2024-09-19T08:42:03.727 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-19T08:42:03.752 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T08:42:03.842 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911876 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7911876' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1747 sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 sleep_before_teardown: 0 subset: 176/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi120.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIGm9EE9hTCXUP1VRU55PFEIcPrffWgu1czeTfdRisZiyadHym5HLydoXK6JJTKqvxY77nulF5JUiO1z9UfoEb0= smithi153.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD5hmKUv6Xv3LWB+Yl0XzGd2upKC1GSDa3taYFP9jY6dXVunwONnf4JxNj0SKxRzq0mVayqe6nYfYNRtSL4bxPs= smithi196.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ4inhnqPtRGpXMEnWi6IrgnX94RILgCNyarRaRFh0wZrGzUwKOS/imkuE1JwjMxdaEv0zmzPyJOWI3d8UOuGjc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-18_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-19T08:42:03.842 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa; will attempt to use it 2024-09-19T08:42:03.843 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks 2024-09-19T08:42:03.843 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-19T08:42:03.845 INFO:teuthology.task.internal:Checking packages... 2024-09-19T08:42:03.867 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2f599a46ad10e37c929e5c43da2f6907ddcd1961' 2024-09-19T08:42:03.867 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-19T08:42:03.867 INFO:teuthology.packaging:ref: None 2024-09-19T08:42:03.867 INFO:teuthology.packaging:tag: None 2024-09-19T08:42:03.868 INFO:teuthology.packaging:branch: squid 2024-09-19T08:42:03.868 INFO:teuthology.packaging:sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:42:03.868 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-19T08:42:04.109 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-363.g6fad5dd7 2024-09-19T08:42:04.111 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-19T08:42:04.117 INFO:teuthology.task.internal:no buildpackages task found 2024-09-19T08:42:04.117 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-19T08:42:04.163 INFO:teuthology.task.internal:Saving configuration 2024-09-19T08:42:04.181 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-19T08:42:04.189 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-19T08:42:04.214 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi120.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911876', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 08:36:35.339533', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIGm9EE9hTCXUP1VRU55PFEIcPrffWgu1czeTfdRisZiyadHym5HLydoXK6JJTKqvxY77nulF5JUiO1z9UfoEb0='} 2024-09-19T08:42:04.239 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi153.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911876', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 08:36:35.342904', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD5hmKUv6Xv3LWB+Yl0XzGd2upKC1GSDa3taYFP9jY6dXVunwONnf4JxNj0SKxRzq0mVayqe6nYfYNRtSL4bxPs='} 2024-09-19T08:42:04.259 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911876', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 08:36:35.341354', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ4inhnqPtRGpXMEnWi6IrgnX94RILgCNyarRaRFh0wZrGzUwKOS/imkuE1JwjMxdaEv0zmzPyJOWI3d8UOuGjc='} 2024-09-19T08:42:04.260 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-19T08:42:04.266 INFO:teuthology.task.internal:roles: ubuntu@smithi120.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-19T08:42:04.266 INFO:teuthology.task.internal:roles: ubuntu@smithi153.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-19T08:42:04.266 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-19T08:42:04.266 INFO:teuthology.run_tasks:Running task console_log... 2024-09-19T08:42:04.351 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4194e8fbe0>, signals=[15]) 2024-09-19T08:42:04.352 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-19T08:42:04.357 INFO:teuthology.task.internal:Opening connections... 2024-09-19T08:42:04.357 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi120.front.sepia.ceph.com 2024-09-19T08:42:04.358 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:42:04.441 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi153.front.sepia.ceph.com 2024-09-19T08:42:04.441 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:42:04.515 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi196.front.sepia.ceph.com 2024-09-19T08:42:04.515 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:42:04.592 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-19T08:42:04.600 DEBUG:teuthology.orchestra.run.smithi120:> uname -m 2024-09-19T08:42:04.619 INFO:teuthology.orchestra.run.smithi120.stdout:x86_64 2024-09-19T08:42:04.620 DEBUG:teuthology.orchestra.run.smithi120:> cat /etc/os-release 2024-09-19T08:42:04.677 INFO:teuthology.orchestra.run.smithi120.stdout:NAME="CentOS Stream" 2024-09-19T08:42:04.677 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION="9" 2024-09-19T08:42:04.678 INFO:teuthology.orchestra.run.smithi120.stdout:ID="centos" 2024-09-19T08:42:04.678 INFO:teuthology.orchestra.run.smithi120.stdout:ID_LIKE="rhel fedora" 2024-09-19T08:42:04.678 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_ID="9" 2024-09-19T08:42:04.678 INFO:teuthology.orchestra.run.smithi120.stdout:PLATFORM_ID="platform:el9" 2024-09-19T08:42:04.678 INFO:teuthology.orchestra.run.smithi120.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T08:42:04.678 INFO:teuthology.orchestra.run.smithi120.stdout:ANSI_COLOR="0;31" 2024-09-19T08:42:04.678 INFO:teuthology.orchestra.run.smithi120.stdout:LOGO="fedora-logo-icon" 2024-09-19T08:42:04.678 INFO:teuthology.orchestra.run.smithi120.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T08:42:04.678 INFO:teuthology.orchestra.run.smithi120.stdout:HOME_URL="https://centos.org/" 2024-09-19T08:42:04.678 INFO:teuthology.orchestra.run.smithi120.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T08:42:04.678 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T08:42:04.678 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T08:42:04.679 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server 2024-09-19T08:42:04.711 DEBUG:teuthology.orchestra.run.smithi153:> uname -m 2024-09-19T08:42:04.730 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64 2024-09-19T08:42:04.730 DEBUG:teuthology.orchestra.run.smithi153:> cat /etc/os-release 2024-09-19T08:42:04.787 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="CentOS Stream" 2024-09-19T08:42:04.787 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="9" 2024-09-19T08:42:04.787 INFO:teuthology.orchestra.run.smithi153.stdout:ID="centos" 2024-09-19T08:42:04.787 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="rhel fedora" 2024-09-19T08:42:04.787 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="9" 2024-09-19T08:42:04.787 INFO:teuthology.orchestra.run.smithi153.stdout:PLATFORM_ID="platform:el9" 2024-09-19T08:42:04.787 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T08:42:04.788 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31" 2024-09-19T08:42:04.788 INFO:teuthology.orchestra.run.smithi153.stdout:LOGO="fedora-logo-icon" 2024-09-19T08:42:04.788 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T08:42:04.788 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://centos.org/" 2024-09-19T08:42:04.788 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T08:42:04.788 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T08:42:04.788 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T08:42:04.788 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2024-09-19T08:42:04.811 DEBUG:teuthology.orchestra.run.smithi196:> uname -m 2024-09-19T08:42:04.830 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2024-09-19T08:42:04.830 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-09-19T08:42:04.888 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="CentOS Stream" 2024-09-19T08:42:04.889 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="9" 2024-09-19T08:42:04.889 INFO:teuthology.orchestra.run.smithi196.stdout:ID="centos" 2024-09-19T08:42:04.889 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="rhel fedora" 2024-09-19T08:42:04.889 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="9" 2024-09-19T08:42:04.889 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el9" 2024-09-19T08:42:04.889 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T08:42:04.889 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2024-09-19T08:42:04.889 INFO:teuthology.orchestra.run.smithi196.stdout:LOGO="fedora-logo-icon" 2024-09-19T08:42:04.889 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T08:42:04.889 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://centos.org/" 2024-09-19T08:42:04.889 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T08:42:04.889 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T08:42:04.889 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T08:42:04.890 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-09-19T08:42:04.910 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-19T08:42:04.920 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-19T08:42:04.924 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-19T08:42:04.925 DEBUG:teuthology.orchestra.run.smithi120:> test '!' -e /home/ubuntu/cephtest 2024-09-19T08:42:04.927 DEBUG:teuthology.orchestra.run.smithi153:> test '!' -e /home/ubuntu/cephtest 2024-09-19T08:42:04.929 DEBUG:teuthology.orchestra.run.smithi196:> test '!' -e /home/ubuntu/cephtest 2024-09-19T08:42:04.947 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-19T08:42:04.953 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-19T08:42:04.953 DEBUG:teuthology.orchestra.run.smithi120:> test -z $(ls -A /var/lib/ceph) 2024-09-19T08:42:04.986 DEBUG:teuthology.orchestra.run.smithi153:> test -z $(ls -A /var/lib/ceph) 2024-09-19T08:42:04.988 DEBUG:teuthology.orchestra.run.smithi196:> test -z $(ls -A /var/lib/ceph) 2024-09-19T08:42:05.065 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-19T08:42:05.140 INFO:teuthology.run_tasks:Running task kernel... 2024-09-19T08:42:05.154 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-19T08:42:05.155 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-19T08:42:05.155 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-19T08:42:05.155 DEBUG:teuthology.orchestra.run.smithi120:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T08:42:05.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:42:05.172 DEBUG:teuthology.orchestra.run.smithi120:> uname -r 2024-09-19T08:42:05.230 INFO:teuthology.orchestra.run.smithi120.stdout:5.14.0-503.el9.x86_64 2024-09-19T08:42:05.230 INFO:teuthology.task.kernel:Running kernel on smithi120: 5.14.0-503.el9.x86_64 2024-09-19T08:42:05.230 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum install -y kernel 2024-09-19T08:42:07.743 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.3 MB 00:00 2024-09-19T08:42:09.561 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-09-19T08:42:13.745 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-19T08:42:15.171 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - Extras packages 87 kB/s | 19 kB 00:00 2024-09-19T08:42:16.100 INFO:teuthology.orchestra.run.smithi120.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-19T08:42:21.723 INFO:teuthology.orchestra.run.smithi120.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-19T08:42:23.629 INFO:teuthology.orchestra.run.smithi120.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-19T08:42:23.629 INFO:teuthology.orchestra.run.smithi120.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-19T08:42:23.723 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-19T08:42:23.731 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-19T08:42:23.731 INFO:teuthology.orchestra.run.smithi120.stdout: Package Architecture Version Repository Size 2024-09-19T08:42:23.731 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-19T08:42:23.732 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-09-19T08:42:23.732 INFO:teuthology.orchestra.run.smithi120.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-19T08:42:23.732 INFO:teuthology.orchestra.run.smithi120.stdout:Installing dependencies: 2024-09-19T08:42:23.732 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-19T08:42:23.732 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-19T08:42:23.732 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-19T08:42:23.732 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:42:23.732 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-09-19T08:42:23.732 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-19T08:42:23.732 INFO:teuthology.orchestra.run.smithi120.stdout:Install 4 Packages 2024-09-19T08:42:23.732 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:42:23.733 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 79 M 2024-09-19T08:42:23.733 INFO:teuthology.orchestra.run.smithi120.stdout:Installed size: 126 M 2024-09-19T08:42:23.733 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-09-19T08:42:23.966 INFO:teuthology.orchestra.run.smithi120.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 371 kB/s | 36 kB 00:00 2024-09-19T08:42:24.436 INFO:teuthology.orchestra.run.smithi120.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-09-19T08:42:24.833 INFO:teuthology.orchestra.run.smithi120.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 36 MB/s | 35 MB 00:00 2024-09-19T08:42:25.100 INFO:teuthology.orchestra.run.smithi120.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 25 MB/s | 29 MB 00:01 2024-09-19T08:42:25.101 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-09-19T08:42:25.101 INFO:teuthology.orchestra.run.smithi120.stdout:Total 58 MB/s | 79 MB 00:01 2024-09-19T08:42:25.778 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-09-19T08:42:26.129 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-09-19T08:42:26.129 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-09-19T08:42:27.094 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-09-19T08:42:27.095 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-09-19T08:42:29.533 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-09-19T08:42:30.246 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-19T08:42:30.329 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T08:42:33.104 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T08:42:33.246 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T08:42:38.080 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T08:42:38.212 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T08:42:43.032 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T08:43:21.425 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T08:43:21.469 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-19T08:43:21.849 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T08:43:21.849 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-19T08:43:21.849 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T08:43:21.849 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T08:43:22.741 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T08:43:22.741 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:43:22.741 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-09-19T08:43:22.741 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-19T08:43:22.741 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-19T08:43:22.741 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-19T08:43:22.741 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-19T08:43:22.741 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:43:22.742 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-19T08:43:23.110 DEBUG:teuthology.orchestra.run.smithi120:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T08:43:23.199 INFO:teuthology.orchestra.run.smithi120.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T08:43:23.199 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T08:43:23.199 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-19T08:43:23.200 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T08:43:23.200 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T08:43:23.200 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-19T08:43:23.200 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi120.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T08:43:23.200 DEBUG:teuthology.orchestra.run.smithi120:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T08:43:23.258 INFO:teuthology.orchestra.run.smithi120.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T08:43:23.258 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T08:43:23.258 DEBUG:teuthology.orchestra.run.smithi120:> sudo rpm -qi grub2-tools 2024-09-19T08:43:23.291 INFO:teuthology.orchestra.run.smithi120.stdout:Name : grub2-tools 2024-09-19T08:43:23.291 INFO:teuthology.orchestra.run.smithi120.stdout:Epoch : 1 2024-09-19T08:43:23.291 INFO:teuthology.orchestra.run.smithi120.stdout:Version : 2.06 2024-09-19T08:43:23.291 INFO:teuthology.orchestra.run.smithi120.stdout:Release : 82.el9 2024-09-19T08:43:23.291 INFO:teuthology.orchestra.run.smithi120.stdout:Architecture: x86_64 2024-09-19T08:43:23.291 INFO:teuthology.orchestra.run.smithi120.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T08:43:23.291 INFO:teuthology.orchestra.run.smithi120.stdout:Group : Unspecified 2024-09-19T08:43:23.291 INFO:teuthology.orchestra.run.smithi120.stdout:Size : 8274670 2024-09-19T08:43:23.291 INFO:teuthology.orchestra.run.smithi120.stdout:License : GPLv3+ 2024-09-19T08:43:23.291 INFO:teuthology.orchestra.run.smithi120.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T08:43:23.292 INFO:teuthology.orchestra.run.smithi120.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T08:43:23.292 INFO:teuthology.orchestra.run.smithi120.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T08:43:23.292 INFO:teuthology.orchestra.run.smithi120.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T08:43:23.292 INFO:teuthology.orchestra.run.smithi120.stdout:Packager : builder@centos.org 2024-09-19T08:43:23.292 INFO:teuthology.orchestra.run.smithi120.stdout:Vendor : CentOS 2024-09-19T08:43:23.292 INFO:teuthology.orchestra.run.smithi120.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T08:43:23.292 INFO:teuthology.orchestra.run.smithi120.stdout:Summary : Support tools for GRUB. 2024-09-19T08:43:23.292 INFO:teuthology.orchestra.run.smithi120.stdout:Description : 2024-09-19T08:43:23.292 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:43:23.292 INFO:teuthology.orchestra.run.smithi120.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T08:43:23.292 INFO:teuthology.orchestra.run.smithi120.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T08:43:23.292 INFO:teuthology.orchestra.run.smithi120.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T08:43:23.292 INFO:teuthology.orchestra.run.smithi120.stdout:hardware devices. 2024-09-19T08:43:23.292 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:43:23.292 INFO:teuthology.orchestra.run.smithi120.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T08:43:23.293 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T08:43:23.293 INFO:teuthology.task.kernel:Updating grub on smithi120 to boot 5.14.0-508.el9.x86_64 2024-09-19T08:43:23.294 DEBUG:teuthology.orchestra.run.smithi120:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T08:43:23.964 INFO:teuthology.orchestra.run.smithi120.stderr:Generating grub configuration file ... 2024-09-19T08:43:25.274 INFO:teuthology.orchestra.run.smithi120.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T08:43:25.300 INFO:teuthology.orchestra.run.smithi120.stderr:done 2024-09-19T08:43:25.302 DEBUG:teuthology.orchestra.run.smithi120:> mktemp 2024-09-19T08:43:25.318 INFO:teuthology.orchestra.run.smithi120.stdout:/tmp/tmp.NPCyQ6Ghjn 2024-09-19T08:43:25.319 DEBUG:teuthology.orchestra.run.smithi120:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.NPCyQ6Ghjn 2024-09-19T08:43:25.385 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 0666 /tmp/tmp.NPCyQ6Ghjn 2024-09-19T08:43:25.531 DEBUG:teuthology.orchestra.remote:smithi120:/tmp/tmp.NPCyQ6Ghjn is 6KB 2024-09-19T08:43:25.581 DEBUG:teuthology.orchestra.run.smithi120:> rm -fr /tmp/tmp.NPCyQ6Ghjn 2024-09-19T08:43:25.597 DEBUG:teuthology.orchestra.run.smithi120:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T08:43:25.668 INFO:teuthology.orchestra.run.smithi120.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T08:43:25.668 INFO:teuthology.orchestra.run.smithi120.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T08:43:25.668 INFO:teuthology.orchestra.run.smithi120.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T08:43:25.669 INFO:teuthology.orchestra.run.smithi120.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-19T08:43:25.670 DEBUG:teuthology.orchestra.run.smithi120:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-19T08:43:25.828 DEBUG:teuthology.orchestra.run.smithi120:> sudo shutdown -r now 2024-09-19T08:43:55.832 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-19T08:43:55.832 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-19T08:43:55.833 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:44:14.269 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.120 2024-09-19T08:44:23.270 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-19T08:44:23.272 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:44:26.333 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.120 2024-09-19T08:44:38.333 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-19T08:44:38.334 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:44:56.765 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.120 2024-09-19T08:45:11.766 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-19T08:45:11.767 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:45:12.185 DEBUG:teuthology.orchestra.run.smithi120:> true 2024-09-19T08:45:12.664 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-19T08:45:12.664 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-19T08:45:12.664 DEBUG:teuthology.orchestra.run.smithi120:> uname -r 2024-09-19T08:45:12.679 INFO:teuthology.orchestra.run.smithi120.stdout:5.14.0-508.el9.x86_64 2024-09-19T08:45:12.679 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-19T08:45:12.679 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T08:45:12.679 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T08:45:13.680 DEBUG:teuthology.parallel:result is None 2024-09-19T08:45:13.681 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-19T08:45:13.694 INFO:teuthology.task.internal:Creating test directory... 2024-09-19T08:45:13.694 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T08:45:13.699 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T08:45:13.703 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T08:45:13.747 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-19T08:45:13.754 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-19T08:45:13.761 INFO:teuthology.task.internal:Creating archive directory... 2024-09-19T08:45:13.761 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T08:45:13.764 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T08:45:13.766 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T08:45:13.822 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-19T08:45:13.829 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-19T08:45:13.829 DEBUG:teuthology.orchestra.run.smithi153:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T08:45:13.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:45:13.846 DEBUG:teuthology.orchestra.run.smithi196:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T08:45:13.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:45:13.864 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T08:45:13.866 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T08:45:13.889 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T08:45:13.924 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T08:45:13.938 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T08:45:13.951 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T08:45:13.957 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T08:45:13.961 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T08:45:13.995 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T08:45:13.997 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-19T08:45:14.004 INFO:teuthology.task.internal:Configuring sudo... 2024-09-19T08:45:14.004 DEBUG:teuthology.orchestra.run.smithi120:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T08:45:14.006 DEBUG:teuthology.orchestra.run.smithi153:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T08:45:14.008 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T08:45:14.037 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-19T08:45:14.046 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-19T08:45:14.046 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T08:45:14.072 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T08:45:14.076 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T08:45:14.095 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T08:45:14.170 DEBUG:teuthology.orchestra.run.smithi120:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T08:45:14.324 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T08:45:14.371 DEBUG:teuthology.orchestra.run.smithi120:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T08:45:14.411 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-19T08:45:14.411 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T08:45:14.476 DEBUG:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T08:45:14.517 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T08:45:14.570 DEBUG:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T08:45:14.654 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T08:45:14.696 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-19T08:45:14.696 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T08:45:14.763 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T08:45:14.809 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T08:45:14.867 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T08:45:14.957 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T08:45:14.996 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-19T08:45:14.997 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T08:45:15.063 DEBUG:teuthology.orchestra.run.smithi120:> sudo service rsyslog restart 2024-09-19T08:45:15.066 DEBUG:teuthology.orchestra.run.smithi153:> sudo service rsyslog restart 2024-09-19T08:45:15.068 DEBUG:teuthology.orchestra.run.smithi196:> sudo service rsyslog restart 2024-09-19T08:45:15.102 INFO:teuthology.orchestra.run.smithi153.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T08:45:15.110 INFO:teuthology.orchestra.run.smithi120.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T08:45:15.139 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T08:45:15.479 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-19T08:45:15.487 INFO:teuthology.task.internal:Starting timer... 2024-09-19T08:45:15.487 INFO:teuthology.run_tasks:Running task pcp... 2024-09-19T08:45:15.509 INFO:teuthology.run_tasks:Running task selinux... 2024-09-19T08:45:15.519 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-19T08:45:15.519 DEBUG:teuthology.orchestra.run.smithi120:> sudo service auditd rotate 2024-09-19T08:45:15.572 INFO:teuthology.orchestra.run.smithi120.stdout:Rotating logs: 2024-09-19T08:45:15.574 DEBUG:teuthology.orchestra.run.smithi153:> sudo service auditd rotate 2024-09-19T08:45:15.629 INFO:teuthology.orchestra.run.smithi153.stdout:Rotating logs: 2024-09-19T08:45:15.631 DEBUG:teuthology.orchestra.run.smithi196:> sudo service auditd rotate 2024-09-19T08:45:15.692 INFO:teuthology.orchestra.run.smithi196.stdout:Rotating logs: 2024-09-19T08:45:15.694 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-19T08:45:15.694 DEBUG:teuthology.orchestra.run.smithi120:> /usr/sbin/getenforce 2024-09-19T08:45:15.718 INFO:teuthology.orchestra.run.smithi120.stdout:Permissive 2024-09-19T08:45:15.718 DEBUG:teuthology.orchestra.run.smithi153:> /usr/sbin/getenforce 2024-09-19T08:45:15.743 INFO:teuthology.orchestra.run.smithi153.stdout:Permissive 2024-09-19T08:45:15.744 DEBUG:teuthology.orchestra.run.smithi196:> /usr/sbin/getenforce 2024-09-19T08:45:15.768 INFO:teuthology.orchestra.run.smithi196.stdout:Permissive 2024-09-19T08:45:15.768 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi120.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi153.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi196.front.sepia.ceph.com': 'permissive'} 2024-09-19T08:45:15.769 DEBUG:teuthology.orchestra.run.smithi120:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T08:45:15.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:45:15.794 DEBUG:teuthology.orchestra.run.smithi153:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T08:45:15.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:45:15.822 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T08:45:15.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:45:15.849 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-19T08:45:15.850 DEBUG:teuthology.orchestra.run.smithi120:> sudo /usr/sbin/setenforce permissive 2024-09-19T08:45:15.883 DEBUG:teuthology.orchestra.run.smithi153:> sudo /usr/sbin/setenforce permissive 2024-09-19T08:45:15.913 DEBUG:teuthology.orchestra.run.smithi196:> sudo /usr/sbin/setenforce permissive 2024-09-19T08:45:15.942 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-19T08:45:15.952 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-19T08:45:15.955 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-19T08:45:16.079 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-19T08:45:16.090 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-19T08:45:16.091 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi120.front.sepia.ceph.com,smithi153.front.sepia.ceph.com,smithi196.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-19T08:49:26.526 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi120.front.sepia.ceph.com'), Remote(name='ubuntu@smithi153.front.sepia.ceph.com'), Remote(name='ubuntu@smithi196.front.sepia.ceph.com')] 2024-09-19T08:49:26.527 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-19T08:49:26.528 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:49:26.607 DEBUG:teuthology.orchestra.run.smithi120:> true 2024-09-19T08:49:26.695 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-19T08:49:26.696 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-09-19T08:49:26.697 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:49:26.770 DEBUG:teuthology.orchestra.run.smithi153:> true 2024-09-19T08:49:26.860 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-09-19T08:49:26.861 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-19T08:49:26.861 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:49:26.938 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-09-19T08:49:27.013 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-19T08:49:27.014 INFO:teuthology.run_tasks:Running task clock... 2024-09-19T08:49:27.025 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-19T08:49:27.025 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T08:49:27.025 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T08:49:27.028 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T08:49:27.028 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T08:49:27.030 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T08:49:27.030 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T08:49:27.056 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T08:49:27.057 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T08:49:27.070 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T08:49:27.071 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T08:49:27.084 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T08:49:27.094 INFO:teuthology.orchestra.run.smithi120.stderr:sudo: ntpd: command not found 2024-09-19T08:49:27.096 INFO:teuthology.orchestra.run.smithi153.stderr:sudo: ntpd: command not found 2024-09-19T08:49:27.100 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T08:49:27.106 INFO:teuthology.orchestra.run.smithi120.stdout:506 Cannot talk to daemon 2024-09-19T08:49:27.107 INFO:teuthology.orchestra.run.smithi153.stdout:506 Cannot talk to daemon 2024-09-19T08:49:27.119 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T08:49:27.121 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T08:49:27.126 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: ntpd: command not found 2024-09-19T08:49:27.133 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T08:49:27.135 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T08:49:27.137 INFO:teuthology.orchestra.run.smithi196.stdout:506 Cannot talk to daemon 2024-09-19T08:49:27.152 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T08:49:27.166 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T08:49:27.191 INFO:teuthology.orchestra.run.smithi120.stderr:bash: line 1: ntpq: command not found 2024-09-19T08:49:27.192 INFO:teuthology.orchestra.run.smithi153.stderr:bash: line 1: ntpq: command not found 2024-09-19T08:49:27.194 INFO:teuthology.orchestra.run.smithi120.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T08:49:27.194 INFO:teuthology.orchestra.run.smithi120.stdout:=============================================================================== 2024-09-19T08:49:27.194 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:49:27.194 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:49:27.195 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:49:27.195 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:49:27.196 INFO:teuthology.orchestra.run.smithi153.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T08:49:27.196 INFO:teuthology.orchestra.run.smithi153.stdout:=============================================================================== 2024-09-19T08:49:27.196 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:49:27.196 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:49:27.196 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:49:27.196 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:49:27.225 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-09-19T08:49:27.228 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T08:49:27.228 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-09-19T08:49:27.228 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:49:27.228 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:49:27.228 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:49:27.228 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:49:27.229 INFO:teuthology.run_tasks:Running task install... 2024-09-19T08:49:27.237 DEBUG:teuthology.task.install:project ceph 2024-09-19T08:49:27.237 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}} 2024-09-19T08:49:27.237 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'} 2024-09-19T08:49:27.237 INFO:teuthology.task.install:Using flavor: default 2024-09-19T08:49:27.242 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-19T08:49:27.242 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-19T08:49:27.243 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T08:49:27.243 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:49:27.245 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T08:49:27.246 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:49:27.246 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T08:49:27.247 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:49:27.393 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T08:49:27.393 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T08:49:27.401 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T08:49:27.401 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T08:49:27.407 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T08:49:27.407 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T08:49:27.504 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T08:49:27.505 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-19T08:49:27.505 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T08:49:27.510 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T08:49:27.510 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-19T08:49:27.510 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T08:49:27.515 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T08:49:27.516 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-19T08:49:27.516 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T08:49:27.530 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T08:49:27.530 DEBUG:teuthology.orchestra.run.smithi120:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T08:49:27.536 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T08:49:27.536 DEBUG:teuthology.orchestra.run.smithi196:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T08:49:27.542 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T08:49:27.542 DEBUG:teuthology.orchestra.run.smithi153:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T08:49:27.596 DEBUG:teuthology.orchestra.run.smithi120:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T08:49:27.602 DEBUG:teuthology.orchestra.run.smithi196:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T08:49:27.608 DEBUG:teuthology.orchestra.run.smithi153:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T08:49:27.671 DEBUG:teuthology.orchestra.run.smithi120:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T08:49:27.677 DEBUG:teuthology.orchestra.run.smithi196:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T08:49:27.683 DEBUG:teuthology.orchestra.run.smithi153:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T08:49:27.735 INFO:teuthology.orchestra.run.smithi120.stdout:check_obsoletes = 1 2024-09-19T08:49:27.737 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum clean all 2024-09-19T08:49:27.741 INFO:teuthology.orchestra.run.smithi196.stdout:check_obsoletes = 1 2024-09-19T08:49:27.743 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum clean all 2024-09-19T08:49:27.748 INFO:teuthology.orchestra.run.smithi153.stdout:check_obsoletes = 1 2024-09-19T08:49:27.749 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum clean all 2024-09-19T08:49:27.964 INFO:teuthology.orchestra.run.smithi120.stdout:45 files removed 2024-09-19T08:49:27.967 INFO:teuthology.orchestra.run.smithi196.stdout:45 files removed 2024-09-19T08:49:27.974 INFO:teuthology.orchestra.run.smithi153.stdout:45 files removed 2024-09-19T08:49:27.990 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T08:49:27.994 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T08:49:28.001 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T08:49:28.537 INFO:teuthology.orchestra.run.smithi153.stdout:ceph packages for x86_64 323 kB/s | 84 kB 00:00 2024-09-19T08:49:28.552 INFO:teuthology.orchestra.run.smithi120.stdout:ceph packages for x86_64 303 kB/s | 84 kB 00:00 2024-09-19T08:49:28.556 INFO:teuthology.orchestra.run.smithi196.stdout:ceph packages for x86_64 294 kB/s | 84 kB 00:00 2024-09-19T08:49:28.813 INFO:teuthology.orchestra.run.smithi153.stdout:ceph noarch packages 51 kB/s | 12 kB 00:00 2024-09-19T08:49:28.845 INFO:teuthology.orchestra.run.smithi120.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-19T08:49:28.864 INFO:teuthology.orchestra.run.smithi196.stdout:ceph noarch packages 44 kB/s | 12 kB 00:00 2024-09-19T08:49:29.080 INFO:teuthology.orchestra.run.smithi153.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-19T08:49:29.104 INFO:teuthology.orchestra.run.smithi120.stdout:ceph source packages 8.6 kB/s | 1.9 kB 00:00 2024-09-19T08:49:29.132 INFO:teuthology.orchestra.run.smithi196.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-19T08:49:29.813 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.3 MB 00:00 2024-09-19T08:49:31.105 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.3 MB 00:01 2024-09-19T08:49:31.857 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - BaseOS 3.1 MB/s | 8.3 MB 00:02 2024-09-19T08:49:32.156 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - AppStream 16 MB/s | 20 MB 00:01 2024-09-19T08:49:32.789 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - AppStream 33 MB/s | 20 MB 00:00 2024-09-19T08:49:33.734 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-19T08:49:37.432 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 6.5 MB 00:01 2024-09-19T08:49:38.226 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - CRB 9.3 MB/s | 6.5 MB 00:00 2024-09-19T08:49:38.331 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.5 MB 00:01 2024-09-19T08:49:39.290 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 19 kB 00:00 2024-09-19T08:49:39.643 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - Extras packages 103 kB/s | 19 kB 00:00 2024-09-19T08:49:40.085 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 19 kB 00:00 2024-09-19T08:49:40.620 INFO:teuthology.orchestra.run.smithi120.stdout:Extra Packages for Enterprise Linux 19 MB/s | 23 MB 00:01 2024-09-19T08:49:40.856 INFO:teuthology.orchestra.run.smithi196.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-19T08:49:41.211 INFO:teuthology.orchestra.run.smithi153.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-19T08:49:46.100 INFO:teuthology.orchestra.run.smithi120.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-19T08:49:46.378 INFO:teuthology.orchestra.run.smithi196.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-19T08:49:46.633 INFO:teuthology.orchestra.run.smithi153.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-09-19T08:49:48.371 INFO:teuthology.orchestra.run.smithi120.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T08:49:48.372 INFO:teuthology.orchestra.run.smithi120.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T08:49:48.385 INFO:teuthology.orchestra.run.smithi120.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T08:49:48.386 INFO:teuthology.orchestra.run.smithi120.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T08:49:48.387 INFO:teuthology.orchestra.run.smithi120.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T08:49:48.388 INFO:teuthology.orchestra.run.smithi120.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T08:49:48.390 INFO:teuthology.orchestra.run.smithi120.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T08:49:48.393 INFO:teuthology.orchestra.run.smithi120.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T08:49:48.394 INFO:teuthology.orchestra.run.smithi120.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T08:49:48.458 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-19T08:49:48.481 INFO:teuthology.orchestra.run.smithi120.stdout:====================================================================================== 2024-09-19T08:49:48.481 INFO:teuthology.orchestra.run.smithi120.stdout: Package Arch Version Repository Size 2024-09-19T08:49:48.481 INFO:teuthology.orchestra.run.smithi120.stdout:====================================================================================== 2024-09-19T08:49:48.482 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-09-19T08:49:48.482 INFO:teuthology.orchestra.run.smithi120.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T08:49:48.482 INFO:teuthology.orchestra.run.smithi120.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T08:49:48.482 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T08:49:48.482 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T08:49:48.482 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T08:49:48.482 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T08:49:48.482 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T08:49:48.482 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T08:49:48.482 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T08:49:48.482 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T08:49:48.482 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T08:49:48.483 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T08:49:48.483 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T08:49:48.483 INFO:teuthology.orchestra.run.smithi120.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T08:49:48.483 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T08:49:48.483 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T08:49:48.483 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T08:49:48.483 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T08:49:48.483 INFO:teuthology.orchestra.run.smithi120.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T08:49:48.483 INFO:teuthology.orchestra.run.smithi120.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T08:49:48.483 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T08:49:48.483 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T08:49:48.483 INFO:teuthology.orchestra.run.smithi120.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T08:49:48.484 INFO:teuthology.orchestra.run.smithi120.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T08:49:48.484 INFO:teuthology.orchestra.run.smithi120.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T08:49:48.484 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T08:49:48.484 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T08:49:48.484 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T08:49:48.484 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T08:49:48.484 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T08:49:48.484 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T08:49:48.484 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T08:49:48.484 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T08:49:48.484 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T08:49:48.484 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T08:49:48.485 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T08:49:48.485 INFO:teuthology.orchestra.run.smithi120.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T08:49:48.485 INFO:teuthology.orchestra.run.smithi120.stdout:Upgrading: 2024-09-19T08:49:48.485 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T08:49:48.485 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T08:49:48.485 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T08:49:48.485 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T08:49:48.485 INFO:teuthology.orchestra.run.smithi120.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T08:49:48.485 INFO:teuthology.orchestra.run.smithi120.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T08:49:48.485 INFO:teuthology.orchestra.run.smithi120.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T08:49:48.485 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T08:49:48.485 INFO:teuthology.orchestra.run.smithi120.stdout:Installing dependencies: 2024-09-19T08:49:48.486 INFO:teuthology.orchestra.run.smithi120.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T08:49:48.486 INFO:teuthology.orchestra.run.smithi120.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T08:49:48.486 INFO:teuthology.orchestra.run.smithi120.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T08:49:48.486 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T08:49:48.486 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T08:49:48.486 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T08:49:48.486 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T08:49:48.486 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T08:49:48.486 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T08:49:48.486 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T08:49:48.486 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T08:49:48.486 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T08:49:48.486 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T08:49:48.487 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T08:49:48.487 INFO:teuthology.orchestra.run.smithi120.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T08:49:48.487 INFO:teuthology.orchestra.run.smithi120.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T08:49:48.487 INFO:teuthology.orchestra.run.smithi120.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T08:49:48.487 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T08:49:48.487 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T08:49:48.487 INFO:teuthology.orchestra.run.smithi120.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T08:49:48.487 INFO:teuthology.orchestra.run.smithi120.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T08:49:48.487 INFO:teuthology.orchestra.run.smithi120.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T08:49:48.487 INFO:teuthology.orchestra.run.smithi120.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T08:49:48.487 INFO:teuthology.orchestra.run.smithi120.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T08:49:48.487 INFO:teuthology.orchestra.run.smithi120.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T08:49:48.487 INFO:teuthology.orchestra.run.smithi120.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T08:49:48.488 INFO:teuthology.orchestra.run.smithi120.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T08:49:48.488 INFO:teuthology.orchestra.run.smithi120.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T08:49:48.488 INFO:teuthology.orchestra.run.smithi120.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T08:49:48.488 INFO:teuthology.orchestra.run.smithi120.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T08:49:48.488 INFO:teuthology.orchestra.run.smithi120.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T08:49:48.488 INFO:teuthology.orchestra.run.smithi120.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T08:49:48.488 INFO:teuthology.orchestra.run.smithi120.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T08:49:48.488 INFO:teuthology.orchestra.run.smithi120.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T08:49:48.488 INFO:teuthology.orchestra.run.smithi120.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T08:49:48.488 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T08:49:48.488 INFO:teuthology.orchestra.run.smithi120.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T08:49:48.488 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T08:49:48.488 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T08:49:48.489 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T08:49:48.489 INFO:teuthology.orchestra.run.smithi120.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T08:49:48.489 INFO:teuthology.orchestra.run.smithi120.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T08:49:48.489 INFO:teuthology.orchestra.run.smithi120.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T08:49:48.489 INFO:teuthology.orchestra.run.smithi120.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T08:49:48.489 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T08:49:48.489 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T08:49:48.489 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T08:49:48.489 INFO:teuthology.orchestra.run.smithi120.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T08:49:48.489 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T08:49:48.489 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T08:49:48.489 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T08:49:48.489 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T08:49:48.490 INFO:teuthology.orchestra.run.smithi120.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T08:49:48.490 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T08:49:48.490 INFO:teuthology.orchestra.run.smithi120.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T08:49:48.490 INFO:teuthology.orchestra.run.smithi120.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T08:49:48.490 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T08:49:48.490 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T08:49:48.490 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T08:49:48.490 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T08:49:48.490 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T08:49:48.490 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T08:49:48.490 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T08:49:48.490 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T08:49:48.490 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T08:49:48.491 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T08:49:48.491 INFO:teuthology.orchestra.run.smithi120.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T08:49:48.491 INFO:teuthology.orchestra.run.smithi120.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T08:49:48.491 INFO:teuthology.orchestra.run.smithi120.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T08:49:48.491 INFO:teuthology.orchestra.run.smithi120.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T08:49:48.491 INFO:teuthology.orchestra.run.smithi120.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T08:49:48.491 INFO:teuthology.orchestra.run.smithi120.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T08:49:48.491 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T08:49:48.491 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T08:49:48.491 INFO:teuthology.orchestra.run.smithi120.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T08:49:48.491 INFO:teuthology.orchestra.run.smithi120.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T08:49:48.491 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T08:49:48.491 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T08:49:48.491 INFO:teuthology.orchestra.run.smithi120.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T08:49:48.492 INFO:teuthology.orchestra.run.smithi120.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T08:49:48.492 INFO:teuthology.orchestra.run.smithi120.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T08:49:48.492 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T08:49:48.492 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T08:49:48.492 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T08:49:48.492 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T08:49:48.492 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T08:49:48.492 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T08:49:48.492 INFO:teuthology.orchestra.run.smithi120.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T08:49:48.492 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T08:49:48.492 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T08:49:48.492 INFO:teuthology.orchestra.run.smithi120.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T08:49:48.492 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T08:49:48.493 INFO:teuthology.orchestra.run.smithi120.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T08:49:48.493 INFO:teuthology.orchestra.run.smithi120.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T08:49:48.493 INFO:teuthology.orchestra.run.smithi120.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T08:49:48.493 INFO:teuthology.orchestra.run.smithi120.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T08:49:48.493 INFO:teuthology.orchestra.run.smithi120.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T08:49:48.493 INFO:teuthology.orchestra.run.smithi120.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T08:49:48.493 INFO:teuthology.orchestra.run.smithi120.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T08:49:48.493 INFO:teuthology.orchestra.run.smithi120.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T08:49:48.493 INFO:teuthology.orchestra.run.smithi120.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T08:49:48.493 INFO:teuthology.orchestra.run.smithi120.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T08:49:48.493 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T08:49:48.493 INFO:teuthology.orchestra.run.smithi120.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T08:49:48.493 INFO:teuthology.orchestra.run.smithi120.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T08:49:48.494 INFO:teuthology.orchestra.run.smithi120.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T08:49:48.494 INFO:teuthology.orchestra.run.smithi120.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T08:49:48.494 INFO:teuthology.orchestra.run.smithi120.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T08:49:48.494 INFO:teuthology.orchestra.run.smithi120.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T08:49:48.494 INFO:teuthology.orchestra.run.smithi120.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T08:49:48.494 INFO:teuthology.orchestra.run.smithi120.stdout:Installing weak dependencies: 2024-09-19T08:49:48.494 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T08:49:48.494 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:49:48.494 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-09-19T08:49:48.494 INFO:teuthology.orchestra.run.smithi120.stdout:====================================================================================== 2024-09-19T08:49:48.494 INFO:teuthology.orchestra.run.smithi120.stdout:Install 148 Packages 2024-09-19T08:49:48.494 INFO:teuthology.orchestra.run.smithi120.stdout:Upgrade 8 Packages 2024-09-19T08:49:48.494 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:49:48.495 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 226 M 2024-09-19T08:49:48.495 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-09-19T08:49:48.530 INFO:teuthology.orchestra.run.smithi196.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T08:49:48.531 INFO:teuthology.orchestra.run.smithi196.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T08:49:48.542 INFO:teuthology.orchestra.run.smithi196.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T08:49:48.543 INFO:teuthology.orchestra.run.smithi196.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T08:49:48.544 INFO:teuthology.orchestra.run.smithi196.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T08:49:48.545 INFO:teuthology.orchestra.run.smithi196.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T08:49:48.546 INFO:teuthology.orchestra.run.smithi196.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T08:49:48.549 INFO:teuthology.orchestra.run.smithi196.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T08:49:48.549 INFO:teuthology.orchestra.run.smithi196.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T08:49:48.613 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-09-19T08:49:48.640 INFO:teuthology.orchestra.run.smithi196.stdout:====================================================================================== 2024-09-19T08:49:48.641 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-09-19T08:49:48.641 INFO:teuthology.orchestra.run.smithi196.stdout:====================================================================================== 2024-09-19T08:49:48.641 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-09-19T08:49:48.641 INFO:teuthology.orchestra.run.smithi196.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T08:49:48.641 INFO:teuthology.orchestra.run.smithi196.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T08:49:48.641 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T08:49:48.641 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T08:49:48.641 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T08:49:48.641 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T08:49:48.641 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T08:49:48.641 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T08:49:48.642 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T08:49:48.642 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T08:49:48.642 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T08:49:48.642 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T08:49:48.642 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T08:49:48.642 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T08:49:48.642 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T08:49:48.642 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T08:49:48.642 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T08:49:48.642 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T08:49:48.642 INFO:teuthology.orchestra.run.smithi196.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T08:49:48.642 INFO:teuthology.orchestra.run.smithi196.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T08:49:48.643 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T08:49:48.643 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T08:49:48.643 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T08:49:48.643 INFO:teuthology.orchestra.run.smithi196.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T08:49:48.643 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T08:49:48.643 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T08:49:48.643 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T08:49:48.643 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T08:49:48.643 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T08:49:48.643 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T08:49:48.643 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T08:49:48.644 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T08:49:48.644 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T08:49:48.644 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T08:49:48.644 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T08:49:48.644 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T08:49:48.644 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T08:49:48.644 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrading: 2024-09-19T08:49:48.644 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T08:49:48.644 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T08:49:48.644 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T08:49:48.644 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T08:49:48.645 INFO:teuthology.orchestra.run.smithi196.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T08:49:48.645 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T08:49:48.645 INFO:teuthology.orchestra.run.smithi196.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T08:49:48.645 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T08:49:48.645 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-09-19T08:49:48.645 INFO:teuthology.orchestra.run.smithi196.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T08:49:48.645 INFO:teuthology.orchestra.run.smithi196.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T08:49:48.645 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T08:49:48.645 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T08:49:48.645 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T08:49:48.645 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T08:49:48.646 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T08:49:48.646 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T08:49:48.646 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T08:49:48.646 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T08:49:48.646 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T08:49:48.646 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T08:49:48.646 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T08:49:48.646 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T08:49:48.646 INFO:teuthology.orchestra.run.smithi196.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T08:49:48.646 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T08:49:48.646 INFO:teuthology.orchestra.run.smithi196.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T08:49:48.646 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T08:49:48.647 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T08:49:48.647 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T08:49:48.647 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T08:49:48.647 INFO:teuthology.orchestra.run.smithi196.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T08:49:48.647 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T08:49:48.647 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T08:49:48.647 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T08:49:48.647 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T08:49:48.647 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T08:49:48.647 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T08:49:48.648 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T08:49:48.648 INFO:teuthology.orchestra.run.smithi196.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T08:49:48.648 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T08:49:48.648 INFO:teuthology.orchestra.run.smithi196.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T08:49:48.648 INFO:teuthology.orchestra.run.smithi196.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T08:49:48.648 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T08:49:48.648 INFO:teuthology.orchestra.run.smithi196.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T08:49:48.648 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T08:49:48.648 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T08:49:48.648 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T08:49:48.648 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T08:49:48.649 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T08:49:48.649 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T08:49:48.649 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T08:49:48.649 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T08:49:48.649 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T08:49:48.649 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T08:49:48.649 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T08:49:48.649 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T08:49:48.649 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T08:49:48.649 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T08:49:48.649 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T08:49:48.650 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T08:49:48.650 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T08:49:48.650 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T08:49:48.650 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T08:49:48.650 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T08:49:48.650 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T08:49:48.650 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T08:49:48.650 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T08:49:48.650 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T08:49:48.650 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T08:49:48.650 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T08:49:48.650 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T08:49:48.651 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T08:49:48.651 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T08:49:48.651 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T08:49:48.651 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T08:49:48.651 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T08:49:48.651 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T08:49:48.651 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T08:49:48.651 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T08:49:48.651 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T08:49:48.651 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T08:49:48.651 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T08:49:48.651 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T08:49:48.652 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T08:49:48.652 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T08:49:48.652 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T08:49:48.652 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T08:49:48.652 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T08:49:48.652 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T08:49:48.652 INFO:teuthology.orchestra.run.smithi196.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T08:49:48.652 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T08:49:48.652 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T08:49:48.652 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T08:49:48.652 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T08:49:48.653 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T08:49:48.653 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T08:49:48.653 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T08:49:48.653 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T08:49:48.653 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T08:49:48.653 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T08:49:48.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T08:49:48.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T08:49:48.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T08:49:48.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T08:49:48.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T08:49:48.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T08:49:48.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T08:49:48.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T08:49:48.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T08:49:48.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T08:49:48.655 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T08:49:48.655 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T08:49:48.655 INFO:teuthology.orchestra.run.smithi196.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T08:49:48.655 INFO:teuthology.orchestra.run.smithi196.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T08:49:48.655 INFO:teuthology.orchestra.run.smithi196.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T08:49:48.655 INFO:teuthology.orchestra.run.smithi196.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T08:49:48.655 INFO:teuthology.orchestra.run.smithi196.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T08:49:48.655 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T08:49:48.655 INFO:teuthology.orchestra.run.smithi196.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T08:49:48.655 INFO:teuthology.orchestra.run.smithi196.stdout:Installing weak dependencies: 2024-09-19T08:49:48.655 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T08:49:48.655 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T08:49:48.656 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-09-19T08:49:48.656 INFO:teuthology.orchestra.run.smithi196.stdout:====================================================================================== 2024-09-19T08:49:48.656 INFO:teuthology.orchestra.run.smithi196.stdout:Install 148 Packages 2024-09-19T08:49:48.656 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrade 8 Packages 2024-09-19T08:49:48.656 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T08:49:48.657 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 226 M 2024-09-19T08:49:48.657 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-09-19T08:49:48.794 INFO:teuthology.orchestra.run.smithi153.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T08:49:48.795 INFO:teuthology.orchestra.run.smithi153.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T08:49:48.806 INFO:teuthology.orchestra.run.smithi153.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T08:49:48.807 INFO:teuthology.orchestra.run.smithi153.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T08:49:48.807 INFO:teuthology.orchestra.run.smithi153.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T08:49:48.808 INFO:teuthology.orchestra.run.smithi153.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T08:49:48.810 INFO:teuthology.orchestra.run.smithi153.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T08:49:48.812 INFO:teuthology.orchestra.run.smithi153.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T08:49:48.813 INFO:teuthology.orchestra.run.smithi153.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T08:49:48.876 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-09-19T08:49:48.894 INFO:teuthology.orchestra.run.smithi153.stdout:====================================================================================== 2024-09-19T08:49:48.894 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2024-09-19T08:49:48.894 INFO:teuthology.orchestra.run.smithi153.stdout:====================================================================================== 2024-09-19T08:49:48.894 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2024-09-19T08:49:48.894 INFO:teuthology.orchestra.run.smithi153.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T08:49:48.894 INFO:teuthology.orchestra.run.smithi153.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T08:49:48.894 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T08:49:48.894 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T08:49:48.894 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T08:49:48.895 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T08:49:48.895 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T08:49:48.895 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T08:49:48.895 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T08:49:48.895 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T08:49:48.895 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T08:49:48.896 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T08:49:48.896 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T08:49:48.896 INFO:teuthology.orchestra.run.smithi153.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T08:49:48.896 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T08:49:48.896 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T08:49:48.896 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T08:49:48.896 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T08:49:48.896 INFO:teuthology.orchestra.run.smithi153.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T08:49:48.896 INFO:teuthology.orchestra.run.smithi153.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T08:49:48.897 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T08:49:48.897 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T08:49:48.897 INFO:teuthology.orchestra.run.smithi153.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T08:49:48.897 INFO:teuthology.orchestra.run.smithi153.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T08:49:48.897 INFO:teuthology.orchestra.run.smithi153.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T08:49:48.897 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T08:49:48.897 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T08:49:48.897 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T08:49:48.897 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T08:49:48.897 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T08:49:48.897 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T08:49:48.897 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T08:49:48.897 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T08:49:48.898 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T08:49:48.898 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T08:49:48.898 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T08:49:48.898 INFO:teuthology.orchestra.run.smithi153.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T08:49:48.898 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrading: 2024-09-19T08:49:48.898 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T08:49:48.898 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T08:49:48.898 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T08:49:48.898 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T08:49:48.898 INFO:teuthology.orchestra.run.smithi153.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T08:49:48.898 INFO:teuthology.orchestra.run.smithi153.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T08:49:48.898 INFO:teuthology.orchestra.run.smithi153.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T08:49:48.898 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T08:49:48.899 INFO:teuthology.orchestra.run.smithi153.stdout:Installing dependencies: 2024-09-19T08:49:48.899 INFO:teuthology.orchestra.run.smithi153.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T08:49:48.899 INFO:teuthology.orchestra.run.smithi153.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T08:49:48.899 INFO:teuthology.orchestra.run.smithi153.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T08:49:48.899 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T08:49:48.899 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T08:49:48.899 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T08:49:48.899 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T08:49:48.899 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T08:49:48.899 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T08:49:48.899 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T08:49:48.899 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T08:49:48.899 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T08:49:48.900 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T08:49:48.900 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T08:49:48.900 INFO:teuthology.orchestra.run.smithi153.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T08:49:48.900 INFO:teuthology.orchestra.run.smithi153.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T08:49:48.900 INFO:teuthology.orchestra.run.smithi153.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T08:49:48.900 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T08:49:48.900 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T08:49:48.900 INFO:teuthology.orchestra.run.smithi153.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T08:49:48.900 INFO:teuthology.orchestra.run.smithi153.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T08:49:48.900 INFO:teuthology.orchestra.run.smithi153.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T08:49:48.900 INFO:teuthology.orchestra.run.smithi153.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T08:49:48.900 INFO:teuthology.orchestra.run.smithi153.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T08:49:48.900 INFO:teuthology.orchestra.run.smithi153.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T08:49:48.900 INFO:teuthology.orchestra.run.smithi153.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T08:49:48.901 INFO:teuthology.orchestra.run.smithi153.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T08:49:48.901 INFO:teuthology.orchestra.run.smithi153.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T08:49:48.901 INFO:teuthology.orchestra.run.smithi153.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T08:49:48.901 INFO:teuthology.orchestra.run.smithi153.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T08:49:48.901 INFO:teuthology.orchestra.run.smithi153.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T08:49:48.901 INFO:teuthology.orchestra.run.smithi153.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T08:49:48.901 INFO:teuthology.orchestra.run.smithi153.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T08:49:48.901 INFO:teuthology.orchestra.run.smithi153.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T08:49:48.901 INFO:teuthology.orchestra.run.smithi153.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T08:49:48.901 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T08:49:48.901 INFO:teuthology.orchestra.run.smithi153.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T08:49:48.901 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T08:49:48.901 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T08:49:48.902 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T08:49:48.902 INFO:teuthology.orchestra.run.smithi153.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T08:49:48.902 INFO:teuthology.orchestra.run.smithi153.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T08:49:48.902 INFO:teuthology.orchestra.run.smithi153.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T08:49:48.902 INFO:teuthology.orchestra.run.smithi153.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T08:49:48.902 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T08:49:48.902 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T08:49:48.902 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T08:49:48.902 INFO:teuthology.orchestra.run.smithi153.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T08:49:48.902 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T08:49:48.902 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T08:49:48.902 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T08:49:48.902 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T08:49:48.903 INFO:teuthology.orchestra.run.smithi153.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T08:49:48.903 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T08:49:48.903 INFO:teuthology.orchestra.run.smithi153.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T08:49:48.903 INFO:teuthology.orchestra.run.smithi153.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T08:49:48.903 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T08:49:48.903 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T08:49:48.903 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T08:49:48.903 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T08:49:48.903 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T08:49:48.903 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T08:49:48.903 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T08:49:48.903 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T08:49:48.904 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T08:49:48.904 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T08:49:48.904 INFO:teuthology.orchestra.run.smithi153.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T08:49:48.904 INFO:teuthology.orchestra.run.smithi153.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T08:49:48.904 INFO:teuthology.orchestra.run.smithi153.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T08:49:48.904 INFO:teuthology.orchestra.run.smithi153.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T08:49:48.904 INFO:teuthology.orchestra.run.smithi153.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T08:49:48.904 INFO:teuthology.orchestra.run.smithi153.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T08:49:48.904 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T08:49:48.905 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T08:49:48.905 INFO:teuthology.orchestra.run.smithi153.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T08:49:48.905 INFO:teuthology.orchestra.run.smithi153.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T08:49:48.905 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T08:49:48.905 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T08:49:48.905 INFO:teuthology.orchestra.run.smithi153.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T08:49:48.905 INFO:teuthology.orchestra.run.smithi153.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T08:49:48.905 INFO:teuthology.orchestra.run.smithi153.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T08:49:48.905 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T08:49:48.905 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T08:49:48.906 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T08:49:48.906 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T08:49:48.906 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T08:49:48.906 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T08:49:48.906 INFO:teuthology.orchestra.run.smithi153.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T08:49:48.906 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T08:49:48.906 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T08:49:48.906 INFO:teuthology.orchestra.run.smithi153.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T08:49:48.906 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T08:49:48.907 INFO:teuthology.orchestra.run.smithi153.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T08:49:48.907 INFO:teuthology.orchestra.run.smithi153.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T08:49:48.907 INFO:teuthology.orchestra.run.smithi153.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T08:49:48.907 INFO:teuthology.orchestra.run.smithi153.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T08:49:48.907 INFO:teuthology.orchestra.run.smithi153.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T08:49:48.907 INFO:teuthology.orchestra.run.smithi153.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T08:49:48.907 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T08:49:48.907 INFO:teuthology.orchestra.run.smithi153.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T08:49:48.907 INFO:teuthology.orchestra.run.smithi153.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T08:49:48.907 INFO:teuthology.orchestra.run.smithi153.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T08:49:48.908 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T08:49:48.908 INFO:teuthology.orchestra.run.smithi153.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T08:49:48.908 INFO:teuthology.orchestra.run.smithi153.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T08:49:48.908 INFO:teuthology.orchestra.run.smithi153.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T08:49:48.908 INFO:teuthology.orchestra.run.smithi153.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T08:49:48.908 INFO:teuthology.orchestra.run.smithi153.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T08:49:48.908 INFO:teuthology.orchestra.run.smithi153.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T08:49:48.908 INFO:teuthology.orchestra.run.smithi153.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T08:49:48.908 INFO:teuthology.orchestra.run.smithi153.stdout:Installing weak dependencies: 2024-09-19T08:49:48.908 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T08:49:48.908 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-19T08:49:48.908 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2024-09-19T08:49:48.908 INFO:teuthology.orchestra.run.smithi153.stdout:====================================================================================== 2024-09-19T08:49:48.909 INFO:teuthology.orchestra.run.smithi153.stdout:Install 148 Packages 2024-09-19T08:49:48.909 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrade 8 Packages 2024-09-19T08:49:48.909 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-19T08:49:48.909 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 226 M 2024-09-19T08:49:48.909 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2024-09-19T08:49:49.293 INFO:teuthology.orchestra.run.smithi120.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 46 kB/s | 6.6 kB 00:00 2024-09-19T08:49:49.405 INFO:teuthology.orchestra.run.smithi196.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 45 kB/s | 6.6 kB 00:00 2024-09-19T08:49:49.543 INFO:teuthology.orchestra.run.smithi153.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 37 kB/s | 6.6 kB 00:00 2024-09-19T08:49:49.635 INFO:teuthology.orchestra.run.smithi120.stdout:(2/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-19T08:49:49.695 INFO:teuthology.orchestra.run.smithi120.stdout:(3/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 2.2 MB/s | 896 kB 00:00 2024-09-19T08:49:49.728 INFO:teuthology.orchestra.run.smithi120.stdout:(4/156): ceph-immutable-object-cache-19.1.1-361 1.5 MB/s | 145 kB 00:00 2024-09-19T08:49:49.895 INFO:teuthology.orchestra.run.smithi120.stdout:(5/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 9.1 MB/s | 1.5 MB 00:00 2024-09-19T08:49:50.185 INFO:teuthology.orchestra.run.smithi153.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 1.4 MB/s | 896 kB 00:00 2024-09-19T08:49:50.196 INFO:teuthology.orchestra.run.smithi196.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 1.1 MB/s | 896 kB 00:00 2024-09-19T08:49:50.252 INFO:teuthology.orchestra.run.smithi153.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 2.1 MB/s | 145 kB 00:00 2024-09-19T08:49:50.254 INFO:teuthology.orchestra.run.smithi120.stdout:(6/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 13 MB/s | 4.7 MB 00:00 2024-09-19T08:49:50.263 INFO:teuthology.orchestra.run.smithi196.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 2.1 MB/s | 145 kB 00:00 2024-09-19T08:49:50.344 INFO:teuthology.orchestra.run.smithi153.stdout:(4/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 5.5 MB/s | 5.4 MB 00:00 2024-09-19T08:49:50.479 INFO:teuthology.orchestra.run.smithi120.stdout:(7/156): ceph-common-19.1.1-361.g2f599a46.el9.x 16 MB/s | 21 MB 00:01 2024-09-19T08:49:50.502 INFO:teuthology.orchestra.run.smithi153.stdout:(5/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 9.6 MB/s | 1.5 MB 00:00 2024-09-19T08:49:50.529 INFO:teuthology.orchestra.run.smithi120.stdout:(8/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 2.9 MB/s | 2.4 MB 00:00 2024-09-19T08:49:50.630 INFO:teuthology.orchestra.run.smithi120.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 251 kB/s | 25 kB 00:00 2024-09-19T08:49:50.986 INFO:teuthology.orchestra.run.smithi153.stdout:(6/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 3.3 MB/s | 2.4 MB 00:00 2024-09-19T08:49:51.328 INFO:teuthology.orchestra.run.smithi153.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 5.7 MB/s | 4.7 MB 00:00 2024-09-19T08:49:51.380 INFO:teuthology.orchestra.run.smithi120.stdout:(10/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 12 MB/s | 11 MB 00:00 2024-09-19T08:49:51.444 INFO:teuthology.orchestra.run.smithi196.stdout:(4/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 2.5 MB/s | 5.4 MB 00:02 2024-09-19T08:49:51.547 INFO:teuthology.orchestra.run.smithi120.stdout:(11/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 13 MB/s | 17 MB 00:01 2024-09-19T08:49:51.572 INFO:teuthology.orchestra.run.smithi120.stdout:(12/156): cephfs-mirror-19.1.1-361.g2f599a46.el 1.2 MB/s | 228 kB 00:00 2024-09-19T08:49:51.597 INFO:teuthology.orchestra.run.smithi120.stdout:(13/156): cephfs-shell-19.1.1-361.g2f599a46.el9 540 kB/s | 27 kB 00:00 2024-09-19T08:49:51.623 INFO:teuthology.orchestra.run.smithi120.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 630 kB/s | 32 kB 00:00 2024-09-19T08:49:51.655 INFO:teuthology.orchestra.run.smithi196.stdout:(5/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 1.7 MB/s | 2.4 MB 00:01 2024-09-19T08:49:51.664 INFO:teuthology.orchestra.run.smithi120.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 11 MB/s | 751 kB 00:00 2024-09-19T08:49:51.690 INFO:teuthology.orchestra.run.smithi120.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.4 MB/s | 163 kB 00:00 2024-09-19T08:49:51.711 INFO:teuthology.orchestra.run.smithi153.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 9.0 MB/s | 21 MB 00:02 2024-09-19T08:49:51.715 INFO:teuthology.orchestra.run.smithi120.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 2.5 MB/s | 127 kB 00:00 2024-09-19T08:49:51.757 INFO:teuthology.orchestra.run.smithi120.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 7.4 MB/s | 506 kB 00:00 2024-09-19T08:49:51.778 INFO:teuthology.orchestra.run.smithi153.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 378 kB/s | 25 kB 00:00 2024-09-19T08:49:51.789 INFO:teuthology.orchestra.run.smithi196.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 4.4 MB/s | 1.5 MB 00:00 2024-09-19T08:49:51.858 INFO:teuthology.orchestra.run.smithi120.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 451 kB/s | 45 kB 00:00 2024-09-19T08:49:51.958 INFO:teuthology.orchestra.run.smithi120.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 1.3 MB/s | 136 kB 00:00 2024-09-19T08:49:52.066 INFO:teuthology.orchestra.run.smithi120.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 1.5 MB/s | 161 kB 00:00 2024-09-19T08:49:52.134 INFO:teuthology.orchestra.run.smithi120.stdout:(22/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 13 MB/s | 5.3 MB 00:00 2024-09-19T08:49:52.159 INFO:teuthology.orchestra.run.smithi120.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 3.4 MB/s | 320 kB 00:00 2024-09-19T08:49:52.226 INFO:teuthology.orchestra.run.smithi120.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 3.2 MB/s | 300 kB 00:00 2024-09-19T08:49:52.251 INFO:teuthology.orchestra.run.smithi120.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.1 MB/s | 99 kB 00:00 2024-09-19T08:49:52.318 INFO:teuthology.orchestra.run.smithi120.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 926 kB/s | 85 kB 00:00 2024-09-19T08:49:52.660 INFO:teuthology.orchestra.run.smithi120.stdout:(27/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 7.6 MB/s | 3.1 MB 00:00 2024-09-19T08:49:52.711 INFO:teuthology.orchestra.run.smithi120.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 635 kB/s | 32 kB 00:00 2024-09-19T08:49:52.754 INFO:teuthology.orchestra.run.smithi153.stdout:(10/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 7.4 MB/s | 11 MB 00:01 2024-09-19T08:49:52.770 INFO:teuthology.orchestra.run.smithi120.stdout:(29/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.4 MB/s | 146 kB 00:00 2024-09-19T08:49:52.820 INFO:teuthology.orchestra.run.smithi153.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 3.4 MB/s | 228 kB 00:00 2024-09-19T08:49:52.904 INFO:teuthology.orchestra.run.smithi153.stdout:(12/156): cephfs-shell-19.1.1-361.g2f599a46.el9 325 kB/s | 27 kB 00:00 2024-09-19T08:49:52.987 INFO:teuthology.orchestra.run.smithi153.stdout:(13/156): libcephfs-devel-19.1.1-361.g2f599a46. 382 kB/s | 32 kB 00:00 2024-09-19T08:49:53.061 INFO:teuthology.orchestra.run.smithi120.stdout:(30/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 230 kB/s | 171 kB 00:00 2024-09-19T08:49:53.129 INFO:teuthology.orchestra.run.smithi153.stdout:(14/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 5.2 MB/s | 751 kB 00:00 2024-09-19T08:49:53.137 INFO:teuthology.orchestra.run.smithi120.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 9.7 MB/s | 3.6 MB 00:00 2024-09-19T08:49:53.196 INFO:teuthology.orchestra.run.smithi153.stdout:(15/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.4 MB/s | 163 kB 00:00 2024-09-19T08:49:53.254 INFO:teuthology.orchestra.run.smithi120.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 2.1 MB/s | 249 kB 00:00 2024-09-19T08:49:53.296 INFO:teuthology.orchestra.run.smithi153.stdout:(16/156): librados-devel-19.1.1-361.g2f599a46.e 1.2 MB/s | 127 kB 00:00 2024-09-19T08:49:53.337 INFO:teuthology.orchestra.run.smithi120.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 592 kB/s | 49 kB 00:00 2024-09-19T08:49:53.421 INFO:teuthology.orchestra.run.smithi120.stdout:(34/156): ceph-prometheus-alerts-19.1.1-361.g2f 197 kB/s | 16 kB 00:00 2024-09-19T08:49:53.455 INFO:teuthology.orchestra.run.smithi153.stdout:(17/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 6.8 MB/s | 17 MB 00:02 2024-09-19T08:49:53.480 INFO:teuthology.orchestra.run.smithi153.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 2.7 MB/s | 506 kB 00:00 2024-09-19T08:49:53.496 INFO:teuthology.orchestra.run.smithi120.stdout:(35/156): ceph-volume-19.1.1-361.g2f599a46.el9. 3.7 MB/s | 285 kB 00:00 2024-09-19T08:49:53.564 INFO:teuthology.orchestra.run.smithi153.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 541 kB/s | 45 kB 00:00 2024-09-19T08:49:53.613 INFO:teuthology.orchestra.run.smithi120.stdout:(36/156): cephadm-19.1.1-361.g2f599a46.el9.noar 6.4 MB/s | 765 kB 00:00 2024-09-19T08:49:53.664 INFO:teuthology.orchestra.run.smithi153.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 1.3 MB/s | 136 kB 00:00 2024-09-19T08:49:53.722 INFO:teuthology.orchestra.run.smithi120.stdout:(37/156): ceph-mgr-diskprediction-local-19.1.1- 11 MB/s | 7.4 MB 00:00 2024-09-19T08:49:53.739 INFO:teuthology.orchestra.run.smithi153.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 2.1 MB/s | 161 kB 00:00 2024-09-19T08:49:53.747 INFO:teuthology.orchestra.run.smithi120.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 136 kB/s | 18 kB 00:00 2024-09-19T08:49:53.815 INFO:teuthology.orchestra.run.smithi153.stdout:(22/156): python3-rados-19.1.1-361.g2f599a46.el 4.2 MB/s | 320 kB 00:00 2024-09-19T08:49:53.881 INFO:teuthology.orchestra.run.smithi120.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 503 kB/s | 80 kB 00:00 2024-09-19T08:49:53.907 INFO:teuthology.orchestra.run.smithi153.stdout:(23/156): python3-rbd-19.1.1-361.g2f599a46.el9. 3.2 MB/s | 300 kB 00:00 2024-09-19T08:49:53.982 INFO:teuthology.orchestra.run.smithi153.stdout:(24/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.3 MB/s | 99 kB 00:00 2024-09-19T08:49:54.006 INFO:teuthology.orchestra.run.smithi120.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.5 MB/s | 191 kB 00:00 2024-09-19T08:49:54.091 INFO:teuthology.orchestra.run.smithi153.stdout:(25/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 8.4 MB/s | 5.3 MB 00:00 2024-09-19T08:49:54.116 INFO:teuthology.orchestra.run.smithi153.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 639 kB/s | 85 kB 00:00 2024-09-19T08:49:54.131 INFO:teuthology.orchestra.run.smithi120.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-09-19T08:49:54.200 INFO:teuthology.orchestra.run.smithi153.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 2.0 MB/s | 171 kB 00:00 2024-09-19T08:49:54.283 INFO:teuthology.orchestra.run.smithi153.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 383 kB/s | 32 kB 00:00 2024-09-19T08:49:54.290 INFO:teuthology.orchestra.run.smithi120.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.4 MB/s | 801 kB 00:00 2024-09-19T08:49:54.367 INFO:teuthology.orchestra.run.smithi153.stdout:(29/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 1.7 MB/s | 146 kB 00:00 2024-09-19T08:49:54.374 INFO:teuthology.orchestra.run.smithi120.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-09-19T08:49:54.467 INFO:teuthology.orchestra.run.smithi153.stdout:(30/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 8.2 MB/s | 3.1 MB 00:00 2024-09-19T08:49:54.516 INFO:teuthology.orchestra.run.smithi120.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 953 kB/s | 135 kB 00:00 2024-09-19T08:49:54.608 INFO:teuthology.orchestra.run.smithi120.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 2.6 MB/s | 1.2 MB 00:00 2024-09-19T08:49:54.633 INFO:teuthology.orchestra.run.smithi120.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-09-19T08:49:54.639 INFO:teuthology.orchestra.run.smithi196.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 1.6 MB/s | 4.7 MB 00:02 2024-09-19T08:49:54.959 INFO:teuthology.orchestra.run.smithi153.stdout:(31/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 15 MB/s | 49 MB 00:03 2024-09-19T08:49:55.043 INFO:teuthology.orchestra.run.smithi153.stdout:(32/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 5.3 MB/s | 3.6 MB 00:00 2024-09-19T08:49:55.068 INFO:teuthology.orchestra.run.smithi153.stdout:(33/156): ceph-mgr-modules-core-19.1.1-361.g2f5 2.3 MB/s | 249 kB 00:00 2024-09-19T08:49:55.084 INFO:teuthology.orchestra.run.smithi120.stdout:(47/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 11 MB/s | 49 MB 00:04 2024-09-19T08:49:55.093 INFO:teuthology.orchestra.run.smithi153.stdout:(34/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 986 kB/s | 49 kB 00:00 2024-09-19T08:49:55.117 INFO:teuthology.orchestra.run.smithi120.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 422 kB/s | 215 kB 00:00 2024-09-19T08:49:55.118 INFO:teuthology.orchestra.run.smithi153.stdout:(35/156): ceph-prometheus-alerts-19.1.1-361.g2f 328 kB/s | 16 kB 00:00 2024-09-19T08:49:55.169 INFO:teuthology.orchestra.run.smithi153.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 3.7 MB/s | 285 kB 00:00 2024-09-19T08:49:55.222 INFO:teuthology.orchestra.run.smithi196.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 3.6 MB/s | 21 MB 00:05 2024-09-19T08:49:55.244 INFO:teuthology.orchestra.run.smithi153.stdout:(37/156): ceph-mgr-diskprediction-local-19.1.1- 9.5 MB/s | 7.4 MB 00:00 2024-09-19T08:49:55.264 INFO:teuthology.orchestra.run.smithi196.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 608 kB/s | 25 kB 00:00 2024-09-19T08:49:55.284 INFO:teuthology.orchestra.run.smithi120.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 641 kB/s | 107 kB 00:00 2024-09-19T08:49:55.289 INFO:teuthology.orchestra.run.smithi153.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 152 kB/s | 18 kB 00:00 2024-09-19T08:49:55.320 INFO:teuthology.orchestra.run.smithi153.stdout:(39/156): cephadm-19.1.1-361.g2f599a46.el9.noar 3.7 MB/s | 765 kB 00:00 2024-09-19T08:49:55.326 INFO:teuthology.orchestra.run.smithi120.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 522 kB/s | 22 kB 00:00 2024-09-19T08:49:55.368 INFO:teuthology.orchestra.run.smithi120.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-09-19T08:49:55.419 INFO:teuthology.orchestra.run.smithi120.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-09-19T08:49:55.444 INFO:teuthology.orchestra.run.smithi120.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 434 kB/s | 33 kB 00:00 2024-09-19T08:49:55.478 INFO:teuthology.orchestra.run.smithi120.stdout:(54/156): automake-1.16.2-8.el9.noarch.rpm 816 kB/s | 689 kB 00:00 2024-09-19T08:49:55.503 INFO:teuthology.orchestra.run.smithi120.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 310 kB/s | 18 kB 00:00 2024-09-19T08:49:55.528 INFO:teuthology.orchestra.run.smithi120.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-09-19T08:49:55.562 INFO:teuthology.orchestra.run.smithi120.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-19T08:49:55.588 INFO:teuthology.orchestra.run.smithi120.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 774 kB/s | 45 kB 00:00 2024-09-19T08:49:55.654 INFO:teuthology.orchestra.run.smithi120.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 8.5 MB/s | 581 kB 00:00 2024-09-19T08:49:55.696 INFO:teuthology.orchestra.run.smithi120.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 590 kB/s | 24 kB 00:00 2024-09-19T08:49:55.721 INFO:teuthology.orchestra.run.smithi120.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-09-19T08:49:55.747 INFO:teuthology.orchestra.run.smithi120.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 940 kB/s | 47 kB 00:00 2024-09-19T08:49:55.770 INFO:teuthology.orchestra.run.smithi153.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 152 kB/s | 80 kB 00:00 2024-09-19T08:49:55.780 INFO:teuthology.orchestra.run.smithi120.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-09-19T08:49:55.822 INFO:teuthology.orchestra.run.smithi120.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-09-19T08:49:55.848 INFO:teuthology.orchestra.run.smithi120.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 575 kB/s | 38 kB 00:00 2024-09-19T08:49:55.878 INFO:teuthology.orchestra.run.smithi153.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 342 kB/s | 191 kB 00:00 2024-09-19T08:49:56.128 INFO:teuthology.orchestra.run.smithi153.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 955 kB/s | 801 kB 00:00 2024-09-19T08:49:56.139 INFO:teuthology.orchestra.run.smithi196.stdout:(10/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 3.9 MB/s | 17 MB 00:04 2024-09-19T08:49:56.153 INFO:teuthology.orchestra.run.smithi153.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 660 kB/s | 253 kB 00:00 2024-09-19T08:49:56.198 INFO:teuthology.orchestra.run.smithi120.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.8 MB/s | 3.0 MB 00:00 2024-09-19T08:49:56.215 INFO:teuthology.orchestra.run.smithi196.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 3.0 MB/s | 228 kB 00:00 2024-09-19T08:49:56.237 INFO:teuthology.orchestra.run.smithi153.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 981 kB/s | 106 kB 00:00 2024-09-19T08:49:56.257 INFO:teuthology.orchestra.run.smithi196.stdout:(12/156): cephfs-shell-19.1.1-361.g2f599a46.el9 654 kB/s | 27 kB 00:00 2024-09-19T08:49:56.262 INFO:teuthology.orchestra.run.smithi153.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-09-19T08:49:56.282 INFO:teuthology.orchestra.run.smithi120.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-09-19T08:49:56.298 INFO:teuthology.orchestra.run.smithi196.stdout:(13/156): libcephfs-devel-19.1.1-361.g2f599a46. 763 kB/s | 32 kB 00:00 2024-09-19T08:49:56.346 INFO:teuthology.orchestra.run.smithi153.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-09-19T08:49:56.380 INFO:teuthology.orchestra.run.smithi153.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-09-19T08:49:56.382 INFO:teuthology.orchestra.run.smithi120.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 8.9 MB/s | 910 kB 00:00 2024-09-19T08:49:56.408 INFO:teuthology.orchestra.run.smithi196.stdout:(14/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 6.8 MB/s | 751 kB 00:00 2024-09-19T08:49:56.424 INFO:teuthology.orchestra.run.smithi120.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 3.2 MB/s | 137 kB 00:00 2024-09-19T08:49:56.438 INFO:teuthology.orchestra.run.smithi153.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 2.2 MB/s | 1.2 MB 00:00 2024-09-19T08:49:56.457 INFO:teuthology.orchestra.run.smithi196.stdout:(15/156): libcephsqlite-19.1.1-361.g2f599a46.el 3.2 MB/s | 163 kB 00:00 2024-09-19T08:49:56.539 INFO:teuthology.orchestra.run.smithi153.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 689 kB 00:00 2024-09-19T08:49:56.558 INFO:teuthology.orchestra.run.smithi196.stdout:(16/156): librados-devel-19.1.1-361.g2f599a46.e 1.2 MB/s | 127 kB 00:00 2024-09-19T08:49:56.564 INFO:teuthology.orchestra.run.smithi153.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 854 kB/s | 107 kB 00:00 2024-09-19T08:49:56.589 INFO:teuthology.orchestra.run.smithi153.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-09-19T08:49:56.622 INFO:teuthology.orchestra.run.smithi153.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 943 kB 00:00 2024-09-19T08:49:56.648 INFO:teuthology.orchestra.run.smithi153.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 559 kB/s | 33 kB 00:00 2024-09-19T08:49:56.649 INFO:teuthology.orchestra.run.smithi120.stdout:(70/156): postgresql-13.16-1.el9.x86_64.rpm 3.5 MB/s | 1.6 MB 00:00 2024-09-19T08:49:56.649 INFO:teuthology.orchestra.run.smithi196.stdout:(17/156): libradosstriper1-19.1.1-361.g2f599a46 5.4 MB/s | 506 kB 00:00 2024-09-19T08:49:56.673 INFO:teuthology.orchestra.run.smithi153.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-09-19T08:49:56.698 INFO:teuthology.orchestra.run.smithi153.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-09-19T08:49:56.723 INFO:teuthology.orchestra.run.smithi153.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-19T08:49:56.749 INFO:teuthology.orchestra.run.smithi153.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-19T08:49:56.774 INFO:teuthology.orchestra.run.smithi153.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 899 kB/s | 45 kB 00:00 2024-09-19T08:49:56.808 INFO:teuthology.orchestra.run.smithi153.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-19T08:49:56.841 INFO:teuthology.orchestra.run.smithi153.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 8.5 MB/s | 581 kB 00:00 2024-09-19T08:49:56.850 INFO:teuthology.orchestra.run.smithi196.stdout:(18/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 4.8 MB/s | 11 MB 00:02 2024-09-19T08:49:56.866 INFO:teuthology.orchestra.run.smithi153.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 418 kB/s | 24 kB 00:00 2024-09-19T08:49:56.892 INFO:teuthology.orchestra.run.smithi153.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-19T08:49:56.900 INFO:teuthology.orchestra.run.smithi196.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 910 kB/s | 45 kB 00:00 2024-09-19T08:49:56.917 INFO:teuthology.orchestra.run.smithi153.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-19T08:49:56.941 INFO:teuthology.orchestra.run.smithi120.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 3.5 MB/s | 1.0 MB 00:00 2024-09-19T08:49:56.942 INFO:teuthology.orchestra.run.smithi153.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-19T08:49:56.959 INFO:teuthology.orchestra.run.smithi196.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 2.3 MB/s | 136 kB 00:00 2024-09-19T08:49:56.968 INFO:teuthology.orchestra.run.smithi153.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 766 kB/s | 38 kB 00:00 2024-09-19T08:49:57.016 INFO:teuthology.orchestra.run.smithi120.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 9.8 MB/s | 5.8 MB 00:00 2024-09-19T08:49:57.026 INFO:teuthology.orchestra.run.smithi196.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 2.4 MB/s | 161 kB 00:00 2024-09-19T08:49:57.026 INFO:teuthology.orchestra.run.smithi153.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.4 MB/s | 3.0 MB 00:00 2024-09-19T08:49:57.042 INFO:teuthology.orchestra.run.smithi120.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 873 kB/s | 87 kB 00:00 2024-09-19T08:49:57.086 INFO:teuthology.orchestra.run.smithi120.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-19T08:49:57.093 INFO:teuthology.orchestra.run.smithi153.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-09-19T08:49:57.101 INFO:teuthology.orchestra.run.smithi196.stdout:(22/156): python3-rados-19.1.1-361.g2f599a46.el 4.2 MB/s | 320 kB 00:00 2024-09-19T08:49:57.160 INFO:teuthology.orchestra.run.smithi153.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 23 MB/s | 4.9 MB 00:00 2024-09-19T08:49:57.176 INFO:teuthology.orchestra.run.smithi120.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-09-19T08:49:57.176 INFO:teuthology.orchestra.run.smithi196.stdout:(23/156): python3-rbd-19.1.1-361.g2f599a46.el9. 3.9 MB/s | 300 kB 00:00 2024-09-19T08:49:57.227 INFO:teuthology.orchestra.run.smithi196.stdout:(24/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.9 MB/s | 99 kB 00:00 2024-09-19T08:49:57.227 INFO:teuthology.orchestra.run.smithi153.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-09-19T08:49:57.243 INFO:teuthology.orchestra.run.smithi120.stdout:(76/156): openblas-openmp-0.3.26-2.el9.x86_64.r 3.5 MB/s | 4.9 MB 00:01 2024-09-19T08:49:57.253 INFO:teuthology.orchestra.run.smithi153.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.5 MB/s | 137 kB 00:00 2024-09-19T08:49:57.277 INFO:teuthology.orchestra.run.smithi196.stdout:(25/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-19T08:49:57.291 INFO:teuthology.orchestra.run.smithi153.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 4.6 MB/s | 910 kB 00:00 2024-09-19T08:49:57.293 INFO:teuthology.orchestra.run.smithi120.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 514 kB/s | 26 kB 00:00 2024-09-19T08:49:57.319 INFO:teuthology.orchestra.run.smithi153.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-09-19T08:49:57.343 INFO:teuthology.orchestra.run.smithi120.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-19T08:49:57.345 INFO:teuthology.orchestra.run.smithi153.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-19T08:49:57.387 INFO:teuthology.orchestra.run.smithi153.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-19T08:49:57.400 INFO:teuthology.orchestra.run.smithi120.stdout:(79/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.1 MB/s | 248 kB 00:00 2024-09-19T08:49:57.427 INFO:teuthology.orchestra.run.smithi120.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-19T08:49:57.436 INFO:teuthology.orchestra.run.smithi196.stdout:(26/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 6.8 MB/s | 5.3 MB 00:00 2024-09-19T08:49:57.453 INFO:teuthology.orchestra.run.smithi120.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 669 kB/s | 35 kB 00:00 2024-09-19T08:49:57.462 INFO:teuthology.orchestra.run.smithi153.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 25 MB/s | 5.8 MB 00:00 2024-09-19T08:49:57.486 INFO:teuthology.orchestra.run.smithi196.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 3.4 MB/s | 171 kB 00:00 2024-09-19T08:49:57.495 INFO:teuthology.orchestra.run.smithi153.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-19T08:49:57.521 INFO:teuthology.orchestra.run.smithi153.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-09-19T08:49:57.536 INFO:teuthology.orchestra.run.smithi196.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 640 kB/s | 32 kB 00:00 2024-09-19T08:49:57.546 INFO:teuthology.orchestra.run.smithi153.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-09-19T08:49:57.563 INFO:teuthology.orchestra.run.smithi153.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 459 kB/s | 19 kB 00:00 2024-09-19T08:49:57.586 INFO:teuthology.orchestra.run.smithi120.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.8 MB/s | 250 kB 00:00 2024-09-19T08:49:57.588 INFO:teuthology.orchestra.run.smithi153.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-09-19T08:49:57.589 INFO:teuthology.orchestra.run.smithi196.stdout:(29/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.7 MB/s | 146 kB 00:00 2024-09-19T08:49:57.613 INFO:teuthology.orchestra.run.smithi153.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-09-19T08:49:57.689 INFO:teuthology.orchestra.run.smithi153.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-19T08:49:57.712 INFO:teuthology.orchestra.run.smithi120.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.7 MB/s | 222 kB 00:00 2024-09-19T08:49:57.714 INFO:teuthology.orchestra.run.smithi153.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-19T08:49:57.739 INFO:teuthology.orchestra.run.smithi153.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-19T08:49:57.765 INFO:teuthology.orchestra.run.smithi153.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-19T08:49:57.790 INFO:teuthology.orchestra.run.smithi153.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 845 kB/s | 42 kB 00:00 2024-09-19T08:49:57.804 INFO:teuthology.orchestra.run.smithi120.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 841 kB/s | 77 kB 00:00 2024-09-19T08:49:57.814 INFO:teuthology.orchestra.run.smithi196.stdout:(30/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 5.8 MB/s | 3.1 MB 00:00 2024-09-19T08:49:57.815 INFO:teuthology.orchestra.run.smithi153.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-09-19T08:49:57.840 INFO:teuthology.orchestra.run.smithi153.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-19T08:49:57.854 INFO:teuthology.orchestra.run.smithi120.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 848 kB/s | 42 kB 00:00 2024-09-19T08:49:57.865 INFO:teuthology.orchestra.run.smithi153.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-19T08:49:57.882 INFO:teuthology.orchestra.run.smithi153.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-19T08:49:57.899 INFO:teuthology.orchestra.run.smithi153.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-09-19T08:49:57.983 INFO:teuthology.orchestra.run.smithi153.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-09-19T08:49:58.000 INFO:teuthology.orchestra.run.smithi153.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 416 kB/s | 42 kB 00:00 2024-09-19T08:49:58.017 INFO:teuthology.orchestra.run.smithi153.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-09-19T08:49:58.039 INFO:teuthology.orchestra.run.smithi196.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 7.9 MB/s | 3.6 MB 00:00 2024-09-19T08:49:58.042 INFO:teuthology.orchestra.run.smithi153.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-09-19T08:49:58.054 INFO:teuthology.orchestra.run.smithi120.stdout:(86/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.7 MB/s | 6.0 MB 00:01 2024-09-19T08:49:58.059 INFO:teuthology.orchestra.run.smithi153.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 860 kB/s | 36 kB 00:00 2024-09-19T08:49:58.080 INFO:teuthology.orchestra.run.smithi120.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 1.2 MB/s | 269 kB 00:00 2024-09-19T08:49:58.084 INFO:teuthology.orchestra.run.smithi153.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-19T08:49:58.106 INFO:teuthology.orchestra.run.smithi196.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 3.7 MB/s | 249 kB 00:00 2024-09-19T08:49:58.110 INFO:teuthology.orchestra.run.smithi153.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-19T08:49:58.113 INFO:teuthology.orchestra.run.smithi120.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-09-19T08:49:58.135 INFO:teuthology.orchestra.run.smithi153.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-19T08:49:58.147 INFO:teuthology.orchestra.run.smithi120.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-09-19T08:49:58.148 INFO:teuthology.orchestra.run.smithi196.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 1.2 MB/s | 49 kB 00:00 2024-09-19T08:49:58.160 INFO:teuthology.orchestra.run.smithi153.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-19T08:49:58.172 INFO:teuthology.orchestra.run.smithi120.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 868 kB/s | 51 kB 00:00 2024-09-19T08:49:58.185 INFO:teuthology.orchestra.run.smithi153.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-19T08:49:58.190 INFO:teuthology.orchestra.run.smithi196.stdout:(34/156): ceph-prometheus-alerts-19.1.1-361.g2f 397 kB/s | 16 kB 00:00 2024-09-19T08:49:58.206 INFO:teuthology.orchestra.run.smithi120.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 914 kB/s | 54 kB 00:00 2024-09-19T08:49:58.211 INFO:teuthology.orchestra.run.smithi153.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-09-19T08:49:58.257 INFO:teuthology.orchestra.run.smithi196.stdout:(35/156): ceph-volume-19.1.1-361.g2f599a46.el9. 4.2 MB/s | 285 kB 00:00 2024-09-19T08:49:58.273 INFO:teuthology.orchestra.run.smithi120.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 626 kB/s | 42 kB 00:00 2024-09-19T08:49:58.340 INFO:teuthology.orchestra.run.smithi120.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 663 kB/s | 44 kB 00:00 2024-09-19T08:49:58.382 INFO:teuthology.orchestra.run.smithi196.stdout:(36/156): cephadm-19.1.1-361.g2f599a46.el9.noar 6.0 MB/s | 765 kB 00:00 2024-09-19T08:49:58.407 INFO:teuthology.orchestra.run.smithi120.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-09-19T08:49:58.424 INFO:teuthology.orchestra.run.smithi196.stdout:(37/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 438 kB/s | 18 kB 00:00 2024-09-19T08:49:58.436 INFO:teuthology.orchestra.run.smithi153.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-09-19T08:49:58.462 INFO:teuthology.orchestra.run.smithi153.stdout:(104/156): lua-devel-5.4.4-4.el9.x86_64.rpm 81 kB/s | 22 kB 00:00 2024-09-19T08:49:58.465 INFO:teuthology.orchestra.run.smithi120.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 618 kB/s | 36 kB 00:00 2024-09-19T08:49:58.520 INFO:teuthology.orchestra.run.smithi153.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 6.4 MB/s | 548 kB 00:00 2024-09-19T08:49:58.532 INFO:teuthology.orchestra.run.smithi120.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 711 kB/s | 47 kB 00:00 2024-09-19T08:49:58.545 INFO:teuthology.orchestra.run.smithi153.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 789 kB/s | 19 kB 00:00 2024-09-19T08:49:58.570 INFO:teuthology.orchestra.run.smithi153.stdout:(107/156): gperftools-libs-2.9.1-3.el9.x86_64.r 2.8 MB/s | 308 kB 00:00 2024-09-19T08:49:58.596 INFO:teuthology.orchestra.run.smithi153.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-19T08:49:58.599 INFO:teuthology.orchestra.run.smithi120.stdout:(97/156): socat-1.7.4.1-6.el9.x86_64.rpm 4.5 MB/s | 304 kB 00:00 2024-09-19T08:49:58.621 INFO:teuthology.orchestra.run.smithi153.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-19T08:49:58.646 INFO:teuthology.orchestra.run.smithi153.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-19T08:49:58.666 INFO:teuthology.orchestra.run.smithi120.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 867 kB/s | 58 kB 00:00 2024-09-19T08:49:58.697 INFO:teuthology.orchestra.run.smithi153.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-19T08:49:58.733 INFO:teuthology.orchestra.run.smithi120.stdout:(99/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 1.4 MB/s | 96 kB 00:00 2024-09-19T08:49:58.755 INFO:teuthology.orchestra.run.smithi153.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-19T08:49:58.792 INFO:teuthology.orchestra.run.smithi120.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-19T08:49:58.805 INFO:teuthology.orchestra.run.smithi153.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-19T08:49:58.825 INFO:teuthology.orchestra.run.smithi196.stdout:(38/156): ceph-mgr-diskprediction-local-19.1.1- 7.3 MB/s | 7.4 MB 00:01 2024-09-19T08:49:58.851 INFO:teuthology.orchestra.run.smithi120.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 769 kB/s | 45 kB 00:00 2024-09-19T08:49:58.872 INFO:teuthology.orchestra.run.smithi153.stdout:(114/156): libarrow-9.0.0-11.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-09-19T08:49:58.897 INFO:teuthology.orchestra.run.smithi153.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 473 kB/s | 43 kB 00:00 2024-09-19T08:49:58.924 INFO:teuthology.orchestra.run.smithi153.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-09-19T08:49:58.943 INFO:teuthology.orchestra.run.smithi120.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 243 kB/s | 22 kB 00:00 2024-09-19T08:49:58.948 INFO:teuthology.orchestra.run.smithi153.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-19T08:49:58.973 INFO:teuthology.orchestra.run.smithi153.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-09-19T08:49:58.998 INFO:teuthology.orchestra.run.smithi153.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-19T08:49:59.032 INFO:teuthology.orchestra.run.smithi153.stdout:(120/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.0 MB/s | 863 kB 00:00 2024-09-19T08:49:59.066 INFO:teuthology.orchestra.run.smithi153.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.2 MB/s | 220 kB 00:00 2024-09-19T08:49:59.091 INFO:teuthology.orchestra.run.smithi153.stdout:(122/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-09-19T08:49:59.100 INFO:teuthology.orchestra.run.smithi196.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 118 kB/s | 80 kB 00:00 2024-09-19T08:49:59.116 INFO:teuthology.orchestra.run.smithi153.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-19T08:49:59.118 INFO:teuthology.orchestra.run.smithi120.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 4.8 MB/s | 863 kB 00:00 2024-09-19T08:49:59.141 INFO:teuthology.orchestra.run.smithi153.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-09-19T08:49:59.167 INFO:teuthology.orchestra.run.smithi153.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 354 kB/s | 18 kB 00:00 2024-09-19T08:49:59.176 INFO:teuthology.orchestra.run.smithi120.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-19T08:49:59.192 INFO:teuthology.orchestra.run.smithi153.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-19T08:49:59.210 INFO:teuthology.orchestra.run.smithi120.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-19T08:49:59.217 INFO:teuthology.orchestra.run.smithi153.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-19T08:49:59.236 INFO:teuthology.orchestra.run.smithi120.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 778 kB/s | 19 kB 00:00 2024-09-19T08:49:59.242 INFO:teuthology.orchestra.run.smithi153.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-19T08:49:59.276 INFO:teuthology.orchestra.run.smithi153.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-19T08:49:59.302 INFO:teuthology.orchestra.run.smithi153.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T08:49:59.335 INFO:teuthology.orchestra.run.smithi153.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-09-19T08:49:59.360 INFO:teuthology.orchestra.run.smithi153.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 990 kB/s | 58 kB 00:00 2024-09-19T08:49:59.378 INFO:teuthology.orchestra.run.smithi120.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 31 MB/s | 4.4 MB 00:00 2024-09-19T08:49:59.386 INFO:teuthology.orchestra.run.smithi153.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 655 kB/s | 16 kB 00:00 2024-09-19T08:49:59.403 INFO:teuthology.orchestra.run.smithi120.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-19T08:49:59.411 INFO:teuthology.orchestra.run.smithi153.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-19T08:49:59.428 INFO:teuthology.orchestra.run.smithi120.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-19T08:49:59.436 INFO:teuthology.orchestra.run.smithi153.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-19T08:49:59.454 INFO:teuthology.orchestra.run.smithi120.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-19T08:49:59.462 INFO:teuthology.orchestra.run.smithi153.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-09-19T08:49:59.487 INFO:teuthology.orchestra.run.smithi153.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-19T08:49:59.512 INFO:teuthology.orchestra.run.smithi153.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-19T08:49:59.521 INFO:teuthology.orchestra.run.smithi120.stdout:(111/156): python3-numpy-1.20.1-5.el9.x86_64.rp 2.4 MB/s | 5.1 MB 00:02 2024-09-19T08:49:59.538 INFO:teuthology.orchestra.run.smithi153.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-19T08:49:59.554 INFO:teuthology.orchestra.run.smithi120.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-09-19T08:49:59.562 INFO:teuthology.orchestra.run.smithi153.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-09-19T08:49:59.588 INFO:teuthology.orchestra.run.smithi153.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 2.9 MB/s | 74 kB 00:00 2024-09-19T08:49:59.588 INFO:teuthology.orchestra.run.smithi120.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 17 MB/s | 548 kB 00:00 2024-09-19T08:49:59.613 INFO:teuthology.orchestra.run.smithi153.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-19T08:49:59.614 INFO:teuthology.orchestra.run.smithi120.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-19T08:49:59.639 INFO:teuthology.orchestra.run.smithi120.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-19T08:49:59.655 INFO:teuthology.orchestra.run.smithi153.stdout:(143/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.3 MB/s | 2.0 MB 00:00 2024-09-19T08:49:59.664 INFO:teuthology.orchestra.run.smithi120.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 562 kB/s | 14 kB 00:00 2024-09-19T08:49:59.688 INFO:teuthology.orchestra.run.smithi153.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-19T08:49:59.689 INFO:teuthology.orchestra.run.smithi120.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-19T08:49:59.714 INFO:teuthology.orchestra.run.smithi153.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 799 kB/s | 20 kB 00:00 2024-09-19T08:49:59.714 INFO:teuthology.orchestra.run.smithi120.stdout:(118/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.2 MB/s | 839 kB 00:00 2024-09-19T08:49:59.739 INFO:teuthology.orchestra.run.smithi153.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T08:49:59.740 INFO:teuthology.orchestra.run.smithi120.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.9 MB/s | 358 kB 00:00 2024-09-19T08:49:59.764 INFO:teuthology.orchestra.run.smithi153.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-09-19T08:49:59.773 INFO:teuthology.orchestra.run.smithi120.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.0 MB/s | 303 kB 00:00 2024-09-19T08:49:59.799 INFO:teuthology.orchestra.run.smithi120.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-19T08:49:59.814 INFO:teuthology.orchestra.run.smithi153.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-19T08:49:59.824 INFO:teuthology.orchestra.run.smithi120.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-19T08:49:59.850 INFO:teuthology.orchestra.run.smithi120.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-19T08:49:59.875 INFO:teuthology.orchestra.run.smithi120.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 702 kB/s | 18 kB 00:00 2024-09-19T08:49:59.900 INFO:teuthology.orchestra.run.smithi120.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 929 kB/s | 23 kB 00:00 2024-09-19T08:49:59.926 INFO:teuthology.orchestra.run.smithi120.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 775 kB/s | 19 kB 00:00 2024-09-19T08:49:59.951 INFO:teuthology.orchestra.run.smithi120.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 789 kB/s | 20 kB 00:00 2024-09-19T08:50:00.010 INFO:teuthology.orchestra.run.smithi120.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.6 MB/s | 2.0 MB 00:00 2024-09-19T08:50:00.015 INFO:teuthology.orchestra.run.smithi153.stdout:(149/156): librados2-19.1.1-361.g2f599a46.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-19T08:50:00.035 INFO:teuthology.orchestra.run.smithi120.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T08:50:00.050 INFO:teuthology.orchestra.run.smithi196.stdout:(40/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 10 MB/s | 49 MB 00:04 2024-09-19T08:50:00.060 INFO:teuthology.orchestra.run.smithi120.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.0 MB/s | 79 kB 00:00 2024-09-19T08:50:00.075 INFO:teuthology.orchestra.run.smithi196.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 196 kB/s | 191 kB 00:00 2024-09-19T08:50:00.102 INFO:teuthology.orchestra.run.smithi120.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-09-19T08:50:00.117 INFO:teuthology.orchestra.run.smithi196.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 620 kB/s | 801 kB 00:01 2024-09-19T08:50:00.128 INFO:teuthology.orchestra.run.smithi120.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 861 kB/s | 58 kB 00:00 2024-09-19T08:50:00.153 INFO:teuthology.orchestra.run.smithi120.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 656 kB/s | 16 kB 00:00 2024-09-19T08:50:00.178 INFO:teuthology.orchestra.run.smithi120.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-19T08:50:00.198 INFO:teuthology.orchestra.run.smithi153.stdout:(150/156): librbd1-19.1.1-361.g2f599a46.el9.x86 7.3 MB/s | 3.2 MB 00:00 2024-09-19T08:50:00.204 INFO:teuthology.orchestra.run.smithi120.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-19T08:50:00.229 INFO:teuthology.orchestra.run.smithi120.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-09-19T08:50:00.252 INFO:teuthology.orchestra.run.smithi120.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 645 kB/s | 31 kB 00:00 2024-09-19T08:50:00.277 INFO:teuthology.orchestra.run.smithi120.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-19T08:50:00.302 INFO:teuthology.orchestra.run.smithi120.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-09-19T08:50:00.353 INFO:teuthology.orchestra.run.smithi120.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-09-19T08:50:00.378 INFO:teuthology.orchestra.run.smithi120.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 986 kB/s | 74 kB 00:00 2024-09-19T08:50:00.404 INFO:teuthology.orchestra.run.smithi120.stdout:(142/156): python3-webob-1.8.8-2.el9.noarch.rpm 4.4 MB/s | 230 kB 00:00 2024-09-19T08:50:00.429 INFO:teuthology.orchestra.run.smithi120.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-19T08:50:00.463 INFO:teuthology.orchestra.run.smithi120.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 7.1 MB/s | 427 kB 00:00 2024-09-19T08:50:00.488 INFO:teuthology.orchestra.run.smithi120.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 340 kB/s | 20 kB 00:00 2024-09-19T08:50:00.530 INFO:teuthology.orchestra.run.smithi120.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-19T08:50:00.581 INFO:teuthology.orchestra.run.smithi120.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-19T08:50:00.615 INFO:teuthology.orchestra.run.smithi153.stdout:(151/156): NetworkManager-team-1.51.0-1.el9.x86 98 kB/s | 41 kB 00:00 2024-09-19T08:50:00.784 INFO:teuthology.orchestra.run.smithi196.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 160 kB/s | 106 kB 00:00 2024-09-19T08:50:00.806 INFO:teuthology.orchestra.run.smithi120.stdout:(148/156): librados2-19.1.1-361.g2f599a46.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-19T08:50:00.809 INFO:teuthology.orchestra.run.smithi196.stdout:(44/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 334 kB/s | 253 kB 00:00 2024-09-19T08:50:00.873 INFO:teuthology.orchestra.run.smithi120.stdout:(149/156): librbd1-19.1.1-361.g2f599a46.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-19T08:50:00.874 INFO:teuthology.orchestra.run.smithi153.stdout:(152/156): NetworkManager-1.51.0-1.el9.x86_64.r 2.2 MB/s | 2.3 MB 00:01 2024-09-19T08:50:01.001 INFO:teuthology.orchestra.run.smithi196.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 1.3 MB/s | 1.2 MB 00:00 2024-09-19T08:50:01.149 INFO:teuthology.orchestra.run.smithi153.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 4.9 MB/s | 1.4 MB 00:00 2024-09-19T08:50:01.174 INFO:teuthology.orchestra.run.smithi153.stdout:(154/156): NetworkManager-tui-1.51.0-1.el9.x86_ 446 kB/s | 249 kB 00:00 2024-09-19T08:50:01.216 INFO:teuthology.orchestra.run.smithi153.stdout:(155/156): NetworkManager-libnm-1.51.0-1.el9.x8 1.5 MB/s | 1.8 MB 00:01 2024-09-19T08:50:01.232 INFO:teuthology.orchestra.run.smithi120.stdout:(150/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.3 MB/s | 16 MB 00:03 2024-09-19T08:50:01.423 INFO:teuthology.orchestra.run.smithi120.stdout:(151/156): NetworkManager-team-1.51.0-1.el9.x86 214 kB/s | 41 kB 00:00 2024-09-19T08:50:01.482 INFO:teuthology.orchestra.run.smithi120.stdout:(152/156): NetworkManager-1.51.0-1.el9.x86_64.r 3.4 MB/s | 2.3 MB 00:00 2024-09-19T08:50:01.501 INFO:teuthology.orchestra.run.smithi196.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 180 kB/s | 125 kB 00:00 2024-09-19T08:50:01.524 INFO:teuthology.orchestra.run.smithi153.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 6.4 MB/s | 2.4 MB 00:00 2024-09-19T08:50:01.531 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2024-09-19T08:50:01.531 INFO:teuthology.orchestra.run.smithi153.stdout:Total 18 MB/s | 226 MB 00:12 2024-09-19T08:50:01.535 INFO:teuthology.orchestra.run.smithi196.stdout:(47/156): python3-pycparser-2.20-6.el9.noarch.r 180 kB/s | 135 kB 00:00 2024-09-19T08:50:01.549 INFO:teuthology.orchestra.run.smithi120.stdout:(153/156): NetworkManager-libnm-1.51.0-1.el9.x8 2.7 MB/s | 1.8 MB 00:00 2024-09-19T08:50:01.608 INFO:teuthology.orchestra.run.smithi120.stdout:(154/156): NetworkManager-tui-1.51.0-1.el9.x86_ 1.3 MB/s | 249 kB 00:00 2024-09-19T08:50:01.694 INFO:teuthology.orchestra.run.smithi196.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 689 kB 00:00 2024-09-19T08:50:01.733 INFO:teuthology.orchestra.run.smithi120.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 5.4 MB/s | 1.4 MB 00:00 2024-09-19T08:50:01.736 INFO:teuthology.orchestra.run.smithi196.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 4.6 MB/s | 943 kB 00:00 2024-09-19T08:50:01.770 INFO:teuthology.orchestra.run.smithi196.stdout:(50/156): python3-urllib3-1.26.5-6.el9.noarch.r 280 kB/s | 215 kB 00:00 2024-09-19T08:50:01.795 INFO:teuthology.orchestra.run.smithi196.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 370 kB/s | 22 kB 00:00 2024-09-19T08:50:01.820 INFO:teuthology.orchestra.run.smithi196.stdout:(52/156): boost-program-options-1.75.0-8.el9.x8 849 kB/s | 107 kB 00:00 2024-09-19T08:50:01.845 INFO:teuthology.orchestra.run.smithi196.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-09-19T08:50:01.887 INFO:teuthology.orchestra.run.smithi196.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 438 kB/s | 18 kB 00:00 2024-09-19T08:50:01.929 INFO:teuthology.orchestra.run.smithi196.stdout:(55/156): flex-2.6.4-9.el9.x86_64.rpm 1.9 MB/s | 313 kB 00:00 2024-09-19T08:50:01.955 INFO:teuthology.orchestra.run.smithi196.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-09-19T08:50:01.983 INFO:teuthology.orchestra.run.smithi120.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 5.5 MB/s | 2.4 MB 00:00 2024-09-19T08:50:01.988 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-09-19T08:50:01.989 INFO:teuthology.orchestra.run.smithi120.stdout:Total 17 MB/s | 226 MB 00:13 2024-09-19T08:50:02.005 INFO:teuthology.orchestra.run.smithi196.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-09-19T08:50:02.030 INFO:teuthology.orchestra.run.smithi196.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 601 kB/s | 45 kB 00:00 2024-09-19T08:50:02.055 INFO:teuthology.orchestra.run.smithi196.stdout:(59/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-09-19T08:50:02.089 INFO:teuthology.orchestra.run.smithi196.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2024-09-19T08:50:02.115 INFO:teuthology.orchestra.run.smithi196.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 418 kB/s | 24 kB 00:00 2024-09-19T08:50:02.140 INFO:teuthology.orchestra.run.smithi196.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-19T08:50:02.165 INFO:teuthology.orchestra.run.smithi196.stdout:(63/156): libtool-2.4.6-46.el9.x86_64.rpm 4.2 MB/s | 581 kB 00:00 2024-09-19T08:50:02.190 INFO:teuthology.orchestra.run.smithi196.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-19T08:50:02.207 INFO:teuthology.orchestra.run.smithi196.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 923 kB/s | 38 kB 00:00 2024-09-19T08:50:02.258 INFO:teuthology.orchestra.run.smithi196.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-19T08:50:02.325 INFO:teuthology.orchestra.run.smithi196.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-09-19T08:50:02.392 INFO:teuthology.orchestra.run.smithi196.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-09-19T08:50:02.425 INFO:teuthology.orchestra.run.smithi196.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 8.9 MB/s | 910 kB 00:00 2024-09-19T08:50:02.459 INFO:teuthology.orchestra.run.smithi196.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-19T08:50:02.517 INFO:teuthology.orchestra.run.smithi196.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-09-19T08:50:02.584 INFO:teuthology.orchestra.run.smithi196.stdout:(72/156): openblas-openmp-0.3.26-2.el9.x86_64.r 13 MB/s | 4.9 MB 00:00 2024-09-19T08:50:02.610 INFO:teuthology.orchestra.run.smithi196.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 952 kB/s | 87 kB 00:00 2024-09-19T08:50:02.685 INFO:teuthology.orchestra.run.smithi196.stdout:(74/156): postgresql-server-13.16-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-09-19T08:50:02.710 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2024-09-19T08:50:02.710 INFO:teuthology.orchestra.run.smithi196.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 611 kB/s | 61 kB 00:00 2024-09-19T08:50:02.735 INFO:teuthology.orchestra.run.smithi196.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-19T08:50:02.769 INFO:teuthology.orchestra.run.smithi196.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-09-19T08:50:02.803 INFO:teuthology.orchestra.run.smithi196.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 383 kB/s | 26 kB 00:00 2024-09-19T08:50:02.828 INFO:teuthology.orchestra.run.smithi196.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 327 kB/s | 19 kB 00:00 2024-09-19T08:50:02.861 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2024-09-19T08:50:02.861 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2024-09-19T08:50:02.862 INFO:teuthology.orchestra.run.smithi196.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-19T08:50:02.887 INFO:teuthology.orchestra.run.smithi196.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 592 kB/s | 35 kB 00:00 2024-09-19T08:50:02.929 INFO:teuthology.orchestra.run.smithi196.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-09-19T08:50:03.004 INFO:teuthology.orchestra.run.smithi196.stdout:(83/156): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-09-19T08:50:03.038 INFO:teuthology.orchestra.run.smithi196.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-09-19T08:50:03.063 INFO:teuthology.orchestra.run.smithi196.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-19T08:50:03.088 INFO:teuthology.orchestra.run.smithi196.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 846 kB/s | 42 kB 00:00 2024-09-19T08:50:03.114 INFO:teuthology.orchestra.run.smithi196.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-19T08:50:03.147 INFO:teuthology.orchestra.run.smithi196.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-09-19T08:50:03.184 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-09-19T08:50:03.214 INFO:teuthology.orchestra.run.smithi196.stdout:(89/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-09-19T08:50:03.240 INFO:teuthology.orchestra.run.smithi196.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 554 kB/s | 51 kB 00:00 2024-09-19T08:50:03.265 INFO:teuthology.orchestra.run.smithi196.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.8 MB/s | 279 kB 00:00 2024-09-19T08:50:03.290 INFO:teuthology.orchestra.run.smithi196.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 711 kB/s | 54 kB 00:00 2024-09-19T08:50:03.332 INFO:teuthology.orchestra.run.smithi196.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-19T08:50:03.335 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-09-19T08:50:03.335 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-09-19T08:50:03.357 INFO:teuthology.orchestra.run.smithi196.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 665 kB/s | 44 kB 00:00 2024-09-19T08:50:03.399 INFO:teuthology.orchestra.run.smithi196.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 869 kB/s | 36 kB 00:00 2024-09-19T08:50:03.424 INFO:teuthology.orchestra.run.smithi196.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-09-19T08:50:03.450 INFO:teuthology.orchestra.run.smithi196.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-19T08:50:03.475 INFO:teuthology.orchestra.run.smithi196.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-19T08:50:03.500 INFO:teuthology.orchestra.run.smithi196.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-19T08:50:03.525 INFO:teuthology.orchestra.run.smithi196.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-19T08:50:03.551 INFO:teuthology.orchestra.run.smithi196.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-19T08:50:03.576 INFO:teuthology.orchestra.run.smithi196.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-09-19T08:50:03.768 INFO:teuthology.orchestra.run.smithi196.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-09-19T08:50:03.793 INFO:teuthology.orchestra.run.smithi196.stdout:(104/156): lua-devel-5.4.4-4.el9.x86_64.rpm 92 kB/s | 22 kB 00:00 2024-09-19T08:50:03.852 INFO:teuthology.orchestra.run.smithi196.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 6.4 MB/s | 548 kB 00:00 2024-09-19T08:50:03.877 INFO:teuthology.orchestra.run.smithi196.stdout:(106/156): gperftools-libs-2.9.1-3.el9.x86_64.r 3.6 MB/s | 308 kB 00:00 2024-09-19T08:50:03.894 INFO:teuthology.orchestra.run.smithi196.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 466 kB/s | 19 kB 00:00 2024-09-19T08:50:03.911 INFO:teuthology.orchestra.run.smithi196.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.5 MB/s | 26 kB 00:00 2024-09-19T08:50:03.945 INFO:teuthology.orchestra.run.smithi196.stdout:(109/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.3 MB/s | 863 kB 00:00 2024-09-19T08:50:03.962 INFO:teuthology.orchestra.run.smithi196.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 962 kB/s | 49 kB 00:00 2024-09-19T08:50:04.004 INFO:teuthology.orchestra.run.smithi196.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.3 MB/s | 151 kB 00:00 2024-09-19T08:50:04.037 INFO:teuthology.orchestra.run.smithi196.stdout:(112/156): libunwind-1.6.2-1.el9.x86_64.rpm 733 kB/s | 67 kB 00:00 2024-09-19T08:50:04.096 INFO:teuthology.orchestra.run.smithi196.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.1 MB/s | 839 kB 00:00 2024-09-19T08:50:04.130 INFO:teuthology.orchestra.run.smithi196.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-09-19T08:50:04.155 INFO:teuthology.orchestra.run.smithi196.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-19T08:50:04.189 INFO:teuthology.orchestra.run.smithi196.stdout:(116/156): python3-asyncssh-2.13.2-5.el9.noarch 3.5 MB/s | 548 kB 00:00 2024-09-19T08:50:04.214 INFO:teuthology.orchestra.run.smithi196.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 241 kB/s | 14 kB 00:00 2024-09-19T08:50:04.239 INFO:teuthology.orchestra.run.smithi196.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-09-19T08:50:04.265 INFO:teuthology.orchestra.run.smithi196.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-19T08:50:04.289 INFO:teuthology.orchestra.run.smithi196.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-19T08:50:04.315 INFO:teuthology.orchestra.run.smithi196.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-19T08:50:04.382 INFO:teuthology.orchestra.run.smithi196.stdout:(122/156): libarrow-9.0.0-11.el9.x86_64.rpm 8.7 MB/s | 4.4 MB 00:00 2024-09-19T08:50:04.407 INFO:teuthology.orchestra.run.smithi196.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-09-19T08:50:04.433 INFO:teuthology.orchestra.run.smithi196.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-09-19T08:50:04.458 INFO:teuthology.orchestra.run.smithi196.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-19T08:50:04.483 INFO:teuthology.orchestra.run.smithi196.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-09-19T08:50:04.508 INFO:teuthology.orchestra.run.smithi196.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-19T08:50:04.534 INFO:teuthology.orchestra.run.smithi196.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-19T08:50:04.559 INFO:teuthology.orchestra.run.smithi196.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T08:50:04.593 INFO:teuthology.orchestra.run.smithi196.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-19T08:50:04.626 INFO:teuthology.orchestra.run.smithi196.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-09-19T08:50:04.652 INFO:teuthology.orchestra.run.smithi196.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 985 kB/s | 58 kB 00:00 2024-09-19T08:50:04.677 INFO:teuthology.orchestra.run.smithi196.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 663 kB/s | 16 kB 00:00 2024-09-19T08:50:04.702 INFO:teuthology.orchestra.run.smithi196.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-19T08:50:04.727 INFO:teuthology.orchestra.run.smithi196.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-19T08:50:04.753 INFO:teuthology.orchestra.run.smithi196.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-19T08:50:04.767 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2024-09-19T08:50:04.768 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2024-09-19T08:50:04.778 INFO:teuthology.orchestra.run.smithi196.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-19T08:50:04.829 INFO:teuthology.orchestra.run.smithi196.stdout:(138/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.8 MB/s | 2.0 MB 00:00 2024-09-19T08:50:04.854 INFO:teuthology.orchestra.run.smithi196.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-19T08:50:04.879 INFO:teuthology.orchestra.run.smithi196.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 586 kB/s | 59 kB 00:00 2024-09-19T08:50:04.905 INFO:teuthology.orchestra.run.smithi196.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-09-19T08:50:04.930 INFO:teuthology.orchestra.run.smithi196.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 985 kB/s | 74 kB 00:00 2024-09-19T08:50:04.955 INFO:teuthology.orchestra.run.smithi196.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-19T08:50:04.980 INFO:teuthology.orchestra.run.smithi196.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-19T08:50:04.997 INFO:teuthology.orchestra.run.smithi196.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 478 kB/s | 20 kB 00:00 2024-09-19T08:50:05.031 INFO:teuthology.orchestra.run.smithi196.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T08:50:05.057 INFO:teuthology.orchestra.run.smithi196.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-19T08:50:05.107 INFO:teuthology.orchestra.run.smithi196.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-19T08:50:05.482 INFO:teuthology.orchestra.run.smithi196.stdout:(149/156): librbd1-19.1.1-361.g2f599a46.el9.x86 7.4 MB/s | 3.2 MB 00:00 2024-09-19T08:50:05.540 INFO:teuthology.orchestra.run.smithi196.stdout:(150/156): librados2-19.1.1-361.g2f599a46.el9.x 6.7 MB/s | 3.4 MB 00:00 2024-09-19T08:50:05.739 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-09-19T08:50:05.740 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-09-19T08:50:06.124 INFO:teuthology.orchestra.run.smithi196.stdout:(151/156): NetworkManager-1.51.0-1.el9.x86_64.r 2.3 MB/s | 2.3 MB 00:01 2024-09-19T08:50:06.149 INFO:teuthology.orchestra.run.smithi196.stdout:(152/156): NetworkManager-team-1.51.0-1.el9.x86 67 kB/s | 41 kB 00:00 2024-09-19T08:50:06.524 INFO:teuthology.orchestra.run.smithi196.stdout:(153/156): NetworkManager-libnm-1.51.0-1.el9.x8 1.7 MB/s | 1.8 MB 00:01 2024-09-19T08:50:06.925 INFO:teuthology.orchestra.run.smithi196.stdout:(154/156): NetworkManager-tui-1.51.0-1.el9.x86_ 311 kB/s | 249 kB 00:00 2024-09-19T08:50:07.116 INFO:teuthology.orchestra.run.smithi196.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 1.4 MB/s | 1.4 MB 00:00 2024-09-19T08:50:07.533 INFO:teuthology.orchestra.run.smithi196.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.4 MB/s | 2.4 MB 00:01 2024-09-19T08:50:07.539 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-09-19T08:50:07.540 INFO:teuthology.orchestra.run.smithi196.stdout:Total 12 MB/s | 226 MB 00:18 2024-09-19T08:50:08.781 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-09-19T08:50:08.935 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-09-19T08:50:08.935 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-09-19T08:50:10.748 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-09-19T08:50:10.748 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-09-19T08:50:11.273 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2024-09-19T08:50:11.379 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T08:50:11.425 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T08:50:11.478 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T08:50:11.513 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T08:50:11.538 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T08:50:11.581 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T08:50:11.955 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T08:50:12.012 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T08:50:12.118 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T08:50:12.134 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-09-19T08:50:12.148 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T08:50:12.231 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T08:50:12.261 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T08:50:12.279 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T08:50:12.360 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T08:50:12.409 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T08:50:12.466 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T08:50:12.494 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T08:50:12.497 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T08:50:12.530 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T08:50:12.556 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T08:50:12.597 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T08:50:12.620 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T08:50:12.621 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T08:50:12.665 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T08:50:12.688 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T08:50:12.712 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T08:50:12.905 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T08:50:12.956 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T08:50:13.031 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T08:50:13.082 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T08:50:13.169 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T08:50:13.204 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T08:50:13.242 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T08:50:13.270 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T08:50:13.328 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T08:50:13.334 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T08:50:13.371 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T08:50:13.536 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T08:50:13.548 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T08:50:13.603 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T08:50:13.638 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T08:50:13.640 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T08:50:13.729 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T08:50:13.750 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T08:50:13.766 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T08:50:13.772 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T08:50:13.801 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T08:50:13.835 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T08:50:13.857 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T08:50:13.866 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T08:50:13.877 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T08:50:13.910 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T08:50:13.942 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T08:50:13.986 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T08:50:14.018 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T08:50:14.046 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T08:50:14.076 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T08:50:14.115 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T08:50:14.164 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T08:50:14.202 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T08:50:14.237 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T08:50:14.252 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T08:50:14.314 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T08:50:14.348 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T08:50:14.391 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T08:50:14.406 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T08:50:14.420 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T08:50:14.491 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T08:50:14.513 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T08:50:14.534 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T08:50:14.599 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T08:50:14.610 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T08:50:14.750 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T08:50:14.804 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T08:50:14.815 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T08:50:14.922 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T08:50:15.016 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T08:50:15.064 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T08:50:15.106 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T08:50:15.179 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T08:50:15.223 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T08:50:15.256 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T08:50:15.300 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T08:50:15.329 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T08:50:15.332 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T08:50:15.360 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T08:50:15.416 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T08:50:15.428 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T08:50:15.445 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T08:50:15.487 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T08:50:15.490 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T08:50:15.525 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T08:50:15.530 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T08:50:15.565 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T08:50:15.567 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T08:50:15.641 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T08:50:15.678 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T08:50:15.720 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T08:50:15.728 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T08:50:15.748 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T08:50:15.787 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T08:50:15.819 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T08:50:15.842 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T08:50:15.863 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T08:50:15.915 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T08:50:15.945 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T08:50:15.946 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T08:50:15.982 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T08:50:16.028 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T08:50:16.064 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T08:50:16.088 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T08:50:16.107 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T08:50:16.141 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T08:50:16.157 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T08:50:16.295 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T08:50:16.342 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T08:50:16.384 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T08:50:16.488 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T08:50:16.526 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T08:50:16.565 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T08:50:16.591 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T08:50:16.616 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T08:50:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T08:50:16.689 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T08:50:16.736 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T08:50:16.766 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T08:50:16.801 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T08:50:16.838 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T08:50:16.890 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T08:50:17.042 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T08:50:17.046 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T08:50:17.092 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T08:50:17.108 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T08:50:17.119 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T08:50:17.154 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T08:50:17.163 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T08:50:17.193 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T08:50:17.224 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T08:50:17.236 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T08:50:17.254 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T08:50:17.269 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T08:50:17.289 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T08:50:17.304 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T08:50:17.322 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T08:50:17.351 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T08:50:17.364 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T08:50:17.382 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-09-19T08:50:17.387 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T08:50:17.398 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T08:50:17.430 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T08:50:17.481 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T08:50:17.485 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T08:50:17.528 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T08:50:17.537 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T08:50:17.568 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T08:50:17.616 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T08:50:17.631 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T08:50:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T08:50:17.672 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T08:50:17.712 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T08:50:17.728 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T08:50:17.840 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T08:50:17.880 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T08:50:17.905 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T08:50:17.958 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T08:50:18.079 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T08:50:18.124 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T08:50:18.172 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T08:50:18.203 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T08:50:18.228 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T08:50:18.288 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T08:50:18.328 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T08:50:18.397 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T08:50:18.453 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T08:50:18.463 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T08:50:18.494 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T08:50:18.499 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T08:50:18.522 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T08:50:18.533 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T08:50:18.541 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T08:50:18.549 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T08:50:18.576 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T08:50:18.621 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T08:50:18.632 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T08:50:18.641 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T08:50:18.678 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T08:50:18.679 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T08:50:18.679 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T08:50:18.716 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T08:50:18.724 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T08:50:18.750 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T08:50:18.784 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T08:50:18.813 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T08:50:18.825 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T08:50:18.829 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T08:50:18.852 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T08:50:18.860 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T08:50:18.884 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T08:50:18.912 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T08:50:18.946 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T08:50:18.990 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T08:50:19.000 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T08:50:19.029 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T08:50:19.030 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T08:50:19.035 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T08:50:19.066 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T08:50:19.075 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T08:50:19.105 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T08:50:19.128 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T08:50:19.134 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T08:50:19.143 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T08:50:19.181 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T08:50:19.315 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T08:50:19.472 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T08:50:19.604 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T08:50:19.690 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T08:50:19.789 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T08:50:19.790 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T08:50:19.902 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T08:50:19.931 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T08:50:19.960 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T08:50:19.993 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T08:50:19.998 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T08:50:20.094 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T08:50:20.110 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T08:50:20.151 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T08:50:20.196 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T08:50:20.217 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T08:50:20.255 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T08:50:20.286 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T08:50:20.290 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T08:50:20.331 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T08:50:20.355 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T08:50:20.399 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T08:50:20.431 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T08:50:20.475 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T08:50:20.516 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T08:50:20.516 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T08:50:20.544 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T08:50:20.554 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T08:50:20.612 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T08:50:20.632 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T08:50:20.666 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T08:50:20.697 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T08:50:20.741 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T08:50:20.816 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T08:50:20.852 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T08:50:20.895 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T08:50:20.924 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T08:50:20.993 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T08:50:21.046 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T08:50:21.120 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T08:50:21.160 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T08:50:21.245 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T08:50:21.253 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T08:50:21.272 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T08:50:21.286 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T08:50:21.305 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T08:50:21.307 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T08:50:21.402 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T08:50:21.624 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T08:50:21.669 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T08:50:21.759 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T08:50:21.790 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T08:50:21.827 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T08:50:21.829 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T08:50:21.912 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T08:50:21.942 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T08:50:21.984 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T08:50:22.024 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T08:50:22.065 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T08:50:22.233 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T08:50:22.302 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T08:50:22.365 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T08:50:22.440 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T08:50:22.480 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T08:50:22.490 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T08:50:22.522 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T08:50:22.525 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T08:50:22.571 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T08:50:22.576 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T08:50:22.598 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T08:50:22.616 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T08:50:22.650 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T08:50:22.658 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T08:50:22.701 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T08:50:22.735 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T08:50:22.772 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T08:50:22.805 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T08:50:22.837 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T08:50:22.886 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T08:50:22.936 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T08:50:23.037 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T08:50:23.078 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T08:50:23.113 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T08:50:23.127 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T08:50:23.152 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T08:50:23.160 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T08:50:23.185 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T08:50:23.242 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T08:50:23.259 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T08:50:23.273 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T08:50:23.314 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T08:50:23.386 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T08:50:23.448 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T08:50:23.496 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T08:50:23.530 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T08:50:23.569 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T08:50:23.600 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T08:50:23.630 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T08:50:23.673 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T08:50:23.707 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T08:50:23.748 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T08:50:23.783 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T08:50:23.814 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T08:50:23.838 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T08:50:23.869 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T08:50:23.909 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T08:50:23.913 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T08:50:23.931 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T08:50:23.944 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T08:50:24.047 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T08:50:24.105 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T08:50:24.824 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T08:50:24.876 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T08:50:24.903 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T08:50:24.910 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T08:50:24.944 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T08:50:25.008 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T08:50:25.065 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T08:50:25.110 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T08:50:25.206 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T08:50:25.246 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T08:50:25.431 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T08:50:25.461 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T08:50:25.489 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T08:50:25.523 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T08:50:25.538 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T08:50:25.597 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T08:50:25.597 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T08:50:25.597 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T08:50:25.597 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-19T08:50:25.624 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T08:50:26.177 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T08:50:26.271 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T08:50:26.301 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T08:50:26.312 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T08:50:26.871 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T08:50:26.924 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T08:50:27.014 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T08:50:27.014 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T08:50:27.014 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T08:50:27.014 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:50:27.051 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T08:50:27.535 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T08:50:27.623 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T08:50:27.651 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T08:50:27.684 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T08:50:27.997 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T08:50:28.039 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T08:50:28.130 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T08:50:28.160 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T08:50:28.197 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T08:50:28.688 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T08:50:28.726 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T08:50:28.789 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T08:50:28.819 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T08:50:31.089 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T08:50:31.132 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T08:50:31.680 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T08:50:31.719 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T08:50:31.816 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T08:50:31.816 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T08:50:31.816 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T08:50:31.816 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T08:50:31.853 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T08:50:37.584 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T08:50:37.585 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /sys 2024-09-19T08:50:37.585 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /proc 2024-09-19T08:50:37.585 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /mnt 2024-09-19T08:50:37.585 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /var/tmp 2024-09-19T08:50:37.585 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /home 2024-09-19T08:50:37.585 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /root 2024-09-19T08:50:37.585 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /tmp 2024-09-19T08:50:37.585 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-19T08:50:37.759 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T08:50:37.774 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T08:50:37.775 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:50:37.775 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T08:50:37.775 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-19T08:50:38.970 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T08:50:38.970 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /sys 2024-09-19T08:50:38.970 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /proc 2024-09-19T08:50:38.970 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /mnt 2024-09-19T08:50:38.971 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /var/tmp 2024-09-19T08:50:38.971 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /home 2024-09-19T08:50:38.971 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /root 2024-09-19T08:50:38.971 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /tmp 2024-09-19T08:50:38.971 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:50:39.140 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T08:50:39.155 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T08:50:39.155 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:50:39.155 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T08:50:39.155 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:50:39.221 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T08:50:39.247 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T08:50:39.248 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:50:39.248 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T08:50:39.248 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T08:50:39.248 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T08:50:39.248 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-19T08:50:39.471 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T08:50:39.494 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T08:50:39.495 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:50:39.495 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T08:50:39.495 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T08:50:39.495 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T08:50:39.495 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-19T08:50:39.922 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T08:50:39.945 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T08:50:39.945 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:50:39.945 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T08:50:39.945 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T08:50:39.945 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T08:50:39.946 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-19T08:50:40.052 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T08:50:40.108 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T08:50:40.164 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T08:50:40.347 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T08:50:40.397 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T08:50:40.475 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T08:50:40.564 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T08:50:40.635 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T08:50:40.662 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T08:50:40.662 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:50:40.662 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T08:50:40.662 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T08:50:40.662 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T08:50:40.662 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:50:40.702 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T08:50:40.747 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T08:50:40.779 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T08:50:40.807 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T08:50:40.895 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T08:50:40.918 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T08:50:40.919 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:50:40.919 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T08:50:40.919 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T08:50:40.919 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T08:50:40.919 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:50:41.244 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T08:50:41.272 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T08:50:41.295 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T08:50:41.349 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T08:50:41.363 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T08:50:41.387 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T08:50:41.387 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:50:41.387 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T08:50:41.387 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T08:50:41.387 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T08:50:41.387 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:50:41.388 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T08:50:41.502 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T08:50:41.567 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T08:50:41.648 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T08:50:41.834 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T08:50:41.882 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T08:50:41.959 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T08:50:42.048 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T08:50:42.178 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T08:50:42.229 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T08:50:42.264 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T08:50:42.292 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T08:50:42.433 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T08:50:42.463 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T08:50:42.731 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T08:50:42.758 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T08:50:42.789 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T08:50:42.853 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T08:50:42.880 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T08:50:43.422 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T08:50:43.447 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T08:50:43.510 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T08:50:43.510 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /sys 2024-09-19T08:50:43.510 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /proc 2024-09-19T08:50:43.510 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /mnt 2024-09-19T08:50:43.510 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /var/tmp 2024-09-19T08:50:43.510 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /home 2024-09-19T08:50:43.510 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /root 2024-09-19T08:50:43.510 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /tmp 2024-09-19T08:50:43.510 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T08:50:43.548 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T08:50:43.690 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T08:50:43.708 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T08:50:43.708 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:50:43.708 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T08:50:43.708 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T08:50:43.709 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T08:50:43.741 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T08:50:43.764 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T08:50:43.764 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:50:43.765 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T08:50:43.765 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T08:50:43.765 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T08:50:43.765 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-19T08:50:43.808 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T08:50:43.820 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T08:50:43.909 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T08:50:43.929 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T08:50:43.956 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T08:50:43.975 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T08:50:44.877 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T08:50:44.919 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T08:50:44.942 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T08:50:44.942 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T08:50:44.942 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:50:44.942 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T08:50:44.942 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T08:50:44.943 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T08:50:44.943 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-19T08:50:44.967 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T08:50:44.986 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T08:50:45.009 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T08:50:45.009 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:50:45.010 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T08:50:45.010 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-19T08:50:45.054 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T08:50:45.070 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T08:50:45.076 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T08:50:45.077 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:50:45.077 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T08:50:45.077 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T08:50:45.077 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T08:50:45.077 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-19T08:50:45.202 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T08:50:45.229 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T08:50:45.229 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:50:45.229 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T08:50:45.229 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T08:50:45.229 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T08:50:45.229 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T08:50:45.230 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T08:50:45.269 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T08:50:45.292 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T08:50:45.293 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:50:45.293 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T08:50:45.293 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T08:50:45.293 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T08:50:45.293 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:50:45.337 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T08:50:45.348 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T08:50:45.379 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T08:50:45.402 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T08:50:45.402 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:50:45.403 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T08:50:45.403 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T08:50:45.403 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T08:50:45.403 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-19T08:50:45.450 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T08:50:45.452 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T08:50:45.476 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T08:50:45.476 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:50:45.476 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T08:50:45.476 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T08:50:45.476 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T08:50:45.477 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T08:50:45.508 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T08:50:45.920 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T08:50:45.945 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T08:50:45.945 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:50:45.946 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T08:50:45.946 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T08:50:45.946 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T08:50:45.946 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T08:50:46.050 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T08:50:46.107 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T08:50:46.161 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T08:50:46.347 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T08:50:46.396 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T08:50:46.438 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T08:50:46.464 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T08:50:46.482 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T08:50:46.488 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T08:50:46.488 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:50:46.488 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T08:50:46.488 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T08:50:46.488 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T08:50:46.488 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:50:46.540 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T08:50:46.562 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T08:50:46.563 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:50:46.563 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T08:50:46.563 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:50:46.578 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T08:50:46.616 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T08:50:46.639 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T08:50:46.639 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:50:46.639 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T08:50:46.639 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T08:50:46.639 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T08:50:46.639 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:50:46.719 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T08:50:46.765 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T08:50:46.795 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T08:50:46.822 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T08:50:46.942 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T08:50:46.964 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T08:50:46.964 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:50:46.965 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T08:50:46.965 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T08:50:46.965 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T08:50:46.965 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:50:47.276 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T08:50:47.304 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T08:50:47.336 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T08:50:47.398 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T08:50:47.427 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T08:50:48.475 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T08:50:48.504 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T08:50:49.468 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T08:50:49.505 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T08:50:49.607 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T08:50:49.768 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T08:50:49.799 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T08:50:49.823 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T08:50:49.824 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:50:49.824 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T08:50:49.824 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T08:50:49.824 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T08:50:49.824 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T08:50:49.883 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T08:50:49.895 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T08:50:49.966 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T08:50:49.992 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T08:50:50.007 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T08:50:50.047 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T08:50:50.048 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T08:50:50.080 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T08:50:50.107 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T08:50:50.292 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T08:50:50.337 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T08:50:50.368 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T08:50:50.397 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T08:50:50.435 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T08:50:50.972 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T08:50:51.010 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T08:50:51.034 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T08:50:51.034 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:50:51.034 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T08:50:51.034 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T08:50:51.035 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T08:50:51.035 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T08:50:51.041 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T08:50:51.074 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T08:50:51.078 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T08:50:51.094 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T08:50:51.101 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T08:50:51.101 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:50:51.101 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T08:50:51.101 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T08:50:51.154 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T08:50:51.177 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T08:50:51.177 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:50:51.177 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T08:50:51.177 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T08:50:51.177 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T08:50:51.177 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T08:50:51.496 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T08:50:51.519 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T08:50:51.519 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:50:51.519 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T08:50:51.520 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T08:50:51.520 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T08:50:51.520 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T08:50:51.554 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T08:50:51.595 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T08:50:51.636 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T08:50:51.669 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T08:50:51.696 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T08:50:51.890 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T08:50:51.923 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T08:50:51.935 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T08:50:51.974 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T08:50:52.009 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T08:50:52.012 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T08:50:52.053 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T08:50:52.058 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T08:50:52.135 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T08:50:52.165 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T08:50:52.190 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T08:50:52.225 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T08:50:52.225 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T08:50:52.309 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T08:50:52.309 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T08:50:52.409 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T08:50:52.434 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T08:50:52.439 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T08:50:52.439 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T08:50:52.611 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T08:50:52.658 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T08:50:52.676 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T08:50:52.695 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T08:50:52.715 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T08:50:52.735 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T08:50:52.767 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T08:50:53.560 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T08:50:53.638 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T08:50:53.696 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T08:50:53.796 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T08:50:53.833 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T08:50:53.858 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T08:50:53.884 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T08:50:53.884 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T08:50:53.968 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T08:50:53.969 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T08:50:54.053 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T08:50:54.078 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T08:50:54.083 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T08:50:54.083 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T08:50:54.263 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T08:50:54.263 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T08:50:54.339 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T08:50:54.403 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T08:50:56.049 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T08:50:56.098 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T08:50:56.148 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T08:50:56.180 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T08:50:56.207 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T08:50:56.386 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T08:50:56.429 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T08:50:56.459 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T08:50:56.489 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T08:50:56.535 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T08:50:57.158 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T08:50:57.242 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T08:50:57.266 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T08:50:58.095 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T08:50:58.188 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T08:50:58.229 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T08:50:58.319 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T08:50:58.350 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T08:50:58.375 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T08:50:58.396 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T08:50:58.396 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T08:50:58.396 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T08:50:58.396 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T08:50:58.396 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T08:50:58.396 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T08:50:58.397 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T08:50:58.397 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T08:50:58.397 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T08:50:58.397 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T08:50:58.397 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T08:50:58.397 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T08:50:58.397 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T08:50:58.397 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T08:50:58.397 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T08:50:58.397 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T08:50:58.397 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T08:50:58.397 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T08:50:58.398 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T08:50:58.398 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T08:50:58.398 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T08:50:58.398 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T08:50:58.398 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T08:50:58.398 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T08:50:58.398 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T08:50:58.398 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T08:50:58.398 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T08:50:58.398 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T08:50:58.406 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T08:50:58.406 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T08:50:58.411 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T08:50:58.411 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T08:50:58.411 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T08:50:58.411 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T08:50:58.412 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T08:50:58.412 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T08:50:58.412 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T08:50:58.412 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T08:50:58.412 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T08:50:58.412 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T08:50:58.412 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T08:50:58.412 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T08:50:58.412 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T08:50:58.412 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T08:50:58.412 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T08:50:58.412 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T08:50:58.412 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T08:50:58.413 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T08:50:58.413 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T08:50:58.413 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T08:50:58.413 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T08:50:58.413 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T08:50:58.413 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T08:50:58.413 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T08:50:58.413 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T08:50:58.413 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T08:50:58.413 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T08:50:58.413 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T08:50:58.413 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T08:50:58.413 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T08:50:58.414 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T08:50:58.414 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T08:50:58.414 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T08:50:58.414 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T08:50:58.414 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T08:50:58.414 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T08:50:58.414 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T08:50:58.414 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T08:50:58.414 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T08:50:58.414 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T08:50:58.414 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T08:50:58.414 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T08:50:58.414 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T08:50:58.415 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T08:50:58.415 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T08:50:58.415 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T08:50:58.415 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T08:50:58.415 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T08:50:58.415 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T08:50:58.415 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T08:50:58.415 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T08:50:58.415 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T08:50:58.415 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T08:50:58.415 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T08:50:58.415 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T08:50:58.415 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T08:50:58.416 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T08:50:58.416 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T08:50:58.416 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T08:50:58.416 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T08:50:58.416 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T08:50:58.416 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T08:50:58.416 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T08:50:58.416 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T08:50:58.416 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T08:50:58.416 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T08:50:58.416 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T08:50:58.416 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T08:50:58.416 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T08:50:58.416 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T08:50:58.417 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T08:50:58.417 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T08:50:58.417 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T08:50:58.417 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T08:50:58.417 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T08:50:58.417 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T08:50:58.417 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T08:50:58.417 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T08:50:58.417 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T08:50:58.417 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T08:50:58.417 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T08:50:58.417 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T08:50:58.417 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T08:50:58.417 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T08:50:58.418 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T08:50:58.418 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T08:50:58.418 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T08:50:58.418 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T08:50:58.418 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T08:50:58.418 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T08:50:58.418 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T08:50:58.418 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T08:50:58.418 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T08:50:58.418 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T08:50:58.418 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T08:50:58.418 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T08:50:58.418 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T08:50:58.419 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T08:50:58.419 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T08:50:58.419 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T08:50:58.419 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T08:50:58.419 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T08:50:58.419 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T08:50:58.420 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T08:50:58.420 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T08:50:58.420 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T08:50:58.420 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T08:50:58.420 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T08:50:58.420 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T08:50:58.420 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T08:50:58.420 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T08:50:58.420 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T08:50:58.420 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T08:50:58.420 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T08:50:58.420 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T08:50:58.420 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T08:50:58.421 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T08:50:58.421 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T08:50:58.421 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T08:50:58.421 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T08:50:58.421 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T08:50:58.421 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T08:50:58.421 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T08:50:58.421 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T08:50:58.421 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T08:50:58.421 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T08:50:58.421 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T08:50:58.421 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T08:50:58.421 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T08:50:58.421 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T08:50:58.422 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T08:50:58.422 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T08:50:58.422 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T08:50:58.422 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T08:50:58.422 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T08:50:58.422 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T08:50:58.494 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T08:50:58.494 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T08:50:58.578 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T08:50:58.603 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T08:50:58.609 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T08:50:58.609 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T08:50:58.796 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T08:50:58.796 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T08:50:58.881 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T08:50:58.944 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T08:51:00.251 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T08:51:00.252 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T08:51:00.252 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T08:51:00.252 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T08:51:00.252 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T08:51:00.252 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T08:51:00.252 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T08:51:00.252 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T08:51:00.252 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T08:51:00.252 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T08:51:00.252 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T08:51:00.252 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T08:51:00.253 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T08:51:00.253 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T08:51:00.253 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T08:51:00.253 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T08:51:00.253 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T08:51:00.253 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T08:51:00.253 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T08:51:00.253 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T08:51:00.253 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T08:51:00.253 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T08:51:00.253 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T08:51:00.253 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T08:51:00.253 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T08:51:00.254 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T08:51:00.254 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T08:51:00.254 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T08:51:00.267 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T08:51:00.267 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T08:51:00.267 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T08:51:00.267 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T08:51:00.267 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T08:51:00.267 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T08:51:00.267 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T08:51:00.267 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T08:51:00.268 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T08:51:00.268 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T08:51:00.268 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T08:51:00.268 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T08:51:00.268 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T08:51:00.268 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T08:51:00.268 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T08:51:00.268 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T08:51:00.268 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T08:51:00.268 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T08:51:00.268 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T08:51:00.268 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T08:51:00.269 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T08:51:00.269 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T08:51:00.269 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T08:51:00.269 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T08:51:00.269 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T08:51:00.269 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T08:51:00.269 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T08:51:00.269 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T08:51:00.269 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T08:51:00.269 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T08:51:00.269 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T08:51:00.269 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T08:51:00.269 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T08:51:00.269 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T08:51:00.270 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T08:51:00.270 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T08:51:00.270 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T08:51:00.270 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T08:51:00.270 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T08:51:00.270 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T08:51:00.270 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T08:51:00.270 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T08:51:00.270 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T08:51:00.270 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T08:51:00.271 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T08:51:00.271 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T08:51:00.271 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T08:51:00.271 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T08:51:00.271 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T08:51:00.271 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T08:51:00.271 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T08:51:00.271 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T08:51:00.271 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T08:51:00.272 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T08:51:00.272 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T08:51:00.272 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T08:51:00.272 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T08:51:00.272 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T08:51:00.272 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T08:51:00.272 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T08:51:00.272 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T08:51:00.272 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T08:51:00.272 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T08:51:00.272 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T08:51:00.272 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T08:51:00.272 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T08:51:00.272 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T08:51:00.273 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T08:51:00.273 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T08:51:00.273 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T08:51:00.273 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T08:51:00.273 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T08:51:00.273 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T08:51:00.273 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T08:51:00.273 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T08:51:00.273 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T08:51:00.273 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T08:51:00.273 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T08:51:00.273 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T08:51:00.273 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T08:51:00.273 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T08:51:00.274 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T08:51:00.274 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T08:51:00.274 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T08:51:00.274 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T08:51:00.274 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T08:51:00.274 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T08:51:00.274 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T08:51:00.274 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T08:51:00.274 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T08:51:00.274 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T08:51:00.274 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T08:51:00.274 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T08:51:00.274 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T08:51:00.274 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T08:51:00.274 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T08:51:00.275 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T08:51:00.275 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T08:51:00.275 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T08:51:00.275 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T08:51:00.275 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T08:51:00.275 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T08:51:00.275 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T08:51:00.275 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T08:51:00.275 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T08:51:00.275 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T08:51:00.275 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T08:51:00.275 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T08:51:00.275 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T08:51:00.275 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T08:51:00.276 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T08:51:00.276 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T08:51:00.276 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T08:51:00.276 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T08:51:00.276 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T08:51:00.276 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T08:51:00.276 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T08:51:00.276 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T08:51:00.276 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T08:51:00.276 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T08:51:00.276 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T08:51:00.276 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T08:51:00.276 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T08:51:00.276 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T08:51:00.277 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T08:51:00.277 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T08:51:00.277 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T08:51:00.277 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T08:51:00.277 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T08:51:00.277 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T08:51:00.277 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T08:51:00.277 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T08:51:00.277 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T08:51:00.277 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T08:51:00.277 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T08:51:00.277 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T08:51:01.038 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T08:51:01.038 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-19T08:51:01.038 INFO:teuthology.orchestra.run.smithi153.stdout:Upgraded: 2024-09-19T08:51:01.038 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T08:51:01.038 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T08:51:01.038 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T08:51:01.038 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T08:51:01.039 INFO:teuthology.orchestra.run.smithi153.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.039 INFO:teuthology.orchestra.run.smithi153.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.039 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T08:51:01.039 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T08:51:01.039 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2024-09-19T08:51:01.039 INFO:teuthology.orchestra.run.smithi153.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T08:51:01.039 INFO:teuthology.orchestra.run.smithi153.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T08:51:01.039 INFO:teuthology.orchestra.run.smithi153.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T08:51:01.039 INFO:teuthology.orchestra.run.smithi153.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T08:51:01.039 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.039 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.039 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.040 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.040 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:01.040 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.040 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.040 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.040 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:01.040 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:01.040 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:01.040 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:01.040 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:01.040 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.040 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.040 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:01.041 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.041 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.041 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.041 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:01.041 INFO:teuthology.orchestra.run.smithi153.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:01.041 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.041 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.041 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:01.041 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T08:51:01.041 INFO:teuthology.orchestra.run.smithi153.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T08:51:01.041 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T08:51:01.041 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T08:51:01.041 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T08:51:01.041 INFO:teuthology.orchestra.run.smithi153.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T08:51:01.042 INFO:teuthology.orchestra.run.smithi153.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T08:51:01.042 INFO:teuthology.orchestra.run.smithi153.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T08:51:01.042 INFO:teuthology.orchestra.run.smithi153.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T08:51:01.042 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T08:51:01.042 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T08:51:01.042 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.042 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.042 INFO:teuthology.orchestra.run.smithi153.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.042 INFO:teuthology.orchestra.run.smithi153.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T08:51:01.042 INFO:teuthology.orchestra.run.smithi153.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T08:51:01.042 INFO:teuthology.orchestra.run.smithi153.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T08:51:01.042 INFO:teuthology.orchestra.run.smithi153.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T08:51:01.042 INFO:teuthology.orchestra.run.smithi153.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T08:51:01.042 INFO:teuthology.orchestra.run.smithi153.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.042 INFO:teuthology.orchestra.run.smithi153.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.043 INFO:teuthology.orchestra.run.smithi153.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T08:51:01.043 INFO:teuthology.orchestra.run.smithi153.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.043 INFO:teuthology.orchestra.run.smithi153.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T08:51:01.043 INFO:teuthology.orchestra.run.smithi153.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T08:51:01.043 INFO:teuthology.orchestra.run.smithi153.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T08:51:01.043 INFO:teuthology.orchestra.run.smithi153.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T08:51:01.043 INFO:teuthology.orchestra.run.smithi153.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T08:51:01.043 INFO:teuthology.orchestra.run.smithi153.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T08:51:01.043 INFO:teuthology.orchestra.run.smithi153.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T08:51:01.043 INFO:teuthology.orchestra.run.smithi153.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T08:51:01.043 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T08:51:01.043 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T08:51:01.043 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T08:51:01.043 INFO:teuthology.orchestra.run.smithi153.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T08:51:01.044 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T08:51:01.044 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T08:51:01.044 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T08:51:01.044 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T08:51:01.044 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T08:51:01.044 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T08:51:01.044 INFO:teuthology.orchestra.run.smithi153.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T08:51:01.044 INFO:teuthology.orchestra.run.smithi153.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T08:51:01.044 INFO:teuthology.orchestra.run.smithi153.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T08:51:01.044 INFO:teuthology.orchestra.run.smithi153.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T08:51:01.044 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T08:51:01.044 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.044 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.044 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.045 INFO:teuthology.orchestra.run.smithi153.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T08:51:01.045 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T08:51:01.045 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T08:51:01.045 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T08:51:01.045 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T08:51:01.045 INFO:teuthology.orchestra.run.smithi153.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T08:51:01.045 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T08:51:01.045 INFO:teuthology.orchestra.run.smithi153.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T08:51:01.045 INFO:teuthology.orchestra.run.smithi153.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T08:51:01.045 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T08:51:01.045 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T08:51:01.045 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T08:51:01.045 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T08:51:01.046 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T08:51:01.046 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T08:51:01.046 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T08:51:01.046 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T08:51:01.046 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T08:51:01.046 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T08:51:01.046 INFO:teuthology.orchestra.run.smithi153.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T08:51:01.046 INFO:teuthology.orchestra.run.smithi153.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T08:51:01.046 INFO:teuthology.orchestra.run.smithi153.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T08:51:01.046 INFO:teuthology.orchestra.run.smithi153.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T08:51:01.046 INFO:teuthology.orchestra.run.smithi153.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T08:51:01.046 INFO:teuthology.orchestra.run.smithi153.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T08:51:01.046 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T08:51:01.047 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T08:51:01.047 INFO:teuthology.orchestra.run.smithi153.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T08:51:01.047 INFO:teuthology.orchestra.run.smithi153.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T08:51:01.047 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T08:51:01.047 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T08:51:01.047 INFO:teuthology.orchestra.run.smithi153.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T08:51:01.047 INFO:teuthology.orchestra.run.smithi153.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T08:51:01.047 INFO:teuthology.orchestra.run.smithi153.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T08:51:01.047 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T08:51:01.047 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T08:51:01.047 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T08:51:01.047 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T08:51:01.047 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T08:51:01.047 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T08:51:01.047 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.048 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.048 INFO:teuthology.orchestra.run.smithi153.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T08:51:01.048 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T08:51:01.048 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T08:51:01.048 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.048 INFO:teuthology.orchestra.run.smithi153.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T08:51:01.048 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T08:51:01.048 INFO:teuthology.orchestra.run.smithi153.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T08:51:01.048 INFO:teuthology.orchestra.run.smithi153.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T08:51:01.048 INFO:teuthology.orchestra.run.smithi153.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T08:51:01.048 INFO:teuthology.orchestra.run.smithi153.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T08:51:01.048 INFO:teuthology.orchestra.run.smithi153.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T08:51:01.048 INFO:teuthology.orchestra.run.smithi153.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T08:51:01.048 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T08:51:01.049 INFO:teuthology.orchestra.run.smithi153.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T08:51:01.049 INFO:teuthology.orchestra.run.smithi153.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T08:51:01.049 INFO:teuthology.orchestra.run.smithi153.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T08:51:01.049 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T08:51:01.049 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T08:51:01.049 INFO:teuthology.orchestra.run.smithi153.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T08:51:01.049 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.049 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.049 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:01.049 INFO:teuthology.orchestra.run.smithi153.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T08:51:01.049 INFO:teuthology.orchestra.run.smithi153.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T08:51:01.049 INFO:teuthology.orchestra.run.smithi153.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T08:51:01.049 INFO:teuthology.orchestra.run.smithi153.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T08:51:01.049 INFO:teuthology.orchestra.run.smithi153.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T08:51:01.050 INFO:teuthology.orchestra.run.smithi153.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T08:51:01.050 INFO:teuthology.orchestra.run.smithi153.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T08:51:01.050 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-19T08:51:01.050 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-09-19T08:51:01.465 DEBUG:teuthology.parallel:result is None 2024-09-19T08:51:02.884 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T08:51:02.884 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:51:02.884 INFO:teuthology.orchestra.run.smithi120.stdout:Upgraded: 2024-09-19T08:51:02.884 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T08:51:02.884 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T08:51:02.885 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T08:51:02.885 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T08:51:02.885 INFO:teuthology.orchestra.run.smithi120.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.885 INFO:teuthology.orchestra.run.smithi120.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.885 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T08:51:02.885 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T08:51:02.885 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-09-19T08:51:02.885 INFO:teuthology.orchestra.run.smithi120.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T08:51:02.885 INFO:teuthology.orchestra.run.smithi120.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T08:51:02.885 INFO:teuthology.orchestra.run.smithi120.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T08:51:02.885 INFO:teuthology.orchestra.run.smithi120.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T08:51:02.885 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.885 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.886 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.886 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.886 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:02.886 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.886 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.886 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.886 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:02.886 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:02.886 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:02.886 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:02.886 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:02.886 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.886 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.887 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:02.887 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.887 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.887 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.887 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:02.887 INFO:teuthology.orchestra.run.smithi120.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:02.887 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.887 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.887 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:02.887 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T08:51:02.887 INFO:teuthology.orchestra.run.smithi120.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T08:51:02.887 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T08:51:02.887 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T08:51:02.888 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T08:51:02.888 INFO:teuthology.orchestra.run.smithi120.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T08:51:02.888 INFO:teuthology.orchestra.run.smithi120.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T08:51:02.888 INFO:teuthology.orchestra.run.smithi120.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T08:51:02.888 INFO:teuthology.orchestra.run.smithi120.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T08:51:02.888 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T08:51:02.888 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T08:51:02.888 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.888 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.888 INFO:teuthology.orchestra.run.smithi120.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.888 INFO:teuthology.orchestra.run.smithi120.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T08:51:02.888 INFO:teuthology.orchestra.run.smithi120.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T08:51:02.888 INFO:teuthology.orchestra.run.smithi120.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T08:51:02.889 INFO:teuthology.orchestra.run.smithi120.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T08:51:02.889 INFO:teuthology.orchestra.run.smithi120.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T08:51:02.889 INFO:teuthology.orchestra.run.smithi120.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.889 INFO:teuthology.orchestra.run.smithi120.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.889 INFO:teuthology.orchestra.run.smithi120.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T08:51:02.889 INFO:teuthology.orchestra.run.smithi120.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.889 INFO:teuthology.orchestra.run.smithi120.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T08:51:02.889 INFO:teuthology.orchestra.run.smithi120.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T08:51:02.889 INFO:teuthology.orchestra.run.smithi120.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T08:51:02.889 INFO:teuthology.orchestra.run.smithi120.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T08:51:02.889 INFO:teuthology.orchestra.run.smithi120.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T08:51:02.889 INFO:teuthology.orchestra.run.smithi120.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T08:51:02.889 INFO:teuthology.orchestra.run.smithi120.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T08:51:02.889 INFO:teuthology.orchestra.run.smithi120.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T08:51:02.890 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T08:51:02.890 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T08:51:02.890 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T08:51:02.890 INFO:teuthology.orchestra.run.smithi120.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T08:51:02.890 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T08:51:02.890 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T08:51:02.890 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T08:51:02.890 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T08:51:02.890 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T08:51:02.890 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T08:51:02.890 INFO:teuthology.orchestra.run.smithi120.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T08:51:02.890 INFO:teuthology.orchestra.run.smithi120.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T08:51:02.890 INFO:teuthology.orchestra.run.smithi120.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T08:51:02.890 INFO:teuthology.orchestra.run.smithi120.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T08:51:02.891 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T08:51:02.891 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.891 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.891 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.891 INFO:teuthology.orchestra.run.smithi120.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T08:51:02.891 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T08:51:02.891 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T08:51:02.891 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T08:51:02.891 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T08:51:02.891 INFO:teuthology.orchestra.run.smithi120.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T08:51:02.891 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T08:51:02.891 INFO:teuthology.orchestra.run.smithi120.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T08:51:02.891 INFO:teuthology.orchestra.run.smithi120.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T08:51:02.891 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T08:51:02.892 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T08:51:02.892 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T08:51:02.892 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T08:51:02.892 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T08:51:02.892 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T08:51:02.892 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T08:51:02.892 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T08:51:02.892 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T08:51:02.892 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T08:51:02.892 INFO:teuthology.orchestra.run.smithi120.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T08:51:02.892 INFO:teuthology.orchestra.run.smithi120.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T08:51:02.892 INFO:teuthology.orchestra.run.smithi120.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T08:51:02.893 INFO:teuthology.orchestra.run.smithi120.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T08:51:02.893 INFO:teuthology.orchestra.run.smithi120.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T08:51:02.893 INFO:teuthology.orchestra.run.smithi120.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T08:51:02.893 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T08:51:02.893 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T08:51:02.893 INFO:teuthology.orchestra.run.smithi120.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T08:51:02.893 INFO:teuthology.orchestra.run.smithi120.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T08:51:02.893 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T08:51:02.893 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T08:51:02.893 INFO:teuthology.orchestra.run.smithi120.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T08:51:02.893 INFO:teuthology.orchestra.run.smithi120.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T08:51:02.893 INFO:teuthology.orchestra.run.smithi120.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T08:51:02.893 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T08:51:02.893 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T08:51:02.894 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T08:51:02.894 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T08:51:02.894 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T08:51:02.894 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T08:51:02.894 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.894 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.894 INFO:teuthology.orchestra.run.smithi120.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T08:51:02.894 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T08:51:02.894 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T08:51:02.894 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.894 INFO:teuthology.orchestra.run.smithi120.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T08:51:02.894 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T08:51:02.894 INFO:teuthology.orchestra.run.smithi120.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T08:51:02.895 INFO:teuthology.orchestra.run.smithi120.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T08:51:02.895 INFO:teuthology.orchestra.run.smithi120.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T08:51:02.895 INFO:teuthology.orchestra.run.smithi120.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T08:51:02.895 INFO:teuthology.orchestra.run.smithi120.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T08:51:02.895 INFO:teuthology.orchestra.run.smithi120.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T08:51:02.895 INFO:teuthology.orchestra.run.smithi120.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T08:51:02.895 INFO:teuthology.orchestra.run.smithi120.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T08:51:02.895 INFO:teuthology.orchestra.run.smithi120.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T08:51:02.895 INFO:teuthology.orchestra.run.smithi120.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T08:51:02.895 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T08:51:02.895 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T08:51:02.895 INFO:teuthology.orchestra.run.smithi120.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T08:51:02.895 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.896 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.896 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:02.896 INFO:teuthology.orchestra.run.smithi120.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T08:51:02.896 INFO:teuthology.orchestra.run.smithi120.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T08:51:02.896 INFO:teuthology.orchestra.run.smithi120.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T08:51:02.896 INFO:teuthology.orchestra.run.smithi120.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T08:51:02.896 INFO:teuthology.orchestra.run.smithi120.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T08:51:02.896 INFO:teuthology.orchestra.run.smithi120.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T08:51:02.896 INFO:teuthology.orchestra.run.smithi120.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T08:51:02.897 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:51:02.897 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-19T08:51:03.302 DEBUG:teuthology.parallel:result is None 2024-09-19T08:51:04.603 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T08:51:04.603 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T08:51:04.603 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T08:51:04.603 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T08:51:04.603 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T08:51:04.603 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T08:51:04.603 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T08:51:04.603 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T08:51:04.603 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T08:51:04.603 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T08:51:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T08:51:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T08:51:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T08:51:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T08:51:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T08:51:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T08:51:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T08:51:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T08:51:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T08:51:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T08:51:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T08:51:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T08:51:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T08:51:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T08:51:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T08:51:04.604 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T08:51:04.605 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T08:51:04.605 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T08:51:04.605 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T08:51:04.605 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T08:51:04.605 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T08:51:04.616 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T08:51:04.616 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T08:51:04.616 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T08:51:04.616 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T08:51:04.616 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T08:51:04.616 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T08:51:04.616 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T08:51:04.616 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T08:51:04.616 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T08:51:04.616 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T08:51:04.616 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T08:51:04.617 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T08:51:04.617 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T08:51:04.617 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T08:51:04.617 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T08:51:04.617 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T08:51:04.617 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T08:51:04.617 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T08:51:04.617 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T08:51:04.617 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T08:51:04.617 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T08:51:04.617 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T08:51:04.617 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T08:51:04.617 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T08:51:04.618 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T08:51:04.618 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T08:51:04.618 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T08:51:04.618 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T08:51:04.618 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T08:51:04.618 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T08:51:04.618 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T08:51:04.618 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T08:51:04.618 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T08:51:04.618 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T08:51:04.618 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T08:51:04.618 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T08:51:04.618 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T08:51:04.619 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T08:51:04.619 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T08:51:04.619 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T08:51:04.619 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T08:51:04.619 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T08:51:04.619 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T08:51:04.619 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T08:51:04.619 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T08:51:04.619 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T08:51:04.619 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T08:51:04.619 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T08:51:04.619 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T08:51:04.619 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T08:51:04.620 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T08:51:04.620 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T08:51:04.620 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T08:51:04.620 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T08:51:04.620 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T08:51:04.620 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T08:51:04.620 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T08:51:04.620 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T08:51:04.620 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T08:51:04.620 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T08:51:04.620 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T08:51:04.620 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T08:51:04.620 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T08:51:04.620 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T08:51:04.621 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T08:51:04.621 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T08:51:04.621 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T08:51:04.621 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T08:51:04.621 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T08:51:04.621 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T08:51:04.621 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T08:51:04.621 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T08:51:04.621 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T08:51:04.621 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T08:51:04.621 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T08:51:04.621 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T08:51:04.622 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T08:51:04.622 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T08:51:04.622 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T08:51:04.622 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T08:51:04.622 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T08:51:04.622 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T08:51:04.622 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T08:51:04.622 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T08:51:04.622 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T08:51:04.622 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T08:51:04.622 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T08:51:04.622 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T08:51:04.622 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T08:51:04.623 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T08:51:04.623 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T08:51:04.623 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T08:51:04.623 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T08:51:04.623 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T08:51:04.623 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T08:51:04.623 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T08:51:04.623 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T08:51:04.623 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T08:51:04.623 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T08:51:04.623 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T08:51:04.623 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T08:51:04.623 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T08:51:04.623 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T08:51:04.623 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T08:51:04.624 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T08:51:04.624 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T08:51:04.624 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T08:51:04.624 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T08:51:04.624 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T08:51:04.624 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T08:51:04.624 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T08:51:04.624 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T08:51:04.624 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T08:51:04.624 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T08:51:04.624 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T08:51:04.624 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T08:51:04.624 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T08:51:04.624 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T08:51:04.624 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T08:51:04.624 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T08:51:04.625 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T08:51:04.625 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T08:51:04.625 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T08:51:04.625 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T08:51:04.625 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T08:51:04.625 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T08:51:04.625 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T08:51:04.625 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T08:51:04.625 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T08:51:04.625 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T08:51:04.625 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T08:51:04.625 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T08:51:04.625 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T08:51:07.081 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T08:51:07.081 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T08:51:07.081 INFO:teuthology.orchestra.run.smithi196.stdout:Upgraded: 2024-09-19T08:51:07.081 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T08:51:07.082 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T08:51:07.082 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T08:51:07.082 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T08:51:07.082 INFO:teuthology.orchestra.run.smithi196.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.082 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.083 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T08:51:07.083 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T08:51:07.083 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-09-19T08:51:07.083 INFO:teuthology.orchestra.run.smithi196.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T08:51:07.083 INFO:teuthology.orchestra.run.smithi196.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T08:51:07.083 INFO:teuthology.orchestra.run.smithi196.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T08:51:07.084 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T08:51:07.084 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.084 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.084 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.084 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.084 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:07.085 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.085 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.085 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.085 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:07.085 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:07.085 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:07.086 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:07.086 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:07.086 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.086 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.086 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:07.086 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.087 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.087 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.087 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:07.087 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:07.087 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.087 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.088 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:51:07.088 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T08:51:07.088 INFO:teuthology.orchestra.run.smithi196.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T08:51:07.088 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T08:51:07.088 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T08:51:07.089 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T08:51:07.089 INFO:teuthology.orchestra.run.smithi196.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T08:51:07.089 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T08:51:07.089 INFO:teuthology.orchestra.run.smithi196.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T08:51:07.089 INFO:teuthology.orchestra.run.smithi196.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T08:51:07.089 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T08:51:07.090 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T08:51:07.090 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.090 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.090 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.090 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T08:51:07.090 INFO:teuthology.orchestra.run.smithi196.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T08:51:07.091 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T08:51:07.091 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T08:51:07.091 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T08:51:07.091 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.091 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.091 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T08:51:07.091 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.092 INFO:teuthology.orchestra.run.smithi196.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T08:51:07.092 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T08:51:07.092 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T08:51:07.092 INFO:teuthology.orchestra.run.smithi196.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T08:51:07.092 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T08:51:07.092 INFO:teuthology.orchestra.run.smithi196.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T08:51:07.093 INFO:teuthology.orchestra.run.smithi196.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T08:51:07.093 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T08:51:07.093 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T08:51:07.093 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T08:51:07.093 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T08:51:07.093 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T08:51:07.094 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T08:51:07.094 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T08:51:07.094 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T08:51:07.094 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T08:51:07.094 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T08:51:07.094 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T08:51:07.095 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T08:51:07.095 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T08:51:07.095 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T08:51:07.095 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T08:51:07.095 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T08:51:07.096 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.096 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.096 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.096 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T08:51:07.096 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T08:51:07.096 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T08:51:07.097 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T08:51:07.097 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T08:51:07.097 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T08:51:07.097 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T08:51:07.097 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T08:51:07.097 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T08:51:07.098 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T08:51:07.098 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T08:51:07.098 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T08:51:07.098 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T08:51:07.098 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T08:51:07.098 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T08:51:07.099 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T08:51:07.099 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T08:51:07.099 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T08:51:07.099 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T08:51:07.099 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T08:51:07.099 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T08:51:07.100 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T08:51:07.100 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T08:51:07.100 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T08:51:07.100 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T08:51:07.100 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T08:51:07.100 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T08:51:07.101 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T08:51:07.101 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T08:51:07.101 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T08:51:07.101 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T08:51:07.101 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T08:51:07.101 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T08:51:07.102 INFO:teuthology.orchestra.run.smithi196.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T08:51:07.102 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T08:51:07.102 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T08:51:07.102 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T08:51:07.103 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T08:51:07.103 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T08:51:07.103 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T08:51:07.103 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.103 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.103 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T08:51:07.104 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T08:51:07.104 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T08:51:07.104 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.104 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T08:51:07.104 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T08:51:07.104 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T08:51:07.105 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T08:51:07.105 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T08:51:07.105 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T08:51:07.105 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T08:51:07.105 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T08:51:07.105 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T08:51:07.106 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T08:51:07.106 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T08:51:07.106 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T08:51:07.106 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T08:51:07.106 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T08:51:07.106 INFO:teuthology.orchestra.run.smithi196.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T08:51:07.106 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.107 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.107 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:51:07.107 INFO:teuthology.orchestra.run.smithi196.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T08:51:07.107 INFO:teuthology.orchestra.run.smithi196.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T08:51:07.107 INFO:teuthology.orchestra.run.smithi196.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T08:51:07.107 INFO:teuthology.orchestra.run.smithi196.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T08:51:07.108 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T08:51:07.108 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T08:51:07.108 INFO:teuthology.orchestra.run.smithi196.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T08:51:07.108 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T08:51:07.108 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-09-19T08:51:07.495 DEBUG:teuthology.parallel:result is None 2024-09-19T08:51:07.496 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:51:07.647 DEBUG:teuthology.orchestra.run.smithi120:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T08:51:07.673 INFO:teuthology.orchestra.run.smithi120.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T08:51:07.674 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T08:51:07.674 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T08:51:07.676 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:51:07.813 DEBUG:teuthology.orchestra.run.smithi153:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T08:51:07.841 INFO:teuthology.orchestra.run.smithi153.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T08:51:07.841 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T08:51:07.841 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T08:51:07.843 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:51:07.980 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T08:51:08.003 INFO:teuthology.orchestra.run.smithi196.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T08:51:08.004 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T08:51:08.004 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T08:51:08.006 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-19T08:51:08.006 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-19T08:51:08.006 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T08:51:08.040 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-19T08:51:08.040 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T08:51:08.073 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-19T08:51:08.073 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T08:51:08.102 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-19T08:51:08.102 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-19T08:51:08.102 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T08:51:08.131 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T08:51:08.199 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-19T08:51:08.199 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T08:51:08.229 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T08:51:08.298 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-19T08:51:08.298 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T08:51:08.323 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T08:51:08.388 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-19T08:51:08.388 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-19T08:51:08.388 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T08:51:08.418 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T08:51:08.486 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-19T08:51:08.487 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T08:51:08.512 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T08:51:08.577 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-19T08:51:08.578 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T08:51:08.602 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T08:51:08.667 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-19T08:51:08.667 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-19T08:51:08.667 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T08:51:08.695 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T08:51:08.763 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-19T08:51:08.763 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T08:51:08.788 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T08:51:08.853 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-19T08:51:08.853 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T08:51:08.877 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T08:51:08.942 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-19T08:51:09.032 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-19T08:51:09.032 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:51:09.033 INFO:tasks.cephadm:Cluster fsid is 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T08:51:09.033 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-19T08:51:09.034 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.120', 'mon.b': '172.21.15.153', 'mon.c': '172.21.15.196'} 2024-09-19T08:51:09.034 INFO:tasks.cephadm:First mon is mon.a on smithi120 2024-09-19T08:51:09.034 INFO:tasks.cephadm:First mgr is x 2024-09-19T08:51:09.034 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-19T08:51:09.034 DEBUG:teuthology.orchestra.run.smithi120:> sudo hostname $(hostname -s) 2024-09-19T08:51:09.063 DEBUG:teuthology.orchestra.run.smithi153:> sudo hostname $(hostname -s) 2024-09-19T08:51:09.089 DEBUG:teuthology.orchestra.run.smithi196:> sudo hostname $(hostname -s) 2024-09-19T08:51:09.114 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-19T08:51:09.114 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:51:09.256 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56106/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-361-g2f599a46', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-361.g2f599a46'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-18 17:52:24.287651', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-19T08:51:09.395 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref squid, sha1 2f599a46ad10e37c929e5c43da2f6907ddcd1961 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:51:09.396 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T08:51:09.396 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T08:51:09.396 DEBUG:teuthology.orchestra.run.smithi120:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T08:51:09.753 INFO:teuthology.orchestra.run.smithi120.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 08:51 /home/ubuntu/cephtest/cephadm 2024-09-19T08:51:09.754 DEBUG:teuthology.orchestra.run.smithi153:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T08:51:10.109 INFO:teuthology.orchestra.run.smithi153.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 08:51 /home/ubuntu/cephtest/cephadm 2024-09-19T08:51:10.109 DEBUG:teuthology.orchestra.run.smithi196:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T08:51:10.497 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 08:51 /home/ubuntu/cephtest/cephadm 2024-09-19T08:51:10.498 DEBUG:teuthology.orchestra.run.smithi120:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T08:51:10.515 DEBUG:teuthology.orchestra.run.smithi153:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T08:51:10.534 DEBUG:teuthology.orchestra.run.smithi196:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T08:51:10.568 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 on all hosts... 2024-09-19T08:51:10.568 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T08:51:10.570 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T08:51:10.577 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T08:51:12.152 INFO:teuthology.orchestra.run.smithi153.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T08:51:12.172 INFO:teuthology.orchestra.run.smithi120.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T08:51:12.211 INFO:teuthology.orchestra.run.smithi196.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T08:51:59.184 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-09-19T08:51:59.184 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:51:59.184 INFO:teuthology.orchestra.run.smithi153.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T08:51:59.184 INFO:teuthology.orchestra.run.smithi153.stdout: "repo_digests": [ 2024-09-19T08:51:59.184 INFO:teuthology.orchestra.run.smithi153.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T08:51:59.185 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-09-19T08:51:59.185 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-09-19T08:52:43.616 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-19T08:52:43.617 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:52:43.617 INFO:teuthology.orchestra.run.smithi196.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T08:52:43.617 INFO:teuthology.orchestra.run.smithi196.stdout: "repo_digests": [ 2024-09-19T08:52:43.617 INFO:teuthology.orchestra.run.smithi196.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T08:52:43.617 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-09-19T08:52:43.617 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-19T08:52:46.125 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T08:52:46.125 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:52:46.126 INFO:teuthology.orchestra.run.smithi120.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T08:52:46.126 INFO:teuthology.orchestra.run.smithi120.stdout: "repo_digests": [ 2024-09-19T08:52:46.126 INFO:teuthology.orchestra.run.smithi120.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T08:52:46.126 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-19T08:52:46.126 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T08:52:46.151 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /etc/ceph 2024-09-19T08:52:46.211 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /etc/ceph 2024-09-19T08:52:46.245 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph 2024-09-19T08:52:46.278 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 777 /etc/ceph 2024-09-19T08:52:46.303 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 777 /etc/ceph 2024-09-19T08:52:46.334 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 777 /etc/ceph 2024-09-19T08:52:46.363 INFO:tasks.cephadm:Writing seed config... 2024-09-19T08:52:46.365 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-19T08:52:46.365 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-19T08:52:46.365 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-19T08:52:46.365 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-19T08:52:46.365 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-19T08:52:46.365 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-19T08:52:46.365 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-19T08:52:46.365 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-19T08:52:46.365 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-19T08:52:46.365 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-19T08:52:46.365 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-19T08:52:46.365 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-19T08:52:46.366 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-19T08:52:46.366 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-19T08:52:46.366 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-19T08:52:46.366 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-19T08:52:46.366 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-19T08:52:46.366 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-19T08:52:46.366 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-19T08:52:46.366 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-19T08:52:46.366 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-19T08:52:46.366 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-19T08:52:46.366 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-19T08:52:46.366 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-19T08:52:46.366 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-19T08:52:46.366 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-19T08:52:46.366 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-19T08:52:46.367 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-19T08:52:46.367 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-19T08:52:46.367 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-19T08:52:46.367 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-19T08:52:46.367 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-19T08:52:46.367 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-19T08:52:46.367 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-19T08:52:46.367 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-19T08:52:46.367 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-19T08:52:46.367 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-19T08:52:46.367 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-19T08:52:46.367 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-19T08:52:46.367 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-19T08:52:46.367 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-19T08:52:46.368 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-19T08:52:46.368 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-19T08:52:46.368 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-19T08:52:46.368 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-19T08:52:46.368 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-19T08:52:46.368 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-19T08:52:46.369 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-19T08:52:46.369 DEBUG:teuthology.orchestra.run.smithi120:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-19T08:52:46.384 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 507393d2-7664-11ef-bceb-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-19T08:52:46.384 DEBUG:teuthology.orchestra.run.smithi120:mon.a> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mon.a.service 2024-09-19T08:52:46.427 DEBUG:teuthology.orchestra.run.smithi120:mgr.x> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mgr.x.service 2024-09-19T08:52:46.469 INFO:tasks.cephadm:Bootstrapping... 2024-09-19T08:52:46.470 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 -v bootstrap --fsid 507393d2-7664-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.120 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-19T08:52:46.671 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-09-19T08:52:46.671 INFO:teuthology.orchestra.run.smithi120.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961', '-v', 'bootstrap', '--fsid', '507393d2-7664-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.120', '--skip-admin-label'] 2024-09-19T08:52:46.671 INFO:teuthology.orchestra.run.smithi120.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-19T08:52:46.671 INFO:teuthology.orchestra.run.smithi120.stdout:Verifying podman|docker is present... 2024-09-19T08:52:46.694 INFO:teuthology.orchestra.run.smithi120.stdout:/bin/podman: stdout 5.2.2 2024-09-19T08:52:46.694 INFO:teuthology.orchestra.run.smithi120.stdout:Verifying lvm2 is present... 2024-09-19T08:52:46.694 INFO:teuthology.orchestra.run.smithi120.stdout:Verifying time synchronization is in place... 2024-09-19T08:52:46.702 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T08:52:46.702 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T08:52:46.708 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T08:52:46.708 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout inactive 2024-09-19T08:52:46.715 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout enabled 2024-09-19T08:52:46.721 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout active 2024-09-19T08:52:46.722 INFO:teuthology.orchestra.run.smithi120.stdout:Unit chronyd.service is enabled and running 2024-09-19T08:52:46.722 INFO:teuthology.orchestra.run.smithi120.stdout:Repeating the final host check... 2024-09-19T08:52:46.743 INFO:teuthology.orchestra.run.smithi120.stdout:/bin/podman: stdout 5.2.2 2024-09-19T08:52:46.743 INFO:teuthology.orchestra.run.smithi120.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-19T08:52:46.743 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl is present 2024-09-19T08:52:46.743 INFO:teuthology.orchestra.run.smithi120.stdout:lvcreate is present 2024-09-19T08:52:46.750 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T08:52:46.750 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T08:52:46.756 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T08:52:46.756 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout inactive 2024-09-19T08:52:46.762 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout enabled 2024-09-19T08:52:46.769 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout active 2024-09-19T08:52:46.769 INFO:teuthology.orchestra.run.smithi120.stdout:Unit chronyd.service is enabled and running 2024-09-19T08:52:46.769 INFO:teuthology.orchestra.run.smithi120.stdout:Host looks OK 2024-09-19T08:52:46.769 INFO:teuthology.orchestra.run.smithi120.stdout:Cluster fsid: 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T08:52:46.769 INFO:teuthology.orchestra.run.smithi120.stdout:Acquiring lock 139835200762880 on /run/cephadm/507393d2-7664-11ef-bceb-c7b262605968.lock 2024-09-19T08:52:46.769 INFO:teuthology.orchestra.run.smithi120.stdout:Lock 139835200762880 acquired on /run/cephadm/507393d2-7664-11ef-bceb-c7b262605968.lock 2024-09-19T08:52:46.770 INFO:teuthology.orchestra.run.smithi120.stdout:Verifying IP 172.21.15.120 port 3300 ... 2024-09-19T08:52:46.770 INFO:teuthology.orchestra.run.smithi120.stdout:Verifying IP 172.21.15.120 port 6789 ... 2024-09-19T08:52:46.770 INFO:teuthology.orchestra.run.smithi120.stdout:Base mon IP(s) is [172.21.15.120:3300, 172.21.15.120:6789], mon addrv is [v2:172.21.15.120:3300,v1:172.21.15.120:6789] 2024-09-19T08:52:46.773 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.120 metric 100 2024-09-19T08:52:46.773 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.120 metric 100 2024-09-19T08:52:46.776 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-19T08:52:46.776 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-19T08:52:46.776 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-19T08:52:46.778 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-19T08:52:46.778 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-19T08:52:46.779 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T08:52:46.779 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-19T08:52:46.779 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:478f/64 scope link noprefixroute 2024-09-19T08:52:46.779 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T08:52:46.779 INFO:teuthology.orchestra.run.smithi120.stdout:Mon IP `172.21.15.120` is in CIDR network `172.21.0.0/20` 2024-09-19T08:52:46.779 INFO:teuthology.orchestra.run.smithi120.stdout:Mon IP `172.21.15.120` is in CIDR network `172.21.0.0/20` 2024-09-19T08:52:46.779 INFO:teuthology.orchestra.run.smithi120.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-19T08:52:46.780 INFO:teuthology.orchestra.run.smithi120.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-19T08:52:46.780 INFO:teuthology.orchestra.run.smithi120.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T08:52:47.228 INFO:teuthology.orchestra.run.smithi120.stdout:/bin/podman: stdout d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T08:52:47.229 INFO:teuthology.orchestra.run.smithi120.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T08:52:47.229 INFO:teuthology.orchestra.run.smithi120.stdout:/bin/podman: stderr Getting image source signatures 2024-09-19T08:52:47.229 INFO:teuthology.orchestra.run.smithi120.stdout:/bin/podman: stderr Copying blob sha256:c4221751d16d26d2a4b7031ebd71ecffffff13c8cf7401cf95ce80dab3b6d67a 2024-09-19T08:52:47.229 INFO:teuthology.orchestra.run.smithi120.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-19T08:52:47.229 INFO:teuthology.orchestra.run.smithi120.stdout:/bin/podman: stderr Copying config sha256:d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T08:52:47.229 INFO:teuthology.orchestra.run.smithi120.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-19T08:52:48.246 INFO:teuthology.orchestra.run.smithi120.stdout:ceph: stdout ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T08:52:48.246 INFO:teuthology.orchestra.run.smithi120.stdout:Ceph version: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T08:52:48.246 INFO:teuthology.orchestra.run.smithi120.stdout:Extracting ceph user uid/gid from container image... 2024-09-19T08:52:58.177 INFO:teuthology.orchestra.run.smithi120.stdout:stat: stdout 167 167 2024-09-19T08:52:58.178 INFO:teuthology.orchestra.run.smithi120.stdout:Creating initial keys... 2024-09-19T08:52:59.554 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph-authtool: stdout AQBq5utmHZsXNRAAfDYaQXXLVLnPtMM998hOWg== 2024-09-19T08:53:00.575 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph-authtool: stdout AQBr5utmgGNUNBAA0T3r49J30FGyaTbSWZOdBg== 2024-09-19T08:53:01.541 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph-authtool: stdout AQBs5utmMqSzNxAAGOMk2fzfadES45Qh4QpW4Q== 2024-09-19T08:53:01.541 INFO:teuthology.orchestra.run.smithi120.stdout:Creating initial monmap... 2024-09-19T08:53:02.481 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T08:53:02.481 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-19T08:53:02.481 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T08:53:02.481 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T08:53:02.481 INFO:teuthology.orchestra.run.smithi120.stdout:monmaptool for a [v2:172.21.15.120:3300,v1:172.21.15.120:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T08:53:02.481 INFO:teuthology.orchestra.run.smithi120.stdout:setting min_mon_release = quincy 2024-09-19T08:53:02.481 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/monmaptool: set fsid to 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T08:53:02.481 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T08:53:02.481 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:53:02.481 INFO:teuthology.orchestra.run.smithi120.stdout:Creating mon... 2024-09-19T08:53:03.427 INFO:teuthology.orchestra.run.smithi120.stdout:create mon.a on 2024-09-19T08:53:03.644 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-19T08:53:03.918 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-19T08:53:04.545 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-507393d2-7664-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-507393d2-7664-11ef-bceb-c7b262605968.target. 2024-09-19T08:53:04.545 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-507393d2-7664-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-507393d2-7664-11ef-bceb-c7b262605968.target. 2024-09-19T08:53:05.497 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-507393d2-7664-11ef-bceb-c7b262605968@mon.a 2024-09-19T08:53:05.497 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Failed to reset failed state of unit ceph-507393d2-7664-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-507393d2-7664-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-19T08:53:05.799 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-507393d2-7664-11ef-bceb-c7b262605968.target.wants/ceph-507393d2-7664-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-507393d2-7664-11ef-bceb-c7b262605968@.service. 2024-09-19T08:53:06.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:06 smithi120 podman[30768]: 2024-09-19 08:53:05.934119644 +0000 UTC m=+0.015582441 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:53:06.387 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:06 smithi120 podman[30768]: 2024-09-19 08:53:06.107556571 +0000 UTC m=+0.189019369 container create f8ae9b73ef38e2cf751c77cfc7d4b2c86cde642191554f78728faf5075208817 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, io.buildah.version=1.37.2) 2024-09-19T08:53:06.388 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:06 smithi120 podman[30768]: 2024-09-19 08:53:06.343318098 +0000 UTC m=+0.424780899 container init f8ae9b73ef38e2cf751c77cfc7d4b2c86cde642191554f78728faf5075208817 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-09-19T08:53:06.388 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:06 smithi120 podman[30768]: 2024-09-19 08:53:06.34658884 +0000 UTC m=+0.428051637 container start f8ae9b73ef38e2cf751c77cfc7d4b2c86cde642191554f78728faf5075208817 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T08:53:06.440 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T08:53:06.440 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout disabled 2024-09-19T08:53:06.447 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T08:53:06.447 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout inactive 2024-09-19T08:53:06.447 INFO:teuthology.orchestra.run.smithi120.stdout:firewalld.service is not enabled 2024-09-19T08:53:06.447 INFO:teuthology.orchestra.run.smithi120.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T08:53:06.447 INFO:teuthology.orchestra.run.smithi120.stdout:Waiting for mon to start... 2024-09-19T08:53:06.447 INFO:teuthology.orchestra.run.smithi120.stdout:Waiting for mon... 2024-09-19T08:53:06.643 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:06 smithi120 ceph-mon[30787]: mkfs 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T08:53:06.643 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:06 smithi120 ceph-mon[30787]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T08:53:06.643 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:06 smithi120 bash[30768]: f8ae9b73ef38e2cf751c77cfc7d4b2c86cde642191554f78728faf5075208817 2024-09-19T08:53:06.643 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:06 smithi120 systemd[1]: Started Ceph mon.a for 507393d2-7664-11ef-bceb-c7b262605968. 2024-09-19T08:53:07.440 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout cluster: 2024-09-19T08:53:07.440 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout id: 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T08:53:07.440 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-19T08:53:07.440 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2024-09-19T08:53:07.440 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout services: 2024-09-19T08:53:07.440 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.524132s) 2024-09-19T08:53:07.441 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-19T08:53:07.441 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-19T08:53:07.441 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2024-09-19T08:53:07.441 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout data: 2024-09-19T08:53:07.441 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-19T08:53:07.441 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-19T08:53:07.441 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-19T08:53:07.441 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout pgs: 2024-09-19T08:53:07.441 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2024-09-19T08:53:07.441 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.837+0000 7f1e2f15a640 1 Processor -- start 2024-09-19T08:53:07.441 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.837+0000 7f1e2f15a640 1 -- start start 2024-09-19T08:53:07.441 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.838+0000 7f1e2f15a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e28107640 0x7f1e28107a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:07.441 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.838+0000 7f1e2f15a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1e28108010 con 0x7f1e28107640 2024-09-19T08:53:07.442 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.838+0000 7f1e2cef6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e28107640 0x7f1e28107a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:07.442 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.838+0000 7f1e2cef6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e28107640 0x7f1e28107a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60608/0 (socket says 172.21.15.120:60608) 2024-09-19T08:53:07.442 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.838+0000 7f1e2cef6640 1 -- 172.21.15.120:0/2389153386 learned_addr learned my addr 172.21.15.120:0/2389153386 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:07.442 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.838+0000 7f1e2cef6640 1 -- 172.21.15.120:0/2389153386 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e28108840 con 0x7f1e28107640 2024-09-19T08:53:07.442 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.840+0000 7f1e2cef6640 1 --2- 172.21.15.120:0/2389153386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e28107640 0x7f1e28107a40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c009820 tx=0x7f1e1c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872a3b3b2e86a63e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:07.442 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.841+0000 7f1e277fe640 1 -- 172.21.15.120:0/2389153386 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e1c037070 con 0x7f1e28107640 2024-09-19T08:53:07.442 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.841+0000 7f1e277fe640 1 -- 172.21.15.120:0/2389153386 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e1c032440 con 0x7f1e28107640 2024-09-19T08:53:07.442 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.841+0000 7f1e277fe640 1 -- 172.21.15.120:0/2389153386 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e1c032740 con 0x7f1e28107640 2024-09-19T08:53:07.442 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.841+0000 7f1e2f15a640 1 -- 172.21.15.120:0/2389153386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e28107640 msgr2=0x7f1e28107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:07.442 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.841+0000 7f1e2f15a640 1 --2- 172.21.15.120:0/2389153386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e28107640 0x7f1e28107a40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c009820 tx=0x7f1e1c02a270 comp rx=0 tx=0).stop 2024-09-19T08:53:07.442 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.841+0000 7f1e2f15a640 1 -- 172.21.15.120:0/2389153386 shutdown_connections 2024-09-19T08:53:07.442 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.841+0000 7f1e2f15a640 1 --2- 172.21.15.120:0/2389153386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e28107640 0x7f1e28107a40 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.443 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.841+0000 7f1e2f15a640 1 -- 172.21.15.120:0/2389153386 >> 172.21.15.120:0/2389153386 conn(0x7f1e28102e10 msgr2=0x7f1e28105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:07.443 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.841+0000 7f1e2f15a640 1 -- 172.21.15.120:0/2389153386 shutdown_connections 2024-09-19T08:53:07.443 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.841+0000 7f1e2f15a640 1 -- 172.21.15.120:0/2389153386 wait complete. 2024-09-19T08:53:07.443 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.842+0000 7f1e2f15a640 1 Processor -- start 2024-09-19T08:53:07.443 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.842+0000 7f1e2f15a640 1 -- start start 2024-09-19T08:53:07.443 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.842+0000 7f1e2f15a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e28107640 0x7f1e2807db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:07.443 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.842+0000 7f1e2f15a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1e28108de0 con 0x7f1e28107640 2024-09-19T08:53:07.443 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.842+0000 7f1e2cef6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e28107640 0x7f1e2807db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:07.443 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.842+0000 7f1e2cef6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e28107640 0x7f1e2807db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60618/0 (socket says 172.21.15.120:60618) 2024-09-19T08:53:07.443 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.842+0000 7f1e2cef6640 1 -- 172.21.15.120:0/317001869 learned_addr learned my addr 172.21.15.120:0/317001869 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:07.443 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.843+0000 7f1e2cef6640 1 -- 172.21.15.120:0/317001869 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2807e0b0 con 0x7f1e28107640 2024-09-19T08:53:07.443 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.843+0000 7f1e2cef6640 1 --2- 172.21.15.120:0/317001869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e28107640 0x7f1e2807db70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c0090b0 tx=0x7f1e1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:07.444 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.843+0000 7f1e25ffb640 1 -- 172.21.15.120:0/317001869 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e1c041070 con 0x7f1e28107640 2024-09-19T08:53:07.444 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.843+0000 7f1e25ffb640 1 -- 172.21.15.120:0/317001869 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e1c03c7a0 con 0x7f1e28107640 2024-09-19T08:53:07.444 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.843+0000 7f1e2f15a640 1 -- 172.21.15.120:0/317001869 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e2807c2a0 con 0x7f1e28107640 2024-09-19T08:53:07.444 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.843+0000 7f1e25ffb640 1 -- 172.21.15.120:0/317001869 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e1c037040 con 0x7f1e28107640 2024-09-19T08:53:07.444 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.843+0000 7f1e2f15a640 1 -- 172.21.15.120:0/317001869 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e2807c7e0 con 0x7f1e28107640 2024-09-19T08:53:07.444 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.843+0000 7f1e25ffb640 1 -- 172.21.15.120:0/317001869 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1e1c044070 con 0x7f1e28107640 2024-09-19T08:53:07.444 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.843+0000 7f1e25ffb640 1 -- 172.21.15.120:0/317001869 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1e1c03d3f0 con 0x7f1e28107640 2024-09-19T08:53:07.444 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.844+0000 7f1e2f15a640 1 -- 172.21.15.120:0/317001869 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e28109430 con 0x7f1e28107640 2024-09-19T08:53:07.444 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.846+0000 7f1e25ffb640 1 -- 172.21.15.120:0/317001869 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f1e1c046450 con 0x7f1e28107640 2024-09-19T08:53:07.444 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.907+0000 7f1e2f15a640 1 -- 172.21.15.120:0/317001869 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f1e2807cee0 con 0x7f1e28107640 2024-09-19T08:53:07.444 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.907+0000 7f1e25ffb640 1 -- 172.21.15.120:0/317001869 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f1e1c046630 con 0x7f1e28107640 2024-09-19T08:53:07.444 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.908+0000 7f1e2f15a640 1 -- 172.21.15.120:0/317001869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e28107640 msgr2=0x7f1e2807db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:07.445 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.908+0000 7f1e2f15a640 1 --2- 172.21.15.120:0/317001869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e28107640 0x7f1e2807db70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c0090b0 tx=0x7f1e1c0046c0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.445 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.908+0000 7f1e2f15a640 1 -- 172.21.15.120:0/317001869 shutdown_connections 2024-09-19T08:53:07.445 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.908+0000 7f1e2f15a640 1 --2- 172.21.15.120:0/317001869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e28107640 0x7f1e2807db70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.445 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.908+0000 7f1e2f15a640 1 -- 172.21.15.120:0/317001869 >> 172.21.15.120:0/317001869 conn(0x7f1e28102e10 msgr2=0x7f1e2818f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:07.445 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.908+0000 7f1e2f15a640 1 -- 172.21.15.120:0/317001869 shutdown_connections 2024-09-19T08:53:07.445 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:06.908+0000 7f1e2f15a640 1 -- 172.21.15.120:0/317001869 wait complete. 2024-09-19T08:53:07.445 INFO:teuthology.orchestra.run.smithi120.stdout:mon is available 2024-09-19T08:53:07.445 INFO:teuthology.orchestra.run.smithi120.stdout:Assimilating anything we can from ceph.conf... 2024-09-19T08:53:07.657 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:07 smithi120 ceph-mon[30787]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T08:53:07.657 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:07 smithi120 ceph-mon[30787]: monmap epoch 1 2024-09-19T08:53:07.657 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:07 smithi120 ceph-mon[30787]: fsid 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T08:53:07.657 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:07 smithi120 ceph-mon[30787]: last_changed 2024-09-19T08:53:01.874293+0000 2024-09-19T08:53:07.657 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:07 smithi120 ceph-mon[30787]: created 2024-09-19T08:53:01.874293+0000 2024-09-19T08:53:07.657 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:07 smithi120 ceph-mon[30787]: min_mon_release 19 (squid) 2024-09-19T08:53:07.657 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:07 smithi120 ceph-mon[30787]: election_strategy: 1 2024-09-19T08:53:07.657 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:07 smithi120 ceph-mon[30787]: 0: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.a 2024-09-19T08:53:07.657 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:07 smithi120 ceph-mon[30787]: fsmap 2024-09-19T08:53:07.657 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:07 smithi120 ceph-mon[30787]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T08:53:07.658 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:07 smithi120 ceph-mon[30787]: mgrmap e1: no daemons active 2024-09-19T08:53:07.658 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:07 smithi120 ceph-mon[30787]: from='client.? 172.21.15.120:0/317001869' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-19T08:53:08.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:08 smithi120 ceph-mon[30787]: from='client.? 172.21.15.120:0/758514619' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T08:53:08.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:08 smithi120 ceph-mon[30787]: from='client.? 172.21.15.120:0/758514619' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-19T08:53:08.454 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2024-09-19T08:53:08.455 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T08:53:08.455 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout fsid = 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T08:53:08.455 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.120:3300,v1:172.21.15.120:6789] 2024-09-19T08:53:08.455 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T08:53:08.455 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T08:53:08.455 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T08:53:08.455 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T08:53:08.455 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2024-09-19T08:53:08.455 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T08:53:08.455 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T08:53:08.455 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2024-09-19T08:53:08.455 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T08:53:08.455 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T08:53:08.456 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T08:53:08.456 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T08:53:08.456 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T08:53:08.456 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T08:53:08.456 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.835+0000 7fccb1356640 1 Processor -- start 2024-09-19T08:53:08.456 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.835+0000 7fccb1356640 1 -- start start 2024-09-19T08:53:08.456 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.835+0000 7fccb1356640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccac106510 0x7fccac106910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:08.456 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.835+0000 7fccb1356640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fccac106ee0 con 0x7fccac106510 2024-09-19T08:53:08.456 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.836+0000 7fccaaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccac106510 0x7fccac106910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:08.456 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.836+0000 7fccaaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccac106510 0x7fccac106910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60624/0 (socket says 172.21.15.120:60624) 2024-09-19T08:53:08.456 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.836+0000 7fccaaffd640 1 -- 172.21.15.120:0/2681701362 learned_addr learned my addr 172.21.15.120:0/2681701362 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:08.456 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.836+0000 7fccaaffd640 1 -- 172.21.15.120:0/2681701362 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccac107710 con 0x7fccac106510 2024-09-19T08:53:08.456 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.836+0000 7fccaaffd640 1 --2- 172.21.15.120:0/2681701362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccac106510 0x7fccac106910 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009e80 tx=0x7fcc9c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5ae4d389627853 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:08.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.837+0000 7fcca9ffb640 1 -- 172.21.15.120:0/2681701362 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc9c036070 con 0x7fccac106510 2024-09-19T08:53:08.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.837+0000 7fcca9ffb640 1 -- 172.21.15.120:0/2681701362 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc9c031400 con 0x7fccac106510 2024-09-19T08:53:08.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.837+0000 7fcca9ffb640 1 -- 172.21.15.120:0/2681701362 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc9c031700 con 0x7fccac106510 2024-09-19T08:53:08.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.837+0000 7fccb1356640 1 -- 172.21.15.120:0/2681701362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccac106510 msgr2=0x7fccac106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:08.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.837+0000 7fccb1356640 1 --2- 172.21.15.120:0/2681701362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccac106510 0x7fccac106910 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009e80 tx=0x7fcc9c02a7a0 comp rx=0 tx=0).stop 2024-09-19T08:53:08.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.837+0000 7fccb1356640 1 -- 172.21.15.120:0/2681701362 shutdown_connections 2024-09-19T08:53:08.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.837+0000 7fccb1356640 1 --2- 172.21.15.120:0/2681701362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccac106510 0x7fccac106910 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:08.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.837+0000 7fccb1356640 1 -- 172.21.15.120:0/2681701362 >> 172.21.15.120:0/2681701362 conn(0x7fccac101ce0 msgr2=0x7fccac104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:08.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.837+0000 7fccb1356640 1 -- 172.21.15.120:0/2681701362 shutdown_connections 2024-09-19T08:53:08.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.837+0000 7fccb1356640 1 -- 172.21.15.120:0/2681701362 wait complete. 2024-09-19T08:53:08.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.838+0000 7fccb1356640 1 Processor -- start 2024-09-19T08:53:08.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.838+0000 7fccb1356640 1 -- start start 2024-09-19T08:53:08.457 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.838+0000 7fccb1356640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccac106510 0x7fccac07ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:08.458 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.838+0000 7fccb1356640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fccac107cb0 con 0x7fccac106510 2024-09-19T08:53:08.458 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.838+0000 7fccaaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccac106510 0x7fccac07ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:08.458 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.838+0000 7fccaaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccac106510 0x7fccac07ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60640/0 (socket says 172.21.15.120:60640) 2024-09-19T08:53:08.458 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.838+0000 7fccaaffd640 1 -- 172.21.15.120:0/758514619 learned_addr learned my addr 172.21.15.120:0/758514619 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:08.458 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.839+0000 7fccaaffd640 1 -- 172.21.15.120:0/758514619 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccac19db40 con 0x7fccac106510 2024-09-19T08:53:08.458 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.839+0000 7fccaaffd640 1 --2- 172.21.15.120:0/758514619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccac106510 0x7fccac07ba50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c034040 tx=0x7fcc9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:08.458 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.839+0000 7fcca3fff640 1 -- 172.21.15.120:0/758514619 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc9c041070 con 0x7fccac106510 2024-09-19T08:53:08.458 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.839+0000 7fcca3fff640 1 -- 172.21.15.120:0/758514619 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc9c03c650 con 0x7fccac106510 2024-09-19T08:53:08.458 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.839+0000 7fccb1356640 1 -- 172.21.15.120:0/758514619 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccac19dd10 con 0x7fccac106510 2024-09-19T08:53:08.458 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.839+0000 7fcca3fff640 1 -- 172.21.15.120:0/758514619 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc9c036040 con 0x7fccac106510 2024-09-19T08:53:08.458 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.839+0000 7fccb1356640 1 -- 172.21.15.120:0/758514619 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccac19e130 con 0x7fccac106510 2024-09-19T08:53:08.458 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.839+0000 7fcca3fff640 1 -- 172.21.15.120:0/758514619 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcc9c044070 con 0x7fccac106510 2024-09-19T08:53:08.459 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.839+0000 7fcca3fff640 1 -- 172.21.15.120:0/758514619 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcc9c046c10 con 0x7fccac106510 2024-09-19T08:53:08.459 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.840+0000 7fccb1356640 1 -- 172.21.15.120:0/758514619 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccac108300 con 0x7fccac106510 2024-09-19T08:53:08.459 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.842+0000 7fcca3fff640 1 -- 172.21.15.120:0/758514619 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fcc9c03c950 con 0x7fccac106510 2024-09-19T08:53:08.459 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.901+0000 7fccb1356640 1 -- 172.21.15.120:0/758514619 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fccac19e540 con 0x7fccac106510 2024-09-19T08:53:08.459 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.904+0000 7fcca3fff640 1 -- 172.21.15.120:0/758514619 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+431 (secure 0 0 0) 0x7fcc9c03cb30 con 0x7fccac106510 2024-09-19T08:53:08.459 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.905+0000 7fcca3fff640 1 -- 172.21.15.120:0/758514619 <== mon.0 v2:172.21.15.120:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcc9c056070 con 0x7fccac106510 2024-09-19T08:53:08.459 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.905+0000 7fccb1356640 1 -- 172.21.15.120:0/758514619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccac106510 msgr2=0x7fccac07ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:08.459 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.905+0000 7fccb1356640 1 --2- 172.21.15.120:0/758514619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccac106510 0x7fccac07ba50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c034040 tx=0x7fcc9c004a40 comp rx=0 tx=0).stop 2024-09-19T08:53:08.459 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.905+0000 7fccb1356640 1 -- 172.21.15.120:0/758514619 shutdown_connections 2024-09-19T08:53:08.459 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.905+0000 7fccb1356640 1 --2- 172.21.15.120:0/758514619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccac106510 0x7fccac07ba50 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:08.459 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.905+0000 7fccb1356640 1 -- 172.21.15.120:0/758514619 >> 172.21.15.120:0/758514619 conn(0x7fccac101ce0 msgr2=0x7fccac18dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:08.459 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.905+0000 7fccb1356640 1 -- 172.21.15.120:0/758514619 shutdown_connections 2024-09-19T08:53:08.460 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:07.905+0000 7fccb1356640 1 -- 172.21.15.120:0/758514619 wait complete. 2024-09-19T08:53:08.460 INFO:teuthology.orchestra.run.smithi120.stdout:Generating new minimal ceph.conf... 2024-09-19T08:53:09.685 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:09 smithi120 ceph-mon[30787]: from='client.? 172.21.15.120:0/1329387417' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:53:09.692 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.041+0000 7f208a199640 1 Processor -- start 2024-09-19T08:53:09.692 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.041+0000 7f208a199640 1 -- start start 2024-09-19T08:53:09.692 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.042+0000 7f208a199640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20841080f0 0x7f208410a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:09.692 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.042+0000 7f208a199640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f208407be50 con 0x7f20841080f0 2024-09-19T08:53:09.692 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.042+0000 7f20837fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20841080f0 0x7f208410a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:09.692 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.042+0000 7f20837fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20841080f0 0x7f208410a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60646/0 (socket says 172.21.15.120:60646) 2024-09-19T08:53:09.692 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.042+0000 7f20837fe640 1 -- 172.21.15.120:0/465109260 learned_addr learned my addr 172.21.15.120:0/465109260 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:09.692 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.042+0000 7f20837fe640 1 -- 172.21.15.120:0/465109260 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f208410aa20 con 0x7f20841080f0 2024-09-19T08:53:09.693 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.043+0000 7f20837fe640 1 --2- 172.21.15.120:0/465109260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20841080f0 0x7f208410a4e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f206c009820 tx=0x7f206c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f0826ae0453ab8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:09.693 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.043+0000 7f20827fc640 1 -- 172.21.15.120:0/465109260 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f206c037070 con 0x7f20841080f0 2024-09-19T08:53:09.693 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.043+0000 7f20827fc640 1 -- 172.21.15.120:0/465109260 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f206c032900 con 0x7f20841080f0 2024-09-19T08:53:09.693 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.043+0000 7f20827fc640 1 -- 172.21.15.120:0/465109260 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f206c030850 con 0x7f20841080f0 2024-09-19T08:53:09.693 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.044+0000 7f208a199640 1 -- 172.21.15.120:0/465109260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20841080f0 msgr2=0x7f208410a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:09.693 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.044+0000 7f208a199640 1 --2- 172.21.15.120:0/465109260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20841080f0 0x7f208410a4e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f206c009820 tx=0x7f206c02a270 comp rx=0 tx=0).stop 2024-09-19T08:53:09.693 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.044+0000 7f208a199640 1 -- 172.21.15.120:0/465109260 shutdown_connections 2024-09-19T08:53:09.693 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.044+0000 7f208a199640 1 --2- 172.21.15.120:0/465109260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20841080f0 0x7f208410a4e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:09.693 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.044+0000 7f208a199640 1 -- 172.21.15.120:0/465109260 >> 172.21.15.120:0/465109260 conn(0x7f2084101db0 msgr2=0x7f20841041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:09.693 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.044+0000 7f208a199640 1 -- 172.21.15.120:0/465109260 shutdown_connections 2024-09-19T08:53:09.693 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.044+0000 7f208a199640 1 -- 172.21.15.120:0/465109260 wait complete. 2024-09-19T08:53:09.693 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.044+0000 7f208a199640 1 Processor -- start 2024-09-19T08:53:09.694 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.045+0000 7f208a199640 1 -- start start 2024-09-19T08:53:09.694 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.045+0000 7f208a199640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20841080f0 0x7f2084107970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:09.694 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.045+0000 7f208a199640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f208410ba60 con 0x7f20841080f0 2024-09-19T08:53:09.694 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.045+0000 7f20837fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20841080f0 0x7f2084107970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:09.694 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.045+0000 7f20837fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20841080f0 0x7f2084107970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60652/0 (socket says 172.21.15.120:60652) 2024-09-19T08:53:09.694 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.045+0000 7f20837fe640 1 -- 172.21.15.120:0/1329387417 learned_addr learned my addr 172.21.15.120:0/1329387417 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:09.694 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.045+0000 7f20837fe640 1 -- 172.21.15.120:0/1329387417 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2084107eb0 con 0x7f20841080f0 2024-09-19T08:53:09.694 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.045+0000 7f20837fe640 1 --2- 172.21.15.120:0/1329387417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20841080f0 0x7f2084107970 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f206c032ce0 tx=0x7f206c032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:09.694 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.045+0000 7f2080ff9640 1 -- 172.21.15.120:0/1329387417 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f206c042070 con 0x7f20841080f0 2024-09-19T08:53:09.694 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.045+0000 7f208a199640 1 -- 172.21.15.120:0/1329387417 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20841060a0 con 0x7f20841080f0 2024-09-19T08:53:09.694 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.045+0000 7f2080ff9640 1 -- 172.21.15.120:0/1329387417 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f206c0308c0 con 0x7f20841080f0 2024-09-19T08:53:09.694 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.045+0000 7f2080ff9640 1 -- 172.21.15.120:0/1329387417 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f206c037040 con 0x7f20841080f0 2024-09-19T08:53:09.694 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.045+0000 7f208a199640 1 -- 172.21.15.120:0/1329387417 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20841065e0 con 0x7f20841080f0 2024-09-19T08:53:09.694 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.046+0000 7f2080ff9640 1 -- 172.21.15.120:0/1329387417 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f206c045070 con 0x7f20841080f0 2024-09-19T08:53:09.695 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.046+0000 7f2080ff9640 1 -- 172.21.15.120:0/1329387417 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f206c047bd0 con 0x7f20841080f0 2024-09-19T08:53:09.695 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.046+0000 7f208a199640 1 -- 172.21.15.120:0/1329387417 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2084108550 con 0x7f20841080f0 2024-09-19T08:53:09.695 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.049+0000 7f2080ff9640 1 -- 172.21.15.120:0/1329387417 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f206c03c830 con 0x7f20841080f0 2024-09-19T08:53:09.695 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.105+0000 7f208a199640 1 -- 172.21.15.120:0/1329387417 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f2084106b10 con 0x7f20841080f0 2024-09-19T08:53:09.695 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.106+0000 7f2080ff9640 1 -- 172.21.15.120:0/1329387417 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7f206c03ca10 con 0x7f20841080f0 2024-09-19T08:53:09.695 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.107+0000 7f208a199640 1 -- 172.21.15.120:0/1329387417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20841080f0 msgr2=0x7f2084107970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:09.695 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.107+0000 7f208a199640 1 --2- 172.21.15.120:0/1329387417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20841080f0 0x7f2084107970 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f206c032ce0 tx=0x7f206c032d10 comp rx=0 tx=0).stop 2024-09-19T08:53:09.695 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.107+0000 7f208a199640 1 -- 172.21.15.120:0/1329387417 shutdown_connections 2024-09-19T08:53:09.695 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.107+0000 7f208a199640 1 --2- 172.21.15.120:0/1329387417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20841080f0 0x7f2084107970 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:09.695 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.107+0000 7f208a199640 1 -- 172.21.15.120:0/1329387417 >> 172.21.15.120:0/1329387417 conn(0x7f2084101db0 msgr2=0x7f2084102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:09.695 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.107+0000 7f208a199640 1 -- 172.21.15.120:0/1329387417 shutdown_connections 2024-09-19T08:53:09.695 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:09.107+0000 7f208a199640 1 -- 172.21.15.120:0/1329387417 wait complete. 2024-09-19T08:53:09.695 INFO:teuthology.orchestra.run.smithi120.stdout:Restarting the monitor... 2024-09-19T08:53:09.955 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:09 smithi120 systemd[1]: Stopping Ceph mon.a for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T08:53:10.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:09 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-a[30783]: 2024-09-19T08:53:09.955+0000 7f79f82c0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:53:10.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:09 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-a[30783]: 2024-09-19T08:53:09.955+0000 7f79f82c0640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-19T08:53:10.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:10 smithi120 podman[31027]: 2024-09-19 08:53:10.078843373 +0000 UTC m=+0.233004390 container died f8ae9b73ef38e2cf751c77cfc7d4b2c86cde642191554f78728faf5075208817 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-19T08:53:10.680 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:10 smithi120 podman[31027]: 2024-09-19 08:53:10.447058134 +0000 UTC m=+0.601219151 container cleanup f8ae9b73ef38e2cf751c77cfc7d4b2c86cde642191554f78728faf5075208817 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T08:53:10.680 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:10 smithi120 bash[31027]: ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-a 2024-09-19T08:53:11.014 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:10 smithi120 podman[31039]: 2024-09-19 08:53:10.681266033 +0000 UTC m=+0.598987354 container remove f8ae9b73ef38e2cf751c77cfc7d4b2c86cde642191554f78728faf5075208817 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2) 2024-09-19T08:53:11.014 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:10 smithi120 systemd[1]: ceph-507393d2-7664-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-19T08:53:11.014 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:10 smithi120 systemd[1]: Stopped Ceph mon.a for 507393d2-7664-11ef-bceb-c7b262605968. 2024-09-19T08:53:11.014 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:10 smithi120 systemd[1]: Starting Ceph mon.a for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T08:53:11.274 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 podman[31131]: 2024-09-19 08:53:10.917849601 +0000 UTC m=+0.015399961 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:53:11.274 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 podman[31131]: 2024-09-19 08:53:11.023907668 +0000 UTC m=+0.121458022 container create 12359571ca9cfae382f0f597a6d85c8061cdf3be1bb0b9eb77f3f9aa369d9ea6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-a, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-19T08:53:11.274 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 podman[31131]: 2024-09-19 08:53:11.251209742 +0000 UTC m=+0.348760094 container init 12359571ca9cfae382f0f597a6d85c8061cdf3be1bb0b9eb77f3f9aa369d9ea6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T08:53:11.275 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 podman[31131]: 2024-09-19 08:53:11.254348766 +0000 UTC m=+0.351899127 container start 12359571ca9cfae382f0f597a6d85c8061cdf3be1bb0b9eb77f3f9aa369d9ea6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-a, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-19T08:53:11.534 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T08:53:11.534 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T08:53:11.534 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: pidfile_write: ignore empty --pid-file 2024-09-19T08:53:11.534 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: load: jerasure load: lrc 2024-09-19T08:53:11.534 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: RocksDB version: 7.9.2 2024-09-19T08:53:11.534 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Git sha 0 2024-09-19T08:53:11.534 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T08:53:11.535 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: DB SUMMARY 2024-09-19T08:53:11.535 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: DB Session ID: 6NBSHRM0DOBON45D8A8N 2024-09-19T08:53:11.535 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: CURRENT file: CURRENT 2024-09-19T08:53:11.535 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T08:53:11.535 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-19T08:53:11.535 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-19T08:53:11.535 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105432 ; 2024-09-19T08:53:11.535 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.error_if_exists: 0 2024-09-19T08:53:11.535 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.create_if_missing: 0 2024-09-19T08:53:11.535 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.paranoid_checks: 1 2024-09-19T08:53:11.535 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T08:53:11.535 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T08:53:11.535 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T08:53:11.535 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.env: 0x55e748239940 2024-09-19T08:53:11.535 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T08:53:11.536 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.info_log: 0x55e74a5016e0 2024-09-19T08:53:11.536 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T08:53:11.536 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.statistics: (nil) 2024-09-19T08:53:11.536 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.use_fsync: 0 2024-09-19T08:53:11.536 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.max_log_file_size: 0 2024-09-19T08:53:11.536 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T08:53:11.536 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T08:53:11.536 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T08:53:11.536 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T08:53:11.536 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.allow_fallocate: 1 2024-09-19T08:53:11.536 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T08:53:11.536 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T08:53:11.536 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.use_direct_reads: 0 2024-09-19T08:53:11.536 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T08:53:11.537 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T08:53:11.537 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.db_log_dir: 2024-09-19T08:53:11.537 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.wal_dir: 2024-09-19T08:53:11.537 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T08:53:11.538 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T08:53:11.538 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T08:53:11.538 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T08:53:11.538 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T08:53:11.538 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T08:53:11.538 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T08:53:11.539 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T08:53:11.539 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.write_buffer_manager: 0x55e74a510dc0 2024-09-19T08:53:11.539 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T08:53:11.539 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T08:53:11.539 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T08:53:11.539 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T08:53:11.539 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T08:53:11.539 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T08:53:11.539 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T08:53:11.539 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T08:53:11.539 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.unordered_write: 0 2024-09-19T08:53:11.539 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T08:53:11.539 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T08:53:11.539 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T08:53:11.540 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T08:53:11.540 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.row_cache: None 2024-09-19T08:53:11.540 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.wal_filter: None 2024-09-19T08:53:11.540 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T08:53:11.540 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T08:53:11.540 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.two_write_queues: 0 2024-09-19T08:53:11.540 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T08:53:11.540 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.wal_compression: 0 2024-09-19T08:53:11.540 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.atomic_flush: 0 2024-09-19T08:53:11.540 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T08:53:11.540 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T08:53:11.540 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T08:53:11.540 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.log_readahead_size: 0 2024-09-19T08:53:11.541 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T08:53:11.541 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T08:53:11.541 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T08:53:11.541 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T08:53:11.541 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T08:53:11.541 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T08:53:11.541 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T08:53:11.541 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.max_background_jobs: 2 2024-09-19T08:53:11.541 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.max_background_compactions: -1 2024-09-19T08:53:11.541 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.max_subcompactions: 1 2024-09-19T08:53:11.541 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T08:53:11.541 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T08:53:11.541 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T08:53:11.541 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T08:53:11.542 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T08:53:11.542 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T08:53:11.542 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T08:53:11.542 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T08:53:11.542 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.max_open_files: -1 2024-09-19T08:53:11.542 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T08:53:11.542 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T08:53:11.542 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T08:53:11.542 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T08:53:11.542 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.max_background_flushes: -1 2024-09-19T08:53:11.542 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Compression algorithms supported: 2024-09-19T08:53:11.542 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: kZSTD supported: 0 2024-09-19T08:53:11.542 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: kXpressCompression supported: 0 2024-09-19T08:53:11.542 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: kBZip2Compression supported: 0 2024-09-19T08:53:11.543 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T08:53:11.543 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: kLZ4Compression supported: 1 2024-09-19T08:53:11.543 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: kZlibCompression supported: 1 2024-09-19T08:53:11.543 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T08:53:11.543 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: kSnappyCompression supported: 1 2024-09-19T08:53:11.543 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T08:53:11.543 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T08:53:11.543 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-19T08:53:11.543 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T08:53:11.543 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T08:53:11.543 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.merge_operator: 2024-09-19T08:53:11.543 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.compaction_filter: None 2024-09-19T08:53:11.543 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T08:53:11.543 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T08:53:11.544 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T08:53:11.544 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T08:53:11.544 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e74a501320) 2024-09-19T08:53:11.544 INFO:journalctl@ceph.mon.a.smithi120.stdout: cache_index_and_filter_blocks: 1 2024-09-19T08:53:11.544 INFO:journalctl@ceph.mon.a.smithi120.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T08:53:11.544 INFO:journalctl@ceph.mon.a.smithi120.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T08:53:11.544 INFO:journalctl@ceph.mon.a.smithi120.stdout: pin_top_level_index_and_filter: 1 2024-09-19T08:53:11.544 INFO:journalctl@ceph.mon.a.smithi120.stdout: index_type: 0 2024-09-19T08:53:11.544 INFO:journalctl@ceph.mon.a.smithi120.stdout: data_block_index_type: 0 2024-09-19T08:53:11.544 INFO:journalctl@ceph.mon.a.smithi120.stdout: index_shortening: 1 2024-09-19T08:53:11.544 INFO:journalctl@ceph.mon.a.smithi120.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T08:53:11.544 INFO:journalctl@ceph.mon.a.smithi120.stdout: checksum: 4 2024-09-19T08:53:11.544 INFO:journalctl@ceph.mon.a.smithi120.stdout: no_block_cache: 0 2024-09-19T08:53:11.544 INFO:journalctl@ceph.mon.a.smithi120.stdout: block_cache: 0x55e74a4f51f0 2024-09-19T08:53:11.545 INFO:journalctl@ceph.mon.a.smithi120.stdout: block_cache_name: BinnedLRUCache 2024-09-19T08:53:11.545 INFO:journalctl@ceph.mon.a.smithi120.stdout: block_cache_options: 2024-09-19T08:53:11.545 INFO:journalctl@ceph.mon.a.smithi120.stdout: capacity : 536870912 2024-09-19T08:53:11.545 INFO:journalctl@ceph.mon.a.smithi120.stdout: num_shard_bits : 4 2024-09-19T08:53:11.545 INFO:journalctl@ceph.mon.a.smithi120.stdout: strict_capacity_limit : 0 2024-09-19T08:53:11.545 INFO:journalctl@ceph.mon.a.smithi120.stdout: high_pri_pool_ratio: 0.000 2024-09-19T08:53:11.545 INFO:journalctl@ceph.mon.a.smithi120.stdout: block_cache_compressed: (nil) 2024-09-19T08:53:11.545 INFO:journalctl@ceph.mon.a.smithi120.stdout: persistent_cache: (nil) 2024-09-19T08:53:11.545 INFO:journalctl@ceph.mon.a.smithi120.stdout: block_size: 4096 2024-09-19T08:53:11.545 INFO:journalctl@ceph.mon.a.smithi120.stdout: block_size_deviation: 10 2024-09-19T08:53:11.545 INFO:journalctl@ceph.mon.a.smithi120.stdout: block_restart_interval: 16 2024-09-19T08:53:11.545 INFO:journalctl@ceph.mon.a.smithi120.stdout: index_block_restart_interval: 1 2024-09-19T08:53:11.545 INFO:journalctl@ceph.mon.a.smithi120.stdout: metadata_block_size: 4096 2024-09-19T08:53:11.545 INFO:journalctl@ceph.mon.a.smithi120.stdout: partition_filters: 0 2024-09-19T08:53:11.545 INFO:journalctl@ceph.mon.a.smithi120.stdout: use_delta_encoding: 1 2024-09-19T08:53:11.546 INFO:journalctl@ceph.mon.a.smithi120.stdout: filter_policy: bloomfilter 2024-09-19T08:53:11.546 INFO:journalctl@ceph.mon.a.smithi120.stdout: whole_key_filtering: 1 2024-09-19T08:53:11.546 INFO:journalctl@ceph.mon.a.smithi120.stdout: verify_compression: 0 2024-09-19T08:53:11.546 INFO:journalctl@ceph.mon.a.smithi120.stdout: read_amp_bytes_per_bit: 0 2024-09-19T08:53:11.546 INFO:journalctl@ceph.mon.a.smithi120.stdout: format_version: 5 2024-09-19T08:53:11.546 INFO:journalctl@ceph.mon.a.smithi120.stdout: enable_index_compression: 1 2024-09-19T08:53:11.546 INFO:journalctl@ceph.mon.a.smithi120.stdout: block_align: 0 2024-09-19T08:53:11.546 INFO:journalctl@ceph.mon.a.smithi120.stdout: max_auto_readahead_size: 262144 2024-09-19T08:53:11.546 INFO:journalctl@ceph.mon.a.smithi120.stdout: prepopulate_block_cache: 0 2024-09-19T08:53:11.546 INFO:journalctl@ceph.mon.a.smithi120.stdout: initial_auto_readahead_size: 8192 2024-09-19T08:53:11.546 INFO:journalctl@ceph.mon.a.smithi120.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T08:53:11.546 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T08:53:11.546 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T08:53:11.547 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.compression: NoCompression 2024-09-19T08:53:11.547 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T08:53:11.547 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T08:53:11.547 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T08:53:11.547 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.num_levels: 7 2024-09-19T08:53:11.547 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T08:53:11.547 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T08:53:11.547 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T08:53:11.547 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T08:53:11.547 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T08:53:11.547 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T08:53:11.547 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T08:53:11.547 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T08:53:11.547 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T08:53:11.548 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T08:53:11.548 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T08:53:11.548 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T08:53:11.548 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T08:53:11.548 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T08:53:11.548 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T08:53:11.548 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T08:53:11.548 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T08:53:11.548 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T08:53:11.548 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T08:53:11.548 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T08:53:11.548 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T08:53:11.548 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T08:53:11.549 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T08:53:11.549 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T08:53:11.549 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T08:53:11.549 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T08:53:11.549 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T08:53:11.549 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T08:53:11.549 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T08:53:11.549 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T08:53:11.549 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T08:53:11.549 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T08:53:11.549 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T08:53:11.549 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T08:53:11.549 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T08:53:11.550 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T08:53:11.550 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T08:53:11.550 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T08:53:11.550 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T08:53:11.550 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T08:53:11.550 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T08:53:11.550 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T08:53:11.550 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T08:53:11.550 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T08:53:11.550 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T08:53:11.550 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T08:53:11.550 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T08:53:11.550 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T08:53:11.550 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T08:53:11.551 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T08:53:11.551 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T08:53:11.551 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T08:53:11.551 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T08:53:11.551 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T08:53:11.551 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.inplace_update_support: 0 2024-09-19T08:53:11.551 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T08:53:11.551 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T08:53:11.551 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T08:53:11.551 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T08:53:11.551 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.bloom_locality: 0 2024-09-19T08:53:11.551 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.max_successive_merges: 0 2024-09-19T08:53:11.551 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T08:53:11.551 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T08:53:11.552 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T08:53:11.552 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T08:53:11.552 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.ttl: 2592000 2024-09-19T08:53:11.552 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T08:53:11.552 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T08:53:11.552 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T08:53:11.552 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.enable_blob_files: false 2024-09-19T08:53:11.552 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.min_blob_size: 0 2024-09-19T08:53:11.552 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T08:53:11.552 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T08:53:11.552 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T08:53:11.552 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T08:53:11.552 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T08:53:11.552 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T08:53:11.553 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T08:53:11.553 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T08:53:11.553 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-19T08:53:11.553 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-19T08:53:11.553 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2008569d-c137-4c03-8585-80872ce06480 2024-09-19T08:53:11.553 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726735991280401, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-19T08:53:11.553 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-19T08:53:11.553 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726735991281341, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100480, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98374, "index_size": 298, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14320, "raw_average_key_size": 49, "raw_value_size": 90627, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726735991, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2008569d-c137-4c03-8585-80872ce06480", "db_session_id": "6NBSHRM0DOBON45D8A8N", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-19T08:53:11.553 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726735991281434, "job": 1, "event": "recovery_finished"} 2024-09-19T08:53:11.553 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-19T08:53:11.553 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T08:53:11.553 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e74a522e00 2024-09-19T08:53:11.553 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: DB pointer 0x55e74a61e000 2024-09-19T08:53:11.554 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T08:53:11.554 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T08:53:11.554 INFO:journalctl@ceph.mon.a.smithi120.stdout: ** DB Stats ** 2024-09-19T08:53:11.554 INFO:journalctl@ceph.mon.a.smithi120.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T08:53:11.554 INFO:journalctl@ceph.mon.a.smithi120.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T08:53:11.554 INFO:journalctl@ceph.mon.a.smithi120.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T08:53:11.554 INFO:journalctl@ceph.mon.a.smithi120.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T08:53:11.554 INFO:journalctl@ceph.mon.a.smithi120.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T08:53:11.554 INFO:journalctl@ceph.mon.a.smithi120.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T08:53:11.554 INFO:journalctl@ceph.mon.a.smithi120.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T08:53:11.554 INFO:journalctl@ceph.mon.a.smithi120.stdout: 2024-09-19T08:53:11.554 INFO:journalctl@ceph.mon.a.smithi120.stdout: ** Compaction Stats [default] ** 2024-09-19T08:53:11.554 INFO:journalctl@ceph.mon.a.smithi120.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T08:53:11.554 INFO:journalctl@ceph.mon.a.smithi120.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T08:53:11.554 INFO:journalctl@ceph.mon.a.smithi120.stdout: L0 2/0 99.99 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 169.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T08:53:11.555 INFO:journalctl@ceph.mon.a.smithi120.stdout: Sum 2/0 99.99 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 169.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T08:53:11.555 INFO:journalctl@ceph.mon.a.smithi120.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 169.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T08:53:11.555 INFO:journalctl@ceph.mon.a.smithi120.stdout: 2024-09-19T08:53:11.555 INFO:journalctl@ceph.mon.a.smithi120.stdout: ** Compaction Stats [default] ** 2024-09-19T08:53:11.555 INFO:journalctl@ceph.mon.a.smithi120.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T08:53:11.555 INFO:journalctl@ceph.mon.a.smithi120.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T08:53:11.555 INFO:journalctl@ceph.mon.a.smithi120.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 169.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T08:53:11.555 INFO:journalctl@ceph.mon.a.smithi120.stdout: 2024-09-19T08:53:11.555 INFO:journalctl@ceph.mon.a.smithi120.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T08:53:11.555 INFO:journalctl@ceph.mon.a.smithi120.stdout: 2024-09-19T08:53:11.555 INFO:journalctl@ceph.mon.a.smithi120.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T08:53:11.555 INFO:journalctl@ceph.mon.a.smithi120.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T08:53:11.555 INFO:journalctl@ceph.mon.a.smithi120.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T08:53:11.555 INFO:journalctl@ceph.mon.a.smithi120.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T08:53:11.556 INFO:journalctl@ceph.mon.a.smithi120.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T08:53:11.556 INFO:journalctl@ceph.mon.a.smithi120.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T08:53:11.556 INFO:journalctl@ceph.mon.a.smithi120.stdout: Cumulative compaction: 0.00 GB write, 31.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T08:53:11.556 INFO:journalctl@ceph.mon.a.smithi120.stdout: Interval compaction: 0.00 GB write, 31.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T08:53:11.556 INFO:journalctl@ceph.mon.a.smithi120.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T08:53:11.556 INFO:journalctl@ceph.mon.a.smithi120.stdout: Block cache BinnedLRUCache@0x55e74a4f51f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-19T08:53:11.556 INFO:journalctl@ceph.mon.a.smithi120.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-19T08:53:11.556 INFO:journalctl@ceph.mon.a.smithi120.stdout: 2024-09-19T08:53:11.556 INFO:journalctl@ceph.mon.a.smithi120.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T08:53:11.556 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: starting mon.a rank 0 at public addrs [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] at bind addrs [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T08:53:11.556 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: mon.a@-1(???) e1 preinit fsid 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T08:53:11.556 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: mon.a@-1(???).mds e1 new map 2024-09-19T08:53:11.556 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: mon.a@-1(???).mds e1 print_map 2024-09-19T08:53:11.556 INFO:journalctl@ceph.mon.a.smithi120.stdout: e1 2024-09-19T08:53:11.557 INFO:journalctl@ceph.mon.a.smithi120.stdout: btime 2024-09-19T08:53:06:386334+0000 2024-09-19T08:53:11.557 INFO:journalctl@ceph.mon.a.smithi120.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T08:53:11.557 INFO:journalctl@ceph.mon.a.smithi120.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T08:53:11.557 INFO:journalctl@ceph.mon.a.smithi120.stdout: legacy client fscid: -1 2024-09-19T08:53:11.557 INFO:journalctl@ceph.mon.a.smithi120.stdout: 2024-09-19T08:53:11.557 INFO:journalctl@ceph.mon.a.smithi120.stdout: No filesystems configured 2024-09-19T08:53:11.557 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T08:53:11.557 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T08:53:11.557 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T08:53:11.557 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T08:53:11.557 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-19T08:53:11.557 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T08:53:11.557 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: monmap epoch 1 2024-09-19T08:53:11.558 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: fsid 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T08:53:11.558 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: last_changed 2024-09-19T08:53:01.874293+0000 2024-09-19T08:53:11.558 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: created 2024-09-19T08:53:01.874293+0000 2024-09-19T08:53:11.558 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: min_mon_release 19 (squid) 2024-09-19T08:53:11.558 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: election_strategy: 1 2024-09-19T08:53:11.558 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: 0: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.a 2024-09-19T08:53:11.558 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: fsmap 2024-09-19T08:53:11.558 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T08:53:11.558 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 ceph-mon[31150]: mgrmap e1: no daemons active 2024-09-19T08:53:11.558 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 bash[31131]: 12359571ca9cfae382f0f597a6d85c8061cdf3be1bb0b9eb77f3f9aa369d9ea6 2024-09-19T08:53:11.558 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:11 smithi120 systemd[1]: Started Ceph mon.a for 507393d2-7664-11ef-bceb-c7b262605968. 2024-09-19T08:53:11.559 INFO:teuthology.orchestra.run.smithi120.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-19T08:53:12.580 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.963+0000 7f125537b640 1 Processor -- start 2024-09-19T08:53:12.580 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.963+0000 7f125537b640 1 -- start start 2024-09-19T08:53:12.580 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.963+0000 7f125537b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1250108100 0x7f125010a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:12.580 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.963+0000 7f125537b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f125007beb0 con 0x7f1250108100 2024-09-19T08:53:12.580 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.964+0000 7f124effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1250108100 0x7f125010a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:12.580 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.964+0000 7f124effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1250108100 0x7f125010a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41648/0 (socket says 172.21.15.120:41648) 2024-09-19T08:53:12.580 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.964+0000 7f124effd640 1 -- 172.21.15.120:0/3245894295 learned_addr learned my addr 172.21.15.120:0/3245894295 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:12.580 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.964+0000 7f124effd640 1 -- 172.21.15.120:0/3245894295 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125010aa30 con 0x7f1250108100 2024-09-19T08:53:12.580 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.966+0000 7f124effd640 1 --2- 172.21.15.120:0/3245894295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1250108100 0x7f125010a4f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1240009820 tx=0x7f124002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41c33737e7dbfd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:12.581 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.966+0000 7f124dffb640 1 -- 172.21.15.120:0/3245894295 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1240037070 con 0x7f1250108100 2024-09-19T08:53:12.581 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.966+0000 7f124dffb640 1 -- 172.21.15.120:0/3245894295 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1240032900 con 0x7f1250108100 2024-09-19T08:53:12.581 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.966+0000 7f124dffb640 1 -- 172.21.15.120:0/3245894295 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1240030850 con 0x7f1250108100 2024-09-19T08:53:12.581 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.967+0000 7f125537b640 1 -- 172.21.15.120:0/3245894295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1250108100 msgr2=0x7f125010a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:12.581 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.967+0000 7f125537b640 1 --2- 172.21.15.120:0/3245894295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1250108100 0x7f125010a4f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1240009820 tx=0x7f124002a270 comp rx=0 tx=0).stop 2024-09-19T08:53:12.581 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.967+0000 7f125537b640 1 -- 172.21.15.120:0/3245894295 shutdown_connections 2024-09-19T08:53:12.581 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.967+0000 7f125537b640 1 --2- 172.21.15.120:0/3245894295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1250108100 0x7f125010a4f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:12.581 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.967+0000 7f125537b640 1 -- 172.21.15.120:0/3245894295 >> 172.21.15.120:0/3245894295 conn(0x7f1250101de0 msgr2=0x7f1250104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:12.581 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.967+0000 7f125537b640 1 -- 172.21.15.120:0/3245894295 shutdown_connections 2024-09-19T08:53:12.581 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.967+0000 7f125537b640 1 -- 172.21.15.120:0/3245894295 wait complete. 2024-09-19T08:53:12.581 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.967+0000 7f125537b640 1 Processor -- start 2024-09-19T08:53:12.581 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.967+0000 7f125537b640 1 -- start start 2024-09-19T08:53:12.581 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.968+0000 7f125537b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1250108100 0x7f125019b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:12.581 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.968+0000 7f125537b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f125010ba60 con 0x7f1250108100 2024-09-19T08:53:12.582 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.968+0000 7f124effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1250108100 0x7f125019b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:12.582 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.968+0000 7f124effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1250108100 0x7f125019b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41656/0 (socket says 172.21.15.120:41656) 2024-09-19T08:53:12.582 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.968+0000 7f124effd640 1 -- 172.21.15.120:0/2754077372 learned_addr learned my addr 172.21.15.120:0/2754077372 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:12.582 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.968+0000 7f124effd640 1 -- 172.21.15.120:0/2754077372 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125019bb20 con 0x7f1250108100 2024-09-19T08:53:12.582 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.968+0000 7f124effd640 1 --2- 172.21.15.120:0/2754077372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1250108100 0x7f125019b5e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1240032ce0 tx=0x7f1240032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:12.582 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.968+0000 7f1237fff640 1 -- 172.21.15.120:0/2754077372 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1240042070 con 0x7f1250108100 2024-09-19T08:53:12.582 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.968+0000 7f1237fff640 1 -- 172.21.15.120:0/2754077372 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f12400308c0 con 0x7f1250108100 2024-09-19T08:53:12.582 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.968+0000 7f1237fff640 1 -- 172.21.15.120:0/2754077372 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1240037040 con 0x7f1250108100 2024-09-19T08:53:12.582 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.968+0000 7f125537b640 1 -- 172.21.15.120:0/2754077372 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f125019bdb0 con 0x7f1250108100 2024-09-19T08:53:12.582 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.968+0000 7f125537b640 1 -- 172.21.15.120:0/2754077372 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f125019eaa0 con 0x7f1250108100 2024-09-19T08:53:12.582 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.969+0000 7f1237fff640 1 -- 172.21.15.120:0/2754077372 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1240045070 con 0x7f1250108100 2024-09-19T08:53:12.582 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.969+0000 7f1237fff640 1 -- 172.21.15.120:0/2754077372 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1240047b80 con 0x7f1250108100 2024-09-19T08:53:12.582 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.969+0000 7f125537b640 1 -- 172.21.15.120:0/2754077372 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1250108560 con 0x7f1250108100 2024-09-19T08:53:12.582 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:11.972+0000 7f1237fff640 1 -- 172.21.15.120:0/2754077372 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f124003c720 con 0x7f1250108100 2024-09-19T08:53:12.583 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:12.028+0000 7f125537b640 1 -- 172.21.15.120:0/2754077372 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f125019ec70 con 0x7f1250108100 2024-09-19T08:53:12.583 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:12.030+0000 7f1237fff640 1 -- 172.21.15.120:0/2754077372 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f124003c900 con 0x7f1250108100 2024-09-19T08:53:12.583 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:12.030+0000 7f125537b640 1 -- 172.21.15.120:0/2754077372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1250108100 msgr2=0x7f125019b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:12.583 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:12.030+0000 7f125537b640 1 --2- 172.21.15.120:0/2754077372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1250108100 0x7f125019b5e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1240032ce0 tx=0x7f1240032d10 comp rx=0 tx=0).stop 2024-09-19T08:53:12.583 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:12.031+0000 7f125537b640 1 -- 172.21.15.120:0/2754077372 shutdown_connections 2024-09-19T08:53:12.583 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:12.031+0000 7f125537b640 1 --2- 172.21.15.120:0/2754077372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1250108100 0x7f125019b5e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:12.583 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:12.031+0000 7f125537b640 1 -- 172.21.15.120:0/2754077372 >> 172.21.15.120:0/2754077372 conn(0x7f1250101de0 msgr2=0x7f12501028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:12.583 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:12.031+0000 7f125537b640 1 -- 172.21.15.120:0/2754077372 shutdown_connections 2024-09-19T08:53:12.583 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:12.031+0000 7f125537b640 1 -- 172.21.15.120:0/2754077372 wait complete. 2024-09-19T08:53:12.583 INFO:teuthology.orchestra.run.smithi120.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-19T08:53:12.589 INFO:teuthology.orchestra.run.smithi120.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-19T08:53:12.589 INFO:teuthology.orchestra.run.smithi120.stdout:Creating mgr... 2024-09-19T08:53:12.590 INFO:teuthology.orchestra.run.smithi120.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-19T08:53:12.590 INFO:teuthology.orchestra.run.smithi120.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-19T08:53:12.809 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-507393d2-7664-11ef-bceb-c7b262605968@mgr.x 2024-09-19T08:53:12.809 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Failed to reset failed state of unit ceph-507393d2-7664-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-507393d2-7664-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-19T08:53:12.966 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-507393d2-7664-11ef-bceb-c7b262605968.target.wants/ceph-507393d2-7664-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-507393d2-7664-11ef-bceb-c7b262605968@.service. 2024-09-19T08:53:13.296 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:13 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2754077372' entity='client.admin' 2024-09-19T08:53:13.484 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T08:53:13.484 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout disabled 2024-09-19T08:53:13.491 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T08:53:13.491 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout inactive 2024-09-19T08:53:13.492 INFO:teuthology.orchestra.run.smithi120.stdout:firewalld.service is not enabled 2024-09-19T08:53:13.492 INFO:teuthology.orchestra.run.smithi120.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T08:53:13.500 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T08:53:13.500 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout disabled 2024-09-19T08:53:13.507 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T08:53:13.507 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout inactive 2024-09-19T08:53:13.507 INFO:teuthology.orchestra.run.smithi120.stdout:firewalld.service is not enabled 2024-09-19T08:53:13.507 INFO:teuthology.orchestra.run.smithi120.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-19T08:53:13.507 INFO:teuthology.orchestra.run.smithi120.stdout:Waiting for mgr to start... 2024-09-19T08:53:13.507 INFO:teuthology.orchestra.run.smithi120.stdout:Waiting for mgr... 2024-09-19T08:53:14.019 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:13 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:13.971+0000 7f577a601040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:53:14.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:14 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2595874428' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T08:53:14.324 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:14 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:14.118+0000 7f577a601040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T08:53:14.642 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:14 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:14.437+0000 7f577a601040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T08:53:14.642 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:14 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:14.498+0000 7f577a601040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T08:53:14.648 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2024-09-19T08:53:14.648 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout { 2024-09-19T08:53:14.648 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "fsid": "507393d2-7664-11ef-bceb-c7b262605968", 2024-09-19T08:53:14.648 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T08:53:14.648 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T08:53:14.648 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T08:53:14.649 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T08:53:14.649 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:53:14.649 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T08:53:14.649 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T08:53:14.649 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 0 2024-09-19T08:53:14.649 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2024-09-19T08:53:14.649 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T08:53:14.649 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T08:53:14.649 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2024-09-19T08:53:14.649 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-19T08:53:14.649 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T08:53:14.649 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:53:14.649 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T08:53:14.649 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T08:53:14.649 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:53:14.650 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T08:53:14.650 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:53:14.650 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T08:53:14.650 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T08:53:14.650 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T08:53:14.650 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T08:53:14.650 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T08:53:14.650 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T08:53:14.650 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:53:14.650 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T08:53:14.650 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T08:53:14.650 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T08:53:14.650 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T08:53:14.650 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T08:53:14.651 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T08:53:14.651 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T08:53:14.651 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T08:53:14.657 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T08:53:14.658 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:53:14.658 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T08:53:14.658 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:53:14.658 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T08:53:06:386334+0000", 2024-09-19T08:53:14.658 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T08:53:14.658 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T08:53:14.658 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:53:14.658 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T08:53:14.658 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T08:53:14.658 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T08:53:14.658 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T08:53:14.658 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T08:53:14.658 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T08:53:14.658 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T08:53:14.659 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2024-09-19T08:53:14.659 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T08:53:14.659 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:53:14.659 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T08:53:14.659 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:53:14.659 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T08:53:06.387009+0000", 2024-09-19T08:53:14.659 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T08:53:14.659 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:53:14.659 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T08:53:14.659 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout } 2024-09-19T08:53:14.659 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.950+0000 7f47aaba2640 1 Processor -- start 2024-09-19T08:53:14.659 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.950+0000 7f47aaba2640 1 -- start start 2024-09-19T08:53:14.659 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.950+0000 7f47aaba2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47a4107630 0x7f47a4107a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:14.659 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.950+0000 7f47aaba2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f47a4108000 con 0x7f47a4107630 2024-09-19T08:53:14.660 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.951+0000 7f47a893e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47a4107630 0x7f47a4107a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:14.660 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.951+0000 7f47a893e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47a4107630 0x7f47a4107a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41676/0 (socket says 172.21.15.120:41676) 2024-09-19T08:53:14.660 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.951+0000 7f47a893e640 1 -- 172.21.15.120:0/2742741137 learned_addr learned my addr 172.21.15.120:0/2742741137 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:14.660 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.951+0000 7f47a893e640 1 -- 172.21.15.120:0/2742741137 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a4108830 con 0x7f47a4107630 2024-09-19T08:53:14.660 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.952+0000 7f47a893e640 1 --2- 172.21.15.120:0/2742741137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47a4107630 0x7f47a4107a30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4798009820 tx=0x7f479802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ff50a25812eb43 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:14.660 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.952+0000 7f47a37fe640 1 -- 172.21.15.120:0/2742741137 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4798037070 con 0x7f47a4107630 2024-09-19T08:53:14.660 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.952+0000 7f47a37fe640 1 -- 172.21.15.120:0/2742741137 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4798032900 con 0x7f47a4107630 2024-09-19T08:53:14.660 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.952+0000 7f47a37fe640 1 -- 172.21.15.120:0/2742741137 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4798030850 con 0x7f47a4107630 2024-09-19T08:53:14.660 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.953+0000 7f47aaba2640 1 -- 172.21.15.120:0/2742741137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47a4107630 msgr2=0x7f47a4107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:14.660 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.953+0000 7f47aaba2640 1 --2- 172.21.15.120:0/2742741137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47a4107630 0x7f47a4107a30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4798009820 tx=0x7f479802a270 comp rx=0 tx=0).stop 2024-09-19T08:53:14.661 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.953+0000 7f47aaba2640 1 -- 172.21.15.120:0/2742741137 shutdown_connections 2024-09-19T08:53:14.661 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.953+0000 7f47aaba2640 1 --2- 172.21.15.120:0/2742741137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47a4107630 0x7f47a4107a30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:14.661 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.953+0000 7f47aaba2640 1 -- 172.21.15.120:0/2742741137 >> 172.21.15.120:0/2742741137 conn(0x7f47a4102de0 msgr2=0x7f47a4105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:14.661 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.953+0000 7f47aaba2640 1 -- 172.21.15.120:0/2742741137 shutdown_connections 2024-09-19T08:53:14.661 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.953+0000 7f47aaba2640 1 -- 172.21.15.120:0/2742741137 wait complete. 2024-09-19T08:53:14.661 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.954+0000 7f47aaba2640 1 Processor -- start 2024-09-19T08:53:14.661 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.954+0000 7f47aaba2640 1 -- start start 2024-09-19T08:53:14.661 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.954+0000 7f47aaba2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47a4107630 0x7f47a419b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:14.661 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.954+0000 7f47aaba2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f47a410bbe0 con 0x7f47a4107630 2024-09-19T08:53:14.661 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.954+0000 7f47a893e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47a4107630 0x7f47a419b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:14.661 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.954+0000 7f47a893e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47a4107630 0x7f47a419b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41688/0 (socket says 172.21.15.120:41688) 2024-09-19T08:53:14.661 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.954+0000 7f47a893e640 1 -- 172.21.15.120:0/2595874428 learned_addr learned my addr 172.21.15.120:0/2595874428 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:14.661 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.954+0000 7f47a893e640 1 -- 172.21.15.120:0/2595874428 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a419bbc0 con 0x7f47a4107630 2024-09-19T08:53:14.661 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.955+0000 7f47a893e640 1 --2- 172.21.15.120:0/2595874428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47a4107630 0x7f47a419b680 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4798009940 tx=0x7f47980326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:14.662 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.955+0000 7f47a1ffb640 1 -- 172.21.15.120:0/2595874428 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4798042070 con 0x7f47a4107630 2024-09-19T08:53:14.662 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.955+0000 7f47a1ffb640 1 -- 172.21.15.120:0/2595874428 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4798032be0 con 0x7f47a4107630 2024-09-19T08:53:14.662 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.955+0000 7f47a1ffb640 1 -- 172.21.15.120:0/2595874428 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4798037040 con 0x7f47a4107630 2024-09-19T08:53:14.662 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.955+0000 7f47aaba2640 1 -- 172.21.15.120:0/2595874428 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a419be50 con 0x7f47a4107630 2024-09-19T08:53:14.662 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.955+0000 7f47aaba2640 1 -- 172.21.15.120:0/2595874428 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a419eb40 con 0x7f47a4107630 2024-09-19T08:53:14.662 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.955+0000 7f47a1ffb640 1 -- 172.21.15.120:0/2595874428 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4798045070 con 0x7f47a4107630 2024-09-19T08:53:14.662 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.955+0000 7f47a1ffb640 1 -- 172.21.15.120:0/2595874428 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f47980463b0 con 0x7f47a4107630 2024-09-19T08:53:14.662 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.956+0000 7f478f7fe640 1 -- 172.21.15.120:0/2595874428 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4784005290 con 0x7f47a4107630 2024-09-19T08:53:14.662 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:13.958+0000 7f47a1ffb640 1 -- 172.21.15.120:0/2595874428 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f4798032d90 con 0x7f47a4107630 2024-09-19T08:53:14.663 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:14.019+0000 7f478f7fe640 1 -- 172.21.15.120:0/2595874428 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4784005850 con 0x7f47a4107630 2024-09-19T08:53:14.663 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:14.019+0000 7f47a1ffb640 1 -- 172.21.15.120:0/2595874428 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f4798047440 con 0x7f47a4107630 2024-09-19T08:53:14.663 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:14.020+0000 7f478f7fe640 1 -- 172.21.15.120:0/2595874428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47a4107630 msgr2=0x7f47a419b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:14.663 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:14.020+0000 7f478f7fe640 1 --2- 172.21.15.120:0/2595874428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47a4107630 0x7f47a419b680 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4798009940 tx=0x7f47980326e0 comp rx=0 tx=0).stop 2024-09-19T08:53:14.663 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:14.021+0000 7f478f7fe640 1 -- 172.21.15.120:0/2595874428 shutdown_connections 2024-09-19T08:53:14.663 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:14.021+0000 7f478f7fe640 1 --2- 172.21.15.120:0/2595874428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47a4107630 0x7f47a419b680 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:14.663 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:14.021+0000 7f478f7fe640 1 -- 172.21.15.120:0/2595874428 >> 172.21.15.120:0/2595874428 conn(0x7f47a4102de0 msgr2=0x7f47a4103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:14.663 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:14.021+0000 7f478f7fe640 1 -- 172.21.15.120:0/2595874428 shutdown_connections 2024-09-19T08:53:14.663 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:14.021+0000 7f478f7fe640 1 -- 172.21.15.120:0/2595874428 wait complete. 2024-09-19T08:53:14.663 INFO:teuthology.orchestra.run.smithi120.stdout:mgr not available, waiting (1/15)... 2024-09-19T08:53:15.074 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:14 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:14.687+0000 7f577a601040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T08:53:15.824 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:15 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:15.561+0000 7f577a601040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T08:53:15.824 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:15 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:15.629+0000 7f577a601040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T08:53:16.627 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:16 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:16.355+0000 7f577a601040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T08:53:16.627 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:16 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:16.476+0000 7f577a601040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T08:53:16.628 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:16 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:16.627+0000 7f577a601040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:53:16.887 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:16 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:16.694+0000 7f577a601040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T08:53:16.888 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:16 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:16.851+0000 7f577a601040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T08:53:17.155 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:16 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:16.925+0000 7f577a601040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T08:53:17.155 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:16 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:16.987+0000 7f577a601040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T08:53:17.155 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:17 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:17.155+0000 7f577a601040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T08:53:17.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:17 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3594519424' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T08:53:17.449 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:17 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:17.229+0000 7f577a601040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T08:53:17.449 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:17 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:17.300+0000 7f577a601040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T08:53:17.449 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:17 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:17.386+0000 7f577a601040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T08:53:17.449 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:17 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:17.448+0000 7f577a601040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T08:53:17.824 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:17 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:17.509+0000 7f577a601040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T08:53:17.861 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2024-09-19T08:53:17.862 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout { 2024-09-19T08:53:17.862 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "fsid": "507393d2-7664-11ef-bceb-c7b262605968", 2024-09-19T08:53:17.862 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T08:53:17.862 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T08:53:17.862 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T08:53:17.862 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T08:53:17.862 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:53:17.862 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T08:53:17.862 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T08:53:17.862 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 0 2024-09-19T08:53:17.862 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2024-09-19T08:53:17.862 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T08:53:17.863 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T08:53:17.863 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2024-09-19T08:53:17.863 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-19T08:53:17.863 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T08:53:17.863 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:53:17.863 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T08:53:17.863 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T08:53:17.863 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:53:17.863 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T08:53:17.863 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:53:17.863 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T08:53:17.863 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T08:53:17.863 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T08:53:17.864 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T08:53:17.864 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T08:53:17.864 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T08:53:17.864 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:53:17.864 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T08:53:17.864 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T08:53:17.864 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T08:53:17.864 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T08:53:17.864 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T08:53:17.864 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T08:53:17.864 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T08:53:17.864 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T08:53:17.864 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T08:53:17.865 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:53:17.865 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T08:53:17.865 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:53:17.865 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T08:53:06:386334+0000", 2024-09-19T08:53:17.865 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T08:53:17.865 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T08:53:17.865 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:53:17.865 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T08:53:17.865 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T08:53:17.865 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T08:53:17.865 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T08:53:17.865 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T08:53:17.865 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T08:53:17.866 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T08:53:17.866 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2024-09-19T08:53:17.866 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T08:53:17.866 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:53:17.866 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T08:53:17.866 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:53:17.866 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T08:53:06.387009+0000", 2024-09-19T08:53:17.866 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T08:53:17.866 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:53:17.866 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T08:53:17.866 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout } 2024-09-19T08:53:17.866 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.091+0000 7f403e978640 1 Processor -- start 2024-09-19T08:53:17.866 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.091+0000 7f403e978640 1 -- start start 2024-09-19T08:53:17.867 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.091+0000 7f403e978640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4038108090 0x7f403810a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:17.867 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.091+0000 7f403e978640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f403807beb0 con 0x7f4038108090 2024-09-19T08:53:17.867 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.092+0000 7f4037fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4038108090 0x7f403810a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:17.867 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.092+0000 7f4037fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4038108090 0x7f403810a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41702/0 (socket says 172.21.15.120:41702) 2024-09-19T08:53:17.867 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.092+0000 7f4037fff640 1 -- 172.21.15.120:0/656912770 learned_addr learned my addr 172.21.15.120:0/656912770 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:17.867 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.092+0000 7f4037fff640 1 -- 172.21.15.120:0/656912770 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403810a9c0 con 0x7f4038108090 2024-09-19T08:53:17.867 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.093+0000 7f4037fff640 1 --2- 172.21.15.120:0/656912770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4038108090 0x7f403810a480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4028009820 tx=0x7f402802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb4b1dc293f53745 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:17.867 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.093+0000 7f4036ffd640 1 -- 172.21.15.120:0/656912770 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4028037070 con 0x7f4038108090 2024-09-19T08:53:17.867 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.093+0000 7f4036ffd640 1 -- 172.21.15.120:0/656912770 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4028032900 con 0x7f4038108090 2024-09-19T08:53:17.867 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.093+0000 7f4036ffd640 1 -- 172.21.15.120:0/656912770 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4028030850 con 0x7f4038108090 2024-09-19T08:53:17.867 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.094+0000 7f403e978640 1 -- 172.21.15.120:0/656912770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4038108090 msgr2=0x7f403810a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:17.867 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.094+0000 7f403e978640 1 --2- 172.21.15.120:0/656912770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4038108090 0x7f403810a480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4028009820 tx=0x7f402802a270 comp rx=0 tx=0).stop 2024-09-19T08:53:17.867 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.094+0000 7f403e978640 1 -- 172.21.15.120:0/656912770 shutdown_connections 2024-09-19T08:53:17.868 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.094+0000 7f403e978640 1 --2- 172.21.15.120:0/656912770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4038108090 0x7f403810a480 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:17.868 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.094+0000 7f403e978640 1 -- 172.21.15.120:0/656912770 >> 172.21.15.120:0/656912770 conn(0x7f4038101db0 msgr2=0x7f40381041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:17.868 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.094+0000 7f403e978640 1 -- 172.21.15.120:0/656912770 shutdown_connections 2024-09-19T08:53:17.868 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.094+0000 7f403e978640 1 -- 172.21.15.120:0/656912770 wait complete. 2024-09-19T08:53:17.868 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.095+0000 7f403e978640 1 Processor -- start 2024-09-19T08:53:17.868 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.095+0000 7f403e978640 1 -- start start 2024-09-19T08:53:17.868 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.095+0000 7f403e978640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4038108090 0x7f403819b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:17.868 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.095+0000 7f403e978640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f403810ba60 con 0x7f4038108090 2024-09-19T08:53:17.868 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.095+0000 7f4037fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4038108090 0x7f403819b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:17.868 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.095+0000 7f4037fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4038108090 0x7f403819b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41716/0 (socket says 172.21.15.120:41716) 2024-09-19T08:53:17.868 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.095+0000 7f4037fff640 1 -- 172.21.15.120:0/3594519424 learned_addr learned my addr 172.21.15.120:0/3594519424 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:17.868 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.095+0000 7f4037fff640 1 -- 172.21.15.120:0/3594519424 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403819bb50 con 0x7f4038108090 2024-09-19T08:53:17.868 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.096+0000 7f4037fff640 1 --2- 172.21.15.120:0/3594519424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4038108090 0x7f403819b610 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4028032ce0 tx=0x7f4028032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:17.869 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.096+0000 7f40357fa640 1 -- 172.21.15.120:0/3594519424 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4028042070 con 0x7f4038108090 2024-09-19T08:53:17.869 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.096+0000 7f40357fa640 1 -- 172.21.15.120:0/3594519424 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f40280308c0 con 0x7f4038108090 2024-09-19T08:53:17.869 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.096+0000 7f40357fa640 1 -- 172.21.15.120:0/3594519424 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4028037040 con 0x7f4038108090 2024-09-19T08:53:17.869 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.096+0000 7f403e978640 1 -- 172.21.15.120:0/3594519424 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f403819bde0 con 0x7f4038108090 2024-09-19T08:53:17.869 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.096+0000 7f403e978640 1 -- 172.21.15.120:0/3594519424 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f403819ead0 con 0x7f4038108090 2024-09-19T08:53:17.869 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.096+0000 7f40357fa640 1 -- 172.21.15.120:0/3594519424 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4028045070 con 0x7f4038108090 2024-09-19T08:53:17.869 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.096+0000 7f40357fa640 1 -- 172.21.15.120:0/3594519424 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f40280463b0 con 0x7f4038108090 2024-09-19T08:53:17.869 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.097+0000 7f403e978640 1 -- 172.21.15.120:0/3594519424 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40381084f0 con 0x7f4038108090 2024-09-19T08:53:17.870 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.101+0000 7f40357fa640 1 -- 172.21.15.120:0/3594519424 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f4028047440 con 0x7f4038108090 2024-09-19T08:53:17.870 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.163+0000 7f403e978640 1 -- 172.21.15.120:0/3594519424 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f403819eca0 con 0x7f4038108090 2024-09-19T08:53:17.870 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.163+0000 7f40357fa640 1 -- 172.21.15.120:0/3594519424 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f4028047620 con 0x7f4038108090 2024-09-19T08:53:17.870 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.165+0000 7f403e978640 1 -- 172.21.15.120:0/3594519424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4038108090 msgr2=0x7f403819b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:17.870 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.165+0000 7f403e978640 1 --2- 172.21.15.120:0/3594519424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4038108090 0x7f403819b610 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4028032ce0 tx=0x7f4028032d10 comp rx=0 tx=0).stop 2024-09-19T08:53:17.870 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.165+0000 7f403e978640 1 -- 172.21.15.120:0/3594519424 shutdown_connections 2024-09-19T08:53:17.870 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.165+0000 7f403e978640 1 --2- 172.21.15.120:0/3594519424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4038108090 0x7f403819b610 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:17.870 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.165+0000 7f403e978640 1 -- 172.21.15.120:0/3594519424 >> 172.21.15.120:0/3594519424 conn(0x7f4038101db0 msgr2=0x7f40381028c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:17.870 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.165+0000 7f403e978640 1 -- 172.21.15.120:0/3594519424 shutdown_connections 2024-09-19T08:53:17.870 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:17.165+0000 7f403e978640 1 -- 172.21.15.120:0/3594519424 wait complete. 2024-09-19T08:53:17.871 INFO:teuthology.orchestra.run.smithi120.stdout:mgr not available, waiting (2/15)... 2024-09-19T08:53:18.574 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:18 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:18.302+0000 7f577a601040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T08:53:18.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:18 smithi120 ceph-mon[31150]: Activating manager daemon x 2024-09-19T08:53:18.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:18 smithi120 ceph-mon[31150]: mgrmap e2: x(active, starting, since 0.0028077s) 2024-09-19T08:53:18.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:18 smithi120 ceph-mon[31150]: from='mgr.14100 172.21.15.120:0/3900685923' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T08:53:18.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:18 smithi120 ceph-mon[31150]: from='mgr.14100 172.21.15.120:0/3900685923' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T08:53:18.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:18 smithi120 ceph-mon[31150]: from='mgr.14100 172.21.15.120:0/3900685923' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T08:53:18.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:18 smithi120 ceph-mon[31150]: from='mgr.14100 172.21.15.120:0/3900685923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:53:18.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:18 smithi120 ceph-mon[31150]: from='mgr.14100 172.21.15.120:0/3900685923' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T08:53:18.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:18 smithi120 ceph-mon[31150]: Manager daemon x is now available 2024-09-19T08:53:18.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:18 smithi120 ceph-mon[31150]: from='mgr.14100 172.21.15.120:0/3900685923' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T08:53:18.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:18 smithi120 ceph-mon[31150]: from='mgr.14100 172.21.15.120:0/3900685923' entity='mgr.x' 2024-09-19T08:53:18.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:18 smithi120 ceph-mon[31150]: from='mgr.14100 172.21.15.120:0/3900685923' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T08:53:18.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:18 smithi120 ceph-mon[31150]: from='mgr.14100 172.21.15.120:0/3900685923' entity='mgr.x' 2024-09-19T08:53:18.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:18 smithi120 ceph-mon[31150]: from='mgr.14100 172.21.15.120:0/3900685923' entity='mgr.x' 2024-09-19T08:53:20.539 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:20 smithi120 ceph-mon[31150]: mgrmap e3: x(active, since 1.00739s) 2024-09-19T08:53:21.073 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2024-09-19T08:53:21.084 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout { 2024-09-19T08:53:21.085 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "fsid": "507393d2-7664-11ef-bceb-c7b262605968", 2024-09-19T08:53:21.085 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T08:53:21.085 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T08:53:21.085 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T08:53:21.085 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T08:53:21.085 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:53:21.085 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T08:53:21.085 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T08:53:21.085 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 0 2024-09-19T08:53:21.085 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2024-09-19T08:53:21.085 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T08:53:21.085 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T08:53:21.086 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2024-09-19T08:53:21.086 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-19T08:53:21.086 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T08:53:21.086 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:53:21.086 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T08:53:21.086 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T08:53:21.086 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:53:21.086 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T08:53:21.086 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:53:21.086 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T08:53:21.086 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T08:53:21.086 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T08:53:21.086 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T08:53:21.087 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T08:53:21.087 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T08:53:21.087 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:53:21.087 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T08:53:21.087 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T08:53:21.087 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T08:53:21.087 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T08:53:21.087 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T08:53:21.087 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T08:53:21.087 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T08:53:21.087 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T08:53:21.087 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T08:53:21.087 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:53:21.087 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T08:53:21.088 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:53:21.088 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T08:53:06:386334+0000", 2024-09-19T08:53:21.088 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T08:53:21.088 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T08:53:21.088 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:53:21.088 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T08:53:21.088 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T08:53:21.088 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T08:53:21.088 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T08:53:21.088 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T08:53:21.088 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T08:53:21.088 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T08:53:21.088 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2024-09-19T08:53:21.088 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T08:53:21.089 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:53:21.089 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T08:53:21.089 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:53:21.089 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T08:53:06.387009+0000", 2024-09-19T08:53:21.089 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T08:53:21.089 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:53:21.089 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T08:53:21.089 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout } 2024-09-19T08:53:21.089 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.286+0000 7fc762cf3640 1 Processor -- start 2024-09-19T08:53:21.089 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.286+0000 7fc762cf3640 1 -- start start 2024-09-19T08:53:21.089 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.286+0000 7fc762cf3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc75c105f00 0x7fc75c106300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:21.089 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.286+0000 7fc762cf3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c07be30 con 0x7fc75c105f00 2024-09-19T08:53:21.089 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.287+0000 7fc760a8f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc75c105f00 0x7fc75c106300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:21.089 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.287+0000 7fc760a8f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc75c105f00 0x7fc75c106300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41780/0 (socket says 172.21.15.120:41780) 2024-09-19T08:53:21.090 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.287+0000 7fc760a8f640 1 -- 172.21.15.120:0/2540583511 learned_addr learned my addr 172.21.15.120:0/2540583511 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:21.090 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.287+0000 7fc760a8f640 1 -- 172.21.15.120:0/2540583511 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75c106840 con 0x7fc75c105f00 2024-09-19T08:53:21.090 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.288+0000 7fc760a8f640 1 --2- 172.21.15.120:0/2540583511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc75c105f00 0x7fc75c106300 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009e80 tx=0x7fc74c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a66c384b7f4b6a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:21.090 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.288+0000 7fc75b7fe640 1 -- 172.21.15.120:0/2540583511 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc74c036070 con 0x7fc75c105f00 2024-09-19T08:53:21.090 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.288+0000 7fc75b7fe640 1 -- 172.21.15.120:0/2540583511 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc74c0318c0 con 0x7fc75c105f00 2024-09-19T08:53:21.090 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.288+0000 7fc75b7fe640 1 -- 172.21.15.120:0/2540583511 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc74c007ab0 con 0x7fc75c105f00 2024-09-19T08:53:21.090 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.288+0000 7fc762cf3640 1 -- 172.21.15.120:0/2540583511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc75c105f00 msgr2=0x7fc75c106300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:21.090 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.288+0000 7fc762cf3640 1 --2- 172.21.15.120:0/2540583511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc75c105f00 0x7fc75c106300 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009e80 tx=0x7fc74c02a7a0 comp rx=0 tx=0).stop 2024-09-19T08:53:21.090 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.289+0000 7fc762cf3640 1 -- 172.21.15.120:0/2540583511 shutdown_connections 2024-09-19T08:53:21.090 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.289+0000 7fc762cf3640 1 --2- 172.21.15.120:0/2540583511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc75c105f00 0x7fc75c106300 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:21.090 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.289+0000 7fc762cf3640 1 -- 172.21.15.120:0/2540583511 >> 172.21.15.120:0/2540583511 conn(0x7fc75c101db0 msgr2=0x7fc75c1041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:21.090 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.289+0000 7fc762cf3640 1 -- 172.21.15.120:0/2540583511 shutdown_connections 2024-09-19T08:53:21.090 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.289+0000 7fc762cf3640 1 -- 172.21.15.120:0/2540583511 wait complete. 2024-09-19T08:53:21.090 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.289+0000 7fc762cf3640 1 Processor -- start 2024-09-19T08:53:21.091 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.289+0000 7fc762cf3640 1 -- start start 2024-09-19T08:53:21.091 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.289+0000 7fc762cf3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc75c105f00 0x7fc75c19d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:21.091 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.289+0000 7fc762cf3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c107940 con 0x7fc75c105f00 2024-09-19T08:53:21.091 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.290+0000 7fc760a8f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc75c105f00 0x7fc75c19d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:21.091 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.290+0000 7fc760a8f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc75c105f00 0x7fc75c19d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41782/0 (socket says 172.21.15.120:41782) 2024-09-19T08:53:21.091 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.290+0000 7fc760a8f640 1 -- 172.21.15.120:0/2321183304 learned_addr learned my addr 172.21.15.120:0/2321183304 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:21.091 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.290+0000 7fc760a8f640 1 -- 172.21.15.120:0/2321183304 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75c19dd00 con 0x7fc75c105f00 2024-09-19T08:53:21.091 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.290+0000 7fc760a8f640 1 --2- 172.21.15.120:0/2321183304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc75c105f00 0x7fc75c19d7c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc74c031ca0 tx=0x7fc74c031cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:21.091 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.290+0000 7fc759ffb640 1 -- 172.21.15.120:0/2321183304 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc74c036050 con 0x7fc75c105f00 2024-09-19T08:53:21.091 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.290+0000 7fc759ffb640 1 -- 172.21.15.120:0/2321183304 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc74c042070 con 0x7fc75c105f00 2024-09-19T08:53:21.091 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.290+0000 7fc759ffb640 1 -- 172.21.15.120:0/2321183304 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc74c03d820 con 0x7fc75c105f00 2024-09-19T08:53:21.091 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.290+0000 7fc762cf3640 1 -- 172.21.15.120:0/2321183304 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc75c19df90 con 0x7fc75c105f00 2024-09-19T08:53:21.091 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.290+0000 7fc762cf3640 1 -- 172.21.15.120:0/2321183304 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc75c1a0c80 con 0x7fc75c105f00 2024-09-19T08:53:21.091 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.291+0000 7fc759ffb640 1 -- 172.21.15.120:0/2321183304 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 3) ==== 46344+0+0 (secure 0 0 0) 0x7fc74c03b610 con 0x7fc75c105f00 2024-09-19T08:53:21.092 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.291+0000 7fc759ffb640 1 --2- 172.21.15.120:0/2321183304 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7fc7440390a0 0x7fc74403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:21.092 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.291+0000 7fc762cf3640 1 -- 172.21.15.120:0/2321183304 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc75c106300 con 0x7fc75c105f00 2024-09-19T08:53:21.092 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.292+0000 7fc759ffb640 1 -- 172.21.15.120:0/2321183304 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc74c007e00 con 0x7fc75c105f00 2024-09-19T08:53:21.092 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.292+0000 7fc75bfff640 1 --2- 172.21.15.120:0/2321183304 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7fc7440390a0 0x7fc74403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:21.092 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.292+0000 7fc75bfff640 1 --2- 172.21.15.120:0/2321183304 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7fc7440390a0 0x7fc74403b560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc7500098c0 tx=0x7fc750006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:21.092 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.297+0000 7fc759ffb640 1 -- 172.21.15.120:0/2321183304 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc74c042220 con 0x7fc75c105f00 2024-09-19T08:53:21.092 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.308+0000 7fc759ffb640 1 -- 172.21.15.120:0/2321183304 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7fc74c06bf10 con 0x7fc75c105f00 2024-09-19T08:53:21.092 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.528+0000 7fc762cf3640 1 -- 172.21.15.120:0/2321183304 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc75c196c30 con 0x7fc75c105f00 2024-09-19T08:53:21.092 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.529+0000 7fc759ffb640 1 -- 172.21.15.120:0/2321183304 <== mon.0 v2:172.21.15.120:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7fc74c0072f0 con 0x7fc75c105f00 2024-09-19T08:53:21.092 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.531+0000 7fc762cf3640 1 -- 172.21.15.120:0/2321183304 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7fc7440390a0 msgr2=0x7fc74403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:21.092 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.531+0000 7fc762cf3640 1 --2- 172.21.15.120:0/2321183304 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7fc7440390a0 0x7fc74403b560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc7500098c0 tx=0x7fc750006db0 comp rx=0 tx=0).stop 2024-09-19T08:53:21.092 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.531+0000 7fc762cf3640 1 -- 172.21.15.120:0/2321183304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc75c105f00 msgr2=0x7fc75c19d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:21.093 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.531+0000 7fc762cf3640 1 --2- 172.21.15.120:0/2321183304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc75c105f00 0x7fc75c19d7c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc74c031ca0 tx=0x7fc74c031cd0 comp rx=0 tx=0).stop 2024-09-19T08:53:21.093 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.531+0000 7fc762cf3640 1 -- 172.21.15.120:0/2321183304 shutdown_connections 2024-09-19T08:53:21.093 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.531+0000 7fc762cf3640 1 --2- 172.21.15.120:0/2321183304 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7fc7440390a0 0x7fc74403b560 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:21.093 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.531+0000 7fc762cf3640 1 --2- 172.21.15.120:0/2321183304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc75c105f00 0x7fc75c19d7c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:21.093 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.531+0000 7fc762cf3640 1 -- 172.21.15.120:0/2321183304 >> 172.21.15.120:0/2321183304 conn(0x7fc75c101db0 msgr2=0x7fc75c191bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:21.093 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.531+0000 7fc762cf3640 1 -- 172.21.15.120:0/2321183304 shutdown_connections 2024-09-19T08:53:21.093 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:20.531+0000 7fc762cf3640 1 -- 172.21.15.120:0/2321183304 wait complete. 2024-09-19T08:53:21.093 INFO:teuthology.orchestra.run.smithi120.stdout:mgr is available 2024-09-19T08:53:21.490 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:21 smithi120 ceph-mon[31150]: mgrmap e4: x(active, since 2s) 2024-09-19T08:53:21.491 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:21 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2321183304' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T08:53:22.222 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2024-09-19T08:53:22.222 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T08:53:22.223 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout fsid = 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T08:53:22.223 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.120:3300,v1:172.21.15.120:6789] 2024-09-19T08:53:22.223 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T08:53:22.223 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T08:53:22.223 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T08:53:22.223 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T08:53:22.223 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2024-09-19T08:53:22.223 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T08:53:22.229 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T08:53:22.229 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2024-09-19T08:53:22.229 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T08:53:22.229 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T08:53:22.229 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T08:53:22.230 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T08:53:22.230 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T08:53:22.230 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T08:53:22.230 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.495+0000 7efe79c15640 1 Processor -- start 2024-09-19T08:53:22.230 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.496+0000 7efe79c15640 1 -- start start 2024-09-19T08:53:22.230 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.496+0000 7efe79c15640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe74107680 0x7efe74107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:22.230 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.496+0000 7efe79c15640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efe74108050 con 0x7efe74107680 2024-09-19T08:53:22.230 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.496+0000 7efe737fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe74107680 0x7efe74107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:22.230 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.496+0000 7efe737fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe74107680 0x7efe74107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38614/0 (socket says 172.21.15.120:38614) 2024-09-19T08:53:22.230 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.496+0000 7efe737fe640 1 -- 172.21.15.120:0/458880503 learned_addr learned my addr 172.21.15.120:0/458880503 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:22.230 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.496+0000 7efe737fe640 1 -- 172.21.15.120:0/458880503 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe74108830 con 0x7efe74107680 2024-09-19T08:53:22.230 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.497+0000 7efe737fe640 1 --2- 172.21.15.120:0/458880503 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe74107680 0x7efe74107a80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efe64009820 tx=0x7efe6402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7807278c5b7cb65 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:22.231 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.497+0000 7efe727fc640 1 -- 172.21.15.120:0/458880503 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe64037070 con 0x7efe74107680 2024-09-19T08:53:22.231 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.497+0000 7efe727fc640 1 -- 172.21.15.120:0/458880503 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efe64032900 con 0x7efe74107680 2024-09-19T08:53:22.231 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.497+0000 7efe727fc640 1 -- 172.21.15.120:0/458880503 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe64030850 con 0x7efe74107680 2024-09-19T08:53:22.231 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.498+0000 7efe79c15640 1 -- 172.21.15.120:0/458880503 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe74107680 msgr2=0x7efe74107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:22.231 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.498+0000 7efe79c15640 1 --2- 172.21.15.120:0/458880503 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe74107680 0x7efe74107a80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efe64009820 tx=0x7efe6402a270 comp rx=0 tx=0).stop 2024-09-19T08:53:22.231 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.498+0000 7efe79c15640 1 -- 172.21.15.120:0/458880503 shutdown_connections 2024-09-19T08:53:22.231 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.498+0000 7efe79c15640 1 --2- 172.21.15.120:0/458880503 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe74107680 0x7efe74107a80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:22.231 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.498+0000 7efe79c15640 1 -- 172.21.15.120:0/458880503 >> 172.21.15.120:0/458880503 conn(0x7efe74102e50 msgr2=0x7efe74105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:22.231 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.498+0000 7efe79c15640 1 -- 172.21.15.120:0/458880503 shutdown_connections 2024-09-19T08:53:22.231 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.498+0000 7efe79c15640 1 -- 172.21.15.120:0/458880503 wait complete. 2024-09-19T08:53:22.231 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.498+0000 7efe79c15640 1 Processor -- start 2024-09-19T08:53:22.231 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.499+0000 7efe79c15640 1 -- start start 2024-09-19T08:53:22.231 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.499+0000 7efe79c15640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe74107680 0x7efe7419b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:22.231 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.499+0000 7efe79c15640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efe7410bbe0 con 0x7efe74107680 2024-09-19T08:53:22.232 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.499+0000 7efe737fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe74107680 0x7efe7419b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:22.232 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.499+0000 7efe737fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe74107680 0x7efe7419b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38616/0 (socket says 172.21.15.120:38616) 2024-09-19T08:53:22.232 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.499+0000 7efe737fe640 1 -- 172.21.15.120:0/1160301098 learned_addr learned my addr 172.21.15.120:0/1160301098 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:22.232 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.499+0000 7efe737fe640 1 -- 172.21.15.120:0/1160301098 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe7419bbc0 con 0x7efe74107680 2024-09-19T08:53:22.232 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.499+0000 7efe737fe640 1 --2- 172.21.15.120:0/1160301098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe74107680 0x7efe7419b680 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efe640097f0 tx=0x7efe64030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:22.232 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.499+0000 7efe70ff9640 1 -- 172.21.15.120:0/1160301098 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe64042070 con 0x7efe74107680 2024-09-19T08:53:22.232 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.499+0000 7efe70ff9640 1 -- 172.21.15.120:0/1160301098 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efe64032ca0 con 0x7efe74107680 2024-09-19T08:53:22.232 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.499+0000 7efe79c15640 1 -- 172.21.15.120:0/1160301098 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe7419be50 con 0x7efe74107680 2024-09-19T08:53:22.232 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.500+0000 7efe70ff9640 1 -- 172.21.15.120:0/1160301098 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe64037040 con 0x7efe74107680 2024-09-19T08:53:22.232 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.500+0000 7efe79c15640 1 -- 172.21.15.120:0/1160301098 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe7419eb40 con 0x7efe74107680 2024-09-19T08:53:22.232 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.500+0000 7efe70ff9640 1 -- 172.21.15.120:0/1160301098 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7efe64045070 con 0x7efe74107680 2024-09-19T08:53:22.232 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.500+0000 7efe79c15640 1 -- 172.21.15.120:0/1160301098 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe7410bd60 con 0x7efe74107680 2024-09-19T08:53:22.232 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.501+0000 7efe70ff9640 1 --2- 172.21.15.120:0/1160301098 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7efe600391b0 0x7efe6003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:22.232 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.501+0000 7efe70ff9640 1 -- 172.21.15.120:0/1160301098 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efe6406c0b0 con 0x7efe74107680 2024-09-19T08:53:22.233 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.501+0000 7efe72ffd640 1 --2- 172.21.15.120:0/1160301098 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7efe600391b0 0x7efe6003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:22.233 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.501+0000 7efe72ffd640 1 --2- 172.21.15.120:0/1160301098 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7efe600391b0 0x7efe6003b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efe5c0098c0 tx=0x7efe5c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:22.233 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.506+0000 7efe70ff9640 1 -- 172.21.15.120:0/1160301098 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efe64045520 con 0x7efe74107680 2024-09-19T08:53:22.233 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.672+0000 7efe79c15640 1 -- 172.21.15.120:0/1160301098 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7efe74107b00 con 0x7efe74107680 2024-09-19T08:53:22.233 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.673+0000 7efe70ff9640 1 -- 172.21.15.120:0/1160301098 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+431 (secure 0 0 0) 0x7efe6403add0 con 0x7efe74107680 2024-09-19T08:53:22.233 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.675+0000 7efe79c15640 1 -- 172.21.15.120:0/1160301098 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7efe600391b0 msgr2=0x7efe6003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:22.233 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.675+0000 7efe79c15640 1 --2- 172.21.15.120:0/1160301098 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7efe600391b0 0x7efe6003b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efe5c0098c0 tx=0x7efe5c006db0 comp rx=0 tx=0).stop 2024-09-19T08:53:22.233 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.675+0000 7efe79c15640 1 -- 172.21.15.120:0/1160301098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe74107680 msgr2=0x7efe7419b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:22.233 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.675+0000 7efe79c15640 1 --2- 172.21.15.120:0/1160301098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe74107680 0x7efe7419b680 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efe640097f0 tx=0x7efe64030ec0 comp rx=0 tx=0).stop 2024-09-19T08:53:22.233 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.675+0000 7efe79c15640 1 -- 172.21.15.120:0/1160301098 shutdown_connections 2024-09-19T08:53:22.233 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.675+0000 7efe79c15640 1 --2- 172.21.15.120:0/1160301098 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7efe600391b0 0x7efe6003b670 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:22.233 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.675+0000 7efe79c15640 1 --2- 172.21.15.120:0/1160301098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe74107680 0x7efe7419b680 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:22.233 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.675+0000 7efe79c15640 1 -- 172.21.15.120:0/1160301098 >> 172.21.15.120:0/1160301098 conn(0x7efe74102e50 msgr2=0x7efe74103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:22.234 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.675+0000 7efe79c15640 1 -- 172.21.15.120:0/1160301098 shutdown_connections 2024-09-19T08:53:22.234 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:21.675+0000 7efe79c15640 1 -- 172.21.15.120:0/1160301098 wait complete. 2024-09-19T08:53:22.234 INFO:teuthology.orchestra.run.smithi120.stdout:Enabling cephadm module... 2024-09-19T08:53:22.537 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:22 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1160301098' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T08:53:23.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:23 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/898458412' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-19T08:53:23.574 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:23 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: ignoring --setuser ceph since I am not root 2024-09-19T08:53:23.574 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:23 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: ignoring --setgroup ceph since I am not root 2024-09-19T08:53:23.575 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:23 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:23.454+0000 7f8e5682a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T08:53:23.575 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:23 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:23.511+0000 7f8e5682a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T08:53:23.916 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:23 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:23.659+0000 7f8e5682a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T08:53:23.916 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:23 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:23.726+0000 7f8e5682a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T08:53:23.917 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:23 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:23.874+0000 7f8e5682a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:53:23.924 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.615+0000 7f572b432640 1 Processor -- start 2024-09-19T08:53:23.924 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.615+0000 7f572b432640 1 -- start start 2024-09-19T08:53:23.924 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.616+0000 7f572b432640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5724107680 0x7f5724107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:23.924 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.616+0000 7f572b432640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5724108050 con 0x7f5724107680 2024-09-19T08:53:23.924 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.616+0000 7f57291ce640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5724107680 0x7f5724107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:23.924 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.616+0000 7f57291ce640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5724107680 0x7f5724107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38628/0 (socket says 172.21.15.120:38628) 2024-09-19T08:53:23.924 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.616+0000 7f57291ce640 1 -- 172.21.15.120:0/2886786064 learned_addr learned my addr 172.21.15.120:0/2886786064 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:23.924 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.616+0000 7f57291ce640 1 -- 172.21.15.120:0/2886786064 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5724108830 con 0x7f5724107680 2024-09-19T08:53:23.924 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.617+0000 7f57291ce640 1 --2- 172.21.15.120:0/2886786064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5724107680 0x7f5724107a80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5720009820 tx=0x7f572002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a40e06d8999a734 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:23.924 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.617+0000 7f5717fff640 1 -- 172.21.15.120:0/2886786064 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5720037070 con 0x7f5724107680 2024-09-19T08:53:23.924 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.617+0000 7f5717fff640 1 -- 172.21.15.120:0/2886786064 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5720032900 con 0x7f5724107680 2024-09-19T08:53:23.924 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.617+0000 7f5717fff640 1 -- 172.21.15.120:0/2886786064 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5720030850 con 0x7f5724107680 2024-09-19T08:53:23.925 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.618+0000 7f572b432640 1 -- 172.21.15.120:0/2886786064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5724107680 msgr2=0x7f5724107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:23.925 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.618+0000 7f572b432640 1 --2- 172.21.15.120:0/2886786064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5724107680 0x7f5724107a80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5720009820 tx=0x7f572002a270 comp rx=0 tx=0).stop 2024-09-19T08:53:23.925 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.618+0000 7f572b432640 1 -- 172.21.15.120:0/2886786064 shutdown_connections 2024-09-19T08:53:23.925 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.618+0000 7f572b432640 1 --2- 172.21.15.120:0/2886786064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5724107680 0x7f5724107a80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.925 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.618+0000 7f572b432640 1 -- 172.21.15.120:0/2886786064 >> 172.21.15.120:0/2886786064 conn(0x7f5724102e50 msgr2=0x7f5724105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:23.925 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.618+0000 7f572b432640 1 -- 172.21.15.120:0/2886786064 shutdown_connections 2024-09-19T08:53:23.925 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.618+0000 7f572b432640 1 -- 172.21.15.120:0/2886786064 wait complete. 2024-09-19T08:53:23.925 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.618+0000 7f572b432640 1 Processor -- start 2024-09-19T08:53:23.925 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.618+0000 7f572b432640 1 -- start start 2024-09-19T08:53:23.925 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.619+0000 7f572b432640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5724107680 0x7f572419b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:23.925 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.619+0000 7f572b432640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f572410bbe0 con 0x7f5724107680 2024-09-19T08:53:23.925 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.619+0000 7f57291ce640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5724107680 0x7f572419b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:23.925 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.619+0000 7f57291ce640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5724107680 0x7f572419b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38642/0 (socket says 172.21.15.120:38642) 2024-09-19T08:53:23.926 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.619+0000 7f57291ce640 1 -- 172.21.15.120:0/898458412 learned_addr learned my addr 172.21.15.120:0/898458412 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:23.926 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.619+0000 7f57291ce640 1 -- 172.21.15.120:0/898458412 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f572419bbc0 con 0x7f5724107680 2024-09-19T08:53:23.926 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.619+0000 7f57291ce640 1 --2- 172.21.15.120:0/898458412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5724107680 0x7f572419b680 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5720009940 tx=0x7f57200326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:23.926 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.619+0000 7f57167fc640 1 -- 172.21.15.120:0/898458412 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5720042070 con 0x7f5724107680 2024-09-19T08:53:23.926 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.619+0000 7f57167fc640 1 -- 172.21.15.120:0/898458412 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5720032be0 con 0x7f5724107680 2024-09-19T08:53:23.926 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.619+0000 7f572b432640 1 -- 172.21.15.120:0/898458412 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f572419be50 con 0x7f5724107680 2024-09-19T08:53:23.926 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.619+0000 7f57167fc640 1 -- 172.21.15.120:0/898458412 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5720037040 con 0x7f5724107680 2024-09-19T08:53:23.926 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.619+0000 7f572b432640 1 -- 172.21.15.120:0/898458412 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f572419eb40 con 0x7f5724107680 2024-09-19T08:53:23.926 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.620+0000 7f57167fc640 1 -- 172.21.15.120:0/898458412 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f5720045070 con 0x7f5724107680 2024-09-19T08:53:23.926 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.620+0000 7f572b432640 1 -- 172.21.15.120:0/898458412 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f572410bd60 con 0x7f5724107680 2024-09-19T08:53:23.926 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.620+0000 7f57167fc640 1 --2- 172.21.15.120:0/898458412 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7f5710039160 0x7f571003b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:23.926 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.620+0000 7f57167fc640 1 -- 172.21.15.120:0/898458412 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f572006c0f0 con 0x7f5724107680 2024-09-19T08:53:23.926 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.620+0000 7f57289cd640 1 --2- 172.21.15.120:0/898458412 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7f5710039160 0x7f571003b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:23.927 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.621+0000 7f57289cd640 1 --2- 172.21.15.120:0/898458412 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7f5710039160 0x7f571003b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f57180098c0 tx=0x7f5718006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:23.927 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.625+0000 7f57167fc640 1 -- 172.21.15.120:0/898458412 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f572004b170 con 0x7f5724107680 2024-09-19T08:53:23.927 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:22.838+0000 7f572b432640 1 -- 172.21.15.120:0/898458412 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f572419f1b0 con 0x7f5724107680 2024-09-19T08:53:23.927 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:23.314+0000 7f57167fc640 1 -- 172.21.15.120:0/898458412 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f5720032d90 con 0x7f5724107680 2024-09-19T08:53:23.927 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:23.316+0000 7f57167fc640 1 -- 172.21.15.120:0/898458412 <== mon.0 v2:172.21.15.120:3300/0 8 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f5720031ca0 con 0x7f5724107680 2024-09-19T08:53:23.927 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:23.317+0000 7f572b432640 1 -- 172.21.15.120:0/898458412 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7f5710039160 msgr2=0x7f571003b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:23.927 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:23.317+0000 7f572b432640 1 --2- 172.21.15.120:0/898458412 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7f5710039160 0x7f571003b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f57180098c0 tx=0x7f5718006db0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.927 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:23.317+0000 7f572b432640 1 -- 172.21.15.120:0/898458412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5724107680 msgr2=0x7f572419b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:23.927 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:23.317+0000 7f572b432640 1 --2- 172.21.15.120:0/898458412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5724107680 0x7f572419b680 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5720009940 tx=0x7f57200326e0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.927 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:23.317+0000 7f572b432640 1 -- 172.21.15.120:0/898458412 shutdown_connections 2024-09-19T08:53:23.927 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:23.317+0000 7f572b432640 1 --2- 172.21.15.120:0/898458412 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7f5710039160 0x7f571003b620 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.927 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:23.317+0000 7f572b432640 1 --2- 172.21.15.120:0/898458412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5724107680 0x7f572419b680 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.927 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:23.317+0000 7f572b432640 1 -- 172.21.15.120:0/898458412 >> 172.21.15.120:0/898458412 conn(0x7f5724102e50 msgr2=0x7f5724103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:23.928 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:23.318+0000 7f572b432640 1 -- 172.21.15.120:0/898458412 shutdown_connections 2024-09-19T08:53:23.928 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:23.318+0000 7f572b432640 1 -- 172.21.15.120:0/898458412 wait complete. 2024-09-19T08:53:24.252 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:24 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:24.029+0000 7f8e5682a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T08:53:24.537 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:24 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/898458412' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-19T08:53:24.538 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:24 smithi120 ceph-mon[31150]: mgrmap e5: x(active, since 5s) 2024-09-19T08:53:24.538 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:24 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:24.353+0000 7f8e5682a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T08:53:24.538 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:24 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:24.416+0000 7f8e5682a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T08:53:24.824 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:24 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:24.609+0000 7f8e5682a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T08:53:25.107 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout { 2024-09-19T08:53:25.108 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-19T08:53:25.108 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T08:53:25.108 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T08:53:25.108 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T08:53:25.108 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout } 2024-09-19T08:53:25.108 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.332+0000 7f6815f74640 1 Processor -- start 2024-09-19T08:53:25.108 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.332+0000 7f6815f74640 1 -- start start 2024-09-19T08:53:25.108 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.332+0000 7f6815f74640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68101080c0 0x7f681010a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:25.108 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.332+0000 7f6815f74640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f681007be70 con 0x7f68101080c0 2024-09-19T08:53:25.108 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.332+0000 7f680f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68101080c0 0x7f681010a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:25.108 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.332+0000 7f680f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68101080c0 0x7f681010a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38658/0 (socket says 172.21.15.120:38658) 2024-09-19T08:53:25.109 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.332+0000 7f680f7fe640 1 -- 172.21.15.120:0/1723080275 learned_addr learned my addr 172.21.15.120:0/1723080275 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:25.109 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.333+0000 7f680f7fe640 1 -- 172.21.15.120:0/1723080275 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f681010a9f0 con 0x7f68101080c0 2024-09-19T08:53:25.109 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.333+0000 7f680f7fe640 1 --2- 172.21.15.120:0/1723080275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68101080c0 0x7f681010a4b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6800009820 tx=0x7f680002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77359f87e62477d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:25.109 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.333+0000 7f680effd640 1 -- 172.21.15.120:0/1723080275 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6800037070 con 0x7f68101080c0 2024-09-19T08:53:25.109 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.334+0000 7f680effd640 1 -- 172.21.15.120:0/1723080275 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6800032900 con 0x7f68101080c0 2024-09-19T08:53:25.109 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.334+0000 7f680effd640 1 -- 172.21.15.120:0/1723080275 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6800030850 con 0x7f68101080c0 2024-09-19T08:53:25.109 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.334+0000 7f6815f74640 1 -- 172.21.15.120:0/1723080275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68101080c0 msgr2=0x7f681010a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:25.109 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.334+0000 7f6815f74640 1 --2- 172.21.15.120:0/1723080275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68101080c0 0x7f681010a4b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6800009820 tx=0x7f680002a270 comp rx=0 tx=0).stop 2024-09-19T08:53:25.109 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.334+0000 7f6815f74640 1 -- 172.21.15.120:0/1723080275 shutdown_connections 2024-09-19T08:53:25.109 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.334+0000 7f6815f74640 1 --2- 172.21.15.120:0/1723080275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68101080c0 0x7f681010a4b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:25.109 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.334+0000 7f6815f74640 1 -- 172.21.15.120:0/1723080275 >> 172.21.15.120:0/1723080275 conn(0x7f6810101ce0 msgr2=0x7f6810104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:25.109 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.334+0000 7f6815f74640 1 -- 172.21.15.120:0/1723080275 shutdown_connections 2024-09-19T08:53:25.109 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.334+0000 7f6815f74640 1 -- 172.21.15.120:0/1723080275 wait complete. 2024-09-19T08:53:25.110 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.335+0000 7f6815f74640 1 Processor -- start 2024-09-19T08:53:25.110 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.335+0000 7f6815f74640 1 -- start start 2024-09-19T08:53:25.110 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.335+0000 7f6815f74640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68101080c0 0x7f681019b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:25.110 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.335+0000 7f6815f74640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f681010ba30 con 0x7f68101080c0 2024-09-19T08:53:25.110 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.335+0000 7f680f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68101080c0 0x7f681019b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:25.110 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.335+0000 7f680f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68101080c0 0x7f681019b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38672/0 (socket says 172.21.15.120:38672) 2024-09-19T08:53:25.110 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.335+0000 7f680f7fe640 1 -- 172.21.15.120:0/198568141 learned_addr learned my addr 172.21.15.120:0/198568141 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:25.110 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.335+0000 7f680f7fe640 1 -- 172.21.15.120:0/198568141 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f681019bb80 con 0x7f68101080c0 2024-09-19T08:53:25.110 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.336+0000 7f680f7fe640 1 --2- 172.21.15.120:0/198568141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68101080c0 0x7f681019b640 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6800031000 tx=0x7f68000326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:25.110 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.336+0000 7f680d7fa640 1 -- 172.21.15.120:0/198568141 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6800042070 con 0x7f68101080c0 2024-09-19T08:53:25.110 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.336+0000 7f680d7fa640 1 -- 172.21.15.120:0/198568141 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6800032be0 con 0x7f68101080c0 2024-09-19T08:53:25.110 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.336+0000 7f6815f74640 1 -- 172.21.15.120:0/198568141 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f681019be10 con 0x7f68101080c0 2024-09-19T08:53:25.110 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.336+0000 7f680d7fa640 1 -- 172.21.15.120:0/198568141 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6800037040 con 0x7f68101080c0 2024-09-19T08:53:25.110 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.336+0000 7f6815f74640 1 -- 172.21.15.120:0/198568141 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f681019eb00 con 0x7f68101080c0 2024-09-19T08:53:25.111 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.337+0000 7f680d7fa640 1 -- 172.21.15.120:0/198568141 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f6800045070 con 0x7f68101080c0 2024-09-19T08:53:25.111 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.337+0000 7f6815f74640 1 -- 172.21.15.120:0/198568141 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6810108520 con 0x7f68101080c0 2024-09-19T08:53:25.111 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.337+0000 7f680d7fa640 1 --2- 172.21.15.120:0/198568141 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7f67fc039200 0x7f67fc03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:25.111 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.337+0000 7f680d7fa640 1 -- 172.21.15.120:0/198568141 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f680006c1a0 con 0x7f68101080c0 2024-09-19T08:53:25.111 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.337+0000 7f6807fff640 1 -- 172.21.15.120:0/198568141 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7f67fc039200 msgr2=0x7f67fc03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.120:6800/2938446665 2024-09-19T08:53:25.111 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.337+0000 7f6807fff640 1 --2- 172.21.15.120:0/198568141 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7f67fc039200 0x7f67fc03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:53:25.111 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.342+0000 7f680d7fa640 1 -- 172.21.15.120:0/198568141 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6800042210 con 0x7f68101080c0 2024-09-19T08:53:25.111 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.538+0000 7f6807fff640 1 -- 172.21.15.120:0/198568141 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7f67fc039200 msgr2=0x7f67fc03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.120:6800/2938446665 2024-09-19T08:53:25.111 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.538+0000 7f6807fff640 1 --2- 172.21.15.120:0/198568141 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7f67fc039200 0x7f67fc03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T08:53:25.111 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.560+0000 7f6815f74640 1 -- 172.21.15.120:0/198568141 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f681019ecd0 con 0x7f68101080c0 2024-09-19T08:53:25.111 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.560+0000 7f680d7fa640 1 -- 172.21.15.120:0/198568141 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f6800032d90 con 0x7f68101080c0 2024-09-19T08:53:25.111 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.562+0000 7f6815f74640 1 -- 172.21.15.120:0/198568141 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7f67fc039200 msgr2=0x7f67fc03b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:53:25.111 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.562+0000 7f6815f74640 1 --2- 172.21.15.120:0/198568141 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7f67fc039200 0x7f67fc03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:25.111 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.562+0000 7f6815f74640 1 -- 172.21.15.120:0/198568141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68101080c0 msgr2=0x7f681019b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:25.112 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.562+0000 7f6815f74640 1 --2- 172.21.15.120:0/198568141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68101080c0 0x7f681019b640 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6800031000 tx=0x7f68000326e0 comp rx=0 tx=0).stop 2024-09-19T08:53:25.112 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.562+0000 7f6815f74640 1 -- 172.21.15.120:0/198568141 shutdown_connections 2024-09-19T08:53:25.112 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.562+0000 7f6815f74640 1 --2- 172.21.15.120:0/198568141 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7f67fc039200 0x7f67fc03b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:25.112 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.562+0000 7f6815f74640 1 --2- 172.21.15.120:0/198568141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68101080c0 0x7f681019b640 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:25.112 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.562+0000 7f6815f74640 1 -- 172.21.15.120:0/198568141 >> 172.21.15.120:0/198568141 conn(0x7f6810101ce0 msgr2=0x7f68101028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:25.112 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.562+0000 7f6815f74640 1 -- 172.21.15.120:0/198568141 shutdown_connections 2024-09-19T08:53:25.112 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:24.562+0000 7f6815f74640 1 -- 172.21.15.120:0/198568141 wait complete. 2024-09-19T08:53:25.112 INFO:teuthology.orchestra.run.smithi120.stdout:Waiting for the mgr to restart... 2024-09-19T08:53:25.112 INFO:teuthology.orchestra.run.smithi120.stdout:Waiting for mgr epoch 5... 2024-09-19T08:53:25.504 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:25 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/198568141' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T08:53:25.824 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:25 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:25.503+0000 7f8e5682a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T08:53:25.824 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:25 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:25.572+0000 7f8e5682a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T08:53:26.389 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:26 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:26.272+0000 7f8e5682a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T08:53:26.738 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:26 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:26.388+0000 7f8e5682a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T08:53:26.738 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:26 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:26.529+0000 7f8e5682a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:53:26.739 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:26 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:26.590+0000 7f8e5682a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T08:53:26.739 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:26 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:26.737+0000 7f8e5682a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T08:53:27.025 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:26 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:26.806+0000 7f8e5682a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T08:53:27.026 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:26 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:26.869+0000 7f8e5682a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T08:53:27.026 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:27 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:27.024+0000 7f8e5682a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T08:53:27.296 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:27 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:27.089+0000 7f8e5682a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T08:53:27.297 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:27 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:27.155+0000 7f8e5682a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T08:53:27.297 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:27 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:27.236+0000 7f8e5682a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T08:53:27.297 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:27 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:27.294+0000 7f8e5682a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T08:53:27.574 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:27 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:27.352+0000 7f8e5682a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T08:53:28.574 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:28 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:28.141+0000 7f8e5682a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T08:53:28.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:28 smithi120 ceph-mon[31150]: Active manager daemon x restarted 2024-09-19T08:53:28.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:28 smithi120 ceph-mon[31150]: Activating manager daemon x 2024-09-19T08:53:28.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:28 smithi120 ceph-mon[31150]: osdmap e2: 0 total, 0 up, 0 in 2024-09-19T08:53:28.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:28 smithi120 ceph-mon[31150]: mgrmap e6: x(active, starting, since 0.00320267s) 2024-09-19T08:53:28.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:28 smithi120 ceph-mon[31150]: from='mgr.14118 172.21.15.120:0/3236336334' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:53:28.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:28 smithi120 ceph-mon[31150]: from='mgr.14118 172.21.15.120:0/3236336334' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T08:53:28.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:28 smithi120 ceph-mon[31150]: from='mgr.14118 172.21.15.120:0/3236336334' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T08:53:28.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:28 smithi120 ceph-mon[31150]: from='mgr.14118 172.21.15.120:0/3236336334' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T08:53:28.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:28 smithi120 ceph-mon[31150]: from='mgr.14118 172.21.15.120:0/3236336334' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T08:53:28.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:28 smithi120 ceph-mon[31150]: Manager daemon x is now available 2024-09-19T08:53:28.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:28 smithi120 ceph-mon[31150]: from='mgr.14118 172.21.15.120:0/3236336334' entity='mgr.x' 2024-09-19T08:53:28.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:28 smithi120 ceph-mon[31150]: from='mgr.14118 172.21.15.120:0/3236336334' entity='mgr.x' 2024-09-19T08:53:28.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:28 smithi120 ceph-mon[31150]: from='mgr.14118 172.21.15.120:0/3236336334' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:53:28.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:28 smithi120 ceph-mon[31150]: from='mgr.14118 172.21.15.120:0/3236336334' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:53:28.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:28 smithi120 ceph-mon[31150]: from='mgr.14118 172.21.15.120:0/3236336334' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T08:53:28.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:28 smithi120 ceph-mon[31150]: from='mgr.14118 172.21.15.120:0/3236336334' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T08:53:29.472 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:29 smithi120 ceph-mon[31150]: Found migration_current of "None". Setting to last migration. 2024-09-19T08:53:29.472 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:29 smithi120 ceph-mon[31150]: from='mgr.14118 172.21.15.120:0/3236336334' entity='mgr.x' 2024-09-19T08:53:29.473 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:29 smithi120 ceph-mon[31150]: from='mgr.14118 172.21.15.120:0/3236336334' entity='mgr.x' 2024-09-19T08:53:29.473 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:29 smithi120 ceph-mon[31150]: mgrmap e7: x(active, since 1.00523s) 2024-09-19T08:53:29.473 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:29 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:29.381+0000 7f8dbdb84640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:08:53:29] ENGINE Error in HTTPServer.serve 2024-09-19T08:53:29.473 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:29 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: Traceback (most recent call last): 2024-09-19T08:53:29.473 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:29 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T08:53:29.473 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:29 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: self._connections.run(self.expiration_interval) 2024-09-19T08:53:29.473 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:29 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T08:53:29.473 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:29 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: self._run(expiration_interval) 2024-09-19T08:53:29.473 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:29 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T08:53:29.474 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:29 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T08:53:29.474 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:29 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T08:53:29.474 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:29 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T08:53:29.474 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:29 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T08:53:29.474 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:29 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: s = self.context.wrap_socket( 2024-09-19T08:53:29.474 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:29 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T08:53:29.474 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:29 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: return self.sslsocket_class._create( 2024-09-19T08:53:29.474 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:29 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T08:53:29.474 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:29 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: self.do_handshake() 2024-09-19T08:53:29.474 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:29 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T08:53:29.474 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:29 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: self._sslobj.do_handshake() 2024-09-19T08:53:29.474 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:29 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T08:53:29.474 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:29 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:29.774 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout { 2024-09-19T08:53:29.775 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-19T08:53:29.775 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T08:53:29.775 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout } 2024-09-19T08:53:29.775 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.513+0000 7f57e7daf640 1 Processor -- start 2024-09-19T08:53:29.775 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.513+0000 7f57e7daf640 1 -- start start 2024-09-19T08:53:29.775 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.513+0000 7f57e7daf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57e0106510 0x7f57e0106910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:29.775 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.513+0000 7f57e7daf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f57e0106ee0 con 0x7f57e0106510 2024-09-19T08:53:29.775 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.514+0000 7f57e5b4b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57e0106510 0x7f57e0106910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:29.775 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.514+0000 7f57e5b4b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57e0106510 0x7f57e0106910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38680/0 (socket says 172.21.15.120:38680) 2024-09-19T08:53:29.775 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.514+0000 7f57e5b4b640 1 -- 172.21.15.120:0/2362813961 learned_addr learned my addr 172.21.15.120:0/2362813961 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:29.775 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.514+0000 7f57e5b4b640 1 -- 172.21.15.120:0/2362813961 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e01076c0 con 0x7f57e0106510 2024-09-19T08:53:29.775 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.515+0000 7f57e5b4b640 1 --2- 172.21.15.120:0/2362813961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57e0106510 0x7f57e0106910 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f57d4009820 tx=0x7f57d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b522e67ab80a578c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:29.776 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.515+0000 7f57e4b49640 1 -- 172.21.15.120:0/2362813961 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57d4037070 con 0x7f57e0106510 2024-09-19T08:53:29.776 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.515+0000 7f57e4b49640 1 -- 172.21.15.120:0/2362813961 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f57d4032900 con 0x7f57e0106510 2024-09-19T08:53:29.776 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.515+0000 7f57e4b49640 1 -- 172.21.15.120:0/2362813961 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57d4030850 con 0x7f57e0106510 2024-09-19T08:53:29.776 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.515+0000 7f57e7daf640 1 -- 172.21.15.120:0/2362813961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57e0106510 msgr2=0x7f57e0106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:29.776 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.515+0000 7f57e7daf640 1 --2- 172.21.15.120:0/2362813961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57e0106510 0x7f57e0106910 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f57d4009820 tx=0x7f57d402a270 comp rx=0 tx=0).stop 2024-09-19T08:53:29.776 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.515+0000 7f57e7daf640 1 -- 172.21.15.120:0/2362813961 shutdown_connections 2024-09-19T08:53:29.776 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.515+0000 7f57e7daf640 1 --2- 172.21.15.120:0/2362813961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57e0106510 0x7f57e0106910 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:29.776 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.515+0000 7f57e7daf640 1 -- 172.21.15.120:0/2362813961 >> 172.21.15.120:0/2362813961 conn(0x7f57e0101ca0 msgr2=0x7f57e01040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:29.776 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.515+0000 7f57e7daf640 1 -- 172.21.15.120:0/2362813961 shutdown_connections 2024-09-19T08:53:29.776 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.515+0000 7f57e7daf640 1 -- 172.21.15.120:0/2362813961 wait complete. 2024-09-19T08:53:29.776 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.516+0000 7f57e7daf640 1 Processor -- start 2024-09-19T08:53:29.776 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.516+0000 7f57e7daf640 1 -- start start 2024-09-19T08:53:29.776 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.516+0000 7f57e7daf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57e0106510 0x7f57e01916f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:29.777 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.516+0000 7f57e7daf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f57e010aa70 con 0x7f57e0106510 2024-09-19T08:53:29.777 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.516+0000 7f57e5b4b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57e0106510 0x7f57e01916f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:29.777 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.516+0000 7f57e5b4b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57e0106510 0x7f57e01916f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38682/0 (socket says 172.21.15.120:38682) 2024-09-19T08:53:29.777 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.516+0000 7f57e5b4b640 1 -- 172.21.15.120:0/3284289316 learned_addr learned my addr 172.21.15.120:0/3284289316 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:29.777 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.517+0000 7f57e5b4b640 1 -- 172.21.15.120:0/3284289316 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e0191c30 con 0x7f57e0106510 2024-09-19T08:53:29.777 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.517+0000 7f57e5b4b640 1 --2- 172.21.15.120:0/3284289316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57e0106510 0x7f57e01916f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f57d40097f0 tx=0x7f57d4030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:29.777 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.517+0000 7f57d2ffd640 1 -- 172.21.15.120:0/3284289316 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57d4037040 con 0x7f57e0106510 2024-09-19T08:53:29.777 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.517+0000 7f57d2ffd640 1 -- 172.21.15.120:0/3284289316 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f57d4040070 con 0x7f57e0106510 2024-09-19T08:53:29.777 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.517+0000 7f57d2ffd640 1 -- 172.21.15.120:0/3284289316 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57d403a390 con 0x7f57e0106510 2024-09-19T08:53:29.777 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.517+0000 7f57e7daf640 1 -- 172.21.15.120:0/3284289316 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57e018fe20 con 0x7f57e0106510 2024-09-19T08:53:29.777 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.517+0000 7f57e7daf640 1 -- 172.21.15.120:0/3284289316 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57e0190300 con 0x7f57e0106510 2024-09-19T08:53:29.777 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.518+0000 7f57d2ffd640 1 -- 172.21.15.120:0/3284289316 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f57d403a5e0 con 0x7f57e0106510 2024-09-19T08:53:29.777 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.518+0000 7f57d2ffd640 1 --2- 172.21.15.120:0/3284289316 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7f57cc039200 0x7f57cc03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:29.777 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.518+0000 7f57d2ffd640 1 -- 172.21.15.120:0/3284289316 --> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f57cc03bdd0 con 0x7f57cc039200 2024-09-19T08:53:29.778 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.518+0000 7f57d2ffd640 1 -- 172.21.15.120:0/3284289316 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f57d406c530 con 0x7f57e0106510 2024-09-19T08:53:29.778 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.518+0000 7f57e534a640 1 -- 172.21.15.120:0/3284289316 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7f57cc039200 msgr2=0x7f57cc03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.120:6800/2938446665 2024-09-19T08:53:29.778 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.518+0000 7f57e534a640 1 --2- 172.21.15.120:0/3284289316 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7f57cc039200 0x7f57cc03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:53:29.778 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.719+0000 7f57e534a640 1 -- 172.21.15.120:0/3284289316 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7f57cc039200 msgr2=0x7f57cc03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.120:6800/2938446665 2024-09-19T08:53:29.778 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:25.719+0000 7f57e534a640 1 --2- 172.21.15.120:0/3284289316 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7f57cc039200 0x7f57cc03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T08:53:29.778 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:26.120+0000 7f57e534a640 1 -- 172.21.15.120:0/3284289316 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7f57cc039200 msgr2=0x7f57cc03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.120:6800/2938446665 2024-09-19T08:53:29.778 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:26.120+0000 7f57e534a640 1 --2- 172.21.15.120:0/3284289316 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7f57cc039200 0x7f57cc03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T08:53:29.778 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:26.921+0000 7f57e534a640 1 -- 172.21.15.120:0/3284289316 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7f57cc039200 msgr2=0x7f57cc03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.120:6800/2938446665 2024-09-19T08:53:29.778 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:26.921+0000 7f57e534a640 1 --2- 172.21.15.120:0/3284289316 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7f57cc039200 0x7f57cc03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T08:53:29.778 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:28.144+0000 7f57d2ffd640 1 -- 172.21.15.120:0/3284289316 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mgrmap(e 6) ==== 46228+0+0 (secure 0 0 0) 0x7f57d406b330 con 0x7f57e0106510 2024-09-19T08:53:29.778 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:28.145+0000 7f57d2ffd640 1 -- 172.21.15.120:0/3284289316 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7f57cc039200 msgr2=0x7f57cc03b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:53:29.778 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:28.145+0000 7f57d2ffd640 1 --2- 172.21.15.120:0/3284289316 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7f57cc039200 0x7f57cc03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:29.778 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:29.145+0000 7f57d2ffd640 1 -- 172.21.15.120:0/3284289316 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f57d4042d00 con 0x7f57e0106510 2024-09-19T08:53:29.778 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:29.145+0000 7f57d2ffd640 1 --2- 172.21.15.120:0/3284289316 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f57cc03cb30 0x7f57cc03ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:29.779 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:29.145+0000 7f57d2ffd640 1 -- 172.21.15.120:0/3284289316 --> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f57cc03bdd0 con 0x7f57cc03cb30 2024-09-19T08:53:29.779 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:29.146+0000 7f57e534a640 1 --2- 172.21.15.120:0/3284289316 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f57cc03cb30 0x7f57cc03ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:29.779 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:29.147+0000 7f57e534a640 1 --2- 172.21.15.120:0/3284289316 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f57cc03cb30 0x7f57cc03ef20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f57dc003980 tx=0x7f57dc0092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:29.779 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:29.147+0000 7f57d2ffd640 1 -- 172.21.15.120:0/3284289316 <== mgr.14118 v2:172.21.15.120:6800/3562882246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f57cc03bdd0 con 0x7f57cc03cb30 2024-09-19T08:53:29.779 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:29.154+0000 7f57e7daf640 1 -- 172.21.15.120:0/3284289316 --> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f57e010abf0 con 0x7f57cc03cb30 2024-09-19T08:53:29.779 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:29.154+0000 7f57d2ffd640 1 -- 172.21.15.120:0/3284289316 <== mgr.14118 v2:172.21.15.120:6800/3562882246 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f57e010abf0 con 0x7f57cc03cb30 2024-09-19T08:53:29.779 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:29.155+0000 7f57e7daf640 1 -- 172.21.15.120:0/3284289316 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f57cc03cb30 msgr2=0x7f57cc03ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:29.779 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:29.155+0000 7f57e7daf640 1 --2- 172.21.15.120:0/3284289316 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f57cc03cb30 0x7f57cc03ef20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f57dc003980 tx=0x7f57dc0092b0 comp rx=0 tx=0).stop 2024-09-19T08:53:29.779 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:29.155+0000 7f57e7daf640 1 -- 172.21.15.120:0/3284289316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57e0106510 msgr2=0x7f57e01916f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:29.779 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:29.155+0000 7f57e7daf640 1 --2- 172.21.15.120:0/3284289316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57e0106510 0x7f57e01916f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f57d40097f0 tx=0x7f57d4030d80 comp rx=0 tx=0).stop 2024-09-19T08:53:29.779 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:29.155+0000 7f57e7daf640 1 -- 172.21.15.120:0/3284289316 shutdown_connections 2024-09-19T08:53:29.779 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:29.155+0000 7f57e7daf640 1 --2- 172.21.15.120:0/3284289316 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f57cc03cb30 0x7f57cc03ef20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:29.779 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:29.155+0000 7f57e7daf640 1 --2- 172.21.15.120:0/3284289316 >> [v2:172.21.15.120:6800/2938446665,v1:172.21.15.120:6801/2938446665] conn(0x7f57cc039200 0x7f57cc03b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:29.779 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:29.155+0000 7f57e7daf640 1 --2- 172.21.15.120:0/3284289316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57e0106510 0x7f57e01916f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:29.779 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:29.155+0000 7f57e7daf640 1 -- 172.21.15.120:0/3284289316 >> 172.21.15.120:0/3284289316 conn(0x7f57e0101ca0 msgr2=0x7f57e0102850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:29.780 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:29.155+0000 7f57e7daf640 1 -- 172.21.15.120:0/3284289316 shutdown_connections 2024-09-19T08:53:29.780 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:29.155+0000 7f57e7daf640 1 -- 172.21.15.120:0/3284289316 wait complete. 2024-09-19T08:53:29.780 INFO:teuthology.orchestra.run.smithi120.stdout:mgr epoch 5 is available 2024-09-19T08:53:29.780 INFO:teuthology.orchestra.run.smithi120.stdout:Setting orchestrator backend to cephadm... 2024-09-19T08:53:30.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:30 smithi120 ceph-mon[31150]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-19T08:53:30.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:30 smithi120 ceph-mon[31150]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-19T08:53:30.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:30 smithi120 ceph-mon[31150]: [19/Sep/2024:08:53:29] ENGINE Bus STARTING 2024-09-19T08:53:30.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:30 smithi120 ceph-mon[31150]: [19/Sep/2024:08:53:29] ENGINE Serving on http://172.21.15.120:8765 2024-09-19T08:53:30.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:30 smithi120 ceph-mon[31150]: from='mgr.14118 172.21.15.120:0/3236336334' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:53:30.903 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.137+0000 7f761d4a5640 1 Processor -- start 2024-09-19T08:53:30.904 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.137+0000 7f761d4a5640 1 -- start start 2024-09-19T08:53:30.904 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.137+0000 7f761d4a5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7618107740 0x7f7618107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:30.904 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.137+0000 7f761d4a5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7618108110 con 0x7f7618107740 2024-09-19T08:53:30.904 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.137+0000 7f7616ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7618107740 0x7f7618107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:30.904 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.137+0000 7f7616ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7618107740 0x7f7618107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38766/0 (socket says 172.21.15.120:38766) 2024-09-19T08:53:30.904 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.137+0000 7f7616ffd640 1 -- 172.21.15.120:0/2456871498 learned_addr learned my addr 172.21.15.120:0/2456871498 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:30.904 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.138+0000 7f7616ffd640 1 -- 172.21.15.120:0/2456871498 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76181088a0 con 0x7f7618107740 2024-09-19T08:53:30.904 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.138+0000 7f7616ffd640 1 --2- 172.21.15.120:0/2456871498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7618107740 0x7f7618107b40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7610009e80 tx=0x7f761002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f240602f77d7e6db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:30.904 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.138+0000 7f7615ffb640 1 -- 172.21.15.120:0/2456871498 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7610036070 con 0x7f7618107740 2024-09-19T08:53:30.904 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.138+0000 7f7615ffb640 1 -- 172.21.15.120:0/2456871498 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f76100318c0 con 0x7f7618107740 2024-09-19T08:53:30.904 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.138+0000 7f7615ffb640 1 -- 172.21.15.120:0/2456871498 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7610007ab0 con 0x7f7618107740 2024-09-19T08:53:30.905 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.139+0000 7f761d4a5640 1 -- 172.21.15.120:0/2456871498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7618107740 msgr2=0x7f7618107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:30.905 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.139+0000 7f761d4a5640 1 --2- 172.21.15.120:0/2456871498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7618107740 0x7f7618107b40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7610009e80 tx=0x7f761002a7a0 comp rx=0 tx=0).stop 2024-09-19T08:53:30.905 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.139+0000 7f761d4a5640 1 -- 172.21.15.120:0/2456871498 shutdown_connections 2024-09-19T08:53:30.905 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.139+0000 7f761d4a5640 1 --2- 172.21.15.120:0/2456871498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7618107740 0x7f7618107b40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:30.905 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.139+0000 7f761d4a5640 1 -- 172.21.15.120:0/2456871498 >> 172.21.15.120:0/2456871498 conn(0x7f7618102eb0 msgr2=0x7f7618105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:30.905 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.139+0000 7f761d4a5640 1 -- 172.21.15.120:0/2456871498 shutdown_connections 2024-09-19T08:53:30.905 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.139+0000 7f761d4a5640 1 -- 172.21.15.120:0/2456871498 wait complete. 2024-09-19T08:53:30.905 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.140+0000 7f761d4a5640 1 Processor -- start 2024-09-19T08:53:30.905 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.140+0000 7f761d4a5640 1 -- start start 2024-09-19T08:53:30.905 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.140+0000 7f761d4a5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7618107740 0x7f761819b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:30.905 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.140+0000 7f761d4a5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f761810bc50 con 0x7f7618107740 2024-09-19T08:53:30.905 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.140+0000 7f7616ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7618107740 0x7f761819b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:30.906 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.140+0000 7f7616ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7618107740 0x7f761819b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38772/0 (socket says 172.21.15.120:38772) 2024-09-19T08:53:30.906 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.140+0000 7f7616ffd640 1 -- 172.21.15.120:0/3277973569 learned_addr learned my addr 172.21.15.120:0/3277973569 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:30.906 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.140+0000 7f7616ffd640 1 -- 172.21.15.120:0/3277973569 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761819bbc0 con 0x7f7618107740 2024-09-19T08:53:30.906 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.140+0000 7f7616ffd640 1 --2- 172.21.15.120:0/3277973569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7618107740 0x7f761819b680 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7610007ce0 tx=0x7f7610031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:30.906 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.140+0000 7f760ffff640 1 -- 172.21.15.120:0/3277973569 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7610036050 con 0x7f7618107740 2024-09-19T08:53:30.906 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.141+0000 7f760ffff640 1 -- 172.21.15.120:0/3277973569 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7610042070 con 0x7f7618107740 2024-09-19T08:53:30.906 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.141+0000 7f761d4a5640 1 -- 172.21.15.120:0/3277973569 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f761819be50 con 0x7f7618107740 2024-09-19T08:53:30.906 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.141+0000 7f760ffff640 1 -- 172.21.15.120:0/3277973569 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f761003d8a0 con 0x7f7618107740 2024-09-19T08:53:30.906 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.141+0000 7f761d4a5640 1 -- 172.21.15.120:0/3277973569 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f761819eb40 con 0x7f7618107740 2024-09-19T08:53:30.906 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.141+0000 7f760ffff640 1 -- 172.21.15.120:0/3277973569 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f761003dba0 con 0x7f7618107740 2024-09-19T08:53:30.906 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.142+0000 7f760ffff640 1 --2- 172.21.15.120:0/3277973569 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f76040390a0 0x7f760403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:30.906 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.142+0000 7f760ffff640 1 -- 172.21.15.120:0/3277973569 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7610007e00 con 0x7f7618107740 2024-09-19T08:53:30.906 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.142+0000 7f761d4a5640 1 -- 172.21.15.120:0/3277973569 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7618107bc0 con 0x7f7618107740 2024-09-19T08:53:30.907 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.142+0000 7f76167fc640 1 --2- 172.21.15.120:0/3277973569 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f76040390a0 0x7f760403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:30.907 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.142+0000 7f76167fc640 1 --2- 172.21.15.120:0/3277973569 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f76040390a0 0x7f760403b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f76000098c0 tx=0x7f7600006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:30.907 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.147+0000 7f760ffff640 1 -- 172.21.15.120:0/3277973569 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7610044310 con 0x7f7618107740 2024-09-19T08:53:30.907 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.206+0000 7f760ffff640 1 -- 172.21.15.120:0/3277973569 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f761003bb30 con 0x7f7618107740 2024-09-19T08:53:30.907 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.327+0000 7f761d4a5640 1 -- 172.21.15.120:0/3277973569 --> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f76181041c0 con 0x7f76040390a0 2024-09-19T08:53:30.907 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.332+0000 7f760ffff640 1 -- 172.21.15.120:0/3277973569 <== mgr.14118 v2:172.21.15.120:6800/3562882246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f76181041c0 con 0x7f76040390a0 2024-09-19T08:53:30.907 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.334+0000 7f761d4a5640 1 -- 172.21.15.120:0/3277973569 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f76040390a0 msgr2=0x7f760403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:30.907 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.334+0000 7f761d4a5640 1 --2- 172.21.15.120:0/3277973569 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f76040390a0 0x7f760403b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f76000098c0 tx=0x7f7600006db0 comp rx=0 tx=0).stop 2024-09-19T08:53:30.907 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.334+0000 7f761d4a5640 1 -- 172.21.15.120:0/3277973569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7618107740 msgr2=0x7f761819b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:30.907 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.334+0000 7f761d4a5640 1 --2- 172.21.15.120:0/3277973569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7618107740 0x7f761819b680 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7610007ce0 tx=0x7f7610031ca0 comp rx=0 tx=0).stop 2024-09-19T08:53:30.907 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.334+0000 7f761d4a5640 1 -- 172.21.15.120:0/3277973569 shutdown_connections 2024-09-19T08:53:30.907 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.334+0000 7f761d4a5640 1 --2- 172.21.15.120:0/3277973569 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f76040390a0 0x7f760403b560 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:30.907 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.334+0000 7f761d4a5640 1 --2- 172.21.15.120:0/3277973569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7618107740 0x7f761819b680 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:30.907 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.334+0000 7f761d4a5640 1 -- 172.21.15.120:0/3277973569 >> 172.21.15.120:0/3277973569 conn(0x7f7618102eb0 msgr2=0x7f7618103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:30.907 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.334+0000 7f761d4a5640 1 -- 172.21.15.120:0/3277973569 shutdown_connections 2024-09-19T08:53:30.908 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:30.334+0000 7f761d4a5640 1 -- 172.21.15.120:0/3277973569 wait complete. 2024-09-19T08:53:31.491 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:31 smithi120 ceph-mon[31150]: [19/Sep/2024:08:53:29] ENGINE Serving on https://172.21.15.120:7150 2024-09-19T08:53:31.491 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:31 smithi120 ceph-mon[31150]: [19/Sep/2024:08:53:29] ENGINE Bus STARTED 2024-09-19T08:53:31.491 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:31 smithi120 ceph-mon[31150]: [19/Sep/2024:08:53:29] ENGINE Error in HTTPServer.serve 2024-09-19T08:53:31.491 INFO:journalctl@ceph.mon.a.smithi120.stdout: Traceback (most recent call last): 2024-09-19T08:53:31.491 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T08:53:31.491 INFO:journalctl@ceph.mon.a.smithi120.stdout: self._connections.run(self.expiration_interval) 2024-09-19T08:53:31.491 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T08:53:31.491 INFO:journalctl@ceph.mon.a.smithi120.stdout: self._run(expiration_interval) 2024-09-19T08:53:31.492 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T08:53:31.492 INFO:journalctl@ceph.mon.a.smithi120.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T08:53:31.492 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T08:53:31.492 INFO:journalctl@ceph.mon.a.smithi120.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T08:53:31.492 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T08:53:31.492 INFO:journalctl@ceph.mon.a.smithi120.stdout: s = self.context.wrap_socket( 2024-09-19T08:53:31.492 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T08:53:31.492 INFO:journalctl@ceph.mon.a.smithi120.stdout: return self.sslsocket_class._create( 2024-09-19T08:53:31.492 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T08:53:31.492 INFO:journalctl@ceph.mon.a.smithi120.stdout: self.do_handshake() 2024-09-19T08:53:31.492 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T08:53:31.492 INFO:journalctl@ceph.mon.a.smithi120.stdout: self._sslobj.do_handshake() 2024-09-19T08:53:31.492 INFO:journalctl@ceph.mon.a.smithi120.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T08:53:31.492 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:31 smithi120 ceph-mon[31150]: mgrmap e8: x(active, since 2s) 2024-09-19T08:53:31.493 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:31 smithi120 ceph-mon[31150]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:53:31.493 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:31 smithi120 ceph-mon[31150]: from='mgr.14118 172.21.15.120:0/3236336334' entity='mgr.x' 2024-09-19T08:53:31.493 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:31 smithi120 ceph-mon[31150]: from='mgr.14118 172.21.15.120:0/3236336334' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:53:32.095 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-19T08:53:32.095 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.311+0000 7f3e4ff34640 1 Processor -- start 2024-09-19T08:53:32.095 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.311+0000 7f3e4ff34640 1 -- start start 2024-09-19T08:53:32.095 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.311+0000 7f3e4ff34640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e48107700 0x7f3e48107b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:32.095 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.311+0000 7f3e4ff34640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3e481080d0 con 0x7f3e48107700 2024-09-19T08:53:32.095 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.312+0000 7f3e4dcd0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e48107700 0x7f3e48107b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:32.096 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.312+0000 7f3e4dcd0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e48107700 0x7f3e48107b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41230/0 (socket says 172.21.15.120:41230) 2024-09-19T08:53:32.096 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.312+0000 7f3e4dcd0640 1 -- 172.21.15.120:0/3974923814 learned_addr learned my addr 172.21.15.120:0/3974923814 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:32.096 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.312+0000 7f3e4dcd0640 1 -- 172.21.15.120:0/3974923814 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e48108860 con 0x7f3e48107700 2024-09-19T08:53:32.096 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.312+0000 7f3e4dcd0640 1 --2- 172.21.15.120:0/3974923814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e48107700 0x7f3e48107b00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3e44009820 tx=0x7f3e4402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5230cb543a285ed5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:32.096 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.313+0000 7f3e4ccce640 1 -- 172.21.15.120:0/3974923814 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e44037070 con 0x7f3e48107700 2024-09-19T08:53:32.096 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.313+0000 7f3e4ccce640 1 -- 172.21.15.120:0/3974923814 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3e44032900 con 0x7f3e48107700 2024-09-19T08:53:32.096 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.313+0000 7f3e4ccce640 1 -- 172.21.15.120:0/3974923814 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e44030850 con 0x7f3e48107700 2024-09-19T08:53:32.096 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.313+0000 7f3e4ff34640 1 -- 172.21.15.120:0/3974923814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e48107700 msgr2=0x7f3e48107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:32.096 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.313+0000 7f3e4ff34640 1 --2- 172.21.15.120:0/3974923814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e48107700 0x7f3e48107b00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3e44009820 tx=0x7f3e4402a270 comp rx=0 tx=0).stop 2024-09-19T08:53:32.096 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.313+0000 7f3e4ff34640 1 -- 172.21.15.120:0/3974923814 shutdown_connections 2024-09-19T08:53:32.096 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.313+0000 7f3e4ff34640 1 --2- 172.21.15.120:0/3974923814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e48107700 0x7f3e48107b00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:32.096 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.313+0000 7f3e4ff34640 1 -- 172.21.15.120:0/3974923814 >> 172.21.15.120:0/3974923814 conn(0x7f3e48102eb0 msgr2=0x7f3e481052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:32.096 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.313+0000 7f3e4ff34640 1 -- 172.21.15.120:0/3974923814 shutdown_connections 2024-09-19T08:53:32.097 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.313+0000 7f3e4ff34640 1 -- 172.21.15.120:0/3974923814 wait complete. 2024-09-19T08:53:32.097 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.314+0000 7f3e4ff34640 1 Processor -- start 2024-09-19T08:53:32.097 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.314+0000 7f3e4ff34640 1 -- start start 2024-09-19T08:53:32.097 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.314+0000 7f3e4ff34640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e48107700 0x7f3e4819b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:32.097 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.314+0000 7f3e4ff34640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4810bc10 con 0x7f3e48107700 2024-09-19T08:53:32.097 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.314+0000 7f3e4dcd0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e48107700 0x7f3e4819b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:32.097 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.314+0000 7f3e4dcd0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e48107700 0x7f3e4819b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41232/0 (socket says 172.21.15.120:41232) 2024-09-19T08:53:32.097 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.314+0000 7f3e4dcd0640 1 -- 172.21.15.120:0/2385674947 learned_addr learned my addr 172.21.15.120:0/2385674947 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:32.097 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.314+0000 7f3e4dcd0640 1 -- 172.21.15.120:0/2385674947 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e4819bbf0 con 0x7f3e48107700 2024-09-19T08:53:32.097 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.315+0000 7f3e4dcd0640 1 --2- 172.21.15.120:0/2385674947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e48107700 0x7f3e4819b6b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3e44009940 tx=0x7f3e440326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:32.097 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.315+0000 7f3e3effd640 1 -- 172.21.15.120:0/2385674947 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e44042070 con 0x7f3e48107700 2024-09-19T08:53:32.097 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.315+0000 7f3e3effd640 1 -- 172.21.15.120:0/2385674947 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3e44032be0 con 0x7f3e48107700 2024-09-19T08:53:32.098 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.315+0000 7f3e3effd640 1 -- 172.21.15.120:0/2385674947 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e44037040 con 0x7f3e48107700 2024-09-19T08:53:32.098 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.315+0000 7f3e4ff34640 1 -- 172.21.15.120:0/2385674947 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e4819be80 con 0x7f3e48107700 2024-09-19T08:53:32.098 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.315+0000 7f3e4ff34640 1 -- 172.21.15.120:0/2385674947 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e4819eb70 con 0x7f3e48107700 2024-09-19T08:53:32.098 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.316+0000 7f3e4ff34640 1 -- 172.21.15.120:0/2385674947 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e4810bd90 con 0x7f3e48107700 2024-09-19T08:53:32.098 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.316+0000 7f3e3effd640 1 -- 172.21.15.120:0/2385674947 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f3e44045070 con 0x7f3e48107700 2024-09-19T08:53:32.098 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.316+0000 7f3e3effd640 1 --2- 172.21.15.120:0/2385674947 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f3e34039200 0x7f3e3403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:32.098 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.316+0000 7f3e3effd640 1 -- 172.21.15.120:0/2385674947 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3e4406c180 con 0x7f3e48107700 2024-09-19T08:53:32.098 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.316+0000 7f3e4d4cf640 1 --2- 172.21.15.120:0/2385674947 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f3e34039200 0x7f3e3403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:32.098 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.317+0000 7f3e4d4cf640 1 --2- 172.21.15.120:0/2385674947 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f3e34039200 0x7f3e3403b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3e380098c0 tx=0x7f3e38006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:32.098 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.321+0000 7f3e3effd640 1 -- 172.21.15.120:0/2385674947 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3e4404b170 con 0x7f3e48107700 2024-09-19T08:53:32.098 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.490+0000 7f3e4ff34640 1 -- 172.21.15.120:0/2385674947 --> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f3e48104180 con 0x7f3e34039200 2024-09-19T08:53:32.098 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.491+0000 7f3e3effd640 1 -- 172.21.15.120:0/2385674947 <== mgr.14118 v2:172.21.15.120:6800/3562882246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f3e48104180 con 0x7f3e34039200 2024-09-19T08:53:32.098 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.493+0000 7f3e4ff34640 1 -- 172.21.15.120:0/2385674947 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f3e34039200 msgr2=0x7f3e3403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:32.099 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.493+0000 7f3e4ff34640 1 --2- 172.21.15.120:0/2385674947 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f3e34039200 0x7f3e3403b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3e380098c0 tx=0x7f3e38006db0 comp rx=0 tx=0).stop 2024-09-19T08:53:32.099 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.493+0000 7f3e4ff34640 1 -- 172.21.15.120:0/2385674947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e48107700 msgr2=0x7f3e4819b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:32.099 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.493+0000 7f3e4ff34640 1 --2- 172.21.15.120:0/2385674947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e48107700 0x7f3e4819b6b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3e44009940 tx=0x7f3e440326e0 comp rx=0 tx=0).stop 2024-09-19T08:53:32.099 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.493+0000 7f3e4ff34640 1 -- 172.21.15.120:0/2385674947 shutdown_connections 2024-09-19T08:53:32.099 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.493+0000 7f3e4ff34640 1 --2- 172.21.15.120:0/2385674947 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f3e34039200 0x7f3e3403b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:32.099 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.493+0000 7f3e4ff34640 1 --2- 172.21.15.120:0/2385674947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e48107700 0x7f3e4819b6b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:32.099 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.493+0000 7f3e4ff34640 1 -- 172.21.15.120:0/2385674947 >> 172.21.15.120:0/2385674947 conn(0x7f3e48102eb0 msgr2=0x7f3e48103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:32.099 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.493+0000 7f3e4ff34640 1 -- 172.21.15.120:0/2385674947 shutdown_connections 2024-09-19T08:53:32.099 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:31.493+0000 7f3e4ff34640 1 -- 172.21.15.120:0/2385674947 wait complete. 2024-09-19T08:53:32.099 INFO:teuthology.orchestra.run.smithi120.stdout:Generating ssh key... 2024-09-19T08:53:32.715 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:32 smithi120 ceph-mon[31150]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:53:33.075 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:32 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: Generating public/private rsa key pair. 2024-09-19T08:53:33.075 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:32 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: Your identification has been saved in /tmp/tmp4j7msao5/key 2024-09-19T08:53:33.075 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:32 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: Your public key has been saved in /tmp/tmp4j7msao5/key.pub 2024-09-19T08:53:33.075 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:32 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: The key fingerprint is: 2024-09-19T08:53:33.075 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:32 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: SHA256:Aff72nxAL8NV6JjeqrLXLcf7yn0DX/zT3TuanwM4Mk8 ceph-507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T08:53:33.075 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:32 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: The key's randomart image is: 2024-09-19T08:53:33.075 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:32 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: +---[RSA 3072]----+ 2024-09-19T08:53:33.075 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:32 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: | . . . | 2024-09-19T08:53:33.076 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:32 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: | o . . .| 2024-09-19T08:53:33.076 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:32 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: | . . + . | 2024-09-19T08:53:33.076 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:32 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: | . .+ o | 2024-09-19T08:53:33.076 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:32 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: | S .+.+ . | 2024-09-19T08:53:33.076 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:32 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: | o E*oo o| 2024-09-19T08:53:33.076 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:32 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: | =.oB+ *| 2024-09-19T08:53:33.076 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:32 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: | . .=+.*BB| 2024-09-19T08:53:33.076 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:32 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: | .+o.o**BX| 2024-09-19T08:53:33.076 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:32 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: +----[SHA256]-----+ 2024-09-19T08:53:33.469 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.537+0000 7f63c491d640 1 Processor -- start 2024-09-19T08:53:33.469 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.538+0000 7f63c491d640 1 -- start start 2024-09-19T08:53:33.469 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.538+0000 7f63c491d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63c01080e0 0x7f63c010a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:33.469 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.538+0000 7f63c491d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f63c007be50 con 0x7f63c01080e0 2024-09-19T08:53:33.469 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.538+0000 7f63be59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63c01080e0 0x7f63c010a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:33.469 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.538+0000 7f63be59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63c01080e0 0x7f63c010a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41238/0 (socket says 172.21.15.120:41238) 2024-09-19T08:53:33.469 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.538+0000 7f63be59c640 1 -- 172.21.15.120:0/3329625494 learned_addr learned my addr 172.21.15.120:0/3329625494 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:33.469 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.538+0000 7f63be59c640 1 -- 172.21.15.120:0/3329625494 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c010aa10 con 0x7f63c01080e0 2024-09-19T08:53:33.469 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.539+0000 7f63be59c640 1 --2- 172.21.15.120:0/3329625494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63c01080e0 0x7f63c010a4d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009820 tx=0x7f63b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70fdd29c8c5d93b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:33.470 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.539+0000 7f63bdd9b640 1 -- 172.21.15.120:0/3329625494 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63b0037070 con 0x7f63c01080e0 2024-09-19T08:53:33.470 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.539+0000 7f63bdd9b640 1 -- 172.21.15.120:0/3329625494 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f63b0032900 con 0x7f63c01080e0 2024-09-19T08:53:33.470 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.539+0000 7f63bdd9b640 1 -- 172.21.15.120:0/3329625494 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63b0030850 con 0x7f63c01080e0 2024-09-19T08:53:33.470 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.540+0000 7f63c491d640 1 -- 172.21.15.120:0/3329625494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63c01080e0 msgr2=0x7f63c010a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:33.470 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.540+0000 7f63c491d640 1 --2- 172.21.15.120:0/3329625494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63c01080e0 0x7f63c010a4d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009820 tx=0x7f63b002a270 comp rx=0 tx=0).stop 2024-09-19T08:53:33.470 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.540+0000 7f63c491d640 1 -- 172.21.15.120:0/3329625494 shutdown_connections 2024-09-19T08:53:33.470 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.540+0000 7f63c491d640 1 --2- 172.21.15.120:0/3329625494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63c01080e0 0x7f63c010a4d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:33.470 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.540+0000 7f63c491d640 1 -- 172.21.15.120:0/3329625494 >> 172.21.15.120:0/3329625494 conn(0x7f63c0101d80 msgr2=0x7f63c01041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:33.470 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.540+0000 7f63c491d640 1 -- 172.21.15.120:0/3329625494 shutdown_connections 2024-09-19T08:53:33.470 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.540+0000 7f63c491d640 1 -- 172.21.15.120:0/3329625494 wait complete. 2024-09-19T08:53:33.470 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.541+0000 7f63c491d640 1 Processor -- start 2024-09-19T08:53:33.470 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.541+0000 7f63c491d640 1 -- start start 2024-09-19T08:53:33.470 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.541+0000 7f63c491d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63c01080e0 0x7f63c0107960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:33.471 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.541+0000 7f63c491d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f63c010ba50 con 0x7f63c01080e0 2024-09-19T08:53:33.471 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.541+0000 7f63be59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63c01080e0 0x7f63c0107960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:33.471 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.541+0000 7f63be59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63c01080e0 0x7f63c0107960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41252/0 (socket says 172.21.15.120:41252) 2024-09-19T08:53:33.471 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.541+0000 7f63be59c640 1 -- 172.21.15.120:0/2284924028 learned_addr learned my addr 172.21.15.120:0/2284924028 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:33.471 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.541+0000 7f63be59c640 1 -- 172.21.15.120:0/2284924028 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c0107ea0 con 0x7f63c01080e0 2024-09-19T08:53:33.471 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.541+0000 7f63be59c640 1 --2- 172.21.15.120:0/2284924028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63c01080e0 0x7f63c0107960 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f63b0032ce0 tx=0x7f63b0032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:33.471 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.541+0000 7f63b77fe640 1 -- 172.21.15.120:0/2284924028 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63b0042070 con 0x7f63c01080e0 2024-09-19T08:53:33.471 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.542+0000 7f63c491d640 1 -- 172.21.15.120:0/2284924028 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c0106090 con 0x7f63c01080e0 2024-09-19T08:53:33.471 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.542+0000 7f63b77fe640 1 -- 172.21.15.120:0/2284924028 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f63b00308c0 con 0x7f63c01080e0 2024-09-19T08:53:33.471 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.542+0000 7f63b77fe640 1 -- 172.21.15.120:0/2284924028 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63b0037040 con 0x7f63c01080e0 2024-09-19T08:53:33.471 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.542+0000 7f63c491d640 1 -- 172.21.15.120:0/2284924028 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c01065d0 con 0x7f63c01080e0 2024-09-19T08:53:33.471 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.542+0000 7f63b77fe640 1 -- 172.21.15.120:0/2284924028 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f63b0045070 con 0x7f63c01080e0 2024-09-19T08:53:33.471 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.542+0000 7f63b77fe640 1 --2- 172.21.15.120:0/2284924028 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f63ac039200 0x7f63ac03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:33.471 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.543+0000 7f63b77fe640 1 -- 172.21.15.120:0/2284924028 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f63b006c320 con 0x7f63c01080e0 2024-09-19T08:53:33.472 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.543+0000 7f63c491d640 1 -- 172.21.15.120:0/2284924028 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63c0108540 con 0x7f63c01080e0 2024-09-19T08:53:33.472 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.543+0000 7f63b7fff640 1 --2- 172.21.15.120:0/2284924028 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f63ac039200 0x7f63ac03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:33.472 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.543+0000 7f63b7fff640 1 --2- 172.21.15.120:0/2284924028 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f63ac039200 0x7f63ac03b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f63a80098c0 tx=0x7f63a8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:33.472 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.547+0000 7f63b77fe640 1 -- 172.21.15.120:0/2284924028 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f63b0045360 con 0x7f63c01080e0 2024-09-19T08:53:33.472 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.715+0000 7f63c491d640 1 -- 172.21.15.120:0/2284924028 --> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f63c0106f00 con 0x7f63ac039200 2024-09-19T08:53:33.472 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.920+0000 7f63b77fe640 1 -- 172.21.15.120:0/2284924028 <== mgr.14118 v2:172.21.15.120:6800/3562882246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f63c0106f00 con 0x7f63ac039200 2024-09-19T08:53:33.472 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.922+0000 7f63c491d640 1 -- 172.21.15.120:0/2284924028 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f63ac039200 msgr2=0x7f63ac03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:33.472 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.922+0000 7f63c491d640 1 --2- 172.21.15.120:0/2284924028 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f63ac039200 0x7f63ac03b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f63a80098c0 tx=0x7f63a8006db0 comp rx=0 tx=0).stop 2024-09-19T08:53:33.472 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.922+0000 7f63c491d640 1 -- 172.21.15.120:0/2284924028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63c01080e0 msgr2=0x7f63c0107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:33.472 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.922+0000 7f63c491d640 1 --2- 172.21.15.120:0/2284924028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63c01080e0 0x7f63c0107960 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f63b0032ce0 tx=0x7f63b0032d10 comp rx=0 tx=0).stop 2024-09-19T08:53:33.472 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.922+0000 7f63c491d640 1 -- 172.21.15.120:0/2284924028 shutdown_connections 2024-09-19T08:53:33.472 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.922+0000 7f63c491d640 1 --2- 172.21.15.120:0/2284924028 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f63ac039200 0x7f63ac03b6c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:33.472 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.922+0000 7f63c491d640 1 --2- 172.21.15.120:0/2284924028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63c01080e0 0x7f63c0107960 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:33.473 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.922+0000 7f63c491d640 1 -- 172.21.15.120:0/2284924028 >> 172.21.15.120:0/2284924028 conn(0x7f63c0101d80 msgr2=0x7f63c01028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:33.473 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.922+0000 7f63c491d640 1 -- 172.21.15.120:0/2284924028 shutdown_connections 2024-09-19T08:53:33.473 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:32.922+0000 7f63c491d640 1 -- 172.21.15.120:0/2284924028 wait complete. 2024-09-19T08:53:34.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:33 smithi120 ceph-mon[31150]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:53:34.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:33 smithi120 ceph-mon[31150]: Generating ssh key... 2024-09-19T08:53:34.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:33 smithi120 ceph-mon[31150]: from='mgr.14118 172.21.15.120:0/3236336334' entity='mgr.x' 2024-09-19T08:53:34.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:33 smithi120 ceph-mon[31150]: from='mgr.14118 172.21.15.120:0/3236336334' entity='mgr.x' 2024-09-19T08:53:34.584 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCVS0BoZyzl2d4UZYb6pmtEVyd73DYS+X46xalomhMZ4cerC1Q4l7Cmk33c9R6V/Gch2MIAHQhNER/jjEDkW+PL22CwOz4QJgzLu1Faa/0hLB557rYfA12X29w2SDQskctQEbVfUXTFPUoOS18bwC5zqkze4jIi9SsXC2qVxTPTlmc4kxPTjau7FgotPgHIAC9mrn0gI9/StdxqLWzr7+YkFKCOqqKfec11elluXZHpdm3CUhfivZ219JI1S0D4AJwmRdIHOTRfd8mxfW88w0OWIMp38dEZnAvidy8Fl/cRC0JnxUhK5NN3qWoCueJS6uExfd2E8bAoFDa6BJN9+mP3cownwirkbs03yPAOCfzwINXyMxIZIl5WBW57SCfWQTtT0KpC4cNKuzs1potGatswqL3lRjDloGGGhyLsZbbOEWkNrBIJGg3eYvh9Xg2yZIfJXEhacPFv4iQw0lh6RcldSXztTEiFEKDUTp0tXbnPfkyDmgSNlVCgYDh+3PnrOHs= ceph-507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T08:53:34.584 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.815+0000 7f2148ce0640 1 Processor -- start 2024-09-19T08:53:34.584 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.816+0000 7f2148ce0640 1 -- start start 2024-09-19T08:53:34.585 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.816+0000 7f2148ce0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2144107680 0x7f2144107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:34.585 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.816+0000 7f2148ce0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2144108050 con 0x7f2144107680 2024-09-19T08:53:34.585 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.816+0000 7f214259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2144107680 0x7f2144107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:34.585 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.816+0000 7f214259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2144107680 0x7f2144107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41258/0 (socket says 172.21.15.120:41258) 2024-09-19T08:53:34.585 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.816+0000 7f214259c640 1 -- 172.21.15.120:0/1760936289 learned_addr learned my addr 172.21.15.120:0/1760936289 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:34.585 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.816+0000 7f214259c640 1 -- 172.21.15.120:0/1760936289 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2144108830 con 0x7f2144107680 2024-09-19T08:53:34.585 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.817+0000 7f214259c640 1 --2- 172.21.15.120:0/1760936289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2144107680 0x7f2144107a80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2134009820 tx=0x7f213402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dabd2036822aa41d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:34.585 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.817+0000 7f2141d9b640 1 -- 172.21.15.120:0/1760936289 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2134037070 con 0x7f2144107680 2024-09-19T08:53:34.585 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.817+0000 7f2141d9b640 1 -- 172.21.15.120:0/1760936289 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2134032900 con 0x7f2144107680 2024-09-19T08:53:34.585 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.817+0000 7f2141d9b640 1 -- 172.21.15.120:0/1760936289 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2134030850 con 0x7f2144107680 2024-09-19T08:53:34.585 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.817+0000 7f2148ce0640 1 -- 172.21.15.120:0/1760936289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2144107680 msgr2=0x7f2144107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:34.585 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.817+0000 7f2148ce0640 1 --2- 172.21.15.120:0/1760936289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2144107680 0x7f2144107a80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2134009820 tx=0x7f213402a270 comp rx=0 tx=0).stop 2024-09-19T08:53:34.585 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.817+0000 7f2148ce0640 1 -- 172.21.15.120:0/1760936289 shutdown_connections 2024-09-19T08:53:34.586 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.817+0000 7f2148ce0640 1 --2- 172.21.15.120:0/1760936289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2144107680 0x7f2144107a80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:34.586 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.818+0000 7f2148ce0640 1 -- 172.21.15.120:0/1760936289 >> 172.21.15.120:0/1760936289 conn(0x7f2144102e50 msgr2=0x7f2144105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:34.586 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.818+0000 7f2148ce0640 1 -- 172.21.15.120:0/1760936289 shutdown_connections 2024-09-19T08:53:34.586 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.818+0000 7f2148ce0640 1 -- 172.21.15.120:0/1760936289 wait complete. 2024-09-19T08:53:34.586 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.818+0000 7f2148ce0640 1 Processor -- start 2024-09-19T08:53:34.586 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.818+0000 7f2148ce0640 1 -- start start 2024-09-19T08:53:34.586 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.819+0000 7f2148ce0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2144107680 0x7f214419b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:34.586 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.819+0000 7f2148ce0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f214410bbe0 con 0x7f2144107680 2024-09-19T08:53:34.586 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.819+0000 7f214259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2144107680 0x7f214419b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:34.586 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.819+0000 7f214259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2144107680 0x7f214419b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41266/0 (socket says 172.21.15.120:41266) 2024-09-19T08:53:34.586 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.819+0000 7f214259c640 1 -- 172.21.15.120:0/1461773464 learned_addr learned my addr 172.21.15.120:0/1461773464 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:34.587 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.819+0000 7f214259c640 1 -- 172.21.15.120:0/1461773464 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214419bbc0 con 0x7f2144107680 2024-09-19T08:53:34.587 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.819+0000 7f214259c640 1 --2- 172.21.15.120:0/1461773464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2144107680 0x7f214419b680 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2134009940 tx=0x7f21340326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:34.587 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.819+0000 7f213b7fe640 1 -- 172.21.15.120:0/1461773464 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2134042070 con 0x7f2144107680 2024-09-19T08:53:34.587 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.819+0000 7f2148ce0640 1 -- 172.21.15.120:0/1461773464 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f214419be50 con 0x7f2144107680 2024-09-19T08:53:34.587 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.819+0000 7f213b7fe640 1 -- 172.21.15.120:0/1461773464 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2134032be0 con 0x7f2144107680 2024-09-19T08:53:34.587 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.819+0000 7f213b7fe640 1 -- 172.21.15.120:0/1461773464 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2134037040 con 0x7f2144107680 2024-09-19T08:53:34.587 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.819+0000 7f2148ce0640 1 -- 172.21.15.120:0/1461773464 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f214419eb40 con 0x7f2144107680 2024-09-19T08:53:34.587 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.820+0000 7f213b7fe640 1 -- 172.21.15.120:0/1461773464 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f2134045070 con 0x7f2144107680 2024-09-19T08:53:34.587 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.820+0000 7f2148ce0640 1 -- 172.21.15.120:0/1461773464 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f214410bd60 con 0x7f2144107680 2024-09-19T08:53:34.587 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.820+0000 7f213b7fe640 1 --2- 172.21.15.120:0/1461773464 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f21300391b0 0x7f213003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:34.588 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.820+0000 7f213b7fe640 1 -- 172.21.15.120:0/1461773464 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f213406c1e0 con 0x7f2144107680 2024-09-19T08:53:34.588 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.820+0000 7f213bfff640 1 --2- 172.21.15.120:0/1461773464 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f21300391b0 0x7f213003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:34.588 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.821+0000 7f213bfff640 1 --2- 172.21.15.120:0/1461773464 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f21300391b0 0x7f213003b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f212c0098c0 tx=0x7f212c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:34.588 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.825+0000 7f213b7fe640 1 -- 172.21.15.120:0/1461773464 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2134032d90 con 0x7f2144107680 2024-09-19T08:53:34.588 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.995+0000 7f2148ce0640 1 -- 172.21.15.120:0/1461773464 --> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f2144104130 con 0x7f21300391b0 2024-09-19T08:53:34.588 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.996+0000 7f213b7fe640 1 -- 172.21.15.120:0/1461773464 <== mgr.14118 v2:172.21.15.120:6800/3562882246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f2144104130 con 0x7f21300391b0 2024-09-19T08:53:34.588 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.998+0000 7f2148ce0640 1 -- 172.21.15.120:0/1461773464 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f21300391b0 msgr2=0x7f213003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:34.588 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.998+0000 7f2148ce0640 1 --2- 172.21.15.120:0/1461773464 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f21300391b0 0x7f213003b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f212c0098c0 tx=0x7f212c006db0 comp rx=0 tx=0).stop 2024-09-19T08:53:34.588 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.998+0000 7f2148ce0640 1 -- 172.21.15.120:0/1461773464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2144107680 msgr2=0x7f214419b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:34.588 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.998+0000 7f2148ce0640 1 --2- 172.21.15.120:0/1461773464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2144107680 0x7f214419b680 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2134009940 tx=0x7f21340326e0 comp rx=0 tx=0).stop 2024-09-19T08:53:34.588 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.998+0000 7f2148ce0640 1 -- 172.21.15.120:0/1461773464 shutdown_connections 2024-09-19T08:53:34.588 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.998+0000 7f2148ce0640 1 --2- 172.21.15.120:0/1461773464 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f21300391b0 0x7f213003b670 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:34.588 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.998+0000 7f2148ce0640 1 --2- 172.21.15.120:0/1461773464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2144107680 0x7f214419b680 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:34.589 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.998+0000 7f2148ce0640 1 -- 172.21.15.120:0/1461773464 >> 172.21.15.120:0/1461773464 conn(0x7f2144102e50 msgr2=0x7f2144103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:34.589 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.998+0000 7f2148ce0640 1 -- 172.21.15.120:0/1461773464 shutdown_connections 2024-09-19T08:53:34.589 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:33.998+0000 7f2148ce0640 1 -- 172.21.15.120:0/1461773464 wait complete. 2024-09-19T08:53:34.589 INFO:teuthology.orchestra.run.smithi120.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-19T08:53:34.589 INFO:teuthology.orchestra.run.smithi120.stdout:Adding key to root@localhost authorized_keys... 2024-09-19T08:53:34.591 INFO:teuthology.orchestra.run.smithi120.stdout:Adding host smithi120... 2024-09-19T08:53:35.178 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:34 smithi120 ceph-mon[31150]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:53:35.967 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:35 smithi120 ceph-mon[31150]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi120", "addr": "172.21.15.120", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:53:36.927 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:36 smithi120 ceph-mon[31150]: Deploying cephadm binary to smithi120 2024-09-19T08:53:38.876 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout Added host 'smithi120' with addr '172.21.15.120' 2024-09-19T08:53:38.876 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.001+0000 7fed489e2640 1 Processor -- start 2024-09-19T08:53:38.876 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.001+0000 7fed489e2640 1 -- start start 2024-09-19T08:53:38.877 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.001+0000 7fed489e2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed44106520 0x7fed44106920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:38.877 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.001+0000 7fed489e2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fed44106ef0 con 0x7fed44106520 2024-09-19T08:53:38.877 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.001+0000 7fed4259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed44106520 0x7fed44106920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:38.877 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.002+0000 7fed4259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed44106520 0x7fed44106920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41282/0 (socket says 172.21.15.120:41282) 2024-09-19T08:53:38.877 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.002+0000 7fed4259c640 1 -- 172.21.15.120:0/1878872897 learned_addr learned my addr 172.21.15.120:0/1878872897 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:38.877 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.002+0000 7fed4259c640 1 -- 172.21.15.120:0/1878872897 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed44107720 con 0x7fed44106520 2024-09-19T08:53:38.877 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.002+0000 7fed4259c640 1 --2- 172.21.15.120:0/1878872897 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed44106520 0x7fed44106920 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fed34009820 tx=0x7fed3402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71418c1547d893 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:38.877 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.003+0000 7fed41d9b640 1 -- 172.21.15.120:0/1878872897 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed34037070 con 0x7fed44106520 2024-09-19T08:53:38.877 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.003+0000 7fed41d9b640 1 -- 172.21.15.120:0/1878872897 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fed34032900 con 0x7fed44106520 2024-09-19T08:53:38.877 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.003+0000 7fed41d9b640 1 -- 172.21.15.120:0/1878872897 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed34030850 con 0x7fed44106520 2024-09-19T08:53:38.877 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.003+0000 7fed489e2640 1 -- 172.21.15.120:0/1878872897 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed44106520 msgr2=0x7fed44106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:38.877 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.003+0000 7fed489e2640 1 --2- 172.21.15.120:0/1878872897 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed44106520 0x7fed44106920 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fed34009820 tx=0x7fed3402a270 comp rx=0 tx=0).stop 2024-09-19T08:53:38.878 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.003+0000 7fed489e2640 1 -- 172.21.15.120:0/1878872897 shutdown_connections 2024-09-19T08:53:38.878 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.003+0000 7fed489e2640 1 --2- 172.21.15.120:0/1878872897 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed44106520 0x7fed44106920 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:38.878 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.003+0000 7fed489e2640 1 -- 172.21.15.120:0/1878872897 >> 172.21.15.120:0/1878872897 conn(0x7fed44101db0 msgr2=0x7fed441041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:38.878 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.003+0000 7fed489e2640 1 -- 172.21.15.120:0/1878872897 shutdown_connections 2024-09-19T08:53:38.878 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.003+0000 7fed489e2640 1 -- 172.21.15.120:0/1878872897 wait complete. 2024-09-19T08:53:38.878 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.004+0000 7fed489e2640 1 Processor -- start 2024-09-19T08:53:38.878 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.004+0000 7fed489e2640 1 -- start start 2024-09-19T08:53:38.878 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.004+0000 7fed489e2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed44106520 0x7fed4407ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:38.878 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.004+0000 7fed489e2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fed4410aad0 con 0x7fed44106520 2024-09-19T08:53:38.878 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.004+0000 7fed4259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed44106520 0x7fed4407ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:38.878 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.004+0000 7fed4259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed44106520 0x7fed4407ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41286/0 (socket says 172.21.15.120:41286) 2024-09-19T08:53:38.878 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.004+0000 7fed4259c640 1 -- 172.21.15.120:0/2507334365 learned_addr learned my addr 172.21.15.120:0/2507334365 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:38.878 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.005+0000 7fed4259c640 1 -- 172.21.15.120:0/2507334365 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed4419e360 con 0x7fed44106520 2024-09-19T08:53:38.879 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.005+0000 7fed4259c640 1 --2- 172.21.15.120:0/2507334365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed44106520 0x7fed4407ba50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fed34009940 tx=0x7fed34032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:38.879 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.005+0000 7fed3bfff640 1 -- 172.21.15.120:0/2507334365 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed34042070 con 0x7fed44106520 2024-09-19T08:53:38.879 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.005+0000 7fed3bfff640 1 -- 172.21.15.120:0/2507334365 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fed340308c0 con 0x7fed44106520 2024-09-19T08:53:38.879 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.005+0000 7fed489e2640 1 -- 172.21.15.120:0/2507334365 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed4419e530 con 0x7fed44106520 2024-09-19T08:53:38.879 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.005+0000 7fed3bfff640 1 -- 172.21.15.120:0/2507334365 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed34037040 con 0x7fed44106520 2024-09-19T08:53:38.879 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.005+0000 7fed489e2640 1 -- 172.21.15.120:0/2507334365 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed4419e8d0 con 0x7fed44106520 2024-09-19T08:53:38.879 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.006+0000 7fed3bfff640 1 -- 172.21.15.120:0/2507334365 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fed34045070 con 0x7fed44106520 2024-09-19T08:53:38.879 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.006+0000 7fed489e2640 1 -- 172.21.15.120:0/2507334365 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed4410ac50 con 0x7fed44106520 2024-09-19T08:53:38.879 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.006+0000 7fed3bfff640 1 --2- 172.21.15.120:0/2507334365 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7fed300391b0 0x7fed3003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:38.879 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.006+0000 7fed3bfff640 1 -- 172.21.15.120:0/2507334365 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fed3406c3f0 con 0x7fed44106520 2024-09-19T08:53:38.879 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.006+0000 7fed39d9b640 1 --2- 172.21.15.120:0/2507334365 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7fed300391b0 0x7fed3003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:38.879 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.007+0000 7fed39d9b640 1 --2- 172.21.15.120:0/2507334365 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7fed300391b0 0x7fed3003b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fed2c0098c0 tx=0x7fed2c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:38.879 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.011+0000 7fed3bfff640 1 -- 172.21.15.120:0/2507334365 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fed3406bd80 con 0x7fed44106520 2024-09-19T08:53:38.879 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:35.179+0000 7fed489e2640 1 -- 172.21.15.120:0/2507334365 --> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi120", "addr": "172.21.15.120", "target": ["mon-mgr", ""]}) -- 0x7fed44102fe0 con 0x7fed300391b0 2024-09-19T08:53:38.880 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:38.341+0000 7fed3bfff640 1 -- 172.21.15.120:0/2507334365 <== mgr.14118 v2:172.21.15.120:6800/3562882246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fed44102fe0 con 0x7fed300391b0 2024-09-19T08:53:38.880 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:38.344+0000 7fed489e2640 1 -- 172.21.15.120:0/2507334365 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7fed300391b0 msgr2=0x7fed3003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:38.880 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:38.344+0000 7fed489e2640 1 --2- 172.21.15.120:0/2507334365 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7fed300391b0 0x7fed3003b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fed2c0098c0 tx=0x7fed2c006db0 comp rx=0 tx=0).stop 2024-09-19T08:53:38.880 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:38.344+0000 7fed489e2640 1 -- 172.21.15.120:0/2507334365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed44106520 msgr2=0x7fed4407ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:38.880 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:38.344+0000 7fed489e2640 1 --2- 172.21.15.120:0/2507334365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed44106520 0x7fed4407ba50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fed34009940 tx=0x7fed34032ce0 comp rx=0 tx=0).stop 2024-09-19T08:53:38.880 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:38.344+0000 7fed489e2640 1 -- 172.21.15.120:0/2507334365 shutdown_connections 2024-09-19T08:53:38.880 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:38.344+0000 7fed489e2640 1 --2- 172.21.15.120:0/2507334365 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7fed300391b0 0x7fed3003b670 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:38.880 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:38.344+0000 7fed489e2640 1 --2- 172.21.15.120:0/2507334365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed44106520 0x7fed4407ba50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:38.880 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:38.344+0000 7fed489e2640 1 -- 172.21.15.120:0/2507334365 >> 172.21.15.120:0/2507334365 conn(0x7fed44101db0 msgr2=0x7fed441028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:38.880 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:38.344+0000 7fed489e2640 1 -- 172.21.15.120:0/2507334365 shutdown_connections 2024-09-19T08:53:38.880 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:38.344+0000 7fed489e2640 1 -- 172.21.15.120:0/2507334365 wait complete. 2024-09-19T08:53:38.880 INFO:teuthology.orchestra.run.smithi120.stdout:Deploying unmanaged mon service... 2024-09-19T08:53:39.522 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:39 smithi120 ceph-mon[31150]: from='mgr.14118 172.21.15.120:0/3236336334' entity='mgr.x' 2024-09-19T08:53:39.522 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:39 smithi120 ceph-mon[31150]: Added host smithi120 2024-09-19T08:53:39.522 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:39 smithi120 ceph-mon[31150]: from='mgr.14118 172.21.15.120:0/3236336334' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:53:40.127 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-19T08:53:40.127 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.328+0000 7faba654b640 1 Processor -- start 2024-09-19T08:53:40.128 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.328+0000 7faba654b640 1 -- start start 2024-09-19T08:53:40.128 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.328+0000 7faba654b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faba0107680 0x7faba0107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:40.128 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.328+0000 7faba654b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faba0108050 con 0x7faba0107680 2024-09-19T08:53:40.128 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.328+0000 7fab9ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faba0107680 0x7faba0107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:40.128 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.328+0000 7fab9ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faba0107680 0x7faba0107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41302/0 (socket says 172.21.15.120:41302) 2024-09-19T08:53:40.128 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.328+0000 7fab9ffff640 1 -- 172.21.15.120:0/1262186627 learned_addr learned my addr 172.21.15.120:0/1262186627 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:40.128 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.329+0000 7fab9ffff640 1 -- 172.21.15.120:0/1262186627 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba0108830 con 0x7faba0107680 2024-09-19T08:53:40.128 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.329+0000 7fab9ffff640 1 --2- 172.21.15.120:0/1262186627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faba0107680 0x7faba0107a80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fab90009820 tx=0x7fab9002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8475bff78b477775 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:40.128 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.329+0000 7fab9effd640 1 -- 172.21.15.120:0/1262186627 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab90037070 con 0x7faba0107680 2024-09-19T08:53:40.128 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.329+0000 7fab9effd640 1 -- 172.21.15.120:0/1262186627 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fab90032900 con 0x7faba0107680 2024-09-19T08:53:40.128 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.329+0000 7fab9effd640 1 -- 172.21.15.120:0/1262186627 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab90030850 con 0x7faba0107680 2024-09-19T08:53:40.128 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.330+0000 7faba654b640 1 -- 172.21.15.120:0/1262186627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faba0107680 msgr2=0x7faba0107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:40.128 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.330+0000 7faba654b640 1 --2- 172.21.15.120:0/1262186627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faba0107680 0x7faba0107a80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fab90009820 tx=0x7fab9002a270 comp rx=0 tx=0).stop 2024-09-19T08:53:40.129 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.330+0000 7faba654b640 1 -- 172.21.15.120:0/1262186627 shutdown_connections 2024-09-19T08:53:40.129 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.330+0000 7faba654b640 1 --2- 172.21.15.120:0/1262186627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faba0107680 0x7faba0107a80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:40.129 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.330+0000 7faba654b640 1 -- 172.21.15.120:0/1262186627 >> 172.21.15.120:0/1262186627 conn(0x7faba0102e50 msgr2=0x7faba0105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:40.129 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.330+0000 7faba654b640 1 -- 172.21.15.120:0/1262186627 shutdown_connections 2024-09-19T08:53:40.129 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.330+0000 7faba654b640 1 -- 172.21.15.120:0/1262186627 wait complete. 2024-09-19T08:53:40.129 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.331+0000 7faba654b640 1 Processor -- start 2024-09-19T08:53:40.129 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.331+0000 7faba654b640 1 -- start start 2024-09-19T08:53:40.129 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.331+0000 7faba654b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faba0107680 0x7faba019b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:40.129 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.331+0000 7faba654b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faba010bbe0 con 0x7faba0107680 2024-09-19T08:53:40.129 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.331+0000 7fab9ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faba0107680 0x7faba019b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:40.129 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.331+0000 7fab9ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faba0107680 0x7faba019b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41312/0 (socket says 172.21.15.120:41312) 2024-09-19T08:53:40.129 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.331+0000 7fab9ffff640 1 -- 172.21.15.120:0/1647275373 learned_addr learned my addr 172.21.15.120:0/1647275373 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:40.129 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.331+0000 7fab9ffff640 1 -- 172.21.15.120:0/1647275373 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba019bbc0 con 0x7faba0107680 2024-09-19T08:53:40.129 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.331+0000 7fab9ffff640 1 --2- 172.21.15.120:0/1647275373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faba0107680 0x7faba019b680 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fab900097f0 tx=0x7fab90030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:40.130 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.332+0000 7fab9d7fa640 1 -- 172.21.15.120:0/1647275373 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab90042070 con 0x7faba0107680 2024-09-19T08:53:40.130 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.332+0000 7fab9d7fa640 1 -- 172.21.15.120:0/1647275373 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fab90032ca0 con 0x7faba0107680 2024-09-19T08:53:40.130 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.332+0000 7faba654b640 1 -- 172.21.15.120:0/1647275373 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faba019be50 con 0x7faba0107680 2024-09-19T08:53:40.130 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.332+0000 7fab9d7fa640 1 -- 172.21.15.120:0/1647275373 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab90037040 con 0x7faba0107680 2024-09-19T08:53:40.130 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.332+0000 7faba654b640 1 -- 172.21.15.120:0/1647275373 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faba019eb40 con 0x7faba0107680 2024-09-19T08:53:40.130 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.333+0000 7fab9d7fa640 1 -- 172.21.15.120:0/1647275373 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fab90045070 con 0x7faba0107680 2024-09-19T08:53:40.130 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.333+0000 7fab9d7fa640 1 --2- 172.21.15.120:0/1647275373 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7fab880391b0 0x7fab8803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:40.130 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.333+0000 7faba654b640 1 -- 172.21.15.120:0/1647275373 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faba010bd60 con 0x7faba0107680 2024-09-19T08:53:40.130 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.333+0000 7fab9d7fa640 1 -- 172.21.15.120:0/1647275373 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fab9006c1a0 con 0x7faba0107680 2024-09-19T08:53:40.130 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.333+0000 7fab9f7fe640 1 --2- 172.21.15.120:0/1647275373 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7fab880391b0 0x7fab8803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:40.130 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.333+0000 7fab9f7fe640 1 --2- 172.21.15.120:0/1647275373 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7fab880391b0 0x7fab8803b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fab940098c0 tx=0x7fab94006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:40.130 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.338+0000 7fab9d7fa640 1 -- 172.21.15.120:0/1647275373 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fab9004b170 con 0x7faba0107680 2024-09-19T08:53:40.130 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.522+0000 7faba654b640 1 -- 172.21.15.120:0/1647275373 --> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7faba0104120 con 0x7fab880391b0 2024-09-19T08:53:40.130 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.526+0000 7fab9d7fa640 1 -- 172.21.15.120:0/1647275373 <== mgr.14118 v2:172.21.15.120:6800/3562882246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7faba0104120 con 0x7fab880391b0 2024-09-19T08:53:40.131 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.529+0000 7faba654b640 1 -- 172.21.15.120:0/1647275373 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7fab880391b0 msgr2=0x7fab8803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:40.131 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.529+0000 7faba654b640 1 --2- 172.21.15.120:0/1647275373 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7fab880391b0 0x7fab8803b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fab940098c0 tx=0x7fab94006db0 comp rx=0 tx=0).stop 2024-09-19T08:53:40.131 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.529+0000 7faba654b640 1 -- 172.21.15.120:0/1647275373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faba0107680 msgr2=0x7faba019b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:40.131 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.529+0000 7faba654b640 1 --2- 172.21.15.120:0/1647275373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faba0107680 0x7faba019b680 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fab900097f0 tx=0x7fab90030ec0 comp rx=0 tx=0).stop 2024-09-19T08:53:40.131 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.529+0000 7faba654b640 1 -- 172.21.15.120:0/1647275373 shutdown_connections 2024-09-19T08:53:40.131 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.529+0000 7faba654b640 1 --2- 172.21.15.120:0/1647275373 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7fab880391b0 0x7fab8803b670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:40.131 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.529+0000 7faba654b640 1 --2- 172.21.15.120:0/1647275373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faba0107680 0x7faba019b680 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:40.131 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.529+0000 7faba654b640 1 -- 172.21.15.120:0/1647275373 >> 172.21.15.120:0/1647275373 conn(0x7faba0102e50 msgr2=0x7faba0103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:40.131 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.529+0000 7faba654b640 1 -- 172.21.15.120:0/1647275373 shutdown_connections 2024-09-19T08:53:40.131 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:39.529+0000 7faba654b640 1 -- 172.21.15.120:0/1647275373 wait complete. 2024-09-19T08:53:40.131 INFO:teuthology.orchestra.run.smithi120.stdout:Deploying unmanaged mgr service... 2024-09-19T08:53:40.728 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:40 smithi120 ceph-mon[31150]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:53:40.728 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:40 smithi120 ceph-mon[31150]: Saving service mon spec with placement count:5 2024-09-19T08:53:40.728 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:40 smithi120 ceph-mon[31150]: from='mgr.14118 172.21.15.120:0/3236336334' entity='mgr.x' 2024-09-19T08:53:42.059 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:41 smithi120 ceph-mon[31150]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:53:42.059 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:41 smithi120 ceph-mon[31150]: Saving service mgr spec with placement count:2 2024-09-19T08:53:42.059 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:41 smithi120 ceph-mon[31150]: from='mgr.14118 172.21.15.120:0/3236336334' entity='mgr.x' 2024-09-19T08:53:42.059 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:41 smithi120 ceph-mon[31150]: from='mgr.14118 172.21.15.120:0/3236336334' entity='mgr.x' 2024-09-19T08:53:42.066 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-19T08:53:42.066 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.535+0000 7f6b5d170640 1 Processor -- start 2024-09-19T08:53:42.066 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.535+0000 7f6b5d170640 1 -- start start 2024-09-19T08:53:42.066 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.535+0000 7f6b5d170640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5807de20 0x7f6b5807c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:42.066 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.535+0000 7f6b5d170640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5807c860 con 0x7f6b5807de20 2024-09-19T08:53:42.066 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.536+0000 7f6b56d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5807de20 0x7f6b5807c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:42.066 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.536+0000 7f6b56d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5807de20 0x7f6b5807c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41314/0 (socket says 172.21.15.120:41314) 2024-09-19T08:53:42.067 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.536+0000 7f6b56d9d640 1 -- 172.21.15.120:0/2874689703 learned_addr learned my addr 172.21.15.120:0/2874689703 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:42.067 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.536+0000 7f6b56d9d640 1 -- 172.21.15.120:0/2874689703 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b5807c9e0 con 0x7f6b5807de20 2024-09-19T08:53:42.067 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.536+0000 7f6b56d9d640 1 --2- 172.21.15.120:0/2874689703 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5807de20 0x7f6b5807c320 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6b48009820 tx=0x7f6b4802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c10e3a2415e090ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:42.067 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.537+0000 7f6b55d9b640 1 -- 172.21.15.120:0/2874689703 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b48037070 con 0x7f6b5807de20 2024-09-19T08:53:42.067 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.537+0000 7f6b55d9b640 1 -- 172.21.15.120:0/2874689703 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6b48032900 con 0x7f6b5807de20 2024-09-19T08:53:42.067 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.537+0000 7f6b55d9b640 1 -- 172.21.15.120:0/2874689703 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b48030850 con 0x7f6b5807de20 2024-09-19T08:53:42.067 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.537+0000 7f6b5d170640 1 -- 172.21.15.120:0/2874689703 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5807de20 msgr2=0x7f6b5807c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:42.067 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.537+0000 7f6b5d170640 1 --2- 172.21.15.120:0/2874689703 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5807de20 0x7f6b5807c320 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6b48009820 tx=0x7f6b4802a270 comp rx=0 tx=0).stop 2024-09-19T08:53:42.067 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.537+0000 7f6b5d170640 1 -- 172.21.15.120:0/2874689703 shutdown_connections 2024-09-19T08:53:42.067 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.537+0000 7f6b5d170640 1 --2- 172.21.15.120:0/2874689703 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5807de20 0x7f6b5807c320 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:42.067 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.537+0000 7f6b5d170640 1 -- 172.21.15.120:0/2874689703 >> 172.21.15.120:0/2874689703 conn(0x7f6b58102eb0 msgr2=0x7f6b58105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:42.067 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.537+0000 7f6b5d170640 1 -- 172.21.15.120:0/2874689703 shutdown_connections 2024-09-19T08:53:42.067 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.537+0000 7f6b5d170640 1 -- 172.21.15.120:0/2874689703 wait complete. 2024-09-19T08:53:42.068 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.538+0000 7f6b5d170640 1 Processor -- start 2024-09-19T08:53:42.068 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.538+0000 7f6b5d170640 1 -- start start 2024-09-19T08:53:42.068 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.538+0000 7f6b5d170640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5807de20 0x7f6b5819d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:42.068 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.538+0000 7f6b5d170640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5810dcd0 con 0x7f6b5807de20 2024-09-19T08:53:42.068 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.538+0000 7f6b56d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5807de20 0x7f6b5819d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:42.068 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.538+0000 7f6b56d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5807de20 0x7f6b5819d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41328/0 (socket says 172.21.15.120:41328) 2024-09-19T08:53:42.068 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.538+0000 7f6b56d9d640 1 -- 172.21.15.120:0/480259149 learned_addr learned my addr 172.21.15.120:0/480259149 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:42.068 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.539+0000 7f6b56d9d640 1 -- 172.21.15.120:0/480259149 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b5819dde0 con 0x7f6b5807de20 2024-09-19T08:53:42.068 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.539+0000 7f6b56d9d640 1 --2- 172.21.15.120:0/480259149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5807de20 0x7f6b5819d8a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6b48031000 tx=0x7f6b480326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:42.068 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.539+0000 7f6b4ffff640 1 -- 172.21.15.120:0/480259149 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b48042070 con 0x7f6b5807de20 2024-09-19T08:53:42.068 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.539+0000 7f6b5d170640 1 -- 172.21.15.120:0/480259149 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b5819e070 con 0x7f6b5807de20 2024-09-19T08:53:42.068 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.539+0000 7f6b4ffff640 1 -- 172.21.15.120:0/480259149 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6b48032be0 con 0x7f6b5807de20 2024-09-19T08:53:42.068 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.539+0000 7f6b4ffff640 1 -- 172.21.15.120:0/480259149 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b48037040 con 0x7f6b5807de20 2024-09-19T08:53:42.069 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.539+0000 7f6b5d170640 1 -- 172.21.15.120:0/480259149 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b581a0d60 con 0x7f6b5807de20 2024-09-19T08:53:42.069 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.540+0000 7f6b4ffff640 1 -- 172.21.15.120:0/480259149 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f6b48045070 con 0x7f6b5807de20 2024-09-19T08:53:42.069 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.540+0000 7f6b5d170640 1 -- 172.21.15.120:0/480259149 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b5807c3a0 con 0x7f6b5807de20 2024-09-19T08:53:42.069 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.540+0000 7f6b4ffff640 1 --2- 172.21.15.120:0/480259149 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f6b440391b0 0x7f6b4403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:42.069 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.540+0000 7f6b4ffff640 1 -- 172.21.15.120:0/480259149 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6b4806c2d0 con 0x7f6b5807de20 2024-09-19T08:53:42.069 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.540+0000 7f6b5659c640 1 --2- 172.21.15.120:0/480259149 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f6b440391b0 0x7f6b4403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:42.069 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.541+0000 7f6b5659c640 1 --2- 172.21.15.120:0/480259149 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f6b440391b0 0x7f6b4403b670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6b400098c0 tx=0x7f6b40006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:42.069 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.545+0000 7f6b4ffff640 1 -- 172.21.15.120:0/480259149 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6b48042210 con 0x7f6b5807de20 2024-09-19T08:53:42.069 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.728+0000 7f6b5d170640 1 -- 172.21.15.120:0/480259149 --> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f6b581041e0 con 0x7f6b440391b0 2024-09-19T08:53:42.069 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.733+0000 7f6b4ffff640 1 -- 172.21.15.120:0/480259149 <== mgr.14118 v2:172.21.15.120:6800/3562882246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6b581041e0 con 0x7f6b440391b0 2024-09-19T08:53:42.069 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.735+0000 7f6b5d170640 1 -- 172.21.15.120:0/480259149 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f6b440391b0 msgr2=0x7f6b4403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:42.069 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.735+0000 7f6b5d170640 1 --2- 172.21.15.120:0/480259149 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f6b440391b0 0x7f6b4403b670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6b400098c0 tx=0x7f6b40006db0 comp rx=0 tx=0).stop 2024-09-19T08:53:42.069 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.735+0000 7f6b5d170640 1 -- 172.21.15.120:0/480259149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5807de20 msgr2=0x7f6b5819d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:42.069 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.735+0000 7f6b5d170640 1 --2- 172.21.15.120:0/480259149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5807de20 0x7f6b5819d8a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6b48031000 tx=0x7f6b480326e0 comp rx=0 tx=0).stop 2024-09-19T08:53:42.070 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.735+0000 7f6b5d170640 1 -- 172.21.15.120:0/480259149 shutdown_connections 2024-09-19T08:53:42.070 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.735+0000 7f6b5d170640 1 --2- 172.21.15.120:0/480259149 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f6b440391b0 0x7f6b4403b670 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:42.070 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.735+0000 7f6b5d170640 1 --2- 172.21.15.120:0/480259149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5807de20 0x7f6b5819d8a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:42.070 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.735+0000 7f6b5d170640 1 -- 172.21.15.120:0/480259149 >> 172.21.15.120:0/480259149 conn(0x7f6b58102eb0 msgr2=0x7f6b58103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:42.070 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.735+0000 7f6b5d170640 1 -- 172.21.15.120:0/480259149 shutdown_connections 2024-09-19T08:53:42.070 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:40.735+0000 7f6b5d170640 1 -- 172.21.15.120:0/480259149 wait complete. 2024-09-19T08:53:43.266 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:43 smithi120 ceph-mon[31150]: from='mgr.14118 172.21.15.120:0/3236336334' entity='mgr.x' 2024-09-19T08:53:43.267 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:43 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1525438484' entity='client.admin' 2024-09-19T08:53:43.274 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.518+0000 7f82a3640640 1 Processor -- start 2024-09-19T08:53:43.274 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.518+0000 7f82a3640640 1 -- start start 2024-09-19T08:53:43.274 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.518+0000 7f82a3640640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f829c107680 0x7f829c107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:43.274 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.518+0000 7f82a3640640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f829c108050 con 0x7f829c107680 2024-09-19T08:53:43.274 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.518+0000 7f82a13dc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f829c107680 0x7f829c107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:43.274 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.518+0000 7f82a13dc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f829c107680 0x7f829c107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33088/0 (socket says 172.21.15.120:33088) 2024-09-19T08:53:43.274 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.518+0000 7f82a13dc640 1 -- 172.21.15.120:0/765719104 learned_addr learned my addr 172.21.15.120:0/765719104 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:43.274 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.518+0000 7f82a13dc640 1 -- 172.21.15.120:0/765719104 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829c108830 con 0x7f829c107680 2024-09-19T08:53:43.274 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.519+0000 7f82a13dc640 1 --2- 172.21.15.120:0/765719104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f829c107680 0x7f829c107a80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8298009820 tx=0x7f829802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1606245a826129e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:43.274 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.519+0000 7f8293fff640 1 -- 172.21.15.120:0/765719104 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8298037070 con 0x7f829c107680 2024-09-19T08:53:43.274 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.519+0000 7f8293fff640 1 -- 172.21.15.120:0/765719104 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8298032920 con 0x7f829c107680 2024-09-19T08:53:43.275 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.519+0000 7f8293fff640 1 -- 172.21.15.120:0/765719104 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8298030810 con 0x7f829c107680 2024-09-19T08:53:43.275 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.520+0000 7f82a3640640 1 -- 172.21.15.120:0/765719104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f829c107680 msgr2=0x7f829c107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:43.275 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.520+0000 7f82a3640640 1 --2- 172.21.15.120:0/765719104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f829c107680 0x7f829c107a80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8298009820 tx=0x7f829802a270 comp rx=0 tx=0).stop 2024-09-19T08:53:43.275 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.520+0000 7f82a3640640 1 -- 172.21.15.120:0/765719104 shutdown_connections 2024-09-19T08:53:43.275 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.520+0000 7f82a3640640 1 --2- 172.21.15.120:0/765719104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f829c107680 0x7f829c107a80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.275 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.520+0000 7f82a3640640 1 -- 172.21.15.120:0/765719104 >> 172.21.15.120:0/765719104 conn(0x7f829c102e50 msgr2=0x7f829c105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:43.275 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.520+0000 7f82a3640640 1 -- 172.21.15.120:0/765719104 shutdown_connections 2024-09-19T08:53:43.275 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.520+0000 7f82a3640640 1 -- 172.21.15.120:0/765719104 wait complete. 2024-09-19T08:53:43.275 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.521+0000 7f82a3640640 1 Processor -- start 2024-09-19T08:53:43.275 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.521+0000 7f82a3640640 1 -- start start 2024-09-19T08:53:43.275 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.521+0000 7f82a3640640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f829c107680 0x7f829c19b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:43.275 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.521+0000 7f82a3640640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f829c10bbe0 con 0x7f829c107680 2024-09-19T08:53:43.275 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.521+0000 7f82a13dc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f829c107680 0x7f829c19b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:43.275 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.521+0000 7f82a13dc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f829c107680 0x7f829c19b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33090/0 (socket says 172.21.15.120:33090) 2024-09-19T08:53:43.276 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.521+0000 7f82a13dc640 1 -- 172.21.15.120:0/1525438484 learned_addr learned my addr 172.21.15.120:0/1525438484 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:43.276 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.521+0000 7f82a13dc640 1 -- 172.21.15.120:0/1525438484 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829c19bbe0 con 0x7f829c107680 2024-09-19T08:53:43.276 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.521+0000 7f82a13dc640 1 --2- 172.21.15.120:0/1525438484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f829c107680 0x7f829c19b6a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8298030560 tx=0x7f8298032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:43.276 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.521+0000 7f82927fc640 1 -- 172.21.15.120:0/1525438484 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8298037040 con 0x7f829c107680 2024-09-19T08:53:43.276 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.521+0000 7f82927fc640 1 -- 172.21.15.120:0/1525438484 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8298040070 con 0x7f829c107680 2024-09-19T08:53:43.276 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.521+0000 7f82a3640640 1 -- 172.21.15.120:0/1525438484 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f829c19be70 con 0x7f829c107680 2024-09-19T08:53:43.276 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.521+0000 7f82927fc640 1 -- 172.21.15.120:0/1525438484 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f829803a6d0 con 0x7f829c107680 2024-09-19T08:53:43.276 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.521+0000 7f82a3640640 1 -- 172.21.15.120:0/1525438484 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f829c19eb60 con 0x7f829c107680 2024-09-19T08:53:43.276 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.522+0000 7f82927fc640 1 -- 172.21.15.120:0/1525438484 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f829803b420 con 0x7f829c107680 2024-09-19T08:53:43.276 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.522+0000 7f82a3640640 1 -- 172.21.15.120:0/1525438484 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f829c10bd60 con 0x7f829c107680 2024-09-19T08:53:43.276 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.522+0000 7f82927fc640 1 --2- 172.21.15.120:0/1525438484 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f82880391b0 0x7f828803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:43.276 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.523+0000 7f82927fc640 1 -- 172.21.15.120:0/1525438484 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f829806c800 con 0x7f829c107680 2024-09-19T08:53:43.277 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.523+0000 7f82a0bdb640 1 --2- 172.21.15.120:0/1525438484 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f82880391b0 0x7f828803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:43.277 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.523+0000 7f82a0bdb640 1 --2- 172.21.15.120:0/1525438484 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f82880391b0 0x7f828803b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f828c0098c0 tx=0x7f828c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:43.277 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.527+0000 7f82927fc640 1 -- 172.21.15.120:0/1525438484 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f829804a200 con 0x7f829c107680 2024-09-19T08:53:43.277 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.697+0000 7f82a3640640 1 -- 172.21.15.120:0/1525438484 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f829c107b00 con 0x7f829c107680 2024-09-19T08:53:43.277 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.700+0000 7f82927fc640 1 -- 172.21.15.120:0/1525438484 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f8298031e80 con 0x7f829c107680 2024-09-19T08:53:43.277 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.702+0000 7f82a3640640 1 -- 172.21.15.120:0/1525438484 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f82880391b0 msgr2=0x7f828803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:43.277 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.702+0000 7f82a3640640 1 --2- 172.21.15.120:0/1525438484 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f82880391b0 0x7f828803b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f828c0098c0 tx=0x7f828c006db0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.277 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.702+0000 7f82a3640640 1 -- 172.21.15.120:0/1525438484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f829c107680 msgr2=0x7f829c19b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:43.277 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.702+0000 7f82a3640640 1 --2- 172.21.15.120:0/1525438484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f829c107680 0x7f829c19b6a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8298030560 tx=0x7f8298032670 comp rx=0 tx=0).stop 2024-09-19T08:53:43.277 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.702+0000 7f82a3640640 1 -- 172.21.15.120:0/1525438484 shutdown_connections 2024-09-19T08:53:43.277 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.702+0000 7f82a3640640 1 --2- 172.21.15.120:0/1525438484 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f82880391b0 0x7f828803b670 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.277 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.702+0000 7f82a3640640 1 --2- 172.21.15.120:0/1525438484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f829c107680 0x7f829c19b6a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.277 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.702+0000 7f82a3640640 1 -- 172.21.15.120:0/1525438484 >> 172.21.15.120:0/1525438484 conn(0x7f829c102e50 msgr2=0x7f829c103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:43.277 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.703+0000 7f82a3640640 1 -- 172.21.15.120:0/1525438484 shutdown_connections 2024-09-19T08:53:43.278 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:42.703+0000 7f82a3640640 1 -- 172.21.15.120:0/1525438484 wait complete. 2024-09-19T08:53:44.693 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.681+0000 7f3321ca4640 1 Processor -- start 2024-09-19T08:53:44.693 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.681+0000 7f3321ca4640 1 -- start start 2024-09-19T08:53:44.693 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.681+0000 7f3321ca4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f331c07de20 0x7f331c07c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:44.693 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.681+0000 7f3321ca4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f331c07c860 con 0x7f331c07de20 2024-09-19T08:53:44.693 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.681+0000 7f331b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f331c07de20 0x7f331c07c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:44.693 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.681+0000 7f331b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f331c07de20 0x7f331c07c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33092/0 (socket says 172.21.15.120:33092) 2024-09-19T08:53:44.693 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.681+0000 7f331b7fe640 1 -- 172.21.15.120:0/1761738579 learned_addr learned my addr 172.21.15.120:0/1761738579 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:44.693 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.682+0000 7f331b7fe640 1 -- 172.21.15.120:0/1761738579 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f331c07c9e0 con 0x7f331c07de20 2024-09-19T08:53:44.693 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.682+0000 7f331b7fe640 1 --2- 172.21.15.120:0/1761738579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f331c07de20 0x7f331c07c320 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3310009820 tx=0x7f331002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf87d7b9e0ab5b6b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:44.693 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.682+0000 7f331a7fc640 1 -- 172.21.15.120:0/1761738579 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3310037070 con 0x7f331c07de20 2024-09-19T08:53:44.694 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.682+0000 7f331a7fc640 1 -- 172.21.15.120:0/1761738579 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3310032920 con 0x7f331c07de20 2024-09-19T08:53:44.694 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.682+0000 7f331a7fc640 1 -- 172.21.15.120:0/1761738579 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3310030810 con 0x7f331c07de20 2024-09-19T08:53:44.694 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.683+0000 7f3321ca4640 1 -- 172.21.15.120:0/1761738579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f331c07de20 msgr2=0x7f331c07c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:44.694 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.683+0000 7f3321ca4640 1 --2- 172.21.15.120:0/1761738579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f331c07de20 0x7f331c07c320 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3310009820 tx=0x7f331002a270 comp rx=0 tx=0).stop 2024-09-19T08:53:44.694 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.683+0000 7f3321ca4640 1 -- 172.21.15.120:0/1761738579 shutdown_connections 2024-09-19T08:53:44.694 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.683+0000 7f3321ca4640 1 --2- 172.21.15.120:0/1761738579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f331c07de20 0x7f331c07c320 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:44.694 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.683+0000 7f3321ca4640 1 -- 172.21.15.120:0/1761738579 >> 172.21.15.120:0/1761738579 conn(0x7f331c102eb0 msgr2=0x7f331c105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:44.694 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.683+0000 7f3321ca4640 1 -- 172.21.15.120:0/1761738579 shutdown_connections 2024-09-19T08:53:44.694 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.683+0000 7f3321ca4640 1 -- 172.21.15.120:0/1761738579 wait complete. 2024-09-19T08:53:44.694 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.684+0000 7f3321ca4640 1 Processor -- start 2024-09-19T08:53:44.694 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.684+0000 7f3321ca4640 1 -- start start 2024-09-19T08:53:44.694 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.684+0000 7f3321ca4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f331c07de20 0x7f331c19d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:44.694 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.684+0000 7f3321ca4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f331c10dcc0 con 0x7f331c07de20 2024-09-19T08:53:44.695 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.684+0000 7f331b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f331c07de20 0x7f331c19d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:44.695 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.684+0000 7f331b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f331c07de20 0x7f331c19d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33102/0 (socket says 172.21.15.120:33102) 2024-09-19T08:53:44.695 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.684+0000 7f331b7fe640 1 -- 172.21.15.120:0/2785407536 learned_addr learned my addr 172.21.15.120:0/2785407536 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:44.695 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.684+0000 7f331b7fe640 1 -- 172.21.15.120:0/2785407536 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f331c19de00 con 0x7f331c07de20 2024-09-19T08:53:44.695 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.684+0000 7f331b7fe640 1 --2- 172.21.15.120:0/2785407536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f331c07de20 0x7f331c19d8c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3310030420 tx=0x7f3310032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:44.695 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.685+0000 7f3318ff9640 1 -- 172.21.15.120:0/2785407536 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3310037040 con 0x7f331c07de20 2024-09-19T08:53:44.695 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.685+0000 7f3318ff9640 1 -- 172.21.15.120:0/2785407536 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3310040070 con 0x7f331c07de20 2024-09-19T08:53:44.695 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.685+0000 7f3321ca4640 1 -- 172.21.15.120:0/2785407536 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f331c19e090 con 0x7f331c07de20 2024-09-19T08:53:44.695 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.685+0000 7f3318ff9640 1 -- 172.21.15.120:0/2785407536 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f331003a6e0 con 0x7f331c07de20 2024-09-19T08:53:44.695 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.685+0000 7f3321ca4640 1 -- 172.21.15.120:0/2785407536 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f331c1a0d80 con 0x7f331c07de20 2024-09-19T08:53:44.696 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.686+0000 7f3318ff9640 1 -- 172.21.15.120:0/2785407536 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f331003a990 con 0x7f331c07de20 2024-09-19T08:53:44.696 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.686+0000 7f3318ff9640 1 --2- 172.21.15.120:0/2785407536 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f3304039200 0x7f330403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:44.696 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.686+0000 7f3318ff9640 1 -- 172.21.15.120:0/2785407536 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f331006d870 con 0x7f331c07de20 2024-09-19T08:53:44.696 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.686+0000 7f331affd640 1 --2- 172.21.15.120:0/2785407536 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f3304039200 0x7f330403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:44.696 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.686+0000 7f3321ca4640 1 -- 172.21.15.120:0/2785407536 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f331c07c3a0 con 0x7f331c07de20 2024-09-19T08:53:44.696 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.686+0000 7f331affd640 1 --2- 172.21.15.120:0/2785407536 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f3304039200 0x7f330403b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f330c0098c0 tx=0x7f330c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:44.697 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.691+0000 7f3318ff9640 1 -- 172.21.15.120:0/2785407536 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3310044c70 con 0x7f331c07de20 2024-09-19T08:53:44.697 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.874+0000 7f3321ca4640 1 -- 172.21.15.120:0/2785407536 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f331c1a0f50 con 0x7f331c07de20 2024-09-19T08:53:44.697 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.876+0000 7f3318ff9640 1 -- 172.21.15.120:0/2785407536 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f3310042e30 con 0x7f331c07de20 2024-09-19T08:53:44.697 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.879+0000 7f3321ca4640 1 -- 172.21.15.120:0/2785407536 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f3304039200 msgr2=0x7f330403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:44.697 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.879+0000 7f3321ca4640 1 --2- 172.21.15.120:0/2785407536 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f3304039200 0x7f330403b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f330c0098c0 tx=0x7f330c006db0 comp rx=0 tx=0).stop 2024-09-19T08:53:44.697 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.879+0000 7f3321ca4640 1 -- 172.21.15.120:0/2785407536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f331c07de20 msgr2=0x7f331c19d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:44.697 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.879+0000 7f3321ca4640 1 --2- 172.21.15.120:0/2785407536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f331c07de20 0x7f331c19d8c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3310030420 tx=0x7f3310032670 comp rx=0 tx=0).stop 2024-09-19T08:53:44.697 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.879+0000 7f3321ca4640 1 -- 172.21.15.120:0/2785407536 shutdown_connections 2024-09-19T08:53:44.697 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.879+0000 7f3321ca4640 1 --2- 172.21.15.120:0/2785407536 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f3304039200 0x7f330403b6c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:44.697 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.879+0000 7f3321ca4640 1 --2- 172.21.15.120:0/2785407536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f331c07de20 0x7f331c19d8c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:44.697 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.879+0000 7f3321ca4640 1 -- 172.21.15.120:0/2785407536 >> 172.21.15.120:0/2785407536 conn(0x7f331c102eb0 msgr2=0x7f331c103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:44.697 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.879+0000 7f3321ca4640 1 -- 172.21.15.120:0/2785407536 shutdown_connections 2024-09-19T08:53:44.697 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:43.879+0000 7f3321ca4640 1 -- 172.21.15.120:0/2785407536 wait complete. 2024-09-19T08:53:44.697 INFO:teuthology.orchestra.run.smithi120.stdout:Enabling the dashboard module... 2024-09-19T08:53:45.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:44 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2785407536' entity='client.admin' 2024-09-19T08:53:46.284 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:46 smithi120 ceph-mon[31150]: from='mgr.14118 172.21.15.120:0/3236336334' entity='mgr.x' 2024-09-19T08:53:46.284 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:46 smithi120 ceph-mon[31150]: from='mgr.14118 172.21.15.120:0/3236336334' entity='mgr.x' 2024-09-19T08:53:46.284 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:46 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1128643399' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-19T08:53:47.062 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:46 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: ignoring --setuser ceph since I am not root 2024-09-19T08:53:47.062 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:46 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: ignoring --setgroup ceph since I am not root 2024-09-19T08:53:47.062 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:46 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:46.855+0000 7f69d9c63040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T08:53:47.062 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:46 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:46.911+0000 7f69d9c63040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T08:53:47.062 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:47 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:47.061+0000 7f69d9c63040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T08:53:47.324 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:47 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:47.128+0000 7f69d9c63040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T08:53:47.324 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:47 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:47.276+0000 7f69d9c63040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:53:47.423 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.485+0000 7f22309b2640 1 Processor -- start 2024-09-19T08:53:47.423 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.485+0000 7f22309b2640 1 -- start start 2024-09-19T08:53:47.423 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.485+0000 7f22309b2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f222c108c40 0x7f222c109040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:47.423 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.485+0000 7f22309b2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f222c109610 con 0x7f222c108c40 2024-09-19T08:53:47.423 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.486+0000 7f222a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f222c108c40 0x7f222c109040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:47.423 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.486+0000 7f222a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f222c108c40 0x7f222c109040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33114/0 (socket says 172.21.15.120:33114) 2024-09-19T08:53:47.423 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.486+0000 7f222a59c640 1 -- 172.21.15.120:0/2300542303 learned_addr learned my addr 172.21.15.120:0/2300542303 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:47.423 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.486+0000 7f222a59c640 1 -- 172.21.15.120:0/2300542303 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222c109790 con 0x7f222c108c40 2024-09-19T08:53:47.423 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.487+0000 7f222a59c640 1 --2- 172.21.15.120:0/2300542303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f222c108c40 0x7f222c109040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f221c009820 tx=0x7f221c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5096b5ea6aa8ee8c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:47.423 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.487+0000 7f2229d9b640 1 -- 172.21.15.120:0/2300542303 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f221c037070 con 0x7f222c108c40 2024-09-19T08:53:47.423 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.487+0000 7f2229d9b640 1 -- 172.21.15.120:0/2300542303 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f221c032920 con 0x7f222c108c40 2024-09-19T08:53:47.423 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.487+0000 7f2229d9b640 1 -- 172.21.15.120:0/2300542303 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f221c030810 con 0x7f222c108c40 2024-09-19T08:53:47.424 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.488+0000 7f22309b2640 1 -- 172.21.15.120:0/2300542303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f222c108c40 msgr2=0x7f222c109040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:47.424 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.488+0000 7f22309b2640 1 --2- 172.21.15.120:0/2300542303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f222c108c40 0x7f222c109040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f221c009820 tx=0x7f221c02a270 comp rx=0 tx=0).stop 2024-09-19T08:53:47.424 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.488+0000 7f22309b2640 1 -- 172.21.15.120:0/2300542303 shutdown_connections 2024-09-19T08:53:47.424 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.488+0000 7f22309b2640 1 --2- 172.21.15.120:0/2300542303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f222c108c40 0x7f222c109040 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:47.424 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.488+0000 7f22309b2640 1 -- 172.21.15.120:0/2300542303 >> 172.21.15.120:0/2300542303 conn(0x7f222c101510 msgr2=0x7f222c103930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:47.424 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.488+0000 7f22309b2640 1 -- 172.21.15.120:0/2300542303 shutdown_connections 2024-09-19T08:53:47.424 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.488+0000 7f22309b2640 1 -- 172.21.15.120:0/2300542303 wait complete. 2024-09-19T08:53:47.424 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.489+0000 7f22309b2640 1 Processor -- start 2024-09-19T08:53:47.424 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.489+0000 7f22309b2640 1 -- start start 2024-09-19T08:53:47.424 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.489+0000 7f22309b2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f222c108c40 0x7f222c199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:47.424 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.489+0000 7f22309b2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f222c1067d0 con 0x7f222c108c40 2024-09-19T08:53:47.424 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.489+0000 7f222a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f222c108c40 0x7f222c199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:47.424 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.489+0000 7f222a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f222c108c40 0x7f222c199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33118/0 (socket says 172.21.15.120:33118) 2024-09-19T08:53:47.424 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.489+0000 7f222a59c640 1 -- 172.21.15.120:0/1128643399 learned_addr learned my addr 172.21.15.120:0/1128643399 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:47.425 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.490+0000 7f222a59c640 1 -- 172.21.15.120:0/1128643399 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222c1999a0 con 0x7f222c108c40 2024-09-19T08:53:47.425 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.490+0000 7f222a59c640 1 --2- 172.21.15.120:0/1128643399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f222c108c40 0x7f222c199460 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f221c030560 tx=0x7f221c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:47.425 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.490+0000 7f22237fe640 1 -- 172.21.15.120:0/1128643399 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f221c037040 con 0x7f222c108c40 2024-09-19T08:53:47.425 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.490+0000 7f22237fe640 1 -- 172.21.15.120:0/1128643399 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f221c040070 con 0x7f222c108c40 2024-09-19T08:53:47.425 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.490+0000 7f22237fe640 1 -- 172.21.15.120:0/1128643399 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f221c03a740 con 0x7f222c108c40 2024-09-19T08:53:47.425 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.491+0000 7f22309b2640 1 -- 172.21.15.120:0/1128643399 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f222c199c30 con 0x7f222c108c40 2024-09-19T08:53:47.425 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.491+0000 7f22309b2640 1 -- 172.21.15.120:0/1128643399 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f222c19c920 con 0x7f222c108c40 2024-09-19T08:53:47.425 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.492+0000 7f22237fe640 1 -- 172.21.15.120:0/1128643399 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f221c03bb90 con 0x7f222c108c40 2024-09-19T08:53:47.425 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.492+0000 7f22309b2640 1 -- 172.21.15.120:0/1128643399 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f222c109040 con 0x7f222c108c40 2024-09-19T08:53:47.425 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.492+0000 7f22237fe640 1 --2- 172.21.15.120:0/1128643399 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f221803d660 0x7f221803fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:47.425 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.492+0000 7f22237fe640 1 -- 172.21.15.120:0/1128643399 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f221c06bd60 con 0x7f222c108c40 2024-09-19T08:53:47.425 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.493+0000 7f2223fff640 1 --2- 172.21.15.120:0/1128643399 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f221803d660 0x7f221803fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:47.425 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.493+0000 7f2223fff640 1 --2- 172.21.15.120:0/1128643399 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f221803d660 0x7f221803fb20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f22140098c0 tx=0x7f2214006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:47.426 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.497+0000 7f22237fe640 1 -- 172.21.15.120:0/1128643399 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f221c03b940 con 0x7f222c108c40 2024-09-19T08:53:47.426 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:45.731+0000 7f22309b2640 1 -- 172.21.15.120:0/1128643399 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f222c02d9a0 con 0x7f222c108c40 2024-09-19T08:53:47.426 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:46.702+0000 7f22237fe640 1 -- 172.21.15.120:0/1128643399 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f221c0328f0 con 0x7f222c108c40 2024-09-19T08:53:47.426 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:46.704+0000 7f22237fe640 1 -- 172.21.15.120:0/1128643399 <== mon.0 v2:172.21.15.120:3300/0 8 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f221c03b420 con 0x7f222c108c40 2024-09-19T08:53:47.426 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:46.705+0000 7f22309b2640 1 -- 172.21.15.120:0/1128643399 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f221803d660 msgr2=0x7f221803fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:47.426 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:46.705+0000 7f22309b2640 1 --2- 172.21.15.120:0/1128643399 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f221803d660 0x7f221803fb20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f22140098c0 tx=0x7f2214006db0 comp rx=0 tx=0).stop 2024-09-19T08:53:47.426 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:46.705+0000 7f22309b2640 1 -- 172.21.15.120:0/1128643399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f222c108c40 msgr2=0x7f222c199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:47.426 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:46.705+0000 7f22309b2640 1 --2- 172.21.15.120:0/1128643399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f222c108c40 0x7f222c199460 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f221c030560 tx=0x7f221c032670 comp rx=0 tx=0).stop 2024-09-19T08:53:47.426 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:46.705+0000 7f22309b2640 1 -- 172.21.15.120:0/1128643399 shutdown_connections 2024-09-19T08:53:47.426 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:46.705+0000 7f22309b2640 1 --2- 172.21.15.120:0/1128643399 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f221803d660 0x7f221803fb20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:47.426 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:46.705+0000 7f22309b2640 1 --2- 172.21.15.120:0/1128643399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f222c108c40 0x7f222c199460 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:47.426 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:46.705+0000 7f22309b2640 1 -- 172.21.15.120:0/1128643399 >> 172.21.15.120:0/1128643399 conn(0x7f222c101510 msgr2=0x7f222c103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:47.426 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:46.705+0000 7f22309b2640 1 -- 172.21.15.120:0/1128643399 shutdown_connections 2024-09-19T08:53:47.426 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:46.706+0000 7f22309b2640 1 -- 172.21.15.120:0/1128643399 wait complete. 2024-09-19T08:53:47.668 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:47 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:47.416+0000 7f69d9c63040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T08:53:47.995 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:47 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1128643399' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-19T08:53:47.995 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:47 smithi120 ceph-mon[31150]: mgrmap e9: x(active, since 18s) 2024-09-19T08:53:47.995 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:47 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:47.740+0000 7f69d9c63040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T08:53:47.995 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:47 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:47.804+0000 7f69d9c63040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T08:53:47.995 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:47 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:47.995+0000 7f69d9c63040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T08:53:48.891 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout { 2024-09-19T08:53:48.891 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-19T08:53:48.891 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T08:53:48.892 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T08:53:48.892 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T08:53:48.892 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout } 2024-09-19T08:53:48.892 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.838+0000 7fdf9b3de640 1 Processor -- start 2024-09-19T08:53:48.892 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.839+0000 7fdf9b3de640 1 -- start start 2024-09-19T08:53:48.892 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.839+0000 7fdf9b3de640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf94107740 0x7fdf94107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:48.892 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.839+0000 7fdf9b3de640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdf94108110 con 0x7fdf94107740 2024-09-19T08:53:48.892 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.839+0000 7fdf9917a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf94107740 0x7fdf94107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:48.892 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.839+0000 7fdf9917a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf94107740 0x7fdf94107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33154/0 (socket says 172.21.15.120:33154) 2024-09-19T08:53:48.892 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.839+0000 7fdf9917a640 1 -- 172.21.15.120:0/810739447 learned_addr learned my addr 172.21.15.120:0/810739447 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:48.892 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.840+0000 7fdf9917a640 1 -- 172.21.15.120:0/810739447 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf941088a0 con 0x7fdf94107740 2024-09-19T08:53:48.892 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.840+0000 7fdf9917a640 1 --2- 172.21.15.120:0/810739447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf94107740 0x7fdf94107b40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdf90009820 tx=0x7fdf9002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03933efd61fe764 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:48.892 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.840+0000 7fdf8bfff640 1 -- 172.21.15.120:0/810739447 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf90037070 con 0x7fdf94107740 2024-09-19T08:53:48.892 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.840+0000 7fdf8bfff640 1 -- 172.21.15.120:0/810739447 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf90032920 con 0x7fdf94107740 2024-09-19T08:53:48.893 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.840+0000 7fdf8bfff640 1 -- 172.21.15.120:0/810739447 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf90030810 con 0x7fdf94107740 2024-09-19T08:53:48.893 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.841+0000 7fdf9b3de640 1 -- 172.21.15.120:0/810739447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf94107740 msgr2=0x7fdf94107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:48.893 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.841+0000 7fdf9b3de640 1 --2- 172.21.15.120:0/810739447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf94107740 0x7fdf94107b40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdf90009820 tx=0x7fdf9002a270 comp rx=0 tx=0).stop 2024-09-19T08:53:48.893 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.841+0000 7fdf9b3de640 1 -- 172.21.15.120:0/810739447 shutdown_connections 2024-09-19T08:53:48.893 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.841+0000 7fdf9b3de640 1 --2- 172.21.15.120:0/810739447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf94107740 0x7fdf94107b40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.893 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.841+0000 7fdf9b3de640 1 -- 172.21.15.120:0/810739447 >> 172.21.15.120:0/810739447 conn(0x7fdf94102eb0 msgr2=0x7fdf94105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:48.893 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.841+0000 7fdf9b3de640 1 -- 172.21.15.120:0/810739447 shutdown_connections 2024-09-19T08:53:48.893 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.841+0000 7fdf9b3de640 1 -- 172.21.15.120:0/810739447 wait complete. 2024-09-19T08:53:48.893 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.842+0000 7fdf9b3de640 1 Processor -- start 2024-09-19T08:53:48.893 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.842+0000 7fdf9b3de640 1 -- start start 2024-09-19T08:53:48.893 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.842+0000 7fdf9b3de640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf94107740 0x7fdf9419b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:48.893 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.842+0000 7fdf9b3de640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9410bc50 con 0x7fdf94107740 2024-09-19T08:53:48.894 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.842+0000 7fdf9917a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf94107740 0x7fdf9419b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:48.894 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.842+0000 7fdf9917a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf94107740 0x7fdf9419b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33160/0 (socket says 172.21.15.120:33160) 2024-09-19T08:53:48.894 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.842+0000 7fdf9917a640 1 -- 172.21.15.120:0/3243999200 learned_addr learned my addr 172.21.15.120:0/3243999200 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:48.894 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.842+0000 7fdf9917a640 1 -- 172.21.15.120:0/3243999200 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9419bc00 con 0x7fdf94107740 2024-09-19T08:53:48.894 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.842+0000 7fdf9917a640 1 --2- 172.21.15.120:0/3243999200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf94107740 0x7fdf9419b6c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdf900097f0 tx=0x7fdf900327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:48.894 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.843+0000 7fdf8a7fc640 1 -- 172.21.15.120:0/3243999200 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf90037040 con 0x7fdf94107740 2024-09-19T08:53:48.894 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.843+0000 7fdf8a7fc640 1 -- 172.21.15.120:0/3243999200 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf90030c40 con 0x7fdf94107740 2024-09-19T08:53:48.894 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.843+0000 7fdf9b3de640 1 -- 172.21.15.120:0/3243999200 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf9419be90 con 0x7fdf94107740 2024-09-19T08:53:48.894 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.843+0000 7fdf8a7fc640 1 -- 172.21.15.120:0/3243999200 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf9003a5b0 con 0x7fdf94107740 2024-09-19T08:53:48.894 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.843+0000 7fdf9b3de640 1 -- 172.21.15.120:0/3243999200 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf9419eb80 con 0x7fdf94107740 2024-09-19T08:53:48.894 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.844+0000 7fdf8a7fc640 1 -- 172.21.15.120:0/3243999200 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7fdf9003b3f0 con 0x7fdf94107740 2024-09-19T08:53:48.895 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.844+0000 7fdf9b3de640 1 -- 172.21.15.120:0/3243999200 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf94107bc0 con 0x7fdf94107740 2024-09-19T08:53:48.895 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.844+0000 7fdf8a7fc640 1 --2- 172.21.15.120:0/3243999200 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7fdf80039260 0x7fdf8003b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:48.895 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.844+0000 7fdf98979640 1 -- 172.21.15.120:0/3243999200 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7fdf80039260 msgr2=0x7fdf8003b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.120:6800/3562882246 2024-09-19T08:53:48.895 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.844+0000 7fdf8a7fc640 1 -- 172.21.15.120:0/3243999200 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdf9006c6d0 con 0x7fdf94107740 2024-09-19T08:53:48.895 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.844+0000 7fdf98979640 1 --2- 172.21.15.120:0/3243999200 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7fdf80039260 0x7fdf8003b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:53:48.895 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:47.849+0000 7fdf8a7fc640 1 -- 172.21.15.120:0/3243999200 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdf90044dc0 con 0x7fdf94107740 2024-09-19T08:53:48.895 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:48.045+0000 7fdf98979640 1 -- 172.21.15.120:0/3243999200 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7fdf80039260 msgr2=0x7fdf8003b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.120:6800/3562882246 2024-09-19T08:53:48.895 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:48.045+0000 7fdf98979640 1 --2- 172.21.15.120:0/3243999200 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7fdf80039260 0x7fdf8003b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T08:53:48.895 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:48.067+0000 7fdf9b3de640 1 -- 172.21.15.120:0/3243999200 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fdf9419f170 con 0x7fdf94107740 2024-09-19T08:53:48.895 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:48.067+0000 7fdf8a7fc640 1 -- 172.21.15.120:0/3243999200 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fdf900313a0 con 0x7fdf94107740 2024-09-19T08:53:48.895 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:48.069+0000 7fdf9b3de640 1 -- 172.21.15.120:0/3243999200 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7fdf80039260 msgr2=0x7fdf8003b720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:53:48.895 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:48.069+0000 7fdf9b3de640 1 --2- 172.21.15.120:0/3243999200 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7fdf80039260 0x7fdf8003b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.896 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:48.069+0000 7fdf9b3de640 1 -- 172.21.15.120:0/3243999200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf94107740 msgr2=0x7fdf9419b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:48.896 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:48.069+0000 7fdf9b3de640 1 --2- 172.21.15.120:0/3243999200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf94107740 0x7fdf9419b6c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdf900097f0 tx=0x7fdf900327b0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.896 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:48.069+0000 7fdf9b3de640 1 -- 172.21.15.120:0/3243999200 shutdown_connections 2024-09-19T08:53:48.896 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:48.069+0000 7fdf9b3de640 1 --2- 172.21.15.120:0/3243999200 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7fdf80039260 0x7fdf8003b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.896 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:48.069+0000 7fdf9b3de640 1 --2- 172.21.15.120:0/3243999200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf94107740 0x7fdf9419b6c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.896 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:48.069+0000 7fdf9b3de640 1 -- 172.21.15.120:0/3243999200 >> 172.21.15.120:0/3243999200 conn(0x7fdf94102eb0 msgr2=0x7fdf94103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:48.896 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:48.069+0000 7fdf9b3de640 1 -- 172.21.15.120:0/3243999200 shutdown_connections 2024-09-19T08:53:48.896 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:48.069+0000 7fdf9b3de640 1 -- 172.21.15.120:0/3243999200 wait complete. 2024-09-19T08:53:48.896 INFO:teuthology.orchestra.run.smithi120.stdout:Waiting for the mgr to restart... 2024-09-19T08:53:48.896 INFO:teuthology.orchestra.run.smithi120.stdout:Waiting for mgr epoch 9... 2024-09-19T08:53:48.960 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:48 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3243999200' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T08:53:48.961 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:48 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:48.890+0000 7f69d9c63040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T08:53:49.273 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:48 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:48.960+0000 7f69d9c63040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T08:53:49.930 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:49 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:49.676+0000 7f69d9c63040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T08:53:49.931 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:49 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:49.789+0000 7f69d9c63040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T08:53:50.204 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:49 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:49.930+0000 7f69d9c63040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:53:50.204 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:49 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:49.991+0000 7f69d9c63040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T08:53:50.204 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:50 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:50.137+0000 7f69d9c63040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T08:53:50.204 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:50 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:50.204+0000 7f69d9c63040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T08:53:50.483 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:50 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:50.264+0000 7f69d9c63040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T08:53:50.483 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:50 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:50.420+0000 7f69d9c63040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T08:53:50.484 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:50 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:50.483+0000 7f69d9c63040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T08:53:50.748 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:50 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:50.549+0000 7f69d9c63040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T08:53:50.748 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:50 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:50.630+0000 7f69d9c63040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T08:53:50.748 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:50 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:50.689+0000 7f69d9c63040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T08:53:50.748 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:50 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:50.748+0000 7f69d9c63040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T08:53:51.824 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:51 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:51.526+0000 7f69d9c63040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T08:53:51.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:51 smithi120 ceph-mon[31150]: Active manager daemon x restarted 2024-09-19T08:53:51.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:51 smithi120 ceph-mon[31150]: Activating manager daemon x 2024-09-19T08:53:51.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:51 smithi120 ceph-mon[31150]: osdmap e3: 0 total, 0 up, 0 in 2024-09-19T08:53:51.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:51 smithi120 ceph-mon[31150]: mgrmap e10: x(active, starting, since 0.00324428s) 2024-09-19T08:53:51.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:53:51.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T08:53:51.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T08:53:51.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T08:53:51.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T08:53:51.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:51 smithi120 ceph-mon[31150]: Manager daemon x is now available 2024-09-19T08:53:51.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:53:51.827 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T08:53:51.827 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T08:53:53.075 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:53 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:53.015+0000 7f693f72c640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:08:53:53] ENGINE Error in HTTPServer.serve 2024-09-19T08:53:53.075 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:53 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: Traceback (most recent call last): 2024-09-19T08:53:53.075 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:53 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T08:53:53.075 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:53 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: self._connections.run(self.expiration_interval) 2024-09-19T08:53:53.075 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:53 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T08:53:53.075 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:53 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: self._run(expiration_interval) 2024-09-19T08:53:53.075 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:53 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T08:53:53.076 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:53 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T08:53:53.076 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:53 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T08:53:53.076 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:53 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T08:53:53.076 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:53 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T08:53:53.076 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:53 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: s = self.context.wrap_socket( 2024-09-19T08:53:53.076 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:53 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T08:53:53.076 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:53 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: return self.sslsocket_class._create( 2024-09-19T08:53:53.076 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:53 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T08:53:53.076 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:53 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: self.do_handshake() 2024-09-19T08:53:53.076 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:53 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T08:53:53.076 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:53 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: self._sslobj.do_handshake() 2024-09-19T08:53:53.076 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:53 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T08:53:53.077 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 08:53:53 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T08:53:53.370 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout { 2024-09-19T08:53:53.370 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-19T08:53:53.370 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T08:53:53.370 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout } 2024-09-19T08:53:53.370 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.274+0000 7f960750d640 1 Processor -- start 2024-09-19T08:53:53.370 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.274+0000 7f960750d640 1 -- start start 2024-09-19T08:53:53.371 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.274+0000 7f960750d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9600106510 0x7f9600106910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:53.371 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.274+0000 7f960750d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9600106ee0 con 0x7f9600106510 2024-09-19T08:53:53.371 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.274+0000 7f96052a9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9600106510 0x7f9600106910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:53.371 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.274+0000 7f96052a9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9600106510 0x7f9600106910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33164/0 (socket says 172.21.15.120:33164) 2024-09-19T08:53:53.371 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.274+0000 7f96052a9640 1 -- 172.21.15.120:0/3744752237 learned_addr learned my addr 172.21.15.120:0/3744752237 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:53.371 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.275+0000 7f96052a9640 1 -- 172.21.15.120:0/3744752237 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96001076c0 con 0x7f9600106510 2024-09-19T08:53:53.371 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.275+0000 7f96052a9640 1 --2- 172.21.15.120:0/3744752237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9600106510 0x7f9600106910 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f95fc009820 tx=0x7f95fc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0e827a72f7186d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:53.371 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.275+0000 7f95f7fff640 1 -- 172.21.15.120:0/3744752237 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95fc037070 con 0x7f9600106510 2024-09-19T08:53:53.371 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.276+0000 7f95f7fff640 1 -- 172.21.15.120:0/3744752237 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95fc032920 con 0x7f9600106510 2024-09-19T08:53:53.371 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.276+0000 7f95f7fff640 1 -- 172.21.15.120:0/3744752237 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95fc030810 con 0x7f9600106510 2024-09-19T08:53:53.371 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.276+0000 7f960750d640 1 -- 172.21.15.120:0/3744752237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9600106510 msgr2=0x7f9600106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:53.371 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.276+0000 7f960750d640 1 --2- 172.21.15.120:0/3744752237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9600106510 0x7f9600106910 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f95fc009820 tx=0x7f95fc02a270 comp rx=0 tx=0).stop 2024-09-19T08:53:53.371 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.276+0000 7f960750d640 1 -- 172.21.15.120:0/3744752237 shutdown_connections 2024-09-19T08:53:53.372 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.276+0000 7f960750d640 1 --2- 172.21.15.120:0/3744752237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9600106510 0x7f9600106910 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.372 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.276+0000 7f960750d640 1 -- 172.21.15.120:0/3744752237 >> 172.21.15.120:0/3744752237 conn(0x7f9600101ca0 msgr2=0x7f96001040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:53.372 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.276+0000 7f960750d640 1 -- 172.21.15.120:0/3744752237 shutdown_connections 2024-09-19T08:53:53.372 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.276+0000 7f960750d640 1 -- 172.21.15.120:0/3744752237 wait complete. 2024-09-19T08:53:53.372 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.277+0000 7f960750d640 1 Processor -- start 2024-09-19T08:53:53.372 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.277+0000 7f960750d640 1 -- start start 2024-09-19T08:53:53.372 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.277+0000 7f960750d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9600106510 0x7f960019b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:53.372 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.277+0000 7f960750d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f960010aa70 con 0x7f9600106510 2024-09-19T08:53:53.372 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.277+0000 7f96052a9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9600106510 0x7f960019b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:53.372 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.277+0000 7f96052a9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9600106510 0x7f960019b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33170/0 (socket says 172.21.15.120:33170) 2024-09-19T08:53:53.372 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.277+0000 7f96052a9640 1 -- 172.21.15.120:0/2769650838 learned_addr learned my addr 172.21.15.120:0/2769650838 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:53.372 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.277+0000 7f96052a9640 1 -- 172.21.15.120:0/2769650838 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960019bb90 con 0x7f9600106510 2024-09-19T08:53:53.372 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.278+0000 7f96052a9640 1 --2- 172.21.15.120:0/2769650838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9600106510 0x7f960019b650 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0097f0 tx=0x7f95fc0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:53.373 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.278+0000 7f95f67fc640 1 -- 172.21.15.120:0/2769650838 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95fc037040 con 0x7f9600106510 2024-09-19T08:53:53.373 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.278+0000 7f960750d640 1 -- 172.21.15.120:0/2769650838 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f960019be20 con 0x7f9600106510 2024-09-19T08:53:53.373 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.278+0000 7f95f67fc640 1 -- 172.21.15.120:0/2769650838 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95fc030cc0 con 0x7f9600106510 2024-09-19T08:53:53.373 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.278+0000 7f960750d640 1 -- 172.21.15.120:0/2769650838 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f960019eb10 con 0x7f9600106510 2024-09-19T08:53:53.373 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.278+0000 7f95f67fc640 1 -- 172.21.15.120:0/2769650838 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95fc03a640 con 0x7f9600106510 2024-09-19T08:53:53.373 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.279+0000 7f95f67fc640 1 -- 172.21.15.120:0/2769650838 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f95fc03b3f0 con 0x7f9600106510 2024-09-19T08:53:53.373 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.279+0000 7f95f67fc640 1 --2- 172.21.15.120:0/2769650838 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f95ec039200 0x7f95ec03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:53.373 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.279+0000 7f95f67fc640 1 -- 172.21.15.120:0/2769650838 --> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f95ec03bdd0 con 0x7f95ec039200 2024-09-19T08:53:53.373 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.279+0000 7f95f67fc640 1 -- 172.21.15.120:0/2769650838 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f95fc06c840 con 0x7f9600106510 2024-09-19T08:53:53.373 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.279+0000 7f9604aa8640 1 -- 172.21.15.120:0/2769650838 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f95ec039200 msgr2=0x7f95ec03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.120:6800/3562882246 2024-09-19T08:53:53.373 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.279+0000 7f9604aa8640 1 --2- 172.21.15.120:0/2769650838 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f95ec039200 0x7f95ec03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:53:53.373 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.480+0000 7f9604aa8640 1 -- 172.21.15.120:0/2769650838 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f95ec039200 msgr2=0x7f95ec03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.120:6800/3562882246 2024-09-19T08:53:53.373 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.480+0000 7f9604aa8640 1 --2- 172.21.15.120:0/2769650838 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f95ec039200 0x7f95ec03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T08:53:53.373 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.881+0000 7f9604aa8640 1 -- 172.21.15.120:0/2769650838 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f95ec039200 msgr2=0x7f95ec03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.120:6800/3562882246 2024-09-19T08:53:53.374 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:49.881+0000 7f9604aa8640 1 --2- 172.21.15.120:0/2769650838 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f95ec039200 0x7f95ec03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T08:53:53.374 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:50.682+0000 7f9604aa8640 1 -- 172.21.15.120:0/2769650838 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f95ec039200 msgr2=0x7f95ec03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.120:6800/3562882246 2024-09-19T08:53:53.374 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:50.682+0000 7f9604aa8640 1 --2- 172.21.15.120:0/2769650838 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f95ec039200 0x7f95ec03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T08:53:53.374 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:51.530+0000 7f95f67fc640 1 -- 172.21.15.120:0/2769650838 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mgrmap(e 10) ==== 46241+0+0 (secure 0 0 0) 0x7f95fc06b2f0 con 0x7f9600106510 2024-09-19T08:53:53.374 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:51.530+0000 7f95f67fc640 1 -- 172.21.15.120:0/2769650838 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f95ec039200 msgr2=0x7f95ec03b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:53:53.374 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:51.530+0000 7f95f67fc640 1 --2- 172.21.15.120:0/2769650838 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f95ec039200 0x7f95ec03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.374 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:52.531+0000 7f95f67fc640 1 -- 172.21.15.120:0/2769650838 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7f95fc049cc0 con 0x7f9600106510 2024-09-19T08:53:53.374 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:52.532+0000 7f95f67fc640 1 --2- 172.21.15.120:0/2769650838 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f95ec03ccc0 0x7f95ec03f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:53.374 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:52.532+0000 7f95f67fc640 1 -- 172.21.15.120:0/2769650838 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f95fc02cbd0 con 0x7f95ec03ccc0 2024-09-19T08:53:53.374 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:52.532+0000 7f9604aa8640 1 --2- 172.21.15.120:0/2769650838 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f95ec03ccc0 0x7f95ec03f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:53.374 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:52.533+0000 7f9604aa8640 1 --2- 172.21.15.120:0/2769650838 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f95ec03ccc0 0x7f95ec03f0b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f95f0003980 tx=0x7f95f0006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:53.374 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:52.533+0000 7f95f67fc640 1 -- 172.21.15.120:0/2769650838 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f95fc02cbd0 con 0x7f95ec03ccc0 2024-09-19T08:53:53.374 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:52.546+0000 7f960750d640 1 -- 172.21.15.120:0/2769650838 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f960010abf0 con 0x7f95ec03ccc0 2024-09-19T08:53:53.375 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:52.546+0000 7f95f67fc640 1 -- 172.21.15.120:0/2769650838 <== mgr.14150 v2:172.21.15.120:6800/3841841273 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f960010abf0 con 0x7f95ec03ccc0 2024-09-19T08:53:53.375 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:52.546+0000 7f960750d640 1 -- 172.21.15.120:0/2769650838 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f95ec03ccc0 msgr2=0x7f95ec03f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:53.375 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:52.546+0000 7f960750d640 1 --2- 172.21.15.120:0/2769650838 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f95ec03ccc0 0x7f95ec03f0b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f95f0003980 tx=0x7f95f0006e10 comp rx=0 tx=0).stop 2024-09-19T08:53:53.375 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:52.546+0000 7f960750d640 1 -- 172.21.15.120:0/2769650838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9600106510 msgr2=0x7f960019b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:53.375 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:52.546+0000 7f960750d640 1 --2- 172.21.15.120:0/2769650838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9600106510 0x7f960019b650 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0097f0 tx=0x7f95fc0327b0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.375 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:52.546+0000 7f960750d640 1 -- 172.21.15.120:0/2769650838 shutdown_connections 2024-09-19T08:53:53.375 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:52.546+0000 7f960750d640 1 --2- 172.21.15.120:0/2769650838 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f95ec03ccc0 0x7f95ec03f0b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.375 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:52.546+0000 7f960750d640 1 --2- 172.21.15.120:0/2769650838 >> [v2:172.21.15.120:6800/3562882246,v1:172.21.15.120:6801/3562882246] conn(0x7f95ec039200 0x7f95ec03b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.375 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:52.546+0000 7f960750d640 1 --2- 172.21.15.120:0/2769650838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9600106510 0x7f960019b650 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.375 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:52.546+0000 7f960750d640 1 -- 172.21.15.120:0/2769650838 >> 172.21.15.120:0/2769650838 conn(0x7f9600101ca0 msgr2=0x7f9600102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:53.375 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:52.546+0000 7f960750d640 1 -- 172.21.15.120:0/2769650838 shutdown_connections 2024-09-19T08:53:53.375 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:52.546+0000 7f960750d640 1 -- 172.21.15.120:0/2769650838 wait complete. 2024-09-19T08:53:53.375 INFO:teuthology.orchestra.run.smithi120.stdout:mgr epoch 9 is available 2024-09-19T08:53:53.375 INFO:teuthology.orchestra.run.smithi120.stdout:Generating a dashboard self-signed certificate... 2024-09-19T08:53:53.741 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:53 smithi120 ceph-mon[31150]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-19T08:53:53.741 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:53 smithi120 ceph-mon[31150]: mgrmap e11: x(active, since 1.00578s) 2024-09-19T08:53:53.741 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:53 smithi120 ceph-mon[31150]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-19T08:53:54.633 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-19T08:53:54.633 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.741+0000 7fe7afcaa640 1 Processor -- start 2024-09-19T08:53:54.634 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.741+0000 7fe7afcaa640 1 -- start start 2024-09-19T08:53:54.634 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.741+0000 7fe7afcaa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7a81080d0 0x7fe7a810a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:54.634 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.741+0000 7fe7afcaa640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a807be70 con 0x7fe7a81080d0 2024-09-19T08:53:54.634 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.742+0000 7fe7ada46640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7a81080d0 0x7fe7a810a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:54.634 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.742+0000 7fe7ada46640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7a81080d0 0x7fe7a810a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38918/0 (socket says 172.21.15.120:38918) 2024-09-19T08:53:54.634 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.742+0000 7fe7ada46640 1 -- 172.21.15.120:0/1360380063 learned_addr learned my addr 172.21.15.120:0/1360380063 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:54.634 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.742+0000 7fe7ada46640 1 -- 172.21.15.120:0/1360380063 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7a810aa00 con 0x7fe7a81080d0 2024-09-19T08:53:54.634 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.743+0000 7fe7ada46640 1 --2- 172.21.15.120:0/1360380063 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7a81080d0 0x7fe7a810a4c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe798009e80 tx=0x7fe79802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9250f52c38df05fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:54.634 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.743+0000 7fe7aca44640 1 -- 172.21.15.120:0/1360380063 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe798036070 con 0x7fe7a81080d0 2024-09-19T08:53:54.634 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.743+0000 7fe7aca44640 1 -- 172.21.15.120:0/1360380063 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7980318e0 con 0x7fe7a81080d0 2024-09-19T08:53:54.634 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.743+0000 7fe7aca44640 1 -- 172.21.15.120:0/1360380063 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe798007a00 con 0x7fe7a81080d0 2024-09-19T08:53:54.634 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.743+0000 7fe7afcaa640 1 -- 172.21.15.120:0/1360380063 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7a81080d0 msgr2=0x7fe7a810a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:54.634 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.743+0000 7fe7afcaa640 1 --2- 172.21.15.120:0/1360380063 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7a81080d0 0x7fe7a810a4c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe798009e80 tx=0x7fe79802a7a0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.635 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.743+0000 7fe7afcaa640 1 -- 172.21.15.120:0/1360380063 shutdown_connections 2024-09-19T08:53:54.635 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.743+0000 7fe7afcaa640 1 --2- 172.21.15.120:0/1360380063 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7a81080d0 0x7fe7a810a4c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.635 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.743+0000 7fe7afcaa640 1 -- 172.21.15.120:0/1360380063 >> 172.21.15.120:0/1360380063 conn(0x7fe7a8101db0 msgr2=0x7fe7a81041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:54.635 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.744+0000 7fe7afcaa640 1 -- 172.21.15.120:0/1360380063 shutdown_connections 2024-09-19T08:53:54.635 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.744+0000 7fe7afcaa640 1 -- 172.21.15.120:0/1360380063 wait complete. 2024-09-19T08:53:54.635 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.744+0000 7fe7afcaa640 1 Processor -- start 2024-09-19T08:53:54.635 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.744+0000 7fe7afcaa640 1 -- start start 2024-09-19T08:53:54.635 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.744+0000 7fe7afcaa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7a81080d0 0x7fe7a819b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:54.635 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.744+0000 7fe7afcaa640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a810ba80 con 0x7fe7a81080d0 2024-09-19T08:53:54.635 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.744+0000 7fe7ada46640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7a81080d0 0x7fe7a819b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:54.635 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.744+0000 7fe7ada46640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7a81080d0 0x7fe7a819b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38930/0 (socket says 172.21.15.120:38930) 2024-09-19T08:53:54.635 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.744+0000 7fe7ada46640 1 -- 172.21.15.120:0/1071929477 learned_addr learned my addr 172.21.15.120:0/1071929477 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:54.635 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.745+0000 7fe7ada46640 1 -- 172.21.15.120:0/1071929477 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7a819bb70 con 0x7fe7a81080d0 2024-09-19T08:53:54.635 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.745+0000 7fe7ada46640 1 --2- 172.21.15.120:0/1071929477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7a81080d0 0x7fe7a819b630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe798031cf0 tx=0x7fe798031d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:54.636 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.745+0000 7fe7a6ffd640 1 -- 172.21.15.120:0/1071929477 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe798036050 con 0x7fe7a81080d0 2024-09-19T08:53:54.636 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.745+0000 7fe7a6ffd640 1 -- 172.21.15.120:0/1071929477 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe798042070 con 0x7fe7a81080d0 2024-09-19T08:53:54.636 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.745+0000 7fe7afcaa640 1 -- 172.21.15.120:0/1071929477 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7a819be00 con 0x7fe7a81080d0 2024-09-19T08:53:54.636 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.745+0000 7fe7a6ffd640 1 -- 172.21.15.120:0/1071929477 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe79803d9b0 con 0x7fe7a81080d0 2024-09-19T08:53:54.636 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.745+0000 7fe7afcaa640 1 -- 172.21.15.120:0/1071929477 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7a819eaf0 con 0x7fe7a81080d0 2024-09-19T08:53:54.636 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.746+0000 7fe7a6ffd640 1 -- 172.21.15.120:0/1071929477 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fe79803dc60 con 0x7fe7a81080d0 2024-09-19T08:53:54.636 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.746+0000 7fe7a6ffd640 1 --2- 172.21.15.120:0/1071929477 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe790039200 0x7fe79003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:54.636 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.746+0000 7fe7a6ffd640 1 -- 172.21.15.120:0/1071929477 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe79806d8b0 con 0x7fe7a81080d0 2024-09-19T08:53:54.636 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.746+0000 7fe7ad245640 1 --2- 172.21.15.120:0/1071929477 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe790039200 0x7fe79003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:54.636 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.746+0000 7fe7afcaa640 1 -- 172.21.15.120:0/1071929477 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7a8108530 con 0x7fe7a81080d0 2024-09-19T08:53:54.636 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.746+0000 7fe7ad245640 1 --2- 172.21.15.120:0/1071929477 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe790039200 0x7fe79003b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe79c0098c0 tx=0x7fe79c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:54.636 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.751+0000 7fe7a6ffd640 1 -- 172.21.15.120:0/1071929477 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe798044310 con 0x7fe7a81080d0 2024-09-19T08:53:54.636 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:53.927+0000 7fe7afcaa640 1 -- 172.21.15.120:0/1071929477 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fe7a8103000 con 0x7fe790039200 2024-09-19T08:53:54.637 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:54.018+0000 7fe7a6ffd640 1 -- 172.21.15.120:0/1071929477 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fe7a8103000 con 0x7fe790039200 2024-09-19T08:53:54.637 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:54.020+0000 7fe7afcaa640 1 -- 172.21.15.120:0/1071929477 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe790039200 msgr2=0x7fe79003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:54.637 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:54.020+0000 7fe7afcaa640 1 --2- 172.21.15.120:0/1071929477 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe790039200 0x7fe79003b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe79c0098c0 tx=0x7fe79c006db0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.637 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:54.020+0000 7fe7afcaa640 1 -- 172.21.15.120:0/1071929477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7a81080d0 msgr2=0x7fe7a819b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:54.637 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:54.020+0000 7fe7afcaa640 1 --2- 172.21.15.120:0/1071929477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7a81080d0 0x7fe7a819b630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe798031cf0 tx=0x7fe798031d20 comp rx=0 tx=0).stop 2024-09-19T08:53:54.637 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:54.020+0000 7fe7afcaa640 1 -- 172.21.15.120:0/1071929477 shutdown_connections 2024-09-19T08:53:54.637 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:54.020+0000 7fe7afcaa640 1 --2- 172.21.15.120:0/1071929477 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe790039200 0x7fe79003b6c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.637 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:54.020+0000 7fe7afcaa640 1 --2- 172.21.15.120:0/1071929477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7a81080d0 0x7fe7a819b630 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.637 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:54.020+0000 7fe7afcaa640 1 -- 172.21.15.120:0/1071929477 >> 172.21.15.120:0/1071929477 conn(0x7fe7a8101db0 msgr2=0x7fe7a81028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:54.637 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:54.020+0000 7fe7afcaa640 1 -- 172.21.15.120:0/1071929477 shutdown_connections 2024-09-19T08:53:54.637 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:54.020+0000 7fe7afcaa640 1 -- 172.21.15.120:0/1071929477 wait complete. 2024-09-19T08:53:54.637 INFO:teuthology.orchestra.run.smithi120.stdout:Creating initial admin user... 2024-09-19T08:53:54.753 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:54 smithi120 ceph-mon[31150]: [19/Sep/2024:08:53:52] ENGINE Bus STARTING 2024-09-19T08:53:54.753 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:54 smithi120 ceph-mon[31150]: [19/Sep/2024:08:53:52] ENGINE Serving on http://172.21.15.120:8765 2024-09-19T08:53:54.753 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:54 smithi120 ceph-mon[31150]: [19/Sep/2024:08:53:53] ENGINE Serving on https://172.21.15.120:7150 2024-09-19T08:53:54.753 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:54 smithi120 ceph-mon[31150]: [19/Sep/2024:08:53:53] ENGINE Bus STARTED 2024-09-19T08:53:54.753 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:54 smithi120 ceph-mon[31150]: [19/Sep/2024:08:53:53] ENGINE Error in HTTPServer.serve 2024-09-19T08:53:54.753 INFO:journalctl@ceph.mon.a.smithi120.stdout: Traceback (most recent call last): 2024-09-19T08:53:54.753 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T08:53:54.753 INFO:journalctl@ceph.mon.a.smithi120.stdout: self._connections.run(self.expiration_interval) 2024-09-19T08:53:54.753 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T08:53:54.753 INFO:journalctl@ceph.mon.a.smithi120.stdout: self._run(expiration_interval) 2024-09-19T08:53:54.753 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T08:53:54.754 INFO:journalctl@ceph.mon.a.smithi120.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T08:53:54.754 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T08:53:54.754 INFO:journalctl@ceph.mon.a.smithi120.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T08:53:54.754 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T08:53:54.754 INFO:journalctl@ceph.mon.a.smithi120.stdout: s = self.context.wrap_socket( 2024-09-19T08:53:54.754 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T08:53:54.754 INFO:journalctl@ceph.mon.a.smithi120.stdout: return self.sslsocket_class._create( 2024-09-19T08:53:54.754 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T08:53:54.754 INFO:journalctl@ceph.mon.a.smithi120.stdout: self.do_handshake() 2024-09-19T08:53:54.754 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T08:53:54.754 INFO:journalctl@ceph.mon.a.smithi120.stdout: self._sslobj.do_handshake() 2024-09-19T08:53:54.754 INFO:journalctl@ceph.mon.a.smithi120.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T08:53:54.754 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:54 smithi120 ceph-mon[31150]: mgrmap e12: x(active, since 2s) 2024-09-19T08:53:54.754 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:54 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:53:54.754 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:54 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:53:54.755 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:54 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:53:56.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:55 smithi120 ceph-mon[31150]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:53:56.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:55 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:53:56.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:55 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:53:56.782 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:56 smithi120 ceph-mon[31150]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:53:56.790 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$J3fxmOcQcKxFjCruNPpvXOMFsCeUk90yOXmxD3VM69FAj.rzKDOiO", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726736035, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-19T08:53:56.790 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.079+0000 7f14ddbcd640 1 Processor -- start 2024-09-19T08:53:56.790 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.079+0000 7f14ddbcd640 1 -- start start 2024-09-19T08:53:56.790 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.079+0000 7f14ddbcd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14d807de20 0x7f14d807c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:56.790 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.079+0000 7f14ddbcd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f14d807c860 con 0x7f14d807de20 2024-09-19T08:53:56.790 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.079+0000 7f14d77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14d807de20 0x7f14d807c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:56.790 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.079+0000 7f14d77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14d807de20 0x7f14d807c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38934/0 (socket says 172.21.15.120:38934) 2024-09-19T08:53:56.790 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.079+0000 7f14d77fe640 1 -- 172.21.15.120:0/206548364 learned_addr learned my addr 172.21.15.120:0/206548364 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:56.790 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.080+0000 7f14d77fe640 1 -- 172.21.15.120:0/206548364 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d807c9e0 con 0x7f14d807de20 2024-09-19T08:53:56.791 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.080+0000 7f14d77fe640 1 --2- 172.21.15.120:0/206548364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14d807de20 0x7f14d807c320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009820 tx=0x7f14c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b46bc89b4f05e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:56.791 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.080+0000 7f14d6ffd640 1 -- 172.21.15.120:0/206548364 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14c8037070 con 0x7f14d807de20 2024-09-19T08:53:56.791 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.080+0000 7f14d6ffd640 1 -- 172.21.15.120:0/206548364 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c8032920 con 0x7f14d807de20 2024-09-19T08:53:56.791 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.080+0000 7f14d6ffd640 1 -- 172.21.15.120:0/206548364 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14c8030810 con 0x7f14d807de20 2024-09-19T08:53:56.791 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.081+0000 7f14ddbcd640 1 -- 172.21.15.120:0/206548364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14d807de20 msgr2=0x7f14d807c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:56.791 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.081+0000 7f14ddbcd640 1 --2- 172.21.15.120:0/206548364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14d807de20 0x7f14d807c320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009820 tx=0x7f14c802a270 comp rx=0 tx=0).stop 2024-09-19T08:53:56.791 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.081+0000 7f14ddbcd640 1 -- 172.21.15.120:0/206548364 shutdown_connections 2024-09-19T08:53:56.791 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.081+0000 7f14ddbcd640 1 --2- 172.21.15.120:0/206548364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14d807de20 0x7f14d807c320 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:56.791 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.081+0000 7f14ddbcd640 1 -- 172.21.15.120:0/206548364 >> 172.21.15.120:0/206548364 conn(0x7f14d8102eb0 msgr2=0x7f14d8105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:56.791 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.081+0000 7f14ddbcd640 1 -- 172.21.15.120:0/206548364 shutdown_connections 2024-09-19T08:53:56.791 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.081+0000 7f14ddbcd640 1 -- 172.21.15.120:0/206548364 wait complete. 2024-09-19T08:53:56.791 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.082+0000 7f14ddbcd640 1 Processor -- start 2024-09-19T08:53:56.791 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.082+0000 7f14ddbcd640 1 -- start start 2024-09-19T08:53:56.791 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.082+0000 7f14ddbcd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14d807de20 0x7f14d819d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:56.792 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.082+0000 7f14ddbcd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f14d810dcc0 con 0x7f14d807de20 2024-09-19T08:53:56.792 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.082+0000 7f14d77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14d807de20 0x7f14d819d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:56.792 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.082+0000 7f14d77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14d807de20 0x7f14d819d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38950/0 (socket says 172.21.15.120:38950) 2024-09-19T08:53:56.792 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.082+0000 7f14d77fe640 1 -- 172.21.15.120:0/2901358627 learned_addr learned my addr 172.21.15.120:0/2901358627 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:56.792 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.082+0000 7f14d77fe640 1 -- 172.21.15.120:0/2901358627 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d819ddd0 con 0x7f14d807de20 2024-09-19T08:53:56.792 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.082+0000 7f14d77fe640 1 --2- 172.21.15.120:0/2901358627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14d807de20 0x7f14d819d890 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f14c80097f0 tx=0x7f14c80327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:56.792 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.082+0000 7f14d57fa640 1 -- 172.21.15.120:0/2901358627 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14c8037040 con 0x7f14d807de20 2024-09-19T08:53:56.792 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.082+0000 7f14d57fa640 1 -- 172.21.15.120:0/2901358627 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c8030c40 con 0x7f14d807de20 2024-09-19T08:53:56.792 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.083+0000 7f14ddbcd640 1 -- 172.21.15.120:0/2901358627 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d819e060 con 0x7f14d807de20 2024-09-19T08:53:56.792 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.083+0000 7f14d57fa640 1 -- 172.21.15.120:0/2901358627 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14c803a5d0 con 0x7f14d807de20 2024-09-19T08:53:56.792 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.083+0000 7f14ddbcd640 1 -- 172.21.15.120:0/2901358627 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d81a0d50 con 0x7f14d807de20 2024-09-19T08:53:56.792 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.083+0000 7f14d57fa640 1 -- 172.21.15.120:0/2901358627 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f14c803b3f0 con 0x7f14d807de20 2024-09-19T08:53:56.792 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.083+0000 7f14ddbcd640 1 -- 172.21.15.120:0/2901358627 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14d807c3a0 con 0x7f14d807de20 2024-09-19T08:53:56.793 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.084+0000 7f14d57fa640 1 --2- 172.21.15.120:0/2901358627 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f14c4039200 0x7f14c403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:56.793 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.084+0000 7f14d57fa640 1 -- 172.21.15.120:0/2901358627 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f14c806c880 con 0x7f14d807de20 2024-09-19T08:53:56.793 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.084+0000 7f14cffff640 1 --2- 172.21.15.120:0/2901358627 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f14c4039200 0x7f14c403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:56.793 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.084+0000 7f14cffff640 1 --2- 172.21.15.120:0/2901358627 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f14c4039200 0x7f14c403b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f14c00098c0 tx=0x7f14c0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:56.793 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.088+0000 7f14d57fa640 1 -- 172.21.15.120:0/2901358627 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f14c80313a0 con 0x7f14d807de20 2024-09-19T08:53:56.793 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.275+0000 7f14ddbcd640 1 -- 172.21.15.120:0/2901358627 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f14d81a0f20 con 0x7f14c4039200 2024-09-19T08:53:56.793 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.499+0000 7f14d57fa640 1 -- 172.21.15.120:0/2901358627 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f14d81a0f20 con 0x7f14c4039200 2024-09-19T08:53:56.793 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.501+0000 7f14ddbcd640 1 -- 172.21.15.120:0/2901358627 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f14c4039200 msgr2=0x7f14c403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:56.793 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.501+0000 7f14ddbcd640 1 --2- 172.21.15.120:0/2901358627 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f14c4039200 0x7f14c403b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f14c00098c0 tx=0x7f14c0006db0 comp rx=0 tx=0).stop 2024-09-19T08:53:56.793 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.501+0000 7f14ddbcd640 1 -- 172.21.15.120:0/2901358627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14d807de20 msgr2=0x7f14d819d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:56.793 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.501+0000 7f14ddbcd640 1 --2- 172.21.15.120:0/2901358627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14d807de20 0x7f14d819d890 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f14c80097f0 tx=0x7f14c80327b0 comp rx=0 tx=0).stop 2024-09-19T08:53:56.793 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.501+0000 7f14ddbcd640 1 -- 172.21.15.120:0/2901358627 shutdown_connections 2024-09-19T08:53:56.793 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.501+0000 7f14ddbcd640 1 --2- 172.21.15.120:0/2901358627 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f14c4039200 0x7f14c403b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:56.794 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.501+0000 7f14ddbcd640 1 --2- 172.21.15.120:0/2901358627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14d807de20 0x7f14d819d890 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:56.794 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.501+0000 7f14ddbcd640 1 -- 172.21.15.120:0/2901358627 >> 172.21.15.120:0/2901358627 conn(0x7f14d8102eb0 msgr2=0x7f14d8103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:56.794 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.501+0000 7f14ddbcd640 1 -- 172.21.15.120:0/2901358627 shutdown_connections 2024-09-19T08:53:56.794 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:55.501+0000 7f14ddbcd640 1 -- 172.21.15.120:0/2901358627 wait complete. 2024-09-19T08:53:56.794 INFO:teuthology.orchestra.run.smithi120.stdout:Fetching dashboard port number... 2024-09-19T08:53:57.989 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:57 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1025539510' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-19T08:53:57.997 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 8443 2024-09-19T08:53:57.997 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.247+0000 7f1dbce60640 1 Processor -- start 2024-09-19T08:53:57.997 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.247+0000 7f1dbce60640 1 -- start start 2024-09-19T08:53:57.997 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.248+0000 7f1dbce60640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1db8107630 0x7f1db8107a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:57.997 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.248+0000 7f1dbce60640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1db8108000 con 0x7f1db8107630 2024-09-19T08:53:57.997 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.248+0000 7f1db659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1db8107630 0x7f1db8107a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:57.997 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.248+0000 7f1db659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1db8107630 0x7f1db8107a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38960/0 (socket says 172.21.15.120:38960) 2024-09-19T08:53:57.997 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.248+0000 7f1db659c640 1 -- 172.21.15.120:0/2817534258 learned_addr learned my addr 172.21.15.120:0/2817534258 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:57.997 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.248+0000 7f1db659c640 1 -- 172.21.15.120:0/2817534258 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db8108830 con 0x7f1db8107630 2024-09-19T08:53:57.997 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.249+0000 7f1db659c640 1 --2- 172.21.15.120:0/2817534258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1db8107630 0x7f1db8107a30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1da8009e80 tx=0x7f1da802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5ab3f828509c6e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:57.998 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.249+0000 7f1db559a640 1 -- 172.21.15.120:0/2817534258 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1da8036070 con 0x7f1db8107630 2024-09-19T08:53:57.998 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.249+0000 7f1db559a640 1 -- 172.21.15.120:0/2817534258 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1da80318e0 con 0x7f1db8107630 2024-09-19T08:53:57.998 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.249+0000 7f1db559a640 1 -- 172.21.15.120:0/2817534258 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1da8007a00 con 0x7f1db8107630 2024-09-19T08:53:57.998 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.249+0000 7f1dbce60640 1 -- 172.21.15.120:0/2817534258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1db8107630 msgr2=0x7f1db8107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:57.998 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.249+0000 7f1dbce60640 1 --2- 172.21.15.120:0/2817534258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1db8107630 0x7f1db8107a30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1da8009e80 tx=0x7f1da802a7a0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.998 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.250+0000 7f1dbce60640 1 -- 172.21.15.120:0/2817534258 shutdown_connections 2024-09-19T08:53:57.998 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.250+0000 7f1dbce60640 1 --2- 172.21.15.120:0/2817534258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1db8107630 0x7f1db8107a30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.998 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.250+0000 7f1dbce60640 1 -- 172.21.15.120:0/2817534258 >> 172.21.15.120:0/2817534258 conn(0x7f1db8102de0 msgr2=0x7f1db8105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:57.998 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.250+0000 7f1dbce60640 1 -- 172.21.15.120:0/2817534258 shutdown_connections 2024-09-19T08:53:57.998 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.250+0000 7f1dbce60640 1 -- 172.21.15.120:0/2817534258 wait complete. 2024-09-19T08:53:57.998 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.250+0000 7f1dbce60640 1 Processor -- start 2024-09-19T08:53:57.998 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.250+0000 7f1dbce60640 1 -- start start 2024-09-19T08:53:57.998 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.250+0000 7f1dbce60640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1db8107630 0x7f1db807db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:57.998 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.250+0000 7f1dbce60640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1db810bbe0 con 0x7f1db8107630 2024-09-19T08:53:57.999 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.251+0000 7f1db659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1db8107630 0x7f1db807db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:57.999 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.251+0000 7f1db659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1db8107630 0x7f1db807db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38964/0 (socket says 172.21.15.120:38964) 2024-09-19T08:53:57.999 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.251+0000 7f1db659c640 1 -- 172.21.15.120:0/1025539510 learned_addr learned my addr 172.21.15.120:0/1025539510 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:57.999 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.251+0000 7f1db659c640 1 -- 172.21.15.120:0/1025539510 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db807e0b0 con 0x7f1db8107630 2024-09-19T08:53:57.999 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.251+0000 7f1db659c640 1 --2- 172.21.15.120:0/1025539510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1db8107630 0x7f1db807db70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1da802acc0 tx=0x7f1da8031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:57.999 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.251+0000 7f1daf7fe640 1 -- 172.21.15.120:0/1025539510 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1da8036050 con 0x7f1db8107630 2024-09-19T08:53:57.999 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.251+0000 7f1daf7fe640 1 -- 172.21.15.120:0/1025539510 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1da8037040 con 0x7f1db8107630 2024-09-19T08:53:57.999 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.251+0000 7f1dbce60640 1 -- 172.21.15.120:0/1025539510 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db807c2a0 con 0x7f1db8107630 2024-09-19T08:53:57.999 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.251+0000 7f1daf7fe640 1 -- 172.21.15.120:0/1025539510 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1da803b920 con 0x7f1db8107630 2024-09-19T08:53:57.999 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.251+0000 7f1dbce60640 1 -- 172.21.15.120:0/1025539510 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db807c7e0 con 0x7f1db8107630 2024-09-19T08:53:57.999 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.252+0000 7f1daf7fe640 1 -- 172.21.15.120:0/1025539510 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f1da8044020 con 0x7f1db8107630 2024-09-19T08:53:57.999 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.252+0000 7f1daf7fe640 1 --2- 172.21.15.120:0/1025539510 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1da4039200 0x7f1da403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:57.999 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.252+0000 7f1daf7fe640 1 -- 172.21.15.120:0/1025539510 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1da8042020 con 0x7f1db8107630 2024-09-19T08:53:57.999 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.252+0000 7f1dbce60640 1 -- 172.21.15.120:0/1025539510 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1db810bd60 con 0x7f1db8107630 2024-09-19T08:53:58.000 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.252+0000 7f1db5d9b640 1 --2- 172.21.15.120:0/1025539510 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1da4039200 0x7f1da403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:58.000 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.253+0000 7f1db5d9b640 1 --2- 172.21.15.120:0/1025539510 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1da4039200 0x7f1da403b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1da00098c0 tx=0x7f1da0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:58.000 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.257+0000 7f1daf7fe640 1 -- 172.21.15.120:0/1025539510 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1da804b020 con 0x7f1db8107630 2024-09-19T08:53:58.000 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.427+0000 7f1dbce60640 1 -- 172.21.15.120:0/1025539510 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f1db81043e0 con 0x7f1db8107630 2024-09-19T08:53:58.000 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.428+0000 7f1daf7fe640 1 -- 172.21.15.120:0/1025539510 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f1da8041c10 con 0x7f1db8107630 2024-09-19T08:53:58.000 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.430+0000 7f1dbce60640 1 -- 172.21.15.120:0/1025539510 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1da4039200 msgr2=0x7f1da403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:58.000 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.430+0000 7f1dbce60640 1 --2- 172.21.15.120:0/1025539510 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1da4039200 0x7f1da403b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1da00098c0 tx=0x7f1da0006db0 comp rx=0 tx=0).stop 2024-09-19T08:53:58.000 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.430+0000 7f1dbce60640 1 -- 172.21.15.120:0/1025539510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1db8107630 msgr2=0x7f1db807db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:58.000 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.430+0000 7f1dbce60640 1 --2- 172.21.15.120:0/1025539510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1db8107630 0x7f1db807db70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1da802acc0 tx=0x7f1da8031cc0 comp rx=0 tx=0).stop 2024-09-19T08:53:58.000 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.430+0000 7f1dbce60640 1 -- 172.21.15.120:0/1025539510 shutdown_connections 2024-09-19T08:53:58.000 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.430+0000 7f1dbce60640 1 --2- 172.21.15.120:0/1025539510 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1da4039200 0x7f1da403b6c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:58.000 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.430+0000 7f1dbce60640 1 --2- 172.21.15.120:0/1025539510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1db8107630 0x7f1db807db70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:58.000 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.430+0000 7f1dbce60640 1 -- 172.21.15.120:0/1025539510 >> 172.21.15.120:0/1025539510 conn(0x7f1db8102de0 msgr2=0x7f1db8103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:58.001 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.430+0000 7f1dbce60640 1 -- 172.21.15.120:0/1025539510 shutdown_connections 2024-09-19T08:53:58.001 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:57.430+0000 7f1dbce60640 1 -- 172.21.15.120:0/1025539510 wait complete. 2024-09-19T08:53:58.005 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T08:53:58.006 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout disabled 2024-09-19T08:53:58.013 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T08:53:58.013 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout inactive 2024-09-19T08:53:58.014 INFO:teuthology.orchestra.run.smithi120.stdout:firewalld.service is not enabled 2024-09-19T08:53:58.014 INFO:teuthology.orchestra.run.smithi120.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-19T08:53:58.015 INFO:teuthology.orchestra.run.smithi120.stdout:Ceph Dashboard is now available at: 2024-09-19T08:53:58.015 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:53:58.015 INFO:teuthology.orchestra.run.smithi120.stdout: URL: https://smithi120.front.sepia.ceph.com:8443/ 2024-09-19T08:53:58.015 INFO:teuthology.orchestra.run.smithi120.stdout: User: admin 2024-09-19T08:53:58.015 INFO:teuthology.orchestra.run.smithi120.stdout: Password: e5a6gvppy1 2024-09-19T08:53:58.015 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:53:58.016 INFO:teuthology.orchestra.run.smithi120.stdout:Saving cluster configuration to /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/config directory 2024-09-19T08:53:59.004 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:58 smithi120 ceph-mon[31150]: mgrmap e13: x(active, since 6s) 2024-09-19T08:53:59.288 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.439+0000 7fa8d5f12640 1 Processor -- start 2024-09-19T08:53:59.288 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.439+0000 7fa8d5f12640 1 -- start start 2024-09-19T08:53:59.288 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.439+0000 7fa8d5f12640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8d0105940 0x7fa8d0105d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:59.288 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.439+0000 7fa8d5f12640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d0106310 con 0x7fa8d0105940 2024-09-19T08:53:59.288 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.439+0000 7fa8cf7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8d0105940 0x7fa8d0105d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:59.288 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.439+0000 7fa8cf7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8d0105940 0x7fa8d0105d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38966/0 (socket says 172.21.15.120:38966) 2024-09-19T08:53:59.288 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.439+0000 7fa8cf7fe640 1 -- 172.21.15.120:0/2414116330 learned_addr learned my addr 172.21.15.120:0/2414116330 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:59.288 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.440+0000 7fa8cf7fe640 1 -- 172.21.15.120:0/2414116330 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d0108920 con 0x7fa8d0105940 2024-09-19T08:53:59.288 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.440+0000 7fa8cf7fe640 1 --2- 172.21.15.120:0/2414116330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8d0105940 0x7fa8d0105d40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009820 tx=0x7fa8c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13292e9f9cea688 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:59.288 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.440+0000 7fa8ceffd640 1 -- 172.21.15.120:0/2414116330 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8c0037070 con 0x7fa8d0105940 2024-09-19T08:53:59.289 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.440+0000 7fa8ceffd640 1 -- 172.21.15.120:0/2414116330 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8c0032920 con 0x7fa8d0105940 2024-09-19T08:53:59.289 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.440+0000 7fa8ceffd640 1 -- 172.21.15.120:0/2414116330 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8c0030810 con 0x7fa8d0105940 2024-09-19T08:53:59.289 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.441+0000 7fa8d5f12640 1 -- 172.21.15.120:0/2414116330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8d0105940 msgr2=0x7fa8d0105d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:59.289 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.441+0000 7fa8d5f12640 1 --2- 172.21.15.120:0/2414116330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8d0105940 0x7fa8d0105d40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009820 tx=0x7fa8c002a270 comp rx=0 tx=0).stop 2024-09-19T08:53:59.289 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.441+0000 7fa8d5f12640 1 -- 172.21.15.120:0/2414116330 shutdown_connections 2024-09-19T08:53:59.289 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.441+0000 7fa8d5f12640 1 --2- 172.21.15.120:0/2414116330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8d0105940 0x7fa8d0105d40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.289 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.441+0000 7fa8d5f12640 1 -- 172.21.15.120:0/2414116330 >> 172.21.15.120:0/2414116330 conn(0x7fa8d0101550 msgr2=0x7fa8d0103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:59.289 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.441+0000 7fa8d5f12640 1 -- 172.21.15.120:0/2414116330 shutdown_connections 2024-09-19T08:53:59.289 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.441+0000 7fa8d5f12640 1 -- 172.21.15.120:0/2414116330 wait complete. 2024-09-19T08:53:59.289 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.442+0000 7fa8d5f12640 1 Processor -- start 2024-09-19T08:53:59.289 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.442+0000 7fa8d5f12640 1 -- start start 2024-09-19T08:53:59.289 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.442+0000 7fa8d5f12640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8d0105940 0x7fa8d019b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:59.289 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.442+0000 7fa8d5f12640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d010bb50 con 0x7fa8d0105940 2024-09-19T08:53:59.289 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.442+0000 7fa8cf7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8d0105940 0x7fa8d019b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:59.290 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.442+0000 7fa8cf7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8d0105940 0x7fa8d019b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38972/0 (socket says 172.21.15.120:38972) 2024-09-19T08:53:59.290 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.442+0000 7fa8cf7fe640 1 -- 172.21.15.120:0/73621367 learned_addr learned my addr 172.21.15.120:0/73621367 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:53:59.290 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.442+0000 7fa8cf7fe640 1 -- 172.21.15.120:0/73621367 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d019bb50 con 0x7fa8d0105940 2024-09-19T08:53:59.290 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.442+0000 7fa8cf7fe640 1 --2- 172.21.15.120:0/73621367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8d0105940 0x7fa8d019b610 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0030560 tx=0x7fa8c0032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:59.290 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.442+0000 7fa8cd7fa640 1 -- 172.21.15.120:0/73621367 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8c0037040 con 0x7fa8d0105940 2024-09-19T08:53:59.290 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.442+0000 7fa8cd7fa640 1 -- 172.21.15.120:0/73621367 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8c0040070 con 0x7fa8d0105940 2024-09-19T08:53:59.290 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.442+0000 7fa8d5f12640 1 -- 172.21.15.120:0/73621367 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8d019bde0 con 0x7fa8d0105940 2024-09-19T08:53:59.290 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.442+0000 7fa8cd7fa640 1 -- 172.21.15.120:0/73621367 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8c003a740 con 0x7fa8d0105940 2024-09-19T08:53:59.290 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.443+0000 7fa8d5f12640 1 -- 172.21.15.120:0/73621367 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8d019ead0 con 0x7fa8d0105940 2024-09-19T08:53:59.290 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.443+0000 7fa8cd7fa640 1 -- 172.21.15.120:0/73621367 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fa8c003a9b0 con 0x7fa8d0105940 2024-09-19T08:53:59.290 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.443+0000 7fa8cd7fa640 1 --2- 172.21.15.120:0/73621367 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa8bc0392a0 0x7fa8bc03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:59.290 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.444+0000 7fa8cd7fa640 1 -- 172.21.15.120:0/73621367 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa8c006cac0 con 0x7fa8d0105940 2024-09-19T08:53:59.290 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.444+0000 7fa8d5f12640 1 -- 172.21.15.120:0/73621367 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8d0105dc0 con 0x7fa8d0105940 2024-09-19T08:53:59.291 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.444+0000 7fa8c7fff640 1 --2- 172.21.15.120:0/73621367 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa8bc0392a0 0x7fa8bc03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:59.291 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.444+0000 7fa8c7fff640 1 --2- 172.21.15.120:0/73621367 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa8bc0392a0 0x7fa8bc03b760 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80098c0 tx=0x7fa8b8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:59.291 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.448+0000 7fa8cd7fa640 1 -- 172.21.15.120:0/73621367 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa8c0039970 con 0x7fa8d0105940 2024-09-19T08:53:59.291 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.703+0000 7fa8d5f12640 1 -- 172.21.15.120:0/73621367 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fa8d019ef60 con 0x7fa8d0105940 2024-09-19T08:53:59.291 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.705+0000 7fa8cd7fa640 1 -- 172.21.15.120:0/73621367 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fa8c0044c70 con 0x7fa8d0105940 2024-09-19T08:53:59.291 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-19T08:53:59.291 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.707+0000 7fa8d5f12640 1 -- 172.21.15.120:0/73621367 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa8bc0392a0 msgr2=0x7fa8bc03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:59.291 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.707+0000 7fa8d5f12640 1 --2- 172.21.15.120:0/73621367 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa8bc0392a0 0x7fa8bc03b760 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80098c0 tx=0x7fa8b8006db0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.291 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.707+0000 7fa8d5f12640 1 -- 172.21.15.120:0/73621367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8d0105940 msgr2=0x7fa8d019b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:59.291 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.707+0000 7fa8d5f12640 1 --2- 172.21.15.120:0/73621367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8d0105940 0x7fa8d019b610 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0030560 tx=0x7fa8c0032670 comp rx=0 tx=0).stop 2024-09-19T08:53:59.291 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.707+0000 7fa8d5f12640 1 -- 172.21.15.120:0/73621367 shutdown_connections 2024-09-19T08:53:59.291 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.707+0000 7fa8d5f12640 1 --2- 172.21.15.120:0/73621367 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa8bc0392a0 0x7fa8bc03b760 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.291 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.707+0000 7fa8d5f12640 1 --2- 172.21.15.120:0/73621367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8d0105940 0x7fa8d019b610 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.291 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.707+0000 7fa8d5f12640 1 -- 172.21.15.120:0/73621367 >> 172.21.15.120:0/73621367 conn(0x7fa8d0101550 msgr2=0x7fa8d01020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:59.291 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.707+0000 7fa8d5f12640 1 -- 172.21.15.120:0/73621367 shutdown_connections 2024-09-19T08:53:59.292 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr 2024-09-19T08:53:58.707+0000 7fa8d5f12640 1 -- 172.21.15.120:0/73621367 wait complete. 2024-09-19T08:53:59.292 INFO:teuthology.orchestra.run.smithi120.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-19T08:53:59.292 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:53:59.292 INFO:teuthology.orchestra.run.smithi120.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-19T08:53:59.292 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:53:59.292 INFO:teuthology.orchestra.run.smithi120.stdout:Or, if you are only running a single cluster on this host: 2024-09-19T08:53:59.292 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:53:59.292 INFO:teuthology.orchestra.run.smithi120.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-19T08:53:59.292 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:53:59.292 INFO:teuthology.orchestra.run.smithi120.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-19T08:53:59.292 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:53:59.292 INFO:teuthology.orchestra.run.smithi120.stdout: ceph telemetry on 2024-09-19T08:53:59.292 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:53:59.292 INFO:teuthology.orchestra.run.smithi120.stdout:For more information see: 2024-09-19T08:53:59.293 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:53:59.293 INFO:teuthology.orchestra.run.smithi120.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-19T08:53:59.293 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:53:59.293 INFO:teuthology.orchestra.run.smithi120.stdout:Bootstrap complete. 2024-09-19T08:53:59.327 INFO:tasks.cephadm:Fetching config... 2024-09-19T08:53:59.327 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-19T08:53:59.327 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-19T08:53:59.343 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-19T08:53:59.344 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-19T08:53:59.344 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-19T08:53:59.399 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-19T08:53:59.399 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-19T08:53:59.399 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-19T08:53:59.464 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-19T08:53:59.464 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-19T08:53:59.464 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-19T08:53:59.519 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-19T08:53:59.520 DEBUG:teuthology.orchestra.run.smithi120:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCVS0BoZyzl2d4UZYb6pmtEVyd73DYS+X46xalomhMZ4cerC1Q4l7Cmk33c9R6V/Gch2MIAHQhNER/jjEDkW+PL22CwOz4QJgzLu1Faa/0hLB557rYfA12X29w2SDQskctQEbVfUXTFPUoOS18bwC5zqkze4jIi9SsXC2qVxTPTlmc4kxPTjau7FgotPgHIAC9mrn0gI9/StdxqLWzr7+YkFKCOqqKfec11elluXZHpdm3CUhfivZ219JI1S0D4AJwmRdIHOTRfd8mxfW88w0OWIMp38dEZnAvidy8Fl/cRC0JnxUhK5NN3qWoCueJS6uExfd2E8bAoFDa6BJN9+mP3cownwirkbs03yPAOCfzwINXyMxIZIl5WBW57SCfWQTtT0KpC4cNKuzs1potGatswqL3lRjDloGGGhyLsZbbOEWkNrBIJGg3eYvh9Xg2yZIfJXEhacPFv4iQw0lh6RcldSXztTEiFEKDUTp0tXbnPfkyDmgSNlVCgYDh+3PnrOHs= ceph-507393d2-7664-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T08:53:59.593 INFO:teuthology.orchestra.run.smithi120.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCVS0BoZyzl2d4UZYb6pmtEVyd73DYS+X46xalomhMZ4cerC1Q4l7Cmk33c9R6V/Gch2MIAHQhNER/jjEDkW+PL22CwOz4QJgzLu1Faa/0hLB557rYfA12X29w2SDQskctQEbVfUXTFPUoOS18bwC5zqkze4jIi9SsXC2qVxTPTlmc4kxPTjau7FgotPgHIAC9mrn0gI9/StdxqLWzr7+YkFKCOqqKfec11elluXZHpdm3CUhfivZ219JI1S0D4AJwmRdIHOTRfd8mxfW88w0OWIMp38dEZnAvidy8Fl/cRC0JnxUhK5NN3qWoCueJS6uExfd2E8bAoFDa6BJN9+mP3cownwirkbs03yPAOCfzwINXyMxIZIl5WBW57SCfWQTtT0KpC4cNKuzs1potGatswqL3lRjDloGGGhyLsZbbOEWkNrBIJGg3eYvh9Xg2yZIfJXEhacPFv4iQw0lh6RcldSXztTEiFEKDUTp0tXbnPfkyDmgSNlVCgYDh+3PnrOHs= ceph-507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T08:53:59.605 DEBUG:teuthology.orchestra.run.smithi153:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCVS0BoZyzl2d4UZYb6pmtEVyd73DYS+X46xalomhMZ4cerC1Q4l7Cmk33c9R6V/Gch2MIAHQhNER/jjEDkW+PL22CwOz4QJgzLu1Faa/0hLB557rYfA12X29w2SDQskctQEbVfUXTFPUoOS18bwC5zqkze4jIi9SsXC2qVxTPTlmc4kxPTjau7FgotPgHIAC9mrn0gI9/StdxqLWzr7+YkFKCOqqKfec11elluXZHpdm3CUhfivZ219JI1S0D4AJwmRdIHOTRfd8mxfW88w0OWIMp38dEZnAvidy8Fl/cRC0JnxUhK5NN3qWoCueJS6uExfd2E8bAoFDa6BJN9+mP3cownwirkbs03yPAOCfzwINXyMxIZIl5WBW57SCfWQTtT0KpC4cNKuzs1potGatswqL3lRjDloGGGhyLsZbbOEWkNrBIJGg3eYvh9Xg2yZIfJXEhacPFv4iQw0lh6RcldSXztTEiFEKDUTp0tXbnPfkyDmgSNlVCgYDh+3PnrOHs= ceph-507393d2-7664-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T08:53:59.646 INFO:teuthology.orchestra.run.smithi153.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCVS0BoZyzl2d4UZYb6pmtEVyd73DYS+X46xalomhMZ4cerC1Q4l7Cmk33c9R6V/Gch2MIAHQhNER/jjEDkW+PL22CwOz4QJgzLu1Faa/0hLB557rYfA12X29w2SDQskctQEbVfUXTFPUoOS18bwC5zqkze4jIi9SsXC2qVxTPTlmc4kxPTjau7FgotPgHIAC9mrn0gI9/StdxqLWzr7+YkFKCOqqKfec11elluXZHpdm3CUhfivZ219JI1S0D4AJwmRdIHOTRfd8mxfW88w0OWIMp38dEZnAvidy8Fl/cRC0JnxUhK5NN3qWoCueJS6uExfd2E8bAoFDa6BJN9+mP3cownwirkbs03yPAOCfzwINXyMxIZIl5WBW57SCfWQTtT0KpC4cNKuzs1potGatswqL3lRjDloGGGhyLsZbbOEWkNrBIJGg3eYvh9Xg2yZIfJXEhacPFv4iQw0lh6RcldSXztTEiFEKDUTp0tXbnPfkyDmgSNlVCgYDh+3PnrOHs= ceph-507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T08:53:59.658 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCVS0BoZyzl2d4UZYb6pmtEVyd73DYS+X46xalomhMZ4cerC1Q4l7Cmk33c9R6V/Gch2MIAHQhNER/jjEDkW+PL22CwOz4QJgzLu1Faa/0hLB557rYfA12X29w2SDQskctQEbVfUXTFPUoOS18bwC5zqkze4jIi9SsXC2qVxTPTlmc4kxPTjau7FgotPgHIAC9mrn0gI9/StdxqLWzr7+YkFKCOqqKfec11elluXZHpdm3CUhfivZ219JI1S0D4AJwmRdIHOTRfd8mxfW88w0OWIMp38dEZnAvidy8Fl/cRC0JnxUhK5NN3qWoCueJS6uExfd2E8bAoFDa6BJN9+mP3cownwirkbs03yPAOCfzwINXyMxIZIl5WBW57SCfWQTtT0KpC4cNKuzs1potGatswqL3lRjDloGGGhyLsZbbOEWkNrBIJGg3eYvh9Xg2yZIfJXEhacPFv4iQw0lh6RcldSXztTEiFEKDUTp0tXbnPfkyDmgSNlVCgYDh+3PnrOHs= ceph-507393d2-7664-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T08:53:59.699 INFO:teuthology.orchestra.run.smithi196.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCVS0BoZyzl2d4UZYb6pmtEVyd73DYS+X46xalomhMZ4cerC1Q4l7Cmk33c9R6V/Gch2MIAHQhNER/jjEDkW+PL22CwOz4QJgzLu1Faa/0hLB557rYfA12X29w2SDQskctQEbVfUXTFPUoOS18bwC5zqkze4jIi9SsXC2qVxTPTlmc4kxPTjau7FgotPgHIAC9mrn0gI9/StdxqLWzr7+YkFKCOqqKfec11elluXZHpdm3CUhfivZ219JI1S0D4AJwmRdIHOTRfd8mxfW88w0OWIMp38dEZnAvidy8Fl/cRC0JnxUhK5NN3qWoCueJS6uExfd2E8bAoFDa6BJN9+mP3cownwirkbs03yPAOCfzwINXyMxIZIl5WBW57SCfWQTtT0KpC4cNKuzs1potGatswqL3lRjDloGGGhyLsZbbOEWkNrBIJGg3eYvh9Xg2yZIfJXEhacPFv4iQw0lh6RcldSXztTEiFEKDUTp0tXbnPfkyDmgSNlVCgYDh+3PnrOHs= ceph-507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T08:53:59.711 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-19T08:53:59.955 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:54:00.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:53:59 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/73621367' entity='client.admin' 2024-09-19T08:54:00.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.472+0000 7fb7cc90b640 1 -- 172.21.15.120:0/4218619242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7c8101640 msgr2=0x7fb7c8101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:00.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.472+0000 7fb7cc90b640 1 --2- 172.21.15.120:0/4218619242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7c8101640 0x7fb7c8101a40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8009880 tx=0x7fb7b802a5d0 comp rx=0 tx=0).stop 2024-09-19T08:54:00.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.472+0000 7fb7cc90b640 1 -- 172.21.15.120:0/4218619242 shutdown_connections 2024-09-19T08:54:00.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.472+0000 7fb7cc90b640 1 --2- 172.21.15.120:0/4218619242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7c8101640 0x7fb7c8101a40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:00.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.472+0000 7fb7cc90b640 1 -- 172.21.15.120:0/4218619242 >> 172.21.15.120:0/4218619242 conn(0x7fb7c80fcdf0 msgr2=0x7fb7c80ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:00.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.472+0000 7fb7cc90b640 1 -- 172.21.15.120:0/4218619242 shutdown_connections 2024-09-19T08:54:00.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.472+0000 7fb7cc90b640 1 -- 172.21.15.120:0/4218619242 wait complete. 2024-09-19T08:54:00.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.473+0000 7fb7cc90b640 1 Processor -- start 2024-09-19T08:54:00.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.473+0000 7fb7cc90b640 1 -- start start 2024-09-19T08:54:00.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.473+0000 7fb7cc90b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7c8101640 0x7fb7c8196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:00.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.473+0000 7fb7cc90b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c8107750 con 0x7fb7c8101640 2024-09-19T08:54:00.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.473+0000 7fb7c659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7c8101640 0x7fb7c8196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:00.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.473+0000 7fb7c659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7c8101640 0x7fb7c8196e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38990/0 (socket says 172.21.15.120:38990) 2024-09-19T08:54:00.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.473+0000 7fb7c659c640 1 -- 172.21.15.120:0/553875352 learned_addr learned my addr 172.21.15.120:0/553875352 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:54:00.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.474+0000 7fb7c659c640 1 -- 172.21.15.120:0/553875352 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c8197390 con 0x7fb7c8101640 2024-09-19T08:54:00.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.474+0000 7fb7c659c640 1 --2- 172.21.15.120:0/553875352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7c8101640 0x7fb7c8196e50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8009230 tx=0x7fb7b80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:00.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.474+0000 7fb7bf7fe640 1 -- 172.21.15.120:0/553875352 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7b803f070 con 0x7fb7c8101640 2024-09-19T08:54:00.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.474+0000 7fb7bf7fe640 1 -- 172.21.15.120:0/553875352 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7b8003ad0 con 0x7fb7c8101640 2024-09-19T08:54:00.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.474+0000 7fb7cc90b640 1 -- 172.21.15.120:0/553875352 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7c8197620 con 0x7fb7c8101640 2024-09-19T08:54:00.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.474+0000 7fb7bf7fe640 1 -- 172.21.15.120:0/553875352 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7b8035030 con 0x7fb7c8101640 2024-09-19T08:54:00.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.474+0000 7fb7cc90b640 1 -- 172.21.15.120:0/553875352 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7c819a310 con 0x7fb7c8101640 2024-09-19T08:54:00.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.475+0000 7fb7bf7fe640 1 -- 172.21.15.120:0/553875352 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fb7b8042070 con 0x7fb7c8101640 2024-09-19T08:54:00.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.475+0000 7fb7cc90b640 1 -- 172.21.15.120:0/553875352 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7c8107ed0 con 0x7fb7c8101640 2024-09-19T08:54:00.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.475+0000 7fb7bf7fe640 1 --2- 172.21.15.120:0/553875352 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb7b40392a0 0x7fb7b403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:00.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.475+0000 7fb7bf7fe640 1 -- 172.21.15.120:0/553875352 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb7b806d0b0 con 0x7fb7c8101640 2024-09-19T08:54:00.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.475+0000 7fb7bffff640 1 --2- 172.21.15.120:0/553875352 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb7b40392a0 0x7fb7b403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:00.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.476+0000 7fb7bffff640 1 --2- 172.21.15.120:0/553875352 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb7b40392a0 0x7fb7b403b760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00098c0 tx=0x7fb7b0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:00.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.481+0000 7fb7bf7fe640 1 -- 172.21.15.120:0/553875352 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb7b802f300 con 0x7fb7c8101640 2024-09-19T08:54:00.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.653+0000 7fb7cc90b640 1 -- 172.21.15.120:0/553875352 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fb7c802d830 con 0x7fb7c8101640 2024-09-19T08:54:00.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.657+0000 7fb7bf7fe640 1 -- 172.21.15.120:0/553875352 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fb7b8003c80 con 0x7fb7c8101640 2024-09-19T08:54:00.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.659+0000 7fb7cc90b640 1 -- 172.21.15.120:0/553875352 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb7b40392a0 msgr2=0x7fb7b403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:00.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.659+0000 7fb7cc90b640 1 --2- 172.21.15.120:0/553875352 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb7b40392a0 0x7fb7b403b760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00098c0 tx=0x7fb7b0006db0 comp rx=0 tx=0).stop 2024-09-19T08:54:00.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.659+0000 7fb7cc90b640 1 -- 172.21.15.120:0/553875352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7c8101640 msgr2=0x7fb7c8196e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:00.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.659+0000 7fb7cc90b640 1 --2- 172.21.15.120:0/553875352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7c8101640 0x7fb7c8196e50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8009230 tx=0x7fb7b80048f0 comp rx=0 tx=0).stop 2024-09-19T08:54:00.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.659+0000 7fb7cc90b640 1 -- 172.21.15.120:0/553875352 shutdown_connections 2024-09-19T08:54:00.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.659+0000 7fb7cc90b640 1 --2- 172.21.15.120:0/553875352 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb7b40392a0 0x7fb7b403b760 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:00.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.659+0000 7fb7cc90b640 1 --2- 172.21.15.120:0/553875352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7c8101640 0x7fb7c8196e50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:00.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.659+0000 7fb7cc90b640 1 -- 172.21.15.120:0/553875352 >> 172.21.15.120:0/553875352 conn(0x7fb7c80fcdf0 msgr2=0x7fb7c80fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:00.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.660+0000 7fb7cc90b640 1 -- 172.21.15.120:0/553875352 shutdown_connections 2024-09-19T08:54:00.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:00.660+0000 7fb7cc90b640 1 -- 172.21.15.120:0/553875352 wait complete. 2024-09-19T08:54:01.773 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-19T08:54:01.773 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-19T08:54:01.971 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:01 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/553875352' entity='client.admin' 2024-09-19T08:54:02.210 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:54:02.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.682+0000 7fe570945640 1 -- 172.21.15.120:0/4239282986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe56c101640 msgr2=0x7fe56c101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:02.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.682+0000 7fe570945640 1 --2- 172.21.15.120:0/4239282986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe56c101640 0x7fe56c101a40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe55c009880 tx=0x7fe55c02a5d0 comp rx=0 tx=0).stop 2024-09-19T08:54:02.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.682+0000 7fe570945640 1 -- 172.21.15.120:0/4239282986 shutdown_connections 2024-09-19T08:54:02.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.682+0000 7fe570945640 1 --2- 172.21.15.120:0/4239282986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe56c101640 0x7fe56c101a40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:02.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.682+0000 7fe570945640 1 -- 172.21.15.120:0/4239282986 >> 172.21.15.120:0/4239282986 conn(0x7fe56c0fcdf0 msgr2=0x7fe56c0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:02.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.682+0000 7fe570945640 1 -- 172.21.15.120:0/4239282986 shutdown_connections 2024-09-19T08:54:02.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.682+0000 7fe570945640 1 -- 172.21.15.120:0/4239282986 wait complete. 2024-09-19T08:54:02.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.683+0000 7fe570945640 1 Processor -- start 2024-09-19T08:54:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.683+0000 7fe570945640 1 -- start start 2024-09-19T08:54:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.683+0000 7fe570945640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe56c101640 0x7fe56c075eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.683+0000 7fe570945640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c107750 con 0x7fe56c101640 2024-09-19T08:54:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.683+0000 7fe56a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe56c101640 0x7fe56c075eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.683+0000 7fe56a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe56c101640 0x7fe56c075eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36228/0 (socket says 172.21.15.120:36228) 2024-09-19T08:54:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.683+0000 7fe56a59c640 1 -- 172.21.15.120:0/383052460 learned_addr learned my addr 172.21.15.120:0/383052460 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:54:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.683+0000 7fe56a59c640 1 -- 172.21.15.120:0/383052460 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56c0763f0 con 0x7fe56c101640 2024-09-19T08:54:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.683+0000 7fe56a59c640 1 --2- 172.21.15.120:0/383052460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe56c101640 0x7fe56c075eb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe55c009230 tx=0x7fe55c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.684+0000 7fe55b7fe640 1 -- 172.21.15.120:0/383052460 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe55c03f070 con 0x7fe56c101640 2024-09-19T08:54:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.684+0000 7fe570945640 1 -- 172.21.15.120:0/383052460 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe56c0745e0 con 0x7fe56c101640 2024-09-19T08:54:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.684+0000 7fe55b7fe640 1 -- 172.21.15.120:0/383052460 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe55c003ad0 con 0x7fe56c101640 2024-09-19T08:54:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.684+0000 7fe55b7fe640 1 -- 172.21.15.120:0/383052460 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe55c035030 con 0x7fe56c101640 2024-09-19T08:54:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.684+0000 7fe570945640 1 -- 172.21.15.120:0/383052460 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe56c074b20 con 0x7fe56c101640 2024-09-19T08:54:02.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.684+0000 7fe55b7fe640 1 -- 172.21.15.120:0/383052460 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fe55c042070 con 0x7fe56c101640 2024-09-19T08:54:02.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.685+0000 7fe55b7fe640 1 --2- 172.21.15.120:0/383052460 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5540392a0 0x7fe55403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:02.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.685+0000 7fe55b7fe640 1 -- 172.21.15.120:0/383052460 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe55c003c80 con 0x7fe56c101640 2024-09-19T08:54:02.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.685+0000 7fe569d9b640 1 --2- 172.21.15.120:0/383052460 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5540392a0 0x7fe55403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:02.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.685+0000 7fe570945640 1 -- 172.21.15.120:0/383052460 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe56c107ed0 con 0x7fe56c101640 2024-09-19T08:54:02.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.685+0000 7fe569d9b640 1 --2- 172.21.15.120:0/383052460 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5540392a0 0x7fe55403b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe5600098c0 tx=0x7fe560006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:02.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.693+0000 7fe55b7fe640 1 -- 172.21.15.120:0/383052460 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe55c02f300 con 0x7fe56c101640 2024-09-19T08:54:02.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.870+0000 7fe570945640 1 -- 172.21.15.120:0/383052460 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fe56c190020 con 0x7fe5540392a0 2024-09-19T08:54:02.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.874+0000 7fe55b7fe640 1 -- 172.21.15.120:0/383052460 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe56c190020 con 0x7fe5540392a0 2024-09-19T08:54:02.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.875+0000 7fe570945640 1 -- 172.21.15.120:0/383052460 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5540392a0 msgr2=0x7fe55403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:02.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.875+0000 7fe570945640 1 --2- 172.21.15.120:0/383052460 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5540392a0 0x7fe55403b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe5600098c0 tx=0x7fe560006db0 comp rx=0 tx=0).stop 2024-09-19T08:54:02.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.875+0000 7fe570945640 1 -- 172.21.15.120:0/383052460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe56c101640 msgr2=0x7fe56c075eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:02.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.875+0000 7fe570945640 1 --2- 172.21.15.120:0/383052460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe56c101640 0x7fe56c075eb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe55c009230 tx=0x7fe55c0048f0 comp rx=0 tx=0).stop 2024-09-19T08:54:02.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.876+0000 7fe570945640 1 -- 172.21.15.120:0/383052460 shutdown_connections 2024-09-19T08:54:02.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.876+0000 7fe570945640 1 --2- 172.21.15.120:0/383052460 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5540392a0 0x7fe55403b760 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:02.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.876+0000 7fe570945640 1 --2- 172.21.15.120:0/383052460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe56c101640 0x7fe56c075eb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:02.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.876+0000 7fe570945640 1 -- 172.21.15.120:0/383052460 >> 172.21.15.120:0/383052460 conn(0x7fe56c0fcdf0 msgr2=0x7fe56c0fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:02.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.876+0000 7fe570945640 1 -- 172.21.15.120:0/383052460 shutdown_connections 2024-09-19T08:54:02.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:02.876+0000 7fe570945640 1 -- 172.21.15.120:0/383052460 wait complete. 2024-09-19T08:54:03.588 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi153 2024-09-19T08:54:03.588 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-19T08:54:03.588 DEBUG:teuthology.orchestra.run.smithi153:> dd of=/etc/ceph/ceph.conf 2024-09-19T08:54:03.607 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-19T08:54:03.607 DEBUG:teuthology.orchestra.run.smithi153:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:54:03.665 INFO:tasks.cephadm:Adding host smithi153 to orchestrator... 2024-09-19T08:54:03.666 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph orch host add smithi153 2024-09-19T08:54:04.043 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:54:04.166 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:03 smithi120 ceph-mon[31150]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:54:04.166 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:03 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:04.551+0000 7f78257b4640 1 -- 172.21.15.120:0/3881644554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7820100530 msgr2=0x7f7820100930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:04.551+0000 7f78257b4640 1 --2- 172.21.15.120:0/3881644554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7820100530 0x7f7820100930 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7810009880 tx=0x7f781002a5d0 comp rx=0 tx=0).stop 2024-09-19T08:54:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:04.551+0000 7f78257b4640 1 -- 172.21.15.120:0/3881644554 shutdown_connections 2024-09-19T08:54:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:04.551+0000 7f78257b4640 1 --2- 172.21.15.120:0/3881644554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7820100530 0x7f7820100930 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:04.551+0000 7f78257b4640 1 -- 172.21.15.120:0/3881644554 >> 172.21.15.120:0/3881644554 conn(0x7f78200fbca0 msgr2=0x7f78200fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:04.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:04.551+0000 7f78257b4640 1 -- 172.21.15.120:0/3881644554 shutdown_connections 2024-09-19T08:54:04.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:04.551+0000 7f78257b4640 1 -- 172.21.15.120:0/3881644554 wait complete. 2024-09-19T08:54:04.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:04.552+0000 7f78257b4640 1 Processor -- start 2024-09-19T08:54:04.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:04.552+0000 7f78257b4640 1 -- start start 2024-09-19T08:54:04.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:04.552+0000 7f78257b4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7820100530 0x7f782006a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:04.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:04.552+0000 7f78257b4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7820106640 con 0x7f7820100530 2024-09-19T08:54:04.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:04.552+0000 7f781effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7820100530 0x7f782006a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:04.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:04.552+0000 7f781effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7820100530 0x7f782006a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36232/0 (socket says 172.21.15.120:36232) 2024-09-19T08:54:04.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:04.552+0000 7f781effd640 1 -- 172.21.15.120:0/1823954005 learned_addr learned my addr 172.21.15.120:0/1823954005 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:54:04.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:04.552+0000 7f781effd640 1 -- 172.21.15.120:0/1823954005 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7820199ad0 con 0x7f7820100530 2024-09-19T08:54:04.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:04.552+0000 7f781effd640 1 --2- 172.21.15.120:0/1823954005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7820100530 0x7f782006a180 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7810004f60 tx=0x7f78100048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:04.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:04.553+0000 7f7807fff640 1 -- 172.21.15.120:0/1823954005 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f781003f070 con 0x7f7820100530 2024-09-19T08:54:04.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:04.553+0000 7f7807fff640 1 -- 172.21.15.120:0/1823954005 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7810004d40 con 0x7f7820100530 2024-09-19T08:54:04.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:04.553+0000 7f7807fff640 1 -- 172.21.15.120:0/1823954005 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7810035030 con 0x7f7820100530 2024-09-19T08:54:04.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:04.553+0000 7f78257b4640 1 -- 172.21.15.120:0/1823954005 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7820199ca0 con 0x7f7820100530 2024-09-19T08:54:04.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:04.553+0000 7f78257b4640 1 -- 172.21.15.120:0/1823954005 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f782019a0c0 con 0x7f7820100530 2024-09-19T08:54:04.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:04.553+0000 7f7807fff640 1 -- 172.21.15.120:0/1823954005 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f7810042070 con 0x7f7820100530 2024-09-19T08:54:04.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:04.554+0000 7f7807fff640 1 --2- 172.21.15.120:0/1823954005 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7808039300 0x7f780803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:04.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:04.554+0000 7f7807fff640 1 -- 172.21.15.120:0/1823954005 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f781006e020 con 0x7f7820100530 2024-09-19T08:54:04.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:04.554+0000 7f781e7fc640 1 --2- 172.21.15.120:0/1823954005 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7808039300 0x7f780803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:04.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:04.554+0000 7f78257b4640 1 -- 172.21.15.120:0/1823954005 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7820106dc0 con 0x7f7820100530 2024-09-19T08:54:04.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:04.554+0000 7f781e7fc640 1 --2- 172.21.15.120:0/1823954005 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7808039300 0x7f780803b7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f78140098c0 tx=0x7f7814006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:04.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:04.559+0000 7f7807fff640 1 -- 172.21.15.120:0/1823954005 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7810042360 con 0x7f7820100530 2024-09-19T08:54:04.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:04.739+0000 7f78257b4640 1 -- 172.21.15.120:0/1823954005 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi153", "target": ["mon-mgr", ""]}) -- 0x7f782002d8d0 con 0x7f7808039300 2024-09-19T08:54:05.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:05.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:05.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi120", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:54:05.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:05.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:54:05.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:04 smithi120 ceph-mon[31150]: Updating smithi120:/etc/ceph/ceph.conf 2024-09-19T08:54:05.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:04 smithi120 ceph-mon[31150]: Updating smithi120:/var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:54:05.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:04 smithi120 ceph-mon[31150]: Updating smithi120:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:54:05.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:04 smithi120 ceph-mon[31150]: Updating smithi120:/var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:54:05.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:04 smithi120 ceph-mon[31150]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi153", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:54:05.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:05.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:05.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:05.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:54:05.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:05.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:54:05.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:06.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:05 smithi120 ceph-mon[31150]: Deploying cephadm binary to smithi153 2024-09-19T08:54:07.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:07.917+0000 7f7807fff640 1 -- 172.21.15.120:0/1823954005 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f782002d8d0 con 0x7f7808039300 2024-09-19T08:54:07.918 INFO:teuthology.orchestra.run.smithi120.stdout:Added host 'smithi153' with addr '172.21.15.153' 2024-09-19T08:54:07.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:07.919+0000 7f78257b4640 1 -- 172.21.15.120:0/1823954005 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7808039300 msgr2=0x7f780803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:07.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:07.919+0000 7f78257b4640 1 --2- 172.21.15.120:0/1823954005 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7808039300 0x7f780803b7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f78140098c0 tx=0x7f7814006db0 comp rx=0 tx=0).stop 2024-09-19T08:54:07.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:07.919+0000 7f78257b4640 1 -- 172.21.15.120:0/1823954005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7820100530 msgr2=0x7f782006a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:07.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:07.919+0000 7f78257b4640 1 --2- 172.21.15.120:0/1823954005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7820100530 0x7f782006a180 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7810004f60 tx=0x7f78100048f0 comp rx=0 tx=0).stop 2024-09-19T08:54:07.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:07.919+0000 7f78257b4640 1 -- 172.21.15.120:0/1823954005 shutdown_connections 2024-09-19T08:54:07.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:07.919+0000 7f78257b4640 1 --2- 172.21.15.120:0/1823954005 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7808039300 0x7f780803b7c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:07.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:07.919+0000 7f78257b4640 1 --2- 172.21.15.120:0/1823954005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7820100530 0x7f782006a180 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:07.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:07.919+0000 7f78257b4640 1 -- 172.21.15.120:0/1823954005 >> 172.21.15.120:0/1823954005 conn(0x7f78200fbca0 msgr2=0x7f78200fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:07.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:07.920+0000 7f78257b4640 1 -- 172.21.15.120:0/1823954005 shutdown_connections 2024-09-19T08:54:07.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:07.920+0000 7f78257b4640 1 -- 172.21.15.120:0/1823954005 wait complete. 2024-09-19T08:54:08.545 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T08:54:08.783 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:54:09.176 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:08 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:09.176 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:08 smithi120 ceph-mon[31150]: Added host smithi153 2024-09-19T08:54:09.176 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:08 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:54:09.176 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:08 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:09.176 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:08 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:09.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.256+0000 7fcceda0f640 1 -- 172.21.15.120:0/3777749225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcce8106590 msgr2=0x7fcce8106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:09.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.256+0000 7fcceda0f640 1 --2- 172.21.15.120:0/3777749225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcce8106590 0x7fcce8106970 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009880 tx=0x7fccd802a5d0 comp rx=0 tx=0).stop 2024-09-19T08:54:09.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.256+0000 7fcceda0f640 1 -- 172.21.15.120:0/3777749225 shutdown_connections 2024-09-19T08:54:09.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.256+0000 7fcceda0f640 1 --2- 172.21.15.120:0/3777749225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcce8106590 0x7fcce8106970 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:09.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.256+0000 7fcceda0f640 1 -- 172.21.15.120:0/3777749225 >> 172.21.15.120:0/3777749225 conn(0x7fcce80fc1a0 msgr2=0x7fcce80fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:09.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.256+0000 7fcceda0f640 1 -- 172.21.15.120:0/3777749225 shutdown_connections 2024-09-19T08:54:09.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.256+0000 7fcceda0f640 1 -- 172.21.15.120:0/3777749225 wait complete. 2024-09-19T08:54:09.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.256+0000 7fcceda0f640 1 Processor -- start 2024-09-19T08:54:09.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.257+0000 7fcceda0f640 1 -- start start 2024-09-19T08:54:09.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.257+0000 7fcceda0f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcce8106590 0x7fcce8197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:09.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.257+0000 7fcceda0f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcce810abc0 con 0x7fcce8106590 2024-09-19T08:54:09.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.257+0000 7fcce6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcce8106590 0x7fcce8197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:09.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.257+0000 7fcce6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcce8106590 0x7fcce8197120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36250/0 (socket says 172.21.15.120:36250) 2024-09-19T08:54:09.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.257+0000 7fcce6ffd640 1 -- 172.21.15.120:0/2659608943 learned_addr learned my addr 172.21.15.120:0/2659608943 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:54:09.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.257+0000 7fcce6ffd640 1 -- 172.21.15.120:0/2659608943 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce8197660 con 0x7fcce8106590 2024-09-19T08:54:09.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.257+0000 7fcce6ffd640 1 --2- 172.21.15.120:0/2659608943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcce8106590 0x7fcce8197120 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009230 tx=0x7fccd8003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:09.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.259+0000 7fcceca0d640 1 -- 172.21.15.120:0/2659608943 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccd803f070 con 0x7fcce8106590 2024-09-19T08:54:09.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.259+0000 7fcceda0f640 1 -- 172.21.15.120:0/2659608943 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce81978f0 con 0x7fcce8106590 2024-09-19T08:54:09.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.259+0000 7fcceda0f640 1 -- 172.21.15.120:0/2659608943 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce819b9f0 con 0x7fcce8106590 2024-09-19T08:54:09.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.259+0000 7fcceca0d640 1 -- 172.21.15.120:0/2659608943 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccd803bc80 con 0x7fcce8106590 2024-09-19T08:54:09.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.259+0000 7fcceca0d640 1 -- 172.21.15.120:0/2659608943 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccd8049060 con 0x7fcce8106590 2024-09-19T08:54:09.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.260+0000 7fcceca0d640 1 -- 172.21.15.120:0/2659608943 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fccd8042070 con 0x7fcce8106590 2024-09-19T08:54:09.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.260+0000 7fcceca0d640 1 --2- 172.21.15.120:0/2659608943 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fccd0039300 0x7fccd003b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:09.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.260+0000 7fcceda0f640 1 -- 172.21.15.120:0/2659608943 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcce8101c00 con 0x7fcce8106590 2024-09-19T08:54:09.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.261+0000 7fcceca0d640 1 -- 172.21.15.120:0/2659608943 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fccd806d050 con 0x7fcce8106590 2024-09-19T08:54:09.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.261+0000 7fcce67fc640 1 --2- 172.21.15.120:0/2659608943 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fccd0039300 0x7fccd003b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:09.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.261+0000 7fcce67fc640 1 --2- 172.21.15.120:0/2659608943 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fccd0039300 0x7fccd003b7c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0098c0 tx=0x7fccdc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:09.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.265+0000 7fcceca0d640 1 -- 172.21.15.120:0/2659608943 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fccd803b7a0 con 0x7fcce8106590 2024-09-19T08:54:09.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.439+0000 7fcceda0f640 1 -- 172.21.15.120:0/2659608943 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcce8106a50 con 0x7fccd0039300 2024-09-19T08:54:09.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.441+0000 7fcceca0d640 1 -- 172.21.15.120:0/2659608943 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7fcce8106a50 con 0x7fccd0039300 2024-09-19T08:54:09.441 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:54:09.442 INFO:teuthology.orchestra.run.smithi120.stdout:[{"addr": "172.21.15.120", "hostname": "smithi120", "labels": [], "status": ""}, {"addr": "172.21.15.153", "hostname": "smithi153", "labels": [], "status": ""}] 2024-09-19T08:54:09.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.443+0000 7fcceda0f640 1 -- 172.21.15.120:0/2659608943 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fccd0039300 msgr2=0x7fccd003b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:09.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.443+0000 7fcceda0f640 1 --2- 172.21.15.120:0/2659608943 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fccd0039300 0x7fccd003b7c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0098c0 tx=0x7fccdc006db0 comp rx=0 tx=0).stop 2024-09-19T08:54:09.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.443+0000 7fcceda0f640 1 -- 172.21.15.120:0/2659608943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcce8106590 msgr2=0x7fcce8197120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:09.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.443+0000 7fcceda0f640 1 --2- 172.21.15.120:0/2659608943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcce8106590 0x7fcce8197120 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009230 tx=0x7fccd8003d40 comp rx=0 tx=0).stop 2024-09-19T08:54:09.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.443+0000 7fcceda0f640 1 -- 172.21.15.120:0/2659608943 shutdown_connections 2024-09-19T08:54:09.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.443+0000 7fcceda0f640 1 --2- 172.21.15.120:0/2659608943 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fccd0039300 0x7fccd003b7c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:09.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.443+0000 7fcceda0f640 1 --2- 172.21.15.120:0/2659608943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcce8106590 0x7fcce8197120 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:09.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.443+0000 7fcceda0f640 1 -- 172.21.15.120:0/2659608943 >> 172.21.15.120:0/2659608943 conn(0x7fcce80fc1a0 msgr2=0x7fcce8069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:09.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.443+0000 7fcceda0f640 1 -- 172.21.15.120:0/2659608943 shutdown_connections 2024-09-19T08:54:09.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:09.443+0000 7fcceda0f640 1 -- 172.21.15.120:0/2659608943 wait complete. 2024-09-19T08:54:10.075 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi196 2024-09-19T08:54:10.075 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-19T08:54:10.075 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.conf 2024-09-19T08:54:10.095 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-19T08:54:10.095 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:54:10.153 INFO:tasks.cephadm:Adding host smithi196 to orchestrator... 2024-09-19T08:54:10.154 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph orch host add smithi196 2024-09-19T08:54:10.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:10 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:10.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:10 smithi120 ceph-mon[31150]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:54:10.392 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:54:10.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:10.795+0000 7fd99438c640 1 -- 172.21.15.120:0/3857392510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd98c107660 msgr2=0x7fd98c107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:10.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:10.795+0000 7fd99438c640 1 --2- 172.21.15.120:0/3857392510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd98c107660 0x7fd98c107a40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd988009c20 tx=0x7fd98802aa90 comp rx=0 tx=0).stop 2024-09-19T08:54:10.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:10.796+0000 7fd99438c640 1 -- 172.21.15.120:0/3857392510 shutdown_connections 2024-09-19T08:54:10.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:10.796+0000 7fd99438c640 1 --2- 172.21.15.120:0/3857392510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd98c107660 0x7fd98c107a40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:10.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:10.796+0000 7fd99438c640 1 -- 172.21.15.120:0/3857392510 >> 172.21.15.120:0/3857392510 conn(0x7fd98c0fd2b0 msgr2=0x7fd98c0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:10.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:10.796+0000 7fd99438c640 1 -- 172.21.15.120:0/3857392510 shutdown_connections 2024-09-19T08:54:10.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:10.796+0000 7fd99438c640 1 -- 172.21.15.120:0/3857392510 wait complete. 2024-09-19T08:54:10.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:10.796+0000 7fd99438c640 1 Processor -- start 2024-09-19T08:54:10.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:10.796+0000 7fd99438c640 1 -- start start 2024-09-19T08:54:10.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:10.797+0000 7fd99438c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd98c107660 0x7fd98c197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:10.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:10.797+0000 7fd99438c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd98c10bc70 con 0x7fd98c107660 2024-09-19T08:54:10.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:10.797+0000 7fd992128640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd98c107660 0x7fd98c197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:10.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:10.797+0000 7fd992128640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd98c107660 0x7fd98c197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36268/0 (socket says 172.21.15.120:36268) 2024-09-19T08:54:10.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:10.797+0000 7fd992128640 1 -- 172.21.15.120:0/2805505617 learned_addr learned my addr 172.21.15.120:0/2805505617 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:54:10.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:10.797+0000 7fd992128640 1 -- 172.21.15.120:0/2805505617 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd98c197650 con 0x7fd98c107660 2024-09-19T08:54:10.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:10.797+0000 7fd992128640 1 --2- 172.21.15.120:0/2805505617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd98c107660 0x7fd98c197110 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd988005e30 tx=0x7fd98802fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:10.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:10.797+0000 7fd9837fe640 1 -- 172.21.15.120:0/2805505617 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd988047070 con 0x7fd98c107660 2024-09-19T08:54:10.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:10.797+0000 7fd9837fe640 1 -- 172.21.15.120:0/2805505617 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd988003df0 con 0x7fd98c107660 2024-09-19T08:54:10.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:10.797+0000 7fd99438c640 1 -- 172.21.15.120:0/2805505617 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd98c197940 con 0x7fd98c107660 2024-09-19T08:54:10.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:10.797+0000 7fd9837fe640 1 -- 172.21.15.120:0/2805505617 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd988033050 con 0x7fd98c107660 2024-09-19T08:54:10.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:10.797+0000 7fd99438c640 1 -- 172.21.15.120:0/2805505617 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd98c19b9e0 con 0x7fd98c107660 2024-09-19T08:54:10.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:10.798+0000 7fd9837fe640 1 -- 172.21.15.120:0/2805505617 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fd98804a070 con 0x7fd98c107660 2024-09-19T08:54:10.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:10.798+0000 7fd99438c640 1 -- 172.21.15.120:0/2805505617 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd98c102d10 con 0x7fd98c107660 2024-09-19T08:54:10.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:10.798+0000 7fd9837fe640 1 --2- 172.21.15.120:0/2805505617 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd9780392a0 0x7fd97803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:10.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:10.798+0000 7fd9837fe640 1 -- 172.21.15.120:0/2805505617 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd98803e030 con 0x7fd98c107660 2024-09-19T08:54:10.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:10.799+0000 7fd991927640 1 --2- 172.21.15.120:0/2805505617 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd9780392a0 0x7fd97803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:10.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:10.799+0000 7fd991927640 1 --2- 172.21.15.120:0/2805505617 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd9780392a0 0x7fd97803b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0098c0 tx=0x7fd97c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:10.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:10.804+0000 7fd9837fe640 1 -- 172.21.15.120:0/2805505617 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd988001250 con 0x7fd98c107660 2024-09-19T08:54:10.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:10.974+0000 7fd99438c640 1 -- 172.21.15.120:0/2805505617 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}) -- 0x7fd98c198470 con 0x7fd9780392a0 2024-09-19T08:54:12.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:12 smithi120 ceph-mon[31150]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:54:12.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:12 smithi120 ceph-mon[31150]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:12.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:12.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:12.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:12.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:12.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi153", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:54:12.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:12.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:54:13.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:13 smithi120 ceph-mon[31150]: Deploying cephadm binary to smithi196 2024-09-19T08:54:13.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:13 smithi120 ceph-mon[31150]: Updating smithi153:/etc/ceph/ceph.conf 2024-09-19T08:54:13.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:13 smithi120 ceph-mon[31150]: Updating smithi153:/var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:54:13.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:13 smithi120 ceph-mon[31150]: Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:54:13.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:13 smithi120 ceph-mon[31150]: Updating smithi153:/var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:54:14.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:14.121+0000 7fd9837fe640 1 -- 172.21.15.120:0/2805505617 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fd98c198470 con 0x7fd9780392a0 2024-09-19T08:54:14.123 INFO:teuthology.orchestra.run.smithi120.stdout:Added host 'smithi196' with addr '172.21.15.196' 2024-09-19T08:54:14.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:14.123+0000 7fd99438c640 1 -- 172.21.15.120:0/2805505617 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd9780392a0 msgr2=0x7fd97803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:14.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:14.123+0000 7fd99438c640 1 --2- 172.21.15.120:0/2805505617 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd9780392a0 0x7fd97803b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0098c0 tx=0x7fd97c006db0 comp rx=0 tx=0).stop 2024-09-19T08:54:14.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:14.123+0000 7fd99438c640 1 -- 172.21.15.120:0/2805505617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd98c107660 msgr2=0x7fd98c197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:14.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:14.123+0000 7fd99438c640 1 --2- 172.21.15.120:0/2805505617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd98c107660 0x7fd98c197110 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd988005e30 tx=0x7fd98802fc10 comp rx=0 tx=0).stop 2024-09-19T08:54:14.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:14.123+0000 7fd99438c640 1 -- 172.21.15.120:0/2805505617 shutdown_connections 2024-09-19T08:54:14.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:14.123+0000 7fd99438c640 1 --2- 172.21.15.120:0/2805505617 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd9780392a0 0x7fd97803b760 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:14.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:14.123+0000 7fd99438c640 1 --2- 172.21.15.120:0/2805505617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd98c107660 0x7fd98c197110 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:14.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:14.124+0000 7fd99438c640 1 -- 172.21.15.120:0/2805505617 >> 172.21.15.120:0/2805505617 conn(0x7fd98c0fd2b0 msgr2=0x7fd98c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:14.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:14.124+0000 7fd99438c640 1 -- 172.21.15.120:0/2805505617 shutdown_connections 2024-09-19T08:54:14.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:14.124+0000 7fd99438c640 1 -- 172.21.15.120:0/2805505617 wait complete. 2024-09-19T08:54:14.729 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:14 smithi120 ceph-mon[31150]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:14.729 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:14 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:14.729 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:14 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:14.729 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:14 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:14.729 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:14 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:14.729 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:14 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:54:14.730 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:14 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:14.759 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T08:54:14.993 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:54:15.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.480+0000 7ff339a4b640 1 -- 172.21.15.120:0/4211541205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff334106590 msgr2=0x7ff334106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:15.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.480+0000 7ff339a4b640 1 --2- 172.21.15.120:0/4211541205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff334106590 0x7ff334106970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff324009880 tx=0x7ff32402a5d0 comp rx=0 tx=0).stop 2024-09-19T08:54:15.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.480+0000 7ff339a4b640 1 -- 172.21.15.120:0/4211541205 shutdown_connections 2024-09-19T08:54:15.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.480+0000 7ff339a4b640 1 --2- 172.21.15.120:0/4211541205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff334106590 0x7ff334106970 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:15.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.480+0000 7ff339a4b640 1 -- 172.21.15.120:0/4211541205 >> 172.21.15.120:0/4211541205 conn(0x7ff3340fc1a0 msgr2=0x7ff3340fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:15.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.480+0000 7ff339a4b640 1 -- 172.21.15.120:0/4211541205 shutdown_connections 2024-09-19T08:54:15.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.480+0000 7ff339a4b640 1 -- 172.21.15.120:0/4211541205 wait complete. 2024-09-19T08:54:15.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.480+0000 7ff339a4b640 1 Processor -- start 2024-09-19T08:54:15.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.481+0000 7ff339a4b640 1 -- start start 2024-09-19T08:54:15.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.481+0000 7ff339a4b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff334106590 0x7ff334197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:15.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.481+0000 7ff339a4b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff33410abc0 con 0x7ff334106590 2024-09-19T08:54:15.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.481+0000 7ff332ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff334106590 0x7ff334197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:15.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.481+0000 7ff332ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff334106590 0x7ff334197130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56114/0 (socket says 172.21.15.120:56114) 2024-09-19T08:54:15.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.481+0000 7ff332ffd640 1 -- 172.21.15.120:0/3929948635 learned_addr learned my addr 172.21.15.120:0/3929948635 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:54:15.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.481+0000 7ff332ffd640 1 -- 172.21.15.120:0/3929948635 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff334197670 con 0x7ff334106590 2024-09-19T08:54:15.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.481+0000 7ff332ffd640 1 --2- 172.21.15.120:0/3929948635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff334106590 0x7ff334197130 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff324009230 tx=0x7ff324003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:15.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.481+0000 7ff338a49640 1 -- 172.21.15.120:0/3929948635 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff32403f070 con 0x7ff334106590 2024-09-19T08:54:15.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.481+0000 7ff338a49640 1 -- 172.21.15.120:0/3929948635 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff32403bc80 con 0x7ff334106590 2024-09-19T08:54:15.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.481+0000 7ff339a4b640 1 -- 172.21.15.120:0/3929948635 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff334197900 con 0x7ff334106590 2024-09-19T08:54:15.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.481+0000 7ff338a49640 1 -- 172.21.15.120:0/3929948635 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff324035030 con 0x7ff334106590 2024-09-19T08:54:15.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.481+0000 7ff339a4b640 1 -- 172.21.15.120:0/3929948635 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff33419ba00 con 0x7ff334106590 2024-09-19T08:54:15.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.482+0000 7ff338a49640 1 -- 172.21.15.120:0/3929948635 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7ff324042070 con 0x7ff334106590 2024-09-19T08:54:15.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.482+0000 7ff339a4b640 1 -- 172.21.15.120:0/3929948635 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff334101c00 con 0x7ff334106590 2024-09-19T08:54:15.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.483+0000 7ff338a49640 1 --2- 172.21.15.120:0/3929948635 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff320039300 0x7ff32003b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:15.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.483+0000 7ff338a49640 1 -- 172.21.15.120:0/3929948635 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff32406d040 con 0x7ff334106590 2024-09-19T08:54:15.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.483+0000 7ff32bfff640 1 --2- 172.21.15.120:0/3929948635 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff320039300 0x7ff32003b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:15.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.483+0000 7ff32bfff640 1 --2- 172.21.15.120:0/3929948635 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff320039300 0x7ff32003b7c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff31c0098c0 tx=0x7ff31c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:15.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.487+0000 7ff338a49640 1 -- 172.21.15.120:0/3929948635 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff3240372d0 con 0x7ff334106590 2024-09-19T08:54:15.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.659+0000 7ff339a4b640 1 -- 172.21.15.120:0/3929948635 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff334198490 con 0x7ff320039300 2024-09-19T08:54:15.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.660+0000 7ff338a49640 1 -- 172.21.15.120:0/3929948635 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7ff334198490 con 0x7ff320039300 2024-09-19T08:54:15.660 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:54:15.661 INFO:teuthology.orchestra.run.smithi120.stdout:[{"addr": "172.21.15.120", "hostname": "smithi120", "labels": [], "status": ""}, {"addr": "172.21.15.153", "hostname": "smithi153", "labels": [], "status": ""}, {"addr": "172.21.15.196", "hostname": "smithi196", "labels": [], "status": ""}] 2024-09-19T08:54:15.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.662+0000 7ff339a4b640 1 -- 172.21.15.120:0/3929948635 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff320039300 msgr2=0x7ff32003b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:15.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.662+0000 7ff339a4b640 1 --2- 172.21.15.120:0/3929948635 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff320039300 0x7ff32003b7c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff31c0098c0 tx=0x7ff31c006db0 comp rx=0 tx=0).stop 2024-09-19T08:54:15.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.662+0000 7ff339a4b640 1 -- 172.21.15.120:0/3929948635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff334106590 msgr2=0x7ff334197130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:15.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.662+0000 7ff339a4b640 1 --2- 172.21.15.120:0/3929948635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff334106590 0x7ff334197130 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff324009230 tx=0x7ff324003d40 comp rx=0 tx=0).stop 2024-09-19T08:54:15.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.663+0000 7ff339a4b640 1 -- 172.21.15.120:0/3929948635 shutdown_connections 2024-09-19T08:54:15.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.663+0000 7ff339a4b640 1 --2- 172.21.15.120:0/3929948635 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff320039300 0x7ff32003b7c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:15.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.663+0000 7ff339a4b640 1 --2- 172.21.15.120:0/3929948635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff334106590 0x7ff334197130 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:15.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.663+0000 7ff339a4b640 1 -- 172.21.15.120:0/3929948635 >> 172.21.15.120:0/3929948635 conn(0x7ff3340fc1a0 msgr2=0x7ff334069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:15.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.663+0000 7ff339a4b640 1 -- 172.21.15.120:0/3929948635 shutdown_connections 2024-09-19T08:54:15.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:15.663+0000 7ff339a4b640 1 -- 172.21.15.120:0/3929948635 wait complete. 2024-09-19T08:54:15.961 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:15 smithi120 ceph-mon[31150]: Added host smithi196 2024-09-19T08:54:15.961 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:15 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:15.961 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:15 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:16.263 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-19T08:54:16.263 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-19T08:54:16.510 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:54:16.797 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:16 smithi120 ceph-mon[31150]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:16.797 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:16 smithi120 ceph-mon[31150]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:54:17.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:16.998+0000 7f5d389c8640 1 -- 172.21.15.120:0/1194759470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d34107690 msgr2=0x7f5d34107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:17.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:16.998+0000 7f5d389c8640 1 --2- 172.21.15.120:0/1194759470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d34107690 0x7f5d34107a70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c009c20 tx=0x7f5d2c02aa90 comp rx=0 tx=0).stop 2024-09-19T08:54:17.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:16.998+0000 7f5d389c8640 1 -- 172.21.15.120:0/1194759470 shutdown_connections 2024-09-19T08:54:17.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:16.999+0000 7f5d389c8640 1 --2- 172.21.15.120:0/1194759470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d34107690 0x7f5d34107a70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:17.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:16.999+0000 7f5d389c8640 1 -- 172.21.15.120:0/1194759470 >> 172.21.15.120:0/1194759470 conn(0x7f5d34076250 msgr2=0x7f5d34076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:17.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:16.999+0000 7f5d389c8640 1 -- 172.21.15.120:0/1194759470 shutdown_connections 2024-09-19T08:54:17.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:16.999+0000 7f5d389c8640 1 -- 172.21.15.120:0/1194759470 wait complete. 2024-09-19T08:54:17.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:16.999+0000 7f5d389c8640 1 Processor -- start 2024-09-19T08:54:17.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:16.999+0000 7f5d389c8640 1 -- start start 2024-09-19T08:54:17.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.000+0000 7f5d389c8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d34107690 0x7f5d34197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:17.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.000+0000 7f5d389c8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3410bce0 con 0x7f5d34107690 2024-09-19T08:54:17.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.000+0000 7f5d3259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d34107690 0x7f5d34197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:17.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.000+0000 7f5d3259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d34107690 0x7f5d34197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56124/0 (socket says 172.21.15.120:56124) 2024-09-19T08:54:17.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.000+0000 7f5d3259c640 1 -- 172.21.15.120:0/4264215085 learned_addr learned my addr 172.21.15.120:0/4264215085 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:54:17.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.000+0000 7f5d3259c640 1 -- 172.21.15.120:0/4264215085 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d34197650 con 0x7f5d34107690 2024-09-19T08:54:17.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.000+0000 7f5d3259c640 1 --2- 172.21.15.120:0/4264215085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d34107690 0x7f5d34197110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c005e30 tx=0x7f5d2c02fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:17.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.000+0000 7f5d2b7fe640 1 -- 172.21.15.120:0/4264215085 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d2c047070 con 0x7f5d34107690 2024-09-19T08:54:17.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.001+0000 7f5d389c8640 1 -- 172.21.15.120:0/4264215085 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d341978e0 con 0x7f5d34107690 2024-09-19T08:54:17.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.001+0000 7f5d2b7fe640 1 -- 172.21.15.120:0/4264215085 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d2c003df0 con 0x7f5d34107690 2024-09-19T08:54:17.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.001+0000 7f5d2b7fe640 1 -- 172.21.15.120:0/4264215085 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d2c033050 con 0x7f5d34107690 2024-09-19T08:54:17.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.001+0000 7f5d389c8640 1 -- 172.21.15.120:0/4264215085 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d3419b9e0 con 0x7f5d34107690 2024-09-19T08:54:17.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.002+0000 7f5d2b7fe640 1 -- 172.21.15.120:0/4264215085 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f5d2c04a070 con 0x7f5d34107690 2024-09-19T08:54:17.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.002+0000 7f5d389c8640 1 -- 172.21.15.120:0/4264215085 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d34102d30 con 0x7f5d34107690 2024-09-19T08:54:17.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.002+0000 7f5d2b7fe640 1 --2- 172.21.15.120:0/4264215085 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5d200392a0 0x7f5d2003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:17.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.002+0000 7f5d2b7fe640 1 -- 172.21.15.120:0/4264215085 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5d2c03e030 con 0x7f5d34107690 2024-09-19T08:54:17.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.002+0000 7f5d31d9b640 1 --2- 172.21.15.120:0/4264215085 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5d200392a0 0x7f5d2003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:17.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.002+0000 7f5d31d9b640 1 --2- 172.21.15.120:0/4264215085 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5d200392a0 0x7f5d2003b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0098c0 tx=0x7f5d1c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:17.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.007+0000 7f5d2b7fe640 1 -- 172.21.15.120:0/4264215085 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5d2c0373b0 con 0x7f5d34107690 2024-09-19T08:54:17.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.183+0000 7f5d389c8640 1 -- 172.21.15.120:0/4264215085 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f5d34198470 con 0x7f5d34107690 2024-09-19T08:54:17.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.696+0000 7f5d2b7fe640 1 -- 172.21.15.120:0/4264215085 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f5d2c0492c0 con 0x7f5d34107690 2024-09-19T08:54:17.696 INFO:teuthology.orchestra.run.smithi120.stderr:adjusted tunables profile to default 2024-09-19T08:54:17.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.699+0000 7f5d389c8640 1 -- 172.21.15.120:0/4264215085 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5d200392a0 msgr2=0x7f5d2003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:17.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.699+0000 7f5d389c8640 1 --2- 172.21.15.120:0/4264215085 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5d200392a0 0x7f5d2003b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0098c0 tx=0x7f5d1c006db0 comp rx=0 tx=0).stop 2024-09-19T08:54:17.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.699+0000 7f5d389c8640 1 -- 172.21.15.120:0/4264215085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d34107690 msgr2=0x7f5d34197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:17.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.699+0000 7f5d389c8640 1 --2- 172.21.15.120:0/4264215085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d34107690 0x7f5d34197110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c005e30 tx=0x7f5d2c02fc10 comp rx=0 tx=0).stop 2024-09-19T08:54:17.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.699+0000 7f5d389c8640 1 -- 172.21.15.120:0/4264215085 shutdown_connections 2024-09-19T08:54:17.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.699+0000 7f5d389c8640 1 --2- 172.21.15.120:0/4264215085 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5d200392a0 0x7f5d2003b760 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:17.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.699+0000 7f5d389c8640 1 --2- 172.21.15.120:0/4264215085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d34107690 0x7f5d34197110 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:17.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.699+0000 7f5d389c8640 1 -- 172.21.15.120:0/4264215085 >> 172.21.15.120:0/4264215085 conn(0x7f5d34076250 msgr2=0x7f5d340fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:17.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.699+0000 7f5d389c8640 1 -- 172.21.15.120:0/4264215085 shutdown_connections 2024-09-19T08:54:17.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:17.699+0000 7f5d389c8640 1 -- 172.21.15.120:0/4264215085 wait complete. 2024-09-19T08:54:17.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:17 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/4264215085' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T08:54:18.447 INFO:tasks.cephadm:Adding mon.a on smithi120 2024-09-19T08:54:18.447 INFO:tasks.cephadm:Adding mon.b on smithi153 2024-09-19T08:54:18.447 INFO:tasks.cephadm:Adding mon.c on smithi196 2024-09-19T08:54:18.447 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi120:172.21.15.120=a;smithi153:172.21.15.153=b;smithi196:172.21.15.196=c' 2024-09-19T08:54:18.692 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:54:19.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:18 smithi120 ceph-mon[31150]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:19.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:18 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/4264215085' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T08:54:19.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:18 smithi120 ceph-mon[31150]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T08:54:19.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:18 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:19.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:18 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:19.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:18 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:19.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:18 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:19.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:18 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:54:19.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:18 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:19.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:18 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:54:19.224 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.223+0000 7fcf534d2640 1 -- 172.21.15.196:0/1415870669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf4c107660 msgr2=0x7fcf4c107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:19.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.223+0000 7fcf534d2640 1 --2- 172.21.15.196:0/1415870669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf4c107660 0x7fcf4c107a40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcf48009880 tx=0x7fcf4802a5d0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.223+0000 7fcf534d2640 1 -- 172.21.15.196:0/1415870669 shutdown_connections 2024-09-19T08:54:19.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.223+0000 7fcf534d2640 1 --2- 172.21.15.196:0/1415870669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf4c107660 0x7fcf4c107a40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.223+0000 7fcf534d2640 1 -- 172.21.15.196:0/1415870669 >> 172.21.15.196:0/1415870669 conn(0x7fcf4c0fd2b0 msgr2=0x7fcf4c0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:19.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.223+0000 7fcf534d2640 1 -- 172.21.15.196:0/1415870669 shutdown_connections 2024-09-19T08:54:19.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.223+0000 7fcf534d2640 1 -- 172.21.15.196:0/1415870669 wait complete. 2024-09-19T08:54:19.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.224+0000 7fcf534d2640 1 Processor -- start 2024-09-19T08:54:19.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.224+0000 7fcf534d2640 1 -- start start 2024-09-19T08:54:19.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.224+0000 7fcf534d2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf4c107660 0x7fcf4c1970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:19.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.225+0000 7fcf534d2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c10bc70 con 0x7fcf4c107660 2024-09-19T08:54:19.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.225+0000 7fcf5126e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf4c107660 0x7fcf4c1970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:19.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.225+0000 7fcf5126e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf4c107660 0x7fcf4c1970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.196:41134/0 (socket says 172.21.15.196:41134) 2024-09-19T08:54:19.233 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.225+0000 7fcf5126e640 1 -- 172.21.15.196:0/4004071966 learned_addr learned my addr 172.21.15.196:0/4004071966 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-19T08:54:19.233 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.225+0000 7fcf5126e640 1 -- 172.21.15.196:0/4004071966 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4c1975e0 con 0x7fcf4c107660 2024-09-19T08:54:19.233 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.225+0000 7fcf5126e640 1 --2- 172.21.15.196:0/4004071966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf4c107660 0x7fcf4c1970a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcf48009230 tx=0x7fcf48003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:19.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.225+0000 7fcf427fc640 1 -- 172.21.15.196:0/4004071966 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf4803f070 con 0x7fcf4c107660 2024-09-19T08:54:19.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.225+0000 7fcf534d2640 1 -- 172.21.15.196:0/4004071966 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf4c1978d0 con 0x7fcf4c107660 2024-09-19T08:54:19.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.225+0000 7fcf427fc640 1 -- 172.21.15.196:0/4004071966 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf4803bc80 con 0x7fcf4c107660 2024-09-19T08:54:19.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.225+0000 7fcf427fc640 1 -- 172.21.15.196:0/4004071966 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf48035030 con 0x7fcf4c107660 2024-09-19T08:54:19.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.225+0000 7fcf534d2640 1 -- 172.21.15.196:0/4004071966 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf4c19b970 con 0x7fcf4c107660 2024-09-19T08:54:19.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.226+0000 7fcf534d2640 1 -- 172.21.15.196:0/4004071966 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf4c102d10 con 0x7fcf4c107660 2024-09-19T08:54:19.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.227+0000 7fcf427fc640 1 -- 172.21.15.196:0/4004071966 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fcf48042070 con 0x7fcf4c107660 2024-09-19T08:54:19.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.227+0000 7fcf427fc640 1 --2- 172.21.15.196:0/4004071966 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcf38039300 0x7fcf3803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:19.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.227+0000 7fcf427fc640 1 -- 172.21.15.196:0/4004071966 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcf4806d6d0 con 0x7fcf4c107660 2024-09-19T08:54:19.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.227+0000 7fcf50a6d640 1 --2- 172.21.15.196:0/4004071966 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcf38039300 0x7fcf3803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:19.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.228+0000 7fcf50a6d640 1 --2- 172.21.15.196:0/4004071966 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcf38039300 0x7fcf3803b7c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0098c0 tx=0x7fcf3c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:19.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.231+0000 7fcf427fc640 1 -- 172.21.15.196:0/4004071966 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcf48049070 con 0x7fcf4c107660 2024-09-19T08:54:19.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.418+0000 7fcf534d2640 1 -- 172.21.15.196:0/4004071966 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi120:172.21.15.120=a;smithi153:172.21.15.153=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}) -- 0x7fcf4c198400 con 0x7fcf38039300 2024-09-19T08:54:19.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.423+0000 7fcf427fc640 1 -- 172.21.15.196:0/4004071966 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fcf4c198400 con 0x7fcf38039300 2024-09-19T08:54:19.423 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mon update... 2024-09-19T08:54:19.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.425+0000 7fcf534d2640 1 -- 172.21.15.196:0/4004071966 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcf38039300 msgr2=0x7fcf3803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:19.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.425+0000 7fcf534d2640 1 --2- 172.21.15.196:0/4004071966 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcf38039300 0x7fcf3803b7c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0098c0 tx=0x7fcf3c006db0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.425+0000 7fcf534d2640 1 -- 172.21.15.196:0/4004071966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf4c107660 msgr2=0x7fcf4c1970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:19.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.425+0000 7fcf534d2640 1 --2- 172.21.15.196:0/4004071966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf4c107660 0x7fcf4c1970a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcf48009230 tx=0x7fcf48003d40 comp rx=0 tx=0).stop 2024-09-19T08:54:19.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.425+0000 7fcf534d2640 1 -- 172.21.15.196:0/4004071966 shutdown_connections 2024-09-19T08:54:19.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.425+0000 7fcf534d2640 1 --2- 172.21.15.196:0/4004071966 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcf38039300 0x7fcf3803b7c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.425+0000 7fcf534d2640 1 --2- 172.21.15.196:0/4004071966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf4c107660 0x7fcf4c1970a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.425+0000 7fcf534d2640 1 -- 172.21.15.196:0/4004071966 >> 172.21.15.196:0/4004071966 conn(0x7fcf4c0fd2b0 msgr2=0x7fcf4c0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:19.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.425+0000 7fcf534d2640 1 -- 172.21.15.196:0/4004071966 shutdown_connections 2024-09-19T08:54:19.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:19.425+0000 7fcf534d2640 1 -- 172.21.15.196:0/4004071966 wait complete. 2024-09-19T08:54:20.014 DEBUG:teuthology.orchestra.run.smithi153:mon.b> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mon.b.service 2024-09-19T08:54:20.017 DEBUG:teuthology.orchestra.run.smithi196:mon.c> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mon.c.service 2024-09-19T08:54:20.019 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T08:54:20.019 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T08:54:20.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:19 smithi120 ceph-mon[31150]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-19T08:54:20.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:19 smithi120 ceph-mon[31150]: Updating smithi196:/var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:54:20.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:19 smithi120 ceph-mon[31150]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:54:20.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:19 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:20.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:19 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:20.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:19 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:20.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:19 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:20.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:19 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:54:20.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:19 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:20.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:19 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:54:20.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:19 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:20.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:19 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:54:20.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:19 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:20.758 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:54:21.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:20 smithi120 ceph-mon[31150]: Updating smithi196:/var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:54:21.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:20 smithi120 ceph-mon[31150]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi120:172.21.15.120=a;smithi153:172.21.15.153=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:54:21.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:20 smithi120 ceph-mon[31150]: Saving service mon spec with placement smithi120:172.21.15.120=a;smithi153:172.21.15.153=b;smithi196:172.21.15.196=c;count:3 2024-09-19T08:54:21.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:20 smithi120 ceph-mon[31150]: Deploying daemon mon.c on smithi196 2024-09-19T08:54:21.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:20 smithi120 ceph-mon[31150]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:21.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.153+0000 7fce78d7e640 1 -- 172.21.15.196:0/1550540740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce74107690 msgr2=0x7fce74107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:21.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.153+0000 7fce78d7e640 1 --2- 172.21.15.196:0/1550540740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce74107690 0x7fce74107a70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009880 tx=0x7fce6c02a5d0 comp rx=0 tx=0).stop 2024-09-19T08:54:21.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.153+0000 7fce78d7e640 1 -- 172.21.15.196:0/1550540740 shutdown_connections 2024-09-19T08:54:21.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.153+0000 7fce78d7e640 1 --2- 172.21.15.196:0/1550540740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce74107690 0x7fce74107a70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:21.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.153+0000 7fce78d7e640 1 -- 172.21.15.196:0/1550540740 >> 172.21.15.196:0/1550540740 conn(0x7fce74076250 msgr2=0x7fce74076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:21.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.153+0000 7fce78d7e640 1 -- 172.21.15.196:0/1550540740 shutdown_connections 2024-09-19T08:54:21.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.153+0000 7fce78d7e640 1 -- 172.21.15.196:0/1550540740 wait complete. 2024-09-19T08:54:21.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.154+0000 7fce78d7e640 1 Processor -- start 2024-09-19T08:54:21.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.154+0000 7fce78d7e640 1 -- start start 2024-09-19T08:54:21.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.154+0000 7fce78d7e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce74107690 0x7fce741993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:21.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.154+0000 7fce78d7e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fce7410bce0 con 0x7fce74107690 2024-09-19T08:54:21.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.154+0000 7fce7259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce74107690 0x7fce741993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:21.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.154+0000 7fce7259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce74107690 0x7fce741993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.196:41166/0 (socket says 172.21.15.196:41166) 2024-09-19T08:54:21.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.154+0000 7fce7259c640 1 -- 172.21.15.196:0/2437368149 learned_addr learned my addr 172.21.15.196:0/2437368149 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-19T08:54:21.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.155+0000 7fce7259c640 1 -- 172.21.15.196:0/2437368149 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce741998e0 con 0x7fce74107690 2024-09-19T08:54:21.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.155+0000 7fce7259c640 1 --2- 172.21.15.196:0/2437368149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce74107690 0x7fce741993a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009230 tx=0x7fce6c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:21.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.155+0000 7fce637fe640 1 -- 172.21.15.196:0/2437368149 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce6c03f070 con 0x7fce74107690 2024-09-19T08:54:21.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.155+0000 7fce637fe640 1 -- 172.21.15.196:0/2437368149 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce6c03bc80 con 0x7fce74107690 2024-09-19T08:54:21.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.155+0000 7fce637fe640 1 -- 172.21.15.196:0/2437368149 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce6c035030 con 0x7fce74107690 2024-09-19T08:54:21.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.155+0000 7fce78d7e640 1 -- 172.21.15.196:0/2437368149 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce74199b70 con 0x7fce74107690 2024-09-19T08:54:21.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.155+0000 7fce78d7e640 1 -- 172.21.15.196:0/2437368149 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce74199fd0 con 0x7fce74107690 2024-09-19T08:54:21.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.156+0000 7fce637fe640 1 -- 172.21.15.196:0/2437368149 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fce6c042070 con 0x7fce74107690 2024-09-19T08:54:21.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.156+0000 7fce78d7e640 1 -- 172.21.15.196:0/2437368149 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce74102d30 con 0x7fce74107690 2024-09-19T08:54:21.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.156+0000 7fce637fe640 1 --2- 172.21.15.196:0/2437368149 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fce5c039300 0x7fce5c03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:21.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.156+0000 7fce637fe640 1 -- 172.21.15.196:0/2437368149 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fce6c06d070 con 0x7fce74107690 2024-09-19T08:54:21.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.156+0000 7fce71d9b640 1 --2- 172.21.15.196:0/2437368149 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fce5c039300 0x7fce5c03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:21.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.157+0000 7fce71d9b640 1 --2- 172.21.15.196:0/2437368149 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fce5c039300 0x7fce5c03b7c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fce6800ad30 tx=0x7fce680093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:21.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.161+0000 7fce637fe640 1 -- 172.21.15.196:0/2437368149 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fce6c0372d0 con 0x7fce74107690 2024-09-19T08:54:21.395 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.395+0000 7fce78d7e640 1 -- 172.21.15.196:0/2437368149 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fce7419a760 con 0x7fce74107690 2024-09-19T08:54:21.395 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.396+0000 7fce637fe640 1 -- 172.21.15.196:0/2437368149 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7fce6c031dd0 con 0x7fce74107690 2024-09-19T08:54:21.396 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T08:54:21.396 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"507393d2-7664-11ef-bceb-c7b262605968","modified":"2024-09-19T08:53:01.874293Z","created":"2024-09-19T08:53:01.874293Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:3300","nonce":0},{"type":"v1","addr":"172.21.15.120:6789","nonce":0}]},"addr":"172.21.15.120:6789/0","public_addr":"172.21.15.120:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T08:54:21.396 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-09-19T08:54:21.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.398+0000 7fce78d7e640 1 -- 172.21.15.196:0/2437368149 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fce5c039300 msgr2=0x7fce5c03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:21.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.398+0000 7fce78d7e640 1 --2- 172.21.15.196:0/2437368149 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fce5c039300 0x7fce5c03b7c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fce6800ad30 tx=0x7fce680093f0 comp rx=0 tx=0).stop 2024-09-19T08:54:21.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.398+0000 7fce78d7e640 1 -- 172.21.15.196:0/2437368149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce74107690 msgr2=0x7fce741993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:21.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.398+0000 7fce78d7e640 1 --2- 172.21.15.196:0/2437368149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce74107690 0x7fce741993a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009230 tx=0x7fce6c003d40 comp rx=0 tx=0).stop 2024-09-19T08:54:21.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.398+0000 7fce78d7e640 1 -- 172.21.15.196:0/2437368149 shutdown_connections 2024-09-19T08:54:21.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.398+0000 7fce78d7e640 1 --2- 172.21.15.196:0/2437368149 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fce5c039300 0x7fce5c03b7c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:21.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.398+0000 7fce78d7e640 1 --2- 172.21.15.196:0/2437368149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce74107690 0x7fce741993a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:21.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.398+0000 7fce78d7e640 1 -- 172.21.15.196:0/2437368149 >> 172.21.15.196:0/2437368149 conn(0x7fce74076250 msgr2=0x7fce741039e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:21.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.398+0000 7fce78d7e640 1 -- 172.21.15.196:0/2437368149 shutdown_connections 2024-09-19T08:54:21.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:21.398+0000 7fce78d7e640 1 -- 172.21.15.196:0/2437368149 wait complete. 2024-09-19T08:54:22.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:21 smithi120 ceph-mon[31150]: from='client.? 172.21.15.196:0/2437368149' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:54:23.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:22 smithi120 ceph-mon[31150]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:23.075 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T08:54:23.076 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T08:54:23.476 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:54:24.166 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.148+0000 7fde6cce7640 1 -- 172.21.15.196:0/2882423758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde681066c0 msgr2=0x7fde68106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:24.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.148+0000 7fde6cce7640 1 --2- 172.21.15.196:0/2882423758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde681066c0 0x7fde68106aa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fde58009880 tx=0x7fde5802a5d0 comp rx=0 tx=0).stop 2024-09-19T08:54:24.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.148+0000 7fde6cce7640 1 -- 172.21.15.196:0/2882423758 shutdown_connections 2024-09-19T08:54:24.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.148+0000 7fde6cce7640 1 --2- 172.21.15.196:0/2882423758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde681066c0 0x7fde68106aa0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:24.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.148+0000 7fde6cce7640 1 -- 172.21.15.196:0/2882423758 >> 172.21.15.196:0/2882423758 conn(0x7fde680fb9a0 msgr2=0x7fde680fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:24.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.148+0000 7fde6cce7640 1 -- 172.21.15.196:0/2882423758 shutdown_connections 2024-09-19T08:54:24.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.148+0000 7fde6cce7640 1 -- 172.21.15.196:0/2882423758 wait complete. 2024-09-19T08:54:24.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.149+0000 7fde6cce7640 1 Processor -- start 2024-09-19T08:54:24.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.149+0000 7fde6cce7640 1 -- start start 2024-09-19T08:54:24.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.149+0000 7fde6cce7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde681066c0 0x7fde68197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:24.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.149+0000 7fde6cce7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fde6810acd0 con 0x7fde681066c0 2024-09-19T08:54:24.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.150+0000 7fde6659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde681066c0 0x7fde68197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:24.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.150+0000 7fde6659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde681066c0 0x7fde68197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.196:41184/0 (socket says 172.21.15.196:41184) 2024-09-19T08:54:24.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.150+0000 7fde6659c640 1 -- 172.21.15.196:0/117145104 learned_addr learned my addr 172.21.15.196:0/117145104 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-19T08:54:24.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.150+0000 7fde6659c640 1 -- 172.21.15.196:0/117145104 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde68197770 con 0x7fde681066c0 2024-09-19T08:54:24.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.150+0000 7fde6659c640 1 --2- 172.21.15.196:0/117145104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde681066c0 0x7fde68197230 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fde58009230 tx=0x7fde58003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:24.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.150+0000 7fde577fe640 1 -- 172.21.15.196:0/117145104 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde5803f070 con 0x7fde681066c0 2024-09-19T08:54:24.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.150+0000 7fde577fe640 1 -- 172.21.15.196:0/117145104 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde5803bc80 con 0x7fde681066c0 2024-09-19T08:54:24.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.150+0000 7fde6cce7640 1 -- 172.21.15.196:0/117145104 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde68197a00 con 0x7fde681066c0 2024-09-19T08:54:24.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.150+0000 7fde577fe640 1 -- 172.21.15.196:0/117145104 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde58035030 con 0x7fde681066c0 2024-09-19T08:54:24.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.150+0000 7fde6cce7640 1 -- 172.21.15.196:0/117145104 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde6819bab0 con 0x7fde681066c0 2024-09-19T08:54:24.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.151+0000 7fde577fe640 1 -- 172.21.15.196:0/117145104 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fde58042070 con 0x7fde681066c0 2024-09-19T08:54:24.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.152+0000 7fde577fe640 1 --2- 172.21.15.196:0/117145104 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fde500392a0 0x7fde5003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:24.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.152+0000 7fde6cce7640 1 -- 172.21.15.196:0/117145104 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde68101d10 con 0x7fde681066c0 2024-09-19T08:54:24.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.152+0000 7fde577fe640 1 -- 172.21.15.196:0/117145104 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fde5806cd90 con 0x7fde681066c0 2024-09-19T08:54:24.169 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.152+0000 7fde65d9b640 1 --2- 172.21.15.196:0/117145104 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fde500392a0 0x7fde5003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:24.169 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.152+0000 7fde65d9b640 1 --2- 172.21.15.196:0/117145104 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fde500392a0 0x7fde5003b760 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fde5c0098c0 tx=0x7fde5c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:24.169 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.157+0000 7fde577fe640 1 -- 172.21.15.196:0/117145104 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fde58031e20 con 0x7fde681066c0 2024-09-19T08:54:24.392 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:24 smithi196 systemd[1]: Starting Ceph mon.c for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T08:54:24.392 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:24 smithi196 podman[35114]: 2024-09-19 08:54:24.391704263 +0000 UTC m=+0.102907390 container create 636bb695af2c65422d97e50340ec937b8d1939efa2b2dd72a7a85113d03798d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-19T08:54:24.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.410+0000 7fde6cce7640 1 -- 172.21.15.196:0/117145104 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fde68106aa0 con 0x7fde681066c0 2024-09-19T08:54:24.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.411+0000 7fde577fe640 1 -- 172.21.15.196:0/117145104 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7fde580372d0 con 0x7fde681066c0 2024-09-19T08:54:24.411 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T08:54:24.411 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"507393d2-7664-11ef-bceb-c7b262605968","modified":"2024-09-19T08:53:01.874293Z","created":"2024-09-19T08:53:01.874293Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:3300","nonce":0},{"type":"v1","addr":"172.21.15.120:6789","nonce":0}]},"addr":"172.21.15.120:6789/0","public_addr":"172.21.15.120:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T08:54:24.412 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-09-19T08:54:24.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.413+0000 7fde6cce7640 1 -- 172.21.15.196:0/117145104 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fde500392a0 msgr2=0x7fde5003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:24.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.413+0000 7fde6cce7640 1 --2- 172.21.15.196:0/117145104 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fde500392a0 0x7fde5003b760 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fde5c0098c0 tx=0x7fde5c006db0 comp rx=0 tx=0).stop 2024-09-19T08:54:24.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.413+0000 7fde6cce7640 1 -- 172.21.15.196:0/117145104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde681066c0 msgr2=0x7fde68197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:24.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.413+0000 7fde6cce7640 1 --2- 172.21.15.196:0/117145104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde681066c0 0x7fde68197230 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fde58009230 tx=0x7fde58003d40 comp rx=0 tx=0).stop 2024-09-19T08:54:24.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.413+0000 7fde6cce7640 1 -- 172.21.15.196:0/117145104 shutdown_connections 2024-09-19T08:54:24.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.413+0000 7fde6cce7640 1 --2- 172.21.15.196:0/117145104 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fde500392a0 0x7fde5003b760 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:24.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.413+0000 7fde6cce7640 1 --2- 172.21.15.196:0/117145104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde681066c0 0x7fde68197230 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:24.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.413+0000 7fde6cce7640 1 -- 172.21.15.196:0/117145104 >> 172.21.15.196:0/117145104 conn(0x7fde680fb9a0 msgr2=0x7fde680fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:24.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.413+0000 7fde6cce7640 1 -- 172.21.15.196:0/117145104 shutdown_connections 2024-09-19T08:54:24.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:24.413+0000 7fde6cce7640 1 -- 172.21.15.196:0/117145104 wait complete. 2024-09-19T08:54:24.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:24 smithi196 podman[35114]: 2024-09-19 08:54:24.304643836 +0000 UTC m=+0.015846975 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:54:25.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:24 smithi120 ceph-mon[31150]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:25.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:24 smithi120 ceph-mon[31150]: from='client.? 172.21.15.196:0/117145104' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:54:25.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 podman[35114]: 2024-09-19 08:54:25.164408215 +0000 UTC m=+0.875611341 container init 636bb695af2c65422d97e50340ec937b8d1939efa2b2dd72a7a85113d03798d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2) 2024-09-19T08:54:25.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 podman[35114]: 2024-09-19 08:54:25.167905654 +0000 UTC m=+0.879108780 container start 636bb695af2c65422d97e50340ec937b8d1939efa2b2dd72a7a85113d03798d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-19T08:54:25.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T08:54:25.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T08:54:25.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: pidfile_write: ignore empty --pid-file 2024-09-19T08:54:25.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: load: jerasure load: lrc 2024-09-19T08:54:25.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: RocksDB version: 7.9.2 2024-09-19T08:54:25.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Git sha 0 2024-09-19T08:54:25.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T08:54:25.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: DB SUMMARY 2024-09-19T08:54:25.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: DB Session ID: 3HPCSRAPRTTI5SJ1XUWU 2024-09-19T08:54:25.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: CURRENT file: CURRENT 2024-09-19T08:54:25.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T08:54:25.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-19T08:54:25.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-19T08:54:25.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-19T08:54:25.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.error_if_exists: 0 2024-09-19T08:54:25.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.create_if_missing: 0 2024-09-19T08:54:25.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.paranoid_checks: 1 2024-09-19T08:54:25.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T08:54:25.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T08:54:25.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T08:54:25.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.env: 0x561424e8a940 2024-09-19T08:54:25.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T08:54:25.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.info_log: 0x5614267ab6e0 2024-09-19T08:54:25.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T08:54:25.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.statistics: (nil) 2024-09-19T08:54:25.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.use_fsync: 0 2024-09-19T08:54:25.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.max_log_file_size: 0 2024-09-19T08:54:25.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T08:54:25.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T08:54:25.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T08:54:25.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T08:54:25.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.allow_fallocate: 1 2024-09-19T08:54:25.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T08:54:25.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T08:54:25.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.use_direct_reads: 0 2024-09-19T08:54:25.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T08:54:25.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T08:54:25.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.db_log_dir: 2024-09-19T08:54:25.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.wal_dir: 2024-09-19T08:54:25.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T08:54:25.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T08:54:25.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T08:54:25.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T08:54:25.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T08:54:25.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T08:54:25.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T08:54:25.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T08:54:25.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.write_buffer_manager: 0x5614267badc0 2024-09-19T08:54:25.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T08:54:25.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T08:54:25.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T08:54:25.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T08:54:25.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T08:54:25.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T08:54:25.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T08:54:25.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T08:54:25.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.unordered_write: 0 2024-09-19T08:54:25.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T08:54:25.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T08:54:25.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T08:54:25.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T08:54:25.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.row_cache: None 2024-09-19T08:54:25.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.wal_filter: None 2024-09-19T08:54:25.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T08:54:25.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T08:54:25.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.two_write_queues: 0 2024-09-19T08:54:25.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T08:54:25.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.wal_compression: 0 2024-09-19T08:54:25.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.atomic_flush: 0 2024-09-19T08:54:25.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T08:54:25.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T08:54:25.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T08:54:25.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.log_readahead_size: 0 2024-09-19T08:54:25.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T08:54:25.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T08:54:25.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T08:54:25.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T08:54:25.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T08:54:25.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T08:54:25.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T08:54:25.444 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.max_background_jobs: 2 2024-09-19T08:54:25.444 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.max_background_compactions: -1 2024-09-19T08:54:25.444 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.max_subcompactions: 1 2024-09-19T08:54:25.444 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T08:54:25.444 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T08:54:25.444 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T08:54:25.444 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T08:54:25.444 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T08:54:25.444 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T08:54:25.444 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T08:54:25.444 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T08:54:25.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.max_open_files: -1 2024-09-19T08:54:25.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T08:54:25.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T08:54:25.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T08:54:25.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T08:54:25.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.max_background_flushes: -1 2024-09-19T08:54:25.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Compression algorithms supported: 2024-09-19T08:54:25.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: kZSTD supported: 0 2024-09-19T08:54:25.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: kXpressCompression supported: 0 2024-09-19T08:54:25.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: kBZip2Compression supported: 0 2024-09-19T08:54:25.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T08:54:25.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: kLZ4Compression supported: 1 2024-09-19T08:54:25.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: kZlibCompression supported: 1 2024-09-19T08:54:25.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T08:54:25.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: kSnappyCompression supported: 1 2024-09-19T08:54:25.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T08:54:25.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T08:54:25.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-19T08:54:25.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T08:54:25.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T08:54:25.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.merge_operator: 2024-09-19T08:54:25.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.compaction_filter: None 2024-09-19T08:54:25.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T08:54:25.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T08:54:25.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T08:54:25.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T08:54:25.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5614267ab320) 2024-09-19T08:54:25.446 INFO:journalctl@ceph.mon.c.smithi196.stdout: cache_index_and_filter_blocks: 1 2024-09-19T08:54:25.447 INFO:journalctl@ceph.mon.c.smithi196.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T08:54:25.447 INFO:journalctl@ceph.mon.c.smithi196.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T08:54:25.447 INFO:journalctl@ceph.mon.c.smithi196.stdout: pin_top_level_index_and_filter: 1 2024-09-19T08:54:25.447 INFO:journalctl@ceph.mon.c.smithi196.stdout: index_type: 0 2024-09-19T08:54:25.447 INFO:journalctl@ceph.mon.c.smithi196.stdout: data_block_index_type: 0 2024-09-19T08:54:25.447 INFO:journalctl@ceph.mon.c.smithi196.stdout: index_shortening: 1 2024-09-19T08:54:25.447 INFO:journalctl@ceph.mon.c.smithi196.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T08:54:25.447 INFO:journalctl@ceph.mon.c.smithi196.stdout: checksum: 4 2024-09-19T08:54:25.447 INFO:journalctl@ceph.mon.c.smithi196.stdout: no_block_cache: 0 2024-09-19T08:54:25.447 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache: 0x56142679f1f0 2024-09-19T08:54:25.447 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache_name: BinnedLRUCache 2024-09-19T08:54:25.447 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache_options: 2024-09-19T08:54:25.447 INFO:journalctl@ceph.mon.c.smithi196.stdout: capacity : 536870912 2024-09-19T08:54:25.448 INFO:journalctl@ceph.mon.c.smithi196.stdout: num_shard_bits : 4 2024-09-19T08:54:25.448 INFO:journalctl@ceph.mon.c.smithi196.stdout: strict_capacity_limit : 0 2024-09-19T08:54:25.448 INFO:journalctl@ceph.mon.c.smithi196.stdout: high_pri_pool_ratio: 0.000 2024-09-19T08:54:25.448 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache_compressed: (nil) 2024-09-19T08:54:25.448 INFO:journalctl@ceph.mon.c.smithi196.stdout: persistent_cache: (nil) 2024-09-19T08:54:25.448 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_size: 4096 2024-09-19T08:54:25.448 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_size_deviation: 10 2024-09-19T08:54:25.448 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_restart_interval: 16 2024-09-19T08:54:25.448 INFO:journalctl@ceph.mon.c.smithi196.stdout: index_block_restart_interval: 1 2024-09-19T08:54:25.448 INFO:journalctl@ceph.mon.c.smithi196.stdout: metadata_block_size: 4096 2024-09-19T08:54:25.448 INFO:journalctl@ceph.mon.c.smithi196.stdout: partition_filters: 0 2024-09-19T08:54:25.448 INFO:journalctl@ceph.mon.c.smithi196.stdout: use_delta_encoding: 1 2024-09-19T08:54:25.448 INFO:journalctl@ceph.mon.c.smithi196.stdout: filter_policy: bloomfilter 2024-09-19T08:54:25.448 INFO:journalctl@ceph.mon.c.smithi196.stdout: whole_key_filtering: 1 2024-09-19T08:54:25.448 INFO:journalctl@ceph.mon.c.smithi196.stdout: verify_compression: 0 2024-09-19T08:54:25.449 INFO:journalctl@ceph.mon.c.smithi196.stdout: read_amp_bytes_per_bit: 0 2024-09-19T08:54:25.449 INFO:journalctl@ceph.mon.c.smithi196.stdout: format_version: 5 2024-09-19T08:54:25.449 INFO:journalctl@ceph.mon.c.smithi196.stdout: enable_index_compression: 1 2024-09-19T08:54:25.449 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_align: 0 2024-09-19T08:54:25.449 INFO:journalctl@ceph.mon.c.smithi196.stdout: max_auto_readahead_size: 262144 2024-09-19T08:54:25.449 INFO:journalctl@ceph.mon.c.smithi196.stdout: prepopulate_block_cache: 0 2024-09-19T08:54:25.449 INFO:journalctl@ceph.mon.c.smithi196.stdout: initial_auto_readahead_size: 8192 2024-09-19T08:54:25.449 INFO:journalctl@ceph.mon.c.smithi196.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T08:54:25.449 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T08:54:25.449 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T08:54:25.450 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.compression: NoCompression 2024-09-19T08:54:25.450 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T08:54:25.450 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T08:54:25.450 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T08:54:25.450 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.num_levels: 7 2024-09-19T08:54:25.450 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T08:54:25.450 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T08:54:25.450 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T08:54:25.450 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T08:54:25.450 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T08:54:25.450 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T08:54:25.450 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T08:54:25.450 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T08:54:25.450 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T08:54:25.451 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T08:54:25.451 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T08:54:25.451 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T08:54:25.451 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T08:54:25.451 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T08:54:25.451 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T08:54:25.451 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T08:54:25.451 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T08:54:25.451 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T08:54:25.451 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T08:54:25.451 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T08:54:25.451 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T08:54:25.451 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T08:54:25.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T08:54:25.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T08:54:25.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T08:54:25.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T08:54:25.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T08:54:25.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T08:54:25.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T08:54:25.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T08:54:25.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T08:54:25.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T08:54:25.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T08:54:25.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T08:54:25.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T08:54:25.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T08:54:25.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T08:54:25.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T08:54:25.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T08:54:25.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T08:54:25.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T08:54:25.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T08:54:25.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T08:54:25.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T08:54:25.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T08:54:25.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T08:54:25.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T08:54:25.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T08:54:25.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T08:54:25.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T08:54:25.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T08:54:25.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T08:54:25.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T08:54:25.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T08:54:25.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.inplace_update_support: 0 2024-09-19T08:54:25.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T08:54:25.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T08:54:25.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T08:54:25.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T08:54:25.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.bloom_locality: 0 2024-09-19T08:54:25.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.max_successive_merges: 0 2024-09-19T08:54:25.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T08:54:25.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T08:54:25.454 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T08:54:25.455 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T08:54:25.455 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.ttl: 2592000 2024-09-19T08:54:25.455 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T08:54:25.455 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T08:54:25.455 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T08:54:25.455 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.enable_blob_files: false 2024-09-19T08:54:25.455 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.min_blob_size: 0 2024-09-19T08:54:25.455 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T08:54:25.455 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T08:54:25.455 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T08:54:25.455 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T08:54:25.455 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T08:54:25.455 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T08:54:25.455 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T08:54:25.456 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T08:54:25.456 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-19T08:54:25.456 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-19T08:54:25.456 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 22b2d295-73e5-4b8b-acc0-ff5cb72c5e60 2024-09-19T08:54:25.456 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726736065194778, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-19T08:54:25.456 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-19T08:54:25.456 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726736065195188, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726736065, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "22b2d295-73e5-4b8b-acc0-ff5cb72c5e60", "db_session_id": "3HPCSRAPRTTI5SJ1XUWU", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-19T08:54:25.456 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726736065195279, "job": 1, "event": "recovery_finished"} 2024-09-19T08:54:25.456 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-19T08:54:25.456 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T08:54:25.456 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5614267cce00 2024-09-19T08:54:25.456 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: DB pointer 0x5614268c8000 2024-09-19T08:54:25.456 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T08:54:25.456 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T08:54:25.457 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** DB Stats ** 2024-09-19T08:54:25.457 INFO:journalctl@ceph.mon.c.smithi196.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T08:54:25.457 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T08:54:25.457 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T08:54:25.457 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T08:54:25.457 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T08:54:25.457 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T08:54:25.457 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T08:54:25.457 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-09-19T08:54:25.457 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** Compaction Stats [default] ** 2024-09-19T08:54:25.457 INFO:journalctl@ceph.mon.c.smithi196.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T08:54:25.457 INFO:journalctl@ceph.mon.c.smithi196.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T08:54:25.457 INFO:journalctl@ceph.mon.c.smithi196.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T08:54:25.457 INFO:journalctl@ceph.mon.c.smithi196.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T08:54:25.458 INFO:journalctl@ceph.mon.c.smithi196.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T08:54:25.458 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-09-19T08:54:25.458 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** Compaction Stats [default] ** 2024-09-19T08:54:25.458 INFO:journalctl@ceph.mon.c.smithi196.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T08:54:25.458 INFO:journalctl@ceph.mon.c.smithi196.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T08:54:25.458 INFO:journalctl@ceph.mon.c.smithi196.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T08:54:25.458 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-09-19T08:54:25.458 INFO:journalctl@ceph.mon.c.smithi196.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T08:54:25.458 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-09-19T08:54:25.458 INFO:journalctl@ceph.mon.c.smithi196.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T08:54:25.458 INFO:journalctl@ceph.mon.c.smithi196.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T08:54:25.458 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T08:54:25.458 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T08:54:25.458 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T08:54:25.459 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T08:54:25.459 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T08:54:25.459 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T08:54:25.459 INFO:journalctl@ceph.mon.c.smithi196.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T08:54:25.459 INFO:journalctl@ceph.mon.c.smithi196.stdout: Block cache BinnedLRUCache@0x56142679f1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-19T08:54:25.459 INFO:journalctl@ceph.mon.c.smithi196.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-19T08:54:25.459 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-09-19T08:54:25.459 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T08:54:25.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-19T08:54:25.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: using public_addr v2:172.21.15.196:0/0 -> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] 2024-09-19T08:54:25.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: starting mon.c rank -1 at public addrs [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] at bind addrs [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T08:54:25.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: mon.c@-1(???) e0 preinit fsid 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T08:54:25.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: mon.c@-1(synchronizing).mds e1 new map 2024-09-19T08:54:25.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-19T08:54:25.460 INFO:journalctl@ceph.mon.c.smithi196.stdout: e1 2024-09-19T08:54:25.460 INFO:journalctl@ceph.mon.c.smithi196.stdout: btime 2024-09-19T08:53:06:386334+0000 2024-09-19T08:54:25.460 INFO:journalctl@ceph.mon.c.smithi196.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T08:54:25.460 INFO:journalctl@ceph.mon.c.smithi196.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T08:54:25.460 INFO:journalctl@ceph.mon.c.smithi196.stdout: legacy client fscid: -1 2024-09-19T08:54:25.460 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-09-19T08:54:25.460 INFO:journalctl@ceph.mon.c.smithi196.stdout: No filesystems configured 2024-09-19T08:54:25.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-19T08:54:25.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-19T08:54:25.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-19T08:54:25.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-19T08:54:25.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-19T08:54:25.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-19T08:54:25.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T08:54:25.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T08:54:25.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T08:54:25.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T08:54:25.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:25.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:54:25.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/4264215085' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T08:54:25.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:25.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/4264215085' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T08:54:25.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T08:54:25.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:25.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:25.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:25.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:25.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:54:25.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:25.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:54:25.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-19T08:54:25.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: Updating smithi196:/var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:54:25.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:54:25.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:25.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:25.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:25.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:25.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:54:25.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:25.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:54:25.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:25.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:54:25.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:25.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: Updating smithi196:/var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:54:25.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi120:172.21.15.120=a;smithi153:172.21.15.153=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:54:25.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: Saving service mon spec with placement smithi120:172.21.15.120=a;smithi153:172.21.15.153=b;smithi196:172.21.15.196=c;count:3 2024-09-19T08:54:25.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: Deploying daemon mon.c on smithi196 2024-09-19T08:54:25.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:25.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: from='client.? 172.21.15.196:0/2437368149' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:54:25.463 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:25.464 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:25.464 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: from='client.? 172.21.15.196:0/117145104' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:54:25.464 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 ceph-mon[35150]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-19T08:54:25.464 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 bash[35114]: 636bb695af2c65422d97e50340ec937b8d1939efa2b2dd72a7a85113d03798d2 2024-09-19T08:54:25.464 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:25 smithi196 systemd[1]: Started Ceph mon.c for 507393d2-7664-11ef-bceb-c7b262605968. 2024-09-19T08:54:26.075 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T08:54:26.075 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T08:54:26.342 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:54:30.385 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.383+0000 7f41d6abb640 1 -- 172.21.15.196:0/2144141557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41c0003530 msgr2=0x7f41c00059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:30.385 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.383+0000 7f41d6abb640 1 --2- 172.21.15.196:0/2144141557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41c0003530 0x7f41c00059c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f41d01016a0 tx=0x7f41c402c3f0 comp rx=0 tx=0).stop 2024-09-19T08:54:30.385 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.383+0000 7f41d6abb640 1 -- 172.21.15.196:0/2144141557 shutdown_connections 2024-09-19T08:54:30.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.383+0000 7f41d6abb640 1 --2- 172.21.15.196:0/2144141557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41c0003530 0x7f41c00059c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:30.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.383+0000 7f41d6abb640 1 -- 172.21.15.196:0/2144141557 >> 172.21.15.196:0/2144141557 conn(0x7f41d00fc220 msgr2=0x7f41d00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:30.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.383+0000 7f41d6abb640 1 -- 172.21.15.196:0/2144141557 shutdown_connections 2024-09-19T08:54:30.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.383+0000 7f41d6abb640 1 -- 172.21.15.196:0/2144141557 wait complete. 2024-09-19T08:54:30.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.384+0000 7f41d6abb640 1 Processor -- start 2024-09-19T08:54:30.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.384+0000 7f41d6abb640 1 -- start start 2024-09-19T08:54:30.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.384+0000 7f41d6abb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41c0003530 0x7f41d0198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:30.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.384+0000 7f41d6abb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41d01985c0 0x7f41d0198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:30.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.384+0000 7f41d6abb640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f41d0108670 con 0x7f41c0003530 2024-09-19T08:54:30.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.384+0000 7f41d6abb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41d01084f0 con 0x7f41d01985c0 2024-09-19T08:54:30.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.384+0000 7f41cffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41d01985c0 0x7f41d0198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:30.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.384+0000 7f41d4857640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41c0003530 0x7f41d0198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:30.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.384+0000 7f41cffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41d01985c0 0x7f41d0198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:41816/0 (socket says 172.21.15.196:41816) 2024-09-19T08:54:30.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.384+0000 7f41cffff640 1 -- 172.21.15.196:0/3167781636 learned_addr learned my addr 172.21.15.196:0/3167781636 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-19T08:54:30.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.384+0000 7f41cffff640 1 -- 172.21.15.196:0/3167781636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41d01985c0 msgr2=0x7f41d0198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-19T08:54:30.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.384+0000 7f41cffff640 1 -- 172.21.15.196:0/3167781636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41d01985c0 msgr2=0x7f41d0198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T08:54:30.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.384+0000 7f41cffff640 1 --2- 172.21.15.196:0/3167781636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41d01985c0 0x7f41d0198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T08:54:30.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.385+0000 7f41cffff640 1 --2- 172.21.15.196:0/3167781636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41d01985c0 0x7f41d0198a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:54:30.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.385+0000 7f41d4857640 1 -- 172.21.15.196:0/3167781636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41d01985c0 msgr2=0x7f41d0198a20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:54:30.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.385+0000 7f41d4857640 1 --2- 172.21.15.196:0/3167781636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41d01985c0 0x7f41d0198a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:30.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.385+0000 7f41d4857640 1 -- 172.21.15.196:0/3167781636 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d01a10f0 con 0x7f41c0003530 2024-09-19T08:54:30.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.385+0000 7f41d4857640 1 --2- 172.21.15.196:0/3167781636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41c0003530 0x7f41d0198080 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f41c4036040 tx=0x7f41c4034bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:30.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.385+0000 7f41cdffb640 1 -- 172.21.15.196:0/3167781636 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f41c40051f0 con 0x7f41c0003530 2024-09-19T08:54:30.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.385+0000 7f41cdffb640 1 -- 172.21.15.196:0/3167781636 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41c4005390 con 0x7f41c0003530 2024-09-19T08:54:30.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.385+0000 7f41d6abb640 1 -- 172.21.15.196:0/3167781636 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d01a12c0 con 0x7f41c0003530 2024-09-19T08:54:30.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.385+0000 7f41cdffb640 1 -- 172.21.15.196:0/3167781636 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f41c40128c0 con 0x7f41c0003530 2024-09-19T08:54:30.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.386+0000 7f41d6abb640 1 -- 172.21.15.196:0/3167781636 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d01a17c0 con 0x7f41c0003530 2024-09-19T08:54:30.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.387+0000 7f41d6abb640 1 -- 172.21.15.196:0/3167781636 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41d0101ca0 con 0x7f41c0003530 2024-09-19T08:54:30.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.387+0000 7f41cdffb640 1 -- 172.21.15.196:0/3167781636 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f41c4004c20 con 0x7f41c0003530 2024-09-19T08:54:30.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.387+0000 7f41cdffb640 1 --2- 172.21.15.196:0/3167781636 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f41b4039040 0x7f41b403b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:30.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.387+0000 7f41cdffb640 1 -- 172.21.15.196:0/3167781636 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f41c406e580 con 0x7f41c0003530 2024-09-19T08:54:30.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.387+0000 7f41cffff640 1 --2- 172.21.15.196:0/3167781636 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f41b4039040 0x7f41b403b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:30.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.388+0000 7f41cffff640 1 --2- 172.21.15.196:0/3167781636 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f41b4039040 0x7f41b403b500 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009100 tx=0x7f41bc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:30.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.392+0000 7f41cdffb640 1 -- 172.21.15.196:0/3167781636 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f41c403c3e0 con 0x7f41c0003530 2024-09-19T08:54:30.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:54:30.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:54:30.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: mon.a calling monitor election 2024-09-19T08:54:30.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:30.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:54:30.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:54:30.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: mon.c calling monitor election 2024-09-19T08:54:30.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:30.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:54:30.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:54:30.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:30.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:54:30.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T08:54:30.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: monmap epoch 2 2024-09-19T08:54:30.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: fsid 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T08:54:30.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: last_changed 2024-09-19T08:54:25.220102+0000 2024-09-19T08:54:30.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: created 2024-09-19T08:53:01.874293+0000 2024-09-19T08:54:30.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: min_mon_release 19 (squid) 2024-09-19T08:54:30.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: election_strategy: 1 2024-09-19T08:54:30.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: 0: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.a 2024-09-19T08:54:30.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-19T08:54:30.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: fsmap 2024-09-19T08:54:30.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T08:54:30.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: mgrmap e13: x(active, since 38s) 2024-09-19T08:54:30.577 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: overall HEALTH_OK 2024-09-19T08:54:30.577 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:30.577 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:30.577 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:30.577 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:54:30.577 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:30 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:30.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.646+0000 7f41d6abb640 1 -- 172.21.15.196:0/3167781636 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f41d002d7c0 con 0x7f41c0003530 2024-09-19T08:54:30.647 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.647+0000 7f41cdffb640 1 -- 172.21.15.196:0/3167781636 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f41c402cbf0 con 0x7f41c0003530 2024-09-19T08:54:30.647 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T08:54:30.647 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":2,"fsid":"507393d2-7664-11ef-bceb-c7b262605968","modified":"2024-09-19T08:54:25.220102Z","created":"2024-09-19T08:53:01.874293Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:3300","nonce":0},{"type":"v1","addr":"172.21.15.120:6789","nonce":0}]},"addr":"172.21.15.120:6789/0","public_addr":"172.21.15.120:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T08:54:30.647 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 2 2024-09-19T08:54:30.648 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:54:30.648 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:54:30.648 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: mon.a calling monitor election 2024-09-19T08:54:30.648 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:30.648 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:54:30.648 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:54:30.648 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: mon.c calling monitor election 2024-09-19T08:54:30.648 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:30.648 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:54:30.648 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:54:30.648 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:30.648 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:54:30.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T08:54:30.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: monmap epoch 2 2024-09-19T08:54:30.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: fsid 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T08:54:30.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: last_changed 2024-09-19T08:54:25.220102+0000 2024-09-19T08:54:30.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: created 2024-09-19T08:53:01.874293+0000 2024-09-19T08:54:30.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: min_mon_release 19 (squid) 2024-09-19T08:54:30.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: election_strategy: 1 2024-09-19T08:54:30.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: 0: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.a 2024-09-19T08:54:30.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-19T08:54:30.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: fsmap 2024-09-19T08:54:30.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T08:54:30.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: mgrmap e13: x(active, since 38s) 2024-09-19T08:54:30.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: overall HEALTH_OK 2024-09-19T08:54:30.649 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:30.650 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:30.650 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:30.650 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:54:30.650 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:30 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:30.650 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.649+0000 7f41d6abb640 1 -- 172.21.15.196:0/3167781636 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f41b4039040 msgr2=0x7f41b403b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:30.650 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.649+0000 7f41d6abb640 1 --2- 172.21.15.196:0/3167781636 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f41b4039040 0x7f41b403b500 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009100 tx=0x7f41bc008040 comp rx=0 tx=0).stop 2024-09-19T08:54:30.650 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.649+0000 7f41d6abb640 1 -- 172.21.15.196:0/3167781636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41c0003530 msgr2=0x7f41d0198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:30.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.649+0000 7f41d6abb640 1 --2- 172.21.15.196:0/3167781636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41c0003530 0x7f41d0198080 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f41c4036040 tx=0x7f41c4034bb0 comp rx=0 tx=0).stop 2024-09-19T08:54:30.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.650+0000 7f41d6abb640 1 -- 172.21.15.196:0/3167781636 shutdown_connections 2024-09-19T08:54:30.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.650+0000 7f41d6abb640 1 --2- 172.21.15.196:0/3167781636 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f41b4039040 0x7f41b403b500 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:30.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.650+0000 7f41d6abb640 1 --2- 172.21.15.196:0/3167781636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41d01985c0 0x7f41d0198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:30.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.650+0000 7f41d6abb640 1 --2- 172.21.15.196:0/3167781636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41c0003530 0x7f41d0198080 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:30.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.650+0000 7f41d6abb640 1 -- 172.21.15.196:0/3167781636 >> 172.21.15.196:0/3167781636 conn(0x7f41d00fc220 msgr2=0x7f41d00fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:30.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.650+0000 7f41d6abb640 1 -- 172.21.15.196:0/3167781636 shutdown_connections 2024-09-19T08:54:30.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:30.650+0000 7f41d6abb640 1 -- 172.21.15.196:0/3167781636 wait complete. 2024-09-19T08:54:31.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:31 smithi120 ceph-mon[31150]: Deploying daemon mon.b on smithi153 2024-09-19T08:54:31.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:31 smithi120 ceph-mon[31150]: from='client.? 172.21.15.196:0/3167781636' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:54:31.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:31 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:54:31.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:31 smithi196 ceph-mon[35150]: Deploying daemon mon.b on smithi153 2024-09-19T08:54:31.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:31 smithi196 ceph-mon[35150]: from='client.? 172.21.15.196:0/3167781636' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:54:31.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:31 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:54:32.221 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T08:54:32.221 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T08:54:32.476 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:54:32.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:32 smithi120 ceph-mon[31150]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:32.595 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:32 smithi196 ceph-mon[35150]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:32.906 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.903+0000 7f9d21e48640 1 -- 172.21.15.196:0/4086024744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d1c107660 msgr2=0x7f9d1c107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:32.906 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.903+0000 7f9d21e48640 1 --2- 172.21.15.196:0/4086024744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d1c107660 0x7f9d1c107a40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009880 tx=0x7f9d0c02a5d0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.906 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.903+0000 7f9d21e48640 1 -- 172.21.15.196:0/4086024744 shutdown_connections 2024-09-19T08:54:32.906 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.903+0000 7f9d21e48640 1 --2- 172.21.15.196:0/4086024744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d1c107660 0x7f9d1c107a40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.906 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.903+0000 7f9d21e48640 1 -- 172.21.15.196:0/4086024744 >> 172.21.15.196:0/4086024744 conn(0x7f9d1c0fd320 msgr2=0x7f9d1c0ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:32.906 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.903+0000 7f9d21e48640 1 -- 172.21.15.196:0/4086024744 shutdown_connections 2024-09-19T08:54:32.906 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.903+0000 7f9d21e48640 1 -- 172.21.15.196:0/4086024744 wait complete. 2024-09-19T08:54:32.906 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.903+0000 7f9d21e48640 1 Processor -- start 2024-09-19T08:54:32.906 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.904+0000 7f9d21e48640 1 -- start start 2024-09-19T08:54:32.906 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.904+0000 7f9d21e48640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d1c107660 0x7f9d1c1995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:32.906 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.904+0000 7f9d21e48640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d1c199af0 0x7f9d1c19de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:32.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.904+0000 7f9d21e48640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c075c30 con 0x7f9d1c107660 2024-09-19T08:54:32.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.904+0000 7f9d21e48640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c075ab0 con 0x7f9d1c199af0 2024-09-19T08:54:32.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.904+0000 7f9d1affd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d1c199af0 0x7f9d1c19de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:32.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.904+0000 7f9d1affd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d1c199af0 0x7f9d1c19de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:41820/0 (socket says 172.21.15.196:41820) 2024-09-19T08:54:32.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.904+0000 7f9d1affd640 1 -- 172.21.15.196:0/1354943950 learned_addr learned my addr 172.21.15.196:0/1354943950 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-19T08:54:32.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.904+0000 7f9d1b7fe640 1 --2- 172.21.15.196:0/1354943950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d1c107660 0x7f9d1c1995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:32.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.904+0000 7f9d1affd640 1 -- 172.21.15.196:0/1354943950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d1c199af0 msgr2=0x7f9d1c19de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-19T08:54:32.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.904+0000 7f9d1affd640 1 -- 172.21.15.196:0/1354943950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d1c199af0 msgr2=0x7f9d1c19de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T08:54:32.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.904+0000 7f9d1affd640 1 --2- 172.21.15.196:0/1354943950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d1c199af0 0x7f9d1c19de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T08:54:32.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.904+0000 7f9d1affd640 1 --2- 172.21.15.196:0/1354943950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d1c199af0 0x7f9d1c19de80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:54:32.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.904+0000 7f9d1b7fe640 1 -- 172.21.15.196:0/1354943950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d1c199af0 msgr2=0x7f9d1c19de80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:54:32.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.904+0000 7f9d1b7fe640 1 --2- 172.21.15.196:0/1354943950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d1c199af0 0x7f9d1c19de80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.904+0000 7f9d1b7fe640 1 -- 172.21.15.196:0/1354943950 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1c19a110 con 0x7f9d1c107660 2024-09-19T08:54:32.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.905+0000 7f9d1b7fe640 1 --2- 172.21.15.196:0/1354943950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d1c107660 0x7f9d1c1995b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c005ab0 tx=0x7f9d0c032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:32.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.905+0000 7f9d18ff9640 1 -- 172.21.15.196:0/1354943950 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f9d0c035070 con 0x7f9d1c107660 2024-09-19T08:54:32.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.905+0000 7f9d21e48640 1 -- 172.21.15.196:0/1354943950 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d1c19e3c0 con 0x7f9d1c107660 2024-09-19T08:54:32.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.905+0000 7f9d18ff9640 1 -- 172.21.15.196:0/1354943950 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d0c032e40 con 0x7f9d1c107660 2024-09-19T08:54:32.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.905+0000 7f9d18ff9640 1 -- 172.21.15.196:0/1354943950 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f9d0c03bc10 con 0x7f9d1c107660 2024-09-19T08:54:32.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.905+0000 7f9d21e48640 1 -- 172.21.15.196:0/1354943950 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d1c19e900 con 0x7f9d1c107660 2024-09-19T08:54:32.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.906+0000 7f9d18ff9640 1 -- 172.21.15.196:0/1354943950 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f9d0c042030 con 0x7f9d1c107660 2024-09-19T08:54:32.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.906+0000 7f9d21e48640 1 -- 172.21.15.196:0/1354943950 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d1c102da0 con 0x7f9d1c107660 2024-09-19T08:54:32.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.906+0000 7f9d18ff9640 1 --2- 172.21.15.196:0/1354943950 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9d04039040 0x7f9d0403b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:32.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.906+0000 7f9d18ff9640 1 -- 172.21.15.196:0/1354943950 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9d0c040070 con 0x7f9d1c107660 2024-09-19T08:54:32.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.906+0000 7f9d1affd640 1 --2- 172.21.15.196:0/1354943950 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9d04039040 0x7f9d0403b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:32.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.907+0000 7f9d1affd640 1 --2- 172.21.15.196:0/1354943950 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9d04039040 0x7f9d0403b500 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009100 tx=0x7f9d10008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:32.911 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:32.912+0000 7f9d18ff9640 1 -- 172.21.15.196:0/1354943950 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9d0c04c350 con 0x7f9d1c107660 2024-09-19T08:54:33.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:33.145+0000 7f9d21e48640 1 -- 172.21.15.196:0/1354943950 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9d1c19ebe0 con 0x7f9d1c107660 2024-09-19T08:54:33.146 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:33.146+0000 7f9d18ff9640 1 -- 172.21.15.196:0/1354943950 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f9d0c04ad50 con 0x7f9d1c107660 2024-09-19T08:54:33.146 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T08:54:33.146 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":2,"fsid":"507393d2-7664-11ef-bceb-c7b262605968","modified":"2024-09-19T08:54:25.220102Z","created":"2024-09-19T08:53:01.874293Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:3300","nonce":0},{"type":"v1","addr":"172.21.15.120:6789","nonce":0}]},"addr":"172.21.15.120:6789/0","public_addr":"172.21.15.120:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T08:54:33.146 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 2 2024-09-19T08:54:33.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:33.149+0000 7f9d21e48640 1 -- 172.21.15.196:0/1354943950 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9d04039040 msgr2=0x7f9d0403b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:33.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:33.149+0000 7f9d21e48640 1 --2- 172.21.15.196:0/1354943950 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9d04039040 0x7f9d0403b500 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009100 tx=0x7f9d10008040 comp rx=0 tx=0).stop 2024-09-19T08:54:33.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:33.149+0000 7f9d21e48640 1 -- 172.21.15.196:0/1354943950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d1c107660 msgr2=0x7f9d1c1995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:33.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:33.149+0000 7f9d21e48640 1 --2- 172.21.15.196:0/1354943950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d1c107660 0x7f9d1c1995b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c005ab0 tx=0x7f9d0c032c50 comp rx=0 tx=0).stop 2024-09-19T08:54:33.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:33.149+0000 7f9d21e48640 1 -- 172.21.15.196:0/1354943950 shutdown_connections 2024-09-19T08:54:33.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:33.149+0000 7f9d21e48640 1 --2- 172.21.15.196:0/1354943950 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9d04039040 0x7f9d0403b500 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:33.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:33.149+0000 7f9d21e48640 1 --2- 172.21.15.196:0/1354943950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d1c199af0 0x7f9d1c19de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:33.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:33.149+0000 7f9d21e48640 1 --2- 172.21.15.196:0/1354943950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d1c107660 0x7f9d1c1995b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:33.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:33.149+0000 7f9d21e48640 1 -- 172.21.15.196:0/1354943950 >> 172.21.15.196:0/1354943950 conn(0x7f9d1c0fd320 msgr2=0x7f9d1c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:33.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:33.149+0000 7f9d21e48640 1 -- 172.21.15.196:0/1354943950 shutdown_connections 2024-09-19T08:54:33.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:33.149+0000 7f9d21e48640 1 -- 172.21.15.196:0/1354943950 wait complete. 2024-09-19T08:54:33.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:33 smithi196 ceph-mon[35150]: from='client.? 172.21.15.196:0/1354943950' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:54:33.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:33 smithi120 ceph-mon[31150]: from='client.? 172.21.15.196:0/1354943950' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:54:34.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:33 smithi153 systemd[1]: Started Ceph mon.b for 507393d2-7664-11ef-bceb-c7b262605968. 2024-09-19T08:54:34.685 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T08:54:34.686 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T08:54:34.941 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:54:39.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:54:39.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: mon.c calling monitor election 2024-09-19T08:54:39.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: mon.a calling monitor election 2024-09-19T08:54:39.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:54:39.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:54:39.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:54:39.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:39.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:54:39.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:54:39.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:39.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:54:39.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:54:39.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T08:54:39.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: monmap epoch 3 2024-09-19T08:54:39.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: fsid 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T08:54:39.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: last_changed 2024-09-19T08:54:33.996914+0000 2024-09-19T08:54:39.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: created 2024-09-19T08:53:01.874293+0000 2024-09-19T08:54:39.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: min_mon_release 19 (squid) 2024-09-19T08:54:39.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: election_strategy: 1 2024-09-19T08:54:39.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: 0: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.a 2024-09-19T08:54:39.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-19T08:54:39.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: 2: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.b 2024-09-19T08:54:39.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: fsmap 2024-09-19T08:54:39.327 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T08:54:39.327 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: mgrmap e13: x(active, since 47s) 2024-09-19T08:54:39.327 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: overall HEALTH_OK 2024-09-19T08:54:39.327 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:39.327 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:39.327 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:39.327 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:39 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:39.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:54:39.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: mon.c calling monitor election 2024-09-19T08:54:39.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: mon.a calling monitor election 2024-09-19T08:54:39.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:54:39.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:54:39.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:54:39.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:39.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:54:39.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:54:39.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:39.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:54:39.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:54:39.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T08:54:39.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: monmap epoch 3 2024-09-19T08:54:39.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: fsid 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T08:54:39.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: last_changed 2024-09-19T08:54:33.996914+0000 2024-09-19T08:54:39.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: created 2024-09-19T08:53:01.874293+0000 2024-09-19T08:54:39.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: min_mon_release 19 (squid) 2024-09-19T08:54:39.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: election_strategy: 1 2024-09-19T08:54:39.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: 0: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.a 2024-09-19T08:54:39.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-19T08:54:39.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: 2: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.b 2024-09-19T08:54:39.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: fsmap 2024-09-19T08:54:39.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T08:54:39.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: mgrmap e13: x(active, since 47s) 2024-09-19T08:54:39.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: overall HEALTH_OK 2024-09-19T08:54:39.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:39.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:39.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:39.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:39 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:39.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.760+0000 7fc4ebfef640 1 -- 172.21.15.196:0/835663524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4cc003530 msgr2=0x7fc4cc0059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:39.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.760+0000 7fc4ebfef640 1 --2- 172.21.15.196:0/835663524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4cc003530 0x7fc4cc0059c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc4e406a1a0 tx=0x7fc4dc02c490 comp rx=0 tx=0).stop 2024-09-19T08:54:39.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.760+0000 7fc4ebfef640 1 -- 172.21.15.196:0/835663524 shutdown_connections 2024-09-19T08:54:39.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.760+0000 7fc4ebfef640 1 --2- 172.21.15.196:0/835663524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4cc003530 0x7fc4cc0059c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:39.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.760+0000 7fc4ebfef640 1 -- 172.21.15.196:0/835663524 >> 172.21.15.196:0/835663524 conn(0x7fc4e40fc280 msgr2=0x7fc4e40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:39.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.760+0000 7fc4ebfef640 1 -- 172.21.15.196:0/835663524 shutdown_connections 2024-09-19T08:54:39.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.760+0000 7fc4ebfef640 1 -- 172.21.15.196:0/835663524 wait complete. 2024-09-19T08:54:39.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.761+0000 7fc4ebfef640 1 Processor -- start 2024-09-19T08:54:39.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.761+0000 7fc4ebfef640 1 -- start start 2024-09-19T08:54:39.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.761+0000 7fc4ebfef640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4cc003530 0x7fc4e4197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:39.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.761+0000 7fc4ebfef640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4e4197980 0x7fc4e419bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:39.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.761+0000 7fc4ebfef640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4e419c250 0x7fc4e419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:39.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.762+0000 7fc4ebfef640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e4109750 con 0x7fc4e419c250 2024-09-19T08:54:39.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.762+0000 7fc4ebfef640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e41095d0 con 0x7fc4cc003530 2024-09-19T08:54:39.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.762+0000 7fc4ebfef640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e41098d0 con 0x7fc4e4197980 2024-09-19T08:54:39.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.762+0000 7fc4e958a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4e4197980 0x7fc4e419bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:39.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.762+0000 7fc4e958a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4e4197980 0x7fc4e419bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:37594/0 (socket says 172.21.15.196:37594) 2024-09-19T08:54:39.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.762+0000 7fc4e958a640 1 -- 172.21.15.196:0/3265528642 learned_addr learned my addr 172.21.15.196:0/3265528642 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-19T08:54:39.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.762+0000 7fc4e958a640 1 -- 172.21.15.196:0/3265528642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4e4197980 msgr2=0x7fc4e419bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-19T08:54:39.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.762+0000 7fc4e958a640 1 -- 172.21.15.196:0/3265528642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4e4197980 msgr2=0x7fc4e419bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T08:54:39.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.762+0000 7fc4e958a640 1 --2- 172.21.15.196:0/3265528642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4e4197980 0x7fc4e419bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T08:54:39.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.762+0000 7fc4ea58c640 1 --2- 172.21.15.196:0/3265528642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4e419c250 0x7fc4e419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:39.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.762+0000 7fc4e9d8b640 1 --2- 172.21.15.196:0/3265528642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4cc003530 0x7fc4e4197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:39.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.762+0000 7fc4e958a640 1 --2- 172.21.15.196:0/3265528642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4e4197980 0x7fc4e419bd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:54:39.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.762+0000 7fc4e9d8b640 1 -- 172.21.15.196:0/3265528642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4cc003530 msgr2=0x7fc4e4197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T08:54:39.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.762+0000 7fc4e9d8b640 1 -- 172.21.15.196:0/3265528642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4cc003530 msgr2=0x7fc4e4197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T08:54:39.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.762+0000 7fc4e9d8b640 1 --2- 172.21.15.196:0/3265528642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4cc003530 0x7fc4e4197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T08:54:39.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.762+0000 7fc4e9d8b640 1 --2- 172.21.15.196:0/3265528642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4cc003530 0x7fc4e4197440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:54:39.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.762+0000 7fc4ea58c640 1 -- 172.21.15.196:0/3265528642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4e4197980 msgr2=0x7fc4e419bd10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:54:39.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.762+0000 7fc4ea58c640 1 --2- 172.21.15.196:0/3265528642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4e4197980 0x7fc4e419bd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:39.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.762+0000 7fc4ea58c640 1 -- 172.21.15.196:0/3265528642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4cc003530 msgr2=0x7fc4e4197440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:54:39.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.762+0000 7fc4ea58c640 1 --2- 172.21.15.196:0/3265528642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4cc003530 0x7fc4e4197440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:39.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.762+0000 7fc4ea58c640 1 -- 172.21.15.196:0/3265528642 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e41a3fc0 con 0x7fc4e419c250 2024-09-19T08:54:39.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.762+0000 7fc4ea58c640 1 --2- 172.21.15.196:0/3265528642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4e419c250 0x7fc4e419c700 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400bd50 tx=0x7fc4d400be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:39.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.763+0000 7fc4e2ffd640 1 -- 172.21.15.196:0/3265528642 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4019070 con 0x7fc4e419c250 2024-09-19T08:54:39.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.763+0000 7fc4ebfef640 1 -- 172.21.15.196:0/3265528642 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e41a41f0 con 0x7fc4e419c250 2024-09-19T08:54:39.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.763+0000 7fc4e2ffd640 1 -- 172.21.15.196:0/3265528642 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4d4014920 con 0x7fc4e419c250 2024-09-19T08:54:39.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.763+0000 7fc4e2ffd640 1 -- 172.21.15.196:0/3265528642 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4012990 con 0x7fc4e419c250 2024-09-19T08:54:39.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.763+0000 7fc4ebfef640 1 -- 172.21.15.196:0/3265528642 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e41a4700 con 0x7fc4e419c250 2024-09-19T08:54:39.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.764+0000 7fc4ebfef640 1 -- 172.21.15.196:0/3265528642 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4e410a900 con 0x7fc4e419c250 2024-09-19T08:54:39.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.764+0000 7fc4e2ffd640 1 -- 172.21.15.196:0/3265528642 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fc4d401c070 con 0x7fc4e419c250 2024-09-19T08:54:39.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.764+0000 7fc4e2ffd640 1 --2- 172.21.15.196:0/3265528642 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc4d0039730 0x7fc4d003bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:39.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.764+0000 7fc4e2ffd640 1 -- 172.21.15.196:0/3265528642 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc4d404c160 con 0x7fc4e419c250 2024-09-19T08:54:39.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.764+0000 7fc4e9d8b640 1 --2- 172.21.15.196:0/3265528642 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc4d0039730 0x7fc4d003bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:39.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.765+0000 7fc4e9d8b640 1 --2- 172.21.15.196:0/3265528642 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc4d0039730 0x7fc4d003bbf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc000c00 tx=0x7fc4dc0325a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:39.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:39.769+0000 7fc4e2ffd640 1 -- 172.21.15.196:0/3265528642 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc4d4013e80 con 0x7fc4e419c250 2024-09-19T08:54:40.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:40.005+0000 7fc4ebfef640 1 -- 172.21.15.196:0/3265528642 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc4e41a4aa0 con 0x7fc4e419c250 2024-09-19T08:54:40.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:40.006+0000 7fc4e2ffd640 1 -- 172.21.15.196:0/3265528642 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7fc4d4013960 con 0x7fc4e419c250 2024-09-19T08:54:40.007 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T08:54:40.007 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":3,"fsid":"507393d2-7664-11ef-bceb-c7b262605968","modified":"2024-09-19T08:54:33.996914Z","created":"2024-09-19T08:53:01.874293Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:3300","nonce":0},{"type":"v1","addr":"172.21.15.120:6789","nonce":0}]},"addr":"172.21.15.120:6789/0","public_addr":"172.21.15.120:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T08:54:40.007 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 3 2024-09-19T08:54:40.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:40.008+0000 7fc4ebfef640 1 -- 172.21.15.196:0/3265528642 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc4d0039730 msgr2=0x7fc4d003bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:40.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:40.008+0000 7fc4ebfef640 1 --2- 172.21.15.196:0/3265528642 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc4d0039730 0x7fc4d003bbf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc000c00 tx=0x7fc4dc0325a0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:40.008+0000 7fc4ebfef640 1 -- 172.21.15.196:0/3265528642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4e419c250 msgr2=0x7fc4e419c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:40.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:40.008+0000 7fc4ebfef640 1 --2- 172.21.15.196:0/3265528642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4e419c250 0x7fc4e419c700 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400bd50 tx=0x7fc4d400be40 comp rx=0 tx=0).stop 2024-09-19T08:54:40.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:40.008+0000 7fc4ebfef640 1 -- 172.21.15.196:0/3265528642 shutdown_connections 2024-09-19T08:54:40.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:40.008+0000 7fc4ebfef640 1 --2- 172.21.15.196:0/3265528642 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc4d0039730 0x7fc4d003bbf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:40.008+0000 7fc4ebfef640 1 --2- 172.21.15.196:0/3265528642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4e419c250 0x7fc4e419c700 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:40.008+0000 7fc4ebfef640 1 --2- 172.21.15.196:0/3265528642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4e4197980 0x7fc4e419bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:40.008+0000 7fc4ebfef640 1 --2- 172.21.15.196:0/3265528642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4cc003530 0x7fc4e4197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:40.008+0000 7fc4ebfef640 1 -- 172.21.15.196:0/3265528642 >> 172.21.15.196:0/3265528642 conn(0x7fc4e40fc280 msgr2=0x7fc4e40fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:40.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:40.008+0000 7fc4ebfef640 1 -- 172.21.15.196:0/3265528642 shutdown_connections 2024-09-19T08:54:40.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:40.008+0000 7fc4ebfef640 1 -- 172.21.15.196:0/3265528642 wait complete. 2024-09-19T08:54:40.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:40 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:54:40.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:40 smithi120 ceph-mon[31150]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:40.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:40 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:54:40.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:40 smithi120 ceph-mon[31150]: from='client.? 172.21.15.196:0/3265528642' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:54:40.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:40 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:54:40.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:40 smithi196 ceph-mon[35150]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:40.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:40 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:54:40.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:40 smithi196 ceph-mon[35150]: from='client.? 172.21.15.196:0/3265528642' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:54:40.804 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-19T08:54:40.804 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-19T08:54:41.052 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:54:41.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:54:41.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: mon.c calling monitor election 2024-09-19T08:54:41.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: mon.a calling monitor election 2024-09-19T08:54:41.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:54:41.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:54:41.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:54:41.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:41.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:54:41.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:54:41.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:41.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:54:41.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:54:41.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T08:54:41.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: monmap epoch 3 2024-09-19T08:54:41.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: fsid 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T08:54:41.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: last_changed 2024-09-19T08:54:33.996914+0000 2024-09-19T08:54:41.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: created 2024-09-19T08:53:01.874293+0000 2024-09-19T08:54:41.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: min_mon_release 19 (squid) 2024-09-19T08:54:41.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: election_strategy: 1 2024-09-19T08:54:41.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: 0: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.a 2024-09-19T08:54:41.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-19T08:54:41.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: 2: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.b 2024-09-19T08:54:41.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: fsmap 2024-09-19T08:54:41.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T08:54:41.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: mgrmap e13: x(active, since 47s) 2024-09-19T08:54:41.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: overall HEALTH_OK 2024-09-19T08:54:41.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:41.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:41.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:41.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:41.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:54:41.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:41.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:54:41.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: from='client.? 172.21.15.196:0/3265528642' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:54:42.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:41 smithi120 ceph-mon[31150]: mon.b calling monitor election 2024-09-19T08:54:42.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:41 smithi120 ceph-mon[31150]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T08:54:42.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:41 smithi120 ceph-mon[31150]: Reconfiguring daemon mon.a on smithi120 2024-09-19T08:54:42.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:41 smithi120 ceph-mon[31150]: mon.b calling monitor election 2024-09-19T08:54:42.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:41 smithi120 ceph-mon[31150]: mon.c calling monitor election 2024-09-19T08:54:42.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:41 smithi120 ceph-mon[31150]: mon.a calling monitor election 2024-09-19T08:54:42.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:41 smithi120 ceph-mon[31150]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T08:54:42.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:41 smithi120 ceph-mon[31150]: monmap epoch 3 2024-09-19T08:54:42.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:41 smithi120 ceph-mon[31150]: fsid 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T08:54:42.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:41 smithi120 ceph-mon[31150]: last_changed 2024-09-19T08:54:33.996914+0000 2024-09-19T08:54:42.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:41 smithi120 ceph-mon[31150]: created 2024-09-19T08:53:01.874293+0000 2024-09-19T08:54:42.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:41 smithi120 ceph-mon[31150]: min_mon_release 19 (squid) 2024-09-19T08:54:42.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:41 smithi120 ceph-mon[31150]: election_strategy: 1 2024-09-19T08:54:42.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:41 smithi120 ceph-mon[31150]: 0: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.a 2024-09-19T08:54:42.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:41 smithi120 ceph-mon[31150]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-19T08:54:42.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:41 smithi120 ceph-mon[31150]: 2: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.b 2024-09-19T08:54:42.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:41 smithi120 ceph-mon[31150]: fsmap 2024-09-19T08:54:42.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:41 smithi120 ceph-mon[31150]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T08:54:42.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:41 smithi120 ceph-mon[31150]: mgrmap e13: x(active, since 49s) 2024-09-19T08:54:42.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:41 smithi120 ceph-mon[31150]: overall HEALTH_OK 2024-09-19T08:54:42.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:41 smithi120 ceph-mon[31150]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:42.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:41 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:42.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.080+0000 7f4987759640 1 -- 172.21.15.120:0/963958124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4980100990 msgr2=0x7f4980100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:42.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.080+0000 7f4987759640 1 --2- 172.21.15.120:0/963958124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4980100990 0x7f4980100d70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4970009c20 tx=0x7f497002aa90 comp rx=0 tx=0).stop 2024-09-19T08:54:42.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.080+0000 7f4987759640 1 -- 172.21.15.120:0/963958124 shutdown_connections 2024-09-19T08:54:42.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.080+0000 7f4987759640 1 --2- 172.21.15.120:0/963958124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4980100990 0x7f4980100d70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.080+0000 7f4987759640 1 -- 172.21.15.120:0/963958124 >> 172.21.15.120:0/963958124 conn(0x7f4980076250 msgr2=0x7f4980076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:42.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.080+0000 7f4987759640 1 -- 172.21.15.120:0/963958124 shutdown_connections 2024-09-19T08:54:42.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.080+0000 7f4987759640 1 -- 172.21.15.120:0/963958124 wait complete. 2024-09-19T08:54:42.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.081+0000 7f4987759640 1 Processor -- start 2024-09-19T08:54:42.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.081+0000 7f4987759640 1 -- start start 2024-09-19T08:54:42.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.081+0000 7f4987759640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4980100990 0x7f4980197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:42.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.081+0000 7f4987759640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4980197850 0x7f498019bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:42.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.081+0000 7f4987759640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f498019c120 0x7f498019c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:42.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.081+0000 7f4987759640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f498010ab40 con 0x7f498019c120 2024-09-19T08:54:42.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.081+0000 7f4987759640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f498010a9c0 con 0x7f4980100990 2024-09-19T08:54:42.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.081+0000 7f4987759640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f498010acc0 con 0x7f4980197850 2024-09-19T08:54:42.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.081+0000 7f4985cf6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f498019c120 0x7f498019c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:42.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.081+0000 7f4985cf6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f498019c120 0x7f498019c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56950/0 (socket says 172.21.15.120:56950) 2024-09-19T08:54:42.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.081+0000 7f4985cf6640 1 -- 172.21.15.120:0/668912324 learned_addr learned my addr 172.21.15.120:0/668912324 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:54:42.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.081+0000 7f49854f5640 1 --2- 172.21.15.120:0/668912324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4980100990 0x7f4980197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:42.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.081+0000 7f4984cf4640 1 --2- 172.21.15.120:0/668912324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4980197850 0x7f498019bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:42.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.081+0000 7f4985cf6640 1 -- 172.21.15.120:0/668912324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4980197850 msgr2=0x7f498019bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:42.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.081+0000 7f4985cf6640 1 --2- 172.21.15.120:0/668912324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4980197850 0x7f498019bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.081+0000 7f4985cf6640 1 -- 172.21.15.120:0/668912324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4980100990 msgr2=0x7f4980197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:42.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.081+0000 7f4985cf6640 1 --2- 172.21.15.120:0/668912324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4980100990 0x7f4980197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.082+0000 7f4985cf6640 1 -- 172.21.15.120:0/668912324 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49801a3e90 con 0x7f498019c120 2024-09-19T08:54:42.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.082+0000 7f4985cf6640 1 --2- 172.21.15.120:0/668912324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f498019c120 0x7f498019c5d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f497800b450 tx=0x7f497800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:42.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.082+0000 7f497e7fc640 1 -- 172.21.15.120:0/668912324 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4978015020 con 0x7f498019c120 2024-09-19T08:54:42.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.082+0000 7f497e7fc640 1 -- 172.21.15.120:0/668912324 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4978011d70 con 0x7f498019c120 2024-09-19T08:54:42.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.082+0000 7f497e7fc640 1 -- 172.21.15.120:0/668912324 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497800f340 con 0x7f498019c120 2024-09-19T08:54:42.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.082+0000 7f4987759640 1 -- 172.21.15.120:0/668912324 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49801a40c0 con 0x7f498019c120 2024-09-19T08:54:42.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.082+0000 7f4987759640 1 -- 172.21.15.120:0/668912324 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49801a4600 con 0x7f498019c120 2024-09-19T08:54:42.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.083+0000 7f497e7fc640 1 -- 172.21.15.120:0/668912324 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f4978018020 con 0x7f498019c120 2024-09-19T08:54:42.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.083+0000 7f4987759640 1 -- 172.21.15.120:0/668912324 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4980100df0 con 0x7f498019c120 2024-09-19T08:54:42.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.083+0000 7f497e7fc640 1 --2- 172.21.15.120:0/668912324 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4968039730 0x7f496803bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:42.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.083+0000 7f497e7fc640 1 -- 172.21.15.120:0/668912324 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f497804c1a0 con 0x7f498019c120 2024-09-19T08:54:42.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.083+0000 7f49854f5640 1 --2- 172.21.15.120:0/668912324 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4968039730 0x7f496803bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:42.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.083+0000 7f49854f5640 1 --2- 172.21.15.120:0/668912324 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4968039730 0x7f496803bbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4970005cc0 tx=0x7f497003a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:42.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.088+0000 7f497e7fc640 1 -- 172.21.15.120:0/668912324 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f497801f020 con 0x7f498019c120 2024-09-19T08:54:42.108 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: mon.b calling monitor election 2024-09-19T08:54:42.108 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T08:54:42.108 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: Reconfiguring daemon mon.a on smithi120 2024-09-19T08:54:42.108 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: mon.b calling monitor election 2024-09-19T08:54:42.108 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: mon.c calling monitor election 2024-09-19T08:54:42.108 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: mon.a calling monitor election 2024-09-19T08:54:42.108 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T08:54:42.109 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: monmap epoch 3 2024-09-19T08:54:42.109 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: fsid 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T08:54:42.109 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: last_changed 2024-09-19T08:54:33.996914+0000 2024-09-19T08:54:42.109 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: created 2024-09-19T08:53:01.874293+0000 2024-09-19T08:54:42.109 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: min_mon_release 19 (squid) 2024-09-19T08:54:42.109 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: election_strategy: 1 2024-09-19T08:54:42.109 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: 0: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.a 2024-09-19T08:54:42.109 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-19T08:54:42.109 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: 2: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.b 2024-09-19T08:54:42.109 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: fsmap 2024-09-19T08:54:42.109 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T08:54:42.109 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: mgrmap e13: x(active, since 49s) 2024-09-19T08:54:42.109 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: overall HEALTH_OK 2024-09-19T08:54:42.109 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:42.109 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:41 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:42.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:41 smithi196 ceph-mon[35150]: mon.b calling monitor election 2024-09-19T08:54:42.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:41 smithi196 ceph-mon[35150]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T08:54:42.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:41 smithi196 ceph-mon[35150]: Reconfiguring daemon mon.a on smithi120 2024-09-19T08:54:42.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:41 smithi196 ceph-mon[35150]: mon.b calling monitor election 2024-09-19T08:54:42.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:41 smithi196 ceph-mon[35150]: mon.c calling monitor election 2024-09-19T08:54:42.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:41 smithi196 ceph-mon[35150]: mon.a calling monitor election 2024-09-19T08:54:42.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:41 smithi196 ceph-mon[35150]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T08:54:42.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:41 smithi196 ceph-mon[35150]: monmap epoch 3 2024-09-19T08:54:42.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:41 smithi196 ceph-mon[35150]: fsid 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T08:54:42.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:41 smithi196 ceph-mon[35150]: last_changed 2024-09-19T08:54:33.996914+0000 2024-09-19T08:54:42.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:41 smithi196 ceph-mon[35150]: created 2024-09-19T08:53:01.874293+0000 2024-09-19T08:54:42.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:41 smithi196 ceph-mon[35150]: min_mon_release 19 (squid) 2024-09-19T08:54:42.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:41 smithi196 ceph-mon[35150]: election_strategy: 1 2024-09-19T08:54:42.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:41 smithi196 ceph-mon[35150]: 0: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.a 2024-09-19T08:54:42.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:41 smithi196 ceph-mon[35150]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-19T08:54:42.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:41 smithi196 ceph-mon[35150]: 2: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.b 2024-09-19T08:54:42.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:41 smithi196 ceph-mon[35150]: fsmap 2024-09-19T08:54:42.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:41 smithi196 ceph-mon[35150]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T08:54:42.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:41 smithi196 ceph-mon[35150]: mgrmap e13: x(active, since 49s) 2024-09-19T08:54:42.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:41 smithi196 ceph-mon[35150]: overall HEALTH_OK 2024-09-19T08:54:42.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:41 smithi196 ceph-mon[35150]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:42.188 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:41 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:42.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.260+0000 7f4987759640 1 -- 172.21.15.120:0/668912324 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f49801a4a30 con 0x7f498019c120 2024-09-19T08:54:42.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.261+0000 7f497e7fc640 1 -- 172.21.15.120:0/668912324 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f497800d040 con 0x7f498019c120 2024-09-19T08:54:42.261 INFO:teuthology.orchestra.run.smithi120.stdout:# minimal ceph.conf for 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T08:54:42.261 INFO:teuthology.orchestra.run.smithi120.stdout:[global] 2024-09-19T08:54:42.261 INFO:teuthology.orchestra.run.smithi120.stdout: fsid = 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T08:54:42.261 INFO:teuthology.orchestra.run.smithi120.stdout: mon_host = [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] 2024-09-19T08:54:42.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.263+0000 7f4987759640 1 -- 172.21.15.120:0/668912324 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4968039730 msgr2=0x7f496803bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:42.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.263+0000 7f4987759640 1 --2- 172.21.15.120:0/668912324 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4968039730 0x7f496803bbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4970005cc0 tx=0x7f497003a040 comp rx=0 tx=0).stop 2024-09-19T08:54:42.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.263+0000 7f4987759640 1 -- 172.21.15.120:0/668912324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f498019c120 msgr2=0x7f498019c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:42.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.263+0000 7f4987759640 1 --2- 172.21.15.120:0/668912324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f498019c120 0x7f498019c5d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f497800b450 tx=0x7f497800b910 comp rx=0 tx=0).stop 2024-09-19T08:54:42.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.263+0000 7f4987759640 1 -- 172.21.15.120:0/668912324 shutdown_connections 2024-09-19T08:54:42.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.263+0000 7f4987759640 1 --2- 172.21.15.120:0/668912324 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4968039730 0x7f496803bbf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.263+0000 7f4987759640 1 --2- 172.21.15.120:0/668912324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f498019c120 0x7f498019c5d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.263+0000 7f4987759640 1 --2- 172.21.15.120:0/668912324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4980197850 0x7f498019bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.263+0000 7f4987759640 1 --2- 172.21.15.120:0/668912324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4980100990 0x7f4980197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.263+0000 7f4987759640 1 -- 172.21.15.120:0/668912324 >> 172.21.15.120:0/668912324 conn(0x7f4980076250 msgr2=0x7f49800696c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:42.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.263+0000 7f4987759640 1 -- 172.21.15.120:0/668912324 shutdown_connections 2024-09-19T08:54:42.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:42.263+0000 7f4987759640 1 -- 172.21.15.120:0/668912324 wait complete. 2024-09-19T08:54:43.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:42 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:54:43.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:42 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/668912324' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:43.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:42 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:43.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:42 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:43.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:42 smithi120 ceph-mon[31150]: Reconfiguring mon.b (monmap changed)... 2024-09-19T08:54:43.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:42 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:54:43.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:42 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T08:54:43.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:42 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:43.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:42 smithi120 ceph-mon[31150]: Reconfiguring daemon mon.b on smithi153 2024-09-19T08:54:43.113 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:42 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:54:43.113 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:42 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/668912324' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:43.113 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:42 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:43.113 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:42 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:43.113 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:42 smithi153 ceph-mon[35071]: Reconfiguring mon.b (monmap changed)... 2024-09-19T08:54:43.113 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:42 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:54:43.113 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:42 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T08:54:43.113 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:42 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:43.114 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:42 smithi153 ceph-mon[35071]: Reconfiguring daemon mon.b on smithi153 2024-09-19T08:54:43.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:42 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:54:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:42 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/668912324' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:42 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:42 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:42 smithi196 ceph-mon[35150]: Reconfiguring mon.b (monmap changed)... 2024-09-19T08:54:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:42 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:54:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:42 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T08:54:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:42 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:42 smithi196 ceph-mon[35150]: Reconfiguring daemon mon.b on smithi153 2024-09-19T08:54:43.205 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-19T08:54:43.205 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-19T08:54:43.205 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T08:54:43.232 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-19T08:54:43.232 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:54:43.297 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-19T08:54:43.297 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T08:54:43.324 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-19T08:54:43.324 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:54:43.389 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-19T08:54:43.389 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T08:54:43.421 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-19T08:54:43.421 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:54:43.490 INFO:tasks.cephadm:Adding mgr.x on smithi120 2024-09-19T08:54:43.490 INFO:tasks.cephadm:Adding mgr.y on smithi153 2024-09-19T08:54:43.490 INFO:tasks.cephadm:Adding mgr.z on smithi196 2024-09-19T08:54:43.490 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi120=x;smithi153=y;smithi196=z' 2024-09-19T08:54:43.772 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:54:44.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:43 smithi196 ceph-mon[35150]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:44.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:43 smithi120 ceph-mon[31150]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:44.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:43 smithi153 ceph-mon[35071]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:44.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.205+0000 7fa00ca65640 1 -- 172.21.15.196:0/3278698576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0081005f0 msgr2=0x7fa0081009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:44.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.205+0000 7fa00ca65640 1 --2- 172.21.15.196:0/3278698576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0081005f0 0x7fa0081009f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8009880 tx=0x7f9ff802a5d0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.205+0000 7fa00ca65640 1 -- 172.21.15.196:0/3278698576 shutdown_connections 2024-09-19T08:54:44.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.205+0000 7fa00ca65640 1 --2- 172.21.15.196:0/3278698576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0081005f0 0x7fa0081009f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.205+0000 7fa00ca65640 1 -- 172.21.15.196:0/3278698576 >> 172.21.15.196:0/3278698576 conn(0x7fa0080fbda0 msgr2=0x7fa0080fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:44.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.205+0000 7fa00ca65640 1 -- 172.21.15.196:0/3278698576 shutdown_connections 2024-09-19T08:54:44.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.205+0000 7fa00ca65640 1 -- 172.21.15.196:0/3278698576 wait complete. 2024-09-19T08:54:44.209 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.206+0000 7fa00ca65640 1 Processor -- start 2024-09-19T08:54:44.209 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.206+0000 7fa00ca65640 1 -- start start 2024-09-19T08:54:44.209 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.206+0000 7fa00ca65640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0081005f0 0x7fa00806af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:44.209 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.207+0000 7fa00ca65640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa008199d30 0x7fa00819c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:44.209 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.207+0000 7fa00ca65640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa00819c660 0x7fa00819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:44.209 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.207+0000 7fa00ca65640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa0081066e0 con 0x7fa00819c660 2024-09-19T08:54:44.210 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.207+0000 7fa00ca65640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa008106560 con 0x7fa0081005f0 2024-09-19T08:54:44.210 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.207+0000 7fa00ca65640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa008106860 con 0x7fa008199d30 2024-09-19T08:54:44.210 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.207+0000 7fa005d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa008199d30 0x7fa00819c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:44.210 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.207+0000 7fa005d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa008199d30 0x7fa00819c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:37602/0 (socket says 172.21.15.196:37602) 2024-09-19T08:54:44.210 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.207+0000 7fa005d9b640 1 -- 172.21.15.196:0/1212891902 learned_addr learned my addr 172.21.15.196:0/1212891902 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-19T08:54:44.210 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.207+0000 7fa006d9d640 1 --2- 172.21.15.196:0/1212891902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa00819c660 0x7fa00819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:44.210 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.207+0000 7fa00659c640 1 --2- 172.21.15.196:0/1212891902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0081005f0 0x7fa00806af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:44.210 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.207+0000 7fa005d9b640 1 -- 172.21.15.196:0/1212891902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0081005f0 msgr2=0x7fa00806af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:44.210 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.207+0000 7fa005d9b640 1 --2- 172.21.15.196:0/1212891902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0081005f0 0x7fa00806af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.210 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.207+0000 7fa005d9b640 1 -- 172.21.15.196:0/1212891902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa00819c660 msgr2=0x7fa00819eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:44.210 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.207+0000 7fa005d9b640 1 --2- 172.21.15.196:0/1212891902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa00819c660 0x7fa00819eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.211 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.207+0000 7fa005d9b640 1 -- 172.21.15.196:0/1212891902 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa00819f430 con 0x7fa008199d30 2024-09-19T08:54:44.211 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.207+0000 7fa005d9b640 1 --2- 172.21.15.196:0/1212891902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa008199d30 0x7fa00819c120 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc00b520 tx=0x7f9ffc00b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:44.211 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.207+0000 7f9ff77fe640 1 -- 172.21.15.196:0/1212891902 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc004230 con 0x7fa008199d30 2024-09-19T08:54:44.211 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.207+0000 7fa00ca65640 1 -- 172.21.15.196:0/1212891902 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa00819f6c0 con 0x7fa008199d30 2024-09-19T08:54:44.211 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.207+0000 7f9ff77fe640 1 -- 172.21.15.196:0/1212891902 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ffc0043d0 con 0x7fa008199d30 2024-09-19T08:54:44.211 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.207+0000 7f9ff77fe640 1 -- 172.21.15.196:0/1212891902 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc003d60 con 0x7fa008199d30 2024-09-19T08:54:44.211 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.207+0000 7fa00ca65640 1 -- 172.21.15.196:0/1212891902 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa00819fc80 con 0x7fa008199d30 2024-09-19T08:54:44.211 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.208+0000 7f9ff77fe640 1 -- 172.21.15.196:0/1212891902 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f9ffc00e070 con 0x7fa008199d30 2024-09-19T08:54:44.211 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.208+0000 7f9ff77fe640 1 --2- 172.21.15.196:0/1212891902 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9ff0039730 0x7f9ff003bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:44.211 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.209+0000 7f9ff77fe640 1 -- 172.21.15.196:0/1212891902 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9ffc04bea0 con 0x7fa008199d30 2024-09-19T08:54:44.211 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.209+0000 7fa00ca65640 1 -- 172.21.15.196:0/1212891902 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa00818f4a0 con 0x7fa008199d30 2024-09-19T08:54:44.211 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.209+0000 7fa00659c640 1 --2- 172.21.15.196:0/1212891902 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9ff0039730 0x7f9ff003bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:44.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.209+0000 7fa00659c640 1 --2- 172.21.15.196:0/1212891902 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9ff0039730 0x7f9ff003bbf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8000c00 tx=0x7f9ff8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:44.214 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.214+0000 7f9ff77fe640 1 -- 172.21.15.196:0/1212891902 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9ffc07b210 con 0x7fa008199d30 2024-09-19T08:54:44.400 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.399+0000 7fa00ca65640 1 -- 172.21.15.196:0/1212891902 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi120=x;smithi153=y;smithi196=z", "target": ["mon-mgr", ""]}) -- 0x7fa0080626c0 con 0x7f9ff0039730 2024-09-19T08:54:44.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.404+0000 7f9ff77fe640 1 -- 172.21.15.196:0/1212891902 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa0080626c0 con 0x7f9ff0039730 2024-09-19T08:54:44.405 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mgr update... 2024-09-19T08:54:44.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.406+0000 7fa00ca65640 1 -- 172.21.15.196:0/1212891902 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9ff0039730 msgr2=0x7f9ff003bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:44.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.406+0000 7fa00ca65640 1 --2- 172.21.15.196:0/1212891902 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9ff0039730 0x7f9ff003bbf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8000c00 tx=0x7f9ff8031400 comp rx=0 tx=0).stop 2024-09-19T08:54:44.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.407+0000 7fa00ca65640 1 -- 172.21.15.196:0/1212891902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa008199d30 msgr2=0x7fa00819c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:44.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.407+0000 7fa00ca65640 1 --2- 172.21.15.196:0/1212891902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa008199d30 0x7fa00819c120 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc00b520 tx=0x7f9ffc00b9e0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.407+0000 7fa00ca65640 1 -- 172.21.15.196:0/1212891902 shutdown_connections 2024-09-19T08:54:44.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.407+0000 7fa00ca65640 1 --2- 172.21.15.196:0/1212891902 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9ff0039730 0x7f9ff003bbf0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.407+0000 7fa00ca65640 1 --2- 172.21.15.196:0/1212891902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa00819c660 0x7fa00819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.407+0000 7fa00ca65640 1 --2- 172.21.15.196:0/1212891902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa008199d30 0x7fa00819c120 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.407+0000 7fa00ca65640 1 --2- 172.21.15.196:0/1212891902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0081005f0 0x7fa00806af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.407+0000 7fa00ca65640 1 -- 172.21.15.196:0/1212891902 >> 172.21.15.196:0/1212891902 conn(0x7fa0080fbda0 msgr2=0x7fa0080fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:44.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.407+0000 7fa00ca65640 1 -- 172.21.15.196:0/1212891902 shutdown_connections 2024-09-19T08:54:44.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:54:44.407+0000 7fa00ca65640 1 -- 172.21.15.196:0/1212891902 wait complete. 2024-09-19T08:54:45.003 DEBUG:teuthology.orchestra.run.smithi153:mgr.y> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mgr.y.service 2024-09-19T08:54:45.005 DEBUG:teuthology.orchestra.run.smithi196:mgr.z> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mgr.z.service 2024-09-19T08:54:45.008 INFO:tasks.cephadm:Deploying OSDs... 2024-09-19T08:54:45.008 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-19T08:54:45.008 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T08:54:45.026 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T08:54:45.026 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_1 2024-09-19T08:54:45.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:44 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:45.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:44 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:45.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:44 smithi120 ceph-mon[31150]: Reconfiguring mon.c (monmap changed)... 2024-09-19T08:54:45.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:44 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:54:45.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:44 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T08:54:45.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:44 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:45.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:44 smithi120 ceph-mon[31150]: Reconfiguring daemon mon.c on smithi196 2024-09-19T08:54:45.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:44 smithi120 ceph-mon[31150]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi120=x;smithi153=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:54:45.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:44 smithi120 ceph-mon[31150]: Saving service mgr spec with placement smithi120=x;smithi153=y;smithi196=z;count:3 2024-09-19T08:54:45.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:44 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:45.082 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T08:54:45.083 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:54:45.083 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-19T08:54:45.083 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:54:45.083 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:54:45.083 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-09-19 08:54:02.518120467 +0000 2024-09-19T08:54:45.083 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-09-19 08:48:40.575834829 +0000 2024-09-19T08:54:45.083 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-09-19 08:48:40.575834829 +0000 2024-09-19T08:54:45.083 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-09-19 08:48:40.575834829 +0000 2024-09-19T08:54:45.083 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T08:54:45.144 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:44 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:45.144 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:44 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:45.144 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:44 smithi196 ceph-mon[35150]: Reconfiguring mon.c (monmap changed)... 2024-09-19T08:54:45.144 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:44 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:54:45.145 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:44 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T08:54:45.145 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:44 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:45.145 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:44 smithi196 ceph-mon[35150]: Reconfiguring daemon mon.c on smithi196 2024-09-19T08:54:45.145 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:44 smithi196 ceph-mon[35150]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi120=x;smithi153=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:54:45.145 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:44 smithi196 ceph-mon[35150]: Saving service mgr spec with placement smithi120=x;smithi153=y;smithi196=z;count:3 2024-09-19T08:54:45.145 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:44 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:45.149 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-09-19T08:54:45.149 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-09-19T08:54:45.149 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000201899 s, 2.5 MB/s 2024-09-19T08:54:45.150 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T08:54:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:44 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:44 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:44 smithi153 ceph-mon[35071]: Reconfiguring mon.c (monmap changed)... 2024-09-19T08:54:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:44 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:54:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:44 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T08:54:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:44 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:44 smithi153 ceph-mon[35071]: Reconfiguring daemon mon.c on smithi196 2024-09-19T08:54:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:44 smithi153 ceph-mon[35071]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi120=x;smithi153=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:54:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:44 smithi153 ceph-mon[35071]: Saving service mgr spec with placement smithi120=x;smithi153=y;smithi196=z;count:3 2024-09-19T08:54:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:44 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:45.207 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_2 2024-09-19T08:54:45.262 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T08:54:45.262 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:54:45.263 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-19T08:54:45.263 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:54:45.263 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:54:45.263 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-09-19 08:54:02.552119664 +0000 2024-09-19T08:54:45.263 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-09-19 08:48:40.903827099 +0000 2024-09-19T08:54:45.263 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-09-19 08:48:40.903827099 +0000 2024-09-19T08:54:45.263 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-09-19 08:48:40.903827099 +0000 2024-09-19T08:54:45.263 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T08:54:45.326 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-09-19T08:54:45.326 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-09-19T08:54:45.326 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000222249 s, 2.3 MB/s 2024-09-19T08:54:45.327 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T08:54:45.383 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_3 2024-09-19T08:54:45.438 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T08:54:45.438 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:54:45.438 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-19T08:54:45.439 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:54:45.439 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:54:45.439 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-09-19 08:54:02.586118860 +0000 2024-09-19T08:54:45.439 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-09-19 08:48:41.253818849 +0000 2024-09-19T08:54:45.439 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-09-19 08:48:41.253818849 +0000 2024-09-19T08:54:45.439 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-09-19 08:48:41.253818849 +0000 2024-09-19T08:54:45.439 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T08:54:45.501 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-09-19T08:54:45.502 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-09-19T08:54:45.502 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000198832 s, 2.6 MB/s 2024-09-19T08:54:45.503 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T08:54:45.559 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_4 2024-09-19T08:54:45.614 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T08:54:45.614 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:54:45.614 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-19T08:54:45.614 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:54:45.614 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:54:45.614 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-09-19 08:54:02.620118057 +0000 2024-09-19T08:54:45.614 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-09-19 08:48:41.588810953 +0000 2024-09-19T08:54:45.614 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-09-19 08:48:41.588810953 +0000 2024-09-19T08:54:45.615 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-09-19 08:48:41.588810953 +0000 2024-09-19T08:54:45.615 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T08:54:45.677 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-09-19T08:54:45.677 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-09-19T08:54:45.677 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000221314 s, 2.3 MB/s 2024-09-19T08:54:45.679 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T08:54:45.735 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-19T08:54:45.735 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T08:54:45.755 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T08:54:45.756 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_1 2024-09-19T08:54:45.814 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T08:54:45.814 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:54:45.814 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-19T08:54:45.814 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:54:45.814 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:54:45.814 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-09-19 08:54:33.444807641 +0000 2024-09-19T08:54:45.814 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-09-19 08:49:07.863369858 +0000 2024-09-19T08:54:45.814 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-09-19 08:49:07.863369858 +0000 2024-09-19T08:54:45.815 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-09-19 08:49:07.863369858 +0000 2024-09-19T08:54:45.815 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T08:54:45.881 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-09-19T08:54:45.881 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-09-19T08:54:45.881 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000282659 s, 1.8 MB/s 2024-09-19T08:54:45.882 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T08:54:45.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:45 smithi196 ceph-mon[35150]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:45.940 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_2 2024-09-19T08:54:45.997 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T08:54:45.997 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:54:45.998 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-09-19T08:54:45.998 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:54:45.998 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:54:45.998 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-09-19 08:54:33.444807641 +0000 2024-09-19T08:54:45.998 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-09-19 08:49:08.219363362 +0000 2024-09-19T08:54:45.998 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-09-19 08:49:08.219363362 +0000 2024-09-19T08:54:45.998 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-09-19 08:49:08.219363362 +0000 2024-09-19T08:54:45.998 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T08:54:46.060 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:45 smithi153 ceph-mon[35071]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:46.063 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-09-19T08:54:46.063 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-09-19T08:54:46.063 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000198977 s, 2.6 MB/s 2024-09-19T08:54:46.064 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T08:54:46.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:45 smithi120 ceph-mon[31150]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:46.122 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_3 2024-09-19T08:54:46.179 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T08:54:46.179 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:54:46.179 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-09-19T08:54:46.179 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:54:46.179 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:54:46.179 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-09-19 08:54:33.445807623 +0000 2024-09-19T08:54:46.179 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-09-19 08:49:08.552357287 +0000 2024-09-19T08:54:46.179 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-09-19 08:49:08.552357287 +0000 2024-09-19T08:54:46.179 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-09-19 08:49:08.552357287 +0000 2024-09-19T08:54:46.180 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T08:54:46.244 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-09-19T08:54:46.245 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-09-19T08:54:46.245 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000208322 s, 2.5 MB/s 2024-09-19T08:54:46.246 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T08:54:46.303 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_4 2024-09-19T08:54:46.359 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T08:54:46.359 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:54:46.359 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-19T08:54:46.359 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:54:46.359 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:54:46.359 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-09-19 08:54:33.445807623 +0000 2024-09-19T08:54:46.359 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-09-19 08:49:08.879351321 +0000 2024-09-19T08:54:46.360 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-09-19 08:49:08.879351321 +0000 2024-09-19T08:54:46.360 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-09-19 08:49:08.879351321 +0000 2024-09-19T08:54:46.360 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T08:54:46.423 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-09-19T08:54:46.424 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-09-19T08:54:46.424 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000222546 s, 2.3 MB/s 2024-09-19T08:54:46.425 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T08:54:46.482 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-19T08:54:46.482 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T08:54:46.498 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T08:54:46.498 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_1 2024-09-19T08:54:46.554 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T08:54:46.554 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:54:46.554 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-19T08:54:46.554 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:54:46.554 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:54:46.554 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-19 08:54:24.094296930 +0000 2024-09-19T08:54:46.555 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-19 08:49:04.346083827 +0000 2024-09-19T08:54:46.555 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-19 08:49:04.346083827 +0000 2024-09-19T08:54:46.555 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-19 08:49:04.346083827 +0000 2024-09-19T08:54:46.555 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T08:54:46.619 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-19T08:54:46.619 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-19T08:54:46.619 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000345306 s, 1.5 MB/s 2024-09-19T08:54:46.620 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T08:54:46.676 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_2 2024-09-19T08:54:46.731 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T08:54:46.731 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:54:46.731 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-19T08:54:46.732 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:54:46.732 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:54:46.732 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-19 08:54:24.095296908 +0000 2024-09-19T08:54:46.732 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-19 08:49:04.673076629 +0000 2024-09-19T08:54:46.732 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-19 08:49:04.673076629 +0000 2024-09-19T08:54:46.732 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-19 08:49:04.673076629 +0000 2024-09-19T08:54:46.732 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T08:54:46.794 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-19T08:54:46.795 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-19T08:54:46.795 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000187522 s, 2.7 MB/s 2024-09-19T08:54:46.796 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T08:54:46.852 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_3 2024-09-19T08:54:46.908 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T08:54:46.908 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:54:46.908 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-19T08:54:46.908 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:54:46.908 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:54:46.908 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-19 08:54:24.095296908 +0000 2024-09-19T08:54:46.908 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-19 08:49:05.008069256 +0000 2024-09-19T08:54:46.908 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-19 08:49:05.008069256 +0000 2024-09-19T08:54:46.909 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-19 08:49:05.008069256 +0000 2024-09-19T08:54:46.909 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T08:54:46.971 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-19T08:54:46.971 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-19T08:54:46.971 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000184522 s, 2.8 MB/s 2024-09-19T08:54:46.972 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T08:54:47.028 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_4 2024-09-19T08:54:47.083 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T08:54:47.084 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:54:47.084 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-19T08:54:47.084 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:54:47.084 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:54:47.084 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-19 08:54:24.095296908 +0000 2024-09-19T08:54:47.084 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-19 08:49:05.338061993 +0000 2024-09-19T08:54:47.084 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-19 08:49:05.338061993 +0000 2024-09-19T08:54:47.084 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-19 08:49:05.338061993 +0000 2024-09-19T08:54:47.084 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T08:54:47.144 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:46 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:47.144 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:46 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:47.144 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:46 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:54:47.144 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:46 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:47.145 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:46 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:54:47.145 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:46 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:47.145 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:46 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T08:54:47.145 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:46 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T08:54:47.145 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:46 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T08:54:47.145 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:46 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:47.145 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:46 smithi196 ceph-mon[35150]: Deploying daemon mgr.z on smithi196 2024-09-19T08:54:47.146 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-19T08:54:47.146 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-19T08:54:47.146 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000172086 s, 3.0 MB/s 2024-09-19T08:54:47.147 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T08:54:47.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:46 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:47.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:46 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:47.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:46 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:54:47.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:46 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:47.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:46 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:54:47.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:46 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:47.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:46 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T08:54:47.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:46 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T08:54:47.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:46 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T08:54:47.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:46 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:47.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:46 smithi153 ceph-mon[35071]: Deploying daemon mgr.z on smithi196 2024-09-19T08:54:47.204 INFO:tasks.cephadm:Deploying osd.0 on smithi120 with /dev/vg_nvme/lv_4... 2024-09-19T08:54:47.204 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T08:54:47.230 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:46 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:47.231 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:46 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:47.231 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:46 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:54:47.231 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:46 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:47.231 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:46 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:54:47.231 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:46 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:47.231 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:46 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T08:54:47.231 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:46 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T08:54:47.231 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:46 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T08:54:47.231 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:46 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:47.231 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:46 smithi120 ceph-mon[31150]: Deploying daemon mgr.z on smithi196 2024-09-19T08:54:47.455 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:54:47.840 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:47 smithi196 systemd[1]: Starting Ceph mgr.z for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T08:54:48.117 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:47 smithi196 ceph-mon[35150]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:48.117 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:47 smithi196 podman[36945]: 2024-09-19 08:54:47.809887523 +0000 UTC m=+0.015473466 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:54:48.117 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:47 smithi196 podman[36945]: 2024-09-19 08:54:47.923867363 +0000 UTC m=+0.129453338 container create 8eab4dfbe448eac4b33d93a7b2ac5ef5d6db3b49f420396293994b119e132e50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-19T08:54:48.117 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:48 smithi196 podman[36945]: 2024-09-19 08:54:48.117945126 +0000 UTC m=+0.323531062 container init 8eab4dfbe448eac4b33d93a7b2ac5ef5d6db3b49f420396293994b119e132e50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-09-19T08:54:48.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:47 smithi153 ceph-mon[35071]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:48.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:47 smithi120 ceph-mon[31150]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:48.435 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:48 smithi196 podman[36945]: 2024-09-19 08:54:48.121078016 +0000 UTC m=+0.326663957 container start 8eab4dfbe448eac4b33d93a7b2ac5ef5d6db3b49f420396293994b119e132e50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-19T08:54:48.436 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:48 smithi196 ceph-mgr[36964]: -- 172.21.15.196:0/554464203 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b59cc0b0e0 con 0x55b59cbe9000 2024-09-19T08:54:48.436 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:48 smithi196 bash[36945]: 8eab4dfbe448eac4b33d93a7b2ac5ef5d6db3b49f420396293994b119e132e50 2024-09-19T08:54:48.436 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:48 smithi196 systemd[1]: Started Ceph mgr.z for 507393d2-7664-11ef-bceb-c7b262605968. 2024-09-19T08:54:48.436 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:48 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z[36960]: 2024-09-19T08:54:48.240+0000 7f9407185040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T08:54:48.436 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:48 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z[36960]: 2024-09-19T08:54:48.300+0000 7f9407185040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T08:54:48.802 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:48 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z[36960]: 2024-09-19T08:54:48.449+0000 7f9407185040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T08:54:48.803 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:48 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z[36960]: 2024-09-19T08:54:48.514+0000 7f9407185040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T08:54:48.803 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:48 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z[36960]: 2024-09-19T08:54:48.662+0000 7f9407185040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:54:48.803 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:48 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z[36960]: 2024-09-19T08:54:48.802+0000 7f9407185040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T08:54:49.349 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:49 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:49.349 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:49 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:49.349 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:49 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:49.349 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:49 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T08:54:49.349 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:49 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T08:54:49.349 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:49 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T08:54:49.349 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:49 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:49.349 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:49 smithi120 ceph-mon[31150]: Deploying daemon mgr.y on smithi153 2024-09-19T08:54:49.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:49 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:49.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:49 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:49.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:49 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:49.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:49 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T08:54:49.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:49 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T08:54:49.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:49 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T08:54:49.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:49 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:49.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:49 smithi196 ceph-mon[35150]: Deploying daemon mgr.y on smithi153 2024-09-19T08:54:49.436 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:49 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z[36960]: 2024-09-19T08:54:49.116+0000 7f9407185040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T08:54:49.436 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:49 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z[36960]: 2024-09-19T08:54:49.177+0000 7f9407185040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T08:54:49.436 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:49 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z[36960]: 2024-09-19T08:54:49.364+0000 7f9407185040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T08:54:49.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:49 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:49.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:49 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:49.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:49 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:49.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:49 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T08:54:49.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:49 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T08:54:49.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:49 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T08:54:49.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:49 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:49.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:49 smithi153 ceph-mon[35071]: Deploying daemon mgr.y on smithi153 2024-09-19T08:54:49.944 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:54:49.970 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi120:vg_nvme/lv_4 2024-09-19T08:54:50.187 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:50 smithi153 systemd[1]: Starting Ceph mgr.y for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T08:54:50.251 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:54:50.501 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:50 smithi153 ceph-mon[35071]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:50.501 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:50 smithi153 podman[36421]: 2024-09-19 08:54:50.127744939 +0000 UTC m=+0.015572817 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:54:50.502 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:50 smithi153 podman[36421]: 2024-09-19 08:54:50.233266446 +0000 UTC m=+0.121094358 container create ae5d6044121a1fd1314d1d25c9d43dc09ba889969a08365181e9a55d18e1a6d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-19T08:54:50.502 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:50 smithi153 podman[36421]: 2024-09-19 08:54:50.435738145 +0000 UTC m=+0.323566018 container init ae5d6044121a1fd1314d1d25c9d43dc09ba889969a08365181e9a55d18e1a6d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-19T08:54:50.502 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:50 smithi153 podman[36421]: 2024-09-19 08:54:50.438432209 +0000 UTC m=+0.326260087 container start ae5d6044121a1fd1314d1d25c9d43dc09ba889969a08365181e9a55d18e1a6d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True) 2024-09-19T08:54:50.502 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:50 smithi153 ceph-mgr[36439]: -- 172.21.15.153:0/2551354206 <== mon.2 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562c60e090e0 con 0x562c60de7000 2024-09-19T08:54:50.502 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:50 smithi153 bash[36421]: ae5d6044121a1fd1314d1d25c9d43dc09ba889969a08365181e9a55d18e1a6d7 2024-09-19T08:54:50.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:50 smithi120 ceph-mon[31150]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:50.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:50 smithi196 ceph-mon[35150]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:50.685 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:50 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z[36960]: 2024-09-19T08:54:50.248+0000 7f9407185040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T08:54:50.685 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:50 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z[36960]: 2024-09-19T08:54:50.317+0000 7f9407185040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T08:54:50.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.683+0000 7ff2abbc0640 1 -- 172.21.15.120:0/4131503812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2a4101cd0 msgr2=0x7ff2a4102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:50.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.683+0000 7ff2abbc0640 1 --2- 172.21.15.120:0/4131503812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2a4101cd0 0x7ff2a4102150 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff294009900 tx=0x7ff29402a410 comp rx=0 tx=0).stop 2024-09-19T08:54:50.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.683+0000 7ff2abbc0640 1 -- 172.21.15.120:0/4131503812 shutdown_connections 2024-09-19T08:54:50.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.683+0000 7ff2abbc0640 1 --2- 172.21.15.120:0/4131503812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2a4102690 0x7ff2a4108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:50.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.683+0000 7ff2abbc0640 1 --2- 172.21.15.120:0/4131503812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2a4101cd0 0x7ff2a4102150 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:50.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.683+0000 7ff2abbc0640 1 --2- 172.21.15.120:0/4131503812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2a4100ad0 0x7ff2a4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:50.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.683+0000 7ff2abbc0640 1 -- 172.21.15.120:0/4131503812 >> 172.21.15.120:0/4131503812 conn(0x7ff2a40fc280 msgr2=0x7ff2a40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:50.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.683+0000 7ff2abbc0640 1 -- 172.21.15.120:0/4131503812 shutdown_connections 2024-09-19T08:54:50.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.683+0000 7ff2abbc0640 1 -- 172.21.15.120:0/4131503812 wait complete. 2024-09-19T08:54:50.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.683+0000 7ff2abbc0640 1 Processor -- start 2024-09-19T08:54:50.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.683+0000 7ff2abbc0640 1 -- start start 2024-09-19T08:54:50.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.683+0000 7ff2abbc0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2a4100ad0 0x7ff2a4197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:50.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.683+0000 7ff2abbc0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2a4101cd0 0x7ff2a4198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:50.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.683+0000 7ff2abbc0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2a4102690 0x7ff2a419f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:50.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.683+0000 7ff2abbc0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a410c730 con 0x7ff2a4101cd0 2024-09-19T08:54:50.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.683+0000 7ff2abbc0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a410c5b0 con 0x7ff2a4100ad0 2024-09-19T08:54:50.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.683+0000 7ff2abbc0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a410c8b0 con 0x7ff2a4102690 2024-09-19T08:54:50.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.683+0000 7ff2a915b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2a4101cd0 0x7ff2a4198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:50.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.684+0000 7ff2a915b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2a4101cd0 0x7ff2a4198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56970/0 (socket says 172.21.15.120:56970) 2024-09-19T08:54:50.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.684+0000 7ff2a915b640 1 -- 172.21.15.120:0/4233603836 learned_addr learned my addr 172.21.15.120:0/4233603836 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:54:50.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.684+0000 7ff2a995c640 1 --2- 172.21.15.120:0/4233603836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2a4100ad0 0x7ff2a4197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:50.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.684+0000 7ff2aa15d640 1 --2- 172.21.15.120:0/4233603836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2a4102690 0x7ff2a419f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:50.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.684+0000 7ff2a915b640 1 -- 172.21.15.120:0/4233603836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2a4102690 msgr2=0x7ff2a419f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:50.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.684+0000 7ff2a915b640 1 --2- 172.21.15.120:0/4233603836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2a4102690 0x7ff2a419f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:50.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.684+0000 7ff2a915b640 1 -- 172.21.15.120:0/4233603836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2a4100ad0 msgr2=0x7ff2a4197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:50.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.684+0000 7ff2a915b640 1 --2- 172.21.15.120:0/4233603836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2a4100ad0 0x7ff2a4197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:50.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.684+0000 7ff2a915b640 1 -- 172.21.15.120:0/4233603836 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a419fb10 con 0x7ff2a4101cd0 2024-09-19T08:54:50.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.684+0000 7ff2a995c640 1 --2- 172.21.15.120:0/4233603836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2a4100ad0 0x7ff2a4197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:54:50.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.684+0000 7ff2a915b640 1 --2- 172.21.15.120:0/4233603836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2a4101cd0 0x7ff2a4198420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff294009230 tx=0x7ff294031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:50.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.684+0000 7ff29affd640 1 -- 172.21.15.120:0/4233603836 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29403f030 con 0x7ff2a4101cd0 2024-09-19T08:54:50.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.684+0000 7ff29affd640 1 -- 172.21.15.120:0/4233603836 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff294003ad0 con 0x7ff2a4101cd0 2024-09-19T08:54:50.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.684+0000 7ff2abbc0640 1 -- 172.21.15.120:0/4233603836 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a419fda0 con 0x7ff2a4101cd0 2024-09-19T08:54:50.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.684+0000 7ff29affd640 1 -- 172.21.15.120:0/4233603836 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294035070 con 0x7ff2a4101cd0 2024-09-19T08:54:50.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.684+0000 7ff2abbc0640 1 -- 172.21.15.120:0/4233603836 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a41a0220 con 0x7ff2a4101cd0 2024-09-19T08:54:50.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.685+0000 7ff29affd640 1 -- 172.21.15.120:0/4233603836 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7ff294042030 con 0x7ff2a4101cd0 2024-09-19T08:54:50.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.685+0000 7ff29affd640 1 --2- 172.21.15.120:0/4233603836 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff290039500 0x7ff29003b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:50.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.685+0000 7ff29affd640 1 -- 172.21.15.120:0/4233603836 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff29406cd20 con 0x7ff2a4101cd0 2024-09-19T08:54:50.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.685+0000 7ff2a995c640 1 --2- 172.21.15.120:0/4233603836 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff290039500 0x7ff29003b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:50.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.686+0000 7ff2abbc0640 1 -- 172.21.15.120:0/4233603836 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2a41a0450 con 0x7ff2a4101cd0 2024-09-19T08:54:50.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.686+0000 7ff2a995c640 1 --2- 172.21.15.120:0/4233603836 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff290039500 0x7ff29003b9c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4101b30 tx=0x7ff2a0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:50.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.690+0000 7ff29affd640 1 -- 172.21.15.120:0/4233603836 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff294003cf0 con 0x7ff2a4101cd0 2024-09-19T08:54:50.779 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:50 smithi153 systemd[1]: Started Ceph mgr.y for 507393d2-7664-11ef-bceb-c7b262605968. 2024-09-19T08:54:50.779 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:50 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y[36435]: 2024-09-19T08:54:50.562+0000 7fce3e20d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T08:54:50.779 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:50 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y[36435]: 2024-09-19T08:54:50.622+0000 7fce3e20d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T08:54:50.779 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:50 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y[36435]: 2024-09-19T08:54:50.779+0000 7fce3e20d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T08:54:50.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:50.865+0000 7ff2abbc0640 1 -- 172.21.15.120:0/4233603836 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff2a406a680 con 0x7ff290039500 2024-09-19T08:54:51.157 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:50 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y[36435]: 2024-09-19T08:54:50.847+0000 7fce3e20d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T08:54:51.158 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:51 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y[36435]: 2024-09-19T08:54:51.005+0000 7fce3e20d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:54:51.158 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:51 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y[36435]: 2024-09-19T08:54:51.157+0000 7fce3e20d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T08:54:51.341 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:51 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z[36960]: 2024-09-19T08:54:51.076+0000 7f9407185040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T08:54:51.342 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:51 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z[36960]: 2024-09-19T08:54:51.195+0000 7f9407185040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T08:54:51.342 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:51 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z[36960]: 2024-09-19T08:54:51.341+0000 7f9407185040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:54:51.630 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:51 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z[36960]: 2024-09-19T08:54:51.407+0000 7f9407185040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T08:54:51.630 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:51 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z[36960]: 2024-09-19T08:54:51.561+0000 7f9407185040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T08:54:51.631 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:51 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z[36960]: 2024-09-19T08:54:51.630+0000 7f9407185040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T08:54:51.631 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:51 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:51.631 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:51 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:51.631 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:51 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:51.631 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:51 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:51.631 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:51 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:54:51.631 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:51 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:54:51.631 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:51 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:54:51.631 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:51 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:51.741 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:51 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y[36435]: 2024-09-19T08:54:51.486+0000 7fce3e20d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T08:54:51.742 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:51 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y[36435]: 2024-09-19T08:54:51.547+0000 7fce3e20d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T08:54:51.742 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:51 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:51.742 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:51 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:51.742 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:51 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:51.743 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:51 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:51.743 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:51 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:54:51.743 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:51 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:54:51.743 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:51 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:54:51.743 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:51 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:51.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:51.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:51.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:51.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:51.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:54:51.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:54:51.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:54:51.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:51.935 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:51 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z[36960]: 2024-09-19T08:54:51.692+0000 7f9407185040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T08:54:51.935 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:51 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z[36960]: 2024-09-19T08:54:51.869+0000 7f9407185040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T08:54:52.187 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:51 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y[36435]: 2024-09-19T08:54:51.741+0000 7fce3e20d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T08:54:52.200 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:51 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z[36960]: 2024-09-19T08:54:51.936+0000 7f9407185040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T08:54:52.200 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:52 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z[36960]: 2024-09-19T08:54:52.003+0000 7f9407185040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T08:54:52.200 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:52 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z[36960]: 2024-09-19T08:54:52.085+0000 7f9407185040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T08:54:52.201 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:52 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z[36960]: 2024-09-19T08:54:52.143+0000 7f9407185040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T08:54:52.201 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:52 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z[36960]: 2024-09-19T08:54:52.200+0000 7f9407185040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T08:54:52.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:52 smithi120 ceph-mon[31150]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:54:52.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:52 smithi120 ceph-mon[31150]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:52.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:52 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:52.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:52 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:52.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:52 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:52.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:52 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:52.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:52 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:52.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:52 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:52.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:52 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:54:52.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:52 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:52.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:52 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T08:54:52.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:52 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T08:54:52.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:52 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:52.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:52 smithi196 ceph-mon[35150]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:54:52.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:52 smithi196 ceph-mon[35150]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:52.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:52 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:52.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:52 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:52.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:52 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:52.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:52 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:52.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:52 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:52.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:52 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:52.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:52 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:54:52.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:52 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:52.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:52 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T08:54:52.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:52 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T08:54:52.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:52 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:52.940 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:52 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y[36435]: 2024-09-19T08:54:52.622+0000 7fce3e20d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T08:54:52.940 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:52 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y[36435]: 2024-09-19T08:54:52.689+0000 7fce3e20d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T08:54:52.941 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:52 smithi153 ceph-mon[35071]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:54:52.941 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:52 smithi153 ceph-mon[35071]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:52.941 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:52 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:52.941 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:52 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:52.941 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:52 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:52.941 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:52 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:52.941 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:52 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:52.941 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:52 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:52.941 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:52 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:54:52.941 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:52 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:54:52.941 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:52 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T08:54:52.942 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:52 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T08:54:52.942 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:52 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:53.435 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 08:54:52 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z[36960]: 2024-09-19T08:54:52.991+0000 7f9407185040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T08:54:53.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:53.582+0000 7ff29affd640 1 -- 172.21.15.120:0/4233603836 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mgrmap(e 14) ==== 92409+0+0 (secure 0 0 0) 0x7ff29403b460 con 0x7ff2a4101cd0 2024-09-19T08:54:53.659 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:53 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y[36435]: 2024-09-19T08:54:53.401+0000 7fce3e20d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T08:54:53.659 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:53 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y[36435]: 2024-09-19T08:54:53.516+0000 7fce3e20d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T08:54:53.659 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:53 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y[36435]: 2024-09-19T08:54:53.659+0000 7fce3e20d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:54:53.659 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:53 smithi153 ceph-mon[35071]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T08:54:53.660 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:53 smithi153 ceph-mon[35071]: Reconfiguring daemon mgr.x on smithi120 2024-09-19T08:54:53.660 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:53 smithi153 ceph-mon[35071]: Standby manager daemon z started 2024-09-19T08:54:53.660 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:53 smithi153 ceph-mon[35071]: from='mgr.? 172.21.15.196:0/375708282' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T08:54:53.660 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:53 smithi153 ceph-mon[35071]: from='mgr.? 172.21.15.196:0/375708282' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:54:53.660 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:53 smithi153 ceph-mon[35071]: from='mgr.? 172.21.15.196:0/375708282' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T08:54:53.660 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:53 smithi153 ceph-mon[35071]: from='mgr.? 172.21.15.196:0/375708282' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:54:53.660 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:53 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1783849192' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33ec3993-576c-428c-bf75-adcb17cd72a8"}]: dispatch 2024-09-19T08:54:53.660 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:53 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1783849192' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33ec3993-576c-428c-bf75-adcb17cd72a8"}]': finished 2024-09-19T08:54:53.660 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:53 smithi153 ceph-mon[35071]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T08:54:53.660 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:53 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:54:53.719 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:53 smithi120 ceph-mon[31150]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T08:54:53.720 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:53 smithi120 ceph-mon[31150]: Reconfiguring daemon mgr.x on smithi120 2024-09-19T08:54:53.720 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:53 smithi120 ceph-mon[31150]: Standby manager daemon z started 2024-09-19T08:54:53.720 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:53 smithi120 ceph-mon[31150]: from='mgr.? 172.21.15.196:0/375708282' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T08:54:53.720 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:53 smithi120 ceph-mon[31150]: from='mgr.? 172.21.15.196:0/375708282' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:54:53.720 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:53 smithi120 ceph-mon[31150]: from='mgr.? 172.21.15.196:0/375708282' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T08:54:53.720 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:53 smithi120 ceph-mon[31150]: from='mgr.? 172.21.15.196:0/375708282' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:54:53.720 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:53 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1783849192' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33ec3993-576c-428c-bf75-adcb17cd72a8"}]: dispatch 2024-09-19T08:54:53.720 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:53 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1783849192' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33ec3993-576c-428c-bf75-adcb17cd72a8"}]': finished 2024-09-19T08:54:53.720 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:53 smithi120 ceph-mon[31150]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T08:54:53.720 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:53 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:54:53.935 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:53 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y[36435]: 2024-09-19T08:54:53.720+0000 7fce3e20d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T08:54:53.935 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:53 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y[36435]: 2024-09-19T08:54:53.868+0000 7fce3e20d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T08:54:53.935 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:53 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y[36435]: 2024-09-19T08:54:53.935+0000 7fce3e20d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T08:54:53.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:53 smithi196 ceph-mon[35150]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T08:54:53.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:53 smithi196 ceph-mon[35150]: Reconfiguring daemon mgr.x on smithi120 2024-09-19T08:54:53.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:53 smithi196 ceph-mon[35150]: Standby manager daemon z started 2024-09-19T08:54:53.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:53 smithi196 ceph-mon[35150]: from='mgr.? 172.21.15.196:0/375708282' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T08:54:53.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:53 smithi196 ceph-mon[35150]: from='mgr.? 172.21.15.196:0/375708282' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:54:53.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:53 smithi196 ceph-mon[35150]: from='mgr.? 172.21.15.196:0/375708282' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T08:54:53.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:53 smithi196 ceph-mon[35150]: from='mgr.? 172.21.15.196:0/375708282' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:54:53.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:53 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1783849192' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33ec3993-576c-428c-bf75-adcb17cd72a8"}]: dispatch 2024-09-19T08:54:53.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:53 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1783849192' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33ec3993-576c-428c-bf75-adcb17cd72a8"}]': finished 2024-09-19T08:54:53.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:53 smithi196 ceph-mon[35150]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T08:54:53.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:53 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:54:54.187 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:53 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y[36435]: 2024-09-19T08:54:53.995+0000 7fce3e20d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T08:54:54.187 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:54 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y[36435]: 2024-09-19T08:54:54.151+0000 7fce3e20d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T08:54:54.480 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:54 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y[36435]: 2024-09-19T08:54:54.216+0000 7fce3e20d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T08:54:54.480 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:54 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y[36435]: 2024-09-19T08:54:54.281+0000 7fce3e20d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T08:54:54.481 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:54 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y[36435]: 2024-09-19T08:54:54.363+0000 7fce3e20d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T08:54:54.481 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:54 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y[36435]: 2024-09-19T08:54:54.422+0000 7fce3e20d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T08:54:54.481 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:54 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y[36435]: 2024-09-19T08:54:54.480+0000 7fce3e20d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T08:54:54.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:54 smithi196 ceph-mon[35150]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:54.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:54 smithi196 ceph-mon[35150]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-19T08:54:54.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:54 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T08:54:54.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:54 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3757308019' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:54:54.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:54 smithi153 ceph-mon[35071]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:54.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:54 smithi153 ceph-mon[35071]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-19T08:54:54.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:54 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T08:54:54.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:54 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3757308019' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:54:55.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:54 smithi120 ceph-mon[31150]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:55.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:54 smithi120 ceph-mon[31150]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-19T08:54:55.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:54 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T08:54:55.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:54 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3757308019' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:54:55.589 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 08:54:55 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y[36435]: 2024-09-19T08:54:55.264+0000 7fce3e20d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T08:54:55.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:54:55.596+0000 7ff29affd640 1 -- 172.21.15.120:0/4233603836 <== mon.0 v2:172.21.15.120:3300/0 8 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7ff29403bc70 con 0x7ff2a4101cd0 2024-09-19T08:54:55.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:55 smithi120 ceph-mon[31150]: Standby manager daemon y started 2024-09-19T08:54:55.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:55 smithi120 ceph-mon[31150]: from='mgr.? 172.21.15.153:0/1423494163' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T08:54:55.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:55 smithi120 ceph-mon[31150]: from='mgr.? 172.21.15.153:0/1423494163' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:54:55.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:55 smithi120 ceph-mon[31150]: from='mgr.? 172.21.15.153:0/1423494163' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T08:54:55.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:55 smithi120 ceph-mon[31150]: from='mgr.? 172.21.15.153:0/1423494163' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:54:55.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:55 smithi196 ceph-mon[35150]: Standby manager daemon y started 2024-09-19T08:54:55.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:55 smithi196 ceph-mon[35150]: from='mgr.? 172.21.15.153:0/1423494163' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T08:54:55.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:55 smithi196 ceph-mon[35150]: from='mgr.? 172.21.15.153:0/1423494163' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:54:55.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:55 smithi196 ceph-mon[35150]: from='mgr.? 172.21.15.153:0/1423494163' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T08:54:55.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:55 smithi196 ceph-mon[35150]: from='mgr.? 172.21.15.153:0/1423494163' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:54:55.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:55 smithi153 ceph-mon[35071]: Standby manager daemon y started 2024-09-19T08:54:55.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:55 smithi153 ceph-mon[35071]: from='mgr.? 172.21.15.153:0/1423494163' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T08:54:55.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:55 smithi153 ceph-mon[35071]: from='mgr.? 172.21.15.153:0/1423494163' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:54:55.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:55 smithi153 ceph-mon[35071]: from='mgr.? 172.21.15.153:0/1423494163' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T08:54:55.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:55 smithi153 ceph-mon[35071]: from='mgr.? 172.21.15.153:0/1423494163' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:54:56.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:56 smithi196 ceph-mon[35150]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:56.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:56 smithi196 ceph-mon[35150]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-19T08:54:56.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:56 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T08:54:56.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:56 smithi153 ceph-mon[35071]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:56.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:56 smithi153 ceph-mon[35071]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-19T08:54:56.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:56 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T08:54:57.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:56 smithi120 ceph-mon[31150]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:57.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:56 smithi120 ceph-mon[31150]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-19T08:54:57.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:56 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T08:54:58.877 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:54:58 smithi120 ceph-mon[31150]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:58.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:54:58 smithi196 ceph-mon[35150]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:54:58.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:54:58 smithi153 ceph-mon[35071]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:00.759 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:00 smithi120 ceph-mon[31150]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:00.759 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:00.759 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:00.759 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:55:00.759 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:00.759 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:55:00.759 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:00.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:00 smithi196 ceph-mon[35150]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:00.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:00.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:00.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:55:00.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:00.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:55:00.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:00.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:00 smithi153 ceph-mon[35071]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:00.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:00.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:00.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:55:00.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:00.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:55:00.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:02.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:02 smithi196 ceph-mon[35150]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:02.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:02 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T08:55:02.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:02 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:02.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:02 smithi153 ceph-mon[35071]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:02.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:02 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T08:55:02.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:02 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:02.944 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:02 smithi120 ceph-mon[31150]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:02.944 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:02 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T08:55:02.945 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:02 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:03.802 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:03 smithi120 ceph-mon[31150]: Deploying daemon osd.0 on smithi120 2024-09-19T08:55:03.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:03 smithi196 ceph-mon[35150]: Deploying daemon osd.0 on smithi120 2024-09-19T08:55:03.936 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:03 smithi153 ceph-mon[35071]: Deploying daemon osd.0 on smithi120 2024-09-19T08:55:04.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:04 smithi196 ceph-mon[35150]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:04.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:04 smithi153 ceph-mon[35071]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:05.024 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:04 smithi120 ceph-mon[31150]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:06.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:06 smithi196 ceph-mon[35150]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:06.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:06 smithi153 ceph-mon[35071]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:07.041 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:06 smithi120 ceph-mon[31150]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:08.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:08 smithi120 ceph-mon[31150]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:08.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:08 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:55:08.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:08 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:08.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:08 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:08.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:08 smithi196 ceph-mon[35150]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:08.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:08 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:55:08.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:08 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:08.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:08 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:08.936 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:08 smithi153 ceph-mon[35071]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:08.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:08 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:55:08.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:08 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:08.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:08 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:10.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:10 smithi120 ceph-mon[31150]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:10.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:10 smithi196 ceph-mon[35150]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:10.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:10 smithi153 ceph-mon[35071]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:12.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:11 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:12.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:11 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:12.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:11 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:12.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:11 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:55:12.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:11 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:12.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:11 smithi120 ceph-mon[31150]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:12.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:11 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:12.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:11 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:12.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:11 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:12.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:11 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:55:12.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:11 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:12.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:11 smithi196 ceph-mon[35150]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:12.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:11 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:12.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:11 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:12.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:11 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:12.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:11 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:55:12.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:11 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:12.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:11 smithi153 ceph-mon[35071]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:12.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:55:12.617+0000 7ff29affd640 1 -- 172.21.15.120:0/4233603836 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff2a406a680 con 0x7ff290039500 2024-09-19T08:55:12.617 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 0 on host 'smithi120' 2024-09-19T08:55:12.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:55:12.621+0000 7ff2abbc0640 1 -- 172.21.15.120:0/4233603836 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff290039500 msgr2=0x7ff29003b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:12.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:55:12.621+0000 7ff2abbc0640 1 --2- 172.21.15.120:0/4233603836 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff290039500 0x7ff29003b9c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4101b30 tx=0x7ff2a0005f90 comp rx=0 tx=0).stop 2024-09-19T08:55:12.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:55:12.621+0000 7ff2abbc0640 1 -- 172.21.15.120:0/4233603836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2a4101cd0 msgr2=0x7ff2a4198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:12.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:55:12.621+0000 7ff2abbc0640 1 --2- 172.21.15.120:0/4233603836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2a4101cd0 0x7ff2a4198420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff294009230 tx=0x7ff294031b60 comp rx=0 tx=0).stop 2024-09-19T08:55:12.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:55:12.621+0000 7ff2abbc0640 1 -- 172.21.15.120:0/4233603836 shutdown_connections 2024-09-19T08:55:12.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:55:12.621+0000 7ff2abbc0640 1 --2- 172.21.15.120:0/4233603836 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff290039500 0x7ff29003b9c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:12.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:55:12.621+0000 7ff2abbc0640 1 --2- 172.21.15.120:0/4233603836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2a4102690 0x7ff2a419f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:12.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:55:12.621+0000 7ff2abbc0640 1 --2- 172.21.15.120:0/4233603836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2a4101cd0 0x7ff2a4198420 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:12.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:55:12.621+0000 7ff2abbc0640 1 --2- 172.21.15.120:0/4233603836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2a4100ad0 0x7ff2a4197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:12.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:55:12.621+0000 7ff2abbc0640 1 -- 172.21.15.120:0/4233603836 >> 172.21.15.120:0/4233603836 conn(0x7ff2a40fc280 msgr2=0x7ff2a4069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:12.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:55:12.621+0000 7ff2abbc0640 1 -- 172.21.15.120:0/4233603836 shutdown_connections 2024-09-19T08:55:12.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:55:12.621+0000 7ff2abbc0640 1 -- 172.21.15.120:0/4233603836 wait complete. 2024-09-19T08:55:13.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:55:13.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:13.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:13.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:12 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:55:13.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:12 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:13.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:12 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:13.186 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:12 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:55:13.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:12 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:13.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:12 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:13.225 DEBUG:teuthology.orchestra.run.smithi120:osd.0> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.0.service 2024-09-19T08:55:13.228 INFO:tasks.cephadm:Deploying osd.1 on smithi153 with /dev/vg_nvme/lv_4... 2024-09-19T08:55:13.228 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T08:55:13.466 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:55:13.864 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:13 smithi153 ceph-mon[35071]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:14.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:13 smithi120 ceph-mon[31150]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:14.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:13 smithi196 ceph-mon[35150]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:14.824 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 19 08:55:14 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-0[45204]: 2024-09-19T08:55:14.558+0000 7fa46bbed640 -1 osd.0 0 log_to_monitors true 2024-09-19T08:55:15.119 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:14 smithi153 ceph-mon[35071]: from='osd.0 [v2:172.21.15.120:6802/4131180019,v1:172.21.15.120:6803/4131180019]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T08:55:15.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:14 smithi196 ceph-mon[35150]: from='osd.0 [v2:172.21.15.120:6802/4131180019,v1:172.21.15.120:6803/4131180019]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T08:55:15.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:14 smithi120 ceph-mon[31150]: from='osd.0 [v2:172.21.15.120:6802/4131180019,v1:172.21.15.120:6803/4131180019]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T08:55:15.911 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-19T08:55:15.937 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi153:vg_nvme/lv_4 2024-09-19T08:55:16.064 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:15 smithi120 ceph-mon[31150]: from='osd.0 [v2:172.21.15.120:6802/4131180019,v1:172.21.15.120:6803/4131180019]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T08:55:16.065 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:15 smithi120 ceph-mon[31150]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T08:55:16.065 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:15 smithi120 ceph-mon[31150]: from='osd.0 [v2:172.21.15.120:6802/4131180019,v1:172.21.15.120:6803/4131180019]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-19T08:55:16.065 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:15 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:55:16.065 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:15 smithi120 ceph-mon[31150]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:16.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:15 smithi196 ceph-mon[35150]: from='osd.0 [v2:172.21.15.120:6802/4131180019,v1:172.21.15.120:6803/4131180019]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T08:55:16.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:15 smithi196 ceph-mon[35150]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T08:55:16.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:15 smithi196 ceph-mon[35150]: from='osd.0 [v2:172.21.15.120:6802/4131180019,v1:172.21.15.120:6803/4131180019]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-19T08:55:16.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:15 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:55:16.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:15 smithi196 ceph-mon[35150]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:16.186 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:15 smithi153 ceph-mon[35071]: from='osd.0 [v2:172.21.15.120:6802/4131180019,v1:172.21.15.120:6803/4131180019]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T08:55:16.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:15 smithi153 ceph-mon[35071]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T08:55:16.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:15 smithi153 ceph-mon[35071]: from='osd.0 [v2:172.21.15.120:6802/4131180019,v1:172.21.15.120:6803/4131180019]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-19T08:55:16.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:15 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:55:16.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:15 smithi153 ceph-mon[35071]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:16.190 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:55:16.324 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 19 08:55:16 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-0[45204]: 2024-09-19T08:55:16.277+0000 7fa467b70640 -1 osd.0 0 waiting for initial osdmap 2024-09-19T08:55:16.324 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 19 08:55:16 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-0[45204]: 2024-09-19T08:55:16.280+0000 7fa45f990640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:55:16.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.613+0000 7eff3ff75640 1 -- 172.21.15.153:0/3410940765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff38104f90 msgr2=0x7eff38107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:16.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.613+0000 7eff3ff75640 1 --2- 172.21.15.153:0/3410940765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff38104f90 0x7eff38107380 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7eff28009860 tx=0x7eff2802a340 comp rx=0 tx=0).stop 2024-09-19T08:55:16.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.613+0000 7eff3ff75640 1 -- 172.21.15.153:0/3410940765 shutdown_connections 2024-09-19T08:55:16.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.613+0000 7eff3ff75640 1 --2- 172.21.15.153:0/3410940765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff381078c0 0x7eff38109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:16.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.613+0000 7eff3ff75640 1 --2- 172.21.15.153:0/3410940765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff38104f90 0x7eff38107380 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:16.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.613+0000 7eff3ff75640 1 --2- 172.21.15.153:0/3410940765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff38102660 0x7eff38104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:16.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.613+0000 7eff3ff75640 1 -- 172.21.15.153:0/3410940765 >> 172.21.15.153:0/3410940765 conn(0x7eff380fc280 msgr2=0x7eff380fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:16.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.613+0000 7eff3ff75640 1 -- 172.21.15.153:0/3410940765 shutdown_connections 2024-09-19T08:55:16.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.613+0000 7eff3ff75640 1 -- 172.21.15.153:0/3410940765 wait complete. 2024-09-19T08:55:16.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.613+0000 7eff3ff75640 1 Processor -- start 2024-09-19T08:55:16.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.614+0000 7eff3ff75640 1 -- start start 2024-09-19T08:55:16.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.614+0000 7eff3ff75640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff38102660 0x7eff38101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:16.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.614+0000 7eff3ff75640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff38104f90 0x7eff38100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:16.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.614+0000 7eff3ff75640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff381078c0 0x7eff38100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:16.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.614+0000 7eff3ff75640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7eff3810d860 con 0x7eff38102660 2024-09-19T08:55:16.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.614+0000 7eff3ff75640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7eff3810d6e0 con 0x7eff38104f90 2024-09-19T08:55:16.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.614+0000 7eff3ff75640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7eff3810d9e0 con 0x7eff381078c0 2024-09-19T08:55:16.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.614+0000 7eff3d510640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff38104f90 0x7eff38100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:16.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.614+0000 7eff3d510640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff38104f90 0x7eff38100610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39090/0 (socket says 172.21.15.153:39090) 2024-09-19T08:55:16.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.614+0000 7eff3d510640 1 -- 172.21.15.153:0/986086169 learned_addr learned my addr 172.21.15.153:0/986086169 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-19T08:55:16.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.614+0000 7eff3dd11640 1 --2- 172.21.15.153:0/986086169 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff38102660 0x7eff38101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:16.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.614+0000 7eff3d510640 1 -- 172.21.15.153:0/986086169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff381078c0 msgr2=0x7eff38100b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:16.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.614+0000 7eff3e512640 1 --2- 172.21.15.153:0/986086169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff381078c0 0x7eff38100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:16.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.614+0000 7eff3d510640 1 --2- 172.21.15.153:0/986086169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff381078c0 0x7eff38100b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:16.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.614+0000 7eff3d510640 1 -- 172.21.15.153:0/986086169 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff38102660 msgr2=0x7eff38101fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:16.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.614+0000 7eff3d510640 1 --2- 172.21.15.153:0/986086169 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff38102660 0x7eff38101fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:16.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.614+0000 7eff3d510640 1 -- 172.21.15.153:0/986086169 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff38101410 con 0x7eff38104f90 2024-09-19T08:55:16.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.614+0000 7eff3e512640 1 --2- 172.21.15.153:0/986086169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff381078c0 0x7eff38100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:55:16.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.614+0000 7eff3d510640 1 --2- 172.21.15.153:0/986086169 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff38104f90 0x7eff38100610 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7eff280305b0 tx=0x7eff28033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:16.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.614+0000 7eff2effd640 1 -- 172.21.15.153:0/986086169 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2803f020 con 0x7eff38104f90 2024-09-19T08:55:16.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.615+0000 7eff3ff75640 1 -- 172.21.15.153:0/986086169 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff381a2040 con 0x7eff38104f90 2024-09-19T08:55:16.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.615+0000 7eff2effd640 1 -- 172.21.15.153:0/986086169 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff28030850 con 0x7eff38104f90 2024-09-19T08:55:16.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.615+0000 7eff2effd640 1 -- 172.21.15.153:0/986086169 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff28037040 con 0x7eff38104f90 2024-09-19T08:55:16.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.615+0000 7eff3ff75640 1 -- 172.21.15.153:0/986086169 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff381a2540 con 0x7eff38104f90 2024-09-19T08:55:16.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.616+0000 7eff3ff75640 1 -- 172.21.15.153:0/986086169 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff380fdd70 con 0x7eff38104f90 2024-09-19T08:55:16.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.617+0000 7eff2effd640 1 -- 172.21.15.153:0/986086169 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7eff28042020 con 0x7eff38104f90 2024-09-19T08:55:16.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.617+0000 7eff2effd640 1 --2- 172.21.15.153:0/986086169 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7eff240a3230 0x7eff240a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:16.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.617+0000 7eff2effd640 1 -- 172.21.15.153:0/986086169 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7eff280ee040 con 0x7eff38104f90 2024-09-19T08:55:16.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.618+0000 7eff3dd11640 1 --2- 172.21.15.153:0/986086169 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7eff240a3230 0x7eff240a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:16.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.618+0000 7eff3dd11640 1 --2- 172.21.15.153:0/986086169 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7eff240a3230 0x7eff240a56f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7eff34009530 tx=0x7eff34009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:16.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.622+0000 7eff2effd640 1 -- 172.21.15.153:0/986086169 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7eff28049020 con 0x7eff38104f90 2024-09-19T08:55:16.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:16.794+0000 7eff3ff75640 1 -- 172.21.15.153:0/986086169 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7eff38062630 con 0x7eff240a3230 2024-09-19T08:55:16.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:16 smithi153 ceph-mon[35071]: from='osd.0 [v2:172.21.15.120:6802/4131180019,v1:172.21.15.120:6803/4131180019]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-19T08:55:16.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:16 smithi153 ceph-mon[35071]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T08:55:16.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:55:16.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:55:16.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:16 smithi153 ceph-mon[35071]: Detected new or changed devices on smithi120 2024-09-19T08:55:16.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:16.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:16.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:55:16.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:16 smithi153 ceph-mon[35071]: Adjusting osd_memory_target on smithi120 to 17088M 2024-09-19T08:55:16.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:16.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:16.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:55:16.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:16.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:16 smithi153 ceph-mon[35071]: from='osd.0 [v2:172.21.15.120:6802/4131180019,v1:172.21.15.120:6803/4131180019]' entity='osd.0' 2024-09-19T08:55:16.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:55:16.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:55:16.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:17.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:16 smithi120 ceph-mon[31150]: from='osd.0 [v2:172.21.15.120:6802/4131180019,v1:172.21.15.120:6803/4131180019]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-19T08:55:17.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:16 smithi120 ceph-mon[31150]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T08:55:17.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:55:17.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:55:17.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:16 smithi120 ceph-mon[31150]: Detected new or changed devices on smithi120 2024-09-19T08:55:17.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:17.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:17.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:55:17.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:16 smithi120 ceph-mon[31150]: Adjusting osd_memory_target on smithi120 to 17088M 2024-09-19T08:55:17.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:17.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:17.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:55:17.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:17.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:16 smithi120 ceph-mon[31150]: from='osd.0 [v2:172.21.15.120:6802/4131180019,v1:172.21.15.120:6803/4131180019]' entity='osd.0' 2024-09-19T08:55:17.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:55:17.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:55:17.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:17.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:16 smithi196 ceph-mon[35150]: from='osd.0 [v2:172.21.15.120:6802/4131180019,v1:172.21.15.120:6803/4131180019]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-19T08:55:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:16 smithi196 ceph-mon[35150]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T08:55:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:55:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:55:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:16 smithi196 ceph-mon[35150]: Detected new or changed devices on smithi120 2024-09-19T08:55:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:55:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:16 smithi196 ceph-mon[35150]: Adjusting osd_memory_target on smithi120 to 17088M 2024-09-19T08:55:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:55:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:17.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:16 smithi196 ceph-mon[35150]: from='osd.0 [v2:172.21.15.120:6802/4131180019,v1:172.21.15.120:6803/4131180019]' entity='osd.0' 2024-09-19T08:55:17.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:55:17.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:55:17.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:18.101 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:17 smithi153 ceph-mon[35071]: purged_snaps scrub starts 2024-09-19T08:55:18.101 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:17 smithi153 ceph-mon[35071]: purged_snaps scrub ok 2024-09-19T08:55:18.101 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:17 smithi153 ceph-mon[35071]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:55:18.101 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:17 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:55:18.101 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:17 smithi153 ceph-mon[35071]: osd.0 [v2:172.21.15.120:6802/4131180019,v1:172.21.15.120:6803/4131180019] boot 2024-09-19T08:55:18.101 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:17 smithi153 ceph-mon[35071]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T08:55:18.102 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:17 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:55:18.102 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:17 smithi153 ceph-mon[35071]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:18.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:17 smithi196 ceph-mon[35150]: purged_snaps scrub starts 2024-09-19T08:55:18.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:17 smithi196 ceph-mon[35150]: purged_snaps scrub ok 2024-09-19T08:55:18.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:17 smithi196 ceph-mon[35150]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:55:18.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:17 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:55:18.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:17 smithi196 ceph-mon[35150]: osd.0 [v2:172.21.15.120:6802/4131180019,v1:172.21.15.120:6803/4131180019] boot 2024-09-19T08:55:18.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:17 smithi196 ceph-mon[35150]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T08:55:18.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:17 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:55:18.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:17 smithi196 ceph-mon[35150]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:18.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:17 smithi120 ceph-mon[31150]: purged_snaps scrub starts 2024-09-19T08:55:18.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:17 smithi120 ceph-mon[31150]: purged_snaps scrub ok 2024-09-19T08:55:18.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:17 smithi120 ceph-mon[31150]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:55:18.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:17 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:55:18.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:17 smithi120 ceph-mon[31150]: osd.0 [v2:172.21.15.120:6802/4131180019,v1:172.21.15.120:6803/4131180019] boot 2024-09-19T08:55:18.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:17 smithi120 ceph-mon[31150]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T08:55:18.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:17 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:55:18.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:17 smithi120 ceph-mon[31150]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:55:19.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:19 smithi120 ceph-mon[31150]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T08:55:19.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:19 smithi120 ceph-mon[31150]: from='client.? 172.21.15.153:0/1538372846' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "450bbebf-2aac-47c6-8474-bbd37fd0a1a0"}]: dispatch 2024-09-19T08:55:19.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:19 smithi120 ceph-mon[31150]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "450bbebf-2aac-47c6-8474-bbd37fd0a1a0"}]: dispatch 2024-09-19T08:55:19.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:19 smithi120 ceph-mon[31150]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "450bbebf-2aac-47c6-8474-bbd37fd0a1a0"}]': finished 2024-09-19T08:55:19.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:19 smithi120 ceph-mon[31150]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T08:55:19.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:19 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:55:19.626 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:19 smithi153 ceph-mon[35071]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T08:55:19.627 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:19 smithi153 ceph-mon[35071]: from='client.? 172.21.15.153:0/1538372846' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "450bbebf-2aac-47c6-8474-bbd37fd0a1a0"}]: dispatch 2024-09-19T08:55:19.627 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:19 smithi153 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "450bbebf-2aac-47c6-8474-bbd37fd0a1a0"}]: dispatch 2024-09-19T08:55:19.627 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:19 smithi153 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "450bbebf-2aac-47c6-8474-bbd37fd0a1a0"}]': finished 2024-09-19T08:55:19.627 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:19 smithi153 ceph-mon[35071]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T08:55:19.627 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:19 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:55:19.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:19 smithi196 ceph-mon[35150]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T08:55:19.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:19 smithi196 ceph-mon[35150]: from='client.? 172.21.15.153:0/1538372846' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "450bbebf-2aac-47c6-8474-bbd37fd0a1a0"}]: dispatch 2024-09-19T08:55:19.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:19 smithi196 ceph-mon[35150]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "450bbebf-2aac-47c6-8474-bbd37fd0a1a0"}]: dispatch 2024-09-19T08:55:19.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:19 smithi196 ceph-mon[35150]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "450bbebf-2aac-47c6-8474-bbd37fd0a1a0"}]': finished 2024-09-19T08:55:19.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:19 smithi196 ceph-mon[35150]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T08:55:19.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:19 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:55:20.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:20 smithi120 ceph-mon[31150]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:20.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:20 smithi120 ceph-mon[31150]: from='client.? 172.21.15.153:0/3975699916' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:55:20.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:20 smithi196 ceph-mon[35150]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:20.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:20 smithi196 ceph-mon[35150]: from='client.? 172.21.15.153:0/3975699916' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:55:20.686 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:20 smithi153 ceph-mon[35071]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:20.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:20 smithi153 ceph-mon[35071]: from='client.? 172.21.15.153:0/3975699916' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:55:22.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:22 smithi196 ceph-mon[35150]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:22.936 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:22 smithi153 ceph-mon[35071]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:23.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:22 smithi120 ceph-mon[31150]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:24.792 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:24 smithi153 ceph-mon[35071]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:24.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:24 smithi196 ceph-mon[35150]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:25.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:24 smithi120 ceph-mon[31150]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:26.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:26 smithi196 ceph-mon[35150]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:26.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:26 smithi153 ceph-mon[35071]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:27.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:26 smithi120 ceph-mon[31150]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:27.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:27 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T08:55:27.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:27 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:27.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:27 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T08:55:27.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:27 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:28.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:27 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T08:55:28.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:27 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:28.884 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:28 smithi153 ceph-mon[35071]: Deploying daemon osd.1 on smithi153 2024-09-19T08:55:28.885 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:28 smithi153 ceph-mon[35071]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:28.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:28 smithi196 ceph-mon[35150]: Deploying daemon osd.1 on smithi153 2024-09-19T08:55:28.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:28 smithi196 ceph-mon[35150]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:29.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:28 smithi120 ceph-mon[31150]: Deploying daemon osd.1 on smithi153 2024-09-19T08:55:29.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:28 smithi120 ceph-mon[31150]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:30.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:30 smithi153 ceph-mon[35071]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:30.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:30 smithi196 ceph-mon[35150]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:31.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:30 smithi120 ceph-mon[31150]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:32.925 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:32 smithi153 ceph-mon[35071]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:32.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:32 smithi196 ceph-mon[35150]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:33.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:32 smithi120 ceph-mon[31150]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:33.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:33 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:55:33.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:33 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:33.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:33 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:33.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:33 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:55:33.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:33 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:33.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:33 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:34.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:33 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:55:34.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:33 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:34.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:33 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:34.910 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:34 smithi153 ceph-mon[35071]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:34.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:34 smithi196 ceph-mon[35150]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:35.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:34 smithi120 ceph-mon[31150]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:36.786 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:36 smithi153 ceph-mon[35071]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:36.787 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:36 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:36.787 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:36 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:36.787 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:36 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:36.787 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:36 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:55:36.787 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:36 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:36.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:36 smithi196 ceph-mon[35150]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:36.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:36 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:36.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:36 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:36.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:36 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:36.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:36 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:55:36.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:36 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:37.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:36 smithi120 ceph-mon[31150]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:37.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:36 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:37.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:36 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:37.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:36 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:37.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:36 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:55:37.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:36 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:37.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:37.083+0000 7eff2effd640 1 -- 172.21.15.153:0/986086169 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7eff38062630 con 0x7eff240a3230 2024-09-19T08:55:37.083 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 1 on host 'smithi153' 2024-09-19T08:55:37.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:37.086+0000 7eff3ff75640 1 -- 172.21.15.153:0/986086169 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7eff240a3230 msgr2=0x7eff240a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:37.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:37.086+0000 7eff3ff75640 1 --2- 172.21.15.153:0/986086169 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7eff240a3230 0x7eff240a56f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7eff34009530 tx=0x7eff34009260 comp rx=0 tx=0).stop 2024-09-19T08:55:37.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:37.086+0000 7eff3ff75640 1 -- 172.21.15.153:0/986086169 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff38104f90 msgr2=0x7eff38100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:37.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:37.086+0000 7eff3ff75640 1 --2- 172.21.15.153:0/986086169 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff38104f90 0x7eff38100610 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7eff280305b0 tx=0x7eff28033d90 comp rx=0 tx=0).stop 2024-09-19T08:55:37.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:37.086+0000 7eff3ff75640 1 -- 172.21.15.153:0/986086169 shutdown_connections 2024-09-19T08:55:37.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:37.086+0000 7eff3ff75640 1 --2- 172.21.15.153:0/986086169 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7eff240a3230 0x7eff240a56f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:37.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:37.086+0000 7eff3ff75640 1 --2- 172.21.15.153:0/986086169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff381078c0 0x7eff38100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:37.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:37.086+0000 7eff3ff75640 1 --2- 172.21.15.153:0/986086169 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff38104f90 0x7eff38100610 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:37.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:37.086+0000 7eff3ff75640 1 --2- 172.21.15.153:0/986086169 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff38102660 0x7eff38101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:37.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:37.086+0000 7eff3ff75640 1 -- 172.21.15.153:0/986086169 >> 172.21.15.153:0/986086169 conn(0x7eff380fc280 msgr2=0x7eff38103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:37.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:37.086+0000 7eff3ff75640 1 -- 172.21.15.153:0/986086169 shutdown_connections 2024-09-19T08:55:37.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:55:37.086+0000 7eff3ff75640 1 -- 172.21.15.153:0/986086169 wait complete. 2024-09-19T08:55:37.657 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:37 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:55:37.657 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:37 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:37.657 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:37 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:37.682 DEBUG:teuthology.orchestra.run.smithi153:osd.1> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.1.service 2024-09-19T08:55:37.685 INFO:tasks.cephadm:Deploying osd.2 on smithi196 with /dev/vg_nvme/lv_4... 2024-09-19T08:55:37.685 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T08:55:37.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:37 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:55:37.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:37 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:37.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:37 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:37.944 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:55:38.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:37 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:55:38.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:37 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:38.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:37 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:38.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:38 smithi196 ceph-mon[35150]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:38.921 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:38 smithi153 ceph-mon[35071]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:39.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:38 smithi120 ceph-mon[31150]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:39.437 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 19 08:55:39 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-1[39526]: 2024-09-19T08:55:39.372+0000 7f1a9a3f3640 -1 osd.1 0 log_to_monitors true 2024-09-19T08:55:39.880 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:39 smithi153 ceph-mon[35071]: from='osd.1 [v2:172.21.15.153:6800/2259730294,v1:172.21.15.153:6801/2259730294]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T08:55:39.880 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:39 smithi153 ceph-mon[35071]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T08:55:39.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:39 smithi196 ceph-mon[35150]: from='osd.1 [v2:172.21.15.153:6800/2259730294,v1:172.21.15.153:6801/2259730294]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T08:55:39.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:39 smithi196 ceph-mon[35150]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T08:55:40.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:39 smithi120 ceph-mon[31150]: from='osd.1 [v2:172.21.15.153:6800/2259730294,v1:172.21.15.153:6801/2259730294]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T08:55:40.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:39 smithi120 ceph-mon[31150]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T08:55:40.357 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T08:55:40.384 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_4 2024-09-19T08:55:40.632 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:55:40.895 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:40 smithi196 ceph-mon[35150]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:40.895 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:40 smithi196 ceph-mon[35150]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T08:55:40.895 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:40 smithi196 ceph-mon[35150]: from='osd.1 [v2:172.21.15.153:6800/2259730294,v1:172.21.15.153:6801/2259730294]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-19T08:55:40.895 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:40 smithi196 ceph-mon[35150]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T08:55:40.895 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:40 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:55:40.895 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:40 smithi196 ceph-mon[35150]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-19T08:55:40.895 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:40 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:40.895 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:40 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:40.895 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:40 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:55:40.896 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:40 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:40.896 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:40 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:40.896 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:40 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:55:40.896 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:40 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:40.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:40 smithi153 ceph-mon[35071]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:40.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:40 smithi153 ceph-mon[35071]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T08:55:40.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:40 smithi153 ceph-mon[35071]: from='osd.1 [v2:172.21.15.153:6800/2259730294,v1:172.21.15.153:6801/2259730294]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-19T08:55:40.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:40 smithi153 ceph-mon[35071]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T08:55:40.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:40 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:55:40.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:40 smithi153 ceph-mon[35071]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-19T08:55:40.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:40 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:40.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:40 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:40.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:40 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:55:40.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:40 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:40.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:40 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:40.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:40 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:55:40.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:40 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:41.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:40 smithi120 ceph-mon[31150]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:41.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:40 smithi120 ceph-mon[31150]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T08:55:41.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:40 smithi120 ceph-mon[31150]: from='osd.1 [v2:172.21.15.153:6800/2259730294,v1:172.21.15.153:6801/2259730294]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-19T08:55:41.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:40 smithi120 ceph-mon[31150]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T08:55:41.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:40 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:55:41.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:40 smithi120 ceph-mon[31150]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-19T08:55:41.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:40 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:41.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:40 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:41.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:40 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:55:41.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:40 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:41.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:40 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:41.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:40 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:55:41.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:40 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:41.099 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.087+0000 7fa9cd750640 1 -- 172.21.15.196:0/215220874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9c81020f0 msgr2=0x7fa9c81024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:41.099 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.087+0000 7fa9cd750640 1 --2- 172.21.15.196:0/215220874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9c81020f0 0x7fa9c81024f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8009930 tx=0x7fa9b802a420 comp rx=0 tx=0).stop 2024-09-19T08:55:41.099 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.087+0000 7fa9cd750640 1 -- 172.21.15.196:0/215220874 shutdown_connections 2024-09-19T08:55:41.099 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.087+0000 7fa9cd750640 1 --2- 172.21.15.196:0/215220874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9c8101760 0x7fa9c8109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:41.099 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.087+0000 7fa9cd750640 1 --2- 172.21.15.196:0/215220874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9c8100da0 0x7fa9c8101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:41.099 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.087+0000 7fa9cd750640 1 --2- 172.21.15.196:0/215220874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9c81020f0 0x7fa9c81024f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:41.099 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.087+0000 7fa9cd750640 1 -- 172.21.15.196:0/215220874 >> 172.21.15.196:0/215220874 conn(0x7fa9c80fba40 msgr2=0x7fa9c80fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:41.099 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.087+0000 7fa9cd750640 1 -- 172.21.15.196:0/215220874 shutdown_connections 2024-09-19T08:55:41.099 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.087+0000 7fa9cd750640 1 -- 172.21.15.196:0/215220874 wait complete. 2024-09-19T08:55:41.099 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.088+0000 7fa9cd750640 1 Processor -- start 2024-09-19T08:55:41.100 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.088+0000 7fa9cd750640 1 -- start start 2024-09-19T08:55:41.100 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.088+0000 7fa9cd750640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9c8100da0 0x7fa9c81007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:41.100 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.088+0000 7fa9cd750640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9c8101760 0x7fa9c819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:41.100 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.088+0000 7fa9cd750640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9c81020f0 0x7fa9c819f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:41.100 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.088+0000 7fa9cd750640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c810c790 con 0x7fa9c81020f0 2024-09-19T08:55:41.100 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.088+0000 7fa9cd750640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c810c610 con 0x7fa9c8101760 2024-09-19T08:55:41.100 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.088+0000 7fa9cd750640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c810c910 con 0x7fa9c8100da0 2024-09-19T08:55:41.100 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.088+0000 7fa9c6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9c8100da0 0x7fa9c81007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:41.100 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.088+0000 7fa9c6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9c8100da0 0x7fa9c81007f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:46240/0 (socket says 172.21.15.196:46240) 2024-09-19T08:55:41.100 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.088+0000 7fa9c6ffd640 1 -- 172.21.15.196:0/3888183160 learned_addr learned my addr 172.21.15.196:0/3888183160 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-19T08:55:41.100 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.088+0000 7fa9c77fe640 1 --2- 172.21.15.196:0/3888183160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9c81020f0 0x7fa9c819f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:41.100 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.088+0000 7fa9bffff640 1 --2- 172.21.15.196:0/3888183160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9c8101760 0x7fa9c819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:41.101 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.089+0000 7fa9c6ffd640 1 -- 172.21.15.196:0/3888183160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9c8101760 msgr2=0x7fa9c819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:41.101 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.089+0000 7fa9c6ffd640 1 --2- 172.21.15.196:0/3888183160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9c8101760 0x7fa9c819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:41.101 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.089+0000 7fa9c6ffd640 1 -- 172.21.15.196:0/3888183160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9c81020f0 msgr2=0x7fa9c819f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:41.101 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.089+0000 7fa9c6ffd640 1 --2- 172.21.15.196:0/3888183160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9c81020f0 0x7fa9c819f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:41.101 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.089+0000 7fa9c6ffd640 1 -- 172.21.15.196:0/3888183160 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c819fc60 con 0x7fa9c8100da0 2024-09-19T08:55:41.101 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.089+0000 7fa9c6ffd640 1 --2- 172.21.15.196:0/3888183160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9c8100da0 0x7fa9c81007f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8009230 tx=0x7fa9b80042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:41.101 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.089+0000 7fa9c4ff9640 1 -- 172.21.15.196:0/3888183160 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b803f030 con 0x7fa9c8100da0 2024-09-19T08:55:41.101 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.089+0000 7fa9c4ff9640 1 -- 172.21.15.196:0/3888183160 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9b8003ce0 con 0x7fa9c8100da0 2024-09-19T08:55:41.101 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.089+0000 7fa9cd750640 1 -- 172.21.15.196:0/3888183160 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c819fe90 con 0x7fa9c8100da0 2024-09-19T08:55:41.101 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.089+0000 7fa9c4ff9640 1 -- 172.21.15.196:0/3888183160 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8035070 con 0x7fa9c8100da0 2024-09-19T08:55:41.101 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.089+0000 7fa9cd750640 1 -- 172.21.15.196:0/3888183160 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c81a02f0 con 0x7fa9c8100da0 2024-09-19T08:55:41.101 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.090+0000 7fa9cd750640 1 -- 172.21.15.196:0/3888183160 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9c8108050 con 0x7fa9c8100da0 2024-09-19T08:55:41.101 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.091+0000 7fa9c4ff9640 1 -- 172.21.15.196:0/3888183160 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7fa9b8042030 con 0x7fa9c8100da0 2024-09-19T08:55:41.102 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.092+0000 7fa9c4ff9640 1 --2- 172.21.15.196:0/3888183160 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9b40a3230 0x7fa9b40a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:41.102 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.092+0000 7fa9c4ff9640 1 -- 172.21.15.196:0/3888183160 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7fa9b80ee2c0 con 0x7fa9c8100da0 2024-09-19T08:55:41.102 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.092+0000 7fa9bffff640 1 --2- 172.21.15.196:0/3888183160 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9b40a3230 0x7fa9b40a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:41.102 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.092+0000 7fa9bffff640 1 --2- 172.21.15.196:0/3888183160 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9b40a3230 0x7fa9b40a56f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8100d70 tx=0x7fa9b000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:41.102 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.096+0000 7fa9c4ff9640 1 -- 172.21.15.196:0/3888183160 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa9b80bc300 con 0x7fa9c8100da0 2024-09-19T08:55:41.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:55:41.272+0000 7fa9cd750640 1 -- 172.21.15.196:0/3888183160 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa9c8062630 con 0x7fa9b40a3230 2024-09-19T08:55:41.437 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 19 08:55:41 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-1[39526]: 2024-09-19T08:55:41.072+0000 7f1a96b8a640 -1 osd.1 0 waiting for initial osdmap 2024-09-19T08:55:41.437 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 19 08:55:41 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-1[39526]: 2024-09-19T08:55:41.075+0000 7f1a8e196640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:55:41.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:41 smithi196 ceph-mon[35150]: Detected new or changed devices on smithi153 2024-09-19T08:55:41.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:41 smithi196 ceph-mon[35150]: Adjusting osd_memory_target on smithi153 to 17088M 2024-09-19T08:55:41.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:41 smithi196 ceph-mon[35150]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-19T08:55:41.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:41 smithi196 ceph-mon[35150]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T08:55:41.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:41 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:55:41.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:41 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:55:41.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:41 smithi196 ceph-mon[35150]: from='osd.1 ' entity='osd.1' 2024-09-19T08:55:41.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:41 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:55:41.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:41 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:55:41.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:41 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:41.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:41 smithi153 ceph-mon[35071]: Detected new or changed devices on smithi153 2024-09-19T08:55:41.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:41 smithi153 ceph-mon[35071]: Adjusting osd_memory_target on smithi153 to 17088M 2024-09-19T08:55:41.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:41 smithi153 ceph-mon[35071]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-19T08:55:41.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:41 smithi153 ceph-mon[35071]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T08:55:41.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:41 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:55:41.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:41 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:55:41.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:41 smithi153 ceph-mon[35071]: from='osd.1 ' entity='osd.1' 2024-09-19T08:55:41.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:41 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:55:41.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:41 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:55:41.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:41 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:42.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:41 smithi120 ceph-mon[31150]: Detected new or changed devices on smithi153 2024-09-19T08:55:42.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:41 smithi120 ceph-mon[31150]: Adjusting osd_memory_target on smithi153 to 17088M 2024-09-19T08:55:42.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:41 smithi120 ceph-mon[31150]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-19T08:55:42.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:41 smithi120 ceph-mon[31150]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T08:55:42.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:41 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:55:42.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:41 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:55:42.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:41 smithi120 ceph-mon[31150]: from='osd.1 ' entity='osd.1' 2024-09-19T08:55:42.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:41 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:55:42.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:41 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:55:42.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:41 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:42.826 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:42 smithi196 ceph-mon[35150]: purged_snaps scrub starts 2024-09-19T08:55:42.827 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:42 smithi196 ceph-mon[35150]: purged_snaps scrub ok 2024-09-19T08:55:42.827 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:42 smithi196 ceph-mon[35150]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:55:42.827 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:42 smithi196 ceph-mon[35150]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:42.827 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:42 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:55:42.827 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:42 smithi196 ceph-mon[35150]: osd.1 [v2:172.21.15.153:6800/2259730294,v1:172.21.15.153:6801/2259730294] boot 2024-09-19T08:55:42.827 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:42 smithi196 ceph-mon[35150]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T08:55:42.827 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:42 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:55:42.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:42 smithi153 ceph-mon[35071]: purged_snaps scrub starts 2024-09-19T08:55:42.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:42 smithi153 ceph-mon[35071]: purged_snaps scrub ok 2024-09-19T08:55:42.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:42 smithi153 ceph-mon[35071]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:55:42.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:42 smithi153 ceph-mon[35071]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:42.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:42 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:55:42.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:42 smithi153 ceph-mon[35071]: osd.1 [v2:172.21.15.153:6800/2259730294,v1:172.21.15.153:6801/2259730294] boot 2024-09-19T08:55:42.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:42 smithi153 ceph-mon[35071]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T08:55:42.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:42 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:55:43.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:42 smithi120 ceph-mon[31150]: purged_snaps scrub starts 2024-09-19T08:55:43.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:42 smithi120 ceph-mon[31150]: purged_snaps scrub ok 2024-09-19T08:55:43.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:42 smithi120 ceph-mon[31150]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:55:43.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:42 smithi120 ceph-mon[31150]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:55:43.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:42 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:55:43.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:42 smithi120 ceph-mon[31150]: osd.1 [v2:172.21.15.153:6800/2259730294,v1:172.21.15.153:6801/2259730294] boot 2024-09-19T08:55:43.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:42 smithi120 ceph-mon[31150]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T08:55:43.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:42 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:55:44.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:44 smithi196 ceph-mon[35150]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T08:55:44.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:44 smithi196 ceph-mon[35150]: from='client.? 172.21.15.196:0/884684010' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac8c4b83-8e8e-4803-b266-262f7873bd85"}]: dispatch 2024-09-19T08:55:44.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:44 smithi196 ceph-mon[35150]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac8c4b83-8e8e-4803-b266-262f7873bd85"}]: dispatch 2024-09-19T08:55:44.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:44 smithi196 ceph-mon[35150]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac8c4b83-8e8e-4803-b266-262f7873bd85"}]': finished 2024-09-19T08:55:44.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:44 smithi196 ceph-mon[35150]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T08:55:44.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:44 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:55:44.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:44 smithi196 ceph-mon[35150]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:55:44.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:44 smithi196 ceph-mon[35150]: from='client.? 172.21.15.196:0/2402457046' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:55:44.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:44 smithi153 ceph-mon[35071]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T08:55:44.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:44 smithi153 ceph-mon[35071]: from='client.? 172.21.15.196:0/884684010' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac8c4b83-8e8e-4803-b266-262f7873bd85"}]: dispatch 2024-09-19T08:55:44.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:44 smithi153 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac8c4b83-8e8e-4803-b266-262f7873bd85"}]: dispatch 2024-09-19T08:55:44.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:44 smithi153 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac8c4b83-8e8e-4803-b266-262f7873bd85"}]': finished 2024-09-19T08:55:44.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:44 smithi153 ceph-mon[35071]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T08:55:44.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:44 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:55:44.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:44 smithi153 ceph-mon[35071]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:55:44.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:44 smithi153 ceph-mon[35071]: from='client.? 172.21.15.196:0/2402457046' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:55:44.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:44 smithi120 ceph-mon[31150]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T08:55:44.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:44 smithi120 ceph-mon[31150]: from='client.? 172.21.15.196:0/884684010' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac8c4b83-8e8e-4803-b266-262f7873bd85"}]: dispatch 2024-09-19T08:55:44.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:44 smithi120 ceph-mon[31150]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac8c4b83-8e8e-4803-b266-262f7873bd85"}]: dispatch 2024-09-19T08:55:44.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:44 smithi120 ceph-mon[31150]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac8c4b83-8e8e-4803-b266-262f7873bd85"}]': finished 2024-09-19T08:55:44.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:44 smithi120 ceph-mon[31150]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T08:55:44.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:44 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:55:44.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:44 smithi120 ceph-mon[31150]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:55:44.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:44 smithi120 ceph-mon[31150]: from='client.? 172.21.15.196:0/2402457046' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:55:46.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:46 smithi196 ceph-mon[35150]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:55:46.936 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:46 smithi153 ceph-mon[35071]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:55:47.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:46 smithi120 ceph-mon[31150]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:55:48.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:48 smithi196 ceph-mon[35150]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:55:48.936 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:48 smithi153 ceph-mon[35071]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:55:49.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:48 smithi120 ceph-mon[31150]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:55:50.903 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:50 smithi196 ceph-mon[35150]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:55:50.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:50 smithi153 ceph-mon[35071]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:55:51.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:50 smithi120 ceph-mon[31150]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:55:52.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:52 smithi196 ceph-mon[35150]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:55:52.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:52 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T08:55:52.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:52 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:52.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:52 smithi196 ceph-mon[35150]: Deploying daemon osd.2 on smithi196 2024-09-19T08:55:52.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:52 smithi153 ceph-mon[35071]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:55:52.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:52 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T08:55:52.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:52 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:52.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:52 smithi153 ceph-mon[35071]: Deploying daemon osd.2 on smithi196 2024-09-19T08:55:53.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:52 smithi120 ceph-mon[31150]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:55:53.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:52 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T08:55:53.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:52 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:53.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:52 smithi120 ceph-mon[31150]: Deploying daemon osd.2 on smithi196 2024-09-19T08:55:54.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:54 smithi196 ceph-mon[35150]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:55:54.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:54 smithi153 ceph-mon[35071]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:55:55.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:54 smithi120 ceph-mon[31150]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:55:56.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:56 smithi196 ceph-mon[35150]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:55:56.936 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:56 smithi153 ceph-mon[35071]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:55:57.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:56 smithi120 ceph-mon[31150]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:55:58.885 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:58 smithi196 ceph-mon[35150]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:55:58.885 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:58 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:55:58.885 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:58 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:58.885 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:55:58 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:58.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:58 smithi153 ceph-mon[35071]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:55:58.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:58 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:55:58.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:58 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:58.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:55:58 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:59.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:58 smithi120 ceph-mon[31150]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:55:59.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:58 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:55:59.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:58 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:55:59.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:55:58 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:00.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:00 smithi196 ceph-mon[35150]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:56:00.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:00.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:00.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:00.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:56:00.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:00.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:00 smithi153 ceph-mon[35071]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:56:00.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:00.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:00.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:00.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:56:00.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:01.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:00 smithi120 ceph-mon[31150]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:56:01.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:01.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:01.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:01.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:56:01.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:01.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:01.818+0000 7fa9c4ff9640 1 -- 172.21.15.196:0/3888183160 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa9c8062630 con 0x7fa9b40a3230 2024-09-19T08:56:01.819 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 2 on host 'smithi196' 2024-09-19T08:56:01.820 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:01.821+0000 7fa9cd750640 1 -- 172.21.15.196:0/3888183160 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9b40a3230 msgr2=0x7fa9b40a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:01.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:01.821+0000 7fa9cd750640 1 --2- 172.21.15.196:0/3888183160 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9b40a3230 0x7fa9b40a56f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8100d70 tx=0x7fa9b000b410 comp rx=0 tx=0).stop 2024-09-19T08:56:01.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:01.821+0000 7fa9cd750640 1 -- 172.21.15.196:0/3888183160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9c8100da0 msgr2=0x7fa9c81007f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:01.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:01.821+0000 7fa9cd750640 1 --2- 172.21.15.196:0/3888183160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9c8100da0 0x7fa9c81007f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8009230 tx=0x7fa9b80042f0 comp rx=0 tx=0).stop 2024-09-19T08:56:01.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:01.821+0000 7fa9cd750640 1 -- 172.21.15.196:0/3888183160 shutdown_connections 2024-09-19T08:56:01.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:01.821+0000 7fa9cd750640 1 --2- 172.21.15.196:0/3888183160 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9b40a3230 0x7fa9b40a56f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:01.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:01.821+0000 7fa9cd750640 1 --2- 172.21.15.196:0/3888183160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9c81020f0 0x7fa9c819f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:01.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:01.821+0000 7fa9cd750640 1 --2- 172.21.15.196:0/3888183160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9c8101760 0x7fa9c819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:01.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:01.821+0000 7fa9cd750640 1 --2- 172.21.15.196:0/3888183160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9c8100da0 0x7fa9c81007f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:01.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:01.821+0000 7fa9cd750640 1 -- 172.21.15.196:0/3888183160 >> 172.21.15.196:0/3888183160 conn(0x7fa9c80fba40 msgr2=0x7fa9c8105220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:01.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:01.821+0000 7fa9cd750640 1 -- 172.21.15.196:0/3888183160 shutdown_connections 2024-09-19T08:56:01.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:01.821+0000 7fa9cd750640 1 -- 172.21.15.196:0/3888183160 wait complete. 2024-09-19T08:56:02.434 DEBUG:teuthology.orchestra.run.smithi196:osd.2> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.2.service 2024-09-19T08:56:02.436 INFO:tasks.cephadm:Deploying osd.3 on smithi120 with /dev/vg_nvme/lv_3... 2024-09-19T08:56:02.436 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T08:56:02.657 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:02 smithi196 ceph-mon[35150]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:56:02.657 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:02 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:56:02.657 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:02 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:02.657 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:02 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:02.675 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:56:02.796 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:02 smithi120 ceph-mon[31150]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:56:02.796 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:02 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:56:02.796 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:02 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:02.796 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:02 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:02.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:02 smithi153 ceph-mon[35071]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:56:02.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:02 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:56:02.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:02 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:02.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:02 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:04.186 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 08:56:03 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-2[40117]: 2024-09-19T08:56:03.923+0000 7f9698a14640 -1 osd.2 0 log_to_monitors true 2024-09-19T08:56:04.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:04 smithi196 ceph-mon[35150]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:56:04.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:04 smithi196 ceph-mon[35150]: from='osd.2 [v2:172.21.15.196:6800/2942286570,v1:172.21.15.196:6801/2942286570]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T08:56:04.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:04 smithi196 ceph-mon[35150]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T08:56:04.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:04 smithi153 ceph-mon[35071]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:56:04.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:04 smithi153 ceph-mon[35071]: from='osd.2 [v2:172.21.15.196:6800/2942286570,v1:172.21.15.196:6801/2942286570]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T08:56:04.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:04 smithi153 ceph-mon[35071]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T08:56:05.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:04 smithi120 ceph-mon[31150]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:56:05.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:04 smithi120 ceph-mon[31150]: from='osd.2 [v2:172.21.15.196:6800/2942286570,v1:172.21.15.196:6801/2942286570]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T08:56:05.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:04 smithi120 ceph-mon[31150]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T08:56:05.164 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:56:05.193 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi120:vg_nvme/lv_3 2024-09-19T08:56:05.454 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:56:05.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:05 smithi120 ceph-mon[31150]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T08:56:05.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:05 smithi120 ceph-mon[31150]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T08:56:05.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:05 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:56:05.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:05 smithi120 ceph-mon[31150]: from='osd.2 [v2:172.21.15.196:6800/2942286570,v1:172.21.15.196:6801/2942286570]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T08:56:05.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:05 smithi120 ceph-mon[31150]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T08:56:05.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:05 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:05.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:05 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:05.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:05 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:05.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:05 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:05.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:05 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:05.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:05 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:56:05.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:05 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:05.935 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 08:56:05 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-2[40117]: 2024-09-19T08:56:05.644+0000 7f9694997640 -1 osd.2 0 waiting for initial osdmap 2024-09-19T08:56:05.935 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 08:56:05 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-2[40117]: 2024-09-19T08:56:05.647+0000 7f96907bf640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:56:05.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:05 smithi196 ceph-mon[35150]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T08:56:05.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:05 smithi196 ceph-mon[35150]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T08:56:05.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:05 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:56:05.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:05 smithi196 ceph-mon[35150]: from='osd.2 [v2:172.21.15.196:6800/2942286570,v1:172.21.15.196:6801/2942286570]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T08:56:05.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:05 smithi196 ceph-mon[35150]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T08:56:05.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:05 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:05.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:05 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:05.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:05 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:05.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:05 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:05.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:05 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:05.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:05 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:56:05.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:05 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:05.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:05 smithi153 ceph-mon[35071]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T08:56:05.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:05 smithi153 ceph-mon[35071]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T08:56:05.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:05 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:56:05.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:05 smithi153 ceph-mon[35071]: from='osd.2 [v2:172.21.15.196:6800/2942286570,v1:172.21.15.196:6801/2942286570]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T08:56:05.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:05 smithi153 ceph-mon[35071]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T08:56:05.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:05 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:05.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:05 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:05.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:05 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:05.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:05 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:05.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:05 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:05.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:05 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:56:05.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:05 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:05.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.909+0000 7f4e594e1640 1 -- 172.21.15.120:0/791042161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e54101c00 msgr2=0x7f4e54102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:05.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.909+0000 7f4e594e1640 1 --2- 172.21.15.120:0/791042161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e54101c00 0x7f4e54102000 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009900 tx=0x7f4e4402a410 comp rx=0 tx=0).stop 2024-09-19T08:56:05.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.909+0000 7f4e594e1640 1 -- 172.21.15.120:0/791042161 shutdown_connections 2024-09-19T08:56:05.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.909+0000 7f4e594e1640 1 --2- 172.21.15.120:0/791042161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e541037c0 0x7f4e5410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:05.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.909+0000 7f4e594e1640 1 --2- 172.21.15.120:0/791042161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e54102e00 0x7f4e54103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:05.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.909+0000 7f4e594e1640 1 --2- 172.21.15.120:0/791042161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e54101c00 0x7f4e54102000 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:05.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.909+0000 7f4e594e1640 1 -- 172.21.15.120:0/791042161 >> 172.21.15.120:0/791042161 conn(0x7f4e540fd390 msgr2=0x7f4e540ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:05.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.909+0000 7f4e594e1640 1 -- 172.21.15.120:0/791042161 shutdown_connections 2024-09-19T08:56:05.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.909+0000 7f4e594e1640 1 -- 172.21.15.120:0/791042161 wait complete. 2024-09-19T08:56:05.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.910+0000 7f4e594e1640 1 Processor -- start 2024-09-19T08:56:05.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.910+0000 7f4e594e1640 1 -- start start 2024-09-19T08:56:05.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.910+0000 7f4e594e1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e54101c00 0x7f4e54197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:05.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.910+0000 7f4e594e1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e54102e00 0x7f4e541984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:05.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.910+0000 7f4e594e1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e541037c0 0x7f4e5419f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:05.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.910+0000 7f4e594e1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5410d8e0 con 0x7f4e541037c0 2024-09-19T08:56:05.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.910+0000 7f4e594e1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5410d760 con 0x7f4e54102e00 2024-09-19T08:56:05.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.910+0000 7f4e594e1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5410da60 con 0x7f4e54101c00 2024-09-19T08:56:05.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.911+0000 7f4e537fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e541037c0 0x7f4e5419f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:05.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.911+0000 7f4e537fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e541037c0 0x7f4e5419f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38924/0 (socket says 172.21.15.120:38924) 2024-09-19T08:56:05.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.911+0000 7f4e537fe640 1 -- 172.21.15.120:0/1775277768 learned_addr learned my addr 172.21.15.120:0/1775277768 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:56:05.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.911+0000 7f4e52ffd640 1 --2- 172.21.15.120:0/1775277768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e54101c00 0x7f4e54197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:05.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.911+0000 7f4e527fc640 1 --2- 172.21.15.120:0/1775277768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e54102e00 0x7f4e541984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:05.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.911+0000 7f4e537fe640 1 -- 172.21.15.120:0/1775277768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e54101c00 msgr2=0x7f4e54197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:05.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.911+0000 7f4e537fe640 1 --2- 172.21.15.120:0/1775277768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e54101c00 0x7f4e54197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:05.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.911+0000 7f4e537fe640 1 -- 172.21.15.120:0/1775277768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e54102e00 msgr2=0x7f4e541984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:05.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.911+0000 7f4e537fe640 1 --2- 172.21.15.120:0/1775277768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e54102e00 0x7f4e541984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:05.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.911+0000 7f4e537fe640 1 -- 172.21.15.120:0/1775277768 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5419fb90 con 0x7f4e541037c0 2024-09-19T08:56:05.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.911+0000 7f4e537fe640 1 --2- 172.21.15.120:0/1775277768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e541037c0 0x7f4e5419f520 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00b970 tx=0x7f4e4c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:05.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.911+0000 7f4e3bfff640 1 -- 172.21.15.120:0/1775277768 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c019070 con 0x7f4e541037c0 2024-09-19T08:56:05.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.911+0000 7f4e594e1640 1 -- 172.21.15.120:0/1775277768 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e5419fe80 con 0x7f4e541037c0 2024-09-19T08:56:05.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.911+0000 7f4e3bfff640 1 -- 172.21.15.120:0/1775277768 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e4c00cb40 con 0x7f4e541037c0 2024-09-19T08:56:05.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.911+0000 7f4e3bfff640 1 -- 172.21.15.120:0/1775277768 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c007650 con 0x7f4e541037c0 2024-09-19T08:56:05.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.911+0000 7f4e594e1640 1 -- 172.21.15.120:0/1775277768 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e541a03b0 con 0x7f4e541037c0 2024-09-19T08:56:05.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.913+0000 7f4e594e1640 1 -- 172.21.15.120:0/1775277768 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e54109020 con 0x7f4e541037c0 2024-09-19T08:56:05.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.913+0000 7f4e3bfff640 1 -- 172.21.15.120:0/1775277768 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f4e4c01c070 con 0x7f4e541037c0 2024-09-19T08:56:05.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.914+0000 7f4e3bfff640 1 --2- 172.21.15.120:0/1775277768 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4e3c0a31e0 0x7f4e3c0a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:05.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.914+0000 7f4e52ffd640 1 --2- 172.21.15.120:0/1775277768 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4e3c0a31e0 0x7f4e3c0a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:05.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.914+0000 7f4e3bfff640 1 -- 172.21.15.120:0/1775277768 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7f4e4c0cd6c0 con 0x7f4e541037c0 2024-09-19T08:56:05.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.914+0000 7f4e52ffd640 1 --2- 172.21.15.120:0/1775277768 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4e3c0a31e0 0x7f4e3c0a56a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4e44005ab0 tx=0x7f4e44000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:05.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:05.918+0000 7f4e3bfff640 1 -- 172.21.15.120:0/1775277768 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4e4c09b4e0 con 0x7f4e541037c0 2024-09-19T08:56:06.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:06.101+0000 7f4e594e1640 1 -- 172.21.15.120:0/1775277768 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4e54062630 con 0x7f4e3c0a31e0 2024-09-19T08:56:06.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:06 smithi120 ceph-mon[31150]: Detected new or changed devices on smithi196 2024-09-19T08:56:06.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:06 smithi120 ceph-mon[31150]: Adjusting osd_memory_target on smithi196 to 17088M 2024-09-19T08:56:06.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:06 smithi120 ceph-mon[31150]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:56:06.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:06 smithi120 ceph-mon[31150]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-19T08:56:06.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:06 smithi120 ceph-mon[31150]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T08:56:06.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:06 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:56:06.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:06 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:56:06.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:06 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:56:06.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:06 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:56:06.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:06 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:06.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:06 smithi196 ceph-mon[35150]: Detected new or changed devices on smithi196 2024-09-19T08:56:06.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:06 smithi196 ceph-mon[35150]: Adjusting osd_memory_target on smithi196 to 17088M 2024-09-19T08:56:06.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:06 smithi196 ceph-mon[35150]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:56:06.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:06 smithi196 ceph-mon[35150]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-19T08:56:06.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:06 smithi196 ceph-mon[35150]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T08:56:06.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:06 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:56:06.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:06 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:56:06.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:06 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:56:06.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:06 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:56:06.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:06 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:06.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:06 smithi153 ceph-mon[35071]: Detected new or changed devices on smithi196 2024-09-19T08:56:06.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:06 smithi153 ceph-mon[35071]: Adjusting osd_memory_target on smithi196 to 17088M 2024-09-19T08:56:06.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:06 smithi153 ceph-mon[35071]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:56:06.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:06 smithi153 ceph-mon[35071]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-19T08:56:06.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:06 smithi153 ceph-mon[35071]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T08:56:06.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:06 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:56:06.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:06 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:56:06.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:06 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:56:06.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:06 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:56:06.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:06 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:08.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:07 smithi120 ceph-mon[31150]: purged_snaps scrub starts 2024-09-19T08:56:08.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:07 smithi120 ceph-mon[31150]: purged_snaps scrub ok 2024-09-19T08:56:08.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:07 smithi120 ceph-mon[31150]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:56:08.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:07 smithi120 ceph-mon[31150]: osd.2 [v2:172.21.15.196:6800/2942286570,v1:172.21.15.196:6801/2942286570] boot 2024-09-19T08:56:08.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:07 smithi120 ceph-mon[31150]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T08:56:08.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:07 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:56:08.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:07 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T08:56:08.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:07 smithi196 ceph-mon[35150]: purged_snaps scrub starts 2024-09-19T08:56:08.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:07 smithi196 ceph-mon[35150]: purged_snaps scrub ok 2024-09-19T08:56:08.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:07 smithi196 ceph-mon[35150]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:56:08.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:07 smithi196 ceph-mon[35150]: osd.2 [v2:172.21.15.196:6800/2942286570,v1:172.21.15.196:6801/2942286570] boot 2024-09-19T08:56:08.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:07 smithi196 ceph-mon[35150]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T08:56:08.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:07 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:56:08.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:07 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T08:56:08.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:07 smithi153 ceph-mon[35071]: purged_snaps scrub starts 2024-09-19T08:56:08.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:07 smithi153 ceph-mon[35071]: purged_snaps scrub ok 2024-09-19T08:56:08.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:07 smithi153 ceph-mon[35071]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:56:08.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:07 smithi153 ceph-mon[35071]: osd.2 [v2:172.21.15.196:6800/2942286570,v1:172.21.15.196:6801/2942286570] boot 2024-09-19T08:56:08.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:07 smithi153 ceph-mon[35071]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T08:56:08.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:07 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:56:08.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:07 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T08:56:08.987 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 19 08:56:08 smithi120 sudo[49639]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T08:56:08.987 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 19 08:56:08 smithi120 sudo[49639]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:56:08.987 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 19 08:56:08 smithi120 sudo[49639]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:56:08.988 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 19 08:56:08 smithi120 sudo[49639]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:56:08.988 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:08 smithi120 ceph-mon[31150]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:56:08.988 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:08 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T08:56:08.988 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:08 smithi120 ceph-mon[31150]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T08:56:08.988 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:08 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T08:56:08.988 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:08 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/605897759' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1d9e507-8f02-403e-87da-9b4966fd8fab"}]: dispatch 2024-09-19T08:56:08.988 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:08 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T08:56:08.988 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:08 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/605897759' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1d9e507-8f02-403e-87da-9b4966fd8fab"}]': finished 2024-09-19T08:56:08.988 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:08 smithi120 ceph-mon[31150]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T08:56:08.988 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:08 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:56:09.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:08 smithi196 ceph-mon[35150]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:56:09.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:08 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T08:56:09.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:08 smithi196 ceph-mon[35150]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T08:56:09.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:08 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T08:56:09.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:08 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/605897759' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1d9e507-8f02-403e-87da-9b4966fd8fab"}]: dispatch 2024-09-19T08:56:09.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:08 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T08:56:09.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:08 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/605897759' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1d9e507-8f02-403e-87da-9b4966fd8fab"}]': finished 2024-09-19T08:56:09.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:08 smithi196 ceph-mon[35150]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T08:56:09.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:08 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:56:09.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:08 smithi153 ceph-mon[35071]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:56:09.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:08 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T08:56:09.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:08 smithi153 ceph-mon[35071]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T08:56:09.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:08 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T08:56:09.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:08 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/605897759' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1d9e507-8f02-403e-87da-9b4966fd8fab"}]: dispatch 2024-09-19T08:56:09.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:08 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T08:56:09.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:08 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/605897759' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1d9e507-8f02-403e-87da-9b4966fd8fab"}]': finished 2024-09-19T08:56:09.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:08 smithi153 ceph-mon[35071]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T08:56:09.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:08 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:56:09.324 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 19 08:56:09 smithi120 sudo[49645]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T08:56:09.324 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 19 08:56:09 smithi120 sudo[49645]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:56:09.324 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 19 08:56:09 smithi120 sudo[49645]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:56:09.324 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 19 08:56:09 smithi120 sudo[49645]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:56:09.550 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 19 08:56:09 smithi153 sudo[43229]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T08:56:09.550 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 19 08:56:09 smithi153 sudo[43229]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:56:09.550 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 19 08:56:09 smithi153 sudo[43229]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:56:09.550 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 19 08:56:09 smithi153 sudo[43229]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:56:09.838 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 19 08:56:09 smithi153 sudo[43232]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T08:56:09.839 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 19 08:56:09 smithi153 sudo[43232]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:56:09.839 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 19 08:56:09 smithi153 sudo[43232]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:56:09.839 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 19 08:56:09 smithi153 sudo[43232]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:56:10.073 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:09 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1093668680' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:56:10.073 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:09 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:10.073 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:09 smithi196 ceph-mon[35150]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T08:56:10.073 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:09 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:56:10.073 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:09 smithi196 ceph-mon[35150]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:10.073 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:09 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:10.074 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 08:56:09 smithi196 sudo[43733]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T08:56:10.074 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 08:56:09 smithi196 sudo[43733]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:56:10.074 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 08:56:09 smithi196 sudo[43733]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:56:10.074 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 08:56:09 smithi196 sudo[43733]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:56:10.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:09 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1093668680' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:56:10.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:09 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:10.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:09 smithi153 ceph-mon[35071]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T08:56:10.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:09 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:56:10.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:09 smithi153 ceph-mon[35071]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:10.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:09 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:10.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:09 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1093668680' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:56:10.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:09 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:10.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:09 smithi120 ceph-mon[31150]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T08:56:10.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:09 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:56:10.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:09 smithi120 ceph-mon[31150]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:10.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:09 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:10.435 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 08:56:10 smithi196 sudo[43736]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T08:56:10.435 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 08:56:10 smithi196 sudo[43736]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:56:10.436 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 08:56:10 smithi196 sudo[43736]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:56:10.436 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 08:56:10 smithi196 sudo[43736]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:56:10.603 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:10 smithi120 sudo[49877]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T08:56:10.603 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:10 smithi120 sudo[49877]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:56:10.603 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:10 smithi120 sudo[49877]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:56:10.603 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:10 smithi120 sudo[49877]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:56:10.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:10.620+0000 7f4e3bfff640 1 -- 172.21.15.120:0/1775277768 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f4e4c00ccf0 con 0x7f4e541037c0 2024-09-19T08:56:11.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:10 smithi120 sudo[50063]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T08:56:11.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:10 smithi120 sudo[50063]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:56:11.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:10 smithi120 sudo[50063]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:56:11.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:10 smithi120 sudo[50063]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:56:11.108 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:10 smithi196 sudo[43739]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T08:56:11.108 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:10 smithi196 sudo[43739]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:56:11.108 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:10 smithi196 sudo[43739]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:56:11.109 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:10 smithi196 sudo[43739]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:56:11.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:11 smithi196 sudo[43743]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T08:56:11.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:11 smithi196 sudo[43743]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:56:11.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:11 smithi196 sudo[43743]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:56:11.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:11 smithi196 sudo[43743]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:56:11.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:11 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:11.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:11 smithi196 ceph-mon[35150]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T08:56:11.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:11 smithi196 ceph-mon[35150]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T08:56:11.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:11 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:56:11.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:11 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:56:11.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:11 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:56:11.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:11 smithi196 ceph-mon[35150]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T08:56:11.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:11 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:56:11.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:11 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:56:11.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:11 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:56:11.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:11 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:11.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:11 smithi120 ceph-mon[31150]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T08:56:11.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:11 smithi120 ceph-mon[31150]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T08:56:11.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:11 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:56:11.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:11 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:56:11.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:11 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:56:11.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:11 smithi120 ceph-mon[31150]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T08:56:11.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:11 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:56:11.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:11 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:56:11.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:11 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:56:11.611 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:11 smithi153 sudo[43235]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T08:56:11.612 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:11 smithi153 sudo[43235]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:56:11.612 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:11 smithi153 sudo[43235]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:56:11.612 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:11 smithi153 sudo[43235]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:56:11.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:11 smithi153 sudo[43238]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T08:56:11.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:11 smithi153 sudo[43238]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:56:11.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:11 smithi153 sudo[43238]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:56:11.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:11 smithi153 sudo[43238]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:56:11.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:11 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:11.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:11 smithi153 ceph-mon[35071]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T08:56:11.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:11 smithi153 ceph-mon[35071]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T08:56:11.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:11 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:56:11.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:11 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:56:11.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:11 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:56:11.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:11 smithi153 ceph-mon[35071]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T08:56:11.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:11 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:56:11.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:11 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:56:11.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:11 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:56:12.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:12 smithi196 ceph-mon[35150]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T08:56:12.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:12 smithi196 ceph-mon[35150]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T08:56:12.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:12 smithi196 ceph-mon[35150]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T08:56:12.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:12 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:56:12.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:12 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:56:12.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:12 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:56:12.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:12 smithi196 ceph-mon[35150]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:12.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:12 smithi196 ceph-mon[35150]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T08:56:12.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:12 smithi153 ceph-mon[35071]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T08:56:12.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:12 smithi153 ceph-mon[35071]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T08:56:12.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:12 smithi153 ceph-mon[35071]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T08:56:12.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:12 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:56:12.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:12 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:56:12.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:12 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:56:12.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:12 smithi153 ceph-mon[35071]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:12.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:12 smithi153 ceph-mon[35071]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T08:56:12.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:12 smithi120 ceph-mon[31150]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T08:56:12.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:12 smithi120 ceph-mon[31150]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T08:56:12.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:12 smithi120 ceph-mon[31150]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T08:56:12.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:56:12.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:56:12.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:56:12.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:12 smithi120 ceph-mon[31150]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:12.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:12 smithi120 ceph-mon[31150]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T08:56:14.894 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:14 smithi120 ceph-mon[31150]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:14.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:14 smithi196 ceph-mon[35150]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:14.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:14 smithi153 ceph-mon[35071]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:16.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:16 smithi120 ceph-mon[31150]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:16.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:16 smithi196 ceph-mon[35150]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:16.940 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:16 smithi153 ceph-mon[35071]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:17.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:17 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T08:56:17.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:17 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:17.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:17 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T08:56:17.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:17 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:18.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:17 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T08:56:18.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:17 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:18.807 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:18 smithi120 ceph-mon[31150]: Deploying daemon osd.3 on smithi120 2024-09-19T08:56:18.808 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:18 smithi120 ceph-mon[31150]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:18.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:18 smithi196 ceph-mon[35150]: Deploying daemon osd.3 on smithi120 2024-09-19T08:56:18.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:18 smithi196 ceph-mon[35150]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:18.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:18 smithi153 ceph-mon[35071]: Deploying daemon osd.3 on smithi120 2024-09-19T08:56:18.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:18 smithi153 ceph-mon[35071]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:20.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:20 smithi120 ceph-mon[31150]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:20.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:20 smithi196 ceph-mon[35150]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:20.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:20 smithi153 ceph-mon[35071]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:22.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:22 smithi120 ceph-mon[31150]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:22.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:22 smithi196 ceph-mon[35150]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:22.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:22 smithi153 ceph-mon[35071]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:23.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:23 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:56:23.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:23 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:23.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:23 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:23.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:23 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:56:23.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:23 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:23.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:23 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:23.936 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:23 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:56:23.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:23 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:23.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:23 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:24.913 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:24 smithi120 ceph-mon[31150]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:24.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:24 smithi196 ceph-mon[35150]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:24.936 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:24 smithi153 ceph-mon[35071]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:26.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:26 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:26.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:26 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:26.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:26 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:26.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:26 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:56:26.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:26 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:26.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:26 smithi120 ceph-mon[31150]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:26.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:26 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:26.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:26 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:26.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:26 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:26.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:26 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:56:26.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:26 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:26.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:26 smithi196 ceph-mon[35150]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:26.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:26 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:26.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:26 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:26.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:26 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:26.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:26 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:56:26.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:26 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:26.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:26 smithi153 ceph-mon[35071]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:27.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:27.175+0000 7f4e3bfff640 1 -- 172.21.15.120:0/1775277768 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4e54062630 con 0x7f4e3c0a31e0 2024-09-19T08:56:27.175 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 3 on host 'smithi120' 2024-09-19T08:56:27.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:27.177+0000 7f4e594e1640 1 -- 172.21.15.120:0/1775277768 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4e3c0a31e0 msgr2=0x7f4e3c0a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:27.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:27.177+0000 7f4e594e1640 1 --2- 172.21.15.120:0/1775277768 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4e3c0a31e0 0x7f4e3c0a56a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4e44005ab0 tx=0x7f4e44000bc0 comp rx=0 tx=0).stop 2024-09-19T08:56:27.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:27.177+0000 7f4e594e1640 1 -- 172.21.15.120:0/1775277768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e541037c0 msgr2=0x7f4e5419f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:27.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:27.177+0000 7f4e594e1640 1 --2- 172.21.15.120:0/1775277768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e541037c0 0x7f4e5419f520 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00b970 tx=0x7f4e4c00be30 comp rx=0 tx=0).stop 2024-09-19T08:56:27.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:27.177+0000 7f4e594e1640 1 -- 172.21.15.120:0/1775277768 shutdown_connections 2024-09-19T08:56:27.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:27.177+0000 7f4e594e1640 1 --2- 172.21.15.120:0/1775277768 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4e3c0a31e0 0x7f4e3c0a56a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:27.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:27.177+0000 7f4e594e1640 1 --2- 172.21.15.120:0/1775277768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e541037c0 0x7f4e5419f520 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:27.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:27.177+0000 7f4e594e1640 1 --2- 172.21.15.120:0/1775277768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e54102e00 0x7f4e541984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:27.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:27.177+0000 7f4e594e1640 1 --2- 172.21.15.120:0/1775277768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e54101c00 0x7f4e54197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:27.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:27.177+0000 7f4e594e1640 1 -- 172.21.15.120:0/1775277768 >> 172.21.15.120:0/1775277768 conn(0x7f4e540fd390 msgr2=0x7f4e541061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:27.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:27.177+0000 7f4e594e1640 1 -- 172.21.15.120:0/1775277768 shutdown_connections 2024-09-19T08:56:27.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:56:27.177+0000 7f4e594e1640 1 -- 172.21.15.120:0/1775277768 wait complete. 2024-09-19T08:56:27.524 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:27 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:56:27.525 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:27 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:27.525 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:27 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:27.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:27 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:56:27.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:27 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:27.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:27 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:27.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:27 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:56:27.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:27 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:27.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:27 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:27.860 DEBUG:teuthology.orchestra.run.smithi120:osd.3> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.3.service 2024-09-19T08:56:27.862 INFO:tasks.cephadm:Deploying osd.4 on smithi153 with /dev/vg_nvme/lv_3... 2024-09-19T08:56:27.863 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T08:56:28.113 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:56:28.568 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:28 smithi153 ceph-mon[35071]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:28.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:28 smithi120 ceph-mon[31150]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:28.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:28 smithi196 ceph-mon[35150]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:29.021 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 19 08:56:28 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-3[51657]: 2024-09-19T08:56:28.693+0000 7fec73ace640 -1 osd.3 0 log_to_monitors true 2024-09-19T08:56:29.543 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:29 smithi153 ceph-mon[35071]: from='osd.3 [v2:172.21.15.120:6810/713205668,v1:172.21.15.120:6811/713205668]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T08:56:29.571 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:29 smithi120 ceph-mon[31150]: from='osd.3 [v2:172.21.15.120:6810/713205668,v1:172.21.15.120:6811/713205668]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T08:56:29.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:29 smithi196 ceph-mon[35150]: from='osd.3 [v2:172.21.15.120:6810/713205668,v1:172.21.15.120:6811/713205668]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T08:56:30.485 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:30 smithi153 ceph-mon[35071]: from='osd.3 [v2:172.21.15.120:6810/713205668,v1:172.21.15.120:6811/713205668]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T08:56:30.485 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:30 smithi153 ceph-mon[35071]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T08:56:30.485 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:30 smithi153 ceph-mon[35071]: from='osd.3 [v2:172.21.15.120:6810/713205668,v1:172.21.15.120:6811/713205668]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-19T08:56:30.485 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:30 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:56:30.485 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:30 smithi153 ceph-mon[35071]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:30.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:30 smithi196 ceph-mon[35150]: from='osd.3 [v2:172.21.15.120:6810/713205668,v1:172.21.15.120:6811/713205668]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T08:56:30.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:30 smithi196 ceph-mon[35150]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T08:56:30.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:30 smithi196 ceph-mon[35150]: from='osd.3 [v2:172.21.15.120:6810/713205668,v1:172.21.15.120:6811/713205668]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-19T08:56:30.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:30 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:56:30.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:30 smithi196 ceph-mon[35150]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:30.641 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-19T08:56:30.668 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi153:vg_nvme/lv_3 2024-09-19T08:56:30.742 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 19 08:56:30 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-3[51657]: 2024-09-19T08:56:30.348+0000 7fec6fa51640 -1 osd.3 0 waiting for initial osdmap 2024-09-19T08:56:30.742 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 19 08:56:30 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-3[51657]: 2024-09-19T08:56:30.354+0000 7fec6b879640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:56:30.742 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:30 smithi120 ceph-mon[31150]: from='osd.3 [v2:172.21.15.120:6810/713205668,v1:172.21.15.120:6811/713205668]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T08:56:30.742 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:30 smithi120 ceph-mon[31150]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T08:56:30.743 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:30 smithi120 ceph-mon[31150]: from='osd.3 [v2:172.21.15.120:6810/713205668,v1:172.21.15.120:6811/713205668]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-19T08:56:30.743 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:30 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:56:30.743 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:30 smithi120 ceph-mon[31150]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:56:30.914 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:56:31.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.368+0000 7fae3a37f640 1 -- 172.21.15.153:0/2564656938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae34101cd0 msgr2=0x7fae34102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:31.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.368+0000 7fae3a37f640 1 --2- 172.21.15.153:0/2564656938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae34101cd0 0x7fae34102150 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fae28009900 tx=0x7fae2802a410 comp rx=0 tx=0).stop 2024-09-19T08:56:31.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.368+0000 7fae3a37f640 1 -- 172.21.15.153:0/2564656938 shutdown_connections 2024-09-19T08:56:31.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.368+0000 7fae3a37f640 1 --2- 172.21.15.153:0/2564656938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae34102690 0x7fae34108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.368+0000 7fae3a37f640 1 --2- 172.21.15.153:0/2564656938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae34101cd0 0x7fae34102150 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.368+0000 7fae3a37f640 1 --2- 172.21.15.153:0/2564656938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae34100ad0 0x7fae34100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.368+0000 7fae3a37f640 1 -- 172.21.15.153:0/2564656938 >> 172.21.15.153:0/2564656938 conn(0x7fae340fc280 msgr2=0x7fae340fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:31.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.368+0000 7fae3a37f640 1 -- 172.21.15.153:0/2564656938 shutdown_connections 2024-09-19T08:56:31.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.368+0000 7fae3a37f640 1 -- 172.21.15.153:0/2564656938 wait complete. 2024-09-19T08:56:31.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.368+0000 7fae3a37f640 1 Processor -- start 2024-09-19T08:56:31.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.368+0000 7fae3a37f640 1 -- start start 2024-09-19T08:56:31.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.369+0000 7fae3a37f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae34100ad0 0x7fae34197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:31.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.369+0000 7fae3a37f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae34101cd0 0x7fae34198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:31.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.369+0000 7fae3a37f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae34102690 0x7fae3419f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:31.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.369+0000 7fae3a37f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fae3410c750 con 0x7fae34101cd0 2024-09-19T08:56:31.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.369+0000 7fae3a37f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fae3410c5d0 con 0x7fae34102690 2024-09-19T08:56:31.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.369+0000 7fae3a37f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae3410c8d0 con 0x7fae34100ad0 2024-09-19T08:56:31.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.369+0000 7fae33fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae34100ad0 0x7fae34197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:31.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.369+0000 7fae3891c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae34102690 0x7fae3419f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:31.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.369+0000 7fae33fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae34100ad0 0x7fae34197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.153:37090/0 (socket says 172.21.15.153:37090) 2024-09-19T08:56:31.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.369+0000 7fae33fff640 1 -- 172.21.15.153:0/1434476355 learned_addr learned my addr 172.21.15.153:0/1434476355 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-19T08:56:31.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.369+0000 7fae337fe640 1 --2- 172.21.15.153:0/1434476355 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae34101cd0 0x7fae34198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:31.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.369+0000 7fae3891c640 1 -- 172.21.15.153:0/1434476355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae34100ad0 msgr2=0x7fae34197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:31.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.369+0000 7fae3891c640 1 --2- 172.21.15.153:0/1434476355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae34100ad0 0x7fae34197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.369+0000 7fae3891c640 1 -- 172.21.15.153:0/1434476355 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae34101cd0 msgr2=0x7fae34198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:31.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.369+0000 7fae3891c640 1 --2- 172.21.15.153:0/1434476355 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae34101cd0 0x7fae34198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.369+0000 7fae3891c640 1 -- 172.21.15.153:0/1434476355 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae3419fb30 con 0x7fae34102690 2024-09-19T08:56:31.394 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.369+0000 7fae33fff640 1 --2- 172.21.15.153:0/1434476355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae34100ad0 0x7fae34197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:56:31.394 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.370+0000 7fae3891c640 1 --2- 172.21.15.153:0/1434476355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae34102690 0x7fae3419f4c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fae2c009fd0 tx=0x7fae2c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:31.394 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.370+0000 7fae317fa640 1 -- 172.21.15.153:0/1434476355 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c00ce20 con 0x7fae34102690 2024-09-19T08:56:31.394 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.370+0000 7fae317fa640 1 -- 172.21.15.153:0/1434476355 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae2c005020 con 0x7fae34102690 2024-09-19T08:56:31.394 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.370+0000 7fae3a37f640 1 -- 172.21.15.153:0/1434476355 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae3419fe20 con 0x7fae34102690 2024-09-19T08:56:31.394 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.370+0000 7fae317fa640 1 -- 172.21.15.153:0/1434476355 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c010720 con 0x7fae34102690 2024-09-19T08:56:31.394 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.370+0000 7fae3a37f640 1 -- 172.21.15.153:0/1434476355 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae341a02d0 con 0x7fae34102690 2024-09-19T08:56:31.394 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.371+0000 7fae3a37f640 1 -- 172.21.15.153:0/1434476355 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae341a0560 con 0x7fae34102690 2024-09-19T08:56:31.394 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.372+0000 7fae317fa640 1 -- 172.21.15.153:0/1434476355 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fae2c005540 con 0x7fae34102690 2024-09-19T08:56:31.394 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.373+0000 7fae317fa640 1 --2- 172.21.15.153:0/1434476355 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fae1c0a3270 0x7fae1c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:31.394 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.373+0000 7fae317fa640 1 -- 172.21.15.153:0/1434476355 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 3702+0+0 (secure 0 0 0) 0x7fae2c0cecd0 con 0x7fae34102690 2024-09-19T08:56:31.394 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.373+0000 7fae33fff640 1 --2- 172.21.15.153:0/1434476355 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fae1c0a3270 0x7fae1c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:31.394 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.373+0000 7fae33fff640 1 --2- 172.21.15.153:0/1434476355 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fae1c0a3270 0x7fae1c0a5730 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fae240104e0 tx=0x7fae2400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:31.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.377+0000 7fae317fa640 1 -- 172.21.15.153:0/1434476355 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fae2c09cbf0 con 0x7fae34102690 2024-09-19T08:56:31.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:31.554+0000 7fae3a37f640 1 -- 172.21.15.153:0/1434476355 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fae341a0730 con 0x7fae1c0a3270 2024-09-19T08:56:31.554 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:31 smithi153 ceph-mon[35071]: from='osd.3 [v2:172.21.15.120:6810/713205668,v1:172.21.15.120:6811/713205668]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-19T08:56:31.554 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:31 smithi153 ceph-mon[35071]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T08:56:31.554 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:31 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:56:31.554 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:31 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:56:31.554 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:31 smithi153 ceph-mon[35071]: Detected new or changed devices on smithi120 2024-09-19T08:56:31.554 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:31 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:31.554 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:31 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:31.554 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:31 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:31.554 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:31 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:31.554 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:31 smithi153 ceph-mon[35071]: Adjusting osd_memory_target on smithi120 to 8544M 2024-09-19T08:56:31.554 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:31 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:31.555 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:31 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:31.555 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:31 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:56:31.555 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:31 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:31.555 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:31 smithi153 ceph-mon[35071]: osd.3 [v2:172.21.15.120:6810/713205668,v1:172.21.15.120:6811/713205668] boot 2024-09-19T08:56:31.555 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:31 smithi153 ceph-mon[35071]: osdmap e24: 4 total, 4 up, 4 in 2024-09-19T08:56:31.555 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:31 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:56:31.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:31 smithi196 ceph-mon[35150]: from='osd.3 [v2:172.21.15.120:6810/713205668,v1:172.21.15.120:6811/713205668]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-19T08:56:31.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:31 smithi196 ceph-mon[35150]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T08:56:31.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:31 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:56:31.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:31 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:56:31.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:31 smithi196 ceph-mon[35150]: Detected new or changed devices on smithi120 2024-09-19T08:56:31.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:31 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:31.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:31 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:31.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:31 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:31.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:31 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:31.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:31 smithi196 ceph-mon[35150]: Adjusting osd_memory_target on smithi120 to 8544M 2024-09-19T08:56:31.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:31 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:31.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:31 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:31.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:31 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:56:31.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:31 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:31.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:31 smithi196 ceph-mon[35150]: osd.3 [v2:172.21.15.120:6810/713205668,v1:172.21.15.120:6811/713205668] boot 2024-09-19T08:56:31.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:31 smithi196 ceph-mon[35150]: osdmap e24: 4 total, 4 up, 4 in 2024-09-19T08:56:31.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:31 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:56:31.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:31 smithi120 ceph-mon[31150]: from='osd.3 [v2:172.21.15.120:6810/713205668,v1:172.21.15.120:6811/713205668]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-19T08:56:31.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:31 smithi120 ceph-mon[31150]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T08:56:31.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:31 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:56:31.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:31 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:56:31.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:31 smithi120 ceph-mon[31150]: Detected new or changed devices on smithi120 2024-09-19T08:56:31.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:31 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:31.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:31 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:31.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:31 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:31.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:31 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:31.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:31 smithi120 ceph-mon[31150]: Adjusting osd_memory_target on smithi120 to 8544M 2024-09-19T08:56:31.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:31 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:31.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:31 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:31.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:31 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:56:31.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:31 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:31.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:31 smithi120 ceph-mon[31150]: osd.3 [v2:172.21.15.120:6810/713205668,v1:172.21.15.120:6811/713205668] boot 2024-09-19T08:56:31.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:31 smithi120 ceph-mon[31150]: osdmap e24: 4 total, 4 up, 4 in 2024-09-19T08:56:31.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:31 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:56:32.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:32 smithi196 ceph-mon[35150]: purged_snaps scrub starts 2024-09-19T08:56:32.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:32 smithi196 ceph-mon[35150]: purged_snaps scrub ok 2024-09-19T08:56:32.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:32 smithi196 ceph-mon[35150]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:56:32.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:32 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:56:32.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:32 smithi196 ceph-mon[35150]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:56:32.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:32 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:56:32.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:32 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:32.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:32 smithi196 ceph-mon[35150]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T08:56:32.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:32 smithi153 ceph-mon[35071]: purged_snaps scrub starts 2024-09-19T08:56:32.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:32 smithi153 ceph-mon[35071]: purged_snaps scrub ok 2024-09-19T08:56:32.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:32 smithi153 ceph-mon[35071]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:56:32.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:32 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:56:32.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:32 smithi153 ceph-mon[35071]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:56:32.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:32 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:56:32.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:32 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:32.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:32 smithi153 ceph-mon[35071]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T08:56:32.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:32 smithi120 ceph-mon[31150]: purged_snaps scrub starts 2024-09-19T08:56:32.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:32 smithi120 ceph-mon[31150]: purged_snaps scrub ok 2024-09-19T08:56:32.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:32 smithi120 ceph-mon[31150]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:56:32.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:32 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:56:32.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:32 smithi120 ceph-mon[31150]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:56:32.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:32 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:56:32.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:32 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:32.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:32 smithi120 ceph-mon[31150]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T08:56:34.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:34 smithi153 ceph-mon[35071]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T08:56:34.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:34 smithi153 ceph-mon[35071]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:56:34.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:34 smithi153 ceph-mon[35071]: from='client.? 172.21.15.153:0/2877776612' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f8bf3b9-0ec4-46f6-a406-d26d7971179e"}]: dispatch 2024-09-19T08:56:34.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:34 smithi153 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f8bf3b9-0ec4-46f6-a406-d26d7971179e"}]: dispatch 2024-09-19T08:56:34.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:34 smithi153 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7f8bf3b9-0ec4-46f6-a406-d26d7971179e"}]': finished 2024-09-19T08:56:34.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:34 smithi153 ceph-mon[35071]: osdmap e27: 5 total, 4 up, 5 in 2024-09-19T08:56:34.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:34 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:56:34.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:34 smithi196 ceph-mon[35150]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T08:56:34.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:34 smithi196 ceph-mon[35150]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:56:34.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:34 smithi196 ceph-mon[35150]: from='client.? 172.21.15.153:0/2877776612' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f8bf3b9-0ec4-46f6-a406-d26d7971179e"}]: dispatch 2024-09-19T08:56:34.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:34 smithi196 ceph-mon[35150]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f8bf3b9-0ec4-46f6-a406-d26d7971179e"}]: dispatch 2024-09-19T08:56:34.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:34 smithi196 ceph-mon[35150]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7f8bf3b9-0ec4-46f6-a406-d26d7971179e"}]': finished 2024-09-19T08:56:34.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:34 smithi196 ceph-mon[35150]: osdmap e27: 5 total, 4 up, 5 in 2024-09-19T08:56:34.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:34 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:56:34.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:34 smithi120 ceph-mon[31150]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T08:56:34.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:34 smithi120 ceph-mon[31150]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:56:34.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:34 smithi120 ceph-mon[31150]: from='client.? 172.21.15.153:0/2877776612' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f8bf3b9-0ec4-46f6-a406-d26d7971179e"}]: dispatch 2024-09-19T08:56:34.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:34 smithi120 ceph-mon[31150]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f8bf3b9-0ec4-46f6-a406-d26d7971179e"}]: dispatch 2024-09-19T08:56:34.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:34 smithi120 ceph-mon[31150]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7f8bf3b9-0ec4-46f6-a406-d26d7971179e"}]': finished 2024-09-19T08:56:34.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:34 smithi120 ceph-mon[31150]: osdmap e27: 5 total, 4 up, 5 in 2024-09-19T08:56:34.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:34 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:56:35.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:35 smithi196 ceph-mon[35150]: from='client.? 172.21.15.153:0/414980848' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:56:35.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:35 smithi196 ceph-mon[35150]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T08:56:35.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:35 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:56:35.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:35 smithi153 ceph-mon[35071]: from='client.? 172.21.15.153:0/414980848' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:56:35.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:35 smithi153 ceph-mon[35071]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T08:56:35.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:35 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:56:35.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:35 smithi120 ceph-mon[31150]: from='client.? 172.21.15.153:0/414980848' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:56:35.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:35 smithi120 ceph-mon[31150]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T08:56:35.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:35 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:56:36.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:36 smithi196 ceph-mon[35150]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:56:36.686 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:36 smithi153 ceph-mon[35071]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:56:36.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:36 smithi120 ceph-mon[31150]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:56:38.774 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:38 smithi153 ceph-mon[35071]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:56:38.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:38 smithi196 ceph-mon[35150]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:56:39.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:38 smithi120 ceph-mon[31150]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:56:40.903 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:40 smithi153 ceph-mon[35071]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-19T08:56:40.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:40 smithi196 ceph-mon[35150]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-19T08:56:41.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:40 smithi120 ceph-mon[31150]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-19T08:56:42.879 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:42 smithi153 ceph-mon[35071]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T08:56:42.906 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:42 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T08:56:42.907 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:42 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:42.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:42 smithi196 ceph-mon[35150]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T08:56:42.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:42 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T08:56:42.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:42 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:43.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:42 smithi120 ceph-mon[31150]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T08:56:43.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:42 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T08:56:43.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:42 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:43.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:43 smithi196 ceph-mon[35150]: Deploying daemon osd.4 on smithi153 2024-09-19T08:56:43.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:43 smithi153 ceph-mon[35071]: Deploying daemon osd.4 on smithi153 2024-09-19T08:56:44.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:43 smithi120 ceph-mon[31150]: Deploying daemon osd.4 on smithi153 2024-09-19T08:56:44.881 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:44 smithi153 ceph-mon[35071]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-19T08:56:44.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:44 smithi196 ceph-mon[35150]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-19T08:56:45.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:44 smithi120 ceph-mon[31150]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-19T08:56:46.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:46 smithi196 ceph-mon[35150]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-19T08:56:46.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:46 smithi153 ceph-mon[35071]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-19T08:56:47.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:46 smithi120 ceph-mon[31150]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-19T08:56:48.888 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:48 smithi120 ceph-mon[31150]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T08:56:48.888 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:48 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:56:48.888 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:48 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:48.888 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:48 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:48.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:48 smithi196 ceph-mon[35150]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T08:56:48.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:48 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:56:48.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:48 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:48.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:48 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:48.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:48 smithi153 ceph-mon[35071]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T08:56:48.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:48 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:56:48.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:48 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:48.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:48 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:50.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:50 smithi196 ceph-mon[35150]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T08:56:50.936 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:50 smithi153 ceph-mon[35071]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T08:56:51.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:50 smithi120 ceph-mon[31150]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T08:56:52.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:52.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:52.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:52.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:56:52.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:52.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:51 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:52.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:51 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:52.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:51 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:52.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:51 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:56:52.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:51 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:52.186 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:51 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:52.186 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:51 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:52.186 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:51 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:52.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:51 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:56:52.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:51 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:52.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:52.541+0000 7fae317fa640 1 -- 172.21.15.153:0/1434476355 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fae341a0730 con 0x7fae1c0a3270 2024-09-19T08:56:52.541 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 4 on host 'smithi153' 2024-09-19T08:56:52.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:52.543+0000 7fae3a37f640 1 -- 172.21.15.153:0/1434476355 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fae1c0a3270 msgr2=0x7fae1c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:52.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:52.543+0000 7fae3a37f640 1 --2- 172.21.15.153:0/1434476355 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fae1c0a3270 0x7fae1c0a5730 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fae240104e0 tx=0x7fae2400b410 comp rx=0 tx=0).stop 2024-09-19T08:56:52.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:52.543+0000 7fae3a37f640 1 -- 172.21.15.153:0/1434476355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae34102690 msgr2=0x7fae3419f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:52.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:52.543+0000 7fae3a37f640 1 --2- 172.21.15.153:0/1434476355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae34102690 0x7fae3419f4c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fae2c009fd0 tx=0x7fae2c00ef10 comp rx=0 tx=0).stop 2024-09-19T08:56:52.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:52.543+0000 7fae3a37f640 1 -- 172.21.15.153:0/1434476355 shutdown_connections 2024-09-19T08:56:52.544 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:52.543+0000 7fae3a37f640 1 --2- 172.21.15.153:0/1434476355 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fae1c0a3270 0x7fae1c0a5730 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:52.544 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:52.543+0000 7fae3a37f640 1 --2- 172.21.15.153:0/1434476355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae34102690 0x7fae3419f4c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:52.544 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:52.543+0000 7fae3a37f640 1 --2- 172.21.15.153:0/1434476355 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae34101cd0 0x7fae34198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:52.544 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:52.543+0000 7fae3a37f640 1 --2- 172.21.15.153:0/1434476355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae34100ad0 0x7fae34197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:52.544 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:52.543+0000 7fae3a37f640 1 -- 172.21.15.153:0/1434476355 >> 172.21.15.153:0/1434476355 conn(0x7fae340fc280 msgr2=0x7fae34069fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:52.544 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:52.543+0000 7fae3a37f640 1 -- 172.21.15.153:0/1434476355 shutdown_connections 2024-09-19T08:56:52.544 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:56:52.543+0000 7fae3a37f640 1 -- 172.21.15.153:0/1434476355 wait complete. 2024-09-19T08:56:53.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:52 smithi120 ceph-mon[31150]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:56:53.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:52 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:56:53.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:52 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:53.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:52 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:53.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:52 smithi196 ceph-mon[35150]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:56:53.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:52 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:56:53.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:52 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:53.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:52 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:53.186 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:52 smithi153 ceph-mon[35071]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:56:53.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:52 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:56:53.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:52 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:53.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:52 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:53.215 DEBUG:teuthology.orchestra.run.smithi153:osd.4> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.4.service 2024-09-19T08:56:53.218 INFO:tasks.cephadm:Deploying osd.5 on smithi196 with /dev/vg_nvme/lv_3... 2024-09-19T08:56:53.218 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T08:56:53.471 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:56:54.654 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 19 08:56:54 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-4[45982]: 2024-09-19T08:56:54.606+0000 7f2dae5fb640 -1 osd.4 0 log_to_monitors true 2024-09-19T08:56:54.821 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:54 smithi196 ceph-mon[35150]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:56:54.821 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:54 smithi196 ceph-mon[35150]: from='osd.4 [v2:172.21.15.153:6808/2680865854,v1:172.21.15.153:6809/2680865854]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T08:56:54.822 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:54 smithi196 ceph-mon[35150]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T08:56:54.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:54 smithi153 ceph-mon[35071]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:56:54.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:54 smithi153 ceph-mon[35071]: from='osd.4 [v2:172.21.15.153:6808/2680865854,v1:172.21.15.153:6809/2680865854]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T08:56:54.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:54 smithi153 ceph-mon[35071]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T08:56:55.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:54 smithi120 ceph-mon[31150]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:56:55.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:54 smithi120 ceph-mon[31150]: from='osd.4 [v2:172.21.15.153:6808/2680865854,v1:172.21.15.153:6809/2680865854]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T08:56:55.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:54 smithi120 ceph-mon[31150]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T08:56:55.937 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 19 08:56:55 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-4[45982]: 2024-09-19T08:56:55.794+0000 7f2daa57e640 -1 osd.4 0 waiting for initial osdmap 2024-09-19T08:56:55.937 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 19 08:56:55 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-4[45982]: 2024-09-19T08:56:55.800+0000 7f2da63a6640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:56:55.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:55 smithi153 ceph-mon[35071]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T08:56:55.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:55 smithi153 ceph-mon[35071]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T08:56:55.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:55 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:56:55.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:55 smithi153 ceph-mon[35071]: from='osd.4 [v2:172.21.15.153:6808/2680865854,v1:172.21.15.153:6809/2680865854]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-19T08:56:55.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:55 smithi153 ceph-mon[35071]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-19T08:56:55.954 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T08:56:55.981 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_3 2024-09-19T08:56:56.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:55 smithi120 ceph-mon[31150]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T08:56:56.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:55 smithi120 ceph-mon[31150]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T08:56:56.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:55 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:56:56.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:55 smithi120 ceph-mon[31150]: from='osd.4 [v2:172.21.15.153:6808/2680865854,v1:172.21.15.153:6809/2680865854]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-19T08:56:56.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:55 smithi120 ceph-mon[31150]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-19T08:56:56.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:55 smithi196 ceph-mon[35150]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T08:56:56.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:55 smithi196 ceph-mon[35150]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T08:56:56.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:55 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:56:56.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:55 smithi196 ceph-mon[35150]: from='osd.4 [v2:172.21.15.153:6808/2680865854,v1:172.21.15.153:6809/2680865854]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-19T08:56:56.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:55 smithi196 ceph-mon[35150]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-19T08:56:56.237 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:56:56.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.687+0000 7fbd5a69d640 1 -- 172.21.15.196:0/358129417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd54102e00 msgr2=0x7fbd54103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:56.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.687+0000 7fbd5a69d640 1 --2- 172.21.15.196:0/358129417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd54102e00 0x7fbd54103280 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c009900 tx=0x7fbd3c02a410 comp rx=0 tx=0).stop 2024-09-19T08:56:56.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.687+0000 7fbd5a69d640 1 -- 172.21.15.196:0/358129417 shutdown_connections 2024-09-19T08:56:56.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.687+0000 7fbd5a69d640 1 --2- 172.21.15.196:0/358129417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd541037c0 0x7fbd5410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.687+0000 7fbd5a69d640 1 --2- 172.21.15.196:0/358129417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd54102e00 0x7fbd54103280 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.687+0000 7fbd5a69d640 1 --2- 172.21.15.196:0/358129417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd54101c00 0x7fbd54102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.687+0000 7fbd5a69d640 1 -- 172.21.15.196:0/358129417 >> 172.21.15.196:0/358129417 conn(0x7fbd540fd390 msgr2=0x7fbd540ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:56.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.687+0000 7fbd5a69d640 1 -- 172.21.15.196:0/358129417 shutdown_connections 2024-09-19T08:56:56.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.687+0000 7fbd5a69d640 1 -- 172.21.15.196:0/358129417 wait complete. 2024-09-19T08:56:56.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.688+0000 7fbd5a69d640 1 Processor -- start 2024-09-19T08:56:56.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.688+0000 7fbd5a69d640 1 -- start start 2024-09-19T08:56:56.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.688+0000 7fbd5a69d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd54101c00 0x7fbd54197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:56.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.688+0000 7fbd5a69d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd54102e00 0x7fbd54198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:56.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.688+0000 7fbd5a69d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd541037c0 0x7fbd5419f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:56.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.688+0000 7fbd5a69d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5410d880 con 0x7fbd541037c0 2024-09-19T08:56:56.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.688+0000 7fbd5a69d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5410d700 con 0x7fbd54101c00 2024-09-19T08:56:56.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.688+0000 7fbd5a69d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5410da00 con 0x7fbd54102e00 2024-09-19T08:56:56.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.688+0000 7fbd537fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd54102e00 0x7fbd54198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:56.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.688+0000 7fbd537fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd54102e00 0x7fbd54198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49880/0 (socket says 172.21.15.196:49880) 2024-09-19T08:56:56.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.688+0000 7fbd537fe640 1 -- 172.21.15.196:0/2165938620 learned_addr learned my addr 172.21.15.196:0/2165938620 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-19T08:56:56.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.689+0000 7fbd58c3a640 1 --2- 172.21.15.196:0/2165938620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd541037c0 0x7fbd5419f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:56.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.689+0000 7fbd53fff640 1 --2- 172.21.15.196:0/2165938620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd54101c00 0x7fbd54197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:56.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.689+0000 7fbd537fe640 1 -- 172.21.15.196:0/2165938620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd54101c00 msgr2=0x7fbd54197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:56.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.689+0000 7fbd537fe640 1 --2- 172.21.15.196:0/2165938620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd54101c00 0x7fbd54197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.689+0000 7fbd537fe640 1 -- 172.21.15.196:0/2165938620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd541037c0 msgr2=0x7fbd5419f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:56.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.689+0000 7fbd537fe640 1 --2- 172.21.15.196:0/2165938620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd541037c0 0x7fbd5419f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.689+0000 7fbd537fe640 1 -- 172.21.15.196:0/2165938620 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5419fb20 con 0x7fbd54102e00 2024-09-19T08:56:56.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.689+0000 7fbd58c3a640 1 --2- 172.21.15.196:0/2165938620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd541037c0 0x7fbd5419f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:56:56.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.689+0000 7fbd537fe640 1 --2- 172.21.15.196:0/2165938620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd54102e00 0x7fbd54198430 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c009230 tx=0x7fbd3c004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:56.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.689+0000 7fbd517fa640 1 -- 172.21.15.196:0/2165938620 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c03f030 con 0x7fbd54102e00 2024-09-19T08:56:56.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.689+0000 7fbd5a69d640 1 -- 172.21.15.196:0/2165938620 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd5419fdb0 con 0x7fbd54102e00 2024-09-19T08:56:56.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.689+0000 7fbd517fa640 1 -- 172.21.15.196:0/2165938620 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd3c003ce0 con 0x7fbd54102e00 2024-09-19T08:56:56.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.689+0000 7fbd517fa640 1 -- 172.21.15.196:0/2165938620 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c035070 con 0x7fbd54102e00 2024-09-19T08:56:56.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.689+0000 7fbd5a69d640 1 -- 172.21.15.196:0/2165938620 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd541a0250 con 0x7fbd54102e00 2024-09-19T08:56:56.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.690+0000 7fbd5a69d640 1 -- 172.21.15.196:0/2165938620 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd54109000 con 0x7fbd54102e00 2024-09-19T08:56:56.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.691+0000 7fbd517fa640 1 -- 172.21.15.196:0/2165938620 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fbd3c042030 con 0x7fbd54102e00 2024-09-19T08:56:56.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.692+0000 7fbd517fa640 1 --2- 172.21.15.196:0/2165938620 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbd400a3270 0x7fbd400a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:56.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.692+0000 7fbd517fa640 1 -- 172.21.15.196:0/2165938620 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3821+0+0 (secure 0 0 0) 0x7fbd3c0eeca0 con 0x7fbd54102e00 2024-09-19T08:56:56.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.692+0000 7fbd53fff640 1 --2- 172.21.15.196:0/2165938620 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbd400a3270 0x7fbd400a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:56.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.692+0000 7fbd53fff640 1 --2- 172.21.15.196:0/2165938620 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbd400a3270 0x7fbd400a5730 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbd54102c60 tx=0x7fbd44005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:56.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.696+0000 7fbd517fa640 1 -- 172.21.15.196:0/2165938620 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbd3c0bcab0 con 0x7fbd54102e00 2024-09-19T08:56:56.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:56:56.873+0000 7fbd5a69d640 1 -- 172.21.15.196:0/2165938620 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fbd54062630 con 0x7fbd400a3270 2024-09-19T08:56:56.874 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:56 smithi196 ceph-mon[35150]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:56:56.874 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:56 smithi196 ceph-mon[35150]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-19T08:56:56.874 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:56 smithi196 ceph-mon[35150]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T08:56:56.874 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:56 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:56:56.874 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:56 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:56:56.875 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:56 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:56.875 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:56 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:56.875 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:56 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:56.875 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:56 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:56.875 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:56 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:56.875 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:56 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:56.875 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:56 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:56:56.875 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:56 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:57.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:56 smithi120 ceph-mon[31150]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:56:57.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:56 smithi120 ceph-mon[31150]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-19T08:56:57.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:56 smithi120 ceph-mon[31150]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T08:56:57.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:56 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:56:57.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:56 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:56:57.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:56 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:57.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:56 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:57.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:56 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:57.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:56 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:57.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:56 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:57.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:56 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:57.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:56 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:56:57.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:56 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:57.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:56 smithi153 ceph-mon[35071]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:56:57.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:56 smithi153 ceph-mon[35071]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-19T08:56:57.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:56 smithi153 ceph-mon[35071]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T08:56:57.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:56 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:56:57.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:56 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:56:57.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:56 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:57.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:56 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:57.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:56 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:57.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:56 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:57.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:56 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:57.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:56 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:57.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:56 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:56:57.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:56 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:56:57.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:57 smithi196 ceph-mon[35150]: purged_snaps scrub starts 2024-09-19T08:56:57.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:57 smithi196 ceph-mon[35150]: purged_snaps scrub ok 2024-09-19T08:56:57.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:57 smithi196 ceph-mon[35150]: Detected new or changed devices on smithi153 2024-09-19T08:56:57.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:57 smithi196 ceph-mon[35150]: Adjusting osd_memory_target on smithi153 to 8544M 2024-09-19T08:56:57.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:57 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:56:57.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:57 smithi196 ceph-mon[35150]: osd.4 [v2:172.21.15.153:6808/2680865854,v1:172.21.15.153:6809/2680865854] boot 2024-09-19T08:56:57.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:57 smithi196 ceph-mon[35150]: osdmap e31: 5 total, 5 up, 5 in 2024-09-19T08:56:57.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:57 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:56:57.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:57 smithi196 ceph-mon[35150]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:56:57.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:57 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:56:57.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:57 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:56:57.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:57 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:57.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:57 smithi196 ceph-mon[35150]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:56:58.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:57 smithi120 ceph-mon[31150]: purged_snaps scrub starts 2024-09-19T08:56:58.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:57 smithi120 ceph-mon[31150]: purged_snaps scrub ok 2024-09-19T08:56:58.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:57 smithi120 ceph-mon[31150]: Detected new or changed devices on smithi153 2024-09-19T08:56:58.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:57 smithi120 ceph-mon[31150]: Adjusting osd_memory_target on smithi153 to 8544M 2024-09-19T08:56:58.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:57 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:56:58.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:57 smithi120 ceph-mon[31150]: osd.4 [v2:172.21.15.153:6808/2680865854,v1:172.21.15.153:6809/2680865854] boot 2024-09-19T08:56:58.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:57 smithi120 ceph-mon[31150]: osdmap e31: 5 total, 5 up, 5 in 2024-09-19T08:56:58.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:57 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:56:58.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:57 smithi120 ceph-mon[31150]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:56:58.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:57 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:56:58.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:57 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:56:58.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:57 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:58.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:57 smithi120 ceph-mon[31150]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:56:58.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:57 smithi153 ceph-mon[35071]: purged_snaps scrub starts 2024-09-19T08:56:58.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:57 smithi153 ceph-mon[35071]: purged_snaps scrub ok 2024-09-19T08:56:58.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:57 smithi153 ceph-mon[35071]: Detected new or changed devices on smithi153 2024-09-19T08:56:58.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:57 smithi153 ceph-mon[35071]: Adjusting osd_memory_target on smithi153 to 8544M 2024-09-19T08:56:58.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:57 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:56:58.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:57 smithi153 ceph-mon[35071]: osd.4 [v2:172.21.15.153:6808/2680865854,v1:172.21.15.153:6809/2680865854] boot 2024-09-19T08:56:58.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:57 smithi153 ceph-mon[35071]: osdmap e31: 5 total, 5 up, 5 in 2024-09-19T08:56:58.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:57 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:56:58.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:57 smithi153 ceph-mon[35071]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:56:58.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:57 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:56:58.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:57 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:56:58.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:57 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:58.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:57 smithi153 ceph-mon[35071]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:56:58.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:58 smithi196 ceph-mon[35150]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T08:56:59.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:58 smithi120 ceph-mon[31150]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T08:56:59.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:58 smithi153 ceph-mon[35071]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T08:57:00.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:59 smithi120 ceph-mon[31150]: from='client.? 172.21.15.196:0/3146125536' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70b270b2-209d-4ff5-875a-04ee7b992a93"}]: dispatch 2024-09-19T08:57:00.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:59 smithi120 ceph-mon[31150]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70b270b2-209d-4ff5-875a-04ee7b992a93"}]: dispatch 2024-09-19T08:57:00.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:59 smithi120 ceph-mon[31150]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70b270b2-209d-4ff5-875a-04ee7b992a93"}]': finished 2024-09-19T08:57:00.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:59 smithi120 ceph-mon[31150]: osdmap e33: 6 total, 5 up, 6 in 2024-09-19T08:57:00.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:59 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:57:00.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:56:59 smithi120 ceph-mon[31150]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:00.088 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:59 smithi153 ceph-mon[35071]: from='client.? 172.21.15.196:0/3146125536' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70b270b2-209d-4ff5-875a-04ee7b992a93"}]: dispatch 2024-09-19T08:57:00.088 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:59 smithi153 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70b270b2-209d-4ff5-875a-04ee7b992a93"}]: dispatch 2024-09-19T08:57:00.089 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:59 smithi153 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70b270b2-209d-4ff5-875a-04ee7b992a93"}]': finished 2024-09-19T08:57:00.089 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:59 smithi153 ceph-mon[35071]: osdmap e33: 6 total, 5 up, 6 in 2024-09-19T08:57:00.089 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:59 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:57:00.089 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:56:59 smithi153 ceph-mon[35071]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:00.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:59 smithi196 ceph-mon[35150]: from='client.? 172.21.15.196:0/3146125536' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70b270b2-209d-4ff5-875a-04ee7b992a93"}]: dispatch 2024-09-19T08:57:00.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:59 smithi196 ceph-mon[35150]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70b270b2-209d-4ff5-875a-04ee7b992a93"}]: dispatch 2024-09-19T08:57:00.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:59 smithi196 ceph-mon[35150]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70b270b2-209d-4ff5-875a-04ee7b992a93"}]': finished 2024-09-19T08:57:00.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:59 smithi196 ceph-mon[35150]: osdmap e33: 6 total, 5 up, 6 in 2024-09-19T08:57:00.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:59 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:57:00.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:56:59 smithi196 ceph-mon[35150]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:01.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:00 smithi120 ceph-mon[31150]: from='client.? 172.21.15.196:0/2089836479' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:57:01.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:00 smithi196 ceph-mon[35150]: from='client.? 172.21.15.196:0/2089836479' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:57:01.186 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:00 smithi153 ceph-mon[35071]: from='client.? 172.21.15.196:0/2089836479' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:57:02.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:01 smithi120 ceph-mon[31150]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:02.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:01 smithi196 ceph-mon[35150]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:02.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:01 smithi153 ceph-mon[35071]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:04.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:04 smithi196 ceph-mon[35150]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:04.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:04 smithi153 ceph-mon[35071]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:05.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:04 smithi120 ceph-mon[31150]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:06.828 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:06 smithi196 ceph-mon[35150]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:06.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:06 smithi153 ceph-mon[35071]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:07.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:06 smithi120 ceph-mon[31150]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:08.871 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:08 smithi196 ceph-mon[35150]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:08.871 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:08 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T08:57:08.871 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:08 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:57:08.872 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:08 smithi196 ceph-mon[35150]: Deploying daemon osd.5 on smithi196 2024-09-19T08:57:08.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:08 smithi153 ceph-mon[35071]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:08.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:08 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T08:57:08.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:08 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:57:08.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:08 smithi153 ceph-mon[35071]: Deploying daemon osd.5 on smithi196 2024-09-19T08:57:09.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:08 smithi120 ceph-mon[31150]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:09.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:08 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T08:57:09.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:08 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:57:09.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:08 smithi120 ceph-mon[31150]: Deploying daemon osd.5 on smithi196 2024-09-19T08:57:10.815 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:10 smithi196 ceph-mon[35150]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:10.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:10 smithi153 ceph-mon[35071]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:11.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:10 smithi120 ceph-mon[31150]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:12.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:12 smithi196 ceph-mon[35150]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:12.936 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:12 smithi153 ceph-mon[35071]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:13.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:12 smithi120 ceph-mon[31150]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:14.912 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:14 smithi196 ceph-mon[35150]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:14.913 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:14 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:57:14.913 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:14 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:14.913 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:14 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:14.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:14 smithi153 ceph-mon[35071]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:14.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:14 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:57:14.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:14 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:14.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:14 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:15.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:14 smithi120 ceph-mon[31150]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:15.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:14 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:57:15.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:14 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:15.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:14 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:16.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:16 smithi196 ceph-mon[35150]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:16.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:16 smithi153 ceph-mon[35071]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:17.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:16 smithi120 ceph-mon[31150]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:18.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:18 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:18.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:18 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:18.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:18 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:57:18.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:18 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:57:18.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:18 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:18.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:18 smithi120 ceph-mon[31150]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:18.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:18 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:18.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:18 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:18.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:18 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:57:18.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:18 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:57:18.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:18 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:18.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:18 smithi196 ceph-mon[35150]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:18.414 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 5 on host 'smithi196' 2024-09-19T08:57:18.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:57:18.413+0000 7fbd517fa640 1 -- 172.21.15.196:0/2165938620 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbd54062630 con 0x7fbd400a3270 2024-09-19T08:57:18.416 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:57:18.416+0000 7fbd5a69d640 1 -- 172.21.15.196:0/2165938620 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbd400a3270 msgr2=0x7fbd400a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:18.416 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:57:18.416+0000 7fbd5a69d640 1 --2- 172.21.15.196:0/2165938620 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbd400a3270 0x7fbd400a5730 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbd54102c60 tx=0x7fbd44005c60 comp rx=0 tx=0).stop 2024-09-19T08:57:18.416 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:57:18.416+0000 7fbd5a69d640 1 -- 172.21.15.196:0/2165938620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd54102e00 msgr2=0x7fbd54198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:18.416 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:57:18.416+0000 7fbd5a69d640 1 --2- 172.21.15.196:0/2165938620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd54102e00 0x7fbd54198430 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c009230 tx=0x7fbd3c004430 comp rx=0 tx=0).stop 2024-09-19T08:57:18.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:57:18.416+0000 7fbd5a69d640 1 -- 172.21.15.196:0/2165938620 shutdown_connections 2024-09-19T08:57:18.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:57:18.416+0000 7fbd5a69d640 1 --2- 172.21.15.196:0/2165938620 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbd400a3270 0x7fbd400a5730 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:18.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:57:18.416+0000 7fbd5a69d640 1 --2- 172.21.15.196:0/2165938620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd541037c0 0x7fbd5419f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:18.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:57:18.416+0000 7fbd5a69d640 1 --2- 172.21.15.196:0/2165938620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd54102e00 0x7fbd54198430 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:18.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:57:18.416+0000 7fbd5a69d640 1 --2- 172.21.15.196:0/2165938620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd54101c00 0x7fbd54197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:18.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:57:18.416+0000 7fbd5a69d640 1 -- 172.21.15.196:0/2165938620 >> 172.21.15.196:0/2165938620 conn(0x7fbd540fd390 msgr2=0x7fbd541061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:18.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:57:18.417+0000 7fbd5a69d640 1 -- 172.21.15.196:0/2165938620 shutdown_connections 2024-09-19T08:57:18.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:57:18.417+0000 7fbd5a69d640 1 -- 172.21.15.196:0/2165938620 wait complete. 2024-09-19T08:57:18.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:18 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:18.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:18 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:18.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:18 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:57:18.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:18 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:57:18.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:18 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:18.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:18 smithi153 ceph-mon[35071]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:19.077 DEBUG:teuthology.orchestra.run.smithi196:osd.5> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.5.service 2024-09-19T08:57:19.079 INFO:tasks.cephadm:Deploying osd.6 on smithi120 with /dev/vg_nvme/lv_2... 2024-09-19T08:57:19.079 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T08:57:19.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:19 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:57:19.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:19 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:19.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:19 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:19.327 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:57:19.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:19 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:57:19.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:19 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:19.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:19 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:19.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:19 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:57:19.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:19 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:19.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:19 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:20.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:20 smithi120 ceph-mon[31150]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:20.381 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:20 smithi196 ceph-mon[35150]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:20.381 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 08:57:20 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T08:57:20.242+0000 7f3436a20640 -1 osd.5 0 log_to_monitors true 2024-09-19T08:57:20.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:20 smithi153 ceph-mon[35071]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:21.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:21 smithi120 ceph-mon[31150]: from='osd.5 [v2:172.21.15.196:6808/3763331116,v1:172.21.15.196:6809/3763331116]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T08:57:21.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:21 smithi120 ceph-mon[31150]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T08:57:21.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:21 smithi196 ceph-mon[35150]: from='osd.5 [v2:172.21.15.196:6808/3763331116,v1:172.21.15.196:6809/3763331116]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T08:57:21.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:21 smithi196 ceph-mon[35150]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T08:57:21.436 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:21 smithi153 ceph-mon[35071]: from='osd.5 [v2:172.21.15.196:6808/3763331116,v1:172.21.15.196:6809/3763331116]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T08:57:21.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:21 smithi153 ceph-mon[35071]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T08:57:21.876 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:57:21.901 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi120:vg_nvme/lv_2 2024-09-19T08:57:22.160 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:57:22.196 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:22 smithi196 ceph-mon[35150]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T08:57:22.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:22 smithi196 ceph-mon[35150]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T08:57:22.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:22 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:57:22.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:22 smithi196 ceph-mon[35150]: from='osd.5 [v2:172.21.15.196:6808/3763331116,v1:172.21.15.196:6809/3763331116]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T08:57:22.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:22 smithi196 ceph-mon[35150]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T08:57:22.198 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:22 smithi196 ceph-mon[35150]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:22.198 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:22 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:22.198 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:22 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:22.198 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:22 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:57:22.198 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:22 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:57:22.198 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:22 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:22.199 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:22 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:57:22.199 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:22 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:57:22.199 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:22 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:22.281 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:22 smithi120 ceph-mon[31150]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T08:57:22.281 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:22 smithi120 ceph-mon[31150]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T08:57:22.281 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:22 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:57:22.281 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:22 smithi120 ceph-mon[31150]: from='osd.5 [v2:172.21.15.196:6808/3763331116,v1:172.21.15.196:6809/3763331116]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T08:57:22.281 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:22 smithi120 ceph-mon[31150]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T08:57:22.281 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:22 smithi120 ceph-mon[31150]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:22.282 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:22 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:22.282 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:22 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:22.282 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:22 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:57:22.282 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:22 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:57:22.282 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:22 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:22.282 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:22 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:57:22.283 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:22 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:57:22.283 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:22 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:22.442 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:22 smithi153 ceph-mon[35071]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T08:57:22.442 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:22 smithi153 ceph-mon[35071]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T08:57:22.442 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:22 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:57:22.442 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:22 smithi153 ceph-mon[35071]: from='osd.5 [v2:172.21.15.196:6808/3763331116,v1:172.21.15.196:6809/3763331116]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T08:57:22.442 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:22 smithi153 ceph-mon[35071]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T08:57:22.442 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:22 smithi153 ceph-mon[35071]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:57:22.442 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:22 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:22.443 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:22 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:22.443 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:22 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:57:22.443 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:22 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:57:22.443 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:22 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:22.443 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:22 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:57:22.443 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:22 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:57:22.443 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:22 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:22.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.602+0000 7fb5323de640 1 -- 172.21.15.120:0/3480584058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb52c1026d0 msgr2=0x7fb52c108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:22.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.602+0000 7fb5323de640 1 --2- 172.21.15.120:0/3480584058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb52c1026d0 0x7fb52c108f60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb524009ca0 tx=0x7fb52402a8c0 comp rx=0 tx=0).stop 2024-09-19T08:57:22.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.602+0000 7fb5323de640 1 -- 172.21.15.120:0/3480584058 shutdown_connections 2024-09-19T08:57:22.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.602+0000 7fb5323de640 1 --2- 172.21.15.120:0/3480584058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb52c1026d0 0x7fb52c108f60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:22.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.602+0000 7fb5323de640 1 --2- 172.21.15.120:0/3480584058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb52c101d10 0x7fb52c102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:22.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.602+0000 7fb5323de640 1 --2- 172.21.15.120:0/3480584058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb52c100b10 0x7fb52c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:22.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.602+0000 7fb5323de640 1 -- 172.21.15.120:0/3480584058 >> 172.21.15.120:0/3480584058 conn(0x7fb52c0fc2a0 msgr2=0x7fb52c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:22.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.602+0000 7fb5323de640 1 -- 172.21.15.120:0/3480584058 shutdown_connections 2024-09-19T08:57:22.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.602+0000 7fb5323de640 1 -- 172.21.15.120:0/3480584058 wait complete. 2024-09-19T08:57:22.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.602+0000 7fb5323de640 1 Processor -- start 2024-09-19T08:57:22.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.603+0000 7fb5323de640 1 -- start start 2024-09-19T08:57:22.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.603+0000 7fb5323de640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb52c100b10 0x7fb52c104aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:22.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.603+0000 7fb5323de640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb52c101d10 0x7fb52c1030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:22.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.603+0000 7fb5323de640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb52c1026d0 0x7fb52c103630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:22.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.603+0000 7fb5323de640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c10c790 con 0x7fb52c1026d0 2024-09-19T08:57:22.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.603+0000 7fb5323de640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c10c610 con 0x7fb52c100b10 2024-09-19T08:57:22.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.603+0000 7fb5323de640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c10c910 con 0x7fb52c101d10 2024-09-19T08:57:22.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.603+0000 7fb53097b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb52c1026d0 0x7fb52c103630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:22.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.603+0000 7fb53097b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb52c1026d0 0x7fb52c103630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59308/0 (socket says 172.21.15.120:59308) 2024-09-19T08:57:22.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.603+0000 7fb53097b640 1 -- 172.21.15.120:0/607198777 learned_addr learned my addr 172.21.15.120:0/607198777 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:57:22.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.603+0000 7fb52bfff640 1 --2- 172.21.15.120:0/607198777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb52c100b10 0x7fb52c104aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:22.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.603+0000 7fb52b7fe640 1 --2- 172.21.15.120:0/607198777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb52c101d10 0x7fb52c1030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:22.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.603+0000 7fb53097b640 1 -- 172.21.15.120:0/607198777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb52c101d10 msgr2=0x7fb52c1030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:22.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.603+0000 7fb53097b640 1 --2- 172.21.15.120:0/607198777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb52c101d10 0x7fb52c1030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:22.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.603+0000 7fb53097b640 1 -- 172.21.15.120:0/607198777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb52c100b10 msgr2=0x7fb52c104aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:22.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.603+0000 7fb53097b640 1 --2- 172.21.15.120:0/607198777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb52c100b10 0x7fb52c104aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:22.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.603+0000 7fb53097b640 1 -- 172.21.15.120:0/607198777 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb52c103ef0 con 0x7fb52c1026d0 2024-09-19T08:57:22.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.603+0000 7fb52b7fe640 1 --2- 172.21.15.120:0/607198777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb52c101d10 0x7fb52c1030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:57:22.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.604+0000 7fb53097b640 1 --2- 172.21.15.120:0/607198777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb52c1026d0 0x7fb52c103630 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb524005e30 tx=0x7fb524031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:22.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.604+0000 7fb5297fa640 1 -- 172.21.15.120:0/607198777 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb524003c30 con 0x7fb52c1026d0 2024-09-19T08:57:22.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.604+0000 7fb5297fa640 1 -- 172.21.15.120:0/607198777 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb524003dd0 con 0x7fb52c1026d0 2024-09-19T08:57:22.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.604+0000 7fb5323de640 1 -- 172.21.15.120:0/607198777 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb52c1a1fd0 con 0x7fb52c1026d0 2024-09-19T08:57:22.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.604+0000 7fb5297fa640 1 -- 172.21.15.120:0/607198777 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52402fde0 con 0x7fb52c1026d0 2024-09-19T08:57:22.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.604+0000 7fb5323de640 1 -- 172.21.15.120:0/607198777 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb52c1a2480 con 0x7fb52c1026d0 2024-09-19T08:57:22.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.605+0000 7fb5323de640 1 -- 172.21.15.120:0/607198777 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb52c1a2770 con 0x7fb52c1026d0 2024-09-19T08:57:22.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.606+0000 7fb5297fa640 1 -- 172.21.15.120:0/607198777 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fb524004050 con 0x7fb52c1026d0 2024-09-19T08:57:22.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.606+0000 7fb5297fa640 1 --2- 172.21.15.120:0/607198777 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb5180a3240 0x7fb5180a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:22.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.606+0000 7fb52bfff640 1 --2- 172.21.15.120:0/607198777 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb5180a3240 0x7fb5180a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:22.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.607+0000 7fb5297fa640 1 -- 172.21.15.120:0/607198777 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4253+0+0 (secure 0 0 0) 0x7fb5240efcf0 con 0x7fb52c1026d0 2024-09-19T08:57:22.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.607+0000 7fb52bfff640 1 --2- 172.21.15.120:0/607198777 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb5180a3240 0x7fb5180a5700 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb51c0058c0 tx=0x7fb51c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:22.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.611+0000 7fb5297fa640 1 -- 172.21.15.120:0/607198777 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb5240bdaf0 con 0x7fb52c1026d0 2024-09-19T08:57:22.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:22.785+0000 7fb5323de640 1 -- 172.21.15.120:0/607198777 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb52c1a2940 con 0x7fb5180a3240 2024-09-19T08:57:22.935 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 08:57:22 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T08:57:22.461+0000 7f34329a3640 -1 osd.5 0 waiting for initial osdmap 2024-09-19T08:57:22.936 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 08:57:22 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T08:57:22.467+0000 7f342a7c3640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:57:23.332 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:23 smithi120 ceph-mon[31150]: Detected new or changed devices on smithi196 2024-09-19T08:57:23.332 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:23 smithi120 ceph-mon[31150]: Adjusting osd_memory_target on smithi196 to 8544M 2024-09-19T08:57:23.332 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:23 smithi120 ceph-mon[31150]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-19T08:57:23.332 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:23 smithi120 ceph-mon[31150]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T08:57:23.332 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:23 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:57:23.332 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:23 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:57:23.332 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:23 smithi120 ceph-mon[31150]: from='osd.5 ' entity='osd.5' 2024-09-19T08:57:23.333 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:23 smithi120 ceph-mon[31150]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:57:23.333 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:23 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:57:23.333 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:23 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:57:23.333 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:23 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:57:23.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:23 smithi196 ceph-mon[35150]: Detected new or changed devices on smithi196 2024-09-19T08:57:23.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:23 smithi196 ceph-mon[35150]: Adjusting osd_memory_target on smithi196 to 8544M 2024-09-19T08:57:23.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:23 smithi196 ceph-mon[35150]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-19T08:57:23.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:23 smithi196 ceph-mon[35150]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T08:57:23.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:23 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:57:23.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:23 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:57:23.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:23 smithi196 ceph-mon[35150]: from='osd.5 ' entity='osd.5' 2024-09-19T08:57:23.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:23 smithi196 ceph-mon[35150]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:57:23.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:23 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:57:23.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:23 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:57:23.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:23 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:57:23.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:23 smithi153 ceph-mon[35071]: Detected new or changed devices on smithi196 2024-09-19T08:57:23.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:23 smithi153 ceph-mon[35071]: Adjusting osd_memory_target on smithi196 to 8544M 2024-09-19T08:57:23.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:23 smithi153 ceph-mon[35071]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-19T08:57:23.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:23 smithi153 ceph-mon[35071]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T08:57:23.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:23 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:57:23.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:23 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:57:23.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:23 smithi153 ceph-mon[35071]: from='osd.5 ' entity='osd.5' 2024-09-19T08:57:23.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:23 smithi153 ceph-mon[35071]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:57:23.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:23 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:57:23.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:23 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:57:23.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:23 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:57:24.185 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:24 smithi120 ceph-mon[31150]: purged_snaps scrub starts 2024-09-19T08:57:24.185 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:24 smithi120 ceph-mon[31150]: purged_snaps scrub ok 2024-09-19T08:57:24.185 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:24 smithi120 ceph-mon[31150]: osd.5 [v2:172.21.15.196:6808/3763331116,v1:172.21.15.196:6809/3763331116] boot 2024-09-19T08:57:24.185 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:24 smithi120 ceph-mon[31150]: osdmap e36: 6 total, 6 up, 6 in 2024-09-19T08:57:24.185 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:24 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:57:24.185 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:24 smithi120 ceph-mon[31150]: pgmap v132: 1 pgs: 1 peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:57:24.185 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:24 smithi120 ceph-mon[31150]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T08:57:24.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:24 smithi196 ceph-mon[35150]: purged_snaps scrub starts 2024-09-19T08:57:24.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:24 smithi196 ceph-mon[35150]: purged_snaps scrub ok 2024-09-19T08:57:24.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:24 smithi196 ceph-mon[35150]: osd.5 [v2:172.21.15.196:6808/3763331116,v1:172.21.15.196:6809/3763331116] boot 2024-09-19T08:57:24.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:24 smithi196 ceph-mon[35150]: osdmap e36: 6 total, 6 up, 6 in 2024-09-19T08:57:24.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:24 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:57:24.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:24 smithi196 ceph-mon[35150]: pgmap v132: 1 pgs: 1 peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:57:24.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:24 smithi196 ceph-mon[35150]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T08:57:24.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:24 smithi153 ceph-mon[35071]: purged_snaps scrub starts 2024-09-19T08:57:24.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:24 smithi153 ceph-mon[35071]: purged_snaps scrub ok 2024-09-19T08:57:24.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:24 smithi153 ceph-mon[35071]: osd.5 [v2:172.21.15.196:6808/3763331116,v1:172.21.15.196:6809/3763331116] boot 2024-09-19T08:57:24.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:24 smithi153 ceph-mon[35071]: osdmap e36: 6 total, 6 up, 6 in 2024-09-19T08:57:24.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:24 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:57:24.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:24 smithi153 ceph-mon[35071]: pgmap v132: 1 pgs: 1 peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:57:24.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:24 smithi153 ceph-mon[35071]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T08:57:26.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:26 smithi196 ceph-mon[35150]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T08:57:26.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:26 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/735069076' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8087f69c-dc31-4522-a18d-4e6ab1b7cb27"}]: dispatch 2024-09-19T08:57:26.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:26 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/735069076' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8087f69c-dc31-4522-a18d-4e6ab1b7cb27"}]': finished 2024-09-19T08:57:26.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:26 smithi196 ceph-mon[35150]: osdmap e39: 7 total, 6 up, 7 in 2024-09-19T08:57:26.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:26 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:57:26.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:26 smithi196 ceph-mon[35150]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:57:26.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:26 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2549537305' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:57:26.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:26 smithi153 ceph-mon[35071]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T08:57:26.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:26 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/735069076' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8087f69c-dc31-4522-a18d-4e6ab1b7cb27"}]: dispatch 2024-09-19T08:57:26.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:26 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/735069076' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8087f69c-dc31-4522-a18d-4e6ab1b7cb27"}]': finished 2024-09-19T08:57:26.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:26 smithi153 ceph-mon[35071]: osdmap e39: 7 total, 6 up, 7 in 2024-09-19T08:57:26.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:26 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:57:26.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:26 smithi153 ceph-mon[35071]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:57:26.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:26 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2549537305' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:57:26.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:26 smithi120 ceph-mon[31150]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T08:57:26.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:26 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/735069076' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8087f69c-dc31-4522-a18d-4e6ab1b7cb27"}]: dispatch 2024-09-19T08:57:26.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:26 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/735069076' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8087f69c-dc31-4522-a18d-4e6ab1b7cb27"}]': finished 2024-09-19T08:57:26.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:26 smithi120 ceph-mon[31150]: osdmap e39: 7 total, 6 up, 7 in 2024-09-19T08:57:26.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:26 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:57:26.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:26 smithi120 ceph-mon[31150]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:57:26.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:26 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2549537305' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:57:27.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:27 smithi120 ceph-mon[31150]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T08:57:27.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:27 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:57:27.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:27 smithi196 ceph-mon[35150]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T08:57:27.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:27 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:57:27.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:27 smithi153 ceph-mon[35071]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T08:57:27.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:27 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:57:28.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:28 smithi120 ceph-mon[31150]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:57:28.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:28 smithi196 ceph-mon[35150]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:57:28.686 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:28 smithi153 ceph-mon[35071]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:57:30.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:30 smithi196 ceph-mon[35150]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-09-19T08:57:30.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:30 smithi153 ceph-mon[35071]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-09-19T08:57:31.063 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:30 smithi120 ceph-mon[31150]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-09-19T08:57:32.748 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:32 smithi120 ceph-mon[31150]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-19T08:57:32.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:32 smithi196 ceph-mon[35150]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-19T08:57:32.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:32 smithi153 ceph-mon[35071]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-19T08:57:34.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:34 smithi120 ceph-mon[31150]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-19T08:57:34.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:34 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T08:57:34.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:34 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:57:34.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:34 smithi120 ceph-mon[31150]: Deploying daemon osd.6 on smithi120 2024-09-19T08:57:34.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:34 smithi196 ceph-mon[35150]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-19T08:57:34.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:34 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T08:57:34.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:34 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:57:34.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:34 smithi196 ceph-mon[35150]: Deploying daemon osd.6 on smithi120 2024-09-19T08:57:34.936 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:34 smithi153 ceph-mon[35071]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-19T08:57:34.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:34 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T08:57:34.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:34 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:57:34.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:34 smithi153 ceph-mon[35071]: Deploying daemon osd.6 on smithi120 2024-09-19T08:57:36.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:36 smithi120 ceph-mon[31150]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T08:57:36.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:36 smithi196 ceph-mon[35150]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T08:57:36.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:36 smithi153 ceph-mon[35071]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T08:57:38.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:38 smithi120 ceph-mon[31150]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-19T08:57:38.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:38 smithi196 ceph-mon[35150]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-19T08:57:38.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:38 smithi153 ceph-mon[35071]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-19T08:57:40.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:40 smithi120 ceph-mon[31150]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T08:57:40.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:40 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:57:40.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:40 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:40.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:40 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:40.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:40 smithi196 ceph-mon[35150]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T08:57:40.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:40 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:57:40.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:40 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:40.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:40 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:40.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:40 smithi153 ceph-mon[35071]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T08:57:40.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:40 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:57:40.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:40 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:40.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:40 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:42.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:42 smithi196 ceph-mon[35150]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:57:42.936 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:42 smithi153 ceph-mon[35071]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:57:43.025 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:42 smithi120 ceph-mon[31150]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:57:43.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:43.062+0000 7fb5297fa640 1 -- 172.21.15.120:0/607198777 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb52c1a2940 con 0x7fb5180a3240 2024-09-19T08:57:43.063 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 6 on host 'smithi120' 2024-09-19T08:57:43.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:43.064+0000 7fb5323de640 1 -- 172.21.15.120:0/607198777 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb5180a3240 msgr2=0x7fb5180a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:43.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:43.064+0000 7fb5323de640 1 --2- 172.21.15.120:0/607198777 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb5180a3240 0x7fb5180a5700 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb51c0058c0 tx=0x7fb51c00a2d0 comp rx=0 tx=0).stop 2024-09-19T08:57:43.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:43.064+0000 7fb5323de640 1 -- 172.21.15.120:0/607198777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb52c1026d0 msgr2=0x7fb52c103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:43.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:43.064+0000 7fb5323de640 1 --2- 172.21.15.120:0/607198777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb52c1026d0 0x7fb52c103630 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb524005e30 tx=0x7fb524031ee0 comp rx=0 tx=0).stop 2024-09-19T08:57:43.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:43.064+0000 7fb5323de640 1 -- 172.21.15.120:0/607198777 shutdown_connections 2024-09-19T08:57:43.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:43.064+0000 7fb5323de640 1 --2- 172.21.15.120:0/607198777 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb5180a3240 0x7fb5180a5700 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:43.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:43.064+0000 7fb5323de640 1 --2- 172.21.15.120:0/607198777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb52c1026d0 0x7fb52c103630 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:43.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:43.064+0000 7fb5323de640 1 --2- 172.21.15.120:0/607198777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb52c101d10 0x7fb52c1030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:43.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:43.064+0000 7fb5323de640 1 --2- 172.21.15.120:0/607198777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb52c100b10 0x7fb52c104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:43.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:43.064+0000 7fb5323de640 1 -- 172.21.15.120:0/607198777 >> 172.21.15.120:0/607198777 conn(0x7fb52c0fc2a0 msgr2=0x7fb52c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:43.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:43.064+0000 7fb5323de640 1 -- 172.21.15.120:0/607198777 shutdown_connections 2024-09-19T08:57:43.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:57:43.064+0000 7fb5323de640 1 -- 172.21.15.120:0/607198777 wait complete. 2024-09-19T08:57:43.764 DEBUG:teuthology.orchestra.run.smithi120:osd.6> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.6.service 2024-09-19T08:57:43.767 INFO:tasks.cephadm:Deploying osd.7 on smithi153 with /dev/vg_nvme/lv_2... 2024-09-19T08:57:43.767 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T08:57:44.023 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:57:44.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:44 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:44.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:44 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:44.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:44 smithi120 ceph-mon[31150]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:57:44.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:44 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:44.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:44 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:44.375 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:44 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:44.375 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:44 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:44.375 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:44 smithi153 ceph-mon[35071]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:57:44.375 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:44 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:44.375 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:44 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:44.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:44 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:44.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:44 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:44.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:44 smithi196 ceph-mon[35150]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:57:44.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:44 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:44.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:44 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:45.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:45 smithi153 ceph-mon[35071]: from='osd.6 [v2:172.21.15.120:6818/4171773765,v1:172.21.15.120:6819/4171773765]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T08:57:45.562 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:45 smithi120 ceph-mon[31150]: from='osd.6 [v2:172.21.15.120:6818/4171773765,v1:172.21.15.120:6819/4171773765]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T08:57:45.563 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 08:57:45 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T08:57:45.307+0000 7f50f44bd640 -1 osd.6 0 log_to_monitors true 2024-09-19T08:57:45.690 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:45 smithi196 ceph-mon[35150]: from='osd.6 [v2:172.21.15.120:6818/4171773765,v1:172.21.15.120:6819/4171773765]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T08:57:46.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:46 smithi196 ceph-mon[35150]: from='osd.6 [v2:172.21.15.120:6818/4171773765,v1:172.21.15.120:6819/4171773765]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T08:57:46.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:46 smithi196 ceph-mon[35150]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T08:57:46.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:46 smithi196 ceph-mon[35150]: from='osd.6 [v2:172.21.15.120:6818/4171773765,v1:172.21.15.120:6819/4171773765]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-19T08:57:46.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:46 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:57:46.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:46 smithi196 ceph-mon[35150]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:57:46.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:46 smithi196 ceph-mon[35150]: from='osd.6 [v2:172.21.15.120:6818/4171773765,v1:172.21.15.120:6819/4171773765]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-19T08:57:46.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:46 smithi196 ceph-mon[35150]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T08:57:46.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:46 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:57:46.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:46 smithi153 ceph-mon[35071]: from='osd.6 [v2:172.21.15.120:6818/4171773765,v1:172.21.15.120:6819/4171773765]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T08:57:46.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:46 smithi153 ceph-mon[35071]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T08:57:46.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:46 smithi153 ceph-mon[35071]: from='osd.6 [v2:172.21.15.120:6818/4171773765,v1:172.21.15.120:6819/4171773765]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-19T08:57:46.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:46 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:57:46.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:46 smithi153 ceph-mon[35071]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:57:46.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:46 smithi153 ceph-mon[35071]: from='osd.6 [v2:172.21.15.120:6818/4171773765,v1:172.21.15.120:6819/4171773765]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-19T08:57:46.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:46 smithi153 ceph-mon[35071]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T08:57:46.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:46 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:57:46.733 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-19T08:57:46.761 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi153:vg_nvme/lv_2 2024-09-19T08:57:46.825 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 08:57:46 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T08:57:46.370+0000 7f50f0440640 -1 osd.6 0 waiting for initial osdmap 2024-09-19T08:57:46.825 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 08:57:46 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T08:57:46.382+0000 7f50ec268640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:57:46.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:46 smithi120 ceph-mon[31150]: from='osd.6 [v2:172.21.15.120:6818/4171773765,v1:172.21.15.120:6819/4171773765]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T08:57:46.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:46 smithi120 ceph-mon[31150]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T08:57:46.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:46 smithi120 ceph-mon[31150]: from='osd.6 [v2:172.21.15.120:6818/4171773765,v1:172.21.15.120:6819/4171773765]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-19T08:57:46.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:46 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:57:46.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:46 smithi120 ceph-mon[31150]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:57:46.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:46 smithi120 ceph-mon[31150]: from='osd.6 [v2:172.21.15.120:6818/4171773765,v1:172.21.15.120:6819/4171773765]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-19T08:57:46.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:46 smithi120 ceph-mon[31150]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T08:57:46.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:46 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:57:47.011 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:57:47.408 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:47 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:57:47.408 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:47 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:47.408 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:47 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:47.408 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:47 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:57:47.408 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:47 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:57:47.408 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:47 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:57:47.408 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:47 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:47.408 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:47 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:57:47.408 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:47 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:57:47.409 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:47 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:47.409 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:47 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:57:47.409 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:47 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:57:47.409 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:47 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:57:47.409 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:47 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:47.409 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:47 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:57:47.508 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.493+0000 7f203f384640 1 -- 172.21.15.153:0/1308253396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2038101d10 msgr2=0x7f2038102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:47.508 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.493+0000 7f203f384640 1 --2- 172.21.15.153:0/1308253396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2038101d10 0x7f2038102190 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2028009900 tx=0x7f202802a410 comp rx=0 tx=0).stop 2024-09-19T08:57:47.508 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.493+0000 7f203f384640 1 -- 172.21.15.153:0/1308253396 shutdown_connections 2024-09-19T08:57:47.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.493+0000 7f203f384640 1 --2- 172.21.15.153:0/1308253396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20381026d0 0x7f2038108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.493+0000 7f203f384640 1 --2- 172.21.15.153:0/1308253396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2038101d10 0x7f2038102190 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.493+0000 7f203f384640 1 --2- 172.21.15.153:0/1308253396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2038100b10 0x7f2038100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.493+0000 7f203f384640 1 -- 172.21.15.153:0/1308253396 >> 172.21.15.153:0/1308253396 conn(0x7f20380fc2a0 msgr2=0x7f20380fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:47.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.493+0000 7f203f384640 1 -- 172.21.15.153:0/1308253396 shutdown_connections 2024-09-19T08:57:47.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.493+0000 7f203f384640 1 -- 172.21.15.153:0/1308253396 wait complete. 2024-09-19T08:57:47.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.494+0000 7f203f384640 1 Processor -- start 2024-09-19T08:57:47.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.494+0000 7f203f384640 1 -- start start 2024-09-19T08:57:47.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.494+0000 7f203f384640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2038100b10 0x7f2038197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:47.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.494+0000 7f203f384640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2038101d10 0x7f2038198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:47.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.494+0000 7f203f384640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20381026d0 0x7f203819f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:47.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.494+0000 7f203f384640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f203810c770 con 0x7f2038100b10 2024-09-19T08:57:47.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.494+0000 7f203f384640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f203810c5f0 con 0x7f2038101d10 2024-09-19T08:57:47.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.494+0000 7f203f384640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f203810c8f0 con 0x7f20381026d0 2024-09-19T08:57:47.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.494+0000 7f203c91f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2038101d10 0x7f2038198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:47.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.494+0000 7f203c91f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2038101d10 0x7f2038198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:59968/0 (socket says 172.21.15.153:59968) 2024-09-19T08:57:47.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.494+0000 7f203c91f640 1 -- 172.21.15.153:0/469980849 learned_addr learned my addr 172.21.15.153:0/469980849 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-19T08:57:47.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.494+0000 7f203d120640 1 --2- 172.21.15.153:0/469980849 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2038100b10 0x7f2038197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:47.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.494+0000 7f203d921640 1 --2- 172.21.15.153:0/469980849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20381026d0 0x7f203819f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:47.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.495+0000 7f203c91f640 1 -- 172.21.15.153:0/469980849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20381026d0 msgr2=0x7f203819f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:47.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.495+0000 7f203c91f640 1 --2- 172.21.15.153:0/469980849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20381026d0 0x7f203819f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.495+0000 7f203c91f640 1 -- 172.21.15.153:0/469980849 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2038100b10 msgr2=0x7f2038197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:47.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.495+0000 7f203c91f640 1 --2- 172.21.15.153:0/469980849 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2038100b10 0x7f2038197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.495+0000 7f203c91f640 1 -- 172.21.15.153:0/469980849 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f203819fb50 con 0x7f2038101d10 2024-09-19T08:57:47.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.495+0000 7f203c91f640 1 --2- 172.21.15.153:0/469980849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2038101d10 0x7f2038198460 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2028009230 tx=0x7f2028004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:47.511 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.495+0000 7f202e7fc640 1 -- 172.21.15.153:0/469980849 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202803f030 con 0x7f2038101d10 2024-09-19T08:57:47.511 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.495+0000 7f202e7fc640 1 -- 172.21.15.153:0/469980849 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2028003ce0 con 0x7f2038101d10 2024-09-19T08:57:47.511 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.495+0000 7f203f384640 1 -- 172.21.15.153:0/469980849 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f203819fde0 con 0x7f2038101d10 2024-09-19T08:57:47.511 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.495+0000 7f202e7fc640 1 -- 172.21.15.153:0/469980849 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2028035070 con 0x7f2038101d10 2024-09-19T08:57:47.511 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.495+0000 7f203f384640 1 -- 172.21.15.153:0/469980849 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20381a0240 con 0x7f2038101d10 2024-09-19T08:57:47.511 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.496+0000 7f203f384640 1 -- 172.21.15.153:0/469980849 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20381a0470 con 0x7f2038101d10 2024-09-19T08:57:47.511 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.497+0000 7f202e7fc640 1 -- 172.21.15.153:0/469980849 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f2028042030 con 0x7f2038101d10 2024-09-19T08:57:47.511 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.498+0000 7f202e7fc640 1 --2- 172.21.15.153:0/469980849 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f20240a32f0 0x7f20240a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:47.511 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.498+0000 7f202e7fc640 1 -- 172.21.15.153:0/469980849 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7f20280ef190 con 0x7f2038101d10 2024-09-19T08:57:47.511 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.498+0000 7f203d120640 1 --2- 172.21.15.153:0/469980849 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f20240a32f0 0x7f20240a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:47.511 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.498+0000 7f203d120640 1 --2- 172.21.15.153:0/469980849 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f20240a32f0 0x7f20240a57b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2038101b70 tx=0x7f203400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:47.511 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.502+0000 7f202e7fc640 1 -- 172.21.15.153:0/469980849 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f20280bcbc0 con 0x7f2038101d10 2024-09-19T08:57:47.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:47 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:57:47.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:47 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:47.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:47 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:47.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:47 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:57:47.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:47 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:57:47.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:47 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:57:47.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:47 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:47.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:47 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:57:47.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:47 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:57:47.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:47 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:47.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:47 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:57:47.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:47 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:57:47.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:47 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:57:47.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:47 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:47.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:47 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:57:47.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:57:47.676+0000 7f203f384640 1 -- 172.21.15.153:0/469980849 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f203806aed0 con 0x7f20240a32f0 2024-09-19T08:57:47.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:47 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:57:47.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:47 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:47.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:47 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:47.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:47 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:57:47.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:47 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:57:47.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:47 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:57:47.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:47 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:47.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:47 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:57:47.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:47 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:57:47.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:47 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:47.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:47 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:57:47.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:47 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:57:47.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:47 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:57:47.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:47 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:57:47.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:47 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:57:48.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:48 smithi196 ceph-mon[35150]: purged_snaps scrub starts 2024-09-19T08:57:48.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:48 smithi196 ceph-mon[35150]: purged_snaps scrub ok 2024-09-19T08:57:48.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:48 smithi196 ceph-mon[35150]: Detected new or changed devices on smithi120 2024-09-19T08:57:48.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:48 smithi196 ceph-mon[35150]: Adjusting osd_memory_target on smithi120 to 5696M 2024-09-19T08:57:48.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:48 smithi196 ceph-mon[35150]: osd.6 [v2:172.21.15.120:6818/4171773765,v1:172.21.15.120:6819/4171773765] boot 2024-09-19T08:57:48.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:48 smithi196 ceph-mon[35150]: osdmap e43: 7 total, 7 up, 7 in 2024-09-19T08:57:48.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:48 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:57:48.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:48 smithi196 ceph-mon[35150]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:57:48.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:48 smithi196 ceph-mon[35150]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:57:48.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:48 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:57:48.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:48 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:57:48.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:48 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:57:48.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:48 smithi153 ceph-mon[35071]: purged_snaps scrub starts 2024-09-19T08:57:48.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:48 smithi153 ceph-mon[35071]: purged_snaps scrub ok 2024-09-19T08:57:48.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:48 smithi153 ceph-mon[35071]: Detected new or changed devices on smithi120 2024-09-19T08:57:48.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:48 smithi153 ceph-mon[35071]: Adjusting osd_memory_target on smithi120 to 5696M 2024-09-19T08:57:48.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:48 smithi153 ceph-mon[35071]: osd.6 [v2:172.21.15.120:6818/4171773765,v1:172.21.15.120:6819/4171773765] boot 2024-09-19T08:57:48.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:48 smithi153 ceph-mon[35071]: osdmap e43: 7 total, 7 up, 7 in 2024-09-19T08:57:48.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:48 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:57:48.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:48 smithi153 ceph-mon[35071]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:57:48.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:48 smithi153 ceph-mon[35071]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:57:48.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:48 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:57:48.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:48 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:57:48.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:48 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:57:48.745 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:48 smithi120 ceph-mon[31150]: purged_snaps scrub starts 2024-09-19T08:57:48.745 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:48 smithi120 ceph-mon[31150]: purged_snaps scrub ok 2024-09-19T08:57:48.745 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:48 smithi120 ceph-mon[31150]: Detected new or changed devices on smithi120 2024-09-19T08:57:48.745 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:48 smithi120 ceph-mon[31150]: Adjusting osd_memory_target on smithi120 to 5696M 2024-09-19T08:57:48.745 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:48 smithi120 ceph-mon[31150]: osd.6 [v2:172.21.15.120:6818/4171773765,v1:172.21.15.120:6819/4171773765] boot 2024-09-19T08:57:48.745 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:48 smithi120 ceph-mon[31150]: osdmap e43: 7 total, 7 up, 7 in 2024-09-19T08:57:48.745 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:48 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:57:48.745 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:48 smithi120 ceph-mon[31150]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:57:48.745 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:48 smithi120 ceph-mon[31150]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:57:48.745 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:48 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:57:48.745 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:48 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:57:48.745 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:48 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:57:49.390 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:49 smithi153 ceph-mon[35071]: osdmap e44: 7 total, 7 up, 7 in 2024-09-19T08:57:49.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:49 smithi196 ceph-mon[35150]: osdmap e44: 7 total, 7 up, 7 in 2024-09-19T08:57:49.709 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:49 smithi120 ceph-mon[31150]: osdmap e44: 7 total, 7 up, 7 in 2024-09-19T08:57:50.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:50 smithi196 ceph-mon[35150]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:57:50.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:50 smithi196 ceph-mon[35150]: from='client.? 172.21.15.153:0/1370170970' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3b1ead3-150f-43fd-8924-f639203f4cdd"}]: dispatch 2024-09-19T08:57:50.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:50 smithi196 ceph-mon[35150]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3b1ead3-150f-43fd-8924-f639203f4cdd"}]: dispatch 2024-09-19T08:57:50.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:50 smithi196 ceph-mon[35150]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3b1ead3-150f-43fd-8924-f639203f4cdd"}]': finished 2024-09-19T08:57:50.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:50 smithi196 ceph-mon[35150]: osdmap e45: 8 total, 7 up, 8 in 2024-09-19T08:57:50.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:50 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:57:50.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:50 smithi153 ceph-mon[35071]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:57:50.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:50 smithi153 ceph-mon[35071]: from='client.? 172.21.15.153:0/1370170970' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3b1ead3-150f-43fd-8924-f639203f4cdd"}]: dispatch 2024-09-19T08:57:50.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:50 smithi153 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3b1ead3-150f-43fd-8924-f639203f4cdd"}]: dispatch 2024-09-19T08:57:50.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:50 smithi153 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3b1ead3-150f-43fd-8924-f639203f4cdd"}]': finished 2024-09-19T08:57:50.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:50 smithi153 ceph-mon[35071]: osdmap e45: 8 total, 7 up, 8 in 2024-09-19T08:57:50.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:50 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:57:50.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:50 smithi120 ceph-mon[31150]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:57:50.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:50 smithi120 ceph-mon[31150]: from='client.? 172.21.15.153:0/1370170970' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3b1ead3-150f-43fd-8924-f639203f4cdd"}]: dispatch 2024-09-19T08:57:50.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:50 smithi120 ceph-mon[31150]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3b1ead3-150f-43fd-8924-f639203f4cdd"}]: dispatch 2024-09-19T08:57:50.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:50 smithi120 ceph-mon[31150]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3b1ead3-150f-43fd-8924-f639203f4cdd"}]': finished 2024-09-19T08:57:50.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:50 smithi120 ceph-mon[31150]: osdmap e45: 8 total, 7 up, 8 in 2024-09-19T08:57:50.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:50 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:57:51.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:51 smithi196 ceph-mon[35150]: from='client.? 172.21.15.153:0/4045623416' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:57:51.686 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:51 smithi153 ceph-mon[35071]: from='client.? 172.21.15.153:0/4045623416' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:57:51.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:51 smithi120 ceph-mon[31150]: from='client.? 172.21.15.153:0/4045623416' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:57:52.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:52 smithi196 ceph-mon[35150]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:57:52.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:52 smithi153 ceph-mon[35071]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:57:52.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:52 smithi120 ceph-mon[31150]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:57:54.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:54 smithi196 ceph-mon[35150]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:57:54.936 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:54 smithi153 ceph-mon[35071]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:57:55.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:54 smithi120 ceph-mon[31150]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:57:56.793 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:56 smithi153 ceph-mon[35071]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:57:56.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:56 smithi196 ceph-mon[35150]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:57:57.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:56 smithi120 ceph-mon[31150]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:57:58.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:58 smithi153 ceph-mon[35071]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:57:58.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:58 smithi196 ceph-mon[35150]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:57:59.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:58 smithi120 ceph-mon[31150]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:57:59.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:59 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T08:57:59.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:59 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:57:59.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:57:59 smithi153 ceph-mon[35071]: Deploying daemon osd.7 on smithi153 2024-09-19T08:57:59.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:59 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T08:57:59.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:59 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:57:59.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:57:59 smithi196 ceph-mon[35150]: Deploying daemon osd.7 on smithi153 2024-09-19T08:58:00.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:59 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T08:58:00.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:59 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:58:00.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:57:59 smithi120 ceph-mon[31150]: Deploying daemon osd.7 on smithi153 2024-09-19T08:58:00.723 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:00 smithi153 ceph-mon[35071]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:58:00.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:00 smithi196 ceph-mon[35150]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:58:01.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:00 smithi120 ceph-mon[31150]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:58:02.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:02 smithi196 ceph-mon[35150]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:58:02.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:02 smithi153 ceph-mon[35071]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:58:03.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:02 smithi120 ceph-mon[31150]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:58:04.739 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:04 smithi153 ceph-mon[35071]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:58:04.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:04 smithi196 ceph-mon[35150]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:58:05.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:04 smithi120 ceph-mon[31150]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:58:05.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:05 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:58:05.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:05 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:05.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:05 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:05.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:05 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:58:05.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:05 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:05.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:05 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:06.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:05 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:58:06.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:05 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:06.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:05 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:06.925 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:06 smithi153 ceph-mon[35071]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:58:06.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:06 smithi196 ceph-mon[35150]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:58:07.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:06 smithi120 ceph-mon[31150]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:58:08.653 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:58:08.651+0000 7f202e7fc640 1 -- 172.21.15.153:0/469980849 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f203806aed0 con 0x7f20240a32f0 2024-09-19T08:58:08.653 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 7 on host 'smithi153' 2024-09-19T08:58:08.654 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:58:08.654+0000 7f203f384640 1 -- 172.21.15.153:0/469980849 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f20240a32f0 msgr2=0x7f20240a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:08.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:58:08.654+0000 7f203f384640 1 --2- 172.21.15.153:0/469980849 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f20240a32f0 0x7f20240a57b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2038101b70 tx=0x7f203400b390 comp rx=0 tx=0).stop 2024-09-19T08:58:08.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:58:08.654+0000 7f203f384640 1 -- 172.21.15.153:0/469980849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2038101d10 msgr2=0x7f2038198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:08.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:58:08.654+0000 7f203f384640 1 --2- 172.21.15.153:0/469980849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2038101d10 0x7f2038198460 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2028009230 tx=0x7f2028004360 comp rx=0 tx=0).stop 2024-09-19T08:58:08.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:58:08.654+0000 7f203f384640 1 -- 172.21.15.153:0/469980849 shutdown_connections 2024-09-19T08:58:08.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:58:08.654+0000 7f203f384640 1 --2- 172.21.15.153:0/469980849 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f20240a32f0 0x7f20240a57b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:08.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:58:08.654+0000 7f203f384640 1 --2- 172.21.15.153:0/469980849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20381026d0 0x7f203819f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:08.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:58:08.654+0000 7f203f384640 1 --2- 172.21.15.153:0/469980849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2038101d10 0x7f2038198460 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:08.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:58:08.654+0000 7f203f384640 1 --2- 172.21.15.153:0/469980849 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2038100b10 0x7f2038197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:08.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:58:08.654+0000 7f203f384640 1 -- 172.21.15.153:0/469980849 >> 172.21.15.153:0/469980849 conn(0x7f20380fc2a0 msgr2=0x7f203806a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:58:08.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:58:08.654+0000 7f203f384640 1 -- 172.21.15.153:0/469980849 shutdown_connections 2024-09-19T08:58:08.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:58:08.654+0000 7f203f384640 1 -- 172.21.15.153:0/469980849 wait complete. 2024-09-19T08:58:08.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:08 smithi196 ceph-mon[35150]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:58:08.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:08 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:08.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:08 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:08.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:08 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:58:08.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:08 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:58:08.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:08 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:08.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:08 smithi153 ceph-mon[35071]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:58:08.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:08 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:08.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:08 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:08.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:08 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:58:08.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:08 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:58:08.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:08 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:09.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:08 smithi120 ceph-mon[31150]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:58:09.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:08 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:09.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:08 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:09.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:08 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:58:09.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:08 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:58:09.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:08 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:09.281 DEBUG:teuthology.orchestra.run.smithi153:osd.7> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.7.service 2024-09-19T08:58:09.283 INFO:tasks.cephadm:Deploying osd.8 on smithi196 with /dev/vg_nvme/lv_2... 2024-09-19T08:58:09.283 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T08:58:09.537 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:58:09.912 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:09 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:58:09.912 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:09 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:09.912 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:09 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:09.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:09 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:58:09.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:09 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:09.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:09 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:10.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:09 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:58:10.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:09 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:10.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:09 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:10.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:10 smithi196 ceph-mon[35150]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:58:10.936 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:10 smithi153 ceph-mon[35071]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:58:11.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:10 smithi120 ceph-mon[31150]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:58:11.422 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 08:58:10 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T08:58:10.962+0000 7f84100cc640 -1 osd.7 0 log_to_monitors true 2024-09-19T08:58:11.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:11 smithi153 ceph-mon[35071]: from='osd.7 [v2:172.21.15.153:6816/2535148453,v1:172.21.15.153:6817/2535148453]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T08:58:11.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:11 smithi153 ceph-mon[35071]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T08:58:11.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:11 smithi196 ceph-mon[35150]: from='osd.7 [v2:172.21.15.153:6816/2535148453,v1:172.21.15.153:6817/2535148453]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T08:58:11.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:11 smithi196 ceph-mon[35150]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T08:58:12.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:11 smithi120 ceph-mon[31150]: from='osd.7 [v2:172.21.15.153:6816/2535148453,v1:172.21.15.153:6817/2535148453]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T08:58:12.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:11 smithi120 ceph-mon[31150]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T08:58:12.311 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T08:58:12.337 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_2 2024-09-19T08:58:12.596 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:58:12.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:12 smithi196 ceph-mon[35150]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:58:12.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:12 smithi196 ceph-mon[35150]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T08:58:12.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:12 smithi196 ceph-mon[35150]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T08:58:12.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:12 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:58:12.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:12 smithi196 ceph-mon[35150]: from='osd.7 [v2:172.21.15.153:6816/2535148453,v1:172.21.15.153:6817/2535148453]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-19T08:58:12.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:12 smithi196 ceph-mon[35150]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-19T08:58:12.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:12 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:12.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:12 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:12.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:12 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:58:12.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:12 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:58:12.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:12 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:58:12.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:12 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:12.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:12 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:58:12.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:12 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:58:12.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:12 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:12.937 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 08:58:12 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T08:58:12.668+0000 7f840c04f640 -1 osd.7 0 waiting for initial osdmap 2024-09-19T08:58:12.937 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 08:58:12 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T08:58:12.676+0000 7f8407e77640 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:58:12.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:12 smithi153 ceph-mon[35071]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:58:12.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:12 smithi153 ceph-mon[35071]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T08:58:12.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:12 smithi153 ceph-mon[35071]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T08:58:12.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:12 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:58:12.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:12 smithi153 ceph-mon[35071]: from='osd.7 [v2:172.21.15.153:6816/2535148453,v1:172.21.15.153:6817/2535148453]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-19T08:58:12.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:12 smithi153 ceph-mon[35071]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-19T08:58:12.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:12 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:12.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:12 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:12.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:12 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:58:12.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:12 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:58:12.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:12 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:58:12.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:12 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:12.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:12 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:58:12.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:12 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:58:12.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:12 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:13.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:12 smithi120 ceph-mon[31150]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:58:13.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:12 smithi120 ceph-mon[31150]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T08:58:13.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:12 smithi120 ceph-mon[31150]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T08:58:13.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:58:13.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:12 smithi120 ceph-mon[31150]: from='osd.7 [v2:172.21.15.153:6816/2535148453,v1:172.21.15.153:6817/2535148453]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-19T08:58:13.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:12 smithi120 ceph-mon[31150]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-19T08:58:13.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:13.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:13.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:58:13.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:58:13.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:58:13.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:13.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:58:13.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:58:13.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:13.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.078+0000 7fd7908eb640 1 -- 172.21.15.196:0/2314466231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd78c100b10 msgr2=0x7fd78c100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:13.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.078+0000 7fd7908eb640 1 --2- 172.21.15.196:0/2314466231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd78c100b10 0x7fd78c100f10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd77c009900 tx=0x7fd77c02a410 comp rx=0 tx=0).stop 2024-09-19T08:58:13.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.078+0000 7fd7908eb640 1 -- 172.21.15.196:0/2314466231 shutdown_connections 2024-09-19T08:58:13.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.078+0000 7fd7908eb640 1 --2- 172.21.15.196:0/2314466231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd78c1026d0 0x7fd78c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:13.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.078+0000 7fd7908eb640 1 --2- 172.21.15.196:0/2314466231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd78c101d10 0x7fd78c102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:13.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.078+0000 7fd7908eb640 1 --2- 172.21.15.196:0/2314466231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd78c100b10 0x7fd78c100f10 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:13.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.078+0000 7fd7908eb640 1 -- 172.21.15.196:0/2314466231 >> 172.21.15.196:0/2314466231 conn(0x7fd78c0fc2a0 msgr2=0x7fd78c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:58:13.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.078+0000 7fd7908eb640 1 -- 172.21.15.196:0/2314466231 shutdown_connections 2024-09-19T08:58:13.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.078+0000 7fd7908eb640 1 -- 172.21.15.196:0/2314466231 wait complete. 2024-09-19T08:58:13.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.079+0000 7fd7908eb640 1 Processor -- start 2024-09-19T08:58:13.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.079+0000 7fd7908eb640 1 -- start start 2024-09-19T08:58:13.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.079+0000 7fd7908eb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd78c100b10 0x7fd78c197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:13.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.079+0000 7fd7908eb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd78c101d10 0x7fd78c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:13.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.079+0000 7fd7908eb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd78c1026d0 0x7fd78c19f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:13.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.079+0000 7fd7908eb640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c10d880 con 0x7fd78c100b10 2024-09-19T08:58:13.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.079+0000 7fd7908eb640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c10d700 con 0x7fd78c1026d0 2024-09-19T08:58:13.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.079+0000 7fd7908eb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c10da00 con 0x7fd78c101d10 2024-09-19T08:58:13.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.079+0000 7fd783fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd78c101d10 0x7fd78c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:13.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.079+0000 7fd783fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd78c101d10 0x7fd78c198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34896/0 (socket says 172.21.15.196:34896) 2024-09-19T08:58:13.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.079+0000 7fd783fff640 1 -- 172.21.15.196:0/1105705339 learned_addr learned my addr 172.21.15.196:0/1105705339 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-19T08:58:13.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.079+0000 7fd78a59c640 1 --2- 172.21.15.196:0/1105705339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd78c100b10 0x7fd78c197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:13.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.079+0000 7fd78ad9d640 1 --2- 172.21.15.196:0/1105705339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd78c1026d0 0x7fd78c19f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:13.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.079+0000 7fd783fff640 1 -- 172.21.15.196:0/1105705339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd78c1026d0 msgr2=0x7fd78c19f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:13.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.079+0000 7fd783fff640 1 --2- 172.21.15.196:0/1105705339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd78c1026d0 0x7fd78c19f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:13.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.080+0000 7fd783fff640 1 -- 172.21.15.196:0/1105705339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd78c100b10 msgr2=0x7fd78c197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:13.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.080+0000 7fd783fff640 1 --2- 172.21.15.196:0/1105705339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd78c100b10 0x7fd78c197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:13.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.080+0000 7fd783fff640 1 -- 172.21.15.196:0/1105705339 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd78c19fb70 con 0x7fd78c101d10 2024-09-19T08:58:13.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.080+0000 7fd783fff640 1 --2- 172.21.15.196:0/1105705339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd78c101d10 0x7fd78c198480 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd77400e9f0 tx=0x7fd77400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:58:13.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.080+0000 7fd7837fe640 1 -- 172.21.15.196:0/1105705339 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77400cd80 con 0x7fd78c101d10 2024-09-19T08:58:13.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.080+0000 7fd7908eb640 1 -- 172.21.15.196:0/1105705339 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd78c19fe60 con 0x7fd78c101d10 2024-09-19T08:58:13.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.080+0000 7fd7837fe640 1 -- 172.21.15.196:0/1105705339 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd774010910 con 0x7fd78c101d10 2024-09-19T08:58:13.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.080+0000 7fd7837fe640 1 -- 172.21.15.196:0/1105705339 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7740188f0 con 0x7fd78c101d10 2024-09-19T08:58:13.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.080+0000 7fd7908eb640 1 -- 172.21.15.196:0/1105705339 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd78c1a0360 con 0x7fd78c101d10 2024-09-19T08:58:13.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.081+0000 7fd7908eb640 1 -- 172.21.15.196:0/1105705339 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd78c107f30 con 0x7fd78c101d10 2024-09-19T08:58:13.098 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.082+0000 7fd7837fe640 1 -- 172.21.15.196:0/1105705339 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fd774010ac0 con 0x7fd78c101d10 2024-09-19T08:58:13.098 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.083+0000 7fd7837fe640 1 --2- 172.21.15.196:0/1105705339 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd7780a3270 0x7fd7780a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:13.098 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.083+0000 7fd7837fe640 1 -- 172.21.15.196:0/1105705339 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5117+0+0 (secure 0 0 0) 0x7fd7740cf350 con 0x7fd78c101d10 2024-09-19T08:58:13.098 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.083+0000 7fd78a59c640 1 --2- 172.21.15.196:0/1105705339 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd7780a3270 0x7fd7780a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:13.098 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.083+0000 7fd78a59c640 1 --2- 172.21.15.196:0/1105705339 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd7780a3270 0x7fd7780a5730 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd77c005ab0 tx=0x7fd77c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:58:13.098 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.086+0000 7fd7837fe640 1 -- 172.21.15.196:0/1105705339 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd77409c6a0 con 0x7fd78c101d10 2024-09-19T08:58:13.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:13.261+0000 7fd7908eb640 1 -- 172.21.15.196:0/1105705339 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd78c062630 con 0x7fd7780a3270 2024-09-19T08:58:13.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:13 smithi196 ceph-mon[35150]: Detected new or changed devices on smithi153 2024-09-19T08:58:13.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:13 smithi196 ceph-mon[35150]: Adjusting osd_memory_target on smithi153 to 5696M 2024-09-19T08:58:13.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:13 smithi196 ceph-mon[35150]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-19T08:58:13.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:13 smithi196 ceph-mon[35150]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T08:58:13.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:13 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:58:13.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:13 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:58:13.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:13 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:58:13.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:13 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:58:13.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:13 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:58:13.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:13 smithi153 ceph-mon[35071]: Detected new or changed devices on smithi153 2024-09-19T08:58:13.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:13 smithi153 ceph-mon[35071]: Adjusting osd_memory_target on smithi153 to 5696M 2024-09-19T08:58:13.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:13 smithi153 ceph-mon[35071]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-19T08:58:13.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:13 smithi153 ceph-mon[35071]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T08:58:13.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:13 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:58:13.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:13 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:58:13.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:13 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:58:13.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:13 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:58:13.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:13 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:58:14.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:13 smithi120 ceph-mon[31150]: Detected new or changed devices on smithi153 2024-09-19T08:58:14.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:13 smithi120 ceph-mon[31150]: Adjusting osd_memory_target on smithi153 to 5696M 2024-09-19T08:58:14.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:13 smithi120 ceph-mon[31150]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-19T08:58:14.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:13 smithi120 ceph-mon[31150]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T08:58:14.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:13 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:58:14.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:13 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:58:14.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:13 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:58:14.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:13 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:58:14.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:13 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:58:14.839 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:14 smithi196 ceph-mon[35150]: purged_snaps scrub starts 2024-09-19T08:58:14.839 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:14 smithi196 ceph-mon[35150]: purged_snaps scrub ok 2024-09-19T08:58:14.839 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:14 smithi196 ceph-mon[35150]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:58:14.839 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:14 smithi196 ceph-mon[35150]: pgmap v168: 1 pgs: 1 unknown; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:58:14.839 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:14 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:58:14.839 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:14 smithi196 ceph-mon[35150]: osd.7 [v2:172.21.15.153:6816/2535148453,v1:172.21.15.153:6817/2535148453] boot 2024-09-19T08:58:14.839 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:14 smithi196 ceph-mon[35150]: osdmap e48: 8 total, 8 up, 8 in 2024-09-19T08:58:14.839 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:14 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:58:14.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:14 smithi153 ceph-mon[35071]: purged_snaps scrub starts 2024-09-19T08:58:14.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:14 smithi153 ceph-mon[35071]: purged_snaps scrub ok 2024-09-19T08:58:14.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:14 smithi153 ceph-mon[35071]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:58:14.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:14 smithi153 ceph-mon[35071]: pgmap v168: 1 pgs: 1 unknown; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:58:14.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:14 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:58:14.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:14 smithi153 ceph-mon[35071]: osd.7 [v2:172.21.15.153:6816/2535148453,v1:172.21.15.153:6817/2535148453] boot 2024-09-19T08:58:14.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:14 smithi153 ceph-mon[35071]: osdmap e48: 8 total, 8 up, 8 in 2024-09-19T08:58:14.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:14 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:58:15.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:14 smithi120 ceph-mon[31150]: purged_snaps scrub starts 2024-09-19T08:58:15.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:14 smithi120 ceph-mon[31150]: purged_snaps scrub ok 2024-09-19T08:58:15.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:14 smithi120 ceph-mon[31150]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:58:15.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:14 smithi120 ceph-mon[31150]: pgmap v168: 1 pgs: 1 unknown; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:58:15.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:14 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:58:15.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:14 smithi120 ceph-mon[31150]: osd.7 [v2:172.21.15.153:6816/2535148453,v1:172.21.15.153:6817/2535148453] boot 2024-09-19T08:58:15.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:14 smithi120 ceph-mon[31150]: osdmap e48: 8 total, 8 up, 8 in 2024-09-19T08:58:15.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:14 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:58:15.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:15 smithi196 ceph-mon[35150]: osdmap e49: 8 total, 8 up, 8 in 2024-09-19T08:58:15.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:15 smithi196 ceph-mon[35150]: from='client.? 172.21.15.196:0/2228152542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3273a2a-968e-4e93-a9cf-3879892bdeab"}]: dispatch 2024-09-19T08:58:15.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:15 smithi196 ceph-mon[35150]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3273a2a-968e-4e93-a9cf-3879892bdeab"}]: dispatch 2024-09-19T08:58:15.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:15 smithi196 ceph-mon[35150]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3273a2a-968e-4e93-a9cf-3879892bdeab"}]': finished 2024-09-19T08:58:15.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:15 smithi196 ceph-mon[35150]: osdmap e50: 9 total, 8 up, 9 in 2024-09-19T08:58:15.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:15 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:58:15.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:15 smithi153 ceph-mon[35071]: osdmap e49: 8 total, 8 up, 8 in 2024-09-19T08:58:15.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:15 smithi153 ceph-mon[35071]: from='client.? 172.21.15.196:0/2228152542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3273a2a-968e-4e93-a9cf-3879892bdeab"}]: dispatch 2024-09-19T08:58:15.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:15 smithi153 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3273a2a-968e-4e93-a9cf-3879892bdeab"}]: dispatch 2024-09-19T08:58:15.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:15 smithi153 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3273a2a-968e-4e93-a9cf-3879892bdeab"}]': finished 2024-09-19T08:58:15.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:15 smithi153 ceph-mon[35071]: osdmap e50: 9 total, 8 up, 9 in 2024-09-19T08:58:15.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:15 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:58:16.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:15 smithi120 ceph-mon[31150]: osdmap e49: 8 total, 8 up, 8 in 2024-09-19T08:58:16.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:15 smithi120 ceph-mon[31150]: from='client.? 172.21.15.196:0/2228152542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3273a2a-968e-4e93-a9cf-3879892bdeab"}]: dispatch 2024-09-19T08:58:16.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:15 smithi120 ceph-mon[31150]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3273a2a-968e-4e93-a9cf-3879892bdeab"}]: dispatch 2024-09-19T08:58:16.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:15 smithi120 ceph-mon[31150]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3273a2a-968e-4e93-a9cf-3879892bdeab"}]': finished 2024-09-19T08:58:16.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:15 smithi120 ceph-mon[31150]: osdmap e50: 9 total, 8 up, 9 in 2024-09-19T08:58:16.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:15 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:58:17.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:16 smithi120 ceph-mon[31150]: pgmap v172: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:17.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:16 smithi120 ceph-mon[31150]: from='client.? 172.21.15.196:0/3843329120' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:58:17.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:16 smithi120 ceph-mon[31150]: osdmap e51: 9 total, 8 up, 9 in 2024-09-19T08:58:17.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:58:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:16 smithi196 ceph-mon[35150]: pgmap v172: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:16 smithi196 ceph-mon[35150]: from='client.? 172.21.15.196:0/3843329120' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:58:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:16 smithi196 ceph-mon[35150]: osdmap e51: 9 total, 8 up, 9 in 2024-09-19T08:58:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:58:17.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:16 smithi153 ceph-mon[35071]: pgmap v172: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:17.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:16 smithi153 ceph-mon[35071]: from='client.? 172.21.15.196:0/3843329120' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:58:17.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:16 smithi153 ceph-mon[35071]: osdmap e51: 9 total, 8 up, 9 in 2024-09-19T08:58:17.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:58:18.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:18 smithi196 ceph-mon[35150]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T08:58:18.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:18 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:58:18.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:18 smithi196 ceph-mon[35150]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:18.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:18 smithi153 ceph-mon[35071]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T08:58:18.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:18 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:58:18.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:18 smithi153 ceph-mon[35071]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:18.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:18 smithi120 ceph-mon[31150]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T08:58:18.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:18 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:58:18.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:18 smithi120 ceph-mon[31150]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:20.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:20 smithi196 ceph-mon[35150]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:20.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:20 smithi153 ceph-mon[35071]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:21.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:20 smithi120 ceph-mon[31150]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:22.902 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:22 smithi196 ceph-mon[35150]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:22.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:22 smithi153 ceph-mon[35071]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:23.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:22 smithi120 ceph-mon[31150]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:24.929 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:24 smithi196 ceph-mon[35150]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:24.929 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:24 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T08:58:24.929 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:24 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:58:24.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:24 smithi153 ceph-mon[35071]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:24.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:24 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T08:58:24.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:24 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:58:25.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:24 smithi120 ceph-mon[31150]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:25.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:24 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T08:58:25.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:24 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:58:25.908 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:25 smithi196 ceph-mon[35150]: Deploying daemon osd.8 on smithi196 2024-09-19T08:58:25.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:25 smithi153 ceph-mon[35071]: Deploying daemon osd.8 on smithi196 2024-09-19T08:58:26.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:25 smithi120 ceph-mon[31150]: Deploying daemon osd.8 on smithi196 2024-09-19T08:58:26.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:26 smithi196 ceph-mon[35150]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:26.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:26 smithi153 ceph-mon[35071]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:27.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:26 smithi120 ceph-mon[31150]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:28.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:28 smithi196 ceph-mon[35150]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:28.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:28 smithi153 ceph-mon[35071]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:29.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:28 smithi120 ceph-mon[31150]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:30.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:30 smithi196 ceph-mon[35150]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:30.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:30 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:58:30.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:30 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:30.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:30 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:30.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:30 smithi153 ceph-mon[35071]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:30.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:30 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:58:30.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:30 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:30.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:30 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:31.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:30 smithi120 ceph-mon[31150]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:31.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:30 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:58:31.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:30 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:31.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:30 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:32.883 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:32 smithi196 ceph-mon[35150]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:32.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:32 smithi153 ceph-mon[35071]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:33.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:32 smithi120 ceph-mon[31150]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:34.037 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:33 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:34.037 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:33 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:34.037 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:33 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:58:34.037 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:33 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:58:34.037 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:33 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:34.037 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:33 smithi196 ceph-mon[35150]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:34.077 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:34.075+0000 7fd7837fe640 1 -- 172.21.15.196:0/1105705339 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd78c062630 con 0x7fd7780a3270 2024-09-19T08:58:34.077 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 8 on host 'smithi196' 2024-09-19T08:58:34.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:34.078+0000 7fd7908eb640 1 -- 172.21.15.196:0/1105705339 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd7780a3270 msgr2=0x7fd7780a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:34.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:34.078+0000 7fd7908eb640 1 --2- 172.21.15.196:0/1105705339 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd7780a3270 0x7fd7780a5730 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd77c005ab0 tx=0x7fd77c000bc0 comp rx=0 tx=0).stop 2024-09-19T08:58:34.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:34.078+0000 7fd7908eb640 1 -- 172.21.15.196:0/1105705339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd78c101d10 msgr2=0x7fd78c198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:34.079 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:34.078+0000 7fd7908eb640 1 --2- 172.21.15.196:0/1105705339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd78c101d10 0x7fd78c198480 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd77400e9f0 tx=0x7fd77400eeb0 comp rx=0 tx=0).stop 2024-09-19T08:58:34.079 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:34.078+0000 7fd7908eb640 1 -- 172.21.15.196:0/1105705339 shutdown_connections 2024-09-19T08:58:34.079 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:34.078+0000 7fd7908eb640 1 --2- 172.21.15.196:0/1105705339 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd7780a3270 0x7fd7780a5730 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:34.079 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:34.078+0000 7fd7908eb640 1 --2- 172.21.15.196:0/1105705339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd78c1026d0 0x7fd78c19f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:34.079 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:34.078+0000 7fd7908eb640 1 --2- 172.21.15.196:0/1105705339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd78c101d10 0x7fd78c198480 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:34.079 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:34.078+0000 7fd7908eb640 1 --2- 172.21.15.196:0/1105705339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd78c100b10 0x7fd78c197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:34.079 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:34.078+0000 7fd7908eb640 1 -- 172.21.15.196:0/1105705339 >> 172.21.15.196:0/1105705339 conn(0x7fd78c0fc2a0 msgr2=0x7fd78c105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:58:34.079 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:34.078+0000 7fd7908eb640 1 -- 172.21.15.196:0/1105705339 shutdown_connections 2024-09-19T08:58:34.079 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:58:34.078+0000 7fd7908eb640 1 -- 172.21.15.196:0/1105705339 wait complete. 2024-09-19T08:58:34.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:33 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:34.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:33 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:34.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:33 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:58:34.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:33 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:58:34.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:33 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:34.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:33 smithi120 ceph-mon[31150]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:34.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:33 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:34.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:33 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:34.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:33 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:58:34.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:33 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:58:34.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:33 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:34.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:33 smithi153 ceph-mon[35071]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:34.767 DEBUG:teuthology.orchestra.run.smithi196:osd.8> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.8.service 2024-09-19T08:58:34.769 INFO:tasks.cephadm:Deploying osd.9 on smithi120 with /dev/vg_nvme/lv_1... 2024-09-19T08:58:34.769 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T08:58:35.024 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:58:35.035 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:34 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:58:35.035 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:34 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:35.035 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:34 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:35.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:34 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:58:35.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:34 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:35.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:34 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:35.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:34 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:58:35.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:34 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:35.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:34 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:36.158 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:35 smithi196 ceph-mon[35150]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:36.158 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 08:58:36 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T08:58:36.130+0000 7fc1b95d6640 -1 osd.8 0 log_to_monitors true 2024-09-19T08:58:36.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:35 smithi120 ceph-mon[31150]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:36.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:35 smithi153 ceph-mon[35071]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:37.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:36 smithi196 ceph-mon[35150]: from='osd.8 [v2:172.21.15.196:6816/186195906,v1:172.21.15.196:6817/186195906]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T08:58:37.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:36 smithi196 ceph-mon[35150]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T08:58:37.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:36 smithi120 ceph-mon[31150]: from='osd.8 [v2:172.21.15.196:6816/186195906,v1:172.21.15.196:6817/186195906]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T08:58:37.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:36 smithi120 ceph-mon[31150]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T08:58:37.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:36 smithi153 ceph-mon[35071]: from='osd.8 [v2:172.21.15.196:6816/186195906,v1:172.21.15.196:6817/186195906]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T08:58:37.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:36 smithi153 ceph-mon[35071]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T08:58:37.737 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:58:37.763 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi120:vg_nvme/lv_1 2024-09-19T08:58:37.991 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:37 smithi120 ceph-mon[31150]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T08:58:37.991 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:37 smithi120 ceph-mon[31150]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T08:58:37.991 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:37 smithi120 ceph-mon[31150]: from='osd.8 [v2:172.21.15.196:6816/186195906,v1:172.21.15.196:6817/186195906]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T08:58:37.991 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:37 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:58:37.991 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:37 smithi120 ceph-mon[31150]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T08:58:37.991 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:37 smithi120 ceph-mon[31150]: Detected new or changed devices on smithi196 2024-09-19T08:58:37.991 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:37 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:37.992 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:37 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:37.992 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:37 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:58:37.992 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:37 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:58:37.992 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:37 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:58:37.992 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:37 smithi120 ceph-mon[31150]: Adjusting osd_memory_target on smithi196 to 5696M 2024-09-19T08:58:37.992 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:37 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:37.992 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:37 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:58:37.992 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:37 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:58:37.992 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:37 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:37.993 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:37 smithi120 ceph-mon[31150]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:37.993 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:37 smithi120 ceph-mon[31150]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-19T08:58:37.993 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:37 smithi120 ceph-mon[31150]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T08:58:37.993 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:37 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:58:38.021 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:58:38.363 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:37 smithi196 ceph-mon[35150]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T08:58:38.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:37 smithi196 ceph-mon[35150]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T08:58:38.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:37 smithi196 ceph-mon[35150]: from='osd.8 [v2:172.21.15.196:6816/186195906,v1:172.21.15.196:6817/186195906]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T08:58:38.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:37 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:58:38.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:37 smithi196 ceph-mon[35150]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T08:58:38.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:37 smithi196 ceph-mon[35150]: Detected new or changed devices on smithi196 2024-09-19T08:58:38.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:37 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:38.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:37 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:38.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:37 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:58:38.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:37 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:58:38.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:37 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:58:38.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:37 smithi196 ceph-mon[35150]: Adjusting osd_memory_target on smithi196 to 5696M 2024-09-19T08:58:38.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:37 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:38.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:37 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:58:38.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:37 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:58:38.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:37 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:38.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:37 smithi196 ceph-mon[35150]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:38.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:37 smithi196 ceph-mon[35150]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-19T08:58:38.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:37 smithi196 ceph-mon[35150]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T08:58:38.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:37 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:58:38.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:37 smithi153 ceph-mon[35071]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T08:58:38.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:37 smithi153 ceph-mon[35071]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T08:58:38.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:37 smithi153 ceph-mon[35071]: from='osd.8 [v2:172.21.15.196:6816/186195906,v1:172.21.15.196:6817/186195906]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T08:58:38.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:37 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:58:38.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:37 smithi153 ceph-mon[35071]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T08:58:38.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:37 smithi153 ceph-mon[35071]: Detected new or changed devices on smithi196 2024-09-19T08:58:38.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:37 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:38.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:37 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:38.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:37 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:58:38.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:37 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:58:38.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:37 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:58:38.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:37 smithi153 ceph-mon[35071]: Adjusting osd_memory_target on smithi196 to 5696M 2024-09-19T08:58:38.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:37 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:38.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:37 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:58:38.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:37 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:58:38.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:37 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:38.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:37 smithi153 ceph-mon[35071]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:38.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:37 smithi153 ceph-mon[35071]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-19T08:58:38.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:37 smithi153 ceph-mon[35071]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T08:58:38.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:37 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:58:38.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.513+0000 7f54aa907640 1 -- 172.21.15.120:0/4009601040 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54a4107920 msgr2=0x7f54a4109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:38.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.513+0000 7f54aa907640 1 --2- 172.21.15.120:0/4009601040 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54a4107920 0x7f54a4109d10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f549c009ca0 tx=0x7f549c02a8c0 comp rx=0 tx=0).stop 2024-09-19T08:58:38.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.514+0000 7f54aa907640 1 -- 172.21.15.120:0/4009601040 shutdown_connections 2024-09-19T08:58:38.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.514+0000 7f54aa907640 1 --2- 172.21.15.120:0/4009601040 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54a4107920 0x7f54a4109d10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:38.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.514+0000 7f54aa907640 1 --2- 172.21.15.120:0/4009601040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54a4104ff0 0x7f54a41073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:38.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.514+0000 7f54aa907640 1 --2- 172.21.15.120:0/4009601040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54a41026c0 0x7f54a4104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:38.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.514+0000 7f54aa907640 1 -- 172.21.15.120:0/4009601040 >> 172.21.15.120:0/4009601040 conn(0x7f54a40fc280 msgr2=0x7f54a40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:58:38.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.514+0000 7f54aa907640 1 -- 172.21.15.120:0/4009601040 shutdown_connections 2024-09-19T08:58:38.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.514+0000 7f54aa907640 1 -- 172.21.15.120:0/4009601040 wait complete. 2024-09-19T08:58:38.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.514+0000 7f54aa907640 1 Processor -- start 2024-09-19T08:58:38.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.514+0000 7f54aa907640 1 -- start start 2024-09-19T08:58:38.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.514+0000 7f54aa907640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54a41026c0 0x7f54a419a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:38.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.514+0000 7f54aa907640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54a4104ff0 0x7f54a419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:38.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.514+0000 7f54aa907640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54a4107920 0x7f54a41a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:38.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.515+0000 7f54aa907640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f54a410d880 con 0x7f54a4107920 2024-09-19T08:58:38.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.515+0000 7f54aa907640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f54a410d700 con 0x7f54a41026c0 2024-09-19T08:58:38.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.515+0000 7f54aa907640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f54a410da00 con 0x7f54a4104ff0 2024-09-19T08:58:38.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.515+0000 7f54a8ea4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54a4107920 0x7f54a41a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:38.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.515+0000 7f54a8ea4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54a4107920 0x7f54a41a1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53228/0 (socket says 172.21.15.120:53228) 2024-09-19T08:58:38.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.515+0000 7f54a8ea4640 1 -- 172.21.15.120:0/1460973984 learned_addr learned my addr 172.21.15.120:0/1460973984 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:58:38.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.515+0000 7f54a37fe640 1 --2- 172.21.15.120:0/1460973984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54a4104ff0 0x7f54a419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:38.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.515+0000 7f54a3fff640 1 --2- 172.21.15.120:0/1460973984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54a41026c0 0x7f54a419a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:38.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.515+0000 7f54a8ea4640 1 -- 172.21.15.120:0/1460973984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54a4104ff0 msgr2=0x7f54a419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:38.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.515+0000 7f54a8ea4640 1 --2- 172.21.15.120:0/1460973984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54a4104ff0 0x7f54a419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:38.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.515+0000 7f54a8ea4640 1 -- 172.21.15.120:0/1460973984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54a41026c0 msgr2=0x7f54a419a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:38.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.515+0000 7f54a8ea4640 1 --2- 172.21.15.120:0/1460973984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54a41026c0 0x7f54a419a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:38.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.515+0000 7f54a8ea4640 1 -- 172.21.15.120:0/1460973984 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a41a1cc0 con 0x7f54a4107920 2024-09-19T08:58:38.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.515+0000 7f54a37fe640 1 --2- 172.21.15.120:0/1460973984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54a4104ff0 0x7f54a419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:58:38.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.515+0000 7f54a8ea4640 1 --2- 172.21.15.120:0/1460973984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54a4107920 0x7f54a41a1650 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f549c005e30 tx=0x7f549c031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:58:38.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.515+0000 7f54a17fa640 1 -- 172.21.15.120:0/1460973984 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c003b70 con 0x7f54a4107920 2024-09-19T08:58:38.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.515+0000 7f54a17fa640 1 -- 172.21.15.120:0/1460973984 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f549c003d10 con 0x7f54a4107920 2024-09-19T08:58:38.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.515+0000 7f54aa907640 1 -- 172.21.15.120:0/1460973984 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a41a1f50 con 0x7f54a4107920 2024-09-19T08:58:38.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.515+0000 7f54a17fa640 1 -- 172.21.15.120:0/1460973984 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c02fd30 con 0x7f54a4107920 2024-09-19T08:58:38.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.516+0000 7f54aa907640 1 -- 172.21.15.120:0/1460973984 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a41a2450 con 0x7f54a4107920 2024-09-19T08:58:38.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.517+0000 7f54aa907640 1 -- 172.21.15.120:0/1460973984 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54a4109410 con 0x7f54a4107920 2024-09-19T08:58:38.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.518+0000 7f54a17fa640 1 -- 172.21.15.120:0/1460973984 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f549c004050 con 0x7f54a4107920 2024-09-19T08:58:38.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.518+0000 7f54a17fa640 1 --2- 172.21.15.120:0/1460973984 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f548c0a32f0 0x7f548c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:38.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.518+0000 7f54a3fff640 1 --2- 172.21.15.120:0/1460973984 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f548c0a32f0 0x7f548c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:38.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.519+0000 7f54a17fa640 1 -- 172.21.15.120:0/1460973984 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5549+0+0 (secure 0 0 0) 0x7f549c0eff70 con 0x7f54a4107920 2024-09-19T08:58:38.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.519+0000 7f54a3fff640 1 --2- 172.21.15.120:0/1460973984 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f548c0a32f0 0x7f548c0a57b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5494005fd0 tx=0x7f549400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:58:38.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.523+0000 7f54a17fa640 1 -- 172.21.15.120:0/1460973984 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f549c0bd880 con 0x7f54a4107920 2024-09-19T08:58:38.685 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 08:58:38 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T08:58:38.363+0000 7fc1b5559640 -1 osd.8 0 waiting for initial osdmap 2024-09-19T08:58:38.685 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 08:58:38 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T08:58:38.373+0000 7fc1ad379640 -1 osd.8 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:58:38.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:58:38.704+0000 7f54aa907640 1 -- 172.21.15.120:0/1460973984 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f54a4062630 con 0x7f548c0a32f0 2024-09-19T08:58:39.269 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:38 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:58:39.269 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:38 smithi120 ceph-mon[31150]: from='osd.8 ' entity='osd.8' 2024-09-19T08:58:39.269 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:38 smithi120 ceph-mon[31150]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:58:39.269 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:38 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:58:39.269 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:38 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:58:39.269 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:38 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:58:39.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:38 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:58:39.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:38 smithi196 ceph-mon[35150]: from='osd.8 ' entity='osd.8' 2024-09-19T08:58:39.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:38 smithi196 ceph-mon[35150]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:58:39.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:38 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:58:39.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:38 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:58:39.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:38 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:58:39.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:38 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:58:39.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:38 smithi153 ceph-mon[35071]: from='osd.8 ' entity='osd.8' 2024-09-19T08:58:39.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:38 smithi153 ceph-mon[35071]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:58:39.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:38 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:58:39.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:38 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:58:39.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:38 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:58:40.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:39 smithi120 ceph-mon[31150]: purged_snaps scrub starts 2024-09-19T08:58:40.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:40 smithi120 ceph-mon[31150]: purged_snaps scrub ok 2024-09-19T08:58:40.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:40 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:58:40.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:40 smithi120 ceph-mon[31150]: osd.8 [v2:172.21.15.196:6816/186195906,v1:172.21.15.196:6817/186195906] boot 2024-09-19T08:58:40.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:40 smithi120 ceph-mon[31150]: osdmap e55: 9 total, 9 up, 9 in 2024-09-19T08:58:40.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:40 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:58:40.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:40 smithi120 ceph-mon[31150]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:40.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:39 smithi196 ceph-mon[35150]: purged_snaps scrub starts 2024-09-19T08:58:40.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:39 smithi196 ceph-mon[35150]: purged_snaps scrub ok 2024-09-19T08:58:40.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:39 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:58:40.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:39 smithi196 ceph-mon[35150]: osd.8 [v2:172.21.15.196:6816/186195906,v1:172.21.15.196:6817/186195906] boot 2024-09-19T08:58:40.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:39 smithi196 ceph-mon[35150]: osdmap e55: 9 total, 9 up, 9 in 2024-09-19T08:58:40.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:39 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:58:40.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:39 smithi196 ceph-mon[35150]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:40.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:39 smithi153 ceph-mon[35071]: purged_snaps scrub starts 2024-09-19T08:58:40.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:39 smithi153 ceph-mon[35071]: purged_snaps scrub ok 2024-09-19T08:58:40.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:39 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:58:40.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:39 smithi153 ceph-mon[35071]: osd.8 [v2:172.21.15.196:6816/186195906,v1:172.21.15.196:6817/186195906] boot 2024-09-19T08:58:40.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:39 smithi153 ceph-mon[35071]: osdmap e55: 9 total, 9 up, 9 in 2024-09-19T08:58:40.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:39 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:58:40.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:39 smithi153 ceph-mon[35071]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:58:41.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:41 smithi120 ceph-mon[31150]: osdmap e56: 9 total, 9 up, 9 in 2024-09-19T08:58:41.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:41 smithi196 ceph-mon[35150]: osdmap e56: 9 total, 9 up, 9 in 2024-09-19T08:58:41.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:41 smithi153 ceph-mon[35071]: osdmap e56: 9 total, 9 up, 9 in 2024-09-19T08:58:42.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:42 smithi120 ceph-mon[31150]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T08:58:42.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:42 smithi120 ceph-mon[31150]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T08:58:42.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:42 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1621294286' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d184513-dab8-45f7-bc08-fa2e8b2f10b7"}]: dispatch 2024-09-19T08:58:42.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:42 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1621294286' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d184513-dab8-45f7-bc08-fa2e8b2f10b7"}]': finished 2024-09-19T08:58:42.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:42 smithi120 ceph-mon[31150]: osdmap e59: 10 total, 9 up, 10 in 2024-09-19T08:58:42.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:42 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:58:42.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:42 smithi120 ceph-mon[31150]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:58:42.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:42 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3596424504' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:58:42.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:42 smithi196 ceph-mon[35150]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T08:58:42.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:42 smithi196 ceph-mon[35150]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T08:58:42.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:42 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1621294286' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d184513-dab8-45f7-bc08-fa2e8b2f10b7"}]: dispatch 2024-09-19T08:58:42.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:42 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1621294286' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d184513-dab8-45f7-bc08-fa2e8b2f10b7"}]': finished 2024-09-19T08:58:42.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:42 smithi196 ceph-mon[35150]: osdmap e59: 10 total, 9 up, 10 in 2024-09-19T08:58:42.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:42 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:58:42.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:42 smithi196 ceph-mon[35150]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:58:42.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:42 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3596424504' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:58:42.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:42 smithi153 ceph-mon[35071]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T08:58:42.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:42 smithi153 ceph-mon[35071]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T08:58:42.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:42 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1621294286' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d184513-dab8-45f7-bc08-fa2e8b2f10b7"}]: dispatch 2024-09-19T08:58:42.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:42 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1621294286' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d184513-dab8-45f7-bc08-fa2e8b2f10b7"}]': finished 2024-09-19T08:58:42.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:42 smithi153 ceph-mon[35071]: osdmap e59: 10 total, 9 up, 10 in 2024-09-19T08:58:42.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:42 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:58:42.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:42 smithi153 ceph-mon[35071]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:58:42.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:42 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3596424504' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:58:44.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:44 smithi196 ceph-mon[35150]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:58:44.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:44 smithi153 ceph-mon[35071]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:58:45.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:44 smithi120 ceph-mon[31150]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:58:46.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:46 smithi196 ceph-mon[35150]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-19T08:58:46.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:46 smithi153 ceph-mon[35071]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-19T08:58:46.945 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:46 smithi120 ceph-mon[31150]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-19T08:58:48.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:48 smithi196 ceph-mon[35150]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-19T08:58:48.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:48 smithi153 ceph-mon[35071]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-19T08:58:48.955 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:48 smithi120 ceph-mon[31150]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-19T08:58:50.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:50 smithi196 ceph-mon[35150]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-19T08:58:50.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:50 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T08:58:50.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:50 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:58:50.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:50 smithi153 ceph-mon[35071]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-19T08:58:50.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:50 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T08:58:50.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:50 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:58:50.991 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:50 smithi120 ceph-mon[31150]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-19T08:58:50.991 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:50 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T08:58:50.991 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:50 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:58:51.795 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:51 smithi120 ceph-mon[31150]: Deploying daemon osd.9 on smithi120 2024-09-19T08:58:51.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:51 smithi196 ceph-mon[35150]: Deploying daemon osd.9 on smithi120 2024-09-19T08:58:51.936 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:51 smithi153 ceph-mon[35071]: Deploying daemon osd.9 on smithi120 2024-09-19T08:58:52.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:52 smithi120 ceph-mon[31150]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-19T08:58:52.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:52 smithi196 ceph-mon[35150]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-19T08:58:52.936 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:52 smithi153 ceph-mon[35071]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-19T08:58:54.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:54 smithi196 ceph-mon[35150]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T08:58:54.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:54 smithi153 ceph-mon[35071]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T08:58:55.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:54 smithi120 ceph-mon[31150]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T08:58:56.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:56 smithi196 ceph-mon[35150]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T08:58:56.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:56 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:58:56.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:56 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:56.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:56 smithi153 ceph-mon[35071]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T08:58:56.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:56 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:58:56.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:56 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:57.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:56 smithi120 ceph-mon[31150]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T08:58:57.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:56 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:58:57.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:56 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:57.774 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:57 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:57.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:57 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:57.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:57 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:58:58.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:58:58 smithi120 ceph-mon[31150]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:58:58.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:58:58 smithi196 ceph-mon[35150]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:58:58.936 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:58:58 smithi153 ceph-mon[35071]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:59:00.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:00.466+0000 7f54a17fa640 1 -- 172.21.15.120:0/1460973984 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f54a4062630 con 0x7f548c0a32f0 2024-09-19T08:59:00.468 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 9 on host 'smithi120' 2024-09-19T08:59:00.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:00.469+0000 7f54aa907640 1 -- 172.21.15.120:0/1460973984 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f548c0a32f0 msgr2=0x7f548c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:00.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:00.469+0000 7f54aa907640 1 --2- 172.21.15.120:0/1460973984 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f548c0a32f0 0x7f548c0a57b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5494005fd0 tx=0x7f549400b390 comp rx=0 tx=0).stop 2024-09-19T08:59:00.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:00.469+0000 7f54aa907640 1 -- 172.21.15.120:0/1460973984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54a4107920 msgr2=0x7f54a41a1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:00.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:00.469+0000 7f54aa907640 1 --2- 172.21.15.120:0/1460973984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54a4107920 0x7f54a41a1650 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f549c005e30 tx=0x7f549c031ee0 comp rx=0 tx=0).stop 2024-09-19T08:59:00.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:00.469+0000 7f54aa907640 1 -- 172.21.15.120:0/1460973984 shutdown_connections 2024-09-19T08:59:00.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:00.469+0000 7f54aa907640 1 --2- 172.21.15.120:0/1460973984 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f548c0a32f0 0x7f548c0a57b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:00.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:00.469+0000 7f54aa907640 1 --2- 172.21.15.120:0/1460973984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54a4107920 0x7f54a41a1650 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:00.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:00.469+0000 7f54aa907640 1 --2- 172.21.15.120:0/1460973984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54a4104ff0 0x7f54a419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:00.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:00.469+0000 7f54aa907640 1 --2- 172.21.15.120:0/1460973984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54a41026c0 0x7f54a419a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:00.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:00.469+0000 7f54aa907640 1 -- 172.21.15.120:0/1460973984 >> 172.21.15.120:0/1460973984 conn(0x7f54a40fc280 msgr2=0x7f54a4101c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:59:00.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:00.469+0000 7f54aa907640 1 -- 172.21.15.120:0/1460973984 shutdown_connections 2024-09-19T08:59:00.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:00.469+0000 7f54aa907640 1 -- 172.21.15.120:0/1460973984 wait complete. 2024-09-19T08:59:00.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:00 smithi120 ceph-mon[31150]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:59:00.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:00.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:00.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:00.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:59:00.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:00.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:59:00.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:00.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:00.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:00 smithi196 ceph-mon[35150]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:59:00.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:00.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:00.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:00.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:59:00.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:00.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:59:00.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:00.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:00.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:00 smithi153 ceph-mon[35071]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:59:00.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:00.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:00.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:00.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:59:00.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:00.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:59:00.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:00.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:01.161 DEBUG:teuthology.orchestra.run.smithi120:osd.9> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.9.service 2024-09-19T08:59:01.163 INFO:tasks.cephadm:Deploying osd.10 on smithi153 with /dev/vg_nvme/lv_1... 2024-09-19T08:59:01.164 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T08:59:01.420 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:59:02.702 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:02 smithi120 ceph-mon[31150]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:59:02.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:02 smithi196 ceph-mon[35150]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:59:02.936 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:02 smithi153 ceph-mon[35071]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:59:03.075 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 08:59:02 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T08:59:02.710+0000 7fda96f89640 -1 osd.9 0 log_to_monitors true 2024-09-19T08:59:03.767 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:03 smithi120 ceph-mon[31150]: from='osd.9 [v2:172.21.15.120:6826/1559320883,v1:172.21.15.120:6827/1559320883]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T08:59:03.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:03 smithi196 ceph-mon[35150]: from='osd.9 [v2:172.21.15.120:6826/1559320883,v1:172.21.15.120:6827/1559320883]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T08:59:03.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:03 smithi153 ceph-mon[35071]: from='osd.9 [v2:172.21.15.120:6826/1559320883,v1:172.21.15.120:6827/1559320883]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T08:59:04.094 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-19T08:59:04.120 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi153:vg_nvme/lv_1 2024-09-19T08:59:04.374 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:59:04.761 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:04 smithi153 ceph-mon[35071]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:59:04.761 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:04 smithi153 ceph-mon[35071]: from='osd.9 [v2:172.21.15.120:6826/1559320883,v1:172.21.15.120:6827/1559320883]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T08:59:04.761 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:04 smithi153 ceph-mon[35071]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T08:59:04.761 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:04 smithi153 ceph-mon[35071]: from='osd.9 [v2:172.21.15.120:6826/1559320883,v1:172.21.15.120:6827/1559320883]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-19T08:59:04.762 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:04 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:59:04.762 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:04 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:04.762 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:04 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:04.762 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:04 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:04.762 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:04 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:04.762 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:04 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:04.762 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:04 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:04.762 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:04 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:04.762 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:04 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:04.762 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:04 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:59:04.762 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:04 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:04.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.845+0000 7f83b67ab640 1 -- 172.21.15.153:0/3933480451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83b0102e00 msgr2=0x7f83b0103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:04.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.845+0000 7f83b67ab640 1 --2- 172.21.15.153:0/3933480451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83b0102e00 0x7f83b0103280 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f83a4009900 tx=0x7f83a402a410 comp rx=0 tx=0).stop 2024-09-19T08:59:04.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.845+0000 7f83b67ab640 1 -- 172.21.15.153:0/3933480451 shutdown_connections 2024-09-19T08:59:04.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.845+0000 7f83b67ab640 1 --2- 172.21.15.153:0/3933480451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83b01037c0 0x7f83b010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:04.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.845+0000 7f83b67ab640 1 --2- 172.21.15.153:0/3933480451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83b0102e00 0x7f83b0103280 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:04.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.845+0000 7f83b67ab640 1 --2- 172.21.15.153:0/3933480451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83b0101c00 0x7f83b0102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:04.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.845+0000 7f83b67ab640 1 -- 172.21.15.153:0/3933480451 >> 172.21.15.153:0/3933480451 conn(0x7f83b00fd390 msgr2=0x7f83b00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:59:04.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.845+0000 7f83b67ab640 1 -- 172.21.15.153:0/3933480451 shutdown_connections 2024-09-19T08:59:04.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.845+0000 7f83b67ab640 1 -- 172.21.15.153:0/3933480451 wait complete. 2024-09-19T08:59:04.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.846+0000 7f83b67ab640 1 Processor -- start 2024-09-19T08:59:04.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.846+0000 7f83b67ab640 1 -- start start 2024-09-19T08:59:04.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.846+0000 7f83b67ab640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83b0101c00 0x7f83b0197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:59:04.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.846+0000 7f83b67ab640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83b0102e00 0x7f83b0198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:59:04.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.846+0000 7f83b67ab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83b01037c0 0x7f83b019f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:59:04.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.846+0000 7f83b67ab640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f83b010d860 con 0x7f83b01037c0 2024-09-19T08:59:04.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.846+0000 7f83b67ab640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f83b010d6e0 con 0x7f83b0102e00 2024-09-19T08:59:04.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.846+0000 7f83b67ab640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f83b010d9e0 con 0x7f83b0101c00 2024-09-19T08:59:04.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.846+0000 7f83af7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83b0102e00 0x7f83b0198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:59:04.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.846+0000 7f83af7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83b0102e00 0x7f83b0198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:51014/0 (socket says 172.21.15.153:51014) 2024-09-19T08:59:04.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.846+0000 7f83af7fe640 1 -- 172.21.15.153:0/3228308572 learned_addr learned my addr 172.21.15.153:0/3228308572 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-19T08:59:04.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.846+0000 7f83b4d48640 1 --2- 172.21.15.153:0/3228308572 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83b01037c0 0x7f83b019f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:59:04.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.846+0000 7f83affff640 1 --2- 172.21.15.153:0/3228308572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83b0101c00 0x7f83b0197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:59:04.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.846+0000 7f83af7fe640 1 -- 172.21.15.153:0/3228308572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83b0101c00 msgr2=0x7f83b0197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:04.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.846+0000 7f83af7fe640 1 --2- 172.21.15.153:0/3228308572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83b0101c00 0x7f83b0197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:04.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.846+0000 7f83af7fe640 1 -- 172.21.15.153:0/3228308572 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83b01037c0 msgr2=0x7f83b019f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:04.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.846+0000 7f83af7fe640 1 --2- 172.21.15.153:0/3228308572 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83b01037c0 0x7f83b019f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:04.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.846+0000 7f83af7fe640 1 -- 172.21.15.153:0/3228308572 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b019fb10 con 0x7f83b0102e00 2024-09-19T08:59:04.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.846+0000 7f83affff640 1 --2- 172.21.15.153:0/3228308572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83b0101c00 0x7f83b0197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:59:04.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.847+0000 7f83af7fe640 1 --2- 172.21.15.153:0/3228308572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83b0102e00 0x7f83b0198420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f83a4009230 tx=0x7f83a4004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:59:04.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.847+0000 7f83ad7fa640 1 -- 172.21.15.153:0/3228308572 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a403f030 con 0x7f83b0102e00 2024-09-19T08:59:04.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.847+0000 7f83ad7fa640 1 -- 172.21.15.153:0/3228308572 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83a4003ce0 con 0x7f83b0102e00 2024-09-19T08:59:04.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.847+0000 7f83ad7fa640 1 -- 172.21.15.153:0/3228308572 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a4035070 con 0x7f83b0102e00 2024-09-19T08:59:04.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.847+0000 7f83b67ab640 1 -- 172.21.15.153:0/3228308572 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83b019fda0 con 0x7f83b0102e00 2024-09-19T08:59:04.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.847+0000 7f83b67ab640 1 -- 172.21.15.153:0/3228308572 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83b01a01d0 con 0x7f83b0102e00 2024-09-19T08:59:04.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.848+0000 7f83b67ab640 1 -- 172.21.15.153:0/3228308572 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83b01090a0 con 0x7f83b0102e00 2024-09-19T08:59:04.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.849+0000 7f83ad7fa640 1 -- 172.21.15.153:0/3228308572 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f83a4042030 con 0x7f83b0102e00 2024-09-19T08:59:04.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.849+0000 7f83ad7fa640 1 --2- 172.21.15.153:0/3228308572 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83980a3270 0x7f83980a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:59:04.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.850+0000 7f83ad7fa640 1 -- 172.21.15.153:0/3228308572 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5981+0+0 (secure 0 0 0) 0x7f83a40ef620 con 0x7f83b0102e00 2024-09-19T08:59:04.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.850+0000 7f83affff640 1 --2- 172.21.15.153:0/3228308572 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83980a3270 0x7f83980a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:59:04.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.850+0000 7f83affff640 1 --2- 172.21.15.153:0/3228308572 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83980a3270 0x7f83980a5730 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f83a0005d00 tx=0x7f83a0005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:59:04.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:04.854+0000 7f83ad7fa640 1 -- 172.21.15.153:0/3228308572 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f83a40bcc60 con 0x7f83b0102e00 2024-09-19T08:59:04.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:04 smithi196 ceph-mon[35150]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:59:04.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:04 smithi196 ceph-mon[35150]: from='osd.9 [v2:172.21.15.120:6826/1559320883,v1:172.21.15.120:6827/1559320883]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T08:59:04.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:04 smithi196 ceph-mon[35150]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T08:59:04.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:04 smithi196 ceph-mon[35150]: from='osd.9 [v2:172.21.15.120:6826/1559320883,v1:172.21.15.120:6827/1559320883]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-19T08:59:04.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:04 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:59:04.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:04 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:04.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:04 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:04.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:04 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:04.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:04 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:04.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:04 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:04.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:04 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:04.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:04 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:04.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:04 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:04.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:04 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:59:04.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:04 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:04.989 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:04 smithi120 ceph-mon[31150]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:59:04.989 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:04 smithi120 ceph-mon[31150]: from='osd.9 [v2:172.21.15.120:6826/1559320883,v1:172.21.15.120:6827/1559320883]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T08:59:04.989 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:04 smithi120 ceph-mon[31150]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T08:59:04.989 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:04 smithi120 ceph-mon[31150]: from='osd.9 [v2:172.21.15.120:6826/1559320883,v1:172.21.15.120:6827/1559320883]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-09-19T08:59:04.989 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:59:04.989 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:04.989 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:04.989 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:04.990 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:04.990 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:04.990 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:04.990 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:04.990 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:04.990 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:59:04.990 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:05.031 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:05.031+0000 7f83b67ab640 1 -- 172.21.15.153:0/3228308572 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f83b0062630 con 0x7f83980a3270 2024-09-19T08:59:05.324 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 08:59:04 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T08:59:04.987+0000 7fda92f0c640 -1 osd.9 0 waiting for initial osdmap 2024-09-19T08:59:05.324 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 08:59:05 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T08:59:05.000+0000 7fda8ad2c640 -1 osd.9 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:59:05.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:05 smithi153 ceph-mon[35071]: Detected new or changed devices on smithi120 2024-09-19T08:59:05.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:05 smithi153 ceph-mon[35071]: Adjusting osd_memory_target on smithi120 to 4272M 2024-09-19T08:59:05.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:05 smithi153 ceph-mon[35071]: from='osd.9 [v2:172.21.15.120:6826/1559320883,v1:172.21.15.120:6827/1559320883]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-19T08:59:05.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:05 smithi153 ceph-mon[35071]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T08:59:05.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:05 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:59:05.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:05 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:59:05.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:05 smithi153 ceph-mon[35071]: from='osd.9 [v2:172.21.15.120:6826/1559320883,v1:172.21.15.120:6827/1559320883]' entity='osd.9' 2024-09-19T08:59:05.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:05 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:59:05.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:05 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:59:05.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:05 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:05.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:05 smithi196 ceph-mon[35150]: Detected new or changed devices on smithi120 2024-09-19T08:59:05.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:05 smithi196 ceph-mon[35150]: Adjusting osd_memory_target on smithi120 to 4272M 2024-09-19T08:59:05.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:05 smithi196 ceph-mon[35150]: from='osd.9 [v2:172.21.15.120:6826/1559320883,v1:172.21.15.120:6827/1559320883]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-19T08:59:05.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:05 smithi196 ceph-mon[35150]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T08:59:05.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:05 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:59:05.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:05 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:59:05.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:05 smithi196 ceph-mon[35150]: from='osd.9 [v2:172.21.15.120:6826/1559320883,v1:172.21.15.120:6827/1559320883]' entity='osd.9' 2024-09-19T08:59:05.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:05 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:59:05.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:05 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:59:05.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:05 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:06.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:05 smithi120 ceph-mon[31150]: Detected new or changed devices on smithi120 2024-09-19T08:59:06.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:05 smithi120 ceph-mon[31150]: Adjusting osd_memory_target on smithi120 to 4272M 2024-09-19T08:59:06.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:05 smithi120 ceph-mon[31150]: from='osd.9 [v2:172.21.15.120:6826/1559320883,v1:172.21.15.120:6827/1559320883]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-19T08:59:06.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:05 smithi120 ceph-mon[31150]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T08:59:06.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:05 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:59:06.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:05 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:59:06.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:05 smithi120 ceph-mon[31150]: from='osd.9 [v2:172.21.15.120:6826/1559320883,v1:172.21.15.120:6827/1559320883]' entity='osd.9' 2024-09-19T08:59:06.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:05 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:59:06.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:05 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:59:06.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:05 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:06.702 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:06 smithi153 ceph-mon[35071]: purged_snaps scrub starts 2024-09-19T08:59:06.703 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:06 smithi153 ceph-mon[35071]: purged_snaps scrub ok 2024-09-19T08:59:06.703 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:06 smithi153 ceph-mon[35071]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:59:06.703 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:06 smithi153 ceph-mon[35071]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:59:06.703 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:06 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:59:06.703 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:06 smithi153 ceph-mon[35071]: osd.9 [v2:172.21.15.120:6826/1559320883,v1:172.21.15.120:6827/1559320883] boot 2024-09-19T08:59:06.703 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:06 smithi153 ceph-mon[35071]: osdmap e62: 10 total, 10 up, 10 in 2024-09-19T08:59:06.703 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:06 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:59:06.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:06 smithi196 ceph-mon[35150]: purged_snaps scrub starts 2024-09-19T08:59:06.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:06 smithi196 ceph-mon[35150]: purged_snaps scrub ok 2024-09-19T08:59:06.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:06 smithi196 ceph-mon[35150]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:59:06.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:06 smithi196 ceph-mon[35150]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:59:06.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:06 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:59:06.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:06 smithi196 ceph-mon[35150]: osd.9 [v2:172.21.15.120:6826/1559320883,v1:172.21.15.120:6827/1559320883] boot 2024-09-19T08:59:06.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:06 smithi196 ceph-mon[35150]: osdmap e62: 10 total, 10 up, 10 in 2024-09-19T08:59:06.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:06 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:59:07.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:06 smithi120 ceph-mon[31150]: purged_snaps scrub starts 2024-09-19T08:59:07.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:06 smithi120 ceph-mon[31150]: purged_snaps scrub ok 2024-09-19T08:59:07.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:06 smithi120 ceph-mon[31150]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:59:07.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:06 smithi120 ceph-mon[31150]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:59:07.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:06 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:59:07.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:06 smithi120 ceph-mon[31150]: osd.9 [v2:172.21.15.120:6826/1559320883,v1:172.21.15.120:6827/1559320883] boot 2024-09-19T08:59:07.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:06 smithi120 ceph-mon[31150]: osdmap e62: 10 total, 10 up, 10 in 2024-09-19T08:59:07.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:06 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:59:07.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:07 smithi196 ceph-mon[35150]: from='client.? 172.21.15.153:0/2806308859' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "797db886-62f9-4316-a45b-4d50512e65d4"}]: dispatch 2024-09-19T08:59:07.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:07 smithi196 ceph-mon[35150]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "797db886-62f9-4316-a45b-4d50512e65d4"}]: dispatch 2024-09-19T08:59:07.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:07 smithi196 ceph-mon[35150]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "797db886-62f9-4316-a45b-4d50512e65d4"}]': finished 2024-09-19T08:59:07.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:07 smithi196 ceph-mon[35150]: osdmap e63: 11 total, 10 up, 11 in 2024-09-19T08:59:07.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:07 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:59:07.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:07 smithi153 ceph-mon[35071]: from='client.? 172.21.15.153:0/2806308859' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "797db886-62f9-4316-a45b-4d50512e65d4"}]: dispatch 2024-09-19T08:59:07.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:07 smithi153 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "797db886-62f9-4316-a45b-4d50512e65d4"}]: dispatch 2024-09-19T08:59:07.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:07 smithi153 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "797db886-62f9-4316-a45b-4d50512e65d4"}]': finished 2024-09-19T08:59:07.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:07 smithi153 ceph-mon[35071]: osdmap e63: 11 total, 10 up, 11 in 2024-09-19T08:59:07.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:07 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:59:08.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:07 smithi120 ceph-mon[31150]: from='client.? 172.21.15.153:0/2806308859' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "797db886-62f9-4316-a45b-4d50512e65d4"}]: dispatch 2024-09-19T08:59:08.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:07 smithi120 ceph-mon[31150]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "797db886-62f9-4316-a45b-4d50512e65d4"}]: dispatch 2024-09-19T08:59:08.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:07 smithi120 ceph-mon[31150]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "797db886-62f9-4316-a45b-4d50512e65d4"}]': finished 2024-09-19T08:59:08.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:07 smithi120 ceph-mon[31150]: osdmap e63: 11 total, 10 up, 11 in 2024-09-19T08:59:08.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:07 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:59:08.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:08 smithi196 ceph-mon[35150]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:08.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:08 smithi196 ceph-mon[35150]: from='client.? 172.21.15.153:0/48416971' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:59:08.936 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:08 smithi153 ceph-mon[35071]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:08.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:08 smithi153 ceph-mon[35071]: from='client.? 172.21.15.153:0/48416971' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:59:09.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:08 smithi120 ceph-mon[31150]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:09.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:08 smithi120 ceph-mon[31150]: from='client.? 172.21.15.153:0/48416971' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:59:10.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:10 smithi196 ceph-mon[35150]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:10.936 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:10 smithi153 ceph-mon[35071]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:11.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:10 smithi120 ceph-mon[31150]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:12.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:12 smithi196 ceph-mon[35150]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:12.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:12 smithi153 ceph-mon[35071]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:13.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:12 smithi120 ceph-mon[31150]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:14.819 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:14 smithi153 ceph-mon[35071]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:14.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:14 smithi196 ceph-mon[35150]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:15.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:14 smithi120 ceph-mon[31150]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:16.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:16 smithi196 ceph-mon[35150]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:16.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T08:59:16.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:16.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:16 smithi153 ceph-mon[35071]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:16.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T08:59:16.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:17.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:16 smithi120 ceph-mon[31150]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:17.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T08:59:17.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:17.878 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:17 smithi153 ceph-mon[35071]: Deploying daemon osd.10 on smithi153 2024-09-19T08:59:17.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:17 smithi196 ceph-mon[35150]: Deploying daemon osd.10 on smithi153 2024-09-19T08:59:18.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:17 smithi120 ceph-mon[31150]: Deploying daemon osd.10 on smithi153 2024-09-19T08:59:18.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:18 smithi196 ceph-mon[35150]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:18.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:18 smithi153 ceph-mon[35071]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:18.975 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:18 smithi120 ceph-mon[31150]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:20.870 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:20 smithi153 ceph-mon[35071]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:20.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:20 smithi196 ceph-mon[35150]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:21.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:20 smithi120 ceph-mon[31150]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:22.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:22 smithi196 ceph-mon[35150]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:22.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:22 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:59:22.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:22 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:22.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:22 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:22.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:22 smithi153 ceph-mon[35071]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:22.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:22 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:59:22.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:22 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:22.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:22 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:23.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:22 smithi120 ceph-mon[31150]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:23.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:22 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:59:23.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:22 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:23.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:22 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:24.850 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:24 smithi153 ceph-mon[35071]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:24.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:24 smithi196 ceph-mon[35150]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:25.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:24 smithi120 ceph-mon[31150]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:25.601 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 10 on host 'smithi153' 2024-09-19T08:59:25.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:25.598+0000 7f83ad7fa640 1 -- 172.21.15.153:0/3228308572 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f83b0062630 con 0x7f83980a3270 2024-09-19T08:59:25.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:25.600+0000 7f83b67ab640 1 -- 172.21.15.153:0/3228308572 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83980a3270 msgr2=0x7f83980a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:25.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:25.600+0000 7f83b67ab640 1 --2- 172.21.15.153:0/3228308572 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83980a3270 0x7f83980a5730 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f83a0005d00 tx=0x7f83a0005ef0 comp rx=0 tx=0).stop 2024-09-19T08:59:25.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:25.600+0000 7f83b67ab640 1 -- 172.21.15.153:0/3228308572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83b0102e00 msgr2=0x7f83b0198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:25.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:25.600+0000 7f83b67ab640 1 --2- 172.21.15.153:0/3228308572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83b0102e00 0x7f83b0198420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f83a4009230 tx=0x7f83a4004360 comp rx=0 tx=0).stop 2024-09-19T08:59:25.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:25.600+0000 7f83b67ab640 1 -- 172.21.15.153:0/3228308572 shutdown_connections 2024-09-19T08:59:25.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:25.600+0000 7f83b67ab640 1 --2- 172.21.15.153:0/3228308572 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83980a3270 0x7f83980a5730 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:25.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:25.600+0000 7f83b67ab640 1 --2- 172.21.15.153:0/3228308572 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83b01037c0 0x7f83b019f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:25.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:25.600+0000 7f83b67ab640 1 --2- 172.21.15.153:0/3228308572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83b0102e00 0x7f83b0198420 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:25.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:25.600+0000 7f83b67ab640 1 --2- 172.21.15.153:0/3228308572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83b0101c00 0x7f83b0197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:25.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:25.600+0000 7f83b67ab640 1 -- 172.21.15.153:0/3228308572 >> 172.21.15.153:0/3228308572 conn(0x7f83b00fd390 msgr2=0x7f83b0106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:59:25.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:25.600+0000 7f83b67ab640 1 -- 172.21.15.153:0/3228308572 shutdown_connections 2024-09-19T08:59:25.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-19T08:59:25.600+0000 7f83b67ab640 1 -- 172.21.15.153:0/3228308572 wait complete. 2024-09-19T08:59:26.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:25 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:26.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:25 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:26.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:25 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:26.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:25 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:59:26.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:25 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:26.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:25 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:59:26.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:25 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:26.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:25 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:26.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:26.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:26.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:26.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:59:26.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:26.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:59:26.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:26.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:26.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:25 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:26.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:25 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:26.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:25 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:26.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:25 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:59:26.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:25 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:26.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:25 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:59:26.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:25 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:26.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:25 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:26.262 DEBUG:teuthology.orchestra.run.smithi153:osd.10> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.10.service 2024-09-19T08:59:26.264 INFO:tasks.cephadm:Deploying osd.11 on smithi196 with /dev/vg_nvme/lv_1... 2024-09-19T08:59:26.265 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T08:59:26.519 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:59:26.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:26 smithi196 ceph-mon[35150]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:26.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:26 smithi153 ceph-mon[35071]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:27.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:26 smithi120 ceph-mon[31150]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:27.821 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 08:59:27 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T08:59:27.653+0000 7f4beb869640 -1 osd.10 0 log_to_monitors true 2024-09-19T08:59:28.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:27 smithi196 ceph-mon[35150]: from='osd.10 [v2:172.21.15.153:6824/3216877560,v1:172.21.15.153:6825/3216877560]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T08:59:28.023 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:27 smithi196 ceph-mon[35150]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T08:59:28.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:27 smithi120 ceph-mon[31150]: from='osd.10 [v2:172.21.15.153:6824/3216877560,v1:172.21.15.153:6825/3216877560]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T08:59:28.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:27 smithi120 ceph-mon[31150]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T08:59:28.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:27 smithi153 ceph-mon[35071]: from='osd.10 [v2:172.21.15.153:6824/3216877560,v1:172.21.15.153:6825/3216877560]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T08:59:28.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:27 smithi153 ceph-mon[35071]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T08:59:29.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:28 smithi196 ceph-mon[35150]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:29.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:28 smithi196 ceph-mon[35150]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T08:59:29.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:28 smithi196 ceph-mon[35150]: osdmap e64: 11 total, 10 up, 11 in 2024-09-19T08:59:29.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:28 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:59:29.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:28 smithi196 ceph-mon[35150]: from='osd.10 [v2:172.21.15.153:6824/3216877560,v1:172.21.15.153:6825/3216877560]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-19T08:59:29.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:28 smithi196 ceph-mon[35150]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-19T08:59:29.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:28 smithi153 ceph-mon[35071]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:29.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:28 smithi153 ceph-mon[35071]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T08:59:29.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:28 smithi153 ceph-mon[35071]: osdmap e64: 11 total, 10 up, 11 in 2024-09-19T08:59:29.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:28 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:59:29.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:28 smithi153 ceph-mon[35071]: from='osd.10 [v2:172.21.15.153:6824/3216877560,v1:172.21.15.153:6825/3216877560]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-19T08:59:29.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:28 smithi153 ceph-mon[35071]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-19T08:59:29.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:28 smithi120 ceph-mon[31150]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:29.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:28 smithi120 ceph-mon[31150]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T08:59:29.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:28 smithi120 ceph-mon[31150]: osdmap e64: 11 total, 10 up, 11 in 2024-09-19T08:59:29.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:28 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:59:29.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:28 smithi120 ceph-mon[31150]: from='osd.10 [v2:172.21.15.153:6824/3216877560,v1:172.21.15.153:6825/3216877560]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-19T08:59:29.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:28 smithi120 ceph-mon[31150]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-09-19T08:59:29.353 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-19T08:59:29.388 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_1 2024-09-19T08:59:29.635 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:59:29.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 08:59:29 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T08:59:29.254+0000 7f4be8000640 -1 osd.10 0 waiting for initial osdmap 2024-09-19T08:59:29.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 08:59:29 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T08:59:29.268+0000 7f4bdf60c640 -1 osd.10 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:59:30.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:29 smithi196 ceph-mon[35150]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-19T08:59:30.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:29 smithi196 ceph-mon[35150]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T08:59:30.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:29 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:59:30.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:29 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:59:30.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:29 smithi196 ceph-mon[35150]: Detected new or changed devices on smithi153 2024-09-19T08:59:30.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:29 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:30.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:29 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:30.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:29 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:30.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:29 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:30.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:29 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:30.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:29 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:30.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:29 smithi196 ceph-mon[35150]: Adjusting osd_memory_target on smithi153 to 4272M 2024-09-19T08:59:30.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:29 smithi196 ceph-mon[35150]: from='osd.10 ' entity='osd.10' 2024-09-19T08:59:30.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:29 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:30.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:29 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:30.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:29 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:59:30.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:29 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:30.022 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:29 smithi196 ceph-mon[35150]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:30.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.104+0000 7f61864f0640 1 -- 172.21.15.196:0/1553769093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6180101c00 msgr2=0x7f6180102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:30.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.104+0000 7f61864f0640 1 --2- 172.21.15.196:0/1553769093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6180101c00 0x7f6180102000 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6170009900 tx=0x7f617002a410 comp rx=0 tx=0).stop 2024-09-19T08:59:30.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.104+0000 7f61864f0640 1 -- 172.21.15.196:0/1553769093 shutdown_connections 2024-09-19T08:59:30.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.104+0000 7f61864f0640 1 --2- 172.21.15.196:0/1553769093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61801037c0 0x7f618010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:30.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.104+0000 7f61864f0640 1 --2- 172.21.15.196:0/1553769093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6180102e00 0x7f6180103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:30.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.104+0000 7f61864f0640 1 --2- 172.21.15.196:0/1553769093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6180101c00 0x7f6180102000 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:30.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.105+0000 7f61864f0640 1 -- 172.21.15.196:0/1553769093 >> 172.21.15.196:0/1553769093 conn(0x7f61800fd390 msgr2=0x7f61800ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:59:30.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.105+0000 7f61864f0640 1 -- 172.21.15.196:0/1553769093 shutdown_connections 2024-09-19T08:59:30.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.105+0000 7f61864f0640 1 -- 172.21.15.196:0/1553769093 wait complete. 2024-09-19T08:59:30.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.105+0000 7f61864f0640 1 Processor -- start 2024-09-19T08:59:30.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.105+0000 7f61864f0640 1 -- start start 2024-09-19T08:59:30.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.105+0000 7f61864f0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6180101c00 0x7f6180197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:59:30.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.105+0000 7f61864f0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6180102e00 0x7f61801983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:59:30.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.105+0000 7f61864f0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61801037c0 0x7f618019f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:59:30.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.105+0000 7f61864f0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f618010d860 con 0x7f6180102e00 2024-09-19T08:59:30.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.105+0000 7f61864f0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f618010d6e0 con 0x7f6180101c00 2024-09-19T08:59:30.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.105+0000 7f61864f0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f618010d9e0 con 0x7f61801037c0 2024-09-19T08:59:30.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.106+0000 7f6184a8d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61801037c0 0x7f618019f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:59:30.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.106+0000 7f6184a8d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61801037c0 0x7f618019f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:42392/0 (socket says 172.21.15.196:42392) 2024-09-19T08:59:30.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.106+0000 7f6184a8d640 1 -- 172.21.15.196:0/4007491362 learned_addr learned my addr 172.21.15.196:0/4007491362 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-19T08:59:30.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.106+0000 7f617ffff640 1 --2- 172.21.15.196:0/4007491362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6180101c00 0x7f6180197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:59:30.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.106+0000 7f617f7fe640 1 --2- 172.21.15.196:0/4007491362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6180102e00 0x7f61801983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:59:30.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.106+0000 7f6184a8d640 1 -- 172.21.15.196:0/4007491362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6180101c00 msgr2=0x7f6180197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:30.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.106+0000 7f6184a8d640 1 --2- 172.21.15.196:0/4007491362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6180101c00 0x7f6180197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:30.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.106+0000 7f6184a8d640 1 -- 172.21.15.196:0/4007491362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6180102e00 msgr2=0x7f61801983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:30.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.106+0000 7f6184a8d640 1 --2- 172.21.15.196:0/4007491362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6180102e00 0x7f61801983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:30.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.106+0000 7f6184a8d640 1 -- 172.21.15.196:0/4007491362 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618019fae0 con 0x7f61801037c0 2024-09-19T08:59:30.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.106+0000 7f6184a8d640 1 --2- 172.21.15.196:0/4007491362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61801037c0 0x7f618019f470 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f617800cce0 tx=0x7f617800a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:59:30.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.106+0000 7f617d7fa640 1 -- 172.21.15.196:0/4007491362 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617800f450 con 0x7f61801037c0 2024-09-19T08:59:30.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.106+0000 7f61864f0640 1 -- 172.21.15.196:0/4007491362 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f618019fdd0 con 0x7f61801037c0 2024-09-19T08:59:30.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.106+0000 7f617d7fa640 1 -- 172.21.15.196:0/4007491362 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f617800fad0 con 0x7f61801037c0 2024-09-19T08:59:30.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.106+0000 7f617d7fa640 1 -- 172.21.15.196:0/4007491362 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178018c10 con 0x7f61801037c0 2024-09-19T08:59:30.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.106+0000 7f61864f0640 1 -- 172.21.15.196:0/4007491362 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61801a0230 con 0x7f61801037c0 2024-09-19T08:59:30.124 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.107+0000 7f61864f0640 1 -- 172.21.15.196:0/4007491362 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6180109020 con 0x7f61801037c0 2024-09-19T08:59:30.124 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.108+0000 7f617d7fa640 1 -- 172.21.15.196:0/4007491362 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f617800f5f0 con 0x7f61801037c0 2024-09-19T08:59:30.124 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.109+0000 7f617d7fa640 1 --2- 172.21.15.196:0/4007491362 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f61680a3270 0x7f61680a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:59:30.124 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.109+0000 7f617d7fa640 1 -- 172.21.15.196:0/4007491362 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6727+0+0 (secure 0 0 0) 0x7f61780cf920 con 0x7f61801037c0 2024-09-19T08:59:30.124 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.109+0000 7f617ffff640 1 --2- 172.21.15.196:0/4007491362 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f61680a3270 0x7f61680a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:59:30.124 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.110+0000 7f617ffff640 1 --2- 172.21.15.196:0/4007491362 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f61680a3270 0x7f61680a5730 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6170005ab0 tx=0x7f6170000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:59:30.124 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.112+0000 7f617d7fa640 1 -- 172.21.15.196:0/4007491362 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6178014030 con 0x7f61801037c0 2024-09-19T08:59:30.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:29 smithi153 ceph-mon[35071]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-19T08:59:30.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:29 smithi153 ceph-mon[35071]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T08:59:30.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:29 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:59:30.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:29 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:59:30.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:29 smithi153 ceph-mon[35071]: Detected new or changed devices on smithi153 2024-09-19T08:59:30.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:29 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:30.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:29 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:30.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:29 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:30.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:29 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:30.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:29 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:30.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:29 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:30.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:29 smithi153 ceph-mon[35071]: Adjusting osd_memory_target on smithi153 to 4272M 2024-09-19T08:59:30.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:29 smithi153 ceph-mon[35071]: from='osd.10 ' entity='osd.10' 2024-09-19T08:59:30.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:29 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:30.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:29 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:30.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:29 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:59:30.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:29 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:30.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:29 smithi153 ceph-mon[35071]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:30.287 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:30.287+0000 7f61864f0640 1 -- 172.21.15.196:0/4007491362 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6180062630 con 0x7f61680a3270 2024-09-19T08:59:30.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:29 smithi120 ceph-mon[31150]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-19T08:59:30.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:29 smithi120 ceph-mon[31150]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T08:59:30.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:29 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:59:30.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:29 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:59:30.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:29 smithi120 ceph-mon[31150]: Detected new or changed devices on smithi153 2024-09-19T08:59:30.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:29 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:30.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:29 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:30.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:29 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:30.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:29 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:30.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:29 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:30.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:29 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:30.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:29 smithi120 ceph-mon[31150]: Adjusting osd_memory_target on smithi153 to 4272M 2024-09-19T08:59:30.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:29 smithi120 ceph-mon[31150]: from='osd.10 ' entity='osd.10' 2024-09-19T08:59:30.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:29 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:30.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:29 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:30.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:29 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:59:30.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:29 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:30.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:29 smithi120 ceph-mon[31150]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:59:30.949 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:30 smithi196 ceph-mon[35150]: purged_snaps scrub starts 2024-09-19T08:59:30.949 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:30 smithi196 ceph-mon[35150]: purged_snaps scrub ok 2024-09-19T08:59:30.949 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:30 smithi196 ceph-mon[35150]: osd.10 [v2:172.21.15.153:6824/3216877560,v1:172.21.15.153:6825/3216877560] boot 2024-09-19T08:59:30.949 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:30 smithi196 ceph-mon[35150]: osdmap e66: 11 total, 11 up, 11 in 2024-09-19T08:59:30.949 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:30 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:59:30.949 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:30 smithi196 ceph-mon[35150]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:59:30.949 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:30 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:59:30.949 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:30 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:59:30.949 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:30 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:31.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:30 smithi120 ceph-mon[31150]: purged_snaps scrub starts 2024-09-19T08:59:31.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:30 smithi120 ceph-mon[31150]: purged_snaps scrub ok 2024-09-19T08:59:31.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:30 smithi120 ceph-mon[31150]: osd.10 [v2:172.21.15.153:6824/3216877560,v1:172.21.15.153:6825/3216877560] boot 2024-09-19T08:59:31.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:30 smithi120 ceph-mon[31150]: osdmap e66: 11 total, 11 up, 11 in 2024-09-19T08:59:31.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:30 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:59:31.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:30 smithi120 ceph-mon[31150]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:59:31.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:30 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:59:31.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:30 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:59:31.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:30 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:31.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:30 smithi153 ceph-mon[35071]: purged_snaps scrub starts 2024-09-19T08:59:31.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:30 smithi153 ceph-mon[35071]: purged_snaps scrub ok 2024-09-19T08:59:31.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:30 smithi153 ceph-mon[35071]: osd.10 [v2:172.21.15.153:6824/3216877560,v1:172.21.15.153:6825/3216877560] boot 2024-09-19T08:59:31.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:30 smithi153 ceph-mon[35071]: osdmap e66: 11 total, 11 up, 11 in 2024-09-19T08:59:31.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:30 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:59:31.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:30 smithi153 ceph-mon[35071]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:59:31.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:30 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:59:31.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:30 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:59:31.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:30 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:32.097 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:31 smithi196 ceph-mon[35150]: osdmap e67: 11 total, 11 up, 11 in 2024-09-19T08:59:32.098 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:31 smithi196 ceph-mon[35150]: pgmap v227: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:59:32.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:31 smithi153 ceph-mon[35071]: osdmap e67: 11 total, 11 up, 11 in 2024-09-19T08:59:32.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:31 smithi153 ceph-mon[35071]: pgmap v227: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:59:32.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:31 smithi120 ceph-mon[31150]: osdmap e67: 11 total, 11 up, 11 in 2024-09-19T08:59:32.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:31 smithi120 ceph-mon[31150]: pgmap v227: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:59:33.131 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:32 smithi196 ceph-mon[35150]: osdmap e68: 11 total, 11 up, 11 in 2024-09-19T08:59:33.131 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:32 smithi196 ceph-mon[35150]: from='client.? 172.21.15.196:0/704108732' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17e0afaa-84eb-4b26-9810-bfa3cc7e0f3c"}]: dispatch 2024-09-19T08:59:33.131 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:32 smithi196 ceph-mon[35150]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17e0afaa-84eb-4b26-9810-bfa3cc7e0f3c"}]: dispatch 2024-09-19T08:59:33.131 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:32 smithi196 ceph-mon[35150]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17e0afaa-84eb-4b26-9810-bfa3cc7e0f3c"}]': finished 2024-09-19T08:59:33.131 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:32 smithi196 ceph-mon[35150]: osdmap e69: 12 total, 11 up, 12 in 2024-09-19T08:59:33.131 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:32 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:59:33.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:32 smithi153 ceph-mon[35071]: osdmap e68: 11 total, 11 up, 11 in 2024-09-19T08:59:33.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:32 smithi153 ceph-mon[35071]: from='client.? 172.21.15.196:0/704108732' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17e0afaa-84eb-4b26-9810-bfa3cc7e0f3c"}]: dispatch 2024-09-19T08:59:33.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:32 smithi153 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17e0afaa-84eb-4b26-9810-bfa3cc7e0f3c"}]: dispatch 2024-09-19T08:59:33.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:32 smithi153 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17e0afaa-84eb-4b26-9810-bfa3cc7e0f3c"}]': finished 2024-09-19T08:59:33.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:32 smithi153 ceph-mon[35071]: osdmap e69: 12 total, 11 up, 12 in 2024-09-19T08:59:33.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:32 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:59:33.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:32 smithi120 ceph-mon[31150]: osdmap e68: 11 total, 11 up, 11 in 2024-09-19T08:59:33.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:32 smithi120 ceph-mon[31150]: from='client.? 172.21.15.196:0/704108732' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17e0afaa-84eb-4b26-9810-bfa3cc7e0f3c"}]: dispatch 2024-09-19T08:59:33.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:32 smithi120 ceph-mon[31150]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17e0afaa-84eb-4b26-9810-bfa3cc7e0f3c"}]: dispatch 2024-09-19T08:59:33.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:32 smithi120 ceph-mon[31150]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17e0afaa-84eb-4b26-9810-bfa3cc7e0f3c"}]': finished 2024-09-19T08:59:33.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:32 smithi120 ceph-mon[31150]: osdmap e69: 12 total, 11 up, 12 in 2024-09-19T08:59:33.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:32 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:59:34.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:33 smithi196 ceph-mon[35150]: from='client.? 172.21.15.196:0/1799059099' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:59:34.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:33 smithi196 ceph-mon[35150]: pgmap v230: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:59:34.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:33 smithi196 ceph-mon[35150]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T08:59:34.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:33 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:59:34.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:33 smithi153 ceph-mon[35071]: from='client.? 172.21.15.196:0/1799059099' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:59:34.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:33 smithi153 ceph-mon[35071]: pgmap v230: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:59:34.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:33 smithi153 ceph-mon[35071]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T08:59:34.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:33 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:59:34.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:33 smithi120 ceph-mon[31150]: from='client.? 172.21.15.196:0/1799059099' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:59:34.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:33 smithi120 ceph-mon[31150]: pgmap v230: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:59:34.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:33 smithi120 ceph-mon[31150]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T08:59:34.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:33 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:59:36.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:36 smithi196 ceph-mon[35150]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-09-19T08:59:36.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:36 smithi153 ceph-mon[35071]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-09-19T08:59:37.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:36 smithi120 ceph-mon[31150]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-09-19T08:59:38.906 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:38 smithi196 ceph-mon[35150]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T08:59:38.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:38 smithi153 ceph-mon[35071]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T08:59:39.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:38 smithi120 ceph-mon[31150]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T08:59:40.928 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:40 smithi153 ceph-mon[35071]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-19T08:59:40.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:40 smithi196 ceph-mon[35150]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-19T08:59:41.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:40 smithi120 ceph-mon[31150]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-19T08:59:41.897 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:41 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T08:59:41.897 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:41 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:41.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:41 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T08:59:41.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:41 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:42.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:41 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T08:59:42.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:41 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:42.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:42 smithi196 ceph-mon[35150]: Deploying daemon osd.11 on smithi196 2024-09-19T08:59:42.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:42 smithi196 ceph-mon[35150]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-19T08:59:42.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:42 smithi153 ceph-mon[35071]: Deploying daemon osd.11 on smithi196 2024-09-19T08:59:42.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:42 smithi153 ceph-mon[35071]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-19T08:59:43.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:42 smithi120 ceph-mon[31150]: Deploying daemon osd.11 on smithi196 2024-09-19T08:59:43.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:42 smithi120 ceph-mon[31150]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-19T08:59:44.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:44 smithi196 ceph-mon[35150]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T08:59:44.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:44 smithi153 ceph-mon[35071]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T08:59:45.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:44 smithi120 ceph-mon[31150]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T08:59:46.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:46 smithi196 ceph-mon[35150]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-19T08:59:46.936 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:46 smithi153 ceph-mon[35071]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-19T08:59:47.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:46 smithi120 ceph-mon[31150]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-19T08:59:47.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:47 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:59:47.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:47 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:47.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:47 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:47.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:47 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:59:47.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:47 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:47.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:47 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:48.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:47 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:59:48.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:47 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:48.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:47 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:48.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:48 smithi196 ceph-mon[35150]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:59:48.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:48 smithi153 ceph-mon[35071]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:59:48.975 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:48 smithi120 ceph-mon[31150]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:59:50.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:50 smithi196 ceph-mon[35150]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:59:50.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:50 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:50.866 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:50 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:50.866 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:50 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:50.866 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:50 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:59:50.866 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:50 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:50.906 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 11 on host 'smithi196' 2024-09-19T08:59:50.906 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:50.906+0000 7f617d7fa640 1 -- 172.21.15.196:0/4007491362 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f6180062630 con 0x7f61680a3270 2024-09-19T08:59:50.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:50.908+0000 7f61864f0640 1 -- 172.21.15.196:0/4007491362 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f61680a3270 msgr2=0x7f61680a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:50.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:50.908+0000 7f61864f0640 1 --2- 172.21.15.196:0/4007491362 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f61680a3270 0x7f61680a5730 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6170005ab0 tx=0x7f6170000bc0 comp rx=0 tx=0).stop 2024-09-19T08:59:50.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:50.908+0000 7f61864f0640 1 -- 172.21.15.196:0/4007491362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61801037c0 msgr2=0x7f618019f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:50.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:50.908+0000 7f61864f0640 1 --2- 172.21.15.196:0/4007491362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61801037c0 0x7f618019f470 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f617800cce0 tx=0x7f617800a620 comp rx=0 tx=0).stop 2024-09-19T08:59:50.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:50.909+0000 7f61864f0640 1 -- 172.21.15.196:0/4007491362 shutdown_connections 2024-09-19T08:59:50.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:50.909+0000 7f61864f0640 1 --2- 172.21.15.196:0/4007491362 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f61680a3270 0x7f61680a5730 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:50.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:50.909+0000 7f61864f0640 1 --2- 172.21.15.196:0/4007491362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61801037c0 0x7f618019f470 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:50.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:50.909+0000 7f61864f0640 1 --2- 172.21.15.196:0/4007491362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6180102e00 0x7f61801983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:50.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:50.909+0000 7f61864f0640 1 --2- 172.21.15.196:0/4007491362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6180101c00 0x7f6180197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:50.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:50.909+0000 7f61864f0640 1 -- 172.21.15.196:0/4007491362 >> 172.21.15.196:0/4007491362 conn(0x7f61800fd390 msgr2=0x7f61801061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:59:50.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:50.909+0000 7f61864f0640 1 -- 172.21.15.196:0/4007491362 shutdown_connections 2024-09-19T08:59:50.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T08:59:50.909+0000 7f61864f0640 1 -- 172.21.15.196:0/4007491362 wait complete. 2024-09-19T08:59:50.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:50 smithi153 ceph-mon[35071]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:59:50.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:50 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:50.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:50 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:50.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:50 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:50.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:50 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:59:50.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:50 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:51.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:50 smithi120 ceph-mon[31150]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:59:51.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:50 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:51.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:50 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:51.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:50 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:51.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:50 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:59:51.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:50 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:51.580 DEBUG:teuthology.orchestra.run.smithi196:osd.11> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.11.service 2024-09-19T08:59:51.582 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-19T08:59:51.583 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T08:59:51.845 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:59:51.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:51 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:59:51.868 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:51 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:51.868 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:51 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:51.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:51 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:59:51.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:51 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:51.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:51 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:51.965 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:59:51.965 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:51.965 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:52.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.369+0000 7ff4436f3640 1 -- 172.21.15.120:0/889825506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff43c101cd0 msgr2=0x7ff43c102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:52.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.369+0000 7ff4436f3640 1 --2- 172.21.15.120:0/889825506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff43c101cd0 0x7ff43c102150 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009900 tx=0x7ff42c02a410 comp rx=0 tx=0).stop 2024-09-19T08:59:52.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.369+0000 7ff4436f3640 1 -- 172.21.15.120:0/889825506 shutdown_connections 2024-09-19T08:59:52.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.369+0000 7ff4436f3640 1 --2- 172.21.15.120:0/889825506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff43c102690 0x7ff43c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:52.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.369+0000 7ff4436f3640 1 --2- 172.21.15.120:0/889825506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff43c101cd0 0x7ff43c102150 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:52.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.369+0000 7ff4436f3640 1 --2- 172.21.15.120:0/889825506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff43c100ad0 0x7ff43c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:52.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.369+0000 7ff4436f3640 1 -- 172.21.15.120:0/889825506 >> 172.21.15.120:0/889825506 conn(0x7ff43c0fc280 msgr2=0x7ff43c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:59:52.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.369+0000 7ff4436f3640 1 -- 172.21.15.120:0/889825506 shutdown_connections 2024-09-19T08:59:52.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.369+0000 7ff4436f3640 1 -- 172.21.15.120:0/889825506 wait complete. 2024-09-19T08:59:52.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.370+0000 7ff4436f3640 1 Processor -- start 2024-09-19T08:59:52.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.370+0000 7ff4436f3640 1 -- start start 2024-09-19T08:59:52.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.370+0000 7ff4436f3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff43c100ad0 0x7ff43c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:59:52.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.370+0000 7ff4436f3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff43c101cd0 0x7ff43c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:59:52.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.370+0000 7ff4436f3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff43c102690 0x7ff43c19f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:59:52.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.370+0000 7ff4436f3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c10c750 con 0x7ff43c100ad0 2024-09-19T08:59:52.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.370+0000 7ff4436f3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c10c5d0 con 0x7ff43c102690 2024-09-19T08:59:52.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.370+0000 7ff4436f3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c10c8d0 con 0x7ff43c101cd0 2024-09-19T08:59:52.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.370+0000 7ff44148f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff43c100ad0 0x7ff43c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:59:52.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.370+0000 7ff44148f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff43c100ad0 0x7ff43c197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51258/0 (socket says 172.21.15.120:51258) 2024-09-19T08:59:52.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.370+0000 7ff44148f640 1 -- 172.21.15.120:0/2916369925 learned_addr learned my addr 172.21.15.120:0/2916369925 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:59:52.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.370+0000 7ff440c8e640 1 --2- 172.21.15.120:0/2916369925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff43c101cd0 0x7ff43c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:59:52.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.370+0000 7ff441c90640 1 --2- 172.21.15.120:0/2916369925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff43c102690 0x7ff43c19f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:59:52.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.370+0000 7ff44148f640 1 -- 172.21.15.120:0/2916369925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff43c101cd0 msgr2=0x7ff43c1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:52.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.370+0000 7ff44148f640 1 --2- 172.21.15.120:0/2916369925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff43c101cd0 0x7ff43c1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:52.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.370+0000 7ff44148f640 1 -- 172.21.15.120:0/2916369925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff43c102690 msgr2=0x7ff43c19f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:52.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.370+0000 7ff44148f640 1 --2- 172.21.15.120:0/2916369925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff43c102690 0x7ff43c19f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:52.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.370+0000 7ff44148f640 1 -- 172.21.15.120:0/2916369925 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43c19fb90 con 0x7ff43c100ad0 2024-09-19T08:59:52.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.371+0000 7ff44148f640 1 --2- 172.21.15.120:0/2916369925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff43c100ad0 0x7ff43c197f60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff43800d770 tx=0x7ff43800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:59:52.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.371+0000 7ff4327fc640 1 -- 172.21.15.120:0/2916369925 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff438017070 con 0x7ff43c100ad0 2024-09-19T08:59:52.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.371+0000 7ff4327fc640 1 -- 172.21.15.120:0/2916369925 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff438009c30 con 0x7ff43c100ad0 2024-09-19T08:59:52.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.371+0000 7ff4436f3640 1 -- 172.21.15.120:0/2916369925 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff43c19fe80 con 0x7ff43c100ad0 2024-09-19T08:59:52.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.371+0000 7ff4327fc640 1 -- 172.21.15.120:0/2916369925 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff438003dd0 con 0x7ff43c100ad0 2024-09-19T08:59:52.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.371+0000 7ff4436f3640 1 -- 172.21.15.120:0/2916369925 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff43c1a02d0 con 0x7ff43c100ad0 2024-09-19T08:59:52.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.372+0000 7ff4436f3640 1 -- 172.21.15.120:0/2916369925 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff43c1a0500 con 0x7ff43c100ad0 2024-09-19T08:59:52.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.374+0000 7ff4327fc640 1 -- 172.21.15.120:0/2916369925 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7ff438004000 con 0x7ff43c100ad0 2024-09-19T08:59:52.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.375+0000 7ff4327fc640 1 --2- 172.21.15.120:0/2916369925 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff4280a3270 0x7ff4280a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:59:52.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.375+0000 7ff440c8e640 1 --2- 172.21.15.120:0/2916369925 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff4280a3270 0x7ff4280a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:59:52.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.375+0000 7ff4327fc640 1 -- 172.21.15.120:0/2916369925 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6809+0+0 (secure 0 0 0) 0x7ff4380cf090 con 0x7ff43c100ad0 2024-09-19T08:59:52.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.375+0000 7ff440c8e640 1 --2- 172.21.15.120:0/2916369925 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff4280a3270 0x7ff4280a5730 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff42c000c00 tx=0x7ff42c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:59:52.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.377+0000 7ff4327fc640 1 -- 172.21.15.120:0/2916369925 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff438009de0 con 0x7ff43c100ad0 2024-09-19T08:59:52.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.566+0000 7ff4436f3640 1 -- 172.21.15.120:0/2916369925 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7ff43c1a0890 con 0x7ff43c100ad0 2024-09-19T08:59:52.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.566+0000 7ff4327fc640 1 -- 172.21.15.120:0/2916369925 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7ff43809bca0 con 0x7ff43c100ad0 2024-09-19T08:59:52.566 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:59:52.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.568+0000 7ff4436f3640 1 -- 172.21.15.120:0/2916369925 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff4280a3270 msgr2=0x7ff4280a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:52.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.568+0000 7ff4436f3640 1 --2- 172.21.15.120:0/2916369925 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff4280a3270 0x7ff4280a5730 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff42c000c00 tx=0x7ff42c031400 comp rx=0 tx=0).stop 2024-09-19T08:59:52.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.568+0000 7ff4436f3640 1 -- 172.21.15.120:0/2916369925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff43c100ad0 msgr2=0x7ff43c197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:52.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.568+0000 7ff4436f3640 1 --2- 172.21.15.120:0/2916369925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff43c100ad0 0x7ff43c197f60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff43800d770 tx=0x7ff43800dc30 comp rx=0 tx=0).stop 2024-09-19T08:59:52.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.568+0000 7ff4436f3640 1 -- 172.21.15.120:0/2916369925 shutdown_connections 2024-09-19T08:59:52.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.568+0000 7ff4436f3640 1 --2- 172.21.15.120:0/2916369925 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff4280a3270 0x7ff4280a5730 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:52.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.568+0000 7ff4436f3640 1 --2- 172.21.15.120:0/2916369925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff43c102690 0x7ff43c19f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:52.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.568+0000 7ff4436f3640 1 --2- 172.21.15.120:0/2916369925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff43c101cd0 0x7ff43c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:52.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.568+0000 7ff4436f3640 1 --2- 172.21.15.120:0/2916369925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff43c100ad0 0x7ff43c197f60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:52.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.568+0000 7ff4436f3640 1 -- 172.21.15.120:0/2916369925 >> 172.21.15.120:0/2916369925 conn(0x7ff43c0fc280 msgr2=0x7ff43c06a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:59:52.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.569+0000 7ff4436f3640 1 -- 172.21.15.120:0/2916369925 shutdown_connections 2024-09-19T08:59:52.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:52.569+0000 7ff4436f3640 1 -- 172.21.15.120:0/2916369925 wait complete. 2024-09-19T08:59:52.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:52 smithi120 ceph-mon[31150]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:59:52.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:52 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2916369925' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T08:59:52.930 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:52 smithi196 ceph-mon[35150]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:59:52.930 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:52 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2916369925' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T08:59:52.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:52 smithi153 ceph-mon[35071]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:59:52.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:52 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2916369925' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T08:59:53.387 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":70,"num_osds":12,"num_up_osds":11,"osd_up_since":1726736369,"num_in_osds":12,"osd_in_since":1726736372,"num_remapped_pgs":0} 2024-09-19T08:59:53.647 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 08:59:53 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T08:59:53.333+0000 7fa96d0db640 -1 osd.11 0 log_to_monitors true 2024-09-19T08:59:53.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:53 smithi196 ceph-mon[35150]: from='osd.11 [v2:172.21.15.196:6824/1857200740,v1:172.21.15.196:6825/1857200740]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T08:59:53.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:53 smithi196 ceph-mon[35150]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T08:59:53.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:53 smithi153 ceph-mon[35071]: from='osd.11 [v2:172.21.15.196:6824/1857200740,v1:172.21.15.196:6825/1857200740]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T08:59:53.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:53 smithi153 ceph-mon[35071]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T08:59:54.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:53 smithi120 ceph-mon[31150]: from='osd.11 [v2:172.21.15.196:6824/1857200740,v1:172.21.15.196:6825/1857200740]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T08:59:54.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:53 smithi120 ceph-mon[31150]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T08:59:54.388 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T08:59:54.642 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:59:54.953 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:54 smithi120 ceph-mon[31150]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:59:54.953 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:54 smithi120 ceph-mon[31150]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T08:59:54.953 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:54 smithi120 ceph-mon[31150]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T08:59:54.953 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:54 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:59:54.953 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:54 smithi120 ceph-mon[31150]: from='osd.11 [v2:172.21.15.196:6824/1857200740,v1:172.21.15.196:6825/1857200740]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T08:59:54.953 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:54 smithi120 ceph-mon[31150]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T08:59:54.954 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:54 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:54.954 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:54 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:54.954 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:54 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:54.954 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:54 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:54.954 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:54 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:54.954 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:54 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:54.954 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:54 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:54.954 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:54 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:54.954 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:54 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:59:54.955 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:54 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:55.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.136+0000 7fb4ecfa1640 1 -- 172.21.15.120:0/3109735768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4e8100950 msgr2=0x7fb4e8100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:55.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.136+0000 7fb4ecfa1640 1 --2- 172.21.15.120:0/3109735768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4e8100950 0x7fb4e8100d50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8009ca0 tx=0x7fb4d802a8c0 comp rx=0 tx=0).stop 2024-09-19T08:59:55.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.136+0000 7fb4ecfa1640 1 -- 172.21.15.120:0/3109735768 shutdown_connections 2024-09-19T08:59:55.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.136+0000 7fb4ecfa1640 1 --2- 172.21.15.120:0/3109735768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4e8102510 0x7fb4e8108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:55.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.136+0000 7fb4ecfa1640 1 --2- 172.21.15.120:0/3109735768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4e8101b50 0x7fb4e8101fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:55.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.136+0000 7fb4ecfa1640 1 --2- 172.21.15.120:0/3109735768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4e8100950 0x7fb4e8100d50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:55.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.136+0000 7fb4ecfa1640 1 -- 172.21.15.120:0/3109735768 >> 172.21.15.120:0/3109735768 conn(0x7fb4e80fc0e0 msgr2=0x7fb4e80fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:59:55.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.136+0000 7fb4ecfa1640 1 -- 172.21.15.120:0/3109735768 shutdown_connections 2024-09-19T08:59:55.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.136+0000 7fb4ecfa1640 1 -- 172.21.15.120:0/3109735768 wait complete. 2024-09-19T08:59:55.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.136+0000 7fb4ecfa1640 1 Processor -- start 2024-09-19T08:59:55.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.136+0000 7fb4ecfa1640 1 -- start start 2024-09-19T08:59:55.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.137+0000 7fb4ecfa1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4e8100950 0x7fb4e806aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:59:55.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.137+0000 7fb4ecfa1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4e8101b50 0x7fb4e819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:59:55.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.137+0000 7fb4ecfa1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4e8102510 0x7fb4e819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:59:55.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.137+0000 7fb4ecfa1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e810c5e0 con 0x7fb4e8101b50 2024-09-19T08:59:55.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.137+0000 7fb4ecfa1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e810c460 con 0x7fb4e8100950 2024-09-19T08:59:55.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.137+0000 7fb4ecfa1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e810c760 con 0x7fb4e8102510 2024-09-19T08:59:55.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.137+0000 7fb4e6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4e8101b50 0x7fb4e819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:59:55.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.137+0000 7fb4e6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4e8101b50 0x7fb4e819ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51272/0 (socket says 172.21.15.120:51272) 2024-09-19T08:59:55.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.137+0000 7fb4e6ffd640 1 -- 172.21.15.120:0/1356827090 learned_addr learned my addr 172.21.15.120:0/1356827090 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:59:55.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.137+0000 7fb4e7fff640 1 --2- 172.21.15.120:0/1356827090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4e8102510 0x7fb4e819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:59:55.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.137+0000 7fb4e77fe640 1 --2- 172.21.15.120:0/1356827090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4e8100950 0x7fb4e806aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:59:55.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.137+0000 7fb4e6ffd640 1 -- 172.21.15.120:0/1356827090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4e8102510 msgr2=0x7fb4e819f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:55.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.137+0000 7fb4e6ffd640 1 --2- 172.21.15.120:0/1356827090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4e8102510 0x7fb4e819f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:55.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.137+0000 7fb4e6ffd640 1 -- 172.21.15.120:0/1356827090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4e8100950 msgr2=0x7fb4e806aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:55.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.137+0000 7fb4e6ffd640 1 --2- 172.21.15.120:0/1356827090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4e8100950 0x7fb4e806aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:55.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.137+0000 7fb4e6ffd640 1 -- 172.21.15.120:0/1356827090 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e819f9c0 con 0x7fb4e8101b50 2024-09-19T08:59:55.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.137+0000 7fb4e6ffd640 1 --2- 172.21.15.120:0/1356827090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4e8101b50 0x7fb4e819ca90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000c890 tx=0x7fb4d000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:59:55.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.137+0000 7fb4e4ff9640 1 -- 172.21.15.120:0/1356827090 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0004220 con 0x7fb4e8101b50 2024-09-19T08:59:55.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.137+0000 7fb4ecfa1640 1 -- 172.21.15.120:0/1356827090 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4e819fc50 con 0x7fb4e8101b50 2024-09-19T08:59:55.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.137+0000 7fb4e4ff9640 1 -- 172.21.15.120:0/1356827090 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4d00043c0 con 0x7fb4e8101b50 2024-09-19T08:59:55.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.137+0000 7fb4e4ff9640 1 -- 172.21.15.120:0/1356827090 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0004a00 con 0x7fb4e8101b50 2024-09-19T08:59:55.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.137+0000 7fb4ecfa1640 1 -- 172.21.15.120:0/1356827090 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4e81a0190 con 0x7fb4e8101b50 2024-09-19T08:59:55.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.138+0000 7fb4ecfa1640 1 -- 172.21.15.120:0/1356827090 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4e8107d70 con 0x7fb4e8101b50 2024-09-19T08:59:55.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.139+0000 7fb4e4ff9640 1 -- 172.21.15.120:0/1356827090 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fb4d00079b0 con 0x7fb4e8101b50 2024-09-19T08:59:55.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.140+0000 7fb4e4ff9640 1 --2- 172.21.15.120:0/1356827090 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb4d40a3240 0x7fb4d40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:59:55.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.140+0000 7fb4e77fe640 1 --2- 172.21.15.120:0/1356827090 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb4d40a3240 0x7fb4d40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:59:55.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.140+0000 7fb4e4ff9640 1 -- 172.21.15.120:0/1356827090 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6847+0+0 (secure 0 0 0) 0x7fb4d00cf420 con 0x7fb4e8101b50 2024-09-19T08:59:55.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.141+0000 7fb4e77fe640 1 --2- 172.21.15.120:0/1356827090 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb4d40a3240 0x7fb4d40a5700 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8006010 tx=0x7fb4d8009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:59:55.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.143+0000 7fb4e4ff9640 1 -- 172.21.15.120:0/1356827090 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb4d0014030 con 0x7fb4e8101b50 2024-09-19T08:59:55.185 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 08:59:54 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T08:59:54.691+0000 7fa96905e640 -1 osd.11 0 waiting for initial osdmap 2024-09-19T08:59:55.185 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 08:59:54 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T08:59:54.702+0000 7fa964e86640 -1 osd.11 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:59:55.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:54 smithi196 ceph-mon[35150]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:59:55.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:54 smithi196 ceph-mon[35150]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T08:59:55.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:54 smithi196 ceph-mon[35150]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T08:59:55.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:54 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:59:55.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:54 smithi196 ceph-mon[35150]: from='osd.11 [v2:172.21.15.196:6824/1857200740,v1:172.21.15.196:6825/1857200740]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T08:59:55.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:54 smithi196 ceph-mon[35150]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T08:59:55.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:54 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:55.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:54 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:55.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:54 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:55.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:54 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:55.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:54 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:55.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:54 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:55.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:54 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:55.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:54 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:55.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:54 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:59:55.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:54 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:55.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:54 smithi153 ceph-mon[35071]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:59:55.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:54 smithi153 ceph-mon[35071]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T08:59:55.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:54 smithi153 ceph-mon[35071]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T08:59:55.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:54 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:59:55.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:54 smithi153 ceph-mon[35071]: from='osd.11 [v2:172.21.15.196:6824/1857200740,v1:172.21.15.196:6825/1857200740]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T08:59:55.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:54 smithi153 ceph-mon[35071]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-19T08:59:55.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:54 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:55.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:54 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:55.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:54 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:55.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:54 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:55.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:54 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:55.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:54 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:59:55.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:54 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:55.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:54 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:59:55.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:54 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:59:55.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:54 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T08:59:55.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.311+0000 7fb4ecfa1640 1 -- 172.21.15.120:0/1356827090 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fb4e802d9c0 con 0x7fb4e8101b50 2024-09-19T08:59:55.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.311+0000 7fb4e4ff9640 1 -- 172.21.15.120:0/1356827090 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7fb4d009c700 con 0x7fb4e8101b50 2024-09-19T08:59:55.311 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:59:55.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.314+0000 7fb4ecfa1640 1 -- 172.21.15.120:0/1356827090 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb4d40a3240 msgr2=0x7fb4d40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:55.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.314+0000 7fb4ecfa1640 1 --2- 172.21.15.120:0/1356827090 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb4d40a3240 0x7fb4d40a5700 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8006010 tx=0x7fb4d8009360 comp rx=0 tx=0).stop 2024-09-19T08:59:55.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.314+0000 7fb4ecfa1640 1 -- 172.21.15.120:0/1356827090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4e8101b50 msgr2=0x7fb4e819ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:55.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.314+0000 7fb4ecfa1640 1 --2- 172.21.15.120:0/1356827090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4e8101b50 0x7fb4e819ca90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000c890 tx=0x7fb4d000cd50 comp rx=0 tx=0).stop 2024-09-19T08:59:55.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.314+0000 7fb4ecfa1640 1 -- 172.21.15.120:0/1356827090 shutdown_connections 2024-09-19T08:59:55.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.314+0000 7fb4ecfa1640 1 --2- 172.21.15.120:0/1356827090 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb4d40a3240 0x7fb4d40a5700 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:55.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.314+0000 7fb4ecfa1640 1 --2- 172.21.15.120:0/1356827090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4e8102510 0x7fb4e819f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:55.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.314+0000 7fb4ecfa1640 1 --2- 172.21.15.120:0/1356827090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4e8101b50 0x7fb4e819ca90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:55.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.314+0000 7fb4ecfa1640 1 --2- 172.21.15.120:0/1356827090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4e8100950 0x7fb4e806aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:55.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.314+0000 7fb4ecfa1640 1 -- 172.21.15.120:0/1356827090 >> 172.21.15.120:0/1356827090 conn(0x7fb4e80fc0e0 msgr2=0x7fb4e8104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:59:55.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.314+0000 7fb4ecfa1640 1 -- 172.21.15.120:0/1356827090 shutdown_connections 2024-09-19T08:59:55.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:55.314+0000 7fb4ecfa1640 1 -- 172.21.15.120:0/1356827090 wait complete. 2024-09-19T08:59:55.985 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:55 smithi120 ceph-mon[31150]: Detected new or changed devices on smithi196 2024-09-19T08:59:55.985 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:55 smithi120 ceph-mon[31150]: Adjusting osd_memory_target on smithi196 to 4272M 2024-09-19T08:59:55.985 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:55 smithi120 ceph-mon[31150]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-19T08:59:55.985 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:55 smithi120 ceph-mon[31150]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T08:59:55.985 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:55 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:59:55.985 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:55 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:59:55.985 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:55 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1356827090' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T08:59:55.985 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:55 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:59:56.011 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":72,"num_osds":12,"num_up_osds":11,"osd_up_since":1726736369,"num_in_osds":12,"osd_in_since":1726736372,"num_remapped_pgs":0} 2024-09-19T08:59:56.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:55 smithi196 ceph-mon[35150]: Detected new or changed devices on smithi196 2024-09-19T08:59:56.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:55 smithi196 ceph-mon[35150]: Adjusting osd_memory_target on smithi196 to 4272M 2024-09-19T08:59:56.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:55 smithi196 ceph-mon[35150]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-19T08:59:56.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:55 smithi196 ceph-mon[35150]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T08:59:56.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:55 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:59:56.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:55 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:59:56.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:55 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1356827090' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T08:59:56.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:55 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:59:56.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:55 smithi153 ceph-mon[35071]: Detected new or changed devices on smithi196 2024-09-19T08:59:56.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:55 smithi153 ceph-mon[35071]: Adjusting osd_memory_target on smithi196 to 4272M 2024-09-19T08:59:56.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:55 smithi153 ceph-mon[35071]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-19T08:59:56.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:55 smithi153 ceph-mon[35071]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T08:59:56.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:55 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:59:56.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:55 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:59:56.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:55 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1356827090' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T08:59:56.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:55 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:59:57.013 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T08:59:57.041 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:56 smithi120 ceph-mon[31150]: purged_snaps scrub starts 2024-09-19T08:59:57.041 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:56 smithi120 ceph-mon[31150]: purged_snaps scrub ok 2024-09-19T08:59:57.041 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:56 smithi120 ceph-mon[31150]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:59:57.041 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:56 smithi120 ceph-mon[31150]: osd.11 [v2:172.21.15.196:6824/1857200740,v1:172.21.15.196:6825/1857200740] boot 2024-09-19T08:59:57.041 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:56 smithi120 ceph-mon[31150]: osdmap e73: 12 total, 12 up, 12 in 2024-09-19T08:59:57.041 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:56 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:59:57.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:56 smithi196 ceph-mon[35150]: purged_snaps scrub starts 2024-09-19T08:59:57.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:56 smithi196 ceph-mon[35150]: purged_snaps scrub ok 2024-09-19T08:59:57.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:56 smithi196 ceph-mon[35150]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:59:57.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:56 smithi196 ceph-mon[35150]: osd.11 [v2:172.21.15.196:6824/1857200740,v1:172.21.15.196:6825/1857200740] boot 2024-09-19T08:59:57.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:56 smithi196 ceph-mon[35150]: osdmap e73: 12 total, 12 up, 12 in 2024-09-19T08:59:57.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:56 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:59:57.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:56 smithi153 ceph-mon[35071]: purged_snaps scrub starts 2024-09-19T08:59:57.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:56 smithi153 ceph-mon[35071]: purged_snaps scrub ok 2024-09-19T08:59:57.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:56 smithi153 ceph-mon[35071]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:59:57.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:56 smithi153 ceph-mon[35071]: osd.11 [v2:172.21.15.196:6824/1857200740,v1:172.21.15.196:6825/1857200740] boot 2024-09-19T08:59:57.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:56 smithi153 ceph-mon[35071]: osdmap e73: 12 total, 12 up, 12 in 2024-09-19T08:59:57.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:56 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:59:57.270 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:59:57.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.726+0000 7f33bd05b640 1 -- 172.21.15.120:0/613602048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33b810e1f0 msgr2=0x7f33b8114a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:57.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.726+0000 7f33bd05b640 1 --2- 172.21.15.120:0/613602048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33b810e1f0 0x7f33b8114a80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009ca0 tx=0x7f33b002a8c0 comp rx=0 tx=0).stop 2024-09-19T08:59:57.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.726+0000 7f33bd05b640 1 -- 172.21.15.120:0/613602048 shutdown_connections 2024-09-19T08:59:57.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.726+0000 7f33bd05b640 1 --2- 172.21.15.120:0/613602048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33b810e1f0 0x7f33b8114a80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:57.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.726+0000 7f33bd05b640 1 --2- 172.21.15.120:0/613602048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33b810d830 0x7f33b810dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:57.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.726+0000 7f33bd05b640 1 --2- 172.21.15.120:0/613602048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33b810c640 0x7f33b810ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:57.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.726+0000 7f33bd05b640 1 -- 172.21.15.120:0/613602048 >> 172.21.15.120:0/613602048 conn(0x7f33b806c860 msgr2=0x7f33b806cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:59:57.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.726+0000 7f33bd05b640 1 -- 172.21.15.120:0/613602048 shutdown_connections 2024-09-19T08:59:57.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.726+0000 7f33bd05b640 1 -- 172.21.15.120:0/613602048 wait complete. 2024-09-19T08:59:57.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.727+0000 7f33bd05b640 1 Processor -- start 2024-09-19T08:59:57.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.727+0000 7f33bd05b640 1 -- start start 2024-09-19T08:59:57.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.727+0000 7f33bd05b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33b810c640 0x7f33b81a2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:59:57.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.727+0000 7f33bd05b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33b810d830 0x7f33b81a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:59:57.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.727+0000 7f33bd05b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33b810e1f0 0x7f33b81a9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:59:57.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.727+0000 7f33bd05b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f33b8118300 con 0x7f33b810e1f0 2024-09-19T08:59:57.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.727+0000 7f33bd05b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f33b8118180 con 0x7f33b810c640 2024-09-19T08:59:57.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.727+0000 7f33bd05b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f33b8118480 con 0x7f33b810d830 2024-09-19T08:59:57.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.728+0000 7f33b759e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33b810e1f0 0x7f33b81a9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:59:57.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.728+0000 7f33b759e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33b810e1f0 0x7f33b81a9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51282/0 (socket says 172.21.15.120:51282) 2024-09-19T08:59:57.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.728+0000 7f33b759e640 1 -- 172.21.15.120:0/3436183726 learned_addr learned my addr 172.21.15.120:0/3436183726 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:59:57.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.728+0000 7f33b659c640 1 --2- 172.21.15.120:0/3436183726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33b810d830 0x7f33b81a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:59:57.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.728+0000 7f33b6d9d640 1 --2- 172.21.15.120:0/3436183726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33b810c640 0x7f33b81a2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:59:57.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.728+0000 7f33b759e640 1 -- 172.21.15.120:0/3436183726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33b810d830 msgr2=0x7f33b81a2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:57.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.728+0000 7f33b759e640 1 --2- 172.21.15.120:0/3436183726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33b810d830 0x7f33b81a2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:57.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.728+0000 7f33b759e640 1 -- 172.21.15.120:0/3436183726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33b810c640 msgr2=0x7f33b81a2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:57.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.728+0000 7f33b759e640 1 --2- 172.21.15.120:0/3436183726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33b810c640 0x7f33b81a2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:57.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.728+0000 7f33b759e640 1 -- 172.21.15.120:0/3436183726 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b81aa550 con 0x7f33b810e1f0 2024-09-19T08:59:57.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.728+0000 7f33b759e640 1 --2- 172.21.15.120:0/3436183726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33b810e1f0 0x7f33b81a9ee0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f33b0031ae0 tx=0x7f33b0031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:59:57.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.728+0000 7f339ffff640 1 -- 172.21.15.120:0/3436183726 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b0003b70 con 0x7f33b810e1f0 2024-09-19T08:59:57.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.728+0000 7f33bd05b640 1 -- 172.21.15.120:0/3436183726 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b81aa7e0 con 0x7f33b810e1f0 2024-09-19T08:59:57.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.728+0000 7f339ffff640 1 -- 172.21.15.120:0/3436183726 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33b0003d10 con 0x7f33b810e1f0 2024-09-19T08:59:57.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.728+0000 7f339ffff640 1 -- 172.21.15.120:0/3436183726 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b002fc20 con 0x7f33b810e1f0 2024-09-19T08:59:57.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.728+0000 7f33bd05b640 1 -- 172.21.15.120:0/3436183726 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b81aada0 con 0x7f33b810e1f0 2024-09-19T08:59:57.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.730+0000 7f33bd05b640 1 -- 172.21.15.120:0/3436183726 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33b8113a30 con 0x7f33b810e1f0 2024-09-19T08:59:57.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.731+0000 7f339ffff640 1 -- 172.21.15.120:0/3436183726 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f33b00046c0 con 0x7f33b810e1f0 2024-09-19T08:59:57.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.731+0000 7f339ffff640 1 --2- 172.21.15.120:0/3436183726 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33a00a3270 0x7f33a00a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:59:57.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.732+0000 7f33b6d9d640 1 --2- 172.21.15.120:0/3436183726 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33a00a3270 0x7f33a00a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:59:57.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.732+0000 7f339ffff640 1 -- 172.21.15.120:0/3436183726 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f33b00f0510 con 0x7f33b810e1f0 2024-09-19T08:59:57.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.732+0000 7f33b6d9d640 1 --2- 172.21.15.120:0/3436183726 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33a00a3270 0x7f33a00a5730 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f33a8005fd0 tx=0x7f33a800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:59:57.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.736+0000 7f339ffff640 1 -- 172.21.15.120:0/3436183726 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f33b00bd6d0 con 0x7f33b810e1f0 2024-09-19T08:59:57.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.903+0000 7f33bd05b640 1 -- 172.21.15.120:0/3436183726 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f33b802d9c0 con 0x7f33b810e1f0 2024-09-19T08:59:57.904 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:57 smithi120 ceph-mon[31150]: osdmap e74: 12 total, 12 up, 12 in 2024-09-19T08:59:57.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.904+0000 7f339ffff640 1 -- 172.21.15.120:0/3436183726 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7f33b00bcfd0 con 0x7f33b810e1f0 2024-09-19T08:59:57.905 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:59:57.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.906+0000 7f33bd05b640 1 -- 172.21.15.120:0/3436183726 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33a00a3270 msgr2=0x7f33a00a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:57.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.906+0000 7f33bd05b640 1 --2- 172.21.15.120:0/3436183726 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33a00a3270 0x7f33a00a5730 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f33a8005fd0 tx=0x7f33a800b390 comp rx=0 tx=0).stop 2024-09-19T08:59:57.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.906+0000 7f33bd05b640 1 -- 172.21.15.120:0/3436183726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33b810e1f0 msgr2=0x7f33b81a9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:57.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.906+0000 7f33bd05b640 1 --2- 172.21.15.120:0/3436183726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33b810e1f0 0x7f33b81a9ee0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f33b0031ae0 tx=0x7f33b0031ee0 comp rx=0 tx=0).stop 2024-09-19T08:59:57.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.906+0000 7f33bd05b640 1 -- 172.21.15.120:0/3436183726 shutdown_connections 2024-09-19T08:59:57.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.906+0000 7f33bd05b640 1 --2- 172.21.15.120:0/3436183726 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33a00a3270 0x7f33a00a5730 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:57.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.906+0000 7f33bd05b640 1 --2- 172.21.15.120:0/3436183726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33b810e1f0 0x7f33b81a9ee0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:57.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.906+0000 7f33bd05b640 1 --2- 172.21.15.120:0/3436183726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33b810d830 0x7f33b81a2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:57.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.906+0000 7f33bd05b640 1 --2- 172.21.15.120:0/3436183726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33b810c640 0x7f33b81a2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:57.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.906+0000 7f33bd05b640 1 -- 172.21.15.120:0/3436183726 >> 172.21.15.120:0/3436183726 conn(0x7f33b806c860 msgr2=0x7f33b8110c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:59:57.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.906+0000 7f33bd05b640 1 -- 172.21.15.120:0/3436183726 shutdown_connections 2024-09-19T08:59:57.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:57.906+0000 7f33bd05b640 1 -- 172.21.15.120:0/3436183726 wait complete. 2024-09-19T08:59:58.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:57 smithi196 ceph-mon[35150]: osdmap e74: 12 total, 12 up, 12 in 2024-09-19T08:59:58.186 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:57 smithi153 ceph-mon[35071]: osdmap e74: 12 total, 12 up, 12 in 2024-09-19T08:59:58.663 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":74,"num_osds":12,"num_up_osds":12,"osd_up_since":1726736395,"num_in_osds":12,"osd_in_since":1726736372,"num_remapped_pgs":0} 2024-09-19T08:59:58.664 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T08:59:58.945 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:59:59.065 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:58 smithi120 ceph-mon[31150]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:59:59.065 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:58 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3436183726' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T08:59:59.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:58 smithi196 ceph-mon[35150]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:59:59.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:58 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3436183726' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T08:59:59.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:58 smithi153 ceph-mon[35071]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:59:59.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:58 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3436183726' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T08:59:59.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.448+0000 7fe6b549b640 1 -- 172.21.15.120:0/3752318038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6b0106a80 msgr2=0x7fe6b0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:59.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.448+0000 7fe6b549b640 1 --2- 172.21.15.120:0/3752318038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6b0106a80 0x7fe6b0106e60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0009930 tx=0x7fe6a002a420 comp rx=0 tx=0).stop 2024-09-19T08:59:59.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.448+0000 7fe6b549b640 1 -- 172.21.15.120:0/3752318038 shutdown_connections 2024-09-19T08:59:59.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.448+0000 7fe6b549b640 1 --2- 172.21.15.120:0/3752318038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6b0101420 0x7fe6b010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:59.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.448+0000 7fe6b549b640 1 --2- 172.21.15.120:0/3752318038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6b0100a80 0x7fe6b0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:59.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.448+0000 7fe6b549b640 1 --2- 172.21.15.120:0/3752318038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6b0106a80 0x7fe6b0106e60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:59.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.448+0000 7fe6b549b640 1 -- 172.21.15.120:0/3752318038 >> 172.21.15.120:0/3752318038 conn(0x7fe6b00fc7a0 msgr2=0x7fe6b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:59:59.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.448+0000 7fe6b549b640 1 -- 172.21.15.120:0/3752318038 shutdown_connections 2024-09-19T08:59:59.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.448+0000 7fe6b549b640 1 -- 172.21.15.120:0/3752318038 wait complete. 2024-09-19T08:59:59.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.449+0000 7fe6b549b640 1 Processor -- start 2024-09-19T08:59:59.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.449+0000 7fe6b549b640 1 -- start start 2024-09-19T08:59:59.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.449+0000 7fe6b549b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6b0100a80 0x7fe6b01984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:59:59.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.449+0000 7fe6b549b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6b0101420 0x7fe6b01989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:59:59.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.449+0000 7fe6b549b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6b0106a80 0x7fe6b019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:59:59.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.449+0000 7fe6b549b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b0110ec0 con 0x7fe6b0106a80 2024-09-19T08:59:59.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.449+0000 7fe6b549b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b0110d40 con 0x7fe6b0101420 2024-09-19T08:59:59.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.449+0000 7fe6b549b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b0111040 con 0x7fe6b0100a80 2024-09-19T08:59:59.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.449+0000 7fe6af7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6b0106a80 0x7fe6b019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:59:59.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.449+0000 7fe6af7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6b0106a80 0x7fe6b019cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51294/0 (socket says 172.21.15.120:51294) 2024-09-19T08:59:59.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.449+0000 7fe6af7fe640 1 -- 172.21.15.120:0/2225614161 learned_addr learned my addr 172.21.15.120:0/2225614161 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T08:59:59.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.449+0000 7fe6ae7fc640 1 --2- 172.21.15.120:0/2225614161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6b0101420 0x7fe6b01989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:59:59.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.449+0000 7fe6aeffd640 1 --2- 172.21.15.120:0/2225614161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6b0100a80 0x7fe6b01984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:59:59.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.449+0000 7fe6af7fe640 1 -- 172.21.15.120:0/2225614161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6b0100a80 msgr2=0x7fe6b01984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:59.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.449+0000 7fe6af7fe640 1 --2- 172.21.15.120:0/2225614161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6b0100a80 0x7fe6b01984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:59.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.449+0000 7fe6af7fe640 1 -- 172.21.15.120:0/2225614161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6b0101420 msgr2=0x7fe6b01989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:59.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.449+0000 7fe6af7fe640 1 --2- 172.21.15.120:0/2225614161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6b0101420 0x7fe6b01989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:59.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.449+0000 7fe6af7fe640 1 -- 172.21.15.120:0/2225614161 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b019d450 con 0x7fe6b0106a80 2024-09-19T08:59:59.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.450+0000 7fe6af7fe640 1 --2- 172.21.15.120:0/2225614161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6b0106a80 0x7fe6b019cd70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800b970 tx=0x7fe6a800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:59:59.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.450+0000 7fe697fff640 1 -- 172.21.15.120:0/2225614161 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8019070 con 0x7fe6b0106a80 2024-09-19T08:59:59.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.450+0000 7fe6b549b640 1 -- 172.21.15.120:0/2225614161 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6b019d740 con 0x7fe6b0106a80 2024-09-19T08:59:59.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.450+0000 7fe697fff640 1 -- 172.21.15.120:0/2225614161 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6a800cb40 con 0x7fe6b0106a80 2024-09-19T08:59:59.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.450+0000 7fe697fff640 1 -- 172.21.15.120:0/2225614161 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8007650 con 0x7fe6b0106a80 2024-09-19T08:59:59.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.450+0000 7fe6b549b640 1 -- 172.21.15.120:0/2225614161 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6b01a5020 con 0x7fe6b0106a80 2024-09-19T08:59:59.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.451+0000 7fe6b549b640 1 -- 172.21.15.120:0/2225614161 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6b0100ee0 con 0x7fe6b0106a80 2024-09-19T08:59:59.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.452+0000 7fe697fff640 1 -- 172.21.15.120:0/2225614161 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fe6a801c070 con 0x7fe6b0106a80 2024-09-19T08:59:59.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.453+0000 7fe697fff640 1 --2- 172.21.15.120:0/2225614161 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe6980a3240 0x7fe6980a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:59:59.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.453+0000 7fe6aeffd640 1 --2- 172.21.15.120:0/2225614161 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe6980a3240 0x7fe6980a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:59:59.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.453+0000 7fe697fff640 1 -- 172.21.15.120:0/2225614161 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fe6a80cef20 con 0x7fe6b0106a80 2024-09-19T08:59:59.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.453+0000 7fe6aeffd640 1 --2- 172.21.15.120:0/2225614161 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe6980a3240 0x7fe6980a5700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0000c00 tx=0x7fe6a0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:59:59.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.456+0000 7fe697fff640 1 -- 172.21.15.120:0/2225614161 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe6a809ba90 con 0x7fe6b0106a80 2024-09-19T08:59:59.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.624+0000 7fe6b549b640 1 -- 172.21.15.120:0/2225614161 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6b0109a70 con 0x7fe6b0106a80 2024-09-19T08:59:59.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.625+0000 7fe697fff640 1 -- 172.21.15.120:0/2225614161 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+18551 (secure 0 0 0) 0x7fe6a809fd90 con 0x7fe6b0106a80 2024-09-19T08:59:59.625 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T08:59:59.625 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":74,"fsid":"507393d2-7664-11ef-bceb-c7b262605968","created":"2024-09-19T08:53:06.386526+0000","modified":"2024-09-19T08:59:56.702544+0000","last_up_change":"2024-09-19T08:59:55.692659+0000","last_in_change":"2024-09-19T08:59:32.808860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:56:07.830047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33ec3993-576c-428c-bf75-adcb17cd72a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6803","nonce":4131180019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6805","nonce":4131180019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6809","nonce":4131180019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6807","nonce":4131180019}]},"public_addr":"172.21.15.120:6803/4131180019","cluster_addr":"172.21.15.120:6805/4131180019","heartbeat_back_addr":"172.21.15.120:6809/4131180019","heartbeat_front_addr":"172.21.15.120:6807/4131180019","state":["exists","up"]},{"osd":1,"uuid":"450bbebf-2aac-47c6-8474-bbd37fd0a1a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6801","nonce":2259730294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6803","nonce":2259730294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6807","nonce":2259730294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6805","nonce":2259730294}]},"public_addr":"172.21.15.153:6801/2259730294","cluster_addr":"172.21.15.153:6803/2259730294","heartbeat_back_addr":"172.21.15.153:6807/2259730294","heartbeat_front_addr":"172.21.15.153:6805/2259730294","state":["exists","up"]},{"osd":2,"uuid":"ac8c4b83-8e8e-4803-b266-262f7873bd85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6801","nonce":2942286570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6803","nonce":2942286570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6807","nonce":2942286570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6805","nonce":2942286570}]},"public_addr":"172.21.15.196:6801/2942286570","cluster_addr":"172.21.15.196:6803/2942286570","heartbeat_back_addr":"172.21.15.196:6807/2942286570","heartbeat_front_addr":"172.21.15.196:6805/2942286570","state":["exists","up"]},{"osd":3,"uuid":"c1d9e507-8f02-403e-87da-9b4966fd8fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6811","nonce":713205668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6813","nonce":713205668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6817","nonce":713205668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6815","nonce":713205668}]},"public_addr":"172.21.15.120:6811/713205668","cluster_addr":"172.21.15.120:6813/713205668","heartbeat_back_addr":"172.21.15.120:6817/713205668","heartbeat_front_addr":"172.21.15.120:6815/713205668","state":["exists","up"]},{"osd":4,"uuid":"7f8bf3b9-0ec4-46f6-a406-d26d7971179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6809","nonce":2680865854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6811","nonce":2680865854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6815","nonce":2680865854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6813","nonce":2680865854}]},"public_addr":"172.21.15.153:6809/2680865854","cluster_addr":"172.21.15.153:6811/2680865854","heartbeat_back_addr":"172.21.15.153:6815/2680865854","heartbeat_front_addr":"172.21.15.153:6813/2680865854","state":["exists","up"]},{"osd":5,"uuid":"70b270b2-209d-4ff5-875a-04ee7b992a93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6809","nonce":3763331116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6811","nonce":3763331116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6815","nonce":3763331116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6813","nonce":3763331116}]},"public_addr":"172.21.15.196:6809/3763331116","cluster_addr":"172.21.15.196:6811/3763331116","heartbeat_back_addr":"172.21.15.196:6815/3763331116","heartbeat_front_addr":"172.21.15.196:6813/3763331116","state":["exists","up"]},{"osd":6,"uuid":"8087f69c-dc31-4522-a18d-4e6ab1b7cb27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6819","nonce":4171773765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6821","nonce":4171773765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6825","nonce":4171773765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6823","nonce":4171773765}]},"public_addr":"172.21.15.120:6819/4171773765","cluster_addr":"172.21.15.120:6821/4171773765","heartbeat_back_addr":"172.21.15.120:6825/4171773765","heartbeat_front_addr":"172.21.15.120:6823/4171773765","state":["exists","up"]},{"osd":7,"uuid":"c3b1ead3-150f-43fd-8924-f639203f4cdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6817","nonce":2535148453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6819","nonce":2535148453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6823","nonce":2535148453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6821","nonce":2535148453}]},"public_addr":"172.21.15.153:6817/2535148453","cluster_addr":"172.21.15.153:6819/2535148453","heartbeat_back_addr":"172.21.15.153:6823/2535148453","heartbeat_front_addr":"172.21.15.153:6821/2535148453","state":["exists","up"]},{"osd":8,"uuid":"a3273a2a-968e-4e93-a9cf-3879892bdeab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6817","nonce":186195906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6819","nonce":186195906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6823","nonce":186195906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6821","nonce":186195906}]},"public_addr":"172.21.15.196:6817/186195906","cluster_addr":"172.21.15.196:6819/186195906","heartbeat_back_addr":"172.21.15.196:6823/186195906","heartbeat_front_addr":"172.21.15.196:6821/186195906","state":["exists","up"]},{"osd":9,"uuid":"6d184513-dab8-45f7-bc08-fa2e8b2f10b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6827","nonce":1559320883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6829","nonce":1559320883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6833","nonce":1559320883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6831","nonce":1559320883}]},"public_addr":"172.21.15.120:6827/1559320883","cluster_addr":"172.21.15.120:6829/1559320883","heartbeat_back_addr":"172.21.15.120:6833/1559320883","heartbeat_front_addr":"172.21.15.120:6831/1559320883","state":["exists","up"]},{"osd":10,"uuid":"797db886-62f9-4316-a45b-4d50512e65d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6825","nonce":3216877560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6827","nonce":3216877560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6831","nonce":3216877560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6829","nonce":3216877560}]},"public_addr":"172.21.15.153:6825/3216877560","cluster_addr":"172.21.15.153:6827/3216877560","heartbeat_back_addr":"172.21.15.153:6831/3216877560","heartbeat_front_addr":"172.21.15.153:6829/3216877560","state":["exists","up"]},{"osd":11,"uuid":"17e0afaa-84eb-4b26-9810-bfa3cc7e0f3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6825","nonce":1857200740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6827","nonce":1857200740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6831","nonce":1857200740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6829","nonce":1857200740}]},"public_addr":"172.21.15.196:6825/1857200740","cluster_addr":"172.21.15.196:6827/1857200740","heartbeat_back_addr":"172.21.15.196:6831/1857200740","heartbeat_front_addr":"172.21.15.196:6829/1857200740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:55:15.595645+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:55:40.405487+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:04.938034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:29.718696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:55.636029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:57:21.257083+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:57:46.316552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:58:11.946863+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:58:37.098578+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:03.759023+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:28.659049+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:54.350475+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:0/3010783707":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6801/3562882246":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6800/3562882246":"2024-09-20T08:53:51.528772+0000","172.21.15.120:0/3224512035":"2024-09-20T08:53:51.528772+0000","172.21.15.120:0/3465105835":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/3317449610":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/2800697451":"2024-09-20T08:53:28.144376+0000","172.21.15.120:6801/2938446665":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/109127432":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6800/2938446665":"2024-09-20T08:53:28.144376+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:59:59.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.627+0000 7fe6b549b640 1 -- 172.21.15.120:0/2225614161 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe6980a3240 msgr2=0x7fe6980a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:59.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.627+0000 7fe6b549b640 1 --2- 172.21.15.120:0/2225614161 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe6980a3240 0x7fe6980a5700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0000c00 tx=0x7fe6a0005a20 comp rx=0 tx=0).stop 2024-09-19T08:59:59.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.627+0000 7fe6b549b640 1 -- 172.21.15.120:0/2225614161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6b0106a80 msgr2=0x7fe6b019cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:59:59.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.627+0000 7fe6b549b640 1 --2- 172.21.15.120:0/2225614161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6b0106a80 0x7fe6b019cd70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800b970 tx=0x7fe6a800be30 comp rx=0 tx=0).stop 2024-09-19T08:59:59.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.627+0000 7fe6b549b640 1 -- 172.21.15.120:0/2225614161 shutdown_connections 2024-09-19T08:59:59.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.627+0000 7fe6b549b640 1 --2- 172.21.15.120:0/2225614161 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe6980a3240 0x7fe6980a5700 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:59.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.627+0000 7fe6b549b640 1 --2- 172.21.15.120:0/2225614161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6b0106a80 0x7fe6b019cd70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:59.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.627+0000 7fe6b549b640 1 --2- 172.21.15.120:0/2225614161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6b0101420 0x7fe6b01989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:59.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.627+0000 7fe6b549b640 1 --2- 172.21.15.120:0/2225614161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6b0100a80 0x7fe6b01984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:59:59.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.627+0000 7fe6b549b640 1 -- 172.21.15.120:0/2225614161 >> 172.21.15.120:0/2225614161 conn(0x7fe6b00fc7a0 msgr2=0x7fe6b006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:59:59.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.627+0000 7fe6b549b640 1 -- 172.21.15.120:0/2225614161 shutdown_connections 2024-09-19T08:59:59.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T08:59:59.627+0000 7fe6b549b640 1 -- 172.21.15.120:0/2225614161 wait complete. 2024-09-19T09:00:00.018 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 08:59:59 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2225614161' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:00:00.087 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 08:59:59 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2225614161' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:00:00.087 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 08:59:59 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2225614161' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:00:00.451 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T08:56:07.830047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T09:00:00.510 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-19T09:00:00.755 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:00:00.974 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:00 smithi120 ceph-mon[31150]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:00.974 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:00 smithi120 ceph-mon[31150]: overall HEALTH_OK 2024-09-19T09:00:01.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:00 smithi196 ceph-mon[35150]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:01.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:00 smithi196 ceph-mon[35150]: overall HEALTH_OK 2024-09-19T09:00:01.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:00 smithi153 ceph-mon[35071]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:01.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:00 smithi153 ceph-mon[35071]: overall HEALTH_OK 2024-09-19T09:00:01.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.371+0000 7fbffc308640 1 -- 172.21.15.120:0/3634882189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbff41027a0 msgr2=0x7fbff41067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:01.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.371+0000 7fbffc308640 1 --2- 172.21.15.120:0/3634882189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbff41027a0 0x7fbff41067f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009900 tx=0x7fbfe402a410 comp rx=0 tx=0).stop 2024-09-19T09:00:01.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.371+0000 7fbffc308640 1 -- 172.21.15.120:0/3634882189 shutdown_connections 2024-09-19T09:00:01.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.371+0000 7fbffc308640 1 --2- 172.21.15.120:0/3634882189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbff4106f20 0x7fbff410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:01.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.371+0000 7fbffc308640 1 --2- 172.21.15.120:0/3634882189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbff41027a0 0x7fbff41067f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:01.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.371+0000 7fbffc308640 1 --2- 172.21.15.120:0/3634882189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbff4101df0 0x7fbff41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:01.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.371+0000 7fbffc308640 1 -- 172.21.15.120:0/3634882189 >> 172.21.15.120:0/3634882189 conn(0x7fbff40fd8f0 msgr2=0x7fbff40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:01.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.371+0000 7fbffc308640 1 -- 172.21.15.120:0/3634882189 shutdown_connections 2024-09-19T09:00:01.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.371+0000 7fbffc308640 1 -- 172.21.15.120:0/3634882189 wait complete. 2024-09-19T09:00:01.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.372+0000 7fbffc308640 1 Processor -- start 2024-09-19T09:00:01.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.372+0000 7fbffc308640 1 -- start start 2024-09-19T09:00:01.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.372+0000 7fbffc308640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbff4101df0 0x7fbff4198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:01.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.372+0000 7fbffc308640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbff41027a0 0x7fbff4198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:01.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.372+0000 7fbffc308640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbff4106f20 0x7fbff419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:01.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.372+0000 7fbffc308640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbff4111f80 con 0x7fbff4101df0 2024-09-19T09:00:01.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.372+0000 7fbffc308640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbff4111e00 con 0x7fbff41027a0 2024-09-19T09:00:01.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.372+0000 7fbffc308640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbff4112100 con 0x7fbff4106f20 2024-09-19T09:00:01.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.372+0000 7fbffa0a4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbff4101df0 0x7fbff4198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:01.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.372+0000 7fbffa0a4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbff4101df0 0x7fbff4198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46440/0 (socket says 172.21.15.120:46440) 2024-09-19T09:00:01.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.372+0000 7fbffa0a4640 1 -- 172.21.15.120:0/2633357907 learned_addr learned my addr 172.21.15.120:0/2633357907 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:00:01.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.372+0000 7fbffa8a5640 1 --2- 172.21.15.120:0/2633357907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbff4106f20 0x7fbff419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:01.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.373+0000 7fbff98a3640 1 --2- 172.21.15.120:0/2633357907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbff41027a0 0x7fbff4198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:01.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.373+0000 7fbffa0a4640 1 -- 172.21.15.120:0/2633357907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbff4106f20 msgr2=0x7fbff419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:01.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.373+0000 7fbffa0a4640 1 --2- 172.21.15.120:0/2633357907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbff4106f20 0x7fbff419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:01.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.373+0000 7fbffa0a4640 1 -- 172.21.15.120:0/2633357907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbff41027a0 msgr2=0x7fbff4198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:01.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.373+0000 7fbffa0a4640 1 --2- 172.21.15.120:0/2633357907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbff41027a0 0x7fbff4198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:01.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.373+0000 7fbffa0a4640 1 -- 172.21.15.120:0/2633357907 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff419d4b0 con 0x7fbff4101df0 2024-09-19T09:00:01.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.373+0000 7fbffa0a4640 1 --2- 172.21.15.120:0/2633357907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbff4101df0 0x7fbff4198500 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbff000d770 tx=0x7fbff000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:01.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.373+0000 7fbfeb7fe640 1 -- 172.21.15.120:0/2633357907 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0017070 con 0x7fbff4101df0 2024-09-19T09:00:01.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.373+0000 7fbffc308640 1 -- 172.21.15.120:0/2633357907 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbff419d7a0 con 0x7fbff4101df0 2024-09-19T09:00:01.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.373+0000 7fbfeb7fe640 1 -- 172.21.15.120:0/2633357907 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbff0005500 con 0x7fbff4101df0 2024-09-19T09:00:01.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.373+0000 7fbfeb7fe640 1 -- 172.21.15.120:0/2633357907 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0004e00 con 0x7fbff4101df0 2024-09-19T09:00:01.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.373+0000 7fbffc308640 1 -- 172.21.15.120:0/2633357907 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbff41a5080 con 0x7fbff4101df0 2024-09-19T09:00:01.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.374+0000 7fbffc308640 1 -- 172.21.15.120:0/2633357907 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbff41032d0 con 0x7fbff4101df0 2024-09-19T09:00:01.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.375+0000 7fbfeb7fe640 1 -- 172.21.15.120:0/2633357907 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fbff0004fa0 con 0x7fbff4101df0 2024-09-19T09:00:01.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.376+0000 7fbfeb7fe640 1 --2- 172.21.15.120:0/2633357907 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbfe00a3240 0x7fbfe00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:01.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.376+0000 7fbff98a3640 1 --2- 172.21.15.120:0/2633357907 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbfe00a3240 0x7fbfe00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:01.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.376+0000 7fbfeb7fe640 1 -- 172.21.15.120:0/2633357907 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fbff00cee40 con 0x7fbff4101df0 2024-09-19T09:00:01.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.376+0000 7fbff98a3640 1 --2- 172.21.15.120:0/2633357907 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbfe00a3240 0x7fbfe00a5700 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009230 tx=0x7fbfe40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:01.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.380+0000 7fbfeb7fe640 1 -- 172.21.15.120:0/2633357907 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbff009b9b0 con 0x7fbff4101df0 2024-09-19T09:00:01.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.547+0000 7fbffc308640 1 -- 172.21.15.120:0/2633357907 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbff4199860 con 0x7fbff4101df0 2024-09-19T09:00:01.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.548+0000 7fbfeb7fe640 1 -- 172.21.15.120:0/2633357907 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) ==== 93+0+10 (secure 0 0 0) 0x7fbff009fcb0 con 0x7fbff4101df0 2024-09-19T09:00:01.548 INFO:teuthology.orchestra.run.smithi120.stdout:pg_num: 1 2024-09-19T09:00:01.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.550+0000 7fbffc308640 1 -- 172.21.15.120:0/2633357907 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbfe00a3240 msgr2=0x7fbfe00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:01.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.550+0000 7fbffc308640 1 --2- 172.21.15.120:0/2633357907 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbfe00a3240 0x7fbfe00a5700 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009230 tx=0x7fbfe40058d0 comp rx=0 tx=0).stop 2024-09-19T09:00:01.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.550+0000 7fbffc308640 1 -- 172.21.15.120:0/2633357907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbff4101df0 msgr2=0x7fbff4198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:01.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.550+0000 7fbffc308640 1 --2- 172.21.15.120:0/2633357907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbff4101df0 0x7fbff4198500 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbff000d770 tx=0x7fbff000dc30 comp rx=0 tx=0).stop 2024-09-19T09:00:01.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.550+0000 7fbffc308640 1 -- 172.21.15.120:0/2633357907 shutdown_connections 2024-09-19T09:00:01.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.550+0000 7fbffc308640 1 --2- 172.21.15.120:0/2633357907 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbfe00a3240 0x7fbfe00a5700 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:01.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.550+0000 7fbffc308640 1 --2- 172.21.15.120:0/2633357907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbff4106f20 0x7fbff419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:01.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.550+0000 7fbffc308640 1 --2- 172.21.15.120:0/2633357907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbff41027a0 0x7fbff4198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:01.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.550+0000 7fbffc308640 1 --2- 172.21.15.120:0/2633357907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbff4101df0 0x7fbff4198500 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:01.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.550+0000 7fbffc308640 1 -- 172.21.15.120:0/2633357907 >> 172.21.15.120:0/2633357907 conn(0x7fbff40fd8f0 msgr2=0x7fbff40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:01.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.550+0000 7fbffc308640 1 -- 172.21.15.120:0/2633357907 shutdown_connections 2024-09-19T09:00:01.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:01.550+0000 7fbffc308640 1 -- 172.21.15.120:0/2633357907 wait complete. 2024-09-19T09:00:01.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:01 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2633357907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T09:00:02.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:01 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2633357907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T09:00:02.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:01 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2633357907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T09:00:02.314 INFO:tasks.cephadm:Adding mds.a on smithi120 2024-09-19T09:00:02.314 INFO:tasks.cephadm:Adding mds.d on smithi120 2024-09-19T09:00:02.314 INFO:tasks.cephadm:Adding mds.g on smithi120 2024-09-19T09:00:02.314 INFO:tasks.cephadm:Adding mds.j on smithi120 2024-09-19T09:00:02.315 INFO:tasks.cephadm:Adding mds.b on smithi153 2024-09-19T09:00:02.315 INFO:tasks.cephadm:Adding mds.e on smithi153 2024-09-19T09:00:02.315 INFO:tasks.cephadm:Adding mds.h on smithi153 2024-09-19T09:00:02.315 INFO:tasks.cephadm:Adding mds.k on smithi153 2024-09-19T09:00:02.315 INFO:tasks.cephadm:Adding mds.c on smithi196 2024-09-19T09:00:02.315 INFO:tasks.cephadm:Adding mds.f on smithi196 2024-09-19T09:00:02.315 INFO:tasks.cephadm:Adding mds.i on smithi196 2024-09-19T09:00:02.315 INFO:tasks.cephadm:Adding mds.l on smithi196 2024-09-19T09:00:02.315 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi120=a;smithi120=d;smithi120=g;smithi120=j;smithi153=b;smithi153=e;smithi153=h;smithi153=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l' 2024-09-19T09:00:02.561 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:00:02.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:02 smithi196 ceph-mon[35150]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:03.040 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:02 smithi120 ceph-mon[31150]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:03.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.080+0000 7f33db946640 1 -- 172.21.15.196:0/1356797305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33d4101c00 msgr2=0x7f33d4102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:03.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.080+0000 7f33db946640 1 --2- 172.21.15.196:0/1356797305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33d4101c00 0x7f33d4102000 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f33c8009900 tx=0x7f33c802a410 comp rx=0 tx=0).stop 2024-09-19T09:00:03.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.080+0000 7f33db946640 1 -- 172.21.15.196:0/1356797305 shutdown_connections 2024-09-19T09:00:03.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.080+0000 7f33db946640 1 --2- 172.21.15.196:0/1356797305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33d41037c0 0x7f33d410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:03.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.080+0000 7f33db946640 1 --2- 172.21.15.196:0/1356797305 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33d4102e00 0x7f33d4103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:03.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.080+0000 7f33db946640 1 --2- 172.21.15.196:0/1356797305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33d4101c00 0x7f33d4102000 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:03.089 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.080+0000 7f33db946640 1 -- 172.21.15.196:0/1356797305 >> 172.21.15.196:0/1356797305 conn(0x7f33d40fd390 msgr2=0x7f33d40ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:03.089 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.080+0000 7f33db946640 1 -- 172.21.15.196:0/1356797305 shutdown_connections 2024-09-19T09:00:03.089 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.080+0000 7f33db946640 1 -- 172.21.15.196:0/1356797305 wait complete. 2024-09-19T09:00:03.089 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.081+0000 7f33db946640 1 Processor -- start 2024-09-19T09:00:03.089 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.081+0000 7f33db946640 1 -- start start 2024-09-19T09:00:03.089 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.081+0000 7f33db946640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33d4101c00 0x7f33d4197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:03.089 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.081+0000 7f33db946640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33d4102e00 0x7f33d41984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:03.089 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.081+0000 7f33db946640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33d41037c0 0x7f33d419f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:03.089 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.081+0000 7f33db946640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f33d410d8e0 con 0x7f33d4101c00 2024-09-19T09:00:03.089 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.081+0000 7f33db946640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f33d410d760 con 0x7f33d41037c0 2024-09-19T09:00:03.089 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.081+0000 7f33db946640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f33d410da60 con 0x7f33d4102e00 2024-09-19T09:00:03.089 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.081+0000 7f33d8ee1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33d4102e00 0x7f33d41984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:03.089 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.081+0000 7f33d8ee1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33d4102e00 0x7f33d41984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:33962/0 (socket says 172.21.15.196:33962) 2024-09-19T09:00:03.090 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.081+0000 7f33d8ee1640 1 -- 172.21.15.196:0/1092297191 learned_addr learned my addr 172.21.15.196:0/1092297191 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-19T09:00:03.090 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.081+0000 7f33d96e2640 1 --2- 172.21.15.196:0/1092297191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33d4101c00 0x7f33d4197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:03.090 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.081+0000 7f33d9ee3640 1 --2- 172.21.15.196:0/1092297191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33d41037c0 0x7f33d419f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:03.090 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.082+0000 7f33d8ee1640 1 -- 172.21.15.196:0/1092297191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33d41037c0 msgr2=0x7f33d419f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:03.090 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.082+0000 7f33d8ee1640 1 --2- 172.21.15.196:0/1092297191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33d41037c0 0x7f33d419f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:03.090 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.082+0000 7f33d8ee1640 1 -- 172.21.15.196:0/1092297191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33d4101c00 msgr2=0x7f33d4197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:03.090 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.082+0000 7f33d8ee1640 1 --2- 172.21.15.196:0/1092297191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33d4101c00 0x7f33d4197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:03.090 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.082+0000 7f33d8ee1640 1 -- 172.21.15.196:0/1092297191 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d419fb90 con 0x7f33d4102e00 2024-09-19T09:00:03.090 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.082+0000 7f33d9ee3640 1 --2- 172.21.15.196:0/1092297191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33d41037c0 0x7f33d419f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:00:03.090 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.082+0000 7f33d8ee1640 1 --2- 172.21.15.196:0/1092297191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33d4102e00 0x7f33d41984a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f33d000d950 tx=0x7f33d000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:03.090 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.082+0000 7f33c67fc640 1 -- 172.21.15.196:0/1092297191 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d0017070 con 0x7f33d4102e00 2024-09-19T09:00:03.090 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.082+0000 7f33c67fc640 1 -- 172.21.15.196:0/1092297191 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33d0009c30 con 0x7f33d4102e00 2024-09-19T09:00:03.090 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.082+0000 7f33db946640 1 -- 172.21.15.196:0/1092297191 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33d419fe80 con 0x7f33d4102e00 2024-09-19T09:00:03.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.082+0000 7f33c67fc640 1 -- 172.21.15.196:0/1092297191 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d000f6b0 con 0x7f33d4102e00 2024-09-19T09:00:03.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.082+0000 7f33db946640 1 -- 172.21.15.196:0/1092297191 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33d41a03b0 con 0x7f33d4102e00 2024-09-19T09:00:03.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.083+0000 7f33db946640 1 -- 172.21.15.196:0/1092297191 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33d4109070 con 0x7f33d4102e00 2024-09-19T09:00:03.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.084+0000 7f33c67fc640 1 -- 172.21.15.196:0/1092297191 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f33d00109d0 con 0x7f33d4102e00 2024-09-19T09:00:03.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.085+0000 7f33c67fc640 1 --2- 172.21.15.196:0/1092297191 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33c00a3270 0x7f33c00a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:03.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.085+0000 7f33c67fc640 1 -- 172.21.15.196:0/1092297191 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f33d00cf100 con 0x7f33d4102e00 2024-09-19T09:00:03.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.085+0000 7f33d96e2640 1 --2- 172.21.15.196:0/1092297191 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33c00a3270 0x7f33c00a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:03.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.085+0000 7f33d96e2640 1 --2- 172.21.15.196:0/1092297191 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33c00a3270 0x7f33c00a5730 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f33c8006010 tx=0x7f33c8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:03.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.089+0000 7f33c67fc640 1 -- 172.21.15.196:0/1092297191 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f33d009c2c0 con 0x7f33d4102e00 2024-09-19T09:00:03.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:02 smithi153 ceph-mon[35071]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:03.264 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.263+0000 7f33db946640 1 -- 172.21.15.196:0/1092297191 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi120=a;smithi120=d;smithi120=g;smithi120=j;smithi153=b;smithi153=e;smithi153=h;smithi153=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}) -- 0x7f33d40627f0 con 0x7f33c00a3270 2024-09-19T09:00:03.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.271+0000 7f33c67fc640 1 -- 172.21.15.196:0/1092297191 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f33d40627f0 con 0x7f33c00a3270 2024-09-19T09:00:03.271 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mds.all update... 2024-09-19T09:00:03.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.273+0000 7f33db946640 1 -- 172.21.15.196:0/1092297191 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33c00a3270 msgr2=0x7f33c00a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:03.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.273+0000 7f33db946640 1 --2- 172.21.15.196:0/1092297191 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33c00a3270 0x7f33c00a5730 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f33c8006010 tx=0x7f33c8000bc0 comp rx=0 tx=0).stop 2024-09-19T09:00:03.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.273+0000 7f33db946640 1 -- 172.21.15.196:0/1092297191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33d4102e00 msgr2=0x7f33d41984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:03.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.273+0000 7f33db946640 1 --2- 172.21.15.196:0/1092297191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33d4102e00 0x7f33d41984a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f33d000d950 tx=0x7f33d000de10 comp rx=0 tx=0).stop 2024-09-19T09:00:03.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.273+0000 7f33db946640 1 -- 172.21.15.196:0/1092297191 shutdown_connections 2024-09-19T09:00:03.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.273+0000 7f33db946640 1 --2- 172.21.15.196:0/1092297191 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33c00a3270 0x7f33c00a5730 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:03.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.273+0000 7f33db946640 1 --2- 172.21.15.196:0/1092297191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33d41037c0 0x7f33d419f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:03.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.273+0000 7f33db946640 1 --2- 172.21.15.196:0/1092297191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33d4102e00 0x7f33d41984a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:03.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.273+0000 7f33db946640 1 --2- 172.21.15.196:0/1092297191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33d4101c00 0x7f33d4197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:03.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.273+0000 7f33db946640 1 -- 172.21.15.196:0/1092297191 >> 172.21.15.196:0/1092297191 conn(0x7f33d40fd390 msgr2=0x7f33d4106240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:03.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.273+0000 7f33db946640 1 -- 172.21.15.196:0/1092297191 shutdown_connections 2024-09-19T09:00:03.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-19T09:00:03.273+0000 7f33db946640 1 -- 172.21.15.196:0/1092297191 wait complete. 2024-09-19T09:00:03.989 DEBUG:teuthology.orchestra.run.smithi120:mds.a> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.a.service 2024-09-19T09:00:03.991 DEBUG:teuthology.orchestra.run.smithi120:mds.d> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.d.service 2024-09-19T09:00:04.034 DEBUG:teuthology.orchestra.run.smithi120:mds.g> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.g.service 2024-09-19T09:00:04.077 DEBUG:teuthology.orchestra.run.smithi120:mds.j> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.j.service 2024-09-19T09:00:04.120 DEBUG:teuthology.orchestra.run.smithi153:mds.b> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.b.service 2024-09-19T09:00:04.123 DEBUG:teuthology.orchestra.run.smithi153:mds.e> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.e.service 2024-09-19T09:00:04.165 DEBUG:teuthology.orchestra.run.smithi153:mds.h> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.h.service 2024-09-19T09:00:04.209 DEBUG:teuthology.orchestra.run.smithi153:mds.k> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.k.service 2024-09-19T09:00:04.252 DEBUG:teuthology.orchestra.run.smithi196:mds.c> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.c.service 2024-09-19T09:00:04.254 DEBUG:teuthology.orchestra.run.smithi196:mds.f> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.f.service 2024-09-19T09:00:04.297 DEBUG:teuthology.orchestra.run.smithi196:mds.i> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.i.service 2024-09-19T09:00:04.340 DEBUG:teuthology.orchestra.run.smithi196:mds.l> sudo journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.l.service 2024-09-19T09:00:04.383 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-19T09:00:04.384 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:00:04.419 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:04 smithi196 ceph-mon[35150]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi120=a;smithi120=d;smithi120=g;smithi120=j;smithi153=b;smithi153=e;smithi153=h;smithi153=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:00:04.420 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:04 smithi196 ceph-mon[35150]: Saving service mds.all spec with placement smithi120=a;smithi120=d;smithi120=g;smithi120=j;smithi153=b;smithi153=e;smithi153=h;smithi153=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-09-19T09:00:04.420 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:04 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:04.420 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:04 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:00:04.420 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:04 smithi196 ceph-mon[35150]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:04.420 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:04 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:04.420 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:04 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:00:04.420 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:04 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:04.420 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:04 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:04.420 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:04 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:04.420 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:04 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:04.420 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:04 smithi196 ceph-mon[35150]: Deploying daemon mds.l on smithi196 2024-09-19T09:00:04.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:04 smithi153 ceph-mon[35071]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi120=a;smithi120=d;smithi120=g;smithi120=j;smithi153=b;smithi153=e;smithi153=h;smithi153=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:00:04.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:04 smithi153 ceph-mon[35071]: Saving service mds.all spec with placement smithi120=a;smithi120=d;smithi120=g;smithi120=j;smithi153=b;smithi153=e;smithi153=h;smithi153=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-09-19T09:00:04.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:04 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:04.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:04 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:00:04.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:04 smithi153 ceph-mon[35071]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:04.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:04 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:04.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:04 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:00:04.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:04 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:04.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:04 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:04.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:04 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:04.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:04 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:04.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:04 smithi153 ceph-mon[35071]: Deploying daemon mds.l on smithi196 2024-09-19T09:00:04.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:04 smithi120 ceph-mon[31150]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi120=a;smithi120=d;smithi120=g;smithi120=j;smithi153=b;smithi153=e;smithi153=h;smithi153=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:00:04.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:04 smithi120 ceph-mon[31150]: Saving service mds.all spec with placement smithi120=a;smithi120=d;smithi120=g;smithi120=j;smithi153=b;smithi153=e;smithi153=h;smithi153=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-09-19T09:00:04.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:04.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:00:04.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:04 smithi120 ceph-mon[31150]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:04.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:04.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:00:04.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:04.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:04.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:04.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:04 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:04.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:04 smithi120 ceph-mon[31150]: Deploying daemon mds.l on smithi196 2024-09-19T09:00:04.624 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:00:05.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.103+0000 7f05d7b47640 1 -- 172.21.15.120:0/3690118482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05d0102ef0 msgr2=0x7f05d010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:05.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.103+0000 7f05d7b47640 1 --2- 172.21.15.120:0/3690118482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05d0102ef0 0x7f05d010da40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009900 tx=0x7f05c002a480 comp rx=0 tx=0).stop 2024-09-19T09:00:05.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.103+0000 7f05d7b47640 1 -- 172.21.15.120:0/3690118482 shutdown_connections 2024-09-19T09:00:05.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.103+0000 7f05d7b47640 1 --2- 172.21.15.120:0/3690118482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05d0102ef0 0x7f05d010da40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:05.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.103+0000 7f05d7b47640 1 --2- 172.21.15.120:0/3690118482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05d0102570 0x7f05d01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:05.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.103+0000 7f05d7b47640 1 --2- 172.21.15.120:0/3690118482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05d0100e20 0x7f05d0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:05.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.103+0000 7f05d7b47640 1 -- 172.21.15.120:0/3690118482 >> 172.21.15.120:0/3690118482 conn(0x7f05d00fc7d0 msgr2=0x7f05d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:05.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.104+0000 7f05d7b47640 1 -- 172.21.15.120:0/3690118482 shutdown_connections 2024-09-19T09:00:05.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.104+0000 7f05d7b47640 1 -- 172.21.15.120:0/3690118482 wait complete. 2024-09-19T09:00:05.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.104+0000 7f05d7b47640 1 Processor -- start 2024-09-19T09:00:05.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.104+0000 7f05d7b47640 1 -- start start 2024-09-19T09:00:05.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.104+0000 7f05d7b47640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05d0100e20 0x7f05d0199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:05.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.104+0000 7f05d7b47640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05d0102570 0x7f05d0199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:05.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.104+0000 7f05d7b47640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05d0102ef0 0x7f05d01a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:05.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.104+0000 7f05d7b47640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f05d0110fc0 con 0x7f05d0100e20 2024-09-19T09:00:05.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.104+0000 7f05d7b47640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f05d0110e40 con 0x7f05d0102ef0 2024-09-19T09:00:05.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.104+0000 7f05d7b47640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f05d0111140 con 0x7f05d0102570 2024-09-19T09:00:05.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.104+0000 7f05d58e3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05d0100e20 0x7f05d0199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:05.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.104+0000 7f05d58e3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05d0100e20 0x7f05d0199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46464/0 (socket says 172.21.15.120:46464) 2024-09-19T09:00:05.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.104+0000 7f05d58e3640 1 -- 172.21.15.120:0/2090368530 learned_addr learned my addr 172.21.15.120:0/2090368530 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:00:05.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.105+0000 7f05d50e2640 1 --2- 172.21.15.120:0/2090368530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05d0102570 0x7f05d0199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:05.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.105+0000 7f05d58e3640 1 -- 172.21.15.120:0/2090368530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05d0102570 msgr2=0x7f05d0199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:05.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.105+0000 7f05d58e3640 1 --2- 172.21.15.120:0/2090368530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05d0102570 0x7f05d0199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:05.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.105+0000 7f05d58e3640 1 -- 172.21.15.120:0/2090368530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05d0102ef0 msgr2=0x7f05d01a43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:05.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.105+0000 7f05d58e3640 1 --2- 172.21.15.120:0/2090368530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05d0102ef0 0x7f05d01a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:05.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.105+0000 7f05d58e3640 1 -- 172.21.15.120:0/2090368530 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d01a4910 con 0x7f05d0100e20 2024-09-19T09:00:05.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.105+0000 7f05d50e2640 1 --2- 172.21.15.120:0/2090368530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05d0102570 0x7f05d0199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:00:05.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.105+0000 7f05d58e3640 1 --2- 172.21.15.120:0/2090368530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05d0100e20 0x7f05d0199350 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f05c800b4a0 tx=0x7f05c800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:05.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.105+0000 7f05ceffd640 1 -- 172.21.15.120:0/2090368530 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c8014070 con 0x7f05d0100e20 2024-09-19T09:00:05.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.105+0000 7f05ceffd640 1 -- 172.21.15.120:0/2090368530 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05c8010af0 con 0x7f05d0100e20 2024-09-19T09:00:05.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.105+0000 7f05ceffd640 1 -- 172.21.15.120:0/2090368530 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c8017d10 con 0x7f05d0100e20 2024-09-19T09:00:05.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.105+0000 7f05d7b47640 1 -- 172.21.15.120:0/2090368530 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05d01a4ae0 con 0x7f05d0100e20 2024-09-19T09:00:05.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.105+0000 7f05d7b47640 1 -- 172.21.15.120:0/2090368530 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05d01a4fa0 con 0x7f05d0100e20 2024-09-19T09:00:05.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.106+0000 7f05d7b47640 1 -- 172.21.15.120:0/2090368530 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05d0101200 con 0x7f05d0100e20 2024-09-19T09:00:05.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.107+0000 7f05ceffd640 1 -- 172.21.15.120:0/2090368530 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f05c8017070 con 0x7f05d0100e20 2024-09-19T09:00:05.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.108+0000 7f05ceffd640 1 --2- 172.21.15.120:0/2090368530 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f05b80a3240 0x7f05b80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:05.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.108+0000 7f05d50e2640 1 --2- 172.21.15.120:0/2090368530 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f05b80a3240 0x7f05b80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:05.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.108+0000 7f05ceffd640 1 -- 172.21.15.120:0/2090368530 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f05c80ce8f0 con 0x7f05d0100e20 2024-09-19T09:00:05.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.108+0000 7f05d50e2640 1 --2- 172.21.15.120:0/2090368530 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f05b80a3240 0x7f05b80a5700 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f05d006aac0 tx=0x7f05c40082f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:05.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.112+0000 7f05ceffd640 1 -- 172.21.15.120:0/2090368530 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f05c809bab0 con 0x7f05d0100e20 2024-09-19T09:00:05.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.312+0000 7f05d7b47640 1 -- 172.21.15.120:0/2090368530 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05d01a5260 con 0x7f05d0100e20 2024-09-19T09:00:05.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.312+0000 7f05ceffd640 1 -- 172.21.15.120:0/2090368530 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f05c809b3b0 con 0x7f05d0100e20 2024-09-19T09:00:05.312 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:00:05.312 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":1,"btime":"2024-09-19T08:53:06:386334+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-19T09:00:05.313 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 1 2024-09-19T09:00:05.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.314+0000 7f05d7b47640 1 -- 172.21.15.120:0/2090368530 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f05b80a3240 msgr2=0x7f05b80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:05.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.314+0000 7f05d7b47640 1 --2- 172.21.15.120:0/2090368530 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f05b80a3240 0x7f05b80a5700 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f05d006aac0 tx=0x7f05c40082f0 comp rx=0 tx=0).stop 2024-09-19T09:00:05.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.314+0000 7f05d7b47640 1 -- 172.21.15.120:0/2090368530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05d0100e20 msgr2=0x7f05d0199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:05.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.314+0000 7f05d7b47640 1 --2- 172.21.15.120:0/2090368530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05d0100e20 0x7f05d0199350 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f05c800b4a0 tx=0x7f05c800b960 comp rx=0 tx=0).stop 2024-09-19T09:00:05.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.314+0000 7f05d7b47640 1 -- 172.21.15.120:0/2090368530 shutdown_connections 2024-09-19T09:00:05.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.314+0000 7f05d7b47640 1 --2- 172.21.15.120:0/2090368530 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f05b80a3240 0x7f05b80a5700 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:05.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.314+0000 7f05d7b47640 1 --2- 172.21.15.120:0/2090368530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05d0102ef0 0x7f05d01a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:05.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.314+0000 7f05d7b47640 1 --2- 172.21.15.120:0/2090368530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05d0102570 0x7f05d0199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:05.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.314+0000 7f05d7b47640 1 --2- 172.21.15.120:0/2090368530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05d0100e20 0x7f05d0199350 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:05.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.314+0000 7f05d7b47640 1 -- 172.21.15.120:0/2090368530 >> 172.21.15.120:0/2090368530 conn(0x7f05d00fc7d0 msgr2=0x7f05d0106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:05.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.314+0000 7f05d7b47640 1 -- 172.21.15.120:0/2090368530 shutdown_connections 2024-09-19T09:00:05.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:05.314+0000 7f05d7b47640 1 -- 172.21.15.120:0/2090368530 wait complete. 2024-09-19T09:00:05.499 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:05 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2090368530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:00:05.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:05 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2090368530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:00:05.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:05 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2090368530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:00:05.892 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 09:00:05 smithi196 systemd[1]: Starting Ceph mds.l for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:00:06.165 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 09:00:05 smithi196 podman[64403]: 2024-09-19 09:00:05.795298205 +0000 UTC m=+0.015683677 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:00:06.166 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 09:00:05 smithi196 podman[64403]: 2024-09-19 09:00:05.903623282 +0000 UTC m=+0.124008748 container create 5dd429730b8796a48b19e9923b8b568c28b95032e43640760c3d9617f8ae21d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-l, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4) 2024-09-19T09:00:06.166 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 09:00:06 smithi196 podman[64403]: 2024-09-19 09:00:06.139646728 +0000 UTC m=+0.360032192 container init 5dd429730b8796a48b19e9923b8b568c28b95032e43640760c3d9617f8ae21d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-l, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-19T09:00:06.166 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 09:00:06 smithi196 podman[64403]: 2024-09-19 09:00:06.142877627 +0000 UTC m=+0.363263093 container start 5dd429730b8796a48b19e9923b8b568c28b95032e43640760c3d9617f8ae21d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-l, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-19T09:00:06.436 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 09:00:06 smithi196 ceph-mds[64422]: -- 172.21.15.196:0/2229639311 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560d6c153860 con 0x560d6c131400 2024-09-19T09:00:06.436 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 09:00:06 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-l[64418]: starting mds.l at 2024-09-19T09:00:06.436 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 09:00:06 smithi196 bash[64403]: 5dd429730b8796a48b19e9923b8b568c28b95032e43640760c3d9617f8ae21d7 2024-09-19T09:00:06.436 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 09:00:06 smithi196 systemd[1]: Started Ceph mds.l for 507393d2-7664-11ef-bceb-c7b262605968. 2024-09-19T09:00:06.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:06 smithi196 ceph-mon[35150]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:06.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:06 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.196:6832/1407049222,v1:172.21.15.196:6833/1407049222] up:boot 2024-09-19T09:00:06.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:06 smithi196 ceph-mon[35150]: fsmap 1 up:standby 2024-09-19T09:00:06.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:06 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T09:00:06.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:06 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:06.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:06 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:06.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:06 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:06.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:06 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:06.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:06 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:06.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:06 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:06.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:06 smithi120 ceph-mon[31150]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:06.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:06 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.196:6832/1407049222,v1:172.21.15.196:6833/1407049222] up:boot 2024-09-19T09:00:06.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:06 smithi120 ceph-mon[31150]: fsmap 1 up:standby 2024-09-19T09:00:06.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:06 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T09:00:06.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:06 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:06.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:06 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:06.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:06 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:06.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:06 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:06.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:06 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:06.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:06 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:06.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:06 smithi153 ceph-mon[35071]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:06.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:06 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.196:6832/1407049222,v1:172.21.15.196:6833/1407049222] up:boot 2024-09-19T09:00:06.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:06 smithi153 ceph-mon[35071]: fsmap 1 up:standby 2024-09-19T09:00:06.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:06 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T09:00:06.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:06 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:06.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:06 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:06.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:06 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:06.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:06 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:06.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:06 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:06.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:06 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:07.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:07 smithi196 ceph-mon[35150]: Deploying daemon mds.k on smithi153 2024-09-19T09:00:07.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:07 smithi153 ceph-mon[35071]: Deploying daemon mds.k on smithi153 2024-09-19T09:00:07.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:07 smithi120 ceph-mon[31150]: Deploying daemon mds.k on smithi153 2024-09-19T09:00:08.179 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:00:08.377 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 19 09:00:08 smithi153 systemd[1]: Starting Ceph mds.k for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:00:08.436 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:00:08.556 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:08 smithi120 ceph-mon[31150]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:08.635 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:08 smithi153 ceph-mon[35071]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:08.636 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 19 09:00:08 smithi153 podman[63582]: 2024-09-19 09:00:08.353607079 +0000 UTC m=+0.015885654 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:00:08.636 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 19 09:00:08 smithi153 podman[63582]: 2024-09-19 09:00:08.486035813 +0000 UTC m=+0.148314436 container create 5c585088ff66e9be5114b6efad06840594f5e0823c090859f836ee7dcd79675f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-k, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:00:08.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:08 smithi196 ceph-mon[35150]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:08.937 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 19 09:00:08 smithi153 podman[63582]: 2024-09-19 09:00:08.721885456 +0000 UTC m=+0.384164031 container init 5c585088ff66e9be5114b6efad06840594f5e0823c090859f836ee7dcd79675f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-k, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=) 2024-09-19T09:00:08.937 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 19 09:00:08 smithi153 podman[63582]: 2024-09-19 09:00:08.724982631 +0000 UTC m=+0.387261214 container start 5c585088ff66e9be5114b6efad06840594f5e0823c090859f836ee7dcd79675f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-k, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True) 2024-09-19T09:00:08.937 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 19 09:00:08 smithi153 ceph-mds[63601]: -- 172.21.15.153:0/1633352950 <== mon.2 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55908fad9860 con 0x55908fab7400 2024-09-19T09:00:08.937 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 19 09:00:08 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-k[63597]: starting mds.k at 2024-09-19T09:00:08.937 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 19 09:00:08 smithi153 bash[63582]: 5c585088ff66e9be5114b6efad06840594f5e0823c090859f836ee7dcd79675f 2024-09-19T09:00:08.938 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 19 09:00:08 smithi153 systemd[1]: Started Ceph mds.k for 507393d2-7664-11ef-bceb-c7b262605968. 2024-09-19T09:00:08.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.924+0000 7f331a5ba640 1 -- 172.21.15.120:0/3684226866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3314102dd0 msgr2=0x7f33141031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:08.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.924+0000 7f331a5ba640 1 --2- 172.21.15.120:0/3684226866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3314102dd0 0x7f33141031b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3304009900 tx=0x7f330402a480 comp rx=0 tx=0).stop 2024-09-19T09:00:08.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.925+0000 7f331a5ba640 1 -- 172.21.15.120:0/3684226866 shutdown_connections 2024-09-19T09:00:08.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.925+0000 7f331a5ba640 1 --2- 172.21.15.120:0/3684226866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f331410c170 0x7f331410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:08.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.925+0000 7f331a5ba640 1 --2- 172.21.15.120:0/3684226866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33141036f0 0x7f331410bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:08.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.925+0000 7f331a5ba640 1 --2- 172.21.15.120:0/3684226866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3314102dd0 0x7f33141031b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:08.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.925+0000 7f331a5ba640 1 -- 172.21.15.120:0/3684226866 >> 172.21.15.120:0/3684226866 conn(0x7f33140fc790 msgr2=0x7f33140febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:08.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.925+0000 7f331a5ba640 1 -- 172.21.15.120:0/3684226866 shutdown_connections 2024-09-19T09:00:08.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.925+0000 7f331a5ba640 1 -- 172.21.15.120:0/3684226866 wait complete. 2024-09-19T09:00:08.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.925+0000 7f331a5ba640 1 Processor -- start 2024-09-19T09:00:08.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.925+0000 7f331a5ba640 1 -- start start 2024-09-19T09:00:08.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.926+0000 7f331a5ba640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3314102dd0 0x7f33141984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:08.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.926+0000 7f331a5ba640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33141036f0 0x7f33141989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:08.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.926+0000 7f331a5ba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f331410c170 0x7f331419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:08.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.926+0000 7f331a5ba640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3314111fa0 con 0x7f3314102dd0 2024-09-19T09:00:08.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.926+0000 7f331a5ba640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3314111e20 con 0x7f33141036f0 2024-09-19T09:00:08.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.926+0000 7f331a5ba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3314112120 con 0x7f331410c170 2024-09-19T09:00:08.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.926+0000 7f3313fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3314102dd0 0x7f33141984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:08.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.926+0000 7f3313fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3314102dd0 0x7f33141984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46480/0 (socket says 172.21.15.120:46480) 2024-09-19T09:00:08.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.926+0000 7f3313fff640 1 -- 172.21.15.120:0/168974613 learned_addr learned my addr 172.21.15.120:0/168974613 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:00:08.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.926+0000 7f33137fe640 1 --2- 172.21.15.120:0/168974613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33141036f0 0x7f33141989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:08.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.926+0000 7f3318b57640 1 --2- 172.21.15.120:0/168974613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f331410c170 0x7f331419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:08.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.926+0000 7f3313fff640 1 -- 172.21.15.120:0/168974613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f331410c170 msgr2=0x7f331419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:08.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.926+0000 7f3313fff640 1 --2- 172.21.15.120:0/168974613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f331410c170 0x7f331419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:08.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.926+0000 7f3313fff640 1 -- 172.21.15.120:0/168974613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33141036f0 msgr2=0x7f33141989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:08.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.926+0000 7f3313fff640 1 --2- 172.21.15.120:0/168974613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33141036f0 0x7f33141989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:08.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.926+0000 7f3313fff640 1 -- 172.21.15.120:0/168974613 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f331419d460 con 0x7f3314102dd0 2024-09-19T09:00:08.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.926+0000 7f33137fe640 1 --2- 172.21.15.120:0/168974613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33141036f0 0x7f33141989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:00:08.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.926+0000 7f3313fff640 1 --2- 172.21.15.120:0/168974613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3314102dd0 0x7f33141984b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3304005ab0 tx=0x7f33040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:08.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.926+0000 7f33117fa640 1 -- 172.21.15.120:0/168974613 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330403f030 con 0x7f3314102dd0 2024-09-19T09:00:08.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.926+0000 7f33117fa640 1 -- 172.21.15.120:0/168974613 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3304003ce0 con 0x7f3314102dd0 2024-09-19T09:00:08.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.926+0000 7f331a5ba640 1 -- 172.21.15.120:0/168974613 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f331419d6f0 con 0x7f3314102dd0 2024-09-19T09:00:08.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.927+0000 7f33117fa640 1 -- 172.21.15.120:0/168974613 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304035070 con 0x7f3314102dd0 2024-09-19T09:00:08.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.927+0000 7f331a5ba640 1 -- 172.21.15.120:0/168974613 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3314100960 con 0x7f3314102dd0 2024-09-19T09:00:08.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.928+0000 7f331a5ba640 1 -- 172.21.15.120:0/168974613 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f331406afd0 con 0x7f3314102dd0 2024-09-19T09:00:08.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.929+0000 7f33117fa640 1 -- 172.21.15.120:0/168974613 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f3304042030 con 0x7f3314102dd0 2024-09-19T09:00:08.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.929+0000 7f33117fa640 1 --2- 172.21.15.120:0/168974613 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33000a3240 0x7f33000a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:08.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.930+0000 7f33117fa640 1 -- 172.21.15.120:0/168974613 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f33040efd60 con 0x7f3314102dd0 2024-09-19T09:00:08.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.930+0000 7f33137fe640 1 --2- 172.21.15.120:0/168974613 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33000a3240 0x7f33000a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:08.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.930+0000 7f33137fe640 1 --2- 172.21.15.120:0/168974613 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33000a3240 0x7f33000a5700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f33140fee00 tx=0x7f32fc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:08.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:08.934+0000 7f33117fa640 1 -- 172.21.15.120:0/168974613 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f33040bcf20 con 0x7f3314102dd0 2024-09-19T09:00:09.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:09.143+0000 7f331a5ba640 1 -- 172.21.15.120:0/168974613 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3314199840 con 0x7f3314102dd0 2024-09-19T09:00:09.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:09.144+0000 7f33117fa640 1 -- 172.21.15.120:0/168974613 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f3314199840 con 0x7f3314102dd0 2024-09-19T09:00:09.144 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:00:09.144 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":2,"btime":"2024-09-19T09:00:06:171004+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T09:00:09.144 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 2 2024-09-19T09:00:09.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:09.147+0000 7f331a5ba640 1 -- 172.21.15.120:0/168974613 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33000a3240 msgr2=0x7f33000a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:09.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:09.147+0000 7f331a5ba640 1 --2- 172.21.15.120:0/168974613 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33000a3240 0x7f33000a5700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f33140fee00 tx=0x7f32fc00a2d0 comp rx=0 tx=0).stop 2024-09-19T09:00:09.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:09.147+0000 7f331a5ba640 1 -- 172.21.15.120:0/168974613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3314102dd0 msgr2=0x7f33141984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:09.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:09.147+0000 7f331a5ba640 1 --2- 172.21.15.120:0/168974613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3314102dd0 0x7f33141984b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3304005ab0 tx=0x7f33040042f0 comp rx=0 tx=0).stop 2024-09-19T09:00:09.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:09.147+0000 7f331a5ba640 1 -- 172.21.15.120:0/168974613 shutdown_connections 2024-09-19T09:00:09.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:09.147+0000 7f331a5ba640 1 --2- 172.21.15.120:0/168974613 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33000a3240 0x7f33000a5700 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:09.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:09.147+0000 7f331a5ba640 1 --2- 172.21.15.120:0/168974613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f331410c170 0x7f331419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:09.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:09.147+0000 7f331a5ba640 1 --2- 172.21.15.120:0/168974613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33141036f0 0x7f33141989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:09.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:09.147+0000 7f331a5ba640 1 --2- 172.21.15.120:0/168974613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3314102dd0 0x7f33141984b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:09.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:09.147+0000 7f331a5ba640 1 -- 172.21.15.120:0/168974613 >> 172.21.15.120:0/168974613 conn(0x7f33140fc790 msgr2=0x7f33140fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:09.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:09.147+0000 7f331a5ba640 1 -- 172.21.15.120:0/168974613 shutdown_connections 2024-09-19T09:00:09.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:09.147+0000 7f331a5ba640 1 -- 172.21.15.120:0/168974613 wait complete. 2024-09-19T09:00:10.177 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:09 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:10.177 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:09 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:10.177 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:09 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:10.177 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:09 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:10.177 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:09 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:10.177 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:09 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:10.177 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:09 smithi120 ceph-mon[31150]: Deploying daemon mds.j on smithi120 2024-09-19T09:00:10.177 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:09 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/168974613' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:00:10.177 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:09 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.153:6832/1522304272,v1:172.21.15.153:6833/1522304272] up:boot 2024-09-19T09:00:10.178 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:09 smithi120 ceph-mon[31150]: fsmap 2 up:standby 2024-09-19T09:00:10.178 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:09 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T09:00:10.178 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:09 smithi120 ceph-mon[31150]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:10.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:09 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:10.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:09 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:10.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:09 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:10.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:09 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:10.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:09 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:10.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:09 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:10.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:09 smithi196 ceph-mon[35150]: Deploying daemon mds.j on smithi120 2024-09-19T09:00:10.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:09 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/168974613' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:00:10.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:09 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.153:6832/1522304272,v1:172.21.15.153:6833/1522304272] up:boot 2024-09-19T09:00:10.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:09 smithi196 ceph-mon[35150]: fsmap 2 up:standby 2024-09-19T09:00:10.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:09 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T09:00:10.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:09 smithi196 ceph-mon[35150]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:10.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:09 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:10.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:09 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:10.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:09 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:10.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:09 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:10.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:09 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:10.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:09 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:10.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:09 smithi153 ceph-mon[35071]: Deploying daemon mds.j on smithi120 2024-09-19T09:00:10.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:09 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/168974613' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:00:10.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:09 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.153:6832/1522304272,v1:172.21.15.153:6833/1522304272] up:boot 2024-09-19T09:00:10.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:09 smithi153 ceph-mon[35071]: fsmap 2 up:standby 2024-09-19T09:00:10.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:09 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T09:00:10.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:09 smithi153 ceph-mon[35071]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:11.825 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 19 09:00:11 smithi120 systemd[1]: Starting Ceph mds.j for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:00:12.006 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:00:12.174 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 19 09:00:11 smithi120 podman[70649]: 2024-09-19 09:00:11.82499081 +0000 UTC m=+0.016047120 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:00:12.174 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 19 09:00:11 smithi120 podman[70649]: 2024-09-19 09:00:11.938100244 +0000 UTC m=+0.129156544 container create d3d45bf31a9201ebead060b2bcc534cf252155e1e07edc020a6fe55e0a0951c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-j, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:00:12.341 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:00:12.463 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 19 09:00:12 smithi120 podman[70649]: 2024-09-19 09:00:12.17454166 +0000 UTC m=+0.365597973 container init d3d45bf31a9201ebead060b2bcc534cf252155e1e07edc020a6fe55e0a0951c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-j, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-09-19T09:00:12.464 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 19 09:00:12 smithi120 podman[70649]: 2024-09-19 09:00:12.1784685 +0000 UTC m=+0.369524800 container start d3d45bf31a9201ebead060b2bcc534cf252155e1e07edc020a6fe55e0a0951c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-19T09:00:12.464 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 19 09:00:12 smithi120 ceph-mds[70687]: -- 172.21.15.120:0/2476122017 <== mon.0 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b96ee91c20 con 0x55b96ee6e800 2024-09-19T09:00:12.464 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 19 09:00:12 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-j[70682]: starting mds.j at 2024-09-19T09:00:12.464 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 19 09:00:12 smithi120 bash[70649]: d3d45bf31a9201ebead060b2bcc534cf252155e1e07edc020a6fe55e0a0951c6 2024-09-19T09:00:12.464 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 19 09:00:12 smithi120 systemd[1]: Started Ceph mds.j for 507393d2-7664-11ef-bceb-c7b262605968. 2024-09-19T09:00:12.778 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:12 smithi120 ceph-mon[31150]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:12.779 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:12.779 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:12.779 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:12.779 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:12.779 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:12.779 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:12.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.859+0000 7fdba3546640 1 -- 172.21.15.120:0/4237541355 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb9c103660 msgr2=0x7fdb9c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:12.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.859+0000 7fdba3546640 1 --2- 172.21.15.120:0/4237541355 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb9c103660 0x7fdb9c10bb50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009900 tx=0x7fdb9802a480 comp rx=0 tx=0).stop 2024-09-19T09:00:12.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.859+0000 7fdba3546640 1 -- 172.21.15.120:0/4237541355 shutdown_connections 2024-09-19T09:00:12.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.859+0000 7fdba3546640 1 --2- 172.21.15.120:0/4237541355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb9c10c170 0x7fdb9c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:12.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.859+0000 7fdba3546640 1 --2- 172.21.15.120:0/4237541355 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb9c103660 0x7fdb9c10bb50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:12.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.859+0000 7fdba3546640 1 --2- 172.21.15.120:0/4237541355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb9c102d40 0x7fdb9c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:12.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.859+0000 7fdba3546640 1 -- 172.21.15.120:0/4237541355 >> 172.21.15.120:0/4237541355 conn(0x7fdb9c0fc7a0 msgr2=0x7fdb9c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:12.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.859+0000 7fdba3546640 1 -- 172.21.15.120:0/4237541355 shutdown_connections 2024-09-19T09:00:12.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.859+0000 7fdba3546640 1 -- 172.21.15.120:0/4237541355 wait complete. 2024-09-19T09:00:12.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.860+0000 7fdba3546640 1 Processor -- start 2024-09-19T09:00:12.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.860+0000 7fdba3546640 1 -- start start 2024-09-19T09:00:12.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.860+0000 7fdba3546640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb9c102d40 0x7fdb9c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:12.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.860+0000 7fdba3546640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb9c103660 0x7fdb9c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:12.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.860+0000 7fdba3546640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb9c10c170 0x7fdb9c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:12.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.860+0000 7fdba3546640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c111fb0 con 0x7fdb9c102d40 2024-09-19T09:00:12.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.860+0000 7fdba3546640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c111e30 con 0x7fdb9c10c170 2024-09-19T09:00:12.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.860+0000 7fdba3546640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c112130 con 0x7fdb9c103660 2024-09-19T09:00:12.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.860+0000 7fdba12e2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb9c102d40 0x7fdb9c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:12.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.860+0000 7fdba12e2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb9c102d40 0x7fdb9c1985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54188/0 (socket says 172.21.15.120:54188) 2024-09-19T09:00:12.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.860+0000 7fdba12e2640 1 -- 172.21.15.120:0/1701427264 learned_addr learned my addr 172.21.15.120:0/1701427264 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:00:12.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.860+0000 7fdba1ae3640 1 --2- 172.21.15.120:0/1701427264 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb9c10c170 0x7fdb9c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:12.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.860+0000 7fdba0ae1640 1 --2- 172.21.15.120:0/1701427264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb9c103660 0x7fdb9c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:12.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.860+0000 7fdba12e2640 1 -- 172.21.15.120:0/1701427264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb9c103660 msgr2=0x7fdb9c198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:12.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.860+0000 7fdba12e2640 1 --2- 172.21.15.120:0/1701427264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb9c103660 0x7fdb9c198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:12.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.860+0000 7fdba12e2640 1 -- 172.21.15.120:0/1701427264 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb9c10c170 msgr2=0x7fdb9c19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:12.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.860+0000 7fdba12e2640 1 --2- 172.21.15.120:0/1701427264 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb9c10c170 0x7fdb9c19ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:12.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.860+0000 7fdba12e2640 1 -- 172.21.15.120:0/1701427264 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9c19d560 con 0x7fdb9c102d40 2024-09-19T09:00:12.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.861+0000 7fdba12e2640 1 --2- 172.21.15.120:0/1701427264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb9c102d40 0x7fdb9c1985b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdb9000b4a0 tx=0x7fdb9000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:12.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.861+0000 7fdb8e7fc640 1 -- 172.21.15.120:0/1701427264 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90014070 con 0x7fdb9c102d40 2024-09-19T09:00:12.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.861+0000 7fdb8e7fc640 1 -- 172.21.15.120:0/1701427264 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb90010af0 con 0x7fdb9c102d40 2024-09-19T09:00:12.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.861+0000 7fdba3546640 1 -- 172.21.15.120:0/1701427264 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb9c19d850 con 0x7fdb9c102d40 2024-09-19T09:00:12.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.861+0000 7fdb8e7fc640 1 -- 172.21.15.120:0/1701427264 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90017d10 con 0x7fdb9c102d40 2024-09-19T09:00:12.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.861+0000 7fdba3546640 1 -- 172.21.15.120:0/1701427264 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb9c1a5130 con 0x7fdb9c102d40 2024-09-19T09:00:12.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.862+0000 7fdba3546640 1 -- 172.21.15.120:0/1701427264 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb9c103120 con 0x7fdb9c102d40 2024-09-19T09:00:12.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.863+0000 7fdb8e7fc640 1 -- 172.21.15.120:0/1701427264 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fdb90017070 con 0x7fdb9c102d40 2024-09-19T09:00:12.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.864+0000 7fdb8e7fc640 1 --2- 172.21.15.120:0/1701427264 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdb880a3240 0x7fdb880a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:12.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.864+0000 7fdba0ae1640 1 --2- 172.21.15.120:0/1701427264 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdb880a3240 0x7fdb880a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:12.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.864+0000 7fdb8e7fc640 1 -- 172.21.15.120:0/1701427264 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fdb900cec50 con 0x7fdb9c102d40 2024-09-19T09:00:12.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.864+0000 7fdba0ae1640 1 --2- 172.21.15.120:0/1701427264 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdb880a3240 0x7fdb880a5700 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c10d680 tx=0x7fdb98005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:12.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:12.867+0000 7fdb8e7fc640 1 -- 172.21.15.120:0/1701427264 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdb9009be10 con 0x7fdb9c102d40 2024-09-19T09:00:12.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:12 smithi196 ceph-mon[35150]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:12.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:12 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:12.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:12 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:12.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:12 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:12.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:12 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:12.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:12 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:12.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:12 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:12.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:12 smithi153 ceph-mon[35071]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:12.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:12 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:12.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:12 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:12.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:12 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:12.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:12 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:12.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:12 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:12.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:12 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:13.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:13.081+0000 7fdba3546640 1 -- 172.21.15.120:0/1701427264 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb9c199910 con 0x7fdb9c102d40 2024-09-19T09:00:13.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:13.081+0000 7fdb8e7fc640 1 -- 172.21.15.120:0/1701427264 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3094 (secure 0 0 0) 0x7fdb9009b710 con 0x7fdb9c102d40 2024-09-19T09:00:13.081 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:00:13.081 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":4,"btime":"2024-09-19T09:00:12:653304+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T09:00:13.082 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 4 2024-09-19T09:00:13.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:13.083+0000 7fdba3546640 1 -- 172.21.15.120:0/1701427264 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdb880a3240 msgr2=0x7fdb880a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:13.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:13.083+0000 7fdba3546640 1 --2- 172.21.15.120:0/1701427264 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdb880a3240 0x7fdb880a5700 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c10d680 tx=0x7fdb98005a20 comp rx=0 tx=0).stop 2024-09-19T09:00:13.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:13.083+0000 7fdba3546640 1 -- 172.21.15.120:0/1701427264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb9c102d40 msgr2=0x7fdb9c1985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:13.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:13.083+0000 7fdba3546640 1 --2- 172.21.15.120:0/1701427264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb9c102d40 0x7fdb9c1985b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdb9000b4a0 tx=0x7fdb9000b960 comp rx=0 tx=0).stop 2024-09-19T09:00:13.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:13.083+0000 7fdba3546640 1 -- 172.21.15.120:0/1701427264 shutdown_connections 2024-09-19T09:00:13.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:13.083+0000 7fdba3546640 1 --2- 172.21.15.120:0/1701427264 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdb880a3240 0x7fdb880a5700 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:13.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:13.083+0000 7fdba3546640 1 --2- 172.21.15.120:0/1701427264 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb9c10c170 0x7fdb9c19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:13.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:13.083+0000 7fdba3546640 1 --2- 172.21.15.120:0/1701427264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb9c103660 0x7fdb9c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:13.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:13.083+0000 7fdba3546640 1 --2- 172.21.15.120:0/1701427264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb9c102d40 0x7fdb9c1985b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:13.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:13.083+0000 7fdba3546640 1 -- 172.21.15.120:0/1701427264 >> 172.21.15.120:0/1701427264 conn(0x7fdb9c0fc7a0 msgr2=0x7fdb9c0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:13.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:13.083+0000 7fdba3546640 1 -- 172.21.15.120:0/1701427264 shutdown_connections 2024-09-19T09:00:13.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:13.083+0000 7fdba3546640 1 -- 172.21.15.120:0/1701427264 wait complete. 2024-09-19T09:00:13.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:13 smithi120 ceph-mon[31150]: Deploying daemon mds.d on smithi120 2024-09-19T09:00:13.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:13 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.120:6834/2395347742,v1:172.21.15.120:6835/2395347742] up:boot 2024-09-19T09:00:13.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:13 smithi120 ceph-mon[31150]: fsmap 3 up:standby 2024-09-19T09:00:13.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:13 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T09:00:13.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:13 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1701427264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:00:13.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:13 smithi196 ceph-mon[35150]: Deploying daemon mds.d on smithi120 2024-09-19T09:00:13.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:13 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.120:6834/2395347742,v1:172.21.15.120:6835/2395347742] up:boot 2024-09-19T09:00:13.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:13 smithi196 ceph-mon[35150]: fsmap 3 up:standby 2024-09-19T09:00:13.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:13 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T09:00:13.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:13 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1701427264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:00:13.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:13 smithi153 ceph-mon[35071]: Deploying daemon mds.d on smithi120 2024-09-19T09:00:13.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:13 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.120:6834/2395347742,v1:172.21.15.120:6835/2395347742] up:boot 2024-09-19T09:00:13.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:13 smithi153 ceph-mon[35071]: fsmap 3 up:standby 2024-09-19T09:00:13.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:13 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T09:00:13.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:13 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1701427264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:00:14.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:14 smithi196 ceph-mon[35150]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:14.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:14 smithi153 ceph-mon[35071]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:15.046 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:14 smithi120 ceph-mon[31150]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:15.046 INFO:journalctl@ceph.mds.d.smithi120.stdout:Sep 19 09:00:14 smithi120 systemd[1]: Starting Ceph mds.d for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:00:15.300 INFO:journalctl@ceph.mds.d.smithi120.stdout:Sep 19 09:00:15 smithi120 podman[71058]: 2024-09-19 09:00:15.157098374 +0000 UTC m=+0.016345625 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:00:15.300 INFO:journalctl@ceph.mds.d.smithi120.stdout:Sep 19 09:00:15 smithi120 podman[71058]: 2024-09-19 09:00:15.298072782 +0000 UTC m=+0.157320036 container create c4e057cca4e0c9d81156ebf49ba3c886c674506b3efb56de11b76f5af24867d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-d, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-19T09:00:15.568 INFO:journalctl@ceph.mds.d.smithi120.stdout:Sep 19 09:00:15 smithi120 podman[71058]: 2024-09-19 09:00:15.542747999 +0000 UTC m=+0.401995255 container init c4e057cca4e0c9d81156ebf49ba3c886c674506b3efb56de11b76f5af24867d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-d, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-19T09:00:15.568 INFO:journalctl@ceph.mds.d.smithi120.stdout:Sep 19 09:00:15 smithi120 podman[71058]: 2024-09-19 09:00:15.545451193 +0000 UTC m=+0.404698446 container start c4e057cca4e0c9d81156ebf49ba3c886c674506b3efb56de11b76f5af24867d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-d, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:00:15.824 INFO:journalctl@ceph.mds.d.smithi120.stdout:Sep 19 09:00:15 smithi120 ceph-mds[71077]: -- 172.21.15.120:0/573622523 <== mon.0 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eeda651860 con 0x55eeda62f400 2024-09-19T09:00:15.824 INFO:journalctl@ceph.mds.d.smithi120.stdout:Sep 19 09:00:15 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-d[71073]: starting mds.d at 2024-09-19T09:00:15.825 INFO:journalctl@ceph.mds.d.smithi120.stdout:Sep 19 09:00:15 smithi120 bash[71058]: c4e057cca4e0c9d81156ebf49ba3c886c674506b3efb56de11b76f5af24867d4 2024-09-19T09:00:15.825 INFO:journalctl@ceph.mds.d.smithi120.stdout:Sep 19 09:00:15 smithi120 systemd[1]: Started Ceph mds.d for 507393d2-7664-11ef-bceb-c7b262605968. 2024-09-19T09:00:16.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:16 smithi196 ceph-mon[35150]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:16.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:16 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.120:6836/217332579,v1:172.21.15.120:6837/217332579] up:boot 2024-09-19T09:00:16.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:16 smithi196 ceph-mon[35150]: fsmap 4 up:standby 2024-09-19T09:00:16.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T09:00:16.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:16.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:16.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:16.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:16.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:16.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:16.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:16 smithi196 ceph-mon[35150]: Deploying daemon mds.e on smithi153 2024-09-19T09:00:16.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:16 smithi153 ceph-mon[35071]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:16.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:16 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.120:6836/217332579,v1:172.21.15.120:6837/217332579] up:boot 2024-09-19T09:00:16.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:16 smithi153 ceph-mon[35071]: fsmap 4 up:standby 2024-09-19T09:00:16.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T09:00:16.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:16.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:16.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:16.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:16.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:16.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:16.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:16 smithi153 ceph-mon[35071]: Deploying daemon mds.e on smithi153 2024-09-19T09:00:17.074 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:00:17.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:16 smithi120 ceph-mon[31150]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:17.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:16 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.120:6836/217332579,v1:172.21.15.120:6837/217332579] up:boot 2024-09-19T09:00:17.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:16 smithi120 ceph-mon[31150]: fsmap 4 up:standby 2024-09-19T09:00:17.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T09:00:17.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:17.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:17.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:17.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:17.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:17.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:17.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:16 smithi120 ceph-mon[31150]: Deploying daemon mds.e on smithi153 2024-09-19T09:00:17.334 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:00:17.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.836+0000 7fb6135e0640 1 -- 172.21.15.120:0/1026789119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb60c101de0 msgr2=0x7fb60c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:17.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.836+0000 7fb6135e0640 1 --2- 172.21.15.120:0/1026789119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb60c101de0 0x7fb60c1021c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc009900 tx=0x7fb5fc02a410 comp rx=0 tx=0).stop 2024-09-19T09:00:17.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.836+0000 7fb6135e0640 1 -- 172.21.15.120:0/1026789119 shutdown_connections 2024-09-19T09:00:17.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.836+0000 7fb6135e0640 1 --2- 172.21.15.120:0/1026789119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb60c106f10 0x7fb60c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:17.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.836+0000 7fb6135e0640 1 --2- 172.21.15.120:0/1026789119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb60c102790 0x7fb60c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:17.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.836+0000 7fb6135e0640 1 --2- 172.21.15.120:0/1026789119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb60c101de0 0x7fb60c1021c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:17.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.836+0000 7fb6135e0640 1 -- 172.21.15.120:0/1026789119 >> 172.21.15.120:0/1026789119 conn(0x7fb60c0fd8c0 msgr2=0x7fb60c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:17.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.836+0000 7fb6135e0640 1 -- 172.21.15.120:0/1026789119 shutdown_connections 2024-09-19T09:00:17.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.836+0000 7fb6135e0640 1 -- 172.21.15.120:0/1026789119 wait complete. 2024-09-19T09:00:17.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.837+0000 7fb6135e0640 1 Processor -- start 2024-09-19T09:00:17.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.837+0000 7fb6135e0640 1 -- start start 2024-09-19T09:00:17.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.837+0000 7fb6135e0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb60c101de0 0x7fb60c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:17.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.837+0000 7fb6135e0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb60c102790 0x7fb60c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:17.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.837+0000 7fb6135e0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb60c106f10 0x7fb60c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:17.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.837+0000 7fb6135e0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c111f70 con 0x7fb60c102790 2024-09-19T09:00:17.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.837+0000 7fb6135e0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c111df0 con 0x7fb60c101de0 2024-09-19T09:00:17.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.837+0000 7fb6135e0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c1120f0 con 0x7fb60c106f10 2024-09-19T09:00:17.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.837+0000 7fb610b7b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb60c102790 0x7fb60c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:17.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.837+0000 7fb610b7b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb60c102790 0x7fb60c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54226/0 (socket says 172.21.15.120:54226) 2024-09-19T09:00:17.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.837+0000 7fb610b7b640 1 -- 172.21.15.120:0/3290580802 learned_addr learned my addr 172.21.15.120:0/3290580802 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:00:17.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.837+0000 7fb61137c640 1 --2- 172.21.15.120:0/3290580802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb60c101de0 0x7fb60c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:17.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.838+0000 7fb610b7b640 1 -- 172.21.15.120:0/3290580802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb60c106f10 msgr2=0x7fb60c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:17.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.838+0000 7fb610b7b640 1 --2- 172.21.15.120:0/3290580802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb60c106f10 0x7fb60c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:17.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.838+0000 7fb610b7b640 1 -- 172.21.15.120:0/3290580802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb60c101de0 msgr2=0x7fb60c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:17.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.838+0000 7fb610b7b640 1 --2- 172.21.15.120:0/3290580802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb60c101de0 0x7fb60c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:17.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.838+0000 7fb610b7b640 1 -- 172.21.15.120:0/3290580802 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60c1a6990 con 0x7fb60c102790 2024-09-19T09:00:17.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.838+0000 7fb610b7b640 1 --2- 172.21.15.120:0/3290580802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb60c102790 0x7fb60c074a30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb60800e9f0 tx=0x7fb60800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:17.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.838+0000 7fb6027fc640 1 -- 172.21.15.120:0/3290580802 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60800cd80 con 0x7fb60c102790 2024-09-19T09:00:17.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.838+0000 7fb6027fc640 1 -- 172.21.15.120:0/3290580802 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb608010910 con 0x7fb60c102790 2024-09-19T09:00:17.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.838+0000 7fb6135e0640 1 -- 172.21.15.120:0/3290580802 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb60c1a6c80 con 0x7fb60c102790 2024-09-19T09:00:17.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.838+0000 7fb6027fc640 1 -- 172.21.15.120:0/3290580802 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb608018990 con 0x7fb60c102790 2024-09-19T09:00:17.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.838+0000 7fb6135e0640 1 -- 172.21.15.120:0/3290580802 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb60c1a71c0 con 0x7fb60c102790 2024-09-19T09:00:17.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.839+0000 7fb6135e0640 1 -- 172.21.15.120:0/3290580802 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb60c1021c0 con 0x7fb60c102790 2024-09-19T09:00:17.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.840+0000 7fb6027fc640 1 -- 172.21.15.120:0/3290580802 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fb608010ac0 con 0x7fb60c102790 2024-09-19T09:00:17.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.841+0000 7fb6027fc640 1 --2- 172.21.15.120:0/3290580802 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb5f80a32c0 0x7fb5f80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:17.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.841+0000 7fb61137c640 1 --2- 172.21.15.120:0/3290580802 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb5f80a32c0 0x7fb5f80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:17.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.841+0000 7fb6027fc640 1 -- 172.21.15.120:0/3290580802 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fb6080cfc20 con 0x7fb60c102790 2024-09-19T09:00:17.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.841+0000 7fb61137c640 1 --2- 172.21.15.120:0/3290580802 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb5f80a32c0 0x7fb5f80a5780 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc009230 tx=0x7fb5fc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:17.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:17.844+0000 7fb6027fc640 1 -- 172.21.15.120:0/3290580802 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb60809cd40 con 0x7fb60c102790 2024-09-19T09:00:17.918 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:00:17 smithi153 systemd[1]: Starting Ceph mds.e for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:00:18.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:18.044+0000 7fb6135e0640 1 -- 172.21.15.120:0/3290580802 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb60c075a20 con 0x7fb60c102790 2024-09-19T09:00:18.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:18.045+0000 7fb6027fc640 1 -- 172.21.15.120:0/3290580802 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3919 (secure 0 0 0) 0x7fb608098070 con 0x7fb60c102790 2024-09-19T09:00:18.045 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:00:18.045 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":5,"btime":"2024-09-19T09:00:15:668406+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T09:00:18.046 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 5 2024-09-19T09:00:18.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:18.047+0000 7fb6135e0640 1 -- 172.21.15.120:0/3290580802 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb5f80a32c0 msgr2=0x7fb5f80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:18.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:18.047+0000 7fb6135e0640 1 --2- 172.21.15.120:0/3290580802 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb5f80a32c0 0x7fb5f80a5780 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc009230 tx=0x7fb5fc0058d0 comp rx=0 tx=0).stop 2024-09-19T09:00:18.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:18.047+0000 7fb6135e0640 1 -- 172.21.15.120:0/3290580802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb60c102790 msgr2=0x7fb60c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:18.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:18.047+0000 7fb6135e0640 1 --2- 172.21.15.120:0/3290580802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb60c102790 0x7fb60c074a30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb60800e9f0 tx=0x7fb60800eeb0 comp rx=0 tx=0).stop 2024-09-19T09:00:18.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:18.047+0000 7fb6135e0640 1 -- 172.21.15.120:0/3290580802 shutdown_connections 2024-09-19T09:00:18.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:18.047+0000 7fb6135e0640 1 --2- 172.21.15.120:0/3290580802 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb5f80a32c0 0x7fb5f80a5780 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:18.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:18.047+0000 7fb6135e0640 1 --2- 172.21.15.120:0/3290580802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb60c106f10 0x7fb60c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:18.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:18.047+0000 7fb6135e0640 1 --2- 172.21.15.120:0/3290580802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb60c102790 0x7fb60c074a30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:18.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:18.047+0000 7fb6135e0640 1 --2- 172.21.15.120:0/3290580802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb60c101de0 0x7fb60c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:18.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:18.047+0000 7fb6135e0640 1 -- 172.21.15.120:0/3290580802 >> 172.21.15.120:0/3290580802 conn(0x7fb60c0fd8c0 msgr2=0x7fb60c0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:18.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:18.047+0000 7fb6135e0640 1 -- 172.21.15.120:0/3290580802 shutdown_connections 2024-09-19T09:00:18.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:18.047+0000 7fb6135e0640 1 -- 172.21.15.120:0/3290580802 wait complete. 2024-09-19T09:00:18.177 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:00:17 smithi153 podman[63895]: 2024-09-19 09:00:17.821662079 +0000 UTC m=+0.015663646 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:00:18.178 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:00:17 smithi153 podman[63895]: 2024-09-19 09:00:17.957651066 +0000 UTC m=+0.151652641 container create 8b60695bc827d25ff69cf5167dff1ccec15fc4ecac3153f1a544b35e4b1a4bea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-e, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-19T09:00:18.178 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:00:18 smithi153 podman[63895]: 2024-09-19 09:00:18.151879327 +0000 UTC m=+0.345880894 container init 8b60695bc827d25ff69cf5167dff1ccec15fc4ecac3153f1a544b35e4b1a4bea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-e, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4) 2024-09-19T09:00:18.178 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:00:18 smithi153 podman[63895]: 2024-09-19 09:00:18.154586578 +0000 UTC m=+0.348588144 container start 8b60695bc827d25ff69cf5167dff1ccec15fc4ecac3153f1a544b35e4b1a4bea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD) 2024-09-19T09:00:18.437 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:00:18 smithi153 ceph-mds[63915]: -- 172.21.15.153:0/1293313082 <== mon.2 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5586506bd860 con 0x55865069a800 2024-09-19T09:00:18.437 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:00:18 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-e[63911]: starting mds.e at 2024-09-19T09:00:18.437 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:00:18 smithi153 bash[63895]: 8b60695bc827d25ff69cf5167dff1ccec15fc4ecac3153f1a544b35e4b1a4bea 2024-09-19T09:00:18.437 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:00:18 smithi153 systemd[1]: Started Ceph mds.e for 507393d2-7664-11ef-bceb-c7b262605968. 2024-09-19T09:00:18.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:18 smithi196 ceph-mon[35150]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:18.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:18 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3290580802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:00:18.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:18 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:18.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:18 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:18.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:18 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:18.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:18 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:18.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:18 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:18.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:18 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:18.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:18 smithi153 ceph-mon[35071]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:18.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:18 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3290580802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:00:18.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:18 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:18.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:18 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:18.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:18 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:18.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:18 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:18.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:18 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:18.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:18 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:19.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:18 smithi120 ceph-mon[31150]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:19.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:18 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3290580802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:00:19.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:18 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:19.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:18 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:19.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:18 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:19.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:18 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:19.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:18 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:19.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:18 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:20.020 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:19 smithi196 ceph-mon[35150]: Deploying daemon mds.f on smithi196 2024-09-19T09:00:20.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:19 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.153:6834/3103515283,v1:172.21.15.153:6835/3103515283] up:boot 2024-09-19T09:00:20.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:19 smithi196 ceph-mon[35150]: fsmap 5 up:standby 2024-09-19T09:00:20.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:19 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:00:20.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:19 smithi120 ceph-mon[31150]: Deploying daemon mds.f on smithi196 2024-09-19T09:00:20.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:19 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.153:6834/3103515283,v1:172.21.15.153:6835/3103515283] up:boot 2024-09-19T09:00:20.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:19 smithi120 ceph-mon[31150]: fsmap 5 up:standby 2024-09-19T09:00:20.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:19 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:00:20.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:19 smithi153 ceph-mon[35071]: Deploying daemon mds.f on smithi196 2024-09-19T09:00:20.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:19 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.153:6834/3103515283,v1:172.21.15.153:6835/3103515283] up:boot 2024-09-19T09:00:20.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:19 smithi153 ceph-mon[35071]: fsmap 5 up:standby 2024-09-19T09:00:20.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:19 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:00:20.417 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 09:00:20 smithi196 systemd[1]: Starting Ceph mds.f for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:00:20.676 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 09:00:20 smithi196 podman[64706]: 2024-09-19 09:00:20.320685046 +0000 UTC m=+0.015855338 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:00:20.676 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 09:00:20 smithi196 podman[64706]: 2024-09-19 09:00:20.449043796 +0000 UTC m=+0.144214147 container create 79f3de5e93f2b475954bd25128db46ee6242d9af109d736359009d90e2897c5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-f, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-19T09:00:20.676 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 09:00:20 smithi196 podman[64706]: 2024-09-19 09:00:20.676815966 +0000 UTC m=+0.371986271 container init 79f3de5e93f2b475954bd25128db46ee6242d9af109d736359009d90e2897c5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-f, ceph=True, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-19T09:00:20.865 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:00:20.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:20 smithi196 ceph-mon[35150]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:20.936 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 09:00:20 smithi196 podman[64706]: 2024-09-19 09:00:20.679547504 +0000 UTC m=+0.374717797 container start 79f3de5e93f2b475954bd25128db46ee6242d9af109d736359009d90e2897c5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2) 2024-09-19T09:00:20.936 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 09:00:20 smithi196 ceph-mds[64725]: -- 172.21.15.196:0/4071679776 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55836ca07860 con 0x55836c9e5000 2024-09-19T09:00:20.936 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 09:00:20 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-f[64721]: starting mds.f at 2024-09-19T09:00:20.936 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 09:00:20 smithi196 bash[64706]: 79f3de5e93f2b475954bd25128db46ee6242d9af109d736359009d90e2897c5c 2024-09-19T09:00:20.936 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 09:00:20 smithi196 systemd[1]: Started Ceph mds.f for 507393d2-7664-11ef-bceb-c7b262605968. 2024-09-19T09:00:21.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:20 smithi120 ceph-mon[31150]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:21.116 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:00:21.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:20 smithi153 ceph-mon[35071]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:21.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.638+0000 7fbfe3d86640 1 -- 172.21.15.120:0/1810387526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbfdc102790 msgr2=0x7fbfdc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:21.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.638+0000 7fbfe3d86640 1 --2- 172.21.15.120:0/1810387526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbfdc102790 0x7fbfdc1067e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc009900 tx=0x7fbfcc02a410 comp rx=0 tx=0).stop 2024-09-19T09:00:21.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.638+0000 7fbfe3d86640 1 -- 172.21.15.120:0/1810387526 shutdown_connections 2024-09-19T09:00:21.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.638+0000 7fbfe3d86640 1 --2- 172.21.15.120:0/1810387526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfdc106f10 0x7fbfdc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:21.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.638+0000 7fbfe3d86640 1 --2- 172.21.15.120:0/1810387526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbfdc102790 0x7fbfdc1067e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:21.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.638+0000 7fbfe3d86640 1 --2- 172.21.15.120:0/1810387526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbfdc101de0 0x7fbfdc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:21.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.638+0000 7fbfe3d86640 1 -- 172.21.15.120:0/1810387526 >> 172.21.15.120:0/1810387526 conn(0x7fbfdc0fd8c0 msgr2=0x7fbfdc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:21.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.638+0000 7fbfe3d86640 1 -- 172.21.15.120:0/1810387526 shutdown_connections 2024-09-19T09:00:21.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.638+0000 7fbfe3d86640 1 -- 172.21.15.120:0/1810387526 wait complete. 2024-09-19T09:00:21.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.639+0000 7fbfe3d86640 1 Processor -- start 2024-09-19T09:00:21.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.639+0000 7fbfe3d86640 1 -- start start 2024-09-19T09:00:21.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.639+0000 7fbfe3d86640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbfdc101de0 0x7fbfdc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:21.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.639+0000 7fbfe3d86640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfdc102790 0x7fbfdc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:21.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.639+0000 7fbfe3d86640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbfdc106f10 0x7fbfdc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:21.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.639+0000 7fbfe3d86640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbfdc111f70 con 0x7fbfdc106f10 2024-09-19T09:00:21.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.639+0000 7fbfe3d86640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbfdc111df0 con 0x7fbfdc101de0 2024-09-19T09:00:21.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.639+0000 7fbfe3d86640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbfdc1120f0 con 0x7fbfdc102790 2024-09-19T09:00:21.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.639+0000 7fbfe2323640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbfdc106f10 0x7fbfdc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:21.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.639+0000 7fbfe2323640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbfdc106f10 0x7fbfdc074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49914/0 (socket says 172.21.15.120:49914) 2024-09-19T09:00:21.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.639+0000 7fbfe2323640 1 -- 172.21.15.120:0/1222864283 learned_addr learned my addr 172.21.15.120:0/1222864283 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:00:21.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.639+0000 7fbfe1b22640 1 --2- 172.21.15.120:0/1222864283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbfdc101de0 0x7fbfdc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:21.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.639+0000 7fbfe1321640 1 --2- 172.21.15.120:0/1222864283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfdc102790 0x7fbfdc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:21.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.640+0000 7fbfe2323640 1 -- 172.21.15.120:0/1222864283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfdc102790 msgr2=0x7fbfdc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:21.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.640+0000 7fbfe2323640 1 --2- 172.21.15.120:0/1222864283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfdc102790 0x7fbfdc074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:21.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.640+0000 7fbfe2323640 1 -- 172.21.15.120:0/1222864283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbfdc101de0 msgr2=0x7fbfdc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:21.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.640+0000 7fbfe2323640 1 --2- 172.21.15.120:0/1222864283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbfdc101de0 0x7fbfdc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:21.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.640+0000 7fbfe2323640 1 -- 172.21.15.120:0/1222864283 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfdc1a6990 con 0x7fbfdc106f10 2024-09-19T09:00:21.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.640+0000 7fbfe1321640 1 --2- 172.21.15.120:0/1222864283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfdc102790 0x7fbfdc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:00:21.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.640+0000 7fbfe2323640 1 --2- 172.21.15.120:0/1222864283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbfdc106f10 0x7fbfdc074f70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbfd400cad0 tx=0x7fbfd400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:21.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.640+0000 7fbfd2ffd640 1 -- 172.21.15.120:0/1222864283 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd400ae20 con 0x7fbfdc106f10 2024-09-19T09:00:21.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.640+0000 7fbfd2ffd640 1 -- 172.21.15.120:0/1222864283 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfd4005020 con 0x7fbfdc106f10 2024-09-19T09:00:21.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.640+0000 7fbfd2ffd640 1 -- 172.21.15.120:0/1222864283 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd400f6f0 con 0x7fbfdc106f10 2024-09-19T09:00:21.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.640+0000 7fbfe3d86640 1 -- 172.21.15.120:0/1222864283 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfdc1a6c80 con 0x7fbfdc106f10 2024-09-19T09:00:21.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.640+0000 7fbfe3d86640 1 -- 172.21.15.120:0/1222864283 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfdc1a71c0 con 0x7fbfdc106f10 2024-09-19T09:00:21.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.641+0000 7fbfe3d86640 1 -- 172.21.15.120:0/1222864283 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfdc1021c0 con 0x7fbfdc106f10 2024-09-19T09:00:21.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.642+0000 7fbfd2ffd640 1 -- 172.21.15.120:0/1222864283 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fbfd40051d0 con 0x7fbfdc106f10 2024-09-19T09:00:21.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.643+0000 7fbfd2ffd640 1 --2- 172.21.15.120:0/1222864283 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbfc80a32c0 0x7fbfc80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:21.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.643+0000 7fbfe1b22640 1 --2- 172.21.15.120:0/1222864283 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbfc80a32c0 0x7fbfc80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:21.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.643+0000 7fbfd2ffd640 1 -- 172.21.15.120:0/1222864283 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fbfd40cf7c0 con 0x7fbfdc106f10 2024-09-19T09:00:21.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.644+0000 7fbfe1b22640 1 --2- 172.21.15.120:0/1222864283 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbfc80a32c0 0x7fbfc80a5780 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800bbe0 tx=0x7fbfd800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:21.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.647+0000 7fbfd2ffd640 1 -- 172.21.15.120:0/1222864283 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbfd4014030 con 0x7fbfdc106f10 2024-09-19T09:00:21.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.851+0000 7fbfe3d86640 1 -- 172.21.15.120:0/1222864283 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbfdc075a20 con 0x7fbfdc106f10 2024-09-19T09:00:21.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.852+0000 7fbfd2ffd640 1 -- 172.21.15.120:0/1222864283 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5575 (secure 0 0 0) 0x7fbfdc075a20 con 0x7fbfdc106f10 2024-09-19T09:00:21.852 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:00:21.852 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":7,"btime":"2024-09-19T09:00:21:689613+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-19T09:00:21.852 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 7 2024-09-19T09:00:21.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.854+0000 7fbfe3d86640 1 -- 172.21.15.120:0/1222864283 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbfc80a32c0 msgr2=0x7fbfc80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:21.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.854+0000 7fbfe3d86640 1 --2- 172.21.15.120:0/1222864283 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbfc80a32c0 0x7fbfc80a5780 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800bbe0 tx=0x7fbfd800b390 comp rx=0 tx=0).stop 2024-09-19T09:00:21.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.854+0000 7fbfe3d86640 1 -- 172.21.15.120:0/1222864283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbfdc106f10 msgr2=0x7fbfdc074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:21.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.854+0000 7fbfe3d86640 1 --2- 172.21.15.120:0/1222864283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbfdc106f10 0x7fbfdc074f70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbfd400cad0 tx=0x7fbfd400cf90 comp rx=0 tx=0).stop 2024-09-19T09:00:21.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.854+0000 7fbfe3d86640 1 -- 172.21.15.120:0/1222864283 shutdown_connections 2024-09-19T09:00:21.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.854+0000 7fbfe3d86640 1 --2- 172.21.15.120:0/1222864283 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbfc80a32c0 0x7fbfc80a5780 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:21.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.854+0000 7fbfe3d86640 1 --2- 172.21.15.120:0/1222864283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbfdc106f10 0x7fbfdc074f70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:21.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.854+0000 7fbfe3d86640 1 --2- 172.21.15.120:0/1222864283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfdc102790 0x7fbfdc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:21.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.854+0000 7fbfe3d86640 1 --2- 172.21.15.120:0/1222864283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbfdc101de0 0x7fbfdc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:21.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.854+0000 7fbfe3d86640 1 -- 172.21.15.120:0/1222864283 >> 172.21.15.120:0/1222864283 conn(0x7fbfdc0fd8c0 msgr2=0x7fbfdc0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:21.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.854+0000 7fbfe3d86640 1 -- 172.21.15.120:0/1222864283 shutdown_connections 2024-09-19T09:00:21.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:21.854+0000 7fbfe3d86640 1 -- 172.21.15.120:0/1222864283 wait complete. 2024-09-19T09:00:21.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:21 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:21.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:21 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:21.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:21 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:21.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:21 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:21.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:21 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:21.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:21 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:21.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:21 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] up:boot 2024-09-19T09:00:21.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:21 smithi196 ceph-mon[35150]: fsmap 6 up:standby 2024-09-19T09:00:21.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:21 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T09:00:21.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:21 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:21.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:21 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:21.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:21 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:21.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:21 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:21.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:21 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:21.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:21 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:21.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:21 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] up:boot 2024-09-19T09:00:21.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:21 smithi153 ceph-mon[35071]: fsmap 6 up:standby 2024-09-19T09:00:21.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:21 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T09:00:22.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:21 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:22.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:21 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:22.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:21 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:22.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:21 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:22.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:21 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:22.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:21 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:22.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:21 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] up:boot 2024-09-19T09:00:22.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:21 smithi120 ceph-mon[31150]: fsmap 6 up:standby 2024-09-19T09:00:22.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:21 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T09:00:22.820 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 09:00:22 smithi196 systemd[1]: Starting Ceph mds.c for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:00:23.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:22 smithi120 ceph-mon[31150]: Deploying daemon mds.c on smithi196 2024-09-19T09:00:23.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:22 smithi120 ceph-mon[31150]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:23.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:22 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1222864283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:00:23.181 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:22 smithi196 ceph-mon[35150]: Deploying daemon mds.c on smithi196 2024-09-19T09:00:23.181 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:22 smithi196 ceph-mon[35150]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:23.181 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:22 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1222864283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:00:23.181 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 09:00:22 smithi196 podman[65006]: 2024-09-19 09:00:22.825713453 +0000 UTC m=+0.016533947 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:00:23.181 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 09:00:22 smithi196 podman[65006]: 2024-09-19 09:00:22.929007991 +0000 UTC m=+0.119828543 container create ae427bccb0d4288f9ed0779455d2aa9a152ceab7bb56829c0f3dd55a064ef3c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-c, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-19T09:00:23.181 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 09:00:23 smithi196 podman[65006]: 2024-09-19 09:00:23.181865235 +0000 UTC m=+0.372685731 container init ae427bccb0d4288f9ed0779455d2aa9a152ceab7bb56829c0f3dd55a064ef3c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True) 2024-09-19T09:00:23.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:22 smithi153 ceph-mon[35071]: Deploying daemon mds.c on smithi196 2024-09-19T09:00:23.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:22 smithi153 ceph-mon[35071]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:23.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:22 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1222864283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:00:23.435 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 09:00:23 smithi196 podman[65006]: 2024-09-19 09:00:23.184820157 +0000 UTC m=+0.375640651 container start ae427bccb0d4288f9ed0779455d2aa9a152ceab7bb56829c0f3dd55a064ef3c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-09-19T09:00:23.436 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 09:00:23 smithi196 ceph-mds[65026]: -- 172.21.15.196:0/2251991099 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ac355a1860 con 0x55ac3557f400 2024-09-19T09:00:23.436 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 09:00:23 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-c[65022]: starting mds.c at 2024-09-19T09:00:23.436 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 09:00:23 smithi196 bash[65006]: ae427bccb0d4288f9ed0779455d2aa9a152ceab7bb56829c0f3dd55a064ef3c8 2024-09-19T09:00:23.436 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 09:00:23 smithi196 systemd[1]: Started Ceph mds.c for 507393d2-7664-11ef-bceb-c7b262605968. 2024-09-19T09:00:24.578 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:00:24.606 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:24 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:24.606 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:24 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:24.606 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:24 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:24.607 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:24 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:24.607 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:24 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:24.607 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:24 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:24.607 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:24 smithi120 ceph-mon[31150]: Deploying daemon mds.h on smithi153 2024-09-19T09:00:24.607 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:24 smithi120 ceph-mon[31150]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:24.607 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:24 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] up:boot 2024-09-19T09:00:24.607 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:24 smithi120 ceph-mon[31150]: fsmap 7 up:standby 2024-09-19T09:00:24.607 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:24 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T09:00:24.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:24 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:24.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:24 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:24.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:24 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:24.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:24 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:24.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:24 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:24.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:24 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:24.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:24 smithi196 ceph-mon[35150]: Deploying daemon mds.h on smithi153 2024-09-19T09:00:24.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:24 smithi196 ceph-mon[35150]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:24.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:24 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] up:boot 2024-09-19T09:00:24.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:24 smithi196 ceph-mon[35150]: fsmap 7 up:standby 2024-09-19T09:00:24.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:24 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T09:00:24.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:24 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:24.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:24 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:24.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:24 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:24.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:24 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:24.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:24 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:24.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:24 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:24.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:24 smithi153 ceph-mon[35071]: Deploying daemon mds.h on smithi153 2024-09-19T09:00:24.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:24 smithi153 ceph-mon[35071]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:24.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:24 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] up:boot 2024-09-19T09:00:24.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:24 smithi153 ceph-mon[35071]: fsmap 7 up:standby 2024-09-19T09:00:24.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:24 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T09:00:24.833 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:00:25.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.339+0000 7f3f160fa640 1 -- 172.21.15.120:0/2213474410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f10101de0 msgr2=0x7f3f101021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:25.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.339+0000 7f3f160fa640 1 --2- 172.21.15.120:0/2213474410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f10101de0 0x7f3f101021c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3f00009900 tx=0x7f3f0002a410 comp rx=0 tx=0).stop 2024-09-19T09:00:25.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.339+0000 7f3f160fa640 1 -- 172.21.15.120:0/2213474410 shutdown_connections 2024-09-19T09:00:25.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.339+0000 7f3f160fa640 1 --2- 172.21.15.120:0/2213474410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f10106f10 0x7f3f1010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:25.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.339+0000 7f3f160fa640 1 --2- 172.21.15.120:0/2213474410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f10102790 0x7f3f101067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:25.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.339+0000 7f3f160fa640 1 --2- 172.21.15.120:0/2213474410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f10101de0 0x7f3f101021c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:25.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.339+0000 7f3f160fa640 1 -- 172.21.15.120:0/2213474410 >> 172.21.15.120:0/2213474410 conn(0x7f3f100fd8c0 msgr2=0x7f3f100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:25.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.339+0000 7f3f160fa640 1 -- 172.21.15.120:0/2213474410 shutdown_connections 2024-09-19T09:00:25.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.339+0000 7f3f160fa640 1 -- 172.21.15.120:0/2213474410 wait complete. 2024-09-19T09:00:25.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.340+0000 7f3f160fa640 1 Processor -- start 2024-09-19T09:00:25.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.340+0000 7f3f160fa640 1 -- start start 2024-09-19T09:00:25.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.340+0000 7f3f160fa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f10101de0 0x7f3f101984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:25.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.340+0000 7f3f160fa640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f10102790 0x7f3f10198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:25.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.340+0000 7f3f160fa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f10106f10 0x7f3f1019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:25.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.340+0000 7f3f160fa640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3f10111f70 con 0x7f3f10106f10 2024-09-19T09:00:25.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.340+0000 7f3f160fa640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3f10111df0 con 0x7f3f10102790 2024-09-19T09:00:25.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.340+0000 7f3f160fa640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101120f0 con 0x7f3f10101de0 2024-09-19T09:00:25.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.340+0000 7f3f0ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f10106f10 0x7f3f1019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:25.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.340+0000 7f3f0ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f10106f10 0x7f3f1019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49932/0 (socket says 172.21.15.120:49932) 2024-09-19T09:00:25.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.340+0000 7f3f0ffff640 1 -- 172.21.15.120:0/343675916 learned_addr learned my addr 172.21.15.120:0/343675916 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:00:25.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.340+0000 7f3f0effd640 1 --2- 172.21.15.120:0/343675916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f10102790 0x7f3f10198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:25.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.340+0000 7f3f0f7fe640 1 --2- 172.21.15.120:0/343675916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f10101de0 0x7f3f101984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:25.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.340+0000 7f3f0ffff640 1 -- 172.21.15.120:0/343675916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f10101de0 msgr2=0x7f3f101984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:25.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.340+0000 7f3f0ffff640 1 --2- 172.21.15.120:0/343675916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f10101de0 0x7f3f101984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:25.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.340+0000 7f3f0ffff640 1 -- 172.21.15.120:0/343675916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f10102790 msgr2=0x7f3f10198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:25.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.340+0000 7f3f0ffff640 1 --2- 172.21.15.120:0/343675916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f10102790 0x7f3f10198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:25.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.340+0000 7f3f0ffff640 1 -- 172.21.15.120:0/343675916 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1019d480 con 0x7f3f10106f10 2024-09-19T09:00:25.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.340+0000 7f3f0effd640 1 --2- 172.21.15.120:0/343675916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f10102790 0x7f3f10198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:00:25.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.341+0000 7f3f0ffff640 1 --2- 172.21.15.120:0/343675916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f10106f10 0x7f3f1019cda0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3f0800eaa0 tx=0x7f3f0800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:25.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.341+0000 7f3f0cff9640 1 -- 172.21.15.120:0/343675916 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0800cdb0 con 0x7f3f10106f10 2024-09-19T09:00:25.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.341+0000 7f3f160fa640 1 -- 172.21.15.120:0/343675916 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f1019d770 con 0x7f3f10106f10 2024-09-19T09:00:25.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.341+0000 7f3f0cff9640 1 -- 172.21.15.120:0/343675916 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f08005020 con 0x7f3f10106f10 2024-09-19T09:00:25.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.341+0000 7f3f0cff9640 1 -- 172.21.15.120:0/343675916 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f080116b0 con 0x7f3f10106f10 2024-09-19T09:00:25.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.341+0000 7f3f160fa640 1 -- 172.21.15.120:0/343675916 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f101a5050 con 0x7f3f10106f10 2024-09-19T09:00:25.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.342+0000 7f3f160fa640 1 -- 172.21.15.120:0/343675916 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f101021c0 con 0x7f3f10106f10 2024-09-19T09:00:25.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.343+0000 7f3f0cff9640 1 -- 172.21.15.120:0/343675916 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f3f08005560 con 0x7f3f10106f10 2024-09-19T09:00:25.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.344+0000 7f3f0cff9640 1 --2- 172.21.15.120:0/343675916 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3efc0a3240 0x7f3efc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:25.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.344+0000 7f3f0f7fe640 1 --2- 172.21.15.120:0/343675916 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3efc0a3240 0x7f3efc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:25.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.344+0000 7f3f0cff9640 1 -- 172.21.15.120:0/343675916 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f3f080cf5a0 con 0x7f3f10106f10 2024-09-19T09:00:25.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.344+0000 7f3f0f7fe640 1 --2- 172.21.15.120:0/343675916 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3efc0a3240 0x7f3efc0a5700 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3f00009230 tx=0x7f3f000058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:25.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.348+0000 7f3f0cff9640 1 -- 172.21.15.120:0/343675916 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3f08016030 con 0x7f3f10106f10 2024-09-19T09:00:25.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.549+0000 7f3f160fa640 1 -- 172.21.15.120:0/343675916 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f10109b60 con 0x7f3f10106f10 2024-09-19T09:00:25.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.550+0000 7f3f0cff9640 1 -- 172.21.15.120:0/343675916 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6400 (secure 0 0 0) 0x7f3f0809c110 con 0x7f3f10106f10 2024-09-19T09:00:25.551 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:00:25.551 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":8,"btime":"2024-09-19T09:00:23:820670+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-19T09:00:25.552 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 8 2024-09-19T09:00:25.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.553+0000 7f3f160fa640 1 -- 172.21.15.120:0/343675916 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3efc0a3240 msgr2=0x7f3efc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:25.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.553+0000 7f3f160fa640 1 --2- 172.21.15.120:0/343675916 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3efc0a3240 0x7f3efc0a5700 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3f00009230 tx=0x7f3f000058d0 comp rx=0 tx=0).stop 2024-09-19T09:00:25.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.553+0000 7f3f160fa640 1 -- 172.21.15.120:0/343675916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f10106f10 msgr2=0x7f3f1019cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:25.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.553+0000 7f3f160fa640 1 --2- 172.21.15.120:0/343675916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f10106f10 0x7f3f1019cda0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3f0800eaa0 tx=0x7f3f0800ef60 comp rx=0 tx=0).stop 2024-09-19T09:00:25.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.553+0000 7f3f160fa640 1 -- 172.21.15.120:0/343675916 shutdown_connections 2024-09-19T09:00:25.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.553+0000 7f3f160fa640 1 --2- 172.21.15.120:0/343675916 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3efc0a3240 0x7f3efc0a5700 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:25.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.553+0000 7f3f160fa640 1 --2- 172.21.15.120:0/343675916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f10106f10 0x7f3f1019cda0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:25.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.553+0000 7f3f160fa640 1 --2- 172.21.15.120:0/343675916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f10102790 0x7f3f10198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:25.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.553+0000 7f3f160fa640 1 --2- 172.21.15.120:0/343675916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f10101de0 0x7f3f101984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:25.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.553+0000 7f3f160fa640 1 -- 172.21.15.120:0/343675916 >> 172.21.15.120:0/343675916 conn(0x7f3f100fd8c0 msgr2=0x7f3f100fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:25.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.553+0000 7f3f160fa640 1 -- 172.21.15.120:0/343675916 shutdown_connections 2024-09-19T09:00:25.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:25.553+0000 7f3f160fa640 1 -- 172.21.15.120:0/343675916 wait complete. 2024-09-19T09:00:25.809 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:25 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/343675916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:00:25.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:25 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/343675916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:00:25.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:25 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/343675916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:00:26.187 INFO:journalctl@ceph.mds.h.smithi153.stdout:Sep 19 09:00:26 smithi153 systemd[1]: Started Ceph mds.h for 507393d2-7664-11ef-bceb-c7b262605968. 2024-09-19T09:00:26.785 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:26 smithi196 ceph-mon[35150]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:26.785 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:26 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:26.785 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:26 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:26.785 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:26 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:26.785 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:26 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:26.785 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:26 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:26.785 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:26 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:26.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:26 smithi153 ceph-mon[35071]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:26.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:26 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:26.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:26 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:26.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:26 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:26.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:26 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:26.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:26 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:26.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:26 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:27.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:26 smithi120 ceph-mon[31150]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:27.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:26 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:27.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:26 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:27.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:26 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:27.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:26 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:27.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:26 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:27.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:26 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:27.782 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:27 smithi196 ceph-mon[35150]: Deploying daemon mds.i on smithi196 2024-09-19T09:00:27.782 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:27 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] up:boot 2024-09-19T09:00:27.782 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:27 smithi196 ceph-mon[35150]: fsmap 8 up:standby 2024-09-19T09:00:27.782 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:27 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T09:00:27.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:27 smithi153 ceph-mon[35071]: Deploying daemon mds.i on smithi196 2024-09-19T09:00:27.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:27 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] up:boot 2024-09-19T09:00:27.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:27 smithi153 ceph-mon[35071]: fsmap 8 up:standby 2024-09-19T09:00:27.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:27 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T09:00:28.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:27 smithi120 ceph-mon[31150]: Deploying daemon mds.i on smithi196 2024-09-19T09:00:28.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:27 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] up:boot 2024-09-19T09:00:28.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:27 smithi120 ceph-mon[31150]: fsmap 8 up:standby 2024-09-19T09:00:28.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:27 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T09:00:28.186 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 09:00:27 smithi196 systemd[1]: Starting Ceph mds.i for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:00:28.234 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:00:28.444 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 09:00:28 smithi196 podman[65310]: 2024-09-19 09:00:28.096099525 +0000 UTC m=+0.015761078 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:00:28.444 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 09:00:28 smithi196 podman[65310]: 2024-09-19 09:00:28.209342686 +0000 UTC m=+0.129004226 container create 92cd60c0b5c868fa3fe800895ca39f073b3a1105e2f492de73487fda2bd1e569 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-i, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD) 2024-09-19T09:00:28.491 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:00:28.928 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:28 smithi120 ceph-mon[31150]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:28.928 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:28 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:28.928 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:28 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:28.928 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:28 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:28.928 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:28 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:28.929 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:28 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:28.929 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:28 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:28.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:28 smithi196 ceph-mon[35150]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:28.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:28 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:28.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:28 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:28.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:28 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:28.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:28 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:28.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:28 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:28.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:28 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:28.936 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 09:00:28 smithi196 podman[65310]: 2024-09-19 09:00:28.445281651 +0000 UTC m=+0.364943189 container init 92cd60c0b5c868fa3fe800895ca39f073b3a1105e2f492de73487fda2bd1e569 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-2f599a4, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-19T09:00:28.936 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 09:00:28 smithi196 podman[65310]: 2024-09-19 09:00:28.448296555 +0000 UTC m=+0.367958094 container start 92cd60c0b5c868fa3fe800895ca39f073b3a1105e2f492de73487fda2bd1e569 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-09-19T09:00:28.936 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 09:00:28 smithi196 ceph-mds[65329]: -- 172.21.15.196:0/3939880741 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d471287860 con 0x55d471264800 2024-09-19T09:00:28.936 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 09:00:28 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-i[65325]: starting mds.i at 2024-09-19T09:00:28.936 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 09:00:28 smithi196 bash[65310]: 92cd60c0b5c868fa3fe800895ca39f073b3a1105e2f492de73487fda2bd1e569 2024-09-19T09:00:28.936 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 09:00:28 smithi196 systemd[1]: Started Ceph mds.i for 507393d2-7664-11ef-bceb-c7b262605968. 2024-09-19T09:00:28.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:28 smithi153 ceph-mon[35071]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:28.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:28 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:28.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:28 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:28.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:28 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:28.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:28 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:28.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:28 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:28.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:28 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:29.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.006+0000 7f8dcec3d640 1 -- 172.21.15.120:0/304116671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8dc81027a0 msgr2=0x7f8dc81067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:29.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.006+0000 7f8dcec3d640 1 --2- 172.21.15.120:0/304116671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8dc81027a0 0x7f8dc81067f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc009900 tx=0x7f8dbc02a410 comp rx=0 tx=0).stop 2024-09-19T09:00:29.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.006+0000 7f8dcec3d640 1 -- 172.21.15.120:0/304116671 shutdown_connections 2024-09-19T09:00:29.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.006+0000 7f8dcec3d640 1 --2- 172.21.15.120:0/304116671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8dc8106f20 0x7f8dc810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:29.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.006+0000 7f8dcec3d640 1 --2- 172.21.15.120:0/304116671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8dc81027a0 0x7f8dc81067f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:29.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.006+0000 7f8dcec3d640 1 --2- 172.21.15.120:0/304116671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dc8101df0 0x7f8dc81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:29.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.006+0000 7f8dcec3d640 1 -- 172.21.15.120:0/304116671 >> 172.21.15.120:0/304116671 conn(0x7f8dc80fd8b0 msgr2=0x7f8dc80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:29.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.006+0000 7f8dcec3d640 1 -- 172.21.15.120:0/304116671 shutdown_connections 2024-09-19T09:00:29.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.006+0000 7f8dcec3d640 1 -- 172.21.15.120:0/304116671 wait complete. 2024-09-19T09:00:29.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.006+0000 7f8dcec3d640 1 Processor -- start 2024-09-19T09:00:29.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.006+0000 7f8dcec3d640 1 -- start start 2024-09-19T09:00:29.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.006+0000 7f8dcec3d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dc8101df0 0x7f8dc819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:29.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.006+0000 7f8dcec3d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8dc81027a0 0x7f8dc819ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:29.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.006+0000 7f8dcec3d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8dc8106f20 0x7f8dc819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:29.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.006+0000 7f8dcec3d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc8111fd0 con 0x7f8dc8106f20 2024-09-19T09:00:29.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.006+0000 7f8dcec3d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc8111e50 con 0x7f8dc81027a0 2024-09-19T09:00:29.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.006+0000 7f8dcec3d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc8112150 con 0x7f8dc8101df0 2024-09-19T09:00:29.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.007+0000 7f8dcd1da640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8dc8106f20 0x7f8dc819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:29.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.007+0000 7f8dcd1da640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8dc8106f20 0x7f8dc819efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49954/0 (socket says 172.21.15.120:49954) 2024-09-19T09:00:29.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.007+0000 7f8dcd1da640 1 -- 172.21.15.120:0/1427271917 learned_addr learned my addr 172.21.15.120:0/1427271917 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:00:29.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.007+0000 7f8dcc9d9640 1 --2- 172.21.15.120:0/1427271917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dc8101df0 0x7f8dc819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:29.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.007+0000 7f8dc7fff640 1 --2- 172.21.15.120:0/1427271917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8dc81027a0 0x7f8dc819ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:29.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.007+0000 7f8dcd1da640 1 -- 172.21.15.120:0/1427271917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dc8101df0 msgr2=0x7f8dc819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:29.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.007+0000 7f8dcd1da640 1 --2- 172.21.15.120:0/1427271917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dc8101df0 0x7f8dc819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:29.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.007+0000 7f8dcd1da640 1 -- 172.21.15.120:0/1427271917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8dc81027a0 msgr2=0x7f8dc819ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:29.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.007+0000 7f8dcd1da640 1 --2- 172.21.15.120:0/1427271917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8dc81027a0 0x7f8dc819ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:29.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.007+0000 7f8dcd1da640 1 -- 172.21.15.120:0/1427271917 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc819f6a0 con 0x7f8dc8106f20 2024-09-19T09:00:29.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.007+0000 7f8dcd1da640 1 --2- 172.21.15.120:0/1427271917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8dc8106f20 0x7f8dc819efc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000cce0 tx=0x7f8dc000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:29.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.007+0000 7f8dc5ffb640 1 -- 172.21.15.120:0/1427271917 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc000f450 con 0x7f8dc8106f20 2024-09-19T09:00:29.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.007+0000 7f8dc5ffb640 1 -- 172.21.15.120:0/1427271917 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8dc000fad0 con 0x7f8dc8106f20 2024-09-19T09:00:29.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.007+0000 7f8dcec3d640 1 -- 172.21.15.120:0/1427271917 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dc819f990 con 0x7f8dc8106f20 2024-09-19T09:00:29.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.007+0000 7f8dc5ffb640 1 -- 172.21.15.120:0/1427271917 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0018c10 con 0x7f8dc8106f20 2024-09-19T09:00:29.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.007+0000 7f8dcec3d640 1 -- 172.21.15.120:0/1427271917 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dc81a7270 con 0x7f8dc8106f20 2024-09-19T09:00:29.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.009+0000 7f8dcec3d640 1 -- 172.21.15.120:0/1427271917 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dc81021d0 con 0x7f8dc8106f20 2024-09-19T09:00:29.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.009+0000 7f8dc5ffb640 1 -- 172.21.15.120:0/1427271917 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f8dc000f5f0 con 0x7f8dc8106f20 2024-09-19T09:00:29.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.010+0000 7f8dc5ffb640 1 --2- 172.21.15.120:0/1427271917 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8db00a32c0 0x7f8db00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:29.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.010+0000 7f8dcc9d9640 1 --2- 172.21.15.120:0/1427271917 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8db00a32c0 0x7f8db00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:29.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.010+0000 7f8dc5ffb640 1 -- 172.21.15.120:0/1427271917 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f8dc00cfa40 con 0x7f8dc8106f20 2024-09-19T09:00:29.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.011+0000 7f8dcc9d9640 1 --2- 172.21.15.120:0/1427271917 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8db00a32c0 0x7f8db00a5780 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8db8005fd0 tx=0x7f8db800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:29.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.015+0000 7f8dc5ffb640 1 -- 172.21.15.120:0/1427271917 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8dc0014030 con 0x7f8dc8106f20 2024-09-19T09:00:29.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.225+0000 7f8dcec3d640 1 -- 172.21.15.120:0/1427271917 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8dc81a7440 con 0x7f8dc8106f20 2024-09-19T09:00:29.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.226+0000 7f8dc5ffb640 1 -- 172.21.15.120:0/1427271917 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8055 (secure 0 0 0) 0x7f8dc009cc00 con 0x7f8dc8106f20 2024-09-19T09:00:29.227 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:00:29.227 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":10,"btime":"2024-09-19T09:00:28:614236+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24442,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-19T09:00:29.227 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 10 2024-09-19T09:00:29.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.228+0000 7f8dcec3d640 1 -- 172.21.15.120:0/1427271917 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8db00a32c0 msgr2=0x7f8db00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:29.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.228+0000 7f8dcec3d640 1 --2- 172.21.15.120:0/1427271917 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8db00a32c0 0x7f8db00a5780 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8db8005fd0 tx=0x7f8db800b390 comp rx=0 tx=0).stop 2024-09-19T09:00:29.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.228+0000 7f8dcec3d640 1 -- 172.21.15.120:0/1427271917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8dc8106f20 msgr2=0x7f8dc819efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:29.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.228+0000 7f8dcec3d640 1 --2- 172.21.15.120:0/1427271917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8dc8106f20 0x7f8dc819efc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000cce0 tx=0x7f8dc000a620 comp rx=0 tx=0).stop 2024-09-19T09:00:29.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.228+0000 7f8dcec3d640 1 -- 172.21.15.120:0/1427271917 shutdown_connections 2024-09-19T09:00:29.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.228+0000 7f8dcec3d640 1 --2- 172.21.15.120:0/1427271917 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8db00a32c0 0x7f8db00a5780 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:29.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.228+0000 7f8dcec3d640 1 --2- 172.21.15.120:0/1427271917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8dc8106f20 0x7f8dc819efc0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:29.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.228+0000 7f8dcec3d640 1 --2- 172.21.15.120:0/1427271917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8dc81027a0 0x7f8dc819ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:29.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.228+0000 7f8dcec3d640 1 --2- 172.21.15.120:0/1427271917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dc8101df0 0x7f8dc819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:29.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.228+0000 7f8dcec3d640 1 -- 172.21.15.120:0/1427271917 >> 172.21.15.120:0/1427271917 conn(0x7f8dc80fd8b0 msgr2=0x7f8dc80fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:29.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.228+0000 7f8dcec3d640 1 -- 172.21.15.120:0/1427271917 shutdown_connections 2024-09-19T09:00:29.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:29.228+0000 7f8dcec3d640 1 -- 172.21.15.120:0/1427271917 wait complete. 2024-09-19T09:00:29.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:29 smithi120 ceph-mon[31150]: Deploying daemon mds.g on smithi120 2024-09-19T09:00:29.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:29 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] up:boot 2024-09-19T09:00:29.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:29 smithi120 ceph-mon[31150]: fsmap 9 up:standby 2024-09-19T09:00:29.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:29 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T09:00:29.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:29 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1427271917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:00:29.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:29 smithi196 ceph-mon[35150]: Deploying daemon mds.g on smithi120 2024-09-19T09:00:29.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:29 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] up:boot 2024-09-19T09:00:29.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:29 smithi196 ceph-mon[35150]: fsmap 9 up:standby 2024-09-19T09:00:29.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:29 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T09:00:29.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:29 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1427271917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:00:29.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:29 smithi153 ceph-mon[35071]: Deploying daemon mds.g on smithi120 2024-09-19T09:00:29.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:29 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] up:boot 2024-09-19T09:00:29.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:29 smithi153 ceph-mon[35071]: fsmap 9 up:standby 2024-09-19T09:00:29.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:29 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T09:00:29.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:29 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1427271917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:00:30.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:30 smithi196 ceph-mon[35150]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:30.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:30 smithi153 ceph-mon[35071]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:31.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:30 smithi120 ceph-mon[31150]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:31.818 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 19 09:00:31 smithi120 systemd[1]: Starting Ceph mds.g for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:00:32.074 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 19 09:00:31 smithi120 podman[71828]: 2024-09-19 09:00:31.720897614 +0000 UTC m=+0.027434463 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:00:32.075 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 19 09:00:31 smithi120 podman[71828]: 2024-09-19 09:00:31.846898091 +0000 UTC m=+0.153434930 container create 19915489a4a6142cc133389ea6329cc8a10cf9055313ecb1379692a65357af5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-g, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-19T09:00:32.574 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 19 09:00:32 smithi120 podman[71828]: 2024-09-19 09:00:32.1000095 +0000 UTC m=+0.406546340 container init 19915489a4a6142cc133389ea6329cc8a10cf9055313ecb1379692a65357af5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:00:32.575 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 19 09:00:32 smithi120 podman[71828]: 2024-09-19 09:00:32.102914012 +0000 UTC m=+0.409450854 container start 19915489a4a6142cc133389ea6329cc8a10cf9055313ecb1379692a65357af5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-g, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-19T09:00:32.575 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 19 09:00:32 smithi120 ceph-mds[71848]: -- 172.21.15.120:0/2115361832 <== mon.0 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55672f955860 con 0x55672f933400 2024-09-19T09:00:32.575 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 19 09:00:32 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-g[71844]: starting mds.g at 2024-09-19T09:00:32.575 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 19 09:00:32 smithi120 bash[71828]: 19915489a4a6142cc133389ea6329cc8a10cf9055313ecb1379692a65357af5e 2024-09-19T09:00:32.575 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 19 09:00:32 smithi120 systemd[1]: Started Ceph mds.g for 507393d2-7664-11ef-bceb-c7b262605968. 2024-09-19T09:00:32.654 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:00:32.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:32 smithi196 ceph-mon[35150]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:32.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:32 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:32.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:32 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:32.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:32 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:32.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:32 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:32.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:32 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:32.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:32 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:32.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:32 smithi153 ceph-mon[35071]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:32.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:32 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:32.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:32 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:32.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:32 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:32.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:32 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:32.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:32 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:32.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:32 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:32.938 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:32 smithi120 ceph-mon[31150]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:32.938 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:32 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:32.938 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:32 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:32.938 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:32 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:32.938 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:32 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:32.938 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:32 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:32.938 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:32 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:33.751 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:33 smithi120 ceph-mon[31150]: Deploying daemon mds.a on smithi120 2024-09-19T09:00:33.751 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:33 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.120:6838/1939272010,v1:172.21.15.120:6839/1939272010] up:boot 2024-09-19T09:00:33.751 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:33 smithi120 ceph-mon[31150]: fsmap 10 up:standby 2024-09-19T09:00:33.751 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:33 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T09:00:33.787 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:00:33.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:33 smithi196 ceph-mon[35150]: Deploying daemon mds.a on smithi120 2024-09-19T09:00:33.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:33 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.120:6838/1939272010,v1:172.21.15.120:6839/1939272010] up:boot 2024-09-19T09:00:33.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:33 smithi196 ceph-mon[35150]: fsmap 10 up:standby 2024-09-19T09:00:33.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:33 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T09:00:33.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:33 smithi153 ceph-mon[35071]: Deploying daemon mds.a on smithi120 2024-09-19T09:00:33.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:33 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.120:6838/1939272010,v1:172.21.15.120:6839/1939272010] up:boot 2024-09-19T09:00:33.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:33 smithi153 ceph-mon[35071]: fsmap 10 up:standby 2024-09-19T09:00:33.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:33 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T09:00:34.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.448+0000 7f3351f31640 1 -- 172.21.15.120:0/2004821653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f334c102790 msgr2=0x7f334c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:34.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.448+0000 7f3351f31640 1 --2- 172.21.15.120:0/2004821653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f334c102790 0x7f334c1067e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3340009900 tx=0x7f334002a410 comp rx=0 tx=0).stop 2024-09-19T09:00:34.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.448+0000 7f3351f31640 1 -- 172.21.15.120:0/2004821653 shutdown_connections 2024-09-19T09:00:34.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.448+0000 7f3351f31640 1 --2- 172.21.15.120:0/2004821653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f334c106f10 0x7f334c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:34.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.448+0000 7f3351f31640 1 --2- 172.21.15.120:0/2004821653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f334c102790 0x7f334c1067e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:34.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.448+0000 7f3351f31640 1 --2- 172.21.15.120:0/2004821653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f334c101de0 0x7f334c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:34.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.448+0000 7f3351f31640 1 -- 172.21.15.120:0/2004821653 >> 172.21.15.120:0/2004821653 conn(0x7f334c0fd8c0 msgr2=0x7f334c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:34.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.448+0000 7f3351f31640 1 -- 172.21.15.120:0/2004821653 shutdown_connections 2024-09-19T09:00:34.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.448+0000 7f3351f31640 1 -- 172.21.15.120:0/2004821653 wait complete. 2024-09-19T09:00:34.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.449+0000 7f3351f31640 1 Processor -- start 2024-09-19T09:00:34.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.449+0000 7f3351f31640 1 -- start start 2024-09-19T09:00:34.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.449+0000 7f3351f31640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f334c101de0 0x7f334c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:34.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.449+0000 7f3351f31640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f334c102790 0x7f334c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:34.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.449+0000 7f3351f31640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f334c106f10 0x7f334c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:34.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.449+0000 7f3351f31640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f334c111f70 con 0x7f334c101de0 2024-09-19T09:00:34.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.449+0000 7f3351f31640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f334c111df0 con 0x7f334c102790 2024-09-19T09:00:34.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.449+0000 7f3351f31640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1120f0 con 0x7f334c106f10 2024-09-19T09:00:34.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.449+0000 7f334b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f334c101de0 0x7f334c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:34.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.449+0000 7f334b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f334c101de0 0x7f334c198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49818/0 (socket says 172.21.15.120:49818) 2024-09-19T09:00:34.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.449+0000 7f334b7fe640 1 -- 172.21.15.120:0/3323088054 learned_addr learned my addr 172.21.15.120:0/3323088054 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:00:34.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.449+0000 7f334affd640 1 --2- 172.21.15.120:0/3323088054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f334c102790 0x7f334c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:34.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.450+0000 7f334bfff640 1 --2- 172.21.15.120:0/3323088054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f334c106f10 0x7f334c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:34.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.450+0000 7f334b7fe640 1 -- 172.21.15.120:0/3323088054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f334c106f10 msgr2=0x7f334c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:34.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.450+0000 7f334b7fe640 1 --2- 172.21.15.120:0/3323088054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f334c106f10 0x7f334c19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:34.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.450+0000 7f334b7fe640 1 -- 172.21.15.120:0/3323088054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f334c102790 msgr2=0x7f334c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:34.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.450+0000 7f334b7fe640 1 --2- 172.21.15.120:0/3323088054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f334c102790 0x7f334c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:34.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.450+0000 7f334b7fe640 1 -- 172.21.15.120:0/3323088054 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f334c19d440 con 0x7f334c101de0 2024-09-19T09:00:34.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.450+0000 7f334affd640 1 --2- 172.21.15.120:0/3323088054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f334c102790 0x7f334c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:00:34.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.450+0000 7f334b7fe640 1 --2- 172.21.15.120:0/3323088054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f334c101de0 0x7f334c198490 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f333c00c890 tx=0x7f333c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:34.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.450+0000 7f3348ff9640 1 -- 172.21.15.120:0/3323088054 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c004220 con 0x7f334c101de0 2024-09-19T09:00:34.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.450+0000 7f3351f31640 1 -- 172.21.15.120:0/3323088054 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f334c19d730 con 0x7f334c101de0 2024-09-19T09:00:34.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.450+0000 7f3348ff9640 1 -- 172.21.15.120:0/3323088054 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f333c0043c0 con 0x7f334c101de0 2024-09-19T09:00:34.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.450+0000 7f3348ff9640 1 -- 172.21.15.120:0/3323088054 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c004a00 con 0x7f334c101de0 2024-09-19T09:00:34.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.450+0000 7f3351f31640 1 -- 172.21.15.120:0/3323088054 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f334c1a5010 con 0x7f334c101de0 2024-09-19T09:00:34.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.451+0000 7f3351f31640 1 -- 172.21.15.120:0/3323088054 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f334c1021c0 con 0x7f334c101de0 2024-09-19T09:00:34.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.452+0000 7f3348ff9640 1 -- 172.21.15.120:0/3323088054 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f333c0079b0 con 0x7f334c101de0 2024-09-19T09:00:34.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.453+0000 7f3348ff9640 1 --2- 172.21.15.120:0/3323088054 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33340a3240 0x7f33340a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:34.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.453+0000 7f334affd640 1 --2- 172.21.15.120:0/3323088054 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33340a3240 0x7f33340a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:34.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.453+0000 7f3348ff9640 1 -- 172.21.15.120:0/3323088054 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f333c0cf220 con 0x7f334c101de0 2024-09-19T09:00:34.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.453+0000 7f334affd640 1 --2- 172.21.15.120:0/3323088054 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33340a3240 0x7f33340a5700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f334c1087d0 tx=0x7f3340005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:34.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.457+0000 7f3348ff9640 1 -- 172.21.15.120:0/3323088054 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f333c014030 con 0x7f334c101de0 2024-09-19T09:00:34.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.673+0000 7f3351f31640 1 -- 172.21.15.120:0/3323088054 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f334c1997f0 con 0x7f334c101de0 2024-09-19T09:00:34.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.673+0000 7f3348ff9640 1 -- 172.21.15.120:0/3323088054 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8884 (secure 0 0 0) 0x7f333c09c3e0 con 0x7f334c101de0 2024-09-19T09:00:34.674 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:00:34.674 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":11,"btime":"2024-09-19T09:00:32:630284+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24442,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-19T09:00:34.674 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 11 2024-09-19T09:00:34.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.675+0000 7f3351f31640 1 -- 172.21.15.120:0/3323088054 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33340a3240 msgr2=0x7f33340a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:34.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.675+0000 7f3351f31640 1 --2- 172.21.15.120:0/3323088054 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33340a3240 0x7f33340a5700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f334c1087d0 tx=0x7f3340005a20 comp rx=0 tx=0).stop 2024-09-19T09:00:34.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.675+0000 7f3351f31640 1 -- 172.21.15.120:0/3323088054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f334c101de0 msgr2=0x7f334c198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:34.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.675+0000 7f3351f31640 1 --2- 172.21.15.120:0/3323088054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f334c101de0 0x7f334c198490 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f333c00c890 tx=0x7f333c00cd50 comp rx=0 tx=0).stop 2024-09-19T09:00:34.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.675+0000 7f3351f31640 1 -- 172.21.15.120:0/3323088054 shutdown_connections 2024-09-19T09:00:34.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.675+0000 7f3351f31640 1 --2- 172.21.15.120:0/3323088054 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33340a3240 0x7f33340a5700 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:34.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.675+0000 7f3351f31640 1 --2- 172.21.15.120:0/3323088054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f334c106f10 0x7f334c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:34.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.675+0000 7f3351f31640 1 --2- 172.21.15.120:0/3323088054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f334c102790 0x7f334c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:34.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.675+0000 7f3351f31640 1 --2- 172.21.15.120:0/3323088054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f334c101de0 0x7f334c198490 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:34.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.675+0000 7f3351f31640 1 -- 172.21.15.120:0/3323088054 >> 172.21.15.120:0/3323088054 conn(0x7f334c0fd8c0 msgr2=0x7f334c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:34.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.676+0000 7f3351f31640 1 -- 172.21.15.120:0/3323088054 shutdown_connections 2024-09-19T09:00:34.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:34.676+0000 7f3351f31640 1 -- 172.21.15.120:0/3323088054 wait complete. 2024-09-19T09:00:34.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:34 smithi196 ceph-mon[35150]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:34.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:34 smithi153 ceph-mon[35071]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:35.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:34 smithi120 ceph-mon[31150]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:35.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:35 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3323088054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:00:35.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:35 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:35.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:35 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:35.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:35 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:35.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:35 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:35.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:35 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:35.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:35 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:35.928 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:35 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3323088054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:00:35.928 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:35 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:35.928 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:35 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:35.928 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:35 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:35.928 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:35 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:35.928 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:35 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:35.928 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:35 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:35.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:35 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3323088054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:00:35.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:35 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:35.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:35 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:35.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:35 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:35.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:35 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:00:35.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:35 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:00:35.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:35 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:36.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:36 smithi196 ceph-mon[35150]: Deploying daemon mds.b on smithi153 2024-09-19T09:00:36.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:36 smithi196 ceph-mon[35150]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:36.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:36 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.120:6840/658447478,v1:172.21.15.120:6841/658447478] up:boot 2024-09-19T09:00:36.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:36 smithi196 ceph-mon[35150]: fsmap 11 up:standby 2024-09-19T09:00:36.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:36 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T09:00:36.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:36 smithi153 ceph-mon[35071]: Deploying daemon mds.b on smithi153 2024-09-19T09:00:36.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:36 smithi153 ceph-mon[35071]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:36.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:36 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.120:6840/658447478,v1:172.21.15.120:6841/658447478] up:boot 2024-09-19T09:00:36.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:36 smithi153 ceph-mon[35071]: fsmap 11 up:standby 2024-09-19T09:00:36.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:36 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T09:00:37.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:36 smithi120 ceph-mon[31150]: Deploying daemon mds.b on smithi153 2024-09-19T09:00:37.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:36 smithi120 ceph-mon[31150]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:37.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:36 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.120:6840/658447478,v1:172.21.15.120:6841/658447478] up:boot 2024-09-19T09:00:37.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:36 smithi120 ceph-mon[31150]: fsmap 11 up:standby 2024-09-19T09:00:37.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:36 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T09:00:37.666 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 19 09:00:37 smithi153 systemd[1]: Starting Ceph mds.b for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:00:37.897 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:00:37.929 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 19 09:00:37 smithi153 podman[64509]: 2024-09-19 09:00:37.569157215 +0000 UTC m=+0.015811688 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:00:37.929 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 19 09:00:37 smithi153 podman[64509]: 2024-09-19 09:00:37.684668004 +0000 UTC m=+0.131322479 container create 3c564dedccd8d27d314462e23175a2e06862c4f6b8359e3b397118358dc7e15b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-b, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-19T09:00:37.929 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 19 09:00:37 smithi153 podman[64509]: 2024-09-19 09:00:37.903882103 +0000 UTC m=+0.350536576 container init 3c564dedccd8d27d314462e23175a2e06862c4f6b8359e3b397118358dc7e15b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-b, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS) 2024-09-19T09:00:37.929 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 19 09:00:37 smithi153 podman[64509]: 2024-09-19 09:00:37.906846879 +0000 UTC m=+0.353501360 container start 3c564dedccd8d27d314462e23175a2e06862c4f6b8359e3b397118358dc7e15b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-b, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-09-19T09:00:38.167 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:00:38.187 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 19 09:00:37 smithi153 ceph-mds[64528]: -- 172.21.15.153:0/1294117678 <== mon.2 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560de7ffb860 con 0x560de7fd9400 2024-09-19T09:00:38.187 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 19 09:00:37 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-b[64524]: starting mds.b at 2024-09-19T09:00:38.187 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 19 09:00:38 smithi153 bash[64509]: 3c564dedccd8d27d314462e23175a2e06862c4f6b8359e3b397118358dc7e15b 2024-09-19T09:00:38.187 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 19 09:00:38 smithi153 systemd[1]: Started Ceph mds.b for 507393d2-7664-11ef-bceb-c7b262605968. 2024-09-19T09:00:38.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.697+0000 7f53b0bc3640 1 -- 172.21.15.120:0/1329248971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53ac101420 msgr2=0x7f53ac10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:38.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.697+0000 7f53b0bc3640 1 --2- 172.21.15.120:0/1329248971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53ac101420 0x7f53ac10d950 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f53a4009930 tx=0x7f53a402a420 comp rx=0 tx=0).stop 2024-09-19T09:00:38.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.697+0000 7f53b0bc3640 1 -- 172.21.15.120:0/1329248971 shutdown_connections 2024-09-19T09:00:38.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.697+0000 7f53b0bc3640 1 --2- 172.21.15.120:0/1329248971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53ac101420 0x7f53ac10d950 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:38.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.697+0000 7f53b0bc3640 1 --2- 172.21.15.120:0/1329248971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53ac100a80 0x7f53ac100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:38.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.697+0000 7f53b0bc3640 1 --2- 172.21.15.120:0/1329248971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53ac106a80 0x7f53ac106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:38.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.697+0000 7f53b0bc3640 1 -- 172.21.15.120:0/1329248971 >> 172.21.15.120:0/1329248971 conn(0x7f53ac0fc7a0 msgr2=0x7f53ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:38.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.697+0000 7f53b0bc3640 1 -- 172.21.15.120:0/1329248971 shutdown_connections 2024-09-19T09:00:38.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.697+0000 7f53b0bc3640 1 -- 172.21.15.120:0/1329248971 wait complete. 2024-09-19T09:00:38.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.697+0000 7f53b0bc3640 1 Processor -- start 2024-09-19T09:00:38.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.697+0000 7f53b0bc3640 1 -- start start 2024-09-19T09:00:38.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.698+0000 7f53b0bc3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53ac100a80 0x7f53ac19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:38.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.698+0000 7f53b0bc3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53ac101420 0x7f53ac19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:38.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.698+0000 7f53b0bc3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53ac106a80 0x7f53ac19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:38.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.698+0000 7f53b0bc3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac111fa0 con 0x7f53ac100a80 2024-09-19T09:00:38.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.698+0000 7f53b0bc3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac111e20 con 0x7f53ac106a80 2024-09-19T09:00:38.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.698+0000 7f53b0bc3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac112120 con 0x7f53ac101420 2024-09-19T09:00:38.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.698+0000 7f53aa59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53ac100a80 0x7f53ac19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:38.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.698+0000 7f53aa59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53ac100a80 0x7f53ac19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49848/0 (socket says 172.21.15.120:49848) 2024-09-19T09:00:38.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.698+0000 7f53aa59c640 1 -- 172.21.15.120:0/1451251303 learned_addr learned my addr 172.21.15.120:0/1451251303 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:00:38.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.698+0000 7f53a9d9b640 1 --2- 172.21.15.120:0/1451251303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53ac101420 0x7f53ac19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:38.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.698+0000 7f53aad9d640 1 --2- 172.21.15.120:0/1451251303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53ac106a80 0x7f53ac19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:38.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.698+0000 7f53aa59c640 1 -- 172.21.15.120:0/1451251303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53ac101420 msgr2=0x7f53ac19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:38.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.698+0000 7f53aa59c640 1 --2- 172.21.15.120:0/1451251303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53ac101420 0x7f53ac19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:38.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.698+0000 7f53aa59c640 1 -- 172.21.15.120:0/1451251303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53ac106a80 msgr2=0x7f53ac19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:38.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.698+0000 7f53aa59c640 1 --2- 172.21.15.120:0/1451251303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53ac106a80 0x7f53ac19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:38.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.698+0000 7f53aa59c640 1 -- 172.21.15.120:0/1451251303 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53ac19f5d0 con 0x7f53ac100a80 2024-09-19T09:00:38.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.698+0000 7f53aa59c640 1 --2- 172.21.15.120:0/1451251303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53ac100a80 0x7f53ac19a620 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f539c00db80 tx=0x7f539c00b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:38.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.698+0000 7f539b7fe640 1 -- 172.21.15.120:0/1451251303 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c017070 con 0x7f53ac100a80 2024-09-19T09:00:38.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.698+0000 7f539b7fe640 1 -- 172.21.15.120:0/1451251303 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f539c003b70 con 0x7f53ac100a80 2024-09-19T09:00:38.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.698+0000 7f53b0bc3640 1 -- 172.21.15.120:0/1451251303 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53ac19f8c0 con 0x7f53ac100a80 2024-09-19T09:00:38.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.698+0000 7f539b7fe640 1 -- 172.21.15.120:0/1451251303 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c0106c0 con 0x7f53ac100a80 2024-09-19T09:00:38.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.699+0000 7f53b0bc3640 1 -- 172.21.15.120:0/1451251303 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53ac1a71a0 con 0x7f53ac100a80 2024-09-19T09:00:38.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.700+0000 7f53b0bc3640 1 -- 172.21.15.120:0/1451251303 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53ac100ee0 con 0x7f53ac100a80 2024-09-19T09:00:38.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.701+0000 7f539b7fe640 1 -- 172.21.15.120:0/1451251303 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f539c004020 con 0x7f53ac100a80 2024-09-19T09:00:38.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.701+0000 7f539b7fe640 1 --2- 172.21.15.120:0/1451251303 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f53940a3240 0x7f53940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:38.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.701+0000 7f53a9d9b640 1 --2- 172.21.15.120:0/1451251303 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f53940a3240 0x7f53940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:38.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.701+0000 7f539b7fe640 1 -- 172.21.15.120:0/1451251303 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f539c0cf510 con 0x7f53ac100a80 2024-09-19T09:00:38.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.702+0000 7f53a9d9b640 1 --2- 172.21.15.120:0/1451251303 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f53940a3240 0x7f53940a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f53ac10d0d0 tx=0x7f53a0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:38.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.705+0000 7f539b7fe640 1 -- 172.21.15.120:0/1451251303 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f539c020070 con 0x7f53ac100a80 2024-09-19T09:00:38.911 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:38 smithi196 ceph-mon[35150]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:38.911 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:38 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:38.911 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:38 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:38.911 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:38 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:38.911 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:38 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:38.911 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:38 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:38.911 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:38 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:00:38.915 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:38 smithi153 ceph-mon[35071]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:38.915 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:38 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:38.915 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:38 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:38.915 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:38 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:38.915 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:38 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:38.915 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:38 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:38.916 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:38 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:00:38.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.918+0000 7f53b0bc3640 1 -- 172.21.15.120:0/1451251303 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53ac1a7440 con 0x7f53ac100a80 2024-09-19T09:00:38.918 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:38 smithi120 ceph-mon[31150]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:38.918 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:38 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:38.918 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:38 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:38.918 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:38 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:38.918 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:38 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:38.918 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:38 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:38.919 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:38 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:00:38.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.919+0000 7f539b7fe640 1 -- 172.21.15.120:0/1451251303 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10539 (secure 0 0 0) 0x7f539c09c630 con 0x7f53ac100a80 2024-09-19T09:00:38.919 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:00:38.919 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":13,"btime":"2024-09-19T09:00:38:651798+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24442,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24461,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-19T09:00:38.920 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 13 2024-09-19T09:00:38.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.921+0000 7f53b0bc3640 1 -- 172.21.15.120:0/1451251303 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f53940a3240 msgr2=0x7f53940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:38.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.921+0000 7f53b0bc3640 1 --2- 172.21.15.120:0/1451251303 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f53940a3240 0x7f53940a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f53ac10d0d0 tx=0x7f53a0005f90 comp rx=0 tx=0).stop 2024-09-19T09:00:38.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.921+0000 7f53b0bc3640 1 -- 172.21.15.120:0/1451251303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53ac100a80 msgr2=0x7f53ac19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:38.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.921+0000 7f53b0bc3640 1 --2- 172.21.15.120:0/1451251303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53ac100a80 0x7f53ac19a620 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f539c00db80 tx=0x7f539c00b630 comp rx=0 tx=0).stop 2024-09-19T09:00:38.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.921+0000 7f53b0bc3640 1 -- 172.21.15.120:0/1451251303 shutdown_connections 2024-09-19T09:00:38.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.921+0000 7f53b0bc3640 1 --2- 172.21.15.120:0/1451251303 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f53940a3240 0x7f53940a5700 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:38.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.921+0000 7f53b0bc3640 1 --2- 172.21.15.120:0/1451251303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53ac106a80 0x7f53ac19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:38.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.921+0000 7f53b0bc3640 1 --2- 172.21.15.120:0/1451251303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53ac101420 0x7f53ac19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:38.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.921+0000 7f53b0bc3640 1 --2- 172.21.15.120:0/1451251303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53ac100a80 0x7f53ac19a620 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:38.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.921+0000 7f53b0bc3640 1 -- 172.21.15.120:0/1451251303 >> 172.21.15.120:0/1451251303 conn(0x7f53ac0fc7a0 msgr2=0x7f53ac10c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:38.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.921+0000 7f53b0bc3640 1 -- 172.21.15.120:0/1451251303 shutdown_connections 2024-09-19T09:00:38.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:38.921+0000 7f53b0bc3640 1 -- 172.21.15.120:0/1451251303 wait complete. 2024-09-19T09:00:39.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:39 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] up:boot 2024-09-19T09:00:39.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:39 smithi120 ceph-mon[31150]: fsmap 12 up:standby 2024-09-19T09:00:39.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:39 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T09:00:39.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:39 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1451251303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:00:39.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:39 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:39.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:39 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:39.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:39 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] up:boot 2024-09-19T09:00:39.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:39 smithi196 ceph-mon[35150]: fsmap 12 up:standby 2024-09-19T09:00:39.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:39 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T09:00:39.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:39 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1451251303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:00:39.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:39 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:39.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:39 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:39.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:39 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] up:boot 2024-09-19T09:00:39.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:39 smithi153 ceph-mon[35071]: fsmap 12 up:standby 2024-09-19T09:00:39.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:39 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T09:00:39.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:39 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1451251303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:00:39.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:39 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:39.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:39 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:40.255 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-19T09:00:40.525 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:00:40.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:40 smithi196 ceph-mon[35150]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:40.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:40 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:40.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:40 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:40.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:40 smithi153 ceph-mon[35071]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:40.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:40 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:40.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:40 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:41.013 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:40 smithi120 ceph-mon[31150]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:41.013 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:40 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:41.014 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:40 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:41.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.093+0000 7efc1205d640 1 -- 172.21.15.120:0/2513667820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc0c102790 msgr2=0x7efc0c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:41.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.093+0000 7efc1205d640 1 --2- 172.21.15.120:0/2513667820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc0c102790 0x7efc0c1067e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009900 tx=0x7efbf402a410 comp rx=0 tx=0).stop 2024-09-19T09:00:41.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.093+0000 7efc1205d640 1 -- 172.21.15.120:0/2513667820 shutdown_connections 2024-09-19T09:00:41.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.093+0000 7efc1205d640 1 --2- 172.21.15.120:0/2513667820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc0c106f10 0x7efc0c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:41.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.093+0000 7efc1205d640 1 --2- 172.21.15.120:0/2513667820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc0c102790 0x7efc0c1067e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:41.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.093+0000 7efc1205d640 1 --2- 172.21.15.120:0/2513667820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc0c101de0 0x7efc0c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:41.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.093+0000 7efc1205d640 1 -- 172.21.15.120:0/2513667820 >> 172.21.15.120:0/2513667820 conn(0x7efc0c0fd880 msgr2=0x7efc0c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:41.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.093+0000 7efc1205d640 1 -- 172.21.15.120:0/2513667820 shutdown_connections 2024-09-19T09:00:41.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.093+0000 7efc1205d640 1 -- 172.21.15.120:0/2513667820 wait complete. 2024-09-19T09:00:41.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.093+0000 7efc1205d640 1 Processor -- start 2024-09-19T09:00:41.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.093+0000 7efc1205d640 1 -- start start 2024-09-19T09:00:41.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.094+0000 7efc1205d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc0c101de0 0x7efc0c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:41.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.094+0000 7efc1205d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc0c102790 0x7efc0c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:41.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.094+0000 7efc1205d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc0c106f10 0x7efc0c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:41.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.094+0000 7efc1205d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c111f30 con 0x7efc0c102790 2024-09-19T09:00:41.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.094+0000 7efc1205d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c111db0 con 0x7efc0c101de0 2024-09-19T09:00:41.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.094+0000 7efc1205d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c1120b0 con 0x7efc0c106f10 2024-09-19T09:00:41.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.094+0000 7efc02ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc0c102790 0x7efc0c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:41.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.094+0000 7efc02ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc0c102790 0x7efc0c198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49872/0 (socket says 172.21.15.120:49872) 2024-09-19T09:00:41.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.094+0000 7efc02ffd640 1 -- 172.21.15.120:0/1526277290 learned_addr learned my addr 172.21.15.120:0/1526277290 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:00:41.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.094+0000 7efc0b7fe640 1 --2- 172.21.15.120:0/1526277290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc0c101de0 0x7efc0c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:41.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.094+0000 7efc0bfff640 1 --2- 172.21.15.120:0/1526277290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc0c106f10 0x7efc0c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:41.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.094+0000 7efc02ffd640 1 -- 172.21.15.120:0/1526277290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc0c106f10 msgr2=0x7efc0c19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:41.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.094+0000 7efc02ffd640 1 --2- 172.21.15.120:0/1526277290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc0c106f10 0x7efc0c19cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:41.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.094+0000 7efc02ffd640 1 -- 172.21.15.120:0/1526277290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc0c101de0 msgr2=0x7efc0c198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:41.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.094+0000 7efc02ffd640 1 --2- 172.21.15.120:0/1526277290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc0c101de0 0x7efc0c198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:41.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.094+0000 7efc02ffd640 1 -- 172.21.15.120:0/1526277290 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0c19d300 con 0x7efc0c102790 2024-09-19T09:00:41.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.094+0000 7efc0bfff640 1 --2- 172.21.15.120:0/1526277290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc0c106f10 0x7efc0c19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:00:41.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.094+0000 7efc02ffd640 1 --2- 172.21.15.120:0/1526277290 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc0c102790 0x7efc0c198890 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efbf4005ab0 tx=0x7efbf40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:41.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.095+0000 7efc097fa640 1 -- 172.21.15.120:0/1526277290 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf403f030 con 0x7efc0c102790 2024-09-19T09:00:41.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.095+0000 7efc1205d640 1 -- 172.21.15.120:0/1526277290 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc0c19d590 con 0x7efc0c102790 2024-09-19T09:00:41.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.095+0000 7efc097fa640 1 -- 172.21.15.120:0/1526277290 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbf4003ce0 con 0x7efc0c102790 2024-09-19T09:00:41.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.095+0000 7efc097fa640 1 -- 172.21.15.120:0/1526277290 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4035070 con 0x7efc0c102790 2024-09-19T09:00:41.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.095+0000 7efc1205d640 1 -- 172.21.15.120:0/1526277290 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc0c1a4ed0 con 0x7efc0c102790 2024-09-19T09:00:41.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.096+0000 7efc1205d640 1 -- 172.21.15.120:0/1526277290 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc0c1032c0 con 0x7efc0c102790 2024-09-19T09:00:41.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.097+0000 7efc097fa640 1 -- 172.21.15.120:0/1526277290 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7efbf4042030 con 0x7efc0c102790 2024-09-19T09:00:41.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.098+0000 7efc097fa640 1 --2- 172.21.15.120:0/1526277290 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efbf80a3240 0x7efbf80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:41.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.098+0000 7efc0b7fe640 1 --2- 172.21.15.120:0/1526277290 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efbf80a3240 0x7efbf80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:41.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.098+0000 7efc097fa640 1 -- 172.21.15.120:0/1526277290 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7efbf40efdf0 con 0x7efc0c102790 2024-09-19T09:00:41.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.098+0000 7efc0b7fe640 1 --2- 172.21.15.120:0/1526277290 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efbf80a3240 0x7efbf80a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efbfc00abe0 tx=0x7efbfc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:41.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.102+0000 7efc097fa640 1 -- 172.21.15.120:0/1526277290 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efbf40bcfb0 con 0x7efc0c102790 2024-09-19T09:00:41.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.290+0000 7efc1205d640 1 -- 172.21.15.120:0/1526277290 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7efc0c10a0f0 con 0x7efc0c102790 2024-09-19T09:00:41.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.290+0000 7efc097fa640 1 -- 172.21.15.120:0/1526277290 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v74) ==== 84+0+65 (secure 0 0 0) 0x7efbf40bc8b0 con 0x7efc0c102790 2024-09-19T09:00:41.291 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:00:41.291 INFO:teuthology.orchestra.run.smithi120.stdout:[ 2024-09-19T09:00:41.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:00:41.291 INFO:teuthology.orchestra.run.smithi120.stdout: "poolnum": 1, 2024-09-19T09:00:41.291 INFO:teuthology.orchestra.run.smithi120.stdout: "poolname": ".mgr" 2024-09-19T09:00:41.291 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:00:41.291 INFO:teuthology.orchestra.run.smithi120.stdout:] 2024-09-19T09:00:41.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.293+0000 7efc1205d640 1 -- 172.21.15.120:0/1526277290 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efbf80a3240 msgr2=0x7efbf80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:41.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.293+0000 7efc1205d640 1 --2- 172.21.15.120:0/1526277290 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efbf80a3240 0x7efbf80a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efbfc00abe0 tx=0x7efbfc00a2d0 comp rx=0 tx=0).stop 2024-09-19T09:00:41.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.293+0000 7efc1205d640 1 -- 172.21.15.120:0/1526277290 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc0c102790 msgr2=0x7efc0c198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:41.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.293+0000 7efc1205d640 1 --2- 172.21.15.120:0/1526277290 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc0c102790 0x7efc0c198890 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efbf4005ab0 tx=0x7efbf40042f0 comp rx=0 tx=0).stop 2024-09-19T09:00:41.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.293+0000 7efc1205d640 1 -- 172.21.15.120:0/1526277290 shutdown_connections 2024-09-19T09:00:41.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.293+0000 7efc1205d640 1 --2- 172.21.15.120:0/1526277290 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efbf80a3240 0x7efbf80a5700 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:41.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.293+0000 7efc1205d640 1 --2- 172.21.15.120:0/1526277290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc0c106f10 0x7efc0c19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:41.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.293+0000 7efc1205d640 1 --2- 172.21.15.120:0/1526277290 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc0c102790 0x7efc0c198890 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:41.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.293+0000 7efc1205d640 1 --2- 172.21.15.120:0/1526277290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc0c101de0 0x7efc0c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:41.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.293+0000 7efc1205d640 1 -- 172.21.15.120:0/1526277290 >> 172.21.15.120:0/1526277290 conn(0x7efc0c0fd880 msgr2=0x7efc0c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:41.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.293+0000 7efc1205d640 1 -- 172.21.15.120:0/1526277290 shutdown_connections 2024-09-19T09:00:41.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:41.293+0000 7efc1205d640 1 -- 172.21.15.120:0/1526277290 wait complete. 2024-09-19T09:00:41.990 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T09:00:41.991 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T09:00:42.531 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:42 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:42.531 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:42 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:42.532 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:42 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:42.532 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:42 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:00:42.532 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:42 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:42.532 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:42 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1526277290' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T09:00:42.532 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:42 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:00:42.532 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:42 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:42.532 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:42 smithi120 ceph-mon[31150]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:42.573 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:42 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:42.573 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:42 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:42.573 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:42 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:42.574 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:42 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:00:42.574 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:42 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:42.574 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:42 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1526277290' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T09:00:42.574 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:42 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:00:42.574 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:42 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:42.574 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:42 smithi196 ceph-mon[35150]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:42.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:42 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:42.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:42 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:42.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:42 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:42.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:42 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:00:42.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:42 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:42.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:42 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1526277290' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T09:00:42.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:42 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:00:42.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:42 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:42.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:42 smithi153 ceph-mon[35071]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:42.817 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:00:43.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.337+0000 7ff9cfee1640 1 -- 172.21.15.120:0/814787970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9c8100e30 msgr2=0x7ff9c8104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:43.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.337+0000 7ff9cfee1640 1 --2- 172.21.15.120:0/814787970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9c8100e30 0x7ff9c8104e80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009900 tx=0x7ff9b802a410 comp rx=0 tx=0).stop 2024-09-19T09:00:43.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.337+0000 7ff9cfee1640 1 -- 172.21.15.120:0/814787970 shutdown_connections 2024-09-19T09:00:43.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.337+0000 7ff9cfee1640 1 --2- 172.21.15.120:0/814787970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c81055b0 0x7ff9c810d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:43.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.337+0000 7ff9cfee1640 1 --2- 172.21.15.120:0/814787970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9c8100e30 0x7ff9c8104e80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:43.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.337+0000 7ff9cfee1640 1 --2- 172.21.15.120:0/814787970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9c8100480 0x7ff9c8100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:43.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.337+0000 7ff9cfee1640 1 -- 172.21.15.120:0/814787970 >> 172.21.15.120:0/814787970 conn(0x7ff9c8077210 msgr2=0x7ff9c80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:43.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.337+0000 7ff9cfee1640 1 -- 172.21.15.120:0/814787970 shutdown_connections 2024-09-19T09:00:43.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.338+0000 7ff9cfee1640 1 -- 172.21.15.120:0/814787970 wait complete. 2024-09-19T09:00:43.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.338+0000 7ff9cfee1640 1 Processor -- start 2024-09-19T09:00:43.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.338+0000 7ff9cfee1640 1 -- start start 2024-09-19T09:00:43.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.338+0000 7ff9cfee1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9c8100480 0x7ff9c8198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:43.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.338+0000 7ff9cfee1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9c8100e30 0x7ff9c8198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:43.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.338+0000 7ff9cfee1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c81055b0 0x7ff9c819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:43.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.338+0000 7ff9cfee1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c8111f90 con 0x7ff9c8100480 2024-09-19T09:00:43.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.338+0000 7ff9cfee1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c8111e10 con 0x7ff9c8100e30 2024-09-19T09:00:43.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.338+0000 7ff9cfee1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c8112110 con 0x7ff9c81055b0 2024-09-19T09:00:43.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.338+0000 7ff9cdc7d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9c8100480 0x7ff9c8198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:43.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.338+0000 7ff9cdc7d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9c8100480 0x7ff9c8198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59890/0 (socket says 172.21.15.120:59890) 2024-09-19T09:00:43.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.338+0000 7ff9cdc7d640 1 -- 172.21.15.120:0/3896524173 learned_addr learned my addr 172.21.15.120:0/3896524173 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:00:43.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.339+0000 7ff9cd47c640 1 --2- 172.21.15.120:0/3896524173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9c8100e30 0x7ff9c8198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:43.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.339+0000 7ff9ce47e640 1 --2- 172.21.15.120:0/3896524173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c81055b0 0x7ff9c819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:43.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.339+0000 7ff9cdc7d640 1 -- 172.21.15.120:0/3896524173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c81055b0 msgr2=0x7ff9c819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:43.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.339+0000 7ff9cdc7d640 1 --2- 172.21.15.120:0/3896524173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c81055b0 0x7ff9c819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:43.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.339+0000 7ff9cdc7d640 1 -- 172.21.15.120:0/3896524173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9c8100e30 msgr2=0x7ff9c8198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:43.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.339+0000 7ff9cdc7d640 1 --2- 172.21.15.120:0/3896524173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9c8100e30 0x7ff9c8198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:43.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.339+0000 7ff9cdc7d640 1 -- 172.21.15.120:0/3896524173 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c819d400 con 0x7ff9c8100480 2024-09-19T09:00:43.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.339+0000 7ff9cd47c640 1 --2- 172.21.15.120:0/3896524173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9c8100e30 0x7ff9c8198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:00:43.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.339+0000 7ff9ce47e640 1 --2- 172.21.15.120:0/3896524173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c81055b0 0x7ff9c819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:00:43.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.339+0000 7ff9cdc7d640 1 --2- 172.21.15.120:0/3896524173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9c8100480 0x7ff9c8198450 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400c890 tx=0x7ff9c400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:43.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.339+0000 7ff9beffd640 1 -- 172.21.15.120:0/3896524173 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4004220 con 0x7ff9c8100480 2024-09-19T09:00:43.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.339+0000 7ff9cfee1640 1 -- 172.21.15.120:0/3896524173 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c819d6f0 con 0x7ff9c8100480 2024-09-19T09:00:43.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.339+0000 7ff9beffd640 1 -- 172.21.15.120:0/3896524173 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9c40043c0 con 0x7ff9c8100480 2024-09-19T09:00:43.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.339+0000 7ff9beffd640 1 -- 172.21.15.120:0/3896524173 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4004a00 con 0x7ff9c8100480 2024-09-19T09:00:43.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.339+0000 7ff9cfee1640 1 -- 172.21.15.120:0/3896524173 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c81a4fd0 con 0x7ff9c8100480 2024-09-19T09:00:43.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.340+0000 7ff9cfee1640 1 -- 172.21.15.120:0/3896524173 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9c8100860 con 0x7ff9c8100480 2024-09-19T09:00:43.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.341+0000 7ff9beffd640 1 -- 172.21.15.120:0/3896524173 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7ff9c40079b0 con 0x7ff9c8100480 2024-09-19T09:00:43.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.342+0000 7ff9beffd640 1 --2- 172.21.15.120:0/3896524173 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff9b40a3240 0x7ff9b40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:43.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.342+0000 7ff9cd47c640 1 --2- 172.21.15.120:0/3896524173 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff9b40a3240 0x7ff9b40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:43.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.342+0000 7ff9beffd640 1 -- 172.21.15.120:0/3896524173 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7ff9c40cf610 con 0x7ff9c8100480 2024-09-19T09:00:43.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.342+0000 7ff9cd47c640 1 --2- 172.21.15.120:0/3896524173 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff9b40a3240 0x7ff9b40a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8000c00 tx=0x7ff9b8032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:43.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.345+0000 7ff9beffd640 1 -- 172.21.15.120:0/3896524173 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff9c4014030 con 0x7ff9c8100480 2024-09-19T09:00:43.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.525+0000 7ff9cfee1640 1 -- 172.21.15.120:0/3896524173 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff9c802d7c0 con 0x7ff9c8100480 2024-09-19T09:00:43.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:43 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:43.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:43 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:43.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:43 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:43.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:43 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:43.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.929+0000 7ff9beffd640 1 -- 172.21.15.120:0/3896524173 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v75) ==== 138+0+0 (secure 0 0 0) 0x7ff9c409c7d0 con 0x7ff9c8100480 2024-09-19T09:00:43.929 INFO:teuthology.orchestra.run.smithi120.stderr:pool 'cephfs_metadata' created 2024-09-19T09:00:43.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.931+0000 7ff9cfee1640 1 -- 172.21.15.120:0/3896524173 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff9b40a3240 msgr2=0x7ff9b40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:43.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.931+0000 7ff9cfee1640 1 --2- 172.21.15.120:0/3896524173 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff9b40a3240 0x7ff9b40a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8000c00 tx=0x7ff9b8032d50 comp rx=0 tx=0).stop 2024-09-19T09:00:43.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.931+0000 7ff9cfee1640 1 -- 172.21.15.120:0/3896524173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9c8100480 msgr2=0x7ff9c8198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:43.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.931+0000 7ff9cfee1640 1 --2- 172.21.15.120:0/3896524173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9c8100480 0x7ff9c8198450 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400c890 tx=0x7ff9c400cd50 comp rx=0 tx=0).stop 2024-09-19T09:00:43.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.931+0000 7ff9cfee1640 1 -- 172.21.15.120:0/3896524173 shutdown_connections 2024-09-19T09:00:43.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.931+0000 7ff9cfee1640 1 --2- 172.21.15.120:0/3896524173 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff9b40a3240 0x7ff9b40a5700 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:43.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.931+0000 7ff9cfee1640 1 --2- 172.21.15.120:0/3896524173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c81055b0 0x7ff9c819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:43.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.931+0000 7ff9cfee1640 1 --2- 172.21.15.120:0/3896524173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9c8100e30 0x7ff9c8198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:43.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.931+0000 7ff9cfee1640 1 --2- 172.21.15.120:0/3896524173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9c8100480 0x7ff9c8198450 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:43.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.931+0000 7ff9cfee1640 1 -- 172.21.15.120:0/3896524173 >> 172.21.15.120:0/3896524173 conn(0x7ff9c8077210 msgr2=0x7ff9c8103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:43.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.931+0000 7ff9cfee1640 1 -- 172.21.15.120:0/3896524173 shutdown_connections 2024-09-19T09:00:43.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:43.931+0000 7ff9cfee1640 1 -- 172.21.15.120:0/3896524173 wait complete. 2024-09-19T09:00:44.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:43 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:44.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:43 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:44.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:43 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:44.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:43 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:44.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:43 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:44.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:43 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:44.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:43 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:44.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:43 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:44.639 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T09:00:44.886 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:00:45.007 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:44 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3896524173' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T09:00:45.007 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:44 smithi120 ceph-mon[31150]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:45.007 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:44 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:45.007 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:44 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:45.007 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:44 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:45.007 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:44 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:00:45.007 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:44 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:45.007 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:44 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3896524173' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T09:00:45.007 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:44 smithi120 ceph-mon[31150]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T09:00:45.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:44 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3896524173' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T09:00:45.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:44 smithi196 ceph-mon[35150]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:45.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:44 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:45.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:44 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:45.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:44 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:45.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:44 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:00:45.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:44 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:45.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:44 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3896524173' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T09:00:45.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:44 smithi196 ceph-mon[35150]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T09:00:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:44 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3896524173' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T09:00:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:44 smithi153 ceph-mon[35071]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:44 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:44 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:44 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:00:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:44 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:00:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:44 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:00:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:44 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3896524173' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T09:00:45.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:44 smithi153 ceph-mon[35071]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T09:00:45.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.374+0000 7fcac4fc7640 1 -- 172.21.15.120:0/2085528990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcac0106f20 msgr2=0x7fcac010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:45.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.374+0000 7fcac4fc7640 1 --2- 172.21.15.120:0/2085528990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcac0106f20 0x7fcac010eaa0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009ca0 tx=0x7fcab802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:00:45.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.374+0000 7fcac4fc7640 1 -- 172.21.15.120:0/2085528990 shutdown_connections 2024-09-19T09:00:45.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.374+0000 7fcac4fc7640 1 --2- 172.21.15.120:0/2085528990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcac0106f20 0x7fcac010eaa0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:45.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.374+0000 7fcac4fc7640 1 --2- 172.21.15.120:0/2085528990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcac01027a0 0x7fcac01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:45.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.374+0000 7fcac4fc7640 1 --2- 172.21.15.120:0/2085528990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcac0101df0 0x7fcac01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:45.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.374+0000 7fcac4fc7640 1 -- 172.21.15.120:0/2085528990 >> 172.21.15.120:0/2085528990 conn(0x7fcac00fd8b0 msgr2=0x7fcac00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:45.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.374+0000 7fcac4fc7640 1 -- 172.21.15.120:0/2085528990 shutdown_connections 2024-09-19T09:00:45.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.374+0000 7fcac4fc7640 1 -- 172.21.15.120:0/2085528990 wait complete. 2024-09-19T09:00:45.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.375+0000 7fcac4fc7640 1 Processor -- start 2024-09-19T09:00:45.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.375+0000 7fcac4fc7640 1 -- start start 2024-09-19T09:00:45.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.375+0000 7fcac4fc7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcac0101df0 0x7fcac019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:45.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.375+0000 7fcac4fc7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcac01027a0 0x7fcac019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:45.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.375+0000 7fcac4fc7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcac0106f20 0x7fcac019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:45.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.375+0000 7fcac4fc7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcac0111fc0 con 0x7fcac0101df0 2024-09-19T09:00:45.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.375+0000 7fcac4fc7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcac0111e40 con 0x7fcac01027a0 2024-09-19T09:00:45.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.375+0000 7fcac4fc7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcac0112140 con 0x7fcac0106f20 2024-09-19T09:00:45.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.375+0000 7fcabe59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcac0101df0 0x7fcac019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:45.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.375+0000 7fcabe59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcac0101df0 0x7fcac019a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59908/0 (socket says 172.21.15.120:59908) 2024-09-19T09:00:45.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.375+0000 7fcabe59c640 1 -- 172.21.15.120:0/1752176642 learned_addr learned my addr 172.21.15.120:0/1752176642 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:00:45.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.375+0000 7fcabed9d640 1 --2- 172.21.15.120:0/1752176642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcac0106f20 0x7fcac019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:45.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.375+0000 7fcabdd9b640 1 --2- 172.21.15.120:0/1752176642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcac01027a0 0x7fcac019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:45.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.375+0000 7fcabe59c640 1 -- 172.21.15.120:0/1752176642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcac0106f20 msgr2=0x7fcac019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:45.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.375+0000 7fcabe59c640 1 --2- 172.21.15.120:0/1752176642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcac0106f20 0x7fcac019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:45.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.375+0000 7fcabe59c640 1 -- 172.21.15.120:0/1752176642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcac01027a0 msgr2=0x7fcac019ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:45.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.375+0000 7fcabe59c640 1 --2- 172.21.15.120:0/1752176642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcac01027a0 0x7fcac019ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:45.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.375+0000 7fcabe59c640 1 -- 172.21.15.120:0/1752176642 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac019f670 con 0x7fcac0101df0 2024-09-19T09:00:45.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.376+0000 7fcabe59c640 1 --2- 172.21.15.120:0/1752176642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcac0101df0 0x7fcac019a6c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcab000b4a0 tx=0x7fcab000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:45.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.376+0000 7fcab77fe640 1 -- 172.21.15.120:0/1752176642 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0014070 con 0x7fcac0101df0 2024-09-19T09:00:45.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.376+0000 7fcac4fc7640 1 -- 172.21.15.120:0/1752176642 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac019f960 con 0x7fcac0101df0 2024-09-19T09:00:45.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.376+0000 7fcab77fe640 1 -- 172.21.15.120:0/1752176642 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab0010af0 con 0x7fcac0101df0 2024-09-19T09:00:45.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.376+0000 7fcab77fe640 1 -- 172.21.15.120:0/1752176642 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0017d10 con 0x7fcac0101df0 2024-09-19T09:00:45.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.376+0000 7fcac4fc7640 1 -- 172.21.15.120:0/1752176642 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac01a7240 con 0x7fcac0101df0 2024-09-19T09:00:45.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.377+0000 7fcac4fc7640 1 -- 172.21.15.120:0/1752176642 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcac01021d0 con 0x7fcac0101df0 2024-09-19T09:00:45.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.378+0000 7fcab77fe640 1 -- 172.21.15.120:0/1752176642 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fcab0017070 con 0x7fcac0101df0 2024-09-19T09:00:45.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.379+0000 7fcab77fe640 1 --2- 172.21.15.120:0/1752176642 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcaac0a3240 0x7fcaac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:45.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.379+0000 7fcabdd9b640 1 --2- 172.21.15.120:0/1752176642 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcaac0a3240 0x7fcaac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:45.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.379+0000 7fcab77fe640 1 -- 172.21.15.120:0/1752176642 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7497+0+0 (secure 0 0 0) 0x7fcab00cec90 con 0x7fcac0101df0 2024-09-19T09:00:45.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.379+0000 7fcabdd9b640 1 --2- 172.21.15.120:0/1752176642 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcaac0a3240 0x7fcaac0a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcac00fde90 tx=0x7fcaa800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:45.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.382+0000 7fcab77fe640 1 -- 172.21.15.120:0/1752176642 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcab009b690 con 0x7fcac0101df0 2024-09-19T09:00:45.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.549+0000 7fcac4fc7640 1 -- 172.21.15.120:0/1752176642 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fcac01a7470 con 0x7fcac0101df0 2024-09-19T09:00:45.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.935+0000 7fcab77fe640 1 -- 172.21.15.120:0/1752176642 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v77) ==== 170+0+0 (secure 0 0 0) 0x7fcab009f990 con 0x7fcac0101df0 2024-09-19T09:00:45.936 INFO:teuthology.orchestra.run.smithi120.stderr:pool 'cephfs_data' created 2024-09-19T09:00:45.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.938+0000 7fcac4fc7640 1 -- 172.21.15.120:0/1752176642 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcaac0a3240 msgr2=0x7fcaac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:45.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.938+0000 7fcac4fc7640 1 --2- 172.21.15.120:0/1752176642 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcaac0a3240 0x7fcaac0a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcac00fde90 tx=0x7fcaa800b390 comp rx=0 tx=0).stop 2024-09-19T09:00:45.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.938+0000 7fcac4fc7640 1 -- 172.21.15.120:0/1752176642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcac0101df0 msgr2=0x7fcac019a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:45.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.938+0000 7fcac4fc7640 1 --2- 172.21.15.120:0/1752176642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcac0101df0 0x7fcac019a6c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcab000b4a0 tx=0x7fcab000b960 comp rx=0 tx=0).stop 2024-09-19T09:00:45.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.938+0000 7fcac4fc7640 1 -- 172.21.15.120:0/1752176642 shutdown_connections 2024-09-19T09:00:45.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.938+0000 7fcac4fc7640 1 --2- 172.21.15.120:0/1752176642 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcaac0a3240 0x7fcaac0a5700 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:45.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.938+0000 7fcac4fc7640 1 --2- 172.21.15.120:0/1752176642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcac0106f20 0x7fcac019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:45.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.938+0000 7fcac4fc7640 1 --2- 172.21.15.120:0/1752176642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcac01027a0 0x7fcac019ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:45.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.938+0000 7fcac4fc7640 1 --2- 172.21.15.120:0/1752176642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcac0101df0 0x7fcac019a6c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:45.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.938+0000 7fcac4fc7640 1 -- 172.21.15.120:0/1752176642 >> 172.21.15.120:0/1752176642 conn(0x7fcac00fd8b0 msgr2=0x7fcac00feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:45.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.938+0000 7fcac4fc7640 1 -- 172.21.15.120:0/1752176642 shutdown_connections 2024-09-19T09:00:45.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:45.938+0000 7fcac4fc7640 1 -- 172.21.15.120:0/1752176642 wait complete. 2024-09-19T09:00:46.252 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:45 smithi120 ceph-mon[31150]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T09:00:46.252 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:45 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1752176642' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T09:00:46.252 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:45 smithi120 ceph-mon[31150]: pgmap v273: 33 pgs: 2 creating+activating, 3 creating+peering, 5 active+clean, 23 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:46.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:45 smithi196 ceph-mon[35150]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T09:00:46.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:45 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1752176642' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T09:00:46.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:45 smithi196 ceph-mon[35150]: pgmap v273: 33 pgs: 2 creating+activating, 3 creating+peering, 5 active+clean, 23 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:46.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:45 smithi153 ceph-mon[35071]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T09:00:46.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:45 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1752176642' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T09:00:46.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:45 smithi153 ceph-mon[35071]: pgmap v273: 33 pgs: 2 creating+activating, 3 creating+peering, 5 active+clean, 23 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:46.682 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T09:00:46.925 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:00:47.233 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:46 smithi120 ceph-mon[31150]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T09:00:47.233 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:46 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1752176642' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T09:00:47.233 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:46 smithi120 ceph-mon[31150]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T09:00:47.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:46 smithi196 ceph-mon[35150]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T09:00:47.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:46 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1752176642' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T09:00:47.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:46 smithi196 ceph-mon[35150]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T09:00:47.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:46 smithi153 ceph-mon[35071]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T09:00:47.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:46 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1752176642' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T09:00:47.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:46 smithi153 ceph-mon[35071]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T09:00:47.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.447+0000 7f521256e640 1 -- 172.21.15.120:0/4257791253 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f520c101df0 msgr2=0x7f520c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:47.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.447+0000 7f521256e640 1 --2- 172.21.15.120:0/4257791253 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f520c101df0 0x7f520c1021d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009900 tx=0x7f51fc02a410 comp rx=0 tx=0).stop 2024-09-19T09:00:47.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.448+0000 7f521256e640 1 -- 172.21.15.120:0/4257791253 shutdown_connections 2024-09-19T09:00:47.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.448+0000 7f521256e640 1 --2- 172.21.15.120:0/4257791253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f520c106f20 0x7f520c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:47.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.448+0000 7f521256e640 1 --2- 172.21.15.120:0/4257791253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f520c1027a0 0x7f520c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:47.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.448+0000 7f521256e640 1 --2- 172.21.15.120:0/4257791253 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f520c101df0 0x7f520c1021d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:47.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.448+0000 7f521256e640 1 -- 172.21.15.120:0/4257791253 >> 172.21.15.120:0/4257791253 conn(0x7f520c0fd8b0 msgr2=0x7f520c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:47.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.448+0000 7f521256e640 1 -- 172.21.15.120:0/4257791253 shutdown_connections 2024-09-19T09:00:47.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.448+0000 7f521256e640 1 -- 172.21.15.120:0/4257791253 wait complete. 2024-09-19T09:00:47.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.448+0000 7f521256e640 1 Processor -- start 2024-09-19T09:00:47.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.448+0000 7f521256e640 1 -- start start 2024-09-19T09:00:47.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.448+0000 7f521256e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f520c101df0 0x7f520c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:47.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.449+0000 7f521256e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f520c1027a0 0x7f520c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:47.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.449+0000 7f521256e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f520c106f20 0x7f520c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:47.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.449+0000 7f521256e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f520c111f80 con 0x7f520c1027a0 2024-09-19T09:00:47.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.449+0000 7f521256e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f520c111e00 con 0x7f520c101df0 2024-09-19T09:00:47.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.449+0000 7f521256e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f520c112100 con 0x7f520c106f20 2024-09-19T09:00:47.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.449+0000 7f5203fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f520c1027a0 0x7f520c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:47.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.449+0000 7f5203fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f520c1027a0 0x7f520c1989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59924/0 (socket says 172.21.15.120:59924) 2024-09-19T09:00:47.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.449+0000 7f5203fff640 1 -- 172.21.15.120:0/3258627469 learned_addr learned my addr 172.21.15.120:0/3258627469 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:00:47.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.449+0000 7f520bfff640 1 --2- 172.21.15.120:0/3258627469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f520c101df0 0x7f520c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:47.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.449+0000 7f5210b0b640 1 --2- 172.21.15.120:0/3258627469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f520c106f20 0x7f520c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:47.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.449+0000 7f5203fff640 1 -- 172.21.15.120:0/3258627469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f520c106f20 msgr2=0x7f520c19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:47.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.449+0000 7f5203fff640 1 --2- 172.21.15.120:0/3258627469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f520c106f20 0x7f520c19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:47.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.449+0000 7f5203fff640 1 -- 172.21.15.120:0/3258627469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f520c101df0 msgr2=0x7f520c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:47.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.449+0000 7f5203fff640 1 --2- 172.21.15.120:0/3258627469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f520c101df0 0x7f520c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:47.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.449+0000 7f5203fff640 1 -- 172.21.15.120:0/3258627469 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520c19d410 con 0x7f520c1027a0 2024-09-19T09:00:47.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.449+0000 7f520bfff640 1 --2- 172.21.15.120:0/3258627469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f520c101df0 0x7f520c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:00:47.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.449+0000 7f5203fff640 1 --2- 172.21.15.120:0/3258627469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f520c1027a0 0x7f520c1989a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f51f400d950 tx=0x7f51f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:47.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.449+0000 7f5209ffb640 1 -- 172.21.15.120:0/3258627469 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4017070 con 0x7f520c1027a0 2024-09-19T09:00:47.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.449+0000 7f5209ffb640 1 -- 172.21.15.120:0/3258627469 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51f4009c30 con 0x7f520c1027a0 2024-09-19T09:00:47.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.449+0000 7f521256e640 1 -- 172.21.15.120:0/3258627469 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520c19d700 con 0x7f520c1027a0 2024-09-19T09:00:47.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.449+0000 7f5209ffb640 1 -- 172.21.15.120:0/3258627469 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f400f6b0 con 0x7f520c1027a0 2024-09-19T09:00:47.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.449+0000 7f521256e640 1 -- 172.21.15.120:0/3258627469 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f520c1a4fe0 con 0x7f520c1027a0 2024-09-19T09:00:47.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.450+0000 7f521256e640 1 -- 172.21.15.120:0/3258627469 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f520c1021d0 con 0x7f520c1027a0 2024-09-19T09:00:47.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.451+0000 7f5209ffb640 1 -- 172.21.15.120:0/3258627469 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f51f40109d0 con 0x7f520c1027a0 2024-09-19T09:00:47.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.452+0000 7f5209ffb640 1 --2- 172.21.15.120:0/3258627469 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f51f80a3240 0x7f51f80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:47.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.452+0000 7f520bfff640 1 --2- 172.21.15.120:0/3258627469 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f51f80a3240 0x7f51f80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:47.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.452+0000 7f5209ffb640 1 -- 172.21.15.120:0/3258627469 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7879+0+0 (secure 0 0 0) 0x7f51f40cf440 con 0x7f520c1027a0 2024-09-19T09:00:47.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.452+0000 7f520bfff640 1 --2- 172.21.15.120:0/3258627469 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f51f80a3240 0x7f51f80a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f51fc005ab0 tx=0x7f51fc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:47.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.455+0000 7f5209ffb640 1 -- 172.21.15.120:0/3258627469 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f51f4009de0 con 0x7f520c1027a0 2024-09-19T09:00:47.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.658+0000 7f521256e640 1 -- 172.21.15.120:0/3258627469 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f520c10a130 con 0x7f520c1027a0 2024-09-19T09:00:47.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.955+0000 7f5209ffb640 1 -- 172.21.15.120:0/3258627469 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f51f409c310 con 0x7f520c1027a0 2024-09-19T09:00:47.955 INFO:teuthology.orchestra.run.smithi120.stderr:new fs with metadata pool 2 and data pool 3 2024-09-19T09:00:47.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.958+0000 7f521256e640 1 -- 172.21.15.120:0/3258627469 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f51f80a3240 msgr2=0x7f51f80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:47.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.958+0000 7f521256e640 1 --2- 172.21.15.120:0/3258627469 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f51f80a3240 0x7f51f80a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f51fc005ab0 tx=0x7f51fc005a20 comp rx=0 tx=0).stop 2024-09-19T09:00:47.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.958+0000 7f521256e640 1 -- 172.21.15.120:0/3258627469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f520c1027a0 msgr2=0x7f520c1989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:47.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.958+0000 7f521256e640 1 --2- 172.21.15.120:0/3258627469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f520c1027a0 0x7f520c1989a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f51f400d950 tx=0x7f51f400de10 comp rx=0 tx=0).stop 2024-09-19T09:00:47.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.958+0000 7f521256e640 1 -- 172.21.15.120:0/3258627469 shutdown_connections 2024-09-19T09:00:47.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.958+0000 7f521256e640 1 --2- 172.21.15.120:0/3258627469 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f51f80a3240 0x7f51f80a5700 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:47.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.958+0000 7f521256e640 1 --2- 172.21.15.120:0/3258627469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f520c106f20 0x7f520c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:47.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.958+0000 7f521256e640 1 --2- 172.21.15.120:0/3258627469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f520c1027a0 0x7f520c1989a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:47.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.958+0000 7f521256e640 1 --2- 172.21.15.120:0/3258627469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f520c101df0 0x7f520c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:47.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.958+0000 7f521256e640 1 -- 172.21.15.120:0/3258627469 >> 172.21.15.120:0/3258627469 conn(0x7f520c0fd8b0 msgr2=0x7f520c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:47.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.958+0000 7f521256e640 1 -- 172.21.15.120:0/3258627469 shutdown_connections 2024-09-19T09:00:47.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:47.958+0000 7f521256e640 1 -- 172.21.15.120:0/3258627469 wait complete. 2024-09-19T09:00:48.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:47 smithi120 ceph-mon[31150]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T09:00:48.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:47 smithi120 ceph-mon[31150]: pgmap v276: 97 pgs: 2 creating+activating, 20 creating+peering, 13 active+clean, 62 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:48.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:47 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3258627469' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T09:00:48.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:47 smithi120 ceph-mon[31150]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T09:00:48.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:47 smithi196 ceph-mon[35150]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T09:00:48.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:47 smithi196 ceph-mon[35150]: pgmap v276: 97 pgs: 2 creating+activating, 20 creating+peering, 13 active+clean, 62 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:48.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:47 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3258627469' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T09:00:48.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:47 smithi196 ceph-mon[35150]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T09:00:48.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:47 smithi153 ceph-mon[35071]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T09:00:48.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:47 smithi153 ceph-mon[35071]: pgmap v276: 97 pgs: 2 creating+activating, 20 creating+peering, 13 active+clean, 62 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:00:48.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:47 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3258627469' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T09:00:48.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:47 smithi153 ceph-mon[35071]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T09:00:48.699 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:00:48.941 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:00:49.231 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:48 smithi120 ceph-mon[31150]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T09:00:49.231 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:48 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3258627469' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T09:00:49.231 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:48 smithi120 ceph-mon[31150]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T09:00:49.231 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:48 smithi120 ceph-mon[31150]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T09:00:49.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:48 smithi196 ceph-mon[35150]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T09:00:49.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:48 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3258627469' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T09:00:49.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:48 smithi196 ceph-mon[35150]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T09:00:49.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:48 smithi196 ceph-mon[35150]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T09:00:49.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:48 smithi153 ceph-mon[35071]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T09:00:49.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:48 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3258627469' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T09:00:49.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:48 smithi153 ceph-mon[35071]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T09:00:49.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:48 smithi153 ceph-mon[35071]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T09:00:49.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.457+0000 7f6e1860f640 1 -- 172.21.15.120:0/825774073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e10102db0 msgr2=0x7f6e10103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:49.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.457+0000 7f6e1860f640 1 --2- 172.21.15.120:0/825774073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e10102db0 0x7f6e10103190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c009930 tx=0x7f6e0c02a420 comp rx=0 tx=0).stop 2024-09-19T09:00:49.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.457+0000 7f6e1860f640 1 -- 172.21.15.120:0/825774073 shutdown_connections 2024-09-19T09:00:49.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.457+0000 7f6e1860f640 1 --2- 172.21.15.120:0/825774073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e101082e0 0x7f6e1010ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:49.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.457+0000 7f6e1860f640 1 --2- 172.21.15.120:0/825774073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e10103760 0x7f6e10107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:49.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.457+0000 7f6e1860f640 1 --2- 172.21.15.120:0/825774073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e10102db0 0x7f6e10103190 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:49.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.457+0000 7f6e1860f640 1 -- 172.21.15.120:0/825774073 >> 172.21.15.120:0/825774073 conn(0x7f6e100fc7d0 msgr2=0x7f6e100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:49.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.457+0000 7f6e1860f640 1 -- 172.21.15.120:0/825774073 shutdown_connections 2024-09-19T09:00:49.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.457+0000 7f6e1860f640 1 -- 172.21.15.120:0/825774073 wait complete. 2024-09-19T09:00:49.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.457+0000 7f6e1860f640 1 Processor -- start 2024-09-19T09:00:49.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.457+0000 7f6e1860f640 1 -- start start 2024-09-19T09:00:49.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.458+0000 7f6e1860f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e10102db0 0x7f6e10198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:49.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.458+0000 7f6e1860f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e10103760 0x7f6e10198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:49.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.458+0000 7f6e1860f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e101082e0 0x7f6e1019cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:49.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.458+0000 7f6e1860f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6e10112110 con 0x7f6e10102db0 2024-09-19T09:00:49.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.458+0000 7f6e1860f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6e10111f90 con 0x7f6e101082e0 2024-09-19T09:00:49.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.458+0000 7f6e1860f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6e10112290 con 0x7f6e10103760 2024-09-19T09:00:49.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.458+0000 7f6e163ab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e10102db0 0x7f6e10198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:49.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.458+0000 7f6e163ab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e10102db0 0x7f6e10198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59944/0 (socket says 172.21.15.120:59944) 2024-09-19T09:00:49.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.458+0000 7f6e163ab640 1 -- 172.21.15.120:0/1527812819 learned_addr learned my addr 172.21.15.120:0/1527812819 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:00:49.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.458+0000 7f6e16bac640 1 --2- 172.21.15.120:0/1527812819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e101082e0 0x7f6e1019cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:49.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.458+0000 7f6e163ab640 1 -- 172.21.15.120:0/1527812819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e10103760 msgr2=0x7f6e10198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:49.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.458+0000 7f6e163ab640 1 --2- 172.21.15.120:0/1527812819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e10103760 0x7f6e10198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:49.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.458+0000 7f6e163ab640 1 -- 172.21.15.120:0/1527812819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e101082e0 msgr2=0x7f6e1019cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:49.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.458+0000 7f6e163ab640 1 --2- 172.21.15.120:0/1527812819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e101082e0 0x7f6e1019cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:49.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.458+0000 7f6e163ab640 1 -- 172.21.15.120:0/1527812819 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e1019d5f0 con 0x7f6e10102db0 2024-09-19T09:00:49.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.458+0000 7f6e163ab640 1 --2- 172.21.15.120:0/1527812819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e10102db0 0x7f6e10198640 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c004770 tx=0x7f6e0c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:49.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.458+0000 7f6e077fe640 1 -- 172.21.15.120:0/1527812819 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c03f070 con 0x7f6e10102db0 2024-09-19T09:00:49.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.458+0000 7f6e1860f640 1 -- 172.21.15.120:0/1527812819 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e1019d880 con 0x7f6e10102db0 2024-09-19T09:00:49.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.458+0000 7f6e077fe640 1 -- 172.21.15.120:0/1527812819 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e0c031c20 con 0x7f6e10102db0 2024-09-19T09:00:49.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.458+0000 7f6e077fe640 1 -- 172.21.15.120:0/1527812819 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c035030 con 0x7f6e10102db0 2024-09-19T09:00:49.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.458+0000 7f6e1860f640 1 -- 172.21.15.120:0/1527812819 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e101a51c0 con 0x7f6e10102db0 2024-09-19T09:00:49.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.460+0000 7f6e1860f640 1 -- 172.21.15.120:0/1527812819 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e10103190 con 0x7f6e10102db0 2024-09-19T09:00:49.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.461+0000 7f6e077fe640 1 -- 172.21.15.120:0/1527812819 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f6e0c042070 con 0x7f6e10102db0 2024-09-19T09:00:49.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.461+0000 7f6e077fe640 1 --2- 172.21.15.120:0/1527812819 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6dfc0a3240 0x7f6dfc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:49.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.461+0000 7f6e15baa640 1 --2- 172.21.15.120:0/1527812819 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6dfc0a3240 0x7f6dfc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:49.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.462+0000 7f6e077fe640 1 -- 172.21.15.120:0/1527812819 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f6e0c0f08a0 con 0x7f6e10102db0 2024-09-19T09:00:49.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.462+0000 7f6e15baa640 1 --2- 172.21.15.120:0/1527812819 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6dfc0a3240 0x7f6dfc0a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6e00005fd0 tx=0x7f6e00005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:49.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.465+0000 7f6e077fe640 1 -- 172.21.15.120:0/1527812819 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6e0c0bd710 con 0x7f6e10102db0 2024-09-19T09:00:49.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.639+0000 7f6e1860f640 1 -- 172.21.15.120:0/1527812819 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e1010a2c0 con 0x7f6e10102db0 2024-09-19T09:00:49.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.641+0000 7f6e077fe640 1 -- 172.21.15.120:0/1527812819 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22362 (secure 0 0 0) 0x7f6e0c0bd010 con 0x7f6e10102db0 2024-09-19T09:00:49.641 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:00:49.641 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":79,"fsid":"507393d2-7664-11ef-bceb-c7b262605968","created":"2024-09-19T08:53:06.386526+0000","modified":"2024-09-19T09:00:47.659465+0000","last_up_change":"2024-09-19T08:59:55.692659+0000","last_in_change":"2024-09-19T08:59:32.808860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:56:07.830047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:00:43.774339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:00:45.786262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33ec3993-576c-428c-bf75-adcb17cd72a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6803","nonce":4131180019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6805","nonce":4131180019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6809","nonce":4131180019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6807","nonce":4131180019}]},"public_addr":"172.21.15.120:6803/4131180019","cluster_addr":"172.21.15.120:6805/4131180019","heartbeat_back_addr":"172.21.15.120:6809/4131180019","heartbeat_front_addr":"172.21.15.120:6807/4131180019","state":["exists","up"]},{"osd":1,"uuid":"450bbebf-2aac-47c6-8474-bbd37fd0a1a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6801","nonce":2259730294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6803","nonce":2259730294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6807","nonce":2259730294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6805","nonce":2259730294}]},"public_addr":"172.21.15.153:6801/2259730294","cluster_addr":"172.21.15.153:6803/2259730294","heartbeat_back_addr":"172.21.15.153:6807/2259730294","heartbeat_front_addr":"172.21.15.153:6805/2259730294","state":["exists","up"]},{"osd":2,"uuid":"ac8c4b83-8e8e-4803-b266-262f7873bd85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6801","nonce":2942286570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6803","nonce":2942286570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6807","nonce":2942286570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6805","nonce":2942286570}]},"public_addr":"172.21.15.196:6801/2942286570","cluster_addr":"172.21.15.196:6803/2942286570","heartbeat_back_addr":"172.21.15.196:6807/2942286570","heartbeat_front_addr":"172.21.15.196:6805/2942286570","state":["exists","up"]},{"osd":3,"uuid":"c1d9e507-8f02-403e-87da-9b4966fd8fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6811","nonce":713205668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6813","nonce":713205668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6817","nonce":713205668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6815","nonce":713205668}]},"public_addr":"172.21.15.120:6811/713205668","cluster_addr":"172.21.15.120:6813/713205668","heartbeat_back_addr":"172.21.15.120:6817/713205668","heartbeat_front_addr":"172.21.15.120:6815/713205668","state":["exists","up"]},{"osd":4,"uuid":"7f8bf3b9-0ec4-46f6-a406-d26d7971179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6809","nonce":2680865854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6811","nonce":2680865854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6815","nonce":2680865854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6813","nonce":2680865854}]},"public_addr":"172.21.15.153:6809/2680865854","cluster_addr":"172.21.15.153:6811/2680865854","heartbeat_back_addr":"172.21.15.153:6815/2680865854","heartbeat_front_addr":"172.21.15.153:6813/2680865854","state":["exists","up"]},{"osd":5,"uuid":"70b270b2-209d-4ff5-875a-04ee7b992a93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6809","nonce":3763331116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6811","nonce":3763331116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6815","nonce":3763331116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6813","nonce":3763331116}]},"public_addr":"172.21.15.196:6809/3763331116","cluster_addr":"172.21.15.196:6811/3763331116","heartbeat_back_addr":"172.21.15.196:6815/3763331116","heartbeat_front_addr":"172.21.15.196:6813/3763331116","state":["exists","up"]},{"osd":6,"uuid":"8087f69c-dc31-4522-a18d-4e6ab1b7cb27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6819","nonce":4171773765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6821","nonce":4171773765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6825","nonce":4171773765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6823","nonce":4171773765}]},"public_addr":"172.21.15.120:6819/4171773765","cluster_addr":"172.21.15.120:6821/4171773765","heartbeat_back_addr":"172.21.15.120:6825/4171773765","heartbeat_front_addr":"172.21.15.120:6823/4171773765","state":["exists","up"]},{"osd":7,"uuid":"c3b1ead3-150f-43fd-8924-f639203f4cdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6817","nonce":2535148453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6819","nonce":2535148453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6823","nonce":2535148453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6821","nonce":2535148453}]},"public_addr":"172.21.15.153:6817/2535148453","cluster_addr":"172.21.15.153:6819/2535148453","heartbeat_back_addr":"172.21.15.153:6823/2535148453","heartbeat_front_addr":"172.21.15.153:6821/2535148453","state":["exists","up"]},{"osd":8,"uuid":"a3273a2a-968e-4e93-a9cf-3879892bdeab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6817","nonce":186195906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6819","nonce":186195906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6823","nonce":186195906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6821","nonce":186195906}]},"public_addr":"172.21.15.196:6817/186195906","cluster_addr":"172.21.15.196:6819/186195906","heartbeat_back_addr":"172.21.15.196:6823/186195906","heartbeat_front_addr":"172.21.15.196:6821/186195906","state":["exists","up"]},{"osd":9,"uuid":"6d184513-dab8-45f7-bc08-fa2e8b2f10b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6827","nonce":1559320883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6829","nonce":1559320883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6833","nonce":1559320883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6831","nonce":1559320883}]},"public_addr":"172.21.15.120:6827/1559320883","cluster_addr":"172.21.15.120:6829/1559320883","heartbeat_back_addr":"172.21.15.120:6833/1559320883","heartbeat_front_addr":"172.21.15.120:6831/1559320883","state":["exists","up"]},{"osd":10,"uuid":"797db886-62f9-4316-a45b-4d50512e65d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6825","nonce":3216877560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6827","nonce":3216877560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6831","nonce":3216877560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6829","nonce":3216877560}]},"public_addr":"172.21.15.153:6825/3216877560","cluster_addr":"172.21.15.153:6827/3216877560","heartbeat_back_addr":"172.21.15.153:6831/3216877560","heartbeat_front_addr":"172.21.15.153:6829/3216877560","state":["exists","up"]},{"osd":11,"uuid":"17e0afaa-84eb-4b26-9810-bfa3cc7e0f3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6825","nonce":1857200740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6827","nonce":1857200740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6831","nonce":1857200740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6829","nonce":1857200740}]},"public_addr":"172.21.15.196:6825/1857200740","cluster_addr":"172.21.15.196:6827/1857200740","heartbeat_back_addr":"172.21.15.196:6831/1857200740","heartbeat_front_addr":"172.21.15.196:6829/1857200740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:55:15.595645+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:55:40.405487+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:04.938034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:29.718696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:55.636029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:57:21.257083+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:57:46.316552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:58:11.946863+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:58:37.098578+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:03.759023+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:28.659049+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:54.350475+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:0/3010783707":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6801/3562882246":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6800/3562882246":"2024-09-20T08:53:51.528772+0000","172.21.15.120:0/3224512035":"2024-09-20T08:53:51.528772+0000","172.21.15.120:0/3465105835":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/3317449610":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/2800697451":"2024-09-20T08:53:28.144376+0000","172.21.15.120:6801/2938446665":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/109127432":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6800/2938446665":"2024-09-20T08:53:28.144376+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:00:49.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.643+0000 7f6e1860f640 1 -- 172.21.15.120:0/1527812819 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6dfc0a3240 msgr2=0x7f6dfc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:49.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.643+0000 7f6e1860f640 1 --2- 172.21.15.120:0/1527812819 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6dfc0a3240 0x7f6dfc0a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6e00005fd0 tx=0x7f6e00005850 comp rx=0 tx=0).stop 2024-09-19T09:00:49.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.643+0000 7f6e1860f640 1 -- 172.21.15.120:0/1527812819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e10102db0 msgr2=0x7f6e10198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:49.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.643+0000 7f6e1860f640 1 --2- 172.21.15.120:0/1527812819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e10102db0 0x7f6e10198640 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c004770 tx=0x7f6e0c003b90 comp rx=0 tx=0).stop 2024-09-19T09:00:49.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.643+0000 7f6e1860f640 1 -- 172.21.15.120:0/1527812819 shutdown_connections 2024-09-19T09:00:49.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.643+0000 7f6e1860f640 1 --2- 172.21.15.120:0/1527812819 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6dfc0a3240 0x7f6dfc0a5700 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:49.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.643+0000 7f6e1860f640 1 --2- 172.21.15.120:0/1527812819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e101082e0 0x7f6e1019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:49.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.643+0000 7f6e1860f640 1 --2- 172.21.15.120:0/1527812819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e10103760 0x7f6e10198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:49.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.643+0000 7f6e1860f640 1 --2- 172.21.15.120:0/1527812819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e10102db0 0x7f6e10198640 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:49.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.643+0000 7f6e1860f640 1 -- 172.21.15.120:0/1527812819 >> 172.21.15.120:0/1527812819 conn(0x7f6e100fc7d0 msgr2=0x7f6e10105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:49.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.643+0000 7f6e1860f640 1 -- 172.21.15.120:0/1527812819 shutdown_connections 2024-09-19T09:00:49.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:49.643+0000 7f6e1860f640 1 -- 172.21.15.120:0/1527812819 wait complete. 2024-09-19T09:00:50.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:49 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] up:active 2024-09-19T09:00:50.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:49 smithi120 ceph-mon[31150]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T09:00:50.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:49 smithi120 ceph-mon[31150]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T09:00:50.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:49 smithi120 ceph-mon[31150]: pgmap v278: 97 pgs: 2 creating+activating, 20 creating+peering, 59 active+clean, 16 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 437 B/s wr, 1 op/s 2024-09-19T09:00:50.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:49 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1527812819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:00:50.385 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:00:50.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:49 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] up:active 2024-09-19T09:00:50.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:49 smithi196 ceph-mon[35150]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T09:00:50.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:49 smithi196 ceph-mon[35150]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T09:00:50.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:49 smithi196 ceph-mon[35150]: pgmap v278: 97 pgs: 2 creating+activating, 20 creating+peering, 59 active+clean, 16 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 437 B/s wr, 1 op/s 2024-09-19T09:00:50.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:49 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1527812819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:00:50.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:49 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] up:active 2024-09-19T09:00:50.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:49 smithi153 ceph-mon[35071]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T09:00:50.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:49 smithi153 ceph-mon[35071]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T09:00:50.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:49 smithi153 ceph-mon[35071]: pgmap v278: 97 pgs: 2 creating+activating, 20 creating+peering, 59 active+clean, 16 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 437 B/s wr, 1 op/s 2024-09-19T09:00:50.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:49 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1527812819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:00:50.635 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:00:51.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:50 smithi120 ceph-mon[31150]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T09:00:51.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:50 smithi120 ceph-mon[31150]: Cluster is now healthy 2024-09-19T09:00:51.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.167+0000 7fb241e86640 1 -- 172.21.15.120:0/3516696176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb23c102790 msgr2=0x7fb23c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:51.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.167+0000 7fb241e86640 1 --2- 172.21.15.120:0/3516696176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb23c102790 0x7fb23c1067e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb224009900 tx=0x7fb22402a410 comp rx=0 tx=0).stop 2024-09-19T09:00:51.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.167+0000 7fb241e86640 1 -- 172.21.15.120:0/3516696176 shutdown_connections 2024-09-19T09:00:51.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.167+0000 7fb241e86640 1 --2- 172.21.15.120:0/3516696176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb23c106f10 0x7fb23c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:51.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.167+0000 7fb241e86640 1 --2- 172.21.15.120:0/3516696176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb23c102790 0x7fb23c1067e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:51.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.167+0000 7fb241e86640 1 --2- 172.21.15.120:0/3516696176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb23c101de0 0x7fb23c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:51.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.167+0000 7fb241e86640 1 -- 172.21.15.120:0/3516696176 >> 172.21.15.120:0/3516696176 conn(0x7fb23c0fd8c0 msgr2=0x7fb23c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:51.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.167+0000 7fb241e86640 1 -- 172.21.15.120:0/3516696176 shutdown_connections 2024-09-19T09:00:51.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.167+0000 7fb241e86640 1 -- 172.21.15.120:0/3516696176 wait complete. 2024-09-19T09:00:51.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.168+0000 7fb241e86640 1 Processor -- start 2024-09-19T09:00:51.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.168+0000 7fb241e86640 1 -- start start 2024-09-19T09:00:51.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.168+0000 7fb241e86640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb23c101de0 0x7fb23c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:51.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.168+0000 7fb241e86640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb23c102790 0x7fb23c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:51.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.168+0000 7fb241e86640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb23c106f10 0x7fb23c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:51.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.168+0000 7fb241e86640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c112010 con 0x7fb23c101de0 2024-09-19T09:00:51.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.168+0000 7fb241e86640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c111e90 con 0x7fb23c102790 2024-09-19T09:00:51.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.168+0000 7fb241e86640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c112190 con 0x7fb23c106f10 2024-09-19T09:00:51.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.168+0000 7fb23b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb23c101de0 0x7fb23c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:51.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.168+0000 7fb23b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb23c101de0 0x7fb23c1985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59966/0 (socket says 172.21.15.120:59966) 2024-09-19T09:00:51.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.168+0000 7fb23b7fe640 1 -- 172.21.15.120:0/3901683017 learned_addr learned my addr 172.21.15.120:0/3901683017 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:00:51.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.168+0000 7fb233fff640 1 --2- 172.21.15.120:0/3901683017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb23c102790 0x7fb23c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:51.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.169+0000 7fb23b7fe640 1 -- 172.21.15.120:0/3901683017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb23c106f10 msgr2=0x7fb23c19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:51.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.169+0000 7fb23b7fe640 1 --2- 172.21.15.120:0/3901683017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb23c106f10 0x7fb23c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:51.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.169+0000 7fb23b7fe640 1 -- 172.21.15.120:0/3901683017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb23c102790 msgr2=0x7fb23c198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:51.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.169+0000 7fb23b7fe640 1 --2- 172.21.15.120:0/3901683017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb23c102790 0x7fb23c198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:51.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.169+0000 7fb23b7fe640 1 -- 172.21.15.120:0/3901683017 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23c19d560 con 0x7fb23c101de0 2024-09-19T09:00:51.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.169+0000 7fb233fff640 1 --2- 172.21.15.120:0/3901683017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb23c102790 0x7fb23c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:00:51.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.169+0000 7fb23b7fe640 1 --2- 172.21.15.120:0/3901683017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb23c101de0 0x7fb23c1985b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00b4a0 tx=0x7fb22c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:51.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.169+0000 7fb2397fa640 1 -- 172.21.15.120:0/3901683017 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c014070 con 0x7fb23c101de0 2024-09-19T09:00:51.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.169+0000 7fb2397fa640 1 -- 172.21.15.120:0/3901683017 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb22c010af0 con 0x7fb23c101de0 2024-09-19T09:00:51.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.169+0000 7fb241e86640 1 -- 172.21.15.120:0/3901683017 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb23c19d850 con 0x7fb23c101de0 2024-09-19T09:00:51.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.169+0000 7fb2397fa640 1 -- 172.21.15.120:0/3901683017 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c017d10 con 0x7fb23c101de0 2024-09-19T09:00:51.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.169+0000 7fb241e86640 1 -- 172.21.15.120:0/3901683017 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb23c1a5130 con 0x7fb23c101de0 2024-09-19T09:00:51.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.170+0000 7fb241e86640 1 -- 172.21.15.120:0/3901683017 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb23c102240 con 0x7fb23c101de0 2024-09-19T09:00:51.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.171+0000 7fb2397fa640 1 -- 172.21.15.120:0/3901683017 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fb22c017070 con 0x7fb23c101de0 2024-09-19T09:00:51.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.172+0000 7fb2397fa640 1 --2- 172.21.15.120:0/3901683017 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb2280a3240 0x7fb2280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:51.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.172+0000 7fb233fff640 1 --2- 172.21.15.120:0/3901683017 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb2280a3240 0x7fb2280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:51.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.172+0000 7fb2397fa640 1 -- 172.21.15.120:0/3901683017 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fb22c0cec40 con 0x7fb23c101de0 2024-09-19T09:00:51.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.172+0000 7fb233fff640 1 --2- 172.21.15.120:0/3901683017 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb2280a3240 0x7fb2280a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0fdd50 tx=0x7fb224005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:51.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.176+0000 7fb2397fa640 1 -- 172.21.15.120:0/3901683017 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb22c09bab0 con 0x7fb23c101de0 2024-09-19T09:00:51.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.343+0000 7fb241e86640 1 -- 172.21.15.120:0/3901683017 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb23c109b60 con 0x7fb23c101de0 2024-09-19T09:00:51.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.344+0000 7fb2397fa640 1 -- 172.21.15.120:0/3901683017 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22362 (secure 0 0 0) 0x7fb22c09b3b0 con 0x7fb23c101de0 2024-09-19T09:00:51.344 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:00:51.345 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":79,"fsid":"507393d2-7664-11ef-bceb-c7b262605968","created":"2024-09-19T08:53:06.386526+0000","modified":"2024-09-19T09:00:47.659465+0000","last_up_change":"2024-09-19T08:59:55.692659+0000","last_in_change":"2024-09-19T08:59:32.808860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:56:07.830047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:00:43.774339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:00:45.786262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33ec3993-576c-428c-bf75-adcb17cd72a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6803","nonce":4131180019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6805","nonce":4131180019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6809","nonce":4131180019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6807","nonce":4131180019}]},"public_addr":"172.21.15.120:6803/4131180019","cluster_addr":"172.21.15.120:6805/4131180019","heartbeat_back_addr":"172.21.15.120:6809/4131180019","heartbeat_front_addr":"172.21.15.120:6807/4131180019","state":["exists","up"]},{"osd":1,"uuid":"450bbebf-2aac-47c6-8474-bbd37fd0a1a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6801","nonce":2259730294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6803","nonce":2259730294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6807","nonce":2259730294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6805","nonce":2259730294}]},"public_addr":"172.21.15.153:6801/2259730294","cluster_addr":"172.21.15.153:6803/2259730294","heartbeat_back_addr":"172.21.15.153:6807/2259730294","heartbeat_front_addr":"172.21.15.153:6805/2259730294","state":["exists","up"]},{"osd":2,"uuid":"ac8c4b83-8e8e-4803-b266-262f7873bd85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6801","nonce":2942286570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6803","nonce":2942286570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6807","nonce":2942286570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6805","nonce":2942286570}]},"public_addr":"172.21.15.196:6801/2942286570","cluster_addr":"172.21.15.196:6803/2942286570","heartbeat_back_addr":"172.21.15.196:6807/2942286570","heartbeat_front_addr":"172.21.15.196:6805/2942286570","state":["exists","up"]},{"osd":3,"uuid":"c1d9e507-8f02-403e-87da-9b4966fd8fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6811","nonce":713205668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6813","nonce":713205668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6817","nonce":713205668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6815","nonce":713205668}]},"public_addr":"172.21.15.120:6811/713205668","cluster_addr":"172.21.15.120:6813/713205668","heartbeat_back_addr":"172.21.15.120:6817/713205668","heartbeat_front_addr":"172.21.15.120:6815/713205668","state":["exists","up"]},{"osd":4,"uuid":"7f8bf3b9-0ec4-46f6-a406-d26d7971179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6809","nonce":2680865854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6811","nonce":2680865854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6815","nonce":2680865854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6813","nonce":2680865854}]},"public_addr":"172.21.15.153:6809/2680865854","cluster_addr":"172.21.15.153:6811/2680865854","heartbeat_back_addr":"172.21.15.153:6815/2680865854","heartbeat_front_addr":"172.21.15.153:6813/2680865854","state":["exists","up"]},{"osd":5,"uuid":"70b270b2-209d-4ff5-875a-04ee7b992a93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6809","nonce":3763331116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6811","nonce":3763331116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6815","nonce":3763331116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6813","nonce":3763331116}]},"public_addr":"172.21.15.196:6809/3763331116","cluster_addr":"172.21.15.196:6811/3763331116","heartbeat_back_addr":"172.21.15.196:6815/3763331116","heartbeat_front_addr":"172.21.15.196:6813/3763331116","state":["exists","up"]},{"osd":6,"uuid":"8087f69c-dc31-4522-a18d-4e6ab1b7cb27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6819","nonce":4171773765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6821","nonce":4171773765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6825","nonce":4171773765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6823","nonce":4171773765}]},"public_addr":"172.21.15.120:6819/4171773765","cluster_addr":"172.21.15.120:6821/4171773765","heartbeat_back_addr":"172.21.15.120:6825/4171773765","heartbeat_front_addr":"172.21.15.120:6823/4171773765","state":["exists","up"]},{"osd":7,"uuid":"c3b1ead3-150f-43fd-8924-f639203f4cdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6817","nonce":2535148453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6819","nonce":2535148453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6823","nonce":2535148453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6821","nonce":2535148453}]},"public_addr":"172.21.15.153:6817/2535148453","cluster_addr":"172.21.15.153:6819/2535148453","heartbeat_back_addr":"172.21.15.153:6823/2535148453","heartbeat_front_addr":"172.21.15.153:6821/2535148453","state":["exists","up"]},{"osd":8,"uuid":"a3273a2a-968e-4e93-a9cf-3879892bdeab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6817","nonce":186195906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6819","nonce":186195906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6823","nonce":186195906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6821","nonce":186195906}]},"public_addr":"172.21.15.196:6817/186195906","cluster_addr":"172.21.15.196:6819/186195906","heartbeat_back_addr":"172.21.15.196:6823/186195906","heartbeat_front_addr":"172.21.15.196:6821/186195906","state":["exists","up"]},{"osd":9,"uuid":"6d184513-dab8-45f7-bc08-fa2e8b2f10b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6827","nonce":1559320883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6829","nonce":1559320883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6833","nonce":1559320883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6831","nonce":1559320883}]},"public_addr":"172.21.15.120:6827/1559320883","cluster_addr":"172.21.15.120:6829/1559320883","heartbeat_back_addr":"172.21.15.120:6833/1559320883","heartbeat_front_addr":"172.21.15.120:6831/1559320883","state":["exists","up"]},{"osd":10,"uuid":"797db886-62f9-4316-a45b-4d50512e65d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6825","nonce":3216877560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6827","nonce":3216877560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6831","nonce":3216877560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6829","nonce":3216877560}]},"public_addr":"172.21.15.153:6825/3216877560","cluster_addr":"172.21.15.153:6827/3216877560","heartbeat_back_addr":"172.21.15.153:6831/3216877560","heartbeat_front_addr":"172.21.15.153:6829/3216877560","state":["exists","up"]},{"osd":11,"uuid":"17e0afaa-84eb-4b26-9810-bfa3cc7e0f3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6825","nonce":1857200740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6827","nonce":1857200740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6831","nonce":1857200740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6829","nonce":1857200740}]},"public_addr":"172.21.15.196:6825/1857200740","cluster_addr":"172.21.15.196:6827/1857200740","heartbeat_back_addr":"172.21.15.196:6831/1857200740","heartbeat_front_addr":"172.21.15.196:6829/1857200740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:55:15.595645+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:55:40.405487+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:04.938034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:29.718696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:55.636029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:57:21.257083+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:57:46.316552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:58:11.946863+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:58:37.098578+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:03.759023+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:28.659049+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:54.350475+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:0/3010783707":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6801/3562882246":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6800/3562882246":"2024-09-20T08:53:51.528772+0000","172.21.15.120:0/3224512035":"2024-09-20T08:53:51.528772+0000","172.21.15.120:0/3465105835":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/3317449610":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/2800697451":"2024-09-20T08:53:28.144376+0000","172.21.15.120:6801/2938446665":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/109127432":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6800/2938446665":"2024-09-20T08:53:28.144376+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:00:51.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.347+0000 7fb241e86640 1 -- 172.21.15.120:0/3901683017 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb2280a3240 msgr2=0x7fb2280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:51.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.347+0000 7fb241e86640 1 --2- 172.21.15.120:0/3901683017 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb2280a3240 0x7fb2280a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0fdd50 tx=0x7fb224005a20 comp rx=0 tx=0).stop 2024-09-19T09:00:51.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.347+0000 7fb241e86640 1 -- 172.21.15.120:0/3901683017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb23c101de0 msgr2=0x7fb23c1985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:51.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.347+0000 7fb241e86640 1 --2- 172.21.15.120:0/3901683017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb23c101de0 0x7fb23c1985b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00b4a0 tx=0x7fb22c00b960 comp rx=0 tx=0).stop 2024-09-19T09:00:51.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.347+0000 7fb241e86640 1 -- 172.21.15.120:0/3901683017 shutdown_connections 2024-09-19T09:00:51.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.347+0000 7fb241e86640 1 --2- 172.21.15.120:0/3901683017 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb2280a3240 0x7fb2280a5700 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:51.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.347+0000 7fb241e86640 1 --2- 172.21.15.120:0/3901683017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb23c106f10 0x7fb23c19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:51.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.347+0000 7fb241e86640 1 --2- 172.21.15.120:0/3901683017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb23c102790 0x7fb23c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:51.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.347+0000 7fb241e86640 1 --2- 172.21.15.120:0/3901683017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb23c101de0 0x7fb23c1985b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:51.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.347+0000 7fb241e86640 1 -- 172.21.15.120:0/3901683017 >> 172.21.15.120:0/3901683017 conn(0x7fb23c0fd8c0 msgr2=0x7fb23c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:51.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.347+0000 7fb241e86640 1 -- 172.21.15.120:0/3901683017 shutdown_connections 2024-09-19T09:00:51.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:51.347+0000 7fb241e86640 1 -- 172.21.15.120:0/3901683017 wait complete. 2024-09-19T09:00:51.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:50 smithi196 ceph-mon[35150]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T09:00:51.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:50 smithi196 ceph-mon[35150]: Cluster is now healthy 2024-09-19T09:00:51.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:50 smithi153 ceph-mon[35071]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T09:00:51.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:50 smithi153 ceph-mon[35071]: Cluster is now healthy 2024-09-19T09:00:52.049 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:51 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3901683017' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:00:52.049 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:51 smithi120 ceph-mon[31150]: pgmap v279: 97 pgs: 9 creating+peering, 88 active+clean; 578 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 5 op/s 2024-09-19T09:00:52.078 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-19T09:00:52.322 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:00:52.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:51 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3901683017' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:00:52.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:51 smithi196 ceph-mon[35150]: pgmap v279: 97 pgs: 9 creating+peering, 88 active+clean; 578 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 5 op/s 2024-09-19T09:00:52.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:51 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3901683017' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:00:52.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:51 smithi153 ceph-mon[35071]: pgmap v279: 97 pgs: 9 creating+peering, 88 active+clean; 578 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 5 op/s 2024-09-19T09:00:52.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.811+0000 7f96481f9640 1 -- 172.21.15.120:0/3911396030 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9640100e20 msgr2=0x7f9640101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:52.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.811+0000 7f96481f9640 1 --2- 172.21.15.120:0/3911396030 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9640100e20 0x7f9640101200 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f963c009900 tx=0x7f963c02a480 comp rx=0 tx=0).stop 2024-09-19T09:00:52.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.811+0000 7f96481f9640 1 -- 172.21.15.120:0/3911396030 shutdown_connections 2024-09-19T09:00:52.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.811+0000 7f96481f9640 1 --2- 172.21.15.120:0/3911396030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9640102ef0 0x7f964010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:52.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.811+0000 7f96481f9640 1 --2- 172.21.15.120:0/3911396030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9640102570 0x7f96401029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:52.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.811+0000 7f96481f9640 1 --2- 172.21.15.120:0/3911396030 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9640100e20 0x7f9640101200 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:52.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.811+0000 7f96481f9640 1 -- 172.21.15.120:0/3911396030 >> 172.21.15.120:0/3911396030 conn(0x7f96400fc7d0 msgr2=0x7f96400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:52.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.811+0000 7f96481f9640 1 -- 172.21.15.120:0/3911396030 shutdown_connections 2024-09-19T09:00:52.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.811+0000 7f96481f9640 1 -- 172.21.15.120:0/3911396030 wait complete. 2024-09-19T09:00:52.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.811+0000 7f96481f9640 1 Processor -- start 2024-09-19T09:00:52.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.811+0000 7f96481f9640 1 -- start start 2024-09-19T09:00:52.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.811+0000 7f96481f9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9640100e20 0x7f96401994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:52.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.811+0000 7f96481f9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9640102570 0x7f9640199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:52.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.811+0000 7f96481f9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9640102ef0 0x7f96401a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:52.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.811+0000 7f96481f9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9640111020 con 0x7f9640102570 2024-09-19T09:00:52.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.811+0000 7f96481f9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9640110ea0 con 0x7f9640102ef0 2024-09-19T09:00:52.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.811+0000 7f96481f9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f96401111a0 con 0x7f9640100e20 2024-09-19T09:00:52.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.812+0000 7f9645794640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9640102570 0x7f9640199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:52.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.812+0000 7f9645794640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9640102570 0x7f9640199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46984/0 (socket says 172.21.15.120:46984) 2024-09-19T09:00:52.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.812+0000 7f9645794640 1 -- 172.21.15.120:0/839319716 learned_addr learned my addr 172.21.15.120:0/839319716 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:00:52.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.812+0000 7f9645f95640 1 --2- 172.21.15.120:0/839319716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9640100e20 0x7f96401994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:52.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.812+0000 7f9645794640 1 -- 172.21.15.120:0/839319716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9640100e20 msgr2=0x7f96401994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:52.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.812+0000 7f9645794640 1 --2- 172.21.15.120:0/839319716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9640100e20 0x7f96401994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:52.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.812+0000 7f9645794640 1 -- 172.21.15.120:0/839319716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9640102ef0 msgr2=0x7f96401a4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:52.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.812+0000 7f9646796640 1 --2- 172.21.15.120:0/839319716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9640102ef0 0x7f96401a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:52.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.812+0000 7f9645794640 1 --2- 172.21.15.120:0/839319716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9640102ef0 0x7f96401a4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:52.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.812+0000 7f9645794640 1 -- 172.21.15.120:0/839319716 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96401a4ab0 con 0x7f9640102570 2024-09-19T09:00:52.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.812+0000 7f9645f95640 1 --2- 172.21.15.120:0/839319716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9640100e20 0x7f96401994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:00:52.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.812+0000 7f9646796640 1 --2- 172.21.15.120:0/839319716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9640102ef0 0x7f96401a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:00:52.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.812+0000 7f9645794640 1 --2- 172.21.15.120:0/839319716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9640102570 0x7f9640199a30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f963000b4a0 tx=0x7f963000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:52.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.812+0000 7f9636ffd640 1 -- 172.21.15.120:0/839319716 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9630014070 con 0x7f9640102570 2024-09-19T09:00:52.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.812+0000 7f9636ffd640 1 -- 172.21.15.120:0/839319716 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9630010af0 con 0x7f9640102570 2024-09-19T09:00:52.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.812+0000 7f96481f9640 1 -- 172.21.15.120:0/839319716 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96401a4c80 con 0x7f9640102570 2024-09-19T09:00:52.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.812+0000 7f9636ffd640 1 -- 172.21.15.120:0/839319716 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9630017d10 con 0x7f9640102570 2024-09-19T09:00:52.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.812+0000 7f96481f9640 1 -- 172.21.15.120:0/839319716 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96401a5190 con 0x7f9640102570 2024-09-19T09:00:52.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.813+0000 7f96481f9640 1 -- 172.21.15.120:0/839319716 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9640101280 con 0x7f9640102570 2024-09-19T09:00:52.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.814+0000 7f9636ffd640 1 -- 172.21.15.120:0/839319716 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f9630017070 con 0x7f9640102570 2024-09-19T09:00:52.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.815+0000 7f9636ffd640 1 --2- 172.21.15.120:0/839319716 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f962c0a3240 0x7f962c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:52.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.815+0000 7f9645f95640 1 --2- 172.21.15.120:0/839319716 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f962c0a3240 0x7f962c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:52.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.816+0000 7f9636ffd640 1 -- 172.21.15.120:0/839319716 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f96300cebe0 con 0x7f9640102570 2024-09-19T09:00:52.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.816+0000 7f9645f95640 1 --2- 172.21.15.120:0/839319716 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f962c0a3240 0x7f962c0a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f963c000c00 tx=0x7f963c0041e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:52.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:52.820+0000 7f9636ffd640 1 -- 172.21.15.120:0/839319716 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f963009ba50 con 0x7f9640102570 2024-09-19T09:00:53.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:53.022+0000 7f96481f9640 1 -- 172.21.15.120:0/839319716 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f9640109090 con 0x7f9640102570 2024-09-19T09:00:53.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:53.074+0000 7f9636ffd640 1 -- 172.21.15.120:0/839319716 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f963009b350 con 0x7f9640102570 2024-09-19T09:00:53.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:53.077+0000 7f96481f9640 1 -- 172.21.15.120:0/839319716 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f962c0a3240 msgr2=0x7f962c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:53.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:53.077+0000 7f96481f9640 1 --2- 172.21.15.120:0/839319716 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f962c0a3240 0x7f962c0a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f963c000c00 tx=0x7f963c0041e0 comp rx=0 tx=0).stop 2024-09-19T09:00:53.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:53.077+0000 7f96481f9640 1 -- 172.21.15.120:0/839319716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9640102570 msgr2=0x7f9640199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:53.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:53.077+0000 7f96481f9640 1 --2- 172.21.15.120:0/839319716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9640102570 0x7f9640199a30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f963000b4a0 tx=0x7f963000b960 comp rx=0 tx=0).stop 2024-09-19T09:00:53.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:53.077+0000 7f96481f9640 1 -- 172.21.15.120:0/839319716 shutdown_connections 2024-09-19T09:00:53.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:53.077+0000 7f96481f9640 1 --2- 172.21.15.120:0/839319716 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f962c0a3240 0x7f962c0a5700 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:53.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:53.077+0000 7f96481f9640 1 --2- 172.21.15.120:0/839319716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9640102ef0 0x7f96401a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:53.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:53.077+0000 7f96481f9640 1 --2- 172.21.15.120:0/839319716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9640102570 0x7f9640199a30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:53.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:53.077+0000 7f96481f9640 1 --2- 172.21.15.120:0/839319716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9640100e20 0x7f96401994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:53.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:53.077+0000 7f96481f9640 1 -- 172.21.15.120:0/839319716 >> 172.21.15.120:0/839319716 conn(0x7f96400fc7d0 msgr2=0x7f96400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:53.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:53.077+0000 7f96481f9640 1 -- 172.21.15.120:0/839319716 shutdown_connections 2024-09-19T09:00:53.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:53.077+0000 7f96481f9640 1 -- 172.21.15.120:0/839319716 wait complete. 2024-09-19T09:00:53.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:53 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/839319716' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T09:00:53.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:53 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/839319716' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T09:00:53.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:53 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/839319716' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T09:00:53.773 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-19T09:00:53.773 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-19T09:00:53.937 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 19 09:00:53 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-b[64524]: 2024-09-19T09:00:53.660+0000 7f1ecf160640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T09:00:54.018 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:00:54.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:54 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/839319716' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T09:00:54.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:54 smithi196 ceph-mon[35150]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T09:00:54.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:54 smithi196 ceph-mon[35150]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-19T09:00:54.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:54 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/839319716' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T09:00:54.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:54 smithi153 ceph-mon[35071]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T09:00:54.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:54 smithi153 ceph-mon[35071]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-19T09:00:54.456 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:54 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/839319716' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T09:00:54.456 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:54 smithi120 ceph-mon[31150]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T09:00:54.456 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:54 smithi120 ceph-mon[31150]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-19T09:00:54.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.538+0000 7fc48cf27640 1 -- 172.21.15.120:0/18145567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc488100a80 msgr2=0x7fc488100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:54.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.538+0000 7fc48cf27640 1 --2- 172.21.15.120:0/18145567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc488100a80 0x7fc488100ee0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009930 tx=0x7fc47c02a420 comp rx=0 tx=0).stop 2024-09-19T09:00:54.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.538+0000 7fc48cf27640 1 -- 172.21.15.120:0/18145567 shutdown_connections 2024-09-19T09:00:54.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.538+0000 7fc48cf27640 1 --2- 172.21.15.120:0/18145567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc488101420 0x7fc48810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:54.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.538+0000 7fc48cf27640 1 --2- 172.21.15.120:0/18145567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc488100a80 0x7fc488100ee0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:54.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.538+0000 7fc48cf27640 1 --2- 172.21.15.120:0/18145567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc488106a80 0x7fc488106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:54.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.538+0000 7fc48cf27640 1 -- 172.21.15.120:0/18145567 >> 172.21.15.120:0/18145567 conn(0x7fc4880fc7a0 msgr2=0x7fc4880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:54.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.538+0000 7fc48cf27640 1 -- 172.21.15.120:0/18145567 shutdown_connections 2024-09-19T09:00:54.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.538+0000 7fc48cf27640 1 -- 172.21.15.120:0/18145567 wait complete. 2024-09-19T09:00:54.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.538+0000 7fc48cf27640 1 Processor -- start 2024-09-19T09:00:54.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.539+0000 7fc48cf27640 1 -- start start 2024-09-19T09:00:54.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.539+0000 7fc48cf27640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc488100a80 0x7fc488198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:54.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.539+0000 7fc48cf27640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc488101420 0x7fc4881989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:54.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.539+0000 7fc48cf27640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc488106a80 0x7fc48819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:54.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.539+0000 7fc48cf27640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc488110e80 con 0x7fc488106a80 2024-09-19T09:00:54.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.539+0000 7fc48cf27640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc488110d00 con 0x7fc488101420 2024-09-19T09:00:54.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.539+0000 7fc48cf27640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc488111000 con 0x7fc488100a80 2024-09-19T09:00:54.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.539+0000 7fc486d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc488106a80 0x7fc48819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:54.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.539+0000 7fc486d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc488106a80 0x7fc48819cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47002/0 (socket says 172.21.15.120:47002) 2024-09-19T09:00:54.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.539+0000 7fc486d9d640 1 -- 172.21.15.120:0/2671128726 learned_addr learned my addr 172.21.15.120:0/2671128726 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:00:54.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.539+0000 7fc48659c640 1 --2- 172.21.15.120:0/2671128726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc488100a80 0x7fc488198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:54.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.539+0000 7fc485d9b640 1 --2- 172.21.15.120:0/2671128726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc488101420 0x7fc4881989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:54.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.539+0000 7fc486d9d640 1 -- 172.21.15.120:0/2671128726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc488100a80 msgr2=0x7fc488198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:54.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.539+0000 7fc486d9d640 1 --2- 172.21.15.120:0/2671128726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc488100a80 0x7fc488198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:54.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.539+0000 7fc486d9d640 1 -- 172.21.15.120:0/2671128726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc488101420 msgr2=0x7fc4881989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:54.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.539+0000 7fc486d9d640 1 --2- 172.21.15.120:0/2671128726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc488101420 0x7fc4881989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:54.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.539+0000 7fc486d9d640 1 -- 172.21.15.120:0/2671128726 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc48819d440 con 0x7fc488106a80 2024-09-19T09:00:54.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.539+0000 7fc486d9d640 1 --2- 172.21.15.120:0/2671128726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc488106a80 0x7fc48819cd60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc48000eaa0 tx=0x7fc48000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:54.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.539+0000 7fc4777fe640 1 -- 172.21.15.120:0/2671128726 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48000cdb0 con 0x7fc488106a80 2024-09-19T09:00:54.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.539+0000 7fc4777fe640 1 -- 172.21.15.120:0/2671128726 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc480005020 con 0x7fc488106a80 2024-09-19T09:00:54.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.539+0000 7fc48cf27640 1 -- 172.21.15.120:0/2671128726 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc48819d730 con 0x7fc488106a80 2024-09-19T09:00:54.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.539+0000 7fc4777fe640 1 -- 172.21.15.120:0/2671128726 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4800116b0 con 0x7fc488106a80 2024-09-19T09:00:54.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.540+0000 7fc48cf27640 1 -- 172.21.15.120:0/2671128726 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4881a5010 con 0x7fc488106a80 2024-09-19T09:00:54.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.540+0000 7fc48cf27640 1 -- 172.21.15.120:0/2671128726 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc488100ee0 con 0x7fc488106a80 2024-09-19T09:00:54.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.542+0000 7fc4777fe640 1 -- 172.21.15.120:0/2671128726 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fc480005560 con 0x7fc488106a80 2024-09-19T09:00:54.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.542+0000 7fc4777fe640 1 --2- 172.21.15.120:0/2671128726 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc4700a3240 0x7fc4700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:54.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.542+0000 7fc48659c640 1 --2- 172.21.15.120:0/2671128726 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc4700a3240 0x7fc4700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:54.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.542+0000 7fc4777fe640 1 -- 172.21.15.120:0/2671128726 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fc4800cf990 con 0x7fc488106a80 2024-09-19T09:00:54.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.543+0000 7fc48659c640 1 --2- 172.21.15.120:0/2671128726 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc4700a3240 0x7fc4700a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc478005fd0 tx=0x7fc47800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:54.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.546+0000 7fc4777fe640 1 -- 172.21.15.120:0/2671128726 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc480016030 con 0x7fc488106a80 2024-09-19T09:00:54.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:54.747+0000 7fc48cf27640 1 -- 172.21.15.120:0/2671128726 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fc488109a70 con 0x7fc488106a80 2024-09-19T09:00:55.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:55.093+0000 7fc4777fe640 1 -- 172.21.15.120:0/2671128726 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fc48009c1b0 con 0x7fc488106a80 2024-09-19T09:00:55.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:55.095+0000 7fc48cf27640 1 -- 172.21.15.120:0/2671128726 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc4700a3240 msgr2=0x7fc4700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:55.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:55.095+0000 7fc48cf27640 1 --2- 172.21.15.120:0/2671128726 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc4700a3240 0x7fc4700a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc478005fd0 tx=0x7fc47800b390 comp rx=0 tx=0).stop 2024-09-19T09:00:55.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:55.095+0000 7fc48cf27640 1 -- 172.21.15.120:0/2671128726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc488106a80 msgr2=0x7fc48819cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:55.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:55.095+0000 7fc48cf27640 1 --2- 172.21.15.120:0/2671128726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc488106a80 0x7fc48819cd60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc48000eaa0 tx=0x7fc48000ef60 comp rx=0 tx=0).stop 2024-09-19T09:00:55.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:55.096+0000 7fc48cf27640 1 -- 172.21.15.120:0/2671128726 shutdown_connections 2024-09-19T09:00:55.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:55.096+0000 7fc48cf27640 1 --2- 172.21.15.120:0/2671128726 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc4700a3240 0x7fc4700a5700 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:55.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:55.096+0000 7fc48cf27640 1 --2- 172.21.15.120:0/2671128726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc488106a80 0x7fc48819cd60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:55.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:55.096+0000 7fc48cf27640 1 --2- 172.21.15.120:0/2671128726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc488101420 0x7fc4881989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:55.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:55.096+0000 7fc48cf27640 1 --2- 172.21.15.120:0/2671128726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc488100a80 0x7fc488198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:55.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:55.096+0000 7fc48cf27640 1 -- 172.21.15.120:0/2671128726 >> 172.21.15.120:0/2671128726 conn(0x7fc4880fc7a0 msgr2=0x7fc48806aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:55.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:55.096+0000 7fc48cf27640 1 -- 172.21.15.120:0/2671128726 shutdown_connections 2024-09-19T09:00:55.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:55.096+0000 7fc48cf27640 1 -- 172.21.15.120:0/2671128726 wait complete. 2024-09-19T09:00:55.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:55 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2671128726' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T09:00:55.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:55 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2671128726' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T09:00:55.460 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:55 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2671128726' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T09:00:55.790 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-19T09:00:56.032 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:00:56.348 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:56 smithi120 ceph-mon[31150]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T09:00:56.349 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:56 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2671128726' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T09:00:56.349 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:56 smithi120 ceph-mon[31150]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T09:00:56.349 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:56 smithi120 ceph-mon[31150]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T09:00:56.349 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:56 smithi120 ceph-mon[31150]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-19T09:00:56.349 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:56 smithi120 ceph-mon[31150]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-19T09:00:56.349 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:56 smithi120 ceph-mon[31150]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-19T09:00:56.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:56 smithi196 ceph-mon[35150]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T09:00:56.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:56 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2671128726' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T09:00:56.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:56 smithi196 ceph-mon[35150]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T09:00:56.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:56 smithi196 ceph-mon[35150]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T09:00:56.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:56 smithi196 ceph-mon[35150]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-19T09:00:56.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:56 smithi196 ceph-mon[35150]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-19T09:00:56.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:56 smithi196 ceph-mon[35150]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-19T09:00:56.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:56 smithi153 ceph-mon[35071]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T09:00:56.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:56 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2671128726' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T09:00:56.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:56 smithi153 ceph-mon[35071]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T09:00:56.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:56 smithi153 ceph-mon[35071]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T09:00:56.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:56 smithi153 ceph-mon[35071]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-19T09:00:56.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:56 smithi153 ceph-mon[35071]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-19T09:00:56.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:56 smithi153 ceph-mon[35071]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-19T09:00:56.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.556+0000 7effec7c6640 1 -- 172.21.15.120:0/3105951964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe4102d40 msgr2=0x7effe4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:56.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.556+0000 7effec7c6640 1 --2- 172.21.15.120:0/3105951964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe4102d40 0x7effe4103120 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7effe0009900 tx=0x7effe002a480 comp rx=0 tx=0).stop 2024-09-19T09:00:56.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.557+0000 7effec7c6640 1 -- 172.21.15.120:0/3105951964 shutdown_connections 2024-09-19T09:00:56.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.557+0000 7effec7c6640 1 --2- 172.21.15.120:0/3105951964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effe410c170 0x7effe410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:56.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.557+0000 7effec7c6640 1 --2- 172.21.15.120:0/3105951964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effe4103660 0x7effe410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:56.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.557+0000 7effec7c6640 1 --2- 172.21.15.120:0/3105951964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe4102d40 0x7effe4103120 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:56.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.557+0000 7effec7c6640 1 -- 172.21.15.120:0/3105951964 >> 172.21.15.120:0/3105951964 conn(0x7effe40fc7a0 msgr2=0x7effe40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:56.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.557+0000 7effec7c6640 1 -- 172.21.15.120:0/3105951964 shutdown_connections 2024-09-19T09:00:56.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.557+0000 7effec7c6640 1 -- 172.21.15.120:0/3105951964 wait complete. 2024-09-19T09:00:56.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.557+0000 7effec7c6640 1 Processor -- start 2024-09-19T09:00:56.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.557+0000 7effec7c6640 1 -- start start 2024-09-19T09:00:56.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.557+0000 7effec7c6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe4102d40 0x7effe4198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:56.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.557+0000 7effec7c6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effe4103660 0x7effe41989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:56.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.557+0000 7effec7c6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effe410c170 0x7effe419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:56.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.557+0000 7effec7c6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7effe4111fb0 con 0x7effe4102d40 2024-09-19T09:00:56.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.557+0000 7effec7c6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7effe4111e30 con 0x7effe4103660 2024-09-19T09:00:56.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.557+0000 7effec7c6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7effe4112130 con 0x7effe410c170 2024-09-19T09:00:56.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.558+0000 7effea562640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe4102d40 0x7effe4198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:56.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.558+0000 7effea562640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe4102d40 0x7effe4198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47022/0 (socket says 172.21.15.120:47022) 2024-09-19T09:00:56.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.558+0000 7effea562640 1 -- 172.21.15.120:0/1985269222 learned_addr learned my addr 172.21.15.120:0/1985269222 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:00:56.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.558+0000 7effead63640 1 --2- 172.21.15.120:0/1985269222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effe410c170 0x7effe419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:56.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.558+0000 7effe9d61640 1 --2- 172.21.15.120:0/1985269222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effe4103660 0x7effe41989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:56.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.558+0000 7effea562640 1 -- 172.21.15.120:0/1985269222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effe410c170 msgr2=0x7effe419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:56.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.558+0000 7effea562640 1 --2- 172.21.15.120:0/1985269222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effe410c170 0x7effe419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:56.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.558+0000 7effea562640 1 -- 172.21.15.120:0/1985269222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effe4103660 msgr2=0x7effe41989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:56.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.558+0000 7effea562640 1 --2- 172.21.15.120:0/1985269222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effe4103660 0x7effe41989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:56.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.558+0000 7effea562640 1 -- 172.21.15.120:0/1985269222 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe419d420 con 0x7effe4102d40 2024-09-19T09:00:56.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.558+0000 7effead63640 1 --2- 172.21.15.120:0/1985269222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effe410c170 0x7effe419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:00:56.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.558+0000 7effea562640 1 --2- 172.21.15.120:0/1985269222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe4102d40 0x7effe4198470 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7effe0006010 tx=0x7effe00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:56.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.558+0000 7effdb7fe640 1 -- 172.21.15.120:0/1985269222 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe003f030 con 0x7effe4102d40 2024-09-19T09:00:56.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.558+0000 7effdb7fe640 1 -- 172.21.15.120:0/1985269222 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effe0003ce0 con 0x7effe4102d40 2024-09-19T09:00:56.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.558+0000 7effdb7fe640 1 -- 172.21.15.120:0/1985269222 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe0035070 con 0x7effe4102d40 2024-09-19T09:00:56.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.558+0000 7effec7c6640 1 -- 172.21.15.120:0/1985269222 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effe419d6b0 con 0x7effe4102d40 2024-09-19T09:00:56.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.558+0000 7effec7c6640 1 -- 172.21.15.120:0/1985269222 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effe41a4ff0 con 0x7effe4102d40 2024-09-19T09:00:56.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.559+0000 7effec7c6640 1 -- 172.21.15.120:0/1985269222 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effe4103120 con 0x7effe4102d40 2024-09-19T09:00:56.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.560+0000 7effdb7fe640 1 -- 172.21.15.120:0/1985269222 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7effe0042030 con 0x7effe4102d40 2024-09-19T09:00:56.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.561+0000 7effdb7fe640 1 --2- 172.21.15.120:0/1985269222 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7effd00a32c0 0x7effd00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:56.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.561+0000 7effe9d61640 1 --2- 172.21.15.120:0/1985269222 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7effd00a32c0 0x7effd00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:56.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.561+0000 7effdb7fe640 1 -- 172.21.15.120:0/1985269222 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7effe00ef960 con 0x7effe4102d40 2024-09-19T09:00:56.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.562+0000 7effe9d61640 1 --2- 172.21.15.120:0/1985269222 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7effd00a32c0 0x7effd00a5780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7effd40058c0 tx=0x7effd4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:56.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.565+0000 7effdb7fe640 1 -- 172.21.15.120:0/1985269222 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7effe00bc890 con 0x7effe4102d40 2024-09-19T09:00:56.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:56.767+0000 7effec7c6640 1 -- 172.21.15.120:0/1985269222 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7effe4107210 con 0x7effe4102d40 2024-09-19T09:00:57.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:57.110+0000 7effdb7fe640 1 -- 172.21.15.120:0/1985269222 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7effe00bc190 con 0x7effe4102d40 2024-09-19T09:00:57.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:57.112+0000 7effec7c6640 1 -- 172.21.15.120:0/1985269222 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7effd00a32c0 msgr2=0x7effd00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:57.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:57.112+0000 7effec7c6640 1 --2- 172.21.15.120:0/1985269222 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7effd00a32c0 0x7effd00a5780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7effd40058c0 tx=0x7effd4005850 comp rx=0 tx=0).stop 2024-09-19T09:00:57.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:57.112+0000 7effec7c6640 1 -- 172.21.15.120:0/1985269222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe4102d40 msgr2=0x7effe4198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:57.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:57.112+0000 7effec7c6640 1 --2- 172.21.15.120:0/1985269222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe4102d40 0x7effe4198470 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7effe0006010 tx=0x7effe00042f0 comp rx=0 tx=0).stop 2024-09-19T09:00:57.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:57.112+0000 7effec7c6640 1 -- 172.21.15.120:0/1985269222 shutdown_connections 2024-09-19T09:00:57.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:57.112+0000 7effec7c6640 1 --2- 172.21.15.120:0/1985269222 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7effd00a32c0 0x7effd00a5780 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:57.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:57.112+0000 7effec7c6640 1 --2- 172.21.15.120:0/1985269222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effe410c170 0x7effe419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:57.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:57.112+0000 7effec7c6640 1 --2- 172.21.15.120:0/1985269222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effe4103660 0x7effe41989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:57.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:57.112+0000 7effec7c6640 1 --2- 172.21.15.120:0/1985269222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe4102d40 0x7effe4198470 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:57.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:57.112+0000 7effec7c6640 1 -- 172.21.15.120:0/1985269222 >> 172.21.15.120:0/1985269222 conn(0x7effe40fc7a0 msgr2=0x7effe40fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:57.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:57.112+0000 7effec7c6640 1 -- 172.21.15.120:0/1985269222 shutdown_connections 2024-09-19T09:00:57.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:57.112+0000 7effec7c6640 1 -- 172.21.15.120:0/1985269222 wait complete. 2024-09-19T09:00:57.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:57 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] up:active 2024-09-19T09:00:57.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:57 smithi196 ceph-mon[35150]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T09:00:57.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:57 smithi196 ceph-mon[35150]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-19T09:00:57.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:57 smithi196 ceph-mon[35150]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-19T09:00:57.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:57 smithi196 ceph-mon[35150]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-19T09:00:57.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:57 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1985269222' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T09:00:57.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:57 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] up:active 2024-09-19T09:00:57.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:57 smithi153 ceph-mon[35071]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T09:00:57.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:57 smithi153 ceph-mon[35071]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-19T09:00:57.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:57 smithi153 ceph-mon[35071]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-19T09:00:57.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:57 smithi153 ceph-mon[35071]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-19T09:00:57.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:57 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1985269222' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T09:00:57.488 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:57 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] up:active 2024-09-19T09:00:57.488 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:57 smithi120 ceph-mon[31150]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T09:00:57.488 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:57 smithi120 ceph-mon[31150]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-19T09:00:57.488 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:57 smithi120 ceph-mon[31150]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-19T09:00:57.488 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:57 smithi120 ceph-mon[31150]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-19T09:00:57.489 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:57 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1985269222' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T09:00:57.857 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-19T09:00:58.100 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:00:58.366 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:58 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1985269222' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T09:00:58.366 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:58 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] up:active 2024-09-19T09:00:58.366 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:58 smithi120 ceph-mon[31150]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T09:00:58.366 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:58 smithi120 ceph-mon[31150]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-19T09:00:58.366 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:58 smithi120 ceph-mon[31150]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-19T09:00:58.366 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:58 smithi120 ceph-mon[31150]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-19T09:00:58.366 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:58 smithi120 ceph-mon[31150]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 6 op/s 2024-09-19T09:00:58.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:58 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1985269222' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T09:00:58.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:58 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] up:active 2024-09-19T09:00:58.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:58 smithi196 ceph-mon[35150]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T09:00:58.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:58 smithi196 ceph-mon[35150]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-19T09:00:58.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:58 smithi196 ceph-mon[35150]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-19T09:00:58.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:58 smithi196 ceph-mon[35150]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-19T09:00:58.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:58 smithi196 ceph-mon[35150]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 6 op/s 2024-09-19T09:00:58.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:58 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1985269222' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T09:00:58.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:58 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] up:active 2024-09-19T09:00:58.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:58 smithi153 ceph-mon[35071]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T09:00:58.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:58 smithi153 ceph-mon[35071]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-19T09:00:58.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:58 smithi153 ceph-mon[35071]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-19T09:00:58.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:58 smithi153 ceph-mon[35071]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-19T09:00:58.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:58 smithi153 ceph-mon[35071]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 6 op/s 2024-09-19T09:00:58.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.608+0000 7fab91dc7640 1 -- 172.21.15.120:0/1454793041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab8c102d40 msgr2=0x7fab8c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:58.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.608+0000 7fab91dc7640 1 --2- 172.21.15.120:0/1454793041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab8c102d40 0x7fab8c103120 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009900 tx=0x7fab7c02a480 comp rx=0 tx=0).stop 2024-09-19T09:00:58.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.608+0000 7fab91dc7640 1 -- 172.21.15.120:0/1454793041 shutdown_connections 2024-09-19T09:00:58.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.608+0000 7fab91dc7640 1 --2- 172.21.15.120:0/1454793041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8c10c170 0x7fab8c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:58.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.608+0000 7fab91dc7640 1 --2- 172.21.15.120:0/1454793041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab8c103660 0x7fab8c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:58.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.608+0000 7fab91dc7640 1 --2- 172.21.15.120:0/1454793041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab8c102d40 0x7fab8c103120 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:58.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.608+0000 7fab91dc7640 1 -- 172.21.15.120:0/1454793041 >> 172.21.15.120:0/1454793041 conn(0x7fab8c0fc7a0 msgr2=0x7fab8c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:58.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.608+0000 7fab91dc7640 1 -- 172.21.15.120:0/1454793041 shutdown_connections 2024-09-19T09:00:58.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.608+0000 7fab91dc7640 1 -- 172.21.15.120:0/1454793041 wait complete. 2024-09-19T09:00:58.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.609+0000 7fab91dc7640 1 Processor -- start 2024-09-19T09:00:58.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.609+0000 7fab91dc7640 1 -- start start 2024-09-19T09:00:58.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.609+0000 7fab91dc7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8c102d40 0x7fab8c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:58.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.609+0000 7fab91dc7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab8c103660 0x7fab8c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:58.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.609+0000 7fab91dc7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab8c10c170 0x7fab8c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:58.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.609+0000 7fab91dc7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c111fb0 con 0x7fab8c10c170 2024-09-19T09:00:58.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.609+0000 7fab91dc7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c111e30 con 0x7fab8c103660 2024-09-19T09:00:58.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.609+0000 7fab91dc7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c112130 con 0x7fab8c102d40 2024-09-19T09:00:58.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.609+0000 7fab8bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab8c10c170 0x7fab8c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:58.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.609+0000 7fab8bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab8c10c170 0x7fab8c19cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47034/0 (socket says 172.21.15.120:47034) 2024-09-19T09:00:58.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.609+0000 7fab8bfff640 1 -- 172.21.15.120:0/4210542292 learned_addr learned my addr 172.21.15.120:0/4210542292 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:00:58.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.609+0000 7fab8b7fe640 1 --2- 172.21.15.120:0/4210542292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8c102d40 0x7fab8c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:58.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.609+0000 7fab8affd640 1 --2- 172.21.15.120:0/4210542292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab8c103660 0x7fab8c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:58.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.609+0000 7fab8bfff640 1 -- 172.21.15.120:0/4210542292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8c102d40 msgr2=0x7fab8c198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:58.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.610+0000 7fab8bfff640 1 --2- 172.21.15.120:0/4210542292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8c102d40 0x7fab8c198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:58.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.610+0000 7fab8bfff640 1 -- 172.21.15.120:0/4210542292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab8c103660 msgr2=0x7fab8c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:58.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.610+0000 7fab8bfff640 1 --2- 172.21.15.120:0/4210542292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab8c103660 0x7fab8c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:58.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.610+0000 7fab8bfff640 1 -- 172.21.15.120:0/4210542292 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8c19d420 con 0x7fab8c10c170 2024-09-19T09:00:58.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.610+0000 7fab8affd640 1 --2- 172.21.15.120:0/4210542292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab8c103660 0x7fab8c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:00:58.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.610+0000 7fab8bfff640 1 --2- 172.21.15.120:0/4210542292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab8c10c170 0x7fab8c19cd40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fab8400b970 tx=0x7fab8400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:58.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.610+0000 7fab88ff9640 1 -- 172.21.15.120:0/4210542292 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84019070 con 0x7fab8c10c170 2024-09-19T09:00:58.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.610+0000 7fab88ff9640 1 -- 172.21.15.120:0/4210542292 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab8400cb40 con 0x7fab8c10c170 2024-09-19T09:00:58.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.610+0000 7fab91dc7640 1 -- 172.21.15.120:0/4210542292 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab8c19d710 con 0x7fab8c10c170 2024-09-19T09:00:58.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.610+0000 7fab88ff9640 1 -- 172.21.15.120:0/4210542292 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84007650 con 0x7fab8c10c170 2024-09-19T09:00:58.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.610+0000 7fab91dc7640 1 -- 172.21.15.120:0/4210542292 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab8c1a4ff0 con 0x7fab8c10c170 2024-09-19T09:00:58.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.611+0000 7fab91dc7640 1 -- 172.21.15.120:0/4210542292 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab8c103120 con 0x7fab8c10c170 2024-09-19T09:00:58.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.612+0000 7fab88ff9640 1 -- 172.21.15.120:0/4210542292 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fab8401c070 con 0x7fab8c10c170 2024-09-19T09:00:58.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.613+0000 7fab88ff9640 1 --2- 172.21.15.120:0/4210542292 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fab740a3240 0x7fab740a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:00:58.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.613+0000 7fab8b7fe640 1 --2- 172.21.15.120:0/4210542292 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fab740a3240 0x7fab740a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:00:58.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.613+0000 7fab88ff9640 1 -- 172.21.15.120:0/4210542292 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fab840cedd0 con 0x7fab8c10c170 2024-09-19T09:00:58.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.613+0000 7fab8b7fe640 1 --2- 172.21.15.120:0/4210542292 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fab740a3240 0x7fab740a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fab7c0096c0 tx=0x7fab7c031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:00:58.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.617+0000 7fab88ff9640 1 -- 172.21.15.120:0/4210542292 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fab8409bc40 con 0x7fab8c10c170 2024-09-19T09:00:58.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:58.818+0000 7fab91dc7640 1 -- 172.21.15.120:0/4210542292 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fab8c106c80 con 0x7fab8c10c170 2024-09-19T09:00:58.937 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 19 09:00:58 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-b[64524]: 2024-09-19T09:00:58.661+0000 7f1ecf160640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-19T09:00:58.937 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 19 09:00:58 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-b[64524]: 2024-09-19T09:00:58.661+0000 7f1ecf160640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-19T09:00:58.937 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 19 09:00:58 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-b[64524]: 2024-09-19T09:00:58.661+0000 7f1ecf160640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-19T09:00:59.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:59.130+0000 7fab88ff9640 1 -- 172.21.15.120:0/4210542292 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7fab8409b540 con 0x7fab8c10c170 2024-09-19T09:00:59.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:59.133+0000 7fab91dc7640 1 -- 172.21.15.120:0/4210542292 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fab740a3240 msgr2=0x7fab740a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:59.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:59.133+0000 7fab91dc7640 1 --2- 172.21.15.120:0/4210542292 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fab740a3240 0x7fab740a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fab7c0096c0 tx=0x7fab7c031490 comp rx=0 tx=0).stop 2024-09-19T09:00:59.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:59.133+0000 7fab91dc7640 1 -- 172.21.15.120:0/4210542292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab8c10c170 msgr2=0x7fab8c19cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:00:59.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:59.133+0000 7fab91dc7640 1 --2- 172.21.15.120:0/4210542292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab8c10c170 0x7fab8c19cd40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fab8400b970 tx=0x7fab8400be30 comp rx=0 tx=0).stop 2024-09-19T09:00:59.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:59.133+0000 7fab91dc7640 1 -- 172.21.15.120:0/4210542292 shutdown_connections 2024-09-19T09:00:59.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:59.133+0000 7fab91dc7640 1 --2- 172.21.15.120:0/4210542292 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fab740a3240 0x7fab740a5700 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:59.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:59.133+0000 7fab91dc7640 1 --2- 172.21.15.120:0/4210542292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab8c10c170 0x7fab8c19cd40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:59.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:59.133+0000 7fab91dc7640 1 --2- 172.21.15.120:0/4210542292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab8c103660 0x7fab8c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:59.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:59.133+0000 7fab91dc7640 1 --2- 172.21.15.120:0/4210542292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8c102d40 0x7fab8c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:00:59.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:59.133+0000 7fab91dc7640 1 -- 172.21.15.120:0/4210542292 >> 172.21.15.120:0/4210542292 conn(0x7fab8c0fc7a0 msgr2=0x7fab8c0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:00:59.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:59.133+0000 7fab91dc7640 1 -- 172.21.15.120:0/4210542292 shutdown_connections 2024-09-19T09:00:59.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:00:59.133+0000 7fab91dc7640 1 -- 172.21.15.120:0/4210542292 wait complete. 2024-09-19T09:00:59.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:59 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] up:active 2024-09-19T09:00:59.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:59 smithi196 ceph-mon[35150]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T09:00:59.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:59 smithi196 ceph-mon[35150]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T09:00:59.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:59 smithi196 ceph-mon[35150]: Cluster is now healthy 2024-09-19T09:00:59.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:59 smithi196 ceph-mon[35150]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-09-19T09:00:59.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:59 smithi196 ceph-mon[35150]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T09:00:59.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:59 smithi196 ceph-mon[35150]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-19T09:00:59.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:00:59 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/4210542292' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T09:00:59.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:59 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] up:active 2024-09-19T09:00:59.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:59 smithi153 ceph-mon[35071]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T09:00:59.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:59 smithi153 ceph-mon[35071]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T09:00:59.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:59 smithi153 ceph-mon[35071]: Cluster is now healthy 2024-09-19T09:00:59.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:59 smithi153 ceph-mon[35071]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-09-19T09:00:59.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:59 smithi153 ceph-mon[35071]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T09:00:59.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:59 smithi153 ceph-mon[35071]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-19T09:00:59.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:00:59 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/4210542292' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T09:00:59.505 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:59 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] up:active 2024-09-19T09:00:59.505 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:59 smithi120 ceph-mon[31150]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T09:00:59.505 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:59 smithi120 ceph-mon[31150]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T09:00:59.505 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:59 smithi120 ceph-mon[31150]: Cluster is now healthy 2024-09-19T09:00:59.505 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:59 smithi120 ceph-mon[31150]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-09-19T09:00:59.505 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:59 smithi120 ceph-mon[31150]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T09:00:59.506 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:59 smithi120 ceph-mon[31150]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-19T09:00:59.506 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:00:59 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/4210542292' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T09:00:59.843 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-09-19T09:00:59.843 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-19T09:01:00.084 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:00.381 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:00 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/4210542292' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T09:01:00.382 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:00 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] up:active 2024-09-19T09:01:00.382 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:00 smithi120 ceph-mon[31150]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-19T09:01:00.382 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:00 smithi120 ceph-mon[31150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:01:00.382 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:00 smithi120 ceph-mon[31150]: pgmap v283: 97 pgs: 97 active+clean; 583 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s wr, 10 op/s 2024-09-19T09:01:00.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:00 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/4210542292' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T09:01:00.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:00 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] up:active 2024-09-19T09:01:00.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:00 smithi196 ceph-mon[35150]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-19T09:01:00.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:00 smithi196 ceph-mon[35150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:01:00.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:00 smithi196 ceph-mon[35150]: pgmap v283: 97 pgs: 97 active+clean; 583 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s wr, 10 op/s 2024-09-19T09:01:00.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:00 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/4210542292' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T09:01:00.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:00 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] up:active 2024-09-19T09:01:00.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:00 smithi153 ceph-mon[35071]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-19T09:01:00.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:00 smithi153 ceph-mon[35071]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:01:00.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:00 smithi153 ceph-mon[35071]: pgmap v283: 97 pgs: 97 active+clean; 583 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s wr, 10 op/s 2024-09-19T09:01:00.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.602+0000 7f9957c64640 1 -- 172.21.15.120:0/3534218284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9950101de0 msgr2=0x7f99501021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:00.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.602+0000 7f9957c64640 1 --2- 172.21.15.120:0/3534218284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9950101de0 0x7f99501021c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f994c009900 tx=0x7f994c02a410 comp rx=0 tx=0).stop 2024-09-19T09:01:00.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.602+0000 7f9957c64640 1 -- 172.21.15.120:0/3534218284 shutdown_connections 2024-09-19T09:01:00.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.602+0000 7f9957c64640 1 --2- 172.21.15.120:0/3534218284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9950106f10 0x7f995010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:00.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.602+0000 7f9957c64640 1 --2- 172.21.15.120:0/3534218284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9950102790 0x7f99501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:00.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.602+0000 7f9957c64640 1 --2- 172.21.15.120:0/3534218284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9950101de0 0x7f99501021c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:00.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.602+0000 7f9957c64640 1 -- 172.21.15.120:0/3534218284 >> 172.21.15.120:0/3534218284 conn(0x7f99500fd8c0 msgr2=0x7f99500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:00.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.602+0000 7f9957c64640 1 -- 172.21.15.120:0/3534218284 shutdown_connections 2024-09-19T09:01:00.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.602+0000 7f9957c64640 1 -- 172.21.15.120:0/3534218284 wait complete. 2024-09-19T09:01:00.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.603+0000 7f9957c64640 1 Processor -- start 2024-09-19T09:01:00.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.603+0000 7f9957c64640 1 -- start start 2024-09-19T09:01:00.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.603+0000 7f9957c64640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9950101de0 0x7f9950198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:00.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.603+0000 7f9957c64640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9950102790 0x7f9950198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:00.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.603+0000 7f9957c64640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9950106f10 0x7f995019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:00.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.603+0000 7f9957c64640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9950111f70 con 0x7f9950101de0 2024-09-19T09:01:00.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.603+0000 7f9957c64640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9950111df0 con 0x7f9950106f10 2024-09-19T09:01:00.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.603+0000 7f9957c64640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f99501120f0 con 0x7f9950102790 2024-09-19T09:01:00.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.603+0000 7f9955a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9950101de0 0x7f9950198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:00.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.603+0000 7f9955a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9950101de0 0x7f9950198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47046/0 (socket says 172.21.15.120:47046) 2024-09-19T09:01:00.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.603+0000 7f9955a00640 1 -- 172.21.15.120:0/4080562950 learned_addr learned my addr 172.21.15.120:0/4080562950 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:00.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.603+0000 7f9956201640 1 --2- 172.21.15.120:0/4080562950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9950106f10 0x7f995019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:00.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.603+0000 7f99551ff640 1 --2- 172.21.15.120:0/4080562950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9950102790 0x7f9950198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:00.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.604+0000 7f9955a00640 1 -- 172.21.15.120:0/4080562950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9950102790 msgr2=0x7f9950198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:00.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.604+0000 7f9955a00640 1 --2- 172.21.15.120:0/4080562950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9950102790 0x7f9950198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:00.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.604+0000 7f9955a00640 1 -- 172.21.15.120:0/4080562950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9950106f10 msgr2=0x7f995019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:00.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.604+0000 7f9955a00640 1 --2- 172.21.15.120:0/4080562950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9950106f10 0x7f995019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:00.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.604+0000 7f9955a00640 1 -- 172.21.15.120:0/4080562950 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f995019d3c0 con 0x7f9950101de0 2024-09-19T09:01:00.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.604+0000 7f99551ff640 1 --2- 172.21.15.120:0/4080562950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9950102790 0x7f9950198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:01:00.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.604+0000 7f9955a00640 1 --2- 172.21.15.120:0/4080562950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9950101de0 0x7f9950198410 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f994c006010 tx=0x7f994c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:00.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.604+0000 7f9942ffd640 1 -- 172.21.15.120:0/4080562950 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c03f030 con 0x7f9950101de0 2024-09-19T09:01:00.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.604+0000 7f9957c64640 1 -- 172.21.15.120:0/4080562950 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f995019d650 con 0x7f9950101de0 2024-09-19T09:01:00.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.604+0000 7f9942ffd640 1 -- 172.21.15.120:0/4080562950 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f994c003ce0 con 0x7f9950101de0 2024-09-19T09:01:00.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.604+0000 7f9942ffd640 1 -- 172.21.15.120:0/4080562950 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c035070 con 0x7f9950101de0 2024-09-19T09:01:00.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.604+0000 7f9957c64640 1 -- 172.21.15.120:0/4080562950 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99501a4f90 con 0x7f9950101de0 2024-09-19T09:01:00.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.605+0000 7f9957c64640 1 -- 172.21.15.120:0/4080562950 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99501032c0 con 0x7f9950101de0 2024-09-19T09:01:00.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.606+0000 7f9942ffd640 1 -- 172.21.15.120:0/4080562950 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f994c042030 con 0x7f9950101de0 2024-09-19T09:01:00.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.607+0000 7f9942ffd640 1 --2- 172.21.15.120:0/4080562950 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f993c0a3240 0x7f993c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:00.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.607+0000 7f99551ff640 1 --2- 172.21.15.120:0/4080562950 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f993c0a3240 0x7f993c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:00.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.607+0000 7f9942ffd640 1 -- 172.21.15.120:0/4080562950 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f994c0ef8e0 con 0x7f9950101de0 2024-09-19T09:01:00.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.607+0000 7f99551ff640 1 --2- 172.21.15.120:0/4080562950 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f993c0a3240 0x7f993c0a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f99500fdd50 tx=0x7f9944005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:00.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.611+0000 7f9942ffd640 1 -- 172.21.15.120:0/4080562950 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f994c0bc810 con 0x7f9950101de0 2024-09-19T09:01:00.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.799+0000 7f9957c64640 1 -- 172.21.15.120:0/4080562950 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9950062630 con 0x7f993c0a3240 2024-09-19T09:01:00.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.821+0000 7f9942ffd640 1 -- 172.21.15.120:0/4080562950 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9950062630 con 0x7f993c0a3240 2024-09-19T09:01:00.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.823+0000 7f9957c64640 1 -- 172.21.15.120:0/4080562950 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f993c0a3240 msgr2=0x7f993c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:00.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.823+0000 7f9957c64640 1 --2- 172.21.15.120:0/4080562950 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f993c0a3240 0x7f993c0a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f99500fdd50 tx=0x7f9944005f90 comp rx=0 tx=0).stop 2024-09-19T09:01:00.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.823+0000 7f9957c64640 1 -- 172.21.15.120:0/4080562950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9950101de0 msgr2=0x7f9950198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:00.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.823+0000 7f9957c64640 1 --2- 172.21.15.120:0/4080562950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9950101de0 0x7f9950198410 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f994c006010 tx=0x7f994c0042f0 comp rx=0 tx=0).stop 2024-09-19T09:01:00.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.823+0000 7f9957c64640 1 -- 172.21.15.120:0/4080562950 shutdown_connections 2024-09-19T09:01:00.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.823+0000 7f9957c64640 1 --2- 172.21.15.120:0/4080562950 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f993c0a3240 0x7f993c0a5700 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:00.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.823+0000 7f9957c64640 1 --2- 172.21.15.120:0/4080562950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9950106f10 0x7f995019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:00.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.823+0000 7f9957c64640 1 --2- 172.21.15.120:0/4080562950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9950102790 0x7f9950198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:00.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.823+0000 7f9957c64640 1 --2- 172.21.15.120:0/4080562950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9950101de0 0x7f9950198410 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:00.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.823+0000 7f9957c64640 1 -- 172.21.15.120:0/4080562950 >> 172.21.15.120:0/4080562950 conn(0x7f99500fd8c0 msgr2=0x7f99500fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:00.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.823+0000 7f9957c64640 1 -- 172.21.15.120:0/4080562950 shutdown_connections 2024-09-19T09:01:00.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:00.823+0000 7f9957c64640 1 -- 172.21.15.120:0/4080562950 wait complete. 2024-09-19T09:01:01.074 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 09:01:00 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T09:01:00.805+0000 7f69457f8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T09:01:01.075 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 09:01:00 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T09:01:00.805+0000 7f69457f8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T09:01:01.075 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 09:01:00 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T09:01:00.805+0000 7f69457f8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T09:01:01.075 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 09:01:00 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T09:01:00.805+0000 7f69457f8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T09:01:01.075 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 09:01:00 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T09:01:00.805+0000 7f69457f8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T09:01:01.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:01 smithi196 ceph-mon[35150]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:01:01.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:01 smithi153 ceph-mon[35071]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:01:01.524 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:01 smithi120 ceph-mon[31150]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:01:01.554 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-19T09:01:01.800 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:02.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:02 smithi196 ceph-mon[35150]: pgmap v284: 97 pgs: 97 active+clean; 591 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 5.5 KiB/s wr, 16 op/s 2024-09-19T09:01:02.255 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:02 smithi120 ceph-mon[31150]: pgmap v284: 97 pgs: 97 active+clean; 591 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 5.5 KiB/s wr, 16 op/s 2024-09-19T09:01:02.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.377+0000 7fee0ce57640 1 -- 172.21.15.120:0/12534159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee0810c170 msgr2=0x7fee0810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.377+0000 7fee0ce57640 1 --2- 172.21.15.120:0/12534159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee0810c170 0x7fee0810e610 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fee00009ca0 tx=0x7fee0002a950 comp rx=0 tx=0).stop 2024-09-19T09:01:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.377+0000 7fee0ce57640 1 -- 172.21.15.120:0/12534159 shutdown_connections 2024-09-19T09:01:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.377+0000 7fee0ce57640 1 --2- 172.21.15.120:0/12534159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee0810c170 0x7fee0810e610 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.377+0000 7fee0ce57640 1 --2- 172.21.15.120:0/12534159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee08103660 0x7fee0810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.377+0000 7fee0ce57640 1 --2- 172.21.15.120:0/12534159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee08102d40 0x7fee08103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.377+0000 7fee0ce57640 1 -- 172.21.15.120:0/12534159 >> 172.21.15.120:0/12534159 conn(0x7fee080fc7a0 msgr2=0x7fee080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.377+0000 7fee0ce57640 1 -- 172.21.15.120:0/12534159 shutdown_connections 2024-09-19T09:01:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.377+0000 7fee0ce57640 1 -- 172.21.15.120:0/12534159 wait complete. 2024-09-19T09:01:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.378+0000 7fee0ce57640 1 Processor -- start 2024-09-19T09:01:02.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.378+0000 7fee0ce57640 1 -- start start 2024-09-19T09:01:02.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.378+0000 7fee0ce57640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee08102d40 0x7fee081983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:02.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.379+0000 7fee0ce57640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee08103660 0x7fee081988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:02.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.379+0000 7fee0ce57640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee0810c170 0x7fee0819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:02.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.379+0000 7fee0ce57640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fee08111f30 con 0x7fee0810c170 2024-09-19T09:01:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.379+0000 7fee0ce57640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fee08111db0 con 0x7fee08103660 2024-09-19T09:01:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.379+0000 7fee0ce57640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fee081120b0 con 0x7fee08102d40 2024-09-19T09:01:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.379+0000 7fee06d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee0810c170 0x7fee0819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.379+0000 7fee0659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee08102d40 0x7fee081983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.379+0000 7fee06d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee0810c170 0x7fee0819cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34116/0 (socket says 172.21.15.120:34116) 2024-09-19T09:01:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.379+0000 7fee06d9d640 1 -- 172.21.15.120:0/3208412425 learned_addr learned my addr 172.21.15.120:0/3208412425 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.379+0000 7fedfdd9b640 1 --2- 172.21.15.120:0/3208412425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee08103660 0x7fee081988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.379+0000 7fee0659c640 1 -- 172.21.15.120:0/3208412425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee08103660 msgr2=0x7fee081988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.379+0000 7fee0659c640 1 --2- 172.21.15.120:0/3208412425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee08103660 0x7fee081988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.379+0000 7fee0659c640 1 -- 172.21.15.120:0/3208412425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee0810c170 msgr2=0x7fee0819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.379+0000 7fee0659c640 1 --2- 172.21.15.120:0/3208412425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee0810c170 0x7fee0819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.379+0000 7fee0659c640 1 -- 172.21.15.120:0/3208412425 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0819d360 con 0x7fee08102d40 2024-09-19T09:01:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.379+0000 7fee06d9d640 1 --2- 172.21.15.120:0/3208412425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee0810c170 0x7fee0819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:01:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.379+0000 7fee0659c640 1 --2- 172.21.15.120:0/3208412425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee08102d40 0x7fee081983b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fedf8005fd0 tx=0x7fedf800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.379+0000 7fedfffff640 1 -- 172.21.15.120:0/3208412425 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8009c30 con 0x7fee08102d40 2024-09-19T09:01:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.379+0000 7fee0ce57640 1 -- 172.21.15.120:0/3208412425 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee0819d650 con 0x7fee08102d40 2024-09-19T09:01:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.379+0000 7fedfffff640 1 -- 172.21.15.120:0/3208412425 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedf8009dd0 con 0x7fee08102d40 2024-09-19T09:01:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.380+0000 7fedfffff640 1 -- 172.21.15.120:0/3208412425 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8010460 con 0x7fee08102d40 2024-09-19T09:01:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.380+0000 7fee0ce57640 1 -- 172.21.15.120:0/3208412425 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee081a4f30 con 0x7fee08102d40 2024-09-19T09:01:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.381+0000 7fee0ce57640 1 -- 172.21.15.120:0/3208412425 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee08103120 con 0x7fee08102d40 2024-09-19T09:01:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.382+0000 7fedfffff640 1 -- 172.21.15.120:0/3208412425 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fedf8003f50 con 0x7fee08102d40 2024-09-19T09:01:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.382+0000 7fedfffff640 1 --2- 172.21.15.120:0/3208412425 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fedf40a32d0 0x7fedf40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.382+0000 7fedfdd9b640 1 --2- 172.21.15.120:0/3208412425 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fedf40a32d0 0x7fedf40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.382+0000 7fedfffff640 1 -- 172.21.15.120:0/3208412425 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fedf80cf8b0 con 0x7fee08102d40 2024-09-19T09:01:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.383+0000 7fedfdd9b640 1 --2- 172.21.15.120:0/3208412425 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fedf40a32d0 0x7fedf40a5790 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fee0810d490 tx=0x7fedf000b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:02.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.387+0000 7fedfffff640 1 -- 172.21.15.120:0/3208412425 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fedf8017020 con 0x7fee08102d40 2024-09-19T09:01:02.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:02 smithi153 ceph-mon[35071]: pgmap v284: 97 pgs: 97 active+clean; 591 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 5.5 KiB/s wr, 16 op/s 2024-09-19T09:01:02.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.577+0000 7fee0ce57640 1 -- 172.21.15.120:0/3208412425 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fee08062630 con 0x7fedf40a32d0 2024-09-19T09:01:02.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.602+0000 7fedfffff640 1 -- 172.21.15.120:0/3208412425 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fee08062630 con 0x7fedf40a32d0 2024-09-19T09:01:02.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.604+0000 7fee0ce57640 1 -- 172.21.15.120:0/3208412425 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fedf40a32d0 msgr2=0x7fedf40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:02.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.604+0000 7fee0ce57640 1 --2- 172.21.15.120:0/3208412425 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fedf40a32d0 0x7fedf40a5790 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fee0810d490 tx=0x7fedf000b2e0 comp rx=0 tx=0).stop 2024-09-19T09:01:02.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.604+0000 7fee0ce57640 1 -- 172.21.15.120:0/3208412425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee08102d40 msgr2=0x7fee081983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:02.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.604+0000 7fee0ce57640 1 --2- 172.21.15.120:0/3208412425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee08102d40 0x7fee081983b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fedf8005fd0 tx=0x7fedf800dde0 comp rx=0 tx=0).stop 2024-09-19T09:01:02.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.604+0000 7fee0ce57640 1 -- 172.21.15.120:0/3208412425 shutdown_connections 2024-09-19T09:01:02.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.604+0000 7fee0ce57640 1 --2- 172.21.15.120:0/3208412425 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fedf40a32d0 0x7fedf40a5790 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:02.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.604+0000 7fee0ce57640 1 --2- 172.21.15.120:0/3208412425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee0810c170 0x7fee0819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:02.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.604+0000 7fee0ce57640 1 --2- 172.21.15.120:0/3208412425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee08103660 0x7fee081988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:02.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.604+0000 7fee0ce57640 1 --2- 172.21.15.120:0/3208412425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee08102d40 0x7fee081983b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:02.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.604+0000 7fee0ce57640 1 -- 172.21.15.120:0/3208412425 >> 172.21.15.120:0/3208412425 conn(0x7fee080fc7a0 msgr2=0x7fee080fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:02.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.604+0000 7fee0ce57640 1 -- 172.21.15.120:0/3208412425 shutdown_connections 2024-09-19T09:01:02.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:02.604+0000 7fee0ce57640 1 -- 172.21.15.120:0/3208412425 wait complete. 2024-09-19T09:01:03.302 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:03 smithi120 ceph-mon[31150]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T09:01:03.302 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:03 smithi120 ceph-mon[31150]: from='client.24514 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:01:03.330 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-19T09:01:03.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:03 smithi196 ceph-mon[35150]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T09:01:03.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:03 smithi196 ceph-mon[35150]: from='client.24514 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:01:03.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:03 smithi153 ceph-mon[35071]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T09:01:03.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:03 smithi153 ceph-mon[35071]: from='client.24514 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:01:03.577 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:03.937 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 19 09:01:03 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-b[64524]: 2024-09-19T09:01:03.661+0000 7f1ecf160640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-19T09:01:04.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.099+0000 7f7b7533b640 1 -- 172.21.15.120:0/1972051540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b70100e20 msgr2=0x7f7b70101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:04.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.099+0000 7f7b7533b640 1 --2- 172.21.15.120:0/1972051540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b70100e20 0x7f7b70101200 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7b60009ca0 tx=0x7f7b6002a950 comp rx=0 tx=0).stop 2024-09-19T09:01:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.099+0000 7f7b7533b640 1 -- 172.21.15.120:0/1972051540 shutdown_connections 2024-09-19T09:01:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.099+0000 7f7b7533b640 1 --2- 172.21.15.120:0/1972051540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b70102ef0 0x7f7b7010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.099+0000 7f7b7533b640 1 --2- 172.21.15.120:0/1972051540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b70102570 0x7f7b701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.099+0000 7f7b7533b640 1 --2- 172.21.15.120:0/1972051540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b70100e20 0x7f7b70101200 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.099+0000 7f7b7533b640 1 -- 172.21.15.120:0/1972051540 >> 172.21.15.120:0/1972051540 conn(0x7f7b700fc7d0 msgr2=0x7f7b700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.099+0000 7f7b7533b640 1 -- 172.21.15.120:0/1972051540 shutdown_connections 2024-09-19T09:01:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.099+0000 7f7b7533b640 1 -- 172.21.15.120:0/1972051540 wait complete. 2024-09-19T09:01:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.099+0000 7f7b7533b640 1 Processor -- start 2024-09-19T09:01:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.100+0000 7f7b7533b640 1 -- start start 2024-09-19T09:01:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.100+0000 7f7b7533b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b70100e20 0x7f7b701984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.100+0000 7f7b7533b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b70102570 0x7f7b70198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.100+0000 7f7b7533b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b70102ef0 0x7f7b7019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.100+0000 7f7b7533b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7b70112060 con 0x7f7b70102570 2024-09-19T09:01:04.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.100+0000 7f7b7533b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7b70111ee0 con 0x7f7b70100e20 2024-09-19T09:01:04.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.100+0000 7f7b7533b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7b701121e0 con 0x7f7b70102ef0 2024-09-19T09:01:04.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.100+0000 7f7b67fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b70102570 0x7f7b70198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:04.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.100+0000 7f7b67fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b70102570 0x7f7b70198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34146/0 (socket says 172.21.15.120:34146) 2024-09-19T09:01:04.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.100+0000 7f7b67fff640 1 -- 172.21.15.120:0/4198873657 learned_addr learned my addr 172.21.15.120:0/4198873657 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:04.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.100+0000 7f7b6f7fe640 1 --2- 172.21.15.120:0/4198873657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b70102ef0 0x7f7b7019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:04.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.100+0000 7f7b67fff640 1 -- 172.21.15.120:0/4198873657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b70102ef0 msgr2=0x7f7b7019cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:04.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.100+0000 7f7b6effd640 1 --2- 172.21.15.120:0/4198873657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b70100e20 0x7f7b701984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:04.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.100+0000 7f7b67fff640 1 --2- 172.21.15.120:0/4198873657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b70102ef0 0x7f7b7019cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:04.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.100+0000 7f7b67fff640 1 -- 172.21.15.120:0/4198873657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b70100e20 msgr2=0x7f7b701984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:04.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.100+0000 7f7b67fff640 1 --2- 172.21.15.120:0/4198873657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b70100e20 0x7f7b701984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:04.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.100+0000 7f7b67fff640 1 -- 172.21.15.120:0/4198873657 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7019d470 con 0x7f7b70102570 2024-09-19T09:01:04.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.100+0000 7f7b6f7fe640 1 --2- 172.21.15.120:0/4198873657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b70102ef0 0x7f7b7019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:01:04.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.100+0000 7f7b6effd640 1 --2- 172.21.15.120:0/4198873657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b70100e20 0x7f7b701984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:01:04.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.100+0000 7f7b67fff640 1 --2- 172.21.15.120:0/4198873657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b70102570 0x7f7b70198a00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7b5800d950 tx=0x7f7b5800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:04.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.101+0000 7f7b6cff9640 1 -- 172.21.15.120:0/4198873657 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b58017070 con 0x7f7b70102570 2024-09-19T09:01:04.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.101+0000 7f7b6cff9640 1 -- 172.21.15.120:0/4198873657 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b58009c30 con 0x7f7b70102570 2024-09-19T09:01:04.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.101+0000 7f7b7533b640 1 -- 172.21.15.120:0/4198873657 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b7019d760 con 0x7f7b70102570 2024-09-19T09:01:04.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.101+0000 7f7b6cff9640 1 -- 172.21.15.120:0/4198873657 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5800f6b0 con 0x7f7b70102570 2024-09-19T09:01:04.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.101+0000 7f7b7533b640 1 -- 172.21.15.120:0/4198873657 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b701a5040 con 0x7f7b70102570 2024-09-19T09:01:04.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.101+0000 7f7b7533b640 1 -- 172.21.15.120:0/4198873657 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b70101200 con 0x7f7b70102570 2024-09-19T09:01:04.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.103+0000 7f7b6cff9640 1 -- 172.21.15.120:0/4198873657 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f7b580109d0 con 0x7f7b70102570 2024-09-19T09:01:04.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.103+0000 7f7b6cff9640 1 --2- 172.21.15.120:0/4198873657 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7b5c0a3350 0x7f7b5c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:04.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.103+0000 7f7b6effd640 1 --2- 172.21.15.120:0/4198873657 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7b5c0a3350 0x7f7b5c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:04.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.103+0000 7f7b6cff9640 1 -- 172.21.15.120:0/4198873657 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f7b580cf570 con 0x7f7b70102570 2024-09-19T09:01:04.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.104+0000 7f7b6effd640 1 --2- 172.21.15.120:0/4198873657 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7b5c0a3350 0x7f7b5c0a5810 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7b60006010 tx=0x7f7b60005f80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:04.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.107+0000 7f7b6cff9640 1 -- 172.21.15.120:0/4198873657 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7b58009de0 con 0x7f7b70102570 2024-09-19T09:01:04.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.300+0000 7f7b7533b640 1 -- 172.21.15.120:0/4198873657 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f7b70062630 con 0x7f7b5c0a3350 2024-09-19T09:01:04.301 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:04 smithi120 ceph-mon[31150]: pgmap v285: 97 pgs: 97 active+clean; 591 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 5.5 KiB/s wr, 19 op/s 2024-09-19T09:01:04.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.307+0000 7f7b6cff9640 1 -- 172.21.15.120:0/4198873657 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f7b70062630 con 0x7f7b5c0a3350 2024-09-19T09:01:04.308 INFO:teuthology.orchestra.run.smithi120.stdout:/volumes/qa/sv_0/cdb0287a-f7d5-48be-9274-616b526ceaa2 2024-09-19T09:01:04.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.309+0000 7f7b7533b640 1 -- 172.21.15.120:0/4198873657 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7b5c0a3350 msgr2=0x7f7b5c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:04.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.309+0000 7f7b7533b640 1 --2- 172.21.15.120:0/4198873657 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7b5c0a3350 0x7f7b5c0a5810 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7b60006010 tx=0x7f7b60005f80 comp rx=0 tx=0).stop 2024-09-19T09:01:04.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.309+0000 7f7b7533b640 1 -- 172.21.15.120:0/4198873657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b70102570 msgr2=0x7f7b70198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:04.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.309+0000 7f7b7533b640 1 --2- 172.21.15.120:0/4198873657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b70102570 0x7f7b70198a00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7b5800d950 tx=0x7f7b5800de10 comp rx=0 tx=0).stop 2024-09-19T09:01:04.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.309+0000 7f7b7533b640 1 -- 172.21.15.120:0/4198873657 shutdown_connections 2024-09-19T09:01:04.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.309+0000 7f7b7533b640 1 --2- 172.21.15.120:0/4198873657 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7b5c0a3350 0x7f7b5c0a5810 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:04.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.309+0000 7f7b7533b640 1 --2- 172.21.15.120:0/4198873657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b70102ef0 0x7f7b7019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:04.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.309+0000 7f7b7533b640 1 --2- 172.21.15.120:0/4198873657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b70102570 0x7f7b70198a00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:04.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.309+0000 7f7b7533b640 1 --2- 172.21.15.120:0/4198873657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b70100e20 0x7f7b701984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:04.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.309+0000 7f7b7533b640 1 -- 172.21.15.120:0/4198873657 >> 172.21.15.120:0/4198873657 conn(0x7f7b700fc7d0 msgr2=0x7f7b700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:04.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.309+0000 7f7b7533b640 1 -- 172.21.15.120:0/4198873657 shutdown_connections 2024-09-19T09:01:04.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:04.309+0000 7f7b7533b640 1 -- 172.21.15.120:0/4198873657 wait complete. 2024-09-19T09:01:04.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:04 smithi196 ceph-mon[35150]: pgmap v285: 97 pgs: 97 active+clean; 591 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 5.5 KiB/s wr, 19 op/s 2024-09-19T09:01:04.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:04 smithi153 ceph-mon[35071]: pgmap v285: 97 pgs: 97 active+clean; 591 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 5.5 KiB/s wr, 19 op/s 2024-09-19T09:01:05.127 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-19T09:01:05.372 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:05.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:05 smithi196 ceph-mon[35150]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:01:05.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:05 smithi153 ceph-mon[35071]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:01:05.492 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:05 smithi120 ceph-mon[31150]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:01:05.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.875+0000 7f411eecf640 1 -- 172.21.15.120:0/3910991254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4118102ef0 msgr2=0x7f411810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:05.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.875+0000 7f411eecf640 1 --2- 172.21.15.120:0/3910991254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4118102ef0 0x7f411810da40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4110009ca0 tx=0x7f411002a950 comp rx=0 tx=0).stop 2024-09-19T09:01:05.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.875+0000 7f411eecf640 1 -- 172.21.15.120:0/3910991254 shutdown_connections 2024-09-19T09:01:05.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.875+0000 7f411eecf640 1 --2- 172.21.15.120:0/3910991254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4118102ef0 0x7f411810da40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:05.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.875+0000 7f411eecf640 1 --2- 172.21.15.120:0/3910991254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4118102570 0x7f41181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:05.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.875+0000 7f411eecf640 1 --2- 172.21.15.120:0/3910991254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4118100e20 0x7f4118101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:05.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.875+0000 7f411eecf640 1 -- 172.21.15.120:0/3910991254 >> 172.21.15.120:0/3910991254 conn(0x7f41180fc7d0 msgr2=0x7f41180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:05.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.875+0000 7f411eecf640 1 -- 172.21.15.120:0/3910991254 shutdown_connections 2024-09-19T09:01:05.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.875+0000 7f411eecf640 1 -- 172.21.15.120:0/3910991254 wait complete. 2024-09-19T09:01:05.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.875+0000 7f411eecf640 1 Processor -- start 2024-09-19T09:01:05.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.875+0000 7f411eecf640 1 -- start start 2024-09-19T09:01:05.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.876+0000 7f411eecf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4118100e20 0x7f411810e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:05.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.876+0000 7f411eecf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4118102570 0x7f411810ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:05.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.876+0000 7f411eecf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4118102ef0 0x7f411810ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:05.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.876+0000 7f411eecf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f41181120a0 con 0x7f4118102570 2024-09-19T09:01:05.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.876+0000 7f411eecf640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4118111f20 con 0x7f4118102ef0 2024-09-19T09:01:05.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.876+0000 7f411eecf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4118112220 con 0x7f4118100e20 2024-09-19T09:01:05.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.876+0000 7f4117fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4118102570 0x7f411810ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:05.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.876+0000 7f4117fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4118102570 0x7f411810ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34168/0 (socket says 172.21.15.120:34168) 2024-09-19T09:01:05.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.876+0000 7f4117fff640 1 -- 172.21.15.120:0/673904141 learned_addr learned my addr 172.21.15.120:0/673904141 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:05.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.876+0000 7f411cc6b640 1 --2- 172.21.15.120:0/673904141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4118100e20 0x7f411810e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:05.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.876+0000 7f411d46c640 1 --2- 172.21.15.120:0/673904141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4118102ef0 0x7f411810ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:05.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.876+0000 7f4117fff640 1 -- 172.21.15.120:0/673904141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4118100e20 msgr2=0x7f411810e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:05.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.876+0000 7f4117fff640 1 --2- 172.21.15.120:0/673904141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4118100e20 0x7f411810e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:05.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.876+0000 7f4117fff640 1 -- 172.21.15.120:0/673904141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4118102ef0 msgr2=0x7f411810ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:05.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.876+0000 7f4117fff640 1 --2- 172.21.15.120:0/673904141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4118102ef0 0x7f411810ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:05.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.876+0000 7f4117fff640 1 -- 172.21.15.120:0/673904141 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41181a6970 con 0x7f4118102570 2024-09-19T09:01:05.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.876+0000 7f4117fff640 1 --2- 172.21.15.120:0/673904141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4118102570 0x7f411810ea10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f410400d950 tx=0x7f410400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:05.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.876+0000 7f4115ffb640 1 -- 172.21.15.120:0/673904141 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4104017070 con 0x7f4118102570 2024-09-19T09:01:05.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.876+0000 7f411eecf640 1 -- 172.21.15.120:0/673904141 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41181a6c60 con 0x7f4118102570 2024-09-19T09:01:05.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.876+0000 7f4115ffb640 1 -- 172.21.15.120:0/673904141 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f410400de80 con 0x7f4118102570 2024-09-19T09:01:05.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.876+0000 7f4115ffb640 1 -- 172.21.15.120:0/673904141 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410400f620 con 0x7f4118102570 2024-09-19T09:01:05.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.877+0000 7f411eecf640 1 -- 172.21.15.120:0/673904141 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41181a71a0 con 0x7f4118102570 2024-09-19T09:01:05.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.878+0000 7f411eecf640 1 -- 172.21.15.120:0/673904141 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4118101200 con 0x7f4118102570 2024-09-19T09:01:05.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.879+0000 7f4115ffb640 1 -- 172.21.15.120:0/673904141 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f4104009c30 con 0x7f4118102570 2024-09-19T09:01:05.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.879+0000 7f4115ffb640 1 --2- 172.21.15.120:0/673904141 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f41080a32d0 0x7f41080a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:05.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.879+0000 7f411cc6b640 1 --2- 172.21.15.120:0/673904141 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f41080a32d0 0x7f41080a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:05.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.879+0000 7f4115ffb640 1 -- 172.21.15.120:0/673904141 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f41040cf4b0 con 0x7f4118102570 2024-09-19T09:01:05.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.880+0000 7f411cc6b640 1 --2- 172.21.15.120:0/673904141 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f41080a32d0 0x7f41080a5790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f410c009990 tx=0x7f410c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:05.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:05.883+0000 7f4115ffb640 1 -- 172.21.15.120:0/673904141 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f410409c220 con 0x7f4118102570 2024-09-19T09:01:06.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:06.073+0000 7f411eecf640 1 -- 172.21.15.120:0/673904141 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4118106f80 con 0x7f41080a32d0 2024-09-19T09:01:06.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:06.096+0000 7f4115ffb640 1 -- 172.21.15.120:0/673904141 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4118106f80 con 0x7f41080a32d0 2024-09-19T09:01:06.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:06.098+0000 7f411eecf640 1 -- 172.21.15.120:0/673904141 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f41080a32d0 msgr2=0x7f41080a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:06.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:06.098+0000 7f411eecf640 1 --2- 172.21.15.120:0/673904141 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f41080a32d0 0x7f41080a5790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f410c009990 tx=0x7f410c009110 comp rx=0 tx=0).stop 2024-09-19T09:01:06.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:06.098+0000 7f411eecf640 1 -- 172.21.15.120:0/673904141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4118102570 msgr2=0x7f411810ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:06.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:06.098+0000 7f411eecf640 1 --2- 172.21.15.120:0/673904141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4118102570 0x7f411810ea10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f410400d950 tx=0x7f410400dd50 comp rx=0 tx=0).stop 2024-09-19T09:01:06.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:06.098+0000 7f411eecf640 1 -- 172.21.15.120:0/673904141 shutdown_connections 2024-09-19T09:01:06.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:06.098+0000 7f411eecf640 1 --2- 172.21.15.120:0/673904141 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f41080a32d0 0x7f41080a5790 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:06.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:06.098+0000 7f411eecf640 1 --2- 172.21.15.120:0/673904141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4118102ef0 0x7f411810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:06.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:06.098+0000 7f411eecf640 1 --2- 172.21.15.120:0/673904141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4118102570 0x7f411810ea10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:06.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:06.098+0000 7f411eecf640 1 --2- 172.21.15.120:0/673904141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4118100e20 0x7f411810e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:06.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:06.098+0000 7f411eecf640 1 -- 172.21.15.120:0/673904141 >> 172.21.15.120:0/673904141 conn(0x7f41180fc7d0 msgr2=0x7f4118106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:06.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:06.098+0000 7f411eecf640 1 -- 172.21.15.120:0/673904141 shutdown_connections 2024-09-19T09:01:06.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:06.098+0000 7f411eecf640 1 -- 172.21.15.120:0/673904141 wait complete. 2024-09-19T09:01:06.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:06 smithi196 ceph-mon[35150]: pgmap v286: 97 pgs: 97 active+clean; 591 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.8 KiB/s wr, 19 op/s 2024-09-19T09:01:06.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:06 smithi153 ceph-mon[35071]: pgmap v286: 97 pgs: 97 active+clean; 591 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.8 KiB/s wr, 19 op/s 2024-09-19T09:01:06.539 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:06 smithi120 ceph-mon[31150]: pgmap v286: 97 pgs: 97 active+clean; 591 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.8 KiB/s wr, 19 op/s 2024-09-19T09:01:06.826 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-19T09:01:07.068 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:07.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:07 smithi196 ceph-mon[35150]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:01:07.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:07 smithi153 ceph-mon[35071]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:01:07.478 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:07 smithi120 ceph-mon[31150]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:01:07.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.561+0000 7fcf59fcb640 1 -- 172.21.15.120:0/2012397252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf54106f20 msgr2=0x7fcf5410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:07.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.561+0000 7fcf59fcb640 1 --2- 172.21.15.120:0/2012397252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf54106f20 0x7fcf5410eaa0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c009ca0 tx=0x7fcf4c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:01:07.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.561+0000 7fcf59fcb640 1 -- 172.21.15.120:0/2012397252 shutdown_connections 2024-09-19T09:01:07.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.561+0000 7fcf59fcb640 1 --2- 172.21.15.120:0/2012397252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf54106f20 0x7fcf5410eaa0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:07.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.561+0000 7fcf59fcb640 1 --2- 172.21.15.120:0/2012397252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf541027a0 0x7fcf541067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:07.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.561+0000 7fcf59fcb640 1 --2- 172.21.15.120:0/2012397252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf54101df0 0x7fcf541021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:07.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.561+0000 7fcf59fcb640 1 -- 172.21.15.120:0/2012397252 >> 172.21.15.120:0/2012397252 conn(0x7fcf540fd8b0 msgr2=0x7fcf540ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:07.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.561+0000 7fcf59fcb640 1 -- 172.21.15.120:0/2012397252 shutdown_connections 2024-09-19T09:01:07.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.561+0000 7fcf59fcb640 1 -- 172.21.15.120:0/2012397252 wait complete. 2024-09-19T09:01:07.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.561+0000 7fcf59fcb640 1 Processor -- start 2024-09-19T09:01:07.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.562+0000 7fcf59fcb640 1 -- start start 2024-09-19T09:01:07.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.562+0000 7fcf59fcb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf54101df0 0x7fcf54198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:07.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.562+0000 7fcf59fcb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf541027a0 0x7fcf54198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:07.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.562+0000 7fcf59fcb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf54106f20 0x7fcf5419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:07.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.562+0000 7fcf59fcb640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcf54112020 con 0x7fcf54106f20 2024-09-19T09:01:07.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.562+0000 7fcf59fcb640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcf54111ea0 con 0x7fcf54101df0 2024-09-19T09:01:07.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.562+0000 7fcf59fcb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcf541121a0 con 0x7fcf541027a0 2024-09-19T09:01:07.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.562+0000 7fcf53fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf54106f20 0x7fcf5419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:07.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.562+0000 7fcf53fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf54106f20 0x7fcf5419cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34188/0 (socket says 172.21.15.120:34188) 2024-09-19T09:01:07.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.562+0000 7fcf53fff640 1 -- 172.21.15.120:0/3634878552 learned_addr learned my addr 172.21.15.120:0/3634878552 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:07.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.562+0000 7fcf537fe640 1 --2- 172.21.15.120:0/3634878552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf54101df0 0x7fcf54198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:07.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.562+0000 7fcf4bfff640 1 --2- 172.21.15.120:0/3634878552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf541027a0 0x7fcf54198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:07.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.562+0000 7fcf53fff640 1 -- 172.21.15.120:0/3634878552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf541027a0 msgr2=0x7fcf54198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:07.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.562+0000 7fcf53fff640 1 --2- 172.21.15.120:0/3634878552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf541027a0 0x7fcf54198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:07.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.562+0000 7fcf53fff640 1 -- 172.21.15.120:0/3634878552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf54101df0 msgr2=0x7fcf54198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:07.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.562+0000 7fcf53fff640 1 --2- 172.21.15.120:0/3634878552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf54101df0 0x7fcf54198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:07.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.562+0000 7fcf53fff640 1 -- 172.21.15.120:0/3634878552 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf5419d4b0 con 0x7fcf54106f20 2024-09-19T09:01:07.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.562+0000 7fcf537fe640 1 --2- 172.21.15.120:0/3634878552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf54101df0 0x7fcf54198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:01:07.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.562+0000 7fcf53fff640 1 --2- 172.21.15.120:0/3634878552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf54106f20 0x7fcf5419cdd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c031a60 tx=0x7fcf4c031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:07.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.563+0000 7fcf517fa640 1 -- 172.21.15.120:0/3634878552 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c003c30 con 0x7fcf54106f20 2024-09-19T09:01:07.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.563+0000 7fcf517fa640 1 -- 172.21.15.120:0/3634878552 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf4c003dd0 con 0x7fcf54106f20 2024-09-19T09:01:07.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.563+0000 7fcf59fcb640 1 -- 172.21.15.120:0/3634878552 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf5419d740 con 0x7fcf54106f20 2024-09-19T09:01:07.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.563+0000 7fcf517fa640 1 -- 172.21.15.120:0/3634878552 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c0047e0 con 0x7fcf54106f20 2024-09-19T09:01:07.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.563+0000 7fcf59fcb640 1 -- 172.21.15.120:0/3634878552 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf541a5080 con 0x7fcf54106f20 2024-09-19T09:01:07.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.564+0000 7fcf59fcb640 1 -- 172.21.15.120:0/3634878552 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf54102250 con 0x7fcf54106f20 2024-09-19T09:01:07.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.565+0000 7fcf517fa640 1 -- 172.21.15.120:0/3634878552 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fcf4c02f790 con 0x7fcf54106f20 2024-09-19T09:01:07.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.565+0000 7fcf517fa640 1 --2- 172.21.15.120:0/3634878552 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcf400a32d0 0x7fcf400a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:07.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.565+0000 7fcf537fe640 1 --2- 172.21.15.120:0/3634878552 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcf400a32d0 0x7fcf400a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:07.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.566+0000 7fcf517fa640 1 -- 172.21.15.120:0/3634878552 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fcf4c0f0e10 con 0x7fcf54106f20 2024-09-19T09:01:07.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.566+0000 7fcf537fe640 1 --2- 172.21.15.120:0/3634878552 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcf400a32d0 0x7fcf400a5790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcf44009930 tx=0x7fcf44008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:07.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.569+0000 7fcf517fa640 1 -- 172.21.15.120:0/3634878552 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcf4c0bdc90 con 0x7fcf54106f20 2024-09-19T09:01:07.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.763+0000 7fcf59fcb640 1 -- 172.21.15.120:0/3634878552 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fcf54062630 con 0x7fcf400a32d0 2024-09-19T09:01:07.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.767+0000 7fcf517fa640 1 -- 172.21.15.120:0/3634878552 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fcf54062630 con 0x7fcf400a32d0 2024-09-19T09:01:07.768 INFO:teuthology.orchestra.run.smithi120.stdout:/volumes/qa/sv_1/f2bee15c-e720-4829-ac9c-1a0401c9d37d 2024-09-19T09:01:07.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.769+0000 7fcf59fcb640 1 -- 172.21.15.120:0/3634878552 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcf400a32d0 msgr2=0x7fcf400a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:07.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.769+0000 7fcf59fcb640 1 --2- 172.21.15.120:0/3634878552 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcf400a32d0 0x7fcf400a5790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcf44009930 tx=0x7fcf44008040 comp rx=0 tx=0).stop 2024-09-19T09:01:07.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.769+0000 7fcf59fcb640 1 -- 172.21.15.120:0/3634878552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf54106f20 msgr2=0x7fcf5419cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:07.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.769+0000 7fcf59fcb640 1 --2- 172.21.15.120:0/3634878552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf54106f20 0x7fcf5419cdd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c031a60 tx=0x7fcf4c031a90 comp rx=0 tx=0).stop 2024-09-19T09:01:07.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.770+0000 7fcf59fcb640 1 -- 172.21.15.120:0/3634878552 shutdown_connections 2024-09-19T09:01:07.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.770+0000 7fcf59fcb640 1 --2- 172.21.15.120:0/3634878552 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcf400a32d0 0x7fcf400a5790 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:07.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.770+0000 7fcf59fcb640 1 --2- 172.21.15.120:0/3634878552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf54106f20 0x7fcf5419cdd0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:07.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.770+0000 7fcf59fcb640 1 --2- 172.21.15.120:0/3634878552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf541027a0 0x7fcf54198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:07.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.770+0000 7fcf59fcb640 1 --2- 172.21.15.120:0/3634878552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf54101df0 0x7fcf54198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:07.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.770+0000 7fcf59fcb640 1 -- 172.21.15.120:0/3634878552 >> 172.21.15.120:0/3634878552 conn(0x7fcf540fd8b0 msgr2=0x7fcf540fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:07.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.770+0000 7fcf59fcb640 1 -- 172.21.15.120:0/3634878552 shutdown_connections 2024-09-19T09:01:07.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:07.770+0000 7fcf59fcb640 1 -- 172.21.15.120:0/3634878552 wait complete. 2024-09-19T09:01:08.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:08 smithi120 ceph-mon[31150]: pgmap v287: 97 pgs: 97 active+clean; 610 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.6 KiB/s wr, 22 op/s 2024-09-19T09:01:08.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:08 smithi120 ceph-mon[31150]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:01:08.430 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:01:08.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:08 smithi196 ceph-mon[35150]: pgmap v287: 97 pgs: 97 active+clean; 610 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.6 KiB/s wr, 22 op/s 2024-09-19T09:01:08.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:08 smithi196 ceph-mon[35150]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:01:08.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:08 smithi153 ceph-mon[35071]: pgmap v287: 97 pgs: 97 active+clean; 610 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.6 KiB/s wr, 22 op/s 2024-09-19T09:01:08.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:08 smithi153 ceph-mon[35071]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:01:08.672 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:09.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.173+0000 7f6d4af09640 1 -- 172.21.15.120:0/2582220041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d44106f10 msgr2=0x7f6d4410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:09.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.173+0000 7f6d4af09640 1 --2- 172.21.15.120:0/2582220041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d44106f10 0x7f6d4410ea90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c009ca0 tx=0x7f6d3c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:01:09.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.173+0000 7f6d4af09640 1 -- 172.21.15.120:0/2582220041 shutdown_connections 2024-09-19T09:01:09.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.173+0000 7f6d4af09640 1 --2- 172.21.15.120:0/2582220041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d44106f10 0x7f6d4410ea90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:09.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.173+0000 7f6d4af09640 1 --2- 172.21.15.120:0/2582220041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d44102790 0x7f6d441067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:09.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.173+0000 7f6d4af09640 1 --2- 172.21.15.120:0/2582220041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d44101de0 0x7f6d441021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:09.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.173+0000 7f6d4af09640 1 -- 172.21.15.120:0/2582220041 >> 172.21.15.120:0/2582220041 conn(0x7f6d440fd8c0 msgr2=0x7f6d440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:09.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.173+0000 7f6d4af09640 1 -- 172.21.15.120:0/2582220041 shutdown_connections 2024-09-19T09:01:09.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.173+0000 7f6d4af09640 1 -- 172.21.15.120:0/2582220041 wait complete. 2024-09-19T09:01:09.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.174+0000 7f6d4af09640 1 Processor -- start 2024-09-19T09:01:09.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.174+0000 7f6d4af09640 1 -- start start 2024-09-19T09:01:09.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.174+0000 7f6d4af09640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d44101de0 0x7f6d440744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:09.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.174+0000 7f6d4af09640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d44102790 0x7f6d44074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:09.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.174+0000 7f6d4af09640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d44106f10 0x7f6d44074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:09.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.174+0000 7f6d4af09640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6d44111f70 con 0x7f6d44106f10 2024-09-19T09:01:09.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.174+0000 7f6d4af09640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6d44111df0 con 0x7f6d44101de0 2024-09-19T09:01:09.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.174+0000 7f6d4af09640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6d441120f0 con 0x7f6d44102790 2024-09-19T09:01:09.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.174+0000 7f6d494a6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d44106f10 0x7f6d44074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:09.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.174+0000 7f6d494a6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d44106f10 0x7f6d44074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34204/0 (socket says 172.21.15.120:34204) 2024-09-19T09:01:09.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.174+0000 7f6d494a6640 1 -- 172.21.15.120:0/1030367816 learned_addr learned my addr 172.21.15.120:0/1030367816 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:09.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.174+0000 7f6d494a6640 1 -- 172.21.15.120:0/1030367816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d44102790 msgr2=0x7f6d44074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:09.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.174+0000 7f6d494a6640 1 --2- 172.21.15.120:0/1030367816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d44102790 0x7f6d44074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:09.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.174+0000 7f6d48ca5640 1 --2- 172.21.15.120:0/1030367816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d44101de0 0x7f6d440744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:09.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.174+0000 7f6d494a6640 1 -- 172.21.15.120:0/1030367816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d44101de0 msgr2=0x7f6d440744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:09.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.175+0000 7f6d494a6640 1 --2- 172.21.15.120:0/1030367816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d44101de0 0x7f6d440744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:09.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.175+0000 7f6d494a6640 1 -- 172.21.15.120:0/1030367816 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d441a6810 con 0x7f6d44106f10 2024-09-19T09:01:09.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.175+0000 7f6d48ca5640 1 --2- 172.21.15.120:0/1030367816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d44101de0 0x7f6d440744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:01:09.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.175+0000 7f6d494a6640 1 --2- 172.21.15.120:0/1030367816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d44106f10 0x7f6d44074f70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c031a60 tx=0x7f6d3c031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:09.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.175+0000 7f6d41ffb640 1 -- 172.21.15.120:0/1030367816 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c003c30 con 0x7f6d44106f10 2024-09-19T09:01:09.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.175+0000 7f6d41ffb640 1 -- 172.21.15.120:0/1030367816 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d3c003dd0 con 0x7f6d44106f10 2024-09-19T09:01:09.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.175+0000 7f6d4af09640 1 -- 172.21.15.120:0/1030367816 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d441a6aa0 con 0x7f6d44106f10 2024-09-19T09:01:09.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.175+0000 7f6d41ffb640 1 -- 172.21.15.120:0/1030367816 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c0047e0 con 0x7f6d44106f10 2024-09-19T09:01:09.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.175+0000 7f6d4af09640 1 -- 172.21.15.120:0/1030367816 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d441a7030 con 0x7f6d44106f10 2024-09-19T09:01:09.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.176+0000 7f6d4af09640 1 -- 172.21.15.120:0/1030367816 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d441021c0 con 0x7f6d44106f10 2024-09-19T09:01:09.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.177+0000 7f6d41ffb640 1 -- 172.21.15.120:0/1030367816 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f6d3c02f790 con 0x7f6d44106f10 2024-09-19T09:01:09.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.178+0000 7f6d41ffb640 1 --2- 172.21.15.120:0/1030367816 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6d340a3350 0x7f6d340a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:09.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.178+0000 7f6d41ffb640 1 -- 172.21.15.120:0/1030367816 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f6d3c0f0c10 con 0x7f6d44106f10 2024-09-19T09:01:09.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.178+0000 7f6d48ca5640 1 --2- 172.21.15.120:0/1030367816 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6d340a3350 0x7f6d340a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:09.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.178+0000 7f6d48ca5640 1 --2- 172.21.15.120:0/1030367816 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6d340a3350 0x7f6d340a5810 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6d380058c0 tx=0x7f6d38005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:09.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.182+0000 7f6d41ffb640 1 -- 172.21.15.120:0/1030367816 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6d3c0bda90 con 0x7f6d44106f10 2024-09-19T09:01:09.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.384+0000 7f6d4af09640 1 -- 172.21.15.120:0/1030367816 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d441a7330 con 0x7f6d44106f10 2024-09-19T09:01:09.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.385+0000 7f6d41ffb640 1 -- 172.21.15.120:0/1030367816 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11988 (secure 0 0 0) 0x7f6d3c0098b0 con 0x7f6d44106f10 2024-09-19T09:01:09.387 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:01:09.387 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":27,"btime":"2024-09-19T09:00:59:134373+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:00:59.134353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:01:09.387 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 27 2024-09-19T09:01:09.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.387+0000 7f6d4af09640 1 -- 172.21.15.120:0/1030367816 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6d340a3350 msgr2=0x7f6d340a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:09.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.387+0000 7f6d4af09640 1 --2- 172.21.15.120:0/1030367816 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6d340a3350 0x7f6d340a5810 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6d380058c0 tx=0x7f6d38005d30 comp rx=0 tx=0).stop 2024-09-19T09:01:09.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.387+0000 7f6d4af09640 1 -- 172.21.15.120:0/1030367816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d44106f10 msgr2=0x7f6d44074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:09.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.387+0000 7f6d4af09640 1 --2- 172.21.15.120:0/1030367816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d44106f10 0x7f6d44074f70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c031a60 tx=0x7f6d3c031a90 comp rx=0 tx=0).stop 2024-09-19T09:01:09.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.387+0000 7f6d4af09640 1 -- 172.21.15.120:0/1030367816 shutdown_connections 2024-09-19T09:01:09.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.387+0000 7f6d4af09640 1 --2- 172.21.15.120:0/1030367816 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6d340a3350 0x7f6d340a5810 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:09.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.387+0000 7f6d4af09640 1 --2- 172.21.15.120:0/1030367816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d44106f10 0x7f6d44074f70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:09.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.387+0000 7f6d4af09640 1 --2- 172.21.15.120:0/1030367816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d44102790 0x7f6d44074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:09.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.387+0000 7f6d4af09640 1 --2- 172.21.15.120:0/1030367816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d44101de0 0x7f6d440744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:09.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.387+0000 7f6d4af09640 1 -- 172.21.15.120:0/1030367816 >> 172.21.15.120:0/1030367816 conn(0x7f6d440fd8c0 msgr2=0x7f6d440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:09.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.387+0000 7f6d4af09640 1 -- 172.21.15.120:0/1030367816 shutdown_connections 2024-09-19T09:01:09.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:09.387+0000 7f6d4af09640 1 -- 172.21.15.120:0/1030367816 wait complete. 2024-09-19T09:01:09.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:09 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1030367816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:01:09.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:09 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1030367816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:01:09.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:09 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1030367816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:01:10.123 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:01:10.369 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:10.710 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:10 smithi120 ceph-mon[31150]: pgmap v288: 97 pgs: 97 active+clean; 610 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.4 KiB/s wr, 22 op/s 2024-09-19T09:01:10.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:10 smithi196 ceph-mon[35150]: pgmap v288: 97 pgs: 97 active+clean; 610 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.4 KiB/s wr, 22 op/s 2024-09-19T09:01:10.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:10 smithi153 ceph-mon[35071]: pgmap v288: 97 pgs: 97 active+clean; 610 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.4 KiB/s wr, 22 op/s 2024-09-19T09:01:10.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.944+0000 7f1daddaa640 1 -- 172.21.15.120:0/2070763739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1da8101420 msgr2=0x7f1da810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:10.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.944+0000 7f1daddaa640 1 --2- 172.21.15.120:0/2070763739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1da8101420 0x7f1da810d950 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1da0009cd0 tx=0x7f1da002a8e0 comp rx=0 tx=0).stop 2024-09-19T09:01:10.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.944+0000 7f1daddaa640 1 -- 172.21.15.120:0/2070763739 shutdown_connections 2024-09-19T09:01:10.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.944+0000 7f1daddaa640 1 --2- 172.21.15.120:0/2070763739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1da8101420 0x7f1da810d950 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:10.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.944+0000 7f1daddaa640 1 --2- 172.21.15.120:0/2070763739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1da8100a80 0x7f1da8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:10.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.944+0000 7f1daddaa640 1 --2- 172.21.15.120:0/2070763739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1da8106a80 0x7f1da8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:10.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.944+0000 7f1daddaa640 1 -- 172.21.15.120:0/2070763739 >> 172.21.15.120:0/2070763739 conn(0x7f1da80fc7a0 msgr2=0x7f1da80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:10.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.944+0000 7f1daddaa640 1 -- 172.21.15.120:0/2070763739 shutdown_connections 2024-09-19T09:01:10.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.944+0000 7f1daddaa640 1 -- 172.21.15.120:0/2070763739 wait complete. 2024-09-19T09:01:10.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.945+0000 7f1daddaa640 1 Processor -- start 2024-09-19T09:01:10.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.945+0000 7f1daddaa640 1 -- start start 2024-09-19T09:01:10.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.945+0000 7f1daddaa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1da8100a80 0x7f1da81992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:10.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.945+0000 7f1daddaa640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1da8101420 0x7f1da8199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:10.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.945+0000 7f1daddaa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1da8106a80 0x7f1da81a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:10.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.945+0000 7f1daddaa640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1da8110ec0 con 0x7f1da8100a80 2024-09-19T09:01:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.945+0000 7f1daddaa640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1da8110d40 con 0x7f1da8101420 2024-09-19T09:01:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.945+0000 7f1daddaa640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1da8111040 con 0x7f1da8106a80 2024-09-19T09:01:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.945+0000 7f1da77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1da8100a80 0x7f1da81992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.945+0000 7f1da77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1da8100a80 0x7f1da81992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34220/0 (socket says 172.21.15.120:34220) 2024-09-19T09:01:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.945+0000 7f1da77fe640 1 -- 172.21.15.120:0/1997311681 learned_addr learned my addr 172.21.15.120:0/1997311681 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.945+0000 7f1d9ffff640 1 --2- 172.21.15.120:0/1997311681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1da8101420 0x7f1da8199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.945+0000 7f1da77fe640 1 -- 172.21.15.120:0/1997311681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1da8106a80 msgr2=0x7f1da81a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.945+0000 7f1da77fe640 1 --2- 172.21.15.120:0/1997311681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1da8106a80 0x7f1da81a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.945+0000 7f1da77fe640 1 -- 172.21.15.120:0/1997311681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1da8101420 msgr2=0x7f1da8199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.945+0000 7f1da77fe640 1 --2- 172.21.15.120:0/1997311681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1da8101420 0x7f1da8199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.945+0000 7f1da77fe640 1 -- 172.21.15.120:0/1997311681 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da81a48b0 con 0x7f1da8100a80 2024-09-19T09:01:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.946+0000 7f1da77fe640 1 --2- 172.21.15.120:0/1997311681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1da8100a80 0x7f1da81992f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1d9800c970 tx=0x7f1d9800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:10.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.946+0000 7f1da57fa640 1 -- 172.21.15.120:0/1997311681 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98007bb0 con 0x7f1da8100a80 2024-09-19T09:01:10.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.946+0000 7f1da57fa640 1 -- 172.21.15.120:0/1997311681 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d98007d50 con 0x7f1da8100a80 2024-09-19T09:01:10.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.946+0000 7f1daddaa640 1 -- 172.21.15.120:0/1997311681 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da81a4a80 con 0x7f1da8100a80 2024-09-19T09:01:10.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.946+0000 7f1da57fa640 1 -- 172.21.15.120:0/1997311681 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9800f450 con 0x7f1da8100a80 2024-09-19T09:01:10.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.946+0000 7f1daddaa640 1 -- 172.21.15.120:0/1997311681 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da81a4f40 con 0x7f1da8100a80 2024-09-19T09:01:10.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.947+0000 7f1daddaa640 1 -- 172.21.15.120:0/1997311681 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1da8100ee0 con 0x7f1da8100a80 2024-09-19T09:01:10.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.948+0000 7f1da57fa640 1 -- 172.21.15.120:0/1997311681 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f1d98005020 con 0x7f1da8100a80 2024-09-19T09:01:10.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.948+0000 7f1da57fa640 1 --2- 172.21.15.120:0/1997311681 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1d940a32d0 0x7f1d940a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:10.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.949+0000 7f1d9ffff640 1 --2- 172.21.15.120:0/1997311681 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1d940a32d0 0x7f1d940a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:10.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.949+0000 7f1da57fa640 1 -- 172.21.15.120:0/1997311681 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f1d980cf780 con 0x7f1da8100a80 2024-09-19T09:01:10.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.949+0000 7f1d9ffff640 1 --2- 172.21.15.120:0/1997311681 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1d940a32d0 0x7f1d940a5790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1da806b2e0 tx=0x7f1d90008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:10.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:10.953+0000 7f1da57fa640 1 -- 172.21.15.120:0/1997311681 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1d98014030 con 0x7f1da8100a80 2024-09-19T09:01:11.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:11.124+0000 7f1daddaa640 1 -- 172.21.15.120:0/1997311681 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1da81a51d0 con 0x7f1da8100a80 2024-09-19T09:01:11.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:11.125+0000 7f1da57fa640 1 -- 172.21.15.120:0/1997311681 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22362 (secure 0 0 0) 0x7f1d9809c4f0 con 0x7f1da8100a80 2024-09-19T09:01:11.127 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:01:11.127 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":79,"fsid":"507393d2-7664-11ef-bceb-c7b262605968","created":"2024-09-19T08:53:06.386526+0000","modified":"2024-09-19T09:00:47.659465+0000","last_up_change":"2024-09-19T08:59:55.692659+0000","last_in_change":"2024-09-19T08:59:32.808860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:56:07.830047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:00:43.774339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:00:45.786262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33ec3993-576c-428c-bf75-adcb17cd72a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6803","nonce":4131180019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6805","nonce":4131180019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6809","nonce":4131180019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6807","nonce":4131180019}]},"public_addr":"172.21.15.120:6803/4131180019","cluster_addr":"172.21.15.120:6805/4131180019","heartbeat_back_addr":"172.21.15.120:6809/4131180019","heartbeat_front_addr":"172.21.15.120:6807/4131180019","state":["exists","up"]},{"osd":1,"uuid":"450bbebf-2aac-47c6-8474-bbd37fd0a1a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6801","nonce":2259730294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6803","nonce":2259730294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6807","nonce":2259730294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6805","nonce":2259730294}]},"public_addr":"172.21.15.153:6801/2259730294","cluster_addr":"172.21.15.153:6803/2259730294","heartbeat_back_addr":"172.21.15.153:6807/2259730294","heartbeat_front_addr":"172.21.15.153:6805/2259730294","state":["exists","up"]},{"osd":2,"uuid":"ac8c4b83-8e8e-4803-b266-262f7873bd85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6801","nonce":2942286570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6803","nonce":2942286570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6807","nonce":2942286570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6805","nonce":2942286570}]},"public_addr":"172.21.15.196:6801/2942286570","cluster_addr":"172.21.15.196:6803/2942286570","heartbeat_back_addr":"172.21.15.196:6807/2942286570","heartbeat_front_addr":"172.21.15.196:6805/2942286570","state":["exists","up"]},{"osd":3,"uuid":"c1d9e507-8f02-403e-87da-9b4966fd8fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6811","nonce":713205668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6813","nonce":713205668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6817","nonce":713205668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6815","nonce":713205668}]},"public_addr":"172.21.15.120:6811/713205668","cluster_addr":"172.21.15.120:6813/713205668","heartbeat_back_addr":"172.21.15.120:6817/713205668","heartbeat_front_addr":"172.21.15.120:6815/713205668","state":["exists","up"]},{"osd":4,"uuid":"7f8bf3b9-0ec4-46f6-a406-d26d7971179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6809","nonce":2680865854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6811","nonce":2680865854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6815","nonce":2680865854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6813","nonce":2680865854}]},"public_addr":"172.21.15.153:6809/2680865854","cluster_addr":"172.21.15.153:6811/2680865854","heartbeat_back_addr":"172.21.15.153:6815/2680865854","heartbeat_front_addr":"172.21.15.153:6813/2680865854","state":["exists","up"]},{"osd":5,"uuid":"70b270b2-209d-4ff5-875a-04ee7b992a93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6809","nonce":3763331116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6811","nonce":3763331116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6815","nonce":3763331116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6813","nonce":3763331116}]},"public_addr":"172.21.15.196:6809/3763331116","cluster_addr":"172.21.15.196:6811/3763331116","heartbeat_back_addr":"172.21.15.196:6815/3763331116","heartbeat_front_addr":"172.21.15.196:6813/3763331116","state":["exists","up"]},{"osd":6,"uuid":"8087f69c-dc31-4522-a18d-4e6ab1b7cb27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6819","nonce":4171773765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6821","nonce":4171773765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6825","nonce":4171773765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6823","nonce":4171773765}]},"public_addr":"172.21.15.120:6819/4171773765","cluster_addr":"172.21.15.120:6821/4171773765","heartbeat_back_addr":"172.21.15.120:6825/4171773765","heartbeat_front_addr":"172.21.15.120:6823/4171773765","state":["exists","up"]},{"osd":7,"uuid":"c3b1ead3-150f-43fd-8924-f639203f4cdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6817","nonce":2535148453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6819","nonce":2535148453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6823","nonce":2535148453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6821","nonce":2535148453}]},"public_addr":"172.21.15.153:6817/2535148453","cluster_addr":"172.21.15.153:6819/2535148453","heartbeat_back_addr":"172.21.15.153:6823/2535148453","heartbeat_front_addr":"172.21.15.153:6821/2535148453","state":["exists","up"]},{"osd":8,"uuid":"a3273a2a-968e-4e93-a9cf-3879892bdeab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6817","nonce":186195906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6819","nonce":186195906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6823","nonce":186195906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6821","nonce":186195906}]},"public_addr":"172.21.15.196:6817/186195906","cluster_addr":"172.21.15.196:6819/186195906","heartbeat_back_addr":"172.21.15.196:6823/186195906","heartbeat_front_addr":"172.21.15.196:6821/186195906","state":["exists","up"]},{"osd":9,"uuid":"6d184513-dab8-45f7-bc08-fa2e8b2f10b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6827","nonce":1559320883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6829","nonce":1559320883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6833","nonce":1559320883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6831","nonce":1559320883}]},"public_addr":"172.21.15.120:6827/1559320883","cluster_addr":"172.21.15.120:6829/1559320883","heartbeat_back_addr":"172.21.15.120:6833/1559320883","heartbeat_front_addr":"172.21.15.120:6831/1559320883","state":["exists","up"]},{"osd":10,"uuid":"797db886-62f9-4316-a45b-4d50512e65d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6825","nonce":3216877560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6827","nonce":3216877560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6831","nonce":3216877560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6829","nonce":3216877560}]},"public_addr":"172.21.15.153:6825/3216877560","cluster_addr":"172.21.15.153:6827/3216877560","heartbeat_back_addr":"172.21.15.153:6831/3216877560","heartbeat_front_addr":"172.21.15.153:6829/3216877560","state":["exists","up"]},{"osd":11,"uuid":"17e0afaa-84eb-4b26-9810-bfa3cc7e0f3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6825","nonce":1857200740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6827","nonce":1857200740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6831","nonce":1857200740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6829","nonce":1857200740}]},"public_addr":"172.21.15.196:6825/1857200740","cluster_addr":"172.21.15.196:6827/1857200740","heartbeat_back_addr":"172.21.15.196:6831/1857200740","heartbeat_front_addr":"172.21.15.196:6829/1857200740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:55:15.595645+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:55:40.405487+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:04.938034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:29.718696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:55.636029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:57:21.257083+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:57:46.316552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:58:11.946863+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:58:37.098578+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:03.759023+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:28.659049+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:54.350475+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:0/3010783707":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6801/3562882246":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6800/3562882246":"2024-09-20T08:53:51.528772+0000","172.21.15.120:0/3224512035":"2024-09-20T08:53:51.528772+0000","172.21.15.120:0/3465105835":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/3317449610":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/2800697451":"2024-09-20T08:53:28.144376+0000","172.21.15.120:6801/2938446665":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/109127432":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6800/2938446665":"2024-09-20T08:53:28.144376+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:01:11.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:11.128+0000 7f1daddaa640 1 -- 172.21.15.120:0/1997311681 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1d940a32d0 msgr2=0x7f1d940a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:11.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:11.128+0000 7f1daddaa640 1 --2- 172.21.15.120:0/1997311681 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1d940a32d0 0x7f1d940a5790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1da806b2e0 tx=0x7f1d90008040 comp rx=0 tx=0).stop 2024-09-19T09:01:11.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:11.128+0000 7f1daddaa640 1 -- 172.21.15.120:0/1997311681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1da8100a80 msgr2=0x7f1da81992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:11.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:11.128+0000 7f1daddaa640 1 --2- 172.21.15.120:0/1997311681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1da8100a80 0x7f1da81992f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1d9800c970 tx=0x7f1d9800ce30 comp rx=0 tx=0).stop 2024-09-19T09:01:11.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:11.128+0000 7f1daddaa640 1 -- 172.21.15.120:0/1997311681 shutdown_connections 2024-09-19T09:01:11.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:11.128+0000 7f1daddaa640 1 --2- 172.21.15.120:0/1997311681 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1d940a32d0 0x7f1d940a5790 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:11.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:11.128+0000 7f1daddaa640 1 --2- 172.21.15.120:0/1997311681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1da8106a80 0x7f1da81a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:11.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:11.128+0000 7f1daddaa640 1 --2- 172.21.15.120:0/1997311681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1da8101420 0x7f1da8199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:11.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:11.128+0000 7f1daddaa640 1 --2- 172.21.15.120:0/1997311681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1da8100a80 0x7f1da81992f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:11.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:11.128+0000 7f1daddaa640 1 -- 172.21.15.120:0/1997311681 >> 172.21.15.120:0/1997311681 conn(0x7f1da80fc7a0 msgr2=0x7f1da80fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:11.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:11.128+0000 7f1daddaa640 1 -- 172.21.15.120:0/1997311681 shutdown_connections 2024-09-19T09:01:11.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:11.128+0000 7f1daddaa640 1 -- 172.21.15.120:0/1997311681 wait complete. 2024-09-19T09:01:11.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:11 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1997311681' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:01:11.857 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T09:01:11.857 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T09:01:11.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:11 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1997311681' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:01:11.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:11 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1997311681' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:01:12.098 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:12.509 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:12 smithi120 ceph-mon[31150]: pgmap v289: 97 pgs: 97 active+clean; 632 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 20 op/s 2024-09-19T09:01:12.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.587+0000 7f7f22c48640 1 -- 172.21.15.120:0/3371559714 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f1c106f00 msgr2=0x7f7f1c10ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:12.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.587+0000 7f7f22c48640 1 --2- 172.21.15.120:0/3371559714 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f1c106f00 0x7f7f1c10ea80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7f14009ca0 tx=0x7f7f1402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:01:12.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.587+0000 7f7f22c48640 1 -- 172.21.15.120:0/3371559714 shutdown_connections 2024-09-19T09:01:12.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.587+0000 7f7f22c48640 1 --2- 172.21.15.120:0/3371559714 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f1c106f00 0x7f7f1c10ea80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:12.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.587+0000 7f7f22c48640 1 --2- 172.21.15.120:0/3371559714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f1c102780 0x7f7f1c1067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:12.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.587+0000 7f7f22c48640 1 --2- 172.21.15.120:0/3371559714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f1c101dd0 0x7f7f1c1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:12.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.587+0000 7f7f22c48640 1 -- 172.21.15.120:0/3371559714 >> 172.21.15.120:0/3371559714 conn(0x7f7f1c0fd8b0 msgr2=0x7f7f1c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:12.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.587+0000 7f7f22c48640 1 -- 172.21.15.120:0/3371559714 shutdown_connections 2024-09-19T09:01:12.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.587+0000 7f7f22c48640 1 -- 172.21.15.120:0/3371559714 wait complete. 2024-09-19T09:01:12.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.588+0000 7f7f22c48640 1 Processor -- start 2024-09-19T09:01:12.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.588+0000 7f7f22c48640 1 -- start start 2024-09-19T09:01:12.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.588+0000 7f7f22c48640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f1c101dd0 0x7f7f1c19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:12.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.588+0000 7f7f22c48640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f1c102780 0x7f7f1c19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:12.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.588+0000 7f7f22c48640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f1c106f00 0x7f7f1c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:12.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.588+0000 7f7f22c48640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c114180 con 0x7f7f1c101dd0 2024-09-19T09:01:12.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.588+0000 7f7f22c48640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c114000 con 0x7f7f1c106f00 2024-09-19T09:01:12.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.588+0000 7f7f22c48640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c114300 con 0x7f7f1c102780 2024-09-19T09:01:12.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.588+0000 7f7f209e4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f1c101dd0 0x7f7f1c19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:12.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.588+0000 7f7f209e4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f1c101dd0 0x7f7f1c19a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42774/0 (socket says 172.21.15.120:42774) 2024-09-19T09:01:12.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.588+0000 7f7f209e4640 1 -- 172.21.15.120:0/1471252935 learned_addr learned my addr 172.21.15.120:0/1471252935 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:12.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.588+0000 7f7f211e5640 1 --2- 172.21.15.120:0/1471252935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f1c106f00 0x7f7f1c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:12.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.588+0000 7f7f1bfff640 1 --2- 172.21.15.120:0/1471252935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f1c102780 0x7f7f1c19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:12.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.588+0000 7f7f209e4640 1 -- 172.21.15.120:0/1471252935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f1c102780 msgr2=0x7f7f1c19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:12.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.588+0000 7f7f209e4640 1 --2- 172.21.15.120:0/1471252935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f1c102780 0x7f7f1c19abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:12.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.588+0000 7f7f209e4640 1 -- 172.21.15.120:0/1471252935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f1c106f00 msgr2=0x7f7f1c19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:12.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.588+0000 7f7f209e4640 1 --2- 172.21.15.120:0/1471252935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f1c106f00 0x7f7f1c19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:12.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.588+0000 7f7f209e4640 1 -- 172.21.15.120:0/1471252935 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1c19f630 con 0x7f7f1c101dd0 2024-09-19T09:01:12.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.588+0000 7f7f1bfff640 1 --2- 172.21.15.120:0/1471252935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f1c102780 0x7f7f1c19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:01:12.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.589+0000 7f7f209e4640 1 --2- 172.21.15.120:0/1471252935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f1c101dd0 0x7f7f1c19a680 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000d950 tx=0x7f7f1000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:12.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.589+0000 7f7f19ffb640 1 -- 172.21.15.120:0/1471252935 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10017070 con 0x7f7f1c101dd0 2024-09-19T09:01:12.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.589+0000 7f7f22c48640 1 -- 172.21.15.120:0/1471252935 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f1c19f920 con 0x7f7f1c101dd0 2024-09-19T09:01:12.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.589+0000 7f7f19ffb640 1 -- 172.21.15.120:0/1471252935 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f10009c30 con 0x7f7f1c101dd0 2024-09-19T09:01:12.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.589+0000 7f7f19ffb640 1 -- 172.21.15.120:0/1471252935 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1000f6b0 con 0x7f7f1c101dd0 2024-09-19T09:01:12.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.589+0000 7f7f22c48640 1 -- 172.21.15.120:0/1471252935 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f1c1a7200 con 0x7f7f1c101dd0 2024-09-19T09:01:12.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.590+0000 7f7f22c48640 1 -- 172.21.15.120:0/1471252935 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f1c1021b0 con 0x7f7f1c101dd0 2024-09-19T09:01:12.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.591+0000 7f7f19ffb640 1 -- 172.21.15.120:0/1471252935 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f7f100109d0 con 0x7f7f1c101dd0 2024-09-19T09:01:12.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.592+0000 7f7f19ffb640 1 --2- 172.21.15.120:0/1471252935 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7f0c0a32d0 0x7f7f0c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:12.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.592+0000 7f7f1bfff640 1 --2- 172.21.15.120:0/1471252935 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7f0c0a32d0 0x7f7f0c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:12.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.592+0000 7f7f19ffb640 1 -- 172.21.15.120:0/1471252935 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f7f100cf610 con 0x7f7f1c101dd0 2024-09-19T09:01:12.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.592+0000 7f7f1bfff640 1 --2- 172.21.15.120:0/1471252935 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7f0c0a32d0 0x7f7f0c0a5790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7f08006fd0 tx=0x7f7f08008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:12.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.596+0000 7f7f19ffb640 1 -- 172.21.15.120:0/1471252935 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7f10009de0 con 0x7f7f1c101dd0 2024-09-19T09:01:12.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.760+0000 7f7f22c48640 1 -- 172.21.15.120:0/1471252935 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7f1c062630 con 0x7f7f0c0a32d0 2024-09-19T09:01:12.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.768+0000 7f7f19ffb640 1 -- 172.21.15.120:0/1471252935 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258866 (secure 0 0 0) 0x7f7f1c062630 con 0x7f7f0c0a32d0 2024-09-19T09:01:12.771 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:01:12.772 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-19T09:01:12.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.770+0000 7f7f22c48640 1 -- 172.21.15.120:0/1471252935 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7f0c0a32d0 msgr2=0x7f7f0c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:12.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.770+0000 7f7f22c48640 1 --2- 172.21.15.120:0/1471252935 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7f0c0a32d0 0x7f7f0c0a5790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7f08006fd0 tx=0x7f7f08008040 comp rx=0 tx=0).stop 2024-09-19T09:01:12.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.770+0000 7f7f22c48640 1 -- 172.21.15.120:0/1471252935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f1c101dd0 msgr2=0x7f7f1c19a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:12.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.770+0000 7f7f22c48640 1 --2- 172.21.15.120:0/1471252935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f1c101dd0 0x7f7f1c19a680 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000d950 tx=0x7f7f1000de10 comp rx=0 tx=0).stop 2024-09-19T09:01:12.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.770+0000 7f7f22c48640 1 -- 172.21.15.120:0/1471252935 shutdown_connections 2024-09-19T09:01:12.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.770+0000 7f7f22c48640 1 --2- 172.21.15.120:0/1471252935 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7f0c0a32d0 0x7f7f0c0a5790 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:12.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.770+0000 7f7f22c48640 1 --2- 172.21.15.120:0/1471252935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f1c106f00 0x7f7f1c19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:12.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.770+0000 7f7f22c48640 1 --2- 172.21.15.120:0/1471252935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f1c102780 0x7f7f1c19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:12.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.770+0000 7f7f22c48640 1 --2- 172.21.15.120:0/1471252935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f1c101dd0 0x7f7f1c19a680 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:12.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.770+0000 7f7f22c48640 1 -- 172.21.15.120:0/1471252935 >> 172.21.15.120:0/1471252935 conn(0x7f7f1c0fd8b0 msgr2=0x7f7f1c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:12.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.770+0000 7f7f22c48640 1 -- 172.21.15.120:0/1471252935 shutdown_connections 2024-09-19T09:01:12.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:12.770+0000 7f7f22c48640 1 -- 172.21.15.120:0/1471252935 wait complete. 2024-09-19T09:01:12.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:12 smithi196 ceph-mon[35150]: pgmap v289: 97 pgs: 97 active+clean; 632 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 20 op/s 2024-09-19T09:01:12.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:12 smithi153 ceph-mon[35071]: pgmap v289: 97 pgs: 97 active+clean; 632 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 20 op/s 2024-09-19T09:01:13.637 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-19T09:01:11.613881+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647020,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":267,"num_read_kb":245,"num_write":410,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":338,"ondisk_log_size":338,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364012,"kb_used_data":10604,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577812,"statfs":{"total":1151940427776,"available":1151567679488,"internally_reserved":0,"allocated":10858496,"data_stored":6840864,"data_compressed":107721,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":50226,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":141,"num_read_kb":136,"num_write":109,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002793"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.969591+0000","last_change":"2024-09-19T09:00:46.966628+0000","last_active":"2024-09-19T09:00:47.969591+0000","last_peered":"2024-09-19T09:00:47.969591+0000","last_clean":"2024-09-19T09:00:47.969591+0000","last_became_active":"2024-09-19T09:00:46.966290+0000","last_became_peered":"2024-09-19T09:00:46.966290+0000","last_unstale":"2024-09-19T09:00:47.969591+0000","last_undegraded":"2024-09-19T09:00:47.969591+0000","last_fullsized":"2024-09-19T09:00:47.969591+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:42.717139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964905+0000","last_change":"2024-09-19T09:00:46.963706+0000","last_active":"2024-09-19T09:00:47.964905+0000","last_peered":"2024-09-19T09:00:47.964905+0000","last_clean":"2024-09-19T09:00:47.964905+0000","last_became_active":"2024-09-19T09:00:46.963303+0000","last_became_peered":"2024-09-19T09:00:46.963303+0000","last_unstale":"2024-09-19T09:00:47.964905+0000","last_undegraded":"2024-09-19T09:00:47.964905+0000","last_fullsized":"2024-09-19T09:00:47.964905+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:07.451500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.963870+0000","last_change":"2024-09-19T09:00:46.967149+0000","last_active":"2024-09-19T09:00:47.963870+0000","last_peered":"2024-09-19T09:00:47.963870+0000","last_clean":"2024-09-19T09:00:47.963870+0000","last_became_active":"2024-09-19T09:00:46.966861+0000","last_became_peered":"2024-09-19T09:00:46.966861+0000","last_unstale":"2024-09-19T09:00:47.963870+0000","last_undegraded":"2024-09-19T09:00:47.963870+0000","last_fullsized":"2024-09-19T09:00:47.963870+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:28.396022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964150+0000","last_change":"2024-09-19T09:00:46.969242+0000","last_active":"2024-09-19T09:00:47.964150+0000","last_peered":"2024-09-19T09:00:47.964150+0000","last_clean":"2024-09-19T09:00:47.964150+0000","last_became_active":"2024-09-19T09:00:46.968546+0000","last_became_peered":"2024-09-19T09:00:46.968546+0000","last_unstale":"2024-09-19T09:00:47.964150+0000","last_undegraded":"2024-09-19T09:00:47.964150+0000","last_fullsized":"2024-09-19T09:00:47.964150+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:43:51.060130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.953778+0000","last_change":"2024-09-19T09:00:46.955501+0000","last_active":"2024-09-19T09:00:47.953778+0000","last_peered":"2024-09-19T09:00:47.953778+0000","last_clean":"2024-09-19T09:00:47.953778+0000","last_became_active":"2024-09-19T09:00:46.955161+0000","last_became_peered":"2024-09-19T09:00:46.955161+0000","last_unstale":"2024-09-19T09:00:47.953778+0000","last_undegraded":"2024-09-19T09:00:47.953778+0000","last_fullsized":"2024-09-19T09:00:47.953778+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:39:19.005931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.965095+0000","last_change":"2024-09-19T09:00:46.962577+0000","last_active":"2024-09-19T09:00:47.965095+0000","last_peered":"2024-09-19T09:00:47.965095+0000","last_clean":"2024-09-19T09:00:47.965095+0000","last_became_active":"2024-09-19T09:00:46.962253+0000","last_became_peered":"2024-09-19T09:00:46.962253+0000","last_unstale":"2024-09-19T09:00:47.965095+0000","last_undegraded":"2024-09-19T09:00:47.965095+0000","last_fullsized":"2024-09-19T09:00:47.965095+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:27.925932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964437+0000","last_change":"2024-09-19T09:00:46.960004+0000","last_active":"2024-09-19T09:00:47.964437+0000","last_peered":"2024-09-19T09:00:47.964437+0000","last_clean":"2024-09-19T09:00:47.964437+0000","last_became_active":"2024-09-19T09:00:46.959664+0000","last_became_peered":"2024-09-19T09:00:46.959664+0000","last_unstale":"2024-09-19T09:00:47.964437+0000","last_undegraded":"2024-09-19T09:00:47.964437+0000","last_fullsized":"2024-09-19T09:00:47.964437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:41.616309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964731+0000","last_change":"2024-09-19T09:00:46.952394+0000","last_active":"2024-09-19T09:00:47.964731+0000","last_peered":"2024-09-19T09:00:47.964731+0000","last_clean":"2024-09-19T09:00:47.964731+0000","last_became_active":"2024-09-19T09:00:46.952084+0000","last_became_peered":"2024-09-19T09:00:46.952084+0000","last_unstale":"2024-09-19T09:00:47.964731+0000","last_undegraded":"2024-09-19T09:00:47.964731+0000","last_fullsized":"2024-09-19T09:00:47.964731+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:21.698423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.953905+0000","last_change":"2024-09-19T09:00:46.961262+0000","last_active":"2024-09-19T09:00:47.953905+0000","last_peered":"2024-09-19T09:00:47.953905+0000","last_clean":"2024-09-19T09:00:47.953905+0000","last_became_active":"2024-09-19T09:00:46.960951+0000","last_became_peered":"2024-09-19T09:00:46.960951+0000","last_unstale":"2024-09-19T09:00:47.953905+0000","last_undegraded":"2024-09-19T09:00:47.953905+0000","last_fullsized":"2024-09-19T09:00:47.953905+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:02.445108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.966156+0000","last_change":"2024-09-19T09:00:46.964806+0000","last_active":"2024-09-19T09:00:47.966156+0000","last_peered":"2024-09-19T09:00:47.966156+0000","last_clean":"2024-09-19T09:00:47.966156+0000","last_became_active":"2024-09-19T09:00:46.964194+0000","last_became_peered":"2024-09-19T09:00:46.964194+0000","last_unstale":"2024-09-19T09:00:47.966156+0000","last_undegraded":"2024-09-19T09:00:47.966156+0000","last_fullsized":"2024-09-19T09:00:47.966156+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:31.621075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.966443+0000","last_change":"2024-09-19T09:00:46.963200+0000","last_active":"2024-09-19T09:00:47.966443+0000","last_peered":"2024-09-19T09:00:47.966443+0000","last_clean":"2024-09-19T09:00:47.966443+0000","last_became_active":"2024-09-19T09:00:46.960496+0000","last_became_peered":"2024-09-19T09:00:46.960496+0000","last_unstale":"2024-09-19T09:00:47.966443+0000","last_undegraded":"2024-09-19T09:00:47.966443+0000","last_fullsized":"2024-09-19T09:00:47.966443+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:59:47.234492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964182+0000","last_change":"2024-09-19T09:00:46.966309+0000","last_active":"2024-09-19T09:00:47.964182+0000","last_peered":"2024-09-19T09:00:47.964182+0000","last_clean":"2024-09-19T09:00:47.964182+0000","last_became_active":"2024-09-19T09:00:46.965970+0000","last_became_peered":"2024-09-19T09:00:46.965970+0000","last_unstale":"2024-09-19T09:00:47.964182+0000","last_undegraded":"2024-09-19T09:00:47.964182+0000","last_fullsized":"2024-09-19T09:00:47.964182+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:38.353206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.952739+0000","last_change":"2024-09-19T09:00:46.954079+0000","last_active":"2024-09-19T09:00:47.952739+0000","last_peered":"2024-09-19T09:00:47.952739+0000","last_clean":"2024-09-19T09:00:47.952739+0000","last_became_active":"2024-09-19T09:00:46.953398+0000","last_became_peered":"2024-09-19T09:00:46.953398+0000","last_unstale":"2024-09-19T09:00:47.952739+0000","last_undegraded":"2024-09-19T09:00:47.952739+0000","last_fullsized":"2024-09-19T09:00:47.952739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:54:27.554800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.963727+0000","last_change":"2024-09-19T09:00:46.965347+0000","last_active":"2024-09-19T09:00:47.963727+0000","last_peered":"2024-09-19T09:00:47.963727+0000","last_clean":"2024-09-19T09:00:47.963727+0000","last_became_active":"2024-09-19T09:00:46.965045+0000","last_became_peered":"2024-09-19T09:00:46.965045+0000","last_unstale":"2024-09-19T09:00:47.963727+0000","last_undegraded":"2024-09-19T09:00:47.963727+0000","last_fullsized":"2024-09-19T09:00:47.963727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:27.316100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964176+0000","last_change":"2024-09-19T09:00:46.962898+0000","last_active":"2024-09-19T09:00:47.964176+0000","last_peered":"2024-09-19T09:00:47.964176+0000","last_clean":"2024-09-19T09:00:47.964176+0000","last_became_active":"2024-09-19T09:00:46.962412+0000","last_became_peered":"2024-09-19T09:00:46.962412+0000","last_unstale":"2024-09-19T09:00:47.964176+0000","last_undegraded":"2024-09-19T09:00:47.964176+0000","last_fullsized":"2024-09-19T09:00:47.964176+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:40.483678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.953179+0000","last_change":"2024-09-19T09:00:46.958147+0000","last_active":"2024-09-19T09:00:47.953179+0000","last_peered":"2024-09-19T09:00:47.953179+0000","last_clean":"2024-09-19T09:00:47.953179+0000","last_became_active":"2024-09-19T09:00:46.957805+0000","last_became_peered":"2024-09-19T09:00:46.957805+0000","last_unstale":"2024-09-19T09:00:47.953179+0000","last_undegraded":"2024-09-19T09:00:47.953179+0000","last_fullsized":"2024-09-19T09:00:47.953179+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:24:17.000064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.969691+0000","last_change":"2024-09-19T09:00:46.967399+0000","last_active":"2024-09-19T09:00:47.969691+0000","last_peered":"2024-09-19T09:00:47.969691+0000","last_clean":"2024-09-19T09:00:47.969691+0000","last_became_active":"2024-09-19T09:00:46.966145+0000","last_became_peered":"2024-09-19T09:00:46.966145+0000","last_unstale":"2024-09-19T09:00:47.969691+0000","last_undegraded":"2024-09-19T09:00:47.969691+0000","last_fullsized":"2024-09-19T09:00:47.969691+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:29.192469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.962791+0000","last_change":"2024-09-19T09:00:46.960956+0000","last_active":"2024-09-19T09:00:47.962791+0000","last_peered":"2024-09-19T09:00:47.962791+0000","last_clean":"2024-09-19T09:00:47.962791+0000","last_became_active":"2024-09-19T09:00:46.960664+0000","last_became_peered":"2024-09-19T09:00:46.960664+0000","last_unstale":"2024-09-19T09:00:47.962791+0000","last_undegraded":"2024-09-19T09:00:47.962791+0000","last_fullsized":"2024-09-19T09:00:47.962791+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:18.724348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.965529+0000","last_change":"2024-09-19T09:00:46.951362+0000","last_active":"2024-09-19T09:00:47.965529+0000","last_peered":"2024-09-19T09:00:47.965529+0000","last_clean":"2024-09-19T09:00:47.965529+0000","last_became_active":"2024-09-19T09:00:46.951004+0000","last_became_peered":"2024-09-19T09:00:46.951004+0000","last_unstale":"2024-09-19T09:00:47.965529+0000","last_undegraded":"2024-09-19T09:00:47.965529+0000","last_fullsized":"2024-09-19T09:00:47.965529+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:43.574425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.969012+0000","last_change":"2024-09-19T09:00:46.966752+0000","last_active":"2024-09-19T09:00:47.969012+0000","last_peered":"2024-09-19T09:00:47.969012+0000","last_clean":"2024-09-19T09:00:47.969012+0000","last_became_active":"2024-09-19T09:00:46.966370+0000","last_became_peered":"2024-09-19T09:00:46.966370+0000","last_unstale":"2024-09-19T09:00:47.969012+0000","last_undegraded":"2024-09-19T09:00:47.969012+0000","last_fullsized":"2024-09-19T09:00:47.969012+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:04.035771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.952829+0000","last_change":"2024-09-19T09:00:46.956369+0000","last_active":"2024-09-19T09:00:47.952829+0000","last_peered":"2024-09-19T09:00:47.952829+0000","last_clean":"2024-09-19T09:00:47.952829+0000","last_became_active":"2024-09-19T09:00:46.956075+0000","last_became_peered":"2024-09-19T09:00:46.956075+0000","last_unstale":"2024-09-19T09:00:47.952829+0000","last_undegraded":"2024-09-19T09:00:47.952829+0000","last_fullsized":"2024-09-19T09:00:47.952829+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:23.985017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.953432+0000","last_change":"2024-09-19T09:00:46.956661+0000","last_active":"2024-09-19T09:00:47.953432+0000","last_peered":"2024-09-19T09:00:47.953432+0000","last_clean":"2024-09-19T09:00:47.953432+0000","last_became_active":"2024-09-19T09:00:46.956291+0000","last_became_peered":"2024-09-19T09:00:46.956291+0000","last_unstale":"2024-09-19T09:00:47.953432+0000","last_undegraded":"2024-09-19T09:00:47.953432+0000","last_fullsized":"2024-09-19T09:00:47.953432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:55.713039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964577+0000","last_change":"2024-09-19T09:00:46.961866+0000","last_active":"2024-09-19T09:00:47.964577+0000","last_peered":"2024-09-19T09:00:47.964577+0000","last_clean":"2024-09-19T09:00:47.964577+0000","last_became_active":"2024-09-19T09:00:46.961469+0000","last_became_peered":"2024-09-19T09:00:46.961469+0000","last_unstale":"2024-09-19T09:00:47.964577+0000","last_undegraded":"2024-09-19T09:00:47.964577+0000","last_fullsized":"2024-09-19T09:00:47.964577+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:47.665419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.967694+0000","last_change":"2024-09-19T09:00:46.948765+0000","last_active":"2024-09-19T09:00:47.967694+0000","last_peered":"2024-09-19T09:00:47.967694+0000","last_clean":"2024-09-19T09:00:47.967694+0000","last_became_active":"2024-09-19T09:00:46.948431+0000","last_became_peered":"2024-09-19T09:00:46.948431+0000","last_unstale":"2024-09-19T09:00:47.967694+0000","last_undegraded":"2024-09-19T09:00:47.967694+0000","last_fullsized":"2024-09-19T09:00:47.967694+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:31.766464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964637+0000","last_change":"2024-09-19T09:00:46.962433+0000","last_active":"2024-09-19T09:00:47.964637+0000","last_peered":"2024-09-19T09:00:47.964637+0000","last_clean":"2024-09-19T09:00:47.964637+0000","last_became_active":"2024-09-19T09:00:46.962153+0000","last_became_peered":"2024-09-19T09:00:46.962153+0000","last_unstale":"2024-09-19T09:00:47.964637+0000","last_undegraded":"2024-09-19T09:00:47.964637+0000","last_fullsized":"2024-09-19T09:00:47.964637+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:37.973611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.965774+0000","last_change":"2024-09-19T09:00:46.953715+0000","last_active":"2024-09-19T09:00:47.965774+0000","last_peered":"2024-09-19T09:00:47.965774+0000","last_clean":"2024-09-19T09:00:47.965774+0000","last_became_active":"2024-09-19T09:00:46.953415+0000","last_became_peered":"2024-09-19T09:00:46.953415+0000","last_unstale":"2024-09-19T09:00:47.965774+0000","last_undegraded":"2024-09-19T09:00:47.965774+0000","last_fullsized":"2024-09-19T09:00:47.965774+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:29.768382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.969440+0000","last_change":"2024-09-19T09:00:46.964746+0000","last_active":"2024-09-19T09:00:47.969440+0000","last_peered":"2024-09-19T09:00:47.969440+0000","last_clean":"2024-09-19T09:00:47.969440+0000","last_became_active":"2024-09-19T09:00:46.964404+0000","last_became_peered":"2024-09-19T09:00:46.964404+0000","last_unstale":"2024-09-19T09:00:47.969440+0000","last_undegraded":"2024-09-19T09:00:47.969440+0000","last_fullsized":"2024-09-19T09:00:47.969440+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:19.841495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.952789+0000","last_change":"2024-09-19T09:00:46.962875+0000","last_active":"2024-09-19T09:00:47.952789+0000","last_peered":"2024-09-19T09:00:47.952789+0000","last_clean":"2024-09-19T09:00:47.952789+0000","last_became_active":"2024-09-19T09:00:46.962561+0000","last_became_peered":"2024-09-19T09:00:46.962561+0000","last_unstale":"2024-09-19T09:00:47.952789+0000","last_undegraded":"2024-09-19T09:00:47.952789+0000","last_fullsized":"2024-09-19T09:00:47.952789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:39.675545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964406+0000","last_change":"2024-09-19T09:00:46.963126+0000","last_active":"2024-09-19T09:00:47.964406+0000","last_peered":"2024-09-19T09:00:47.964406+0000","last_clean":"2024-09-19T09:00:47.964406+0000","last_became_active":"2024-09-19T09:00:46.962620+0000","last_became_peered":"2024-09-19T09:00:46.962620+0000","last_unstale":"2024-09-19T09:00:47.964406+0000","last_undegraded":"2024-09-19T09:00:47.964406+0000","last_fullsized":"2024-09-19T09:00:47.964406+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:59.723280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964649+0000","last_change":"2024-09-19T09:00:46.953026+0000","last_active":"2024-09-19T09:00:47.964649+0000","last_peered":"2024-09-19T09:00:47.964649+0000","last_clean":"2024-09-19T09:00:47.964649+0000","last_became_active":"2024-09-19T09:00:46.952596+0000","last_became_peered":"2024-09-19T09:00:46.952596+0000","last_unstale":"2024-09-19T09:00:47.964649+0000","last_undegraded":"2024-09-19T09:00:47.964649+0000","last_fullsized":"2024-09-19T09:00:47.964649+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:58.524116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964488+0000","last_change":"2024-09-19T09:00:46.968964+0000","last_active":"2024-09-19T09:00:47.964488+0000","last_peered":"2024-09-19T09:00:47.964488+0000","last_clean":"2024-09-19T09:00:47.964488+0000","last_became_active":"2024-09-19T09:00:46.968505+0000","last_became_peered":"2024-09-19T09:00:46.968505+0000","last_unstale":"2024-09-19T09:00:47.964488+0000","last_undegraded":"2024-09-19T09:00:47.964488+0000","last_fullsized":"2024-09-19T09:00:47.964488+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:17:33.335541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.969060+0000","last_change":"2024-09-19T09:00:46.961520+0000","last_active":"2024-09-19T09:00:47.969060+0000","last_peered":"2024-09-19T09:00:47.969060+0000","last_clean":"2024-09-19T09:00:47.969060+0000","last_became_active":"2024-09-19T09:00:46.961140+0000","last_became_peered":"2024-09-19T09:00:46.961140+0000","last_unstale":"2024-09-19T09:00:47.969060+0000","last_undegraded":"2024-09-19T09:00:47.969060+0000","last_fullsized":"2024-09-19T09:00:47.969060+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:03.120173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.953710+0000","last_change":"2024-09-19T09:00:46.955995+0000","last_active":"2024-09-19T09:00:47.953710+0000","last_peered":"2024-09-19T09:00:47.953710+0000","last_clean":"2024-09-19T09:00:47.953710+0000","last_became_active":"2024-09-19T09:00:46.954678+0000","last_became_peered":"2024-09-19T09:00:46.954678+0000","last_unstale":"2024-09-19T09:00:47.953710+0000","last_undegraded":"2024-09-19T09:00:47.953710+0000","last_fullsized":"2024-09-19T09:00:47.953710+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:10.314575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:01:06.154929+0000","last_change":"2024-09-19T09:00:44.946000+0000","last_active":"2024-09-19T09:01:06.154929+0000","last_peered":"2024-09-19T09:01:06.154929+0000","last_clean":"2024-09-19T09:01:06.154929+0000","last_became_active":"2024-09-19T09:00:44.945583+0000","last_became_peered":"2024-09-19T09:00:44.945583+0000","last_unstale":"2024-09-19T09:01:06.154929+0000","last_undegraded":"2024-09-19T09:01:06.154929+0000","last_fullsized":"2024-09-19T09:01:06.154929+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:53:19.109697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.967983+0000","last_change":"2024-09-19T09:00:46.962774+0000","last_active":"2024-09-19T09:00:47.967983+0000","last_peered":"2024-09-19T09:00:47.967983+0000","last_clean":"2024-09-19T09:00:47.967983+0000","last_became_active":"2024-09-19T09:00:46.962435+0000","last_became_peered":"2024-09-19T09:00:46.962435+0000","last_unstale":"2024-09-19T09:00:47.967983+0000","last_undegraded":"2024-09-19T09:00:47.967983+0000","last_fullsized":"2024-09-19T09:00:47.967983+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:55.481141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:57.126925+0000","last_change":"2024-09-19T09:00:44.942435+0000","last_active":"2024-09-19T09:00:57.126925+0000","last_peered":"2024-09-19T09:00:57.126925+0000","last_clean":"2024-09-19T09:00:57.126925+0000","last_became_active":"2024-09-19T09:00:44.942067+0000","last_became_peered":"2024-09-19T09:00:44.942067+0000","last_unstale":"2024-09-19T09:00:57.126925+0000","last_undegraded":"2024-09-19T09:00:57.126925+0000","last_fullsized":"2024-09-19T09:00:57.126925+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:16.476506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:56.118780+0000","last_change":"2024-09-19T09:00:44.937944+0000","last_active":"2024-09-19T09:00:56.118780+0000","last_peered":"2024-09-19T09:00:56.118780+0000","last_clean":"2024-09-19T09:00:56.118780+0000","last_became_active":"2024-09-19T09:00:44.937623+0000","last_became_peered":"2024-09-19T09:00:44.937623+0000","last_unstale":"2024-09-19T09:00:56.118780+0000","last_undegraded":"2024-09-19T09:00:56.118780+0000","last_fullsized":"2024-09-19T09:00:56.118780+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:49:08.730015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964590+0000","last_change":"2024-09-19T09:00:46.959932+0000","last_active":"2024-09-19T09:00:47.964590+0000","last_peered":"2024-09-19T09:00:47.964590+0000","last_clean":"2024-09-19T09:00:47.964590+0000","last_became_active":"2024-09-19T09:00:46.959637+0000","last_became_peered":"2024-09-19T09:00:46.959637+0000","last_unstale":"2024-09-19T09:00:47.964590+0000","last_undegraded":"2024-09-19T09:00:47.964590+0000","last_fullsized":"2024-09-19T09:00:47.964590+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:55.337669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.140550+0000","last_change":"2024-09-19T09:00:44.944882+0000","last_active":"2024-09-19T09:00:58.140550+0000","last_peered":"2024-09-19T09:00:58.140550+0000","last_clean":"2024-09-19T09:00:58.140550+0000","last_became_active":"2024-09-19T09:00:44.944116+0000","last_became_peered":"2024-09-19T09:00:44.944116+0000","last_unstale":"2024-09-19T09:00:58.140550+0000","last_undegraded":"2024-09-19T09:00:58.140550+0000","last_fullsized":"2024-09-19T09:00:58.140550+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:54:58.555913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.962700+0000","last_change":"2024-09-19T09:00:46.956859+0000","last_active":"2024-09-19T09:00:47.962700+0000","last_peered":"2024-09-19T09:00:47.962700+0000","last_clean":"2024-09-19T09:00:47.962700+0000","last_became_active":"2024-09-19T09:00:46.956561+0000","last_became_peered":"2024-09-19T09:00:46.956561+0000","last_unstale":"2024-09-19T09:00:47.962700+0000","last_undegraded":"2024-09-19T09:00:47.962700+0000","last_fullsized":"2024-09-19T09:00:47.962700+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:09:04.125591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":46,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:01:10.159859+0000","last_change":"2024-09-19T09:00:44.945959+0000","last_active":"2024-09-19T09:01:10.159859+0000","last_peered":"2024-09-19T09:01:10.159859+0000","last_clean":"2024-09-19T09:01:10.159859+0000","last_became_active":"2024-09-19T09:00:44.945640+0000","last_became_peered":"2024-09-19T09:00:44.945640+0000","last_unstale":"2024-09-19T09:01:10.159859+0000","last_undegraded":"2024-09-19T09:01:10.159859+0000","last_fullsized":"2024-09-19T09:01:10.159859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:02.523834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.953728+0000","last_change":"2024-09-19T09:00:46.953677+0000","last_active":"2024-09-19T09:00:47.953728+0000","last_peered":"2024-09-19T09:00:47.953728+0000","last_clean":"2024-09-19T09:00:47.953728+0000","last_became_active":"2024-09-19T09:00:46.953357+0000","last_became_peered":"2024-09-19T09:00:46.953357+0000","last_unstale":"2024-09-19T09:00:47.953728+0000","last_undegraded":"2024-09-19T09:00:47.953728+0000","last_fullsized":"2024-09-19T09:00:47.953728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:48.963003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.139991+0000","last_change":"2024-09-19T09:00:44.945777+0000","last_active":"2024-09-19T09:00:58.139991+0000","last_peered":"2024-09-19T09:00:58.139991+0000","last_clean":"2024-09-19T09:00:58.139991+0000","last_became_active":"2024-09-19T09:00:44.945450+0000","last_became_peered":"2024-09-19T09:00:44.945450+0000","last_unstale":"2024-09-19T09:00:58.139991+0000","last_undegraded":"2024-09-19T09:00:58.139991+0000","last_fullsized":"2024-09-19T09:00:58.139991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:44.656451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.965513+0000","last_change":"2024-09-19T09:00:46.962285+0000","last_active":"2024-09-19T09:00:47.965513+0000","last_peered":"2024-09-19T09:00:47.965513+0000","last_clean":"2024-09-19T09:00:47.965513+0000","last_became_active":"2024-09-19T09:00:46.961982+0000","last_became_peered":"2024-09-19T09:00:46.961982+0000","last_unstale":"2024-09-19T09:00:47.965513+0000","last_undegraded":"2024-09-19T09:00:47.965513+0000","last_fullsized":"2024-09-19T09:00:47.965513+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:41.658501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:57.125258+0000","last_change":"2024-09-19T09:00:45.241425+0000","last_active":"2024-09-19T09:00:57.125258+0000","last_peered":"2024-09-19T09:00:57.125258+0000","last_clean":"2024-09-19T09:00:57.125258+0000","last_became_active":"2024-09-19T09:00:45.241032+0000","last_became_peered":"2024-09-19T09:00:45.241032+0000","last_unstale":"2024-09-19T09:00:57.125258+0000","last_undegraded":"2024-09-19T09:00:57.125258+0000","last_fullsized":"2024-09-19T09:00:57.125258+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:00.707855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.953388+0000","last_change":"2024-09-19T09:00:46.962436+0000","last_active":"2024-09-19T09:00:47.953388+0000","last_peered":"2024-09-19T09:00:47.953388+0000","last_clean":"2024-09-19T09:00:47.953388+0000","last_became_active":"2024-09-19T09:00:46.962114+0000","last_became_peered":"2024-09-19T09:00:46.962114+0000","last_unstale":"2024-09-19T09:00:47.953388+0000","last_undegraded":"2024-09-19T09:00:47.953388+0000","last_fullsized":"2024-09-19T09:00:47.953388+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:41.761705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'27","reported_seq":75,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:01:10.164619+0000","last_change":"2024-09-19T09:00:44.945187+0000","last_active":"2024-09-19T09:01:10.164619+0000","last_peered":"2024-09-19T09:01:10.164619+0000","last_clean":"2024-09-19T09:01:10.164619+0000","last_became_active":"2024-09-19T09:00:44.944763+0000","last_became_peered":"2024-09-19T09:00:44.944763+0000","last_unstale":"2024-09-19T09:01:10.164619+0000","last_undegraded":"2024-09-19T09:01:10.164619+0000","last_fullsized":"2024-09-19T09:01:10.164619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:06.052994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49603,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":30,"num_read_kb":63,"num_write":29,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.954165+0000","last_change":"2024-09-19T09:00:46.960991+0000","last_active":"2024-09-19T09:00:47.954165+0000","last_peered":"2024-09-19T09:00:47.954165+0000","last_clean":"2024-09-19T09:00:47.954165+0000","last_became_active":"2024-09-19T09:00:46.958962+0000","last_became_peered":"2024-09-19T09:00:46.958962+0000","last_unstale":"2024-09-19T09:00:47.954165+0000","last_undegraded":"2024-09-19T09:00:47.954165+0000","last_fullsized":"2024-09-19T09:00:47.954165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:21.795694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:59.142128+0000","last_change":"2024-09-19T09:00:44.946176+0000","last_active":"2024-09-19T09:00:59.142128+0000","last_peered":"2024-09-19T09:00:59.142128+0000","last_clean":"2024-09-19T09:00:59.142128+0000","last_became_active":"2024-09-19T09:00:44.945869+0000","last_became_peered":"2024-09-19T09:00:44.945869+0000","last_unstale":"2024-09-19T09:00:59.142128+0000","last_undegraded":"2024-09-19T09:00:59.142128+0000","last_fullsized":"2024-09-19T09:00:59.142128+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:16.022184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.952567+0000","last_change":"2024-09-19T09:00:46.956345+0000","last_active":"2024-09-19T09:00:47.952567+0000","last_peered":"2024-09-19T09:00:47.952567+0000","last_clean":"2024-09-19T09:00:47.952567+0000","last_became_active":"2024-09-19T09:00:46.956057+0000","last_became_peered":"2024-09-19T09:00:46.956057+0000","last_unstale":"2024-09-19T09:00:47.952567+0000","last_undegraded":"2024-09-19T09:00:47.952567+0000","last_fullsized":"2024-09-19T09:00:47.952567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:27.510661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.140102+0000","last_change":"2024-09-19T09:00:44.945667+0000","last_active":"2024-09-19T09:00:58.140102+0000","last_peered":"2024-09-19T09:00:58.140102+0000","last_clean":"2024-09-19T09:00:58.140102+0000","last_became_active":"2024-09-19T09:00:44.945308+0000","last_became_peered":"2024-09-19T09:00:44.945308+0000","last_unstale":"2024-09-19T09:00:58.140102+0000","last_undegraded":"2024-09-19T09:00:58.140102+0000","last_fullsized":"2024-09-19T09:00:58.140102+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:00:11.304913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.965242+0000","last_change":"2024-09-19T09:00:46.965088+0000","last_active":"2024-09-19T09:00:47.965242+0000","last_peered":"2024-09-19T09:00:47.965242+0000","last_clean":"2024-09-19T09:00:47.965242+0000","last_became_active":"2024-09-19T09:00:46.964777+0000","last_became_peered":"2024-09-19T09:00:46.964777+0000","last_unstale":"2024-09-19T09:00:47.965242+0000","last_undegraded":"2024-09-19T09:00:47.965242+0000","last_fullsized":"2024-09-19T09:00:47.965242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:32.958984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:01:06.157592+0000","last_change":"2024-09-19T09:00:44.946193+0000","last_active":"2024-09-19T09:01:06.157592+0000","last_peered":"2024-09-19T09:01:06.157592+0000","last_clean":"2024-09-19T09:01:06.157592+0000","last_became_active":"2024-09-19T09:00:44.945168+0000","last_became_peered":"2024-09-19T09:00:44.945168+0000","last_unstale":"2024-09-19T09:01:06.157592+0000","last_undegraded":"2024-09-19T09:01:06.157592+0000","last_fullsized":"2024-09-19T09:01:06.157592+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:21.230689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964246+0000","last_change":"2024-09-19T09:00:46.959425+0000","last_active":"2024-09-19T09:00:47.964246+0000","last_peered":"2024-09-19T09:00:47.964246+0000","last_clean":"2024-09-19T09:00:47.964246+0000","last_became_active":"2024-09-19T09:00:46.958764+0000","last_became_peered":"2024-09-19T09:00:46.958764+0000","last_unstale":"2024-09-19T09:00:47.964246+0000","last_undegraded":"2024-09-19T09:00:47.964246+0000","last_fullsized":"2024-09-19T09:00:47.964246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:08:27.371080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.140076+0000","last_change":"2024-09-19T09:00:45.240084+0000","last_active":"2024-09-19T09:00:58.140076+0000","last_peered":"2024-09-19T09:00:58.140076+0000","last_clean":"2024-09-19T09:00:58.140076+0000","last_became_active":"2024-09-19T09:00:45.239222+0000","last_became_peered":"2024-09-19T09:00:45.239222+0000","last_unstale":"2024-09-19T09:00:58.140076+0000","last_undegraded":"2024-09-19T09:00:58.140076+0000","last_fullsized":"2024-09-19T09:00:58.140076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:00:56.643063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.968850+0000","last_change":"2024-09-19T09:00:46.963380+0000","last_active":"2024-09-19T09:00:47.968850+0000","last_peered":"2024-09-19T09:00:47.968850+0000","last_clean":"2024-09-19T09:00:47.968850+0000","last_became_active":"2024-09-19T09:00:46.962605+0000","last_became_peered":"2024-09-19T09:00:46.962605+0000","last_unstale":"2024-09-19T09:00:47.968850+0000","last_undegraded":"2024-09-19T09:00:47.968850+0000","last_fullsized":"2024-09-19T09:00:47.968850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:20:59.424753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:57.115299+0000","last_change":"2024-09-19T09:00:44.946138+0000","last_active":"2024-09-19T09:00:57.115299+0000","last_peered":"2024-09-19T09:00:57.115299+0000","last_clean":"2024-09-19T09:00:57.115299+0000","last_became_active":"2024-09-19T09:00:44.945793+0000","last_became_peered":"2024-09-19T09:00:44.945793+0000","last_unstale":"2024-09-19T09:00:57.115299+0000","last_undegraded":"2024-09-19T09:00:57.115299+0000","last_fullsized":"2024-09-19T09:00:57.115299+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:01.341560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.969341+0000","last_change":"2024-09-19T09:00:46.970360+0000","last_active":"2024-09-19T09:00:47.969341+0000","last_peered":"2024-09-19T09:00:47.969341+0000","last_clean":"2024-09-19T09:00:47.969341+0000","last_became_active":"2024-09-19T09:00:46.969971+0000","last_became_peered":"2024-09-19T09:00:46.969971+0000","last_unstale":"2024-09-19T09:00:47.969341+0000","last_undegraded":"2024-09-19T09:00:47.969341+0000","last_fullsized":"2024-09-19T09:00:47.969341+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:48.812158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.139613+0000","last_change":"2024-09-19T09:00:44.941713+0000","last_active":"2024-09-19T09:00:58.139613+0000","last_peered":"2024-09-19T09:00:58.139613+0000","last_clean":"2024-09-19T09:00:58.139613+0000","last_became_active":"2024-09-19T09:00:44.941373+0000","last_became_peered":"2024-09-19T09:00:44.941373+0000","last_unstale":"2024-09-19T09:00:58.139613+0000","last_undegraded":"2024-09-19T09:00:58.139613+0000","last_fullsized":"2024-09-19T09:00:58.139613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:31.157803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.965581+0000","last_change":"2024-09-19T09:00:46.959383+0000","last_active":"2024-09-19T09:00:47.965581+0000","last_peered":"2024-09-19T09:00:47.965581+0000","last_clean":"2024-09-19T09:00:47.965581+0000","last_became_active":"2024-09-19T09:00:46.958860+0000","last_became_peered":"2024-09-19T09:00:46.958860+0000","last_unstale":"2024-09-19T09:00:47.965581+0000","last_undegraded":"2024-09-19T09:00:47.965581+0000","last_fullsized":"2024-09-19T09:00:47.965581+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:32:04.423017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.139240+0000","last_change":"2024-09-19T09:00:44.944797+0000","last_active":"2024-09-19T09:00:58.139240+0000","last_peered":"2024-09-19T09:00:58.139240+0000","last_clean":"2024-09-19T09:00:58.139240+0000","last_became_active":"2024-09-19T09:00:44.944259+0000","last_became_peered":"2024-09-19T09:00:44.944259+0000","last_unstale":"2024-09-19T09:00:58.139240+0000","last_undegraded":"2024-09-19T09:00:58.139240+0000","last_fullsized":"2024-09-19T09:00:58.139240+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:58.996217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:49.684985+0000","last_change":"2024-09-19T09:00:46.964998+0000","last_active":"2024-09-19T09:00:49.684985+0000","last_peered":"2024-09-19T09:00:49.684985+0000","last_clean":"2024-09-19T09:00:49.684985+0000","last_became_active":"2024-09-19T09:00:46.964562+0000","last_became_peered":"2024-09-19T09:00:46.964562+0000","last_unstale":"2024-09-19T09:00:49.684985+0000","last_undegraded":"2024-09-19T09:00:49.684985+0000","last_fullsized":"2024-09-19T09:00:49.684985+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:00.089800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:01:06.154761+0000","last_change":"2024-09-19T09:00:45.239770+0000","last_active":"2024-09-19T09:01:06.154761+0000","last_peered":"2024-09-19T09:01:06.154761+0000","last_clean":"2024-09-19T09:01:06.154761+0000","last_became_active":"2024-09-19T09:00:45.239385+0000","last_became_peered":"2024-09-19T09:00:45.239385+0000","last_unstale":"2024-09-19T09:01:06.154761+0000","last_undegraded":"2024-09-19T09:01:06.154761+0000","last_fullsized":"2024-09-19T09:01:06.154761+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:21.109900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.966028+0000","last_change":"2024-09-19T09:00:46.967505+0000","last_active":"2024-09-19T09:00:47.966028+0000","last_peered":"2024-09-19T09:00:47.966028+0000","last_clean":"2024-09-19T09:00:47.966028+0000","last_became_active":"2024-09-19T09:00:46.967195+0000","last_became_peered":"2024-09-19T09:00:46.967195+0000","last_unstale":"2024-09-19T09:00:47.966028+0000","last_undegraded":"2024-09-19T09:00:47.966028+0000","last_fullsized":"2024-09-19T09:00:47.966028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:52:28.917333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":330,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964506+0000","last_change":"2024-09-19T08:59:33.821924+0000","last_active":"2024-09-19T09:00:47.964506+0000","last_peered":"2024-09-19T09:00:47.964506+0000","last_clean":"2024-09-19T09:00:47.964506+0000","last_became_active":"2024-09-19T08:59:33.821404+0000","last_became_peered":"2024-09-19T08:59:33.821404+0000","last_unstale":"2024-09-19T09:00:47.964506+0000","last_undegraded":"2024-09-19T09:00:47.964506+0000","last_fullsized":"2024-09-19T09:00:47.964506+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:56:07.830480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:56:07.830480+0000","last_clean_scrub_stamp":"2024-09-19T08:56:07.830480+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:21.763918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:59.142206+0000","last_change":"2024-09-19T09:00:44.943714+0000","last_active":"2024-09-19T09:00:59.142206+0000","last_peered":"2024-09-19T09:00:59.142206+0000","last_clean":"2024-09-19T09:00:59.142206+0000","last_became_active":"2024-09-19T09:00:44.943362+0000","last_became_peered":"2024-09-19T09:00:44.943362+0000","last_unstale":"2024-09-19T09:00:59.142206+0000","last_undegraded":"2024-09-19T09:00:59.142206+0000","last_fullsized":"2024-09-19T09:00:59.142206+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:30.015648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964922+0000","last_change":"2024-09-19T09:00:46.956784+0000","last_active":"2024-09-19T09:00:47.964922+0000","last_peered":"2024-09-19T09:00:47.964922+0000","last_clean":"2024-09-19T09:00:47.964922+0000","last_became_active":"2024-09-19T09:00:46.956443+0000","last_became_peered":"2024-09-19T09:00:46.956443+0000","last_unstale":"2024-09-19T09:00:47.964922+0000","last_undegraded":"2024-09-19T09:00:47.964922+0000","last_fullsized":"2024-09-19T09:00:47.964922+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:18.918788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:56.119049+0000","last_change":"2024-09-19T09:00:44.947796+0000","last_active":"2024-09-19T09:00:56.119049+0000","last_peered":"2024-09-19T09:00:56.119049+0000","last_clean":"2024-09-19T09:00:56.119049+0000","last_became_active":"2024-09-19T09:00:44.946949+0000","last_became_peered":"2024-09-19T09:00:44.946949+0000","last_unstale":"2024-09-19T09:00:56.119049+0000","last_undegraded":"2024-09-19T09:00:56.119049+0000","last_fullsized":"2024-09-19T09:00:56.119049+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:42.210186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.952537+0000","last_change":"2024-09-19T09:00:46.963856+0000","last_active":"2024-09-19T09:00:47.952537+0000","last_peered":"2024-09-19T09:00:47.952537+0000","last_clean":"2024-09-19T09:00:47.952537+0000","last_became_active":"2024-09-19T09:00:46.963543+0000","last_became_peered":"2024-09-19T09:00:46.963543+0000","last_unstale":"2024-09-19T09:00:47.952537+0000","last_undegraded":"2024-09-19T09:00:47.952537+0000","last_fullsized":"2024-09-19T09:00:47.952537+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:31.041028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:01:07.158294+0000","last_change":"2024-09-19T09:00:45.239269+0000","last_active":"2024-09-19T09:01:07.158294+0000","last_peered":"2024-09-19T09:01:07.158294+0000","last_clean":"2024-09-19T09:01:07.158294+0000","last_became_active":"2024-09-19T09:00:45.238929+0000","last_became_peered":"2024-09-19T09:00:45.238929+0000","last_unstale":"2024-09-19T09:01:07.158294+0000","last_undegraded":"2024-09-19T09:01:07.158294+0000","last_fullsized":"2024-09-19T09:01:07.158294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:46.684148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.968730+0000","last_change":"2024-09-19T09:00:46.963595+0000","last_active":"2024-09-19T09:00:47.968730+0000","last_peered":"2024-09-19T09:00:47.968730+0000","last_clean":"2024-09-19T09:00:47.968730+0000","last_became_active":"2024-09-19T09:00:46.963243+0000","last_became_peered":"2024-09-19T09:00:46.963243+0000","last_unstale":"2024-09-19T09:00:47.968730+0000","last_undegraded":"2024-09-19T09:00:47.968730+0000","last_fullsized":"2024-09-19T09:00:47.968730+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:01.122996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":35,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:01:08.156777+0000","last_change":"2024-09-19T09:00:44.941550+0000","last_active":"2024-09-19T09:01:08.156777+0000","last_peered":"2024-09-19T09:01:08.156777+0000","last_clean":"2024-09-19T09:01:08.156777+0000","last_became_active":"2024-09-19T09:00:44.941145+0000","last_became_peered":"2024-09-19T09:00:44.941145+0000","last_unstale":"2024-09-19T09:01:08.156777+0000","last_undegraded":"2024-09-19T09:01:08.156777+0000","last_fullsized":"2024-09-19T09:01:08.156777+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:53.893887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964440+0000","last_change":"2024-09-19T09:00:46.963473+0000","last_active":"2024-09-19T09:00:47.964440+0000","last_peered":"2024-09-19T09:00:47.964440+0000","last_clean":"2024-09-19T09:00:47.964440+0000","last_became_active":"2024-09-19T09:00:46.963090+0000","last_became_peered":"2024-09-19T09:00:46.963090+0000","last_unstale":"2024-09-19T09:00:47.964440+0000","last_undegraded":"2024-09-19T09:00:47.964440+0000","last_fullsized":"2024-09-19T09:00:47.964440+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:01.492385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.962784+0000","last_change":"2024-09-19T09:00:45.239566+0000","last_active":"2024-09-19T09:00:47.962784+0000","last_peered":"2024-09-19T09:00:47.962784+0000","last_clean":"2024-09-19T09:00:47.962784+0000","last_became_active":"2024-09-19T09:00:45.239250+0000","last_became_peered":"2024-09-19T09:00:45.239250+0000","last_unstale":"2024-09-19T09:00:47.962784+0000","last_undegraded":"2024-09-19T09:00:47.962784+0000","last_fullsized":"2024-09-19T09:00:47.962784+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:44:21.739311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:50.839180+0000","last_change":"2024-09-19T09:00:46.955344+0000","last_active":"2024-09-19T09:00:50.839180+0000","last_peered":"2024-09-19T09:00:50.839180+0000","last_clean":"2024-09-19T09:00:50.839180+0000","last_became_active":"2024-09-19T09:00:46.954961+0000","last_became_peered":"2024-09-19T09:00:46.954961+0000","last_unstale":"2024-09-19T09:00:50.839180+0000","last_undegraded":"2024-09-19T09:00:50.839180+0000","last_fullsized":"2024-09-19T09:00:50.839180+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:55.743556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.141069+0000","last_change":"2024-09-19T09:00:45.241514+0000","last_active":"2024-09-19T09:00:58.141069+0000","last_peered":"2024-09-19T09:00:58.141069+0000","last_clean":"2024-09-19T09:00:58.141069+0000","last_became_active":"2024-09-19T09:00:45.241097+0000","last_became_peered":"2024-09-19T09:00:45.241097+0000","last_unstale":"2024-09-19T09:00:58.141069+0000","last_undegraded":"2024-09-19T09:00:58.141069+0000","last_fullsized":"2024-09-19T09:00:58.141069+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:35:38.323393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.952874+0000","last_change":"2024-09-19T09:00:46.955194+0000","last_active":"2024-09-19T09:00:47.952874+0000","last_peered":"2024-09-19T09:00:47.952874+0000","last_clean":"2024-09-19T09:00:47.952874+0000","last_became_active":"2024-09-19T09:00:46.954771+0000","last_became_peered":"2024-09-19T09:00:46.954771+0000","last_unstale":"2024-09-19T09:00:47.952874+0000","last_undegraded":"2024-09-19T09:00:47.952874+0000","last_fullsized":"2024-09-19T09:00:47.952874+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:18.945035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'9","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:01:00.963438+0000","last_change":"2024-09-19T09:00:44.938554+0000","last_active":"2024-09-19T09:01:00.963438+0000","last_peered":"2024-09-19T09:01:00.963438+0000","last_clean":"2024-09-19T09:01:00.963438+0000","last_became_active":"2024-09-19T09:00:44.938261+0000","last_became_peered":"2024-09-19T09:00:44.938261+0000","last_unstale":"2024-09-19T09:01:00.963438+0000","last_undegraded":"2024-09-19T09:01:00.963438+0000","last_fullsized":"2024-09-19T09:01:00.963438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:24.293764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964806+0000","last_change":"2024-09-19T09:00:46.960435+0000","last_active":"2024-09-19T09:00:47.964806+0000","last_peered":"2024-09-19T09:00:47.964806+0000","last_clean":"2024-09-19T09:00:47.964806+0000","last_became_active":"2024-09-19T09:00:46.960138+0000","last_became_peered":"2024-09-19T09:00:46.960138+0000","last_unstale":"2024-09-19T09:00:47.964806+0000","last_undegraded":"2024-09-19T09:00:47.964806+0000","last_fullsized":"2024-09-19T09:00:47.964806+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:59:50.853344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.139592+0000","last_change":"2024-09-19T09:00:44.941479+0000","last_active":"2024-09-19T09:00:58.139592+0000","last_peered":"2024-09-19T09:00:58.139592+0000","last_clean":"2024-09-19T09:00:58.139592+0000","last_became_active":"2024-09-19T09:00:44.937197+0000","last_became_peered":"2024-09-19T09:00:44.937197+0000","last_unstale":"2024-09-19T09:00:58.139592+0000","last_undegraded":"2024-09-19T09:00:58.139592+0000","last_fullsized":"2024-09-19T09:00:58.139592+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:53:17.385884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.965041+0000","last_change":"2024-09-19T09:00:46.970319+0000","last_active":"2024-09-19T09:00:47.965041+0000","last_peered":"2024-09-19T09:00:47.965041+0000","last_clean":"2024-09-19T09:00:47.965041+0000","last_became_active":"2024-09-19T09:00:46.969899+0000","last_became_peered":"2024-09-19T09:00:46.969899+0000","last_unstale":"2024-09-19T09:00:47.965041+0000","last_undegraded":"2024-09-19T09:00:47.965041+0000","last_fullsized":"2024-09-19T09:00:47.965041+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:26:21.379231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:55.109457+0000","last_change":"2024-09-19T09:00:45.239525+0000","last_active":"2024-09-19T09:00:55.109457+0000","last_peered":"2024-09-19T09:00:55.109457+0000","last_clean":"2024-09-19T09:00:55.109457+0000","last_became_active":"2024-09-19T09:00:45.239213+0000","last_became_peered":"2024-09-19T09:00:45.239213+0000","last_unstale":"2024-09-19T09:00:55.109457+0000","last_undegraded":"2024-09-19T09:00:55.109457+0000","last_fullsized":"2024-09-19T09:00:55.109457+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:32.525025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.953631+0000","last_change":"2024-09-19T09:00:46.958573+0000","last_active":"2024-09-19T09:00:47.953631+0000","last_peered":"2024-09-19T09:00:47.953631+0000","last_clean":"2024-09-19T09:00:47.953631+0000","last_became_active":"2024-09-19T09:00:46.958274+0000","last_became_peered":"2024-09-19T09:00:46.958274+0000","last_unstale":"2024-09-19T09:00:47.953631+0000","last_undegraded":"2024-09-19T09:00:47.953631+0000","last_fullsized":"2024-09-19T09:00:47.953631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:17.249122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.139345+0000","last_change":"2024-09-19T09:00:44.945194+0000","last_active":"2024-09-19T09:00:58.139345+0000","last_peered":"2024-09-19T09:00:58.139345+0000","last_clean":"2024-09-19T09:00:58.139345+0000","last_became_active":"2024-09-19T09:00:44.944931+0000","last_became_peered":"2024-09-19T09:00:44.944931+0000","last_unstale":"2024-09-19T09:00:58.139345+0000","last_undegraded":"2024-09-19T09:00:58.139345+0000","last_fullsized":"2024-09-19T09:00:58.139345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:11.181150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.969572+0000","last_change":"2024-09-19T09:00:46.966895+0000","last_active":"2024-09-19T09:00:47.969572+0000","last_peered":"2024-09-19T09:00:47.969572+0000","last_clean":"2024-09-19T09:00:47.969572+0000","last_became_active":"2024-09-19T09:00:46.966124+0000","last_became_peered":"2024-09-19T09:00:46.966124+0000","last_unstale":"2024-09-19T09:00:47.969572+0000","last_undegraded":"2024-09-19T09:00:47.969572+0000","last_fullsized":"2024-09-19T09:00:47.969572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:14:04.320863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:57.127047+0000","last_change":"2024-09-19T09:00:44.944167+0000","last_active":"2024-09-19T09:00:57.127047+0000","last_peered":"2024-09-19T09:00:57.127047+0000","last_clean":"2024-09-19T09:00:57.127047+0000","last_became_active":"2024-09-19T09:00:44.943670+0000","last_became_peered":"2024-09-19T09:00:44.943670+0000","last_unstale":"2024-09-19T09:00:57.127047+0000","last_undegraded":"2024-09-19T09:00:57.127047+0000","last_fullsized":"2024-09-19T09:00:57.127047+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:05.596877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964961+0000","last_change":"2024-09-19T09:00:46.958581+0000","last_active":"2024-09-19T09:00:47.964961+0000","last_peered":"2024-09-19T09:00:47.964961+0000","last_clean":"2024-09-19T09:00:47.964961+0000","last_became_active":"2024-09-19T09:00:46.957782+0000","last_became_peered":"2024-09-19T09:00:46.957782+0000","last_unstale":"2024-09-19T09:00:47.964961+0000","last_undegraded":"2024-09-19T09:00:47.964961+0000","last_fullsized":"2024-09-19T09:00:47.964961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:37:19.488538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:55.110727+0000","last_change":"2024-09-19T09:00:44.945351+0000","last_active":"2024-09-19T09:00:55.110727+0000","last_peered":"2024-09-19T09:00:55.110727+0000","last_clean":"2024-09-19T09:00:55.110727+0000","last_became_active":"2024-09-19T09:00:44.944991+0000","last_became_peered":"2024-09-19T09:00:44.944991+0000","last_unstale":"2024-09-19T09:00:55.110727+0000","last_undegraded":"2024-09-19T09:00:55.110727+0000","last_fullsized":"2024-09-19T09:00:55.110727+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:45:07.581872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964606+0000","last_change":"2024-09-19T09:00:46.956340+0000","last_active":"2024-09-19T09:00:47.964606+0000","last_peered":"2024-09-19T09:00:47.964606+0000","last_clean":"2024-09-19T09:00:47.964606+0000","last_became_active":"2024-09-19T09:00:46.955956+0000","last_became_peered":"2024-09-19T09:00:46.955956+0000","last_unstale":"2024-09-19T09:00:47.964606+0000","last_undegraded":"2024-09-19T09:00:47.964606+0000","last_fullsized":"2024-09-19T09:00:47.964606+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:34.863428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.968838+0000","last_change":"2024-09-19T09:00:46.963657+0000","last_active":"2024-09-19T09:00:47.968838+0000","last_peered":"2024-09-19T09:00:47.968838+0000","last_clean":"2024-09-19T09:00:47.968838+0000","last_became_active":"2024-09-19T09:00:46.963134+0000","last_became_peered":"2024-09-19T09:00:46.963134+0000","last_unstale":"2024-09-19T09:00:47.968838+0000","last_undegraded":"2024-09-19T09:00:47.968838+0000","last_fullsized":"2024-09-19T09:00:47.968838+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:01.098570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.141281+0000","last_change":"2024-09-19T09:00:44.945382+0000","last_active":"2024-09-19T09:00:58.141281+0000","last_peered":"2024-09-19T09:00:58.141281+0000","last_clean":"2024-09-19T09:00:58.141281+0000","last_became_active":"2024-09-19T09:00:44.945061+0000","last_became_peered":"2024-09-19T09:00:44.945061+0000","last_unstale":"2024-09-19T09:00:58.141281+0000","last_undegraded":"2024-09-19T09:00:58.141281+0000","last_fullsized":"2024-09-19T09:00:58.141281+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:07:12.747709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.953216+0000","last_change":"2024-09-19T09:00:46.956354+0000","last_active":"2024-09-19T09:00:47.953216+0000","last_peered":"2024-09-19T09:00:47.953216+0000","last_clean":"2024-09-19T09:00:47.953216+0000","last_became_active":"2024-09-19T09:00:46.955964+0000","last_became_peered":"2024-09-19T09:00:46.955964+0000","last_unstale":"2024-09-19T09:00:47.953216+0000","last_undegraded":"2024-09-19T09:00:47.953216+0000","last_fullsized":"2024-09-19T09:00:47.953216+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:05:33.243479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:59.136540+0000","last_change":"2024-09-19T09:00:44.944983+0000","last_active":"2024-09-19T09:00:59.136540+0000","last_peered":"2024-09-19T09:00:59.136540+0000","last_clean":"2024-09-19T09:00:59.136540+0000","last_became_active":"2024-09-19T09:00:44.944625+0000","last_became_peered":"2024-09-19T09:00:44.944625+0000","last_unstale":"2024-09-19T09:00:59.136540+0000","last_undegraded":"2024-09-19T09:00:59.136540+0000","last_fullsized":"2024-09-19T09:00:59.136540+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:49:50.818863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.965462+0000","last_change":"2024-09-19T09:00:46.966448+0000","last_active":"2024-09-19T09:00:47.965462+0000","last_peered":"2024-09-19T09:00:47.965462+0000","last_clean":"2024-09-19T09:00:47.965462+0000","last_became_active":"2024-09-19T09:00:46.966135+0000","last_became_peered":"2024-09-19T09:00:46.966135+0000","last_unstale":"2024-09-19T09:00:47.965462+0000","last_undegraded":"2024-09-19T09:00:47.965462+0000","last_fullsized":"2024-09-19T09:00:47.965462+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:42.652877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.967059+0000","last_change":"2024-09-19T09:00:45.239405+0000","last_active":"2024-09-19T09:00:47.967059+0000","last_peered":"2024-09-19T09:00:47.967059+0000","last_clean":"2024-09-19T09:00:47.967059+0000","last_became_active":"2024-09-19T09:00:45.239002+0000","last_became_peered":"2024-09-19T09:00:45.239002+0000","last_unstale":"2024-09-19T09:00:47.967059+0000","last_undegraded":"2024-09-19T09:00:47.967059+0000","last_fullsized":"2024-09-19T09:00:47.967059+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:46:03.934975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.965411+0000","last_change":"2024-09-19T09:00:46.951635+0000","last_active":"2024-09-19T09:00:47.965411+0000","last_peered":"2024-09-19T09:00:47.965411+0000","last_clean":"2024-09-19T09:00:47.965411+0000","last_became_active":"2024-09-19T09:00:46.951325+0000","last_became_peered":"2024-09-19T09:00:46.951325+0000","last_unstale":"2024-09-19T09:00:47.965411+0000","last_undegraded":"2024-09-19T09:00:47.965411+0000","last_fullsized":"2024-09-19T09:00:47.965411+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:17.360950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":40,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:01:05.152125+0000","last_change":"2024-09-19T09:00:44.942662+0000","last_active":"2024-09-19T09:01:05.152125+0000","last_peered":"2024-09-19T09:01:05.152125+0000","last_clean":"2024-09-19T09:01:05.152125+0000","last_became_active":"2024-09-19T09:00:44.942397+0000","last_became_peered":"2024-09-19T09:00:44.942397+0000","last_unstale":"2024-09-19T09:01:05.152125+0000","last_undegraded":"2024-09-19T09:01:05.152125+0000","last_fullsized":"2024-09-19T09:01:05.152125+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:46:35.247703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":56418,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":141,"num_read_kb":136,"num_write":227,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":512000,"data_stored":152178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":212166,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612624,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":860160,"data_stored":410932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841558,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":1051957,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972379,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":410862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201312,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":411240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430246,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":1003760,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593771,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":410579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822703,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":1003045,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986229,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":410833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215162,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":410404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":452777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36232,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708920,"statfs":{"total":95995035648,"available":95957934080,"internally_reserved":0,"allocated":909312,"data_stored":452557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":411918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":42817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":43037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":51732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:01:13.642 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T09:01:13.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:13 smithi120 ceph-mon[31150]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:01:13.887 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:13.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:13 smithi196 ceph-mon[35150]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:01:13.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:13 smithi153 ceph-mon[35071]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:01:14.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.356+0000 7f169702b640 1 -- 172.21.15.120:0/3862000325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16901027c0 msgr2=0x7f1690106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:14.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.356+0000 7f169702b640 1 --2- 172.21.15.120:0/3862000325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16901027c0 0x7f1690106810 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1678009900 tx=0x7f167802a410 comp rx=0 tx=0).stop 2024-09-19T09:01:14.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.356+0000 7f169702b640 1 -- 172.21.15.120:0/3862000325 shutdown_connections 2024-09-19T09:01:14.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.356+0000 7f169702b640 1 --2- 172.21.15.120:0/3862000325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1690106f40 0x7f169010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:14.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.356+0000 7f169702b640 1 --2- 172.21.15.120:0/3862000325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16901027c0 0x7f1690106810 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:14.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.356+0000 7f169702b640 1 --2- 172.21.15.120:0/3862000325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1690101e10 0x7f16901021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:14.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.356+0000 7f169702b640 1 -- 172.21.15.120:0/3862000325 >> 172.21.15.120:0/3862000325 conn(0x7f16900fd8d0 msgr2=0x7f16900ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:14.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.356+0000 7f169702b640 1 -- 172.21.15.120:0/3862000325 shutdown_connections 2024-09-19T09:01:14.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.356+0000 7f169702b640 1 -- 172.21.15.120:0/3862000325 wait complete. 2024-09-19T09:01:14.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.357+0000 7f169702b640 1 Processor -- start 2024-09-19T09:01:14.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.357+0000 7f169702b640 1 -- start start 2024-09-19T09:01:14.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.357+0000 7f169702b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1690101e10 0x7f1690198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:14.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.357+0000 7f169702b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16901027c0 0x7f1690198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:14.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.357+0000 7f169702b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1690106f40 0x7f169019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:14.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.357+0000 7f169702b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1690111fe0 con 0x7f1690101e10 2024-09-19T09:01:14.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.357+0000 7f169702b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1690111e60 con 0x7f16901027c0 2024-09-19T09:01:14.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.357+0000 7f169702b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1690112160 con 0x7f1690106f40 2024-09-19T09:01:14.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.357+0000 7f1694dc7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1690101e10 0x7f1690198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:14.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.357+0000 7f1694dc7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1690101e10 0x7f1690198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42790/0 (socket says 172.21.15.120:42790) 2024-09-19T09:01:14.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.357+0000 7f1694dc7640 1 -- 172.21.15.120:0/3387203399 learned_addr learned my addr 172.21.15.120:0/3387203399 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:14.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.358+0000 7f16955c8640 1 --2- 172.21.15.120:0/3387203399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1690106f40 0x7f169019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:14.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.358+0000 7f168ffff640 1 --2- 172.21.15.120:0/3387203399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16901027c0 0x7f1690198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:14.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.358+0000 7f1694dc7640 1 -- 172.21.15.120:0/3387203399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1690106f40 msgr2=0x7f169019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:14.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.358+0000 7f1694dc7640 1 --2- 172.21.15.120:0/3387203399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1690106f40 0x7f169019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:14.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.358+0000 7f1694dc7640 1 -- 172.21.15.120:0/3387203399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16901027c0 msgr2=0x7f1690198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:14.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.358+0000 7f1694dc7640 1 --2- 172.21.15.120:0/3387203399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16901027c0 0x7f1690198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:14.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.358+0000 7f1694dc7640 1 -- 172.21.15.120:0/3387203399 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169019d4b0 con 0x7f1690101e10 2024-09-19T09:01:14.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.358+0000 7f1694dc7640 1 --2- 172.21.15.120:0/3387203399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1690101e10 0x7f1690198500 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f168000cce0 tx=0x7f168000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:14.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.358+0000 7f168dffb640 1 -- 172.21.15.120:0/3387203399 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1680004b20 con 0x7f1690101e10 2024-09-19T09:01:14.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.358+0000 7f168dffb640 1 -- 172.21.15.120:0/3387203399 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1680004cc0 con 0x7f1690101e10 2024-09-19T09:01:14.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.358+0000 7f169702b640 1 -- 172.21.15.120:0/3387203399 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f169019d7a0 con 0x7f1690101e10 2024-09-19T09:01:14.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.358+0000 7f168dffb640 1 -- 172.21.15.120:0/3387203399 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168000f8c0 con 0x7f1690101e10 2024-09-19T09:01:14.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.358+0000 7f169702b640 1 -- 172.21.15.120:0/3387203399 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16901a5080 con 0x7f1690101e10 2024-09-19T09:01:14.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.359+0000 7f169702b640 1 -- 172.21.15.120:0/3387203399 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16901021f0 con 0x7f1690101e10 2024-09-19T09:01:14.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.360+0000 7f168dffb640 1 -- 172.21.15.120:0/3387203399 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f168000fa60 con 0x7f1690101e10 2024-09-19T09:01:14.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.361+0000 7f168dffb640 1 --2- 172.21.15.120:0/3387203399 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f167c0a3350 0x7f167c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:14.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.361+0000 7f168ffff640 1 --2- 172.21.15.120:0/3387203399 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f167c0a3350 0x7f167c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:14.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.361+0000 7f168dffb640 1 -- 172.21.15.120:0/3387203399 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f16800cf9d0 con 0x7f1690101e10 2024-09-19T09:01:14.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.362+0000 7f168ffff640 1 --2- 172.21.15.120:0/3387203399 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f167c0a3350 0x7f167c0a5810 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f16900ff7d0 tx=0x7f1678031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:14.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.365+0000 7f168dffb640 1 -- 172.21.15.120:0/3387203399 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1680014030 con 0x7f1690101e10 2024-09-19T09:01:14.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.531+0000 7f169702b640 1 -- 172.21.15.120:0/3387203399 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f169010a180 con 0x7f167c0a3350 2024-09-19T09:01:14.531 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:14 smithi120 ceph-mon[31150]: pgmap v290: 97 pgs: 97 active+clean; 632 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 KiB/s wr, 16 op/s 2024-09-19T09:01:14.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.537+0000 7f168dffb640 1 -- 172.21.15.120:0/3387203399 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258868 (secure 0 0 0) 0x7f169010a180 con 0x7f167c0a3350 2024-09-19T09:01:14.539 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:01:14.540 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-19T09:01:14.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.540+0000 7f169702b640 1 -- 172.21.15.120:0/3387203399 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f167c0a3350 msgr2=0x7f167c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:14.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.540+0000 7f169702b640 1 --2- 172.21.15.120:0/3387203399 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f167c0a3350 0x7f167c0a5810 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f16900ff7d0 tx=0x7f1678031490 comp rx=0 tx=0).stop 2024-09-19T09:01:14.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.540+0000 7f169702b640 1 -- 172.21.15.120:0/3387203399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1690101e10 msgr2=0x7f1690198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:14.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.540+0000 7f169702b640 1 --2- 172.21.15.120:0/3387203399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1690101e10 0x7f1690198500 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f168000cce0 tx=0x7f168000a620 comp rx=0 tx=0).stop 2024-09-19T09:01:14.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.540+0000 7f169702b640 1 -- 172.21.15.120:0/3387203399 shutdown_connections 2024-09-19T09:01:14.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.540+0000 7f169702b640 1 --2- 172.21.15.120:0/3387203399 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f167c0a3350 0x7f167c0a5810 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:14.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.540+0000 7f169702b640 1 --2- 172.21.15.120:0/3387203399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1690106f40 0x7f169019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:14.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.540+0000 7f169702b640 1 --2- 172.21.15.120:0/3387203399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16901027c0 0x7f1690198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:14.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.540+0000 7f169702b640 1 --2- 172.21.15.120:0/3387203399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1690101e10 0x7f1690198500 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:14.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.540+0000 7f169702b640 1 -- 172.21.15.120:0/3387203399 >> 172.21.15.120:0/3387203399 conn(0x7f16900fd8d0 msgr2=0x7f16900fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:14.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.540+0000 7f169702b640 1 -- 172.21.15.120:0/3387203399 shutdown_connections 2024-09-19T09:01:14.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:14.540+0000 7f169702b640 1 -- 172.21.15.120:0/3387203399 wait complete. 2024-09-19T09:01:14.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:14 smithi196 ceph-mon[35150]: pgmap v290: 97 pgs: 97 active+clean; 632 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 KiB/s wr, 16 op/s 2024-09-19T09:01:14.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:14 smithi153 ceph-mon[35071]: pgmap v290: 97 pgs: 97 active+clean; 632 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 KiB/s wr, 16 op/s 2024-09-19T09:01:15.378 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-09-19T09:01:13.614377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647020,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":292,"num_read_kb":255,"num_write":417,"num_write_kb":3060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364028,"kb_used_data":10620,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577796,"statfs":{"total":1151940427776,"available":1151567663104,"internally_reserved":0,"allocated":10874880,"data_stored":6857940,"data_compressed":107721,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":42140,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":146,"num_read_kb":129,"num_write":47,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002838"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.969591+0000","last_change":"2024-09-19T09:00:46.966628+0000","last_active":"2024-09-19T09:00:47.969591+0000","last_peered":"2024-09-19T09:00:47.969591+0000","last_clean":"2024-09-19T09:00:47.969591+0000","last_became_active":"2024-09-19T09:00:46.966290+0000","last_became_peered":"2024-09-19T09:00:46.966290+0000","last_unstale":"2024-09-19T09:00:47.969591+0000","last_undegraded":"2024-09-19T09:00:47.969591+0000","last_fullsized":"2024-09-19T09:00:47.969591+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:42.717139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964905+0000","last_change":"2024-09-19T09:00:46.963706+0000","last_active":"2024-09-19T09:00:47.964905+0000","last_peered":"2024-09-19T09:00:47.964905+0000","last_clean":"2024-09-19T09:00:47.964905+0000","last_became_active":"2024-09-19T09:00:46.963303+0000","last_became_peered":"2024-09-19T09:00:46.963303+0000","last_unstale":"2024-09-19T09:00:47.964905+0000","last_undegraded":"2024-09-19T09:00:47.964905+0000","last_fullsized":"2024-09-19T09:00:47.964905+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:07.451500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.963870+0000","last_change":"2024-09-19T09:00:46.967149+0000","last_active":"2024-09-19T09:00:47.963870+0000","last_peered":"2024-09-19T09:00:47.963870+0000","last_clean":"2024-09-19T09:00:47.963870+0000","last_became_active":"2024-09-19T09:00:46.966861+0000","last_became_peered":"2024-09-19T09:00:46.966861+0000","last_unstale":"2024-09-19T09:00:47.963870+0000","last_undegraded":"2024-09-19T09:00:47.963870+0000","last_fullsized":"2024-09-19T09:00:47.963870+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:28.396022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964150+0000","last_change":"2024-09-19T09:00:46.969242+0000","last_active":"2024-09-19T09:00:47.964150+0000","last_peered":"2024-09-19T09:00:47.964150+0000","last_clean":"2024-09-19T09:00:47.964150+0000","last_became_active":"2024-09-19T09:00:46.968546+0000","last_became_peered":"2024-09-19T09:00:46.968546+0000","last_unstale":"2024-09-19T09:00:47.964150+0000","last_undegraded":"2024-09-19T09:00:47.964150+0000","last_fullsized":"2024-09-19T09:00:47.964150+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:43:51.060130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.953778+0000","last_change":"2024-09-19T09:00:46.955501+0000","last_active":"2024-09-19T09:00:47.953778+0000","last_peered":"2024-09-19T09:00:47.953778+0000","last_clean":"2024-09-19T09:00:47.953778+0000","last_became_active":"2024-09-19T09:00:46.955161+0000","last_became_peered":"2024-09-19T09:00:46.955161+0000","last_unstale":"2024-09-19T09:00:47.953778+0000","last_undegraded":"2024-09-19T09:00:47.953778+0000","last_fullsized":"2024-09-19T09:00:47.953778+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:39:19.005931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.965095+0000","last_change":"2024-09-19T09:00:46.962577+0000","last_active":"2024-09-19T09:00:47.965095+0000","last_peered":"2024-09-19T09:00:47.965095+0000","last_clean":"2024-09-19T09:00:47.965095+0000","last_became_active":"2024-09-19T09:00:46.962253+0000","last_became_peered":"2024-09-19T09:00:46.962253+0000","last_unstale":"2024-09-19T09:00:47.965095+0000","last_undegraded":"2024-09-19T09:00:47.965095+0000","last_fullsized":"2024-09-19T09:00:47.965095+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:27.925932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964437+0000","last_change":"2024-09-19T09:00:46.960004+0000","last_active":"2024-09-19T09:00:47.964437+0000","last_peered":"2024-09-19T09:00:47.964437+0000","last_clean":"2024-09-19T09:00:47.964437+0000","last_became_active":"2024-09-19T09:00:46.959664+0000","last_became_peered":"2024-09-19T09:00:46.959664+0000","last_unstale":"2024-09-19T09:00:47.964437+0000","last_undegraded":"2024-09-19T09:00:47.964437+0000","last_fullsized":"2024-09-19T09:00:47.964437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:41.616309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964731+0000","last_change":"2024-09-19T09:00:46.952394+0000","last_active":"2024-09-19T09:00:47.964731+0000","last_peered":"2024-09-19T09:00:47.964731+0000","last_clean":"2024-09-19T09:00:47.964731+0000","last_became_active":"2024-09-19T09:00:46.952084+0000","last_became_peered":"2024-09-19T09:00:46.952084+0000","last_unstale":"2024-09-19T09:00:47.964731+0000","last_undegraded":"2024-09-19T09:00:47.964731+0000","last_fullsized":"2024-09-19T09:00:47.964731+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:21.698423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.953905+0000","last_change":"2024-09-19T09:00:46.961262+0000","last_active":"2024-09-19T09:00:47.953905+0000","last_peered":"2024-09-19T09:00:47.953905+0000","last_clean":"2024-09-19T09:00:47.953905+0000","last_became_active":"2024-09-19T09:00:46.960951+0000","last_became_peered":"2024-09-19T09:00:46.960951+0000","last_unstale":"2024-09-19T09:00:47.953905+0000","last_undegraded":"2024-09-19T09:00:47.953905+0000","last_fullsized":"2024-09-19T09:00:47.953905+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:02.445108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.966156+0000","last_change":"2024-09-19T09:00:46.964806+0000","last_active":"2024-09-19T09:00:47.966156+0000","last_peered":"2024-09-19T09:00:47.966156+0000","last_clean":"2024-09-19T09:00:47.966156+0000","last_became_active":"2024-09-19T09:00:46.964194+0000","last_became_peered":"2024-09-19T09:00:46.964194+0000","last_unstale":"2024-09-19T09:00:47.966156+0000","last_undegraded":"2024-09-19T09:00:47.966156+0000","last_fullsized":"2024-09-19T09:00:47.966156+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:31.621075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.966443+0000","last_change":"2024-09-19T09:00:46.963200+0000","last_active":"2024-09-19T09:00:47.966443+0000","last_peered":"2024-09-19T09:00:47.966443+0000","last_clean":"2024-09-19T09:00:47.966443+0000","last_became_active":"2024-09-19T09:00:46.960496+0000","last_became_peered":"2024-09-19T09:00:46.960496+0000","last_unstale":"2024-09-19T09:00:47.966443+0000","last_undegraded":"2024-09-19T09:00:47.966443+0000","last_fullsized":"2024-09-19T09:00:47.966443+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:59:47.234492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964182+0000","last_change":"2024-09-19T09:00:46.966309+0000","last_active":"2024-09-19T09:00:47.964182+0000","last_peered":"2024-09-19T09:00:47.964182+0000","last_clean":"2024-09-19T09:00:47.964182+0000","last_became_active":"2024-09-19T09:00:46.965970+0000","last_became_peered":"2024-09-19T09:00:46.965970+0000","last_unstale":"2024-09-19T09:00:47.964182+0000","last_undegraded":"2024-09-19T09:00:47.964182+0000","last_fullsized":"2024-09-19T09:00:47.964182+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:38.353206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.952739+0000","last_change":"2024-09-19T09:00:46.954079+0000","last_active":"2024-09-19T09:00:47.952739+0000","last_peered":"2024-09-19T09:00:47.952739+0000","last_clean":"2024-09-19T09:00:47.952739+0000","last_became_active":"2024-09-19T09:00:46.953398+0000","last_became_peered":"2024-09-19T09:00:46.953398+0000","last_unstale":"2024-09-19T09:00:47.952739+0000","last_undegraded":"2024-09-19T09:00:47.952739+0000","last_fullsized":"2024-09-19T09:00:47.952739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:54:27.554800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.963727+0000","last_change":"2024-09-19T09:00:46.965347+0000","last_active":"2024-09-19T09:00:47.963727+0000","last_peered":"2024-09-19T09:00:47.963727+0000","last_clean":"2024-09-19T09:00:47.963727+0000","last_became_active":"2024-09-19T09:00:46.965045+0000","last_became_peered":"2024-09-19T09:00:46.965045+0000","last_unstale":"2024-09-19T09:00:47.963727+0000","last_undegraded":"2024-09-19T09:00:47.963727+0000","last_fullsized":"2024-09-19T09:00:47.963727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:27.316100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964176+0000","last_change":"2024-09-19T09:00:46.962898+0000","last_active":"2024-09-19T09:00:47.964176+0000","last_peered":"2024-09-19T09:00:47.964176+0000","last_clean":"2024-09-19T09:00:47.964176+0000","last_became_active":"2024-09-19T09:00:46.962412+0000","last_became_peered":"2024-09-19T09:00:46.962412+0000","last_unstale":"2024-09-19T09:00:47.964176+0000","last_undegraded":"2024-09-19T09:00:47.964176+0000","last_fullsized":"2024-09-19T09:00:47.964176+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:40.483678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.953179+0000","last_change":"2024-09-19T09:00:46.958147+0000","last_active":"2024-09-19T09:00:47.953179+0000","last_peered":"2024-09-19T09:00:47.953179+0000","last_clean":"2024-09-19T09:00:47.953179+0000","last_became_active":"2024-09-19T09:00:46.957805+0000","last_became_peered":"2024-09-19T09:00:46.957805+0000","last_unstale":"2024-09-19T09:00:47.953179+0000","last_undegraded":"2024-09-19T09:00:47.953179+0000","last_fullsized":"2024-09-19T09:00:47.953179+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:24:17.000064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.969691+0000","last_change":"2024-09-19T09:00:46.967399+0000","last_active":"2024-09-19T09:00:47.969691+0000","last_peered":"2024-09-19T09:00:47.969691+0000","last_clean":"2024-09-19T09:00:47.969691+0000","last_became_active":"2024-09-19T09:00:46.966145+0000","last_became_peered":"2024-09-19T09:00:46.966145+0000","last_unstale":"2024-09-19T09:00:47.969691+0000","last_undegraded":"2024-09-19T09:00:47.969691+0000","last_fullsized":"2024-09-19T09:00:47.969691+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:29.192469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.962791+0000","last_change":"2024-09-19T09:00:46.960956+0000","last_active":"2024-09-19T09:00:47.962791+0000","last_peered":"2024-09-19T09:00:47.962791+0000","last_clean":"2024-09-19T09:00:47.962791+0000","last_became_active":"2024-09-19T09:00:46.960664+0000","last_became_peered":"2024-09-19T09:00:46.960664+0000","last_unstale":"2024-09-19T09:00:47.962791+0000","last_undegraded":"2024-09-19T09:00:47.962791+0000","last_fullsized":"2024-09-19T09:00:47.962791+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:18.724348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.965529+0000","last_change":"2024-09-19T09:00:46.951362+0000","last_active":"2024-09-19T09:00:47.965529+0000","last_peered":"2024-09-19T09:00:47.965529+0000","last_clean":"2024-09-19T09:00:47.965529+0000","last_became_active":"2024-09-19T09:00:46.951004+0000","last_became_peered":"2024-09-19T09:00:46.951004+0000","last_unstale":"2024-09-19T09:00:47.965529+0000","last_undegraded":"2024-09-19T09:00:47.965529+0000","last_fullsized":"2024-09-19T09:00:47.965529+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:43.574425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.969012+0000","last_change":"2024-09-19T09:00:46.966752+0000","last_active":"2024-09-19T09:00:47.969012+0000","last_peered":"2024-09-19T09:00:47.969012+0000","last_clean":"2024-09-19T09:00:47.969012+0000","last_became_active":"2024-09-19T09:00:46.966370+0000","last_became_peered":"2024-09-19T09:00:46.966370+0000","last_unstale":"2024-09-19T09:00:47.969012+0000","last_undegraded":"2024-09-19T09:00:47.969012+0000","last_fullsized":"2024-09-19T09:00:47.969012+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:04.035771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.952829+0000","last_change":"2024-09-19T09:00:46.956369+0000","last_active":"2024-09-19T09:00:47.952829+0000","last_peered":"2024-09-19T09:00:47.952829+0000","last_clean":"2024-09-19T09:00:47.952829+0000","last_became_active":"2024-09-19T09:00:46.956075+0000","last_became_peered":"2024-09-19T09:00:46.956075+0000","last_unstale":"2024-09-19T09:00:47.952829+0000","last_undegraded":"2024-09-19T09:00:47.952829+0000","last_fullsized":"2024-09-19T09:00:47.952829+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:23.985017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.953432+0000","last_change":"2024-09-19T09:00:46.956661+0000","last_active":"2024-09-19T09:00:47.953432+0000","last_peered":"2024-09-19T09:00:47.953432+0000","last_clean":"2024-09-19T09:00:47.953432+0000","last_became_active":"2024-09-19T09:00:46.956291+0000","last_became_peered":"2024-09-19T09:00:46.956291+0000","last_unstale":"2024-09-19T09:00:47.953432+0000","last_undegraded":"2024-09-19T09:00:47.953432+0000","last_fullsized":"2024-09-19T09:00:47.953432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:55.713039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964577+0000","last_change":"2024-09-19T09:00:46.961866+0000","last_active":"2024-09-19T09:00:47.964577+0000","last_peered":"2024-09-19T09:00:47.964577+0000","last_clean":"2024-09-19T09:00:47.964577+0000","last_became_active":"2024-09-19T09:00:46.961469+0000","last_became_peered":"2024-09-19T09:00:46.961469+0000","last_unstale":"2024-09-19T09:00:47.964577+0000","last_undegraded":"2024-09-19T09:00:47.964577+0000","last_fullsized":"2024-09-19T09:00:47.964577+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:47.665419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.967694+0000","last_change":"2024-09-19T09:00:46.948765+0000","last_active":"2024-09-19T09:00:47.967694+0000","last_peered":"2024-09-19T09:00:47.967694+0000","last_clean":"2024-09-19T09:00:47.967694+0000","last_became_active":"2024-09-19T09:00:46.948431+0000","last_became_peered":"2024-09-19T09:00:46.948431+0000","last_unstale":"2024-09-19T09:00:47.967694+0000","last_undegraded":"2024-09-19T09:00:47.967694+0000","last_fullsized":"2024-09-19T09:00:47.967694+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:31.766464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964637+0000","last_change":"2024-09-19T09:00:46.962433+0000","last_active":"2024-09-19T09:00:47.964637+0000","last_peered":"2024-09-19T09:00:47.964637+0000","last_clean":"2024-09-19T09:00:47.964637+0000","last_became_active":"2024-09-19T09:00:46.962153+0000","last_became_peered":"2024-09-19T09:00:46.962153+0000","last_unstale":"2024-09-19T09:00:47.964637+0000","last_undegraded":"2024-09-19T09:00:47.964637+0000","last_fullsized":"2024-09-19T09:00:47.964637+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:37.973611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.965774+0000","last_change":"2024-09-19T09:00:46.953715+0000","last_active":"2024-09-19T09:00:47.965774+0000","last_peered":"2024-09-19T09:00:47.965774+0000","last_clean":"2024-09-19T09:00:47.965774+0000","last_became_active":"2024-09-19T09:00:46.953415+0000","last_became_peered":"2024-09-19T09:00:46.953415+0000","last_unstale":"2024-09-19T09:00:47.965774+0000","last_undegraded":"2024-09-19T09:00:47.965774+0000","last_fullsized":"2024-09-19T09:00:47.965774+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:29.768382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.969440+0000","last_change":"2024-09-19T09:00:46.964746+0000","last_active":"2024-09-19T09:00:47.969440+0000","last_peered":"2024-09-19T09:00:47.969440+0000","last_clean":"2024-09-19T09:00:47.969440+0000","last_became_active":"2024-09-19T09:00:46.964404+0000","last_became_peered":"2024-09-19T09:00:46.964404+0000","last_unstale":"2024-09-19T09:00:47.969440+0000","last_undegraded":"2024-09-19T09:00:47.969440+0000","last_fullsized":"2024-09-19T09:00:47.969440+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:19.841495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.952789+0000","last_change":"2024-09-19T09:00:46.962875+0000","last_active":"2024-09-19T09:00:47.952789+0000","last_peered":"2024-09-19T09:00:47.952789+0000","last_clean":"2024-09-19T09:00:47.952789+0000","last_became_active":"2024-09-19T09:00:46.962561+0000","last_became_peered":"2024-09-19T09:00:46.962561+0000","last_unstale":"2024-09-19T09:00:47.952789+0000","last_undegraded":"2024-09-19T09:00:47.952789+0000","last_fullsized":"2024-09-19T09:00:47.952789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:39.675545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964406+0000","last_change":"2024-09-19T09:00:46.963126+0000","last_active":"2024-09-19T09:00:47.964406+0000","last_peered":"2024-09-19T09:00:47.964406+0000","last_clean":"2024-09-19T09:00:47.964406+0000","last_became_active":"2024-09-19T09:00:46.962620+0000","last_became_peered":"2024-09-19T09:00:46.962620+0000","last_unstale":"2024-09-19T09:00:47.964406+0000","last_undegraded":"2024-09-19T09:00:47.964406+0000","last_fullsized":"2024-09-19T09:00:47.964406+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:59.723280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964649+0000","last_change":"2024-09-19T09:00:46.953026+0000","last_active":"2024-09-19T09:00:47.964649+0000","last_peered":"2024-09-19T09:00:47.964649+0000","last_clean":"2024-09-19T09:00:47.964649+0000","last_became_active":"2024-09-19T09:00:46.952596+0000","last_became_peered":"2024-09-19T09:00:46.952596+0000","last_unstale":"2024-09-19T09:00:47.964649+0000","last_undegraded":"2024-09-19T09:00:47.964649+0000","last_fullsized":"2024-09-19T09:00:47.964649+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:58.524116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964488+0000","last_change":"2024-09-19T09:00:46.968964+0000","last_active":"2024-09-19T09:00:47.964488+0000","last_peered":"2024-09-19T09:00:47.964488+0000","last_clean":"2024-09-19T09:00:47.964488+0000","last_became_active":"2024-09-19T09:00:46.968505+0000","last_became_peered":"2024-09-19T09:00:46.968505+0000","last_unstale":"2024-09-19T09:00:47.964488+0000","last_undegraded":"2024-09-19T09:00:47.964488+0000","last_fullsized":"2024-09-19T09:00:47.964488+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:17:33.335541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.969060+0000","last_change":"2024-09-19T09:00:46.961520+0000","last_active":"2024-09-19T09:00:47.969060+0000","last_peered":"2024-09-19T09:00:47.969060+0000","last_clean":"2024-09-19T09:00:47.969060+0000","last_became_active":"2024-09-19T09:00:46.961140+0000","last_became_peered":"2024-09-19T09:00:46.961140+0000","last_unstale":"2024-09-19T09:00:47.969060+0000","last_undegraded":"2024-09-19T09:00:47.969060+0000","last_fullsized":"2024-09-19T09:00:47.969060+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:03.120173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.953710+0000","last_change":"2024-09-19T09:00:46.955995+0000","last_active":"2024-09-19T09:00:47.953710+0000","last_peered":"2024-09-19T09:00:47.953710+0000","last_clean":"2024-09-19T09:00:47.953710+0000","last_became_active":"2024-09-19T09:00:46.954678+0000","last_became_peered":"2024-09-19T09:00:46.954678+0000","last_unstale":"2024-09-19T09:00:47.953710+0000","last_undegraded":"2024-09-19T09:00:47.953710+0000","last_fullsized":"2024-09-19T09:00:47.953710+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:10.314575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":39,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:01:11.161958+0000","last_change":"2024-09-19T09:00:44.946000+0000","last_active":"2024-09-19T09:01:11.161958+0000","last_peered":"2024-09-19T09:01:11.161958+0000","last_clean":"2024-09-19T09:01:11.161958+0000","last_became_active":"2024-09-19T09:00:44.945583+0000","last_became_peered":"2024-09-19T09:00:44.945583+0000","last_unstale":"2024-09-19T09:01:11.161958+0000","last_undegraded":"2024-09-19T09:01:11.161958+0000","last_fullsized":"2024-09-19T09:01:11.161958+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:53:19.109697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.967983+0000","last_change":"2024-09-19T09:00:46.962774+0000","last_active":"2024-09-19T09:00:47.967983+0000","last_peered":"2024-09-19T09:00:47.967983+0000","last_clean":"2024-09-19T09:00:47.967983+0000","last_became_active":"2024-09-19T09:00:46.962435+0000","last_became_peered":"2024-09-19T09:00:46.962435+0000","last_unstale":"2024-09-19T09:00:47.967983+0000","last_undegraded":"2024-09-19T09:00:47.967983+0000","last_fullsized":"2024-09-19T09:00:47.967983+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:55.481141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:57.126925+0000","last_change":"2024-09-19T09:00:44.942435+0000","last_active":"2024-09-19T09:00:57.126925+0000","last_peered":"2024-09-19T09:00:57.126925+0000","last_clean":"2024-09-19T09:00:57.126925+0000","last_became_active":"2024-09-19T09:00:44.942067+0000","last_became_peered":"2024-09-19T09:00:44.942067+0000","last_unstale":"2024-09-19T09:00:57.126925+0000","last_undegraded":"2024-09-19T09:00:57.126925+0000","last_fullsized":"2024-09-19T09:00:57.126925+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:16.476506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:56.118780+0000","last_change":"2024-09-19T09:00:44.937944+0000","last_active":"2024-09-19T09:00:56.118780+0000","last_peered":"2024-09-19T09:00:56.118780+0000","last_clean":"2024-09-19T09:00:56.118780+0000","last_became_active":"2024-09-19T09:00:44.937623+0000","last_became_peered":"2024-09-19T09:00:44.937623+0000","last_unstale":"2024-09-19T09:00:56.118780+0000","last_undegraded":"2024-09-19T09:00:56.118780+0000","last_fullsized":"2024-09-19T09:00:56.118780+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:49:08.730015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964590+0000","last_change":"2024-09-19T09:00:46.959932+0000","last_active":"2024-09-19T09:00:47.964590+0000","last_peered":"2024-09-19T09:00:47.964590+0000","last_clean":"2024-09-19T09:00:47.964590+0000","last_became_active":"2024-09-19T09:00:46.959637+0000","last_became_peered":"2024-09-19T09:00:46.959637+0000","last_unstale":"2024-09-19T09:00:47.964590+0000","last_undegraded":"2024-09-19T09:00:47.964590+0000","last_fullsized":"2024-09-19T09:00:47.964590+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:55.337669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.140550+0000","last_change":"2024-09-19T09:00:44.944882+0000","last_active":"2024-09-19T09:00:58.140550+0000","last_peered":"2024-09-19T09:00:58.140550+0000","last_clean":"2024-09-19T09:00:58.140550+0000","last_became_active":"2024-09-19T09:00:44.944116+0000","last_became_peered":"2024-09-19T09:00:44.944116+0000","last_unstale":"2024-09-19T09:00:58.140550+0000","last_undegraded":"2024-09-19T09:00:58.140550+0000","last_fullsized":"2024-09-19T09:00:58.140550+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:54:58.555913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.962700+0000","last_change":"2024-09-19T09:00:46.956859+0000","last_active":"2024-09-19T09:00:47.962700+0000","last_peered":"2024-09-19T09:00:47.962700+0000","last_clean":"2024-09-19T09:00:47.962700+0000","last_became_active":"2024-09-19T09:00:46.956561+0000","last_became_peered":"2024-09-19T09:00:46.956561+0000","last_unstale":"2024-09-19T09:00:47.962700+0000","last_undegraded":"2024-09-19T09:00:47.962700+0000","last_fullsized":"2024-09-19T09:00:47.962700+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:09:04.125591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":46,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:01:10.159859+0000","last_change":"2024-09-19T09:00:44.945959+0000","last_active":"2024-09-19T09:01:10.159859+0000","last_peered":"2024-09-19T09:01:10.159859+0000","last_clean":"2024-09-19T09:01:10.159859+0000","last_became_active":"2024-09-19T09:00:44.945640+0000","last_became_peered":"2024-09-19T09:00:44.945640+0000","last_unstale":"2024-09-19T09:01:10.159859+0000","last_undegraded":"2024-09-19T09:01:10.159859+0000","last_fullsized":"2024-09-19T09:01:10.159859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:02.523834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.953728+0000","last_change":"2024-09-19T09:00:46.953677+0000","last_active":"2024-09-19T09:00:47.953728+0000","last_peered":"2024-09-19T09:00:47.953728+0000","last_clean":"2024-09-19T09:00:47.953728+0000","last_became_active":"2024-09-19T09:00:46.953357+0000","last_became_peered":"2024-09-19T09:00:46.953357+0000","last_unstale":"2024-09-19T09:00:47.953728+0000","last_undegraded":"2024-09-19T09:00:47.953728+0000","last_fullsized":"2024-09-19T09:00:47.953728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:48.963003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.139991+0000","last_change":"2024-09-19T09:00:44.945777+0000","last_active":"2024-09-19T09:00:58.139991+0000","last_peered":"2024-09-19T09:00:58.139991+0000","last_clean":"2024-09-19T09:00:58.139991+0000","last_became_active":"2024-09-19T09:00:44.945450+0000","last_became_peered":"2024-09-19T09:00:44.945450+0000","last_unstale":"2024-09-19T09:00:58.139991+0000","last_undegraded":"2024-09-19T09:00:58.139991+0000","last_fullsized":"2024-09-19T09:00:58.139991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:44.656451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.965513+0000","last_change":"2024-09-19T09:00:46.962285+0000","last_active":"2024-09-19T09:00:47.965513+0000","last_peered":"2024-09-19T09:00:47.965513+0000","last_clean":"2024-09-19T09:00:47.965513+0000","last_became_active":"2024-09-19T09:00:46.961982+0000","last_became_peered":"2024-09-19T09:00:46.961982+0000","last_unstale":"2024-09-19T09:00:47.965513+0000","last_undegraded":"2024-09-19T09:00:47.965513+0000","last_fullsized":"2024-09-19T09:00:47.965513+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:41.658501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:57.125258+0000","last_change":"2024-09-19T09:00:45.241425+0000","last_active":"2024-09-19T09:00:57.125258+0000","last_peered":"2024-09-19T09:00:57.125258+0000","last_clean":"2024-09-19T09:00:57.125258+0000","last_became_active":"2024-09-19T09:00:45.241032+0000","last_became_peered":"2024-09-19T09:00:45.241032+0000","last_unstale":"2024-09-19T09:00:57.125258+0000","last_undegraded":"2024-09-19T09:00:57.125258+0000","last_fullsized":"2024-09-19T09:00:57.125258+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:00.707855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.953388+0000","last_change":"2024-09-19T09:00:46.962436+0000","last_active":"2024-09-19T09:00:47.953388+0000","last_peered":"2024-09-19T09:00:47.953388+0000","last_clean":"2024-09-19T09:00:47.953388+0000","last_became_active":"2024-09-19T09:00:46.962114+0000","last_became_peered":"2024-09-19T09:00:46.962114+0000","last_unstale":"2024-09-19T09:00:47.953388+0000","last_undegraded":"2024-09-19T09:00:47.953388+0000","last_fullsized":"2024-09-19T09:00:47.953388+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:41.761705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'27","reported_seq":75,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:01:10.164619+0000","last_change":"2024-09-19T09:00:44.945187+0000","last_active":"2024-09-19T09:01:10.164619+0000","last_peered":"2024-09-19T09:01:10.164619+0000","last_clean":"2024-09-19T09:01:10.164619+0000","last_became_active":"2024-09-19T09:00:44.944763+0000","last_became_peered":"2024-09-19T09:00:44.944763+0000","last_unstale":"2024-09-19T09:01:10.164619+0000","last_undegraded":"2024-09-19T09:01:10.164619+0000","last_fullsized":"2024-09-19T09:01:10.164619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:06.052994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49603,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":30,"num_read_kb":63,"num_write":29,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.954165+0000","last_change":"2024-09-19T09:00:46.960991+0000","last_active":"2024-09-19T09:00:47.954165+0000","last_peered":"2024-09-19T09:00:47.954165+0000","last_clean":"2024-09-19T09:00:47.954165+0000","last_became_active":"2024-09-19T09:00:46.958962+0000","last_became_peered":"2024-09-19T09:00:46.958962+0000","last_unstale":"2024-09-19T09:00:47.954165+0000","last_undegraded":"2024-09-19T09:00:47.954165+0000","last_fullsized":"2024-09-19T09:00:47.954165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:21.795694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:59.142128+0000","last_change":"2024-09-19T09:00:44.946176+0000","last_active":"2024-09-19T09:00:59.142128+0000","last_peered":"2024-09-19T09:00:59.142128+0000","last_clean":"2024-09-19T09:00:59.142128+0000","last_became_active":"2024-09-19T09:00:44.945869+0000","last_became_peered":"2024-09-19T09:00:44.945869+0000","last_unstale":"2024-09-19T09:00:59.142128+0000","last_undegraded":"2024-09-19T09:00:59.142128+0000","last_fullsized":"2024-09-19T09:00:59.142128+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:16.022184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.952567+0000","last_change":"2024-09-19T09:00:46.956345+0000","last_active":"2024-09-19T09:00:47.952567+0000","last_peered":"2024-09-19T09:00:47.952567+0000","last_clean":"2024-09-19T09:00:47.952567+0000","last_became_active":"2024-09-19T09:00:46.956057+0000","last_became_peered":"2024-09-19T09:00:46.956057+0000","last_unstale":"2024-09-19T09:00:47.952567+0000","last_undegraded":"2024-09-19T09:00:47.952567+0000","last_fullsized":"2024-09-19T09:00:47.952567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:27.510661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.140102+0000","last_change":"2024-09-19T09:00:44.945667+0000","last_active":"2024-09-19T09:00:58.140102+0000","last_peered":"2024-09-19T09:00:58.140102+0000","last_clean":"2024-09-19T09:00:58.140102+0000","last_became_active":"2024-09-19T09:00:44.945308+0000","last_became_peered":"2024-09-19T09:00:44.945308+0000","last_unstale":"2024-09-19T09:00:58.140102+0000","last_undegraded":"2024-09-19T09:00:58.140102+0000","last_fullsized":"2024-09-19T09:00:58.140102+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:00:11.304913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.965242+0000","last_change":"2024-09-19T09:00:46.965088+0000","last_active":"2024-09-19T09:00:47.965242+0000","last_peered":"2024-09-19T09:00:47.965242+0000","last_clean":"2024-09-19T09:00:47.965242+0000","last_became_active":"2024-09-19T09:00:46.964777+0000","last_became_peered":"2024-09-19T09:00:46.964777+0000","last_unstale":"2024-09-19T09:00:47.965242+0000","last_undegraded":"2024-09-19T09:00:47.965242+0000","last_fullsized":"2024-09-19T09:00:47.965242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:32.958984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":41,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:01:11.164753+0000","last_change":"2024-09-19T09:00:44.946193+0000","last_active":"2024-09-19T09:01:11.164753+0000","last_peered":"2024-09-19T09:01:11.164753+0000","last_clean":"2024-09-19T09:01:11.164753+0000","last_became_active":"2024-09-19T09:00:44.945168+0000","last_became_peered":"2024-09-19T09:00:44.945168+0000","last_unstale":"2024-09-19T09:01:11.164753+0000","last_undegraded":"2024-09-19T09:01:11.164753+0000","last_fullsized":"2024-09-19T09:01:11.164753+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:21.230689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964246+0000","last_change":"2024-09-19T09:00:46.959425+0000","last_active":"2024-09-19T09:00:47.964246+0000","last_peered":"2024-09-19T09:00:47.964246+0000","last_clean":"2024-09-19T09:00:47.964246+0000","last_became_active":"2024-09-19T09:00:46.958764+0000","last_became_peered":"2024-09-19T09:00:46.958764+0000","last_unstale":"2024-09-19T09:00:47.964246+0000","last_undegraded":"2024-09-19T09:00:47.964246+0000","last_fullsized":"2024-09-19T09:00:47.964246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:08:27.371080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.140076+0000","last_change":"2024-09-19T09:00:45.240084+0000","last_active":"2024-09-19T09:00:58.140076+0000","last_peered":"2024-09-19T09:00:58.140076+0000","last_clean":"2024-09-19T09:00:58.140076+0000","last_became_active":"2024-09-19T09:00:45.239222+0000","last_became_peered":"2024-09-19T09:00:45.239222+0000","last_unstale":"2024-09-19T09:00:58.140076+0000","last_undegraded":"2024-09-19T09:00:58.140076+0000","last_fullsized":"2024-09-19T09:00:58.140076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:00:56.643063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.968850+0000","last_change":"2024-09-19T09:00:46.963380+0000","last_active":"2024-09-19T09:00:47.968850+0000","last_peered":"2024-09-19T09:00:47.968850+0000","last_clean":"2024-09-19T09:00:47.968850+0000","last_became_active":"2024-09-19T09:00:46.962605+0000","last_became_peered":"2024-09-19T09:00:46.962605+0000","last_unstale":"2024-09-19T09:00:47.968850+0000","last_undegraded":"2024-09-19T09:00:47.968850+0000","last_fullsized":"2024-09-19T09:00:47.968850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:20:59.424753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:57.115299+0000","last_change":"2024-09-19T09:00:44.946138+0000","last_active":"2024-09-19T09:00:57.115299+0000","last_peered":"2024-09-19T09:00:57.115299+0000","last_clean":"2024-09-19T09:00:57.115299+0000","last_became_active":"2024-09-19T09:00:44.945793+0000","last_became_peered":"2024-09-19T09:00:44.945793+0000","last_unstale":"2024-09-19T09:00:57.115299+0000","last_undegraded":"2024-09-19T09:00:57.115299+0000","last_fullsized":"2024-09-19T09:00:57.115299+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:01.341560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.969341+0000","last_change":"2024-09-19T09:00:46.970360+0000","last_active":"2024-09-19T09:00:47.969341+0000","last_peered":"2024-09-19T09:00:47.969341+0000","last_clean":"2024-09-19T09:00:47.969341+0000","last_became_active":"2024-09-19T09:00:46.969971+0000","last_became_peered":"2024-09-19T09:00:46.969971+0000","last_unstale":"2024-09-19T09:00:47.969341+0000","last_undegraded":"2024-09-19T09:00:47.969341+0000","last_fullsized":"2024-09-19T09:00:47.969341+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:48.812158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.139613+0000","last_change":"2024-09-19T09:00:44.941713+0000","last_active":"2024-09-19T09:00:58.139613+0000","last_peered":"2024-09-19T09:00:58.139613+0000","last_clean":"2024-09-19T09:00:58.139613+0000","last_became_active":"2024-09-19T09:00:44.941373+0000","last_became_peered":"2024-09-19T09:00:44.941373+0000","last_unstale":"2024-09-19T09:00:58.139613+0000","last_undegraded":"2024-09-19T09:00:58.139613+0000","last_fullsized":"2024-09-19T09:00:58.139613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:31.157803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.965581+0000","last_change":"2024-09-19T09:00:46.959383+0000","last_active":"2024-09-19T09:00:47.965581+0000","last_peered":"2024-09-19T09:00:47.965581+0000","last_clean":"2024-09-19T09:00:47.965581+0000","last_became_active":"2024-09-19T09:00:46.958860+0000","last_became_peered":"2024-09-19T09:00:46.958860+0000","last_unstale":"2024-09-19T09:00:47.965581+0000","last_undegraded":"2024-09-19T09:00:47.965581+0000","last_fullsized":"2024-09-19T09:00:47.965581+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:32:04.423017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.139240+0000","last_change":"2024-09-19T09:00:44.944797+0000","last_active":"2024-09-19T09:00:58.139240+0000","last_peered":"2024-09-19T09:00:58.139240+0000","last_clean":"2024-09-19T09:00:58.139240+0000","last_became_active":"2024-09-19T09:00:44.944259+0000","last_became_peered":"2024-09-19T09:00:44.944259+0000","last_unstale":"2024-09-19T09:00:58.139240+0000","last_undegraded":"2024-09-19T09:00:58.139240+0000","last_fullsized":"2024-09-19T09:00:58.139240+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:58.996217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:49.684985+0000","last_change":"2024-09-19T09:00:46.964998+0000","last_active":"2024-09-19T09:00:49.684985+0000","last_peered":"2024-09-19T09:00:49.684985+0000","last_clean":"2024-09-19T09:00:49.684985+0000","last_became_active":"2024-09-19T09:00:46.964562+0000","last_became_peered":"2024-09-19T09:00:46.964562+0000","last_unstale":"2024-09-19T09:00:49.684985+0000","last_undegraded":"2024-09-19T09:00:49.684985+0000","last_fullsized":"2024-09-19T09:00:49.684985+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:00.089800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":39,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:01:11.161775+0000","last_change":"2024-09-19T09:00:45.239770+0000","last_active":"2024-09-19T09:01:11.161775+0000","last_peered":"2024-09-19T09:01:11.161775+0000","last_clean":"2024-09-19T09:01:11.161775+0000","last_became_active":"2024-09-19T09:00:45.239385+0000","last_became_peered":"2024-09-19T09:00:45.239385+0000","last_unstale":"2024-09-19T09:01:11.161775+0000","last_undegraded":"2024-09-19T09:01:11.161775+0000","last_fullsized":"2024-09-19T09:01:11.161775+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:21.109900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.966028+0000","last_change":"2024-09-19T09:00:46.967505+0000","last_active":"2024-09-19T09:00:47.966028+0000","last_peered":"2024-09-19T09:00:47.966028+0000","last_clean":"2024-09-19T09:00:47.966028+0000","last_became_active":"2024-09-19T09:00:46.967195+0000","last_became_peered":"2024-09-19T09:00:46.967195+0000","last_unstale":"2024-09-19T09:00:47.966028+0000","last_undegraded":"2024-09-19T09:00:47.966028+0000","last_fullsized":"2024-09-19T09:00:47.966028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:52:28.917333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":330,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964506+0000","last_change":"2024-09-19T08:59:33.821924+0000","last_active":"2024-09-19T09:00:47.964506+0000","last_peered":"2024-09-19T09:00:47.964506+0000","last_clean":"2024-09-19T09:00:47.964506+0000","last_became_active":"2024-09-19T08:59:33.821404+0000","last_became_peered":"2024-09-19T08:59:33.821404+0000","last_unstale":"2024-09-19T09:00:47.964506+0000","last_undegraded":"2024-09-19T09:00:47.964506+0000","last_fullsized":"2024-09-19T09:00:47.964506+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:56:07.830480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:56:07.830480+0000","last_clean_scrub_stamp":"2024-09-19T08:56:07.830480+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:21.763918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:59.142206+0000","last_change":"2024-09-19T09:00:44.943714+0000","last_active":"2024-09-19T09:00:59.142206+0000","last_peered":"2024-09-19T09:00:59.142206+0000","last_clean":"2024-09-19T09:00:59.142206+0000","last_became_active":"2024-09-19T09:00:44.943362+0000","last_became_peered":"2024-09-19T09:00:44.943362+0000","last_unstale":"2024-09-19T09:00:59.142206+0000","last_undegraded":"2024-09-19T09:00:59.142206+0000","last_fullsized":"2024-09-19T09:00:59.142206+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:30.015648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964922+0000","last_change":"2024-09-19T09:00:46.956784+0000","last_active":"2024-09-19T09:00:47.964922+0000","last_peered":"2024-09-19T09:00:47.964922+0000","last_clean":"2024-09-19T09:00:47.964922+0000","last_became_active":"2024-09-19T09:00:46.956443+0000","last_became_peered":"2024-09-19T09:00:46.956443+0000","last_unstale":"2024-09-19T09:00:47.964922+0000","last_undegraded":"2024-09-19T09:00:47.964922+0000","last_fullsized":"2024-09-19T09:00:47.964922+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:18.918788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:56.119049+0000","last_change":"2024-09-19T09:00:44.947796+0000","last_active":"2024-09-19T09:00:56.119049+0000","last_peered":"2024-09-19T09:00:56.119049+0000","last_clean":"2024-09-19T09:00:56.119049+0000","last_became_active":"2024-09-19T09:00:44.946949+0000","last_became_peered":"2024-09-19T09:00:44.946949+0000","last_unstale":"2024-09-19T09:00:56.119049+0000","last_undegraded":"2024-09-19T09:00:56.119049+0000","last_fullsized":"2024-09-19T09:00:56.119049+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:42.210186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.952537+0000","last_change":"2024-09-19T09:00:46.963856+0000","last_active":"2024-09-19T09:00:47.952537+0000","last_peered":"2024-09-19T09:00:47.952537+0000","last_clean":"2024-09-19T09:00:47.952537+0000","last_became_active":"2024-09-19T09:00:46.963543+0000","last_became_peered":"2024-09-19T09:00:46.963543+0000","last_unstale":"2024-09-19T09:00:47.952537+0000","last_undegraded":"2024-09-19T09:00:47.952537+0000","last_fullsized":"2024-09-19T09:00:47.952537+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:31.041028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":39,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:01:12.165450+0000","last_change":"2024-09-19T09:00:45.239269+0000","last_active":"2024-09-19T09:01:12.165450+0000","last_peered":"2024-09-19T09:01:12.165450+0000","last_clean":"2024-09-19T09:01:12.165450+0000","last_became_active":"2024-09-19T09:00:45.238929+0000","last_became_peered":"2024-09-19T09:00:45.238929+0000","last_unstale":"2024-09-19T09:01:12.165450+0000","last_undegraded":"2024-09-19T09:01:12.165450+0000","last_fullsized":"2024-09-19T09:01:12.165450+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:46.684148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.968730+0000","last_change":"2024-09-19T09:00:46.963595+0000","last_active":"2024-09-19T09:00:47.968730+0000","last_peered":"2024-09-19T09:00:47.968730+0000","last_clean":"2024-09-19T09:00:47.968730+0000","last_became_active":"2024-09-19T09:00:46.963243+0000","last_became_peered":"2024-09-19T09:00:46.963243+0000","last_unstale":"2024-09-19T09:00:47.968730+0000","last_undegraded":"2024-09-19T09:00:47.968730+0000","last_fullsized":"2024-09-19T09:00:47.968730+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:01.122996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":35,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:01:08.156777+0000","last_change":"2024-09-19T09:00:44.941550+0000","last_active":"2024-09-19T09:01:08.156777+0000","last_peered":"2024-09-19T09:01:08.156777+0000","last_clean":"2024-09-19T09:01:08.156777+0000","last_became_active":"2024-09-19T09:00:44.941145+0000","last_became_peered":"2024-09-19T09:00:44.941145+0000","last_unstale":"2024-09-19T09:01:08.156777+0000","last_undegraded":"2024-09-19T09:01:08.156777+0000","last_fullsized":"2024-09-19T09:01:08.156777+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:53.893887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964440+0000","last_change":"2024-09-19T09:00:46.963473+0000","last_active":"2024-09-19T09:00:47.964440+0000","last_peered":"2024-09-19T09:00:47.964440+0000","last_clean":"2024-09-19T09:00:47.964440+0000","last_became_active":"2024-09-19T09:00:46.963090+0000","last_became_peered":"2024-09-19T09:00:46.963090+0000","last_unstale":"2024-09-19T09:00:47.964440+0000","last_undegraded":"2024-09-19T09:00:47.964440+0000","last_fullsized":"2024-09-19T09:00:47.964440+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:01.492385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.962784+0000","last_change":"2024-09-19T09:00:45.239566+0000","last_active":"2024-09-19T09:00:47.962784+0000","last_peered":"2024-09-19T09:00:47.962784+0000","last_clean":"2024-09-19T09:00:47.962784+0000","last_became_active":"2024-09-19T09:00:45.239250+0000","last_became_peered":"2024-09-19T09:00:45.239250+0000","last_unstale":"2024-09-19T09:00:47.962784+0000","last_undegraded":"2024-09-19T09:00:47.962784+0000","last_fullsized":"2024-09-19T09:00:47.962784+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:44:21.739311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:50.839180+0000","last_change":"2024-09-19T09:00:46.955344+0000","last_active":"2024-09-19T09:00:50.839180+0000","last_peered":"2024-09-19T09:00:50.839180+0000","last_clean":"2024-09-19T09:00:50.839180+0000","last_became_active":"2024-09-19T09:00:46.954961+0000","last_became_peered":"2024-09-19T09:00:46.954961+0000","last_unstale":"2024-09-19T09:00:50.839180+0000","last_undegraded":"2024-09-19T09:00:50.839180+0000","last_fullsized":"2024-09-19T09:00:50.839180+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:55.743556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.141069+0000","last_change":"2024-09-19T09:00:45.241514+0000","last_active":"2024-09-19T09:00:58.141069+0000","last_peered":"2024-09-19T09:00:58.141069+0000","last_clean":"2024-09-19T09:00:58.141069+0000","last_became_active":"2024-09-19T09:00:45.241097+0000","last_became_peered":"2024-09-19T09:00:45.241097+0000","last_unstale":"2024-09-19T09:00:58.141069+0000","last_undegraded":"2024-09-19T09:00:58.141069+0000","last_fullsized":"2024-09-19T09:00:58.141069+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:35:38.323393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.952874+0000","last_change":"2024-09-19T09:00:46.955194+0000","last_active":"2024-09-19T09:00:47.952874+0000","last_peered":"2024-09-19T09:00:47.952874+0000","last_clean":"2024-09-19T09:00:47.952874+0000","last_became_active":"2024-09-19T09:00:46.954771+0000","last_became_peered":"2024-09-19T09:00:46.954771+0000","last_unstale":"2024-09-19T09:00:47.952874+0000","last_undegraded":"2024-09-19T09:00:47.952874+0000","last_fullsized":"2024-09-19T09:00:47.952874+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:18.945035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'12","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:01:07.964867+0000","last_change":"2024-09-19T09:00:44.938554+0000","last_active":"2024-09-19T09:01:07.964867+0000","last_peered":"2024-09-19T09:01:07.964867+0000","last_clean":"2024-09-19T09:01:07.964867+0000","last_became_active":"2024-09-19T09:00:44.938261+0000","last_became_peered":"2024-09-19T09:00:44.938261+0000","last_unstale":"2024-09-19T09:01:07.964867+0000","last_undegraded":"2024-09-19T09:01:07.964867+0000","last_fullsized":"2024-09-19T09:01:07.964867+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:24.293764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964806+0000","last_change":"2024-09-19T09:00:46.960435+0000","last_active":"2024-09-19T09:00:47.964806+0000","last_peered":"2024-09-19T09:00:47.964806+0000","last_clean":"2024-09-19T09:00:47.964806+0000","last_became_active":"2024-09-19T09:00:46.960138+0000","last_became_peered":"2024-09-19T09:00:46.960138+0000","last_unstale":"2024-09-19T09:00:47.964806+0000","last_undegraded":"2024-09-19T09:00:47.964806+0000","last_fullsized":"2024-09-19T09:00:47.964806+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:59:50.853344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.139592+0000","last_change":"2024-09-19T09:00:44.941479+0000","last_active":"2024-09-19T09:00:58.139592+0000","last_peered":"2024-09-19T09:00:58.139592+0000","last_clean":"2024-09-19T09:00:58.139592+0000","last_became_active":"2024-09-19T09:00:44.937197+0000","last_became_peered":"2024-09-19T09:00:44.937197+0000","last_unstale":"2024-09-19T09:00:58.139592+0000","last_undegraded":"2024-09-19T09:00:58.139592+0000","last_fullsized":"2024-09-19T09:00:58.139592+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:53:17.385884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.965041+0000","last_change":"2024-09-19T09:00:46.970319+0000","last_active":"2024-09-19T09:00:47.965041+0000","last_peered":"2024-09-19T09:00:47.965041+0000","last_clean":"2024-09-19T09:00:47.965041+0000","last_became_active":"2024-09-19T09:00:46.969899+0000","last_became_peered":"2024-09-19T09:00:46.969899+0000","last_unstale":"2024-09-19T09:00:47.965041+0000","last_undegraded":"2024-09-19T09:00:47.965041+0000","last_fullsized":"2024-09-19T09:00:47.965041+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:26:21.379231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:55.109457+0000","last_change":"2024-09-19T09:00:45.239525+0000","last_active":"2024-09-19T09:00:55.109457+0000","last_peered":"2024-09-19T09:00:55.109457+0000","last_clean":"2024-09-19T09:00:55.109457+0000","last_became_active":"2024-09-19T09:00:45.239213+0000","last_became_peered":"2024-09-19T09:00:45.239213+0000","last_unstale":"2024-09-19T09:00:55.109457+0000","last_undegraded":"2024-09-19T09:00:55.109457+0000","last_fullsized":"2024-09-19T09:00:55.109457+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:32.525025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.953631+0000","last_change":"2024-09-19T09:00:46.958573+0000","last_active":"2024-09-19T09:00:47.953631+0000","last_peered":"2024-09-19T09:00:47.953631+0000","last_clean":"2024-09-19T09:00:47.953631+0000","last_became_active":"2024-09-19T09:00:46.958274+0000","last_became_peered":"2024-09-19T09:00:46.958274+0000","last_unstale":"2024-09-19T09:00:47.953631+0000","last_undegraded":"2024-09-19T09:00:47.953631+0000","last_fullsized":"2024-09-19T09:00:47.953631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:17.249122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.139345+0000","last_change":"2024-09-19T09:00:44.945194+0000","last_active":"2024-09-19T09:00:58.139345+0000","last_peered":"2024-09-19T09:00:58.139345+0000","last_clean":"2024-09-19T09:00:58.139345+0000","last_became_active":"2024-09-19T09:00:44.944931+0000","last_became_peered":"2024-09-19T09:00:44.944931+0000","last_unstale":"2024-09-19T09:00:58.139345+0000","last_undegraded":"2024-09-19T09:00:58.139345+0000","last_fullsized":"2024-09-19T09:00:58.139345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:11.181150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.969572+0000","last_change":"2024-09-19T09:00:46.966895+0000","last_active":"2024-09-19T09:00:47.969572+0000","last_peered":"2024-09-19T09:00:47.969572+0000","last_clean":"2024-09-19T09:00:47.969572+0000","last_became_active":"2024-09-19T09:00:46.966124+0000","last_became_peered":"2024-09-19T09:00:46.966124+0000","last_unstale":"2024-09-19T09:00:47.969572+0000","last_undegraded":"2024-09-19T09:00:47.969572+0000","last_fullsized":"2024-09-19T09:00:47.969572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:14:04.320863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:57.127047+0000","last_change":"2024-09-19T09:00:44.944167+0000","last_active":"2024-09-19T09:00:57.127047+0000","last_peered":"2024-09-19T09:00:57.127047+0000","last_clean":"2024-09-19T09:00:57.127047+0000","last_became_active":"2024-09-19T09:00:44.943670+0000","last_became_peered":"2024-09-19T09:00:44.943670+0000","last_unstale":"2024-09-19T09:00:57.127047+0000","last_undegraded":"2024-09-19T09:00:57.127047+0000","last_fullsized":"2024-09-19T09:00:57.127047+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:05.596877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964961+0000","last_change":"2024-09-19T09:00:46.958581+0000","last_active":"2024-09-19T09:00:47.964961+0000","last_peered":"2024-09-19T09:00:47.964961+0000","last_clean":"2024-09-19T09:00:47.964961+0000","last_became_active":"2024-09-19T09:00:46.957782+0000","last_became_peered":"2024-09-19T09:00:46.957782+0000","last_unstale":"2024-09-19T09:00:47.964961+0000","last_undegraded":"2024-09-19T09:00:47.964961+0000","last_fullsized":"2024-09-19T09:00:47.964961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:37:19.488538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:55.110727+0000","last_change":"2024-09-19T09:00:44.945351+0000","last_active":"2024-09-19T09:00:55.110727+0000","last_peered":"2024-09-19T09:00:55.110727+0000","last_clean":"2024-09-19T09:00:55.110727+0000","last_became_active":"2024-09-19T09:00:44.944991+0000","last_became_peered":"2024-09-19T09:00:44.944991+0000","last_unstale":"2024-09-19T09:00:55.110727+0000","last_undegraded":"2024-09-19T09:00:55.110727+0000","last_fullsized":"2024-09-19T09:00:55.110727+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:45:07.581872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964606+0000","last_change":"2024-09-19T09:00:46.956340+0000","last_active":"2024-09-19T09:00:47.964606+0000","last_peered":"2024-09-19T09:00:47.964606+0000","last_clean":"2024-09-19T09:00:47.964606+0000","last_became_active":"2024-09-19T09:00:46.955956+0000","last_became_peered":"2024-09-19T09:00:46.955956+0000","last_unstale":"2024-09-19T09:00:47.964606+0000","last_undegraded":"2024-09-19T09:00:47.964606+0000","last_fullsized":"2024-09-19T09:00:47.964606+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:34.863428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.968838+0000","last_change":"2024-09-19T09:00:46.963657+0000","last_active":"2024-09-19T09:00:47.968838+0000","last_peered":"2024-09-19T09:00:47.968838+0000","last_clean":"2024-09-19T09:00:47.968838+0000","last_became_active":"2024-09-19T09:00:46.963134+0000","last_became_peered":"2024-09-19T09:00:46.963134+0000","last_unstale":"2024-09-19T09:00:47.968838+0000","last_undegraded":"2024-09-19T09:00:47.968838+0000","last_fullsized":"2024-09-19T09:00:47.968838+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:01.098570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.141281+0000","last_change":"2024-09-19T09:00:44.945382+0000","last_active":"2024-09-19T09:00:58.141281+0000","last_peered":"2024-09-19T09:00:58.141281+0000","last_clean":"2024-09-19T09:00:58.141281+0000","last_became_active":"2024-09-19T09:00:44.945061+0000","last_became_peered":"2024-09-19T09:00:44.945061+0000","last_unstale":"2024-09-19T09:00:58.141281+0000","last_undegraded":"2024-09-19T09:00:58.141281+0000","last_fullsized":"2024-09-19T09:00:58.141281+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:07:12.747709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.953216+0000","last_change":"2024-09-19T09:00:46.956354+0000","last_active":"2024-09-19T09:00:47.953216+0000","last_peered":"2024-09-19T09:00:47.953216+0000","last_clean":"2024-09-19T09:00:47.953216+0000","last_became_active":"2024-09-19T09:00:46.955964+0000","last_became_peered":"2024-09-19T09:00:46.955964+0000","last_unstale":"2024-09-19T09:00:47.953216+0000","last_undegraded":"2024-09-19T09:00:47.953216+0000","last_fullsized":"2024-09-19T09:00:47.953216+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:05:33.243479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:59.136540+0000","last_change":"2024-09-19T09:00:44.944983+0000","last_active":"2024-09-19T09:00:59.136540+0000","last_peered":"2024-09-19T09:00:59.136540+0000","last_clean":"2024-09-19T09:00:59.136540+0000","last_became_active":"2024-09-19T09:00:44.944625+0000","last_became_peered":"2024-09-19T09:00:44.944625+0000","last_unstale":"2024-09-19T09:00:59.136540+0000","last_undegraded":"2024-09-19T09:00:59.136540+0000","last_fullsized":"2024-09-19T09:00:59.136540+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:49:50.818863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.965462+0000","last_change":"2024-09-19T09:00:46.966448+0000","last_active":"2024-09-19T09:00:47.965462+0000","last_peered":"2024-09-19T09:00:47.965462+0000","last_clean":"2024-09-19T09:00:47.965462+0000","last_became_active":"2024-09-19T09:00:46.966135+0000","last_became_peered":"2024-09-19T09:00:46.966135+0000","last_unstale":"2024-09-19T09:00:47.965462+0000","last_undegraded":"2024-09-19T09:00:47.965462+0000","last_fullsized":"2024-09-19T09:00:47.965462+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:42.652877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.967059+0000","last_change":"2024-09-19T09:00:45.239405+0000","last_active":"2024-09-19T09:00:47.967059+0000","last_peered":"2024-09-19T09:00:47.967059+0000","last_clean":"2024-09-19T09:00:47.967059+0000","last_became_active":"2024-09-19T09:00:45.239002+0000","last_became_peered":"2024-09-19T09:00:45.239002+0000","last_unstale":"2024-09-19T09:00:47.967059+0000","last_undegraded":"2024-09-19T09:00:47.967059+0000","last_fullsized":"2024-09-19T09:00:47.967059+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:46:03.934975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.965411+0000","last_change":"2024-09-19T09:00:46.951635+0000","last_active":"2024-09-19T09:00:47.965411+0000","last_peered":"2024-09-19T09:00:47.965411+0000","last_clean":"2024-09-19T09:00:47.965411+0000","last_became_active":"2024-09-19T09:00:46.951325+0000","last_became_peered":"2024-09-19T09:00:46.951325+0000","last_unstale":"2024-09-19T09:00:47.965411+0000","last_undegraded":"2024-09-19T09:00:47.965411+0000","last_fullsized":"2024-09-19T09:00:47.965411+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:17.360950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":45,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:01:10.158975+0000","last_change":"2024-09-19T09:00:44.942662+0000","last_active":"2024-09-19T09:01:10.158975+0000","last_peered":"2024-09-19T09:01:10.158975+0000","last_clean":"2024-09-19T09:01:10.158975+0000","last_became_active":"2024-09-19T09:00:44.942397+0000","last_became_peered":"2024-09-19T09:00:44.942397+0000","last_unstale":"2024-09-19T09:01:10.158975+0000","last_undegraded":"2024-09-19T09:01:10.158975+0000","last_fullsized":"2024-09-19T09:01:10.158975+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:46:35.247703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":56418,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":166,"num_read_kb":146,"num_write":234,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":169254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":212166,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612625,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":860160,"data_stored":410932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841558,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":1051957,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972380,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":410862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201312,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":411240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430246,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":1003760,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593772,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":410579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822704,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":1003045,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986230,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":410833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215162,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":410404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":461315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":461095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":411918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":51355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":51575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":51732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:01:15.382 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T09:01:15.383 INFO:tasks.cephadm:Setting up client nodes... 2024-09-19T09:01:15.383 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-19T09:01:15.634 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:15.754 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:15 smithi120 ceph-mon[31150]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:01:15.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:15 smithi196 ceph-mon[35150]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:01:15.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:15 smithi153 ceph-mon[35071]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:01:16.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.117+0000 7f5d2661b640 1 -- 172.21.15.120:0/4269323536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d20101c00 msgr2=0x7f5d20102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:16.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.117+0000 7f5d2661b640 1 --2- 172.21.15.120:0/4269323536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d20101c00 0x7f5d20102000 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5d10009900 tx=0x7f5d1002a410 comp rx=0 tx=0).stop 2024-09-19T09:01:16.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.117+0000 7f5d2661b640 1 -- 172.21.15.120:0/4269323536 shutdown_connections 2024-09-19T09:01:16.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.117+0000 7f5d2661b640 1 --2- 172.21.15.120:0/4269323536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d201037c0 0x7f5d2010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:16.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.117+0000 7f5d2661b640 1 --2- 172.21.15.120:0/4269323536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d20102e00 0x7f5d20103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:16.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.117+0000 7f5d2661b640 1 --2- 172.21.15.120:0/4269323536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d20101c00 0x7f5d20102000 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:16.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.117+0000 7f5d2661b640 1 -- 172.21.15.120:0/4269323536 >> 172.21.15.120:0/4269323536 conn(0x7f5d200fd390 msgr2=0x7f5d200ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:16.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.117+0000 7f5d2661b640 1 -- 172.21.15.120:0/4269323536 shutdown_connections 2024-09-19T09:01:16.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.117+0000 7f5d2661b640 1 -- 172.21.15.120:0/4269323536 wait complete. 2024-09-19T09:01:16.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.118+0000 7f5d2661b640 1 Processor -- start 2024-09-19T09:01:16.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.118+0000 7f5d2661b640 1 -- start start 2024-09-19T09:01:16.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.118+0000 7f5d2661b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d20101c00 0x7f5d20075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:16.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.118+0000 7f5d2661b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d20102e00 0x7f5d200745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:16.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.118+0000 7f5d2661b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d201037c0 0x7f5d20074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:16.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.118+0000 7f5d2661b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2010d860 con 0x7f5d20102e00 2024-09-19T09:01:16.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.118+0000 7f5d2661b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2010d6e0 con 0x7f5d201037c0 2024-09-19T09:01:16.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.118+0000 7f5d2661b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2010d9e0 con 0x7f5d20101c00 2024-09-19T09:01:16.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.118+0000 7f5d1f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d20102e00 0x7f5d200745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:16.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.118+0000 7f5d1f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d20102e00 0x7f5d200745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42810/0 (socket says 172.21.15.120:42810) 2024-09-19T09:01:16.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.118+0000 7f5d1f7fe640 1 -- 172.21.15.120:0/1470771368 learned_addr learned my addr 172.21.15.120:0/1470771368 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:16.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.118+0000 7f5d24bb8640 1 --2- 172.21.15.120:0/1470771368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d201037c0 0x7f5d20074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:16.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.118+0000 7f5d1f7fe640 1 -- 172.21.15.120:0/1470771368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d20101c00 msgr2=0x7f5d20075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:16.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.118+0000 7f5d1f7fe640 1 --2- 172.21.15.120:0/1470771368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d20101c00 0x7f5d20075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:16.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.118+0000 7f5d1f7fe640 1 -- 172.21.15.120:0/1470771368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d201037c0 msgr2=0x7f5d20074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:16.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.118+0000 7f5d1f7fe640 1 --2- 172.21.15.120:0/1470771368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d201037c0 0x7f5d20074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:16.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.118+0000 7f5d1f7fe640 1 -- 172.21.15.120:0/1470771368 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d200753e0 con 0x7f5d20102e00 2024-09-19T09:01:16.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.119+0000 7f5d1f7fe640 1 --2- 172.21.15.120:0/1470771368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d20102e00 0x7f5d200745e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5d1400d950 tx=0x7f5d1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:16.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.119+0000 7f5d1d7fa640 1 -- 172.21.15.120:0/1470771368 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d14017070 con 0x7f5d20102e00 2024-09-19T09:01:16.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.119+0000 7f5d1d7fa640 1 -- 172.21.15.120:0/1470771368 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d14009c30 con 0x7f5d20102e00 2024-09-19T09:01:16.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.119+0000 7f5d2661b640 1 -- 172.21.15.120:0/1470771368 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d201a2090 con 0x7f5d20102e00 2024-09-19T09:01:16.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.119+0000 7f5d1d7fa640 1 -- 172.21.15.120:0/1470771368 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1400f6b0 con 0x7f5d20102e00 2024-09-19T09:01:16.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.119+0000 7f5d2661b640 1 -- 172.21.15.120:0/1470771368 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d201a25f0 con 0x7f5d20102e00 2024-09-19T09:01:16.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.120+0000 7f5d2661b640 1 -- 172.21.15.120:0/1470771368 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d20109000 con 0x7f5d20102e00 2024-09-19T09:01:16.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.121+0000 7f5d1d7fa640 1 -- 172.21.15.120:0/1470771368 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f5d140109d0 con 0x7f5d20102e00 2024-09-19T09:01:16.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.122+0000 7f5d1d7fa640 1 --2- 172.21.15.120:0/1470771368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5d080a3350 0x7f5d080a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:16.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.122+0000 7f5d1ffff640 1 --2- 172.21.15.120:0/1470771368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5d080a3350 0x7f5d080a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:16.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.122+0000 7f5d1d7fa640 1 -- 172.21.15.120:0/1470771368 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f5d140cf610 con 0x7f5d20102e00 2024-09-19T09:01:16.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.122+0000 7f5d1ffff640 1 --2- 172.21.15.120:0/1470771368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5d080a3350 0x7f5d080a5810 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5d100096c0 tx=0x7f5d10031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:16.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.126+0000 7f5d1d7fa640 1 -- 172.21.15.120:0/1470771368 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5d14009de0 con 0x7f5d20102e00 2024-09-19T09:01:16.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.358+0000 7f5d2661b640 1 -- 172.21.15.120:0/1470771368 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f5d2002d7c0 con 0x7f5d20102e00 2024-09-19T09:01:16.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.361+0000 7f5d1d7fa640 1 -- 172.21.15.120:0/1470771368 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f5d1409c280 con 0x7f5d20102e00 2024-09-19T09:01:16.361 INFO:teuthology.orchestra.run.smithi120.stdout:[client.0] 2024-09-19T09:01:16.361 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBc6Otm+tBzFRAAyGLxUTt41QrPaJPwJgkRKA== 2024-09-19T09:01:16.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.364+0000 7f5d2661b640 1 -- 172.21.15.120:0/1470771368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5d080a3350 msgr2=0x7f5d080a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:16.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.364+0000 7f5d2661b640 1 --2- 172.21.15.120:0/1470771368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5d080a3350 0x7f5d080a5810 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5d100096c0 tx=0x7f5d10031400 comp rx=0 tx=0).stop 2024-09-19T09:01:16.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.364+0000 7f5d2661b640 1 -- 172.21.15.120:0/1470771368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d20102e00 msgr2=0x7f5d200745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:16.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.364+0000 7f5d2661b640 1 --2- 172.21.15.120:0/1470771368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d20102e00 0x7f5d200745e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5d1400d950 tx=0x7f5d1400de10 comp rx=0 tx=0).stop 2024-09-19T09:01:16.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.364+0000 7f5d2661b640 1 -- 172.21.15.120:0/1470771368 shutdown_connections 2024-09-19T09:01:16.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.364+0000 7f5d2661b640 1 --2- 172.21.15.120:0/1470771368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5d080a3350 0x7f5d080a5810 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:16.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.364+0000 7f5d2661b640 1 --2- 172.21.15.120:0/1470771368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d201037c0 0x7f5d20074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:16.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.364+0000 7f5d2661b640 1 --2- 172.21.15.120:0/1470771368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d20102e00 0x7f5d200745e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:16.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.364+0000 7f5d2661b640 1 --2- 172.21.15.120:0/1470771368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d20101c00 0x7f5d20075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:16.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.364+0000 7f5d2661b640 1 -- 172.21.15.120:0/1470771368 >> 172.21.15.120:0/1470771368 conn(0x7f5d200fd390 msgr2=0x7f5d201061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:16.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.364+0000 7f5d2661b640 1 -- 172.21.15.120:0/1470771368 shutdown_connections 2024-09-19T09:01:16.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:16.364+0000 7f5d2661b640 1 -- 172.21.15.120:0/1470771368 wait complete. 2024-09-19T09:01:16.787 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:16 smithi120 ceph-mon[31150]: pgmap v291: 97 pgs: 97 active+clean; 632 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-19T09:01:16.788 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:16 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1470771368' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T09:01:16.788 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:16 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1470771368' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T09:01:16.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:16 smithi196 ceph-mon[35150]: pgmap v291: 97 pgs: 97 active+clean; 632 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-19T09:01:16.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:16 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1470771368' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T09:01:16.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:16 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1470771368' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T09:01:16.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:16 smithi153 ceph-mon[35071]: pgmap v291: 97 pgs: 97 active+clean; 632 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-19T09:01:16.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:16 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1470771368' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T09:01:16.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:16 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1470771368' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T09:01:17.221 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-19T09:01:17.221 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-19T09:01:17.222 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-19T09:01:17.258 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-19T09:01:17.258 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-19T09:01:17.259 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-19T09:01:17.543 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:18.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.042+0000 7f15c290e640 1 -- 172.21.15.120:0/614516701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15bc100e20 msgr2=0x7f15bc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:18.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.042+0000 7f15c290e640 1 --2- 172.21.15.120:0/614516701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15bc100e20 0x7f15bc101200 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009900 tx=0x7f15ac02a480 comp rx=0 tx=0).stop 2024-09-19T09:01:18.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.043+0000 7f15c290e640 1 -- 172.21.15.120:0/614516701 shutdown_connections 2024-09-19T09:01:18.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.043+0000 7f15c290e640 1 --2- 172.21.15.120:0/614516701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15bc102ef0 0x7f15bc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:18.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.043+0000 7f15c290e640 1 --2- 172.21.15.120:0/614516701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15bc102570 0x7f15bc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:18.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.043+0000 7f15c290e640 1 --2- 172.21.15.120:0/614516701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15bc100e20 0x7f15bc101200 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:18.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.043+0000 7f15c290e640 1 -- 172.21.15.120:0/614516701 >> 172.21.15.120:0/614516701 conn(0x7f15bc0fc7d0 msgr2=0x7f15bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:18.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.043+0000 7f15c290e640 1 -- 172.21.15.120:0/614516701 shutdown_connections 2024-09-19T09:01:18.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.043+0000 7f15c290e640 1 -- 172.21.15.120:0/614516701 wait complete. 2024-09-19T09:01:18.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.043+0000 7f15c290e640 1 Processor -- start 2024-09-19T09:01:18.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.043+0000 7f15c290e640 1 -- start start 2024-09-19T09:01:18.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.043+0000 7f15c290e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15bc100e20 0x7f15bc1973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:18.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.043+0000 7f15c290e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15bc102570 0x7f15bc1978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:18.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.043+0000 7f15c290e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15bc102ef0 0x7f15bc19bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:18.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.044+0000 7f15c290e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc110f80 con 0x7f15bc102ef0 2024-09-19T09:01:18.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.044+0000 7f15c290e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc110e00 con 0x7f15bc102570 2024-09-19T09:01:18.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.044+0000 7f15c290e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc111100 con 0x7f15bc100e20 2024-09-19T09:01:18.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.044+0000 7f15c0eab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15bc102ef0 0x7f15bc19bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:18.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.044+0000 7f15c0eab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15bc102ef0 0x7f15bc19bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42826/0 (socket says 172.21.15.120:42826) 2024-09-19T09:01:18.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.044+0000 7f15c0eab640 1 -- 172.21.15.120:0/2882187746 learned_addr learned my addr 172.21.15.120:0/2882187746 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:18.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.044+0000 7f15bbfff640 1 --2- 172.21.15.120:0/2882187746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15bc100e20 0x7f15bc1973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:18.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.044+0000 7f15bb7fe640 1 --2- 172.21.15.120:0/2882187746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15bc102570 0x7f15bc1978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:18.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.044+0000 7f15c0eab640 1 -- 172.21.15.120:0/2882187746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15bc100e20 msgr2=0x7f15bc1973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:18.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.044+0000 7f15c0eab640 1 --2- 172.21.15.120:0/2882187746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15bc100e20 0x7f15bc1973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:18.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.044+0000 7f15c0eab640 1 -- 172.21.15.120:0/2882187746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15bc102570 msgr2=0x7f15bc1978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:18.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.044+0000 7f15c0eab640 1 --2- 172.21.15.120:0/2882187746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15bc102570 0x7f15bc1978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:18.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.044+0000 7f15c0eab640 1 -- 172.21.15.120:0/2882187746 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15bc19c360 con 0x7f15bc102ef0 2024-09-19T09:01:18.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.044+0000 7f15c0eab640 1 --2- 172.21.15.120:0/2882187746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15bc102ef0 0x7f15bc19bc80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f15b400ebb0 tx=0x7f15b4007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:18.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.044+0000 7f15b97fa640 1 -- 172.21.15.120:0/2882187746 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b4007ca0 con 0x7f15bc102ef0 2024-09-19T09:01:18.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.044+0000 7f15b97fa640 1 -- 172.21.15.120:0/2882187746 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15b4007e40 con 0x7f15bc102ef0 2024-09-19T09:01:18.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.044+0000 7f15c290e640 1 -- 172.21.15.120:0/2882187746 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15bc19c5f0 con 0x7f15bc102ef0 2024-09-19T09:01:18.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.044+0000 7f15b97fa640 1 -- 172.21.15.120:0/2882187746 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b40058a0 con 0x7f15bc102ef0 2024-09-19T09:01:18.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.044+0000 7f15c290e640 1 -- 172.21.15.120:0/2882187746 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15bc069f70 con 0x7f15bc102ef0 2024-09-19T09:01:18.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.045+0000 7f15c290e640 1 -- 172.21.15.120:0/2882187746 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15bc1021b0 con 0x7f15bc102ef0 2024-09-19T09:01:18.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.046+0000 7f15b97fa640 1 -- 172.21.15.120:0/2882187746 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f15b4004a40 con 0x7f15bc102ef0 2024-09-19T09:01:18.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.047+0000 7f15b97fa640 1 --2- 172.21.15.120:0/2882187746 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f15a40a32d0 0x7f15a40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:18.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.047+0000 7f15bbfff640 1 --2- 172.21.15.120:0/2882187746 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f15a40a32d0 0x7f15a40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:18.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.047+0000 7f15b97fa640 1 -- 172.21.15.120:0/2882187746 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f15b40d0e90 con 0x7f15bc102ef0 2024-09-19T09:01:18.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.047+0000 7f15bbfff640 1 --2- 172.21.15.120:0/2882187746 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f15a40a32d0 0x7f15a40a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009230 tx=0x7f15ac0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:18.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.051+0000 7f15b97fa640 1 -- 172.21.15.120:0/2882187746 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f15b409dc00 con 0x7f15bc102ef0 2024-09-19T09:01:18.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.263+0000 7f15c290e640 1 -- 172.21.15.120:0/2882187746 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f15bc198710 con 0x7f15bc102ef0 2024-09-19T09:01:18.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.269+0000 7f15b97fa640 1 -- 172.21.15.120:0/2882187746 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264059 (secure 0 0 0) 0x7f15b409d500 con 0x7f15bc102ef0 2024-09-19T09:01:18.272 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:01:18.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.272+0000 7f15c290e640 1 -- 172.21.15.120:0/2882187746 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f15a40a32d0 msgr2=0x7f15a40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:18.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.272+0000 7f15c290e640 1 --2- 172.21.15.120:0/2882187746 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f15a40a32d0 0x7f15a40a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009230 tx=0x7f15ac0058d0 comp rx=0 tx=0).stop 2024-09-19T09:01:18.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.272+0000 7f15c290e640 1 -- 172.21.15.120:0/2882187746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15bc102ef0 msgr2=0x7f15bc19bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:18.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.272+0000 7f15c290e640 1 --2- 172.21.15.120:0/2882187746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15bc102ef0 0x7f15bc19bc80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f15b400ebb0 tx=0x7f15b4007590 comp rx=0 tx=0).stop 2024-09-19T09:01:18.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.272+0000 7f15c290e640 1 -- 172.21.15.120:0/2882187746 shutdown_connections 2024-09-19T09:01:18.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.272+0000 7f15c290e640 1 --2- 172.21.15.120:0/2882187746 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f15a40a32d0 0x7f15a40a5790 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:18.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.272+0000 7f15c290e640 1 --2- 172.21.15.120:0/2882187746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15bc102ef0 0x7f15bc19bc80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:18.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.272+0000 7f15c290e640 1 --2- 172.21.15.120:0/2882187746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15bc102570 0x7f15bc1978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:18.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.272+0000 7f15c290e640 1 --2- 172.21.15.120:0/2882187746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15bc100e20 0x7f15bc1973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:18.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.272+0000 7f15c290e640 1 -- 172.21.15.120:0/2882187746 >> 172.21.15.120:0/2882187746 conn(0x7f15bc0fc7d0 msgr2=0x7f15bc0fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:18.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.272+0000 7f15c290e640 1 -- 172.21.15.120:0/2882187746 shutdown_connections 2024-09-19T09:01:18.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:18.272+0000 7f15c290e640 1 -- 172.21.15.120:0/2882187746 wait complete. 2024-09-19T09:01:18.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:18 smithi120 ceph-mon[31150]: pgmap v292: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-09-19T09:01:18.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:18 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2882187746' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T09:01:18.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:18 smithi196 ceph-mon[35150]: pgmap v292: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-09-19T09:01:18.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:18 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2882187746' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T09:01:18.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:18 smithi153 ceph-mon[35071]: pgmap v292: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-09-19T09:01:18.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:18 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2882187746' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T09:01:19.094 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3841841273},{"type":"v1","addr":"172.21.15.120:6801","nonce":3841841273}]},"active_addr":"172.21.15.120:6801/3841841273","active_change":"2024-09-19T08:53:51.528880+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.120:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.120:0","nonce":3651871805}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.120:0","nonce":3713436834}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.120:0","nonce":2672339263}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.120:0","nonce":2066332567}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.120:0","nonce":2715033501}]}]} 2024-09-19T09:01:19.100 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-19T09:01:19.100 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-19T09:01:19.100 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:01:19.349 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:19.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.889+0000 7f1e2b49a640 1 -- 172.21.15.120:0/656786493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e24106f10 msgr2=0x7f1e2410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:19.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.889+0000 7f1e2b49a640 1 --2- 172.21.15.120:0/656786493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e24106f10 0x7f1e2410ea90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c009ca0 tx=0x7f1e1c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:01:19.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.889+0000 7f1e2b49a640 1 -- 172.21.15.120:0/656786493 shutdown_connections 2024-09-19T09:01:19.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.889+0000 7f1e2b49a640 1 --2- 172.21.15.120:0/656786493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e24106f10 0x7f1e2410ea90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:19.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.889+0000 7f1e2b49a640 1 --2- 172.21.15.120:0/656786493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e24102790 0x7f1e241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:19.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.889+0000 7f1e2b49a640 1 --2- 172.21.15.120:0/656786493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e24101de0 0x7f1e241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:19.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.889+0000 7f1e2b49a640 1 -- 172.21.15.120:0/656786493 >> 172.21.15.120:0/656786493 conn(0x7f1e240fd8c0 msgr2=0x7f1e240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:19.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.890+0000 7f1e2b49a640 1 -- 172.21.15.120:0/656786493 shutdown_connections 2024-09-19T09:01:19.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.890+0000 7f1e2b49a640 1 -- 172.21.15.120:0/656786493 wait complete. 2024-09-19T09:01:19.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.890+0000 7f1e2b49a640 1 Processor -- start 2024-09-19T09:01:19.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.890+0000 7f1e2b49a640 1 -- start start 2024-09-19T09:01:19.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.891+0000 7f1e2b49a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e24101de0 0x7f1e24198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:19.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.891+0000 7f1e2b49a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e24102790 0x7f1e24198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:19.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.891+0000 7f1e2b49a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e24106f10 0x7f1e2419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:19.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.891+0000 7f1e2b49a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1e24111f90 con 0x7f1e24102790 2024-09-19T09:01:19.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.891+0000 7f1e2b49a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1e24111e10 con 0x7f1e24106f10 2024-09-19T09:01:19.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.891+0000 7f1e2b49a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1e24112110 con 0x7f1e24101de0 2024-09-19T09:01:19.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.891+0000 7f1e28a35640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e24102790 0x7f1e24198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:19.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.891+0000 7f1e28a35640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e24102790 0x7f1e24198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42846/0 (socket says 172.21.15.120:42846) 2024-09-19T09:01:19.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.891+0000 7f1e28a35640 1 -- 172.21.15.120:0/2459728401 learned_addr learned my addr 172.21.15.120:0/2459728401 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:19.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.891+0000 7f1e29a37640 1 --2- 172.21.15.120:0/2459728401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e24106f10 0x7f1e2419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:19.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.891+0000 7f1e29236640 1 --2- 172.21.15.120:0/2459728401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e24101de0 0x7f1e24198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:19.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.891+0000 7f1e28a35640 1 -- 172.21.15.120:0/2459728401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e24101de0 msgr2=0x7f1e24198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:19.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.891+0000 7f1e28a35640 1 --2- 172.21.15.120:0/2459728401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e24101de0 0x7f1e24198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:19.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.891+0000 7f1e28a35640 1 -- 172.21.15.120:0/2459728401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e24106f10 msgr2=0x7f1e2419ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:19.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.891+0000 7f1e28a35640 1 --2- 172.21.15.120:0/2459728401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e24106f10 0x7f1e2419ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:19.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.891+0000 7f1e28a35640 1 -- 172.21.15.120:0/2459728401 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2419d500 con 0x7f1e24102790 2024-09-19T09:01:19.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.891+0000 7f1e28a35640 1 --2- 172.21.15.120:0/2459728401 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e24102790 0x7f1e24198a90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400b4a0 tx=0x7f1e1400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:19.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.891+0000 7f1e1a7fc640 1 -- 172.21.15.120:0/2459728401 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e14014070 con 0x7f1e24102790 2024-09-19T09:01:19.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.891+0000 7f1e2b49a640 1 -- 172.21.15.120:0/2459728401 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e2419d7f0 con 0x7f1e24102790 2024-09-19T09:01:19.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.891+0000 7f1e1a7fc640 1 -- 172.21.15.120:0/2459728401 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e14010af0 con 0x7f1e24102790 2024-09-19T09:01:19.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.891+0000 7f1e1a7fc640 1 -- 172.21.15.120:0/2459728401 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e14017d10 con 0x7f1e24102790 2024-09-19T09:01:19.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.891+0000 7f1e2b49a640 1 -- 172.21.15.120:0/2459728401 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e241a50d0 con 0x7f1e24102790 2024-09-19T09:01:19.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.892+0000 7f1e2b49a640 1 -- 172.21.15.120:0/2459728401 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e241021c0 con 0x7f1e24102790 2024-09-19T09:01:19.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.893+0000 7f1e1a7fc640 1 -- 172.21.15.120:0/2459728401 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f1e14017070 con 0x7f1e24102790 2024-09-19T09:01:19.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.894+0000 7f1e1a7fc640 1 --2- 172.21.15.120:0/2459728401 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1e100a32d0 0x7f1e100a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:19.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.894+0000 7f1e29236640 1 --2- 172.21.15.120:0/2459728401 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1e100a32d0 0x7f1e100a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:19.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.894+0000 7f1e1a7fc640 1 -- 172.21.15.120:0/2459728401 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f1e140ceb90 con 0x7f1e24102790 2024-09-19T09:01:19.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.895+0000 7f1e29236640 1 --2- 172.21.15.120:0/2459728401 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1e100a32d0 0x7f1e100a5790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1e20005fd0 tx=0x7f1e20005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:19.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:19.898+0000 7f1e1a7fc640 1 -- 172.21.15.120:0/2459728401 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1e1409ba10 con 0x7f1e24102790 2024-09-19T09:01:20.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:20.067+0000 7f1e2b49a640 1 -- 172.21.15.120:0/2459728401 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e2410a180 con 0x7f1e24102790 2024-09-19T09:01:20.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:20.069+0000 7f1e1a7fc640 1 -- 172.21.15.120:0/2459728401 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22362 (secure 0 0 0) 0x7f1e1409b310 con 0x7f1e24102790 2024-09-19T09:01:20.069 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:01:20.069 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":79,"fsid":"507393d2-7664-11ef-bceb-c7b262605968","created":"2024-09-19T08:53:06.386526+0000","modified":"2024-09-19T09:00:47.659465+0000","last_up_change":"2024-09-19T08:59:55.692659+0000","last_in_change":"2024-09-19T08:59:32.808860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:56:07.830047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:00:43.774339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:00:45.786262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33ec3993-576c-428c-bf75-adcb17cd72a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6803","nonce":4131180019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6805","nonce":4131180019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6809","nonce":4131180019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6807","nonce":4131180019}]},"public_addr":"172.21.15.120:6803/4131180019","cluster_addr":"172.21.15.120:6805/4131180019","heartbeat_back_addr":"172.21.15.120:6809/4131180019","heartbeat_front_addr":"172.21.15.120:6807/4131180019","state":["exists","up"]},{"osd":1,"uuid":"450bbebf-2aac-47c6-8474-bbd37fd0a1a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6801","nonce":2259730294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6803","nonce":2259730294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6807","nonce":2259730294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6805","nonce":2259730294}]},"public_addr":"172.21.15.153:6801/2259730294","cluster_addr":"172.21.15.153:6803/2259730294","heartbeat_back_addr":"172.21.15.153:6807/2259730294","heartbeat_front_addr":"172.21.15.153:6805/2259730294","state":["exists","up"]},{"osd":2,"uuid":"ac8c4b83-8e8e-4803-b266-262f7873bd85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6801","nonce":2942286570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6803","nonce":2942286570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6807","nonce":2942286570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6805","nonce":2942286570}]},"public_addr":"172.21.15.196:6801/2942286570","cluster_addr":"172.21.15.196:6803/2942286570","heartbeat_back_addr":"172.21.15.196:6807/2942286570","heartbeat_front_addr":"172.21.15.196:6805/2942286570","state":["exists","up"]},{"osd":3,"uuid":"c1d9e507-8f02-403e-87da-9b4966fd8fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6811","nonce":713205668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6813","nonce":713205668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6817","nonce":713205668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6815","nonce":713205668}]},"public_addr":"172.21.15.120:6811/713205668","cluster_addr":"172.21.15.120:6813/713205668","heartbeat_back_addr":"172.21.15.120:6817/713205668","heartbeat_front_addr":"172.21.15.120:6815/713205668","state":["exists","up"]},{"osd":4,"uuid":"7f8bf3b9-0ec4-46f6-a406-d26d7971179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6809","nonce":2680865854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6811","nonce":2680865854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6815","nonce":2680865854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6813","nonce":2680865854}]},"public_addr":"172.21.15.153:6809/2680865854","cluster_addr":"172.21.15.153:6811/2680865854","heartbeat_back_addr":"172.21.15.153:6815/2680865854","heartbeat_front_addr":"172.21.15.153:6813/2680865854","state":["exists","up"]},{"osd":5,"uuid":"70b270b2-209d-4ff5-875a-04ee7b992a93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6809","nonce":3763331116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6811","nonce":3763331116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6815","nonce":3763331116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6813","nonce":3763331116}]},"public_addr":"172.21.15.196:6809/3763331116","cluster_addr":"172.21.15.196:6811/3763331116","heartbeat_back_addr":"172.21.15.196:6815/3763331116","heartbeat_front_addr":"172.21.15.196:6813/3763331116","state":["exists","up"]},{"osd":6,"uuid":"8087f69c-dc31-4522-a18d-4e6ab1b7cb27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6819","nonce":4171773765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6821","nonce":4171773765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6825","nonce":4171773765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6823","nonce":4171773765}]},"public_addr":"172.21.15.120:6819/4171773765","cluster_addr":"172.21.15.120:6821/4171773765","heartbeat_back_addr":"172.21.15.120:6825/4171773765","heartbeat_front_addr":"172.21.15.120:6823/4171773765","state":["exists","up"]},{"osd":7,"uuid":"c3b1ead3-150f-43fd-8924-f639203f4cdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6817","nonce":2535148453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6819","nonce":2535148453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6823","nonce":2535148453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6821","nonce":2535148453}]},"public_addr":"172.21.15.153:6817/2535148453","cluster_addr":"172.21.15.153:6819/2535148453","heartbeat_back_addr":"172.21.15.153:6823/2535148453","heartbeat_front_addr":"172.21.15.153:6821/2535148453","state":["exists","up"]},{"osd":8,"uuid":"a3273a2a-968e-4e93-a9cf-3879892bdeab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6817","nonce":186195906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6819","nonce":186195906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6823","nonce":186195906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6821","nonce":186195906}]},"public_addr":"172.21.15.196:6817/186195906","cluster_addr":"172.21.15.196:6819/186195906","heartbeat_back_addr":"172.21.15.196:6823/186195906","heartbeat_front_addr":"172.21.15.196:6821/186195906","state":["exists","up"]},{"osd":9,"uuid":"6d184513-dab8-45f7-bc08-fa2e8b2f10b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6827","nonce":1559320883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6829","nonce":1559320883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6833","nonce":1559320883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6831","nonce":1559320883}]},"public_addr":"172.21.15.120:6827/1559320883","cluster_addr":"172.21.15.120:6829/1559320883","heartbeat_back_addr":"172.21.15.120:6833/1559320883","heartbeat_front_addr":"172.21.15.120:6831/1559320883","state":["exists","up"]},{"osd":10,"uuid":"797db886-62f9-4316-a45b-4d50512e65d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6825","nonce":3216877560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6827","nonce":3216877560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6831","nonce":3216877560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6829","nonce":3216877560}]},"public_addr":"172.21.15.153:6825/3216877560","cluster_addr":"172.21.15.153:6827/3216877560","heartbeat_back_addr":"172.21.15.153:6831/3216877560","heartbeat_front_addr":"172.21.15.153:6829/3216877560","state":["exists","up"]},{"osd":11,"uuid":"17e0afaa-84eb-4b26-9810-bfa3cc7e0f3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6825","nonce":1857200740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6827","nonce":1857200740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6831","nonce":1857200740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6829","nonce":1857200740}]},"public_addr":"172.21.15.196:6825/1857200740","cluster_addr":"172.21.15.196:6827/1857200740","heartbeat_back_addr":"172.21.15.196:6831/1857200740","heartbeat_front_addr":"172.21.15.196:6829/1857200740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:55:15.595645+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:55:40.405487+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:04.938034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:29.718696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:55.636029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:57:21.257083+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:57:46.316552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:58:11.946863+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:58:37.098578+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:03.759023+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:28.659049+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:54.350475+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:0/3010783707":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6801/3562882246":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6800/3562882246":"2024-09-20T08:53:51.528772+0000","172.21.15.120:0/3224512035":"2024-09-20T08:53:51.528772+0000","172.21.15.120:0/3465105835":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/3317449610":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/2800697451":"2024-09-20T08:53:28.144376+0000","172.21.15.120:6801/2938446665":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/109127432":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6800/2938446665":"2024-09-20T08:53:28.144376+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:01:20.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:20.071+0000 7f1e2b49a640 1 -- 172.21.15.120:0/2459728401 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1e100a32d0 msgr2=0x7f1e100a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:20.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:20.071+0000 7f1e2b49a640 1 --2- 172.21.15.120:0/2459728401 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1e100a32d0 0x7f1e100a5790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1e20005fd0 tx=0x7f1e20005850 comp rx=0 tx=0).stop 2024-09-19T09:01:20.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:20.071+0000 7f1e2b49a640 1 -- 172.21.15.120:0/2459728401 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e24102790 msgr2=0x7f1e24198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:20.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:20.071+0000 7f1e2b49a640 1 --2- 172.21.15.120:0/2459728401 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e24102790 0x7f1e24198a90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400b4a0 tx=0x7f1e1400b960 comp rx=0 tx=0).stop 2024-09-19T09:01:20.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:20.071+0000 7f1e2b49a640 1 -- 172.21.15.120:0/2459728401 shutdown_connections 2024-09-19T09:01:20.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:20.071+0000 7f1e2b49a640 1 --2- 172.21.15.120:0/2459728401 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1e100a32d0 0x7f1e100a5790 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:20.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:20.071+0000 7f1e2b49a640 1 --2- 172.21.15.120:0/2459728401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e24106f10 0x7f1e2419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:20.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:20.071+0000 7f1e2b49a640 1 --2- 172.21.15.120:0/2459728401 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e24102790 0x7f1e24198a90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:20.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:20.071+0000 7f1e2b49a640 1 --2- 172.21.15.120:0/2459728401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e24101de0 0x7f1e24198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:20.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:20.071+0000 7f1e2b49a640 1 -- 172.21.15.120:0/2459728401 >> 172.21.15.120:0/2459728401 conn(0x7f1e240fd8c0 msgr2=0x7f1e240fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:20.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:20.071+0000 7f1e2b49a640 1 -- 172.21.15.120:0/2459728401 shutdown_connections 2024-09-19T09:01:20.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:20.071+0000 7f1e2b49a640 1 -- 172.21.15.120:0/2459728401 wait complete. 2024-09-19T09:01:20.703 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-19T09:01:20.703 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:01:20.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:20 smithi196 ceph-mon[35150]: pgmap v293: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-19T09:01:20.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:20 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2459728401' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:01:20.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:20 smithi153 ceph-mon[35071]: pgmap v293: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-19T09:01:20.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:20 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2459728401' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:01:20.950 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:21.070 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:20 smithi120 ceph-mon[31150]: pgmap v293: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-19T09:01:21.071 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:20 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2459728401' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:01:21.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.473+0000 7f31acfcc640 1 -- 172.21.15.120:0/2833397777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31a8103660 msgr2=0x7f31a810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:21.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.473+0000 7f31acfcc640 1 --2- 172.21.15.120:0/2833397777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31a8103660 0x7f31a810bb50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3198009900 tx=0x7f319802a480 comp rx=0 tx=0).stop 2024-09-19T09:01:21.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.473+0000 7f31acfcc640 1 -- 172.21.15.120:0/2833397777 shutdown_connections 2024-09-19T09:01:21.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.473+0000 7f31acfcc640 1 --2- 172.21.15.120:0/2833397777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31a810c170 0x7f31a810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:21.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.473+0000 7f31acfcc640 1 --2- 172.21.15.120:0/2833397777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31a8103660 0x7f31a810bb50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:21.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.473+0000 7f31acfcc640 1 --2- 172.21.15.120:0/2833397777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31a8102d40 0x7f31a8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:21.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.473+0000 7f31acfcc640 1 -- 172.21.15.120:0/2833397777 >> 172.21.15.120:0/2833397777 conn(0x7f31a80fc7a0 msgr2=0x7f31a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:21.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.474+0000 7f31acfcc640 1 -- 172.21.15.120:0/2833397777 shutdown_connections 2024-09-19T09:01:21.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.474+0000 7f31acfcc640 1 -- 172.21.15.120:0/2833397777 wait complete. 2024-09-19T09:01:21.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.474+0000 7f31acfcc640 1 Processor -- start 2024-09-19T09:01:21.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.474+0000 7f31acfcc640 1 -- start start 2024-09-19T09:01:21.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.474+0000 7f31acfcc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31a8102d40 0x7f31a81009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:21.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.474+0000 7f31acfcc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31a8103660 0x7f31a8100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:21.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.475+0000 7f31acfcc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31a810c170 0x7f31a8101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:21.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.475+0000 7f31acfcc640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f31a8111fb0 con 0x7f31a810c170 2024-09-19T09:01:21.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.475+0000 7f31acfcc640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f31a8111e30 con 0x7f31a8103660 2024-09-19T09:01:21.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.475+0000 7f31acfcc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f31a8112130 con 0x7f31a8102d40 2024-09-19T09:01:21.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.475+0000 7f31a6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31a810c170 0x7f31a8101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:21.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.475+0000 7f31a6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31a810c170 0x7f31a8101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55406/0 (socket says 172.21.15.120:55406) 2024-09-19T09:01:21.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.475+0000 7f31a6d9d640 1 -- 172.21.15.120:0/2521907351 learned_addr learned my addr 172.21.15.120:0/2521907351 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:21.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.475+0000 7f31a5d9b640 1 --2- 172.21.15.120:0/2521907351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31a8103660 0x7f31a8100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:21.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.475+0000 7f31a659c640 1 --2- 172.21.15.120:0/2521907351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31a8102d40 0x7f31a81009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:21.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.475+0000 7f31a6d9d640 1 -- 172.21.15.120:0/2521907351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31a8102d40 msgr2=0x7f31a81009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:21.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.475+0000 7f31a6d9d640 1 --2- 172.21.15.120:0/2521907351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31a8102d40 0x7f31a81009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:21.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.475+0000 7f31a6d9d640 1 -- 172.21.15.120:0/2521907351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31a8103660 msgr2=0x7f31a8100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:21.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.475+0000 7f31a6d9d640 1 --2- 172.21.15.120:0/2521907351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31a8103660 0x7f31a8100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:21.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.475+0000 7f31a6d9d640 1 -- 172.21.15.120:0/2521907351 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a81a6920 con 0x7f31a810c170 2024-09-19T09:01:21.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.475+0000 7f31a6d9d640 1 --2- 172.21.15.120:0/2521907351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31a810c170 0x7f31a8101420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f31a000cce0 tx=0x7f31a000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:21.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.475+0000 7f31977fe640 1 -- 172.21.15.120:0/2521907351 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a000f450 con 0x7f31a810c170 2024-09-19T09:01:21.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.475+0000 7f31acfcc640 1 -- 172.21.15.120:0/2521907351 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a81a6c10 con 0x7f31a810c170 2024-09-19T09:01:21.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.475+0000 7f31977fe640 1 -- 172.21.15.120:0/2521907351 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31a000fad0 con 0x7f31a810c170 2024-09-19T09:01:21.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.475+0000 7f31977fe640 1 -- 172.21.15.120:0/2521907351 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a0018c10 con 0x7f31a810c170 2024-09-19T09:01:21.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.475+0000 7f31acfcc640 1 -- 172.21.15.120:0/2521907351 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a81a7120 con 0x7f31a810c170 2024-09-19T09:01:21.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.476+0000 7f31acfcc640 1 -- 172.21.15.120:0/2521907351 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31a8103120 con 0x7f31a810c170 2024-09-19T09:01:21.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.477+0000 7f31977fe640 1 -- 172.21.15.120:0/2521907351 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f31a000f5f0 con 0x7f31a810c170 2024-09-19T09:01:21.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.478+0000 7f31977fe640 1 --2- 172.21.15.120:0/2521907351 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f31900a32d0 0x7f31900a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:21.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.478+0000 7f31a659c640 1 --2- 172.21.15.120:0/2521907351 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f31900a32d0 0x7f31900a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:21.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.478+0000 7f31977fe640 1 -- 172.21.15.120:0/2521907351 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f31a00d0060 con 0x7f31a810c170 2024-09-19T09:01:21.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.479+0000 7f31a659c640 1 --2- 172.21.15.120:0/2521907351 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f31900a32d0 0x7f31900a5790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f319c009890 tx=0x7f319c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:21.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.483+0000 7f31977fe640 1 -- 172.21.15.120:0/2521907351 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f31a0014030 con 0x7f31a810c170 2024-09-19T09:01:21.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.649+0000 7f31acfcc640 1 -- 172.21.15.120:0/2521907351 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f31a81071e0 con 0x7f31a810c170 2024-09-19T09:01:21.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.650+0000 7f31977fe640 1 -- 172.21.15.120:0/2521907351 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22362 (secure 0 0 0) 0x7f31a009cd50 con 0x7f31a810c170 2024-09-19T09:01:21.650 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:01:21.650 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":79,"fsid":"507393d2-7664-11ef-bceb-c7b262605968","created":"2024-09-19T08:53:06.386526+0000","modified":"2024-09-19T09:00:47.659465+0000","last_up_change":"2024-09-19T08:59:55.692659+0000","last_in_change":"2024-09-19T08:59:32.808860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:56:07.830047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:00:43.774339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:00:45.786262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33ec3993-576c-428c-bf75-adcb17cd72a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6803","nonce":4131180019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6805","nonce":4131180019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6809","nonce":4131180019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6807","nonce":4131180019}]},"public_addr":"172.21.15.120:6803/4131180019","cluster_addr":"172.21.15.120:6805/4131180019","heartbeat_back_addr":"172.21.15.120:6809/4131180019","heartbeat_front_addr":"172.21.15.120:6807/4131180019","state":["exists","up"]},{"osd":1,"uuid":"450bbebf-2aac-47c6-8474-bbd37fd0a1a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6801","nonce":2259730294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6803","nonce":2259730294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6807","nonce":2259730294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6805","nonce":2259730294}]},"public_addr":"172.21.15.153:6801/2259730294","cluster_addr":"172.21.15.153:6803/2259730294","heartbeat_back_addr":"172.21.15.153:6807/2259730294","heartbeat_front_addr":"172.21.15.153:6805/2259730294","state":["exists","up"]},{"osd":2,"uuid":"ac8c4b83-8e8e-4803-b266-262f7873bd85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6801","nonce":2942286570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6803","nonce":2942286570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6807","nonce":2942286570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6805","nonce":2942286570}]},"public_addr":"172.21.15.196:6801/2942286570","cluster_addr":"172.21.15.196:6803/2942286570","heartbeat_back_addr":"172.21.15.196:6807/2942286570","heartbeat_front_addr":"172.21.15.196:6805/2942286570","state":["exists","up"]},{"osd":3,"uuid":"c1d9e507-8f02-403e-87da-9b4966fd8fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6811","nonce":713205668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6813","nonce":713205668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6817","nonce":713205668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6815","nonce":713205668}]},"public_addr":"172.21.15.120:6811/713205668","cluster_addr":"172.21.15.120:6813/713205668","heartbeat_back_addr":"172.21.15.120:6817/713205668","heartbeat_front_addr":"172.21.15.120:6815/713205668","state":["exists","up"]},{"osd":4,"uuid":"7f8bf3b9-0ec4-46f6-a406-d26d7971179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6809","nonce":2680865854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6811","nonce":2680865854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6815","nonce":2680865854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6813","nonce":2680865854}]},"public_addr":"172.21.15.153:6809/2680865854","cluster_addr":"172.21.15.153:6811/2680865854","heartbeat_back_addr":"172.21.15.153:6815/2680865854","heartbeat_front_addr":"172.21.15.153:6813/2680865854","state":["exists","up"]},{"osd":5,"uuid":"70b270b2-209d-4ff5-875a-04ee7b992a93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6809","nonce":3763331116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6811","nonce":3763331116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6815","nonce":3763331116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6813","nonce":3763331116}]},"public_addr":"172.21.15.196:6809/3763331116","cluster_addr":"172.21.15.196:6811/3763331116","heartbeat_back_addr":"172.21.15.196:6815/3763331116","heartbeat_front_addr":"172.21.15.196:6813/3763331116","state":["exists","up"]},{"osd":6,"uuid":"8087f69c-dc31-4522-a18d-4e6ab1b7cb27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6819","nonce":4171773765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6821","nonce":4171773765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6825","nonce":4171773765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6823","nonce":4171773765}]},"public_addr":"172.21.15.120:6819/4171773765","cluster_addr":"172.21.15.120:6821/4171773765","heartbeat_back_addr":"172.21.15.120:6825/4171773765","heartbeat_front_addr":"172.21.15.120:6823/4171773765","state":["exists","up"]},{"osd":7,"uuid":"c3b1ead3-150f-43fd-8924-f639203f4cdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6817","nonce":2535148453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6819","nonce":2535148453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6823","nonce":2535148453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6821","nonce":2535148453}]},"public_addr":"172.21.15.153:6817/2535148453","cluster_addr":"172.21.15.153:6819/2535148453","heartbeat_back_addr":"172.21.15.153:6823/2535148453","heartbeat_front_addr":"172.21.15.153:6821/2535148453","state":["exists","up"]},{"osd":8,"uuid":"a3273a2a-968e-4e93-a9cf-3879892bdeab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6817","nonce":186195906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6819","nonce":186195906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6823","nonce":186195906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6821","nonce":186195906}]},"public_addr":"172.21.15.196:6817/186195906","cluster_addr":"172.21.15.196:6819/186195906","heartbeat_back_addr":"172.21.15.196:6823/186195906","heartbeat_front_addr":"172.21.15.196:6821/186195906","state":["exists","up"]},{"osd":9,"uuid":"6d184513-dab8-45f7-bc08-fa2e8b2f10b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6827","nonce":1559320883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6829","nonce":1559320883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6833","nonce":1559320883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6831","nonce":1559320883}]},"public_addr":"172.21.15.120:6827/1559320883","cluster_addr":"172.21.15.120:6829/1559320883","heartbeat_back_addr":"172.21.15.120:6833/1559320883","heartbeat_front_addr":"172.21.15.120:6831/1559320883","state":["exists","up"]},{"osd":10,"uuid":"797db886-62f9-4316-a45b-4d50512e65d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6825","nonce":3216877560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6827","nonce":3216877560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6831","nonce":3216877560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6829","nonce":3216877560}]},"public_addr":"172.21.15.153:6825/3216877560","cluster_addr":"172.21.15.153:6827/3216877560","heartbeat_back_addr":"172.21.15.153:6831/3216877560","heartbeat_front_addr":"172.21.15.153:6829/3216877560","state":["exists","up"]},{"osd":11,"uuid":"17e0afaa-84eb-4b26-9810-bfa3cc7e0f3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6825","nonce":1857200740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6827","nonce":1857200740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6831","nonce":1857200740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6829","nonce":1857200740}]},"public_addr":"172.21.15.196:6825/1857200740","cluster_addr":"172.21.15.196:6827/1857200740","heartbeat_back_addr":"172.21.15.196:6831/1857200740","heartbeat_front_addr":"172.21.15.196:6829/1857200740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:55:15.595645+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:55:40.405487+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:04.938034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:29.718696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:55.636029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:57:21.257083+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:57:46.316552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:58:11.946863+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:58:37.098578+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:03.759023+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:28.659049+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:54.350475+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:0/3010783707":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6801/3562882246":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6800/3562882246":"2024-09-20T08:53:51.528772+0000","172.21.15.120:0/3224512035":"2024-09-20T08:53:51.528772+0000","172.21.15.120:0/3465105835":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/3317449610":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/2800697451":"2024-09-20T08:53:28.144376+0000","172.21.15.120:6801/2938446665":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/109127432":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6800/2938446665":"2024-09-20T08:53:28.144376+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:01:21.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.652+0000 7f31acfcc640 1 -- 172.21.15.120:0/2521907351 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f31900a32d0 msgr2=0x7f31900a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:21.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.652+0000 7f31acfcc640 1 --2- 172.21.15.120:0/2521907351 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f31900a32d0 0x7f31900a5790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f319c009890 tx=0x7f319c008040 comp rx=0 tx=0).stop 2024-09-19T09:01:21.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.652+0000 7f31acfcc640 1 -- 172.21.15.120:0/2521907351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31a810c170 msgr2=0x7f31a8101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:21.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.652+0000 7f31acfcc640 1 --2- 172.21.15.120:0/2521907351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31a810c170 0x7f31a8101420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f31a000cce0 tx=0x7f31a000a620 comp rx=0 tx=0).stop 2024-09-19T09:01:21.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.652+0000 7f31acfcc640 1 -- 172.21.15.120:0/2521907351 shutdown_connections 2024-09-19T09:01:21.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.652+0000 7f31acfcc640 1 --2- 172.21.15.120:0/2521907351 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f31900a32d0 0x7f31900a5790 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:21.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.652+0000 7f31acfcc640 1 --2- 172.21.15.120:0/2521907351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31a810c170 0x7f31a8101420 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:21.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.652+0000 7f31acfcc640 1 --2- 172.21.15.120:0/2521907351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31a8103660 0x7f31a8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:21.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.652+0000 7f31acfcc640 1 --2- 172.21.15.120:0/2521907351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31a8102d40 0x7f31a81009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:21.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.652+0000 7f31acfcc640 1 -- 172.21.15.120:0/2521907351 >> 172.21.15.120:0/2521907351 conn(0x7f31a80fc7a0 msgr2=0x7f31a80fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:21.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.652+0000 7f31acfcc640 1 -- 172.21.15.120:0/2521907351 shutdown_connections 2024-09-19T09:01:21.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:21.652+0000 7f31acfcc640 1 -- 172.21.15.120:0/2521907351 wait complete. 2024-09-19T09:01:21.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:21 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2521907351' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:01:21.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:21 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2521907351' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:01:22.008 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:21 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2521907351' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:01:22.443 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-19T09:01:22.443 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-19T09:01:22.444 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-19T09:01:22.444 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-19T09:01:22.445 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-19T09:01:22.445 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-19T09:01:22.445 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-19T09:01:22.446 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-19T09:01:22.447 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-19T09:01:22.447 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-19T09:01:22.448 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-19T09:01:22.448 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-19T09:01:22.935 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:22 smithi196 ceph-mon[35150]: pgmap v294: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.5 KiB/s wr, 12 op/s 2024-09-19T09:01:22.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:22 smithi153 ceph-mon[35071]: pgmap v294: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.5 KiB/s wr, 12 op/s 2024-09-19T09:01:23.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:22 smithi120 ceph-mon[31150]: pgmap v294: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.5 KiB/s wr, 12 op/s 2024-09-19T09:01:23.138 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:23.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.680+0000 7f1a1b3b2640 1 -- 172.21.15.120:0/3004635152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1a14100e20 msgr2=0x7f1a14101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:23.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.680+0000 7f1a1b3b2640 1 --2- 172.21.15.120:0/3004635152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1a14100e20 0x7f1a14101200 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c009900 tx=0x7f1a0c02a480 comp rx=0 tx=0).stop 2024-09-19T09:01:23.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.680+0000 7f1a1b3b2640 1 -- 172.21.15.120:0/3004635152 shutdown_connections 2024-09-19T09:01:23.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.680+0000 7f1a1b3b2640 1 --2- 172.21.15.120:0/3004635152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a14102ef0 0x7f1a1410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:23.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.680+0000 7f1a1b3b2640 1 --2- 172.21.15.120:0/3004635152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a14102570 0x7f1a141029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:23.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.680+0000 7f1a1b3b2640 1 --2- 172.21.15.120:0/3004635152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1a14100e20 0x7f1a14101200 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:23.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.680+0000 7f1a1b3b2640 1 -- 172.21.15.120:0/3004635152 >> 172.21.15.120:0/3004635152 conn(0x7f1a140fc7d0 msgr2=0x7f1a140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:23.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.680+0000 7f1a1b3b2640 1 -- 172.21.15.120:0/3004635152 shutdown_connections 2024-09-19T09:01:23.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.680+0000 7f1a1b3b2640 1 -- 172.21.15.120:0/3004635152 wait complete. 2024-09-19T09:01:23.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.681+0000 7f1a1b3b2640 1 Processor -- start 2024-09-19T09:01:23.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.681+0000 7f1a1b3b2640 1 -- start start 2024-09-19T09:01:23.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.681+0000 7f1a1b3b2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a14100e20 0x7f1a14199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:23.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.681+0000 7f1a1b3b2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a14102570 0x7f1a14199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:23.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.681+0000 7f1a1b3b2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1a14102ef0 0x7f1a141a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:23.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.681+0000 7f1a1b3b2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1a14110fa0 con 0x7f1a14102ef0 2024-09-19T09:01:23.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.681+0000 7f1a1b3b2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1a14110e20 con 0x7f1a14100e20 2024-09-19T09:01:23.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.681+0000 7f1a1b3b2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1a14111120 con 0x7f1a14102570 2024-09-19T09:01:23.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.681+0000 7f1a1994f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1a14102ef0 0x7f1a141a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:23.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.681+0000 7f1a1994f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1a14102ef0 0x7f1a141a44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55432/0 (socket says 172.21.15.120:55432) 2024-09-19T09:01:23.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.681+0000 7f1a1994f640 1 -- 172.21.15.120:0/796769249 learned_addr learned my addr 172.21.15.120:0/796769249 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:23.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.681+0000 7f1a1914e640 1 --2- 172.21.15.120:0/796769249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a14100e20 0x7f1a14199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:23.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.681+0000 7f1a1894d640 1 --2- 172.21.15.120:0/796769249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a14102570 0x7f1a14199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:23.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.682+0000 7f1a1994f640 1 -- 172.21.15.120:0/796769249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a14102570 msgr2=0x7f1a14199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:23.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.682+0000 7f1a1994f640 1 --2- 172.21.15.120:0/796769249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a14102570 0x7f1a14199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:23.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.682+0000 7f1a1994f640 1 -- 172.21.15.120:0/796769249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a14100e20 msgr2=0x7f1a14199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:23.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.682+0000 7f1a1994f640 1 --2- 172.21.15.120:0/796769249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a14100e20 0x7f1a14199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:23.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.682+0000 7f1a1994f640 1 -- 172.21.15.120:0/796769249 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a141a49f0 con 0x7f1a14102ef0 2024-09-19T09:01:23.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.682+0000 7f1a1994f640 1 --2- 172.21.15.120:0/796769249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1a14102ef0 0x7f1a141a44b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400cad0 tx=0x7f1a0400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:23.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.682+0000 7f1a127fc640 1 -- 172.21.15.120:0/796769249 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a04004d20 con 0x7f1a14102ef0 2024-09-19T09:01:23.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.682+0000 7f1a127fc640 1 -- 172.21.15.120:0/796769249 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a0400ad10 con 0x7f1a14102ef0 2024-09-19T09:01:23.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.683+0000 7f1a1b3b2640 1 -- 172.21.15.120:0/796769249 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a14197220 con 0x7f1a14102ef0 2024-09-19T09:01:23.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.683+0000 7f1a127fc640 1 -- 172.21.15.120:0/796769249 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a04005880 con 0x7f1a14102ef0 2024-09-19T09:01:23.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.683+0000 7f1a1b3b2640 1 -- 172.21.15.120:0/796769249 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a14197760 con 0x7f1a14102ef0 2024-09-19T09:01:23.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.683+0000 7f1a1b3b2640 1 -- 172.21.15.120:0/796769249 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1a14062900 con 0x7f1a14102ef0 2024-09-19T09:01:23.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.685+0000 7f1a127fc640 1 -- 172.21.15.120:0/796769249 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f1a0400a830 con 0x7f1a14102ef0 2024-09-19T09:01:23.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.685+0000 7f1a127fc640 1 --2- 172.21.15.120:0/796769249 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f19fc0a3350 0x7f19fc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:23.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.685+0000 7f1a1914e640 1 --2- 172.21.15.120:0/796769249 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f19fc0a3350 0x7f19fc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:23.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.685+0000 7f1a127fc640 1 -- 172.21.15.120:0/796769249 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f1a040d86a0 con 0x7f1a14102ef0 2024-09-19T09:01:23.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.686+0000 7f1a127fc640 1 --2- 172.21.15.120:0/796769249 >> [v2:172.21.15.120:6818/4171773765,v1:172.21.15.120:6819/4171773765] conn(0x7f19fc0acba0 0x7f19fc0aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:23.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.686+0000 7f1a127fc640 1 -- 172.21.15.120:0/796769249 --> [v2:172.21.15.120:6818/4171773765,v1:172.21.15.120:6819/4171773765] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1a04005a20 con 0x7f19fc0acba0 2024-09-19T09:01:23.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.686+0000 7f1a127fc640 1 -- 172.21.15.120:0/796769249 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f1a040d8b80 con 0x7f1a14102ef0 2024-09-19T09:01:23.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.686+0000 7f1a1894d640 1 --2- 172.21.15.120:0/796769249 >> [v2:172.21.15.120:6818/4171773765,v1:172.21.15.120:6819/4171773765] conn(0x7f19fc0acba0 0x7f19fc0aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:23.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.686+0000 7f1a1914e640 1 --2- 172.21.15.120:0/796769249 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f19fc0a3350 0x7f19fc0a5810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c000c00 tx=0x7f1a0c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:23.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.686+0000 7f1a1894d640 1 --2- 172.21.15.120:0/796769249 >> [v2:172.21.15.120:6818/4171773765,v1:172.21.15.120:6819/4171773765] conn(0x7f19fc0acba0 0x7f19fc0aefc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:23.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.687+0000 7f1a127fc640 1 -- 172.21.15.120:0/796769249 <== osd.6 v2:172.21.15.120:6818/4171773765 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1a04005a20 con 0x7f19fc0acba0 2024-09-19T09:01:23.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.700+0000 7f1a1b3b2640 1 -- 172.21.15.120:0/796769249 --> [v2:172.21.15.120:6818/4171773765,v1:172.21.15.120:6819/4171773765] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1a14062d00 con 0x7f19fc0acba0 2024-09-19T09:01:23.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.700+0000 7f1a127fc640 1 -- 172.21.15.120:0/796769249 <== osd.6 v2:172.21.15.120:6818/4171773765 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1a14062d00 con 0x7f19fc0acba0 2024-09-19T09:01:23.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.700+0000 7f1a1b3b2640 1 -- 172.21.15.120:0/796769249 >> [v2:172.21.15.120:6818/4171773765,v1:172.21.15.120:6819/4171773765] conn(0x7f19fc0acba0 msgr2=0x7f19fc0aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:23.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.700+0000 7f1a1b3b2640 1 --2- 172.21.15.120:0/796769249 >> [v2:172.21.15.120:6818/4171773765,v1:172.21.15.120:6819/4171773765] conn(0x7f19fc0acba0 0x7f19fc0aefc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:23.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.701+0000 7f1a1b3b2640 1 -- 172.21.15.120:0/796769249 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f19fc0a3350 msgr2=0x7f19fc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:23.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.701+0000 7f1a1b3b2640 1 --2- 172.21.15.120:0/796769249 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f19fc0a3350 0x7f19fc0a5810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c000c00 tx=0x7f1a0c0058d0 comp rx=0 tx=0).stop 2024-09-19T09:01:23.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.701+0000 7f1a1b3b2640 1 -- 172.21.15.120:0/796769249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1a14102ef0 msgr2=0x7f1a141a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:23.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.701+0000 7f1a1b3b2640 1 --2- 172.21.15.120:0/796769249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1a14102ef0 0x7f1a141a44b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400cad0 tx=0x7f1a0400ced0 comp rx=0 tx=0).stop 2024-09-19T09:01:23.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.701+0000 7f1a1994f640 1 -- 172.21.15.120:0/796769249 reap_dead start 2024-09-19T09:01:23.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.701+0000 7f1a1b3b2640 1 -- 172.21.15.120:0/796769249 shutdown_connections 2024-09-19T09:01:23.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.701+0000 7f1a1b3b2640 1 -- 172.21.15.120:0/796769249 >> 172.21.15.120:0/796769249 conn(0x7f1a140fc7d0 msgr2=0x7f1a140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:23.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.701+0000 7f1a1b3b2640 1 -- 172.21.15.120:0/796769249 shutdown_connections 2024-09-19T09:01:23.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:23.701+0000 7f1a1b3b2640 1 -- 172.21.15.120:0/796769249 wait complete. 2024-09-19T09:01:24.474 INFO:teuthology.orchestra.run.smithi120.stdout:184683593775 2024-09-19T09:01:24.475 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-19T09:01:24.557 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:24.557 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:24.558 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:24.558 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:24.558 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:24.563 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:24.563 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:24.563 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:24.566 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:24.567 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:24.572 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:24.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:24 smithi196 ceph-mon[35150]: pgmap v295: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-19T09:01:24.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:24 smithi153 ceph-mon[35071]: pgmap v295: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-19T09:01:25.004 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:24 smithi120 ceph-mon[31150]: pgmap v295: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-19T09:01:25.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.465+0000 7f281d643640 1 -- 172.21.15.120:0/3307337194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2818106a80 msgr2=0x7f2818106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:25.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.465+0000 7f281d643640 1 --2- 172.21.15.120:0/3307337194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2818106a80 0x7f2818106e60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2808009930 tx=0x7f280802a420 comp rx=0 tx=0).stop 2024-09-19T09:01:25.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.465+0000 7f281d643640 1 -- 172.21.15.120:0/3307337194 shutdown_connections 2024-09-19T09:01:25.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.465+0000 7f281d643640 1 --2- 172.21.15.120:0/3307337194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2818101420 0x7f281810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:25.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.465+0000 7f281d643640 1 --2- 172.21.15.120:0/3307337194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2818100a80 0x7f2818100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:25.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.465+0000 7f281d643640 1 --2- 172.21.15.120:0/3307337194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2818106a80 0x7f2818106e60 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:25.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.465+0000 7f281d643640 1 -- 172.21.15.120:0/3307337194 >> 172.21.15.120:0/3307337194 conn(0x7f28180fc7a0 msgr2=0x7f28180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:25.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.465+0000 7f281d643640 1 -- 172.21.15.120:0/3307337194 shutdown_connections 2024-09-19T09:01:25.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.465+0000 7f281d643640 1 -- 172.21.15.120:0/3307337194 wait complete. 2024-09-19T09:01:25.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.466+0000 7f281d643640 1 Processor -- start 2024-09-19T09:01:25.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.466+0000 7f281d643640 1 -- start start 2024-09-19T09:01:25.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.466+0000 7f281d643640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2818100a80 0x7f2818197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:25.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.466+0000 7f281d643640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2818101420 0x7f2818197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:25.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.466+0000 7f281d643640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2818106a80 0x7f281819bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:25.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.466+0000 7f281d643640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2818110f20 con 0x7f2818100a80 2024-09-19T09:01:25.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.466+0000 7f281d643640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2818110da0 con 0x7f2818106a80 2024-09-19T09:01:25.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.466+0000 7f281d643640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f28181110a0 con 0x7f2818101420 2024-09-19T09:01:25.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.466+0000 7f2816ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2818100a80 0x7f2818197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:25.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.466+0000 7f2816ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2818100a80 0x7f2818197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55478/0 (socket says 172.21.15.120:55478) 2024-09-19T09:01:25.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.466+0000 7f2816ffd640 1 -- 172.21.15.120:0/2126657277 learned_addr learned my addr 172.21.15.120:0/2126657277 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:25.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.466+0000 7f28177fe640 1 --2- 172.21.15.120:0/2126657277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2818106a80 0x7f281819bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:25.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.466+0000 7f2816ffd640 1 -- 172.21.15.120:0/2126657277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2818101420 msgr2=0x7f2818197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:25.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.466+0000 7f2816ffd640 1 --2- 172.21.15.120:0/2126657277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2818101420 0x7f2818197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:25.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.466+0000 7f2816ffd640 1 -- 172.21.15.120:0/2126657277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2818106a80 msgr2=0x7f281819bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:25.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.466+0000 7f2816ffd640 1 --2- 172.21.15.120:0/2126657277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2818106a80 0x7f281819bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:25.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.466+0000 7f2816ffd640 1 -- 172.21.15.120:0/2126657277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281819c4d0 con 0x7f2818100a80 2024-09-19T09:01:25.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.467+0000 7f2816ffd640 1 --2- 172.21.15.120:0/2126657277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2818100a80 0x7f2818197520 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2808009a50 tx=0x7f2808004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:25.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.467+0000 7f2814ff9640 1 -- 172.21.15.120:0/2126657277 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280803f030 con 0x7f2818100a80 2024-09-19T09:01:25.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.467+0000 7f281d643640 1 -- 172.21.15.120:0/2126657277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f281819c760 con 0x7f2818100a80 2024-09-19T09:01:25.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.467+0000 7f2814ff9640 1 -- 172.21.15.120:0/2126657277 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2808003ce0 con 0x7f2818100a80 2024-09-19T09:01:25.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.467+0000 7f2814ff9640 1 -- 172.21.15.120:0/2126657277 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2808035070 con 0x7f2818100a80 2024-09-19T09:01:25.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.467+0000 7f281d643640 1 -- 172.21.15.120:0/2126657277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28181a40a0 con 0x7f2818100a80 2024-09-19T09:01:25.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.468+0000 7f281d643640 1 -- 172.21.15.120:0/2126657277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2818062980 con 0x7f2818100a80 2024-09-19T09:01:25.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.469+0000 7f2814ff9640 1 -- 172.21.15.120:0/2126657277 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f2808042030 con 0x7f2818100a80 2024-09-19T09:01:25.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.470+0000 7f2814ff9640 1 --2- 172.21.15.120:0/2126657277 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f28040a3350 0x7f28040a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:25.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.470+0000 7f280ffff640 1 --2- 172.21.15.120:0/2126657277 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f28040a3350 0x7f28040a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:25.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.470+0000 7f2814ff9640 1 -- 172.21.15.120:0/2126657277 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f28080f0350 con 0x7f2818100a80 2024-09-19T09:01:25.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.470+0000 7f2814ff9640 1 --2- 172.21.15.120:0/2126657277 >> [v2:172.21.15.120:6826/1559320883,v1:172.21.15.120:6827/1559320883] conn(0x7f28040acba0 0x7f28040aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:25.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.470+0000 7f2814ff9640 1 -- 172.21.15.120:0/2126657277 --> [v2:172.21.15.120:6826/1559320883,v1:172.21.15.120:6827/1559320883] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2808003b30 con 0x7f28040acba0 2024-09-19T09:01:25.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.470+0000 7f2814ff9640 1 -- 172.21.15.120:0/2126657277 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f28080f0830 con 0x7f2818100a80 2024-09-19T09:01:25.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.470+0000 7f28177fe640 1 --2- 172.21.15.120:0/2126657277 >> [v2:172.21.15.120:6826/1559320883,v1:172.21.15.120:6827/1559320883] conn(0x7f28040acba0 0x7f28040aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:25.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.470+0000 7f280ffff640 1 --2- 172.21.15.120:0/2126657277 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f28040a3350 0x7f28040a5810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f28180fee70 tx=0x7f2800006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:25.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.470+0000 7f28177fe640 1 --2- 172.21.15.120:0/2126657277 >> [v2:172.21.15.120:6826/1559320883,v1:172.21.15.120:6827/1559320883] conn(0x7f28040acba0 0x7f28040aefc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:25.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.471+0000 7f2814ff9640 1 -- 172.21.15.120:0/2126657277 <== osd.9 v2:172.21.15.120:6826/1559320883 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2808003b30 con 0x7f28040acba0 2024-09-19T09:01:25.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.484+0000 7f281d643640 1 -- 172.21.15.120:0/2126657277 --> [v2:172.21.15.120:6826/1559320883,v1:172.21.15.120:6827/1559320883] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2818062d80 con 0x7f28040acba0 2024-09-19T09:01:25.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.484+0000 7f2814ff9640 1 -- 172.21.15.120:0/2126657277 <== osd.9 v2:172.21.15.120:6826/1559320883 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2818062d80 con 0x7f28040acba0 2024-09-19T09:01:25.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.485+0000 7f281d643640 1 -- 172.21.15.120:0/2126657277 >> [v2:172.21.15.120:6826/1559320883,v1:172.21.15.120:6827/1559320883] conn(0x7f28040acba0 msgr2=0x7f28040aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:25.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.485+0000 7f281d643640 1 --2- 172.21.15.120:0/2126657277 >> [v2:172.21.15.120:6826/1559320883,v1:172.21.15.120:6827/1559320883] conn(0x7f28040acba0 0x7f28040aefc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:25.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.485+0000 7f281d643640 1 -- 172.21.15.120:0/2126657277 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f28040a3350 msgr2=0x7f28040a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:25.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.485+0000 7f281d643640 1 --2- 172.21.15.120:0/2126657277 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f28040a3350 0x7f28040a5810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f28180fee70 tx=0x7f2800006c90 comp rx=0 tx=0).stop 2024-09-19T09:01:25.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.485+0000 7f281d643640 1 -- 172.21.15.120:0/2126657277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2818100a80 msgr2=0x7f2818197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:25.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.485+0000 7f281d643640 1 --2- 172.21.15.120:0/2126657277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2818100a80 0x7f2818197520 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2808009a50 tx=0x7f2808004270 comp rx=0 tx=0).stop 2024-09-19T09:01:25.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.485+0000 7f28177fe640 1 -- 172.21.15.120:0/2126657277 reap_dead start 2024-09-19T09:01:25.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.485+0000 7f281d643640 1 -- 172.21.15.120:0/2126657277 shutdown_connections 2024-09-19T09:01:25.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.485+0000 7f281d643640 1 -- 172.21.15.120:0/2126657277 >> 172.21.15.120:0/2126657277 conn(0x7f28180fc7a0 msgr2=0x7f28180fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:25.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.485+0000 7f281d643640 1 -- 172.21.15.120:0/2126657277 shutdown_connections 2024-09-19T09:01:25.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.485+0000 7f281d643640 1 -- 172.21.15.120:0/2126657277 wait complete. 2024-09-19T09:01:26.233 INFO:teuthology.orchestra.run.smithi120.stdout:266287972383 2024-09-19T09:01:26.234 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-19T09:01:26.758 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:26 smithi120 ceph-mon[31150]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-19T09:01:26.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:26 smithi196 ceph-mon[35150]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-19T09:01:26.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:26 smithi153 ceph-mon[35071]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-19T09:01:27.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.338+0000 7ff9f0f1e640 1 -- 172.21.15.120:0/2696983027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9ec100e20 msgr2=0x7ff9ec101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:27.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.338+0000 7ff9f0f1e640 1 --2- 172.21.15.120:0/2696983027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9ec100e20 0x7ff9ec101200 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc009900 tx=0x7ff9dc02a480 comp rx=0 tx=0).stop 2024-09-19T09:01:27.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.338+0000 7ff9f0f1e640 1 -- 172.21.15.120:0/2696983027 shutdown_connections 2024-09-19T09:01:27.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.338+0000 7ff9f0f1e640 1 --2- 172.21.15.120:0/2696983027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9ec102ef0 0x7ff9ec10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:27.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.338+0000 7ff9f0f1e640 1 --2- 172.21.15.120:0/2696983027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9ec102570 0x7ff9ec1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:27.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.338+0000 7ff9f0f1e640 1 --2- 172.21.15.120:0/2696983027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9ec100e20 0x7ff9ec101200 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:27.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.338+0000 7ff9f0f1e640 1 -- 172.21.15.120:0/2696983027 >> 172.21.15.120:0/2696983027 conn(0x7ff9ec0fc7d0 msgr2=0x7ff9ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:27.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.338+0000 7ff9f0f1e640 1 -- 172.21.15.120:0/2696983027 shutdown_connections 2024-09-19T09:01:27.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.338+0000 7ff9f0f1e640 1 -- 172.21.15.120:0/2696983027 wait complete. 2024-09-19T09:01:27.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.339+0000 7ff9f0f1e640 1 Processor -- start 2024-09-19T09:01:27.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.339+0000 7ff9f0f1e640 1 -- start start 2024-09-19T09:01:27.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.339+0000 7ff9f0f1e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9ec100e20 0x7ff9ec199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:27.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.339+0000 7ff9f0f1e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9ec102570 0x7ff9ec199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:27.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.339+0000 7ff9f0f1e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9ec102ef0 0x7ff9ec1a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:27.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.339+0000 7ff9f0f1e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec110f80 con 0x7ff9ec102570 2024-09-19T09:01:27.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.339+0000 7ff9f0f1e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec110e00 con 0x7ff9ec100e20 2024-09-19T09:01:27.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.339+0000 7ff9f0f1e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec111100 con 0x7ff9ec102ef0 2024-09-19T09:01:27.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.340+0000 7ff9e3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9ec102570 0x7ff9ec199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:27.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.340+0000 7ff9e3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9ec102570 0x7ff9ec199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55456/0 (socket says 172.21.15.120:55456) 2024-09-19T09:01:27.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.340+0000 7ff9e3fff640 1 -- 172.21.15.120:0/1630396343 learned_addr learned my addr 172.21.15.120:0/1630396343 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:27.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.340+0000 7ff9ead9d640 1 --2- 172.21.15.120:0/1630396343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9ec102ef0 0x7ff9ec1a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:27.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.340+0000 7ff9ea59c640 1 --2- 172.21.15.120:0/1630396343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9ec100e20 0x7ff9ec199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:27.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.340+0000 7ff9e3fff640 1 -- 172.21.15.120:0/1630396343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9ec102ef0 msgr2=0x7ff9ec1a43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:27.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.340+0000 7ff9e3fff640 1 --2- 172.21.15.120:0/1630396343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9ec102ef0 0x7ff9ec1a43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:27.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.340+0000 7ff9e3fff640 1 -- 172.21.15.120:0/1630396343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9ec100e20 msgr2=0x7ff9ec199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:27.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.340+0000 7ff9e3fff640 1 --2- 172.21.15.120:0/1630396343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9ec100e20 0x7ff9ec199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:27.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.340+0000 7ff9e3fff640 1 -- 172.21.15.120:0/1630396343 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9ec1a4910 con 0x7ff9ec102570 2024-09-19T09:01:27.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.340+0000 7ff9e3fff640 1 --2- 172.21.15.120:0/1630396343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9ec102570 0x7ff9ec199890 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400c890 tx=0x7ff9d400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:27.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.340+0000 7ff9e37fe640 1 -- 172.21.15.120:0/1630396343 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4004220 con 0x7ff9ec102570 2024-09-19T09:01:27.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.340+0000 7ff9e37fe640 1 -- 172.21.15.120:0/1630396343 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9d40043c0 con 0x7ff9ec102570 2024-09-19T09:01:27.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.340+0000 7ff9f0f1e640 1 -- 172.21.15.120:0/1630396343 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9ec1a4ae0 con 0x7ff9ec102570 2024-09-19T09:01:27.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.340+0000 7ff9e37fe640 1 -- 172.21.15.120:0/1630396343 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40049e0 con 0x7ff9ec102570 2024-09-19T09:01:27.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.340+0000 7ff9f0f1e640 1 -- 172.21.15.120:0/1630396343 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9ec1a5020 con 0x7ff9ec102570 2024-09-19T09:01:27.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.341+0000 7ff9f0f1e640 1 -- 172.21.15.120:0/1630396343 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff9ec062940 con 0x7ff9ec102570 2024-09-19T09:01:27.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.343+0000 7ff9e37fe640 1 -- 172.21.15.120:0/1630396343 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ff9d40079b0 con 0x7ff9ec102570 2024-09-19T09:01:27.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.343+0000 7ff9e37fe640 1 --2- 172.21.15.120:0/1630396343 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff9d80a3350 0x7ff9d80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:27.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.343+0000 7ff9ea59c640 1 --2- 172.21.15.120:0/1630396343 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff9d80a3350 0x7ff9d80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:27.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.343+0000 7ff9e37fe640 1 -- 172.21.15.120:0/1630396343 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7ff9d40cf9a0 con 0x7ff9ec102570 2024-09-19T09:01:27.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.343+0000 7ff9e37fe640 1 --2- 172.21.15.120:0/1630396343 >> [v2:172.21.15.153:6816/2535148453,v1:172.21.15.153:6817/2535148453] conn(0x7ff9d80acba0 0x7ff9d80aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:27.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.344+0000 7ff9e37fe640 1 -- 172.21.15.120:0/1630396343 --> [v2:172.21.15.153:6816/2535148453,v1:172.21.15.153:6817/2535148453] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff9d80af670 con 0x7ff9d80acba0 2024-09-19T09:01:27.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.344+0000 7ff9e37fe640 1 -- 172.21.15.120:0/1630396343 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7ff9d40cfe80 con 0x7ff9ec102570 2024-09-19T09:01:27.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.344+0000 7ff9ea59c640 1 --2- 172.21.15.120:0/1630396343 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff9d80a3350 0x7ff9d80a5810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc000c00 tx=0x7ff9dc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:27.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.344+0000 7ff9ead9d640 1 --2- 172.21.15.120:0/1630396343 >> [v2:172.21.15.153:6816/2535148453,v1:172.21.15.153:6817/2535148453] conn(0x7ff9d80acba0 0x7ff9d80aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:27.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.344+0000 7ff9ead9d640 1 --2- 172.21.15.120:0/1630396343 >> [v2:172.21.15.153:6816/2535148453,v1:172.21.15.153:6817/2535148453] conn(0x7ff9d80acba0 0x7ff9d80aefc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:27.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.345+0000 7ff9e37fe640 1 -- 172.21.15.120:0/1630396343 <== osd.7 v2:172.21.15.153:6816/2535148453 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff9d80af670 con 0x7ff9d80acba0 2024-09-19T09:01:27.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.359+0000 7ff9f0f1e640 1 -- 172.21.15.120:0/1630396343 --> [v2:172.21.15.153:6816/2535148453,v1:172.21.15.153:6817/2535148453] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff9ec062dc0 con 0x7ff9d80acba0 2024-09-19T09:01:27.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.359+0000 7ff9e37fe640 1 -- 172.21.15.120:0/1630396343 <== osd.7 v2:172.21.15.153:6816/2535148453 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff9ec062dc0 con 0x7ff9d80acba0 2024-09-19T09:01:27.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.359+0000 7ff9f0f1e640 1 -- 172.21.15.120:0/1630396343 >> [v2:172.21.15.153:6816/2535148453,v1:172.21.15.153:6817/2535148453] conn(0x7ff9d80acba0 msgr2=0x7ff9d80aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:27.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.359+0000 7ff9f0f1e640 1 --2- 172.21.15.120:0/1630396343 >> [v2:172.21.15.153:6816/2535148453,v1:172.21.15.153:6817/2535148453] conn(0x7ff9d80acba0 0x7ff9d80aefc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:27.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.359+0000 7ff9f0f1e640 1 -- 172.21.15.120:0/1630396343 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff9d80a3350 msgr2=0x7ff9d80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:27.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.359+0000 7ff9f0f1e640 1 --2- 172.21.15.120:0/1630396343 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff9d80a3350 0x7ff9d80a5810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc000c00 tx=0x7ff9dc0058d0 comp rx=0 tx=0).stop 2024-09-19T09:01:27.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.359+0000 7ff9f0f1e640 1 -- 172.21.15.120:0/1630396343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9ec102570 msgr2=0x7ff9ec199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:27.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.359+0000 7ff9f0f1e640 1 --2- 172.21.15.120:0/1630396343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9ec102570 0x7ff9ec199890 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400c890 tx=0x7ff9d400cd50 comp rx=0 tx=0).stop 2024-09-19T09:01:27.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.359+0000 7ff9ead9d640 1 -- 172.21.15.120:0/1630396343 reap_dead start 2024-09-19T09:01:27.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.359+0000 7ff9f0f1e640 1 -- 172.21.15.120:0/1630396343 shutdown_connections 2024-09-19T09:01:27.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.359+0000 7ff9f0f1e640 1 -- 172.21.15.120:0/1630396343 >> 172.21.15.120:0/1630396343 conn(0x7ff9ec0fc7d0 msgr2=0x7ff9ec0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:27.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.360+0000 7ff9f0f1e640 1 -- 172.21.15.120:0/1630396343 shutdown_connections 2024-09-19T09:01:27.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:25.360+0000 7ff9f0f1e640 1 -- 172.21.15.120:0/1630396343 wait complete. 2024-09-19T09:01:28.242 INFO:teuthology.orchestra.run.smithi120.stdout:206158430250 2024-09-19T09:01:28.242 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-19T09:01:28.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.633+0000 7f4dee023640 1 -- 172.21.15.120:0/1793381810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4de8102790 msgr2=0x7f4de81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:28.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.633+0000 7f4dee023640 1 --2- 172.21.15.120:0/1793381810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4de8102790 0x7f4de81067e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc009900 tx=0x7f4ddc02a410 comp rx=0 tx=0).stop 2024-09-19T09:01:28.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.633+0000 7f4dee023640 1 -- 172.21.15.120:0/1793381810 shutdown_connections 2024-09-19T09:01:28.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.633+0000 7f4dee023640 1 --2- 172.21.15.120:0/1793381810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4de8106f10 0x7f4de810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:28.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.633+0000 7f4dee023640 1 --2- 172.21.15.120:0/1793381810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4de8102790 0x7f4de81067e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:28.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.633+0000 7f4dee023640 1 --2- 172.21.15.120:0/1793381810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4de8101de0 0x7f4de81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:28.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.633+0000 7f4dee023640 1 -- 172.21.15.120:0/1793381810 >> 172.21.15.120:0/1793381810 conn(0x7f4de80fd8c0 msgr2=0x7f4de80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:28.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.633+0000 7f4dee023640 1 -- 172.21.15.120:0/1793381810 shutdown_connections 2024-09-19T09:01:28.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.633+0000 7f4dee023640 1 -- 172.21.15.120:0/1793381810 wait complete. 2024-09-19T09:01:28.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.633+0000 7f4dee023640 1 Processor -- start 2024-09-19T09:01:28.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.634+0000 7f4dee023640 1 -- start start 2024-09-19T09:01:28.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.634+0000 7f4dee023640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4de8101de0 0x7f4de80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:28.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.634+0000 7f4dee023640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4de8102790 0x7f4de8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:28.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.634+0000 7f4dee023640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4de8106f10 0x7f4de8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:28.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.634+0000 7f4dee023640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4de8112010 con 0x7f4de8101de0 2024-09-19T09:01:28.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.634+0000 7f4dee023640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4de8111e90 con 0x7f4de8106f10 2024-09-19T09:01:28.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.634+0000 7f4dee023640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4de8112190 con 0x7f4de8102790 2024-09-19T09:01:28.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.634+0000 7f4de77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4de8101de0 0x7f4de80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:28.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.634+0000 7f4de77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4de8101de0 0x7f4de80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55484/0 (socket says 172.21.15.120:55484) 2024-09-19T09:01:28.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.634+0000 7f4de77fe640 1 -- 172.21.15.120:0/208189739 learned_addr learned my addr 172.21.15.120:0/208189739 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:28.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.634+0000 7f4de6ffd640 1 --2- 172.21.15.120:0/208189739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4de8102790 0x7f4de8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:28.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.634+0000 7f4de77fe640 1 -- 172.21.15.120:0/208189739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4de8102790 msgr2=0x7f4de8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:28.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.634+0000 7f4de77fe640 1 --2- 172.21.15.120:0/208189739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4de8102790 0x7f4de8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:28.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.634+0000 7f4de77fe640 1 -- 172.21.15.120:0/208189739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4de8106f10 msgr2=0x7f4de8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:28.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.634+0000 7f4de77fe640 1 --2- 172.21.15.120:0/208189739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4de8106f10 0x7f4de8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:28.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.634+0000 7f4de77fe640 1 -- 172.21.15.120:0/208189739 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de81a6a70 con 0x7f4de8101de0 2024-09-19T09:01:28.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.634+0000 7f4de6ffd640 1 --2- 172.21.15.120:0/208189739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4de8102790 0x7f4de8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:01:28.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.634+0000 7f4de77fe640 1 --2- 172.21.15.120:0/208189739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4de8101de0 0x7f4de80744f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800ba50 tx=0x7f4dd800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:28.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.634+0000 7f4de4ff9640 1 -- 172.21.15.120:0/208189739 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8019070 con 0x7f4de8101de0 2024-09-19T09:01:28.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.635+0000 7f4dee023640 1 -- 172.21.15.120:0/208189739 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4de81a6d60 con 0x7f4de8101de0 2024-09-19T09:01:28.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.635+0000 7f4de4ff9640 1 -- 172.21.15.120:0/208189739 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dd800cbe0 con 0x7f4de8101de0 2024-09-19T09:01:28.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.635+0000 7f4de4ff9640 1 -- 172.21.15.120:0/208189739 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd80076e0 con 0x7f4de8101de0 2024-09-19T09:01:28.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.635+0000 7f4dee023640 1 -- 172.21.15.120:0/208189739 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4de81a71f0 con 0x7f4de8101de0 2024-09-19T09:01:28.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.636+0000 7f4dee023640 1 -- 172.21.15.120:0/208189739 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4de80628c0 con 0x7f4de8101de0 2024-09-19T09:01:28.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.637+0000 7f4de4ff9640 1 -- 172.21.15.120:0/208189739 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f4dd801c070 con 0x7f4de8101de0 2024-09-19T09:01:28.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.637+0000 7f4de4ff9640 1 --2- 172.21.15.120:0/208189739 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4dd00a32d0 0x7f4dd00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:28.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.638+0000 7f4de6ffd640 1 --2- 172.21.15.120:0/208189739 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4dd00a32d0 0x7f4dd00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:28.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.638+0000 7f4de4ff9640 1 -- 172.21.15.120:0/208189739 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f4dd80cf6c0 con 0x7f4de8101de0 2024-09-19T09:01:28.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.638+0000 7f4de4ff9640 1 --2- 172.21.15.120:0/208189739 >> [v2:172.21.15.153:6800/2259730294,v1:172.21.15.153:6801/2259730294] conn(0x7f4dd00acb20 0x7f4dd00aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:28.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.638+0000 7f4de4ff9640 1 -- 172.21.15.120:0/208189739 --> [v2:172.21.15.153:6800/2259730294,v1:172.21.15.153:6801/2259730294] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4dd00af5f0 con 0x7f4dd00acb20 2024-09-19T09:01:28.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.638+0000 7f4de4ff9640 1 -- 172.21.15.120:0/208189739 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f4dd80cfba0 con 0x7f4de8101de0 2024-09-19T09:01:28.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.638+0000 7f4de6ffd640 1 --2- 172.21.15.120:0/208189739 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4dd00a32d0 0x7f4dd00a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc000c00 tx=0x7f4ddc032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:28.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.638+0000 7f4de7fff640 1 --2- 172.21.15.120:0/208189739 >> [v2:172.21.15.153:6800/2259730294,v1:172.21.15.153:6801/2259730294] conn(0x7f4dd00acb20 0x7f4dd00aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:28.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.638+0000 7f4de7fff640 1 --2- 172.21.15.120:0/208189739 >> [v2:172.21.15.153:6800/2259730294,v1:172.21.15.153:6801/2259730294] conn(0x7f4dd00acb20 0x7f4dd00aef40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:28.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.639+0000 7f4de4ff9640 1 -- 172.21.15.120:0/208189739 <== osd.1 v2:172.21.15.153:6800/2259730294 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4dd00af5f0 con 0x7f4dd00acb20 2024-09-19T09:01:28.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.654+0000 7f4dee023640 1 -- 172.21.15.120:0/208189739 --> [v2:172.21.15.153:6800/2259730294,v1:172.21.15.153:6801/2259730294] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4de8062d40 con 0x7f4dd00acb20 2024-09-19T09:01:28.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.654+0000 7f4de4ff9640 1 -- 172.21.15.120:0/208189739 <== osd.1 v2:172.21.15.153:6800/2259730294 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4de8062d40 con 0x7f4dd00acb20 2024-09-19T09:01:28.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.654+0000 7f4dee023640 1 -- 172.21.15.120:0/208189739 >> [v2:172.21.15.153:6800/2259730294,v1:172.21.15.153:6801/2259730294] conn(0x7f4dd00acb20 msgr2=0x7f4dd00aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:28.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.654+0000 7f4dee023640 1 --2- 172.21.15.120:0/208189739 >> [v2:172.21.15.153:6800/2259730294,v1:172.21.15.153:6801/2259730294] conn(0x7f4dd00acb20 0x7f4dd00aef40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:28.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.654+0000 7f4dee023640 1 -- 172.21.15.120:0/208189739 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4dd00a32d0 msgr2=0x7f4dd00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:28.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.654+0000 7f4dee023640 1 --2- 172.21.15.120:0/208189739 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4dd00a32d0 0x7f4dd00a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc000c00 tx=0x7f4ddc032d50 comp rx=0 tx=0).stop 2024-09-19T09:01:28.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.654+0000 7f4dee023640 1 -- 172.21.15.120:0/208189739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4de8101de0 msgr2=0x7f4de80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:28.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.654+0000 7f4dee023640 1 --2- 172.21.15.120:0/208189739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4de8101de0 0x7f4de80744f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800ba50 tx=0x7f4dd800bf10 comp rx=0 tx=0).stop 2024-09-19T09:01:28.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.654+0000 7f4de7fff640 1 -- 172.21.15.120:0/208189739 reap_dead start 2024-09-19T09:01:28.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.654+0000 7f4dee023640 1 -- 172.21.15.120:0/208189739 shutdown_connections 2024-09-19T09:01:28.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.654+0000 7f4dee023640 1 -- 172.21.15.120:0/208189739 >> 172.21.15.120:0/208189739 conn(0x7f4de80fd8c0 msgr2=0x7f4de80fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:28.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.655+0000 7f4dee023640 1 -- 172.21.15.120:0/208189739 shutdown_connections 2024-09-19T09:01:28.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:28.655+0000 7f4dee023640 1 -- 172.21.15.120:0/208189739 wait complete. 2024-09-19T09:01:28.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:28 smithi120 ceph-mon[31150]: pgmap v297: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T09:01:29.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:28 smithi196 ceph-mon[35150]: pgmap v297: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T09:01:29.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:28 smithi153 ceph-mon[35071]: pgmap v297: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T09:01:29.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.598+0000 7faf87a65640 1 -- 172.21.15.120:0/2516918730 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf80101240 msgr2=0x7faf80101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:29.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.598+0000 7faf87a65640 1 --2- 172.21.15.120:0/2516918730 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf80101240 0x7faf80101680 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faf70009900 tx=0x7faf7002a410 comp rx=0 tx=0).stop 2024-09-19T09:01:29.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.598+0000 7faf87a65640 1 -- 172.21.15.120:0/2516918730 shutdown_connections 2024-09-19T09:01:29.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.598+0000 7faf87a65640 1 --2- 172.21.15.120:0/2516918730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf80101bc0 0x7faf8010d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:29.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.598+0000 7faf87a65640 1 --2- 172.21.15.120:0/2516918730 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf80101240 0x7faf80101680 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:29.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.598+0000 7faf87a65640 1 --2- 172.21.15.120:0/2516918730 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf80105a90 0x7faf80105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:29.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.598+0000 7faf87a65640 1 -- 172.21.15.120:0/2516918730 >> 172.21.15.120:0/2516918730 conn(0x7faf80077210 msgr2=0x7faf800fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:29.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.598+0000 7faf87a65640 1 -- 172.21.15.120:0/2516918730 shutdown_connections 2024-09-19T09:01:29.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.599+0000 7faf87a65640 1 -- 172.21.15.120:0/2516918730 wait complete. 2024-09-19T09:01:29.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.599+0000 7faf87a65640 1 Processor -- start 2024-09-19T09:01:29.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.599+0000 7faf87a65640 1 -- start start 2024-09-19T09:01:29.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.599+0000 7faf87a65640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf80101240 0x7faf80197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:29.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.599+0000 7faf87a65640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf80101bc0 0x7faf801975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:29.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.599+0000 7faf87a65640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf80105a90 0x7faf80197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:29.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.599+0000 7faf87a65640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faf80110f70 con 0x7faf80105a90 2024-09-19T09:01:29.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.599+0000 7faf87a65640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faf80110df0 con 0x7faf80101bc0 2024-09-19T09:01:29.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.599+0000 7faf87a65640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faf801110f0 con 0x7faf80101240 2024-09-19T09:01:29.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.599+0000 7faf86002640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf80105a90 0x7faf80197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:29.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.599+0000 7faf86002640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf80105a90 0x7faf80197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55504/0 (socket says 172.21.15.120:55504) 2024-09-19T09:01:29.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.599+0000 7faf86002640 1 -- 172.21.15.120:0/3550967039 learned_addr learned my addr 172.21.15.120:0/3550967039 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:29.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.600+0000 7faf85801640 1 --2- 172.21.15.120:0/3550967039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf80101240 0x7faf80197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:29.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.600+0000 7faf85000640 1 --2- 172.21.15.120:0/3550967039 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf80101bc0 0x7faf801975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:29.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.600+0000 7faf86002640 1 -- 172.21.15.120:0/3550967039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf80101240 msgr2=0x7faf80197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:29.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.600+0000 7faf86002640 1 --2- 172.21.15.120:0/3550967039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf80101240 0x7faf80197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:29.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.600+0000 7faf86002640 1 -- 172.21.15.120:0/3550967039 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf80101bc0 msgr2=0x7faf801975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:29.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.600+0000 7faf86002640 1 --2- 172.21.15.120:0/3550967039 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf80101bc0 0x7faf801975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:29.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.600+0000 7faf86002640 1 -- 172.21.15.120:0/3550967039 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf801a6900 con 0x7faf80105a90 2024-09-19T09:01:29.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.600+0000 7faf86002640 1 --2- 172.21.15.120:0/3550967039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf80105a90 0x7faf80197b10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faf7800eaa0 tx=0x7faf7800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:29.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.600+0000 7faf767fc640 1 -- 172.21.15.120:0/3550967039 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7800cdb0 con 0x7faf80105a90 2024-09-19T09:01:29.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.600+0000 7faf767fc640 1 -- 172.21.15.120:0/3550967039 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf78005020 con 0x7faf80105a90 2024-09-19T09:01:29.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.600+0000 7faf87a65640 1 -- 172.21.15.120:0/3550967039 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf801a6bf0 con 0x7faf80105a90 2024-09-19T09:01:29.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.600+0000 7faf767fc640 1 -- 172.21.15.120:0/3550967039 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf780116b0 con 0x7faf80105a90 2024-09-19T09:01:29.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.600+0000 7faf87a65640 1 -- 172.21.15.120:0/3550967039 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf801a7180 con 0x7faf80105a90 2024-09-19T09:01:29.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.601+0000 7faf87a65640 1 -- 172.21.15.120:0/3550967039 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faf801042a0 con 0x7faf80105a90 2024-09-19T09:01:29.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.602+0000 7faf767fc640 1 -- 172.21.15.120:0/3550967039 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7faf78005560 con 0x7faf80105a90 2024-09-19T09:01:29.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.603+0000 7faf767fc640 1 --2- 172.21.15.120:0/3550967039 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faf6c0a3350 0x7faf6c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:29.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.603+0000 7faf85801640 1 --2- 172.21.15.120:0/3550967039 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faf6c0a3350 0x7faf6c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:29.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.603+0000 7faf767fc640 1 -- 172.21.15.120:0/3550967039 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7faf780cff40 con 0x7faf80105a90 2024-09-19T09:01:29.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.603+0000 7faf767fc640 1 --2- 172.21.15.120:0/3550967039 >> [v2:172.21.15.153:6824/3216877560,v1:172.21.15.153:6825/3216877560] conn(0x7faf6c0acba0 0x7faf6c0aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:29.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.603+0000 7faf767fc640 1 -- 172.21.15.120:0/3550967039 --> [v2:172.21.15.153:6824/3216877560,v1:172.21.15.153:6825/3216877560] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faf78014ec0 con 0x7faf6c0acba0 2024-09-19T09:01:29.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.603+0000 7faf767fc640 1 -- 172.21.15.120:0/3550967039 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7faf780d0420 con 0x7faf80105a90 2024-09-19T09:01:29.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.603+0000 7faf85801640 1 --2- 172.21.15.120:0/3550967039 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faf6c0a3350 0x7faf6c0a5810 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faf7c005e10 tx=0x7faf7c005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:29.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.603+0000 7faf85000640 1 --2- 172.21.15.120:0/3550967039 >> [v2:172.21.15.153:6824/3216877560,v1:172.21.15.153:6825/3216877560] conn(0x7faf6c0acba0 0x7faf6c0aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:29.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.604+0000 7faf85000640 1 --2- 172.21.15.120:0/3550967039 >> [v2:172.21.15.153:6824/3216877560,v1:172.21.15.153:6825/3216877560] conn(0x7faf6c0acba0 0x7faf6c0aefc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:29.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.605+0000 7faf767fc640 1 -- 172.21.15.120:0/3550967039 <== osd.10 v2:172.21.15.153:6824/3216877560 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faf78014ec0 con 0x7faf6c0acba0 2024-09-19T09:01:29.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.618+0000 7faf87a65640 1 -- 172.21.15.120:0/3550967039 --> [v2:172.21.15.153:6824/3216877560,v1:172.21.15.153:6825/3216877560] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faf80102250 con 0x7faf6c0acba0 2024-09-19T09:01:29.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.619+0000 7faf767fc640 1 -- 172.21.15.120:0/3550967039 <== osd.10 v2:172.21.15.153:6824/3216877560 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7faf80102250 con 0x7faf6c0acba0 2024-09-19T09:01:29.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.619+0000 7faf87a65640 1 -- 172.21.15.120:0/3550967039 >> [v2:172.21.15.153:6824/3216877560,v1:172.21.15.153:6825/3216877560] conn(0x7faf6c0acba0 msgr2=0x7faf6c0aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:29.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.619+0000 7faf87a65640 1 --2- 172.21.15.120:0/3550967039 >> [v2:172.21.15.153:6824/3216877560,v1:172.21.15.153:6825/3216877560] conn(0x7faf6c0acba0 0x7faf6c0aefc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:29.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.619+0000 7faf87a65640 1 -- 172.21.15.120:0/3550967039 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faf6c0a3350 msgr2=0x7faf6c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:29.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.619+0000 7faf87a65640 1 --2- 172.21.15.120:0/3550967039 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faf6c0a3350 0x7faf6c0a5810 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faf7c005e10 tx=0x7faf7c005da0 comp rx=0 tx=0).stop 2024-09-19T09:01:29.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.619+0000 7faf87a65640 1 -- 172.21.15.120:0/3550967039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf80105a90 msgr2=0x7faf80197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:29.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.619+0000 7faf87a65640 1 --2- 172.21.15.120:0/3550967039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf80105a90 0x7faf80197b10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faf7800eaa0 tx=0x7faf7800ef60 comp rx=0 tx=0).stop 2024-09-19T09:01:29.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.619+0000 7faf86002640 1 -- 172.21.15.120:0/3550967039 reap_dead start 2024-09-19T09:01:29.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.619+0000 7faf87a65640 1 -- 172.21.15.120:0/3550967039 shutdown_connections 2024-09-19T09:01:29.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.619+0000 7faf87a65640 1 -- 172.21.15.120:0/3550967039 >> 172.21.15.120:0/3550967039 conn(0x7faf80077210 msgr2=0x7faf80103900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:29.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.619+0000 7faf87a65640 1 -- 172.21.15.120:0/3550967039 shutdown_connections 2024-09-19T09:01:29.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:29.619+0000 7faf87a65640 1 -- 172.21.15.120:0/3550967039 wait complete. 2024-09-19T09:01:30.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.008+0000 7f37cc07f640 1 -- 172.21.15.120:0/1075826166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37c4106a80 msgr2=0x7f37c4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:30.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.008+0000 7f37cc07f640 1 --2- 172.21.15.120:0/1075826166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37c4106a80 0x7f37c4106e60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f37c0009930 tx=0x7f37c002a420 comp rx=0 tx=0).stop 2024-09-19T09:01:30.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.008+0000 7f37cc07f640 1 -- 172.21.15.120:0/1075826166 shutdown_connections 2024-09-19T09:01:30.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.008+0000 7f37cc07f640 1 --2- 172.21.15.120:0/1075826166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37c4101420 0x7f37c410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:30.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.008+0000 7f37cc07f640 1 --2- 172.21.15.120:0/1075826166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37c4100a80 0x7f37c4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:30.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.008+0000 7f37cc07f640 1 --2- 172.21.15.120:0/1075826166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37c4106a80 0x7f37c4106e60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:30.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.008+0000 7f37cc07f640 1 -- 172.21.15.120:0/1075826166 >> 172.21.15.120:0/1075826166 conn(0x7f37c40fc7a0 msgr2=0x7f37c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:30.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.008+0000 7f37cc07f640 1 -- 172.21.15.120:0/1075826166 shutdown_connections 2024-09-19T09:01:30.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.008+0000 7f37cc07f640 1 -- 172.21.15.120:0/1075826166 wait complete. 2024-09-19T09:01:30.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.008+0000 7f37cc07f640 1 Processor -- start 2024-09-19T09:01:30.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.008+0000 7f37cc07f640 1 -- start start 2024-09-19T09:01:30.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.009+0000 7f37cc07f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37c4100a80 0x7f37c4197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:30.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.009+0000 7f37cc07f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37c4101420 0x7f37c41978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:30.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.009+0000 7f37cc07f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37c4106a80 0x7f37c419bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:30.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.009+0000 7f37cc07f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f37c4110e80 con 0x7f37c4106a80 2024-09-19T09:01:30.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.009+0000 7f37cc07f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f37c4110d00 con 0x7f37c4100a80 2024-09-19T09:01:30.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.009+0000 7f37cc07f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37c4111000 con 0x7f37c4101420 2024-09-19T09:01:30.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.009+0000 7f37ca61c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37c4106a80 0x7f37c419bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:30.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.009+0000 7f37ca61c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37c4106a80 0x7f37c419bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55520/0 (socket says 172.21.15.120:55520) 2024-09-19T09:01:30.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.009+0000 7f37ca61c640 1 -- 172.21.15.120:0/4159255892 learned_addr learned my addr 172.21.15.120:0/4159255892 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:30.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.009+0000 7f37c9e1b640 1 --2- 172.21.15.120:0/4159255892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37c4100a80 0x7f37c4197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:30.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.009+0000 7f37c961a640 1 --2- 172.21.15.120:0/4159255892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37c4101420 0x7f37c41978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:30.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.009+0000 7f37ca61c640 1 -- 172.21.15.120:0/4159255892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37c4101420 msgr2=0x7f37c41978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:30.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.009+0000 7f37ca61c640 1 --2- 172.21.15.120:0/4159255892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37c4101420 0x7f37c41978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:30.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.009+0000 7f37ca61c640 1 -- 172.21.15.120:0/4159255892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37c4100a80 msgr2=0x7f37c4197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:30.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.009+0000 7f37ca61c640 1 --2- 172.21.15.120:0/4159255892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37c4100a80 0x7f37c4197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:30.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.009+0000 7f37ca61c640 1 -- 172.21.15.120:0/4159255892 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c419c330 con 0x7f37c4106a80 2024-09-19T09:01:30.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.009+0000 7f37c961a640 1 --2- 172.21.15.120:0/4159255892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37c4101420 0x7f37c41978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:01:30.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.009+0000 7f37ca61c640 1 --2- 172.21.15.120:0/4159255892 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37c4106a80 0x7f37c419bc50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009fd0 tx=0x7f37bc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:30.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.009+0000 7f37baffd640 1 -- 172.21.15.120:0/4159255892 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc017070 con 0x7f37c4106a80 2024-09-19T09:01:30.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.009+0000 7f37baffd640 1 -- 172.21.15.120:0/4159255892 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37bc003e50 con 0x7f37c4106a80 2024-09-19T09:01:30.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.009+0000 7f37cc07f640 1 -- 172.21.15.120:0/4159255892 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c419c620 con 0x7f37c4106a80 2024-09-19T09:01:30.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.009+0000 7f37baffd640 1 -- 172.21.15.120:0/4159255892 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc0106c0 con 0x7f37c4106a80 2024-09-19T09:01:30.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.009+0000 7f37cc07f640 1 -- 172.21.15.120:0/4159255892 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c41a3f00 con 0x7f37c4106a80 2024-09-19T09:01:30.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.010+0000 7f37cc07f640 1 -- 172.21.15.120:0/4159255892 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f37c4062980 con 0x7f37c4106a80 2024-09-19T09:01:30.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.012+0000 7f37baffd640 1 -- 172.21.15.120:0/4159255892 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f37bc003970 con 0x7f37c4106a80 2024-09-19T09:01:30.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.012+0000 7f37baffd640 1 --2- 172.21.15.120:0/4159255892 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f37b00a32d0 0x7f37b00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:30.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.012+0000 7f37c9e1b640 1 --2- 172.21.15.120:0/4159255892 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f37b00a32d0 0x7f37b00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:30.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.013+0000 7f37baffd640 1 -- 172.21.15.120:0/4159255892 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f37bc0cfa10 con 0x7f37c4106a80 2024-09-19T09:01:30.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.013+0000 7f37baffd640 1 --2- 172.21.15.120:0/4159255892 >> [v2:172.21.15.153:6808/2680865854,v1:172.21.15.153:6809/2680865854] conn(0x7f37b00acb20 0x7f37b00aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:30.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.013+0000 7f37baffd640 1 -- 172.21.15.120:0/4159255892 --> [v2:172.21.15.153:6808/2680865854,v1:172.21.15.153:6809/2680865854] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f37bc01de50 con 0x7f37b00acb20 2024-09-19T09:01:30.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.013+0000 7f37baffd640 1 -- 172.21.15.120:0/4159255892 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f37bc0cfef0 con 0x7f37c4106a80 2024-09-19T09:01:30.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.013+0000 7f37c9e1b640 1 --2- 172.21.15.120:0/4159255892 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f37b00a32d0 0x7f37b00a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f37c0009900 tx=0x7f37c00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:30.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.013+0000 7f37c961a640 1 --2- 172.21.15.120:0/4159255892 >> [v2:172.21.15.153:6808/2680865854,v1:172.21.15.153:6809/2680865854] conn(0x7f37b00acb20 0x7f37b00aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:30.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.013+0000 7f37c961a640 1 --2- 172.21.15.120:0/4159255892 >> [v2:172.21.15.153:6808/2680865854,v1:172.21.15.153:6809/2680865854] conn(0x7f37b00acb20 0x7f37b00aef40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:30.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.014+0000 7f37baffd640 1 -- 172.21.15.120:0/4159255892 <== osd.4 v2:172.21.15.153:6808/2680865854 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f37bc01de50 con 0x7f37b00acb20 2024-09-19T09:01:30.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.028+0000 7f37cc07f640 1 -- 172.21.15.120:0/4159255892 --> [v2:172.21.15.153:6808/2680865854,v1:172.21.15.153:6809/2680865854] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f37c4062e00 con 0x7f37b00acb20 2024-09-19T09:01:30.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.028+0000 7f37baffd640 1 -- 172.21.15.120:0/4159255892 <== osd.4 v2:172.21.15.153:6808/2680865854 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f37c4062e00 con 0x7f37b00acb20 2024-09-19T09:01:30.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.029+0000 7f37cc07f640 1 -- 172.21.15.120:0/4159255892 >> [v2:172.21.15.153:6808/2680865854,v1:172.21.15.153:6809/2680865854] conn(0x7f37b00acb20 msgr2=0x7f37b00aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:30.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.029+0000 7f37cc07f640 1 --2- 172.21.15.120:0/4159255892 >> [v2:172.21.15.153:6808/2680865854,v1:172.21.15.153:6809/2680865854] conn(0x7f37b00acb20 0x7f37b00aef40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:30.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.029+0000 7f37cc07f640 1 -- 172.21.15.120:0/4159255892 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f37b00a32d0 msgr2=0x7f37b00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:30.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.029+0000 7f37cc07f640 1 --2- 172.21.15.120:0/4159255892 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f37b00a32d0 0x7f37b00a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f37c0009900 tx=0x7f37c00058d0 comp rx=0 tx=0).stop 2024-09-19T09:01:30.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.029+0000 7f37cc07f640 1 -- 172.21.15.120:0/4159255892 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37c4106a80 msgr2=0x7f37c419bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:30.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.029+0000 7f37cc07f640 1 --2- 172.21.15.120:0/4159255892 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37c4106a80 0x7f37c419bc50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009fd0 tx=0x7f37bc00b260 comp rx=0 tx=0).stop 2024-09-19T09:01:30.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.029+0000 7f37ca61c640 1 -- 172.21.15.120:0/4159255892 reap_dead start 2024-09-19T09:01:30.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.029+0000 7f37cc07f640 1 -- 172.21.15.120:0/4159255892 shutdown_connections 2024-09-19T09:01:30.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.029+0000 7f37cc07f640 1 -- 172.21.15.120:0/4159255892 >> 172.21.15.120:0/4159255892 conn(0x7f37c40fc7a0 msgr2=0x7f37c40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:30.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.029+0000 7f37cc07f640 1 -- 172.21.15.120:0/4159255892 shutdown_connections 2024-09-19T09:01:30.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.029+0000 7f37cc07f640 1 -- 172.21.15.120:0/4159255892 wait complete. 2024-09-19T09:01:30.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.469+0000 7fb6a3cd7640 1 -- 172.21.15.120:0/2257797988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb69c102ef0 msgr2=0x7fb69c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:30.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.469+0000 7fb6a3cd7640 1 --2- 172.21.15.120:0/2257797988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb69c102ef0 0x7fb69c10da40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb694009ca0 tx=0x7fb69402a950 comp rx=0 tx=0).stop 2024-09-19T09:01:30.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.469+0000 7fb6a3cd7640 1 -- 172.21.15.120:0/2257797988 shutdown_connections 2024-09-19T09:01:30.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.469+0000 7fb6a3cd7640 1 --2- 172.21.15.120:0/2257797988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb69c102ef0 0x7fb69c10da40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:30.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.469+0000 7fb6a3cd7640 1 --2- 172.21.15.120:0/2257797988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb69c102570 0x7fb69c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:30.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.469+0000 7fb6a3cd7640 1 --2- 172.21.15.120:0/2257797988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb69c100e20 0x7fb69c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:30.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.469+0000 7fb6a3cd7640 1 -- 172.21.15.120:0/2257797988 >> 172.21.15.120:0/2257797988 conn(0x7fb69c0fc7d0 msgr2=0x7fb69c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:30.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.469+0000 7fb6a3cd7640 1 -- 172.21.15.120:0/2257797988 shutdown_connections 2024-09-19T09:01:30.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.469+0000 7fb6a3cd7640 1 -- 172.21.15.120:0/2257797988 wait complete. 2024-09-19T09:01:30.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.470+0000 7fb6a3cd7640 1 Processor -- start 2024-09-19T09:01:30.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.470+0000 7fb6a3cd7640 1 -- start start 2024-09-19T09:01:30.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.470+0000 7fb6a3cd7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb69c100e20 0x7fb69c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:30.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.470+0000 7fb6a3cd7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb69c102570 0x7fb69c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:30.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.470+0000 7fb6a3cd7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb69c102ef0 0x7fb69c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:30.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.470+0000 7fb6a3cd7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb69c110f80 con 0x7fb69c100e20 2024-09-19T09:01:30.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.470+0000 7fb6a3cd7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb69c110e00 con 0x7fb69c102ef0 2024-09-19T09:01:30.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.470+0000 7fb6a3cd7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb69c111100 con 0x7fb69c102570 2024-09-19T09:01:30.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.470+0000 7fb6a1a73640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb69c100e20 0x7fb69c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:30.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.470+0000 7fb6a1a73640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb69c100e20 0x7fb69c198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55548/0 (socket says 172.21.15.120:55548) 2024-09-19T09:01:30.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.470+0000 7fb6a1a73640 1 -- 172.21.15.120:0/3999428922 learned_addr learned my addr 172.21.15.120:0/3999428922 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:30.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.470+0000 7fb6a1272640 1 --2- 172.21.15.120:0/3999428922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb69c102570 0x7fb69c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:30.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.470+0000 7fb6a2274640 1 --2- 172.21.15.120:0/3999428922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb69c102ef0 0x7fb69c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:30.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.470+0000 7fb6a1a73640 1 -- 172.21.15.120:0/3999428922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb69c102570 msgr2=0x7fb69c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:30.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.470+0000 7fb6a1a73640 1 --2- 172.21.15.120:0/3999428922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb69c102570 0x7fb69c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:30.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.470+0000 7fb6a1a73640 1 -- 172.21.15.120:0/3999428922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb69c102ef0 msgr2=0x7fb69c19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:30.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.470+0000 7fb6a1a73640 1 --2- 172.21.15.120:0/3999428922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb69c102ef0 0x7fb69c19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:30.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.470+0000 7fb6a1a73640 1 -- 172.21.15.120:0/3999428922 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb69c19d420 con 0x7fb69c100e20 2024-09-19T09:01:30.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.470+0000 7fb6a1272640 1 --2- 172.21.15.120:0/3999428922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb69c102570 0x7fb69c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:01:30.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.470+0000 7fb6a2274640 1 --2- 172.21.15.120:0/3999428922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb69c102ef0 0x7fb69c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:01:30.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.470+0000 7fb6a1a73640 1 --2- 172.21.15.120:0/3999428922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb69c100e20 0x7fb69c198470 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb69800c890 tx=0x7fb69800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:30.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.471+0000 7fb692ffd640 1 -- 172.21.15.120:0/3999428922 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb698004220 con 0x7fb69c100e20 2024-09-19T09:01:30.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.471+0000 7fb692ffd640 1 -- 172.21.15.120:0/3999428922 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6980043c0 con 0x7fb69c100e20 2024-09-19T09:01:30.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.471+0000 7fb6a3cd7640 1 -- 172.21.15.120:0/3999428922 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb69c19d710 con 0x7fb69c100e20 2024-09-19T09:01:30.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.471+0000 7fb692ffd640 1 -- 172.21.15.120:0/3999428922 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6980049e0 con 0x7fb69c100e20 2024-09-19T09:01:30.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.471+0000 7fb6a3cd7640 1 -- 172.21.15.120:0/3999428922 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb69c1a4ff0 con 0x7fb69c100e20 2024-09-19T09:01:30.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.471+0000 7fb6a3cd7640 1 -- 172.21.15.120:0/3999428922 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb69c062900 con 0x7fb69c100e20 2024-09-19T09:01:30.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.473+0000 7fb692ffd640 1 -- 172.21.15.120:0/3999428922 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fb6980079b0 con 0x7fb69c100e20 2024-09-19T09:01:30.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.473+0000 7fb692ffd640 1 --2- 172.21.15.120:0/3999428922 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb6880a32d0 0x7fb6880a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:30.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.473+0000 7fb6a1272640 1 --2- 172.21.15.120:0/3999428922 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb6880a32d0 0x7fb6880a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:30.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.473+0000 7fb692ffd640 1 -- 172.21.15.120:0/3999428922 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fb6980cfbe0 con 0x7fb69c100e20 2024-09-19T09:01:30.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.474+0000 7fb692ffd640 1 --2- 172.21.15.120:0/3999428922 >> [v2:172.21.15.196:6816/186195906,v1:172.21.15.196:6817/186195906] conn(0x7fb6880acb20 0x7fb6880aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:30.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.474+0000 7fb692ffd640 1 -- 172.21.15.120:0/3999428922 --> [v2:172.21.15.196:6816/186195906,v1:172.21.15.196:6817/186195906] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb6880af5f0 con 0x7fb6880acb20 2024-09-19T09:01:30.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.474+0000 7fb692ffd640 1 -- 172.21.15.120:0/3999428922 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fb6980d00c0 con 0x7fb69c100e20 2024-09-19T09:01:30.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.474+0000 7fb6a1272640 1 --2- 172.21.15.120:0/3999428922 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb6880a32d0 0x7fb6880a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb69c0ff060 tx=0x7fb68c005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:30.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.474+0000 7fb6a2274640 1 --2- 172.21.15.120:0/3999428922 >> [v2:172.21.15.196:6816/186195906,v1:172.21.15.196:6817/186195906] conn(0x7fb6880acb20 0x7fb6880aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:30.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.474+0000 7fb6a2274640 1 --2- 172.21.15.120:0/3999428922 >> [v2:172.21.15.196:6816/186195906,v1:172.21.15.196:6817/186195906] conn(0x7fb6880acb20 0x7fb6880aef40 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:30.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.475+0000 7fb692ffd640 1 -- 172.21.15.120:0/3999428922 <== osd.8 v2:172.21.15.196:6816/186195906 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb6880af5f0 con 0x7fb6880acb20 2024-09-19T09:01:30.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.489+0000 7fb6a3cd7640 1 -- 172.21.15.120:0/3999428922 --> [v2:172.21.15.196:6816/186195906,v1:172.21.15.196:6817/186195906] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb69c062d80 con 0x7fb6880acb20 2024-09-19T09:01:30.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.489+0000 7fb692ffd640 1 -- 172.21.15.120:0/3999428922 <== osd.8 v2:172.21.15.196:6816/186195906 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb69c062d80 con 0x7fb6880acb20 2024-09-19T09:01:30.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.489+0000 7fb6a3cd7640 1 -- 172.21.15.120:0/3999428922 >> [v2:172.21.15.196:6816/186195906,v1:172.21.15.196:6817/186195906] conn(0x7fb6880acb20 msgr2=0x7fb6880aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:30.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.489+0000 7fb6a3cd7640 1 --2- 172.21.15.120:0/3999428922 >> [v2:172.21.15.196:6816/186195906,v1:172.21.15.196:6817/186195906] conn(0x7fb6880acb20 0x7fb6880aef40 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:30.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.490+0000 7fb6a3cd7640 1 -- 172.21.15.120:0/3999428922 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb6880a32d0 msgr2=0x7fb6880a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:30.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.490+0000 7fb6a3cd7640 1 --2- 172.21.15.120:0/3999428922 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb6880a32d0 0x7fb6880a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb69c0ff060 tx=0x7fb68c005e50 comp rx=0 tx=0).stop 2024-09-19T09:01:30.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.490+0000 7fb6a3cd7640 1 -- 172.21.15.120:0/3999428922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb69c100e20 msgr2=0x7fb69c198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:30.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.490+0000 7fb6a3cd7640 1 --2- 172.21.15.120:0/3999428922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb69c100e20 0x7fb69c198470 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb69800c890 tx=0x7fb69800cd50 comp rx=0 tx=0).stop 2024-09-19T09:01:30.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.490+0000 7fb6a2274640 1 -- 172.21.15.120:0/3999428922 reap_dead start 2024-09-19T09:01:30.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.490+0000 7fb6a3cd7640 1 -- 172.21.15.120:0/3999428922 shutdown_connections 2024-09-19T09:01:30.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.490+0000 7fb6a3cd7640 1 -- 172.21.15.120:0/3999428922 >> 172.21.15.120:0/3999428922 conn(0x7fb69c0fc7d0 msgr2=0x7fb69c06a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:30.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.490+0000 7fb6a3cd7640 1 -- 172.21.15.120:0/3999428922 shutdown_connections 2024-09-19T09:01:30.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:30.490+0000 7fb6a3cd7640 1 -- 172.21.15.120:0/3999428922 wait complete. 2024-09-19T09:01:30.988 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:30 smithi196 ceph-mon[35150]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-19T09:01:30.989 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:30 smithi153 ceph-mon[35071]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-19T09:01:31.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:30 smithi120 ceph-mon[31150]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-19T09:01:31.309 INFO:teuthology.orchestra.run.smithi120.stdout:55834574920 2024-09-19T09:01:31.309 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-19T09:01:31.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.780+0000 7f7451d37640 1 -- 172.21.15.120:0/1295790660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f744c100de0 msgr2=0x7f744c104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:31.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.780+0000 7f7451d37640 1 --2- 172.21.15.120:0/1295790660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f744c100de0 0x7f744c104e30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7434009900 tx=0x7f743402a410 comp rx=0 tx=0).stop 2024-09-19T09:01:31.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.780+0000 7f7451d37640 1 -- 172.21.15.120:0/1295790660 shutdown_connections 2024-09-19T09:01:31.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.780+0000 7f7451d37640 1 --2- 172.21.15.120:0/1295790660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f744c105560 0x7f744c10d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:31.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.780+0000 7f7451d37640 1 --2- 172.21.15.120:0/1295790660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f744c100de0 0x7f744c104e30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:31.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.780+0000 7f7451d37640 1 --2- 172.21.15.120:0/1295790660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f744c100430 0x7f744c100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:31.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.780+0000 7f7451d37640 1 -- 172.21.15.120:0/1295790660 >> 172.21.15.120:0/1295790660 conn(0x7f744c077230 msgr2=0x7f744c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:31.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.780+0000 7f7451d37640 1 -- 172.21.15.120:0/1295790660 shutdown_connections 2024-09-19T09:01:31.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.780+0000 7f7451d37640 1 -- 172.21.15.120:0/1295790660 wait complete. 2024-09-19T09:01:31.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.780+0000 7f7451d37640 1 Processor -- start 2024-09-19T09:01:31.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.780+0000 7f7451d37640 1 -- start start 2024-09-19T09:01:31.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.780+0000 7f7451d37640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f744c100430 0x7f744c1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:31.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.781+0000 7f7451d37640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f744c100de0 0x7f744c198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:31.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.781+0000 7f7451d37640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f744c105560 0x7f744c19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:31.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.781+0000 7f7451d37640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f744c110e60 con 0x7f744c100430 2024-09-19T09:01:31.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.781+0000 7f7451d37640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f744c110ce0 con 0x7f744c100de0 2024-09-19T09:01:31.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.781+0000 7f7451d37640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f744c110fe0 con 0x7f744c105560 2024-09-19T09:01:31.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.781+0000 7f744b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f744c100430 0x7f744c1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:31.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.781+0000 7f744b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f744c100430 0x7f744c1984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36928/0 (socket says 172.21.15.120:36928) 2024-09-19T09:01:31.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.781+0000 7f744b7fe640 1 -- 172.21.15.120:0/1220071885 learned_addr learned my addr 172.21.15.120:0/1220071885 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:31.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.781+0000 7f7443fff640 1 --2- 172.21.15.120:0/1220071885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f744c100de0 0x7f744c198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:31.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.781+0000 7f744bfff640 1 --2- 172.21.15.120:0/1220071885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f744c105560 0x7f744c19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:31.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.781+0000 7f744b7fe640 1 -- 172.21.15.120:0/1220071885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f744c105560 msgr2=0x7f744c19cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:31.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.781+0000 7f744b7fe640 1 --2- 172.21.15.120:0/1220071885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f744c105560 0x7f744c19cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:31.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.781+0000 7f744b7fe640 1 -- 172.21.15.120:0/1220071885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f744c100de0 msgr2=0x7f744c198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:31.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.781+0000 7f744b7fe640 1 --2- 172.21.15.120:0/1220071885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f744c100de0 0x7f744c198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:31.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.781+0000 7f744b7fe640 1 -- 172.21.15.120:0/1220071885 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f744c19d490 con 0x7f744c100430 2024-09-19T09:01:31.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.781+0000 7f744b7fe640 1 --2- 172.21.15.120:0/1220071885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f744c100430 0x7f744c1984e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f743c00b4a0 tx=0x7f743c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:31.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.781+0000 7f74497fa640 1 -- 172.21.15.120:0/1220071885 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743c014070 con 0x7f744c100430 2024-09-19T09:01:31.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.781+0000 7f7451d37640 1 -- 172.21.15.120:0/1220071885 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f744c19d780 con 0x7f744c100430 2024-09-19T09:01:31.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.781+0000 7f74497fa640 1 -- 172.21.15.120:0/1220071885 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f743c010af0 con 0x7f744c100430 2024-09-19T09:01:31.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.781+0000 7f74497fa640 1 -- 172.21.15.120:0/1220071885 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743c017d10 con 0x7f744c100430 2024-09-19T09:01:31.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.781+0000 7f7451d37640 1 -- 172.21.15.120:0/1220071885 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f744c1a5060 con 0x7f744c100430 2024-09-19T09:01:31.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.782+0000 7f7451d37640 1 -- 172.21.15.120:0/1220071885 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f744c062940 con 0x7f744c100430 2024-09-19T09:01:31.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.783+0000 7f74497fa640 1 -- 172.21.15.120:0/1220071885 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f743c017070 con 0x7f744c100430 2024-09-19T09:01:31.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.784+0000 7f74497fa640 1 --2- 172.21.15.120:0/1220071885 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f74380a32d0 0x7f74380a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:31.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.784+0000 7f7443fff640 1 --2- 172.21.15.120:0/1220071885 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f74380a32d0 0x7f74380a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:31.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.784+0000 7f74497fa640 1 -- 172.21.15.120:0/1220071885 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f743c0cefb0 con 0x7f744c100430 2024-09-19T09:01:31.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.784+0000 7f74497fa640 1 --2- 172.21.15.120:0/1220071885 >> [v2:172.21.15.120:6802/4131180019,v1:172.21.15.120:6803/4131180019] conn(0x7f74380acb20 0x7f74380aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:31.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.784+0000 7f74497fa640 1 -- 172.21.15.120:0/1220071885 --> [v2:172.21.15.120:6802/4131180019,v1:172.21.15.120:6803/4131180019] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f743c00dec0 con 0x7f74380acb20 2024-09-19T09:01:31.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.784+0000 7f74497fa640 1 -- 172.21.15.120:0/1220071885 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f743c0cf490 con 0x7f744c100430 2024-09-19T09:01:31.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.784+0000 7f744bfff640 1 --2- 172.21.15.120:0/1220071885 >> [v2:172.21.15.120:6802/4131180019,v1:172.21.15.120:6803/4131180019] conn(0x7f74380acb20 0x7f74380aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:31.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.785+0000 7f7443fff640 1 --2- 172.21.15.120:0/1220071885 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f74380a32d0 0x7f74380a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7434009230 tx=0x7f74340058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:31.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.785+0000 7f744bfff640 1 --2- 172.21.15.120:0/1220071885 >> [v2:172.21.15.120:6802/4131180019,v1:172.21.15.120:6803/4131180019] conn(0x7f74380acb20 0x7f74380aef40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:31.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.785+0000 7f74497fa640 1 -- 172.21.15.120:0/1220071885 <== osd.0 v2:172.21.15.120:6802/4131180019 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f743c00dec0 con 0x7f74380acb20 2024-09-19T09:01:31.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.799+0000 7f7451d37640 1 -- 172.21.15.120:0/1220071885 --> [v2:172.21.15.120:6802/4131180019,v1:172.21.15.120:6803/4131180019] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f744c062dc0 con 0x7f74380acb20 2024-09-19T09:01:31.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.799+0000 7f74497fa640 1 -- 172.21.15.120:0/1220071885 <== osd.0 v2:172.21.15.120:6802/4131180019 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f744c062dc0 con 0x7f74380acb20 2024-09-19T09:01:31.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.799+0000 7f7451d37640 1 -- 172.21.15.120:0/1220071885 >> [v2:172.21.15.120:6802/4131180019,v1:172.21.15.120:6803/4131180019] conn(0x7f74380acb20 msgr2=0x7f74380aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:31.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.799+0000 7f7451d37640 1 --2- 172.21.15.120:0/1220071885 >> [v2:172.21.15.120:6802/4131180019,v1:172.21.15.120:6803/4131180019] conn(0x7f74380acb20 0x7f74380aef40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:31.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.799+0000 7f7451d37640 1 -- 172.21.15.120:0/1220071885 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f74380a32d0 msgr2=0x7f74380a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:31.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.799+0000 7f7451d37640 1 --2- 172.21.15.120:0/1220071885 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f74380a32d0 0x7f74380a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7434009230 tx=0x7f74340058d0 comp rx=0 tx=0).stop 2024-09-19T09:01:31.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.799+0000 7f7451d37640 1 -- 172.21.15.120:0/1220071885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f744c100430 msgr2=0x7f744c1984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:31.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.799+0000 7f7451d37640 1 --2- 172.21.15.120:0/1220071885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f744c100430 0x7f744c1984e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f743c00b4a0 tx=0x7f743c00b960 comp rx=0 tx=0).stop 2024-09-19T09:01:31.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.799+0000 7f744bfff640 1 -- 172.21.15.120:0/1220071885 reap_dead start 2024-09-19T09:01:31.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.799+0000 7f7451d37640 1 -- 172.21.15.120:0/1220071885 shutdown_connections 2024-09-19T09:01:31.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.799+0000 7f7451d37640 1 -- 172.21.15.120:0/1220071885 >> 172.21.15.120:0/1220071885 conn(0x7f744c077230 msgr2=0x7f744c0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:31.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.799+0000 7f7451d37640 1 -- 172.21.15.120:0/1220071885 shutdown_connections 2024-09-19T09:01:31.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:31.800+0000 7f7451d37640 1 -- 172.21.15.120:0/1220071885 wait complete. 2024-09-19T09:01:32.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.222+0000 7f24ad86f640 1 -- 172.21.15.120:0/2692235518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24a8100d20 msgr2=0x7f24a8106860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:32.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.222+0000 7f24ad86f640 1 --2- 172.21.15.120:0/2692235518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24a8100d20 0x7f24a8106860 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2490009930 tx=0x7f249002a420 comp rx=0 tx=0).stop 2024-09-19T09:01:32.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.222+0000 7f24ad86f640 1 -- 172.21.15.120:0/2692235518 shutdown_connections 2024-09-19T09:01:32.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.222+0000 7f24ad86f640 1 --2- 172.21.15.120:0/2692235518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24a8106fc0 0x7f24a810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:32.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.222+0000 7f24ad86f640 1 --2- 172.21.15.120:0/2692235518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24a8100d20 0x7f24a8106860 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:32.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.222+0000 7f24ad86f640 1 --2- 172.21.15.120:0/2692235518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24a8100370 0x7f24a8100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:32.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.222+0000 7f24ad86f640 1 -- 172.21.15.120:0/2692235518 >> 172.21.15.120:0/2692235518 conn(0x7f24a8077210 msgr2=0x7f24a80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:32.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.222+0000 7f24ad86f640 1 -- 172.21.15.120:0/2692235518 shutdown_connections 2024-09-19T09:01:32.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.222+0000 7f24ad86f640 1 -- 172.21.15.120:0/2692235518 wait complete. 2024-09-19T09:01:32.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.223+0000 7f24ad86f640 1 Processor -- start 2024-09-19T09:01:32.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.223+0000 7f24ad86f640 1 -- start start 2024-09-19T09:01:32.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.223+0000 7f24ad86f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24a8100370 0x7f24a81012f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:32.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.223+0000 7f24ad86f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24a8100d20 0x7f24a8101830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:32.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.223+0000 7f24ad86f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24a8106fc0 0x7f24a8101d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:32.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.223+0000 7f24ad86f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f24a8111fa0 con 0x7f24a8100370 2024-09-19T09:01:32.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.223+0000 7f24ad86f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f24a8111e20 con 0x7f24a8100d20 2024-09-19T09:01:32.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.223+0000 7f24ad86f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f24a8112120 con 0x7f24a8106fc0 2024-09-19T09:01:32.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.223+0000 7f24a6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24a8100370 0x7f24a81012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:32.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.223+0000 7f24a6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24a8100370 0x7f24a81012f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36956/0 (socket says 172.21.15.120:36956) 2024-09-19T09:01:32.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.223+0000 7f24a6ffd640 1 -- 172.21.15.120:0/1961977629 learned_addr learned my addr 172.21.15.120:0/1961977629 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:32.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.223+0000 7f24a77fe640 1 --2- 172.21.15.120:0/1961977629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24a8106fc0 0x7f24a8101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:32.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.223+0000 7f249e7fc640 1 --2- 172.21.15.120:0/1961977629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24a8100d20 0x7f24a8101830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:32.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.224+0000 7f24a6ffd640 1 -- 172.21.15.120:0/1961977629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24a8106fc0 msgr2=0x7f24a8101d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:32.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.224+0000 7f24a6ffd640 1 --2- 172.21.15.120:0/1961977629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24a8106fc0 0x7f24a8101d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:32.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.224+0000 7f24a6ffd640 1 -- 172.21.15.120:0/1961977629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24a8100d20 msgr2=0x7f24a8101830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:32.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.224+0000 7f24a6ffd640 1 --2- 172.21.15.120:0/1961977629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24a8100d20 0x7f24a8101830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:32.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.224+0000 7f24a6ffd640 1 -- 172.21.15.120:0/1961977629 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a81a6900 con 0x7f24a8100370 2024-09-19T09:01:32.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.224+0000 7f24a6ffd640 1 --2- 172.21.15.120:0/1961977629 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24a8100370 0x7f24a81012f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f249800d950 tx=0x7f249800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:32.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.224+0000 7f24ac86d640 1 -- 172.21.15.120:0/1961977629 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2498017070 con 0x7f24a8100370 2024-09-19T09:01:32.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.224+0000 7f24ac86d640 1 -- 172.21.15.120:0/1961977629 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2498009c30 con 0x7f24a8100370 2024-09-19T09:01:32.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.224+0000 7f24ad86f640 1 -- 172.21.15.120:0/1961977629 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a81a6bf0 con 0x7f24a8100370 2024-09-19T09:01:32.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.224+0000 7f24ac86d640 1 -- 172.21.15.120:0/1961977629 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249800f6b0 con 0x7f24a8100370 2024-09-19T09:01:32.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.224+0000 7f24ad86f640 1 -- 172.21.15.120:0/1961977629 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a81a7100 con 0x7f24a8100370 2024-09-19T09:01:32.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.225+0000 7f24ad86f640 1 -- 172.21.15.120:0/1961977629 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f24a8062940 con 0x7f24a8100370 2024-09-19T09:01:32.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.226+0000 7f24ac86d640 1 -- 172.21.15.120:0/1961977629 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f24980109d0 con 0x7f24a8100370 2024-09-19T09:01:32.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.227+0000 7f24ac86d640 1 --2- 172.21.15.120:0/1961977629 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f24940a32d0 0x7f24940a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:32.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.227+0000 7f249e7fc640 1 --2- 172.21.15.120:0/1961977629 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f24940a32d0 0x7f24940a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:32.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.227+0000 7f24ac86d640 1 -- 172.21.15.120:0/1961977629 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f24980cf490 con 0x7f24a8100370 2024-09-19T09:01:32.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.227+0000 7f24ac86d640 1 --2- 172.21.15.120:0/1961977629 >> [v2:172.21.15.196:6808/3763331116,v1:172.21.15.196:6809/3763331116] conn(0x7f24940acb20 0x7f24940aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:32.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.227+0000 7f24ac86d640 1 -- 172.21.15.120:0/1961977629 --> [v2:172.21.15.196:6808/3763331116,v1:172.21.15.196:6809/3763331116] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2498004470 con 0x7f24940acb20 2024-09-19T09:01:32.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.227+0000 7f24ac86d640 1 -- 172.21.15.120:0/1961977629 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f24980d6050 con 0x7f24a8100370 2024-09-19T09:01:32.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.227+0000 7f249e7fc640 1 --2- 172.21.15.120:0/1961977629 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f24940a32d0 0x7f24940a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2490004770 tx=0x7f24900058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:32.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.227+0000 7f24a77fe640 1 --2- 172.21.15.120:0/1961977629 >> [v2:172.21.15.196:6808/3763331116,v1:172.21.15.196:6809/3763331116] conn(0x7f24940acb20 0x7f24940aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:32.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.228+0000 7f24a77fe640 1 --2- 172.21.15.120:0/1961977629 >> [v2:172.21.15.196:6808/3763331116,v1:172.21.15.196:6809/3763331116] conn(0x7f24940acb20 0x7f24940aef40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:32.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.229+0000 7f24ac86d640 1 -- 172.21.15.120:0/1961977629 <== osd.5 v2:172.21.15.196:6808/3763331116 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2498004470 con 0x7f24940acb20 2024-09-19T09:01:32.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.242+0000 7f24ad86f640 1 -- 172.21.15.120:0/1961977629 --> [v2:172.21.15.196:6808/3763331116,v1:172.21.15.196:6809/3763331116] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f24a8062dc0 con 0x7f24940acb20 2024-09-19T09:01:32.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.243+0000 7f24ac86d640 1 -- 172.21.15.120:0/1961977629 <== osd.5 v2:172.21.15.196:6808/3763331116 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f24a8062dc0 con 0x7f24940acb20 2024-09-19T09:01:32.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.243+0000 7f24ad86f640 1 -- 172.21.15.120:0/1961977629 >> [v2:172.21.15.196:6808/3763331116,v1:172.21.15.196:6809/3763331116] conn(0x7f24940acb20 msgr2=0x7f24940aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:32.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.243+0000 7f24ad86f640 1 --2- 172.21.15.120:0/1961977629 >> [v2:172.21.15.196:6808/3763331116,v1:172.21.15.196:6809/3763331116] conn(0x7f24940acb20 0x7f24940aef40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:32.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.243+0000 7f24ad86f640 1 -- 172.21.15.120:0/1961977629 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f24940a32d0 msgr2=0x7f24940a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:32.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.243+0000 7f24ad86f640 1 --2- 172.21.15.120:0/1961977629 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f24940a32d0 0x7f24940a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2490004770 tx=0x7f24900058d0 comp rx=0 tx=0).stop 2024-09-19T09:01:32.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.243+0000 7f24ad86f640 1 -- 172.21.15.120:0/1961977629 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24a8100370 msgr2=0x7f24a81012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:32.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.243+0000 7f24ad86f640 1 --2- 172.21.15.120:0/1961977629 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24a8100370 0x7f24a81012f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f249800d950 tx=0x7f249800de10 comp rx=0 tx=0).stop 2024-09-19T09:01:32.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.243+0000 7f24a77fe640 1 -- 172.21.15.120:0/1961977629 reap_dead start 2024-09-19T09:01:32.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.243+0000 7f24ad86f640 1 -- 172.21.15.120:0/1961977629 shutdown_connections 2024-09-19T09:01:32.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.243+0000 7f24ad86f640 1 -- 172.21.15.120:0/1961977629 >> 172.21.15.120:0/1961977629 conn(0x7f24a8077210 msgr2=0x7f24a8105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:32.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.243+0000 7f24ad86f640 1 -- 172.21.15.120:0/1961977629 shutdown_connections 2024-09-19T09:01:32.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.243+0000 7f24ad86f640 1 -- 172.21.15.120:0/1961977629 wait complete. 2024-09-19T09:01:32.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:32 smithi120 ceph-mon[31150]: pgmap v299: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T09:01:33.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.990+0000 7fbb96859640 1 -- 172.21.15.120:0/3227145362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb90102570 msgr2=0x7fbb901029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:33.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.990+0000 7fbb96859640 1 --2- 172.21.15.120:0/3227145362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb90102570 0x7fbb901029b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009900 tx=0x7fbb8402a480 comp rx=0 tx=0).stop 2024-09-19T09:01:33.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.990+0000 7fbb96859640 1 -- 172.21.15.120:0/3227145362 shutdown_connections 2024-09-19T09:01:33.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.990+0000 7fbb96859640 1 --2- 172.21.15.120:0/3227145362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb90102ef0 0x7fbb9010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:33.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.990+0000 7fbb96859640 1 --2- 172.21.15.120:0/3227145362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb90102570 0x7fbb901029b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:33.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.990+0000 7fbb96859640 1 --2- 172.21.15.120:0/3227145362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb90100e20 0x7fbb90101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:33.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.990+0000 7fbb96859640 1 -- 172.21.15.120:0/3227145362 >> 172.21.15.120:0/3227145362 conn(0x7fbb900fc7d0 msgr2=0x7fbb900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:33.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.990+0000 7fbb96859640 1 -- 172.21.15.120:0/3227145362 shutdown_connections 2024-09-19T09:01:33.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.990+0000 7fbb96859640 1 -- 172.21.15.120:0/3227145362 wait complete. 2024-09-19T09:01:33.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.991+0000 7fbb96859640 1 Processor -- start 2024-09-19T09:01:33.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.991+0000 7fbb96859640 1 -- start start 2024-09-19T09:01:33.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.991+0000 7fbb96859640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb90100e20 0x7fbb900fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:33.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.991+0000 7fbb96859640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb90102570 0x7fbb900fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:33.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.991+0000 7fbb96859640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb90102ef0 0x7fbb900fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:33.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.991+0000 7fbb96859640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbb90110f80 con 0x7fbb90100e20 2024-09-19T09:01:33.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.991+0000 7fbb96859640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbb90110e00 con 0x7fbb90102570 2024-09-19T09:01:33.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.991+0000 7fbb96859640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbb90111100 con 0x7fbb90102ef0 2024-09-19T09:01:33.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.991+0000 7fbb8ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb90100e20 0x7fbb900fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:33.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.992+0000 7fbb8ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb90100e20 0x7fbb900fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36964/0 (socket says 172.21.15.120:36964) 2024-09-19T09:01:33.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.992+0000 7fbb8ffff640 1 -- 172.21.15.120:0/1626506634 learned_addr learned my addr 172.21.15.120:0/1626506634 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:33.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.992+0000 7fbb94df6640 1 --2- 172.21.15.120:0/1626506634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb90102ef0 0x7fbb900fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:33.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.992+0000 7fbb8f7fe640 1 --2- 172.21.15.120:0/1626506634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb90102570 0x7fbb900fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:33.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.992+0000 7fbb8ffff640 1 -- 172.21.15.120:0/1626506634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb90102ef0 msgr2=0x7fbb900fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:33.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.992+0000 7fbb8ffff640 1 --2- 172.21.15.120:0/1626506634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb90102ef0 0x7fbb900fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:33.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.992+0000 7fbb8ffff640 1 -- 172.21.15.120:0/1626506634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb90102570 msgr2=0x7fbb900fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:33.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.992+0000 7fbb8ffff640 1 --2- 172.21.15.120:0/1626506634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb90102570 0x7fbb900fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:33.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.992+0000 7fbb8ffff640 1 -- 172.21.15.120:0/1626506634 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb901a6a50 con 0x7fbb90100e20 2024-09-19T09:01:33.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.992+0000 7fbb8ffff640 1 --2- 172.21.15.120:0/1626506634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb90100e20 0x7fbb900fcc80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000d950 tx=0x7fbb8000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:33.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.992+0000 7fbb8d7fa640 1 -- 172.21.15.120:0/1626506634 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb80017070 con 0x7fbb90100e20 2024-09-19T09:01:33.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.992+0000 7fbb96859640 1 -- 172.21.15.120:0/1626506634 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb901a6d40 con 0x7fbb90100e20 2024-09-19T09:01:33.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.992+0000 7fbb8d7fa640 1 -- 172.21.15.120:0/1626506634 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb80009c30 con 0x7fbb90100e20 2024-09-19T09:01:33.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.992+0000 7fbb8d7fa640 1 -- 172.21.15.120:0/1626506634 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8000f6b0 con 0x7fbb90100e20 2024-09-19T09:01:33.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.992+0000 7fbb96859640 1 -- 172.21.15.120:0/1626506634 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb901a71f0 con 0x7fbb90100e20 2024-09-19T09:01:33.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.993+0000 7fbb96859640 1 -- 172.21.15.120:0/1626506634 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbb900628c0 con 0x7fbb90100e20 2024-09-19T09:01:33.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.994+0000 7fbb8d7fa640 1 -- 172.21.15.120:0/1626506634 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fbb800109d0 con 0x7fbb90100e20 2024-09-19T09:01:33.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.995+0000 7fbb8d7fa640 1 --2- 172.21.15.120:0/1626506634 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbb780a32d0 0x7fbb780a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:33.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.995+0000 7fbb8f7fe640 1 --2- 172.21.15.120:0/1626506634 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbb780a32d0 0x7fbb780a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:33.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.995+0000 7fbb8d7fa640 1 -- 172.21.15.120:0/1626506634 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fbb800cf870 con 0x7fbb90100e20 2024-09-19T09:01:33.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.995+0000 7fbb8d7fa640 1 --2- 172.21.15.120:0/1626506634 >> [v2:172.21.15.196:6824/1857200740,v1:172.21.15.196:6825/1857200740] conn(0x7fbb780acb20 0x7fbb780aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:33.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.995+0000 7fbb8d7fa640 1 -- 172.21.15.120:0/1626506634 --> [v2:172.21.15.196:6824/1857200740,v1:172.21.15.196:6825/1857200740] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbb780af5f0 con 0x7fbb780acb20 2024-09-19T09:01:33.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.995+0000 7fbb8d7fa640 1 -- 172.21.15.120:0/1626506634 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fbb800cfd50 con 0x7fbb90100e20 2024-09-19T09:01:33.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.996+0000 7fbb8f7fe640 1 --2- 172.21.15.120:0/1626506634 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbb780a32d0 0x7fbb780a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009230 tx=0x7fbb840058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:33.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.996+0000 7fbb94df6640 1 --2- 172.21.15.120:0/1626506634 >> [v2:172.21.15.196:6824/1857200740,v1:172.21.15.196:6825/1857200740] conn(0x7fbb780acb20 0x7fbb780aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:33.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.996+0000 7fbb94df6640 1 --2- 172.21.15.120:0/1626506634 >> [v2:172.21.15.196:6824/1857200740,v1:172.21.15.196:6825/1857200740] conn(0x7fbb780acb20 0x7fbb780aef40 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:33.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:32.997+0000 7fbb8d7fa640 1 -- 172.21.15.120:0/1626506634 <== osd.11 v2:172.21.15.196:6824/1857200740 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbb780af5f0 con 0x7fbb780acb20 2024-09-19T09:01:33.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.010+0000 7fbb96859640 1 -- 172.21.15.120:0/1626506634 --> [v2:172.21.15.196:6824/1857200740,v1:172.21.15.196:6825/1857200740] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbb90062cc0 con 0x7fbb780acb20 2024-09-19T09:01:33.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.010+0000 7fbb8d7fa640 1 -- 172.21.15.120:0/1626506634 <== osd.11 v2:172.21.15.196:6824/1857200740 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbb90062cc0 con 0x7fbb780acb20 2024-09-19T09:01:33.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.011+0000 7fbb96859640 1 -- 172.21.15.120:0/1626506634 >> [v2:172.21.15.196:6824/1857200740,v1:172.21.15.196:6825/1857200740] conn(0x7fbb780acb20 msgr2=0x7fbb780aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:33.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.011+0000 7fbb96859640 1 --2- 172.21.15.120:0/1626506634 >> [v2:172.21.15.196:6824/1857200740,v1:172.21.15.196:6825/1857200740] conn(0x7fbb780acb20 0x7fbb780aef40 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:33.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.011+0000 7fbb96859640 1 -- 172.21.15.120:0/1626506634 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbb780a32d0 msgr2=0x7fbb780a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:33.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.011+0000 7fbb96859640 1 --2- 172.21.15.120:0/1626506634 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbb780a32d0 0x7fbb780a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009230 tx=0x7fbb840058d0 comp rx=0 tx=0).stop 2024-09-19T09:01:33.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.011+0000 7fbb96859640 1 -- 172.21.15.120:0/1626506634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb90100e20 msgr2=0x7fbb900fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:33.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.011+0000 7fbb96859640 1 --2- 172.21.15.120:0/1626506634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb90100e20 0x7fbb900fcc80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000d950 tx=0x7fbb8000de10 comp rx=0 tx=0).stop 2024-09-19T09:01:33.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.011+0000 7fbb94df6640 1 -- 172.21.15.120:0/1626506634 reap_dead start 2024-09-19T09:01:33.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.011+0000 7fbb96859640 1 -- 172.21.15.120:0/1626506634 shutdown_connections 2024-09-19T09:01:33.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.011+0000 7fbb96859640 1 -- 172.21.15.120:0/1626506634 >> 172.21.15.120:0/1626506634 conn(0x7fbb900fc7d0 msgr2=0x7fbb9006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:33.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.011+0000 7fbb96859640 1 -- 172.21.15.120:0/1626506634 shutdown_connections 2024-09-19T09:01:33.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.011+0000 7fbb96859640 1 -- 172.21.15.120:0/1626506634 wait complete. 2024-09-19T09:01:33.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:32 smithi196 ceph-mon[35150]: pgmap v299: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T09:01:33.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:32 smithi153 ceph-mon[35071]: pgmap v299: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T09:01:33.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.937+0000 7f267485b640 1 -- 172.21.15.120:0/3194631320 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2670101de0 msgr2=0x7f26701021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:33.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.937+0000 7f267485b640 1 --2- 172.21.15.120:0/3194631320 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2670101de0 0x7f26701021c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2660009900 tx=0x7f266002a410 comp rx=0 tx=0).stop 2024-09-19T09:01:33.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.937+0000 7f267485b640 1 -- 172.21.15.120:0/3194631320 shutdown_connections 2024-09-19T09:01:33.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.937+0000 7f267485b640 1 --2- 172.21.15.120:0/3194631320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2670106f10 0x7f267010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:33.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.937+0000 7f267485b640 1 --2- 172.21.15.120:0/3194631320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2670102790 0x7f26701067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:33.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.937+0000 7f267485b640 1 --2- 172.21.15.120:0/3194631320 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2670101de0 0x7f26701021c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:33.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.937+0000 7f267485b640 1 -- 172.21.15.120:0/3194631320 >> 172.21.15.120:0/3194631320 conn(0x7f26700fd8c0 msgr2=0x7f26700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:33.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.937+0000 7f267485b640 1 -- 172.21.15.120:0/3194631320 shutdown_connections 2024-09-19T09:01:33.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.937+0000 7f267485b640 1 -- 172.21.15.120:0/3194631320 wait complete. 2024-09-19T09:01:33.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.938+0000 7f267485b640 1 Processor -- start 2024-09-19T09:01:33.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.938+0000 7f267485b640 1 -- start start 2024-09-19T09:01:33.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.938+0000 7f267485b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2670101de0 0x7f2670198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:33.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.938+0000 7f267485b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2670102790 0x7f2670198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:33.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.938+0000 7f267485b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2670106f10 0x7f267019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:33.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.938+0000 7f267485b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2670111f90 con 0x7f2670101de0 2024-09-19T09:01:33.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.938+0000 7f267485b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2670111e10 con 0x7f2670106f10 2024-09-19T09:01:33.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.938+0000 7f267485b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2670112110 con 0x7f2670102790 2024-09-19T09:01:33.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.938+0000 7f266e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2670101de0 0x7f2670198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:33.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.938+0000 7f266e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2670101de0 0x7f2670198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36978/0 (socket says 172.21.15.120:36978) 2024-09-19T09:01:33.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.938+0000 7f266e59c640 1 -- 172.21.15.120:0/3240261774 learned_addr learned my addr 172.21.15.120:0/3240261774 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:33.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.939+0000 7f266dd9b640 1 --2- 172.21.15.120:0/3240261774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2670102790 0x7f2670198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:33.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.939+0000 7f266e59c640 1 -- 172.21.15.120:0/3240261774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2670102790 msgr2=0x7f2670198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:33.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.939+0000 7f266e59c640 1 --2- 172.21.15.120:0/3240261774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2670102790 0x7f2670198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:33.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.939+0000 7f266e59c640 1 -- 172.21.15.120:0/3240261774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2670106f10 msgr2=0x7f267019cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:33.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.939+0000 7f266e59c640 1 --2- 172.21.15.120:0/3240261774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2670106f10 0x7f267019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:33.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.939+0000 7f266e59c640 1 -- 172.21.15.120:0/3240261774 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267019d4c0 con 0x7f2670101de0 2024-09-19T09:01:33.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.939+0000 7f266dd9b640 1 --2- 172.21.15.120:0/3240261774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2670102790 0x7f2670198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:01:33.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.939+0000 7f266e59c640 1 --2- 172.21.15.120:0/3240261774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2670101de0 0x7f2670198510 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2660006010 tx=0x7f26600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:33.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.939+0000 7f265f7fe640 1 -- 172.21.15.120:0/3240261774 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266003f030 con 0x7f2670101de0 2024-09-19T09:01:33.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.939+0000 7f265f7fe640 1 -- 172.21.15.120:0/3240261774 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2660003ce0 con 0x7f2670101de0 2024-09-19T09:01:33.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.939+0000 7f267485b640 1 -- 172.21.15.120:0/3240261774 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f267019d750 con 0x7f2670101de0 2024-09-19T09:01:33.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.939+0000 7f265f7fe640 1 -- 172.21.15.120:0/3240261774 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2660035070 con 0x7f2670101de0 2024-09-19T09:01:33.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.939+0000 7f267485b640 1 -- 172.21.15.120:0/3240261774 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26701a5090 con 0x7f2670101de0 2024-09-19T09:01:33.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.940+0000 7f267485b640 1 -- 172.21.15.120:0/3240261774 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2670062980 con 0x7f2670101de0 2024-09-19T09:01:33.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.941+0000 7f265f7fe640 1 -- 172.21.15.120:0/3240261774 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f2660042030 con 0x7f2670101de0 2024-09-19T09:01:33.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.942+0000 7f265f7fe640 1 --2- 172.21.15.120:0/3240261774 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f26580a32d0 0x7f26580a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:33.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.942+0000 7f266dd9b640 1 --2- 172.21.15.120:0/3240261774 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f26580a32d0 0x7f26580a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:33.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.942+0000 7f265f7fe640 1 -- 172.21.15.120:0/3240261774 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f26600f0060 con 0x7f2670101de0 2024-09-19T09:01:33.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.942+0000 7f265f7fe640 1 --2- 172.21.15.120:0/3240261774 >> [v2:172.21.15.120:6810/713205668,v1:172.21.15.120:6811/713205668] conn(0x7f26580acb20 0x7f26580aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:33.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.942+0000 7f265f7fe640 1 -- 172.21.15.120:0/3240261774 --> [v2:172.21.15.120:6810/713205668,v1:172.21.15.120:6811/713205668] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f266003be50 con 0x7f26580acb20 2024-09-19T09:01:33.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.942+0000 7f265f7fe640 1 -- 172.21.15.120:0/3240261774 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f26600f7050 con 0x7f2670101de0 2024-09-19T09:01:33.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.942+0000 7f266ed9d640 1 --2- 172.21.15.120:0/3240261774 >> [v2:172.21.15.120:6810/713205668,v1:172.21.15.120:6811/713205668] conn(0x7f26580acb20 0x7f26580aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:33.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.942+0000 7f266dd9b640 1 --2- 172.21.15.120:0/3240261774 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f26580a32d0 0x7f26580a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2664006fd0 tx=0x7f2664008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:33.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.942+0000 7f266ed9d640 1 --2- 172.21.15.120:0/3240261774 >> [v2:172.21.15.120:6810/713205668,v1:172.21.15.120:6811/713205668] conn(0x7f26580acb20 0x7f26580aef40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:33.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.943+0000 7f265f7fe640 1 -- 172.21.15.120:0/3240261774 <== osd.3 v2:172.21.15.120:6810/713205668 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f266003be50 con 0x7f26580acb20 2024-09-19T09:01:33.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.956+0000 7f267485b640 1 -- 172.21.15.120:0/3240261774 --> [v2:172.21.15.120:6810/713205668,v1:172.21.15.120:6811/713205668] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2670062e00 con 0x7f26580acb20 2024-09-19T09:01:33.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.956+0000 7f265f7fe640 1 -- 172.21.15.120:0/3240261774 <== osd.3 v2:172.21.15.120:6810/713205668 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2670062e00 con 0x7f26580acb20 2024-09-19T09:01:33.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.957+0000 7f267485b640 1 -- 172.21.15.120:0/3240261774 >> [v2:172.21.15.120:6810/713205668,v1:172.21.15.120:6811/713205668] conn(0x7f26580acb20 msgr2=0x7f26580aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:33.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.957+0000 7f267485b640 1 --2- 172.21.15.120:0/3240261774 >> [v2:172.21.15.120:6810/713205668,v1:172.21.15.120:6811/713205668] conn(0x7f26580acb20 0x7f26580aef40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:33.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.957+0000 7f267485b640 1 -- 172.21.15.120:0/3240261774 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f26580a32d0 msgr2=0x7f26580a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:33.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.957+0000 7f267485b640 1 --2- 172.21.15.120:0/3240261774 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f26580a32d0 0x7f26580a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2664006fd0 tx=0x7f2664008040 comp rx=0 tx=0).stop 2024-09-19T09:01:33.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.957+0000 7f267485b640 1 -- 172.21.15.120:0/3240261774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2670101de0 msgr2=0x7f2670198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:33.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.957+0000 7f267485b640 1 --2- 172.21.15.120:0/3240261774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2670101de0 0x7f2670198510 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2660006010 tx=0x7f26600042f0 comp rx=0 tx=0).stop 2024-09-19T09:01:33.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.957+0000 7f266ed9d640 1 -- 172.21.15.120:0/3240261774 reap_dead start 2024-09-19T09:01:33.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.957+0000 7f267485b640 1 -- 172.21.15.120:0/3240261774 shutdown_connections 2024-09-19T09:01:33.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.957+0000 7f267485b640 1 -- 172.21.15.120:0/3240261774 >> 172.21.15.120:0/3240261774 conn(0x7f26700fd8c0 msgr2=0x7f26700fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:33.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.957+0000 7f267485b640 1 -- 172.21.15.120:0/3240261774 shutdown_connections 2024-09-19T09:01:33.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:33.957+0000 7f267485b640 1 -- 172.21.15.120:0/3240261774 wait complete. 2024-09-19T09:01:34.649 INFO:teuthology.orchestra.run.smithi120.stdout:236223201317 2024-09-19T09:01:34.649 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-19T09:01:34.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:34 smithi120 ceph-mon[31150]: pgmap v300: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T09:01:35.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:34 smithi196 ceph-mon[35150]: pgmap v300: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T09:01:35.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:34 smithi153 ceph-mon[35071]: pgmap v300: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T09:01:35.192 INFO:teuthology.orchestra.run.smithi120.stdout:283467841562 2024-09-19T09:01:35.192 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-19T09:01:35.500 INFO:teuthology.orchestra.run.smithi120.stdout:313532612630 2024-09-19T09:01:35.501 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-19T09:01:35.913 INFO:teuthology.orchestra.run.smithi120.stdout:103079215167 2024-09-19T09:01:35.914 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-19T09:01:36.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.063+0000 7f35fe5ad640 1 -- 172.21.15.120:0/3008588282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35f8102570 msgr2=0x7f35f81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:36.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.063+0000 7f35fe5ad640 1 --2- 172.21.15.120:0/3008588282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35f8102570 0x7f35f81029b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f35ec009900 tx=0x7f35ec02a480 comp rx=0 tx=0).stop 2024-09-19T09:01:36.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.063+0000 7f35fe5ad640 1 -- 172.21.15.120:0/3008588282 shutdown_connections 2024-09-19T09:01:36.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.063+0000 7f35fe5ad640 1 --2- 172.21.15.120:0/3008588282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35f8102ef0 0x7f35f810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:36.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.063+0000 7f35fe5ad640 1 --2- 172.21.15.120:0/3008588282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35f8102570 0x7f35f81029b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:36.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.063+0000 7f35fe5ad640 1 --2- 172.21.15.120:0/3008588282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35f8100e20 0x7f35f8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:36.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.063+0000 7f35fe5ad640 1 -- 172.21.15.120:0/3008588282 >> 172.21.15.120:0/3008588282 conn(0x7f35f80fc7d0 msgr2=0x7f35f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:36.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.063+0000 7f35fe5ad640 1 -- 172.21.15.120:0/3008588282 shutdown_connections 2024-09-19T09:01:36.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.063+0000 7f35fe5ad640 1 -- 172.21.15.120:0/3008588282 wait complete. 2024-09-19T09:01:36.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.064+0000 7f35fe5ad640 1 Processor -- start 2024-09-19T09:01:36.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.064+0000 7f35fe5ad640 1 -- start start 2024-09-19T09:01:36.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.065+0000 7f35fe5ad640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35f8100e20 0x7f35f8199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:36.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.065+0000 7f35fe5ad640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35f8102570 0x7f35f81998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:36.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.065+0000 7f35fe5ad640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35f8102ef0 0x7f35f81a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:36.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.065+0000 7f35fe5ad640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f35f8110f80 con 0x7f35f8100e20 2024-09-19T09:01:36.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.065+0000 7f35fe5ad640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f35f8110e00 con 0x7f35f8102570 2024-09-19T09:01:36.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.065+0000 7f35fe5ad640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f35f8111100 con 0x7f35f8102ef0 2024-09-19T09:01:36.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.065+0000 7f35f7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35f8100e20 0x7f35f8199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:36.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.065+0000 7f35f7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35f8100e20 0x7f35f8199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37006/0 (socket says 172.21.15.120:37006) 2024-09-19T09:01:36.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.065+0000 7f35f7fff640 1 -- 172.21.15.120:0/3520111294 learned_addr learned my addr 172.21.15.120:0/3520111294 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:36.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.065+0000 7f35f77fe640 1 --2- 172.21.15.120:0/3520111294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35f8102570 0x7f35f81998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:36.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.065+0000 7f35fcb4a640 1 --2- 172.21.15.120:0/3520111294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35f8102ef0 0x7f35f81a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:36.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.065+0000 7f35f7fff640 1 -- 172.21.15.120:0/3520111294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35f8102ef0 msgr2=0x7f35f81a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:36.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.065+0000 7f35f7fff640 1 --2- 172.21.15.120:0/3520111294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35f8102ef0 0x7f35f81a4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:36.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.065+0000 7f35f7fff640 1 -- 172.21.15.120:0/3520111294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35f8102570 msgr2=0x7f35f81998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:36.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.065+0000 7f35f7fff640 1 --2- 172.21.15.120:0/3520111294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35f8102570 0x7f35f81998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:36.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.065+0000 7f35f7fff640 1 -- 172.21.15.120:0/3520111294 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f81a4950 con 0x7f35f8100e20 2024-09-19T09:01:36.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.065+0000 7f35fcb4a640 1 --2- 172.21.15.120:0/3520111294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35f8102ef0 0x7f35f81a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:01:36.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.065+0000 7f35f7fff640 1 --2- 172.21.15.120:0/3520111294 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35f8100e20 0x7f35f8199390 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f35e800d770 tx=0x7f35e800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:36.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.066+0000 7f35f57fa640 1 -- 172.21.15.120:0/3520111294 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e8017070 con 0x7f35f8100e20 2024-09-19T09:01:36.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.066+0000 7f35fe5ad640 1 -- 172.21.15.120:0/3520111294 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f81a4b20 con 0x7f35f8100e20 2024-09-19T09:01:36.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.066+0000 7f35f57fa640 1 -- 172.21.15.120:0/3520111294 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35e8005500 con 0x7f35f8100e20 2024-09-19T09:01:36.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.066+0000 7f35f57fa640 1 -- 172.21.15.120:0/3520111294 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e8004e00 con 0x7f35f8100e20 2024-09-19T09:01:36.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.066+0000 7f35fe5ad640 1 -- 172.21.15.120:0/3520111294 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f81a4fe0 con 0x7f35f8100e20 2024-09-19T09:01:36.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.067+0000 7f35fe5ad640 1 -- 172.21.15.120:0/3520111294 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f35f8062900 con 0x7f35f8100e20 2024-09-19T09:01:36.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.068+0000 7f35f57fa640 1 -- 172.21.15.120:0/3520111294 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f35e8004fa0 con 0x7f35f8100e20 2024-09-19T09:01:36.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.068+0000 7f35f57fa640 1 --2- 172.21.15.120:0/3520111294 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f35e00a32d0 0x7f35e00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:36.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.069+0000 7f35f77fe640 1 --2- 172.21.15.120:0/3520111294 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f35e00a32d0 0x7f35e00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:36.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.069+0000 7f35f57fa640 1 -- 172.21.15.120:0/3520111294 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f35e80cf750 con 0x7f35f8100e20 2024-09-19T09:01:36.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.069+0000 7f35f57fa640 1 --2- 172.21.15.120:0/3520111294 >> [v2:172.21.15.196:6800/2942286570,v1:172.21.15.196:6801/2942286570] conn(0x7f35e00acb20 0x7f35e00aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:36.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.069+0000 7f35f57fa640 1 -- 172.21.15.120:0/3520111294 --> [v2:172.21.15.196:6800/2942286570,v1:172.21.15.196:6801/2942286570] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f35e8001d30 con 0x7f35e00acb20 2024-09-19T09:01:36.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.069+0000 7f35f57fa640 1 -- 172.21.15.120:0/3520111294 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f35e80cfc30 con 0x7f35f8100e20 2024-09-19T09:01:36.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.069+0000 7f35f77fe640 1 --2- 172.21.15.120:0/3520111294 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f35e00a32d0 0x7f35e00a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f35ec009230 tx=0x7f35ec0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:36.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.069+0000 7f35fcb4a640 1 --2- 172.21.15.120:0/3520111294 >> [v2:172.21.15.196:6800/2942286570,v1:172.21.15.196:6801/2942286570] conn(0x7f35e00acb20 0x7f35e00aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:36.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.069+0000 7f35fcb4a640 1 --2- 172.21.15.120:0/3520111294 >> [v2:172.21.15.196:6800/2942286570,v1:172.21.15.196:6801/2942286570] conn(0x7f35e00acb20 0x7f35e00aef40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:36.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.070+0000 7f35f57fa640 1 -- 172.21.15.120:0/3520111294 <== osd.2 v2:172.21.15.196:6800/2942286570 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f35e8001d30 con 0x7f35e00acb20 2024-09-19T09:01:36.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.084+0000 7f35fe5ad640 1 -- 172.21.15.120:0/3520111294 --> [v2:172.21.15.196:6800/2942286570,v1:172.21.15.196:6801/2942286570] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f35f8062d80 con 0x7f35e00acb20 2024-09-19T09:01:36.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.084+0000 7f35f57fa640 1 -- 172.21.15.120:0/3520111294 <== osd.2 v2:172.21.15.196:6800/2942286570 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f35f8062d80 con 0x7f35e00acb20 2024-09-19T09:01:36.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.085+0000 7f35fe5ad640 1 -- 172.21.15.120:0/3520111294 >> [v2:172.21.15.196:6800/2942286570,v1:172.21.15.196:6801/2942286570] conn(0x7f35e00acb20 msgr2=0x7f35e00aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:36.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.085+0000 7f35fe5ad640 1 --2- 172.21.15.120:0/3520111294 >> [v2:172.21.15.196:6800/2942286570,v1:172.21.15.196:6801/2942286570] conn(0x7f35e00acb20 0x7f35e00aef40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:36.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.085+0000 7f35fe5ad640 1 -- 172.21.15.120:0/3520111294 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f35e00a32d0 msgr2=0x7f35e00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:36.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.085+0000 7f35fe5ad640 1 --2- 172.21.15.120:0/3520111294 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f35e00a32d0 0x7f35e00a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f35ec009230 tx=0x7f35ec0058d0 comp rx=0 tx=0).stop 2024-09-19T09:01:36.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.085+0000 7f35fe5ad640 1 -- 172.21.15.120:0/3520111294 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35f8100e20 msgr2=0x7f35f8199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:36.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.085+0000 7f35fe5ad640 1 --2- 172.21.15.120:0/3520111294 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35f8100e20 0x7f35f8199390 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f35e800d770 tx=0x7f35e800dc30 comp rx=0 tx=0).stop 2024-09-19T09:01:36.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.085+0000 7f35fcb4a640 1 -- 172.21.15.120:0/3520111294 reap_dead start 2024-09-19T09:01:36.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.085+0000 7f35fe5ad640 1 -- 172.21.15.120:0/3520111294 shutdown_connections 2024-09-19T09:01:36.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.085+0000 7f35fe5ad640 1 -- 172.21.15.120:0/3520111294 >> 172.21.15.120:0/3520111294 conn(0x7f35f80fc7d0 msgr2=0x7f35f8105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:36.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.085+0000 7f35fe5ad640 1 -- 172.21.15.120:0/3520111294 shutdown_connections 2024-09-19T09:01:36.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:36.085+0000 7f35fe5ad640 1 -- 172.21.15.120:0/3520111294 wait complete. 2024-09-19T09:01:36.489 INFO:teuthology.orchestra.run.smithi120.stdout:154618822708 2024-09-19T09:01:36.489 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-19T09:01:36.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:36 smithi120 ceph-mon[31150]: pgmap v301: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T09:01:37.006 INFO:teuthology.orchestra.run.smithi120.stdout:34359738446 2024-09-19T09:01:37.007 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-19T09:01:37.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:36 smithi196 ceph-mon[35150]: pgmap v301: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T09:01:37.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:36 smithi153 ceph-mon[35071]: pgmap v301: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T09:01:37.492 INFO:teuthology.orchestra.run.smithi120.stdout:77309411397 2024-09-19T09:01:37.492 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-19T09:01:37.937 INFO:teuthology.orchestra.run.smithi120.stdout:133143986234 2024-09-19T09:01:37.937 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-19T09:01:38.003 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:38.368 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:38.369 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:38.369 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:38.369 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:38.373 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:38.374 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:38.376 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:38.376 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:38.386 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:38.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.630+0000 7fd24a0de640 1 -- 172.21.15.120:0/3527408104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd244102570 msgr2=0x7fd2441029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:38.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.630+0000 7fd24a0de640 1 --2- 172.21.15.120:0/3527408104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd244102570 0x7fd2441029b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd22c009900 tx=0x7fd22c02a480 comp rx=0 tx=0).stop 2024-09-19T09:01:38.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.630+0000 7fd24a0de640 1 -- 172.21.15.120:0/3527408104 shutdown_connections 2024-09-19T09:01:38.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.630+0000 7fd24a0de640 1 --2- 172.21.15.120:0/3527408104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd244102ef0 0x7fd24410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:38.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.630+0000 7fd24a0de640 1 --2- 172.21.15.120:0/3527408104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd244102570 0x7fd2441029b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:38.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.630+0000 7fd24a0de640 1 --2- 172.21.15.120:0/3527408104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd244100e20 0x7fd244101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:38.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.630+0000 7fd24a0de640 1 -- 172.21.15.120:0/3527408104 >> 172.21.15.120:0/3527408104 conn(0x7fd2440fc7d0 msgr2=0x7fd2440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:38.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.630+0000 7fd24a0de640 1 -- 172.21.15.120:0/3527408104 shutdown_connections 2024-09-19T09:01:38.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.630+0000 7fd24a0de640 1 -- 172.21.15.120:0/3527408104 wait complete. 2024-09-19T09:01:38.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.630+0000 7fd24a0de640 1 Processor -- start 2024-09-19T09:01:38.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.630+0000 7fd24a0de640 1 -- start start 2024-09-19T09:01:38.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.631+0000 7fd24a0de640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd244100e20 0x7fd2441993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:38.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.631+0000 7fd24a0de640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd244102570 0x7fd244199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:38.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.631+0000 7fd24a0de640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd244102ef0 0x7fd2441a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:38.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.631+0000 7fd24a0de640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd244110fa0 con 0x7fd244100e20 2024-09-19T09:01:38.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.631+0000 7fd24a0de640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd244110e20 con 0x7fd244102ef0 2024-09-19T09:01:38.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.631+0000 7fd24a0de640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd244111120 con 0x7fd244102570 2024-09-19T09:01:38.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.631+0000 7fd2437fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd244100e20 0x7fd2441993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:38.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.631+0000 7fd2437fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd244100e20 0x7fd2441993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37030/0 (socket says 172.21.15.120:37030) 2024-09-19T09:01:38.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.631+0000 7fd2437fe640 1 -- 172.21.15.120:0/3288312432 learned_addr learned my addr 172.21.15.120:0/3288312432 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:38.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.631+0000 7fd23affd640 1 --2- 172.21.15.120:0/3288312432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd244102570 0x7fd244199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:38.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.631+0000 7fd2437fe640 1 -- 172.21.15.120:0/3288312432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd244102570 msgr2=0x7fd244199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:38.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.631+0000 7fd2437fe640 1 --2- 172.21.15.120:0/3288312432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd244102570 0x7fd244199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:38.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.631+0000 7fd243fff640 1 --2- 172.21.15.120:0/3288312432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd244102ef0 0x7fd2441a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:38.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.631+0000 7fd2437fe640 1 -- 172.21.15.120:0/3288312432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd244102ef0 msgr2=0x7fd2441a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:38.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.631+0000 7fd2437fe640 1 --2- 172.21.15.120:0/3288312432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd244102ef0 0x7fd2441a4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:38.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.631+0000 7fd2437fe640 1 -- 172.21.15.120:0/3288312432 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2441a4990 con 0x7fd244100e20 2024-09-19T09:01:38.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.632+0000 7fd2437fe640 1 --2- 172.21.15.120:0/3288312432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd244100e20 0x7fd2441993d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd23400b4a0 tx=0x7fd23400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:38.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.632+0000 7fd2417fa640 1 -- 172.21.15.120:0/3288312432 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd234014070 con 0x7fd244100e20 2024-09-19T09:01:38.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.632+0000 7fd2417fa640 1 -- 172.21.15.120:0/3288312432 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd234010af0 con 0x7fd244100e20 2024-09-19T09:01:38.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.632+0000 7fd24a0de640 1 -- 172.21.15.120:0/3288312432 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2441a4b60 con 0x7fd244100e20 2024-09-19T09:01:38.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.632+0000 7fd2417fa640 1 -- 172.21.15.120:0/3288312432 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd234017d10 con 0x7fd244100e20 2024-09-19T09:01:38.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.632+0000 7fd24a0de640 1 -- 172.21.15.120:0/3288312432 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2441a5020 con 0x7fd244100e20 2024-09-19T09:01:38.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.633+0000 7fd24a0de640 1 -- 172.21.15.120:0/3288312432 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd244101200 con 0x7fd244100e20 2024-09-19T09:01:38.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.634+0000 7fd2417fa640 1 -- 172.21.15.120:0/3288312432 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fd234017070 con 0x7fd244100e20 2024-09-19T09:01:38.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.635+0000 7fd2417fa640 1 --2- 172.21.15.120:0/3288312432 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd2300a32d0 0x7fd2300a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:38.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.635+0000 7fd23affd640 1 --2- 172.21.15.120:0/3288312432 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd2300a32d0 0x7fd2300a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:38.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.635+0000 7fd2417fa640 1 -- 172.21.15.120:0/3288312432 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fd2340cefe0 con 0x7fd244100e20 2024-09-19T09:01:38.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.635+0000 7fd23affd640 1 --2- 172.21.15.120:0/3288312432 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd2300a32d0 0x7fd2300a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd22c000c00 tx=0x7fd22c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:38.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.638+0000 7fd2417fa640 1 -- 172.21.15.120:0/3288312432 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd23409bdc0 con 0x7fd244100e20 2024-09-19T09:01:38.761 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:38 smithi120 ceph-mon[31150]: pgmap v302: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T09:01:38.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.816+0000 7fd24a0de640 1 -- 172.21.15.120:0/3288312432 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fd244109090 con 0x7fd244100e20 2024-09-19T09:01:38.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.817+0000 7fd2417fa640 1 -- 172.21.15.120:0/3288312432 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd23409b6c0 con 0x7fd244100e20 2024-09-19T09:01:38.817 INFO:teuthology.orchestra.run.smithi120.stdout:266287972386 2024-09-19T09:01:38.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.819+0000 7fd24a0de640 1 -- 172.21.15.120:0/3288312432 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd2300a32d0 msgr2=0x7fd2300a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:38.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.819+0000 7fd24a0de640 1 --2- 172.21.15.120:0/3288312432 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd2300a32d0 0x7fd2300a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd22c000c00 tx=0x7fd22c0058d0 comp rx=0 tx=0).stop 2024-09-19T09:01:38.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.819+0000 7fd24a0de640 1 -- 172.21.15.120:0/3288312432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd244100e20 msgr2=0x7fd2441993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:38.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.819+0000 7fd24a0de640 1 --2- 172.21.15.120:0/3288312432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd244100e20 0x7fd2441993d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd23400b4a0 tx=0x7fd23400b960 comp rx=0 tx=0).stop 2024-09-19T09:01:38.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.819+0000 7fd24a0de640 1 -- 172.21.15.120:0/3288312432 shutdown_connections 2024-09-19T09:01:38.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.819+0000 7fd24a0de640 1 --2- 172.21.15.120:0/3288312432 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd2300a32d0 0x7fd2300a5790 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:38.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.819+0000 7fd24a0de640 1 --2- 172.21.15.120:0/3288312432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd244102ef0 0x7fd2441a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:38.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.819+0000 7fd24a0de640 1 --2- 172.21.15.120:0/3288312432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd244102570 0x7fd244199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:38.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.819+0000 7fd24a0de640 1 --2- 172.21.15.120:0/3288312432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd244100e20 0x7fd2441993d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:38.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.819+0000 7fd24a0de640 1 -- 172.21.15.120:0/3288312432 >> 172.21.15.120:0/3288312432 conn(0x7fd2440fc7d0 msgr2=0x7fd2441068e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:38.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.819+0000 7fd24a0de640 1 -- 172.21.15.120:0/3288312432 shutdown_connections 2024-09-19T09:01:38.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:38.819+0000 7fd24a0de640 1 -- 172.21.15.120:0/3288312432 wait complete. 2024-09-19T09:01:39.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:38 smithi196 ceph-mon[35150]: pgmap v302: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T09:01:39.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:38 smithi153 ceph-mon[35071]: pgmap v302: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T09:01:39.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.697+0000 7f1357733640 1 -- 172.21.15.120:0/2336431568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13501027a0 msgr2=0x7f13501067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:39.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.697+0000 7f1357733640 1 --2- 172.21.15.120:0/2336431568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13501027a0 0x7f13501067f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1344009900 tx=0x7f134402a410 comp rx=0 tx=0).stop 2024-09-19T09:01:39.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.697+0000 7f1357733640 1 -- 172.21.15.120:0/2336431568 shutdown_connections 2024-09-19T09:01:39.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.697+0000 7f1357733640 1 --2- 172.21.15.120:0/2336431568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1350106f20 0x7f135010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:39.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.697+0000 7f1357733640 1 --2- 172.21.15.120:0/2336431568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13501027a0 0x7f13501067f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:39.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.697+0000 7f1357733640 1 --2- 172.21.15.120:0/2336431568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1350101df0 0x7f13501021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:39.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.697+0000 7f1357733640 1 -- 172.21.15.120:0/2336431568 >> 172.21.15.120:0/2336431568 conn(0x7f13500fd8b0 msgr2=0x7f13500ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:39.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.697+0000 7f1357733640 1 -- 172.21.15.120:0/2336431568 shutdown_connections 2024-09-19T09:01:39.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.697+0000 7f1357733640 1 -- 172.21.15.120:0/2336431568 wait complete. 2024-09-19T09:01:39.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.697+0000 7f1357733640 1 Processor -- start 2024-09-19T09:01:39.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.698+0000 7f1357733640 1 -- start start 2024-09-19T09:01:39.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.698+0000 7f1357733640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1350101df0 0x7f135019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:39.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.698+0000 7f1357733640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13501027a0 0x7f135019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:39.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.698+0000 7f1357733640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1350106f20 0x7f135019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:39.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.698+0000 7f1357733640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1350111f40 con 0x7f13501027a0 2024-09-19T09:01:39.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.698+0000 7f1357733640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1350111dc0 con 0x7f1350106f20 2024-09-19T09:01:39.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.698+0000 7f1357733640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f13501120c0 con 0x7f1350101df0 2024-09-19T09:01:39.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.698+0000 7f1354cce640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13501027a0 0x7f135019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:39.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.698+0000 7f1354cce640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13501027a0 0x7f135019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37056/0 (socket says 172.21.15.120:37056) 2024-09-19T09:01:39.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.698+0000 7f1354cce640 1 -- 172.21.15.120:0/459724811 learned_addr learned my addr 172.21.15.120:0/459724811 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:39.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.698+0000 7f1355cd0640 1 --2- 172.21.15.120:0/459724811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1350106f20 0x7f135019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:39.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.698+0000 7f1354cce640 1 -- 172.21.15.120:0/459724811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1350101df0 msgr2=0x7f135019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:39.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.698+0000 7f1354cce640 1 --2- 172.21.15.120:0/459724811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1350101df0 0x7f135019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:39.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.698+0000 7f1354cce640 1 -- 172.21.15.120:0/459724811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1350106f20 msgr2=0x7f135019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:39.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.698+0000 7f1354cce640 1 --2- 172.21.15.120:0/459724811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1350106f20 0x7f135019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:39.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.698+0000 7f1354cce640 1 -- 172.21.15.120:0/459724811 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135019f5e0 con 0x7f13501027a0 2024-09-19T09:01:39.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.698+0000 7f1355cd0640 1 --2- 172.21.15.120:0/459724811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1350106f20 0x7f135019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:01:39.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.698+0000 7f1354cce640 1 --2- 172.21.15.120:0/459724811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13501027a0 0x7f135019ab70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1344006010 tx=0x7f13440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:39.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.698+0000 7f13427fc640 1 -- 172.21.15.120:0/459724811 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134403f030 con 0x7f13501027a0 2024-09-19T09:01:39.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.698+0000 7f13427fc640 1 -- 172.21.15.120:0/459724811 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1344003ce0 con 0x7f13501027a0 2024-09-19T09:01:39.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.698+0000 7f1357733640 1 -- 172.21.15.120:0/459724811 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f135019f870 con 0x7f13501027a0 2024-09-19T09:01:39.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.698+0000 7f13427fc640 1 -- 172.21.15.120:0/459724811 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344035070 con 0x7f13501027a0 2024-09-19T09:01:39.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.698+0000 7f1357733640 1 -- 172.21.15.120:0/459724811 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13501a71b0 con 0x7f13501027a0 2024-09-19T09:01:39.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.699+0000 7f1357733640 1 -- 172.21.15.120:0/459724811 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13501032d0 con 0x7f13501027a0 2024-09-19T09:01:39.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.701+0000 7f13427fc640 1 -- 172.21.15.120:0/459724811 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f1344042030 con 0x7f13501027a0 2024-09-19T09:01:39.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.701+0000 7f13427fc640 1 --2- 172.21.15.120:0/459724811 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f133c0a32d0 0x7f133c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:39.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.701+0000 7f13554cf640 1 --2- 172.21.15.120:0/459724811 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f133c0a32d0 0x7f133c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:39.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.701+0000 7f13427fc640 1 -- 172.21.15.120:0/459724811 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f13440f0300 con 0x7f13501027a0 2024-09-19T09:01:39.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.702+0000 7f13554cf640 1 --2- 172.21.15.120:0/459724811 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f133c0a32d0 0x7f133c0a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f134c005fd0 tx=0x7f134c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:39.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.704+0000 7f13427fc640 1 -- 172.21.15.120:0/459724811 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f13440bd130 con 0x7f13501027a0 2024-09-19T09:01:39.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.884+0000 7f1357733640 1 -- 172.21.15.120:0/459724811 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f13501a7380 con 0x7f13501027a0 2024-09-19T09:01:39.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.885+0000 7f13427fc640 1 -- 172.21.15.120:0/459724811 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f13440bca30 con 0x7f13501027a0 2024-09-19T09:01:39.885 INFO:teuthology.orchestra.run.smithi120.stdout:313532612631 2024-09-19T09:01:39.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.887+0000 7f1357733640 1 -- 172.21.15.120:0/459724811 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f133c0a32d0 msgr2=0x7f133c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:39.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.887+0000 7f1357733640 1 --2- 172.21.15.120:0/459724811 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f133c0a32d0 0x7f133c0a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f134c005fd0 tx=0x7f134c005850 comp rx=0 tx=0).stop 2024-09-19T09:01:39.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.887+0000 7f1357733640 1 -- 172.21.15.120:0/459724811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13501027a0 msgr2=0x7f135019ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:39.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.887+0000 7f1357733640 1 --2- 172.21.15.120:0/459724811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13501027a0 0x7f135019ab70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1344006010 tx=0x7f13440042f0 comp rx=0 tx=0).stop 2024-09-19T09:01:39.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.887+0000 7f1357733640 1 -- 172.21.15.120:0/459724811 shutdown_connections 2024-09-19T09:01:39.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.887+0000 7f1357733640 1 --2- 172.21.15.120:0/459724811 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f133c0a32d0 0x7f133c0a5790 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:39.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.887+0000 7f1357733640 1 --2- 172.21.15.120:0/459724811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1350106f20 0x7f135019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:39.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.887+0000 7f1357733640 1 --2- 172.21.15.120:0/459724811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13501027a0 0x7f135019ab70 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:39.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.887+0000 7f1357733640 1 --2- 172.21.15.120:0/459724811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1350101df0 0x7f135019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:39.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.887+0000 7f1357733640 1 -- 172.21.15.120:0/459724811 >> 172.21.15.120:0/459724811 conn(0x7f13500fd8b0 msgr2=0x7f13500feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:39.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.887+0000 7f1357733640 1 -- 172.21.15.120:0/459724811 shutdown_connections 2024-09-19T09:01:39.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:39.887+0000 7f1357733640 1 -- 172.21.15.120:0/459724811 wait complete. 2024-09-19T09:01:39.962 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:39 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3288312432' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T09:01:40.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:39 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3288312432' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T09:01:40.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:39 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3288312432' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T09:01:40.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.504+0000 7fedf2325640 1 -- 172.21.15.120:0/913380872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fedec106ba0 msgr2=0x7fedec106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:40.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.504+0000 7fedf2325640 1 --2- 172.21.15.120:0/913380872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fedec106ba0 0x7fedec106f80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7feddc009900 tx=0x7feddc02a410 comp rx=0 tx=0).stop 2024-09-19T09:01:40.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.504+0000 7fedf2325640 1 -- 172.21.15.120:0/913380872 shutdown_connections 2024-09-19T09:01:40.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.504+0000 7fedf2325640 1 --2- 172.21.15.120:0/913380872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fedec1005c0 0x7fedec10ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:40.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.504+0000 7fedf2325640 1 --2- 172.21.15.120:0/913380872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedec0ffc20 0x7fedec100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:40.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.504+0000 7fedf2325640 1 --2- 172.21.15.120:0/913380872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fedec106ba0 0x7fedec106f80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:40.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.504+0000 7fedf2325640 1 -- 172.21.15.120:0/913380872 >> 172.21.15.120:0/913380872 conn(0x7fedec0fba90 msgr2=0x7fedec0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:40.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.504+0000 7fedf2325640 1 -- 172.21.15.120:0/913380872 shutdown_connections 2024-09-19T09:01:40.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.504+0000 7fedf2325640 1 -- 172.21.15.120:0/913380872 wait complete. 2024-09-19T09:01:40.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.505+0000 7fedf2325640 1 Processor -- start 2024-09-19T09:01:40.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.505+0000 7fedf2325640 1 -- start start 2024-09-19T09:01:40.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.505+0000 7fedf2325640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fedec0ffc20 0x7fedec198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:40.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.505+0000 7fedf2325640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fedec1005c0 0x7fedec198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:40.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.505+0000 7fedf2325640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedec106ba0 0x7fedec19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:40.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.505+0000 7fedf2325640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fedec1120b0 con 0x7fedec1005c0 2024-09-19T09:01:40.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.505+0000 7fedf2325640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fedec111f30 con 0x7fedec0ffc20 2024-09-19T09:01:40.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.505+0000 7fedf2325640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fedec112230 con 0x7fedec106ba0 2024-09-19T09:01:40.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.505+0000 7fedeb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fedec1005c0 0x7fedec198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:40.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.505+0000 7fedeb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fedec1005c0 0x7fedec198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37074/0 (socket says 172.21.15.120:37074) 2024-09-19T09:01:40.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.505+0000 7fedeb7fe640 1 -- 172.21.15.120:0/1589697474 learned_addr learned my addr 172.21.15.120:0/1589697474 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:40.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.505+0000 7fedebfff640 1 --2- 172.21.15.120:0/1589697474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fedec0ffc20 0x7fedec198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:40.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.505+0000 7fedeb7fe640 1 -- 172.21.15.120:0/1589697474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedec106ba0 msgr2=0x7fedec19ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:40.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.505+0000 7fedeb7fe640 1 --2- 172.21.15.120:0/1589697474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedec106ba0 0x7fedec19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:40.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.505+0000 7fedeb7fe640 1 -- 172.21.15.120:0/1589697474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fedec0ffc20 msgr2=0x7fedec198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:40.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.505+0000 7fedeb7fe640 1 --2- 172.21.15.120:0/1589697474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fedec0ffc20 0x7fedec198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:40.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.505+0000 7fedeb7fe640 1 -- 172.21.15.120:0/1589697474 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedec19d520 con 0x7fedec1005c0 2024-09-19T09:01:40.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.506+0000 7fedebfff640 1 --2- 172.21.15.120:0/1589697474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fedec0ffc20 0x7fedec198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:01:40.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.506+0000 7fedeb7fe640 1 --2- 172.21.15.120:0/1589697474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fedec1005c0 0x7fedec198ab0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fede000b4a0 tx=0x7fede000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:40.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.506+0000 7fede97fa640 1 -- 172.21.15.120:0/1589697474 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0014070 con 0x7fedec1005c0 2024-09-19T09:01:40.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.506+0000 7fedf2325640 1 -- 172.21.15.120:0/1589697474 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedec19d810 con 0x7fedec1005c0 2024-09-19T09:01:40.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.506+0000 7fede97fa640 1 -- 172.21.15.120:0/1589697474 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fede0010af0 con 0x7fedec1005c0 2024-09-19T09:01:40.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.506+0000 7fede97fa640 1 -- 172.21.15.120:0/1589697474 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0017d10 con 0x7fedec1005c0 2024-09-19T09:01:40.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.506+0000 7fedf2325640 1 -- 172.21.15.120:0/1589697474 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedec1a50f0 con 0x7fedec1005c0 2024-09-19T09:01:40.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.507+0000 7fedf2325640 1 -- 172.21.15.120:0/1589697474 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedec100080 con 0x7fedec1005c0 2024-09-19T09:01:40.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.508+0000 7fede97fa640 1 -- 172.21.15.120:0/1589697474 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fede0017070 con 0x7fedec1005c0 2024-09-19T09:01:40.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.508+0000 7fede97fa640 1 --2- 172.21.15.120:0/1589697474 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fedd40a32d0 0x7fedd40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:40.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.509+0000 7fedebfff640 1 --2- 172.21.15.120:0/1589697474 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fedd40a32d0 0x7fedd40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:40.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.509+0000 7fede97fa640 1 -- 172.21.15.120:0/1589697474 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fede00cecd0 con 0x7fedec1005c0 2024-09-19T09:01:40.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.509+0000 7fedebfff640 1 --2- 172.21.15.120:0/1589697474 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fedd40a32d0 0x7fedd40a5790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7feddc005ab0 tx=0x7feddc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:40.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.513+0000 7fede97fa640 1 -- 172.21.15.120:0/1589697474 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fede009bb50 con 0x7fedec1005c0 2024-09-19T09:01:40.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.689+0000 7fedf2325640 1 -- 172.21.15.120:0/1589697474 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fedec02d7c0 con 0x7fedec1005c0 2024-09-19T09:01:40.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.690+0000 7fede97fa640 1 -- 172.21.15.120:0/1589697474 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fede009b450 con 0x7fedec1005c0 2024-09-19T09:01:40.690 INFO:teuthology.orchestra.run.smithi120.stdout:184683593778 2024-09-19T09:01:40.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.692+0000 7fedf2325640 1 -- 172.21.15.120:0/1589697474 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fedd40a32d0 msgr2=0x7fedd40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:40.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.692+0000 7fedf2325640 1 --2- 172.21.15.120:0/1589697474 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fedd40a32d0 0x7fedd40a5790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7feddc005ab0 tx=0x7feddc000bc0 comp rx=0 tx=0).stop 2024-09-19T09:01:40.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.692+0000 7fedf2325640 1 -- 172.21.15.120:0/1589697474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fedec1005c0 msgr2=0x7fedec198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:40.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.692+0000 7fedf2325640 1 --2- 172.21.15.120:0/1589697474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fedec1005c0 0x7fedec198ab0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fede000b4a0 tx=0x7fede000b960 comp rx=0 tx=0).stop 2024-09-19T09:01:40.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.692+0000 7fedf2325640 1 -- 172.21.15.120:0/1589697474 shutdown_connections 2024-09-19T09:01:40.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.692+0000 7fedf2325640 1 --2- 172.21.15.120:0/1589697474 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fedd40a32d0 0x7fedd40a5790 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:40.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.692+0000 7fedf2325640 1 --2- 172.21.15.120:0/1589697474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedec106ba0 0x7fedec19ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:40.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.692+0000 7fedf2325640 1 --2- 172.21.15.120:0/1589697474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fedec1005c0 0x7fedec198ab0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:40.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.692+0000 7fedf2325640 1 --2- 172.21.15.120:0/1589697474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fedec0ffc20 0x7fedec198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:40.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.692+0000 7fedf2325640 1 -- 172.21.15.120:0/1589697474 >> 172.21.15.120:0/1589697474 conn(0x7fedec0fba90 msgr2=0x7fedec0fc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:40.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.692+0000 7fedf2325640 1 -- 172.21.15.120:0/1589697474 shutdown_connections 2024-09-19T09:01:40.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.692+0000 7fedf2325640 1 -- 172.21.15.120:0/1589697474 wait complete. 2024-09-19T09:01:41.045 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:40 smithi120 ceph-mon[31150]: pgmap v303: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T09:01:41.045 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:40 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/459724811' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T09:01:41.045 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:40 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1589697474' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T09:01:41.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:40 smithi196 ceph-mon[35150]: pgmap v303: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T09:01:41.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:40 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/459724811' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T09:01:41.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:40 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1589697474' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T09:01:41.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:40 smithi153 ceph-mon[35071]: pgmap v303: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T09:01:41.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:40 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/459724811' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T09:01:41.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:40 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1589697474' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T09:01:41.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.865+0000 7f64b2845640 1 -- 172.21.15.120:0/3526010254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ac102570 msgr2=0x7f64ac1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:41.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.865+0000 7f64b2845640 1 --2- 172.21.15.120:0/3526010254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ac102570 0x7f64ac1029b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f64a0009900 tx=0x7f64a002a480 comp rx=0 tx=0).stop 2024-09-19T09:01:41.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.865+0000 7f64b2845640 1 -- 172.21.15.120:0/3526010254 shutdown_connections 2024-09-19T09:01:41.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.865+0000 7f64b2845640 1 --2- 172.21.15.120:0/3526010254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ac102ef0 0x7f64ac10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:41.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.865+0000 7f64b2845640 1 --2- 172.21.15.120:0/3526010254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ac102570 0x7f64ac1029b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:41.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.865+0000 7f64b2845640 1 --2- 172.21.15.120:0/3526010254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64ac100e20 0x7f64ac101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:41.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.865+0000 7f64b2845640 1 -- 172.21.15.120:0/3526010254 >> 172.21.15.120:0/3526010254 conn(0x7f64ac0fc7d0 msgr2=0x7f64ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:41.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.865+0000 7f64b2845640 1 -- 172.21.15.120:0/3526010254 shutdown_connections 2024-09-19T09:01:41.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.865+0000 7f64b2845640 1 -- 172.21.15.120:0/3526010254 wait complete. 2024-09-19T09:01:41.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.866+0000 7f64b2845640 1 Processor -- start 2024-09-19T09:01:41.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.866+0000 7f64b2845640 1 -- start start 2024-09-19T09:01:41.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.866+0000 7f64b2845640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64ac100e20 0x7f64ac198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:41.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.866+0000 7f64b2845640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ac102570 0x7f64ac198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:41.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.866+0000 7f64b2845640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ac102ef0 0x7f64ac19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:41.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.866+0000 7f64b2845640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac112110 con 0x7f64ac102570 2024-09-19T09:01:41.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.866+0000 7f64b2845640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac111f90 con 0x7f64ac102ef0 2024-09-19T09:01:41.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.866+0000 7f64b2845640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac112290 con 0x7f64ac100e20 2024-09-19T09:01:41.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.867+0000 7f64ab7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ac102570 0x7f64ac198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:41.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.867+0000 7f64ab7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ac102570 0x7f64ac198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37084/0 (socket says 172.21.15.120:37084) 2024-09-19T09:01:41.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.867+0000 7f64ab7fe640 1 -- 172.21.15.120:0/3656265103 learned_addr learned my addr 172.21.15.120:0/3656265103 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:41.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.867+0000 7f64b0de2640 1 --2- 172.21.15.120:0/3656265103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ac102ef0 0x7f64ac19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:41.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.867+0000 7f64abfff640 1 --2- 172.21.15.120:0/3656265103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64ac100e20 0x7f64ac198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:41.553 INFO:teuthology.orchestra.run.smithi120.stdout:154618822710 2024-09-19T09:01:41.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.867+0000 7f64ab7fe640 1 -- 172.21.15.120:0/3656265103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64ac100e20 msgr2=0x7f64ac198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:41.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.867+0000 7f64ab7fe640 1 --2- 172.21.15.120:0/3656265103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64ac100e20 0x7f64ac198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:41.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.867+0000 7f64ab7fe640 1 -- 172.21.15.120:0/3656265103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ac102ef0 msgr2=0x7f64ac19ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:41.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.867+0000 7f64ab7fe640 1 --2- 172.21.15.120:0/3656265103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ac102ef0 0x7f64ac19ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:41.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.867+0000 7f64ab7fe640 1 -- 172.21.15.120:0/3656265103 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ac19d510 con 0x7f64ac102570 2024-09-19T09:01:41.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.867+0000 7f64b0de2640 1 --2- 172.21.15.120:0/3656265103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ac102ef0 0x7f64ac19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:01:41.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.867+0000 7f64ab7fe640 1 --2- 172.21.15.120:0/3656265103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ac102570 0x7f64ac198aa0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f64a0006010 tx=0x7f64a0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:41.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.867+0000 7f64a97fa640 1 -- 172.21.15.120:0/3656265103 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a003f070 con 0x7f64ac102570 2024-09-19T09:01:41.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.867+0000 7f64a97fa640 1 -- 172.21.15.120:0/3656265103 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64a0031e30 con 0x7f64ac102570 2024-09-19T09:01:41.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.867+0000 7f64b2845640 1 -- 172.21.15.120:0/3656265103 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64ac19d7a0 con 0x7f64ac102570 2024-09-19T09:01:41.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.867+0000 7f64a97fa640 1 -- 172.21.15.120:0/3656265103 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a0035030 con 0x7f64ac102570 2024-09-19T09:01:41.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.867+0000 7f64b2845640 1 -- 172.21.15.120:0/3656265103 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64ac1a50e0 con 0x7f64ac102570 2024-09-19T09:01:41.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.868+0000 7f64b2845640 1 -- 172.21.15.120:0/3656265103 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64ac101200 con 0x7f64ac102570 2024-09-19T09:01:41.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.869+0000 7f64a97fa640 1 -- 172.21.15.120:0/3656265103 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f64a0042070 con 0x7f64ac102570 2024-09-19T09:01:41.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.870+0000 7f64a97fa640 1 --2- 172.21.15.120:0/3656265103 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f64940a3350 0x7f64940a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:41.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.870+0000 7f64abfff640 1 --2- 172.21.15.120:0/3656265103 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f64940a3350 0x7f64940a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:41.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.870+0000 7f64a97fa640 1 -- 172.21.15.120:0/3656265103 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f64a00f0820 con 0x7f64ac102570 2024-09-19T09:01:41.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.870+0000 7f64abfff640 1 --2- 172.21.15.120:0/3656265103 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f64940a3350 0x7f64940a5810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f649c009890 tx=0x7f649c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:41.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:40.873+0000 7f64a97fa640 1 -- 172.21.15.120:0/3656265103 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f64a00bd620 con 0x7f64ac102570 2024-09-19T09:01:41.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.045+0000 7f64b2845640 1 -- 172.21.15.120:0/3656265103 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f64ac1090c0 con 0x7f64ac102570 2024-09-19T09:01:41.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.045+0000 7f64a97fa640 1 -- 172.21.15.120:0/3656265103 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f64a00bcf20 con 0x7f64ac102570 2024-09-19T09:01:41.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.047+0000 7f64b2845640 1 -- 172.21.15.120:0/3656265103 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f64940a3350 msgr2=0x7f64940a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:41.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.047+0000 7f64b2845640 1 --2- 172.21.15.120:0/3656265103 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f64940a3350 0x7f64940a5810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f649c009890 tx=0x7f649c008040 comp rx=0 tx=0).stop 2024-09-19T09:01:41.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.047+0000 7f64b2845640 1 -- 172.21.15.120:0/3656265103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ac102570 msgr2=0x7f64ac198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:41.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.047+0000 7f64b2845640 1 --2- 172.21.15.120:0/3656265103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ac102570 0x7f64ac198aa0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f64a0006010 tx=0x7f64a0003c80 comp rx=0 tx=0).stop 2024-09-19T09:01:41.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.047+0000 7f64b2845640 1 -- 172.21.15.120:0/3656265103 shutdown_connections 2024-09-19T09:01:41.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.047+0000 7f64b2845640 1 --2- 172.21.15.120:0/3656265103 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f64940a3350 0x7f64940a5810 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:41.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.047+0000 7f64b2845640 1 --2- 172.21.15.120:0/3656265103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ac102ef0 0x7f64ac19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:41.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.047+0000 7f64b2845640 1 --2- 172.21.15.120:0/3656265103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ac102570 0x7f64ac198aa0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:41.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.047+0000 7f64b2845640 1 --2- 172.21.15.120:0/3656265103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64ac100e20 0x7f64ac198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:41.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.047+0000 7f64b2845640 1 -- 172.21.15.120:0/3656265103 >> 172.21.15.120:0/3656265103 conn(0x7f64ac0fc7d0 msgr2=0x7f64ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:41.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.047+0000 7f64b2845640 1 -- 172.21.15.120:0/3656265103 shutdown_connections 2024-09-19T09:01:41.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.047+0000 7f64b2845640 1 -- 172.21.15.120:0/3656265103 wait complete. 2024-09-19T09:01:41.976 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:41 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3656265103' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T09:01:41.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.976+0000 7f92e12ac640 1 -- 172.21.15.120:0/1203437239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92dc102570 msgr2=0x7f92dc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:41.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.976+0000 7f92e12ac640 1 --2- 172.21.15.120:0/1203437239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92dc102570 0x7f92dc1029b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f92c4009900 tx=0x7f92c402a480 comp rx=0 tx=0).stop 2024-09-19T09:01:41.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.976+0000 7f92e12ac640 1 -- 172.21.15.120:0/1203437239 shutdown_connections 2024-09-19T09:01:41.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.976+0000 7f92e12ac640 1 --2- 172.21.15.120:0/1203437239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92dc102ef0 0x7f92dc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:41.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.976+0000 7f92e12ac640 1 --2- 172.21.15.120:0/1203437239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92dc102570 0x7f92dc1029b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:41.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.976+0000 7f92e12ac640 1 --2- 172.21.15.120:0/1203437239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92dc100e20 0x7f92dc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:41.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.976+0000 7f92e12ac640 1 -- 172.21.15.120:0/1203437239 >> 172.21.15.120:0/1203437239 conn(0x7f92dc0fc7d0 msgr2=0x7f92dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:41.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.976+0000 7f92e12ac640 1 -- 172.21.15.120:0/1203437239 shutdown_connections 2024-09-19T09:01:41.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.976+0000 7f92e12ac640 1 -- 172.21.15.120:0/1203437239 wait complete. 2024-09-19T09:01:41.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.977+0000 7f92e12ac640 1 Processor -- start 2024-09-19T09:01:41.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.977+0000 7f92e12ac640 1 -- start start 2024-09-19T09:01:41.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.977+0000 7f92e12ac640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92dc100e20 0x7f92dc198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:41.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.977+0000 7f92e12ac640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92dc102570 0x7f92dc198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:41.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.977+0000 7f92e12ac640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92dc102ef0 0x7f92dc19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:42.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.977+0000 7f92e12ac640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc110fa0 con 0x7f92dc102ef0 2024-09-19T09:01:42.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.977+0000 7f92e12ac640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc110e20 con 0x7f92dc102570 2024-09-19T09:01:42.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.977+0000 7f92e12ac640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc111120 con 0x7f92dc100e20 2024-09-19T09:01:42.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.977+0000 7f92db7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92dc102ef0 0x7f92dc19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:42.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.977+0000 7f92db7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92dc102ef0 0x7f92dc19ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59554/0 (socket says 172.21.15.120:59554) 2024-09-19T09:01:42.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.977+0000 7f92db7fe640 1 -- 172.21.15.120:0/2014489118 learned_addr learned my addr 172.21.15.120:0/2014489118 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:42.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.977+0000 7f92d27fc640 1 --2- 172.21.15.120:0/2014489118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92dc102570 0x7f92dc198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:42.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.977+0000 7f92db7fe640 1 -- 172.21.15.120:0/2014489118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92dc100e20 msgr2=0x7f92dc198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:42.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.977+0000 7f92db7fe640 1 --2- 172.21.15.120:0/2014489118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92dc100e20 0x7f92dc198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:42.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.977+0000 7f92db7fe640 1 -- 172.21.15.120:0/2014489118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92dc102570 msgr2=0x7f92dc198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:42.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.977+0000 7f92db7fe640 1 --2- 172.21.15.120:0/2014489118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92dc102570 0x7f92dc198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:42.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.977+0000 7f92db7fe640 1 -- 172.21.15.120:0/2014489118 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92dc19d510 con 0x7f92dc102ef0 2024-09-19T09:01:42.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.978+0000 7f92db7fe640 1 --2- 172.21.15.120:0/2014489118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92dc102ef0 0x7f92dc19ce30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f92d400b970 tx=0x7f92d400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:42.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.978+0000 7f92d8ff9640 1 -- 172.21.15.120:0/2014489118 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d4019070 con 0x7f92dc102ef0 2024-09-19T09:01:42.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.978+0000 7f92e12ac640 1 -- 172.21.15.120:0/2014489118 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92dc19d800 con 0x7f92dc102ef0 2024-09-19T09:01:42.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.978+0000 7f92d8ff9640 1 -- 172.21.15.120:0/2014489118 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92d400cb40 con 0x7f92dc102ef0 2024-09-19T09:01:42.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.978+0000 7f92d8ff9640 1 -- 172.21.15.120:0/2014489118 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d4007610 con 0x7f92dc102ef0 2024-09-19T09:01:42.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.978+0000 7f92e12ac640 1 -- 172.21.15.120:0/2014489118 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92dc1a50e0 con 0x7f92dc102ef0 2024-09-19T09:01:42.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.979+0000 7f92e12ac640 1 -- 172.21.15.120:0/2014489118 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92dc101200 con 0x7f92dc102ef0 2024-09-19T09:01:42.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.980+0000 7f92d8ff9640 1 -- 172.21.15.120:0/2014489118 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f92d401c070 con 0x7f92dc102ef0 2024-09-19T09:01:42.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.981+0000 7f92d8ff9640 1 --2- 172.21.15.120:0/2014489118 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92c80a32d0 0x7f92c80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:42.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.981+0000 7f92daffd640 1 --2- 172.21.15.120:0/2014489118 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92c80a32d0 0x7f92c80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:42.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.981+0000 7f92d8ff9640 1 -- 172.21.15.120:0/2014489118 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f92d40cf3d0 con 0x7f92dc102ef0 2024-09-19T09:01:42.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.981+0000 7f92daffd640 1 --2- 172.21.15.120:0/2014489118 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92c80a32d0 0x7f92c80a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f92cc00a8f0 tx=0x7f92cc005bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:42.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:41.984+0000 7f92d8ff9640 1 -- 172.21.15.120:0/2014489118 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f92d409c150 con 0x7f92dc102ef0 2024-09-19T09:01:42.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.158+0000 7f92e12ac640 1 -- 172.21.15.120:0/2014489118 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f92dc10a0a0 con 0x7f92dc102ef0 2024-09-19T09:01:42.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.158+0000 7f92d8ff9640 1 -- 172.21.15.120:0/2014489118 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f92d409ba50 con 0x7f92dc102ef0 2024-09-19T09:01:42.158 INFO:teuthology.orchestra.run.smithi120.stdout:34359738448 2024-09-19T09:01:42.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.161+0000 7f92e12ac640 1 -- 172.21.15.120:0/2014489118 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92c80a32d0 msgr2=0x7f92c80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:42.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.161+0000 7f92e12ac640 1 --2- 172.21.15.120:0/2014489118 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92c80a32d0 0x7f92c80a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f92cc00a8f0 tx=0x7f92cc005bf0 comp rx=0 tx=0).stop 2024-09-19T09:01:42.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.161+0000 7f92e12ac640 1 -- 172.21.15.120:0/2014489118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92dc102ef0 msgr2=0x7f92dc19ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:42.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.161+0000 7f92e12ac640 1 --2- 172.21.15.120:0/2014489118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92dc102ef0 0x7f92dc19ce30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f92d400b970 tx=0x7f92d400be30 comp rx=0 tx=0).stop 2024-09-19T09:01:42.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.161+0000 7f92e12ac640 1 -- 172.21.15.120:0/2014489118 shutdown_connections 2024-09-19T09:01:42.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.161+0000 7f92e12ac640 1 --2- 172.21.15.120:0/2014489118 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92c80a32d0 0x7f92c80a5790 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:42.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.161+0000 7f92e12ac640 1 --2- 172.21.15.120:0/2014489118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92dc102ef0 0x7f92dc19ce30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:42.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.161+0000 7f92e12ac640 1 --2- 172.21.15.120:0/2014489118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92dc102570 0x7f92dc198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:42.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.161+0000 7f92e12ac640 1 --2- 172.21.15.120:0/2014489118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92dc100e20 0x7f92dc198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:42.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.161+0000 7f92e12ac640 1 -- 172.21.15.120:0/2014489118 >> 172.21.15.120:0/2014489118 conn(0x7f92dc0fc7d0 msgr2=0x7f92dc06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:42.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.161+0000 7f92e12ac640 1 -- 172.21.15.120:0/2014489118 shutdown_connections 2024-09-19T09:01:42.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.161+0000 7f92e12ac640 1 -- 172.21.15.120:0/2014489118 wait complete. 2024-09-19T09:01:42.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.179+0000 7f8bd91e6640 1 -- 172.21.15.120:0/2338738415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bd4106f20 msgr2=0x7f8bd410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:42.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.179+0000 7f8bd91e6640 1 --2- 172.21.15.120:0/2338738415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bd4106f20 0x7f8bd410eaa0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009ca0 tx=0x7f8bcc02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:01:42.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.179+0000 7f8bd91e6640 1 -- 172.21.15.120:0/2338738415 shutdown_connections 2024-09-19T09:01:42.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.179+0000 7f8bd91e6640 1 --2- 172.21.15.120:0/2338738415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bd4106f20 0x7f8bd410eaa0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:42.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.179+0000 7f8bd91e6640 1 --2- 172.21.15.120:0/2338738415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bd41027a0 0x7f8bd41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:42.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.179+0000 7f8bd91e6640 1 --2- 172.21.15.120:0/2338738415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bd4101df0 0x7f8bd41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:42.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.179+0000 7f8bd91e6640 1 -- 172.21.15.120:0/2338738415 >> 172.21.15.120:0/2338738415 conn(0x7f8bd40fd8f0 msgr2=0x7f8bd40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:42.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.179+0000 7f8bd91e6640 1 -- 172.21.15.120:0/2338738415 shutdown_connections 2024-09-19T09:01:42.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.179+0000 7f8bd91e6640 1 -- 172.21.15.120:0/2338738415 wait complete. 2024-09-19T09:01:42.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.180+0000 7f8bd91e6640 1 Processor -- start 2024-09-19T09:01:42.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.180+0000 7f8bd91e6640 1 -- start start 2024-09-19T09:01:42.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.180+0000 7f8bd91e6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bd4101df0 0x7f8bd41983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:42.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.180+0000 7f8bd91e6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bd41027a0 0x7f8bd4198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:42.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.180+0000 7f8bd91e6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bd4106f20 0x7f8bd419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:42.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.181+0000 7f8bd91e6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd4111fa0 con 0x7f8bd41027a0 2024-09-19T09:01:42.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.181+0000 7f8bd91e6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd4111e20 con 0x7f8bd4106f20 2024-09-19T09:01:42.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.181+0000 7f8bd91e6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd4112120 con 0x7f8bd4101df0 2024-09-19T09:01:42.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.181+0000 7f8bd259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bd41027a0 0x7f8bd4198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:42.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.181+0000 7f8bd259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bd41027a0 0x7f8bd4198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59566/0 (socket says 172.21.15.120:59566) 2024-09-19T09:01:42.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.181+0000 7f8bd259c640 1 -- 172.21.15.120:0/2329516579 learned_addr learned my addr 172.21.15.120:0/2329516579 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:42.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.181+0000 7f8bd2d9d640 1 --2- 172.21.15.120:0/2329516579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bd4101df0 0x7f8bd41983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:42.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.181+0000 7f8bd359e640 1 --2- 172.21.15.120:0/2329516579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bd4106f20 0x7f8bd419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:42.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.181+0000 7f8bd259c640 1 -- 172.21.15.120:0/2329516579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bd4101df0 msgr2=0x7f8bd41983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:42.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.181+0000 7f8bd259c640 1 --2- 172.21.15.120:0/2329516579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bd4101df0 0x7f8bd41983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:42.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.181+0000 7f8bd259c640 1 -- 172.21.15.120:0/2329516579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bd4106f20 msgr2=0x7f8bd419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:42.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.181+0000 7f8bd259c640 1 --2- 172.21.15.120:0/2329516579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bd4106f20 0x7f8bd419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:42.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.181+0000 7f8bd259c640 1 -- 172.21.15.120:0/2329516579 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd419d380 con 0x7f8bd41027a0 2024-09-19T09:01:42.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.181+0000 7f8bd259c640 1 --2- 172.21.15.120:0/2329516579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bd41027a0 0x7f8bd4198910 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800b4a0 tx=0x7f8bc800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:42.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.181+0000 7f8bbbfff640 1 -- 172.21.15.120:0/2329516579 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8015020 con 0x7f8bd41027a0 2024-09-19T09:01:42.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.182+0000 7f8bbbfff640 1 -- 172.21.15.120:0/2329516579 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bc8011d10 con 0x7f8bd41027a0 2024-09-19T09:01:42.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.182+0000 7f8bbbfff640 1 -- 172.21.15.120:0/2329516579 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc800f2e0 con 0x7f8bd41027a0 2024-09-19T09:01:42.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.182+0000 7f8bd91e6640 1 -- 172.21.15.120:0/2329516579 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd419d670 con 0x7f8bd41027a0 2024-09-19T09:01:42.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.182+0000 7f8bd91e6640 1 -- 172.21.15.120:0/2329516579 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd41a4f50 con 0x7f8bd41027a0 2024-09-19T09:01:42.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.183+0000 7f8bd91e6640 1 -- 172.21.15.120:0/2329516579 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bd41021d0 con 0x7f8bd41027a0 2024-09-19T09:01:42.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.185+0000 7f8bbbfff640 1 -- 172.21.15.120:0/2329516579 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f8bc8018020 con 0x7f8bd41027a0 2024-09-19T09:01:42.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.185+0000 7f8bbbfff640 1 --2- 172.21.15.120:0/2329516579 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8bbc0a32d0 0x7f8bbc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:42.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.186+0000 7f8bd2d9d640 1 --2- 172.21.15.120:0/2329516579 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8bbc0a32d0 0x7f8bbc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:42.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.186+0000 7f8bbbfff640 1 -- 172.21.15.120:0/2329516579 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f8bc80cf3f0 con 0x7f8bd41027a0 2024-09-19T09:01:42.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:41 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3656265103' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T09:01:42.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.186+0000 7f8bd2d9d640 1 --2- 172.21.15.120:0/2329516579 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8bbc0a32d0 0x7f8bbc0a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400abe0 tx=0x7f8bc400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:42.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:41 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3656265103' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T09:01:42.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.189+0000 7f8bbbfff640 1 -- 172.21.15.120:0/2329516579 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8bc801f020 con 0x7f8bd41027a0 2024-09-19T09:01:42.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.367+0000 7f8bd91e6640 1 -- 172.21.15.120:0/2329516579 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f8bd4199730 con 0x7f8bd41027a0 2024-09-19T09:01:42.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.368+0000 7f8bbbfff640 1 -- 172.21.15.120:0/2329516579 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f8bd4199730 con 0x7f8bd41027a0 2024-09-19T09:01:42.368 INFO:teuthology.orchestra.run.smithi120.stdout:283467841565 2024-09-19T09:01:42.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.370+0000 7f8bd91e6640 1 -- 172.21.15.120:0/2329516579 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8bbc0a32d0 msgr2=0x7f8bbc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:42.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.370+0000 7f8bd91e6640 1 --2- 172.21.15.120:0/2329516579 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8bbc0a32d0 0x7f8bbc0a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400abe0 tx=0x7f8bc400a2d0 comp rx=0 tx=0).stop 2024-09-19T09:01:42.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.370+0000 7f8bd91e6640 1 -- 172.21.15.120:0/2329516579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bd41027a0 msgr2=0x7f8bd4198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:42.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.370+0000 7f8bd91e6640 1 --2- 172.21.15.120:0/2329516579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bd41027a0 0x7f8bd4198910 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800b4a0 tx=0x7f8bc800b8a0 comp rx=0 tx=0).stop 2024-09-19T09:01:42.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.370+0000 7f8bd91e6640 1 -- 172.21.15.120:0/2329516579 shutdown_connections 2024-09-19T09:01:42.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.370+0000 7f8bd91e6640 1 --2- 172.21.15.120:0/2329516579 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8bbc0a32d0 0x7f8bbc0a5790 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:42.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.370+0000 7f8bd91e6640 1 --2- 172.21.15.120:0/2329516579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bd4106f20 0x7f8bd419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:42.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.370+0000 7f8bd91e6640 1 --2- 172.21.15.120:0/2329516579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bd41027a0 0x7f8bd4198910 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:42.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.370+0000 7f8bd91e6640 1 --2- 172.21.15.120:0/2329516579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bd4101df0 0x7f8bd41983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:42.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.370+0000 7f8bd91e6640 1 -- 172.21.15.120:0/2329516579 >> 172.21.15.120:0/2329516579 conn(0x7f8bd40fd8f0 msgr2=0x7f8bd40fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:42.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.370+0000 7f8bd91e6640 1 -- 172.21.15.120:0/2329516579 shutdown_connections 2024-09-19T09:01:42.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.370+0000 7f8bd91e6640 1 -- 172.21.15.120:0/2329516579 wait complete. 2024-09-19T09:01:42.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:42 smithi120 ceph-mon[31150]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T09:01:42.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:42 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2014489118' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T09:01:42.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:42 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2329516579' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T09:01:42.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:42 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/4250332389' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T09:01:43.126 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841562 got 283467841565 for osd.10 2024-09-19T09:01:43.126 DEBUG:teuthology.parallel:result is None 2024-09-19T09:01:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:42 smithi196 ceph-mon[35150]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T09:01:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:42 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2014489118' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T09:01:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:42 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2329516579' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T09:01:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:42 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/4250332389' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T09:01:43.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:42 smithi153 ceph-mon[35071]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T09:01:43.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:42 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2014489118' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T09:01:43.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:42 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2329516579' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T09:01:43.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:42 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/4250332389' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T09:01:43.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.353+0000 7fbf4af4a640 1 -- 172.21.15.120:0/2413073030 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf44100e20 msgr2=0x7fbf44101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:43.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.353+0000 7fbf4af4a640 1 --2- 172.21.15.120:0/2413073030 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf44100e20 0x7fbf44101200 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009900 tx=0x7fbf3802a480 comp rx=0 tx=0).stop 2024-09-19T09:01:43.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.353+0000 7fbf4af4a640 1 -- 172.21.15.120:0/2413073030 shutdown_connections 2024-09-19T09:01:43.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.353+0000 7fbf4af4a640 1 --2- 172.21.15.120:0/2413073030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf44102ef0 0x7fbf4410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:43.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.353+0000 7fbf4af4a640 1 --2- 172.21.15.120:0/2413073030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf44102570 0x7fbf441029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:43.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.353+0000 7fbf4af4a640 1 --2- 172.21.15.120:0/2413073030 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf44100e20 0x7fbf44101200 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:43.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.353+0000 7fbf4af4a640 1 -- 172.21.15.120:0/2413073030 >> 172.21.15.120:0/2413073030 conn(0x7fbf440fc7d0 msgr2=0x7fbf440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:43.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.353+0000 7fbf4af4a640 1 -- 172.21.15.120:0/2413073030 shutdown_connections 2024-09-19T09:01:43.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.353+0000 7fbf4af4a640 1 -- 172.21.15.120:0/2413073030 wait complete. 2024-09-19T09:01:43.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.354+0000 7fbf4af4a640 1 Processor -- start 2024-09-19T09:01:43.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.354+0000 7fbf4af4a640 1 -- start start 2024-09-19T09:01:43.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.354+0000 7fbf4af4a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf44100e20 0x7fbf441993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:43.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.354+0000 7fbf4af4a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf44102570 0x7fbf44199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:43.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.354+0000 7fbf4af4a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf44102ef0 0x7fbf441a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:43.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.354+0000 7fbf4af4a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbf44110f80 con 0x7fbf44102ef0 2024-09-19T09:01:43.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.354+0000 7fbf4af4a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbf44110e00 con 0x7fbf44100e20 2024-09-19T09:01:43.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.354+0000 7fbf4af4a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbf44111100 con 0x7fbf44102570 2024-09-19T09:01:43.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.354+0000 7fbf494e7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf44102ef0 0x7fbf441a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:43.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.354+0000 7fbf494e7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf44102ef0 0x7fbf441a4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59584/0 (socket says 172.21.15.120:59584) 2024-09-19T09:01:43.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.354+0000 7fbf494e7640 1 -- 172.21.15.120:0/4250332389 learned_addr learned my addr 172.21.15.120:0/4250332389 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:43.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.354+0000 7fbf48ce6640 1 --2- 172.21.15.120:0/4250332389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf44100e20 0x7fbf441993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:43.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.355+0000 7fbf43fff640 1 --2- 172.21.15.120:0/4250332389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf44102570 0x7fbf44199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:43.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.355+0000 7fbf494e7640 1 -- 172.21.15.120:0/4250332389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf44102570 msgr2=0x7fbf44199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:43.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.355+0000 7fbf494e7640 1 --2- 172.21.15.120:0/4250332389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf44102570 0x7fbf44199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:43.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.355+0000 7fbf494e7640 1 -- 172.21.15.120:0/4250332389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf44100e20 msgr2=0x7fbf441993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:43.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.355+0000 7fbf494e7640 1 --2- 172.21.15.120:0/4250332389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf44100e20 0x7fbf441993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:43.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.355+0000 7fbf494e7640 1 -- 172.21.15.120:0/4250332389 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf441a4990 con 0x7fbf44102ef0 2024-09-19T09:01:43.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.355+0000 7fbf43fff640 1 --2- 172.21.15.120:0/4250332389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf44102570 0x7fbf44199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:01:43.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.355+0000 7fbf48ce6640 1 --2- 172.21.15.120:0/4250332389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf44100e20 0x7fbf441993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:01:43.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.355+0000 7fbf494e7640 1 --2- 172.21.15.120:0/4250332389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf44102ef0 0x7fbf441a4450 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00cad0 tx=0x7fbf3c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:43.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.355+0000 7fbf41ffb640 1 -- 172.21.15.120:0/4250332389 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c00ae20 con 0x7fbf44102ef0 2024-09-19T09:01:43.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.355+0000 7fbf4af4a640 1 -- 172.21.15.120:0/4250332389 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf441a4b60 con 0x7fbf44102ef0 2024-09-19T09:01:43.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.355+0000 7fbf41ffb640 1 -- 172.21.15.120:0/4250332389 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf3c005020 con 0x7fbf44102ef0 2024-09-19T09:01:43.208 INFO:teuthology.orchestra.run.smithi120.stdout:236223201319 2024-09-19T09:01:43.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.355+0000 7fbf41ffb640 1 -- 172.21.15.120:0/4250332389 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c00f6f0 con 0x7fbf44102ef0 2024-09-19T09:01:43.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.355+0000 7fbf4af4a640 1 -- 172.21.15.120:0/4250332389 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf441a5070 con 0x7fbf44102ef0 2024-09-19T09:01:43.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.356+0000 7fbf4af4a640 1 -- 172.21.15.120:0/4250332389 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf44101200 con 0x7fbf44102ef0 2024-09-19T09:01:43.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.357+0000 7fbf41ffb640 1 -- 172.21.15.120:0/4250332389 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fbf3c0051d0 con 0x7fbf44102ef0 2024-09-19T09:01:43.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.358+0000 7fbf41ffb640 1 --2- 172.21.15.120:0/4250332389 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbf340a32d0 0x7fbf340a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:43.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.358+0000 7fbf48ce6640 1 --2- 172.21.15.120:0/4250332389 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbf340a32d0 0x7fbf340a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:43.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.358+0000 7fbf41ffb640 1 -- 172.21.15.120:0/4250332389 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fbf3c0cf990 con 0x7fbf44102ef0 2024-09-19T09:01:43.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.358+0000 7fbf48ce6640 1 --2- 172.21.15.120:0/4250332389 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbf340a32d0 0x7fbf340a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbf38005ab0 tx=0x7fbf38005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:43.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.362+0000 7fbf41ffb640 1 -- 172.21.15.120:0/4250332389 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbf3c014030 con 0x7fbf44102ef0 2024-09-19T09:01:43.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.543+0000 7fbf4af4a640 1 -- 172.21.15.120:0/4250332389 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fbf44109090 con 0x7fbf44102ef0 2024-09-19T09:01:43.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.544+0000 7fbf41ffb640 1 -- 172.21.15.120:0/4250332389 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbf3c09c700 con 0x7fbf44102ef0 2024-09-19T09:01:43.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.546+0000 7fbf4af4a640 1 -- 172.21.15.120:0/4250332389 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbf340a32d0 msgr2=0x7fbf340a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:43.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.546+0000 7fbf4af4a640 1 --2- 172.21.15.120:0/4250332389 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbf340a32d0 0x7fbf340a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbf38005ab0 tx=0x7fbf38005a20 comp rx=0 tx=0).stop 2024-09-19T09:01:43.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.546+0000 7fbf4af4a640 1 -- 172.21.15.120:0/4250332389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf44102ef0 msgr2=0x7fbf441a4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:43.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.546+0000 7fbf4af4a640 1 --2- 172.21.15.120:0/4250332389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf44102ef0 0x7fbf441a4450 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00cad0 tx=0x7fbf3c00cf90 comp rx=0 tx=0).stop 2024-09-19T09:01:43.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.546+0000 7fbf4af4a640 1 -- 172.21.15.120:0/4250332389 shutdown_connections 2024-09-19T09:01:43.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.546+0000 7fbf4af4a640 1 --2- 172.21.15.120:0/4250332389 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbf340a32d0 0x7fbf340a5790 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:43.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.546+0000 7fbf4af4a640 1 --2- 172.21.15.120:0/4250332389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf44102ef0 0x7fbf441a4450 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:43.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.546+0000 7fbf4af4a640 1 --2- 172.21.15.120:0/4250332389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf44102570 0x7fbf44199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:43.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.546+0000 7fbf4af4a640 1 --2- 172.21.15.120:0/4250332389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf44100e20 0x7fbf441993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:43.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.546+0000 7fbf4af4a640 1 -- 172.21.15.120:0/4250332389 >> 172.21.15.120:0/4250332389 conn(0x7fbf440fc7d0 msgr2=0x7fbf440fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:43.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.546+0000 7fbf4af4a640 1 -- 172.21.15.120:0/4250332389 shutdown_connections 2024-09-19T09:01:43.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:42.546+0000 7fbf4af4a640 1 -- 172.21.15.120:0/4250332389 wait complete. 2024-09-19T09:01:43.803 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593775 got 184683593778 for osd.6 2024-09-19T09:01:43.803 DEBUG:teuthology.parallel:result is None 2024-09-19T09:01:44.811 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972383 got 266287972386 for osd.9 2024-09-19T09:01:44.811 DEBUG:teuthology.parallel:result is None 2024-09-19T09:01:44.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:44 smithi196 ceph-mon[35150]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T09:01:44.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:44 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:01:44.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:44 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2036995598' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T09:01:44.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:44 smithi153 ceph-mon[35071]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T09:01:44.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:44 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:01:44.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:44 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2036995598' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T09:01:45.035 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:44 smithi120 ceph-mon[31150]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T09:01:45.035 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:44 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:01:45.035 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:44 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2036995598' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T09:01:45.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.114+0000 7f61bb3ee640 1 -- 172.21.15.120:0/2302665776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61b410c180 msgr2=0x7f61b410e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:45.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.114+0000 7f61bb3ee640 1 --2- 172.21.15.120:0/2302665776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61b410c180 0x7f61b410e600 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f61ac009ca0 tx=0x7f61ac02a950 comp rx=0 tx=0).stop 2024-09-19T09:01:45.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.114+0000 7f61bb3ee640 1 -- 172.21.15.120:0/2302665776 shutdown_connections 2024-09-19T09:01:45.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.114+0000 7f61bb3ee640 1 --2- 172.21.15.120:0/2302665776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61b410c180 0x7f61b410e600 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:45.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.114+0000 7f61bb3ee640 1 --2- 172.21.15.120:0/2302665776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61b41036c0 0x7f61b410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:45.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.114+0000 7f61bb3ee640 1 --2- 172.21.15.120:0/2302665776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61b4102da0 0x7f61b4103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:45.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.114+0000 7f61bb3ee640 1 -- 172.21.15.120:0/2302665776 >> 172.21.15.120:0/2302665776 conn(0x7f61b40fc7a0 msgr2=0x7f61b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:45.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.115+0000 7f61bb3ee640 1 -- 172.21.15.120:0/2302665776 shutdown_connections 2024-09-19T09:01:45.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.115+0000 7f61bb3ee640 1 -- 172.21.15.120:0/2302665776 wait complete. 2024-09-19T09:01:45.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.115+0000 7f61bb3ee640 1 Processor -- start 2024-09-19T09:01:45.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.115+0000 7f61bb3ee640 1 -- start start 2024-09-19T09:01:45.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.115+0000 7f61bb3ee640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61b4102da0 0x7f61b4100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:45.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.115+0000 7f61bb3ee640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61b41036c0 0x7f61b4100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:45.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.115+0000 7f61bb3ee640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61b410c180 0x7f61b4101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:45.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.115+0000 7f61bb3ee640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f61b4111f60 con 0x7f61b410c180 2024-09-19T09:01:45.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.115+0000 7f61bb3ee640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f61b4111de0 con 0x7f61b41036c0 2024-09-19T09:01:45.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.115+0000 7f61bb3ee640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61b41120e0 con 0x7f61b4102da0 2024-09-19T09:01:45.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.116+0000 7f61b998b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61b410c180 0x7f61b4101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:45.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.116+0000 7f61b8989640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61b41036c0 0x7f61b4100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:45.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.116+0000 7f61b998b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61b410c180 0x7f61b4101480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59626/0 (socket says 172.21.15.120:59626) 2024-09-19T09:01:45.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.116+0000 7f61b918a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61b4102da0 0x7f61b4100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:45.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.116+0000 7f61b998b640 1 -- 172.21.15.120:0/55626255 learned_addr learned my addr 172.21.15.120:0/55626255 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:45.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.116+0000 7f61b998b640 1 -- 172.21.15.120:0/55626255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61b4102da0 msgr2=0x7f61b4100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:45.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.116+0000 7f61b998b640 1 --2- 172.21.15.120:0/55626255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61b4102da0 0x7f61b4100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:45.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.116+0000 7f61b998b640 1 -- 172.21.15.120:0/55626255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61b41036c0 msgr2=0x7f61b4100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:45.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.116+0000 7f61b998b640 1 --2- 172.21.15.120:0/55626255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61b41036c0 0x7f61b4100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:45.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.116+0000 7f61b998b640 1 -- 172.21.15.120:0/55626255 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b41a6900 con 0x7f61b410c180 2024-09-19T09:01:45.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.116+0000 7f61b998b640 1 --2- 172.21.15.120:0/55626255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61b410c180 0x7f61b4101480 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f61ac02ae70 tx=0x7f61ac02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:45.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.116+0000 7f61a67fc640 1 -- 172.21.15.120:0/55626255 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ac034040 con 0x7f61b410c180 2024-09-19T09:01:45.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.116+0000 7f61bb3ee640 1 -- 172.21.15.120:0/55626255 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b41a6b90 con 0x7f61b410c180 2024-09-19T09:01:45.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.116+0000 7f61a67fc640 1 -- 172.21.15.120:0/55626255 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61ac030920 con 0x7f61b410c180 2024-09-19T09:01:45.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.116+0000 7f61a67fc640 1 -- 172.21.15.120:0/55626255 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ac045480 con 0x7f61b410c180 2024-09-19T09:01:45.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.116+0000 7f61bb3ee640 1 -- 172.21.15.120:0/55626255 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b41a7070 con 0x7f61b410c180 2024-09-19T09:01:45.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.117+0000 7f61bb3ee640 1 -- 172.21.15.120:0/55626255 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61b4103180 con 0x7f61b410c180 2024-09-19T09:01:45.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.118+0000 7f61a67fc640 1 -- 172.21.15.120:0/55626255 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f61ac037020 con 0x7f61b410c180 2024-09-19T09:01:45.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.119+0000 7f61a67fc640 1 --2- 172.21.15.120:0/55626255 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f61a00a32d0 0x7f61a00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:45.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.119+0000 7f61b918a640 1 --2- 172.21.15.120:0/55626255 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f61a00a32d0 0x7f61a00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:45.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.119+0000 7f61a67fc640 1 -- 172.21.15.120:0/55626255 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f61ac0f07f0 con 0x7f61b410c180 2024-09-19T09:01:45.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.119+0000 7f61b918a640 1 --2- 172.21.15.120:0/55626255 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f61a00a32d0 0x7f61a00a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f61a8005fd0 tx=0x7f61a800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:45.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.122+0000 7f61a67fc640 1 -- 172.21.15.120:0/55626255 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f61ac081210 con 0x7f61b410c180 2024-09-19T09:01:45.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.294+0000 7f61bb3ee640 1 -- 172.21.15.120:0/55626255 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f61b4107220 con 0x7f61b410c180 2024-09-19T09:01:45.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.294+0000 7f61a67fc640 1 -- 172.21.15.120:0/55626255 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f61ac0bd670 con 0x7f61b410c180 2024-09-19T09:01:45.295 INFO:teuthology.orchestra.run.smithi120.stdout:206158430253 2024-09-19T09:01:45.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.297+0000 7f61bb3ee640 1 -- 172.21.15.120:0/55626255 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f61a00a32d0 msgr2=0x7f61a00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:45.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.297+0000 7f61bb3ee640 1 --2- 172.21.15.120:0/55626255 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f61a00a32d0 0x7f61a00a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f61a8005fd0 tx=0x7f61a800b390 comp rx=0 tx=0).stop 2024-09-19T09:01:45.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.297+0000 7f61bb3ee640 1 -- 172.21.15.120:0/55626255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61b410c180 msgr2=0x7f61b4101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:45.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.297+0000 7f61bb3ee640 1 --2- 172.21.15.120:0/55626255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61b410c180 0x7f61b4101480 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f61ac02ae70 tx=0x7f61ac02f5b0 comp rx=0 tx=0).stop 2024-09-19T09:01:45.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.297+0000 7f61bb3ee640 1 -- 172.21.15.120:0/55626255 shutdown_connections 2024-09-19T09:01:45.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.297+0000 7f61bb3ee640 1 --2- 172.21.15.120:0/55626255 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f61a00a32d0 0x7f61a00a5790 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:45.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.297+0000 7f61bb3ee640 1 --2- 172.21.15.120:0/55626255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61b410c180 0x7f61b4101480 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:45.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.297+0000 7f61bb3ee640 1 --2- 172.21.15.120:0/55626255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61b41036c0 0x7f61b4100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:45.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.297+0000 7f61bb3ee640 1 --2- 172.21.15.120:0/55626255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61b4102da0 0x7f61b4100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:45.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.297+0000 7f61bb3ee640 1 -- 172.21.15.120:0/55626255 >> 172.21.15.120:0/55626255 conn(0x7f61b40fc7a0 msgr2=0x7f61b40fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:45.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.297+0000 7f61bb3ee640 1 -- 172.21.15.120:0/55626255 shutdown_connections 2024-09-19T09:01:45.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:45.297+0000 7f61bb3ee640 1 -- 172.21.15.120:0/55626255 wait complete. 2024-09-19T09:01:45.777 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612630 got 313532612631 for osd.11 2024-09-19T09:01:45.777 DEBUG:teuthology.parallel:result is None 2024-09-19T09:01:46.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:45 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/55626255' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T09:01:46.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:45 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:01:46.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:45 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:01:46.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:45 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:01:46.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:45 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:01:46.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:45 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/55626255' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T09:01:46.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:45 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:01:46.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:45 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:01:46.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:45 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:01:46.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:45 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:01:46.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:45 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/55626255' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T09:01:46.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:45 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:01:46.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:45 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:01:46.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:45 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:01:46.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:45 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:01:46.225 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201317 got 236223201319 for osd.8 2024-09-19T09:01:46.225 DEBUG:teuthology.parallel:result is None 2024-09-19T09:01:46.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.252+0000 7f9af997e640 1 -- 172.21.15.120:0/678907132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9af4102ef0 msgr2=0x7f9af410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:46.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.252+0000 7f9af997e640 1 --2- 172.21.15.120:0/678907132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9af4102ef0 0x7f9af410da40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9aec009900 tx=0x7f9aec02a480 comp rx=0 tx=0).stop 2024-09-19T09:01:46.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.252+0000 7f9af997e640 1 -- 172.21.15.120:0/678907132 shutdown_connections 2024-09-19T09:01:46.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.252+0000 7f9af997e640 1 --2- 172.21.15.120:0/678907132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9af4102ef0 0x7f9af410da40 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:46.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.252+0000 7f9af997e640 1 --2- 172.21.15.120:0/678907132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9af4102570 0x7f9af41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:46.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.252+0000 7f9af997e640 1 --2- 172.21.15.120:0/678907132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af4100e20 0x7f9af4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:46.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.252+0000 7f9af997e640 1 -- 172.21.15.120:0/678907132 >> 172.21.15.120:0/678907132 conn(0x7f9af40fc7d0 msgr2=0x7f9af40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:46.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.252+0000 7f9af997e640 1 -- 172.21.15.120:0/678907132 shutdown_connections 2024-09-19T09:01:46.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.252+0000 7f9af997e640 1 -- 172.21.15.120:0/678907132 wait complete. 2024-09-19T09:01:46.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.253+0000 7f9af997e640 1 Processor -- start 2024-09-19T09:01:46.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.253+0000 7f9af997e640 1 -- start start 2024-09-19T09:01:46.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.254+0000 7f9af997e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9af4100e20 0x7f9af41984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:46.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.254+0000 7f9af997e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9af4102570 0x7f9af4198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:46.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.254+0000 7f9af997e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af4102ef0 0x7f9af419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:46.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.254+0000 7f9af997e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41120c0 con 0x7f9af4102570 2024-09-19T09:01:46.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.254+0000 7f9af997e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9af4111f40 con 0x7f9af4102ef0 2024-09-19T09:01:46.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.254+0000 7f9af997e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9af4112240 con 0x7f9af4100e20 2024-09-19T09:01:46.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.254+0000 7f9af27fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9af4102570 0x7f9af4198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:46.330 INFO:teuthology.orchestra.run.smithi120.stdout:103079215169 2024-09-19T09:01:46.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.254+0000 7f9af27fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9af4102570 0x7f9af4198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59604/0 (socket says 172.21.15.120:59604) 2024-09-19T09:01:46.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.254+0000 7f9af27fc640 1 -- 172.21.15.120:0/2036995598 learned_addr learned my addr 172.21.15.120:0/2036995598 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:46.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.254+0000 7f9af2ffd640 1 --2- 172.21.15.120:0/2036995598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9af4100e20 0x7f9af41984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:46.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.254+0000 7f9af37fe640 1 --2- 172.21.15.120:0/2036995598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af4102ef0 0x7f9af419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:46.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.254+0000 7f9af27fc640 1 -- 172.21.15.120:0/2036995598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9af4100e20 msgr2=0x7f9af41984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:46.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.254+0000 7f9af27fc640 1 --2- 172.21.15.120:0/2036995598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9af4100e20 0x7f9af41984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:46.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.254+0000 7f9af27fc640 1 -- 172.21.15.120:0/2036995598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af4102ef0 msgr2=0x7f9af419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:46.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.254+0000 7f9af27fc640 1 --2- 172.21.15.120:0/2036995598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af4102ef0 0x7f9af419cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:46.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.254+0000 7f9af27fc640 1 -- 172.21.15.120:0/2036995598 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af419d470 con 0x7f9af4102570 2024-09-19T09:01:46.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.254+0000 7f9af27fc640 1 --2- 172.21.15.120:0/2036995598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9af4102570 0x7f9af4198a00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800d770 tx=0x7f9ae800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:46.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.254+0000 7f9af897c640 1 -- 172.21.15.120:0/2036995598 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8017070 con 0x7f9af4102570 2024-09-19T09:01:46.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.254+0000 7f9af897c640 1 -- 172.21.15.120:0/2036995598 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ae8005500 con 0x7f9af4102570 2024-09-19T09:01:46.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.254+0000 7f9af997e640 1 -- 172.21.15.120:0/2036995598 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af419d760 con 0x7f9af4102570 2024-09-19T09:01:46.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.254+0000 7f9af897c640 1 -- 172.21.15.120:0/2036995598 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8004e00 con 0x7f9af4102570 2024-09-19T09:01:46.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.254+0000 7f9af997e640 1 -- 172.21.15.120:0/2036995598 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af41a5040 con 0x7f9af4102570 2024-09-19T09:01:46.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.255+0000 7f9af997e640 1 -- 172.21.15.120:0/2036995598 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9af4101200 con 0x7f9af4102570 2024-09-19T09:01:46.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.256+0000 7f9af897c640 1 -- 172.21.15.120:0/2036995598 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f9ae8004fa0 con 0x7f9af4102570 2024-09-19T09:01:46.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.257+0000 7f9af897c640 1 --2- 172.21.15.120:0/2036995598 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9adc0a32d0 0x7f9adc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:46.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.257+0000 7f9af2ffd640 1 --2- 172.21.15.120:0/2036995598 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9adc0a32d0 0x7f9adc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:46.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.257+0000 7f9af897c640 1 -- 172.21.15.120:0/2036995598 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f9ae80cf2d0 con 0x7f9af4102570 2024-09-19T09:01:46.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.258+0000 7f9af2ffd640 1 --2- 172.21.15.120:0/2036995598 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9adc0a32d0 0x7f9adc0a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9ae400a400 tx=0x7f9ae400d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:46.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.262+0000 7f9af897c640 1 -- 172.21.15.120:0/2036995598 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9ae809b9f0 con 0x7f9af4102570 2024-09-19T09:01:46.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.436+0000 7f9af997e640 1 -- 172.21.15.120:0/2036995598 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f9af41a5210 con 0x7f9af4102570 2024-09-19T09:01:46.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.436+0000 7f9af897c640 1 -- 172.21.15.120:0/2036995598 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9ae809fcf0 con 0x7f9af4102570 2024-09-19T09:01:46.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.438+0000 7f9af997e640 1 -- 172.21.15.120:0/2036995598 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9adc0a32d0 msgr2=0x7f9adc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:46.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.438+0000 7f9af997e640 1 --2- 172.21.15.120:0/2036995598 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9adc0a32d0 0x7f9adc0a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9ae400a400 tx=0x7f9ae400d040 comp rx=0 tx=0).stop 2024-09-19T09:01:46.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.438+0000 7f9af997e640 1 -- 172.21.15.120:0/2036995598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9af4102570 msgr2=0x7f9af4198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:46.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.438+0000 7f9af997e640 1 --2- 172.21.15.120:0/2036995598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9af4102570 0x7f9af4198a00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800d770 tx=0x7f9ae800dc30 comp rx=0 tx=0).stop 2024-09-19T09:01:46.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.438+0000 7f9af997e640 1 -- 172.21.15.120:0/2036995598 shutdown_connections 2024-09-19T09:01:46.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.438+0000 7f9af997e640 1 --2- 172.21.15.120:0/2036995598 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9adc0a32d0 0x7f9adc0a5790 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:46.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.438+0000 7f9af997e640 1 --2- 172.21.15.120:0/2036995598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af4102ef0 0x7f9af419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:46.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.438+0000 7f9af997e640 1 --2- 172.21.15.120:0/2036995598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9af4102570 0x7f9af4198a00 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:46.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.438+0000 7f9af997e640 1 --2- 172.21.15.120:0/2036995598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9af4100e20 0x7f9af41984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:46.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.438+0000 7f9af997e640 1 -- 172.21.15.120:0/2036995598 >> 172.21.15.120:0/2036995598 conn(0x7f9af40fc7d0 msgr2=0x7f9af4106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:46.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.438+0000 7f9af997e640 1 -- 172.21.15.120:0/2036995598 shutdown_connections 2024-09-19T09:01:46.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:44.438+0000 7f9af997e640 1 -- 172.21.15.120:0/2036995598 wait complete. 2024-09-19T09:01:47.016 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:46 smithi120 ceph-mon[31150]: pgmap v306: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:01:47.050 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738446 got 34359738448 for osd.0 2024-09-19T09:01:47.051 DEBUG:teuthology.parallel:result is None 2024-09-19T09:01:47.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:46 smithi196 ceph-mon[35150]: pgmap v306: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:01:47.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:46 smithi153 ceph-mon[35071]: pgmap v306: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:01:47.807 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430250 got 206158430253 for osd.7 2024-09-19T09:01:47.807 DEBUG:teuthology.parallel:result is None 2024-09-19T09:01:48.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:47 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/318610386' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T09:01:48.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:47 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/318610386' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T09:01:48.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:47 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/318610386' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T09:01:48.249 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215167 got 103079215169 for osd.3 2024-09-19T09:01:48.250 DEBUG:teuthology.parallel:result is None 2024-09-19T09:01:48.738 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822708 got 154618822710 for osd.5 2024-09-19T09:01:48.739 DEBUG:teuthology.parallel:result is None 2024-09-19T09:01:48.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.219+0000 7fd4bef84640 1 -- 172.21.15.120:0/3316861687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4b8101420 msgr2=0x7fd4b810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:48.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.219+0000 7fd4bef84640 1 --2- 172.21.15.120:0/3316861687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4b8101420 0x7fd4b810d950 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0009cd0 tx=0x7fd4b002a8e0 comp rx=0 tx=0).stop 2024-09-19T09:01:48.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.219+0000 7fd4bef84640 1 -- 172.21.15.120:0/3316861687 shutdown_connections 2024-09-19T09:01:48.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.219+0000 7fd4bef84640 1 --2- 172.21.15.120:0/3316861687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4b8101420 0x7fd4b810d950 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:48.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.219+0000 7fd4bef84640 1 --2- 172.21.15.120:0/3316861687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4b8100a80 0x7fd4b8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:48.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.219+0000 7fd4bef84640 1 --2- 172.21.15.120:0/3316861687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4b8106a80 0x7fd4b8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:48.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.219+0000 7fd4bef84640 1 -- 172.21.15.120:0/3316861687 >> 172.21.15.120:0/3316861687 conn(0x7fd4b80fc7a0 msgr2=0x7fd4b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:48.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.219+0000 7fd4bef84640 1 -- 172.21.15.120:0/3316861687 shutdown_connections 2024-09-19T09:01:48.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.219+0000 7fd4bef84640 1 -- 172.21.15.120:0/3316861687 wait complete. 2024-09-19T09:01:48.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.220+0000 7fd4bef84640 1 Processor -- start 2024-09-19T09:01:48.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.220+0000 7fd4bef84640 1 -- start start 2024-09-19T09:01:48.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.220+0000 7fd4bef84640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4b8100a80 0x7fd4b81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:48.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.220+0000 7fd4bef84640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4b8101420 0x7fd4b8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:48.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.220+0000 7fd4bef84640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4b8106a80 0x7fd4b819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:48.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.220+0000 7fd4bef84640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b8110ea0 con 0x7fd4b8101420 2024-09-19T09:01:48.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.220+0000 7fd4bef84640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b8110d20 con 0x7fd4b8106a80 2024-09-19T09:01:48.872 INFO:teuthology.orchestra.run.smithi120.stdout:55834574923 2024-09-19T09:01:48.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.220+0000 7fd4bef84640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b8111020 con 0x7fd4b8100a80 2024-09-19T09:01:48.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.220+0000 7fd4b7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4b8101420 0x7fd4b8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:48.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.220+0000 7fd4b7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4b8101420 0x7fd4b8198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59642/0 (socket says 172.21.15.120:59642) 2024-09-19T09:01:48.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.220+0000 7fd4b7fff640 1 -- 172.21.15.120:0/318610386 learned_addr learned my addr 172.21.15.120:0/318610386 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:48.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.220+0000 7fd4bd521640 1 --2- 172.21.15.120:0/318610386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4b8106a80 0x7fd4b819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:48.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.220+0000 7fd4b7fff640 1 -- 172.21.15.120:0/318610386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4b8100a80 msgr2=0x7fd4b81983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:48.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.220+0000 7fd4b7fff640 1 --2- 172.21.15.120:0/318610386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4b8100a80 0x7fd4b81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:48.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.220+0000 7fd4b7fff640 1 -- 172.21.15.120:0/318610386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4b8106a80 msgr2=0x7fd4b819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:48.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.220+0000 7fd4b7fff640 1 --2- 172.21.15.120:0/318610386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4b8106a80 0x7fd4b819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:48.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.220+0000 7fd4b7fff640 1 -- 172.21.15.120:0/318610386 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b819d3a0 con 0x7fd4b8101420 2024-09-19T09:01:48.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.220+0000 7fd4bd521640 1 --2- 172.21.15.120:0/318610386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4b8106a80 0x7fd4b819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:01:48.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.220+0000 7fd4b7fff640 1 --2- 172.21.15.120:0/318610386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4b8101420 0x7fd4b8198930 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd4a400c920 tx=0x7fd4a400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:48.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.221+0000 7fd4b5ffb640 1 -- 172.21.15.120:0/318610386 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4007c70 con 0x7fd4b8101420 2024-09-19T09:01:48.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.221+0000 7fd4b5ffb640 1 -- 172.21.15.120:0/318610386 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4a4007e10 con 0x7fd4b8101420 2024-09-19T09:01:48.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.221+0000 7fd4bef84640 1 -- 172.21.15.120:0/318610386 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4b819d690 con 0x7fd4b8101420 2024-09-19T09:01:48.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.221+0000 7fd4b5ffb640 1 -- 172.21.15.120:0/318610386 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4018890 con 0x7fd4b8101420 2024-09-19T09:01:48.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.221+0000 7fd4bef84640 1 -- 172.21.15.120:0/318610386 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4b81a4f70 con 0x7fd4b8101420 2024-09-19T09:01:48.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.222+0000 7fd4bef84640 1 -- 172.21.15.120:0/318610386 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4b8100ee0 con 0x7fd4b8101420 2024-09-19T09:01:48.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.223+0000 7fd4b5ffb640 1 -- 172.21.15.120:0/318610386 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fd4a4018a70 con 0x7fd4b8101420 2024-09-19T09:01:48.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.224+0000 7fd4b5ffb640 1 --2- 172.21.15.120:0/318610386 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd4a80a3350 0x7fd4a80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:48.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.224+0000 7fd4bcd20640 1 --2- 172.21.15.120:0/318610386 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd4a80a3350 0x7fd4a80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:48.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.224+0000 7fd4b5ffb640 1 -- 172.21.15.120:0/318610386 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fd4a40cfac0 con 0x7fd4b8101420 2024-09-19T09:01:48.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.224+0000 7fd4bcd20640 1 --2- 172.21.15.120:0/318610386 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd4a80a3350 0x7fd4a80a5810 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac009710 tx=0x7fd4ac006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:48.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.228+0000 7fd4b5ffb640 1 -- 172.21.15.120:0/318610386 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd4a4014030 con 0x7fd4b8101420 2024-09-19T09:01:48.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.399+0000 7fd4bef84640 1 -- 172.21.15.120:0/318610386 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fd4b810a080 con 0x7fd4b8101420 2024-09-19T09:01:48.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.400+0000 7fd4b5ffb640 1 -- 172.21.15.120:0/318610386 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd4a409c830 con 0x7fd4b8101420 2024-09-19T09:01:48.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.402+0000 7fd4bef84640 1 -- 172.21.15.120:0/318610386 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd4a80a3350 msgr2=0x7fd4a80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:48.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.402+0000 7fd4bef84640 1 --2- 172.21.15.120:0/318610386 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd4a80a3350 0x7fd4a80a5810 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac009710 tx=0x7fd4ac006c90 comp rx=0 tx=0).stop 2024-09-19T09:01:48.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.402+0000 7fd4bef84640 1 -- 172.21.15.120:0/318610386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4b8101420 msgr2=0x7fd4b8198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:48.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.402+0000 7fd4bef84640 1 --2- 172.21.15.120:0/318610386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4b8101420 0x7fd4b8198930 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd4a400c920 tx=0x7fd4a400cde0 comp rx=0 tx=0).stop 2024-09-19T09:01:48.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.402+0000 7fd4bef84640 1 -- 172.21.15.120:0/318610386 shutdown_connections 2024-09-19T09:01:48.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.402+0000 7fd4bef84640 1 --2- 172.21.15.120:0/318610386 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd4a80a3350 0x7fd4a80a5810 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:48.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.402+0000 7fd4bef84640 1 --2- 172.21.15.120:0/318610386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4b8106a80 0x7fd4b819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:48.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.402+0000 7fd4bef84640 1 --2- 172.21.15.120:0/318610386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4b8101420 0x7fd4b8198930 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:48.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.402+0000 7fd4bef84640 1 --2- 172.21.15.120:0/318610386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4b8100a80 0x7fd4b81983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:48.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.402+0000 7fd4bef84640 1 -- 172.21.15.120:0/318610386 >> 172.21.15.120:0/318610386 conn(0x7fd4b80fc7a0 msgr2=0x7fd4b806a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:48.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.402+0000 7fd4bef84640 1 -- 172.21.15.120:0/318610386 shutdown_connections 2024-09-19T09:01:48.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:47.402+0000 7fd4bef84640 1 -- 172.21.15.120:0/318610386 wait complete. 2024-09-19T09:01:49.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:48 smithi120 ceph-mon[31150]: pgmap v307: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T09:01:49.088 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:49.089 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:49.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:48 smithi196 ceph-mon[35150]: pgmap v307: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T09:01:49.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:48 smithi153 ceph-mon[35071]: pgmap v307: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T09:01:49.543 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574920 got 55834574923 for osd.1 2024-09-19T09:01:49.543 DEBUG:teuthology.parallel:result is None 2024-09-19T09:01:50.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.242+0000 7f17a3850640 1 -- 172.21.15.120:0/2316910066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f179c102570 msgr2=0x7f179c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:50.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.242+0000 7f17a3850640 1 --2- 172.21.15.120:0/2316910066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f179c102570 0x7f179c1029b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1790009900 tx=0x7f179002a480 comp rx=0 tx=0).stop 2024-09-19T09:01:50.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.242+0000 7f17a3850640 1 -- 172.21.15.120:0/2316910066 shutdown_connections 2024-09-19T09:01:50.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.242+0000 7f17a3850640 1 --2- 172.21.15.120:0/2316910066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f179c102ef0 0x7f179c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:50.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.242+0000 7f17a3850640 1 --2- 172.21.15.120:0/2316910066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f179c102570 0x7f179c1029b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:50.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.242+0000 7f17a3850640 1 --2- 172.21.15.120:0/2316910066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f179c100e20 0x7f179c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:50.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.242+0000 7f17a3850640 1 -- 172.21.15.120:0/2316910066 >> 172.21.15.120:0/2316910066 conn(0x7f179c0fc7d0 msgr2=0x7f179c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:50.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.242+0000 7f17a3850640 1 -- 172.21.15.120:0/2316910066 shutdown_connections 2024-09-19T09:01:50.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.242+0000 7f17a3850640 1 -- 172.21.15.120:0/2316910066 wait complete. 2024-09-19T09:01:50.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.243+0000 7f17a3850640 1 Processor -- start 2024-09-19T09:01:50.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.243+0000 7f17a3850640 1 -- start start 2024-09-19T09:01:50.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.243+0000 7f17a3850640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f179c100e20 0x7f179c1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.243+0000 7f17a3850640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f179c102570 0x7f179c198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.243+0000 7f17a3850640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f179c102ef0 0x7f179c19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.243+0000 7f17a3850640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f179c110f80 con 0x7f179c100e20 2024-09-19T09:01:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.243+0000 7f17a3850640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f179c110e00 con 0x7f179c102ef0 2024-09-19T09:01:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.243+0000 7f17a3850640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f179c111100 con 0x7f179c102570 2024-09-19T09:01:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.243+0000 7f17a15ec640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f179c100e20 0x7f179c1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.243+0000 7f17a15ec640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f179c100e20 0x7f179c1985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59660/0 (socket says 172.21.15.120:59660) 2024-09-19T09:01:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.243+0000 7f17a15ec640 1 -- 172.21.15.120:0/2970428957 learned_addr learned my addr 172.21.15.120:0/2970428957 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.243+0000 7f17a0deb640 1 --2- 172.21.15.120:0/2970428957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f179c102570 0x7f179c198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.243+0000 7f17a1ded640 1 --2- 172.21.15.120:0/2970428957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f179c102ef0 0x7f179c19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.243+0000 7f17a15ec640 1 -- 172.21.15.120:0/2970428957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f179c102570 msgr2=0x7f179c198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.243+0000 7f17a15ec640 1 --2- 172.21.15.120:0/2970428957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f179c102570 0x7f179c198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.243+0000 7f17a15ec640 1 -- 172.21.15.120:0/2970428957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f179c102ef0 msgr2=0x7f179c19ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.243+0000 7f17a15ec640 1 --2- 172.21.15.120:0/2970428957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f179c102ef0 0x7f179c19ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.243+0000 7f17a15ec640 1 -- 172.21.15.120:0/2970428957 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179c19d550 con 0x7f179c100e20 2024-09-19T09:01:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.243+0000 7f17a15ec640 1 --2- 172.21.15.120:0/2970428957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f179c100e20 0x7f179c1985a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f179800d950 tx=0x7f179800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.244+0000 7f178e7fc640 1 -- 172.21.15.120:0/2970428957 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1798017070 con 0x7f179c100e20 2024-09-19T09:01:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.244+0000 7f17a3850640 1 -- 172.21.15.120:0/2970428957 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f179c19d840 con 0x7f179c100e20 2024-09-19T09:01:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.244+0000 7f178e7fc640 1 -- 172.21.15.120:0/2970428957 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1798009c30 con 0x7f179c100e20 2024-09-19T09:01:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.244+0000 7f178e7fc640 1 -- 172.21.15.120:0/2970428957 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179800f6b0 con 0x7f179c100e20 2024-09-19T09:01:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.244+0000 7f17a3850640 1 -- 172.21.15.120:0/2970428957 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f179c1a5120 con 0x7f179c100e20 2024-09-19T09:01:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.245+0000 7f17a3850640 1 -- 172.21.15.120:0/2970428957 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f179c101200 con 0x7f179c100e20 2024-09-19T09:01:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.246+0000 7f178e7fc640 1 -- 172.21.15.120:0/2970428957 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f17980109d0 con 0x7f179c100e20 2024-09-19T09:01:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.247+0000 7f178e7fc640 1 --2- 172.21.15.120:0/2970428957 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f17880a32d0 0x7f17880a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.247+0000 7f17a0deb640 1 --2- 172.21.15.120:0/2970428957 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f17880a32d0 0x7f17880a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.247+0000 7f178e7fc640 1 -- 172.21.15.120:0/2970428957 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f17980cf710 con 0x7f179c100e20 2024-09-19T09:01:50.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.247+0000 7f17a0deb640 1 --2- 172.21.15.120:0/2970428957 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f17880a32d0 0x7f17880a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f179c1000b0 tx=0x7f1790005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:50.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.251+0000 7f178e7fc640 1 -- 172.21.15.120:0/2970428957 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1798009de0 con 0x7f179c100e20 2024-09-19T09:01:50.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.426+0000 7f17a3850640 1 -- 172.21.15.120:0/2970428957 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f179c10a0a0 con 0x7f179c100e20 2024-09-19T09:01:50.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.427+0000 7f178e7fc640 1 -- 172.21.15.120:0/2970428957 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f179809c380 con 0x7f179c100e20 2024-09-19T09:01:50.428 INFO:teuthology.orchestra.run.smithi120.stdout:133143986238 2024-09-19T09:01:50.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.429+0000 7f17a3850640 1 -- 172.21.15.120:0/2970428957 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f17880a32d0 msgr2=0x7f17880a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:50.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.429+0000 7f17a3850640 1 --2- 172.21.15.120:0/2970428957 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f17880a32d0 0x7f17880a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f179c1000b0 tx=0x7f1790005a20 comp rx=0 tx=0).stop 2024-09-19T09:01:50.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.429+0000 7f17a3850640 1 -- 172.21.15.120:0/2970428957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f179c100e20 msgr2=0x7f179c1985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:50.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.429+0000 7f17a3850640 1 --2- 172.21.15.120:0/2970428957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f179c100e20 0x7f179c1985a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f179800d950 tx=0x7f179800de10 comp rx=0 tx=0).stop 2024-09-19T09:01:50.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.429+0000 7f17a3850640 1 -- 172.21.15.120:0/2970428957 shutdown_connections 2024-09-19T09:01:50.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.429+0000 7f17a3850640 1 --2- 172.21.15.120:0/2970428957 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f17880a32d0 0x7f17880a5790 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:50.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.429+0000 7f17a3850640 1 --2- 172.21.15.120:0/2970428957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f179c102ef0 0x7f179c19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:50.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.429+0000 7f17a3850640 1 --2- 172.21.15.120:0/2970428957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f179c102570 0x7f179c198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:50.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.429+0000 7f17a3850640 1 --2- 172.21.15.120:0/2970428957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f179c100e20 0x7f179c1985a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:50.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.429+0000 7f17a3850640 1 -- 172.21.15.120:0/2970428957 >> 172.21.15.120:0/2970428957 conn(0x7f179c0fc7d0 msgr2=0x7f179c06a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:50.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.430+0000 7f17a3850640 1 -- 172.21.15.120:0/2970428957 shutdown_connections 2024-09-19T09:01:50.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.430+0000 7f17a3850640 1 -- 172.21.15.120:0/2970428957 wait complete. 2024-09-19T09:01:50.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.485+0000 7f0994343640 1 -- 172.21.15.120:0/1934113126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f098c1036e0 msgr2=0x7f098c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:50.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.485+0000 7f0994343640 1 --2- 172.21.15.120:0/1934113126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f098c1036e0 0x7f098c107b90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f097c009930 tx=0x7f097c02a420 comp rx=0 tx=0).stop 2024-09-19T09:01:50.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.485+0000 7f0994343640 1 -- 172.21.15.120:0/1934113126 shutdown_connections 2024-09-19T09:01:50.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.485+0000 7f0994343640 1 --2- 172.21.15.120:0/1934113126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f098c1082a0 0x7f098c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:50.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.485+0000 7f0994343640 1 --2- 172.21.15.120:0/1934113126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f098c1036e0 0x7f098c107b90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:50.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.485+0000 7f0994343640 1 --2- 172.21.15.120:0/1934113126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f098c102d30 0x7f098c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:50.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.485+0000 7f0994343640 1 -- 172.21.15.120:0/1934113126 >> 172.21.15.120:0/1934113126 conn(0x7f098c0fc7d0 msgr2=0x7f098c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:50.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.485+0000 7f0994343640 1 -- 172.21.15.120:0/1934113126 shutdown_connections 2024-09-19T09:01:50.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.485+0000 7f0994343640 1 -- 172.21.15.120:0/1934113126 wait complete. 2024-09-19T09:01:50.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.486+0000 7f0994343640 1 Processor -- start 2024-09-19T09:01:50.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.486+0000 7f0994343640 1 -- start start 2024-09-19T09:01:50.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.486+0000 7f0994343640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f098c102d30 0x7f098c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:50.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.486+0000 7f0994343640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f098c1036e0 0x7f098c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:50.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.486+0000 7f0994343640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f098c1082a0 0x7f098c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:50.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.486+0000 7f0994343640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f098c1120a0 con 0x7f098c102d30 2024-09-19T09:01:50.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.486+0000 7f0994343640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f098c111f20 con 0x7f098c1082a0 2024-09-19T09:01:50.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.486+0000 7f0994343640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f098c112220 con 0x7f098c1036e0 2024-09-19T09:01:50.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.486+0000 7f09920df640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f098c102d30 0x7f098c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:50.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.486+0000 7f09920df640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f098c102d30 0x7f098c100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59680/0 (socket says 172.21.15.120:59680) 2024-09-19T09:01:50.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.486+0000 7f09920df640 1 -- 172.21.15.120:0/2623499724 learned_addr learned my addr 172.21.15.120:0/2623499724 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:50.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.486+0000 7f09918de640 1 --2- 172.21.15.120:0/2623499724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f098c1036e0 0x7f098c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:50.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.487+0000 7f09928e0640 1 --2- 172.21.15.120:0/2623499724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f098c1082a0 0x7f098c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:50.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.487+0000 7f09920df640 1 -- 172.21.15.120:0/2623499724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f098c1036e0 msgr2=0x7f098c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:50.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.487+0000 7f09920df640 1 --2- 172.21.15.120:0/2623499724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f098c1036e0 0x7f098c100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:50.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.487+0000 7f09920df640 1 -- 172.21.15.120:0/2623499724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f098c1082a0 msgr2=0x7f098c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:50.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.487+0000 7f09920df640 1 --2- 172.21.15.120:0/2623499724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f098c1082a0 0x7f098c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:50.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.487+0000 7f09920df640 1 -- 172.21.15.120:0/2623499724 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f098c1a6a20 con 0x7f098c102d30 2024-09-19T09:01:50.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.487+0000 7f09918de640 1 --2- 172.21.15.120:0/2623499724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f098c1036e0 0x7f098c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:01:50.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.487+0000 7f09920df640 1 --2- 172.21.15.120:0/2623499724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f098c102d30 0x7f098c100990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f098800d950 tx=0x7f098800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:50.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.487+0000 7f09837fe640 1 -- 172.21.15.120:0/2623499724 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0988017030 con 0x7f098c102d30 2024-09-19T09:01:50.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.487+0000 7f09837fe640 1 -- 172.21.15.120:0/2623499724 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0988004020 con 0x7f098c102d30 2024-09-19T09:01:50.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.487+0000 7f09837fe640 1 -- 172.21.15.120:0/2623499724 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09880047a0 con 0x7f098c102d30 2024-09-19T09:01:50.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.487+0000 7f0994343640 1 -- 172.21.15.120:0/2623499724 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f098c1a6d10 con 0x7f098c102d30 2024-09-19T09:01:50.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.487+0000 7f0994343640 1 -- 172.21.15.120:0/2623499724 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f098c1a71c0 con 0x7f098c102d30 2024-09-19T09:01:50.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.488+0000 7f0994343640 1 -- 172.21.15.120:0/2623499724 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f098c103110 con 0x7f098c102d30 2024-09-19T09:01:50.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.490+0000 7f09837fe640 1 -- 172.21.15.120:0/2623499724 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f09880041d0 con 0x7f098c102d30 2024-09-19T09:01:50.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.490+0000 7f09837fe640 1 --2- 172.21.15.120:0/2623499724 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f09780a32d0 0x7f09780a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:50.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.490+0000 7f09918de640 1 --2- 172.21.15.120:0/2623499724 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f09780a32d0 0x7f09780a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:50.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.491+0000 7f09837fe640 1 -- 172.21.15.120:0/2623499724 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f09880cf370 con 0x7f098c102d30 2024-09-19T09:01:50.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.491+0000 7f09918de640 1 --2- 172.21.15.120:0/2623499724 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f09780a32d0 0x7f09780a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f097c009900 tx=0x7f097c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:50.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.495+0000 7f09837fe640 1 -- 172.21.15.120:0/2623499724 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f098809c0e0 con 0x7f098c102d30 2024-09-19T09:01:50.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.672+0000 7f0994343640 1 -- 172.21.15.120:0/2623499724 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f098c10a2b0 con 0x7f098c102d30 2024-09-19T09:01:50.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.672+0000 7f09837fe640 1 -- 172.21.15.120:0/2623499724 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f098809b9e0 con 0x7f098c102d30 2024-09-19T09:01:50.674 INFO:teuthology.orchestra.run.smithi120.stdout:77309411400 2024-09-19T09:01:50.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.674+0000 7f0994343640 1 -- 172.21.15.120:0/2623499724 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f09780a32d0 msgr2=0x7f09780a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:50.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.674+0000 7f0994343640 1 --2- 172.21.15.120:0/2623499724 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f09780a32d0 0x7f09780a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f097c009900 tx=0x7f097c005a20 comp rx=0 tx=0).stop 2024-09-19T09:01:50.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.674+0000 7f0994343640 1 -- 172.21.15.120:0/2623499724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f098c102d30 msgr2=0x7f098c100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:50.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.674+0000 7f0994343640 1 --2- 172.21.15.120:0/2623499724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f098c102d30 0x7f098c100990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f098800d950 tx=0x7f098800de10 comp rx=0 tx=0).stop 2024-09-19T09:01:50.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.675+0000 7f0994343640 1 -- 172.21.15.120:0/2623499724 shutdown_connections 2024-09-19T09:01:50.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.675+0000 7f0994343640 1 --2- 172.21.15.120:0/2623499724 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f09780a32d0 0x7f09780a5790 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:50.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.675+0000 7f0994343640 1 --2- 172.21.15.120:0/2623499724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f098c1082a0 0x7f098c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:50.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.675+0000 7f0994343640 1 --2- 172.21.15.120:0/2623499724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f098c1036e0 0x7f098c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:50.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.675+0000 7f0994343640 1 --2- 172.21.15.120:0/2623499724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f098c102d30 0x7f098c100990 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:50.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.675+0000 7f0994343640 1 -- 172.21.15.120:0/2623499724 >> 172.21.15.120:0/2623499724 conn(0x7f098c0fc7d0 msgr2=0x7f098c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:50.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.675+0000 7f0994343640 1 -- 172.21.15.120:0/2623499724 shutdown_connections 2024-09-19T09:01:50.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:50.675+0000 7f0994343640 1 -- 172.21.15.120:0/2623499724 wait complete. 2024-09-19T09:01:51.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:50 smithi120 ceph-mon[31150]: pgmap v308: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T09:01:51.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:50 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2970428957' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T09:01:51.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:50 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2623499724' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T09:01:51.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:50 smithi196 ceph-mon[35150]: pgmap v308: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T09:01:51.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:50 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2970428957' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T09:01:51.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:50 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2623499724' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T09:01:51.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:50 smithi153 ceph-mon[35071]: pgmap v308: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T09:01:51.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:50 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2970428957' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T09:01:51.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:50 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2623499724' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T09:01:51.284 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986234 got 133143986238 for osd.4 2024-09-19T09:01:51.285 DEBUG:teuthology.parallel:result is None 2024-09-19T09:01:52.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-09-19T09:01:52.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-19T09:01:52.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-09-19T09:01:52.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-09-19T09:01:52.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-09-19T09:01:52.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-09-19T09:01:52.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]: dispatch 2024-09-19T09:01:52.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-09-19T09:01:52.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:51 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-19T09:01:52.123 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411397 got 77309411400 for osd.2 2024-09-19T09:01:52.124 DEBUG:teuthology.parallel:result is None 2024-09-19T09:01:52.124 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T09:01:52.124 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T09:01:52.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:51 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-09-19T09:01:52.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:51 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-19T09:01:52.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:51 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-09-19T09:01:52.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:51 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-09-19T09:01:52.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:51 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-09-19T09:01:52.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:51 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-09-19T09:01:52.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:51 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]: dispatch 2024-09-19T09:01:52.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:51 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-09-19T09:01:52.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:51 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-19T09:01:52.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:51 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-09-19T09:01:52.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:51 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-19T09:01:52.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:51 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-09-19T09:01:52.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:51 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-09-19T09:01:52.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:51 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-09-19T09:01:52.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:51 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-09-19T09:01:52.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:51 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]: dispatch 2024-09-19T09:01:52.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:51 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-09-19T09:01:52.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:51 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-19T09:01:53.016 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:53.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:52 smithi120 ceph-mon[31150]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T09:01:53.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:52 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-09-19T09:01:53.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:52 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-19T09:01:53.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:52 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-19T09:01:53.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:52 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-09-19T09:01:53.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:52 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-09-19T09:01:53.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:52 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-19T09:01:53.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:52 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]': finished 2024-09-19T09:01:53.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:52 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-09-19T09:01:53.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:52 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-19T09:01:53.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:52 smithi120 ceph-mon[31150]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T09:01:53.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:52 smithi196 ceph-mon[35150]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T09:01:53.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:52 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-09-19T09:01:53.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:52 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-19T09:01:53.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:52 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-19T09:01:53.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:52 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-09-19T09:01:53.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:52 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-09-19T09:01:53.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:52 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-19T09:01:53.188 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:52 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]': finished 2024-09-19T09:01:53.188 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:52 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-09-19T09:01:53.188 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:52 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-19T09:01:53.188 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:52 smithi196 ceph-mon[35150]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T09:01:53.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:52 smithi153 ceph-mon[35071]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T09:01:53.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:52 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-09-19T09:01:53.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:52 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-19T09:01:53.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:52 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-19T09:01:53.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:52 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-09-19T09:01:53.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:52 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-09-19T09:01:53.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:52 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-19T09:01:53.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:52 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]': finished 2024-09-19T09:01:53.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:52 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-09-19T09:01:53.190 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:52 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-19T09:01:53.190 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:52 smithi153 ceph-mon[35071]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T09:01:53.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.529+0000 7f85df598640 1 -- 172.21.15.120:0/3273674010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85d8100e20 msgr2=0x7f85d8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:53.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.529+0000 7f85df598640 1 --2- 172.21.15.120:0/3273674010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85d8100e20 0x7f85d8101200 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009900 tx=0x7f85d402a480 comp rx=0 tx=0).stop 2024-09-19T09:01:53.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.529+0000 7f85df598640 1 -- 172.21.15.120:0/3273674010 shutdown_connections 2024-09-19T09:01:53.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.529+0000 7f85df598640 1 --2- 172.21.15.120:0/3273674010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85d8102ef0 0x7f85d810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:53.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.529+0000 7f85df598640 1 --2- 172.21.15.120:0/3273674010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85d8102570 0x7f85d81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:53.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.529+0000 7f85df598640 1 --2- 172.21.15.120:0/3273674010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85d8100e20 0x7f85d8101200 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:53.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.529+0000 7f85df598640 1 -- 172.21.15.120:0/3273674010 >> 172.21.15.120:0/3273674010 conn(0x7f85d80fc7d0 msgr2=0x7f85d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:53.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.529+0000 7f85df598640 1 -- 172.21.15.120:0/3273674010 shutdown_connections 2024-09-19T09:01:53.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.529+0000 7f85df598640 1 -- 172.21.15.120:0/3273674010 wait complete. 2024-09-19T09:01:53.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.530+0000 7f85df598640 1 Processor -- start 2024-09-19T09:01:53.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.530+0000 7f85df598640 1 -- start start 2024-09-19T09:01:53.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.530+0000 7f85df598640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85d8100e20 0x7f85d8197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:53.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.530+0000 7f85df598640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85d8102570 0x7f85d8197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:53.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.530+0000 7f85df598640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85d8102ef0 0x7f85d819be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:53.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.530+0000 7f85df598640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8111020 con 0x7f85d8102570 2024-09-19T09:01:53.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.530+0000 7f85df598640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8110ea0 con 0x7f85d8100e20 2024-09-19T09:01:53.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.530+0000 7f85df598640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f85d81111a0 con 0x7f85d8102ef0 2024-09-19T09:01:53.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.530+0000 7f85dcb33640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85d8102570 0x7f85d8197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:53.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.530+0000 7f85dcb33640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85d8102570 0x7f85d8197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52820/0 (socket says 172.21.15.120:52820) 2024-09-19T09:01:53.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.530+0000 7f85dcb33640 1 -- 172.21.15.120:0/1666228017 learned_addr learned my addr 172.21.15.120:0/1666228017 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:53.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.530+0000 7f85ddb35640 1 --2- 172.21.15.120:0/1666228017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85d8102ef0 0x7f85d819be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:53.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.530+0000 7f85dcb33640 1 -- 172.21.15.120:0/1666228017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85d8102ef0 msgr2=0x7f85d819be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:53.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.530+0000 7f85dcb33640 1 --2- 172.21.15.120:0/1666228017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85d8102ef0 0x7f85d819be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:53.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.530+0000 7f85dd334640 1 --2- 172.21.15.120:0/1666228017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85d8100e20 0x7f85d8197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:53.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.530+0000 7f85dcb33640 1 -- 172.21.15.120:0/1666228017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85d8100e20 msgr2=0x7f85d8197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:53.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.530+0000 7f85dcb33640 1 --2- 172.21.15.120:0/1666228017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85d8100e20 0x7f85d8197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:53.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.530+0000 7f85dcb33640 1 -- 172.21.15.120:0/1666228017 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d819c530 con 0x7f85d8102570 2024-09-19T09:01:53.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.531+0000 7f85dcb33640 1 --2- 172.21.15.120:0/1666228017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85d8102570 0x7f85d8197ac0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f85c800b4a0 tx=0x7f85c800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:53.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.531+0000 7f85ce7fc640 1 -- 172.21.15.120:0/1666228017 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8014070 con 0x7f85d8102570 2024-09-19T09:01:53.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.531+0000 7f85ce7fc640 1 -- 172.21.15.120:0/1666228017 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c8010af0 con 0x7f85d8102570 2024-09-19T09:01:53.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.531+0000 7f85df598640 1 -- 172.21.15.120:0/1666228017 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d819c820 con 0x7f85d8102570 2024-09-19T09:01:53.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.531+0000 7f85ce7fc640 1 -- 172.21.15.120:0/1666228017 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8017d10 con 0x7f85d8102570 2024-09-19T09:01:53.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.531+0000 7f85df598640 1 -- 172.21.15.120:0/1666228017 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d81a4100 con 0x7f85d8102570 2024-09-19T09:01:53.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.532+0000 7f85df598640 1 -- 172.21.15.120:0/1666228017 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85d8101280 con 0x7f85d8102570 2024-09-19T09:01:53.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.534+0000 7f85ce7fc640 1 -- 172.21.15.120:0/1666228017 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f85c8017070 con 0x7f85d8102570 2024-09-19T09:01:53.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.534+0000 7f85ce7fc640 1 --2- 172.21.15.120:0/1666228017 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f85c40a32d0 0x7f85c40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:53.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.534+0000 7f85dd334640 1 --2- 172.21.15.120:0/1666228017 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f85c40a32d0 0x7f85c40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:53.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.534+0000 7f85ce7fc640 1 -- 172.21.15.120:0/1666228017 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f85c80cf1a0 con 0x7f85d8102570 2024-09-19T09:01:53.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.535+0000 7f85dd334640 1 --2- 172.21.15.120:0/1666228017 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f85c40a32d0 0x7f85c40a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f85d40096c0 tx=0x7f85d4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:53.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.538+0000 7f85ce7fc640 1 -- 172.21.15.120:0/1666228017 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f85c809be70 con 0x7f85d8102570 2024-09-19T09:01:53.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.730+0000 7f85df598640 1 -- 172.21.15.120:0/1666228017 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f85d8062630 con 0x7f85c40a32d0 2024-09-19T09:01:53.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.737+0000 7f85ce7fc640 1 -- 172.21.15.120:0/1666228017 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258986 (secure 0 0 0) 0x7f85d8062630 con 0x7f85c40a32d0 2024-09-19T09:01:53.741 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:01:53.742 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-19T09:01:53.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.740+0000 7f85df598640 1 -- 172.21.15.120:0/1666228017 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f85c40a32d0 msgr2=0x7f85c40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:53.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.740+0000 7f85df598640 1 --2- 172.21.15.120:0/1666228017 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f85c40a32d0 0x7f85c40a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f85d40096c0 tx=0x7f85d4000bc0 comp rx=0 tx=0).stop 2024-09-19T09:01:53.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.740+0000 7f85df598640 1 -- 172.21.15.120:0/1666228017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85d8102570 msgr2=0x7f85d8197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:53.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.740+0000 7f85df598640 1 --2- 172.21.15.120:0/1666228017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85d8102570 0x7f85d8197ac0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f85c800b4a0 tx=0x7f85c800b960 comp rx=0 tx=0).stop 2024-09-19T09:01:53.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.740+0000 7f85df598640 1 -- 172.21.15.120:0/1666228017 shutdown_connections 2024-09-19T09:01:53.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.740+0000 7f85df598640 1 --2- 172.21.15.120:0/1666228017 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f85c40a32d0 0x7f85c40a5790 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:53.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.740+0000 7f85df598640 1 --2- 172.21.15.120:0/1666228017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85d8102ef0 0x7f85d819be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:53.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.740+0000 7f85df598640 1 --2- 172.21.15.120:0/1666228017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85d8102570 0x7f85d8197ac0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:53.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.740+0000 7f85df598640 1 --2- 172.21.15.120:0/1666228017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85d8100e20 0x7f85d8197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:53.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.741+0000 7f85df598640 1 -- 172.21.15.120:0/1666228017 >> 172.21.15.120:0/1666228017 conn(0x7f85d80fc7d0 msgr2=0x7f85d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:53.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.741+0000 7f85df598640 1 -- 172.21.15.120:0/1666228017 shutdown_connections 2024-09-19T09:01:53.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:53.741+0000 7f85df598640 1 -- 172.21.15.120:0/1666228017 wait complete. 2024-09-19T09:01:54.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:53 smithi120 ceph-mon[31150]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T09:01:54.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:53 smithi196 ceph-mon[35150]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T09:01:54.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:53 smithi153 ceph-mon[35071]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T09:01:54.490 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2024-09-19T09:01:53.623477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659727,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":695,"num_read_kb":469,"num_write":426,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":290,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364204,"kb_used_data":10764,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577620,"statfs":{"total":1151940427776,"available":1151567482880,"internally_reserved":0,"allocated":11022336,"data_stored":6973731,"data_compressed":138549,"data_compressed_allocated":184320,"data_compressed_original":1867776,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":95,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001729"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.969591+0000","last_change":"2024-09-19T09:00:46.966628+0000","last_active":"2024-09-19T09:00:47.969591+0000","last_peered":"2024-09-19T09:00:47.969591+0000","last_clean":"2024-09-19T09:00:47.969591+0000","last_became_active":"2024-09-19T09:00:46.966290+0000","last_became_peered":"2024-09-19T09:00:46.966290+0000","last_unstale":"2024-09-19T09:00:47.969591+0000","last_undegraded":"2024-09-19T09:00:47.969591+0000","last_fullsized":"2024-09-19T09:00:47.969591+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:42.717139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.754728+0000","last_change":"2024-09-19T09:00:46.963706+0000","last_active":"2024-09-19T09:01:52.754728+0000","last_peered":"2024-09-19T09:01:52.754728+0000","last_clean":"2024-09-19T09:01:52.754728+0000","last_became_active":"2024-09-19T09:00:46.963303+0000","last_became_peered":"2024-09-19T09:00:46.963303+0000","last_unstale":"2024-09-19T09:01:52.754728+0000","last_undegraded":"2024-09-19T09:01:52.754728+0000","last_fullsized":"2024-09-19T09:01:52.754728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:07.451500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.754557+0000","last_change":"2024-09-19T09:00:46.967149+0000","last_active":"2024-09-19T09:01:52.754557+0000","last_peered":"2024-09-19T09:01:52.754557+0000","last_clean":"2024-09-19T09:01:52.754557+0000","last_became_active":"2024-09-19T09:00:46.966861+0000","last_became_peered":"2024-09-19T09:00:46.966861+0000","last_unstale":"2024-09-19T09:01:52.754557+0000","last_undegraded":"2024-09-19T09:01:52.754557+0000","last_fullsized":"2024-09-19T09:01:52.754557+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:28.396022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.750139+0000","last_change":"2024-09-19T09:00:46.969242+0000","last_active":"2024-09-19T09:01:51.750139+0000","last_peered":"2024-09-19T09:01:51.750139+0000","last_clean":"2024-09-19T09:01:51.750139+0000","last_became_active":"2024-09-19T09:00:46.968546+0000","last_became_peered":"2024-09-19T09:00:46.968546+0000","last_unstale":"2024-09-19T09:01:51.750139+0000","last_undegraded":"2024-09-19T09:01:51.750139+0000","last_fullsized":"2024-09-19T09:01:51.750139+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:43:51.060130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.953778+0000","last_change":"2024-09-19T09:00:46.955501+0000","last_active":"2024-09-19T09:00:47.953778+0000","last_peered":"2024-09-19T09:00:47.953778+0000","last_clean":"2024-09-19T09:00:47.953778+0000","last_became_active":"2024-09-19T09:00:46.955161+0000","last_became_peered":"2024-09-19T09:00:46.955161+0000","last_unstale":"2024-09-19T09:00:47.953778+0000","last_undegraded":"2024-09-19T09:00:47.953778+0000","last_fullsized":"2024-09-19T09:00:47.953778+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:39:19.005931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.965095+0000","last_change":"2024-09-19T09:00:46.962577+0000","last_active":"2024-09-19T09:00:47.965095+0000","last_peered":"2024-09-19T09:00:47.965095+0000","last_clean":"2024-09-19T09:00:47.965095+0000","last_became_active":"2024-09-19T09:00:46.962253+0000","last_became_peered":"2024-09-19T09:00:46.962253+0000","last_unstale":"2024-09-19T09:00:47.965095+0000","last_undegraded":"2024-09-19T09:00:47.965095+0000","last_fullsized":"2024-09-19T09:00:47.965095+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:27.925932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964437+0000","last_change":"2024-09-19T09:00:46.960004+0000","last_active":"2024-09-19T09:00:47.964437+0000","last_peered":"2024-09-19T09:00:47.964437+0000","last_clean":"2024-09-19T09:00:47.964437+0000","last_became_active":"2024-09-19T09:00:46.959664+0000","last_became_peered":"2024-09-19T09:00:46.959664+0000","last_unstale":"2024-09-19T09:00:47.964437+0000","last_undegraded":"2024-09-19T09:00:47.964437+0000","last_fullsized":"2024-09-19T09:00:47.964437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:41.616309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964731+0000","last_change":"2024-09-19T09:00:46.952394+0000","last_active":"2024-09-19T09:00:47.964731+0000","last_peered":"2024-09-19T09:00:47.964731+0000","last_clean":"2024-09-19T09:00:47.964731+0000","last_became_active":"2024-09-19T09:00:46.952084+0000","last_became_peered":"2024-09-19T09:00:46.952084+0000","last_unstale":"2024-09-19T09:00:47.964731+0000","last_undegraded":"2024-09-19T09:00:47.964731+0000","last_fullsized":"2024-09-19T09:00:47.964731+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:21.698423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.750363+0000","last_change":"2024-09-19T09:00:46.961262+0000","last_active":"2024-09-19T09:01:51.750363+0000","last_peered":"2024-09-19T09:01:51.750363+0000","last_clean":"2024-09-19T09:01:51.750363+0000","last_became_active":"2024-09-19T09:00:46.960951+0000","last_became_peered":"2024-09-19T09:00:46.960951+0000","last_unstale":"2024-09-19T09:01:51.750363+0000","last_undegraded":"2024-09-19T09:01:51.750363+0000","last_fullsized":"2024-09-19T09:01:51.750363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:02.445108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.966156+0000","last_change":"2024-09-19T09:00:46.964806+0000","last_active":"2024-09-19T09:00:47.966156+0000","last_peered":"2024-09-19T09:00:47.966156+0000","last_clean":"2024-09-19T09:00:47.966156+0000","last_became_active":"2024-09-19T09:00:46.964194+0000","last_became_peered":"2024-09-19T09:00:46.964194+0000","last_unstale":"2024-09-19T09:00:47.966156+0000","last_undegraded":"2024-09-19T09:00:47.966156+0000","last_fullsized":"2024-09-19T09:00:47.966156+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:31.621075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.966443+0000","last_change":"2024-09-19T09:00:46.963200+0000","last_active":"2024-09-19T09:00:47.966443+0000","last_peered":"2024-09-19T09:00:47.966443+0000","last_clean":"2024-09-19T09:00:47.966443+0000","last_became_active":"2024-09-19T09:00:46.960496+0000","last_became_peered":"2024-09-19T09:00:46.960496+0000","last_unstale":"2024-09-19T09:00:47.966443+0000","last_undegraded":"2024-09-19T09:00:47.966443+0000","last_fullsized":"2024-09-19T09:00:47.966443+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:59:47.234492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.750254+0000","last_change":"2024-09-19T09:00:46.966309+0000","last_active":"2024-09-19T09:01:51.750254+0000","last_peered":"2024-09-19T09:01:51.750254+0000","last_clean":"2024-09-19T09:01:51.750254+0000","last_became_active":"2024-09-19T09:00:46.965970+0000","last_became_peered":"2024-09-19T09:00:46.965970+0000","last_unstale":"2024-09-19T09:01:51.750254+0000","last_undegraded":"2024-09-19T09:01:51.750254+0000","last_fullsized":"2024-09-19T09:01:51.750254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:38.353206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.748099+0000","last_change":"2024-09-19T09:00:46.954079+0000","last_active":"2024-09-19T09:01:52.748099+0000","last_peered":"2024-09-19T09:01:52.748099+0000","last_clean":"2024-09-19T09:01:52.748099+0000","last_became_active":"2024-09-19T09:00:46.953398+0000","last_became_peered":"2024-09-19T09:00:46.953398+0000","last_unstale":"2024-09-19T09:01:52.748099+0000","last_undegraded":"2024-09-19T09:01:52.748099+0000","last_fullsized":"2024-09-19T09:01:52.748099+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:54:27.554800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.754336+0000","last_change":"2024-09-19T09:00:46.965347+0000","last_active":"2024-09-19T09:01:52.754336+0000","last_peered":"2024-09-19T09:01:52.754336+0000","last_clean":"2024-09-19T09:01:52.754336+0000","last_became_active":"2024-09-19T09:00:46.965045+0000","last_became_peered":"2024-09-19T09:00:46.965045+0000","last_unstale":"2024-09-19T09:01:52.754336+0000","last_undegraded":"2024-09-19T09:01:52.754336+0000","last_fullsized":"2024-09-19T09:01:52.754336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:27.316100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.754183+0000","last_change":"2024-09-19T09:00:46.962898+0000","last_active":"2024-09-19T09:01:52.754183+0000","last_peered":"2024-09-19T09:01:52.754183+0000","last_clean":"2024-09-19T09:01:52.754183+0000","last_became_active":"2024-09-19T09:00:46.962412+0000","last_became_peered":"2024-09-19T09:00:46.962412+0000","last_unstale":"2024-09-19T09:01:52.754183+0000","last_undegraded":"2024-09-19T09:01:52.754183+0000","last_fullsized":"2024-09-19T09:01:52.754183+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:40.483678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.749841+0000","last_change":"2024-09-19T09:00:46.958147+0000","last_active":"2024-09-19T09:01:51.749841+0000","last_peered":"2024-09-19T09:01:51.749841+0000","last_clean":"2024-09-19T09:01:51.749841+0000","last_became_active":"2024-09-19T09:00:46.957805+0000","last_became_peered":"2024-09-19T09:00:46.957805+0000","last_unstale":"2024-09-19T09:01:51.749841+0000","last_undegraded":"2024-09-19T09:01:51.749841+0000","last_fullsized":"2024-09-19T09:01:51.749841+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:24:17.000064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.969691+0000","last_change":"2024-09-19T09:00:46.967399+0000","last_active":"2024-09-19T09:00:47.969691+0000","last_peered":"2024-09-19T09:00:47.969691+0000","last_clean":"2024-09-19T09:00:47.969691+0000","last_became_active":"2024-09-19T09:00:46.966145+0000","last_became_peered":"2024-09-19T09:00:46.966145+0000","last_unstale":"2024-09-19T09:00:47.969691+0000","last_undegraded":"2024-09-19T09:00:47.969691+0000","last_fullsized":"2024-09-19T09:00:47.969691+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:29.192469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.962791+0000","last_change":"2024-09-19T09:00:46.960956+0000","last_active":"2024-09-19T09:00:47.962791+0000","last_peered":"2024-09-19T09:00:47.962791+0000","last_clean":"2024-09-19T09:00:47.962791+0000","last_became_active":"2024-09-19T09:00:46.960664+0000","last_became_peered":"2024-09-19T09:00:46.960664+0000","last_unstale":"2024-09-19T09:00:47.962791+0000","last_undegraded":"2024-09-19T09:00:47.962791+0000","last_fullsized":"2024-09-19T09:00:47.962791+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:18.724348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.753580+0000","last_change":"2024-09-19T09:00:46.951362+0000","last_active":"2024-09-19T09:01:51.753580+0000","last_peered":"2024-09-19T09:01:51.753580+0000","last_clean":"2024-09-19T09:01:51.753580+0000","last_became_active":"2024-09-19T09:00:46.951004+0000","last_became_peered":"2024-09-19T09:00:46.951004+0000","last_unstale":"2024-09-19T09:01:51.753580+0000","last_undegraded":"2024-09-19T09:01:51.753580+0000","last_fullsized":"2024-09-19T09:01:51.753580+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:43.574425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.969012+0000","last_change":"2024-09-19T09:00:46.966752+0000","last_active":"2024-09-19T09:00:47.969012+0000","last_peered":"2024-09-19T09:00:47.969012+0000","last_clean":"2024-09-19T09:00:47.969012+0000","last_became_active":"2024-09-19T09:00:46.966370+0000","last_became_peered":"2024-09-19T09:00:46.966370+0000","last_unstale":"2024-09-19T09:00:47.969012+0000","last_undegraded":"2024-09-19T09:00:47.969012+0000","last_fullsized":"2024-09-19T09:00:47.969012+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:04.035771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.748112+0000","last_change":"2024-09-19T09:00:46.956369+0000","last_active":"2024-09-19T09:01:52.748112+0000","last_peered":"2024-09-19T09:01:52.748112+0000","last_clean":"2024-09-19T09:01:52.748112+0000","last_became_active":"2024-09-19T09:00:46.956075+0000","last_became_peered":"2024-09-19T09:00:46.956075+0000","last_unstale":"2024-09-19T09:01:52.748112+0000","last_undegraded":"2024-09-19T09:01:52.748112+0000","last_fullsized":"2024-09-19T09:01:52.748112+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:23.985017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.750892+0000","last_change":"2024-09-19T09:00:46.956661+0000","last_active":"2024-09-19T09:01:51.750892+0000","last_peered":"2024-09-19T09:01:51.750892+0000","last_clean":"2024-09-19T09:01:51.750892+0000","last_became_active":"2024-09-19T09:00:46.956291+0000","last_became_peered":"2024-09-19T09:00:46.956291+0000","last_unstale":"2024-09-19T09:01:51.750892+0000","last_undegraded":"2024-09-19T09:01:51.750892+0000","last_fullsized":"2024-09-19T09:01:51.750892+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:55.713039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964577+0000","last_change":"2024-09-19T09:00:46.961866+0000","last_active":"2024-09-19T09:00:47.964577+0000","last_peered":"2024-09-19T09:00:47.964577+0000","last_clean":"2024-09-19T09:00:47.964577+0000","last_became_active":"2024-09-19T09:00:46.961469+0000","last_became_peered":"2024-09-19T09:00:46.961469+0000","last_unstale":"2024-09-19T09:00:47.964577+0000","last_undegraded":"2024-09-19T09:00:47.964577+0000","last_fullsized":"2024-09-19T09:00:47.964577+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:47.665419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751305+0000","last_change":"2024-09-19T09:00:46.948765+0000","last_active":"2024-09-19T09:01:52.751305+0000","last_peered":"2024-09-19T09:01:52.751305+0000","last_clean":"2024-09-19T09:01:52.751305+0000","last_became_active":"2024-09-19T09:00:46.948431+0000","last_became_peered":"2024-09-19T09:00:46.948431+0000","last_unstale":"2024-09-19T09:01:52.751305+0000","last_undegraded":"2024-09-19T09:01:52.751305+0000","last_fullsized":"2024-09-19T09:01:52.751305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:31.766464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964637+0000","last_change":"2024-09-19T09:00:46.962433+0000","last_active":"2024-09-19T09:00:47.964637+0000","last_peered":"2024-09-19T09:00:47.964637+0000","last_clean":"2024-09-19T09:00:47.964637+0000","last_became_active":"2024-09-19T09:00:46.962153+0000","last_became_peered":"2024-09-19T09:00:46.962153+0000","last_unstale":"2024-09-19T09:00:47.964637+0000","last_undegraded":"2024-09-19T09:00:47.964637+0000","last_fullsized":"2024-09-19T09:00:47.964637+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:37.973611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.753893+0000","last_change":"2024-09-19T09:00:46.953715+0000","last_active":"2024-09-19T09:01:51.753893+0000","last_peered":"2024-09-19T09:01:51.753893+0000","last_clean":"2024-09-19T09:01:51.753893+0000","last_became_active":"2024-09-19T09:00:46.953415+0000","last_became_peered":"2024-09-19T09:00:46.953415+0000","last_unstale":"2024-09-19T09:01:51.753893+0000","last_undegraded":"2024-09-19T09:01:51.753893+0000","last_fullsized":"2024-09-19T09:01:51.753893+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:29.768382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.969440+0000","last_change":"2024-09-19T09:00:46.964746+0000","last_active":"2024-09-19T09:00:47.969440+0000","last_peered":"2024-09-19T09:00:47.969440+0000","last_clean":"2024-09-19T09:00:47.969440+0000","last_became_active":"2024-09-19T09:00:46.964404+0000","last_became_peered":"2024-09-19T09:00:46.964404+0000","last_unstale":"2024-09-19T09:00:47.969440+0000","last_undegraded":"2024-09-19T09:00:47.969440+0000","last_fullsized":"2024-09-19T09:00:47.969440+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:19.841495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.748134+0000","last_change":"2024-09-19T09:00:46.962875+0000","last_active":"2024-09-19T09:01:52.748134+0000","last_peered":"2024-09-19T09:01:52.748134+0000","last_clean":"2024-09-19T09:01:52.748134+0000","last_became_active":"2024-09-19T09:00:46.962561+0000","last_became_peered":"2024-09-19T09:00:46.962561+0000","last_unstale":"2024-09-19T09:01:52.748134+0000","last_undegraded":"2024-09-19T09:01:52.748134+0000","last_fullsized":"2024-09-19T09:01:52.748134+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:39.675545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.754430+0000","last_change":"2024-09-19T09:00:46.963126+0000","last_active":"2024-09-19T09:01:52.754430+0000","last_peered":"2024-09-19T09:01:52.754430+0000","last_clean":"2024-09-19T09:01:52.754430+0000","last_became_active":"2024-09-19T09:00:46.962620+0000","last_became_peered":"2024-09-19T09:00:46.962620+0000","last_unstale":"2024-09-19T09:01:52.754430+0000","last_undegraded":"2024-09-19T09:01:52.754430+0000","last_fullsized":"2024-09-19T09:01:52.754430+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:59.723280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.753128+0000","last_change":"2024-09-19T09:00:46.953026+0000","last_active":"2024-09-19T09:01:51.753128+0000","last_peered":"2024-09-19T09:01:51.753128+0000","last_clean":"2024-09-19T09:01:51.753128+0000","last_became_active":"2024-09-19T09:00:46.952596+0000","last_became_peered":"2024-09-19T09:00:46.952596+0000","last_unstale":"2024-09-19T09:01:51.753128+0000","last_undegraded":"2024-09-19T09:01:51.753128+0000","last_fullsized":"2024-09-19T09:01:51.753128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:58.524116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.753052+0000","last_change":"2024-09-19T09:00:46.968964+0000","last_active":"2024-09-19T09:01:51.753052+0000","last_peered":"2024-09-19T09:01:51.753052+0000","last_clean":"2024-09-19T09:01:51.753052+0000","last_became_active":"2024-09-19T09:00:46.968505+0000","last_became_peered":"2024-09-19T09:00:46.968505+0000","last_unstale":"2024-09-19T09:01:51.753052+0000","last_undegraded":"2024-09-19T09:01:51.753052+0000","last_fullsized":"2024-09-19T09:01:51.753052+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:17:33.335541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.969060+0000","last_change":"2024-09-19T09:00:46.961520+0000","last_active":"2024-09-19T09:00:47.969060+0000","last_peered":"2024-09-19T09:00:47.969060+0000","last_clean":"2024-09-19T09:00:47.969060+0000","last_became_active":"2024-09-19T09:00:46.961140+0000","last_became_peered":"2024-09-19T09:00:46.961140+0000","last_unstale":"2024-09-19T09:00:47.969060+0000","last_undegraded":"2024-09-19T09:00:47.969060+0000","last_fullsized":"2024-09-19T09:00:47.969060+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:03.120173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.953710+0000","last_change":"2024-09-19T09:00:46.955995+0000","last_active":"2024-09-19T09:00:47.953710+0000","last_peered":"2024-09-19T09:00:47.953710+0000","last_clean":"2024-09-19T09:00:47.953710+0000","last_became_active":"2024-09-19T09:00:46.954678+0000","last_became_peered":"2024-09-19T09:00:46.954678+0000","last_unstale":"2024-09-19T09:00:47.953710+0000","last_undegraded":"2024-09-19T09:00:47.953710+0000","last_fullsized":"2024-09-19T09:00:47.953710+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:10.314575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":83,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752664+0000","last_change":"2024-09-19T09:00:44.946000+0000","last_active":"2024-09-19T09:01:52.752664+0000","last_peered":"2024-09-19T09:01:52.752664+0000","last_clean":"2024-09-19T09:01:52.752664+0000","last_became_active":"2024-09-19T09:00:44.945583+0000","last_became_peered":"2024-09-19T09:00:44.945583+0000","last_unstale":"2024-09-19T09:01:52.752664+0000","last_undegraded":"2024-09-19T09:01:52.752664+0000","last_fullsized":"2024-09-19T09:01:52.752664+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:53:19.109697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.904443+0000","last_change":"2024-09-19T09:01:52.904443+0000","last_active":"2024-09-19T09:01:52.904443+0000","last_peered":"2024-09-19T09:01:52.904443+0000","last_clean":"2024-09-19T09:01:52.904443+0000","last_became_active":"2024-09-19T09:01:52.904077+0000","last_became_peered":"2024-09-19T09:01:52.904077+0000","last_unstale":"2024-09-19T09:01:52.904443+0000","last_undegraded":"2024-09-19T09:01:52.904443+0000","last_fullsized":"2024-09-19T09:01:52.904443+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:50.467090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:57.126925+0000","last_change":"2024-09-19T09:00:44.942435+0000","last_active":"2024-09-19T09:00:57.126925+0000","last_peered":"2024-09-19T09:00:57.126925+0000","last_clean":"2024-09-19T09:00:57.126925+0000","last_became_active":"2024-09-19T09:00:44.942067+0000","last_became_peered":"2024-09-19T09:00:44.942067+0000","last_unstale":"2024-09-19T09:00:57.126925+0000","last_undegraded":"2024-09-19T09:00:57.126925+0000","last_fullsized":"2024-09-19T09:00:57.126925+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:16.476506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.747900+0000","last_change":"2024-09-19T09:00:44.937944+0000","last_active":"2024-09-19T09:01:52.747900+0000","last_peered":"2024-09-19T09:01:52.747900+0000","last_clean":"2024-09-19T09:01:52.747900+0000","last_became_active":"2024-09-19T09:00:44.937623+0000","last_became_peered":"2024-09-19T09:00:44.937623+0000","last_unstale":"2024-09-19T09:01:52.747900+0000","last_undegraded":"2024-09-19T09:01:52.747900+0000","last_fullsized":"2024-09-19T09:01:52.747900+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:49:08.730015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964590+0000","last_change":"2024-09-19T09:00:46.959932+0000","last_active":"2024-09-19T09:00:47.964590+0000","last_peered":"2024-09-19T09:00:47.964590+0000","last_clean":"2024-09-19T09:00:47.964590+0000","last_became_active":"2024-09-19T09:00:46.959637+0000","last_became_peered":"2024-09-19T09:00:46.959637+0000","last_unstale":"2024-09-19T09:00:47.964590+0000","last_undegraded":"2024-09-19T09:00:47.964590+0000","last_fullsized":"2024-09-19T09:00:47.964590+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:55.337669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751513+0000","last_change":"2024-09-19T09:00:44.944882+0000","last_active":"2024-09-19T09:01:52.751513+0000","last_peered":"2024-09-19T09:01:52.751513+0000","last_clean":"2024-09-19T09:01:52.751513+0000","last_became_active":"2024-09-19T09:00:44.944116+0000","last_became_peered":"2024-09-19T09:00:44.944116+0000","last_unstale":"2024-09-19T09:01:52.751513+0000","last_undegraded":"2024-09-19T09:01:52.751513+0000","last_fullsized":"2024-09-19T09:01:52.751513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:54:58.555913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.962700+0000","last_change":"2024-09-19T09:00:46.956859+0000","last_active":"2024-09-19T09:00:47.962700+0000","last_peered":"2024-09-19T09:00:47.962700+0000","last_clean":"2024-09-19T09:00:47.962700+0000","last_became_active":"2024-09-19T09:00:46.956561+0000","last_became_peered":"2024-09-19T09:00:46.956561+0000","last_unstale":"2024-09-19T09:00:47.962700+0000","last_undegraded":"2024-09-19T09:00:47.962700+0000","last_fullsized":"2024-09-19T09:00:47.962700+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:09:04.125591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":126,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:01:50.214439+0000","last_change":"2024-09-19T09:00:44.945959+0000","last_active":"2024-09-19T09:01:50.214439+0000","last_peered":"2024-09-19T09:01:50.214439+0000","last_clean":"2024-09-19T09:01:50.214439+0000","last_became_active":"2024-09-19T09:00:44.945640+0000","last_became_peered":"2024-09-19T09:00:44.945640+0000","last_unstale":"2024-09-19T09:01:50.214439+0000","last_undegraded":"2024-09-19T09:01:50.214439+0000","last_fullsized":"2024-09-19T09:01:50.214439+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:02.523834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.750570+0000","last_change":"2024-09-19T09:00:46.953677+0000","last_active":"2024-09-19T09:01:51.750570+0000","last_peered":"2024-09-19T09:01:51.750570+0000","last_clean":"2024-09-19T09:01:51.750570+0000","last_became_active":"2024-09-19T09:00:46.953357+0000","last_became_peered":"2024-09-19T09:00:46.953357+0000","last_unstale":"2024-09-19T09:01:51.750570+0000","last_undegraded":"2024-09-19T09:01:51.750570+0000","last_fullsized":"2024-09-19T09:01:51.750570+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:48.963003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.139991+0000","last_change":"2024-09-19T09:00:44.945777+0000","last_active":"2024-09-19T09:00:58.139991+0000","last_peered":"2024-09-19T09:00:58.139991+0000","last_clean":"2024-09-19T09:00:58.139991+0000","last_became_active":"2024-09-19T09:00:44.945450+0000","last_became_peered":"2024-09-19T09:00:44.945450+0000","last_unstale":"2024-09-19T09:00:58.139991+0000","last_undegraded":"2024-09-19T09:00:58.139991+0000","last_fullsized":"2024-09-19T09:00:58.139991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:44.656451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.754949+0000","last_change":"2024-09-19T09:00:46.962285+0000","last_active":"2024-09-19T09:01:52.754949+0000","last_peered":"2024-09-19T09:01:52.754949+0000","last_clean":"2024-09-19T09:01:52.754949+0000","last_became_active":"2024-09-19T09:00:46.961982+0000","last_became_peered":"2024-09-19T09:00:46.961982+0000","last_unstale":"2024-09-19T09:01:52.754949+0000","last_undegraded":"2024-09-19T09:01:52.754949+0000","last_fullsized":"2024-09-19T09:01:52.754949+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:41.658501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752480+0000","last_change":"2024-09-19T09:00:45.241425+0000","last_active":"2024-09-19T09:01:52.752480+0000","last_peered":"2024-09-19T09:01:52.752480+0000","last_clean":"2024-09-19T09:01:52.752480+0000","last_became_active":"2024-09-19T09:00:45.241032+0000","last_became_peered":"2024-09-19T09:00:45.241032+0000","last_unstale":"2024-09-19T09:01:52.752480+0000","last_undegraded":"2024-09-19T09:01:52.752480+0000","last_fullsized":"2024-09-19T09:01:52.752480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:00.707855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.750496+0000","last_change":"2024-09-19T09:00:46.962436+0000","last_active":"2024-09-19T09:01:51.750496+0000","last_peered":"2024-09-19T09:01:51.750496+0000","last_clean":"2024-09-19T09:01:51.750496+0000","last_became_active":"2024-09-19T09:00:46.962114+0000","last_became_peered":"2024-09-19T09:00:46.962114+0000","last_unstale":"2024-09-19T09:01:51.750496+0000","last_undegraded":"2024-09-19T09:01:51.750496+0000","last_fullsized":"2024-09-19T09:01:51.750496+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:41.761705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'32","reported_seq":163,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:01:50.214436+0000","last_change":"2024-09-19T09:00:44.945187+0000","last_active":"2024-09-19T09:01:50.214436+0000","last_peered":"2024-09-19T09:01:50.214436+0000","last_clean":"2024-09-19T09:01:50.214436+0000","last_became_active":"2024-09-19T09:00:44.944763+0000","last_became_peered":"2024-09-19T09:00:44.944763+0000","last_unstale":"2024-09-19T09:01:50.214436+0000","last_undegraded":"2024-09-19T09:01:50.214436+0000","last_fullsized":"2024-09-19T09:01:50.214436+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:06.052994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62310,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":113,"num_read_kb":117,"num_write":34,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.954165+0000","last_change":"2024-09-19T09:00:46.960991+0000","last_active":"2024-09-19T09:00:47.954165+0000","last_peered":"2024-09-19T09:00:47.954165+0000","last_clean":"2024-09-19T09:00:47.954165+0000","last_became_active":"2024-09-19T09:00:46.958962+0000","last_became_peered":"2024-09-19T09:00:46.958962+0000","last_unstale":"2024-09-19T09:00:47.954165+0000","last_undegraded":"2024-09-19T09:00:47.954165+0000","last_fullsized":"2024-09-19T09:00:47.954165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:21.795694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.753490+0000","last_change":"2024-09-19T09:00:44.946176+0000","last_active":"2024-09-19T09:01:51.753490+0000","last_peered":"2024-09-19T09:01:51.753490+0000","last_clean":"2024-09-19T09:01:51.753490+0000","last_became_active":"2024-09-19T09:00:44.945869+0000","last_became_peered":"2024-09-19T09:00:44.945869+0000","last_unstale":"2024-09-19T09:01:51.753490+0000","last_undegraded":"2024-09-19T09:01:51.753490+0000","last_fullsized":"2024-09-19T09:01:51.753490+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:16.022184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.747950+0000","last_change":"2024-09-19T09:00:46.956345+0000","last_active":"2024-09-19T09:01:52.747950+0000","last_peered":"2024-09-19T09:01:52.747950+0000","last_clean":"2024-09-19T09:01:52.747950+0000","last_became_active":"2024-09-19T09:00:46.956057+0000","last_became_peered":"2024-09-19T09:00:46.956057+0000","last_unstale":"2024-09-19T09:01:52.747950+0000","last_undegraded":"2024-09-19T09:01:52.747950+0000","last_fullsized":"2024-09-19T09:01:52.747950+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:27.510661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752205+0000","last_change":"2024-09-19T09:00:44.945667+0000","last_active":"2024-09-19T09:01:52.752205+0000","last_peered":"2024-09-19T09:01:52.752205+0000","last_clean":"2024-09-19T09:01:52.752205+0000","last_became_active":"2024-09-19T09:00:44.945308+0000","last_became_peered":"2024-09-19T09:00:44.945308+0000","last_unstale":"2024-09-19T09:01:52.752205+0000","last_undegraded":"2024-09-19T09:01:52.752205+0000","last_fullsized":"2024-09-19T09:01:52.752205+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:00:11.304913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.965242+0000","last_change":"2024-09-19T09:00:46.965088+0000","last_active":"2024-09-19T09:00:47.965242+0000","last_peered":"2024-09-19T09:00:47.965242+0000","last_clean":"2024-09-19T09:00:47.965242+0000","last_became_active":"2024-09-19T09:00:46.964777+0000","last_became_peered":"2024-09-19T09:00:46.964777+0000","last_unstale":"2024-09-19T09:00:47.965242+0000","last_undegraded":"2024-09-19T09:00:47.965242+0000","last_fullsized":"2024-09-19T09:00:47.965242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:32.958984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":83,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.752534+0000","last_change":"2024-09-19T09:00:44.946193+0000","last_active":"2024-09-19T09:01:51.752534+0000","last_peered":"2024-09-19T09:01:51.752534+0000","last_clean":"2024-09-19T09:01:51.752534+0000","last_became_active":"2024-09-19T09:00:44.945168+0000","last_became_peered":"2024-09-19T09:00:44.945168+0000","last_unstale":"2024-09-19T09:01:51.752534+0000","last_undegraded":"2024-09-19T09:01:51.752534+0000","last_fullsized":"2024-09-19T09:01:51.752534+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:21.230689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964246+0000","last_change":"2024-09-19T09:00:46.959425+0000","last_active":"2024-09-19T09:00:47.964246+0000","last_peered":"2024-09-19T09:00:47.964246+0000","last_clean":"2024-09-19T09:00:47.964246+0000","last_became_active":"2024-09-19T09:00:46.958764+0000","last_became_peered":"2024-09-19T09:00:46.958764+0000","last_unstale":"2024-09-19T09:00:47.964246+0000","last_undegraded":"2024-09-19T09:00:47.964246+0000","last_fullsized":"2024-09-19T09:00:47.964246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:08:27.371080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.140076+0000","last_change":"2024-09-19T09:00:45.240084+0000","last_active":"2024-09-19T09:00:58.140076+0000","last_peered":"2024-09-19T09:00:58.140076+0000","last_clean":"2024-09-19T09:00:58.140076+0000","last_became_active":"2024-09-19T09:00:45.239222+0000","last_became_peered":"2024-09-19T09:00:45.239222+0000","last_unstale":"2024-09-19T09:00:58.140076+0000","last_undegraded":"2024-09-19T09:00:58.140076+0000","last_fullsized":"2024-09-19T09:00:58.140076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:00:56.643063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.757684+0000","last_change":"2024-09-19T09:01:52.757684+0000","last_active":"2024-09-19T09:01:52.757684+0000","last_peered":"2024-09-19T09:01:52.757684+0000","last_clean":"2024-09-19T09:01:52.757684+0000","last_became_active":"2024-09-19T09:01:52.756969+0000","last_became_peered":"2024-09-19T09:01:52.756969+0000","last_unstale":"2024-09-19T09:01:52.757684+0000","last_undegraded":"2024-09-19T09:01:52.757684+0000","last_fullsized":"2024-09-19T09:01:52.757684+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:24:47.117996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:57.115299+0000","last_change":"2024-09-19T09:00:44.946138+0000","last_active":"2024-09-19T09:00:57.115299+0000","last_peered":"2024-09-19T09:00:57.115299+0000","last_clean":"2024-09-19T09:00:57.115299+0000","last_became_active":"2024-09-19T09:00:44.945793+0000","last_became_peered":"2024-09-19T09:00:44.945793+0000","last_unstale":"2024-09-19T09:00:57.115299+0000","last_undegraded":"2024-09-19T09:00:57.115299+0000","last_fullsized":"2024-09-19T09:00:57.115299+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:01.341560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.969341+0000","last_change":"2024-09-19T09:00:46.970360+0000","last_active":"2024-09-19T09:00:47.969341+0000","last_peered":"2024-09-19T09:00:47.969341+0000","last_clean":"2024-09-19T09:00:47.969341+0000","last_became_active":"2024-09-19T09:00:46.969971+0000","last_became_peered":"2024-09-19T09:00:46.969971+0000","last_unstale":"2024-09-19T09:00:47.969341+0000","last_undegraded":"2024-09-19T09:00:47.969341+0000","last_fullsized":"2024-09-19T09:00:47.969341+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:48.812158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.748305+0000","last_change":"2024-09-19T09:00:44.941713+0000","last_active":"2024-09-19T09:01:52.748305+0000","last_peered":"2024-09-19T09:01:52.748305+0000","last_clean":"2024-09-19T09:01:52.748305+0000","last_became_active":"2024-09-19T09:00:44.941373+0000","last_became_peered":"2024-09-19T09:00:44.941373+0000","last_unstale":"2024-09-19T09:01:52.748305+0000","last_undegraded":"2024-09-19T09:01:52.748305+0000","last_fullsized":"2024-09-19T09:01:52.748305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:31.157803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.965581+0000","last_change":"2024-09-19T09:00:46.959383+0000","last_active":"2024-09-19T09:00:47.965581+0000","last_peered":"2024-09-19T09:00:47.965581+0000","last_clean":"2024-09-19T09:00:47.965581+0000","last_became_active":"2024-09-19T09:00:46.958860+0000","last_became_peered":"2024-09-19T09:00:46.958860+0000","last_unstale":"2024-09-19T09:00:47.965581+0000","last_undegraded":"2024-09-19T09:00:47.965581+0000","last_fullsized":"2024-09-19T09:00:47.965581+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:32:04.423017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752778+0000","last_change":"2024-09-19T09:00:44.944797+0000","last_active":"2024-09-19T09:01:52.752778+0000","last_peered":"2024-09-19T09:01:52.752778+0000","last_clean":"2024-09-19T09:01:52.752778+0000","last_became_active":"2024-09-19T09:00:44.944259+0000","last_became_peered":"2024-09-19T09:00:44.944259+0000","last_unstale":"2024-09-19T09:01:52.752778+0000","last_undegraded":"2024-09-19T09:01:52.752778+0000","last_fullsized":"2024-09-19T09:01:52.752778+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:58.996217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:49.684985+0000","last_change":"2024-09-19T09:00:46.964998+0000","last_active":"2024-09-19T09:00:49.684985+0000","last_peered":"2024-09-19T09:00:49.684985+0000","last_clean":"2024-09-19T09:00:49.684985+0000","last_became_active":"2024-09-19T09:00:46.964562+0000","last_became_peered":"2024-09-19T09:00:46.964562+0000","last_unstale":"2024-09-19T09:00:49.684985+0000","last_undegraded":"2024-09-19T09:00:49.684985+0000","last_fullsized":"2024-09-19T09:00:49.684985+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:00.089800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":81,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.753839+0000","last_change":"2024-09-19T09:00:45.239770+0000","last_active":"2024-09-19T09:01:51.753839+0000","last_peered":"2024-09-19T09:01:51.753839+0000","last_clean":"2024-09-19T09:01:51.753839+0000","last_became_active":"2024-09-19T09:00:45.239385+0000","last_became_peered":"2024-09-19T09:00:45.239385+0000","last_unstale":"2024-09-19T09:01:51.753839+0000","last_undegraded":"2024-09-19T09:01:51.753839+0000","last_fullsized":"2024-09-19T09:01:51.753839+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:21.109900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.966028+0000","last_change":"2024-09-19T09:00:46.967505+0000","last_active":"2024-09-19T09:00:47.966028+0000","last_peered":"2024-09-19T09:00:47.966028+0000","last_clean":"2024-09-19T09:00:47.966028+0000","last_became_active":"2024-09-19T09:00:46.967195+0000","last_became_peered":"2024-09-19T09:00:46.967195+0000","last_unstale":"2024-09-19T09:00:47.966028+0000","last_undegraded":"2024-09-19T09:00:47.966028+0000","last_fullsized":"2024-09-19T09:00:47.966028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:52:28.917333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":330,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964506+0000","last_change":"2024-09-19T08:59:33.821924+0000","last_active":"2024-09-19T09:00:47.964506+0000","last_peered":"2024-09-19T09:00:47.964506+0000","last_clean":"2024-09-19T09:00:47.964506+0000","last_became_active":"2024-09-19T08:59:33.821404+0000","last_became_peered":"2024-09-19T08:59:33.821404+0000","last_unstale":"2024-09-19T09:00:47.964506+0000","last_undegraded":"2024-09-19T09:00:47.964506+0000","last_fullsized":"2024-09-19T09:00:47.964506+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:56:07.830480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:56:07.830480+0000","last_clean_scrub_stamp":"2024-09-19T08:56:07.830480+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:21.763918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.751940+0000","last_change":"2024-09-19T09:00:44.943714+0000","last_active":"2024-09-19T09:01:51.751940+0000","last_peered":"2024-09-19T09:01:51.751940+0000","last_clean":"2024-09-19T09:01:51.751940+0000","last_became_active":"2024-09-19T09:00:44.943362+0000","last_became_peered":"2024-09-19T09:00:44.943362+0000","last_unstale":"2024-09-19T09:01:51.751940+0000","last_undegraded":"2024-09-19T09:01:51.751940+0000","last_fullsized":"2024-09-19T09:01:51.751940+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:30.015648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964922+0000","last_change":"2024-09-19T09:00:46.956784+0000","last_active":"2024-09-19T09:00:47.964922+0000","last_peered":"2024-09-19T09:00:47.964922+0000","last_clean":"2024-09-19T09:00:47.964922+0000","last_became_active":"2024-09-19T09:00:46.956443+0000","last_became_peered":"2024-09-19T09:00:46.956443+0000","last_unstale":"2024-09-19T09:00:47.964922+0000","last_undegraded":"2024-09-19T09:00:47.964922+0000","last_fullsized":"2024-09-19T09:00:47.964922+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:18.918788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:56.119049+0000","last_change":"2024-09-19T09:00:44.947796+0000","last_active":"2024-09-19T09:00:56.119049+0000","last_peered":"2024-09-19T09:00:56.119049+0000","last_clean":"2024-09-19T09:00:56.119049+0000","last_became_active":"2024-09-19T09:00:44.946949+0000","last_became_peered":"2024-09-19T09:00:44.946949+0000","last_unstale":"2024-09-19T09:00:56.119049+0000","last_undegraded":"2024-09-19T09:00:56.119049+0000","last_fullsized":"2024-09-19T09:00:56.119049+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:42.210186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.747892+0000","last_change":"2024-09-19T09:00:46.963856+0000","last_active":"2024-09-19T09:01:52.747892+0000","last_peered":"2024-09-19T09:01:52.747892+0000","last_clean":"2024-09-19T09:01:52.747892+0000","last_became_active":"2024-09-19T09:00:46.963543+0000","last_became_peered":"2024-09-19T09:00:46.963543+0000","last_unstale":"2024-09-19T09:01:52.747892+0000","last_undegraded":"2024-09-19T09:01:52.747892+0000","last_fullsized":"2024-09-19T09:01:52.747892+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:31.041028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":83,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.748056+0000","last_change":"2024-09-19T09:00:45.239269+0000","last_active":"2024-09-19T09:01:52.748056+0000","last_peered":"2024-09-19T09:01:52.748056+0000","last_clean":"2024-09-19T09:01:52.748056+0000","last_became_active":"2024-09-19T09:00:45.238929+0000","last_became_peered":"2024-09-19T09:00:45.238929+0000","last_unstale":"2024-09-19T09:01:52.748056+0000","last_undegraded":"2024-09-19T09:01:52.748056+0000","last_fullsized":"2024-09-19T09:01:52.748056+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:46.684148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751371+0000","last_change":"2024-09-19T09:00:46.963595+0000","last_active":"2024-09-19T09:01:52.751371+0000","last_peered":"2024-09-19T09:01:52.751371+0000","last_clean":"2024-09-19T09:01:52.751371+0000","last_became_active":"2024-09-19T09:00:46.963243+0000","last_became_peered":"2024-09-19T09:00:46.963243+0000","last_unstale":"2024-09-19T09:01:52.751371+0000","last_undegraded":"2024-09-19T09:01:52.751371+0000","last_fullsized":"2024-09-19T09:01:52.751371+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:01.122996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":75,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:01:48.211356+0000","last_change":"2024-09-19T09:00:44.941550+0000","last_active":"2024-09-19T09:01:48.211356+0000","last_peered":"2024-09-19T09:01:48.211356+0000","last_clean":"2024-09-19T09:01:48.211356+0000","last_became_active":"2024-09-19T09:00:44.941145+0000","last_became_peered":"2024-09-19T09:00:44.941145+0000","last_unstale":"2024-09-19T09:01:48.211356+0000","last_undegraded":"2024-09-19T09:01:48.211356+0000","last_fullsized":"2024-09-19T09:01:48.211356+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:53.893887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964440+0000","last_change":"2024-09-19T09:00:46.963473+0000","last_active":"2024-09-19T09:00:47.964440+0000","last_peered":"2024-09-19T09:00:47.964440+0000","last_clean":"2024-09-19T09:00:47.964440+0000","last_became_active":"2024-09-19T09:00:46.963090+0000","last_became_peered":"2024-09-19T09:00:46.963090+0000","last_unstale":"2024-09-19T09:00:47.964440+0000","last_undegraded":"2024-09-19T09:00:47.964440+0000","last_fullsized":"2024-09-19T09:00:47.964440+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:01.492385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.962784+0000","last_change":"2024-09-19T09:00:45.239566+0000","last_active":"2024-09-19T09:00:47.962784+0000","last_peered":"2024-09-19T09:00:47.962784+0000","last_clean":"2024-09-19T09:00:47.962784+0000","last_became_active":"2024-09-19T09:00:45.239250+0000","last_became_peered":"2024-09-19T09:00:45.239250+0000","last_unstale":"2024-09-19T09:00:47.962784+0000","last_undegraded":"2024-09-19T09:00:47.962784+0000","last_fullsized":"2024-09-19T09:00:47.962784+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:44:21.739311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:50.839180+0000","last_change":"2024-09-19T09:00:46.955344+0000","last_active":"2024-09-19T09:00:50.839180+0000","last_peered":"2024-09-19T09:00:50.839180+0000","last_clean":"2024-09-19T09:00:50.839180+0000","last_became_active":"2024-09-19T09:00:46.954961+0000","last_became_peered":"2024-09-19T09:00:46.954961+0000","last_unstale":"2024-09-19T09:00:50.839180+0000","last_undegraded":"2024-09-19T09:00:50.839180+0000","last_fullsized":"2024-09-19T09:00:50.839180+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:55.743556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.141069+0000","last_change":"2024-09-19T09:00:45.241514+0000","last_active":"2024-09-19T09:00:58.141069+0000","last_peered":"2024-09-19T09:00:58.141069+0000","last_clean":"2024-09-19T09:00:58.141069+0000","last_became_active":"2024-09-19T09:00:45.241097+0000","last_became_peered":"2024-09-19T09:00:45.241097+0000","last_unstale":"2024-09-19T09:00:58.141069+0000","last_undegraded":"2024-09-19T09:00:58.141069+0000","last_fullsized":"2024-09-19T09:00:58.141069+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:35:38.323393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.748177+0000","last_change":"2024-09-19T09:00:46.955194+0000","last_active":"2024-09-19T09:01:52.748177+0000","last_peered":"2024-09-19T09:01:52.748177+0000","last_clean":"2024-09-19T09:01:52.748177+0000","last_became_active":"2024-09-19T09:00:46.954771+0000","last_became_peered":"2024-09-19T09:00:46.954771+0000","last_unstale":"2024-09-19T09:01:52.748177+0000","last_undegraded":"2024-09-19T09:01:52.748177+0000","last_fullsized":"2024-09-19T09:01:52.748177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:18.945035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'14","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:01:11.965733+0000","last_change":"2024-09-19T09:00:44.938554+0000","last_active":"2024-09-19T09:01:11.965733+0000","last_peered":"2024-09-19T09:01:11.965733+0000","last_clean":"2024-09-19T09:01:11.965733+0000","last_became_active":"2024-09-19T09:00:44.938261+0000","last_became_peered":"2024-09-19T09:00:44.938261+0000","last_unstale":"2024-09-19T09:01:11.965733+0000","last_undegraded":"2024-09-19T09:01:11.965733+0000","last_fullsized":"2024-09-19T09:01:11.965733+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:24.293764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964806+0000","last_change":"2024-09-19T09:00:46.960435+0000","last_active":"2024-09-19T09:00:47.964806+0000","last_peered":"2024-09-19T09:00:47.964806+0000","last_clean":"2024-09-19T09:00:47.964806+0000","last_became_active":"2024-09-19T09:00:46.960138+0000","last_became_peered":"2024-09-19T09:00:46.960138+0000","last_unstale":"2024-09-19T09:00:47.964806+0000","last_undegraded":"2024-09-19T09:00:47.964806+0000","last_fullsized":"2024-09-19T09:00:47.964806+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:59:50.853344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.750263+0000","last_change":"2024-09-19T09:00:44.941479+0000","last_active":"2024-09-19T09:01:51.750263+0000","last_peered":"2024-09-19T09:01:51.750263+0000","last_clean":"2024-09-19T09:01:51.750263+0000","last_became_active":"2024-09-19T09:00:44.937197+0000","last_became_peered":"2024-09-19T09:00:44.937197+0000","last_unstale":"2024-09-19T09:01:51.750263+0000","last_undegraded":"2024-09-19T09:01:51.750263+0000","last_fullsized":"2024-09-19T09:01:51.750263+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:53:17.385884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.754752+0000","last_change":"2024-09-19T09:00:46.970319+0000","last_active":"2024-09-19T09:01:52.754752+0000","last_peered":"2024-09-19T09:01:52.754752+0000","last_clean":"2024-09-19T09:01:52.754752+0000","last_became_active":"2024-09-19T09:00:46.969899+0000","last_became_peered":"2024-09-19T09:00:46.969899+0000","last_unstale":"2024-09-19T09:01:52.754752+0000","last_undegraded":"2024-09-19T09:01:52.754752+0000","last_fullsized":"2024-09-19T09:01:52.754752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:26:21.379231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.750046+0000","last_change":"2024-09-19T09:00:45.239525+0000","last_active":"2024-09-19T09:01:51.750046+0000","last_peered":"2024-09-19T09:01:51.750046+0000","last_clean":"2024-09-19T09:01:51.750046+0000","last_became_active":"2024-09-19T09:00:45.239213+0000","last_became_peered":"2024-09-19T09:00:45.239213+0000","last_unstale":"2024-09-19T09:01:51.750046+0000","last_undegraded":"2024-09-19T09:01:51.750046+0000","last_fullsized":"2024-09-19T09:01:51.750046+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:32.525025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.953631+0000","last_change":"2024-09-19T09:00:46.958573+0000","last_active":"2024-09-19T09:00:47.953631+0000","last_peered":"2024-09-19T09:00:47.953631+0000","last_clean":"2024-09-19T09:00:47.953631+0000","last_became_active":"2024-09-19T09:00:46.958274+0000","last_became_peered":"2024-09-19T09:00:46.958274+0000","last_unstale":"2024-09-19T09:00:47.953631+0000","last_undegraded":"2024-09-19T09:00:47.953631+0000","last_fullsized":"2024-09-19T09:00:47.953631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:17.249122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.139345+0000","last_change":"2024-09-19T09:00:44.945194+0000","last_active":"2024-09-19T09:00:58.139345+0000","last_peered":"2024-09-19T09:00:58.139345+0000","last_clean":"2024-09-19T09:00:58.139345+0000","last_became_active":"2024-09-19T09:00:44.944931+0000","last_became_peered":"2024-09-19T09:00:44.944931+0000","last_unstale":"2024-09-19T09:00:58.139345+0000","last_undegraded":"2024-09-19T09:00:58.139345+0000","last_fullsized":"2024-09-19T09:00:58.139345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:11.181150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.969572+0000","last_change":"2024-09-19T09:00:46.966895+0000","last_active":"2024-09-19T09:00:47.969572+0000","last_peered":"2024-09-19T09:00:47.969572+0000","last_clean":"2024-09-19T09:00:47.969572+0000","last_became_active":"2024-09-19T09:00:46.966124+0000","last_became_peered":"2024-09-19T09:00:46.966124+0000","last_unstale":"2024-09-19T09:00:47.969572+0000","last_undegraded":"2024-09-19T09:00:47.969572+0000","last_fullsized":"2024-09-19T09:00:47.969572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:14:04.320863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.753137+0000","last_change":"2024-09-19T09:00:44.944167+0000","last_active":"2024-09-19T09:01:51.753137+0000","last_peered":"2024-09-19T09:01:51.753137+0000","last_clean":"2024-09-19T09:01:51.753137+0000","last_became_active":"2024-09-19T09:00:44.943670+0000","last_became_peered":"2024-09-19T09:00:44.943670+0000","last_unstale":"2024-09-19T09:01:51.753137+0000","last_undegraded":"2024-09-19T09:01:51.753137+0000","last_fullsized":"2024-09-19T09:01:51.753137+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:05.596877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964961+0000","last_change":"2024-09-19T09:00:46.958581+0000","last_active":"2024-09-19T09:00:47.964961+0000","last_peered":"2024-09-19T09:00:47.964961+0000","last_clean":"2024-09-19T09:00:47.964961+0000","last_became_active":"2024-09-19T09:00:46.957782+0000","last_became_peered":"2024-09-19T09:00:46.957782+0000","last_unstale":"2024-09-19T09:00:47.964961+0000","last_undegraded":"2024-09-19T09:00:47.964961+0000","last_fullsized":"2024-09-19T09:00:47.964961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:37:19.488538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:55.110727+0000","last_change":"2024-09-19T09:00:44.945351+0000","last_active":"2024-09-19T09:00:55.110727+0000","last_peered":"2024-09-19T09:00:55.110727+0000","last_clean":"2024-09-19T09:00:55.110727+0000","last_became_active":"2024-09-19T09:00:44.944991+0000","last_became_peered":"2024-09-19T09:00:44.944991+0000","last_unstale":"2024-09-19T09:00:55.110727+0000","last_undegraded":"2024-09-19T09:00:55.110727+0000","last_fullsized":"2024-09-19T09:00:55.110727+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:45:07.581872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964606+0000","last_change":"2024-09-19T09:00:46.956340+0000","last_active":"2024-09-19T09:00:47.964606+0000","last_peered":"2024-09-19T09:00:47.964606+0000","last_clean":"2024-09-19T09:00:47.964606+0000","last_became_active":"2024-09-19T09:00:46.955956+0000","last_became_peered":"2024-09-19T09:00:46.955956+0000","last_unstale":"2024-09-19T09:00:47.964606+0000","last_undegraded":"2024-09-19T09:00:47.964606+0000","last_fullsized":"2024-09-19T09:00:47.964606+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:34.863428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752335+0000","last_change":"2024-09-19T09:00:46.963657+0000","last_active":"2024-09-19T09:01:52.752335+0000","last_peered":"2024-09-19T09:01:52.752335+0000","last_clean":"2024-09-19T09:01:52.752335+0000","last_became_active":"2024-09-19T09:00:46.963134+0000","last_became_peered":"2024-09-19T09:00:46.963134+0000","last_unstale":"2024-09-19T09:01:52.752335+0000","last_undegraded":"2024-09-19T09:01:52.752335+0000","last_fullsized":"2024-09-19T09:01:52.752335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:01.098570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.141281+0000","last_change":"2024-09-19T09:00:44.945382+0000","last_active":"2024-09-19T09:00:58.141281+0000","last_peered":"2024-09-19T09:00:58.141281+0000","last_clean":"2024-09-19T09:00:58.141281+0000","last_became_active":"2024-09-19T09:00:44.945061+0000","last_became_peered":"2024-09-19T09:00:44.945061+0000","last_unstale":"2024-09-19T09:00:58.141281+0000","last_undegraded":"2024-09-19T09:00:58.141281+0000","last_fullsized":"2024-09-19T09:00:58.141281+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:07:12.747709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.750394+0000","last_change":"2024-09-19T09:00:46.956354+0000","last_active":"2024-09-19T09:01:51.750394+0000","last_peered":"2024-09-19T09:01:51.750394+0000","last_clean":"2024-09-19T09:01:51.750394+0000","last_became_active":"2024-09-19T09:00:46.955964+0000","last_became_peered":"2024-09-19T09:00:46.955964+0000","last_unstale":"2024-09-19T09:01:51.750394+0000","last_undegraded":"2024-09-19T09:01:51.750394+0000","last_fullsized":"2024-09-19T09:01:51.750394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:05:33.243479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:59.136540+0000","last_change":"2024-09-19T09:00:44.944983+0000","last_active":"2024-09-19T09:00:59.136540+0000","last_peered":"2024-09-19T09:00:59.136540+0000","last_clean":"2024-09-19T09:00:59.136540+0000","last_became_active":"2024-09-19T09:00:44.944625+0000","last_became_peered":"2024-09-19T09:00:44.944625+0000","last_unstale":"2024-09-19T09:00:59.136540+0000","last_undegraded":"2024-09-19T09:00:59.136540+0000","last_fullsized":"2024-09-19T09:00:59.136540+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:49:50.818863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.754929+0000","last_change":"2024-09-19T09:00:46.966448+0000","last_active":"2024-09-19T09:01:52.754929+0000","last_peered":"2024-09-19T09:01:52.754929+0000","last_clean":"2024-09-19T09:01:52.754929+0000","last_became_active":"2024-09-19T09:00:46.966135+0000","last_became_peered":"2024-09-19T09:00:46.966135+0000","last_unstale":"2024-09-19T09:01:52.754929+0000","last_undegraded":"2024-09-19T09:01:52.754929+0000","last_fullsized":"2024-09-19T09:01:52.754929+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:42.652877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.967059+0000","last_change":"2024-09-19T09:00:45.239405+0000","last_active":"2024-09-19T09:00:47.967059+0000","last_peered":"2024-09-19T09:00:47.967059+0000","last_clean":"2024-09-19T09:00:47.967059+0000","last_became_active":"2024-09-19T09:00:45.239002+0000","last_became_peered":"2024-09-19T09:00:45.239002+0000","last_unstale":"2024-09-19T09:00:47.967059+0000","last_undegraded":"2024-09-19T09:00:47.967059+0000","last_fullsized":"2024-09-19T09:00:47.967059+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:46:03.934975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.965411+0000","last_change":"2024-09-19T09:00:46.951635+0000","last_active":"2024-09-19T09:00:47.965411+0000","last_peered":"2024-09-19T09:00:47.965411+0000","last_clean":"2024-09-19T09:00:47.965411+0000","last_became_active":"2024-09-19T09:00:46.951325+0000","last_became_peered":"2024-09-19T09:00:46.951325+0000","last_unstale":"2024-09-19T09:00:47.965411+0000","last_undegraded":"2024-09-19T09:00:47.965411+0000","last_fullsized":"2024-09-19T09:00:47.965411+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:17.360950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":85,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:01:50.213568+0000","last_change":"2024-09-19T09:00:44.942662+0000","last_active":"2024-09-19T09:01:50.213568+0000","last_peered":"2024-09-19T09:01:50.213568+0000","last_clean":"2024-09-19T09:01:50.213568+0000","last_became_active":"2024-09-19T09:00:44.942397+0000","last_became_peered":"2024-09-19T09:00:44.942397+0000","last_unstale":"2024-09-19T09:01:50.213568+0000","last_undegraded":"2024-09-19T09:01:50.213568+0000","last_fullsized":"2024-09-19T09:01:50.213568+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:46:35.247703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69125,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":569,"num_read_kb":360,"num_write":243,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":207375,"data_compressed":4815,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":212166,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612634,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":860160,"data_stored":410932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841567,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1015808,"data_stored":1064664,"data_compressed":37512,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972389,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":410862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201321,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":411240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430255,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":1003760,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593781,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":892928,"data_stored":427768,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822713,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":970752,"data_stored":1020234,"data_compressed":41690,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986239,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":880640,"data_stored":419534,"data_compressed":2888,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215171,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":410404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":962560,"data_stored":482723,"data_compressed":4493,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":942080,"data_stored":482503,"data_compressed":4493,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":913408,"data_stored":429107,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":64062,"data_compressed":1605,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64282,"data_compressed":1605,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":64439,"data_compressed":1605,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:01:54.501 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T09:01:54.745 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:55.030 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:54 smithi120 ceph-mon[31150]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T09:01:55.030 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:54 smithi120 ceph-mon[31150]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:01:55.030 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:54 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:01:55.030 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:54 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:01:55.030 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:54 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:01:55.031 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:54 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:01:55.031 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:54 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:01:55.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:54 smithi196 ceph-mon[35150]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T09:01:55.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:54 smithi196 ceph-mon[35150]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:01:55.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:54 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:01:55.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:54 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:01:55.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:54 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:01:55.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:54 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:01:55.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:54 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:01:55.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:54 smithi153 ceph-mon[35071]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T09:01:55.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:54 smithi153 ceph-mon[35071]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:01:55.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:54 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:01:55.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:54 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:01:55.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:54 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:01:55.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:54 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:01:55.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:54 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:01:55.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.272+0000 7f59be10b640 1 -- 172.21.15.120:0/3153354835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59b8108300 msgr2=0x7f59b810ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:55.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.272+0000 7f59be10b640 1 --2- 172.21.15.120:0/3153354835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59b8108300 0x7f59b810ec70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f59b0009930 tx=0x7f59b002a420 comp rx=0 tx=0).stop 2024-09-19T09:01:55.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.273+0000 7f59be10b640 1 -- 172.21.15.120:0/3153354835 shutdown_connections 2024-09-19T09:01:55.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.273+0000 7f59be10b640 1 --2- 172.21.15.120:0/3153354835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59b8108300 0x7f59b810ec70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:55.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.273+0000 7f59be10b640 1 --2- 172.21.15.120:0/3153354835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59b8103780 0x7f59b8107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:55.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.273+0000 7f59be10b640 1 --2- 172.21.15.120:0/3153354835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59b8102dd0 0x7f59b81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:55.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.273+0000 7f59be10b640 1 -- 172.21.15.120:0/3153354835 >> 172.21.15.120:0/3153354835 conn(0x7f59b80fc7d0 msgr2=0x7f59b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:55.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.273+0000 7f59be10b640 1 -- 172.21.15.120:0/3153354835 shutdown_connections 2024-09-19T09:01:55.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.273+0000 7f59be10b640 1 -- 172.21.15.120:0/3153354835 wait complete. 2024-09-19T09:01:55.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.273+0000 7f59be10b640 1 Processor -- start 2024-09-19T09:01:55.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.273+0000 7f59be10b640 1 -- start start 2024-09-19T09:01:55.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.274+0000 7f59be10b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59b8102dd0 0x7f59b81984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:55.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.274+0000 7f59be10b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59b8103780 0x7f59b8198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:55.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.274+0000 7f59be10b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59b8108300 0x7f59b819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:55.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.274+0000 7f59be10b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f59b81120d0 con 0x7f59b8102dd0 2024-09-19T09:01:55.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.274+0000 7f59be10b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f59b8111f50 con 0x7f59b8103780 2024-09-19T09:01:55.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.274+0000 7f59be10b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f59b8112250 con 0x7f59b8108300 2024-09-19T09:01:55.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.274+0000 7f59b77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59b8102dd0 0x7f59b81984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:55.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.274+0000 7f59b77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59b8102dd0 0x7f59b81984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52832/0 (socket says 172.21.15.120:52832) 2024-09-19T09:01:55.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.274+0000 7f59b77fe640 1 -- 172.21.15.120:0/1110176748 learned_addr learned my addr 172.21.15.120:0/1110176748 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:55.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.274+0000 7f59b6ffd640 1 --2- 172.21.15.120:0/1110176748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59b8103780 0x7f59b8198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:55.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.274+0000 7f59b7fff640 1 --2- 172.21.15.120:0/1110176748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59b8108300 0x7f59b819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:55.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.274+0000 7f59b77fe640 1 -- 172.21.15.120:0/1110176748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59b8108300 msgr2=0x7f59b819cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:55.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.274+0000 7f59b77fe640 1 --2- 172.21.15.120:0/1110176748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59b8108300 0x7f59b819cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:55.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.274+0000 7f59b77fe640 1 -- 172.21.15.120:0/1110176748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59b8103780 msgr2=0x7f59b8198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:55.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.274+0000 7f59b77fe640 1 --2- 172.21.15.120:0/1110176748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59b8103780 0x7f59b8198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:55.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.274+0000 7f59b77fe640 1 -- 172.21.15.120:0/1110176748 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b819d4a0 con 0x7f59b8102dd0 2024-09-19T09:01:55.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.274+0000 7f59b77fe640 1 --2- 172.21.15.120:0/1110176748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59b8102dd0 0x7f59b81984f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f59a800cce0 tx=0x7f59a800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:55.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.274+0000 7f59b4ff9640 1 -- 172.21.15.120:0/1110176748 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a8004b20 con 0x7f59b8102dd0 2024-09-19T09:01:55.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.274+0000 7f59be10b640 1 -- 172.21.15.120:0/1110176748 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59b819d790 con 0x7f59b8102dd0 2024-09-19T09:01:55.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.274+0000 7f59b4ff9640 1 -- 172.21.15.120:0/1110176748 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59a8004cc0 con 0x7f59b8102dd0 2024-09-19T09:01:55.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.274+0000 7f59b4ff9640 1 -- 172.21.15.120:0/1110176748 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a800f6b0 con 0x7f59b8102dd0 2024-09-19T09:01:55.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.275+0000 7f59be10b640 1 -- 172.21.15.120:0/1110176748 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59b81a5070 con 0x7f59b8102dd0 2024-09-19T09:01:55.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.275+0000 7f59be10b640 1 -- 172.21.15.120:0/1110176748 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59b81031b0 con 0x7f59b8102dd0 2024-09-19T09:01:55.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.277+0000 7f59b4ff9640 1 -- 172.21.15.120:0/1110176748 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f59a8005000 con 0x7f59b8102dd0 2024-09-19T09:01:55.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.278+0000 7f59b4ff9640 1 --2- 172.21.15.120:0/1110176748 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f59a00a32d0 0x7f59a00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:55.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.278+0000 7f59b6ffd640 1 --2- 172.21.15.120:0/1110176748 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f59a00a32d0 0x7f59a00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:55.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.278+0000 7f59b4ff9640 1 -- 172.21.15.120:0/1110176748 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f59a80cfba0 con 0x7f59b8102dd0 2024-09-19T09:01:55.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.278+0000 7f59b6ffd640 1 --2- 172.21.15.120:0/1110176748 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f59a00a32d0 0x7f59a00a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f59b80fedd0 tx=0x7f59ac00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:55.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.281+0000 7f59b4ff9640 1 -- 172.21.15.120:0/1110176748 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f59a8014030 con 0x7f59b8102dd0 2024-09-19T09:01:55.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.444+0000 7f59be10b640 1 -- 172.21.15.120:0/1110176748 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f59b8107450 con 0x7f59a00a32d0 2024-09-19T09:01:55.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.450+0000 7f59b4ff9640 1 -- 172.21.15.120:0/1110176748 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258986 (secure 0 0 0) 0x7f59b8107450 con 0x7f59a00a32d0 2024-09-19T09:01:55.452 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:01:55.453 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-19T09:01:55.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.452+0000 7f59be10b640 1 -- 172.21.15.120:0/1110176748 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f59a00a32d0 msgr2=0x7f59a00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:55.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.452+0000 7f59be10b640 1 --2- 172.21.15.120:0/1110176748 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f59a00a32d0 0x7f59a00a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f59b80fedd0 tx=0x7f59ac00a2d0 comp rx=0 tx=0).stop 2024-09-19T09:01:55.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.452+0000 7f59be10b640 1 -- 172.21.15.120:0/1110176748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59b8102dd0 msgr2=0x7f59b81984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:55.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.452+0000 7f59be10b640 1 --2- 172.21.15.120:0/1110176748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59b8102dd0 0x7f59b81984f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f59a800cce0 tx=0x7f59a800a620 comp rx=0 tx=0).stop 2024-09-19T09:01:55.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.452+0000 7f59be10b640 1 -- 172.21.15.120:0/1110176748 shutdown_connections 2024-09-19T09:01:55.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.452+0000 7f59be10b640 1 --2- 172.21.15.120:0/1110176748 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f59a00a32d0 0x7f59a00a5790 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:55.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.453+0000 7f59be10b640 1 --2- 172.21.15.120:0/1110176748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59b8108300 0x7f59b819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:55.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.453+0000 7f59be10b640 1 --2- 172.21.15.120:0/1110176748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59b8103780 0x7f59b8198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:55.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.453+0000 7f59be10b640 1 --2- 172.21.15.120:0/1110176748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59b8102dd0 0x7f59b81984f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:55.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.453+0000 7f59be10b640 1 -- 172.21.15.120:0/1110176748 >> 172.21.15.120:0/1110176748 conn(0x7f59b80fc7d0 msgr2=0x7f59b8105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:55.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.453+0000 7f59be10b640 1 -- 172.21.15.120:0/1110176748 shutdown_connections 2024-09-19T09:01:55.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:55.453+0000 7f59be10b640 1 -- 172.21.15.120:0/1110176748 wait complete. 2024-09-19T09:01:56.188 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2024-09-19T09:01:53.623477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659727,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":695,"num_read_kb":469,"num_write":426,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":290,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364204,"kb_used_data":10764,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577620,"statfs":{"total":1151940427776,"available":1151567482880,"internally_reserved":0,"allocated":11022336,"data_stored":6973731,"data_compressed":138549,"data_compressed_allocated":184320,"data_compressed_original":1867776,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":95,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001729"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.969591+0000","last_change":"2024-09-19T09:00:46.966628+0000","last_active":"2024-09-19T09:00:47.969591+0000","last_peered":"2024-09-19T09:00:47.969591+0000","last_clean":"2024-09-19T09:00:47.969591+0000","last_became_active":"2024-09-19T09:00:46.966290+0000","last_became_peered":"2024-09-19T09:00:46.966290+0000","last_unstale":"2024-09-19T09:00:47.969591+0000","last_undegraded":"2024-09-19T09:00:47.969591+0000","last_fullsized":"2024-09-19T09:00:47.969591+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:42.717139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.754728+0000","last_change":"2024-09-19T09:00:46.963706+0000","last_active":"2024-09-19T09:01:52.754728+0000","last_peered":"2024-09-19T09:01:52.754728+0000","last_clean":"2024-09-19T09:01:52.754728+0000","last_became_active":"2024-09-19T09:00:46.963303+0000","last_became_peered":"2024-09-19T09:00:46.963303+0000","last_unstale":"2024-09-19T09:01:52.754728+0000","last_undegraded":"2024-09-19T09:01:52.754728+0000","last_fullsized":"2024-09-19T09:01:52.754728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:07.451500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.754557+0000","last_change":"2024-09-19T09:00:46.967149+0000","last_active":"2024-09-19T09:01:52.754557+0000","last_peered":"2024-09-19T09:01:52.754557+0000","last_clean":"2024-09-19T09:01:52.754557+0000","last_became_active":"2024-09-19T09:00:46.966861+0000","last_became_peered":"2024-09-19T09:00:46.966861+0000","last_unstale":"2024-09-19T09:01:52.754557+0000","last_undegraded":"2024-09-19T09:01:52.754557+0000","last_fullsized":"2024-09-19T09:01:52.754557+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:28.396022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.750139+0000","last_change":"2024-09-19T09:00:46.969242+0000","last_active":"2024-09-19T09:01:51.750139+0000","last_peered":"2024-09-19T09:01:51.750139+0000","last_clean":"2024-09-19T09:01:51.750139+0000","last_became_active":"2024-09-19T09:00:46.968546+0000","last_became_peered":"2024-09-19T09:00:46.968546+0000","last_unstale":"2024-09-19T09:01:51.750139+0000","last_undegraded":"2024-09-19T09:01:51.750139+0000","last_fullsized":"2024-09-19T09:01:51.750139+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:43:51.060130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.953778+0000","last_change":"2024-09-19T09:00:46.955501+0000","last_active":"2024-09-19T09:00:47.953778+0000","last_peered":"2024-09-19T09:00:47.953778+0000","last_clean":"2024-09-19T09:00:47.953778+0000","last_became_active":"2024-09-19T09:00:46.955161+0000","last_became_peered":"2024-09-19T09:00:46.955161+0000","last_unstale":"2024-09-19T09:00:47.953778+0000","last_undegraded":"2024-09-19T09:00:47.953778+0000","last_fullsized":"2024-09-19T09:00:47.953778+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:39:19.005931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.965095+0000","last_change":"2024-09-19T09:00:46.962577+0000","last_active":"2024-09-19T09:00:47.965095+0000","last_peered":"2024-09-19T09:00:47.965095+0000","last_clean":"2024-09-19T09:00:47.965095+0000","last_became_active":"2024-09-19T09:00:46.962253+0000","last_became_peered":"2024-09-19T09:00:46.962253+0000","last_unstale":"2024-09-19T09:00:47.965095+0000","last_undegraded":"2024-09-19T09:00:47.965095+0000","last_fullsized":"2024-09-19T09:00:47.965095+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:27.925932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964437+0000","last_change":"2024-09-19T09:00:46.960004+0000","last_active":"2024-09-19T09:00:47.964437+0000","last_peered":"2024-09-19T09:00:47.964437+0000","last_clean":"2024-09-19T09:00:47.964437+0000","last_became_active":"2024-09-19T09:00:46.959664+0000","last_became_peered":"2024-09-19T09:00:46.959664+0000","last_unstale":"2024-09-19T09:00:47.964437+0000","last_undegraded":"2024-09-19T09:00:47.964437+0000","last_fullsized":"2024-09-19T09:00:47.964437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:41.616309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964731+0000","last_change":"2024-09-19T09:00:46.952394+0000","last_active":"2024-09-19T09:00:47.964731+0000","last_peered":"2024-09-19T09:00:47.964731+0000","last_clean":"2024-09-19T09:00:47.964731+0000","last_became_active":"2024-09-19T09:00:46.952084+0000","last_became_peered":"2024-09-19T09:00:46.952084+0000","last_unstale":"2024-09-19T09:00:47.964731+0000","last_undegraded":"2024-09-19T09:00:47.964731+0000","last_fullsized":"2024-09-19T09:00:47.964731+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:21.698423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.750363+0000","last_change":"2024-09-19T09:00:46.961262+0000","last_active":"2024-09-19T09:01:51.750363+0000","last_peered":"2024-09-19T09:01:51.750363+0000","last_clean":"2024-09-19T09:01:51.750363+0000","last_became_active":"2024-09-19T09:00:46.960951+0000","last_became_peered":"2024-09-19T09:00:46.960951+0000","last_unstale":"2024-09-19T09:01:51.750363+0000","last_undegraded":"2024-09-19T09:01:51.750363+0000","last_fullsized":"2024-09-19T09:01:51.750363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:02.445108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.966156+0000","last_change":"2024-09-19T09:00:46.964806+0000","last_active":"2024-09-19T09:00:47.966156+0000","last_peered":"2024-09-19T09:00:47.966156+0000","last_clean":"2024-09-19T09:00:47.966156+0000","last_became_active":"2024-09-19T09:00:46.964194+0000","last_became_peered":"2024-09-19T09:00:46.964194+0000","last_unstale":"2024-09-19T09:00:47.966156+0000","last_undegraded":"2024-09-19T09:00:47.966156+0000","last_fullsized":"2024-09-19T09:00:47.966156+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:31.621075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.966443+0000","last_change":"2024-09-19T09:00:46.963200+0000","last_active":"2024-09-19T09:00:47.966443+0000","last_peered":"2024-09-19T09:00:47.966443+0000","last_clean":"2024-09-19T09:00:47.966443+0000","last_became_active":"2024-09-19T09:00:46.960496+0000","last_became_peered":"2024-09-19T09:00:46.960496+0000","last_unstale":"2024-09-19T09:00:47.966443+0000","last_undegraded":"2024-09-19T09:00:47.966443+0000","last_fullsized":"2024-09-19T09:00:47.966443+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:59:47.234492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.750254+0000","last_change":"2024-09-19T09:00:46.966309+0000","last_active":"2024-09-19T09:01:51.750254+0000","last_peered":"2024-09-19T09:01:51.750254+0000","last_clean":"2024-09-19T09:01:51.750254+0000","last_became_active":"2024-09-19T09:00:46.965970+0000","last_became_peered":"2024-09-19T09:00:46.965970+0000","last_unstale":"2024-09-19T09:01:51.750254+0000","last_undegraded":"2024-09-19T09:01:51.750254+0000","last_fullsized":"2024-09-19T09:01:51.750254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:38.353206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.748099+0000","last_change":"2024-09-19T09:00:46.954079+0000","last_active":"2024-09-19T09:01:52.748099+0000","last_peered":"2024-09-19T09:01:52.748099+0000","last_clean":"2024-09-19T09:01:52.748099+0000","last_became_active":"2024-09-19T09:00:46.953398+0000","last_became_peered":"2024-09-19T09:00:46.953398+0000","last_unstale":"2024-09-19T09:01:52.748099+0000","last_undegraded":"2024-09-19T09:01:52.748099+0000","last_fullsized":"2024-09-19T09:01:52.748099+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:54:27.554800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.754336+0000","last_change":"2024-09-19T09:00:46.965347+0000","last_active":"2024-09-19T09:01:52.754336+0000","last_peered":"2024-09-19T09:01:52.754336+0000","last_clean":"2024-09-19T09:01:52.754336+0000","last_became_active":"2024-09-19T09:00:46.965045+0000","last_became_peered":"2024-09-19T09:00:46.965045+0000","last_unstale":"2024-09-19T09:01:52.754336+0000","last_undegraded":"2024-09-19T09:01:52.754336+0000","last_fullsized":"2024-09-19T09:01:52.754336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:27.316100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.754183+0000","last_change":"2024-09-19T09:00:46.962898+0000","last_active":"2024-09-19T09:01:52.754183+0000","last_peered":"2024-09-19T09:01:52.754183+0000","last_clean":"2024-09-19T09:01:52.754183+0000","last_became_active":"2024-09-19T09:00:46.962412+0000","last_became_peered":"2024-09-19T09:00:46.962412+0000","last_unstale":"2024-09-19T09:01:52.754183+0000","last_undegraded":"2024-09-19T09:01:52.754183+0000","last_fullsized":"2024-09-19T09:01:52.754183+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:40.483678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.749841+0000","last_change":"2024-09-19T09:00:46.958147+0000","last_active":"2024-09-19T09:01:51.749841+0000","last_peered":"2024-09-19T09:01:51.749841+0000","last_clean":"2024-09-19T09:01:51.749841+0000","last_became_active":"2024-09-19T09:00:46.957805+0000","last_became_peered":"2024-09-19T09:00:46.957805+0000","last_unstale":"2024-09-19T09:01:51.749841+0000","last_undegraded":"2024-09-19T09:01:51.749841+0000","last_fullsized":"2024-09-19T09:01:51.749841+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:24:17.000064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.969691+0000","last_change":"2024-09-19T09:00:46.967399+0000","last_active":"2024-09-19T09:00:47.969691+0000","last_peered":"2024-09-19T09:00:47.969691+0000","last_clean":"2024-09-19T09:00:47.969691+0000","last_became_active":"2024-09-19T09:00:46.966145+0000","last_became_peered":"2024-09-19T09:00:46.966145+0000","last_unstale":"2024-09-19T09:00:47.969691+0000","last_undegraded":"2024-09-19T09:00:47.969691+0000","last_fullsized":"2024-09-19T09:00:47.969691+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:29.192469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.962791+0000","last_change":"2024-09-19T09:00:46.960956+0000","last_active":"2024-09-19T09:00:47.962791+0000","last_peered":"2024-09-19T09:00:47.962791+0000","last_clean":"2024-09-19T09:00:47.962791+0000","last_became_active":"2024-09-19T09:00:46.960664+0000","last_became_peered":"2024-09-19T09:00:46.960664+0000","last_unstale":"2024-09-19T09:00:47.962791+0000","last_undegraded":"2024-09-19T09:00:47.962791+0000","last_fullsized":"2024-09-19T09:00:47.962791+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:18.724348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.753580+0000","last_change":"2024-09-19T09:00:46.951362+0000","last_active":"2024-09-19T09:01:51.753580+0000","last_peered":"2024-09-19T09:01:51.753580+0000","last_clean":"2024-09-19T09:01:51.753580+0000","last_became_active":"2024-09-19T09:00:46.951004+0000","last_became_peered":"2024-09-19T09:00:46.951004+0000","last_unstale":"2024-09-19T09:01:51.753580+0000","last_undegraded":"2024-09-19T09:01:51.753580+0000","last_fullsized":"2024-09-19T09:01:51.753580+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:43.574425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.969012+0000","last_change":"2024-09-19T09:00:46.966752+0000","last_active":"2024-09-19T09:00:47.969012+0000","last_peered":"2024-09-19T09:00:47.969012+0000","last_clean":"2024-09-19T09:00:47.969012+0000","last_became_active":"2024-09-19T09:00:46.966370+0000","last_became_peered":"2024-09-19T09:00:46.966370+0000","last_unstale":"2024-09-19T09:00:47.969012+0000","last_undegraded":"2024-09-19T09:00:47.969012+0000","last_fullsized":"2024-09-19T09:00:47.969012+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:04.035771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.748112+0000","last_change":"2024-09-19T09:00:46.956369+0000","last_active":"2024-09-19T09:01:52.748112+0000","last_peered":"2024-09-19T09:01:52.748112+0000","last_clean":"2024-09-19T09:01:52.748112+0000","last_became_active":"2024-09-19T09:00:46.956075+0000","last_became_peered":"2024-09-19T09:00:46.956075+0000","last_unstale":"2024-09-19T09:01:52.748112+0000","last_undegraded":"2024-09-19T09:01:52.748112+0000","last_fullsized":"2024-09-19T09:01:52.748112+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:23.985017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.750892+0000","last_change":"2024-09-19T09:00:46.956661+0000","last_active":"2024-09-19T09:01:51.750892+0000","last_peered":"2024-09-19T09:01:51.750892+0000","last_clean":"2024-09-19T09:01:51.750892+0000","last_became_active":"2024-09-19T09:00:46.956291+0000","last_became_peered":"2024-09-19T09:00:46.956291+0000","last_unstale":"2024-09-19T09:01:51.750892+0000","last_undegraded":"2024-09-19T09:01:51.750892+0000","last_fullsized":"2024-09-19T09:01:51.750892+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:55.713039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964577+0000","last_change":"2024-09-19T09:00:46.961866+0000","last_active":"2024-09-19T09:00:47.964577+0000","last_peered":"2024-09-19T09:00:47.964577+0000","last_clean":"2024-09-19T09:00:47.964577+0000","last_became_active":"2024-09-19T09:00:46.961469+0000","last_became_peered":"2024-09-19T09:00:46.961469+0000","last_unstale":"2024-09-19T09:00:47.964577+0000","last_undegraded":"2024-09-19T09:00:47.964577+0000","last_fullsized":"2024-09-19T09:00:47.964577+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:47.665419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751305+0000","last_change":"2024-09-19T09:00:46.948765+0000","last_active":"2024-09-19T09:01:52.751305+0000","last_peered":"2024-09-19T09:01:52.751305+0000","last_clean":"2024-09-19T09:01:52.751305+0000","last_became_active":"2024-09-19T09:00:46.948431+0000","last_became_peered":"2024-09-19T09:00:46.948431+0000","last_unstale":"2024-09-19T09:01:52.751305+0000","last_undegraded":"2024-09-19T09:01:52.751305+0000","last_fullsized":"2024-09-19T09:01:52.751305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:31.766464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964637+0000","last_change":"2024-09-19T09:00:46.962433+0000","last_active":"2024-09-19T09:00:47.964637+0000","last_peered":"2024-09-19T09:00:47.964637+0000","last_clean":"2024-09-19T09:00:47.964637+0000","last_became_active":"2024-09-19T09:00:46.962153+0000","last_became_peered":"2024-09-19T09:00:46.962153+0000","last_unstale":"2024-09-19T09:00:47.964637+0000","last_undegraded":"2024-09-19T09:00:47.964637+0000","last_fullsized":"2024-09-19T09:00:47.964637+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:37.973611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.753893+0000","last_change":"2024-09-19T09:00:46.953715+0000","last_active":"2024-09-19T09:01:51.753893+0000","last_peered":"2024-09-19T09:01:51.753893+0000","last_clean":"2024-09-19T09:01:51.753893+0000","last_became_active":"2024-09-19T09:00:46.953415+0000","last_became_peered":"2024-09-19T09:00:46.953415+0000","last_unstale":"2024-09-19T09:01:51.753893+0000","last_undegraded":"2024-09-19T09:01:51.753893+0000","last_fullsized":"2024-09-19T09:01:51.753893+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:29.768382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.969440+0000","last_change":"2024-09-19T09:00:46.964746+0000","last_active":"2024-09-19T09:00:47.969440+0000","last_peered":"2024-09-19T09:00:47.969440+0000","last_clean":"2024-09-19T09:00:47.969440+0000","last_became_active":"2024-09-19T09:00:46.964404+0000","last_became_peered":"2024-09-19T09:00:46.964404+0000","last_unstale":"2024-09-19T09:00:47.969440+0000","last_undegraded":"2024-09-19T09:00:47.969440+0000","last_fullsized":"2024-09-19T09:00:47.969440+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:19.841495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.748134+0000","last_change":"2024-09-19T09:00:46.962875+0000","last_active":"2024-09-19T09:01:52.748134+0000","last_peered":"2024-09-19T09:01:52.748134+0000","last_clean":"2024-09-19T09:01:52.748134+0000","last_became_active":"2024-09-19T09:00:46.962561+0000","last_became_peered":"2024-09-19T09:00:46.962561+0000","last_unstale":"2024-09-19T09:01:52.748134+0000","last_undegraded":"2024-09-19T09:01:52.748134+0000","last_fullsized":"2024-09-19T09:01:52.748134+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:39.675545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.754430+0000","last_change":"2024-09-19T09:00:46.963126+0000","last_active":"2024-09-19T09:01:52.754430+0000","last_peered":"2024-09-19T09:01:52.754430+0000","last_clean":"2024-09-19T09:01:52.754430+0000","last_became_active":"2024-09-19T09:00:46.962620+0000","last_became_peered":"2024-09-19T09:00:46.962620+0000","last_unstale":"2024-09-19T09:01:52.754430+0000","last_undegraded":"2024-09-19T09:01:52.754430+0000","last_fullsized":"2024-09-19T09:01:52.754430+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:59.723280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.753128+0000","last_change":"2024-09-19T09:00:46.953026+0000","last_active":"2024-09-19T09:01:51.753128+0000","last_peered":"2024-09-19T09:01:51.753128+0000","last_clean":"2024-09-19T09:01:51.753128+0000","last_became_active":"2024-09-19T09:00:46.952596+0000","last_became_peered":"2024-09-19T09:00:46.952596+0000","last_unstale":"2024-09-19T09:01:51.753128+0000","last_undegraded":"2024-09-19T09:01:51.753128+0000","last_fullsized":"2024-09-19T09:01:51.753128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:58.524116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.753052+0000","last_change":"2024-09-19T09:00:46.968964+0000","last_active":"2024-09-19T09:01:51.753052+0000","last_peered":"2024-09-19T09:01:51.753052+0000","last_clean":"2024-09-19T09:01:51.753052+0000","last_became_active":"2024-09-19T09:00:46.968505+0000","last_became_peered":"2024-09-19T09:00:46.968505+0000","last_unstale":"2024-09-19T09:01:51.753052+0000","last_undegraded":"2024-09-19T09:01:51.753052+0000","last_fullsized":"2024-09-19T09:01:51.753052+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:17:33.335541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.969060+0000","last_change":"2024-09-19T09:00:46.961520+0000","last_active":"2024-09-19T09:00:47.969060+0000","last_peered":"2024-09-19T09:00:47.969060+0000","last_clean":"2024-09-19T09:00:47.969060+0000","last_became_active":"2024-09-19T09:00:46.961140+0000","last_became_peered":"2024-09-19T09:00:46.961140+0000","last_unstale":"2024-09-19T09:00:47.969060+0000","last_undegraded":"2024-09-19T09:00:47.969060+0000","last_fullsized":"2024-09-19T09:00:47.969060+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:03.120173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.953710+0000","last_change":"2024-09-19T09:00:46.955995+0000","last_active":"2024-09-19T09:00:47.953710+0000","last_peered":"2024-09-19T09:00:47.953710+0000","last_clean":"2024-09-19T09:00:47.953710+0000","last_became_active":"2024-09-19T09:00:46.954678+0000","last_became_peered":"2024-09-19T09:00:46.954678+0000","last_unstale":"2024-09-19T09:00:47.953710+0000","last_undegraded":"2024-09-19T09:00:47.953710+0000","last_fullsized":"2024-09-19T09:00:47.953710+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:10.314575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":83,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752664+0000","last_change":"2024-09-19T09:00:44.946000+0000","last_active":"2024-09-19T09:01:52.752664+0000","last_peered":"2024-09-19T09:01:52.752664+0000","last_clean":"2024-09-19T09:01:52.752664+0000","last_became_active":"2024-09-19T09:00:44.945583+0000","last_became_peered":"2024-09-19T09:00:44.945583+0000","last_unstale":"2024-09-19T09:01:52.752664+0000","last_undegraded":"2024-09-19T09:01:52.752664+0000","last_fullsized":"2024-09-19T09:01:52.752664+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:53:19.109697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.904443+0000","last_change":"2024-09-19T09:01:52.904443+0000","last_active":"2024-09-19T09:01:52.904443+0000","last_peered":"2024-09-19T09:01:52.904443+0000","last_clean":"2024-09-19T09:01:52.904443+0000","last_became_active":"2024-09-19T09:01:52.904077+0000","last_became_peered":"2024-09-19T09:01:52.904077+0000","last_unstale":"2024-09-19T09:01:52.904443+0000","last_undegraded":"2024-09-19T09:01:52.904443+0000","last_fullsized":"2024-09-19T09:01:52.904443+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:50.467090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:57.126925+0000","last_change":"2024-09-19T09:00:44.942435+0000","last_active":"2024-09-19T09:00:57.126925+0000","last_peered":"2024-09-19T09:00:57.126925+0000","last_clean":"2024-09-19T09:00:57.126925+0000","last_became_active":"2024-09-19T09:00:44.942067+0000","last_became_peered":"2024-09-19T09:00:44.942067+0000","last_unstale":"2024-09-19T09:00:57.126925+0000","last_undegraded":"2024-09-19T09:00:57.126925+0000","last_fullsized":"2024-09-19T09:00:57.126925+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:16.476506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.747900+0000","last_change":"2024-09-19T09:00:44.937944+0000","last_active":"2024-09-19T09:01:52.747900+0000","last_peered":"2024-09-19T09:01:52.747900+0000","last_clean":"2024-09-19T09:01:52.747900+0000","last_became_active":"2024-09-19T09:00:44.937623+0000","last_became_peered":"2024-09-19T09:00:44.937623+0000","last_unstale":"2024-09-19T09:01:52.747900+0000","last_undegraded":"2024-09-19T09:01:52.747900+0000","last_fullsized":"2024-09-19T09:01:52.747900+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:49:08.730015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964590+0000","last_change":"2024-09-19T09:00:46.959932+0000","last_active":"2024-09-19T09:00:47.964590+0000","last_peered":"2024-09-19T09:00:47.964590+0000","last_clean":"2024-09-19T09:00:47.964590+0000","last_became_active":"2024-09-19T09:00:46.959637+0000","last_became_peered":"2024-09-19T09:00:46.959637+0000","last_unstale":"2024-09-19T09:00:47.964590+0000","last_undegraded":"2024-09-19T09:00:47.964590+0000","last_fullsized":"2024-09-19T09:00:47.964590+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:55.337669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751513+0000","last_change":"2024-09-19T09:00:44.944882+0000","last_active":"2024-09-19T09:01:52.751513+0000","last_peered":"2024-09-19T09:01:52.751513+0000","last_clean":"2024-09-19T09:01:52.751513+0000","last_became_active":"2024-09-19T09:00:44.944116+0000","last_became_peered":"2024-09-19T09:00:44.944116+0000","last_unstale":"2024-09-19T09:01:52.751513+0000","last_undegraded":"2024-09-19T09:01:52.751513+0000","last_fullsized":"2024-09-19T09:01:52.751513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:54:58.555913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.962700+0000","last_change":"2024-09-19T09:00:46.956859+0000","last_active":"2024-09-19T09:00:47.962700+0000","last_peered":"2024-09-19T09:00:47.962700+0000","last_clean":"2024-09-19T09:00:47.962700+0000","last_became_active":"2024-09-19T09:00:46.956561+0000","last_became_peered":"2024-09-19T09:00:46.956561+0000","last_unstale":"2024-09-19T09:00:47.962700+0000","last_undegraded":"2024-09-19T09:00:47.962700+0000","last_fullsized":"2024-09-19T09:00:47.962700+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:09:04.125591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":126,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:01:50.214439+0000","last_change":"2024-09-19T09:00:44.945959+0000","last_active":"2024-09-19T09:01:50.214439+0000","last_peered":"2024-09-19T09:01:50.214439+0000","last_clean":"2024-09-19T09:01:50.214439+0000","last_became_active":"2024-09-19T09:00:44.945640+0000","last_became_peered":"2024-09-19T09:00:44.945640+0000","last_unstale":"2024-09-19T09:01:50.214439+0000","last_undegraded":"2024-09-19T09:01:50.214439+0000","last_fullsized":"2024-09-19T09:01:50.214439+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:02.523834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.750570+0000","last_change":"2024-09-19T09:00:46.953677+0000","last_active":"2024-09-19T09:01:51.750570+0000","last_peered":"2024-09-19T09:01:51.750570+0000","last_clean":"2024-09-19T09:01:51.750570+0000","last_became_active":"2024-09-19T09:00:46.953357+0000","last_became_peered":"2024-09-19T09:00:46.953357+0000","last_unstale":"2024-09-19T09:01:51.750570+0000","last_undegraded":"2024-09-19T09:01:51.750570+0000","last_fullsized":"2024-09-19T09:01:51.750570+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:48.963003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.139991+0000","last_change":"2024-09-19T09:00:44.945777+0000","last_active":"2024-09-19T09:00:58.139991+0000","last_peered":"2024-09-19T09:00:58.139991+0000","last_clean":"2024-09-19T09:00:58.139991+0000","last_became_active":"2024-09-19T09:00:44.945450+0000","last_became_peered":"2024-09-19T09:00:44.945450+0000","last_unstale":"2024-09-19T09:00:58.139991+0000","last_undegraded":"2024-09-19T09:00:58.139991+0000","last_fullsized":"2024-09-19T09:00:58.139991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:44.656451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.754949+0000","last_change":"2024-09-19T09:00:46.962285+0000","last_active":"2024-09-19T09:01:52.754949+0000","last_peered":"2024-09-19T09:01:52.754949+0000","last_clean":"2024-09-19T09:01:52.754949+0000","last_became_active":"2024-09-19T09:00:46.961982+0000","last_became_peered":"2024-09-19T09:00:46.961982+0000","last_unstale":"2024-09-19T09:01:52.754949+0000","last_undegraded":"2024-09-19T09:01:52.754949+0000","last_fullsized":"2024-09-19T09:01:52.754949+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:41.658501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752480+0000","last_change":"2024-09-19T09:00:45.241425+0000","last_active":"2024-09-19T09:01:52.752480+0000","last_peered":"2024-09-19T09:01:52.752480+0000","last_clean":"2024-09-19T09:01:52.752480+0000","last_became_active":"2024-09-19T09:00:45.241032+0000","last_became_peered":"2024-09-19T09:00:45.241032+0000","last_unstale":"2024-09-19T09:01:52.752480+0000","last_undegraded":"2024-09-19T09:01:52.752480+0000","last_fullsized":"2024-09-19T09:01:52.752480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:00.707855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.750496+0000","last_change":"2024-09-19T09:00:46.962436+0000","last_active":"2024-09-19T09:01:51.750496+0000","last_peered":"2024-09-19T09:01:51.750496+0000","last_clean":"2024-09-19T09:01:51.750496+0000","last_became_active":"2024-09-19T09:00:46.962114+0000","last_became_peered":"2024-09-19T09:00:46.962114+0000","last_unstale":"2024-09-19T09:01:51.750496+0000","last_undegraded":"2024-09-19T09:01:51.750496+0000","last_fullsized":"2024-09-19T09:01:51.750496+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:41.761705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'32","reported_seq":163,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:01:50.214436+0000","last_change":"2024-09-19T09:00:44.945187+0000","last_active":"2024-09-19T09:01:50.214436+0000","last_peered":"2024-09-19T09:01:50.214436+0000","last_clean":"2024-09-19T09:01:50.214436+0000","last_became_active":"2024-09-19T09:00:44.944763+0000","last_became_peered":"2024-09-19T09:00:44.944763+0000","last_unstale":"2024-09-19T09:01:50.214436+0000","last_undegraded":"2024-09-19T09:01:50.214436+0000","last_fullsized":"2024-09-19T09:01:50.214436+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:06.052994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62310,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":113,"num_read_kb":117,"num_write":34,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.954165+0000","last_change":"2024-09-19T09:00:46.960991+0000","last_active":"2024-09-19T09:00:47.954165+0000","last_peered":"2024-09-19T09:00:47.954165+0000","last_clean":"2024-09-19T09:00:47.954165+0000","last_became_active":"2024-09-19T09:00:46.958962+0000","last_became_peered":"2024-09-19T09:00:46.958962+0000","last_unstale":"2024-09-19T09:00:47.954165+0000","last_undegraded":"2024-09-19T09:00:47.954165+0000","last_fullsized":"2024-09-19T09:00:47.954165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:21.795694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.753490+0000","last_change":"2024-09-19T09:00:44.946176+0000","last_active":"2024-09-19T09:01:51.753490+0000","last_peered":"2024-09-19T09:01:51.753490+0000","last_clean":"2024-09-19T09:01:51.753490+0000","last_became_active":"2024-09-19T09:00:44.945869+0000","last_became_peered":"2024-09-19T09:00:44.945869+0000","last_unstale":"2024-09-19T09:01:51.753490+0000","last_undegraded":"2024-09-19T09:01:51.753490+0000","last_fullsized":"2024-09-19T09:01:51.753490+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:16.022184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.747950+0000","last_change":"2024-09-19T09:00:46.956345+0000","last_active":"2024-09-19T09:01:52.747950+0000","last_peered":"2024-09-19T09:01:52.747950+0000","last_clean":"2024-09-19T09:01:52.747950+0000","last_became_active":"2024-09-19T09:00:46.956057+0000","last_became_peered":"2024-09-19T09:00:46.956057+0000","last_unstale":"2024-09-19T09:01:52.747950+0000","last_undegraded":"2024-09-19T09:01:52.747950+0000","last_fullsized":"2024-09-19T09:01:52.747950+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:27.510661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752205+0000","last_change":"2024-09-19T09:00:44.945667+0000","last_active":"2024-09-19T09:01:52.752205+0000","last_peered":"2024-09-19T09:01:52.752205+0000","last_clean":"2024-09-19T09:01:52.752205+0000","last_became_active":"2024-09-19T09:00:44.945308+0000","last_became_peered":"2024-09-19T09:00:44.945308+0000","last_unstale":"2024-09-19T09:01:52.752205+0000","last_undegraded":"2024-09-19T09:01:52.752205+0000","last_fullsized":"2024-09-19T09:01:52.752205+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:00:11.304913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.965242+0000","last_change":"2024-09-19T09:00:46.965088+0000","last_active":"2024-09-19T09:00:47.965242+0000","last_peered":"2024-09-19T09:00:47.965242+0000","last_clean":"2024-09-19T09:00:47.965242+0000","last_became_active":"2024-09-19T09:00:46.964777+0000","last_became_peered":"2024-09-19T09:00:46.964777+0000","last_unstale":"2024-09-19T09:00:47.965242+0000","last_undegraded":"2024-09-19T09:00:47.965242+0000","last_fullsized":"2024-09-19T09:00:47.965242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:32.958984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":83,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.752534+0000","last_change":"2024-09-19T09:00:44.946193+0000","last_active":"2024-09-19T09:01:51.752534+0000","last_peered":"2024-09-19T09:01:51.752534+0000","last_clean":"2024-09-19T09:01:51.752534+0000","last_became_active":"2024-09-19T09:00:44.945168+0000","last_became_peered":"2024-09-19T09:00:44.945168+0000","last_unstale":"2024-09-19T09:01:51.752534+0000","last_undegraded":"2024-09-19T09:01:51.752534+0000","last_fullsized":"2024-09-19T09:01:51.752534+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:21.230689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964246+0000","last_change":"2024-09-19T09:00:46.959425+0000","last_active":"2024-09-19T09:00:47.964246+0000","last_peered":"2024-09-19T09:00:47.964246+0000","last_clean":"2024-09-19T09:00:47.964246+0000","last_became_active":"2024-09-19T09:00:46.958764+0000","last_became_peered":"2024-09-19T09:00:46.958764+0000","last_unstale":"2024-09-19T09:00:47.964246+0000","last_undegraded":"2024-09-19T09:00:47.964246+0000","last_fullsized":"2024-09-19T09:00:47.964246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:08:27.371080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.140076+0000","last_change":"2024-09-19T09:00:45.240084+0000","last_active":"2024-09-19T09:00:58.140076+0000","last_peered":"2024-09-19T09:00:58.140076+0000","last_clean":"2024-09-19T09:00:58.140076+0000","last_became_active":"2024-09-19T09:00:45.239222+0000","last_became_peered":"2024-09-19T09:00:45.239222+0000","last_unstale":"2024-09-19T09:00:58.140076+0000","last_undegraded":"2024-09-19T09:00:58.140076+0000","last_fullsized":"2024-09-19T09:00:58.140076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:00:56.643063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.757684+0000","last_change":"2024-09-19T09:01:52.757684+0000","last_active":"2024-09-19T09:01:52.757684+0000","last_peered":"2024-09-19T09:01:52.757684+0000","last_clean":"2024-09-19T09:01:52.757684+0000","last_became_active":"2024-09-19T09:01:52.756969+0000","last_became_peered":"2024-09-19T09:01:52.756969+0000","last_unstale":"2024-09-19T09:01:52.757684+0000","last_undegraded":"2024-09-19T09:01:52.757684+0000","last_fullsized":"2024-09-19T09:01:52.757684+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:24:47.117996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:57.115299+0000","last_change":"2024-09-19T09:00:44.946138+0000","last_active":"2024-09-19T09:00:57.115299+0000","last_peered":"2024-09-19T09:00:57.115299+0000","last_clean":"2024-09-19T09:00:57.115299+0000","last_became_active":"2024-09-19T09:00:44.945793+0000","last_became_peered":"2024-09-19T09:00:44.945793+0000","last_unstale":"2024-09-19T09:00:57.115299+0000","last_undegraded":"2024-09-19T09:00:57.115299+0000","last_fullsized":"2024-09-19T09:00:57.115299+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:01.341560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.969341+0000","last_change":"2024-09-19T09:00:46.970360+0000","last_active":"2024-09-19T09:00:47.969341+0000","last_peered":"2024-09-19T09:00:47.969341+0000","last_clean":"2024-09-19T09:00:47.969341+0000","last_became_active":"2024-09-19T09:00:46.969971+0000","last_became_peered":"2024-09-19T09:00:46.969971+0000","last_unstale":"2024-09-19T09:00:47.969341+0000","last_undegraded":"2024-09-19T09:00:47.969341+0000","last_fullsized":"2024-09-19T09:00:47.969341+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:48.812158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.748305+0000","last_change":"2024-09-19T09:00:44.941713+0000","last_active":"2024-09-19T09:01:52.748305+0000","last_peered":"2024-09-19T09:01:52.748305+0000","last_clean":"2024-09-19T09:01:52.748305+0000","last_became_active":"2024-09-19T09:00:44.941373+0000","last_became_peered":"2024-09-19T09:00:44.941373+0000","last_unstale":"2024-09-19T09:01:52.748305+0000","last_undegraded":"2024-09-19T09:01:52.748305+0000","last_fullsized":"2024-09-19T09:01:52.748305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:31.157803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.965581+0000","last_change":"2024-09-19T09:00:46.959383+0000","last_active":"2024-09-19T09:00:47.965581+0000","last_peered":"2024-09-19T09:00:47.965581+0000","last_clean":"2024-09-19T09:00:47.965581+0000","last_became_active":"2024-09-19T09:00:46.958860+0000","last_became_peered":"2024-09-19T09:00:46.958860+0000","last_unstale":"2024-09-19T09:00:47.965581+0000","last_undegraded":"2024-09-19T09:00:47.965581+0000","last_fullsized":"2024-09-19T09:00:47.965581+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:32:04.423017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752778+0000","last_change":"2024-09-19T09:00:44.944797+0000","last_active":"2024-09-19T09:01:52.752778+0000","last_peered":"2024-09-19T09:01:52.752778+0000","last_clean":"2024-09-19T09:01:52.752778+0000","last_became_active":"2024-09-19T09:00:44.944259+0000","last_became_peered":"2024-09-19T09:00:44.944259+0000","last_unstale":"2024-09-19T09:01:52.752778+0000","last_undegraded":"2024-09-19T09:01:52.752778+0000","last_fullsized":"2024-09-19T09:01:52.752778+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:58.996217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:49.684985+0000","last_change":"2024-09-19T09:00:46.964998+0000","last_active":"2024-09-19T09:00:49.684985+0000","last_peered":"2024-09-19T09:00:49.684985+0000","last_clean":"2024-09-19T09:00:49.684985+0000","last_became_active":"2024-09-19T09:00:46.964562+0000","last_became_peered":"2024-09-19T09:00:46.964562+0000","last_unstale":"2024-09-19T09:00:49.684985+0000","last_undegraded":"2024-09-19T09:00:49.684985+0000","last_fullsized":"2024-09-19T09:00:49.684985+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:00.089800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":81,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.753839+0000","last_change":"2024-09-19T09:00:45.239770+0000","last_active":"2024-09-19T09:01:51.753839+0000","last_peered":"2024-09-19T09:01:51.753839+0000","last_clean":"2024-09-19T09:01:51.753839+0000","last_became_active":"2024-09-19T09:00:45.239385+0000","last_became_peered":"2024-09-19T09:00:45.239385+0000","last_unstale":"2024-09-19T09:01:51.753839+0000","last_undegraded":"2024-09-19T09:01:51.753839+0000","last_fullsized":"2024-09-19T09:01:51.753839+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:21.109900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.966028+0000","last_change":"2024-09-19T09:00:46.967505+0000","last_active":"2024-09-19T09:00:47.966028+0000","last_peered":"2024-09-19T09:00:47.966028+0000","last_clean":"2024-09-19T09:00:47.966028+0000","last_became_active":"2024-09-19T09:00:46.967195+0000","last_became_peered":"2024-09-19T09:00:46.967195+0000","last_unstale":"2024-09-19T09:00:47.966028+0000","last_undegraded":"2024-09-19T09:00:47.966028+0000","last_fullsized":"2024-09-19T09:00:47.966028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:52:28.917333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":330,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964506+0000","last_change":"2024-09-19T08:59:33.821924+0000","last_active":"2024-09-19T09:00:47.964506+0000","last_peered":"2024-09-19T09:00:47.964506+0000","last_clean":"2024-09-19T09:00:47.964506+0000","last_became_active":"2024-09-19T08:59:33.821404+0000","last_became_peered":"2024-09-19T08:59:33.821404+0000","last_unstale":"2024-09-19T09:00:47.964506+0000","last_undegraded":"2024-09-19T09:00:47.964506+0000","last_fullsized":"2024-09-19T09:00:47.964506+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:56:07.830480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:56:07.830480+0000","last_clean_scrub_stamp":"2024-09-19T08:56:07.830480+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:21.763918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.751940+0000","last_change":"2024-09-19T09:00:44.943714+0000","last_active":"2024-09-19T09:01:51.751940+0000","last_peered":"2024-09-19T09:01:51.751940+0000","last_clean":"2024-09-19T09:01:51.751940+0000","last_became_active":"2024-09-19T09:00:44.943362+0000","last_became_peered":"2024-09-19T09:00:44.943362+0000","last_unstale":"2024-09-19T09:01:51.751940+0000","last_undegraded":"2024-09-19T09:01:51.751940+0000","last_fullsized":"2024-09-19T09:01:51.751940+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:30.015648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964922+0000","last_change":"2024-09-19T09:00:46.956784+0000","last_active":"2024-09-19T09:00:47.964922+0000","last_peered":"2024-09-19T09:00:47.964922+0000","last_clean":"2024-09-19T09:00:47.964922+0000","last_became_active":"2024-09-19T09:00:46.956443+0000","last_became_peered":"2024-09-19T09:00:46.956443+0000","last_unstale":"2024-09-19T09:00:47.964922+0000","last_undegraded":"2024-09-19T09:00:47.964922+0000","last_fullsized":"2024-09-19T09:00:47.964922+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:18.918788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:56.119049+0000","last_change":"2024-09-19T09:00:44.947796+0000","last_active":"2024-09-19T09:00:56.119049+0000","last_peered":"2024-09-19T09:00:56.119049+0000","last_clean":"2024-09-19T09:00:56.119049+0000","last_became_active":"2024-09-19T09:00:44.946949+0000","last_became_peered":"2024-09-19T09:00:44.946949+0000","last_unstale":"2024-09-19T09:00:56.119049+0000","last_undegraded":"2024-09-19T09:00:56.119049+0000","last_fullsized":"2024-09-19T09:00:56.119049+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:42.210186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.747892+0000","last_change":"2024-09-19T09:00:46.963856+0000","last_active":"2024-09-19T09:01:52.747892+0000","last_peered":"2024-09-19T09:01:52.747892+0000","last_clean":"2024-09-19T09:01:52.747892+0000","last_became_active":"2024-09-19T09:00:46.963543+0000","last_became_peered":"2024-09-19T09:00:46.963543+0000","last_unstale":"2024-09-19T09:01:52.747892+0000","last_undegraded":"2024-09-19T09:01:52.747892+0000","last_fullsized":"2024-09-19T09:01:52.747892+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:31.041028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":83,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.748056+0000","last_change":"2024-09-19T09:00:45.239269+0000","last_active":"2024-09-19T09:01:52.748056+0000","last_peered":"2024-09-19T09:01:52.748056+0000","last_clean":"2024-09-19T09:01:52.748056+0000","last_became_active":"2024-09-19T09:00:45.238929+0000","last_became_peered":"2024-09-19T09:00:45.238929+0000","last_unstale":"2024-09-19T09:01:52.748056+0000","last_undegraded":"2024-09-19T09:01:52.748056+0000","last_fullsized":"2024-09-19T09:01:52.748056+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:46.684148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751371+0000","last_change":"2024-09-19T09:00:46.963595+0000","last_active":"2024-09-19T09:01:52.751371+0000","last_peered":"2024-09-19T09:01:52.751371+0000","last_clean":"2024-09-19T09:01:52.751371+0000","last_became_active":"2024-09-19T09:00:46.963243+0000","last_became_peered":"2024-09-19T09:00:46.963243+0000","last_unstale":"2024-09-19T09:01:52.751371+0000","last_undegraded":"2024-09-19T09:01:52.751371+0000","last_fullsized":"2024-09-19T09:01:52.751371+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:01.122996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":75,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:01:48.211356+0000","last_change":"2024-09-19T09:00:44.941550+0000","last_active":"2024-09-19T09:01:48.211356+0000","last_peered":"2024-09-19T09:01:48.211356+0000","last_clean":"2024-09-19T09:01:48.211356+0000","last_became_active":"2024-09-19T09:00:44.941145+0000","last_became_peered":"2024-09-19T09:00:44.941145+0000","last_unstale":"2024-09-19T09:01:48.211356+0000","last_undegraded":"2024-09-19T09:01:48.211356+0000","last_fullsized":"2024-09-19T09:01:48.211356+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:53.893887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964440+0000","last_change":"2024-09-19T09:00:46.963473+0000","last_active":"2024-09-19T09:00:47.964440+0000","last_peered":"2024-09-19T09:00:47.964440+0000","last_clean":"2024-09-19T09:00:47.964440+0000","last_became_active":"2024-09-19T09:00:46.963090+0000","last_became_peered":"2024-09-19T09:00:46.963090+0000","last_unstale":"2024-09-19T09:00:47.964440+0000","last_undegraded":"2024-09-19T09:00:47.964440+0000","last_fullsized":"2024-09-19T09:00:47.964440+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:01.492385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.962784+0000","last_change":"2024-09-19T09:00:45.239566+0000","last_active":"2024-09-19T09:00:47.962784+0000","last_peered":"2024-09-19T09:00:47.962784+0000","last_clean":"2024-09-19T09:00:47.962784+0000","last_became_active":"2024-09-19T09:00:45.239250+0000","last_became_peered":"2024-09-19T09:00:45.239250+0000","last_unstale":"2024-09-19T09:00:47.962784+0000","last_undegraded":"2024-09-19T09:00:47.962784+0000","last_fullsized":"2024-09-19T09:00:47.962784+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:44:21.739311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:50.839180+0000","last_change":"2024-09-19T09:00:46.955344+0000","last_active":"2024-09-19T09:00:50.839180+0000","last_peered":"2024-09-19T09:00:50.839180+0000","last_clean":"2024-09-19T09:00:50.839180+0000","last_became_active":"2024-09-19T09:00:46.954961+0000","last_became_peered":"2024-09-19T09:00:46.954961+0000","last_unstale":"2024-09-19T09:00:50.839180+0000","last_undegraded":"2024-09-19T09:00:50.839180+0000","last_fullsized":"2024-09-19T09:00:50.839180+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:55.743556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.141069+0000","last_change":"2024-09-19T09:00:45.241514+0000","last_active":"2024-09-19T09:00:58.141069+0000","last_peered":"2024-09-19T09:00:58.141069+0000","last_clean":"2024-09-19T09:00:58.141069+0000","last_became_active":"2024-09-19T09:00:45.241097+0000","last_became_peered":"2024-09-19T09:00:45.241097+0000","last_unstale":"2024-09-19T09:00:58.141069+0000","last_undegraded":"2024-09-19T09:00:58.141069+0000","last_fullsized":"2024-09-19T09:00:58.141069+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:35:38.323393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.748177+0000","last_change":"2024-09-19T09:00:46.955194+0000","last_active":"2024-09-19T09:01:52.748177+0000","last_peered":"2024-09-19T09:01:52.748177+0000","last_clean":"2024-09-19T09:01:52.748177+0000","last_became_active":"2024-09-19T09:00:46.954771+0000","last_became_peered":"2024-09-19T09:00:46.954771+0000","last_unstale":"2024-09-19T09:01:52.748177+0000","last_undegraded":"2024-09-19T09:01:52.748177+0000","last_fullsized":"2024-09-19T09:01:52.748177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:18.945035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'14","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:01:11.965733+0000","last_change":"2024-09-19T09:00:44.938554+0000","last_active":"2024-09-19T09:01:11.965733+0000","last_peered":"2024-09-19T09:01:11.965733+0000","last_clean":"2024-09-19T09:01:11.965733+0000","last_became_active":"2024-09-19T09:00:44.938261+0000","last_became_peered":"2024-09-19T09:00:44.938261+0000","last_unstale":"2024-09-19T09:01:11.965733+0000","last_undegraded":"2024-09-19T09:01:11.965733+0000","last_fullsized":"2024-09-19T09:01:11.965733+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:24.293764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964806+0000","last_change":"2024-09-19T09:00:46.960435+0000","last_active":"2024-09-19T09:00:47.964806+0000","last_peered":"2024-09-19T09:00:47.964806+0000","last_clean":"2024-09-19T09:00:47.964806+0000","last_became_active":"2024-09-19T09:00:46.960138+0000","last_became_peered":"2024-09-19T09:00:46.960138+0000","last_unstale":"2024-09-19T09:00:47.964806+0000","last_undegraded":"2024-09-19T09:00:47.964806+0000","last_fullsized":"2024-09-19T09:00:47.964806+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:59:50.853344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.750263+0000","last_change":"2024-09-19T09:00:44.941479+0000","last_active":"2024-09-19T09:01:51.750263+0000","last_peered":"2024-09-19T09:01:51.750263+0000","last_clean":"2024-09-19T09:01:51.750263+0000","last_became_active":"2024-09-19T09:00:44.937197+0000","last_became_peered":"2024-09-19T09:00:44.937197+0000","last_unstale":"2024-09-19T09:01:51.750263+0000","last_undegraded":"2024-09-19T09:01:51.750263+0000","last_fullsized":"2024-09-19T09:01:51.750263+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:53:17.385884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.754752+0000","last_change":"2024-09-19T09:00:46.970319+0000","last_active":"2024-09-19T09:01:52.754752+0000","last_peered":"2024-09-19T09:01:52.754752+0000","last_clean":"2024-09-19T09:01:52.754752+0000","last_became_active":"2024-09-19T09:00:46.969899+0000","last_became_peered":"2024-09-19T09:00:46.969899+0000","last_unstale":"2024-09-19T09:01:52.754752+0000","last_undegraded":"2024-09-19T09:01:52.754752+0000","last_fullsized":"2024-09-19T09:01:52.754752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:26:21.379231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.750046+0000","last_change":"2024-09-19T09:00:45.239525+0000","last_active":"2024-09-19T09:01:51.750046+0000","last_peered":"2024-09-19T09:01:51.750046+0000","last_clean":"2024-09-19T09:01:51.750046+0000","last_became_active":"2024-09-19T09:00:45.239213+0000","last_became_peered":"2024-09-19T09:00:45.239213+0000","last_unstale":"2024-09-19T09:01:51.750046+0000","last_undegraded":"2024-09-19T09:01:51.750046+0000","last_fullsized":"2024-09-19T09:01:51.750046+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:32.525025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.953631+0000","last_change":"2024-09-19T09:00:46.958573+0000","last_active":"2024-09-19T09:00:47.953631+0000","last_peered":"2024-09-19T09:00:47.953631+0000","last_clean":"2024-09-19T09:00:47.953631+0000","last_became_active":"2024-09-19T09:00:46.958274+0000","last_became_peered":"2024-09-19T09:00:46.958274+0000","last_unstale":"2024-09-19T09:00:47.953631+0000","last_undegraded":"2024-09-19T09:00:47.953631+0000","last_fullsized":"2024-09-19T09:00:47.953631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:17.249122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.139345+0000","last_change":"2024-09-19T09:00:44.945194+0000","last_active":"2024-09-19T09:00:58.139345+0000","last_peered":"2024-09-19T09:00:58.139345+0000","last_clean":"2024-09-19T09:00:58.139345+0000","last_became_active":"2024-09-19T09:00:44.944931+0000","last_became_peered":"2024-09-19T09:00:44.944931+0000","last_unstale":"2024-09-19T09:00:58.139345+0000","last_undegraded":"2024-09-19T09:00:58.139345+0000","last_fullsized":"2024-09-19T09:00:58.139345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:11.181150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.969572+0000","last_change":"2024-09-19T09:00:46.966895+0000","last_active":"2024-09-19T09:00:47.969572+0000","last_peered":"2024-09-19T09:00:47.969572+0000","last_clean":"2024-09-19T09:00:47.969572+0000","last_became_active":"2024-09-19T09:00:46.966124+0000","last_became_peered":"2024-09-19T09:00:46.966124+0000","last_unstale":"2024-09-19T09:00:47.969572+0000","last_undegraded":"2024-09-19T09:00:47.969572+0000","last_fullsized":"2024-09-19T09:00:47.969572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:14:04.320863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.753137+0000","last_change":"2024-09-19T09:00:44.944167+0000","last_active":"2024-09-19T09:01:51.753137+0000","last_peered":"2024-09-19T09:01:51.753137+0000","last_clean":"2024-09-19T09:01:51.753137+0000","last_became_active":"2024-09-19T09:00:44.943670+0000","last_became_peered":"2024-09-19T09:00:44.943670+0000","last_unstale":"2024-09-19T09:01:51.753137+0000","last_undegraded":"2024-09-19T09:01:51.753137+0000","last_fullsized":"2024-09-19T09:01:51.753137+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:05.596877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964961+0000","last_change":"2024-09-19T09:00:46.958581+0000","last_active":"2024-09-19T09:00:47.964961+0000","last_peered":"2024-09-19T09:00:47.964961+0000","last_clean":"2024-09-19T09:00:47.964961+0000","last_became_active":"2024-09-19T09:00:46.957782+0000","last_became_peered":"2024-09-19T09:00:46.957782+0000","last_unstale":"2024-09-19T09:00:47.964961+0000","last_undegraded":"2024-09-19T09:00:47.964961+0000","last_fullsized":"2024-09-19T09:00:47.964961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:37:19.488538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:55.110727+0000","last_change":"2024-09-19T09:00:44.945351+0000","last_active":"2024-09-19T09:00:55.110727+0000","last_peered":"2024-09-19T09:00:55.110727+0000","last_clean":"2024-09-19T09:00:55.110727+0000","last_became_active":"2024-09-19T09:00:44.944991+0000","last_became_peered":"2024-09-19T09:00:44.944991+0000","last_unstale":"2024-09-19T09:00:55.110727+0000","last_undegraded":"2024-09-19T09:00:55.110727+0000","last_fullsized":"2024-09-19T09:00:55.110727+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:45:07.581872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.964606+0000","last_change":"2024-09-19T09:00:46.956340+0000","last_active":"2024-09-19T09:00:47.964606+0000","last_peered":"2024-09-19T09:00:47.964606+0000","last_clean":"2024-09-19T09:00:47.964606+0000","last_became_active":"2024-09-19T09:00:46.955956+0000","last_became_peered":"2024-09-19T09:00:46.955956+0000","last_unstale":"2024-09-19T09:00:47.964606+0000","last_undegraded":"2024-09-19T09:00:47.964606+0000","last_fullsized":"2024-09-19T09:00:47.964606+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:34.863428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752335+0000","last_change":"2024-09-19T09:00:46.963657+0000","last_active":"2024-09-19T09:01:52.752335+0000","last_peered":"2024-09-19T09:01:52.752335+0000","last_clean":"2024-09-19T09:01:52.752335+0000","last_became_active":"2024-09-19T09:00:46.963134+0000","last_became_peered":"2024-09-19T09:00:46.963134+0000","last_unstale":"2024-09-19T09:01:52.752335+0000","last_undegraded":"2024-09-19T09:01:52.752335+0000","last_fullsized":"2024-09-19T09:01:52.752335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:01.098570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:58.141281+0000","last_change":"2024-09-19T09:00:44.945382+0000","last_active":"2024-09-19T09:00:58.141281+0000","last_peered":"2024-09-19T09:00:58.141281+0000","last_clean":"2024-09-19T09:00:58.141281+0000","last_became_active":"2024-09-19T09:00:44.945061+0000","last_became_peered":"2024-09-19T09:00:44.945061+0000","last_unstale":"2024-09-19T09:00:58.141281+0000","last_undegraded":"2024-09-19T09:00:58.141281+0000","last_fullsized":"2024-09-19T09:00:58.141281+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:07:12.747709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T09:01:51.750394+0000","last_change":"2024-09-19T09:00:46.956354+0000","last_active":"2024-09-19T09:01:51.750394+0000","last_peered":"2024-09-19T09:01:51.750394+0000","last_clean":"2024-09-19T09:01:51.750394+0000","last_became_active":"2024-09-19T09:00:46.955964+0000","last_became_peered":"2024-09-19T09:00:46.955964+0000","last_unstale":"2024-09-19T09:01:51.750394+0000","last_undegraded":"2024-09-19T09:01:51.750394+0000","last_fullsized":"2024-09-19T09:01:51.750394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:05:33.243479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:59.136540+0000","last_change":"2024-09-19T09:00:44.944983+0000","last_active":"2024-09-19T09:00:59.136540+0000","last_peered":"2024-09-19T09:00:59.136540+0000","last_clean":"2024-09-19T09:00:59.136540+0000","last_became_active":"2024-09-19T09:00:44.944625+0000","last_became_peered":"2024-09-19T09:00:44.944625+0000","last_unstale":"2024-09-19T09:00:59.136540+0000","last_undegraded":"2024-09-19T09:00:59.136540+0000","last_fullsized":"2024-09-19T09:00:59.136540+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:49:50.818863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.754929+0000","last_change":"2024-09-19T09:00:46.966448+0000","last_active":"2024-09-19T09:01:52.754929+0000","last_peered":"2024-09-19T09:01:52.754929+0000","last_clean":"2024-09-19T09:01:52.754929+0000","last_became_active":"2024-09-19T09:00:46.966135+0000","last_became_peered":"2024-09-19T09:00:46.966135+0000","last_unstale":"2024-09-19T09:01:52.754929+0000","last_undegraded":"2024-09-19T09:01:52.754929+0000","last_fullsized":"2024-09-19T09:01:52.754929+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:42.652877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.967059+0000","last_change":"2024-09-19T09:00:45.239405+0000","last_active":"2024-09-19T09:00:47.967059+0000","last_peered":"2024-09-19T09:00:47.967059+0000","last_clean":"2024-09-19T09:00:47.967059+0000","last_became_active":"2024-09-19T09:00:45.239002+0000","last_became_peered":"2024-09-19T09:00:45.239002+0000","last_unstale":"2024-09-19T09:00:47.967059+0000","last_undegraded":"2024-09-19T09:00:47.967059+0000","last_fullsized":"2024-09-19T09:00:47.967059+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:46:03.934975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:00:47.965411+0000","last_change":"2024-09-19T09:00:46.951635+0000","last_active":"2024-09-19T09:00:47.965411+0000","last_peered":"2024-09-19T09:00:47.965411+0000","last_clean":"2024-09-19T09:00:47.965411+0000","last_became_active":"2024-09-19T09:00:46.951325+0000","last_became_peered":"2024-09-19T09:00:46.951325+0000","last_unstale":"2024-09-19T09:00:47.965411+0000","last_undegraded":"2024-09-19T09:00:47.965411+0000","last_fullsized":"2024-09-19T09:00:47.965411+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:17.360950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":85,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T09:01:50.213568+0000","last_change":"2024-09-19T09:00:44.942662+0000","last_active":"2024-09-19T09:01:50.213568+0000","last_peered":"2024-09-19T09:01:50.213568+0000","last_clean":"2024-09-19T09:01:50.213568+0000","last_became_active":"2024-09-19T09:00:44.942397+0000","last_became_peered":"2024-09-19T09:00:44.942397+0000","last_unstale":"2024-09-19T09:01:50.213568+0000","last_undegraded":"2024-09-19T09:01:50.213568+0000","last_fullsized":"2024-09-19T09:01:50.213568+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:46:35.247703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69125,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":569,"num_read_kb":360,"num_write":243,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":207375,"data_compressed":4815,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":212166,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612634,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":860160,"data_stored":410932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841567,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1015808,"data_stored":1064664,"data_compressed":37512,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972389,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":410862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201321,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":411240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430255,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":1003760,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593781,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":892928,"data_stored":427768,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822713,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":970752,"data_stored":1020234,"data_compressed":41690,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986239,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":880640,"data_stored":419534,"data_compressed":2888,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215171,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":410404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":962560,"data_stored":482723,"data_compressed":4493,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":942080,"data_stored":482503,"data_compressed":4493,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":913408,"data_stored":429107,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":64062,"data_compressed":1605,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64282,"data_compressed":1605,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":64439,"data_compressed":1605,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:01:56.192 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T09:01:56.193 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-19T09:01:56.193 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-19T09:01:56.193 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-19T09:01:56.435 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:56.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:56 smithi120 ceph-mon[31150]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:01:56.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:56 smithi120 ceph-mon[31150]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s; 13 B/s, 1 objects/s recovering 2024-09-19T09:01:56.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.935+0000 7f8bc7a74640 1 -- 172.21.15.120:0/2294994214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bc0100ca0 msgr2=0x7f8bc0106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:56.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.935+0000 7f8bc7a74640 1 --2- 172.21.15.120:0/2294994214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bc0100ca0 0x7f8bc0106840 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009930 tx=0x7f8bb002a420 comp rx=0 tx=0).stop 2024-09-19T09:01:56.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.935+0000 7f8bc7a74640 1 -- 172.21.15.120:0/2294994214 shutdown_connections 2024-09-19T09:01:56.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.935+0000 7f8bc7a74640 1 --2- 172.21.15.120:0/2294994214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bc0106f50 0x7f8bc010eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:56.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.935+0000 7f8bc7a74640 1 --2- 172.21.15.120:0/2294994214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bc0100ca0 0x7f8bc0106840 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:56.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.935+0000 7f8bc7a74640 1 --2- 172.21.15.120:0/2294994214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bc01002f0 0x7f8bc01006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:56.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.935+0000 7f8bc7a74640 1 -- 172.21.15.120:0/2294994214 >> 172.21.15.120:0/2294994214 conn(0x7f8bc0077210 msgr2=0x7f8bc00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:56.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.935+0000 7f8bc7a74640 1 -- 172.21.15.120:0/2294994214 shutdown_connections 2024-09-19T09:01:56.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.935+0000 7f8bc7a74640 1 -- 172.21.15.120:0/2294994214 wait complete. 2024-09-19T09:01:56.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.936+0000 7f8bc7a74640 1 Processor -- start 2024-09-19T09:01:56.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.936+0000 7f8bc7a74640 1 -- start start 2024-09-19T09:01:56.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.936+0000 7f8bc7a74640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bc01002f0 0x7f8bc01984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:56.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.936+0000 7f8bc7a74640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bc0100ca0 0x7f8bc0198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:56.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.936+0000 7f8bc7a74640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bc0106f50 0x7f8bc019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:56.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.936+0000 7f8bc7a74640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc0111fa0 con 0x7f8bc0106f50 2024-09-19T09:01:56.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.936+0000 7f8bc7a74640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc0111e20 con 0x7f8bc0100ca0 2024-09-19T09:01:56.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.936+0000 7f8bc7a74640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc0112120 con 0x7f8bc01002f0 2024-09-19T09:01:56.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.936+0000 7f8bc6011640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bc0106f50 0x7f8bc019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:56.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.936+0000 7f8bc6011640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bc0106f50 0x7f8bc019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52844/0 (socket says 172.21.15.120:52844) 2024-09-19T09:01:56.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.936+0000 7f8bc6011640 1 -- 172.21.15.120:0/4282074651 learned_addr learned my addr 172.21.15.120:0/4282074651 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:56.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.936+0000 7f8bc500f640 1 --2- 172.21.15.120:0/4282074651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bc0100ca0 0x7f8bc0198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:56.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.936+0000 7f8bc5810640 1 --2- 172.21.15.120:0/4282074651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bc01002f0 0x7f8bc01984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:56.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.936+0000 7f8bc6011640 1 -- 172.21.15.120:0/4282074651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bc01002f0 msgr2=0x7f8bc01984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:56.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.936+0000 7f8bc6011640 1 --2- 172.21.15.120:0/4282074651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bc01002f0 0x7f8bc01984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:56.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.936+0000 7f8bc6011640 1 -- 172.21.15.120:0/4282074651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bc0100ca0 msgr2=0x7f8bc0198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:56.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.936+0000 7f8bc6011640 1 --2- 172.21.15.120:0/4282074651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bc0100ca0 0x7f8bc0198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:56.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.936+0000 7f8bc6011640 1 -- 172.21.15.120:0/4282074651 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc019d480 con 0x7f8bc0106f50 2024-09-19T09:01:56.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.937+0000 7f8bc6011640 1 --2- 172.21.15.120:0/4282074651 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bc0106f50 0x7f8bc019cda0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8bb800cad0 tx=0x7f8bb800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:56.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.937+0000 7f8bb6ffd640 1 -- 172.21.15.120:0/4282074651 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb800ae20 con 0x7f8bc0106f50 2024-09-19T09:01:56.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.937+0000 7f8bc7a74640 1 -- 172.21.15.120:0/4282074651 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc019d770 con 0x7f8bc0106f50 2024-09-19T09:01:56.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.937+0000 7f8bb6ffd640 1 -- 172.21.15.120:0/4282074651 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bb8005020 con 0x7f8bc0106f50 2024-09-19T09:01:56.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.937+0000 7f8bb6ffd640 1 -- 172.21.15.120:0/4282074651 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb800f6b0 con 0x7f8bc0106f50 2024-09-19T09:01:56.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.937+0000 7f8bc7a74640 1 -- 172.21.15.120:0/4282074651 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc01a5050 con 0x7f8bc0106f50 2024-09-19T09:01:56.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.938+0000 7f8bc7a74640 1 -- 172.21.15.120:0/4282074651 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bc01006d0 con 0x7f8bc0106f50 2024-09-19T09:01:56.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.939+0000 7f8bb6ffd640 1 -- 172.21.15.120:0/4282074651 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f8bb80051d0 con 0x7f8bc0106f50 2024-09-19T09:01:56.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.939+0000 7f8bb6ffd640 1 --2- 172.21.15.120:0/4282074651 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8bac0a32d0 0x7f8bac0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:56.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.940+0000 7f8bc5810640 1 --2- 172.21.15.120:0/4282074651 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8bac0a32d0 0x7f8bac0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:56.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.940+0000 7f8bb6ffd640 1 -- 172.21.15.120:0/4282074651 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f8bb80cfd80 con 0x7f8bc0106f50 2024-09-19T09:01:56.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.940+0000 7f8bc5810640 1 --2- 172.21.15.120:0/4282074651 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8bac0a32d0 0x7f8bac0a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0058c0 tx=0x7f8bbc005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:56.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:56.944+0000 7f8bb6ffd640 1 -- 172.21.15.120:0/4282074651 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8bb8014030 con 0x7f8bc0106f50 2024-09-19T09:01:57.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:57.148+0000 7f8bc7a74640 1 -- 172.21.15.120:0/4282074651 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8bc002d7c0 con 0x7f8bc0106f50 2024-09-19T09:01:57.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:57.149+0000 7f8bb6ffd640 1 -- 172.21.15.120:0/4282074651 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f8bb809c940 con 0x7f8bc0106f50 2024-09-19T09:01:57.149 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:01:57.149 INFO:teuthology.orchestra.run.smithi120.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T09:01:57.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:57.151+0000 7f8bc7a74640 1 -- 172.21.15.120:0/4282074651 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8bac0a32d0 msgr2=0x7f8bac0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:57.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:57.151+0000 7f8bc7a74640 1 --2- 172.21.15.120:0/4282074651 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8bac0a32d0 0x7f8bac0a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0058c0 tx=0x7f8bbc005c60 comp rx=0 tx=0).stop 2024-09-19T09:01:57.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:57.151+0000 7f8bc7a74640 1 -- 172.21.15.120:0/4282074651 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bc0106f50 msgr2=0x7f8bc019cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:57.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:57.151+0000 7f8bc7a74640 1 --2- 172.21.15.120:0/4282074651 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bc0106f50 0x7f8bc019cda0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8bb800cad0 tx=0x7f8bb800cf90 comp rx=0 tx=0).stop 2024-09-19T09:01:57.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:57.151+0000 7f8bc7a74640 1 -- 172.21.15.120:0/4282074651 shutdown_connections 2024-09-19T09:01:57.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:57.151+0000 7f8bc7a74640 1 --2- 172.21.15.120:0/4282074651 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8bac0a32d0 0x7f8bac0a5790 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:57.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:57.151+0000 7f8bc7a74640 1 --2- 172.21.15.120:0/4282074651 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bc0106f50 0x7f8bc019cda0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:57.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:57.151+0000 7f8bc7a74640 1 --2- 172.21.15.120:0/4282074651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bc0100ca0 0x7f8bc0198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:57.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:57.151+0000 7f8bc7a74640 1 --2- 172.21.15.120:0/4282074651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bc01002f0 0x7f8bc01984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:57.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:57.151+0000 7f8bc7a74640 1 -- 172.21.15.120:0/4282074651 >> 172.21.15.120:0/4282074651 conn(0x7f8bc0077210 msgr2=0x7f8bc00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:57.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:57.151+0000 7f8bc7a74640 1 -- 172.21.15.120:0/4282074651 shutdown_connections 2024-09-19T09:01:57.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:57.151+0000 7f8bc7a74640 1 -- 172.21.15.120:0/4282074651 wait complete. 2024-09-19T09:01:57.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:56 smithi196 ceph-mon[35150]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:01:57.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:56 smithi196 ceph-mon[35150]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s; 13 B/s, 1 objects/s recovering 2024-09-19T09:01:57.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:56 smithi153 ceph-mon[35071]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:01:57.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:56 smithi153 ceph-mon[35071]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s; 13 B/s, 1 objects/s recovering 2024-09-19T09:01:57.866 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:57 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/4282074651' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T09:01:57.867 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-19T09:01:57.867 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:01:58.112 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:01:58.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:57 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/4282074651' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T09:01:58.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:57 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/4282074651' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T09:01:58.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.580+0000 7f0ee0be2640 1 -- 172.21.15.120:0/885886826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0edc100e20 msgr2=0x7f0edc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:58.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.580+0000 7f0ee0be2640 1 --2- 172.21.15.120:0/885886826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0edc100e20 0x7f0edc101200 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009900 tx=0x7f0ecc02a480 comp rx=0 tx=0).stop 2024-09-19T09:01:58.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.580+0000 7f0ee0be2640 1 -- 172.21.15.120:0/885886826 shutdown_connections 2024-09-19T09:01:58.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.580+0000 7f0ee0be2640 1 --2- 172.21.15.120:0/885886826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0edc102ef0 0x7f0edc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:58.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.580+0000 7f0ee0be2640 1 --2- 172.21.15.120:0/885886826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0edc102570 0x7f0edc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:58.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.580+0000 7f0ee0be2640 1 --2- 172.21.15.120:0/885886826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0edc100e20 0x7f0edc101200 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:58.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.580+0000 7f0ee0be2640 1 -- 172.21.15.120:0/885886826 >> 172.21.15.120:0/885886826 conn(0x7f0edc0fc7d0 msgr2=0x7f0edc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:58.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.580+0000 7f0ee0be2640 1 -- 172.21.15.120:0/885886826 shutdown_connections 2024-09-19T09:01:58.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.580+0000 7f0ee0be2640 1 -- 172.21.15.120:0/885886826 wait complete. 2024-09-19T09:01:58.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.581+0000 7f0ee0be2640 1 Processor -- start 2024-09-19T09:01:58.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.581+0000 7f0ee0be2640 1 -- start start 2024-09-19T09:01:58.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.581+0000 7f0ee0be2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0edc100e20 0x7f0edc197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:58.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.581+0000 7f0ee0be2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0edc102570 0x7f0edc1975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:58.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.581+0000 7f0ee0be2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0edc102ef0 0x7f0edc197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:58.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.581+0000 7f0ee0be2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc110fd0 con 0x7f0edc102ef0 2024-09-19T09:01:58.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.581+0000 7f0ee0be2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc110e50 con 0x7f0edc102570 2024-09-19T09:01:58.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.581+0000 7f0ee0be2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc111150 con 0x7f0edc100e20 2024-09-19T09:01:58.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.581+0000 7f0edad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0edc102ef0 0x7f0edc197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:58.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.581+0000 7f0edad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0edc102ef0 0x7f0edc197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52862/0 (socket says 172.21.15.120:52862) 2024-09-19T09:01:58.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.581+0000 7f0edad9d640 1 -- 172.21.15.120:0/545429722 learned_addr learned my addr 172.21.15.120:0/545429722 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:01:58.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.581+0000 7f0eda59c640 1 --2- 172.21.15.120:0/545429722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0edc100e20 0x7f0edc197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:58.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.581+0000 7f0ed9d9b640 1 --2- 172.21.15.120:0/545429722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0edc102570 0x7f0edc1975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:58.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.581+0000 7f0edad9d640 1 -- 172.21.15.120:0/545429722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0edc100e20 msgr2=0x7f0edc197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:58.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.581+0000 7f0edad9d640 1 --2- 172.21.15.120:0/545429722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0edc100e20 0x7f0edc197080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:58.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.581+0000 7f0edad9d640 1 -- 172.21.15.120:0/545429722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0edc102570 msgr2=0x7f0edc1975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:58.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.581+0000 7f0edad9d640 1 --2- 172.21.15.120:0/545429722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0edc102570 0x7f0edc1975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:58.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.581+0000 7f0edad9d640 1 -- 172.21.15.120:0/545429722 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0edc1a68f0 con 0x7f0edc102ef0 2024-09-19T09:01:58.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.582+0000 7f0edad9d640 1 --2- 172.21.15.120:0/545429722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0edc102ef0 0x7f0edc197b00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400cad0 tx=0x7f0ed400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:58.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.582+0000 7f0ed37fe640 1 -- 172.21.15.120:0/545429722 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed400ae20 con 0x7f0edc102ef0 2024-09-19T09:01:58.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.582+0000 7f0ee0be2640 1 -- 172.21.15.120:0/545429722 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0edc1a6be0 con 0x7f0edc102ef0 2024-09-19T09:01:58.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.582+0000 7f0ed37fe640 1 -- 172.21.15.120:0/545429722 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ed4005020 con 0x7f0edc102ef0 2024-09-19T09:01:58.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.582+0000 7f0ed37fe640 1 -- 172.21.15.120:0/545429722 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed400f6b0 con 0x7f0edc102ef0 2024-09-19T09:01:58.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.582+0000 7f0ee0be2640 1 -- 172.21.15.120:0/545429722 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0edc1a7120 con 0x7f0edc102ef0 2024-09-19T09:01:58.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.583+0000 7f0ee0be2640 1 -- 172.21.15.120:0/545429722 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0edc101200 con 0x7f0edc102ef0 2024-09-19T09:01:58.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.584+0000 7f0ed37fe640 1 -- 172.21.15.120:0/545429722 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f0ed40051d0 con 0x7f0edc102ef0 2024-09-19T09:01:58.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.584+0000 7f0ed37fe640 1 --2- 172.21.15.120:0/545429722 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0ec80a32d0 0x7f0ec80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:01:58.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.585+0000 7f0eda59c640 1 --2- 172.21.15.120:0/545429722 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0ec80a32d0 0x7f0ec80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:01:58.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.585+0000 7f0ed37fe640 1 -- 172.21.15.120:0/545429722 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f0ed40cfee0 con 0x7f0edc102ef0 2024-09-19T09:01:58.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.585+0000 7f0eda59c640 1 --2- 172.21.15.120:0/545429722 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0ec80a32d0 0x7f0ec80a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc000c00 tx=0x7f0ecc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:01:58.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.589+0000 7f0ed37fe640 1 -- 172.21.15.120:0/545429722 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0ed4014030 con 0x7f0edc102ef0 2024-09-19T09:01:58.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.790+0000 7f0ee0be2640 1 -- 172.21.15.120:0/545429722 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0edc1a74a0 con 0x7f0edc102ef0 2024-09-19T09:01:58.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.791+0000 7f0ed37fe640 1 -- 172.21.15.120:0/545429722 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11988 (secure 0 0 0) 0x7f0ed409ca60 con 0x7f0edc102ef0 2024-09-19T09:01:58.791 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:01:58.791 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":27,"btime":"2024-09-19T09:00:59:134373+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:00:59.134353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:01:58.791 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 27 2024-09-19T09:01:58.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.793+0000 7f0ee0be2640 1 -- 172.21.15.120:0/545429722 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0ec80a32d0 msgr2=0x7f0ec80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:58.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.793+0000 7f0ee0be2640 1 --2- 172.21.15.120:0/545429722 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0ec80a32d0 0x7f0ec80a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc000c00 tx=0x7f0ecc005a20 comp rx=0 tx=0).stop 2024-09-19T09:01:58.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.793+0000 7f0ee0be2640 1 -- 172.21.15.120:0/545429722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0edc102ef0 msgr2=0x7f0edc197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:01:58.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.793+0000 7f0ee0be2640 1 --2- 172.21.15.120:0/545429722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0edc102ef0 0x7f0edc197b00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400cad0 tx=0x7f0ed400cf90 comp rx=0 tx=0).stop 2024-09-19T09:01:58.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.793+0000 7f0ee0be2640 1 -- 172.21.15.120:0/545429722 shutdown_connections 2024-09-19T09:01:58.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.793+0000 7f0ee0be2640 1 --2- 172.21.15.120:0/545429722 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0ec80a32d0 0x7f0ec80a5790 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:58.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.793+0000 7f0ee0be2640 1 --2- 172.21.15.120:0/545429722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0edc102ef0 0x7f0edc197b00 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:58.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.793+0000 7f0ee0be2640 1 --2- 172.21.15.120:0/545429722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0edc102570 0x7f0edc1975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:58.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.793+0000 7f0ee0be2640 1 --2- 172.21.15.120:0/545429722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0edc100e20 0x7f0edc197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:01:58.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.793+0000 7f0ee0be2640 1 -- 172.21.15.120:0/545429722 >> 172.21.15.120:0/545429722 conn(0x7f0edc0fc7d0 msgr2=0x7f0edc106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:01:58.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.794+0000 7f0ee0be2640 1 -- 172.21.15.120:0/545429722 shutdown_connections 2024-09-19T09:01:58.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:01:58.794+0000 7f0ee0be2640 1 -- 172.21.15.120:0/545429722 wait complete. 2024-09-19T09:01:59.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:58 smithi120 ceph-mon[31150]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 170 B/s, 2 objects/s recovering 2024-09-19T09:01:59.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:01:58 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/545429722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:01:59.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:58 smithi196 ceph-mon[35150]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 170 B/s, 2 objects/s recovering 2024-09-19T09:01:59.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:01:58 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/545429722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:01:59.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:58 smithi153 ceph-mon[35071]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 170 B/s, 2 objects/s recovering 2024-09-19T09:01:59.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:01:58 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/545429722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:01:59.428 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:01:59.676 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:00.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.173+0000 7fa1a49a1640 1 -- 172.21.15.120:0/3349813384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1a01068d0 msgr2=0x7fa1a0106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:00.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.173+0000 7fa1a49a1640 1 --2- 172.21.15.120:0/3349813384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1a01068d0 0x7fa1a0106cb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa190009930 tx=0x7fa19002a420 comp rx=0 tx=0).stop 2024-09-19T09:02:00.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.173+0000 7fa1a49a1640 1 -- 172.21.15.120:0/3349813384 shutdown_connections 2024-09-19T09:02:00.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.173+0000 7fa1a49a1640 1 --2- 172.21.15.120:0/3349813384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1a0101270 0x7fa1a010d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:00.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.173+0000 7fa1a49a1640 1 --2- 172.21.15.120:0/3349813384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1a01008d0 0x7fa1a0100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:00.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.173+0000 7fa1a49a1640 1 --2- 172.21.15.120:0/3349813384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1a01068d0 0x7fa1a0106cb0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:00.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.173+0000 7fa1a49a1640 1 -- 172.21.15.120:0/3349813384 >> 172.21.15.120:0/3349813384 conn(0x7fa1a00fc630 msgr2=0x7fa1a00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:00.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.173+0000 7fa1a49a1640 1 -- 172.21.15.120:0/3349813384 shutdown_connections 2024-09-19T09:02:00.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.173+0000 7fa1a49a1640 1 -- 172.21.15.120:0/3349813384 wait complete. 2024-09-19T09:02:00.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.174+0000 7fa1a49a1640 1 Processor -- start 2024-09-19T09:02:00.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.174+0000 7fa1a49a1640 1 -- start start 2024-09-19T09:02:00.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.174+0000 7fa1a49a1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1a01008d0 0x7fa1a0197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:00.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.174+0000 7fa1a49a1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1a0101270 0x7fa1a01976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:00.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.174+0000 7fa1a49a1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1a01068d0 0x7fa1a019ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:00.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.174+0000 7fa1a49a1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a0110d40 con 0x7fa1a01008d0 2024-09-19T09:02:00.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.174+0000 7fa1a49a1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a0110bc0 con 0x7fa1a01068d0 2024-09-19T09:02:00.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.174+0000 7fa1a49a1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a0110ec0 con 0x7fa1a0101270 2024-09-19T09:02:00.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.174+0000 7fa19f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1a01008d0 0x7fa1a0197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:00.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.174+0000 7fa19f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1a01008d0 0x7fa1a0197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52876/0 (socket says 172.21.15.120:52876) 2024-09-19T09:02:00.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.174+0000 7fa19f7fe640 1 -- 172.21.15.120:0/3864061643 learned_addr learned my addr 172.21.15.120:0/3864061643 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:00.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.174+0000 7fa197fff640 1 --2- 172.21.15.120:0/3864061643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1a0101270 0x7fa1a01976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:00.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.174+0000 7fa19f7fe640 1 -- 172.21.15.120:0/3864061643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1a0101270 msgr2=0x7fa1a01976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:00.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.174+0000 7fa19ffff640 1 --2- 172.21.15.120:0/3864061643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1a01068d0 0x7fa1a019ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:00.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.174+0000 7fa19f7fe640 1 --2- 172.21.15.120:0/3864061643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1a0101270 0x7fa1a01976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:00.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.174+0000 7fa19f7fe640 1 -- 172.21.15.120:0/3864061643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1a01068d0 msgr2=0x7fa1a019ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:00.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.174+0000 7fa19f7fe640 1 --2- 172.21.15.120:0/3864061643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1a01068d0 0x7fa1a019ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:00.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.174+0000 7fa19f7fe640 1 -- 172.21.15.120:0/3864061643 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a019c130 con 0x7fa1a01008d0 2024-09-19T09:02:00.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.175+0000 7fa197fff640 1 --2- 172.21.15.120:0/3864061643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1a0101270 0x7fa1a01976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:02:00.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.175+0000 7fa19f7fe640 1 --2- 172.21.15.120:0/3864061643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1a01008d0 0x7fa1a0197180 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa19002a940 tx=0x7fa190003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:00.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.175+0000 7fa19d7fa640 1 -- 172.21.15.120:0/3864061643 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19003f070 con 0x7fa1a01008d0 2024-09-19T09:02:00.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.175+0000 7fa19d7fa640 1 -- 172.21.15.120:0/3864061643 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa19003b460 con 0x7fa1a01008d0 2024-09-19T09:02:00.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.175+0000 7fa1a49a1640 1 -- 172.21.15.120:0/3864061643 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a019c3c0 con 0x7fa1a01008d0 2024-09-19T09:02:00.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.175+0000 7fa19d7fa640 1 -- 172.21.15.120:0/3864061643 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa190035030 con 0x7fa1a01008d0 2024-09-19T09:02:00.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.175+0000 7fa1a49a1640 1 -- 172.21.15.120:0/3864061643 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a01a3d00 con 0x7fa1a01008d0 2024-09-19T09:02:00.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.176+0000 7fa1a49a1640 1 -- 172.21.15.120:0/3864061643 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a0100d30 con 0x7fa1a01008d0 2024-09-19T09:02:00.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.177+0000 7fa19d7fa640 1 -- 172.21.15.120:0/3864061643 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fa190042070 con 0x7fa1a01008d0 2024-09-19T09:02:00.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.177+0000 7fa19d7fa640 1 --2- 172.21.15.120:0/3864061643 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa18c0a3350 0x7fa18c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:00.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.178+0000 7fa197fff640 1 --2- 172.21.15.120:0/3864061643 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa18c0a3350 0x7fa18c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:00.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.178+0000 7fa19d7fa640 1 -- 172.21.15.120:0/3864061643 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa1900f03c0 con 0x7fa1a01008d0 2024-09-19T09:02:00.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.178+0000 7fa197fff640 1 --2- 172.21.15.120:0/3864061643 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa18c0a3350 0x7fa18c0a5810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00feb10 tx=0x7fa188005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:00.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.181+0000 7fa19d7fa640 1 -- 172.21.15.120:0/3864061643 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa1900bcf80 con 0x7fa1a01008d0 2024-09-19T09:02:00.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.384+0000 7fa1a49a1640 1 -- 172.21.15.120:0/3864061643 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1a01a3f30 con 0x7fa1a01008d0 2024-09-19T09:02:00.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.385+0000 7fa19d7fa640 1 -- 172.21.15.120:0/3864061643 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11988 (secure 0 0 0) 0x7fa1900bc880 con 0x7fa1a01008d0 2024-09-19T09:02:00.385 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:02:00.386 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":27,"btime":"2024-09-19T09:00:59:134373+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:00:59.134353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:02:00.386 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 27 2024-09-19T09:02:00.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.387+0000 7fa1a49a1640 1 -- 172.21.15.120:0/3864061643 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa18c0a3350 msgr2=0x7fa18c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:00.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.387+0000 7fa1a49a1640 1 --2- 172.21.15.120:0/3864061643 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa18c0a3350 0x7fa18c0a5810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00feb10 tx=0x7fa188005d90 comp rx=0 tx=0).stop 2024-09-19T09:02:00.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.387+0000 7fa1a49a1640 1 -- 172.21.15.120:0/3864061643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1a01008d0 msgr2=0x7fa1a0197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:00.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.387+0000 7fa1a49a1640 1 --2- 172.21.15.120:0/3864061643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1a01008d0 0x7fa1a0197180 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa19002a940 tx=0x7fa190003c00 comp rx=0 tx=0).stop 2024-09-19T09:02:00.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.388+0000 7fa1a49a1640 1 -- 172.21.15.120:0/3864061643 shutdown_connections 2024-09-19T09:02:00.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.388+0000 7fa1a49a1640 1 --2- 172.21.15.120:0/3864061643 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa18c0a3350 0x7fa18c0a5810 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:00.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.388+0000 7fa1a49a1640 1 --2- 172.21.15.120:0/3864061643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1a01068d0 0x7fa1a019ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:00.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.388+0000 7fa1a49a1640 1 --2- 172.21.15.120:0/3864061643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1a0101270 0x7fa1a01976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:00.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.388+0000 7fa1a49a1640 1 --2- 172.21.15.120:0/3864061643 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1a01008d0 0x7fa1a0197180 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:00.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.388+0000 7fa1a49a1640 1 -- 172.21.15.120:0/3864061643 >> 172.21.15.120:0/3864061643 conn(0x7fa1a00fc630 msgr2=0x7fa1a00fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:00.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.388+0000 7fa1a49a1640 1 -- 172.21.15.120:0/3864061643 shutdown_connections 2024-09-19T09:02:00.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:00.388+0000 7fa1a49a1640 1 -- 172.21.15.120:0/3864061643 wait complete. 2024-09-19T09:02:01.015 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:00 smithi120 ceph-mon[31150]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 170 B/s, 2 objects/s recovering 2024-09-19T09:02:01.016 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:00 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3864061643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:02:01.044 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:02:01.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:00 smithi196 ceph-mon[35150]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 170 B/s, 2 objects/s recovering 2024-09-19T09:02:01.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:00 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3864061643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:02:01.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:00 smithi153 ceph-mon[35071]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 170 B/s, 2 objects/s recovering 2024-09-19T09:02:01.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:00 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3864061643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:02:01.291 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:01.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.786+0000 7f63d4572640 1 -- 172.21.15.120:0/3763791787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63cc103780 msgr2=0x7f63cc107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:01.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.786+0000 7f63d4572640 1 --2- 172.21.15.120:0/3763791787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63cc103780 0x7f63cc107bc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f63bc009930 tx=0x7f63bc02a420 comp rx=0 tx=0).stop 2024-09-19T09:02:01.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.786+0000 7f63d4572640 1 -- 172.21.15.120:0/3763791787 shutdown_connections 2024-09-19T09:02:01.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.786+0000 7f63d4572640 1 --2- 172.21.15.120:0/3763791787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63cc108300 0x7f63cc10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:01.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.786+0000 7f63d4572640 1 --2- 172.21.15.120:0/3763791787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63cc103780 0x7f63cc107bc0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:01.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.786+0000 7f63d4572640 1 --2- 172.21.15.120:0/3763791787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63cc102dd0 0x7f63cc1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:01.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.786+0000 7f63d4572640 1 -- 172.21.15.120:0/3763791787 >> 172.21.15.120:0/3763791787 conn(0x7f63cc0fc7d0 msgr2=0x7f63cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:01.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.787+0000 7f63d4572640 1 -- 172.21.15.120:0/3763791787 shutdown_connections 2024-09-19T09:02:01.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.787+0000 7f63d4572640 1 -- 172.21.15.120:0/3763791787 wait complete. 2024-09-19T09:02:01.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.787+0000 7f63d4572640 1 Processor -- start 2024-09-19T09:02:01.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.787+0000 7f63d4572640 1 -- start start 2024-09-19T09:02:01.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.787+0000 7f63d4572640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63cc102dd0 0x7f63cc19e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:01.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.787+0000 7f63d4572640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63cc103780 0x7f63cc19e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:01.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.787+0000 7f63d4572640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63cc108300 0x7f63cc198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:01.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.787+0000 7f63d4572640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f63cc1121a0 con 0x7f63cc103780 2024-09-19T09:02:01.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.787+0000 7f63d4572640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f63cc112020 con 0x7f63cc102dd0 2024-09-19T09:02:01.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.787+0000 7f63d4572640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f63cc112320 con 0x7f63cc108300 2024-09-19T09:02:01.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.787+0000 7f63d1b0d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63cc103780 0x7f63cc19e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:01.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.787+0000 7f63d1b0d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63cc103780 0x7f63cc19e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57704/0 (socket says 172.21.15.120:57704) 2024-09-19T09:02:01.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.787+0000 7f63d1b0d640 1 -- 172.21.15.120:0/3391942552 learned_addr learned my addr 172.21.15.120:0/3391942552 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:01.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.788+0000 7f63d2b0f640 1 --2- 172.21.15.120:0/3391942552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63cc108300 0x7f63cc198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:01.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.788+0000 7f63d230e640 1 --2- 172.21.15.120:0/3391942552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63cc102dd0 0x7f63cc19e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:01.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.788+0000 7f63d1b0d640 1 -- 172.21.15.120:0/3391942552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63cc108300 msgr2=0x7f63cc198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:01.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.788+0000 7f63d1b0d640 1 --2- 172.21.15.120:0/3391942552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63cc108300 0x7f63cc198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:01.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.788+0000 7f63d1b0d640 1 -- 172.21.15.120:0/3391942552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63cc102dd0 msgr2=0x7f63cc19e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:01.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.788+0000 7f63d1b0d640 1 --2- 172.21.15.120:0/3391942552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63cc102dd0 0x7f63cc19e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:01.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.788+0000 7f63d1b0d640 1 -- 172.21.15.120:0/3391942552 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63cc198d30 con 0x7f63cc103780 2024-09-19T09:02:01.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.788+0000 7f63d1b0d640 1 --2- 172.21.15.120:0/3391942552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63cc103780 0x7f63cc19e980 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f63bc02a940 tx=0x7f63bc003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:01.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.788+0000 7f63c37fe640 1 -- 172.21.15.120:0/3391942552 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc03f070 con 0x7f63cc103780 2024-09-19T09:02:01.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.788+0000 7f63c37fe640 1 -- 172.21.15.120:0/3391942552 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63bc03b460 con 0x7f63cc103780 2024-09-19T09:02:01.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.788+0000 7f63d4572640 1 -- 172.21.15.120:0/3391942552 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63cc198fc0 con 0x7f63cc103780 2024-09-19T09:02:01.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.788+0000 7f63c37fe640 1 -- 172.21.15.120:0/3391942552 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc035030 con 0x7f63cc103780 2024-09-19T09:02:01.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.788+0000 7f63d4572640 1 -- 172.21.15.120:0/3391942552 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63cc1a5310 con 0x7f63cc103780 2024-09-19T09:02:01.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.789+0000 7f63d4572640 1 -- 172.21.15.120:0/3391942552 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63cc103230 con 0x7f63cc103780 2024-09-19T09:02:01.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.790+0000 7f63c37fe640 1 -- 172.21.15.120:0/3391942552 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f63bc042070 con 0x7f63cc103780 2024-09-19T09:02:01.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.791+0000 7f63c37fe640 1 --2- 172.21.15.120:0/3391942552 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f63b80a32d0 0x7f63b80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:01.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.791+0000 7f63d230e640 1 --2- 172.21.15.120:0/3391942552 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f63b80a32d0 0x7f63b80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:01.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.791+0000 7f63c37fe640 1 -- 172.21.15.120:0/3391942552 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f63bc0f05b0 con 0x7f63cc103780 2024-09-19T09:02:01.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.792+0000 7f63d230e640 1 --2- 172.21.15.120:0/3391942552 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f63b80a32d0 0x7f63b80a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f63c8009f20 tx=0x7f63c80094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:01.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.795+0000 7f63c37fe640 1 -- 172.21.15.120:0/3391942552 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f63bc0bd170 con 0x7f63cc103780 2024-09-19T09:02:01.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.964+0000 7f63d4572640 1 -- 172.21.15.120:0/3391942552 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f63cc10a290 con 0x7f63cc103780 2024-09-19T09:02:01.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.965+0000 7f63c37fe640 1 -- 172.21.15.120:0/3391942552 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22844 (secure 0 0 0) 0x7f63bc0bca70 con 0x7f63cc103780 2024-09-19T09:02:01.966 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:02:01.966 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":81,"fsid":"507393d2-7664-11ef-bceb-c7b262605968","created":"2024-09-19T08:53:06.386526+0000","modified":"2024-09-19T09:01:52.744203+0000","last_up_change":"2024-09-19T08:59:55.692659+0000","last_in_change":"2024-09-19T08:59:32.808860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:56:07.830047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:00:43.774339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:00:45.786262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33ec3993-576c-428c-bf75-adcb17cd72a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6803","nonce":4131180019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6805","nonce":4131180019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6809","nonce":4131180019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6807","nonce":4131180019}]},"public_addr":"172.21.15.120:6803/4131180019","cluster_addr":"172.21.15.120:6805/4131180019","heartbeat_back_addr":"172.21.15.120:6809/4131180019","heartbeat_front_addr":"172.21.15.120:6807/4131180019","state":["exists","up"]},{"osd":1,"uuid":"450bbebf-2aac-47c6-8474-bbd37fd0a1a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6801","nonce":2259730294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6803","nonce":2259730294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6807","nonce":2259730294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6805","nonce":2259730294}]},"public_addr":"172.21.15.153:6801/2259730294","cluster_addr":"172.21.15.153:6803/2259730294","heartbeat_back_addr":"172.21.15.153:6807/2259730294","heartbeat_front_addr":"172.21.15.153:6805/2259730294","state":["exists","up"]},{"osd":2,"uuid":"ac8c4b83-8e8e-4803-b266-262f7873bd85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6801","nonce":2942286570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6803","nonce":2942286570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6807","nonce":2942286570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6805","nonce":2942286570}]},"public_addr":"172.21.15.196:6801/2942286570","cluster_addr":"172.21.15.196:6803/2942286570","heartbeat_back_addr":"172.21.15.196:6807/2942286570","heartbeat_front_addr":"172.21.15.196:6805/2942286570","state":["exists","up"]},{"osd":3,"uuid":"c1d9e507-8f02-403e-87da-9b4966fd8fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6811","nonce":713205668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6813","nonce":713205668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6817","nonce":713205668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6815","nonce":713205668}]},"public_addr":"172.21.15.120:6811/713205668","cluster_addr":"172.21.15.120:6813/713205668","heartbeat_back_addr":"172.21.15.120:6817/713205668","heartbeat_front_addr":"172.21.15.120:6815/713205668","state":["exists","up"]},{"osd":4,"uuid":"7f8bf3b9-0ec4-46f6-a406-d26d7971179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6809","nonce":2680865854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6811","nonce":2680865854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6815","nonce":2680865854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6813","nonce":2680865854}]},"public_addr":"172.21.15.153:6809/2680865854","cluster_addr":"172.21.15.153:6811/2680865854","heartbeat_back_addr":"172.21.15.153:6815/2680865854","heartbeat_front_addr":"172.21.15.153:6813/2680865854","state":["exists","up"]},{"osd":5,"uuid":"70b270b2-209d-4ff5-875a-04ee7b992a93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6809","nonce":3763331116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6811","nonce":3763331116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6815","nonce":3763331116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6813","nonce":3763331116}]},"public_addr":"172.21.15.196:6809/3763331116","cluster_addr":"172.21.15.196:6811/3763331116","heartbeat_back_addr":"172.21.15.196:6815/3763331116","heartbeat_front_addr":"172.21.15.196:6813/3763331116","state":["exists","up"]},{"osd":6,"uuid":"8087f69c-dc31-4522-a18d-4e6ab1b7cb27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6819","nonce":4171773765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6821","nonce":4171773765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6825","nonce":4171773765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6823","nonce":4171773765}]},"public_addr":"172.21.15.120:6819/4171773765","cluster_addr":"172.21.15.120:6821/4171773765","heartbeat_back_addr":"172.21.15.120:6825/4171773765","heartbeat_front_addr":"172.21.15.120:6823/4171773765","state":["exists","up"]},{"osd":7,"uuid":"c3b1ead3-150f-43fd-8924-f639203f4cdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6817","nonce":2535148453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6819","nonce":2535148453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6823","nonce":2535148453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6821","nonce":2535148453}]},"public_addr":"172.21.15.153:6817/2535148453","cluster_addr":"172.21.15.153:6819/2535148453","heartbeat_back_addr":"172.21.15.153:6823/2535148453","heartbeat_front_addr":"172.21.15.153:6821/2535148453","state":["exists","up"]},{"osd":8,"uuid":"a3273a2a-968e-4e93-a9cf-3879892bdeab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6817","nonce":186195906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6819","nonce":186195906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6823","nonce":186195906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6821","nonce":186195906}]},"public_addr":"172.21.15.196:6817/186195906","cluster_addr":"172.21.15.196:6819/186195906","heartbeat_back_addr":"172.21.15.196:6823/186195906","heartbeat_front_addr":"172.21.15.196:6821/186195906","state":["exists","up"]},{"osd":9,"uuid":"6d184513-dab8-45f7-bc08-fa2e8b2f10b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6827","nonce":1559320883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6829","nonce":1559320883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6833","nonce":1559320883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6831","nonce":1559320883}]},"public_addr":"172.21.15.120:6827/1559320883","cluster_addr":"172.21.15.120:6829/1559320883","heartbeat_back_addr":"172.21.15.120:6833/1559320883","heartbeat_front_addr":"172.21.15.120:6831/1559320883","state":["exists","up"]},{"osd":10,"uuid":"797db886-62f9-4316-a45b-4d50512e65d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6825","nonce":3216877560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6827","nonce":3216877560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6831","nonce":3216877560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6829","nonce":3216877560}]},"public_addr":"172.21.15.153:6825/3216877560","cluster_addr":"172.21.15.153:6827/3216877560","heartbeat_back_addr":"172.21.15.153:6831/3216877560","heartbeat_front_addr":"172.21.15.153:6829/3216877560","state":["exists","up"]},{"osd":11,"uuid":"17e0afaa-84eb-4b26-9810-bfa3cc7e0f3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6825","nonce":1857200740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6827","nonce":1857200740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6831","nonce":1857200740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6829","nonce":1857200740}]},"public_addr":"172.21.15.196:6825/1857200740","cluster_addr":"172.21.15.196:6827/1857200740","heartbeat_back_addr":"172.21.15.196:6831/1857200740","heartbeat_front_addr":"172.21.15.196:6829/1857200740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:55:15.595645+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:55:40.405487+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:04.938034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:29.718696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:55.636029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:57:21.257083+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:57:46.316552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:58:11.946863+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:58:37.098578+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:03.759023+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:28.659049+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:54.350475+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:0/3010783707":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6801/3562882246":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6800/3562882246":"2024-09-20T08:53:51.528772+0000","172.21.15.120:0/3224512035":"2024-09-20T08:53:51.528772+0000","172.21.15.120:0/3465105835":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/3317449610":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/2800697451":"2024-09-20T08:53:28.144376+0000","172.21.15.120:6801/2938446665":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/109127432":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6800/2938446665":"2024-09-20T08:53:28.144376+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:02:01.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.967+0000 7f63d4572640 1 -- 172.21.15.120:0/3391942552 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f63b80a32d0 msgr2=0x7f63b80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:01.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.967+0000 7f63d4572640 1 --2- 172.21.15.120:0/3391942552 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f63b80a32d0 0x7f63b80a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f63c8009f20 tx=0x7f63c80094f0 comp rx=0 tx=0).stop 2024-09-19T09:02:01.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.967+0000 7f63d4572640 1 -- 172.21.15.120:0/3391942552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63cc103780 msgr2=0x7f63cc19e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:01.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.967+0000 7f63d4572640 1 --2- 172.21.15.120:0/3391942552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63cc103780 0x7f63cc19e980 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f63bc02a940 tx=0x7f63bc003c00 comp rx=0 tx=0).stop 2024-09-19T09:02:01.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.968+0000 7f63d4572640 1 -- 172.21.15.120:0/3391942552 shutdown_connections 2024-09-19T09:02:01.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.968+0000 7f63d4572640 1 --2- 172.21.15.120:0/3391942552 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f63b80a32d0 0x7f63b80a5790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:01.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.968+0000 7f63d4572640 1 --2- 172.21.15.120:0/3391942552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63cc108300 0x7f63cc198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:01.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.968+0000 7f63d4572640 1 --2- 172.21.15.120:0/3391942552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63cc103780 0x7f63cc19e980 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:01.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.968+0000 7f63d4572640 1 --2- 172.21.15.120:0/3391942552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63cc102dd0 0x7f63cc19e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:01.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.968+0000 7f63d4572640 1 -- 172.21.15.120:0/3391942552 >> 172.21.15.120:0/3391942552 conn(0x7f63cc0fc7d0 msgr2=0x7f63cc105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:01.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.968+0000 7f63d4572640 1 -- 172.21.15.120:0/3391942552 shutdown_connections 2024-09-19T09:02:01.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:01.968+0000 7f63d4572640 1 -- 172.21.15.120:0/3391942552 wait complete. 2024-09-19T09:02:02.659 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:02:02.906 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:03.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:02 smithi120 ceph-mon[31150]: pgmap v316: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 9 op/s; 138 B/s, 1 objects/s recovering 2024-09-19T09:02:03.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:02 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3391942552' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:02:03.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:02 smithi196 ceph-mon[35150]: pgmap v316: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 9 op/s; 138 B/s, 1 objects/s recovering 2024-09-19T09:02:03.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:02 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3391942552' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:02:03.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:02 smithi153 ceph-mon[35071]: pgmap v316: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 9 op/s; 138 B/s, 1 objects/s recovering 2024-09-19T09:02:03.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:02 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3391942552' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:02:03.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.397+0000 7f9c4684e640 1 -- 172.21.15.120:0/3161677425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c40106f10 msgr2=0x7f9c4010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:03.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.397+0000 7f9c4684e640 1 --2- 172.21.15.120:0/3161677425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c40106f10 0x7f9c4010ea90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9c38009ca0 tx=0x7f9c3802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:02:03.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.397+0000 7f9c4684e640 1 -- 172.21.15.120:0/3161677425 shutdown_connections 2024-09-19T09:02:03.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.397+0000 7f9c4684e640 1 --2- 172.21.15.120:0/3161677425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c40106f10 0x7f9c4010ea90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:03.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.397+0000 7f9c4684e640 1 --2- 172.21.15.120:0/3161677425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c40102790 0x7f9c401067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:03.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.397+0000 7f9c4684e640 1 --2- 172.21.15.120:0/3161677425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c40101de0 0x7f9c401021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:03.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.397+0000 7f9c4684e640 1 -- 172.21.15.120:0/3161677425 >> 172.21.15.120:0/3161677425 conn(0x7f9c400fd8c0 msgr2=0x7f9c400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:03.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.397+0000 7f9c4684e640 1 -- 172.21.15.120:0/3161677425 shutdown_connections 2024-09-19T09:02:03.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.397+0000 7f9c4684e640 1 -- 172.21.15.120:0/3161677425 wait complete. 2024-09-19T09:02:03.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.397+0000 7f9c4684e640 1 Processor -- start 2024-09-19T09:02:03.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.397+0000 7f9c4684e640 1 -- start start 2024-09-19T09:02:03.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.398+0000 7f9c4684e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c40101de0 0x7f9c40198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:03.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.398+0000 7f9c4684e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c40102790 0x7f9c40198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:03.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.398+0000 7f9c4684e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c40106f10 0x7f9c4019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:03.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.398+0000 7f9c4684e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9c40112060 con 0x7f9c40102790 2024-09-19T09:02:03.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.398+0000 7f9c4684e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9c40111ee0 con 0x7f9c40106f10 2024-09-19T09:02:03.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.398+0000 7f9c4684e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9c401121e0 con 0x7f9c40101de0 2024-09-19T09:02:03.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.398+0000 7f9c37fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c40102790 0x7f9c40198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:03.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.398+0000 7f9c37fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c40102790 0x7f9c40198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57708/0 (socket says 172.21.15.120:57708) 2024-09-19T09:02:03.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.398+0000 7f9c37fff640 1 -- 172.21.15.120:0/1895535192 learned_addr learned my addr 172.21.15.120:0/1895535192 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:03.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.398+0000 7f9c3ffff640 1 --2- 172.21.15.120:0/1895535192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c40101de0 0x7f9c40198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:03.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.398+0000 7f9c44deb640 1 --2- 172.21.15.120:0/1895535192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c40106f10 0x7f9c4019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:03.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.398+0000 7f9c37fff640 1 -- 172.21.15.120:0/1895535192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c40101de0 msgr2=0x7f9c40198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:03.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.398+0000 7f9c37fff640 1 --2- 172.21.15.120:0/1895535192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c40101de0 0x7f9c40198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:03.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.398+0000 7f9c37fff640 1 -- 172.21.15.120:0/1895535192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c40106f10 msgr2=0x7f9c4019ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:03.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.398+0000 7f9c37fff640 1 --2- 172.21.15.120:0/1895535192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c40106f10 0x7f9c4019ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:03.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.398+0000 7f9c37fff640 1 -- 172.21.15.120:0/1895535192 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4019d4f0 con 0x7f9c40102790 2024-09-19T09:02:03.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.398+0000 7f9c3ffff640 1 --2- 172.21.15.120:0/1895535192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c40101de0 0x7f9c40198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:02:03.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.398+0000 7f9c44deb640 1 --2- 172.21.15.120:0/1895535192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c40106f10 0x7f9c4019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:02:03.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.398+0000 7f9c37fff640 1 --2- 172.21.15.120:0/1895535192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c40102790 0x7f9c40198a80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9c2800b4a0 tx=0x7f9c2800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:03.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.398+0000 7f9c3dffb640 1 -- 172.21.15.120:0/1895535192 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c28014070 con 0x7f9c40102790 2024-09-19T09:02:03.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.399+0000 7f9c3dffb640 1 -- 172.21.15.120:0/1895535192 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c28010af0 con 0x7f9c40102790 2024-09-19T09:02:03.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.399+0000 7f9c4684e640 1 -- 172.21.15.120:0/1895535192 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c4019d7e0 con 0x7f9c40102790 2024-09-19T09:02:03.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.399+0000 7f9c3dffb640 1 -- 172.21.15.120:0/1895535192 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c28017d10 con 0x7f9c40102790 2024-09-19T09:02:03.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.399+0000 7f9c4684e640 1 -- 172.21.15.120:0/1895535192 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c401a50c0 con 0x7f9c40102790 2024-09-19T09:02:03.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.400+0000 7f9c4684e640 1 -- 172.21.15.120:0/1895535192 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c40102240 con 0x7f9c40102790 2024-09-19T09:02:03.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.401+0000 7f9c3dffb640 1 -- 172.21.15.120:0/1895535192 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f9c28017070 con 0x7f9c40102790 2024-09-19T09:02:03.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.401+0000 7f9c3dffb640 1 --2- 172.21.15.120:0/1895535192 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9c2c0a32d0 0x7f9c2c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:03.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.402+0000 7f9c3ffff640 1 --2- 172.21.15.120:0/1895535192 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9c2c0a32d0 0x7f9c2c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:03.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.402+0000 7f9c3dffb640 1 -- 172.21.15.120:0/1895535192 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f9c280cee10 con 0x7f9c40102790 2024-09-19T09:02:03.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.402+0000 7f9c3ffff640 1 --2- 172.21.15.120:0/1895535192 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9c2c0a32d0 0x7f9c2c0a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9c30006fd0 tx=0x7f9c30008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:03.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.406+0000 7f9c3dffb640 1 -- 172.21.15.120:0/1895535192 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9c2809bae0 con 0x7f9c40102790 2024-09-19T09:02:03.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.607+0000 7f9c4684e640 1 -- 172.21.15.120:0/1895535192 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c4002d7c0 con 0x7f9c40102790 2024-09-19T09:02:03.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.608+0000 7f9c3dffb640 1 -- 172.21.15.120:0/1895535192 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11988 (secure 0 0 0) 0x7f9c2809b3e0 con 0x7f9c40102790 2024-09-19T09:02:03.608 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:02:03.608 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":27,"btime":"2024-09-19T09:00:59:134373+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:00:59.134353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:02:03.608 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 27 2024-09-19T09:02:03.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.610+0000 7f9c4684e640 1 -- 172.21.15.120:0/1895535192 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9c2c0a32d0 msgr2=0x7f9c2c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:03.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.610+0000 7f9c4684e640 1 --2- 172.21.15.120:0/1895535192 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9c2c0a32d0 0x7f9c2c0a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9c30006fd0 tx=0x7f9c30008040 comp rx=0 tx=0).stop 2024-09-19T09:02:03.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.610+0000 7f9c4684e640 1 -- 172.21.15.120:0/1895535192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c40102790 msgr2=0x7f9c40198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:03.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.610+0000 7f9c4684e640 1 --2- 172.21.15.120:0/1895535192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c40102790 0x7f9c40198a80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9c2800b4a0 tx=0x7f9c2800b960 comp rx=0 tx=0).stop 2024-09-19T09:02:03.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.610+0000 7f9c4684e640 1 -- 172.21.15.120:0/1895535192 shutdown_connections 2024-09-19T09:02:03.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.610+0000 7f9c4684e640 1 --2- 172.21.15.120:0/1895535192 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9c2c0a32d0 0x7f9c2c0a5790 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:03.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.610+0000 7f9c4684e640 1 --2- 172.21.15.120:0/1895535192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c40106f10 0x7f9c4019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:03.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.610+0000 7f9c4684e640 1 --2- 172.21.15.120:0/1895535192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c40102790 0x7f9c40198a80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:03.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.610+0000 7f9c4684e640 1 --2- 172.21.15.120:0/1895535192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c40101de0 0x7f9c40198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:03.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.610+0000 7f9c4684e640 1 -- 172.21.15.120:0/1895535192 >> 172.21.15.120:0/1895535192 conn(0x7f9c400fd8c0 msgr2=0x7f9c400feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:03.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.610+0000 7f9c4684e640 1 -- 172.21.15.120:0/1895535192 shutdown_connections 2024-09-19T09:02:03.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:03.610+0000 7f9c4684e640 1 -- 172.21.15.120:0/1895535192 wait complete. 2024-09-19T09:02:04.066 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:03 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1895535192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:02:04.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:03 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1895535192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:02:04.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:03 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1895535192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:02:04.406 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T09:00:47.659717+0000', 'modified': '2024-09-19T09:00:59.134353+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24461, 'mds_1': 24446, 'mds_2': 24442, 'mds_3': 24430, 'mds_4': 24421}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14601': {'gid': 14601, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.120:6839/1939272010', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6838', 'nonce': 1939272010}, {'type': 'v1', 'addr': '172.21.15.120:6839', 'nonce': 1939272010}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14613': {'gid': 14613, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.120:6841/658447478', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6840', 'nonce': 658447478}, {'type': 'v1', 'addr': '172.21.15.120:6841', 'nonce': 658447478}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24392': {'gid': 24392, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.153:6833/1522304272', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6832', 'nonce': 1522304272}, {'type': 'v1', 'addr': '172.21.15.153:6833', 'nonce': 1522304272}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24400': {'gid': 24400, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6833/1407049222', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 1407049222}, {'type': 'v1', 'addr': '172.21.15.196:6833', 'nonce': 1407049222}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24419': {'gid': 24419, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.153:6835/3103515283', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6834', 'nonce': 3103515283}, {'type': 'v1', 'addr': '172.21.15.153:6835', 'nonce': 3103515283}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.196:6835/2141168828', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 2141168828}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 2141168828}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.196:6837/776889709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6836', 'nonce': 776889709}, {'type': 'v1', 'addr': '172.21.15.196:6837', 'nonce': 776889709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.196:6839/862542635', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6838', 'nonce': 862542635}, {'type': 'v1', 'addr': '172.21.15.196:6839', 'nonce': 862542635}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.153:6837/3048520124', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6836', 'nonce': 3048520124}, {'type': 'v1', 'addr': '172.21.15.153:6837', 'nonce': 3048520124}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.153:6839/3290519661', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6838', 'nonce': 3290519661}, {'type': 'v1', 'addr': '172.21.15.153:6839', 'nonce': 3290519661}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24461, 'qdb_cluster': [24461, 24446, 24442, 24430, 24421]} 2024-09-19T09:02:04.406 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T09:02:04.406 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-19T09:02:04.653 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:05.047 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:04 smithi120 ceph-mon[31150]: pgmap v317: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s; 125 B/s, 1 objects/s recovering 2024-09-19T09:02:05.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.126+0000 7f86d1256640 1 -- 172.21.15.120:0/1917922540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86cc106a80 msgr2=0x7f86cc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:05.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.126+0000 7f86d1256640 1 --2- 172.21.15.120:0/1917922540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86cc106a80 0x7f86cc106e60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f86bc009930 tx=0x7f86bc02a420 comp rx=0 tx=0).stop 2024-09-19T09:02:05.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.126+0000 7f86d1256640 1 -- 172.21.15.120:0/1917922540 shutdown_connections 2024-09-19T09:02:05.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.126+0000 7f86d1256640 1 --2- 172.21.15.120:0/1917922540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86cc101420 0x7f86cc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.126+0000 7f86d1256640 1 --2- 172.21.15.120:0/1917922540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86cc100a80 0x7f86cc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.126+0000 7f86d1256640 1 --2- 172.21.15.120:0/1917922540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86cc106a80 0x7f86cc106e60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.126+0000 7f86d1256640 1 -- 172.21.15.120:0/1917922540 >> 172.21.15.120:0/1917922540 conn(0x7f86cc0fc7a0 msgr2=0x7f86cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:05.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.126+0000 7f86d1256640 1 -- 172.21.15.120:0/1917922540 shutdown_connections 2024-09-19T09:02:05.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.126+0000 7f86d1256640 1 -- 172.21.15.120:0/1917922540 wait complete. 2024-09-19T09:02:05.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.127+0000 7f86d1256640 1 Processor -- start 2024-09-19T09:02:05.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.127+0000 7f86d1256640 1 -- start start 2024-09-19T09:02:05.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.127+0000 7f86d1256640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86cc100a80 0x7f86cc198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:05.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.127+0000 7f86d1256640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86cc101420 0x7f86cc198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:05.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.127+0000 7f86d1256640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86cc106a80 0x7f86cc19ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:05.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.127+0000 7f86d1256640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f86cc110f20 con 0x7f86cc100a80 2024-09-19T09:02:05.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.127+0000 7f86d1256640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f86cc110da0 con 0x7f86cc101420 2024-09-19T09:02:05.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.127+0000 7f86d1256640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f86cc1110a0 con 0x7f86cc106a80 2024-09-19T09:02:05.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.127+0000 7f86cad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86cc100a80 0x7f86cc198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:05.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.127+0000 7f86cad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86cc100a80 0x7f86cc198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57718/0 (socket says 172.21.15.120:57718) 2024-09-19T09:02:05.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.127+0000 7f86cad9d640 1 -- 172.21.15.120:0/2728605313 learned_addr learned my addr 172.21.15.120:0/2728605313 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:05.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.127+0000 7f86c259c640 1 --2- 172.21.15.120:0/2728605313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86cc101420 0x7f86cc198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:05.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.128+0000 7f86cb59e640 1 --2- 172.21.15.120:0/2728605313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86cc106a80 0x7f86cc19ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:05.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.128+0000 7f86cad9d640 1 -- 172.21.15.120:0/2728605313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86cc106a80 msgr2=0x7f86cc19ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:05.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.128+0000 7f86cad9d640 1 --2- 172.21.15.120:0/2728605313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86cc106a80 0x7f86cc19ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.128+0000 7f86cad9d640 1 -- 172.21.15.120:0/2728605313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86cc101420 msgr2=0x7f86cc198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:05.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.128+0000 7f86cad9d640 1 --2- 172.21.15.120:0/2728605313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86cc101420 0x7f86cc198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.128+0000 7f86cad9d640 1 -- 172.21.15.120:0/2728605313 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86cc19d5b0 con 0x7f86cc100a80 2024-09-19T09:02:05.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.128+0000 7f86cad9d640 1 --2- 172.21.15.120:0/2728605313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86cc100a80 0x7f86cc198600 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f86bc009a50 tx=0x7f86bc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:05.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.128+0000 7f86c8d99640 1 -- 172.21.15.120:0/2728605313 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc03f030 con 0x7f86cc100a80 2024-09-19T09:02:05.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.128+0000 7f86d1256640 1 -- 172.21.15.120:0/2728605313 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86cc19d840 con 0x7f86cc100a80 2024-09-19T09:02:05.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.128+0000 7f86c8d99640 1 -- 172.21.15.120:0/2728605313 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86bc003ad0 con 0x7f86cc100a80 2024-09-19T09:02:05.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.128+0000 7f86c8d99640 1 -- 172.21.15.120:0/2728605313 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc035070 con 0x7f86cc100a80 2024-09-19T09:02:05.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.128+0000 7f86d1256640 1 -- 172.21.15.120:0/2728605313 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86cc1a5180 con 0x7f86cc100a80 2024-09-19T09:02:05.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.130+0000 7f86c8d99640 1 -- 172.21.15.120:0/2728605313 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f86bc042030 con 0x7f86cc100a80 2024-09-19T09:02:05.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.131+0000 7f86d362f740 1 Processor -- start 2024-09-19T09:02:05.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.131+0000 7f86d362f740 1 -- start start 2024-09-19T09:02:05.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.131+0000 7f86d362f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a430cf39d0 0x55a430cf3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:05.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.131+0000 7f86d362f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a430cf42f0 0x55a430cfe670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:05.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.131+0000 7f86d362f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a430cfebb0 0x55a430d00fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:05.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.131+0000 7f86d362f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a430c5a090 con 0x55a430cfebb0 2024-09-19T09:02:05.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.131+0000 7f86d362f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a430c01a70 con 0x55a430cf42f0 2024-09-19T09:02:05.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.131+0000 7f86d362f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a430c56050 con 0x55a430cf39d0 2024-09-19T09:02:05.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.131+0000 7f86c8d99640 1 --2- 172.21.15.120:0/2728605313 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f86b80a32d0 0x7f86b80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:05.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.131+0000 7f86c259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a430cfebb0 0x55a430d00fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:05.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.131+0000 7f86c259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a430cfebb0 0x55a430d00fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57724/0 (socket says 172.21.15.120:57724) 2024-09-19T09:02:05.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.131+0000 7f86c259c640 1 -- 172.21.15.120:0/103804316 learned_addr learned my addr 172.21.15.120:0/103804316 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:05.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.131+0000 7f86cb59e640 1 --2- 172.21.15.120:0/103804316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a430cf39d0 0x55a430cf3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:05.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.131+0000 7f86cb59e640 1 --2- 172.21.15.120:0/2728605313 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f86b80a32d0 0x7f86b80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:05.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.131+0000 7f86c8d99640 1 -- 172.21.15.120:0/2728605313 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f86bc0f0280 con 0x7f86cc100a80 2024-09-19T09:02:05.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.131+0000 7f86cad9d640 1 --2- 172.21.15.120:0/103804316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a430cf42f0 0x55a430cfe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:05.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.132+0000 7f86c259c640 1 -- 172.21.15.120:0/103804316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a430cf39d0 msgr2=0x55a430cf3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:05.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.132+0000 7f86c259c640 1 --2- 172.21.15.120:0/103804316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a430cf39d0 0x55a430cf3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.132+0000 7f86c259c640 1 -- 172.21.15.120:0/103804316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a430cf42f0 msgr2=0x55a430cfe670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:05.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.132+0000 7f86c259c640 1 --2- 172.21.15.120:0/103804316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a430cf42f0 0x55a430cfe670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.132+0000 7f86c259c640 1 -- 172.21.15.120:0/103804316 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a430c0e130 con 0x55a430cfebb0 2024-09-19T09:02:05.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.132+0000 7f86cad9d640 1 --2- 172.21.15.120:0/103804316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a430cf42f0 0x55a430cfe670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:02:05.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.132+0000 7f86c259c640 1 --2- 172.21.15.120:0/103804316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a430cfebb0 0x55a430d00fa0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f86b4009a40 tx=0x7f86b4009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbbc06d39bdb951b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:05.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.132+0000 7f86cb59e640 1 --2- 172.21.15.120:0/2728605313 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f86b80a32d0 0x7f86b80a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f86c40070b0 tx=0x7f86c40093d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:05.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.132+0000 7f86c159a640 1 -- 172.21.15.120:0/103804316 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b4017070 con 0x55a430cfebb0 2024-09-19T09:02:05.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.132+0000 7f86c159a640 1 -- 172.21.15.120:0/103804316 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86b4013470 con 0x55a430cfebb0 2024-09-19T09:02:05.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.132+0000 7f86c159a640 1 -- 172.21.15.120:0/103804316 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b4012630 con 0x55a430cfebb0 2024-09-19T09:02:05.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.133+0000 7f86d362f740 1 -- 172.21.15.120:0/103804316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a430cfebb0 msgr2=0x55a430d00fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:05.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.133+0000 7f86d362f740 1 --2- 172.21.15.120:0/103804316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a430cfebb0 0x55a430d00fa0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f86b4009a40 tx=0x7f86b4009f00 comp rx=0 tx=0).stop 2024-09-19T09:02:05.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.133+0000 7f86d362f740 1 -- 172.21.15.120:0/103804316 shutdown_connections 2024-09-19T09:02:05.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.133+0000 7f86d362f740 1 --2- 172.21.15.120:0/103804316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a430cfebb0 0x55a430d00fa0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.133+0000 7f86d362f740 1 --2- 172.21.15.120:0/103804316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a430cf42f0 0x55a430cfe670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.133+0000 7f86d362f740 1 --2- 172.21.15.120:0/103804316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a430cf39d0 0x55a430cf3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.133+0000 7f86d362f740 1 -- 172.21.15.120:0/103804316 >> 172.21.15.120:0/103804316 conn(0x55a430ceff60 msgr2=0x55a430cf0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:05.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.133+0000 7f86d362f740 1 -- 172.21.15.120:0/103804316 shutdown_connections 2024-09-19T09:02:05.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.133+0000 7f86d362f740 1 -- 172.21.15.120:0/103804316 wait complete. 2024-09-19T09:02:05.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.133+0000 7f86d362f740 1 -- start start 2024-09-19T09:02:05.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.133+0000 7f86d362f740 1 Processor -- start 2024-09-19T09:02:05.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.133+0000 7f86d362f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a430cf39d0 0x55a430cfe380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:05.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.133+0000 7f86d362f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a430cf42f0 0x55a430cfb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:05.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.133+0000 7f86d362f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a430cfebb0 0x55a430cfbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:05.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.133+0000 7f86d362f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a430d01780 con 0x55a430cf39d0 2024-09-19T09:02:05.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.133+0000 7f86d362f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a430c14f20 con 0x55a430cfebb0 2024-09-19T09:02:05.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.133+0000 7f86d362f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a430d01900 con 0x55a430cf42f0 2024-09-19T09:02:05.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.133+0000 7f86d362f740 10 client.0 _ll_register_callbacks cb 0x55a430bde030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:02:05.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.133+0000 7f86cad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a430cf39d0 0x55a430cfe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:05.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.134+0000 7f86cad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a430cf39d0 0x55a430cfe380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57734/0 (socket says 172.21.15.120:57734) 2024-09-19T09:02:05.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.134+0000 7f86cad9d640 1 -- 172.21.15.120:0/3990858771 learned_addr learned my addr 172.21.15.120:0/3990858771 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:05.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.134+0000 7f86cb59e640 1 --2- 172.21.15.120:0/3990858771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a430cfebb0 0x55a430cfbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:05.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.134+0000 7f86c259c640 1 --2- 172.21.15.120:0/3990858771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a430cf42f0 0x55a430cfb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:05.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.134+0000 7f86cad9d640 1 -- 172.21.15.120:0/3990858771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a430cf42f0 msgr2=0x55a430cfb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:05.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.134+0000 7f86cad9d640 1 --2- 172.21.15.120:0/3990858771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a430cf42f0 0x55a430cfb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.134+0000 7f86cad9d640 1 -- 172.21.15.120:0/3990858771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a430cfebb0 msgr2=0x55a430cfbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:05.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.134+0000 7f86cad9d640 1 --2- 172.21.15.120:0/3990858771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a430cfebb0 0x55a430cfbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.134+0000 7f86cad9d640 1 -- 172.21.15.120:0/3990858771 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4309cb2b0 con 0x55a430cf39d0 2024-09-19T09:02:05.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.134+0000 7f86c259c640 1 --2- 172.21.15.120:0/3990858771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a430cf42f0 0x55a430cfb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:02:05.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.134+0000 7f86cad9d640 1 --2- 172.21.15.120:0/3990858771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a430cf39d0 0x55a430cfe380 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0b9110 tx=0x7f86bc009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e4bbafbc294caa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:05.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.134+0000 7f86b27fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:02:05.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.134+0000 7f86b27fc640 1 -- 172.21.15.120:0/3990858771 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc049040 con 0x55a430cf39d0 2024-09-19T09:02:05.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.134+0000 7f86b27fc640 1 -- 172.21.15.120:0/3990858771 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f86bc003c80 con 0x55a430cf39d0 2024-09-19T09:02:05.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.134+0000 7f86d362f740 1 -- 172.21.15.120:0/3990858771 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86b4013470 con 0x55a430cf39d0 2024-09-19T09:02:05.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.134+0000 7f86b27fc640 1 -- 172.21.15.120:0/3990858771 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f86bc0bce40 con 0x55a430cf39d0 2024-09-19T09:02:05.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.134+0000 7f86d362f740 10 client.14967 fetch_fsmap learned FSMap version 27 2024-09-19T09:02:05.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.134+0000 7f86d362f740 1 -- 172.21.15.120:0/3990858771 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a430d0cf20 con 0x55a430cf39d0 2024-09-19T09:02:05.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.134+0000 7f86b27fc640 1 -- 172.21.15.120:0/3990858771 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86b4013470 con 0x55a430cf39d0 2024-09-19T09:02:05.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.135+0000 7f86b27fc640 1 -- 172.21.15.120:0/3990858771 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f86bc0c4f20 con 0x55a430cf39d0 2024-09-19T09:02:05.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.135+0000 7f86d362f740 10 client.14967 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:02:05.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.135+0000 7f86d362f740 10 client.14967 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T09:02:05.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.135+0000 7f86d362f740 20 client.14967 populate_metadata read hostname 'smithi120' 2024-09-19T09:02:05.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.135+0000 7f86d362f740 1 --2- 172.21.15.120:0/3990858771 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55a430d0f660 0x55a430d10ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:05.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.135+0000 7f86d362f740 4 client.14967 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:02:05.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.135+0000 7f86d362f740 1 -- 172.21.15.120:0/3990858771 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a430c67600 con 0x55a430d0f660 2024-09-19T09:02:05.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.135+0000 7f86c259c640 1 --2- 172.21.15.120:0/3990858771 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55a430d0f660 0x55a430d10ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:05.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.135+0000 7f86c259c640 1 --2- 172.21.15.120:0/3990858771 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55a430d0f660 0x55a430d10ad0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=35b02296a69dbd5f server_cookie=826ecec767d9ebdb in_seq=0 out_seq=0 2024-09-19T09:02:05.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.135+0000 7f86b27fc640 10 client.14967 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:02:05.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.136+0000 7f86b27fc640 1 -- 172.21.15.120:0/3990858771 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a430c67600 con 0x55a430d0f660 2024-09-19T09:02:05.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.136+0000 7f86b27fc640 10 client.14967 handle_command_reply: tid=0 2024-09-19T09:02:05.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.136+0000 7f86d362f740 1 client.14967 shutdown 2024-09-19T09:02:05.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.136+0000 7f86d362f740 1 -- 172.21.15.120:0/3990858771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a430cf39d0 msgr2=0x55a430cfe380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:05.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.136+0000 7f86d362f740 1 --2- 172.21.15.120:0/3990858771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a430cf39d0 0x55a430cfe380 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0b9110 tx=0x7f86bc009680 comp rx=0 tx=0).stop 2024-09-19T09:02:05.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.136+0000 7f86d362f740 1 -- 172.21.15.120:0/3990858771 shutdown_connections 2024-09-19T09:02:05.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.136+0000 7f86d362f740 1 --2- 172.21.15.120:0/3990858771 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55a430d0f660 0x55a430d10ad0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.136+0000 7f86d362f740 1 --2- 172.21.15.120:0/3990858771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a430cfebb0 0x55a430cfbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.136+0000 7f86d362f740 1 --2- 172.21.15.120:0/3990858771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a430cf42f0 0x55a430cfb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.136+0000 7f86b27fc640 0 client.14967 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:02:05.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.136+0000 7f86d362f740 1 --2- 172.21.15.120:0/3990858771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a430cf39d0 0x55a430cfe380 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.136+0000 7f86d362f740 1 -- 172.21.15.120:0/3990858771 >> 172.21.15.120:0/3990858771 conn(0x55a430ceff60 msgr2=0x55a430cf5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:05.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.136+0000 7f86d362f740 1 -- 172.21.15.120:0/3990858771 shutdown_connections 2024-09-19T09:02:05.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.136+0000 7f86d362f740 1 -- 172.21.15.120:0/3990858771 wait complete. 2024-09-19T09:02:05.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.137+0000 7f86d362f740 20 client.14967 trim_cache size 0 max 16384 2024-09-19T09:02:05.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.154+0000 7f86d362f740 1 Processor -- start 2024-09-19T09:02:05.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.154+0000 7f86d362f740 1 -- start start 2024-09-19T09:02:05.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.154+0000 7f86d362f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a430cf39d0 0x55a430d10400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:05.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.154+0000 7f86d362f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a430cf42f0 0x55a430ceace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:05.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.154+0000 7f86d362f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a430cfebb0 0x55a430d0ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:05.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.154+0000 7f86d362f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a430d0ee10 con 0x55a430cf39d0 2024-09-19T09:02:05.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.154+0000 7f86d362f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a430d0f4e0 con 0x55a430cf42f0 2024-09-19T09:02:05.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.154+0000 7f86d362f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a430d0fbb0 con 0x55a430cfebb0 2024-09-19T09:02:05.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.154+0000 7f86cad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a430cf39d0 0x55a430d10400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:05.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.154+0000 7f86cad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a430cf39d0 0x55a430d10400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57748/0 (socket says 172.21.15.120:57748) 2024-09-19T09:02:05.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.155+0000 7f86cad9d640 1 -- 172.21.15.120:0/1017709765 learned_addr learned my addr 172.21.15.120:0/1017709765 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:05.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.155+0000 7f86cb59e640 1 --2- 172.21.15.120:0/1017709765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a430cfebb0 0x55a430d0ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:05.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.155+0000 7f86cad9d640 1 -- 172.21.15.120:0/1017709765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a430cfebb0 msgr2=0x55a430d0ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:05.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.155+0000 7f86cad9d640 1 --2- 172.21.15.120:0/1017709765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a430cfebb0 0x55a430d0ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.155+0000 7f86c259c640 1 --2- 172.21.15.120:0/1017709765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a430cf42f0 0x55a430ceace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:05.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.155+0000 7f86cad9d640 1 -- 172.21.15.120:0/1017709765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a430cf42f0 msgr2=0x55a430ceace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:05.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.155+0000 7f86cad9d640 1 --2- 172.21.15.120:0/1017709765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a430cf42f0 0x55a430ceace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.155+0000 7f86cad9d640 1 -- 172.21.15.120:0/1017709765 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a430cefd90 con 0x55a430cf39d0 2024-09-19T09:02:05.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.155+0000 7f86cad9d640 1 --2- 172.21.15.120:0/1017709765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a430cf39d0 0x55a430d10400 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f86bc031550 tx=0x7f86bc0bac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc05241f7afc6440 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:05.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.155+0000 7f86c0d99640 1 -- 172.21.15.120:0/1017709765 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc049070 con 0x55a430cf39d0 2024-09-19T09:02:05.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.155+0000 7f86c0d99640 1 -- 172.21.15.120:0/1017709765 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86bc0bc740 con 0x55a430cf39d0 2024-09-19T09:02:05.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.155+0000 7f86c0d99640 1 -- 172.21.15.120:0/1017709765 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc0c4f20 con 0x55a430cf39d0 2024-09-19T09:02:05.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.156+0000 7f86d362f740 1 -- 172.21.15.120:0/1017709765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a430cf39d0 msgr2=0x55a430d10400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:05.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.156+0000 7f86d362f740 1 --2- 172.21.15.120:0/1017709765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a430cf39d0 0x55a430d10400 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f86bc031550 tx=0x7f86bc0bac50 comp rx=0 tx=0).stop 2024-09-19T09:02:05.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.156+0000 7f86d362f740 1 -- 172.21.15.120:0/1017709765 shutdown_connections 2024-09-19T09:02:05.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.156+0000 7f86d362f740 1 --2- 172.21.15.120:0/1017709765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a430cfebb0 0x55a430d0ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.156+0000 7f86d362f740 1 --2- 172.21.15.120:0/1017709765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a430cf42f0 0x55a430ceace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.156+0000 7f86d362f740 1 --2- 172.21.15.120:0/1017709765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a430cf39d0 0x55a430d10400 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.156+0000 7f86d362f740 1 -- 172.21.15.120:0/1017709765 >> 172.21.15.120:0/1017709765 conn(0x55a430ceff60 msgr2=0x55a430d0e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:05.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.156+0000 7f86d362f740 1 -- 172.21.15.120:0/1017709765 shutdown_connections 2024-09-19T09:02:05.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.156+0000 7f86d362f740 1 -- 172.21.15.120:0/1017709765 wait complete. 2024-09-19T09:02:05.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.156+0000 7f86d362f740 1 -- start start 2024-09-19T09:02:05.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.156+0000 7f86d362f740 1 Processor -- start 2024-09-19T09:02:05.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.157+0000 7f86d362f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a430cf39d0 0x55a430cf5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:05.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.157+0000 7f86d362f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a430cf42f0 0x55a430cf6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:05.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.157+0000 7f86d362f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a430cfebb0 0x55a430d17c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:05.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.157+0000 7f86d362f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a430d10950 con 0x55a430cf39d0 2024-09-19T09:02:05.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.157+0000 7f86d362f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a430d10280 con 0x55a430cf42f0 2024-09-19T09:02:05.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.157+0000 7f86d362f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a430d11020 con 0x55a430cfebb0 2024-09-19T09:02:05.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.157+0000 7f86d362f740 10 client.0 _ll_register_callbacks cb 0x7f86bc005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:02:05.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.157+0000 7f86cad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a430cf39d0 0x55a430cf5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:05.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.157+0000 7f86cad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a430cf39d0 0x55a430cf5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57750/0 (socket says 172.21.15.120:57750) 2024-09-19T09:02:05.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.157+0000 7f86cad9d640 1 -- 172.21.15.120:0/2947734637 learned_addr learned my addr 172.21.15.120:0/2947734637 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:05.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.157+0000 7f86c259c640 1 --2- 172.21.15.120:0/2947734637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a430cf42f0 0x55a430cf6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:05.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.157+0000 7f86cb59e640 1 --2- 172.21.15.120:0/2947734637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a430cfebb0 0x55a430d17c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:05.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.157+0000 7f86cad9d640 1 -- 172.21.15.120:0/2947734637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a430cfebb0 msgr2=0x55a430d17c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:05.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.157+0000 7f86cad9d640 1 --2- 172.21.15.120:0/2947734637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a430cfebb0 0x55a430d17c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.157+0000 7f86cad9d640 1 -- 172.21.15.120:0/2947734637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a430cf42f0 msgr2=0x55a430cf6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:05.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.157+0000 7f86cad9d640 1 --2- 172.21.15.120:0/2947734637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a430cf42f0 0x55a430cf6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.157+0000 7f86cad9d640 1 -- 172.21.15.120:0/2947734637 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a430d18270 con 0x55a430cf39d0 2024-09-19T09:02:05.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.157+0000 7f86cad9d640 1 --2- 172.21.15.120:0/2947734637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a430cf39d0 0x55a430cf5be0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0ee1b0 tx=0x7f86bc03b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d85dc9b68cf184 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:05.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.157+0000 7f86b27fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:02:05.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.157+0000 7f86b27fc640 1 -- 172.21.15.120:0/2947734637 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc0a1060 con 0x55a430cf39d0 2024-09-19T09:02:05.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.157+0000 7f86b27fc640 1 -- 172.21.15.120:0/2947734637 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f86bc0bbb90 con 0x55a430cf39d0 2024-09-19T09:02:05.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.157+0000 7f86d362f740 1 -- 172.21.15.120:0/2947734637 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86bc0bc740 con 0x55a430cf39d0 2024-09-19T09:02:05.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.157+0000 7f86b27fc640 1 -- 172.21.15.120:0/2947734637 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f86bc049070 con 0x55a430cf39d0 2024-09-19T09:02:05.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.157+0000 7f86d362f740 10 client.14973 fetch_fsmap learned FSMap version 27 2024-09-19T09:02:05.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.157+0000 7f86d362f740 1 -- 172.21.15.120:0/2947734637 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a430d18270 con 0x55a430cf39d0 2024-09-19T09:02:05.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.158+0000 7f86b27fc640 1 -- 172.21.15.120:0/2947734637 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86bc0bc740 con 0x55a430cf39d0 2024-09-19T09:02:05.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.158+0000 7f86b27fc640 1 -- 172.21.15.120:0/2947734637 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f86bc0873f0 con 0x55a430cf39d0 2024-09-19T09:02:05.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.158+0000 7f86d362f740 10 client.14973 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:02:05.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.158+0000 7f86d362f740 10 client.14973 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T09:02:05.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.158+0000 7f86d362f740 20 client.14973 populate_metadata read hostname 'smithi120' 2024-09-19T09:02:05.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.158+0000 7f86d362f740 1 --2- 172.21.15.120:0/2947734637 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55a430d7dbf0 0x55a430d7c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:05.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.158+0000 7f86d362f740 4 client.14973 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:02:05.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.158+0000 7f86d362f740 1 -- 172.21.15.120:0/2947734637 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "status"}) -- 0x55a430c61b10 con 0x55a430d7dbf0 2024-09-19T09:02:05.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.158+0000 7f86c259c640 1 --2- 172.21.15.120:0/2947734637 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55a430d7dbf0 0x55a430d7c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:05.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.158+0000 7f86c259c640 1 --2- 172.21.15.120:0/2947734637 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55a430d7dbf0 0x55a430d7c900 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9c5e70ee0718dabc server_cookie=d657cb94629142c7 in_seq=0 out_seq=0 2024-09-19T09:02:05.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.158+0000 7f86b27fc640 10 client.14973 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:02:05.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.159+0000 7f86b27fc640 1 -- 172.21.15.120:0/2947734637 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a430c61b10 con 0x55a430d7dbf0 2024-09-19T09:02:05.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.159+0000 7f86b27fc640 10 client.14973 handle_command_reply: tid=0 2024-09-19T09:02:05.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.159+0000 7f86d362f740 1 client.14973 shutdown 2024-09-19T09:02:05.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.159+0000 7f86d362f740 1 -- 172.21.15.120:0/2947734637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a430cf39d0 msgr2=0x55a430cf5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:05.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.159+0000 7f86d362f740 1 --2- 172.21.15.120:0/2947734637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a430cf39d0 0x55a430cf5be0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0ee1b0 tx=0x7f86bc03b720 comp rx=0 tx=0).stop 2024-09-19T09:02:05.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.159+0000 7f86d362f740 1 -- 172.21.15.120:0/2947734637 shutdown_connections 2024-09-19T09:02:05.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.159+0000 7f86d362f740 1 --2- 172.21.15.120:0/2947734637 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55a430d7dbf0 0x55a430d7c900 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.159+0000 7f86b27fc640 0 client.14973 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:02:05.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.159+0000 7f86d362f740 1 --2- 172.21.15.120:0/2947734637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a430cfebb0 0x55a430d17c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.159+0000 7f86d362f740 1 --2- 172.21.15.120:0/2947734637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a430cf42f0 0x55a430cf6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.159+0000 7f86d362f740 1 --2- 172.21.15.120:0/2947734637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a430cf39d0 0x55a430cf5be0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.159+0000 7f86d362f740 1 -- 172.21.15.120:0/2947734637 >> 172.21.15.120:0/2947734637 conn(0x55a430ceff60 msgr2=0x55a430cf9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:05.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.159+0000 7f86d362f740 1 -- 172.21.15.120:0/2947734637 shutdown_connections 2024-09-19T09:02:05.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.159+0000 7f86d362f740 1 -- 172.21.15.120:0/2947734637 wait complete. 2024-09-19T09:02:05.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.159+0000 7f86d362f740 20 client.14973 trim_cache size 0 max 16384 2024-09-19T09:02:05.175 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:02:05.175 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_fsid": "507393d2-7664-11ef-bceb-c7b262605968", 2024-09-19T09:02:05.175 INFO:teuthology.orchestra.run.smithi120.stdout: "whoami": 4, 2024-09-19T09:02:05.176 INFO:teuthology.orchestra.run.smithi120.stdout: "id": 24421, 2024-09-19T09:02:05.176 INFO:teuthology.orchestra.run.smithi120.stdout: "want_state": "up:active", 2024-09-19T09:02:05.176 INFO:teuthology.orchestra.run.smithi120.stdout: "state": "up:active", 2024-09-19T09:02:05.176 INFO:teuthology.orchestra.run.smithi120.stdout: "fs_name": "cephfs", 2024-09-19T09:02:05.176 INFO:teuthology.orchestra.run.smithi120.stdout: "rank_uptime": 67.027567570000002, 2024-09-19T09:02:05.176 INFO:teuthology.orchestra.run.smithi120.stdout: "mdsmap_epoch": 27, 2024-09-19T09:02:05.176 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch": 81, 2024-09-19T09:02:05.176 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch_barrier": 79, 2024-09-19T09:02:05.176 INFO:teuthology.orchestra.run.smithi120.stdout: "uptime": 104.45410712499999 2024-09-19T09:02:05.176 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:02:05.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.160+0000 7f86d1256640 1 -- 172.21.15.120:0/2728605313 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f86b80a32d0 msgr2=0x7f86b80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:05.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.160+0000 7f86d1256640 1 --2- 172.21.15.120:0/2728605313 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f86b80a32d0 0x7f86b80a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f86c40070b0 tx=0x7f86c40093d0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.160+0000 7f86d1256640 1 -- 172.21.15.120:0/2728605313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86cc100a80 msgr2=0x7f86cc198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:05.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.160+0000 7f86d1256640 1 --2- 172.21.15.120:0/2728605313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86cc100a80 0x7f86cc198600 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f86bc009a50 tx=0x7f86bc004270 comp rx=0 tx=0).stop 2024-09-19T09:02:05.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.160+0000 7f86d1256640 1 -- 172.21.15.120:0/2728605313 shutdown_connections 2024-09-19T09:02:05.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.160+0000 7f86d1256640 1 --2- 172.21.15.120:0/2728605313 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f86b80a32d0 0x7f86b80a5790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.160+0000 7f86d1256640 1 --2- 172.21.15.120:0/2728605313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86cc106a80 0x7f86cc19ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.160+0000 7f86d1256640 1 --2- 172.21.15.120:0/2728605313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86cc101420 0x7f86cc198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.160+0000 7f86d1256640 1 --2- 172.21.15.120:0/2728605313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86cc100a80 0x7f86cc198600 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:05.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.160+0000 7f86d1256640 1 -- 172.21.15.120:0/2728605313 >> 172.21.15.120:0/2728605313 conn(0x7f86cc0fc7a0 msgr2=0x7f86cc06ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:05.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.160+0000 7f86d1256640 1 -- 172.21.15.120:0/2728605313 shutdown_connections 2024-09-19T09:02:05.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:05.160+0000 7f86d1256640 1 -- 172.21.15.120:0/2728605313 wait complete. 2024-09-19T09:02:05.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:04 smithi196 ceph-mon[35150]: pgmap v317: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s; 125 B/s, 1 objects/s recovering 2024-09-19T09:02:05.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:04 smithi153 ceph-mon[35071]: pgmap v317: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s; 125 B/s, 1 objects/s recovering 2024-09-19T09:02:06.006 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T09:02:06.250 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:06.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.755+0000 7fd41ed48640 1 -- 172.21.15.120:0/3100700271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd418101de0 msgr2=0x7fd4181021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:06.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.755+0000 7fd41ed48640 1 --2- 172.21.15.120:0/3100700271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd418101de0 0x7fd4181021c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd408009900 tx=0x7fd40802a410 comp rx=0 tx=0).stop 2024-09-19T09:02:06.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.755+0000 7fd41ed48640 1 -- 172.21.15.120:0/3100700271 shutdown_connections 2024-09-19T09:02:06.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.755+0000 7fd41ed48640 1 --2- 172.21.15.120:0/3100700271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd418106f10 0x7fd41810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.755+0000 7fd41ed48640 1 --2- 172.21.15.120:0/3100700271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd418102790 0x7fd4181067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.755+0000 7fd41ed48640 1 --2- 172.21.15.120:0/3100700271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd418101de0 0x7fd4181021c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.755+0000 7fd41ed48640 1 -- 172.21.15.120:0/3100700271 >> 172.21.15.120:0/3100700271 conn(0x7fd4180fd8c0 msgr2=0x7fd4180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:06.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.755+0000 7fd41ed48640 1 -- 172.21.15.120:0/3100700271 shutdown_connections 2024-09-19T09:02:06.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.755+0000 7fd41ed48640 1 -- 172.21.15.120:0/3100700271 wait complete. 2024-09-19T09:02:06.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.756+0000 7fd41ed48640 1 Processor -- start 2024-09-19T09:02:06.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.756+0000 7fd41ed48640 1 -- start start 2024-09-19T09:02:06.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.756+0000 7fd41ed48640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd418101de0 0x7fd4181985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:06.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.756+0000 7fd41ed48640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd418102790 0x7fd418198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:06.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.756+0000 7fd41ed48640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd418106f10 0x7fd41819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:06.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.756+0000 7fd41ed48640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd418112010 con 0x7fd418102790 2024-09-19T09:02:06.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.756+0000 7fd41ed48640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd418111e90 con 0x7fd418106f10 2024-09-19T09:02:06.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.756+0000 7fd41ed48640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd418112190 con 0x7fd418101de0 2024-09-19T09:02:06.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.756+0000 7fd417fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd418102790 0x7fd418198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:06.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.757+0000 7fd417fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd418102790 0x7fd418198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57764/0 (socket says 172.21.15.120:57764) 2024-09-19T09:02:06.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.757+0000 7fd417fff640 1 -- 172.21.15.120:0/3920301692 learned_addr learned my addr 172.21.15.120:0/3920301692 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:06.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.757+0000 7fd41cae4640 1 --2- 172.21.15.120:0/3920301692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd418101de0 0x7fd4181985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:06.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.757+0000 7fd417fff640 1 -- 172.21.15.120:0/3920301692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd418101de0 msgr2=0x7fd4181985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:06.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.757+0000 7fd417fff640 1 --2- 172.21.15.120:0/3920301692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd418101de0 0x7fd4181985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.757+0000 7fd417fff640 1 -- 172.21.15.120:0/3920301692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd418106f10 msgr2=0x7fd41819ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:06.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.757+0000 7fd41d2e5640 1 --2- 172.21.15.120:0/3920301692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd418106f10 0x7fd41819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:06.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.757+0000 7fd417fff640 1 --2- 172.21.15.120:0/3920301692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd418106f10 0x7fd41819ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.757+0000 7fd417fff640 1 -- 172.21.15.120:0/3920301692 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd41819d560 con 0x7fd418102790 2024-09-19T09:02:06.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.757+0000 7fd41d2e5640 1 --2- 172.21.15.120:0/3920301692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd418106f10 0x7fd41819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:02:06.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.757+0000 7fd417fff640 1 --2- 172.21.15.120:0/3920301692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd418102790 0x7fd418198af0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00b4a0 tx=0x7fd40c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:06.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.757+0000 7fd415ffb640 1 -- 172.21.15.120:0/3920301692 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c014070 con 0x7fd418102790 2024-09-19T09:02:06.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.757+0000 7fd415ffb640 1 -- 172.21.15.120:0/3920301692 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd40c010af0 con 0x7fd418102790 2024-09-19T09:02:06.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.757+0000 7fd41ed48640 1 -- 172.21.15.120:0/3920301692 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd41819d850 con 0x7fd418102790 2024-09-19T09:02:06.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.757+0000 7fd415ffb640 1 -- 172.21.15.120:0/3920301692 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c017d10 con 0x7fd418102790 2024-09-19T09:02:06.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.757+0000 7fd41ed48640 1 -- 172.21.15.120:0/3920301692 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4181a5130 con 0x7fd418102790 2024-09-19T09:02:06.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.759+0000 7fd415ffb640 1 -- 172.21.15.120:0/3920301692 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fd40c017070 con 0x7fd418102790 2024-09-19T09:02:06.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.760+0000 7fd415ffb640 1 --2- 172.21.15.120:0/3920301692 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd4000a32d0 0x7fd4000a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:06.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.760+0000 7fd421121740 1 Processor -- start 2024-09-19T09:02:06.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.760+0000 7fd421121740 1 -- start start 2024-09-19T09:02:06.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.760+0000 7fd415ffb640 1 -- 172.21.15.120:0/3920301692 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fd40c0ced60 con 0x7fd418102790 2024-09-19T09:02:06.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.760+0000 7fd41cae4640 1 --2- 172.21.15.120:0/3920301692 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd4000a32d0 0x7fd4000a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:06.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.760+0000 7fd421121740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df083a69d0 0x55df083a6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:06.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.760+0000 7fd421121740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df083a72f0 0x55df083b1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:06.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.760+0000 7fd421121740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df083b1bb0 0x55df083b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:06.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.760+0000 7fd421121740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55df0830d090 con 0x55df083a69d0 2024-09-19T09:02:06.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.760+0000 7fd421121740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55df082b4a70 con 0x55df083b1bb0 2024-09-19T09:02:06.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.760+0000 7fd421121740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55df08309050 con 0x55df083a72f0 2024-09-19T09:02:06.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.760+0000 7fd41cae4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df083a69d0 0x55df083a6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:06.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.760+0000 7fd41cae4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df083a69d0 0x55df083a6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57774/0 (socket says 172.21.15.120:57774) 2024-09-19T09:02:06.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.760+0000 7fd41cae4640 1 -- 172.21.15.120:0/3396516988 learned_addr learned my addr 172.21.15.120:0/3396516988 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:06.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.760+0000 7fd417fff640 1 --2- 172.21.15.120:0/3396516988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df083a72f0 0x55df083b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:06.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.761+0000 7fd41d2e5640 1 --2- 172.21.15.120:0/3396516988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df083b1bb0 0x55df083b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:06.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.761+0000 7fd41cae4640 1 --2- 172.21.15.120:0/3920301692 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd4000a32d0 0x7fd4000a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd408000c00 tx=0x7fd408031690 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:06.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.761+0000 7fd41cae4640 1 -- 172.21.15.120:0/3396516988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df083a72f0 msgr2=0x55df083b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:06.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.761+0000 7fd41cae4640 1 --2- 172.21.15.120:0/3396516988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df083a72f0 0x55df083b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.761+0000 7fd41cae4640 1 -- 172.21.15.120:0/3396516988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df083b1bb0 msgr2=0x55df083b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:06.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.761+0000 7fd41cae4640 1 --2- 172.21.15.120:0/3396516988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df083b1bb0 0x55df083b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.761+0000 7fd41cae4640 1 -- 172.21.15.120:0/3396516988 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df082c1130 con 0x55df083a69d0 2024-09-19T09:02:06.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.761+0000 7fd41cae4640 1 --2- 172.21.15.120:0/3396516988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df083a69d0 0x55df083a6db0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd408009900 tx=0x7fd408045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc93790a79cb8f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:06.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.761+0000 7fd3feffd640 1 -- 172.21.15.120:0/3396516988 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd408035040 con 0x55df083a69d0 2024-09-19T09:02:06.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.761+0000 7fd3feffd640 1 -- 172.21.15.120:0/3396516988 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4080044b0 con 0x55df083a69d0 2024-09-19T09:02:06.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.761+0000 7fd3feffd640 1 -- 172.21.15.120:0/3396516988 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40803caf0 con 0x55df083a69d0 2024-09-19T09:02:06.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.762+0000 7fd421121740 1 -- 172.21.15.120:0/3396516988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df083a69d0 msgr2=0x55df083a6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:06.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.762+0000 7fd421121740 1 --2- 172.21.15.120:0/3396516988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df083a69d0 0x55df083a6db0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd408009900 tx=0x7fd408045000 comp rx=0 tx=0).stop 2024-09-19T09:02:06.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.762+0000 7fd421121740 1 -- 172.21.15.120:0/3396516988 shutdown_connections 2024-09-19T09:02:06.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.762+0000 7fd421121740 1 --2- 172.21.15.120:0/3396516988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df083b1bb0 0x55df083b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.762+0000 7fd421121740 1 --2- 172.21.15.120:0/3396516988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df083a72f0 0x55df083b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.762+0000 7fd421121740 1 --2- 172.21.15.120:0/3396516988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df083a69d0 0x55df083a6db0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.762+0000 7fd421121740 1 -- 172.21.15.120:0/3396516988 >> 172.21.15.120:0/3396516988 conn(0x55df083a2f60 msgr2=0x55df083a3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:06.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.762+0000 7fd421121740 1 -- 172.21.15.120:0/3396516988 shutdown_connections 2024-09-19T09:02:06.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.762+0000 7fd421121740 1 -- 172.21.15.120:0/3396516988 wait complete. 2024-09-19T09:02:06.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.762+0000 7fd421121740 1 -- start start 2024-09-19T09:02:06.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.762+0000 7fd421121740 1 Processor -- start 2024-09-19T09:02:06.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.763+0000 7fd421121740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df083a69d0 0x55df083b1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:06.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.763+0000 7fd421121740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df083a72f0 0x55df083ae760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:06.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.763+0000 7fd421121740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df083b1bb0 0x55df083aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:06.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.763+0000 7fd421121740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55df083b8b90 con 0x55df083a72f0 2024-09-19T09:02:06.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.763+0000 7fd421121740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55df082c7f20 con 0x55df083a69d0 2024-09-19T09:02:06.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.763+0000 7fd421121740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55df083b8d10 con 0x55df083b1bb0 2024-09-19T09:02:06.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.763+0000 7fd421121740 10 client.0 _ll_register_callbacks cb 0x55df08291030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:02:06.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.763+0000 7fd417fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df083a72f0 0x55df083ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:06.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.763+0000 7fd417fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df083a72f0 0x55df083ae760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57788/0 (socket says 172.21.15.120:57788) 2024-09-19T09:02:06.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.763+0000 7fd417fff640 1 -- 172.21.15.120:0/1079305070 learned_addr learned my addr 172.21.15.120:0/1079305070 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:06.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.763+0000 7fd41d2e5640 1 --2- 172.21.15.120:0/1079305070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df083b1bb0 0x55df083aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:06.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.763+0000 7fd41cae4640 1 --2- 172.21.15.120:0/1079305070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df083a69d0 0x55df083b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:06.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.763+0000 7fd417fff640 1 -- 172.21.15.120:0/1079305070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df083b1bb0 msgr2=0x55df083aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:06.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.763+0000 7fd417fff640 1 --2- 172.21.15.120:0/1079305070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df083b1bb0 0x55df083aeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.763+0000 7fd417fff640 1 -- 172.21.15.120:0/1079305070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df083a69d0 msgr2=0x55df083b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:06.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.763+0000 7fd417fff640 1 --2- 172.21.15.120:0/1079305070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df083a69d0 0x55df083b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.763+0000 7fd417fff640 1 -- 172.21.15.120:0/1079305070 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df0807e2b0 con 0x55df083a72f0 2024-09-19T09:02:06.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.763+0000 7fd417fff640 1 --2- 172.21.15.120:0/1079305070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df083a72f0 0x55df083ae760 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd40c097790 tx=0x7fd40c017730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=993b5969a9eb7f5d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:06.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.763+0000 7fd3f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:02:06.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.763+0000 7fd3f7fff640 1 -- 172.21.15.120:0/1079305070 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c01e040 con 0x55df083a72f0 2024-09-19T09:02:06.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.763+0000 7fd3f7fff640 1 -- 172.21.15.120:0/1079305070 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fd40c010ca0 con 0x55df083a72f0 2024-09-19T09:02:06.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.763+0000 7fd421121740 1 -- 172.21.15.120:0/1079305070 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4080044b0 con 0x55df083a72f0 2024-09-19T09:02:06.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.763+0000 7fd3f7fff640 1 -- 172.21.15.120:0/1079305070 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd40c0a3fa0 con 0x55df083a72f0 2024-09-19T09:02:06.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.763+0000 7fd421121740 10 client.14985 fetch_fsmap learned FSMap version 27 2024-09-19T09:02:06.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.763+0000 7fd421121740 1 -- 172.21.15.120:0/1079305070 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55df083c4480 con 0x55df083a72f0 2024-09-19T09:02:06.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.763+0000 7fd3f7fff640 1 -- 172.21.15.120:0/1079305070 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4080044b0 con 0x55df083a72f0 2024-09-19T09:02:06.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.764+0000 7fd3f7fff640 1 -- 172.21.15.120:0/1079305070 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd40c09a780 con 0x55df083a72f0 2024-09-19T09:02:06.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.764+0000 7fd421121740 10 client.14985 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:02:06.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.764+0000 7fd421121740 10 client.14985 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T09:02:06.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.764+0000 7fd421121740 20 client.14985 populate_metadata read hostname 'smithi120' 2024-09-19T09:02:06.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.764+0000 7fd421121740 1 --2- 172.21.15.120:0/1079305070 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55df083c6ab0 0x55df083c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:06.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.764+0000 7fd421121740 4 client.14985 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:02:06.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.764+0000 7fd421121740 1 -- 172.21.15.120:0/1079305070 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df0831a600 con 0x55df083c6ab0 2024-09-19T09:02:06.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.764+0000 7fd41cae4640 1 --2- 172.21.15.120:0/1079305070 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55df083c6ab0 0x55df083c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:06.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.764+0000 7fd3f7fff640 10 client.14985 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:02:06.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.764+0000 7fd41cae4640 1 --2- 172.21.15.120:0/1079305070 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55df083c6ab0 0x55df083c7f20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=51b30da0bdbcadf1 server_cookie=43cc633b2fdd2c32 in_seq=0 out_seq=0 2024-09-19T09:02:06.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.765+0000 7fd3f7fff640 1 -- 172.21.15.120:0/1079305070 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55df0831a600 con 0x55df083c6ab0 2024-09-19T09:02:06.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.765+0000 7fd3f7fff640 10 client.14985 handle_command_reply: tid=0 2024-09-19T09:02:06.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.765+0000 7fd421121740 1 client.14985 shutdown 2024-09-19T09:02:06.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.765+0000 7fd421121740 1 -- 172.21.15.120:0/1079305070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df083a72f0 msgr2=0x55df083ae760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:06.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.765+0000 7fd421121740 1 --2- 172.21.15.120:0/1079305070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df083a72f0 0x55df083ae760 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd40c097790 tx=0x7fd40c017730 comp rx=0 tx=0).stop 2024-09-19T09:02:06.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.765+0000 7fd421121740 1 -- 172.21.15.120:0/1079305070 shutdown_connections 2024-09-19T09:02:06.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.765+0000 7fd421121740 1 --2- 172.21.15.120:0/1079305070 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55df083c6ab0 0x55df083c7f20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.765+0000 7fd421121740 1 --2- 172.21.15.120:0/1079305070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df083b1bb0 0x55df083aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.765+0000 7fd421121740 1 --2- 172.21.15.120:0/1079305070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df083a72f0 0x55df083ae760 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.765+0000 7fd3f7fff640 0 client.14985 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:02:06.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.765+0000 7fd421121740 1 --2- 172.21.15.120:0/1079305070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df083a69d0 0x55df083b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.765+0000 7fd421121740 1 -- 172.21.15.120:0/1079305070 >> 172.21.15.120:0/1079305070 conn(0x55df083a2f60 msgr2=0x55df083a8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:06.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.766+0000 7fd421121740 1 -- 172.21.15.120:0/1079305070 shutdown_connections 2024-09-19T09:02:06.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.766+0000 7fd421121740 1 -- 172.21.15.120:0/1079305070 wait complete. 2024-09-19T09:02:06.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.766+0000 7fd421121740 20 client.14985 trim_cache size 0 max 16384 2024-09-19T09:02:06.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.783+0000 7fd421121740 1 Processor -- start 2024-09-19T09:02:06.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.783+0000 7fd421121740 1 -- start start 2024-09-19T09:02:06.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.784+0000 7fd421121740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df083a69d0 0x55df083ae020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:06.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.784+0000 7fd421121740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df083a72f0 0x55df083ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:06.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.784+0000 7fd421121740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df083b1bb0 0x55df083aec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:06.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.784+0000 7fd421121740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55df083c6260 con 0x55df083a72f0 2024-09-19T09:02:06.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.784+0000 7fd421121740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55df083c6930 con 0x55df083a69d0 2024-09-19T09:02:06.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.784+0000 7fd421121740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55df083c7000 con 0x55df083b1bb0 2024-09-19T09:02:06.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.784+0000 7fd417fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df083a72f0 0x55df083ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:06.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.784+0000 7fd417fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df083a72f0 0x55df083ae720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57796/0 (socket says 172.21.15.120:57796) 2024-09-19T09:02:06.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.784+0000 7fd417fff640 1 -- 172.21.15.120:0/1830567352 learned_addr learned my addr 172.21.15.120:0/1830567352 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:06.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.784+0000 7fd41cae4640 1 --2- 172.21.15.120:0/1830567352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df083a69d0 0x55df083ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:06.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.784+0000 7fd41d2e5640 1 --2- 172.21.15.120:0/1830567352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df083b1bb0 0x55df083aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:06.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.784+0000 7fd417fff640 1 -- 172.21.15.120:0/1830567352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df083b1bb0 msgr2=0x55df083aec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:06.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.784+0000 7fd417fff640 1 --2- 172.21.15.120:0/1830567352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df083b1bb0 0x55df083aec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.784+0000 7fd417fff640 1 -- 172.21.15.120:0/1830567352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df083a69d0 msgr2=0x55df083ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:06.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.784+0000 7fd417fff640 1 --2- 172.21.15.120:0/1830567352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df083a69d0 0x55df083ae020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.784+0000 7fd417fff640 1 -- 172.21.15.120:0/1830567352 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df083a7120 con 0x55df083a72f0 2024-09-19T09:02:06.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.784+0000 7fd41d2e5640 1 --2- 172.21.15.120:0/1830567352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df083b1bb0 0x55df083aec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:02:06.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.784+0000 7fd417fff640 1 --2- 172.21.15.120:0/1830567352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df083a72f0 0x55df083ae720 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0bb1a0 tx=0x7fd40c09f860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a34ccb47810e9e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:06.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.784+0000 7fd3fe7fc640 1 -- 172.21.15.120:0/1830567352 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c01e070 con 0x55df083a72f0 2024-09-19T09:02:06.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.784+0000 7fd3fe7fc640 1 -- 172.21.15.120:0/1830567352 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd40c0b4240 con 0x55df083a72f0 2024-09-19T09:02:06.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.784+0000 7fd3fe7fc640 1 -- 172.21.15.120:0/1830567352 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c09a780 con 0x55df083a72f0 2024-09-19T09:02:06.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.785+0000 7fd421121740 1 -- 172.21.15.120:0/1830567352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df083a72f0 msgr2=0x55df083ae720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:06.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.785+0000 7fd421121740 1 --2- 172.21.15.120:0/1830567352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df083a72f0 0x55df083ae720 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0bb1a0 tx=0x7fd40c09f860 comp rx=0 tx=0).stop 2024-09-19T09:02:06.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.785+0000 7fd421121740 1 -- 172.21.15.120:0/1830567352 shutdown_connections 2024-09-19T09:02:06.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.785+0000 7fd421121740 1 --2- 172.21.15.120:0/1830567352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df083b1bb0 0x55df083aec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.785+0000 7fd421121740 1 --2- 172.21.15.120:0/1830567352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df083a72f0 0x55df083ae720 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.785+0000 7fd421121740 1 --2- 172.21.15.120:0/1830567352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df083a69d0 0x55df083ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.785+0000 7fd421121740 1 -- 172.21.15.120:0/1830567352 >> 172.21.15.120:0/1830567352 conn(0x55df083a2f60 msgr2=0x55df083ad030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:06.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.785+0000 7fd421121740 1 -- 172.21.15.120:0/1830567352 shutdown_connections 2024-09-19T09:02:06.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.785+0000 7fd421121740 1 -- 172.21.15.120:0/1830567352 wait complete. 2024-09-19T09:02:06.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.785+0000 7fd421121740 1 -- start start 2024-09-19T09:02:06.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.786+0000 7fd421121740 1 Processor -- start 2024-09-19T09:02:06.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.786+0000 7fd421121740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df083a69d0 0x55df083cb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:06.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.786+0000 7fd421121740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df083a72f0 0x55df083cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:06.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.786+0000 7fd421121740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df083b1bb0 0x55df08429180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:06.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.786+0000 7fd421121740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55df083c7da0 con 0x55df083a72f0 2024-09-19T09:02:06.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.786+0000 7fd421121740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55df083c76d0 con 0x55df083a69d0 2024-09-19T09:02:06.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.786+0000 7fd421121740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55df083c8470 con 0x55df083b1bb0 2024-09-19T09:02:06.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.786+0000 7fd421121740 10 client.0 _ll_register_callbacks cb 0x7fd40c09acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:02:06.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.786+0000 7fd417fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df083a72f0 0x55df083cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:06.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.786+0000 7fd417fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df083a72f0 0x55df083cbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57812/0 (socket says 172.21.15.120:57812) 2024-09-19T09:02:06.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.786+0000 7fd417fff640 1 -- 172.21.15.120:0/3260806920 learned_addr learned my addr 172.21.15.120:0/3260806920 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:06.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.786+0000 7fd41cae4640 1 --2- 172.21.15.120:0/3260806920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df083a69d0 0x55df083cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:06.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.786+0000 7fd417fff640 1 -- 172.21.15.120:0/3260806920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df083b1bb0 msgr2=0x55df08429180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:06.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.786+0000 7fd417fff640 1 --2- 172.21.15.120:0/3260806920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df083b1bb0 0x55df08429180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.786+0000 7fd417fff640 1 -- 172.21.15.120:0/3260806920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df083a69d0 msgr2=0x55df083cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:06.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.786+0000 7fd417fff640 1 --2- 172.21.15.120:0/3260806920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df083a69d0 0x55df083cb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.786+0000 7fd417fff640 1 -- 172.21.15.120:0/3260806920 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df08429820 con 0x55df083a72f0 2024-09-19T09:02:06.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.786+0000 7fd417fff640 1 --2- 172.21.15.120:0/3260806920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df083a72f0 0x55df083cbe40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd40c09b030 tx=0x7fd40c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=654baf2d82869bcd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:06.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.786+0000 7fd3f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:02:06.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.787+0000 7fd3f7fff640 1 -- 172.21.15.120:0/3260806920 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c0bb1d0 con 0x55df083a72f0 2024-09-19T09:02:06.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.787+0000 7fd3f7fff640 1 -- 172.21.15.120:0/3260806920 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fd40c0bb370 con 0x55df083a72f0 2024-09-19T09:02:06.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.787+0000 7fd421121740 1 -- 172.21.15.120:0/3260806920 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd40c0b4240 con 0x55df083a72f0 2024-09-19T09:02:06.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.787+0000 7fd3f7fff640 1 -- 172.21.15.120:0/3260806920 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd40c01e070 con 0x55df083a72f0 2024-09-19T09:02:06.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.787+0000 7fd421121740 10 client.14991 fetch_fsmap learned FSMap version 27 2024-09-19T09:02:06.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.787+0000 7fd421121740 1 -- 172.21.15.120:0/3260806920 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55df08430a70 con 0x55df083a72f0 2024-09-19T09:02:06.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.787+0000 7fd3f7fff640 1 -- 172.21.15.120:0/3260806920 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd40c0b4240 con 0x55df083a72f0 2024-09-19T09:02:06.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.787+0000 7fd3f7fff640 1 -- 172.21.15.120:0/3260806920 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd40c09a1f0 con 0x55df083a72f0 2024-09-19T09:02:06.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.787+0000 7fd421121740 10 client.14991 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:02:06.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.787+0000 7fd421121740 10 client.14991 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T09:02:06.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.787+0000 7fd421121740 20 client.14991 populate_metadata read hostname 'smithi120' 2024-09-19T09:02:06.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.787+0000 7fd421121740 1 --2- 172.21.15.120:0/3260806920 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55df083c6ab0 0x55df08433f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:06.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.787+0000 7fd421121740 4 client.14991 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:02:06.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.787+0000 7fd421121740 1 -- 172.21.15.120:0/3260806920 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "status"}) -- 0x55df08314b10 con 0x55df083c6ab0 2024-09-19T09:02:06.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.788+0000 7fd41cae4640 1 --2- 172.21.15.120:0/3260806920 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55df083c6ab0 0x55df08433f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:06.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.788+0000 7fd3f7fff640 10 client.14991 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:02:06.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.788+0000 7fd41cae4640 1 --2- 172.21.15.120:0/3260806920 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55df083c6ab0 0x55df08433f30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2c9603e8275d8a18 server_cookie=cec2aedfe3c69153 in_seq=0 out_seq=0 2024-09-19T09:02:06.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.788+0000 7fd3f7fff640 1 -- 172.21.15.120:0/3260806920 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55df08314b10 con 0x55df083c6ab0 2024-09-19T09:02:06.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.788+0000 7fd3f7fff640 10 client.14991 handle_command_reply: tid=0 2024-09-19T09:02:06.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.788+0000 7fd421121740 1 client.14991 shutdown 2024-09-19T09:02:06.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.788+0000 7fd421121740 1 -- 172.21.15.120:0/3260806920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df083a72f0 msgr2=0x55df083cbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:06.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.788+0000 7fd421121740 1 --2- 172.21.15.120:0/3260806920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df083a72f0 0x55df083cbe40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd40c09b030 tx=0x7fd40c000bc0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.788+0000 7fd421121740 1 -- 172.21.15.120:0/3260806920 shutdown_connections 2024-09-19T09:02:06.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.788+0000 7fd421121740 1 --2- 172.21.15.120:0/3260806920 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55df083c6ab0 0x55df08433f30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.788+0000 7fd421121740 1 --2- 172.21.15.120:0/3260806920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df083b1bb0 0x55df08429180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.788+0000 7fd421121740 1 --2- 172.21.15.120:0/3260806920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df083a72f0 0x55df083cbe40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.788+0000 7fd421121740 1 --2- 172.21.15.120:0/3260806920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df083a69d0 0x55df083cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.788+0000 7fd421121740 1 -- 172.21.15.120:0/3260806920 >> 172.21.15.120:0/3260806920 conn(0x55df083a2f60 msgr2=0x55df083ae720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:06.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.788+0000 7fd3f7fff640 0 client.14991 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:02:06.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.788+0000 7fd421121740 1 -- 172.21.15.120:0/3260806920 shutdown_connections 2024-09-19T09:02:06.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.788+0000 7fd421121740 1 -- 172.21.15.120:0/3260806920 wait complete. 2024-09-19T09:02:06.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.789+0000 7fd421121740 20 client.14991 trim_cache size 0 max 16384 2024-09-19T09:02:06.797 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:02:06.797 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_fsid": "507393d2-7664-11ef-bceb-c7b262605968", 2024-09-19T09:02:06.797 INFO:teuthology.orchestra.run.smithi120.stdout: "whoami": 3, 2024-09-19T09:02:06.797 INFO:teuthology.orchestra.run.smithi120.stdout: "id": 24430, 2024-09-19T09:02:06.797 INFO:teuthology.orchestra.run.smithi120.stdout: "want_state": "up:active", 2024-09-19T09:02:06.797 INFO:teuthology.orchestra.run.smithi120.stdout: "state": "up:active", 2024-09-19T09:02:06.798 INFO:teuthology.orchestra.run.smithi120.stdout: "fs_name": "cephfs", 2024-09-19T09:02:06.798 INFO:teuthology.orchestra.run.smithi120.stdout: "rank_uptime": 69.671187434999993, 2024-09-19T09:02:06.798 INFO:teuthology.orchestra.run.smithi120.stdout: "mdsmap_epoch": 27, 2024-09-19T09:02:06.798 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch": 81, 2024-09-19T09:02:06.798 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch_barrier": 79, 2024-09-19T09:02:06.798 INFO:teuthology.orchestra.run.smithi120.stdout: "uptime": 103.579230045 2024-09-19T09:02:06.798 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:02:06.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.789+0000 7fd41ed48640 1 -- 172.21.15.120:0/3920301692 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd4000a32d0 msgr2=0x7fd4000a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:06.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.789+0000 7fd41ed48640 1 --2- 172.21.15.120:0/3920301692 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd4000a32d0 0x7fd4000a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd408000c00 tx=0x7fd408031690 comp rx=0 tx=0).stop 2024-09-19T09:02:06.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.789+0000 7fd41ed48640 1 -- 172.21.15.120:0/3920301692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd418102790 msgr2=0x7fd418198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:06.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.789+0000 7fd41ed48640 1 --2- 172.21.15.120:0/3920301692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd418102790 0x7fd418198af0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00b4a0 tx=0x7fd40c00b960 comp rx=0 tx=0).stop 2024-09-19T09:02:06.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.789+0000 7fd41ed48640 1 -- 172.21.15.120:0/3920301692 shutdown_connections 2024-09-19T09:02:06.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.789+0000 7fd41ed48640 1 --2- 172.21.15.120:0/3920301692 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd4000a32d0 0x7fd4000a5790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.789+0000 7fd41ed48640 1 --2- 172.21.15.120:0/3920301692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd418106f10 0x7fd41819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.789+0000 7fd41ed48640 1 --2- 172.21.15.120:0/3920301692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd418102790 0x7fd418198af0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.789+0000 7fd41ed48640 1 --2- 172.21.15.120:0/3920301692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd418101de0 0x7fd4181985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:06.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.789+0000 7fd41ed48640 1 -- 172.21.15.120:0/3920301692 >> 172.21.15.120:0/3920301692 conn(0x7fd4180fd8c0 msgr2=0x7fd4180fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:06.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.789+0000 7fd41ed48640 1 -- 172.21.15.120:0/3920301692 shutdown_connections 2024-09-19T09:02:06.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:06.789+0000 7fd41ed48640 1 -- 172.21.15.120:0/3920301692 wait complete. 2024-09-19T09:02:07.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:06 smithi120 ceph-mon[31150]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 113 B/s, 1 objects/s recovering 2024-09-19T09:02:07.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:06 smithi196 ceph-mon[35150]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 113 B/s, 1 objects/s recovering 2024-09-19T09:02:07.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:06 smithi153 ceph-mon[35071]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 113 B/s, 1 objects/s recovering 2024-09-19T09:02:07.695 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T09:02:07.941 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:08.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.402+0000 7fcb2ff94640 1 -- 172.21.15.120:0/897867780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb28102570 msgr2=0x7fcb281029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:08.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.402+0000 7fcb2ff94640 1 --2- 172.21.15.120:0/897867780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb28102570 0x7fcb281029b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcb24009900 tx=0x7fcb2402a480 comp rx=0 tx=0).stop 2024-09-19T09:02:08.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.402+0000 7fcb2ff94640 1 -- 172.21.15.120:0/897867780 shutdown_connections 2024-09-19T09:02:08.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.402+0000 7fcb2ff94640 1 --2- 172.21.15.120:0/897867780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb28102ef0 0x7fcb2810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.402+0000 7fcb2ff94640 1 --2- 172.21.15.120:0/897867780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb28102570 0x7fcb281029b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.402+0000 7fcb2ff94640 1 --2- 172.21.15.120:0/897867780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb28100e20 0x7fcb28101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.402+0000 7fcb2ff94640 1 -- 172.21.15.120:0/897867780 >> 172.21.15.120:0/897867780 conn(0x7fcb280fc7d0 msgr2=0x7fcb280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:08.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.402+0000 7fcb2ff94640 1 -- 172.21.15.120:0/897867780 shutdown_connections 2024-09-19T09:02:08.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.402+0000 7fcb2ff94640 1 -- 172.21.15.120:0/897867780 wait complete. 2024-09-19T09:02:08.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.402+0000 7fcb2ff94640 1 Processor -- start 2024-09-19T09:02:08.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.402+0000 7fcb2ff94640 1 -- start start 2024-09-19T09:02:08.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.402+0000 7fcb2ff94640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb28100e20 0x7fcb2819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:08.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.403+0000 7fcb2ff94640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb28102570 0x7fcb2819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:08.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.403+0000 7fcb2ff94640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb28102ef0 0x7fcb2819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:08.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.403+0000 7fcb2ff94640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb28110fa0 con 0x7fcb28102ef0 2024-09-19T09:02:08.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.403+0000 7fcb2ff94640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcb28110e20 con 0x7fcb28102570 2024-09-19T09:02:08.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.403+0000 7fcb2ff94640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcb28111120 con 0x7fcb28100e20 2024-09-19T09:02:08.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.403+0000 7fcb2e531640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb28102ef0 0x7fcb2819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:08.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.403+0000 7fcb2e531640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb28102ef0 0x7fcb2819ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57828/0 (socket says 172.21.15.120:57828) 2024-09-19T09:02:08.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.403+0000 7fcb2e531640 1 -- 172.21.15.120:0/3520554613 learned_addr learned my addr 172.21.15.120:0/3520554613 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:08.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.403+0000 7fcb2d52f640 1 --2- 172.21.15.120:0/3520554613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb28102570 0x7fcb2819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:08.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.403+0000 7fcb2dd30640 1 --2- 172.21.15.120:0/3520554613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb28100e20 0x7fcb2819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:08.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.403+0000 7fcb2e531640 1 -- 172.21.15.120:0/3520554613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb28100e20 msgr2=0x7fcb2819a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:08.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.403+0000 7fcb2e531640 1 --2- 172.21.15.120:0/3520554613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb28100e20 0x7fcb2819a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.403+0000 7fcb2e531640 1 -- 172.21.15.120:0/3520554613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb28102570 msgr2=0x7fcb2819abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:08.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.403+0000 7fcb2e531640 1 --2- 172.21.15.120:0/3520554613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb28102570 0x7fcb2819abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.403+0000 7fcb2e531640 1 -- 172.21.15.120:0/3520554613 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb2819f660 con 0x7fcb28102ef0 2024-09-19T09:02:08.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.403+0000 7fcb2e531640 1 --2- 172.21.15.120:0/3520554613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb28102ef0 0x7fcb2819ef80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000b970 tx=0x7fcb2000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:08.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.403+0000 7fcb1affd640 1 -- 172.21.15.120:0/3520554613 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb20019070 con 0x7fcb28102ef0 2024-09-19T09:02:08.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.403+0000 7fcb1affd640 1 -- 172.21.15.120:0/3520554613 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb20014440 con 0x7fcb28102ef0 2024-09-19T09:02:08.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.403+0000 7fcb2ff94640 1 -- 172.21.15.120:0/3520554613 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb2819f950 con 0x7fcb28102ef0 2024-09-19T09:02:08.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.403+0000 7fcb1affd640 1 -- 172.21.15.120:0/3520554613 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb200075d0 con 0x7fcb28102ef0 2024-09-19T09:02:08.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.403+0000 7fcb2ff94640 1 -- 172.21.15.120:0/3520554613 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb281a7230 con 0x7fcb28102ef0 2024-09-19T09:02:08.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.406+0000 7fcb1affd640 1 -- 172.21.15.120:0/3520554613 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fcb2001c070 con 0x7fcb28102ef0 2024-09-19T09:02:08.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.406+0000 7fcb3236d740 1 Processor -- start 2024-09-19T09:02:08.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.406+0000 7fcb3236d740 1 -- start start 2024-09-19T09:02:08.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.406+0000 7fcb3236d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e0538c9d0 0x557e0538cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:08.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.406+0000 7fcb3236d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e0538d2f0 0x557e05397670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:08.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.406+0000 7fcb3236d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e05397bb0 0x557e05399fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:08.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.406+0000 7fcb3236d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557e052f3090 con 0x557e0538d2f0 2024-09-19T09:02:08.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.406+0000 7fcb3236d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557e0529aa70 con 0x557e0538c9d0 2024-09-19T09:02:08.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.406+0000 7fcb3236d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557e052ef050 con 0x557e05397bb0 2024-09-19T09:02:08.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.406+0000 7fcb2dd30640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e0538d2f0 0x557e05397670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:08.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.406+0000 7fcb2dd30640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e0538d2f0 0x557e05397670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57842/0 (socket says 172.21.15.120:57842) 2024-09-19T09:02:08.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.406+0000 7fcb2dd30640 1 -- 172.21.15.120:0/4038731592 learned_addr learned my addr 172.21.15.120:0/4038731592 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:08.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.406+0000 7fcb1affd640 1 --2- 172.21.15.120:0/3520554613 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcb140a32d0 0x7fcb140a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:08.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.406+0000 7fcb2d52f640 1 --2- 172.21.15.120:0/4038731592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e0538c9d0 0x557e0538cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:08.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.407+0000 7fcb2d52f640 1 --2- 172.21.15.120:0/4038731592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e05397bb0 0x557e05399fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:08.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.407+0000 7fcb2e531640 1 --2- 172.21.15.120:0/3520554613 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcb140a32d0 0x7fcb140a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:08.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.407+0000 7fcb2dd30640 1 -- 172.21.15.120:0/4038731592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e05397bb0 msgr2=0x557e05399fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:08.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.407+0000 7fcb2dd30640 1 --2- 172.21.15.120:0/4038731592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e05397bb0 0x557e05399fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.407+0000 7fcb2dd30640 1 -- 172.21.15.120:0/4038731592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e0538c9d0 msgr2=0x557e0538cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:08.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.407+0000 7fcb2dd30640 1 --2- 172.21.15.120:0/4038731592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e0538c9d0 0x557e0538cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.407+0000 7fcb2dd30640 1 -- 172.21.15.120:0/4038731592 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e052a7130 con 0x557e0538d2f0 2024-09-19T09:02:08.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.407+0000 7fcb2d52f640 1 --2- 172.21.15.120:0/4038731592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e0538c9d0 0x557e0538cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:02:08.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.407+0000 7fcb1affd640 1 -- 172.21.15.120:0/3520554613 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fcb200cef30 con 0x7fcb28102ef0 2024-09-19T09:02:08.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.407+0000 7fcb2dd30640 1 --2- 172.21.15.120:0/4038731592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e0538d2f0 0x557e05397670 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00ad00 tx=0x7fcb1c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792fda7384a00470 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:08.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.407+0000 7fcb2e531640 1 --2- 172.21.15.120:0/3520554613 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcb140a32d0 0x7fcb140a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcb2009b2b0 tx=0x7fcb200c62d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:08.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.407+0000 7fcb13fff640 1 -- 172.21.15.120:0/4038731592 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c01d070 con 0x557e0538d2f0 2024-09-19T09:02:08.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.407+0000 7fcb13fff640 1 -- 172.21.15.120:0/4038731592 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb1c018b20 con 0x557e0538d2f0 2024-09-19T09:02:08.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.407+0000 7fcb13fff640 1 -- 172.21.15.120:0/4038731592 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c010bc0 con 0x557e0538d2f0 2024-09-19T09:02:08.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.408+0000 7fcb3236d740 1 -- 172.21.15.120:0/4038731592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e0538d2f0 msgr2=0x557e05397670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:08.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.408+0000 7fcb3236d740 1 --2- 172.21.15.120:0/4038731592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e0538d2f0 0x557e05397670 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00ad00 tx=0x7fcb1c00f790 comp rx=0 tx=0).stop 2024-09-19T09:02:08.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.408+0000 7fcb3236d740 1 -- 172.21.15.120:0/4038731592 shutdown_connections 2024-09-19T09:02:08.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.408+0000 7fcb3236d740 1 --2- 172.21.15.120:0/4038731592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e05397bb0 0x557e05399fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.408+0000 7fcb3236d740 1 --2- 172.21.15.120:0/4038731592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e0538d2f0 0x557e05397670 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.408+0000 7fcb3236d740 1 --2- 172.21.15.120:0/4038731592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e0538c9d0 0x557e0538cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.408+0000 7fcb3236d740 1 -- 172.21.15.120:0/4038731592 >> 172.21.15.120:0/4038731592 conn(0x557e05388f60 msgr2=0x557e05389340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:08.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.408+0000 7fcb3236d740 1 -- 172.21.15.120:0/4038731592 shutdown_connections 2024-09-19T09:02:08.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.408+0000 7fcb3236d740 1 -- 172.21.15.120:0/4038731592 wait complete. 2024-09-19T09:02:08.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.408+0000 7fcb3236d740 1 -- start start 2024-09-19T09:02:08.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.408+0000 7fcb3236d740 1 Processor -- start 2024-09-19T09:02:08.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.409+0000 7fcb3236d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e0538c9d0 0x557e05397450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:08.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.409+0000 7fcb3236d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e0538d2f0 0x557e053947a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:08.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.409+0000 7fcb3236d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e05397bb0 0x557e05394ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:08.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.409+0000 7fcb3236d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557e0539a780 con 0x557e0538c9d0 2024-09-19T09:02:08.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.409+0000 7fcb3236d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557e052adf20 con 0x557e05397bb0 2024-09-19T09:02:08.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.409+0000 7fcb3236d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557e0539a900 con 0x557e0538d2f0 2024-09-19T09:02:08.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.409+0000 7fcb3236d740 10 client.0 _ll_register_callbacks cb 0x557e05277030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:02:08.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.409+0000 7fcb2d52f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e0538c9d0 0x557e05397450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:08.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.409+0000 7fcb2d52f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e0538c9d0 0x557e05397450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57846/0 (socket says 172.21.15.120:57846) 2024-09-19T09:02:08.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.409+0000 7fcb2d52f640 1 -- 172.21.15.120:0/1595009822 learned_addr learned my addr 172.21.15.120:0/1595009822 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:08.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.409+0000 7fcb2d52f640 1 -- 172.21.15.120:0/1595009822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e0538d2f0 msgr2=0x557e053947a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:08.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.409+0000 7fcb2d52f640 1 --2- 172.21.15.120:0/1595009822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e0538d2f0 0x557e053947a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.409+0000 7fcb2d52f640 1 -- 172.21.15.120:0/1595009822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e05397bb0 msgr2=0x557e05394ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:08.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.409+0000 7fcb2d52f640 1 --2- 172.21.15.120:0/1595009822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e05397bb0 0x557e05394ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.409+0000 7fcb2d52f640 1 -- 172.21.15.120:0/1595009822 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e050642b0 con 0x557e0538c9d0 2024-09-19T09:02:08.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.409+0000 7fcb2d52f640 1 --2- 172.21.15.120:0/1595009822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e0538c9d0 0x557e05397450 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcb24009900 tx=0x7fcb240045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e5bd015a222716 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:08.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.409+0000 7fcb117fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:02:08.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.409+0000 7fcb117fa640 1 -- 172.21.15.120:0/1595009822 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24043030 con 0x557e0538c9d0 2024-09-19T09:02:08.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.409+0000 7fcb3236d740 1 -- 172.21.15.120:0/1595009822 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb1c018b20 con 0x557e0538c9d0 2024-09-19T09:02:08.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.409+0000 7fcb117fa640 1 -- 172.21.15.120:0/1595009822 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fcb24003940 con 0x557e0538c9d0 2024-09-19T09:02:08.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.410+0000 7fcb117fa640 1 -- 172.21.15.120:0/1595009822 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcb24035040 con 0x557e0538c9d0 2024-09-19T09:02:08.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.410+0000 7fcb3236d740 10 client.15003 fetch_fsmap learned FSMap version 27 2024-09-19T09:02:08.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.410+0000 7fcb3236d740 1 -- 172.21.15.120:0/1595009822 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557e053a60f0 con 0x557e0538c9d0 2024-09-19T09:02:08.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.410+0000 7fcb117fa640 1 -- 172.21.15.120:0/1595009822 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb1c018b20 con 0x557e0538c9d0 2024-09-19T09:02:08.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.410+0000 7fcb117fa640 1 -- 172.21.15.120:0/1595009822 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcb2403d480 con 0x557e0538c9d0 2024-09-19T09:02:08.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.410+0000 7fcb3236d740 10 client.15003 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:02:08.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.410+0000 7fcb3236d740 10 client.15003 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T09:02:08.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.410+0000 7fcb3236d740 20 client.15003 populate_metadata read hostname 'smithi120' 2024-09-19T09:02:08.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.410+0000 7fcb3236d740 1 --2- 172.21.15.120:0/1595009822 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557e053a87a0 0x557e053a9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:08.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.410+0000 7fcb3236d740 4 client.15003 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:02:08.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.410+0000 7fcb3236d740 1 -- 172.21.15.120:0/1595009822 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e05300600 con 0x557e053a87a0 2024-09-19T09:02:08.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.410+0000 7fcb2dd30640 1 --2- 172.21.15.120:0/1595009822 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557e053a87a0 0x557e053a9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:08.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.411+0000 7fcb2dd30640 1 --2- 172.21.15.120:0/1595009822 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557e053a87a0 0x557e053a9c10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=289dc808faf91a90 server_cookie=adc4199c13ccfa86 in_seq=0 out_seq=0 2024-09-19T09:02:08.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.411+0000 7fcb117fa640 10 client.15003 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:02:08.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.411+0000 7fcb117fa640 1 -- 172.21.15.120:0/1595009822 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557e05300600 con 0x557e053a87a0 2024-09-19T09:02:08.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.411+0000 7fcb117fa640 10 client.15003 handle_command_reply: tid=0 2024-09-19T09:02:08.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.411+0000 7fcb3236d740 1 client.15003 shutdown 2024-09-19T09:02:08.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.412+0000 7fcb3236d740 1 -- 172.21.15.120:0/1595009822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e0538c9d0 msgr2=0x557e05397450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:08.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.412+0000 7fcb3236d740 1 --2- 172.21.15.120:0/1595009822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e0538c9d0 0x557e05397450 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcb24009900 tx=0x7fcb240045d0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.412+0000 7fcb3236d740 1 -- 172.21.15.120:0/1595009822 shutdown_connections 2024-09-19T09:02:08.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.412+0000 7fcb3236d740 1 --2- 172.21.15.120:0/1595009822 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557e053a87a0 0x557e053a9c10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.412+0000 7fcb3236d740 1 --2- 172.21.15.120:0/1595009822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e05397bb0 0x557e05394ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.412+0000 7fcb3236d740 1 --2- 172.21.15.120:0/1595009822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e0538d2f0 0x557e053947a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.412+0000 7fcb3236d740 1 --2- 172.21.15.120:0/1595009822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e0538c9d0 0x557e05397450 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.412+0000 7fcb117fa640 0 client.15003 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:02:08.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.412+0000 7fcb3236d740 1 -- 172.21.15.120:0/1595009822 >> 172.21.15.120:0/1595009822 conn(0x557e05388f60 msgr2=0x557e0538ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:08.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.412+0000 7fcb3236d740 1 -- 172.21.15.120:0/1595009822 shutdown_connections 2024-09-19T09:02:08.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.412+0000 7fcb3236d740 1 -- 172.21.15.120:0/1595009822 wait complete. 2024-09-19T09:02:08.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.412+0000 7fcb3236d740 20 client.15003 trim_cache size 0 max 16384 2024-09-19T09:02:08.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.429+0000 7fcb3236d740 1 Processor -- start 2024-09-19T09:02:08.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.429+0000 7fcb3236d740 1 -- start start 2024-09-19T09:02:08.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.429+0000 7fcb3236d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e0538c9d0 0x557e053a9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:08.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.429+0000 7fcb3236d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e0538d2f0 0x557e05383ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:08.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.429+0000 7fcb3236d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e05397bb0 0x557e05393070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:08.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.429+0000 7fcb3236d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557e053a7f50 con 0x557e0538d2f0 2024-09-19T09:02:08.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.429+0000 7fcb3236d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557e053a8620 con 0x557e0538c9d0 2024-09-19T09:02:08.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.429+0000 7fcb3236d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557e053a8cf0 con 0x557e05397bb0 2024-09-19T09:02:08.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.430+0000 7fcb2dd30640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e0538d2f0 0x557e05383ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:08.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.430+0000 7fcb2dd30640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e0538d2f0 0x557e05383ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57852/0 (socket says 172.21.15.120:57852) 2024-09-19T09:02:08.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.430+0000 7fcb2dd30640 1 -- 172.21.15.120:0/1821881286 learned_addr learned my addr 172.21.15.120:0/1821881286 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:08.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.430+0000 7fcb2d52f640 1 --2- 172.21.15.120:0/1821881286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e05397bb0 0x557e05393070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:08.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.430+0000 7fcb2dd30640 1 -- 172.21.15.120:0/1821881286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e05397bb0 msgr2=0x557e05393070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:08.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.430+0000 7fcb2dd30640 1 --2- 172.21.15.120:0/1821881286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e05397bb0 0x557e05393070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.430+0000 7fcb2dd30640 1 -- 172.21.15.120:0/1821881286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e0538c9d0 msgr2=0x557e053a9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:08.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.430+0000 7fcb2dd30640 1 --2- 172.21.15.120:0/1821881286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e0538c9d0 0x557e053a9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.430+0000 7fcb2dd30640 1 -- 172.21.15.120:0/1821881286 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e0538c800 con 0x557e0538d2f0 2024-09-19T09:02:08.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.430+0000 7fcb2dd30640 1 --2- 172.21.15.120:0/1821881286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e0538d2f0 0x557e05383ce0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c0107d0 tx=0x7fcb1c010760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8bd7c5cbd6d091b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:08.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.430+0000 7fcb137fe640 1 -- 172.21.15.120:0/1821881286 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c01d040 con 0x557e0538d2f0 2024-09-19T09:02:08.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.430+0000 7fcb137fe640 1 -- 172.21.15.120:0/1821881286 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb1c029040 con 0x557e0538d2f0 2024-09-19T09:02:08.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.430+0000 7fcb137fe640 1 -- 172.21.15.120:0/1821881286 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c005540 con 0x557e0538d2f0 2024-09-19T09:02:08.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.431+0000 7fcb3236d740 1 -- 172.21.15.120:0/1821881286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e0538d2f0 msgr2=0x557e05383ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:08.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.431+0000 7fcb3236d740 1 --2- 172.21.15.120:0/1821881286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e0538d2f0 0x557e05383ce0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c0107d0 tx=0x7fcb1c010760 comp rx=0 tx=0).stop 2024-09-19T09:02:08.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.431+0000 7fcb3236d740 1 -- 172.21.15.120:0/1821881286 shutdown_connections 2024-09-19T09:02:08.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.431+0000 7fcb3236d740 1 --2- 172.21.15.120:0/1821881286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e05397bb0 0x557e05393070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.431+0000 7fcb3236d740 1 --2- 172.21.15.120:0/1821881286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e0538d2f0 0x557e05383ce0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.431+0000 7fcb3236d740 1 --2- 172.21.15.120:0/1821881286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e0538c9d0 0x557e053a9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.431+0000 7fcb3236d740 1 -- 172.21.15.120:0/1821881286 >> 172.21.15.120:0/1821881286 conn(0x557e05388f60 msgr2=0x557e05394760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:08.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.431+0000 7fcb3236d740 1 -- 172.21.15.120:0/1821881286 shutdown_connections 2024-09-19T09:02:08.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.431+0000 7fcb3236d740 1 -- 172.21.15.120:0/1821881286 wait complete. 2024-09-19T09:02:08.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.431+0000 7fcb3236d740 1 -- start start 2024-09-19T09:02:08.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.431+0000 7fcb3236d740 1 Processor -- start 2024-09-19T09:02:08.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.432+0000 7fcb3236d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e0538c9d0 0x557e053b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:08.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.432+0000 7fcb3236d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e0538d2f0 0x557e053b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:08.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.432+0000 7fcb3236d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e05397bb0 0x557e053ad460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:08.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.432+0000 7fcb3236d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557e053a9a90 con 0x557e0538d2f0 2024-09-19T09:02:08.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.432+0000 7fcb3236d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557e053a93c0 con 0x557e0538c9d0 2024-09-19T09:02:08.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.432+0000 7fcb3236d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557e053aa160 con 0x557e05397bb0 2024-09-19T09:02:08.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.432+0000 7fcb3236d740 10 client.0 _ll_register_callbacks cb 0x7fcb24009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:02:08.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.432+0000 7fcb2dd30640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e0538d2f0 0x557e053b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:08.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.432+0000 7fcb2dd30640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e0538d2f0 0x557e053b24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57866/0 (socket says 172.21.15.120:57866) 2024-09-19T09:02:08.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.432+0000 7fcb2dd30640 1 -- 172.21.15.120:0/1200254646 learned_addr learned my addr 172.21.15.120:0/1200254646 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:08.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.432+0000 7fcb2d52f640 1 --2- 172.21.15.120:0/1200254646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e0538c9d0 0x557e053b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:08.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.432+0000 7fcb2d52f640 1 --2- 172.21.15.120:0/1200254646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e05397bb0 0x557e053ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:08.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.432+0000 7fcb2dd30640 1 -- 172.21.15.120:0/1200254646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e05397bb0 msgr2=0x557e053ad460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:08.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.432+0000 7fcb2dd30640 1 --2- 172.21.15.120:0/1200254646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e05397bb0 0x557e053ad460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.432+0000 7fcb2dd30640 1 -- 172.21.15.120:0/1200254646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e0538c9d0 msgr2=0x557e053b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:08.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.432+0000 7fcb2dd30640 1 --2- 172.21.15.120:0/1200254646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e0538c9d0 0x557e053b1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.432+0000 7fcb2dd30640 1 -- 172.21.15.120:0/1200254646 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e053adc90 con 0x557e0538d2f0 2024-09-19T09:02:08.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.432+0000 7fcb2dd30640 1 --2- 172.21.15.120:0/1200254646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e0538d2f0 0x557e053b24d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00fc90 tx=0x7fcb1c005ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4255b5c709fccf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:08.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.432+0000 7fcb117fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:02:08.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.432+0000 7fcb117fa640 1 -- 172.21.15.120:0/1200254646 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c01d040 con 0x557e0538d2f0 2024-09-19T09:02:08.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.432+0000 7fcb117fa640 1 -- 172.21.15.120:0/1200254646 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fcb1c0052c0 con 0x557e0538d2f0 2024-09-19T09:02:08.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.432+0000 7fcb3236d740 1 -- 172.21.15.120:0/1200254646 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb1c029040 con 0x557e0538d2f0 2024-09-19T09:02:08.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.432+0000 7fcb117fa640 1 -- 172.21.15.120:0/1200254646 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcb1c00fd40 con 0x557e0538d2f0 2024-09-19T09:02:08.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.432+0000 7fcb3236d740 10 client.15009 fetch_fsmap learned FSMap version 27 2024-09-19T09:02:08.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.433+0000 7fcb3236d740 1 -- 172.21.15.120:0/1200254646 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557e05412990 con 0x557e0538d2f0 2024-09-19T09:02:08.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.433+0000 7fcb117fa640 1 -- 172.21.15.120:0/1200254646 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb1c029040 con 0x557e0538d2f0 2024-09-19T09:02:08.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.433+0000 7fcb117fa640 1 -- 172.21.15.120:0/1200254646 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcb1c004a30 con 0x557e0538d2f0 2024-09-19T09:02:08.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.433+0000 7fcb3236d740 10 client.15009 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:02:08.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.433+0000 7fcb3236d740 10 client.15009 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T09:02:08.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.433+0000 7fcb3236d740 20 client.15009 populate_metadata read hostname 'smithi120' 2024-09-19T09:02:08.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.433+0000 7fcb3236d740 1 --2- 172.21.15.120:0/1200254646 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557e05416ee0 0x557e05415bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:08.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.433+0000 7fcb3236d740 4 client.15009 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:02:08.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.433+0000 7fcb3236d740 1 -- 172.21.15.120:0/1200254646 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "status"}) -- 0x557e052fab10 con 0x557e05416ee0 2024-09-19T09:02:08.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.433+0000 7fcb2d52f640 1 --2- 172.21.15.120:0/1200254646 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557e05416ee0 0x557e05415bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:08.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.433+0000 7fcb2d52f640 1 --2- 172.21.15.120:0/1200254646 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557e05416ee0 0x557e05415bf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6e2197088600bb2b server_cookie=211da4dabdee1bed in_seq=0 out_seq=0 2024-09-19T09:02:08.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.433+0000 7fcb117fa640 10 client.15009 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:02:08.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.434+0000 7fcb117fa640 1 -- 172.21.15.120:0/1200254646 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557e052fab10 con 0x557e05416ee0 2024-09-19T09:02:08.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.434+0000 7fcb117fa640 10 client.15009 handle_command_reply: tid=0 2024-09-19T09:02:08.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.434+0000 7fcb3236d740 1 client.15009 shutdown 2024-09-19T09:02:08.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.434+0000 7fcb3236d740 1 -- 172.21.15.120:0/1200254646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e0538d2f0 msgr2=0x557e053b24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:08.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.434+0000 7fcb3236d740 1 --2- 172.21.15.120:0/1200254646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e0538d2f0 0x557e053b24d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00fc90 tx=0x7fcb1c005ed0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.434+0000 7fcb3236d740 1 -- 172.21.15.120:0/1200254646 shutdown_connections 2024-09-19T09:02:08.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.434+0000 7fcb3236d740 1 --2- 172.21.15.120:0/1200254646 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557e05416ee0 0x557e05415bf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.434+0000 7fcb3236d740 1 --2- 172.21.15.120:0/1200254646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e05397bb0 0x557e053ad460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.434+0000 7fcb3236d740 1 --2- 172.21.15.120:0/1200254646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e0538d2f0 0x557e053b24d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.434+0000 7fcb3236d740 1 --2- 172.21.15.120:0/1200254646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e0538c9d0 0x557e053b1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.434+0000 7fcb117fa640 0 client.15009 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:02:08.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.434+0000 7fcb3236d740 1 -- 172.21.15.120:0/1200254646 >> 172.21.15.120:0/1200254646 conn(0x557e05388f60 msgr2=0x557e05392c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:08.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.434+0000 7fcb3236d740 1 -- 172.21.15.120:0/1200254646 shutdown_connections 2024-09-19T09:02:08.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.434+0000 7fcb3236d740 1 -- 172.21.15.120:0/1200254646 wait complete. 2024-09-19T09:02:08.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.434+0000 7fcb3236d740 20 client.15009 trim_cache size 0 max 16384 2024-09-19T09:02:08.442 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:02:08.442 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_fsid": "507393d2-7664-11ef-bceb-c7b262605968", 2024-09-19T09:02:08.442 INFO:teuthology.orchestra.run.smithi120.stdout: "whoami": 2, 2024-09-19T09:02:08.442 INFO:teuthology.orchestra.run.smithi120.stdout: "id": 24442, 2024-09-19T09:02:08.442 INFO:teuthology.orchestra.run.smithi120.stdout: "want_state": "up:active", 2024-09-19T09:02:08.442 INFO:teuthology.orchestra.run.smithi120.stdout: "state": "up:active", 2024-09-19T09:02:08.442 INFO:teuthology.orchestra.run.smithi120.stdout: "fs_name": "cephfs", 2024-09-19T09:02:08.442 INFO:teuthology.orchestra.run.smithi120.stdout: "rank_uptime": 72.323791923000002, 2024-09-19T09:02:08.442 INFO:teuthology.orchestra.run.smithi120.stdout: "mdsmap_epoch": 27, 2024-09-19T09:02:08.442 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch": 81, 2024-09-19T09:02:08.442 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch_barrier": 79, 2024-09-19T09:02:08.442 INFO:teuthology.orchestra.run.smithi120.stdout: "uptime": 99.960385052999996 2024-09-19T09:02:08.443 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:02:08.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.435+0000 7fcb2ff94640 1 -- 172.21.15.120:0/3520554613 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcb140a32d0 msgr2=0x7fcb140a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:08.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.435+0000 7fcb2ff94640 1 --2- 172.21.15.120:0/3520554613 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcb140a32d0 0x7fcb140a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcb2009b2b0 tx=0x7fcb200c62d0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.435+0000 7fcb2ff94640 1 -- 172.21.15.120:0/3520554613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb28102ef0 msgr2=0x7fcb2819ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:08.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.435+0000 7fcb2ff94640 1 --2- 172.21.15.120:0/3520554613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb28102ef0 0x7fcb2819ef80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000b970 tx=0x7fcb2000bd70 comp rx=0 tx=0).stop 2024-09-19T09:02:08.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.435+0000 7fcb2ff94640 1 -- 172.21.15.120:0/3520554613 shutdown_connections 2024-09-19T09:02:08.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.435+0000 7fcb2ff94640 1 --2- 172.21.15.120:0/3520554613 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcb140a32d0 0x7fcb140a5790 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.435+0000 7fcb2ff94640 1 --2- 172.21.15.120:0/3520554613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb28102ef0 0x7fcb2819ef80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.435+0000 7fcb2ff94640 1 --2- 172.21.15.120:0/3520554613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb28102570 0x7fcb2819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.435+0000 7fcb2ff94640 1 --2- 172.21.15.120:0/3520554613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb28100e20 0x7fcb2819a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:08.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.435+0000 7fcb2ff94640 1 -- 172.21.15.120:0/3520554613 >> 172.21.15.120:0/3520554613 conn(0x7fcb280fc7d0 msgr2=0x7fcb28069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:08.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.435+0000 7fcb2ff94640 1 -- 172.21.15.120:0/3520554613 shutdown_connections 2024-09-19T09:02:08.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:08.435+0000 7fcb2ff94640 1 -- 172.21.15.120:0/3520554613 wait complete. 2024-09-19T09:02:09.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:08 smithi120 ceph-mon[31150]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s; 104 B/s, 0 objects/s recovering 2024-09-19T09:02:09.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:08 smithi196 ceph-mon[35150]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s; 104 B/s, 0 objects/s recovering 2024-09-19T09:02:09.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:08 smithi153 ceph-mon[35071]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s; 104 B/s, 0 objects/s recovering 2024-09-19T09:02:09.263 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T09:02:09.505 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:10.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.012+0000 7f6ba48b1640 1 -- 172.21.15.120:0/677728351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ba0101df0 msgr2=0x7f6ba01021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:10.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.012+0000 7f6ba48b1640 1 --2- 172.21.15.120:0/677728351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ba0101df0 0x7f6ba01021d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6b90009900 tx=0x7f6b9002a410 comp rx=0 tx=0).stop 2024-09-19T09:02:10.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.012+0000 7f6ba48b1640 1 -- 172.21.15.120:0/677728351 shutdown_connections 2024-09-19T09:02:10.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.012+0000 7f6ba48b1640 1 --2- 172.21.15.120:0/677728351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ba0106f20 0x7f6ba010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.012+0000 7f6ba48b1640 1 --2- 172.21.15.120:0/677728351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ba01027a0 0x7f6ba01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.012+0000 7f6ba48b1640 1 --2- 172.21.15.120:0/677728351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ba0101df0 0x7f6ba01021d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.012+0000 7f6ba48b1640 1 -- 172.21.15.120:0/677728351 >> 172.21.15.120:0/677728351 conn(0x7f6ba00fd8f0 msgr2=0x7f6ba00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:10.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.013+0000 7f6ba48b1640 1 -- 172.21.15.120:0/677728351 shutdown_connections 2024-09-19T09:02:10.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.013+0000 7f6ba48b1640 1 -- 172.21.15.120:0/677728351 wait complete. 2024-09-19T09:02:10.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.013+0000 7f6ba48b1640 1 Processor -- start 2024-09-19T09:02:10.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.013+0000 7f6ba48b1640 1 -- start start 2024-09-19T09:02:10.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.013+0000 7f6ba48b1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ba0101df0 0x7f6ba0198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:10.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.013+0000 7f6ba48b1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ba01027a0 0x7f6ba0198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:10.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.013+0000 7f6ba48b1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ba0106f20 0x7f6ba019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:10.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.013+0000 7f6ba48b1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba0111fa0 con 0x7f6ba01027a0 2024-09-19T09:02:10.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.013+0000 7f6ba48b1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba0111e20 con 0x7f6ba0106f20 2024-09-19T09:02:10.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.013+0000 7f6ba48b1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba0112120 con 0x7f6ba0101df0 2024-09-19T09:02:10.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.014+0000 7f6b9dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ba01027a0 0x7f6ba0198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:10.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.014+0000 7f6b9dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ba01027a0 0x7f6ba0198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57876/0 (socket says 172.21.15.120:57876) 2024-09-19T09:02:10.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.014+0000 7f6b9dd9b640 1 -- 172.21.15.120:0/4054886887 learned_addr learned my addr 172.21.15.120:0/4054886887 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:10.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.014+0000 7f6b9ed9d640 1 --2- 172.21.15.120:0/4054886887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ba0106f20 0x7f6ba019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:10.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.014+0000 7f6b9e59c640 1 --2- 172.21.15.120:0/4054886887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ba0101df0 0x7f6ba0198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:10.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.014+0000 7f6b9dd9b640 1 -- 172.21.15.120:0/4054886887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ba0101df0 msgr2=0x7f6ba0198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:10.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.014+0000 7f6b9dd9b640 1 --2- 172.21.15.120:0/4054886887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ba0101df0 0x7f6ba0198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.014+0000 7f6b9dd9b640 1 -- 172.21.15.120:0/4054886887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ba0106f20 msgr2=0x7f6ba019cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:10.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.014+0000 7f6b9dd9b640 1 --2- 172.21.15.120:0/4054886887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ba0106f20 0x7f6ba019cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.014+0000 7f6b9dd9b640 1 -- 172.21.15.120:0/4054886887 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba019d3f0 con 0x7f6ba01027a0 2024-09-19T09:02:10.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.014+0000 7f6b9dd9b640 1 --2- 172.21.15.120:0/4054886887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ba01027a0 0x7f6ba0198980 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400c920 tx=0x7f6b9400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:10.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.014+0000 7f6b8f7fe640 1 -- 172.21.15.120:0/4054886887 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b94007c70 con 0x7f6ba01027a0 2024-09-19T09:02:10.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.014+0000 7f6ba48b1640 1 -- 172.21.15.120:0/4054886887 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba019d6e0 con 0x7f6ba01027a0 2024-09-19T09:02:10.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.014+0000 7f6b8f7fe640 1 -- 172.21.15.120:0/4054886887 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b94007e10 con 0x7f6ba01027a0 2024-09-19T09:02:10.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.014+0000 7f6b8f7fe640 1 -- 172.21.15.120:0/4054886887 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b94018890 con 0x7f6ba01027a0 2024-09-19T09:02:10.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.014+0000 7f6ba48b1640 1 -- 172.21.15.120:0/4054886887 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba01a4fc0 con 0x7f6ba01027a0 2024-09-19T09:02:10.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.016+0000 7f6b8f7fe640 1 -- 172.21.15.120:0/4054886887 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f6b94018a70 con 0x7f6ba01027a0 2024-09-19T09:02:10.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.017+0000 7f6ba6c8a740 1 Processor -- start 2024-09-19T09:02:10.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.017+0000 7f6ba6c8a740 1 -- start start 2024-09-19T09:02:10.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.017+0000 7f6ba6c8a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56211b0bc9d0 0x56211b0bcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:10.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.017+0000 7f6ba6c8a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211b0bd2f0 0x56211b0c7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:10.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.017+0000 7f6ba6c8a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56211b0c7bb0 0x56211b0c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:10.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.017+0000 7f6ba6c8a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56211b023090 con 0x56211b0bc9d0 2024-09-19T09:02:10.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.017+0000 7f6ba6c8a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56211afcaa70 con 0x56211b0bd2f0 2024-09-19T09:02:10.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.017+0000 7f6ba6c8a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56211b01f050 con 0x56211b0c7bb0 2024-09-19T09:02:10.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.017+0000 7f6b9ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56211b0bc9d0 0x56211b0bcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:10.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.017+0000 7f6b9ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56211b0bc9d0 0x56211b0bcdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57884/0 (socket says 172.21.15.120:57884) 2024-09-19T09:02:10.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.017+0000 7f6b9ed9d640 1 -- 172.21.15.120:0/1001468916 learned_addr learned my addr 172.21.15.120:0/1001468916 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:10.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.017+0000 7f6b9e59c640 1 --2- 172.21.15.120:0/1001468916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211b0bd2f0 0x56211b0c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:10.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.017+0000 7f6b8f7fe640 1 --2- 172.21.15.120:0/4054886887 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6b880a32d0 0x7f6b880a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:10.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.017+0000 7f6b9ed9d640 1 -- 172.21.15.120:0/1001468916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56211b0c7bb0 msgr2=0x56211b0c9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:10.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.017+0000 7f6b9ed9d640 1 --2- 172.21.15.120:0/1001468916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56211b0c7bb0 0x56211b0c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.017+0000 7f6b9ed9d640 1 -- 172.21.15.120:0/1001468916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211b0bd2f0 msgr2=0x56211b0c7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:10.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.017+0000 7f6b9ed9d640 1 --2- 172.21.15.120:0/1001468916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211b0bd2f0 0x56211b0c7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.017+0000 7f6b9ed9d640 1 -- 172.21.15.120:0/1001468916 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56211afd7130 con 0x56211b0bc9d0 2024-09-19T09:02:10.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.018+0000 7f6b9ed9d640 1 --2- 172.21.15.120:0/4054886887 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6b880a32d0 0x7f6b880a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:10.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.018+0000 7f6b8f7fe640 1 -- 172.21.15.120:0/4054886887 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f6b940cfb40 con 0x7f6ba01027a0 2024-09-19T09:02:10.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.018+0000 7f6b9ed9d640 1 --2- 172.21.15.120:0/1001468916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56211b0bc9d0 0x56211b0bcdb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6b9800a5d0 tx=0x7f6b9800aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2c49f2b8a83d03 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:10.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.018+0000 7f6b8cff9640 1 -- 172.21.15.120:0/1001468916 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9801a040 con 0x56211b0bc9d0 2024-09-19T09:02:10.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.018+0000 7f6b8cff9640 1 -- 172.21.15.120:0/1001468916 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b98021ba0 con 0x56211b0bc9d0 2024-09-19T09:02:10.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.018+0000 7f6b8cff9640 1 -- 172.21.15.120:0/1001468916 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9801f350 con 0x56211b0bc9d0 2024-09-19T09:02:10.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.018+0000 7f6b9ed9d640 1 --2- 172.21.15.120:0/4054886887 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6b880a32d0 0x7f6b880a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6b98020000 tx=0x7f6b98021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:10.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.019+0000 7f6ba6c8a740 1 -- 172.21.15.120:0/1001468916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56211b0bc9d0 msgr2=0x56211b0bcdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:10.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.019+0000 7f6ba6c8a740 1 --2- 172.21.15.120:0/1001468916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56211b0bc9d0 0x56211b0bcdb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6b9800a5d0 tx=0x7f6b9800aa90 comp rx=0 tx=0).stop 2024-09-19T09:02:10.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.019+0000 7f6ba6c8a740 1 -- 172.21.15.120:0/1001468916 shutdown_connections 2024-09-19T09:02:10.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.019+0000 7f6ba6c8a740 1 --2- 172.21.15.120:0/1001468916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56211b0c7bb0 0x56211b0c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.019+0000 7f6ba6c8a740 1 --2- 172.21.15.120:0/1001468916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211b0bd2f0 0x56211b0c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.019+0000 7f6ba6c8a740 1 --2- 172.21.15.120:0/1001468916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56211b0bc9d0 0x56211b0bcdb0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.019+0000 7f6ba6c8a740 1 -- 172.21.15.120:0/1001468916 >> 172.21.15.120:0/1001468916 conn(0x56211b0b8f60 msgr2=0x56211b0b9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:10.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.019+0000 7f6ba6c8a740 1 -- 172.21.15.120:0/1001468916 shutdown_connections 2024-09-19T09:02:10.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.019+0000 7f6ba6c8a740 1 -- 172.21.15.120:0/1001468916 wait complete. 2024-09-19T09:02:10.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.019+0000 7f6ba6c8a740 1 -- start start 2024-09-19T09:02:10.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.019+0000 7f6ba6c8a740 1 Processor -- start 2024-09-19T09:02:10.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.019+0000 7f6ba6c8a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211b0bc9d0 0x56211b0c7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:10.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.019+0000 7f6ba6c8a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56211b0bd2f0 0x56211b0c47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:10.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.019+0000 7f6ba6c8a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56211b0c7bb0 0x56211b0c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:10.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.019+0000 7f6ba6c8a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56211b0ca780 con 0x56211b0bd2f0 2024-09-19T09:02:10.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.019+0000 7f6ba6c8a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56211afddf20 con 0x56211b0bc9d0 2024-09-19T09:02:10.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.019+0000 7f6ba6c8a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56211b0ca900 con 0x56211b0c7bb0 2024-09-19T09:02:10.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.019+0000 7f6ba6c8a740 10 client.0 _ll_register_callbacks cb 0x56211afa7030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:02:10.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.019+0000 7f6b9dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56211b0bd2f0 0x56211b0c47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:10.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.019+0000 7f6b9dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56211b0bd2f0 0x56211b0c47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57896/0 (socket says 172.21.15.120:57896) 2024-09-19T09:02:10.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.019+0000 7f6b9dd9b640 1 -- 172.21.15.120:0/3073650454 learned_addr learned my addr 172.21.15.120:0/3073650454 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:10.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.020+0000 7f6b9e59c640 1 --2- 172.21.15.120:0/3073650454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211b0bc9d0 0x56211b0c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:10.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.020+0000 7f6b9ed9d640 1 --2- 172.21.15.120:0/3073650454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56211b0c7bb0 0x56211b0c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:10.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.020+0000 7f6b9dd9b640 1 -- 172.21.15.120:0/3073650454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56211b0c7bb0 msgr2=0x56211b0c4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:10.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.020+0000 7f6b9dd9b640 1 --2- 172.21.15.120:0/3073650454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56211b0c7bb0 0x56211b0c4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.020+0000 7f6b9dd9b640 1 -- 172.21.15.120:0/3073650454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211b0bc9d0 msgr2=0x56211b0c7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:10.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.020+0000 7f6b9dd9b640 1 --2- 172.21.15.120:0/3073650454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211b0bc9d0 0x56211b0c7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.020+0000 7f6b9dd9b640 1 -- 172.21.15.120:0/3073650454 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56211ad942b0 con 0x56211b0bd2f0 2024-09-19T09:02:10.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.020+0000 7f6b9dd9b640 1 --2- 172.21.15.120:0/3073650454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56211b0bd2f0 0x56211b0c47d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6b9409bd00 tx=0x7f6b9409ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f916493667ebdee4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:10.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.020+0000 7f6b85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:02:10.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.020+0000 7f6b85ffb640 1 -- 172.21.15.120:0/3073650454 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b940a03b0 con 0x56211b0bd2f0 2024-09-19T09:02:10.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.020+0000 7f6b85ffb640 1 -- 172.21.15.120:0/3073650454 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f6b94018d60 con 0x56211b0bd2f0 2024-09-19T09:02:10.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.020+0000 7f6ba6c8a740 1 -- 172.21.15.120:0/3073650454 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b98021ba0 con 0x56211b0bd2f0 2024-09-19T09:02:10.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.020+0000 7f6b85ffb640 1 -- 172.21.15.120:0/3073650454 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6b940a4c70 con 0x56211b0bd2f0 2024-09-19T09:02:10.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.020+0000 7f6ba6c8a740 10 client.15021 fetch_fsmap learned FSMap version 27 2024-09-19T09:02:10.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.020+0000 7f6ba6c8a740 1 -- 172.21.15.120:0/3073650454 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56211b0d6130 con 0x56211b0bd2f0 2024-09-19T09:02:10.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.020+0000 7f6b85ffb640 1 -- 172.21.15.120:0/3073650454 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b98021ba0 con 0x56211b0bd2f0 2024-09-19T09:02:10.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.020+0000 7f6b85ffb640 1 -- 172.21.15.120:0/3073650454 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6b9409b450 con 0x56211b0bd2f0 2024-09-19T09:02:10.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.021+0000 7f6ba6c8a740 10 client.15021 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:02:10.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.021+0000 7f6ba6c8a740 10 client.15021 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T09:02:10.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.021+0000 7f6ba6c8a740 20 client.15021 populate_metadata read hostname 'smithi120' 2024-09-19T09:02:10.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.021+0000 7f6ba6c8a740 1 --2- 172.21.15.120:0/3073650454 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x56211b0d85f0 0x56211b0d9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:10.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.021+0000 7f6ba6c8a740 4 client.15021 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:02:10.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.021+0000 7f6ba6c8a740 1 -- 172.21.15.120:0/3073650454 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56211b030600 con 0x56211b0d85f0 2024-09-19T09:02:10.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.021+0000 7f6b9e59c640 1 --2- 172.21.15.120:0/3073650454 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x56211b0d85f0 0x56211b0d9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:10.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.021+0000 7f6b85ffb640 10 client.15021 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:02:10.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.021+0000 7f6b9e59c640 1 --2- 172.21.15.120:0/3073650454 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x56211b0d85f0 0x56211b0d9a60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4e2452312c902bea server_cookie=23d940db08fd7860 in_seq=0 out_seq=0 2024-09-19T09:02:10.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.022+0000 7f6b85ffb640 1 -- 172.21.15.120:0/3073650454 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56211b030600 con 0x56211b0d85f0 2024-09-19T09:02:10.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.022+0000 7f6b85ffb640 10 client.15021 handle_command_reply: tid=0 2024-09-19T09:02:10.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.022+0000 7f6ba6c8a740 1 client.15021 shutdown 2024-09-19T09:02:10.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.022+0000 7f6ba6c8a740 1 -- 172.21.15.120:0/3073650454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56211b0bd2f0 msgr2=0x56211b0c47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:10.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.022+0000 7f6ba6c8a740 1 --2- 172.21.15.120:0/3073650454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56211b0bd2f0 0x56211b0c47d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6b9409bd00 tx=0x7f6b9409ba20 comp rx=0 tx=0).stop 2024-09-19T09:02:10.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.022+0000 7f6ba6c8a740 1 -- 172.21.15.120:0/3073650454 shutdown_connections 2024-09-19T09:02:10.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.022+0000 7f6ba6c8a740 1 --2- 172.21.15.120:0/3073650454 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x56211b0d85f0 0x56211b0d9a60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.022+0000 7f6b85ffb640 0 client.15021 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:02:10.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.022+0000 7f6ba6c8a740 1 --2- 172.21.15.120:0/3073650454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56211b0c7bb0 0x56211b0c4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.022+0000 7f6ba6c8a740 1 --2- 172.21.15.120:0/3073650454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56211b0bd2f0 0x56211b0c47d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.022+0000 7f6ba6c8a740 1 --2- 172.21.15.120:0/3073650454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211b0bc9d0 0x56211b0c7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.022+0000 7f6ba6c8a740 1 -- 172.21.15.120:0/3073650454 >> 172.21.15.120:0/3073650454 conn(0x56211b0b8f60 msgr2=0x56211b0bec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:10.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.022+0000 7f6ba6c8a740 1 -- 172.21.15.120:0/3073650454 shutdown_connections 2024-09-19T09:02:10.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.022+0000 7f6ba6c8a740 1 -- 172.21.15.120:0/3073650454 wait complete. 2024-09-19T09:02:10.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.022+0000 7f6ba6c8a740 20 client.15021 trim_cache size 0 max 16384 2024-09-19T09:02:10.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.040+0000 7f6ba6c8a740 1 Processor -- start 2024-09-19T09:02:10.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.040+0000 7f6ba6c8a740 1 -- start start 2024-09-19T09:02:10.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.040+0000 7f6ba6c8a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56211b0bc9d0 0x56211b0d9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:10.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.040+0000 7f6ba6c8a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56211b0bd2f0 0x56211b0c4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:10.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.040+0000 7f6ba6c8a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211b0c7bb0 0x56211b0c30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:10.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.040+0000 7f6ba6c8a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56211b0d7da0 con 0x56211b0bd2f0 2024-09-19T09:02:10.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.040+0000 7f6ba6c8a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56211b0d8470 con 0x56211b0c7bb0 2024-09-19T09:02:10.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.040+0000 7f6ba6c8a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56211b0d8b40 con 0x56211b0bc9d0 2024-09-19T09:02:10.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.040+0000 7f6b9dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56211b0bd2f0 0x56211b0c4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:10.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.040+0000 7f6b9dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56211b0bd2f0 0x56211b0c4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57898/0 (socket says 172.21.15.120:57898) 2024-09-19T09:02:10.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.040+0000 7f6b9dd9b640 1 -- 172.21.15.120:0/1621420737 learned_addr learned my addr 172.21.15.120:0/1621420737 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:10.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.040+0000 7f6b9e59c640 1 --2- 172.21.15.120:0/1621420737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56211b0bc9d0 0x56211b0d9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:10.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.040+0000 7f6b9ed9d640 1 --2- 172.21.15.120:0/1621420737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211b0c7bb0 0x56211b0c30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:10.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.040+0000 7f6b9dd9b640 1 -- 172.21.15.120:0/1621420737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56211b0bc9d0 msgr2=0x56211b0d9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:10.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.040+0000 7f6b9dd9b640 1 --2- 172.21.15.120:0/1621420737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56211b0bc9d0 0x56211b0d9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.040+0000 7f6b9dd9b640 1 -- 172.21.15.120:0/1621420737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211b0c7bb0 msgr2=0x56211b0c30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:10.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.040+0000 7f6b9dd9b640 1 --2- 172.21.15.120:0/1621420737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211b0c7bb0 0x56211b0c30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.040+0000 7f6b9dd9b640 1 -- 172.21.15.120:0/1621420737 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56211b0c2d20 con 0x56211b0bd2f0 2024-09-19T09:02:10.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.041+0000 7f6b9ed9d640 1 --2- 172.21.15.120:0/1621420737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211b0c7bb0 0x56211b0c30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:02:10.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.041+0000 7f6b9e59c640 1 --2- 172.21.15.120:0/1621420737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56211b0bc9d0 0x56211b0d9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:02:10.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.041+0000 7f6b9dd9b640 1 --2- 172.21.15.120:0/1621420737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56211b0bd2f0 0x56211b0c4090 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6b940cda70 tx=0x7f6b9400fea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b885ab8bbfc717 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:10.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.041+0000 7f6b87fff640 1 -- 172.21.15.120:0/1621420737 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9407e050 con 0x56211b0bd2f0 2024-09-19T09:02:10.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.041+0000 7f6b87fff640 1 -- 172.21.15.120:0/1621420737 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b940a03b0 con 0x56211b0bd2f0 2024-09-19T09:02:10.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.041+0000 7f6b87fff640 1 -- 172.21.15.120:0/1621420737 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b94096b70 con 0x56211b0bd2f0 2024-09-19T09:02:10.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.041+0000 7f6ba6c8a740 1 -- 172.21.15.120:0/1621420737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56211b0bd2f0 msgr2=0x56211b0c4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:10.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.041+0000 7f6ba6c8a740 1 --2- 172.21.15.120:0/1621420737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56211b0bd2f0 0x56211b0c4090 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6b940cda70 tx=0x7f6b9400fea0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.041+0000 7f6ba6c8a740 1 -- 172.21.15.120:0/1621420737 shutdown_connections 2024-09-19T09:02:10.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.041+0000 7f6ba6c8a740 1 --2- 172.21.15.120:0/1621420737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211b0c7bb0 0x56211b0c30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.041+0000 7f6ba6c8a740 1 --2- 172.21.15.120:0/1621420737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56211b0bd2f0 0x56211b0c4090 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.041+0000 7f6ba6c8a740 1 --2- 172.21.15.120:0/1621420737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56211b0bc9d0 0x56211b0d9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.041+0000 7f6ba6c8a740 1 -- 172.21.15.120:0/1621420737 >> 172.21.15.120:0/1621420737 conn(0x56211b0b8f60 msgr2=0x56211b0d75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:10.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.042+0000 7f6ba6c8a740 1 -- 172.21.15.120:0/1621420737 shutdown_connections 2024-09-19T09:02:10.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.042+0000 7f6ba6c8a740 1 -- 172.21.15.120:0/1621420737 wait complete. 2024-09-19T09:02:10.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.042+0000 7f6ba6c8a740 1 -- start start 2024-09-19T09:02:10.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.042+0000 7f6ba6c8a740 1 Processor -- start 2024-09-19T09:02:10.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.042+0000 7f6ba6c8a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56211b0bc9d0 0x56211b0e2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:10.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.042+0000 7f6ba6c8a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56211b0bd2f0 0x56211b0e2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:10.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.042+0000 7f6ba6c8a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211b0c7bb0 0x56211b0db4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:10.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.042+0000 7f6ba6c8a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56211b0d98e0 con 0x56211b0bc9d0 2024-09-19T09:02:10.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.042+0000 7f6ba6c8a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56211b0d9210 con 0x56211b0c7bb0 2024-09-19T09:02:10.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.042+0000 7f6ba6c8a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56211b0d9fb0 con 0x56211b0bd2f0 2024-09-19T09:02:10.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.042+0000 7f6ba6c8a740 10 client.0 _ll_register_callbacks cb 0x7f6b9400c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:02:10.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.042+0000 7f6b9e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56211b0bc9d0 0x56211b0e2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:10.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.042+0000 7f6b9e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56211b0bc9d0 0x56211b0e2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57900/0 (socket says 172.21.15.120:57900) 2024-09-19T09:02:10.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.042+0000 7f6b9e59c640 1 -- 172.21.15.120:0/2277435191 learned_addr learned my addr 172.21.15.120:0/2277435191 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:10.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.042+0000 7f6b9ed9d640 1 --2- 172.21.15.120:0/2277435191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211b0c7bb0 0x56211b0db4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:10.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.042+0000 7f6b9e59c640 1 -- 172.21.15.120:0/2277435191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56211b0bd2f0 msgr2=0x56211b0e2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:10.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.042+0000 7f6b9e59c640 1 --2- 172.21.15.120:0/2277435191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56211b0bd2f0 0x56211b0e2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.042+0000 7f6b9e59c640 1 -- 172.21.15.120:0/2277435191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211b0c7bb0 msgr2=0x56211b0db4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:10.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.042+0000 7f6b9e59c640 1 --2- 172.21.15.120:0/2277435191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211b0c7bb0 0x56211b0db4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.042+0000 7f6b9e59c640 1 -- 172.21.15.120:0/2277435191 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56211b0dbd00 con 0x56211b0bc9d0 2024-09-19T09:02:10.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.043+0000 7f6b9ed9d640 1 --2- 172.21.15.120:0/2277435191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211b0c7bb0 0x56211b0db4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:02:10.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.043+0000 7f6b9e59c640 1 --2- 172.21.15.120:0/2277435191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56211b0bc9d0 0x56211b0e2010 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6b9002a930 tx=0x7f6b9002a9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9893e441cb2b29 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:10.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.043+0000 7f6b85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:02:10.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.043+0000 7f6b85ffb640 1 -- 172.21.15.120:0/2277435191 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9004d070 con 0x56211b0bc9d0 2024-09-19T09:02:10.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.043+0000 7f6b85ffb640 1 -- 172.21.15.120:0/2277435191 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f6b90004450 con 0x56211b0bc9d0 2024-09-19T09:02:10.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.043+0000 7f6ba6c8a740 1 -- 172.21.15.120:0/2277435191 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b940a03b0 con 0x56211b0bc9d0 2024-09-19T09:02:10.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.043+0000 7f6b85ffb640 1 -- 172.21.15.120:0/2277435191 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6b90058050 con 0x56211b0bc9d0 2024-09-19T09:02:10.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.043+0000 7f6ba6c8a740 10 client.15027 fetch_fsmap learned FSMap version 27 2024-09-19T09:02:10.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.043+0000 7f6ba6c8a740 1 -- 172.21.15.120:0/2277435191 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56211b0e0750 con 0x56211b0bc9d0 2024-09-19T09:02:10.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.043+0000 7f6b85ffb640 1 -- 172.21.15.120:0/2277435191 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b940a03b0 con 0x56211b0bc9d0 2024-09-19T09:02:10.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.043+0000 7f6b85ffb640 1 -- 172.21.15.120:0/2277435191 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6b90004cc0 con 0x56211b0bc9d0 2024-09-19T09:02:10.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.043+0000 7f6ba6c8a740 10 client.15027 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:02:10.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.043+0000 7f6ba6c8a740 10 client.15027 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T09:02:10.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.043+0000 7f6ba6c8a740 20 client.15027 populate_metadata read hostname 'smithi120' 2024-09-19T09:02:10.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.043+0000 7f6ba6c8a740 1 --2- 172.21.15.120:0/2277435191 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x56211b147600 0x56211b146310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:10.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.043+0000 7f6ba6c8a740 4 client.15027 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:02:10.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.043+0000 7f6ba6c8a740 1 -- 172.21.15.120:0/2277435191 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "status"}) -- 0x56211b02ab10 con 0x56211b147600 2024-09-19T09:02:10.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.044+0000 7f6b9dd9b640 1 --2- 172.21.15.120:0/2277435191 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x56211b147600 0x56211b146310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:10.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.044+0000 7f6b85ffb640 10 client.15027 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:02:10.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.044+0000 7f6b9dd9b640 1 --2- 172.21.15.120:0/2277435191 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x56211b147600 0x56211b146310 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1d6f64308278275d server_cookie=6ee58ab98458c01e in_seq=0 out_seq=0 2024-09-19T09:02:10.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.044+0000 7f6b85ffb640 1 -- 172.21.15.120:0/2277435191 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56211b02ab10 con 0x56211b147600 2024-09-19T09:02:10.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.044+0000 7f6b85ffb640 10 client.15027 handle_command_reply: tid=0 2024-09-19T09:02:10.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.044+0000 7f6ba6c8a740 1 client.15027 shutdown 2024-09-19T09:02:10.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.044+0000 7f6ba6c8a740 1 -- 172.21.15.120:0/2277435191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56211b0bc9d0 msgr2=0x56211b0e2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:10.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.044+0000 7f6ba6c8a740 1 --2- 172.21.15.120:0/2277435191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56211b0bc9d0 0x56211b0e2010 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6b9002a930 tx=0x7f6b9002a9b0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.044+0000 7f6ba6c8a740 1 -- 172.21.15.120:0/2277435191 shutdown_connections 2024-09-19T09:02:10.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.044+0000 7f6ba6c8a740 1 --2- 172.21.15.120:0/2277435191 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x56211b147600 0x56211b146310 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.044+0000 7f6ba6c8a740 1 --2- 172.21.15.120:0/2277435191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56211b0c7bb0 0x56211b0db4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.044+0000 7f6ba6c8a740 1 --2- 172.21.15.120:0/2277435191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56211b0bd2f0 0x56211b0e2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.044+0000 7f6ba6c8a740 1 --2- 172.21.15.120:0/2277435191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56211b0bc9d0 0x56211b0e2010 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.044+0000 7f6ba6c8a740 1 -- 172.21.15.120:0/2277435191 >> 172.21.15.120:0/2277435191 conn(0x56211b0b8f60 msgr2=0x56211b0d75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:10.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.044+0000 7f6b85ffb640 0 client.15027 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:02:10.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.045+0000 7f6ba6c8a740 1 -- 172.21.15.120:0/2277435191 shutdown_connections 2024-09-19T09:02:10.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.045+0000 7f6ba6c8a740 1 -- 172.21.15.120:0/2277435191 wait complete. 2024-09-19T09:02:10.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.045+0000 7f6ba6c8a740 20 client.15027 trim_cache size 0 max 16384 2024-09-19T09:02:10.061 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:02:10.061 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_fsid": "507393d2-7664-11ef-bceb-c7b262605968", 2024-09-19T09:02:10.061 INFO:teuthology.orchestra.run.smithi120.stdout: "whoami": 1, 2024-09-19T09:02:10.062 INFO:teuthology.orchestra.run.smithi120.stdout: "id": 24446, 2024-09-19T09:02:10.062 INFO:teuthology.orchestra.run.smithi120.stdout: "want_state": "up:active", 2024-09-19T09:02:10.062 INFO:teuthology.orchestra.run.smithi120.stdout: "state": "up:active", 2024-09-19T09:02:10.062 INFO:teuthology.orchestra.run.smithi120.stdout: "fs_name": "cephfs", 2024-09-19T09:02:10.062 INFO:teuthology.orchestra.run.smithi120.stdout: "rank_uptime": 74.944445380999994, 2024-09-19T09:02:10.062 INFO:teuthology.orchestra.run.smithi120.stdout: "mdsmap_epoch": 27, 2024-09-19T09:02:10.062 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch": 81, 2024-09-19T09:02:10.062 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch_barrier": 79, 2024-09-19T09:02:10.062 INFO:teuthology.orchestra.run.smithi120.stdout: "uptime": 104.076128426 2024-09-19T09:02:10.062 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:02:10.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.045+0000 7f6ba48b1640 1 -- 172.21.15.120:0/4054886887 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6b880a32d0 msgr2=0x7f6b880a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:10.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.045+0000 7f6ba48b1640 1 --2- 172.21.15.120:0/4054886887 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6b880a32d0 0x7f6b880a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6b98020000 tx=0x7f6b98021000 comp rx=0 tx=0).stop 2024-09-19T09:02:10.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.045+0000 7f6ba48b1640 1 -- 172.21.15.120:0/4054886887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ba01027a0 msgr2=0x7f6ba0198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:10.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.045+0000 7f6ba48b1640 1 --2- 172.21.15.120:0/4054886887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ba01027a0 0x7f6ba0198980 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400c920 tx=0x7f6b9400cde0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.045+0000 7f6ba48b1640 1 -- 172.21.15.120:0/4054886887 shutdown_connections 2024-09-19T09:02:10.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.045+0000 7f6ba48b1640 1 --2- 172.21.15.120:0/4054886887 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6b880a32d0 0x7f6b880a5790 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.045+0000 7f6ba48b1640 1 --2- 172.21.15.120:0/4054886887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ba0106f20 0x7f6ba019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.045+0000 7f6ba48b1640 1 --2- 172.21.15.120:0/4054886887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ba01027a0 0x7f6ba0198980 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.045+0000 7f6ba48b1640 1 --2- 172.21.15.120:0/4054886887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ba0101df0 0x7f6ba0198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:10.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.045+0000 7f6ba48b1640 1 -- 172.21.15.120:0/4054886887 >> 172.21.15.120:0/4054886887 conn(0x7f6ba00fd8f0 msgr2=0x7f6ba00fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:10.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.045+0000 7f6ba48b1640 1 -- 172.21.15.120:0/4054886887 shutdown_connections 2024-09-19T09:02:10.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:10.045+0000 7f6ba48b1640 1 -- 172.21.15.120:0/4054886887 wait complete. 2024-09-19T09:02:10.786 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T09:02:11.031 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:11.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:10 smithi120 ceph-mon[31150]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T09:02:11.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:10 smithi196 ceph-mon[35150]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T09:02:11.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:10 smithi153 ceph-mon[35071]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T09:02:11.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.530+0000 7f39759ed640 1 -- 172.21.15.120:0/1998850957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3970102570 msgr2=0x7f39701029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:11.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.530+0000 7f39759ed640 1 --2- 172.21.15.120:0/1998850957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3970102570 0x7f39701029b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3958009900 tx=0x7f395802a480 comp rx=0 tx=0).stop 2024-09-19T09:02:11.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.530+0000 7f39759ed640 1 -- 172.21.15.120:0/1998850957 shutdown_connections 2024-09-19T09:02:11.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.530+0000 7f39759ed640 1 --2- 172.21.15.120:0/1998850957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3970102ef0 0x7f397010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.530+0000 7f39759ed640 1 --2- 172.21.15.120:0/1998850957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3970102570 0x7f39701029b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.530+0000 7f39759ed640 1 --2- 172.21.15.120:0/1998850957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3970100e20 0x7f3970101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.530+0000 7f39759ed640 1 -- 172.21.15.120:0/1998850957 >> 172.21.15.120:0/1998850957 conn(0x7f39700fc7d0 msgr2=0x7f39700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:11.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.530+0000 7f39759ed640 1 -- 172.21.15.120:0/1998850957 shutdown_connections 2024-09-19T09:02:11.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.530+0000 7f39759ed640 1 -- 172.21.15.120:0/1998850957 wait complete. 2024-09-19T09:02:11.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.531+0000 7f39759ed640 1 Processor -- start 2024-09-19T09:02:11.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.531+0000 7f39759ed640 1 -- start start 2024-09-19T09:02:11.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.531+0000 7f39759ed640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3970100e20 0x7f39701993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:11.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.531+0000 7f39759ed640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3970102570 0x7f3970199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:11.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.531+0000 7f39759ed640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3970102ef0 0x7f39701a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:11.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.531+0000 7f39759ed640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3970110ff0 con 0x7f3970102570 2024-09-19T09:02:11.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.531+0000 7f39759ed640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3970110e70 con 0x7f3970100e20 2024-09-19T09:02:11.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.531+0000 7f39759ed640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3970111170 con 0x7f3970102ef0 2024-09-19T09:02:11.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.531+0000 7f396e7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3970102570 0x7f3970199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:11.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.531+0000 7f396e7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3970102570 0x7f3970199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43662/0 (socket says 172.21.15.120:43662) 2024-09-19T09:02:11.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.531+0000 7f396e7fc640 1 -- 172.21.15.120:0/3973281929 learned_addr learned my addr 172.21.15.120:0/3973281929 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:11.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.531+0000 7f396effd640 1 --2- 172.21.15.120:0/3973281929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3970100e20 0x7f39701993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:11.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.531+0000 7f396f7fe640 1 --2- 172.21.15.120:0/3973281929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3970102ef0 0x7f39701a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:11.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.532+0000 7f396e7fc640 1 -- 172.21.15.120:0/3973281929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3970102ef0 msgr2=0x7f39701a4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:11.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.532+0000 7f396e7fc640 1 --2- 172.21.15.120:0/3973281929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3970102ef0 0x7f39701a4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.532+0000 7f396e7fc640 1 -- 172.21.15.120:0/3973281929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3970100e20 msgr2=0x7f39701993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:11.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.532+0000 7f396e7fc640 1 --2- 172.21.15.120:0/3973281929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3970100e20 0x7f39701993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.532+0000 7f396e7fc640 1 -- 172.21.15.120:0/3973281929 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39701a4980 con 0x7f3970102570 2024-09-19T09:02:11.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.532+0000 7f396f7fe640 1 --2- 172.21.15.120:0/3973281929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3970102ef0 0x7f39701a4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:02:11.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.532+0000 7f396effd640 1 --2- 172.21.15.120:0/3973281929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3970100e20 0x7f39701993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:02:11.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.532+0000 7f396e7fc640 1 --2- 172.21.15.120:0/3973281929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3970102570 0x7f3970199900 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3958006010 tx=0x7f3958003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:11.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.532+0000 7f39749eb640 1 -- 172.21.15.120:0/3973281929 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395803f070 con 0x7f3970102570 2024-09-19T09:02:11.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.532+0000 7f39749eb640 1 -- 172.21.15.120:0/3973281929 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3958003e30 con 0x7f3970102570 2024-09-19T09:02:11.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.532+0000 7f39759ed640 1 -- 172.21.15.120:0/3973281929 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39701a4b50 con 0x7f3970102570 2024-09-19T09:02:11.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.532+0000 7f39749eb640 1 -- 172.21.15.120:0/3973281929 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3958035030 con 0x7f3970102570 2024-09-19T09:02:11.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.532+0000 7f39759ed640 1 -- 172.21.15.120:0/3973281929 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39701a4fa0 con 0x7f3970102570 2024-09-19T09:02:11.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.534+0000 7f39749eb640 1 -- 172.21.15.120:0/3973281929 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f3958042070 con 0x7f3970102570 2024-09-19T09:02:11.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.535+0000 7f3977dc6740 1 Processor -- start 2024-09-19T09:02:11.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.535+0000 7f3977dc6740 1 -- start start 2024-09-19T09:02:11.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.535+0000 7f3977dc6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5593806318b0 0x559380631c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:11.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.535+0000 7f39749eb640 1 --2- 172.21.15.120:0/3973281929 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f395c0a32d0 0x7f395c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:11.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.535+0000 7f3977dc6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5593806321d0 0x559380640a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:11.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.535+0000 7f3977dc6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559380640fc0 0x5593806433b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:11.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.535+0000 7f3977dc6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559380598090 con 0x5593806321d0 2024-09-19T09:02:11.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.535+0000 7f3977dc6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55938053fa70 con 0x5593806318b0 2024-09-19T09:02:11.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.535+0000 7f3977dc6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559380594050 con 0x559380640fc0 2024-09-19T09:02:11.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.535+0000 7f396effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5593806321d0 0x559380640a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:11.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.535+0000 7f396effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5593806321d0 0x559380640a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43672/0 (socket says 172.21.15.120:43672) 2024-09-19T09:02:11.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.535+0000 7f396effd640 1 -- 172.21.15.120:0/2980972613 learned_addr learned my addr 172.21.15.120:0/2980972613 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:11.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.535+0000 7f39749eb640 1 -- 172.21.15.120:0/3973281929 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f39580f05e0 con 0x7f3970102570 2024-09-19T09:02:11.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.535+0000 7f396e7fc640 1 --2- 172.21.15.120:0/3973281929 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f395c0a32d0 0x7f395c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:11.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.535+0000 7f396f7fe640 1 --2- 172.21.15.120:0/2980972613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5593806318b0 0x559380631c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:11.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.535+0000 7f396f7fe640 1 --2- 172.21.15.120:0/2980972613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559380640fc0 0x5593806433b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:11.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.535+0000 7f396effd640 1 -- 172.21.15.120:0/2980972613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559380640fc0 msgr2=0x5593806433b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:11.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.535+0000 7f396effd640 1 --2- 172.21.15.120:0/2980972613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559380640fc0 0x5593806433b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.535+0000 7f396effd640 1 -- 172.21.15.120:0/2980972613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5593806318b0 msgr2=0x559380631c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:11.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.535+0000 7f396effd640 1 --2- 172.21.15.120:0/2980972613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5593806318b0 0x559380631c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.535+0000 7f396effd640 1 -- 172.21.15.120:0/2980972613 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55938054c130 con 0x5593806321d0 2024-09-19T09:02:11.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.535+0000 7f396f7fe640 1 --2- 172.21.15.120:0/2980972613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5593806318b0 0x559380631c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:02:11.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.536+0000 7f396e7fc640 1 --2- 172.21.15.120:0/3973281929 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f395c0a32d0 0x7f395c0a5790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f39580edf80 tx=0x7f3958036b60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:11.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.536+0000 7f396effd640 1 --2- 172.21.15.120:0/2980972613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5593806321d0 0x559380640a80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f396800af40 tx=0x7f396800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5efa96352f4e33fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:11.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.536+0000 7f3965ffb640 1 -- 172.21.15.120:0/2980972613 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968004d20 con 0x5593806321d0 2024-09-19T09:02:11.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.536+0000 7f3965ffb640 1 -- 172.21.15.120:0/2980972613 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3968011b30 con 0x5593806321d0 2024-09-19T09:02:11.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.536+0000 7f3965ffb640 1 -- 172.21.15.120:0/2980972613 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968005700 con 0x5593806321d0 2024-09-19T09:02:11.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.536+0000 7f3977dc6740 1 -- 172.21.15.120:0/2980972613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5593806321d0 msgr2=0x559380640a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:11.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.536+0000 7f3977dc6740 1 --2- 172.21.15.120:0/2980972613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5593806321d0 0x559380640a80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f396800af40 tx=0x7f396800f7f0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.536+0000 7f3977dc6740 1 -- 172.21.15.120:0/2980972613 shutdown_connections 2024-09-19T09:02:11.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.536+0000 7f3977dc6740 1 --2- 172.21.15.120:0/2980972613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559380640fc0 0x5593806433b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.536+0000 7f3977dc6740 1 --2- 172.21.15.120:0/2980972613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5593806321d0 0x559380640a80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.536+0000 7f3977dc6740 1 --2- 172.21.15.120:0/2980972613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5593806318b0 0x559380631c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.536+0000 7f3977dc6740 1 -- 172.21.15.120:0/2980972613 >> 172.21.15.120:0/2980972613 conn(0x55938062df60 msgr2=0x55938062e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:11.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.537+0000 7f3977dc6740 1 -- 172.21.15.120:0/2980972613 shutdown_connections 2024-09-19T09:02:11.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.537+0000 7f3977dc6740 1 -- 172.21.15.120:0/2980972613 wait complete. 2024-09-19T09:02:11.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.537+0000 7f3977dc6740 1 -- start start 2024-09-19T09:02:11.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.537+0000 7f3977dc6740 1 Processor -- start 2024-09-19T09:02:11.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.537+0000 7f3977dc6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5593806318b0 0x559380640800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:11.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.537+0000 7f3977dc6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593806321d0 0x55938063dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:11.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.537+0000 7f3977dc6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559380640fc0 0x55938063e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:11.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.537+0000 7f3977dc6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559380643b90 con 0x559380640fc0 2024-09-19T09:02:11.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.537+0000 7f3977dc6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559380552f20 con 0x5593806318b0 2024-09-19T09:02:11.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.537+0000 7f3977dc6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559380643d10 con 0x5593806321d0 2024-09-19T09:02:11.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.537+0000 7f3977dc6740 10 client.0 _ll_register_callbacks cb 0x55938051c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:02:11.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.538+0000 7f396f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559380640fc0 0x55938063e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:11.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.538+0000 7f396f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559380640fc0 0x55938063e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43686/0 (socket says 172.21.15.120:43686) 2024-09-19T09:02:11.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.538+0000 7f396f7fe640 1 -- 172.21.15.120:0/931189997 learned_addr learned my addr 172.21.15.120:0/931189997 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:11.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.538+0000 7f396f7fe640 1 --2- 172.21.15.120:0/931189997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5593806318b0 0x559380640800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:11.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.538+0000 7f396effd640 1 --2- 172.21.15.120:0/931189997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593806321d0 0x55938063dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:11.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.538+0000 7f396f7fe640 1 -- 172.21.15.120:0/931189997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593806321d0 msgr2=0x55938063dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:11.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.538+0000 7f396f7fe640 1 --2- 172.21.15.120:0/931189997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593806321d0 0x55938063dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.538+0000 7f396f7fe640 1 -- 172.21.15.120:0/931189997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5593806318b0 msgr2=0x559380640800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:11.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.538+0000 7f396f7fe640 1 --2- 172.21.15.120:0/931189997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5593806318b0 0x559380640800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.538+0000 7f396f7fe640 1 -- 172.21.15.120:0/931189997 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593803092b0 con 0x559380640fc0 2024-09-19T09:02:11.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.538+0000 7f396effd640 1 --2- 172.21.15.120:0/931189997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593806321d0 0x55938063dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:02:11.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.538+0000 7f396f7fe640 1 --2- 172.21.15.120:0/931189997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559380640fc0 0x55938063e0e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f396000c320 tx=0x7f3960008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ddce82d6cab2868 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:11.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.538+0000 7f394effd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:02:11.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.538+0000 7f394effd640 1 -- 172.21.15.120:0/931189997 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396001c020 con 0x559380640fc0 2024-09-19T09:02:11.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.538+0000 7f394effd640 1 -- 172.21.15.120:0/931189997 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f3960012b80 con 0x559380640fc0 2024-09-19T09:02:11.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.538+0000 7f3977dc6740 1 -- 172.21.15.120:0/931189997 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3968011b30 con 0x559380640fc0 2024-09-19T09:02:11.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.538+0000 7f394effd640 1 -- 172.21.15.120:0/931189997 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f396000f040 con 0x559380640fc0 2024-09-19T09:02:11.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.538+0000 7f3977dc6740 10 client.15039 fetch_fsmap learned FSMap version 27 2024-09-19T09:02:11.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.538+0000 7f3977dc6740 1 -- 172.21.15.120:0/931189997 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55938064f570 con 0x559380640fc0 2024-09-19T09:02:11.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.538+0000 7f394effd640 1 -- 172.21.15.120:0/931189997 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3968011b30 con 0x559380640fc0 2024-09-19T09:02:11.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.539+0000 7f394effd640 1 -- 172.21.15.120:0/931189997 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3960015040 con 0x559380640fc0 2024-09-19T09:02:11.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.539+0000 7f3977dc6740 10 client.15039 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:02:11.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.539+0000 7f3977dc6740 10 client.15039 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T09:02:11.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.539+0000 7f3977dc6740 20 client.15039 populate_metadata read hostname 'smithi120' 2024-09-19T09:02:11.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.539+0000 7f3977dc6740 1 --2- 172.21.15.120:0/931189997 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559380651c20 0x559380653090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:11.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.539+0000 7f3977dc6740 4 client.15039 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:02:11.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.539+0000 7f3977dc6740 1 -- 172.21.15.120:0/931189997 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593805a5600 con 0x559380651c20 2024-09-19T09:02:11.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.539+0000 7f396effd640 1 --2- 172.21.15.120:0/931189997 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559380651c20 0x559380653090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:11.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.539+0000 7f394effd640 10 client.15039 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:02:11.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.539+0000 7f396effd640 1 --2- 172.21.15.120:0/931189997 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559380651c20 0x559380653090 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18c75a392ccbdc31 server_cookie=66ff777f0000399b in_seq=0 out_seq=0 2024-09-19T09:02:11.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.540+0000 7f394effd640 1 -- 172.21.15.120:0/931189997 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5593805a5600 con 0x559380651c20 2024-09-19T09:02:11.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.540+0000 7f394effd640 10 client.15039 handle_command_reply: tid=0 2024-09-19T09:02:11.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.540+0000 7f3977dc6740 1 client.15039 shutdown 2024-09-19T09:02:11.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.540+0000 7f3977dc6740 1 -- 172.21.15.120:0/931189997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559380640fc0 msgr2=0x55938063e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:11.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.540+0000 7f3977dc6740 1 --2- 172.21.15.120:0/931189997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559380640fc0 0x55938063e0e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f396000c320 tx=0x7f3960008040 comp rx=0 tx=0).stop 2024-09-19T09:02:11.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.540+0000 7f3977dc6740 1 -- 172.21.15.120:0/931189997 shutdown_connections 2024-09-19T09:02:11.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.540+0000 7f3977dc6740 1 --2- 172.21.15.120:0/931189997 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559380651c20 0x559380653090 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.540+0000 7f3977dc6740 1 --2- 172.21.15.120:0/931189997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559380640fc0 0x55938063e0e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.540+0000 7f3977dc6740 1 --2- 172.21.15.120:0/931189997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593806321d0 0x55938063dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.540+0000 7f3977dc6740 1 --2- 172.21.15.120:0/931189997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5593806318b0 0x559380640800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.540+0000 7f394effd640 0 client.15039 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:02:11.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.540+0000 7f3977dc6740 1 -- 172.21.15.120:0/931189997 >> 172.21.15.120:0/931189997 conn(0x55938062df60 msgr2=0x559380633cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:11.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.541+0000 7f3977dc6740 1 -- 172.21.15.120:0/931189997 shutdown_connections 2024-09-19T09:02:11.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.541+0000 7f3977dc6740 1 -- 172.21.15.120:0/931189997 wait complete. 2024-09-19T09:02:11.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.541+0000 7f3977dc6740 20 client.15039 trim_cache size 0 max 16384 2024-09-19T09:02:11.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.558+0000 7f3977dc6740 1 Processor -- start 2024-09-19T09:02:11.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.558+0000 7f3977dc6740 1 -- start start 2024-09-19T09:02:11.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.558+0000 7f3977dc6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5593806318b0 0x559380651550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:11.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.558+0000 7f3977dc6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5593806321d0 0x55938063c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:11.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.558+0000 7f3977dc6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559380640fc0 0x55938063c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:11.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.558+0000 7f3977dc6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5593806513d0 con 0x5593806321d0 2024-09-19T09:02:11.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.558+0000 7f3977dc6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559380651aa0 con 0x5593806318b0 2024-09-19T09:02:11.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.558+0000 7f3977dc6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559380652170 con 0x559380640fc0 2024-09-19T09:02:11.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.559+0000 7f396effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5593806321d0 0x55938063c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:11.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.559+0000 7f396effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5593806321d0 0x55938063c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43698/0 (socket says 172.21.15.120:43698) 2024-09-19T09:02:11.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.559+0000 7f396effd640 1 -- 172.21.15.120:0/2473737543 learned_addr learned my addr 172.21.15.120:0/2473737543 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:11.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.559+0000 7f396f7fe640 1 --2- 172.21.15.120:0/2473737543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559380640fc0 0x55938063c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:11.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.559+0000 7f396effd640 1 -- 172.21.15.120:0/2473737543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559380640fc0 msgr2=0x55938063c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:11.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.559+0000 7f396effd640 1 --2- 172.21.15.120:0/2473737543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559380640fc0 0x55938063c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.559+0000 7f396effd640 1 -- 172.21.15.120:0/2473737543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5593806318b0 msgr2=0x559380651550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:11.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.559+0000 7f396effd640 1 --2- 172.21.15.120:0/2473737543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5593806318b0 0x559380651550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.559+0000 7f396effd640 1 -- 172.21.15.120:0/2473737543 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559380632000 con 0x5593806321d0 2024-09-19T09:02:11.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.559+0000 7f396f7fe640 1 --2- 172.21.15.120:0/2473737543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559380640fc0 0x55938063c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:02:11.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.559+0000 7f396effd640 1 --2- 172.21.15.120:0/2473737543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5593806321d0 0x55938063c470 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3968005310 tx=0x7f3968005340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595cd0bc53a62e40 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:11.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.559+0000 7f39657fa640 1 -- 172.21.15.120:0/2473737543 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968005470 con 0x5593806321d0 2024-09-19T09:02:11.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.559+0000 7f39657fa640 1 -- 172.21.15.120:0/2473737543 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39680049e0 con 0x5593806321d0 2024-09-19T09:02:11.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.559+0000 7f39657fa640 1 -- 172.21.15.120:0/2473737543 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968019670 con 0x5593806321d0 2024-09-19T09:02:11.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.560+0000 7f3977dc6740 1 -- 172.21.15.120:0/2473737543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5593806321d0 msgr2=0x55938063c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:11.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.560+0000 7f3977dc6740 1 --2- 172.21.15.120:0/2473737543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5593806321d0 0x55938063c470 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3968005310 tx=0x7f3968005340 comp rx=0 tx=0).stop 2024-09-19T09:02:11.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.560+0000 7f3977dc6740 1 -- 172.21.15.120:0/2473737543 shutdown_connections 2024-09-19T09:02:11.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.560+0000 7f3977dc6740 1 --2- 172.21.15.120:0/2473737543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559380640fc0 0x55938063c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.560+0000 7f3977dc6740 1 --2- 172.21.15.120:0/2473737543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5593806321d0 0x55938063c470 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.560+0000 7f3977dc6740 1 --2- 172.21.15.120:0/2473737543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5593806318b0 0x559380651550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.560+0000 7f3977dc6740 1 -- 172.21.15.120:0/2473737543 >> 172.21.15.120:0/2473737543 conn(0x55938062df60 msgr2=0x559380628ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:11.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.560+0000 7f3977dc6740 1 -- 172.21.15.120:0/2473737543 shutdown_connections 2024-09-19T09:02:11.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.560+0000 7f3977dc6740 1 -- 172.21.15.120:0/2473737543 wait complete. 2024-09-19T09:02:11.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.560+0000 7f3977dc6740 1 -- start start 2024-09-19T09:02:11.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.560+0000 7f3977dc6740 1 Processor -- start 2024-09-19T09:02:11.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.561+0000 7f3977dc6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5593806318b0 0x55938063a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:11.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.561+0000 7f3977dc6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5593806321d0 0x55938063aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:11.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.561+0000 7f3977dc6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559380640fc0 0x55938065a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:11.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.561+0000 7f3977dc6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559380652f10 con 0x5593806321d0 2024-09-19T09:02:11.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.561+0000 7f3977dc6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559380652840 con 0x5593806318b0 2024-09-19T09:02:11.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.561+0000 7f3977dc6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5593806535e0 con 0x559380640fc0 2024-09-19T09:02:11.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.561+0000 7f3977dc6740 10 client.0 _ll_register_callbacks cb 0x7f396000c650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:02:11.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.561+0000 7f396effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5593806321d0 0x55938063aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:11.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.561+0000 7f396effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5593806321d0 0x55938063aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43708/0 (socket says 172.21.15.120:43708) 2024-09-19T09:02:11.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.561+0000 7f396f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5593806318b0 0x55938063a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:11.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.561+0000 7f396effd640 1 -- 172.21.15.120:0/1869137628 learned_addr learned my addr 172.21.15.120:0/1869137628 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:11.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.561+0000 7f396f7fe640 1 --2- 172.21.15.120:0/1869137628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559380640fc0 0x55938065a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:11.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.561+0000 7f396effd640 1 -- 172.21.15.120:0/1869137628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559380640fc0 msgr2=0x55938065a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:11.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.561+0000 7f396effd640 1 --2- 172.21.15.120:0/1869137628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559380640fc0 0x55938065a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.561+0000 7f396effd640 1 -- 172.21.15.120:0/1869137628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5593806318b0 msgr2=0x55938063a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:11.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.561+0000 7f396effd640 1 --2- 172.21.15.120:0/1869137628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5593806318b0 0x55938063a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.561+0000 7f396effd640 1 -- 172.21.15.120:0/1869137628 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55938065a730 con 0x5593806321d0 2024-09-19T09:02:11.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.561+0000 7f396effd640 1 --2- 172.21.15.120:0/1869137628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5593806321d0 0x55938063aad0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f396800af10 tx=0x7f3968011650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38092c38e483282 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:11.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.561+0000 7f394effd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:02:11.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.561+0000 7f394effd640 1 -- 172.21.15.120:0/1869137628 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968003b70 con 0x5593806321d0 2024-09-19T09:02:11.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.561+0000 7f394effd640 1 -- 172.21.15.120:0/1869137628 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f3968003d10 con 0x5593806321d0 2024-09-19T09:02:11.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.561+0000 7f3977dc6740 1 -- 172.21.15.120:0/1869137628 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39680049e0 con 0x5593806321d0 2024-09-19T09:02:11.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.562+0000 7f394effd640 1 -- 172.21.15.120:0/1869137628 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3968005440 con 0x5593806321d0 2024-09-19T09:02:11.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.562+0000 7f3977dc6740 10 client.15045 fetch_fsmap learned FSMap version 27 2024-09-19T09:02:11.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.562+0000 7f3977dc6740 1 -- 172.21.15.120:0/1869137628 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55938065b960 con 0x5593806321d0 2024-09-19T09:02:11.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.562+0000 7f394effd640 1 -- 172.21.15.120:0/1869137628 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39680049e0 con 0x5593806321d0 2024-09-19T09:02:11.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.562+0000 7f394effd640 1 -- 172.21.15.120:0/1869137628 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3968011ce0 con 0x5593806321d0 2024-09-19T09:02:11.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.562+0000 7f3977dc6740 10 client.15045 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:02:11.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.562+0000 7f3977dc6740 10 client.15045 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T09:02:11.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.562+0000 7f3977dc6740 20 client.15045 populate_metadata read hostname 'smithi120' 2024-09-19T09:02:11.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.562+0000 7f3977dc6740 1 --2- 172.21.15.120:0/1869137628 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5593806c00f0 0x5593806bee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:11.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.562+0000 7f3977dc6740 4 client.15045 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:02:11.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.562+0000 7f3977dc6740 1 -- 172.21.15.120:0/1869137628 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "status"}) -- 0x55938059fb10 con 0x5593806c00f0 2024-09-19T09:02:11.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.562+0000 7f396f7fe640 1 --2- 172.21.15.120:0/1869137628 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5593806c00f0 0x5593806bee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:11.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.563+0000 7f396f7fe640 1 --2- 172.21.15.120:0/1869137628 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5593806c00f0 0x5593806bee00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6efa035a03ff8bb server_cookie=be3d34b44fac6a01 in_seq=0 out_seq=0 2024-09-19T09:02:11.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.563+0000 7f394effd640 10 client.15045 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:02:11.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.563+0000 7f394effd640 1 -- 172.21.15.120:0/1869137628 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55938059fb10 con 0x5593806c00f0 2024-09-19T09:02:11.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.563+0000 7f394effd640 10 client.15045 handle_command_reply: tid=0 2024-09-19T09:02:11.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.563+0000 7f3977dc6740 1 client.15045 shutdown 2024-09-19T09:02:11.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.563+0000 7f3977dc6740 1 -- 172.21.15.120:0/1869137628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5593806321d0 msgr2=0x55938063aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:11.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.563+0000 7f3977dc6740 1 --2- 172.21.15.120:0/1869137628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5593806321d0 0x55938063aad0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f396800af10 tx=0x7f3968011650 comp rx=0 tx=0).stop 2024-09-19T09:02:11.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.563+0000 7f3977dc6740 1 -- 172.21.15.120:0/1869137628 shutdown_connections 2024-09-19T09:02:11.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.563+0000 7f3977dc6740 1 --2- 172.21.15.120:0/1869137628 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5593806c00f0 0x5593806bee00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.563+0000 7f3977dc6740 1 --2- 172.21.15.120:0/1869137628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559380640fc0 0x55938065a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.563+0000 7f3977dc6740 1 --2- 172.21.15.120:0/1869137628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5593806321d0 0x55938063aad0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.563+0000 7f3977dc6740 1 --2- 172.21.15.120:0/1869137628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5593806318b0 0x55938063a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.563+0000 7f3977dc6740 1 -- 172.21.15.120:0/1869137628 >> 172.21.15.120:0/1869137628 conn(0x55938062df60 msgr2=0x5593806512e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:11.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.563+0000 7f394effd640 0 client.15045 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:02:11.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.563+0000 7f3977dc6740 1 -- 172.21.15.120:0/1869137628 shutdown_connections 2024-09-19T09:02:11.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.563+0000 7f3977dc6740 1 -- 172.21.15.120:0/1869137628 wait complete. 2024-09-19T09:02:11.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.563+0000 7f3977dc6740 20 client.15045 trim_cache size 0 max 16384 2024-09-19T09:02:11.570 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:02:11.570 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_fsid": "507393d2-7664-11ef-bceb-c7b262605968", 2024-09-19T09:02:11.570 INFO:teuthology.orchestra.run.smithi120.stdout: "whoami": 0, 2024-09-19T09:02:11.570 INFO:teuthology.orchestra.run.smithi120.stdout: "id": 24461, 2024-09-19T09:02:11.570 INFO:teuthology.orchestra.run.smithi120.stdout: "want_state": "up:active", 2024-09-19T09:02:11.570 INFO:teuthology.orchestra.run.smithi120.stdout: "state": "up:active", 2024-09-19T09:02:11.570 INFO:teuthology.orchestra.run.smithi120.stdout: "fs_name": "cephfs", 2024-09-19T09:02:11.570 INFO:teuthology.orchestra.run.smithi120.stdout: "rank_uptime": 83.606919954000006, 2024-09-19T09:02:11.570 INFO:teuthology.orchestra.run.smithi120.stdout: "mdsmap_epoch": 27, 2024-09-19T09:02:11.570 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch": 81, 2024-09-19T09:02:11.570 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch_barrier": 79, 2024-09-19T09:02:11.570 INFO:teuthology.orchestra.run.smithi120.stdout: "uptime": 93.631254185000003 2024-09-19T09:02:11.571 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:02:11.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.564+0000 7f39759ed640 1 -- 172.21.15.120:0/3973281929 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f395c0a32d0 msgr2=0x7f395c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:11.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.564+0000 7f39759ed640 1 --2- 172.21.15.120:0/3973281929 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f395c0a32d0 0x7f395c0a5790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f39580edf80 tx=0x7f3958036b60 comp rx=0 tx=0).stop 2024-09-19T09:02:11.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.564+0000 7f39759ed640 1 -- 172.21.15.120:0/3973281929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3970102570 msgr2=0x7f3970199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:11.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.564+0000 7f39759ed640 1 --2- 172.21.15.120:0/3973281929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3970102570 0x7f3970199900 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3958006010 tx=0x7f3958003c10 comp rx=0 tx=0).stop 2024-09-19T09:02:11.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.564+0000 7f39759ed640 1 -- 172.21.15.120:0/3973281929 shutdown_connections 2024-09-19T09:02:11.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.564+0000 7f39759ed640 1 --2- 172.21.15.120:0/3973281929 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f395c0a32d0 0x7f395c0a5790 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.564+0000 7f39759ed640 1 --2- 172.21.15.120:0/3973281929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3970102ef0 0x7f39701a4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.564+0000 7f39759ed640 1 --2- 172.21.15.120:0/3973281929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3970102570 0x7f3970199900 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.564+0000 7f39759ed640 1 --2- 172.21.15.120:0/3973281929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3970100e20 0x7f39701993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:11.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.564+0000 7f39759ed640 1 -- 172.21.15.120:0/3973281929 >> 172.21.15.120:0/3973281929 conn(0x7f39700fc7d0 msgr2=0x7f39701068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:11.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.564+0000 7f39759ed640 1 -- 172.21.15.120:0/3973281929 shutdown_connections 2024-09-19T09:02:11.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:11.564+0000 7f39759ed640 1 -- 172.21.15.120:0/3973281929 wait complete. 2024-09-19T09:02:12.374 INFO:tasks.cephadm:Setup complete, yielding 2024-09-19T09:02:12.374 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T09:02:12.384 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi120.front.sepia.ceph.com 2024-09-19T09:02:12.385 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-19T09:02:12.630 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:13.063 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:12 smithi120 ceph-mon[31150]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T09:02:13.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.144+0000 7f9f4b0a0640 1 -- 172.21.15.120:0/3998242675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f441026d0 msgr2=0x7f9f44108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:13.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.144+0000 7f9f4b0a0640 1 --2- 172.21.15.120:0/3998242675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f441026d0 0x7f9f44108f60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009900 tx=0x7f9f3c02a410 comp rx=0 tx=0).stop 2024-09-19T09:02:13.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.144+0000 7f9f4b0a0640 1 -- 172.21.15.120:0/3998242675 shutdown_connections 2024-09-19T09:02:13.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.144+0000 7f9f4b0a0640 1 --2- 172.21.15.120:0/3998242675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f441026d0 0x7f9f44108f60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:13.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.144+0000 7f9f4b0a0640 1 --2- 172.21.15.120:0/3998242675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f44101d10 0x7f9f44102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:13.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.144+0000 7f9f4b0a0640 1 --2- 172.21.15.120:0/3998242675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f44100b10 0x7f9f44100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:13.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.144+0000 7f9f4b0a0640 1 -- 172.21.15.120:0/3998242675 >> 172.21.15.120:0/3998242675 conn(0x7f9f440fc2a0 msgr2=0x7f9f440fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:13.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.144+0000 7f9f4b0a0640 1 -- 172.21.15.120:0/3998242675 shutdown_connections 2024-09-19T09:02:13.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.144+0000 7f9f4b0a0640 1 -- 172.21.15.120:0/3998242675 wait complete. 2024-09-19T09:02:13.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.145+0000 7f9f4b0a0640 1 Processor -- start 2024-09-19T09:02:13.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.145+0000 7f9f4b0a0640 1 -- start start 2024-09-19T09:02:13.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.145+0000 7f9f4b0a0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f44100b10 0x7f9f44198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:13.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.145+0000 7f9f4b0a0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f44101d10 0x7f9f44198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:13.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.145+0000 7f9f4b0a0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f441026d0 0x7f9f4419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:13.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.145+0000 7f9f4b0a0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4410d9a0 con 0x7f9f441026d0 2024-09-19T09:02:13.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.145+0000 7f9f4b0a0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4410d820 con 0x7f9f44101d10 2024-09-19T09:02:13.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.145+0000 7f9f4b0a0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4410db20 con 0x7f9f44100b10 2024-09-19T09:02:13.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.145+0000 7f9f4963d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f441026d0 0x7f9f4419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:13.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.145+0000 7f9f4963d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f441026d0 0x7f9f4419f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43730/0 (socket says 172.21.15.120:43730) 2024-09-19T09:02:13.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.145+0000 7f9f4963d640 1 -- 172.21.15.120:0/1315256311 learned_addr learned my addr 172.21.15.120:0/1315256311 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:13.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.145+0000 7f9f48e3c640 1 --2- 172.21.15.120:0/1315256311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f44100b10 0x7f9f44198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:13.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.145+0000 7f9f43fff640 1 --2- 172.21.15.120:0/1315256311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f44101d10 0x7f9f44198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:13.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.145+0000 7f9f4963d640 1 -- 172.21.15.120:0/1315256311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f44100b10 msgr2=0x7f9f44198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:13.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.145+0000 7f9f4963d640 1 --2- 172.21.15.120:0/1315256311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f44100b10 0x7f9f44198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:13.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.145+0000 7f9f4963d640 1 -- 172.21.15.120:0/1315256311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f44101d10 msgr2=0x7f9f44198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:13.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.145+0000 7f9f4963d640 1 --2- 172.21.15.120:0/1315256311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f44101d10 0x7f9f44198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:13.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.145+0000 7f9f4963d640 1 -- 172.21.15.120:0/1315256311 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4419fc70 con 0x7f9f441026d0 2024-09-19T09:02:13.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.146+0000 7f9f4963d640 1 --2- 172.21.15.120:0/1315256311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f441026d0 0x7f9f4419f600 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0314e0 tx=0x7f9f3c0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:13.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.146+0000 7f9f41ffb640 1 -- 172.21.15.120:0/1315256311 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c03f030 con 0x7f9f441026d0 2024-09-19T09:02:13.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.146+0000 7f9f41ffb640 1 -- 172.21.15.120:0/1315256311 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f3c003ce0 con 0x7f9f441026d0 2024-09-19T09:02:13.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.146+0000 7f9f4b0a0640 1 -- 172.21.15.120:0/1315256311 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f4419ff00 con 0x7f9f441026d0 2024-09-19T09:02:13.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.146+0000 7f9f41ffb640 1 -- 172.21.15.120:0/1315256311 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c035070 con 0x7f9f441026d0 2024-09-19T09:02:13.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.146+0000 7f9f4b0a0640 1 -- 172.21.15.120:0/1315256311 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f441a03d0 con 0x7f9f441026d0 2024-09-19T09:02:13.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.147+0000 7f9f4b0a0640 1 -- 172.21.15.120:0/1315256311 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f44107f10 con 0x7f9f441026d0 2024-09-19T09:02:13.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.148+0000 7f9f41ffb640 1 -- 172.21.15.120:0/1315256311 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f9f3c042030 con 0x7f9f441026d0 2024-09-19T09:02:13.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.149+0000 7f9f41ffb640 1 --2- 172.21.15.120:0/1315256311 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9f300a3300 0x7f9f300a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:13.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.149+0000 7f9f48e3c640 1 --2- 172.21.15.120:0/1315256311 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9f300a3300 0x7f9f300a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:13.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.149+0000 7f9f41ffb640 1 -- 172.21.15.120:0/1315256311 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f9f3c0f05b0 con 0x7f9f441026d0 2024-09-19T09:02:13.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.150+0000 7f9f48e3c640 1 --2- 172.21.15.120:0/1315256311 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9f300a3300 0x7f9f300a57c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9f44101b70 tx=0x7f9f2c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:13.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.153+0000 7f9f41ffb640 1 -- 172.21.15.120:0/1315256311 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9f3c0bd130 con 0x7f9f441026d0 2024-09-19T09:02:13.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:12 smithi196 ceph-mon[35150]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T09:02:13.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:12 smithi153 ceph-mon[35071]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T09:02:13.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.338+0000 7f9f4b0a0640 1 -- 172.21.15.120:0/1315256311 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f9f44062630 con 0x7f9f300a3300 2024-09-19T09:02:13.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.339+0000 7f9f41ffb640 1 -- 172.21.15.120:0/1315256311 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f9f44062630 con 0x7f9f300a3300 2024-09-19T09:02:13.339 INFO:teuthology.orchestra.run.smithi120.stdout:Backend: cephadm 2024-09-19T09:02:13.339 INFO:teuthology.orchestra.run.smithi120.stdout:Available: Yes 2024-09-19T09:02:13.339 INFO:teuthology.orchestra.run.smithi120.stdout:Paused: No 2024-09-19T09:02:13.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.341+0000 7f9f4b0a0640 1 -- 172.21.15.120:0/1315256311 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9f300a3300 msgr2=0x7f9f300a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:13.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.341+0000 7f9f4b0a0640 1 --2- 172.21.15.120:0/1315256311 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9f300a3300 0x7f9f300a57c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9f44101b70 tx=0x7f9f2c009110 comp rx=0 tx=0).stop 2024-09-19T09:02:13.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.341+0000 7f9f4b0a0640 1 -- 172.21.15.120:0/1315256311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f441026d0 msgr2=0x7f9f4419f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:13.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.341+0000 7f9f4b0a0640 1 --2- 172.21.15.120:0/1315256311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f441026d0 0x7f9f4419f600 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0314e0 tx=0x7f9f3c0043c0 comp rx=0 tx=0).stop 2024-09-19T09:02:13.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.341+0000 7f9f4b0a0640 1 -- 172.21.15.120:0/1315256311 shutdown_connections 2024-09-19T09:02:13.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.341+0000 7f9f4b0a0640 1 --2- 172.21.15.120:0/1315256311 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9f300a3300 0x7f9f300a57c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:13.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.341+0000 7f9f4b0a0640 1 --2- 172.21.15.120:0/1315256311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f441026d0 0x7f9f4419f600 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:13.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.341+0000 7f9f4b0a0640 1 --2- 172.21.15.120:0/1315256311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f44101d10 0x7f9f44198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:13.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.341+0000 7f9f4b0a0640 1 --2- 172.21.15.120:0/1315256311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f44100b10 0x7f9f44198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:13.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.341+0000 7f9f4b0a0640 1 -- 172.21.15.120:0/1315256311 >> 172.21.15.120:0/1315256311 conn(0x7f9f440fc2a0 msgr2=0x7f9f441050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:13.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.341+0000 7f9f4b0a0640 1 -- 172.21.15.120:0/1315256311 shutdown_connections 2024-09-19T09:02:13.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:13.341+0000 7f9f4b0a0640 1 -- 172.21.15.120:0/1315256311 wait complete. 2024-09-19T09:02:14.040 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-19T09:02:14.284 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:14.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.728+0000 7fe554223640 1 -- 172.21.15.120:0/1080183669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe54c102e00 msgr2=0x7fe54c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:14.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.729+0000 7fe554223640 1 --2- 172.21.15.120:0/1080183669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe54c102e00 0x7fe54c103280 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe548009900 tx=0x7fe54802a410 comp rx=0 tx=0).stop 2024-09-19T09:02:14.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.729+0000 7fe554223640 1 -- 172.21.15.120:0/1080183669 shutdown_connections 2024-09-19T09:02:14.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.729+0000 7fe554223640 1 --2- 172.21.15.120:0/1080183669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe54c1037c0 0x7fe54c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:14.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.729+0000 7fe554223640 1 --2- 172.21.15.120:0/1080183669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe54c102e00 0x7fe54c103280 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:14.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.729+0000 7fe554223640 1 --2- 172.21.15.120:0/1080183669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe54c101c00 0x7fe54c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:14.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.729+0000 7fe554223640 1 -- 172.21.15.120:0/1080183669 >> 172.21.15.120:0/1080183669 conn(0x7fe54c0fd390 msgr2=0x7fe54c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:14.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.729+0000 7fe554223640 1 -- 172.21.15.120:0/1080183669 shutdown_connections 2024-09-19T09:02:14.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.729+0000 7fe554223640 1 -- 172.21.15.120:0/1080183669 wait complete. 2024-09-19T09:02:14.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.729+0000 7fe554223640 1 Processor -- start 2024-09-19T09:02:14.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.729+0000 7fe554223640 1 -- start start 2024-09-19T09:02:14.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.729+0000 7fe554223640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe54c101c00 0x7fe54c197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:14.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.729+0000 7fe554223640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe54c102e00 0x7fe54c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:14.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.729+0000 7fe554223640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe54c1037c0 0x7fe54c19f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:14.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.729+0000 7fe554223640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c10d930 con 0x7fe54c101c00 2024-09-19T09:02:14.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.729+0000 7fe554223640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c10d7b0 con 0x7fe54c1037c0 2024-09-19T09:02:14.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.729+0000 7fe554223640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c10dab0 con 0x7fe54c102e00 2024-09-19T09:02:14.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.730+0000 7fe551fbf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe54c101c00 0x7fe54c197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:14.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.730+0000 7fe551fbf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe54c101c00 0x7fe54c197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43744/0 (socket says 172.21.15.120:43744) 2024-09-19T09:02:14.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.730+0000 7fe551fbf640 1 -- 172.21.15.120:0/3053053749 learned_addr learned my addr 172.21.15.120:0/3053053749 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:14.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.730+0000 7fe5517be640 1 --2- 172.21.15.120:0/3053053749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe54c102e00 0x7fe54c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:14.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.730+0000 7fe5527c0640 1 --2- 172.21.15.120:0/3053053749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe54c1037c0 0x7fe54c19f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:14.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.730+0000 7fe551fbf640 1 -- 172.21.15.120:0/3053053749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe54c102e00 msgr2=0x7fe54c1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:14.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.730+0000 7fe551fbf640 1 --2- 172.21.15.120:0/3053053749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe54c102e00 0x7fe54c1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:14.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.730+0000 7fe551fbf640 1 -- 172.21.15.120:0/3053053749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe54c1037c0 msgr2=0x7fe54c19f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:14.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.730+0000 7fe551fbf640 1 --2- 172.21.15.120:0/3053053749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe54c1037c0 0x7fe54c19f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:14.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.730+0000 7fe551fbf640 1 -- 172.21.15.120:0/3053053749 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54c19fbb0 con 0x7fe54c101c00 2024-09-19T09:02:14.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.730+0000 7fe551fbf640 1 --2- 172.21.15.120:0/3053053749 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe54c101c00 0x7fe54c197f80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe54000b450 tx=0x7fe54000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:14.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.730+0000 7fe53effd640 1 -- 172.21.15.120:0/3053053749 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe540015020 con 0x7fe54c101c00 2024-09-19T09:02:14.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.730+0000 7fe53effd640 1 -- 172.21.15.120:0/3053053749 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe540011d70 con 0x7fe54c101c00 2024-09-19T09:02:14.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.730+0000 7fe554223640 1 -- 172.21.15.120:0/3053053749 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54c19fea0 con 0x7fe54c101c00 2024-09-19T09:02:14.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.730+0000 7fe53effd640 1 -- 172.21.15.120:0/3053053749 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54000f340 con 0x7fe54c101c00 2024-09-19T09:02:14.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.730+0000 7fe554223640 1 -- 172.21.15.120:0/3053053749 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe54c1a0380 con 0x7fe54c101c00 2024-09-19T09:02:14.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.731+0000 7fe554223640 1 -- 172.21.15.120:0/3053053749 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe54c108ff0 con 0x7fe54c101c00 2024-09-19T09:02:14.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.732+0000 7fe53effd640 1 -- 172.21.15.120:0/3053053749 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe540018020 con 0x7fe54c101c00 2024-09-19T09:02:14.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.733+0000 7fe53effd640 1 --2- 172.21.15.120:0/3053053749 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5380a3380 0x7fe5380a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:14.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.733+0000 7fe5517be640 1 --2- 172.21.15.120:0/3053053749 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5380a3380 0x7fe5380a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:14.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.733+0000 7fe53effd640 1 -- 172.21.15.120:0/3053053749 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe5400cf350 con 0x7fe54c101c00 2024-09-19T09:02:14.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.733+0000 7fe5517be640 1 --2- 172.21.15.120:0/3053053749 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5380a3380 0x7fe5380a5840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe5480314e0 tx=0x7fe548000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:14.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.737+0000 7fe53effd640 1 -- 172.21.15.120:0/3053053749 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe54001f020 con 0x7fe54c101c00 2024-09-19T09:02:14.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.913+0000 7fe554223640 1 -- 172.21.15.120:0/3053053749 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fe54c062630 con 0x7fe5380a3380 2024-09-19T09:02:14.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.924+0000 7fe53effd640 1 -- 172.21.15.120:0/3053053749 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7fe54c062630 con 0x7fe5380a3380 2024-09-19T09:02:14.924 INFO:teuthology.orchestra.run.smithi120.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-19T09:02:14.925 INFO:teuthology.orchestra.run.smithi120.stdout:mds.a smithi120 running (100s) 21s ago 99s 19.5M - 19.1.1-361-g2f599a46 d2baa855c067 e6ec996a145b 2024-09-19T09:02:14.925 INFO:teuthology.orchestra.run.smithi120.stdout:mds.b smithi153 running (97s) 29s ago 96s 28.9M - 19.1.1-361-g2f599a46 d2baa855c067 3c564dedccd8 2024-09-19T09:02:14.925 INFO:teuthology.orchestra.run.smithi120.stdout:mds.c smithi196 running (112s) 29s ago 111s 17.8M - 19.1.1-361-g2f599a46 d2baa855c067 ae427bccb0d4 2024-09-19T09:02:14.925 INFO:teuthology.orchestra.run.smithi120.stdout:mds.d smithi120 running (119s) 21s ago 119s 14.2M - 19.1.1-361-g2f599a46 d2baa855c067 c4e057cca4e0 2024-09-19T09:02:14.925 INFO:teuthology.orchestra.run.smithi120.stdout:mds.e smithi153 running (117s) 29s ago 116s 19.4M - 19.1.1-361-g2f599a46 d2baa855c067 8b60695bc827 2024-09-19T09:02:14.925 INFO:teuthology.orchestra.run.smithi120.stdout:mds.f smithi196 running (114s) 29s ago 114s 20.1M - 19.1.1-361-g2f599a46 d2baa855c067 79f3de5e93f2 2024-09-19T09:02:14.925 INFO:teuthology.orchestra.run.smithi120.stdout:mds.g smithi120 running (103s) 21s ago 102s 17.3M - 19.1.1-361-g2f599a46 d2baa855c067 19915489a4a6 2024-09-19T09:02:14.925 INFO:teuthology.orchestra.run.smithi120.stdout:mds.h smithi153 running (109s) 29s ago 108s 17.2M - 19.1.1-361-g2f599a46 d2baa855c067 f6118a9454ab 2024-09-19T09:02:14.925 INFO:teuthology.orchestra.run.smithi120.stdout:mds.i smithi196 running (106s) 29s ago 106s 19.5M - 19.1.1-361-g2f599a46 d2baa855c067 92cd60c0b5c8 2024-09-19T09:02:14.925 INFO:teuthology.orchestra.run.smithi120.stdout:mds.j smithi120 running (2m) 21s ago 2m 16.0M - 19.1.1-361-g2f599a46 d2baa855c067 d3d45bf31a92 2024-09-19T09:02:14.925 INFO:teuthology.orchestra.run.smithi120.stdout:mds.k smithi153 running (2m) 29s ago 2m 16.1M - 19.1.1-361-g2f599a46 d2baa855c067 5c585088ff66 2024-09-19T09:02:14.925 INFO:teuthology.orchestra.run.smithi120.stdout:mds.l smithi196 running (2m) 29s ago 2m 17.0M - 19.1.1-361-g2f599a46 d2baa855c067 5dd429730b87 2024-09-19T09:02:14.926 INFO:teuthology.orchestra.run.smithi120.stdout:mgr.x smithi120 *:9283,8765 running (9m) 21s ago 9m 509M - 19.1.1-361-g2f599a46 d2baa855c067 fc4ae395f642 2024-09-19T09:02:14.926 INFO:teuthology.orchestra.run.smithi120.stdout:mgr.y smithi153 *:8443,8765 running (7m) 29s ago 7m 444M - 19.1.1-361-g2f599a46 d2baa855c067 ae5d6044121a 2024-09-19T09:02:14.926 INFO:teuthology.orchestra.run.smithi120.stdout:mgr.z smithi196 *:8443,8765 running (7m) 29s ago 7m 444M - 19.1.1-361-g2f599a46 d2baa855c067 8eab4dfbe448 2024-09-19T09:02:14.926 INFO:teuthology.orchestra.run.smithi120.stdout:mon.a smithi120 running (9m) 21s ago 9m 50.1M 2048M 19.1.1-361-g2f599a46 d2baa855c067 12359571ca9c 2024-09-19T09:02:14.926 INFO:teuthology.orchestra.run.smithi120.stdout:mon.b smithi153 running (7m) 29s ago 7m 42.3M 2048M 19.1.1-361-g2f599a46 d2baa855c067 d4890304681e 2024-09-19T09:02:14.926 INFO:teuthology.orchestra.run.smithi120.stdout:mon.c smithi196 running (7m) 29s ago 7m 38.7M 2048M 19.1.1-361-g2f599a46 d2baa855c067 636bb695af2c 2024-09-19T09:02:14.926 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0 smithi120 running (7m) 21s ago 7m 55.7M 4272M 19.1.1-361-g2f599a46 d2baa855c067 6aa2a34f6e58 2024-09-19T09:02:14.926 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1 smithi153 running (6m) 29s ago 6m 57.2M 4272M 19.1.1-361-g2f599a46 d2baa855c067 c7e3e511f17d 2024-09-19T09:02:14.926 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2 smithi196 running (6m) 29s ago 6m 49.8M 4272M 19.1.1-361-g2f599a46 d2baa855c067 c6102df5df55 2024-09-19T09:02:14.926 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3 smithi120 running (5m) 21s ago 5m 46.5M 4272M 19.1.1-361-g2f599a46 d2baa855c067 06cfca9b5a2e 2024-09-19T09:02:14.926 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4 smithi153 running (5m) 29s ago 5m 43.3M 4272M 19.1.1-361-g2f599a46 d2baa855c067 7458524620cb 2024-09-19T09:02:14.926 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5 smithi196 running (5m) 29s ago 5m 53.7M 4272M 19.1.1-361-g2f599a46 d2baa855c067 d04c42c0ee6d 2024-09-19T09:02:14.926 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6 smithi120 running (4m) 21s ago 4m 44.9M 4272M 19.1.1-361-g2f599a46 d2baa855c067 3b2af5d32d05 2024-09-19T09:02:14.927 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7 smithi153 running (4m) 29s ago 4m 43.6M 4272M 19.1.1-361-g2f599a46 d2baa855c067 217858a438ca 2024-09-19T09:02:14.927 INFO:teuthology.orchestra.run.smithi120.stdout:osd.8 smithi196 running (3m) 29s ago 3m 55.2M 4272M 19.1.1-361-g2f599a46 d2baa855c067 e640ff09be7a 2024-09-19T09:02:14.927 INFO:teuthology.orchestra.run.smithi120.stdout:osd.9 smithi120 running (3m) 21s ago 3m 50.7M 4272M 19.1.1-361-g2f599a46 d2baa855c067 f875f2996a9c 2024-09-19T09:02:14.927 INFO:teuthology.orchestra.run.smithi120.stdout:osd.10 smithi153 running (2m) 29s ago 2m 56.4M 4272M 19.1.1-361-g2f599a46 d2baa855c067 34e1434a3c2e 2024-09-19T09:02:14.927 INFO:teuthology.orchestra.run.smithi120.stdout:osd.11 smithi196 running (2m) 29s ago 2m 39.5M 4272M 19.1.1-361-g2f599a46 d2baa855c067 56c6c4d0cbee 2024-09-19T09:02:14.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.926+0000 7fe554223640 1 -- 172.21.15.120:0/3053053749 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5380a3380 msgr2=0x7fe5380a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:14.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.926+0000 7fe554223640 1 --2- 172.21.15.120:0/3053053749 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5380a3380 0x7fe5380a5840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe5480314e0 tx=0x7fe548000bc0 comp rx=0 tx=0).stop 2024-09-19T09:02:14.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.926+0000 7fe554223640 1 -- 172.21.15.120:0/3053053749 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe54c101c00 msgr2=0x7fe54c197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:14.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.926+0000 7fe554223640 1 --2- 172.21.15.120:0/3053053749 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe54c101c00 0x7fe54c197f80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe54000b450 tx=0x7fe54000b910 comp rx=0 tx=0).stop 2024-09-19T09:02:14.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.926+0000 7fe554223640 1 -- 172.21.15.120:0/3053053749 shutdown_connections 2024-09-19T09:02:14.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.926+0000 7fe554223640 1 --2- 172.21.15.120:0/3053053749 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5380a3380 0x7fe5380a5840 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:14.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.926+0000 7fe554223640 1 --2- 172.21.15.120:0/3053053749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe54c1037c0 0x7fe54c19f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:14.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.926+0000 7fe554223640 1 --2- 172.21.15.120:0/3053053749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe54c102e00 0x7fe54c1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:14.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.926+0000 7fe554223640 1 --2- 172.21.15.120:0/3053053749 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe54c101c00 0x7fe54c197f80 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:14.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.926+0000 7fe554223640 1 -- 172.21.15.120:0/3053053749 >> 172.21.15.120:0/3053053749 conn(0x7fe54c0fd390 msgr2=0x7fe54c1061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:14.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.927+0000 7fe554223640 1 -- 172.21.15.120:0/3053053749 shutdown_connections 2024-09-19T09:02:14.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:14.927+0000 7fe554223640 1 -- 172.21.15.120:0/3053053749 wait complete. 2024-09-19T09:02:15.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:14 smithi120 ceph-mon[31150]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:02:15.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:14 smithi120 ceph-mon[31150]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T09:02:15.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:14 smithi196 ceph-mon[35150]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:02:15.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:14 smithi196 ceph-mon[35150]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T09:02:15.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:14 smithi153 ceph-mon[35071]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:02:15.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:14 smithi153 ceph-mon[35071]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T09:02:15.650 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-19T09:02:15.896 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:16.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.347+0000 7f8185c5d640 1 -- 172.21.15.120:0/1077595384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81801037d0 msgr2=0x7f818010a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:16.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.347+0000 7f8185c5d640 1 --2- 172.21.15.120:0/1077595384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81801037d0 0x7f818010a060 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8178009ca0 tx=0x7f817802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:02:16.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.347+0000 7f8185c5d640 1 -- 172.21.15.120:0/1077595384 shutdown_connections 2024-09-19T09:02:16.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.347+0000 7f8185c5d640 1 --2- 172.21.15.120:0/1077595384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81801037d0 0x7f818010a060 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:16.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.347+0000 7f8185c5d640 1 --2- 172.21.15.120:0/1077595384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8180102e10 0x7f8180103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:16.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.347+0000 7f8185c5d640 1 --2- 172.21.15.120:0/1077595384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8180101c10 0x7f8180102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:16.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.347+0000 7f8185c5d640 1 -- 172.21.15.120:0/1077595384 >> 172.21.15.120:0/1077595384 conn(0x7f81800fd3c0 msgr2=0x7f81800ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:16.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.347+0000 7f8185c5d640 1 -- 172.21.15.120:0/1077595384 shutdown_connections 2024-09-19T09:02:16.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.347+0000 7f8185c5d640 1 -- 172.21.15.120:0/1077595384 wait complete. 2024-09-19T09:02:16.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.348+0000 7f8185c5d640 1 Processor -- start 2024-09-19T09:02:16.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.348+0000 7f8185c5d640 1 -- start start 2024-09-19T09:02:16.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.348+0000 7f8185c5d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8180101c10 0x7f8180197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:16.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.348+0000 7f8185c5d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8180102e10 0x7f8180198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:16.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.348+0000 7f8185c5d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81801037d0 0x7f818019f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:16.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.348+0000 7f8185c5d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f818010d890 con 0x7f81801037d0 2024-09-19T09:02:16.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.349+0000 7f8185c5d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f818010d710 con 0x7f8180102e10 2024-09-19T09:02:16.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.349+0000 7f8185c5d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f818010da10 con 0x7f8180101c10 2024-09-19T09:02:16.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.349+0000 7f817ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81801037d0 0x7f818019f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:16.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.349+0000 7f817ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81801037d0 0x7f818019f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43764/0 (socket says 172.21.15.120:43764) 2024-09-19T09:02:16.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.349+0000 7f817ffff640 1 -- 172.21.15.120:0/1218901483 learned_addr learned my addr 172.21.15.120:0/1218901483 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:16.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.349+0000 7f817f7fe640 1 --2- 172.21.15.120:0/1218901483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8180101c10 0x7f8180197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:16.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.349+0000 7f817effd640 1 --2- 172.21.15.120:0/1218901483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8180102e10 0x7f8180198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:16.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.349+0000 7f817ffff640 1 -- 172.21.15.120:0/1218901483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8180101c10 msgr2=0x7f8180197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:16.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.349+0000 7f817ffff640 1 --2- 172.21.15.120:0/1218901483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8180101c10 0x7f8180197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:16.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.349+0000 7f817ffff640 1 -- 172.21.15.120:0/1218901483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8180102e10 msgr2=0x7f8180198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:16.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.349+0000 7f817ffff640 1 --2- 172.21.15.120:0/1218901483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8180102e10 0x7f8180198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:16.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.349+0000 7f817ffff640 1 -- 172.21.15.120:0/1218901483 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f818019faf0 con 0x7f81801037d0 2024-09-19T09:02:16.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.349+0000 7f817ffff640 1 --2- 172.21.15.120:0/1218901483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81801037d0 0x7f818019f480 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8178005e30 tx=0x7f8178031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:16.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.349+0000 7f817cff9640 1 -- 172.21.15.120:0/1218901483 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8178003c30 con 0x7f81801037d0 2024-09-19T09:02:16.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.349+0000 7f817cff9640 1 -- 172.21.15.120:0/1218901483 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8178003dd0 con 0x7f81801037d0 2024-09-19T09:02:16.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.349+0000 7f817cff9640 1 -- 172.21.15.120:0/1218901483 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817802fde0 con 0x7f81801037d0 2024-09-19T09:02:16.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.349+0000 7f8185c5d640 1 -- 172.21.15.120:0/1218901483 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f818019fd80 con 0x7f81801037d0 2024-09-19T09:02:16.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.349+0000 7f8185c5d640 1 -- 172.21.15.120:0/1218901483 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81801a0290 con 0x7f81801037d0 2024-09-19T09:02:16.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.350+0000 7f8185c5d640 1 -- 172.21.15.120:0/1218901483 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8180109060 con 0x7f81801037d0 2024-09-19T09:02:16.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.352+0000 7f817cff9640 1 -- 172.21.15.120:0/1218901483 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f8178004050 con 0x7f81801037d0 2024-09-19T09:02:16.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.352+0000 7f817cff9640 1 --2- 172.21.15.120:0/1218901483 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f81680a3380 0x7f81680a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:16.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.352+0000 7f817f7fe640 1 --2- 172.21.15.120:0/1218901483 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f81680a3380 0x7f81680a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:16.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.352+0000 7f817cff9640 1 -- 172.21.15.120:0/1218901483 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f81780f0a90 con 0x7f81801037d0 2024-09-19T09:02:16.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.353+0000 7f817f7fe640 1 --2- 172.21.15.120:0/1218901483 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f81680a3380 0x7f81680a5840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8180102c70 tx=0x7f817400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:16.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.356+0000 7f817cff9640 1 -- 172.21.15.120:0/1218901483 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f81780bd760 con 0x7f81801037d0 2024-09-19T09:02:16.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.529+0000 7f8185c5d640 1 -- 172.21.15.120:0/1218901483 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f8180062630 con 0x7f81680a3380 2024-09-19T09:02:16.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.534+0000 7f817cff9640 1 -- 172.21.15.120:0/1218901483 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f8180062630 con 0x7f81680a3380 2024-09-19T09:02:16.535 INFO:teuthology.orchestra.run.smithi120.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-19T09:02:16.535 INFO:teuthology.orchestra.run.smithi120.stdout:mds.all 12/12 31s ago 2m smithi120=a;smithi120=d;smithi120=g;smithi120=j;smithi153=b;smithi153=e;smithi153=h;smithi153=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-09-19T09:02:16.535 INFO:teuthology.orchestra.run.smithi120.stdout:mgr 3/3 31s ago 7m smithi120=x;smithi153=y;smithi196=z;count:3 2024-09-19T09:02:16.535 INFO:teuthology.orchestra.run.smithi120.stdout:mon 3/3 31s ago 7m smithi120:172.21.15.120=a;smithi153:172.21.15.153=b;smithi196:172.21.15.196=c;count:3 2024-09-19T09:02:16.535 INFO:teuthology.orchestra.run.smithi120.stdout:osd 12 31s ago - 2024-09-19T09:02:16.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.537+0000 7f8185c5d640 1 -- 172.21.15.120:0/1218901483 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f81680a3380 msgr2=0x7f81680a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:16.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.537+0000 7f8185c5d640 1 --2- 172.21.15.120:0/1218901483 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f81680a3380 0x7f81680a5840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8180102c70 tx=0x7f817400a2d0 comp rx=0 tx=0).stop 2024-09-19T09:02:16.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.537+0000 7f8185c5d640 1 -- 172.21.15.120:0/1218901483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81801037d0 msgr2=0x7f818019f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:16.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.537+0000 7f8185c5d640 1 --2- 172.21.15.120:0/1218901483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81801037d0 0x7f818019f480 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8178005e30 tx=0x7f8178031ee0 comp rx=0 tx=0).stop 2024-09-19T09:02:16.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.537+0000 7f8185c5d640 1 -- 172.21.15.120:0/1218901483 shutdown_connections 2024-09-19T09:02:16.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.537+0000 7f8185c5d640 1 --2- 172.21.15.120:0/1218901483 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f81680a3380 0x7f81680a5840 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:16.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.537+0000 7f8185c5d640 1 --2- 172.21.15.120:0/1218901483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81801037d0 0x7f818019f480 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:16.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.537+0000 7f8185c5d640 1 --2- 172.21.15.120:0/1218901483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8180102e10 0x7f8180198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:16.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.537+0000 7f8185c5d640 1 --2- 172.21.15.120:0/1218901483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8180101c10 0x7f8180197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:16.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.537+0000 7f8185c5d640 1 -- 172.21.15.120:0/1218901483 >> 172.21.15.120:0/1218901483 conn(0x7f81800fd3c0 msgr2=0x7f8180106230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:16.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.537+0000 7f8185c5d640 1 -- 172.21.15.120:0/1218901483 shutdown_connections 2024-09-19T09:02:16.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:16.537+0000 7f8185c5d640 1 -- 172.21.15.120:0/1218901483 wait complete. 2024-09-19T09:02:17.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:16 smithi196 ceph-mon[35150]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:02:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:16 smithi196 ceph-mon[35150]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:02:17.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:16 smithi153 ceph-mon[35071]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:02:17.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:16 smithi153 ceph-mon[35071]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:02:17.241 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:16 smithi120 ceph-mon[31150]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:02:17.241 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:16 smithi120 ceph-mon[31150]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:02:17.267 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-19T09:02:17.510 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:17.912 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:17 smithi120 ceph-mon[31150]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:02:18.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.993+0000 7fa9fac61640 1 -- 172.21.15.120:0/2025909677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9f4100b10 msgr2=0x7fa9f4100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:18.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.993+0000 7fa9fac61640 1 --2- 172.21.15.120:0/2025909677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9f4100b10 0x7fa9f4100f10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8009900 tx=0x7fa9e802a410 comp rx=0 tx=0).stop 2024-09-19T09:02:18.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.993+0000 7fa9fac61640 1 -- 172.21.15.120:0/2025909677 shutdown_connections 2024-09-19T09:02:18.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.993+0000 7fa9fac61640 1 --2- 172.21.15.120:0/2025909677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9f41026d0 0x7fa9f4108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:18.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.993+0000 7fa9fac61640 1 --2- 172.21.15.120:0/2025909677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9f4101d10 0x7fa9f4102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:18.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.993+0000 7fa9fac61640 1 --2- 172.21.15.120:0/2025909677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9f4100b10 0x7fa9f4100f10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:18.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.993+0000 7fa9fac61640 1 -- 172.21.15.120:0/2025909677 >> 172.21.15.120:0/2025909677 conn(0x7fa9f40fc2a0 msgr2=0x7fa9f40fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:18.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.993+0000 7fa9fac61640 1 -- 172.21.15.120:0/2025909677 shutdown_connections 2024-09-19T09:02:18.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.993+0000 7fa9fac61640 1 -- 172.21.15.120:0/2025909677 wait complete. 2024-09-19T09:02:18.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.994+0000 7fa9fac61640 1 Processor -- start 2024-09-19T09:02:18.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.994+0000 7fa9fac61640 1 -- start start 2024-09-19T09:02:18.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.994+0000 7fa9fac61640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9f4100b10 0x7fa9f406b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:18.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.994+0000 7fa9fac61640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9f4101d10 0x7fa9f419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:18.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.994+0000 7fa9fac61640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9f41026d0 0x7fa9f419f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:18.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.994+0000 7fa9fac61640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f410c790 con 0x7fa9f4101d10 2024-09-19T09:02:18.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.994+0000 7fa9fac61640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f410c610 con 0x7fa9f41026d0 2024-09-19T09:02:18.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.994+0000 7fa9fac61640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f410c910 con 0x7fa9f4100b10 2024-09-19T09:02:18.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.994+0000 7fa9f3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9f4101d10 0x7fa9f419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:18.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.994+0000 7fa9f3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9f4101d10 0x7fa9f419cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43774/0 (socket says 172.21.15.120:43774) 2024-09-19T09:02:18.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.994+0000 7fa9f3fff640 1 -- 172.21.15.120:0/3685962529 learned_addr learned my addr 172.21.15.120:0/3685962529 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:18.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.994+0000 7fa9f91fe640 1 --2- 172.21.15.120:0/3685962529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9f41026d0 0x7fa9f419f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:18.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.994+0000 7fa9f3fff640 1 -- 172.21.15.120:0/3685962529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9f4100b10 msgr2=0x7fa9f406b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:18.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.994+0000 7fa9f3fff640 1 --2- 172.21.15.120:0/3685962529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9f4100b10 0x7fa9f406b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:18.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.994+0000 7fa9f3fff640 1 -- 172.21.15.120:0/3685962529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9f41026d0 msgr2=0x7fa9f419f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:18.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.994+0000 7fa9f3fff640 1 --2- 172.21.15.120:0/3685962529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9f41026d0 0x7fa9f419f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:18.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.994+0000 7fa9f3fff640 1 -- 172.21.15.120:0/3685962529 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f419fc10 con 0x7fa9f4101d10 2024-09-19T09:02:18.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.994+0000 7fa9f91fe640 1 --2- 172.21.15.120:0/3685962529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9f41026d0 0x7fa9f419f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:02:18.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.994+0000 7fa9f3fff640 1 --2- 172.21.15.120:0/3685962529 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9f4101d10 0x7fa9f419cce0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa9e000c970 tx=0x7fa9e000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:18.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.995+0000 7fa9f1ffb640 1 -- 172.21.15.120:0/3685962529 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0007bb0 con 0x7fa9f4101d10 2024-09-19T09:02:18.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.995+0000 7fa9f1ffb640 1 -- 172.21.15.120:0/3685962529 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e0007d50 con 0x7fa9f4101d10 2024-09-19T09:02:18.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.995+0000 7fa9fac61640 1 -- 172.21.15.120:0/3685962529 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9f419fea0 con 0x7fa9f4101d10 2024-09-19T09:02:18.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.995+0000 7fa9f1ffb640 1 -- 172.21.15.120:0/3685962529 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0018800 con 0x7fa9f4101d10 2024-09-19T09:02:18.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.995+0000 7fa9fac61640 1 -- 172.21.15.120:0/3685962529 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9f41a0350 con 0x7fa9f4101d10 2024-09-19T09:02:18.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.996+0000 7fa9fac61640 1 -- 172.21.15.120:0/3685962529 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9f4107f60 con 0x7fa9f4101d10 2024-09-19T09:02:18.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.997+0000 7fa9f1ffb640 1 -- 172.21.15.120:0/3685962529 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fa9e000f450 con 0x7fa9f4101d10 2024-09-19T09:02:18.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.998+0000 7fa9f1ffb640 1 --2- 172.21.15.120:0/3685962529 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9e40a3350 0x7fa9e40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:18.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.998+0000 7fa9f89fd640 1 --2- 172.21.15.120:0/3685962529 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9e40a3350 0x7fa9e40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:18.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.998+0000 7fa9f1ffb640 1 -- 172.21.15.120:0/3685962529 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa9e00cfb10 con 0x7fa9f4101d10 2024-09-19T09:02:18.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:17.998+0000 7fa9f89fd640 1 --2- 172.21.15.120:0/3685962529 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9e40a3350 0x7fa9e40a5810 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa9e80096c0 tx=0x7fa9e8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:18.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:18.002+0000 7fa9f1ffb640 1 -- 172.21.15.120:0/3685962529 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa9e0014030 con 0x7fa9f4101d10 2024-09-19T09:02:18.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:18.174+0000 7fa9fac61640 1 -- 172.21.15.120:0/3685962529 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fa9f4062630 con 0x7fa9e40a3350 2024-09-19T09:02:18.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:18.176+0000 7fa9f1ffb640 1 -- 172.21.15.120:0/3685962529 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fa9f4062630 con 0x7fa9e40a3350 2024-09-19T09:02:18.176 INFO:teuthology.orchestra.run.smithi120.stdout:HOST ADDR LABELS STATUS 2024-09-19T09:02:18.176 INFO:teuthology.orchestra.run.smithi120.stdout:smithi120 172.21.15.120 2024-09-19T09:02:18.176 INFO:teuthology.orchestra.run.smithi120.stdout:smithi153 172.21.15.153 2024-09-19T09:02:18.176 INFO:teuthology.orchestra.run.smithi120.stdout:smithi196 172.21.15.196 2024-09-19T09:02:18.176 INFO:teuthology.orchestra.run.smithi120.stdout:3 hosts in cluster 2024-09-19T09:02:18.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:18.178+0000 7fa9fac61640 1 -- 172.21.15.120:0/3685962529 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9e40a3350 msgr2=0x7fa9e40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:18.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:18.178+0000 7fa9fac61640 1 --2- 172.21.15.120:0/3685962529 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9e40a3350 0x7fa9e40a5810 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa9e80096c0 tx=0x7fa9e8031400 comp rx=0 tx=0).stop 2024-09-19T09:02:18.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:18.178+0000 7fa9fac61640 1 -- 172.21.15.120:0/3685962529 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9f4101d10 msgr2=0x7fa9f419cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:18.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:18.178+0000 7fa9fac61640 1 --2- 172.21.15.120:0/3685962529 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9f4101d10 0x7fa9f419cce0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa9e000c970 tx=0x7fa9e000ce30 comp rx=0 tx=0).stop 2024-09-19T09:02:18.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:18.178+0000 7fa9fac61640 1 -- 172.21.15.120:0/3685962529 shutdown_connections 2024-09-19T09:02:18.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:18.178+0000 7fa9fac61640 1 --2- 172.21.15.120:0/3685962529 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9e40a3350 0x7fa9e40a5810 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:18.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:18.178+0000 7fa9fac61640 1 --2- 172.21.15.120:0/3685962529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9f41026d0 0x7fa9f419f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:18.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:18.178+0000 7fa9fac61640 1 --2- 172.21.15.120:0/3685962529 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9f4101d10 0x7fa9f419cce0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:18.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:18.178+0000 7fa9fac61640 1 --2- 172.21.15.120:0/3685962529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9f4100b10 0x7fa9f406b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:18.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:18.178+0000 7fa9fac61640 1 -- 172.21.15.120:0/3685962529 >> 172.21.15.120:0/3685962529 conn(0x7fa9f40fc2a0 msgr2=0x7fa9f4105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:18.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:18.178+0000 7fa9fac61640 1 -- 172.21.15.120:0/3685962529 shutdown_connections 2024-09-19T09:02:18.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:18.178+0000 7fa9fac61640 1 -- 172.21.15.120:0/3685962529 wait complete. 2024-09-19T09:02:18.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:17 smithi196 ceph-mon[35150]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:02:18.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:17 smithi153 ceph-mon[35071]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:02:18.921 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-19T09:02:19.172 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:19.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:18 smithi196 ceph-mon[35150]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T09:02:19.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:18 smithi153 ceph-mon[35071]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T09:02:19.293 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:18 smithi120 ceph-mon[31150]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T09:02:19.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.656+0000 7f6a80da6640 1 -- 172.21.15.120:0/25985129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a7c101c00 msgr2=0x7f6a7c102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:19.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.656+0000 7f6a80da6640 1 --2- 172.21.15.120:0/25985129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a7c101c00 0x7f6a7c102000 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009900 tx=0x7f6a6c02a410 comp rx=0 tx=0).stop 2024-09-19T09:02:19.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.656+0000 7f6a80da6640 1 -- 172.21.15.120:0/25985129 shutdown_connections 2024-09-19T09:02:19.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.656+0000 7f6a80da6640 1 --2- 172.21.15.120:0/25985129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a7c1037c0 0x7f6a7c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:19.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.656+0000 7f6a80da6640 1 --2- 172.21.15.120:0/25985129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a7c102e00 0x7f6a7c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:19.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.656+0000 7f6a80da6640 1 --2- 172.21.15.120:0/25985129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a7c101c00 0x7f6a7c102000 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:19.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.656+0000 7f6a80da6640 1 -- 172.21.15.120:0/25985129 >> 172.21.15.120:0/25985129 conn(0x7f6a7c0fd390 msgr2=0x7f6a7c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:19.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.656+0000 7f6a80da6640 1 -- 172.21.15.120:0/25985129 shutdown_connections 2024-09-19T09:02:19.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.656+0000 7f6a80da6640 1 -- 172.21.15.120:0/25985129 wait complete. 2024-09-19T09:02:19.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.657+0000 7f6a80da6640 1 Processor -- start 2024-09-19T09:02:19.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.657+0000 7f6a80da6640 1 -- start start 2024-09-19T09:02:19.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.657+0000 7f6a80da6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a7c101c00 0x7f6a7c197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:19.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.657+0000 7f6a80da6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a7c102e00 0x7f6a7c198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:19.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.657+0000 7f6a80da6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a7c1037c0 0x7f6a7c19f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:19.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.657+0000 7f6a80da6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7c10d8e0 con 0x7f6a7c1037c0 2024-09-19T09:02:19.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.657+0000 7f6a80da6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7c10d760 con 0x7f6a7c101c00 2024-09-19T09:02:19.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.657+0000 7f6a80da6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7c10da60 con 0x7f6a7c102e00 2024-09-19T09:02:19.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.657+0000 7f6a7ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a7c1037c0 0x7f6a7c19f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:19.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.657+0000 7f6a7ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a7c1037c0 0x7f6a7c19f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43802/0 (socket says 172.21.15.120:43802) 2024-09-19T09:02:19.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.657+0000 7f6a7ad9d640 1 -- 172.21.15.120:0/556442778 learned_addr learned my addr 172.21.15.120:0/556442778 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:19.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.657+0000 7f6a7a59c640 1 --2- 172.21.15.120:0/556442778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a7c101c00 0x7f6a7c197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:19.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.657+0000 7f6a79d9b640 1 --2- 172.21.15.120:0/556442778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a7c102e00 0x7f6a7c198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:19.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.657+0000 7f6a7ad9d640 1 -- 172.21.15.120:0/556442778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a7c102e00 msgr2=0x7f6a7c198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:19.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.657+0000 7f6a7ad9d640 1 --2- 172.21.15.120:0/556442778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a7c102e00 0x7f6a7c198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:19.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.657+0000 7f6a7ad9d640 1 -- 172.21.15.120:0/556442778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a7c101c00 msgr2=0x7f6a7c197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:19.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.657+0000 7f6a7ad9d640 1 --2- 172.21.15.120:0/556442778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a7c101c00 0x7f6a7c197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:19.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.657+0000 7f6a7ad9d640 1 -- 172.21.15.120:0/556442778 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a7c19fc20 con 0x7f6a7c1037c0 2024-09-19T09:02:19.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.658+0000 7f6a7ad9d640 1 --2- 172.21.15.120:0/556442778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a7c1037c0 0x7f6a7c19f5b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6a7400cad0 tx=0x7f6a7400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:19.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.658+0000 7f6a6b7fe640 1 -- 172.21.15.120:0/556442778 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7400ad60 con 0x7f6a7c1037c0 2024-09-19T09:02:19.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.658+0000 7f6a6b7fe640 1 -- 172.21.15.120:0/556442778 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a74005020 con 0x7f6a7c1037c0 2024-09-19T09:02:19.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.658+0000 7f6a80da6640 1 -- 172.21.15.120:0/556442778 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a7c19ff10 con 0x7f6a7c1037c0 2024-09-19T09:02:19.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.658+0000 7f6a6b7fe640 1 -- 172.21.15.120:0/556442778 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7400f6b0 con 0x7f6a7c1037c0 2024-09-19T09:02:19.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.658+0000 7f6a80da6640 1 -- 172.21.15.120:0/556442778 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a7c1a03f0 con 0x7f6a7c1037c0 2024-09-19T09:02:19.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.659+0000 7f6a80da6640 1 -- 172.21.15.120:0/556442778 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a7c108fa0 con 0x7f6a7c1037c0 2024-09-19T09:02:19.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.660+0000 7f6a6b7fe640 1 -- 172.21.15.120:0/556442778 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f6a74005560 con 0x7f6a7c1037c0 2024-09-19T09:02:19.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.661+0000 7f6a6b7fe640 1 --2- 172.21.15.120:0/556442778 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6a640a3300 0x7f6a640a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:19.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.661+0000 7f6a7a59c640 1 --2- 172.21.15.120:0/556442778 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6a640a3300 0x7f6a640a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:19.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.661+0000 7f6a6b7fe640 1 -- 172.21.15.120:0/556442778 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f6a740cfa30 con 0x7f6a7c1037c0 2024-09-19T09:02:19.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.661+0000 7f6a7a59c640 1 --2- 172.21.15.120:0/556442778 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6a640a3300 0x7f6a640a57c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c005ab0 tx=0x7f6a6c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:19.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.665+0000 7f6a6b7fe640 1 -- 172.21.15.120:0/556442778 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6a74014030 con 0x7f6a7c1037c0 2024-09-19T09:02:19.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.838+0000 7f6a80da6640 1 -- 172.21.15.120:0/556442778 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f6a7c062630 con 0x7f6a640a3300 2024-09-19T09:02:19.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.840+0000 7f6a6b7fe640 1 -- 172.21.15.120:0/556442778 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f6a7c062630 con 0x7f6a640a3300 2024-09-19T09:02:19.840 INFO:teuthology.orchestra.run.smithi120.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-19T09:02:19.840 INFO:teuthology.orchestra.run.smithi120.stdout:smithi120 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400XV400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-19T09:02:19.840 INFO:teuthology.orchestra.run.smithi120.stdout:smithi153 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300MV400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-19T09:02:19.840 INFO:teuthology.orchestra.run.smithi120.stdout:smithi196 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6246000Z400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-19T09:02:19.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.842+0000 7f6a80da6640 1 -- 172.21.15.120:0/556442778 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6a640a3300 msgr2=0x7f6a640a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:19.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.842+0000 7f6a80da6640 1 --2- 172.21.15.120:0/556442778 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6a640a3300 0x7f6a640a57c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c005ab0 tx=0x7f6a6c000bc0 comp rx=0 tx=0).stop 2024-09-19T09:02:19.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.842+0000 7f6a80da6640 1 -- 172.21.15.120:0/556442778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a7c1037c0 msgr2=0x7f6a7c19f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:19.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.842+0000 7f6a80da6640 1 --2- 172.21.15.120:0/556442778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a7c1037c0 0x7f6a7c19f5b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6a7400cad0 tx=0x7f6a7400ced0 comp rx=0 tx=0).stop 2024-09-19T09:02:19.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.842+0000 7f6a80da6640 1 -- 172.21.15.120:0/556442778 shutdown_connections 2024-09-19T09:02:19.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.842+0000 7f6a80da6640 1 --2- 172.21.15.120:0/556442778 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6a640a3300 0x7f6a640a57c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:19.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.842+0000 7f6a80da6640 1 --2- 172.21.15.120:0/556442778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a7c1037c0 0x7f6a7c19f5b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:19.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.842+0000 7f6a80da6640 1 --2- 172.21.15.120:0/556442778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a7c102e00 0x7f6a7c198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:19.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.842+0000 7f6a80da6640 1 --2- 172.21.15.120:0/556442778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a7c101c00 0x7f6a7c197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:19.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.842+0000 7f6a80da6640 1 -- 172.21.15.120:0/556442778 >> 172.21.15.120:0/556442778 conn(0x7f6a7c0fd390 msgr2=0x7f6a7c106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:19.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.842+0000 7f6a80da6640 1 -- 172.21.15.120:0/556442778 shutdown_connections 2024-09-19T09:02:19.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:19.842+0000 7f6a80da6640 1 -- 172.21.15.120:0/556442778 wait complete. 2024-09-19T09:02:20.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:19 smithi196 ceph-mon[35150]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:02:20.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:19 smithi153 ceph-mon[35071]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:02:20.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:19 smithi120 ceph-mon[31150]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:02:20.657 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T09:02:20.667 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi120.front.sepia.ceph.com 2024-09-19T09:02:20.668 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-19T09:02:20.915 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:21.036 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:20 smithi120 ceph-mon[31150]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T09:02:21.036 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:20 smithi120 ceph-mon[31150]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:02:21.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:20 smithi196 ceph-mon[35150]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T09:02:21.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:20 smithi196 ceph-mon[35150]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:02:21.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:20 smithi153 ceph-mon[35071]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T09:02:21.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:20 smithi153 ceph-mon[35071]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:02:21.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.405+0000 7faf5a0ad640 1 -- 172.21.15.120:0/1536665362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf54100b10 msgr2=0x7faf54100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:21.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.405+0000 7faf5a0ad640 1 --2- 172.21.15.120:0/1536665362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf54100b10 0x7faf54100f10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faf44009ca0 tx=0x7faf4402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:02:21.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.405+0000 7faf5a0ad640 1 -- 172.21.15.120:0/1536665362 shutdown_connections 2024-09-19T09:02:21.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.405+0000 7faf5a0ad640 1 --2- 172.21.15.120:0/1536665362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf541026d0 0x7faf54108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:21.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.405+0000 7faf5a0ad640 1 --2- 172.21.15.120:0/1536665362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf54101d10 0x7faf54102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:21.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.405+0000 7faf5a0ad640 1 --2- 172.21.15.120:0/1536665362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf54100b10 0x7faf54100f10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:21.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.405+0000 7faf5a0ad640 1 -- 172.21.15.120:0/1536665362 >> 172.21.15.120:0/1536665362 conn(0x7faf540fc2a0 msgr2=0x7faf540fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:21.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.405+0000 7faf5a0ad640 1 -- 172.21.15.120:0/1536665362 shutdown_connections 2024-09-19T09:02:21.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.405+0000 7faf5a0ad640 1 -- 172.21.15.120:0/1536665362 wait complete. 2024-09-19T09:02:21.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.406+0000 7faf5a0ad640 1 Processor -- start 2024-09-19T09:02:21.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.406+0000 7faf5a0ad640 1 -- start start 2024-09-19T09:02:21.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.406+0000 7faf5a0ad640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf54100b10 0x7faf54197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:21.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.406+0000 7faf5a0ad640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf54101d10 0x7faf54198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:21.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.406+0000 7faf5a0ad640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf541026d0 0x7faf5419f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:21.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.406+0000 7faf5a0ad640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faf5410d900 con 0x7faf54101d10 2024-09-19T09:02:21.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.406+0000 7faf5a0ad640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faf5410d780 con 0x7faf54100b10 2024-09-19T09:02:21.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.406+0000 7faf5a0ad640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faf5410da80 con 0x7faf541026d0 2024-09-19T09:02:21.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.406+0000 7faf52ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf54101d10 0x7faf54198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:21.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.406+0000 7faf52ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf54101d10 0x7faf54198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49848/0 (socket says 172.21.15.120:49848) 2024-09-19T09:02:21.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.406+0000 7faf52ffd640 1 -- 172.21.15.120:0/3894970280 learned_addr learned my addr 172.21.15.120:0/3894970280 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:21.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.407+0000 7faf52ffd640 1 -- 172.21.15.120:0/3894970280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf541026d0 msgr2=0x7faf5419f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:21.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.407+0000 7faf52ffd640 1 --2- 172.21.15.120:0/3894970280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf541026d0 0x7faf5419f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:21.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.407+0000 7faf52ffd640 1 -- 172.21.15.120:0/3894970280 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf54100b10 msgr2=0x7faf54197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:21.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.407+0000 7faf52ffd640 1 --2- 172.21.15.120:0/3894970280 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf54100b10 0x7faf54197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:21.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.407+0000 7faf52ffd640 1 -- 172.21.15.120:0/3894970280 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf5419fb50 con 0x7faf54101d10 2024-09-19T09:02:21.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.407+0000 7faf52ffd640 1 --2- 172.21.15.120:0/3894970280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf54101d10 0x7faf54198460 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00c920 tx=0x7faf3c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:21.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.407+0000 7faf50ff9640 1 -- 172.21.15.120:0/3894970280 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3c007c70 con 0x7faf54101d10 2024-09-19T09:02:21.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.407+0000 7faf50ff9640 1 -- 172.21.15.120:0/3894970280 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf3c007e10 con 0x7faf54101d10 2024-09-19T09:02:21.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.407+0000 7faf5a0ad640 1 -- 172.21.15.120:0/3894970280 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf5419fe40 con 0x7faf54101d10 2024-09-19T09:02:21.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.407+0000 7faf50ff9640 1 -- 172.21.15.120:0/3894970280 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3c018890 con 0x7faf54101d10 2024-09-19T09:02:21.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.407+0000 7faf5a0ad640 1 -- 172.21.15.120:0/3894970280 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf541a0370 con 0x7faf54101d10 2024-09-19T09:02:21.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.408+0000 7faf5a0ad640 1 -- 172.21.15.120:0/3894970280 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf54107e60 con 0x7faf54101d10 2024-09-19T09:02:21.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.409+0000 7faf50ff9640 1 -- 172.21.15.120:0/3894970280 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7faf3c018a30 con 0x7faf54101d10 2024-09-19T09:02:21.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.410+0000 7faf50ff9640 1 --2- 172.21.15.120:0/3894970280 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faf400a3300 0x7faf400a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:21.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.410+0000 7faf537fe640 1 --2- 172.21.15.120:0/3894970280 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faf400a3300 0x7faf400a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:21.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.410+0000 7faf50ff9640 1 -- 172.21.15.120:0/3894970280 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7faf3c0cfc60 con 0x7faf54101d10 2024-09-19T09:02:21.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.410+0000 7faf537fe640 1 --2- 172.21.15.120:0/3894970280 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faf400a3300 0x7faf400a57c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faf44005e30 tx=0x7faf44004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:21.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.414+0000 7faf50ff9640 1 -- 172.21.15.120:0/3894970280 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7faf3c014030 con 0x7faf54101d10 2024-09-19T09:02:21.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.646+0000 7faf5a0ad640 1 -- 172.21.15.120:0/3894970280 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7faf5402d9f0 con 0x7faf54101d10 2024-09-19T09:02:21.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.646+0000 7faf50ff9640 1 -- 172.21.15.120:0/3894970280 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3138 (secure 0 0 0) 0x7faf3c01f300 con 0x7faf54101d10 2024-09-19T09:02:21.647 INFO:teuthology.orchestra.run.smithi120.stdout:e27 2024-09-19T09:02:21.647 INFO:teuthology.orchestra.run.smithi120.stdout:btime 2024-09-19T09:00:59:134373+0000 2024-09-19T09:02:21.647 INFO:teuthology.orchestra.run.smithi120.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T09:02:21.647 INFO:teuthology.orchestra.run.smithi120.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T09:02:21.647 INFO:teuthology.orchestra.run.smithi120.stdout:legacy client fscid: 1 2024-09-19T09:02:21.647 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:02:21.647 INFO:teuthology.orchestra.run.smithi120.stdout:Filesystem 'cephfs' (1) 2024-09-19T09:02:21.647 INFO:teuthology.orchestra.run.smithi120.stdout:fs_name cephfs 2024-09-19T09:02:21.648 INFO:teuthology.orchestra.run.smithi120.stdout:epoch 27 2024-09-19T09:02:21.648 INFO:teuthology.orchestra.run.smithi120.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-19T09:02:21.648 INFO:teuthology.orchestra.run.smithi120.stdout:created 2024-09-19T09:00:47.659717+0000 2024-09-19T09:02:21.648 INFO:teuthology.orchestra.run.smithi120.stdout:modified 2024-09-19T09:00:59.134353+0000 2024-09-19T09:02:21.648 INFO:teuthology.orchestra.run.smithi120.stdout:tableserver 0 2024-09-19T09:02:21.648 INFO:teuthology.orchestra.run.smithi120.stdout:root 0 2024-09-19T09:02:21.648 INFO:teuthology.orchestra.run.smithi120.stdout:session_timeout 300 2024-09-19T09:02:21.648 INFO:teuthology.orchestra.run.smithi120.stdout:session_autoclose 300 2024-09-19T09:02:21.648 INFO:teuthology.orchestra.run.smithi120.stdout:max_file_size 1099511627776 2024-09-19T09:02:21.648 INFO:teuthology.orchestra.run.smithi120.stdout:max_xattr_size 65536 2024-09-19T09:02:21.648 INFO:teuthology.orchestra.run.smithi120.stdout:required_client_features {} 2024-09-19T09:02:21.648 INFO:teuthology.orchestra.run.smithi120.stdout:last_failure 0 2024-09-19T09:02:21.649 INFO:teuthology.orchestra.run.smithi120.stdout:last_failure_osd_epoch 0 2024-09-19T09:02:21.649 INFO:teuthology.orchestra.run.smithi120.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T09:02:21.649 INFO:teuthology.orchestra.run.smithi120.stdout:max_mds 5 2024-09-19T09:02:21.649 INFO:teuthology.orchestra.run.smithi120.stdout:in 0,1,2,3,4 2024-09-19T09:02:21.649 INFO:teuthology.orchestra.run.smithi120.stdout:up {0=24461,1=24446,2=24442,3=24430,4=24421} 2024-09-19T09:02:21.649 INFO:teuthology.orchestra.run.smithi120.stdout:failed 2024-09-19T09:02:21.649 INFO:teuthology.orchestra.run.smithi120.stdout:damaged 2024-09-19T09:02:21.649 INFO:teuthology.orchestra.run.smithi120.stdout:stopped 2024-09-19T09:02:21.649 INFO:teuthology.orchestra.run.smithi120.stdout:data_pools [3] 2024-09-19T09:02:21.649 INFO:teuthology.orchestra.run.smithi120.stdout:metadata_pool 2 2024-09-19T09:02:21.649 INFO:teuthology.orchestra.run.smithi120.stdout:inline_data disabled 2024-09-19T09:02:21.650 INFO:teuthology.orchestra.run.smithi120.stdout:balancer 2024-09-19T09:02:21.650 INFO:teuthology.orchestra.run.smithi120.stdout:bal_rank_mask -1 2024-09-19T09:02:21.650 INFO:teuthology.orchestra.run.smithi120.stdout:standby_count_wanted 0 2024-09-19T09:02:21.650 INFO:teuthology.orchestra.run.smithi120.stdout:qdb_cluster leader: 24461 members: 24461,24446,24442,24430,24421 2024-09-19T09:02:21.650 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.b{0:24461} state up:active seq 4 addr [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:02:21.650 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.e{0:24419} state up:standby-replay seq 1 addr [v2:172.21.15.153:6834/3103515283,v1:172.21.15.153:6835/3103515283] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:02:21.650 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.h{1:24446} state up:active seq 9 addr [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:02:21.650 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.l{1:24400} state up:standby-replay seq 1 addr [v2:172.21.15.196:6832/1407049222,v1:172.21.15.196:6833/1407049222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:02:21.650 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.i{2:24442} state up:active seq 8 addr [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:02:21.650 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.k{2:24392} state up:standby-replay seq 1 addr [v2:172.21.15.153:6832/1522304272,v1:172.21.15.153:6833/1522304272] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:02:21.650 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.c{3:24430} state up:active seq 10 addr [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:02:21.651 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.a{3:14613} state up:standby-replay seq 1 addr [v2:172.21.15.120:6840/658447478,v1:172.21.15.120:6841/658447478] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:02:21.651 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.f{4:24421} state up:active seq 11 addr [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:02:21.651 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.g{4:14601} state up:standby-replay seq 1 addr [v2:172.21.15.120:6838/1939272010,v1:172.21.15.120:6839/1939272010] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:02:21.651 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:02:21.651 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:02:21.651 INFO:teuthology.orchestra.run.smithi120.stdout:Standby daemons: 2024-09-19T09:02:21.651 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:02:21.651 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.j{-1:14544} state up:standby seq 1 addr [v2:172.21.15.120:6834/2395347742,v1:172.21.15.120:6835/2395347742] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:02:21.651 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.d{-1:14556} state up:standby seq 1 addr [v2:172.21.15.120:6836/217332579,v1:172.21.15.120:6837/217332579] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:02:21.652 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 27 2024-09-19T09:02:21.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.648+0000 7faf5a0ad640 1 -- 172.21.15.120:0/3894970280 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faf400a3300 msgr2=0x7faf400a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:21.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.648+0000 7faf5a0ad640 1 --2- 172.21.15.120:0/3894970280 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faf400a3300 0x7faf400a57c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faf44005e30 tx=0x7faf44004a40 comp rx=0 tx=0).stop 2024-09-19T09:02:21.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.648+0000 7faf5a0ad640 1 -- 172.21.15.120:0/3894970280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf54101d10 msgr2=0x7faf54198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:21.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.648+0000 7faf5a0ad640 1 --2- 172.21.15.120:0/3894970280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf54101d10 0x7faf54198460 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00c920 tx=0x7faf3c00cde0 comp rx=0 tx=0).stop 2024-09-19T09:02:21.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.649+0000 7faf5a0ad640 1 -- 172.21.15.120:0/3894970280 shutdown_connections 2024-09-19T09:02:21.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.649+0000 7faf5a0ad640 1 --2- 172.21.15.120:0/3894970280 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faf400a3300 0x7faf400a57c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:21.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.649+0000 7faf5a0ad640 1 --2- 172.21.15.120:0/3894970280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf541026d0 0x7faf5419f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:21.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.649+0000 7faf5a0ad640 1 --2- 172.21.15.120:0/3894970280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf54101d10 0x7faf54198460 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:21.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.649+0000 7faf5a0ad640 1 --2- 172.21.15.120:0/3894970280 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf54100b10 0x7faf54197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:21.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.649+0000 7faf5a0ad640 1 -- 172.21.15.120:0/3894970280 >> 172.21.15.120:0/3894970280 conn(0x7faf540fc2a0 msgr2=0x7faf54105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:21.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.649+0000 7faf5a0ad640 1 -- 172.21.15.120:0/3894970280 shutdown_connections 2024-09-19T09:02:21.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:21.649+0000 7faf5a0ad640 1 -- 172.21.15.120:0/3894970280 wait complete. 2024-09-19T09:02:22.066 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:21 smithi120 ceph-mon[31150]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T09:02:22.066 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:21 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3894970280' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T09:02:22.188 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:21 smithi196 ceph-mon[35150]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T09:02:22.188 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:21 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3894970280' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T09:02:22.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:21 smithi153 ceph-mon[35071]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T09:02:22.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:21 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3894970280' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T09:02:22.543 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-19T09:02:22.789 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:23.177 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:22 smithi120 ceph-mon[31150]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T09:02:23.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:22 smithi196 ceph-mon[35150]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T09:02:23.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:22 smithi153 ceph-mon[35071]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T09:02:23.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.257+0000 7f8460901640 1 -- 172.21.15.120:0/3851047753 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f845c102690 msgr2=0x7f845c108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:23.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.257+0000 7f8460901640 1 --2- 172.21.15.120:0/3851047753 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f845c102690 0x7f845c108f20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f844c009900 tx=0x7f844c02a410 comp rx=0 tx=0).stop 2024-09-19T09:02:23.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.257+0000 7f8460901640 1 -- 172.21.15.120:0/3851047753 shutdown_connections 2024-09-19T09:02:23.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.257+0000 7f8460901640 1 --2- 172.21.15.120:0/3851047753 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f845c102690 0x7f845c108f20 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:23.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.257+0000 7f8460901640 1 --2- 172.21.15.120:0/3851047753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f845c101cd0 0x7f845c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:23.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.257+0000 7f8460901640 1 --2- 172.21.15.120:0/3851047753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f845c100ad0 0x7f845c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:23.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.257+0000 7f8460901640 1 -- 172.21.15.120:0/3851047753 >> 172.21.15.120:0/3851047753 conn(0x7f845c0fc280 msgr2=0x7f845c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:23.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.257+0000 7f8460901640 1 -- 172.21.15.120:0/3851047753 shutdown_connections 2024-09-19T09:02:23.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.257+0000 7f8460901640 1 -- 172.21.15.120:0/3851047753 wait complete. 2024-09-19T09:02:23.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.258+0000 7f8460901640 1 Processor -- start 2024-09-19T09:02:23.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.258+0000 7f8460901640 1 -- start start 2024-09-19T09:02:23.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.258+0000 7f8460901640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f845c100ad0 0x7f845c06aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:23.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.258+0000 7f8460901640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f845c101cd0 0x7f845c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:23.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.258+0000 7f8460901640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f845c102690 0x7f845c19f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:23.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.258+0000 7f8460901640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f845c10c7b0 con 0x7f845c100ad0 2024-09-19T09:02:23.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.258+0000 7f8460901640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f845c10c630 con 0x7f845c102690 2024-09-19T09:02:23.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.258+0000 7f8460901640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f845c10c930 con 0x7f845c101cd0 2024-09-19T09:02:23.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.258+0000 7f845a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f845c100ad0 0x7f845c06aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:23.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.258+0000 7f845a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f845c100ad0 0x7f845c06aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49860/0 (socket says 172.21.15.120:49860) 2024-09-19T09:02:23.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.258+0000 7f845a59c640 1 -- 172.21.15.120:0/4090497825 learned_addr learned my addr 172.21.15.120:0/4090497825 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:23.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.258+0000 7f8459d9b640 1 --2- 172.21.15.120:0/4090497825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f845c101cd0 0x7f845c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:23.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.258+0000 7f845ad9d640 1 --2- 172.21.15.120:0/4090497825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f845c102690 0x7f845c19f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:23.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.258+0000 7f845a59c640 1 -- 172.21.15.120:0/4090497825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f845c101cd0 msgr2=0x7f845c19cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:23.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.258+0000 7f845a59c640 1 --2- 172.21.15.120:0/4090497825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f845c101cd0 0x7f845c19cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:23.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.258+0000 7f845a59c640 1 -- 172.21.15.120:0/4090497825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f845c102690 msgr2=0x7f845c19f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:23.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.258+0000 7f845a59c640 1 --2- 172.21.15.120:0/4090497825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f845c102690 0x7f845c19f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:23.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.258+0000 7f845a59c640 1 -- 172.21.15.120:0/4090497825 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f845c19fb00 con 0x7f845c100ad0 2024-09-19T09:02:23.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.258+0000 7f8459d9b640 1 --2- 172.21.15.120:0/4090497825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f845c101cd0 0x7f845c19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:02:23.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.259+0000 7f845a59c640 1 --2- 172.21.15.120:0/4090497825 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f845c100ad0 0x7f845c06aad0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f845400ccb0 tx=0x7f845400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:23.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.259+0000 7f84537fe640 1 -- 172.21.15.120:0/4090497825 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454003b70 con 0x7f845c100ad0 2024-09-19T09:02:23.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.259+0000 7f8460901640 1 -- 172.21.15.120:0/4090497825 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f845c19fd90 con 0x7f845c100ad0 2024-09-19T09:02:23.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.259+0000 7f84537fe640 1 -- 172.21.15.120:0/4090497825 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f845400ae50 con 0x7f845c100ad0 2024-09-19T09:02:23.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.259+0000 7f84537fe640 1 -- 172.21.15.120:0/4090497825 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845400f810 con 0x7f845c100ad0 2024-09-19T09:02:23.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.259+0000 7f8460901640 1 -- 172.21.15.120:0/4090497825 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f845c1a02f0 con 0x7f845c100ad0 2024-09-19T09:02:23.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.260+0000 7f8460901640 1 -- 172.21.15.120:0/4090497825 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f845c107e20 con 0x7f845c100ad0 2024-09-19T09:02:23.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.261+0000 7f84537fe640 1 -- 172.21.15.120:0/4090497825 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f845400f9b0 con 0x7f845c100ad0 2024-09-19T09:02:23.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.262+0000 7f84537fe640 1 --2- 172.21.15.120:0/4090497825 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f84480a3240 0x7f84480a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:23.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.262+0000 7f8459d9b640 1 --2- 172.21.15.120:0/4090497825 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f84480a3240 0x7f84480a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:23.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.262+0000 7f84537fe640 1 -- 172.21.15.120:0/4090497825 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f84540cfb40 con 0x7f845c100ad0 2024-09-19T09:02:23.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.262+0000 7f8459d9b640 1 --2- 172.21.15.120:0/4090497825 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f84480a3240 0x7f84480a5700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f845c06b050 tx=0x7f8444008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:23.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.266+0000 7f84537fe640 1 -- 172.21.15.120:0/4090497825 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8454014070 con 0x7f845c100ad0 2024-09-19T09:02:23.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.434+0000 7f8460901640 1 -- 172.21.15.120:0/4090497825 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f845c062ec0 con 0x7f845c100ad0 2024-09-19T09:02:23.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.435+0000 7f84537fe640 1 -- 172.21.15.120:0/4090497825 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v81) ==== 56+0+5342 (secure 0 0 0) 0x7f845409c860 con 0x7f845c100ad0 2024-09-19T09:02:23.435 INFO:teuthology.orchestra.run.smithi120.stdout:epoch 81 2024-09-19T09:02:23.435 INFO:teuthology.orchestra.run.smithi120.stdout:fsid 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T09:02:23.435 INFO:teuthology.orchestra.run.smithi120.stdout:created 2024-09-19T08:53:06.386526+0000 2024-09-19T09:02:23.435 INFO:teuthology.orchestra.run.smithi120.stdout:modified 2024-09-19T09:01:52.744203+0000 2024-09-19T09:02:23.436 INFO:teuthology.orchestra.run.smithi120.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-19T09:02:23.436 INFO:teuthology.orchestra.run.smithi120.stdout:crush_version 26 2024-09-19T09:02:23.436 INFO:teuthology.orchestra.run.smithi120.stdout:full_ratio 0.95 2024-09-19T09:02:23.436 INFO:teuthology.orchestra.run.smithi120.stdout:backfillfull_ratio 0.9 2024-09-19T09:02:23.436 INFO:teuthology.orchestra.run.smithi120.stdout:nearfull_ratio 0.85 2024-09-19T09:02:23.436 INFO:teuthology.orchestra.run.smithi120.stdout:require_min_compat_client luminous 2024-09-19T09:02:23.436 INFO:teuthology.orchestra.run.smithi120.stdout:min_compat_client luminous 2024-09-19T09:02:23.436 INFO:teuthology.orchestra.run.smithi120.stdout:require_osd_release squid 2024-09-19T09:02:23.436 INFO:teuthology.orchestra.run.smithi120.stdout:stretch_mode_enabled false 2024-09-19T09:02:23.436 INFO:teuthology.orchestra.run.smithi120.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-19T09:02:23.436 INFO:teuthology.orchestra.run.smithi120.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-19T09:02:23.436 INFO:teuthology.orchestra.run.smithi120.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-19T09:02:23.437 INFO:teuthology.orchestra.run.smithi120.stdout:max_osd 12 2024-09-19T09:02:23.437 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0 up in weight 1 up_from 8 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6802/4131180019,v1:172.21.15.120:6803/4131180019] [v2:172.21.15.120:6804/4131180019,v1:172.21.15.120:6805/4131180019] exists,up 33ec3993-576c-428c-bf75-adcb17cd72a8 2024-09-19T09:02:23.437 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1 up in weight 1 up_from 13 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6800/2259730294,v1:172.21.15.153:6801/2259730294] [v2:172.21.15.153:6802/2259730294,v1:172.21.15.153:6803/2259730294] exists,up 450bbebf-2aac-47c6-8474-bbd37fd0a1a0 2024-09-19T09:02:23.437 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2 up in weight 1 up_from 18 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6800/2942286570,v1:172.21.15.196:6801/2942286570] [v2:172.21.15.196:6802/2942286570,v1:172.21.15.196:6803/2942286570] exists,up ac8c4b83-8e8e-4803-b266-262f7873bd85 2024-09-19T09:02:23.437 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3 up in weight 1 up_from 24 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6810/713205668,v1:172.21.15.120:6811/713205668] [v2:172.21.15.120:6812/713205668,v1:172.21.15.120:6813/713205668] exists,up c1d9e507-8f02-403e-87da-9b4966fd8fab 2024-09-19T09:02:23.437 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4 up in weight 1 up_from 31 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6808/2680865854,v1:172.21.15.153:6809/2680865854] [v2:172.21.15.153:6810/2680865854,v1:172.21.15.153:6811/2680865854] exists,up 7f8bf3b9-0ec4-46f6-a406-d26d7971179e 2024-09-19T09:02:23.437 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5 up in weight 1 up_from 36 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6808/3763331116,v1:172.21.15.196:6809/3763331116] [v2:172.21.15.196:6810/3763331116,v1:172.21.15.196:6811/3763331116] exists,up 70b270b2-209d-4ff5-875a-04ee7b992a93 2024-09-19T09:02:23.437 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6 up in weight 1 up_from 43 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6818/4171773765,v1:172.21.15.120:6819/4171773765] [v2:172.21.15.120:6820/4171773765,v1:172.21.15.120:6821/4171773765] exists,up 8087f69c-dc31-4522-a18d-4e6ab1b7cb27 2024-09-19T09:02:23.437 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7 up in weight 1 up_from 48 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6816/2535148453,v1:172.21.15.153:6817/2535148453] [v2:172.21.15.153:6818/2535148453,v1:172.21.15.153:6819/2535148453] exists,up c3b1ead3-150f-43fd-8924-f639203f4cdd 2024-09-19T09:02:23.437 INFO:teuthology.orchestra.run.smithi120.stdout:osd.8 up in weight 1 up_from 55 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6816/186195906,v1:172.21.15.196:6817/186195906] [v2:172.21.15.196:6818/186195906,v1:172.21.15.196:6819/186195906] exists,up a3273a2a-968e-4e93-a9cf-3879892bdeab 2024-09-19T09:02:23.437 INFO:teuthology.orchestra.run.smithi120.stdout:osd.9 up in weight 1 up_from 62 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6826/1559320883,v1:172.21.15.120:6827/1559320883] [v2:172.21.15.120:6828/1559320883,v1:172.21.15.120:6829/1559320883] exists,up 6d184513-dab8-45f7-bc08-fa2e8b2f10b7 2024-09-19T09:02:23.437 INFO:teuthology.orchestra.run.smithi120.stdout:osd.10 up in weight 1 up_from 66 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6824/3216877560,v1:172.21.15.153:6825/3216877560] [v2:172.21.15.153:6826/3216877560,v1:172.21.15.153:6827/3216877560] exists,up 797db886-62f9-4316-a45b-4d50512e65d4 2024-09-19T09:02:23.437 INFO:teuthology.orchestra.run.smithi120.stdout:osd.11 up in weight 1 up_from 73 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6824/1857200740,v1:172.21.15.196:6825/1857200740] [v2:172.21.15.196:6826/1857200740,v1:172.21.15.196:6827/1857200740] exists,up 17e0afaa-84eb-4b26-9810-bfa3cc7e0f3c 2024-09-19T09:02:23.438 INFO:teuthology.orchestra.run.smithi120.stdout:pg_upmap_items 2.10 [2,1] 2024-09-19T09:02:23.438 INFO:teuthology.orchestra.run.smithi120.stdout:pg_upmap_items 2.16 [2,4] 2024-09-19T09:02:23.438 INFO:teuthology.orchestra.run.smithi120.stdout:pg_upmap_items 2.1c [8,11] 2024-09-19T09:02:23.438 INFO:teuthology.orchestra.run.smithi120.stdout:pg_upmap_items 2.1f [7,11] 2024-09-19T09:02:23.438 INFO:teuthology.orchestra.run.smithi120.stdout:pg_upmap_items 3.7 [3,9] 2024-09-19T09:02:23.438 INFO:teuthology.orchestra.run.smithi120.stdout:pg_upmap_items 3.10 [5,9] 2024-09-19T09:02:23.438 INFO:teuthology.orchestra.run.smithi120.stdout:pg_upmap_items 3.11 [4,6] 2024-09-19T09:02:23.438 INFO:teuthology.orchestra.run.smithi120.stdout:pg_upmap_items 3.1b [4,8] 2024-09-19T09:02:23.438 INFO:teuthology.orchestra.run.smithi120.stdout:pg_upmap_items 3.36 [4,9] 2024-09-19T09:02:23.438 INFO:teuthology.orchestra.run.smithi120.stdout:blocklist 172.21.15.120:0/3010783707 expires 2024-09-20T08:53:51.528772+0000 2024-09-19T09:02:23.438 INFO:teuthology.orchestra.run.smithi120.stdout:blocklist 172.21.15.120:6801/3562882246 expires 2024-09-20T08:53:51.528772+0000 2024-09-19T09:02:23.438 INFO:teuthology.orchestra.run.smithi120.stdout:blocklist 172.21.15.120:6800/3562882246 expires 2024-09-20T08:53:51.528772+0000 2024-09-19T09:02:23.438 INFO:teuthology.orchestra.run.smithi120.stdout:blocklist 172.21.15.120:0/3224512035 expires 2024-09-20T08:53:51.528772+0000 2024-09-19T09:02:23.438 INFO:teuthology.orchestra.run.smithi120.stdout:blocklist 172.21.15.120:0/3465105835 expires 2024-09-20T08:53:28.144376+0000 2024-09-19T09:02:23.438 INFO:teuthology.orchestra.run.smithi120.stdout:blocklist 172.21.15.120:0/3317449610 expires 2024-09-20T08:53:28.144376+0000 2024-09-19T09:02:23.439 INFO:teuthology.orchestra.run.smithi120.stdout:blocklist 172.21.15.120:0/2800697451 expires 2024-09-20T08:53:28.144376+0000 2024-09-19T09:02:23.439 INFO:teuthology.orchestra.run.smithi120.stdout:blocklist 172.21.15.120:6801/2938446665 expires 2024-09-20T08:53:28.144376+0000 2024-09-19T09:02:23.439 INFO:teuthology.orchestra.run.smithi120.stdout:blocklist 172.21.15.120:0/109127432 expires 2024-09-20T08:53:51.528772+0000 2024-09-19T09:02:23.439 INFO:teuthology.orchestra.run.smithi120.stdout:blocklist 172.21.15.120:6800/2938446665 expires 2024-09-20T08:53:28.144376+0000 2024-09-19T09:02:23.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.439+0000 7f8460901640 1 -- 172.21.15.120:0/4090497825 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f84480a3240 msgr2=0x7f84480a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:23.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.439+0000 7f8460901640 1 --2- 172.21.15.120:0/4090497825 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f84480a3240 0x7f84480a5700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f845c06b050 tx=0x7f8444008040 comp rx=0 tx=0).stop 2024-09-19T09:02:23.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.439+0000 7f8460901640 1 -- 172.21.15.120:0/4090497825 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f845c100ad0 msgr2=0x7f845c06aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:23.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.439+0000 7f8460901640 1 --2- 172.21.15.120:0/4090497825 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f845c100ad0 0x7f845c06aad0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f845400ccb0 tx=0x7f845400a620 comp rx=0 tx=0).stop 2024-09-19T09:02:23.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.439+0000 7f8460901640 1 -- 172.21.15.120:0/4090497825 shutdown_connections 2024-09-19T09:02:23.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.439+0000 7f8460901640 1 --2- 172.21.15.120:0/4090497825 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f84480a3240 0x7f84480a5700 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:23.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.439+0000 7f8460901640 1 --2- 172.21.15.120:0/4090497825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f845c102690 0x7f845c19f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:23.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.439+0000 7f8460901640 1 --2- 172.21.15.120:0/4090497825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f845c101cd0 0x7f845c19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:23.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.439+0000 7f8460901640 1 --2- 172.21.15.120:0/4090497825 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f845c100ad0 0x7f845c06aad0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:23.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.439+0000 7f8460901640 1 -- 172.21.15.120:0/4090497825 >> 172.21.15.120:0/4090497825 conn(0x7f845c0fc280 msgr2=0x7f845c104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:23.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.439+0000 7f8460901640 1 -- 172.21.15.120:0/4090497825 shutdown_connections 2024-09-19T09:02:23.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:23.439+0000 7f8460901640 1 -- 172.21.15.120:0/4090497825 wait complete. 2024-09-19T09:02:24.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:23 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/4090497825' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T09:02:24.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:23 smithi120 ceph-mon[31150]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-19T09:02:24.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:23 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/4090497825' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T09:02:24.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:23 smithi196 ceph-mon[35150]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-19T09:02:24.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:23 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/4090497825' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T09:02:24.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:23 smithi153 ceph-mon[35071]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-19T09:02:24.196 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-19T09:02:24.207 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:02:24.452 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:24.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.933+0000 7f5c36db0640 1 -- 172.21.15.120:0/1749233338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c3010c1a0 msgr2=0x7f5c3010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:24.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.933+0000 7f5c36db0640 1 --2- 172.21.15.120:0/1749233338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c3010c1a0 0x7f5c3010e610 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5c28009ca0 tx=0x7f5c2802a950 comp rx=0 tx=0).stop 2024-09-19T09:02:24.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.933+0000 7f5c36db0640 1 -- 172.21.15.120:0/1749233338 shutdown_connections 2024-09-19T09:02:24.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.933+0000 7f5c36db0640 1 --2- 172.21.15.120:0/1749233338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c3010c1a0 0x7f5c3010e610 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:24.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.933+0000 7f5c36db0640 1 --2- 172.21.15.120:0/1749233338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c30103720 0x7f5c3010bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:24.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.933+0000 7f5c36db0640 1 --2- 172.21.15.120:0/1749233338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c30102e00 0x7f5c301031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:24.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.933+0000 7f5c36db0640 1 -- 172.21.15.120:0/1749233338 >> 172.21.15.120:0/1749233338 conn(0x7f5c300fc7a0 msgr2=0x7f5c300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:24.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.933+0000 7f5c36db0640 1 -- 172.21.15.120:0/1749233338 shutdown_connections 2024-09-19T09:02:24.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.933+0000 7f5c36db0640 1 -- 172.21.15.120:0/1749233338 wait complete. 2024-09-19T09:02:24.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.933+0000 7f5c36db0640 1 Processor -- start 2024-09-19T09:02:24.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.933+0000 7f5c36db0640 1 -- start start 2024-09-19T09:02:24.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.934+0000 7f5c36db0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c30102e00 0x7f5c301983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:24.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.934+0000 7f5c36db0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c30103720 0x7f5c30198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:24.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.934+0000 7f5c36db0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c3010c1a0 0x7f5c3019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:24.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.934+0000 7f5c36db0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5c30111fb0 con 0x7f5c30102e00 2024-09-19T09:02:24.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.934+0000 7f5c36db0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5c30111e30 con 0x7f5c30103720 2024-09-19T09:02:24.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.934+0000 7f5c36db0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5c30112130 con 0x7f5c3010c1a0 2024-09-19T09:02:24.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.934+0000 7f5c34b4c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c30102e00 0x7f5c301983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:24.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.934+0000 7f5c34b4c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c30102e00 0x7f5c301983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49868/0 (socket says 172.21.15.120:49868) 2024-09-19T09:02:24.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.934+0000 7f5c34b4c640 1 -- 172.21.15.120:0/1324537699 learned_addr learned my addr 172.21.15.120:0/1324537699 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:24.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.934+0000 7f5c3534d640 1 --2- 172.21.15.120:0/1324537699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c3010c1a0 0x7f5c3019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:24.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.934+0000 7f5c34b4c640 1 -- 172.21.15.120:0/1324537699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c3010c1a0 msgr2=0x7f5c3019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:24.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.934+0000 7f5c34b4c640 1 --2- 172.21.15.120:0/1324537699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c3010c1a0 0x7f5c3019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:24.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.934+0000 7f5c34b4c640 1 -- 172.21.15.120:0/1324537699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c30103720 msgr2=0x7f5c30198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:24.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.934+0000 7f5c2ffff640 1 --2- 172.21.15.120:0/1324537699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c30103720 0x7f5c30198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:24.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.934+0000 7f5c34b4c640 1 --2- 172.21.15.120:0/1324537699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c30103720 0x7f5c30198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:24.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.934+0000 7f5c34b4c640 1 -- 172.21.15.120:0/1324537699 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c3019d3a0 con 0x7f5c30102e00 2024-09-19T09:02:24.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.934+0000 7f5c34b4c640 1 --2- 172.21.15.120:0/1324537699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c30102e00 0x7f5c301983f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5c2400b4a0 tx=0x7f5c2400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:24.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.934+0000 7f5c2dffb640 1 -- 172.21.15.120:0/1324537699 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c24014070 con 0x7f5c30102e00 2024-09-19T09:02:24.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.934+0000 7f5c2dffb640 1 -- 172.21.15.120:0/1324537699 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c24010af0 con 0x7f5c30102e00 2024-09-19T09:02:24.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.934+0000 7f5c36db0640 1 -- 172.21.15.120:0/1324537699 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c3019d690 con 0x7f5c30102e00 2024-09-19T09:02:24.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.934+0000 7f5c2dffb640 1 -- 172.21.15.120:0/1324537699 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c24017d10 con 0x7f5c30102e00 2024-09-19T09:02:24.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.935+0000 7f5c36db0640 1 -- 172.21.15.120:0/1324537699 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c301a4f70 con 0x7f5c30102e00 2024-09-19T09:02:24.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.936+0000 7f5c36db0640 1 -- 172.21.15.120:0/1324537699 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c301031e0 con 0x7f5c30102e00 2024-09-19T09:02:24.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.937+0000 7f5c2dffb640 1 -- 172.21.15.120:0/1324537699 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f5c24017070 con 0x7f5c30102e00 2024-09-19T09:02:24.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.937+0000 7f5c2dffb640 1 --2- 172.21.15.120:0/1324537699 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5c200a3240 0x7f5c200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:24.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.937+0000 7f5c2ffff640 1 --2- 172.21.15.120:0/1324537699 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5c200a3240 0x7f5c200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:24.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.937+0000 7f5c2dffb640 1 -- 172.21.15.120:0/1324537699 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f5c240ced90 con 0x7f5c30102e00 2024-09-19T09:02:24.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.938+0000 7f5c2ffff640 1 --2- 172.21.15.120:0/1324537699 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5c200a3240 0x7f5c200a5700 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5c3010d4c0 tx=0x7f5c1c005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:24.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:24.941+0000 7f5c2dffb640 1 -- 172.21.15.120:0/1324537699 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5c2409bab0 con 0x7f5c30102e00 2024-09-19T09:02:25.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:25.141+0000 7f5c36db0640 1 -- 172.21.15.120:0/1324537699 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c301a5170 con 0x7f5c30102e00 2024-09-19T09:02:25.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:25.142+0000 7f5c2dffb640 1 -- 172.21.15.120:0/1324537699 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11988 (secure 0 0 0) 0x7f5c2409b3b0 con 0x7f5c30102e00 2024-09-19T09:02:25.142 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:02:25.142 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":27,"btime":"2024-09-19T09:00:59:134373+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:00:59.134353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:02:25.142 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 27 2024-09-19T09:02:25.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:25.144+0000 7f5c36db0640 1 -- 172.21.15.120:0/1324537699 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5c200a3240 msgr2=0x7f5c200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:25.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:25.144+0000 7f5c36db0640 1 --2- 172.21.15.120:0/1324537699 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5c200a3240 0x7f5c200a5700 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5c3010d4c0 tx=0x7f5c1c005cc0 comp rx=0 tx=0).stop 2024-09-19T09:02:25.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:25.144+0000 7f5c36db0640 1 -- 172.21.15.120:0/1324537699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c30102e00 msgr2=0x7f5c301983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:25.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:25.144+0000 7f5c36db0640 1 --2- 172.21.15.120:0/1324537699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c30102e00 0x7f5c301983f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5c2400b4a0 tx=0x7f5c2400b960 comp rx=0 tx=0).stop 2024-09-19T09:02:25.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:25.144+0000 7f5c36db0640 1 -- 172.21.15.120:0/1324537699 shutdown_connections 2024-09-19T09:02:25.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:25.144+0000 7f5c36db0640 1 --2- 172.21.15.120:0/1324537699 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5c200a3240 0x7f5c200a5700 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:25.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:25.144+0000 7f5c36db0640 1 --2- 172.21.15.120:0/1324537699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c3010c1a0 0x7f5c3019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:25.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:25.144+0000 7f5c36db0640 1 --2- 172.21.15.120:0/1324537699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c30103720 0x7f5c30198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:25.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:25.144+0000 7f5c36db0640 1 --2- 172.21.15.120:0/1324537699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c30102e00 0x7f5c301983f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:25.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:25.144+0000 7f5c36db0640 1 -- 172.21.15.120:0/1324537699 >> 172.21.15.120:0/1324537699 conn(0x7f5c300fc7a0 msgr2=0x7f5c300fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:25.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:25.144+0000 7f5c36db0640 1 -- 172.21.15.120:0/1324537699 shutdown_connections 2024-09-19T09:02:25.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:25.144+0000 7f5c36db0640 1 -- 172.21.15.120:0/1324537699 wait complete. 2024-09-19T09:02:25.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:25 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1324537699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:02:25.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:25 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1324537699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:02:25.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:25 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1324537699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:02:25.834 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:02:26.078 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:26.465 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:26 smithi120 ceph-mon[31150]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-19T09:02:26.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.545+0000 7ff2ccc31640 1 -- 172.21.15.120:0/552019946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2c8100e20 msgr2=0x7ff2c8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:26.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.545+0000 7ff2ccc31640 1 --2- 172.21.15.120:0/552019946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2c8100e20 0x7ff2c8101200 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8009900 tx=0x7ff2b802a480 comp rx=0 tx=0).stop 2024-09-19T09:02:26.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.545+0000 7ff2ccc31640 1 -- 172.21.15.120:0/552019946 shutdown_connections 2024-09-19T09:02:26.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.545+0000 7ff2ccc31640 1 --2- 172.21.15.120:0/552019946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2c8102ef0 0x7ff2c810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:26.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.545+0000 7ff2ccc31640 1 --2- 172.21.15.120:0/552019946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2c8102570 0x7ff2c81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:26.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.545+0000 7ff2ccc31640 1 --2- 172.21.15.120:0/552019946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2c8100e20 0x7ff2c8101200 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:26.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.545+0000 7ff2ccc31640 1 -- 172.21.15.120:0/552019946 >> 172.21.15.120:0/552019946 conn(0x7ff2c80fc7d0 msgr2=0x7ff2c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:26.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.545+0000 7ff2ccc31640 1 -- 172.21.15.120:0/552019946 shutdown_connections 2024-09-19T09:02:26.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.545+0000 7ff2ccc31640 1 -- 172.21.15.120:0/552019946 wait complete. 2024-09-19T09:02:26.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.546+0000 7ff2ccc31640 1 Processor -- start 2024-09-19T09:02:26.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.546+0000 7ff2ccc31640 1 -- start start 2024-09-19T09:02:26.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.546+0000 7ff2ccc31640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2c8100e20 0x7ff2c8199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:26.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.546+0000 7ff2ccc31640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2c8102570 0x7ff2c8199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:26.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.546+0000 7ff2ccc31640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2c8102ef0 0x7ff2c81a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:26.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.546+0000 7ff2ccc31640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c8110f80 con 0x7ff2c8102570 2024-09-19T09:02:26.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.546+0000 7ff2ccc31640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c8110e00 con 0x7ff2c8102ef0 2024-09-19T09:02:26.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.546+0000 7ff2ccc31640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c8111100 con 0x7ff2c8100e20 2024-09-19T09:02:26.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.546+0000 7ff2c5d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2c8102570 0x7ff2c8199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:26.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.546+0000 7ff2c5d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2c8102570 0x7ff2c8199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49896/0 (socket says 172.21.15.120:49896) 2024-09-19T09:02:26.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.546+0000 7ff2c5d9b640 1 -- 172.21.15.120:0/3434602944 learned_addr learned my addr 172.21.15.120:0/3434602944 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:26.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.546+0000 7ff2c5d9b640 1 -- 172.21.15.120:0/3434602944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2c8100e20 msgr2=0x7ff2c8199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:26.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.546+0000 7ff2c5d9b640 1 --2- 172.21.15.120:0/3434602944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2c8100e20 0x7ff2c8199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:26.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.546+0000 7ff2c6d9d640 1 --2- 172.21.15.120:0/3434602944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2c8102ef0 0x7ff2c81a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:26.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.546+0000 7ff2c5d9b640 1 -- 172.21.15.120:0/3434602944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2c8102ef0 msgr2=0x7ff2c81a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:26.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.546+0000 7ff2c5d9b640 1 --2- 172.21.15.120:0/3434602944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2c8102ef0 0x7ff2c81a43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:26.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.546+0000 7ff2c5d9b640 1 -- 172.21.15.120:0/3434602944 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c81a48f0 con 0x7ff2c8102570 2024-09-19T09:02:26.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.546+0000 7ff2c6d9d640 1 --2- 172.21.15.120:0/3434602944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2c8102ef0 0x7ff2c81a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:02:26.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.547+0000 7ff2c5d9b640 1 --2- 172.21.15.120:0/3434602944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2c8102570 0x7ff2c8199870 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000c890 tx=0x7ff2b000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:26.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.547+0000 7ff2bf7fe640 1 -- 172.21.15.120:0/3434602944 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0004220 con 0x7ff2c8102570 2024-09-19T09:02:26.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.547+0000 7ff2ccc31640 1 -- 172.21.15.120:0/3434602944 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c81a4ac0 con 0x7ff2c8102570 2024-09-19T09:02:26.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.547+0000 7ff2bf7fe640 1 -- 172.21.15.120:0/3434602944 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2b00043c0 con 0x7ff2c8102570 2024-09-19T09:02:26.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.547+0000 7ff2bf7fe640 1 -- 172.21.15.120:0/3434602944 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b00049e0 con 0x7ff2c8102570 2024-09-19T09:02:26.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.547+0000 7ff2ccc31640 1 -- 172.21.15.120:0/3434602944 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c81a4fd0 con 0x7ff2c8102570 2024-09-19T09:02:26.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.548+0000 7ff2ccc31640 1 -- 172.21.15.120:0/3434602944 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c8101200 con 0x7ff2c8102570 2024-09-19T09:02:26.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.549+0000 7ff2bf7fe640 1 -- 172.21.15.120:0/3434602944 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7ff2b00079b0 con 0x7ff2c8102570 2024-09-19T09:02:26.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.550+0000 7ff2bf7fe640 1 --2- 172.21.15.120:0/3434602944 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff2b40a3240 0x7ff2b40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:26.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.550+0000 7ff2c659c640 1 --2- 172.21.15.120:0/3434602944 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff2b40a3240 0x7ff2b40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:26.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.550+0000 7ff2bf7fe640 1 -- 172.21.15.120:0/3434602944 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7ff2b00cfa50 con 0x7ff2c8102570 2024-09-19T09:02:26.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.550+0000 7ff2c659c640 1 --2- 172.21.15.120:0/3434602944 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff2b40a3240 0x7ff2b40a5700 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80096c0 tx=0x7ff2b8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:26.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.553+0000 7ff2bf7fe640 1 -- 172.21.15.120:0/3434602944 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff2b0014030 con 0x7ff2c8102570 2024-09-19T09:02:26.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:26 smithi196 ceph-mon[35150]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-19T09:02:26.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:26 smithi153 ceph-mon[35071]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-19T09:02:26.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.756+0000 7ff2ccc31640 1 -- 172.21.15.120:0/3434602944 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2c806a560 con 0x7ff2c8102570 2024-09-19T09:02:26.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.756+0000 7ff2bf7fe640 1 -- 172.21.15.120:0/3434602944 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11988 (secure 0 0 0) 0x7ff2c806a560 con 0x7ff2c8102570 2024-09-19T09:02:26.758 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:02:26.758 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":27,"btime":"2024-09-19T09:00:59:134373+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:00:59.134353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:02:26.758 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 27 2024-09-19T09:02:26.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.759+0000 7ff2ccc31640 1 -- 172.21.15.120:0/3434602944 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff2b40a3240 msgr2=0x7ff2b40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:26.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.759+0000 7ff2ccc31640 1 --2- 172.21.15.120:0/3434602944 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff2b40a3240 0x7ff2b40a5700 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80096c0 tx=0x7ff2b8000bc0 comp rx=0 tx=0).stop 2024-09-19T09:02:26.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.759+0000 7ff2ccc31640 1 -- 172.21.15.120:0/3434602944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2c8102570 msgr2=0x7ff2c8199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:26.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.759+0000 7ff2ccc31640 1 --2- 172.21.15.120:0/3434602944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2c8102570 0x7ff2c8199870 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000c890 tx=0x7ff2b000cd50 comp rx=0 tx=0).stop 2024-09-19T09:02:26.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.759+0000 7ff2ccc31640 1 -- 172.21.15.120:0/3434602944 shutdown_connections 2024-09-19T09:02:26.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.759+0000 7ff2ccc31640 1 --2- 172.21.15.120:0/3434602944 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff2b40a3240 0x7ff2b40a5700 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:26.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.759+0000 7ff2ccc31640 1 --2- 172.21.15.120:0/3434602944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2c8102ef0 0x7ff2c81a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:26.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.759+0000 7ff2ccc31640 1 --2- 172.21.15.120:0/3434602944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2c8102570 0x7ff2c8199870 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:26.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.759+0000 7ff2ccc31640 1 --2- 172.21.15.120:0/3434602944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2c8100e20 0x7ff2c8199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:26.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.759+0000 7ff2ccc31640 1 -- 172.21.15.120:0/3434602944 >> 172.21.15.120:0/3434602944 conn(0x7ff2c80fc7d0 msgr2=0x7ff2c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:26.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.759+0000 7ff2ccc31640 1 -- 172.21.15.120:0/3434602944 shutdown_connections 2024-09-19T09:02:26.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:26.759+0000 7ff2ccc31640 1 -- 172.21.15.120:0/3434602944 wait complete. 2024-09-19T09:02:27.496 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:27 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3434602944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:02:27.526 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:02:27.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:27 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3434602944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:02:27.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:27 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3434602944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:02:27.773 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:28.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.238+0000 7f2a5a609640 1 -- 172.21.15.120:0/582891346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a54106f10 msgr2=0x7f2a5410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:28.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.238+0000 7f2a5a609640 1 --2- 172.21.15.120:0/582891346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a54106f10 0x7f2a5410ea90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009ca0 tx=0x7f2a4c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:02:28.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.238+0000 7f2a5a609640 1 -- 172.21.15.120:0/582891346 shutdown_connections 2024-09-19T09:02:28.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.238+0000 7f2a5a609640 1 --2- 172.21.15.120:0/582891346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a54106f10 0x7f2a5410ea90 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:28.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.238+0000 7f2a5a609640 1 --2- 172.21.15.120:0/582891346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a54102790 0x7f2a541067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.238+0000 7f2a5a609640 1 --2- 172.21.15.120:0/582891346 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a54101de0 0x7f2a541021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.238+0000 7f2a5a609640 1 -- 172.21.15.120:0/582891346 >> 172.21.15.120:0/582891346 conn(0x7f2a540fd8c0 msgr2=0x7f2a540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.238+0000 7f2a5a609640 1 -- 172.21.15.120:0/582891346 shutdown_connections 2024-09-19T09:02:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.238+0000 7f2a5a609640 1 -- 172.21.15.120:0/582891346 wait complete. 2024-09-19T09:02:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.239+0000 7f2a5a609640 1 Processor -- start 2024-09-19T09:02:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.239+0000 7f2a5a609640 1 -- start start 2024-09-19T09:02:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.239+0000 7f2a5a609640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a54101de0 0x7f2a541985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.239+0000 7f2a5a609640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a54102790 0x7f2a54198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.239+0000 7f2a5a609640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a54106f10 0x7f2a5419cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.239+0000 7f2a5a609640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2a54112010 con 0x7f2a54102790 2024-09-19T09:02:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.239+0000 7f2a5a609640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2a54111e90 con 0x7f2a54101de0 2024-09-19T09:02:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.239+0000 7f2a5a609640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a54112190 con 0x7f2a54106f10 2024-09-19T09:02:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.239+0000 7f2a537fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a54102790 0x7f2a54198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:28.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.239+0000 7f2a537fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a54102790 0x7f2a54198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49914/0 (socket says 172.21.15.120:49914) 2024-09-19T09:02:28.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.239+0000 7f2a537fe640 1 -- 172.21.15.120:0/3694857466 learned_addr learned my addr 172.21.15.120:0/3694857466 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:28.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.239+0000 7f2a53fff640 1 --2- 172.21.15.120:0/3694857466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a54101de0 0x7f2a541985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:28.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.239+0000 7f2a58ba6640 1 --2- 172.21.15.120:0/3694857466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a54106f10 0x7f2a5419cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:28.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.239+0000 7f2a537fe640 1 -- 172.21.15.120:0/3694857466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a54106f10 msgr2=0x7f2a5419cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:28.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.239+0000 7f2a537fe640 1 --2- 172.21.15.120:0/3694857466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a54106f10 0x7f2a5419cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:28.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.239+0000 7f2a537fe640 1 -- 172.21.15.120:0/3694857466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a54101de0 msgr2=0x7f2a541985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:28.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.239+0000 7f2a537fe640 1 --2- 172.21.15.120:0/3694857466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a54101de0 0x7f2a541985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:28.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.239+0000 7f2a537fe640 1 -- 172.21.15.120:0/3694857466 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5419d5a0 con 0x7f2a54102790 2024-09-19T09:02:28.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.239+0000 7f2a53fff640 1 --2- 172.21.15.120:0/3694857466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a54101de0 0x7f2a541985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:02:28.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.240+0000 7f2a537fe640 1 --2- 172.21.15.120:0/3694857466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a54102790 0x7f2a54198b30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800b4a0 tx=0x7f2a4800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:28.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.240+0000 7f2a517fa640 1 -- 172.21.15.120:0/3694857466 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a48014070 con 0x7f2a54102790 2024-09-19T09:02:28.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.240+0000 7f2a517fa640 1 -- 172.21.15.120:0/3694857466 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a48010af0 con 0x7f2a54102790 2024-09-19T09:02:28.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.240+0000 7f2a5a609640 1 -- 172.21.15.120:0/3694857466 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a5419d890 con 0x7f2a54102790 2024-09-19T09:02:28.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.240+0000 7f2a517fa640 1 -- 172.21.15.120:0/3694857466 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a48017d10 con 0x7f2a54102790 2024-09-19T09:02:28.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.240+0000 7f2a5a609640 1 -- 172.21.15.120:0/3694857466 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a541a5170 con 0x7f2a54102790 2024-09-19T09:02:28.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.241+0000 7f2a5a609640 1 -- 172.21.15.120:0/3694857466 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a54102240 con 0x7f2a54102790 2024-09-19T09:02:28.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.242+0000 7f2a517fa640 1 -- 172.21.15.120:0/3694857466 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f2a48017070 con 0x7f2a54102790 2024-09-19T09:02:28.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.242+0000 7f2a517fa640 1 --2- 172.21.15.120:0/3694857466 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2a3c0a32c0 0x7f2a3c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:28.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.243+0000 7f2a53fff640 1 --2- 172.21.15.120:0/3694857466 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2a3c0a32c0 0x7f2a3c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:28.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.243+0000 7f2a517fa640 1 -- 172.21.15.120:0/3694857466 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f2a480ced20 con 0x7f2a54102790 2024-09-19T09:02:28.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.243+0000 7f2a53fff640 1 --2- 172.21.15.120:0/3694857466 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2a3c0a32c0 0x7f2a3c0a5780 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2a44009930 tx=0x7f2a44008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:28.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.247+0000 7f2a517fa640 1 -- 172.21.15.120:0/3694857466 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2a4809ba40 con 0x7f2a54102790 2024-09-19T09:02:28.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.417+0000 7f2a5a609640 1 -- 172.21.15.120:0/3694857466 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a5410a0f0 con 0x7f2a54102790 2024-09-19T09:02:28.419 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:28 smithi120 ceph-mon[31150]: pgmap v329: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-19T09:02:28.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.419+0000 7f2a517fa640 1 -- 172.21.15.120:0/3694857466 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22844 (secure 0 0 0) 0x7f2a4809b340 con 0x7f2a54102790 2024-09-19T09:02:28.420 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:02:28.420 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":81,"fsid":"507393d2-7664-11ef-bceb-c7b262605968","created":"2024-09-19T08:53:06.386526+0000","modified":"2024-09-19T09:01:52.744203+0000","last_up_change":"2024-09-19T08:59:55.692659+0000","last_in_change":"2024-09-19T08:59:32.808860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:56:07.830047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:00:43.774339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:00:45.786262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33ec3993-576c-428c-bf75-adcb17cd72a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6803","nonce":4131180019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6805","nonce":4131180019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6809","nonce":4131180019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6807","nonce":4131180019}]},"public_addr":"172.21.15.120:6803/4131180019","cluster_addr":"172.21.15.120:6805/4131180019","heartbeat_back_addr":"172.21.15.120:6809/4131180019","heartbeat_front_addr":"172.21.15.120:6807/4131180019","state":["exists","up"]},{"osd":1,"uuid":"450bbebf-2aac-47c6-8474-bbd37fd0a1a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6801","nonce":2259730294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6803","nonce":2259730294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6807","nonce":2259730294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6805","nonce":2259730294}]},"public_addr":"172.21.15.153:6801/2259730294","cluster_addr":"172.21.15.153:6803/2259730294","heartbeat_back_addr":"172.21.15.153:6807/2259730294","heartbeat_front_addr":"172.21.15.153:6805/2259730294","state":["exists","up"]},{"osd":2,"uuid":"ac8c4b83-8e8e-4803-b266-262f7873bd85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6801","nonce":2942286570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6803","nonce":2942286570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6807","nonce":2942286570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6805","nonce":2942286570}]},"public_addr":"172.21.15.196:6801/2942286570","cluster_addr":"172.21.15.196:6803/2942286570","heartbeat_back_addr":"172.21.15.196:6807/2942286570","heartbeat_front_addr":"172.21.15.196:6805/2942286570","state":["exists","up"]},{"osd":3,"uuid":"c1d9e507-8f02-403e-87da-9b4966fd8fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6811","nonce":713205668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6813","nonce":713205668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6817","nonce":713205668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6815","nonce":713205668}]},"public_addr":"172.21.15.120:6811/713205668","cluster_addr":"172.21.15.120:6813/713205668","heartbeat_back_addr":"172.21.15.120:6817/713205668","heartbeat_front_addr":"172.21.15.120:6815/713205668","state":["exists","up"]},{"osd":4,"uuid":"7f8bf3b9-0ec4-46f6-a406-d26d7971179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6809","nonce":2680865854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6811","nonce":2680865854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6815","nonce":2680865854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6813","nonce":2680865854}]},"public_addr":"172.21.15.153:6809/2680865854","cluster_addr":"172.21.15.153:6811/2680865854","heartbeat_back_addr":"172.21.15.153:6815/2680865854","heartbeat_front_addr":"172.21.15.153:6813/2680865854","state":["exists","up"]},{"osd":5,"uuid":"70b270b2-209d-4ff5-875a-04ee7b992a93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6809","nonce":3763331116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6811","nonce":3763331116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6815","nonce":3763331116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6813","nonce":3763331116}]},"public_addr":"172.21.15.196:6809/3763331116","cluster_addr":"172.21.15.196:6811/3763331116","heartbeat_back_addr":"172.21.15.196:6815/3763331116","heartbeat_front_addr":"172.21.15.196:6813/3763331116","state":["exists","up"]},{"osd":6,"uuid":"8087f69c-dc31-4522-a18d-4e6ab1b7cb27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6819","nonce":4171773765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6821","nonce":4171773765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6825","nonce":4171773765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6823","nonce":4171773765}]},"public_addr":"172.21.15.120:6819/4171773765","cluster_addr":"172.21.15.120:6821/4171773765","heartbeat_back_addr":"172.21.15.120:6825/4171773765","heartbeat_front_addr":"172.21.15.120:6823/4171773765","state":["exists","up"]},{"osd":7,"uuid":"c3b1ead3-150f-43fd-8924-f639203f4cdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6817","nonce":2535148453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6819","nonce":2535148453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6823","nonce":2535148453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6821","nonce":2535148453}]},"public_addr":"172.21.15.153:6817/2535148453","cluster_addr":"172.21.15.153:6819/2535148453","heartbeat_back_addr":"172.21.15.153:6823/2535148453","heartbeat_front_addr":"172.21.15.153:6821/2535148453","state":["exists","up"]},{"osd":8,"uuid":"a3273a2a-968e-4e93-a9cf-3879892bdeab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6817","nonce":186195906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6819","nonce":186195906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6823","nonce":186195906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6821","nonce":186195906}]},"public_addr":"172.21.15.196:6817/186195906","cluster_addr":"172.21.15.196:6819/186195906","heartbeat_back_addr":"172.21.15.196:6823/186195906","heartbeat_front_addr":"172.21.15.196:6821/186195906","state":["exists","up"]},{"osd":9,"uuid":"6d184513-dab8-45f7-bc08-fa2e8b2f10b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6827","nonce":1559320883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6829","nonce":1559320883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6833","nonce":1559320883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6831","nonce":1559320883}]},"public_addr":"172.21.15.120:6827/1559320883","cluster_addr":"172.21.15.120:6829/1559320883","heartbeat_back_addr":"172.21.15.120:6833/1559320883","heartbeat_front_addr":"172.21.15.120:6831/1559320883","state":["exists","up"]},{"osd":10,"uuid":"797db886-62f9-4316-a45b-4d50512e65d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6825","nonce":3216877560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6827","nonce":3216877560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6831","nonce":3216877560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6829","nonce":3216877560}]},"public_addr":"172.21.15.153:6825/3216877560","cluster_addr":"172.21.15.153:6827/3216877560","heartbeat_back_addr":"172.21.15.153:6831/3216877560","heartbeat_front_addr":"172.21.15.153:6829/3216877560","state":["exists","up"]},{"osd":11,"uuid":"17e0afaa-84eb-4b26-9810-bfa3cc7e0f3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6825","nonce":1857200740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6827","nonce":1857200740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6831","nonce":1857200740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6829","nonce":1857200740}]},"public_addr":"172.21.15.196:6825/1857200740","cluster_addr":"172.21.15.196:6827/1857200740","heartbeat_back_addr":"172.21.15.196:6831/1857200740","heartbeat_front_addr":"172.21.15.196:6829/1857200740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:55:15.595645+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:55:40.405487+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:04.938034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:29.718696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:55.636029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:57:21.257083+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:57:46.316552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:58:11.946863+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:58:37.098578+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:03.759023+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:28.659049+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:54.350475+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:0/3010783707":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6801/3562882246":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6800/3562882246":"2024-09-20T08:53:51.528772+0000","172.21.15.120:0/3224512035":"2024-09-20T08:53:51.528772+0000","172.21.15.120:0/3465105835":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/3317449610":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/2800697451":"2024-09-20T08:53:28.144376+0000","172.21.15.120:6801/2938446665":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/109127432":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6800/2938446665":"2024-09-20T08:53:28.144376+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:02:28.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.421+0000 7f2a5a609640 1 -- 172.21.15.120:0/3694857466 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2a3c0a32c0 msgr2=0x7f2a3c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:28.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.421+0000 7f2a5a609640 1 --2- 172.21.15.120:0/3694857466 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2a3c0a32c0 0x7f2a3c0a5780 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2a44009930 tx=0x7f2a44008040 comp rx=0 tx=0).stop 2024-09-19T09:02:28.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.421+0000 7f2a5a609640 1 -- 172.21.15.120:0/3694857466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a54102790 msgr2=0x7f2a54198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:28.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.421+0000 7f2a5a609640 1 --2- 172.21.15.120:0/3694857466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a54102790 0x7f2a54198b30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800b4a0 tx=0x7f2a4800b960 comp rx=0 tx=0).stop 2024-09-19T09:02:28.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.421+0000 7f2a5a609640 1 -- 172.21.15.120:0/3694857466 shutdown_connections 2024-09-19T09:02:28.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.421+0000 7f2a5a609640 1 --2- 172.21.15.120:0/3694857466 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2a3c0a32c0 0x7f2a3c0a5780 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:28.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.421+0000 7f2a5a609640 1 --2- 172.21.15.120:0/3694857466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a54106f10 0x7f2a5419cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:28.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.421+0000 7f2a5a609640 1 --2- 172.21.15.120:0/3694857466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a54102790 0x7f2a54198b30 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:28.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.421+0000 7f2a5a609640 1 --2- 172.21.15.120:0/3694857466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a54101de0 0x7f2a541985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:28.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.421+0000 7f2a5a609640 1 -- 172.21.15.120:0/3694857466 >> 172.21.15.120:0/3694857466 conn(0x7f2a540fd8c0 msgr2=0x7f2a540fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:28.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.421+0000 7f2a5a609640 1 -- 172.21.15.120:0/3694857466 shutdown_connections 2024-09-19T09:02:28.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:28.421+0000 7f2a5a609640 1 -- 172.21.15.120:0/3694857466 wait complete. 2024-09-19T09:02:28.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:28 smithi196 ceph-mon[35150]: pgmap v329: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-19T09:02:28.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:28 smithi153 ceph-mon[35071]: pgmap v329: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-19T09:02:29.158 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T09:00:47.659717+0000', 'modified': '2024-09-19T09:00:59.134353+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24461, 'mds_1': 24446, 'mds_2': 24442, 'mds_3': 24430, 'mds_4': 24421}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14601': {'gid': 14601, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.120:6839/1939272010', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6838', 'nonce': 1939272010}, {'type': 'v1', 'addr': '172.21.15.120:6839', 'nonce': 1939272010}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14613': {'gid': 14613, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.120:6841/658447478', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6840', 'nonce': 658447478}, {'type': 'v1', 'addr': '172.21.15.120:6841', 'nonce': 658447478}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24392': {'gid': 24392, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.153:6833/1522304272', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6832', 'nonce': 1522304272}, {'type': 'v1', 'addr': '172.21.15.153:6833', 'nonce': 1522304272}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24400': {'gid': 24400, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6833/1407049222', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 1407049222}, {'type': 'v1', 'addr': '172.21.15.196:6833', 'nonce': 1407049222}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24419': {'gid': 24419, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.153:6835/3103515283', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6834', 'nonce': 3103515283}, {'type': 'v1', 'addr': '172.21.15.153:6835', 'nonce': 3103515283}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.196:6835/2141168828', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 2141168828}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 2141168828}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.196:6837/776889709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6836', 'nonce': 776889709}, {'type': 'v1', 'addr': '172.21.15.196:6837', 'nonce': 776889709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.196:6839/862542635', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6838', 'nonce': 862542635}, {'type': 'v1', 'addr': '172.21.15.196:6839', 'nonce': 862542635}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.153:6837/3048520124', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6836', 'nonce': 3048520124}, {'type': 'v1', 'addr': '172.21.15.153:6837', 'nonce': 3048520124}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.153:6839/3290519661', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6838', 'nonce': 3290519661}, {'type': 'v1', 'addr': '172.21.15.153:6839', 'nonce': 3290519661}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24461, 'qdb_cluster': [24461, 24446, 24442, 24430, 24421]} 2024-09-19T09:02:29.158 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T09:02:29.158 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-19T09:02:29.407 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:29.527 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:29 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3694857466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:02:29.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:29 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3694857466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:02:29.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:29 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3694857466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:02:29.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.883+0000 7f25956cc640 1 -- 172.21.15.120:0/3623810657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2590100a80 msgr2=0x7f2590100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:29.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.883+0000 7f25956cc640 1 --2- 172.21.15.120:0/3623810657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2590100a80 0x7f2590100ee0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2578009930 tx=0x7f257802a420 comp rx=0 tx=0).stop 2024-09-19T09:02:29.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.883+0000 7f25956cc640 1 -- 172.21.15.120:0/3623810657 shutdown_connections 2024-09-19T09:02:29.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.883+0000 7f25956cc640 1 --2- 172.21.15.120:0/3623810657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2590101420 0x7f259010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.883+0000 7f25956cc640 1 --2- 172.21.15.120:0/3623810657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2590100a80 0x7f2590100ee0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.883+0000 7f25956cc640 1 --2- 172.21.15.120:0/3623810657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2590106a80 0x7f2590106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.883+0000 7f25956cc640 1 -- 172.21.15.120:0/3623810657 >> 172.21.15.120:0/3623810657 conn(0x7f25900fc7a0 msgr2=0x7f25900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:29.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.883+0000 7f25956cc640 1 -- 172.21.15.120:0/3623810657 shutdown_connections 2024-09-19T09:02:29.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.883+0000 7f25956cc640 1 -- 172.21.15.120:0/3623810657 wait complete. 2024-09-19T09:02:29.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.884+0000 7f25956cc640 1 Processor -- start 2024-09-19T09:02:29.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.884+0000 7f25956cc640 1 -- start start 2024-09-19T09:02:29.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.884+0000 7f25956cc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2590100a80 0x7f2590198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:29.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.884+0000 7f25956cc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2590101420 0x7f25901989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:29.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.884+0000 7f25956cc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2590106a80 0x7f259019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:29.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.884+0000 7f25956cc640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2590110e80 con 0x7f2590100a80 2024-09-19T09:02:29.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.884+0000 7f25956cc640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2590110d00 con 0x7f2590101420 2024-09-19T09:02:29.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.884+0000 7f25956cc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2590111000 con 0x7f2590106a80 2024-09-19T09:02:29.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.884+0000 7f258effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2590100a80 0x7f2590198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:29.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.884+0000 7f258effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2590100a80 0x7f2590198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49924/0 (socket says 172.21.15.120:49924) 2024-09-19T09:02:29.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.884+0000 7f258effd640 1 -- 172.21.15.120:0/72711119 learned_addr learned my addr 172.21.15.120:0/72711119 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:29.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.884+0000 7f258effd640 1 -- 172.21.15.120:0/72711119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2590106a80 msgr2=0x7f259019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:29.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.884+0000 7f258effd640 1 --2- 172.21.15.120:0/72711119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2590106a80 0x7f259019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.884+0000 7f258effd640 1 -- 172.21.15.120:0/72711119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2590101420 msgr2=0x7f25901989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:29.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.884+0000 7f258effd640 1 --2- 172.21.15.120:0/72711119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2590101420 0x7f25901989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.884+0000 7f258effd640 1 -- 172.21.15.120:0/72711119 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f259019d410 con 0x7f2590100a80 2024-09-19T09:02:29.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.885+0000 7f258effd640 1 --2- 172.21.15.120:0/72711119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2590100a80 0x7f2590198460 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f258000c890 tx=0x7f258000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:29.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.885+0000 7f258cff9640 1 -- 172.21.15.120:0/72711119 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580004220 con 0x7f2590100a80 2024-09-19T09:02:29.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.885+0000 7f25956cc640 1 -- 172.21.15.120:0/72711119 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f259019d700 con 0x7f2590100a80 2024-09-19T09:02:29.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.885+0000 7f258cff9640 1 -- 172.21.15.120:0/72711119 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25800043c0 con 0x7f2590100a80 2024-09-19T09:02:29.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.885+0000 7f25956cc640 1 -- 172.21.15.120:0/72711119 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25901a4fe0 con 0x7f2590100a80 2024-09-19T09:02:29.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.885+0000 7f258cff9640 1 -- 172.21.15.120:0/72711119 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258001d060 con 0x7f2590100a80 2024-09-19T09:02:29.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.887+0000 7f258cff9640 1 -- 172.21.15.120:0/72711119 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f25800079b0 con 0x7f2590100a80 2024-09-19T09:02:29.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.888+0000 7f2597aa5740 1 Processor -- start 2024-09-19T09:02:29.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.888+0000 7f2597aa5740 1 -- start start 2024-09-19T09:02:29.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.888+0000 7f2597aa5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc3321f9d0 0x55bc3321fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:29.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.888+0000 7f2597aa5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc332202f0 0x55bc3322a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:29.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.888+0000 7f2597aa5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc3322abb0 0x55bc3322cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:29.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.888+0000 7f2597aa5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bc33186090 con 0x55bc3322abb0 2024-09-19T09:02:29.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.888+0000 7f2597aa5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bc3312da70 con 0x55bc3321f9d0 2024-09-19T09:02:29.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.888+0000 7f258cff9640 1 --2- 172.21.15.120:0/72711119 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f257c0a3240 0x7f257c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:29.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.888+0000 7f2597aa5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bc33182050 con 0x55bc332202f0 2024-09-19T09:02:29.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.888+0000 7f2587fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc3322abb0 0x55bc3322cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:29.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.888+0000 7f2587fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc3322abb0 0x55bc3322cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49938/0 (socket says 172.21.15.120:49938) 2024-09-19T09:02:29.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.888+0000 7f2587fff640 1 -- 172.21.15.120:0/2495256110 learned_addr learned my addr 172.21.15.120:0/2495256110 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:29.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.888+0000 7f258f7fe640 1 --2- 172.21.15.120:0/72711119 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f257c0a3240 0x7f257c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:29.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.888+0000 7f258cff9640 1 -- 172.21.15.120:0/72711119 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f25800cf630 con 0x7f2590100a80 2024-09-19T09:02:29.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.888+0000 7f258effd640 1 --2- 172.21.15.120:0/2495256110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc332202f0 0x55bc3322a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:29.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.888+0000 7f258f7fe640 1 --2- 172.21.15.120:0/2495256110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc3321f9d0 0x55bc3321fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:29.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.889+0000 7f2587fff640 1 -- 172.21.15.120:0/2495256110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc332202f0 msgr2=0x55bc3322a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:29.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.889+0000 7f2587fff640 1 --2- 172.21.15.120:0/2495256110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc332202f0 0x55bc3322a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.889+0000 7f2587fff640 1 -- 172.21.15.120:0/2495256110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc3321f9d0 msgr2=0x55bc3321fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:29.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.889+0000 7f2587fff640 1 --2- 172.21.15.120:0/2495256110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc3321f9d0 0x55bc3321fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.889+0000 7f2587fff640 1 -- 172.21.15.120:0/2495256110 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc3313a130 con 0x55bc3322abb0 2024-09-19T09:02:29.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.889+0000 7f258f7fe640 1 --2- 172.21.15.120:0/72711119 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f257c0a3240 0x7f257c0a5700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2588013970 tx=0x7f2588009b10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:29.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.889+0000 7f2587fff640 1 --2- 172.21.15.120:0/2495256110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc3322abb0 0x55bc3322cfa0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2578005eb0 tx=0x7f2578032c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769b09291e8262c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:29.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.889+0000 7f25857fa640 1 -- 172.21.15.120:0/2495256110 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2578040070 con 0x55bc3322abb0 2024-09-19T09:02:29.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.889+0000 7f25857fa640 1 -- 172.21.15.120:0/2495256110 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25780317b0 con 0x55bc3322abb0 2024-09-19T09:02:29.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.889+0000 7f25857fa640 1 -- 172.21.15.120:0/2495256110 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2578035040 con 0x55bc3322abb0 2024-09-19T09:02:29.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.890+0000 7f2597aa5740 1 -- 172.21.15.120:0/2495256110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc3322abb0 msgr2=0x55bc3322cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:29.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.890+0000 7f2597aa5740 1 --2- 172.21.15.120:0/2495256110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc3322abb0 0x55bc3322cfa0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2578005eb0 tx=0x7f2578032c30 comp rx=0 tx=0).stop 2024-09-19T09:02:29.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.890+0000 7f2597aa5740 1 -- 172.21.15.120:0/2495256110 shutdown_connections 2024-09-19T09:02:29.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.890+0000 7f2597aa5740 1 --2- 172.21.15.120:0/2495256110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc3322abb0 0x55bc3322cfa0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.890+0000 7f2597aa5740 1 --2- 172.21.15.120:0/2495256110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc332202f0 0x55bc3322a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.890+0000 7f2597aa5740 1 --2- 172.21.15.120:0/2495256110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc3321f9d0 0x55bc3321fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.890+0000 7f2597aa5740 1 -- 172.21.15.120:0/2495256110 >> 172.21.15.120:0/2495256110 conn(0x55bc3321bf60 msgr2=0x55bc3321c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:29.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.890+0000 7f2597aa5740 1 -- 172.21.15.120:0/2495256110 shutdown_connections 2024-09-19T09:02:29.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.890+0000 7f2597aa5740 1 -- 172.21.15.120:0/2495256110 wait complete. 2024-09-19T09:02:29.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.890+0000 7f2597aa5740 1 -- start start 2024-09-19T09:02:29.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.890+0000 7f2597aa5740 1 Processor -- start 2024-09-19T09:02:29.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.890+0000 7f2597aa5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc3321f9d0 0x55bc3322a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:29.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.890+0000 7f2597aa5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc332202f0 0x55bc33227700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:29.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.890+0000 7f2597aa5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc3322abb0 0x55bc33227c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:29.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.890+0000 7f2597aa5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bc3322d780 con 0x55bc3322abb0 2024-09-19T09:02:29.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.890+0000 7f2597aa5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bc33140f20 con 0x55bc332202f0 2024-09-19T09:02:29.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.890+0000 7f2597aa5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bc3322d900 con 0x55bc3321f9d0 2024-09-19T09:02:29.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.891+0000 7f2597aa5740 10 client.0 _ll_register_callbacks cb 0x55bc3310a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:02:29.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.891+0000 7f258f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc3322abb0 0x55bc33227c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:29.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.891+0000 7f258f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc3322abb0 0x55bc33227c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49940/0 (socket says 172.21.15.120:49940) 2024-09-19T09:02:29.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.891+0000 7f258f7fe640 1 -- 172.21.15.120:0/1431164752 learned_addr learned my addr 172.21.15.120:0/1431164752 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:29.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.891+0000 7f2587fff640 1 --2- 172.21.15.120:0/1431164752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc332202f0 0x55bc33227700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:29.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.891+0000 7f258effd640 1 --2- 172.21.15.120:0/1431164752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc3321f9d0 0x55bc3322a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:29.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.891+0000 7f258f7fe640 1 -- 172.21.15.120:0/1431164752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc3321f9d0 msgr2=0x55bc3322a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:29.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.891+0000 7f258f7fe640 1 --2- 172.21.15.120:0/1431164752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc3321f9d0 0x55bc3322a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.891+0000 7f258f7fe640 1 -- 172.21.15.120:0/1431164752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc332202f0 msgr2=0x55bc33227700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:29.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.891+0000 7f258f7fe640 1 --2- 172.21.15.120:0/1431164752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc332202f0 0x55bc33227700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.891+0000 7f258f7fe640 1 -- 172.21.15.120:0/1431164752 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc32ef72b0 con 0x55bc3322abb0 2024-09-19T09:02:29.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.891+0000 7f2587fff640 1 --2- 172.21.15.120:0/1431164752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc332202f0 0x55bc33227700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:02:29.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.891+0000 7f258f7fe640 1 --2- 172.21.15.120:0/1431164752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc3322abb0 0x55bc33227c40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f25880099c0 tx=0x7f2588005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30c7393c0c13bb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:29.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.891+0000 7f25767fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:02:29.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.891+0000 7f25767fc640 1 -- 172.21.15.120:0/1431164752 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2588024070 con 0x55bc3322abb0 2024-09-19T09:02:29.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.891+0000 7f2597aa5740 1 -- 172.21.15.120:0/1431164752 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25780317b0 con 0x55bc3322abb0 2024-09-19T09:02:29.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.891+0000 7f25767fc640 1 -- 172.21.15.120:0/1431164752 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f2588003970 con 0x55bc3322abb0 2024-09-19T09:02:29.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.891+0000 7f25767fc640 1 -- 172.21.15.120:0/1431164752 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f258802f050 con 0x55bc3322abb0 2024-09-19T09:02:29.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.891+0000 7f2597aa5740 10 client.15117 fetch_fsmap learned FSMap version 27 2024-09-19T09:02:29.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.891+0000 7f2597aa5740 1 -- 172.21.15.120:0/1431164752 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bc332390f0 con 0x55bc3322abb0 2024-09-19T09:02:29.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.892+0000 7f25767fc640 1 -- 172.21.15.120:0/1431164752 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25780317b0 con 0x55bc3322abb0 2024-09-19T09:02:29.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.892+0000 7f25767fc640 1 -- 172.21.15.120:0/1431164752 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f258801d070 con 0x55bc3322abb0 2024-09-19T09:02:29.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.892+0000 7f2597aa5740 10 client.15117 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:02:29.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.892+0000 7f2597aa5740 10 client.15117 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T09:02:29.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.892+0000 7f2597aa5740 20 client.15117 populate_metadata read hostname 'smithi120' 2024-09-19T09:02:29.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.892+0000 7f2597aa5740 1 --2- 172.21.15.120:0/1431164752 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55bc3323b790 0x55bc3323cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:29.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.892+0000 7f2597aa5740 4 client.15117 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:02:29.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.892+0000 7f2597aa5740 1 -- 172.21.15.120:0/1431164752 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc33193600 con 0x55bc3323b790 2024-09-19T09:02:29.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.892+0000 7f258effd640 1 --2- 172.21.15.120:0/1431164752 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55bc3323b790 0x55bc3323cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:29.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.893+0000 7f25767fc640 10 client.15117 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:02:29.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.893+0000 7f258effd640 1 --2- 172.21.15.120:0/1431164752 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55bc3323b790 0x55bc3323cc00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b94678321de36156 server_cookie=69e6a2c521f790d1 in_seq=0 out_seq=0 2024-09-19T09:02:29.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.893+0000 7f25767fc640 1 -- 172.21.15.120:0/1431164752 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc33193600 con 0x55bc3323b790 2024-09-19T09:02:29.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.893+0000 7f25767fc640 10 client.15117 handle_command_reply: tid=0 2024-09-19T09:02:29.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.893+0000 7f2597aa5740 1 client.15117 shutdown 2024-09-19T09:02:29.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.893+0000 7f2597aa5740 1 -- 172.21.15.120:0/1431164752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc3322abb0 msgr2=0x55bc33227c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:29.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.893+0000 7f2597aa5740 1 --2- 172.21.15.120:0/1431164752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc3322abb0 0x55bc33227c40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f25880099c0 tx=0x7f2588005e00 comp rx=0 tx=0).stop 2024-09-19T09:02:29.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.894+0000 7f2597aa5740 1 -- 172.21.15.120:0/1431164752 shutdown_connections 2024-09-19T09:02:29.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.894+0000 7f2597aa5740 1 --2- 172.21.15.120:0/1431164752 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55bc3323b790 0x55bc3323cc00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.894+0000 7f2597aa5740 1 --2- 172.21.15.120:0/1431164752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc3322abb0 0x55bc33227c40 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.894+0000 7f2597aa5740 1 --2- 172.21.15.120:0/1431164752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc332202f0 0x55bc33227700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.894+0000 7f25767fc640 0 client.15117 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:02:29.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.894+0000 7f2597aa5740 1 --2- 172.21.15.120:0/1431164752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc3321f9d0 0x55bc3322a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.894+0000 7f2597aa5740 1 -- 172.21.15.120:0/1431164752 >> 172.21.15.120:0/1431164752 conn(0x55bc3321bf60 msgr2=0x55bc33221c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:29.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.894+0000 7f2597aa5740 1 -- 172.21.15.120:0/1431164752 shutdown_connections 2024-09-19T09:02:29.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.894+0000 7f2597aa5740 1 -- 172.21.15.120:0/1431164752 wait complete. 2024-09-19T09:02:29.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.894+0000 7f2597aa5740 20 client.15117 trim_cache size 0 max 16384 2024-09-19T09:02:29.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.911+0000 7f2597aa5740 1 Processor -- start 2024-09-19T09:02:29.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.911+0000 7f2597aa5740 1 -- start start 2024-09-19T09:02:29.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.911+0000 7f2597aa5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc3321f9d0 0x55bc3323b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:29.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.911+0000 7f2597aa5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc332202f0 0x55bc3323be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:29.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.911+0000 7f2597aa5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc3322abb0 0x55bc33226fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:29.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.911+0000 7f2597aa5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bc3323af40 con 0x55bc332202f0 2024-09-19T09:02:29.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.911+0000 7f2597aa5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bc3323b610 con 0x55bc3321f9d0 2024-09-19T09:02:29.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.911+0000 7f2597aa5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bc3323bce0 con 0x55bc3322abb0 2024-09-19T09:02:29.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.911+0000 7f2587fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc332202f0 0x55bc3323be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:29.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.911+0000 7f2587fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc332202f0 0x55bc3323be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49946/0 (socket says 172.21.15.120:49946) 2024-09-19T09:02:29.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.911+0000 7f2587fff640 1 -- 172.21.15.120:0/2132626516 learned_addr learned my addr 172.21.15.120:0/2132626516 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:29.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.911+0000 7f2587fff640 1 -- 172.21.15.120:0/2132626516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc3322abb0 msgr2=0x55bc33226fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:29.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.911+0000 7f2587fff640 1 --2- 172.21.15.120:0/2132626516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc3322abb0 0x55bc33226fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.911+0000 7f2587fff640 1 -- 172.21.15.120:0/2132626516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc3321f9d0 msgr2=0x55bc3323b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:29.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.911+0000 7f2587fff640 1 --2- 172.21.15.120:0/2132626516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc3321f9d0 0x55bc3323b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.911+0000 7f2587fff640 1 -- 172.21.15.120:0/2132626516 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc3322a070 con 0x55bc332202f0 2024-09-19T09:02:29.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.912+0000 7f2587fff640 1 --2- 172.21.15.120:0/2132626516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc332202f0 0x55bc3323be60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2578033000 tx=0x7f2578032f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29385733beeb089 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:29.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.912+0000 7f2584ff9640 1 -- 172.21.15.120:0/2132626516 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2578040040 con 0x55bc332202f0 2024-09-19T09:02:29.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.912+0000 7f2584ff9640 1 -- 172.21.15.120:0/2132626516 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2578009520 con 0x55bc332202f0 2024-09-19T09:02:29.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.912+0000 7f2584ff9640 1 -- 172.21.15.120:0/2132626516 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2578035040 con 0x55bc332202f0 2024-09-19T09:02:29.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.912+0000 7f2597aa5740 1 -- 172.21.15.120:0/2132626516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc332202f0 msgr2=0x55bc3323be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:29.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.912+0000 7f2597aa5740 1 --2- 172.21.15.120:0/2132626516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc332202f0 0x55bc3323be60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2578033000 tx=0x7f2578032f10 comp rx=0 tx=0).stop 2024-09-19T09:02:29.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.912+0000 7f2597aa5740 1 -- 172.21.15.120:0/2132626516 shutdown_connections 2024-09-19T09:02:29.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.912+0000 7f2597aa5740 1 --2- 172.21.15.120:0/2132626516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc3322abb0 0x55bc33226fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.913+0000 7f2597aa5740 1 --2- 172.21.15.120:0/2132626516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc332202f0 0x55bc3323be60 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.913+0000 7f2597aa5740 1 --2- 172.21.15.120:0/2132626516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc3321f9d0 0x55bc3323b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.913+0000 7f2597aa5740 1 -- 172.21.15.120:0/2132626516 >> 172.21.15.120:0/2132626516 conn(0x55bc3321bf60 msgr2=0x55bc3321c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:29.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.913+0000 7f2597aa5740 1 -- 172.21.15.120:0/2132626516 shutdown_connections 2024-09-19T09:02:29.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.913+0000 7f2597aa5740 1 -- 172.21.15.120:0/2132626516 wait complete. 2024-09-19T09:02:29.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.913+0000 7f2597aa5740 1 -- start start 2024-09-19T09:02:29.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.913+0000 7f2597aa5740 1 Processor -- start 2024-09-19T09:02:29.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.913+0000 7f2597aa5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc3321f9d0 0x55bc3323de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:29.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.913+0000 7f2597aa5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc332202f0 0x55bc3323e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:29.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.913+0000 7f2597aa5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc3322abb0 0x55bc3323e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:29.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.913+0000 7f2597aa5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bc3323ca80 con 0x55bc3321f9d0 2024-09-19T09:02:29.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.913+0000 7f2597aa5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bc3323c3b0 con 0x55bc332202f0 2024-09-19T09:02:29.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.913+0000 7f2597aa5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bc3323d150 con 0x55bc3322abb0 2024-09-19T09:02:29.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.913+0000 7f2597aa5740 10 client.0 _ll_register_callbacks cb 0x7f2588013f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:02:29.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.913+0000 7f258effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc3321f9d0 0x55bc3323de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:29.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.913+0000 7f258effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc3321f9d0 0x55bc3323de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49962/0 (socket says 172.21.15.120:49962) 2024-09-19T09:02:29.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.913+0000 7f258effd640 1 -- 172.21.15.120:0/879977284 learned_addr learned my addr 172.21.15.120:0/879977284 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:29.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.913+0000 7f2587fff640 1 --2- 172.21.15.120:0/879977284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc332202f0 0x55bc3323e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:29.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.913+0000 7f258effd640 1 -- 172.21.15.120:0/879977284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc3322abb0 msgr2=0x55bc3323e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:29.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.913+0000 7f258effd640 1 --2- 172.21.15.120:0/879977284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc3322abb0 0x55bc3323e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.913+0000 7f258effd640 1 -- 172.21.15.120:0/879977284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc332202f0 msgr2=0x55bc3323e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:29.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.913+0000 7f258effd640 1 --2- 172.21.15.120:0/879977284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc332202f0 0x55bc3323e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.913+0000 7f258effd640 1 -- 172.21.15.120:0/879977284 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc332a3f60 con 0x55bc3321f9d0 2024-09-19T09:02:29.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.913+0000 7f2587fff640 1 --2- 172.21.15.120:0/879977284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc332202f0 0x55bc3323e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:02:29.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.914+0000 7f258effd640 1 --2- 172.21.15.120:0/879977284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc3321f9d0 0x55bc3323de00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2580005840 tx=0x7f258009b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7144a9d7e0c45cde server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:29.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.914+0000 7f25767fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:02:29.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.914+0000 7f25767fc640 1 -- 172.21.15.120:0/879977284 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258009bc10 con 0x55bc3321f9d0 2024-09-19T09:02:29.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.914+0000 7f25767fc640 1 -- 172.21.15.120:0/879977284 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f2580098280 con 0x55bc3321f9d0 2024-09-19T09:02:29.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.914+0000 7f2597aa5740 1 -- 172.21.15.120:0/879977284 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2578009520 con 0x55bc3321f9d0 2024-09-19T09:02:29.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.914+0000 7f25767fc640 1 -- 172.21.15.120:0/879977284 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f258009ffc0 con 0x55bc3321f9d0 2024-09-19T09:02:29.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.914+0000 7f2597aa5740 10 client.15123 fetch_fsmap learned FSMap version 27 2024-09-19T09:02:29.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.914+0000 7f2597aa5740 1 -- 172.21.15.120:0/879977284 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bc332a52c0 con 0x55bc3321f9d0 2024-09-19T09:02:29.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.914+0000 7f25767fc640 1 -- 172.21.15.120:0/879977284 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2578009520 con 0x55bc3321f9d0 2024-09-19T09:02:29.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.914+0000 7f25767fc640 1 -- 172.21.15.120:0/879977284 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f25800ba490 con 0x55bc3321f9d0 2024-09-19T09:02:29.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.914+0000 7f2597aa5740 10 client.15123 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:02:29.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.914+0000 7f2597aa5740 10 client.15123 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T09:02:29.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.914+0000 7f2597aa5740 20 client.15123 populate_metadata read hostname 'smithi120' 2024-09-19T09:02:29.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.914+0000 7f2597aa5740 1 --2- 172.21.15.120:0/879977284 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55bc332a9ea0 0x55bc332a8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:29.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.914+0000 7f2597aa5740 4 client.15123 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:02:29.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.914+0000 7f2597aa5740 1 -- 172.21.15.120:0/879977284 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "status"}) -- 0x55bc3318db10 con 0x55bc332a9ea0 2024-09-19T09:02:29.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.915+0000 7f2587fff640 1 --2- 172.21.15.120:0/879977284 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55bc332a9ea0 0x55bc332a8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:29.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.915+0000 7f25767fc640 10 client.15123 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:02:29.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.915+0000 7f2587fff640 1 --2- 172.21.15.120:0/879977284 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55bc332a9ea0 0x55bc332a8bb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=84388f5d29b85b0d server_cookie=367f75ee7a31dda4 in_seq=0 out_seq=0 2024-09-19T09:02:29.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.915+0000 7f25767fc640 1 -- 172.21.15.120:0/879977284 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55bc3318db10 con 0x55bc332a9ea0 2024-09-19T09:02:29.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.915+0000 7f25767fc640 10 client.15123 handle_command_reply: tid=0 2024-09-19T09:02:29.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.915+0000 7f2597aa5740 1 client.15123 shutdown 2024-09-19T09:02:29.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.915+0000 7f2597aa5740 1 -- 172.21.15.120:0/879977284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc3321f9d0 msgr2=0x55bc3323de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:29.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.915+0000 7f2597aa5740 1 --2- 172.21.15.120:0/879977284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc3321f9d0 0x55bc3323de00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2580005840 tx=0x7f258009b9e0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.915+0000 7f2597aa5740 1 -- 172.21.15.120:0/879977284 shutdown_connections 2024-09-19T09:02:29.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.915+0000 7f2597aa5740 1 --2- 172.21.15.120:0/879977284 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55bc332a9ea0 0x55bc332a8bb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.915+0000 7f2597aa5740 1 --2- 172.21.15.120:0/879977284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc3322abb0 0x55bc3323e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.915+0000 7f25767fc640 0 client.15123 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:02:29.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.915+0000 7f2597aa5740 1 --2- 172.21.15.120:0/879977284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc332202f0 0x55bc3323e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.915+0000 7f2597aa5740 1 --2- 172.21.15.120:0/879977284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc3321f9d0 0x55bc3323de00 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.915+0000 7f2597aa5740 1 -- 172.21.15.120:0/879977284 >> 172.21.15.120:0/879977284 conn(0x55bc3321bf60 msgr2=0x55bc332276c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:29.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.915+0000 7f2597aa5740 1 -- 172.21.15.120:0/879977284 shutdown_connections 2024-09-19T09:02:29.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.915+0000 7f2597aa5740 1 -- 172.21.15.120:0/879977284 wait complete. 2024-09-19T09:02:29.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.915+0000 7f2597aa5740 20 client.15123 trim_cache size 0 max 16384 2024-09-19T09:02:29.937 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:02:29.937 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_fsid": "507393d2-7664-11ef-bceb-c7b262605968", 2024-09-19T09:02:29.937 INFO:teuthology.orchestra.run.smithi120.stdout: "whoami": 4, 2024-09-19T09:02:29.937 INFO:teuthology.orchestra.run.smithi120.stdout: "id": 24421, 2024-09-19T09:02:29.937 INFO:teuthology.orchestra.run.smithi120.stdout: "want_state": "up:active", 2024-09-19T09:02:29.937 INFO:teuthology.orchestra.run.smithi120.stdout: "state": "up:active", 2024-09-19T09:02:29.937 INFO:teuthology.orchestra.run.smithi120.stdout: "fs_name": "cephfs", 2024-09-19T09:02:29.938 INFO:teuthology.orchestra.run.smithi120.stdout: "rank_uptime": 91.784941821000004, 2024-09-19T09:02:29.938 INFO:teuthology.orchestra.run.smithi120.stdout: "mdsmap_epoch": 27, 2024-09-19T09:02:29.938 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch": 81, 2024-09-19T09:02:29.938 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch_barrier": 79, 2024-09-19T09:02:29.938 INFO:teuthology.orchestra.run.smithi120.stdout: "uptime": 129.21148124300001 2024-09-19T09:02:29.938 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:02:29.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.916+0000 7f25956cc640 1 -- 172.21.15.120:0/72711119 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f257c0a3240 msgr2=0x7f257c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:29.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.916+0000 7f25956cc640 1 --2- 172.21.15.120:0/72711119 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f257c0a3240 0x7f257c0a5700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2588013970 tx=0x7f2588009b10 comp rx=0 tx=0).stop 2024-09-19T09:02:29.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.916+0000 7f25956cc640 1 -- 172.21.15.120:0/72711119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2590100a80 msgr2=0x7f2590198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:29.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.916+0000 7f25956cc640 1 --2- 172.21.15.120:0/72711119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2590100a80 0x7f2590198460 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f258000c890 tx=0x7f258000cd50 comp rx=0 tx=0).stop 2024-09-19T09:02:29.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.916+0000 7f25956cc640 1 -- 172.21.15.120:0/72711119 shutdown_connections 2024-09-19T09:02:29.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.916+0000 7f25956cc640 1 --2- 172.21.15.120:0/72711119 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f257c0a3240 0x7f257c0a5700 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.916+0000 7f25956cc640 1 --2- 172.21.15.120:0/72711119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2590106a80 0x7f259019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.916+0000 7f25956cc640 1 --2- 172.21.15.120:0/72711119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2590101420 0x7f25901989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.916+0000 7f25956cc640 1 --2- 172.21.15.120:0/72711119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2590100a80 0x7f2590198460 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:29.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.916+0000 7f25956cc640 1 -- 172.21.15.120:0/72711119 >> 172.21.15.120:0/72711119 conn(0x7f25900fc7a0 msgr2=0x7f259006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:29.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.916+0000 7f25956cc640 1 -- 172.21.15.120:0/72711119 shutdown_connections 2024-09-19T09:02:29.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:29.916+0000 7f25956cc640 1 -- 172.21.15.120:0/72711119 wait complete. 2024-09-19T09:02:30.552 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:30 smithi120 ceph-mon[31150]: pgmap v330: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-19T09:02:30.587 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:30 smithi196 ceph-mon[35150]: pgmap v330: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-19T09:02:30.588 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:30 smithi153 ceph-mon[35071]: pgmap v330: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-19T09:02:30.705 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T09:02:30.950 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:31.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.435+0000 7f177358f640 1 -- 172.21.15.120:0/1499021626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f176c10c170 msgr2=0x7f176c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:31.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.435+0000 7f177358f640 1 --2- 172.21.15.120:0/1499021626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f176c10c170 0x7f176c10e610 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1764009ca0 tx=0x7f176402a950 comp rx=0 tx=0).stop 2024-09-19T09:02:31.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.435+0000 7f177358f640 1 -- 172.21.15.120:0/1499021626 shutdown_connections 2024-09-19T09:02:31.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.435+0000 7f177358f640 1 --2- 172.21.15.120:0/1499021626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f176c10c170 0x7f176c10e610 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.435+0000 7f177358f640 1 --2- 172.21.15.120:0/1499021626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f176c103660 0x7f176c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.435+0000 7f177358f640 1 --2- 172.21.15.120:0/1499021626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f176c102d40 0x7f176c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.435+0000 7f177358f640 1 -- 172.21.15.120:0/1499021626 >> 172.21.15.120:0/1499021626 conn(0x7f176c0fc7a0 msgr2=0x7f176c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:31.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.436+0000 7f177358f640 1 -- 172.21.15.120:0/1499021626 shutdown_connections 2024-09-19T09:02:31.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.436+0000 7f177358f640 1 -- 172.21.15.120:0/1499021626 wait complete. 2024-09-19T09:02:31.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.436+0000 7f177358f640 1 Processor -- start 2024-09-19T09:02:31.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.436+0000 7f177358f640 1 -- start start 2024-09-19T09:02:31.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.436+0000 7f177358f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f176c102d40 0x7f176c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:31.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.436+0000 7f177358f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f176c103660 0x7f176c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:31.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.436+0000 7f177358f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f176c10c170 0x7f176c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:31.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.436+0000 7f177358f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f176c111f70 con 0x7f176c102d40 2024-09-19T09:02:31.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.436+0000 7f177358f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f176c111df0 con 0x7f176c10c170 2024-09-19T09:02:31.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.436+0000 7f177358f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f176c1120f0 con 0x7f176c103660 2024-09-19T09:02:31.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.436+0000 7f177132b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f176c102d40 0x7f176c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:31.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.436+0000 7f177132b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f176c102d40 0x7f176c1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35630/0 (socket says 172.21.15.120:35630) 2024-09-19T09:02:31.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.436+0000 7f177132b640 1 -- 172.21.15.120:0/338058308 learned_addr learned my addr 172.21.15.120:0/338058308 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:31.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.437+0000 7f1770b2a640 1 --2- 172.21.15.120:0/338058308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f176c103660 0x7f176c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:31.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.437+0000 7f177132b640 1 -- 172.21.15.120:0/338058308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f176c103660 msgr2=0x7f176c198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:31.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.437+0000 7f177132b640 1 --2- 172.21.15.120:0/338058308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f176c103660 0x7f176c198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.437+0000 7f177132b640 1 -- 172.21.15.120:0/338058308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f176c10c170 msgr2=0x7f176c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:31.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.437+0000 7f177132b640 1 --2- 172.21.15.120:0/338058308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f176c10c170 0x7f176c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.437+0000 7f177132b640 1 -- 172.21.15.120:0/338058308 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f176c19d380 con 0x7f176c102d40 2024-09-19T09:02:31.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.437+0000 7f1770b2a640 1 --2- 172.21.15.120:0/338058308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f176c103660 0x7f176c198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:02:31.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.437+0000 7f177132b640 1 --2- 172.21.15.120:0/338058308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f176c102d40 0x7f176c1983d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f176800d770 tx=0x7f176800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:31.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.437+0000 7f17627fc640 1 -- 172.21.15.120:0/338058308 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1768017070 con 0x7f176c102d40 2024-09-19T09:02:31.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.437+0000 7f17627fc640 1 -- 172.21.15.120:0/338058308 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1768005500 con 0x7f176c102d40 2024-09-19T09:02:31.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.437+0000 7f177358f640 1 -- 172.21.15.120:0/338058308 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f176c19d670 con 0x7f176c102d40 2024-09-19T09:02:31.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.437+0000 7f17627fc640 1 -- 172.21.15.120:0/338058308 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1768004e00 con 0x7f176c102d40 2024-09-19T09:02:31.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.437+0000 7f177358f640 1 -- 172.21.15.120:0/338058308 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f176c1a4f50 con 0x7f176c102d40 2024-09-19T09:02:31.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.439+0000 7f17627fc640 1 -- 172.21.15.120:0/338058308 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f1768004fa0 con 0x7f176c102d40 2024-09-19T09:02:31.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.440+0000 7f1775968740 1 Processor -- start 2024-09-19T09:02:31.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.440+0000 7f1775968740 1 -- start start 2024-09-19T09:02:31.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.440+0000 7f1775968740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558481e7e9d0 0x558481e7edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:31.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.440+0000 7f1775968740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558481e7f2f0 0x558481e89670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:31.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.440+0000 7f1775968740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558481e89bb0 0x558481e8bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:31.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.440+0000 7f1775968740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558481de5090 con 0x558481e7e9d0 2024-09-19T09:02:31.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.440+0000 7f1775968740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558481d8ca70 con 0x558481e89bb0 2024-09-19T09:02:31.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.440+0000 7f1775968740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558481de1050 con 0x558481e7f2f0 2024-09-19T09:02:31.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.440+0000 7f17627fc640 1 --2- 172.21.15.120:0/338058308 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f17580a3240 0x7f17580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:31.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.440+0000 7f1771b2c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558481e7e9d0 0x558481e7edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:31.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.440+0000 7f1771b2c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558481e7e9d0 0x558481e7edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35636/0 (socket says 172.21.15.120:35636) 2024-09-19T09:02:31.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.440+0000 7f1771b2c640 1 -- 172.21.15.120:0/3051356060 learned_addr learned my addr 172.21.15.120:0/3051356060 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:31.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.440+0000 7f177132b640 1 --2- 172.21.15.120:0/3051356060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558481e7f2f0 0x558481e89670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:31.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.440+0000 7f17627fc640 1 -- 172.21.15.120:0/338058308 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f17680cf480 con 0x7f176c102d40 2024-09-19T09:02:31.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.440+0000 7f1771b2c640 1 --2- 172.21.15.120:0/338058308 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f17580a3240 0x7f17580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:31.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.440+0000 7f1770b2a640 1 --2- 172.21.15.120:0/3051356060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558481e89bb0 0x558481e8bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:31.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.441+0000 7f1771b2c640 1 -- 172.21.15.120:0/3051356060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558481e7f2f0 msgr2=0x558481e89670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:31.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.441+0000 7f1771b2c640 1 --2- 172.21.15.120:0/3051356060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558481e7f2f0 0x558481e89670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.441+0000 7f1771b2c640 1 -- 172.21.15.120:0/3051356060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558481e89bb0 msgr2=0x558481e8bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:31.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.441+0000 7f1771b2c640 1 --2- 172.21.15.120:0/3051356060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558481e89bb0 0x558481e8bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.441+0000 7f1771b2c640 1 -- 172.21.15.120:0/3051356060 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558481d99130 con 0x558481e7e9d0 2024-09-19T09:02:31.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.441+0000 7f1771b2c640 1 --2- 172.21.15.120:0/3051356060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558481e7e9d0 0x558481e7edb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1764009430 tx=0x7f176402fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a013e478777682 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:31.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.441+0000 7f17577fe640 1 -- 172.21.15.120:0/3051356060 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1764034040 con 0x558481e7e9d0 2024-09-19T09:02:31.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.441+0000 7f17577fe640 1 -- 172.21.15.120:0/3051356060 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f176404c070 con 0x558481e7e9d0 2024-09-19T09:02:31.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.441+0000 7f17577fe640 1 -- 172.21.15.120:0/3051356060 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176403b470 con 0x558481e7e9d0 2024-09-19T09:02:31.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.441+0000 7f1771b2c640 1 --2- 172.21.15.120:0/338058308 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f17580a3240 0x7f17580a5700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f176404b000 tx=0x7f176404c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:31.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.442+0000 7f1775968740 1 -- 172.21.15.120:0/3051356060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558481e7e9d0 msgr2=0x558481e7edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:31.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.442+0000 7f1775968740 1 --2- 172.21.15.120:0/3051356060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558481e7e9d0 0x558481e7edb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1764009430 tx=0x7f176402fc80 comp rx=0 tx=0).stop 2024-09-19T09:02:31.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.442+0000 7f1775968740 1 -- 172.21.15.120:0/3051356060 shutdown_connections 2024-09-19T09:02:31.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.442+0000 7f1775968740 1 --2- 172.21.15.120:0/3051356060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558481e89bb0 0x558481e8bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.442+0000 7f1775968740 1 --2- 172.21.15.120:0/3051356060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558481e7f2f0 0x558481e89670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.442+0000 7f1775968740 1 --2- 172.21.15.120:0/3051356060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558481e7e9d0 0x558481e7edb0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.442+0000 7f1775968740 1 -- 172.21.15.120:0/3051356060 >> 172.21.15.120:0/3051356060 conn(0x558481e7af60 msgr2=0x558481e7b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:31.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.442+0000 7f1775968740 1 -- 172.21.15.120:0/3051356060 shutdown_connections 2024-09-19T09:02:31.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.442+0000 7f1775968740 1 -- 172.21.15.120:0/3051356060 wait complete. 2024-09-19T09:02:31.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.442+0000 7f1775968740 1 -- start start 2024-09-19T09:02:31.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.442+0000 7f1775968740 1 Processor -- start 2024-09-19T09:02:31.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.443+0000 7f1775968740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558481e7e9d0 0x558481e89450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:31.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.443+0000 7f1775968740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558481e7f2f0 0x558481e867a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:31.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.443+0000 7f1775968740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558481e89bb0 0x558481e86ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:31.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.443+0000 7f1775968740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558481e8c780 con 0x558481e7f2f0 2024-09-19T09:02:31.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.443+0000 7f1775968740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558481d9ff20 con 0x558481e89bb0 2024-09-19T09:02:31.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.443+0000 7f1775968740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558481e8c900 con 0x558481e7e9d0 2024-09-19T09:02:31.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.443+0000 7f1775968740 10 client.0 _ll_register_callbacks cb 0x558481d69030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:02:31.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.443+0000 7f1770b2a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558481e7f2f0 0x558481e867a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:31.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.443+0000 7f1770b2a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558481e7f2f0 0x558481e867a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35644/0 (socket says 172.21.15.120:35644) 2024-09-19T09:02:31.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.443+0000 7f1770b2a640 1 -- 172.21.15.120:0/3970223880 learned_addr learned my addr 172.21.15.120:0/3970223880 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:31.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.443+0000 7f177132b640 1 --2- 172.21.15.120:0/3970223880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558481e7e9d0 0x558481e89450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:31.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.443+0000 7f1771b2c640 1 --2- 172.21.15.120:0/3970223880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558481e89bb0 0x558481e86ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:31.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.443+0000 7f1770b2a640 1 -- 172.21.15.120:0/3970223880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558481e7e9d0 msgr2=0x558481e89450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:31.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.443+0000 7f1770b2a640 1 --2- 172.21.15.120:0/3970223880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558481e7e9d0 0x558481e89450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.443+0000 7f1770b2a640 1 -- 172.21.15.120:0/3970223880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558481e89bb0 msgr2=0x558481e86ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:31.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.443+0000 7f1770b2a640 1 --2- 172.21.15.120:0/3970223880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558481e89bb0 0x558481e86ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.443+0000 7f1770b2a640 1 -- 172.21.15.120:0/3970223880 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558481b562b0 con 0x558481e7f2f0 2024-09-19T09:02:31.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.443+0000 7f1770b2a640 1 --2- 172.21.15.120:0/3970223880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558481e7f2f0 0x558481e867a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f175c010490 tx=0x7f175c010950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc986067bd7daa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:31.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.443+0000 7f1754ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:02:31.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.443+0000 7f1754ff9640 1 -- 172.21.15.120:0/3970223880 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c005220 con 0x558481e7f2f0 2024-09-19T09:02:31.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.443+0000 7f1754ff9640 1 -- 172.21.15.120:0/3970223880 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f175c0053c0 con 0x558481e7f2f0 2024-09-19T09:02:31.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.443+0000 7f1775968740 1 -- 172.21.15.120:0/3970223880 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f176404c070 con 0x558481e7f2f0 2024-09-19T09:02:31.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.444+0000 7f1754ff9640 1 -- 172.21.15.120:0/3970223880 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f175c0055c0 con 0x558481e7f2f0 2024-09-19T09:02:31.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.444+0000 7f1775968740 10 client.15135 fetch_fsmap learned FSMap version 27 2024-09-19T09:02:31.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.444+0000 7f1775968740 1 -- 172.21.15.120:0/3970223880 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558481e98150 con 0x558481e7f2f0 2024-09-19T09:02:31.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.444+0000 7f1754ff9640 1 -- 172.21.15.120:0/3970223880 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f176404c070 con 0x558481e7f2f0 2024-09-19T09:02:31.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.444+0000 7f1754ff9640 1 -- 172.21.15.120:0/3970223880 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f175c00f040 con 0x558481e7f2f0 2024-09-19T09:02:31.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.444+0000 7f1775968740 10 client.15135 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:02:31.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.444+0000 7f1775968740 10 client.15135 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T09:02:31.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.444+0000 7f1775968740 20 client.15135 populate_metadata read hostname 'smithi120' 2024-09-19T09:02:31.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.444+0000 7f1775968740 1 --2- 172.21.15.120:0/3970223880 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558481e9a830 0x558481e9bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:31.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.444+0000 7f1775968740 4 client.15135 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:02:31.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.444+0000 7f1775968740 1 -- 172.21.15.120:0/3970223880 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558481df2600 con 0x558481e9a830 2024-09-19T09:02:31.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.444+0000 7f177132b640 1 --2- 172.21.15.120:0/3970223880 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558481e9a830 0x558481e9bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:31.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.445+0000 7f1754ff9640 10 client.15135 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:02:31.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.445+0000 7f177132b640 1 --2- 172.21.15.120:0/3970223880 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558481e9a830 0x558481e9bca0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=989a2d8a81e1719c server_cookie=9275d0cf9feaec41 in_seq=0 out_seq=0 2024-09-19T09:02:31.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.445+0000 7f1754ff9640 1 -- 172.21.15.120:0/3970223880 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558481df2600 con 0x558481e9a830 2024-09-19T09:02:31.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.445+0000 7f1754ff9640 10 client.15135 handle_command_reply: tid=0 2024-09-19T09:02:31.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.445+0000 7f1775968740 1 client.15135 shutdown 2024-09-19T09:02:31.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.445+0000 7f1775968740 1 -- 172.21.15.120:0/3970223880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558481e7f2f0 msgr2=0x558481e867a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:31.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.445+0000 7f1775968740 1 --2- 172.21.15.120:0/3970223880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558481e7f2f0 0x558481e867a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f175c010490 tx=0x7f175c010950 comp rx=0 tx=0).stop 2024-09-19T09:02:31.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.446+0000 7f1775968740 1 -- 172.21.15.120:0/3970223880 shutdown_connections 2024-09-19T09:02:31.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.446+0000 7f1775968740 1 --2- 172.21.15.120:0/3970223880 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558481e9a830 0x558481e9bca0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.446+0000 7f1775968740 1 --2- 172.21.15.120:0/3970223880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558481e89bb0 0x558481e86ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.446+0000 7f1775968740 1 --2- 172.21.15.120:0/3970223880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558481e7f2f0 0x558481e867a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.446+0000 7f1775968740 1 --2- 172.21.15.120:0/3970223880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558481e7e9d0 0x558481e89450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.446+0000 7f1754ff9640 0 client.15135 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:02:31.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.446+0000 7f1775968740 1 -- 172.21.15.120:0/3970223880 >> 172.21.15.120:0/3970223880 conn(0x558481e7af60 msgr2=0x558481e80c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:31.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.446+0000 7f1775968740 1 -- 172.21.15.120:0/3970223880 shutdown_connections 2024-09-19T09:02:31.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.446+0000 7f1775968740 1 -- 172.21.15.120:0/3970223880 wait complete. 2024-09-19T09:02:31.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.446+0000 7f1775968740 20 client.15135 trim_cache size 0 max 16384 2024-09-19T09:02:31.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.464+0000 7f1775968740 1 Processor -- start 2024-09-19T09:02:31.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.464+0000 7f1775968740 1 -- start start 2024-09-19T09:02:31.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.464+0000 7f1775968740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558481e7e9d0 0x558481e9bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:31.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.464+0000 7f1775968740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558481e7f2f0 0x558481ea3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:31.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.464+0000 7f1775968740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558481e89bb0 0x558481ea44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:31.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.464+0000 7f1775968740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558481e99fe0 con 0x558481e7e9d0 2024-09-19T09:02:31.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.464+0000 7f1775968740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558481e9a6b0 con 0x558481e7f2f0 2024-09-19T09:02:31.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.464+0000 7f1775968740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558481e9ad80 con 0x558481e89bb0 2024-09-19T09:02:31.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.464+0000 7f177132b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558481e7e9d0 0x558481e9bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:31.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.465+0000 7f177132b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558481e7e9d0 0x558481e9bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35654/0 (socket says 172.21.15.120:35654) 2024-09-19T09:02:31.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.465+0000 7f177132b640 1 -- 172.21.15.120:0/3650153342 learned_addr learned my addr 172.21.15.120:0/3650153342 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:31.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.465+0000 7f1770b2a640 1 --2- 172.21.15.120:0/3650153342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558481e7f2f0 0x558481ea3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:31.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.465+0000 7f1771b2c640 1 --2- 172.21.15.120:0/3650153342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558481e89bb0 0x558481ea44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:31.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.465+0000 7f177132b640 1 -- 172.21.15.120:0/3650153342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558481e89bb0 msgr2=0x558481ea44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:31.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.465+0000 7f177132b640 1 --2- 172.21.15.120:0/3650153342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558481e89bb0 0x558481ea44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.465+0000 7f177132b640 1 -- 172.21.15.120:0/3650153342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558481e7f2f0 msgr2=0x558481ea3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:31.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.465+0000 7f177132b640 1 --2- 172.21.15.120:0/3650153342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558481e7f2f0 0x558481ea3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.465+0000 7f177132b640 1 -- 172.21.15.120:0/3650153342 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558481e89070 con 0x558481e7e9d0 2024-09-19T09:02:31.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.465+0000 7f1770b2a640 1 --2- 172.21.15.120:0/3650153342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558481e7f2f0 0x558481ea3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:02:31.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.465+0000 7f1771b2c640 1 --2- 172.21.15.120:0/3650153342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558481e89bb0 0x558481ea44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:02:31.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.465+0000 7f177132b640 1 --2- 172.21.15.120:0/3650153342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558481e7e9d0 0x558481e9bca0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f176800d740 tx=0x7f1768016500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3031d73def8da6a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:31.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.465+0000 7f1756ffd640 1 -- 172.21.15.120:0/3650153342 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1768020040 con 0x558481e7e9d0 2024-09-19T09:02:31.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.465+0000 7f1756ffd640 1 -- 172.21.15.120:0/3650153342 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17680b4070 con 0x558481e7e9d0 2024-09-19T09:02:31.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.465+0000 7f1756ffd640 1 -- 172.21.15.120:0/3650153342 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1768099df0 con 0x558481e7e9d0 2024-09-19T09:02:31.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.466+0000 7f1775968740 1 -- 172.21.15.120:0/3650153342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558481e7e9d0 msgr2=0x558481e9bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:31.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.466+0000 7f1775968740 1 --2- 172.21.15.120:0/3650153342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558481e7e9d0 0x558481e9bca0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f176800d740 tx=0x7f1768016500 comp rx=0 tx=0).stop 2024-09-19T09:02:31.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.466+0000 7f1775968740 1 -- 172.21.15.120:0/3650153342 shutdown_connections 2024-09-19T09:02:31.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.466+0000 7f1775968740 1 --2- 172.21.15.120:0/3650153342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558481e89bb0 0x558481ea44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.466+0000 7f1775968740 1 --2- 172.21.15.120:0/3650153342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558481e7f2f0 0x558481ea3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.466+0000 7f1775968740 1 --2- 172.21.15.120:0/3650153342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558481e7e9d0 0x558481e9bca0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.466+0000 7f1775968740 1 -- 172.21.15.120:0/3650153342 >> 172.21.15.120:0/3650153342 conn(0x558481e7af60 msgr2=0x558481e85070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:31.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.466+0000 7f1775968740 1 -- 172.21.15.120:0/3650153342 shutdown_connections 2024-09-19T09:02:31.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.466+0000 7f1775968740 1 -- 172.21.15.120:0/3650153342 wait complete. 2024-09-19T09:02:31.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.466+0000 7f1775968740 1 -- start start 2024-09-19T09:02:31.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.466+0000 7f1775968740 1 Processor -- start 2024-09-19T09:02:31.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.467+0000 7f1775968740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558481e7e9d0 0x558481e9f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:31.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.467+0000 7f1775968740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558481e7f2f0 0x558481e9f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:31.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.467+0000 7f1775968740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558481e89bb0 0x558481f02f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:31.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.467+0000 7f1775968740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558481e9bb20 con 0x558481e7f2f0 2024-09-19T09:02:31.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.467+0000 7f1775968740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558481e9b450 con 0x558481e7e9d0 2024-09-19T09:02:31.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.467+0000 7f1775968740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558481e9c1f0 con 0x558481e89bb0 2024-09-19T09:02:31.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.467+0000 7f1775968740 10 client.0 _ll_register_callbacks cb 0x7f176c06b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:02:31.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.467+0000 7f1770b2a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558481e7f2f0 0x558481e9f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:31.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.467+0000 7f1770b2a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558481e7f2f0 0x558481e9f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35658/0 (socket says 172.21.15.120:35658) 2024-09-19T09:02:31.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.467+0000 7f1770b2a640 1 -- 172.21.15.120:0/36653176 learned_addr learned my addr 172.21.15.120:0/36653176 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:31.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.467+0000 7f177132b640 1 --2- 172.21.15.120:0/36653176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558481e7e9d0 0x558481e9f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:31.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.467+0000 7f1771b2c640 1 --2- 172.21.15.120:0/36653176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558481e89bb0 0x558481f02f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:31.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.467+0000 7f1770b2a640 1 -- 172.21.15.120:0/36653176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558481e89bb0 msgr2=0x558481f02f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:31.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.467+0000 7f1770b2a640 1 --2- 172.21.15.120:0/36653176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558481e89bb0 0x558481f02f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.467+0000 7f1770b2a640 1 -- 172.21.15.120:0/36653176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558481e7e9d0 msgr2=0x558481e9f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:31.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.467+0000 7f1770b2a640 1 --2- 172.21.15.120:0/36653176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558481e7e9d0 0x558481e9f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.467+0000 7f1770b2a640 1 -- 172.21.15.120:0/36653176 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558481f034f0 con 0x558481e7f2f0 2024-09-19T09:02:31.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.467+0000 7f1770b2a640 1 --2- 172.21.15.120:0/36653176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558481e7f2f0 0x558481e9f980 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x558481e87de0 tx=0x7f175c011080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251ac856ee431782 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:31.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.467+0000 7f1754ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:02:31.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.467+0000 7f1754ff9640 1 -- 172.21.15.120:0/36653176 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c011bb0 con 0x558481e7f2f0 2024-09-19T09:02:31.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.468+0000 7f1754ff9640 1 -- 172.21.15.120:0/36653176 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f175c011d50 con 0x558481e7f2f0 2024-09-19T09:02:31.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.468+0000 7f1775968740 1 -- 172.21.15.120:0/36653176 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17680b4070 con 0x558481e7f2f0 2024-09-19T09:02:31.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.468+0000 7f1754ff9640 1 -- 172.21.15.120:0/36653176 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f175c012720 con 0x558481e7f2f0 2024-09-19T09:02:31.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.468+0000 7f1775968740 10 client.15141 fetch_fsmap learned FSMap version 27 2024-09-19T09:02:31.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.468+0000 7f1775968740 1 -- 172.21.15.120:0/36653176 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558481f04740 con 0x558481e7f2f0 2024-09-19T09:02:31.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.468+0000 7f1754ff9640 1 -- 172.21.15.120:0/36653176 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17680b4070 con 0x558481e7f2f0 2024-09-19T09:02:31.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.468+0000 7f1754ff9640 1 -- 172.21.15.120:0/36653176 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f175c027b10 con 0x558481e7f2f0 2024-09-19T09:02:31.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.468+0000 7f1775968740 10 client.15141 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:02:31.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.468+0000 7f1775968740 10 client.15141 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T09:02:31.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.468+0000 7f1775968740 20 client.15141 populate_metadata read hostname 'smithi120' 2024-09-19T09:02:31.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.468+0000 7f1775968740 1 --2- 172.21.15.120:0/36653176 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558481f09310 0x558481f08020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:31.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.468+0000 7f1775968740 4 client.15141 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:02:31.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.468+0000 7f1775968740 1 -- 172.21.15.120:0/36653176 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "status"}) -- 0x558481decb10 con 0x558481f09310 2024-09-19T09:02:31.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.468+0000 7f177132b640 1 --2- 172.21.15.120:0/36653176 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558481f09310 0x558481f08020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:31.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.469+0000 7f177132b640 1 --2- 172.21.15.120:0/36653176 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558481f09310 0x558481f08020 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=92434affb6d6cded server_cookie=a3ab4dabf8462ccb in_seq=0 out_seq=0 2024-09-19T09:02:31.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.469+0000 7f1754ff9640 10 client.15141 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:02:31.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.469+0000 7f1754ff9640 1 -- 172.21.15.120:0/36653176 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x558481decb10 con 0x558481f09310 2024-09-19T09:02:31.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.469+0000 7f1754ff9640 10 client.15141 handle_command_reply: tid=0 2024-09-19T09:02:31.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.469+0000 7f1775968740 1 client.15141 shutdown 2024-09-19T09:02:31.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.469+0000 7f1775968740 1 -- 172.21.15.120:0/36653176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558481e7f2f0 msgr2=0x558481e9f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:31.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.469+0000 7f1775968740 1 --2- 172.21.15.120:0/36653176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558481e7f2f0 0x558481e9f980 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x558481e87de0 tx=0x7f175c011080 comp rx=0 tx=0).stop 2024-09-19T09:02:31.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.469+0000 7f1775968740 1 -- 172.21.15.120:0/36653176 shutdown_connections 2024-09-19T09:02:31.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.469+0000 7f1775968740 1 --2- 172.21.15.120:0/36653176 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558481f09310 0x558481f08020 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.469+0000 7f1775968740 1 --2- 172.21.15.120:0/36653176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558481e89bb0 0x558481f02f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.469+0000 7f1775968740 1 --2- 172.21.15.120:0/36653176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558481e7f2f0 0x558481e9f980 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.469+0000 7f1775968740 1 --2- 172.21.15.120:0/36653176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558481e7e9d0 0x558481e9f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.469+0000 7f1775968740 1 -- 172.21.15.120:0/36653176 >> 172.21.15.120:0/36653176 conn(0x558481e7af60 msgr2=0x558481e85450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:31.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.469+0000 7f1754ff9640 0 client.15141 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:02:31.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.469+0000 7f1775968740 1 -- 172.21.15.120:0/36653176 shutdown_connections 2024-09-19T09:02:31.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.469+0000 7f1775968740 1 -- 172.21.15.120:0/36653176 wait complete. 2024-09-19T09:02:31.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.469+0000 7f1775968740 20 client.15141 trim_cache size 0 max 16384 2024-09-19T09:02:31.485 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:02:31.485 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_fsid": "507393d2-7664-11ef-bceb-c7b262605968", 2024-09-19T09:02:31.485 INFO:teuthology.orchestra.run.smithi120.stdout: "whoami": 3, 2024-09-19T09:02:31.485 INFO:teuthology.orchestra.run.smithi120.stdout: "id": 24430, 2024-09-19T09:02:31.485 INFO:teuthology.orchestra.run.smithi120.stdout: "want_state": "up:active", 2024-09-19T09:02:31.486 INFO:teuthology.orchestra.run.smithi120.stdout: "state": "up:active", 2024-09-19T09:02:31.486 INFO:teuthology.orchestra.run.smithi120.stdout: "fs_name": "cephfs", 2024-09-19T09:02:31.486 INFO:teuthology.orchestra.run.smithi120.stdout: "rank_uptime": 94.353022357, 2024-09-19T09:02:31.486 INFO:teuthology.orchestra.run.smithi120.stdout: "mdsmap_epoch": 27, 2024-09-19T09:02:31.486 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch": 81, 2024-09-19T09:02:31.486 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch_barrier": 79, 2024-09-19T09:02:31.486 INFO:teuthology.orchestra.run.smithi120.stdout: "uptime": 128.26106456100001 2024-09-19T09:02:31.486 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:02:31.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.470+0000 7f177358f640 1 -- 172.21.15.120:0/338058308 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f17580a3240 msgr2=0x7f17580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:31.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.470+0000 7f177358f640 1 --2- 172.21.15.120:0/338058308 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f17580a3240 0x7f17580a5700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f176404b000 tx=0x7f176404c000 comp rx=0 tx=0).stop 2024-09-19T09:02:31.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.470+0000 7f177358f640 1 -- 172.21.15.120:0/338058308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f176c102d40 msgr2=0x7f176c1983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:31.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.470+0000 7f177358f640 1 --2- 172.21.15.120:0/338058308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f176c102d40 0x7f176c1983d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f176800d770 tx=0x7f176800dc30 comp rx=0 tx=0).stop 2024-09-19T09:02:31.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.470+0000 7f177358f640 1 -- 172.21.15.120:0/338058308 shutdown_connections 2024-09-19T09:02:31.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.470+0000 7f177358f640 1 --2- 172.21.15.120:0/338058308 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f17580a3240 0x7f17580a5700 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.470+0000 7f177358f640 1 --2- 172.21.15.120:0/338058308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f176c10c170 0x7f176c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.470+0000 7f177358f640 1 --2- 172.21.15.120:0/338058308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f176c103660 0x7f176c198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.470+0000 7f177358f640 1 --2- 172.21.15.120:0/338058308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f176c102d40 0x7f176c1983d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:31.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.470+0000 7f177358f640 1 -- 172.21.15.120:0/338058308 >> 172.21.15.120:0/338058308 conn(0x7f176c0fc7a0 msgr2=0x7f176c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:31.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.470+0000 7f177358f640 1 -- 172.21.15.120:0/338058308 shutdown_connections 2024-09-19T09:02:31.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:31.470+0000 7f177358f640 1 -- 172.21.15.120:0/338058308 wait complete. 2024-09-19T09:02:32.292 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T09:02:32.535 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:32.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:32 smithi196 ceph-mon[35150]: pgmap v331: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-19T09:02:32.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:32 smithi153 ceph-mon[35071]: pgmap v331: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-19T09:02:32.962 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:32 smithi120 ceph-mon[31150]: pgmap v331: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-19T09:02:33.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.040+0000 7fe6a3adb640 1 -- 172.21.15.120:0/3688643392 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe69c106a80 msgr2=0x7fe69c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:33.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.040+0000 7fe6a3adb640 1 --2- 172.21.15.120:0/3688643392 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe69c106a80 0x7fe69c106e60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe694009930 tx=0x7fe69402a420 comp rx=0 tx=0).stop 2024-09-19T09:02:33.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.040+0000 7fe6a3adb640 1 -- 172.21.15.120:0/3688643392 shutdown_connections 2024-09-19T09:02:33.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.040+0000 7fe6a3adb640 1 --2- 172.21.15.120:0/3688643392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe69c101420 0x7fe69c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.040+0000 7fe6a3adb640 1 --2- 172.21.15.120:0/3688643392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe69c100a80 0x7fe69c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.040+0000 7fe6a3adb640 1 --2- 172.21.15.120:0/3688643392 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe69c106a80 0x7fe69c106e60 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.040+0000 7fe6a3adb640 1 -- 172.21.15.120:0/3688643392 >> 172.21.15.120:0/3688643392 conn(0x7fe69c0fc7a0 msgr2=0x7fe69c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:33.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.040+0000 7fe6a3adb640 1 -- 172.21.15.120:0/3688643392 shutdown_connections 2024-09-19T09:02:33.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.040+0000 7fe6a3adb640 1 -- 172.21.15.120:0/3688643392 wait complete. 2024-09-19T09:02:33.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.041+0000 7fe6a3adb640 1 Processor -- start 2024-09-19T09:02:33.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.041+0000 7fe6a3adb640 1 -- start start 2024-09-19T09:02:33.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.041+0000 7fe6a3adb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe69c100a80 0x7fe69c1974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:33.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.041+0000 7fe6a3adb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe69c101420 0x7fe69c197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:33.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.041+0000 7fe6a3adb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe69c106a80 0x7fe69c19bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:33.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.041+0000 7fe6a3adb640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe69c110f20 con 0x7fe69c106a80 2024-09-19T09:02:33.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.041+0000 7fe6a3adb640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe69c110da0 con 0x7fe69c101420 2024-09-19T09:02:33.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.041+0000 7fe6a3adb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe69c1110a0 con 0x7fe69c100a80 2024-09-19T09:02:33.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.041+0000 7fe6a2078640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe69c106a80 0x7fe69c19bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:33.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.041+0000 7fe6a2078640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe69c106a80 0x7fe69c19bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35668/0 (socket says 172.21.15.120:35668) 2024-09-19T09:02:33.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.041+0000 7fe6a2078640 1 -- 172.21.15.120:0/53950189 learned_addr learned my addr 172.21.15.120:0/53950189 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:33.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.041+0000 7fe6a1076640 1 --2- 172.21.15.120:0/53950189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe69c101420 0x7fe69c197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:33.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.041+0000 7fe6a1877640 1 --2- 172.21.15.120:0/53950189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe69c100a80 0x7fe69c1974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:33.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.041+0000 7fe6a2078640 1 -- 172.21.15.120:0/53950189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe69c100a80 msgr2=0x7fe69c1974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:33.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.041+0000 7fe6a2078640 1 --2- 172.21.15.120:0/53950189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe69c100a80 0x7fe69c1974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.041+0000 7fe6a2078640 1 -- 172.21.15.120:0/53950189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe69c101420 msgr2=0x7fe69c197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:33.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.041+0000 7fe6a2078640 1 --2- 172.21.15.120:0/53950189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe69c101420 0x7fe69c197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.041+0000 7fe6a2078640 1 -- 172.21.15.120:0/53950189 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69c19c490 con 0x7fe69c106a80 2024-09-19T09:02:33.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.041+0000 7fe6a1076640 1 --2- 172.21.15.120:0/53950189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe69c101420 0x7fe69c197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:02:33.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.041+0000 7fe6a2078640 1 --2- 172.21.15.120:0/53950189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe69c106a80 0x7fe69c19bdb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe68c00b970 tx=0x7fe68c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:33.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.041+0000 7fe69affd640 1 -- 172.21.15.120:0/53950189 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c019070 con 0x7fe69c106a80 2024-09-19T09:02:33.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.041+0000 7fe6a3adb640 1 -- 172.21.15.120:0/53950189 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe69c19c780 con 0x7fe69c106a80 2024-09-19T09:02:33.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.041+0000 7fe69affd640 1 -- 172.21.15.120:0/53950189 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe68c00cb40 con 0x7fe69c106a80 2024-09-19T09:02:33.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.042+0000 7fe69affd640 1 -- 172.21.15.120:0/53950189 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c007650 con 0x7fe69c106a80 2024-09-19T09:02:33.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.042+0000 7fe6a3adb640 1 -- 172.21.15.120:0/53950189 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe69c1a4060 con 0x7fe69c106a80 2024-09-19T09:02:33.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.044+0000 7fe69affd640 1 -- 172.21.15.120:0/53950189 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fe68c01c070 con 0x7fe69c106a80 2024-09-19T09:02:33.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.044+0000 7fe6a5eb4740 1 Processor -- start 2024-09-19T09:02:33.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.044+0000 7fe6a5eb4740 1 -- start start 2024-09-19T09:02:33.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.044+0000 7fe69affd640 1 --2- 172.21.15.120:0/53950189 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe6840a3240 0x7fe6840a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:33.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.044+0000 7fe6a5eb4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad723be9d0 0x55ad723bedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:33.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.044+0000 7fe6a5eb4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad723bf2f0 0x55ad723c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:33.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.044+0000 7fe6a5eb4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad723c9bb0 0x55ad723cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:33.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.044+0000 7fe6a5eb4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ad72325090 con 0x55ad723bf2f0 2024-09-19T09:02:33.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.044+0000 7fe6a5eb4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ad722cca70 con 0x55ad723be9d0 2024-09-19T09:02:33.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.044+0000 7fe6a5eb4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ad72321050 con 0x55ad723c9bb0 2024-09-19T09:02:33.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.045+0000 7fe6a1076640 1 --2- 172.21.15.120:0/53950189 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe6840a3240 0x7fe6840a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:33.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.045+0000 7fe69affd640 1 -- 172.21.15.120:0/53950189 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe68c0cefb0 con 0x7fe69c106a80 2024-09-19T09:02:33.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.045+0000 7fe6a1877640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad723be9d0 0x55ad723bedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:33.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.045+0000 7fe6a1076640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad723bf2f0 0x55ad723c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:33.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.045+0000 7fe6a1877640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad723be9d0 0x55ad723bedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:48734/0 (socket says 172.21.15.120:48734) 2024-09-19T09:02:33.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.045+0000 7fe6a1877640 1 -- 172.21.15.120:0/1505261193 learned_addr learned my addr 172.21.15.120:0/1505261193 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:33.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.045+0000 7fe6a2078640 1 --2- 172.21.15.120:0/1505261193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad723c9bb0 0x55ad723cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:33.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.045+0000 7fe6a1076640 1 -- 172.21.15.120:0/1505261193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad723c9bb0 msgr2=0x55ad723cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:33.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.045+0000 7fe6a1076640 1 --2- 172.21.15.120:0/1505261193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad723c9bb0 0x55ad723cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.045+0000 7fe6a1076640 1 -- 172.21.15.120:0/1505261193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad723be9d0 msgr2=0x55ad723bedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:33.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.045+0000 7fe6a1076640 1 --2- 172.21.15.120:0/1505261193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad723be9d0 0x55ad723bedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.045+0000 7fe6a1076640 1 -- 172.21.15.120:0/1505261193 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad722d9130 con 0x55ad723bf2f0 2024-09-19T09:02:33.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.045+0000 7fe6a1877640 1 --2- 172.21.15.120:0/1505261193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad723be9d0 0x55ad723bedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:02:33.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.045+0000 7fe6a1076640 1 --2- 172.21.15.120:0/53950189 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe6840a3240 0x7fe6840a5700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe69c0fee30 tx=0x7fe690009820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:33.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.045+0000 7fe6a1076640 1 --2- 172.21.15.120:0/1505261193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad723bf2f0 0x55ad723c9670 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe690006d10 tx=0x7fe69001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40b9e895d7faaa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:33.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.045+0000 7fe683fff640 1 -- 172.21.15.120:0/1505261193 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690026070 con 0x55ad723bf2f0 2024-09-19T09:02:33.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.045+0000 7fe683fff640 1 -- 172.21.15.120:0/1505261193 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe690021920 con 0x55ad723bf2f0 2024-09-19T09:02:33.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.045+0000 7fe683fff640 1 -- 172.21.15.120:0/1505261193 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69000f040 con 0x55ad723bf2f0 2024-09-19T09:02:33.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.046+0000 7fe6a5eb4740 1 -- 172.21.15.120:0/1505261193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad723bf2f0 msgr2=0x55ad723c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:33.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.046+0000 7fe6a5eb4740 1 --2- 172.21.15.120:0/1505261193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad723bf2f0 0x55ad723c9670 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe690006d10 tx=0x7fe69001a000 comp rx=0 tx=0).stop 2024-09-19T09:02:33.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.046+0000 7fe6a5eb4740 1 -- 172.21.15.120:0/1505261193 shutdown_connections 2024-09-19T09:02:33.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.046+0000 7fe6a5eb4740 1 --2- 172.21.15.120:0/1505261193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad723c9bb0 0x55ad723cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.046+0000 7fe6a5eb4740 1 --2- 172.21.15.120:0/1505261193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad723bf2f0 0x55ad723c9670 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.046+0000 7fe6a5eb4740 1 --2- 172.21.15.120:0/1505261193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad723be9d0 0x55ad723bedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.046+0000 7fe6a5eb4740 1 -- 172.21.15.120:0/1505261193 >> 172.21.15.120:0/1505261193 conn(0x55ad723baf60 msgr2=0x55ad723bb340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:33.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.046+0000 7fe6a5eb4740 1 -- 172.21.15.120:0/1505261193 shutdown_connections 2024-09-19T09:02:33.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.046+0000 7fe6a5eb4740 1 -- 172.21.15.120:0/1505261193 wait complete. 2024-09-19T09:02:33.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.047+0000 7fe6a5eb4740 1 -- start start 2024-09-19T09:02:33.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.047+0000 7fe6a5eb4740 1 Processor -- start 2024-09-19T09:02:33.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.047+0000 7fe6a5eb4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad723be9d0 0x55ad723c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:33.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.047+0000 7fe6a5eb4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad723bf2f0 0x55ad723c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:33.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.047+0000 7fe6a5eb4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad723c9bb0 0x55ad723c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:33.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.047+0000 7fe6a5eb4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ad723cc780 con 0x55ad723bf2f0 2024-09-19T09:02:33.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.047+0000 7fe6a5eb4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ad722dff20 con 0x55ad723be9d0 2024-09-19T09:02:33.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.047+0000 7fe6a5eb4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ad723cc900 con 0x55ad723c9bb0 2024-09-19T09:02:33.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.047+0000 7fe6a5eb4740 10 client.0 _ll_register_callbacks cb 0x55ad722a9030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:02:33.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.047+0000 7fe6a1076640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad723bf2f0 0x55ad723c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:33.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.047+0000 7fe6a1076640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad723bf2f0 0x55ad723c6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35686/0 (socket says 172.21.15.120:35686) 2024-09-19T09:02:33.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.047+0000 7fe6a1076640 1 -- 172.21.15.120:0/2985786677 learned_addr learned my addr 172.21.15.120:0/2985786677 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:33.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.047+0000 7fe6a1877640 1 --2- 172.21.15.120:0/2985786677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad723be9d0 0x55ad723c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:33.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.047+0000 7fe6a1076640 1 -- 172.21.15.120:0/2985786677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad723c9bb0 msgr2=0x55ad723c6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:33.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.047+0000 7fe6a2078640 1 --2- 172.21.15.120:0/2985786677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad723c9bb0 0x55ad723c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:33.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.047+0000 7fe6a1076640 1 --2- 172.21.15.120:0/2985786677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad723c9bb0 0x55ad723c6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.047+0000 7fe6a1076640 1 -- 172.21.15.120:0/2985786677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad723be9d0 msgr2=0x55ad723c94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:33.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.047+0000 7fe6a1076640 1 --2- 172.21.15.120:0/2985786677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad723be9d0 0x55ad723c94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.047+0000 7fe6a1076640 1 -- 172.21.15.120:0/2985786677 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad720962b0 con 0x55ad723bf2f0 2024-09-19T09:02:33.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.048+0000 7fe6a1076640 1 --2- 172.21.15.120:0/2985786677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad723bf2f0 0x55ad723c6830 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe690020660 tx=0x7fe690009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ef956770ba5df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:33.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.048+0000 7fe6817fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:02:33.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.048+0000 7fe6817fa640 1 -- 172.21.15.120:0/2985786677 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690026040 con 0x55ad723bf2f0 2024-09-19T09:02:33.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.048+0000 7fe6a5eb4740 1 -- 172.21.15.120:0/2985786677 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe690021920 con 0x55ad723bf2f0 2024-09-19T09:02:33.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.048+0000 7fe6817fa640 1 -- 172.21.15.120:0/2985786677 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe690021ad0 con 0x55ad723bf2f0 2024-09-19T09:02:33.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.048+0000 7fe6817fa640 1 -- 172.21.15.120:0/2985786677 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe690021d10 con 0x55ad723bf2f0 2024-09-19T09:02:33.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.048+0000 7fe6a5eb4740 10 client.15153 fetch_fsmap learned FSMap version 27 2024-09-19T09:02:33.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.048+0000 7fe6a5eb4740 1 -- 172.21.15.120:0/2985786677 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ad723d8130 con 0x55ad723bf2f0 2024-09-19T09:02:33.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.048+0000 7fe6817fa640 1 -- 172.21.15.120:0/2985786677 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe690021920 con 0x55ad723bf2f0 2024-09-19T09:02:33.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.048+0000 7fe6817fa640 1 -- 172.21.15.120:0/2985786677 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe69002c5c0 con 0x55ad723bf2f0 2024-09-19T09:02:33.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.048+0000 7fe6a5eb4740 10 client.15153 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:02:33.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.048+0000 7fe6a5eb4740 10 client.15153 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T09:02:33.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.048+0000 7fe6a5eb4740 20 client.15153 populate_metadata read hostname 'smithi120' 2024-09-19T09:02:33.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.048+0000 7fe6a5eb4740 1 --2- 172.21.15.120:0/2985786677 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55ad723da4b0 0x55ad723db920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:33.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.048+0000 7fe6a5eb4740 4 client.15153 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:02:33.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.048+0000 7fe6a5eb4740 1 -- 172.21.15.120:0/2985786677 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad72332600 con 0x55ad723da4b0 2024-09-19T09:02:33.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.049+0000 7fe6a1877640 1 --2- 172.21.15.120:0/2985786677 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55ad723da4b0 0x55ad723db920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:33.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.049+0000 7fe6817fa640 10 client.15153 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:02:33.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.049+0000 7fe6a1877640 1 --2- 172.21.15.120:0/2985786677 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55ad723da4b0 0x55ad723db920 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d19b01236866e641 server_cookie=dbe68bf50a4645df in_seq=0 out_seq=0 2024-09-19T09:02:33.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.050+0000 7fe6817fa640 1 -- 172.21.15.120:0/2985786677 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ad72332600 con 0x55ad723da4b0 2024-09-19T09:02:33.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.050+0000 7fe6817fa640 10 client.15153 handle_command_reply: tid=0 2024-09-19T09:02:33.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.050+0000 7fe6a5eb4740 1 client.15153 shutdown 2024-09-19T09:02:33.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.050+0000 7fe6a5eb4740 1 -- 172.21.15.120:0/2985786677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad723bf2f0 msgr2=0x55ad723c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:33.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.050+0000 7fe6a5eb4740 1 --2- 172.21.15.120:0/2985786677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad723bf2f0 0x55ad723c6830 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe690020660 tx=0x7fe690009b30 comp rx=0 tx=0).stop 2024-09-19T09:02:33.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.050+0000 7fe6a5eb4740 1 -- 172.21.15.120:0/2985786677 shutdown_connections 2024-09-19T09:02:33.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.050+0000 7fe6a5eb4740 1 --2- 172.21.15.120:0/2985786677 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55ad723da4b0 0x55ad723db920 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.050+0000 7fe6a5eb4740 1 --2- 172.21.15.120:0/2985786677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad723c9bb0 0x55ad723c6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.050+0000 7fe6a5eb4740 1 --2- 172.21.15.120:0/2985786677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad723bf2f0 0x55ad723c6830 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.050+0000 7fe6a5eb4740 1 --2- 172.21.15.120:0/2985786677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad723be9d0 0x55ad723c94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.050+0000 7fe6a5eb4740 1 -- 172.21.15.120:0/2985786677 >> 172.21.15.120:0/2985786677 conn(0x55ad723baf60 msgr2=0x55ad723c0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:33.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.050+0000 7fe6817fa640 0 client.15153 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:02:33.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.050+0000 7fe6a5eb4740 1 -- 172.21.15.120:0/2985786677 shutdown_connections 2024-09-19T09:02:33.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.050+0000 7fe6a5eb4740 1 -- 172.21.15.120:0/2985786677 wait complete. 2024-09-19T09:02:33.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.050+0000 7fe6a5eb4740 20 client.15153 trim_cache size 0 max 16384 2024-09-19T09:02:33.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.067+0000 7fe6a5eb4740 1 Processor -- start 2024-09-19T09:02:33.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.068+0000 7fe6a5eb4740 1 -- start start 2024-09-19T09:02:33.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.068+0000 7fe6a5eb4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad723be9d0 0x55ad723da4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:33.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.068+0000 7fe6a5eb4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad723bf2f0 0x55ad723c3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:33.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.068+0000 7fe6a5eb4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad723c9bb0 0x55ad723c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:33.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.068+0000 7fe6a5eb4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ad723d9c60 con 0x55ad723c9bb0 2024-09-19T09:02:33.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.068+0000 7fe6a5eb4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ad723da330 con 0x55ad723bf2f0 2024-09-19T09:02:33.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.068+0000 7fe6a5eb4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ad723daa00 con 0x55ad723be9d0 2024-09-19T09:02:33.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.068+0000 7fe6a2078640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad723c9bb0 0x55ad723c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:33.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.068+0000 7fe6a2078640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad723c9bb0 0x55ad723c3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35692/0 (socket says 172.21.15.120:35692) 2024-09-19T09:02:33.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.068+0000 7fe6a2078640 1 -- 172.21.15.120:0/3749813065 learned_addr learned my addr 172.21.15.120:0/3749813065 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:33.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.068+0000 7fe6a1877640 1 --2- 172.21.15.120:0/3749813065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad723be9d0 0x55ad723da4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:33.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.068+0000 7fe6a2078640 1 -- 172.21.15.120:0/3749813065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad723be9d0 msgr2=0x55ad723da4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:33.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.068+0000 7fe6a2078640 1 --2- 172.21.15.120:0/3749813065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad723be9d0 0x55ad723da4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.068+0000 7fe6a2078640 1 -- 172.21.15.120:0/3749813065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad723bf2f0 msgr2=0x55ad723c3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:33.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.068+0000 7fe6a2078640 1 --2- 172.21.15.120:0/3749813065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad723bf2f0 0x55ad723c3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.068+0000 7fe6a2078640 1 -- 172.21.15.120:0/3749813065 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad723c9810 con 0x55ad723c9bb0 2024-09-19T09:02:33.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.068+0000 7fe6a2078640 1 --2- 172.21.15.120:0/3749813065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad723c9bb0 0x55ad723c3ff0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe68c097fa0 tx=0x7fe68c0cc980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7350dfb7b7dd91b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:33.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.068+0000 7fe6837fe640 1 -- 172.21.15.120:0/3749813065 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c09bcd0 con 0x55ad723c9bb0 2024-09-19T09:02:33.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.068+0000 7fe6837fe640 1 -- 172.21.15.120:0/3749813065 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe68c078be0 con 0x55ad723c9bb0 2024-09-19T09:02:33.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.068+0000 7fe6837fe640 1 -- 172.21.15.120:0/3749813065 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c0931b0 con 0x55ad723c9bb0 2024-09-19T09:02:33.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.069+0000 7fe6a5eb4740 1 -- 172.21.15.120:0/3749813065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad723c9bb0 msgr2=0x55ad723c3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:33.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.069+0000 7fe6a5eb4740 1 --2- 172.21.15.120:0/3749813065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad723c9bb0 0x55ad723c3ff0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe68c097fa0 tx=0x7fe68c0cc980 comp rx=0 tx=0).stop 2024-09-19T09:02:33.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.069+0000 7fe6a5eb4740 1 -- 172.21.15.120:0/3749813065 shutdown_connections 2024-09-19T09:02:33.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.069+0000 7fe6a5eb4740 1 --2- 172.21.15.120:0/3749813065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad723c9bb0 0x55ad723c3ff0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.069+0000 7fe6a5eb4740 1 --2- 172.21.15.120:0/3749813065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad723bf2f0 0x55ad723c3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.069+0000 7fe6a5eb4740 1 --2- 172.21.15.120:0/3749813065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad723be9d0 0x55ad723da4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.069+0000 7fe6a5eb4740 1 -- 172.21.15.120:0/3749813065 >> 172.21.15.120:0/3749813065 conn(0x55ad723baf60 msgr2=0x55ad723b5ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:33.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.069+0000 7fe6a5eb4740 1 -- 172.21.15.120:0/3749813065 shutdown_connections 2024-09-19T09:02:33.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.069+0000 7fe6a5eb4740 1 -- 172.21.15.120:0/3749813065 wait complete. 2024-09-19T09:02:33.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.069+0000 7fe6a5eb4740 1 -- start start 2024-09-19T09:02:33.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.069+0000 7fe6a5eb4740 1 Processor -- start 2024-09-19T09:02:33.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.070+0000 7fe6a5eb4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad723be9d0 0x55ad723e4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:33.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.070+0000 7fe6a5eb4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad723bf2f0 0x55ad723dd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:33.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.070+0000 7fe6a5eb4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad723c9bb0 0x55ad723dd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:33.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.070+0000 7fe6a5eb4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ad723db7a0 con 0x55ad723c9bb0 2024-09-19T09:02:33.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.070+0000 7fe6a5eb4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ad723db0d0 con 0x55ad723be9d0 2024-09-19T09:02:33.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.070+0000 7fe6a5eb4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ad723dbe70 con 0x55ad723bf2f0 2024-09-19T09:02:33.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.070+0000 7fe6a5eb4740 10 client.0 _ll_register_callbacks cb 0x7fe690024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:02:33.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.070+0000 7fe6a2078640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad723c9bb0 0x55ad723dd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:33.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.070+0000 7fe6a2078640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad723c9bb0 0x55ad723dd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35694/0 (socket says 172.21.15.120:35694) 2024-09-19T09:02:33.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.070+0000 7fe6a2078640 1 -- 172.21.15.120:0/4139990147 learned_addr learned my addr 172.21.15.120:0/4139990147 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:33.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.070+0000 7fe6a1076640 1 --2- 172.21.15.120:0/4139990147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad723bf2f0 0x55ad723dd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:33.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.070+0000 7fe6a2078640 1 -- 172.21.15.120:0/4139990147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad723bf2f0 msgr2=0x55ad723dd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:33.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.070+0000 7fe6a1877640 1 --2- 172.21.15.120:0/4139990147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad723be9d0 0x55ad723e4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:33.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.070+0000 7fe6a2078640 1 --2- 172.21.15.120:0/4139990147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad723bf2f0 0x55ad723dd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.070+0000 7fe6a2078640 1 -- 172.21.15.120:0/4139990147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad723be9d0 msgr2=0x55ad723e4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:33.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.070+0000 7fe6a2078640 1 --2- 172.21.15.120:0/4139990147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad723be9d0 0x55ad723e4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.070+0000 7fe6a2078640 1 -- 172.21.15.120:0/4139990147 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad723ddee0 con 0x55ad723c9bb0 2024-09-19T09:02:33.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.070+0000 7fe6a2078640 1 --2- 172.21.15.120:0/4139990147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad723c9bb0 0x55ad723dd920 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe68c093410 tx=0x7fe68c00c600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d938b786d0529154 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:33.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.070+0000 7fe6817fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:02:33.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.070+0000 7fe6817fa640 1 -- 172.21.15.120:0/4139990147 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c0b4070 con 0x55ad723c9bb0 2024-09-19T09:02:33.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.070+0000 7fe6817fa640 1 -- 172.21.15.120:0/4139990147 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe68c097c40 con 0x55ad723c9bb0 2024-09-19T09:02:33.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.070+0000 7fe6a5eb4740 1 -- 172.21.15.120:0/4139990147 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe68c078be0 con 0x55ad723c9bb0 2024-09-19T09:02:33.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.071+0000 7fe6817fa640 1 -- 172.21.15.120:0/4139990147 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe68c0931b0 con 0x55ad723c9bb0 2024-09-19T09:02:33.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.071+0000 7fe6a5eb4740 10 client.15159 fetch_fsmap learned FSMap version 27 2024-09-19T09:02:33.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.071+0000 7fe6a5eb4740 1 -- 172.21.15.120:0/4139990147 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ad723e2930 con 0x55ad723c9bb0 2024-09-19T09:02:33.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.071+0000 7fe6817fa640 1 -- 172.21.15.120:0/4139990147 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe68c078be0 con 0x55ad723c9bb0 2024-09-19T09:02:33.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.071+0000 7fe6817fa640 1 -- 172.21.15.120:0/4139990147 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe68c09aa20 con 0x55ad723c9bb0 2024-09-19T09:02:33.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.071+0000 7fe6a5eb4740 10 client.15159 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:02:33.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.071+0000 7fe6a5eb4740 10 client.15159 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T09:02:33.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.071+0000 7fe6a5eb4740 20 client.15159 populate_metadata read hostname 'smithi120' 2024-09-19T09:02:33.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.071+0000 7fe6a5eb4740 1 --2- 172.21.15.120:0/4139990147 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55ad72448f80 0x55ad72447c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:33.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.071+0000 7fe6a5eb4740 4 client.15159 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:02:33.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.071+0000 7fe6a5eb4740 1 -- 172.21.15.120:0/4139990147 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "status"}) -- 0x55ad7232cb10 con 0x55ad72448f80 2024-09-19T09:02:33.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.071+0000 7fe6a1877640 1 --2- 172.21.15.120:0/4139990147 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55ad72448f80 0x55ad72447c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:33.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.072+0000 7fe6817fa640 10 client.15159 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:02:33.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.072+0000 7fe6a1877640 1 --2- 172.21.15.120:0/4139990147 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55ad72448f80 0x55ad72447c90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=337fb8d9be83b524 server_cookie=a124aed15a122de5 in_seq=0 out_seq=0 2024-09-19T09:02:33.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.072+0000 7fe6817fa640 1 -- 172.21.15.120:0/4139990147 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ad7232cb10 con 0x55ad72448f80 2024-09-19T09:02:33.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.072+0000 7fe6817fa640 10 client.15159 handle_command_reply: tid=0 2024-09-19T09:02:33.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.072+0000 7fe6a5eb4740 1 client.15159 shutdown 2024-09-19T09:02:33.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.072+0000 7fe6a5eb4740 1 -- 172.21.15.120:0/4139990147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad723c9bb0 msgr2=0x55ad723dd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:33.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.072+0000 7fe6a5eb4740 1 --2- 172.21.15.120:0/4139990147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad723c9bb0 0x55ad723dd920 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe68c093410 tx=0x7fe68c00c600 comp rx=0 tx=0).stop 2024-09-19T09:02:33.092 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:02:33.093 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_fsid": "507393d2-7664-11ef-bceb-c7b262605968", 2024-09-19T09:02:33.093 INFO:teuthology.orchestra.run.smithi120.stdout: "whoami": 2, 2024-09-19T09:02:33.093 INFO:teuthology.orchestra.run.smithi120.stdout: "id": 24442, 2024-09-19T09:02:33.093 INFO:teuthology.orchestra.run.smithi120.stdout: "want_state": "up:active", 2024-09-19T09:02:33.093 INFO:teuthology.orchestra.run.smithi120.stdout: "state": "up:active", 2024-09-19T09:02:33.093 INFO:teuthology.orchestra.run.smithi120.stdout: "fs_name": "cephfs", 2024-09-19T09:02:33.093 INFO:teuthology.orchestra.run.smithi120.stdout: "rank_uptime": 96.963125833999996, 2024-09-19T09:02:33.093 INFO:teuthology.orchestra.run.smithi120.stdout: "mdsmap_epoch": 27, 2024-09-19T09:02:33.093 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch": 81, 2024-09-19T09:02:33.093 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch_barrier": 79, 2024-09-19T09:02:33.093 INFO:teuthology.orchestra.run.smithi120.stdout: "uptime": 124.599719619 2024-09-19T09:02:33.093 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:02:33.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.072+0000 7fe6a5eb4740 1 -- 172.21.15.120:0/4139990147 shutdown_connections 2024-09-19T09:02:33.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.072+0000 7fe6a5eb4740 1 --2- 172.21.15.120:0/4139990147 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55ad72448f80 0x55ad72447c90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.072+0000 7fe6a5eb4740 1 --2- 172.21.15.120:0/4139990147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad723c9bb0 0x55ad723dd920 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.072+0000 7fe6a5eb4740 1 --2- 172.21.15.120:0/4139990147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ad723bf2f0 0x55ad723dd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.072+0000 7fe6a5eb4740 1 --2- 172.21.15.120:0/4139990147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad723be9d0 0x55ad723e4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.072+0000 7fe6817fa640 0 client.15159 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:02:33.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.072+0000 7fe6a5eb4740 1 -- 172.21.15.120:0/4139990147 >> 172.21.15.120:0/4139990147 conn(0x55ad723baf60 msgr2=0x55ad723c7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:33.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.072+0000 7fe6a5eb4740 1 -- 172.21.15.120:0/4139990147 shutdown_connections 2024-09-19T09:02:33.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.072+0000 7fe6a5eb4740 1 -- 172.21.15.120:0/4139990147 wait complete. 2024-09-19T09:02:33.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.072+0000 7fe6a5eb4740 20 client.15159 trim_cache size 0 max 16384 2024-09-19T09:02:33.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.073+0000 7fe6a3adb640 1 -- 172.21.15.120:0/53950189 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe6840a3240 msgr2=0x7fe6840a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:33.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.073+0000 7fe6a3adb640 1 --2- 172.21.15.120:0/53950189 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe6840a3240 0x7fe6840a5700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe69c0fee30 tx=0x7fe690009820 comp rx=0 tx=0).stop 2024-09-19T09:02:33.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.073+0000 7fe6a3adb640 1 -- 172.21.15.120:0/53950189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe69c106a80 msgr2=0x7fe69c19bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:33.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.073+0000 7fe6a3adb640 1 --2- 172.21.15.120:0/53950189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe69c106a80 0x7fe69c19bdb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe68c00b970 tx=0x7fe68c00be30 comp rx=0 tx=0).stop 2024-09-19T09:02:33.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.073+0000 7fe6a3adb640 1 -- 172.21.15.120:0/53950189 shutdown_connections 2024-09-19T09:02:33.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.073+0000 7fe6a3adb640 1 --2- 172.21.15.120:0/53950189 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe6840a3240 0x7fe6840a5700 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.073+0000 7fe6a3adb640 1 --2- 172.21.15.120:0/53950189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe69c106a80 0x7fe69c19bdb0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.073+0000 7fe6a3adb640 1 --2- 172.21.15.120:0/53950189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe69c101420 0x7fe69c197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.073+0000 7fe6a3adb640 1 --2- 172.21.15.120:0/53950189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe69c100a80 0x7fe69c1974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:33.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.073+0000 7fe6a3adb640 1 -- 172.21.15.120:0/53950189 >> 172.21.15.120:0/53950189 conn(0x7fe69c0fc7a0 msgr2=0x7fe69c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:33.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.073+0000 7fe6a3adb640 1 -- 172.21.15.120:0/53950189 shutdown_connections 2024-09-19T09:02:33.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:33.073+0000 7fe6a3adb640 1 -- 172.21.15.120:0/53950189 wait complete. 2024-09-19T09:02:33.879 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T09:02:34.125 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:34.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.751+0000 7f6241be5640 1 -- 172.21.15.120:0/2046102328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f623c102d40 msgr2=0x7f623c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:34.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.751+0000 7f6241be5640 1 --2- 172.21.15.120:0/2046102328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f623c102d40 0x7f623c103120 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f622c009ca0 tx=0x7f622c02a950 comp rx=0 tx=0).stop 2024-09-19T09:02:34.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.751+0000 7f6241be5640 1 -- 172.21.15.120:0/2046102328 shutdown_connections 2024-09-19T09:02:34.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.751+0000 7f6241be5640 1 --2- 172.21.15.120:0/2046102328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f623c10c170 0x7f623c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.751+0000 7f6241be5640 1 --2- 172.21.15.120:0/2046102328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f623c103660 0x7f623c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.751+0000 7f6241be5640 1 --2- 172.21.15.120:0/2046102328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f623c102d40 0x7f623c103120 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.751+0000 7f6241be5640 1 -- 172.21.15.120:0/2046102328 >> 172.21.15.120:0/2046102328 conn(0x7f623c0fc7a0 msgr2=0x7f623c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:34.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.751+0000 7f6241be5640 1 -- 172.21.15.120:0/2046102328 shutdown_connections 2024-09-19T09:02:34.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.751+0000 7f6241be5640 1 -- 172.21.15.120:0/2046102328 wait complete. 2024-09-19T09:02:34.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.751+0000 7f6241be5640 1 Processor -- start 2024-09-19T09:02:34.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.752+0000 7f6241be5640 1 -- start start 2024-09-19T09:02:34.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.752+0000 7f6241be5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f623c102d40 0x7f623c1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:34.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.752+0000 7f6241be5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f623c103660 0x7f623c198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:34.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.752+0000 7f6241be5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f623c10c170 0x7f623c19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:34.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.752+0000 7f6241be5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f623c112030 con 0x7f623c103660 2024-09-19T09:02:34.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.752+0000 7f6241be5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f623c111eb0 con 0x7f623c10c170 2024-09-19T09:02:34.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.752+0000 7f6241be5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f623c1121b0 con 0x7f623c102d40 2024-09-19T09:02:34.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.752+0000 7f623affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f623c103660 0x7f623c198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:34.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.752+0000 7f623affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f623c103660 0x7f623c198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35716/0 (socket says 172.21.15.120:35716) 2024-09-19T09:02:34.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.752+0000 7f623affd640 1 -- 172.21.15.120:0/555802328 learned_addr learned my addr 172.21.15.120:0/555802328 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:34.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.752+0000 7f623b7fe640 1 --2- 172.21.15.120:0/555802328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f623c102d40 0x7f623c1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:34.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.752+0000 7f623bfff640 1 --2- 172.21.15.120:0/555802328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f623c10c170 0x7f623c19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:34.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.752+0000 7f623affd640 1 -- 172.21.15.120:0/555802328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f623c102d40 msgr2=0x7f623c1985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:34.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.752+0000 7f623affd640 1 --2- 172.21.15.120:0/555802328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f623c102d40 0x7f623c1985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.752+0000 7f623affd640 1 -- 172.21.15.120:0/555802328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f623c10c170 msgr2=0x7f623c19ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:34.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.752+0000 7f623affd640 1 --2- 172.21.15.120:0/555802328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f623c10c170 0x7f623c19ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.752+0000 7f623affd640 1 -- 172.21.15.120:0/555802328 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f623c19d550 con 0x7f623c103660 2024-09-19T09:02:34.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.752+0000 7f623bfff640 1 --2- 172.21.15.120:0/555802328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f623c10c170 0x7f623c19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:02:34.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.752+0000 7f623affd640 1 --2- 172.21.15.120:0/555802328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f623c103660 0x7f623c198ae0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f622400b3f0 tx=0x7f622400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:34.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.753+0000 7f6238ff9640 1 -- 172.21.15.120:0/555802328 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6224015020 con 0x7f623c103660 2024-09-19T09:02:34.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.753+0000 7f6238ff9640 1 -- 172.21.15.120:0/555802328 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6224011d10 con 0x7f623c103660 2024-09-19T09:02:34.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.753+0000 7f6241be5640 1 -- 172.21.15.120:0/555802328 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f623c19d840 con 0x7f623c103660 2024-09-19T09:02:34.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.753+0000 7f6238ff9640 1 -- 172.21.15.120:0/555802328 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622400f2e0 con 0x7f623c103660 2024-09-19T09:02:34.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.753+0000 7f6241be5640 1 -- 172.21.15.120:0/555802328 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f623c1a5120 con 0x7f623c103660 2024-09-19T09:02:34.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.755+0000 7f6238ff9640 1 -- 172.21.15.120:0/555802328 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f6224018020 con 0x7f623c103660 2024-09-19T09:02:34.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.755+0000 7f6243fbe740 1 Processor -- start 2024-09-19T09:02:34.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.755+0000 7f6243fbe740 1 -- start start 2024-09-19T09:02:34.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.755+0000 7f6243fbe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562b1aafc8f0 0x562b1aafccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:34.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.755+0000 7f6243fbe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b1aafd210 0x562b1ab0ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:34.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.755+0000 7f6243fbe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b1ab0bfc0 0x562b1ab0e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:34.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.755+0000 7f6243fbe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562b1aa63090 con 0x562b1ab0bfc0 2024-09-19T09:02:34.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.755+0000 7f6243fbe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562b1aa0aa70 con 0x562b1aafc8f0 2024-09-19T09:02:34.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.755+0000 7f6243fbe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562b1aa5f050 con 0x562b1aafd210 2024-09-19T09:02:34.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.756+0000 7f623affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b1ab0bfc0 0x562b1ab0e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:34.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.756+0000 7f623affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b1ab0bfc0 0x562b1ab0e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35718/0 (socket says 172.21.15.120:35718) 2024-09-19T09:02:34.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.756+0000 7f623affd640 1 -- 172.21.15.120:0/3880456294 learned_addr learned my addr 172.21.15.120:0/3880456294 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:34.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.756+0000 7f623b7fe640 1 --2- 172.21.15.120:0/3880456294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b1aafd210 0x562b1ab0ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:34.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.756+0000 7f623bfff640 1 --2- 172.21.15.120:0/3880456294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562b1aafc8f0 0x562b1aafccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:34.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.756+0000 7f623affd640 1 -- 172.21.15.120:0/3880456294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b1aafd210 msgr2=0x562b1ab0ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:34.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.756+0000 7f623affd640 1 --2- 172.21.15.120:0/3880456294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b1aafd210 0x562b1ab0ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.756+0000 7f623affd640 1 -- 172.21.15.120:0/3880456294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562b1aafc8f0 msgr2=0x562b1aafccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:34.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.756+0000 7f623affd640 1 --2- 172.21.15.120:0/3880456294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562b1aafc8f0 0x562b1aafccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.756+0000 7f623affd640 1 -- 172.21.15.120:0/3880456294 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b1aa17130 con 0x562b1ab0bfc0 2024-09-19T09:02:34.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.756+0000 7f6238ff9640 1 --2- 172.21.15.120:0/555802328 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f62280a3240 0x7f62280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:34.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.756+0000 7f623affd640 1 --2- 172.21.15.120:0/3880456294 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b1ab0bfc0 0x562b1ab0e3b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f62240d74b0 tx=0x7f62240d7970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ba0b04a76b3528 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:34.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.756+0000 7f623bfff640 1 --2- 172.21.15.120:0/555802328 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f62280a3240 0x7f62280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:34.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.756+0000 7f6231ffb640 1 -- 172.21.15.120:0/3880456294 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6224097ae0 con 0x562b1ab0bfc0 2024-09-19T09:02:34.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.756+0000 7f6238ff9640 1 -- 172.21.15.120:0/555802328 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f62240cf220 con 0x7f623c103660 2024-09-19T09:02:34.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.756+0000 7f6231ffb640 1 -- 172.21.15.120:0/3880456294 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6224010eb0 con 0x562b1ab0bfc0 2024-09-19T09:02:34.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.756+0000 7f6231ffb640 1 -- 172.21.15.120:0/3880456294 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62240be4b0 con 0x562b1ab0bfc0 2024-09-19T09:02:34.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.757+0000 7f6243fbe740 1 -- 172.21.15.120:0/3880456294 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b1ab0bfc0 msgr2=0x562b1ab0e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:34.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.757+0000 7f623bfff640 1 --2- 172.21.15.120:0/555802328 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f62280a3240 0x7f62280a5700 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6234006fd0 tx=0x7f6234008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:34.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.757+0000 7f6243fbe740 1 --2- 172.21.15.120:0/3880456294 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b1ab0bfc0 0x562b1ab0e3b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f62240d74b0 tx=0x7f62240d7970 comp rx=0 tx=0).stop 2024-09-19T09:02:34.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.757+0000 7f6243fbe740 1 -- 172.21.15.120:0/3880456294 shutdown_connections 2024-09-19T09:02:34.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.757+0000 7f6243fbe740 1 --2- 172.21.15.120:0/3880456294 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b1ab0bfc0 0x562b1ab0e3b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.757+0000 7f6243fbe740 1 --2- 172.21.15.120:0/3880456294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b1aafd210 0x562b1ab0ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.757+0000 7f6243fbe740 1 --2- 172.21.15.120:0/3880456294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562b1aafc8f0 0x562b1aafccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.757+0000 7f6243fbe740 1 -- 172.21.15.120:0/3880456294 >> 172.21.15.120:0/3880456294 conn(0x562b1aaf8f60 msgr2=0x562b1aaf9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:34.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.757+0000 7f6243fbe740 1 -- 172.21.15.120:0/3880456294 shutdown_connections 2024-09-19T09:02:34.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.757+0000 7f6243fbe740 1 -- 172.21.15.120:0/3880456294 wait complete. 2024-09-19T09:02:34.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.758+0000 7f6243fbe740 1 -- start start 2024-09-19T09:02:34.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.758+0000 7f6243fbe740 1 Processor -- start 2024-09-19T09:02:34.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.758+0000 7f6243fbe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b1aafc8f0 0x562b1ab0b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:34.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.758+0000 7f6243fbe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562b1aafd210 0x562b1ab08c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:34.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.758+0000 7f6243fbe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b1ab0bfc0 0x562b1ab09140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:34.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.758+0000 7f6243fbe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562b1ab0e8f0 con 0x562b1ab0bfc0 2024-09-19T09:02:34.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.758+0000 7f6243fbe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562b1aa1df20 con 0x562b1aafd210 2024-09-19T09:02:34.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.758+0000 7f6243fbe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562b1ab0ea70 con 0x562b1aafc8f0 2024-09-19T09:02:34.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.758+0000 7f6243fbe740 10 client.0 _ll_register_callbacks cb 0x562b1a9e7030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:02:34.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.758+0000 7f623bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b1ab0bfc0 0x562b1ab09140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:34.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.758+0000 7f623bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b1ab0bfc0 0x562b1ab09140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35720/0 (socket says 172.21.15.120:35720) 2024-09-19T09:02:34.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.758+0000 7f623bfff640 1 -- 172.21.15.120:0/3678940303 learned_addr learned my addr 172.21.15.120:0/3678940303 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:34.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.758+0000 7f623affd640 1 --2- 172.21.15.120:0/3678940303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562b1aafd210 0x562b1ab08c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:34.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.758+0000 7f623b7fe640 1 --2- 172.21.15.120:0/3678940303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b1aafc8f0 0x562b1ab0b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:34.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.758+0000 7f623bfff640 1 -- 172.21.15.120:0/3678940303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b1aafc8f0 msgr2=0x562b1ab0b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:34.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.759+0000 7f623bfff640 1 --2- 172.21.15.120:0/3678940303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b1aafc8f0 0x562b1ab0b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.759+0000 7f623bfff640 1 -- 172.21.15.120:0/3678940303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562b1aafd210 msgr2=0x562b1ab08c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:34.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.759+0000 7f623bfff640 1 --2- 172.21.15.120:0/3678940303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562b1aafd210 0x562b1ab08c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.759+0000 7f623bfff640 1 -- 172.21.15.120:0/3678940303 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b1a7d42b0 con 0x562b1ab0bfc0 2024-09-19T09:02:34.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.759+0000 7f623b7fe640 1 --2- 172.21.15.120:0/3678940303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b1aafc8f0 0x562b1ab0b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:02:34.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.759+0000 7f623affd640 1 --2- 172.21.15.120:0/3678940303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562b1aafd210 0x562b1ab08c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:02:34.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.759+0000 7f623bfff640 1 --2- 172.21.15.120:0/3678940303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b1ab0bfc0 0x562b1ab09140 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f62340136e0 tx=0x7f62340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0fbd33be5ead85 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:34.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.759+0000 7f621affd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:02:34.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.759+0000 7f621affd640 1 -- 172.21.15.120:0/3678940303 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623401a050 con 0x562b1ab0bfc0 2024-09-19T09:02:34.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.759+0000 7f621affd640 1 -- 172.21.15.120:0/3678940303 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f6234014750 con 0x562b1ab0bfc0 2024-09-19T09:02:34.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.759+0000 7f6243fbe740 1 -- 172.21.15.120:0/3678940303 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562b1aa70600 con 0x562b1ab0bfc0 2024-09-19T09:02:34.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.759+0000 7f621affd640 1 -- 172.21.15.120:0/3678940303 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f62340059c0 con 0x562b1ab0bfc0 2024-09-19T09:02:34.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.759+0000 7f6243fbe740 10 client.15171 fetch_fsmap learned FSMap version 27 2024-09-19T09:02:34.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.759+0000 7f6243fbe740 1 -- 172.21.15.120:0/3678940303 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562b1ab1a300 con 0x562b1ab0bfc0 2024-09-19T09:02:34.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.759+0000 7f621affd640 1 -- 172.21.15.120:0/3678940303 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562b1aa70600 con 0x562b1ab0bfc0 2024-09-19T09:02:34.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.759+0000 7f621affd640 1 -- 172.21.15.120:0/3678940303 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f62340062f0 con 0x562b1ab0bfc0 2024-09-19T09:02:34.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.759+0000 7f6243fbe740 10 client.15171 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:02:34.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.759+0000 7f6243fbe740 10 client.15171 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T09:02:34.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.759+0000 7f6243fbe740 20 client.15171 populate_metadata read hostname 'smithi120' 2024-09-19T09:02:34.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.759+0000 7f6243fbe740 1 --2- 172.21.15.120:0/3678940303 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x562b1ab1c980 0x562b1ab1ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:34.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.760+0000 7f6243fbe740 4 client.15171 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:02:34.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.760+0000 7f6243fbe740 1 -- 172.21.15.120:0/3678940303 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b1aa6ab10 con 0x562b1ab1c980 2024-09-19T09:02:34.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.760+0000 7f623b7fe640 1 --2- 172.21.15.120:0/3678940303 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x562b1ab1c980 0x562b1ab1ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:34.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.760+0000 7f621affd640 10 client.15171 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:02:34.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.760+0000 7f623b7fe640 1 --2- 172.21.15.120:0/3678940303 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x562b1ab1c980 0x562b1ab1ddf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=865fea76ecb700c7 server_cookie=f9f8334ec4041461 in_seq=0 out_seq=0 2024-09-19T09:02:34.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.761+0000 7f621affd640 1 -- 172.21.15.120:0/3678940303 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562b1aa6ab10 con 0x562b1ab1c980 2024-09-19T09:02:34.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.761+0000 7f621affd640 10 client.15171 handle_command_reply: tid=0 2024-09-19T09:02:34.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.761+0000 7f6243fbe740 1 client.15171 shutdown 2024-09-19T09:02:34.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.761+0000 7f6243fbe740 1 -- 172.21.15.120:0/3678940303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b1ab0bfc0 msgr2=0x562b1ab09140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:34.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.761+0000 7f6243fbe740 1 --2- 172.21.15.120:0/3678940303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b1ab0bfc0 0x562b1ab09140 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f62340136e0 tx=0x7f62340046c0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.761+0000 7f6243fbe740 1 -- 172.21.15.120:0/3678940303 shutdown_connections 2024-09-19T09:02:34.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.761+0000 7f6243fbe740 1 --2- 172.21.15.120:0/3678940303 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x562b1ab1c980 0x562b1ab1ddf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.761+0000 7f6243fbe740 1 --2- 172.21.15.120:0/3678940303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b1ab0bfc0 0x562b1ab09140 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.761+0000 7f6243fbe740 1 --2- 172.21.15.120:0/3678940303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562b1aafd210 0x562b1ab08c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.761+0000 7f6243fbe740 1 --2- 172.21.15.120:0/3678940303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b1aafc8f0 0x562b1ab0b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.761+0000 7f6243fbe740 1 -- 172.21.15.120:0/3678940303 >> 172.21.15.120:0/3678940303 conn(0x562b1aaf8f60 msgr2=0x562b1aafebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:34.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.761+0000 7f621affd640 0 client.15171 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:02:34.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.761+0000 7f6243fbe740 1 -- 172.21.15.120:0/3678940303 shutdown_connections 2024-09-19T09:02:34.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.761+0000 7f6243fbe740 1 -- 172.21.15.120:0/3678940303 wait complete. 2024-09-19T09:02:34.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.761+0000 7f6243fbe740 20 client.15171 trim_cache size 0 max 16384 2024-09-19T09:02:34.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.779+0000 7f6243fbe740 1 Processor -- start 2024-09-19T09:02:34.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.779+0000 7f6243fbe740 1 -- start start 2024-09-19T09:02:34.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.779+0000 7f6243fbe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b1aafc8f0 0x562b1ab074d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:34.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.779+0000 7f6243fbe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562b1aafd210 0x562b1ab07a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:34.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.779+0000 7f6243fbe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b1ab0bfc0 0x562b1ab05e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:34.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.779+0000 7f6243fbe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562b1ab1c130 con 0x562b1ab0bfc0 2024-09-19T09:02:34.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.779+0000 7f6243fbe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562b1ab1c800 con 0x562b1aafd210 2024-09-19T09:02:34.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.779+0000 7f6243fbe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562b1ab1ced0 con 0x562b1aafc8f0 2024-09-19T09:02:34.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.779+0000 7f623bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b1ab0bfc0 0x562b1ab05e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:34.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.779+0000 7f623bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b1ab0bfc0 0x562b1ab05e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35730/0 (socket says 172.21.15.120:35730) 2024-09-19T09:02:34.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.779+0000 7f623bfff640 1 -- 172.21.15.120:0/3544482615 learned_addr learned my addr 172.21.15.120:0/3544482615 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:34.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.779+0000 7f623affd640 1 --2- 172.21.15.120:0/3544482615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562b1aafd210 0x562b1ab07a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:34.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.779+0000 7f623b7fe640 1 --2- 172.21.15.120:0/3544482615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b1aafc8f0 0x562b1ab074d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:34.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.779+0000 7f623bfff640 1 -- 172.21.15.120:0/3544482615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b1aafc8f0 msgr2=0x562b1ab074d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:34.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.779+0000 7f623bfff640 1 --2- 172.21.15.120:0/3544482615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b1aafc8f0 0x562b1ab074d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.779+0000 7f623bfff640 1 -- 172.21.15.120:0/3544482615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562b1aafd210 msgr2=0x562b1ab07a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:34.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.779+0000 7f623bfff640 1 --2- 172.21.15.120:0/3544482615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562b1aafd210 0x562b1ab07a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.779+0000 7f623bfff640 1 -- 172.21.15.120:0/3544482615 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b1ab09f80 con 0x562b1ab0bfc0 2024-09-19T09:02:34.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.780+0000 7f623bfff640 1 --2- 172.21.15.120:0/3544482615 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b1ab0bfc0 0x562b1ab05e60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f62340066e0 tx=0x7f6234006a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a37036c1aaefd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:34.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.780+0000 7f62317fa640 1 -- 172.21.15.120:0/3544482615 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6234038070 con 0x562b1ab0bfc0 2024-09-19T09:02:34.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.780+0000 7f62317fa640 1 -- 172.21.15.120:0/3544482615 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6234024070 con 0x562b1ab0bfc0 2024-09-19T09:02:34.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.780+0000 7f62317fa640 1 -- 172.21.15.120:0/3544482615 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623401a040 con 0x562b1ab0bfc0 2024-09-19T09:02:34.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.780+0000 7f6243fbe740 1 -- 172.21.15.120:0/3544482615 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b1ab0bfc0 msgr2=0x562b1ab05e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:34.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.780+0000 7f6243fbe740 1 --2- 172.21.15.120:0/3544482615 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b1ab0bfc0 0x562b1ab05e60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f62340066e0 tx=0x7f6234006a00 comp rx=0 tx=0).stop 2024-09-19T09:02:34.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.780+0000 7f6243fbe740 1 -- 172.21.15.120:0/3544482615 shutdown_connections 2024-09-19T09:02:34.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.780+0000 7f6243fbe740 1 --2- 172.21.15.120:0/3544482615 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b1ab0bfc0 0x562b1ab05e60 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.780+0000 7f6243fbe740 1 --2- 172.21.15.120:0/3544482615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562b1aafd210 0x562b1ab07a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.780+0000 7f6243fbe740 1 --2- 172.21.15.120:0/3544482615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b1aafc8f0 0x562b1ab074d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.780+0000 7f6243fbe740 1 -- 172.21.15.120:0/3544482615 >> 172.21.15.120:0/3544482615 conn(0x562b1aaf8f60 msgr2=0x562b1ab08bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:34.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.781+0000 7f6243fbe740 1 -- 172.21.15.120:0/3544482615 shutdown_connections 2024-09-19T09:02:34.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.781+0000 7f6243fbe740 1 -- 172.21.15.120:0/3544482615 wait complete. 2024-09-19T09:02:34.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.781+0000 7f6243fbe740 1 -- start start 2024-09-19T09:02:34.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.781+0000 7f6243fbe740 1 Processor -- start 2024-09-19T09:02:34.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.781+0000 7f6243fbe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b1aafc8f0 0x562b1ab26190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:34.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.781+0000 7f6243fbe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562b1aafd210 0x562b1ab266d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:34.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.781+0000 7f6243fbe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b1ab0bfc0 0x562b1ab1f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:34.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.781+0000 7f6243fbe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562b1ab1dc70 con 0x562b1aafc8f0 2024-09-19T09:02:34.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.781+0000 7f6243fbe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562b1ab1d5a0 con 0x562b1aafd210 2024-09-19T09:02:34.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.781+0000 7f6243fbe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562b1ab1e340 con 0x562b1ab0bfc0 2024-09-19T09:02:34.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.781+0000 7f6243fbe740 10 client.0 _ll_register_callbacks cb 0x7f623400a1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:02:34.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.781+0000 7f623b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b1aafc8f0 0x562b1ab26190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:34.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.781+0000 7f623b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b1aafc8f0 0x562b1ab26190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35742/0 (socket says 172.21.15.120:35742) 2024-09-19T09:02:34.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.781+0000 7f623b7fe640 1 -- 172.21.15.120:0/21711865 learned_addr learned my addr 172.21.15.120:0/21711865 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:34.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.782+0000 7f623bfff640 1 --2- 172.21.15.120:0/21711865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b1ab0bfc0 0x562b1ab1f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:34.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.782+0000 7f623affd640 1 --2- 172.21.15.120:0/21711865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562b1aafd210 0x562b1ab266d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:34.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.782+0000 7f623b7fe640 1 -- 172.21.15.120:0/21711865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b1ab0bfc0 msgr2=0x562b1ab1f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:34.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.782+0000 7f623b7fe640 1 --2- 172.21.15.120:0/21711865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b1ab0bfc0 0x562b1ab1f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.782+0000 7f623b7fe640 1 -- 172.21.15.120:0/21711865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562b1aafd210 msgr2=0x562b1ab266d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:34.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.782+0000 7f623b7fe640 1 --2- 172.21.15.120:0/21711865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562b1aafd210 0x562b1ab266d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.782+0000 7f623b7fe640 1 -- 172.21.15.120:0/21711865 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b1ab1fd60 con 0x562b1aafc8f0 2024-09-19T09:02:34.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.782+0000 7f623affd640 1 --2- 172.21.15.120:0/21711865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562b1aafd210 0x562b1ab266d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:02:34.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.782+0000 7f623b7fe640 1 --2- 172.21.15.120:0/21711865 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b1aafc8f0 0x562b1ab26190 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f622c006010 tx=0x7f622c02f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743657a96692d2f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:34.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.782+0000 7f621affd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:02:34.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.782+0000 7f621affd640 1 -- 172.21.15.120:0/21711865 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c034040 con 0x562b1aafc8f0 2024-09-19T09:02:34.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.782+0000 7f621affd640 1 -- 172.21.15.120:0/21711865 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f622c02fd50 con 0x562b1aafc8f0 2024-09-19T09:02:34.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.782+0000 7f6243fbe740 1 -- 172.21.15.120:0/21711865 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6234024070 con 0x562b1aafc8f0 2024-09-19T09:02:34.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.782+0000 7f621affd640 1 -- 172.21.15.120:0/21711865 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f622c003970 con 0x562b1aafc8f0 2024-09-19T09:02:34.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.782+0000 7f6243fbe740 10 client.15177 fetch_fsmap learned FSMap version 27 2024-09-19T09:02:34.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.782+0000 7f6243fbe740 1 -- 172.21.15.120:0/21711865 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562b1ab24790 con 0x562b1aafc8f0 2024-09-19T09:02:34.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.782+0000 7f621affd640 1 -- 172.21.15.120:0/21711865 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6234024070 con 0x562b1aafc8f0 2024-09-19T09:02:34.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.783+0000 7f621affd640 1 -- 172.21.15.120:0/21711865 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f622c0453c0 con 0x562b1aafc8f0 2024-09-19T09:02:34.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.783+0000 7f6243fbe740 10 client.15177 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:02:34.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.783+0000 7f6243fbe740 10 client.15177 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T09:02:34.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.783+0000 7f6243fbe740 20 client.15177 populate_metadata read hostname 'smithi120' 2024-09-19T09:02:34.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.783+0000 7f6243fbe740 1 --2- 172.21.15.120:0/21711865 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x562b1ab8b700 0x562b1ab8a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:34.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.783+0000 7f6243fbe740 4 client.15177 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:02:34.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.783+0000 7f6243fbe740 1 -- 172.21.15.120:0/21711865 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "status"}) -- 0x562b1a9e70e0 con 0x562b1ab8b700 2024-09-19T09:02:34.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.783+0000 7f623affd640 1 --2- 172.21.15.120:0/21711865 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x562b1ab8b700 0x562b1ab8a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:34.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.783+0000 7f623affd640 1 --2- 172.21.15.120:0/21711865 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x562b1ab8b700 0x562b1ab8a410 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f01f4bbbecd0e0c9 server_cookie=8428a1837e674d9e in_seq=0 out_seq=0 2024-09-19T09:02:34.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.783+0000 7f621affd640 10 client.15177 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:02:34.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.784+0000 7f621affd640 1 -- 172.21.15.120:0/21711865 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562b1a9e70e0 con 0x562b1ab8b700 2024-09-19T09:02:34.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.784+0000 7f621affd640 10 client.15177 handle_command_reply: tid=0 2024-09-19T09:02:34.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.784+0000 7f6243fbe740 1 client.15177 shutdown 2024-09-19T09:02:34.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.784+0000 7f6243fbe740 1 -- 172.21.15.120:0/21711865 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b1aafc8f0 msgr2=0x562b1ab26190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:34.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.784+0000 7f6243fbe740 1 --2- 172.21.15.120:0/21711865 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b1aafc8f0 0x562b1ab26190 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f622c006010 tx=0x7f622c02f640 comp rx=0 tx=0).stop 2024-09-19T09:02:34.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.784+0000 7f6243fbe740 1 -- 172.21.15.120:0/21711865 shutdown_connections 2024-09-19T09:02:34.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.784+0000 7f6243fbe740 1 --2- 172.21.15.120:0/21711865 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x562b1ab8b700 0x562b1ab8a410 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.784+0000 7f621affd640 0 client.15177 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:02:34.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.784+0000 7f6243fbe740 1 --2- 172.21.15.120:0/21711865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b1ab0bfc0 0x562b1ab1f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.784+0000 7f6243fbe740 1 --2- 172.21.15.120:0/21711865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562b1aafd210 0x562b1ab266d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.784+0000 7f6243fbe740 1 --2- 172.21.15.120:0/21711865 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562b1aafc8f0 0x562b1ab26190 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.784+0000 7f6243fbe740 1 -- 172.21.15.120:0/21711865 >> 172.21.15.120:0/21711865 conn(0x562b1aaf8f60 msgr2=0x562b1ab07490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:34.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.784+0000 7f6243fbe740 1 -- 172.21.15.120:0/21711865 shutdown_connections 2024-09-19T09:02:34.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.784+0000 7f6243fbe740 1 -- 172.21.15.120:0/21711865 wait complete. 2024-09-19T09:02:34.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.784+0000 7f6243fbe740 20 client.15177 trim_cache size 0 max 16384 2024-09-19T09:02:34.793 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:02:34.793 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_fsid": "507393d2-7664-11ef-bceb-c7b262605968", 2024-09-19T09:02:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: "whoami": 1, 2024-09-19T09:02:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: "id": 24446, 2024-09-19T09:02:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: "want_state": "up:active", 2024-09-19T09:02:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: "state": "up:active", 2024-09-19T09:02:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: "fs_name": "cephfs", 2024-09-19T09:02:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: "rank_uptime": 99.684793306000003, 2024-09-19T09:02:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: "mdsmap_epoch": 27, 2024-09-19T09:02:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch": 81, 2024-09-19T09:02:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch_barrier": 79, 2024-09-19T09:02:34.794 INFO:teuthology.orchestra.run.smithi120.stdout: "uptime": 128.81647740400001 2024-09-19T09:02:34.794 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:02:34.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.785+0000 7f6241be5640 1 -- 172.21.15.120:0/555802328 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f62280a3240 msgr2=0x7f62280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:34.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.785+0000 7f6241be5640 1 --2- 172.21.15.120:0/555802328 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f62280a3240 0x7f62280a5700 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6234006fd0 tx=0x7f6234008040 comp rx=0 tx=0).stop 2024-09-19T09:02:34.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.785+0000 7f6241be5640 1 -- 172.21.15.120:0/555802328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f623c103660 msgr2=0x7f623c198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:34.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.785+0000 7f6241be5640 1 --2- 172.21.15.120:0/555802328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f623c103660 0x7f623c198ae0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f622400b3f0 tx=0x7f622400b8b0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.785+0000 7f6241be5640 1 -- 172.21.15.120:0/555802328 shutdown_connections 2024-09-19T09:02:34.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.785+0000 7f6241be5640 1 --2- 172.21.15.120:0/555802328 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f62280a3240 0x7f62280a5700 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.785+0000 7f6241be5640 1 --2- 172.21.15.120:0/555802328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f623c10c170 0x7f623c19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.785+0000 7f6241be5640 1 --2- 172.21.15.120:0/555802328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f623c103660 0x7f623c198ae0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.785+0000 7f6241be5640 1 --2- 172.21.15.120:0/555802328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f623c102d40 0x7f623c1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:34.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.785+0000 7f6241be5640 1 -- 172.21.15.120:0/555802328 >> 172.21.15.120:0/555802328 conn(0x7f623c0fc7a0 msgr2=0x7f623c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:34.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.785+0000 7f6241be5640 1 -- 172.21.15.120:0/555802328 shutdown_connections 2024-09-19T09:02:34.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:34.785+0000 7f6241be5640 1 -- 172.21.15.120:0/555802328 wait complete. 2024-09-19T09:02:35.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:34 smithi120 ceph-mon[31150]: pgmap v332: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-19T09:02:35.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:34 smithi196 ceph-mon[35150]: pgmap v332: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-19T09:02:35.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:34 smithi153 ceph-mon[35071]: pgmap v332: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-19T09:02:35.706 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T09:02:35.950 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:36.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.446+0000 7feb21cdb640 1 -- 172.21.15.120:0/1252114281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb1c102110 msgr2=0x7feb1c10d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:36.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.446+0000 7feb21cdb640 1 --2- 172.21.15.120:0/1252114281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb1c102110 0x7feb1c10d890 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7feb14009ca0 tx=0x7feb1402a950 comp rx=0 tx=0).stop 2024-09-19T09:02:36.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.446+0000 7feb21cdb640 1 -- 172.21.15.120:0/1252114281 shutdown_connections 2024-09-19T09:02:36.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.446+0000 7feb21cdb640 1 --2- 172.21.15.120:0/1252114281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb1c102110 0x7feb1c10d890 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.446+0000 7feb21cdb640 1 --2- 172.21.15.120:0/1252114281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb1c101790 0x7feb1c101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.446+0000 7feb21cdb640 1 --2- 172.21.15.120:0/1252114281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb1c105ba0 0x7feb1c105f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.446+0000 7feb21cdb640 1 -- 172.21.15.120:0/1252114281 >> 172.21.15.120:0/1252114281 conn(0x7feb1c077210 msgr2=0x7feb1c077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:36.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.446+0000 7feb21cdb640 1 -- 172.21.15.120:0/1252114281 shutdown_connections 2024-09-19T09:02:36.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.446+0000 7feb21cdb640 1 -- 172.21.15.120:0/1252114281 wait complete. 2024-09-19T09:02:36.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.447+0000 7feb21cdb640 1 Processor -- start 2024-09-19T09:02:36.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.447+0000 7feb21cdb640 1 -- start start 2024-09-19T09:02:36.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.447+0000 7feb21cdb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb1c101790 0x7feb1c1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:36.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.447+0000 7feb21cdb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb1c102110 0x7feb1c198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:36.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.447+0000 7feb21cdb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb1c105ba0 0x7feb1c19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:36.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.447+0000 7feb21cdb640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c110e60 con 0x7feb1c102110 2024-09-19T09:02:36.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.447+0000 7feb21cdb640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c110ce0 con 0x7feb1c101790 2024-09-19T09:02:36.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.447+0000 7feb21cdb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c110fe0 con 0x7feb1c105ba0 2024-09-19T09:02:36.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.447+0000 7feb1affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb1c102110 0x7feb1c198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:36.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.447+0000 7feb1affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb1c102110 0x7feb1c198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35756/0 (socket says 172.21.15.120:35756) 2024-09-19T09:02:36.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.447+0000 7feb1affd640 1 -- 172.21.15.120:0/400238212 learned_addr learned my addr 172.21.15.120:0/400238212 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:36.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.447+0000 7feb1b7fe640 1 --2- 172.21.15.120:0/400238212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb1c101790 0x7feb1c1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:36.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.447+0000 7feb1bfff640 1 --2- 172.21.15.120:0/400238212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb1c105ba0 0x7feb1c19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:36.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.448+0000 7feb1affd640 1 -- 172.21.15.120:0/400238212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb1c105ba0 msgr2=0x7feb1c19cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:36.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.448+0000 7feb1affd640 1 --2- 172.21.15.120:0/400238212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb1c105ba0 0x7feb1c19cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.448+0000 7feb1affd640 1 -- 172.21.15.120:0/400238212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb1c101790 msgr2=0x7feb1c1982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:36.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.448+0000 7feb1affd640 1 --2- 172.21.15.120:0/400238212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb1c101790 0x7feb1c1982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.448+0000 7feb1affd640 1 -- 172.21.15.120:0/400238212 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1c19d280 con 0x7feb1c102110 2024-09-19T09:02:36.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.448+0000 7feb1b7fe640 1 --2- 172.21.15.120:0/400238212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb1c101790 0x7feb1c1982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:02:36.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.448+0000 7feb1affd640 1 --2- 172.21.15.120:0/400238212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb1c102110 0x7feb1c198810 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7feb1000d950 tx=0x7feb1000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:36.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.448+0000 7feb18ff9640 1 -- 172.21.15.120:0/400238212 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10017070 con 0x7feb1c102110 2024-09-19T09:02:36.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.448+0000 7feb21cdb640 1 -- 172.21.15.120:0/400238212 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb1c19d570 con 0x7feb1c102110 2024-09-19T09:02:36.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.448+0000 7feb18ff9640 1 -- 172.21.15.120:0/400238212 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb10009c30 con 0x7feb1c102110 2024-09-19T09:02:36.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.448+0000 7feb18ff9640 1 -- 172.21.15.120:0/400238212 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1000f6b0 con 0x7feb1c102110 2024-09-19T09:02:36.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.448+0000 7feb21cdb640 1 -- 172.21.15.120:0/400238212 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb1c1a4e50 con 0x7feb1c102110 2024-09-19T09:02:36.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.450+0000 7feb18ff9640 1 -- 172.21.15.120:0/400238212 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7feb100109d0 con 0x7feb1c102110 2024-09-19T09:02:36.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.451+0000 7feb18ff9640 1 --2- 172.21.15.120:0/400238212 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7feb040a3240 0x7feb040a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:36.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.451+0000 7feb1b7fe640 1 --2- 172.21.15.120:0/400238212 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7feb040a3240 0x7feb040a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:36.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.451+0000 7feb18ff9640 1 -- 172.21.15.120:0/400238212 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7feb100cf630 con 0x7feb1c102110 2024-09-19T09:02:36.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.451+0000 7feb240b4740 1 Processor -- start 2024-09-19T09:02:36.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.451+0000 7feb240b4740 1 -- start start 2024-09-19T09:02:36.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.451+0000 7feb1b7fe640 1 --2- 172.21.15.120:0/400238212 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7feb040a3240 0x7feb040a5700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feb0c0058c0 tx=0x7feb0c005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:36.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.451+0000 7feb240b4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2f000e9d0 0x55a2f000edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:36.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.451+0000 7feb240b4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f000f2f0 0x55a2f0019670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:36.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.451+0000 7feb240b4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2f0019bb0 0x55a2f001bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:36.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.451+0000 7feb240b4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a2eff75090 con 0x55a2f000f2f0 2024-09-19T09:02:36.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.451+0000 7feb240b4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a2eff1ca70 con 0x55a2f000e9d0 2024-09-19T09:02:36.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.451+0000 7feb240b4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a2eff71050 con 0x55a2f0019bb0 2024-09-19T09:02:36.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.452+0000 7feb1affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f000f2f0 0x55a2f0019670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:36.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.452+0000 7feb1affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f000f2f0 0x55a2f0019670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35768/0 (socket says 172.21.15.120:35768) 2024-09-19T09:02:36.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.452+0000 7feb1affd640 1 -- 172.21.15.120:0/1903705371 learned_addr learned my addr 172.21.15.120:0/1903705371 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:36.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.452+0000 7feb1bfff640 1 --2- 172.21.15.120:0/1903705371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2f0019bb0 0x55a2f001bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:36.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.452+0000 7feb1b7fe640 1 --2- 172.21.15.120:0/1903705371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2f000e9d0 0x55a2f000edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:36.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.452+0000 7feb1affd640 1 -- 172.21.15.120:0/1903705371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2f0019bb0 msgr2=0x55a2f001bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:36.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.452+0000 7feb1affd640 1 --2- 172.21.15.120:0/1903705371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2f0019bb0 0x55a2f001bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.452+0000 7feb1affd640 1 -- 172.21.15.120:0/1903705371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2f000e9d0 msgr2=0x55a2f000edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:36.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.452+0000 7feb1affd640 1 --2- 172.21.15.120:0/1903705371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2f000e9d0 0x55a2f000edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.452+0000 7feb1affd640 1 -- 172.21.15.120:0/1903705371 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2eff29130 con 0x55a2f000f2f0 2024-09-19T09:02:36.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.452+0000 7feb1affd640 1 --2- 172.21.15.120:0/1903705371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f000f2f0 0x55a2f0019670 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7feb100a4390 tx=0x7feb10009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724b2873ff6ecb03 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:36.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.452+0000 7feb01ffb640 1 -- 172.21.15.120:0/1903705371 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb100b1070 con 0x55a2f000f2f0 2024-09-19T09:02:36.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.452+0000 7feb01ffb640 1 -- 172.21.15.120:0/1903705371 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb100a0000 con 0x55a2f000f2f0 2024-09-19T09:02:36.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.452+0000 7feb01ffb640 1 -- 172.21.15.120:0/1903705371 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10020040 con 0x55a2f000f2f0 2024-09-19T09:02:36.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.453+0000 7feb240b4740 1 -- 172.21.15.120:0/1903705371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f000f2f0 msgr2=0x55a2f0019670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:36.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.453+0000 7feb240b4740 1 --2- 172.21.15.120:0/1903705371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f000f2f0 0x55a2f0019670 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7feb100a4390 tx=0x7feb10009de0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.453+0000 7feb240b4740 1 -- 172.21.15.120:0/1903705371 shutdown_connections 2024-09-19T09:02:36.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.453+0000 7feb240b4740 1 --2- 172.21.15.120:0/1903705371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2f0019bb0 0x55a2f001bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.453+0000 7feb240b4740 1 --2- 172.21.15.120:0/1903705371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f000f2f0 0x55a2f0019670 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.453+0000 7feb240b4740 1 --2- 172.21.15.120:0/1903705371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2f000e9d0 0x55a2f000edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.453+0000 7feb240b4740 1 -- 172.21.15.120:0/1903705371 >> 172.21.15.120:0/1903705371 conn(0x55a2f000af60 msgr2=0x55a2f000b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:36.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.453+0000 7feb240b4740 1 -- 172.21.15.120:0/1903705371 shutdown_connections 2024-09-19T09:02:36.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.453+0000 7feb240b4740 1 -- 172.21.15.120:0/1903705371 wait complete. 2024-09-19T09:02:36.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.453+0000 7feb240b4740 1 -- start start 2024-09-19T09:02:36.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.454+0000 7feb240b4740 1 Processor -- start 2024-09-19T09:02:36.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.454+0000 7feb240b4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f000e9d0 0x55a2f0019480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:36.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.454+0000 7feb240b4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2f000f2f0 0x55a2f00167d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:36.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.454+0000 7feb240b4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2f0019bb0 0x55a2f0016d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:36.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.454+0000 7feb240b4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a2f001c720 con 0x55a2f000e9d0 2024-09-19T09:02:36.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.454+0000 7feb240b4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a2eff2ff20 con 0x55a2f0019bb0 2024-09-19T09:02:36.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.454+0000 7feb240b4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a2f001c8a0 con 0x55a2f000f2f0 2024-09-19T09:02:36.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.454+0000 7feb240b4740 10 client.0 _ll_register_callbacks cb 0x55a2efef9030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:02:36.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.454+0000 7feb1b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f000e9d0 0x55a2f0019480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:36.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.454+0000 7feb1b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f000e9d0 0x55a2f0019480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35770/0 (socket says 172.21.15.120:35770) 2024-09-19T09:02:36.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.454+0000 7feb1b7fe640 1 -- 172.21.15.120:0/1471538116 learned_addr learned my addr 172.21.15.120:0/1471538116 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:36.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.454+0000 7feb1affd640 1 --2- 172.21.15.120:0/1471538116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2f000f2f0 0x55a2f00167d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:36.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.454+0000 7feb1b7fe640 1 -- 172.21.15.120:0/1471538116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2f000f2f0 msgr2=0x55a2f00167d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:36.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.454+0000 7feb1bfff640 1 --2- 172.21.15.120:0/1471538116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2f0019bb0 0x55a2f0016d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:36.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.454+0000 7feb1b7fe640 1 --2- 172.21.15.120:0/1471538116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2f000f2f0 0x55a2f00167d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.454+0000 7feb1b7fe640 1 -- 172.21.15.120:0/1471538116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2f0019bb0 msgr2=0x55a2f0016d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:36.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.454+0000 7feb1b7fe640 1 --2- 172.21.15.120:0/1471538116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2f0019bb0 0x55a2f0016d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.454+0000 7feb1b7fe640 1 -- 172.21.15.120:0/1471538116 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2efce62b0 con 0x55a2f000e9d0 2024-09-19T09:02:36.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.454+0000 7feb1affd640 1 --2- 172.21.15.120:0/1471538116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2f000f2f0 0x55a2f00167d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:02:36.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.454+0000 7feb1b7fe640 1 --2- 172.21.15.120:0/1471538116 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f000e9d0 0x55a2f0019480 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7feb0c0109e0 tx=0x7feb0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5a1e3ca388003 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:36.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.454+0000 7feafaffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:02:36.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.454+0000 7feafaffd640 1 -- 172.21.15.120:0/1471538116 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c00fd50 con 0x55a2f000e9d0 2024-09-19T09:02:36.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.455+0000 7feafaffd640 1 -- 172.21.15.120:0/1471538116 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7feb0c005020 con 0x55a2f000e9d0 2024-09-19T09:02:36.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.455+0000 7feb240b4740 1 -- 172.21.15.120:0/1471538116 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb100a0000 con 0x55a2f000e9d0 2024-09-19T09:02:36.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.455+0000 7feafaffd640 1 -- 172.21.15.120:0/1471538116 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feb0c005220 con 0x55a2f000e9d0 2024-09-19T09:02:36.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.455+0000 7feb240b4740 10 client.15189 fetch_fsmap learned FSMap version 27 2024-09-19T09:02:36.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.455+0000 7feb240b4740 1 -- 172.21.15.120:0/1471538116 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a2f0028120 con 0x55a2f000e9d0 2024-09-19T09:02:36.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.455+0000 7feafaffd640 1 -- 172.21.15.120:0/1471538116 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb100a0000 con 0x55a2f000e9d0 2024-09-19T09:02:36.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.455+0000 7feafaffd640 1 -- 172.21.15.120:0/1471538116 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7feb0c01e430 con 0x55a2f000e9d0 2024-09-19T09:02:36.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.455+0000 7feb240b4740 10 client.15189 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:02:36.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.455+0000 7feb240b4740 10 client.15189 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T09:02:36.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.455+0000 7feb240b4740 20 client.15189 populate_metadata read hostname 'smithi120' 2024-09-19T09:02:36.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.455+0000 7feb240b4740 1 --2- 172.21.15.120:0/1471538116 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a2f002a7a0 0x55a2f002bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:36.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.455+0000 7feb240b4740 4 client.15189 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:02:36.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.455+0000 7feb240b4740 1 -- 172.21.15.120:0/1471538116 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2eff82600 con 0x55a2f002a7a0 2024-09-19T09:02:36.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.455+0000 7feb1affd640 1 --2- 172.21.15.120:0/1471538116 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a2f002a7a0 0x55a2f002bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:36.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.456+0000 7feafaffd640 10 client.15189 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:02:36.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.456+0000 7feb1affd640 1 --2- 172.21.15.120:0/1471538116 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a2f002a7a0 0x55a2f002bc10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3f6ca900f01a789 server_cookie=70ff4758cb4ee6e1 in_seq=0 out_seq=0 2024-09-19T09:02:36.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.457+0000 7feafaffd640 1 -- 172.21.15.120:0/1471538116 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a2eff82600 con 0x55a2f002a7a0 2024-09-19T09:02:36.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.457+0000 7feafaffd640 10 client.15189 handle_command_reply: tid=0 2024-09-19T09:02:36.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.457+0000 7feb240b4740 1 client.15189 shutdown 2024-09-19T09:02:36.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.457+0000 7feb240b4740 1 -- 172.21.15.120:0/1471538116 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f000e9d0 msgr2=0x55a2f0019480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:36.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.457+0000 7feb240b4740 1 --2- 172.21.15.120:0/1471538116 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f000e9d0 0x55a2f0019480 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7feb0c0109e0 tx=0x7feb0c0046c0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.457+0000 7feb240b4740 1 -- 172.21.15.120:0/1471538116 shutdown_connections 2024-09-19T09:02:36.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.457+0000 7feb240b4740 1 --2- 172.21.15.120:0/1471538116 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a2f002a7a0 0x55a2f002bc10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.457+0000 7feb240b4740 1 --2- 172.21.15.120:0/1471538116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2f0019bb0 0x55a2f0016d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.457+0000 7feb240b4740 1 --2- 172.21.15.120:0/1471538116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2f000f2f0 0x55a2f00167d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.457+0000 7feb240b4740 1 --2- 172.21.15.120:0/1471538116 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f000e9d0 0x55a2f0019480 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.457+0000 7feb240b4740 1 -- 172.21.15.120:0/1471538116 >> 172.21.15.120:0/1471538116 conn(0x55a2f000af60 msgr2=0x55a2f0010c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:36.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.457+0000 7feafaffd640 0 client.15189 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:02:36.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.457+0000 7feb240b4740 1 -- 172.21.15.120:0/1471538116 shutdown_connections 2024-09-19T09:02:36.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.457+0000 7feb240b4740 1 -- 172.21.15.120:0/1471538116 wait complete. 2024-09-19T09:02:36.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.457+0000 7feb240b4740 20 client.15189 trim_cache size 0 max 16384 2024-09-19T09:02:36.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.475+0000 7feb240b4740 1 Processor -- start 2024-09-19T09:02:36.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.475+0000 7feb240b4740 1 -- start start 2024-09-19T09:02:36.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.475+0000 7feb240b4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2f000e9d0 0x55a2f002a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:36.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.475+0000 7feb240b4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f000f2f0 0x55a2f002b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:36.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.475+0000 7feb240b4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2f0019bb0 0x55a2f0016790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:36.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.475+0000 7feb240b4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a2f0029f50 con 0x55a2f000f2f0 2024-09-19T09:02:36.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.475+0000 7feb240b4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a2f002a620 con 0x55a2f0019bb0 2024-09-19T09:02:36.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.475+0000 7feb240b4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a2f002acf0 con 0x55a2f000e9d0 2024-09-19T09:02:36.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.475+0000 7feb1affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f000f2f0 0x55a2f002b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:36.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.475+0000 7feb1affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f000f2f0 0x55a2f002b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35774/0 (socket says 172.21.15.120:35774) 2024-09-19T09:02:36.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.475+0000 7feb1affd640 1 -- 172.21.15.120:0/3712613219 learned_addr learned my addr 172.21.15.120:0/3712613219 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:36.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.475+0000 7feb1affd640 1 -- 172.21.15.120:0/3712613219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2f000e9d0 msgr2=0x55a2f002a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:36.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.475+0000 7feb1bfff640 1 --2- 172.21.15.120:0/3712613219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2f0019bb0 0x55a2f0016790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:36.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.475+0000 7feb1b7fe640 1 --2- 172.21.15.120:0/3712613219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2f000e9d0 0x55a2f002a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:36.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.475+0000 7feb1affd640 1 --2- 172.21.15.120:0/3712613219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2f000e9d0 0x55a2f002a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.475+0000 7feb1affd640 1 -- 172.21.15.120:0/3712613219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2f0019bb0 msgr2=0x55a2f0016790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:36.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.475+0000 7feb1affd640 1 --2- 172.21.15.120:0/3712613219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2f0019bb0 0x55a2f0016790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.475+0000 7feb1affd640 1 -- 172.21.15.120:0/3712613219 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2f000fee0 con 0x55a2f000f2f0 2024-09-19T09:02:36.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.475+0000 7feb1b7fe640 1 --2- 172.21.15.120:0/3712613219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2f000e9d0 0x55a2f002a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:02:36.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.475+0000 7feb1affd640 1 --2- 172.21.15.120:0/3712613219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f000f2f0 0x55a2f002b540 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7feb100cf8d0 tx=0x7feb10010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b12ae63c2cdaea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:36.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.476+0000 7feb017fa640 1 -- 172.21.15.120:0/3712613219 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb100b1070 con 0x55a2f000f2f0 2024-09-19T09:02:36.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.476+0000 7feb017fa640 1 -- 172.21.15.120:0/3712613219 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb100b4e50 con 0x55a2f000f2f0 2024-09-19T09:02:36.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.476+0000 7feb017fa640 1 -- 172.21.15.120:0/3712613219 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10020040 con 0x55a2f000f2f0 2024-09-19T09:02:36.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.477+0000 7feb240b4740 1 -- 172.21.15.120:0/3712613219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f000f2f0 msgr2=0x55a2f002b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:36.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.477+0000 7feb240b4740 1 --2- 172.21.15.120:0/3712613219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f000f2f0 0x55a2f002b540 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7feb100cf8d0 tx=0x7feb10010d70 comp rx=0 tx=0).stop 2024-09-19T09:02:36.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.477+0000 7feb240b4740 1 -- 172.21.15.120:0/3712613219 shutdown_connections 2024-09-19T09:02:36.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.477+0000 7feb240b4740 1 --2- 172.21.15.120:0/3712613219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2f0019bb0 0x55a2f0016790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.477+0000 7feb240b4740 1 --2- 172.21.15.120:0/3712613219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f000f2f0 0x55a2f002b540 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.477+0000 7feb240b4740 1 --2- 172.21.15.120:0/3712613219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2f000e9d0 0x55a2f002a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.477+0000 7feb240b4740 1 -- 172.21.15.120:0/3712613219 >> 172.21.15.120:0/3712613219 conn(0x55a2f000af60 msgr2=0x55a2f0015d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:36.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.477+0000 7feb240b4740 1 -- 172.21.15.120:0/3712613219 shutdown_connections 2024-09-19T09:02:36.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.477+0000 7feb240b4740 1 -- 172.21.15.120:0/3712613219 wait complete. 2024-09-19T09:02:36.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.477+0000 7feb240b4740 1 -- start start 2024-09-19T09:02:36.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.477+0000 7feb240b4740 1 Processor -- start 2024-09-19T09:02:36.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.478+0000 7feb240b4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2f000e9d0 0x55a2f0034320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:36.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.478+0000 7feb240b4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2f000f2f0 0x55a2f00320e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:36.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.478+0000 7feb240b4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f0019bb0 0x55a2f0032620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:36.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.478+0000 7feb240b4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a2f002ba90 con 0x55a2f0019bb0 2024-09-19T09:02:36.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.478+0000 7feb240b4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a2f002b3c0 con 0x55a2f000e9d0 2024-09-19T09:02:36.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.478+0000 7feb240b4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a2f002c160 con 0x55a2f000f2f0 2024-09-19T09:02:36.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.478+0000 7feb240b4740 10 client.0 _ll_register_callbacks cb 0x7feb0c014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:02:36.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.478+0000 7feb1bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f0019bb0 0x55a2f0032620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:36.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.478+0000 7feb1bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f0019bb0 0x55a2f0032620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35784/0 (socket says 172.21.15.120:35784) 2024-09-19T09:02:36.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.478+0000 7feb1bfff640 1 -- 172.21.15.120:0/464663741 learned_addr learned my addr 172.21.15.120:0/464663741 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:36.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.478+0000 7feb1affd640 1 --2- 172.21.15.120:0/464663741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2f000f2f0 0x55a2f00320e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:36.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.478+0000 7feb1b7fe640 1 --2- 172.21.15.120:0/464663741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2f000e9d0 0x55a2f0034320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:36.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.478+0000 7feb1bfff640 1 -- 172.21.15.120:0/464663741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2f000f2f0 msgr2=0x55a2f00320e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:36.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.478+0000 7feb1bfff640 1 --2- 172.21.15.120:0/464663741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2f000f2f0 0x55a2f00320e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.478+0000 7feb1bfff640 1 -- 172.21.15.120:0/464663741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2f000e9d0 msgr2=0x55a2f0034320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:36.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.478+0000 7feb1bfff640 1 --2- 172.21.15.120:0/464663741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2f000e9d0 0x55a2f0034320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.478+0000 7feb1bfff640 1 -- 172.21.15.120:0/464663741 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2f0032b90 con 0x55a2f0019bb0 2024-09-19T09:02:36.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.478+0000 7feb1bfff640 1 --2- 172.21.15.120:0/464663741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f0019bb0 0x55a2f0032620 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7feb14005e30 tx=0x7feb1402ffb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431185f3da9d9e47 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:36.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.478+0000 7feafaffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:02:36.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.478+0000 7feafaffd640 1 -- 172.21.15.120:0/464663741 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14049060 con 0x55a2f0019bb0 2024-09-19T09:02:36.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.479+0000 7feafaffd640 1 -- 172.21.15.120:0/464663741 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7feb14003970 con 0x55a2f0019bb0 2024-09-19T09:02:36.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.479+0000 7feb240b4740 1 -- 172.21.15.120:0/464663741 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb100b4e50 con 0x55a2f0019bb0 2024-09-19T09:02:36.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.479+0000 7feafaffd640 1 -- 172.21.15.120:0/464663741 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feb14034040 con 0x55a2f0019bb0 2024-09-19T09:02:36.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.479+0000 7feb240b4740 10 client.15195 fetch_fsmap learned FSMap version 27 2024-09-19T09:02:36.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.479+0000 7feb240b4740 1 -- 172.21.15.120:0/464663741 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a2f002d7f0 con 0x55a2f0019bb0 2024-09-19T09:02:36.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.479+0000 7feafaffd640 1 -- 172.21.15.120:0/464663741 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb100b4e50 con 0x55a2f0019bb0 2024-09-19T09:02:36.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.479+0000 7feafaffd640 1 -- 172.21.15.120:0/464663741 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7feb1404e070 con 0x55a2f0019bb0 2024-09-19T09:02:36.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.479+0000 7feb240b4740 10 client.15195 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:02:36.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.479+0000 7feb240b4740 10 client.15195 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T09:02:36.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.479+0000 7feb240b4740 20 client.15195 populate_metadata read hostname 'smithi120' 2024-09-19T09:02:36.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.479+0000 7feb240b4740 1 --2- 172.21.15.120:0/464663741 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a2f0098790 0x55a2f00974a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:36.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.479+0000 7feb240b4740 4 client.15195 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:02:36.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.479+0000 7feb240b4740 1 -- 172.21.15.120:0/464663741 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "status"}) -- 0x55a2eff7cb10 con 0x55a2f0098790 2024-09-19T09:02:36.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.480+0000 7feb1b7fe640 1 --2- 172.21.15.120:0/464663741 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a2f0098790 0x55a2f00974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:36.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.480+0000 7feb1b7fe640 1 --2- 172.21.15.120:0/464663741 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a2f0098790 0x55a2f00974a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a5ef8a4f8b10dde server_cookie=b41f119df701b79f in_seq=0 out_seq=0 2024-09-19T09:02:36.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.480+0000 7feafaffd640 10 client.15195 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:02:36.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.480+0000 7feafaffd640 1 -- 172.21.15.120:0/464663741 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x55a2eff7cb10 con 0x55a2f0098790 2024-09-19T09:02:36.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.480+0000 7feafaffd640 10 client.15195 handle_command_reply: tid=0 2024-09-19T09:02:36.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.480+0000 7feb240b4740 1 client.15195 shutdown 2024-09-19T09:02:36.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.480+0000 7feb240b4740 1 -- 172.21.15.120:0/464663741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f0019bb0 msgr2=0x55a2f0032620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:36.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.480+0000 7feb240b4740 1 --2- 172.21.15.120:0/464663741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f0019bb0 0x55a2f0032620 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7feb14005e30 tx=0x7feb1402ffb0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.480+0000 7feb240b4740 1 -- 172.21.15.120:0/464663741 shutdown_connections 2024-09-19T09:02:36.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.480+0000 7feb240b4740 1 --2- 172.21.15.120:0/464663741 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a2f0098790 0x55a2f00974a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.480+0000 7feb240b4740 1 --2- 172.21.15.120:0/464663741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2f0019bb0 0x55a2f0032620 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.480+0000 7feb240b4740 1 --2- 172.21.15.120:0/464663741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a2f000f2f0 0x55a2f00320e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.480+0000 7feb240b4740 1 --2- 172.21.15.120:0/464663741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a2f000e9d0 0x55a2f0034320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.480+0000 7feb240b4740 1 -- 172.21.15.120:0/464663741 >> 172.21.15.120:0/464663741 conn(0x55a2f000af60 msgr2=0x55a2f0015d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:36.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.480+0000 7feafaffd640 0 client.15195 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:02:36.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.481+0000 7feb240b4740 1 -- 172.21.15.120:0/464663741 shutdown_connections 2024-09-19T09:02:36.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.481+0000 7feb240b4740 1 -- 172.21.15.120:0/464663741 wait complete. 2024-09-19T09:02:36.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.481+0000 7feb240b4740 20 client.15195 trim_cache size 0 max 16384 2024-09-19T09:02:36.496 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:02:36.497 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_fsid": "507393d2-7664-11ef-bceb-c7b262605968", 2024-09-19T09:02:36.497 INFO:teuthology.orchestra.run.smithi120.stdout: "whoami": 0, 2024-09-19T09:02:36.498 INFO:teuthology.orchestra.run.smithi120.stdout: "id": 24461, 2024-09-19T09:02:36.498 INFO:teuthology.orchestra.run.smithi120.stdout: "want_state": "up:active", 2024-09-19T09:02:36.498 INFO:teuthology.orchestra.run.smithi120.stdout: "state": "up:active", 2024-09-19T09:02:36.498 INFO:teuthology.orchestra.run.smithi120.stdout: "fs_name": "cephfs", 2024-09-19T09:02:36.498 INFO:teuthology.orchestra.run.smithi120.stdout: "rank_uptime": 108.52423947, 2024-09-19T09:02:36.498 INFO:teuthology.orchestra.run.smithi120.stdout: "mdsmap_epoch": 27, 2024-09-19T09:02:36.498 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch": 81, 2024-09-19T09:02:36.498 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch_barrier": 79, 2024-09-19T09:02:36.498 INFO:teuthology.orchestra.run.smithi120.stdout: "uptime": 118.5485741 2024-09-19T09:02:36.499 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:02:36.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.481+0000 7feb21cdb640 1 -- 172.21.15.120:0/400238212 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7feb040a3240 msgr2=0x7feb040a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:36.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.481+0000 7feb21cdb640 1 --2- 172.21.15.120:0/400238212 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7feb040a3240 0x7feb040a5700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feb0c0058c0 tx=0x7feb0c005ec0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.481+0000 7feb21cdb640 1 -- 172.21.15.120:0/400238212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb1c102110 msgr2=0x7feb1c198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:36.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.481+0000 7feb21cdb640 1 --2- 172.21.15.120:0/400238212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb1c102110 0x7feb1c198810 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7feb1000d950 tx=0x7feb1000de10 comp rx=0 tx=0).stop 2024-09-19T09:02:36.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.481+0000 7feb21cdb640 1 -- 172.21.15.120:0/400238212 shutdown_connections 2024-09-19T09:02:36.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.481+0000 7feb21cdb640 1 --2- 172.21.15.120:0/400238212 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7feb040a3240 0x7feb040a5700 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.481+0000 7feb21cdb640 1 --2- 172.21.15.120:0/400238212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb1c105ba0 0x7feb1c19cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.481+0000 7feb21cdb640 1 --2- 172.21.15.120:0/400238212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb1c102110 0x7feb1c198810 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.481+0000 7feb21cdb640 1 --2- 172.21.15.120:0/400238212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb1c101790 0x7feb1c1982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:36.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.481+0000 7feb21cdb640 1 -- 172.21.15.120:0/400238212 >> 172.21.15.120:0/400238212 conn(0x7feb1c077210 msgr2=0x7feb1c0ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:36.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.481+0000 7feb21cdb640 1 -- 172.21.15.120:0/400238212 shutdown_connections 2024-09-19T09:02:36.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:36.481+0000 7feb21cdb640 1 -- 172.21.15.120:0/400238212 wait complete. 2024-09-19T09:02:37.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:36 smithi120 ceph-mon[31150]: pgmap v333: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T09:02:37.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:36 smithi196 ceph-mon[35150]: pgmap v333: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T09:02:37.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:36 smithi153 ceph-mon[35071]: pgmap v333: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T09:02:37.199 INFO:teuthology.run_tasks:Running task sequential... 2024-09-19T09:02:37.209 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-19T09:02:37.209 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-19T09:02:37.210 INFO:teuthology.task.print:Enabling mgr modules 2024-09-19T09:02:37.210 INFO:teuthology.run_tasks:Running task kclient... 2024-09-19T09:02:37.247 INFO:tasks.kclient:Mounting kernel clients... 2024-09-19T09:02:37.248 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-19T09:02:37.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:02:37.248 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list 2024-09-19T09:02:37.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:02:37.266 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link delete ceph-brx 2024-09-19T09:02:37.333 INFO:teuthology.orchestra.run.smithi120.stderr:Cannot find device "ceph-brx" 2024-09-19T09:02:37.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:02:37.335 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-19T09:02:37.335 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T09:02:37.335 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T09:02:37.335 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-19T09:02:37.630 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:38.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.098+0000 7f9c98e1e640 1 -- 172.21.15.120:0/1606486594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c94102570 msgr2=0x7f9c941029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:38.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.098+0000 7f9c98e1e640 1 --2- 172.21.15.120:0/1606486594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c94102570 0x7f9c941029b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c009900 tx=0x7f9c7c02a480 comp rx=0 tx=0).stop 2024-09-19T09:02:38.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.098+0000 7f9c98e1e640 1 -- 172.21.15.120:0/1606486594 shutdown_connections 2024-09-19T09:02:38.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.098+0000 7f9c98e1e640 1 --2- 172.21.15.120:0/1606486594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c94102ef0 0x7f9c9410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:38.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.098+0000 7f9c98e1e640 1 --2- 172.21.15.120:0/1606486594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c94102570 0x7f9c941029b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:38.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.098+0000 7f9c98e1e640 1 --2- 172.21.15.120:0/1606486594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c94100e20 0x7f9c94101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:38.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.098+0000 7f9c98e1e640 1 -- 172.21.15.120:0/1606486594 >> 172.21.15.120:0/1606486594 conn(0x7f9c940fc7d0 msgr2=0x7f9c940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:38.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.098+0000 7f9c98e1e640 1 -- 172.21.15.120:0/1606486594 shutdown_connections 2024-09-19T09:02:38.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.098+0000 7f9c98e1e640 1 -- 172.21.15.120:0/1606486594 wait complete. 2024-09-19T09:02:38.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.098+0000 7f9c98e1e640 1 Processor -- start 2024-09-19T09:02:38.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.099+0000 7f9c98e1e640 1 -- start start 2024-09-19T09:02:38.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.099+0000 7f9c98e1e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c94100e20 0x7f9c94199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:38.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.099+0000 7f9c98e1e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c94102570 0x7f9c94199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:38.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.099+0000 7f9c98e1e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c94102ef0 0x7f9c941a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:38.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.099+0000 7f9c98e1e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9c94110fa0 con 0x7f9c94102ef0 2024-09-19T09:02:38.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.099+0000 7f9c98e1e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9c94110e20 con 0x7f9c94102570 2024-09-19T09:02:38.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.099+0000 7f9c98e1e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9c94111120 con 0x7f9c94100e20 2024-09-19T09:02:38.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.099+0000 7f9c92d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c94102ef0 0x7f9c941a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:38.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.099+0000 7f9c92d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c94102ef0 0x7f9c941a43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35800/0 (socket says 172.21.15.120:35800) 2024-09-19T09:02:38.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.099+0000 7f9c92d9d640 1 -- 172.21.15.120:0/1944156274 learned_addr learned my addr 172.21.15.120:0/1944156274 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:38.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.099+0000 7f9c91d9b640 1 --2- 172.21.15.120:0/1944156274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c94102570 0x7f9c94199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:38.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.099+0000 7f9c9259c640 1 --2- 172.21.15.120:0/1944156274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c94100e20 0x7f9c94199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:38.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.099+0000 7f9c92d9d640 1 -- 172.21.15.120:0/1944156274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c94100e20 msgr2=0x7f9c94199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:38.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.099+0000 7f9c92d9d640 1 --2- 172.21.15.120:0/1944156274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c94100e20 0x7f9c94199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:38.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.099+0000 7f9c92d9d640 1 -- 172.21.15.120:0/1944156274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c94102570 msgr2=0x7f9c94199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:38.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.099+0000 7f9c92d9d640 1 --2- 172.21.15.120:0/1944156274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c94102570 0x7f9c94199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:38.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.099+0000 7f9c92d9d640 1 -- 172.21.15.120:0/1944156274 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c941a48f0 con 0x7f9c94102ef0 2024-09-19T09:02:38.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.099+0000 7f9c92d9d640 1 --2- 172.21.15.120:0/1944156274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c94102ef0 0x7f9c941a43b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9c84009fd0 tx=0x7f9c8400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:38.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.099+0000 7f9c8b7fe640 1 -- 172.21.15.120:0/1944156274 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c84011830 con 0x7f9c94102ef0 2024-09-19T09:02:38.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.099+0000 7f9c98e1e640 1 -- 172.21.15.120:0/1944156274 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c941a4ac0 con 0x7f9c94102ef0 2024-09-19T09:02:38.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.099+0000 7f9c8b7fe640 1 -- 172.21.15.120:0/1944156274 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c84016070 con 0x7f9c94102ef0 2024-09-19T09:02:38.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.099+0000 7f9c8b7fe640 1 -- 172.21.15.120:0/1944156274 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8400f2e0 con 0x7f9c94102ef0 2024-09-19T09:02:38.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.100+0000 7f9c98e1e640 1 -- 172.21.15.120:0/1944156274 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c941a5000 con 0x7f9c94102ef0 2024-09-19T09:02:38.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.100+0000 7f9c98e1e640 1 -- 172.21.15.120:0/1944156274 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c94101200 con 0x7f9c94102ef0 2024-09-19T09:02:38.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.102+0000 7f9c8b7fe640 1 -- 172.21.15.120:0/1944156274 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f9c840119d0 con 0x7f9c94102ef0 2024-09-19T09:02:38.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.102+0000 7f9c8b7fe640 1 --2- 172.21.15.120:0/1944156274 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9c800a3240 0x7f9c800a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:38.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.103+0000 7f9c9259c640 1 --2- 172.21.15.120:0/1944156274 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9c800a3240 0x7f9c800a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:38.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.103+0000 7f9c8b7fe640 1 -- 172.21.15.120:0/1944156274 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f9c840cf680 con 0x7f9c94102ef0 2024-09-19T09:02:38.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.103+0000 7f9c9259c640 1 --2- 172.21.15.120:0/1944156274 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9c800a3240 0x7f9c800a5700 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c007350 tx=0x7f9c8c0072e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:38.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.105+0000 7f9c8b7fe640 1 -- 172.21.15.120:0/1944156274 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9c8409c3a0 con 0x7f9c94102ef0 2024-09-19T09:02:38.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.308+0000 7f9c98e1e640 1 -- 172.21.15.120:0/1944156274 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9c941a5290 con 0x7f9c94102ef0 2024-09-19T09:02:38.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.309+0000 7f9c8b7fe640 1 -- 172.21.15.120:0/1944156274 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f9c8409bca0 con 0x7f9c94102ef0 2024-09-19T09:02:38.309 INFO:teuthology.orchestra.run.smithi120.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T09:02:38.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.311+0000 7f9c98e1e640 1 -- 172.21.15.120:0/1944156274 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9c800a3240 msgr2=0x7f9c800a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:38.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.311+0000 7f9c98e1e640 1 --2- 172.21.15.120:0/1944156274 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9c800a3240 0x7f9c800a5700 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c007350 tx=0x7f9c8c0072e0 comp rx=0 tx=0).stop 2024-09-19T09:02:38.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.311+0000 7f9c98e1e640 1 -- 172.21.15.120:0/1944156274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c94102ef0 msgr2=0x7f9c941a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:38.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.311+0000 7f9c98e1e640 1 --2- 172.21.15.120:0/1944156274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c94102ef0 0x7f9c941a43b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9c84009fd0 tx=0x7f9c8400b8e0 comp rx=0 tx=0).stop 2024-09-19T09:02:38.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.311+0000 7f9c98e1e640 1 -- 172.21.15.120:0/1944156274 shutdown_connections 2024-09-19T09:02:38.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.311+0000 7f9c98e1e640 1 --2- 172.21.15.120:0/1944156274 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9c800a3240 0x7f9c800a5700 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:38.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.311+0000 7f9c98e1e640 1 --2- 172.21.15.120:0/1944156274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c94102ef0 0x7f9c941a43b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:38.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.311+0000 7f9c98e1e640 1 --2- 172.21.15.120:0/1944156274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c94102570 0x7f9c94199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:38.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.311+0000 7f9c98e1e640 1 --2- 172.21.15.120:0/1944156274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c94100e20 0x7f9c94199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:38.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.311+0000 7f9c98e1e640 1 -- 172.21.15.120:0/1944156274 >> 172.21.15.120:0/1944156274 conn(0x7f9c940fc7d0 msgr2=0x7f9c941068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:38.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.311+0000 7f9c98e1e640 1 -- 172.21.15.120:0/1944156274 shutdown_connections 2024-09-19T09:02:38.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:38.311+0000 7f9c98e1e640 1 -- 172.21.15.120:0/1944156274 wait complete. 2024-09-19T09:02:39.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:38 smithi120 ceph-mon[31150]: pgmap v334: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T09:02:39.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:38 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1944156274' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T09:02:39.163 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T09:02:39.163 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T09:02:39.163 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi120.front.sepia.ceph.com 2024-09-19T09:02:39.163 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T09:02:39.163 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T09:02:39.163 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T09:02:39.163 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T09:02:39.164 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T09:02:39.164 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T09:02:39.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:02:39.164 DEBUG:teuthology.orchestra.run.smithi120:> ip addr 2024-09-19T09:02:39.182 INFO:teuthology.orchestra.run.smithi120.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T09:02:39.182 INFO:teuthology.orchestra.run.smithi120.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T09:02:39.182 INFO:teuthology.orchestra.run.smithi120.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T09:02:39.182 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-09-19T09:02:39.182 INFO:teuthology.orchestra.run.smithi120.stdout: inet6 ::1/128 scope host 2024-09-19T09:02:39.182 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-09-19T09:02:39.182 INFO:teuthology.orchestra.run.smithi120.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T09:02:39.182 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:d8:a1:26 brd ff:ff:ff:ff:ff:ff 2024-09-19T09:02:39.182 INFO:teuthology.orchestra.run.smithi120.stdout: altname enp7s0f0 2024-09-19T09:02:39.182 INFO:teuthology.orchestra.run.smithi120.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T09:02:39.183 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:d8:a1:27 brd ff:ff:ff:ff:ff:ff 2024-09-19T09:02:39.183 INFO:teuthology.orchestra.run.smithi120.stdout: altname enp7s0f1 2024-09-19T09:02:39.183 INFO:teuthology.orchestra.run.smithi120.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T09:02:39.183 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:8f:47:8e brd ff:ff:ff:ff:ff:ff 2024-09-19T09:02:39.183 INFO:teuthology.orchestra.run.smithi120.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T09:02:39.183 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:8f:47:8f brd ff:ff:ff:ff:ff:ff 2024-09-19T09:02:39.183 INFO:teuthology.orchestra.run.smithi120.stdout: inet 172.21.15.120/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-19T09:02:39.183 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft 42147sec preferred_lft 42147sec 2024-09-19T09:02:39.183 INFO:teuthology.orchestra.run.smithi120.stdout: inet6 fe80::ec4:7aff:fe8f:478f/64 scope link noprefixroute 2024-09-19T09:02:39.183 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-09-19T09:02:39.184 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T09:02:39.184 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T09:02:39.184 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-09-19T09:02:39.184 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link add name ceph-brx type bridge 2024-09-19T09:02:39.184 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip addr flush dev ceph-brx 2024-09-19T09:02:39.184 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link set ceph-brx up 2024-09-19T09:02:39.184 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T09:02:39.184 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-09-19T09:02:39.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:38 smithi196 ceph-mon[35150]: pgmap v334: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T09:02:39.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:38 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1944156274' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T09:02:39.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:38 smithi153 ceph-mon[35071]: pgmap v334: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T09:02:39.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:38 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1944156274' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T09:02:39.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T09:02:39.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T09:02:39.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:02:39.407 DEBUG:teuthology.orchestra.run.smithi120:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T09:02:39.432 INFO:teuthology.orchestra.run.smithi120.stdout:1 2024-09-19T09:02:39.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:02:39.434 DEBUG:teuthology.orchestra.run.smithi120:> route 2024-09-19T09:02:39.500 INFO:teuthology.orchestra.run.smithi120.stdout:Kernel IP routing table 2024-09-19T09:02:39.500 INFO:teuthology.orchestra.run.smithi120.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T09:02:39.500 INFO:teuthology.orchestra.run.smithi120.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T09:02:39.500 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T09:02:39.500 INFO:teuthology.orchestra.run.smithi120.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T09:02:39.501 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T09:02:39.501 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-09-19T09:02:39.501 DEBUG:teuthology.orchestra.run.smithi120:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T09:02:39.501 DEBUG:teuthology.orchestra.run.smithi120:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T09:02:39.501 DEBUG:teuthology.orchestra.run.smithi120:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T09:02:39.501 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-09-19T09:02:39.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T09:02:39.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T09:02:39.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:02:39.800 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list 2024-09-19T09:02:39.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:02:39.816 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list-id 2024-09-19T09:02:39.872 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T09:02:39.872 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-09-19T09:02:39.872 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T09:02:39.872 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-19T09:02:39.872 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-09-19T09:02:39.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T09:02:39.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T09:02:39.989 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-19T09:02:39.989 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T09:02:39.990 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-09-19T09:02:39.990 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-19T09:02:39.990 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T09:02:39.990 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-19T09:02:39.990 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-19T09:02:39.990 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-19T09:02:39.990 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-09-19T09:02:40.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T09:02:40.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T09:02:40.118 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T09:02:40.118 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-09-19T09:02:40.118 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link set brx.0 up 2024-09-19T09:02:40.118 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T09:02:40.118 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-09-19T09:02:40.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T09:02:40.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T09:02:40.194 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T09:02:40.194 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T09:02:40.210 INFO:teuthology.orchestra.run.smithi120.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T09:02:40.210 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T09:02:40.210 DEBUG:teuthology.orchestra.run.smithi120:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T09:02:40.266 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-19T09:02:40.266 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-19T09:02:40.266 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/f2bee15c-e720-4829-ac9c-1a0401c9d37d 2024-09-19T09:02:40.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:02:40.266 DEBUG:teuthology.orchestra.run.smithi120:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/f2bee15c-e720-4829-ac9c-1a0401c9d37d /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-09-19T09:02:40.376 INFO:teuthology.orchestra.run.smithi120.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-09-19T09:02:40.376 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-09-19T09:02:40.514 INFO:teuthology.orchestra.run.smithi120.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-09-19T09:02:40.514 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-09-19T09:02:40.514 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: resolved to: "172.21.15.120:3300,172.21.15.153:3300,172.21.15.196:3300" 2024-09-19T09:02:40.515 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: trying mount with new device syntax: 0@507393d2-7664-11ef-bceb-c7b262605968.cephfs=/volumes/qa/sv_1/f2bee15c-e720-4829-ac9c-1a0401c9d37d 2024-09-19T09:02:40.515 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,name=0,key=0,mon_addr=172.21.15.120:3300/172.21.15.153:3300/172.21.15.196:3300" will pass to kernel 2024-09-19T09:02:40.515 INFO:teuthology.orchestra.run.smithi120.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-19T09:02:40.516 INFO:teuthology.orchestra.run.smithi120.stdout: You just mounted a file system that supports labels which does not 2024-09-19T09:02:40.516 INFO:teuthology.orchestra.run.smithi120.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-19T09:02:40.516 INFO:teuthology.orchestra.run.smithi120.stdout: applications will generate AVC messages and not be allowed access to 2024-09-19T09:02:40.516 INFO:teuthology.orchestra.run.smithi120.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-19T09:02:40.517 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-19T09:02:40.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:02:40.517 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T09:02:40.547 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T09:02:40.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T09:02:40.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T09:02:40.643 INFO:teuthology.orchestra.run.smithi120.stdout:client15204 2024-09-19T09:02:40.644 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-19T09:02:40.644 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T09:02:40.661 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T09:02:40.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T09:02:40.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T09:02:40.759 INFO:teuthology.orchestra.run.smithi120.stdout:client15204 2024-09-19T09:02:40.759 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T09:02:40.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T09:02:40.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T09:02:40.816 INFO:teuthology.orchestra.run.smithi120.stdout:client15204 2024-09-19T09:02:40.816 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/507393d2-7664-11ef-bceb-c7b262605968.client15204/status') 2024-09-19T09:02:40.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T09:02:40.874 INFO:teuthology.orchestra.run.smithi120.stdout:instance: client.15204 (3)192.168.0.1:0/4144453080 2024-09-19T09:02:40.875 INFO:teuthology.orchestra.run.smithi120.stdout:blocklisted: false 2024-09-19T09:02:40.875 INFO:teuthology.orchestra.run.smithi120.stderr:0+1 records in 2024-09-19T09:02:40.875 INFO:teuthology.orchestra.run.smithi120.stderr:0+1 records out 2024-09-19T09:02:40.875 INFO:teuthology.orchestra.run.smithi120.stderr:70 bytes copied, 2.2367e-05 s, 3.1 MB/s 2024-09-19T09:02:40.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T09:02:40.882 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T09:02:40.892 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T09:02:40.893 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi120.front.sepia.ceph.com 2024-09-19T09:02:40.893 DEBUG:teuthology.orchestra.run.smithi120:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-19T09:02:40.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.993+0000 7fabf74e6640 1 -- 172.21.15.120:0/1645245749 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabf0104f70 msgr2=0x7fabf0107360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:40.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.993+0000 7fabf74e6640 1 --2- 172.21.15.120:0/1645245749 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabf0104f70 0x7fabf0107360 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fabe4009900 tx=0x7fabe402a410 comp rx=0 tx=0).stop 2024-09-19T09:02:40.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.993+0000 7fabf74e6640 1 -- 172.21.15.120:0/1645245749 shutdown_connections 2024-09-19T09:02:40.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.993+0000 7fabf74e6640 1 --2- 172.21.15.120:0/1645245749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabf01078a0 0x7fabf0109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:40.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.993+0000 7fabf74e6640 1 --2- 172.21.15.120:0/1645245749 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabf0104f70 0x7fabf0107360 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:40.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.993+0000 7fabf74e6640 1 --2- 172.21.15.120:0/1645245749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabf0102640 0x7fabf0104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:40.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.993+0000 7fabf74e6640 1 -- 172.21.15.120:0/1645245749 >> 172.21.15.120:0/1645245749 conn(0x7fabf00fc1e0 msgr2=0x7fabf00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:40.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.993+0000 7fabf74e6640 1 -- 172.21.15.120:0/1645245749 shutdown_connections 2024-09-19T09:02:40.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.993+0000 7fabf74e6640 1 -- 172.21.15.120:0/1645245749 wait complete. 2024-09-19T09:02:40.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.994+0000 7fabf74e6640 1 Processor -- start 2024-09-19T09:02:40.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.994+0000 7fabf74e6640 1 -- start start 2024-09-19T09:02:40.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.994+0000 7fabf74e6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabf0102640 0x7fabf0101fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:40.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.994+0000 7fabf74e6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabf0104f70 0x7fabf01005f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:40.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.994+0000 7fabf74e6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabf01078a0 0x7fabf0100b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:40.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.994+0000 7fabf74e6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fabf010d870 con 0x7fabf0104f70 2024-09-19T09:02:40.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.994+0000 7fabf74e6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fabf010d6f0 con 0x7fabf0102640 2024-09-19T09:02:40.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.994+0000 7fabf74e6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fabf010d9f0 con 0x7fabf01078a0 2024-09-19T09:02:40.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.994+0000 7fabf4a81640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabf0104f70 0x7fabf01005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:40.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.994+0000 7fabf4a81640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabf0104f70 0x7fabf01005f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35822/0 (socket says 172.21.15.120:35822) 2024-09-19T09:02:40.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.994+0000 7fabf4a81640 1 -- 172.21.15.120:0/2362455028 learned_addr learned my addr 172.21.15.120:0/2362455028 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:40.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.994+0000 7fabf5282640 1 --2- 172.21.15.120:0/2362455028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabf0102640 0x7fabf0101fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:40.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.994+0000 7fabf4a81640 1 -- 172.21.15.120:0/2362455028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabf01078a0 msgr2=0x7fabf0100b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:40.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.994+0000 7fabf4a81640 1 --2- 172.21.15.120:0/2362455028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabf01078a0 0x7fabf0100b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:40.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.994+0000 7fabf4a81640 1 -- 172.21.15.120:0/2362455028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabf0102640 msgr2=0x7fabf0101fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:40.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.994+0000 7fabf4a81640 1 --2- 172.21.15.120:0/2362455028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabf0102640 0x7fabf0101fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:40.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.994+0000 7fabf4a81640 1 -- 172.21.15.120:0/2362455028 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf01013c0 con 0x7fabf0104f70 2024-09-19T09:02:40.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.994+0000 7fabf4a81640 1 --2- 172.21.15.120:0/2362455028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabf0104f70 0x7fabf01005f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fabe4031610 tx=0x7fabe40043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:40.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.994+0000 7fabee7fc640 1 -- 172.21.15.120:0/2362455028 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe403f030 con 0x7fabf0104f70 2024-09-19T09:02:40.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.995+0000 7fabee7fc640 1 -- 172.21.15.120:0/2362455028 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabe4003ce0 con 0x7fabf0104f70 2024-09-19T09:02:40.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.995+0000 7fabee7fc640 1 -- 172.21.15.120:0/2362455028 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe4035070 con 0x7fabf0104f70 2024-09-19T09:02:40.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.995+0000 7fabf74e6640 1 -- 172.21.15.120:0/2362455028 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf01a1fa0 con 0x7fabf0104f70 2024-09-19T09:02:40.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.995+0000 7fabf74e6640 1 -- 172.21.15.120:0/2362455028 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf01a24e0 con 0x7fabf0104f70 2024-09-19T09:02:40.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.995+0000 7fabf74e6640 1 -- 172.21.15.120:0/2362455028 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabf00fdcd0 con 0x7fabf0104f70 2024-09-19T09:02:40.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.997+0000 7fabee7fc640 1 -- 172.21.15.120:0/2362455028 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fabe4042030 con 0x7fabf0104f70 2024-09-19T09:02:40.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.998+0000 7fabee7fc640 1 --2- 172.21.15.120:0/2362455028 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fabd80a3370 0x7fabd80a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:40.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.998+0000 7fabf5282640 1 --2- 172.21.15.120:0/2362455028 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fabd80a3370 0x7fabd80a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:40.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.998+0000 7fabee7fc640 1 -- 172.21.15.120:0/2362455028 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fabe40f0120 con 0x7fabf0104f70 2024-09-19T09:02:40.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:40.999+0000 7fabf5282640 1 --2- 172.21.15.120:0/2362455028 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fabd80a3370 0x7fabd80a5830 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fabe8005fd0 tx=0x7fabe8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:41.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.001+0000 7fabee7fc640 1 -- 172.21.15.120:0/2362455028 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fabe40bce40 con 0x7fabf0104f70 2024-09-19T09:02:41.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:40 smithi120 ceph-mon[31150]: pgmap v335: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T09:02:41.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:40 smithi196 ceph-mon[35150]: pgmap v335: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T09:02:41.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:40 smithi153 ceph-mon[35071]: pgmap v335: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T09:02:41.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.206+0000 7fabf74e6640 1 -- 172.21.15.120:0/2362455028 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fabf002d7c0 con 0x7fabf0104f70 2024-09-19T09:02:41.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.701+0000 7fabee7fc640 1 -- 172.21.15.120:0/2362455028 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v28) ==== 118+0+0 (secure 0 0 0) 0x7fabe40bc740 con 0x7fabf0104f70 2024-09-19T09:02:41.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.703+0000 7fabf74e6640 1 -- 172.21.15.120:0/2362455028 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fabd80a3370 msgr2=0x7fabd80a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:41.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.703+0000 7fabf74e6640 1 --2- 172.21.15.120:0/2362455028 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fabd80a3370 0x7fabd80a5830 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fabe8005fd0 tx=0x7fabe8005850 comp rx=0 tx=0).stop 2024-09-19T09:02:41.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.703+0000 7fabf74e6640 1 -- 172.21.15.120:0/2362455028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabf0104f70 msgr2=0x7fabf01005f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:41.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.703+0000 7fabf74e6640 1 --2- 172.21.15.120:0/2362455028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabf0104f70 0x7fabf01005f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fabe4031610 tx=0x7fabe40043b0 comp rx=0 tx=0).stop 2024-09-19T09:02:41.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.703+0000 7fabf74e6640 1 -- 172.21.15.120:0/2362455028 shutdown_connections 2024-09-19T09:02:41.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.703+0000 7fabf74e6640 1 --2- 172.21.15.120:0/2362455028 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fabd80a3370 0x7fabd80a5830 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:41.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.703+0000 7fabf74e6640 1 --2- 172.21.15.120:0/2362455028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabf01078a0 0x7fabf0100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:41.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.703+0000 7fabf74e6640 1 --2- 172.21.15.120:0/2362455028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabf0104f70 0x7fabf01005f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:41.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.703+0000 7fabf74e6640 1 --2- 172.21.15.120:0/2362455028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabf0102640 0x7fabf0101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:41.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.703+0000 7fabf74e6640 1 -- 172.21.15.120:0/2362455028 >> 172.21.15.120:0/2362455028 conn(0x7fabf00fc1e0 msgr2=0x7fabf0102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:41.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.704+0000 7fabf74e6640 1 -- 172.21.15.120:0/2362455028 shutdown_connections 2024-09-19T09:02:41.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.704+0000 7fabf74e6640 1 -- 172.21.15.120:0/2362455028 wait complete. 2024-09-19T09:02:41.717 DEBUG:teuthology.orchestra.run.smithi120:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-19T09:02:41.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.823+0000 7fb38c39a640 1 -- 172.21.15.120:0/3455959055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb384102610 msgr2=0x7fb384108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:41.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.823+0000 7fb38c39a640 1 --2- 172.21.15.120:0/3455959055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb384102610 0x7fb384108ea0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb374009ca0 tx=0x7fb37402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:02:41.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.823+0000 7fb38c39a640 1 -- 172.21.15.120:0/3455959055 shutdown_connections 2024-09-19T09:02:41.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.823+0000 7fb38c39a640 1 --2- 172.21.15.120:0/3455959055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb384102610 0x7fb384108ea0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:41.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.823+0000 7fb38c39a640 1 --2- 172.21.15.120:0/3455959055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb384101c50 0x7fb3841020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:41.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.823+0000 7fb38c39a640 1 --2- 172.21.15.120:0/3455959055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb384100a60 0x7fb384100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:41.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.823+0000 7fb38c39a640 1 -- 172.21.15.120:0/3455959055 >> 172.21.15.120:0/3455959055 conn(0x7fb3840fc1e0 msgr2=0x7fb3840fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:41.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.823+0000 7fb38c39a640 1 -- 172.21.15.120:0/3455959055 shutdown_connections 2024-09-19T09:02:41.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.823+0000 7fb38c39a640 1 -- 172.21.15.120:0/3455959055 wait complete. 2024-09-19T09:02:41.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.823+0000 7fb38c39a640 1 Processor -- start 2024-09-19T09:02:41.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.824+0000 7fb38c39a640 1 -- start start 2024-09-19T09:02:41.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.824+0000 7fb38c39a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb384100a60 0x7fb384197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:41.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.824+0000 7fb38c39a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb384101c50 0x7fb384198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:41.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.824+0000 7fb38c39a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb384102610 0x7fb38419f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:41.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.824+0000 7fb38c39a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb38410d830 con 0x7fb384102610 2024-09-19T09:02:41.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.824+0000 7fb38c39a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb38410d6b0 con 0x7fb384101c50 2024-09-19T09:02:41.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.824+0000 7fb38c39a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb38410d9b0 con 0x7fb384100a60 2024-09-19T09:02:41.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.824+0000 7fb38a937640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb384102610 0x7fb38419f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:41.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.824+0000 7fb38a937640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb384102610 0x7fb38419f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48318/0 (socket says 172.21.15.120:48318) 2024-09-19T09:02:41.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.824+0000 7fb38a937640 1 -- 172.21.15.120:0/2579718390 learned_addr learned my addr 172.21.15.120:0/2579718390 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:41.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.824+0000 7fb38a136640 1 --2- 172.21.15.120:0/2579718390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb384100a60 0x7fb384197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:41.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.824+0000 7fb389935640 1 --2- 172.21.15.120:0/2579718390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb384101c50 0x7fb384198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:41.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.824+0000 7fb38a937640 1 -- 172.21.15.120:0/2579718390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb384100a60 msgr2=0x7fb384197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:41.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.824+0000 7fb38a937640 1 --2- 172.21.15.120:0/2579718390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb384100a60 0x7fb384197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:41.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.824+0000 7fb38a937640 1 -- 172.21.15.120:0/2579718390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb384101c50 msgr2=0x7fb384198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:41.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.824+0000 7fb38a937640 1 --2- 172.21.15.120:0/2579718390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb384101c50 0x7fb384198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:41.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.824+0000 7fb38a937640 1 -- 172.21.15.120:0/2579718390 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb38419f950 con 0x7fb384102610 2024-09-19T09:02:41.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.824+0000 7fb389935640 1 --2- 172.21.15.120:0/2579718390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb384101c50 0x7fb384198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:02:41.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.824+0000 7fb38a937640 1 --2- 172.21.15.120:0/2579718390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb384102610 0x7fb38419f2b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb374005e30 tx=0x7fb374031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:41.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.824+0000 7fb3837fe640 1 -- 172.21.15.120:0/2579718390 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb374003c30 con 0x7fb384102610 2024-09-19T09:02:41.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.825+0000 7fb3837fe640 1 -- 172.21.15.120:0/2579718390 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb374003dd0 con 0x7fb384102610 2024-09-19T09:02:41.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.825+0000 7fb38c39a640 1 -- 172.21.15.120:0/2579718390 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb38419fbe0 con 0x7fb384102610 2024-09-19T09:02:41.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.825+0000 7fb3837fe640 1 -- 172.21.15.120:0/2579718390 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37402fde0 con 0x7fb384102610 2024-09-19T09:02:41.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.825+0000 7fb38c39a640 1 -- 172.21.15.120:0/2579718390 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3841a0130 con 0x7fb384102610 2024-09-19T09:02:41.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.825+0000 7fb38c39a640 1 -- 172.21.15.120:0/2579718390 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb384107d80 con 0x7fb384102610 2024-09-19T09:02:41.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.827+0000 7fb3837fe640 1 -- 172.21.15.120:0/2579718390 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fb374004050 con 0x7fb384102610 2024-09-19T09:02:41.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.828+0000 7fb3837fe640 1 --2- 172.21.15.120:0/2579718390 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb36c0a32f0 0x7fb36c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:41.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.828+0000 7fb38a136640 1 --2- 172.21.15.120:0/2579718390 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb36c0a32f0 0x7fb36c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:41.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.828+0000 7fb3837fe640 1 -- 172.21.15.120:0/2579718390 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb3740f0f70 con 0x7fb384102610 2024-09-19T09:02:41.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.828+0000 7fb38a136640 1 --2- 172.21.15.120:0/2579718390 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb36c0a32f0 0x7fb36c0a57b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb384199210 tx=0x7fb37c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:41.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:41.831+0000 7fb3837fe640 1 -- 172.21.15.120:0/2579718390 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb3740bdc90 con 0x7fb384102610 2024-09-19T09:02:42.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.021+0000 7fb38c39a640 1 -- 172.21.15.120:0/2579718390 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fb3840626d0 con 0x7fb36c0a32f0 2024-09-19T09:02:42.027 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:41 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2362455028' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T09:02:42.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.039+0000 7fb3837fe640 1 -- 172.21.15.120:0/2579718390 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fb3840626d0 con 0x7fb36c0a32f0 2024-09-19T09:02:42.040 INFO:teuthology.orchestra.run.smithi120.stdout:{} 2024-09-19T09:02:42.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.041+0000 7fb38c39a640 1 -- 172.21.15.120:0/2579718390 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb36c0a32f0 msgr2=0x7fb36c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:42.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.041+0000 7fb38c39a640 1 --2- 172.21.15.120:0/2579718390 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb36c0a32f0 0x7fb36c0a57b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb384199210 tx=0x7fb37c005f90 comp rx=0 tx=0).stop 2024-09-19T09:02:42.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.041+0000 7fb38c39a640 1 -- 172.21.15.120:0/2579718390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb384102610 msgr2=0x7fb38419f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:42.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.041+0000 7fb38c39a640 1 --2- 172.21.15.120:0/2579718390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb384102610 0x7fb38419f2b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb374005e30 tx=0x7fb374031ee0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.042+0000 7fb38c39a640 1 -- 172.21.15.120:0/2579718390 shutdown_connections 2024-09-19T09:02:42.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.042+0000 7fb38c39a640 1 --2- 172.21.15.120:0/2579718390 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb36c0a32f0 0x7fb36c0a57b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.042+0000 7fb38c39a640 1 --2- 172.21.15.120:0/2579718390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb384102610 0x7fb38419f2b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.042+0000 7fb38c39a640 1 --2- 172.21.15.120:0/2579718390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb384101c50 0x7fb384198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.042+0000 7fb38c39a640 1 --2- 172.21.15.120:0/2579718390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb384100a60 0x7fb384197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.042+0000 7fb38c39a640 1 -- 172.21.15.120:0/2579718390 >> 172.21.15.120:0/2579718390 conn(0x7fb3840fc1e0 msgr2=0x7fb384104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:42.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.042+0000 7fb38c39a640 1 -- 172.21.15.120:0/2579718390 shutdown_connections 2024-09-19T09:02:42.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.042+0000 7fb38c39a640 1 -- 172.21.15.120:0/2579718390 wait complete. 2024-09-19T09:02:42.053 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-19T09:02:42.065 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-19T09:02:42.081 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T09:02:42.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.181+0000 7f7d29add640 1 -- 172.21.15.120:0/2466753525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d241025f0 msgr2=0x7f7d241049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:42.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.181+0000 7f7d29add640 1 --2- 172.21.15.120:0/2466753525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d241025f0 0x7f7d241049e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7d14009900 tx=0x7f7d1402a410 comp rx=0 tx=0).stop 2024-09-19T09:02:42.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.181+0000 7f7d29add640 1 -- 172.21.15.120:0/2466753525 shutdown_connections 2024-09-19T09:02:42.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.181+0000 7f7d29add640 1 --2- 172.21.15.120:0/2466753525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d24107850 0x7f7d24109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.181+0000 7f7d29add640 1 --2- 172.21.15.120:0/2466753525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d24104f20 0x7f7d24107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.181+0000 7f7d29add640 1 --2- 172.21.15.120:0/2466753525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d241025f0 0x7f7d241049e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.181+0000 7f7d29add640 1 -- 172.21.15.120:0/2466753525 >> 172.21.15.120:0/2466753525 conn(0x7f7d240fc170 msgr2=0x7f7d240fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:42.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.181+0000 7f7d29add640 1 -- 172.21.15.120:0/2466753525 shutdown_connections 2024-09-19T09:02:42.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.181+0000 7f7d29add640 1 -- 172.21.15.120:0/2466753525 wait complete. 2024-09-19T09:02:42.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.181+0000 7f7d29add640 1 Processor -- start 2024-09-19T09:02:42.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.182+0000 7f7d29add640 1 -- start start 2024-09-19T09:02:42.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.182+0000 7f7d29add640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d241025f0 0x7f7d24197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:42.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.182+0000 7f7d29add640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d24104f20 0x7f7d241983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:42.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.182+0000 7f7d29add640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d24107850 0x7f7d2419f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:42.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.182+0000 7f7d29add640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2410d7e0 con 0x7f7d241025f0 2024-09-19T09:02:42.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.182+0000 7f7d29add640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2410d660 con 0x7f7d24107850 2024-09-19T09:02:42.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.182+0000 7f7d29add640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2410d960 con 0x7f7d24104f20 2024-09-19T09:02:42.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.182+0000 7f7d237fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d241025f0 0x7f7d24197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:42.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.182+0000 7f7d237fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d241025f0 0x7f7d24197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48362/0 (socket says 172.21.15.120:48362) 2024-09-19T09:02:42.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.182+0000 7f7d237fe640 1 -- 172.21.15.120:0/3268788602 learned_addr learned my addr 172.21.15.120:0/3268788602 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:42.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.182+0000 7f7d22ffd640 1 --2- 172.21.15.120:0/3268788602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d24104f20 0x7f7d241983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:42.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.182+0000 7f7d237fe640 1 -- 172.21.15.120:0/3268788602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d24104f20 msgr2=0x7f7d241983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:42.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.182+0000 7f7d237fe640 1 --2- 172.21.15.120:0/3268788602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d24104f20 0x7f7d241983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.182+0000 7f7d237fe640 1 -- 172.21.15.120:0/3268788602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d24107850 msgr2=0x7f7d2419f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:42.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.182+0000 7f7d23fff640 1 --2- 172.21.15.120:0/3268788602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d24107850 0x7f7d2419f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:42.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.182+0000 7f7d237fe640 1 --2- 172.21.15.120:0/3268788602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d24107850 0x7f7d2419f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.182+0000 7f7d237fe640 1 -- 172.21.15.120:0/3268788602 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2419fac0 con 0x7f7d241025f0 2024-09-19T09:02:42.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.182+0000 7f7d22ffd640 1 --2- 172.21.15.120:0/3268788602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d24104f20 0x7f7d241983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:02:42.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.182+0000 7f7d237fe640 1 --2- 172.21.15.120:0/3268788602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d241025f0 0x7f7d24197ea0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7d14009230 tx=0x7f7d14004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:42.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.183+0000 7f7d20ff9640 1 -- 172.21.15.120:0/3268788602 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1403f030 con 0x7f7d241025f0 2024-09-19T09:02:42.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.183+0000 7f7d29add640 1 -- 172.21.15.120:0/3268788602 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d2419fd50 con 0x7f7d241025f0 2024-09-19T09:02:42.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.183+0000 7f7d20ff9640 1 -- 172.21.15.120:0/3268788602 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d14003ce0 con 0x7f7d241025f0 2024-09-19T09:02:42.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.183+0000 7f7d20ff9640 1 -- 172.21.15.120:0/3268788602 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d14035070 con 0x7f7d241025f0 2024-09-19T09:02:42.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.183+0000 7f7d29add640 1 -- 172.21.15.120:0/3268788602 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d241a0180 con 0x7f7d241025f0 2024-09-19T09:02:42.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:41 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2362455028' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T09:02:42.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.184+0000 7f7d29add640 1 -- 172.21.15.120:0/3268788602 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d240fdc60 con 0x7f7d241025f0 2024-09-19T09:02:42.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.185+0000 7f7d20ff9640 1 -- 172.21.15.120:0/3268788602 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f7d14042030 con 0x7f7d241025f0 2024-09-19T09:02:42.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.185+0000 7f7d20ff9640 1 --2- 172.21.15.120:0/3268788602 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7d0c0a3250 0x7f7d0c0a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:42.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.186+0000 7f7d20ff9640 1 -- 172.21.15.120:0/3268788602 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f7d140f0390 con 0x7f7d241025f0 2024-09-19T09:02:42.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.186+0000 7f7d22ffd640 1 --2- 172.21.15.120:0/3268788602 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7d0c0a3250 0x7f7d0c0a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:42.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.186+0000 7f7d22ffd640 1 --2- 172.21.15.120:0/3268788602 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7d0c0a3250 0x7f7d0c0a5710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7d241993b0 tx=0x7f7d18005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:42.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:41 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2362455028' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T09:02:42.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.188+0000 7f7d20ff9640 1 -- 172.21.15.120:0/3268788602 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7d140bd0b0 con 0x7f7d241025f0 2024-09-19T09:02:42.325 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 09:02:42 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T09:02:42.026+0000 7f69457f8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T09:02:42.325 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 09:02:42 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T09:02:42.026+0000 7f69457f8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T09:02:42.325 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 09:02:42 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T09:02:42.026+0000 7f69457f8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T09:02:42.325 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 09:02:42 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T09:02:42.026+0000 7f69457f8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T09:02:42.325 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 09:02:42 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T09:02:42.026+0000 7f69457f8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T09:02:42.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.358+0000 7f7d29add640 1 -- 172.21.15.120:0/3268788602 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d24062ef0 con 0x7f7d241025f0 2024-09-19T09:02:42.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.359+0000 7f7d20ff9640 1 -- 172.21.15.120:0/3268788602 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22844 (secure 0 0 0) 0x7f7d140bc9b0 con 0x7f7d241025f0 2024-09-19T09:02:42.360 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:02:42.361 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":81,"fsid":"507393d2-7664-11ef-bceb-c7b262605968","created":"2024-09-19T08:53:06.386526+0000","modified":"2024-09-19T09:01:52.744203+0000","last_up_change":"2024-09-19T08:59:55.692659+0000","last_in_change":"2024-09-19T08:59:32.808860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:56:07.830047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:00:43.774339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:00:45.786262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33ec3993-576c-428c-bf75-adcb17cd72a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6803","nonce":4131180019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6805","nonce":4131180019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6809","nonce":4131180019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6807","nonce":4131180019}]},"public_addr":"172.21.15.120:6803/4131180019","cluster_addr":"172.21.15.120:6805/4131180019","heartbeat_back_addr":"172.21.15.120:6809/4131180019","heartbeat_front_addr":"172.21.15.120:6807/4131180019","state":["exists","up"]},{"osd":1,"uuid":"450bbebf-2aac-47c6-8474-bbd37fd0a1a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6801","nonce":2259730294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6803","nonce":2259730294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6807","nonce":2259730294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6805","nonce":2259730294}]},"public_addr":"172.21.15.153:6801/2259730294","cluster_addr":"172.21.15.153:6803/2259730294","heartbeat_back_addr":"172.21.15.153:6807/2259730294","heartbeat_front_addr":"172.21.15.153:6805/2259730294","state":["exists","up"]},{"osd":2,"uuid":"ac8c4b83-8e8e-4803-b266-262f7873bd85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6801","nonce":2942286570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6803","nonce":2942286570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6807","nonce":2942286570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6805","nonce":2942286570}]},"public_addr":"172.21.15.196:6801/2942286570","cluster_addr":"172.21.15.196:6803/2942286570","heartbeat_back_addr":"172.21.15.196:6807/2942286570","heartbeat_front_addr":"172.21.15.196:6805/2942286570","state":["exists","up"]},{"osd":3,"uuid":"c1d9e507-8f02-403e-87da-9b4966fd8fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6811","nonce":713205668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6813","nonce":713205668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6817","nonce":713205668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6815","nonce":713205668}]},"public_addr":"172.21.15.120:6811/713205668","cluster_addr":"172.21.15.120:6813/713205668","heartbeat_back_addr":"172.21.15.120:6817/713205668","heartbeat_front_addr":"172.21.15.120:6815/713205668","state":["exists","up"]},{"osd":4,"uuid":"7f8bf3b9-0ec4-46f6-a406-d26d7971179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6809","nonce":2680865854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6811","nonce":2680865854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6815","nonce":2680865854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6813","nonce":2680865854}]},"public_addr":"172.21.15.153:6809/2680865854","cluster_addr":"172.21.15.153:6811/2680865854","heartbeat_back_addr":"172.21.15.153:6815/2680865854","heartbeat_front_addr":"172.21.15.153:6813/2680865854","state":["exists","up"]},{"osd":5,"uuid":"70b270b2-209d-4ff5-875a-04ee7b992a93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6809","nonce":3763331116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6811","nonce":3763331116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6815","nonce":3763331116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6813","nonce":3763331116}]},"public_addr":"172.21.15.196:6809/3763331116","cluster_addr":"172.21.15.196:6811/3763331116","heartbeat_back_addr":"172.21.15.196:6815/3763331116","heartbeat_front_addr":"172.21.15.196:6813/3763331116","state":["exists","up"]},{"osd":6,"uuid":"8087f69c-dc31-4522-a18d-4e6ab1b7cb27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6819","nonce":4171773765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6821","nonce":4171773765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6825","nonce":4171773765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6823","nonce":4171773765}]},"public_addr":"172.21.15.120:6819/4171773765","cluster_addr":"172.21.15.120:6821/4171773765","heartbeat_back_addr":"172.21.15.120:6825/4171773765","heartbeat_front_addr":"172.21.15.120:6823/4171773765","state":["exists","up"]},{"osd":7,"uuid":"c3b1ead3-150f-43fd-8924-f639203f4cdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6817","nonce":2535148453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6819","nonce":2535148453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6823","nonce":2535148453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6821","nonce":2535148453}]},"public_addr":"172.21.15.153:6817/2535148453","cluster_addr":"172.21.15.153:6819/2535148453","heartbeat_back_addr":"172.21.15.153:6823/2535148453","heartbeat_front_addr":"172.21.15.153:6821/2535148453","state":["exists","up"]},{"osd":8,"uuid":"a3273a2a-968e-4e93-a9cf-3879892bdeab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6817","nonce":186195906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6819","nonce":186195906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6823","nonce":186195906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6821","nonce":186195906}]},"public_addr":"172.21.15.196:6817/186195906","cluster_addr":"172.21.15.196:6819/186195906","heartbeat_back_addr":"172.21.15.196:6823/186195906","heartbeat_front_addr":"172.21.15.196:6821/186195906","state":["exists","up"]},{"osd":9,"uuid":"6d184513-dab8-45f7-bc08-fa2e8b2f10b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6827","nonce":1559320883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6829","nonce":1559320883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6833","nonce":1559320883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6831","nonce":1559320883}]},"public_addr":"172.21.15.120:6827/1559320883","cluster_addr":"172.21.15.120:6829/1559320883","heartbeat_back_addr":"172.21.15.120:6833/1559320883","heartbeat_front_addr":"172.21.15.120:6831/1559320883","state":["exists","up"]},{"osd":10,"uuid":"797db886-62f9-4316-a45b-4d50512e65d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6825","nonce":3216877560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6827","nonce":3216877560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6831","nonce":3216877560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6829","nonce":3216877560}]},"public_addr":"172.21.15.153:6825/3216877560","cluster_addr":"172.21.15.153:6827/3216877560","heartbeat_back_addr":"172.21.15.153:6831/3216877560","heartbeat_front_addr":"172.21.15.153:6829/3216877560","state":["exists","up"]},{"osd":11,"uuid":"17e0afaa-84eb-4b26-9810-bfa3cc7e0f3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6825","nonce":1857200740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6827","nonce":1857200740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6831","nonce":1857200740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6829","nonce":1857200740}]},"public_addr":"172.21.15.196:6825/1857200740","cluster_addr":"172.21.15.196:6827/1857200740","heartbeat_back_addr":"172.21.15.196:6831/1857200740","heartbeat_front_addr":"172.21.15.196:6829/1857200740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:55:15.595645+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:55:40.405487+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:04.938034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:29.718696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:55.636029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:57:21.257083+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:57:46.316552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:58:11.946863+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:58:37.098578+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:03.759023+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:28.659049+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:54.350475+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:0/3010783707":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6801/3562882246":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6800/3562882246":"2024-09-20T08:53:51.528772+0000","172.21.15.120:0/3224512035":"2024-09-20T08:53:51.528772+0000","172.21.15.120:0/3465105835":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/3317449610":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/2800697451":"2024-09-20T08:53:28.144376+0000","172.21.15.120:6801/2938446665":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/109127432":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6800/2938446665":"2024-09-20T08:53:28.144376+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:02:42.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.362+0000 7f7d29add640 1 -- 172.21.15.120:0/3268788602 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7d0c0a3250 msgr2=0x7f7d0c0a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:42.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.362+0000 7f7d29add640 1 --2- 172.21.15.120:0/3268788602 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7d0c0a3250 0x7f7d0c0a5710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7d241993b0 tx=0x7f7d18005d30 comp rx=0 tx=0).stop 2024-09-19T09:02:42.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.362+0000 7f7d29add640 1 -- 172.21.15.120:0/3268788602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d241025f0 msgr2=0x7f7d24197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:42.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.362+0000 7f7d29add640 1 --2- 172.21.15.120:0/3268788602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d241025f0 0x7f7d24197ea0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7d14009230 tx=0x7f7d14004360 comp rx=0 tx=0).stop 2024-09-19T09:02:42.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.362+0000 7f7d29add640 1 -- 172.21.15.120:0/3268788602 shutdown_connections 2024-09-19T09:02:42.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.362+0000 7f7d29add640 1 --2- 172.21.15.120:0/3268788602 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7d0c0a3250 0x7f7d0c0a5710 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.362+0000 7f7d29add640 1 --2- 172.21.15.120:0/3268788602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d24107850 0x7f7d2419f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.362+0000 7f7d29add640 1 --2- 172.21.15.120:0/3268788602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d24104f20 0x7f7d241983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.362+0000 7f7d29add640 1 --2- 172.21.15.120:0/3268788602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d241025f0 0x7f7d24197ea0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.362+0000 7f7d29add640 1 -- 172.21.15.120:0/3268788602 >> 172.21.15.120:0/3268788602 conn(0x7f7d240fc170 msgr2=0x7f7d24102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:42.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.362+0000 7f7d29add640 1 -- 172.21.15.120:0/3268788602 shutdown_connections 2024-09-19T09:02:42.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.362+0000 7f7d29add640 1 -- 172.21.15.120:0/3268788602 wait complete. 2024-09-19T09:02:42.374 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T08:56:07.830047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T09:00:43.774339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T09:00:45.786262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T09:02:42.375 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T09:02:42.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.481+0000 7f70f8ae4640 1 -- 172.21.15.120:0/1390730270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70f4102da0 msgr2=0x7f70f4103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:42.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.481+0000 7f70f8ae4640 1 --2- 172.21.15.120:0/1390730270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70f4102da0 0x7f70f4103220 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f70e8009900 tx=0x7f70e802a410 comp rx=0 tx=0).stop 2024-09-19T09:02:42.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.481+0000 7f70f8ae4640 1 -- 172.21.15.120:0/1390730270 shutdown_connections 2024-09-19T09:02:42.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.481+0000 7f70f8ae4640 1 --2- 172.21.15.120:0/1390730270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70f4103760 0x7f70f4109ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.481+0000 7f70f8ae4640 1 --2- 172.21.15.120:0/1390730270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70f4102da0 0x7f70f4103220 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.481+0000 7f70f8ae4640 1 --2- 172.21.15.120:0/1390730270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70f40ffdd0 0x7f70f41001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.481+0000 7f70f8ae4640 1 -- 172.21.15.120:0/1390730270 >> 172.21.15.120:0/1390730270 conn(0x7f70f40fb910 msgr2=0x7f70f40fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:42.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.481+0000 7f70f8ae4640 1 -- 172.21.15.120:0/1390730270 shutdown_connections 2024-09-19T09:02:42.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.481+0000 7f70f8ae4640 1 -- 172.21.15.120:0/1390730270 wait complete. 2024-09-19T09:02:42.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.482+0000 7f70f8ae4640 1 Processor -- start 2024-09-19T09:02:42.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.482+0000 7f70f8ae4640 1 -- start start 2024-09-19T09:02:42.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.482+0000 7f70f8ae4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70f40ffdd0 0x7f70f4197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:42.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.482+0000 7f70f8ae4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70f4102da0 0x7f70f41983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:42.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.482+0000 7f70f8ae4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70f4103760 0x7f70f419f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:42.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.482+0000 7f70f8ae4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f70f410d800 con 0x7f70f4102da0 2024-09-19T09:02:42.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.482+0000 7f70f8ae4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f70f410d680 con 0x7f70f4103760 2024-09-19T09:02:42.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.482+0000 7f70f8ae4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f70f410d980 con 0x7f70f40ffdd0 2024-09-19T09:02:42.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.482+0000 7f70f1d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70f4102da0 0x7f70f41983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:42.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.482+0000 7f70f1d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70f4102da0 0x7f70f41983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48376/0 (socket says 172.21.15.120:48376) 2024-09-19T09:02:42.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.482+0000 7f70f1d9b640 1 -- 172.21.15.120:0/1037061916 learned_addr learned my addr 172.21.15.120:0/1037061916 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:42.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.482+0000 7f70f259c640 1 --2- 172.21.15.120:0/1037061916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70f40ffdd0 0x7f70f4197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:42.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.482+0000 7f70f1d9b640 1 -- 172.21.15.120:0/1037061916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70f40ffdd0 msgr2=0x7f70f4197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:42.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.482+0000 7f70f1d9b640 1 --2- 172.21.15.120:0/1037061916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70f40ffdd0 0x7f70f4197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.482+0000 7f70f1d9b640 1 -- 172.21.15.120:0/1037061916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70f4103760 msgr2=0x7f70f419f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:42.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.482+0000 7f70f1d9b640 1 --2- 172.21.15.120:0/1037061916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70f4103760 0x7f70f419f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.482+0000 7f70f1d9b640 1 -- 172.21.15.120:0/1037061916 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f419fad0 con 0x7f70f4102da0 2024-09-19T09:02:42.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.482+0000 7f70f259c640 1 --2- 172.21.15.120:0/1037061916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70f40ffdd0 0x7f70f4197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:02:42.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.483+0000 7f70f1d9b640 1 --2- 172.21.15.120:0/1037061916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70f4102da0 0x7f70f41983f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f70e8009230 tx=0x7f70e8004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:42.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.483+0000 7f70e37fe640 1 -- 172.21.15.120:0/1037061916 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e803f030 con 0x7f70f4102da0 2024-09-19T09:02:42.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.483+0000 7f70e37fe640 1 -- 172.21.15.120:0/1037061916 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70e8003ce0 con 0x7f70f4102da0 2024-09-19T09:02:42.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.483+0000 7f70f8ae4640 1 -- 172.21.15.120:0/1037061916 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70f419fd60 con 0x7f70f4102da0 2024-09-19T09:02:42.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.483+0000 7f70e37fe640 1 -- 172.21.15.120:0/1037061916 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e8035070 con 0x7f70f4102da0 2024-09-19T09:02:42.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.483+0000 7f70f8ae4640 1 -- 172.21.15.120:0/1037061916 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70f41a0270 con 0x7f70f4102da0 2024-09-19T09:02:42.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.484+0000 7f70f8ae4640 1 -- 172.21.15.120:0/1037061916 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70f4109010 con 0x7f70f4102da0 2024-09-19T09:02:42.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.485+0000 7f70e37fe640 1 -- 172.21.15.120:0/1037061916 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f70e8042030 con 0x7f70f4102da0 2024-09-19T09:02:42.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.485+0000 7f70e37fe640 1 --2- 172.21.15.120:0/1037061916 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f70dc0a3250 0x7f70dc0a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:42.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.486+0000 7f70e37fe640 1 -- 172.21.15.120:0/1037061916 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f70e80f0020 con 0x7f70f4102da0 2024-09-19T09:02:42.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.486+0000 7f70f259c640 1 --2- 172.21.15.120:0/1037061916 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f70dc0a3250 0x7f70dc0a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:42.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.486+0000 7f70f259c640 1 --2- 172.21.15.120:0/1037061916 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f70dc0a3250 0x7f70dc0a5710 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f70e40058c0 tx=0x7f70e4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:42.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.489+0000 7f70e37fe640 1 -- 172.21.15.120:0/1037061916 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f70e80bcd40 con 0x7f70f4102da0 2024-09-19T09:02:42.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.658+0000 7f70f8ae4640 1 -- 172.21.15.120:0/1037061916 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f70f4062ef0 con 0x7f70f4102da0 2024-09-19T09:02:42.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.659+0000 7f70e37fe640 1 -- 172.21.15.120:0/1037061916 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f70e80b8070 con 0x7f70f4102da0 2024-09-19T09:02:42.659 INFO:teuthology.orchestra.run.smithi120.stdout:pg_num: 1 2024-09-19T09:02:42.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.661+0000 7f70f8ae4640 1 -- 172.21.15.120:0/1037061916 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f70dc0a3250 msgr2=0x7f70dc0a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:42.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.661+0000 7f70f8ae4640 1 --2- 172.21.15.120:0/1037061916 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f70dc0a3250 0x7f70dc0a5710 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f70e40058c0 tx=0x7f70e4005850 comp rx=0 tx=0).stop 2024-09-19T09:02:42.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.661+0000 7f70f8ae4640 1 -- 172.21.15.120:0/1037061916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70f4102da0 msgr2=0x7f70f41983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:42.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.661+0000 7f70f8ae4640 1 --2- 172.21.15.120:0/1037061916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70f4102da0 0x7f70f41983f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f70e8009230 tx=0x7f70e8004360 comp rx=0 tx=0).stop 2024-09-19T09:02:42.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.661+0000 7f70f8ae4640 1 -- 172.21.15.120:0/1037061916 shutdown_connections 2024-09-19T09:02:42.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.661+0000 7f70f8ae4640 1 --2- 172.21.15.120:0/1037061916 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f70dc0a3250 0x7f70dc0a5710 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.661+0000 7f70f8ae4640 1 --2- 172.21.15.120:0/1037061916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70f4103760 0x7f70f419f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.661+0000 7f70f8ae4640 1 --2- 172.21.15.120:0/1037061916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70f4102da0 0x7f70f41983f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.661+0000 7f70f8ae4640 1 --2- 172.21.15.120:0/1037061916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70f40ffdd0 0x7f70f4197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.661+0000 7f70f8ae4640 1 -- 172.21.15.120:0/1037061916 >> 172.21.15.120:0/1037061916 conn(0x7f70f40fb910 msgr2=0x7f70f41061e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:42.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.661+0000 7f70f8ae4640 1 -- 172.21.15.120:0/1037061916 shutdown_connections 2024-09-19T09:02:42.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.661+0000 7f70f8ae4640 1 -- 172.21.15.120:0/1037061916 wait complete. 2024-09-19T09:02:42.673 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T09:02:42.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.777+0000 7fb89694b640 1 -- 172.21.15.120:0/2582366906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb890101b20 msgr2=0x7fb890101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:42.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.777+0000 7fb89694b640 1 --2- 172.21.15.120:0/2582366906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb890101b20 0x7fb890101f20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb880009900 tx=0x7fb88002a410 comp rx=0 tx=0).stop 2024-09-19T09:02:42.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.777+0000 7fb89694b640 1 -- 172.21.15.120:0/2582366906 shutdown_connections 2024-09-19T09:02:42.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.777+0000 7fb89694b640 1 --2- 172.21.15.120:0/2582366906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8901036d0 0x7fb890109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.777+0000 7fb89694b640 1 --2- 172.21.15.120:0/2582366906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb890102d10 0x7fb890103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.777+0000 7fb89694b640 1 --2- 172.21.15.120:0/2582366906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb890101b20 0x7fb890101f20 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.777+0000 7fb89694b640 1 -- 172.21.15.120:0/2582366906 >> 172.21.15.120:0/2582366906 conn(0x7fb8900fd2a0 msgr2=0x7fb8900ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:42.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.777+0000 7fb89694b640 1 -- 172.21.15.120:0/2582366906 shutdown_connections 2024-09-19T09:02:42.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.777+0000 7fb89694b640 1 -- 172.21.15.120:0/2582366906 wait complete. 2024-09-19T09:02:42.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.777+0000 7fb89694b640 1 Processor -- start 2024-09-19T09:02:42.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.777+0000 7fb89694b640 1 -- start start 2024-09-19T09:02:42.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.778+0000 7fb89694b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb890101b20 0x7fb890197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:42.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.778+0000 7fb89694b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb890102d10 0x7fb890198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:42.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.778+0000 7fb89694b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8901036d0 0x7fb89019f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:42.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.778+0000 7fb89694b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb89010d800 con 0x7fb890101b20 2024-09-19T09:02:42.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.778+0000 7fb89694b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb89010d680 con 0x7fb8901036d0 2024-09-19T09:02:42.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.778+0000 7fb89694b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb89010d980 con 0x7fb890102d10 2024-09-19T09:02:42.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.778+0000 7fb88ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb890101b20 0x7fb890197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:42.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.778+0000 7fb88ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb890101b20 0x7fb890197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48394/0 (socket says 172.21.15.120:48394) 2024-09-19T09:02:42.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.778+0000 7fb88ffff640 1 -- 172.21.15.120:0/1026213428 learned_addr learned my addr 172.21.15.120:0/1026213428 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:42.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.778+0000 7fb894ee8640 1 --2- 172.21.15.120:0/1026213428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8901036d0 0x7fb89019f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:42.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.778+0000 7fb88f7fe640 1 --2- 172.21.15.120:0/1026213428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb890102d10 0x7fb890198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:42.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.778+0000 7fb88ffff640 1 -- 172.21.15.120:0/1026213428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb890102d10 msgr2=0x7fb890198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:42.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.778+0000 7fb88ffff640 1 --2- 172.21.15.120:0/1026213428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb890102d10 0x7fb890198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.778+0000 7fb88ffff640 1 -- 172.21.15.120:0/1026213428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8901036d0 msgr2=0x7fb89019f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:42.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.778+0000 7fb88ffff640 1 --2- 172.21.15.120:0/1026213428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8901036d0 0x7fb89019f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.778+0000 7fb88ffff640 1 -- 172.21.15.120:0/1026213428 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb89019fb10 con 0x7fb890101b20 2024-09-19T09:02:42.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.778+0000 7fb894ee8640 1 --2- 172.21.15.120:0/1026213428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8901036d0 0x7fb89019f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:02:42.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.778+0000 7fb88ffff640 1 --2- 172.21.15.120:0/1026213428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb890101b20 0x7fb890197ef0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb880031610 tx=0x7fb8800043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:42.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.778+0000 7fb88d7fa640 1 -- 172.21.15.120:0/1026213428 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88003f030 con 0x7fb890101b20 2024-09-19T09:02:42.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.778+0000 7fb88d7fa640 1 -- 172.21.15.120:0/1026213428 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb880003ce0 con 0x7fb890101b20 2024-09-19T09:02:42.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.778+0000 7fb88d7fa640 1 -- 172.21.15.120:0/1026213428 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb880035070 con 0x7fb890101b20 2024-09-19T09:02:42.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.778+0000 7fb89694b640 1 -- 172.21.15.120:0/1026213428 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb89019fda0 con 0x7fb890101b20 2024-09-19T09:02:42.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.778+0000 7fb89694b640 1 -- 172.21.15.120:0/1026213428 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8901a0250 con 0x7fb890101b20 2024-09-19T09:02:42.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.779+0000 7fb89694b640 1 -- 172.21.15.120:0/1026213428 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb890108f60 con 0x7fb890101b20 2024-09-19T09:02:42.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.781+0000 7fb88d7fa640 1 -- 172.21.15.120:0/1026213428 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fb880042030 con 0x7fb890101b20 2024-09-19T09:02:42.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.781+0000 7fb88d7fa640 1 --2- 172.21.15.120:0/1026213428 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb8780a3250 0x7fb8780a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:42.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.781+0000 7fb88f7fe640 1 --2- 172.21.15.120:0/1026213428 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb8780a3250 0x7fb8780a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:42.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.781+0000 7fb88d7fa640 1 -- 172.21.15.120:0/1026213428 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb8800f0210 con 0x7fb890101b20 2024-09-19T09:02:42.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.782+0000 7fb88f7fe640 1 --2- 172.21.15.120:0/1026213428 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb8780a3250 0x7fb8780a5710 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb890199400 tx=0x7fb884008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:42.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.784+0000 7fb88d7fa640 1 -- 172.21.15.120:0/1026213428 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb8800bcf30 con 0x7fb890101b20 2024-09-19T09:02:42.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.952+0000 7fb89694b640 1 -- 172.21.15.120:0/1026213428 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb890062e90 con 0x7fb890101b20 2024-09-19T09:02:42.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.952+0000 7fb88d7fa640 1 -- 172.21.15.120:0/1026213428 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) ==== 104+0+11 (secure 0 0 0) 0x7fb8800bc830 con 0x7fb890101b20 2024-09-19T09:02:42.953 INFO:teuthology.orchestra.run.smithi120.stdout:pg_num: 32 2024-09-19T09:02:42.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.954+0000 7fb89694b640 1 -- 172.21.15.120:0/1026213428 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb8780a3250 msgr2=0x7fb8780a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:42.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.954+0000 7fb89694b640 1 --2- 172.21.15.120:0/1026213428 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb8780a3250 0x7fb8780a5710 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb890199400 tx=0x7fb884008040 comp rx=0 tx=0).stop 2024-09-19T09:02:42.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.955+0000 7fb89694b640 1 -- 172.21.15.120:0/1026213428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb890101b20 msgr2=0x7fb890197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:42.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.955+0000 7fb89694b640 1 --2- 172.21.15.120:0/1026213428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb890101b20 0x7fb890197ef0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb880031610 tx=0x7fb8800043b0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.955+0000 7fb89694b640 1 -- 172.21.15.120:0/1026213428 shutdown_connections 2024-09-19T09:02:42.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.955+0000 7fb89694b640 1 --2- 172.21.15.120:0/1026213428 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb8780a3250 0x7fb8780a5710 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.955+0000 7fb89694b640 1 --2- 172.21.15.120:0/1026213428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8901036d0 0x7fb89019f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.955+0000 7fb89694b640 1 --2- 172.21.15.120:0/1026213428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb890102d10 0x7fb890198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.955+0000 7fb89694b640 1 --2- 172.21.15.120:0/1026213428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb890101b20 0x7fb890197ef0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:42.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.955+0000 7fb89694b640 1 -- 172.21.15.120:0/1026213428 >> 172.21.15.120:0/1026213428 conn(0x7fb8900fd2a0 msgr2=0x7fb890106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:42.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.955+0000 7fb89694b640 1 -- 172.21.15.120:0/1026213428 shutdown_connections 2024-09-19T09:02:42.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:42.955+0000 7fb89694b640 1 -- 172.21.15.120:0/1026213428 wait complete. 2024-09-19T09:02:42.965 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:42 smithi120 ceph-mon[31150]: pgmap v336: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T09:02:42.965 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:42 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2362455028' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-19T09:02:42.965 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:42 smithi120 ceph-mon[31150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:02:42.965 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:42 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3268788602' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:02:42.965 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:42 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1037061916' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T09:02:42.967 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T09:02:43.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.070+0000 7f1723453640 1 -- 172.21.15.120:0/2877159177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f171c1025e0 msgr2=0x7f171c108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:43.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.070+0000 7f1723453640 1 --2- 172.21.15.120:0/2877159177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f171c1025e0 0x7f171c108e70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1714009ca0 tx=0x7f171402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:02:43.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.070+0000 7f1723453640 1 -- 172.21.15.120:0/2877159177 shutdown_connections 2024-09-19T09:02:43.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.070+0000 7f1723453640 1 --2- 172.21.15.120:0/2877159177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f171c1025e0 0x7f171c108e70 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:43.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.070+0000 7f1723453640 1 --2- 172.21.15.120:0/2877159177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f171c101c20 0x7f171c1020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:43.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.070+0000 7f1723453640 1 --2- 172.21.15.120:0/2877159177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f171c100a30 0x7f171c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:43.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.070+0000 7f1723453640 1 -- 172.21.15.120:0/2877159177 >> 172.21.15.120:0/2877159177 conn(0x7f171c0fc1b0 msgr2=0x7f171c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:43.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.070+0000 7f1723453640 1 -- 172.21.15.120:0/2877159177 shutdown_connections 2024-09-19T09:02:43.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.070+0000 7f1723453640 1 -- 172.21.15.120:0/2877159177 wait complete. 2024-09-19T09:02:43.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.071+0000 7f1723453640 1 Processor -- start 2024-09-19T09:02:43.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.071+0000 7f1723453640 1 -- start start 2024-09-19T09:02:43.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.071+0000 7f1723453640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f171c100a30 0x7f171c196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:43.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.071+0000 7f1723453640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f171c101c20 0x7f171c197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:43.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.071+0000 7f1723453640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f171c1025e0 0x7f171c19e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:43.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.071+0000 7f1723453640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f171c10c750 con 0x7f171c1025e0 2024-09-19T09:02:43.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.071+0000 7f1723453640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f171c10c5d0 con 0x7f171c100a30 2024-09-19T09:02:43.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.071+0000 7f1723453640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f171c10c8d0 con 0x7f171c101c20 2024-09-19T09:02:43.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.071+0000 7f17219f0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f171c1025e0 0x7f171c19e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:43.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.071+0000 7f17219f0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f171c1025e0 0x7f171c19e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48424/0 (socket says 172.21.15.120:48424) 2024-09-19T09:02:43.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.071+0000 7f17219f0640 1 -- 172.21.15.120:0/1453128460 learned_addr learned my addr 172.21.15.120:0/1453128460 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:43.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.071+0000 7f17211ef640 1 --2- 172.21.15.120:0/1453128460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f171c100a30 0x7f171c196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:43.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.071+0000 7f17219f0640 1 -- 172.21.15.120:0/1453128460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f171c101c20 msgr2=0x7f171c197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:43.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.071+0000 7f17219f0640 1 --2- 172.21.15.120:0/1453128460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f171c101c20 0x7f171c197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:43.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.071+0000 7f17219f0640 1 -- 172.21.15.120:0/1453128460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f171c100a30 msgr2=0x7f171c196e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:43.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.071+0000 7f17219f0640 1 --2- 172.21.15.120:0/1453128460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f171c100a30 0x7f171c196e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:43.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.071+0000 7f17219f0640 1 -- 172.21.15.120:0/1453128460 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f171c19ea50 con 0x7f171c1025e0 2024-09-19T09:02:43.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.072+0000 7f17219f0640 1 --2- 172.21.15.120:0/1453128460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f171c1025e0 0x7f171c19e3e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1714031ee0 tx=0x7f1714031f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:43.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.072+0000 7f17127fc640 1 -- 172.21.15.120:0/1453128460 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1714049070 con 0x7f171c1025e0 2024-09-19T09:02:43.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.072+0000 7f17127fc640 1 -- 172.21.15.120:0/1453128460 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1714003b80 con 0x7f171c1025e0 2024-09-19T09:02:43.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.072+0000 7f1723453640 1 -- 172.21.15.120:0/1453128460 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f171c06a360 con 0x7f171c1025e0 2024-09-19T09:02:43.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.072+0000 7f17127fc640 1 -- 172.21.15.120:0/1453128460 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171402f9d0 con 0x7f171c1025e0 2024-09-19T09:02:43.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.072+0000 7f1723453640 1 -- 172.21.15.120:0/1453128460 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f171c06a980 con 0x7f171c1025e0 2024-09-19T09:02:43.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.073+0000 7f1723453640 1 -- 172.21.15.120:0/1453128460 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f171c1078d0 con 0x7f171c1025e0 2024-09-19T09:02:43.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.074+0000 7f17127fc640 1 -- 172.21.15.120:0/1453128460 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f171404c070 con 0x7f171c1025e0 2024-09-19T09:02:43.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.075+0000 7f17127fc640 1 --2- 172.21.15.120:0/1453128460 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f17080a32d0 0x7f17080a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:43.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.075+0000 7f17211ef640 1 --2- 172.21.15.120:0/1453128460 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f17080a32d0 0x7f17080a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:43.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.075+0000 7f17127fc640 1 -- 172.21.15.120:0/1453128460 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f17140f1fd0 con 0x7f171c1025e0 2024-09-19T09:02:43.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.075+0000 7f17211ef640 1 --2- 172.21.15.120:0/1453128460 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f17080a32d0 0x7f17080a5790 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f171c101a80 tx=0x7f1718009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:43.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.079+0000 7f17127fc640 1 -- 172.21.15.120:0/1453128460 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1714081210 con 0x7f171c1025e0 2024-09-19T09:02:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:42 smithi196 ceph-mon[35150]: pgmap v336: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T09:02:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:42 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2362455028' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-19T09:02:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:42 smithi196 ceph-mon[35150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:02:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:42 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3268788602' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:02:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:42 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1037061916' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T09:02:43.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:42 smithi153 ceph-mon[35071]: pgmap v336: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T09:02:43.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:42 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2362455028' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-19T09:02:43.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:42 smithi153 ceph-mon[35071]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:02:43.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:42 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3268788602' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:02:43.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:42 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1037061916' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T09:02:43.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.245+0000 7f1723453640 1 -- 172.21.15.120:0/1453128460 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f171c19ec80 con 0x7f171c1025e0 2024-09-19T09:02:43.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.246+0000 7f17127fc640 1 -- 172.21.15.120:0/1453128460 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) ==== 100+0+11 (secure 0 0 0) 0x7f17140becf0 con 0x7f171c1025e0 2024-09-19T09:02:43.246 INFO:teuthology.orchestra.run.smithi120.stdout:pg_num: 64 2024-09-19T09:02:43.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.248+0000 7f1723453640 1 -- 172.21.15.120:0/1453128460 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f17080a32d0 msgr2=0x7f17080a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:43.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.248+0000 7f1723453640 1 --2- 172.21.15.120:0/1453128460 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f17080a32d0 0x7f17080a5790 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f171c101a80 tx=0x7f1718009110 comp rx=0 tx=0).stop 2024-09-19T09:02:43.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.248+0000 7f1723453640 1 -- 172.21.15.120:0/1453128460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f171c1025e0 msgr2=0x7f171c19e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:43.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.248+0000 7f1723453640 1 --2- 172.21.15.120:0/1453128460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f171c1025e0 0x7f171c19e3e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1714031ee0 tx=0x7f1714031f10 comp rx=0 tx=0).stop 2024-09-19T09:02:43.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.248+0000 7f1723453640 1 -- 172.21.15.120:0/1453128460 shutdown_connections 2024-09-19T09:02:43.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.248+0000 7f1723453640 1 --2- 172.21.15.120:0/1453128460 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f17080a32d0 0x7f17080a5790 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:43.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.248+0000 7f1723453640 1 --2- 172.21.15.120:0/1453128460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f171c1025e0 0x7f171c19e3e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:43.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.248+0000 7f1723453640 1 --2- 172.21.15.120:0/1453128460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f171c101c20 0x7f171c197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:43.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.248+0000 7f1723453640 1 --2- 172.21.15.120:0/1453128460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f171c100a30 0x7f171c196e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:43.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.248+0000 7f1723453640 1 -- 172.21.15.120:0/1453128460 >> 172.21.15.120:0/1453128460 conn(0x7f171c0fc1b0 msgr2=0x7f171c105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:43.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.248+0000 7f1723453640 1 -- 172.21.15.120:0/1453128460 shutdown_connections 2024-09-19T09:02:43.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:43.248+0000 7f1723453640 1 -- 172.21.15.120:0/1453128460 wait complete. 2024-09-19T09:02:43.259 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-19T09:02:43.260 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:02:43.507 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:43.966 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:43 smithi120 ceph-mon[31150]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:02:43.966 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:43 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1026213428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T09:02:43.966 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:43 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1453128460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T09:02:44.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.046+0000 7f166e0c6640 1 -- 172.21.15.120:0/728755736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1668106a80 msgr2=0x7f1668106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:44.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.046+0000 7f166e0c6640 1 --2- 172.21.15.120:0/728755736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1668106a80 0x7f1668106e60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1658009930 tx=0x7f165802a420 comp rx=0 tx=0).stop 2024-09-19T09:02:44.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.046+0000 7f166e0c6640 1 -- 172.21.15.120:0/728755736 shutdown_connections 2024-09-19T09:02:44.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.046+0000 7f166e0c6640 1 --2- 172.21.15.120:0/728755736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1668101420 0x7f166810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:44.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.046+0000 7f166e0c6640 1 --2- 172.21.15.120:0/728755736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1668100a80 0x7f1668100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:44.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.046+0000 7f166e0c6640 1 --2- 172.21.15.120:0/728755736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1668106a80 0x7f1668106e60 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:44.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.046+0000 7f166e0c6640 1 -- 172.21.15.120:0/728755736 >> 172.21.15.120:0/728755736 conn(0x7f16680fc7a0 msgr2=0x7f16680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:44.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.046+0000 7f166e0c6640 1 -- 172.21.15.120:0/728755736 shutdown_connections 2024-09-19T09:02:44.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.046+0000 7f166e0c6640 1 -- 172.21.15.120:0/728755736 wait complete. 2024-09-19T09:02:44.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.047+0000 7f166e0c6640 1 Processor -- start 2024-09-19T09:02:44.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.047+0000 7f166e0c6640 1 -- start start 2024-09-19T09:02:44.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.048+0000 7f166e0c6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1668100a80 0x7f1668197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:44.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.048+0000 7f166e0c6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1668101420 0x7f1668197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:44.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.048+0000 7f166e0c6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1668106a80 0x7f166819bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:44.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.048+0000 7f166e0c6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1668110e80 con 0x7f1668100a80 2024-09-19T09:02:44.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.048+0000 7f166e0c6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1668110d00 con 0x7f1668101420 2024-09-19T09:02:44.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.048+0000 7f166e0c6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1668111000 con 0x7f1668106a80 2024-09-19T09:02:44.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.048+0000 7f16677fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1668100a80 0x7f1668197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:44.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.048+0000 7f16677fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1668100a80 0x7f1668197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48444/0 (socket says 172.21.15.120:48444) 2024-09-19T09:02:44.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.048+0000 7f16677fe640 1 -- 172.21.15.120:0/4179332293 learned_addr learned my addr 172.21.15.120:0/4179332293 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:44.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.048+0000 7f1666ffd640 1 --2- 172.21.15.120:0/4179332293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1668101420 0x7f1668197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:44.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.048+0000 7f1667fff640 1 --2- 172.21.15.120:0/4179332293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1668106a80 0x7f166819bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:44.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.048+0000 7f16677fe640 1 -- 172.21.15.120:0/4179332293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1668106a80 msgr2=0x7f166819bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:44.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.048+0000 7f16677fe640 1 --2- 172.21.15.120:0/4179332293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1668106a80 0x7f166819bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:44.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.048+0000 7f16677fe640 1 -- 172.21.15.120:0/4179332293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1668101420 msgr2=0x7f1668197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:44.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.048+0000 7f16677fe640 1 --2- 172.21.15.120:0/4179332293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1668101420 0x7f1668197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:44.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.048+0000 7f16677fe640 1 -- 172.21.15.120:0/4179332293 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166819c2b0 con 0x7f1668100a80 2024-09-19T09:02:44.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.048+0000 7f16677fe640 1 --2- 172.21.15.120:0/4179332293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1668100a80 0x7f1668197300 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1658009a50 tx=0x7f1658004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:44.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.048+0000 7f1664ff9640 1 -- 172.21.15.120:0/4179332293 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165803f030 con 0x7f1668100a80 2024-09-19T09:02:44.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.049+0000 7f1664ff9640 1 -- 172.21.15.120:0/4179332293 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1658003ce0 con 0x7f1668100a80 2024-09-19T09:02:44.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.049+0000 7f1664ff9640 1 -- 172.21.15.120:0/4179332293 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1658035070 con 0x7f1668100a80 2024-09-19T09:02:44.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.049+0000 7f166e0c6640 1 -- 172.21.15.120:0/4179332293 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166819c540 con 0x7f1668100a80 2024-09-19T09:02:44.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.049+0000 7f166e0c6640 1 -- 172.21.15.120:0/4179332293 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16681a3e80 con 0x7f1668100a80 2024-09-19T09:02:44.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.050+0000 7f166e0c6640 1 -- 172.21.15.120:0/4179332293 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16681021c0 con 0x7f1668100a80 2024-09-19T09:02:44.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.052+0000 7f1664ff9640 1 -- 172.21.15.120:0/4179332293 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f1658042030 con 0x7f1668100a80 2024-09-19T09:02:44.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.052+0000 7f1664ff9640 1 --2- 172.21.15.120:0/4179332293 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16500a32c0 0x7f16500a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:44.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.052+0000 7f1666ffd640 1 --2- 172.21.15.120:0/4179332293 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16500a32c0 0x7f16500a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:44.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.052+0000 7f1664ff9640 1 -- 172.21.15.120:0/4179332293 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f16580f0270 con 0x7f1668100a80 2024-09-19T09:02:44.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.053+0000 7f1666ffd640 1 --2- 172.21.15.120:0/4179332293 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16500a32c0 0x7f16500a5780 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f16680fed70 tx=0x7f165c00a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:44.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.055+0000 7f1664ff9640 1 -- 172.21.15.120:0/4179332293 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f16580bcf90 con 0x7f1668100a80 2024-09-19T09:02:44.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:43 smithi196 ceph-mon[35150]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:02:44.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:43 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1026213428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T09:02:44.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:43 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1453128460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T09:02:44.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:43 smithi153 ceph-mon[35071]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:02:44.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:43 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1026213428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T09:02:44.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:43 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1453128460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T09:02:44.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.259+0000 7f166e0c6640 1 -- 172.21.15.120:0/4179332293 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1668100ee0 con 0x7f1668100a80 2024-09-19T09:02:44.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.259+0000 7f1664ff9640 1 -- 172.21.15.120:0/4179332293 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f1668100ee0 con 0x7f1668100a80 2024-09-19T09:02:44.260 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:02:44.260 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":28,"btime":"2024-09-19T09:02:41:696728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:02:41.207357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:02:44.260 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 28 2024-09-19T09:02:44.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.262+0000 7f166e0c6640 1 -- 172.21.15.120:0/4179332293 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16500a32c0 msgr2=0x7f16500a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:44.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.262+0000 7f166e0c6640 1 --2- 172.21.15.120:0/4179332293 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16500a32c0 0x7f16500a5780 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f16680fed70 tx=0x7f165c00a250 comp rx=0 tx=0).stop 2024-09-19T09:02:44.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.262+0000 7f166e0c6640 1 -- 172.21.15.120:0/4179332293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1668100a80 msgr2=0x7f1668197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:44.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.262+0000 7f166e0c6640 1 --2- 172.21.15.120:0/4179332293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1668100a80 0x7f1668197300 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1658009a50 tx=0x7f1658004270 comp rx=0 tx=0).stop 2024-09-19T09:02:44.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.262+0000 7f166e0c6640 1 -- 172.21.15.120:0/4179332293 shutdown_connections 2024-09-19T09:02:44.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.262+0000 7f166e0c6640 1 --2- 172.21.15.120:0/4179332293 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16500a32c0 0x7f16500a5780 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:44.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.262+0000 7f166e0c6640 1 --2- 172.21.15.120:0/4179332293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1668106a80 0x7f166819bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:44.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.262+0000 7f166e0c6640 1 --2- 172.21.15.120:0/4179332293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1668101420 0x7f1668197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:44.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.262+0000 7f166e0c6640 1 --2- 172.21.15.120:0/4179332293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1668100a80 0x7f1668197300 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:44.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.262+0000 7f166e0c6640 1 -- 172.21.15.120:0/4179332293 >> 172.21.15.120:0/4179332293 conn(0x7f16680fc7a0 msgr2=0x7f16680fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:44.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.263+0000 7f166e0c6640 1 -- 172.21.15.120:0/4179332293 shutdown_connections 2024-09-19T09:02:44.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:44.263+0000 7f166e0c6640 1 -- 172.21.15.120:0/4179332293 wait complete. 2024-09-19T09:02:45.048 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:44 smithi120 ceph-mon[31150]: pgmap v337: 97 pgs: 97 active+clean; 646 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T09:02:45.048 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:44 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/4179332293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:02:45.076 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-19T09:02:45.076 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-19T09:02:45.077 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T09:02:45.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.178+0000 7f002e247640 1 -- 172.21.15.120:0/1051859808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0028101c20 msgr2=0x7f00281020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:45.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.178+0000 7f002e247640 1 --2- 172.21.15.120:0/1051859808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0028101c20 0x7f00281020a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f001c009900 tx=0x7f001c02a410 comp rx=0 tx=0).stop 2024-09-19T09:02:45.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.179+0000 7f002e247640 1 -- 172.21.15.120:0/1051859808 shutdown_connections 2024-09-19T09:02:45.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.179+0000 7f002e247640 1 --2- 172.21.15.120:0/1051859808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00281025e0 0x7f0028108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:45.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.179+0000 7f002e247640 1 --2- 172.21.15.120:0/1051859808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0028101c20 0x7f00281020a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:45.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.179+0000 7f002e247640 1 --2- 172.21.15.120:0/1051859808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0028100a30 0x7f0028100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:45.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.179+0000 7f002e247640 1 -- 172.21.15.120:0/1051859808 >> 172.21.15.120:0/1051859808 conn(0x7f00280fc190 msgr2=0x7f00280fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:45.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.179+0000 7f002e247640 1 -- 172.21.15.120:0/1051859808 shutdown_connections 2024-09-19T09:02:45.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.179+0000 7f002e247640 1 -- 172.21.15.120:0/1051859808 wait complete. 2024-09-19T09:02:45.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.179+0000 7f002e247640 1 Processor -- start 2024-09-19T09:02:45.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.179+0000 7f002e247640 1 -- start start 2024-09-19T09:02:45.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.179+0000 7f002e247640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0028100a30 0x7f002806af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:45.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.179+0000 7f002e247640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0028101c20 0x7f002819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:45.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.179+0000 7f002e247640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00281025e0 0x7f002819f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:45.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.179+0000 7f002e247640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f002810c690 con 0x7f0028100a30 2024-09-19T09:02:45.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.179+0000 7f002e247640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f002810c510 con 0x7f00281025e0 2024-09-19T09:02:45.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.179+0000 7f002e247640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f002810c810 con 0x7f0028101c20 2024-09-19T09:02:45.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.180+0000 7f00277fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0028100a30 0x7f002806af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:45.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.180+0000 7f00277fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0028100a30 0x7f002806af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48452/0 (socket says 172.21.15.120:48452) 2024-09-19T09:02:45.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.180+0000 7f00277fe640 1 -- 172.21.15.120:0/3375398837 learned_addr learned my addr 172.21.15.120:0/3375398837 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:45.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.180+0000 7f0026ffd640 1 --2- 172.21.15.120:0/3375398837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0028101c20 0x7f002819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:45.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.180+0000 7f00277fe640 1 -- 172.21.15.120:0/3375398837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0028101c20 msgr2=0x7f002819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:45.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.180+0000 7f00277fe640 1 --2- 172.21.15.120:0/3375398837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0028101c20 0x7f002819cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:45.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.180+0000 7f0027fff640 1 --2- 172.21.15.120:0/3375398837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00281025e0 0x7f002819f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:45.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.180+0000 7f00277fe640 1 -- 172.21.15.120:0/3375398837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00281025e0 msgr2=0x7f002819f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:45.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.180+0000 7f00277fe640 1 --2- 172.21.15.120:0/3375398837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00281025e0 0x7f002819f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:45.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.180+0000 7f00277fe640 1 -- 172.21.15.120:0/3375398837 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f002819fb30 con 0x7f0028100a30 2024-09-19T09:02:45.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.180+0000 7f0027fff640 1 --2- 172.21.15.120:0/3375398837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00281025e0 0x7f002819f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:02:45.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.180+0000 7f00277fe640 1 --2- 172.21.15.120:0/3375398837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0028100a30 0x7f002806af30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f001800ebc0 tx=0x7f001800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:45.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.180+0000 7f0024ff9640 1 -- 172.21.15.120:0/3375398837 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0018003b70 con 0x7f0028100a30 2024-09-19T09:02:45.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.180+0000 7f0024ff9640 1 -- 172.21.15.120:0/3375398837 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0018003d10 con 0x7f0028100a30 2024-09-19T09:02:45.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.180+0000 7f002e247640 1 -- 172.21.15.120:0/3375398837 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f002819fdc0 con 0x7f0028100a30 2024-09-19T09:02:45.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.180+0000 7f0024ff9640 1 -- 172.21.15.120:0/3375398837 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0018019980 con 0x7f0028100a30 2024-09-19T09:02:45.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.180+0000 7f002e247640 1 -- 172.21.15.120:0/3375398837 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00281a0280 con 0x7f0028100a30 2024-09-19T09:02:45.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.181+0000 7f002e247640 1 -- 172.21.15.120:0/3375398837 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0028107e70 con 0x7f0028100a30 2024-09-19T09:02:45.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.182+0000 7f0024ff9640 1 -- 172.21.15.120:0/3375398837 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f0018019b20 con 0x7f0028100a30 2024-09-19T09:02:45.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.183+0000 7f0024ff9640 1 --2- 172.21.15.120:0/3375398837 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f00100a3360 0x7f00100a5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:45.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.183+0000 7f0026ffd640 1 --2- 172.21.15.120:0/3375398837 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f00100a3360 0x7f00100a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:45.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.183+0000 7f0024ff9640 1 -- 172.21.15.120:0/3375398837 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f00180d0390 con 0x7f0028100a30 2024-09-19T09:02:45.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.183+0000 7f0026ffd640 1 --2- 172.21.15.120:0/3375398837 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f00100a3360 0x7f00100a5820 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f002806aca0 tx=0x7f001c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:44 smithi153 ceph-mon[35071]: pgmap v337: 97 pgs: 97 active+clean; 646 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T09:02:45.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:44 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/4179332293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:02:45.188 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:44 smithi196 ceph-mon[35150]: pgmap v337: 97 pgs: 97 active+clean; 646 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T09:02:45.188 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:44 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/4179332293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:02:45.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.187+0000 7f0024ff9640 1 -- 172.21.15.120:0/3375398837 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f001809cf50 con 0x7f0028100a30 2024-09-19T09:02:45.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.353+0000 7f002e247640 1 -- 172.21.15.120:0/3375398837 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0028062630 con 0x7f00100a3360 2024-09-19T09:02:45.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.359+0000 7f0024ff9640 1 -- 172.21.15.120:0/3375398837 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259138 (secure 0 0 0) 0x7f0028062630 con 0x7f00100a3360 2024-09-19T09:02:45.363 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:02:45.364 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-19T09:02:45.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.362+0000 7f002e247640 1 -- 172.21.15.120:0/3375398837 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f00100a3360 msgr2=0x7f00100a5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:45.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.362+0000 7f002e247640 1 --2- 172.21.15.120:0/3375398837 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f00100a3360 0x7f00100a5820 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f002806aca0 tx=0x7f001c0315b0 comp rx=0 tx=0).stop 2024-09-19T09:02:45.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.362+0000 7f002e247640 1 -- 172.21.15.120:0/3375398837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0028100a30 msgr2=0x7f002806af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:45.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.362+0000 7f002e247640 1 --2- 172.21.15.120:0/3375398837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0028100a30 0x7f002806af30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f001800ebc0 tx=0x7f001800c6a0 comp rx=0 tx=0).stop 2024-09-19T09:02:45.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.362+0000 7f002e247640 1 -- 172.21.15.120:0/3375398837 shutdown_connections 2024-09-19T09:02:45.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.362+0000 7f002e247640 1 --2- 172.21.15.120:0/3375398837 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f00100a3360 0x7f00100a5820 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:45.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.362+0000 7f002e247640 1 --2- 172.21.15.120:0/3375398837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00281025e0 0x7f002819f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:45.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.362+0000 7f002e247640 1 --2- 172.21.15.120:0/3375398837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0028101c20 0x7f002819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:45.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.362+0000 7f002e247640 1 --2- 172.21.15.120:0/3375398837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0028100a30 0x7f002806af30 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:45.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.362+0000 7f002e247640 1 -- 172.21.15.120:0/3375398837 >> 172.21.15.120:0/3375398837 conn(0x7f00280fc190 msgr2=0x7f0028105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:45.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.362+0000 7f002e247640 1 -- 172.21.15.120:0/3375398837 shutdown_connections 2024-09-19T09:02:45.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.362+0000 7f002e247640 1 -- 172.21.15.120:0/3375398837 wait complete. 2024-09-19T09:02:45.373 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-09-19T09:02:43.635484+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661126,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1193,"num_read_kb":718,"num_write":444,"num_write_kb":3089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1182102,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364360,"kb_used_data":10952,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577464,"statfs":{"total":1151940427776,"available":1151567323136,"internally_reserved":0,"allocated":11214848,"data_stored":7114931,"data_compressed":181932,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1009,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":127,"num_read_kb":62,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003011"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624417+0000","last_change":"2024-09-19T09:00:46.966628+0000","last_active":"2024-09-19T09:02:21.624417+0000","last_peered":"2024-09-19T09:02:21.624417+0000","last_clean":"2024-09-19T09:02:21.624417+0000","last_became_active":"2024-09-19T09:00:46.966290+0000","last_became_peered":"2024-09-19T09:00:46.966290+0000","last_unstale":"2024-09-19T09:02:21.624417+0000","last_undegraded":"2024-09-19T09:02:21.624417+0000","last_fullsized":"2024-09-19T09:02:21.624417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:42.717139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621317+0000","last_change":"2024-09-19T09:00:46.963706+0000","last_active":"2024-09-19T09:02:21.621317+0000","last_peered":"2024-09-19T09:02:21.621317+0000","last_clean":"2024-09-19T09:02:21.621317+0000","last_became_active":"2024-09-19T09:00:46.963303+0000","last_became_peered":"2024-09-19T09:00:46.963303+0000","last_unstale":"2024-09-19T09:02:21.621317+0000","last_undegraded":"2024-09-19T09:02:21.621317+0000","last_fullsized":"2024-09-19T09:02:21.621317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:07.451500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620146+0000","last_change":"2024-09-19T09:00:46.967149+0000","last_active":"2024-09-19T09:02:21.620146+0000","last_peered":"2024-09-19T09:02:21.620146+0000","last_clean":"2024-09-19T09:02:21.620146+0000","last_became_active":"2024-09-19T09:00:46.966861+0000","last_became_peered":"2024-09-19T09:00:46.966861+0000","last_unstale":"2024-09-19T09:02:21.620146+0000","last_undegraded":"2024-09-19T09:02:21.620146+0000","last_fullsized":"2024-09-19T09:02:21.620146+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:28.396022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.603363+0000","last_change":"2024-09-19T09:00:46.969242+0000","last_active":"2024-09-19T09:02:21.603363+0000","last_peered":"2024-09-19T09:02:21.603363+0000","last_clean":"2024-09-19T09:02:21.603363+0000","last_became_active":"2024-09-19T09:00:46.968546+0000","last_became_peered":"2024-09-19T09:00:46.968546+0000","last_unstale":"2024-09-19T09:02:21.603363+0000","last_undegraded":"2024-09-19T09:02:21.603363+0000","last_fullsized":"2024-09-19T09:02:21.603363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:43:51.060130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.602962+0000","last_change":"2024-09-19T09:00:46.955501+0000","last_active":"2024-09-19T09:02:21.602962+0000","last_peered":"2024-09-19T09:02:21.602962+0000","last_clean":"2024-09-19T09:02:21.602962+0000","last_became_active":"2024-09-19T09:00:46.955161+0000","last_became_peered":"2024-09-19T09:00:46.955161+0000","last_unstale":"2024-09-19T09:02:21.602962+0000","last_undegraded":"2024-09-19T09:02:21.602962+0000","last_fullsized":"2024-09-19T09:02:21.602962+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:39:19.005931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624332+0000","last_change":"2024-09-19T09:00:46.962577+0000","last_active":"2024-09-19T09:02:21.624332+0000","last_peered":"2024-09-19T09:02:21.624332+0000","last_clean":"2024-09-19T09:02:21.624332+0000","last_became_active":"2024-09-19T09:00:46.962253+0000","last_became_peered":"2024-09-19T09:00:46.962253+0000","last_unstale":"2024-09-19T09:02:21.624332+0000","last_undegraded":"2024-09-19T09:02:21.624332+0000","last_fullsized":"2024-09-19T09:02:21.624332+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:27.925932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621157+0000","last_change":"2024-09-19T09:00:46.960004+0000","last_active":"2024-09-19T09:02:21.621157+0000","last_peered":"2024-09-19T09:02:21.621157+0000","last_clean":"2024-09-19T09:02:21.621157+0000","last_became_active":"2024-09-19T09:00:46.959664+0000","last_became_peered":"2024-09-19T09:00:46.959664+0000","last_unstale":"2024-09-19T09:02:21.621157+0000","last_undegraded":"2024-09-19T09:02:21.621157+0000","last_fullsized":"2024-09-19T09:02:21.621157+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:41.616309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622980+0000","last_change":"2024-09-19T09:00:46.952394+0000","last_active":"2024-09-19T09:02:21.622980+0000","last_peered":"2024-09-19T09:02:21.622980+0000","last_clean":"2024-09-19T09:02:21.622980+0000","last_became_active":"2024-09-19T09:00:46.952084+0000","last_became_peered":"2024-09-19T09:00:46.952084+0000","last_unstale":"2024-09-19T09:02:21.622980+0000","last_undegraded":"2024-09-19T09:02:21.622980+0000","last_fullsized":"2024-09-19T09:02:21.622980+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:21.698423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.632916+0000","last_change":"2024-09-19T09:00:46.961262+0000","last_active":"2024-09-19T09:02:21.632916+0000","last_peered":"2024-09-19T09:02:21.632916+0000","last_clean":"2024-09-19T09:02:21.632916+0000","last_became_active":"2024-09-19T09:00:46.960951+0000","last_became_peered":"2024-09-19T09:00:46.960951+0000","last_unstale":"2024-09-19T09:02:21.632916+0000","last_undegraded":"2024-09-19T09:02:21.632916+0000","last_fullsized":"2024-09-19T09:02:21.632916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:02.445108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.625661+0000","last_change":"2024-09-19T09:01:52.758459+0000","last_active":"2024-09-19T09:02:21.625661+0000","last_peered":"2024-09-19T09:02:21.625661+0000","last_clean":"2024-09-19T09:02:21.625661+0000","last_became_active":"2024-09-19T09:01:52.758073+0000","last_became_peered":"2024-09-19T09:01:52.758073+0000","last_unstale":"2024-09-19T09:02:21.625661+0000","last_undegraded":"2024-09-19T09:02:21.625661+0000","last_fullsized":"2024-09-19T09:02:21.625661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:46:32.983316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.634093+0000","last_change":"2024-09-19T09:00:46.963200+0000","last_active":"2024-09-19T09:02:21.634093+0000","last_peered":"2024-09-19T09:02:21.634093+0000","last_clean":"2024-09-19T09:02:21.634093+0000","last_became_active":"2024-09-19T09:00:46.960496+0000","last_became_peered":"2024-09-19T09:00:46.960496+0000","last_unstale":"2024-09-19T09:02:21.634093+0000","last_undegraded":"2024-09-19T09:02:21.634093+0000","last_fullsized":"2024-09-19T09:02:21.634093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:59:47.234492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.618494+0000","last_change":"2024-09-19T09:00:46.966309+0000","last_active":"2024-09-19T09:02:21.618494+0000","last_peered":"2024-09-19T09:02:21.618494+0000","last_clean":"2024-09-19T09:02:21.618494+0000","last_became_active":"2024-09-19T09:00:46.965970+0000","last_became_peered":"2024-09-19T09:00:46.965970+0000","last_unstale":"2024-09-19T09:02:21.618494+0000","last_undegraded":"2024-09-19T09:02:21.618494+0000","last_fullsized":"2024-09-19T09:02:21.618494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:38.353206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622484+0000","last_change":"2024-09-19T09:00:46.954079+0000","last_active":"2024-09-19T09:02:21.622484+0000","last_peered":"2024-09-19T09:02:21.622484+0000","last_clean":"2024-09-19T09:02:21.622484+0000","last_became_active":"2024-09-19T09:00:46.953398+0000","last_became_peered":"2024-09-19T09:00:46.953398+0000","last_unstale":"2024-09-19T09:02:21.622484+0000","last_undegraded":"2024-09-19T09:02:21.622484+0000","last_fullsized":"2024-09-19T09:02:21.622484+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:54:27.554800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621192+0000","last_change":"2024-09-19T09:00:46.965347+0000","last_active":"2024-09-19T09:02:21.621192+0000","last_peered":"2024-09-19T09:02:21.621192+0000","last_clean":"2024-09-19T09:02:21.621192+0000","last_became_active":"2024-09-19T09:00:46.965045+0000","last_became_peered":"2024-09-19T09:00:46.965045+0000","last_unstale":"2024-09-19T09:02:21.621192+0000","last_undegraded":"2024-09-19T09:02:21.621192+0000","last_fullsized":"2024-09-19T09:02:21.621192+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:27.316100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620943+0000","last_change":"2024-09-19T09:00:46.962898+0000","last_active":"2024-09-19T09:02:21.620943+0000","last_peered":"2024-09-19T09:02:21.620943+0000","last_clean":"2024-09-19T09:02:21.620943+0000","last_became_active":"2024-09-19T09:00:46.962412+0000","last_became_peered":"2024-09-19T09:00:46.962412+0000","last_unstale":"2024-09-19T09:02:21.620943+0000","last_undegraded":"2024-09-19T09:02:21.620943+0000","last_fullsized":"2024-09-19T09:02:21.620943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:40.483678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.627060+0000","last_change":"2024-09-19T09:00:46.958147+0000","last_active":"2024-09-19T09:02:21.627060+0000","last_peered":"2024-09-19T09:02:21.627060+0000","last_clean":"2024-09-19T09:02:21.627060+0000","last_became_active":"2024-09-19T09:00:46.957805+0000","last_became_peered":"2024-09-19T09:00:46.957805+0000","last_unstale":"2024-09-19T09:02:21.627060+0000","last_undegraded":"2024-09-19T09:02:21.627060+0000","last_fullsized":"2024-09-19T09:02:21.627060+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:24:17.000064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624642+0000","last_change":"2024-09-19T09:00:46.967399+0000","last_active":"2024-09-19T09:02:21.624642+0000","last_peered":"2024-09-19T09:02:21.624642+0000","last_clean":"2024-09-19T09:02:21.624642+0000","last_became_active":"2024-09-19T09:00:46.966145+0000","last_became_peered":"2024-09-19T09:00:46.966145+0000","last_unstale":"2024-09-19T09:02:21.624642+0000","last_undegraded":"2024-09-19T09:02:21.624642+0000","last_fullsized":"2024-09-19T09:02:21.624642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:29.192469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.614692+0000","last_change":"2024-09-19T09:00:46.960956+0000","last_active":"2024-09-19T09:02:21.614692+0000","last_peered":"2024-09-19T09:02:21.614692+0000","last_clean":"2024-09-19T09:02:21.614692+0000","last_became_active":"2024-09-19T09:00:46.960664+0000","last_became_peered":"2024-09-19T09:00:46.960664+0000","last_unstale":"2024-09-19T09:02:21.614692+0000","last_undegraded":"2024-09-19T09:02:21.614692+0000","last_fullsized":"2024-09-19T09:02:21.614692+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:18.724348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622878+0000","last_change":"2024-09-19T09:00:46.951362+0000","last_active":"2024-09-19T09:02:21.622878+0000","last_peered":"2024-09-19T09:02:21.622878+0000","last_clean":"2024-09-19T09:02:21.622878+0000","last_became_active":"2024-09-19T09:00:46.951004+0000","last_became_peered":"2024-09-19T09:00:46.951004+0000","last_unstale":"2024-09-19T09:02:21.622878+0000","last_undegraded":"2024-09-19T09:02:21.622878+0000","last_fullsized":"2024-09-19T09:02:21.622878+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:43.574425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.623346+0000","last_change":"2024-09-19T09:00:46.966752+0000","last_active":"2024-09-19T09:02:21.623346+0000","last_peered":"2024-09-19T09:02:21.623346+0000","last_clean":"2024-09-19T09:02:21.623346+0000","last_became_active":"2024-09-19T09:00:46.966370+0000","last_became_peered":"2024-09-19T09:00:46.966370+0000","last_unstale":"2024-09-19T09:02:21.623346+0000","last_undegraded":"2024-09-19T09:02:21.623346+0000","last_fullsized":"2024-09-19T09:02:21.623346+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:04.035771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621434+0000","last_change":"2024-09-19T09:00:46.956369+0000","last_active":"2024-09-19T09:02:21.621434+0000","last_peered":"2024-09-19T09:02:21.621434+0000","last_clean":"2024-09-19T09:02:21.621434+0000","last_became_active":"2024-09-19T09:00:46.956075+0000","last_became_peered":"2024-09-19T09:00:46.956075+0000","last_unstale":"2024-09-19T09:02:21.621434+0000","last_undegraded":"2024-09-19T09:02:21.621434+0000","last_fullsized":"2024-09-19T09:02:21.621434+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:23.985017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633789+0000","last_change":"2024-09-19T09:00:46.956661+0000","last_active":"2024-09-19T09:02:21.633789+0000","last_peered":"2024-09-19T09:02:21.633789+0000","last_clean":"2024-09-19T09:02:21.633789+0000","last_became_active":"2024-09-19T09:00:46.956291+0000","last_became_peered":"2024-09-19T09:00:46.956291+0000","last_unstale":"2024-09-19T09:02:21.633789+0000","last_undegraded":"2024-09-19T09:02:21.633789+0000","last_fullsized":"2024-09-19T09:02:21.633789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:55.713039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.616782+0000","last_change":"2024-09-19T09:00:46.961866+0000","last_active":"2024-09-19T09:02:21.616782+0000","last_peered":"2024-09-19T09:02:21.616782+0000","last_clean":"2024-09-19T09:02:21.616782+0000","last_became_active":"2024-09-19T09:00:46.961469+0000","last_became_peered":"2024-09-19T09:00:46.961469+0000","last_unstale":"2024-09-19T09:02:21.616782+0000","last_undegraded":"2024-09-19T09:02:21.616782+0000","last_fullsized":"2024-09-19T09:02:21.616782+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:47.665419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622833+0000","last_change":"2024-09-19T09:00:46.948765+0000","last_active":"2024-09-19T09:02:21.622833+0000","last_peered":"2024-09-19T09:02:21.622833+0000","last_clean":"2024-09-19T09:02:21.622833+0000","last_became_active":"2024-09-19T09:00:46.948431+0000","last_became_peered":"2024-09-19T09:00:46.948431+0000","last_unstale":"2024-09-19T09:02:21.622833+0000","last_undegraded":"2024-09-19T09:02:21.622833+0000","last_fullsized":"2024-09-19T09:02:21.622833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:31.766464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.618165+0000","last_change":"2024-09-19T09:00:46.962433+0000","last_active":"2024-09-19T09:02:21.618165+0000","last_peered":"2024-09-19T09:02:21.618165+0000","last_clean":"2024-09-19T09:02:21.618165+0000","last_became_active":"2024-09-19T09:00:46.962153+0000","last_became_peered":"2024-09-19T09:00:46.962153+0000","last_unstale":"2024-09-19T09:02:21.618165+0000","last_undegraded":"2024-09-19T09:02:21.618165+0000","last_fullsized":"2024-09-19T09:02:21.618165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:37.973611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.623402+0000","last_change":"2024-09-19T09:00:46.953715+0000","last_active":"2024-09-19T09:02:21.623402+0000","last_peered":"2024-09-19T09:02:21.623402+0000","last_clean":"2024-09-19T09:02:21.623402+0000","last_became_active":"2024-09-19T09:00:46.953415+0000","last_became_peered":"2024-09-19T09:00:46.953415+0000","last_unstale":"2024-09-19T09:02:21.623402+0000","last_undegraded":"2024-09-19T09:02:21.623402+0000","last_fullsized":"2024-09-19T09:02:21.623402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:29.768382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.615199+0000","last_change":"2024-09-19T09:00:46.964746+0000","last_active":"2024-09-19T09:02:21.615199+0000","last_peered":"2024-09-19T09:02:21.615199+0000","last_clean":"2024-09-19T09:02:21.615199+0000","last_became_active":"2024-09-19T09:00:46.964404+0000","last_became_peered":"2024-09-19T09:00:46.964404+0000","last_unstale":"2024-09-19T09:02:21.615199+0000","last_undegraded":"2024-09-19T09:02:21.615199+0000","last_fullsized":"2024-09-19T09:02:21.615199+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:19.841495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.619156+0000","last_change":"2024-09-19T09:00:46.962875+0000","last_active":"2024-09-19T09:02:21.619156+0000","last_peered":"2024-09-19T09:02:21.619156+0000","last_clean":"2024-09-19T09:02:21.619156+0000","last_became_active":"2024-09-19T09:00:46.962561+0000","last_became_peered":"2024-09-19T09:00:46.962561+0000","last_unstale":"2024-09-19T09:02:21.619156+0000","last_undegraded":"2024-09-19T09:02:21.619156+0000","last_fullsized":"2024-09-19T09:02:21.619156+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:39.675545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.612887+0000","last_change":"2024-09-19T09:00:46.963126+0000","last_active":"2024-09-19T09:02:21.612887+0000","last_peered":"2024-09-19T09:02:21.612887+0000","last_clean":"2024-09-19T09:02:21.612887+0000","last_became_active":"2024-09-19T09:00:46.962620+0000","last_became_peered":"2024-09-19T09:00:46.962620+0000","last_unstale":"2024-09-19T09:02:21.612887+0000","last_undegraded":"2024-09-19T09:02:21.612887+0000","last_fullsized":"2024-09-19T09:02:21.612887+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:59.723280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620254+0000","last_change":"2024-09-19T09:00:46.953026+0000","last_active":"2024-09-19T09:02:21.620254+0000","last_peered":"2024-09-19T09:02:21.620254+0000","last_clean":"2024-09-19T09:02:21.620254+0000","last_became_active":"2024-09-19T09:00:46.952596+0000","last_became_peered":"2024-09-19T09:00:46.952596+0000","last_unstale":"2024-09-19T09:02:21.620254+0000","last_undegraded":"2024-09-19T09:02:21.620254+0000","last_fullsized":"2024-09-19T09:02:21.620254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:58.524116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624169+0000","last_change":"2024-09-19T09:00:46.968964+0000","last_active":"2024-09-19T09:02:21.624169+0000","last_peered":"2024-09-19T09:02:21.624169+0000","last_clean":"2024-09-19T09:02:21.624169+0000","last_became_active":"2024-09-19T09:00:46.968505+0000","last_became_peered":"2024-09-19T09:00:46.968505+0000","last_unstale":"2024-09-19T09:02:21.624169+0000","last_undegraded":"2024-09-19T09:02:21.624169+0000","last_fullsized":"2024-09-19T09:02:21.624169+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:17:33.335541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.625205+0000","last_change":"2024-09-19T09:00:46.961520+0000","last_active":"2024-09-19T09:02:21.625205+0000","last_peered":"2024-09-19T09:02:21.625205+0000","last_clean":"2024-09-19T09:02:21.625205+0000","last_became_active":"2024-09-19T09:00:46.961140+0000","last_became_peered":"2024-09-19T09:00:46.961140+0000","last_unstale":"2024-09-19T09:02:21.625205+0000","last_undegraded":"2024-09-19T09:02:21.625205+0000","last_fullsized":"2024-09-19T09:02:21.625205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:03.120173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621084+0000","last_change":"2024-09-19T09:00:46.955995+0000","last_active":"2024-09-19T09:02:21.621084+0000","last_peered":"2024-09-19T09:02:21.621084+0000","last_clean":"2024-09-19T09:02:21.621084+0000","last_became_active":"2024-09-19T09:00:46.954678+0000","last_became_peered":"2024-09-19T09:00:46.954678+0000","last_unstale":"2024-09-19T09:02:21.621084+0000","last_undegraded":"2024-09-19T09:02:21.621084+0000","last_fullsized":"2024-09-19T09:02:21.621084+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:10.314575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":135,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:41.284935+0000","last_change":"2024-09-19T09:00:44.946000+0000","last_active":"2024-09-19T09:02:41.284935+0000","last_peered":"2024-09-19T09:02:41.284935+0000","last_clean":"2024-09-19T09:02:41.284935+0000","last_became_active":"2024-09-19T09:00:44.945583+0000","last_became_peered":"2024-09-19T09:00:44.945583+0000","last_unstale":"2024-09-19T09:02:41.284935+0000","last_undegraded":"2024-09-19T09:02:41.284935+0000","last_fullsized":"2024-09-19T09:02:41.284935+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:53:19.109697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624079+0000","last_change":"2024-09-19T09:01:52.904443+0000","last_active":"2024-09-19T09:02:21.624079+0000","last_peered":"2024-09-19T09:02:21.624079+0000","last_clean":"2024-09-19T09:02:21.624079+0000","last_became_active":"2024-09-19T09:01:52.904077+0000","last_became_peered":"2024-09-19T09:01:52.904077+0000","last_unstale":"2024-09-19T09:02:21.624079+0000","last_undegraded":"2024-09-19T09:02:21.624079+0000","last_fullsized":"2024-09-19T09:02:21.624079+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:50.467090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.748381+0000","last_change":"2024-09-19T09:00:44.942435+0000","last_active":"2024-09-19T09:01:52.748381+0000","last_peered":"2024-09-19T09:01:52.748381+0000","last_clean":"2024-09-19T09:01:52.748381+0000","last_became_active":"2024-09-19T09:00:44.942067+0000","last_became_peered":"2024-09-19T09:00:44.942067+0000","last_unstale":"2024-09-19T09:01:52.748381+0000","last_undegraded":"2024-09-19T09:01:52.748381+0000","last_fullsized":"2024-09-19T09:01:52.748381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:16.476506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.747900+0000","last_change":"2024-09-19T09:00:44.937944+0000","last_active":"2024-09-19T09:01:52.747900+0000","last_peered":"2024-09-19T09:01:52.747900+0000","last_clean":"2024-09-19T09:01:52.747900+0000","last_became_active":"2024-09-19T09:00:44.937623+0000","last_became_peered":"2024-09-19T09:00:44.937623+0000","last_unstale":"2024-09-19T09:01:52.747900+0000","last_undegraded":"2024-09-19T09:01:52.747900+0000","last_fullsized":"2024-09-19T09:01:52.747900+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:49:08.730015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620184+0000","last_change":"2024-09-19T09:00:46.959932+0000","last_active":"2024-09-19T09:02:21.620184+0000","last_peered":"2024-09-19T09:02:21.620184+0000","last_clean":"2024-09-19T09:02:21.620184+0000","last_became_active":"2024-09-19T09:00:46.959637+0000","last_became_peered":"2024-09-19T09:00:46.959637+0000","last_unstale":"2024-09-19T09:02:21.620184+0000","last_undegraded":"2024-09-19T09:02:21.620184+0000","last_fullsized":"2024-09-19T09:02:21.620184+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:55.337669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751513+0000","last_change":"2024-09-19T09:00:44.944882+0000","last_active":"2024-09-19T09:01:52.751513+0000","last_peered":"2024-09-19T09:01:52.751513+0000","last_clean":"2024-09-19T09:01:52.751513+0000","last_became_active":"2024-09-19T09:00:44.944116+0000","last_became_peered":"2024-09-19T09:00:44.944116+0000","last_unstale":"2024-09-19T09:01:52.751513+0000","last_undegraded":"2024-09-19T09:01:52.751513+0000","last_fullsized":"2024-09-19T09:01:52.751513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:54:58.555913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.615953+0000","last_change":"2024-09-19T09:00:46.956859+0000","last_active":"2024-09-19T09:02:21.615953+0000","last_peered":"2024-09-19T09:02:21.615953+0000","last_clean":"2024-09-19T09:02:21.615953+0000","last_became_active":"2024-09-19T09:00:46.956561+0000","last_became_peered":"2024-09-19T09:00:46.956561+0000","last_unstale":"2024-09-19T09:02:21.615953+0000","last_undegraded":"2024-09-19T09:02:21.615953+0000","last_fullsized":"2024-09-19T09:02:21.615953+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:09:04.125591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'5","reported_seq":231,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:40.309800+0000","last_change":"2024-09-19T09:00:44.945959+0000","last_active":"2024-09-19T09:02:40.309800+0000","last_peered":"2024-09-19T09:02:40.309800+0000","last_clean":"2024-09-19T09:02:40.309800+0000","last_became_active":"2024-09-19T09:00:44.945640+0000","last_became_peered":"2024-09-19T09:00:44.945640+0000","last_unstale":"2024-09-19T09:02:40.309800+0000","last_undegraded":"2024-09-19T09:02:40.309800+0000","last_fullsized":"2024-09-19T09:02:40.309800+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:02.523834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":204,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.632897+0000","last_change":"2024-09-19T09:00:46.953677+0000","last_active":"2024-09-19T09:02:21.632897+0000","last_peered":"2024-09-19T09:02:21.632897+0000","last_clean":"2024-09-19T09:02:21.632897+0000","last_became_active":"2024-09-19T09:00:46.953357+0000","last_became_peered":"2024-09-19T09:00:46.953357+0000","last_unstale":"2024-09-19T09:02:21.632897+0000","last_undegraded":"2024-09-19T09:02:21.632897+0000","last_fullsized":"2024-09-19T09:02:21.632897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:48.963003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.893433+0000","last_change":"2024-09-19T09:01:52.893433+0000","last_active":"2024-09-19T09:01:52.893433+0000","last_peered":"2024-09-19T09:01:52.893433+0000","last_clean":"2024-09-19T09:01:52.893433+0000","last_became_active":"2024-09-19T09:01:52.889836+0000","last_became_peered":"2024-09-19T09:01:52.889836+0000","last_unstale":"2024-09-19T09:01:52.893433+0000","last_undegraded":"2024-09-19T09:01:52.893433+0000","last_fullsized":"2024-09-19T09:01:52.893433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:08:22.339717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622288+0000","last_change":"2024-09-19T09:00:46.962285+0000","last_active":"2024-09-19T09:02:21.622288+0000","last_peered":"2024-09-19T09:02:21.622288+0000","last_clean":"2024-09-19T09:02:21.622288+0000","last_became_active":"2024-09-19T09:00:46.961982+0000","last_became_peered":"2024-09-19T09:00:46.961982+0000","last_unstale":"2024-09-19T09:02:21.622288+0000","last_undegraded":"2024-09-19T09:02:21.622288+0000","last_fullsized":"2024-09-19T09:02:21.622288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:41.658501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752480+0000","last_change":"2024-09-19T09:00:45.241425+0000","last_active":"2024-09-19T09:01:52.752480+0000","last_peered":"2024-09-19T09:01:52.752480+0000","last_clean":"2024-09-19T09:01:52.752480+0000","last_became_active":"2024-09-19T09:00:45.241032+0000","last_became_peered":"2024-09-19T09:00:45.241032+0000","last_unstale":"2024-09-19T09:01:52.752480+0000","last_undegraded":"2024-09-19T09:01:52.752480+0000","last_fullsized":"2024-09-19T09:01:52.752480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:00.707855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.626097+0000","last_change":"2024-09-19T09:00:46.962436+0000","last_active":"2024-09-19T09:02:21.626097+0000","last_peered":"2024-09-19T09:02:21.626097+0000","last_clean":"2024-09-19T09:02:21.626097+0000","last_became_active":"2024-09-19T09:00:46.962114+0000","last_became_peered":"2024-09-19T09:00:46.962114+0000","last_unstale":"2024-09-19T09:02:21.626097+0000","last_undegraded":"2024-09-19T09:02:21.626097+0000","last_fullsized":"2024-09-19T09:02:21.626097+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:41.761705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'37","reported_seq":273,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:40.544906+0000","last_change":"2024-09-19T09:00:44.945187+0000","last_active":"2024-09-19T09:02:40.544906+0000","last_peered":"2024-09-19T09:02:40.544906+0000","last_clean":"2024-09-19T09:02:40.544906+0000","last_became_active":"2024-09-19T09:00:44.944763+0000","last_became_peered":"2024-09-19T09:00:44.944763+0000","last_unstale":"2024-09-19T09:02:40.544906+0000","last_undegraded":"2024-09-19T09:02:40.544906+0000","last_fullsized":"2024-09-19T09:02:40.544906+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:06.052994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62867,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":214,"num_read_kb":168,"num_write":39,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.630413+0000","last_change":"2024-09-19T09:00:46.960991+0000","last_active":"2024-09-19T09:02:21.630413+0000","last_peered":"2024-09-19T09:02:21.630413+0000","last_clean":"2024-09-19T09:02:21.630413+0000","last_became_active":"2024-09-19T09:00:46.958962+0000","last_became_peered":"2024-09-19T09:00:46.958962+0000","last_unstale":"2024-09-19T09:02:21.630413+0000","last_undegraded":"2024-09-19T09:02:21.630413+0000","last_fullsized":"2024-09-19T09:02:21.630413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:21.795694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.755250+0000","last_change":"2024-09-19T09:00:44.946176+0000","last_active":"2024-09-19T09:01:52.755250+0000","last_peered":"2024-09-19T09:01:52.755250+0000","last_clean":"2024-09-19T09:01:52.755250+0000","last_became_active":"2024-09-19T09:00:44.945869+0000","last_became_peered":"2024-09-19T09:00:44.945869+0000","last_unstale":"2024-09-19T09:01:52.755250+0000","last_undegraded":"2024-09-19T09:01:52.755250+0000","last_fullsized":"2024-09-19T09:01:52.755250+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:16.022184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621664+0000","last_change":"2024-09-19T09:00:46.956345+0000","last_active":"2024-09-19T09:02:21.621664+0000","last_peered":"2024-09-19T09:02:21.621664+0000","last_clean":"2024-09-19T09:02:21.621664+0000","last_became_active":"2024-09-19T09:00:46.956057+0000","last_became_peered":"2024-09-19T09:00:46.956057+0000","last_unstale":"2024-09-19T09:02:21.621664+0000","last_undegraded":"2024-09-19T09:02:21.621664+0000","last_fullsized":"2024-09-19T09:02:21.621664+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:27.510661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752205+0000","last_change":"2024-09-19T09:00:44.945667+0000","last_active":"2024-09-19T09:01:52.752205+0000","last_peered":"2024-09-19T09:01:52.752205+0000","last_clean":"2024-09-19T09:01:52.752205+0000","last_became_active":"2024-09-19T09:00:44.945308+0000","last_became_peered":"2024-09-19T09:00:44.945308+0000","last_unstale":"2024-09-19T09:01:52.752205+0000","last_undegraded":"2024-09-19T09:01:52.752205+0000","last_fullsized":"2024-09-19T09:01:52.752205+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:00:11.304913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.629868+0000","last_change":"2024-09-19T09:00:46.965088+0000","last_active":"2024-09-19T09:02:21.629868+0000","last_peered":"2024-09-19T09:02:21.629868+0000","last_clean":"2024-09-19T09:02:21.629868+0000","last_became_active":"2024-09-19T09:00:46.964777+0000","last_became_peered":"2024-09-19T09:00:46.964777+0000","last_unstale":"2024-09-19T09:02:21.629868+0000","last_undegraded":"2024-09-19T09:02:21.629868+0000","last_fullsized":"2024-09-19T09:02:21.629868+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:32.958984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'7","reported_seq":137,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:41.284063+0000","last_change":"2024-09-19T09:00:44.946193+0000","last_active":"2024-09-19T09:02:41.284063+0000","last_peered":"2024-09-19T09:02:41.284063+0000","last_clean":"2024-09-19T09:02:41.284063+0000","last_became_active":"2024-09-19T09:00:44.945168+0000","last_became_peered":"2024-09-19T09:00:44.945168+0000","last_unstale":"2024-09-19T09:02:41.284063+0000","last_undegraded":"2024-09-19T09:02:41.284063+0000","last_fullsized":"2024-09-19T09:02:41.284063+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:21.230689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.627048+0000","last_change":"2024-09-19T09:01:52.754058+0000","last_active":"2024-09-19T09:02:21.627048+0000","last_peered":"2024-09-19T09:02:21.627048+0000","last_clean":"2024-09-19T09:02:21.627048+0000","last_became_active":"2024-09-19T09:01:52.753674+0000","last_became_peered":"2024-09-19T09:01:52.753674+0000","last_unstale":"2024-09-19T09:02:21.627048+0000","last_undegraded":"2024-09-19T09:02:21.627048+0000","last_fullsized":"2024-09-19T09:02:21.627048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:05.665121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":64,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.774430+0000","last_change":"2024-09-19T09:01:52.774430+0000","last_active":"2024-09-19T09:01:52.774430+0000","last_peered":"2024-09-19T09:01:52.774430+0000","last_clean":"2024-09-19T09:01:52.774430+0000","last_became_active":"2024-09-19T09:01:52.756083+0000","last_became_peered":"2024-09-19T09:01:52.756083+0000","last_unstale":"2024-09-19T09:01:52.774430+0000","last_undegraded":"2024-09-19T09:01:52.774430+0000","last_fullsized":"2024-09-19T09:01:52.774430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:52.639490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633068+0000","last_change":"2024-09-19T09:01:52.757684+0000","last_active":"2024-09-19T09:02:21.633068+0000","last_peered":"2024-09-19T09:02:21.633068+0000","last_clean":"2024-09-19T09:02:21.633068+0000","last_became_active":"2024-09-19T09:01:52.756969+0000","last_became_peered":"2024-09-19T09:01:52.756969+0000","last_unstale":"2024-09-19T09:02:21.633068+0000","last_undegraded":"2024-09-19T09:02:21.633068+0000","last_fullsized":"2024-09-19T09:02:21.633068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:24:47.117996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752185+0000","last_change":"2024-09-19T09:00:44.946138+0000","last_active":"2024-09-19T09:01:52.752185+0000","last_peered":"2024-09-19T09:01:52.752185+0000","last_clean":"2024-09-19T09:01:52.752185+0000","last_became_active":"2024-09-19T09:00:44.945793+0000","last_became_peered":"2024-09-19T09:00:44.945793+0000","last_unstale":"2024-09-19T09:01:52.752185+0000","last_undegraded":"2024-09-19T09:01:52.752185+0000","last_fullsized":"2024-09-19T09:01:52.752185+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:01.341560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620195+0000","last_change":"2024-09-19T09:00:46.970360+0000","last_active":"2024-09-19T09:02:21.620195+0000","last_peered":"2024-09-19T09:02:21.620195+0000","last_clean":"2024-09-19T09:02:21.620195+0000","last_became_active":"2024-09-19T09:00:46.969971+0000","last_became_peered":"2024-09-19T09:00:46.969971+0000","last_unstale":"2024-09-19T09:02:21.620195+0000","last_undegraded":"2024-09-19T09:02:21.620195+0000","last_fullsized":"2024-09-19T09:02:21.620195+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:48.812158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.748305+0000","last_change":"2024-09-19T09:00:44.941713+0000","last_active":"2024-09-19T09:01:52.748305+0000","last_peered":"2024-09-19T09:01:52.748305+0000","last_clean":"2024-09-19T09:01:52.748305+0000","last_became_active":"2024-09-19T09:00:44.941373+0000","last_became_peered":"2024-09-19T09:00:44.941373+0000","last_unstale":"2024-09-19T09:01:52.748305+0000","last_undegraded":"2024-09-19T09:01:52.748305+0000","last_fullsized":"2024-09-19T09:01:52.748305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:31.157803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.630100+0000","last_change":"2024-09-19T09:00:46.959383+0000","last_active":"2024-09-19T09:02:21.630100+0000","last_peered":"2024-09-19T09:02:21.630100+0000","last_clean":"2024-09-19T09:02:21.630100+0000","last_became_active":"2024-09-19T09:00:46.958860+0000","last_became_peered":"2024-09-19T09:00:46.958860+0000","last_unstale":"2024-09-19T09:02:21.630100+0000","last_undegraded":"2024-09-19T09:02:21.630100+0000","last_fullsized":"2024-09-19T09:02:21.630100+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:32:04.423017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752778+0000","last_change":"2024-09-19T09:00:44.944797+0000","last_active":"2024-09-19T09:01:52.752778+0000","last_peered":"2024-09-19T09:01:52.752778+0000","last_clean":"2024-09-19T09:01:52.752778+0000","last_became_active":"2024-09-19T09:00:44.944259+0000","last_became_peered":"2024-09-19T09:00:44.944259+0000","last_unstale":"2024-09-19T09:01:52.752778+0000","last_undegraded":"2024-09-19T09:01:52.752778+0000","last_fullsized":"2024-09-19T09:01:52.752778+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:58.996217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624674+0000","last_change":"2024-09-19T09:00:46.964998+0000","last_active":"2024-09-19T09:02:21.624674+0000","last_peered":"2024-09-19T09:02:21.624674+0000","last_clean":"2024-09-19T09:02:21.624674+0000","last_became_active":"2024-09-19T09:00:46.964562+0000","last_became_peered":"2024-09-19T09:00:46.964562+0000","last_unstale":"2024-09-19T09:02:21.624674+0000","last_undegraded":"2024-09-19T09:02:21.624674+0000","last_fullsized":"2024-09-19T09:02:21.624674+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:00.089800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":135,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:42.313326+0000","last_change":"2024-09-19T09:00:45.239770+0000","last_active":"2024-09-19T09:02:42.313326+0000","last_peered":"2024-09-19T09:02:42.313326+0000","last_clean":"2024-09-19T09:02:42.313326+0000","last_became_active":"2024-09-19T09:00:45.239385+0000","last_became_peered":"2024-09-19T09:00:45.239385+0000","last_unstale":"2024-09-19T09:02:42.313326+0000","last_undegraded":"2024-09-19T09:02:42.313326+0000","last_fullsized":"2024-09-19T09:02:42.313326+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:21.109900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.634073+0000","last_change":"2024-09-19T09:00:46.967505+0000","last_active":"2024-09-19T09:02:21.634073+0000","last_peered":"2024-09-19T09:02:21.634073+0000","last_clean":"2024-09-19T09:02:21.634073+0000","last_became_active":"2024-09-19T09:00:46.967195+0000","last_became_peered":"2024-09-19T09:00:46.967195+0000","last_unstale":"2024-09-19T09:02:21.634073+0000","last_undegraded":"2024-09-19T09:02:21.634073+0000","last_fullsized":"2024-09-19T09:02:21.634073+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:52:28.917333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":335,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751112+0000","last_change":"2024-09-19T08:59:33.821924+0000","last_active":"2024-09-19T09:01:52.751112+0000","last_peered":"2024-09-19T09:01:52.751112+0000","last_clean":"2024-09-19T09:01:52.751112+0000","last_became_active":"2024-09-19T08:59:33.821404+0000","last_became_peered":"2024-09-19T08:59:33.821404+0000","last_unstale":"2024-09-19T09:01:52.751112+0000","last_undegraded":"2024-09-19T09:01:52.751112+0000","last_fullsized":"2024-09-19T09:01:52.751112+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:56:07.830480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:56:07.830480+0000","last_clean_scrub_stamp":"2024-09-19T08:56:07.830480+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:21.763918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.888480+0000","last_change":"2024-09-19T09:00:44.943714+0000","last_active":"2024-09-19T09:01:52.888480+0000","last_peered":"2024-09-19T09:01:52.888480+0000","last_clean":"2024-09-19T09:01:52.888480+0000","last_became_active":"2024-09-19T09:00:44.943362+0000","last_became_peered":"2024-09-19T09:00:44.943362+0000","last_unstale":"2024-09-19T09:01:52.888480+0000","last_undegraded":"2024-09-19T09:01:52.888480+0000","last_fullsized":"2024-09-19T09:01:52.888480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:30.015648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633592+0000","last_change":"2024-09-19T09:00:46.956784+0000","last_active":"2024-09-19T09:02:21.633592+0000","last_peered":"2024-09-19T09:02:21.633592+0000","last_clean":"2024-09-19T09:02:21.633592+0000","last_became_active":"2024-09-19T09:00:46.956443+0000","last_became_peered":"2024-09-19T09:00:46.956443+0000","last_unstale":"2024-09-19T09:02:21.633592+0000","last_undegraded":"2024-09-19T09:02:21.633592+0000","last_fullsized":"2024-09-19T09:02:21.633592+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:18.918788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.750940+0000","last_change":"2024-09-19T09:00:44.947796+0000","last_active":"2024-09-19T09:01:52.750940+0000","last_peered":"2024-09-19T09:01:52.750940+0000","last_clean":"2024-09-19T09:01:52.750940+0000","last_became_active":"2024-09-19T09:00:44.946949+0000","last_became_peered":"2024-09-19T09:00:44.946949+0000","last_unstale":"2024-09-19T09:01:52.750940+0000","last_undegraded":"2024-09-19T09:01:52.750940+0000","last_fullsized":"2024-09-19T09:01:52.750940+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:42.210186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.619434+0000","last_change":"2024-09-19T09:00:46.963856+0000","last_active":"2024-09-19T09:02:21.619434+0000","last_peered":"2024-09-19T09:02:21.619434+0000","last_clean":"2024-09-19T09:02:21.619434+0000","last_became_active":"2024-09-19T09:00:46.963543+0000","last_became_peered":"2024-09-19T09:00:46.963543+0000","last_unstale":"2024-09-19T09:02:21.619434+0000","last_undegraded":"2024-09-19T09:02:21.619434+0000","last_fullsized":"2024-09-19T09:02:21.619434+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:31.041028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'6","reported_seq":134,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:42.284796+0000","last_change":"2024-09-19T09:00:45.239269+0000","last_active":"2024-09-19T09:02:42.284796+0000","last_peered":"2024-09-19T09:02:42.284796+0000","last_clean":"2024-09-19T09:02:42.284796+0000","last_became_active":"2024-09-19T09:00:45.238929+0000","last_became_peered":"2024-09-19T09:00:45.238929+0000","last_unstale":"2024-09-19T09:02:42.284796+0000","last_undegraded":"2024-09-19T09:02:42.284796+0000","last_fullsized":"2024-09-19T09:02:42.284796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:46.684148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622349+0000","last_change":"2024-09-19T09:00:46.963595+0000","last_active":"2024-09-19T09:02:21.622349+0000","last_peered":"2024-09-19T09:02:21.622349+0000","last_clean":"2024-09-19T09:02:21.622349+0000","last_became_active":"2024-09-19T09:00:46.963243+0000","last_became_peered":"2024-09-19T09:00:46.963243+0000","last_unstale":"2024-09-19T09:02:21.622349+0000","last_undegraded":"2024-09-19T09:02:21.622349+0000","last_fullsized":"2024-09-19T09:02:21.622349+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:01.122996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":128,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:37.843621+0000","last_change":"2024-09-19T09:00:44.941550+0000","last_active":"2024-09-19T09:02:37.843621+0000","last_peered":"2024-09-19T09:02:37.843621+0000","last_clean":"2024-09-19T09:02:37.843621+0000","last_became_active":"2024-09-19T09:00:44.941145+0000","last_became_peered":"2024-09-19T09:00:44.941145+0000","last_unstale":"2024-09-19T09:02:37.843621+0000","last_undegraded":"2024-09-19T09:02:37.843621+0000","last_fullsized":"2024-09-19T09:02:37.843621+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:53.893887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.612369+0000","last_change":"2024-09-19T09:00:46.963473+0000","last_active":"2024-09-19T09:02:21.612369+0000","last_peered":"2024-09-19T09:02:21.612369+0000","last_clean":"2024-09-19T09:02:21.612369+0000","last_became_active":"2024-09-19T09:00:46.963090+0000","last_became_peered":"2024-09-19T09:00:46.963090+0000","last_unstale":"2024-09-19T09:02:21.612369+0000","last_undegraded":"2024-09-19T09:02:21.612369+0000","last_fullsized":"2024-09-19T09:02:21.612369+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:01.492385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.750853+0000","last_change":"2024-09-19T09:00:45.239566+0000","last_active":"2024-09-19T09:01:52.750853+0000","last_peered":"2024-09-19T09:01:52.750853+0000","last_clean":"2024-09-19T09:01:52.750853+0000","last_became_active":"2024-09-19T09:00:45.239250+0000","last_became_peered":"2024-09-19T09:00:45.239250+0000","last_unstale":"2024-09-19T09:01:52.750853+0000","last_undegraded":"2024-09-19T09:01:52.750853+0000","last_fullsized":"2024-09-19T09:01:52.750853+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:44:21.739311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.614525+0000","last_change":"2024-09-19T09:00:46.955344+0000","last_active":"2024-09-19T09:02:21.614525+0000","last_peered":"2024-09-19T09:02:21.614525+0000","last_clean":"2024-09-19T09:02:21.614525+0000","last_became_active":"2024-09-19T09:00:46.954961+0000","last_became_peered":"2024-09-19T09:00:46.954961+0000","last_unstale":"2024-09-19T09:02:21.614525+0000","last_undegraded":"2024-09-19T09:02:21.614525+0000","last_fullsized":"2024-09-19T09:02:21.614525+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:55.743556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752072+0000","last_change":"2024-09-19T09:00:45.241514+0000","last_active":"2024-09-19T09:01:52.752072+0000","last_peered":"2024-09-19T09:01:52.752072+0000","last_clean":"2024-09-19T09:01:52.752072+0000","last_became_active":"2024-09-19T09:00:45.241097+0000","last_became_peered":"2024-09-19T09:00:45.241097+0000","last_unstale":"2024-09-19T09:01:52.752072+0000","last_undegraded":"2024-09-19T09:01:52.752072+0000","last_fullsized":"2024-09-19T09:01:52.752072+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:35:38.323393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620312+0000","last_change":"2024-09-19T09:00:46.955194+0000","last_active":"2024-09-19T09:02:21.620312+0000","last_peered":"2024-09-19T09:02:21.620312+0000","last_clean":"2024-09-19T09:02:21.620312+0000","last_became_active":"2024-09-19T09:00:46.954771+0000","last_became_peered":"2024-09-19T09:00:46.954771+0000","last_unstale":"2024-09-19T09:02:21.620312+0000","last_undegraded":"2024-09-19T09:02:21.620312+0000","last_fullsized":"2024-09-19T09:02:21.620312+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:18.945035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'16","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.975541+0000","last_change":"2024-09-19T09:00:44.938554+0000","last_active":"2024-09-19T09:02:21.975541+0000","last_peered":"2024-09-19T09:02:21.975541+0000","last_clean":"2024-09-19T09:02:21.975541+0000","last_became_active":"2024-09-19T09:00:44.938261+0000","last_became_peered":"2024-09-19T09:00:44.938261+0000","last_unstale":"2024-09-19T09:02:21.975541+0000","last_undegraded":"2024-09-19T09:02:21.975541+0000","last_fullsized":"2024-09-19T09:02:21.975541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:24.293764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.614207+0000","last_change":"2024-09-19T09:01:52.755528+0000","last_active":"2024-09-19T09:02:21.614207+0000","last_peered":"2024-09-19T09:02:21.614207+0000","last_clean":"2024-09-19T09:02:21.614207+0000","last_became_active":"2024-09-19T09:01:52.755188+0000","last_became_peered":"2024-09-19T09:01:52.755188+0000","last_unstale":"2024-09-19T09:02:21.614207+0000","last_undegraded":"2024-09-19T09:02:21.614207+0000","last_fullsized":"2024-09-19T09:02:21.614207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:49:34.988733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.888813+0000","last_change":"2024-09-19T09:00:44.941479+0000","last_active":"2024-09-19T09:01:52.888813+0000","last_peered":"2024-09-19T09:01:52.888813+0000","last_clean":"2024-09-19T09:01:52.888813+0000","last_became_active":"2024-09-19T09:00:44.937197+0000","last_became_peered":"2024-09-19T09:00:44.937197+0000","last_unstale":"2024-09-19T09:01:52.888813+0000","last_undegraded":"2024-09-19T09:01:52.888813+0000","last_fullsized":"2024-09-19T09:01:52.888813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:53:17.385884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621026+0000","last_change":"2024-09-19T09:00:46.970319+0000","last_active":"2024-09-19T09:02:21.621026+0000","last_peered":"2024-09-19T09:02:21.621026+0000","last_clean":"2024-09-19T09:02:21.621026+0000","last_became_active":"2024-09-19T09:00:46.969899+0000","last_became_peered":"2024-09-19T09:00:46.969899+0000","last_unstale":"2024-09-19T09:02:21.621026+0000","last_undegraded":"2024-09-19T09:02:21.621026+0000","last_fullsized":"2024-09-19T09:02:21.621026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:26:21.379231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751656+0000","last_change":"2024-09-19T09:00:45.239525+0000","last_active":"2024-09-19T09:01:52.751656+0000","last_peered":"2024-09-19T09:01:52.751656+0000","last_clean":"2024-09-19T09:01:52.751656+0000","last_became_active":"2024-09-19T09:00:45.239213+0000","last_became_peered":"2024-09-19T09:00:45.239213+0000","last_unstale":"2024-09-19T09:01:52.751656+0000","last_undegraded":"2024-09-19T09:01:52.751656+0000","last_fullsized":"2024-09-19T09:01:52.751656+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:32.525025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.584763+0000","last_change":"2024-09-19T09:00:46.958573+0000","last_active":"2024-09-19T09:02:21.584763+0000","last_peered":"2024-09-19T09:02:21.584763+0000","last_clean":"2024-09-19T09:02:21.584763+0000","last_became_active":"2024-09-19T09:00:46.958274+0000","last_became_peered":"2024-09-19T09:00:46.958274+0000","last_unstale":"2024-09-19T09:02:21.584763+0000","last_undegraded":"2024-09-19T09:02:21.584763+0000","last_fullsized":"2024-09-19T09:02:21.584763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:17.249122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.903274+0000","last_change":"2024-09-19T09:00:44.945194+0000","last_active":"2024-09-19T09:01:52.903274+0000","last_peered":"2024-09-19T09:01:52.903274+0000","last_clean":"2024-09-19T09:01:52.903274+0000","last_became_active":"2024-09-19T09:00:44.944931+0000","last_became_peered":"2024-09-19T09:00:44.944931+0000","last_unstale":"2024-09-19T09:01:52.903274+0000","last_undegraded":"2024-09-19T09:01:52.903274+0000","last_fullsized":"2024-09-19T09:01:52.903274+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:11.181150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621916+0000","last_change":"2024-09-19T09:00:46.966895+0000","last_active":"2024-09-19T09:02:21.621916+0000","last_peered":"2024-09-19T09:02:21.621916+0000","last_clean":"2024-09-19T09:02:21.621916+0000","last_became_active":"2024-09-19T09:00:46.966124+0000","last_became_peered":"2024-09-19T09:00:46.966124+0000","last_unstale":"2024-09-19T09:02:21.621916+0000","last_undegraded":"2024-09-19T09:02:21.621916+0000","last_fullsized":"2024-09-19T09:02:21.621916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:14:04.320863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:41.120672+0000","last_change":"2024-09-19T09:00:44.944167+0000","last_active":"2024-09-19T09:02:41.120672+0000","last_peered":"2024-09-19T09:02:41.120672+0000","last_clean":"2024-09-19T09:02:41.120672+0000","last_became_active":"2024-09-19T09:00:44.943670+0000","last_became_peered":"2024-09-19T09:00:44.943670+0000","last_unstale":"2024-09-19T09:02:41.120672+0000","last_undegraded":"2024-09-19T09:02:41.120672+0000","last_fullsized":"2024-09-19T09:02:41.120672+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:05.596877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.615278+0000","last_change":"2024-09-19T09:00:46.958581+0000","last_active":"2024-09-19T09:02:21.615278+0000","last_peered":"2024-09-19T09:02:21.615278+0000","last_clean":"2024-09-19T09:02:21.615278+0000","last_became_active":"2024-09-19T09:00:46.957782+0000","last_became_peered":"2024-09-19T09:00:46.957782+0000","last_unstale":"2024-09-19T09:02:21.615278+0000","last_undegraded":"2024-09-19T09:02:21.615278+0000","last_fullsized":"2024-09-19T09:02:21.615278+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:37:19.488538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751779+0000","last_change":"2024-09-19T09:00:44.945351+0000","last_active":"2024-09-19T09:01:52.751779+0000","last_peered":"2024-09-19T09:01:52.751779+0000","last_clean":"2024-09-19T09:01:52.751779+0000","last_became_active":"2024-09-19T09:00:44.944991+0000","last_became_peered":"2024-09-19T09:00:44.944991+0000","last_unstale":"2024-09-19T09:01:52.751779+0000","last_undegraded":"2024-09-19T09:01:52.751779+0000","last_fullsized":"2024-09-19T09:01:52.751779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:45:07.581872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633316+0000","last_change":"2024-09-19T09:00:46.956340+0000","last_active":"2024-09-19T09:02:21.633316+0000","last_peered":"2024-09-19T09:02:21.633316+0000","last_clean":"2024-09-19T09:02:21.633316+0000","last_became_active":"2024-09-19T09:00:46.955956+0000","last_became_peered":"2024-09-19T09:00:46.955956+0000","last_unstale":"2024-09-19T09:02:21.633316+0000","last_undegraded":"2024-09-19T09:02:21.633316+0000","last_fullsized":"2024-09-19T09:02:21.633316+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:34.863428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633515+0000","last_change":"2024-09-19T09:00:46.963657+0000","last_active":"2024-09-19T09:02:21.633515+0000","last_peered":"2024-09-19T09:02:21.633515+0000","last_clean":"2024-09-19T09:02:21.633515+0000","last_became_active":"2024-09-19T09:00:46.963134+0000","last_became_peered":"2024-09-19T09:00:46.963134+0000","last_unstale":"2024-09-19T09:02:21.633515+0000","last_undegraded":"2024-09-19T09:02:21.633515+0000","last_fullsized":"2024-09-19T09:02:21.633515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:01.098570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.764939+0000","last_change":"2024-09-19T09:01:52.764438+0000","last_active":"2024-09-19T09:01:52.764939+0000","last_peered":"2024-09-19T09:01:52.764939+0000","last_clean":"2024-09-19T09:01:52.764939+0000","last_became_active":"2024-09-19T09:01:52.755188+0000","last_became_peered":"2024-09-19T09:01:52.755188+0000","last_unstale":"2024-09-19T09:01:52.764939+0000","last_undegraded":"2024-09-19T09:01:52.764939+0000","last_fullsized":"2024-09-19T09:01:52.764939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:40.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.632994+0000","last_change":"2024-09-19T09:00:46.956354+0000","last_active":"2024-09-19T09:02:21.632994+0000","last_peered":"2024-09-19T09:02:21.632994+0000","last_clean":"2024-09-19T09:02:21.632994+0000","last_became_active":"2024-09-19T09:00:46.955964+0000","last_became_peered":"2024-09-19T09:00:46.955964+0000","last_unstale":"2024-09-19T09:02:21.632994+0000","last_undegraded":"2024-09-19T09:02:21.632994+0000","last_fullsized":"2024-09-19T09:02:21.632994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:05:33.243479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751944+0000","last_change":"2024-09-19T09:00:44.944983+0000","last_active":"2024-09-19T09:01:52.751944+0000","last_peered":"2024-09-19T09:01:52.751944+0000","last_clean":"2024-09-19T09:01:52.751944+0000","last_became_active":"2024-09-19T09:00:44.944625+0000","last_became_peered":"2024-09-19T09:00:44.944625+0000","last_unstale":"2024-09-19T09:01:52.751944+0000","last_undegraded":"2024-09-19T09:01:52.751944+0000","last_fullsized":"2024-09-19T09:01:52.751944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:49:50.818863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.619748+0000","last_change":"2024-09-19T09:00:46.966448+0000","last_active":"2024-09-19T09:02:21.619748+0000","last_peered":"2024-09-19T09:02:21.619748+0000","last_clean":"2024-09-19T09:02:21.619748+0000","last_became_active":"2024-09-19T09:00:46.966135+0000","last_became_peered":"2024-09-19T09:00:46.966135+0000","last_unstale":"2024-09-19T09:02:21.619748+0000","last_undegraded":"2024-09-19T09:02:21.619748+0000","last_fullsized":"2024-09-19T09:02:21.619748+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:42.652877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.903626+0000","last_change":"2024-09-19T09:00:45.239405+0000","last_active":"2024-09-19T09:01:52.903626+0000","last_peered":"2024-09-19T09:01:52.903626+0000","last_clean":"2024-09-19T09:01:52.903626+0000","last_became_active":"2024-09-19T09:00:45.239002+0000","last_became_peered":"2024-09-19T09:00:45.239002+0000","last_unstale":"2024-09-19T09:01:52.903626+0000","last_undegraded":"2024-09-19T09:01:52.903626+0000","last_fullsized":"2024-09-19T09:01:52.903626+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:46:03.934975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621992+0000","last_change":"2024-09-19T09:00:46.951635+0000","last_active":"2024-09-19T09:02:21.621992+0000","last_peered":"2024-09-19T09:02:21.621992+0000","last_clean":"2024-09-19T09:02:21.621992+0000","last_became_active":"2024-09-19T09:00:46.951325+0000","last_became_peered":"2024-09-19T09:00:46.951325+0000","last_unstale":"2024-09-19T09:02:21.621992+0000","last_undegraded":"2024-09-19T09:02:21.621992+0000","last_fullsized":"2024-09-19T09:02:21.621992+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:17.360950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":158,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:39.845934+0000","last_change":"2024-09-19T09:01:52.766049+0000","last_active":"2024-09-19T09:02:39.845934+0000","last_peered":"2024-09-19T09:02:39.845934+0000","last_clean":"2024-09-19T09:02:39.845934+0000","last_became_active":"2024-09-19T09:01:52.755263+0000","last_became_peered":"2024-09-19T09:01:52.755263+0000","last_unstale":"2024-09-19T09:02:39.845934+0000","last_undegraded":"2024-09-19T09:02:39.845934+0000","last_fullsized":"2024-09-19T09:02:39.845934+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:16.968329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70524,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1067,"num_read_kb":609,"num_write":261,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1366,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":220285,"data_compressed":4815,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":212166,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612644,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":429375,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841577,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":1082243,"data_compressed":43295,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972399,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":428051,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201331,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":427847,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430265,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":1020418,"data_compressed":41690,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593791,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":427768,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822723,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":1020542,"data_compressed":41690,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986249,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":428163,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215181,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":427593,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":497064,"data_compressed":7388,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":966656,"data_stored":496367,"data_compressed":7388,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":429500,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":69438,"data_compressed":1605,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":70135,"data_compressed":1605,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65137,"data_compressed":1605,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:02:45.379 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T09:02:45.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.513+0000 7f37c50c9640 1 -- 172.21.15.120:0/1972895740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37c0101b30 msgr2=0x7f37c0101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:45.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.513+0000 7f37c50c9640 1 --2- 172.21.15.120:0/1972895740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37c0101b30 0x7f37c0101f30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009900 tx=0x7f37b002a410 comp rx=0 tx=0).stop 2024-09-19T09:02:45.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.513+0000 7f37c50c9640 1 -- 172.21.15.120:0/1972895740 shutdown_connections 2024-09-19T09:02:45.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.513+0000 7f37c50c9640 1 --2- 172.21.15.120:0/1972895740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37c01036e0 0x7f37c0109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:45.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.513+0000 7f37c50c9640 1 --2- 172.21.15.120:0/1972895740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37c0102d20 0x7f37c01031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:45.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.513+0000 7f37c50c9640 1 --2- 172.21.15.120:0/1972895740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37c0101b30 0x7f37c0101f30 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:45.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.513+0000 7f37c50c9640 1 -- 172.21.15.120:0/1972895740 >> 172.21.15.120:0/1972895740 conn(0x7f37c00fd2b0 msgr2=0x7f37c00ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:45.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.513+0000 7f37c50c9640 1 -- 172.21.15.120:0/1972895740 shutdown_connections 2024-09-19T09:02:45.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.513+0000 7f37c50c9640 1 -- 172.21.15.120:0/1972895740 wait complete. 2024-09-19T09:02:45.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.514+0000 7f37c50c9640 1 Processor -- start 2024-09-19T09:02:45.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.514+0000 7f37c50c9640 1 -- start start 2024-09-19T09:02:45.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.514+0000 7f37c50c9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37c0101b30 0x7f37c0197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:45.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.514+0000 7f37c50c9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37c0102d20 0x7f37c01983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:45.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.514+0000 7f37c50c9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37c01036e0 0x7f37c019f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:45.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.514+0000 7f37c50c9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f37c010d7b0 con 0x7f37c01036e0 2024-09-19T09:02:45.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.514+0000 7f37c50c9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f37c010d630 con 0x7f37c0102d20 2024-09-19T09:02:45.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.514+0000 7f37c50c9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37c010d930 con 0x7f37c0101b30 2024-09-19T09:02:45.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.514+0000 7f37bf59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37c01036e0 0x7f37c019f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:45.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.514+0000 7f37bf59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37c01036e0 0x7f37c019f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48460/0 (socket says 172.21.15.120:48460) 2024-09-19T09:02:45.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.514+0000 7f37bf59e640 1 -- 172.21.15.120:0/3750100380 learned_addr learned my addr 172.21.15.120:0/3750100380 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:45.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.514+0000 7f37be59c640 1 --2- 172.21.15.120:0/3750100380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37c0102d20 0x7f37c01983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:45.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.514+0000 7f37bed9d640 1 --2- 172.21.15.120:0/3750100380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37c0101b30 0x7f37c0197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:45.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.514+0000 7f37bf59e640 1 -- 172.21.15.120:0/3750100380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37c0101b30 msgr2=0x7f37c0197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:45.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.514+0000 7f37bf59e640 1 --2- 172.21.15.120:0/3750100380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37c0101b30 0x7f37c0197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:45.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.514+0000 7f37bf59e640 1 -- 172.21.15.120:0/3750100380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37c0102d20 msgr2=0x7f37c01983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:45.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.514+0000 7f37bf59e640 1 --2- 172.21.15.120:0/3750100380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37c0102d20 0x7f37c01983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:45.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.514+0000 7f37bf59e640 1 -- 172.21.15.120:0/3750100380 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c019fa80 con 0x7f37c01036e0 2024-09-19T09:02:45.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.515+0000 7f37bf59e640 1 --2- 172.21.15.120:0/3750100380 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37c01036e0 0x7f37c019f410 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f37b800eaa0 tx=0x7f37b800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:45.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.515+0000 7f37a7fff640 1 -- 172.21.15.120:0/3750100380 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b800cdb0 con 0x7f37c01036e0 2024-09-19T09:02:45.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.515+0000 7f37a7fff640 1 -- 172.21.15.120:0/3750100380 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37b8005020 con 0x7f37c01036e0 2024-09-19T09:02:45.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.515+0000 7f37c50c9640 1 -- 172.21.15.120:0/3750100380 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c019fd70 con 0x7f37c01036e0 2024-09-19T09:02:45.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.515+0000 7f37a7fff640 1 -- 172.21.15.120:0/3750100380 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b80116b0 con 0x7f37c01036e0 2024-09-19T09:02:45.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.515+0000 7f37c50c9640 1 -- 172.21.15.120:0/3750100380 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c01a0260 con 0x7f37c01036e0 2024-09-19T09:02:45.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.516+0000 7f37c50c9640 1 -- 172.21.15.120:0/3750100380 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37c0108fc0 con 0x7f37c01036e0 2024-09-19T09:02:45.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.517+0000 7f37a7fff640 1 -- 172.21.15.120:0/3750100380 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f37b8005560 con 0x7f37c01036e0 2024-09-19T09:02:45.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.517+0000 7f37a7fff640 1 --2- 172.21.15.120:0/3750100380 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f37a80a32e0 0x7f37a80a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:45.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.517+0000 7f37bed9d640 1 --2- 172.21.15.120:0/3750100380 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f37a80a32e0 0x7f37a80a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:45.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.518+0000 7f37a7fff640 1 -- 172.21.15.120:0/3750100380 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f37b80cfda0 con 0x7f37c01036e0 2024-09-19T09:02:45.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.518+0000 7f37bed9d640 1 --2- 172.21.15.120:0/3750100380 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f37a80a32e0 0x7f37a80a57a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f37b0031610 tx=0x7f37b0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:45.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.521+0000 7f37a7fff640 1 -- 172.21.15.120:0/3750100380 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f37b8016030 con 0x7f37c01036e0 2024-09-19T09:02:45.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.685+0000 7f37c50c9640 1 -- 172.21.15.120:0/3750100380 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f37c0062630 con 0x7f37a80a32e0 2024-09-19T09:02:45.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.691+0000 7f37a7fff640 1 -- 172.21.15.120:0/3750100380 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259138 (secure 0 0 0) 0x7f37c0062630 con 0x7f37a80a32e0 2024-09-19T09:02:45.694 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:02:45.695 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-19T09:02:45.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.694+0000 7f37c50c9640 1 -- 172.21.15.120:0/3750100380 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f37a80a32e0 msgr2=0x7f37a80a57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:45.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.694+0000 7f37c50c9640 1 --2- 172.21.15.120:0/3750100380 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f37a80a32e0 0x7f37a80a57a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f37b0031610 tx=0x7f37b0000bc0 comp rx=0 tx=0).stop 2024-09-19T09:02:45.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.694+0000 7f37c50c9640 1 -- 172.21.15.120:0/3750100380 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37c01036e0 msgr2=0x7f37c019f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:45.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.694+0000 7f37c50c9640 1 --2- 172.21.15.120:0/3750100380 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37c01036e0 0x7f37c019f410 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f37b800eaa0 tx=0x7f37b800ef60 comp rx=0 tx=0).stop 2024-09-19T09:02:45.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.695+0000 7f37c50c9640 1 -- 172.21.15.120:0/3750100380 shutdown_connections 2024-09-19T09:02:45.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.695+0000 7f37c50c9640 1 --2- 172.21.15.120:0/3750100380 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f37a80a32e0 0x7f37a80a57a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:45.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.695+0000 7f37c50c9640 1 --2- 172.21.15.120:0/3750100380 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37c01036e0 0x7f37c019f410 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:45.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.695+0000 7f37c50c9640 1 --2- 172.21.15.120:0/3750100380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37c0102d20 0x7f37c01983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:45.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.695+0000 7f37c50c9640 1 --2- 172.21.15.120:0/3750100380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37c0101b30 0x7f37c0197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:45.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.695+0000 7f37c50c9640 1 -- 172.21.15.120:0/3750100380 >> 172.21.15.120:0/3750100380 conn(0x7f37c00fd2b0 msgr2=0x7f37c0106190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:45.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.695+0000 7f37c50c9640 1 -- 172.21.15.120:0/3750100380 shutdown_connections 2024-09-19T09:02:45.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.695+0000 7f37c50c9640 1 -- 172.21.15.120:0/3750100380 wait complete. 2024-09-19T09:02:45.705 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-09-19T09:02:45.636047+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661126,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1198,"num_read_kb":718,"num_write":444,"num_write_kb":3089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1182102,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364360,"kb_used_data":10952,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577464,"statfs":{"total":1151940427776,"available":1151567323136,"internally_reserved":0,"allocated":11214848,"data_stored":7115239,"data_compressed":181932,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1009,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":52,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003143"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624417+0000","last_change":"2024-09-19T09:00:46.966628+0000","last_active":"2024-09-19T09:02:21.624417+0000","last_peered":"2024-09-19T09:02:21.624417+0000","last_clean":"2024-09-19T09:02:21.624417+0000","last_became_active":"2024-09-19T09:00:46.966290+0000","last_became_peered":"2024-09-19T09:00:46.966290+0000","last_unstale":"2024-09-19T09:02:21.624417+0000","last_undegraded":"2024-09-19T09:02:21.624417+0000","last_fullsized":"2024-09-19T09:02:21.624417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:42.717139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621317+0000","last_change":"2024-09-19T09:00:46.963706+0000","last_active":"2024-09-19T09:02:21.621317+0000","last_peered":"2024-09-19T09:02:21.621317+0000","last_clean":"2024-09-19T09:02:21.621317+0000","last_became_active":"2024-09-19T09:00:46.963303+0000","last_became_peered":"2024-09-19T09:00:46.963303+0000","last_unstale":"2024-09-19T09:02:21.621317+0000","last_undegraded":"2024-09-19T09:02:21.621317+0000","last_fullsized":"2024-09-19T09:02:21.621317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:07.451500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620146+0000","last_change":"2024-09-19T09:00:46.967149+0000","last_active":"2024-09-19T09:02:21.620146+0000","last_peered":"2024-09-19T09:02:21.620146+0000","last_clean":"2024-09-19T09:02:21.620146+0000","last_became_active":"2024-09-19T09:00:46.966861+0000","last_became_peered":"2024-09-19T09:00:46.966861+0000","last_unstale":"2024-09-19T09:02:21.620146+0000","last_undegraded":"2024-09-19T09:02:21.620146+0000","last_fullsized":"2024-09-19T09:02:21.620146+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:28.396022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.603363+0000","last_change":"2024-09-19T09:00:46.969242+0000","last_active":"2024-09-19T09:02:21.603363+0000","last_peered":"2024-09-19T09:02:21.603363+0000","last_clean":"2024-09-19T09:02:21.603363+0000","last_became_active":"2024-09-19T09:00:46.968546+0000","last_became_peered":"2024-09-19T09:00:46.968546+0000","last_unstale":"2024-09-19T09:02:21.603363+0000","last_undegraded":"2024-09-19T09:02:21.603363+0000","last_fullsized":"2024-09-19T09:02:21.603363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:43:51.060130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.602962+0000","last_change":"2024-09-19T09:00:46.955501+0000","last_active":"2024-09-19T09:02:21.602962+0000","last_peered":"2024-09-19T09:02:21.602962+0000","last_clean":"2024-09-19T09:02:21.602962+0000","last_became_active":"2024-09-19T09:00:46.955161+0000","last_became_peered":"2024-09-19T09:00:46.955161+0000","last_unstale":"2024-09-19T09:02:21.602962+0000","last_undegraded":"2024-09-19T09:02:21.602962+0000","last_fullsized":"2024-09-19T09:02:21.602962+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:39:19.005931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624332+0000","last_change":"2024-09-19T09:00:46.962577+0000","last_active":"2024-09-19T09:02:21.624332+0000","last_peered":"2024-09-19T09:02:21.624332+0000","last_clean":"2024-09-19T09:02:21.624332+0000","last_became_active":"2024-09-19T09:00:46.962253+0000","last_became_peered":"2024-09-19T09:00:46.962253+0000","last_unstale":"2024-09-19T09:02:21.624332+0000","last_undegraded":"2024-09-19T09:02:21.624332+0000","last_fullsized":"2024-09-19T09:02:21.624332+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:27.925932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621157+0000","last_change":"2024-09-19T09:00:46.960004+0000","last_active":"2024-09-19T09:02:21.621157+0000","last_peered":"2024-09-19T09:02:21.621157+0000","last_clean":"2024-09-19T09:02:21.621157+0000","last_became_active":"2024-09-19T09:00:46.959664+0000","last_became_peered":"2024-09-19T09:00:46.959664+0000","last_unstale":"2024-09-19T09:02:21.621157+0000","last_undegraded":"2024-09-19T09:02:21.621157+0000","last_fullsized":"2024-09-19T09:02:21.621157+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:41.616309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622980+0000","last_change":"2024-09-19T09:00:46.952394+0000","last_active":"2024-09-19T09:02:21.622980+0000","last_peered":"2024-09-19T09:02:21.622980+0000","last_clean":"2024-09-19T09:02:21.622980+0000","last_became_active":"2024-09-19T09:00:46.952084+0000","last_became_peered":"2024-09-19T09:00:46.952084+0000","last_unstale":"2024-09-19T09:02:21.622980+0000","last_undegraded":"2024-09-19T09:02:21.622980+0000","last_fullsized":"2024-09-19T09:02:21.622980+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:21.698423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.632916+0000","last_change":"2024-09-19T09:00:46.961262+0000","last_active":"2024-09-19T09:02:21.632916+0000","last_peered":"2024-09-19T09:02:21.632916+0000","last_clean":"2024-09-19T09:02:21.632916+0000","last_became_active":"2024-09-19T09:00:46.960951+0000","last_became_peered":"2024-09-19T09:00:46.960951+0000","last_unstale":"2024-09-19T09:02:21.632916+0000","last_undegraded":"2024-09-19T09:02:21.632916+0000","last_fullsized":"2024-09-19T09:02:21.632916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:02.445108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.625661+0000","last_change":"2024-09-19T09:01:52.758459+0000","last_active":"2024-09-19T09:02:21.625661+0000","last_peered":"2024-09-19T09:02:21.625661+0000","last_clean":"2024-09-19T09:02:21.625661+0000","last_became_active":"2024-09-19T09:01:52.758073+0000","last_became_peered":"2024-09-19T09:01:52.758073+0000","last_unstale":"2024-09-19T09:02:21.625661+0000","last_undegraded":"2024-09-19T09:02:21.625661+0000","last_fullsized":"2024-09-19T09:02:21.625661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:46:32.983316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.634093+0000","last_change":"2024-09-19T09:00:46.963200+0000","last_active":"2024-09-19T09:02:21.634093+0000","last_peered":"2024-09-19T09:02:21.634093+0000","last_clean":"2024-09-19T09:02:21.634093+0000","last_became_active":"2024-09-19T09:00:46.960496+0000","last_became_peered":"2024-09-19T09:00:46.960496+0000","last_unstale":"2024-09-19T09:02:21.634093+0000","last_undegraded":"2024-09-19T09:02:21.634093+0000","last_fullsized":"2024-09-19T09:02:21.634093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:59:47.234492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.618494+0000","last_change":"2024-09-19T09:00:46.966309+0000","last_active":"2024-09-19T09:02:21.618494+0000","last_peered":"2024-09-19T09:02:21.618494+0000","last_clean":"2024-09-19T09:02:21.618494+0000","last_became_active":"2024-09-19T09:00:46.965970+0000","last_became_peered":"2024-09-19T09:00:46.965970+0000","last_unstale":"2024-09-19T09:02:21.618494+0000","last_undegraded":"2024-09-19T09:02:21.618494+0000","last_fullsized":"2024-09-19T09:02:21.618494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:38.353206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622484+0000","last_change":"2024-09-19T09:00:46.954079+0000","last_active":"2024-09-19T09:02:21.622484+0000","last_peered":"2024-09-19T09:02:21.622484+0000","last_clean":"2024-09-19T09:02:21.622484+0000","last_became_active":"2024-09-19T09:00:46.953398+0000","last_became_peered":"2024-09-19T09:00:46.953398+0000","last_unstale":"2024-09-19T09:02:21.622484+0000","last_undegraded":"2024-09-19T09:02:21.622484+0000","last_fullsized":"2024-09-19T09:02:21.622484+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:54:27.554800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621192+0000","last_change":"2024-09-19T09:00:46.965347+0000","last_active":"2024-09-19T09:02:21.621192+0000","last_peered":"2024-09-19T09:02:21.621192+0000","last_clean":"2024-09-19T09:02:21.621192+0000","last_became_active":"2024-09-19T09:00:46.965045+0000","last_became_peered":"2024-09-19T09:00:46.965045+0000","last_unstale":"2024-09-19T09:02:21.621192+0000","last_undegraded":"2024-09-19T09:02:21.621192+0000","last_fullsized":"2024-09-19T09:02:21.621192+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:27.316100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620943+0000","last_change":"2024-09-19T09:00:46.962898+0000","last_active":"2024-09-19T09:02:21.620943+0000","last_peered":"2024-09-19T09:02:21.620943+0000","last_clean":"2024-09-19T09:02:21.620943+0000","last_became_active":"2024-09-19T09:00:46.962412+0000","last_became_peered":"2024-09-19T09:00:46.962412+0000","last_unstale":"2024-09-19T09:02:21.620943+0000","last_undegraded":"2024-09-19T09:02:21.620943+0000","last_fullsized":"2024-09-19T09:02:21.620943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:40.483678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.627060+0000","last_change":"2024-09-19T09:00:46.958147+0000","last_active":"2024-09-19T09:02:21.627060+0000","last_peered":"2024-09-19T09:02:21.627060+0000","last_clean":"2024-09-19T09:02:21.627060+0000","last_became_active":"2024-09-19T09:00:46.957805+0000","last_became_peered":"2024-09-19T09:00:46.957805+0000","last_unstale":"2024-09-19T09:02:21.627060+0000","last_undegraded":"2024-09-19T09:02:21.627060+0000","last_fullsized":"2024-09-19T09:02:21.627060+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:24:17.000064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624642+0000","last_change":"2024-09-19T09:00:46.967399+0000","last_active":"2024-09-19T09:02:21.624642+0000","last_peered":"2024-09-19T09:02:21.624642+0000","last_clean":"2024-09-19T09:02:21.624642+0000","last_became_active":"2024-09-19T09:00:46.966145+0000","last_became_peered":"2024-09-19T09:00:46.966145+0000","last_unstale":"2024-09-19T09:02:21.624642+0000","last_undegraded":"2024-09-19T09:02:21.624642+0000","last_fullsized":"2024-09-19T09:02:21.624642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:29.192469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.614692+0000","last_change":"2024-09-19T09:00:46.960956+0000","last_active":"2024-09-19T09:02:21.614692+0000","last_peered":"2024-09-19T09:02:21.614692+0000","last_clean":"2024-09-19T09:02:21.614692+0000","last_became_active":"2024-09-19T09:00:46.960664+0000","last_became_peered":"2024-09-19T09:00:46.960664+0000","last_unstale":"2024-09-19T09:02:21.614692+0000","last_undegraded":"2024-09-19T09:02:21.614692+0000","last_fullsized":"2024-09-19T09:02:21.614692+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:18.724348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622878+0000","last_change":"2024-09-19T09:00:46.951362+0000","last_active":"2024-09-19T09:02:21.622878+0000","last_peered":"2024-09-19T09:02:21.622878+0000","last_clean":"2024-09-19T09:02:21.622878+0000","last_became_active":"2024-09-19T09:00:46.951004+0000","last_became_peered":"2024-09-19T09:00:46.951004+0000","last_unstale":"2024-09-19T09:02:21.622878+0000","last_undegraded":"2024-09-19T09:02:21.622878+0000","last_fullsized":"2024-09-19T09:02:21.622878+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:43.574425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.623346+0000","last_change":"2024-09-19T09:00:46.966752+0000","last_active":"2024-09-19T09:02:21.623346+0000","last_peered":"2024-09-19T09:02:21.623346+0000","last_clean":"2024-09-19T09:02:21.623346+0000","last_became_active":"2024-09-19T09:00:46.966370+0000","last_became_peered":"2024-09-19T09:00:46.966370+0000","last_unstale":"2024-09-19T09:02:21.623346+0000","last_undegraded":"2024-09-19T09:02:21.623346+0000","last_fullsized":"2024-09-19T09:02:21.623346+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:04.035771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621434+0000","last_change":"2024-09-19T09:00:46.956369+0000","last_active":"2024-09-19T09:02:21.621434+0000","last_peered":"2024-09-19T09:02:21.621434+0000","last_clean":"2024-09-19T09:02:21.621434+0000","last_became_active":"2024-09-19T09:00:46.956075+0000","last_became_peered":"2024-09-19T09:00:46.956075+0000","last_unstale":"2024-09-19T09:02:21.621434+0000","last_undegraded":"2024-09-19T09:02:21.621434+0000","last_fullsized":"2024-09-19T09:02:21.621434+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:23.985017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633789+0000","last_change":"2024-09-19T09:00:46.956661+0000","last_active":"2024-09-19T09:02:21.633789+0000","last_peered":"2024-09-19T09:02:21.633789+0000","last_clean":"2024-09-19T09:02:21.633789+0000","last_became_active":"2024-09-19T09:00:46.956291+0000","last_became_peered":"2024-09-19T09:00:46.956291+0000","last_unstale":"2024-09-19T09:02:21.633789+0000","last_undegraded":"2024-09-19T09:02:21.633789+0000","last_fullsized":"2024-09-19T09:02:21.633789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:55.713039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.616782+0000","last_change":"2024-09-19T09:00:46.961866+0000","last_active":"2024-09-19T09:02:21.616782+0000","last_peered":"2024-09-19T09:02:21.616782+0000","last_clean":"2024-09-19T09:02:21.616782+0000","last_became_active":"2024-09-19T09:00:46.961469+0000","last_became_peered":"2024-09-19T09:00:46.961469+0000","last_unstale":"2024-09-19T09:02:21.616782+0000","last_undegraded":"2024-09-19T09:02:21.616782+0000","last_fullsized":"2024-09-19T09:02:21.616782+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:47.665419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622833+0000","last_change":"2024-09-19T09:00:46.948765+0000","last_active":"2024-09-19T09:02:21.622833+0000","last_peered":"2024-09-19T09:02:21.622833+0000","last_clean":"2024-09-19T09:02:21.622833+0000","last_became_active":"2024-09-19T09:00:46.948431+0000","last_became_peered":"2024-09-19T09:00:46.948431+0000","last_unstale":"2024-09-19T09:02:21.622833+0000","last_undegraded":"2024-09-19T09:02:21.622833+0000","last_fullsized":"2024-09-19T09:02:21.622833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:31.766464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.618165+0000","last_change":"2024-09-19T09:00:46.962433+0000","last_active":"2024-09-19T09:02:21.618165+0000","last_peered":"2024-09-19T09:02:21.618165+0000","last_clean":"2024-09-19T09:02:21.618165+0000","last_became_active":"2024-09-19T09:00:46.962153+0000","last_became_peered":"2024-09-19T09:00:46.962153+0000","last_unstale":"2024-09-19T09:02:21.618165+0000","last_undegraded":"2024-09-19T09:02:21.618165+0000","last_fullsized":"2024-09-19T09:02:21.618165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:37.973611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.623402+0000","last_change":"2024-09-19T09:00:46.953715+0000","last_active":"2024-09-19T09:02:21.623402+0000","last_peered":"2024-09-19T09:02:21.623402+0000","last_clean":"2024-09-19T09:02:21.623402+0000","last_became_active":"2024-09-19T09:00:46.953415+0000","last_became_peered":"2024-09-19T09:00:46.953415+0000","last_unstale":"2024-09-19T09:02:21.623402+0000","last_undegraded":"2024-09-19T09:02:21.623402+0000","last_fullsized":"2024-09-19T09:02:21.623402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:29.768382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.615199+0000","last_change":"2024-09-19T09:00:46.964746+0000","last_active":"2024-09-19T09:02:21.615199+0000","last_peered":"2024-09-19T09:02:21.615199+0000","last_clean":"2024-09-19T09:02:21.615199+0000","last_became_active":"2024-09-19T09:00:46.964404+0000","last_became_peered":"2024-09-19T09:00:46.964404+0000","last_unstale":"2024-09-19T09:02:21.615199+0000","last_undegraded":"2024-09-19T09:02:21.615199+0000","last_fullsized":"2024-09-19T09:02:21.615199+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:19.841495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.619156+0000","last_change":"2024-09-19T09:00:46.962875+0000","last_active":"2024-09-19T09:02:21.619156+0000","last_peered":"2024-09-19T09:02:21.619156+0000","last_clean":"2024-09-19T09:02:21.619156+0000","last_became_active":"2024-09-19T09:00:46.962561+0000","last_became_peered":"2024-09-19T09:00:46.962561+0000","last_unstale":"2024-09-19T09:02:21.619156+0000","last_undegraded":"2024-09-19T09:02:21.619156+0000","last_fullsized":"2024-09-19T09:02:21.619156+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:39.675545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.612887+0000","last_change":"2024-09-19T09:00:46.963126+0000","last_active":"2024-09-19T09:02:21.612887+0000","last_peered":"2024-09-19T09:02:21.612887+0000","last_clean":"2024-09-19T09:02:21.612887+0000","last_became_active":"2024-09-19T09:00:46.962620+0000","last_became_peered":"2024-09-19T09:00:46.962620+0000","last_unstale":"2024-09-19T09:02:21.612887+0000","last_undegraded":"2024-09-19T09:02:21.612887+0000","last_fullsized":"2024-09-19T09:02:21.612887+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:59.723280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620254+0000","last_change":"2024-09-19T09:00:46.953026+0000","last_active":"2024-09-19T09:02:21.620254+0000","last_peered":"2024-09-19T09:02:21.620254+0000","last_clean":"2024-09-19T09:02:21.620254+0000","last_became_active":"2024-09-19T09:00:46.952596+0000","last_became_peered":"2024-09-19T09:00:46.952596+0000","last_unstale":"2024-09-19T09:02:21.620254+0000","last_undegraded":"2024-09-19T09:02:21.620254+0000","last_fullsized":"2024-09-19T09:02:21.620254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:58.524116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624169+0000","last_change":"2024-09-19T09:00:46.968964+0000","last_active":"2024-09-19T09:02:21.624169+0000","last_peered":"2024-09-19T09:02:21.624169+0000","last_clean":"2024-09-19T09:02:21.624169+0000","last_became_active":"2024-09-19T09:00:46.968505+0000","last_became_peered":"2024-09-19T09:00:46.968505+0000","last_unstale":"2024-09-19T09:02:21.624169+0000","last_undegraded":"2024-09-19T09:02:21.624169+0000","last_fullsized":"2024-09-19T09:02:21.624169+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:17:33.335541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.625205+0000","last_change":"2024-09-19T09:00:46.961520+0000","last_active":"2024-09-19T09:02:21.625205+0000","last_peered":"2024-09-19T09:02:21.625205+0000","last_clean":"2024-09-19T09:02:21.625205+0000","last_became_active":"2024-09-19T09:00:46.961140+0000","last_became_peered":"2024-09-19T09:00:46.961140+0000","last_unstale":"2024-09-19T09:02:21.625205+0000","last_undegraded":"2024-09-19T09:02:21.625205+0000","last_fullsized":"2024-09-19T09:02:21.625205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:03.120173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621084+0000","last_change":"2024-09-19T09:00:46.955995+0000","last_active":"2024-09-19T09:02:21.621084+0000","last_peered":"2024-09-19T09:02:21.621084+0000","last_clean":"2024-09-19T09:02:21.621084+0000","last_became_active":"2024-09-19T09:00:46.954678+0000","last_became_peered":"2024-09-19T09:00:46.954678+0000","last_unstale":"2024-09-19T09:02:21.621084+0000","last_undegraded":"2024-09-19T09:02:21.621084+0000","last_fullsized":"2024-09-19T09:02:21.621084+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:10.314575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":135,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:41.284935+0000","last_change":"2024-09-19T09:00:44.946000+0000","last_active":"2024-09-19T09:02:41.284935+0000","last_peered":"2024-09-19T09:02:41.284935+0000","last_clean":"2024-09-19T09:02:41.284935+0000","last_became_active":"2024-09-19T09:00:44.945583+0000","last_became_peered":"2024-09-19T09:00:44.945583+0000","last_unstale":"2024-09-19T09:02:41.284935+0000","last_undegraded":"2024-09-19T09:02:41.284935+0000","last_fullsized":"2024-09-19T09:02:41.284935+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:53:19.109697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624079+0000","last_change":"2024-09-19T09:01:52.904443+0000","last_active":"2024-09-19T09:02:21.624079+0000","last_peered":"2024-09-19T09:02:21.624079+0000","last_clean":"2024-09-19T09:02:21.624079+0000","last_became_active":"2024-09-19T09:01:52.904077+0000","last_became_peered":"2024-09-19T09:01:52.904077+0000","last_unstale":"2024-09-19T09:02:21.624079+0000","last_undegraded":"2024-09-19T09:02:21.624079+0000","last_fullsized":"2024-09-19T09:02:21.624079+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:50.467090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.748381+0000","last_change":"2024-09-19T09:00:44.942435+0000","last_active":"2024-09-19T09:01:52.748381+0000","last_peered":"2024-09-19T09:01:52.748381+0000","last_clean":"2024-09-19T09:01:52.748381+0000","last_became_active":"2024-09-19T09:00:44.942067+0000","last_became_peered":"2024-09-19T09:00:44.942067+0000","last_unstale":"2024-09-19T09:01:52.748381+0000","last_undegraded":"2024-09-19T09:01:52.748381+0000","last_fullsized":"2024-09-19T09:01:52.748381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:16.476506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.747900+0000","last_change":"2024-09-19T09:00:44.937944+0000","last_active":"2024-09-19T09:01:52.747900+0000","last_peered":"2024-09-19T09:01:52.747900+0000","last_clean":"2024-09-19T09:01:52.747900+0000","last_became_active":"2024-09-19T09:00:44.937623+0000","last_became_peered":"2024-09-19T09:00:44.937623+0000","last_unstale":"2024-09-19T09:01:52.747900+0000","last_undegraded":"2024-09-19T09:01:52.747900+0000","last_fullsized":"2024-09-19T09:01:52.747900+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:49:08.730015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620184+0000","last_change":"2024-09-19T09:00:46.959932+0000","last_active":"2024-09-19T09:02:21.620184+0000","last_peered":"2024-09-19T09:02:21.620184+0000","last_clean":"2024-09-19T09:02:21.620184+0000","last_became_active":"2024-09-19T09:00:46.959637+0000","last_became_peered":"2024-09-19T09:00:46.959637+0000","last_unstale":"2024-09-19T09:02:21.620184+0000","last_undegraded":"2024-09-19T09:02:21.620184+0000","last_fullsized":"2024-09-19T09:02:21.620184+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:55.337669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751513+0000","last_change":"2024-09-19T09:00:44.944882+0000","last_active":"2024-09-19T09:01:52.751513+0000","last_peered":"2024-09-19T09:01:52.751513+0000","last_clean":"2024-09-19T09:01:52.751513+0000","last_became_active":"2024-09-19T09:00:44.944116+0000","last_became_peered":"2024-09-19T09:00:44.944116+0000","last_unstale":"2024-09-19T09:01:52.751513+0000","last_undegraded":"2024-09-19T09:01:52.751513+0000","last_fullsized":"2024-09-19T09:01:52.751513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:54:58.555913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.615953+0000","last_change":"2024-09-19T09:00:46.956859+0000","last_active":"2024-09-19T09:02:21.615953+0000","last_peered":"2024-09-19T09:02:21.615953+0000","last_clean":"2024-09-19T09:02:21.615953+0000","last_became_active":"2024-09-19T09:00:46.956561+0000","last_became_peered":"2024-09-19T09:00:46.956561+0000","last_unstale":"2024-09-19T09:02:21.615953+0000","last_undegraded":"2024-09-19T09:02:21.615953+0000","last_fullsized":"2024-09-19T09:02:21.615953+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:09:04.125591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'5","reported_seq":231,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:40.309800+0000","last_change":"2024-09-19T09:00:44.945959+0000","last_active":"2024-09-19T09:02:40.309800+0000","last_peered":"2024-09-19T09:02:40.309800+0000","last_clean":"2024-09-19T09:02:40.309800+0000","last_became_active":"2024-09-19T09:00:44.945640+0000","last_became_peered":"2024-09-19T09:00:44.945640+0000","last_unstale":"2024-09-19T09:02:40.309800+0000","last_undegraded":"2024-09-19T09:02:40.309800+0000","last_fullsized":"2024-09-19T09:02:40.309800+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:02.523834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":204,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.632897+0000","last_change":"2024-09-19T09:00:46.953677+0000","last_active":"2024-09-19T09:02:21.632897+0000","last_peered":"2024-09-19T09:02:21.632897+0000","last_clean":"2024-09-19T09:02:21.632897+0000","last_became_active":"2024-09-19T09:00:46.953357+0000","last_became_peered":"2024-09-19T09:00:46.953357+0000","last_unstale":"2024-09-19T09:02:21.632897+0000","last_undegraded":"2024-09-19T09:02:21.632897+0000","last_fullsized":"2024-09-19T09:02:21.632897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:48.963003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.893433+0000","last_change":"2024-09-19T09:01:52.893433+0000","last_active":"2024-09-19T09:01:52.893433+0000","last_peered":"2024-09-19T09:01:52.893433+0000","last_clean":"2024-09-19T09:01:52.893433+0000","last_became_active":"2024-09-19T09:01:52.889836+0000","last_became_peered":"2024-09-19T09:01:52.889836+0000","last_unstale":"2024-09-19T09:01:52.893433+0000","last_undegraded":"2024-09-19T09:01:52.893433+0000","last_fullsized":"2024-09-19T09:01:52.893433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:08:22.339717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622288+0000","last_change":"2024-09-19T09:00:46.962285+0000","last_active":"2024-09-19T09:02:21.622288+0000","last_peered":"2024-09-19T09:02:21.622288+0000","last_clean":"2024-09-19T09:02:21.622288+0000","last_became_active":"2024-09-19T09:00:46.961982+0000","last_became_peered":"2024-09-19T09:00:46.961982+0000","last_unstale":"2024-09-19T09:02:21.622288+0000","last_undegraded":"2024-09-19T09:02:21.622288+0000","last_fullsized":"2024-09-19T09:02:21.622288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:41.658501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752480+0000","last_change":"2024-09-19T09:00:45.241425+0000","last_active":"2024-09-19T09:01:52.752480+0000","last_peered":"2024-09-19T09:01:52.752480+0000","last_clean":"2024-09-19T09:01:52.752480+0000","last_became_active":"2024-09-19T09:00:45.241032+0000","last_became_peered":"2024-09-19T09:00:45.241032+0000","last_unstale":"2024-09-19T09:01:52.752480+0000","last_undegraded":"2024-09-19T09:01:52.752480+0000","last_fullsized":"2024-09-19T09:01:52.752480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:00.707855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.626097+0000","last_change":"2024-09-19T09:00:46.962436+0000","last_active":"2024-09-19T09:02:21.626097+0000","last_peered":"2024-09-19T09:02:21.626097+0000","last_clean":"2024-09-19T09:02:21.626097+0000","last_became_active":"2024-09-19T09:00:46.962114+0000","last_became_peered":"2024-09-19T09:00:46.962114+0000","last_unstale":"2024-09-19T09:02:21.626097+0000","last_undegraded":"2024-09-19T09:02:21.626097+0000","last_fullsized":"2024-09-19T09:02:21.626097+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:41.761705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'37","reported_seq":273,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:40.544906+0000","last_change":"2024-09-19T09:00:44.945187+0000","last_active":"2024-09-19T09:02:40.544906+0000","last_peered":"2024-09-19T09:02:40.544906+0000","last_clean":"2024-09-19T09:02:40.544906+0000","last_became_active":"2024-09-19T09:00:44.944763+0000","last_became_peered":"2024-09-19T09:00:44.944763+0000","last_unstale":"2024-09-19T09:02:40.544906+0000","last_undegraded":"2024-09-19T09:02:40.544906+0000","last_fullsized":"2024-09-19T09:02:40.544906+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:06.052994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62867,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":214,"num_read_kb":168,"num_write":39,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.630413+0000","last_change":"2024-09-19T09:00:46.960991+0000","last_active":"2024-09-19T09:02:21.630413+0000","last_peered":"2024-09-19T09:02:21.630413+0000","last_clean":"2024-09-19T09:02:21.630413+0000","last_became_active":"2024-09-19T09:00:46.958962+0000","last_became_peered":"2024-09-19T09:00:46.958962+0000","last_unstale":"2024-09-19T09:02:21.630413+0000","last_undegraded":"2024-09-19T09:02:21.630413+0000","last_fullsized":"2024-09-19T09:02:21.630413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:21.795694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.755250+0000","last_change":"2024-09-19T09:00:44.946176+0000","last_active":"2024-09-19T09:01:52.755250+0000","last_peered":"2024-09-19T09:01:52.755250+0000","last_clean":"2024-09-19T09:01:52.755250+0000","last_became_active":"2024-09-19T09:00:44.945869+0000","last_became_peered":"2024-09-19T09:00:44.945869+0000","last_unstale":"2024-09-19T09:01:52.755250+0000","last_undegraded":"2024-09-19T09:01:52.755250+0000","last_fullsized":"2024-09-19T09:01:52.755250+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:16.022184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621664+0000","last_change":"2024-09-19T09:00:46.956345+0000","last_active":"2024-09-19T09:02:21.621664+0000","last_peered":"2024-09-19T09:02:21.621664+0000","last_clean":"2024-09-19T09:02:21.621664+0000","last_became_active":"2024-09-19T09:00:46.956057+0000","last_became_peered":"2024-09-19T09:00:46.956057+0000","last_unstale":"2024-09-19T09:02:21.621664+0000","last_undegraded":"2024-09-19T09:02:21.621664+0000","last_fullsized":"2024-09-19T09:02:21.621664+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:27.510661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752205+0000","last_change":"2024-09-19T09:00:44.945667+0000","last_active":"2024-09-19T09:01:52.752205+0000","last_peered":"2024-09-19T09:01:52.752205+0000","last_clean":"2024-09-19T09:01:52.752205+0000","last_became_active":"2024-09-19T09:00:44.945308+0000","last_became_peered":"2024-09-19T09:00:44.945308+0000","last_unstale":"2024-09-19T09:01:52.752205+0000","last_undegraded":"2024-09-19T09:01:52.752205+0000","last_fullsized":"2024-09-19T09:01:52.752205+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:00:11.304913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.629868+0000","last_change":"2024-09-19T09:00:46.965088+0000","last_active":"2024-09-19T09:02:21.629868+0000","last_peered":"2024-09-19T09:02:21.629868+0000","last_clean":"2024-09-19T09:02:21.629868+0000","last_became_active":"2024-09-19T09:00:46.964777+0000","last_became_peered":"2024-09-19T09:00:46.964777+0000","last_unstale":"2024-09-19T09:02:21.629868+0000","last_undegraded":"2024-09-19T09:02:21.629868+0000","last_fullsized":"2024-09-19T09:02:21.629868+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:32.958984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'7","reported_seq":137,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:41.284063+0000","last_change":"2024-09-19T09:00:44.946193+0000","last_active":"2024-09-19T09:02:41.284063+0000","last_peered":"2024-09-19T09:02:41.284063+0000","last_clean":"2024-09-19T09:02:41.284063+0000","last_became_active":"2024-09-19T09:00:44.945168+0000","last_became_peered":"2024-09-19T09:00:44.945168+0000","last_unstale":"2024-09-19T09:02:41.284063+0000","last_undegraded":"2024-09-19T09:02:41.284063+0000","last_fullsized":"2024-09-19T09:02:41.284063+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:21.230689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.627048+0000","last_change":"2024-09-19T09:01:52.754058+0000","last_active":"2024-09-19T09:02:21.627048+0000","last_peered":"2024-09-19T09:02:21.627048+0000","last_clean":"2024-09-19T09:02:21.627048+0000","last_became_active":"2024-09-19T09:01:52.753674+0000","last_became_peered":"2024-09-19T09:01:52.753674+0000","last_unstale":"2024-09-19T09:02:21.627048+0000","last_undegraded":"2024-09-19T09:02:21.627048+0000","last_fullsized":"2024-09-19T09:02:21.627048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:05.665121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":64,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.774430+0000","last_change":"2024-09-19T09:01:52.774430+0000","last_active":"2024-09-19T09:01:52.774430+0000","last_peered":"2024-09-19T09:01:52.774430+0000","last_clean":"2024-09-19T09:01:52.774430+0000","last_became_active":"2024-09-19T09:01:52.756083+0000","last_became_peered":"2024-09-19T09:01:52.756083+0000","last_unstale":"2024-09-19T09:01:52.774430+0000","last_undegraded":"2024-09-19T09:01:52.774430+0000","last_fullsized":"2024-09-19T09:01:52.774430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:52.639490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633068+0000","last_change":"2024-09-19T09:01:52.757684+0000","last_active":"2024-09-19T09:02:21.633068+0000","last_peered":"2024-09-19T09:02:21.633068+0000","last_clean":"2024-09-19T09:02:21.633068+0000","last_became_active":"2024-09-19T09:01:52.756969+0000","last_became_peered":"2024-09-19T09:01:52.756969+0000","last_unstale":"2024-09-19T09:02:21.633068+0000","last_undegraded":"2024-09-19T09:02:21.633068+0000","last_fullsized":"2024-09-19T09:02:21.633068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:24:47.117996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752185+0000","last_change":"2024-09-19T09:00:44.946138+0000","last_active":"2024-09-19T09:01:52.752185+0000","last_peered":"2024-09-19T09:01:52.752185+0000","last_clean":"2024-09-19T09:01:52.752185+0000","last_became_active":"2024-09-19T09:00:44.945793+0000","last_became_peered":"2024-09-19T09:00:44.945793+0000","last_unstale":"2024-09-19T09:01:52.752185+0000","last_undegraded":"2024-09-19T09:01:52.752185+0000","last_fullsized":"2024-09-19T09:01:52.752185+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:01.341560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620195+0000","last_change":"2024-09-19T09:00:46.970360+0000","last_active":"2024-09-19T09:02:21.620195+0000","last_peered":"2024-09-19T09:02:21.620195+0000","last_clean":"2024-09-19T09:02:21.620195+0000","last_became_active":"2024-09-19T09:00:46.969971+0000","last_became_peered":"2024-09-19T09:00:46.969971+0000","last_unstale":"2024-09-19T09:02:21.620195+0000","last_undegraded":"2024-09-19T09:02:21.620195+0000","last_fullsized":"2024-09-19T09:02:21.620195+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:48.812158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.748305+0000","last_change":"2024-09-19T09:00:44.941713+0000","last_active":"2024-09-19T09:01:52.748305+0000","last_peered":"2024-09-19T09:01:52.748305+0000","last_clean":"2024-09-19T09:01:52.748305+0000","last_became_active":"2024-09-19T09:00:44.941373+0000","last_became_peered":"2024-09-19T09:00:44.941373+0000","last_unstale":"2024-09-19T09:01:52.748305+0000","last_undegraded":"2024-09-19T09:01:52.748305+0000","last_fullsized":"2024-09-19T09:01:52.748305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:31.157803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.630100+0000","last_change":"2024-09-19T09:00:46.959383+0000","last_active":"2024-09-19T09:02:21.630100+0000","last_peered":"2024-09-19T09:02:21.630100+0000","last_clean":"2024-09-19T09:02:21.630100+0000","last_became_active":"2024-09-19T09:00:46.958860+0000","last_became_peered":"2024-09-19T09:00:46.958860+0000","last_unstale":"2024-09-19T09:02:21.630100+0000","last_undegraded":"2024-09-19T09:02:21.630100+0000","last_fullsized":"2024-09-19T09:02:21.630100+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:32:04.423017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752778+0000","last_change":"2024-09-19T09:00:44.944797+0000","last_active":"2024-09-19T09:01:52.752778+0000","last_peered":"2024-09-19T09:01:52.752778+0000","last_clean":"2024-09-19T09:01:52.752778+0000","last_became_active":"2024-09-19T09:00:44.944259+0000","last_became_peered":"2024-09-19T09:00:44.944259+0000","last_unstale":"2024-09-19T09:01:52.752778+0000","last_undegraded":"2024-09-19T09:01:52.752778+0000","last_fullsized":"2024-09-19T09:01:52.752778+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:58.996217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624674+0000","last_change":"2024-09-19T09:00:46.964998+0000","last_active":"2024-09-19T09:02:21.624674+0000","last_peered":"2024-09-19T09:02:21.624674+0000","last_clean":"2024-09-19T09:02:21.624674+0000","last_became_active":"2024-09-19T09:00:46.964562+0000","last_became_peered":"2024-09-19T09:00:46.964562+0000","last_unstale":"2024-09-19T09:02:21.624674+0000","last_undegraded":"2024-09-19T09:02:21.624674+0000","last_fullsized":"2024-09-19T09:02:21.624674+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:00.089800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":135,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:42.313326+0000","last_change":"2024-09-19T09:00:45.239770+0000","last_active":"2024-09-19T09:02:42.313326+0000","last_peered":"2024-09-19T09:02:42.313326+0000","last_clean":"2024-09-19T09:02:42.313326+0000","last_became_active":"2024-09-19T09:00:45.239385+0000","last_became_peered":"2024-09-19T09:00:45.239385+0000","last_unstale":"2024-09-19T09:02:42.313326+0000","last_undegraded":"2024-09-19T09:02:42.313326+0000","last_fullsized":"2024-09-19T09:02:42.313326+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:21.109900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.634073+0000","last_change":"2024-09-19T09:00:46.967505+0000","last_active":"2024-09-19T09:02:21.634073+0000","last_peered":"2024-09-19T09:02:21.634073+0000","last_clean":"2024-09-19T09:02:21.634073+0000","last_became_active":"2024-09-19T09:00:46.967195+0000","last_became_peered":"2024-09-19T09:00:46.967195+0000","last_unstale":"2024-09-19T09:02:21.634073+0000","last_undegraded":"2024-09-19T09:02:21.634073+0000","last_fullsized":"2024-09-19T09:02:21.634073+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:52:28.917333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":335,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751112+0000","last_change":"2024-09-19T08:59:33.821924+0000","last_active":"2024-09-19T09:01:52.751112+0000","last_peered":"2024-09-19T09:01:52.751112+0000","last_clean":"2024-09-19T09:01:52.751112+0000","last_became_active":"2024-09-19T08:59:33.821404+0000","last_became_peered":"2024-09-19T08:59:33.821404+0000","last_unstale":"2024-09-19T09:01:52.751112+0000","last_undegraded":"2024-09-19T09:01:52.751112+0000","last_fullsized":"2024-09-19T09:01:52.751112+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:56:07.830480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:56:07.830480+0000","last_clean_scrub_stamp":"2024-09-19T08:56:07.830480+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:21.763918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.888480+0000","last_change":"2024-09-19T09:00:44.943714+0000","last_active":"2024-09-19T09:01:52.888480+0000","last_peered":"2024-09-19T09:01:52.888480+0000","last_clean":"2024-09-19T09:01:52.888480+0000","last_became_active":"2024-09-19T09:00:44.943362+0000","last_became_peered":"2024-09-19T09:00:44.943362+0000","last_unstale":"2024-09-19T09:01:52.888480+0000","last_undegraded":"2024-09-19T09:01:52.888480+0000","last_fullsized":"2024-09-19T09:01:52.888480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:30.015648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633592+0000","last_change":"2024-09-19T09:00:46.956784+0000","last_active":"2024-09-19T09:02:21.633592+0000","last_peered":"2024-09-19T09:02:21.633592+0000","last_clean":"2024-09-19T09:02:21.633592+0000","last_became_active":"2024-09-19T09:00:46.956443+0000","last_became_peered":"2024-09-19T09:00:46.956443+0000","last_unstale":"2024-09-19T09:02:21.633592+0000","last_undegraded":"2024-09-19T09:02:21.633592+0000","last_fullsized":"2024-09-19T09:02:21.633592+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:18.918788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.750994+0000","last_change":"2024-09-19T09:00:44.947796+0000","last_active":"2024-09-19T09:01:52.750994+0000","last_peered":"2024-09-19T09:01:52.750994+0000","last_clean":"2024-09-19T09:01:52.750994+0000","last_became_active":"2024-09-19T09:00:44.946949+0000","last_became_peered":"2024-09-19T09:00:44.946949+0000","last_unstale":"2024-09-19T09:01:52.750994+0000","last_undegraded":"2024-09-19T09:01:52.750994+0000","last_fullsized":"2024-09-19T09:01:52.750994+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:42.210186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.619434+0000","last_change":"2024-09-19T09:00:46.963856+0000","last_active":"2024-09-19T09:02:21.619434+0000","last_peered":"2024-09-19T09:02:21.619434+0000","last_clean":"2024-09-19T09:02:21.619434+0000","last_became_active":"2024-09-19T09:00:46.963543+0000","last_became_peered":"2024-09-19T09:00:46.963543+0000","last_unstale":"2024-09-19T09:02:21.619434+0000","last_undegraded":"2024-09-19T09:02:21.619434+0000","last_fullsized":"2024-09-19T09:02:21.619434+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:31.041028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'6","reported_seq":134,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:42.284796+0000","last_change":"2024-09-19T09:00:45.239269+0000","last_active":"2024-09-19T09:02:42.284796+0000","last_peered":"2024-09-19T09:02:42.284796+0000","last_clean":"2024-09-19T09:02:42.284796+0000","last_became_active":"2024-09-19T09:00:45.238929+0000","last_became_peered":"2024-09-19T09:00:45.238929+0000","last_unstale":"2024-09-19T09:02:42.284796+0000","last_undegraded":"2024-09-19T09:02:42.284796+0000","last_fullsized":"2024-09-19T09:02:42.284796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:46.684148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622349+0000","last_change":"2024-09-19T09:00:46.963595+0000","last_active":"2024-09-19T09:02:21.622349+0000","last_peered":"2024-09-19T09:02:21.622349+0000","last_clean":"2024-09-19T09:02:21.622349+0000","last_became_active":"2024-09-19T09:00:46.963243+0000","last_became_peered":"2024-09-19T09:00:46.963243+0000","last_unstale":"2024-09-19T09:02:21.622349+0000","last_undegraded":"2024-09-19T09:02:21.622349+0000","last_fullsized":"2024-09-19T09:02:21.622349+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:01.122996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":133,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:42.850379+0000","last_change":"2024-09-19T09:00:44.941550+0000","last_active":"2024-09-19T09:02:42.850379+0000","last_peered":"2024-09-19T09:02:42.850379+0000","last_clean":"2024-09-19T09:02:42.850379+0000","last_became_active":"2024-09-19T09:00:44.941145+0000","last_became_peered":"2024-09-19T09:00:44.941145+0000","last_unstale":"2024-09-19T09:02:42.850379+0000","last_undegraded":"2024-09-19T09:02:42.850379+0000","last_fullsized":"2024-09-19T09:02:42.850379+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:53.893887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.612369+0000","last_change":"2024-09-19T09:00:46.963473+0000","last_active":"2024-09-19T09:02:21.612369+0000","last_peered":"2024-09-19T09:02:21.612369+0000","last_clean":"2024-09-19T09:02:21.612369+0000","last_became_active":"2024-09-19T09:00:46.963090+0000","last_became_peered":"2024-09-19T09:00:46.963090+0000","last_unstale":"2024-09-19T09:02:21.612369+0000","last_undegraded":"2024-09-19T09:02:21.612369+0000","last_fullsized":"2024-09-19T09:02:21.612369+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:01.492385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.750853+0000","last_change":"2024-09-19T09:00:45.239566+0000","last_active":"2024-09-19T09:01:52.750853+0000","last_peered":"2024-09-19T09:01:52.750853+0000","last_clean":"2024-09-19T09:01:52.750853+0000","last_became_active":"2024-09-19T09:00:45.239250+0000","last_became_peered":"2024-09-19T09:00:45.239250+0000","last_unstale":"2024-09-19T09:01:52.750853+0000","last_undegraded":"2024-09-19T09:01:52.750853+0000","last_fullsized":"2024-09-19T09:01:52.750853+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:44:21.739311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.614525+0000","last_change":"2024-09-19T09:00:46.955344+0000","last_active":"2024-09-19T09:02:21.614525+0000","last_peered":"2024-09-19T09:02:21.614525+0000","last_clean":"2024-09-19T09:02:21.614525+0000","last_became_active":"2024-09-19T09:00:46.954961+0000","last_became_peered":"2024-09-19T09:00:46.954961+0000","last_unstale":"2024-09-19T09:02:21.614525+0000","last_undegraded":"2024-09-19T09:02:21.614525+0000","last_fullsized":"2024-09-19T09:02:21.614525+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:55.743556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752072+0000","last_change":"2024-09-19T09:00:45.241514+0000","last_active":"2024-09-19T09:01:52.752072+0000","last_peered":"2024-09-19T09:01:52.752072+0000","last_clean":"2024-09-19T09:01:52.752072+0000","last_became_active":"2024-09-19T09:00:45.241097+0000","last_became_peered":"2024-09-19T09:00:45.241097+0000","last_unstale":"2024-09-19T09:01:52.752072+0000","last_undegraded":"2024-09-19T09:01:52.752072+0000","last_fullsized":"2024-09-19T09:01:52.752072+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:35:38.323393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620312+0000","last_change":"2024-09-19T09:00:46.955194+0000","last_active":"2024-09-19T09:02:21.620312+0000","last_peered":"2024-09-19T09:02:21.620312+0000","last_clean":"2024-09-19T09:02:21.620312+0000","last_became_active":"2024-09-19T09:00:46.954771+0000","last_became_peered":"2024-09-19T09:00:46.954771+0000","last_unstale":"2024-09-19T09:02:21.620312+0000","last_undegraded":"2024-09-19T09:02:21.620312+0000","last_fullsized":"2024-09-19T09:02:21.620312+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:18.945035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'16","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.975541+0000","last_change":"2024-09-19T09:00:44.938554+0000","last_active":"2024-09-19T09:02:21.975541+0000","last_peered":"2024-09-19T09:02:21.975541+0000","last_clean":"2024-09-19T09:02:21.975541+0000","last_became_active":"2024-09-19T09:00:44.938261+0000","last_became_peered":"2024-09-19T09:00:44.938261+0000","last_unstale":"2024-09-19T09:02:21.975541+0000","last_undegraded":"2024-09-19T09:02:21.975541+0000","last_fullsized":"2024-09-19T09:02:21.975541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:24.293764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.614207+0000","last_change":"2024-09-19T09:01:52.755528+0000","last_active":"2024-09-19T09:02:21.614207+0000","last_peered":"2024-09-19T09:02:21.614207+0000","last_clean":"2024-09-19T09:02:21.614207+0000","last_became_active":"2024-09-19T09:01:52.755188+0000","last_became_peered":"2024-09-19T09:01:52.755188+0000","last_unstale":"2024-09-19T09:02:21.614207+0000","last_undegraded":"2024-09-19T09:02:21.614207+0000","last_fullsized":"2024-09-19T09:02:21.614207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:49:34.988733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.888813+0000","last_change":"2024-09-19T09:00:44.941479+0000","last_active":"2024-09-19T09:01:52.888813+0000","last_peered":"2024-09-19T09:01:52.888813+0000","last_clean":"2024-09-19T09:01:52.888813+0000","last_became_active":"2024-09-19T09:00:44.937197+0000","last_became_peered":"2024-09-19T09:00:44.937197+0000","last_unstale":"2024-09-19T09:01:52.888813+0000","last_undegraded":"2024-09-19T09:01:52.888813+0000","last_fullsized":"2024-09-19T09:01:52.888813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:53:17.385884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621026+0000","last_change":"2024-09-19T09:00:46.970319+0000","last_active":"2024-09-19T09:02:21.621026+0000","last_peered":"2024-09-19T09:02:21.621026+0000","last_clean":"2024-09-19T09:02:21.621026+0000","last_became_active":"2024-09-19T09:00:46.969899+0000","last_became_peered":"2024-09-19T09:00:46.969899+0000","last_unstale":"2024-09-19T09:02:21.621026+0000","last_undegraded":"2024-09-19T09:02:21.621026+0000","last_fullsized":"2024-09-19T09:02:21.621026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:26:21.379231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751656+0000","last_change":"2024-09-19T09:00:45.239525+0000","last_active":"2024-09-19T09:01:52.751656+0000","last_peered":"2024-09-19T09:01:52.751656+0000","last_clean":"2024-09-19T09:01:52.751656+0000","last_became_active":"2024-09-19T09:00:45.239213+0000","last_became_peered":"2024-09-19T09:00:45.239213+0000","last_unstale":"2024-09-19T09:01:52.751656+0000","last_undegraded":"2024-09-19T09:01:52.751656+0000","last_fullsized":"2024-09-19T09:01:52.751656+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:32.525025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.584763+0000","last_change":"2024-09-19T09:00:46.958573+0000","last_active":"2024-09-19T09:02:21.584763+0000","last_peered":"2024-09-19T09:02:21.584763+0000","last_clean":"2024-09-19T09:02:21.584763+0000","last_became_active":"2024-09-19T09:00:46.958274+0000","last_became_peered":"2024-09-19T09:00:46.958274+0000","last_unstale":"2024-09-19T09:02:21.584763+0000","last_undegraded":"2024-09-19T09:02:21.584763+0000","last_fullsized":"2024-09-19T09:02:21.584763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:17.249122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.903274+0000","last_change":"2024-09-19T09:00:44.945194+0000","last_active":"2024-09-19T09:01:52.903274+0000","last_peered":"2024-09-19T09:01:52.903274+0000","last_clean":"2024-09-19T09:01:52.903274+0000","last_became_active":"2024-09-19T09:00:44.944931+0000","last_became_peered":"2024-09-19T09:00:44.944931+0000","last_unstale":"2024-09-19T09:01:52.903274+0000","last_undegraded":"2024-09-19T09:01:52.903274+0000","last_fullsized":"2024-09-19T09:01:52.903274+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:11.181150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621916+0000","last_change":"2024-09-19T09:00:46.966895+0000","last_active":"2024-09-19T09:02:21.621916+0000","last_peered":"2024-09-19T09:02:21.621916+0000","last_clean":"2024-09-19T09:02:21.621916+0000","last_became_active":"2024-09-19T09:00:46.966124+0000","last_became_peered":"2024-09-19T09:00:46.966124+0000","last_unstale":"2024-09-19T09:02:21.621916+0000","last_undegraded":"2024-09-19T09:02:21.621916+0000","last_fullsized":"2024-09-19T09:02:21.621916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:14:04.320863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:41.120672+0000","last_change":"2024-09-19T09:00:44.944167+0000","last_active":"2024-09-19T09:02:41.120672+0000","last_peered":"2024-09-19T09:02:41.120672+0000","last_clean":"2024-09-19T09:02:41.120672+0000","last_became_active":"2024-09-19T09:00:44.943670+0000","last_became_peered":"2024-09-19T09:00:44.943670+0000","last_unstale":"2024-09-19T09:02:41.120672+0000","last_undegraded":"2024-09-19T09:02:41.120672+0000","last_fullsized":"2024-09-19T09:02:41.120672+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:05.596877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.615278+0000","last_change":"2024-09-19T09:00:46.958581+0000","last_active":"2024-09-19T09:02:21.615278+0000","last_peered":"2024-09-19T09:02:21.615278+0000","last_clean":"2024-09-19T09:02:21.615278+0000","last_became_active":"2024-09-19T09:00:46.957782+0000","last_became_peered":"2024-09-19T09:00:46.957782+0000","last_unstale":"2024-09-19T09:02:21.615278+0000","last_undegraded":"2024-09-19T09:02:21.615278+0000","last_fullsized":"2024-09-19T09:02:21.615278+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:37:19.488538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751779+0000","last_change":"2024-09-19T09:00:44.945351+0000","last_active":"2024-09-19T09:01:52.751779+0000","last_peered":"2024-09-19T09:01:52.751779+0000","last_clean":"2024-09-19T09:01:52.751779+0000","last_became_active":"2024-09-19T09:00:44.944991+0000","last_became_peered":"2024-09-19T09:00:44.944991+0000","last_unstale":"2024-09-19T09:01:52.751779+0000","last_undegraded":"2024-09-19T09:01:52.751779+0000","last_fullsized":"2024-09-19T09:01:52.751779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:45:07.581872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633316+0000","last_change":"2024-09-19T09:00:46.956340+0000","last_active":"2024-09-19T09:02:21.633316+0000","last_peered":"2024-09-19T09:02:21.633316+0000","last_clean":"2024-09-19T09:02:21.633316+0000","last_became_active":"2024-09-19T09:00:46.955956+0000","last_became_peered":"2024-09-19T09:00:46.955956+0000","last_unstale":"2024-09-19T09:02:21.633316+0000","last_undegraded":"2024-09-19T09:02:21.633316+0000","last_fullsized":"2024-09-19T09:02:21.633316+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:34.863428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633515+0000","last_change":"2024-09-19T09:00:46.963657+0000","last_active":"2024-09-19T09:02:21.633515+0000","last_peered":"2024-09-19T09:02:21.633515+0000","last_clean":"2024-09-19T09:02:21.633515+0000","last_became_active":"2024-09-19T09:00:46.963134+0000","last_became_peered":"2024-09-19T09:00:46.963134+0000","last_unstale":"2024-09-19T09:02:21.633515+0000","last_undegraded":"2024-09-19T09:02:21.633515+0000","last_fullsized":"2024-09-19T09:02:21.633515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:01.098570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.764939+0000","last_change":"2024-09-19T09:01:52.764438+0000","last_active":"2024-09-19T09:01:52.764939+0000","last_peered":"2024-09-19T09:01:52.764939+0000","last_clean":"2024-09-19T09:01:52.764939+0000","last_became_active":"2024-09-19T09:01:52.755188+0000","last_became_peered":"2024-09-19T09:01:52.755188+0000","last_unstale":"2024-09-19T09:01:52.764939+0000","last_undegraded":"2024-09-19T09:01:52.764939+0000","last_fullsized":"2024-09-19T09:01:52.764939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:40.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.632994+0000","last_change":"2024-09-19T09:00:46.956354+0000","last_active":"2024-09-19T09:02:21.632994+0000","last_peered":"2024-09-19T09:02:21.632994+0000","last_clean":"2024-09-19T09:02:21.632994+0000","last_became_active":"2024-09-19T09:00:46.955964+0000","last_became_peered":"2024-09-19T09:00:46.955964+0000","last_unstale":"2024-09-19T09:02:21.632994+0000","last_undegraded":"2024-09-19T09:02:21.632994+0000","last_fullsized":"2024-09-19T09:02:21.632994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:05:33.243479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751944+0000","last_change":"2024-09-19T09:00:44.944983+0000","last_active":"2024-09-19T09:01:52.751944+0000","last_peered":"2024-09-19T09:01:52.751944+0000","last_clean":"2024-09-19T09:01:52.751944+0000","last_became_active":"2024-09-19T09:00:44.944625+0000","last_became_peered":"2024-09-19T09:00:44.944625+0000","last_unstale":"2024-09-19T09:01:52.751944+0000","last_undegraded":"2024-09-19T09:01:52.751944+0000","last_fullsized":"2024-09-19T09:01:52.751944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:49:50.818863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.619748+0000","last_change":"2024-09-19T09:00:46.966448+0000","last_active":"2024-09-19T09:02:21.619748+0000","last_peered":"2024-09-19T09:02:21.619748+0000","last_clean":"2024-09-19T09:02:21.619748+0000","last_became_active":"2024-09-19T09:00:46.966135+0000","last_became_peered":"2024-09-19T09:00:46.966135+0000","last_unstale":"2024-09-19T09:02:21.619748+0000","last_undegraded":"2024-09-19T09:02:21.619748+0000","last_fullsized":"2024-09-19T09:02:21.619748+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:42.652877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.903626+0000","last_change":"2024-09-19T09:00:45.239405+0000","last_active":"2024-09-19T09:01:52.903626+0000","last_peered":"2024-09-19T09:01:52.903626+0000","last_clean":"2024-09-19T09:01:52.903626+0000","last_became_active":"2024-09-19T09:00:45.239002+0000","last_became_peered":"2024-09-19T09:00:45.239002+0000","last_unstale":"2024-09-19T09:01:52.903626+0000","last_undegraded":"2024-09-19T09:01:52.903626+0000","last_fullsized":"2024-09-19T09:01:52.903626+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:46:03.934975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621992+0000","last_change":"2024-09-19T09:00:46.951635+0000","last_active":"2024-09-19T09:02:21.621992+0000","last_peered":"2024-09-19T09:02:21.621992+0000","last_clean":"2024-09-19T09:02:21.621992+0000","last_became_active":"2024-09-19T09:00:46.951325+0000","last_became_peered":"2024-09-19T09:00:46.951325+0000","last_unstale":"2024-09-19T09:02:21.621992+0000","last_undegraded":"2024-09-19T09:02:21.621992+0000","last_fullsized":"2024-09-19T09:02:21.621992+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:17.360950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":158,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:39.845934+0000","last_change":"2024-09-19T09:01:52.766049+0000","last_active":"2024-09-19T09:02:39.845934+0000","last_peered":"2024-09-19T09:02:39.845934+0000","last_clean":"2024-09-19T09:02:39.845934+0000","last_became_active":"2024-09-19T09:01:52.755263+0000","last_became_peered":"2024-09-19T09:01:52.755263+0000","last_unstale":"2024-09-19T09:02:39.845934+0000","last_undegraded":"2024-09-19T09:02:39.845934+0000","last_fullsized":"2024-09-19T09:02:39.845934+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:16.968329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70524,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1072,"num_read_kb":609,"num_write":261,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1366,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":220593,"data_compressed":4815,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":212166,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612644,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":429375,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841577,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":1082243,"data_compressed":43295,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972399,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":428051,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201332,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":427847,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430266,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":1020726,"data_compressed":41690,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593791,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":427768,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822723,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":1020542,"data_compressed":41690,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986249,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":428163,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215181,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":427593,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":497064,"data_compressed":7388,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":966656,"data_stored":496367,"data_compressed":7388,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":429500,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":69438,"data_compressed":1605,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":70135,"data_compressed":1605,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65137,"data_compressed":1605,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:02:45.710 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-19T09:02:45.710 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T09:02:45.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.845+0000 7f1244f6d640 1 -- 172.21.15.120:0/49122814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1240101d60 msgr2=0x7f12401085f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:45.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.845+0000 7f1244f6d640 1 --2- 172.21.15.120:0/49122814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1240101d60 0x7f12401085f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1238009ca0 tx=0x7f123802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:02:45.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.845+0000 7f1244f6d640 1 -- 172.21.15.120:0/49122814 shutdown_connections 2024-09-19T09:02:45.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.845+0000 7f1244f6d640 1 --2- 172.21.15.120:0/49122814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1240101d60 0x7f12401085f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:45.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.845+0000 7f1244f6d640 1 --2- 172.21.15.120:0/49122814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12401013a0 0x7f1240101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:45.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.845+0000 7f1244f6d640 1 --2- 172.21.15.120:0/49122814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12401001b0 0x7f12401005b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:45.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.845+0000 7f1244f6d640 1 -- 172.21.15.120:0/49122814 >> 172.21.15.120:0/49122814 conn(0x7f12400fb950 msgr2=0x7f12400fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:45.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.845+0000 7f1244f6d640 1 -- 172.21.15.120:0/49122814 shutdown_connections 2024-09-19T09:02:45.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.845+0000 7f1244f6d640 1 -- 172.21.15.120:0/49122814 wait complete. 2024-09-19T09:02:45.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.846+0000 7f1244f6d640 1 Processor -- start 2024-09-19T09:02:45.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.846+0000 7f1244f6d640 1 -- start start 2024-09-19T09:02:45.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.846+0000 7f1244f6d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12401001b0 0x7f1240109b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:45.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.846+0000 7f1244f6d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12401013a0 0x7f124019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:45.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.846+0000 7f1244f6d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1240101d60 0x7f124019f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:45.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.846+0000 7f1244f6d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f124010c6b0 con 0x7f1240101d60 2024-09-19T09:02:45.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.846+0000 7f1244f6d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f124010c530 con 0x7f12401013a0 2024-09-19T09:02:45.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.846+0000 7f1244f6d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f124010c830 con 0x7f12401001b0 2024-09-19T09:02:45.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.846+0000 7f123ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1240101d60 0x7f124019f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:45.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.846+0000 7f123ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1240101d60 0x7f124019f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48478/0 (socket says 172.21.15.120:48478) 2024-09-19T09:02:45.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.846+0000 7f123ed9d640 1 -- 172.21.15.120:0/308257262 learned_addr learned my addr 172.21.15.120:0/308257262 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:45.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.846+0000 7f1237fff640 1 --2- 172.21.15.120:0/308257262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12401013a0 0x7f124019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:45.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.847+0000 7f123e59c640 1 --2- 172.21.15.120:0/308257262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12401001b0 0x7f1240109b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:45.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.847+0000 7f123ed9d640 1 -- 172.21.15.120:0/308257262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12401001b0 msgr2=0x7f1240109b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:45.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.847+0000 7f123ed9d640 1 --2- 172.21.15.120:0/308257262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12401001b0 0x7f1240109b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:45.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.847+0000 7f123ed9d640 1 -- 172.21.15.120:0/308257262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12401013a0 msgr2=0x7f124019cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:45.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.847+0000 7f123ed9d640 1 --2- 172.21.15.120:0/308257262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12401013a0 0x7f124019cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:45.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.847+0000 7f123ed9d640 1 -- 172.21.15.120:0/308257262 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124019fac0 con 0x7f1240101d60 2024-09-19T09:02:45.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.847+0000 7f123ed9d640 1 --2- 172.21.15.120:0/308257262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1240101d60 0x7f124019f1b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1238005e30 tx=0x7f1238031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:45.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.847+0000 7f12377fe640 1 -- 172.21.15.120:0/308257262 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1238003c30 con 0x7f1240101d60 2024-09-19T09:02:45.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.847+0000 7f12377fe640 1 -- 172.21.15.120:0/308257262 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1238003dd0 con 0x7f1240101d60 2024-09-19T09:02:45.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.847+0000 7f1244f6d640 1 -- 172.21.15.120:0/308257262 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f124019fcf0 con 0x7f1240101d60 2024-09-19T09:02:45.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.847+0000 7f12377fe640 1 -- 172.21.15.120:0/308257262 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123802fde0 con 0x7f1240101d60 2024-09-19T09:02:45.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.847+0000 7f1244f6d640 1 -- 172.21.15.120:0/308257262 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12401a01e0 con 0x7f1240101d60 2024-09-19T09:02:45.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.848+0000 7f1244f6d640 1 -- 172.21.15.120:0/308257262 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12401075a0 con 0x7f1240101d60 2024-09-19T09:02:45.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.849+0000 7f12377fe640 1 -- 172.21.15.120:0/308257262 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f1238004050 con 0x7f1240101d60 2024-09-19T09:02:45.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.850+0000 7f12377fe640 1 --2- 172.21.15.120:0/308257262 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f122c0a32e0 0x7f122c0a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:45.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.850+0000 7f123e59c640 1 --2- 172.21.15.120:0/308257262 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f122c0a32e0 0x7f122c0a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:45.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.850+0000 7f12377fe640 1 -- 172.21.15.120:0/308257262 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f12380f0eb0 con 0x7f1240101d60 2024-09-19T09:02:45.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.850+0000 7f123e59c640 1 --2- 172.21.15.120:0/308257262 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f122c0a32e0 0x7f122c0a57a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1230005fd0 tx=0x7f1230005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:45.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:45.854+0000 7f12377fe640 1 -- 172.21.15.120:0/308257262 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f12380bdca0 con 0x7f1240101d60 2024-09-19T09:02:46.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.018+0000 7f1244f6d640 1 -- 172.21.15.120:0/308257262 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1240062630 con 0x7f122c0a32e0 2024-09-19T09:02:46.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.024+0000 7f12377fe640 1 -- 172.21.15.120:0/308257262 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259138 (secure 0 0 0) 0x7f1240062630 con 0x7f122c0a32e0 2024-09-19T09:02:46.027 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:02:46.028 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-19T09:02:46.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.027+0000 7f1244f6d640 1 -- 172.21.15.120:0/308257262 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f122c0a32e0 msgr2=0x7f122c0a57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:46.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.027+0000 7f1244f6d640 1 --2- 172.21.15.120:0/308257262 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f122c0a32e0 0x7f122c0a57a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1230005fd0 tx=0x7f1230005c70 comp rx=0 tx=0).stop 2024-09-19T09:02:46.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.027+0000 7f1244f6d640 1 -- 172.21.15.120:0/308257262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1240101d60 msgr2=0x7f124019f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:46.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.027+0000 7f1244f6d640 1 --2- 172.21.15.120:0/308257262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1240101d60 0x7f124019f1b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1238005e30 tx=0x7f1238031ee0 comp rx=0 tx=0).stop 2024-09-19T09:02:46.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.027+0000 7f1244f6d640 1 -- 172.21.15.120:0/308257262 shutdown_connections 2024-09-19T09:02:46.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.027+0000 7f1244f6d640 1 --2- 172.21.15.120:0/308257262 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f122c0a32e0 0x7f122c0a57a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:46.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.027+0000 7f1244f6d640 1 --2- 172.21.15.120:0/308257262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1240101d60 0x7f124019f1b0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:46.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.027+0000 7f1244f6d640 1 --2- 172.21.15.120:0/308257262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12401013a0 0x7f124019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:46.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.027+0000 7f1244f6d640 1 --2- 172.21.15.120:0/308257262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12401001b0 0x7f1240109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:46.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.027+0000 7f1244f6d640 1 -- 172.21.15.120:0/308257262 >> 172.21.15.120:0/308257262 conn(0x7f12400fb950 msgr2=0x7f1240104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:46.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.027+0000 7f1244f6d640 1 -- 172.21.15.120:0/308257262 shutdown_connections 2024-09-19T09:02:46.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.027+0000 7f1244f6d640 1 -- 172.21.15.120:0/308257262 wait complete. 2024-09-19T09:02:46.037 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:45 smithi120 ceph-mon[31150]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-19T09:02:46.038 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-09-19T09:02:45.636047+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661126,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1198,"num_read_kb":718,"num_write":444,"num_write_kb":3089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1182102,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364360,"kb_used_data":10952,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577464,"statfs":{"total":1151940427776,"available":1151567323136,"internally_reserved":0,"allocated":11214848,"data_stored":7115239,"data_compressed":181932,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1009,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":52,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003143"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624417+0000","last_change":"2024-09-19T09:00:46.966628+0000","last_active":"2024-09-19T09:02:21.624417+0000","last_peered":"2024-09-19T09:02:21.624417+0000","last_clean":"2024-09-19T09:02:21.624417+0000","last_became_active":"2024-09-19T09:00:46.966290+0000","last_became_peered":"2024-09-19T09:00:46.966290+0000","last_unstale":"2024-09-19T09:02:21.624417+0000","last_undegraded":"2024-09-19T09:02:21.624417+0000","last_fullsized":"2024-09-19T09:02:21.624417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:42.717139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621317+0000","last_change":"2024-09-19T09:00:46.963706+0000","last_active":"2024-09-19T09:02:21.621317+0000","last_peered":"2024-09-19T09:02:21.621317+0000","last_clean":"2024-09-19T09:02:21.621317+0000","last_became_active":"2024-09-19T09:00:46.963303+0000","last_became_peered":"2024-09-19T09:00:46.963303+0000","last_unstale":"2024-09-19T09:02:21.621317+0000","last_undegraded":"2024-09-19T09:02:21.621317+0000","last_fullsized":"2024-09-19T09:02:21.621317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:07.451500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620146+0000","last_change":"2024-09-19T09:00:46.967149+0000","last_active":"2024-09-19T09:02:21.620146+0000","last_peered":"2024-09-19T09:02:21.620146+0000","last_clean":"2024-09-19T09:02:21.620146+0000","last_became_active":"2024-09-19T09:00:46.966861+0000","last_became_peered":"2024-09-19T09:00:46.966861+0000","last_unstale":"2024-09-19T09:02:21.620146+0000","last_undegraded":"2024-09-19T09:02:21.620146+0000","last_fullsized":"2024-09-19T09:02:21.620146+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:28.396022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.603363+0000","last_change":"2024-09-19T09:00:46.969242+0000","last_active":"2024-09-19T09:02:21.603363+0000","last_peered":"2024-09-19T09:02:21.603363+0000","last_clean":"2024-09-19T09:02:21.603363+0000","last_became_active":"2024-09-19T09:00:46.968546+0000","last_became_peered":"2024-09-19T09:00:46.968546+0000","last_unstale":"2024-09-19T09:02:21.603363+0000","last_undegraded":"2024-09-19T09:02:21.603363+0000","last_fullsized":"2024-09-19T09:02:21.603363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:43:51.060130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.602962+0000","last_change":"2024-09-19T09:00:46.955501+0000","last_active":"2024-09-19T09:02:21.602962+0000","last_peered":"2024-09-19T09:02:21.602962+0000","last_clean":"2024-09-19T09:02:21.602962+0000","last_became_active":"2024-09-19T09:00:46.955161+0000","last_became_peered":"2024-09-19T09:00:46.955161+0000","last_unstale":"2024-09-19T09:02:21.602962+0000","last_undegraded":"2024-09-19T09:02:21.602962+0000","last_fullsized":"2024-09-19T09:02:21.602962+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:39:19.005931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624332+0000","last_change":"2024-09-19T09:00:46.962577+0000","last_active":"2024-09-19T09:02:21.624332+0000","last_peered":"2024-09-19T09:02:21.624332+0000","last_clean":"2024-09-19T09:02:21.624332+0000","last_became_active":"2024-09-19T09:00:46.962253+0000","last_became_peered":"2024-09-19T09:00:46.962253+0000","last_unstale":"2024-09-19T09:02:21.624332+0000","last_undegraded":"2024-09-19T09:02:21.624332+0000","last_fullsized":"2024-09-19T09:02:21.624332+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:27.925932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621157+0000","last_change":"2024-09-19T09:00:46.960004+0000","last_active":"2024-09-19T09:02:21.621157+0000","last_peered":"2024-09-19T09:02:21.621157+0000","last_clean":"2024-09-19T09:02:21.621157+0000","last_became_active":"2024-09-19T09:00:46.959664+0000","last_became_peered":"2024-09-19T09:00:46.959664+0000","last_unstale":"2024-09-19T09:02:21.621157+0000","last_undegraded":"2024-09-19T09:02:21.621157+0000","last_fullsized":"2024-09-19T09:02:21.621157+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:41.616309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622980+0000","last_change":"2024-09-19T09:00:46.952394+0000","last_active":"2024-09-19T09:02:21.622980+0000","last_peered":"2024-09-19T09:02:21.622980+0000","last_clean":"2024-09-19T09:02:21.622980+0000","last_became_active":"2024-09-19T09:00:46.952084+0000","last_became_peered":"2024-09-19T09:00:46.952084+0000","last_unstale":"2024-09-19T09:02:21.622980+0000","last_undegraded":"2024-09-19T09:02:21.622980+0000","last_fullsized":"2024-09-19T09:02:21.622980+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:21.698423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.632916+0000","last_change":"2024-09-19T09:00:46.961262+0000","last_active":"2024-09-19T09:02:21.632916+0000","last_peered":"2024-09-19T09:02:21.632916+0000","last_clean":"2024-09-19T09:02:21.632916+0000","last_became_active":"2024-09-19T09:00:46.960951+0000","last_became_peered":"2024-09-19T09:00:46.960951+0000","last_unstale":"2024-09-19T09:02:21.632916+0000","last_undegraded":"2024-09-19T09:02:21.632916+0000","last_fullsized":"2024-09-19T09:02:21.632916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:02.445108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.625661+0000","last_change":"2024-09-19T09:01:52.758459+0000","last_active":"2024-09-19T09:02:21.625661+0000","last_peered":"2024-09-19T09:02:21.625661+0000","last_clean":"2024-09-19T09:02:21.625661+0000","last_became_active":"2024-09-19T09:01:52.758073+0000","last_became_peered":"2024-09-19T09:01:52.758073+0000","last_unstale":"2024-09-19T09:02:21.625661+0000","last_undegraded":"2024-09-19T09:02:21.625661+0000","last_fullsized":"2024-09-19T09:02:21.625661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:46:32.983316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.634093+0000","last_change":"2024-09-19T09:00:46.963200+0000","last_active":"2024-09-19T09:02:21.634093+0000","last_peered":"2024-09-19T09:02:21.634093+0000","last_clean":"2024-09-19T09:02:21.634093+0000","last_became_active":"2024-09-19T09:00:46.960496+0000","last_became_peered":"2024-09-19T09:00:46.960496+0000","last_unstale":"2024-09-19T09:02:21.634093+0000","last_undegraded":"2024-09-19T09:02:21.634093+0000","last_fullsized":"2024-09-19T09:02:21.634093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:59:47.234492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.618494+0000","last_change":"2024-09-19T09:00:46.966309+0000","last_active":"2024-09-19T09:02:21.618494+0000","last_peered":"2024-09-19T09:02:21.618494+0000","last_clean":"2024-09-19T09:02:21.618494+0000","last_became_active":"2024-09-19T09:00:46.965970+0000","last_became_peered":"2024-09-19T09:00:46.965970+0000","last_unstale":"2024-09-19T09:02:21.618494+0000","last_undegraded":"2024-09-19T09:02:21.618494+0000","last_fullsized":"2024-09-19T09:02:21.618494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:38.353206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622484+0000","last_change":"2024-09-19T09:00:46.954079+0000","last_active":"2024-09-19T09:02:21.622484+0000","last_peered":"2024-09-19T09:02:21.622484+0000","last_clean":"2024-09-19T09:02:21.622484+0000","last_became_active":"2024-09-19T09:00:46.953398+0000","last_became_peered":"2024-09-19T09:00:46.953398+0000","last_unstale":"2024-09-19T09:02:21.622484+0000","last_undegraded":"2024-09-19T09:02:21.622484+0000","last_fullsized":"2024-09-19T09:02:21.622484+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:54:27.554800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621192+0000","last_change":"2024-09-19T09:00:46.965347+0000","last_active":"2024-09-19T09:02:21.621192+0000","last_peered":"2024-09-19T09:02:21.621192+0000","last_clean":"2024-09-19T09:02:21.621192+0000","last_became_active":"2024-09-19T09:00:46.965045+0000","last_became_peered":"2024-09-19T09:00:46.965045+0000","last_unstale":"2024-09-19T09:02:21.621192+0000","last_undegraded":"2024-09-19T09:02:21.621192+0000","last_fullsized":"2024-09-19T09:02:21.621192+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:27.316100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620943+0000","last_change":"2024-09-19T09:00:46.962898+0000","last_active":"2024-09-19T09:02:21.620943+0000","last_peered":"2024-09-19T09:02:21.620943+0000","last_clean":"2024-09-19T09:02:21.620943+0000","last_became_active":"2024-09-19T09:00:46.962412+0000","last_became_peered":"2024-09-19T09:00:46.962412+0000","last_unstale":"2024-09-19T09:02:21.620943+0000","last_undegraded":"2024-09-19T09:02:21.620943+0000","last_fullsized":"2024-09-19T09:02:21.620943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:40.483678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.627060+0000","last_change":"2024-09-19T09:00:46.958147+0000","last_active":"2024-09-19T09:02:21.627060+0000","last_peered":"2024-09-19T09:02:21.627060+0000","last_clean":"2024-09-19T09:02:21.627060+0000","last_became_active":"2024-09-19T09:00:46.957805+0000","last_became_peered":"2024-09-19T09:00:46.957805+0000","last_unstale":"2024-09-19T09:02:21.627060+0000","last_undegraded":"2024-09-19T09:02:21.627060+0000","last_fullsized":"2024-09-19T09:02:21.627060+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:24:17.000064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624642+0000","last_change":"2024-09-19T09:00:46.967399+0000","last_active":"2024-09-19T09:02:21.624642+0000","last_peered":"2024-09-19T09:02:21.624642+0000","last_clean":"2024-09-19T09:02:21.624642+0000","last_became_active":"2024-09-19T09:00:46.966145+0000","last_became_peered":"2024-09-19T09:00:46.966145+0000","last_unstale":"2024-09-19T09:02:21.624642+0000","last_undegraded":"2024-09-19T09:02:21.624642+0000","last_fullsized":"2024-09-19T09:02:21.624642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:29.192469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.614692+0000","last_change":"2024-09-19T09:00:46.960956+0000","last_active":"2024-09-19T09:02:21.614692+0000","last_peered":"2024-09-19T09:02:21.614692+0000","last_clean":"2024-09-19T09:02:21.614692+0000","last_became_active":"2024-09-19T09:00:46.960664+0000","last_became_peered":"2024-09-19T09:00:46.960664+0000","last_unstale":"2024-09-19T09:02:21.614692+0000","last_undegraded":"2024-09-19T09:02:21.614692+0000","last_fullsized":"2024-09-19T09:02:21.614692+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:18.724348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622878+0000","last_change":"2024-09-19T09:00:46.951362+0000","last_active":"2024-09-19T09:02:21.622878+0000","last_peered":"2024-09-19T09:02:21.622878+0000","last_clean":"2024-09-19T09:02:21.622878+0000","last_became_active":"2024-09-19T09:00:46.951004+0000","last_became_peered":"2024-09-19T09:00:46.951004+0000","last_unstale":"2024-09-19T09:02:21.622878+0000","last_undegraded":"2024-09-19T09:02:21.622878+0000","last_fullsized":"2024-09-19T09:02:21.622878+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:43.574425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.623346+0000","last_change":"2024-09-19T09:00:46.966752+0000","last_active":"2024-09-19T09:02:21.623346+0000","last_peered":"2024-09-19T09:02:21.623346+0000","last_clean":"2024-09-19T09:02:21.623346+0000","last_became_active":"2024-09-19T09:00:46.966370+0000","last_became_peered":"2024-09-19T09:00:46.966370+0000","last_unstale":"2024-09-19T09:02:21.623346+0000","last_undegraded":"2024-09-19T09:02:21.623346+0000","last_fullsized":"2024-09-19T09:02:21.623346+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:04.035771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621434+0000","last_change":"2024-09-19T09:00:46.956369+0000","last_active":"2024-09-19T09:02:21.621434+0000","last_peered":"2024-09-19T09:02:21.621434+0000","last_clean":"2024-09-19T09:02:21.621434+0000","last_became_active":"2024-09-19T09:00:46.956075+0000","last_became_peered":"2024-09-19T09:00:46.956075+0000","last_unstale":"2024-09-19T09:02:21.621434+0000","last_undegraded":"2024-09-19T09:02:21.621434+0000","last_fullsized":"2024-09-19T09:02:21.621434+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:23.985017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633789+0000","last_change":"2024-09-19T09:00:46.956661+0000","last_active":"2024-09-19T09:02:21.633789+0000","last_peered":"2024-09-19T09:02:21.633789+0000","last_clean":"2024-09-19T09:02:21.633789+0000","last_became_active":"2024-09-19T09:00:46.956291+0000","last_became_peered":"2024-09-19T09:00:46.956291+0000","last_unstale":"2024-09-19T09:02:21.633789+0000","last_undegraded":"2024-09-19T09:02:21.633789+0000","last_fullsized":"2024-09-19T09:02:21.633789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:55.713039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.616782+0000","last_change":"2024-09-19T09:00:46.961866+0000","last_active":"2024-09-19T09:02:21.616782+0000","last_peered":"2024-09-19T09:02:21.616782+0000","last_clean":"2024-09-19T09:02:21.616782+0000","last_became_active":"2024-09-19T09:00:46.961469+0000","last_became_peered":"2024-09-19T09:00:46.961469+0000","last_unstale":"2024-09-19T09:02:21.616782+0000","last_undegraded":"2024-09-19T09:02:21.616782+0000","last_fullsized":"2024-09-19T09:02:21.616782+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:47.665419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622833+0000","last_change":"2024-09-19T09:00:46.948765+0000","last_active":"2024-09-19T09:02:21.622833+0000","last_peered":"2024-09-19T09:02:21.622833+0000","last_clean":"2024-09-19T09:02:21.622833+0000","last_became_active":"2024-09-19T09:00:46.948431+0000","last_became_peered":"2024-09-19T09:00:46.948431+0000","last_unstale":"2024-09-19T09:02:21.622833+0000","last_undegraded":"2024-09-19T09:02:21.622833+0000","last_fullsized":"2024-09-19T09:02:21.622833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:31.766464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.618165+0000","last_change":"2024-09-19T09:00:46.962433+0000","last_active":"2024-09-19T09:02:21.618165+0000","last_peered":"2024-09-19T09:02:21.618165+0000","last_clean":"2024-09-19T09:02:21.618165+0000","last_became_active":"2024-09-19T09:00:46.962153+0000","last_became_peered":"2024-09-19T09:00:46.962153+0000","last_unstale":"2024-09-19T09:02:21.618165+0000","last_undegraded":"2024-09-19T09:02:21.618165+0000","last_fullsized":"2024-09-19T09:02:21.618165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:37.973611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.623402+0000","last_change":"2024-09-19T09:00:46.953715+0000","last_active":"2024-09-19T09:02:21.623402+0000","last_peered":"2024-09-19T09:02:21.623402+0000","last_clean":"2024-09-19T09:02:21.623402+0000","last_became_active":"2024-09-19T09:00:46.953415+0000","last_became_peered":"2024-09-19T09:00:46.953415+0000","last_unstale":"2024-09-19T09:02:21.623402+0000","last_undegraded":"2024-09-19T09:02:21.623402+0000","last_fullsized":"2024-09-19T09:02:21.623402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:29.768382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.615199+0000","last_change":"2024-09-19T09:00:46.964746+0000","last_active":"2024-09-19T09:02:21.615199+0000","last_peered":"2024-09-19T09:02:21.615199+0000","last_clean":"2024-09-19T09:02:21.615199+0000","last_became_active":"2024-09-19T09:00:46.964404+0000","last_became_peered":"2024-09-19T09:00:46.964404+0000","last_unstale":"2024-09-19T09:02:21.615199+0000","last_undegraded":"2024-09-19T09:02:21.615199+0000","last_fullsized":"2024-09-19T09:02:21.615199+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:19.841495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.619156+0000","last_change":"2024-09-19T09:00:46.962875+0000","last_active":"2024-09-19T09:02:21.619156+0000","last_peered":"2024-09-19T09:02:21.619156+0000","last_clean":"2024-09-19T09:02:21.619156+0000","last_became_active":"2024-09-19T09:00:46.962561+0000","last_became_peered":"2024-09-19T09:00:46.962561+0000","last_unstale":"2024-09-19T09:02:21.619156+0000","last_undegraded":"2024-09-19T09:02:21.619156+0000","last_fullsized":"2024-09-19T09:02:21.619156+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:39.675545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.612887+0000","last_change":"2024-09-19T09:00:46.963126+0000","last_active":"2024-09-19T09:02:21.612887+0000","last_peered":"2024-09-19T09:02:21.612887+0000","last_clean":"2024-09-19T09:02:21.612887+0000","last_became_active":"2024-09-19T09:00:46.962620+0000","last_became_peered":"2024-09-19T09:00:46.962620+0000","last_unstale":"2024-09-19T09:02:21.612887+0000","last_undegraded":"2024-09-19T09:02:21.612887+0000","last_fullsized":"2024-09-19T09:02:21.612887+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:59.723280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620254+0000","last_change":"2024-09-19T09:00:46.953026+0000","last_active":"2024-09-19T09:02:21.620254+0000","last_peered":"2024-09-19T09:02:21.620254+0000","last_clean":"2024-09-19T09:02:21.620254+0000","last_became_active":"2024-09-19T09:00:46.952596+0000","last_became_peered":"2024-09-19T09:00:46.952596+0000","last_unstale":"2024-09-19T09:02:21.620254+0000","last_undegraded":"2024-09-19T09:02:21.620254+0000","last_fullsized":"2024-09-19T09:02:21.620254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:58.524116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624169+0000","last_change":"2024-09-19T09:00:46.968964+0000","last_active":"2024-09-19T09:02:21.624169+0000","last_peered":"2024-09-19T09:02:21.624169+0000","last_clean":"2024-09-19T09:02:21.624169+0000","last_became_active":"2024-09-19T09:00:46.968505+0000","last_became_peered":"2024-09-19T09:00:46.968505+0000","last_unstale":"2024-09-19T09:02:21.624169+0000","last_undegraded":"2024-09-19T09:02:21.624169+0000","last_fullsized":"2024-09-19T09:02:21.624169+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:17:33.335541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.625205+0000","last_change":"2024-09-19T09:00:46.961520+0000","last_active":"2024-09-19T09:02:21.625205+0000","last_peered":"2024-09-19T09:02:21.625205+0000","last_clean":"2024-09-19T09:02:21.625205+0000","last_became_active":"2024-09-19T09:00:46.961140+0000","last_became_peered":"2024-09-19T09:00:46.961140+0000","last_unstale":"2024-09-19T09:02:21.625205+0000","last_undegraded":"2024-09-19T09:02:21.625205+0000","last_fullsized":"2024-09-19T09:02:21.625205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:03.120173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621084+0000","last_change":"2024-09-19T09:00:46.955995+0000","last_active":"2024-09-19T09:02:21.621084+0000","last_peered":"2024-09-19T09:02:21.621084+0000","last_clean":"2024-09-19T09:02:21.621084+0000","last_became_active":"2024-09-19T09:00:46.954678+0000","last_became_peered":"2024-09-19T09:00:46.954678+0000","last_unstale":"2024-09-19T09:02:21.621084+0000","last_undegraded":"2024-09-19T09:02:21.621084+0000","last_fullsized":"2024-09-19T09:02:21.621084+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:10.314575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":135,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:41.284935+0000","last_change":"2024-09-19T09:00:44.946000+0000","last_active":"2024-09-19T09:02:41.284935+0000","last_peered":"2024-09-19T09:02:41.284935+0000","last_clean":"2024-09-19T09:02:41.284935+0000","last_became_active":"2024-09-19T09:00:44.945583+0000","last_became_peered":"2024-09-19T09:00:44.945583+0000","last_unstale":"2024-09-19T09:02:41.284935+0000","last_undegraded":"2024-09-19T09:02:41.284935+0000","last_fullsized":"2024-09-19T09:02:41.284935+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:53:19.109697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624079+0000","last_change":"2024-09-19T09:01:52.904443+0000","last_active":"2024-09-19T09:02:21.624079+0000","last_peered":"2024-09-19T09:02:21.624079+0000","last_clean":"2024-09-19T09:02:21.624079+0000","last_became_active":"2024-09-19T09:01:52.904077+0000","last_became_peered":"2024-09-19T09:01:52.904077+0000","last_unstale":"2024-09-19T09:02:21.624079+0000","last_undegraded":"2024-09-19T09:02:21.624079+0000","last_fullsized":"2024-09-19T09:02:21.624079+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:50.467090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.748381+0000","last_change":"2024-09-19T09:00:44.942435+0000","last_active":"2024-09-19T09:01:52.748381+0000","last_peered":"2024-09-19T09:01:52.748381+0000","last_clean":"2024-09-19T09:01:52.748381+0000","last_became_active":"2024-09-19T09:00:44.942067+0000","last_became_peered":"2024-09-19T09:00:44.942067+0000","last_unstale":"2024-09-19T09:01:52.748381+0000","last_undegraded":"2024-09-19T09:01:52.748381+0000","last_fullsized":"2024-09-19T09:01:52.748381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:16.476506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.747900+0000","last_change":"2024-09-19T09:00:44.937944+0000","last_active":"2024-09-19T09:01:52.747900+0000","last_peered":"2024-09-19T09:01:52.747900+0000","last_clean":"2024-09-19T09:01:52.747900+0000","last_became_active":"2024-09-19T09:00:44.937623+0000","last_became_peered":"2024-09-19T09:00:44.937623+0000","last_unstale":"2024-09-19T09:01:52.747900+0000","last_undegraded":"2024-09-19T09:01:52.747900+0000","last_fullsized":"2024-09-19T09:01:52.747900+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:49:08.730015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620184+0000","last_change":"2024-09-19T09:00:46.959932+0000","last_active":"2024-09-19T09:02:21.620184+0000","last_peered":"2024-09-19T09:02:21.620184+0000","last_clean":"2024-09-19T09:02:21.620184+0000","last_became_active":"2024-09-19T09:00:46.959637+0000","last_became_peered":"2024-09-19T09:00:46.959637+0000","last_unstale":"2024-09-19T09:02:21.620184+0000","last_undegraded":"2024-09-19T09:02:21.620184+0000","last_fullsized":"2024-09-19T09:02:21.620184+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:55.337669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751513+0000","last_change":"2024-09-19T09:00:44.944882+0000","last_active":"2024-09-19T09:01:52.751513+0000","last_peered":"2024-09-19T09:01:52.751513+0000","last_clean":"2024-09-19T09:01:52.751513+0000","last_became_active":"2024-09-19T09:00:44.944116+0000","last_became_peered":"2024-09-19T09:00:44.944116+0000","last_unstale":"2024-09-19T09:01:52.751513+0000","last_undegraded":"2024-09-19T09:01:52.751513+0000","last_fullsized":"2024-09-19T09:01:52.751513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:54:58.555913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.615953+0000","last_change":"2024-09-19T09:00:46.956859+0000","last_active":"2024-09-19T09:02:21.615953+0000","last_peered":"2024-09-19T09:02:21.615953+0000","last_clean":"2024-09-19T09:02:21.615953+0000","last_became_active":"2024-09-19T09:00:46.956561+0000","last_became_peered":"2024-09-19T09:00:46.956561+0000","last_unstale":"2024-09-19T09:02:21.615953+0000","last_undegraded":"2024-09-19T09:02:21.615953+0000","last_fullsized":"2024-09-19T09:02:21.615953+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:09:04.125591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'5","reported_seq":231,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:40.309800+0000","last_change":"2024-09-19T09:00:44.945959+0000","last_active":"2024-09-19T09:02:40.309800+0000","last_peered":"2024-09-19T09:02:40.309800+0000","last_clean":"2024-09-19T09:02:40.309800+0000","last_became_active":"2024-09-19T09:00:44.945640+0000","last_became_peered":"2024-09-19T09:00:44.945640+0000","last_unstale":"2024-09-19T09:02:40.309800+0000","last_undegraded":"2024-09-19T09:02:40.309800+0000","last_fullsized":"2024-09-19T09:02:40.309800+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:02.523834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":204,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.632897+0000","last_change":"2024-09-19T09:00:46.953677+0000","last_active":"2024-09-19T09:02:21.632897+0000","last_peered":"2024-09-19T09:02:21.632897+0000","last_clean":"2024-09-19T09:02:21.632897+0000","last_became_active":"2024-09-19T09:00:46.953357+0000","last_became_peered":"2024-09-19T09:00:46.953357+0000","last_unstale":"2024-09-19T09:02:21.632897+0000","last_undegraded":"2024-09-19T09:02:21.632897+0000","last_fullsized":"2024-09-19T09:02:21.632897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:48.963003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.893433+0000","last_change":"2024-09-19T09:01:52.893433+0000","last_active":"2024-09-19T09:01:52.893433+0000","last_peered":"2024-09-19T09:01:52.893433+0000","last_clean":"2024-09-19T09:01:52.893433+0000","last_became_active":"2024-09-19T09:01:52.889836+0000","last_became_peered":"2024-09-19T09:01:52.889836+0000","last_unstale":"2024-09-19T09:01:52.893433+0000","last_undegraded":"2024-09-19T09:01:52.893433+0000","last_fullsized":"2024-09-19T09:01:52.893433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:08:22.339717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622288+0000","last_change":"2024-09-19T09:00:46.962285+0000","last_active":"2024-09-19T09:02:21.622288+0000","last_peered":"2024-09-19T09:02:21.622288+0000","last_clean":"2024-09-19T09:02:21.622288+0000","last_became_active":"2024-09-19T09:00:46.961982+0000","last_became_peered":"2024-09-19T09:00:46.961982+0000","last_unstale":"2024-09-19T09:02:21.622288+0000","last_undegraded":"2024-09-19T09:02:21.622288+0000","last_fullsized":"2024-09-19T09:02:21.622288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:41.658501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752480+0000","last_change":"2024-09-19T09:00:45.241425+0000","last_active":"2024-09-19T09:01:52.752480+0000","last_peered":"2024-09-19T09:01:52.752480+0000","last_clean":"2024-09-19T09:01:52.752480+0000","last_became_active":"2024-09-19T09:00:45.241032+0000","last_became_peered":"2024-09-19T09:00:45.241032+0000","last_unstale":"2024-09-19T09:01:52.752480+0000","last_undegraded":"2024-09-19T09:01:52.752480+0000","last_fullsized":"2024-09-19T09:01:52.752480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:00.707855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.626097+0000","last_change":"2024-09-19T09:00:46.962436+0000","last_active":"2024-09-19T09:02:21.626097+0000","last_peered":"2024-09-19T09:02:21.626097+0000","last_clean":"2024-09-19T09:02:21.626097+0000","last_became_active":"2024-09-19T09:00:46.962114+0000","last_became_peered":"2024-09-19T09:00:46.962114+0000","last_unstale":"2024-09-19T09:02:21.626097+0000","last_undegraded":"2024-09-19T09:02:21.626097+0000","last_fullsized":"2024-09-19T09:02:21.626097+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:41.761705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'37","reported_seq":273,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:40.544906+0000","last_change":"2024-09-19T09:00:44.945187+0000","last_active":"2024-09-19T09:02:40.544906+0000","last_peered":"2024-09-19T09:02:40.544906+0000","last_clean":"2024-09-19T09:02:40.544906+0000","last_became_active":"2024-09-19T09:00:44.944763+0000","last_became_peered":"2024-09-19T09:00:44.944763+0000","last_unstale":"2024-09-19T09:02:40.544906+0000","last_undegraded":"2024-09-19T09:02:40.544906+0000","last_fullsized":"2024-09-19T09:02:40.544906+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:06.052994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62867,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":214,"num_read_kb":168,"num_write":39,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.630413+0000","last_change":"2024-09-19T09:00:46.960991+0000","last_active":"2024-09-19T09:02:21.630413+0000","last_peered":"2024-09-19T09:02:21.630413+0000","last_clean":"2024-09-19T09:02:21.630413+0000","last_became_active":"2024-09-19T09:00:46.958962+0000","last_became_peered":"2024-09-19T09:00:46.958962+0000","last_unstale":"2024-09-19T09:02:21.630413+0000","last_undegraded":"2024-09-19T09:02:21.630413+0000","last_fullsized":"2024-09-19T09:02:21.630413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:21.795694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.755250+0000","last_change":"2024-09-19T09:00:44.946176+0000","last_active":"2024-09-19T09:01:52.755250+0000","last_peered":"2024-09-19T09:01:52.755250+0000","last_clean":"2024-09-19T09:01:52.755250+0000","last_became_active":"2024-09-19T09:00:44.945869+0000","last_became_peered":"2024-09-19T09:00:44.945869+0000","last_unstale":"2024-09-19T09:01:52.755250+0000","last_undegraded":"2024-09-19T09:01:52.755250+0000","last_fullsized":"2024-09-19T09:01:52.755250+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:16.022184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621664+0000","last_change":"2024-09-19T09:00:46.956345+0000","last_active":"2024-09-19T09:02:21.621664+0000","last_peered":"2024-09-19T09:02:21.621664+0000","last_clean":"2024-09-19T09:02:21.621664+0000","last_became_active":"2024-09-19T09:00:46.956057+0000","last_became_peered":"2024-09-19T09:00:46.956057+0000","last_unstale":"2024-09-19T09:02:21.621664+0000","last_undegraded":"2024-09-19T09:02:21.621664+0000","last_fullsized":"2024-09-19T09:02:21.621664+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:27.510661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752205+0000","last_change":"2024-09-19T09:00:44.945667+0000","last_active":"2024-09-19T09:01:52.752205+0000","last_peered":"2024-09-19T09:01:52.752205+0000","last_clean":"2024-09-19T09:01:52.752205+0000","last_became_active":"2024-09-19T09:00:44.945308+0000","last_became_peered":"2024-09-19T09:00:44.945308+0000","last_unstale":"2024-09-19T09:01:52.752205+0000","last_undegraded":"2024-09-19T09:01:52.752205+0000","last_fullsized":"2024-09-19T09:01:52.752205+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:00:11.304913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.629868+0000","last_change":"2024-09-19T09:00:46.965088+0000","last_active":"2024-09-19T09:02:21.629868+0000","last_peered":"2024-09-19T09:02:21.629868+0000","last_clean":"2024-09-19T09:02:21.629868+0000","last_became_active":"2024-09-19T09:00:46.964777+0000","last_became_peered":"2024-09-19T09:00:46.964777+0000","last_unstale":"2024-09-19T09:02:21.629868+0000","last_undegraded":"2024-09-19T09:02:21.629868+0000","last_fullsized":"2024-09-19T09:02:21.629868+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:32.958984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'7","reported_seq":137,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:41.284063+0000","last_change":"2024-09-19T09:00:44.946193+0000","last_active":"2024-09-19T09:02:41.284063+0000","last_peered":"2024-09-19T09:02:41.284063+0000","last_clean":"2024-09-19T09:02:41.284063+0000","last_became_active":"2024-09-19T09:00:44.945168+0000","last_became_peered":"2024-09-19T09:00:44.945168+0000","last_unstale":"2024-09-19T09:02:41.284063+0000","last_undegraded":"2024-09-19T09:02:41.284063+0000","last_fullsized":"2024-09-19T09:02:41.284063+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:21.230689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.627048+0000","last_change":"2024-09-19T09:01:52.754058+0000","last_active":"2024-09-19T09:02:21.627048+0000","last_peered":"2024-09-19T09:02:21.627048+0000","last_clean":"2024-09-19T09:02:21.627048+0000","last_became_active":"2024-09-19T09:01:52.753674+0000","last_became_peered":"2024-09-19T09:01:52.753674+0000","last_unstale":"2024-09-19T09:02:21.627048+0000","last_undegraded":"2024-09-19T09:02:21.627048+0000","last_fullsized":"2024-09-19T09:02:21.627048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:05.665121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":64,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.774430+0000","last_change":"2024-09-19T09:01:52.774430+0000","last_active":"2024-09-19T09:01:52.774430+0000","last_peered":"2024-09-19T09:01:52.774430+0000","last_clean":"2024-09-19T09:01:52.774430+0000","last_became_active":"2024-09-19T09:01:52.756083+0000","last_became_peered":"2024-09-19T09:01:52.756083+0000","last_unstale":"2024-09-19T09:01:52.774430+0000","last_undegraded":"2024-09-19T09:01:52.774430+0000","last_fullsized":"2024-09-19T09:01:52.774430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:52.639490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633068+0000","last_change":"2024-09-19T09:01:52.757684+0000","last_active":"2024-09-19T09:02:21.633068+0000","last_peered":"2024-09-19T09:02:21.633068+0000","last_clean":"2024-09-19T09:02:21.633068+0000","last_became_active":"2024-09-19T09:01:52.756969+0000","last_became_peered":"2024-09-19T09:01:52.756969+0000","last_unstale":"2024-09-19T09:02:21.633068+0000","last_undegraded":"2024-09-19T09:02:21.633068+0000","last_fullsized":"2024-09-19T09:02:21.633068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:24:47.117996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752185+0000","last_change":"2024-09-19T09:00:44.946138+0000","last_active":"2024-09-19T09:01:52.752185+0000","last_peered":"2024-09-19T09:01:52.752185+0000","last_clean":"2024-09-19T09:01:52.752185+0000","last_became_active":"2024-09-19T09:00:44.945793+0000","last_became_peered":"2024-09-19T09:00:44.945793+0000","last_unstale":"2024-09-19T09:01:52.752185+0000","last_undegraded":"2024-09-19T09:01:52.752185+0000","last_fullsized":"2024-09-19T09:01:52.752185+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:01.341560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620195+0000","last_change":"2024-09-19T09:00:46.970360+0000","last_active":"2024-09-19T09:02:21.620195+0000","last_peered":"2024-09-19T09:02:21.620195+0000","last_clean":"2024-09-19T09:02:21.620195+0000","last_became_active":"2024-09-19T09:00:46.969971+0000","last_became_peered":"2024-09-19T09:00:46.969971+0000","last_unstale":"2024-09-19T09:02:21.620195+0000","last_undegraded":"2024-09-19T09:02:21.620195+0000","last_fullsized":"2024-09-19T09:02:21.620195+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:48.812158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.748305+0000","last_change":"2024-09-19T09:00:44.941713+0000","last_active":"2024-09-19T09:01:52.748305+0000","last_peered":"2024-09-19T09:01:52.748305+0000","last_clean":"2024-09-19T09:01:52.748305+0000","last_became_active":"2024-09-19T09:00:44.941373+0000","last_became_peered":"2024-09-19T09:00:44.941373+0000","last_unstale":"2024-09-19T09:01:52.748305+0000","last_undegraded":"2024-09-19T09:01:52.748305+0000","last_fullsized":"2024-09-19T09:01:52.748305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:31.157803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.630100+0000","last_change":"2024-09-19T09:00:46.959383+0000","last_active":"2024-09-19T09:02:21.630100+0000","last_peered":"2024-09-19T09:02:21.630100+0000","last_clean":"2024-09-19T09:02:21.630100+0000","last_became_active":"2024-09-19T09:00:46.958860+0000","last_became_peered":"2024-09-19T09:00:46.958860+0000","last_unstale":"2024-09-19T09:02:21.630100+0000","last_undegraded":"2024-09-19T09:02:21.630100+0000","last_fullsized":"2024-09-19T09:02:21.630100+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:32:04.423017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752778+0000","last_change":"2024-09-19T09:00:44.944797+0000","last_active":"2024-09-19T09:01:52.752778+0000","last_peered":"2024-09-19T09:01:52.752778+0000","last_clean":"2024-09-19T09:01:52.752778+0000","last_became_active":"2024-09-19T09:00:44.944259+0000","last_became_peered":"2024-09-19T09:00:44.944259+0000","last_unstale":"2024-09-19T09:01:52.752778+0000","last_undegraded":"2024-09-19T09:01:52.752778+0000","last_fullsized":"2024-09-19T09:01:52.752778+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:58.996217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624674+0000","last_change":"2024-09-19T09:00:46.964998+0000","last_active":"2024-09-19T09:02:21.624674+0000","last_peered":"2024-09-19T09:02:21.624674+0000","last_clean":"2024-09-19T09:02:21.624674+0000","last_became_active":"2024-09-19T09:00:46.964562+0000","last_became_peered":"2024-09-19T09:00:46.964562+0000","last_unstale":"2024-09-19T09:02:21.624674+0000","last_undegraded":"2024-09-19T09:02:21.624674+0000","last_fullsized":"2024-09-19T09:02:21.624674+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:00.089800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":135,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:42.313326+0000","last_change":"2024-09-19T09:00:45.239770+0000","last_active":"2024-09-19T09:02:42.313326+0000","last_peered":"2024-09-19T09:02:42.313326+0000","last_clean":"2024-09-19T09:02:42.313326+0000","last_became_active":"2024-09-19T09:00:45.239385+0000","last_became_peered":"2024-09-19T09:00:45.239385+0000","last_unstale":"2024-09-19T09:02:42.313326+0000","last_undegraded":"2024-09-19T09:02:42.313326+0000","last_fullsized":"2024-09-19T09:02:42.313326+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:21.109900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.634073+0000","last_change":"2024-09-19T09:00:46.967505+0000","last_active":"2024-09-19T09:02:21.634073+0000","last_peered":"2024-09-19T09:02:21.634073+0000","last_clean":"2024-09-19T09:02:21.634073+0000","last_became_active":"2024-09-19T09:00:46.967195+0000","last_became_peered":"2024-09-19T09:00:46.967195+0000","last_unstale":"2024-09-19T09:02:21.634073+0000","last_undegraded":"2024-09-19T09:02:21.634073+0000","last_fullsized":"2024-09-19T09:02:21.634073+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:52:28.917333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":335,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751112+0000","last_change":"2024-09-19T08:59:33.821924+0000","last_active":"2024-09-19T09:01:52.751112+0000","last_peered":"2024-09-19T09:01:52.751112+0000","last_clean":"2024-09-19T09:01:52.751112+0000","last_became_active":"2024-09-19T08:59:33.821404+0000","last_became_peered":"2024-09-19T08:59:33.821404+0000","last_unstale":"2024-09-19T09:01:52.751112+0000","last_undegraded":"2024-09-19T09:01:52.751112+0000","last_fullsized":"2024-09-19T09:01:52.751112+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:56:07.830480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:56:07.830480+0000","last_clean_scrub_stamp":"2024-09-19T08:56:07.830480+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:21.763918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.888480+0000","last_change":"2024-09-19T09:00:44.943714+0000","last_active":"2024-09-19T09:01:52.888480+0000","last_peered":"2024-09-19T09:01:52.888480+0000","last_clean":"2024-09-19T09:01:52.888480+0000","last_became_active":"2024-09-19T09:00:44.943362+0000","last_became_peered":"2024-09-19T09:00:44.943362+0000","last_unstale":"2024-09-19T09:01:52.888480+0000","last_undegraded":"2024-09-19T09:01:52.888480+0000","last_fullsized":"2024-09-19T09:01:52.888480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:30.015648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633592+0000","last_change":"2024-09-19T09:00:46.956784+0000","last_active":"2024-09-19T09:02:21.633592+0000","last_peered":"2024-09-19T09:02:21.633592+0000","last_clean":"2024-09-19T09:02:21.633592+0000","last_became_active":"2024-09-19T09:00:46.956443+0000","last_became_peered":"2024-09-19T09:00:46.956443+0000","last_unstale":"2024-09-19T09:02:21.633592+0000","last_undegraded":"2024-09-19T09:02:21.633592+0000","last_fullsized":"2024-09-19T09:02:21.633592+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:18.918788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.750994+0000","last_change":"2024-09-19T09:00:44.947796+0000","last_active":"2024-09-19T09:01:52.750994+0000","last_peered":"2024-09-19T09:01:52.750994+0000","last_clean":"2024-09-19T09:01:52.750994+0000","last_became_active":"2024-09-19T09:00:44.946949+0000","last_became_peered":"2024-09-19T09:00:44.946949+0000","last_unstale":"2024-09-19T09:01:52.750994+0000","last_undegraded":"2024-09-19T09:01:52.750994+0000","last_fullsized":"2024-09-19T09:01:52.750994+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:42.210186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.619434+0000","last_change":"2024-09-19T09:00:46.963856+0000","last_active":"2024-09-19T09:02:21.619434+0000","last_peered":"2024-09-19T09:02:21.619434+0000","last_clean":"2024-09-19T09:02:21.619434+0000","last_became_active":"2024-09-19T09:00:46.963543+0000","last_became_peered":"2024-09-19T09:00:46.963543+0000","last_unstale":"2024-09-19T09:02:21.619434+0000","last_undegraded":"2024-09-19T09:02:21.619434+0000","last_fullsized":"2024-09-19T09:02:21.619434+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:31.041028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'6","reported_seq":134,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:42.284796+0000","last_change":"2024-09-19T09:00:45.239269+0000","last_active":"2024-09-19T09:02:42.284796+0000","last_peered":"2024-09-19T09:02:42.284796+0000","last_clean":"2024-09-19T09:02:42.284796+0000","last_became_active":"2024-09-19T09:00:45.238929+0000","last_became_peered":"2024-09-19T09:00:45.238929+0000","last_unstale":"2024-09-19T09:02:42.284796+0000","last_undegraded":"2024-09-19T09:02:42.284796+0000","last_fullsized":"2024-09-19T09:02:42.284796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:46.684148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622349+0000","last_change":"2024-09-19T09:00:46.963595+0000","last_active":"2024-09-19T09:02:21.622349+0000","last_peered":"2024-09-19T09:02:21.622349+0000","last_clean":"2024-09-19T09:02:21.622349+0000","last_became_active":"2024-09-19T09:00:46.963243+0000","last_became_peered":"2024-09-19T09:00:46.963243+0000","last_unstale":"2024-09-19T09:02:21.622349+0000","last_undegraded":"2024-09-19T09:02:21.622349+0000","last_fullsized":"2024-09-19T09:02:21.622349+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:01.122996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":133,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:42.850379+0000","last_change":"2024-09-19T09:00:44.941550+0000","last_active":"2024-09-19T09:02:42.850379+0000","last_peered":"2024-09-19T09:02:42.850379+0000","last_clean":"2024-09-19T09:02:42.850379+0000","last_became_active":"2024-09-19T09:00:44.941145+0000","last_became_peered":"2024-09-19T09:00:44.941145+0000","last_unstale":"2024-09-19T09:02:42.850379+0000","last_undegraded":"2024-09-19T09:02:42.850379+0000","last_fullsized":"2024-09-19T09:02:42.850379+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:53.893887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.612369+0000","last_change":"2024-09-19T09:00:46.963473+0000","last_active":"2024-09-19T09:02:21.612369+0000","last_peered":"2024-09-19T09:02:21.612369+0000","last_clean":"2024-09-19T09:02:21.612369+0000","last_became_active":"2024-09-19T09:00:46.963090+0000","last_became_peered":"2024-09-19T09:00:46.963090+0000","last_unstale":"2024-09-19T09:02:21.612369+0000","last_undegraded":"2024-09-19T09:02:21.612369+0000","last_fullsized":"2024-09-19T09:02:21.612369+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:01.492385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.750853+0000","last_change":"2024-09-19T09:00:45.239566+0000","last_active":"2024-09-19T09:01:52.750853+0000","last_peered":"2024-09-19T09:01:52.750853+0000","last_clean":"2024-09-19T09:01:52.750853+0000","last_became_active":"2024-09-19T09:00:45.239250+0000","last_became_peered":"2024-09-19T09:00:45.239250+0000","last_unstale":"2024-09-19T09:01:52.750853+0000","last_undegraded":"2024-09-19T09:01:52.750853+0000","last_fullsized":"2024-09-19T09:01:52.750853+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:44:21.739311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.614525+0000","last_change":"2024-09-19T09:00:46.955344+0000","last_active":"2024-09-19T09:02:21.614525+0000","last_peered":"2024-09-19T09:02:21.614525+0000","last_clean":"2024-09-19T09:02:21.614525+0000","last_became_active":"2024-09-19T09:00:46.954961+0000","last_became_peered":"2024-09-19T09:00:46.954961+0000","last_unstale":"2024-09-19T09:02:21.614525+0000","last_undegraded":"2024-09-19T09:02:21.614525+0000","last_fullsized":"2024-09-19T09:02:21.614525+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:55.743556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752072+0000","last_change":"2024-09-19T09:00:45.241514+0000","last_active":"2024-09-19T09:01:52.752072+0000","last_peered":"2024-09-19T09:01:52.752072+0000","last_clean":"2024-09-19T09:01:52.752072+0000","last_became_active":"2024-09-19T09:00:45.241097+0000","last_became_peered":"2024-09-19T09:00:45.241097+0000","last_unstale":"2024-09-19T09:01:52.752072+0000","last_undegraded":"2024-09-19T09:01:52.752072+0000","last_fullsized":"2024-09-19T09:01:52.752072+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:35:38.323393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620312+0000","last_change":"2024-09-19T09:00:46.955194+0000","last_active":"2024-09-19T09:02:21.620312+0000","last_peered":"2024-09-19T09:02:21.620312+0000","last_clean":"2024-09-19T09:02:21.620312+0000","last_became_active":"2024-09-19T09:00:46.954771+0000","last_became_peered":"2024-09-19T09:00:46.954771+0000","last_unstale":"2024-09-19T09:02:21.620312+0000","last_undegraded":"2024-09-19T09:02:21.620312+0000","last_fullsized":"2024-09-19T09:02:21.620312+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:18.945035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'16","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.975541+0000","last_change":"2024-09-19T09:00:44.938554+0000","last_active":"2024-09-19T09:02:21.975541+0000","last_peered":"2024-09-19T09:02:21.975541+0000","last_clean":"2024-09-19T09:02:21.975541+0000","last_became_active":"2024-09-19T09:00:44.938261+0000","last_became_peered":"2024-09-19T09:00:44.938261+0000","last_unstale":"2024-09-19T09:02:21.975541+0000","last_undegraded":"2024-09-19T09:02:21.975541+0000","last_fullsized":"2024-09-19T09:02:21.975541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:24.293764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.614207+0000","last_change":"2024-09-19T09:01:52.755528+0000","last_active":"2024-09-19T09:02:21.614207+0000","last_peered":"2024-09-19T09:02:21.614207+0000","last_clean":"2024-09-19T09:02:21.614207+0000","last_became_active":"2024-09-19T09:01:52.755188+0000","last_became_peered":"2024-09-19T09:01:52.755188+0000","last_unstale":"2024-09-19T09:02:21.614207+0000","last_undegraded":"2024-09-19T09:02:21.614207+0000","last_fullsized":"2024-09-19T09:02:21.614207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:49:34.988733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.888813+0000","last_change":"2024-09-19T09:00:44.941479+0000","last_active":"2024-09-19T09:01:52.888813+0000","last_peered":"2024-09-19T09:01:52.888813+0000","last_clean":"2024-09-19T09:01:52.888813+0000","last_became_active":"2024-09-19T09:00:44.937197+0000","last_became_peered":"2024-09-19T09:00:44.937197+0000","last_unstale":"2024-09-19T09:01:52.888813+0000","last_undegraded":"2024-09-19T09:01:52.888813+0000","last_fullsized":"2024-09-19T09:01:52.888813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:53:17.385884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621026+0000","last_change":"2024-09-19T09:00:46.970319+0000","last_active":"2024-09-19T09:02:21.621026+0000","last_peered":"2024-09-19T09:02:21.621026+0000","last_clean":"2024-09-19T09:02:21.621026+0000","last_became_active":"2024-09-19T09:00:46.969899+0000","last_became_peered":"2024-09-19T09:00:46.969899+0000","last_unstale":"2024-09-19T09:02:21.621026+0000","last_undegraded":"2024-09-19T09:02:21.621026+0000","last_fullsized":"2024-09-19T09:02:21.621026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:26:21.379231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751656+0000","last_change":"2024-09-19T09:00:45.239525+0000","last_active":"2024-09-19T09:01:52.751656+0000","last_peered":"2024-09-19T09:01:52.751656+0000","last_clean":"2024-09-19T09:01:52.751656+0000","last_became_active":"2024-09-19T09:00:45.239213+0000","last_became_peered":"2024-09-19T09:00:45.239213+0000","last_unstale":"2024-09-19T09:01:52.751656+0000","last_undegraded":"2024-09-19T09:01:52.751656+0000","last_fullsized":"2024-09-19T09:01:52.751656+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:32.525025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.584763+0000","last_change":"2024-09-19T09:00:46.958573+0000","last_active":"2024-09-19T09:02:21.584763+0000","last_peered":"2024-09-19T09:02:21.584763+0000","last_clean":"2024-09-19T09:02:21.584763+0000","last_became_active":"2024-09-19T09:00:46.958274+0000","last_became_peered":"2024-09-19T09:00:46.958274+0000","last_unstale":"2024-09-19T09:02:21.584763+0000","last_undegraded":"2024-09-19T09:02:21.584763+0000","last_fullsized":"2024-09-19T09:02:21.584763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:17.249122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.903274+0000","last_change":"2024-09-19T09:00:44.945194+0000","last_active":"2024-09-19T09:01:52.903274+0000","last_peered":"2024-09-19T09:01:52.903274+0000","last_clean":"2024-09-19T09:01:52.903274+0000","last_became_active":"2024-09-19T09:00:44.944931+0000","last_became_peered":"2024-09-19T09:00:44.944931+0000","last_unstale":"2024-09-19T09:01:52.903274+0000","last_undegraded":"2024-09-19T09:01:52.903274+0000","last_fullsized":"2024-09-19T09:01:52.903274+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:11.181150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621916+0000","last_change":"2024-09-19T09:00:46.966895+0000","last_active":"2024-09-19T09:02:21.621916+0000","last_peered":"2024-09-19T09:02:21.621916+0000","last_clean":"2024-09-19T09:02:21.621916+0000","last_became_active":"2024-09-19T09:00:46.966124+0000","last_became_peered":"2024-09-19T09:00:46.966124+0000","last_unstale":"2024-09-19T09:02:21.621916+0000","last_undegraded":"2024-09-19T09:02:21.621916+0000","last_fullsized":"2024-09-19T09:02:21.621916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:14:04.320863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:41.120672+0000","last_change":"2024-09-19T09:00:44.944167+0000","last_active":"2024-09-19T09:02:41.120672+0000","last_peered":"2024-09-19T09:02:41.120672+0000","last_clean":"2024-09-19T09:02:41.120672+0000","last_became_active":"2024-09-19T09:00:44.943670+0000","last_became_peered":"2024-09-19T09:00:44.943670+0000","last_unstale":"2024-09-19T09:02:41.120672+0000","last_undegraded":"2024-09-19T09:02:41.120672+0000","last_fullsized":"2024-09-19T09:02:41.120672+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:05.596877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.615278+0000","last_change":"2024-09-19T09:00:46.958581+0000","last_active":"2024-09-19T09:02:21.615278+0000","last_peered":"2024-09-19T09:02:21.615278+0000","last_clean":"2024-09-19T09:02:21.615278+0000","last_became_active":"2024-09-19T09:00:46.957782+0000","last_became_peered":"2024-09-19T09:00:46.957782+0000","last_unstale":"2024-09-19T09:02:21.615278+0000","last_undegraded":"2024-09-19T09:02:21.615278+0000","last_fullsized":"2024-09-19T09:02:21.615278+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:37:19.488538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751779+0000","last_change":"2024-09-19T09:00:44.945351+0000","last_active":"2024-09-19T09:01:52.751779+0000","last_peered":"2024-09-19T09:01:52.751779+0000","last_clean":"2024-09-19T09:01:52.751779+0000","last_became_active":"2024-09-19T09:00:44.944991+0000","last_became_peered":"2024-09-19T09:00:44.944991+0000","last_unstale":"2024-09-19T09:01:52.751779+0000","last_undegraded":"2024-09-19T09:01:52.751779+0000","last_fullsized":"2024-09-19T09:01:52.751779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:45:07.581872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633316+0000","last_change":"2024-09-19T09:00:46.956340+0000","last_active":"2024-09-19T09:02:21.633316+0000","last_peered":"2024-09-19T09:02:21.633316+0000","last_clean":"2024-09-19T09:02:21.633316+0000","last_became_active":"2024-09-19T09:00:46.955956+0000","last_became_peered":"2024-09-19T09:00:46.955956+0000","last_unstale":"2024-09-19T09:02:21.633316+0000","last_undegraded":"2024-09-19T09:02:21.633316+0000","last_fullsized":"2024-09-19T09:02:21.633316+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:34.863428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633515+0000","last_change":"2024-09-19T09:00:46.963657+0000","last_active":"2024-09-19T09:02:21.633515+0000","last_peered":"2024-09-19T09:02:21.633515+0000","last_clean":"2024-09-19T09:02:21.633515+0000","last_became_active":"2024-09-19T09:00:46.963134+0000","last_became_peered":"2024-09-19T09:00:46.963134+0000","last_unstale":"2024-09-19T09:02:21.633515+0000","last_undegraded":"2024-09-19T09:02:21.633515+0000","last_fullsized":"2024-09-19T09:02:21.633515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:01.098570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.764939+0000","last_change":"2024-09-19T09:01:52.764438+0000","last_active":"2024-09-19T09:01:52.764939+0000","last_peered":"2024-09-19T09:01:52.764939+0000","last_clean":"2024-09-19T09:01:52.764939+0000","last_became_active":"2024-09-19T09:01:52.755188+0000","last_became_peered":"2024-09-19T09:01:52.755188+0000","last_unstale":"2024-09-19T09:01:52.764939+0000","last_undegraded":"2024-09-19T09:01:52.764939+0000","last_fullsized":"2024-09-19T09:01:52.764939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:40.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.632994+0000","last_change":"2024-09-19T09:00:46.956354+0000","last_active":"2024-09-19T09:02:21.632994+0000","last_peered":"2024-09-19T09:02:21.632994+0000","last_clean":"2024-09-19T09:02:21.632994+0000","last_became_active":"2024-09-19T09:00:46.955964+0000","last_became_peered":"2024-09-19T09:00:46.955964+0000","last_unstale":"2024-09-19T09:02:21.632994+0000","last_undegraded":"2024-09-19T09:02:21.632994+0000","last_fullsized":"2024-09-19T09:02:21.632994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:05:33.243479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751944+0000","last_change":"2024-09-19T09:00:44.944983+0000","last_active":"2024-09-19T09:01:52.751944+0000","last_peered":"2024-09-19T09:01:52.751944+0000","last_clean":"2024-09-19T09:01:52.751944+0000","last_became_active":"2024-09-19T09:00:44.944625+0000","last_became_peered":"2024-09-19T09:00:44.944625+0000","last_unstale":"2024-09-19T09:01:52.751944+0000","last_undegraded":"2024-09-19T09:01:52.751944+0000","last_fullsized":"2024-09-19T09:01:52.751944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:49:50.818863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.619748+0000","last_change":"2024-09-19T09:00:46.966448+0000","last_active":"2024-09-19T09:02:21.619748+0000","last_peered":"2024-09-19T09:02:21.619748+0000","last_clean":"2024-09-19T09:02:21.619748+0000","last_became_active":"2024-09-19T09:00:46.966135+0000","last_became_peered":"2024-09-19T09:00:46.966135+0000","last_unstale":"2024-09-19T09:02:21.619748+0000","last_undegraded":"2024-09-19T09:02:21.619748+0000","last_fullsized":"2024-09-19T09:02:21.619748+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:42.652877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.903626+0000","last_change":"2024-09-19T09:00:45.239405+0000","last_active":"2024-09-19T09:01:52.903626+0000","last_peered":"2024-09-19T09:01:52.903626+0000","last_clean":"2024-09-19T09:01:52.903626+0000","last_became_active":"2024-09-19T09:00:45.239002+0000","last_became_peered":"2024-09-19T09:00:45.239002+0000","last_unstale":"2024-09-19T09:01:52.903626+0000","last_undegraded":"2024-09-19T09:01:52.903626+0000","last_fullsized":"2024-09-19T09:01:52.903626+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:46:03.934975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621992+0000","last_change":"2024-09-19T09:00:46.951635+0000","last_active":"2024-09-19T09:02:21.621992+0000","last_peered":"2024-09-19T09:02:21.621992+0000","last_clean":"2024-09-19T09:02:21.621992+0000","last_became_active":"2024-09-19T09:00:46.951325+0000","last_became_peered":"2024-09-19T09:00:46.951325+0000","last_unstale":"2024-09-19T09:02:21.621992+0000","last_undegraded":"2024-09-19T09:02:21.621992+0000","last_fullsized":"2024-09-19T09:02:21.621992+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:17.360950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":158,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:39.845934+0000","last_change":"2024-09-19T09:01:52.766049+0000","last_active":"2024-09-19T09:02:39.845934+0000","last_peered":"2024-09-19T09:02:39.845934+0000","last_clean":"2024-09-19T09:02:39.845934+0000","last_became_active":"2024-09-19T09:01:52.755263+0000","last_became_peered":"2024-09-19T09:01:52.755263+0000","last_unstale":"2024-09-19T09:02:39.845934+0000","last_undegraded":"2024-09-19T09:02:39.845934+0000","last_fullsized":"2024-09-19T09:02:39.845934+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:16.968329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70524,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1072,"num_read_kb":609,"num_write":261,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1366,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":220593,"data_compressed":4815,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":212166,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612644,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":429375,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841577,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":1082243,"data_compressed":43295,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972399,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":428051,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201332,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":427847,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430266,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":1020726,"data_compressed":41690,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593791,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":427768,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822723,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":1020542,"data_compressed":41690,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986249,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":428163,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215181,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":427593,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":497064,"data_compressed":7388,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":966656,"data_stored":496367,"data_compressed":7388,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":429500,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":69438,"data_compressed":1605,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":70135,"data_compressed":1605,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65137,"data_compressed":1605,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:02:46.042 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:02:46.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:45 smithi196 ceph-mon[35150]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-19T09:02:46.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:45 smithi153 ceph-mon[35071]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-19T09:02:46.323 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:46.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.774+0000 7f2e8cc74640 1 -- 172.21.15.120:0/2553271623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e88101de0 msgr2=0x7f2e881021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:46.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.774+0000 7f2e8cc74640 1 --2- 172.21.15.120:0/2553271623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e88101de0 0x7f2e881021c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2e78009900 tx=0x7f2e7802a410 comp rx=0 tx=0).stop 2024-09-19T09:02:46.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.775+0000 7f2e8cc74640 1 -- 172.21.15.120:0/2553271623 shutdown_connections 2024-09-19T09:02:46.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.775+0000 7f2e8cc74640 1 --2- 172.21.15.120:0/2553271623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e88106f10 0x7f2e8810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:46.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.775+0000 7f2e8cc74640 1 --2- 172.21.15.120:0/2553271623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e88102790 0x7f2e881067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:46.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.775+0000 7f2e8cc74640 1 --2- 172.21.15.120:0/2553271623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e88101de0 0x7f2e881021c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:46.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.775+0000 7f2e8cc74640 1 -- 172.21.15.120:0/2553271623 >> 172.21.15.120:0/2553271623 conn(0x7f2e880fd8c0 msgr2=0x7f2e880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:46.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.775+0000 7f2e8cc74640 1 -- 172.21.15.120:0/2553271623 shutdown_connections 2024-09-19T09:02:46.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.775+0000 7f2e8cc74640 1 -- 172.21.15.120:0/2553271623 wait complete. 2024-09-19T09:02:46.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.775+0000 7f2e8cc74640 1 Processor -- start 2024-09-19T09:02:46.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.775+0000 7f2e8cc74640 1 -- start start 2024-09-19T09:02:46.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.776+0000 7f2e8cc74640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e88101de0 0x7f2e881985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:46.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.776+0000 7f2e8cc74640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e88102790 0x7f2e88198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:46.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.776+0000 7f2e8cc74640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e88106f10 0x7f2e8819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:46.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.776+0000 7f2e8cc74640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2e88112010 con 0x7f2e88106f10 2024-09-19T09:02:46.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.776+0000 7f2e8cc74640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2e88111e90 con 0x7f2e88101de0 2024-09-19T09:02:46.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.776+0000 7f2e8cc74640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2e88112190 con 0x7f2e88102790 2024-09-19T09:02:46.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.776+0000 7f2e86d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e88106f10 0x7f2e8819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:46.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.776+0000 7f2e86d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e88106f10 0x7f2e8819ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48488/0 (socket says 172.21.15.120:48488) 2024-09-19T09:02:46.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.776+0000 7f2e86d9d640 1 -- 172.21.15.120:0/603628758 learned_addr learned my addr 172.21.15.120:0/603628758 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:46.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.776+0000 7f2e85d9b640 1 --2- 172.21.15.120:0/603628758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e88102790 0x7f2e88198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:46.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.776+0000 7f2e8659c640 1 --2- 172.21.15.120:0/603628758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e88101de0 0x7f2e881985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:46.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.776+0000 7f2e86d9d640 1 -- 172.21.15.120:0/603628758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e88102790 msgr2=0x7f2e88198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:46.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.776+0000 7f2e86d9d640 1 --2- 172.21.15.120:0/603628758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e88102790 0x7f2e88198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:46.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.776+0000 7f2e86d9d640 1 -- 172.21.15.120:0/603628758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e88101de0 msgr2=0x7f2e881985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:46.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.776+0000 7f2e86d9d640 1 --2- 172.21.15.120:0/603628758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e88101de0 0x7f2e881985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:46.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.776+0000 7f2e86d9d640 1 -- 172.21.15.120:0/603628758 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8819d560 con 0x7f2e88106f10 2024-09-19T09:02:46.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.776+0000 7f2e8659c640 1 --2- 172.21.15.120:0/603628758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e88101de0 0x7f2e881985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:02:46.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.776+0000 7f2e86d9d640 1 --2- 172.21.15.120:0/603628758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e88106f10 0x7f2e8819ce80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000b970 tx=0x7f2e8000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:46.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.776+0000 7f2e777fe640 1 -- 172.21.15.120:0/603628758 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e80019070 con 0x7f2e88106f10 2024-09-19T09:02:46.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.776+0000 7f2e8cc74640 1 -- 172.21.15.120:0/603628758 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e8819d850 con 0x7f2e88106f10 2024-09-19T09:02:46.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.776+0000 7f2e777fe640 1 -- 172.21.15.120:0/603628758 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e8000cb40 con 0x7f2e88106f10 2024-09-19T09:02:46.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.776+0000 7f2e777fe640 1 -- 172.21.15.120:0/603628758 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e80007650 con 0x7f2e88106f10 2024-09-19T09:02:46.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.776+0000 7f2e8cc74640 1 -- 172.21.15.120:0/603628758 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e881a5130 con 0x7f2e88106f10 2024-09-19T09:02:46.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.777+0000 7f2e8cc74640 1 -- 172.21.15.120:0/603628758 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e88102240 con 0x7f2e88106f10 2024-09-19T09:02:46.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.778+0000 7f2e777fe640 1 -- 172.21.15.120:0/603628758 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f2e8001c070 con 0x7f2e88106f10 2024-09-19T09:02:46.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.779+0000 7f2e777fe640 1 --2- 172.21.15.120:0/603628758 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2e700a32d0 0x7f2e700a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:46.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.779+0000 7f2e8659c640 1 --2- 172.21.15.120:0/603628758 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2e700a32d0 0x7f2e700a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:46.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.779+0000 7f2e777fe640 1 -- 172.21.15.120:0/603628758 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f2e800cefa0 con 0x7f2e88106f10 2024-09-19T09:02:46.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.779+0000 7f2e8659c640 1 --2- 172.21.15.120:0/603628758 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2e700a32d0 0x7f2e700a5790 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2e78000c00 tx=0x7f2e78032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:46.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.783+0000 7f2e777fe640 1 -- 172.21.15.120:0/603628758 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2e8009bc70 con 0x7f2e88106f10 2024-09-19T09:02:46.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.987+0000 7f2e8cc74640 1 -- 172.21.15.120:0/603628758 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e8810a150 con 0x7f2e88106f10 2024-09-19T09:02:46.988 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:46 smithi120 ceph-mon[31150]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:02:46.988 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:46 smithi120 ceph-mon[31150]: pgmap v338: 97 pgs: 97 active+clean; 646 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-19T09:02:46.988 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:46 smithi120 ceph-mon[31150]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:02:46.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.988+0000 7f2e777fe640 1 -- 172.21.15.120:0/603628758 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f2e800bb070 con 0x7f2e88106f10 2024-09-19T09:02:46.988 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:02:46.989 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":28,"btime":"2024-09-19T09:02:41:696728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:02:41.207357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:02:46.989 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 28 2024-09-19T09:02:46.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.990+0000 7f2e8cc74640 1 -- 172.21.15.120:0/603628758 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2e700a32d0 msgr2=0x7f2e700a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:46.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.990+0000 7f2e8cc74640 1 --2- 172.21.15.120:0/603628758 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2e700a32d0 0x7f2e700a5790 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2e78000c00 tx=0x7f2e78032d50 comp rx=0 tx=0).stop 2024-09-19T09:02:46.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.990+0000 7f2e8cc74640 1 -- 172.21.15.120:0/603628758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e88106f10 msgr2=0x7f2e8819ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:46.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.990+0000 7f2e8cc74640 1 --2- 172.21.15.120:0/603628758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e88106f10 0x7f2e8819ce80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000b970 tx=0x7f2e8000be30 comp rx=0 tx=0).stop 2024-09-19T09:02:46.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.990+0000 7f2e8cc74640 1 -- 172.21.15.120:0/603628758 shutdown_connections 2024-09-19T09:02:46.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.990+0000 7f2e8cc74640 1 --2- 172.21.15.120:0/603628758 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2e700a32d0 0x7f2e700a5790 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:46.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.990+0000 7f2e8cc74640 1 --2- 172.21.15.120:0/603628758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e88106f10 0x7f2e8819ce80 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:46.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.990+0000 7f2e8cc74640 1 --2- 172.21.15.120:0/603628758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e88102790 0x7f2e88198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:46.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.990+0000 7f2e8cc74640 1 --2- 172.21.15.120:0/603628758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e88101de0 0x7f2e881985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:46.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.990+0000 7f2e8cc74640 1 -- 172.21.15.120:0/603628758 >> 172.21.15.120:0/603628758 conn(0x7f2e880fd8c0 msgr2=0x7f2e880fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:46.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.990+0000 7f2e8cc74640 1 -- 172.21.15.120:0/603628758 shutdown_connections 2024-09-19T09:02:46.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:46.990+0000 7f2e8cc74640 1 -- 172.21.15.120:0/603628758 wait complete. 2024-09-19T09:02:47.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:46 smithi196 ceph-mon[35150]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:02:47.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:46 smithi196 ceph-mon[35150]: pgmap v338: 97 pgs: 97 active+clean; 646 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-19T09:02:47.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:46 smithi196 ceph-mon[35150]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:02:47.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:46 smithi153 ceph-mon[35071]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:02:47.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:46 smithi153 ceph-mon[35071]: pgmap v338: 97 pgs: 97 active+clean; 646 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-19T09:02:47.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:46 smithi153 ceph-mon[35071]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:02:47.812 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:02:48.059 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:48.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:47 smithi120 ceph-mon[31150]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:02:48.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:47 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/603628758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:02:48.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:47 smithi196 ceph-mon[35150]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:02:48.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:47 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/603628758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:02:48.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:47 smithi153 ceph-mon[35071]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:02:48.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:47 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/603628758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:02:48.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.508+0000 7f73d9d0d640 1 -- 172.21.15.120:0/3047885492 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d4106f10 msgr2=0x7f73d410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:48.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.508+0000 7f73d9d0d640 1 --2- 172.21.15.120:0/3047885492 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d4106f10 0x7f73d410ea90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009ca0 tx=0x7f73cc02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:02:48.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.509+0000 7f73d9d0d640 1 -- 172.21.15.120:0/3047885492 shutdown_connections 2024-09-19T09:02:48.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.509+0000 7f73d9d0d640 1 --2- 172.21.15.120:0/3047885492 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d4106f10 0x7f73d410ea90 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:48.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.509+0000 7f73d9d0d640 1 --2- 172.21.15.120:0/3047885492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73d4102790 0x7f73d41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:48.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.509+0000 7f73d9d0d640 1 --2- 172.21.15.120:0/3047885492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73d4101de0 0x7f73d41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:48.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.509+0000 7f73d9d0d640 1 -- 172.21.15.120:0/3047885492 >> 172.21.15.120:0/3047885492 conn(0x7f73d40fd8c0 msgr2=0x7f73d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:48.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.509+0000 7f73d9d0d640 1 -- 172.21.15.120:0/3047885492 shutdown_connections 2024-09-19T09:02:48.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.509+0000 7f73d9d0d640 1 -- 172.21.15.120:0/3047885492 wait complete. 2024-09-19T09:02:48.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.509+0000 7f73d9d0d640 1 Processor -- start 2024-09-19T09:02:48.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.509+0000 7f73d9d0d640 1 -- start start 2024-09-19T09:02:48.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.509+0000 7f73d9d0d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d4101de0 0x7f73d4198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:48.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.510+0000 7f73d9d0d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73d4102790 0x7f73d4198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:48.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.510+0000 7f73d9d0d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73d4106f10 0x7f73d419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:48.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.510+0000 7f73d9d0d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f73d4111f70 con 0x7f73d4101de0 2024-09-19T09:02:48.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.510+0000 7f73d9d0d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f73d4111df0 con 0x7f73d4102790 2024-09-19T09:02:48.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.510+0000 7f73d9d0d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f73d41120f0 con 0x7f73d4106f10 2024-09-19T09:02:48.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.510+0000 7f73d37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d4101de0 0x7f73d4198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:48.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.510+0000 7f73d37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d4101de0 0x7f73d4198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48512/0 (socket says 172.21.15.120:48512) 2024-09-19T09:02:48.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.510+0000 7f73d37fe640 1 -- 172.21.15.120:0/1947746436 learned_addr learned my addr 172.21.15.120:0/1947746436 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:48.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.510+0000 7f73caffd640 1 --2- 172.21.15.120:0/1947746436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73d4102790 0x7f73d4198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:48.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.510+0000 7f73d3fff640 1 --2- 172.21.15.120:0/1947746436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73d4106f10 0x7f73d419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:48.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.510+0000 7f73d37fe640 1 -- 172.21.15.120:0/1947746436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73d4106f10 msgr2=0x7f73d419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:48.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.510+0000 7f73d37fe640 1 --2- 172.21.15.120:0/1947746436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73d4106f10 0x7f73d419cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:48.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.510+0000 7f73d37fe640 1 -- 172.21.15.120:0/1947746436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73d4102790 msgr2=0x7f73d4198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:48.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.510+0000 7f73d37fe640 1 --2- 172.21.15.120:0/1947746436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73d4102790 0x7f73d4198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:48.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.510+0000 7f73d37fe640 1 -- 172.21.15.120:0/1947746436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d419d2e0 con 0x7f73d4101de0 2024-09-19T09:02:48.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.510+0000 7f73d3fff640 1 --2- 172.21.15.120:0/1947746436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73d4106f10 0x7f73d419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:02:48.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.510+0000 7f73d37fe640 1 --2- 172.21.15.120:0/1947746436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d4101de0 0x7f73d4198330 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f73c400c920 tx=0x7f73c400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:48.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.510+0000 7f73d17fa640 1 -- 172.21.15.120:0/1947746436 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c4007c70 con 0x7f73d4101de0 2024-09-19T09:02:48.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.510+0000 7f73d17fa640 1 -- 172.21.15.120:0/1947746436 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73c4007e10 con 0x7f73d4101de0 2024-09-19T09:02:48.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.510+0000 7f73d9d0d640 1 -- 172.21.15.120:0/1947746436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73d419d5d0 con 0x7f73d4101de0 2024-09-19T09:02:48.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.510+0000 7f73d17fa640 1 -- 172.21.15.120:0/1947746436 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c4018890 con 0x7f73d4101de0 2024-09-19T09:02:48.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.510+0000 7f73d9d0d640 1 -- 172.21.15.120:0/1947746436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73d41a4eb0 con 0x7f73d4101de0 2024-09-19T09:02:48.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.511+0000 7f73d9d0d640 1 -- 172.21.15.120:0/1947746436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73d41021c0 con 0x7f73d4101de0 2024-09-19T09:02:48.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.513+0000 7f73d17fa640 1 -- 172.21.15.120:0/1947746436 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f73c4018a70 con 0x7f73d4101de0 2024-09-19T09:02:48.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.513+0000 7f73d17fa640 1 --2- 172.21.15.120:0/1947746436 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f73c00a32d0 0x7f73c00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:48.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.513+0000 7f73caffd640 1 --2- 172.21.15.120:0/1947746436 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f73c00a32d0 0x7f73c00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:48.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.514+0000 7f73d17fa640 1 -- 172.21.15.120:0/1947746436 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f73c40cfbf0 con 0x7f73d4101de0 2024-09-19T09:02:48.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.514+0000 7f73caffd640 1 --2- 172.21.15.120:0/1947746436 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f73c00a32d0 0x7f73c00a5790 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f73d4108880 tx=0x7f73bc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:48.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.517+0000 7f73d17fa640 1 -- 172.21.15.120:0/1947746436 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f73c4014030 con 0x7f73d4101de0 2024-09-19T09:02:48.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.685+0000 7f73d9d0d640 1 -- 172.21.15.120:0/1947746436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73d4199690 con 0x7f73d4101de0 2024-09-19T09:02:48.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.686+0000 7f73d17fa640 1 -- 172.21.15.120:0/1947746436 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22844 (secure 0 0 0) 0x7f73c409c7b0 con 0x7f73d4101de0 2024-09-19T09:02:48.687 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:02:48.687 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":81,"fsid":"507393d2-7664-11ef-bceb-c7b262605968","created":"2024-09-19T08:53:06.386526+0000","modified":"2024-09-19T09:01:52.744203+0000","last_up_change":"2024-09-19T08:59:55.692659+0000","last_in_change":"2024-09-19T08:59:32.808860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:56:07.830047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:00:43.774339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:00:45.786262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33ec3993-576c-428c-bf75-adcb17cd72a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6803","nonce":4131180019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6805","nonce":4131180019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6809","nonce":4131180019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6807","nonce":4131180019}]},"public_addr":"172.21.15.120:6803/4131180019","cluster_addr":"172.21.15.120:6805/4131180019","heartbeat_back_addr":"172.21.15.120:6809/4131180019","heartbeat_front_addr":"172.21.15.120:6807/4131180019","state":["exists","up"]},{"osd":1,"uuid":"450bbebf-2aac-47c6-8474-bbd37fd0a1a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6801","nonce":2259730294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6803","nonce":2259730294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6807","nonce":2259730294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6805","nonce":2259730294}]},"public_addr":"172.21.15.153:6801/2259730294","cluster_addr":"172.21.15.153:6803/2259730294","heartbeat_back_addr":"172.21.15.153:6807/2259730294","heartbeat_front_addr":"172.21.15.153:6805/2259730294","state":["exists","up"]},{"osd":2,"uuid":"ac8c4b83-8e8e-4803-b266-262f7873bd85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6801","nonce":2942286570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6803","nonce":2942286570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6807","nonce":2942286570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6805","nonce":2942286570}]},"public_addr":"172.21.15.196:6801/2942286570","cluster_addr":"172.21.15.196:6803/2942286570","heartbeat_back_addr":"172.21.15.196:6807/2942286570","heartbeat_front_addr":"172.21.15.196:6805/2942286570","state":["exists","up"]},{"osd":3,"uuid":"c1d9e507-8f02-403e-87da-9b4966fd8fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6811","nonce":713205668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6813","nonce":713205668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6817","nonce":713205668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6815","nonce":713205668}]},"public_addr":"172.21.15.120:6811/713205668","cluster_addr":"172.21.15.120:6813/713205668","heartbeat_back_addr":"172.21.15.120:6817/713205668","heartbeat_front_addr":"172.21.15.120:6815/713205668","state":["exists","up"]},{"osd":4,"uuid":"7f8bf3b9-0ec4-46f6-a406-d26d7971179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6809","nonce":2680865854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6811","nonce":2680865854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6815","nonce":2680865854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6813","nonce":2680865854}]},"public_addr":"172.21.15.153:6809/2680865854","cluster_addr":"172.21.15.153:6811/2680865854","heartbeat_back_addr":"172.21.15.153:6815/2680865854","heartbeat_front_addr":"172.21.15.153:6813/2680865854","state":["exists","up"]},{"osd":5,"uuid":"70b270b2-209d-4ff5-875a-04ee7b992a93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6809","nonce":3763331116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6811","nonce":3763331116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6815","nonce":3763331116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6813","nonce":3763331116}]},"public_addr":"172.21.15.196:6809/3763331116","cluster_addr":"172.21.15.196:6811/3763331116","heartbeat_back_addr":"172.21.15.196:6815/3763331116","heartbeat_front_addr":"172.21.15.196:6813/3763331116","state":["exists","up"]},{"osd":6,"uuid":"8087f69c-dc31-4522-a18d-4e6ab1b7cb27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6819","nonce":4171773765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6821","nonce":4171773765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6825","nonce":4171773765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6823","nonce":4171773765}]},"public_addr":"172.21.15.120:6819/4171773765","cluster_addr":"172.21.15.120:6821/4171773765","heartbeat_back_addr":"172.21.15.120:6825/4171773765","heartbeat_front_addr":"172.21.15.120:6823/4171773765","state":["exists","up"]},{"osd":7,"uuid":"c3b1ead3-150f-43fd-8924-f639203f4cdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6817","nonce":2535148453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6819","nonce":2535148453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6823","nonce":2535148453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6821","nonce":2535148453}]},"public_addr":"172.21.15.153:6817/2535148453","cluster_addr":"172.21.15.153:6819/2535148453","heartbeat_back_addr":"172.21.15.153:6823/2535148453","heartbeat_front_addr":"172.21.15.153:6821/2535148453","state":["exists","up"]},{"osd":8,"uuid":"a3273a2a-968e-4e93-a9cf-3879892bdeab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6817","nonce":186195906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6819","nonce":186195906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6823","nonce":186195906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6821","nonce":186195906}]},"public_addr":"172.21.15.196:6817/186195906","cluster_addr":"172.21.15.196:6819/186195906","heartbeat_back_addr":"172.21.15.196:6823/186195906","heartbeat_front_addr":"172.21.15.196:6821/186195906","state":["exists","up"]},{"osd":9,"uuid":"6d184513-dab8-45f7-bc08-fa2e8b2f10b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6827","nonce":1559320883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6829","nonce":1559320883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6833","nonce":1559320883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6831","nonce":1559320883}]},"public_addr":"172.21.15.120:6827/1559320883","cluster_addr":"172.21.15.120:6829/1559320883","heartbeat_back_addr":"172.21.15.120:6833/1559320883","heartbeat_front_addr":"172.21.15.120:6831/1559320883","state":["exists","up"]},{"osd":10,"uuid":"797db886-62f9-4316-a45b-4d50512e65d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6825","nonce":3216877560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6827","nonce":3216877560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6831","nonce":3216877560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6829","nonce":3216877560}]},"public_addr":"172.21.15.153:6825/3216877560","cluster_addr":"172.21.15.153:6827/3216877560","heartbeat_back_addr":"172.21.15.153:6831/3216877560","heartbeat_front_addr":"172.21.15.153:6829/3216877560","state":["exists","up"]},{"osd":11,"uuid":"17e0afaa-84eb-4b26-9810-bfa3cc7e0f3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6825","nonce":1857200740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6827","nonce":1857200740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6831","nonce":1857200740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6829","nonce":1857200740}]},"public_addr":"172.21.15.196:6825/1857200740","cluster_addr":"172.21.15.196:6827/1857200740","heartbeat_back_addr":"172.21.15.196:6831/1857200740","heartbeat_front_addr":"172.21.15.196:6829/1857200740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:55:15.595645+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:55:40.405487+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:04.938034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:29.718696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:55.636029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:57:21.257083+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:57:46.316552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:58:11.946863+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:58:37.098578+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:03.759023+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:28.659049+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:54.350475+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:0/3010783707":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6801/3562882246":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6800/3562882246":"2024-09-20T08:53:51.528772+0000","172.21.15.120:0/3224512035":"2024-09-20T08:53:51.528772+0000","172.21.15.120:0/3465105835":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/3317449610":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/2800697451":"2024-09-20T08:53:28.144376+0000","172.21.15.120:6801/2938446665":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/109127432":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6800/2938446665":"2024-09-20T08:53:28.144376+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:02:48.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.689+0000 7f73d9d0d640 1 -- 172.21.15.120:0/1947746436 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f73c00a32d0 msgr2=0x7f73c00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:48.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.689+0000 7f73d9d0d640 1 --2- 172.21.15.120:0/1947746436 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f73c00a32d0 0x7f73c00a5790 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f73d4108880 tx=0x7f73bc00b390 comp rx=0 tx=0).stop 2024-09-19T09:02:48.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.689+0000 7f73d9d0d640 1 -- 172.21.15.120:0/1947746436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d4101de0 msgr2=0x7f73d4198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:48.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.689+0000 7f73d9d0d640 1 --2- 172.21.15.120:0/1947746436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d4101de0 0x7f73d4198330 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f73c400c920 tx=0x7f73c400cde0 comp rx=0 tx=0).stop 2024-09-19T09:02:48.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.689+0000 7f73d9d0d640 1 -- 172.21.15.120:0/1947746436 shutdown_connections 2024-09-19T09:02:48.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.689+0000 7f73d9d0d640 1 --2- 172.21.15.120:0/1947746436 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f73c00a32d0 0x7f73c00a5790 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:48.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.689+0000 7f73d9d0d640 1 --2- 172.21.15.120:0/1947746436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73d4106f10 0x7f73d419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:48.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.689+0000 7f73d9d0d640 1 --2- 172.21.15.120:0/1947746436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73d4102790 0x7f73d4198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:48.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.689+0000 7f73d9d0d640 1 --2- 172.21.15.120:0/1947746436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d4101de0 0x7f73d4198330 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:48.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.689+0000 7f73d9d0d640 1 -- 172.21.15.120:0/1947746436 >> 172.21.15.120:0/1947746436 conn(0x7f73d40fd8c0 msgr2=0x7f73d40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:48.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.689+0000 7f73d9d0d640 1 -- 172.21.15.120:0/1947746436 shutdown_connections 2024-09-19T09:02:48.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:48.689+0000 7f73d9d0d640 1 -- 172.21.15.120:0/1947746436 wait complete. 2024-09-19T09:02:49.057 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:48 smithi120 ceph-mon[31150]: pgmap v339: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-19T09:02:49.057 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:48 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1947746436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:02:49.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:48 smithi196 ceph-mon[35150]: pgmap v339: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-19T09:02:49.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:48 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1947746436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:02:49.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:48 smithi153 ceph-mon[35071]: pgmap v339: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-19T09:02:49.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:48 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1947746436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:02:49.481 DEBUG:tasks.fwd_scrub:Yielding 2024-09-19T09:02:49.481 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-19T09:02:49.494 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi120.front.sepia.ceph.com 2024-09-19T09:02:49.494 DEBUG:teuthology.orchestra.run.smithi120:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-19T09:02:49.495 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-19T09:02:49.495 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:02:49.495 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:02:49.498 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-19T09:02:49.517 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T09:02:49.517 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T09:02:49.763 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:49.781 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:50.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.318+0000 7f92e5ef3640 1 -- 172.21.15.120:0/481529450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92e010c170 msgr2=0x7f92e010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:50.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.318+0000 7f92e5ef3640 1 --2- 172.21.15.120:0/481529450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92e010c170 0x7f92e010e610 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009ca0 tx=0x7f92d802a950 comp rx=0 tx=0).stop 2024-09-19T09:02:50.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.318+0000 7f92e5ef3640 1 -- 172.21.15.120:0/481529450 shutdown_connections 2024-09-19T09:02:50.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.318+0000 7f92e5ef3640 1 --2- 172.21.15.120:0/481529450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92e010c170 0x7f92e010e610 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.318+0000 7f92e5ef3640 1 --2- 172.21.15.120:0/481529450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92e0103660 0x7f92e010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.318+0000 7f92e5ef3640 1 --2- 172.21.15.120:0/481529450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92e0102d40 0x7f92e0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.318+0000 7f92e5ef3640 1 -- 172.21.15.120:0/481529450 >> 172.21.15.120:0/481529450 conn(0x7f92e00fc7a0 msgr2=0x7f92e00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:50.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.318+0000 7f92e5ef3640 1 -- 172.21.15.120:0/481529450 shutdown_connections 2024-09-19T09:02:50.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.318+0000 7f92e5ef3640 1 -- 172.21.15.120:0/481529450 wait complete. 2024-09-19T09:02:50.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.318+0000 7f92e5ef3640 1 Processor -- start 2024-09-19T09:02:50.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.318+0000 7f92e5ef3640 1 -- start start 2024-09-19T09:02:50.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.319+0000 7f92e5ef3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92e0102d40 0x7f92e01009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:50.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.319+0000 7f92e5ef3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92e0103660 0x7f92e0100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:50.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.319+0000 7f92e5ef3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92e010c170 0x7f92e0101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:50.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.319+0000 7f92e5ef3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f92e0111f70 con 0x7f92e0103660 2024-09-19T09:02:50.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.319+0000 7f92e5ef3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f92e0111df0 con 0x7f92e010c170 2024-09-19T09:02:50.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.319+0000 7f92e5ef3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f92e01120f0 con 0x7f92e0102d40 2024-09-19T09:02:50.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.319+0000 7f92deffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92e0103660 0x7f92e0100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:50.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.319+0000 7f92deffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92e0103660 0x7f92e0100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48520/0 (socket says 172.21.15.120:48520) 2024-09-19T09:02:50.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.319+0000 7f92deffd640 1 -- 172.21.15.120:0/1985155130 learned_addr learned my addr 172.21.15.120:0/1985155130 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:50.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.319+0000 7f92dffff640 1 --2- 172.21.15.120:0/1985155130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92e010c170 0x7f92e0101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:50.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.319+0000 7f92df7fe640 1 --2- 172.21.15.120:0/1985155130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92e0102d40 0x7f92e01009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:50.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.319+0000 7f92deffd640 1 -- 172.21.15.120:0/1985155130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92e0102d40 msgr2=0x7f92e01009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:50.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.319+0000 7f92deffd640 1 --2- 172.21.15.120:0/1985155130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92e0102d40 0x7f92e01009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.319+0000 7f92deffd640 1 -- 172.21.15.120:0/1985155130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92e010c170 msgr2=0x7f92e0101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:50.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.319+0000 7f92deffd640 1 --2- 172.21.15.120:0/1985155130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92e010c170 0x7f92e0101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.319+0000 7f92deffd640 1 -- 172.21.15.120:0/1985155130 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e01a68d0 con 0x7f92e0103660 2024-09-19T09:02:50.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.319+0000 7f92deffd640 1 --2- 172.21.15.120:0/1985155130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92e0103660 0x7f92e0100ee0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f92d000d740 tx=0x7f92d000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:50.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.319+0000 7f92dcff9640 1 -- 172.21.15.120:0/1985155130 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d0017070 con 0x7f92e0103660 2024-09-19T09:02:50.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.320+0000 7f92dcff9640 1 -- 172.21.15.120:0/1985155130 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92d000de50 con 0x7f92e0103660 2024-09-19T09:02:50.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.320+0000 7f92e5ef3640 1 -- 172.21.15.120:0/1985155130 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92e01a6bc0 con 0x7f92e0103660 2024-09-19T09:02:50.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.320+0000 7f92dcff9640 1 -- 172.21.15.120:0/1985155130 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d000f620 con 0x7f92e0103660 2024-09-19T09:02:50.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.320+0000 7f92e5ef3640 1 -- 172.21.15.120:0/1985155130 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92e01a7100 con 0x7f92e0103660 2024-09-19T09:02:50.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.322+0000 7f92dcff9640 1 -- 172.21.15.120:0/1985155130 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f92d0009c30 con 0x7f92e0103660 2024-09-19T09:02:50.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.322+0000 7f92dcff9640 1 --2- 172.21.15.120:0/1985155130 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92c80a32d0 0x7f92c80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:50.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.322+0000 7f92e82cc740 1 Processor -- start 2024-09-19T09:02:50.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.322+0000 7f92e82cc740 1 -- start start 2024-09-19T09:02:50.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.322+0000 7f92dcff9640 1 -- 172.21.15.120:0/1985155130 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f92d00cf5f0 con 0x7f92e0103660 2024-09-19T09:02:50.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.322+0000 7f92df7fe640 1 --2- 172.21.15.120:0/1985155130 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92c80a32d0 0x7f92c80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:50.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.323+0000 7f92e82cc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e29b8b60 0x55b1e29b8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:50.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.323+0000 7f92e82cc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e29b9480 0x55b1e29c3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:50.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.323+0000 7f92e82cc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e29c3d40 0x55b1e29c6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:50.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.323+0000 7f92e82cc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b1e291e280 con 0x55b1e29b8b60 2024-09-19T09:02:50.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.323+0000 7f92e82cc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b1e28c5b10 con 0x55b1e29b9480 2024-09-19T09:02:50.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.323+0000 7f92e82cc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b1e291a240 con 0x55b1e29c3d40 2024-09-19T09:02:50.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.323+0000 7f92df7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e29b8b60 0x55b1e29b8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:50.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.323+0000 7f92df7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e29b8b60 0x55b1e29b8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48532/0 (socket says 172.21.15.120:48532) 2024-09-19T09:02:50.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.323+0000 7f92deffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e29b9480 0x55b1e29c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:50.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.323+0000 7f92df7fe640 1 -- 172.21.15.120:0/1034144699 learned_addr learned my addr 172.21.15.120:0/1034144699 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:50.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.323+0000 7f92df7fe640 1 --2- 172.21.15.120:0/1985155130 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92c80a32d0 0x7f92c80a5790 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f92d4005fd0 tx=0x7f92d400af30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:50.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.323+0000 7f92dffff640 1 --2- 172.21.15.120:0/1034144699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e29c3d40 0x55b1e29c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:50.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.323+0000 7f92deffd640 1 -- 172.21.15.120:0/1034144699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e29c3d40 msgr2=0x55b1e29c6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:50.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.323+0000 7f92deffd640 1 --2- 172.21.15.120:0/1034144699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e29c3d40 0x55b1e29c6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.323+0000 7f92deffd640 1 -- 172.21.15.120:0/1034144699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e29b8b60 msgr2=0x55b1e29b8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:50.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.323+0000 7f92deffd640 1 --2- 172.21.15.120:0/1034144699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e29b8b60 0x55b1e29b8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.323+0000 7f92deffd640 1 -- 172.21.15.120:0/1034144699 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1e28d21f0 con 0x55b1e29b9480 2024-09-19T09:02:50.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.323+0000 7f92dffff640 1 --2- 172.21.15.120:0/1034144699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e29c3d40 0x55b1e29c6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:02:50.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.324+0000 7f92deffd640 1 --2- 172.21.15.120:0/1034144699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e29b9480 0x55b1e29c3800 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f92d00ccfa0 tx=0x7f92d009bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40f05321400333bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:50.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.324+0000 7f92c5ffb640 1 -- 172.21.15.120:0/1034144699 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d0020070 con 0x55b1e29b9480 2024-09-19T09:02:50.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.324+0000 7f92c5ffb640 1 -- 172.21.15.120:0/1034144699 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92d0099db0 con 0x55b1e29b9480 2024-09-19T09:02:50.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.324+0000 7f92c5ffb640 1 -- 172.21.15.120:0/1034144699 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d009b160 con 0x55b1e29b9480 2024-09-19T09:02:50.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.325+0000 7f92e82cc740 1 -- 172.21.15.120:0/1034144699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e29b9480 msgr2=0x55b1e29c3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:50.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.325+0000 7f92e82cc740 1 --2- 172.21.15.120:0/1034144699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e29b9480 0x55b1e29c3800 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f92d00ccfa0 tx=0x7f92d009bb70 comp rx=0 tx=0).stop 2024-09-19T09:02:50.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.325+0000 7f92e82cc740 1 -- 172.21.15.120:0/1034144699 shutdown_connections 2024-09-19T09:02:50.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.325+0000 7f92e82cc740 1 --2- 172.21.15.120:0/1034144699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e29c3d40 0x55b1e29c6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.325+0000 7f92e82cc740 1 --2- 172.21.15.120:0/1034144699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e29b9480 0x55b1e29c3800 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.325+0000 7f92e82cc740 1 --2- 172.21.15.120:0/1034144699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e29b8b60 0x55b1e29b8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.325+0000 7f92e82cc740 1 -- 172.21.15.120:0/1034144699 >> 172.21.15.120:0/1034144699 conn(0x55b1e29b50f0 msgr2=0x55b1e29b54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:50.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.325+0000 7f92e82cc740 1 -- 172.21.15.120:0/1034144699 shutdown_connections 2024-09-19T09:02:50.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.325+0000 7f92e82cc740 1 -- 172.21.15.120:0/1034144699 wait complete. 2024-09-19T09:02:50.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.325+0000 7f92e82cc740 1 -- start start 2024-09-19T09:02:50.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.325+0000 7f92e82cc740 1 Processor -- start 2024-09-19T09:02:50.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.326+0000 7f92e82cc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e29b8b60 0x55b1e29c0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:50.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.326+0000 7f92e82cc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e29b9480 0x55b1e29c1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:50.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.326+0000 7f92e82cc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e29c3d40 0x55b1e29c16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:50.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.326+0000 7f92e82cc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b1e29c6910 con 0x55b1e29b9480 2024-09-19T09:02:50.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.326+0000 7f92e82cc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b1e28d8f10 con 0x55b1e29c3d40 2024-09-19T09:02:50.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.326+0000 7f92e82cc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b1e29c6a90 con 0x55b1e29b8b60 2024-09-19T09:02:50.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.326+0000 7f92e82cc740 10 client.0 _ll_register_callbacks cb 0x55b1e28a1a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:02:50.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.326+0000 7f92deffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e29b9480 0x55b1e29c1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:50.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.326+0000 7f92deffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e29b9480 0x55b1e29c1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48548/0 (socket says 172.21.15.120:48548) 2024-09-19T09:02:50.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.326+0000 7f92dffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e29c3d40 0x55b1e29c16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:50.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.326+0000 7f92deffd640 1 -- 172.21.15.120:0/418637779 learned_addr learned my addr 172.21.15.120:0/418637779 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:50.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.326+0000 7f92df7fe640 1 --2- 172.21.15.120:0/418637779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e29b8b60 0x55b1e29c0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:50.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.326+0000 7f92deffd640 1 -- 172.21.15.120:0/418637779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e29b8b60 msgr2=0x55b1e29c0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:50.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.326+0000 7f92deffd640 1 --2- 172.21.15.120:0/418637779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e29b8b60 0x55b1e29c0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.326+0000 7f92deffd640 1 -- 172.21.15.120:0/418637779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e29c3d40 msgr2=0x55b1e29c16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:50.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.326+0000 7f92deffd640 1 --2- 172.21.15.120:0/418637779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e29c3d40 0x55b1e29c16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.326+0000 7f92deffd640 1 -- 172.21.15.120:0/418637779 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1e268f6a0 con 0x55b1e29b9480 2024-09-19T09:02:50.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.326+0000 7f92dffff640 1 --2- 172.21.15.120:0/418637779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e29c3d40 0x55b1e29c16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:02:50.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.326+0000 7f92deffd640 1 --2- 172.21.15.120:0/418637779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e29b9480 0x55b1e29c1180 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f92d0005fd0 tx=0x7f92d0099f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b796cdc4bce61374 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:50.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.326+0000 7f92beffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:02:50.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.326+0000 7f92beffd640 1 -- 172.21.15.120:0/418637779 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d00b3070 con 0x55b1e29b9480 2024-09-19T09:02:50.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.326+0000 7f92e82cc740 1 -- 172.21.15.120:0/418637779 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92d0099db0 con 0x55b1e29b9480 2024-09-19T09:02:50.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.326+0000 7f92beffd640 1 -- 172.21.15.120:0/418637779 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f92d00ba1f0 con 0x55b1e29b9480 2024-09-19T09:02:50.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.327+0000 7f92beffd640 1 -- 172.21.15.120:0/418637779 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f92d0020050 con 0x55b1e29b9480 2024-09-19T09:02:50.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.327+0000 7f92e82cc740 10 client.15294 fetch_fsmap learned FSMap version 28 2024-09-19T09:02:50.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.327+0000 7f92e82cc740 1 -- 172.21.15.120:0/418637779 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b1e29d2540 con 0x55b1e29b9480 2024-09-19T09:02:50.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.327+0000 7f92beffd640 1 -- 172.21.15.120:0/418637779 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92d0099db0 con 0x55b1e29b9480 2024-09-19T09:02:50.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.327+0000 7f92beffd640 1 -- 172.21.15.120:0/418637779 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f92d0097800 con 0x55b1e29b9480 2024-09-19T09:02:50.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.327+0000 7f92e82cc740 10 client.15294 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:02:50.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.327+0000 7f92e82cc740 10 client.15294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:02:50.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.327+0000 7f92e82cc740 20 client.15294 populate_metadata read hostname 'smithi120' 2024-09-19T09:02:50.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.327+0000 7f92e82cc740 1 --2- 172.21.15.120:0/418637779 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b1e29d3890 0x55b1e29d5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:50.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.327+0000 7f92e82cc740 4 client.15294 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:02:50.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.327+0000 7f92e82cc740 1 -- 172.21.15.120:0/418637779 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1e292b7f0 con 0x55b1e29d3890 2024-09-19T09:02:50.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.327+0000 7f92df7fe640 1 --2- 172.21.15.120:0/418637779 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b1e29d3890 0x55b1e29d5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:50.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.328+0000 7f92beffd640 10 client.15294 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:02:50.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.328+0000 7f92df7fe640 1 --2- 172.21.15.120:0/418637779 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b1e29d3890 0x55b1e29d5c80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd3760e128988707 server_cookie=baa35e358c716899 in_seq=0 out_seq=0 2024-09-19T09:02:50.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.329+0000 7f92beffd640 1 -- 172.21.15.120:0/418637779 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b1e292b7f0 con 0x55b1e29d3890 2024-09-19T09:02:50.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.329+0000 7f92beffd640 10 client.15294 handle_command_reply: tid=0 2024-09-19T09:02:50.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.329+0000 7f92e82cc740 1 client.15294 shutdown 2024-09-19T09:02:50.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.329+0000 7f92e82cc740 1 -- 172.21.15.120:0/418637779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e29b9480 msgr2=0x55b1e29c1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:50.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.329+0000 7f92e82cc740 1 --2- 172.21.15.120:0/418637779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e29b9480 0x55b1e29c1180 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f92d0005fd0 tx=0x7f92d0099f60 comp rx=0 tx=0).stop 2024-09-19T09:02:50.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.329+0000 7f92e82cc740 1 -- 172.21.15.120:0/418637779 shutdown_connections 2024-09-19T09:02:50.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.329+0000 7f92e82cc740 1 --2- 172.21.15.120:0/418637779 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b1e29d3890 0x55b1e29d5c80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.329+0000 7f92e82cc740 1 --2- 172.21.15.120:0/418637779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e29c3d40 0x55b1e29c16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.329+0000 7f92e82cc740 1 --2- 172.21.15.120:0/418637779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e29b9480 0x55b1e29c1180 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.329+0000 7f92beffd640 0 client.15294 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:02:50.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.329+0000 7f92e82cc740 1 --2- 172.21.15.120:0/418637779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e29b8b60 0x55b1e29c0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.329+0000 7f92e82cc740 1 -- 172.21.15.120:0/418637779 >> 172.21.15.120:0/418637779 conn(0x55b1e29b50f0 msgr2=0x55b1e29bae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:50.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.329+0000 7f92e82cc740 1 -- 172.21.15.120:0/418637779 shutdown_connections 2024-09-19T09:02:50.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.329+0000 7f92e82cc740 1 -- 172.21.15.120:0/418637779 wait complete. 2024-09-19T09:02:50.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.329+0000 7f92e82cc740 20 client.15294 trim_cache size 0 max 16384 2024-09-19T09:02:50.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.350+0000 7f92e82cc740 1 Processor -- start 2024-09-19T09:02:50.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.350+0000 7f92e82cc740 1 -- start start 2024-09-19T09:02:50.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.351+0000 7f92e82cc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e29b8b60 0x55b1e29c1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:50.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.351+0000 7f92e82cc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e29b9480 0x55b1e29d8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:50.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.351+0000 7f92e82cc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e29c3d40 0x55b1e29d91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:50.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.351+0000 7f92e82cc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f92e01af780 con 0x55b1e29b9480 2024-09-19T09:02:50.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.351+0000 7f92e82cc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f92e01b12c0 con 0x55b1e29b8b60 2024-09-19T09:02:50.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.351+0000 7f92e82cc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f92e01afe50 con 0x55b1e29c3d40 2024-09-19T09:02:50.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.351+0000 7f92deffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e29b9480 0x55b1e29d8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:50.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.351+0000 7f92deffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e29b9480 0x55b1e29d8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48550/0 (socket says 172.21.15.120:48550) 2024-09-19T09:02:50.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.351+0000 7f92deffd640 1 -- 172.21.15.120:0/656772679 learned_addr learned my addr 172.21.15.120:0/656772679 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:50.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.351+0000 7f92df7fe640 1 --2- 172.21.15.120:0/656772679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e29b8b60 0x55b1e29c1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:50.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.351+0000 7f92deffd640 1 -- 172.21.15.120:0/656772679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e29c3d40 msgr2=0x55b1e29d91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:50.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.351+0000 7f92deffd640 1 --2- 172.21.15.120:0/656772679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e29c3d40 0x55b1e29d91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.351+0000 7f92deffd640 1 -- 172.21.15.120:0/656772679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e29b8b60 msgr2=0x55b1e29c1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:50.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.351+0000 7f92deffd640 1 --2- 172.21.15.120:0/656772679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e29b8b60 0x55b1e29c1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.351+0000 7f92deffd640 1 -- 172.21.15.120:0/656772679 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1e29c3510 con 0x55b1e29b9480 2024-09-19T09:02:50.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.351+0000 7f92df7fe640 1 --2- 172.21.15.120:0/656772679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e29b8b60 0x55b1e29c1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:02:50.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.351+0000 7f92deffd640 1 --2- 172.21.15.120:0/656772679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e29b9480 0x55b1e29d8ca0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f92d009ba40 tx=0x7f92d009bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff6c0e2e20b506d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:50.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.352+0000 7f92c57fa640 1 -- 172.21.15.120:0/656772679 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d00b3030 con 0x55b1e29b9480 2024-09-19T09:02:50.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.352+0000 7f92c57fa640 1 -- 172.21.15.120:0/656772679 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92d0099db0 con 0x55b1e29b9480 2024-09-19T09:02:50.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.352+0000 7f92c57fa640 1 -- 172.21.15.120:0/656772679 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d0097800 con 0x55b1e29b9480 2024-09-19T09:02:50.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.353+0000 7f92e82cc740 1 -- 172.21.15.120:0/656772679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e29b9480 msgr2=0x55b1e29d8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:50.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.353+0000 7f92e82cc740 1 --2- 172.21.15.120:0/656772679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e29b9480 0x55b1e29d8ca0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f92d009ba40 tx=0x7f92d009bb70 comp rx=0 tx=0).stop 2024-09-19T09:02:50.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.353+0000 7f92e82cc740 1 -- 172.21.15.120:0/656772679 shutdown_connections 2024-09-19T09:02:50.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.353+0000 7f92e82cc740 1 --2- 172.21.15.120:0/656772679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e29c3d40 0x55b1e29d91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.353+0000 7f92e82cc740 1 --2- 172.21.15.120:0/656772679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e29b9480 0x55b1e29d8ca0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.353+0000 7f92e82cc740 1 --2- 172.21.15.120:0/656772679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e29b8b60 0x55b1e29c1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.353+0000 7f92e82cc740 1 -- 172.21.15.120:0/656772679 >> 172.21.15.120:0/656772679 conn(0x55b1e29b50f0 msgr2=0x55b1e29c2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:50.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.353+0000 7f92e82cc740 1 -- 172.21.15.120:0/656772679 shutdown_connections 2024-09-19T09:02:50.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.353+0000 7f92e82cc740 1 -- 172.21.15.120:0/656772679 wait complete. 2024-09-19T09:02:50.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.354+0000 7f92e82cc740 1 -- start start 2024-09-19T09:02:50.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.354+0000 7f92e82cc740 1 Processor -- start 2024-09-19T09:02:50.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.354+0000 7f92e82cc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e29b8b60 0x55b1e29d6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:50.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.354+0000 7f92e82cc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e29b9480 0x55b1e29d7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:50.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.354+0000 7f92e82cc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e29c3d40 0x55b1e29d7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:50.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.354+0000 7f92e82cc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f92e01b0bf0 con 0x55b1e29c3d40 2024-09-19T09:02:50.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.354+0000 7f92e82cc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f92e01b0520 con 0x55b1e29b8b60 2024-09-19T09:02:50.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.354+0000 7f92e82cc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f92e01ad740 con 0x55b1e29b9480 2024-09-19T09:02:50.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.354+0000 7f92e82cc740 10 client.0 _ll_register_callbacks cb 0x7f92d4016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:02:50.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.354+0000 7f92dffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e29c3d40 0x55b1e29d7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:50.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.355+0000 7f92dffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e29c3d40 0x55b1e29d7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48562/0 (socket says 172.21.15.120:48562) 2024-09-19T09:02:50.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.355+0000 7f92dffff640 1 -- 172.21.15.120:0/2986312885 learned_addr learned my addr 172.21.15.120:0/2986312885 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:50.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.355+0000 7f92deffd640 1 --2- 172.21.15.120:0/2986312885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e29b9480 0x55b1e29d7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:50.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.355+0000 7f92df7fe640 1 --2- 172.21.15.120:0/2986312885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e29b8b60 0x55b1e29d6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:50.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.355+0000 7f92dffff640 1 -- 172.21.15.120:0/2986312885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e29b9480 msgr2=0x55b1e29d7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:50.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.355+0000 7f92dffff640 1 --2- 172.21.15.120:0/2986312885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e29b9480 0x55b1e29d7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.355+0000 7f92dffff640 1 -- 172.21.15.120:0/2986312885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e29b8b60 msgr2=0x55b1e29d6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:50.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.355+0000 7f92dffff640 1 --2- 172.21.15.120:0/2986312885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e29b8b60 0x55b1e29d6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.355+0000 7f92dffff640 1 -- 172.21.15.120:0/2986312885 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1e2a3d3d0 con 0x55b1e29c3d40 2024-09-19T09:02:50.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.355+0000 7f92dffff640 1 --2- 172.21.15.120:0/2986312885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e29c3d40 0x55b1e29d7680 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009ca0 tx=0x7f92d802fe80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a55aaec333c3398 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:50.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.355+0000 7f92beffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:02:50.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.355+0000 7f92beffd640 1 -- 172.21.15.120:0/2986312885 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8034040 con 0x55b1e29c3d40 2024-09-19T09:02:50.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.355+0000 7f92beffd640 1 -- 172.21.15.120:0/2986312885 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f92d8003d80 con 0x55b1e29c3d40 2024-09-19T09:02:50.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.355+0000 7f92e82cc740 1 -- 172.21.15.120:0/2986312885 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92d0099db0 con 0x55b1e29c3d40 2024-09-19T09:02:50.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.355+0000 7f92beffd640 1 -- 172.21.15.120:0/2986312885 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f92d8004040 con 0x55b1e29c3d40 2024-09-19T09:02:50.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.355+0000 7f92e82cc740 10 client.15300 fetch_fsmap learned FSMap version 28 2024-09-19T09:02:50.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.356+0000 7f92e82cc740 1 -- 172.21.15.120:0/2986312885 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b1e2a3e620 con 0x55b1e29c3d40 2024-09-19T09:02:50.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.356+0000 7f92beffd640 1 -- 172.21.15.120:0/2986312885 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92d0099db0 con 0x55b1e29c3d40 2024-09-19T09:02:50.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.356+0000 7f92beffd640 1 -- 172.21.15.120:0/2986312885 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f92d80309a0 con 0x55b1e29c3d40 2024-09-19T09:02:50.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.356+0000 7f92e82cc740 10 client.15300 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:02:50.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.356+0000 7f92e82cc740 10 client.15300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:02:50.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.356+0000 7f92e82cc740 20 client.15300 populate_metadata read hostname 'smithi120' 2024-09-19T09:02:50.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.356+0000 7f92e82cc740 1 --2- 172.21.15.120:0/2986312885 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b1e2a3edf0 0x55b1e2a41210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:50.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.356+0000 7f92e82cc740 4 client.15300 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:02:50.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.356+0000 7f92e82cc740 1 -- 172.21.15.120:0/2986312885 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b1e2925d00 con 0x55b1e2a3edf0 2024-09-19T09:02:50.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.356+0000 7f92df7fe640 1 --2- 172.21.15.120:0/2986312885 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b1e2a3edf0 0x55b1e2a41210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:50.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.357+0000 7f92beffd640 10 client.15300 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:02:50.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.357+0000 7f92df7fe640 1 --2- 172.21.15.120:0/2986312885 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b1e2a3edf0 0x55b1e2a41210 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc68a461a260cad8 server_cookie=efea58fac549f1c9 in_seq=0 out_seq=0 2024-09-19T09:02:50.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.358+0000 7f92beffd640 1 -- 172.21.15.120:0/2986312885 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b1e2925d00 con 0x55b1e2a3edf0 2024-09-19T09:02:50.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.358+0000 7f92beffd640 10 client.15300 handle_command_reply: tid=0 2024-09-19T09:02:50.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.358+0000 7f92e82cc740 1 client.15300 shutdown 2024-09-19T09:02:50.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.358+0000 7f92e82cc740 1 -- 172.21.15.120:0/2986312885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e29c3d40 msgr2=0x55b1e29d7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:50.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.358+0000 7f92e82cc740 1 --2- 172.21.15.120:0/2986312885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e29c3d40 0x55b1e29d7680 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009ca0 tx=0x7f92d802fe80 comp rx=0 tx=0).stop 2024-09-19T09:02:50.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.358+0000 7f92e82cc740 1 -- 172.21.15.120:0/2986312885 shutdown_connections 2024-09-19T09:02:50.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.358+0000 7f92e82cc740 1 --2- 172.21.15.120:0/2986312885 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b1e2a3edf0 0x55b1e2a41210 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.358+0000 7f92e82cc740 1 --2- 172.21.15.120:0/2986312885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e29c3d40 0x55b1e29d7680 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.358+0000 7f92e82cc740 1 --2- 172.21.15.120:0/2986312885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e29b9480 0x55b1e29d7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.358+0000 7f92e82cc740 1 --2- 172.21.15.120:0/2986312885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e29b8b60 0x55b1e29d6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.358+0000 7f92e82cc740 1 -- 172.21.15.120:0/2986312885 >> 172.21.15.120:0/2986312885 conn(0x55b1e29b50f0 msgr2=0x55b1e29c2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:50.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.358+0000 7f92beffd640 0 client.15300 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:02:50.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.358+0000 7f92e82cc740 1 -- 172.21.15.120:0/2986312885 shutdown_connections 2024-09-19T09:02:50.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.358+0000 7f92e82cc740 1 -- 172.21.15.120:0/2986312885 wait complete. 2024-09-19T09:02:50.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.359+0000 7f92e82cc740 20 client.15300 trim_cache size 0 max 16384 2024-09-19T09:02:50.367 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:02:50.367 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:02:50.367 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "b9550c65-d6e1-4b1d-afa4-a29d12acb60a", 2024-09-19T09:02:50.367 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:02:50.368 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:02:50.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.359+0000 7f92e5ef3640 1 -- 172.21.15.120:0/1985155130 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92c80a32d0 msgr2=0x7f92c80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:50.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.359+0000 7f92e5ef3640 1 --2- 172.21.15.120:0/1985155130 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92c80a32d0 0x7f92c80a5790 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f92d4005fd0 tx=0x7f92d400af30 comp rx=0 tx=0).stop 2024-09-19T09:02:50.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.359+0000 7f92e5ef3640 1 -- 172.21.15.120:0/1985155130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92e0103660 msgr2=0x7f92e0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:50.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.359+0000 7f92e5ef3640 1 --2- 172.21.15.120:0/1985155130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92e0103660 0x7f92e0100ee0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f92d000d740 tx=0x7f92d000dc00 comp rx=0 tx=0).stop 2024-09-19T09:02:50.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.359+0000 7f92e5ef3640 1 -- 172.21.15.120:0/1985155130 shutdown_connections 2024-09-19T09:02:50.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.359+0000 7f92e5ef3640 1 --2- 172.21.15.120:0/1985155130 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92c80a32d0 0x7f92c80a5790 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.359+0000 7f92e5ef3640 1 --2- 172.21.15.120:0/1985155130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92e010c170 0x7f92e0101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.359+0000 7f92e5ef3640 1 --2- 172.21.15.120:0/1985155130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92e0103660 0x7f92e0100ee0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.359+0000 7f92e5ef3640 1 --2- 172.21.15.120:0/1985155130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92e0102d40 0x7f92e01009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:50.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.359+0000 7f92e5ef3640 1 -- 172.21.15.120:0/1985155130 >> 172.21.15.120:0/1985155130 conn(0x7f92e00fc7a0 msgr2=0x7f92e00fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:50.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.359+0000 7f92e5ef3640 1 -- 172.21.15.120:0/1985155130 shutdown_connections 2024-09-19T09:02:50.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:50.359+0000 7f92e5ef3640 1 -- 172.21.15.120:0/1985155130 wait complete. 2024-09-19T09:02:50.962 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:50 smithi120 ceph-mon[31150]: pgmap v340: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-09-19T09:02:51.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.042+0000 7f4401e72640 1 -- 172.21.15.120:0/3730576893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43fc101420 msgr2=0x7f43fc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:51.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.042+0000 7f4401e72640 1 --2- 172.21.15.120:0/3730576893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43fc101420 0x7f43fc10d950 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f43f4009cd0 tx=0x7f43f402a8e0 comp rx=0 tx=0).stop 2024-09-19T09:02:51.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.042+0000 7f4401e72640 1 -- 172.21.15.120:0/3730576893 shutdown_connections 2024-09-19T09:02:51.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.042+0000 7f4401e72640 1 --2- 172.21.15.120:0/3730576893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43fc101420 0x7f43fc10d950 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:51.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.042+0000 7f4401e72640 1 --2- 172.21.15.120:0/3730576893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43fc100a80 0x7f43fc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:51.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.042+0000 7f4401e72640 1 --2- 172.21.15.120:0/3730576893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43fc106a80 0x7f43fc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:51.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.042+0000 7f4401e72640 1 -- 172.21.15.120:0/3730576893 >> 172.21.15.120:0/3730576893 conn(0x7f43fc0fc7a0 msgr2=0x7f43fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:51.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.042+0000 7f4401e72640 1 -- 172.21.15.120:0/3730576893 shutdown_connections 2024-09-19T09:02:51.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.042+0000 7f4401e72640 1 -- 172.21.15.120:0/3730576893 wait complete. 2024-09-19T09:02:51.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.043+0000 7f4401e72640 1 Processor -- start 2024-09-19T09:02:51.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.043+0000 7f4401e72640 1 -- start start 2024-09-19T09:02:51.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.043+0000 7f4401e72640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43fc100a80 0x7f43fc0fcca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:51.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.043+0000 7f4401e72640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43fc101420 0x7f43fc0fd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:51.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.043+0000 7f4401e72640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43fc106a80 0x7f43fc0fd720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:51.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.043+0000 7f4401e72640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f43fc110e80 con 0x7f43fc106a80 2024-09-19T09:02:51.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.043+0000 7f4401e72640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f43fc110d00 con 0x7f43fc100a80 2024-09-19T09:02:51.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.043+0000 7f4401e72640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43fc111000 con 0x7f43fc101420 2024-09-19T09:02:51.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.043+0000 7f43fbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43fc106a80 0x7f43fc0fd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:51.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.043+0000 7f43fbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43fc106a80 0x7f43fc0fd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48578/0 (socket says 172.21.15.120:48578) 2024-09-19T09:02:51.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.043+0000 7f43fbfff640 1 -- 172.21.15.120:0/1926190737 learned_addr learned my addr 172.21.15.120:0/1926190737 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:51.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.043+0000 7f43fb7fe640 1 --2- 172.21.15.120:0/1926190737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43fc100a80 0x7f43fc0fcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:51.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.043+0000 7f43fbfff640 1 -- 172.21.15.120:0/1926190737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43fc101420 msgr2=0x7f43fc0fd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:51.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.043+0000 7f43fbfff640 1 --2- 172.21.15.120:0/1926190737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43fc101420 0x7f43fc0fd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:51.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.043+0000 7f43fbfff640 1 -- 172.21.15.120:0/1926190737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43fc100a80 msgr2=0x7f43fc0fcca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:51.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.043+0000 7f43fbfff640 1 --2- 172.21.15.120:0/1926190737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43fc100a80 0x7f43fc0fcca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:51.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.043+0000 7f43fbfff640 1 -- 172.21.15.120:0/1926190737 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43fc1a68e0 con 0x7f43fc106a80 2024-09-19T09:02:51.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.043+0000 7f43fbfff640 1 --2- 172.21.15.120:0/1926190737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43fc106a80 0x7f43fc0fd720 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f43f4005cc0 tx=0x7f43f4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:51.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.044+0000 7f43f97fa640 1 -- 172.21.15.120:0/1926190737 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f4004040 con 0x7f43fc106a80 2024-09-19T09:02:51.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.044+0000 7f43f97fa640 1 -- 172.21.15.120:0/1926190737 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43f40046c0 con 0x7f43fc106a80 2024-09-19T09:02:51.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.044+0000 7f4401e72640 1 -- 172.21.15.120:0/1926190737 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43fc1a6b70 con 0x7f43fc106a80 2024-09-19T09:02:51.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.044+0000 7f43f97fa640 1 -- 172.21.15.120:0/1926190737 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f4046480 con 0x7f43fc106a80 2024-09-19T09:02:51.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.044+0000 7f4401e72640 1 -- 172.21.15.120:0/1926190737 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43fc1a7100 con 0x7f43fc106a80 2024-09-19T09:02:51.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.045+0000 7f4401e72640 1 -- 172.21.15.120:0/1926190737 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43fc100ee0 con 0x7f43fc106a80 2024-09-19T09:02:51.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.046+0000 7f43f97fa640 1 -- 172.21.15.120:0/1926190737 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f43f40305a0 con 0x7f43fc106a80 2024-09-19T09:02:51.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.046+0000 7f43f97fa640 1 --2- 172.21.15.120:0/1926190737 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43e80a3350 0x7f43e80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:51.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.046+0000 7f43fb7fe640 1 --2- 172.21.15.120:0/1926190737 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43e80a3350 0x7f43e80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:51.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.046+0000 7f43f97fa640 1 -- 172.21.15.120:0/1926190737 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f43f402e030 con 0x7f43fc106a80 2024-09-19T09:02:51.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.047+0000 7f43fb7fe640 1 --2- 172.21.15.120:0/1926190737 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43e80a3350 0x7f43e80a5810 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f43ec00cac0 tx=0x7f43ec00c4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:51.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.050+0000 7f43f97fa640 1 -- 172.21.15.120:0/1926190737 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f43f404b060 con 0x7f43fc106a80 2024-09-19T09:02:51.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:50 smithi196 ceph-mon[35150]: pgmap v340: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-09-19T09:02:51.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:50 smithi153 ceph-mon[35071]: pgmap v340: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-09-19T09:02:51.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.220+0000 7f4401e72640 1 -- 172.21.15.120:0/1926190737 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f43fc062630 con 0x7f43e80a3350 2024-09-19T09:02:51.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.227+0000 7f43f97fa640 1 -- 172.21.15.120:0/1926190737 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259139 (secure 0 0 0) 0x7f43fc062630 con 0x7f43e80a3350 2024-09-19T09:02:51.229 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:02:51.230 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-19T09:02:51.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.229+0000 7f4401e72640 1 -- 172.21.15.120:0/1926190737 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43e80a3350 msgr2=0x7f43e80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:51.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.229+0000 7f4401e72640 1 --2- 172.21.15.120:0/1926190737 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43e80a3350 0x7f43e80a5810 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f43ec00cac0 tx=0x7f43ec00c4a0 comp rx=0 tx=0).stop 2024-09-19T09:02:51.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.230+0000 7f4401e72640 1 -- 172.21.15.120:0/1926190737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43fc106a80 msgr2=0x7f43fc0fd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:51.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.230+0000 7f4401e72640 1 --2- 172.21.15.120:0/1926190737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43fc106a80 0x7f43fc0fd720 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f43f4005cc0 tx=0x7f43f4003970 comp rx=0 tx=0).stop 2024-09-19T09:02:51.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.230+0000 7f4401e72640 1 -- 172.21.15.120:0/1926190737 shutdown_connections 2024-09-19T09:02:51.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.230+0000 7f4401e72640 1 --2- 172.21.15.120:0/1926190737 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43e80a3350 0x7f43e80a5810 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:51.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.230+0000 7f4401e72640 1 --2- 172.21.15.120:0/1926190737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43fc106a80 0x7f43fc0fd720 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:51.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.230+0000 7f4401e72640 1 --2- 172.21.15.120:0/1926190737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43fc101420 0x7f43fc0fd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:51.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.230+0000 7f4401e72640 1 --2- 172.21.15.120:0/1926190737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43fc100a80 0x7f43fc0fcca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:51.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.230+0000 7f4401e72640 1 -- 172.21.15.120:0/1926190737 >> 172.21.15.120:0/1926190737 conn(0x7f43fc0fc7a0 msgr2=0x7f43fc06a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:51.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.230+0000 7f4401e72640 1 -- 172.21.15.120:0/1926190737 shutdown_connections 2024-09-19T09:02:51.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:51.230+0000 7f4401e72640 1 -- 172.21.15.120:0/1926190737 wait complete. 2024-09-19T09:02:51.908 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:02:52.038 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:51 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:02:52.038 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:51 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:02:52.039 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:51 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:02:52.039 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:51 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:02:52.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:51 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:02:52.384 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:51 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:02:52.384 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:51 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:02:52.384 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:51 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:02:52.385 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:51 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:02:52.385 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:51 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:02:52.385 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:51 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:02:52.386 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:51 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:02:52.411 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2024-09-19T09:02:49.637085+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668353,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1246,"num_read_kb":752,"num_write":449,"num_write_kb":3099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1182102,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364364,"kb_used_data":10956,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577460,"statfs":{"total":1151940427776,"available":1151567319040,"internally_reserved":0,"allocated":11218944,"data_stored":7128207,"data_compressed":181932,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8236,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":125,"num_read_kb":66,"num_write":16,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003185"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624417+0000","last_change":"2024-09-19T09:00:46.966628+0000","last_active":"2024-09-19T09:02:21.624417+0000","last_peered":"2024-09-19T09:02:21.624417+0000","last_clean":"2024-09-19T09:02:21.624417+0000","last_became_active":"2024-09-19T09:00:46.966290+0000","last_became_peered":"2024-09-19T09:00:46.966290+0000","last_unstale":"2024-09-19T09:02:21.624417+0000","last_undegraded":"2024-09-19T09:02:21.624417+0000","last_fullsized":"2024-09-19T09:02:21.624417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:42.717139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621317+0000","last_change":"2024-09-19T09:00:46.963706+0000","last_active":"2024-09-19T09:02:21.621317+0000","last_peered":"2024-09-19T09:02:21.621317+0000","last_clean":"2024-09-19T09:02:21.621317+0000","last_became_active":"2024-09-19T09:00:46.963303+0000","last_became_peered":"2024-09-19T09:00:46.963303+0000","last_unstale":"2024-09-19T09:02:21.621317+0000","last_undegraded":"2024-09-19T09:02:21.621317+0000","last_fullsized":"2024-09-19T09:02:21.621317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:07.451500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620146+0000","last_change":"2024-09-19T09:00:46.967149+0000","last_active":"2024-09-19T09:02:21.620146+0000","last_peered":"2024-09-19T09:02:21.620146+0000","last_clean":"2024-09-19T09:02:21.620146+0000","last_became_active":"2024-09-19T09:00:46.966861+0000","last_became_peered":"2024-09-19T09:00:46.966861+0000","last_unstale":"2024-09-19T09:02:21.620146+0000","last_undegraded":"2024-09-19T09:02:21.620146+0000","last_fullsized":"2024-09-19T09:02:21.620146+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:28.396022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.605530+0000","last_change":"2024-09-19T09:00:46.969242+0000","last_active":"2024-09-19T09:02:21.605530+0000","last_peered":"2024-09-19T09:02:21.605530+0000","last_clean":"2024-09-19T09:02:21.605530+0000","last_became_active":"2024-09-19T09:00:46.968546+0000","last_became_peered":"2024-09-19T09:00:46.968546+0000","last_unstale":"2024-09-19T09:02:21.605530+0000","last_undegraded":"2024-09-19T09:02:21.605530+0000","last_fullsized":"2024-09-19T09:02:21.605530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:43:51.060130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.602962+0000","last_change":"2024-09-19T09:00:46.955501+0000","last_active":"2024-09-19T09:02:21.602962+0000","last_peered":"2024-09-19T09:02:21.602962+0000","last_clean":"2024-09-19T09:02:21.602962+0000","last_became_active":"2024-09-19T09:00:46.955161+0000","last_became_peered":"2024-09-19T09:00:46.955161+0000","last_unstale":"2024-09-19T09:02:21.602962+0000","last_undegraded":"2024-09-19T09:02:21.602962+0000","last_fullsized":"2024-09-19T09:02:21.602962+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:39:19.005931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624332+0000","last_change":"2024-09-19T09:00:46.962577+0000","last_active":"2024-09-19T09:02:21.624332+0000","last_peered":"2024-09-19T09:02:21.624332+0000","last_clean":"2024-09-19T09:02:21.624332+0000","last_became_active":"2024-09-19T09:00:46.962253+0000","last_became_peered":"2024-09-19T09:00:46.962253+0000","last_unstale":"2024-09-19T09:02:21.624332+0000","last_undegraded":"2024-09-19T09:02:21.624332+0000","last_fullsized":"2024-09-19T09:02:21.624332+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:27.925932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621157+0000","last_change":"2024-09-19T09:00:46.960004+0000","last_active":"2024-09-19T09:02:21.621157+0000","last_peered":"2024-09-19T09:02:21.621157+0000","last_clean":"2024-09-19T09:02:21.621157+0000","last_became_active":"2024-09-19T09:00:46.959664+0000","last_became_peered":"2024-09-19T09:00:46.959664+0000","last_unstale":"2024-09-19T09:02:21.621157+0000","last_undegraded":"2024-09-19T09:02:21.621157+0000","last_fullsized":"2024-09-19T09:02:21.621157+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:41.616309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622980+0000","last_change":"2024-09-19T09:00:46.952394+0000","last_active":"2024-09-19T09:02:21.622980+0000","last_peered":"2024-09-19T09:02:21.622980+0000","last_clean":"2024-09-19T09:02:21.622980+0000","last_became_active":"2024-09-19T09:00:46.952084+0000","last_became_peered":"2024-09-19T09:00:46.952084+0000","last_unstale":"2024-09-19T09:02:21.622980+0000","last_undegraded":"2024-09-19T09:02:21.622980+0000","last_fullsized":"2024-09-19T09:02:21.622980+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:21.698423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.632916+0000","last_change":"2024-09-19T09:00:46.961262+0000","last_active":"2024-09-19T09:02:21.632916+0000","last_peered":"2024-09-19T09:02:21.632916+0000","last_clean":"2024-09-19T09:02:21.632916+0000","last_became_active":"2024-09-19T09:00:46.960951+0000","last_became_peered":"2024-09-19T09:00:46.960951+0000","last_unstale":"2024-09-19T09:02:21.632916+0000","last_undegraded":"2024-09-19T09:02:21.632916+0000","last_fullsized":"2024-09-19T09:02:21.632916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:02.445108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.625661+0000","last_change":"2024-09-19T09:01:52.758459+0000","last_active":"2024-09-19T09:02:21.625661+0000","last_peered":"2024-09-19T09:02:21.625661+0000","last_clean":"2024-09-19T09:02:21.625661+0000","last_became_active":"2024-09-19T09:01:52.758073+0000","last_became_peered":"2024-09-19T09:01:52.758073+0000","last_unstale":"2024-09-19T09:02:21.625661+0000","last_undegraded":"2024-09-19T09:02:21.625661+0000","last_fullsized":"2024-09-19T09:02:21.625661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:46:32.983316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.634093+0000","last_change":"2024-09-19T09:00:46.963200+0000","last_active":"2024-09-19T09:02:21.634093+0000","last_peered":"2024-09-19T09:02:21.634093+0000","last_clean":"2024-09-19T09:02:21.634093+0000","last_became_active":"2024-09-19T09:00:46.960496+0000","last_became_peered":"2024-09-19T09:00:46.960496+0000","last_unstale":"2024-09-19T09:02:21.634093+0000","last_undegraded":"2024-09-19T09:02:21.634093+0000","last_fullsized":"2024-09-19T09:02:21.634093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:59:47.234492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.618494+0000","last_change":"2024-09-19T09:00:46.966309+0000","last_active":"2024-09-19T09:02:21.618494+0000","last_peered":"2024-09-19T09:02:21.618494+0000","last_clean":"2024-09-19T09:02:21.618494+0000","last_became_active":"2024-09-19T09:00:46.965970+0000","last_became_peered":"2024-09-19T09:00:46.965970+0000","last_unstale":"2024-09-19T09:02:21.618494+0000","last_undegraded":"2024-09-19T09:02:21.618494+0000","last_fullsized":"2024-09-19T09:02:21.618494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:38.353206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622484+0000","last_change":"2024-09-19T09:00:46.954079+0000","last_active":"2024-09-19T09:02:21.622484+0000","last_peered":"2024-09-19T09:02:21.622484+0000","last_clean":"2024-09-19T09:02:21.622484+0000","last_became_active":"2024-09-19T09:00:46.953398+0000","last_became_peered":"2024-09-19T09:00:46.953398+0000","last_unstale":"2024-09-19T09:02:21.622484+0000","last_undegraded":"2024-09-19T09:02:21.622484+0000","last_fullsized":"2024-09-19T09:02:21.622484+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:54:27.554800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621192+0000","last_change":"2024-09-19T09:00:46.965347+0000","last_active":"2024-09-19T09:02:21.621192+0000","last_peered":"2024-09-19T09:02:21.621192+0000","last_clean":"2024-09-19T09:02:21.621192+0000","last_became_active":"2024-09-19T09:00:46.965045+0000","last_became_peered":"2024-09-19T09:00:46.965045+0000","last_unstale":"2024-09-19T09:02:21.621192+0000","last_undegraded":"2024-09-19T09:02:21.621192+0000","last_fullsized":"2024-09-19T09:02:21.621192+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:27.316100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620943+0000","last_change":"2024-09-19T09:00:46.962898+0000","last_active":"2024-09-19T09:02:21.620943+0000","last_peered":"2024-09-19T09:02:21.620943+0000","last_clean":"2024-09-19T09:02:21.620943+0000","last_became_active":"2024-09-19T09:00:46.962412+0000","last_became_peered":"2024-09-19T09:00:46.962412+0000","last_unstale":"2024-09-19T09:02:21.620943+0000","last_undegraded":"2024-09-19T09:02:21.620943+0000","last_fullsized":"2024-09-19T09:02:21.620943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:40.483678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.627060+0000","last_change":"2024-09-19T09:00:46.958147+0000","last_active":"2024-09-19T09:02:21.627060+0000","last_peered":"2024-09-19T09:02:21.627060+0000","last_clean":"2024-09-19T09:02:21.627060+0000","last_became_active":"2024-09-19T09:00:46.957805+0000","last_became_peered":"2024-09-19T09:00:46.957805+0000","last_unstale":"2024-09-19T09:02:21.627060+0000","last_undegraded":"2024-09-19T09:02:21.627060+0000","last_fullsized":"2024-09-19T09:02:21.627060+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:24:17.000064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624642+0000","last_change":"2024-09-19T09:00:46.967399+0000","last_active":"2024-09-19T09:02:21.624642+0000","last_peered":"2024-09-19T09:02:21.624642+0000","last_clean":"2024-09-19T09:02:21.624642+0000","last_became_active":"2024-09-19T09:00:46.966145+0000","last_became_peered":"2024-09-19T09:00:46.966145+0000","last_unstale":"2024-09-19T09:02:21.624642+0000","last_undegraded":"2024-09-19T09:02:21.624642+0000","last_fullsized":"2024-09-19T09:02:21.624642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:29.192469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.614692+0000","last_change":"2024-09-19T09:00:46.960956+0000","last_active":"2024-09-19T09:02:21.614692+0000","last_peered":"2024-09-19T09:02:21.614692+0000","last_clean":"2024-09-19T09:02:21.614692+0000","last_became_active":"2024-09-19T09:00:46.960664+0000","last_became_peered":"2024-09-19T09:00:46.960664+0000","last_unstale":"2024-09-19T09:02:21.614692+0000","last_undegraded":"2024-09-19T09:02:21.614692+0000","last_fullsized":"2024-09-19T09:02:21.614692+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:18.724348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622878+0000","last_change":"2024-09-19T09:00:46.951362+0000","last_active":"2024-09-19T09:02:21.622878+0000","last_peered":"2024-09-19T09:02:21.622878+0000","last_clean":"2024-09-19T09:02:21.622878+0000","last_became_active":"2024-09-19T09:00:46.951004+0000","last_became_peered":"2024-09-19T09:00:46.951004+0000","last_unstale":"2024-09-19T09:02:21.622878+0000","last_undegraded":"2024-09-19T09:02:21.622878+0000","last_fullsized":"2024-09-19T09:02:21.622878+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:43.574425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.623346+0000","last_change":"2024-09-19T09:00:46.966752+0000","last_active":"2024-09-19T09:02:21.623346+0000","last_peered":"2024-09-19T09:02:21.623346+0000","last_clean":"2024-09-19T09:02:21.623346+0000","last_became_active":"2024-09-19T09:00:46.966370+0000","last_became_peered":"2024-09-19T09:00:46.966370+0000","last_unstale":"2024-09-19T09:02:21.623346+0000","last_undegraded":"2024-09-19T09:02:21.623346+0000","last_fullsized":"2024-09-19T09:02:21.623346+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:04.035771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621434+0000","last_change":"2024-09-19T09:00:46.956369+0000","last_active":"2024-09-19T09:02:21.621434+0000","last_peered":"2024-09-19T09:02:21.621434+0000","last_clean":"2024-09-19T09:02:21.621434+0000","last_became_active":"2024-09-19T09:00:46.956075+0000","last_became_peered":"2024-09-19T09:00:46.956075+0000","last_unstale":"2024-09-19T09:02:21.621434+0000","last_undegraded":"2024-09-19T09:02:21.621434+0000","last_fullsized":"2024-09-19T09:02:21.621434+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:23.985017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633789+0000","last_change":"2024-09-19T09:00:46.956661+0000","last_active":"2024-09-19T09:02:21.633789+0000","last_peered":"2024-09-19T09:02:21.633789+0000","last_clean":"2024-09-19T09:02:21.633789+0000","last_became_active":"2024-09-19T09:00:46.956291+0000","last_became_peered":"2024-09-19T09:00:46.956291+0000","last_unstale":"2024-09-19T09:02:21.633789+0000","last_undegraded":"2024-09-19T09:02:21.633789+0000","last_fullsized":"2024-09-19T09:02:21.633789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:55.713039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.616782+0000","last_change":"2024-09-19T09:00:46.961866+0000","last_active":"2024-09-19T09:02:21.616782+0000","last_peered":"2024-09-19T09:02:21.616782+0000","last_clean":"2024-09-19T09:02:21.616782+0000","last_became_active":"2024-09-19T09:00:46.961469+0000","last_became_peered":"2024-09-19T09:00:46.961469+0000","last_unstale":"2024-09-19T09:02:21.616782+0000","last_undegraded":"2024-09-19T09:02:21.616782+0000","last_fullsized":"2024-09-19T09:02:21.616782+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:47.665419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622833+0000","last_change":"2024-09-19T09:00:46.948765+0000","last_active":"2024-09-19T09:02:21.622833+0000","last_peered":"2024-09-19T09:02:21.622833+0000","last_clean":"2024-09-19T09:02:21.622833+0000","last_became_active":"2024-09-19T09:00:46.948431+0000","last_became_peered":"2024-09-19T09:00:46.948431+0000","last_unstale":"2024-09-19T09:02:21.622833+0000","last_undegraded":"2024-09-19T09:02:21.622833+0000","last_fullsized":"2024-09-19T09:02:21.622833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:31.766464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.618165+0000","last_change":"2024-09-19T09:00:46.962433+0000","last_active":"2024-09-19T09:02:21.618165+0000","last_peered":"2024-09-19T09:02:21.618165+0000","last_clean":"2024-09-19T09:02:21.618165+0000","last_became_active":"2024-09-19T09:00:46.962153+0000","last_became_peered":"2024-09-19T09:00:46.962153+0000","last_unstale":"2024-09-19T09:02:21.618165+0000","last_undegraded":"2024-09-19T09:02:21.618165+0000","last_fullsized":"2024-09-19T09:02:21.618165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:37.973611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.623402+0000","last_change":"2024-09-19T09:00:46.953715+0000","last_active":"2024-09-19T09:02:21.623402+0000","last_peered":"2024-09-19T09:02:21.623402+0000","last_clean":"2024-09-19T09:02:21.623402+0000","last_became_active":"2024-09-19T09:00:46.953415+0000","last_became_peered":"2024-09-19T09:00:46.953415+0000","last_unstale":"2024-09-19T09:02:21.623402+0000","last_undegraded":"2024-09-19T09:02:21.623402+0000","last_fullsized":"2024-09-19T09:02:21.623402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:29.768382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.615199+0000","last_change":"2024-09-19T09:00:46.964746+0000","last_active":"2024-09-19T09:02:21.615199+0000","last_peered":"2024-09-19T09:02:21.615199+0000","last_clean":"2024-09-19T09:02:21.615199+0000","last_became_active":"2024-09-19T09:00:46.964404+0000","last_became_peered":"2024-09-19T09:00:46.964404+0000","last_unstale":"2024-09-19T09:02:21.615199+0000","last_undegraded":"2024-09-19T09:02:21.615199+0000","last_fullsized":"2024-09-19T09:02:21.615199+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:19.841495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.619156+0000","last_change":"2024-09-19T09:00:46.962875+0000","last_active":"2024-09-19T09:02:21.619156+0000","last_peered":"2024-09-19T09:02:21.619156+0000","last_clean":"2024-09-19T09:02:21.619156+0000","last_became_active":"2024-09-19T09:00:46.962561+0000","last_became_peered":"2024-09-19T09:00:46.962561+0000","last_unstale":"2024-09-19T09:02:21.619156+0000","last_undegraded":"2024-09-19T09:02:21.619156+0000","last_fullsized":"2024-09-19T09:02:21.619156+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:39.675545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.612887+0000","last_change":"2024-09-19T09:00:46.963126+0000","last_active":"2024-09-19T09:02:21.612887+0000","last_peered":"2024-09-19T09:02:21.612887+0000","last_clean":"2024-09-19T09:02:21.612887+0000","last_became_active":"2024-09-19T09:00:46.962620+0000","last_became_peered":"2024-09-19T09:00:46.962620+0000","last_unstale":"2024-09-19T09:02:21.612887+0000","last_undegraded":"2024-09-19T09:02:21.612887+0000","last_fullsized":"2024-09-19T09:02:21.612887+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:59.723280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620254+0000","last_change":"2024-09-19T09:00:46.953026+0000","last_active":"2024-09-19T09:02:21.620254+0000","last_peered":"2024-09-19T09:02:21.620254+0000","last_clean":"2024-09-19T09:02:21.620254+0000","last_became_active":"2024-09-19T09:00:46.952596+0000","last_became_peered":"2024-09-19T09:00:46.952596+0000","last_unstale":"2024-09-19T09:02:21.620254+0000","last_undegraded":"2024-09-19T09:02:21.620254+0000","last_fullsized":"2024-09-19T09:02:21.620254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:58.524116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624169+0000","last_change":"2024-09-19T09:00:46.968964+0000","last_active":"2024-09-19T09:02:21.624169+0000","last_peered":"2024-09-19T09:02:21.624169+0000","last_clean":"2024-09-19T09:02:21.624169+0000","last_became_active":"2024-09-19T09:00:46.968505+0000","last_became_peered":"2024-09-19T09:00:46.968505+0000","last_unstale":"2024-09-19T09:02:21.624169+0000","last_undegraded":"2024-09-19T09:02:21.624169+0000","last_fullsized":"2024-09-19T09:02:21.624169+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:17:33.335541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.625205+0000","last_change":"2024-09-19T09:00:46.961520+0000","last_active":"2024-09-19T09:02:21.625205+0000","last_peered":"2024-09-19T09:02:21.625205+0000","last_clean":"2024-09-19T09:02:21.625205+0000","last_became_active":"2024-09-19T09:00:46.961140+0000","last_became_peered":"2024-09-19T09:00:46.961140+0000","last_unstale":"2024-09-19T09:02:21.625205+0000","last_undegraded":"2024-09-19T09:02:21.625205+0000","last_fullsized":"2024-09-19T09:02:21.625205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:03.120173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621084+0000","last_change":"2024-09-19T09:00:46.955995+0000","last_active":"2024-09-19T09:02:21.621084+0000","last_peered":"2024-09-19T09:02:21.621084+0000","last_clean":"2024-09-19T09:02:21.621084+0000","last_became_active":"2024-09-19T09:00:46.954678+0000","last_became_peered":"2024-09-19T09:00:46.954678+0000","last_unstale":"2024-09-19T09:02:21.621084+0000","last_undegraded":"2024-09-19T09:02:21.621084+0000","last_fullsized":"2024-09-19T09:02:21.621084+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:10.314575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":140,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:46.292343+0000","last_change":"2024-09-19T09:00:44.946000+0000","last_active":"2024-09-19T09:02:46.292343+0000","last_peered":"2024-09-19T09:02:46.292343+0000","last_clean":"2024-09-19T09:02:46.292343+0000","last_became_active":"2024-09-19T09:00:44.945583+0000","last_became_peered":"2024-09-19T09:00:44.945583+0000","last_unstale":"2024-09-19T09:02:46.292343+0000","last_undegraded":"2024-09-19T09:02:46.292343+0000","last_fullsized":"2024-09-19T09:02:46.292343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:53:19.109697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624079+0000","last_change":"2024-09-19T09:01:52.904443+0000","last_active":"2024-09-19T09:02:21.624079+0000","last_peered":"2024-09-19T09:02:21.624079+0000","last_clean":"2024-09-19T09:02:21.624079+0000","last_became_active":"2024-09-19T09:01:52.904077+0000","last_became_peered":"2024-09-19T09:01:52.904077+0000","last_unstale":"2024-09-19T09:02:21.624079+0000","last_undegraded":"2024-09-19T09:02:21.624079+0000","last_fullsized":"2024-09-19T09:02:21.624079+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:50.467090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.748381+0000","last_change":"2024-09-19T09:00:44.942435+0000","last_active":"2024-09-19T09:01:52.748381+0000","last_peered":"2024-09-19T09:01:52.748381+0000","last_clean":"2024-09-19T09:01:52.748381+0000","last_became_active":"2024-09-19T09:00:44.942067+0000","last_became_peered":"2024-09-19T09:00:44.942067+0000","last_unstale":"2024-09-19T09:01:52.748381+0000","last_undegraded":"2024-09-19T09:01:52.748381+0000","last_fullsized":"2024-09-19T09:01:52.748381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:16.476506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.747900+0000","last_change":"2024-09-19T09:00:44.937944+0000","last_active":"2024-09-19T09:01:52.747900+0000","last_peered":"2024-09-19T09:01:52.747900+0000","last_clean":"2024-09-19T09:01:52.747900+0000","last_became_active":"2024-09-19T09:00:44.937623+0000","last_became_peered":"2024-09-19T09:00:44.937623+0000","last_unstale":"2024-09-19T09:01:52.747900+0000","last_undegraded":"2024-09-19T09:01:52.747900+0000","last_fullsized":"2024-09-19T09:01:52.747900+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:49:08.730015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620184+0000","last_change":"2024-09-19T09:00:46.959932+0000","last_active":"2024-09-19T09:02:21.620184+0000","last_peered":"2024-09-19T09:02:21.620184+0000","last_clean":"2024-09-19T09:02:21.620184+0000","last_became_active":"2024-09-19T09:00:46.959637+0000","last_became_peered":"2024-09-19T09:00:46.959637+0000","last_unstale":"2024-09-19T09:02:21.620184+0000","last_undegraded":"2024-09-19T09:02:21.620184+0000","last_fullsized":"2024-09-19T09:02:21.620184+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:55.337669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751513+0000","last_change":"2024-09-19T09:00:44.944882+0000","last_active":"2024-09-19T09:01:52.751513+0000","last_peered":"2024-09-19T09:01:52.751513+0000","last_clean":"2024-09-19T09:01:52.751513+0000","last_became_active":"2024-09-19T09:00:44.944116+0000","last_became_peered":"2024-09-19T09:00:44.944116+0000","last_unstale":"2024-09-19T09:01:52.751513+0000","last_undegraded":"2024-09-19T09:01:52.751513+0000","last_fullsized":"2024-09-19T09:01:52.751513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:54:58.555913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.615953+0000","last_change":"2024-09-19T09:00:46.956859+0000","last_active":"2024-09-19T09:02:21.615953+0000","last_peered":"2024-09-19T09:02:21.615953+0000","last_clean":"2024-09-19T09:02:21.615953+0000","last_became_active":"2024-09-19T09:00:46.956561+0000","last_became_peered":"2024-09-19T09:00:46.956561+0000","last_unstale":"2024-09-19T09:02:21.615953+0000","last_undegraded":"2024-09-19T09:02:21.615953+0000","last_fullsized":"2024-09-19T09:02:21.615953+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:09:04.125591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'6","reported_seq":242,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:45.290203+0000","last_change":"2024-09-19T09:00:44.945959+0000","last_active":"2024-09-19T09:02:45.290203+0000","last_peered":"2024-09-19T09:02:45.290203+0000","last_clean":"2024-09-19T09:02:45.290203+0000","last_became_active":"2024-09-19T09:00:44.945640+0000","last_became_peered":"2024-09-19T09:00:44.945640+0000","last_unstale":"2024-09-19T09:02:45.290203+0000","last_undegraded":"2024-09-19T09:02:45.290203+0000","last_fullsized":"2024-09-19T09:02:45.290203+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:02.523834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":214,"num_read_kb":214,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.632897+0000","last_change":"2024-09-19T09:00:46.953677+0000","last_active":"2024-09-19T09:02:21.632897+0000","last_peered":"2024-09-19T09:02:21.632897+0000","last_clean":"2024-09-19T09:02:21.632897+0000","last_became_active":"2024-09-19T09:00:46.953357+0000","last_became_peered":"2024-09-19T09:00:46.953357+0000","last_unstale":"2024-09-19T09:02:21.632897+0000","last_undegraded":"2024-09-19T09:02:21.632897+0000","last_fullsized":"2024-09-19T09:02:21.632897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:48.963003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.893433+0000","last_change":"2024-09-19T09:01:52.893433+0000","last_active":"2024-09-19T09:01:52.893433+0000","last_peered":"2024-09-19T09:01:52.893433+0000","last_clean":"2024-09-19T09:01:52.893433+0000","last_became_active":"2024-09-19T09:01:52.889836+0000","last_became_peered":"2024-09-19T09:01:52.889836+0000","last_unstale":"2024-09-19T09:01:52.893433+0000","last_undegraded":"2024-09-19T09:01:52.893433+0000","last_fullsized":"2024-09-19T09:01:52.893433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:08:22.339717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622288+0000","last_change":"2024-09-19T09:00:46.962285+0000","last_active":"2024-09-19T09:02:21.622288+0000","last_peered":"2024-09-19T09:02:21.622288+0000","last_clean":"2024-09-19T09:02:21.622288+0000","last_became_active":"2024-09-19T09:00:46.961982+0000","last_became_peered":"2024-09-19T09:00:46.961982+0000","last_unstale":"2024-09-19T09:02:21.622288+0000","last_undegraded":"2024-09-19T09:02:21.622288+0000","last_fullsized":"2024-09-19T09:02:21.622288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:41.658501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752480+0000","last_change":"2024-09-19T09:00:45.241425+0000","last_active":"2024-09-19T09:01:52.752480+0000","last_peered":"2024-09-19T09:01:52.752480+0000","last_clean":"2024-09-19T09:01:52.752480+0000","last_became_active":"2024-09-19T09:00:45.241032+0000","last_became_peered":"2024-09-19T09:00:45.241032+0000","last_unstale":"2024-09-19T09:01:52.752480+0000","last_undegraded":"2024-09-19T09:01:52.752480+0000","last_fullsized":"2024-09-19T09:01:52.752480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:00.707855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.626097+0000","last_change":"2024-09-19T09:00:46.962436+0000","last_active":"2024-09-19T09:02:21.626097+0000","last_peered":"2024-09-19T09:02:21.626097+0000","last_clean":"2024-09-19T09:02:21.626097+0000","last_became_active":"2024-09-19T09:00:46.962114+0000","last_became_peered":"2024-09-19T09:00:46.962114+0000","last_unstale":"2024-09-19T09:02:21.626097+0000","last_undegraded":"2024-09-19T09:02:21.626097+0000","last_fullsized":"2024-09-19T09:02:21.626097+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:41.761705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'39","reported_seq":288,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:45.290288+0000","last_change":"2024-09-19T09:00:44.945187+0000","last_active":"2024-09-19T09:02:45.290288+0000","last_peered":"2024-09-19T09:02:45.290288+0000","last_clean":"2024-09-19T09:02:45.290288+0000","last_became_active":"2024-09-19T09:00:44.944763+0000","last_became_peered":"2024-09-19T09:00:44.944763+0000","last_unstale":"2024-09-19T09:02:45.290288+0000","last_undegraded":"2024-09-19T09:02:45.290288+0000","last_fullsized":"2024-09-19T09:02:45.290288+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:06.052994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70094,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":227,"num_read_kb":182,"num_write":41,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.630413+0000","last_change":"2024-09-19T09:00:46.960991+0000","last_active":"2024-09-19T09:02:21.630413+0000","last_peered":"2024-09-19T09:02:21.630413+0000","last_clean":"2024-09-19T09:02:21.630413+0000","last_became_active":"2024-09-19T09:00:46.958962+0000","last_became_peered":"2024-09-19T09:00:46.958962+0000","last_unstale":"2024-09-19T09:02:21.630413+0000","last_undegraded":"2024-09-19T09:02:21.630413+0000","last_fullsized":"2024-09-19T09:02:21.630413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:21.795694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.755250+0000","last_change":"2024-09-19T09:00:44.946176+0000","last_active":"2024-09-19T09:01:52.755250+0000","last_peered":"2024-09-19T09:01:52.755250+0000","last_clean":"2024-09-19T09:01:52.755250+0000","last_became_active":"2024-09-19T09:00:44.945869+0000","last_became_peered":"2024-09-19T09:00:44.945869+0000","last_unstale":"2024-09-19T09:01:52.755250+0000","last_undegraded":"2024-09-19T09:01:52.755250+0000","last_fullsized":"2024-09-19T09:01:52.755250+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:16.022184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621664+0000","last_change":"2024-09-19T09:00:46.956345+0000","last_active":"2024-09-19T09:02:21.621664+0000","last_peered":"2024-09-19T09:02:21.621664+0000","last_clean":"2024-09-19T09:02:21.621664+0000","last_became_active":"2024-09-19T09:00:46.956057+0000","last_became_peered":"2024-09-19T09:00:46.956057+0000","last_unstale":"2024-09-19T09:02:21.621664+0000","last_undegraded":"2024-09-19T09:02:21.621664+0000","last_fullsized":"2024-09-19T09:02:21.621664+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:27.510661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752205+0000","last_change":"2024-09-19T09:00:44.945667+0000","last_active":"2024-09-19T09:01:52.752205+0000","last_peered":"2024-09-19T09:01:52.752205+0000","last_clean":"2024-09-19T09:01:52.752205+0000","last_became_active":"2024-09-19T09:00:44.945308+0000","last_became_peered":"2024-09-19T09:00:44.945308+0000","last_unstale":"2024-09-19T09:01:52.752205+0000","last_undegraded":"2024-09-19T09:01:52.752205+0000","last_fullsized":"2024-09-19T09:01:52.752205+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:00:11.304913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.629868+0000","last_change":"2024-09-19T09:00:46.965088+0000","last_active":"2024-09-19T09:02:21.629868+0000","last_peered":"2024-09-19T09:02:21.629868+0000","last_clean":"2024-09-19T09:02:21.629868+0000","last_became_active":"2024-09-19T09:00:46.964777+0000","last_became_peered":"2024-09-19T09:00:46.964777+0000","last_unstale":"2024-09-19T09:02:21.629868+0000","last_undegraded":"2024-09-19T09:02:21.629868+0000","last_fullsized":"2024-09-19T09:02:21.629868+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:32.958984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'7","reported_seq":142,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:46.291326+0000","last_change":"2024-09-19T09:00:44.946193+0000","last_active":"2024-09-19T09:02:46.291326+0000","last_peered":"2024-09-19T09:02:46.291326+0000","last_clean":"2024-09-19T09:02:46.291326+0000","last_became_active":"2024-09-19T09:00:44.945168+0000","last_became_peered":"2024-09-19T09:00:44.945168+0000","last_unstale":"2024-09-19T09:02:46.291326+0000","last_undegraded":"2024-09-19T09:02:46.291326+0000","last_fullsized":"2024-09-19T09:02:46.291326+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:21.230689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.627048+0000","last_change":"2024-09-19T09:01:52.754058+0000","last_active":"2024-09-19T09:02:21.627048+0000","last_peered":"2024-09-19T09:02:21.627048+0000","last_clean":"2024-09-19T09:02:21.627048+0000","last_became_active":"2024-09-19T09:01:52.753674+0000","last_became_peered":"2024-09-19T09:01:52.753674+0000","last_unstale":"2024-09-19T09:02:21.627048+0000","last_undegraded":"2024-09-19T09:02:21.627048+0000","last_fullsized":"2024-09-19T09:02:21.627048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:05.665121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":64,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.774430+0000","last_change":"2024-09-19T09:01:52.774430+0000","last_active":"2024-09-19T09:01:52.774430+0000","last_peered":"2024-09-19T09:01:52.774430+0000","last_clean":"2024-09-19T09:01:52.774430+0000","last_became_active":"2024-09-19T09:01:52.756083+0000","last_became_peered":"2024-09-19T09:01:52.756083+0000","last_unstale":"2024-09-19T09:01:52.774430+0000","last_undegraded":"2024-09-19T09:01:52.774430+0000","last_fullsized":"2024-09-19T09:01:52.774430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:52.639490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633068+0000","last_change":"2024-09-19T09:01:52.757684+0000","last_active":"2024-09-19T09:02:21.633068+0000","last_peered":"2024-09-19T09:02:21.633068+0000","last_clean":"2024-09-19T09:02:21.633068+0000","last_became_active":"2024-09-19T09:01:52.756969+0000","last_became_peered":"2024-09-19T09:01:52.756969+0000","last_unstale":"2024-09-19T09:02:21.633068+0000","last_undegraded":"2024-09-19T09:02:21.633068+0000","last_fullsized":"2024-09-19T09:02:21.633068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:24:47.117996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752185+0000","last_change":"2024-09-19T09:00:44.946138+0000","last_active":"2024-09-19T09:01:52.752185+0000","last_peered":"2024-09-19T09:01:52.752185+0000","last_clean":"2024-09-19T09:01:52.752185+0000","last_became_active":"2024-09-19T09:00:44.945793+0000","last_became_peered":"2024-09-19T09:00:44.945793+0000","last_unstale":"2024-09-19T09:01:52.752185+0000","last_undegraded":"2024-09-19T09:01:52.752185+0000","last_fullsized":"2024-09-19T09:01:52.752185+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:01.341560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620195+0000","last_change":"2024-09-19T09:00:46.970360+0000","last_active":"2024-09-19T09:02:21.620195+0000","last_peered":"2024-09-19T09:02:21.620195+0000","last_clean":"2024-09-19T09:02:21.620195+0000","last_became_active":"2024-09-19T09:00:46.969971+0000","last_became_peered":"2024-09-19T09:00:46.969971+0000","last_unstale":"2024-09-19T09:02:21.620195+0000","last_undegraded":"2024-09-19T09:02:21.620195+0000","last_fullsized":"2024-09-19T09:02:21.620195+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:48.812158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.748305+0000","last_change":"2024-09-19T09:00:44.941713+0000","last_active":"2024-09-19T09:01:52.748305+0000","last_peered":"2024-09-19T09:01:52.748305+0000","last_clean":"2024-09-19T09:01:52.748305+0000","last_became_active":"2024-09-19T09:00:44.941373+0000","last_became_peered":"2024-09-19T09:00:44.941373+0000","last_unstale":"2024-09-19T09:01:52.748305+0000","last_undegraded":"2024-09-19T09:01:52.748305+0000","last_fullsized":"2024-09-19T09:01:52.748305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:31.157803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.630100+0000","last_change":"2024-09-19T09:00:46.959383+0000","last_active":"2024-09-19T09:02:21.630100+0000","last_peered":"2024-09-19T09:02:21.630100+0000","last_clean":"2024-09-19T09:02:21.630100+0000","last_became_active":"2024-09-19T09:00:46.958860+0000","last_became_peered":"2024-09-19T09:00:46.958860+0000","last_unstale":"2024-09-19T09:02:21.630100+0000","last_undegraded":"2024-09-19T09:02:21.630100+0000","last_fullsized":"2024-09-19T09:02:21.630100+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:32:04.423017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752778+0000","last_change":"2024-09-19T09:00:44.944797+0000","last_active":"2024-09-19T09:01:52.752778+0000","last_peered":"2024-09-19T09:01:52.752778+0000","last_clean":"2024-09-19T09:01:52.752778+0000","last_became_active":"2024-09-19T09:00:44.944259+0000","last_became_peered":"2024-09-19T09:00:44.944259+0000","last_unstale":"2024-09-19T09:01:52.752778+0000","last_undegraded":"2024-09-19T09:01:52.752778+0000","last_fullsized":"2024-09-19T09:01:52.752778+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:58.996217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624674+0000","last_change":"2024-09-19T09:00:46.964998+0000","last_active":"2024-09-19T09:02:21.624674+0000","last_peered":"2024-09-19T09:02:21.624674+0000","last_clean":"2024-09-19T09:02:21.624674+0000","last_became_active":"2024-09-19T09:00:46.964562+0000","last_became_peered":"2024-09-19T09:00:46.964562+0000","last_unstale":"2024-09-19T09:02:21.624674+0000","last_undegraded":"2024-09-19T09:02:21.624674+0000","last_fullsized":"2024-09-19T09:02:21.624674+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:00.089800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":140,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:46.319890+0000","last_change":"2024-09-19T09:00:45.239770+0000","last_active":"2024-09-19T09:02:46.319890+0000","last_peered":"2024-09-19T09:02:46.319890+0000","last_clean":"2024-09-19T09:02:46.319890+0000","last_became_active":"2024-09-19T09:00:45.239385+0000","last_became_peered":"2024-09-19T09:00:45.239385+0000","last_unstale":"2024-09-19T09:02:46.319890+0000","last_undegraded":"2024-09-19T09:02:46.319890+0000","last_fullsized":"2024-09-19T09:02:46.319890+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:21.109900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.634073+0000","last_change":"2024-09-19T09:00:46.967505+0000","last_active":"2024-09-19T09:02:21.634073+0000","last_peered":"2024-09-19T09:02:21.634073+0000","last_clean":"2024-09-19T09:02:21.634073+0000","last_became_active":"2024-09-19T09:00:46.967195+0000","last_became_peered":"2024-09-19T09:00:46.967195+0000","last_unstale":"2024-09-19T09:02:21.634073+0000","last_undegraded":"2024-09-19T09:02:21.634073+0000","last_fullsized":"2024-09-19T09:02:21.634073+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:52:28.917333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":335,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751112+0000","last_change":"2024-09-19T08:59:33.821924+0000","last_active":"2024-09-19T09:01:52.751112+0000","last_peered":"2024-09-19T09:01:52.751112+0000","last_clean":"2024-09-19T09:01:52.751112+0000","last_became_active":"2024-09-19T08:59:33.821404+0000","last_became_peered":"2024-09-19T08:59:33.821404+0000","last_unstale":"2024-09-19T09:01:52.751112+0000","last_undegraded":"2024-09-19T09:01:52.751112+0000","last_fullsized":"2024-09-19T09:01:52.751112+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:56:07.830480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:56:07.830480+0000","last_clean_scrub_stamp":"2024-09-19T08:56:07.830480+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:21.763918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.888480+0000","last_change":"2024-09-19T09:00:44.943714+0000","last_active":"2024-09-19T09:01:52.888480+0000","last_peered":"2024-09-19T09:01:52.888480+0000","last_clean":"2024-09-19T09:01:52.888480+0000","last_became_active":"2024-09-19T09:00:44.943362+0000","last_became_peered":"2024-09-19T09:00:44.943362+0000","last_unstale":"2024-09-19T09:01:52.888480+0000","last_undegraded":"2024-09-19T09:01:52.888480+0000","last_fullsized":"2024-09-19T09:01:52.888480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:30.015648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633592+0000","last_change":"2024-09-19T09:00:46.956784+0000","last_active":"2024-09-19T09:02:21.633592+0000","last_peered":"2024-09-19T09:02:21.633592+0000","last_clean":"2024-09-19T09:02:21.633592+0000","last_became_active":"2024-09-19T09:00:46.956443+0000","last_became_peered":"2024-09-19T09:00:46.956443+0000","last_unstale":"2024-09-19T09:02:21.633592+0000","last_undegraded":"2024-09-19T09:02:21.633592+0000","last_fullsized":"2024-09-19T09:02:21.633592+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:18.918788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.750994+0000","last_change":"2024-09-19T09:00:44.947796+0000","last_active":"2024-09-19T09:01:52.750994+0000","last_peered":"2024-09-19T09:01:52.750994+0000","last_clean":"2024-09-19T09:01:52.750994+0000","last_became_active":"2024-09-19T09:00:44.946949+0000","last_became_peered":"2024-09-19T09:00:44.946949+0000","last_unstale":"2024-09-19T09:01:52.750994+0000","last_undegraded":"2024-09-19T09:01:52.750994+0000","last_fullsized":"2024-09-19T09:01:52.750994+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:42.210186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.619434+0000","last_change":"2024-09-19T09:00:46.963856+0000","last_active":"2024-09-19T09:02:21.619434+0000","last_peered":"2024-09-19T09:02:21.619434+0000","last_clean":"2024-09-19T09:02:21.619434+0000","last_became_active":"2024-09-19T09:00:46.963543+0000","last_became_peered":"2024-09-19T09:00:46.963543+0000","last_unstale":"2024-09-19T09:02:21.619434+0000","last_undegraded":"2024-09-19T09:02:21.619434+0000","last_fullsized":"2024-09-19T09:02:21.619434+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:31.041028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'6","reported_seq":139,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:47.292119+0000","last_change":"2024-09-19T09:00:45.239269+0000","last_active":"2024-09-19T09:02:47.292119+0000","last_peered":"2024-09-19T09:02:47.292119+0000","last_clean":"2024-09-19T09:02:47.292119+0000","last_became_active":"2024-09-19T09:00:45.238929+0000","last_became_peered":"2024-09-19T09:00:45.238929+0000","last_unstale":"2024-09-19T09:02:47.292119+0000","last_undegraded":"2024-09-19T09:02:47.292119+0000","last_fullsized":"2024-09-19T09:02:47.292119+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:46.684148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622578+0000","last_change":"2024-09-19T09:00:46.963595+0000","last_active":"2024-09-19T09:02:21.622578+0000","last_peered":"2024-09-19T09:02:21.622578+0000","last_clean":"2024-09-19T09:02:21.622578+0000","last_became_active":"2024-09-19T09:00:46.963243+0000","last_became_peered":"2024-09-19T09:00:46.963243+0000","last_unstale":"2024-09-19T09:02:21.622578+0000","last_undegraded":"2024-09-19T09:02:21.622578+0000","last_fullsized":"2024-09-19T09:02:21.622578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:01.122996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":133,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:42.850379+0000","last_change":"2024-09-19T09:00:44.941550+0000","last_active":"2024-09-19T09:02:42.850379+0000","last_peered":"2024-09-19T09:02:42.850379+0000","last_clean":"2024-09-19T09:02:42.850379+0000","last_became_active":"2024-09-19T09:00:44.941145+0000","last_became_peered":"2024-09-19T09:00:44.941145+0000","last_unstale":"2024-09-19T09:02:42.850379+0000","last_undegraded":"2024-09-19T09:02:42.850379+0000","last_fullsized":"2024-09-19T09:02:42.850379+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:53.893887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.612369+0000","last_change":"2024-09-19T09:00:46.963473+0000","last_active":"2024-09-19T09:02:21.612369+0000","last_peered":"2024-09-19T09:02:21.612369+0000","last_clean":"2024-09-19T09:02:21.612369+0000","last_became_active":"2024-09-19T09:00:46.963090+0000","last_became_peered":"2024-09-19T09:00:46.963090+0000","last_unstale":"2024-09-19T09:02:21.612369+0000","last_undegraded":"2024-09-19T09:02:21.612369+0000","last_fullsized":"2024-09-19T09:02:21.612369+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:01.492385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.750853+0000","last_change":"2024-09-19T09:00:45.239566+0000","last_active":"2024-09-19T09:01:52.750853+0000","last_peered":"2024-09-19T09:01:52.750853+0000","last_clean":"2024-09-19T09:01:52.750853+0000","last_became_active":"2024-09-19T09:00:45.239250+0000","last_became_peered":"2024-09-19T09:00:45.239250+0000","last_unstale":"2024-09-19T09:01:52.750853+0000","last_undegraded":"2024-09-19T09:01:52.750853+0000","last_fullsized":"2024-09-19T09:01:52.750853+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:44:21.739311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.614525+0000","last_change":"2024-09-19T09:00:46.955344+0000","last_active":"2024-09-19T09:02:21.614525+0000","last_peered":"2024-09-19T09:02:21.614525+0000","last_clean":"2024-09-19T09:02:21.614525+0000","last_became_active":"2024-09-19T09:00:46.954961+0000","last_became_peered":"2024-09-19T09:00:46.954961+0000","last_unstale":"2024-09-19T09:02:21.614525+0000","last_undegraded":"2024-09-19T09:02:21.614525+0000","last_fullsized":"2024-09-19T09:02:21.614525+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:55.743556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752072+0000","last_change":"2024-09-19T09:00:45.241514+0000","last_active":"2024-09-19T09:01:52.752072+0000","last_peered":"2024-09-19T09:01:52.752072+0000","last_clean":"2024-09-19T09:01:52.752072+0000","last_became_active":"2024-09-19T09:00:45.241097+0000","last_became_peered":"2024-09-19T09:00:45.241097+0000","last_unstale":"2024-09-19T09:01:52.752072+0000","last_undegraded":"2024-09-19T09:01:52.752072+0000","last_fullsized":"2024-09-19T09:01:52.752072+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:35:38.323393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620312+0000","last_change":"2024-09-19T09:00:46.955194+0000","last_active":"2024-09-19T09:02:21.620312+0000","last_peered":"2024-09-19T09:02:21.620312+0000","last_clean":"2024-09-19T09:02:21.620312+0000","last_became_active":"2024-09-19T09:00:46.954771+0000","last_became_peered":"2024-09-19T09:00:46.954771+0000","last_unstale":"2024-09-19T09:02:21.620312+0000","last_undegraded":"2024-09-19T09:02:21.620312+0000","last_fullsized":"2024-09-19T09:02:21.620312+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:18.945035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'17","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:40.978139+0000","last_change":"2024-09-19T09:00:44.938554+0000","last_active":"2024-09-19T09:02:40.978139+0000","last_peered":"2024-09-19T09:02:40.978139+0000","last_clean":"2024-09-19T09:02:40.978139+0000","last_became_active":"2024-09-19T09:00:44.938261+0000","last_became_peered":"2024-09-19T09:00:44.938261+0000","last_unstale":"2024-09-19T09:02:40.978139+0000","last_undegraded":"2024-09-19T09:02:40.978139+0000","last_fullsized":"2024-09-19T09:02:40.978139+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:24.293764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.614207+0000","last_change":"2024-09-19T09:01:52.755528+0000","last_active":"2024-09-19T09:02:21.614207+0000","last_peered":"2024-09-19T09:02:21.614207+0000","last_clean":"2024-09-19T09:02:21.614207+0000","last_became_active":"2024-09-19T09:01:52.755188+0000","last_became_peered":"2024-09-19T09:01:52.755188+0000","last_unstale":"2024-09-19T09:02:21.614207+0000","last_undegraded":"2024-09-19T09:02:21.614207+0000","last_fullsized":"2024-09-19T09:02:21.614207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:49:34.988733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.888813+0000","last_change":"2024-09-19T09:00:44.941479+0000","last_active":"2024-09-19T09:01:52.888813+0000","last_peered":"2024-09-19T09:01:52.888813+0000","last_clean":"2024-09-19T09:01:52.888813+0000","last_became_active":"2024-09-19T09:00:44.937197+0000","last_became_peered":"2024-09-19T09:00:44.937197+0000","last_unstale":"2024-09-19T09:01:52.888813+0000","last_undegraded":"2024-09-19T09:01:52.888813+0000","last_fullsized":"2024-09-19T09:01:52.888813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:53:17.385884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621026+0000","last_change":"2024-09-19T09:00:46.970319+0000","last_active":"2024-09-19T09:02:21.621026+0000","last_peered":"2024-09-19T09:02:21.621026+0000","last_clean":"2024-09-19T09:02:21.621026+0000","last_became_active":"2024-09-19T09:00:46.969899+0000","last_became_peered":"2024-09-19T09:00:46.969899+0000","last_unstale":"2024-09-19T09:02:21.621026+0000","last_undegraded":"2024-09-19T09:02:21.621026+0000","last_fullsized":"2024-09-19T09:02:21.621026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:26:21.379231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751656+0000","last_change":"2024-09-19T09:00:45.239525+0000","last_active":"2024-09-19T09:01:52.751656+0000","last_peered":"2024-09-19T09:01:52.751656+0000","last_clean":"2024-09-19T09:01:52.751656+0000","last_became_active":"2024-09-19T09:00:45.239213+0000","last_became_peered":"2024-09-19T09:00:45.239213+0000","last_unstale":"2024-09-19T09:01:52.751656+0000","last_undegraded":"2024-09-19T09:01:52.751656+0000","last_fullsized":"2024-09-19T09:01:52.751656+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:32.525025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.584763+0000","last_change":"2024-09-19T09:00:46.958573+0000","last_active":"2024-09-19T09:02:21.584763+0000","last_peered":"2024-09-19T09:02:21.584763+0000","last_clean":"2024-09-19T09:02:21.584763+0000","last_became_active":"2024-09-19T09:00:46.958274+0000","last_became_peered":"2024-09-19T09:00:46.958274+0000","last_unstale":"2024-09-19T09:02:21.584763+0000","last_undegraded":"2024-09-19T09:02:21.584763+0000","last_fullsized":"2024-09-19T09:02:21.584763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:17.249122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.903274+0000","last_change":"2024-09-19T09:00:44.945194+0000","last_active":"2024-09-19T09:01:52.903274+0000","last_peered":"2024-09-19T09:01:52.903274+0000","last_clean":"2024-09-19T09:01:52.903274+0000","last_became_active":"2024-09-19T09:00:44.944931+0000","last_became_peered":"2024-09-19T09:00:44.944931+0000","last_unstale":"2024-09-19T09:01:52.903274+0000","last_undegraded":"2024-09-19T09:01:52.903274+0000","last_fullsized":"2024-09-19T09:01:52.903274+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:11.181150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621916+0000","last_change":"2024-09-19T09:00:46.966895+0000","last_active":"2024-09-19T09:02:21.621916+0000","last_peered":"2024-09-19T09:02:21.621916+0000","last_clean":"2024-09-19T09:02:21.621916+0000","last_became_active":"2024-09-19T09:00:46.966124+0000","last_became_peered":"2024-09-19T09:00:46.966124+0000","last_unstale":"2024-09-19T09:02:21.621916+0000","last_undegraded":"2024-09-19T09:02:21.621916+0000","last_fullsized":"2024-09-19T09:02:21.621916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:14:04.320863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:41.120672+0000","last_change":"2024-09-19T09:00:44.944167+0000","last_active":"2024-09-19T09:02:41.120672+0000","last_peered":"2024-09-19T09:02:41.120672+0000","last_clean":"2024-09-19T09:02:41.120672+0000","last_became_active":"2024-09-19T09:00:44.943670+0000","last_became_peered":"2024-09-19T09:00:44.943670+0000","last_unstale":"2024-09-19T09:02:41.120672+0000","last_undegraded":"2024-09-19T09:02:41.120672+0000","last_fullsized":"2024-09-19T09:02:41.120672+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:05.596877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.615278+0000","last_change":"2024-09-19T09:00:46.958581+0000","last_active":"2024-09-19T09:02:21.615278+0000","last_peered":"2024-09-19T09:02:21.615278+0000","last_clean":"2024-09-19T09:02:21.615278+0000","last_became_active":"2024-09-19T09:00:46.957782+0000","last_became_peered":"2024-09-19T09:00:46.957782+0000","last_unstale":"2024-09-19T09:02:21.615278+0000","last_undegraded":"2024-09-19T09:02:21.615278+0000","last_fullsized":"2024-09-19T09:02:21.615278+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:37:19.488538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751779+0000","last_change":"2024-09-19T09:00:44.945351+0000","last_active":"2024-09-19T09:01:52.751779+0000","last_peered":"2024-09-19T09:01:52.751779+0000","last_clean":"2024-09-19T09:01:52.751779+0000","last_became_active":"2024-09-19T09:00:44.944991+0000","last_became_peered":"2024-09-19T09:00:44.944991+0000","last_unstale":"2024-09-19T09:01:52.751779+0000","last_undegraded":"2024-09-19T09:01:52.751779+0000","last_fullsized":"2024-09-19T09:01:52.751779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:45:07.581872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633316+0000","last_change":"2024-09-19T09:00:46.956340+0000","last_active":"2024-09-19T09:02:21.633316+0000","last_peered":"2024-09-19T09:02:21.633316+0000","last_clean":"2024-09-19T09:02:21.633316+0000","last_became_active":"2024-09-19T09:00:46.955956+0000","last_became_peered":"2024-09-19T09:00:46.955956+0000","last_unstale":"2024-09-19T09:02:21.633316+0000","last_undegraded":"2024-09-19T09:02:21.633316+0000","last_fullsized":"2024-09-19T09:02:21.633316+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:34.863428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633515+0000","last_change":"2024-09-19T09:00:46.963657+0000","last_active":"2024-09-19T09:02:21.633515+0000","last_peered":"2024-09-19T09:02:21.633515+0000","last_clean":"2024-09-19T09:02:21.633515+0000","last_became_active":"2024-09-19T09:00:46.963134+0000","last_became_peered":"2024-09-19T09:00:46.963134+0000","last_unstale":"2024-09-19T09:02:21.633515+0000","last_undegraded":"2024-09-19T09:02:21.633515+0000","last_fullsized":"2024-09-19T09:02:21.633515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:01.098570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.764939+0000","last_change":"2024-09-19T09:01:52.764438+0000","last_active":"2024-09-19T09:01:52.764939+0000","last_peered":"2024-09-19T09:01:52.764939+0000","last_clean":"2024-09-19T09:01:52.764939+0000","last_became_active":"2024-09-19T09:01:52.755188+0000","last_became_peered":"2024-09-19T09:01:52.755188+0000","last_unstale":"2024-09-19T09:01:52.764939+0000","last_undegraded":"2024-09-19T09:01:52.764939+0000","last_fullsized":"2024-09-19T09:01:52.764939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:40.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.632994+0000","last_change":"2024-09-19T09:00:46.956354+0000","last_active":"2024-09-19T09:02:21.632994+0000","last_peered":"2024-09-19T09:02:21.632994+0000","last_clean":"2024-09-19T09:02:21.632994+0000","last_became_active":"2024-09-19T09:00:46.955964+0000","last_became_peered":"2024-09-19T09:00:46.955964+0000","last_unstale":"2024-09-19T09:02:21.632994+0000","last_undegraded":"2024-09-19T09:02:21.632994+0000","last_fullsized":"2024-09-19T09:02:21.632994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:05:33.243479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751944+0000","last_change":"2024-09-19T09:00:44.944983+0000","last_active":"2024-09-19T09:01:52.751944+0000","last_peered":"2024-09-19T09:01:52.751944+0000","last_clean":"2024-09-19T09:01:52.751944+0000","last_became_active":"2024-09-19T09:00:44.944625+0000","last_became_peered":"2024-09-19T09:00:44.944625+0000","last_unstale":"2024-09-19T09:01:52.751944+0000","last_undegraded":"2024-09-19T09:01:52.751944+0000","last_fullsized":"2024-09-19T09:01:52.751944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:49:50.818863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.619748+0000","last_change":"2024-09-19T09:00:46.966448+0000","last_active":"2024-09-19T09:02:21.619748+0000","last_peered":"2024-09-19T09:02:21.619748+0000","last_clean":"2024-09-19T09:02:21.619748+0000","last_became_active":"2024-09-19T09:00:46.966135+0000","last_became_peered":"2024-09-19T09:00:46.966135+0000","last_unstale":"2024-09-19T09:02:21.619748+0000","last_undegraded":"2024-09-19T09:02:21.619748+0000","last_fullsized":"2024-09-19T09:02:21.619748+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:42.652877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.903658+0000","last_change":"2024-09-19T09:00:45.239405+0000","last_active":"2024-09-19T09:01:52.903658+0000","last_peered":"2024-09-19T09:01:52.903658+0000","last_clean":"2024-09-19T09:01:52.903658+0000","last_became_active":"2024-09-19T09:00:45.239002+0000","last_became_peered":"2024-09-19T09:00:45.239002+0000","last_unstale":"2024-09-19T09:01:52.903658+0000","last_undegraded":"2024-09-19T09:01:52.903658+0000","last_fullsized":"2024-09-19T09:01:52.903658+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:46:03.934975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624975+0000","last_change":"2024-09-19T09:00:46.951635+0000","last_active":"2024-09-19T09:02:21.624975+0000","last_peered":"2024-09-19T09:02:21.624975+0000","last_clean":"2024-09-19T09:02:21.624975+0000","last_became_active":"2024-09-19T09:00:46.951325+0000","last_became_peered":"2024-09-19T09:00:46.951325+0000","last_unstale":"2024-09-19T09:02:21.624975+0000","last_undegraded":"2024-09-19T09:02:21.624975+0000","last_fullsized":"2024-09-19T09:02:21.624975+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:17.360950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":163,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:44.867191+0000","last_change":"2024-09-19T09:01:52.766049+0000","last_active":"2024-09-19T09:02:44.867191+0000","last_peered":"2024-09-19T09:02:44.867191+0000","last_clean":"2024-09-19T09:02:44.867191+0000","last_became_active":"2024-09-19T09:01:52.755263+0000","last_became_peered":"2024-09-19T09:01:52.755263+0000","last_unstale":"2024-09-19T09:02:44.867191+0000","last_undegraded":"2024-09-19T09:02:44.867191+0000","last_fullsized":"2024-09-19T09:02:44.867191+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:16.968329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":77751,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1120,"num_read_kb":643,"num_write":266,"num_write_kb":159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1366,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":233253,"data_compressed":4815,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":212166,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612645,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":429768,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841578,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1044480,"data_stored":1089778,"data_compressed":43295,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972400,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":428051,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201332,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":427847,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430266,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":1020726,"data_compressed":41690,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593792,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":427768,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822724,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":1020542,"data_compressed":41690,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986250,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":428163,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215182,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":427593,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":499584,"data_compressed":7388,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":966656,"data_stored":498887,"data_compressed":7388,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":429500,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71958,"data_compressed":1605,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":72655,"data_compressed":1605,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":72364,"data_compressed":1605,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:02:52.416 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T09:02:52.491 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:52.949 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:52 smithi120 ceph-mon[31150]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:02:52.950 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:52 smithi120 ceph-mon[31150]: pgmap v341: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-19T09:02:53.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.031+0000 7ff4b8b6f640 1 -- 172.21.15.120:0/3238301052 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4b41027a0 msgr2=0x7ff4b41067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:53.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.031+0000 7ff4b8b6f640 1 --2- 172.21.15.120:0/3238301052 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4b41027a0 0x7ff4b41067f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009900 tx=0x7ff49c02a410 comp rx=0 tx=0).stop 2024-09-19T09:02:53.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.031+0000 7ff4b8b6f640 1 -- 172.21.15.120:0/3238301052 shutdown_connections 2024-09-19T09:02:53.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.031+0000 7ff4b8b6f640 1 --2- 172.21.15.120:0/3238301052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4b4106f20 0x7ff4b410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.031+0000 7ff4b8b6f640 1 --2- 172.21.15.120:0/3238301052 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4b41027a0 0x7ff4b41067f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.031+0000 7ff4b8b6f640 1 --2- 172.21.15.120:0/3238301052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4b4101df0 0x7ff4b41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.031+0000 7ff4b8b6f640 1 -- 172.21.15.120:0/3238301052 >> 172.21.15.120:0/3238301052 conn(0x7ff4b40fd8b0 msgr2=0x7ff4b40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:53.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.031+0000 7ff4b8b6f640 1 -- 172.21.15.120:0/3238301052 shutdown_connections 2024-09-19T09:02:53.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.031+0000 7ff4b8b6f640 1 -- 172.21.15.120:0/3238301052 wait complete. 2024-09-19T09:02:53.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.032+0000 7ff4b8b6f640 1 Processor -- start 2024-09-19T09:02:53.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.032+0000 7ff4b8b6f640 1 -- start start 2024-09-19T09:02:53.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.032+0000 7ff4b8b6f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4b4101df0 0x7ff4b40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:53.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.032+0000 7ff4b8b6f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4b41027a0 0x7ff4b4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:53.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.032+0000 7ff4b8b6f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4b4106f20 0x7ff4b4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:53.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.032+0000 7ff4b8b6f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b4111f80 con 0x7ff4b4106f20 2024-09-19T09:02:53.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.032+0000 7ff4b8b6f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b4111e00 con 0x7ff4b4101df0 2024-09-19T09:02:53.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.032+0000 7ff4b8b6f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b4112100 con 0x7ff4b41027a0 2024-09-19T09:02:53.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.032+0000 7ff4b2d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4b4106f20 0x7ff4b4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:53.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.032+0000 7ff4b2d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4b4106f20 0x7ff4b4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51606/0 (socket says 172.21.15.120:51606) 2024-09-19T09:02:53.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.032+0000 7ff4b2d9d640 1 -- 172.21.15.120:0/2635699284 learned_addr learned my addr 172.21.15.120:0/2635699284 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:53.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.033+0000 7ff4b259c640 1 --2- 172.21.15.120:0/2635699284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4b4101df0 0x7ff4b40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:53.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.033+0000 7ff4b1d9b640 1 --2- 172.21.15.120:0/2635699284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4b41027a0 0x7ff4b4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:53.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.033+0000 7ff4b2d9d640 1 -- 172.21.15.120:0/2635699284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4b41027a0 msgr2=0x7ff4b4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:53.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.033+0000 7ff4b2d9d640 1 --2- 172.21.15.120:0/2635699284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4b41027a0 0x7ff4b4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.033+0000 7ff4b2d9d640 1 -- 172.21.15.120:0/2635699284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4b4101df0 msgr2=0x7ff4b40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:53.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.033+0000 7ff4b2d9d640 1 --2- 172.21.15.120:0/2635699284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4b4101df0 0x7ff4b40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.033+0000 7ff4b2d9d640 1 -- 172.21.15.120:0/2635699284 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b41a68a0 con 0x7ff4b4106f20 2024-09-19T09:02:53.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.033+0000 7ff4b2d9d640 1 --2- 172.21.15.120:0/2635699284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4b4106f20 0x7ff4b4074f70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400d750 tx=0x7ff4a400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:53.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.033+0000 7ff4ab7fe640 1 -- 172.21.15.120:0/2635699284 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4017070 con 0x7ff4b4106f20 2024-09-19T09:02:53.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.033+0000 7ff4b8b6f640 1 -- 172.21.15.120:0/2635699284 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b41a6b90 con 0x7ff4b4106f20 2024-09-19T09:02:53.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.033+0000 7ff4ab7fe640 1 -- 172.21.15.120:0/2635699284 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a4005500 con 0x7ff4b4106f20 2024-09-19T09:02:53.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.033+0000 7ff4ab7fe640 1 -- 172.21.15.120:0/2635699284 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4004d90 con 0x7ff4b4106f20 2024-09-19T09:02:53.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.033+0000 7ff4b8b6f640 1 -- 172.21.15.120:0/2635699284 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b41a7080 con 0x7ff4b4106f20 2024-09-19T09:02:53.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.036+0000 7ff4ab7fe640 1 -- 172.21.15.120:0/2635699284 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7ff4a4004f30 con 0x7ff4b4106f20 2024-09-19T09:02:53.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.036+0000 7ff4baf48740 1 Processor -- start 2024-09-19T09:02:53.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.036+0000 7ff4baf48740 1 -- start start 2024-09-19T09:02:53.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.037+0000 7ff4baf48740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e0d4f64a0 0x560e0d4f6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:53.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.037+0000 7ff4baf48740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e0d4f6dc0 0x560e0d501140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:53.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.037+0000 7ff4baf48740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e0d501680 0x560e0d503a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:53.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.037+0000 7ff4baf48740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560e0d45c2e0 con 0x560e0d4f64a0 2024-09-19T09:02:53.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.037+0000 7ff4baf48740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560e0d403ce0 con 0x560e0d4f6dc0 2024-09-19T09:02:53.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.037+0000 7ff4baf48740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560e0d4582a0 con 0x560e0d501680 2024-09-19T09:02:53.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.037+0000 7ff4b1d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e0d4f64a0 0x560e0d4f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:53.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.037+0000 7ff4b1d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e0d4f64a0 0x560e0d4f6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51610/0 (socket says 172.21.15.120:51610) 2024-09-19T09:02:53.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.037+0000 7ff4b1d9b640 1 -- 172.21.15.120:0/1541296667 learned_addr learned my addr 172.21.15.120:0/1541296667 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:53.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.037+0000 7ff4b259c640 1 --2- 172.21.15.120:0/1541296667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e0d4f6dc0 0x560e0d501140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:53.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.037+0000 7ff4ab7fe640 1 --2- 172.21.15.120:0/2635699284 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff4a00a32d0 0x7ff4a00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:53.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.037+0000 7ff4b1d9b640 1 --2- 172.21.15.120:0/1541296667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e0d501680 0x560e0d503a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:53.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.037+0000 7ff4b2d9d640 1 --2- 172.21.15.120:0/2635699284 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff4a00a32d0 0x7ff4a00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:53.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.037+0000 7ff4b259c640 1 -- 172.21.15.120:0/1541296667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e0d501680 msgr2=0x560e0d503a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:53.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.037+0000 7ff4b259c640 1 --2- 172.21.15.120:0/1541296667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e0d501680 0x560e0d503a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.037+0000 7ff4b259c640 1 -- 172.21.15.120:0/1541296667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e0d4f64a0 msgr2=0x560e0d4f6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:53.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.037+0000 7ff4b259c640 1 --2- 172.21.15.120:0/1541296667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e0d4f64a0 0x560e0d4f6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.037+0000 7ff4b259c640 1 -- 172.21.15.120:0/1541296667 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e0d3e8440 con 0x560e0d4f6dc0 2024-09-19T09:02:53.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.037+0000 7ff4b1d9b640 1 --2- 172.21.15.120:0/1541296667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e0d4f64a0 0x560e0d4f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:02:53.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.037+0000 7ff4ab7fe640 1 -- 172.21.15.120:0/2635699284 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7ff4a40cf410 con 0x7ff4b4106f20 2024-09-19T09:02:53.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.037+0000 7ff4b259c640 1 --2- 172.21.15.120:0/1541296667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e0d4f6dc0 0x560e0d501140 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac009fb0 tx=0x7ff4ac010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66e20084e399a593 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:53.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.038+0000 7ff4b2d9d640 1 --2- 172.21.15.120:0/2635699284 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff4a00a32d0 0x7ff4a00a5790 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff4a40982a0 tx=0x7ff4a4086040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:53.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.038+0000 7ff4a8ff9640 1 -- 172.21.15.120:0/1541296667 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac016070 con 0x560e0d4f6dc0 2024-09-19T09:02:53.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.038+0000 7ff4a8ff9640 1 -- 172.21.15.120:0/1541296667 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4ac004d20 con 0x560e0d4f6dc0 2024-09-19T09:02:53.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.038+0000 7ff4a8ff9640 1 -- 172.21.15.120:0/1541296667 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac0053c0 con 0x560e0d4f6dc0 2024-09-19T09:02:53.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.038+0000 7ff4baf48740 1 -- 172.21.15.120:0/1541296667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e0d4f6dc0 msgr2=0x560e0d501140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:53.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.038+0000 7ff4baf48740 1 --2- 172.21.15.120:0/1541296667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e0d4f6dc0 0x560e0d501140 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac009fb0 tx=0x7ff4ac010800 comp rx=0 tx=0).stop 2024-09-19T09:02:53.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.038+0000 7ff4baf48740 1 -- 172.21.15.120:0/1541296667 shutdown_connections 2024-09-19T09:02:53.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.038+0000 7ff4baf48740 1 --2- 172.21.15.120:0/1541296667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e0d501680 0x560e0d503a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.038+0000 7ff4baf48740 1 --2- 172.21.15.120:0/1541296667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e0d4f6dc0 0x560e0d501140 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.038+0000 7ff4baf48740 1 --2- 172.21.15.120:0/1541296667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e0d4f64a0 0x560e0d4f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.038+0000 7ff4baf48740 1 -- 172.21.15.120:0/1541296667 >> 172.21.15.120:0/1541296667 conn(0x560e0d4f2a30 msgr2=0x560e0d4f2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:53.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.039+0000 7ff4baf48740 1 -- 172.21.15.120:0/1541296667 shutdown_connections 2024-09-19T09:02:53.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.039+0000 7ff4baf48740 1 -- 172.21.15.120:0/1541296667 wait complete. 2024-09-19T09:02:53.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.039+0000 7ff4baf48740 1 -- start start 2024-09-19T09:02:53.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.039+0000 7ff4baf48740 1 Processor -- start 2024-09-19T09:02:53.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.039+0000 7ff4baf48740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e0d4f64a0 0x560e0d500f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:53.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.039+0000 7ff4baf48740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e0d4f6dc0 0x560e0d4fe1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:53.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.039+0000 7ff4baf48740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e0d501680 0x560e0d4fe730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:53.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.039+0000 7ff4baf48740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560e0d504250 con 0x560e0d4f6dc0 2024-09-19T09:02:53.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.039+0000 7ff4baf48740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560e0d417050 con 0x560e0d4f64a0 2024-09-19T09:02:53.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.039+0000 7ff4baf48740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560e0d5043d0 con 0x560e0d501680 2024-09-19T09:02:53.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.039+0000 7ff4baf48740 10 client.0 _ll_register_callbacks cb 0x560e0d457160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:02:53.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.039+0000 7ff4b259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e0d4f6dc0 0x560e0d4fe1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:53.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.039+0000 7ff4b259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e0d4f6dc0 0x560e0d4fe1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51614/0 (socket says 172.21.15.120:51614) 2024-09-19T09:02:53.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.039+0000 7ff4b1d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e0d4f64a0 0x560e0d500f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:53.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.039+0000 7ff4b259c640 1 -- 172.21.15.120:0/333837491 learned_addr learned my addr 172.21.15.120:0/333837491 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:53.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.039+0000 7ff4b1d9b640 1 --2- 172.21.15.120:0/333837491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e0d501680 0x560e0d4fe730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:53.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.040+0000 7ff4b259c640 1 -- 172.21.15.120:0/333837491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e0d501680 msgr2=0x560e0d4fe730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:53.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.040+0000 7ff4b259c640 1 --2- 172.21.15.120:0/333837491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e0d501680 0x560e0d4fe730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.040+0000 7ff4b259c640 1 -- 172.21.15.120:0/333837491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e0d4f64a0 msgr2=0x560e0d500f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:53.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.040+0000 7ff4b259c640 1 --2- 172.21.15.120:0/333837491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e0d4f64a0 0x560e0d500f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.040+0000 7ff4b259c640 1 -- 172.21.15.120:0/333837491 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e0d1cd3c0 con 0x560e0d4f6dc0 2024-09-19T09:02:53.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.040+0000 7ff4b1d9b640 1 --2- 172.21.15.120:0/333837491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e0d4f64a0 0x560e0d500f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:02:53.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.040+0000 7ff4b259c640 1 --2- 172.21.15.120:0/333837491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e0d4f6dc0 0x560e0d4fe1f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac005c10 tx=0x7ff4ac012dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9edc35a51b1da0d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:53.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.040+0000 7ff499ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:02:53.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.040+0000 7ff499ffb640 1 -- 172.21.15.120:0/333837491 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac016030 con 0x560e0d4f6dc0 2024-09-19T09:02:53.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.040+0000 7ff499ffb640 1 -- 172.21.15.120:0/333837491 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7ff4ac003b80 con 0x560e0d4f6dc0 2024-09-19T09:02:53.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.040+0000 7ff4baf48740 1 -- 172.21.15.120:0/333837491 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4ac004d20 con 0x560e0d4f6dc0 2024-09-19T09:02:53.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.040+0000 7ff499ffb640 1 -- 172.21.15.120:0/333837491 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff4ac003d80 con 0x560e0d4f6dc0 2024-09-19T09:02:53.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.040+0000 7ff4baf48740 10 client.15318 fetch_fsmap learned FSMap version 28 2024-09-19T09:02:53.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.040+0000 7ff4baf48740 1 -- 172.21.15.120:0/333837491 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560e0d50fba0 con 0x560e0d4f6dc0 2024-09-19T09:02:53.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.040+0000 7ff499ffb640 1 -- 172.21.15.120:0/333837491 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4ac004d20 con 0x560e0d4f6dc0 2024-09-19T09:02:53.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.041+0000 7ff499ffb640 1 -- 172.21.15.120:0/333837491 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff4ac00f040 con 0x560e0d4f6dc0 2024-09-19T09:02:53.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.041+0000 7ff4baf48740 10 client.15318 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:02:53.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.041+0000 7ff4baf48740 10 client.15318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:02:53.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.041+0000 7ff4baf48740 20 client.15318 populate_metadata read hostname 'smithi120' 2024-09-19T09:02:53.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.041+0000 7ff4baf48740 1 --2- 172.21.15.120:0/333837491 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560e0d510fb0 0x560e0d5133a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:53.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.041+0000 7ff4baf48740 4 client.15318 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:02:53.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.041+0000 7ff4baf48740 1 -- 172.21.15.120:0/333837491 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e0d469850 con 0x560e0d510fb0 2024-09-19T09:02:53.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.041+0000 7ff4b1d9b640 1 --2- 172.21.15.120:0/333837491 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560e0d510fb0 0x560e0d5133a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:53.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.041+0000 7ff499ffb640 10 client.15318 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:02:53.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.041+0000 7ff4b1d9b640 1 --2- 172.21.15.120:0/333837491 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560e0d510fb0 0x560e0d5133a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e563703db131a7b5 server_cookie=16154f3249b01467 in_seq=0 out_seq=0 2024-09-19T09:02:53.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.042+0000 7ff499ffb640 1 -- 172.21.15.120:0/333837491 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e0d469850 con 0x560e0d510fb0 2024-09-19T09:02:53.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.042+0000 7ff499ffb640 10 client.15318 handle_command_reply: tid=0 2024-09-19T09:02:53.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.042+0000 7ff4baf48740 1 client.15318 shutdown 2024-09-19T09:02:53.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.042+0000 7ff4baf48740 1 -- 172.21.15.120:0/333837491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e0d4f6dc0 msgr2=0x560e0d4fe1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:53.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.042+0000 7ff4baf48740 1 --2- 172.21.15.120:0/333837491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e0d4f6dc0 0x560e0d4fe1f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac005c10 tx=0x7ff4ac012dc0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.042+0000 7ff4baf48740 1 -- 172.21.15.120:0/333837491 shutdown_connections 2024-09-19T09:02:53.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.042+0000 7ff4baf48740 1 --2- 172.21.15.120:0/333837491 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560e0d510fb0 0x560e0d5133a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.042+0000 7ff4baf48740 1 --2- 172.21.15.120:0/333837491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e0d501680 0x560e0d4fe730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.042+0000 7ff4baf48740 1 --2- 172.21.15.120:0/333837491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e0d4f6dc0 0x560e0d4fe1f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.042+0000 7ff499ffb640 0 client.15318 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:02:53.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.042+0000 7ff4baf48740 1 --2- 172.21.15.120:0/333837491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e0d4f64a0 0x560e0d500f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.042+0000 7ff4baf48740 1 -- 172.21.15.120:0/333837491 >> 172.21.15.120:0/333837491 conn(0x560e0d4f2a30 msgr2=0x560e0d4f1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:53.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.042+0000 7ff4baf48740 1 -- 172.21.15.120:0/333837491 shutdown_connections 2024-09-19T09:02:53.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.042+0000 7ff4baf48740 1 -- 172.21.15.120:0/333837491 wait complete. 2024-09-19T09:02:53.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.042+0000 7ff4baf48740 20 client.15318 trim_cache size 0 max 16384 2024-09-19T09:02:53.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.060+0000 7ff4baf48740 1 Processor -- start 2024-09-19T09:02:53.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.060+0000 7ff4baf48740 1 -- start start 2024-09-19T09:02:53.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.060+0000 7ff4baf48740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e0d4f64a0 0x560e0d50d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:53.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.060+0000 7ff4baf48740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e0d4f6dc0 0x560e0d518b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:53.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.060+0000 7ff4baf48740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e0d501680 0x560e0d519070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:53.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.060+0000 7ff4baf48740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b41af740 con 0x560e0d4f64a0 2024-09-19T09:02:53.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.060+0000 7ff4baf48740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b41b1280 con 0x560e0d4f6dc0 2024-09-19T09:02:53.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.060+0000 7ff4baf48740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b41afe10 con 0x560e0d501680 2024-09-19T09:02:53.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.060+0000 7ff4b1d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e0d4f64a0 0x560e0d50d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:53.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.060+0000 7ff4b1d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e0d4f64a0 0x560e0d50d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51622/0 (socket says 172.21.15.120:51622) 2024-09-19T09:02:53.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.060+0000 7ff4b1d9b640 1 -- 172.21.15.120:0/2819194313 learned_addr learned my addr 172.21.15.120:0/2819194313 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:53.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.060+0000 7ff4b1d9b640 1 -- 172.21.15.120:0/2819194313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e0d501680 msgr2=0x560e0d519070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:53.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.060+0000 7ff4b1d9b640 1 --2- 172.21.15.120:0/2819194313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e0d501680 0x560e0d519070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.060+0000 7ff4b259c640 1 --2- 172.21.15.120:0/2819194313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e0d4f6dc0 0x560e0d518b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:53.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.060+0000 7ff4b1d9b640 1 -- 172.21.15.120:0/2819194313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e0d4f6dc0 msgr2=0x560e0d518b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:53.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.060+0000 7ff4b1d9b640 1 --2- 172.21.15.120:0/2819194313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e0d4f6dc0 0x560e0d518b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.060+0000 7ff4b1d9b640 1 -- 172.21.15.120:0/2819194313 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e0d4f6220 con 0x560e0d4f64a0 2024-09-19T09:02:53.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.060+0000 7ff4b1d9b640 1 --2- 172.21.15.120:0/2819194313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e0d4f64a0 0x560e0d50d4b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff49c02a930 tx=0x7ff49c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b14cba15837d4cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:53.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.061+0000 7ff49bfff640 1 -- 172.21.15.120:0/2819194313 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49c035070 con 0x560e0d4f64a0 2024-09-19T09:02:53.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.061+0000 7ff49bfff640 1 -- 172.21.15.120:0/2819194313 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff49c0319a0 con 0x560e0d4f64a0 2024-09-19T09:02:53.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.061+0000 7ff49bfff640 1 -- 172.21.15.120:0/2819194313 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49c003810 con 0x560e0d4f64a0 2024-09-19T09:02:53.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.061+0000 7ff4baf48740 1 -- 172.21.15.120:0/2819194313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e0d4f64a0 msgr2=0x560e0d50d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:53.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.061+0000 7ff4baf48740 1 --2- 172.21.15.120:0/2819194313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e0d4f64a0 0x560e0d50d4b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff49c02a930 tx=0x7ff49c005b50 comp rx=0 tx=0).stop 2024-09-19T09:02:53.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.061+0000 7ff4baf48740 1 -- 172.21.15.120:0/2819194313 shutdown_connections 2024-09-19T09:02:53.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.061+0000 7ff4baf48740 1 --2- 172.21.15.120:0/2819194313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e0d501680 0x560e0d519070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.061+0000 7ff4baf48740 1 --2- 172.21.15.120:0/2819194313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e0d4f6dc0 0x560e0d518b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.061+0000 7ff4baf48740 1 --2- 172.21.15.120:0/2819194313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e0d4f64a0 0x560e0d50d4b0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.061+0000 7ff4baf48740 1 -- 172.21.15.120:0/2819194313 >> 172.21.15.120:0/2819194313 conn(0x560e0d4f2a30 msgr2=0x560e0d500270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:53.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.061+0000 7ff4baf48740 1 -- 172.21.15.120:0/2819194313 shutdown_connections 2024-09-19T09:02:53.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.061+0000 7ff4baf48740 1 -- 172.21.15.120:0/2819194313 wait complete. 2024-09-19T09:02:53.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.062+0000 7ff4baf48740 1 -- start start 2024-09-19T09:02:53.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.062+0000 7ff4baf48740 1 Processor -- start 2024-09-19T09:02:53.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.062+0000 7ff4baf48740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e0d4f64a0 0x560e0d572e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:53.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.062+0000 7ff4baf48740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e0d4f6dc0 0x560e0d5733a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:53.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.062+0000 7ff4baf48740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e0d501680 0x560e0d506bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:53.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.062+0000 7ff4baf48740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b41b0bb0 con 0x560e0d4f6dc0 2024-09-19T09:02:53.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.062+0000 7ff4baf48740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b41b04e0 con 0x560e0d4f64a0 2024-09-19T09:02:53.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.062+0000 7ff4baf48740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b41ad700 con 0x560e0d501680 2024-09-19T09:02:53.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.062+0000 7ff4baf48740 10 client.0 _ll_register_callbacks cb 0x7ff49c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:02:53.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.062+0000 7ff4b259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e0d4f6dc0 0x560e0d5733a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:53.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.062+0000 7ff4b259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e0d4f6dc0 0x560e0d5733a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51626/0 (socket says 172.21.15.120:51626) 2024-09-19T09:02:53.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.062+0000 7ff4b259c640 1 -- 172.21.15.120:0/3636891778 learned_addr learned my addr 172.21.15.120:0/3636891778 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:53.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.062+0000 7ff4b1d9b640 1 --2- 172.21.15.120:0/3636891778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e0d4f64a0 0x560e0d572e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:53.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.062+0000 7ff4b1d9b640 1 --2- 172.21.15.120:0/3636891778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e0d501680 0x560e0d506bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:53.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.062+0000 7ff4b259c640 1 -- 172.21.15.120:0/3636891778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e0d501680 msgr2=0x560e0d506bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:53.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.062+0000 7ff4b259c640 1 --2- 172.21.15.120:0/3636891778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e0d501680 0x560e0d506bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.062+0000 7ff4b259c640 1 -- 172.21.15.120:0/3636891778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e0d4f64a0 msgr2=0x560e0d572e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:02:53.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.062+0000 7ff4b259c640 1 --2- 172.21.15.120:0/3636891778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e0d4f64a0 0x560e0d572e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.062+0000 7ff4b259c640 1 -- 172.21.15.120:0/3636891778 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e0d5073c0 con 0x560e0d4f6dc0 2024-09-19T09:02:53.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.062+0000 7ff4b1d9b640 1 --2- 172.21.15.120:0/3636891778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e0d501680 0x560e0d506bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:02:53.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.063+0000 7ff4b259c640 1 --2- 172.21.15.120:0/3636891778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e0d4f6dc0 0x560e0d5733a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac006010 tx=0x7ff4ac003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427a839ce8c46b86 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:53.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.063+0000 7ff499ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:02:53.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.063+0000 7ff499ffb640 1 -- 172.21.15.120:0/3636891778 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac034060 con 0x560e0d4f6dc0 2024-09-19T09:02:53.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.063+0000 7ff499ffb640 1 -- 172.21.15.120:0/3636891778 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7ff4ac003f20 con 0x560e0d4f6dc0 2024-09-19T09:02:53.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.063+0000 7ff4baf48740 1 -- 172.21.15.120:0/3636891778 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff49c0319a0 con 0x560e0d4f6dc0 2024-09-19T09:02:53.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.063+0000 7ff499ffb640 1 -- 172.21.15.120:0/3636891778 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff4ac016070 con 0x560e0d4f6dc0 2024-09-19T09:02:53.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.063+0000 7ff4baf48740 10 client.15324 fetch_fsmap learned FSMap version 28 2024-09-19T09:02:53.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.063+0000 7ff4baf48740 1 -- 172.21.15.120:0/3636891778 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560e0d57b630 con 0x560e0d4f6dc0 2024-09-19T09:02:53.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.063+0000 7ff499ffb640 1 -- 172.21.15.120:0/3636891778 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff49c0319a0 con 0x560e0d4f6dc0 2024-09-19T09:02:53.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.063+0000 7ff499ffb640 1 -- 172.21.15.120:0/3636891778 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff4ac0256a0 con 0x560e0d4f6dc0 2024-09-19T09:02:53.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.063+0000 7ff4baf48740 10 client.15324 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:02:53.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.063+0000 7ff4baf48740 10 client.15324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:02:53.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.063+0000 7ff4baf48740 20 client.15324 populate_metadata read hostname 'smithi120' 2024-09-19T09:02:53.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.063+0000 7ff4baf48740 1 --2- 172.21.15.120:0/3636891778 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560e0d57bcf0 0x560e0d57e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:53.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.063+0000 7ff4baf48740 4 client.15324 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:02:53.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.063+0000 7ff4baf48740 1 -- 172.21.15.120:0/3636891778 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560e0d463d60 con 0x560e0d57bcf0 2024-09-19T09:02:53.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.064+0000 7ff4b1d9b640 1 --2- 172.21.15.120:0/3636891778 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560e0d57bcf0 0x560e0d57e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:53.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.064+0000 7ff4b1d9b640 1 --2- 172.21.15.120:0/3636891778 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560e0d57bcf0 0x560e0d57e0e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55ede49392bcebe0 server_cookie=176b11ea0f1e4550 in_seq=0 out_seq=0 2024-09-19T09:02:53.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.064+0000 7ff499ffb640 10 client.15324 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:02:53.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.064+0000 7ff499ffb640 1 -- 172.21.15.120:0/3636891778 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560e0d463d60 con 0x560e0d57bcf0 2024-09-19T09:02:53.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.064+0000 7ff499ffb640 10 client.15324 handle_command_reply: tid=0 2024-09-19T09:02:53.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.064+0000 7ff4baf48740 1 client.15324 shutdown 2024-09-19T09:02:53.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.064+0000 7ff4baf48740 1 -- 172.21.15.120:0/3636891778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e0d4f6dc0 msgr2=0x560e0d5733a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:53.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.064+0000 7ff4baf48740 1 --2- 172.21.15.120:0/3636891778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e0d4f6dc0 0x560e0d5733a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac006010 tx=0x7ff4ac003b80 comp rx=0 tx=0).stop 2024-09-19T09:02:53.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.064+0000 7ff4baf48740 1 -- 172.21.15.120:0/3636891778 shutdown_connections 2024-09-19T09:02:53.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.064+0000 7ff4baf48740 1 --2- 172.21.15.120:0/3636891778 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560e0d57bcf0 0x560e0d57e0e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.064+0000 7ff4baf48740 1 --2- 172.21.15.120:0/3636891778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e0d501680 0x560e0d506bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.064+0000 7ff4baf48740 1 --2- 172.21.15.120:0/3636891778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e0d4f6dc0 0x560e0d5733a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.064+0000 7ff4baf48740 1 --2- 172.21.15.120:0/3636891778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e0d4f64a0 0x560e0d572e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.064+0000 7ff4baf48740 1 -- 172.21.15.120:0/3636891778 >> 172.21.15.120:0/3636891778 conn(0x560e0d4f2a30 msgr2=0x560e0d4fc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:53.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.064+0000 7ff499ffb640 0 client.15324 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:02:53.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.064+0000 7ff4baf48740 1 -- 172.21.15.120:0/3636891778 shutdown_connections 2024-09-19T09:02:53.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.064+0000 7ff4baf48740 1 -- 172.21.15.120:0/3636891778 wait complete. 2024-09-19T09:02:53.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.065+0000 7ff4baf48740 20 client.15324 trim_cache size 0 max 16384 2024-09-19T09:02:53.077 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:02:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:02:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-19T09:02:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: "b9550c65-d6e1-4b1d-afa4-a29d12acb60a": { 2024-09-19T09:02:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-19T09:02:53.077 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "b9550c65-d6e1-4b1d-afa4-a29d12acb60a", 2024-09-19T09:02:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-19T09:02:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:02:53.078 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:02:53.078 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:02:53.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.065+0000 7ff4b8b6f640 1 -- 172.21.15.120:0/2635699284 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff4a00a32d0 msgr2=0x7ff4a00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:53.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.065+0000 7ff4b8b6f640 1 --2- 172.21.15.120:0/2635699284 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff4a00a32d0 0x7ff4a00a5790 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff4a40982a0 tx=0x7ff4a4086040 comp rx=0 tx=0).stop 2024-09-19T09:02:53.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.065+0000 7ff4b8b6f640 1 -- 172.21.15.120:0/2635699284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4b4106f20 msgr2=0x7ff4b4074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:53.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.065+0000 7ff4b8b6f640 1 --2- 172.21.15.120:0/2635699284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4b4106f20 0x7ff4b4074f70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400d750 tx=0x7ff4a400dc10 comp rx=0 tx=0).stop 2024-09-19T09:02:53.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.065+0000 7ff4b8b6f640 1 -- 172.21.15.120:0/2635699284 shutdown_connections 2024-09-19T09:02:53.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.065+0000 7ff4b8b6f640 1 --2- 172.21.15.120:0/2635699284 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff4a00a32d0 0x7ff4a00a5790 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.065+0000 7ff4b8b6f640 1 --2- 172.21.15.120:0/2635699284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4b4106f20 0x7ff4b4074f70 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.065+0000 7ff4b8b6f640 1 --2- 172.21.15.120:0/2635699284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4b41027a0 0x7ff4b4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.065+0000 7ff4b8b6f640 1 --2- 172.21.15.120:0/2635699284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4b4101df0 0x7ff4b40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:53.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.065+0000 7ff4b8b6f640 1 -- 172.21.15.120:0/2635699284 >> 172.21.15.120:0/2635699284 conn(0x7ff4b40fd8b0 msgr2=0x7ff4b40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:53.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.065+0000 7ff4b8b6f640 1 -- 172.21.15.120:0/2635699284 shutdown_connections 2024-09-19T09:02:53.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:53.065+0000 7ff4b8b6f640 1 -- 172.21.15.120:0/2635699284 wait complete. 2024-09-19T09:02:53.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:52 smithi196 ceph-mon[35150]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:02:53.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:52 smithi196 ceph-mon[35150]: pgmap v341: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-19T09:02:53.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:52 smithi153 ceph-mon[35071]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:02:53.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:52 smithi153 ceph-mon[35071]: pgmap v341: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-19T09:02:53.803 INFO:tasks.cephfs.filesystem:scrub status for tag:b9550c65-d6e1-4b1d-afa4-a29d12acb60a - {'path': '/', 'tag': 'b9550c65-d6e1-4b1d-afa4-a29d12acb60a', 'options': 'recursive,force'} 2024-09-19T09:02:53.887 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:54.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.381+0000 7fd62e00c640 1 -- 172.21.15.120:0/1795295331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6281082a0 msgr2=0x7fd62810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:54.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.381+0000 7fd62e00c640 1 --2- 172.21.15.120:0/1795295331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6281082a0 0x7fd62810ec40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd620009cd0 tx=0x7fd62002a8e0 comp rx=0 tx=0).stop 2024-09-19T09:02:54.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.381+0000 7fd62e00c640 1 -- 172.21.15.120:0/1795295331 shutdown_connections 2024-09-19T09:02:54.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.381+0000 7fd62e00c640 1 --2- 172.21.15.120:0/1795295331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6281082a0 0x7fd62810ec40 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:54.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.381+0000 7fd62e00c640 1 --2- 172.21.15.120:0/1795295331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6281036e0 0x7fd628107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:54.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.381+0000 7fd62e00c640 1 --2- 172.21.15.120:0/1795295331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd628102d30 0x7fd628103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:54.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.381+0000 7fd62e00c640 1 -- 172.21.15.120:0/1795295331 >> 172.21.15.120:0/1795295331 conn(0x7fd6280fc7d0 msgr2=0x7fd6280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:54.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.381+0000 7fd62e00c640 1 -- 172.21.15.120:0/1795295331 shutdown_connections 2024-09-19T09:02:54.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.381+0000 7fd62e00c640 1 -- 172.21.15.120:0/1795295331 wait complete. 2024-09-19T09:02:54.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.381+0000 7fd62e00c640 1 Processor -- start 2024-09-19T09:02:54.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.381+0000 7fd62e00c640 1 -- start start 2024-09-19T09:02:54.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.382+0000 7fd62e00c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd628102d30 0x7fd628198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:54.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.382+0000 7fd62e00c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6281036e0 0x7fd628198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:54.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.382+0000 7fd62e00c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6281082a0 0x7fd62819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:54.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.382+0000 7fd62e00c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd6281120a0 con 0x7fd6281036e0 2024-09-19T09:02:54.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.382+0000 7fd62e00c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd628111f20 con 0x7fd6281082a0 2024-09-19T09:02:54.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.382+0000 7fd62e00c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd628112220 con 0x7fd628102d30 2024-09-19T09:02:54.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.382+0000 7fd61ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6281036e0 0x7fd628198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:54.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.382+0000 7fd61ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6281036e0 0x7fd628198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51642/0 (socket says 172.21.15.120:51642) 2024-09-19T09:02:54.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.382+0000 7fd61ffff640 1 -- 172.21.15.120:0/1559624111 learned_addr learned my addr 172.21.15.120:0/1559624111 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:54.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.382+0000 7fd6277fe640 1 --2- 172.21.15.120:0/1559624111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd628102d30 0x7fd628198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:54.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.382+0000 7fd627fff640 1 --2- 172.21.15.120:0/1559624111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6281082a0 0x7fd62819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:54.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.382+0000 7fd61ffff640 1 -- 172.21.15.120:0/1559624111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd628102d30 msgr2=0x7fd628198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:54.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.382+0000 7fd61ffff640 1 --2- 172.21.15.120:0/1559624111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd628102d30 0x7fd628198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:54.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.382+0000 7fd61ffff640 1 -- 172.21.15.120:0/1559624111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6281082a0 msgr2=0x7fd62819ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:54.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.382+0000 7fd61ffff640 1 --2- 172.21.15.120:0/1559624111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6281082a0 0x7fd62819ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:54.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.382+0000 7fd61ffff640 1 -- 172.21.15.120:0/1559624111 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd62819d4f0 con 0x7fd6281036e0 2024-09-19T09:02:54.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.382+0000 7fd61ffff640 1 --2- 172.21.15.120:0/1559624111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6281036e0 0x7fd628198a80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd61000d740 tx=0x7fd61000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:54.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.383+0000 7fd6257fa640 1 -- 172.21.15.120:0/1559624111 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610017070 con 0x7fd6281036e0 2024-09-19T09:02:54.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.383+0000 7fd6257fa640 1 -- 172.21.15.120:0/1559624111 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd61000de50 con 0x7fd6281036e0 2024-09-19T09:02:54.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.383+0000 7fd62e00c640 1 -- 172.21.15.120:0/1559624111 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd62819d7e0 con 0x7fd6281036e0 2024-09-19T09:02:54.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.383+0000 7fd6257fa640 1 -- 172.21.15.120:0/1559624111 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61000f6b0 con 0x7fd6281036e0 2024-09-19T09:02:54.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.383+0000 7fd62e00c640 1 -- 172.21.15.120:0/1559624111 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6281a50c0 con 0x7fd6281036e0 2024-09-19T09:02:54.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.384+0000 7fd62e00c640 1 -- 172.21.15.120:0/1559624111 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd628103110 con 0x7fd6281036e0 2024-09-19T09:02:54.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.385+0000 7fd6257fa640 1 -- 172.21.15.120:0/1559624111 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7fd61001a070 con 0x7fd6281036e0 2024-09-19T09:02:54.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.386+0000 7fd6257fa640 1 --2- 172.21.15.120:0/1559624111 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd6140a32d0 0x7fd6140a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:54.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.386+0000 7fd6257fa640 1 -- 172.21.15.120:0/1559624111 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fd6100cf340 con 0x7fd6281036e0 2024-09-19T09:02:54.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.386+0000 7fd6277fe640 1 --2- 172.21.15.120:0/1559624111 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd6140a32d0 0x7fd6140a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:54.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.386+0000 7fd6277fe640 1 --2- 172.21.15.120:0/1559624111 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd6140a32d0 0x7fd6140a5790 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd61800bbe0 tx=0x7fd61800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:54.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.389+0000 7fd6257fa640 1 -- 172.21.15.120:0/1559624111 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd61009b8b0 con 0x7fd6281036e0 2024-09-19T09:02:54.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.562+0000 7fd62e00c640 1 -- 172.21.15.120:0/1559624111 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd628107420 con 0x7fd6140a32d0 2024-09-19T09:02:54.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.569+0000 7fd6257fa640 1 -- 172.21.15.120:0/1559624111 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259139 (secure 0 0 0) 0x7fd628107420 con 0x7fd6140a32d0 2024-09-19T09:02:54.572 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:02:54.573 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-19T09:02:54.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.573+0000 7fd62e00c640 1 -- 172.21.15.120:0/1559624111 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd6140a32d0 msgr2=0x7fd6140a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:54.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.573+0000 7fd62e00c640 1 --2- 172.21.15.120:0/1559624111 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd6140a32d0 0x7fd6140a5790 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd61800bbe0 tx=0x7fd61800b390 comp rx=0 tx=0).stop 2024-09-19T09:02:54.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.573+0000 7fd62e00c640 1 -- 172.21.15.120:0/1559624111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6281036e0 msgr2=0x7fd628198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:54.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.573+0000 7fd62e00c640 1 --2- 172.21.15.120:0/1559624111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6281036e0 0x7fd628198a80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd61000d740 tx=0x7fd61000dc00 comp rx=0 tx=0).stop 2024-09-19T09:02:54.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.573+0000 7fd62e00c640 1 -- 172.21.15.120:0/1559624111 shutdown_connections 2024-09-19T09:02:54.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.573+0000 7fd62e00c640 1 --2- 172.21.15.120:0/1559624111 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd6140a32d0 0x7fd6140a5790 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:54.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.573+0000 7fd62e00c640 1 --2- 172.21.15.120:0/1559624111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6281082a0 0x7fd62819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:54.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.573+0000 7fd62e00c640 1 --2- 172.21.15.120:0/1559624111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6281036e0 0x7fd628198a80 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:54.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.573+0000 7fd62e00c640 1 --2- 172.21.15.120:0/1559624111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd628102d30 0x7fd628198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:54.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.573+0000 7fd62e00c640 1 -- 172.21.15.120:0/1559624111 >> 172.21.15.120:0/1559624111 conn(0x7fd6280fc7d0 msgr2=0x7fd628105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:54.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.574+0000 7fd62e00c640 1 -- 172.21.15.120:0/1559624111 shutdown_connections 2024-09-19T09:02:54.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:54.574+0000 7fd62e00c640 1 -- 172.21.15.120:0/1559624111 wait complete. 2024-09-19T09:02:54.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:54 smithi120 ceph-mon[31150]: pgmap v342: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-19T09:02:54.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:54 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:02:54.996 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:54 smithi153 ceph-mon[35071]: pgmap v342: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-19T09:02:54.997 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:54 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:02:55.049 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:54 smithi196 ceph-mon[35150]: pgmap v342: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-19T09:02:55.050 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:54 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:02:55.313 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2024-09-19T09:02:53.638132+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668353,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1298,"num_read_kb":779,"num_write":449,"num_write_kb":3099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1182102,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364364,"kb_used_data":10956,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577460,"statfs":{"total":1151940427776,"available":1151567319040,"internally_reserved":0,"allocated":11218944,"data_stored":7128207,"data_compressed":181932,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7928,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":73,"num_write":13,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003261"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624586+0000","last_change":"2024-09-19T09:00:46.966628+0000","last_active":"2024-09-19T09:02:21.624586+0000","last_peered":"2024-09-19T09:02:21.624586+0000","last_clean":"2024-09-19T09:02:21.624586+0000","last_became_active":"2024-09-19T09:00:46.966290+0000","last_became_peered":"2024-09-19T09:00:46.966290+0000","last_unstale":"2024-09-19T09:02:21.624586+0000","last_undegraded":"2024-09-19T09:02:21.624586+0000","last_fullsized":"2024-09-19T09:02:21.624586+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:42.717139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621872+0000","last_change":"2024-09-19T09:00:46.963706+0000","last_active":"2024-09-19T09:02:21.621872+0000","last_peered":"2024-09-19T09:02:21.621872+0000","last_clean":"2024-09-19T09:02:21.621872+0000","last_became_active":"2024-09-19T09:00:46.963303+0000","last_became_peered":"2024-09-19T09:00:46.963303+0000","last_unstale":"2024-09-19T09:02:21.621872+0000","last_undegraded":"2024-09-19T09:02:21.621872+0000","last_fullsized":"2024-09-19T09:02:21.621872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:07.451500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620429+0000","last_change":"2024-09-19T09:00:46.967149+0000","last_active":"2024-09-19T09:02:21.620429+0000","last_peered":"2024-09-19T09:02:21.620429+0000","last_clean":"2024-09-19T09:02:21.620429+0000","last_became_active":"2024-09-19T09:00:46.966861+0000","last_became_peered":"2024-09-19T09:00:46.966861+0000","last_unstale":"2024-09-19T09:02:21.620429+0000","last_undegraded":"2024-09-19T09:02:21.620429+0000","last_fullsized":"2024-09-19T09:02:21.620429+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:28.396022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.605530+0000","last_change":"2024-09-19T09:00:46.969242+0000","last_active":"2024-09-19T09:02:21.605530+0000","last_peered":"2024-09-19T09:02:21.605530+0000","last_clean":"2024-09-19T09:02:21.605530+0000","last_became_active":"2024-09-19T09:00:46.968546+0000","last_became_peered":"2024-09-19T09:00:46.968546+0000","last_unstale":"2024-09-19T09:02:21.605530+0000","last_undegraded":"2024-09-19T09:02:21.605530+0000","last_fullsized":"2024-09-19T09:02:21.605530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:43:51.060130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.602962+0000","last_change":"2024-09-19T09:00:46.955501+0000","last_active":"2024-09-19T09:02:21.602962+0000","last_peered":"2024-09-19T09:02:21.602962+0000","last_clean":"2024-09-19T09:02:21.602962+0000","last_became_active":"2024-09-19T09:00:46.955161+0000","last_became_peered":"2024-09-19T09:00:46.955161+0000","last_unstale":"2024-09-19T09:02:21.602962+0000","last_undegraded":"2024-09-19T09:02:21.602962+0000","last_fullsized":"2024-09-19T09:02:21.602962+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:39:19.005931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624332+0000","last_change":"2024-09-19T09:00:46.962577+0000","last_active":"2024-09-19T09:02:21.624332+0000","last_peered":"2024-09-19T09:02:21.624332+0000","last_clean":"2024-09-19T09:02:21.624332+0000","last_became_active":"2024-09-19T09:00:46.962253+0000","last_became_peered":"2024-09-19T09:00:46.962253+0000","last_unstale":"2024-09-19T09:02:21.624332+0000","last_undegraded":"2024-09-19T09:02:21.624332+0000","last_fullsized":"2024-09-19T09:02:21.624332+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:27.925932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621157+0000","last_change":"2024-09-19T09:00:46.960004+0000","last_active":"2024-09-19T09:02:21.621157+0000","last_peered":"2024-09-19T09:02:21.621157+0000","last_clean":"2024-09-19T09:02:21.621157+0000","last_became_active":"2024-09-19T09:00:46.959664+0000","last_became_peered":"2024-09-19T09:00:46.959664+0000","last_unstale":"2024-09-19T09:02:21.621157+0000","last_undegraded":"2024-09-19T09:02:21.621157+0000","last_fullsized":"2024-09-19T09:02:21.621157+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:41.616309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622980+0000","last_change":"2024-09-19T09:00:46.952394+0000","last_active":"2024-09-19T09:02:21.622980+0000","last_peered":"2024-09-19T09:02:21.622980+0000","last_clean":"2024-09-19T09:02:21.622980+0000","last_became_active":"2024-09-19T09:00:46.952084+0000","last_became_peered":"2024-09-19T09:00:46.952084+0000","last_unstale":"2024-09-19T09:02:21.622980+0000","last_undegraded":"2024-09-19T09:02:21.622980+0000","last_fullsized":"2024-09-19T09:02:21.622980+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:21.698423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.632916+0000","last_change":"2024-09-19T09:00:46.961262+0000","last_active":"2024-09-19T09:02:21.632916+0000","last_peered":"2024-09-19T09:02:21.632916+0000","last_clean":"2024-09-19T09:02:21.632916+0000","last_became_active":"2024-09-19T09:00:46.960951+0000","last_became_peered":"2024-09-19T09:00:46.960951+0000","last_unstale":"2024-09-19T09:02:21.632916+0000","last_undegraded":"2024-09-19T09:02:21.632916+0000","last_fullsized":"2024-09-19T09:02:21.632916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:02.445108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.625661+0000","last_change":"2024-09-19T09:01:52.758459+0000","last_active":"2024-09-19T09:02:21.625661+0000","last_peered":"2024-09-19T09:02:21.625661+0000","last_clean":"2024-09-19T09:02:21.625661+0000","last_became_active":"2024-09-19T09:01:52.758073+0000","last_became_peered":"2024-09-19T09:01:52.758073+0000","last_unstale":"2024-09-19T09:02:21.625661+0000","last_undegraded":"2024-09-19T09:02:21.625661+0000","last_fullsized":"2024-09-19T09:02:21.625661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:46:32.983316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.634093+0000","last_change":"2024-09-19T09:00:46.963200+0000","last_active":"2024-09-19T09:02:21.634093+0000","last_peered":"2024-09-19T09:02:21.634093+0000","last_clean":"2024-09-19T09:02:21.634093+0000","last_became_active":"2024-09-19T09:00:46.960496+0000","last_became_peered":"2024-09-19T09:00:46.960496+0000","last_unstale":"2024-09-19T09:02:21.634093+0000","last_undegraded":"2024-09-19T09:02:21.634093+0000","last_fullsized":"2024-09-19T09:02:21.634093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:59:47.234492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.618494+0000","last_change":"2024-09-19T09:00:46.966309+0000","last_active":"2024-09-19T09:02:21.618494+0000","last_peered":"2024-09-19T09:02:21.618494+0000","last_clean":"2024-09-19T09:02:21.618494+0000","last_became_active":"2024-09-19T09:00:46.965970+0000","last_became_peered":"2024-09-19T09:00:46.965970+0000","last_unstale":"2024-09-19T09:02:21.618494+0000","last_undegraded":"2024-09-19T09:02:21.618494+0000","last_fullsized":"2024-09-19T09:02:21.618494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:38.353206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622484+0000","last_change":"2024-09-19T09:00:46.954079+0000","last_active":"2024-09-19T09:02:21.622484+0000","last_peered":"2024-09-19T09:02:21.622484+0000","last_clean":"2024-09-19T09:02:21.622484+0000","last_became_active":"2024-09-19T09:00:46.953398+0000","last_became_peered":"2024-09-19T09:00:46.953398+0000","last_unstale":"2024-09-19T09:02:21.622484+0000","last_undegraded":"2024-09-19T09:02:21.622484+0000","last_fullsized":"2024-09-19T09:02:21.622484+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:54:27.554800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621192+0000","last_change":"2024-09-19T09:00:46.965347+0000","last_active":"2024-09-19T09:02:21.621192+0000","last_peered":"2024-09-19T09:02:21.621192+0000","last_clean":"2024-09-19T09:02:21.621192+0000","last_became_active":"2024-09-19T09:00:46.965045+0000","last_became_peered":"2024-09-19T09:00:46.965045+0000","last_unstale":"2024-09-19T09:02:21.621192+0000","last_undegraded":"2024-09-19T09:02:21.621192+0000","last_fullsized":"2024-09-19T09:02:21.621192+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:27.316100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620943+0000","last_change":"2024-09-19T09:00:46.962898+0000","last_active":"2024-09-19T09:02:21.620943+0000","last_peered":"2024-09-19T09:02:21.620943+0000","last_clean":"2024-09-19T09:02:21.620943+0000","last_became_active":"2024-09-19T09:00:46.962412+0000","last_became_peered":"2024-09-19T09:00:46.962412+0000","last_unstale":"2024-09-19T09:02:21.620943+0000","last_undegraded":"2024-09-19T09:02:21.620943+0000","last_fullsized":"2024-09-19T09:02:21.620943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:40.483678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.627060+0000","last_change":"2024-09-19T09:00:46.958147+0000","last_active":"2024-09-19T09:02:21.627060+0000","last_peered":"2024-09-19T09:02:21.627060+0000","last_clean":"2024-09-19T09:02:21.627060+0000","last_became_active":"2024-09-19T09:00:46.957805+0000","last_became_peered":"2024-09-19T09:00:46.957805+0000","last_unstale":"2024-09-19T09:02:21.627060+0000","last_undegraded":"2024-09-19T09:02:21.627060+0000","last_fullsized":"2024-09-19T09:02:21.627060+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:24:17.000064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624642+0000","last_change":"2024-09-19T09:00:46.967399+0000","last_active":"2024-09-19T09:02:21.624642+0000","last_peered":"2024-09-19T09:02:21.624642+0000","last_clean":"2024-09-19T09:02:21.624642+0000","last_became_active":"2024-09-19T09:00:46.966145+0000","last_became_peered":"2024-09-19T09:00:46.966145+0000","last_unstale":"2024-09-19T09:02:21.624642+0000","last_undegraded":"2024-09-19T09:02:21.624642+0000","last_fullsized":"2024-09-19T09:02:21.624642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:29.192469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.614692+0000","last_change":"2024-09-19T09:00:46.960956+0000","last_active":"2024-09-19T09:02:21.614692+0000","last_peered":"2024-09-19T09:02:21.614692+0000","last_clean":"2024-09-19T09:02:21.614692+0000","last_became_active":"2024-09-19T09:00:46.960664+0000","last_became_peered":"2024-09-19T09:00:46.960664+0000","last_unstale":"2024-09-19T09:02:21.614692+0000","last_undegraded":"2024-09-19T09:02:21.614692+0000","last_fullsized":"2024-09-19T09:02:21.614692+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:18.724348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622878+0000","last_change":"2024-09-19T09:00:46.951362+0000","last_active":"2024-09-19T09:02:21.622878+0000","last_peered":"2024-09-19T09:02:21.622878+0000","last_clean":"2024-09-19T09:02:21.622878+0000","last_became_active":"2024-09-19T09:00:46.951004+0000","last_became_peered":"2024-09-19T09:00:46.951004+0000","last_unstale":"2024-09-19T09:02:21.622878+0000","last_undegraded":"2024-09-19T09:02:21.622878+0000","last_fullsized":"2024-09-19T09:02:21.622878+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:43.574425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.623346+0000","last_change":"2024-09-19T09:00:46.966752+0000","last_active":"2024-09-19T09:02:21.623346+0000","last_peered":"2024-09-19T09:02:21.623346+0000","last_clean":"2024-09-19T09:02:21.623346+0000","last_became_active":"2024-09-19T09:00:46.966370+0000","last_became_peered":"2024-09-19T09:00:46.966370+0000","last_unstale":"2024-09-19T09:02:21.623346+0000","last_undegraded":"2024-09-19T09:02:21.623346+0000","last_fullsized":"2024-09-19T09:02:21.623346+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:04.035771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621434+0000","last_change":"2024-09-19T09:00:46.956369+0000","last_active":"2024-09-19T09:02:21.621434+0000","last_peered":"2024-09-19T09:02:21.621434+0000","last_clean":"2024-09-19T09:02:21.621434+0000","last_became_active":"2024-09-19T09:00:46.956075+0000","last_became_peered":"2024-09-19T09:00:46.956075+0000","last_unstale":"2024-09-19T09:02:21.621434+0000","last_undegraded":"2024-09-19T09:02:21.621434+0000","last_fullsized":"2024-09-19T09:02:21.621434+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:23.985017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633789+0000","last_change":"2024-09-19T09:00:46.956661+0000","last_active":"2024-09-19T09:02:21.633789+0000","last_peered":"2024-09-19T09:02:21.633789+0000","last_clean":"2024-09-19T09:02:21.633789+0000","last_became_active":"2024-09-19T09:00:46.956291+0000","last_became_peered":"2024-09-19T09:00:46.956291+0000","last_unstale":"2024-09-19T09:02:21.633789+0000","last_undegraded":"2024-09-19T09:02:21.633789+0000","last_fullsized":"2024-09-19T09:02:21.633789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:55.713039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.616782+0000","last_change":"2024-09-19T09:00:46.961866+0000","last_active":"2024-09-19T09:02:21.616782+0000","last_peered":"2024-09-19T09:02:21.616782+0000","last_clean":"2024-09-19T09:02:21.616782+0000","last_became_active":"2024-09-19T09:00:46.961469+0000","last_became_peered":"2024-09-19T09:00:46.961469+0000","last_unstale":"2024-09-19T09:02:21.616782+0000","last_undegraded":"2024-09-19T09:02:21.616782+0000","last_fullsized":"2024-09-19T09:02:21.616782+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:47.665419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622833+0000","last_change":"2024-09-19T09:00:46.948765+0000","last_active":"2024-09-19T09:02:21.622833+0000","last_peered":"2024-09-19T09:02:21.622833+0000","last_clean":"2024-09-19T09:02:21.622833+0000","last_became_active":"2024-09-19T09:00:46.948431+0000","last_became_peered":"2024-09-19T09:00:46.948431+0000","last_unstale":"2024-09-19T09:02:21.622833+0000","last_undegraded":"2024-09-19T09:02:21.622833+0000","last_fullsized":"2024-09-19T09:02:21.622833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:31.766464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.618165+0000","last_change":"2024-09-19T09:00:46.962433+0000","last_active":"2024-09-19T09:02:21.618165+0000","last_peered":"2024-09-19T09:02:21.618165+0000","last_clean":"2024-09-19T09:02:21.618165+0000","last_became_active":"2024-09-19T09:00:46.962153+0000","last_became_peered":"2024-09-19T09:00:46.962153+0000","last_unstale":"2024-09-19T09:02:21.618165+0000","last_undegraded":"2024-09-19T09:02:21.618165+0000","last_fullsized":"2024-09-19T09:02:21.618165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:37.973611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.623402+0000","last_change":"2024-09-19T09:00:46.953715+0000","last_active":"2024-09-19T09:02:21.623402+0000","last_peered":"2024-09-19T09:02:21.623402+0000","last_clean":"2024-09-19T09:02:21.623402+0000","last_became_active":"2024-09-19T09:00:46.953415+0000","last_became_peered":"2024-09-19T09:00:46.953415+0000","last_unstale":"2024-09-19T09:02:21.623402+0000","last_undegraded":"2024-09-19T09:02:21.623402+0000","last_fullsized":"2024-09-19T09:02:21.623402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:29.768382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.615199+0000","last_change":"2024-09-19T09:00:46.964746+0000","last_active":"2024-09-19T09:02:21.615199+0000","last_peered":"2024-09-19T09:02:21.615199+0000","last_clean":"2024-09-19T09:02:21.615199+0000","last_became_active":"2024-09-19T09:00:46.964404+0000","last_became_peered":"2024-09-19T09:00:46.964404+0000","last_unstale":"2024-09-19T09:02:21.615199+0000","last_undegraded":"2024-09-19T09:02:21.615199+0000","last_fullsized":"2024-09-19T09:02:21.615199+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:19.841495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.619156+0000","last_change":"2024-09-19T09:00:46.962875+0000","last_active":"2024-09-19T09:02:21.619156+0000","last_peered":"2024-09-19T09:02:21.619156+0000","last_clean":"2024-09-19T09:02:21.619156+0000","last_became_active":"2024-09-19T09:00:46.962561+0000","last_became_peered":"2024-09-19T09:00:46.962561+0000","last_unstale":"2024-09-19T09:02:21.619156+0000","last_undegraded":"2024-09-19T09:02:21.619156+0000","last_fullsized":"2024-09-19T09:02:21.619156+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:39.675545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620607+0000","last_change":"2024-09-19T09:00:46.963126+0000","last_active":"2024-09-19T09:02:21.620607+0000","last_peered":"2024-09-19T09:02:21.620607+0000","last_clean":"2024-09-19T09:02:21.620607+0000","last_became_active":"2024-09-19T09:00:46.962620+0000","last_became_peered":"2024-09-19T09:00:46.962620+0000","last_unstale":"2024-09-19T09:02:21.620607+0000","last_undegraded":"2024-09-19T09:02:21.620607+0000","last_fullsized":"2024-09-19T09:02:21.620607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:59.723280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620432+0000","last_change":"2024-09-19T09:00:46.953026+0000","last_active":"2024-09-19T09:02:21.620432+0000","last_peered":"2024-09-19T09:02:21.620432+0000","last_clean":"2024-09-19T09:02:21.620432+0000","last_became_active":"2024-09-19T09:00:46.952596+0000","last_became_peered":"2024-09-19T09:00:46.952596+0000","last_unstale":"2024-09-19T09:02:21.620432+0000","last_undegraded":"2024-09-19T09:02:21.620432+0000","last_fullsized":"2024-09-19T09:02:21.620432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:58.524116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624315+0000","last_change":"2024-09-19T09:00:46.968964+0000","last_active":"2024-09-19T09:02:21.624315+0000","last_peered":"2024-09-19T09:02:21.624315+0000","last_clean":"2024-09-19T09:02:21.624315+0000","last_became_active":"2024-09-19T09:00:46.968505+0000","last_became_peered":"2024-09-19T09:00:46.968505+0000","last_unstale":"2024-09-19T09:02:21.624315+0000","last_undegraded":"2024-09-19T09:02:21.624315+0000","last_fullsized":"2024-09-19T09:02:21.624315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:17:33.335541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.625323+0000","last_change":"2024-09-19T09:00:46.961520+0000","last_active":"2024-09-19T09:02:21.625323+0000","last_peered":"2024-09-19T09:02:21.625323+0000","last_clean":"2024-09-19T09:02:21.625323+0000","last_became_active":"2024-09-19T09:00:46.961140+0000","last_became_peered":"2024-09-19T09:00:46.961140+0000","last_unstale":"2024-09-19T09:02:21.625323+0000","last_undegraded":"2024-09-19T09:02:21.625323+0000","last_fullsized":"2024-09-19T09:02:21.625323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:03.120173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621084+0000","last_change":"2024-09-19T09:00:46.955995+0000","last_active":"2024-09-19T09:02:21.621084+0000","last_peered":"2024-09-19T09:02:21.621084+0000","last_clean":"2024-09-19T09:02:21.621084+0000","last_became_active":"2024-09-19T09:00:46.954678+0000","last_became_peered":"2024-09-19T09:00:46.954678+0000","last_unstale":"2024-09-19T09:02:21.621084+0000","last_undegraded":"2024-09-19T09:02:21.621084+0000","last_fullsized":"2024-09-19T09:02:21.621084+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:10.314575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":145,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:51.298522+0000","last_change":"2024-09-19T09:00:44.946000+0000","last_active":"2024-09-19T09:02:51.298522+0000","last_peered":"2024-09-19T09:02:51.298522+0000","last_clean":"2024-09-19T09:02:51.298522+0000","last_became_active":"2024-09-19T09:00:44.945583+0000","last_became_peered":"2024-09-19T09:00:44.945583+0000","last_unstale":"2024-09-19T09:02:51.298522+0000","last_undegraded":"2024-09-19T09:02:51.298522+0000","last_fullsized":"2024-09-19T09:02:51.298522+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:53:19.109697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624079+0000","last_change":"2024-09-19T09:01:52.904443+0000","last_active":"2024-09-19T09:02:21.624079+0000","last_peered":"2024-09-19T09:02:21.624079+0000","last_clean":"2024-09-19T09:02:21.624079+0000","last_became_active":"2024-09-19T09:01:52.904077+0000","last_became_peered":"2024-09-19T09:01:52.904077+0000","last_unstale":"2024-09-19T09:02:21.624079+0000","last_undegraded":"2024-09-19T09:02:21.624079+0000","last_fullsized":"2024-09-19T09:02:21.624079+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:50.467090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.748381+0000","last_change":"2024-09-19T09:00:44.942435+0000","last_active":"2024-09-19T09:01:52.748381+0000","last_peered":"2024-09-19T09:01:52.748381+0000","last_clean":"2024-09-19T09:01:52.748381+0000","last_became_active":"2024-09-19T09:00:44.942067+0000","last_became_peered":"2024-09-19T09:00:44.942067+0000","last_unstale":"2024-09-19T09:01:52.748381+0000","last_undegraded":"2024-09-19T09:01:52.748381+0000","last_fullsized":"2024-09-19T09:01:52.748381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:16.476506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.747900+0000","last_change":"2024-09-19T09:00:44.937944+0000","last_active":"2024-09-19T09:01:52.747900+0000","last_peered":"2024-09-19T09:01:52.747900+0000","last_clean":"2024-09-19T09:01:52.747900+0000","last_became_active":"2024-09-19T09:00:44.937623+0000","last_became_peered":"2024-09-19T09:00:44.937623+0000","last_unstale":"2024-09-19T09:01:52.747900+0000","last_undegraded":"2024-09-19T09:01:52.747900+0000","last_fullsized":"2024-09-19T09:01:52.747900+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:49:08.730015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620184+0000","last_change":"2024-09-19T09:00:46.959932+0000","last_active":"2024-09-19T09:02:21.620184+0000","last_peered":"2024-09-19T09:02:21.620184+0000","last_clean":"2024-09-19T09:02:21.620184+0000","last_became_active":"2024-09-19T09:00:46.959637+0000","last_became_peered":"2024-09-19T09:00:46.959637+0000","last_unstale":"2024-09-19T09:02:21.620184+0000","last_undegraded":"2024-09-19T09:02:21.620184+0000","last_fullsized":"2024-09-19T09:02:21.620184+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:55.337669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751513+0000","last_change":"2024-09-19T09:00:44.944882+0000","last_active":"2024-09-19T09:01:52.751513+0000","last_peered":"2024-09-19T09:01:52.751513+0000","last_clean":"2024-09-19T09:01:52.751513+0000","last_became_active":"2024-09-19T09:00:44.944116+0000","last_became_peered":"2024-09-19T09:00:44.944116+0000","last_unstale":"2024-09-19T09:01:52.751513+0000","last_undegraded":"2024-09-19T09:01:52.751513+0000","last_fullsized":"2024-09-19T09:01:52.751513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:54:58.555913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.615953+0000","last_change":"2024-09-19T09:00:46.956859+0000","last_active":"2024-09-19T09:02:21.615953+0000","last_peered":"2024-09-19T09:02:21.615953+0000","last_clean":"2024-09-19T09:02:21.615953+0000","last_became_active":"2024-09-19T09:00:46.956561+0000","last_became_peered":"2024-09-19T09:00:46.956561+0000","last_unstale":"2024-09-19T09:02:21.615953+0000","last_undegraded":"2024-09-19T09:02:21.615953+0000","last_fullsized":"2024-09-19T09:02:21.615953+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:09:04.125591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'6","reported_seq":252,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:50.296741+0000","last_change":"2024-09-19T09:00:44.945959+0000","last_active":"2024-09-19T09:02:50.296741+0000","last_peered":"2024-09-19T09:02:50.296741+0000","last_clean":"2024-09-19T09:02:50.296741+0000","last_became_active":"2024-09-19T09:00:44.945640+0000","last_became_peered":"2024-09-19T09:00:44.945640+0000","last_unstale":"2024-09-19T09:02:50.296741+0000","last_undegraded":"2024-09-19T09:02:50.296741+0000","last_fullsized":"2024-09-19T09:02:50.296741+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:02.523834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":224,"num_read_kb":224,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.632897+0000","last_change":"2024-09-19T09:00:46.953677+0000","last_active":"2024-09-19T09:02:21.632897+0000","last_peered":"2024-09-19T09:02:21.632897+0000","last_clean":"2024-09-19T09:02:21.632897+0000","last_became_active":"2024-09-19T09:00:46.953357+0000","last_became_peered":"2024-09-19T09:00:46.953357+0000","last_unstale":"2024-09-19T09:02:21.632897+0000","last_undegraded":"2024-09-19T09:02:21.632897+0000","last_fullsized":"2024-09-19T09:02:21.632897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:48.963003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.893433+0000","last_change":"2024-09-19T09:01:52.893433+0000","last_active":"2024-09-19T09:01:52.893433+0000","last_peered":"2024-09-19T09:01:52.893433+0000","last_clean":"2024-09-19T09:01:52.893433+0000","last_became_active":"2024-09-19T09:01:52.889836+0000","last_became_peered":"2024-09-19T09:01:52.889836+0000","last_unstale":"2024-09-19T09:01:52.893433+0000","last_undegraded":"2024-09-19T09:01:52.893433+0000","last_fullsized":"2024-09-19T09:01:52.893433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:08:22.339717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622288+0000","last_change":"2024-09-19T09:00:46.962285+0000","last_active":"2024-09-19T09:02:21.622288+0000","last_peered":"2024-09-19T09:02:21.622288+0000","last_clean":"2024-09-19T09:02:21.622288+0000","last_became_active":"2024-09-19T09:00:46.961982+0000","last_became_peered":"2024-09-19T09:00:46.961982+0000","last_unstale":"2024-09-19T09:02:21.622288+0000","last_undegraded":"2024-09-19T09:02:21.622288+0000","last_fullsized":"2024-09-19T09:02:21.622288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:41.658501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752480+0000","last_change":"2024-09-19T09:00:45.241425+0000","last_active":"2024-09-19T09:01:52.752480+0000","last_peered":"2024-09-19T09:01:52.752480+0000","last_clean":"2024-09-19T09:01:52.752480+0000","last_became_active":"2024-09-19T09:00:45.241032+0000","last_became_peered":"2024-09-19T09:00:45.241032+0000","last_unstale":"2024-09-19T09:01:52.752480+0000","last_undegraded":"2024-09-19T09:01:52.752480+0000","last_fullsized":"2024-09-19T09:01:52.752480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:00.707855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.626097+0000","last_change":"2024-09-19T09:00:46.962436+0000","last_active":"2024-09-19T09:02:21.626097+0000","last_peered":"2024-09-19T09:02:21.626097+0000","last_clean":"2024-09-19T09:02:21.626097+0000","last_became_active":"2024-09-19T09:00:46.962114+0000","last_became_peered":"2024-09-19T09:00:46.962114+0000","last_unstale":"2024-09-19T09:02:21.626097+0000","last_undegraded":"2024-09-19T09:02:21.626097+0000","last_fullsized":"2024-09-19T09:02:21.626097+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:41.761705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'39","reported_seq":299,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:50.296657+0000","last_change":"2024-09-19T09:00:44.945187+0000","last_active":"2024-09-19T09:02:50.296657+0000","last_peered":"2024-09-19T09:02:50.296657+0000","last_clean":"2024-09-19T09:02:50.296657+0000","last_became_active":"2024-09-19T09:00:44.944763+0000","last_became_peered":"2024-09-19T09:00:44.944763+0000","last_unstale":"2024-09-19T09:02:50.296657+0000","last_undegraded":"2024-09-19T09:02:50.296657+0000","last_fullsized":"2024-09-19T09:02:50.296657+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:06.052994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70094,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":238,"num_read_kb":188,"num_write":41,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.630413+0000","last_change":"2024-09-19T09:00:46.960991+0000","last_active":"2024-09-19T09:02:21.630413+0000","last_peered":"2024-09-19T09:02:21.630413+0000","last_clean":"2024-09-19T09:02:21.630413+0000","last_became_active":"2024-09-19T09:00:46.958962+0000","last_became_peered":"2024-09-19T09:00:46.958962+0000","last_unstale":"2024-09-19T09:02:21.630413+0000","last_undegraded":"2024-09-19T09:02:21.630413+0000","last_fullsized":"2024-09-19T09:02:21.630413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:21.795694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.755250+0000","last_change":"2024-09-19T09:00:44.946176+0000","last_active":"2024-09-19T09:01:52.755250+0000","last_peered":"2024-09-19T09:01:52.755250+0000","last_clean":"2024-09-19T09:01:52.755250+0000","last_became_active":"2024-09-19T09:00:44.945869+0000","last_became_peered":"2024-09-19T09:00:44.945869+0000","last_unstale":"2024-09-19T09:01:52.755250+0000","last_undegraded":"2024-09-19T09:01:52.755250+0000","last_fullsized":"2024-09-19T09:01:52.755250+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:16.022184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621664+0000","last_change":"2024-09-19T09:00:46.956345+0000","last_active":"2024-09-19T09:02:21.621664+0000","last_peered":"2024-09-19T09:02:21.621664+0000","last_clean":"2024-09-19T09:02:21.621664+0000","last_became_active":"2024-09-19T09:00:46.956057+0000","last_became_peered":"2024-09-19T09:00:46.956057+0000","last_unstale":"2024-09-19T09:02:21.621664+0000","last_undegraded":"2024-09-19T09:02:21.621664+0000","last_fullsized":"2024-09-19T09:02:21.621664+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:27.510661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752205+0000","last_change":"2024-09-19T09:00:44.945667+0000","last_active":"2024-09-19T09:01:52.752205+0000","last_peered":"2024-09-19T09:01:52.752205+0000","last_clean":"2024-09-19T09:01:52.752205+0000","last_became_active":"2024-09-19T09:00:44.945308+0000","last_became_peered":"2024-09-19T09:00:44.945308+0000","last_unstale":"2024-09-19T09:01:52.752205+0000","last_undegraded":"2024-09-19T09:01:52.752205+0000","last_fullsized":"2024-09-19T09:01:52.752205+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:00:11.304913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.629868+0000","last_change":"2024-09-19T09:00:46.965088+0000","last_active":"2024-09-19T09:02:21.629868+0000","last_peered":"2024-09-19T09:02:21.629868+0000","last_clean":"2024-09-19T09:02:21.629868+0000","last_became_active":"2024-09-19T09:00:46.964777+0000","last_became_peered":"2024-09-19T09:00:46.964777+0000","last_unstale":"2024-09-19T09:02:21.629868+0000","last_undegraded":"2024-09-19T09:02:21.629868+0000","last_fullsized":"2024-09-19T09:02:21.629868+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:32.958984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'7","reported_seq":147,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:51.297989+0000","last_change":"2024-09-19T09:00:44.946193+0000","last_active":"2024-09-19T09:02:51.297989+0000","last_peered":"2024-09-19T09:02:51.297989+0000","last_clean":"2024-09-19T09:02:51.297989+0000","last_became_active":"2024-09-19T09:00:44.945168+0000","last_became_peered":"2024-09-19T09:00:44.945168+0000","last_unstale":"2024-09-19T09:02:51.297989+0000","last_undegraded":"2024-09-19T09:02:51.297989+0000","last_fullsized":"2024-09-19T09:02:51.297989+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:21.230689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.627048+0000","last_change":"2024-09-19T09:01:52.754058+0000","last_active":"2024-09-19T09:02:21.627048+0000","last_peered":"2024-09-19T09:02:21.627048+0000","last_clean":"2024-09-19T09:02:21.627048+0000","last_became_active":"2024-09-19T09:01:52.753674+0000","last_became_peered":"2024-09-19T09:01:52.753674+0000","last_unstale":"2024-09-19T09:02:21.627048+0000","last_undegraded":"2024-09-19T09:02:21.627048+0000","last_fullsized":"2024-09-19T09:02:21.627048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:05.665121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":64,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.774430+0000","last_change":"2024-09-19T09:01:52.774430+0000","last_active":"2024-09-19T09:01:52.774430+0000","last_peered":"2024-09-19T09:01:52.774430+0000","last_clean":"2024-09-19T09:01:52.774430+0000","last_became_active":"2024-09-19T09:01:52.756083+0000","last_became_peered":"2024-09-19T09:01:52.756083+0000","last_unstale":"2024-09-19T09:01:52.774430+0000","last_undegraded":"2024-09-19T09:01:52.774430+0000","last_fullsized":"2024-09-19T09:01:52.774430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:52.639490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633068+0000","last_change":"2024-09-19T09:01:52.757684+0000","last_active":"2024-09-19T09:02:21.633068+0000","last_peered":"2024-09-19T09:02:21.633068+0000","last_clean":"2024-09-19T09:02:21.633068+0000","last_became_active":"2024-09-19T09:01:52.756969+0000","last_became_peered":"2024-09-19T09:01:52.756969+0000","last_unstale":"2024-09-19T09:02:21.633068+0000","last_undegraded":"2024-09-19T09:02:21.633068+0000","last_fullsized":"2024-09-19T09:02:21.633068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:24:47.117996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752185+0000","last_change":"2024-09-19T09:00:44.946138+0000","last_active":"2024-09-19T09:01:52.752185+0000","last_peered":"2024-09-19T09:01:52.752185+0000","last_clean":"2024-09-19T09:01:52.752185+0000","last_became_active":"2024-09-19T09:00:44.945793+0000","last_became_peered":"2024-09-19T09:00:44.945793+0000","last_unstale":"2024-09-19T09:01:52.752185+0000","last_undegraded":"2024-09-19T09:01:52.752185+0000","last_fullsized":"2024-09-19T09:01:52.752185+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:01.341560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620195+0000","last_change":"2024-09-19T09:00:46.970360+0000","last_active":"2024-09-19T09:02:21.620195+0000","last_peered":"2024-09-19T09:02:21.620195+0000","last_clean":"2024-09-19T09:02:21.620195+0000","last_became_active":"2024-09-19T09:00:46.969971+0000","last_became_peered":"2024-09-19T09:00:46.969971+0000","last_unstale":"2024-09-19T09:02:21.620195+0000","last_undegraded":"2024-09-19T09:02:21.620195+0000","last_fullsized":"2024-09-19T09:02:21.620195+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:48.812158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.748305+0000","last_change":"2024-09-19T09:00:44.941713+0000","last_active":"2024-09-19T09:01:52.748305+0000","last_peered":"2024-09-19T09:01:52.748305+0000","last_clean":"2024-09-19T09:01:52.748305+0000","last_became_active":"2024-09-19T09:00:44.941373+0000","last_became_peered":"2024-09-19T09:00:44.941373+0000","last_unstale":"2024-09-19T09:01:52.748305+0000","last_undegraded":"2024-09-19T09:01:52.748305+0000","last_fullsized":"2024-09-19T09:01:52.748305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:31.157803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.630100+0000","last_change":"2024-09-19T09:00:46.959383+0000","last_active":"2024-09-19T09:02:21.630100+0000","last_peered":"2024-09-19T09:02:21.630100+0000","last_clean":"2024-09-19T09:02:21.630100+0000","last_became_active":"2024-09-19T09:00:46.958860+0000","last_became_peered":"2024-09-19T09:00:46.958860+0000","last_unstale":"2024-09-19T09:02:21.630100+0000","last_undegraded":"2024-09-19T09:02:21.630100+0000","last_fullsized":"2024-09-19T09:02:21.630100+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:32:04.423017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752778+0000","last_change":"2024-09-19T09:00:44.944797+0000","last_active":"2024-09-19T09:01:52.752778+0000","last_peered":"2024-09-19T09:01:52.752778+0000","last_clean":"2024-09-19T09:01:52.752778+0000","last_became_active":"2024-09-19T09:00:44.944259+0000","last_became_peered":"2024-09-19T09:00:44.944259+0000","last_unstale":"2024-09-19T09:01:52.752778+0000","last_undegraded":"2024-09-19T09:01:52.752778+0000","last_fullsized":"2024-09-19T09:01:52.752778+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:58.996217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624674+0000","last_change":"2024-09-19T09:00:46.964998+0000","last_active":"2024-09-19T09:02:21.624674+0000","last_peered":"2024-09-19T09:02:21.624674+0000","last_clean":"2024-09-19T09:02:21.624674+0000","last_became_active":"2024-09-19T09:00:46.964562+0000","last_became_peered":"2024-09-19T09:00:46.964562+0000","last_unstale":"2024-09-19T09:02:21.624674+0000","last_undegraded":"2024-09-19T09:02:21.624674+0000","last_fullsized":"2024-09-19T09:02:21.624674+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:00.089800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":145,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:51.327878+0000","last_change":"2024-09-19T09:00:45.239770+0000","last_active":"2024-09-19T09:02:51.327878+0000","last_peered":"2024-09-19T09:02:51.327878+0000","last_clean":"2024-09-19T09:02:51.327878+0000","last_became_active":"2024-09-19T09:00:45.239385+0000","last_became_peered":"2024-09-19T09:00:45.239385+0000","last_unstale":"2024-09-19T09:02:51.327878+0000","last_undegraded":"2024-09-19T09:02:51.327878+0000","last_fullsized":"2024-09-19T09:02:51.327878+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:21.109900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.634073+0000","last_change":"2024-09-19T09:00:46.967505+0000","last_active":"2024-09-19T09:02:21.634073+0000","last_peered":"2024-09-19T09:02:21.634073+0000","last_clean":"2024-09-19T09:02:21.634073+0000","last_became_active":"2024-09-19T09:00:46.967195+0000","last_became_peered":"2024-09-19T09:00:46.967195+0000","last_unstale":"2024-09-19T09:02:21.634073+0000","last_undegraded":"2024-09-19T09:02:21.634073+0000","last_fullsized":"2024-09-19T09:02:21.634073+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:52:28.917333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":335,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751112+0000","last_change":"2024-09-19T08:59:33.821924+0000","last_active":"2024-09-19T09:01:52.751112+0000","last_peered":"2024-09-19T09:01:52.751112+0000","last_clean":"2024-09-19T09:01:52.751112+0000","last_became_active":"2024-09-19T08:59:33.821404+0000","last_became_peered":"2024-09-19T08:59:33.821404+0000","last_unstale":"2024-09-19T09:01:52.751112+0000","last_undegraded":"2024-09-19T09:01:52.751112+0000","last_fullsized":"2024-09-19T09:01:52.751112+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:56:07.830480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:56:07.830480+0000","last_clean_scrub_stamp":"2024-09-19T08:56:07.830480+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:21.763918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.888536+0000","last_change":"2024-09-19T09:00:44.943714+0000","last_active":"2024-09-19T09:01:52.888536+0000","last_peered":"2024-09-19T09:01:52.888536+0000","last_clean":"2024-09-19T09:01:52.888536+0000","last_became_active":"2024-09-19T09:00:44.943362+0000","last_became_peered":"2024-09-19T09:00:44.943362+0000","last_unstale":"2024-09-19T09:01:52.888536+0000","last_undegraded":"2024-09-19T09:01:52.888536+0000","last_fullsized":"2024-09-19T09:01:52.888536+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:30.015648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633682+0000","last_change":"2024-09-19T09:00:46.956784+0000","last_active":"2024-09-19T09:02:21.633682+0000","last_peered":"2024-09-19T09:02:21.633682+0000","last_clean":"2024-09-19T09:02:21.633682+0000","last_became_active":"2024-09-19T09:00:46.956443+0000","last_became_peered":"2024-09-19T09:00:46.956443+0000","last_unstale":"2024-09-19T09:02:21.633682+0000","last_undegraded":"2024-09-19T09:02:21.633682+0000","last_fullsized":"2024-09-19T09:02:21.633682+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:18.918788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.750994+0000","last_change":"2024-09-19T09:00:44.947796+0000","last_active":"2024-09-19T09:01:52.750994+0000","last_peered":"2024-09-19T09:01:52.750994+0000","last_clean":"2024-09-19T09:01:52.750994+0000","last_became_active":"2024-09-19T09:00:44.946949+0000","last_became_peered":"2024-09-19T09:00:44.946949+0000","last_unstale":"2024-09-19T09:01:52.750994+0000","last_undegraded":"2024-09-19T09:01:52.750994+0000","last_fullsized":"2024-09-19T09:01:52.750994+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:42.210186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.619949+0000","last_change":"2024-09-19T09:00:46.963856+0000","last_active":"2024-09-19T09:02:21.619949+0000","last_peered":"2024-09-19T09:02:21.619949+0000","last_clean":"2024-09-19T09:02:21.619949+0000","last_became_active":"2024-09-19T09:00:46.963543+0000","last_became_peered":"2024-09-19T09:00:46.963543+0000","last_unstale":"2024-09-19T09:02:21.619949+0000","last_undegraded":"2024-09-19T09:02:21.619949+0000","last_fullsized":"2024-09-19T09:02:21.619949+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:31.041028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'6","reported_seq":145,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:53.300234+0000","last_change":"2024-09-19T09:00:45.239269+0000","last_active":"2024-09-19T09:02:53.300234+0000","last_peered":"2024-09-19T09:02:53.300234+0000","last_clean":"2024-09-19T09:02:53.300234+0000","last_became_active":"2024-09-19T09:00:45.238929+0000","last_became_peered":"2024-09-19T09:00:45.238929+0000","last_unstale":"2024-09-19T09:02:53.300234+0000","last_undegraded":"2024-09-19T09:02:53.300234+0000","last_fullsized":"2024-09-19T09:02:53.300234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:46.684148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622578+0000","last_change":"2024-09-19T09:00:46.963595+0000","last_active":"2024-09-19T09:02:21.622578+0000","last_peered":"2024-09-19T09:02:21.622578+0000","last_clean":"2024-09-19T09:02:21.622578+0000","last_became_active":"2024-09-19T09:00:46.963243+0000","last_became_peered":"2024-09-19T09:00:46.963243+0000","last_unstale":"2024-09-19T09:02:21.622578+0000","last_undegraded":"2024-09-19T09:02:21.622578+0000","last_fullsized":"2024-09-19T09:02:21.622578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:01.122996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":139,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:48.873167+0000","last_change":"2024-09-19T09:00:44.941550+0000","last_active":"2024-09-19T09:02:48.873167+0000","last_peered":"2024-09-19T09:02:48.873167+0000","last_clean":"2024-09-19T09:02:48.873167+0000","last_became_active":"2024-09-19T09:00:44.941145+0000","last_became_peered":"2024-09-19T09:00:44.941145+0000","last_unstale":"2024-09-19T09:02:48.873167+0000","last_undegraded":"2024-09-19T09:02:48.873167+0000","last_fullsized":"2024-09-19T09:02:48.873167+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:53.893887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.612369+0000","last_change":"2024-09-19T09:00:46.963473+0000","last_active":"2024-09-19T09:02:21.612369+0000","last_peered":"2024-09-19T09:02:21.612369+0000","last_clean":"2024-09-19T09:02:21.612369+0000","last_became_active":"2024-09-19T09:00:46.963090+0000","last_became_peered":"2024-09-19T09:00:46.963090+0000","last_unstale":"2024-09-19T09:02:21.612369+0000","last_undegraded":"2024-09-19T09:02:21.612369+0000","last_fullsized":"2024-09-19T09:02:21.612369+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:01.492385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.750901+0000","last_change":"2024-09-19T09:00:45.239566+0000","last_active":"2024-09-19T09:01:52.750901+0000","last_peered":"2024-09-19T09:01:52.750901+0000","last_clean":"2024-09-19T09:01:52.750901+0000","last_became_active":"2024-09-19T09:00:45.239250+0000","last_became_peered":"2024-09-19T09:00:45.239250+0000","last_unstale":"2024-09-19T09:01:52.750901+0000","last_undegraded":"2024-09-19T09:01:52.750901+0000","last_fullsized":"2024-09-19T09:01:52.750901+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:44:21.739311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.614525+0000","last_change":"2024-09-19T09:00:46.955344+0000","last_active":"2024-09-19T09:02:21.614525+0000","last_peered":"2024-09-19T09:02:21.614525+0000","last_clean":"2024-09-19T09:02:21.614525+0000","last_became_active":"2024-09-19T09:00:46.954961+0000","last_became_peered":"2024-09-19T09:00:46.954961+0000","last_unstale":"2024-09-19T09:02:21.614525+0000","last_undegraded":"2024-09-19T09:02:21.614525+0000","last_fullsized":"2024-09-19T09:02:21.614525+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:55.743556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752141+0000","last_change":"2024-09-19T09:00:45.241514+0000","last_active":"2024-09-19T09:01:52.752141+0000","last_peered":"2024-09-19T09:01:52.752141+0000","last_clean":"2024-09-19T09:01:52.752141+0000","last_became_active":"2024-09-19T09:00:45.241097+0000","last_became_peered":"2024-09-19T09:00:45.241097+0000","last_unstale":"2024-09-19T09:01:52.752141+0000","last_undegraded":"2024-09-19T09:01:52.752141+0000","last_fullsized":"2024-09-19T09:01:52.752141+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:35:38.323393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620427+0000","last_change":"2024-09-19T09:00:46.955194+0000","last_active":"2024-09-19T09:02:21.620427+0000","last_peered":"2024-09-19T09:02:21.620427+0000","last_clean":"2024-09-19T09:02:21.620427+0000","last_became_active":"2024-09-19T09:00:46.954771+0000","last_became_peered":"2024-09-19T09:00:46.954771+0000","last_unstale":"2024-09-19T09:02:21.620427+0000","last_undegraded":"2024-09-19T09:02:21.620427+0000","last_fullsized":"2024-09-19T09:02:21.620427+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:18.945035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'17","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:40.978139+0000","last_change":"2024-09-19T09:00:44.938554+0000","last_active":"2024-09-19T09:02:40.978139+0000","last_peered":"2024-09-19T09:02:40.978139+0000","last_clean":"2024-09-19T09:02:40.978139+0000","last_became_active":"2024-09-19T09:00:44.938261+0000","last_became_peered":"2024-09-19T09:00:44.938261+0000","last_unstale":"2024-09-19T09:02:40.978139+0000","last_undegraded":"2024-09-19T09:02:40.978139+0000","last_fullsized":"2024-09-19T09:02:40.978139+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:24.293764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.614207+0000","last_change":"2024-09-19T09:01:52.755528+0000","last_active":"2024-09-19T09:02:21.614207+0000","last_peered":"2024-09-19T09:02:21.614207+0000","last_clean":"2024-09-19T09:02:21.614207+0000","last_became_active":"2024-09-19T09:01:52.755188+0000","last_became_peered":"2024-09-19T09:01:52.755188+0000","last_unstale":"2024-09-19T09:02:21.614207+0000","last_undegraded":"2024-09-19T09:02:21.614207+0000","last_fullsized":"2024-09-19T09:02:21.614207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:49:34.988733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.888813+0000","last_change":"2024-09-19T09:00:44.941479+0000","last_active":"2024-09-19T09:01:52.888813+0000","last_peered":"2024-09-19T09:01:52.888813+0000","last_clean":"2024-09-19T09:01:52.888813+0000","last_became_active":"2024-09-19T09:00:44.937197+0000","last_became_peered":"2024-09-19T09:00:44.937197+0000","last_unstale":"2024-09-19T09:01:52.888813+0000","last_undegraded":"2024-09-19T09:01:52.888813+0000","last_fullsized":"2024-09-19T09:01:52.888813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:53:17.385884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621026+0000","last_change":"2024-09-19T09:00:46.970319+0000","last_active":"2024-09-19T09:02:21.621026+0000","last_peered":"2024-09-19T09:02:21.621026+0000","last_clean":"2024-09-19T09:02:21.621026+0000","last_became_active":"2024-09-19T09:00:46.969899+0000","last_became_peered":"2024-09-19T09:00:46.969899+0000","last_unstale":"2024-09-19T09:02:21.621026+0000","last_undegraded":"2024-09-19T09:02:21.621026+0000","last_fullsized":"2024-09-19T09:02:21.621026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:26:21.379231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751656+0000","last_change":"2024-09-19T09:00:45.239525+0000","last_active":"2024-09-19T09:01:52.751656+0000","last_peered":"2024-09-19T09:01:52.751656+0000","last_clean":"2024-09-19T09:01:52.751656+0000","last_became_active":"2024-09-19T09:00:45.239213+0000","last_became_peered":"2024-09-19T09:00:45.239213+0000","last_unstale":"2024-09-19T09:01:52.751656+0000","last_undegraded":"2024-09-19T09:01:52.751656+0000","last_fullsized":"2024-09-19T09:01:52.751656+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:32.525025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.584763+0000","last_change":"2024-09-19T09:00:46.958573+0000","last_active":"2024-09-19T09:02:21.584763+0000","last_peered":"2024-09-19T09:02:21.584763+0000","last_clean":"2024-09-19T09:02:21.584763+0000","last_became_active":"2024-09-19T09:00:46.958274+0000","last_became_peered":"2024-09-19T09:00:46.958274+0000","last_unstale":"2024-09-19T09:02:21.584763+0000","last_undegraded":"2024-09-19T09:02:21.584763+0000","last_fullsized":"2024-09-19T09:02:21.584763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:17.249122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.903274+0000","last_change":"2024-09-19T09:00:44.945194+0000","last_active":"2024-09-19T09:01:52.903274+0000","last_peered":"2024-09-19T09:01:52.903274+0000","last_clean":"2024-09-19T09:01:52.903274+0000","last_became_active":"2024-09-19T09:00:44.944931+0000","last_became_peered":"2024-09-19T09:00:44.944931+0000","last_unstale":"2024-09-19T09:01:52.903274+0000","last_undegraded":"2024-09-19T09:01:52.903274+0000","last_fullsized":"2024-09-19T09:01:52.903274+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:11.181150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621916+0000","last_change":"2024-09-19T09:00:46.966895+0000","last_active":"2024-09-19T09:02:21.621916+0000","last_peered":"2024-09-19T09:02:21.621916+0000","last_clean":"2024-09-19T09:02:21.621916+0000","last_became_active":"2024-09-19T09:00:46.966124+0000","last_became_peered":"2024-09-19T09:00:46.966124+0000","last_unstale":"2024-09-19T09:02:21.621916+0000","last_undegraded":"2024-09-19T09:02:21.621916+0000","last_fullsized":"2024-09-19T09:02:21.621916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:14:04.320863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:41.120672+0000","last_change":"2024-09-19T09:00:44.944167+0000","last_active":"2024-09-19T09:02:41.120672+0000","last_peered":"2024-09-19T09:02:41.120672+0000","last_clean":"2024-09-19T09:02:41.120672+0000","last_became_active":"2024-09-19T09:00:44.943670+0000","last_became_peered":"2024-09-19T09:00:44.943670+0000","last_unstale":"2024-09-19T09:02:41.120672+0000","last_undegraded":"2024-09-19T09:02:41.120672+0000","last_fullsized":"2024-09-19T09:02:41.120672+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:05.596877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.615278+0000","last_change":"2024-09-19T09:00:46.958581+0000","last_active":"2024-09-19T09:02:21.615278+0000","last_peered":"2024-09-19T09:02:21.615278+0000","last_clean":"2024-09-19T09:02:21.615278+0000","last_became_active":"2024-09-19T09:00:46.957782+0000","last_became_peered":"2024-09-19T09:00:46.957782+0000","last_unstale":"2024-09-19T09:02:21.615278+0000","last_undegraded":"2024-09-19T09:02:21.615278+0000","last_fullsized":"2024-09-19T09:02:21.615278+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:37:19.488538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751779+0000","last_change":"2024-09-19T09:00:44.945351+0000","last_active":"2024-09-19T09:01:52.751779+0000","last_peered":"2024-09-19T09:01:52.751779+0000","last_clean":"2024-09-19T09:01:52.751779+0000","last_became_active":"2024-09-19T09:00:44.944991+0000","last_became_peered":"2024-09-19T09:00:44.944991+0000","last_unstale":"2024-09-19T09:01:52.751779+0000","last_undegraded":"2024-09-19T09:01:52.751779+0000","last_fullsized":"2024-09-19T09:01:52.751779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:45:07.581872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633316+0000","last_change":"2024-09-19T09:00:46.956340+0000","last_active":"2024-09-19T09:02:21.633316+0000","last_peered":"2024-09-19T09:02:21.633316+0000","last_clean":"2024-09-19T09:02:21.633316+0000","last_became_active":"2024-09-19T09:00:46.955956+0000","last_became_peered":"2024-09-19T09:00:46.955956+0000","last_unstale":"2024-09-19T09:02:21.633316+0000","last_undegraded":"2024-09-19T09:02:21.633316+0000","last_fullsized":"2024-09-19T09:02:21.633316+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:34.863428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633515+0000","last_change":"2024-09-19T09:00:46.963657+0000","last_active":"2024-09-19T09:02:21.633515+0000","last_peered":"2024-09-19T09:02:21.633515+0000","last_clean":"2024-09-19T09:02:21.633515+0000","last_became_active":"2024-09-19T09:00:46.963134+0000","last_became_peered":"2024-09-19T09:00:46.963134+0000","last_unstale":"2024-09-19T09:02:21.633515+0000","last_undegraded":"2024-09-19T09:02:21.633515+0000","last_fullsized":"2024-09-19T09:02:21.633515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:01.098570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.764939+0000","last_change":"2024-09-19T09:01:52.764438+0000","last_active":"2024-09-19T09:01:52.764939+0000","last_peered":"2024-09-19T09:01:52.764939+0000","last_clean":"2024-09-19T09:01:52.764939+0000","last_became_active":"2024-09-19T09:01:52.755188+0000","last_became_peered":"2024-09-19T09:01:52.755188+0000","last_unstale":"2024-09-19T09:01:52.764939+0000","last_undegraded":"2024-09-19T09:01:52.764939+0000","last_fullsized":"2024-09-19T09:01:52.764939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:40.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.632994+0000","last_change":"2024-09-19T09:00:46.956354+0000","last_active":"2024-09-19T09:02:21.632994+0000","last_peered":"2024-09-19T09:02:21.632994+0000","last_clean":"2024-09-19T09:02:21.632994+0000","last_became_active":"2024-09-19T09:00:46.955964+0000","last_became_peered":"2024-09-19T09:00:46.955964+0000","last_unstale":"2024-09-19T09:02:21.632994+0000","last_undegraded":"2024-09-19T09:02:21.632994+0000","last_fullsized":"2024-09-19T09:02:21.632994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:05:33.243479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751944+0000","last_change":"2024-09-19T09:00:44.944983+0000","last_active":"2024-09-19T09:01:52.751944+0000","last_peered":"2024-09-19T09:01:52.751944+0000","last_clean":"2024-09-19T09:01:52.751944+0000","last_became_active":"2024-09-19T09:00:44.944625+0000","last_became_peered":"2024-09-19T09:00:44.944625+0000","last_unstale":"2024-09-19T09:01:52.751944+0000","last_undegraded":"2024-09-19T09:01:52.751944+0000","last_fullsized":"2024-09-19T09:01:52.751944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:49:50.818863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.619916+0000","last_change":"2024-09-19T09:00:46.966448+0000","last_active":"2024-09-19T09:02:21.619916+0000","last_peered":"2024-09-19T09:02:21.619916+0000","last_clean":"2024-09-19T09:02:21.619916+0000","last_became_active":"2024-09-19T09:00:46.966135+0000","last_became_peered":"2024-09-19T09:00:46.966135+0000","last_unstale":"2024-09-19T09:02:21.619916+0000","last_undegraded":"2024-09-19T09:02:21.619916+0000","last_fullsized":"2024-09-19T09:02:21.619916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:42.652877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.903658+0000","last_change":"2024-09-19T09:00:45.239405+0000","last_active":"2024-09-19T09:01:52.903658+0000","last_peered":"2024-09-19T09:01:52.903658+0000","last_clean":"2024-09-19T09:01:52.903658+0000","last_became_active":"2024-09-19T09:00:45.239002+0000","last_became_peered":"2024-09-19T09:00:45.239002+0000","last_unstale":"2024-09-19T09:01:52.903658+0000","last_undegraded":"2024-09-19T09:01:52.903658+0000","last_fullsized":"2024-09-19T09:01:52.903658+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:46:03.934975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624975+0000","last_change":"2024-09-19T09:00:46.951635+0000","last_active":"2024-09-19T09:02:21.624975+0000","last_peered":"2024-09-19T09:02:21.624975+0000","last_clean":"2024-09-19T09:02:21.624975+0000","last_became_active":"2024-09-19T09:00:46.951325+0000","last_became_peered":"2024-09-19T09:00:46.951325+0000","last_unstale":"2024-09-19T09:02:21.624975+0000","last_undegraded":"2024-09-19T09:02:21.624975+0000","last_fullsized":"2024-09-19T09:02:21.624975+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:17.360950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":170,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:50.875269+0000","last_change":"2024-09-19T09:01:52.766049+0000","last_active":"2024-09-19T09:02:50.875269+0000","last_peered":"2024-09-19T09:02:50.875269+0000","last_clean":"2024-09-19T09:02:50.875269+0000","last_became_active":"2024-09-19T09:01:52.755263+0000","last_became_peered":"2024-09-19T09:01:52.755263+0000","last_unstale":"2024-09-19T09:02:50.875269+0000","last_undegraded":"2024-09-19T09:02:50.875269+0000","last_fullsized":"2024-09-19T09:02:50.875269+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:16.968329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":77751,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1172,"num_read_kb":670,"num_write":266,"num_write_kb":159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1366,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":233253,"data_compressed":4815,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":212166,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612646,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":429768,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841579,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1044480,"data_stored":1089778,"data_compressed":43295,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972401,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":428051,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201333,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":427847,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430267,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":1020726,"data_compressed":41690,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593793,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":427768,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822725,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":1020542,"data_compressed":41690,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986251,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":428163,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215183,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":427593,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":499584,"data_compressed":7388,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":966656,"data_stored":498887,"data_compressed":7388,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":429500,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71958,"data_compressed":1605,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":72655,"data_compressed":1605,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":72364,"data_compressed":1605,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:02:55.318 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T09:02:55.318 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T09:02:56.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:55 smithi120 ceph-mon[31150]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:02:56.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:55 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:02:56.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:55 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:02:56.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:55 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:02:56.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:55 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:02:56.087 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:56.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:55 smithi196 ceph-mon[35150]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:02:56.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:55 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:02:56.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:55 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:02:56.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:55 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:02:56.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:55 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:02:56.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:55 smithi153 ceph-mon[35071]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:02:56.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:55 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:02:56.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:55 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:02:56.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:55 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:02:56.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:55 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:02:56.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.595+0000 7f392ec99640 1 -- 172.21.15.120:0/3712890296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3928106a80 msgr2=0x7f3928106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:56.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.595+0000 7f392ec99640 1 --2- 172.21.15.120:0/3712890296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3928106a80 0x7f3928106e60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f391c009930 tx=0x7f391c02a420 comp rx=0 tx=0).stop 2024-09-19T09:02:56.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.595+0000 7f392ec99640 1 -- 172.21.15.120:0/3712890296 shutdown_connections 2024-09-19T09:02:56.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.595+0000 7f392ec99640 1 --2- 172.21.15.120:0/3712890296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3928101420 0x7f392810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:56.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.595+0000 7f392ec99640 1 --2- 172.21.15.120:0/3712890296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3928100a80 0x7f3928100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:56.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.595+0000 7f392ec99640 1 --2- 172.21.15.120:0/3712890296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3928106a80 0x7f3928106e60 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:56.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.595+0000 7f392ec99640 1 -- 172.21.15.120:0/3712890296 >> 172.21.15.120:0/3712890296 conn(0x7f39280fc7a0 msgr2=0x7f39280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:56.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.595+0000 7f392ec99640 1 -- 172.21.15.120:0/3712890296 shutdown_connections 2024-09-19T09:02:56.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.595+0000 7f392ec99640 1 -- 172.21.15.120:0/3712890296 wait complete. 2024-09-19T09:02:56.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.596+0000 7f392ec99640 1 Processor -- start 2024-09-19T09:02:56.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.596+0000 7f392ec99640 1 -- start start 2024-09-19T09:02:56.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.596+0000 7f392ec99640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3928100a80 0x7f3928198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:56.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.596+0000 7f392ec99640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3928101420 0x7f39281989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:56.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.596+0000 7f392ec99640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3928106a80 0x7f392819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:56.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.596+0000 7f392ec99640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3928111fa0 con 0x7f3928101420 2024-09-19T09:02:56.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.596+0000 7f392ec99640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3928111e20 con 0x7f3928106a80 2024-09-19T09:02:56.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.596+0000 7f392ec99640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3928112120 con 0x7f3928100a80 2024-09-19T09:02:56.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.597+0000 7f392ca35640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3928100a80 0x7f3928198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:56.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.597+0000 7f3927fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3928101420 0x7f39281989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:56.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.597+0000 7f392ca35640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3928100a80 0x7f3928198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:59430/0 (socket says 172.21.15.120:59430) 2024-09-19T09:02:56.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.597+0000 7f3927fff640 1 --2- 172.21.15.120:0/106971443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3928101420 0x7f39281989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51656/0 (socket says 172.21.15.120:51656) 2024-09-19T09:02:56.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.597+0000 7f392ca35640 1 -- 172.21.15.120:0/106971443 learned_addr learned my addr 172.21.15.120:0/106971443 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:56.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.597+0000 7f392d236640 1 --2- 172.21.15.120:0/106971443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3928106a80 0x7f392819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:56.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.597+0000 7f392ca35640 1 -- 172.21.15.120:0/106971443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3928106a80 msgr2=0x7f392819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:56.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.597+0000 7f392ca35640 1 --2- 172.21.15.120:0/106971443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3928106a80 0x7f392819cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:56.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.597+0000 7f392ca35640 1 -- 172.21.15.120:0/106971443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3928101420 msgr2=0x7f39281989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:56.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.597+0000 7f392ca35640 1 --2- 172.21.15.120:0/106971443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3928101420 0x7f39281989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:56.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.597+0000 7f392ca35640 1 -- 172.21.15.120:0/106971443 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f392819d430 con 0x7f3928100a80 2024-09-19T09:02:56.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.597+0000 7f3927fff640 1 --2- 172.21.15.120:0/106971443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3928101420 0x7f39281989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:02:56.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.597+0000 7f392d236640 1 --2- 172.21.15.120:0/106971443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3928106a80 0x7f392819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:02:56.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.597+0000 7f392ca35640 1 --2- 172.21.15.120:0/106971443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3928100a80 0x7f3928198480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f391c009230 tx=0x7f391c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:56.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.597+0000 7f3925ffb640 1 -- 172.21.15.120:0/106971443 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391c03f030 con 0x7f3928100a80 2024-09-19T09:02:56.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.597+0000 7f392ec99640 1 -- 172.21.15.120:0/106971443 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f392819d6c0 con 0x7f3928100a80 2024-09-19T09:02:56.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.597+0000 7f3925ffb640 1 -- 172.21.15.120:0/106971443 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f391c003ad0 con 0x7f3928100a80 2024-09-19T09:02:56.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.597+0000 7f3925ffb640 1 -- 172.21.15.120:0/106971443 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391c035070 con 0x7f3928100a80 2024-09-19T09:02:56.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.598+0000 7f392ec99640 1 -- 172.21.15.120:0/106971443 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39281a5000 con 0x7f3928100a80 2024-09-19T09:02:56.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.599+0000 7f392ec99640 1 -- 172.21.15.120:0/106971443 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39281021c0 con 0x7f3928100a80 2024-09-19T09:02:56.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.600+0000 7f3925ffb640 1 -- 172.21.15.120:0/106971443 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f391c042030 con 0x7f3928100a80 2024-09-19T09:02:56.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.600+0000 7f3925ffb640 1 --2- 172.21.15.120:0/106971443 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f39180a32d0 0x7f39180a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:56.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.600+0000 7f3927fff640 1 --2- 172.21.15.120:0/106971443 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f39180a32d0 0x7f39180a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:56.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.600+0000 7f3925ffb640 1 -- 172.21.15.120:0/106971443 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f391c0f0200 con 0x7f3928100a80 2024-09-19T09:02:56.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.601+0000 7f3927fff640 1 --2- 172.21.15.120:0/106971443 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f39180a32d0 0x7f39180a5790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f39281060a0 tx=0x7f391400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:56.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.604+0000 7f3925ffb640 1 -- 172.21.15.120:0/106971443 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f391c0bcd80 con 0x7f3928100a80 2024-09-19T09:02:56.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.778+0000 7f392ec99640 1 -- 172.21.15.120:0/106971443 --> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3928109060 con 0x7f39180a32d0 2024-09-19T09:02:56.779 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:56 smithi120 ceph-mon[31150]: pgmap v343: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-19T09:02:56.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.785+0000 7f3925ffb640 1 -- 172.21.15.120:0/106971443 <== mgr.14150 v2:172.21.15.120:6800/3841841273 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259138 (secure 0 0 0) 0x7f391401b440 con 0x7f39180a32d0 2024-09-19T09:02:56.787 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:02:56.788 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-19T09:02:56.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.788+0000 7f392ec99640 1 -- 172.21.15.120:0/106971443 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f39180a32d0 msgr2=0x7f39180a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:56.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.788+0000 7f392ec99640 1 --2- 172.21.15.120:0/106971443 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f39180a32d0 0x7f39180a5790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f39281060a0 tx=0x7f391400b390 comp rx=0 tx=0).stop 2024-09-19T09:02:56.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.788+0000 7f392ec99640 1 -- 172.21.15.120:0/106971443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3928100a80 msgr2=0x7f3928198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:56.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.788+0000 7f392ec99640 1 --2- 172.21.15.120:0/106971443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3928100a80 0x7f3928198480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f391c009230 tx=0x7f391c004270 comp rx=0 tx=0).stop 2024-09-19T09:02:56.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.788+0000 7f392ec99640 1 -- 172.21.15.120:0/106971443 shutdown_connections 2024-09-19T09:02:56.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.788+0000 7f392ec99640 1 --2- 172.21.15.120:0/106971443 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f39180a32d0 0x7f39180a5790 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:56.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.788+0000 7f392ec99640 1 --2- 172.21.15.120:0/106971443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3928106a80 0x7f392819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:56.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.788+0000 7f392ec99640 1 --2- 172.21.15.120:0/106971443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3928101420 0x7f39281989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:56.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.788+0000 7f392ec99640 1 --2- 172.21.15.120:0/106971443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3928100a80 0x7f3928198480 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:56.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.788+0000 7f392ec99640 1 -- 172.21.15.120:0/106971443 >> 172.21.15.120:0/106971443 conn(0x7f39280fc7a0 msgr2=0x7f39280fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:56.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.788+0000 7f392ec99640 1 -- 172.21.15.120:0/106971443 shutdown_connections 2024-09-19T09:02:56.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:56.788+0000 7f392ec99640 1 -- 172.21.15.120:0/106971443 wait complete. 2024-09-19T09:02:57.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:56 smithi196 ceph-mon[35150]: pgmap v343: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-19T09:02:57.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:56 smithi153 ceph-mon[35071]: pgmap v343: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-19T09:02:57.469 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-09-19T09:02:55.638760+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668353,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1303,"num_read_kb":779,"num_write":449,"num_write_kb":3099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1182102,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364364,"kb_used_data":10956,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577460,"statfs":{"total":1151940427776,"available":1151567319040,"internally_reserved":0,"allocated":11218944,"data_stored":7128207,"data_compressed":181932,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7227,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":61,"num_write":5,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003275"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624586+0000","last_change":"2024-09-19T09:00:46.966628+0000","last_active":"2024-09-19T09:02:21.624586+0000","last_peered":"2024-09-19T09:02:21.624586+0000","last_clean":"2024-09-19T09:02:21.624586+0000","last_became_active":"2024-09-19T09:00:46.966290+0000","last_became_peered":"2024-09-19T09:00:46.966290+0000","last_unstale":"2024-09-19T09:02:21.624586+0000","last_undegraded":"2024-09-19T09:02:21.624586+0000","last_fullsized":"2024-09-19T09:02:21.624586+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:42.717139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621872+0000","last_change":"2024-09-19T09:00:46.963706+0000","last_active":"2024-09-19T09:02:21.621872+0000","last_peered":"2024-09-19T09:02:21.621872+0000","last_clean":"2024-09-19T09:02:21.621872+0000","last_became_active":"2024-09-19T09:00:46.963303+0000","last_became_peered":"2024-09-19T09:00:46.963303+0000","last_unstale":"2024-09-19T09:02:21.621872+0000","last_undegraded":"2024-09-19T09:02:21.621872+0000","last_fullsized":"2024-09-19T09:02:21.621872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:07.451500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620429+0000","last_change":"2024-09-19T09:00:46.967149+0000","last_active":"2024-09-19T09:02:21.620429+0000","last_peered":"2024-09-19T09:02:21.620429+0000","last_clean":"2024-09-19T09:02:21.620429+0000","last_became_active":"2024-09-19T09:00:46.966861+0000","last_became_peered":"2024-09-19T09:00:46.966861+0000","last_unstale":"2024-09-19T09:02:21.620429+0000","last_undegraded":"2024-09-19T09:02:21.620429+0000","last_fullsized":"2024-09-19T09:02:21.620429+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:28.396022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.605530+0000","last_change":"2024-09-19T09:00:46.969242+0000","last_active":"2024-09-19T09:02:21.605530+0000","last_peered":"2024-09-19T09:02:21.605530+0000","last_clean":"2024-09-19T09:02:21.605530+0000","last_became_active":"2024-09-19T09:00:46.968546+0000","last_became_peered":"2024-09-19T09:00:46.968546+0000","last_unstale":"2024-09-19T09:02:21.605530+0000","last_undegraded":"2024-09-19T09:02:21.605530+0000","last_fullsized":"2024-09-19T09:02:21.605530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:43:51.060130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.602962+0000","last_change":"2024-09-19T09:00:46.955501+0000","last_active":"2024-09-19T09:02:21.602962+0000","last_peered":"2024-09-19T09:02:21.602962+0000","last_clean":"2024-09-19T09:02:21.602962+0000","last_became_active":"2024-09-19T09:00:46.955161+0000","last_became_peered":"2024-09-19T09:00:46.955161+0000","last_unstale":"2024-09-19T09:02:21.602962+0000","last_undegraded":"2024-09-19T09:02:21.602962+0000","last_fullsized":"2024-09-19T09:02:21.602962+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:39:19.005931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624332+0000","last_change":"2024-09-19T09:00:46.962577+0000","last_active":"2024-09-19T09:02:21.624332+0000","last_peered":"2024-09-19T09:02:21.624332+0000","last_clean":"2024-09-19T09:02:21.624332+0000","last_became_active":"2024-09-19T09:00:46.962253+0000","last_became_peered":"2024-09-19T09:00:46.962253+0000","last_unstale":"2024-09-19T09:02:21.624332+0000","last_undegraded":"2024-09-19T09:02:21.624332+0000","last_fullsized":"2024-09-19T09:02:21.624332+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:27.925932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621157+0000","last_change":"2024-09-19T09:00:46.960004+0000","last_active":"2024-09-19T09:02:21.621157+0000","last_peered":"2024-09-19T09:02:21.621157+0000","last_clean":"2024-09-19T09:02:21.621157+0000","last_became_active":"2024-09-19T09:00:46.959664+0000","last_became_peered":"2024-09-19T09:00:46.959664+0000","last_unstale":"2024-09-19T09:02:21.621157+0000","last_undegraded":"2024-09-19T09:02:21.621157+0000","last_fullsized":"2024-09-19T09:02:21.621157+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:41.616309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622980+0000","last_change":"2024-09-19T09:00:46.952394+0000","last_active":"2024-09-19T09:02:21.622980+0000","last_peered":"2024-09-19T09:02:21.622980+0000","last_clean":"2024-09-19T09:02:21.622980+0000","last_became_active":"2024-09-19T09:00:46.952084+0000","last_became_peered":"2024-09-19T09:00:46.952084+0000","last_unstale":"2024-09-19T09:02:21.622980+0000","last_undegraded":"2024-09-19T09:02:21.622980+0000","last_fullsized":"2024-09-19T09:02:21.622980+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:21.698423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.632916+0000","last_change":"2024-09-19T09:00:46.961262+0000","last_active":"2024-09-19T09:02:21.632916+0000","last_peered":"2024-09-19T09:02:21.632916+0000","last_clean":"2024-09-19T09:02:21.632916+0000","last_became_active":"2024-09-19T09:00:46.960951+0000","last_became_peered":"2024-09-19T09:00:46.960951+0000","last_unstale":"2024-09-19T09:02:21.632916+0000","last_undegraded":"2024-09-19T09:02:21.632916+0000","last_fullsized":"2024-09-19T09:02:21.632916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:02.445108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.625661+0000","last_change":"2024-09-19T09:01:52.758459+0000","last_active":"2024-09-19T09:02:21.625661+0000","last_peered":"2024-09-19T09:02:21.625661+0000","last_clean":"2024-09-19T09:02:21.625661+0000","last_became_active":"2024-09-19T09:01:52.758073+0000","last_became_peered":"2024-09-19T09:01:52.758073+0000","last_unstale":"2024-09-19T09:02:21.625661+0000","last_undegraded":"2024-09-19T09:02:21.625661+0000","last_fullsized":"2024-09-19T09:02:21.625661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:46:32.983316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.634093+0000","last_change":"2024-09-19T09:00:46.963200+0000","last_active":"2024-09-19T09:02:21.634093+0000","last_peered":"2024-09-19T09:02:21.634093+0000","last_clean":"2024-09-19T09:02:21.634093+0000","last_became_active":"2024-09-19T09:00:46.960496+0000","last_became_peered":"2024-09-19T09:00:46.960496+0000","last_unstale":"2024-09-19T09:02:21.634093+0000","last_undegraded":"2024-09-19T09:02:21.634093+0000","last_fullsized":"2024-09-19T09:02:21.634093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:59:47.234492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.618494+0000","last_change":"2024-09-19T09:00:46.966309+0000","last_active":"2024-09-19T09:02:21.618494+0000","last_peered":"2024-09-19T09:02:21.618494+0000","last_clean":"2024-09-19T09:02:21.618494+0000","last_became_active":"2024-09-19T09:00:46.965970+0000","last_became_peered":"2024-09-19T09:00:46.965970+0000","last_unstale":"2024-09-19T09:02:21.618494+0000","last_undegraded":"2024-09-19T09:02:21.618494+0000","last_fullsized":"2024-09-19T09:02:21.618494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:38.353206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622484+0000","last_change":"2024-09-19T09:00:46.954079+0000","last_active":"2024-09-19T09:02:21.622484+0000","last_peered":"2024-09-19T09:02:21.622484+0000","last_clean":"2024-09-19T09:02:21.622484+0000","last_became_active":"2024-09-19T09:00:46.953398+0000","last_became_peered":"2024-09-19T09:00:46.953398+0000","last_unstale":"2024-09-19T09:02:21.622484+0000","last_undegraded":"2024-09-19T09:02:21.622484+0000","last_fullsized":"2024-09-19T09:02:21.622484+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:54:27.554800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621192+0000","last_change":"2024-09-19T09:00:46.965347+0000","last_active":"2024-09-19T09:02:21.621192+0000","last_peered":"2024-09-19T09:02:21.621192+0000","last_clean":"2024-09-19T09:02:21.621192+0000","last_became_active":"2024-09-19T09:00:46.965045+0000","last_became_peered":"2024-09-19T09:00:46.965045+0000","last_unstale":"2024-09-19T09:02:21.621192+0000","last_undegraded":"2024-09-19T09:02:21.621192+0000","last_fullsized":"2024-09-19T09:02:21.621192+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:27.316100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620943+0000","last_change":"2024-09-19T09:00:46.962898+0000","last_active":"2024-09-19T09:02:21.620943+0000","last_peered":"2024-09-19T09:02:21.620943+0000","last_clean":"2024-09-19T09:02:21.620943+0000","last_became_active":"2024-09-19T09:00:46.962412+0000","last_became_peered":"2024-09-19T09:00:46.962412+0000","last_unstale":"2024-09-19T09:02:21.620943+0000","last_undegraded":"2024-09-19T09:02:21.620943+0000","last_fullsized":"2024-09-19T09:02:21.620943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:40.483678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.627060+0000","last_change":"2024-09-19T09:00:46.958147+0000","last_active":"2024-09-19T09:02:21.627060+0000","last_peered":"2024-09-19T09:02:21.627060+0000","last_clean":"2024-09-19T09:02:21.627060+0000","last_became_active":"2024-09-19T09:00:46.957805+0000","last_became_peered":"2024-09-19T09:00:46.957805+0000","last_unstale":"2024-09-19T09:02:21.627060+0000","last_undegraded":"2024-09-19T09:02:21.627060+0000","last_fullsized":"2024-09-19T09:02:21.627060+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:24:17.000064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624642+0000","last_change":"2024-09-19T09:00:46.967399+0000","last_active":"2024-09-19T09:02:21.624642+0000","last_peered":"2024-09-19T09:02:21.624642+0000","last_clean":"2024-09-19T09:02:21.624642+0000","last_became_active":"2024-09-19T09:00:46.966145+0000","last_became_peered":"2024-09-19T09:00:46.966145+0000","last_unstale":"2024-09-19T09:02:21.624642+0000","last_undegraded":"2024-09-19T09:02:21.624642+0000","last_fullsized":"2024-09-19T09:02:21.624642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:29.192469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.614692+0000","last_change":"2024-09-19T09:00:46.960956+0000","last_active":"2024-09-19T09:02:21.614692+0000","last_peered":"2024-09-19T09:02:21.614692+0000","last_clean":"2024-09-19T09:02:21.614692+0000","last_became_active":"2024-09-19T09:00:46.960664+0000","last_became_peered":"2024-09-19T09:00:46.960664+0000","last_unstale":"2024-09-19T09:02:21.614692+0000","last_undegraded":"2024-09-19T09:02:21.614692+0000","last_fullsized":"2024-09-19T09:02:21.614692+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:18.724348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622878+0000","last_change":"2024-09-19T09:00:46.951362+0000","last_active":"2024-09-19T09:02:21.622878+0000","last_peered":"2024-09-19T09:02:21.622878+0000","last_clean":"2024-09-19T09:02:21.622878+0000","last_became_active":"2024-09-19T09:00:46.951004+0000","last_became_peered":"2024-09-19T09:00:46.951004+0000","last_unstale":"2024-09-19T09:02:21.622878+0000","last_undegraded":"2024-09-19T09:02:21.622878+0000","last_fullsized":"2024-09-19T09:02:21.622878+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:43.574425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.623346+0000","last_change":"2024-09-19T09:00:46.966752+0000","last_active":"2024-09-19T09:02:21.623346+0000","last_peered":"2024-09-19T09:02:21.623346+0000","last_clean":"2024-09-19T09:02:21.623346+0000","last_became_active":"2024-09-19T09:00:46.966370+0000","last_became_peered":"2024-09-19T09:00:46.966370+0000","last_unstale":"2024-09-19T09:02:21.623346+0000","last_undegraded":"2024-09-19T09:02:21.623346+0000","last_fullsized":"2024-09-19T09:02:21.623346+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:04.035771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621434+0000","last_change":"2024-09-19T09:00:46.956369+0000","last_active":"2024-09-19T09:02:21.621434+0000","last_peered":"2024-09-19T09:02:21.621434+0000","last_clean":"2024-09-19T09:02:21.621434+0000","last_became_active":"2024-09-19T09:00:46.956075+0000","last_became_peered":"2024-09-19T09:00:46.956075+0000","last_unstale":"2024-09-19T09:02:21.621434+0000","last_undegraded":"2024-09-19T09:02:21.621434+0000","last_fullsized":"2024-09-19T09:02:21.621434+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:23.985017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633789+0000","last_change":"2024-09-19T09:00:46.956661+0000","last_active":"2024-09-19T09:02:21.633789+0000","last_peered":"2024-09-19T09:02:21.633789+0000","last_clean":"2024-09-19T09:02:21.633789+0000","last_became_active":"2024-09-19T09:00:46.956291+0000","last_became_peered":"2024-09-19T09:00:46.956291+0000","last_unstale":"2024-09-19T09:02:21.633789+0000","last_undegraded":"2024-09-19T09:02:21.633789+0000","last_fullsized":"2024-09-19T09:02:21.633789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:55.713039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.616782+0000","last_change":"2024-09-19T09:00:46.961866+0000","last_active":"2024-09-19T09:02:21.616782+0000","last_peered":"2024-09-19T09:02:21.616782+0000","last_clean":"2024-09-19T09:02:21.616782+0000","last_became_active":"2024-09-19T09:00:46.961469+0000","last_became_peered":"2024-09-19T09:00:46.961469+0000","last_unstale":"2024-09-19T09:02:21.616782+0000","last_undegraded":"2024-09-19T09:02:21.616782+0000","last_fullsized":"2024-09-19T09:02:21.616782+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:47.665419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622833+0000","last_change":"2024-09-19T09:00:46.948765+0000","last_active":"2024-09-19T09:02:21.622833+0000","last_peered":"2024-09-19T09:02:21.622833+0000","last_clean":"2024-09-19T09:02:21.622833+0000","last_became_active":"2024-09-19T09:00:46.948431+0000","last_became_peered":"2024-09-19T09:00:46.948431+0000","last_unstale":"2024-09-19T09:02:21.622833+0000","last_undegraded":"2024-09-19T09:02:21.622833+0000","last_fullsized":"2024-09-19T09:02:21.622833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:31.766464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.618165+0000","last_change":"2024-09-19T09:00:46.962433+0000","last_active":"2024-09-19T09:02:21.618165+0000","last_peered":"2024-09-19T09:02:21.618165+0000","last_clean":"2024-09-19T09:02:21.618165+0000","last_became_active":"2024-09-19T09:00:46.962153+0000","last_became_peered":"2024-09-19T09:00:46.962153+0000","last_unstale":"2024-09-19T09:02:21.618165+0000","last_undegraded":"2024-09-19T09:02:21.618165+0000","last_fullsized":"2024-09-19T09:02:21.618165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:37.973611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.623402+0000","last_change":"2024-09-19T09:00:46.953715+0000","last_active":"2024-09-19T09:02:21.623402+0000","last_peered":"2024-09-19T09:02:21.623402+0000","last_clean":"2024-09-19T09:02:21.623402+0000","last_became_active":"2024-09-19T09:00:46.953415+0000","last_became_peered":"2024-09-19T09:00:46.953415+0000","last_unstale":"2024-09-19T09:02:21.623402+0000","last_undegraded":"2024-09-19T09:02:21.623402+0000","last_fullsized":"2024-09-19T09:02:21.623402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:29.768382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.615199+0000","last_change":"2024-09-19T09:00:46.964746+0000","last_active":"2024-09-19T09:02:21.615199+0000","last_peered":"2024-09-19T09:02:21.615199+0000","last_clean":"2024-09-19T09:02:21.615199+0000","last_became_active":"2024-09-19T09:00:46.964404+0000","last_became_peered":"2024-09-19T09:00:46.964404+0000","last_unstale":"2024-09-19T09:02:21.615199+0000","last_undegraded":"2024-09-19T09:02:21.615199+0000","last_fullsized":"2024-09-19T09:02:21.615199+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:19.841495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.619156+0000","last_change":"2024-09-19T09:00:46.962875+0000","last_active":"2024-09-19T09:02:21.619156+0000","last_peered":"2024-09-19T09:02:21.619156+0000","last_clean":"2024-09-19T09:02:21.619156+0000","last_became_active":"2024-09-19T09:00:46.962561+0000","last_became_peered":"2024-09-19T09:00:46.962561+0000","last_unstale":"2024-09-19T09:02:21.619156+0000","last_undegraded":"2024-09-19T09:02:21.619156+0000","last_fullsized":"2024-09-19T09:02:21.619156+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:39.675545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620607+0000","last_change":"2024-09-19T09:00:46.963126+0000","last_active":"2024-09-19T09:02:21.620607+0000","last_peered":"2024-09-19T09:02:21.620607+0000","last_clean":"2024-09-19T09:02:21.620607+0000","last_became_active":"2024-09-19T09:00:46.962620+0000","last_became_peered":"2024-09-19T09:00:46.962620+0000","last_unstale":"2024-09-19T09:02:21.620607+0000","last_undegraded":"2024-09-19T09:02:21.620607+0000","last_fullsized":"2024-09-19T09:02:21.620607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:59.723280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620432+0000","last_change":"2024-09-19T09:00:46.953026+0000","last_active":"2024-09-19T09:02:21.620432+0000","last_peered":"2024-09-19T09:02:21.620432+0000","last_clean":"2024-09-19T09:02:21.620432+0000","last_became_active":"2024-09-19T09:00:46.952596+0000","last_became_peered":"2024-09-19T09:00:46.952596+0000","last_unstale":"2024-09-19T09:02:21.620432+0000","last_undegraded":"2024-09-19T09:02:21.620432+0000","last_fullsized":"2024-09-19T09:02:21.620432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:58.524116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624315+0000","last_change":"2024-09-19T09:00:46.968964+0000","last_active":"2024-09-19T09:02:21.624315+0000","last_peered":"2024-09-19T09:02:21.624315+0000","last_clean":"2024-09-19T09:02:21.624315+0000","last_became_active":"2024-09-19T09:00:46.968505+0000","last_became_peered":"2024-09-19T09:00:46.968505+0000","last_unstale":"2024-09-19T09:02:21.624315+0000","last_undegraded":"2024-09-19T09:02:21.624315+0000","last_fullsized":"2024-09-19T09:02:21.624315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:17:33.335541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.625323+0000","last_change":"2024-09-19T09:00:46.961520+0000","last_active":"2024-09-19T09:02:21.625323+0000","last_peered":"2024-09-19T09:02:21.625323+0000","last_clean":"2024-09-19T09:02:21.625323+0000","last_became_active":"2024-09-19T09:00:46.961140+0000","last_became_peered":"2024-09-19T09:00:46.961140+0000","last_unstale":"2024-09-19T09:02:21.625323+0000","last_undegraded":"2024-09-19T09:02:21.625323+0000","last_fullsized":"2024-09-19T09:02:21.625323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:03.120173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621084+0000","last_change":"2024-09-19T09:00:46.955995+0000","last_active":"2024-09-19T09:02:21.621084+0000","last_peered":"2024-09-19T09:02:21.621084+0000","last_clean":"2024-09-19T09:02:21.621084+0000","last_became_active":"2024-09-19T09:00:46.954678+0000","last_became_peered":"2024-09-19T09:00:46.954678+0000","last_unstale":"2024-09-19T09:02:21.621084+0000","last_undegraded":"2024-09-19T09:02:21.621084+0000","last_fullsized":"2024-09-19T09:02:21.621084+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:10.314575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":145,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:51.298522+0000","last_change":"2024-09-19T09:00:44.946000+0000","last_active":"2024-09-19T09:02:51.298522+0000","last_peered":"2024-09-19T09:02:51.298522+0000","last_clean":"2024-09-19T09:02:51.298522+0000","last_became_active":"2024-09-19T09:00:44.945583+0000","last_became_peered":"2024-09-19T09:00:44.945583+0000","last_unstale":"2024-09-19T09:02:51.298522+0000","last_undegraded":"2024-09-19T09:02:51.298522+0000","last_fullsized":"2024-09-19T09:02:51.298522+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:53:19.109697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624079+0000","last_change":"2024-09-19T09:01:52.904443+0000","last_active":"2024-09-19T09:02:21.624079+0000","last_peered":"2024-09-19T09:02:21.624079+0000","last_clean":"2024-09-19T09:02:21.624079+0000","last_became_active":"2024-09-19T09:01:52.904077+0000","last_became_peered":"2024-09-19T09:01:52.904077+0000","last_unstale":"2024-09-19T09:02:21.624079+0000","last_undegraded":"2024-09-19T09:02:21.624079+0000","last_fullsized":"2024-09-19T09:02:21.624079+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:50.467090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.748381+0000","last_change":"2024-09-19T09:00:44.942435+0000","last_active":"2024-09-19T09:01:52.748381+0000","last_peered":"2024-09-19T09:01:52.748381+0000","last_clean":"2024-09-19T09:01:52.748381+0000","last_became_active":"2024-09-19T09:00:44.942067+0000","last_became_peered":"2024-09-19T09:00:44.942067+0000","last_unstale":"2024-09-19T09:01:52.748381+0000","last_undegraded":"2024-09-19T09:01:52.748381+0000","last_fullsized":"2024-09-19T09:01:52.748381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:16.476506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.747900+0000","last_change":"2024-09-19T09:00:44.937944+0000","last_active":"2024-09-19T09:01:52.747900+0000","last_peered":"2024-09-19T09:01:52.747900+0000","last_clean":"2024-09-19T09:01:52.747900+0000","last_became_active":"2024-09-19T09:00:44.937623+0000","last_became_peered":"2024-09-19T09:00:44.937623+0000","last_unstale":"2024-09-19T09:01:52.747900+0000","last_undegraded":"2024-09-19T09:01:52.747900+0000","last_fullsized":"2024-09-19T09:01:52.747900+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:49:08.730015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620184+0000","last_change":"2024-09-19T09:00:46.959932+0000","last_active":"2024-09-19T09:02:21.620184+0000","last_peered":"2024-09-19T09:02:21.620184+0000","last_clean":"2024-09-19T09:02:21.620184+0000","last_became_active":"2024-09-19T09:00:46.959637+0000","last_became_peered":"2024-09-19T09:00:46.959637+0000","last_unstale":"2024-09-19T09:02:21.620184+0000","last_undegraded":"2024-09-19T09:02:21.620184+0000","last_fullsized":"2024-09-19T09:02:21.620184+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:55.337669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751513+0000","last_change":"2024-09-19T09:00:44.944882+0000","last_active":"2024-09-19T09:01:52.751513+0000","last_peered":"2024-09-19T09:01:52.751513+0000","last_clean":"2024-09-19T09:01:52.751513+0000","last_became_active":"2024-09-19T09:00:44.944116+0000","last_became_peered":"2024-09-19T09:00:44.944116+0000","last_unstale":"2024-09-19T09:01:52.751513+0000","last_undegraded":"2024-09-19T09:01:52.751513+0000","last_fullsized":"2024-09-19T09:01:52.751513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:54:58.555913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.615953+0000","last_change":"2024-09-19T09:00:46.956859+0000","last_active":"2024-09-19T09:02:21.615953+0000","last_peered":"2024-09-19T09:02:21.615953+0000","last_clean":"2024-09-19T09:02:21.615953+0000","last_became_active":"2024-09-19T09:00:46.956561+0000","last_became_peered":"2024-09-19T09:00:46.956561+0000","last_unstale":"2024-09-19T09:02:21.615953+0000","last_undegraded":"2024-09-19T09:02:21.615953+0000","last_fullsized":"2024-09-19T09:02:21.615953+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:09:04.125591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'6","reported_seq":252,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:50.296741+0000","last_change":"2024-09-19T09:00:44.945959+0000","last_active":"2024-09-19T09:02:50.296741+0000","last_peered":"2024-09-19T09:02:50.296741+0000","last_clean":"2024-09-19T09:02:50.296741+0000","last_became_active":"2024-09-19T09:00:44.945640+0000","last_became_peered":"2024-09-19T09:00:44.945640+0000","last_unstale":"2024-09-19T09:02:50.296741+0000","last_undegraded":"2024-09-19T09:02:50.296741+0000","last_fullsized":"2024-09-19T09:02:50.296741+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:02.523834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":224,"num_read_kb":224,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.632897+0000","last_change":"2024-09-19T09:00:46.953677+0000","last_active":"2024-09-19T09:02:21.632897+0000","last_peered":"2024-09-19T09:02:21.632897+0000","last_clean":"2024-09-19T09:02:21.632897+0000","last_became_active":"2024-09-19T09:00:46.953357+0000","last_became_peered":"2024-09-19T09:00:46.953357+0000","last_unstale":"2024-09-19T09:02:21.632897+0000","last_undegraded":"2024-09-19T09:02:21.632897+0000","last_fullsized":"2024-09-19T09:02:21.632897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:48.963003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.893433+0000","last_change":"2024-09-19T09:01:52.893433+0000","last_active":"2024-09-19T09:01:52.893433+0000","last_peered":"2024-09-19T09:01:52.893433+0000","last_clean":"2024-09-19T09:01:52.893433+0000","last_became_active":"2024-09-19T09:01:52.889836+0000","last_became_peered":"2024-09-19T09:01:52.889836+0000","last_unstale":"2024-09-19T09:01:52.893433+0000","last_undegraded":"2024-09-19T09:01:52.893433+0000","last_fullsized":"2024-09-19T09:01:52.893433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:08:22.339717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622288+0000","last_change":"2024-09-19T09:00:46.962285+0000","last_active":"2024-09-19T09:02:21.622288+0000","last_peered":"2024-09-19T09:02:21.622288+0000","last_clean":"2024-09-19T09:02:21.622288+0000","last_became_active":"2024-09-19T09:00:46.961982+0000","last_became_peered":"2024-09-19T09:00:46.961982+0000","last_unstale":"2024-09-19T09:02:21.622288+0000","last_undegraded":"2024-09-19T09:02:21.622288+0000","last_fullsized":"2024-09-19T09:02:21.622288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:41.658501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752480+0000","last_change":"2024-09-19T09:00:45.241425+0000","last_active":"2024-09-19T09:01:52.752480+0000","last_peered":"2024-09-19T09:01:52.752480+0000","last_clean":"2024-09-19T09:01:52.752480+0000","last_became_active":"2024-09-19T09:00:45.241032+0000","last_became_peered":"2024-09-19T09:00:45.241032+0000","last_unstale":"2024-09-19T09:01:52.752480+0000","last_undegraded":"2024-09-19T09:01:52.752480+0000","last_fullsized":"2024-09-19T09:01:52.752480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:00.707855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.626097+0000","last_change":"2024-09-19T09:00:46.962436+0000","last_active":"2024-09-19T09:02:21.626097+0000","last_peered":"2024-09-19T09:02:21.626097+0000","last_clean":"2024-09-19T09:02:21.626097+0000","last_became_active":"2024-09-19T09:00:46.962114+0000","last_became_peered":"2024-09-19T09:00:46.962114+0000","last_unstale":"2024-09-19T09:02:21.626097+0000","last_undegraded":"2024-09-19T09:02:21.626097+0000","last_fullsized":"2024-09-19T09:02:21.626097+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:41.761705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'39","reported_seq":299,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:50.296657+0000","last_change":"2024-09-19T09:00:44.945187+0000","last_active":"2024-09-19T09:02:50.296657+0000","last_peered":"2024-09-19T09:02:50.296657+0000","last_clean":"2024-09-19T09:02:50.296657+0000","last_became_active":"2024-09-19T09:00:44.944763+0000","last_became_peered":"2024-09-19T09:00:44.944763+0000","last_unstale":"2024-09-19T09:02:50.296657+0000","last_undegraded":"2024-09-19T09:02:50.296657+0000","last_fullsized":"2024-09-19T09:02:50.296657+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:06.052994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70094,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":238,"num_read_kb":188,"num_write":41,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.630413+0000","last_change":"2024-09-19T09:00:46.960991+0000","last_active":"2024-09-19T09:02:21.630413+0000","last_peered":"2024-09-19T09:02:21.630413+0000","last_clean":"2024-09-19T09:02:21.630413+0000","last_became_active":"2024-09-19T09:00:46.958962+0000","last_became_peered":"2024-09-19T09:00:46.958962+0000","last_unstale":"2024-09-19T09:02:21.630413+0000","last_undegraded":"2024-09-19T09:02:21.630413+0000","last_fullsized":"2024-09-19T09:02:21.630413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:21.795694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.755250+0000","last_change":"2024-09-19T09:00:44.946176+0000","last_active":"2024-09-19T09:01:52.755250+0000","last_peered":"2024-09-19T09:01:52.755250+0000","last_clean":"2024-09-19T09:01:52.755250+0000","last_became_active":"2024-09-19T09:00:44.945869+0000","last_became_peered":"2024-09-19T09:00:44.945869+0000","last_unstale":"2024-09-19T09:01:52.755250+0000","last_undegraded":"2024-09-19T09:01:52.755250+0000","last_fullsized":"2024-09-19T09:01:52.755250+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:16.022184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621664+0000","last_change":"2024-09-19T09:00:46.956345+0000","last_active":"2024-09-19T09:02:21.621664+0000","last_peered":"2024-09-19T09:02:21.621664+0000","last_clean":"2024-09-19T09:02:21.621664+0000","last_became_active":"2024-09-19T09:00:46.956057+0000","last_became_peered":"2024-09-19T09:00:46.956057+0000","last_unstale":"2024-09-19T09:02:21.621664+0000","last_undegraded":"2024-09-19T09:02:21.621664+0000","last_fullsized":"2024-09-19T09:02:21.621664+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:27.510661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752205+0000","last_change":"2024-09-19T09:00:44.945667+0000","last_active":"2024-09-19T09:01:52.752205+0000","last_peered":"2024-09-19T09:01:52.752205+0000","last_clean":"2024-09-19T09:01:52.752205+0000","last_became_active":"2024-09-19T09:00:44.945308+0000","last_became_peered":"2024-09-19T09:00:44.945308+0000","last_unstale":"2024-09-19T09:01:52.752205+0000","last_undegraded":"2024-09-19T09:01:52.752205+0000","last_fullsized":"2024-09-19T09:01:52.752205+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:00:11.304913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.629868+0000","last_change":"2024-09-19T09:00:46.965088+0000","last_active":"2024-09-19T09:02:21.629868+0000","last_peered":"2024-09-19T09:02:21.629868+0000","last_clean":"2024-09-19T09:02:21.629868+0000","last_became_active":"2024-09-19T09:00:46.964777+0000","last_became_peered":"2024-09-19T09:00:46.964777+0000","last_unstale":"2024-09-19T09:02:21.629868+0000","last_undegraded":"2024-09-19T09:02:21.629868+0000","last_fullsized":"2024-09-19T09:02:21.629868+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:32.958984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'7","reported_seq":147,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:51.297989+0000","last_change":"2024-09-19T09:00:44.946193+0000","last_active":"2024-09-19T09:02:51.297989+0000","last_peered":"2024-09-19T09:02:51.297989+0000","last_clean":"2024-09-19T09:02:51.297989+0000","last_became_active":"2024-09-19T09:00:44.945168+0000","last_became_peered":"2024-09-19T09:00:44.945168+0000","last_unstale":"2024-09-19T09:02:51.297989+0000","last_undegraded":"2024-09-19T09:02:51.297989+0000","last_fullsized":"2024-09-19T09:02:51.297989+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:21.230689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.627048+0000","last_change":"2024-09-19T09:01:52.754058+0000","last_active":"2024-09-19T09:02:21.627048+0000","last_peered":"2024-09-19T09:02:21.627048+0000","last_clean":"2024-09-19T09:02:21.627048+0000","last_became_active":"2024-09-19T09:01:52.753674+0000","last_became_peered":"2024-09-19T09:01:52.753674+0000","last_unstale":"2024-09-19T09:02:21.627048+0000","last_undegraded":"2024-09-19T09:02:21.627048+0000","last_fullsized":"2024-09-19T09:02:21.627048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:05.665121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":64,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.774430+0000","last_change":"2024-09-19T09:01:52.774430+0000","last_active":"2024-09-19T09:01:52.774430+0000","last_peered":"2024-09-19T09:01:52.774430+0000","last_clean":"2024-09-19T09:01:52.774430+0000","last_became_active":"2024-09-19T09:01:52.756083+0000","last_became_peered":"2024-09-19T09:01:52.756083+0000","last_unstale":"2024-09-19T09:01:52.774430+0000","last_undegraded":"2024-09-19T09:01:52.774430+0000","last_fullsized":"2024-09-19T09:01:52.774430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:52.639490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633068+0000","last_change":"2024-09-19T09:01:52.757684+0000","last_active":"2024-09-19T09:02:21.633068+0000","last_peered":"2024-09-19T09:02:21.633068+0000","last_clean":"2024-09-19T09:02:21.633068+0000","last_became_active":"2024-09-19T09:01:52.756969+0000","last_became_peered":"2024-09-19T09:01:52.756969+0000","last_unstale":"2024-09-19T09:02:21.633068+0000","last_undegraded":"2024-09-19T09:02:21.633068+0000","last_fullsized":"2024-09-19T09:02:21.633068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:24:47.117996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752185+0000","last_change":"2024-09-19T09:00:44.946138+0000","last_active":"2024-09-19T09:01:52.752185+0000","last_peered":"2024-09-19T09:01:52.752185+0000","last_clean":"2024-09-19T09:01:52.752185+0000","last_became_active":"2024-09-19T09:00:44.945793+0000","last_became_peered":"2024-09-19T09:00:44.945793+0000","last_unstale":"2024-09-19T09:01:52.752185+0000","last_undegraded":"2024-09-19T09:01:52.752185+0000","last_fullsized":"2024-09-19T09:01:52.752185+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:01.341560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620195+0000","last_change":"2024-09-19T09:00:46.970360+0000","last_active":"2024-09-19T09:02:21.620195+0000","last_peered":"2024-09-19T09:02:21.620195+0000","last_clean":"2024-09-19T09:02:21.620195+0000","last_became_active":"2024-09-19T09:00:46.969971+0000","last_became_peered":"2024-09-19T09:00:46.969971+0000","last_unstale":"2024-09-19T09:02:21.620195+0000","last_undegraded":"2024-09-19T09:02:21.620195+0000","last_fullsized":"2024-09-19T09:02:21.620195+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:48.812158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.748305+0000","last_change":"2024-09-19T09:00:44.941713+0000","last_active":"2024-09-19T09:01:52.748305+0000","last_peered":"2024-09-19T09:01:52.748305+0000","last_clean":"2024-09-19T09:01:52.748305+0000","last_became_active":"2024-09-19T09:00:44.941373+0000","last_became_peered":"2024-09-19T09:00:44.941373+0000","last_unstale":"2024-09-19T09:01:52.748305+0000","last_undegraded":"2024-09-19T09:01:52.748305+0000","last_fullsized":"2024-09-19T09:01:52.748305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:31.157803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.630100+0000","last_change":"2024-09-19T09:00:46.959383+0000","last_active":"2024-09-19T09:02:21.630100+0000","last_peered":"2024-09-19T09:02:21.630100+0000","last_clean":"2024-09-19T09:02:21.630100+0000","last_became_active":"2024-09-19T09:00:46.958860+0000","last_became_peered":"2024-09-19T09:00:46.958860+0000","last_unstale":"2024-09-19T09:02:21.630100+0000","last_undegraded":"2024-09-19T09:02:21.630100+0000","last_fullsized":"2024-09-19T09:02:21.630100+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:32:04.423017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752778+0000","last_change":"2024-09-19T09:00:44.944797+0000","last_active":"2024-09-19T09:01:52.752778+0000","last_peered":"2024-09-19T09:01:52.752778+0000","last_clean":"2024-09-19T09:01:52.752778+0000","last_became_active":"2024-09-19T09:00:44.944259+0000","last_became_peered":"2024-09-19T09:00:44.944259+0000","last_unstale":"2024-09-19T09:01:52.752778+0000","last_undegraded":"2024-09-19T09:01:52.752778+0000","last_fullsized":"2024-09-19T09:01:52.752778+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:58.996217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624674+0000","last_change":"2024-09-19T09:00:46.964998+0000","last_active":"2024-09-19T09:02:21.624674+0000","last_peered":"2024-09-19T09:02:21.624674+0000","last_clean":"2024-09-19T09:02:21.624674+0000","last_became_active":"2024-09-19T09:00:46.964562+0000","last_became_peered":"2024-09-19T09:00:46.964562+0000","last_unstale":"2024-09-19T09:02:21.624674+0000","last_undegraded":"2024-09-19T09:02:21.624674+0000","last_fullsized":"2024-09-19T09:02:21.624674+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:00.089800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":145,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:51.327878+0000","last_change":"2024-09-19T09:00:45.239770+0000","last_active":"2024-09-19T09:02:51.327878+0000","last_peered":"2024-09-19T09:02:51.327878+0000","last_clean":"2024-09-19T09:02:51.327878+0000","last_became_active":"2024-09-19T09:00:45.239385+0000","last_became_peered":"2024-09-19T09:00:45.239385+0000","last_unstale":"2024-09-19T09:02:51.327878+0000","last_undegraded":"2024-09-19T09:02:51.327878+0000","last_fullsized":"2024-09-19T09:02:51.327878+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:21.109900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.634073+0000","last_change":"2024-09-19T09:00:46.967505+0000","last_active":"2024-09-19T09:02:21.634073+0000","last_peered":"2024-09-19T09:02:21.634073+0000","last_clean":"2024-09-19T09:02:21.634073+0000","last_became_active":"2024-09-19T09:00:46.967195+0000","last_became_peered":"2024-09-19T09:00:46.967195+0000","last_unstale":"2024-09-19T09:02:21.634073+0000","last_undegraded":"2024-09-19T09:02:21.634073+0000","last_fullsized":"2024-09-19T09:02:21.634073+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:52:28.917333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":335,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751112+0000","last_change":"2024-09-19T08:59:33.821924+0000","last_active":"2024-09-19T09:01:52.751112+0000","last_peered":"2024-09-19T09:01:52.751112+0000","last_clean":"2024-09-19T09:01:52.751112+0000","last_became_active":"2024-09-19T08:59:33.821404+0000","last_became_peered":"2024-09-19T08:59:33.821404+0000","last_unstale":"2024-09-19T09:01:52.751112+0000","last_undegraded":"2024-09-19T09:01:52.751112+0000","last_fullsized":"2024-09-19T09:01:52.751112+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:56:07.830480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:56:07.830480+0000","last_clean_scrub_stamp":"2024-09-19T08:56:07.830480+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:21.763918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.888536+0000","last_change":"2024-09-19T09:00:44.943714+0000","last_active":"2024-09-19T09:01:52.888536+0000","last_peered":"2024-09-19T09:01:52.888536+0000","last_clean":"2024-09-19T09:01:52.888536+0000","last_became_active":"2024-09-19T09:00:44.943362+0000","last_became_peered":"2024-09-19T09:00:44.943362+0000","last_unstale":"2024-09-19T09:01:52.888536+0000","last_undegraded":"2024-09-19T09:01:52.888536+0000","last_fullsized":"2024-09-19T09:01:52.888536+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:30.015648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633682+0000","last_change":"2024-09-19T09:00:46.956784+0000","last_active":"2024-09-19T09:02:21.633682+0000","last_peered":"2024-09-19T09:02:21.633682+0000","last_clean":"2024-09-19T09:02:21.633682+0000","last_became_active":"2024-09-19T09:00:46.956443+0000","last_became_peered":"2024-09-19T09:00:46.956443+0000","last_unstale":"2024-09-19T09:02:21.633682+0000","last_undegraded":"2024-09-19T09:02:21.633682+0000","last_fullsized":"2024-09-19T09:02:21.633682+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:18.918788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.750994+0000","last_change":"2024-09-19T09:00:44.947796+0000","last_active":"2024-09-19T09:01:52.750994+0000","last_peered":"2024-09-19T09:01:52.750994+0000","last_clean":"2024-09-19T09:01:52.750994+0000","last_became_active":"2024-09-19T09:00:44.946949+0000","last_became_peered":"2024-09-19T09:00:44.946949+0000","last_unstale":"2024-09-19T09:01:52.750994+0000","last_undegraded":"2024-09-19T09:01:52.750994+0000","last_fullsized":"2024-09-19T09:01:52.750994+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:42.210186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.619949+0000","last_change":"2024-09-19T09:00:46.963856+0000","last_active":"2024-09-19T09:02:21.619949+0000","last_peered":"2024-09-19T09:02:21.619949+0000","last_clean":"2024-09-19T09:02:21.619949+0000","last_became_active":"2024-09-19T09:00:46.963543+0000","last_became_peered":"2024-09-19T09:00:46.963543+0000","last_unstale":"2024-09-19T09:02:21.619949+0000","last_undegraded":"2024-09-19T09:02:21.619949+0000","last_fullsized":"2024-09-19T09:02:21.619949+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:31.041028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'6","reported_seq":145,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:53.300234+0000","last_change":"2024-09-19T09:00:45.239269+0000","last_active":"2024-09-19T09:02:53.300234+0000","last_peered":"2024-09-19T09:02:53.300234+0000","last_clean":"2024-09-19T09:02:53.300234+0000","last_became_active":"2024-09-19T09:00:45.238929+0000","last_became_peered":"2024-09-19T09:00:45.238929+0000","last_unstale":"2024-09-19T09:02:53.300234+0000","last_undegraded":"2024-09-19T09:02:53.300234+0000","last_fullsized":"2024-09-19T09:02:53.300234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:46.684148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.622578+0000","last_change":"2024-09-19T09:00:46.963595+0000","last_active":"2024-09-19T09:02:21.622578+0000","last_peered":"2024-09-19T09:02:21.622578+0000","last_clean":"2024-09-19T09:02:21.622578+0000","last_became_active":"2024-09-19T09:00:46.963243+0000","last_became_peered":"2024-09-19T09:00:46.963243+0000","last_unstale":"2024-09-19T09:02:21.622578+0000","last_undegraded":"2024-09-19T09:02:21.622578+0000","last_fullsized":"2024-09-19T09:02:21.622578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:01.122996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":144,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:52.878760+0000","last_change":"2024-09-19T09:00:44.941550+0000","last_active":"2024-09-19T09:02:52.878760+0000","last_peered":"2024-09-19T09:02:52.878760+0000","last_clean":"2024-09-19T09:02:52.878760+0000","last_became_active":"2024-09-19T09:00:44.941145+0000","last_became_peered":"2024-09-19T09:00:44.941145+0000","last_unstale":"2024-09-19T09:02:52.878760+0000","last_undegraded":"2024-09-19T09:02:52.878760+0000","last_fullsized":"2024-09-19T09:02:52.878760+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:53.893887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.612369+0000","last_change":"2024-09-19T09:00:46.963473+0000","last_active":"2024-09-19T09:02:21.612369+0000","last_peered":"2024-09-19T09:02:21.612369+0000","last_clean":"2024-09-19T09:02:21.612369+0000","last_became_active":"2024-09-19T09:00:46.963090+0000","last_became_peered":"2024-09-19T09:00:46.963090+0000","last_unstale":"2024-09-19T09:02:21.612369+0000","last_undegraded":"2024-09-19T09:02:21.612369+0000","last_fullsized":"2024-09-19T09:02:21.612369+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:01.492385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.750901+0000","last_change":"2024-09-19T09:00:45.239566+0000","last_active":"2024-09-19T09:01:52.750901+0000","last_peered":"2024-09-19T09:01:52.750901+0000","last_clean":"2024-09-19T09:01:52.750901+0000","last_became_active":"2024-09-19T09:00:45.239250+0000","last_became_peered":"2024-09-19T09:00:45.239250+0000","last_unstale":"2024-09-19T09:01:52.750901+0000","last_undegraded":"2024-09-19T09:01:52.750901+0000","last_fullsized":"2024-09-19T09:01:52.750901+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:44:21.739311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.614525+0000","last_change":"2024-09-19T09:00:46.955344+0000","last_active":"2024-09-19T09:02:21.614525+0000","last_peered":"2024-09-19T09:02:21.614525+0000","last_clean":"2024-09-19T09:02:21.614525+0000","last_became_active":"2024-09-19T09:00:46.954961+0000","last_became_peered":"2024-09-19T09:00:46.954961+0000","last_unstale":"2024-09-19T09:02:21.614525+0000","last_undegraded":"2024-09-19T09:02:21.614525+0000","last_fullsized":"2024-09-19T09:02:21.614525+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:55.743556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.752141+0000","last_change":"2024-09-19T09:00:45.241514+0000","last_active":"2024-09-19T09:01:52.752141+0000","last_peered":"2024-09-19T09:01:52.752141+0000","last_clean":"2024-09-19T09:01:52.752141+0000","last_became_active":"2024-09-19T09:00:45.241097+0000","last_became_peered":"2024-09-19T09:00:45.241097+0000","last_unstale":"2024-09-19T09:01:52.752141+0000","last_undegraded":"2024-09-19T09:01:52.752141+0000","last_fullsized":"2024-09-19T09:01:52.752141+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:35:38.323393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.620427+0000","last_change":"2024-09-19T09:00:46.955194+0000","last_active":"2024-09-19T09:02:21.620427+0000","last_peered":"2024-09-19T09:02:21.620427+0000","last_clean":"2024-09-19T09:02:21.620427+0000","last_became_active":"2024-09-19T09:00:46.954771+0000","last_became_peered":"2024-09-19T09:00:46.954771+0000","last_unstale":"2024-09-19T09:02:21.620427+0000","last_undegraded":"2024-09-19T09:02:21.620427+0000","last_fullsized":"2024-09-19T09:02:21.620427+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:18.945035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'17","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:40.978139+0000","last_change":"2024-09-19T09:00:44.938554+0000","last_active":"2024-09-19T09:02:40.978139+0000","last_peered":"2024-09-19T09:02:40.978139+0000","last_clean":"2024-09-19T09:02:40.978139+0000","last_became_active":"2024-09-19T09:00:44.938261+0000","last_became_peered":"2024-09-19T09:00:44.938261+0000","last_unstale":"2024-09-19T09:02:40.978139+0000","last_undegraded":"2024-09-19T09:02:40.978139+0000","last_fullsized":"2024-09-19T09:02:40.978139+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:24.293764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.614207+0000","last_change":"2024-09-19T09:01:52.755528+0000","last_active":"2024-09-19T09:02:21.614207+0000","last_peered":"2024-09-19T09:02:21.614207+0000","last_clean":"2024-09-19T09:02:21.614207+0000","last_became_active":"2024-09-19T09:01:52.755188+0000","last_became_peered":"2024-09-19T09:01:52.755188+0000","last_unstale":"2024-09-19T09:02:21.614207+0000","last_undegraded":"2024-09-19T09:02:21.614207+0000","last_fullsized":"2024-09-19T09:02:21.614207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:49:34.988733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.888813+0000","last_change":"2024-09-19T09:00:44.941479+0000","last_active":"2024-09-19T09:01:52.888813+0000","last_peered":"2024-09-19T09:01:52.888813+0000","last_clean":"2024-09-19T09:01:52.888813+0000","last_became_active":"2024-09-19T09:00:44.937197+0000","last_became_peered":"2024-09-19T09:00:44.937197+0000","last_unstale":"2024-09-19T09:01:52.888813+0000","last_undegraded":"2024-09-19T09:01:52.888813+0000","last_fullsized":"2024-09-19T09:01:52.888813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:53:17.385884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621026+0000","last_change":"2024-09-19T09:00:46.970319+0000","last_active":"2024-09-19T09:02:21.621026+0000","last_peered":"2024-09-19T09:02:21.621026+0000","last_clean":"2024-09-19T09:02:21.621026+0000","last_became_active":"2024-09-19T09:00:46.969899+0000","last_became_peered":"2024-09-19T09:00:46.969899+0000","last_unstale":"2024-09-19T09:02:21.621026+0000","last_undegraded":"2024-09-19T09:02:21.621026+0000","last_fullsized":"2024-09-19T09:02:21.621026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:26:21.379231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751656+0000","last_change":"2024-09-19T09:00:45.239525+0000","last_active":"2024-09-19T09:01:52.751656+0000","last_peered":"2024-09-19T09:01:52.751656+0000","last_clean":"2024-09-19T09:01:52.751656+0000","last_became_active":"2024-09-19T09:00:45.239213+0000","last_became_peered":"2024-09-19T09:00:45.239213+0000","last_unstale":"2024-09-19T09:01:52.751656+0000","last_undegraded":"2024-09-19T09:01:52.751656+0000","last_fullsized":"2024-09-19T09:01:52.751656+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:32.525025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.584763+0000","last_change":"2024-09-19T09:00:46.958573+0000","last_active":"2024-09-19T09:02:21.584763+0000","last_peered":"2024-09-19T09:02:21.584763+0000","last_clean":"2024-09-19T09:02:21.584763+0000","last_became_active":"2024-09-19T09:00:46.958274+0000","last_became_peered":"2024-09-19T09:00:46.958274+0000","last_unstale":"2024-09-19T09:02:21.584763+0000","last_undegraded":"2024-09-19T09:02:21.584763+0000","last_fullsized":"2024-09-19T09:02:21.584763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:17.249122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.903274+0000","last_change":"2024-09-19T09:00:44.945194+0000","last_active":"2024-09-19T09:01:52.903274+0000","last_peered":"2024-09-19T09:01:52.903274+0000","last_clean":"2024-09-19T09:01:52.903274+0000","last_became_active":"2024-09-19T09:00:44.944931+0000","last_became_peered":"2024-09-19T09:00:44.944931+0000","last_unstale":"2024-09-19T09:01:52.903274+0000","last_undegraded":"2024-09-19T09:01:52.903274+0000","last_fullsized":"2024-09-19T09:01:52.903274+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:11.181150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.621916+0000","last_change":"2024-09-19T09:00:46.966895+0000","last_active":"2024-09-19T09:02:21.621916+0000","last_peered":"2024-09-19T09:02:21.621916+0000","last_clean":"2024-09-19T09:02:21.621916+0000","last_became_active":"2024-09-19T09:00:46.966124+0000","last_became_peered":"2024-09-19T09:00:46.966124+0000","last_unstale":"2024-09-19T09:02:21.621916+0000","last_undegraded":"2024-09-19T09:02:21.621916+0000","last_fullsized":"2024-09-19T09:02:21.621916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:14:04.320863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:41.120672+0000","last_change":"2024-09-19T09:00:44.944167+0000","last_active":"2024-09-19T09:02:41.120672+0000","last_peered":"2024-09-19T09:02:41.120672+0000","last_clean":"2024-09-19T09:02:41.120672+0000","last_became_active":"2024-09-19T09:00:44.943670+0000","last_became_peered":"2024-09-19T09:00:44.943670+0000","last_unstale":"2024-09-19T09:02:41.120672+0000","last_undegraded":"2024-09-19T09:02:41.120672+0000","last_fullsized":"2024-09-19T09:02:41.120672+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:05.596877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.615278+0000","last_change":"2024-09-19T09:00:46.958581+0000","last_active":"2024-09-19T09:02:21.615278+0000","last_peered":"2024-09-19T09:02:21.615278+0000","last_clean":"2024-09-19T09:02:21.615278+0000","last_became_active":"2024-09-19T09:00:46.957782+0000","last_became_peered":"2024-09-19T09:00:46.957782+0000","last_unstale":"2024-09-19T09:02:21.615278+0000","last_undegraded":"2024-09-19T09:02:21.615278+0000","last_fullsized":"2024-09-19T09:02:21.615278+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:37:19.488538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751779+0000","last_change":"2024-09-19T09:00:44.945351+0000","last_active":"2024-09-19T09:01:52.751779+0000","last_peered":"2024-09-19T09:01:52.751779+0000","last_clean":"2024-09-19T09:01:52.751779+0000","last_became_active":"2024-09-19T09:00:44.944991+0000","last_became_peered":"2024-09-19T09:00:44.944991+0000","last_unstale":"2024-09-19T09:01:52.751779+0000","last_undegraded":"2024-09-19T09:01:52.751779+0000","last_fullsized":"2024-09-19T09:01:52.751779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:45:07.581872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633316+0000","last_change":"2024-09-19T09:00:46.956340+0000","last_active":"2024-09-19T09:02:21.633316+0000","last_peered":"2024-09-19T09:02:21.633316+0000","last_clean":"2024-09-19T09:02:21.633316+0000","last_became_active":"2024-09-19T09:00:46.955956+0000","last_became_peered":"2024-09-19T09:00:46.955956+0000","last_unstale":"2024-09-19T09:02:21.633316+0000","last_undegraded":"2024-09-19T09:02:21.633316+0000","last_fullsized":"2024-09-19T09:02:21.633316+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:34.863428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.633515+0000","last_change":"2024-09-19T09:00:46.963657+0000","last_active":"2024-09-19T09:02:21.633515+0000","last_peered":"2024-09-19T09:02:21.633515+0000","last_clean":"2024-09-19T09:02:21.633515+0000","last_became_active":"2024-09-19T09:00:46.963134+0000","last_became_peered":"2024-09-19T09:00:46.963134+0000","last_unstale":"2024-09-19T09:02:21.633515+0000","last_undegraded":"2024-09-19T09:02:21.633515+0000","last_fullsized":"2024-09-19T09:02:21.633515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:01.098570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.764939+0000","last_change":"2024-09-19T09:01:52.764438+0000","last_active":"2024-09-19T09:01:52.764939+0000","last_peered":"2024-09-19T09:01:52.764939+0000","last_clean":"2024-09-19T09:01:52.764939+0000","last_became_active":"2024-09-19T09:01:52.755188+0000","last_became_peered":"2024-09-19T09:01:52.755188+0000","last_unstale":"2024-09-19T09:01:52.764939+0000","last_undegraded":"2024-09-19T09:01:52.764939+0000","last_fullsized":"2024-09-19T09:01:52.764939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:40.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.632994+0000","last_change":"2024-09-19T09:00:46.956354+0000","last_active":"2024-09-19T09:02:21.632994+0000","last_peered":"2024-09-19T09:02:21.632994+0000","last_clean":"2024-09-19T09:02:21.632994+0000","last_became_active":"2024-09-19T09:00:46.955964+0000","last_became_peered":"2024-09-19T09:00:46.955964+0000","last_unstale":"2024-09-19T09:02:21.632994+0000","last_undegraded":"2024-09-19T09:02:21.632994+0000","last_fullsized":"2024-09-19T09:02:21.632994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:05:33.243479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.751944+0000","last_change":"2024-09-19T09:00:44.944983+0000","last_active":"2024-09-19T09:01:52.751944+0000","last_peered":"2024-09-19T09:01:52.751944+0000","last_clean":"2024-09-19T09:01:52.751944+0000","last_became_active":"2024-09-19T09:00:44.944625+0000","last_became_peered":"2024-09-19T09:00:44.944625+0000","last_unstale":"2024-09-19T09:01:52.751944+0000","last_undegraded":"2024-09-19T09:01:52.751944+0000","last_fullsized":"2024-09-19T09:01:52.751944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:49:50.818863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.619916+0000","last_change":"2024-09-19T09:00:46.966448+0000","last_active":"2024-09-19T09:02:21.619916+0000","last_peered":"2024-09-19T09:02:21.619916+0000","last_clean":"2024-09-19T09:02:21.619916+0000","last_became_active":"2024-09-19T09:00:46.966135+0000","last_became_peered":"2024-09-19T09:00:46.966135+0000","last_unstale":"2024-09-19T09:02:21.619916+0000","last_undegraded":"2024-09-19T09:02:21.619916+0000","last_fullsized":"2024-09-19T09:02:21.619916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:42.652877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:01:52.903658+0000","last_change":"2024-09-19T09:00:45.239405+0000","last_active":"2024-09-19T09:01:52.903658+0000","last_peered":"2024-09-19T09:01:52.903658+0000","last_clean":"2024-09-19T09:01:52.903658+0000","last_became_active":"2024-09-19T09:00:45.239002+0000","last_became_peered":"2024-09-19T09:00:45.239002+0000","last_unstale":"2024-09-19T09:01:52.903658+0000","last_undegraded":"2024-09-19T09:01:52.903658+0000","last_fullsized":"2024-09-19T09:01:52.903658+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:46:03.934975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:21.624975+0000","last_change":"2024-09-19T09:00:46.951635+0000","last_active":"2024-09-19T09:02:21.624975+0000","last_peered":"2024-09-19T09:02:21.624975+0000","last_clean":"2024-09-19T09:02:21.624975+0000","last_became_active":"2024-09-19T09:00:46.951325+0000","last_became_peered":"2024-09-19T09:00:46.951325+0000","last_unstale":"2024-09-19T09:02:21.624975+0000","last_undegraded":"2024-09-19T09:02:21.624975+0000","last_fullsized":"2024-09-19T09:02:21.624975+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:45.931684+0000","last_clean_scrub_stamp":"2024-09-19T09:00:45.931684+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:17.360950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":170,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:02:50.875269+0000","last_change":"2024-09-19T09:01:52.766049+0000","last_active":"2024-09-19T09:02:50.875269+0000","last_peered":"2024-09-19T09:02:50.875269+0000","last_clean":"2024-09-19T09:02:50.875269+0000","last_became_active":"2024-09-19T09:01:52.755263+0000","last_became_peered":"2024-09-19T09:01:52.755263+0000","last_unstale":"2024-09-19T09:02:50.875269+0000","last_undegraded":"2024-09-19T09:02:50.875269+0000","last_fullsized":"2024-09-19T09:02:50.875269+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:00:43.921914+0000","last_clean_scrub_stamp":"2024-09-19T09:00:43.921914+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:16.968329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":77751,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1177,"num_read_kb":670,"num_write":266,"num_write_kb":159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1366,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":233253,"data_compressed":4815,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":212166,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612646,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":429768,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841579,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1044480,"data_stored":1089778,"data_compressed":43295,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972401,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":428051,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201334,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":427847,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430268,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":1020726,"data_compressed":41690,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593793,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":427768,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822725,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":1020542,"data_compressed":41690,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986251,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":428163,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215183,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":427593,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":499584,"data_compressed":7388,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":966656,"data_stored":498887,"data_compressed":7388,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":429500,"data_compressed":5783,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71958,"data_compressed":1605,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":72655,"data_compressed":1605,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":72364,"data_compressed":1605,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:02:57.473 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:02:57.757 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:02:58.027 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:57 smithi120 ceph-mon[31150]: from='client.24847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:02:58.027 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:57 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:02:58.027 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:57 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:02:58.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:57 smithi196 ceph-mon[35150]: from='client.24847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:02:58.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:57 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:02:58.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:57 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:02:58.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:57 smithi153 ceph-mon[35071]: from='client.24847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:02:58.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:57 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:02:58.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:57 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:02:58.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.260+0000 7f7b5d8e2640 1 -- 172.21.15.120:0/3010238945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b581027c0 msgr2=0x7f7b58106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:58.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.260+0000 7f7b5d8e2640 1 --2- 172.21.15.120:0/3010238945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b581027c0 0x7f7b58106810 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7b40009900 tx=0x7f7b4002a410 comp rx=0 tx=0).stop 2024-09-19T09:02:58.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.260+0000 7f7b5d8e2640 1 -- 172.21.15.120:0/3010238945 shutdown_connections 2024-09-19T09:02:58.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.260+0000 7f7b5d8e2640 1 --2- 172.21.15.120:0/3010238945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b58106f40 0x7f7b5810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:58.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.260+0000 7f7b5d8e2640 1 --2- 172.21.15.120:0/3010238945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b581027c0 0x7f7b58106810 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:58.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.260+0000 7f7b5d8e2640 1 --2- 172.21.15.120:0/3010238945 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b58101e10 0x7f7b581021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:58.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.260+0000 7f7b5d8e2640 1 -- 172.21.15.120:0/3010238945 >> 172.21.15.120:0/3010238945 conn(0x7f7b580fd8d0 msgr2=0x7f7b580ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:58.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.260+0000 7f7b5d8e2640 1 -- 172.21.15.120:0/3010238945 shutdown_connections 2024-09-19T09:02:58.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.260+0000 7f7b5d8e2640 1 -- 172.21.15.120:0/3010238945 wait complete. 2024-09-19T09:02:58.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.260+0000 7f7b5d8e2640 1 Processor -- start 2024-09-19T09:02:58.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.260+0000 7f7b5d8e2640 1 -- start start 2024-09-19T09:02:58.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.261+0000 7f7b5d8e2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b58101e10 0x7f7b58198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:58.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.261+0000 7f7b5d8e2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b581027c0 0x7f7b58198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:58.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.261+0000 7f7b5d8e2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b58106f40 0x7f7b5819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:58.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.261+0000 7f7b5d8e2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7b58111fa0 con 0x7f7b58101e10 2024-09-19T09:02:58.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.261+0000 7f7b5d8e2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7b58111e20 con 0x7f7b581027c0 2024-09-19T09:02:58.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.261+0000 7f7b5d8e2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7b58112120 con 0x7f7b58106f40 2024-09-19T09:02:58.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.261+0000 7f7b56ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b58101e10 0x7f7b58198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:58.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.261+0000 7f7b56ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b58101e10 0x7f7b58198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51680/0 (socket says 172.21.15.120:51680) 2024-09-19T09:02:58.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.261+0000 7f7b56ffd640 1 -- 172.21.15.120:0/2477974966 learned_addr learned my addr 172.21.15.120:0/2477974966 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:02:58.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.261+0000 7f7b4e7fc640 1 --2- 172.21.15.120:0/2477974966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b581027c0 0x7f7b58198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:58.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.261+0000 7f7b577fe640 1 --2- 172.21.15.120:0/2477974966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b58106f40 0x7f7b5819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:58.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.261+0000 7f7b56ffd640 1 -- 172.21.15.120:0/2477974966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b58106f40 msgr2=0x7f7b5819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:58.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.261+0000 7f7b56ffd640 1 --2- 172.21.15.120:0/2477974966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b58106f40 0x7f7b5819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:58.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.261+0000 7f7b56ffd640 1 -- 172.21.15.120:0/2477974966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b581027c0 msgr2=0x7f7b58198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:58.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.261+0000 7f7b56ffd640 1 --2- 172.21.15.120:0/2477974966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b581027c0 0x7f7b58198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:58.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.261+0000 7f7b56ffd640 1 -- 172.21.15.120:0/2477974966 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b5819d3f0 con 0x7f7b58101e10 2024-09-19T09:02:58.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.261+0000 7f7b577fe640 1 --2- 172.21.15.120:0/2477974966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b58106f40 0x7f7b5819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:02:58.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.261+0000 7f7b4e7fc640 1 --2- 172.21.15.120:0/2477974966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b581027c0 0x7f7b58198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:02:58.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.261+0000 7f7b56ffd640 1 --2- 172.21.15.120:0/2477974966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b58101e10 0x7f7b58198440 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7b4800d950 tx=0x7f7b4800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:58.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.262+0000 7f7b5c8e0640 1 -- 172.21.15.120:0/2477974966 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b48017070 con 0x7f7b58101e10 2024-09-19T09:02:58.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.262+0000 7f7b5c8e0640 1 -- 172.21.15.120:0/2477974966 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b48009c30 con 0x7f7b58101e10 2024-09-19T09:02:58.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.262+0000 7f7b5c8e0640 1 -- 172.21.15.120:0/2477974966 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4800f6b0 con 0x7f7b58101e10 2024-09-19T09:02:58.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.262+0000 7f7b5d8e2640 1 -- 172.21.15.120:0/2477974966 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b5819d6e0 con 0x7f7b58101e10 2024-09-19T09:02:58.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.262+0000 7f7b5d8e2640 1 -- 172.21.15.120:0/2477974966 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b581a4fc0 con 0x7f7b58101e10 2024-09-19T09:02:58.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.263+0000 7f7b5d8e2640 1 -- 172.21.15.120:0/2477974966 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b581021f0 con 0x7f7b58101e10 2024-09-19T09:02:58.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.264+0000 7f7b5c8e0640 1 -- 172.21.15.120:0/2477974966 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f7b480109d0 con 0x7f7b58101e10 2024-09-19T09:02:58.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.265+0000 7f7b5c8e0640 1 --2- 172.21.15.120:0/2477974966 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7b440a32d0 0x7f7b440a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:02:58.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.265+0000 7f7b4e7fc640 1 --2- 172.21.15.120:0/2477974966 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7b440a32d0 0x7f7b440a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:02:58.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.265+0000 7f7b5c8e0640 1 -- 172.21.15.120:0/2477974966 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f7b480cf7e0 con 0x7f7b58101e10 2024-09-19T09:02:58.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.265+0000 7f7b4e7fc640 1 --2- 172.21.15.120:0/2477974966 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7b440a32d0 0x7f7b440a5790 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7b580fdcf0 tx=0x7f7b40005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:02:58.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.269+0000 7f7b5c8e0640 1 -- 172.21.15.120:0/2477974966 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7b48009de0 con 0x7f7b58101e10 2024-09-19T09:02:58.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.474+0000 7f7b5d8e2640 1 -- 172.21.15.120:0/2477974966 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b581997a0 con 0x7f7b58101e10 2024-09-19T09:02:58.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.475+0000 7f7b5c8e0640 1 -- 172.21.15.120:0/2477974966 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f7b4809c3a0 con 0x7f7b58101e10 2024-09-19T09:02:58.475 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:02:58.475 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":28,"btime":"2024-09-19T09:02:41:696728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:02:41.207357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:02:58.475 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 28 2024-09-19T09:02:58.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.477+0000 7f7b5d8e2640 1 -- 172.21.15.120:0/2477974966 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7b440a32d0 msgr2=0x7f7b440a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:58.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.477+0000 7f7b5d8e2640 1 --2- 172.21.15.120:0/2477974966 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7b440a32d0 0x7f7b440a5790 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7b580fdcf0 tx=0x7f7b40005a20 comp rx=0 tx=0).stop 2024-09-19T09:02:58.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.477+0000 7f7b5d8e2640 1 -- 172.21.15.120:0/2477974966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b58101e10 msgr2=0x7f7b58198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:02:58.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.477+0000 7f7b5d8e2640 1 --2- 172.21.15.120:0/2477974966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b58101e10 0x7f7b58198440 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7b4800d950 tx=0x7f7b4800de10 comp rx=0 tx=0).stop 2024-09-19T09:02:58.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.477+0000 7f7b5d8e2640 1 -- 172.21.15.120:0/2477974966 shutdown_connections 2024-09-19T09:02:58.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.477+0000 7f7b5d8e2640 1 --2- 172.21.15.120:0/2477974966 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7b440a32d0 0x7f7b440a5790 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:58.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.477+0000 7f7b5d8e2640 1 --2- 172.21.15.120:0/2477974966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b58106f40 0x7f7b5819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:58.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.477+0000 7f7b5d8e2640 1 --2- 172.21.15.120:0/2477974966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b581027c0 0x7f7b58198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:58.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.477+0000 7f7b5d8e2640 1 --2- 172.21.15.120:0/2477974966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b58101e10 0x7f7b58198440 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:02:58.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.477+0000 7f7b5d8e2640 1 -- 172.21.15.120:0/2477974966 >> 172.21.15.120:0/2477974966 conn(0x7f7b580fd8d0 msgr2=0x7f7b580fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:02:58.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.477+0000 7f7b5d8e2640 1 -- 172.21.15.120:0/2477974966 shutdown_connections 2024-09-19T09:02:58.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:02:58.477+0000 7f7b5d8e2640 1 -- 172.21.15.120:0/2477974966 wait complete. 2024-09-19T09:02:58.850 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:58 smithi120 ceph-mon[31150]: pgmap v344: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-19T09:02:58.850 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:58 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:02:58.850 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:58 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:02:58.850 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:58 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:02:58.850 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:02:58 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2477974966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:02:59.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:58 smithi196 ceph-mon[35150]: pgmap v344: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-19T09:02:59.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:58 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:02:59.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:58 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:02:59.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:58 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:02:59.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:02:58 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2477974966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:02:59.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:58 smithi153 ceph-mon[35071]: pgmap v344: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-19T09:02:59.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:58 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:02:59.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:58 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:02:59.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:58 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:02:59.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:02:58 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2477974966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:02:59.204 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:02:59.449 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:03:00.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.003+0000 7f788efe5640 1 -- 172.21.15.120:0/1568546171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7888101de0 msgr2=0x7f78881021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:00.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.003+0000 7f788efe5640 1 --2- 172.21.15.120:0/1568546171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7888101de0 0x7f78881021c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f787c009900 tx=0x7f787c02a410 comp rx=0 tx=0).stop 2024-09-19T09:03:00.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.003+0000 7f788efe5640 1 -- 172.21.15.120:0/1568546171 shutdown_connections 2024-09-19T09:03:00.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.003+0000 7f788efe5640 1 --2- 172.21.15.120:0/1568546171 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7888106f10 0x7f788810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:00.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.003+0000 7f788efe5640 1 --2- 172.21.15.120:0/1568546171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7888102790 0x7f78881067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:00.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.003+0000 7f788efe5640 1 --2- 172.21.15.120:0/1568546171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7888101de0 0x7f78881021c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:00.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.003+0000 7f788efe5640 1 -- 172.21.15.120:0/1568546171 >> 172.21.15.120:0/1568546171 conn(0x7f78880fd8c0 msgr2=0x7f78880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:00.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.003+0000 7f788efe5640 1 -- 172.21.15.120:0/1568546171 shutdown_connections 2024-09-19T09:03:00.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.003+0000 7f788efe5640 1 -- 172.21.15.120:0/1568546171 wait complete. 2024-09-19T09:03:00.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.003+0000 7f788efe5640 1 Processor -- start 2024-09-19T09:03:00.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.003+0000 7f788efe5640 1 -- start start 2024-09-19T09:03:00.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.003+0000 7f788efe5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7888101de0 0x7f78880744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:00.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.003+0000 7f788efe5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7888102790 0x7f7888074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:00.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.003+0000 7f788efe5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7888106f10 0x7f7888074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:00.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.003+0000 7f788efe5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7888111f30 con 0x7f7888106f10 2024-09-19T09:03:00.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.003+0000 7f788efe5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7888111db0 con 0x7f7888101de0 2024-09-19T09:03:00.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.003+0000 7f788efe5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f78881120b0 con 0x7f7888102790 2024-09-19T09:03:00.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.004+0000 7f788d582640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7888106f10 0x7f7888074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:00.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.004+0000 7f788d582640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7888106f10 0x7f7888074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51710/0 (socket says 172.21.15.120:51710) 2024-09-19T09:03:00.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.004+0000 7f788d582640 1 -- 172.21.15.120:0/725347347 learned_addr learned my addr 172.21.15.120:0/725347347 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:00.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.004+0000 7f788cd81640 1 --2- 172.21.15.120:0/725347347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7888101de0 0x7f78880744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:00.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.004+0000 7f7887fff640 1 --2- 172.21.15.120:0/725347347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7888102790 0x7f7888074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:00.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.004+0000 7f788d582640 1 -- 172.21.15.120:0/725347347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7888102790 msgr2=0x7f7888074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:00.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.004+0000 7f788d582640 1 --2- 172.21.15.120:0/725347347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7888102790 0x7f7888074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:00.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.004+0000 7f788d582640 1 -- 172.21.15.120:0/725347347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7888101de0 msgr2=0x7f78880744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:00.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.004+0000 7f788d582640 1 --2- 172.21.15.120:0/725347347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7888101de0 0x7f78880744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:00.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.004+0000 7f788d582640 1 -- 172.21.15.120:0/725347347 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78881a6900 con 0x7f7888106f10 2024-09-19T09:03:00.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.004+0000 7f788d582640 1 --2- 172.21.15.120:0/725347347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7888106f10 0x7f7888074f70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f788000cad0 tx=0x7f788000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:00.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.004+0000 7f7885ffb640 1 -- 172.21.15.120:0/725347347 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788000ae20 con 0x7f7888106f10 2024-09-19T09:03:00.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.004+0000 7f7885ffb640 1 -- 172.21.15.120:0/725347347 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7880005020 con 0x7f7888106f10 2024-09-19T09:03:00.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.004+0000 7f788efe5640 1 -- 172.21.15.120:0/725347347 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78881a6bf0 con 0x7f7888106f10 2024-09-19T09:03:00.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.004+0000 7f7885ffb640 1 -- 172.21.15.120:0/725347347 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788000f6f0 con 0x7f7888106f10 2024-09-19T09:03:00.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.005+0000 7f788efe5640 1 -- 172.21.15.120:0/725347347 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78881a7130 con 0x7f7888106f10 2024-09-19T09:03:00.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.005+0000 7f788efe5640 1 -- 172.21.15.120:0/725347347 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78881021c0 con 0x7f7888106f10 2024-09-19T09:03:00.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.006+0000 7f7885ffb640 1 -- 172.21.15.120:0/725347347 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f78800051d0 con 0x7f7888106f10 2024-09-19T09:03:00.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.007+0000 7f7885ffb640 1 --2- 172.21.15.120:0/725347347 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f78780a3350 0x7f78780a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:00.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.007+0000 7f788cd81640 1 --2- 172.21.15.120:0/725347347 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f78780a3350 0x7f78780a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:00.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.007+0000 7f7885ffb640 1 -- 172.21.15.120:0/725347347 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f78800cfb10 con 0x7f7888106f10 2024-09-19T09:03:00.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.008+0000 7f788cd81640 1 --2- 172.21.15.120:0/725347347 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f78780a3350 0x7f78780a5810 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f787c005ab0 tx=0x7f787c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:00.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.011+0000 7f7885ffb640 1 -- 172.21.15.120:0/725347347 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7880014030 con 0x7f7888106f10 2024-09-19T09:03:00.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.213+0000 7f788efe5640 1 -- 172.21.15.120:0/725347347 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f788810a0f0 con 0x7f7888106f10 2024-09-19T09:03:00.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.214+0000 7f7885ffb640 1 -- 172.21.15.120:0/725347347 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f788009c6d0 con 0x7f7888106f10 2024-09-19T09:03:00.214 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:03:00.214 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":28,"btime":"2024-09-19T09:02:41:696728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:02:41.207357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:03:00.214 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 28 2024-09-19T09:03:00.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.216+0000 7f788efe5640 1 -- 172.21.15.120:0/725347347 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f78780a3350 msgr2=0x7f78780a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:00.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.216+0000 7f788efe5640 1 --2- 172.21.15.120:0/725347347 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f78780a3350 0x7f78780a5810 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f787c005ab0 tx=0x7f787c005a20 comp rx=0 tx=0).stop 2024-09-19T09:03:00.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.216+0000 7f788efe5640 1 -- 172.21.15.120:0/725347347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7888106f10 msgr2=0x7f7888074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:00.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.216+0000 7f788efe5640 1 --2- 172.21.15.120:0/725347347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7888106f10 0x7f7888074f70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f788000cad0 tx=0x7f788000cf90 comp rx=0 tx=0).stop 2024-09-19T09:03:00.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.216+0000 7f788efe5640 1 -- 172.21.15.120:0/725347347 shutdown_connections 2024-09-19T09:03:00.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.216+0000 7f788efe5640 1 --2- 172.21.15.120:0/725347347 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f78780a3350 0x7f78780a5810 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:00.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.216+0000 7f788efe5640 1 --2- 172.21.15.120:0/725347347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7888106f10 0x7f7888074f70 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:00.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.216+0000 7f788efe5640 1 --2- 172.21.15.120:0/725347347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7888102790 0x7f7888074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:00.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.216+0000 7f788efe5640 1 --2- 172.21.15.120:0/725347347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7888101de0 0x7f78880744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:00.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.216+0000 7f788efe5640 1 -- 172.21.15.120:0/725347347 >> 172.21.15.120:0/725347347 conn(0x7f78880fd8c0 msgr2=0x7f78880fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:00.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.216+0000 7f788efe5640 1 -- 172.21.15.120:0/725347347 shutdown_connections 2024-09-19T09:03:00.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:00.216+0000 7f788efe5640 1 -- 172.21.15.120:0/725347347 wait complete. 2024-09-19T09:03:00.912 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:00 smithi120 ceph-mon[31150]: pgmap v345: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-19T09:03:00.912 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:00 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/725347347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:03:00.941 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:03:01.185 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:03:01.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:00 smithi196 ceph-mon[35150]: pgmap v345: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-19T09:03:01.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:00 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/725347347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:03:01.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:00 smithi153 ceph-mon[35071]: pgmap v345: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-19T09:03:01.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:00 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/725347347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:03:01.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.678+0000 7f7d384ec640 1 -- 172.21.15.120:0/2068583060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d3010e480 msgr2=0x7f7d30110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:01.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.678+0000 7f7d384ec640 1 --2- 172.21.15.120:0/2068583060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d3010e480 0x7f7d30110870 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7d28009ca0 tx=0x7f7d2802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:03:01.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.678+0000 7f7d384ec640 1 -- 172.21.15.120:0/2068583060 shutdown_connections 2024-09-19T09:03:01.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.678+0000 7f7d384ec640 1 --2- 172.21.15.120:0/2068583060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d3010e480 0x7f7d30110870 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:01.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.678+0000 7f7d384ec640 1 --2- 172.21.15.120:0/2068583060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d30101280 0x7f7d3010df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:01.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.678+0000 7f7d384ec640 1 --2- 172.21.15.120:0/2068583060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d301008d0 0x7f7d30100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:01.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.678+0000 7f7d384ec640 1 -- 172.21.15.120:0/2068583060 >> 172.21.15.120:0/2068583060 conn(0x7f7d300fc7a0 msgr2=0x7f7d300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:01.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.678+0000 7f7d384ec640 1 -- 172.21.15.120:0/2068583060 shutdown_connections 2024-09-19T09:03:01.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.679+0000 7f7d384ec640 1 -- 172.21.15.120:0/2068583060 wait complete. 2024-09-19T09:03:01.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.679+0000 7f7d384ec640 1 Processor -- start 2024-09-19T09:03:01.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.679+0000 7f7d384ec640 1 -- start start 2024-09-19T09:03:01.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.679+0000 7f7d384ec640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d301008d0 0x7f7d3019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:01.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.679+0000 7f7d384ec640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d30101280 0x7f7d3019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:01.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.679+0000 7f7d384ec640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d3010e480 0x7f7d3019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:01.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.679+0000 7f7d384ec640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7d30114160 con 0x7f7d30101280 2024-09-19T09:03:01.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.679+0000 7f7d384ec640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7d30113fe0 con 0x7f7d301008d0 2024-09-19T09:03:01.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.679+0000 7f7d384ec640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7d301142e0 con 0x7f7d3010e480 2024-09-19T09:03:01.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.679+0000 7f7d35a87640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d30101280 0x7f7d3019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:01.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.679+0000 7f7d35a87640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d30101280 0x7f7d3019aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56086/0 (socket says 172.21.15.120:56086) 2024-09-19T09:03:01.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.679+0000 7f7d35a87640 1 -- 172.21.15.120:0/2206060800 learned_addr learned my addr 172.21.15.120:0/2206060800 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:01.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.679+0000 7f7d36288640 1 --2- 172.21.15.120:0/2206060800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d301008d0 0x7f7d3019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:01.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.680+0000 7f7d36a89640 1 --2- 172.21.15.120:0/2206060800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d3010e480 0x7f7d3019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:01.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.680+0000 7f7d35a87640 1 -- 172.21.15.120:0/2206060800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d3010e480 msgr2=0x7f7d3019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:01.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.680+0000 7f7d35a87640 1 --2- 172.21.15.120:0/2206060800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d3010e480 0x7f7d3019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:01.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.680+0000 7f7d35a87640 1 -- 172.21.15.120:0/2206060800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d301008d0 msgr2=0x7f7d3019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:01.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.680+0000 7f7d35a87640 1 --2- 172.21.15.120:0/2206060800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d301008d0 0x7f7d3019a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:01.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.680+0000 7f7d35a87640 1 -- 172.21.15.120:0/2206060800 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3019f610 con 0x7f7d30101280 2024-09-19T09:03:01.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.680+0000 7f7d35a87640 1 --2- 172.21.15.120:0/2206060800 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d30101280 0x7f7d3019aba0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000d950 tx=0x7f7d2000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:01.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.680+0000 7f7d277fe640 1 -- 172.21.15.120:0/2206060800 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20017070 con 0x7f7d30101280 2024-09-19T09:03:01.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.680+0000 7f7d277fe640 1 -- 172.21.15.120:0/2206060800 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d20009c30 con 0x7f7d30101280 2024-09-19T09:03:01.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.680+0000 7f7d384ec640 1 -- 172.21.15.120:0/2206060800 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d3019f900 con 0x7f7d30101280 2024-09-19T09:03:01.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.680+0000 7f7d277fe640 1 -- 172.21.15.120:0/2206060800 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2000f6b0 con 0x7f7d30101280 2024-09-19T09:03:01.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.680+0000 7f7d384ec640 1 -- 172.21.15.120:0/2206060800 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d301a71e0 con 0x7f7d30101280 2024-09-19T09:03:01.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.681+0000 7f7d384ec640 1 -- 172.21.15.120:0/2206060800 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d30100cb0 con 0x7f7d30101280 2024-09-19T09:03:01.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.682+0000 7f7d277fe640 1 -- 172.21.15.120:0/2206060800 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f7d200109d0 con 0x7f7d30101280 2024-09-19T09:03:01.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.683+0000 7f7d277fe640 1 --2- 172.21.15.120:0/2206060800 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7d1c0a3350 0x7f7d1c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:01.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.683+0000 7f7d277fe640 1 -- 172.21.15.120:0/2206060800 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f7d200cf5f0 con 0x7f7d30101280 2024-09-19T09:03:01.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.683+0000 7f7d36288640 1 --2- 172.21.15.120:0/2206060800 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7d1c0a3350 0x7f7d1c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:01.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.683+0000 7f7d36288640 1 --2- 172.21.15.120:0/2206060800 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7d1c0a3350 0x7f7d1c0a5810 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00bbe0 tx=0x7f7d2c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:01.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.687+0000 7f7d277fe640 1 -- 172.21.15.120:0/2206060800 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7d20009de0 con 0x7f7d30101280 2024-09-19T09:03:01.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.856+0000 7f7d384ec640 1 -- 172.21.15.120:0/2206060800 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d30109670 con 0x7f7d30101280 2024-09-19T09:03:01.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.857+0000 7f7d277fe640 1 -- 172.21.15.120:0/2206060800 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22844 (secure 0 0 0) 0x7f7d2009c1b0 con 0x7f7d30101280 2024-09-19T09:03:01.858 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:03:01.858 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":81,"fsid":"507393d2-7664-11ef-bceb-c7b262605968","created":"2024-09-19T08:53:06.386526+0000","modified":"2024-09-19T09:01:52.744203+0000","last_up_change":"2024-09-19T08:59:55.692659+0000","last_in_change":"2024-09-19T08:59:32.808860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:56:07.830047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:00:43.774339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:00:45.786262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33ec3993-576c-428c-bf75-adcb17cd72a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6803","nonce":4131180019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6805","nonce":4131180019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6809","nonce":4131180019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6807","nonce":4131180019}]},"public_addr":"172.21.15.120:6803/4131180019","cluster_addr":"172.21.15.120:6805/4131180019","heartbeat_back_addr":"172.21.15.120:6809/4131180019","heartbeat_front_addr":"172.21.15.120:6807/4131180019","state":["exists","up"]},{"osd":1,"uuid":"450bbebf-2aac-47c6-8474-bbd37fd0a1a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6801","nonce":2259730294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6803","nonce":2259730294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6807","nonce":2259730294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6805","nonce":2259730294}]},"public_addr":"172.21.15.153:6801/2259730294","cluster_addr":"172.21.15.153:6803/2259730294","heartbeat_back_addr":"172.21.15.153:6807/2259730294","heartbeat_front_addr":"172.21.15.153:6805/2259730294","state":["exists","up"]},{"osd":2,"uuid":"ac8c4b83-8e8e-4803-b266-262f7873bd85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6801","nonce":2942286570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6803","nonce":2942286570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6807","nonce":2942286570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6805","nonce":2942286570}]},"public_addr":"172.21.15.196:6801/2942286570","cluster_addr":"172.21.15.196:6803/2942286570","heartbeat_back_addr":"172.21.15.196:6807/2942286570","heartbeat_front_addr":"172.21.15.196:6805/2942286570","state":["exists","up"]},{"osd":3,"uuid":"c1d9e507-8f02-403e-87da-9b4966fd8fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6811","nonce":713205668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6813","nonce":713205668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6817","nonce":713205668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6815","nonce":713205668}]},"public_addr":"172.21.15.120:6811/713205668","cluster_addr":"172.21.15.120:6813/713205668","heartbeat_back_addr":"172.21.15.120:6817/713205668","heartbeat_front_addr":"172.21.15.120:6815/713205668","state":["exists","up"]},{"osd":4,"uuid":"7f8bf3b9-0ec4-46f6-a406-d26d7971179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6809","nonce":2680865854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6811","nonce":2680865854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6815","nonce":2680865854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6813","nonce":2680865854}]},"public_addr":"172.21.15.153:6809/2680865854","cluster_addr":"172.21.15.153:6811/2680865854","heartbeat_back_addr":"172.21.15.153:6815/2680865854","heartbeat_front_addr":"172.21.15.153:6813/2680865854","state":["exists","up"]},{"osd":5,"uuid":"70b270b2-209d-4ff5-875a-04ee7b992a93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6809","nonce":3763331116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6811","nonce":3763331116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6815","nonce":3763331116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6813","nonce":3763331116}]},"public_addr":"172.21.15.196:6809/3763331116","cluster_addr":"172.21.15.196:6811/3763331116","heartbeat_back_addr":"172.21.15.196:6815/3763331116","heartbeat_front_addr":"172.21.15.196:6813/3763331116","state":["exists","up"]},{"osd":6,"uuid":"8087f69c-dc31-4522-a18d-4e6ab1b7cb27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6819","nonce":4171773765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6821","nonce":4171773765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6825","nonce":4171773765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6823","nonce":4171773765}]},"public_addr":"172.21.15.120:6819/4171773765","cluster_addr":"172.21.15.120:6821/4171773765","heartbeat_back_addr":"172.21.15.120:6825/4171773765","heartbeat_front_addr":"172.21.15.120:6823/4171773765","state":["exists","up"]},{"osd":7,"uuid":"c3b1ead3-150f-43fd-8924-f639203f4cdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6817","nonce":2535148453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6819","nonce":2535148453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6823","nonce":2535148453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6821","nonce":2535148453}]},"public_addr":"172.21.15.153:6817/2535148453","cluster_addr":"172.21.15.153:6819/2535148453","heartbeat_back_addr":"172.21.15.153:6823/2535148453","heartbeat_front_addr":"172.21.15.153:6821/2535148453","state":["exists","up"]},{"osd":8,"uuid":"a3273a2a-968e-4e93-a9cf-3879892bdeab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6817","nonce":186195906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6819","nonce":186195906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6823","nonce":186195906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6821","nonce":186195906}]},"public_addr":"172.21.15.196:6817/186195906","cluster_addr":"172.21.15.196:6819/186195906","heartbeat_back_addr":"172.21.15.196:6823/186195906","heartbeat_front_addr":"172.21.15.196:6821/186195906","state":["exists","up"]},{"osd":9,"uuid":"6d184513-dab8-45f7-bc08-fa2e8b2f10b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6827","nonce":1559320883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6829","nonce":1559320883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6833","nonce":1559320883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6831","nonce":1559320883}]},"public_addr":"172.21.15.120:6827/1559320883","cluster_addr":"172.21.15.120:6829/1559320883","heartbeat_back_addr":"172.21.15.120:6833/1559320883","heartbeat_front_addr":"172.21.15.120:6831/1559320883","state":["exists","up"]},{"osd":10,"uuid":"797db886-62f9-4316-a45b-4d50512e65d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6825","nonce":3216877560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6827","nonce":3216877560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6831","nonce":3216877560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6829","nonce":3216877560}]},"public_addr":"172.21.15.153:6825/3216877560","cluster_addr":"172.21.15.153:6827/3216877560","heartbeat_back_addr":"172.21.15.153:6831/3216877560","heartbeat_front_addr":"172.21.15.153:6829/3216877560","state":["exists","up"]},{"osd":11,"uuid":"17e0afaa-84eb-4b26-9810-bfa3cc7e0f3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6825","nonce":1857200740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6827","nonce":1857200740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6831","nonce":1857200740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6829","nonce":1857200740}]},"public_addr":"172.21.15.196:6825/1857200740","cluster_addr":"172.21.15.196:6827/1857200740","heartbeat_back_addr":"172.21.15.196:6831/1857200740","heartbeat_front_addr":"172.21.15.196:6829/1857200740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:55:15.595645+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:55:40.405487+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:04.938034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:29.718696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:55.636029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:57:21.257083+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:57:46.316552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:58:11.946863+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:58:37.098578+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:03.759023+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:28.659049+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:54.350475+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:0/3010783707":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6801/3562882246":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6800/3562882246":"2024-09-20T08:53:51.528772+0000","172.21.15.120:0/3224512035":"2024-09-20T08:53:51.528772+0000","172.21.15.120:0/3465105835":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/3317449610":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/2800697451":"2024-09-20T08:53:28.144376+0000","172.21.15.120:6801/2938446665":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/109127432":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6800/2938446665":"2024-09-20T08:53:28.144376+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:03:01.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.860+0000 7f7d384ec640 1 -- 172.21.15.120:0/2206060800 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7d1c0a3350 msgr2=0x7f7d1c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:01.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.860+0000 7f7d384ec640 1 --2- 172.21.15.120:0/2206060800 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7d1c0a3350 0x7f7d1c0a5810 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00bbe0 tx=0x7f7d2c00b390 comp rx=0 tx=0).stop 2024-09-19T09:03:01.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.860+0000 7f7d384ec640 1 -- 172.21.15.120:0/2206060800 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d30101280 msgr2=0x7f7d3019aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:01.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.860+0000 7f7d384ec640 1 --2- 172.21.15.120:0/2206060800 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d30101280 0x7f7d3019aba0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000d950 tx=0x7f7d2000de10 comp rx=0 tx=0).stop 2024-09-19T09:03:01.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.860+0000 7f7d384ec640 1 -- 172.21.15.120:0/2206060800 shutdown_connections 2024-09-19T09:03:01.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.860+0000 7f7d384ec640 1 --2- 172.21.15.120:0/2206060800 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7d1c0a3350 0x7f7d1c0a5810 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:01.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.860+0000 7f7d384ec640 1 --2- 172.21.15.120:0/2206060800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d3010e480 0x7f7d3019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:01.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.860+0000 7f7d384ec640 1 --2- 172.21.15.120:0/2206060800 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d30101280 0x7f7d3019aba0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:01.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.860+0000 7f7d384ec640 1 --2- 172.21.15.120:0/2206060800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d301008d0 0x7f7d3019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:01.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.860+0000 7f7d384ec640 1 -- 172.21.15.120:0/2206060800 >> 172.21.15.120:0/2206060800 conn(0x7f7d300fc7a0 msgr2=0x7f7d300feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:01.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.860+0000 7f7d384ec640 1 -- 172.21.15.120:0/2206060800 shutdown_connections 2024-09-19T09:03:01.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:01.860+0000 7f7d384ec640 1 -- 172.21.15.120:0/2206060800 wait complete. 2024-09-19T09:03:02.577 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-19T09:03:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.631+0000 7f0944de4640 1 -- 172.21.15.120:0/2996659623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f09401025f0 msgr2=0x7f0940108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.631+0000 7f0944de4640 1 --2- 172.21.15.120:0/2996659623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f09401025f0 0x7f0940108e80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0938009ca0 tx=0x7f093802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.632+0000 7f0944de4640 1 -- 172.21.15.120:0/2996659623 shutdown_connections 2024-09-19T09:03:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.632+0000 7f0944de4640 1 --2- 172.21.15.120:0/2996659623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f09401025f0 0x7f0940108e80 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.632+0000 7f0944de4640 1 --2- 172.21.15.120:0/2996659623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0940101c30 0x7f09401020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.632+0000 7f0944de4640 1 --2- 172.21.15.120:0/2996659623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0940100a40 0x7f0940100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.632+0000 7f0944de4640 1 -- 172.21.15.120:0/2996659623 >> 172.21.15.120:0/2996659623 conn(0x7f09400fc1e0 msgr2=0x7f09400fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.632+0000 7f0944de4640 1 -- 172.21.15.120:0/2996659623 shutdown_connections 2024-09-19T09:03:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.632+0000 7f0944de4640 1 -- 172.21.15.120:0/2996659623 wait complete. 2024-09-19T09:03:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.632+0000 7f0944de4640 1 Processor -- start 2024-09-19T09:03:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.632+0000 7f0944de4640 1 -- start start 2024-09-19T09:03:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.632+0000 7f0944de4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0940100a40 0x7f094010b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.633+0000 7f0944de4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0940101c30 0x7f0940109a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.633+0000 7f0944de4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09401025f0 0x7f0940109f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.633+0000 7f0944de4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f094010d850 con 0x7f0940101c30 2024-09-19T09:03:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.633+0000 7f0944de4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f094010d6d0 con 0x7f0940100a40 2024-09-19T09:03:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.633+0000 7f0944de4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f094010d9d0 con 0x7f09401025f0 2024-09-19T09:03:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.633+0000 7f0937fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0940101c30 0x7f0940109a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.633+0000 7f093e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0940100a40 0x7f094010b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.633+0000 7f093e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0940100a40 0x7f094010b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:36458/0 (socket says 172.21.15.120:36458) 2024-09-19T09:03:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.633+0000 7f0937fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0940101c30 0x7f0940109a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56108/0 (socket says 172.21.15.120:56108) 2024-09-19T09:03:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.633+0000 7f093e59c640 1 -- 172.21.15.120:0/1982858665 learned_addr learned my addr 172.21.15.120:0/1982858665 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.633+0000 7f093ed9d640 1 --2- 172.21.15.120:0/1982858665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09401025f0 0x7f0940109f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.633+0000 7f0937fff640 1 -- 172.21.15.120:0/1982858665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09401025f0 msgr2=0x7f0940109f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.633+0000 7f0937fff640 1 --2- 172.21.15.120:0/1982858665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09401025f0 0x7f0940109f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.633+0000 7f0937fff640 1 -- 172.21.15.120:0/1982858665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0940100a40 msgr2=0x7f094010b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.633+0000 7f0937fff640 1 --2- 172.21.15.120:0/1982858665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0940100a40 0x7f094010b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.633+0000 7f0937fff640 1 -- 172.21.15.120:0/1982858665 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f094010a7d0 con 0x7f0940101c30 2024-09-19T09:03:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.633+0000 7f093e59c640 1 --2- 172.21.15.120:0/1982858665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0940100a40 0x7f094010b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:03:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.633+0000 7f0937fff640 1 --2- 172.21.15.120:0/1982858665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0940101c30 0x7f0940109a00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0928005fd0 tx=0x7f092800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.633+0000 7f09377fe640 1 -- 172.21.15.120:0/1982858665 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928009c30 con 0x7f0940101c30 2024-09-19T09:03:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.633+0000 7f09377fe640 1 -- 172.21.15.120:0/1982858665 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0928009dd0 con 0x7f0940101c30 2024-09-19T09:03:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.634+0000 7f0944de4640 1 -- 172.21.15.120:0/1982858665 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09401a1f10 con 0x7f0940101c30 2024-09-19T09:03:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.634+0000 7f09377fe640 1 -- 172.21.15.120:0/1982858665 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928010460 con 0x7f0940101c30 2024-09-19T09:03:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.634+0000 7f0944de4640 1 -- 172.21.15.120:0/1982858665 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09401a2370 con 0x7f0940101c30 2024-09-19T09:03:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.636+0000 7f09377fe640 1 -- 172.21.15.120:0/1982858665 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f0928010600 con 0x7f0940101c30 2024-09-19T09:03:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.636+0000 7f0947c57740 1 Processor -- start 2024-09-19T09:03:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.636+0000 7f0947c57740 1 -- start start 2024-09-19T09:03:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.636+0000 7f0947c57740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c29d1e29a0 0x55c29d1e2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.636+0000 7f0947c57740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c29d1e32e0 0x55c29d1e7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.636+0000 7f0947c57740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c29d1e8370 0x55c29d1ea830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.636+0000 7f0947c57740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c29d1462f0 con 0x55c29d1e32e0 2024-09-19T09:03:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.636+0000 7f0947c57740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c29d0ee270 con 0x55c29d1e29a0 2024-09-19T09:03:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.636+0000 7f0947c57740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c29d1422b0 con 0x55c29d1e8370 2024-09-19T09:03:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.636+0000 7f093e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c29d1e32e0 0x55c29d1e7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.636+0000 7f093e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c29d1e32e0 0x55c29d1e7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56118/0 (socket says 172.21.15.120:56118) 2024-09-19T09:03:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.636+0000 7f093e59c640 1 -- 172.21.15.120:0/2251476524 learned_addr learned my addr 172.21.15.120:0/2251476524 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.637+0000 7f09377fe640 1 --2- 172.21.15.120:0/1982858665 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f092c0a3390 0x7f092c0a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.637+0000 7f0937fff640 1 --2- 172.21.15.120:0/2251476524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c29d1e8370 0x55c29d1ea830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.637+0000 7f093ed9d640 1 --2- 172.21.15.120:0/2251476524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c29d1e29a0 0x55c29d1e2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.637+0000 7f093ed9d640 1 --2- 172.21.15.120:0/1982858665 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f092c0a3390 0x7f092c0a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.637+0000 7f093e59c640 1 -- 172.21.15.120:0/2251476524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c29d1e8370 msgr2=0x55c29d1ea830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.637+0000 7f093e59c640 1 --2- 172.21.15.120:0/2251476524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c29d1e8370 0x55c29d1ea830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.637+0000 7f093e59c640 1 -- 172.21.15.120:0/2251476524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c29d1e29a0 msgr2=0x55c29d1e2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.637+0000 7f093e59c640 1 --2- 172.21.15.120:0/2251476524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c29d1e29a0 0x55c29d1e2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.637+0000 7f093e59c640 1 -- 172.21.15.120:0/2251476524 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c29d0ed340 con 0x55c29d1e32e0 2024-09-19T09:03:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.637+0000 7f09377fe640 1 -- 172.21.15.120:0/1982858665 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f09280cf990 con 0x7f0940101c30 2024-09-19T09:03:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.637+0000 7f093e59c640 1 --2- 172.21.15.120:0/2251476524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c29d1e32e0 0x55c29d1e7da0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0930009ed0 tx=0x7f093000e5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5d10e4508e909f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.637+0000 7f093ed9d640 1 --2- 172.21.15.120:0/1982858665 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f092c0a3390 0x7f092c0a5850 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0938009430 tx=0x7f0938005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.637+0000 7f0934ff9640 1 -- 172.21.15.120:0/2251476524 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930017070 con 0x55c29d1e32e0 2024-09-19T09:03:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.637+0000 7f0934ff9640 1 -- 172.21.15.120:0/2251476524 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0930013950 con 0x55c29d1e32e0 2024-09-19T09:03:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.637+0000 7f0934ff9640 1 -- 172.21.15.120:0/2251476524 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930012a10 con 0x55c29d1e32e0 2024-09-19T09:03:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.638+0000 7f0947c57740 1 -- 172.21.15.120:0/2251476524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c29d1e32e0 msgr2=0x55c29d1e7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.638+0000 7f0947c57740 1 --2- 172.21.15.120:0/2251476524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c29d1e32e0 0x55c29d1e7da0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0930009ed0 tx=0x7f093000e5a0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.638+0000 7f0947c57740 1 -- 172.21.15.120:0/2251476524 shutdown_connections 2024-09-19T09:03:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.638+0000 7f0947c57740 1 --2- 172.21.15.120:0/2251476524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c29d1e8370 0x55c29d1ea830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.638+0000 7f0947c57740 1 --2- 172.21.15.120:0/2251476524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c29d1e32e0 0x55c29d1e7da0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.638+0000 7f0947c57740 1 --2- 172.21.15.120:0/2251476524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c29d1e29a0 0x55c29d1e2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.638+0000 7f0947c57740 1 -- 172.21.15.120:0/2251476524 >> 172.21.15.120:0/2251476524 conn(0x55c29d1de010 msgr2=0x55c29d1de410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.638+0000 7f0947c57740 1 -- 172.21.15.120:0/2251476524 shutdown_connections 2024-09-19T09:03:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.638+0000 7f0947c57740 1 -- 172.21.15.120:0/2251476524 wait complete. 2024-09-19T09:03:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.638+0000 7f0947c57740 1 -- start start 2024-09-19T09:03:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.639+0000 7f0947c57740 1 Processor -- start 2024-09-19T09:03:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.639+0000 7f0947c57740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c29d1e29a0 0x55c29d1ecb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.639+0000 7f0947c57740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c29d1e32e0 0x55c29d1ed0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.639+0000 7f0947c57740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c29d1e8370 0x55c29d1f40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.639+0000 7f0947c57740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c29d1eb340 con 0x55c29d1e8370 2024-09-19T09:03:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.639+0000 7f0947c57740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c29d0cf290 con 0x55c29d1e29a0 2024-09-19T09:03:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.639+0000 7f0947c57740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c29d1eb4c0 con 0x55c29d1e32e0 2024-09-19T09:03:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.639+0000 7f0947c57740 10 client.0 _ll_register_callbacks cb 0x55c29d100140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.639+0000 7f093ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c29d1e8370 0x55c29d1f40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.639+0000 7f093ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c29d1e8370 0x55c29d1f40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56122/0 (socket says 172.21.15.120:56122) 2024-09-19T09:03:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.639+0000 7f093ed9d640 1 -- 172.21.15.120:0/3861873436 learned_addr learned my addr 172.21.15.120:0/3861873436 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.639+0000 7f093e59c640 1 --2- 172.21.15.120:0/3861873436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c29d1e29a0 0x55c29d1ecb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.639+0000 7f093ed9d640 1 -- 172.21.15.120:0/3861873436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c29d1e32e0 msgr2=0x55c29d1ed0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.639+0000 7f0937fff640 1 --2- 172.21.15.120:0/3861873436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c29d1e32e0 0x55c29d1ed0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.639+0000 7f093ed9d640 1 --2- 172.21.15.120:0/3861873436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c29d1e32e0 0x55c29d1ed0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.639+0000 7f093ed9d640 1 -- 172.21.15.120:0/3861873436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c29d1e29a0 msgr2=0x55c29d1ecb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.639+0000 7f093ed9d640 1 --2- 172.21.15.120:0/3861873436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c29d1e29a0 0x55c29d1ecb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.639+0000 7f093ed9d640 1 -- 172.21.15.120:0/3861873436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c29ce59210 con 0x55c29d1e8370 2024-09-19T09:03:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.639+0000 7f093e59c640 1 --2- 172.21.15.120:0/3861873436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c29d1e29a0 0x55c29d1ecb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:03:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.639+0000 7f093ed9d640 1 --2- 172.21.15.120:0/3861873436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c29d1e8370 0x55c29d1f40f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0938035000 tx=0x7f09380095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a802963be3a7fbe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.639+0000 7f0925ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.639+0000 7f0925ffb640 1 -- 172.21.15.120:0/3861873436 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0938030b40 con 0x55c29d1e8370 2024-09-19T09:03:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.640+0000 7f0925ffb640 1 -- 172.21.15.120:0/3861873436 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f0938030ce0 con 0x55c29d1e8370 2024-09-19T09:03:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.640+0000 7f0947c57740 1 -- 172.21.15.120:0/3861873436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0930013950 con 0x55c29d1e8370 2024-09-19T09:03:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.640+0000 7f0925ffb640 1 -- 172.21.15.120:0/3861873436 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0938054480 con 0x55c29d1e8370 2024-09-19T09:03:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.640+0000 7f0947c57740 10 client.15366 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.640+0000 7f0947c57740 1 -- 172.21.15.120:0/3861873436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c29cf25620 con 0x55c29d1e8370 2024-09-19T09:03:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.640+0000 7f0925ffb640 1 -- 172.21.15.120:0/3861873436 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0930013950 con 0x55c29d1e8370 2024-09-19T09:03:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.640+0000 7f0925ffb640 1 -- 172.21.15.120:0/3861873436 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0938049520 con 0x55c29d1e8370 2024-09-19T09:03:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.640+0000 7f0947c57740 10 client.15366 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.640+0000 7f0947c57740 10 client.15366 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:03:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.640+0000 7f0947c57740 20 client.15366 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.640+0000 7f0947c57740 1 --2- 172.21.15.120:0/3861873436 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c29d1f8800 0x55c29d1facc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.640+0000 7f0947c57740 4 client.15366 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:03:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.640+0000 7f0947c57740 1 -- 172.21.15.120:0/3861873436 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c29d153a20 con 0x55c29d1f8800 2024-09-19T09:03:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.640+0000 7f093e59c640 1 --2- 172.21.15.120:0/3861873436 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c29d1f8800 0x55c29d1facc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.641+0000 7f0925ffb640 10 client.15366 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.641+0000 7f093e59c640 1 --2- 172.21.15.120:0/3861873436 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c29d1f8800 0x55c29d1facc0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7f47b98c7eed5b4 server_cookie=da1f25e2fd678cd8 in_seq=0 out_seq=0 2024-09-19T09:03:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.641+0000 7f0925ffb640 1 -- 172.21.15.120:0/3861873436 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c29d153a20 con 0x55c29d1f8800 2024-09-19T09:03:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.641+0000 7f0925ffb640 10 client.15366 handle_command_reply: tid=0 2024-09-19T09:03:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.641+0000 7f0947c57740 1 client.15366 shutdown 2024-09-19T09:03:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.642+0000 7f0947c57740 1 -- 172.21.15.120:0/3861873436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c29d1e8370 msgr2=0x55c29d1f40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.642+0000 7f0947c57740 1 --2- 172.21.15.120:0/3861873436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c29d1e8370 0x55c29d1f40f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0938035000 tx=0x7f09380095c0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.642+0000 7f0947c57740 1 -- 172.21.15.120:0/3861873436 shutdown_connections 2024-09-19T09:03:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.642+0000 7f0947c57740 1 --2- 172.21.15.120:0/3861873436 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c29d1f8800 0x55c29d1facc0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.642+0000 7f0947c57740 1 --2- 172.21.15.120:0/3861873436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c29d1e8370 0x55c29d1f40f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.642+0000 7f0925ffb640 0 client.15366 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.642+0000 7f0947c57740 1 --2- 172.21.15.120:0/3861873436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c29d1e32e0 0x55c29d1ed0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.642+0000 7f0947c57740 1 --2- 172.21.15.120:0/3861873436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c29d1e29a0 0x55c29d1ecb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.642+0000 7f0947c57740 1 -- 172.21.15.120:0/3861873436 >> 172.21.15.120:0/3861873436 conn(0x55c29d1de010 msgr2=0x55c29d1df820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.642+0000 7f0947c57740 1 -- 172.21.15.120:0/3861873436 shutdown_connections 2024-09-19T09:03:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.642+0000 7f0947c57740 1 -- 172.21.15.120:0/3861873436 wait complete. 2024-09-19T09:03:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.642+0000 7f0947c57740 20 client.15366 trim_cache size 0 max 16384 2024-09-19T09:03:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.656+0000 7f0947c57740 1 Processor -- start 2024-09-19T09:03:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.656+0000 7f0947c57740 1 -- start start 2024-09-19T09:03:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.656+0000 7f0947c57740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c29d1e29a0 0x55c29d1dd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.656+0000 7f0947c57740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c29d1e32e0 0x55c29d1f40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.656+0000 7f0947c57740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c29d1e8370 0x55c29d1ff530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.656+0000 7f0947c57740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f09401ab2a0 con 0x55c29d1e8370 2024-09-19T09:03:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.656+0000 7f0947c57740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f09401acde0 con 0x55c29d1e32e0 2024-09-19T09:03:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.656+0000 7f0947c57740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f09401ab970 con 0x55c29d1e29a0 2024-09-19T09:03:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.656+0000 7f093ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c29d1e8370 0x55c29d1ff530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.656+0000 7f093ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c29d1e8370 0x55c29d1ff530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56130/0 (socket says 172.21.15.120:56130) 2024-09-19T09:03:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.656+0000 7f093ed9d640 1 -- 172.21.15.120:0/3904031143 learned_addr learned my addr 172.21.15.120:0/3904031143 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.656+0000 7f0937fff640 1 --2- 172.21.15.120:0/3904031143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c29d1e32e0 0x55c29d1f40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.656+0000 7f093e59c640 1 --2- 172.21.15.120:0/3904031143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c29d1e29a0 0x55c29d1dd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.656+0000 7f093ed9d640 1 -- 172.21.15.120:0/3904031143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c29d1e29a0 msgr2=0x55c29d1dd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.656+0000 7f093ed9d640 1 --2- 172.21.15.120:0/3904031143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c29d1e29a0 0x55c29d1dd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.656+0000 7f093ed9d640 1 -- 172.21.15.120:0/3904031143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c29d1e32e0 msgr2=0x55c29d1f40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.656+0000 7f093ed9d640 1 --2- 172.21.15.120:0/3904031143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c29d1e32e0 0x55c29d1f40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.656+0000 7f093ed9d640 1 -- 172.21.15.120:0/3904031143 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c29d1e37c0 con 0x55c29d1e8370 2024-09-19T09:03:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.656+0000 7f0937fff640 1 --2- 172.21.15.120:0/3904031143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c29d1e32e0 0x55c29d1f40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:03:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.657+0000 7f093ed9d640 1 --2- 172.21.15.120:0/3904031143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c29d1e8370 0x55c29d1ff530 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0938009bf0 tx=0x7f0938031db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a881a15d0b0df5eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.657+0000 7f0927fff640 1 -- 172.21.15.120:0/3904031143 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0938031860 con 0x55c29d1e8370 2024-09-19T09:03:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.657+0000 7f0927fff640 1 -- 172.21.15.120:0/3904031143 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0938036070 con 0x55c29d1e8370 2024-09-19T09:03:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.657+0000 7f0927fff640 1 -- 172.21.15.120:0/3904031143 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0938054620 con 0x55c29d1e8370 2024-09-19T09:03:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.657+0000 7f0947c57740 1 -- 172.21.15.120:0/3904031143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c29d1e8370 msgr2=0x55c29d1ff530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.657+0000 7f0947c57740 1 --2- 172.21.15.120:0/3904031143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c29d1e8370 0x55c29d1ff530 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0938009bf0 tx=0x7f0938031db0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.657+0000 7f0947c57740 1 -- 172.21.15.120:0/3904031143 shutdown_connections 2024-09-19T09:03:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.657+0000 7f0947c57740 1 --2- 172.21.15.120:0/3904031143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c29d1e8370 0x55c29d1ff530 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.657+0000 7f0947c57740 1 --2- 172.21.15.120:0/3904031143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c29d1e32e0 0x55c29d1f40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.657+0000 7f0947c57740 1 --2- 172.21.15.120:0/3904031143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c29d1e29a0 0x55c29d1dd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.657+0000 7f0947c57740 1 -- 172.21.15.120:0/3904031143 >> 172.21.15.120:0/3904031143 conn(0x55c29d1de010 msgr2=0x55c29d1dc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.658+0000 7f0947c57740 1 -- 172.21.15.120:0/3904031143 shutdown_connections 2024-09-19T09:03:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.658+0000 7f0947c57740 1 -- 172.21.15.120:0/3904031143 wait complete. 2024-09-19T09:03:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.658+0000 7f0947c57740 1 -- start start 2024-09-19T09:03:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.658+0000 7f0947c57740 1 Processor -- start 2024-09-19T09:03:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.658+0000 7f0947c57740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c29d1e29a0 0x55c29d1f1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.658+0000 7f0947c57740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c29d1e32e0 0x55c29d1f1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.658+0000 7f0947c57740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c29d1e8370 0x55c29d1f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.658+0000 7f0947c57740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f09401ac710 con 0x55c29d1e32e0 2024-09-19T09:03:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.658+0000 7f0947c57740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f09401ac040 con 0x55c29d1e29a0 2024-09-19T09:03:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.658+0000 7f0947c57740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f09401a9260 con 0x55c29d1e8370 2024-09-19T09:03:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.658+0000 7f0947c57740 10 client.0 _ll_register_callbacks cb 0x7f0938031f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.658+0000 7f0937fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c29d1e32e0 0x55c29d1f1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.658+0000 7f0937fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c29d1e32e0 0x55c29d1f1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56146/0 (socket says 172.21.15.120:56146) 2024-09-19T09:03:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.658+0000 7f0937fff640 1 -- 172.21.15.120:0/3082669132 learned_addr learned my addr 172.21.15.120:0/3082669132 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.658+0000 7f093ed9d640 1 --2- 172.21.15.120:0/3082669132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c29d1e8370 0x55c29d1f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.658+0000 7f0937fff640 1 -- 172.21.15.120:0/3082669132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c29d1e8370 msgr2=0x55c29d1f2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.658+0000 7f0937fff640 1 --2- 172.21.15.120:0/3082669132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c29d1e8370 0x55c29d1f2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.658+0000 7f0937fff640 1 -- 172.21.15.120:0/3082669132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c29d1e29a0 msgr2=0x55c29d1f1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.658+0000 7f0937fff640 1 --2- 172.21.15.120:0/3082669132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c29d1e29a0 0x55c29d1f1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.658+0000 7f0937fff640 1 -- 172.21.15.120:0/3082669132 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c29d1f32a0 con 0x55c29d1e32e0 2024-09-19T09:03:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.658+0000 7f093ed9d640 1 --2- 172.21.15.120:0/3082669132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c29d1e8370 0x55c29d1f2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:03:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.659+0000 7f0937fff640 1 --2- 172.21.15.120:0/3082669132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c29d1e32e0 0x55c29d1f1d60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0928009610 tx=0x7f092809b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e1630af0d5f070 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.659+0000 7f0925ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.659+0000 7f0925ffb640 1 -- 172.21.15.120:0/3082669132 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928020040 con 0x55c29d1e32e0 2024-09-19T09:03:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.659+0000 7f0925ffb640 1 -- 172.21.15.120:0/3082669132 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f09280035f0 con 0x55c29d1e32e0 2024-09-19T09:03:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.659+0000 7f0947c57740 1 -- 172.21.15.120:0/3082669132 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0938036070 con 0x55c29d1e32e0 2024-09-19T09:03:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.659+0000 7f0925ffb640 1 -- 172.21.15.120:0/3082669132 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f09280a4ac0 con 0x55c29d1e32e0 2024-09-19T09:03:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.659+0000 7f0947c57740 10 client.15372 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.659+0000 7f0947c57740 1 -- 172.21.15.120:0/3082669132 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c29d2635e0 con 0x55c29d1e32e0 2024-09-19T09:03:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.659+0000 7f0925ffb640 1 -- 172.21.15.120:0/3082669132 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0938036070 con 0x55c29d1e32e0 2024-09-19T09:03:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.659+0000 7f0925ffb640 1 -- 172.21.15.120:0/3082669132 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f092809b350 con 0x55c29d1e32e0 2024-09-19T09:03:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.659+0000 7f0947c57740 10 client.15372 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.659+0000 7f0947c57740 10 client.15372 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:03:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.659+0000 7f0947c57740 20 client.15372 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.659+0000 7f0947c57740 1 --2- 172.21.15.120:0/3082669132 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c29d1f8800 0x55c29d265fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.659+0000 7f0947c57740 4 client.15372 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:03:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.659+0000 7f0947c57740 1 -- 172.21.15.120:0/3082669132 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c29d14dd70 con 0x55c29d1f8800 2024-09-19T09:03:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.660+0000 7f093e59c640 1 --2- 172.21.15.120:0/3082669132 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c29d1f8800 0x55c29d265fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.660+0000 7f093e59c640 1 --2- 172.21.15.120:0/3082669132 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c29d1f8800 0x55c29d265fa0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faf3f268df93a0e7 server_cookie=cf875d24c1cdff69 in_seq=0 out_seq=0 2024-09-19T09:03:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.660+0000 7f0925ffb640 10 client.15372 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.670+0000 7f0925ffb640 1 -- 172.21.15.120:0/3082669132 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c29d14dd70 con 0x55c29d1f8800 2024-09-19T09:03:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.670+0000 7f0925ffb640 10 client.15372 handle_command_reply: tid=0 2024-09-19T09:03:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.670+0000 7f0947c57740 1 client.15372 shutdown 2024-09-19T09:03:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.670+0000 7f0947c57740 1 -- 172.21.15.120:0/3082669132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c29d1e32e0 msgr2=0x55c29d1f1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.670+0000 7f0947c57740 1 --2- 172.21.15.120:0/3082669132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c29d1e32e0 0x55c29d1f1d60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0928009610 tx=0x7f092809b860 comp rx=0 tx=0).stop 2024-09-19T09:03:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.670+0000 7f0947c57740 1 -- 172.21.15.120:0/3082669132 shutdown_connections 2024-09-19T09:03:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.671+0000 7f0947c57740 1 --2- 172.21.15.120:0/3082669132 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c29d1f8800 0x55c29d265fa0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.671+0000 7f0947c57740 1 --2- 172.21.15.120:0/3082669132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c29d1e8370 0x55c29d1f2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.671+0000 7f0947c57740 1 --2- 172.21.15.120:0/3082669132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c29d1e32e0 0x55c29d1f1d60 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.671+0000 7f0947c57740 1 --2- 172.21.15.120:0/3082669132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c29d1e29a0 0x55c29d1f1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.671+0000 7f0925ffb640 0 client.15372 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.671+0000 7f0947c57740 1 -- 172.21.15.120:0/3082669132 >> 172.21.15.120:0/3082669132 conn(0x55c29d1de010 msgr2=0x55c29d1df6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.671+0000 7f0947c57740 1 -- 172.21.15.120:0/3082669132 shutdown_connections 2024-09-19T09:03:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.671+0000 7f0947c57740 1 -- 172.21.15.120:0/3082669132 wait complete. 2024-09-19T09:03:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.671+0000 7f0947c57740 20 client.15372 trim_cache size 0 max 16384 2024-09-19T09:03:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:03:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:03:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:03:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:03:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.671+0000 7f0944de4640 1 -- 172.21.15.120:0/1982858665 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f092c0a3390 msgr2=0x7f092c0a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.671+0000 7f0944de4640 1 --2- 172.21.15.120:0/1982858665 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f092c0a3390 0x7f092c0a5850 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0938009430 tx=0x7f0938005d80 comp rx=0 tx=0).stop 2024-09-19T09:03:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.671+0000 7f0944de4640 1 -- 172.21.15.120:0/1982858665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0940101c30 msgr2=0x7f0940109a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.671+0000 7f0944de4640 1 --2- 172.21.15.120:0/1982858665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0940101c30 0x7f0940109a00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0928005fd0 tx=0x7f092800dde0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.671+0000 7f0944de4640 1 -- 172.21.15.120:0/1982858665 shutdown_connections 2024-09-19T09:03:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.671+0000 7f0944de4640 1 --2- 172.21.15.120:0/1982858665 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f092c0a3390 0x7f092c0a5850 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.671+0000 7f0944de4640 1 --2- 172.21.15.120:0/1982858665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09401025f0 0x7f0940109f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.671+0000 7f0944de4640 1 --2- 172.21.15.120:0/1982858665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0940101c30 0x7f0940109a00 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.671+0000 7f0944de4640 1 --2- 172.21.15.120:0/1982858665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0940100a40 0x7f094010b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.671+0000 7f0944de4640 1 -- 172.21.15.120:0/1982858665 >> 172.21.15.120:0/1982858665 conn(0x7f09400fc1e0 msgr2=0x7f0940104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.671+0000 7f0944de4640 1 -- 172.21.15.120:0/1982858665 shutdown_connections 2024-09-19T09:03:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:02.671+0000 7f0944de4640 1 -- 172.21.15.120:0/1982858665 wait complete. 2024-09-19T09:03:02.829 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:03:02.949 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:02 smithi120 ceph-mon[31150]: pgmap v346: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-19T09:03:02.949 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:02 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2206060800' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:03:03.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:02 smithi196 ceph-mon[35150]: pgmap v346: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-19T09:03:03.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:02 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2206060800' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:03:03.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:02 smithi153 ceph-mon[35071]: pgmap v346: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-19T09:03:03.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:02 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2206060800' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:03:04.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:03 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:03:04.102 INFO:teuthology.orchestra.run.smithi120.stdout:{"run_dir":"/var/run/ceph"} 2024-09-19T09:03:04.102 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-19T09:03:04.102 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 1747 2024-09-19T09:03:04.102 DEBUG:tasks.quiescer:Yielding 2024-09-19T09:03:04.103 INFO:teuthology.run_tasks:Running task workunit... 2024-09-19T09:03:04.116 INFO:tasks.workunit:Pulling workunits from ref 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T09:03:04.116 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-19T09:03:04.116 INFO:tasks.workunit:timeout=3h 2024-09-19T09:03:04.116 INFO:tasks.workunit:cleanup=True 2024-09-19T09:03:04.117 DEBUG:teuthology.orchestra.run.smithi120:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-19T09:03:04.117 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:03:04.133 INFO:teuthology.orchestra.run.smithi120.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-19T09:03:04.133 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-19T09:03:04.133 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 6ch/108d Inode: 1099511628283 Links: 2 2024-09-19T09:03:04.134 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T09:03:04.134 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-19T09:03:04.134 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-09-19 09:01:06.079351395 +0000 2024-09-19T09:03:04.134 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-09-19 09:01:06.079351395 +0000 2024-09-19T09:03:04.134 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-09-19 09:02:40.541632092 +0000 2024-09-19T09:03:04.134 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-09-19 09:01:06.079351395 +0000 2024-09-19T09:03:04.134 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-19T09:03:04.134 DEBUG:teuthology.orchestra.run.smithi120:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-19T09:03:04.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:03 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:03:04.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:03 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:03:04.208 DEBUG:teuthology.orchestra.run.smithi120:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T09:03:04.264 INFO:tasks.workunit.client.0.smithi120.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-19T09:03:04.369 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:03:04.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.919+0000 7f0bf33bf640 1 -- 172.21.15.120:0/2512368007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bec101e10 msgr2=0x7f0bec1021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:04.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.919+0000 7f0bf33bf640 1 --2- 172.21.15.120:0/2512368007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bec101e10 0x7f0bec1021f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0be8009900 tx=0x7f0be802a410 comp rx=0 tx=0).stop 2024-09-19T09:03:04.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.919+0000 7f0bf33bf640 1 -- 172.21.15.120:0/2512368007 shutdown_connections 2024-09-19T09:03:04.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.919+0000 7f0bf33bf640 1 --2- 172.21.15.120:0/2512368007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bec106f40 0x7f0bec10eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:04.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.919+0000 7f0bf33bf640 1 --2- 172.21.15.120:0/2512368007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0bec1027c0 0x7f0bec106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:04.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.919+0000 7f0bf33bf640 1 --2- 172.21.15.120:0/2512368007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bec101e10 0x7f0bec1021f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:04.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.919+0000 7f0bf33bf640 1 -- 172.21.15.120:0/2512368007 >> 172.21.15.120:0/2512368007 conn(0x7f0bec0fd8d0 msgr2=0x7f0bec0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:04.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.919+0000 7f0bf33bf640 1 -- 172.21.15.120:0/2512368007 shutdown_connections 2024-09-19T09:03:04.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.919+0000 7f0bf33bf640 1 -- 172.21.15.120:0/2512368007 wait complete. 2024-09-19T09:03:04.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.920+0000 7f0bf33bf640 1 Processor -- start 2024-09-19T09:03:04.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.920+0000 7f0bf33bf640 1 -- start start 2024-09-19T09:03:04.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.920+0000 7f0bf33bf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bec101e10 0x7f0bec198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:04.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.920+0000 7f0bf33bf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0bec1027c0 0x7f0bec198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:04.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.920+0000 7f0bf33bf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bec106f40 0x7f0bec19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:04.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.920+0000 7f0bf33bf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0bec111fa0 con 0x7f0bec101e10 2024-09-19T09:03:04.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.920+0000 7f0bf33bf640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0bec111e20 con 0x7f0bec1027c0 2024-09-19T09:03:04.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.920+0000 7f0bf33bf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0bec112120 con 0x7f0bec106f40 2024-09-19T09:03:04.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.920+0000 7f0bf115b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bec101e10 0x7f0bec198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:04.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.920+0000 7f0bf115b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bec101e10 0x7f0bec198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56168/0 (socket says 172.21.15.120:56168) 2024-09-19T09:03:04.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.920+0000 7f0bf115b640 1 -- 172.21.15.120:0/2389056746 learned_addr learned my addr 172.21.15.120:0/2389056746 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:04.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.920+0000 7f0bf095a640 1 --2- 172.21.15.120:0/2389056746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0bec1027c0 0x7f0bec198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:04.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.920+0000 7f0bf115b640 1 -- 172.21.15.120:0/2389056746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bec106f40 msgr2=0x7f0bec19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:04.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.920+0000 7f0bf195c640 1 --2- 172.21.15.120:0/2389056746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bec106f40 0x7f0bec19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:04.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.920+0000 7f0bf115b640 1 --2- 172.21.15.120:0/2389056746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bec106f40 0x7f0bec19ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:04.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.920+0000 7f0bf115b640 1 -- 172.21.15.120:0/2389056746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0bec1027c0 msgr2=0x7f0bec198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:04.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.920+0000 7f0bf115b640 1 --2- 172.21.15.120:0/2389056746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0bec1027c0 0x7f0bec198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:04.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.920+0000 7f0bf115b640 1 -- 172.21.15.120:0/2389056746 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bec19d3d0 con 0x7f0bec101e10 2024-09-19T09:03:04.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.920+0000 7f0bf095a640 1 --2- 172.21.15.120:0/2389056746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0bec1027c0 0x7f0bec198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:03:04.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.921+0000 7f0bf115b640 1 --2- 172.21.15.120:0/2389056746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bec101e10 0x7f0bec198420 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0be8006010 tx=0x7f0be80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:04.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.921+0000 7f0be27fc640 1 -- 172.21.15.120:0/2389056746 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be803f030 con 0x7f0bec101e10 2024-09-19T09:03:04.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.921+0000 7f0bf33bf640 1 -- 172.21.15.120:0/2389056746 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bec19d660 con 0x7f0bec101e10 2024-09-19T09:03:04.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.921+0000 7f0be27fc640 1 -- 172.21.15.120:0/2389056746 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0be8003ce0 con 0x7f0bec101e10 2024-09-19T09:03:04.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.921+0000 7f0be27fc640 1 -- 172.21.15.120:0/2389056746 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be8035070 con 0x7f0bec101e10 2024-09-19T09:03:04.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.921+0000 7f0bf33bf640 1 -- 172.21.15.120:0/2389056746 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bec1a4fa0 con 0x7f0bec101e10 2024-09-19T09:03:04.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.922+0000 7f0bf33bf640 1 -- 172.21.15.120:0/2389056746 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bec1032f0 con 0x7f0bec101e10 2024-09-19T09:03:04.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.923+0000 7f0be27fc640 1 -- 172.21.15.120:0/2389056746 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f0be8042030 con 0x7f0bec101e10 2024-09-19T09:03:04.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.924+0000 7f0be27fc640 1 --2- 172.21.15.120:0/2389056746 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0bd80a32d0 0x7f0bd80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:04.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.924+0000 7f0bf095a640 1 --2- 172.21.15.120:0/2389056746 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0bd80a32d0 0x7f0bd80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:04.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.924+0000 7f0be27fc640 1 -- 172.21.15.120:0/2389056746 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f0be80efc10 con 0x7f0bec101e10 2024-09-19T09:03:04.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.924+0000 7f0bf095a640 1 --2- 172.21.15.120:0/2389056746 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0bd80a32d0 0x7f0bd80a5790 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0bec108800 tx=0x7f0bdc005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:04.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:04.928+0000 7f0be27fc640 1 -- 172.21.15.120:0/2389056746 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0be80bc890 con 0x7f0bec101e10 2024-09-19T09:03:05.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:04 smithi120 ceph-mon[31150]: pgmap v347: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-19T09:03:05.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:05.136+0000 7f0bf33bf640 1 -- 172.21.15.120:0/2389056746 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0bec10a120 con 0x7f0bec101e10 2024-09-19T09:03:05.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:05.137+0000 7f0be27fc640 1 -- 172.21.15.120:0/2389056746 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f0be80bc190 con 0x7f0bec101e10 2024-09-19T09:03:05.137 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:03:05.137 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":28,"btime":"2024-09-19T09:02:41:696728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:02:41.207357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:03:05.138 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 28 2024-09-19T09:03:05.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:05.139+0000 7f0bf33bf640 1 -- 172.21.15.120:0/2389056746 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0bd80a32d0 msgr2=0x7f0bd80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:05.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:05.139+0000 7f0bf33bf640 1 --2- 172.21.15.120:0/2389056746 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0bd80a32d0 0x7f0bd80a5790 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0bec108800 tx=0x7f0bdc005e50 comp rx=0 tx=0).stop 2024-09-19T09:03:05.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:05.139+0000 7f0bf33bf640 1 -- 172.21.15.120:0/2389056746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bec101e10 msgr2=0x7f0bec198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:05.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:05.139+0000 7f0bf33bf640 1 --2- 172.21.15.120:0/2389056746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bec101e10 0x7f0bec198420 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0be8006010 tx=0x7f0be80042f0 comp rx=0 tx=0).stop 2024-09-19T09:03:05.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:05.139+0000 7f0bf33bf640 1 -- 172.21.15.120:0/2389056746 shutdown_connections 2024-09-19T09:03:05.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:05.139+0000 7f0bf33bf640 1 --2- 172.21.15.120:0/2389056746 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0bd80a32d0 0x7f0bd80a5790 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:05.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:05.139+0000 7f0bf33bf640 1 --2- 172.21.15.120:0/2389056746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bec106f40 0x7f0bec19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:05.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:05.139+0000 7f0bf33bf640 1 --2- 172.21.15.120:0/2389056746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0bec1027c0 0x7f0bec198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:05.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:05.139+0000 7f0bf33bf640 1 --2- 172.21.15.120:0/2389056746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bec101e10 0x7f0bec198420 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:05.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:05.139+0000 7f0bf33bf640 1 -- 172.21.15.120:0/2389056746 >> 172.21.15.120:0/2389056746 conn(0x7f0bec0fd8d0 msgr2=0x7f0bec0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:05.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:05.139+0000 7f0bf33bf640 1 -- 172.21.15.120:0/2389056746 shutdown_connections 2024-09-19T09:03:05.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:05.140+0000 7f0bf33bf640 1 -- 172.21.15.120:0/2389056746 wait complete. 2024-09-19T09:03:05.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:04 smithi196 ceph-mon[35150]: pgmap v347: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-19T09:03:05.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:04 smithi153 ceph-mon[35071]: pgmap v347: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-19T09:03:05.953 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:05 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2389056746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:03:05.980 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T09:00:47.659717+0000', 'modified': '2024-09-19T09:02:41.207357+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24461, 'mds_1': 24446, 'mds_2': 24442, 'mds_3': 24430, 'mds_4': 24421}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14601': {'gid': 14601, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.120:6839/1939272010', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6838', 'nonce': 1939272010}, {'type': 'v1', 'addr': '172.21.15.120:6839', 'nonce': 1939272010}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14613': {'gid': 14613, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.120:6841/658447478', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6840', 'nonce': 658447478}, {'type': 'v1', 'addr': '172.21.15.120:6841', 'nonce': 658447478}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24392': {'gid': 24392, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.153:6833/1522304272', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6832', 'nonce': 1522304272}, {'type': 'v1', 'addr': '172.21.15.153:6833', 'nonce': 1522304272}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24400': {'gid': 24400, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6833/1407049222', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 1407049222}, {'type': 'v1', 'addr': '172.21.15.196:6833', 'nonce': 1407049222}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24419': {'gid': 24419, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.153:6835/3103515283', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6834', 'nonce': 3103515283}, {'type': 'v1', 'addr': '172.21.15.153:6835', 'nonce': 3103515283}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.196:6835/2141168828', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 2141168828}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 2141168828}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.196:6837/776889709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6836', 'nonce': 776889709}, {'type': 'v1', 'addr': '172.21.15.196:6837', 'nonce': 776889709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.196:6839/862542635', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6838', 'nonce': 862542635}, {'type': 'v1', 'addr': '172.21.15.196:6839', 'nonce': 862542635}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.153:6837/3048520124', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6836', 'nonce': 3048520124}, {'type': 'v1', 'addr': '172.21.15.153:6837', 'nonce': 3048520124}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.153:6839/3290519661', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6838', 'nonce': 3290519661}, {'type': 'v1', 'addr': '172.21.15.153:6839', 'nonce': 3290519661}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24461, 'qdb_cluster': [24421, 24430, 24442, 24446, 24461]} 2024-09-19T09:03:05.981 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T09:03:05.981 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-19T09:03:06.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:05 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2389056746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:03:06.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:05 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2389056746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:03:06.265 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:03:06.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.810+0000 7fd94c982640 1 -- 172.21.15.120:0/3948856323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd948106a80 msgr2=0x7fd948106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:06.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.810+0000 7fd94c982640 1 --2- 172.21.15.120:0/3948856323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd948106a80 0x7fd948106e60 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd938009930 tx=0x7fd93802a420 comp rx=0 tx=0).stop 2024-09-19T09:03:06.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.810+0000 7fd94c982640 1 -- 172.21.15.120:0/3948856323 shutdown_connections 2024-09-19T09:03:06.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.810+0000 7fd94c982640 1 --2- 172.21.15.120:0/3948856323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd948101420 0x7fd94810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.810+0000 7fd94c982640 1 --2- 172.21.15.120:0/3948856323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd948100a80 0x7fd948100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.810+0000 7fd94c982640 1 --2- 172.21.15.120:0/3948856323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd948106a80 0x7fd948106e60 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.810+0000 7fd94c982640 1 -- 172.21.15.120:0/3948856323 >> 172.21.15.120:0/3948856323 conn(0x7fd9480fc7a0 msgr2=0x7fd9480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:06.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.810+0000 7fd94c982640 1 -- 172.21.15.120:0/3948856323 shutdown_connections 2024-09-19T09:03:06.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.810+0000 7fd94c982640 1 -- 172.21.15.120:0/3948856323 wait complete. 2024-09-19T09:03:06.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.811+0000 7fd94c982640 1 Processor -- start 2024-09-19T09:03:06.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.811+0000 7fd94c982640 1 -- start start 2024-09-19T09:03:06.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.811+0000 7fd94c982640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd948100a80 0x7fd948198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:06.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.811+0000 7fd94c982640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd948101420 0x7fd948198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:06.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.811+0000 7fd94c982640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd948106a80 0x7fd94819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:06.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.811+0000 7fd94c982640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd948111fa0 con 0x7fd948106a80 2024-09-19T09:03:06.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.811+0000 7fd94c982640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd948111e20 con 0x7fd948101420 2024-09-19T09:03:06.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.811+0000 7fd94c982640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd948112120 con 0x7fd948100a80 2024-09-19T09:03:06.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.811+0000 7fd946d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd948106a80 0x7fd94819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:06.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.811+0000 7fd946d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd948106a80 0x7fd94819cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56184/0 (socket says 172.21.15.120:56184) 2024-09-19T09:03:06.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.811+0000 7fd946d9d640 1 -- 172.21.15.120:0/2471276996 learned_addr learned my addr 172.21.15.120:0/2471276996 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:06.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.811+0000 7fd94659c640 1 --2- 172.21.15.120:0/2471276996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd948100a80 0x7fd948198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:06.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.812+0000 7fd945d9b640 1 --2- 172.21.15.120:0/2471276996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd948101420 0x7fd948198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:06.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.812+0000 7fd946d9d640 1 -- 172.21.15.120:0/2471276996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd948100a80 msgr2=0x7fd948198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:06.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.812+0000 7fd946d9d640 1 --2- 172.21.15.120:0/2471276996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd948100a80 0x7fd948198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.812+0000 7fd946d9d640 1 -- 172.21.15.120:0/2471276996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd948101420 msgr2=0x7fd948198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:06.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.812+0000 7fd946d9d640 1 --2- 172.21.15.120:0/2471276996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd948101420 0x7fd948198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.812+0000 7fd946d9d640 1 -- 172.21.15.120:0/2471276996 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94819d4d0 con 0x7fd948106a80 2024-09-19T09:03:06.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.812+0000 7fd946d9d640 1 --2- 172.21.15.120:0/2471276996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd948106a80 0x7fd94819cdf0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd94000eaa0 tx=0x7fd94000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:06.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.812+0000 7fd9377fe640 1 -- 172.21.15.120:0/2471276996 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94000cdb0 con 0x7fd948106a80 2024-09-19T09:03:06.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.812+0000 7fd9377fe640 1 -- 172.21.15.120:0/2471276996 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd940005020 con 0x7fd948106a80 2024-09-19T09:03:06.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.812+0000 7fd94c982640 1 -- 172.21.15.120:0/2471276996 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd94819d7c0 con 0x7fd948106a80 2024-09-19T09:03:06.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.812+0000 7fd9377fe640 1 -- 172.21.15.120:0/2471276996 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9400116b0 con 0x7fd948106a80 2024-09-19T09:03:06.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.812+0000 7fd94c982640 1 -- 172.21.15.120:0/2471276996 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9481a50a0 con 0x7fd948106a80 2024-09-19T09:03:06.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.814+0000 7fd9377fe640 1 -- 172.21.15.120:0/2471276996 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7fd940005560 con 0x7fd948106a80 2024-09-19T09:03:06.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.815+0000 7fd94ed5b740 1 Processor -- start 2024-09-19T09:03:06.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.815+0000 7fd94ed5b740 1 -- start start 2024-09-19T09:03:06.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.815+0000 7fd94ed5b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e05ca389d0 0x55e05ca38db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:06.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.815+0000 7fd94ed5b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e05ca392f0 0x55e05ca43670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:06.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.815+0000 7fd94ed5b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e05ca43bb0 0x55e05ca45fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:06.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.815+0000 7fd94ed5b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e05c99f090 con 0x55e05ca392f0 2024-09-19T09:03:06.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.815+0000 7fd94ed5b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e05c946a70 con 0x55e05ca389d0 2024-09-19T09:03:06.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.815+0000 7fd94ed5b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e05c99b050 con 0x55e05ca43bb0 2024-09-19T09:03:06.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.815+0000 7fd94659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e05ca392f0 0x55e05ca43670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:06.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.815+0000 7fd94659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e05ca392f0 0x55e05ca43670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56194/0 (socket says 172.21.15.120:56194) 2024-09-19T09:03:06.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.815+0000 7fd94659c640 1 -- 172.21.15.120:0/1420247045 learned_addr learned my addr 172.21.15.120:0/1420247045 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:06.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.815+0000 7fd945d9b640 1 --2- 172.21.15.120:0/1420247045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e05ca43bb0 0x55e05ca45fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:06.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.815+0000 7fd94659c640 1 -- 172.21.15.120:0/1420247045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e05ca43bb0 msgr2=0x55e05ca45fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:06.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.815+0000 7fd945d9b640 1 --2- 172.21.15.120:0/1420247045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e05ca389d0 0x55e05ca38db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:06.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.815+0000 7fd94659c640 1 --2- 172.21.15.120:0/1420247045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e05ca43bb0 0x55e05ca45fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.815+0000 7fd94659c640 1 -- 172.21.15.120:0/1420247045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e05ca389d0 msgr2=0x55e05ca38db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:06.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.815+0000 7fd94659c640 1 --2- 172.21.15.120:0/1420247045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e05ca389d0 0x55e05ca38db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.815+0000 7fd9377fe640 1 --2- 172.21.15.120:0/2471276996 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd9300a32d0 0x7fd9300a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:06.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.815+0000 7fd94659c640 1 -- 172.21.15.120:0/1420247045 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e05c953130 con 0x55e05ca392f0 2024-09-19T09:03:06.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.816+0000 7fd946d9d640 1 --2- 172.21.15.120:0/2471276996 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd9300a32d0 0x7fd9300a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:06.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.816+0000 7fd94659c640 1 --2- 172.21.15.120:0/1420247045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e05ca392f0 0x55e05ca43670 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd938009930 tx=0x7fd938031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2dc672f1c8638f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:06.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.816+0000 7fd9377fe640 1 -- 172.21.15.120:0/2471276996 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fd9400cfd20 con 0x7fd948106a80 2024-09-19T09:03:06.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.816+0000 7fd934ff9640 1 -- 172.21.15.120:0/1420247045 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd938040070 con 0x55e05ca392f0 2024-09-19T09:03:06.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.816+0000 7fd934ff9640 1 -- 172.21.15.120:0/1420247045 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd938005020 con 0x55e05ca392f0 2024-09-19T09:03:06.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.816+0000 7fd934ff9640 1 -- 172.21.15.120:0/1420247045 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd938035040 con 0x55e05ca392f0 2024-09-19T09:03:06.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.817+0000 7fd94ed5b740 1 -- 172.21.15.120:0/1420247045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e05ca392f0 msgr2=0x55e05ca43670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:06.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.817+0000 7fd946d9d640 1 --2- 172.21.15.120:0/2471276996 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd9300a32d0 0x7fd9300a5790 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd940098bb0 tx=0x7fd9400c6fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:06.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.817+0000 7fd94ed5b740 1 --2- 172.21.15.120:0/1420247045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e05ca392f0 0x55e05ca43670 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd938009930 tx=0x7fd938031630 comp rx=0 tx=0).stop 2024-09-19T09:03:06.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.817+0000 7fd94ed5b740 1 -- 172.21.15.120:0/1420247045 shutdown_connections 2024-09-19T09:03:06.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.817+0000 7fd94ed5b740 1 --2- 172.21.15.120:0/1420247045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e05ca43bb0 0x55e05ca45fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.817+0000 7fd94ed5b740 1 --2- 172.21.15.120:0/1420247045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e05ca392f0 0x55e05ca43670 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.817+0000 7fd94ed5b740 1 --2- 172.21.15.120:0/1420247045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e05ca389d0 0x55e05ca38db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.817+0000 7fd94ed5b740 1 -- 172.21.15.120:0/1420247045 >> 172.21.15.120:0/1420247045 conn(0x55e05ca34f60 msgr2=0x55e05ca35340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:06.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.817+0000 7fd94ed5b740 1 -- 172.21.15.120:0/1420247045 shutdown_connections 2024-09-19T09:03:06.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.817+0000 7fd94ed5b740 1 -- 172.21.15.120:0/1420247045 wait complete. 2024-09-19T09:03:06.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.817+0000 7fd94ed5b740 1 -- start start 2024-09-19T09:03:06.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.817+0000 7fd94ed5b740 1 Processor -- start 2024-09-19T09:03:06.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.818+0000 7fd94ed5b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e05ca389d0 0x55e05ca43350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:06.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.818+0000 7fd94ed5b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e05ca392f0 0x55e05ca406a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:06.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.818+0000 7fd94ed5b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e05ca43bb0 0x55e05ca40be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:06.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.818+0000 7fd94ed5b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e05ca46780 con 0x55e05ca392f0 2024-09-19T09:03:06.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.818+0000 7fd94ed5b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e05c959f20 con 0x55e05ca389d0 2024-09-19T09:03:06.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.818+0000 7fd94ed5b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e05ca46900 con 0x55e05ca43bb0 2024-09-19T09:03:06.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.818+0000 7fd94ed5b740 10 client.0 _ll_register_callbacks cb 0x55e05c923030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:06.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.818+0000 7fd94659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e05ca392f0 0x55e05ca406a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:06.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.818+0000 7fd94659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e05ca392f0 0x55e05ca406a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56204/0 (socket says 172.21.15.120:56204) 2024-09-19T09:03:06.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.818+0000 7fd94659c640 1 -- 172.21.15.120:0/3311231472 learned_addr learned my addr 172.21.15.120:0/3311231472 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:06.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.818+0000 7fd945d9b640 1 --2- 172.21.15.120:0/3311231472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e05ca389d0 0x55e05ca43350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:06.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.818+0000 7fd945d9b640 1 --2- 172.21.15.120:0/3311231472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e05ca43bb0 0x55e05ca40be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:06.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.818+0000 7fd94659c640 1 -- 172.21.15.120:0/3311231472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e05ca43bb0 msgr2=0x55e05ca40be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:06.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.818+0000 7fd94659c640 1 --2- 172.21.15.120:0/3311231472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e05ca43bb0 0x55e05ca40be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.818+0000 7fd94659c640 1 -- 172.21.15.120:0/3311231472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e05ca389d0 msgr2=0x55e05ca43350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:06.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.818+0000 7fd94659c640 1 --2- 172.21.15.120:0/3311231472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e05ca389d0 0x55e05ca43350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.818+0000 7fd94659c640 1 -- 172.21.15.120:0/3311231472 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e05c7102b0 con 0x55e05ca392f0 2024-09-19T09:03:06.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.818+0000 7fd945d9b640 1 --2- 172.21.15.120:0/3311231472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e05ca389d0 0x55e05ca43350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:03:06.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.818+0000 7fd94659c640 1 --2- 172.21.15.120:0/3311231472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e05ca392f0 0x55e05ca406a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd938005940 tx=0x7fd938009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41bfc6fc247c2d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:06.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.818+0000 7fd92dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:06.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.818+0000 7fd92dffb640 1 -- 172.21.15.120:0/3311231472 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd938040070 con 0x55e05ca392f0 2024-09-19T09:03:06.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.818+0000 7fd92dffb640 1 -- 172.21.15.120:0/3311231472 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fd9380054e0 con 0x55e05ca392f0 2024-09-19T09:03:06.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.818+0000 7fd94ed5b740 1 -- 172.21.15.120:0/3311231472 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd938005020 con 0x55e05ca392f0 2024-09-19T09:03:06.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.819+0000 7fd92dffb640 1 -- 172.21.15.120:0/3311231472 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd938035030 con 0x55e05ca392f0 2024-09-19T09:03:06.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.819+0000 7fd94ed5b740 10 client.15390 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:06.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.819+0000 7fd94ed5b740 1 -- 172.21.15.120:0/3311231472 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e05ca51f90 con 0x55e05ca392f0 2024-09-19T09:03:06.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.819+0000 7fd92dffb640 1 -- 172.21.15.120:0/3311231472 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd938005020 con 0x55e05ca392f0 2024-09-19T09:03:06.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.819+0000 7fd92dffb640 1 -- 172.21.15.120:0/3311231472 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd93803ca10 con 0x55e05ca392f0 2024-09-19T09:03:06.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.819+0000 7fd94ed5b740 10 client.15390 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:06.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.819+0000 7fd94ed5b740 10 client.15390 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T09:03:06.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.819+0000 7fd94ed5b740 20 client.15390 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:06.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.819+0000 7fd94ed5b740 1 --2- 172.21.15.120:0/3311231472 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55e05ca54120 0x55e05ca55590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:06.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.819+0000 7fd94ed5b740 4 client.15390 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:03:06.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.819+0000 7fd94ed5b740 1 -- 172.21.15.120:0/3311231472 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e05c9ac600 con 0x55e05ca54120 2024-09-19T09:03:06.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.819+0000 7fd945d9b640 1 --2- 172.21.15.120:0/3311231472 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55e05ca54120 0x55e05ca55590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:06.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.820+0000 7fd945d9b640 1 --2- 172.21.15.120:0/3311231472 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55e05ca54120 0x55e05ca55590 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=20acb7c953f99fea server_cookie=1caa2e48dee11b28 in_seq=0 out_seq=0 2024-09-19T09:03:06.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.820+0000 7fd92dffb640 10 client.15390 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:03:06.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.820+0000 7fd92dffb640 1 -- 172.21.15.120:0/3311231472 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e05c9ac600 con 0x55e05ca54120 2024-09-19T09:03:06.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.820+0000 7fd92dffb640 10 client.15390 handle_command_reply: tid=0 2024-09-19T09:03:06.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.820+0000 7fd94ed5b740 1 client.15390 shutdown 2024-09-19T09:03:06.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.820+0000 7fd94ed5b740 1 -- 172.21.15.120:0/3311231472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e05ca392f0 msgr2=0x55e05ca406a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:06.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.820+0000 7fd94ed5b740 1 --2- 172.21.15.120:0/3311231472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e05ca392f0 0x55e05ca406a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd938005940 tx=0x7fd938009660 comp rx=0 tx=0).stop 2024-09-19T09:03:06.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.821+0000 7fd94ed5b740 1 -- 172.21.15.120:0/3311231472 shutdown_connections 2024-09-19T09:03:06.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.821+0000 7fd94ed5b740 1 --2- 172.21.15.120:0/3311231472 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55e05ca54120 0x55e05ca55590 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.821+0000 7fd94ed5b740 1 --2- 172.21.15.120:0/3311231472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e05ca43bb0 0x55e05ca40be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.821+0000 7fd94ed5b740 1 --2- 172.21.15.120:0/3311231472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e05ca392f0 0x55e05ca406a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.821+0000 7fd92dffb640 0 client.15390 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:03:06.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.821+0000 7fd94ed5b740 1 --2- 172.21.15.120:0/3311231472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e05ca389d0 0x55e05ca43350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.821+0000 7fd94ed5b740 1 -- 172.21.15.120:0/3311231472 >> 172.21.15.120:0/3311231472 conn(0x55e05ca34f60 msgr2=0x55e05ca3ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:06.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.821+0000 7fd94ed5b740 1 -- 172.21.15.120:0/3311231472 shutdown_connections 2024-09-19T09:03:06.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.821+0000 7fd94ed5b740 1 -- 172.21.15.120:0/3311231472 wait complete. 2024-09-19T09:03:06.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.821+0000 7fd94ed5b740 20 client.15390 trim_cache size 0 max 16384 2024-09-19T09:03:06.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.838+0000 7fd94ed5b740 1 Processor -- start 2024-09-19T09:03:06.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.838+0000 7fd94ed5b740 1 -- start start 2024-09-19T09:03:06.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.839+0000 7fd94ed5b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e05ca389d0 0x55e05ca55590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:06.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.839+0000 7fd94ed5b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e05ca392f0 0x55e05ca3d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:06.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.839+0000 7fd94ed5b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e05ca43bb0 0x55e05ca3de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:06.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.839+0000 7fd94ed5b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e05ca538d0 con 0x55e05ca389d0 2024-09-19T09:03:06.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.839+0000 7fd94ed5b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e05ca53fa0 con 0x55e05ca392f0 2024-09-19T09:03:06.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.839+0000 7fd94ed5b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e05ca54670 con 0x55e05ca43bb0 2024-09-19T09:03:06.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.839+0000 7fd945d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e05ca389d0 0x55e05ca55590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:06.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.839+0000 7fd945d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e05ca389d0 0x55e05ca55590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56220/0 (socket says 172.21.15.120:56220) 2024-09-19T09:03:06.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.839+0000 7fd945d9b640 1 -- 172.21.15.120:0/1566078686 learned_addr learned my addr 172.21.15.120:0/1566078686 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:06.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.839+0000 7fd94659c640 1 --2- 172.21.15.120:0/1566078686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e05ca392f0 0x55e05ca3d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:06.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.839+0000 7fd945d9b640 1 --2- 172.21.15.120:0/1566078686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e05ca43bb0 0x55e05ca3de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:06.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.839+0000 7fd945d9b640 1 -- 172.21.15.120:0/1566078686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e05ca43bb0 msgr2=0x55e05ca3de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:06.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.839+0000 7fd945d9b640 1 --2- 172.21.15.120:0/1566078686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e05ca43bb0 0x55e05ca3de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.839+0000 7fd945d9b640 1 -- 172.21.15.120:0/1566078686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e05ca392f0 msgr2=0x55e05ca3d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:06.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.839+0000 7fd945d9b640 1 --2- 172.21.15.120:0/1566078686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e05ca392f0 0x55e05ca3d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.839+0000 7fd945d9b640 1 -- 172.21.15.120:0/1566078686 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e05ca39120 con 0x55e05ca389d0 2024-09-19T09:03:06.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.839+0000 7fd945d9b640 1 --2- 172.21.15.120:0/1566078686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e05ca389d0 0x55e05ca55590 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x55e05ca43b80 tx=0x7fd93c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c4e0fe480e61d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:06.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.839+0000 7fd92ffff640 1 -- 172.21.15.120:0/1566078686 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c00f040 con 0x55e05ca389d0 2024-09-19T09:03:06.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.840+0000 7fd92ffff640 1 -- 172.21.15.120:0/1566078686 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd93c01c030 con 0x55e05ca389d0 2024-09-19T09:03:06.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.840+0000 7fd92ffff640 1 -- 172.21.15.120:0/1566078686 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c0149f0 con 0x55e05ca389d0 2024-09-19T09:03:06.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.840+0000 7fd94ed5b740 1 -- 172.21.15.120:0/1566078686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e05ca389d0 msgr2=0x55e05ca55590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:06.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.840+0000 7fd94ed5b740 1 --2- 172.21.15.120:0/1566078686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e05ca389d0 0x55e05ca55590 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x55e05ca43b80 tx=0x7fd93c008040 comp rx=0 tx=0).stop 2024-09-19T09:03:06.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.840+0000 7fd94ed5b740 1 -- 172.21.15.120:0/1566078686 shutdown_connections 2024-09-19T09:03:06.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.840+0000 7fd94ed5b740 1 --2- 172.21.15.120:0/1566078686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e05ca43bb0 0x55e05ca3de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.840+0000 7fd94ed5b740 1 --2- 172.21.15.120:0/1566078686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e05ca392f0 0x55e05ca3d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.840+0000 7fd94ed5b740 1 --2- 172.21.15.120:0/1566078686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e05ca389d0 0x55e05ca55590 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.840+0000 7fd94ed5b740 1 -- 172.21.15.120:0/1566078686 >> 172.21.15.120:0/1566078686 conn(0x55e05ca34f60 msgr2=0x55e05ca42610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:06.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.840+0000 7fd94ed5b740 1 -- 172.21.15.120:0/1566078686 shutdown_connections 2024-09-19T09:03:06.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.840+0000 7fd94ed5b740 1 -- 172.21.15.120:0/1566078686 wait complete. 2024-09-19T09:03:06.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.841+0000 7fd94ed5b740 1 -- start start 2024-09-19T09:03:06.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.841+0000 7fd94ed5b740 1 Processor -- start 2024-09-19T09:03:06.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.841+0000 7fd94ed5b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e05ca389d0 0x55e05ca5da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:06.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.841+0000 7fd94ed5b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e05ca392f0 0x55e05ca5dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:06.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.841+0000 7fd94ed5b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e05ca43bb0 0x55e05ca56de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:06.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.841+0000 7fd94ed5b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e05ca55410 con 0x55e05ca392f0 2024-09-19T09:03:06.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.841+0000 7fd94ed5b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e05ca54d40 con 0x55e05ca389d0 2024-09-19T09:03:06.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.841+0000 7fd94ed5b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e05ca55ae0 con 0x55e05ca43bb0 2024-09-19T09:03:06.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.841+0000 7fd94ed5b740 10 client.0 _ll_register_callbacks cb 0x7fd938009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:06.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.841+0000 7fd94659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e05ca392f0 0x55e05ca5dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:06.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.841+0000 7fd94659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e05ca392f0 0x55e05ca5dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56236/0 (socket says 172.21.15.120:56236) 2024-09-19T09:03:06.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.841+0000 7fd94659c640 1 -- 172.21.15.120:0/3305685497 learned_addr learned my addr 172.21.15.120:0/3305685497 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:06.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.841+0000 7fd945d9b640 1 --2- 172.21.15.120:0/3305685497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e05ca389d0 0x55e05ca5da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:06.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.841+0000 7fd945d9b640 1 --2- 172.21.15.120:0/3305685497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e05ca43bb0 0x55e05ca56de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:06.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.841+0000 7fd94659c640 1 -- 172.21.15.120:0/3305685497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e05ca43bb0 msgr2=0x55e05ca56de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:06.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.841+0000 7fd94659c640 1 --2- 172.21.15.120:0/3305685497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e05ca43bb0 0x55e05ca56de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.841+0000 7fd94659c640 1 -- 172.21.15.120:0/3305685497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e05ca389d0 msgr2=0x55e05ca5da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:06.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.841+0000 7fd94659c640 1 --2- 172.21.15.120:0/3305685497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e05ca389d0 0x55e05ca5da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.841+0000 7fd94659c640 1 -- 172.21.15.120:0/3305685497 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e05ca57610 con 0x55e05ca392f0 2024-09-19T09:03:06.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.841+0000 7fd945d9b640 1 --2- 172.21.15.120:0/3305685497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e05ca43bb0 0x55e05ca56de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:03:06.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.842+0000 7fd94659c640 1 --2- 172.21.15.120:0/3305685497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e05ca392f0 0x55e05ca5dfa0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x55e05ca3ff30 tx=0x7fd938001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd175bac1cad9b1c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:06.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.842+0000 7fd92dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:06.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.842+0000 7fd92dffb640 1 -- 172.21.15.120:0/3305685497 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd938040040 con 0x55e05ca392f0 2024-09-19T09:03:06.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.842+0000 7fd92dffb640 1 -- 172.21.15.120:0/3305685497 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fd93803c540 con 0x55e05ca392f0 2024-09-19T09:03:06.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.842+0000 7fd94ed5b740 1 -- 172.21.15.120:0/3305685497 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd93c01c030 con 0x55e05ca392f0 2024-09-19T09:03:06.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.842+0000 7fd92dffb640 1 -- 172.21.15.120:0/3305685497 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd93803c740 con 0x55e05ca392f0 2024-09-19T09:03:06.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.842+0000 7fd94ed5b740 10 client.15396 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:06.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.842+0000 7fd94ed5b740 1 -- 172.21.15.120:0/3305685497 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e05ca5c040 con 0x55e05ca392f0 2024-09-19T09:03:06.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.842+0000 7fd92dffb640 1 -- 172.21.15.120:0/3305685497 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd93c01c030 con 0x55e05ca392f0 2024-09-19T09:03:06.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.842+0000 7fd92dffb640 1 -- 172.21.15.120:0/3305685497 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd938065050 con 0x55e05ca392f0 2024-09-19T09:03:06.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.842+0000 7fd94ed5b740 10 client.15396 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:06.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.842+0000 7fd94ed5b740 10 client.15396 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T09:03:06.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.842+0000 7fd94ed5b740 20 client.15396 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:06.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.842+0000 7fd94ed5b740 1 --2- 172.21.15.120:0/3305685497 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55e05cac2e00 0x55e05cac0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:06.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.842+0000 7fd94ed5b740 4 client.15396 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:03:06.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.842+0000 7fd94ed5b740 1 -- 172.21.15.120:0/3305685497 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "status"}) -- 0x55e05c9a6b10 con 0x55e05cac2e00 2024-09-19T09:03:06.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.842+0000 7fd945d9b640 1 --2- 172.21.15.120:0/3305685497 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55e05cac2e00 0x55e05cac0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:06.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.843+0000 7fd945d9b640 1 --2- 172.21.15.120:0/3305685497 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55e05cac2e00 0x55e05cac0bd0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=395cfb42297466dd server_cookie=5f8de3053a30d8b4 in_seq=0 out_seq=0 2024-09-19T09:03:06.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.843+0000 7fd92dffb640 10 client.15396 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:03:06.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.843+0000 7fd92dffb640 1 -- 172.21.15.120:0/3305685497 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e05c9a6b10 con 0x55e05cac2e00 2024-09-19T09:03:06.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.843+0000 7fd92dffb640 10 client.15396 handle_command_reply: tid=0 2024-09-19T09:03:06.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.843+0000 7fd94ed5b740 1 client.15396 shutdown 2024-09-19T09:03:06.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.843+0000 7fd94ed5b740 1 -- 172.21.15.120:0/3305685497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e05ca392f0 msgr2=0x55e05ca5dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:06.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.843+0000 7fd94ed5b740 1 --2- 172.21.15.120:0/3305685497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e05ca392f0 0x55e05ca5dfa0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x55e05ca3ff30 tx=0x7fd938001e50 comp rx=0 tx=0).stop 2024-09-19T09:03:06.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.843+0000 7fd94ed5b740 1 -- 172.21.15.120:0/3305685497 shutdown_connections 2024-09-19T09:03:06.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.843+0000 7fd94ed5b740 1 --2- 172.21.15.120:0/3305685497 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55e05cac2e00 0x55e05cac0bd0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.843+0000 7fd94ed5b740 1 --2- 172.21.15.120:0/3305685497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e05ca43bb0 0x55e05ca56de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.843+0000 7fd94ed5b740 1 --2- 172.21.15.120:0/3305685497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e05ca392f0 0x55e05ca5dfa0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.843+0000 7fd92dffb640 0 client.15396 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:03:06.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.843+0000 7fd94ed5b740 1 --2- 172.21.15.120:0/3305685497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e05ca389d0 0x55e05ca5da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.843+0000 7fd94ed5b740 1 -- 172.21.15.120:0/3305685497 >> 172.21.15.120:0/3305685497 conn(0x55e05ca34f60 msgr2=0x55e05ca41d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:06.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.843+0000 7fd94ed5b740 1 -- 172.21.15.120:0/3305685497 shutdown_connections 2024-09-19T09:03:06.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.843+0000 7fd94ed5b740 1 -- 172.21.15.120:0/3305685497 wait complete. 2024-09-19T09:03:06.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.843+0000 7fd94ed5b740 20 client.15396 trim_cache size 0 max 16384 2024-09-19T09:03:06.853 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:03:06.853 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_fsid": "507393d2-7664-11ef-bceb-c7b262605968", 2024-09-19T09:03:06.853 INFO:teuthology.orchestra.run.smithi120.stdout: "whoami": 4, 2024-09-19T09:03:06.853 INFO:teuthology.orchestra.run.smithi120.stdout: "id": 24421, 2024-09-19T09:03:06.853 INFO:teuthology.orchestra.run.smithi120.stdout: "want_state": "up:active", 2024-09-19T09:03:06.854 INFO:teuthology.orchestra.run.smithi120.stdout: "state": "up:active", 2024-09-19T09:03:06.854 INFO:teuthology.orchestra.run.smithi120.stdout: "fs_name": "cephfs", 2024-09-19T09:03:06.854 INFO:teuthology.orchestra.run.smithi120.stdout: "rank_uptime": 128.712989077, 2024-09-19T09:03:06.854 INFO:teuthology.orchestra.run.smithi120.stdout: "mdsmap_epoch": 28, 2024-09-19T09:03:06.854 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch": 81, 2024-09-19T09:03:06.854 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch_barrier": 81, 2024-09-19T09:03:06.854 INFO:teuthology.orchestra.run.smithi120.stdout: "uptime": 166.13952820399999 2024-09-19T09:03:06.854 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:03:06.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.844+0000 7fd94c982640 1 -- 172.21.15.120:0/2471276996 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd9300a32d0 msgr2=0x7fd9300a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:06.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.844+0000 7fd94c982640 1 --2- 172.21.15.120:0/2471276996 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd9300a32d0 0x7fd9300a5790 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd940098bb0 tx=0x7fd9400c6fb0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.844+0000 7fd94c982640 1 -- 172.21.15.120:0/2471276996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd948106a80 msgr2=0x7fd94819cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:06.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.844+0000 7fd94c982640 1 --2- 172.21.15.120:0/2471276996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd948106a80 0x7fd94819cdf0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd94000eaa0 tx=0x7fd94000ef60 comp rx=0 tx=0).stop 2024-09-19T09:03:06.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.844+0000 7fd94c982640 1 -- 172.21.15.120:0/2471276996 shutdown_connections 2024-09-19T09:03:06.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.844+0000 7fd94c982640 1 --2- 172.21.15.120:0/2471276996 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd9300a32d0 0x7fd9300a5790 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.844+0000 7fd94c982640 1 --2- 172.21.15.120:0/2471276996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd948106a80 0x7fd94819cdf0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.844+0000 7fd94c982640 1 --2- 172.21.15.120:0/2471276996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd948101420 0x7fd948198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.844+0000 7fd94c982640 1 --2- 172.21.15.120:0/2471276996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd948100a80 0x7fd948198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:06.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.844+0000 7fd94c982640 1 -- 172.21.15.120:0/2471276996 >> 172.21.15.120:0/2471276996 conn(0x7fd9480fc7a0 msgr2=0x7fd9480fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:06.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.844+0000 7fd94c982640 1 -- 172.21.15.120:0/2471276996 shutdown_connections 2024-09-19T09:03:06.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:06.844+0000 7fd94c982640 1 -- 172.21.15.120:0/2471276996 wait complete. 2024-09-19T09:03:07.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:06 smithi120 ceph-mon[31150]: pgmap v348: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T09:03:07.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:06 smithi196 ceph-mon[35150]: pgmap v348: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T09:03:07.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:06 smithi153 ceph-mon[35071]: pgmap v348: 97 pgs: 97 active+clean; 653 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T09:03:07.559 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T09:03:07.806 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:03:08.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.296+0000 7f3c426a5640 1 -- 172.21.15.120:0/1600220328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c3c101420 msgr2=0x7f3c3c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:08.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.296+0000 7f3c426a5640 1 --2- 172.21.15.120:0/1600220328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c3c101420 0x7f3c3c10d950 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3c34009cd0 tx=0x7f3c3402a8e0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.296+0000 7f3c426a5640 1 -- 172.21.15.120:0/1600220328 shutdown_connections 2024-09-19T09:03:08.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.296+0000 7f3c426a5640 1 --2- 172.21.15.120:0/1600220328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c3c101420 0x7f3c3c10d950 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.296+0000 7f3c426a5640 1 --2- 172.21.15.120:0/1600220328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c3c100a80 0x7f3c3c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.296+0000 7f3c426a5640 1 --2- 172.21.15.120:0/1600220328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c3c106a80 0x7f3c3c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.296+0000 7f3c426a5640 1 -- 172.21.15.120:0/1600220328 >> 172.21.15.120:0/1600220328 conn(0x7f3c3c0fc7a0 msgr2=0x7f3c3c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:08.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.296+0000 7f3c426a5640 1 -- 172.21.15.120:0/1600220328 shutdown_connections 2024-09-19T09:03:08.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.296+0000 7f3c426a5640 1 -- 172.21.15.120:0/1600220328 wait complete. 2024-09-19T09:03:08.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.296+0000 7f3c426a5640 1 Processor -- start 2024-09-19T09:03:08.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.296+0000 7f3c426a5640 1 -- start start 2024-09-19T09:03:08.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.297+0000 7f3c426a5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c3c100a80 0x7f3c3c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:08.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.297+0000 7f3c426a5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c3c101420 0x7f3c3c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:08.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.297+0000 7f3c426a5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c3c106a80 0x7f3c3c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:08.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.297+0000 7f3c426a5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3c110e80 con 0x7f3c3c100a80 2024-09-19T09:03:08.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.297+0000 7f3c426a5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3c110d00 con 0x7f3c3c101420 2024-09-19T09:03:08.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.297+0000 7f3c426a5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3c111000 con 0x7f3c3c106a80 2024-09-19T09:03:08.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.297+0000 7f3c3bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c3c100a80 0x7f3c3c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:08.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.297+0000 7f3c3bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c3c100a80 0x7f3c3c198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56254/0 (socket says 172.21.15.120:56254) 2024-09-19T09:03:08.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.297+0000 7f3c3bfff640 1 -- 172.21.15.120:0/4172576288 learned_addr learned my addr 172.21.15.120:0/4172576288 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:08.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.297+0000 7f3c3b7fe640 1 --2- 172.21.15.120:0/4172576288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c3c101420 0x7f3c3c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:08.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.297+0000 7f3c40c42640 1 --2- 172.21.15.120:0/4172576288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c3c106a80 0x7f3c3c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:08.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.297+0000 7f3c3bfff640 1 -- 172.21.15.120:0/4172576288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c3c106a80 msgr2=0x7f3c3c19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:08.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.297+0000 7f3c3bfff640 1 --2- 172.21.15.120:0/4172576288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c3c106a80 0x7f3c3c19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.297+0000 7f3c3bfff640 1 -- 172.21.15.120:0/4172576288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c3c101420 msgr2=0x7f3c3c198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:08.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.297+0000 7f3c3bfff640 1 --2- 172.21.15.120:0/4172576288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c3c101420 0x7f3c3c198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.297+0000 7f3c3bfff640 1 -- 172.21.15.120:0/4172576288 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c3c19d3d0 con 0x7f3c3c100a80 2024-09-19T09:03:08.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.297+0000 7f3c40c42640 1 --2- 172.21.15.120:0/4172576288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c3c106a80 0x7f3c3c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:03:08.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.297+0000 7f3c3bfff640 1 --2- 172.21.15.120:0/4172576288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c3c100a80 0x7f3c3c198420 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00d950 tx=0x7f3c2c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:08.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.297+0000 7f3c397fa640 1 -- 172.21.15.120:0/4172576288 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c017070 con 0x7f3c3c100a80 2024-09-19T09:03:08.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.297+0000 7f3c426a5640 1 -- 172.21.15.120:0/4172576288 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c3c19d6c0 con 0x7f3c3c100a80 2024-09-19T09:03:08.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.297+0000 7f3c397fa640 1 -- 172.21.15.120:0/4172576288 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c2c009c30 con 0x7f3c3c100a80 2024-09-19T09:03:08.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.297+0000 7f3c397fa640 1 -- 172.21.15.120:0/4172576288 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c00f6b0 con 0x7f3c3c100a80 2024-09-19T09:03:08.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.298+0000 7f3c426a5640 1 -- 172.21.15.120:0/4172576288 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c3c1a4fa0 con 0x7f3c3c100a80 2024-09-19T09:03:08.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.300+0000 7f3c397fa640 1 -- 172.21.15.120:0/4172576288 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f3c2c0109d0 con 0x7f3c3c100a80 2024-09-19T09:03:08.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.300+0000 7f3c397fa640 1 --2- 172.21.15.120:0/4172576288 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3c280a32d0 0x7f3c280a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:08.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.300+0000 7f3c44a7e740 1 Processor -- start 2024-09-19T09:03:08.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.300+0000 7f3c44a7e740 1 -- start start 2024-09-19T09:03:08.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.300+0000 7f3c3b7fe640 1 --2- 172.21.15.120:0/4172576288 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3c280a32d0 0x7f3c280a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:08.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.300+0000 7f3c397fa640 1 -- 172.21.15.120:0/4172576288 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f3c2c0cf6e0 con 0x7f3c3c100a80 2024-09-19T09:03:08.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.301+0000 7f3c44a7e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f6e3e8b0 0x5595f6e3ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:08.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.301+0000 7f3c44a7e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595f6e3f1d0 0x5595f6e4da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:08.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.301+0000 7f3c44a7e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595f6e4dfc0 0x5595f6e503b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:08.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.301+0000 7f3c44a7e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5595f6da5090 con 0x5595f6e4dfc0 2024-09-19T09:03:08.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.301+0000 7f3c44a7e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5595f6d4ca70 con 0x5595f6e3f1d0 2024-09-19T09:03:08.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.301+0000 7f3c44a7e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5595f6da1050 con 0x5595f6e3e8b0 2024-09-19T09:03:08.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.301+0000 7f3c40c42640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595f6e4dfc0 0x5595f6e503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:08.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.301+0000 7f3c40c42640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595f6e4dfc0 0x5595f6e503b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56258/0 (socket says 172.21.15.120:56258) 2024-09-19T09:03:08.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.301+0000 7f3c40c42640 1 -- 172.21.15.120:0/1375712120 learned_addr learned my addr 172.21.15.120:0/1375712120 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:08.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.301+0000 7f3c3b7fe640 1 --2- 172.21.15.120:0/1375712120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595f6e3f1d0 0x5595f6e4da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:08.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.301+0000 7f3c3bfff640 1 --2- 172.21.15.120:0/1375712120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f6e3e8b0 0x5595f6e3ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:08.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.301+0000 7f3c3b7fe640 1 --2- 172.21.15.120:0/4172576288 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3c280a32d0 0x7f3c280a5790 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3c24005fd0 tx=0x7f3c24005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:08.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.301+0000 7f3c40c42640 1 -- 172.21.15.120:0/1375712120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f6e3e8b0 msgr2=0x5595f6e3ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:08.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.301+0000 7f3c40c42640 1 --2- 172.21.15.120:0/1375712120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f6e3e8b0 0x5595f6e3ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.301+0000 7f3c40c42640 1 -- 172.21.15.120:0/1375712120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595f6e3f1d0 msgr2=0x5595f6e4da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:08.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.301+0000 7f3c40c42640 1 --2- 172.21.15.120:0/1375712120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595f6e3f1d0 0x5595f6e4da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.301+0000 7f3c40c42640 1 -- 172.21.15.120:0/1375712120 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595f6d59130 con 0x5595f6e4dfc0 2024-09-19T09:03:08.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.301+0000 7f3c40c42640 1 --2- 172.21.15.120:0/1375712120 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595f6e4dfc0 0x5595f6e503b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3c34005e30 tx=0x7f3c34009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc8ff8fbce0a4f2a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:08.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.301+0000 7f3c327fc640 1 -- 172.21.15.120:0/1375712120 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c340318c0 con 0x5595f6e4dfc0 2024-09-19T09:03:08.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.301+0000 7f3c327fc640 1 -- 172.21.15.120:0/1375712120 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c34036040 con 0x5595f6e4dfc0 2024-09-19T09:03:08.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.301+0000 7f3c327fc640 1 -- 172.21.15.120:0/1375712120 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c34004600 con 0x5595f6e4dfc0 2024-09-19T09:03:08.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.302+0000 7f3c44a7e740 1 -- 172.21.15.120:0/1375712120 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595f6e4dfc0 msgr2=0x5595f6e503b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:08.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.302+0000 7f3c44a7e740 1 --2- 172.21.15.120:0/1375712120 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595f6e4dfc0 0x5595f6e503b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3c34005e30 tx=0x7f3c34009a20 comp rx=0 tx=0).stop 2024-09-19T09:03:08.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.302+0000 7f3c44a7e740 1 -- 172.21.15.120:0/1375712120 shutdown_connections 2024-09-19T09:03:08.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.302+0000 7f3c44a7e740 1 --2- 172.21.15.120:0/1375712120 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595f6e4dfc0 0x5595f6e503b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.302+0000 7f3c44a7e740 1 --2- 172.21.15.120:0/1375712120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595f6e3f1d0 0x5595f6e4da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.302+0000 7f3c44a7e740 1 --2- 172.21.15.120:0/1375712120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f6e3e8b0 0x5595f6e3ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.302+0000 7f3c44a7e740 1 -- 172.21.15.120:0/1375712120 >> 172.21.15.120:0/1375712120 conn(0x5595f6e3af60 msgr2=0x5595f6e3b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:08.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.302+0000 7f3c44a7e740 1 -- 172.21.15.120:0/1375712120 shutdown_connections 2024-09-19T09:03:08.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.302+0000 7f3c44a7e740 1 -- 172.21.15.120:0/1375712120 wait complete. 2024-09-19T09:03:08.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.302+0000 7f3c44a7e740 1 -- start start 2024-09-19T09:03:08.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.303+0000 7f3c44a7e740 1 Processor -- start 2024-09-19T09:03:08.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.303+0000 7f3c44a7e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595f6e3e8b0 0x5595f6e4d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:08.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.303+0000 7f3c44a7e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595f6e3f1d0 0x5595f6e4abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:08.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.303+0000 7f3c44a7e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f6e4dfc0 0x5595f6e4b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:08.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.303+0000 7f3c44a7e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5595f6e50b90 con 0x5595f6e3e8b0 2024-09-19T09:03:08.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.303+0000 7f3c44a7e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5595f6d5ff20 con 0x5595f6e3f1d0 2024-09-19T09:03:08.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.303+0000 7f3c44a7e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5595f6e50d10 con 0x5595f6e4dfc0 2024-09-19T09:03:08.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.303+0000 7f3c44a7e740 10 client.0 _ll_register_callbacks cb 0x5595f6d29030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:08.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.303+0000 7f3c3bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595f6e3e8b0 0x5595f6e4d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:08.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.303+0000 7f3c3bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595f6e3e8b0 0x5595f6e4d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56264/0 (socket says 172.21.15.120:56264) 2024-09-19T09:03:08.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.303+0000 7f3c3bfff640 1 -- 172.21.15.120:0/2938946641 learned_addr learned my addr 172.21.15.120:0/2938946641 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:08.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.303+0000 7f3c3b7fe640 1 --2- 172.21.15.120:0/2938946641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595f6e3f1d0 0x5595f6e4abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:08.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.303+0000 7f3c3bfff640 1 -- 172.21.15.120:0/2938946641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f6e4dfc0 msgr2=0x5595f6e4b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:08.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.303+0000 7f3c3bfff640 1 --2- 172.21.15.120:0/2938946641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f6e4dfc0 0x5595f6e4b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.303+0000 7f3c3bfff640 1 -- 172.21.15.120:0/2938946641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595f6e3f1d0 msgr2=0x5595f6e4abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:08.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.303+0000 7f3c3bfff640 1 --2- 172.21.15.120:0/2938946641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595f6e3f1d0 0x5595f6e4abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.303+0000 7f3c3bfff640 1 -- 172.21.15.120:0/2938946641 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595f6b162b0 con 0x5595f6e3e8b0 2024-09-19T09:03:08.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.303+0000 7f3c3b7fe640 1 --2- 172.21.15.120:0/2938946641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595f6e3f1d0 0x5595f6e4abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:03:08.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.304+0000 7f3c3bfff640 1 --2- 172.21.15.120:0/2938946641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595f6e3e8b0 0x5595f6e4d860 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c09b8a0 tx=0x7f3c2c09c2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3c0eb782e4ebb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:08.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.304+0000 7f3c1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:08.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.304+0000 7f3c1b7fe640 1 -- 172.21.15.120:0/2938946641 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c020040 con 0x5595f6e3e8b0 2024-09-19T09:03:08.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.304+0000 7f3c1b7fe640 1 -- 172.21.15.120:0/2938946641 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f3c2c09aff0 con 0x5595f6e3e8b0 2024-09-19T09:03:08.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.304+0000 7f3c44a7e740 1 -- 172.21.15.120:0/2938946641 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c34036040 con 0x5595f6e3e8b0 2024-09-19T09:03:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.304+0000 7f3c1b7fe640 1 -- 172.21.15.120:0/2938946641 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3c2c09ff50 con 0x5595f6e3e8b0 2024-09-19T09:03:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.304+0000 7f3c44a7e740 10 client.15408 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.304+0000 7f3c44a7e740 1 -- 172.21.15.120:0/2938946641 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5595f6e5c540 con 0x5595f6e3e8b0 2024-09-19T09:03:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.304+0000 7f3c1b7fe640 1 -- 172.21.15.120:0/2938946641 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c34036040 con 0x5595f6e3e8b0 2024-09-19T09:03:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.304+0000 7f3c1b7fe640 1 -- 172.21.15.120:0/2938946641 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3c2c09aa60 con 0x5595f6e3e8b0 2024-09-19T09:03:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.304+0000 7f3c44a7e740 10 client.15408 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.304+0000 7f3c44a7e740 10 client.15408 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T09:03:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.304+0000 7f3c44a7e740 20 client.15408 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.304+0000 7f3c44a7e740 1 --2- 172.21.15.120:0/2938946641 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5595f6e5ec30 0x5595f6e600a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.304+0000 7f3c44a7e740 4 client.15408 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:03:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.304+0000 7f3c44a7e740 1 -- 172.21.15.120:0/2938946641 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595f6db2600 con 0x5595f6e5ec30 2024-09-19T09:03:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.305+0000 7f3c3b7fe640 1 --2- 172.21.15.120:0/2938946641 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5595f6e5ec30 0x5595f6e600a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.305+0000 7f3c3b7fe640 1 --2- 172.21.15.120:0/2938946641 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5595f6e5ec30 0x5595f6e600a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f837d02ecfae0b55 server_cookie=c46f97c562ef07ed in_seq=0 out_seq=0 2024-09-19T09:03:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.305+0000 7f3c1b7fe640 10 client.15408 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:03:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.306+0000 7f3c1b7fe640 1 -- 172.21.15.120:0/2938946641 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5595f6db2600 con 0x5595f6e5ec30 2024-09-19T09:03:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.306+0000 7f3c1b7fe640 10 client.15408 handle_command_reply: tid=0 2024-09-19T09:03:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.306+0000 7f3c44a7e740 1 client.15408 shutdown 2024-09-19T09:03:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.306+0000 7f3c44a7e740 1 -- 172.21.15.120:0/2938946641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595f6e3e8b0 msgr2=0x5595f6e4d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.306+0000 7f3c44a7e740 1 --2- 172.21.15.120:0/2938946641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595f6e3e8b0 0x5595f6e4d860 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c09b8a0 tx=0x7f3c2c09c2a0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.306+0000 7f3c44a7e740 1 -- 172.21.15.120:0/2938946641 shutdown_connections 2024-09-19T09:03:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.306+0000 7f3c44a7e740 1 --2- 172.21.15.120:0/2938946641 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5595f6e5ec30 0x5595f6e600a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.306+0000 7f3c44a7e740 1 --2- 172.21.15.120:0/2938946641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f6e4dfc0 0x5595f6e4b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.306+0000 7f3c44a7e740 1 --2- 172.21.15.120:0/2938946641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595f6e3f1d0 0x5595f6e4abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.306+0000 7f3c44a7e740 1 --2- 172.21.15.120:0/2938946641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595f6e3e8b0 0x5595f6e4d860 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.306+0000 7f3c44a7e740 1 -- 172.21.15.120:0/2938946641 >> 172.21.15.120:0/2938946641 conn(0x5595f6e3af60 msgr2=0x5595f6e40cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.306+0000 7f3c1b7fe640 0 client.15408 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:03:08.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.306+0000 7f3c44a7e740 1 -- 172.21.15.120:0/2938946641 shutdown_connections 2024-09-19T09:03:08.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.306+0000 7f3c44a7e740 1 -- 172.21.15.120:0/2938946641 wait complete. 2024-09-19T09:03:08.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.306+0000 7f3c44a7e740 20 client.15408 trim_cache size 0 max 16384 2024-09-19T09:03:08.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.323+0000 7f3c44a7e740 1 Processor -- start 2024-09-19T09:03:08.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.324+0000 7f3c44a7e740 1 -- start start 2024-09-19T09:03:08.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.324+0000 7f3c44a7e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595f6e3e8b0 0x5595f6e5e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:08.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.324+0000 7f3c44a7e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f6e3f1d0 0x5595f6e5f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:08.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.324+0000 7f3c44a7e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595f6e4dfc0 0x5595f6e46130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:08.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.324+0000 7f3c44a7e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5595f6e5e3e0 con 0x5595f6e3e8b0 2024-09-19T09:03:08.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.324+0000 7f3c44a7e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5595f6e5eab0 con 0x5595f6e4dfc0 2024-09-19T09:03:08.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.324+0000 7f3c44a7e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5595f6e5f180 con 0x5595f6e3f1d0 2024-09-19T09:03:08.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.324+0000 7f3c3bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595f6e3e8b0 0x5595f6e5e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:08.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.324+0000 7f3c3bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595f6e3e8b0 0x5595f6e5e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56276/0 (socket says 172.21.15.120:56276) 2024-09-19T09:03:08.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.324+0000 7f3c3bfff640 1 -- 172.21.15.120:0/17912473 learned_addr learned my addr 172.21.15.120:0/17912473 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:08.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.324+0000 7f3c3b7fe640 1 --2- 172.21.15.120:0/17912473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f6e3f1d0 0x5595f6e5f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:08.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.324+0000 7f3c3bfff640 1 -- 172.21.15.120:0/17912473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f6e3f1d0 msgr2=0x5595f6e5f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:08.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.324+0000 7f3c3bfff640 1 --2- 172.21.15.120:0/17912473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f6e3f1d0 0x5595f6e5f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.324+0000 7f3c3bfff640 1 -- 172.21.15.120:0/17912473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595f6e4dfc0 msgr2=0x5595f6e46130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:08.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.324+0000 7f3c3bfff640 1 --2- 172.21.15.120:0/17912473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595f6e4dfc0 0x5595f6e46130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.324+0000 7f3c3bfff640 1 -- 172.21.15.120:0/17912473 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595f6e4b6f0 con 0x5595f6e3e8b0 2024-09-19T09:03:08.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.324+0000 7f3c3b7fe640 1 --2- 172.21.15.120:0/17912473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f6e3f1d0 0x5595f6e5f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:03:08.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.324+0000 7f3c3bfff640 1 --2- 172.21.15.120:0/17912473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595f6e3e8b0 0x5595f6e5e560 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c09bb30 tx=0x7f3c2c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa0234116551787 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:08.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.324+0000 7f3c31ffb640 1 -- 172.21.15.120:0/17912473 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c09aa60 con 0x5595f6e3e8b0 2024-09-19T09:03:08.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.324+0000 7f3c31ffb640 1 -- 172.21.15.120:0/17912473 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c2c09ac00 con 0x5595f6e3e8b0 2024-09-19T09:03:08.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.324+0000 7f3c31ffb640 1 -- 172.21.15.120:0/17912473 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c0978f0 con 0x5595f6e3e8b0 2024-09-19T09:03:08.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.325+0000 7f3c44a7e740 1 -- 172.21.15.120:0/17912473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595f6e3e8b0 msgr2=0x5595f6e5e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:08.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.325+0000 7f3c44a7e740 1 --2- 172.21.15.120:0/17912473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595f6e3e8b0 0x5595f6e5e560 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c09bb30 tx=0x7f3c2c005880 comp rx=0 tx=0).stop 2024-09-19T09:03:08.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.325+0000 7f3c44a7e740 1 -- 172.21.15.120:0/17912473 shutdown_connections 2024-09-19T09:03:08.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.325+0000 7f3c44a7e740 1 --2- 172.21.15.120:0/17912473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595f6e4dfc0 0x5595f6e46130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.325+0000 7f3c44a7e740 1 --2- 172.21.15.120:0/17912473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f6e3f1d0 0x5595f6e5f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.325+0000 7f3c44a7e740 1 --2- 172.21.15.120:0/17912473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595f6e3e8b0 0x5595f6e5e560 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.325+0000 7f3c44a7e740 1 -- 172.21.15.120:0/17912473 >> 172.21.15.120:0/17912473 conn(0x5595f6e3af60 msgr2=0x5595f6e45a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:08.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.325+0000 7f3c44a7e740 1 -- 172.21.15.120:0/17912473 shutdown_connections 2024-09-19T09:03:08.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.325+0000 7f3c44a7e740 1 -- 172.21.15.120:0/17912473 wait complete. 2024-09-19T09:03:08.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.325+0000 7f3c44a7e740 1 -- start start 2024-09-19T09:03:08.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.326+0000 7f3c44a7e740 1 Processor -- start 2024-09-19T09:03:08.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.326+0000 7f3c44a7e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f6e3e8b0 0x5595f6e65070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:08.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.326+0000 7f3c44a7e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595f6e3f1d0 0x5595f6e655b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:08.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.326+0000 7f3c44a7e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595f6e4dfc0 0x5595f6e61840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:08.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.326+0000 7f3c44a7e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5595f6e5ff20 con 0x5595f6e3f1d0 2024-09-19T09:03:08.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.326+0000 7f3c44a7e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5595f6e5f850 con 0x5595f6e4dfc0 2024-09-19T09:03:08.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.326+0000 7f3c44a7e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5595f6e605f0 con 0x5595f6e3e8b0 2024-09-19T09:03:08.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.326+0000 7f3c44a7e740 10 client.0 _ll_register_callbacks cb 0x7f3c2c09b570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:08.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.326+0000 7f3c3b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595f6e3f1d0 0x5595f6e655b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:08.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.326+0000 7f3c3b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595f6e3f1d0 0x5595f6e655b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56288/0 (socket says 172.21.15.120:56288) 2024-09-19T09:03:08.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.326+0000 7f3c3b7fe640 1 -- 172.21.15.120:0/3928726825 learned_addr learned my addr 172.21.15.120:0/3928726825 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:08.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.326+0000 7f3c3bfff640 1 --2- 172.21.15.120:0/3928726825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f6e3e8b0 0x5595f6e65070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:08.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.326+0000 7f3c3b7fe640 1 -- 172.21.15.120:0/3928726825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f6e3e8b0 msgr2=0x5595f6e65070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:08.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.326+0000 7f3c3b7fe640 1 --2- 172.21.15.120:0/3928726825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f6e3e8b0 0x5595f6e65070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.326+0000 7f3c3b7fe640 1 -- 172.21.15.120:0/3928726825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595f6e4dfc0 msgr2=0x5595f6e61840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:08.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.326+0000 7f3c40c42640 1 --2- 172.21.15.120:0/3928726825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595f6e4dfc0 0x5595f6e61840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:08.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.326+0000 7f3c3b7fe640 1 --2- 172.21.15.120:0/3928726825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595f6e4dfc0 0x5595f6e61840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.326+0000 7f3c3b7fe640 1 -- 172.21.15.120:0/3928726825 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595f6e61e80 con 0x5595f6e3f1d0 2024-09-19T09:03:08.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.326+0000 7f3c3b7fe640 1 --2- 172.21.15.120:0/3928726825 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595f6e3f1d0 0x5595f6e655b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3c2400c010 tx=0x7f3c24004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87bde7056a1e6ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:08.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.326+0000 7f3c1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:08.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.327+0000 7f3c1b7fe640 1 -- 172.21.15.120:0/3928726825 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2402d070 con 0x5595f6e3f1d0 2024-09-19T09:03:08.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.327+0000 7f3c1b7fe640 1 -- 172.21.15.120:0/3928726825 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f3c24029740 con 0x5595f6e3f1d0 2024-09-19T09:03:08.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.327+0000 7f3c44a7e740 1 -- 172.21.15.120:0/3928726825 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c2c09ac00 con 0x5595f6e3f1d0 2024-09-19T09:03:08.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.327+0000 7f3c1b7fe640 1 -- 172.21.15.120:0/3928726825 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3c24039050 con 0x5595f6e3f1d0 2024-09-19T09:03:08.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.327+0000 7f3c44a7e740 10 client.15414 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:08.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.327+0000 7f3c44a7e740 1 -- 172.21.15.120:0/3928726825 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5595f6e66830 con 0x5595f6e3f1d0 2024-09-19T09:03:08.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.327+0000 7f3c1b7fe640 1 -- 172.21.15.120:0/3928726825 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c2c09ac00 con 0x5595f6e3f1d0 2024-09-19T09:03:08.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.327+0000 7f3c1b7fe640 1 -- 172.21.15.120:0/3928726825 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3c2401f430 con 0x5595f6e3f1d0 2024-09-19T09:03:08.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.327+0000 7f3c44a7e740 10 client.15414 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:08.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.327+0000 7f3c44a7e740 10 client.15414 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T09:03:08.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.327+0000 7f3c44a7e740 20 client.15414 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:08.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.327+0000 7f3c44a7e740 1 --2- 172.21.15.120:0/3928726825 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5595f6ecd810 0x5595f6ecc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:08.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.327+0000 7f3c44a7e740 4 client.15414 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:03:08.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.327+0000 7f3c44a7e740 1 -- 172.21.15.120:0/3928726825 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "status"}) -- 0x5595f6dacb10 con 0x5595f6ecd810 2024-09-19T09:03:08.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.327+0000 7f3c3bfff640 1 --2- 172.21.15.120:0/3928726825 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5595f6ecd810 0x5595f6ecc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:08.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.328+0000 7f3c1b7fe640 10 client.15414 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:03:08.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.328+0000 7f3c3bfff640 1 --2- 172.21.15.120:0/3928726825 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5595f6ecd810 0x5595f6ecc520 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a59a78d0ea7c9679 server_cookie=5266f4de0111bd4b in_seq=0 out_seq=0 2024-09-19T09:03:08.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.328+0000 7f3c1b7fe640 1 -- 172.21.15.120:0/3928726825 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5595f6dacb10 con 0x5595f6ecd810 2024-09-19T09:03:08.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.328+0000 7f3c1b7fe640 10 client.15414 handle_command_reply: tid=0 2024-09-19T09:03:08.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.328+0000 7f3c44a7e740 1 client.15414 shutdown 2024-09-19T09:03:08.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.328+0000 7f3c44a7e740 1 -- 172.21.15.120:0/3928726825 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595f6e3f1d0 msgr2=0x5595f6e655b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:08.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.328+0000 7f3c44a7e740 1 --2- 172.21.15.120:0/3928726825 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595f6e3f1d0 0x5595f6e655b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3c2400c010 tx=0x7f3c24004730 comp rx=0 tx=0).stop 2024-09-19T09:03:08.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.328+0000 7f3c44a7e740 1 -- 172.21.15.120:0/3928726825 shutdown_connections 2024-09-19T09:03:08.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.328+0000 7f3c44a7e740 1 --2- 172.21.15.120:0/3928726825 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5595f6ecd810 0x5595f6ecc520 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.328+0000 7f3c44a7e740 1 --2- 172.21.15.120:0/3928726825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595f6e4dfc0 0x5595f6e61840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.328+0000 7f3c44a7e740 1 --2- 172.21.15.120:0/3928726825 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595f6e3f1d0 0x5595f6e655b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.328+0000 7f3c1b7fe640 0 client.15414 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:03:08.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.328+0000 7f3c44a7e740 1 --2- 172.21.15.120:0/3928726825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595f6e3e8b0 0x5595f6e65070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.328+0000 7f3c44a7e740 1 -- 172.21.15.120:0/3928726825 >> 172.21.15.120:0/3928726825 conn(0x5595f6e3af60 msgr2=0x5595f6e46240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:08.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.328+0000 7f3c44a7e740 1 -- 172.21.15.120:0/3928726825 shutdown_connections 2024-09-19T09:03:08.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.328+0000 7f3c44a7e740 1 -- 172.21.15.120:0/3928726825 wait complete. 2024-09-19T09:03:08.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.328+0000 7f3c44a7e740 20 client.15414 trim_cache size 0 max 16384 2024-09-19T09:03:08.360 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:03:08.360 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_fsid": "507393d2-7664-11ef-bceb-c7b262605968", 2024-09-19T09:03:08.360 INFO:teuthology.orchestra.run.smithi120.stdout: "whoami": 3, 2024-09-19T09:03:08.360 INFO:teuthology.orchestra.run.smithi120.stdout: "id": 24430, 2024-09-19T09:03:08.360 INFO:teuthology.orchestra.run.smithi120.stdout: "want_state": "up:active", 2024-09-19T09:03:08.361 INFO:teuthology.orchestra.run.smithi120.stdout: "state": "up:active", 2024-09-19T09:03:08.361 INFO:teuthology.orchestra.run.smithi120.stdout: "fs_name": "cephfs", 2024-09-19T09:03:08.361 INFO:teuthology.orchestra.run.smithi120.stdout: "rank_uptime": 131.21212273200001, 2024-09-19T09:03:08.361 INFO:teuthology.orchestra.run.smithi120.stdout: "mdsmap_epoch": 28, 2024-09-19T09:03:08.361 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch": 81, 2024-09-19T09:03:08.361 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch_barrier": 81, 2024-09-19T09:03:08.361 INFO:teuthology.orchestra.run.smithi120.stdout: "uptime": 165.12016503999999 2024-09-19T09:03:08.361 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:03:08.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.329+0000 7f3c426a5640 1 -- 172.21.15.120:0/4172576288 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3c280a32d0 msgr2=0x7f3c280a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:08.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.329+0000 7f3c426a5640 1 --2- 172.21.15.120:0/4172576288 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3c280a32d0 0x7f3c280a5790 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3c24005fd0 tx=0x7f3c24005850 comp rx=0 tx=0).stop 2024-09-19T09:03:08.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.329+0000 7f3c426a5640 1 -- 172.21.15.120:0/4172576288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c3c100a80 msgr2=0x7f3c3c198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:08.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.329+0000 7f3c426a5640 1 --2- 172.21.15.120:0/4172576288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c3c100a80 0x7f3c3c198420 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00d950 tx=0x7f3c2c00de10 comp rx=0 tx=0).stop 2024-09-19T09:03:08.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.329+0000 7f3c426a5640 1 -- 172.21.15.120:0/4172576288 shutdown_connections 2024-09-19T09:03:08.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.329+0000 7f3c426a5640 1 --2- 172.21.15.120:0/4172576288 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3c280a32d0 0x7f3c280a5790 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.329+0000 7f3c426a5640 1 --2- 172.21.15.120:0/4172576288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c3c106a80 0x7f3c3c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.329+0000 7f3c426a5640 1 --2- 172.21.15.120:0/4172576288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c3c101420 0x7f3c3c198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.329+0000 7f3c426a5640 1 --2- 172.21.15.120:0/4172576288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c3c100a80 0x7f3c3c198420 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:08.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.329+0000 7f3c426a5640 1 -- 172.21.15.120:0/4172576288 >> 172.21.15.120:0/4172576288 conn(0x7f3c3c0fc7a0 msgr2=0x7f3c3c104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:08.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.329+0000 7f3c426a5640 1 -- 172.21.15.120:0/4172576288 shutdown_connections 2024-09-19T09:03:08.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:08.329+0000 7f3c426a5640 1 -- 172.21.15.120:0/4172576288 wait complete. 2024-09-19T09:03:09.031 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:08 smithi120 ceph-mon[31150]: pgmap v349: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.4 KiB/s wr, 14 op/s 2024-09-19T09:03:09.059 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T09:03:09.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:08 smithi196 ceph-mon[35150]: pgmap v349: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.4 KiB/s wr, 14 op/s 2024-09-19T09:03:09.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:08 smithi153 ceph-mon[35071]: pgmap v349: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.4 KiB/s wr, 14 op/s 2024-09-19T09:03:09.304 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:03:09.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.809+0000 7fe34ba97640 1 -- 172.21.15.120:0/2557284836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe344100a80 msgr2=0x7fe344100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:09.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.809+0000 7fe34ba97640 1 --2- 172.21.15.120:0/2557284836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe344100a80 0x7fe344100ee0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe338009930 tx=0x7fe33802a420 comp rx=0 tx=0).stop 2024-09-19T09:03:09.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.809+0000 7fe34ba97640 1 -- 172.21.15.120:0/2557284836 shutdown_connections 2024-09-19T09:03:09.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.809+0000 7fe34ba97640 1 --2- 172.21.15.120:0/2557284836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe344101420 0x7fe34410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.809+0000 7fe34ba97640 1 --2- 172.21.15.120:0/2557284836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe344100a80 0x7fe344100ee0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.809+0000 7fe34ba97640 1 --2- 172.21.15.120:0/2557284836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe344106a80 0x7fe344106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.809+0000 7fe34ba97640 1 -- 172.21.15.120:0/2557284836 >> 172.21.15.120:0/2557284836 conn(0x7fe3440fc7a0 msgr2=0x7fe3440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:09.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.809+0000 7fe34ba97640 1 -- 172.21.15.120:0/2557284836 shutdown_connections 2024-09-19T09:03:09.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.809+0000 7fe34ba97640 1 -- 172.21.15.120:0/2557284836 wait complete. 2024-09-19T09:03:09.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.809+0000 7fe34ba97640 1 Processor -- start 2024-09-19T09:03:09.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.809+0000 7fe34ba97640 1 -- start start 2024-09-19T09:03:09.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.809+0000 7fe34ba97640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe344100a80 0x7fe3440fcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:09.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.810+0000 7fe34ba97640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe344101420 0x7fe3440fd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:09.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.810+0000 7fe34ba97640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe344106a80 0x7fe3440fd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:09.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.810+0000 7fe34ba97640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe344110ea0 con 0x7fe344101420 2024-09-19T09:03:09.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.810+0000 7fe34ba97640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe344110d20 con 0x7fe344100a80 2024-09-19T09:03:09.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.810+0000 7fe34ba97640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe344111020 con 0x7fe344106a80 2024-09-19T09:03:09.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.810+0000 7fe349032640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe344101420 0x7fe3440fd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:09.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.810+0000 7fe349032640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe344101420 0x7fe3440fd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56312/0 (socket says 172.21.15.120:56312) 2024-09-19T09:03:09.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.810+0000 7fe349032640 1 -- 172.21.15.120:0/2445112358 learned_addr learned my addr 172.21.15.120:0/2445112358 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:09.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.810+0000 7fe34a034640 1 --2- 172.21.15.120:0/2445112358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe344106a80 0x7fe3440fd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:09.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.810+0000 7fe349833640 1 --2- 172.21.15.120:0/2445112358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe344100a80 0x7fe3440fcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:09.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.810+0000 7fe349032640 1 -- 172.21.15.120:0/2445112358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe344106a80 msgr2=0x7fe3440fd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:09.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.810+0000 7fe349032640 1 --2- 172.21.15.120:0/2445112358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe344106a80 0x7fe3440fd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.810+0000 7fe349032640 1 -- 172.21.15.120:0/2445112358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe344100a80 msgr2=0x7fe3440fcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:09.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.810+0000 7fe349032640 1 --2- 172.21.15.120:0/2445112358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe344100a80 0x7fe3440fcdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.810+0000 7fe349032640 1 -- 172.21.15.120:0/2445112358 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3441a6980 con 0x7fe344101420 2024-09-19T09:03:09.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.810+0000 7fe349032640 1 --2- 172.21.15.120:0/2445112358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe344101420 0x7fe3440fd2f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe338009230 tx=0x7fe338004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:09.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.810+0000 7fe336ffd640 1 -- 172.21.15.120:0/2445112358 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33803f030 con 0x7fe344101420 2024-09-19T09:03:09.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.810+0000 7fe336ffd640 1 -- 172.21.15.120:0/2445112358 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe338003ad0 con 0x7fe344101420 2024-09-19T09:03:09.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.810+0000 7fe336ffd640 1 -- 172.21.15.120:0/2445112358 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338035070 con 0x7fe344101420 2024-09-19T09:03:09.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.810+0000 7fe34ba97640 1 -- 172.21.15.120:0/2445112358 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3441a6c10 con 0x7fe344101420 2024-09-19T09:03:09.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.810+0000 7fe34ba97640 1 -- 172.21.15.120:0/2445112358 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3441a70f0 con 0x7fe344101420 2024-09-19T09:03:09.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.812+0000 7fe336ffd640 1 -- 172.21.15.120:0/2445112358 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7fe338042030 con 0x7fe344101420 2024-09-19T09:03:09.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.813+0000 7fe34de70740 1 Processor -- start 2024-09-19T09:03:09.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.813+0000 7fe34de70740 1 -- start start 2024-09-19T09:03:09.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.813+0000 7fe34de70740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56084a3419d0 0x56084a341db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:09.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.813+0000 7fe34de70740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56084a3422f0 0x56084a34c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:09.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.813+0000 7fe336ffd640 1 --2- 172.21.15.120:0/2445112358 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe3300a32d0 0x7fe3300a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:09.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.813+0000 7fe34de70740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56084a34cbb0 0x56084a34efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:09.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.813+0000 7fe34de70740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56084a2a8090 con 0x56084a3419d0 2024-09-19T09:03:09.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.813+0000 7fe34de70740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56084a24fa70 con 0x56084a3422f0 2024-09-19T09:03:09.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.813+0000 7fe34de70740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56084a2a4050 con 0x56084a34cbb0 2024-09-19T09:03:09.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.813+0000 7fe34a034640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56084a3419d0 0x56084a341db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:09.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.814+0000 7fe349032640 1 --2- 172.21.15.120:0/2445112358 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe3300a32d0 0x7fe3300a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:09.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.814+0000 7fe34a034640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56084a3419d0 0x56084a341db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56322/0 (socket says 172.21.15.120:56322) 2024-09-19T09:03:09.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.814+0000 7fe34a034640 1 -- 172.21.15.120:0/1458755100 learned_addr learned my addr 172.21.15.120:0/1458755100 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:09.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.814+0000 7fe336ffd640 1 -- 172.21.15.120:0/2445112358 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe3380efee0 con 0x7fe344101420 2024-09-19T09:03:09.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.814+0000 7fe34a034640 1 --2- 172.21.15.120:0/1458755100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56084a34cbb0 0x56084a34efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:09.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.814+0000 7fe349833640 1 --2- 172.21.15.120:0/1458755100 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56084a3422f0 0x56084a34c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:09.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.814+0000 7fe34a034640 1 -- 172.21.15.120:0/1458755100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56084a34cbb0 msgr2=0x56084a34efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:09.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.814+0000 7fe34a034640 1 --2- 172.21.15.120:0/1458755100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56084a34cbb0 0x56084a34efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.814+0000 7fe34a034640 1 -- 172.21.15.120:0/1458755100 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56084a3422f0 msgr2=0x56084a34c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:09.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.814+0000 7fe34a034640 1 --2- 172.21.15.120:0/1458755100 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56084a3422f0 0x56084a34c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.814+0000 7fe34a034640 1 -- 172.21.15.120:0/1458755100 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56084a25c130 con 0x56084a3419d0 2024-09-19T09:03:09.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.814+0000 7fe349032640 1 --2- 172.21.15.120:0/2445112358 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe3300a32d0 0x7fe3300a5790 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe3380bc330 tx=0x7fe338049770 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:09.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.814+0000 7fe34a034640 1 --2- 172.21.15.120:0/1458755100 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56084a3419d0 0x56084a341db0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe33c005c20 tx=0x7fe33c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=116db4b82d34918 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:09.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.814+0000 7fe32ffff640 1 -- 172.21.15.120:0/1458755100 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c01a030 con 0x56084a3419d0 2024-09-19T09:03:09.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.814+0000 7fe32ffff640 1 -- 172.21.15.120:0/1458755100 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe33c012910 con 0x56084a3419d0 2024-09-19T09:03:09.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.814+0000 7fe32ffff640 1 -- 172.21.15.120:0/1458755100 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c005480 con 0x56084a3419d0 2024-09-19T09:03:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.815+0000 7fe34de70740 1 -- 172.21.15.120:0/1458755100 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56084a3419d0 msgr2=0x56084a341db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.815+0000 7fe34de70740 1 --2- 172.21.15.120:0/1458755100 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56084a3419d0 0x56084a341db0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe33c005c20 tx=0x7fe33c009f90 comp rx=0 tx=0).stop 2024-09-19T09:03:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.815+0000 7fe34de70740 1 -- 172.21.15.120:0/1458755100 shutdown_connections 2024-09-19T09:03:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.815+0000 7fe34de70740 1 --2- 172.21.15.120:0/1458755100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56084a34cbb0 0x56084a34efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.815+0000 7fe34de70740 1 --2- 172.21.15.120:0/1458755100 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56084a3422f0 0x56084a34c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.815+0000 7fe34de70740 1 --2- 172.21.15.120:0/1458755100 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56084a3419d0 0x56084a341db0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.815+0000 7fe34de70740 1 -- 172.21.15.120:0/1458755100 >> 172.21.15.120:0/1458755100 conn(0x56084a33df60 msgr2=0x56084a33e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.815+0000 7fe34de70740 1 -- 172.21.15.120:0/1458755100 shutdown_connections 2024-09-19T09:03:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.815+0000 7fe34de70740 1 -- 172.21.15.120:0/1458755100 wait complete. 2024-09-19T09:03:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.815+0000 7fe34de70740 1 -- start start 2024-09-19T09:03:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.815+0000 7fe34de70740 1 Processor -- start 2024-09-19T09:03:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.816+0000 7fe34de70740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56084a3419d0 0x56084a34c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.816+0000 7fe34de70740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56084a3422f0 0x56084a349740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.816+0000 7fe34de70740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56084a34cbb0 0x56084a349c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.816+0000 7fe34de70740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56084a34f780 con 0x56084a3419d0 2024-09-19T09:03:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.816+0000 7fe34de70740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56084a262f20 con 0x56084a3422f0 2024-09-19T09:03:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.816+0000 7fe34de70740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56084a34f900 con 0x56084a34cbb0 2024-09-19T09:03:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.816+0000 7fe34de70740 10 client.0 _ll_register_callbacks cb 0x56084a22c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.816+0000 7fe34a034640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56084a3419d0 0x56084a34c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.816+0000 7fe34a034640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56084a3419d0 0x56084a34c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56332/0 (socket says 172.21.15.120:56332) 2024-09-19T09:03:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.816+0000 7fe34a034640 1 -- 172.21.15.120:0/1156849487 learned_addr learned my addr 172.21.15.120:0/1156849487 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.816+0000 7fe349833640 1 --2- 172.21.15.120:0/1156849487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56084a3422f0 0x56084a349740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.816+0000 7fe34a034640 1 -- 172.21.15.120:0/1156849487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56084a34cbb0 msgr2=0x56084a349c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.816+0000 7fe34a034640 1 --2- 172.21.15.120:0/1156849487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56084a34cbb0 0x56084a349c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.816+0000 7fe34a034640 1 -- 172.21.15.120:0/1156849487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56084a3422f0 msgr2=0x56084a349740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.816+0000 7fe34a034640 1 --2- 172.21.15.120:0/1156849487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56084a3422f0 0x56084a349740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.816+0000 7fe34a034640 1 -- 172.21.15.120:0/1156849487 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56084a0192b0 con 0x56084a3419d0 2024-09-19T09:03:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.816+0000 7fe34a034640 1 --2- 172.21.15.120:0/1156849487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56084a3419d0 0x56084a34c3f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe33c005f40 tx=0x7fe33c005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449f1a45407033ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.816+0000 7fe32d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.817+0000 7fe32d7fa640 1 -- 172.21.15.120:0/1156849487 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c01a070 con 0x56084a3419d0 2024-09-19T09:03:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.817+0000 7fe32d7fa640 1 -- 172.21.15.120:0/1156849487 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe33c004e10 con 0x56084a3419d0 2024-09-19T09:03:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.817+0000 7fe34de70740 1 -- 172.21.15.120:0/1156849487 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe33c012910 con 0x56084a3419d0 2024-09-19T09:03:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.817+0000 7fe32d7fa640 1 -- 172.21.15.120:0/1156849487 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe33c005260 con 0x56084a3419d0 2024-09-19T09:03:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.817+0000 7fe34de70740 10 client.15426 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.817+0000 7fe34de70740 1 -- 172.21.15.120:0/1156849487 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56084a35b0c0 con 0x56084a3419d0 2024-09-19T09:03:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.817+0000 7fe32d7fa640 1 -- 172.21.15.120:0/1156849487 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe33c012910 con 0x56084a3419d0 2024-09-19T09:03:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.817+0000 7fe32d7fa640 1 -- 172.21.15.120:0/1156849487 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe33c027070 con 0x56084a3419d0 2024-09-19T09:03:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.817+0000 7fe34de70740 10 client.15426 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.817+0000 7fe34de70740 10 client.15426 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T09:03:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.817+0000 7fe34de70740 20 client.15426 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.817+0000 7fe34de70740 1 --2- 172.21.15.120:0/1156849487 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56084a35d530 0x56084a35e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.817+0000 7fe34de70740 4 client.15426 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:03:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.817+0000 7fe34de70740 1 -- 172.21.15.120:0/1156849487 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56084a2b5600 con 0x56084a35d530 2024-09-19T09:03:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.818+0000 7fe349833640 1 --2- 172.21.15.120:0/1156849487 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56084a35d530 0x56084a35e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.818+0000 7fe32d7fa640 10 client.15426 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:03:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.818+0000 7fe349833640 1 --2- 172.21.15.120:0/1156849487 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56084a35d530 0x56084a35e9a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e03b0d5308e70b22 server_cookie=51950db75b1c37d3 in_seq=0 out_seq=0 2024-09-19T09:03:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.819+0000 7fe32d7fa640 1 -- 172.21.15.120:0/1156849487 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56084a2b5600 con 0x56084a35d530 2024-09-19T09:03:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.819+0000 7fe32d7fa640 10 client.15426 handle_command_reply: tid=0 2024-09-19T09:03:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.819+0000 7fe34de70740 1 client.15426 shutdown 2024-09-19T09:03:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.819+0000 7fe34de70740 1 -- 172.21.15.120:0/1156849487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56084a3419d0 msgr2=0x56084a34c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.819+0000 7fe34de70740 1 --2- 172.21.15.120:0/1156849487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56084a3419d0 0x56084a34c3f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe33c005f40 tx=0x7fe33c005cf0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.819+0000 7fe34de70740 1 -- 172.21.15.120:0/1156849487 shutdown_connections 2024-09-19T09:03:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.819+0000 7fe34de70740 1 --2- 172.21.15.120:0/1156849487 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56084a35d530 0x56084a35e9a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.819+0000 7fe34de70740 1 --2- 172.21.15.120:0/1156849487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56084a34cbb0 0x56084a349c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.819+0000 7fe34de70740 1 --2- 172.21.15.120:0/1156849487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56084a3422f0 0x56084a349740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.819+0000 7fe34de70740 1 --2- 172.21.15.120:0/1156849487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56084a3419d0 0x56084a34c3f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.819+0000 7fe32d7fa640 0 client.15426 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:03:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.819+0000 7fe34de70740 1 -- 172.21.15.120:0/1156849487 >> 172.21.15.120:0/1156849487 conn(0x56084a33df60 msgr2=0x56084a343c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.819+0000 7fe34de70740 1 -- 172.21.15.120:0/1156849487 shutdown_connections 2024-09-19T09:03:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.819+0000 7fe34de70740 1 -- 172.21.15.120:0/1156849487 wait complete. 2024-09-19T09:03:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.819+0000 7fe34de70740 20 client.15426 trim_cache size 0 max 16384 2024-09-19T09:03:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.836+0000 7fe34de70740 1 Processor -- start 2024-09-19T09:03:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.836+0000 7fe34de70740 1 -- start start 2024-09-19T09:03:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.837+0000 7fe34de70740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56084a3419d0 0x56084a35e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.837+0000 7fe34de70740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56084a3422f0 0x56084a35ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.837+0000 7fe34de70740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56084a34cbb0 0x56084a338ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.837+0000 7fe34de70740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56084a35cce0 con 0x56084a3419d0 2024-09-19T09:03:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.837+0000 7fe34de70740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56084a35d3b0 con 0x56084a3422f0 2024-09-19T09:03:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.837+0000 7fe34de70740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56084a35da80 con 0x56084a34cbb0 2024-09-19T09:03:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.837+0000 7fe34a034640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56084a3419d0 0x56084a35e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.837+0000 7fe34a034640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56084a3419d0 0x56084a35e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56340/0 (socket says 172.21.15.120:56340) 2024-09-19T09:03:09.907 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:03:09.907 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_fsid": "507393d2-7664-11ef-bceb-c7b262605968", 2024-09-19T09:03:09.907 INFO:teuthology.orchestra.run.smithi120.stdout: "whoami": 2, 2024-09-19T09:03:09.907 INFO:teuthology.orchestra.run.smithi120.stdout: "id": 24442, 2024-09-19T09:03:09.907 INFO:teuthology.orchestra.run.smithi120.stdout: "want_state": "up:active", 2024-09-19T09:03:09.907 INFO:teuthology.orchestra.run.smithi120.stdout: "state": "up:active", 2024-09-19T09:03:09.907 INFO:teuthology.orchestra.run.smithi120.stdout: "fs_name": "cephfs", 2024-09-19T09:03:09.907 INFO:teuthology.orchestra.run.smithi120.stdout: "rank_uptime": 133.73213972100001, 2024-09-19T09:03:09.907 INFO:teuthology.orchestra.run.smithi120.stdout: "mdsmap_epoch": 28, 2024-09-19T09:03:09.907 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch": 81, 2024-09-19T09:03:09.907 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch_barrier": 81, 2024-09-19T09:03:09.907 INFO:teuthology.orchestra.run.smithi120.stdout: "uptime": 161.36873194699999 2024-09-19T09:03:09.907 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:03:09.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.837+0000 7fe34a034640 1 -- 172.21.15.120:0/1669280692 learned_addr learned my addr 172.21.15.120:0/1669280692 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:09.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.837+0000 7fe34a034640 1 -- 172.21.15.120:0/1669280692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56084a34cbb0 msgr2=0x56084a338ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:09.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.837+0000 7fe34a034640 1 --2- 172.21.15.120:0/1669280692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56084a34cbb0 0x56084a338ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.837+0000 7fe34a034640 1 -- 172.21.15.120:0/1669280692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56084a3422f0 msgr2=0x56084a35ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:09.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.837+0000 7fe34a034640 1 --2- 172.21.15.120:0/1669280692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56084a3422f0 0x56084a35ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.837+0000 7fe34a034640 1 -- 172.21.15.120:0/1669280692 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56084a342120 con 0x56084a3419d0 2024-09-19T09:03:09.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.837+0000 7fe34a034640 1 --2- 172.21.15.120:0/1669280692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56084a3419d0 0x56084a35e9a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe33c014590 tx=0x7fe33c005c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=895d77f289681c0b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:09.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.837+0000 7fe32f7fe640 1 -- 172.21.15.120:0/1669280692 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c01a040 con 0x56084a3419d0 2024-09-19T09:03:09.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.837+0000 7fe32f7fe640 1 -- 172.21.15.120:0/1669280692 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe33c01b040 con 0x56084a3419d0 2024-09-19T09:03:09.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.837+0000 7fe32f7fe640 1 -- 172.21.15.120:0/1669280692 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c005400 con 0x56084a3419d0 2024-09-19T09:03:09.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.838+0000 7fe34de70740 1 -- 172.21.15.120:0/1669280692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56084a3419d0 msgr2=0x56084a35e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:09.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.838+0000 7fe34de70740 1 --2- 172.21.15.120:0/1669280692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56084a3419d0 0x56084a35e9a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe33c014590 tx=0x7fe33c005c10 comp rx=0 tx=0).stop 2024-09-19T09:03:09.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.838+0000 7fe34de70740 1 -- 172.21.15.120:0/1669280692 shutdown_connections 2024-09-19T09:03:09.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.838+0000 7fe34de70740 1 --2- 172.21.15.120:0/1669280692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56084a34cbb0 0x56084a338ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.838+0000 7fe34de70740 1 --2- 172.21.15.120:0/1669280692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56084a3422f0 0x56084a35ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.838+0000 7fe34de70740 1 --2- 172.21.15.120:0/1669280692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56084a3419d0 0x56084a35e9a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.838+0000 7fe34de70740 1 -- 172.21.15.120:0/1669280692 >> 172.21.15.120:0/1669280692 conn(0x56084a33df60 msgr2=0x56084a35c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:09.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.838+0000 7fe34de70740 1 -- 172.21.15.120:0/1669280692 shutdown_connections 2024-09-19T09:03:09.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.838+0000 7fe34de70740 1 -- 172.21.15.120:0/1669280692 wait complete. 2024-09-19T09:03:09.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.838+0000 7fe34de70740 1 -- start start 2024-09-19T09:03:09.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.838+0000 7fe34de70740 1 Processor -- start 2024-09-19T09:03:09.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.839+0000 7fe34de70740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56084a3419d0 0x56084a366e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:09.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.839+0000 7fe34de70740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56084a3422f0 0x56084a3673b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:09.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.839+0000 7fe34de70740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56084a34cbb0 0x56084a3601c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:09.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.839+0000 7fe34de70740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56084a35e820 con 0x56084a3422f0 2024-09-19T09:03:09.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.839+0000 7fe34de70740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56084a35e150 con 0x56084a3419d0 2024-09-19T09:03:09.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.839+0000 7fe34de70740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56084a35eef0 con 0x56084a34cbb0 2024-09-19T09:03:09.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.839+0000 7fe34de70740 10 client.0 _ll_register_callbacks cb 0x7fe33c014790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:09.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.839+0000 7fe349833640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56084a3422f0 0x56084a3673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:09.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.839+0000 7fe349833640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56084a3422f0 0x56084a3673b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56350/0 (socket says 172.21.15.120:56350) 2024-09-19T09:03:09.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.839+0000 7fe349833640 1 -- 172.21.15.120:0/2817842416 learned_addr learned my addr 172.21.15.120:0/2817842416 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:09.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.839+0000 7fe34a034640 1 --2- 172.21.15.120:0/2817842416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56084a3419d0 0x56084a366e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:09.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.839+0000 7fe34a034640 1 --2- 172.21.15.120:0/2817842416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56084a34cbb0 0x56084a3601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:09.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.839+0000 7fe349833640 1 -- 172.21.15.120:0/2817842416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56084a34cbb0 msgr2=0x56084a3601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:09.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.839+0000 7fe349833640 1 --2- 172.21.15.120:0/2817842416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56084a34cbb0 0x56084a3601c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.839+0000 7fe349833640 1 -- 172.21.15.120:0/2817842416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56084a3419d0 msgr2=0x56084a366e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:09.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.839+0000 7fe349833640 1 --2- 172.21.15.120:0/2817842416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56084a3419d0 0x56084a366e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.839+0000 7fe349833640 1 -- 172.21.15.120:0/2817842416 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56084a3609f0 con 0x56084a3422f0 2024-09-19T09:03:09.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.839+0000 7fe34a034640 1 --2- 172.21.15.120:0/2817842416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56084a34cbb0 0x56084a3601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:03:09.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.839+0000 7fe349833640 1 --2- 172.21.15.120:0/2817842416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56084a3422f0 0x56084a3673b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe340006e10 tx=0x7fe3400098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6abb7120c53d95b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:09.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.839+0000 7fe32d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:09.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.839+0000 7fe32d7fa640 1 -- 172.21.15.120:0/2817842416 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe340021070 con 0x56084a3422f0 2024-09-19T09:03:09.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.839+0000 7fe32d7fa640 1 -- 172.21.15.120:0/2817842416 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe34001d250 con 0x56084a3422f0 2024-09-19T09:03:09.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.839+0000 7fe34de70740 1 -- 172.21.15.120:0/2817842416 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe33c01b040 con 0x56084a3422f0 2024-09-19T09:03:09.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.840+0000 7fe32d7fa640 1 -- 172.21.15.120:0/2817842416 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe34002d050 con 0x56084a3422f0 2024-09-19T09:03:09.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.840+0000 7fe34de70740 10 client.15432 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:09.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.840+0000 7fe34de70740 1 -- 172.21.15.120:0/2817842416 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56084a365530 con 0x56084a3422f0 2024-09-19T09:03:09.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.840+0000 7fe32d7fa640 1 -- 172.21.15.120:0/2817842416 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe33c01b040 con 0x56084a3422f0 2024-09-19T09:03:09.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.840+0000 7fe32d7fa640 1 -- 172.21.15.120:0/2817842416 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe3400256b0 con 0x56084a3422f0 2024-09-19T09:03:09.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.840+0000 7fe34de70740 10 client.15432 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:09.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.840+0000 7fe34de70740 10 client.15432 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T09:03:09.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.840+0000 7fe34de70740 20 client.15432 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:09.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.840+0000 7fe34de70740 1 --2- 172.21.15.120:0/2817842416 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56084a3cc330 0x56084a3cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:09.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.840+0000 7fe34de70740 4 client.15432 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:03:09.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.840+0000 7fe34de70740 1 -- 172.21.15.120:0/2817842416 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "status"}) -- 0x56084a2afb10 con 0x56084a3cc330 2024-09-19T09:03:09.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.840+0000 7fe34a034640 1 --2- 172.21.15.120:0/2817842416 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56084a3cc330 0x56084a3cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:09.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.841+0000 7fe34a034640 1 --2- 172.21.15.120:0/2817842416 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56084a3cc330 0x56084a3cb040 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d2a801cddd80581e server_cookie=e9d09fc658144334 in_seq=0 out_seq=0 2024-09-19T09:03:09.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.841+0000 7fe32d7fa640 10 client.15432 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:03:09.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.841+0000 7fe32d7fa640 1 -- 172.21.15.120:0/2817842416 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56084a2afb10 con 0x56084a3cc330 2024-09-19T09:03:09.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.841+0000 7fe32d7fa640 10 client.15432 handle_command_reply: tid=0 2024-09-19T09:03:09.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.841+0000 7fe34de70740 1 client.15432 shutdown 2024-09-19T09:03:09.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.841+0000 7fe34de70740 1 -- 172.21.15.120:0/2817842416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56084a3422f0 msgr2=0x56084a3673b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:09.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.841+0000 7fe34de70740 1 --2- 172.21.15.120:0/2817842416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56084a3422f0 0x56084a3673b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe340006e10 tx=0x7fe3400098e0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.841+0000 7fe34de70740 1 -- 172.21.15.120:0/2817842416 shutdown_connections 2024-09-19T09:03:09.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.841+0000 7fe34de70740 1 --2- 172.21.15.120:0/2817842416 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56084a3cc330 0x56084a3cb040 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.841+0000 7fe34de70740 1 --2- 172.21.15.120:0/2817842416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56084a34cbb0 0x56084a3601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.841+0000 7fe32d7fa640 0 client.15432 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:03:09.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.841+0000 7fe34de70740 1 --2- 172.21.15.120:0/2817842416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56084a3422f0 0x56084a3673b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.841+0000 7fe34de70740 1 --2- 172.21.15.120:0/2817842416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56084a3419d0 0x56084a366e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.841+0000 7fe34de70740 1 -- 172.21.15.120:0/2817842416 >> 172.21.15.120:0/2817842416 conn(0x56084a33df60 msgr2=0x56084a338ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:09.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.841+0000 7fe34de70740 1 -- 172.21.15.120:0/2817842416 shutdown_connections 2024-09-19T09:03:09.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.841+0000 7fe34de70740 1 -- 172.21.15.120:0/2817842416 wait complete. 2024-09-19T09:03:09.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.841+0000 7fe34de70740 20 client.15432 trim_cache size 0 max 16384 2024-09-19T09:03:09.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.842+0000 7fe34ba97640 1 -- 172.21.15.120:0/2445112358 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe3300a32d0 msgr2=0x7fe3300a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:09.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.842+0000 7fe34ba97640 1 --2- 172.21.15.120:0/2445112358 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe3300a32d0 0x7fe3300a5790 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe3380bc330 tx=0x7fe338049770 comp rx=0 tx=0).stop 2024-09-19T09:03:09.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.842+0000 7fe34ba97640 1 -- 172.21.15.120:0/2445112358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe344101420 msgr2=0x7fe3440fd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:09.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.842+0000 7fe34ba97640 1 --2- 172.21.15.120:0/2445112358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe344101420 0x7fe3440fd2f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe338009230 tx=0x7fe338004270 comp rx=0 tx=0).stop 2024-09-19T09:03:09.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.842+0000 7fe34ba97640 1 -- 172.21.15.120:0/2445112358 shutdown_connections 2024-09-19T09:03:09.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.842+0000 7fe34ba97640 1 --2- 172.21.15.120:0/2445112358 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe3300a32d0 0x7fe3300a5790 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.842+0000 7fe34ba97640 1 --2- 172.21.15.120:0/2445112358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe344106a80 0x7fe3440fd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.842+0000 7fe34ba97640 1 --2- 172.21.15.120:0/2445112358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe344101420 0x7fe3440fd2f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.842+0000 7fe34ba97640 1 --2- 172.21.15.120:0/2445112358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe344100a80 0x7fe3440fcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:09.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.842+0000 7fe34ba97640 1 -- 172.21.15.120:0/2445112358 >> 172.21.15.120:0/2445112358 conn(0x7fe3440fc7a0 msgr2=0x7fe34406aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:09.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.842+0000 7fe34ba97640 1 -- 172.21.15.120:0/2445112358 shutdown_connections 2024-09-19T09:03:09.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:09.842+0000 7fe34ba97640 1 -- 172.21.15.120:0/2445112358 wait complete. 2024-09-19T09:03:10.564 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T09:03:10.812 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:03:10.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:10 smithi120 ceph-mon[31150]: pgmap v350: 97 pgs: 97 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.6 KiB/s wr, 14 op/s 2024-09-19T09:03:11.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:10 smithi196 ceph-mon[35150]: pgmap v350: 97 pgs: 97 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.6 KiB/s wr, 14 op/s 2024-09-19T09:03:11.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:10 smithi153 ceph-mon[35071]: pgmap v350: 97 pgs: 97 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.6 KiB/s wr, 14 op/s 2024-09-19T09:03:11.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.309+0000 7f78083f9640 1 -- 172.21.15.120:0/230460248 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7800106a80 msgr2=0x7f7800106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:11.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.309+0000 7f78083f9640 1 --2- 172.21.15.120:0/230460248 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7800106a80 0x7f7800106e60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f77f0009930 tx=0x7f77f002a420 comp rx=0 tx=0).stop 2024-09-19T09:03:11.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.309+0000 7f78083f9640 1 -- 172.21.15.120:0/230460248 shutdown_connections 2024-09-19T09:03:11.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.309+0000 7f78083f9640 1 --2- 172.21.15.120:0/230460248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7800101420 0x7f780010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.309+0000 7f78083f9640 1 --2- 172.21.15.120:0/230460248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7800100a80 0x7f7800100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.309+0000 7f78083f9640 1 --2- 172.21.15.120:0/230460248 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7800106a80 0x7f7800106e60 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.309+0000 7f78083f9640 1 -- 172.21.15.120:0/230460248 >> 172.21.15.120:0/230460248 conn(0x7f78000fc7a0 msgr2=0x7f78000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:11.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.309+0000 7f78083f9640 1 -- 172.21.15.120:0/230460248 shutdown_connections 2024-09-19T09:03:11.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.309+0000 7f78083f9640 1 -- 172.21.15.120:0/230460248 wait complete. 2024-09-19T09:03:11.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.310+0000 7f78083f9640 1 Processor -- start 2024-09-19T09:03:11.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.310+0000 7f78083f9640 1 -- start start 2024-09-19T09:03:11.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.310+0000 7f78083f9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7800100a80 0x7f7800197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:11.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.310+0000 7f78083f9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7800101420 0x7f78001978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:11.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.310+0000 7f78083f9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7800106a80 0x7f780019bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:11.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.310+0000 7f78083f9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7800110e80 con 0x7f7800100a80 2024-09-19T09:03:11.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.310+0000 7f78083f9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7800110d00 con 0x7f7800101420 2024-09-19T09:03:11.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.310+0000 7f78083f9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7800111000 con 0x7f7800106a80 2024-09-19T09:03:11.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.310+0000 7f7806195640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7800100a80 0x7f7800197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:11.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.310+0000 7f7806195640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7800100a80 0x7f7800197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59750/0 (socket says 172.21.15.120:59750) 2024-09-19T09:03:11.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.310+0000 7f7806195640 1 -- 172.21.15.120:0/3026997029 learned_addr learned my addr 172.21.15.120:0/3026997029 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:11.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.310+0000 7f7805994640 1 --2- 172.21.15.120:0/3026997029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7800101420 0x7f78001978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:11.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.310+0000 7f7806195640 1 -- 172.21.15.120:0/3026997029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7800106a80 msgr2=0x7f780019bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:11.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.310+0000 7f7806195640 1 --2- 172.21.15.120:0/3026997029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7800106a80 0x7f780019bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.310+0000 7f7806195640 1 -- 172.21.15.120:0/3026997029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7800101420 msgr2=0x7f78001978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:11.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.310+0000 7f7806195640 1 --2- 172.21.15.120:0/3026997029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7800101420 0x7f78001978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.310+0000 7f7806195640 1 -- 172.21.15.120:0/3026997029 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780019c310 con 0x7f7800100a80 2024-09-19T09:03:11.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.311+0000 7f7806195640 1 --2- 172.21.15.120:0/3026997029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7800100a80 0x7f7800197360 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f77f0009a50 tx=0x7f77f0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:11.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.311+0000 7f77f77fe640 1 -- 172.21.15.120:0/3026997029 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f003f030 con 0x7f7800100a80 2024-09-19T09:03:11.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.311+0000 7f77f77fe640 1 -- 172.21.15.120:0/3026997029 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77f0003ce0 con 0x7f7800100a80 2024-09-19T09:03:11.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.311+0000 7f78083f9640 1 -- 172.21.15.120:0/3026997029 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f780019c5a0 con 0x7f7800100a80 2024-09-19T09:03:11.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.311+0000 7f77f77fe640 1 -- 172.21.15.120:0/3026997029 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0035070 con 0x7f7800100a80 2024-09-19T09:03:11.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.311+0000 7f78083f9640 1 -- 172.21.15.120:0/3026997029 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78001a3ee0 con 0x7f7800100a80 2024-09-19T09:03:11.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.313+0000 7f77f77fe640 1 -- 172.21.15.120:0/3026997029 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f77f0042030 con 0x7f7800100a80 2024-09-19T09:03:11.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.313+0000 7f780a7d2740 1 Processor -- start 2024-09-19T09:03:11.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.314+0000 7f780a7d2740 1 -- start start 2024-09-19T09:03:11.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.314+0000 7f780a7d2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56059e1739d0 0x56059e173db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:11.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.314+0000 7f780a7d2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56059e1742f0 0x56059e17e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:11.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.314+0000 7f780a7d2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56059e17ebb0 0x56059e180fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:11.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.314+0000 7f780a7d2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56059e0da090 con 0x56059e17ebb0 2024-09-19T09:03:11.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.314+0000 7f780a7d2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56059e081a70 con 0x56059e1742f0 2024-09-19T09:03:11.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.314+0000 7f780a7d2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56059e0d6050 con 0x56059e1739d0 2024-09-19T09:03:11.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.314+0000 7f7805994640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56059e17ebb0 0x56059e180fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:11.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.314+0000 7f7805994640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56059e17ebb0 0x56059e180fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59758/0 (socket says 172.21.15.120:59758) 2024-09-19T09:03:11.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.314+0000 7f7805994640 1 -- 172.21.15.120:0/1382053806 learned_addr learned my addr 172.21.15.120:0/1382053806 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:11.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.314+0000 7f7806195640 1 --2- 172.21.15.120:0/1382053806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56059e1742f0 0x56059e17e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:11.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.314+0000 7f77f77fe640 1 --2- 172.21.15.120:0/3026997029 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f77ec0a32d0 0x7f77ec0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:11.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.314+0000 7f7806996640 1 --2- 172.21.15.120:0/1382053806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56059e1739d0 0x56059e173db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:11.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.314+0000 7f7805994640 1 -- 172.21.15.120:0/1382053806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56059e1739d0 msgr2=0x56059e173db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:11.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.314+0000 7f7805994640 1 --2- 172.21.15.120:0/1382053806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56059e1739d0 0x56059e173db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.314+0000 7f7805994640 1 -- 172.21.15.120:0/1382053806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56059e1742f0 msgr2=0x56059e17e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:11.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.315+0000 7f7805994640 1 --2- 172.21.15.120:0/1382053806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56059e1742f0 0x56059e17e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.315+0000 7f7805994640 1 -- 172.21.15.120:0/1382053806 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56059e08e130 con 0x56059e17ebb0 2024-09-19T09:03:11.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.315+0000 7f7806195640 1 --2- 172.21.15.120:0/1382053806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56059e1742f0 0x56059e17e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:03:11.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.315+0000 7f7806996640 1 --2- 172.21.15.120:0/3026997029 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f77ec0a32d0 0x7f77ec0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:11.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.315+0000 7f77f77fe640 1 -- 172.21.15.120:0/3026997029 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f77f00f0180 con 0x7f7800100a80 2024-09-19T09:03:11.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.315+0000 7f7805994640 1 --2- 172.21.15.120:0/1382053806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56059e17ebb0 0x56059e180fa0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00cbc0 tx=0x7f77fc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6eb6b77d817be8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:11.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.315+0000 7f77f4dea640 1 -- 172.21.15.120:0/1382053806 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc010630 con 0x56059e17ebb0 2024-09-19T09:03:11.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.315+0000 7f77f4dea640 1 -- 172.21.15.120:0/1382053806 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77fc010cb0 con 0x56059e17ebb0 2024-09-19T09:03:11.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.315+0000 7f77f4dea640 1 -- 172.21.15.120:0/1382053806 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc01ac70 con 0x56059e17ebb0 2024-09-19T09:03:11.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.316+0000 7f780a7d2740 1 -- 172.21.15.120:0/1382053806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56059e17ebb0 msgr2=0x56059e180fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:11.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.316+0000 7f780a7d2740 1 --2- 172.21.15.120:0/1382053806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56059e17ebb0 0x56059e180fa0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00cbc0 tx=0x7f77fc00e720 comp rx=0 tx=0).stop 2024-09-19T09:03:11.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.316+0000 7f7806996640 1 --2- 172.21.15.120:0/3026997029 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f77ec0a32d0 0x7f77ec0a5790 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f77f800acc0 tx=0x7f77f8004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:11.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.316+0000 7f780a7d2740 1 -- 172.21.15.120:0/1382053806 shutdown_connections 2024-09-19T09:03:11.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.316+0000 7f780a7d2740 1 --2- 172.21.15.120:0/1382053806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56059e17ebb0 0x56059e180fa0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.316+0000 7f780a7d2740 1 --2- 172.21.15.120:0/1382053806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56059e1742f0 0x56059e17e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.316+0000 7f780a7d2740 1 --2- 172.21.15.120:0/1382053806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56059e1739d0 0x56059e173db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.316+0000 7f780a7d2740 1 -- 172.21.15.120:0/1382053806 >> 172.21.15.120:0/1382053806 conn(0x56059e16ff60 msgr2=0x56059e170340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:11.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.316+0000 7f780a7d2740 1 -- 172.21.15.120:0/1382053806 shutdown_connections 2024-09-19T09:03:11.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.316+0000 7f780a7d2740 1 -- 172.21.15.120:0/1382053806 wait complete. 2024-09-19T09:03:11.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.316+0000 7f780a7d2740 1 -- start start 2024-09-19T09:03:11.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.316+0000 7f780a7d2740 1 Processor -- start 2024-09-19T09:03:11.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.316+0000 7f780a7d2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56059e1739d0 0x56059e17e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:11.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.316+0000 7f780a7d2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56059e1742f0 0x56059e17b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:11.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.316+0000 7f780a7d2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56059e17ebb0 0x56059e17bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:11.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.316+0000 7f780a7d2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56059e183990 con 0x56059e1742f0 2024-09-19T09:03:11.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.316+0000 7f780a7d2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56059e094f20 con 0x56059e17ebb0 2024-09-19T09:03:11.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.316+0000 7f780a7d2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56059e183b10 con 0x56059e1739d0 2024-09-19T09:03:11.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.317+0000 7f780a7d2740 10 client.0 _ll_register_callbacks cb 0x56059e05e030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:11.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.317+0000 7f7805994640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56059e1742f0 0x56059e17b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:11.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.317+0000 7f7805994640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56059e1742f0 0x56059e17b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59770/0 (socket says 172.21.15.120:59770) 2024-09-19T09:03:11.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.317+0000 7f7805994640 1 -- 172.21.15.120:0/3951090780 learned_addr learned my addr 172.21.15.120:0/3951090780 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:11.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.317+0000 7f7806195640 1 --2- 172.21.15.120:0/3951090780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56059e1739d0 0x56059e17e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:11.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.317+0000 7f7806996640 1 --2- 172.21.15.120:0/3951090780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56059e17ebb0 0x56059e17bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:11.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.317+0000 7f7805994640 1 -- 172.21.15.120:0/3951090780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56059e1739d0 msgr2=0x56059e17e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:11.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.317+0000 7f7805994640 1 --2- 172.21.15.120:0/3951090780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56059e1739d0 0x56059e17e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.317+0000 7f7805994640 1 -- 172.21.15.120:0/3951090780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56059e17ebb0 msgr2=0x56059e17bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:11.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.317+0000 7f7805994640 1 --2- 172.21.15.120:0/3951090780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56059e17ebb0 0x56059e17bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.317+0000 7f7805994640 1 -- 172.21.15.120:0/3951090780 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56059de4b2b0 con 0x56059e1742f0 2024-09-19T09:03:11.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.317+0000 7f7806195640 1 --2- 172.21.15.120:0/3951090780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56059e1739d0 0x56059e17e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:03:11.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.317+0000 7f7805994640 1 --2- 172.21.15.120:0/3951090780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56059e1742f0 0x56059e17b710 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f77fc01a890 tx=0x7f77fc005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d602b5653fb82f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:11.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.317+0000 7f77e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:11.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.317+0000 7f77e9ffb640 1 -- 172.21.15.120:0/3951090780 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc010e60 con 0x56059e1742f0 2024-09-19T09:03:11.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.317+0000 7f77e9ffb640 1 -- 172.21.15.120:0/3951090780 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f77fc005260 con 0x56059e1742f0 2024-09-19T09:03:11.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.317+0000 7f780a7d2740 1 -- 172.21.15.120:0/3951090780 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77fc010cb0 con 0x56059e1742f0 2024-09-19T09:03:11.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.317+0000 7f77e9ffb640 1 -- 172.21.15.120:0/3951090780 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f77fc01a410 con 0x56059e1742f0 2024-09-19T09:03:11.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.317+0000 7f780a7d2740 10 client.15444 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:11.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.317+0000 7f780a7d2740 1 -- 172.21.15.120:0/3951090780 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56059e18f220 con 0x56059e1742f0 2024-09-19T09:03:11.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.317+0000 7f77e9ffb640 1 -- 172.21.15.120:0/3951090780 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77fc010cb0 con 0x56059e1742f0 2024-09-19T09:03:11.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.318+0000 7f77e9ffb640 1 -- 172.21.15.120:0/3951090780 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f77fc02c050 con 0x56059e1742f0 2024-09-19T09:03:11.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.318+0000 7f780a7d2740 10 client.15444 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:11.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.318+0000 7f780a7d2740 10 client.15444 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T09:03:11.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.318+0000 7f780a7d2740 20 client.15444 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:11.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.318+0000 7f780a7d2740 1 --2- 172.21.15.120:0/3951090780 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x56059e191480 0x56059e1928f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:11.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.318+0000 7f780a7d2740 4 client.15444 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:03:11.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.318+0000 7f780a7d2740 1 -- 172.21.15.120:0/3951090780 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56059e0e7600 con 0x56059e191480 2024-09-19T09:03:11.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.318+0000 7f7806195640 1 --2- 172.21.15.120:0/3951090780 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x56059e191480 0x56059e1928f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:11.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.318+0000 7f77e9ffb640 10 client.15444 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:03:11.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.318+0000 7f7806195640 1 --2- 172.21.15.120:0/3951090780 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x56059e191480 0x56059e1928f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5c9724701b577f98 server_cookie=603ff66b8d97e88c in_seq=0 out_seq=0 2024-09-19T09:03:11.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.319+0000 7f77e9ffb640 1 -- 172.21.15.120:0/3951090780 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56059e0e7600 con 0x56059e191480 2024-09-19T09:03:11.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.319+0000 7f77e9ffb640 10 client.15444 handle_command_reply: tid=0 2024-09-19T09:03:11.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.319+0000 7f780a7d2740 1 client.15444 shutdown 2024-09-19T09:03:11.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.319+0000 7f780a7d2740 1 -- 172.21.15.120:0/3951090780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56059e1742f0 msgr2=0x56059e17b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:11.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.319+0000 7f780a7d2740 1 --2- 172.21.15.120:0/3951090780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56059e1742f0 0x56059e17b710 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f77fc01a890 tx=0x7f77fc005d40 comp rx=0 tx=0).stop 2024-09-19T09:03:11.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.319+0000 7f780a7d2740 1 -- 172.21.15.120:0/3951090780 shutdown_connections 2024-09-19T09:03:11.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.319+0000 7f780a7d2740 1 --2- 172.21.15.120:0/3951090780 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x56059e191480 0x56059e1928f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.319+0000 7f780a7d2740 1 --2- 172.21.15.120:0/3951090780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56059e17ebb0 0x56059e17bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.319+0000 7f780a7d2740 1 --2- 172.21.15.120:0/3951090780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56059e1742f0 0x56059e17b710 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.319+0000 7f77e9ffb640 0 client.15444 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:03:11.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.319+0000 7f780a7d2740 1 --2- 172.21.15.120:0/3951090780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56059e1739d0 0x56059e17e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.319+0000 7f780a7d2740 1 -- 172.21.15.120:0/3951090780 >> 172.21.15.120:0/3951090780 conn(0x56059e16ff60 msgr2=0x56059e175c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:11.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.319+0000 7f780a7d2740 1 -- 172.21.15.120:0/3951090780 shutdown_connections 2024-09-19T09:03:11.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.319+0000 7f780a7d2740 1 -- 172.21.15.120:0/3951090780 wait complete. 2024-09-19T09:03:11.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.320+0000 7f780a7d2740 20 client.15444 trim_cache size 0 max 16384 2024-09-19T09:03:11.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.337+0000 7f780a7d2740 1 Processor -- start 2024-09-19T09:03:11.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.337+0000 7f780a7d2740 1 -- start start 2024-09-19T09:03:11.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.337+0000 7f780a7d2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56059e1739d0 0x56059e1928f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:11.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.337+0000 7f780a7d2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56059e1742f0 0x56059e16ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:11.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.337+0000 7f780a7d2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56059e17ebb0 0x56059e17b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:11.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.337+0000 7f780a7d2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56059e190c30 con 0x56059e17ebb0 2024-09-19T09:03:11.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.337+0000 7f780a7d2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56059e191300 con 0x56059e1742f0 2024-09-19T09:03:11.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.337+0000 7f780a7d2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56059e1919d0 con 0x56059e1739d0 2024-09-19T09:03:11.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.337+0000 7f7806996640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56059e17ebb0 0x56059e17b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:11.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.337+0000 7f7806996640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56059e17ebb0 0x56059e17b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59780/0 (socket says 172.21.15.120:59780) 2024-09-19T09:03:11.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.337+0000 7f7806996640 1 -- 172.21.15.120:0/3241938625 learned_addr learned my addr 172.21.15.120:0/3241938625 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:11.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.337+0000 7f7806195640 1 --2- 172.21.15.120:0/3241938625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56059e1739d0 0x56059e1928f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:11.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.338+0000 7f7805994640 1 --2- 172.21.15.120:0/3241938625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56059e1742f0 0x56059e16ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:11.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.338+0000 7f7806996640 1 -- 172.21.15.120:0/3241938625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56059e1739d0 msgr2=0x56059e1928f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:11.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.338+0000 7f7806996640 1 --2- 172.21.15.120:0/3241938625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56059e1739d0 0x56059e1928f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.338+0000 7f7806996640 1 -- 172.21.15.120:0/3241938625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56059e1742f0 msgr2=0x56059e16ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:11.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.338+0000 7f7806996640 1 --2- 172.21.15.120:0/3241938625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56059e1742f0 0x56059e16ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.338+0000 7f7806996640 1 -- 172.21.15.120:0/3241938625 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56059e174120 con 0x56059e17ebb0 2024-09-19T09:03:11.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.338+0000 7f7806996640 1 --2- 172.21.15.120:0/3241938625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56059e17ebb0 0x56059e17b6e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f77f80115d0 tx=0x7f77f8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b50267b28deddf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:11.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.338+0000 7f77ebfff640 1 -- 172.21.15.120:0/3241938625 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f801d920 con 0x56059e17ebb0 2024-09-19T09:03:11.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.338+0000 7f77ebfff640 1 -- 172.21.15.120:0/3241938625 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77f8022070 con 0x56059e17ebb0 2024-09-19T09:03:11.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.338+0000 7f77ebfff640 1 -- 172.21.15.120:0/3241938625 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f801e440 con 0x56059e17ebb0 2024-09-19T09:03:11.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.339+0000 7f780a7d2740 1 -- 172.21.15.120:0/3241938625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56059e17ebb0 msgr2=0x56059e17b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:11.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.339+0000 7f780a7d2740 1 --2- 172.21.15.120:0/3241938625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56059e17ebb0 0x56059e17b6e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f77f80115d0 tx=0x7f77f8012000 comp rx=0 tx=0).stop 2024-09-19T09:03:11.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.339+0000 7f780a7d2740 1 -- 172.21.15.120:0/3241938625 shutdown_connections 2024-09-19T09:03:11.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.339+0000 7f780a7d2740 1 --2- 172.21.15.120:0/3241938625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56059e17ebb0 0x56059e17b6e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.339+0000 7f780a7d2740 1 --2- 172.21.15.120:0/3241938625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56059e1742f0 0x56059e16ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.339+0000 7f780a7d2740 1 --2- 172.21.15.120:0/3241938625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56059e1739d0 0x56059e1928f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.339+0000 7f780a7d2740 1 -- 172.21.15.120:0/3241938625 >> 172.21.15.120:0/3241938625 conn(0x56059e16ff60 msgr2=0x56059e17afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:11.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.339+0000 7f780a7d2740 1 -- 172.21.15.120:0/3241938625 shutdown_connections 2024-09-19T09:03:11.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.339+0000 7f780a7d2740 1 -- 172.21.15.120:0/3241938625 wait complete. 2024-09-19T09:03:11.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.339+0000 7f780a7d2740 1 -- start start 2024-09-19T09:03:11.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.339+0000 7f780a7d2740 1 Processor -- start 2024-09-19T09:03:11.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.340+0000 7f780a7d2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56059e1739d0 0x56059e19b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:11.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.340+0000 7f780a7d2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56059e1742f0 0x56059e196410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:11.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.340+0000 7f780a7d2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56059e17ebb0 0x56059e196950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:11.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.340+0000 7f780a7d2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56059e192770 con 0x56059e1742f0 2024-09-19T09:03:11.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.340+0000 7f780a7d2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56059e1920a0 con 0x56059e1739d0 2024-09-19T09:03:11.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.340+0000 7f780a7d2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56059e192e40 con 0x56059e17ebb0 2024-09-19T09:03:11.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.340+0000 7f780a7d2740 10 client.0 _ll_register_callbacks cb 0x7f77fc014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:11.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.340+0000 7f7805994640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56059e1742f0 0x56059e196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:11.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.340+0000 7f7805994640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56059e1742f0 0x56059e196410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59796/0 (socket says 172.21.15.120:59796) 2024-09-19T09:03:11.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.340+0000 7f7805994640 1 -- 172.21.15.120:0/3227141217 learned_addr learned my addr 172.21.15.120:0/3227141217 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:11.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.340+0000 7f7806195640 1 --2- 172.21.15.120:0/3227141217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56059e1739d0 0x56059e19b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:11.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.340+0000 7f7806996640 1 --2- 172.21.15.120:0/3227141217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56059e17ebb0 0x56059e196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:11.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.340+0000 7f7805994640 1 -- 172.21.15.120:0/3227141217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56059e17ebb0 msgr2=0x56059e196950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:11.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.340+0000 7f7805994640 1 --2- 172.21.15.120:0/3227141217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56059e17ebb0 0x56059e196950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.340+0000 7f7805994640 1 -- 172.21.15.120:0/3227141217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56059e1739d0 msgr2=0x56059e19b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:11.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.340+0000 7f7805994640 1 --2- 172.21.15.120:0/3227141217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56059e1739d0 0x56059e19b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.340+0000 7f7805994640 1 -- 172.21.15.120:0/3227141217 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56059e196f10 con 0x56059e1742f0 2024-09-19T09:03:11.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.340+0000 7f7806996640 1 --2- 172.21.15.120:0/3227141217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56059e17ebb0 0x56059e196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:03:11.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.340+0000 7f7805994640 1 --2- 172.21.15.120:0/3227141217 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56059e1742f0 0x56059e196410 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f77fc004710 tx=0x7f77fc010640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76eb4542c4b2dde server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:11.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.340+0000 7f77e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:11.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.340+0000 7f77e9ffb640 1 -- 172.21.15.120:0/3227141217 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc01a6f0 con 0x56059e1742f0 2024-09-19T09:03:11.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.340+0000 7f77e9ffb640 1 -- 172.21.15.120:0/3227141217 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f77fc01ae00 con 0x56059e1742f0 2024-09-19T09:03:11.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.340+0000 7f780a7d2740 1 -- 172.21.15.120:0/3227141217 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77f8022070 con 0x56059e1742f0 2024-09-19T09:03:11.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.341+0000 7f77e9ffb640 1 -- 172.21.15.120:0/3227141217 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f77fc00c430 con 0x56059e1742f0 2024-09-19T09:03:11.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.341+0000 7f780a7d2740 10 client.15450 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:11.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.341+0000 7f780a7d2740 1 -- 172.21.15.120:0/3227141217 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56059e1fb920 con 0x56059e1742f0 2024-09-19T09:03:11.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.341+0000 7f77e9ffb640 1 -- 172.21.15.120:0/3227141217 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77f8022070 con 0x56059e1742f0 2024-09-19T09:03:11.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.341+0000 7f77e9ffb640 1 -- 172.21.15.120:0/3227141217 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f77fc02c040 con 0x56059e1742f0 2024-09-19T09:03:11.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.341+0000 7f780a7d2740 10 client.15450 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:11.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.341+0000 7f780a7d2740 10 client.15450 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T09:03:11.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.341+0000 7f780a7d2740 20 client.15450 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:11.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.341+0000 7f780a7d2740 1 --2- 172.21.15.120:0/3227141217 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x56059e2004c0 0x56059e1ff1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:11.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.341+0000 7f780a7d2740 4 client.15450 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:03:11.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.341+0000 7f780a7d2740 1 -- 172.21.15.120:0/3227141217 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "status"}) -- 0x56059e0e1b10 con 0x56059e2004c0 2024-09-19T09:03:11.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.341+0000 7f7806195640 1 --2- 172.21.15.120:0/3227141217 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x56059e2004c0 0x56059e1ff1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:11.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.342+0000 7f77e9ffb640 10 client.15450 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:03:11.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.342+0000 7f7806195640 1 --2- 172.21.15.120:0/3227141217 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x56059e2004c0 0x56059e1ff1d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8012ffc0be5fbe0f server_cookie=125e17c3755e4ac in_seq=0 out_seq=0 2024-09-19T09:03:11.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.342+0000 7f77e9ffb640 1 -- 172.21.15.120:0/3227141217 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56059e0e1b10 con 0x56059e2004c0 2024-09-19T09:03:11.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.342+0000 7f77e9ffb640 10 client.15450 handle_command_reply: tid=0 2024-09-19T09:03:11.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.342+0000 7f780a7d2740 1 client.15450 shutdown 2024-09-19T09:03:11.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.342+0000 7f780a7d2740 1 -- 172.21.15.120:0/3227141217 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56059e1742f0 msgr2=0x56059e196410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:11.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.342+0000 7f780a7d2740 1 --2- 172.21.15.120:0/3227141217 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56059e1742f0 0x56059e196410 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f77fc004710 tx=0x7f77fc010640 comp rx=0 tx=0).stop 2024-09-19T09:03:11.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.342+0000 7f780a7d2740 1 -- 172.21.15.120:0/3227141217 shutdown_connections 2024-09-19T09:03:11.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.342+0000 7f780a7d2740 1 --2- 172.21.15.120:0/3227141217 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x56059e2004c0 0x56059e1ff1d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.342+0000 7f77e9ffb640 0 client.15450 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:03:11.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.342+0000 7f780a7d2740 1 --2- 172.21.15.120:0/3227141217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56059e17ebb0 0x56059e196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.342+0000 7f780a7d2740 1 --2- 172.21.15.120:0/3227141217 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56059e1742f0 0x56059e196410 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.342+0000 7f780a7d2740 1 --2- 172.21.15.120:0/3227141217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56059e1739d0 0x56059e19b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.342+0000 7f780a7d2740 1 -- 172.21.15.120:0/3227141217 >> 172.21.15.120:0/3227141217 conn(0x56059e16ff60 msgr2=0x56059e17b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:11.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.342+0000 7f780a7d2740 1 -- 172.21.15.120:0/3227141217 shutdown_connections 2024-09-19T09:03:11.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.342+0000 7f780a7d2740 1 -- 172.21.15.120:0/3227141217 wait complete. 2024-09-19T09:03:11.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.342+0000 7f780a7d2740 20 client.15450 trim_cache size 0 max 16384 2024-09-19T09:03:11.351 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:03:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_fsid": "507393d2-7664-11ef-bceb-c7b262605968", 2024-09-19T09:03:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: "whoami": 1, 2024-09-19T09:03:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: "id": 24446, 2024-09-19T09:03:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: "want_state": "up:active", 2024-09-19T09:03:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: "state": "up:active", 2024-09-19T09:03:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: "fs_name": "cephfs", 2024-09-19T09:03:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: "rank_uptime": 136.24303309999999, 2024-09-19T09:03:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: "mdsmap_epoch": 28, 2024-09-19T09:03:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch": 81, 2024-09-19T09:03:11.352 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch_barrier": 81, 2024-09-19T09:03:11.352 INFO:teuthology.orchestra.run.smithi120.stdout: "uptime": 165.37471560700001 2024-09-19T09:03:11.352 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:03:11.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.343+0000 7f78083f9640 1 -- 172.21.15.120:0/3026997029 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f77ec0a32d0 msgr2=0x7f77ec0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:11.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.343+0000 7f78083f9640 1 --2- 172.21.15.120:0/3026997029 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f77ec0a32d0 0x7f77ec0a5790 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f77f800acc0 tx=0x7f77f8004a40 comp rx=0 tx=0).stop 2024-09-19T09:03:11.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.343+0000 7f78083f9640 1 -- 172.21.15.120:0/3026997029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7800100a80 msgr2=0x7f7800197360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:11.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.343+0000 7f78083f9640 1 --2- 172.21.15.120:0/3026997029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7800100a80 0x7f7800197360 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f77f0009a50 tx=0x7f77f0004270 comp rx=0 tx=0).stop 2024-09-19T09:03:11.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.343+0000 7f78083f9640 1 -- 172.21.15.120:0/3026997029 shutdown_connections 2024-09-19T09:03:11.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.343+0000 7f78083f9640 1 --2- 172.21.15.120:0/3026997029 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f77ec0a32d0 0x7f77ec0a5790 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.343+0000 7f78083f9640 1 --2- 172.21.15.120:0/3026997029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7800106a80 0x7f780019bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.343+0000 7f78083f9640 1 --2- 172.21.15.120:0/3026997029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7800101420 0x7f78001978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.343+0000 7f78083f9640 1 --2- 172.21.15.120:0/3026997029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7800100a80 0x7f7800197360 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:11.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.343+0000 7f78083f9640 1 -- 172.21.15.120:0/3026997029 >> 172.21.15.120:0/3026997029 conn(0x7f78000fc7a0 msgr2=0x7f78000fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:11.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.343+0000 7f78083f9640 1 -- 172.21.15.120:0/3026997029 shutdown_connections 2024-09-19T09:03:11.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:11.343+0000 7f78083f9640 1 -- 172.21.15.120:0/3026997029 wait complete. 2024-09-19T09:03:12.065 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T09:03:12.311 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:03:12.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.786+0000 7f136335a640 1 -- 172.21.15.120:0/2271890190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f135c106f00 msgr2=0x7f135c10ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:12.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.786+0000 7f136335a640 1 --2- 172.21.15.120:0/2271890190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f135c106f00 0x7f135c10ea80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1354009ca0 tx=0x7f135402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.786+0000 7f136335a640 1 -- 172.21.15.120:0/2271890190 shutdown_connections 2024-09-19T09:03:12.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.786+0000 7f136335a640 1 --2- 172.21.15.120:0/2271890190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f135c106f00 0x7f135c10ea80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.786+0000 7f136335a640 1 --2- 172.21.15.120:0/2271890190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f135c102780 0x7f135c1067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.786+0000 7f136335a640 1 --2- 172.21.15.120:0/2271890190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f135c101dd0 0x7f135c1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.786+0000 7f136335a640 1 -- 172.21.15.120:0/2271890190 >> 172.21.15.120:0/2271890190 conn(0x7f135c0fd8b0 msgr2=0x7f135c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:12.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.786+0000 7f136335a640 1 -- 172.21.15.120:0/2271890190 shutdown_connections 2024-09-19T09:03:12.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.786+0000 7f136335a640 1 -- 172.21.15.120:0/2271890190 wait complete. 2024-09-19T09:03:12.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.787+0000 7f136335a640 1 Processor -- start 2024-09-19T09:03:12.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.787+0000 7f136335a640 1 -- start start 2024-09-19T09:03:12.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.787+0000 7f136335a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f135c101dd0 0x7f135c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:12.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.787+0000 7f136335a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f135c102780 0x7f135c198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:12.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.787+0000 7f136335a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f135c106f00 0x7f135c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:12.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.787+0000 7f136335a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f135c111f60 con 0x7f135c106f00 2024-09-19T09:03:12.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.787+0000 7f136335a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f135c111de0 con 0x7f135c102780 2024-09-19T09:03:12.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.787+0000 7f136335a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f135c1120e0 con 0x7f135c101dd0 2024-09-19T09:03:12.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.787+0000 7f13618f7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f135c106f00 0x7f135c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:12.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.787+0000 7f13618f7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f135c106f00 0x7f135c19ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59814/0 (socket says 172.21.15.120:59814) 2024-09-19T09:03:12.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.787+0000 7f13618f7640 1 -- 172.21.15.120:0/3688191506 learned_addr learned my addr 172.21.15.120:0/3688191506 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:12.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.787+0000 7f13610f6640 1 --2- 172.21.15.120:0/3688191506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f135c101dd0 0x7f135c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:12.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.788+0000 7f13608f5640 1 --2- 172.21.15.120:0/3688191506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f135c102780 0x7f135c198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:12.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.788+0000 7f13618f7640 1 -- 172.21.15.120:0/3688191506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f135c101dd0 msgr2=0x7f135c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:12.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.788+0000 7f13618f7640 1 --2- 172.21.15.120:0/3688191506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f135c101dd0 0x7f135c1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.788+0000 7f13618f7640 1 -- 172.21.15.120:0/3688191506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f135c102780 msgr2=0x7f135c198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:12.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.788+0000 7f13618f7640 1 --2- 172.21.15.120:0/3688191506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f135c102780 0x7f135c198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.788+0000 7f13618f7640 1 -- 172.21.15.120:0/3688191506 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135c19d390 con 0x7f135c106f00 2024-09-19T09:03:12.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.788+0000 7f13608f5640 1 --2- 172.21.15.120:0/3688191506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f135c102780 0x7f135c198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:03:12.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.788+0000 7f13618f7640 1 --2- 172.21.15.120:0/3688191506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f135c106f00 0x7f135c19ccb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1354031a20 tx=0x7f1354031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:12.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.788+0000 7f13527fc640 1 -- 172.21.15.120:0/3688191506 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1354003b70 con 0x7f135c106f00 2024-09-19T09:03:12.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.788+0000 7f136335a640 1 -- 172.21.15.120:0/3688191506 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f135c19d620 con 0x7f135c106f00 2024-09-19T09:03:12.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.788+0000 7f136335a640 1 -- 172.21.15.120:0/3688191506 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f135c1a4f60 con 0x7f135c106f00 2024-09-19T09:03:12.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.788+0000 7f13527fc640 1 -- 172.21.15.120:0/3688191506 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1354003d10 con 0x7f135c106f00 2024-09-19T09:03:12.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.788+0000 7f13527fc640 1 -- 172.21.15.120:0/3688191506 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135402fc40 con 0x7f135c106f00 2024-09-19T09:03:12.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.790+0000 7f13527fc640 1 -- 172.21.15.120:0/3688191506 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f1354004050 con 0x7f135c106f00 2024-09-19T09:03:12.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.791+0000 7f1365733740 1 Processor -- start 2024-09-19T09:03:12.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.791+0000 7f1365733740 1 -- start start 2024-09-19T09:03:12.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.791+0000 7f1365733740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56210ff539d0 0x56210ff53db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:12.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.791+0000 7f1365733740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56210ff542f0 0x56210ff5e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:12.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.791+0000 7f1365733740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56210ff5ebb0 0x56210ff60fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:12.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.791+0000 7f1365733740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56210feba090 con 0x56210ff539d0 2024-09-19T09:03:12.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.791+0000 7f1365733740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56210fe61a70 con 0x56210ff5ebb0 2024-09-19T09:03:12.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.791+0000 7f1365733740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56210feb6050 con 0x56210ff542f0 2024-09-19T09:03:12.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.791+0000 7f13608f5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56210ff539d0 0x56210ff53db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:12.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.791+0000 7f13608f5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56210ff539d0 0x56210ff53db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59828/0 (socket says 172.21.15.120:59828) 2024-09-19T09:03:12.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.791+0000 7f13608f5640 1 -- 172.21.15.120:0/383723184 learned_addr learned my addr 172.21.15.120:0/383723184 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:12.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.791+0000 7f13527fc640 1 --2- 172.21.15.120:0/3688191506 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f13480a3350 0x7f13480a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:12.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.792+0000 7f13610f6640 1 --2- 172.21.15.120:0/383723184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56210ff542f0 0x56210ff5e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:12.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.792+0000 7f13618f7640 1 --2- 172.21.15.120:0/3688191506 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f13480a3350 0x7f13480a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:12.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.792+0000 7f13608f5640 1 -- 172.21.15.120:0/383723184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56210ff542f0 msgr2=0x56210ff5e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:12.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.792+0000 7f13608f5640 1 --2- 172.21.15.120:0/383723184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56210ff542f0 0x56210ff5e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.792+0000 7f13608f5640 1 -- 172.21.15.120:0/383723184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56210ff5ebb0 msgr2=0x56210ff60fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:12.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.792+0000 7f13608f5640 1 --2- 172.21.15.120:0/383723184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56210ff5ebb0 0x56210ff60fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.792+0000 7f13527fc640 1 -- 172.21.15.120:0/3688191506 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f13540f0c30 con 0x7f135c106f00 2024-09-19T09:03:12.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.792+0000 7f13608f5640 1 -- 172.21.15.120:0/383723184 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56210fe6e130 con 0x56210ff539d0 2024-09-19T09:03:12.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.792+0000 7f13608f5640 1 --2- 172.21.15.120:0/383723184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56210ff539d0 0x56210ff53db0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f134c009ce0 tx=0x7f134c012760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7ecd7b6bcff1ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:12.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.792+0000 7f13618f7640 1 --2- 172.21.15.120:0/3688191506 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f13480a3350 0x7f13480a5810 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f13540bcf90 tx=0x7f1354090110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:12.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.792+0000 7f13477fe640 1 -- 172.21.15.120:0/383723184 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c01b400 con 0x56210ff539d0 2024-09-19T09:03:12.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.792+0000 7f13477fe640 1 -- 172.21.15.120:0/383723184 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f134c0138e0 con 0x56210ff539d0 2024-09-19T09:03:12.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.792+0000 7f13477fe640 1 -- 172.21.15.120:0/383723184 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c00e930 con 0x56210ff539d0 2024-09-19T09:03:12.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.793+0000 7f1365733740 1 -- 172.21.15.120:0/383723184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56210ff539d0 msgr2=0x56210ff53db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:12.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.793+0000 7f1365733740 1 --2- 172.21.15.120:0/383723184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56210ff539d0 0x56210ff53db0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f134c009ce0 tx=0x7f134c012760 comp rx=0 tx=0).stop 2024-09-19T09:03:12.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.793+0000 7f1365733740 1 -- 172.21.15.120:0/383723184 shutdown_connections 2024-09-19T09:03:12.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.793+0000 7f1365733740 1 --2- 172.21.15.120:0/383723184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56210ff5ebb0 0x56210ff60fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.793+0000 7f1365733740 1 --2- 172.21.15.120:0/383723184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56210ff542f0 0x56210ff5e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.793+0000 7f1365733740 1 --2- 172.21.15.120:0/383723184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56210ff539d0 0x56210ff53db0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.793+0000 7f1365733740 1 -- 172.21.15.120:0/383723184 >> 172.21.15.120:0/383723184 conn(0x56210ff4ff60 msgr2=0x56210ff50340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:12.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.793+0000 7f1365733740 1 -- 172.21.15.120:0/383723184 shutdown_connections 2024-09-19T09:03:12.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.793+0000 7f1365733740 1 -- 172.21.15.120:0/383723184 wait complete. 2024-09-19T09:03:12.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.793+0000 7f1365733740 1 -- start start 2024-09-19T09:03:12.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.793+0000 7f1365733740 1 Processor -- start 2024-09-19T09:03:12.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.794+0000 7f1365733740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56210ff539d0 0x56210ff5e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:12.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.794+0000 7f1365733740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56210ff542f0 0x56210ff5b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:12.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.794+0000 7f1365733740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56210ff5ebb0 0x56210ff5bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:12.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.794+0000 7f1365733740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56210ff61780 con 0x56210ff539d0 2024-09-19T09:03:12.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.794+0000 7f1365733740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56210fe74f20 con 0x56210ff5ebb0 2024-09-19T09:03:12.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.794+0000 7f1365733740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56210ff61900 con 0x56210ff542f0 2024-09-19T09:03:12.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.794+0000 7f1365733740 10 client.0 _ll_register_callbacks cb 0x56210fe3e030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:12.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.794+0000 7f13608f5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56210ff539d0 0x56210ff5e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:12.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.794+0000 7f13608f5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56210ff539d0 0x56210ff5e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59834/0 (socket says 172.21.15.120:59834) 2024-09-19T09:03:12.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.794+0000 7f13608f5640 1 -- 172.21.15.120:0/2844167499 learned_addr learned my addr 172.21.15.120:0/2844167499 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:12.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.794+0000 7f13610f6640 1 --2- 172.21.15.120:0/2844167499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56210ff542f0 0x56210ff5b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:12.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.794+0000 7f13608f5640 1 --2- 172.21.15.120:0/2844167499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56210ff5ebb0 0x56210ff5bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:12.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.794+0000 7f13608f5640 1 -- 172.21.15.120:0/2844167499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56210ff542f0 msgr2=0x56210ff5b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:12.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.794+0000 7f13608f5640 1 --2- 172.21.15.120:0/2844167499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56210ff542f0 0x56210ff5b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.794+0000 7f13608f5640 1 -- 172.21.15.120:0/2844167499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56210ff5ebb0 msgr2=0x56210ff5bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:12.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.794+0000 7f13608f5640 1 --2- 172.21.15.120:0/2844167499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56210ff5ebb0 0x56210ff5bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.794+0000 7f13608f5640 1 -- 172.21.15.120:0/2844167499 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56210fc2b2b0 con 0x56210ff539d0 2024-09-19T09:03:12.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.795+0000 7f13608f5640 1 --2- 172.21.15.120:0/2844167499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56210ff539d0 0x56210ff5e4a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f134c00e410 tx=0x7f134c005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415bd8d3e4823146 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:12.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.795+0000 7f1344ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:12.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.795+0000 7f1344ff9640 1 -- 172.21.15.120:0/2844167499 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c01f030 con 0x56210ff539d0 2024-09-19T09:03:12.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.795+0000 7f1344ff9640 1 -- 172.21.15.120:0/2844167499 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f134c004020 con 0x56210ff539d0 2024-09-19T09:03:12.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.795+0000 7f1365733740 1 -- 172.21.15.120:0/2844167499 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f134c0138e0 con 0x56210ff539d0 2024-09-19T09:03:12.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.795+0000 7f1344ff9640 1 -- 172.21.15.120:0/2844167499 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f134c013e20 con 0x56210ff539d0 2024-09-19T09:03:12.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.795+0000 7f1365733740 10 client.15462 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:12.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.795+0000 7f1365733740 1 -- 172.21.15.120:0/2844167499 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56210ff6d0f0 con 0x56210ff539d0 2024-09-19T09:03:12.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.795+0000 7f1344ff9640 1 -- 172.21.15.120:0/2844167499 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f134c0138e0 con 0x56210ff539d0 2024-09-19T09:03:12.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.795+0000 7f1344ff9640 1 -- 172.21.15.120:0/2844167499 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f134c00f040 con 0x56210ff539d0 2024-09-19T09:03:12.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.795+0000 7f1365733740 10 client.15462 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:12.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.795+0000 7f1365733740 10 client.15462 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T09:03:12.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.795+0000 7f1365733740 20 client.15462 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:12.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.795+0000 7f1365733740 1 --2- 172.21.15.120:0/2844167499 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56210ff6f3d0 0x56210ff70840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:12.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.795+0000 7f1365733740 4 client.15462 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:03:12.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.795+0000 7f1365733740 1 -- 172.21.15.120:0/2844167499 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56210fec7600 con 0x56210ff6f3d0 2024-09-19T09:03:12.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.796+0000 7f13610f6640 1 --2- 172.21.15.120:0/2844167499 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56210ff6f3d0 0x56210ff70840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:12.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.796+0000 7f13610f6640 1 --2- 172.21.15.120:0/2844167499 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56210ff6f3d0 0x56210ff70840 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d20236bfd6c3966 server_cookie=a1cfabcd8af48512 in_seq=0 out_seq=0 2024-09-19T09:03:12.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.796+0000 7f1344ff9640 10 client.15462 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:12.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.797+0000 7f1344ff9640 1 -- 172.21.15.120:0/2844167499 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56210fec7600 con 0x56210ff6f3d0 2024-09-19T09:03:12.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.797+0000 7f1344ff9640 10 client.15462 handle_command_reply: tid=0 2024-09-19T09:03:12.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.797+0000 7f1365733740 1 client.15462 shutdown 2024-09-19T09:03:12.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.797+0000 7f1365733740 1 -- 172.21.15.120:0/2844167499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56210ff539d0 msgr2=0x56210ff5e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:12.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.797+0000 7f1365733740 1 --2- 172.21.15.120:0/2844167499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56210ff539d0 0x56210ff5e4a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f134c00e410 tx=0x7f134c005f10 comp rx=0 tx=0).stop 2024-09-19T09:03:12.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.797+0000 7f1365733740 1 -- 172.21.15.120:0/2844167499 shutdown_connections 2024-09-19T09:03:12.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.797+0000 7f1365733740 1 --2- 172.21.15.120:0/2844167499 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56210ff6f3d0 0x56210ff70840 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.797+0000 7f1365733740 1 --2- 172.21.15.120:0/2844167499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56210ff5ebb0 0x56210ff5bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.797+0000 7f1365733740 1 --2- 172.21.15.120:0/2844167499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56210ff542f0 0x56210ff5b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.797+0000 7f1365733740 1 --2- 172.21.15.120:0/2844167499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56210ff539d0 0x56210ff5e4a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.797+0000 7f1344ff9640 0 client.15462 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:12.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.797+0000 7f1365733740 1 -- 172.21.15.120:0/2844167499 >> 172.21.15.120:0/2844167499 conn(0x56210ff4ff60 msgr2=0x56210ff55c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:12.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.797+0000 7f1365733740 1 -- 172.21.15.120:0/2844167499 shutdown_connections 2024-09-19T09:03:12.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.797+0000 7f1365733740 1 -- 172.21.15.120:0/2844167499 wait complete. 2024-09-19T09:03:12.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.797+0000 7f1365733740 20 client.15462 trim_cache size 0 max 16384 2024-09-19T09:03:12.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.814+0000 7f1365733740 1 Processor -- start 2024-09-19T09:03:12.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.814+0000 7f1365733740 1 -- start start 2024-09-19T09:03:12.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.815+0000 7f1365733740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56210ff539d0 0x56210ff4ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:12.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.815+0000 7f1365733740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56210ff542f0 0x56210ff78f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:12.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.815+0000 7f1365733740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56210ff5ebb0 0x56210ff58a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:12.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.815+0000 7f1365733740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56210ff6eb80 con 0x56210ff5ebb0 2024-09-19T09:03:12.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.815+0000 7f1365733740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56210ff6f250 con 0x56210ff542f0 2024-09-19T09:03:12.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.815+0000 7f1365733740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56210ff6f920 con 0x56210ff539d0 2024-09-19T09:03:12.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.815+0000 7f13608f5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56210ff5ebb0 0x56210ff58a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:12.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.815+0000 7f13608f5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56210ff5ebb0 0x56210ff58a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59838/0 (socket says 172.21.15.120:59838) 2024-09-19T09:03:12.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.815+0000 7f13608f5640 1 -- 172.21.15.120:0/1986218186 learned_addr learned my addr 172.21.15.120:0/1986218186 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:12.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.815+0000 7f13608f5640 1 -- 172.21.15.120:0/1986218186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56210ff539d0 msgr2=0x56210ff4ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:12.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.815+0000 7f13608f5640 1 --2- 172.21.15.120:0/1986218186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56210ff539d0 0x56210ff4ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.815+0000 7f13608f5640 1 -- 172.21.15.120:0/1986218186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56210ff542f0 msgr2=0x56210ff78f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:12.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.815+0000 7f13608f5640 1 --2- 172.21.15.120:0/1986218186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56210ff542f0 0x56210ff78f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.815+0000 7f13608f5640 1 -- 172.21.15.120:0/1986218186 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56210ff5e0c0 con 0x56210ff5ebb0 2024-09-19T09:03:12.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.815+0000 7f13608f5640 1 --2- 172.21.15.120:0/1986218186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56210ff5ebb0 0x56210ff58a50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f134c00e460 tx=0x7f134c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=701ace6d9f636f63 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:12.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.815+0000 7f1346ffd640 1 -- 172.21.15.120:0/1986218186 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c009650 con 0x56210ff5ebb0 2024-09-19T09:03:12.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.815+0000 7f1346ffd640 1 -- 172.21.15.120:0/1986218186 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f134c027040 con 0x56210ff5ebb0 2024-09-19T09:03:12.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.815+0000 7f1346ffd640 1 -- 172.21.15.120:0/1986218186 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c0138c0 con 0x56210ff5ebb0 2024-09-19T09:03:12.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.816+0000 7f1365733740 1 -- 172.21.15.120:0/1986218186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56210ff5ebb0 msgr2=0x56210ff58a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:12.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.816+0000 7f1365733740 1 --2- 172.21.15.120:0/1986218186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56210ff5ebb0 0x56210ff58a50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f134c00e460 tx=0x7f134c004ac0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.816+0000 7f1365733740 1 -- 172.21.15.120:0/1986218186 shutdown_connections 2024-09-19T09:03:12.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.816+0000 7f1365733740 1 --2- 172.21.15.120:0/1986218186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56210ff5ebb0 0x56210ff58a50 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.816+0000 7f1365733740 1 --2- 172.21.15.120:0/1986218186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56210ff542f0 0x56210ff78f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.816+0000 7f1365733740 1 --2- 172.21.15.120:0/1986218186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56210ff539d0 0x56210ff4ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.816+0000 7f1365733740 1 -- 172.21.15.120:0/1986218186 >> 172.21.15.120:0/1986218186 conn(0x56210ff4ff60 msgr2=0x56210ff5a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:12.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.816+0000 7f1365733740 1 -- 172.21.15.120:0/1986218186 shutdown_connections 2024-09-19T09:03:12.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.816+0000 7f1365733740 1 -- 172.21.15.120:0/1986218186 wait complete. 2024-09-19T09:03:12.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.816+0000 7f1365733740 1 -- start start 2024-09-19T09:03:12.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.816+0000 7f1365733740 1 Processor -- start 2024-09-19T09:03:12.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.817+0000 7f1365733740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56210ff539d0 0x56210ff72a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:12.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.817+0000 7f1365733740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56210ff542f0 0x56210ff72fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:12.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.817+0000 7f1365733740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56210ff73510 0x56210ffdcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:12.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.817+0000 7f1365733740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56210ff706c0 con 0x56210ff542f0 2024-09-19T09:03:12.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.817+0000 7f1365733740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56210ff6fff0 con 0x56210ff539d0 2024-09-19T09:03:12.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.817+0000 7f1365733740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56210ff70d90 con 0x56210ff73510 2024-09-19T09:03:12.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.817+0000 7f1365733740 10 client.0 _ll_register_callbacks cb 0x7f134c009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:12.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.817+0000 7f13610f6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56210ff542f0 0x56210ff72fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:12.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.817+0000 7f13610f6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56210ff542f0 0x56210ff72fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59846/0 (socket says 172.21.15.120:59846) 2024-09-19T09:03:12.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.817+0000 7f13610f6640 1 -- 172.21.15.120:0/103260905 learned_addr learned my addr 172.21.15.120:0/103260905 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:12.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.817+0000 7f13608f5640 1 --2- 172.21.15.120:0/103260905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56210ff73510 0x56210ffdcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:12.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.817+0000 7f13608f5640 1 --2- 172.21.15.120:0/103260905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56210ff539d0 0x56210ff72a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:12.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.817+0000 7f13610f6640 1 -- 172.21.15.120:0/103260905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56210ff73510 msgr2=0x56210ffdcb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:12.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.817+0000 7f13610f6640 1 --2- 172.21.15.120:0/103260905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56210ff73510 0x56210ffdcb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.817+0000 7f13610f6640 1 -- 172.21.15.120:0/103260905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56210ff539d0 msgr2=0x56210ff72a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:12.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.817+0000 7f13610f6640 1 --2- 172.21.15.120:0/103260905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56210ff539d0 0x56210ff72a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.817+0000 7f13610f6640 1 -- 172.21.15.120:0/103260905 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56210ffdd050 con 0x56210ff542f0 2024-09-19T09:03:12.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.817+0000 7f13608f5640 1 --2- 172.21.15.120:0/103260905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56210ff73510 0x56210ffdcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:03:12.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.817+0000 7f13610f6640 1 --2- 172.21.15.120:0/103260905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56210ff542f0 0x56210ff72fd0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x56210ff5ce30 tx=0x7f1358004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9821805d1e60d7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:12.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.817+0000 7f1344ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:12.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.817+0000 7f1344ff9640 1 -- 172.21.15.120:0/103260905 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1358010630 con 0x56210ff542f0 2024-09-19T09:03:12.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.817+0000 7f1344ff9640 1 -- 172.21.15.120:0/103260905 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f13580107d0 con 0x56210ff542f0 2024-09-19T09:03:12.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.817+0000 7f1365733740 1 -- 172.21.15.120:0/103260905 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f134c027040 con 0x56210ff542f0 2024-09-19T09:03:12.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.818+0000 7f1344ff9640 1 -- 172.21.15.120:0/103260905 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f135802f050 con 0x56210ff542f0 2024-09-19T09:03:12.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.818+0000 7f1365733740 10 client.15468 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:12.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.818+0000 7f1365733740 1 -- 172.21.15.120:0/103260905 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56210ffde1e0 con 0x56210ff542f0 2024-09-19T09:03:12.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.818+0000 7f1344ff9640 1 -- 172.21.15.120:0/103260905 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f134c027040 con 0x56210ff542f0 2024-09-19T09:03:12.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.818+0000 7f1344ff9640 1 -- 172.21.15.120:0/103260905 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1358026070 con 0x56210ff542f0 2024-09-19T09:03:12.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.818+0000 7f1365733740 10 client.15468 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:12.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.818+0000 7f1365733740 10 client.15468 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T09:03:12.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.818+0000 7f1365733740 20 client.15468 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:12.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.818+0000 7f1365733740 1 --2- 172.21.15.120:0/103260905 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56210ffe2db0 0x56210ffe1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:12.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.818+0000 7f1365733740 4 client.15468 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:03:12.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.818+0000 7f1365733740 1 -- 172.21.15.120:0/103260905 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "status"}) -- 0x56210fec1b10 con 0x56210ffe2db0 2024-09-19T09:03:12.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.818+0000 7f13608f5640 1 --2- 172.21.15.120:0/103260905 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56210ffe2db0 0x56210ffe1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:12.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.819+0000 7f1344ff9640 10 client.15468 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:12.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.819+0000 7f13608f5640 1 --2- 172.21.15.120:0/103260905 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56210ffe2db0 0x56210ffe1ac0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d658ad80566e5500 server_cookie=cd7fd8d5d2ed9324 in_seq=0 out_seq=0 2024-09-19T09:03:12.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.819+0000 7f1344ff9640 1 -- 172.21.15.120:0/103260905 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56210fec1b10 con 0x56210ffe2db0 2024-09-19T09:03:12.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.819+0000 7f1344ff9640 10 client.15468 handle_command_reply: tid=0 2024-09-19T09:03:12.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.819+0000 7f1365733740 1 client.15468 shutdown 2024-09-19T09:03:12.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.819+0000 7f1365733740 1 -- 172.21.15.120:0/103260905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56210ff542f0 msgr2=0x56210ff72fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:12.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.819+0000 7f1365733740 1 --2- 172.21.15.120:0/103260905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56210ff542f0 0x56210ff72fd0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x56210ff5ce30 tx=0x7f1358004560 comp rx=0 tx=0).stop 2024-09-19T09:03:12.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.819+0000 7f1365733740 1 -- 172.21.15.120:0/103260905 shutdown_connections 2024-09-19T09:03:12.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.819+0000 7f1365733740 1 --2- 172.21.15.120:0/103260905 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56210ffe2db0 0x56210ffe1ac0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.819+0000 7f1365733740 1 --2- 172.21.15.120:0/103260905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56210ff73510 0x56210ffdcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.819+0000 7f1344ff9640 0 client.15468 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:12.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.819+0000 7f1365733740 1 --2- 172.21.15.120:0/103260905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56210ff542f0 0x56210ff72fd0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.819+0000 7f1365733740 1 --2- 172.21.15.120:0/103260905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56210ff539d0 0x56210ff72a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.819+0000 7f1365733740 1 -- 172.21.15.120:0/103260905 >> 172.21.15.120:0/103260905 conn(0x56210ff4ff60 msgr2=0x56210ff5a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:12.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.819+0000 7f1365733740 1 -- 172.21.15.120:0/103260905 shutdown_connections 2024-09-19T09:03:12.846 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:03:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_fsid": "507393d2-7664-11ef-bceb-c7b262605968", 2024-09-19T09:03:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "whoami": 0, 2024-09-19T09:03:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "id": 24461, 2024-09-19T09:03:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "want_state": "up:active", 2024-09-19T09:03:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "state": "up:active", 2024-09-19T09:03:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "fs_name": "cephfs", 2024-09-19T09:03:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "rank_uptime": 144.86383447899999, 2024-09-19T09:03:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "mdsmap_epoch": 28, 2024-09-19T09:03:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch": 81, 2024-09-19T09:03:12.847 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch_barrier": 81, 2024-09-19T09:03:12.847 INFO:teuthology.orchestra.run.smithi120.stdout: "uptime": 154.88816944199999 2024-09-19T09:03:12.847 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:03:12.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.819+0000 7f1365733740 1 -- 172.21.15.120:0/103260905 wait complete. 2024-09-19T09:03:12.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.819+0000 7f1365733740 20 client.15468 trim_cache size 0 max 16384 2024-09-19T09:03:12.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.820+0000 7f136335a640 1 -- 172.21.15.120:0/3688191506 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f13480a3350 msgr2=0x7f13480a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:12.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.820+0000 7f136335a640 1 --2- 172.21.15.120:0/3688191506 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f13480a3350 0x7f13480a5810 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f13540bcf90 tx=0x7f1354090110 comp rx=0 tx=0).stop 2024-09-19T09:03:12.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.820+0000 7f136335a640 1 -- 172.21.15.120:0/3688191506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f135c106f00 msgr2=0x7f135c19ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:12.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.820+0000 7f136335a640 1 --2- 172.21.15.120:0/3688191506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f135c106f00 0x7f135c19ccb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1354031a20 tx=0x7f1354031a50 comp rx=0 tx=0).stop 2024-09-19T09:03:12.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.820+0000 7f136335a640 1 -- 172.21.15.120:0/3688191506 shutdown_connections 2024-09-19T09:03:12.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.820+0000 7f136335a640 1 --2- 172.21.15.120:0/3688191506 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f13480a3350 0x7f13480a5810 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.820+0000 7f136335a640 1 --2- 172.21.15.120:0/3688191506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f135c106f00 0x7f135c19ccb0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.820+0000 7f136335a640 1 --2- 172.21.15.120:0/3688191506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f135c102780 0x7f135c198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.820+0000 7f136335a640 1 --2- 172.21.15.120:0/3688191506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f135c101dd0 0x7f135c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:12.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.820+0000 7f136335a640 1 -- 172.21.15.120:0/3688191506 >> 172.21.15.120:0/3688191506 conn(0x7f135c0fd8b0 msgr2=0x7f135c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:12.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.820+0000 7f136335a640 1 -- 172.21.15.120:0/3688191506 shutdown_connections 2024-09-19T09:03:12.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:12.820+0000 7f136335a640 1 -- 172.21.15.120:0/3688191506 wait complete. 2024-09-19T09:03:13.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:12 smithi120 ceph-mon[31150]: pgmap v351: 97 pgs: 97 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.6 KiB/s wr, 15 op/s 2024-09-19T09:03:13.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:12 smithi196 ceph-mon[35150]: pgmap v351: 97 pgs: 97 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.6 KiB/s wr, 15 op/s 2024-09-19T09:03:13.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:12 smithi153 ceph-mon[35071]: pgmap v351: 97 pgs: 97 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.6 KiB/s wr, 15 op/s 2024-09-19T09:03:13.567 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-19T09:03:15.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:14 smithi120 ceph-mon[31150]: pgmap v352: 97 pgs: 97 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.6 KiB/s wr, 14 op/s 2024-09-19T09:03:15.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:14 smithi196 ceph-mon[35150]: pgmap v352: 97 pgs: 97 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.6 KiB/s wr, 14 op/s 2024-09-19T09:03:15.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:14 smithi153 ceph-mon[35071]: pgmap v352: 97 pgs: 97 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.6 KiB/s wr, 14 op/s 2024-09-19T09:03:15.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.767+0000 7fbb104c0640 1 -- 172.21.15.120:0/463470131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb08102d50 msgr2=0x7fbb081031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:15.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.767+0000 7fbb104c0640 1 --2- 172.21.15.120:0/463470131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb08102d50 0x7fbb081031d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009900 tx=0x7fbafc02a3f0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.767+0000 7fbb104c0640 1 -- 172.21.15.120:0/463470131 shutdown_connections 2024-09-19T09:03:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.767+0000 7fbb104c0640 1 --2- 172.21.15.120:0/463470131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb08103710 0x7fbb08109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.767+0000 7fbb104c0640 1 --2- 172.21.15.120:0/463470131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb08102d50 0x7fbb081031d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.767+0000 7fbb104c0640 1 --2- 172.21.15.120:0/463470131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb08101b60 0x7fbb08101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.767+0000 7fbb104c0640 1 -- 172.21.15.120:0/463470131 >> 172.21.15.120:0/463470131 conn(0x7fbb080fd2e0 msgr2=0x7fbb080ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.767+0000 7fbb104c0640 1 -- 172.21.15.120:0/463470131 shutdown_connections 2024-09-19T09:03:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.767+0000 7fbb104c0640 1 -- 172.21.15.120:0/463470131 wait complete. 2024-09-19T09:03:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.768+0000 7fbb104c0640 1 Processor -- start 2024-09-19T09:03:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.768+0000 7fbb104c0640 1 -- start start 2024-09-19T09:03:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.768+0000 7fbb104c0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb08101b60 0x7fbb08197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.768+0000 7fbb104c0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb08102d50 0x7fbb081982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.768+0000 7fbb104c0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb08103710 0x7fbb0819f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.768+0000 7fbb104c0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0810d7c0 con 0x7fbb08102d50 2024-09-19T09:03:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.768+0000 7fbb104c0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0810d640 con 0x7fbb08101b60 2024-09-19T09:03:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.768+0000 7fbb104c0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0810d940 con 0x7fbb08103710 2024-09-19T09:03:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.768+0000 7fbb0da5b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb08102d50 0x7fbb081982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.768+0000 7fbb0da5b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb08102d50 0x7fbb081982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59854/0 (socket says 172.21.15.120:59854) 2024-09-19T09:03:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.768+0000 7fbb0da5b640 1 -- 172.21.15.120:0/2375065456 learned_addr learned my addr 172.21.15.120:0/2375065456 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.768+0000 7fbb0da5b640 1 -- 172.21.15.120:0/2375065456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb08103710 msgr2=0x7fbb0819f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.768+0000 7fbb0da5b640 1 --2- 172.21.15.120:0/2375065456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb08103710 0x7fbb0819f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.768+0000 7fbb0da5b640 1 -- 172.21.15.120:0/2375065456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb08101b60 msgr2=0x7fbb08197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.768+0000 7fbb0e25c640 1 --2- 172.21.15.120:0/2375065456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb08101b60 0x7fbb08197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.768+0000 7fbb0da5b640 1 --2- 172.21.15.120:0/2375065456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb08101b60 0x7fbb08197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.768+0000 7fbb0da5b640 1 -- 172.21.15.120:0/2375065456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0819f9b0 con 0x7fbb08102d50 2024-09-19T09:03:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.769+0000 7fbb0da5b640 1 --2- 172.21.15.120:0/2375065456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb08102d50 0x7fbb081982e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009230 tx=0x7fbafc004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.769+0000 7fbafb7fe640 1 -- 172.21.15.120:0/2375065456 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc03f030 con 0x7fbb08102d50 2024-09-19T09:03:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.769+0000 7fbafb7fe640 1 -- 172.21.15.120:0/2375065456 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbafc003ce0 con 0x7fbb08102d50 2024-09-19T09:03:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.769+0000 7fbb104c0640 1 -- 172.21.15.120:0/2375065456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb0819fc40 con 0x7fbb08102d50 2024-09-19T09:03:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.769+0000 7fbafb7fe640 1 -- 172.21.15.120:0/2375065456 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc035070 con 0x7fbb08102d50 2024-09-19T09:03:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.769+0000 7fbb104c0640 1 -- 172.21.15.120:0/2375065456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb081a0130 con 0x7fbb08102d50 2024-09-19T09:03:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.771+0000 7fbafb7fe640 1 -- 172.21.15.120:0/2375065456 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7fbafc042030 con 0x7fbb08102d50 2024-09-19T09:03:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.772+0000 7fbb13254740 1 Processor -- start 2024-09-19T09:03:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.772+0000 7fbb13254740 1 -- start start 2024-09-19T09:03:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.772+0000 7fbafb7fe640 1 --2- 172.21.15.120:0/2375065456 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbaf40a3390 0x7fbaf40a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.772+0000 7fbb13254740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56542a9ee9a0 0x56542a9eeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.772+0000 7fbb13254740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56542a9ef2e0 0x56542a9f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.772+0000 7fbb13254740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56542a9f4370 0x56542a9f6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.772+0000 7fbb13254740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56542a9522f0 con 0x56542a9ee9a0 2024-09-19T09:03:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.772+0000 7fbb13254740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56542a8fa270 con 0x56542a9ef2e0 2024-09-19T09:03:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.772+0000 7fbb13254740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56542a94e2b0 con 0x56542a9f4370 2024-09-19T09:03:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.772+0000 7fbb0e25c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56542a9ee9a0 0x56542a9eeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.773+0000 7fbb0ea5d640 1 --2- 172.21.15.120:0/2375065456 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbaf40a3390 0x7fbaf40a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.773+0000 7fbafb7fe640 1 -- 172.21.15.120:0/2375065456 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fbafc0f05d0 con 0x7fbb08102d50 2024-09-19T09:03:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.773+0000 7fbb0e25c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56542a9ee9a0 0x56542a9eeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59864/0 (socket says 172.21.15.120:59864) 2024-09-19T09:03:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.773+0000 7fbb0e25c640 1 -- 172.21.15.120:0/1502138872 learned_addr learned my addr 172.21.15.120:0/1502138872 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.773+0000 7fbb0ea5d640 1 --2- 172.21.15.120:0/1502138872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56542a9f4370 0x56542a9f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.773+0000 7fbb0da5b640 1 --2- 172.21.15.120:0/1502138872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56542a9ef2e0 0x56542a9f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.773+0000 7fbb0e25c640 1 -- 172.21.15.120:0/1502138872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56542a9f4370 msgr2=0x56542a9f6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.773+0000 7fbb0e25c640 1 --2- 172.21.15.120:0/1502138872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56542a9f4370 0x56542a9f6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.773+0000 7fbb0e25c640 1 -- 172.21.15.120:0/1502138872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56542a9ef2e0 msgr2=0x56542a9f3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.773+0000 7fbb0e25c640 1 --2- 172.21.15.120:0/1502138872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56542a9ef2e0 0x56542a9f3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.773+0000 7fbb0e25c640 1 -- 172.21.15.120:0/1502138872 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56542a8f9340 con 0x56542a9ee9a0 2024-09-19T09:03:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.773+0000 7fbb0ea5d640 1 --2- 172.21.15.120:0/2375065456 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbaf40a3390 0x7fbaf40a5850 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbb00005fd0 tx=0x7fbb000098f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.773+0000 7fbb0e25c640 1 --2- 172.21.15.120:0/1502138872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56542a9ee9a0 0x56542a9eeda0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400bb10 tx=0x7fbb0400d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8789a9e41fbd1e7a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.773+0000 7fbaf8bfe640 1 -- 172.21.15.120:0/1502138872 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0400de50 con 0x56542a9ee9a0 2024-09-19T09:03:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.773+0000 7fbaf8bfe640 1 -- 172.21.15.120:0/1502138872 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb04017900 con 0x56542a9ee9a0 2024-09-19T09:03:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.773+0000 7fbaf8bfe640 1 -- 172.21.15.120:0/1502138872 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0400f9e0 con 0x56542a9ee9a0 2024-09-19T09:03:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.774+0000 7fbb13254740 1 -- 172.21.15.120:0/1502138872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56542a9ee9a0 msgr2=0x56542a9eeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.774+0000 7fbb13254740 1 --2- 172.21.15.120:0/1502138872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56542a9ee9a0 0x56542a9eeda0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400bb10 tx=0x7fbb0400d340 comp rx=0 tx=0).stop 2024-09-19T09:03:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.774+0000 7fbb13254740 1 -- 172.21.15.120:0/1502138872 shutdown_connections 2024-09-19T09:03:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.774+0000 7fbb13254740 1 --2- 172.21.15.120:0/1502138872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56542a9f4370 0x56542a9f6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.774+0000 7fbb13254740 1 --2- 172.21.15.120:0/1502138872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56542a9ef2e0 0x56542a9f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.774+0000 7fbb13254740 1 --2- 172.21.15.120:0/1502138872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56542a9ee9a0 0x56542a9eeda0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.774+0000 7fbb13254740 1 -- 172.21.15.120:0/1502138872 >> 172.21.15.120:0/1502138872 conn(0x56542a9ea010 msgr2=0x56542a9ea410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.774+0000 7fbb13254740 1 -- 172.21.15.120:0/1502138872 shutdown_connections 2024-09-19T09:03:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.774+0000 7fbb13254740 1 -- 172.21.15.120:0/1502138872 wait complete. 2024-09-19T09:03:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.774+0000 7fbb13254740 1 -- start start 2024-09-19T09:03:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.775+0000 7fbb13254740 1 Processor -- start 2024-09-19T09:03:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.775+0000 7fbb13254740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56542a9ee9a0 0x56542a9f8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.775+0000 7fbb13254740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56542a9ef2e0 0x56542a9f90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.775+0000 7fbb13254740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56542a9f4370 0x56542aa000f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.775+0000 7fbb13254740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56542a9f7340 con 0x56542a9ef2e0 2024-09-19T09:03:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.775+0000 7fbb13254740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56542a8db290 con 0x56542a9ee9a0 2024-09-19T09:03:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.775+0000 7fbb13254740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56542a9f74c0 con 0x56542a9f4370 2024-09-19T09:03:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.775+0000 7fbb13254740 10 client.0 _ll_register_callbacks cb 0x56542a90c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.775+0000 7fbb0da5b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56542a9ef2e0 0x56542a9f90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.775+0000 7fbb0da5b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56542a9ef2e0 0x56542a9f90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59872/0 (socket says 172.21.15.120:59872) 2024-09-19T09:03:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.775+0000 7fbb0da5b640 1 -- 172.21.15.120:0/54039169 learned_addr learned my addr 172.21.15.120:0/54039169 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.775+0000 7fbb0e25c640 1 --2- 172.21.15.120:0/54039169 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56542a9ee9a0 0x56542a9f8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.775+0000 7fbb0da5b640 1 -- 172.21.15.120:0/54039169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56542a9f4370 msgr2=0x56542aa000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.775+0000 7fbb0ea5d640 1 --2- 172.21.15.120:0/54039169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56542a9f4370 0x56542aa000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.775+0000 7fbb0da5b640 1 --2- 172.21.15.120:0/54039169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56542a9f4370 0x56542aa000f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.775+0000 7fbb0da5b640 1 -- 172.21.15.120:0/54039169 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56542a9ee9a0 msgr2=0x56542a9f8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.775+0000 7fbb0da5b640 1 --2- 172.21.15.120:0/54039169 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56542a9ee9a0 0x56542a9f8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.775+0000 7fbb0da5b640 1 -- 172.21.15.120:0/54039169 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56542a665210 con 0x56542a9ef2e0 2024-09-19T09:03:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.775+0000 7fbb0e25c640 1 --2- 172.21.15.120:0/54039169 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56542a9ee9a0 0x56542a9f8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:03:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.775+0000 7fbb0da5b640 1 --2- 172.21.15.120:0/54039169 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56542a9ef2e0 0x56542a9f90b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009a20 tx=0x7fbafc0babd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393cb1501a47ef22 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.775+0000 7fbaf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.775+0000 7fbaf1ffb640 1 -- 172.21.15.120:0/54039169 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc049040 con 0x56542a9ef2e0 2024-09-19T09:03:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.776+0000 7fbaf1ffb640 1 -- 172.21.15.120:0/54039169 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fbafc03b460 con 0x56542a9ef2e0 2024-09-19T09:03:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.776+0000 7fbb13254740 1 -- 172.21.15.120:0/54039169 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb04017900 con 0x56542a9ef2e0 2024-09-19T09:03:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.776+0000 7fbaf1ffb640 1 -- 172.21.15.120:0/54039169 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbafc0bca90 con 0x56542a9ef2e0 2024-09-19T09:03:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.776+0000 7fbb13254740 10 client.15480 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.776+0000 7fbb13254740 1 -- 172.21.15.120:0/54039169 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56542a731620 con 0x56542a9ef2e0 2024-09-19T09:03:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.776+0000 7fbaf1ffb640 1 -- 172.21.15.120:0/54039169 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb04017900 con 0x56542a9ef2e0 2024-09-19T09:03:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.776+0000 7fbaf1ffb640 1 -- 172.21.15.120:0/54039169 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbafc0a06d0 con 0x56542a9ef2e0 2024-09-19T09:03:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.776+0000 7fbb13254740 10 client.15480 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.776+0000 7fbb13254740 10 client.15480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:03:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.776+0000 7fbb13254740 20 client.15480 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.776+0000 7fbb13254740 1 --2- 172.21.15.120:0/54039169 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56542aa047b0 0x56542aa06c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.776+0000 7fbb13254740 4 client.15480 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:03:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.776+0000 7fbb13254740 1 -- 172.21.15.120:0/54039169 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56542a95fa20 con 0x56542aa047b0 2024-09-19T09:03:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.777+0000 7fbb0e25c640 1 --2- 172.21.15.120:0/54039169 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56542aa047b0 0x56542aa06c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.777+0000 7fbaf1ffb640 10 client.15480 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.777+0000 7fbb0e25c640 1 --2- 172.21.15.120:0/54039169 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56542aa047b0 0x56542aa06c20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cbecc7cd1cd6f0c server_cookie=1a5768f8d7fc1fa6 in_seq=0 out_seq=0 2024-09-19T09:03:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.778+0000 7fbaf1ffb640 1 -- 172.21.15.120:0/54039169 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56542a95fa20 con 0x56542aa047b0 2024-09-19T09:03:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.778+0000 7fbaf1ffb640 10 client.15480 handle_command_reply: tid=0 2024-09-19T09:03:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.778+0000 7fbb13254740 1 client.15480 shutdown 2024-09-19T09:03:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.778+0000 7fbb13254740 1 -- 172.21.15.120:0/54039169 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56542a9ef2e0 msgr2=0x56542a9f90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.778+0000 7fbb13254740 1 --2- 172.21.15.120:0/54039169 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56542a9ef2e0 0x56542a9f90b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009a20 tx=0x7fbafc0babd0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.778+0000 7fbb13254740 1 -- 172.21.15.120:0/54039169 shutdown_connections 2024-09-19T09:03:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.778+0000 7fbb13254740 1 --2- 172.21.15.120:0/54039169 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56542aa047b0 0x56542aa06c20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.778+0000 7fbb13254740 1 --2- 172.21.15.120:0/54039169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56542a9f4370 0x56542aa000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.778+0000 7fbb13254740 1 --2- 172.21.15.120:0/54039169 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56542a9ef2e0 0x56542a9f90b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.778+0000 7fbb13254740 1 --2- 172.21.15.120:0/54039169 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56542a9ee9a0 0x56542a9f8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.778+0000 7fbaf1ffb640 0 client.15480 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:15.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.778+0000 7fbb13254740 1 -- 172.21.15.120:0/54039169 >> 172.21.15.120:0/54039169 conn(0x56542a9ea010 msgr2=0x56542a9eb820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:15.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.778+0000 7fbb13254740 1 -- 172.21.15.120:0/54039169 shutdown_connections 2024-09-19T09:03:15.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.778+0000 7fbb13254740 1 -- 172.21.15.120:0/54039169 wait complete. 2024-09-19T09:03:15.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.778+0000 7fbb13254740 20 client.15480 trim_cache size 0 max 16384 2024-09-19T09:03:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.794+0000 7fbb13254740 1 Processor -- start 2024-09-19T09:03:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.794+0000 7fbb13254740 1 -- start start 2024-09-19T09:03:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.794+0000 7fbb13254740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56542a9ee9a0 0x56542a9eaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.794+0000 7fbb13254740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56542a9ef2e0 0x56542aa000b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.794+0000 7fbb13254740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56542a9f4370 0x56542a9eb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.794+0000 7fbb13254740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbb081a8f20 con 0x56542a9ee9a0 2024-09-19T09:03:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.794+0000 7fbb13254740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbb081aaa60 con 0x56542a9f4370 2024-09-19T09:03:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.794+0000 7fbb13254740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbb081a95f0 con 0x56542a9ef2e0 2024-09-19T09:03:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.794+0000 7fbb0e25c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56542a9ee9a0 0x56542a9eaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.794+0000 7fbb0e25c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56542a9ee9a0 0x56542a9eaec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59880/0 (socket says 172.21.15.120:59880) 2024-09-19T09:03:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.794+0000 7fbb0e25c640 1 -- 172.21.15.120:0/4038402407 learned_addr learned my addr 172.21.15.120:0/4038402407 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.794+0000 7fbb0da5b640 1 --2- 172.21.15.120:0/4038402407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56542a9ef2e0 0x56542aa000b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.794+0000 7fbb0e25c640 1 -- 172.21.15.120:0/4038402407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56542a9ef2e0 msgr2=0x56542aa000b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.794+0000 7fbb0e25c640 1 --2- 172.21.15.120:0/4038402407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56542a9ef2e0 0x56542aa000b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.794+0000 7fbb0e25c640 1 -- 172.21.15.120:0/4038402407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56542a9f4370 msgr2=0x56542a9eb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.794+0000 7fbb0e25c640 1 --2- 172.21.15.120:0/4038402407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56542a9f4370 0x56542a9eb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.794+0000 7fbb0e25c640 1 -- 172.21.15.120:0/4038402407 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56542a9ef7c0 con 0x56542a9ee9a0 2024-09-19T09:03:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.794+0000 7fbb0e25c640 1 --2- 172.21.15.120:0/4038402407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56542a9ee9a0 0x56542a9eaec0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fbb04000c00 tx=0x7fbb04005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e709779ba0f4e492 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.794+0000 7fbaf3fff640 1 -- 172.21.15.120:0/4038402407 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0402e070 con 0x56542a9ee9a0 2024-09-19T09:03:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.794+0000 7fbaf3fff640 1 -- 172.21.15.120:0/4038402407 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb0400ddd0 con 0x56542a9ee9a0 2024-09-19T09:03:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.794+0000 7fbaf3fff640 1 -- 172.21.15.120:0/4038402407 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb040288d0 con 0x56542a9ee9a0 2024-09-19T09:03:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.794+0000 7fbb13254740 1 -- 172.21.15.120:0/4038402407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56542a9ee9a0 msgr2=0x56542a9eaec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.794+0000 7fbb13254740 1 --2- 172.21.15.120:0/4038402407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56542a9ee9a0 0x56542a9eaec0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fbb04000c00 tx=0x7fbb04005f00 comp rx=0 tx=0).stop 2024-09-19T09:03:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.794+0000 7fbb13254740 1 -- 172.21.15.120:0/4038402407 shutdown_connections 2024-09-19T09:03:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.794+0000 7fbb13254740 1 --2- 172.21.15.120:0/4038402407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56542a9f4370 0x56542a9eb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.794+0000 7fbb13254740 1 --2- 172.21.15.120:0/4038402407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56542a9ef2e0 0x56542aa000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.794+0000 7fbb13254740 1 --2- 172.21.15.120:0/4038402407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56542a9ee9a0 0x56542a9eaec0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.794+0000 7fbb13254740 1 -- 172.21.15.120:0/4038402407 >> 172.21.15.120:0/4038402407 conn(0x56542a9ea010 msgr2=0x56542a9f8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.795+0000 7fbb13254740 1 -- 172.21.15.120:0/4038402407 shutdown_connections 2024-09-19T09:03:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.795+0000 7fbb13254740 1 -- 172.21.15.120:0/4038402407 wait complete. 2024-09-19T09:03:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.795+0000 7fbb13254740 1 -- start start 2024-09-19T09:03:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.795+0000 7fbb13254740 1 Processor -- start 2024-09-19T09:03:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.795+0000 7fbb13254740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56542a9ee9a0 0x56542a9fdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.795+0000 7fbb13254740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56542a9ef2e0 0x56542a9fe280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.795+0000 7fbb13254740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56542a9f4370 0x56542a9ff240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.795+0000 7fbb13254740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbb081aa390 con 0x56542a9ee9a0 2024-09-19T09:03:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.795+0000 7fbb13254740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbb081a9cc0 con 0x56542a9ef2e0 2024-09-19T09:03:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.795+0000 7fbb13254740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbb081a6ee0 con 0x56542a9f4370 2024-09-19T09:03:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.795+0000 7fbb13254740 10 client.0 _ll_register_callbacks cb 0x7fbafc003e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.795+0000 7fbb0e25c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56542a9ee9a0 0x56542a9fdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.795+0000 7fbb0e25c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56542a9ee9a0 0x56542a9fdd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59894/0 (socket says 172.21.15.120:59894) 2024-09-19T09:03:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.795+0000 7fbb0e25c640 1 -- 172.21.15.120:0/1347796691 learned_addr learned my addr 172.21.15.120:0/1347796691 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.795+0000 7fbb0ea5d640 1 --2- 172.21.15.120:0/1347796691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56542a9f4370 0x56542a9ff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.795+0000 7fbb0da5b640 1 --2- 172.21.15.120:0/1347796691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56542a9ef2e0 0x56542a9fe280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.795+0000 7fbb0e25c640 1 -- 172.21.15.120:0/1347796691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56542a9f4370 msgr2=0x56542a9ff240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.795+0000 7fbb0e25c640 1 --2- 172.21.15.120:0/1347796691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56542a9f4370 0x56542a9ff240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.795+0000 7fbb0e25c640 1 -- 172.21.15.120:0/1347796691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56542a9ef2e0 msgr2=0x56542a9fe280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.795+0000 7fbb0e25c640 1 --2- 172.21.15.120:0/1347796691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56542a9ef2e0 0x56542a9fe280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.795+0000 7fbb0e25c640 1 -- 172.21.15.120:0/1347796691 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56542a9ff850 con 0x56542a9ee9a0 2024-09-19T09:03:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.795+0000 7fbb0ea5d640 1 --2- 172.21.15.120:0/1347796691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56542a9f4370 0x56542a9ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:03:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.795+0000 7fbb0da5b640 1 --2- 172.21.15.120:0/1347796691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56542a9ef2e0 0x56542a9fe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:03:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.796+0000 7fbb0e25c640 1 --2- 172.21.15.120:0/1347796691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56542a9ee9a0 0x56542a9fdd40 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400f460 tx=0x7fbb0400b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467620165091d73b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.796+0000 7fbaf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.796+0000 7fbaf1ffb640 1 -- 172.21.15.120:0/1347796691 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04039060 con 0x56542a9ee9a0 2024-09-19T09:03:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.796+0000 7fbaf1ffb640 1 -- 172.21.15.120:0/1347796691 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fbb04028910 con 0x56542a9ee9a0 2024-09-19T09:03:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.796+0000 7fbb13254740 1 -- 172.21.15.120:0/1347796691 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb0400ddd0 con 0x56542a9ee9a0 2024-09-19T09:03:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.796+0000 7fbaf1ffb640 1 -- 172.21.15.120:0/1347796691 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbb0402e040 con 0x56542a9ee9a0 2024-09-19T09:03:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.796+0000 7fbb13254740 10 client.15486 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.796+0000 7fbb13254740 1 -- 172.21.15.120:0/1347796691 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56542aa6f980 con 0x56542a9ee9a0 2024-09-19T09:03:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.796+0000 7fbaf1ffb640 1 -- 172.21.15.120:0/1347796691 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb0400ddd0 con 0x56542a9ee9a0 2024-09-19T09:03:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.796+0000 7fbaf1ffb640 1 -- 172.21.15.120:0/1347796691 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbb040053b0 con 0x56542a9ee9a0 2024-09-19T09:03:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.796+0000 7fbb13254740 10 client.15486 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.796+0000 7fbb13254740 10 client.15486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:03:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.796+0000 7fbb13254740 20 client.15486 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.796+0000 7fbb13254740 1 --2- 172.21.15.120:0/1347796691 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56542aa70100 0x56542aa72520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.796+0000 7fbb13254740 4 client.15486 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:03:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.796+0000 7fbb13254740 1 -- 172.21.15.120:0/1347796691 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56542a959d70 con 0x56542aa70100 2024-09-19T09:03:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.797+0000 7fbb0da5b640 1 --2- 172.21.15.120:0/1347796691 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56542aa70100 0x56542aa72520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.797+0000 7fbb0da5b640 1 --2- 172.21.15.120:0/1347796691 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56542aa70100 0x56542aa72520 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66c3f98a7d59b331 server_cookie=1d17b53e8a99bf0f in_seq=0 out_seq=0 2024-09-19T09:03:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.797+0000 7fbaf1ffb640 10 client.15486 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.804+0000 7fbaf1ffb640 1 -- 172.21.15.120:0/1347796691 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56542a959d70 con 0x56542aa70100 2024-09-19T09:03:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.804+0000 7fbaf1ffb640 10 client.15486 handle_command_reply: tid=0 2024-09-19T09:03:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.804+0000 7fbb13254740 1 client.15486 shutdown 2024-09-19T09:03:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.805+0000 7fbb13254740 1 -- 172.21.15.120:0/1347796691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56542a9ee9a0 msgr2=0x56542a9fdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.805+0000 7fbb13254740 1 --2- 172.21.15.120:0/1347796691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56542a9ee9a0 0x56542a9fdd40 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400f460 tx=0x7fbb0400b600 comp rx=0 tx=0).stop 2024-09-19T09:03:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.805+0000 7fbb13254740 1 -- 172.21.15.120:0/1347796691 shutdown_connections 2024-09-19T09:03:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.805+0000 7fbb13254740 1 --2- 172.21.15.120:0/1347796691 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56542aa70100 0x56542aa72520 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.805+0000 7fbb13254740 1 --2- 172.21.15.120:0/1347796691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56542a9f4370 0x56542a9ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.805+0000 7fbb13254740 1 --2- 172.21.15.120:0/1347796691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56542a9ef2e0 0x56542a9fe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.805+0000 7fbb13254740 1 --2- 172.21.15.120:0/1347796691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56542a9ee9a0 0x56542a9fdd40 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.805+0000 7fbaf1ffb640 0 client.15486 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.805+0000 7fbb13254740 1 -- 172.21.15.120:0/1347796691 >> 172.21.15.120:0/1347796691 conn(0x56542a9ea010 msgr2=0x56542a9e89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.805+0000 7fbb13254740 1 -- 172.21.15.120:0/1347796691 shutdown_connections 2024-09-19T09:03:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.805+0000 7fbb13254740 1 -- 172.21.15.120:0/1347796691 wait complete. 2024-09-19T09:03:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.805+0000 7fbb13254740 20 client.15486 trim_cache size 0 max 16384 2024-09-19T09:03:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:03:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:03:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:03:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:03:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.805+0000 7fbb104c0640 1 -- 172.21.15.120:0/2375065456 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbaf40a3390 msgr2=0x7fbaf40a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.805+0000 7fbb104c0640 1 --2- 172.21.15.120:0/2375065456 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbaf40a3390 0x7fbaf40a5850 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbb00005fd0 tx=0x7fbb000098f0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.805+0000 7fbb104c0640 1 -- 172.21.15.120:0/2375065456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb08102d50 msgr2=0x7fbb081982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.805+0000 7fbb104c0640 1 --2- 172.21.15.120:0/2375065456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb08102d50 0x7fbb081982e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009230 tx=0x7fbafc004360 comp rx=0 tx=0).stop 2024-09-19T09:03:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.805+0000 7fbb104c0640 1 -- 172.21.15.120:0/2375065456 shutdown_connections 2024-09-19T09:03:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.805+0000 7fbb104c0640 1 --2- 172.21.15.120:0/2375065456 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbaf40a3390 0x7fbaf40a5850 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.805+0000 7fbb104c0640 1 --2- 172.21.15.120:0/2375065456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb08103710 0x7fbb0819f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.805+0000 7fbb104c0640 1 --2- 172.21.15.120:0/2375065456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb08102d50 0x7fbb081982e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.805+0000 7fbb104c0640 1 --2- 172.21.15.120:0/2375065456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb08101b60 0x7fbb08197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.805+0000 7fbb104c0640 1 -- 172.21.15.120:0/2375065456 >> 172.21.15.120:0/2375065456 conn(0x7fbb080fd2e0 msgr2=0x7fbb081060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.806+0000 7fbb104c0640 1 -- 172.21.15.120:0/2375065456 shutdown_connections 2024-09-19T09:03:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:15.806+0000 7fbb104c0640 1 -- 172.21.15.120:0/2375065456 wait complete. 2024-09-19T09:03:17.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:16 smithi120 ceph-mon[31150]: pgmap v353: 97 pgs: 97 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-19T09:03:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:16 smithi196 ceph-mon[35150]: pgmap v353: 97 pgs: 97 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-19T09:03:17.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:16 smithi153 ceph-mon[35071]: pgmap v353: 97 pgs: 97 active+clean; 663 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-19T09:03:19.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:18 smithi120 ceph-mon[31150]: pgmap v354: 97 pgs: 97 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.4 KiB/s wr, 14 op/s 2024-09-19T09:03:19.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:18 smithi196 ceph-mon[35150]: pgmap v354: 97 pgs: 97 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.4 KiB/s wr, 14 op/s 2024-09-19T09:03:19.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:18 smithi153 ceph-mon[35071]: pgmap v354: 97 pgs: 97 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.4 KiB/s wr, 14 op/s 2024-09-19T09:03:21.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:20 smithi120 ceph-mon[31150]: pgmap v355: 97 pgs: 97 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 937 B/s wr, 12 op/s 2024-09-19T09:03:21.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:20 smithi196 ceph-mon[35150]: pgmap v355: 97 pgs: 97 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 937 B/s wr, 12 op/s 2024-09-19T09:03:21.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:20 smithi153 ceph-mon[35071]: pgmap v355: 97 pgs: 97 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 937 B/s wr, 12 op/s 2024-09-19T09:03:23.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:22 smithi120 ceph-mon[31150]: pgmap v356: 97 pgs: 97 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-19T09:03:23.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:22 smithi196 ceph-mon[35150]: pgmap v356: 97 pgs: 97 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-19T09:03:23.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:22 smithi153 ceph-mon[35071]: pgmap v356: 97 pgs: 97 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-19T09:03:23.804 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:03:24.066 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:03:24.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.589+0000 7f427ab69640 1 -- 172.21.15.120:0/3431329422 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4274102570 msgr2=0x7f42741029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:24.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.589+0000 7f427ab69640 1 --2- 172.21.15.120:0/3431329422 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4274102570 0x7f42741029b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4260009900 tx=0x7f426002a480 comp rx=0 tx=0).stop 2024-09-19T09:03:24.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.589+0000 7f427ab69640 1 -- 172.21.15.120:0/3431329422 shutdown_connections 2024-09-19T09:03:24.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.589+0000 7f427ab69640 1 --2- 172.21.15.120:0/3431329422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4274102ef0 0x7f427410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.589+0000 7f427ab69640 1 --2- 172.21.15.120:0/3431329422 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4274102570 0x7f42741029b0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.589+0000 7f427ab69640 1 --2- 172.21.15.120:0/3431329422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4274100e20 0x7f4274101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.589+0000 7f427ab69640 1 -- 172.21.15.120:0/3431329422 >> 172.21.15.120:0/3431329422 conn(0x7f42740fc7d0 msgr2=0x7f42740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:24.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.589+0000 7f427ab69640 1 -- 172.21.15.120:0/3431329422 shutdown_connections 2024-09-19T09:03:24.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.589+0000 7f427ab69640 1 -- 172.21.15.120:0/3431329422 wait complete. 2024-09-19T09:03:24.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.590+0000 7f427ab69640 1 Processor -- start 2024-09-19T09:03:24.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.590+0000 7f427ab69640 1 -- start start 2024-09-19T09:03:24.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.590+0000 7f427ab69640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4274100e20 0x7f42741993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:24.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.590+0000 7f427ab69640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4274102570 0x7f4274199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:24.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.590+0000 7f427ab69640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4274102ef0 0x7f42741a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:24.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.590+0000 7f427ab69640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4274110fa0 con 0x7f4274102570 2024-09-19T09:03:24.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.590+0000 7f427ab69640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4274110e20 con 0x7f4274102ef0 2024-09-19T09:03:24.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.590+0000 7f427ab69640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4274111120 con 0x7f4274100e20 2024-09-19T09:03:24.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.590+0000 7f4273fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4274102570 0x7f4274199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:24.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.590+0000 7f4273fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4274102570 0x7f4274199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58900/0 (socket says 172.21.15.120:58900) 2024-09-19T09:03:24.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.590+0000 7f4273fff640 1 -- 172.21.15.120:0/811581276 learned_addr learned my addr 172.21.15.120:0/811581276 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:24.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.590+0000 7f4278905640 1 --2- 172.21.15.120:0/811581276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4274100e20 0x7f42741993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:24.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.591+0000 7f4273fff640 1 -- 172.21.15.120:0/811581276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4274100e20 msgr2=0x7f42741993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:24.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.591+0000 7f4273fff640 1 --2- 172.21.15.120:0/811581276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4274100e20 0x7f42741993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.591+0000 7f4273fff640 1 -- 172.21.15.120:0/811581276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4274102ef0 msgr2=0x7f42741a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:24.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.591+0000 7f4273fff640 1 --2- 172.21.15.120:0/811581276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4274102ef0 0x7f42741a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.591+0000 7f4273fff640 1 -- 172.21.15.120:0/811581276 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42741a4990 con 0x7f4274102570 2024-09-19T09:03:24.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.591+0000 7f4278905640 1 --2- 172.21.15.120:0/811581276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4274100e20 0x7f42741993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:03:24.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.591+0000 7f4273fff640 1 --2- 172.21.15.120:0/811581276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4274102570 0x7f4274199910 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4260006010 tx=0x7f4260003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:24.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.591+0000 7f4271ffb640 1 -- 172.21.15.120:0/811581276 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426003f070 con 0x7f4274102570 2024-09-19T09:03:24.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.591+0000 7f427ab69640 1 -- 172.21.15.120:0/811581276 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42741a4b60 con 0x7f4274102570 2024-09-19T09:03:24.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.591+0000 7f4271ffb640 1 -- 172.21.15.120:0/811581276 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4260031e30 con 0x7f4274102570 2024-09-19T09:03:24.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.591+0000 7f4271ffb640 1 -- 172.21.15.120:0/811581276 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4260035030 con 0x7f4274102570 2024-09-19T09:03:24.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.591+0000 7f427ab69640 1 -- 172.21.15.120:0/811581276 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42741a4fc0 con 0x7f4274102570 2024-09-19T09:03:24.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.593+0000 7f4271ffb640 1 -- 172.21.15.120:0/811581276 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f4260042070 con 0x7f4274102570 2024-09-19T09:03:24.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.594+0000 7f4271ffb640 1 --2- 172.21.15.120:0/811581276 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f42640a32d0 0x7f42640a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:24.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.594+0000 7f4271ffb640 1 -- 172.21.15.120:0/811581276 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f42600f04a0 con 0x7f4274102570 2024-09-19T09:03:24.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.594+0000 7f4278905640 1 --2- 172.21.15.120:0/811581276 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f42640a32d0 0x7f42640a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:24.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.594+0000 7f4278905640 1 --2- 172.21.15.120:0/811581276 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f42640a32d0 0x7f42640a5790 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4268009930 tx=0x7f4268008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:24.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.595+0000 7f427cf42740 1 Processor -- start 2024-09-19T09:03:24.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.595+0000 7f427cf42740 1 -- start start 2024-09-19T09:03:24.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.595+0000 7f427cf42740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3eba564a0 0x55c3eba56880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:24.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.595+0000 7f427cf42740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3eba56dc0 0x55c3eba61140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:24.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.595+0000 7f427cf42740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c3eba61680 0x55c3eba63a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:24.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.595+0000 7f427cf42740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c3eb9bc2e0 con 0x55c3eba61680 2024-09-19T09:03:24.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.595+0000 7f427cf42740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c3eb963ce0 con 0x55c3eba564a0 2024-09-19T09:03:24.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.595+0000 7f427cf42740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c3eb9b82a0 con 0x55c3eba56dc0 2024-09-19T09:03:24.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.595+0000 7f4279106640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c3eba61680 0x55c3eba63a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:24.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.595+0000 7f4279106640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c3eba61680 0x55c3eba63a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58912/0 (socket says 172.21.15.120:58912) 2024-09-19T09:03:24.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.595+0000 7f4279106640 1 -- 172.21.15.120:0/2620978118 learned_addr learned my addr 172.21.15.120:0/2620978118 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:24.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.595+0000 7f4273fff640 1 --2- 172.21.15.120:0/2620978118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3eba56dc0 0x55c3eba61140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:24.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.595+0000 7f4278905640 1 --2- 172.21.15.120:0/2620978118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3eba564a0 0x55c3eba56880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:24.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.595+0000 7f4279106640 1 -- 172.21.15.120:0/2620978118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3eba56dc0 msgr2=0x55c3eba61140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:24.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.595+0000 7f4279106640 1 --2- 172.21.15.120:0/2620978118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3eba56dc0 0x55c3eba61140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.595+0000 7f4279106640 1 -- 172.21.15.120:0/2620978118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3eba564a0 msgr2=0x55c3eba56880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:24.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.595+0000 7f4279106640 1 --2- 172.21.15.120:0/2620978118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3eba564a0 0x55c3eba56880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.595+0000 7f4279106640 1 -- 172.21.15.120:0/2620978118 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3eb948440 con 0x55c3eba61680 2024-09-19T09:03:24.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.596+0000 7f4279106640 1 --2- 172.21.15.120:0/2620978118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c3eba61680 0x55c3eba63a70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f426c007cf0 tx=0x7f426c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c53df54ab87161d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:24.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.596+0000 7f425effd640 1 -- 172.21.15.120:0/2620978118 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c014070 con 0x55c3eba61680 2024-09-19T09:03:24.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.596+0000 7f425effd640 1 -- 172.21.15.120:0/2620978118 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f426c00fb30 con 0x55c3eba61680 2024-09-19T09:03:24.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.596+0000 7f425effd640 1 -- 172.21.15.120:0/2620978118 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c00ebc0 con 0x55c3eba61680 2024-09-19T09:03:24.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.596+0000 7f427cf42740 1 -- 172.21.15.120:0/2620978118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c3eba61680 msgr2=0x55c3eba63a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:24.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.596+0000 7f427cf42740 1 --2- 172.21.15.120:0/2620978118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c3eba61680 0x55c3eba63a70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f426c007cf0 tx=0x7f426c00c6a0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.597+0000 7f427cf42740 1 -- 172.21.15.120:0/2620978118 shutdown_connections 2024-09-19T09:03:24.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.597+0000 7f427cf42740 1 --2- 172.21.15.120:0/2620978118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c3eba61680 0x55c3eba63a70 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.597+0000 7f427cf42740 1 --2- 172.21.15.120:0/2620978118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3eba56dc0 0x55c3eba61140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.597+0000 7f427cf42740 1 --2- 172.21.15.120:0/2620978118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3eba564a0 0x55c3eba56880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.597+0000 7f427cf42740 1 -- 172.21.15.120:0/2620978118 >> 172.21.15.120:0/2620978118 conn(0x55c3eba52a30 msgr2=0x55c3eba52e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:24.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.597+0000 7f427cf42740 1 -- 172.21.15.120:0/2620978118 shutdown_connections 2024-09-19T09:03:24.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.597+0000 7f427cf42740 1 -- 172.21.15.120:0/2620978118 wait complete. 2024-09-19T09:03:24.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.597+0000 7f427cf42740 1 -- start start 2024-09-19T09:03:24.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.597+0000 7f427cf42740 1 Processor -- start 2024-09-19T09:03:24.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.598+0000 7f427cf42740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3eba564a0 0x55c3eba60f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:24.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.598+0000 7f427cf42740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c3eba56dc0 0x55c3eba5e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:24.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.598+0000 7f427cf42740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3eba61680 0x55c3eba5e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:24.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.598+0000 7f427cf42740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c3eba64250 con 0x55c3eba56dc0 2024-09-19T09:03:24.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.598+0000 7f427cf42740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c3eb977050 con 0x55c3eba564a0 2024-09-19T09:03:24.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.598+0000 7f427cf42740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c3eba643d0 con 0x55c3eba61680 2024-09-19T09:03:24.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.598+0000 7f427cf42740 10 client.0 _ll_register_callbacks cb 0x55c3eb9b7160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:24.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.598+0000 7f4273fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c3eba56dc0 0x55c3eba5e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:24.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.598+0000 7f4273fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c3eba56dc0 0x55c3eba5e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58922/0 (socket says 172.21.15.120:58922) 2024-09-19T09:03:24.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.598+0000 7f4273fff640 1 -- 172.21.15.120:0/3782725009 learned_addr learned my addr 172.21.15.120:0/3782725009 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:24.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.598+0000 7f4278905640 1 --2- 172.21.15.120:0/3782725009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3eba564a0 0x55c3eba60f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:24.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.598+0000 7f4279106640 1 --2- 172.21.15.120:0/3782725009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3eba61680 0x55c3eba5e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:24.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.598+0000 7f4273fff640 1 -- 172.21.15.120:0/3782725009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3eba61680 msgr2=0x55c3eba5e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:24.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.598+0000 7f4273fff640 1 --2- 172.21.15.120:0/3782725009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3eba61680 0x55c3eba5e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.598+0000 7f4273fff640 1 -- 172.21.15.120:0/3782725009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3eba564a0 msgr2=0x55c3eba60f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:24.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.598+0000 7f4273fff640 1 --2- 172.21.15.120:0/3782725009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3eba564a0 0x55c3eba60f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.598+0000 7f4273fff640 1 -- 172.21.15.120:0/3782725009 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3eb72d3c0 con 0x55c3eba56dc0 2024-09-19T09:03:24.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.598+0000 7f4278905640 1 --2- 172.21.15.120:0/3782725009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3eba564a0 0x55c3eba60f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:03:24.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.598+0000 7f4279106640 1 --2- 172.21.15.120:0/3782725009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3eba61680 0x55c3eba5e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:03:24.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.598+0000 7f4273fff640 1 --2- 172.21.15.120:0/3782725009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c3eba56dc0 0x55c3eba5e210 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f42600bc660 tx=0x7f42600bc2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6bae103532c4c14 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:24.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.598+0000 7f4253fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:24.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.598+0000 7f4253fff640 1 -- 172.21.15.120:0/3782725009 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4260049030 con 0x55c3eba56dc0 2024-09-19T09:03:24.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.599+0000 7f4253fff640 1 -- 172.21.15.120:0/3782725009 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f42600b8fd0 con 0x55c3eba56dc0 2024-09-19T09:03:24.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.599+0000 7f427cf42740 1 -- 172.21.15.120:0/3782725009 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f426c00fb30 con 0x55c3eba56dc0 2024-09-19T09:03:24.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.599+0000 7f4253fff640 1 -- 172.21.15.120:0/3782725009 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f42600c0d10 con 0x55c3eba56dc0 2024-09-19T09:03:24.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.599+0000 7f427cf42740 10 client.15498 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:24.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.599+0000 7f427cf42740 1 -- 172.21.15.120:0/3782725009 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c3eba6fbc0 con 0x55c3eba56dc0 2024-09-19T09:03:24.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.599+0000 7f4253fff640 1 -- 172.21.15.120:0/3782725009 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f426c00fb30 con 0x55c3eba56dc0 2024-09-19T09:03:24.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.599+0000 7f4253fff640 1 -- 172.21.15.120:0/3782725009 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f42600db1a0 con 0x55c3eba56dc0 2024-09-19T09:03:24.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.599+0000 7f427cf42740 10 client.15498 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:24.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.599+0000 7f427cf42740 10 client.15498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:03:24.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.599+0000 7f427cf42740 20 client.15498 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:24.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.599+0000 7f427cf42740 1 --2- 172.21.15.120:0/3782725009 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c3eba70fd0 0x55c3eba73450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:24.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.599+0000 7f427cf42740 4 client.15498 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:03:24.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.599+0000 7f427cf42740 1 -- 172.21.15.120:0/3782725009 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3eb9c9850 con 0x55c3eba70fd0 2024-09-19T09:03:24.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.599+0000 7f4278905640 1 --2- 172.21.15.120:0/3782725009 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c3eba70fd0 0x55c3eba73450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:24.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.600+0000 7f4253fff640 10 client.15498 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:24.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.600+0000 7f4278905640 1 --2- 172.21.15.120:0/3782725009 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c3eba70fd0 0x55c3eba73450 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c44e59cdd69d77c0 server_cookie=b528f80ad4c81b32 in_seq=0 out_seq=0 2024-09-19T09:03:24.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.601+0000 7f4253fff640 1 -- 172.21.15.120:0/3782725009 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c3eb9c9850 con 0x55c3eba70fd0 2024-09-19T09:03:24.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.601+0000 7f4253fff640 10 client.15498 handle_command_reply: tid=0 2024-09-19T09:03:24.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.601+0000 7f427cf42740 1 client.15498 shutdown 2024-09-19T09:03:24.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.601+0000 7f427cf42740 1 -- 172.21.15.120:0/3782725009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c3eba56dc0 msgr2=0x55c3eba5e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:24.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.601+0000 7f427cf42740 1 --2- 172.21.15.120:0/3782725009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c3eba56dc0 0x55c3eba5e210 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f42600bc660 tx=0x7f42600bc2c0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.601+0000 7f427cf42740 1 -- 172.21.15.120:0/3782725009 shutdown_connections 2024-09-19T09:03:24.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.601+0000 7f427cf42740 1 --2- 172.21.15.120:0/3782725009 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c3eba70fd0 0x55c3eba73450 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.601+0000 7f427cf42740 1 --2- 172.21.15.120:0/3782725009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3eba61680 0x55c3eba5e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.601+0000 7f427cf42740 1 --2- 172.21.15.120:0/3782725009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c3eba56dc0 0x55c3eba5e210 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.601+0000 7f427cf42740 1 --2- 172.21.15.120:0/3782725009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3eba564a0 0x55c3eba60f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.601+0000 7f4253fff640 0 client.15498 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:24.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.601+0000 7f427cf42740 1 -- 172.21.15.120:0/3782725009 >> 172.21.15.120:0/3782725009 conn(0x55c3eba52a30 msgr2=0x55c3eba58710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:24.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.601+0000 7f427cf42740 1 -- 172.21.15.120:0/3782725009 shutdown_connections 2024-09-19T09:03:24.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.601+0000 7f427cf42740 1 -- 172.21.15.120:0/3782725009 wait complete. 2024-09-19T09:03:24.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.601+0000 7f427cf42740 20 client.15498 trim_cache size 0 max 16384 2024-09-19T09:03:24.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.633+0000 7f427cf42740 1 Processor -- start 2024-09-19T09:03:24.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.633+0000 7f427cf42740 1 -- start start 2024-09-19T09:03:24.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.633+0000 7f427cf42740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c3eba52a30 0x55c3eba6d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:24.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.633+0000 7f427cf42740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3eba564a0 0x55c3eba59760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:24.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.633+0000 7f427cf42740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3eba56dc0 0x55c3eba78d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:24.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.633+0000 7f427cf42740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f42741ad550 con 0x55c3eba52a30 2024-09-19T09:03:24.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.633+0000 7f427cf42740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f42741af090 con 0x55c3eba56dc0 2024-09-19T09:03:24.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.633+0000 7f427cf42740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f42741adc20 con 0x55c3eba564a0 2024-09-19T09:03:24.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.633+0000 7f4278905640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c3eba52a30 0x55c3eba6d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:24.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.634+0000 7f4278905640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c3eba52a30 0x55c3eba6d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58924/0 (socket says 172.21.15.120:58924) 2024-09-19T09:03:24.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.634+0000 7f4278905640 1 -- 172.21.15.120:0/4104738287 learned_addr learned my addr 172.21.15.120:0/4104738287 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:24.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.634+0000 7f4273fff640 1 --2- 172.21.15.120:0/4104738287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3eba564a0 0x55c3eba59760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:24.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.634+0000 7f4279106640 1 --2- 172.21.15.120:0/4104738287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3eba56dc0 0x55c3eba78d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:24.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.634+0000 7f4278905640 1 -- 172.21.15.120:0/4104738287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3eba564a0 msgr2=0x55c3eba59760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:24.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.634+0000 7f4278905640 1 --2- 172.21.15.120:0/4104738287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3eba564a0 0x55c3eba59760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.634+0000 7f4278905640 1 -- 172.21.15.120:0/4104738287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3eba56dc0 msgr2=0x55c3eba78d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:24.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.634+0000 7f4278905640 1 --2- 172.21.15.120:0/4104738287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3eba56dc0 0x55c3eba78d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.634+0000 7f4278905640 1 -- 172.21.15.120:0/4104738287 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3eba53ba0 con 0x55c3eba52a30 2024-09-19T09:03:24.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.634+0000 7f4278905640 1 --2- 172.21.15.120:0/4104738287 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c3eba52a30 0x55c3eba6d4b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4268009fd0 tx=0x7f4268013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c75ed321278cd8a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:24.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.634+0000 7f425e7fc640 1 -- 172.21.15.120:0/4104738287 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4268030070 con 0x55c3eba52a30 2024-09-19T09:03:24.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.634+0000 7f425e7fc640 1 -- 172.21.15.120:0/4104738287 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4268024430 con 0x55c3eba52a30 2024-09-19T09:03:24.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.634+0000 7f425e7fc640 1 -- 172.21.15.120:0/4104738287 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426801d040 con 0x55c3eba52a30 2024-09-19T09:03:24.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.635+0000 7f427cf42740 1 -- 172.21.15.120:0/4104738287 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c3eba52a30 msgr2=0x55c3eba6d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:24.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.635+0000 7f427cf42740 1 --2- 172.21.15.120:0/4104738287 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c3eba52a30 0x55c3eba6d4b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4268009fd0 tx=0x7f4268013000 comp rx=0 tx=0).stop 2024-09-19T09:03:24.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.635+0000 7f427cf42740 1 -- 172.21.15.120:0/4104738287 shutdown_connections 2024-09-19T09:03:24.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.635+0000 7f427cf42740 1 --2- 172.21.15.120:0/4104738287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3eba56dc0 0x55c3eba78d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.635+0000 7f427cf42740 1 --2- 172.21.15.120:0/4104738287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3eba564a0 0x55c3eba59760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.635+0000 7f427cf42740 1 --2- 172.21.15.120:0/4104738287 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c3eba52a30 0x55c3eba6d4b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.635+0000 7f427cf42740 1 -- 172.21.15.120:0/4104738287 >> 172.21.15.120:0/4104738287 conn(0x55c3eba5e8b0 msgr2=0x55c3eba59050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:24.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.635+0000 7f427cf42740 1 -- 172.21.15.120:0/4104738287 shutdown_connections 2024-09-19T09:03:24.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.635+0000 7f427cf42740 1 -- 172.21.15.120:0/4104738287 wait complete. 2024-09-19T09:03:24.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.636+0000 7f427cf42740 1 -- start start 2024-09-19T09:03:24.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.636+0000 7f427cf42740 1 Processor -- start 2024-09-19T09:03:24.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.636+0000 7f427cf42740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c3eba52a30 0x55c3eba69d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:24.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.636+0000 7f427cf42740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3eba564a0 0x55c3eba65040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:24.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.637+0000 7f427cf42740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3eba56dc0 0x55c3eba65580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:24.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.637+0000 7f427cf42740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f42741ae9c0 con 0x55c3eba52a30 2024-09-19T09:03:24.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.637+0000 7f427cf42740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f42741ae2f0 con 0x55c3eba564a0 2024-09-19T09:03:24.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.637+0000 7f427cf42740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f42741ab510 con 0x55c3eba56dc0 2024-09-19T09:03:24.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.637+0000 7f4278905640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c3eba52a30 0x55c3eba69d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:24.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.637+0000 7f4278905640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c3eba52a30 0x55c3eba69d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58936/0 (socket says 172.21.15.120:58936) 2024-09-19T09:03:24.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.637+0000 7f4278905640 1 -- 172.21.15.120:0/1161453518 learned_addr learned my addr 172.21.15.120:0/1161453518 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:24.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.637+0000 7f427cf42740 10 client.0 _ll_register_callbacks cb 0x7f4260009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:24.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.637+0000 7f4273fff640 1 --2- 172.21.15.120:0/1161453518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3eba564a0 0x55c3eba65040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:24.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.637+0000 7f4278905640 1 -- 172.21.15.120:0/1161453518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3eba56dc0 msgr2=0x55c3eba65580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:24.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.637+0000 7f4278905640 1 --2- 172.21.15.120:0/1161453518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3eba56dc0 0x55c3eba65580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.637+0000 7f4278905640 1 -- 172.21.15.120:0/1161453518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3eba564a0 msgr2=0x55c3eba65040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:24.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.637+0000 7f4278905640 1 --2- 172.21.15.120:0/1161453518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3eba564a0 0x55c3eba65040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.637+0000 7f4278905640 1 -- 172.21.15.120:0/1161453518 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3eba65d60 con 0x55c3eba52a30 2024-09-19T09:03:24.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.637+0000 7f4273fff640 1 --2- 172.21.15.120:0/1161453518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3eba564a0 0x55c3eba65040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:03:24.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.637+0000 7f4278905640 1 --2- 172.21.15.120:0/1161453518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c3eba52a30 0x55c3eba69d10 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f426800d530 tx=0x7f4268024690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999c577767f868b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:24.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.637+0000 7f4253fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:24.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.637+0000 7f4253fff640 1 -- 172.21.15.120:0/1161453518 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4268039060 con 0x55c3eba52a30 2024-09-19T09:03:24.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.637+0000 7f4253fff640 1 -- 172.21.15.120:0/1161453518 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f426800f040 con 0x55c3eba52a30 2024-09-19T09:03:24.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.637+0000 7f427cf42740 1 -- 172.21.15.120:0/1161453518 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4268024430 con 0x55c3eba52a30 2024-09-19T09:03:24.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.638+0000 7f4253fff640 1 -- 172.21.15.120:0/1161453518 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4268030030 con 0x55c3eba52a30 2024-09-19T09:03:24.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.638+0000 7f427cf42740 10 client.15504 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:24.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.638+0000 7f427cf42740 1 -- 172.21.15.120:0/1161453518 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c3ebae0100 con 0x55c3eba52a30 2024-09-19T09:03:24.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.638+0000 7f4253fff640 1 -- 172.21.15.120:0/1161453518 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4268024430 con 0x55c3eba52a30 2024-09-19T09:03:24.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.638+0000 7f4253fff640 1 -- 172.21.15.120:0/1161453518 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f426803e070 con 0x55c3eba52a30 2024-09-19T09:03:24.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.638+0000 7f427cf42740 10 client.15504 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:24.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.638+0000 7f427cf42740 10 client.15504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:03:24.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.638+0000 7f427cf42740 20 client.15504 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:24.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.638+0000 7f427cf42740 1 --2- 172.21.15.120:0/1161453518 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c3eba61680 0x55c3ebae26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:24.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.638+0000 7f427cf42740 4 client.15504 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:03:24.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.638+0000 7f427cf42740 1 -- 172.21.15.120:0/1161453518 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c3eb9c3d60 con 0x55c3eba61680 2024-09-19T09:03:24.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.638+0000 7f4273fff640 1 --2- 172.21.15.120:0/1161453518 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c3eba61680 0x55c3ebae26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:24.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.639+0000 7f4273fff640 1 --2- 172.21.15.120:0/1161453518 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c3eba61680 0x55c3ebae26d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd95dc01bcea125c server_cookie=1f83d1cd10772fe2 in_seq=0 out_seq=0 2024-09-19T09:03:24.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.639+0000 7f4253fff640 10 client.15504 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:24.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.639+0000 7f4253fff640 1 -- 172.21.15.120:0/1161453518 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c3eb9c3d60 con 0x55c3eba61680 2024-09-19T09:03:24.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.639+0000 7f4253fff640 10 client.15504 handle_command_reply: tid=0 2024-09-19T09:03:24.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.639+0000 7f427cf42740 1 client.15504 shutdown 2024-09-19T09:03:24.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.639+0000 7f427cf42740 1 -- 172.21.15.120:0/1161453518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c3eba52a30 msgr2=0x55c3eba69d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:24.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.639+0000 7f427cf42740 1 --2- 172.21.15.120:0/1161453518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c3eba52a30 0x55c3eba69d10 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f426800d530 tx=0x7f4268024690 comp rx=0 tx=0).stop 2024-09-19T09:03:24.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.639+0000 7f427cf42740 1 -- 172.21.15.120:0/1161453518 shutdown_connections 2024-09-19T09:03:24.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.639+0000 7f427cf42740 1 --2- 172.21.15.120:0/1161453518 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c3eba61680 0x55c3ebae26d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.639+0000 7f427cf42740 1 --2- 172.21.15.120:0/1161453518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3eba56dc0 0x55c3eba65580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.639+0000 7f4253fff640 0 client.15504 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:24.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.639+0000 7f427cf42740 1 --2- 172.21.15.120:0/1161453518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c3eba564a0 0x55c3eba65040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.639+0000 7f427cf42740 1 --2- 172.21.15.120:0/1161453518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c3eba52a30 0x55c3eba69d10 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.639+0000 7f427cf42740 1 -- 172.21.15.120:0/1161453518 >> 172.21.15.120:0/1161453518 conn(0x55c3eba5e8b0 msgr2=0x55c3eba59850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:24.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.639+0000 7f427cf42740 1 -- 172.21.15.120:0/1161453518 shutdown_connections 2024-09-19T09:03:24.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.639+0000 7f427cf42740 1 -- 172.21.15.120:0/1161453518 wait complete. 2024-09-19T09:03:24.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.639+0000 7f427cf42740 20 client.15504 trim_cache size 0 max 16384 2024-09-19T09:03:24.644 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:03:24.644 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:03:24.644 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:03:24.644 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:03:24.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.640+0000 7f427ab69640 1 -- 172.21.15.120:0/811581276 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f42640a32d0 msgr2=0x7f42640a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:24.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.640+0000 7f427ab69640 1 --2- 172.21.15.120:0/811581276 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f42640a32d0 0x7f42640a5790 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4268009930 tx=0x7f4268008040 comp rx=0 tx=0).stop 2024-09-19T09:03:24.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.640+0000 7f427ab69640 1 -- 172.21.15.120:0/811581276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4274102570 msgr2=0x7f4274199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:24.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.640+0000 7f427ab69640 1 --2- 172.21.15.120:0/811581276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4274102570 0x7f4274199910 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4260006010 tx=0x7f4260003c80 comp rx=0 tx=0).stop 2024-09-19T09:03:24.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.640+0000 7f427ab69640 1 -- 172.21.15.120:0/811581276 shutdown_connections 2024-09-19T09:03:24.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.640+0000 7f427ab69640 1 --2- 172.21.15.120:0/811581276 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f42640a32d0 0x7f42640a5790 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.640+0000 7f427ab69640 1 --2- 172.21.15.120:0/811581276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4274102ef0 0x7f42741a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.640+0000 7f427ab69640 1 --2- 172.21.15.120:0/811581276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4274102570 0x7f4274199910 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.640+0000 7f427ab69640 1 --2- 172.21.15.120:0/811581276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4274100e20 0x7f42741993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:24.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.640+0000 7f427ab69640 1 -- 172.21.15.120:0/811581276 >> 172.21.15.120:0/811581276 conn(0x7f42740fc7d0 msgr2=0x7f42741068e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:24.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.640+0000 7f427ab69640 1 -- 172.21.15.120:0/811581276 shutdown_connections 2024-09-19T09:03:24.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:24.640+0000 7f427ab69640 1 -- 172.21.15.120:0/811581276 wait complete. 2024-09-19T09:03:25.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:24 smithi120 ceph-mon[31150]: pgmap v357: 97 pgs: 97 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T09:03:25.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:24 smithi196 ceph-mon[35150]: pgmap v357: 97 pgs: 97 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T09:03:25.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:24 smithi153 ceph-mon[35071]: pgmap v357: 97 pgs: 97 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T09:03:25.383 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:03:25.383 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:03:25.634 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:03:26.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.261+0000 7f435d297640 1 -- 172.21.15.120:0/1760232307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4358102ef0 msgr2=0x7f435810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:26.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.261+0000 7f435d297640 1 --2- 172.21.15.120:0/1760232307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4358102ef0 0x7f435810da40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4350009ca0 tx=0x7f435002a950 comp rx=0 tx=0).stop 2024-09-19T09:03:26.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.261+0000 7f435d297640 1 -- 172.21.15.120:0/1760232307 shutdown_connections 2024-09-19T09:03:26.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.261+0000 7f435d297640 1 --2- 172.21.15.120:0/1760232307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4358102ef0 0x7f435810da40 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:26.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.261+0000 7f435d297640 1 --2- 172.21.15.120:0/1760232307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4358102570 0x7f43581029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:26.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.261+0000 7f435d297640 1 --2- 172.21.15.120:0/1760232307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4358100e20 0x7f4358101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:26.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.261+0000 7f435d297640 1 -- 172.21.15.120:0/1760232307 >> 172.21.15.120:0/1760232307 conn(0x7f43580fc7d0 msgr2=0x7f43580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:26.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.261+0000 7f435d297640 1 -- 172.21.15.120:0/1760232307 shutdown_connections 2024-09-19T09:03:26.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.261+0000 7f435d297640 1 -- 172.21.15.120:0/1760232307 wait complete. 2024-09-19T09:03:26.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.262+0000 7f435d297640 1 Processor -- start 2024-09-19T09:03:26.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.262+0000 7f435d297640 1 -- start start 2024-09-19T09:03:26.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.262+0000 7f435d297640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4358100e20 0x7f4358199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:26.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.262+0000 7f435d297640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4358102570 0x7f4358199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:26.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.262+0000 7f435d297640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4358102ef0 0x7f43581a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:26.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.262+0000 7f435d297640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4358110fa0 con 0x7f4358102ef0 2024-09-19T09:03:26.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.262+0000 7f435d297640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4358110e20 con 0x7f4358102570 2024-09-19T09:03:26.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.262+0000 7f435d297640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4358111120 con 0x7f4358100e20 2024-09-19T09:03:26.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.262+0000 7f43577fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4358102ef0 0x7f43581a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:26.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.262+0000 7f43577fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4358102ef0 0x7f43581a43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58954/0 (socket says 172.21.15.120:58954) 2024-09-19T09:03:26.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.262+0000 7f43577fe640 1 -- 172.21.15.120:0/4155831213 learned_addr learned my addr 172.21.15.120:0/4155831213 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:26.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.262+0000 7f43567fc640 1 --2- 172.21.15.120:0/4155831213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4358102570 0x7f4358199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:26.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.262+0000 7f4356ffd640 1 --2- 172.21.15.120:0/4155831213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4358100e20 0x7f4358199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:26.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.263+0000 7f43577fe640 1 -- 172.21.15.120:0/4155831213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4358100e20 msgr2=0x7f4358199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:26.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.263+0000 7f43577fe640 1 --2- 172.21.15.120:0/4155831213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4358100e20 0x7f4358199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:26.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.263+0000 7f43577fe640 1 -- 172.21.15.120:0/4155831213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4358102570 msgr2=0x7f4358199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:26.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.263+0000 7f43577fe640 1 --2- 172.21.15.120:0/4155831213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4358102570 0x7f4358199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:26.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.263+0000 7f43577fe640 1 -- 172.21.15.120:0/4155831213 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43581a4910 con 0x7f4358102ef0 2024-09-19T09:03:26.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.263+0000 7f43577fe640 1 --2- 172.21.15.120:0/4155831213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4358102ef0 0x7f43581a43d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f435002ae70 tx=0x7f435002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:26.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.263+0000 7f433ffff640 1 -- 172.21.15.120:0/4155831213 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4350034040 con 0x7f4358102ef0 2024-09-19T09:03:26.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.263+0000 7f435d297640 1 -- 172.21.15.120:0/4155831213 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43581a4ae0 con 0x7f4358102ef0 2024-09-19T09:03:26.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.263+0000 7f435d297640 1 -- 172.21.15.120:0/4155831213 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43581a4f90 con 0x7f4358102ef0 2024-09-19T09:03:26.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.263+0000 7f433ffff640 1 -- 172.21.15.120:0/4155831213 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4350003d80 con 0x7f4358102ef0 2024-09-19T09:03:26.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.263+0000 7f433ffff640 1 -- 172.21.15.120:0/4155831213 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4350045770 con 0x7f4358102ef0 2024-09-19T09:03:26.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.264+0000 7f435d297640 1 -- 172.21.15.120:0/4155831213 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4344005290 con 0x7f4358102ef0 2024-09-19T09:03:26.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.266+0000 7f433ffff640 1 -- 172.21.15.120:0/4155831213 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f4350037020 con 0x7f4358102ef0 2024-09-19T09:03:26.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.267+0000 7f433ffff640 1 --2- 172.21.15.120:0/4155831213 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43400a32d0 0x7f43400a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:26.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.267+0000 7f433ffff640 1 -- 172.21.15.120:0/4155831213 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f43500f0940 con 0x7f4358102ef0 2024-09-19T09:03:26.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.267+0000 7f4356ffd640 1 --2- 172.21.15.120:0/4155831213 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43400a32d0 0x7f43400a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:26.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.267+0000 7f4356ffd640 1 --2- 172.21.15.120:0/4155831213 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43400a32d0 0x7f43400a5790 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4348005fd0 tx=0x7f4348005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:26.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.271+0000 7f433ffff640 1 -- 172.21.15.120:0/4155831213 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4350081210 con 0x7f4358102ef0 2024-09-19T09:03:26.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.492+0000 7f435d297640 1 -- 172.21.15.120:0/4155831213 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4344005580 con 0x7f4358102ef0 2024-09-19T09:03:26.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.493+0000 7f433ffff640 1 -- 172.21.15.120:0/4155831213 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f43500383b0 con 0x7f4358102ef0 2024-09-19T09:03:26.493 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:03:26.493 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":28,"btime":"2024-09-19T09:02:41:696728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:02:41.207357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:03:26.493 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 28 2024-09-19T09:03:26.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.495+0000 7f435d297640 1 -- 172.21.15.120:0/4155831213 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43400a32d0 msgr2=0x7f43400a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:26.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.495+0000 7f435d297640 1 --2- 172.21.15.120:0/4155831213 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43400a32d0 0x7f43400a5790 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4348005fd0 tx=0x7f4348005850 comp rx=0 tx=0).stop 2024-09-19T09:03:26.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.495+0000 7f435d297640 1 -- 172.21.15.120:0/4155831213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4358102ef0 msgr2=0x7f43581a43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:26.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.495+0000 7f435d297640 1 --2- 172.21.15.120:0/4155831213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4358102ef0 0x7f43581a43d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f435002ae70 tx=0x7f435002f5d0 comp rx=0 tx=0).stop 2024-09-19T09:03:26.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.495+0000 7f435d297640 1 -- 172.21.15.120:0/4155831213 shutdown_connections 2024-09-19T09:03:26.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.495+0000 7f435d297640 1 --2- 172.21.15.120:0/4155831213 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43400a32d0 0x7f43400a5790 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:26.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.495+0000 7f435d297640 1 --2- 172.21.15.120:0/4155831213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4358102ef0 0x7f43581a43d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:26.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.495+0000 7f435d297640 1 --2- 172.21.15.120:0/4155831213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4358102570 0x7f4358199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:26.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.495+0000 7f435d297640 1 --2- 172.21.15.120:0/4155831213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4358100e20 0x7f4358199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:26.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.495+0000 7f435d297640 1 -- 172.21.15.120:0/4155831213 >> 172.21.15.120:0/4155831213 conn(0x7f43580fc7d0 msgr2=0x7f4358106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:26.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.495+0000 7f435d297640 1 -- 172.21.15.120:0/4155831213 shutdown_connections 2024-09-19T09:03:26.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:26.495+0000 7f435d297640 1 -- 172.21.15.120:0/4155831213 wait complete. 2024-09-19T09:03:26.877 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:26 smithi120 ceph-mon[31150]: pgmap v358: 97 pgs: 97 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T09:03:26.877 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:26 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/4155831213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:03:27.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:26 smithi196 ceph-mon[35150]: pgmap v358: 97 pgs: 97 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T09:03:27.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:26 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/4155831213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:03:27.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:26 smithi153 ceph-mon[35071]: pgmap v358: 97 pgs: 97 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T09:03:27.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:26 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/4155831213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:03:27.232 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:03:27.486 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:03:28.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.027+0000 7fd611288640 1 -- 172.21.15.120:0/1787017316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60c106a70 msgr2=0x7fd60c106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:28.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.027+0000 7fd611288640 1 --2- 172.21.15.120:0/1787017316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60c106a70 0x7fd60c106e50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009930 tx=0x7fd5fc02a420 comp rx=0 tx=0).stop 2024-09-19T09:03:28.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.027+0000 7fd611288640 1 -- 172.21.15.120:0/1787017316 shutdown_connections 2024-09-19T09:03:28.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.027+0000 7fd611288640 1 --2- 172.21.15.120:0/1787017316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd60c101410 0x7fd60c10d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.027+0000 7fd611288640 1 --2- 172.21.15.120:0/1787017316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd60c100a70 0x7fd60c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.027+0000 7fd611288640 1 --2- 172.21.15.120:0/1787017316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60c106a70 0x7fd60c106e50 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.027+0000 7fd611288640 1 -- 172.21.15.120:0/1787017316 >> 172.21.15.120:0/1787017316 conn(0x7fd60c0fc790 msgr2=0x7fd60c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:28.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.027+0000 7fd611288640 1 -- 172.21.15.120:0/1787017316 shutdown_connections 2024-09-19T09:03:28.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.027+0000 7fd611288640 1 -- 172.21.15.120:0/1787017316 wait complete. 2024-09-19T09:03:28.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.028+0000 7fd611288640 1 Processor -- start 2024-09-19T09:03:28.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.028+0000 7fd611288640 1 -- start start 2024-09-19T09:03:28.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.028+0000 7fd611288640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60c100a70 0x7fd60c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.028+0000 7fd611288640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd60c101410 0x7fd60c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.028+0000 7fd611288640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd60c106a70 0x7fd60c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.028+0000 7fd611288640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c110e70 con 0x7fd60c100a70 2024-09-19T09:03:28.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.028+0000 7fd611288640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c110cf0 con 0x7fd60c106a70 2024-09-19T09:03:28.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.028+0000 7fd611288640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c110ff0 con 0x7fd60c101410 2024-09-19T09:03:28.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.028+0000 7fd60affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60c100a70 0x7fd60c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.028+0000 7fd60affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60c100a70 0x7fd60c198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58974/0 (socket says 172.21.15.120:58974) 2024-09-19T09:03:28.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.028+0000 7fd60affd640 1 -- 172.21.15.120:0/624169502 learned_addr learned my addr 172.21.15.120:0/624169502 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:28.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.028+0000 7fd60b7fe640 1 --2- 172.21.15.120:0/624169502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd60c106a70 0x7fd60c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.028+0000 7fd60a7fc640 1 --2- 172.21.15.120:0/624169502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd60c101410 0x7fd60c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.029+0000 7fd60affd640 1 -- 172.21.15.120:0/624169502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd60c101410 msgr2=0x7fd60c198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:28.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.029+0000 7fd60affd640 1 --2- 172.21.15.120:0/624169502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd60c101410 0x7fd60c198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.029+0000 7fd60affd640 1 -- 172.21.15.120:0/624169502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd60c106a70 msgr2=0x7fd60c19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:28.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.029+0000 7fd60affd640 1 --2- 172.21.15.120:0/624169502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd60c106a70 0x7fd60c19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.029+0000 7fd60affd640 1 -- 172.21.15.120:0/624169502 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60c19d3f0 con 0x7fd60c100a70 2024-09-19T09:03:28.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.029+0000 7fd60affd640 1 --2- 172.21.15.120:0/624169502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60c100a70 0x7fd60c198440 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009230 tx=0x7fd5fc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:28.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.029+0000 7fd5f3fff640 1 -- 172.21.15.120:0/624169502 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc03f030 con 0x7fd60c100a70 2024-09-19T09:03:28.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.029+0000 7fd5f3fff640 1 -- 172.21.15.120:0/624169502 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5fc003ad0 con 0x7fd60c100a70 2024-09-19T09:03:28.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.029+0000 7fd5f3fff640 1 -- 172.21.15.120:0/624169502 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc035070 con 0x7fd60c100a70 2024-09-19T09:03:28.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.029+0000 7fd611288640 1 -- 172.21.15.120:0/624169502 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd60c19d680 con 0x7fd60c100a70 2024-09-19T09:03:28.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.029+0000 7fd611288640 1 -- 172.21.15.120:0/624169502 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd60c1a4fc0 con 0x7fd60c100a70 2024-09-19T09:03:28.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.031+0000 7fd5f3fff640 1 -- 172.21.15.120:0/624169502 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7fd5fc042030 con 0x7fd60c100a70 2024-09-19T09:03:28.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.032+0000 7fd613661740 1 Processor -- start 2024-09-19T09:03:28.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.032+0000 7fd613661740 1 -- start start 2024-09-19T09:03:28.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.032+0000 7fd613661740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d15a2c2b40 0x55d15a2c1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.032+0000 7fd613661740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d15a2c15d0 0x55d15a2c1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.032+0000 7fd613661740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d15a2cc370 0x55d15a2ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.032+0000 7fd613661740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d15a2272f0 con 0x55d15a2cc370 2024-09-19T09:03:28.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.032+0000 7fd613661740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d15a1cecf0 con 0x55d15a2c15d0 2024-09-19T09:03:28.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.032+0000 7fd613661740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d15a2232b0 con 0x55d15a2c2b40 2024-09-19T09:03:28.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.032+0000 7fd5f3fff640 1 --2- 172.21.15.120:0/624169502 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd5f40a32d0 0x7fd5f40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.032+0000 7fd60a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d15a2cc370 0x55d15a2ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.032+0000 7fd60a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d15a2cc370 0x55d15a2ce760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58978/0 (socket says 172.21.15.120:58978) 2024-09-19T09:03:28.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.032+0000 7fd60a7fc640 1 -- 172.21.15.120:0/2456563342 learned_addr learned my addr 172.21.15.120:0/2456563342 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:28.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.032+0000 7fd60b7fe640 1 --2- 172.21.15.120:0/624169502 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd5f40a32d0 0x7fd5f40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.032+0000 7fd5f3fff640 1 -- 172.21.15.120:0/624169502 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fd5fc0efef0 con 0x7fd60c100a70 2024-09-19T09:03:28.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.032+0000 7fd60affd640 1 --2- 172.21.15.120:0/2456563342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d15a2c15d0 0x55d15a2c1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.033+0000 7fd60b7fe640 1 --2- 172.21.15.120:0/2456563342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d15a2c2b40 0x55d15a2c1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.033+0000 7fd60a7fc640 1 -- 172.21.15.120:0/2456563342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d15a2c2b40 msgr2=0x55d15a2c1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:28.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.033+0000 7fd60a7fc640 1 --2- 172.21.15.120:0/2456563342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d15a2c2b40 0x55d15a2c1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.033+0000 7fd60a7fc640 1 -- 172.21.15.120:0/2456563342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d15a2c15d0 msgr2=0x55d15a2c1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:28.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.033+0000 7fd60a7fc640 1 --2- 172.21.15.120:0/2456563342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d15a2c15d0 0x55d15a2c1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.033+0000 7fd60a7fc640 1 -- 172.21.15.120:0/2456563342 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d15a1b33a0 con 0x55d15a2cc370 2024-09-19T09:03:28.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.033+0000 7fd60b7fe640 1 --2- 172.21.15.120:0/624169502 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd5f40a32d0 0x7fd5f40a5790 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd6040098b0 tx=0x7fd604012a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:28.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.033+0000 7fd60a7fc640 1 --2- 172.21.15.120:0/2456563342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d15a2cc370 0x55d15a2ce760 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd60000bd00 tx=0x7fd6000107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77622e233fff4377 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:28.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.033+0000 7fd5f17fa640 1 -- 172.21.15.120:0/2456563342 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600004030 con 0x55d15a2cc370 2024-09-19T09:03:28.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.033+0000 7fd5f17fa640 1 -- 172.21.15.120:0/2456563342 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6000041d0 con 0x55d15a2cc370 2024-09-19T09:03:28.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.033+0000 7fd5f17fa640 1 -- 172.21.15.120:0/2456563342 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6000117b0 con 0x55d15a2cc370 2024-09-19T09:03:28.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.034+0000 7fd613661740 1 -- 172.21.15.120:0/2456563342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d15a2cc370 msgr2=0x55d15a2ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:28.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.034+0000 7fd613661740 1 --2- 172.21.15.120:0/2456563342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d15a2cc370 0x55d15a2ce760 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd60000bd00 tx=0x7fd6000107a0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.034+0000 7fd613661740 1 -- 172.21.15.120:0/2456563342 shutdown_connections 2024-09-19T09:03:28.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.034+0000 7fd613661740 1 --2- 172.21.15.120:0/2456563342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d15a2cc370 0x55d15a2ce760 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.034+0000 7fd613661740 1 --2- 172.21.15.120:0/2456563342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d15a2c15d0 0x55d15a2c1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.034+0000 7fd613661740 1 --2- 172.21.15.120:0/2456563342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d15a2c2b40 0x55d15a2c1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.034+0000 7fd613661740 1 -- 172.21.15.120:0/2456563342 >> 172.21.15.120:0/2456563342 conn(0x55d15a28a310 msgr2=0x55d15a2bdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:28.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.034+0000 7fd613661740 1 -- 172.21.15.120:0/2456563342 shutdown_connections 2024-09-19T09:03:28.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.034+0000 7fd613661740 1 -- 172.21.15.120:0/2456563342 wait complete. 2024-09-19T09:03:28.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.034+0000 7fd613661740 1 -- start start 2024-09-19T09:03:28.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.034+0000 7fd613661740 1 Processor -- start 2024-09-19T09:03:28.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.035+0000 7fd613661740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d15a2c15d0 0x55d15a2cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.035+0000 7fd613661740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d15a2c2b40 0x55d15a2c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.035+0000 7fd613661740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d15a2cc370 0x55d15a2c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.035+0000 7fd613661740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d15a2cf070 con 0x55d15a2cc370 2024-09-19T09:03:28.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.035+0000 7fd613661740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d15a1e2060 con 0x55d15a2c2b40 2024-09-19T09:03:28.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.035+0000 7fd613661740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d15a2cf1f0 con 0x55d15a2c15d0 2024-09-19T09:03:28.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.035+0000 7fd613661740 10 client.0 _ll_register_callbacks cb 0x55d159f358c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:28.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.035+0000 7fd60b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d15a2cc370 0x55d15a2c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.035+0000 7fd60b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d15a2cc370 0x55d15a2c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58980/0 (socket says 172.21.15.120:58980) 2024-09-19T09:03:28.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.035+0000 7fd60b7fe640 1 -- 172.21.15.120:0/456896600 learned_addr learned my addr 172.21.15.120:0/456896600 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:28.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.035+0000 7fd60a7fc640 1 --2- 172.21.15.120:0/456896600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d15a2c2b40 0x55d15a2c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.035+0000 7fd60affd640 1 --2- 172.21.15.120:0/456896600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d15a2c15d0 0x55d15a2cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.035+0000 7fd60b7fe640 1 -- 172.21.15.120:0/456896600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d15a2c15d0 msgr2=0x55d15a2cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:28.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.035+0000 7fd60b7fe640 1 --2- 172.21.15.120:0/456896600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d15a2c15d0 0x55d15a2cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.035+0000 7fd60b7fe640 1 -- 172.21.15.120:0/456896600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d15a2c2b40 msgr2=0x55d15a2c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:28.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.035+0000 7fd60b7fe640 1 --2- 172.21.15.120:0/456896600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d15a2c2b40 0x55d15a2c90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.035+0000 7fd60b7fe640 1 -- 172.21.15.120:0/456896600 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d159f983c0 con 0x55d15a2cc370 2024-09-19T09:03:28.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.035+0000 7fd60affd640 1 --2- 172.21.15.120:0/456896600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d15a2c15d0 0x55d15a2cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:03:28.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.035+0000 7fd60b7fe640 1 --2- 172.21.15.120:0/456896600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d15a2cc370 0x55d15a2c9840 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd604010b60 tx=0x7fd604009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a29b9f96a651b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:28.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.035+0000 7fd5eeff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:28.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.035+0000 7fd5eeff5640 1 -- 172.21.15.120:0/456896600 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604023030 con 0x55d15a2cc370 2024-09-19T09:03:28.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.035+0000 7fd5eeff5640 1 -- 172.21.15.120:0/456896600 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fd60401ccf0 con 0x55d15a2cc370 2024-09-19T09:03:28.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.035+0000 7fd613661740 1 -- 172.21.15.120:0/456896600 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6000041d0 con 0x55d15a2cc370 2024-09-19T09:03:28.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.036+0000 7fd5eeff5640 1 -- 172.21.15.120:0/456896600 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd60401f420 con 0x55d15a2cc370 2024-09-19T09:03:28.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.036+0000 7fd613661740 10 client.15522 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:28.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.036+0000 7fd613661740 1 -- 172.21.15.120:0/456896600 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d15a2dabc0 con 0x55d15a2cc370 2024-09-19T09:03:28.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.036+0000 7fd5eeff5640 1 -- 172.21.15.120:0/456896600 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6000041d0 con 0x55d15a2cc370 2024-09-19T09:03:28.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.036+0000 7fd5eeff5640 1 -- 172.21.15.120:0/456896600 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd60400f040 con 0x55d15a2cc370 2024-09-19T09:03:28.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.036+0000 7fd613661740 10 client.15522 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:28.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.036+0000 7fd613661740 10 client.15522 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:03:28.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.036+0000 7fd613661740 20 client.15522 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:28.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.036+0000 7fd613661740 1 --2- 172.21.15.120:0/456896600 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55d15a2dc080 0x55d15a2de520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.036+0000 7fd613661740 4 client.15522 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:03:28.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.036+0000 7fd613661740 1 -- 172.21.15.120:0/456896600 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d15a234860 con 0x55d15a2dc080 2024-09-19T09:03:28.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.036+0000 7fd60affd640 1 --2- 172.21.15.120:0/456896600 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55d15a2dc080 0x55d15a2de520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.037+0000 7fd5eeff5640 10 client.15522 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:03:28.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.037+0000 7fd60affd640 1 --2- 172.21.15.120:0/456896600 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55d15a2dc080 0x55d15a2de520 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e163a874a6060881 server_cookie=f65e837d949f09a0 in_seq=0 out_seq=0 2024-09-19T09:03:28.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.038+0000 7fd5eeff5640 1 -- 172.21.15.120:0/456896600 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d15a234860 con 0x55d15a2dc080 2024-09-19T09:03:28.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.038+0000 7fd5eeff5640 10 client.15522 handle_command_reply: tid=0 2024-09-19T09:03:28.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.038+0000 7fd613661740 1 client.15522 shutdown 2024-09-19T09:03:28.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.038+0000 7fd613661740 1 -- 172.21.15.120:0/456896600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d15a2cc370 msgr2=0x55d15a2c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:28.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.038+0000 7fd613661740 1 --2- 172.21.15.120:0/456896600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d15a2cc370 0x55d15a2c9840 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd604010b60 tx=0x7fd604009600 comp rx=0 tx=0).stop 2024-09-19T09:03:28.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.038+0000 7fd613661740 1 -- 172.21.15.120:0/456896600 shutdown_connections 2024-09-19T09:03:28.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.038+0000 7fd613661740 1 --2- 172.21.15.120:0/456896600 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55d15a2dc080 0x55d15a2de520 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.038+0000 7fd5eeff5640 0 client.15522 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:03:28.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.038+0000 7fd613661740 1 --2- 172.21.15.120:0/456896600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d15a2cc370 0x55d15a2c9840 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.038+0000 7fd613661740 1 --2- 172.21.15.120:0/456896600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d15a2c2b40 0x55d15a2c90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.038+0000 7fd613661740 1 --2- 172.21.15.120:0/456896600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d15a2c15d0 0x55d15a2cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.038+0000 7fd613661740 1 -- 172.21.15.120:0/456896600 >> 172.21.15.120:0/456896600 conn(0x55d15a28a310 msgr2=0x55d15a2c3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:28.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.038+0000 7fd613661740 1 -- 172.21.15.120:0/456896600 shutdown_connections 2024-09-19T09:03:28.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.038+0000 7fd613661740 1 -- 172.21.15.120:0/456896600 wait complete. 2024-09-19T09:03:28.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.038+0000 7fd613661740 20 client.15522 trim_cache size 0 max 16384 2024-09-19T09:03:28.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.051+0000 7fd613661740 1 Processor -- start 2024-09-19T09:03:28.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.051+0000 7fd613661740 1 -- start start 2024-09-19T09:03:28.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.051+0000 7fd613661740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d15a2c15d0 0x55d15a2c4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.051+0000 7fd613661740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d15a2c2b40 0x55d15a2c4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.051+0000 7fd613661740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d15a2cc370 0x55d15a2e3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.051+0000 7fd613661740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c1ad590 con 0x55d15a2c2b40 2024-09-19T09:03:28.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.051+0000 7fd613661740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c1af0d0 con 0x55d15a2cc370 2024-09-19T09:03:28.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.051+0000 7fd613661740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c1adc60 con 0x55d15a2c15d0 2024-09-19T09:03:28.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.052+0000 7fd60a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d15a2c2b40 0x55d15a2c4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.052+0000 7fd60a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d15a2c2b40 0x55d15a2c4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58988/0 (socket says 172.21.15.120:58988) 2024-09-19T09:03:28.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.052+0000 7fd60a7fc640 1 -- 172.21.15.120:0/2509140285 learned_addr learned my addr 172.21.15.120:0/2509140285 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:28.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.052+0000 7fd60affd640 1 --2- 172.21.15.120:0/2509140285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d15a2c15d0 0x55d15a2c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.052+0000 7fd60b7fe640 1 --2- 172.21.15.120:0/2509140285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d15a2cc370 0x55d15a2e3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.052+0000 7fd60a7fc640 1 -- 172.21.15.120:0/2509140285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d15a2c15d0 msgr2=0x55d15a2c4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:28.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.052+0000 7fd60a7fc640 1 --2- 172.21.15.120:0/2509140285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d15a2c15d0 0x55d15a2c4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.052+0000 7fd60a7fc640 1 -- 172.21.15.120:0/2509140285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d15a2cc370 msgr2=0x55d15a2e3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:28.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.052+0000 7fd60a7fc640 1 --2- 172.21.15.120:0/2509140285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d15a2cc370 0x55d15a2e3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.052+0000 7fd60a7fc640 1 -- 172.21.15.120:0/2509140285 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d15a2c7600 con 0x55d15a2c2b40 2024-09-19T09:03:28.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.052+0000 7fd60a7fc640 1 --2- 172.21.15.120:0/2509140285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d15a2c2b40 0x55d15a2c4660 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd60000bd00 tx=0x7fd600003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98be9f6832aef5ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:28.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.052+0000 7fd5f0ff9640 1 -- 172.21.15.120:0/2509140285 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60002b070 con 0x55d15a2c2b40 2024-09-19T09:03:28.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.052+0000 7fd5f0ff9640 1 -- 172.21.15.120:0/2509140285 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd600005520 con 0x55d15a2c2b40 2024-09-19T09:03:28.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.052+0000 7fd5f0ff9640 1 -- 172.21.15.120:0/2509140285 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6000116a0 con 0x55d15a2c2b40 2024-09-19T09:03:28.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.053+0000 7fd613661740 1 -- 172.21.15.120:0/2509140285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d15a2c2b40 msgr2=0x55d15a2c4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:28.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.053+0000 7fd613661740 1 --2- 172.21.15.120:0/2509140285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d15a2c2b40 0x55d15a2c4660 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd60000bd00 tx=0x7fd600003f60 comp rx=0 tx=0).stop 2024-09-19T09:03:28.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.053+0000 7fd613661740 1 -- 172.21.15.120:0/2509140285 shutdown_connections 2024-09-19T09:03:28.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.053+0000 7fd613661740 1 --2- 172.21.15.120:0/2509140285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d15a2cc370 0x55d15a2e3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.053+0000 7fd613661740 1 --2- 172.21.15.120:0/2509140285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d15a2c2b40 0x55d15a2c4660 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.053+0000 7fd613661740 1 --2- 172.21.15.120:0/2509140285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d15a2c15d0 0x55d15a2c4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.053+0000 7fd613661740 1 -- 172.21.15.120:0/2509140285 >> 172.21.15.120:0/2509140285 conn(0x55d15a28a310 msgr2=0x55d15a2c81a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:28.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.053+0000 7fd613661740 1 -- 172.21.15.120:0/2509140285 shutdown_connections 2024-09-19T09:03:28.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.053+0000 7fd613661740 1 -- 172.21.15.120:0/2509140285 wait complete. 2024-09-19T09:03:28.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.053+0000 7fd613661740 1 -- start start 2024-09-19T09:03:28.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.054+0000 7fd613661740 1 Processor -- start 2024-09-19T09:03:28.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.054+0000 7fd613661740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d15a2c15d0 0x55d15a2d01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.054+0000 7fd613661740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d15a2c2b40 0x55d15a2d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.054+0000 7fd613661740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d15a2cc370 0x55d15a2d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.054+0000 7fd613661740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c1aea00 con 0x55d15a2c2b40 2024-09-19T09:03:28.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.054+0000 7fd613661740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c1ae330 con 0x55d15a2c15d0 2024-09-19T09:03:28.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.054+0000 7fd613661740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c1ab550 con 0x55d15a2cc370 2024-09-19T09:03:28.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.054+0000 7fd613661740 10 client.0 _ll_register_callbacks cb 0x7fd6040106c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:28.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.054+0000 7fd60a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d15a2c2b40 0x55d15a2d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.054+0000 7fd60a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d15a2c2b40 0x55d15a2d0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58994/0 (socket says 172.21.15.120:58994) 2024-09-19T09:03:28.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.054+0000 7fd60a7fc640 1 -- 172.21.15.120:0/1984080074 learned_addr learned my addr 172.21.15.120:0/1984080074 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:28.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.054+0000 7fd60affd640 1 --2- 172.21.15.120:0/1984080074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d15a2c15d0 0x55d15a2d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.054+0000 7fd60b7fe640 1 --2- 172.21.15.120:0/1984080074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d15a2cc370 0x55d15a2d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.054+0000 7fd60a7fc640 1 -- 172.21.15.120:0/1984080074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d15a2cc370 msgr2=0x55d15a2d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:28.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.054+0000 7fd60a7fc640 1 --2- 172.21.15.120:0/1984080074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d15a2cc370 0x55d15a2d0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.054+0000 7fd60a7fc640 1 -- 172.21.15.120:0/1984080074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d15a2c15d0 msgr2=0x55d15a2d01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:28.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.054+0000 7fd60a7fc640 1 --2- 172.21.15.120:0/1984080074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d15a2c15d0 0x55d15a2d01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.054+0000 7fd60a7fc640 1 -- 172.21.15.120:0/1984080074 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d15a345e10 con 0x55d15a2c2b40 2024-09-19T09:03:28.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.054+0000 7fd60affd640 1 --2- 172.21.15.120:0/1984080074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d15a2c15d0 0x55d15a2d01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:03:28.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.054+0000 7fd60a7fc640 1 --2- 172.21.15.120:0/1984080074 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d15a2c2b40 0x55d15a2d0710 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd60000b3d0 tx=0x7fd600005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a8653db87e6401 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:28.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.054+0000 7fd5eeff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:28.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.055+0000 7fd5eeff5640 1 -- 172.21.15.120:0/1984080074 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60002b070 con 0x55d15a2c2b40 2024-09-19T09:03:28.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.055+0000 7fd5eeff5640 1 -- 172.21.15.120:0/1984080074 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fd6000035f0 con 0x55d15a2c2b40 2024-09-19T09:03:28.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.055+0000 7fd613661740 1 -- 172.21.15.120:0/1984080074 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd600005520 con 0x55d15a2c2b40 2024-09-19T09:03:28.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.055+0000 7fd5eeff5640 1 -- 172.21.15.120:0/1984080074 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd6000037f0 con 0x55d15a2c2b40 2024-09-19T09:03:28.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.055+0000 7fd613661740 10 client.15528 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:28.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.055+0000 7fd613661740 1 -- 172.21.15.120:0/1984080074 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d15a3470f0 con 0x55d15a2c2b40 2024-09-19T09:03:28.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.055+0000 7fd5eeff5640 1 -- 172.21.15.120:0/1984080074 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd600005520 con 0x55d15a2c2b40 2024-09-19T09:03:28.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.055+0000 7fd5eeff5640 1 -- 172.21.15.120:0/1984080074 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd60001b420 con 0x55d15a2c2b40 2024-09-19T09:03:28.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.055+0000 7fd613661740 10 client.15528 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:28.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.055+0000 7fd613661740 10 client.15528 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:03:28.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.055+0000 7fd613661740 20 client.15528 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:28.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.055+0000 7fd613661740 1 --2- 172.21.15.120:0/1984080074 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55d15a2dc080 0x55d15a349530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.055+0000 7fd613661740 4 client.15528 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:03:28.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.055+0000 7fd613661740 1 -- 172.21.15.120:0/1984080074 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d15a22ed70 con 0x55d15a2dc080 2024-09-19T09:03:28.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.056+0000 7fd60affd640 1 --2- 172.21.15.120:0/1984080074 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55d15a2dc080 0x55d15a349530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.056+0000 7fd5eeff5640 10 client.15528 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:03:28.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.056+0000 7fd60affd640 1 --2- 172.21.15.120:0/1984080074 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55d15a2dc080 0x55d15a349530 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6f8962118ae70836 server_cookie=7346e87c1b786655 in_seq=0 out_seq=0 2024-09-19T09:03:28.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.056+0000 7fd5eeff5640 1 -- 172.21.15.120:0/1984080074 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d15a22ed70 con 0x55d15a2dc080 2024-09-19T09:03:28.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.056+0000 7fd5eeff5640 10 client.15528 handle_command_reply: tid=0 2024-09-19T09:03:28.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.056+0000 7fd613661740 1 client.15528 shutdown 2024-09-19T09:03:28.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.056+0000 7fd613661740 1 -- 172.21.15.120:0/1984080074 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d15a2c2b40 msgr2=0x55d15a2d0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:28.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.056+0000 7fd613661740 1 --2- 172.21.15.120:0/1984080074 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d15a2c2b40 0x55d15a2d0710 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd60000b3d0 tx=0x7fd600005740 comp rx=0 tx=0).stop 2024-09-19T09:03:28.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.056+0000 7fd613661740 1 -- 172.21.15.120:0/1984080074 shutdown_connections 2024-09-19T09:03:28.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.056+0000 7fd613661740 1 --2- 172.21.15.120:0/1984080074 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55d15a2dc080 0x55d15a349530 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.056+0000 7fd613661740 1 --2- 172.21.15.120:0/1984080074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d15a2cc370 0x55d15a2d0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.056+0000 7fd613661740 1 --2- 172.21.15.120:0/1984080074 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d15a2c2b40 0x55d15a2d0710 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.057+0000 7fd613661740 1 --2- 172.21.15.120:0/1984080074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d15a2c15d0 0x55d15a2d01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.057+0000 7fd5eeff5640 0 client.15528 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:03:28.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.057+0000 7fd613661740 1 -- 172.21.15.120:0/1984080074 >> 172.21.15.120:0/1984080074 conn(0x55d15a28a310 msgr2=0x55d15a2ca910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:28.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.057+0000 7fd613661740 1 -- 172.21.15.120:0/1984080074 shutdown_connections 2024-09-19T09:03:28.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.057+0000 7fd613661740 1 -- 172.21.15.120:0/1984080074 wait complete. 2024-09-19T09:03:28.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.057+0000 7fd613661740 20 client.15528 trim_cache size 0 max 16384 2024-09-19T09:03:28.072 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:03:28.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.058+0000 7fd611288640 1 -- 172.21.15.120:0/624169502 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd5f40a32d0 msgr2=0x7fd5f40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:28.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.058+0000 7fd611288640 1 --2- 172.21.15.120:0/624169502 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd5f40a32d0 0x7fd5f40a5790 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd6040098b0 tx=0x7fd604012a90 comp rx=0 tx=0).stop 2024-09-19T09:03:28.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.058+0000 7fd611288640 1 -- 172.21.15.120:0/624169502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60c100a70 msgr2=0x7fd60c198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:28.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.058+0000 7fd611288640 1 --2- 172.21.15.120:0/624169502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60c100a70 0x7fd60c198440 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009230 tx=0x7fd5fc004270 comp rx=0 tx=0).stop 2024-09-19T09:03:28.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.058+0000 7fd611288640 1 -- 172.21.15.120:0/624169502 shutdown_connections 2024-09-19T09:03:28.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.058+0000 7fd611288640 1 --2- 172.21.15.120:0/624169502 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd5f40a32d0 0x7fd5f40a5790 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.058+0000 7fd611288640 1 --2- 172.21.15.120:0/624169502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd60c106a70 0x7fd60c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.058+0000 7fd611288640 1 --2- 172.21.15.120:0/624169502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd60c101410 0x7fd60c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.058+0000 7fd611288640 1 --2- 172.21.15.120:0/624169502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60c100a70 0x7fd60c198440 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.058+0000 7fd611288640 1 -- 172.21.15.120:0/624169502 >> 172.21.15.120:0/624169502 conn(0x7fd60c0fc790 msgr2=0x7fd60c06a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:28.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.058+0000 7fd611288640 1 -- 172.21.15.120:0/624169502 shutdown_connections 2024-09-19T09:03:28.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:28.058+0000 7fd611288640 1 -- 172.21.15.120:0/624169502 wait complete. 2024-09-19T09:03:28.568 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 11.2 2024-09-19T09:03:28.569 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:03:28.878 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:28 smithi120 ceph-mon[31150]: pgmap v359: 97 pgs: 97 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T09:03:28.880 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:03:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.915+0000 7f3315d11640 1 -- 172.21.15.120:0/1720586047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33100ffbc0 msgr2=0x7f3310100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.915+0000 7f3315d11640 1 --2- 172.21.15.120:0/1720586047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33100ffbc0 0x7f3310100040 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3304009900 tx=0x7f330402a410 comp rx=0 tx=0).stop 2024-09-19T09:03:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.916+0000 7f3315d11640 1 -- 172.21.15.120:0/1720586047 shutdown_connections 2024-09-19T09:03:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.916+0000 7f3315d11640 1 --2- 172.21.15.120:0/1720586047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3310100580 0x7f3310100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.916+0000 7f3315d11640 1 --2- 172.21.15.120:0/1720586047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33100ffbc0 0x7f3310100040 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.916+0000 7f3315d11640 1 --2- 172.21.15.120:0/1720586047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33101020d0 0x7f33101024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.916+0000 7f3315d11640 1 -- 172.21.15.120:0/1720586047 >> 172.21.15.120:0/1720586047 conn(0x7f33100fb930 msgr2=0x7f33100fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.916+0000 7f3315d11640 1 -- 172.21.15.120:0/1720586047 shutdown_connections 2024-09-19T09:03:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.916+0000 7f3315d11640 1 -- 172.21.15.120:0/1720586047 wait complete. 2024-09-19T09:03:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.917+0000 7f3315d11640 1 Processor -- start 2024-09-19T09:03:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.917+0000 7f3315d11640 1 -- start start 2024-09-19T09:03:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.917+0000 7f3315d11640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33100ffbc0 0x7f3310199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.917+0000 7f3315d11640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3310100580 0x7f331019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.917+0000 7f3315d11640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33101020d0 0x7f33101a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.917+0000 7f3315d11640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f331010f9f0 con 0x7f33101020d0 2024-09-19T09:03:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.917+0000 7f3315d11640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f331010f870 con 0x7f33100ffbc0 2024-09-19T09:03:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.917+0000 7f3315d11640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f331010fb70 con 0x7f3310100580 2024-09-19T09:03:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.917+0000 7f330ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33101020d0 0x7f33101a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.917+0000 7f330ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33101020d0 0x7f33101a1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59012/0 (socket says 172.21.15.120:59012) 2024-09-19T09:03:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.917+0000 7f330ffff640 1 -- 172.21.15.120:0/4133004670 learned_addr learned my addr 172.21.15.120:0/4133004670 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.917+0000 7f330effd640 1 --2- 172.21.15.120:0/4133004670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3310100580 0x7f331019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.917+0000 7f330f7fe640 1 --2- 172.21.15.120:0/4133004670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33100ffbc0 0x7f3310199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.918+0000 7f330effd640 1 -- 172.21.15.120:0/4133004670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33100ffbc0 msgr2=0x7f3310199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.918+0000 7f330effd640 1 --2- 172.21.15.120:0/4133004670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33100ffbc0 0x7f3310199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.918+0000 7f330effd640 1 -- 172.21.15.120:0/4133004670 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33101020d0 msgr2=0x7f33101a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.918+0000 7f330effd640 1 --2- 172.21.15.120:0/4133004670 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33101020d0 0x7f33101a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.918+0000 7f330effd640 1 -- 172.21.15.120:0/4133004670 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33101a1bc0 con 0x7f3310100580 2024-09-19T09:03:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.918+0000 7f330ffff640 1 --2- 172.21.15.120:0/4133004670 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33101020d0 0x7f33101a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:03:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.918+0000 7f330effd640 1 --2- 172.21.15.120:0/4133004670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3310100580 0x7f331019a4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3304009230 tx=0x7f3304004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.918+0000 7f330cff9640 1 -- 172.21.15.120:0/4133004670 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330403f070 con 0x7f3310100580 2024-09-19T09:03:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.918+0000 7f3315d11640 1 -- 172.21.15.120:0/4133004670 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33101a1e50 con 0x7f3310100580 2024-09-19T09:03:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.918+0000 7f330cff9640 1 -- 172.21.15.120:0/4133004670 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3304003cd0 con 0x7f3310100580 2024-09-19T09:03:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.918+0000 7f330cff9640 1 -- 172.21.15.120:0/4133004670 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304035070 con 0x7f3310100580 2024-09-19T09:03:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.919+0000 7f3315d11640 1 -- 172.21.15.120:0/4133004670 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33101a2260 con 0x7f3310100580 2024-09-19T09:03:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.921+0000 7f3318aa6740 1 Processor -- start 2024-09-19T09:03:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.921+0000 7f3318aa6740 1 -- start start 2024-09-19T09:03:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.921+0000 7f3318aa6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f32c2b09a0 0x55f32c2b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.921+0000 7f3318aa6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32c2b12e0 0x55f32c2b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.921+0000 7f3318aa6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32c2b6370 0x55f32c2b8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.921+0000 7f3318aa6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f32c2142f0 con 0x55f32c2b6370 2024-09-19T09:03:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.921+0000 7f3318aa6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f32c1bc270 con 0x55f32c2b12e0 2024-09-19T09:03:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.921+0000 7f3318aa6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f32c2102b0 con 0x55f32c2b09a0 2024-09-19T09:03:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.922+0000 7f330cff9640 1 -- 172.21.15.120:0/4133004670 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f3304043420 con 0x7f3310100580 2024-09-19T09:03:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.922+0000 7f330ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f32c2b09a0 0x55f32c2b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.922+0000 7f330effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32c2b6370 0x55f32c2b8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.922+0000 7f330ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f32c2b09a0 0x55f32c2b0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:45034/0 (socket says 172.21.15.120:45034) 2024-09-19T09:03:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.922+0000 7f330ffff640 1 -- 172.21.15.120:0/928214833 learned_addr learned my addr 172.21.15.120:0/928214833 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.922+0000 7f330f7fe640 1 --2- 172.21.15.120:0/928214833 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32c2b12e0 0x55f32c2b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.922+0000 7f330cff9640 1 --2- 172.21.15.120:0/4133004670 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f32f80a3390 0x7f32f80a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.923+0000 7f330cff9640 1 -- 172.21.15.120:0/4133004670 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f33040efe20 con 0x7f3310100580 2024-09-19T09:03:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.923+0000 7f330effd640 1 -- 172.21.15.120:0/928214833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f32c2b09a0 msgr2=0x55f32c2b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.923+0000 7f330ffff640 1 --2- 172.21.15.120:0/4133004670 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f32f80a3390 0x7f32f80a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.923+0000 7f330effd640 1 --2- 172.21.15.120:0/928214833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f32c2b09a0 0x55f32c2b0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.923+0000 7f330effd640 1 -- 172.21.15.120:0/928214833 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32c2b12e0 msgr2=0x55f32c2b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.923+0000 7f330effd640 1 --2- 172.21.15.120:0/928214833 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32c2b12e0 0x55f32c2b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.923+0000 7f330effd640 1 -- 172.21.15.120:0/928214833 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f32c1bb340 con 0x55f32c2b6370 2024-09-19T09:03:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.923+0000 7f330f7fe640 1 --2- 172.21.15.120:0/928214833 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32c2b12e0 0x55f32c2b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:03:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.923+0000 7f330effd640 1 --2- 172.21.15.120:0/928214833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32c2b6370 0x55f32c2b8830 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f33040ed7d0 tx=0x7f3304009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f14d394046cd48 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.923+0000 7f330ffff640 1 --2- 172.21.15.120:0/4133004670 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f32f80a3390 0x7f32f80a5850 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f330800f600 tx=0x7f3308004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.923+0000 7f32f5ffb640 1 -- 172.21.15.120:0/928214833 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304048030 con 0x55f32c2b6370 2024-09-19T09:03:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.923+0000 7f32f5ffb640 1 -- 172.21.15.120:0/928214833 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33040bc390 con 0x55f32c2b6370 2024-09-19T09:03:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.923+0000 7f32f5ffb640 1 -- 172.21.15.120:0/928214833 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33040c4f50 con 0x55f32c2b6370 2024-09-19T09:03:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.924+0000 7f3318aa6740 1 -- 172.21.15.120:0/928214833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32c2b6370 msgr2=0x55f32c2b8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.924+0000 7f3318aa6740 1 --2- 172.21.15.120:0/928214833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32c2b6370 0x55f32c2b8830 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f33040ed7d0 tx=0x7f3304009260 comp rx=0 tx=0).stop 2024-09-19T09:03:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.925+0000 7f3318aa6740 1 -- 172.21.15.120:0/928214833 shutdown_connections 2024-09-19T09:03:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.925+0000 7f3318aa6740 1 --2- 172.21.15.120:0/928214833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32c2b6370 0x55f32c2b8830 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.925+0000 7f3318aa6740 1 --2- 172.21.15.120:0/928214833 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32c2b12e0 0x55f32c2b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.925+0000 7f3318aa6740 1 --2- 172.21.15.120:0/928214833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f32c2b09a0 0x55f32c2b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.925+0000 7f3318aa6740 1 -- 172.21.15.120:0/928214833 >> 172.21.15.120:0/928214833 conn(0x55f32c2ac010 msgr2=0x55f32c2ac410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.925+0000 7f3318aa6740 1 -- 172.21.15.120:0/928214833 shutdown_connections 2024-09-19T09:03:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.925+0000 7f3318aa6740 1 -- 172.21.15.120:0/928214833 wait complete. 2024-09-19T09:03:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.925+0000 7f3318aa6740 1 -- start start 2024-09-19T09:03:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.925+0000 7f3318aa6740 1 Processor -- start 2024-09-19T09:03:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.926+0000 7f3318aa6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f32c2b09a0 0x55f32c2bab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.926+0000 7f3318aa6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32c2b12e0 0x55f32c2bb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.926+0000 7f3318aa6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32c2c0090 0x55f32c2c2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.926+0000 7f3318aa6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f32c2b9100 con 0x55f32c2b12e0 2024-09-19T09:03:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.926+0000 7f3318aa6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f32c19d290 con 0x55f32c2c0090 2024-09-19T09:03:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.926+0000 7f3318aa6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f32c2b9280 con 0x55f32c2b09a0 2024-09-19T09:03:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.926+0000 7f3318aa6740 10 client.0 _ll_register_callbacks cb 0x55f32c1ce140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.926+0000 7f330ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32c2c0090 0x55f32c2c2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.926+0000 7f330effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32c2b12e0 0x55f32c2bb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.926+0000 7f330ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32c2c0090 0x55f32c2c2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:34780/0 (socket says 172.21.15.120:34780) 2024-09-19T09:03:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.926+0000 7f330ffff640 1 -- 172.21.15.120:0/284122336 learned_addr learned my addr 172.21.15.120:0/284122336 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.926+0000 7f330f7fe640 1 --2- 172.21.15.120:0/284122336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f32c2b09a0 0x55f32c2bab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.926+0000 7f330ffff640 1 -- 172.21.15.120:0/284122336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f32c2b09a0 msgr2=0x55f32c2bab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.926+0000 7f330ffff640 1 --2- 172.21.15.120:0/284122336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f32c2b09a0 0x55f32c2bab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.926+0000 7f330ffff640 1 -- 172.21.15.120:0/284122336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32c2b12e0 msgr2=0x55f32c2bb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.926+0000 7f330ffff640 1 --2- 172.21.15.120:0/284122336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32c2b12e0 0x55f32c2bb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.926+0000 7f330ffff640 1 -- 172.21.15.120:0/284122336 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f32bf27210 con 0x55f32c2c0090 2024-09-19T09:03:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.926+0000 7f330effd640 1 --2- 172.21.15.120:0/284122336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32c2b12e0 0x55f32c2bb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:03:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.927+0000 7f330ffff640 1 --2- 172.21.15.120:0/284122336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32c2c0090 0x55f32c2c2480 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f33080106b0 tx=0x7f3308004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d18f21e4d1bb62a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.927+0000 7f32eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:03:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.927+0000 7f32eeffd640 1 -- 172.21.15.120:0/284122336 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3308004d20 con 0x55f32c2c0090 2024-09-19T09:03:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.927+0000 7f3318aa6740 1 -- 172.21.15.120:0/284122336 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33040bc390 con 0x55f32c2c0090 2024-09-19T09:03:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.927+0000 7f32eeffd640 1 -- 172.21.15.120:0/284122336 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f33080204c0 con 0x55f32c2c0090 2024-09-19T09:03:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.927+0000 7f32eeffd640 1 -- 172.21.15.120:0/284122336 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f33080206c0 con 0x55f32c2c0090 2024-09-19T09:03:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.927+0000 7f3318aa6740 10 client.24998 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.927+0000 7f3318aa6740 1 -- 172.21.15.120:0/284122336 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f32bff3620 con 0x55f32c2c0090 2024-09-19T09:03:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.927+0000 7f32eeffd640 1 -- 172.21.15.120:0/284122336 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33040bc390 con 0x55f32c2c0090 2024-09-19T09:03:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.928+0000 7f32eeffd640 1 -- 172.21.15.120:0/284122336 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f330802aa20 con 0x55f32c2c0090 2024-09-19T09:03:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.928+0000 7f3318aa6740 10 client.24998 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.928+0000 7f3318aa6740 10 client.24998 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:03:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.928+0000 7f3318aa6740 20 client.24998 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.928+0000 7f3318aa6740 1 --2- 172.21.15.120:0/284122336 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f32c2c6c20 0x55f32c2c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.928+0000 7f3318aa6740 4 client.24998 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:03:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.928+0000 7f3318aa6740 1 -- 172.21.15.120:0/284122336 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f32c221a20 con 0x55f32c2c6c20 2024-09-19T09:03:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.928+0000 7f330f7fe640 1 --2- 172.21.15.120:0/284122336 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f32c2c6c20 0x55f32c2c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.929+0000 7f32eeffd640 10 client.24998 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.929+0000 7f330f7fe640 1 --2- 172.21.15.120:0/284122336 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f32c2c6c20 0x55f32c2c9070 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b06e121453147997 server_cookie=59c555fa6e0b6c97 in_seq=0 out_seq=0 2024-09-19T09:03:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.930+0000 7f32eeffd640 1 -- 172.21.15.120:0/284122336 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f32c221a20 con 0x55f32c2c6c20 2024-09-19T09:03:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.930+0000 7f32eeffd640 10 client.24998 handle_command_reply: tid=0 2024-09-19T09:03:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.930+0000 7f3318aa6740 1 client.24998 shutdown 2024-09-19T09:03:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.930+0000 7f3318aa6740 1 -- 172.21.15.120:0/284122336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32c2c0090 msgr2=0x55f32c2c2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.930+0000 7f3318aa6740 1 --2- 172.21.15.120:0/284122336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32c2c0090 0x55f32c2c2480 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f33080106b0 tx=0x7f3308004a40 comp rx=0 tx=0).stop 2024-09-19T09:03:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.930+0000 7f3318aa6740 1 -- 172.21.15.120:0/284122336 shutdown_connections 2024-09-19T09:03:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.930+0000 7f3318aa6740 1 --2- 172.21.15.120:0/284122336 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f32c2c6c20 0x55f32c2c9070 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.930+0000 7f3318aa6740 1 --2- 172.21.15.120:0/284122336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32c2c0090 0x55f32c2c2480 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.930+0000 7f3318aa6740 1 --2- 172.21.15.120:0/284122336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32c2b12e0 0x55f32c2bb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.930+0000 7f3318aa6740 1 --2- 172.21.15.120:0/284122336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f32c2b09a0 0x55f32c2bab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.930+0000 7f3318aa6740 1 -- 172.21.15.120:0/284122336 >> 172.21.15.120:0/284122336 conn(0x55f32c2ac010 msgr2=0x55f32c2ad820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.930+0000 7f32eeffd640 0 client.24998 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.931+0000 7f3318aa6740 1 -- 172.21.15.120:0/284122336 shutdown_connections 2024-09-19T09:03:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.931+0000 7f3318aa6740 1 -- 172.21.15.120:0/284122336 wait complete. 2024-09-19T09:03:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.931+0000 7f3318aa6740 20 client.24998 trim_cache size 0 max 16384 2024-09-19T09:03:28.939 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:03:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.948+0000 7f3318aa6740 1 Processor -- start 2024-09-19T09:03:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.948+0000 7f3318aa6740 1 -- start start 2024-09-19T09:03:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.948+0000 7f3318aa6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f32c2b09a0 0x55f32c2acec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.948+0000 7f3318aa6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32c2b12e0 0x55f32c2ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.948+0000 7f3318aa6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32c2c0090 0x55f32c2ad940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.948+0000 7f3318aa6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f33101ab0d0 con 0x55f32c2b12e0 2024-09-19T09:03:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.948+0000 7f3318aa6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f33101acc10 con 0x55f32c2c0090 2024-09-19T09:03:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.948+0000 7f3318aa6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f33101ab7a0 con 0x55f32c2b09a0 2024-09-19T09:03:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.948+0000 7f330effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32c2b12e0 0x55f32c2ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.948+0000 7f330effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32c2b12e0 0x55f32c2ad400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59044/0 (socket says 172.21.15.120:59044) 2024-09-19T09:03:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.948+0000 7f330effd640 1 -- 172.21.15.120:0/1556656883 learned_addr learned my addr 172.21.15.120:0/1556656883 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.948+0000 7f330f7fe640 1 --2- 172.21.15.120:0/1556656883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f32c2b09a0 0x55f32c2acec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.948+0000 7f330ffff640 1 --2- 172.21.15.120:0/1556656883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32c2c0090 0x55f32c2ad940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.948+0000 7f330effd640 1 -- 172.21.15.120:0/1556656883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f32c2b09a0 msgr2=0x55f32c2acec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.948+0000 7f330effd640 1 --2- 172.21.15.120:0/1556656883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f32c2b09a0 0x55f32c2acec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.948+0000 7f330effd640 1 -- 172.21.15.120:0/1556656883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32c2c0090 msgr2=0x55f32c2ad940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.948+0000 7f330effd640 1 --2- 172.21.15.120:0/1556656883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32c2c0090 0x55f32c2ad940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.949+0000 7f330effd640 1 -- 172.21.15.120:0/1556656883 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f32c2abe30 con 0x55f32c2b12e0 2024-09-19T09:03:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.949+0000 7f330f7fe640 1 --2- 172.21.15.120:0/1556656883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f32c2b09a0 0x55f32c2acec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:03:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.949+0000 7f330ffff640 1 --2- 172.21.15.120:0/1556656883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32c2c0090 0x55f32c2ad940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:03:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.949+0000 7f330effd640 1 --2- 172.21.15.120:0/1556656883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32c2b12e0 0x55f32c2ad400 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f33040c4a20 tx=0x7f33040f3a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be47eaa8cde12039 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.949+0000 7f32f57fa640 1 -- 172.21.15.120:0/1556656883 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304048070 con 0x55f32c2b12e0 2024-09-19T09:03:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.949+0000 7f32f57fa640 1 -- 172.21.15.120:0/1556656883 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33040f37a0 con 0x55f32c2b12e0 2024-09-19T09:03:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.949+0000 7f32f57fa640 1 -- 172.21.15.120:0/1556656883 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33040b8a00 con 0x55f32c2b12e0 2024-09-19T09:03:28.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.950+0000 7f3318aa6740 1 -- 172.21.15.120:0/1556656883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32c2b12e0 msgr2=0x55f32c2ad400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:28.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.950+0000 7f3318aa6740 1 --2- 172.21.15.120:0/1556656883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32c2b12e0 0x55f32c2ad400 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f33040c4a20 tx=0x7f33040f3a40 comp rx=0 tx=0).stop 2024-09-19T09:03:28.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.950+0000 7f3318aa6740 1 -- 172.21.15.120:0/1556656883 shutdown_connections 2024-09-19T09:03:28.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.950+0000 7f3318aa6740 1 --2- 172.21.15.120:0/1556656883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32c2c0090 0x55f32c2ad940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.950+0000 7f3318aa6740 1 --2- 172.21.15.120:0/1556656883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32c2b12e0 0x55f32c2ad400 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.950+0000 7f3318aa6740 1 --2- 172.21.15.120:0/1556656883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f32c2b09a0 0x55f32c2acec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.950+0000 7f3318aa6740 1 -- 172.21.15.120:0/1556656883 >> 172.21.15.120:0/1556656883 conn(0x55f32c2ac010 msgr2=0x55f32c2bab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.950+0000 7f3318aa6740 1 -- 172.21.15.120:0/1556656883 shutdown_connections 2024-09-19T09:03:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.950+0000 7f3318aa6740 1 -- 172.21.15.120:0/1556656883 wait complete. 2024-09-19T09:03:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.951+0000 7f3318aa6740 1 -- start start 2024-09-19T09:03:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.951+0000 7f3318aa6740 1 Processor -- start 2024-09-19T09:03:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.951+0000 7f3318aa6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f32c2b09a0 0x55f32c2dae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.951+0000 7f3318aa6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32c2b12e0 0x55f32c2d7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.951+0000 7f3318aa6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32c2c0090 0x55f32c2d7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.951+0000 7f3318aa6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f33101ac540 con 0x55f32c2b12e0 2024-09-19T09:03:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.951+0000 7f3318aa6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f33101abe70 con 0x55f32c2c0090 2024-09-19T09:03:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.951+0000 7f3318aa6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f33101a9090 con 0x55f32c2b09a0 2024-09-19T09:03:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.951+0000 7f3318aa6740 10 client.0 _ll_register_callbacks cb 0x7f3308011a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.951+0000 7f330effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32c2b12e0 0x55f32c2d7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.951+0000 7f330effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32c2b12e0 0x55f32c2d7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59056/0 (socket says 172.21.15.120:59056) 2024-09-19T09:03:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.951+0000 7f330effd640 1 -- 172.21.15.120:0/2298404341 learned_addr learned my addr 172.21.15.120:0/2298404341 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.951+0000 7f330f7fe640 1 --2- 172.21.15.120:0/2298404341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f32c2b09a0 0x55f32c2dae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.952+0000 7f330ffff640 1 --2- 172.21.15.120:0/2298404341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32c2c0090 0x55f32c2d7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.952+0000 7f330effd640 1 -- 172.21.15.120:0/2298404341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f32c2b09a0 msgr2=0x55f32c2dae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.952+0000 7f330effd640 1 --2- 172.21.15.120:0/2298404341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f32c2b09a0 0x55f32c2dae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.952+0000 7f330effd640 1 -- 172.21.15.120:0/2298404341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32c2c0090 msgr2=0x55f32c2d7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.952+0000 7f330effd640 1 --2- 172.21.15.120:0/2298404341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32c2c0090 0x55f32c2d7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.952+0000 7f330effd640 1 -- 172.21.15.120:0/2298404341 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f32c32f7c0 con 0x55f32c2b12e0 2024-09-19T09:03:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.952+0000 7f330f7fe640 1 --2- 172.21.15.120:0/2298404341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f32c2b09a0 0x55f32c2dae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:03:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.952+0000 7f330ffff640 1 --2- 172.21.15.120:0/2298404341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32c2c0090 0x55f32c2d7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:03:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.952+0000 7f330effd640 1 --2- 172.21.15.120:0/2298404341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32c2b12e0 0x55f32c2d7930 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3304006010 tx=0x7f33040315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bbe95154b62dfe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.952+0000 7f32eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.952+0000 7f32eeffd640 1 -- 172.21.15.120:0/2298404341 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304048070 con 0x55f32c2b12e0 2024-09-19T09:03:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.952+0000 7f32eeffd640 1 -- 172.21.15.120:0/2298404341 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f330403b460 con 0x55f32c2b12e0 2024-09-19T09:03:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.952+0000 7f3318aa6740 1 -- 172.21.15.120:0/2298404341 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33040f37a0 con 0x55f32c2b12e0 2024-09-19T09:03:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.952+0000 7f32eeffd640 1 -- 172.21.15.120:0/2298404341 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f33040bbcf0 con 0x55f32c2b12e0 2024-09-19T09:03:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.952+0000 7f3318aa6740 10 client.15546 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.952+0000 7f3318aa6740 1 -- 172.21.15.120:0/2298404341 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f32c330a00 con 0x55f32c2b12e0 2024-09-19T09:03:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.953+0000 7f32eeffd640 1 -- 172.21.15.120:0/2298404341 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33040f37a0 con 0x55f32c2b12e0 2024-09-19T09:03:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.953+0000 7f32eeffd640 1 -- 172.21.15.120:0/2298404341 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f330404b060 con 0x55f32c2b12e0 2024-09-19T09:03:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.953+0000 7f3318aa6740 10 client.15546 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.953+0000 7f3318aa6740 10 client.15546 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:03:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.953+0000 7f3318aa6740 20 client.15546 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.953+0000 7f3318aa6740 1 --2- 172.21.15.120:0/2298404341 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f32c331110 0x55f32c333500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.953+0000 7f3318aa6740 4 client.15546 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:03:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.953+0000 7f3318aa6740 1 -- 172.21.15.120:0/2298404341 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f32c21bd70 con 0x55f32c331110 2024-09-19T09:03:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.954+0000 7f330f7fe640 1 --2- 172.21.15.120:0/2298404341 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f32c331110 0x55f32c333500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.954+0000 7f32eeffd640 10 client.15546 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.954+0000 7f330f7fe640 1 --2- 172.21.15.120:0/2298404341 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f32c331110 0x55f32c333500 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=929b86ccf6f2402f server_cookie=d95fdad9e2b6407a in_seq=0 out_seq=0 2024-09-19T09:03:28.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.958+0000 7f32eeffd640 1 -- 172.21.15.120:0/2298404341 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f32c21bd70 con 0x55f32c331110 2024-09-19T09:03:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.958+0000 7f32eeffd640 10 client.15546 handle_command_reply: tid=0 2024-09-19T09:03:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.958+0000 7f3318aa6740 1 client.15546 shutdown 2024-09-19T09:03:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.958+0000 7f3318aa6740 1 -- 172.21.15.120:0/2298404341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32c2b12e0 msgr2=0x55f32c2d7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.958+0000 7f3318aa6740 1 --2- 172.21.15.120:0/2298404341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32c2b12e0 0x55f32c2d7930 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3304006010 tx=0x7f33040315b0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.958+0000 7f3318aa6740 1 -- 172.21.15.120:0/2298404341 shutdown_connections 2024-09-19T09:03:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.958+0000 7f3318aa6740 1 --2- 172.21.15.120:0/2298404341 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f32c331110 0x55f32c333500 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.958+0000 7f3318aa6740 1 --2- 172.21.15.120:0/2298404341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32c2c0090 0x55f32c2d7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.958+0000 7f3318aa6740 1 --2- 172.21.15.120:0/2298404341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32c2b12e0 0x55f32c2d7930 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.958+0000 7f3318aa6740 1 --2- 172.21.15.120:0/2298404341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f32c2b09a0 0x55f32c2dae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.958+0000 7f3318aa6740 1 -- 172.21.15.120:0/2298404341 >> 172.21.15.120:0/2298404341 conn(0x55f32c2ac010 msgr2=0x55f32c2aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.958+0000 7f32eeffd640 0 client.15546 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.958+0000 7f3318aa6740 1 -- 172.21.15.120:0/2298404341 shutdown_connections 2024-09-19T09:03:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.958+0000 7f3318aa6740 1 -- 172.21.15.120:0/2298404341 wait complete. 2024-09-19T09:03:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.959+0000 7f3318aa6740 20 client.15546 trim_cache size 0 max 16384 2024-09-19T09:03:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:03:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:03:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:03:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:03:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.959+0000 7f3315d11640 1 -- 172.21.15.120:0/4133004670 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f32f80a3390 msgr2=0x7f32f80a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.959+0000 7f3315d11640 1 --2- 172.21.15.120:0/4133004670 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f32f80a3390 0x7f32f80a5850 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f330800f600 tx=0x7f3308004ab0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.959+0000 7f3315d11640 1 -- 172.21.15.120:0/4133004670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3310100580 msgr2=0x7f331019a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.959+0000 7f3315d11640 1 --2- 172.21.15.120:0/4133004670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3310100580 0x7f331019a4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3304009230 tx=0x7f3304004270 comp rx=0 tx=0).stop 2024-09-19T09:03:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.959+0000 7f3315d11640 1 -- 172.21.15.120:0/4133004670 shutdown_connections 2024-09-19T09:03:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.959+0000 7f3315d11640 1 --2- 172.21.15.120:0/4133004670 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f32f80a3390 0x7f32f80a5850 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.959+0000 7f3315d11640 1 --2- 172.21.15.120:0/4133004670 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33101020d0 0x7f33101a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.959+0000 7f3315d11640 1 --2- 172.21.15.120:0/4133004670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3310100580 0x7f331019a4e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.959+0000 7f3315d11640 1 --2- 172.21.15.120:0/4133004670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33100ffbc0 0x7f3310199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.959+0000 7f3315d11640 1 -- 172.21.15.120:0/4133004670 >> 172.21.15.120:0/4133004670 conn(0x7f33100fb930 msgr2=0x7f33100fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.959+0000 7f3315d11640 1 -- 172.21.15.120:0/4133004670 shutdown_connections 2024-09-19T09:03:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:28.959+0000 7f3315d11640 1 -- 172.21.15.120:0/4133004670 wait complete. 2024-09-19T09:03:29.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:28 smithi196 ceph-mon[35150]: pgmap v359: 97 pgs: 97 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T09:03:29.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:28 smithi153 ceph-mon[35071]: pgmap v359: 97 pgs: 97 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T09:03:29.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.466+0000 7f1c58c4b640 1 -- 172.21.15.120:0/3498945562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c54102790 msgr2=0x7f1c541067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:29.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.466+0000 7f1c58c4b640 1 --2- 172.21.15.120:0/3498945562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c54102790 0x7f1c541067e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009900 tx=0x7f1c3c02a410 comp rx=0 tx=0).stop 2024-09-19T09:03:29.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.466+0000 7f1c58c4b640 1 -- 172.21.15.120:0/3498945562 shutdown_connections 2024-09-19T09:03:29.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.466+0000 7f1c58c4b640 1 --2- 172.21.15.120:0/3498945562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c54106f10 0x7f1c5410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:29.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.466+0000 7f1c58c4b640 1 --2- 172.21.15.120:0/3498945562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c54102790 0x7f1c541067e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:29.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.466+0000 7f1c58c4b640 1 --2- 172.21.15.120:0/3498945562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c54101de0 0x7f1c541021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:29.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.466+0000 7f1c58c4b640 1 -- 172.21.15.120:0/3498945562 >> 172.21.15.120:0/3498945562 conn(0x7f1c540fd8e0 msgr2=0x7f1c540ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:29.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.466+0000 7f1c58c4b640 1 -- 172.21.15.120:0/3498945562 shutdown_connections 2024-09-19T09:03:29.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.466+0000 7f1c58c4b640 1 -- 172.21.15.120:0/3498945562 wait complete. 2024-09-19T09:03:29.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.467+0000 7f1c58c4b640 1 Processor -- start 2024-09-19T09:03:29.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.467+0000 7f1c58c4b640 1 -- start start 2024-09-19T09:03:29.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.467+0000 7f1c58c4b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c54101de0 0x7f1c54198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:29.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.467+0000 7f1c58c4b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c54102790 0x7f1c541989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:29.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.467+0000 7f1c58c4b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c54106f10 0x7f1c5419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:29.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.467+0000 7f1c58c4b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1c54111f30 con 0x7f1c54101de0 2024-09-19T09:03:29.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.467+0000 7f1c58c4b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1c54111db0 con 0x7f1c54106f10 2024-09-19T09:03:29.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.467+0000 7f1c58c4b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1c541120b0 con 0x7f1c54102790 2024-09-19T09:03:29.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.468+0000 7f1c5259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c54101de0 0x7f1c54198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:29.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.468+0000 7f1c5259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c54101de0 0x7f1c54198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59072/0 (socket says 172.21.15.120:59072) 2024-09-19T09:03:29.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.468+0000 7f1c5259c640 1 -- 172.21.15.120:0/4146364220 learned_addr learned my addr 172.21.15.120:0/4146364220 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:29.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.468+0000 7f1c4bfff640 1 --2- 172.21.15.120:0/4146364220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c54102790 0x7f1c541989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:29.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.468+0000 7f1c52d9d640 1 --2- 172.21.15.120:0/4146364220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c54106f10 0x7f1c5419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:29.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.468+0000 7f1c5259c640 1 -- 172.21.15.120:0/4146364220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c54102790 msgr2=0x7f1c541989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:29.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.468+0000 7f1c5259c640 1 --2- 172.21.15.120:0/4146364220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c54102790 0x7f1c541989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:29.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.468+0000 7f1c5259c640 1 -- 172.21.15.120:0/4146364220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c54106f10 msgr2=0x7f1c5419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:29.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.468+0000 7f1c5259c640 1 --2- 172.21.15.120:0/4146364220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c54106f10 0x7f1c5419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:29.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.468+0000 7f1c5259c640 1 -- 172.21.15.120:0/4146364220 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5419d440 con 0x7f1c54101de0 2024-09-19T09:03:29.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.468+0000 7f1c5259c640 1 --2- 172.21.15.120:0/4146364220 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c54101de0 0x7f1c54198490 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400d950 tx=0x7f1c4400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:29.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.468+0000 7f1c4b7fe640 1 -- 172.21.15.120:0/4146364220 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44017030 con 0x7f1c54101de0 2024-09-19T09:03:29.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.468+0000 7f1c58c4b640 1 -- 172.21.15.120:0/4146364220 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c5419d730 con 0x7f1c54101de0 2024-09-19T09:03:29.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.468+0000 7f1c4b7fe640 1 -- 172.21.15.120:0/4146364220 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c44004020 con 0x7f1c54101de0 2024-09-19T09:03:29.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.468+0000 7f1c4b7fe640 1 -- 172.21.15.120:0/4146364220 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c440047a0 con 0x7f1c54101de0 2024-09-19T09:03:29.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.468+0000 7f1c58c4b640 1 -- 172.21.15.120:0/4146364220 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c54074420 con 0x7f1c54101de0 2024-09-19T09:03:29.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.470+0000 7f1c58c4b640 1 -- 172.21.15.120:0/4146364220 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c541021c0 con 0x7f1c54101de0 2024-09-19T09:03:29.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.471+0000 7f1c4b7fe640 1 -- 172.21.15.120:0/4146364220 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f1c440041d0 con 0x7f1c54101de0 2024-09-19T09:03:29.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.471+0000 7f1c4b7fe640 1 --2- 172.21.15.120:0/4146364220 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1c400a32d0 0x7f1c400a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:29.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.471+0000 7f1c4bfff640 1 --2- 172.21.15.120:0/4146364220 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1c400a32d0 0x7f1c400a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:29.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.471+0000 7f1c4b7fe640 1 -- 172.21.15.120:0/4146364220 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f1c440cf480 con 0x7f1c54101de0 2024-09-19T09:03:29.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.472+0000 7f1c4bfff640 1 --2- 172.21.15.120:0/4146364220 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1c400a32d0 0x7f1c400a5790 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009230 tx=0x7f1c3c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:29.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.476+0000 7f1c4b7fe640 1 -- 172.21.15.120:0/4146364220 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1c4409c040 con 0x7f1c54101de0 2024-09-19T09:03:29.538 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:03:29.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.762+0000 7f1c58c4b640 1 -- 172.21.15.120:0/4146364220 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c5410a0f0 con 0x7f1c54101de0 2024-09-19T09:03:29.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.763+0000 7f1c4b7fe640 1 -- 172.21.15.120:0/4146364220 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f1c4409b940 con 0x7f1c54101de0 2024-09-19T09:03:29.763 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:03:29.763 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":28,"btime":"2024-09-19T09:02:41:696728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:02:41.207357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:03:29.763 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 28 2024-09-19T09:03:29.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.765+0000 7f1c58c4b640 1 -- 172.21.15.120:0/4146364220 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1c400a32d0 msgr2=0x7f1c400a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:29.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.765+0000 7f1c58c4b640 1 --2- 172.21.15.120:0/4146364220 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1c400a32d0 0x7f1c400a5790 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009230 tx=0x7f1c3c0058d0 comp rx=0 tx=0).stop 2024-09-19T09:03:29.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.765+0000 7f1c58c4b640 1 -- 172.21.15.120:0/4146364220 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c54101de0 msgr2=0x7f1c54198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:29.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.765+0000 7f1c58c4b640 1 --2- 172.21.15.120:0/4146364220 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c54101de0 0x7f1c54198490 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400d950 tx=0x7f1c4400de10 comp rx=0 tx=0).stop 2024-09-19T09:03:29.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.765+0000 7f1c58c4b640 1 -- 172.21.15.120:0/4146364220 shutdown_connections 2024-09-19T09:03:29.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.765+0000 7f1c58c4b640 1 --2- 172.21.15.120:0/4146364220 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1c400a32d0 0x7f1c400a5790 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:29.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.765+0000 7f1c58c4b640 1 --2- 172.21.15.120:0/4146364220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c54106f10 0x7f1c5419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:29.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.765+0000 7f1c58c4b640 1 --2- 172.21.15.120:0/4146364220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c54102790 0x7f1c541989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:29.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.765+0000 7f1c58c4b640 1 --2- 172.21.15.120:0/4146364220 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c54101de0 0x7f1c54198490 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:29.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.765+0000 7f1c58c4b640 1 -- 172.21.15.120:0/4146364220 >> 172.21.15.120:0/4146364220 conn(0x7f1c540fd8e0 msgr2=0x7f1c540fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:29.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.765+0000 7f1c58c4b640 1 -- 172.21.15.120:0/4146364220 shutdown_connections 2024-09-19T09:03:29.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:29.765+0000 7f1c58c4b640 1 -- 172.21.15.120:0/4146364220 wait complete. 2024-09-19T09:03:30.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:29 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/4146364220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:03:30.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:29 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/4146364220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:03:30.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:29 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/4146364220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:03:30.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.223+0000 7f4c2c594640 1 -- 172.21.15.120:0/2418773145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c24106f20 msgr2=0x7f4c2410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:30.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.223+0000 7f4c2c594640 1 --2- 172.21.15.120:0/2418773145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c24106f20 0x7f4c2410eaa0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009ca0 tx=0x7f4c1c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.224+0000 7f4c2c594640 1 -- 172.21.15.120:0/2418773145 shutdown_connections 2024-09-19T09:03:30.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.224+0000 7f4c2c594640 1 --2- 172.21.15.120:0/2418773145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c24106f20 0x7f4c2410eaa0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.224+0000 7f4c2c594640 1 --2- 172.21.15.120:0/2418773145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c241027a0 0x7f4c241067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.224+0000 7f4c2c594640 1 --2- 172.21.15.120:0/2418773145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c24101df0 0x7f4c241021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.224+0000 7f4c2c594640 1 -- 172.21.15.120:0/2418773145 >> 172.21.15.120:0/2418773145 conn(0x7f4c240fd8f0 msgr2=0x7f4c240ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:30.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.224+0000 7f4c2c594640 1 -- 172.21.15.120:0/2418773145 shutdown_connections 2024-09-19T09:03:30.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.224+0000 7f4c2c594640 1 -- 172.21.15.120:0/2418773145 wait complete. 2024-09-19T09:03:30.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.224+0000 7f4c2c594640 1 Processor -- start 2024-09-19T09:03:30.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.224+0000 7f4c2c594640 1 -- start start 2024-09-19T09:03:30.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.224+0000 7f4c2c594640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c24101df0 0x7f4c241983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:30.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.224+0000 7f4c2c594640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c241027a0 0x7f4c24198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:30.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.224+0000 7f4c2c594640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c24106f20 0x7f4c2419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:30.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.225+0000 7f4c2c594640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4c24111f80 con 0x7f4c24106f20 2024-09-19T09:03:30.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.225+0000 7f4c2c594640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4c24111e00 con 0x7f4c24101df0 2024-09-19T09:03:30.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.225+0000 7f4c2c594640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4c24112100 con 0x7f4c241027a0 2024-09-19T09:03:30.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.225+0000 7f4c2ab31640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c24106f20 0x7f4c2419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:30.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.225+0000 7f4c2ab31640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c24106f20 0x7f4c2419cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59096/0 (socket says 172.21.15.120:59096) 2024-09-19T09:03:30.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.225+0000 7f4c2ab31640 1 -- 172.21.15.120:0/2122498974 learned_addr learned my addr 172.21.15.120:0/2122498974 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:30.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.225+0000 7f4c29b2f640 1 --2- 172.21.15.120:0/2122498974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c241027a0 0x7f4c24198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:30.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.225+0000 7f4c2a330640 1 --2- 172.21.15.120:0/2122498974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c24101df0 0x7f4c241983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:30.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.225+0000 7f4c2ab31640 1 -- 172.21.15.120:0/2122498974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c241027a0 msgr2=0x7f4c24198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:30.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.225+0000 7f4c2ab31640 1 --2- 172.21.15.120:0/2122498974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c241027a0 0x7f4c24198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.225+0000 7f4c2ab31640 1 -- 172.21.15.120:0/2122498974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c24101df0 msgr2=0x7f4c241983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:30.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.225+0000 7f4c2ab31640 1 --2- 172.21.15.120:0/2122498974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c24101df0 0x7f4c241983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.225+0000 7f4c2ab31640 1 -- 172.21.15.120:0/2122498974 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c2419d370 con 0x7f4c24106f20 2024-09-19T09:03:30.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.225+0000 7f4c2ab31640 1 --2- 172.21.15.120:0/2122498974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c24106f20 0x7f4c2419cc90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c031a20 tx=0x7f4c1c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:30.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.225+0000 7f4c1b7fe640 1 -- 172.21.15.120:0/2122498974 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c003c30 con 0x7f4c24106f20 2024-09-19T09:03:30.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.225+0000 7f4c1b7fe640 1 -- 172.21.15.120:0/2122498974 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c1c003dd0 con 0x7f4c24106f20 2024-09-19T09:03:30.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.225+0000 7f4c2c594640 1 -- 172.21.15.120:0/2122498974 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c2419d600 con 0x7f4c24106f20 2024-09-19T09:03:30.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.225+0000 7f4c1b7fe640 1 -- 172.21.15.120:0/2122498974 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c02fcf0 con 0x7f4c24106f20 2024-09-19T09:03:30.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.226+0000 7f4c2c594640 1 -- 172.21.15.120:0/2122498974 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c241a4f40 con 0x7f4c24106f20 2024-09-19T09:03:30.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.228+0000 7f4c1b7fe640 1 -- 172.21.15.120:0/2122498974 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f4c1c004050 con 0x7f4c24106f20 2024-09-19T09:03:30.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.228+0000 7f4c2e96d740 1 Processor -- start 2024-09-19T09:03:30.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.228+0000 7f4c2e96d740 1 -- start start 2024-09-19T09:03:30.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.229+0000 7f4c2e96d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599bfa8df50 0x5599bfa8c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:30.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.229+0000 7f4c2e96d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599bfa8c9e0 0x5599bfa8ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:30.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.229+0000 7f4c2e96d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599bfa97780 0x5599bfa99b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:30.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.229+0000 7f4c2e96d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5599bf9ee2f0 con 0x5599bfa8df50 2024-09-19T09:03:30.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.229+0000 7f4c1b7fe640 1 --2- 172.21.15.120:0/2122498974 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4c100a3350 0x7f4c100a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:30.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.229+0000 7f4c2e96d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5599bf995cf0 con 0x5599bfa8c9e0 2024-09-19T09:03:30.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.229+0000 7f4c2e96d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5599bf9ea2b0 con 0x5599bfa97780 2024-09-19T09:03:30.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.229+0000 7f4c29b2f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599bfa8df50 0x5599bfa8c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:30.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.229+0000 7f4c2ab31640 1 --2- 172.21.15.120:0/2122498974 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4c100a3350 0x7f4c100a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:30.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.229+0000 7f4c2a330640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599bfa8c9e0 0x5599bfa8ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:30.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.229+0000 7f4c29b2f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599bfa8df50 0x5599bfa8c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59104/0 (socket says 172.21.15.120:59104) 2024-09-19T09:03:30.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.229+0000 7f4c29b2f640 1 -- 172.21.15.120:0/941801116 learned_addr learned my addr 172.21.15.120:0/941801116 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:30.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.229+0000 7f4c29b2f640 1 --2- 172.21.15.120:0/941801116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599bfa97780 0x5599bfa99b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:30.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.229+0000 7f4c1b7fe640 1 -- 172.21.15.120:0/2122498974 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f4c1c0f0ca0 con 0x7f4c24106f20 2024-09-19T09:03:30.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.229+0000 7f4c29b2f640 1 -- 172.21.15.120:0/941801116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599bfa97780 msgr2=0x5599bfa99b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:30.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.229+0000 7f4c29b2f640 1 --2- 172.21.15.120:0/941801116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599bfa97780 0x5599bfa99b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.229+0000 7f4c29b2f640 1 -- 172.21.15.120:0/941801116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599bfa8c9e0 msgr2=0x5599bfa8ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:30.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.229+0000 7f4c29b2f640 1 --2- 172.21.15.120:0/941801116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599bfa8c9e0 0x5599bfa8ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.229+0000 7f4c29b2f640 1 -- 172.21.15.120:0/941801116 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599bf97a3a0 con 0x5599bfa8df50 2024-09-19T09:03:30.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.229+0000 7f4c2a330640 1 --2- 172.21.15.120:0/941801116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599bfa8c9e0 0x5599bfa8ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:03:30.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.229+0000 7f4c2ab31640 1 --2- 172.21.15.120:0/2122498974 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4c100a3350 0x7f4c100a5810 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c0bd3b0 tx=0x7f4c1c037c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:30.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.230+0000 7f4c29b2f640 1 --2- 172.21.15.120:0/941801116 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599bfa8df50 0x5599bfa8c4a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4c14004650 tx=0x7f4c140041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa33289480cc031 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:30.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.230+0000 7f4c18ff9640 1 -- 172.21.15.120:0/941801116 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c14005020 con 0x5599bfa8df50 2024-09-19T09:03:30.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.230+0000 7f4c18ff9640 1 -- 172.21.15.120:0/941801116 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c140056a0 con 0x5599bfa8df50 2024-09-19T09:03:30.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.230+0000 7f4c18ff9640 1 -- 172.21.15.120:0/941801116 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c140106c0 con 0x5599bfa8df50 2024-09-19T09:03:30.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.230+0000 7f4c2e96d740 1 -- 172.21.15.120:0/941801116 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599bfa8df50 msgr2=0x5599bfa8c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:30.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.230+0000 7f4c2e96d740 1 --2- 172.21.15.120:0/941801116 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599bfa8df50 0x5599bfa8c4a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4c14004650 tx=0x7f4c140041d0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.231+0000 7f4c2e96d740 1 -- 172.21.15.120:0/941801116 shutdown_connections 2024-09-19T09:03:30.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.231+0000 7f4c2e96d740 1 --2- 172.21.15.120:0/941801116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599bfa97780 0x5599bfa99b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.231+0000 7f4c2e96d740 1 --2- 172.21.15.120:0/941801116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599bfa8c9e0 0x5599bfa8ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.231+0000 7f4c2e96d740 1 --2- 172.21.15.120:0/941801116 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599bfa8df50 0x5599bfa8c4a0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.231+0000 7f4c2e96d740 1 -- 172.21.15.120:0/941801116 >> 172.21.15.120:0/941801116 conn(0x5599bfa51310 msgr2=0x5599bfa89070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:30.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.231+0000 7f4c2e96d740 1 -- 172.21.15.120:0/941801116 shutdown_connections 2024-09-19T09:03:30.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.231+0000 7f4c2e96d740 1 -- 172.21.15.120:0/941801116 wait complete. 2024-09-19T09:03:30.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.231+0000 7f4c2e96d740 1 -- start start 2024-09-19T09:03:30.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.231+0000 7f4c2e96d740 1 Processor -- start 2024-09-19T09:03:30.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.231+0000 7f4c2e96d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599bfa8c9e0 0x5599bfa97130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:30.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.231+0000 7f4c2e96d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599bfa8df50 0x5599bfa943e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:30.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.231+0000 7f4c2e96d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599bfa97780 0x5599bfa94980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:30.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.231+0000 7f4c2e96d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5599bfa9a420 con 0x5599bfa97780 2024-09-19T09:03:30.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.231+0000 7f4c2e96d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5599bf9a9060 con 0x5599bfa8c9e0 2024-09-19T09:03:30.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.231+0000 7f4c2e96d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5599bfa9a5a0 con 0x5599bfa8df50 2024-09-19T09:03:30.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.231+0000 7f4c2e96d740 10 client.0 _ll_register_callbacks cb 0x5599bf6fc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:30.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.232+0000 7f4c29b2f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599bfa97780 0x5599bfa94980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:30.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.232+0000 7f4c29b2f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599bfa97780 0x5599bfa94980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59118/0 (socket says 172.21.15.120:59118) 2024-09-19T09:03:30.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.232+0000 7f4c29b2f640 1 -- 172.21.15.120:0/4076727045 learned_addr learned my addr 172.21.15.120:0/4076727045 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:30.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.232+0000 7f4c2a330640 1 --2- 172.21.15.120:0/4076727045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599bfa8df50 0x5599bfa943e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:30.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.232+0000 7f4c29b2f640 1 --2- 172.21.15.120:0/4076727045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599bfa8c9e0 0x5599bfa97130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:30.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.232+0000 7f4c29b2f640 1 -- 172.21.15.120:0/4076727045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599bfa8df50 msgr2=0x5599bfa943e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:30.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.232+0000 7f4c29b2f640 1 --2- 172.21.15.120:0/4076727045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599bfa8df50 0x5599bfa943e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.232+0000 7f4c29b2f640 1 -- 172.21.15.120:0/4076727045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599bfa8c9e0 msgr2=0x5599bfa97130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:30.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.232+0000 7f4c29b2f640 1 --2- 172.21.15.120:0/4076727045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599bfa8c9e0 0x5599bfa97130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.232+0000 7f4c29b2f640 1 -- 172.21.15.120:0/4076727045 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599bf75f3c0 con 0x5599bfa97780 2024-09-19T09:03:30.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.232+0000 7f4c29b2f640 1 --2- 172.21.15.120:0/4076727045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599bfa97780 0x5599bfa94980 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4c14005940 tx=0x7f4c14004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385fce04d21ed992 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:30.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.232+0000 7f4c0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:30.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.232+0000 7f4c0dffb640 1 -- 172.21.15.120:0/4076727045 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c140051e0 con 0x5599bfa97780 2024-09-19T09:03:30.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.232+0000 7f4c0dffb640 1 -- 172.21.15.120:0/4076727045 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f4c14005380 con 0x5599bfa97780 2024-09-19T09:03:30.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.232+0000 7f4c2e96d740 1 -- 172.21.15.120:0/4076727045 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c140056a0 con 0x5599bfa97780 2024-09-19T09:03:30.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.232+0000 7f4c0dffb640 1 -- 172.21.15.120:0/4076727045 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4c14010460 con 0x5599bfa97780 2024-09-19T09:03:30.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.232+0000 7f4c2e96d740 10 client.15564 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:30.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.232+0000 7f4c2e96d740 1 -- 172.21.15.120:0/4076727045 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5599bfaa5e70 con 0x5599bfa97780 2024-09-19T09:03:30.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.233+0000 7f4c0dffb640 1 -- 172.21.15.120:0/4076727045 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c140056a0 con 0x5599bfa97780 2024-09-19T09:03:30.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.233+0000 7f4c0dffb640 1 -- 172.21.15.120:0/4076727045 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4c140304e0 con 0x5599bfa97780 2024-09-19T09:03:30.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.233+0000 7f4c2e96d740 10 client.15564 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:30.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.233+0000 7f4c2e96d740 10 client.15564 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:03:30.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.233+0000 7f4c2e96d740 20 client.15564 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:30.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.233+0000 7f4c2e96d740 1 --2- 172.21.15.120:0/4076727045 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5599bfaa71c0 0x5599bfaa95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:30.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.233+0000 7f4c2e96d740 4 client.15564 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:03:30.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.233+0000 7f4c2e96d740 1 -- 172.21.15.120:0/4076727045 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599bf9fb860 con 0x5599bfaa71c0 2024-09-19T09:03:30.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.233+0000 7f4c2a330640 1 --2- 172.21.15.120:0/4076727045 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5599bfaa71c0 0x5599bfaa95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:30.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.234+0000 7f4c2a330640 1 --2- 172.21.15.120:0/4076727045 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5599bfaa71c0 0x5599bfaa95b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6970175656597541 server_cookie=902f87fc0055a249 in_seq=0 out_seq=0 2024-09-19T09:03:30.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.234+0000 7f4c0dffb640 10 client.15564 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:03:30.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.234+0000 7f4c0dffb640 1 -- 172.21.15.120:0/4076727045 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5599bf9fb860 con 0x5599bfaa71c0 2024-09-19T09:03:30.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.234+0000 7f4c0dffb640 10 client.15564 handle_command_reply: tid=0 2024-09-19T09:03:30.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.234+0000 7f4c2e96d740 1 client.15564 shutdown 2024-09-19T09:03:30.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.234+0000 7f4c2e96d740 1 -- 172.21.15.120:0/4076727045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599bfa97780 msgr2=0x5599bfa94980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:30.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.234+0000 7f4c2e96d740 1 --2- 172.21.15.120:0/4076727045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599bfa97780 0x5599bfa94980 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4c14005940 tx=0x7f4c14004c80 comp rx=0 tx=0).stop 2024-09-19T09:03:30.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.235+0000 7f4c2e96d740 1 -- 172.21.15.120:0/4076727045 shutdown_connections 2024-09-19T09:03:30.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.235+0000 7f4c2e96d740 1 --2- 172.21.15.120:0/4076727045 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5599bfaa71c0 0x5599bfaa95b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.235+0000 7f4c2e96d740 1 --2- 172.21.15.120:0/4076727045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599bfa97780 0x5599bfa94980 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.235+0000 7f4c2e96d740 1 --2- 172.21.15.120:0/4076727045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599bfa8df50 0x5599bfa943e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.235+0000 7f4c0dffb640 0 client.15564 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:03:30.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.235+0000 7f4c2e96d740 1 --2- 172.21.15.120:0/4076727045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599bfa8c9e0 0x5599bfa97130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.235+0000 7f4c2e96d740 1 -- 172.21.15.120:0/4076727045 >> 172.21.15.120:0/4076727045 conn(0x5599bfa51310 msgr2=0x5599bfa8ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:30.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.235+0000 7f4c2e96d740 1 -- 172.21.15.120:0/4076727045 shutdown_connections 2024-09-19T09:03:30.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.235+0000 7f4c2e96d740 1 -- 172.21.15.120:0/4076727045 wait complete. 2024-09-19T09:03:30.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.235+0000 7f4c2e96d740 20 client.15564 trim_cache size 0 max 16384 2024-09-19T09:03:30.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.248+0000 7f4c2e96d740 1 Processor -- start 2024-09-19T09:03:30.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.248+0000 7f4c2e96d740 1 -- start start 2024-09-19T09:03:30.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.248+0000 7f4c2e96d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599bfa8c9e0 0x5599bfaa36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:30.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.248+0000 7f4c2e96d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599bfa8df50 0x5599bfaaed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:30.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.248+0000 7f4c2e96d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599bfa97780 0x5599bfaaf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:30.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.248+0000 7f4c2e96d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4c241ab380 con 0x5599bfa8df50 2024-09-19T09:03:30.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.248+0000 7f4c2e96d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4c241acec0 con 0x5599bfa97780 2024-09-19T09:03:30.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.248+0000 7f4c2e96d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4c241aba50 con 0x5599bfa8c9e0 2024-09-19T09:03:30.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.248+0000 7f4c2a330640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599bfa8df50 0x5599bfaaed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:30.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.248+0000 7f4c2a330640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599bfa8df50 0x5599bfaaed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59120/0 (socket says 172.21.15.120:59120) 2024-09-19T09:03:30.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.248+0000 7f4c2a330640 1 -- 172.21.15.120:0/3972467582 learned_addr learned my addr 172.21.15.120:0/3972467582 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:30.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.248+0000 7f4c29b2f640 1 --2- 172.21.15.120:0/3972467582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599bfa8c9e0 0x5599bfaa36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:30.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.248+0000 7f4c29b2f640 1 --2- 172.21.15.120:0/3972467582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599bfa97780 0x5599bfaaf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:30.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.248+0000 7f4c2a330640 1 -- 172.21.15.120:0/3972467582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599bfa8c9e0 msgr2=0x5599bfaa36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:30.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.248+0000 7f4c2a330640 1 --2- 172.21.15.120:0/3972467582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599bfa8c9e0 0x5599bfaa36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.248+0000 7f4c2a330640 1 -- 172.21.15.120:0/3972467582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599bfa97780 msgr2=0x5599bfaaf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:30.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.248+0000 7f4c2a330640 1 --2- 172.21.15.120:0/3972467582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599bfa97780 0x5599bfaaf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.248+0000 7f4c2a330640 1 -- 172.21.15.120:0/3972467582 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599bfa8d710 con 0x5599bfa8df50 2024-09-19T09:03:30.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.248+0000 7f4c29b2f640 1 --2- 172.21.15.120:0/3972467582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599bfa97780 0x5599bfaaf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:03:30.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.249+0000 7f4c2a330640 1 --2- 172.21.15.120:0/3972467582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599bfa8df50 0x5599bfaaed40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x5599bfa93c70 tx=0x7f4c2000de30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49cc4b34dfb2ac8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:30.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.249+0000 7f4c0ffff640 1 -- 172.21.15.120:0/3972467582 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c20003da0 con 0x5599bfa8df50 2024-09-19T09:03:30.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.249+0000 7f4c0ffff640 1 -- 172.21.15.120:0/3972467582 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c20005500 con 0x5599bfa8df50 2024-09-19T09:03:30.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.249+0000 7f4c0ffff640 1 -- 172.21.15.120:0/3972467582 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2001d6d0 con 0x5599bfa8df50 2024-09-19T09:03:30.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.249+0000 7f4c2e96d740 1 -- 172.21.15.120:0/3972467582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599bfa8df50 msgr2=0x5599bfaaed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:30.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.250+0000 7f4c2e96d740 1 --2- 172.21.15.120:0/3972467582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599bfa8df50 0x5599bfaaed40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x5599bfa93c70 tx=0x7f4c2000de30 comp rx=0 tx=0).stop 2024-09-19T09:03:30.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.250+0000 7f4c2e96d740 1 -- 172.21.15.120:0/3972467582 shutdown_connections 2024-09-19T09:03:30.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.250+0000 7f4c2e96d740 1 --2- 172.21.15.120:0/3972467582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599bfa97780 0x5599bfaaf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.250+0000 7f4c2e96d740 1 --2- 172.21.15.120:0/3972467582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599bfa8df50 0x5599bfaaed40 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.250+0000 7f4c2e96d740 1 --2- 172.21.15.120:0/3972467582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599bfa8c9e0 0x5599bfaa36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.250+0000 7f4c2e96d740 1 -- 172.21.15.120:0/3972467582 >> 172.21.15.120:0/3972467582 conn(0x5599bfa51310 msgr2=0x5599bfa96460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:30.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.250+0000 7f4c2e96d740 1 -- 172.21.15.120:0/3972467582 shutdown_connections 2024-09-19T09:03:30.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.250+0000 7f4c2e96d740 1 -- 172.21.15.120:0/3972467582 wait complete. 2024-09-19T09:03:30.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.250+0000 7f4c2e96d740 1 -- start start 2024-09-19T09:03:30.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.250+0000 7f4c2e96d740 1 Processor -- start 2024-09-19T09:03:30.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.250+0000 7f4c2e96d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599bfa8c9e0 0x5599bfa9b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:30.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.250+0000 7f4c2e96d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599bfa8df50 0x5599bfa9bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:30.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.250+0000 7f4c2e96d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599bfa97780 0x5599bfa9c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:30.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.250+0000 7f4c2e96d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4c241ac7f0 con 0x5599bfa8c9e0 2024-09-19T09:03:30.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.250+0000 7f4c2e96d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4c241ac120 con 0x5599bfa8df50 2024-09-19T09:03:30.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.250+0000 7f4c2e96d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4c241a9340 con 0x5599bfa97780 2024-09-19T09:03:30.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.250+0000 7f4c2e96d740 10 client.0 _ll_register_callbacks cb 0x7f4c2000d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:30.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.250+0000 7f4c29b2f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599bfa8c9e0 0x5599bfa9b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:30.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.250+0000 7f4c29b2f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599bfa8c9e0 0x5599bfa9b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59136/0 (socket says 172.21.15.120:59136) 2024-09-19T09:03:30.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.250+0000 7f4c29b2f640 1 -- 172.21.15.120:0/2398201130 learned_addr learned my addr 172.21.15.120:0/2398201130 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:30.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.251+0000 7f4c29b2f640 1 -- 172.21.15.120:0/2398201130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599bfa97780 msgr2=0x5599bfa9c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:30.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.251+0000 7f4c29b2f640 1 --2- 172.21.15.120:0/2398201130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599bfa97780 0x5599bfa9c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.251+0000 7f4c29b2f640 1 -- 172.21.15.120:0/2398201130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599bfa8df50 msgr2=0x5599bfa9bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:30.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.251+0000 7f4c29b2f640 1 --2- 172.21.15.120:0/2398201130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599bfa8df50 0x5599bfa9bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.251+0000 7f4c29b2f640 1 -- 172.21.15.120:0/2398201130 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599bfb10f10 con 0x5599bfa8c9e0 2024-09-19T09:03:30.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.251+0000 7f4c29b2f640 1 --2- 172.21.15.120:0/2398201130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599bfa8c9e0 0x5599bfa9b7a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4c14005910 tx=0x7f4c1400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e37eaf9596c13b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:30.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.251+0000 7f4c0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:30.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.251+0000 7f4c0dffb640 1 -- 172.21.15.120:0/2398201130 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c14036070 con 0x5599bfa8c9e0 2024-09-19T09:03:30.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.251+0000 7f4c0dffb640 1 -- 172.21.15.120:0/2398201130 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f4c14030650 con 0x5599bfa8c9e0 2024-09-19T09:03:30.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.251+0000 7f4c2e96d740 1 -- 172.21.15.120:0/2398201130 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c20005500 con 0x5599bfa8c9e0 2024-09-19T09:03:30.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.251+0000 7f4c0dffb640 1 -- 172.21.15.120:0/2398201130 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4c1400b950 con 0x5599bfa8c9e0 2024-09-19T09:03:30.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.251+0000 7f4c2e96d740 10 client.15570 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:30.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.251+0000 7f4c2e96d740 1 -- 172.21.15.120:0/2398201130 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5599bfb12120 con 0x5599bfa8c9e0 2024-09-19T09:03:30.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.252+0000 7f4c0dffb640 1 -- 172.21.15.120:0/2398201130 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c20005500 con 0x5599bfa8c9e0 2024-09-19T09:03:30.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.252+0000 7f4c0dffb640 1 -- 172.21.15.120:0/2398201130 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4c140051d0 con 0x5599bfa8c9e0 2024-09-19T09:03:30.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.252+0000 7f4c2e96d740 10 client.15570 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:30.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.252+0000 7f4c2e96d740 10 client.15570 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:03:30.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.252+0000 7f4c2e96d740 20 client.15570 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:30.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.252+0000 7f4c2e96d740 1 --2- 172.21.15.120:0/2398201130 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5599bfaa71c0 0x5599bfb148d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:30.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.252+0000 7f4c2e96d740 4 client.15570 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:03:30.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.252+0000 7f4c2e96d740 1 -- 172.21.15.120:0/2398201130 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5599bf9f5d70 con 0x5599bfaa71c0 2024-09-19T09:03:30.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.252+0000 7f4c2a330640 1 --2- 172.21.15.120:0/2398201130 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5599bfaa71c0 0x5599bfb148d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:30.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.253+0000 7f4c0dffb640 10 client.15570 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:03:30.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.253+0000 7f4c2a330640 1 --2- 172.21.15.120:0/2398201130 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5599bfaa71c0 0x5599bfb148d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d8407d0d82a57cb6 server_cookie=e5137560acb40c81 in_seq=0 out_seq=0 2024-09-19T09:03:30.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.253+0000 7f4c0dffb640 1 -- 172.21.15.120:0/2398201130 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5599bf9f5d70 con 0x5599bfaa71c0 2024-09-19T09:03:30.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.253+0000 7f4c0dffb640 10 client.15570 handle_command_reply: tid=0 2024-09-19T09:03:30.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.253+0000 7f4c2e96d740 1 client.15570 shutdown 2024-09-19T09:03:30.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.253+0000 7f4c2e96d740 1 -- 172.21.15.120:0/2398201130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599bfa8c9e0 msgr2=0x5599bfa9b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:30.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.253+0000 7f4c2e96d740 1 --2- 172.21.15.120:0/2398201130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599bfa8c9e0 0x5599bfa9b7a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4c14005910 tx=0x7f4c1400b810 comp rx=0 tx=0).stop 2024-09-19T09:03:30.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.253+0000 7f4c2e96d740 1 -- 172.21.15.120:0/2398201130 shutdown_connections 2024-09-19T09:03:30.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.253+0000 7f4c2e96d740 1 --2- 172.21.15.120:0/2398201130 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5599bfaa71c0 0x5599bfb148d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.253+0000 7f4c2e96d740 1 --2- 172.21.15.120:0/2398201130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5599bfa97780 0x5599bfa9c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.253+0000 7f4c2e96d740 1 --2- 172.21.15.120:0/2398201130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5599bfa8df50 0x5599bfa9bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.253+0000 7f4c0dffb640 0 client.15570 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:03:30.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.253+0000 7f4c2e96d740 1 --2- 172.21.15.120:0/2398201130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5599bfa8c9e0 0x5599bfa9b7a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.253+0000 7f4c2e96d740 1 -- 172.21.15.120:0/2398201130 >> 172.21.15.120:0/2398201130 conn(0x5599bfa51310 msgr2=0x5599bfa834e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:30.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.253+0000 7f4c2e96d740 1 -- 172.21.15.120:0/2398201130 shutdown_connections 2024-09-19T09:03:30.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.253+0000 7f4c2e96d740 1 -- 172.21.15.120:0/2398201130 wait complete. 2024-09-19T09:03:30.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.253+0000 7f4c2e96d740 20 client.15570 trim_cache size 0 max 16384 2024-09-19T09:03:30.270 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:03:30.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.254+0000 7f4c2c594640 1 -- 172.21.15.120:0/2122498974 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4c100a3350 msgr2=0x7f4c100a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:30.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.254+0000 7f4c2c594640 1 --2- 172.21.15.120:0/2122498974 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4c100a3350 0x7f4c100a5810 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c0bd3b0 tx=0x7f4c1c037c40 comp rx=0 tx=0).stop 2024-09-19T09:03:30.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.254+0000 7f4c2c594640 1 -- 172.21.15.120:0/2122498974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c24106f20 msgr2=0x7f4c2419cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:30.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.254+0000 7f4c2c594640 1 --2- 172.21.15.120:0/2122498974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c24106f20 0x7f4c2419cc90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c031a20 tx=0x7f4c1c031a50 comp rx=0 tx=0).stop 2024-09-19T09:03:30.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.254+0000 7f4c2c594640 1 -- 172.21.15.120:0/2122498974 shutdown_connections 2024-09-19T09:03:30.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.254+0000 7f4c2c594640 1 --2- 172.21.15.120:0/2122498974 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4c100a3350 0x7f4c100a5810 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.254+0000 7f4c2c594640 1 --2- 172.21.15.120:0/2122498974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c24106f20 0x7f4c2419cc90 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.254+0000 7f4c2c594640 1 --2- 172.21.15.120:0/2122498974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c241027a0 0x7f4c24198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.254+0000 7f4c2c594640 1 --2- 172.21.15.120:0/2122498974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c24101df0 0x7f4c241983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:30.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.254+0000 7f4c2c594640 1 -- 172.21.15.120:0/2122498974 >> 172.21.15.120:0/2122498974 conn(0x7f4c240fd8f0 msgr2=0x7f4c240fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:30.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.254+0000 7f4c2c594640 1 -- 172.21.15.120:0/2122498974 shutdown_connections 2024-09-19T09:03:30.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:30.254+0000 7f4c2c594640 1 -- 172.21.15.120:0/2122498974 wait complete. 2024-09-19T09:03:31.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:30 smithi120 ceph-mon[31150]: pgmap v360: 97 pgs: 97 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-19T09:03:31.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:30 smithi196 ceph-mon[35150]: pgmap v360: 97 pgs: 97 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-19T09:03:31.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:30 smithi153 ceph-mon[35071]: pgmap v360: 97 pgs: 97 active+clean; 664 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-19T09:03:32.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:31 smithi196 ceph-mon[35150]: pgmap v361: 97 pgs: 97 active+clean; 665 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-19T09:03:32.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:31 smithi153 ceph-mon[35071]: pgmap v361: 97 pgs: 97 active+clean; 665 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-19T09:03:32.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:31 smithi120 ceph-mon[31150]: pgmap v361: 97 pgs: 97 active+clean; 665 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-19T09:03:32.885 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:03:35.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:34 smithi120 ceph-mon[31150]: pgmap v362: 97 pgs: 97 active+clean; 665 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T09:03:35.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:34 smithi196 ceph-mon[35150]: pgmap v362: 97 pgs: 97 active+clean; 665 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T09:03:35.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:34 smithi153 ceph-mon[35071]: pgmap v362: 97 pgs: 97 active+clean; 665 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T09:03:36.107 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db / --timeout 90 --expiration 131.2 --await-for 45.0' 2024-09-19T09:03:36.107 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db / --timeout 90 --expiration 131.2 --await-for 45.0 2024-09-19T09:03:36.243 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:03:36.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.827+0000 7fcc7ed9f640 1 -- 172.21.15.120:0/647145366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc781008d0 msgr2=0x7fcc78100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:36.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.827+0000 7fcc7ed9f640 1 --2- 172.21.15.120:0/647145366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc781008d0 0x7fcc78100d30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fcc68009930 tx=0x7fcc6802a420 comp rx=0 tx=0).stop 2024-09-19T09:03:36.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.827+0000 7fcc7ed9f640 1 -- 172.21.15.120:0/647145366 shutdown_connections 2024-09-19T09:03:36.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.827+0000 7fcc7ed9f640 1 --2- 172.21.15.120:0/647145366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc78101270 0x7fcc7810d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.827+0000 7fcc7ed9f640 1 --2- 172.21.15.120:0/647145366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc781008d0 0x7fcc78100d30 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.827+0000 7fcc7ed9f640 1 --2- 172.21.15.120:0/647145366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc781068d0 0x7fcc78106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.827+0000 7fcc7ed9f640 1 -- 172.21.15.120:0/647145366 >> 172.21.15.120:0/647145366 conn(0x7fcc780fc610 msgr2=0x7fcc780fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:36.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.828+0000 7fcc7ed9f640 1 -- 172.21.15.120:0/647145366 shutdown_connections 2024-09-19T09:03:36.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.828+0000 7fcc7ed9f640 1 -- 172.21.15.120:0/647145366 wait complete. 2024-09-19T09:03:36.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.828+0000 7fcc7ed9f640 1 Processor -- start 2024-09-19T09:03:36.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.829+0000 7fcc7ed9f640 1 -- start start 2024-09-19T09:03:36.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.829+0000 7fcc7ed9f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc781008d0 0x7fcc781982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:36.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.829+0000 7fcc7ed9f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc78101270 0x7fcc78198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:36.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.829+0000 7fcc7ed9f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc781068d0 0x7fcc7819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:36.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.829+0000 7fcc7d59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc78101270 0x7fcc78198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:36.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.829+0000 7fcc7d59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc78101270 0x7fcc78198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41748/0 (socket says 172.21.15.120:41748) 2024-09-19T09:03:36.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.829+0000 7fcc7ed9f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcc78110d20 con 0x7fcc78101270 2024-09-19T09:03:36.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.829+0000 7fcc7d59c640 1 -- 172.21.15.120:0/965020887 learned_addr learned my addr 172.21.15.120:0/965020887 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:36.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.829+0000 7fcc7ed9f640 1 -- 172.21.15.120:0/965020887 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcc78110ba0 con 0x7fcc781008d0 2024-09-19T09:03:36.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.829+0000 7fcc7ed9f640 1 -- 172.21.15.120:0/965020887 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcc78110ea0 con 0x7fcc781068d0 2024-09-19T09:03:36.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.829+0000 7fcc7dd9d640 1 --2- 172.21.15.120:0/965020887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc781008d0 0x7fcc781982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:36.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.829+0000 7fcc7e59e640 1 --2- 172.21.15.120:0/965020887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc781068d0 0x7fcc7819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:36.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.830+0000 7fcc7dd9d640 1 -- 172.21.15.120:0/965020887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc781068d0 msgr2=0x7fcc7819cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:36.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.830+0000 7fcc7dd9d640 1 --2- 172.21.15.120:0/965020887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc781068d0 0x7fcc7819cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.830+0000 7fcc7dd9d640 1 -- 172.21.15.120:0/965020887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc78101270 msgr2=0x7fcc78198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:36.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.830+0000 7fcc7dd9d640 1 --2- 172.21.15.120:0/965020887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc78101270 0x7fcc78198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.830+0000 7fcc7dd9d640 1 -- 172.21.15.120:0/965020887 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7819d290 con 0x7fcc781008d0 2024-09-19T09:03:36.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.830+0000 7fcc7dd9d640 1 --2- 172.21.15.120:0/965020887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc781008d0 0x7fcc781982e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcc74009e60 tx=0x7fcc7400edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:36.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.830+0000 7fcc6effd640 1 -- 172.21.15.120:0/965020887 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc74020070 con 0x7fcc781008d0 2024-09-19T09:03:36.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.830+0000 7fcc6effd640 1 -- 172.21.15.120:0/965020887 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc7400ccc0 con 0x7fcc781008d0 2024-09-19T09:03:36.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.830+0000 7fcc7ed9f640 1 -- 172.21.15.120:0/965020887 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc7819d580 con 0x7fcc781008d0 2024-09-19T09:03:36.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.830+0000 7fcc6effd640 1 -- 172.21.15.120:0/965020887 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7401bb40 con 0x7fcc781008d0 2024-09-19T09:03:36.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.830+0000 7fcc7ed9f640 1 -- 172.21.15.120:0/965020887 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc780fcbc0 con 0x7fcc781008d0 2024-09-19T09:03:36.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.833+0000 7fcc6effd640 1 -- 172.21.15.120:0/965020887 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7fcc74023070 con 0x7fcc781008d0 2024-09-19T09:03:36.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.834+0000 7fcc823da740 1 Processor -- start 2024-09-19T09:03:36.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.834+0000 7fcc6effd640 1 --2- 172.21.15.120:0/965020887 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcc640a32d0 0x7fcc640a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:36.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.835+0000 7fcc6effd640 1 -- 172.21.15.120:0/965020887 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fcc740d3a00 con 0x7fcc781008d0 2024-09-19T09:03:36.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.835+0000 7fcc823da740 1 -- start start 2024-09-19T09:03:36.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.835+0000 7fcc7e59e640 1 --2- 172.21.15.120:0/965020887 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcc640a32d0 0x7fcc640a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:36.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.835+0000 7fcc823da740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651975afd20 0x5651975ae270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:36.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.835+0000 7fcc823da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651975ae7b0 0x5651975aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:36.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.835+0000 7fcc823da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651975b9550 0x5651975bb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:36.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.835+0000 7fcc823da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5651975142f0 con 0x5651975b9550 2024-09-19T09:03:36.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.835+0000 7fcc823da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5651974bbcf0 con 0x5651975ae7b0 2024-09-19T09:03:36.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.835+0000 7fcc823da740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5651975102b0 con 0x5651975afd20 2024-09-19T09:03:36.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.835+0000 7fcc7d59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651975ae7b0 0x5651975aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:36.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.835+0000 7fcc7d59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651975ae7b0 0x5651975aebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:50010/0 (socket says 172.21.15.120:50010) 2024-09-19T09:03:36.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.835+0000 7fcc7d59c640 1 -- 172.21.15.120:0/3792621102 learned_addr learned my addr 172.21.15.120:0/3792621102 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:36.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.836+0000 7fcc7d59c640 1 -- 172.21.15.120:0/3792621102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651975afd20 msgr2=0x5651975ae270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:03:36.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.836+0000 7fcc7d59c640 1 --2- 172.21.15.120:0/3792621102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651975afd20 0x5651975ae270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.836+0000 7fcc7d59c640 1 -- 172.21.15.120:0/3792621102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651975b9550 msgr2=0x5651975bb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:36.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.836+0000 7fcc7d59c640 1 --2- 172.21.15.120:0/3792621102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651975b9550 0x5651975bb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.836+0000 7fcc7d59c640 1 -- 172.21.15.120:0/3792621102 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651974a03a0 con 0x5651975ae7b0 2024-09-19T09:03:36.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.836+0000 7fcc7e59e640 1 --2- 172.21.15.120:0/965020887 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcc640a32d0 0x7fcc640a5790 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcc70005fd0 tx=0x7fcc70009af0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:36.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.836+0000 7fcc7d59c640 1 --2- 172.21.15.120:0/3792621102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651975ae7b0 0x5651975aebf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcc68005e90 tx=0x7fcc68004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7d8ccd3757e28ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:36.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.837+0000 7fcc63fff640 1 -- 172.21.15.120:0/3792621102 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc68035040 con 0x5651975ae7b0 2024-09-19T09:03:36.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.837+0000 7fcc63fff640 1 -- 172.21.15.120:0/3792621102 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc68005520 con 0x5651975ae7b0 2024-09-19T09:03:36.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.837+0000 7fcc63fff640 1 -- 172.21.15.120:0/3792621102 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc680035f0 con 0x5651975ae7b0 2024-09-19T09:03:36.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.838+0000 7fcc823da740 1 -- 172.21.15.120:0/3792621102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651975ae7b0 msgr2=0x5651975aebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:36.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.838+0000 7fcc823da740 1 --2- 172.21.15.120:0/3792621102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651975ae7b0 0x5651975aebf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcc68005e90 tx=0x7fcc68004040 comp rx=0 tx=0).stop 2024-09-19T09:03:36.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.838+0000 7fcc823da740 1 -- 172.21.15.120:0/3792621102 shutdown_connections 2024-09-19T09:03:36.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.838+0000 7fcc823da740 1 --2- 172.21.15.120:0/3792621102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651975b9550 0x5651975bb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.838+0000 7fcc823da740 1 --2- 172.21.15.120:0/3792621102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651975ae7b0 0x5651975aebf0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.838+0000 7fcc823da740 1 --2- 172.21.15.120:0/3792621102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651975afd20 0x5651975ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.838+0000 7fcc823da740 1 -- 172.21.15.120:0/3792621102 >> 172.21.15.120:0/3792621102 conn(0x565197577310 msgr2=0x5651975aae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:36.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.838+0000 7fcc823da740 1 -- 172.21.15.120:0/3792621102 shutdown_connections 2024-09-19T09:03:36.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.838+0000 7fcc823da740 1 -- 172.21.15.120:0/3792621102 wait complete. 2024-09-19T09:03:36.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.838+0000 7fcc823da740 1 -- start start 2024-09-19T09:03:36.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.839+0000 7fcc823da740 1 Processor -- start 2024-09-19T09:03:36.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.839+0000 7fcc823da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651975ae7b0 0x5651975b8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:36.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.839+0000 7fcc823da740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651975afd20 0x5651975b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:36.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.839+0000 7fcc823da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651975b9550 0x5651975b6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:36.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.840+0000 7fcc7dd9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651975ae7b0 0x5651975b8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:36.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.840+0000 7fcc7dd9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651975ae7b0 0x5651975b8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41754/0 (socket says 172.21.15.120:41754) 2024-09-19T09:03:36.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.840+0000 7fcc7dd9d640 1 -- 172.21.15.120:0/1445531620 learned_addr learned my addr 172.21.15.120:0/1445531620 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:36.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.840+0000 7fcc823da740 1 -- 172.21.15.120:0/1445531620 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5651975bc1f0 con 0x5651975ae7b0 2024-09-19T09:03:36.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.840+0000 7fcc823da740 1 -- 172.21.15.120:0/1445531620 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5651974cf060 con 0x5651975b9550 2024-09-19T09:03:36.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.840+0000 7fcc823da740 1 -- 172.21.15.120:0/1445531620 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5651975bc370 con 0x5651975afd20 2024-09-19T09:03:36.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.840+0000 7fcc7e59e640 1 --2- 172.21.15.120:0/1445531620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651975b9550 0x5651975b6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:36.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.840+0000 7fcc7d59c640 1 --2- 172.21.15.120:0/1445531620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651975afd20 0x5651975b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:36.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.840+0000 7fcc823da740 10 client.0 _ll_register_callbacks cb 0x5651972228c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:36.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.840+0000 7fcc7e59e640 1 -- 172.21.15.120:0/1445531620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651975afd20 msgr2=0x5651975b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:36.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.840+0000 7fcc7e59e640 1 --2- 172.21.15.120:0/1445531620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651975afd20 0x5651975b62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.840+0000 7fcc7e59e640 1 -- 172.21.15.120:0/1445531620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651975ae7b0 msgr2=0x5651975b8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:36.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.840+0000 7fcc7e59e640 1 --2- 172.21.15.120:0/1445531620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651975ae7b0 0x5651975b8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.840+0000 7fcc7e59e640 1 -- 172.21.15.120:0/1445531620 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651972853c0 con 0x5651975b9550 2024-09-19T09:03:36.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.840+0000 7fcc7dd9d640 1 --2- 172.21.15.120:0/1445531620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651975ae7b0 0x5651975b8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:03:36.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.841+0000 7fcc823da740 1 -- 172.21.15.120:0/1445531620 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc68005520 con 0x5651975b9550 2024-09-19T09:03:36.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.841+0000 7fcc7e59e640 1 --2- 172.21.15.120:0/1445531620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651975b9550 0x5651975b6a30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcc70009990 tx=0x7fcc70004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbbc2dd0cfdaa34c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:36.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.841+0000 7fcc617fa640 10 client.25031 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:03:36.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.841+0000 7fcc617fa640 1 -- 172.21.15.120:0/1445531620 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7002d050 con 0x5651975b9550 2024-09-19T09:03:36.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.841+0000 7fcc617fa640 1 -- 172.21.15.120:0/1445531620 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fcc7000f890 con 0x5651975b9550 2024-09-19T09:03:36.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.841+0000 7fcc617fa640 1 -- 172.21.15.120:0/1445531620 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcc70023070 con 0x5651975b9550 2024-09-19T09:03:36.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.841+0000 7fcc823da740 10 client.25031 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:36.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.841+0000 7fcc823da740 1 -- 172.21.15.120:0/1445531620 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({fsmap=28}) -- 0x5651975c7ca0 con 0x5651975b9550 2024-09-19T09:03:36.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.842+0000 7fcc617fa640 1 -- 172.21.15.120:0/1445531620 <== mon.2 v2:172.21.15.153:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcc70030020 con 0x5651975b9550 2024-09-19T09:03:36.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.842+0000 7fcc823da740 10 client.25031 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:36.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.842+0000 7fcc823da740 10 client.25031 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:03:36.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.842+0000 7fcc823da740 20 client.25031 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:36.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.842+0000 7fcc823da740 1 --2- 172.21.15.120:0/1445531620 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5651975c90a0 0x5651975cb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:36.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.842+0000 7fcc823da740 4 client.25031 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:03:36.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.842+0000 7fcc823da740 1 -- 172.21.15.120:0/1445531620 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565197521860 con 0x5651975c90a0 2024-09-19T09:03:36.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.842+0000 7fcc7dd9d640 1 --2- 172.21.15.120:0/1445531620 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5651975c90a0 0x5651975cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:36.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.843+0000 7fcc617fa640 10 client.25031 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:03:36.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.843+0000 7fcc7dd9d640 1 --2- 172.21.15.120:0/1445531620 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5651975c90a0 0x5651975cb490 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=66f33dd03af2fff8 server_cookie=1858a959e84a5ca4 in_seq=0 out_seq=0 2024-09-19T09:03:36.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.844+0000 7fcc617fa640 1 -- 172.21.15.120:0/1445531620 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565197521860 con 0x5651975c90a0 2024-09-19T09:03:36.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.844+0000 7fcc617fa640 10 client.25031 handle_command_reply: tid=0 2024-09-19T09:03:36.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.844+0000 7fcc823da740 1 client.25031 shutdown 2024-09-19T09:03:36.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.844+0000 7fcc823da740 1 -- 172.21.15.120:0/1445531620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651975b9550 msgr2=0x5651975b6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:36.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.844+0000 7fcc823da740 1 --2- 172.21.15.120:0/1445531620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651975b9550 0x5651975b6a30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcc70009990 tx=0x7fcc70004ab0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.844+0000 7fcc823da740 1 -- 172.21.15.120:0/1445531620 shutdown_connections 2024-09-19T09:03:36.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.844+0000 7fcc823da740 1 --2- 172.21.15.120:0/1445531620 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5651975c90a0 0x5651975cb490 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.844+0000 7fcc617fa640 0 client.25031 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:03:36.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.844+0000 7fcc823da740 1 --2- 172.21.15.120:0/1445531620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651975b9550 0x5651975b6a30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.844+0000 7fcc823da740 1 --2- 172.21.15.120:0/1445531620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651975afd20 0x5651975b62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.844+0000 7fcc823da740 1 --2- 172.21.15.120:0/1445531620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651975ae7b0 0x5651975b8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.844+0000 7fcc823da740 1 -- 172.21.15.120:0/1445531620 >> 172.21.15.120:0/1445531620 conn(0x565197577310 msgr2=0x5651975b0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:36.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.844+0000 7fcc823da740 1 -- 172.21.15.120:0/1445531620 shutdown_connections 2024-09-19T09:03:36.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.844+0000 7fcc823da740 1 -- 172.21.15.120:0/1445531620 wait complete. 2024-09-19T09:03:36.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.844+0000 7fcc823da740 20 client.25031 trim_cache size 0 max 16384 2024-09-19T09:03:36.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.869+0000 7fcc823da740 1 Processor -- start 2024-09-19T09:03:36.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.869+0000 7fcc823da740 1 -- start start 2024-09-19T09:03:36.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.869+0000 7fcc823da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651975ae7b0 0x5651975aa790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:36.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.869+0000 7fcc823da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651975afd20 0x5651975aacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:36.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.869+0000 7fcc823da740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651975b9550 0x5651975d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:36.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.869+0000 7fcc823da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcc781ab1d0 con 0x5651975ae7b0 2024-09-19T09:03:36.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.869+0000 7fcc823da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcc781acd10 con 0x5651975afd20 2024-09-19T09:03:36.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.869+0000 7fcc823da740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcc781ab8a0 con 0x5651975b9550 2024-09-19T09:03:36.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.869+0000 7fcc7dd9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651975ae7b0 0x5651975aa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:36.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.869+0000 7fcc7dd9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651975ae7b0 0x5651975aa790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41758/0 (socket says 172.21.15.120:41758) 2024-09-19T09:03:36.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.869+0000 7fcc7dd9d640 1 -- 172.21.15.120:0/4117059262 learned_addr learned my addr 172.21.15.120:0/4117059262 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:36.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.869+0000 7fcc7d59c640 1 --2- 172.21.15.120:0/4117059262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651975afd20 0x5651975aacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:36.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.869+0000 7fcc7e59e640 1 --2- 172.21.15.120:0/4117059262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651975b9550 0x5651975d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:36.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.870+0000 7fcc7dd9d640 1 -- 172.21.15.120:0/4117059262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651975b9550 msgr2=0x5651975d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:36.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.870+0000 7fcc7dd9d640 1 --2- 172.21.15.120:0/4117059262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651975b9550 0x5651975d0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.870+0000 7fcc7dd9d640 1 -- 172.21.15.120:0/4117059262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651975afd20 msgr2=0x5651975aacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:36.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.870+0000 7fcc7dd9d640 1 --2- 172.21.15.120:0/4117059262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651975afd20 0x5651975aacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.870+0000 7fcc7dd9d640 1 -- 172.21.15.120:0/4117059262 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651975cb800 con 0x5651975ae7b0 2024-09-19T09:03:36.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.870+0000 7fcc7dd9d640 1 --2- 172.21.15.120:0/4117059262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651975ae7b0 0x5651975aa790 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fcc7400c6a0 tx=0x7fcc7401b440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a027465364597e75 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:36.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.870+0000 7fcc637fe640 1 -- 172.21.15.120:0/4117059262 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc740a4340 con 0x5651975ae7b0 2024-09-19T09:03:36.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.870+0000 7fcc637fe640 1 -- 172.21.15.120:0/4117059262 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc740a8c00 con 0x5651975ae7b0 2024-09-19T09:03:36.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.870+0000 7fcc637fe640 1 -- 172.21.15.120:0/4117059262 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc740a8770 con 0x5651975ae7b0 2024-09-19T09:03:36.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.871+0000 7fcc823da740 1 -- 172.21.15.120:0/4117059262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651975ae7b0 msgr2=0x5651975aa790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:36.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.871+0000 7fcc823da740 1 --2- 172.21.15.120:0/4117059262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651975ae7b0 0x5651975aa790 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fcc7400c6a0 tx=0x7fcc7401b440 comp rx=0 tx=0).stop 2024-09-19T09:03:36.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.871+0000 7fcc823da740 1 -- 172.21.15.120:0/4117059262 shutdown_connections 2024-09-19T09:03:36.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.871+0000 7fcc823da740 1 --2- 172.21.15.120:0/4117059262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651975b9550 0x5651975d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.871+0000 7fcc823da740 1 --2- 172.21.15.120:0/4117059262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651975afd20 0x5651975aacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.871+0000 7fcc823da740 1 --2- 172.21.15.120:0/4117059262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651975ae7b0 0x5651975aa790 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.871+0000 7fcc823da740 1 -- 172.21.15.120:0/4117059262 >> 172.21.15.120:0/4117059262 conn(0x565197577310 msgr2=0x5651975b5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:36.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.872+0000 7fcc823da740 1 -- 172.21.15.120:0/4117059262 shutdown_connections 2024-09-19T09:03:36.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.872+0000 7fcc823da740 1 -- 172.21.15.120:0/4117059262 wait complete. 2024-09-19T09:03:36.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.872+0000 7fcc823da740 1 -- start start 2024-09-19T09:03:36.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.872+0000 7fcc823da740 1 Processor -- start 2024-09-19T09:03:36.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.873+0000 7fcc823da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651975ae7b0 0x5651975c1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:36.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.873+0000 7fcc823da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651975afd20 0x5651975bcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:36.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.873+0000 7fcc823da740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651975b9550 0x5651975bd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:36.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.873+0000 7fcc823da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcc781ac640 con 0x5651975ae7b0 2024-09-19T09:03:36.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.873+0000 7fcc823da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcc781abf70 con 0x5651975afd20 2024-09-19T09:03:36.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.873+0000 7fcc823da740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcc781a91b0 con 0x5651975b9550 2024-09-19T09:03:36.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.873+0000 7fcc7dd9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651975ae7b0 0x5651975c1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:36.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.873+0000 7fcc823da740 10 client.0 _ll_register_callbacks cb 0x7fcc700098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:36.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.873+0000 7fcc7dd9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651975ae7b0 0x5651975c1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41762/0 (socket says 172.21.15.120:41762) 2024-09-19T09:03:36.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.873+0000 7fcc7dd9d640 1 -- 172.21.15.120:0/294493950 learned_addr learned my addr 172.21.15.120:0/294493950 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:36.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.873+0000 7fcc7dd9d640 1 -- 172.21.15.120:0/294493950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651975b9550 msgr2=0x5651975bd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:36.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.873+0000 7fcc7dd9d640 1 --2- 172.21.15.120:0/294493950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651975b9550 0x5651975bd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.873+0000 7fcc7dd9d640 1 -- 172.21.15.120:0/294493950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651975afd20 msgr2=0x5651975bcf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:36.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.873+0000 7fcc7dd9d640 1 --2- 172.21.15.120:0/294493950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651975afd20 0x5651975bcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.873+0000 7fcc7dd9d640 1 -- 172.21.15.120:0/294493950 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651975bda60 con 0x5651975ae7b0 2024-09-19T09:03:36.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.874+0000 7fcc7dd9d640 1 --2- 172.21.15.120:0/294493950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651975ae7b0 0x5651975c1b60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcc7409fd40 tx=0x7fcc740c0170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9afccf54ab2ed818 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:36.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.874+0000 7fcc617fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:36.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.874+0000 7fcc617fa640 1 -- 172.21.15.120:0/294493950 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc740d15c0 con 0x5651975ae7b0 2024-09-19T09:03:36.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.874+0000 7fcc617fa640 1 -- 172.21.15.120:0/294493950 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fcc740bebd0 con 0x5651975ae7b0 2024-09-19T09:03:36.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.874+0000 7fcc823da740 1 -- 172.21.15.120:0/294493950 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc740a8c00 con 0x5651975ae7b0 2024-09-19T09:03:36.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.874+0000 7fcc617fa640 1 -- 172.21.15.120:0/294493950 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcc740ae050 con 0x5651975ae7b0 2024-09-19T09:03:36.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.874+0000 7fcc823da740 10 client.15585 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:36.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.874+0000 7fcc823da740 1 -- 172.21.15.120:0/294493950 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565197633ae0 con 0x5651975ae7b0 2024-09-19T09:03:36.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.875+0000 7fcc617fa640 1 -- 172.21.15.120:0/294493950 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc740a8c00 con 0x5651975ae7b0 2024-09-19T09:03:36.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.875+0000 7fcc617fa640 1 -- 172.21.15.120:0/294493950 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcc7401bce0 con 0x5651975ae7b0 2024-09-19T09:03:36.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.875+0000 7fcc823da740 10 client.15585 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:36.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.875+0000 7fcc823da740 10 client.15585 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:03:36.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.875+0000 7fcc823da740 20 client.15585 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:36.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.875+0000 7fcc823da740 1 --2- 172.21.15.120:0/294493950 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x565197634190 0x5651976365c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:36.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.875+0000 7fcc823da740 4 client.15585 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:03:36.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.875+0000 7fcc823da740 1 -- 172.21.15.120:0/294493950 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56519751bd70 con 0x565197634190 2024-09-19T09:03:36.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.876+0000 7fcc7d59c640 1 --2- 172.21.15.120:0/294493950 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x565197634190 0x5651976365c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:36.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.876+0000 7fcc7d59c640 1 --2- 172.21.15.120:0/294493950 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x565197634190 0x5651976365c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bcbd33b56029a1c1 server_cookie=ff60c03dd45b2403 in_seq=0 out_seq=0 2024-09-19T09:03:36.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.876+0000 7fcc617fa640 10 client.15585 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:03:36.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.876+0000 7fcc617fa640 1 -- 172.21.15.120:0/294493950 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56519751bd70 con 0x565197634190 2024-09-19T09:03:36.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.876+0000 7fcc617fa640 10 client.15585 handle_command_reply: tid=0 2024-09-19T09:03:36.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.876+0000 7fcc823da740 1 client.15585 shutdown 2024-09-19T09:03:36.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.877+0000 7fcc823da740 1 -- 172.21.15.120:0/294493950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651975ae7b0 msgr2=0x5651975c1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:36.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.877+0000 7fcc823da740 1 --2- 172.21.15.120:0/294493950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651975ae7b0 0x5651975c1b60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcc7409fd40 tx=0x7fcc740c0170 comp rx=0 tx=0).stop 2024-09-19T09:03:36.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.877+0000 7fcc823da740 1 -- 172.21.15.120:0/294493950 shutdown_connections 2024-09-19T09:03:36.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.877+0000 7fcc823da740 1 --2- 172.21.15.120:0/294493950 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x565197634190 0x5651976365c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.877+0000 7fcc823da740 1 --2- 172.21.15.120:0/294493950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651975b9550 0x5651975bd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.877+0000 7fcc823da740 1 --2- 172.21.15.120:0/294493950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651975afd20 0x5651975bcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.877+0000 7fcc823da740 1 --2- 172.21.15.120:0/294493950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651975ae7b0 0x5651975c1b60 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.877+0000 7fcc823da740 1 -- 172.21.15.120:0/294493950 >> 172.21.15.120:0/294493950 conn(0x565197577310 msgr2=0x5651975b7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:36.880 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:03:36.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.877+0000 7fcc617fa640 0 client.15585 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:03:36.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.877+0000 7fcc823da740 1 -- 172.21.15.120:0/294493950 shutdown_connections 2024-09-19T09:03:36.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.877+0000 7fcc823da740 1 -- 172.21.15.120:0/294493950 wait complete. 2024-09-19T09:03:36.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.877+0000 7fcc823da740 20 client.15585 trim_cache size 0 max 16384 2024-09-19T09:03:36.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.878+0000 7fcc7ed9f640 1 -- 172.21.15.120:0/965020887 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcc640a32d0 msgr2=0x7fcc640a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:36.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.878+0000 7fcc7ed9f640 1 --2- 172.21.15.120:0/965020887 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcc640a32d0 0x7fcc640a5790 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcc70005fd0 tx=0x7fcc70009af0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.878+0000 7fcc7ed9f640 1 -- 172.21.15.120:0/965020887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc781008d0 msgr2=0x7fcc781982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:36.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.878+0000 7fcc7ed9f640 1 --2- 172.21.15.120:0/965020887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc781008d0 0x7fcc781982e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcc74009e60 tx=0x7fcc7400edc0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.878+0000 7fcc7ed9f640 1 -- 172.21.15.120:0/965020887 shutdown_connections 2024-09-19T09:03:36.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.878+0000 7fcc7ed9f640 1 --2- 172.21.15.120:0/965020887 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcc640a32d0 0x7fcc640a5790 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.878+0000 7fcc7ed9f640 1 --2- 172.21.15.120:0/965020887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc781068d0 0x7fcc7819cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.878+0000 7fcc7ed9f640 1 --2- 172.21.15.120:0/965020887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc78101270 0x7fcc78198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.878+0000 7fcc7ed9f640 1 --2- 172.21.15.120:0/965020887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc781008d0 0x7fcc781982e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:36.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.878+0000 7fcc7ed9f640 1 -- 172.21.15.120:0/965020887 >> 172.21.15.120:0/965020887 conn(0x7fcc780fc610 msgr2=0x7fcc7806a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:36.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.878+0000 7fcc7ed9f640 1 -- 172.21.15.120:0/965020887 shutdown_connections 2024-09-19T09:03:36.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:36.878+0000 7fcc7ed9f640 1 -- 172.21.15.120:0/965020887 wait complete. 2024-09-19T09:03:36.882 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:03:37.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:36 smithi120 ceph-mon[31150]: pgmap v363: 97 pgs: 97 active+clean; 665 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T09:03:37.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:36 smithi196 ceph-mon[35150]: pgmap v363: 97 pgs: 97 active+clean; 665 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T09:03:37.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:36 smithi153 ceph-mon[35071]: pgmap v363: 97 pgs: 97 active+clean; 665 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T09:03:37.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.788+0000 7fdae6a7f640 1 -- 172.21.15.120:0/2100189331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae0106f40 msgr2=0x7fdae010eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:37.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.788+0000 7fdae6a7f640 1 --2- 172.21.15.120:0/2100189331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae0106f40 0x7fdae010eac0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fdad8009ca0 tx=0x7fdad802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.788+0000 7fdae6a7f640 1 -- 172.21.15.120:0/2100189331 shutdown_connections 2024-09-19T09:03:37.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.788+0000 7fdae6a7f640 1 --2- 172.21.15.120:0/2100189331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae0106f40 0x7fdae010eac0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.788+0000 7fdae6a7f640 1 --2- 172.21.15.120:0/2100189331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae01027c0 0x7fdae0106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.788+0000 7fdae6a7f640 1 --2- 172.21.15.120:0/2100189331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdae0101e10 0x7fdae01021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.788+0000 7fdae6a7f640 1 -- 172.21.15.120:0/2100189331 >> 172.21.15.120:0/2100189331 conn(0x7fdae00fd8d0 msgr2=0x7fdae00ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:37.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.788+0000 7fdae6a7f640 1 -- 172.21.15.120:0/2100189331 shutdown_connections 2024-09-19T09:03:37.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.788+0000 7fdae6a7f640 1 -- 172.21.15.120:0/2100189331 wait complete. 2024-09-19T09:03:37.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.789+0000 7fdae6a7f640 1 Processor -- start 2024-09-19T09:03:37.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.789+0000 7fdae6a7f640 1 -- start start 2024-09-19T09:03:37.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.789+0000 7fdae6a7f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae0101e10 0x7fdae0198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:37.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.789+0000 7fdae6a7f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae01027c0 0x7fdae01988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:37.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.789+0000 7fdae6a7f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdae0106f40 0x7fdae019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:37.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.789+0000 7fdae6a7f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdae0111ff0 con 0x7fdae01027c0 2024-09-19T09:03:37.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.789+0000 7fdae6a7f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdae0111e70 con 0x7fdae0101e10 2024-09-19T09:03:37.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.789+0000 7fdae6a7f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdae0112170 con 0x7fdae0106f40 2024-09-19T09:03:37.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.790+0000 7fdadffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae01027c0 0x7fdae01988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:37.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.790+0000 7fdadffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae01027c0 0x7fdae01988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41782/0 (socket says 172.21.15.120:41782) 2024-09-19T09:03:37.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.790+0000 7fdadffff640 1 -- 172.21.15.120:0/3981456850 learned_addr learned my addr 172.21.15.120:0/3981456850 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:37.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.790+0000 7fdae501c640 1 --2- 172.21.15.120:0/3981456850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdae0106f40 0x7fdae019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:37.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.790+0000 7fdae481b640 1 --2- 172.21.15.120:0/3981456850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae0101e10 0x7fdae0198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:37.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.790+0000 7fdadffff640 1 -- 172.21.15.120:0/3981456850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdae0106f40 msgr2=0x7fdae019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:37.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.790+0000 7fdadffff640 1 --2- 172.21.15.120:0/3981456850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdae0106f40 0x7fdae019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.790+0000 7fdadffff640 1 -- 172.21.15.120:0/3981456850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae0101e10 msgr2=0x7fdae0198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:37.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.790+0000 7fdadffff640 1 --2- 172.21.15.120:0/3981456850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae0101e10 0x7fdae0198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.790+0000 7fdadffff640 1 -- 172.21.15.120:0/3981456850 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae019d330 con 0x7fdae01027c0 2024-09-19T09:03:37.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.790+0000 7fdadffff640 1 --2- 172.21.15.120:0/3981456850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae01027c0 0x7fdae01988c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00c920 tx=0x7fdacc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:37.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.790+0000 7fdaddffb640 1 -- 172.21.15.120:0/3981456850 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc007c70 con 0x7fdae01027c0 2024-09-19T09:03:37.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.790+0000 7fdaddffb640 1 -- 172.21.15.120:0/3981456850 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdacc007e10 con 0x7fdae01027c0 2024-09-19T09:03:37.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.790+0000 7fdaddffb640 1 -- 172.21.15.120:0/3981456850 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc018890 con 0x7fdae01027c0 2024-09-19T09:03:37.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.790+0000 7fdae6a7f640 1 -- 172.21.15.120:0/3981456850 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdae019d620 con 0x7fdae01027c0 2024-09-19T09:03:37.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.791+0000 7fdae6a7f640 1 -- 172.21.15.120:0/3981456850 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdae0074420 con 0x7fdae01027c0 2024-09-19T09:03:37.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.794+0000 7fdaddffb640 1 -- 172.21.15.120:0/3981456850 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7fdacc018a70 con 0x7fdae01027c0 2024-09-19T09:03:37.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.794+0000 7fdae8e58740 1 Processor -- start 2024-09-19T09:03:37.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.795+0000 7fdaddffb640 1 --2- 172.21.15.120:0/3981456850 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdad00a3420 0x7fdad00a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:37.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.795+0000 7fdae8e58740 1 -- start start 2024-09-19T09:03:37.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.795+0000 7fdaddffb640 1 -- 172.21.15.120:0/3981456850 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fdacc09a080 con 0x7fdae01027c0 2024-09-19T09:03:37.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.795+0000 7fdae8e58740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba757df760 0x55ba757dfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:37.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.795+0000 7fdae501c640 1 --2- 172.21.15.120:0/3981456850 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdad00a3420 0x7fdad00a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:37.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.795+0000 7fdae8e58740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba757e0080 0x55ba757ea400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:37.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.795+0000 7fdae8e58740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba757ea940 0x55ba757ecd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:37.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.795+0000 7fdae8e58740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ba757452a0 con 0x55ba757df760 2024-09-19T09:03:37.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.795+0000 7fdae8e58740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ba756ecb10 con 0x55ba757ea940 2024-09-19T09:03:37.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.795+0000 7fdae8e58740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ba75741260 con 0x55ba757e0080 2024-09-19T09:03:37.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.795+0000 7fdae481b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba757df760 0x55ba757dfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:37.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.795+0000 7fdae481b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba757df760 0x55ba757dfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41796/0 (socket says 172.21.15.120:41796) 2024-09-19T09:03:37.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.795+0000 7fdae481b640 1 -- 172.21.15.120:0/2812084691 learned_addr learned my addr 172.21.15.120:0/2812084691 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:37.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.796+0000 7fdadffff640 1 --2- 172.21.15.120:0/2812084691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba757e0080 0x55ba757ea400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:37.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.796+0000 7fdae501c640 1 --2- 172.21.15.120:0/2812084691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba757ea940 0x55ba757ecd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:37.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.796+0000 7fdae481b640 1 -- 172.21.15.120:0/2812084691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba757e0080 msgr2=0x55ba757ea400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:37.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.796+0000 7fdae481b640 1 --2- 172.21.15.120:0/2812084691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba757e0080 0x55ba757ea400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.796+0000 7fdae481b640 1 -- 172.21.15.120:0/2812084691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba757ea940 msgr2=0x55ba757ecd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:37.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.796+0000 7fdae481b640 1 --2- 172.21.15.120:0/2812084691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba757ea940 0x55ba757ecd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.796+0000 7fdae481b640 1 -- 172.21.15.120:0/2812084691 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba754b6b90 con 0x55ba757df760 2024-09-19T09:03:37.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.796+0000 7fdae481b640 1 --2- 172.21.15.120:0/2812084691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba757df760 0x55ba757dfb40 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fdad4009c10 tx=0x7fdad400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1294dc8e0be4bfc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:37.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.797+0000 7fdae501c640 1 --2- 172.21.15.120:0/3981456850 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdad00a3420 0x7fdad00a58e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdad8006010 tx=0x7fdad80099d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:37.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.797+0000 7fdacaffd640 1 -- 172.21.15.120:0/2812084691 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4016070 con 0x55ba757df760 2024-09-19T09:03:37.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.797+0000 7fdacaffd640 1 -- 172.21.15.120:0/2812084691 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad4005500 con 0x55ba757df760 2024-09-19T09:03:37.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.798+0000 7fdacaffd640 1 -- 172.21.15.120:0/2812084691 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad40116d0 con 0x55ba757df760 2024-09-19T09:03:37.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.798+0000 7fdae8e58740 1 -- 172.21.15.120:0/2812084691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba757df760 msgr2=0x55ba757dfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:37.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.798+0000 7fdae8e58740 1 --2- 172.21.15.120:0/2812084691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba757df760 0x55ba757dfb40 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fdad4009c10 tx=0x7fdad400e6c0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.798+0000 7fdae8e58740 1 -- 172.21.15.120:0/2812084691 shutdown_connections 2024-09-19T09:03:37.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.798+0000 7fdae8e58740 1 --2- 172.21.15.120:0/2812084691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba757ea940 0x55ba757ecd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.798+0000 7fdae8e58740 1 --2- 172.21.15.120:0/2812084691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba757e0080 0x55ba757ea400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.798+0000 7fdae8e58740 1 --2- 172.21.15.120:0/2812084691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba757df760 0x55ba757dfb40 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.798+0000 7fdae8e58740 1 -- 172.21.15.120:0/2812084691 >> 172.21.15.120:0/2812084691 conn(0x55ba757dbcf0 msgr2=0x55ba757dc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:37.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.798+0000 7fdae8e58740 1 -- 172.21.15.120:0/2812084691 shutdown_connections 2024-09-19T09:03:37.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.798+0000 7fdae8e58740 1 -- 172.21.15.120:0/2812084691 wait complete. 2024-09-19T09:03:37.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.798+0000 7fdae8e58740 1 -- start start 2024-09-19T09:03:37.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.798+0000 7fdae8e58740 1 Processor -- start 2024-09-19T09:03:37.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.799+0000 7fdae8e58740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba757e0080 0x55ba757e79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:37.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.799+0000 7fdae8e58740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba757ea940 0x55ba757e7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:37.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.799+0000 7fdae8e58740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba757e8470 0x55ba757ff280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:37.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.799+0000 7fdae8e58740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ba757f5dc0 con 0x55ba757ea940 2024-09-19T09:03:37.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.799+0000 7fdae8e58740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ba756cd6e0 con 0x55ba757e8470 2024-09-19T09:03:37.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.799+0000 7fdae8e58740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ba757f5f40 con 0x55ba757e0080 2024-09-19T09:03:37.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.799+0000 7fdae8e58740 10 client.0 _ll_register_callbacks cb 0x55ba756f1a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:37.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.799+0000 7fdae501c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba757e8470 0x55ba757ff280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:37.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.799+0000 7fdae501c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba757e8470 0x55ba757ff280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:50070/0 (socket says 172.21.15.120:50070) 2024-09-19T09:03:37.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.799+0000 7fdae501c640 1 -- 172.21.15.120:0/3160161375 learned_addr learned my addr 172.21.15.120:0/3160161375 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:37.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.799+0000 7fdadffff640 1 --2- 172.21.15.120:0/3160161375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba757ea940 0x55ba757e7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:37.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.799+0000 7fdae481b640 1 --2- 172.21.15.120:0/3160161375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba757e0080 0x55ba757e79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:37.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.800+0000 7fdae501c640 1 -- 172.21.15.120:0/3160161375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba757e0080 msgr2=0x55ba757e79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:37.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.800+0000 7fdae501c640 1 --2- 172.21.15.120:0/3160161375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba757e0080 0x55ba757e79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.800+0000 7fdae501c640 1 -- 172.21.15.120:0/3160161375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba757ea940 msgr2=0x55ba757e7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:37.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.800+0000 7fdae501c640 1 --2- 172.21.15.120:0/3160161375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba757ea940 0x55ba757e7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.800+0000 7fdae501c640 1 -- 172.21.15.120:0/3160161375 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba757ff8c0 con 0x55ba757e8470 2024-09-19T09:03:37.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.801+0000 7fdae501c640 1 --2- 172.21.15.120:0/3160161375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba757e8470 0x55ba757ff280 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdad8031f80 tx=0x7fdad802a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34d7cf3ab56dd18f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:37.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.801+0000 7fdabffff640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:03:37.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.801+0000 7fdabffff640 1 -- 172.21.15.120:0/3160161375 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad8004440 con 0x55ba757e8470 2024-09-19T09:03:37.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.801+0000 7fdabffff640 1 -- 172.21.15.120:0/3160161375 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fdad80045e0 con 0x55ba757e8470 2024-09-19T09:03:37.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.802+0000 7fdae8e58740 1 -- 172.21.15.120:0/3160161375 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdad4005500 con 0x55ba757e8470 2024-09-19T09:03:37.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.802+0000 7fdabffff640 1 -- 172.21.15.120:0/3160161375 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdad80047e0 con 0x55ba757e8470 2024-09-19T09:03:37.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.802+0000 7fdae8e58740 10 client.25037 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:37.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.802+0000 7fdae8e58740 1 -- 172.21.15.120:0/3160161375 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ba75802230 con 0x55ba757e8470 2024-09-19T09:03:37.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.802+0000 7fdabffff640 1 -- 172.21.15.120:0/3160161375 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad804f070 con 0x55ba757e8470 2024-09-19T09:03:37.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.802+0000 7fdabffff640 1 -- 172.21.15.120:0/3160161375 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdad8049070 con 0x55ba757e8470 2024-09-19T09:03:37.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.802+0000 7fdae8e58740 10 client.25037 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:37.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.802+0000 7fdae8e58740 10 client.25037 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:03:37.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.802+0000 7fdae8e58740 20 client.25037 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:37.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.802+0000 7fdae8e58740 1 --2- 172.21.15.120:0/3160161375 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ba75803620 0x55ba75805a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:37.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.802+0000 7fdae8e58740 4 client.25037 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:03:37.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.802+0000 7fdae8e58740 1 -- 172.21.15.120:0/3160161375 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdad4005500 con 0x55ba75803620 2024-09-19T09:03:37.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.803+0000 7fdae481b640 1 --2- 172.21.15.120:0/3160161375 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ba75803620 0x55ba75805a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:37.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.803+0000 7fdabffff640 10 client.25037 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:37.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.804+0000 7fdae481b640 1 --2- 172.21.15.120:0/3160161375 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ba75803620 0x55ba75805a80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c46821ee0f82aad server_cookie=a2f72bbe96d7d4eb in_seq=0 out_seq=0 2024-09-19T09:03:37.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.804+0000 7fdabffff640 1 -- 172.21.15.120:0/3160161375 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fdad4005500 con 0x55ba75803620 2024-09-19T09:03:37.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.804+0000 7fdabffff640 10 client.25037 handle_command_reply: tid=0 2024-09-19T09:03:37.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.805+0000 7fdae8e58740 1 client.25037 shutdown 2024-09-19T09:03:37.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.805+0000 7fdae8e58740 1 -- 172.21.15.120:0/3160161375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba757e8470 msgr2=0x55ba757ff280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:37.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.805+0000 7fdae8e58740 1 --2- 172.21.15.120:0/3160161375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba757e8470 0x55ba757ff280 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdad8031f80 tx=0x7fdad802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.805+0000 7fdae8e58740 1 -- 172.21.15.120:0/3160161375 shutdown_connections 2024-09-19T09:03:37.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.805+0000 7fdae8e58740 1 --2- 172.21.15.120:0/3160161375 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ba75803620 0x55ba75805a80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.805+0000 7fdae8e58740 1 --2- 172.21.15.120:0/3160161375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba757e8470 0x55ba757ff280 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.805+0000 7fdae8e58740 1 --2- 172.21.15.120:0/3160161375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba757ea940 0x55ba757e7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.805+0000 7fdae8e58740 1 --2- 172.21.15.120:0/3160161375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba757e0080 0x55ba757e79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.805+0000 7fdabffff640 0 client.25037 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:37.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.805+0000 7fdae8e58740 1 -- 172.21.15.120:0/3160161375 >> 172.21.15.120:0/3160161375 conn(0x55ba757dbcf0 msgr2=0x55ba757e19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:37.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.805+0000 7fdae8e58740 1 -- 172.21.15.120:0/3160161375 shutdown_connections 2024-09-19T09:03:37.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.805+0000 7fdae8e58740 1 -- 172.21.15.120:0/3160161375 wait complete. 2024-09-19T09:03:37.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.805+0000 7fdae8e58740 20 client.25037 trim_cache size 0 max 16384 2024-09-19T09:03:37.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.835+0000 7fdae8e58740 1 Processor -- start 2024-09-19T09:03:37.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.835+0000 7fdae8e58740 1 -- start start 2024-09-19T09:03:37.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.835+0000 7fdae8e58740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba757e0080 0x55ba757ff070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:37.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.835+0000 7fdae8e58740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba757e8470 0x55ba7580b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:37.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.835+0000 7fdae8e58740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba757ea940 0x55ba7580b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:37.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.836+0000 7fdae8e58740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdae01ab2d0 con 0x55ba757e0080 2024-09-19T09:03:37.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.836+0000 7fdae8e58740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdae01ace10 con 0x55ba757e8470 2024-09-19T09:03:37.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.836+0000 7fdae8e58740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdae01ab9a0 con 0x55ba757ea940 2024-09-19T09:03:37.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.836+0000 7fdae481b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba757e0080 0x55ba757ff070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:37.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.836+0000 7fdae481b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba757e0080 0x55ba757ff070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41814/0 (socket says 172.21.15.120:41814) 2024-09-19T09:03:37.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.836+0000 7fdae481b640 1 -- 172.21.15.120:0/2983114637 learned_addr learned my addr 172.21.15.120:0/2983114637 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:37.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.836+0000 7fdadffff640 1 --2- 172.21.15.120:0/2983114637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba757e8470 0x55ba7580b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:37.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.836+0000 7fdae501c640 1 --2- 172.21.15.120:0/2983114637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba757ea940 0x55ba7580b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:37.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.836+0000 7fdadffff640 1 -- 172.21.15.120:0/2983114637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba757ea940 msgr2=0x55ba7580b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:37.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.836+0000 7fdadffff640 1 --2- 172.21.15.120:0/2983114637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba757ea940 0x55ba7580b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.836+0000 7fdadffff640 1 -- 172.21.15.120:0/2983114637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba757e0080 msgr2=0x55ba757ff070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:37.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.836+0000 7fdadffff640 1 --2- 172.21.15.120:0/2983114637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba757e0080 0x55ba757ff070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.836+0000 7fdadffff640 1 -- 172.21.15.120:0/2983114637 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba757df550 con 0x55ba757e8470 2024-09-19T09:03:37.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.836+0000 7fdae481b640 1 --2- 172.21.15.120:0/2983114637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba757e0080 0x55ba757ff070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:03:37.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.836+0000 7fdadffff640 1 --2- 172.21.15.120:0/2983114637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba757e8470 0x55ba7580b390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0cd4d0 tx=0x7fdacc0bc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2057a09c30b28b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:37.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.837+0000 7fdaca7fc640 1 -- 172.21.15.120:0/2983114637 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc0a4c50 con 0x55ba757e8470 2024-09-19T09:03:37.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.837+0000 7fdaca7fc640 1 -- 172.21.15.120:0/2983114637 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdacc09a2e0 con 0x55ba757e8470 2024-09-19T09:03:37.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.837+0000 7fdaca7fc640 1 -- 172.21.15.120:0/2983114637 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc0987e0 con 0x55ba757e8470 2024-09-19T09:03:37.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.837+0000 7fdae8e58740 1 -- 172.21.15.120:0/2983114637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba757e8470 msgr2=0x55ba7580b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:37.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.837+0000 7fdae8e58740 1 --2- 172.21.15.120:0/2983114637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba757e8470 0x55ba7580b390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0cd4d0 tx=0x7fdacc0bc040 comp rx=0 tx=0).stop 2024-09-19T09:03:37.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.838+0000 7fdae8e58740 1 -- 172.21.15.120:0/2983114637 shutdown_connections 2024-09-19T09:03:37.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.838+0000 7fdae8e58740 1 --2- 172.21.15.120:0/2983114637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba757ea940 0x55ba7580b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.838+0000 7fdae8e58740 1 --2- 172.21.15.120:0/2983114637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba757e8470 0x55ba7580b390 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.838+0000 7fdae8e58740 1 --2- 172.21.15.120:0/2983114637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba757e0080 0x55ba757ff070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.838+0000 7fdae8e58740 1 -- 172.21.15.120:0/2983114637 >> 172.21.15.120:0/2983114637 conn(0x55ba757dbcf0 msgr2=0x55ba757e9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:37.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.838+0000 7fdae8e58740 1 -- 172.21.15.120:0/2983114637 shutdown_connections 2024-09-19T09:03:37.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.838+0000 7fdae8e58740 1 -- 172.21.15.120:0/2983114637 wait complete. 2024-09-19T09:03:37.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.838+0000 7fdae8e58740 1 -- start start 2024-09-19T09:03:37.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.838+0000 7fdae8e58740 1 Processor -- start 2024-09-19T09:03:37.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.839+0000 7fdae8e58740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba757e0080 0x55ba757fb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:37.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.839+0000 7fdae8e58740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba757e8470 0x55ba757fb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:37.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.839+0000 7fdae8e58740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba757ea940 0x55ba757f81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:37.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.839+0000 7fdae8e58740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdae01ac740 con 0x55ba757e0080 2024-09-19T09:03:37.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.839+0000 7fdae8e58740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdae01ac070 con 0x55ba757ea940 2024-09-19T09:03:37.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.839+0000 7fdae8e58740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdae01a9290 con 0x55ba757e8470 2024-09-19T09:03:37.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.839+0000 7fdae8e58740 10 client.0 _ll_register_callbacks cb 0x7fdad8003d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:37.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.839+0000 7fdae481b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba757e0080 0x55ba757fb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:37.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.839+0000 7fdae481b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba757e0080 0x55ba757fb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41826/0 (socket says 172.21.15.120:41826) 2024-09-19T09:03:37.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.839+0000 7fdae481b640 1 -- 172.21.15.120:0/2460731375 learned_addr learned my addr 172.21.15.120:0/2460731375 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:37.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.839+0000 7fdadffff640 1 --2- 172.21.15.120:0/2460731375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba757e8470 0x55ba757fb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:37.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.839+0000 7fdae481b640 1 -- 172.21.15.120:0/2460731375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba757e8470 msgr2=0x55ba757fb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:37.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.839+0000 7fdae481b640 1 --2- 172.21.15.120:0/2460731375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba757e8470 0x55ba757fb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.839+0000 7fdae481b640 1 -- 172.21.15.120:0/2460731375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba757ea940 msgr2=0x55ba757f81b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:03:37.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.839+0000 7fdae481b640 1 --2- 172.21.15.120:0/2460731375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba757ea940 0x55ba757f81b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.839+0000 7fdae481b640 1 -- 172.21.15.120:0/2460731375 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba757f8820 con 0x55ba757e0080 2024-09-19T09:03:37.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.839+0000 7fdadffff640 1 --2- 172.21.15.120:0/2460731375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba757e8470 0x55ba757fb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:03:37.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.840+0000 7fdae481b640 1 --2- 172.21.15.120:0/2460731375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba757e0080 0x55ba757fb3c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x55ba757dc8f0 tx=0x7fdad40058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46db11b49563caf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:37.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.840+0000 7fdabffff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:37.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.840+0000 7fdabffff640 1 -- 172.21.15.120:0/2460731375 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4016040 con 0x55ba757e0080 2024-09-19T09:03:37.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.840+0000 7fdabffff640 1 -- 172.21.15.120:0/2460731375 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fdad4004620 con 0x55ba757e0080 2024-09-19T09:03:37.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.840+0000 7fdae8e58740 1 -- 172.21.15.120:0/2460731375 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdacc09a2e0 con 0x55ba757e0080 2024-09-19T09:03:37.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.840+0000 7fdabffff640 1 -- 172.21.15.120:0/2460731375 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdad40097c0 con 0x55ba757e0080 2024-09-19T09:03:37.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.840+0000 7fdae8e58740 10 client.15603 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:37.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.840+0000 7fdae8e58740 1 -- 172.21.15.120:0/2460731375 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ba7586ecb0 con 0x55ba757e0080 2024-09-19T09:03:37.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.840+0000 7fdabffff640 1 -- 172.21.15.120:0/2460731375 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdacc09a2e0 con 0x55ba757e0080 2024-09-19T09:03:37.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.840+0000 7fdabffff640 1 -- 172.21.15.120:0/2460731375 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdad400f040 con 0x55ba757e0080 2024-09-19T09:03:37.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.841+0000 7fdae8e58740 10 client.15603 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:37.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.841+0000 7fdae8e58740 10 client.15603 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:03:37.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.841+0000 7fdae8e58740 20 client.15603 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:37.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.841+0000 7fdae8e58740 1 --2- 172.21.15.120:0/2460731375 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ba7586f260 0x55ba75871650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:37.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.841+0000 7fdae8e58740 4 client.15603 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.2, "await_for": 45.0}] 2024-09-19T09:03:37.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.841+0000 7fdae8e58740 1 -- 172.21.15.120:0/2460731375 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.2, "await_for": 45.0}) -- 0x55ba757529d0 con 0x55ba7586f260 2024-09-19T09:03:37.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.841+0000 7fdadffff640 1 --2- 172.21.15.120:0/2460731375 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ba7586f260 0x55ba75871650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:37.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.842+0000 7fdadffff640 1 --2- 172.21.15.120:0/2460731375 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ba7586f260 0x55ba75871650 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b2b6da013a83603 server_cookie=92d95dc22dd1c240 in_seq=0 out_seq=0 2024-09-19T09:03:37.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.842+0000 7fdabffff640 10 client.15603 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:37.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.850+0000 7fdabffff640 1 -- 172.21.15.120:0/2460731375 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55ba757529d0 con 0x55ba7586f260 2024-09-19T09:03:37.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.850+0000 7fdabffff640 10 client.15603 handle_command_reply: tid=0 2024-09-19T09:03:37.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.850+0000 7fdae8e58740 1 client.15603 shutdown 2024-09-19T09:03:37.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.850+0000 7fdae8e58740 1 -- 172.21.15.120:0/2460731375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba757e0080 msgr2=0x55ba757fb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:37.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.850+0000 7fdae8e58740 1 --2- 172.21.15.120:0/2460731375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba757e0080 0x55ba757fb3c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x55ba757dc8f0 tx=0x7fdad40058e0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.850+0000 7fdae8e58740 1 -- 172.21.15.120:0/2460731375 shutdown_connections 2024-09-19T09:03:37.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.850+0000 7fdae8e58740 1 --2- 172.21.15.120:0/2460731375 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ba7586f260 0x55ba75871650 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.850+0000 7fdae8e58740 1 --2- 172.21.15.120:0/2460731375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba757ea940 0x55ba757f81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.850+0000 7fdae8e58740 1 --2- 172.21.15.120:0/2460731375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba757e8470 0x55ba757fb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.850+0000 7fdae8e58740 1 --2- 172.21.15.120:0/2460731375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba757e0080 0x55ba757fb3c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.850+0000 7fdae8e58740 1 -- 172.21.15.120:0/2460731375 >> 172.21.15.120:0/2460731375 conn(0x55ba757dbcf0 msgr2=0x55ba757e11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:37.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.850+0000 7fdabffff640 0 client.15603 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:37.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.850+0000 7fdae8e58740 1 -- 172.21.15.120:0/2460731375 shutdown_connections 2024-09-19T09:03:37.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.850+0000 7fdae8e58740 1 -- 172.21.15.120:0/2460731375 wait complete. 2024-09-19T09:03:37.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.851+0000 7fdae8e58740 20 client.15603 trim_cache size 0 max 16384 2024-09-19T09:03:37.852 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:03:37.852 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 28, 2024-09-19T09:03:37.852 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:03:37.852 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 2, 2024-09-19T09:03:37.852 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:03:37.853 INFO:teuthology.orchestra.run.smithi120.stdout: "27390328": { 2024-09-19T09:03:37.853 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 2, 2024-09-19T09:03:37.853 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:03:37.853 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:03:37.853 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:03:37.853 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:03:37.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:03:37.853 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:03:37.853 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 131.2, 2024-09-19T09:03:37.853 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:03:37.853 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:03:37.853 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:03:37.853 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:03:37.854 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:03:37.854 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:03:37.854 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:03:37.854 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:03:37.854 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:03:37.854 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:03:37.854 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:03:37.854 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:03:37.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.851+0000 7fdae6a7f640 1 -- 172.21.15.120:0/3981456850 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdad00a3420 msgr2=0x7fdad00a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:37.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.851+0000 7fdae6a7f640 1 --2- 172.21.15.120:0/3981456850 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdad00a3420 0x7fdad00a58e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdad8006010 tx=0x7fdad80099d0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.851+0000 7fdae6a7f640 1 -- 172.21.15.120:0/3981456850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae01027c0 msgr2=0x7fdae01988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:37.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.851+0000 7fdae6a7f640 1 --2- 172.21.15.120:0/3981456850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae01027c0 0x7fdae01988c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00c920 tx=0x7fdacc00cde0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.851+0000 7fdae6a7f640 1 -- 172.21.15.120:0/3981456850 shutdown_connections 2024-09-19T09:03:37.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.851+0000 7fdae6a7f640 1 --2- 172.21.15.120:0/3981456850 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdad00a3420 0x7fdad00a58e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.851+0000 7fdae6a7f640 1 --2- 172.21.15.120:0/3981456850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdae0106f40 0x7fdae019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.851+0000 7fdae6a7f640 1 --2- 172.21.15.120:0/3981456850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae01027c0 0x7fdae01988c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.851+0000 7fdae6a7f640 1 --2- 172.21.15.120:0/3981456850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae0101e10 0x7fdae0198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:37.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.851+0000 7fdae6a7f640 1 -- 172.21.15.120:0/3981456850 >> 172.21.15.120:0/3981456850 conn(0x7fdae00fd8d0 msgr2=0x7fdae00ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:37.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.852+0000 7fdae6a7f640 1 -- 172.21.15.120:0/3981456850 shutdown_connections 2024-09-19T09:03:37.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:37.852+0000 7fdae6a7f640 1 -- 172.21.15.120:0/3981456850 wait complete. 2024-09-19T09:03:38.298 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:03:38.935 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:38 smithi120 ceph-mon[31150]: pgmap v364: 97 pgs: 97 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-19T09:03:38.970 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '27390328', quiesce took 10.4 seconds. Will release after: 0.7999999999999989 2024-09-19T09:03:39.042 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:03:39.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:38 smithi196 ceph-mon[35150]: pgmap v364: 97 pgs: 97 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-19T09:03:39.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:38 smithi153 ceph-mon[35071]: pgmap v364: 97 pgs: 97 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-19T09:03:39.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.553+0000 7f4d90b14640 1 -- 172.21.15.120:0/601696007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d8c102db0 msgr2=0x7f4d8c103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:39.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.553+0000 7f4d90b14640 1 --2- 172.21.15.120:0/601696007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d8c102db0 0x7f4d8c103190 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c009930 tx=0x7f4d7c02a420 comp rx=0 tx=0).stop 2024-09-19T09:03:39.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.553+0000 7f4d90b14640 1 -- 172.21.15.120:0/601696007 shutdown_connections 2024-09-19T09:03:39.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.553+0000 7f4d90b14640 1 --2- 172.21.15.120:0/601696007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d8c1082e0 0x7f4d8c10ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.553+0000 7f4d90b14640 1 --2- 172.21.15.120:0/601696007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d8c103760 0x7f4d8c107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.553+0000 7f4d90b14640 1 --2- 172.21.15.120:0/601696007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d8c102db0 0x7f4d8c103190 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.553+0000 7f4d90b14640 1 -- 172.21.15.120:0/601696007 >> 172.21.15.120:0/601696007 conn(0x7f4d8c0fc7d0 msgr2=0x7f4d8c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:39.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.554+0000 7f4d90b14640 1 -- 172.21.15.120:0/601696007 shutdown_connections 2024-09-19T09:03:39.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.554+0000 7f4d90b14640 1 -- 172.21.15.120:0/601696007 wait complete. 2024-09-19T09:03:39.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.555+0000 7f4d90b14640 1 Processor -- start 2024-09-19T09:03:39.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.555+0000 7f4d90b14640 1 -- start start 2024-09-19T09:03:39.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.555+0000 7f4d90b14640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d8c102db0 0x7f4d8c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:39.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.555+0000 7f4d90b14640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d8c103760 0x7f4d8c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:39.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.555+0000 7f4d90b14640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d8c1082e0 0x7f4d8c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:39.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.555+0000 7f4d90b14640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c1120b0 con 0x7f4d8c1082e0 2024-09-19T09:03:39.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.555+0000 7f4d90b14640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c111f30 con 0x7f4d8c102db0 2024-09-19T09:03:39.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.555+0000 7f4d90b14640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c112230 con 0x7f4d8c103760 2024-09-19T09:03:39.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.555+0000 7f4d8ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d8c1082e0 0x7f4d8c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:39.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.555+0000 7f4d8ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d8c1082e0 0x7f4d8c19ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41830/0 (socket says 172.21.15.120:41830) 2024-09-19T09:03:39.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.555+0000 7f4d8ad9d640 1 -- 172.21.15.120:0/1977921407 learned_addr learned my addr 172.21.15.120:0/1977921407 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:39.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.555+0000 7f4d8a59c640 1 --2- 172.21.15.120:0/1977921407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d8c102db0 0x7f4d8c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:39.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.556+0000 7f4d89d9b640 1 --2- 172.21.15.120:0/1977921407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d8c103760 0x7f4d8c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:39.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.556+0000 7f4d8ad9d640 1 -- 172.21.15.120:0/1977921407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d8c103760 msgr2=0x7f4d8c198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:39.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.556+0000 7f4d8ad9d640 1 --2- 172.21.15.120:0/1977921407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d8c103760 0x7f4d8c198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.556+0000 7f4d8ad9d640 1 -- 172.21.15.120:0/1977921407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d8c102db0 msgr2=0x7f4d8c1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:39.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.556+0000 7f4d8ad9d640 1 --2- 172.21.15.120:0/1977921407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d8c102db0 0x7f4d8c1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.556+0000 7f4d8ad9d640 1 -- 172.21.15.120:0/1977921407 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8c19d560 con 0x7f4d8c1082e0 2024-09-19T09:03:39.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.556+0000 7f4d8a59c640 1 --2- 172.21.15.120:0/1977921407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d8c102db0 0x7f4d8c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:03:39.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.556+0000 7f4d8ad9d640 1 --2- 172.21.15.120:0/1977921407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d8c1082e0 0x7f4d8c19ce80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400cad0 tx=0x7f4d8400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:39.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.556+0000 7f4d7b7fe640 1 -- 172.21.15.120:0/1977921407 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8400ae20 con 0x7f4d8c1082e0 2024-09-19T09:03:39.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.556+0000 7f4d7b7fe640 1 -- 172.21.15.120:0/1977921407 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d84011750 con 0x7f4d8c1082e0 2024-09-19T09:03:39.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.556+0000 7f4d90b14640 1 -- 172.21.15.120:0/1977921407 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d8c19d850 con 0x7f4d8c1082e0 2024-09-19T09:03:39.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.556+0000 7f4d7b7fe640 1 -- 172.21.15.120:0/1977921407 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d84005520 con 0x7f4d8c1082e0 2024-09-19T09:03:39.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.556+0000 7f4d90b14640 1 -- 172.21.15.120:0/1977921407 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d8c100940 con 0x7f4d8c1082e0 2024-09-19T09:03:39.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.559+0000 7f4d92eed740 1 Processor -- start 2024-09-19T09:03:39.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.560+0000 7f4d7b7fe640 1 -- 172.21.15.120:0/1977921407 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f4d84011900 con 0x7f4d8c1082e0 2024-09-19T09:03:39.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.560+0000 7f4d92eed740 1 -- start start 2024-09-19T09:03:39.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.560+0000 7f4d92eed740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3f2a7db40 0x55b3f2a7c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:39.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.560+0000 7f4d92eed740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3f2a7c5d0 0x55b3f2a7ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:39.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.561+0000 7f4d92eed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3f2a87370 0x55b3f2a89760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:39.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.561+0000 7f4d92eed740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b3f29e22f0 con 0x55b3f2a7db40 2024-09-19T09:03:39.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.561+0000 7f4d92eed740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b3f2989cf0 con 0x55b3f2a7c5d0 2024-09-19T09:03:39.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.561+0000 7f4d92eed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b3f29de2b0 con 0x55b3f2a87370 2024-09-19T09:03:39.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.561+0000 7f4d89d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3f2a7db40 0x55b3f2a7c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:39.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.561+0000 7f4d8a59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3f2a7c5d0 0x55b3f2a7ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:39.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.561+0000 7f4d8a59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3f2a7c5d0 0x55b3f2a7ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:50116/0 (socket says 172.21.15.120:50116) 2024-09-19T09:03:39.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.561+0000 7f4d8a59c640 1 -- 172.21.15.120:0/414853999 learned_addr learned my addr 172.21.15.120:0/414853999 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:39.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.561+0000 7f4d7b7fe640 1 --2- 172.21.15.120:0/1977921407 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4d740a3350 0x7f4d740a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:39.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.561+0000 7f4d8a59c640 1 -- 172.21.15.120:0/414853999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3f2a87370 msgr2=0x55b3f2a89760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:03:39.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.561+0000 7f4d8a59c640 1 --2- 172.21.15.120:0/414853999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3f2a87370 0x55b3f2a89760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.561+0000 7f4d8a59c640 1 -- 172.21.15.120:0/414853999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3f2a7db40 msgr2=0x55b3f2a7c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:39.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.561+0000 7f4d7b7fe640 1 -- 172.21.15.120:0/1977921407 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f4d840d3910 con 0x7f4d8c1082e0 2024-09-19T09:03:39.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.561+0000 7f4d8ad9d640 1 --2- 172.21.15.120:0/1977921407 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4d740a3350 0x7f4d740a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:39.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.561+0000 7f4d8a59c640 1 --2- 172.21.15.120:0/414853999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3f2a7db40 0x55b3f2a7c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.561+0000 7f4d8a59c640 1 -- 172.21.15.120:0/414853999 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3f296e3a0 con 0x55b3f2a7c5d0 2024-09-19T09:03:39.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.561+0000 7f4d89d9b640 1 --2- 172.21.15.120:0/414853999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3f2a7db40 0x55b3f2a7c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:03:39.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.562+0000 7f4d8a59c640 1 --2- 172.21.15.120:0/414853999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3f2a7c5d0 0x55b3f2a7ca10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c000c00 tx=0x7f4d7c0045d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bbe2a2d948f8279 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:39.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.562+0000 7f4d8ad9d640 1 --2- 172.21.15.120:0/1977921407 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4d740a3350 0x7f4d740a5810 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4d840a8620 tx=0x7f4d84013040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:39.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.562+0000 7f4d78ff9640 1 -- 172.21.15.120:0/414853999 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c035030 con 0x55b3f2a7c5d0 2024-09-19T09:03:39.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.562+0000 7f4d78ff9640 1 -- 172.21.15.120:0/414853999 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d7c040070 con 0x55b3f2a7c5d0 2024-09-19T09:03:39.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.562+0000 7f4d78ff9640 1 -- 172.21.15.120:0/414853999 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c045470 con 0x55b3f2a7c5d0 2024-09-19T09:03:39.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.563+0000 7f4d92eed740 1 -- 172.21.15.120:0/414853999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3f2a7c5d0 msgr2=0x55b3f2a7ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:39.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.563+0000 7f4d92eed740 1 --2- 172.21.15.120:0/414853999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3f2a7c5d0 0x55b3f2a7ca10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c000c00 tx=0x7f4d7c0045d0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.563+0000 7f4d92eed740 1 -- 172.21.15.120:0/414853999 shutdown_connections 2024-09-19T09:03:39.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.563+0000 7f4d92eed740 1 --2- 172.21.15.120:0/414853999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3f2a87370 0x55b3f2a89760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.563+0000 7f4d92eed740 1 --2- 172.21.15.120:0/414853999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3f2a7c5d0 0x55b3f2a7ca10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.563+0000 7f4d92eed740 1 --2- 172.21.15.120:0/414853999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3f2a7db40 0x55b3f2a7c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.563+0000 7f4d92eed740 1 -- 172.21.15.120:0/414853999 >> 172.21.15.120:0/414853999 conn(0x55b3f2a45310 msgr2=0x55b3f2a78c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:39.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.564+0000 7f4d92eed740 1 -- 172.21.15.120:0/414853999 shutdown_connections 2024-09-19T09:03:39.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.564+0000 7f4d92eed740 1 -- 172.21.15.120:0/414853999 wait complete. 2024-09-19T09:03:39.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.564+0000 7f4d92eed740 1 -- start start 2024-09-19T09:03:39.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.564+0000 7f4d92eed740 1 Processor -- start 2024-09-19T09:03:39.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.564+0000 7f4d92eed740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3f2a7c5d0 0x55b3f2a86e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:39.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.564+0000 7f4d92eed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3f2a7db40 0x55b3f2a840c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:39.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.564+0000 7f4d92eed740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3f2a87370 0x55b3f2a84850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:39.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.565+0000 7f4d92eed740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b3f2a8a010 con 0x55b3f2a87370 2024-09-19T09:03:39.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.565+0000 7f4d92eed740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b3f299d060 con 0x55b3f2a7c5d0 2024-09-19T09:03:39.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.565+0000 7f4d92eed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b3f2a8a190 con 0x55b3f2a7db40 2024-09-19T09:03:39.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.565+0000 7f4d92eed740 10 client.0 _ll_register_callbacks cb 0x55b3f26f08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:39.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.565+0000 7f4d89d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3f2a87370 0x55b3f2a84850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:39.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.565+0000 7f4d89d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3f2a87370 0x55b3f2a84850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41842/0 (socket says 172.21.15.120:41842) 2024-09-19T09:03:39.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.565+0000 7f4d89d9b640 1 -- 172.21.15.120:0/1670409631 learned_addr learned my addr 172.21.15.120:0/1670409631 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:39.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.565+0000 7f4d8a59c640 1 --2- 172.21.15.120:0/1670409631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3f2a7db40 0x55b3f2a840c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:39.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.565+0000 7f4d89d9b640 1 --2- 172.21.15.120:0/1670409631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3f2a7c5d0 0x55b3f2a86e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:39.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.565+0000 7f4d89d9b640 1 -- 172.21.15.120:0/1670409631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3f2a7db40 msgr2=0x55b3f2a840c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:39.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.565+0000 7f4d89d9b640 1 --2- 172.21.15.120:0/1670409631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3f2a7db40 0x55b3f2a840c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.565+0000 7f4d89d9b640 1 -- 172.21.15.120:0/1670409631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3f2a7c5d0 msgr2=0x55b3f2a86e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:39.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.565+0000 7f4d89d9b640 1 --2- 172.21.15.120:0/1670409631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3f2a7c5d0 0x55b3f2a86e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.565+0000 7f4d89d9b640 1 -- 172.21.15.120:0/1670409631 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3f27533c0 con 0x55b3f2a87370 2024-09-19T09:03:39.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.565+0000 7f4d8a59c640 1 --2- 172.21.15.120:0/1670409631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3f2a7db40 0x55b3f2a840c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:03:39.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.565+0000 7f4d89d9b640 1 --2- 172.21.15.120:0/1670409631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3f2a87370 0x55b3f2a84850 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0697d0 tx=0x7f4d8000c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368666738947731d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:39.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.565+0000 7f4d71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:39.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.566+0000 7f4d71ffb640 1 -- 172.21.15.120:0/1670409631 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d80004020 con 0x55b3f2a87370 2024-09-19T09:03:39.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.566+0000 7f4d71ffb640 1 -- 172.21.15.120:0/1670409631 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f4d800041c0 con 0x55b3f2a87370 2024-09-19T09:03:39.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.566+0000 7f4d92eed740 1 -- 172.21.15.120:0/1670409631 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d7c040070 con 0x55b3f2a87370 2024-09-19T09:03:39.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.566+0000 7f4d71ffb640 1 -- 172.21.15.120:0/1670409631 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4d800043c0 con 0x55b3f2a87370 2024-09-19T09:03:39.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.566+0000 7f4d92eed740 10 client.15615 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:39.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.566+0000 7f4d92eed740 1 -- 172.21.15.120:0/1670409631 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b3f2a95bf0 con 0x55b3f2a87370 2024-09-19T09:03:39.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.566+0000 7f4d71ffb640 1 -- 172.21.15.120:0/1670409631 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d80003ad0 con 0x55b3f2a87370 2024-09-19T09:03:39.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.566+0000 7f4d71ffb640 1 -- 172.21.15.120:0/1670409631 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4d80020990 con 0x55b3f2a87370 2024-09-19T09:03:39.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.566+0000 7f4d92eed740 10 client.15615 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:39.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.566+0000 7f4d92eed740 10 client.15615 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:03:39.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.566+0000 7f4d92eed740 20 client.15615 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:39.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.566+0000 7f4d92eed740 1 --2- 172.21.15.120:0/1670409631 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55b3f2a970b0 0x55b3f2a994a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:39.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.566+0000 7f4d92eed740 4 client.15615 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:03:39.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.566+0000 7f4d92eed740 1 -- 172.21.15.120:0/1670409631 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4d7c040070 con 0x55b3f2a970b0 2024-09-19T09:03:39.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.567+0000 7f4d8a59c640 1 --2- 172.21.15.120:0/1670409631 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55b3f2a970b0 0x55b3f2a994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:39.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.567+0000 7f4d8a59c640 1 --2- 172.21.15.120:0/1670409631 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55b3f2a970b0 0x55b3f2a994a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=928d40d425039b26 server_cookie=cd7957547cc6df2e in_seq=0 out_seq=0 2024-09-19T09:03:39.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.567+0000 7f4d71ffb640 10 client.15615 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:03:39.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.568+0000 7f4d71ffb640 1 -- 172.21.15.120:0/1670409631 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f4d7c040070 con 0x55b3f2a970b0 2024-09-19T09:03:39.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.568+0000 7f4d71ffb640 10 client.15615 handle_command_reply: tid=0 2024-09-19T09:03:39.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.568+0000 7f4d92eed740 1 client.15615 shutdown 2024-09-19T09:03:39.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.568+0000 7f4d92eed740 1 -- 172.21.15.120:0/1670409631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3f2a87370 msgr2=0x55b3f2a84850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:39.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.568+0000 7f4d92eed740 1 --2- 172.21.15.120:0/1670409631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3f2a87370 0x55b3f2a84850 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0697d0 tx=0x7f4d8000c6c0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.568+0000 7f4d92eed740 1 -- 172.21.15.120:0/1670409631 shutdown_connections 2024-09-19T09:03:39.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.568+0000 7f4d92eed740 1 --2- 172.21.15.120:0/1670409631 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55b3f2a970b0 0x55b3f2a994a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.568+0000 7f4d71ffb640 0 client.15615 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:03:39.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.568+0000 7f4d92eed740 1 --2- 172.21.15.120:0/1670409631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3f2a87370 0x55b3f2a84850 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.568+0000 7f4d92eed740 1 --2- 172.21.15.120:0/1670409631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3f2a7db40 0x55b3f2a840c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.568+0000 7f4d92eed740 1 --2- 172.21.15.120:0/1670409631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3f2a7c5d0 0x55b3f2a86e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.568+0000 7f4d92eed740 1 -- 172.21.15.120:0/1670409631 >> 172.21.15.120:0/1670409631 conn(0x55b3f2a45310 msgr2=0x55b3f2a7e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:39.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.569+0000 7f4d92eed740 1 -- 172.21.15.120:0/1670409631 shutdown_connections 2024-09-19T09:03:39.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.569+0000 7f4d92eed740 1 -- 172.21.15.120:0/1670409631 wait complete. 2024-09-19T09:03:39.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.569+0000 7f4d92eed740 20 client.15615 trim_cache size 0 max 16384 2024-09-19T09:03:39.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.593+0000 7f4d92eed740 1 Processor -- start 2024-09-19T09:03:39.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.593+0000 7f4d92eed740 1 -- start start 2024-09-19T09:03:39.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.593+0000 7f4d92eed740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3f2a7c5d0 0x55b3f2a78d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:39.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.593+0000 7f4d92eed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3f2a7db40 0x55b3f2a77bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:39.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.593+0000 7f4d92eed740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3f2a87370 0x55b3f2a9ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:39.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.593+0000 7f4d92eed740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c1ab4d0 con 0x55b3f2a87370 2024-09-19T09:03:39.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.593+0000 7f4d92eed740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c1ad010 con 0x55b3f2a7c5d0 2024-09-19T09:03:39.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.593+0000 7f4d92eed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c1abba0 con 0x55b3f2a7db40 2024-09-19T09:03:39.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.594+0000 7f4d89d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3f2a87370 0x55b3f2a9ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:39.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.594+0000 7f4d89d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3f2a87370 0x55b3f2a9ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41858/0 (socket says 172.21.15.120:41858) 2024-09-19T09:03:39.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.594+0000 7f4d89d9b640 1 -- 172.21.15.120:0/204787758 learned_addr learned my addr 172.21.15.120:0/204787758 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:39.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.594+0000 7f4d8a59c640 1 --2- 172.21.15.120:0/204787758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3f2a7db40 0x55b3f2a77bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:39.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.594+0000 7f4d89d9b640 1 --2- 172.21.15.120:0/204787758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3f2a7c5d0 0x55b3f2a78d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:39.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.594+0000 7f4d89d9b640 1 -- 172.21.15.120:0/204787758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3f2a7db40 msgr2=0x55b3f2a77bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:39.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.594+0000 7f4d89d9b640 1 --2- 172.21.15.120:0/204787758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3f2a7db40 0x55b3f2a77bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.594+0000 7f4d89d9b640 1 -- 172.21.15.120:0/204787758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3f2a7c5d0 msgr2=0x55b3f2a78d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:39.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.594+0000 7f4d89d9b640 1 --2- 172.21.15.120:0/204787758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3f2a7c5d0 0x55b3f2a78d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.594+0000 7f4d89d9b640 1 -- 172.21.15.120:0/204787758 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3f2a87180 con 0x55b3f2a87370 2024-09-19T09:03:39.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.594+0000 7f4d89d9b640 1 --2- 172.21.15.120:0/204787758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3f2a87370 0x55b3f2a9ed00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4d8000d010 tx=0x7f4d80005f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e1da2707eb9b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:39.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.594+0000 7f4d73fff640 1 -- 172.21.15.120:0/204787758 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d80031020 con 0x55b3f2a87370 2024-09-19T09:03:39.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.595+0000 7f4d73fff640 1 -- 172.21.15.120:0/204787758 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d80017040 con 0x55b3f2a87370 2024-09-19T09:03:39.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.595+0000 7f4d73fff640 1 -- 172.21.15.120:0/204787758 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8002b6c0 con 0x55b3f2a87370 2024-09-19T09:03:39.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.595+0000 7f4d92eed740 1 -- 172.21.15.120:0/204787758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3f2a87370 msgr2=0x55b3f2a9ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:39.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.595+0000 7f4d92eed740 1 --2- 172.21.15.120:0/204787758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3f2a87370 0x55b3f2a9ed00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4d8000d010 tx=0x7f4d80005f20 comp rx=0 tx=0).stop 2024-09-19T09:03:39.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.595+0000 7f4d92eed740 1 -- 172.21.15.120:0/204787758 shutdown_connections 2024-09-19T09:03:39.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.595+0000 7f4d92eed740 1 --2- 172.21.15.120:0/204787758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3f2a87370 0x55b3f2a9ed00 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.595+0000 7f4d92eed740 1 --2- 172.21.15.120:0/204787758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3f2a7db40 0x55b3f2a77bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.595+0000 7f4d92eed740 1 --2- 172.21.15.120:0/204787758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3f2a7c5d0 0x55b3f2a78d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.595+0000 7f4d92eed740 1 -- 172.21.15.120:0/204787758 >> 172.21.15.120:0/204787758 conn(0x55b3f2a45310 msgr2=0x55b3f2a774e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:39.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.596+0000 7f4d92eed740 1 -- 172.21.15.120:0/204787758 shutdown_connections 2024-09-19T09:03:39.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.596+0000 7f4d92eed740 1 -- 172.21.15.120:0/204787758 wait complete. 2024-09-19T09:03:39.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.596+0000 7f4d92eed740 1 -- start start 2024-09-19T09:03:39.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.596+0000 7f4d92eed740 1 Processor -- start 2024-09-19T09:03:39.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.596+0000 7f4d92eed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3f2a7c5d0 0x55b3f2a8f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:39.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.596+0000 7f4d92eed740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3f2a7db40 0x55b3f2a8aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:39.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.596+0000 7f4d92eed740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3f2a970b0 0x55b3f2a8b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:39.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.596+0000 7f4d92eed740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c1ac940 con 0x55b3f2a970b0 2024-09-19T09:03:39.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.596+0000 7f4d92eed740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c1ac270 con 0x55b3f2a7db40 2024-09-19T09:03:39.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.596+0000 7f4d92eed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c1a9490 con 0x55b3f2a7c5d0 2024-09-19T09:03:39.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.596+0000 7f4d92eed740 10 client.0 _ll_register_callbacks cb 0x7f4d8000c9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:39.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.596+0000 7f4d89d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3f2a970b0 0x55b3f2a8b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:39.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.597+0000 7f4d89d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3f2a970b0 0x55b3f2a8b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41870/0 (socket says 172.21.15.120:41870) 2024-09-19T09:03:39.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.597+0000 7f4d89d9b640 1 -- 172.21.15.120:0/2190341805 learned_addr learned my addr 172.21.15.120:0/2190341805 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:39.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.597+0000 7f4d8a59c640 1 --2- 172.21.15.120:0/2190341805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3f2a7db40 0x55b3f2a8aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:39.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.597+0000 7f4d8a59c640 1 -- 172.21.15.120:0/2190341805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3f2a7c5d0 msgr2=0x55b3f2a8f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:39.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.597+0000 7f4d8a59c640 1 --2- 172.21.15.120:0/2190341805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3f2a7c5d0 0x55b3f2a8f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.597+0000 7f4d8a59c640 1 -- 172.21.15.120:0/2190341805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3f2a970b0 msgr2=0x55b3f2a8b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:39.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.597+0000 7f4d8a59c640 1 --2- 172.21.15.120:0/2190341805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3f2a970b0 0x55b3f2a8b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.597+0000 7f4d8a59c640 1 -- 172.21.15.120:0/2190341805 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3f2a8b620 con 0x55b3f2a7db40 2024-09-19T09:03:39.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.597+0000 7f4d89d9b640 1 --2- 172.21.15.120:0/2190341805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3f2a970b0 0x55b3f2a8b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:03:39.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.597+0000 7f4d8a59c640 1 --2- 172.21.15.120:0/2190341805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3f2a7db40 0x55b3f2a8aba0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c009900 tx=0x7f4d7c031b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c9765c25cd6a2c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:39.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.597+0000 7f4d71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:03:39.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.598+0000 7f4d71ffb640 1 -- 172.21.15.120:0/2190341805 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c035040 con 0x55b3f2a7db40 2024-09-19T09:03:39.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.598+0000 7f4d92eed740 1 -- 172.21.15.120:0/2190341805 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d80017040 con 0x55b3f2a7db40 2024-09-19T09:03:39.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.598+0000 7f4d71ffb640 1 -- 172.21.15.120:0/2190341805 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f4d7c003a30 con 0x55b3f2a7db40 2024-09-19T09:03:39.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.598+0000 7f4d71ffb640 1 -- 172.21.15.120:0/2190341805 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4d7c003c30 con 0x55b3f2a7db40 2024-09-19T09:03:39.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.598+0000 7f4d92eed740 10 client.25055 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:39.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.598+0000 7f4d92eed740 1 -- 172.21.15.120:0/2190341805 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b3f2b0a250 con 0x55b3f2a7db40 2024-09-19T09:03:39.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.598+0000 7f4d71ffb640 1 -- 172.21.15.120:0/2190341805 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d80017040 con 0x55b3f2a7db40 2024-09-19T09:03:39.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.599+0000 7f4d71ffb640 1 -- 172.21.15.120:0/2190341805 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4d7c051420 con 0x55b3f2a7db40 2024-09-19T09:03:39.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.599+0000 7f4d92eed740 10 client.25055 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:39.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.599+0000 7f4d92eed740 10 client.25055 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:03:39.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.599+0000 7f4d92eed740 20 client.25055 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:39.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.599+0000 7f4d92eed740 1 --2- 172.21.15.120:0/2190341805 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55b3f2b0a540 0x55b3f2b0c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:39.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.599+0000 7f4d92eed740 4 client.25055 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:03:39.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.599+0000 7f4d92eed740 1 -- 172.21.15.120:0/2190341805 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b3f29ef860 con 0x55b3f2b0a540 2024-09-19T09:03:39.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.599+0000 7f4d89d9b640 1 --2- 172.21.15.120:0/2190341805 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55b3f2b0a540 0x55b3f2b0c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:39.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.599+0000 7f4d71ffb640 10 client.25055 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:03:39.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.599+0000 7f4d89d9b640 1 --2- 172.21.15.120:0/2190341805 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55b3f2b0a540 0x55b3f2b0c930 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a755ab16b552b0d6 server_cookie=8ec7874f5fed297b in_seq=0 out_seq=0 2024-09-19T09:03:39.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.599+0000 7f4d71ffb640 1 -- 172.21.15.120:0/2190341805 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b3f29ef860 con 0x55b3f2b0a540 2024-09-19T09:03:39.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.599+0000 7f4d71ffb640 10 client.25055 handle_command_reply: tid=0 2024-09-19T09:03:39.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.599+0000 7f4d92eed740 1 client.25055 shutdown 2024-09-19T09:03:39.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.599+0000 7f4d92eed740 1 -- 172.21.15.120:0/2190341805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3f2a7db40 msgr2=0x55b3f2a8aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:39.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.599+0000 7f4d92eed740 1 --2- 172.21.15.120:0/2190341805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3f2a7db40 0x55b3f2a8aba0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c009900 tx=0x7f4d7c031b40 comp rx=0 tx=0).stop 2024-09-19T09:03:39.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.600+0000 7f4d92eed740 1 -- 172.21.15.120:0/2190341805 shutdown_connections 2024-09-19T09:03:39.605 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:03:39.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.600+0000 7f4d92eed740 1 --2- 172.21.15.120:0/2190341805 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55b3f2b0a540 0x55b3f2b0c930 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.600+0000 7f4d92eed740 1 --2- 172.21.15.120:0/2190341805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3f2a970b0 0x55b3f2a8b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.600+0000 7f4d71ffb640 0 client.25055 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:03:39.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.600+0000 7f4d92eed740 1 --2- 172.21.15.120:0/2190341805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3f2a7db40 0x55b3f2a8aba0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.600+0000 7f4d92eed740 1 --2- 172.21.15.120:0/2190341805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3f2a7c5d0 0x55b3f2a8f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.600+0000 7f4d92eed740 1 -- 172.21.15.120:0/2190341805 >> 172.21.15.120:0/2190341805 conn(0x55b3f2a45310 msgr2=0x55b3f2a774e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:39.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.600+0000 7f4d92eed740 1 -- 172.21.15.120:0/2190341805 shutdown_connections 2024-09-19T09:03:39.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.600+0000 7f4d92eed740 1 -- 172.21.15.120:0/2190341805 wait complete. 2024-09-19T09:03:39.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.600+0000 7f4d92eed740 20 client.25055 trim_cache size 0 max 16384 2024-09-19T09:03:39.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.600+0000 7f4d90b14640 1 -- 172.21.15.120:0/1977921407 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4d740a3350 msgr2=0x7f4d740a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:39.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.600+0000 7f4d90b14640 1 --2- 172.21.15.120:0/1977921407 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4d740a3350 0x7f4d740a5810 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4d840a8620 tx=0x7f4d84013040 comp rx=0 tx=0).stop 2024-09-19T09:03:39.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.600+0000 7f4d90b14640 1 -- 172.21.15.120:0/1977921407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d8c1082e0 msgr2=0x7f4d8c19ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:39.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.600+0000 7f4d90b14640 1 --2- 172.21.15.120:0/1977921407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d8c1082e0 0x7f4d8c19ce80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400cad0 tx=0x7f4d8400cf90 comp rx=0 tx=0).stop 2024-09-19T09:03:39.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.600+0000 7f4d90b14640 1 -- 172.21.15.120:0/1977921407 shutdown_connections 2024-09-19T09:03:39.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.600+0000 7f4d90b14640 1 --2- 172.21.15.120:0/1977921407 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4d740a3350 0x7f4d740a5810 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.600+0000 7f4d90b14640 1 --2- 172.21.15.120:0/1977921407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d8c1082e0 0x7f4d8c19ce80 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.600+0000 7f4d90b14640 1 --2- 172.21.15.120:0/1977921407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d8c103760 0x7f4d8c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.600+0000 7f4d90b14640 1 --2- 172.21.15.120:0/1977921407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d8c102db0 0x7f4d8c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:39.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.600+0000 7f4d90b14640 1 -- 172.21.15.120:0/1977921407 >> 172.21.15.120:0/1977921407 conn(0x7f4d8c0fc7d0 msgr2=0x7f4d8c105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:39.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.601+0000 7f4d90b14640 1 -- 172.21.15.120:0/1977921407 shutdown_connections 2024-09-19T09:03:39.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:39.601+0000 7f4d90b14640 1 -- 172.21.15.120:0/1977921407 wait complete. 2024-09-19T09:03:39.771 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:03:40.482 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:03:40.602 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:03:41.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:40 smithi120 ceph-mon[31150]: pgmap v365: 97 pgs: 97 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-19T09:03:41.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:40 smithi196 ceph-mon[35150]: pgmap v365: 97 pgs: 97 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-19T09:03:41.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:40 smithi153 ceph-mon[35071]: pgmap v365: 97 pgs: 97 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-19T09:03:41.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.201+0000 7ff10ae61640 1 -- 172.21.15.120:0/3196413462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff104100a80 msgr2=0x7ff104100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:41.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.201+0000 7ff10ae61640 1 --2- 172.21.15.120:0/3196413462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff104100a80 0x7ff104100ee0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0009890 tx=0x7ff0f002a380 comp rx=0 tx=0).stop 2024-09-19T09:03:41.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.201+0000 7ff10ae61640 1 -- 172.21.15.120:0/3196413462 shutdown_connections 2024-09-19T09:03:41.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.201+0000 7ff10ae61640 1 --2- 172.21.15.120:0/3196413462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff104101420 0x7ff10410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:41.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.201+0000 7ff10ae61640 1 --2- 172.21.15.120:0/3196413462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff104100a80 0x7ff104100ee0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:41.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.201+0000 7ff10ae61640 1 --2- 172.21.15.120:0/3196413462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff104106a80 0x7ff104106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:41.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.201+0000 7ff10ae61640 1 -- 172.21.15.120:0/3196413462 >> 172.21.15.120:0/3196413462 conn(0x7ff1040fc7a0 msgr2=0x7ff1040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:41.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.201+0000 7ff10ae61640 1 -- 172.21.15.120:0/3196413462 shutdown_connections 2024-09-19T09:03:41.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.201+0000 7ff10ae61640 1 -- 172.21.15.120:0/3196413462 wait complete. 2024-09-19T09:03:41.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.202+0000 7ff10ae61640 1 Processor -- start 2024-09-19T09:03:41.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.202+0000 7ff10ae61640 1 -- start start 2024-09-19T09:03:41.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.203+0000 7ff10ae61640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff104101420 0x7ff1041974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:41.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.203+0000 7ff10ae61640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff104106a80 0x7ff104197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:41.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.203+0000 7ff10ae61640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff10419bda0 0x7ff10419c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:41.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.203+0000 7ff10ae61640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff104110e80 con 0x7ff104106a80 2024-09-19T09:03:41.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.203+0000 7ff10ae61640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff104110d00 con 0x7ff104101420 2024-09-19T09:03:41.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.203+0000 7ff10ae61640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff104111000 con 0x7ff10419bda0 2024-09-19T09:03:41.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.203+0000 7ff108bfd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff104101420 0x7ff1041974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:41.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.204+0000 7ff103fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff104106a80 0x7ff104197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:41.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.204+0000 7ff1093fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff10419bda0 0x7ff10419c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:41.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.204+0000 7ff103fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff104106a80 0x7ff104197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41878/0 (socket says 172.21.15.120:41878) 2024-09-19T09:03:41.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.204+0000 7ff103fff640 1 -- 172.21.15.120:0/1146357580 learned_addr learned my addr 172.21.15.120:0/1146357580 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:41.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.204+0000 7ff1093fe640 1 -- 172.21.15.120:0/1146357580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff104101420 msgr2=0x7ff1041974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:41.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.204+0000 7ff1093fe640 1 --2- 172.21.15.120:0/1146357580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff104101420 0x7ff1041974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:41.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.204+0000 7ff1093fe640 1 -- 172.21.15.120:0/1146357580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff104106a80 msgr2=0x7ff104197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:41.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.204+0000 7ff1093fe640 1 --2- 172.21.15.120:0/1146357580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff104106a80 0x7ff104197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:41.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.204+0000 7ff1093fe640 1 -- 172.21.15.120:0/1146357580 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff10419c860 con 0x7ff10419bda0 2024-09-19T09:03:41.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.204+0000 7ff103fff640 1 --2- 172.21.15.120:0/1146357580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff104106a80 0x7ff104197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:03:41.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.205+0000 7ff1093fe640 1 --2- 172.21.15.120:0/1146357580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff10419bda0 0x7ff10419c250 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00cab0 tx=0x7ff0fc00cf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:41.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.205+0000 7ff101ffb640 1 -- 172.21.15.120:0/1146357580 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc00ae20 con 0x7ff10419bda0 2024-09-19T09:03:41.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.205+0000 7ff101ffb640 1 -- 172.21.15.120:0/1146357580 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0fc00f930 con 0x7ff10419bda0 2024-09-19T09:03:41.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.205+0000 7ff101ffb640 1 -- 172.21.15.120:0/1146357580 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc018b30 con 0x7ff10419bda0 2024-09-19T09:03:41.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.205+0000 7ff10ae61640 1 -- 172.21.15.120:0/1146357580 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff10419cb50 con 0x7ff10419bda0 2024-09-19T09:03:41.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.205+0000 7ff10ae61640 1 -- 172.21.15.120:0/1146357580 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff10406a9c0 con 0x7ff10419bda0 2024-09-19T09:03:41.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.207+0000 7ff10ae61640 1 -- 172.21.15.120:0/1146357580 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff104100ee0 con 0x7ff10419bda0 2024-09-19T09:03:41.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.209+0000 7ff101ffb640 1 -- 172.21.15.120:0/1146357580 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7ff0fc00fae0 con 0x7ff10419bda0 2024-09-19T09:03:41.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.209+0000 7ff101ffb640 1 --2- 172.21.15.120:0/1146357580 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff0f40a3350 0x7ff0f40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:41.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.210+0000 7ff101ffb640 1 -- 172.21.15.120:0/1146357580 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7ff0fc0d06d0 con 0x7ff10419bda0 2024-09-19T09:03:41.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.210+0000 7ff108bfd640 1 --2- 172.21.15.120:0/1146357580 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff0f40a3350 0x7ff0f40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:41.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.211+0000 7ff108bfd640 1 --2- 172.21.15.120:0/1146357580 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff0f40a3350 0x7ff0f40a5810 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8005fd0 tx=0x7ff0f8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:41.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.212+0000 7ff101ffb640 1 -- 172.21.15.120:0/1146357580 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff0fc014030 con 0x7ff10419bda0 2024-09-19T09:03:41.240 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:03:41.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.556+0000 7ff10ae61640 1 -- 172.21.15.120:0/1146357580 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff104198830 con 0x7ff10419bda0 2024-09-19T09:03:41.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.557+0000 7ff101ffb640 1 -- 172.21.15.120:0/1146357580 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7ff104198830 con 0x7ff10419bda0 2024-09-19T09:03:41.558 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:03:41.558 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":28,"btime":"2024-09-19T09:02:41:696728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:02:41.207357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:03:41.558 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 28 2024-09-19T09:03:41.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.560+0000 7ff10ae61640 1 -- 172.21.15.120:0/1146357580 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff0f40a3350 msgr2=0x7ff0f40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:41.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.560+0000 7ff10ae61640 1 --2- 172.21.15.120:0/1146357580 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff0f40a3350 0x7ff0f40a5810 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8005fd0 tx=0x7ff0f8009260 comp rx=0 tx=0).stop 2024-09-19T09:03:41.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.560+0000 7ff10ae61640 1 -- 172.21.15.120:0/1146357580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff10419bda0 msgr2=0x7ff10419c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:41.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.560+0000 7ff10ae61640 1 --2- 172.21.15.120:0/1146357580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff10419bda0 0x7ff10419c250 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00cab0 tx=0x7ff0fc00cf70 comp rx=0 tx=0).stop 2024-09-19T09:03:41.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.561+0000 7ff10ae61640 1 -- 172.21.15.120:0/1146357580 shutdown_connections 2024-09-19T09:03:41.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.561+0000 7ff10ae61640 1 --2- 172.21.15.120:0/1146357580 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff0f40a3350 0x7ff0f40a5810 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:41.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.561+0000 7ff10ae61640 1 --2- 172.21.15.120:0/1146357580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff10419bda0 0x7ff10419c250 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:41.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.561+0000 7ff10ae61640 1 --2- 172.21.15.120:0/1146357580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff104106a80 0x7ff104197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:41.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.561+0000 7ff10ae61640 1 --2- 172.21.15.120:0/1146357580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff104101420 0x7ff1041974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:41.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.561+0000 7ff10ae61640 1 -- 172.21.15.120:0/1146357580 >> 172.21.15.120:0/1146357580 conn(0x7ff1040fc7a0 msgr2=0x7ff1040fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:41.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.561+0000 7ff10ae61640 1 -- 172.21.15.120:0/1146357580 shutdown_connections 2024-09-19T09:03:41.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.562+0000 7ff10ae61640 1 -- 172.21.15.120:0/1146357580 wait complete. 2024-09-19T09:03:41.972 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:41 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1146357580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:03:41.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.969+0000 7f45c08c9640 1 -- 172.21.15.120:0/3934713308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45bc1036e0 msgr2=0x7f45bc107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:41.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.969+0000 7f45c08c9640 1 --2- 172.21.15.120:0/3934713308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45bc1036e0 0x7f45bc107b90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009930 tx=0x7f45b002a420 comp rx=0 tx=0).stop 2024-09-19T09:03:41.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.969+0000 7f45c08c9640 1 -- 172.21.15.120:0/3934713308 shutdown_connections 2024-09-19T09:03:41.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.969+0000 7f45c08c9640 1 --2- 172.21.15.120:0/3934713308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45bc1082a0 0x7f45bc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:41.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.969+0000 7f45c08c9640 1 --2- 172.21.15.120:0/3934713308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45bc1036e0 0x7f45bc107b90 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:41.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.969+0000 7f45c08c9640 1 --2- 172.21.15.120:0/3934713308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45bc102d30 0x7f45bc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:41.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.969+0000 7f45c08c9640 1 -- 172.21.15.120:0/3934713308 >> 172.21.15.120:0/3934713308 conn(0x7f45bc0fc7d0 msgr2=0x7f45bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:41.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.969+0000 7f45c08c9640 1 -- 172.21.15.120:0/3934713308 shutdown_connections 2024-09-19T09:03:41.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.969+0000 7f45c08c9640 1 -- 172.21.15.120:0/3934713308 wait complete. 2024-09-19T09:03:41.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.970+0000 7f45c08c9640 1 Processor -- start 2024-09-19T09:03:41.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.970+0000 7f45c08c9640 1 -- start start 2024-09-19T09:03:41.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.970+0000 7f45c08c9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45bc102d30 0x7f45bc100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:41.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.970+0000 7f45c08c9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45bc1036e0 0x7f45bc100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:41.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.971+0000 7f45c08c9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45bc1082a0 0x7f45bc101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:41.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.971+0000 7f45c08c9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc112060 con 0x7f45bc1082a0 2024-09-19T09:03:41.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.971+0000 7f45c08c9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc111ee0 con 0x7f45bc1036e0 2024-09-19T09:03:41.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.971+0000 7f45c08c9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc1121e0 con 0x7f45bc102d30 2024-09-19T09:03:41.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.971+0000 7f45ba59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45bc102d30 0x7f45bc100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:41.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.971+0000 7f45ba59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45bc102d30 0x7f45bc100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:50466/0 (socket says 172.21.15.120:50466) 2024-09-19T09:03:41.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.971+0000 7f45ba59c640 1 -- 172.21.15.120:0/3691204134 learned_addr learned my addr 172.21.15.120:0/3691204134 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:42.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.971+0000 7f45bad9d640 1 --2- 172.21.15.120:0/3691204134 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45bc1082a0 0x7f45bc101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.971+0000 7f45b9d9b640 1 --2- 172.21.15.120:0/3691204134 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45bc1036e0 0x7f45bc100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.971+0000 7f45ba59c640 1 -- 172.21.15.120:0/3691204134 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45bc1036e0 msgr2=0x7f45bc100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:42.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.971+0000 7f45ba59c640 1 --2- 172.21.15.120:0/3691204134 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45bc1036e0 0x7f45bc100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.971+0000 7f45ba59c640 1 -- 172.21.15.120:0/3691204134 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45bc1082a0 msgr2=0x7f45bc101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:42.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.971+0000 7f45ba59c640 1 --2- 172.21.15.120:0/3691204134 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45bc1082a0 0x7f45bc101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.971+0000 7f45ba59c640 1 -- 172.21.15.120:0/3691204134 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45bc1a69c0 con 0x7f45bc102d30 2024-09-19T09:03:42.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.971+0000 7f45bad9d640 1 --2- 172.21.15.120:0/3691204134 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45bc1082a0 0x7f45bc101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:03:42.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.971+0000 7f45ba59c640 1 --2- 172.21.15.120:0/3691204134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45bc102d30 0x7f45bc100990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00c920 tx=0x7f45ac00cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:42.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.972+0000 7f45ab7fe640 1 -- 172.21.15.120:0/3691204134 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac007c70 con 0x7f45bc102d30 2024-09-19T09:03:42.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.972+0000 7f45ab7fe640 1 -- 172.21.15.120:0/3691204134 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45ac007e10 con 0x7f45bc102d30 2024-09-19T09:03:42.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.972+0000 7f45c08c9640 1 -- 172.21.15.120:0/3691204134 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45bc1a6cb0 con 0x7f45bc102d30 2024-09-19T09:03:42.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.972+0000 7f45ab7fe640 1 -- 172.21.15.120:0/3691204134 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac005260 con 0x7f45bc102d30 2024-09-19T09:03:42.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.972+0000 7f45c08c9640 1 -- 172.21.15.120:0/3691204134 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45bc1a71c0 con 0x7f45bc102d30 2024-09-19T09:03:42.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.975+0000 7f45ab7fe640 1 -- 172.21.15.120:0/3691204134 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f45ac005400 con 0x7f45bc102d30 2024-09-19T09:03:42.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.975+0000 7f45c2ca2740 1 Processor -- start 2024-09-19T09:03:42.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.975+0000 7f45c2ca2740 1 -- start start 2024-09-19T09:03:42.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.976+0000 7f45c2ca2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561dc6dae350 0x561dc6dac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.976+0000 7f45c2ca2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dc6dacde0 0x561dc6dad220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.976+0000 7f45c2ca2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561dc6db7b80 0x561dc6db9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.976+0000 7f45c2ca2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561dc6d0a2f0 con 0x561dc6db7b80 2024-09-19T09:03:42.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.976+0000 7f45c2ca2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561dc6cb1cf0 con 0x561dc6dae350 2024-09-19T09:03:42.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.976+0000 7f45c2ca2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561dc6d062b0 con 0x561dc6dacde0 2024-09-19T09:03:42.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.976+0000 7f45b9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561dc6db7b80 0x561dc6db9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.976+0000 7f45b9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561dc6db7b80 0x561dc6db9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51462/0 (socket says 172.21.15.120:51462) 2024-09-19T09:03:42.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.976+0000 7f45b9d9b640 1 -- 172.21.15.120:0/3489933167 learned_addr learned my addr 172.21.15.120:0/3489933167 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:42.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.976+0000 7f45ab7fe640 1 --2- 172.21.15.120:0/3691204134 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f45a40a3350 0x7f45a40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.976+0000 7f45ba59c640 1 --2- 172.21.15.120:0/3489933167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dc6dacde0 0x561dc6dad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.976+0000 7f45b9d9b640 1 -- 172.21.15.120:0/3489933167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dc6dacde0 msgr2=0x561dc6dad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:42.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.976+0000 7f45b9d9b640 1 --2- 172.21.15.120:0/3489933167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dc6dacde0 0x561dc6dad220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.976+0000 7f45ab7fe640 1 -- 172.21.15.120:0/3691204134 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f45ac0cfb80 con 0x7f45bc102d30 2024-09-19T09:03:42.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.976+0000 7f45b9d9b640 1 -- 172.21.15.120:0/3489933167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561dc6dae350 msgr2=0x561dc6dac8a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:03:42.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.976+0000 7f45b9d9b640 1 --2- 172.21.15.120:0/3489933167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561dc6dae350 0x561dc6dac8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.976+0000 7f45b9d9b640 1 -- 172.21.15.120:0/3489933167 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dc6c963a0 con 0x561dc6db7b80 2024-09-19T09:03:42.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.977+0000 7f45ba59c640 1 --2- 172.21.15.120:0/3489933167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dc6dacde0 0x561dc6dad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:03:42.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.977+0000 7f45bad9d640 1 --2- 172.21.15.120:0/3691204134 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f45a40a3350 0x7f45a40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.977+0000 7f45b9d9b640 1 --2- 172.21.15.120:0/3489933167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561dc6db7b80 0x561dc6db9f70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009900 tx=0x7f45b0032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986482c47bfac229 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:42.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.977+0000 7f45a8ff9640 1 -- 172.21.15.120:0/3489933167 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b0040070 con 0x561dc6db7b80 2024-09-19T09:03:42.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.977+0000 7f45a8ff9640 1 -- 172.21.15.120:0/3489933167 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45b00317b0 con 0x561dc6db7b80 2024-09-19T09:03:42.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.977+0000 7f45a8ff9640 1 -- 172.21.15.120:0/3489933167 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b0035040 con 0x561dc6db7b80 2024-09-19T09:03:42.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.978+0000 7f45bad9d640 1 --2- 172.21.15.120:0/3691204134 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f45a40a3350 0x7f45a40a5810 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f45b4005fd0 tx=0x7f45b4009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:42.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.978+0000 7f45c2ca2740 1 -- 172.21.15.120:0/3489933167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561dc6db7b80 msgr2=0x561dc6db9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:42.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.978+0000 7f45c2ca2740 1 --2- 172.21.15.120:0/3489933167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561dc6db7b80 0x561dc6db9f70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009900 tx=0x7f45b0032cb0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.979+0000 7f45c2ca2740 1 -- 172.21.15.120:0/3489933167 shutdown_connections 2024-09-19T09:03:42.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.979+0000 7f45c2ca2740 1 --2- 172.21.15.120:0/3489933167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561dc6db7b80 0x561dc6db9f70 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.979+0000 7f45c2ca2740 1 --2- 172.21.15.120:0/3489933167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dc6dacde0 0x561dc6dad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.979+0000 7f45c2ca2740 1 --2- 172.21.15.120:0/3489933167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561dc6dae350 0x561dc6dac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.979+0000 7f45c2ca2740 1 -- 172.21.15.120:0/3489933167 >> 172.21.15.120:0/3489933167 conn(0x561dc6d6d310 msgr2=0x561dc6da9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:42.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.979+0000 7f45c2ca2740 1 -- 172.21.15.120:0/3489933167 shutdown_connections 2024-09-19T09:03:42.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.979+0000 7f45c2ca2740 1 -- 172.21.15.120:0/3489933167 wait complete. 2024-09-19T09:03:42.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.980+0000 7f45c2ca2740 1 -- start start 2024-09-19T09:03:42.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.980+0000 7f45c2ca2740 1 Processor -- start 2024-09-19T09:03:42.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.981+0000 7f45c2ca2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561dc6dacde0 0x561dc6db74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.981+0000 7f45c2ca2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dc6dae350 0x561dc6db4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.981+0000 7f45c2ca2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561dc6db7b80 0x561dc6db4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.981+0000 7f45c2ca2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561dc6dba820 con 0x561dc6db7b80 2024-09-19T09:03:42.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.981+0000 7f45c2ca2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561dc6cc5060 con 0x561dc6dacde0 2024-09-19T09:03:42.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.981+0000 7f45c2ca2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561dc6dba9a0 con 0x561dc6dae350 2024-09-19T09:03:42.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.981+0000 7f45bad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561dc6db7b80 0x561dc6db4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.981+0000 7f45ba59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561dc6dacde0 0x561dc6db74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.981+0000 7f45bad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561dc6db7b80 0x561dc6db4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51470/0 (socket says 172.21.15.120:51470) 2024-09-19T09:03:42.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.981+0000 7f45bad9d640 1 -- 172.21.15.120:0/3915618051 learned_addr learned my addr 172.21.15.120:0/3915618051 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:42.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.981+0000 7f45c2ca2740 10 client.0 _ll_register_callbacks cb 0x561dc6a188c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:42.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.981+0000 7f45b9d9b640 1 --2- 172.21.15.120:0/3915618051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dc6dae350 0x561dc6db4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.982+0000 7f45ba59c640 1 -- 172.21.15.120:0/3915618051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dc6dae350 msgr2=0x561dc6db4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:42.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.982+0000 7f45ba59c640 1 --2- 172.21.15.120:0/3915618051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dc6dae350 0x561dc6db4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.982+0000 7f45ba59c640 1 -- 172.21.15.120:0/3915618051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561dc6db7b80 msgr2=0x561dc6db4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:42.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.982+0000 7f45ba59c640 1 --2- 172.21.15.120:0/3915618051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561dc6db7b80 0x561dc6db4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.982+0000 7f45ba59c640 1 -- 172.21.15.120:0/3915618051 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dc6a7b3c0 con 0x561dc6dacde0 2024-09-19T09:03:42.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.982+0000 7f45b9d9b640 1 --2- 172.21.15.120:0/3915618051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dc6dae350 0x561dc6db4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:03:42.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.982+0000 7f45bad9d640 1 --2- 172.21.15.120:0/3915618051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561dc6db7b80 0x561dc6db4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:03:42.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.982+0000 7f45ba59c640 1 --2- 172.21.15.120:0/3915618051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561dc6dacde0 0x561dc6db74b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f45ac0cd4f0 tx=0x7f45ac01def0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8028eb71fc8438c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:42.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.982+0000 7f45a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:03:42.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.982+0000 7f45a1ffb640 1 -- 172.21.15.120:0/3915618051 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac09c700 con 0x561dc6dacde0 2024-09-19T09:03:42.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.982+0000 7f45c2ca2740 1 -- 172.21.15.120:0/3915618051 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45b00317b0 con 0x561dc6dacde0 2024-09-19T09:03:42.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.982+0000 7f45a1ffb640 1 -- 172.21.15.120:0/3915618051 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f45ac00fb30 con 0x561dc6dacde0 2024-09-19T09:03:42.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.983+0000 7f45a1ffb640 1 -- 172.21.15.120:0/3915618051 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f45ac09c000 con 0x561dc6dacde0 2024-09-19T09:03:42.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.983+0000 7f45c2ca2740 10 client.25064 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:42.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.983+0000 7f45c2ca2740 1 -- 172.21.15.120:0/3915618051 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561dc6dc6240 con 0x561dc6dacde0 2024-09-19T09:03:42.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.983+0000 7f45a1ffb640 1 -- 172.21.15.120:0/3915618051 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45ac0a03b0 con 0x561dc6dacde0 2024-09-19T09:03:42.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.983+0000 7f45a1ffb640 1 -- 172.21.15.120:0/3915618051 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f45ac0b8070 con 0x561dc6dacde0 2024-09-19T09:03:42.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.983+0000 7f45c2ca2740 10 client.25064 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:42.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.983+0000 7f45c2ca2740 10 client.25064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:03:42.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.983+0000 7f45c2ca2740 20 client.25064 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:42.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.983+0000 7f45c2ca2740 1 --2- 172.21.15.120:0/3915618051 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561dc6dc7a80 0x561dc6dc9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.983+0000 7f45c2ca2740 4 client.25064 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:03:42.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.983+0000 7f45c2ca2740 1 -- 172.21.15.120:0/3915618051 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f45b00317b0 con 0x561dc6dc7a80 2024-09-19T09:03:42.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.984+0000 7f45b9d9b640 1 --2- 172.21.15.120:0/3915618051 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561dc6dc7a80 0x561dc6dc9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.984+0000 7f45b9d9b640 1 --2- 172.21.15.120:0/3915618051 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561dc6dc7a80 0x561dc6dc9ed0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a52f4b15f31aacfd server_cookie=c656dcefc82f934 in_seq=0 out_seq=0 2024-09-19T09:03:42.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.984+0000 7f45a1ffb640 10 client.25064 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:42.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.985+0000 7f45a1ffb640 1 -- 172.21.15.120:0/3915618051 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f45b00317b0 con 0x561dc6dc7a80 2024-09-19T09:03:42.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.985+0000 7f45a1ffb640 10 client.25064 handle_command_reply: tid=0 2024-09-19T09:03:42.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.985+0000 7f45c2ca2740 1 client.25064 shutdown 2024-09-19T09:03:42.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.985+0000 7f45c2ca2740 1 -- 172.21.15.120:0/3915618051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561dc6dacde0 msgr2=0x561dc6db74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:42.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.985+0000 7f45c2ca2740 1 --2- 172.21.15.120:0/3915618051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561dc6dacde0 0x561dc6db74b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f45ac0cd4f0 tx=0x7f45ac01def0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.985+0000 7f45c2ca2740 1 -- 172.21.15.120:0/3915618051 shutdown_connections 2024-09-19T09:03:42.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.985+0000 7f45c2ca2740 1 --2- 172.21.15.120:0/3915618051 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561dc6dc7a80 0x561dc6dc9ed0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.985+0000 7f45c2ca2740 1 --2- 172.21.15.120:0/3915618051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561dc6db7b80 0x561dc6db4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.985+0000 7f45c2ca2740 1 --2- 172.21.15.120:0/3915618051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dc6dae350 0x561dc6db4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.985+0000 7f45c2ca2740 1 --2- 172.21.15.120:0/3915618051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561dc6dacde0 0x561dc6db74b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.985+0000 7f45c2ca2740 1 -- 172.21.15.120:0/3915618051 >> 172.21.15.120:0/3915618051 conn(0x561dc6d6d310 msgr2=0x561dc6daf170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:42.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.986+0000 7f45a1ffb640 0 client.25064 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:42.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.986+0000 7f45c2ca2740 1 -- 172.21.15.120:0/3915618051 shutdown_connections 2024-09-19T09:03:42.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.986+0000 7f45c2ca2740 1 -- 172.21.15.120:0/3915618051 wait complete. 2024-09-19T09:03:42.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:41.986+0000 7f45c2ca2740 20 client.25064 trim_cache size 0 max 16384 2024-09-19T09:03:42.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.011+0000 7f45c2ca2740 1 Processor -- start 2024-09-19T09:03:42.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.011+0000 7f45c2ca2740 1 -- start start 2024-09-19T09:03:42.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.011+0000 7f45c2ca2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dc6d6d310 0x561dc6dc3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.011+0000 7f45c2ca2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561dc6dacde0 0x561dc6dc9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.011+0000 7f45b9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561dc6dacde0 0x561dc6dc9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.012+0000 7f45b9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561dc6dacde0 0x561dc6dc9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51474/0 (socket says 172.21.15.120:51474) 2024-09-19T09:03:42.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.012+0000 7f45b9d9b640 1 -- 172.21.15.120:0/1994707148 learned_addr learned my addr 172.21.15.120:0/1994707148 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:42.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.012+0000 7f45c2ca2740 1 --2- 172.21.15.120:0/1994707148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561dc6dae350 0x561dc6dcf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.012+0000 7f45ba59c640 1 --2- 172.21.15.120:0/1994707148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dc6d6d310 0x561dc6dc3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.012+0000 7f45c2ca2740 1 -- 172.21.15.120:0/1994707148 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc1ab3e0 con 0x561dc6dacde0 2024-09-19T09:03:42.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.012+0000 7f45c2ca2740 1 -- 172.21.15.120:0/1994707148 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc1acf20 con 0x561dc6dae350 2024-09-19T09:03:42.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.012+0000 7f45c2ca2740 1 -- 172.21.15.120:0/1994707148 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc1abab0 con 0x561dc6d6d310 2024-09-19T09:03:42.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.012+0000 7f45ba59c640 1 -- 172.21.15.120:0/1994707148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561dc6dae350 msgr2=0x561dc6dcf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:42.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.012+0000 7f45ba59c640 1 --2- 172.21.15.120:0/1994707148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561dc6dae350 0x561dc6dcf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.012+0000 7f45ba59c640 1 -- 172.21.15.120:0/1994707148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561dc6dacde0 msgr2=0x561dc6dc9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:42.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.012+0000 7f45ba59c640 1 --2- 172.21.15.120:0/1994707148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561dc6dacde0 0x561dc6dc9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.012+0000 7f45ba59c640 1 -- 172.21.15.120:0/1994707148 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dc6db2cb0 con 0x561dc6d6d310 2024-09-19T09:03:42.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.012+0000 7f45b9d9b640 1 --2- 172.21.15.120:0/1994707148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561dc6dacde0 0x561dc6dc9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:03:42.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.012+0000 7f45ba59c640 1 --2- 172.21.15.120:0/1994707148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dc6d6d310 0x561dc6dc3b20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f45ac007590 tx=0x7f45ac0057b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43bee012c1d788ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:42.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.013+0000 7f45a3fff640 1 -- 172.21.15.120:0/1994707148 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac09c700 con 0x561dc6d6d310 2024-09-19T09:03:42.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.013+0000 7f45a3fff640 1 -- 172.21.15.120:0/1994707148 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45ac05a070 con 0x561dc6d6d310 2024-09-19T09:03:42.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.013+0000 7f45a3fff640 1 -- 172.21.15.120:0/1994707148 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac013e50 con 0x561dc6d6d310 2024-09-19T09:03:42.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.014+0000 7f45c2ca2740 1 -- 172.21.15.120:0/1994707148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dc6d6d310 msgr2=0x561dc6dc3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:42.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.014+0000 7f45c2ca2740 1 --2- 172.21.15.120:0/1994707148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dc6d6d310 0x561dc6dc3b20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f45ac007590 tx=0x7f45ac0057b0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.014+0000 7f45c2ca2740 1 -- 172.21.15.120:0/1994707148 shutdown_connections 2024-09-19T09:03:42.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.014+0000 7f45c2ca2740 1 --2- 172.21.15.120:0/1994707148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561dc6dae350 0x561dc6dcf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.014+0000 7f45c2ca2740 1 --2- 172.21.15.120:0/1994707148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561dc6dacde0 0x561dc6dc9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.014+0000 7f45c2ca2740 1 --2- 172.21.15.120:0/1994707148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dc6d6d310 0x561dc6dc3b20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.014+0000 7f45c2ca2740 1 -- 172.21.15.120:0/1994707148 >> 172.21.15.120:0/1994707148 conn(0x561dc6db4e00 msgr2=0x561dc6db3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:42.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.014+0000 7f45c2ca2740 1 -- 172.21.15.120:0/1994707148 shutdown_connections 2024-09-19T09:03:42.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.014+0000 7f45c2ca2740 1 -- 172.21.15.120:0/1994707148 wait complete. 2024-09-19T09:03:42.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.015+0000 7f45c2ca2740 1 -- start start 2024-09-19T09:03:42.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.015+0000 7f45c2ca2740 1 Processor -- start 2024-09-19T09:03:42.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.015+0000 7f45c2ca2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561dc6d6d310 0x561dc6dc0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.015+0000 7f45c2ca2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dc6dacde0 0x561dc6db1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.016+0000 7f45ba59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561dc6d6d310 0x561dc6dc0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.016+0000 7f45ba59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561dc6d6d310 0x561dc6dc0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51476/0 (socket says 172.21.15.120:51476) 2024-09-19T09:03:42.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.016+0000 7f45ba59c640 1 -- 172.21.15.120:0/1965368441 learned_addr learned my addr 172.21.15.120:0/1965368441 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:42.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.016+0000 7f45c2ca2740 1 --2- 172.21.15.120:0/1965368441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561dc6dae350 0x561dc6db17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.016+0000 7f45c2ca2740 1 -- 172.21.15.120:0/1965368441 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc1ac850 con 0x561dc6d6d310 2024-09-19T09:03:42.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.016+0000 7f45c2ca2740 1 -- 172.21.15.120:0/1965368441 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc1ac180 con 0x561dc6dae350 2024-09-19T09:03:42.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.016+0000 7f45b9d9b640 1 --2- 172.21.15.120:0/1965368441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dc6dacde0 0x561dc6db1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.016+0000 7f45c2ca2740 1 -- 172.21.15.120:0/1965368441 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc1a93c0 con 0x561dc6dacde0 2024-09-19T09:03:42.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.016+0000 7f45bad9d640 1 --2- 172.21.15.120:0/1965368441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561dc6dae350 0x561dc6db17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.017+0000 7f45ba59c640 1 -- 172.21.15.120:0/1965368441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dc6dacde0 msgr2=0x561dc6db1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:42.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.017+0000 7f45ba59c640 1 --2- 172.21.15.120:0/1965368441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dc6dacde0 0x561dc6db1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.017+0000 7f45ba59c640 1 -- 172.21.15.120:0/1965368441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561dc6dae350 msgr2=0x561dc6db17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:42.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.017+0000 7f45ba59c640 1 --2- 172.21.15.120:0/1965368441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561dc6dae350 0x561dc6db17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.017+0000 7f45ba59c640 1 -- 172.21.15.120:0/1965368441 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dc6db1df0 con 0x561dc6d6d310 2024-09-19T09:03:42.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.017+0000 7f45c2ca2740 10 client.0 _ll_register_callbacks cb 0x7f45ac09bea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:42.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.017+0000 7f45c2ca2740 1 -- 172.21.15.120:0/1965368441 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45ac05a070 con 0x561dc6d6d310 2024-09-19T09:03:42.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.018+0000 7f45ba59c640 1 --2- 172.21.15.120:0/1965368441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561dc6d6d310 0x561dc6dc0230 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f45ac098300 tx=0x7f45ac0a4df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4540e6c3bada0ca5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:42.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.018+0000 7f45a1ffb640 10 client.15645 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:42.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.018+0000 7f45a1ffb640 1 -- 172.21.15.120:0/1965368441 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac013df0 con 0x561dc6d6d310 2024-09-19T09:03:42.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.018+0000 7f45a1ffb640 1 -- 172.21.15.120:0/1965368441 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f45ac096b70 con 0x561dc6d6d310 2024-09-19T09:03:42.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.018+0000 7f45a1ffb640 1 -- 172.21.15.120:0/1965368441 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f45ac096d90 con 0x561dc6d6d310 2024-09-19T09:03:42.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.019+0000 7f45c2ca2740 10 client.15645 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:42.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.019+0000 7f45c2ca2740 1 -- 172.21.15.120:0/1965368441 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({fsmap=28}) -- 0x561dc6e32280 con 0x561dc6d6d310 2024-09-19T09:03:42.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.019+0000 7f45a1ffb640 1 -- 172.21.15.120:0/1965368441 <== mon.0 v2:172.21.15.120:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f45ac0b8050 con 0x561dc6d6d310 2024-09-19T09:03:42.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.020+0000 7f45c2ca2740 10 client.15645 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:42.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.020+0000 7f45c2ca2740 10 client.15645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:03:42.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.020+0000 7f45c2ca2740 20 client.15645 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:42.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.020+0000 7f45c2ca2740 1 --2- 172.21.15.120:0/1965368441 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561dc6db7b80 0x561dc6e34750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.020+0000 7f45c2ca2740 4 client.15645 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:03:42.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.020+0000 7f45c2ca2740 1 -- 172.21.15.120:0/1965368441 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561dc6dca260 con 0x561dc6db7b80 2024-09-19T09:03:42.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.021+0000 7f45b9d9b640 1 --2- 172.21.15.120:0/1965368441 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561dc6db7b80 0x561dc6e34750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.021+0000 7f45b9d9b640 1 --2- 172.21.15.120:0/1965368441 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561dc6db7b80 0x561dc6e34750 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eda2cb2cff24f1b server_cookie=c073daf96a8e0cdf in_seq=0 out_seq=0 2024-09-19T09:03:42.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.021+0000 7f45a1ffb640 10 client.15645 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:42.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.022+0000 7f45a1ffb640 1 -- 172.21.15.120:0/1965368441 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561dc6dca260 con 0x561dc6db7b80 2024-09-19T09:03:42.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.022+0000 7f45a1ffb640 10 client.15645 handle_command_reply: tid=0 2024-09-19T09:03:42.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.022+0000 7f45c2ca2740 1 client.15645 shutdown 2024-09-19T09:03:42.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.022+0000 7f45c2ca2740 1 -- 172.21.15.120:0/1965368441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561dc6d6d310 msgr2=0x561dc6dc0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:42.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.022+0000 7f45c2ca2740 1 --2- 172.21.15.120:0/1965368441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561dc6d6d310 0x561dc6dc0230 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f45ac098300 tx=0x7f45ac0a4df0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.023+0000 7f45c2ca2740 1 -- 172.21.15.120:0/1965368441 shutdown_connections 2024-09-19T09:03:42.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.023+0000 7f45c2ca2740 1 --2- 172.21.15.120:0/1965368441 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561dc6db7b80 0x561dc6e34750 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.023+0000 7f45c2ca2740 1 --2- 172.21.15.120:0/1965368441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561dc6dae350 0x561dc6db17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.023+0000 7f45a1ffb640 0 client.15645 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:42.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.023+0000 7f45c2ca2740 1 --2- 172.21.15.120:0/1965368441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dc6dacde0 0x561dc6db1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.024+0000 7f45c2ca2740 1 --2- 172.21.15.120:0/1965368441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561dc6d6d310 0x561dc6dc0230 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.024+0000 7f45c2ca2740 1 -- 172.21.15.120:0/1965368441 >> 172.21.15.120:0/1965368441 conn(0x561dc6db4e00 msgr2=0x561dc6dae920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:42.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.024+0000 7f45c2ca2740 1 -- 172.21.15.120:0/1965368441 shutdown_connections 2024-09-19T09:03:42.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.024+0000 7f45c2ca2740 1 -- 172.21.15.120:0/1965368441 wait complete. 2024-09-19T09:03:42.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.024+0000 7f45c2ca2740 20 client.15645 trim_cache size 0 max 16384 2024-09-19T09:03:42.025 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:03:42.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.025+0000 7f45c08c9640 1 -- 172.21.15.120:0/3691204134 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f45a40a3350 msgr2=0x7f45a40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:42.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.025+0000 7f45c08c9640 1 --2- 172.21.15.120:0/3691204134 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f45a40a3350 0x7f45a40a5810 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f45b4005fd0 tx=0x7f45b4009600 comp rx=0 tx=0).stop 2024-09-19T09:03:42.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.025+0000 7f45c08c9640 1 -- 172.21.15.120:0/3691204134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45bc102d30 msgr2=0x7f45bc100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:42.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.025+0000 7f45c08c9640 1 --2- 172.21.15.120:0/3691204134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45bc102d30 0x7f45bc100990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00c920 tx=0x7f45ac00cde0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.025+0000 7f45c08c9640 1 -- 172.21.15.120:0/3691204134 shutdown_connections 2024-09-19T09:03:42.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.025+0000 7f45c08c9640 1 --2- 172.21.15.120:0/3691204134 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f45a40a3350 0x7f45a40a5810 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.025+0000 7f45c08c9640 1 --2- 172.21.15.120:0/3691204134 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45bc1082a0 0x7f45bc101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.025+0000 7f45c08c9640 1 --2- 172.21.15.120:0/3691204134 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45bc1036e0 0x7f45bc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.025+0000 7f45c08c9640 1 --2- 172.21.15.120:0/3691204134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45bc102d30 0x7f45bc100990 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.025+0000 7f45c08c9640 1 -- 172.21.15.120:0/3691204134 >> 172.21.15.120:0/3691204134 conn(0x7f45bc0fc7d0 msgr2=0x7f45bc105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:42.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.026+0000 7f45c08c9640 1 -- 172.21.15.120:0/3691204134 shutdown_connections 2024-09-19T09:03:42.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:42.026+0000 7f45c08c9640 1 -- 172.21.15.120:0/3691204134 wait complete. 2024-09-19T09:03:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.069+0000 7fdbfe331640 1 -- 172.21.15.120:0/2300955581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbf8101d90 msgr2=0x7fdbf8108620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.069+0000 7fdbfe331640 1 --2- 172.21.15.120:0/2300955581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbf8101d90 0x7fdbf8108620 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0009ca0 tx=0x7fdbf002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.069+0000 7fdbfe331640 1 -- 172.21.15.120:0/2300955581 shutdown_connections 2024-09-19T09:03:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.069+0000 7fdbfe331640 1 --2- 172.21.15.120:0/2300955581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbf8101d90 0x7fdbf8108620 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.069+0000 7fdbfe331640 1 --2- 172.21.15.120:0/2300955581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbf81013d0 0x7fdbf8101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.069+0000 7fdbfe331640 1 --2- 172.21.15.120:0/2300955581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbf81001e0 0x7fdbf81005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.069+0000 7fdbfe331640 1 -- 172.21.15.120:0/2300955581 >> 172.21.15.120:0/2300955581 conn(0x7fdbf80fb940 msgr2=0x7fdbf80fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.069+0000 7fdbfe331640 1 -- 172.21.15.120:0/2300955581 shutdown_connections 2024-09-19T09:03:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.069+0000 7fdbfe331640 1 -- 172.21.15.120:0/2300955581 wait complete. 2024-09-19T09:03:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.070+0000 7fdbfe331640 1 Processor -- start 2024-09-19T09:03:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.070+0000 7fdbfe331640 1 -- start start 2024-09-19T09:03:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.070+0000 7fdbfe331640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbf81001e0 0x7fdbf8195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.070+0000 7fdbfe331640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbf81013d0 0x7fdbf8196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.070+0000 7fdbfe331640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbf8101d90 0x7fdbf819d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.070+0000 7fdbfe331640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf810b660 con 0x7fdbf8101d90 2024-09-19T09:03:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.070+0000 7fdbfe331640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf810b4e0 con 0x7fdbf81013d0 2024-09-19T09:03:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.070+0000 7fdbfe331640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf810b7e0 con 0x7fdbf81001e0 2024-09-19T09:03:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.070+0000 7fdbfc8ce640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbf8101d90 0x7fdbf819d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.070+0000 7fdbfc8ce640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbf8101d90 0x7fdbf819d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51504/0 (socket says 172.21.15.120:51504) 2024-09-19T09:03:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.070+0000 7fdbfc8ce640 1 -- 172.21.15.120:0/1297795112 learned_addr learned my addr 172.21.15.120:0/1297795112 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.070+0000 7fdbf7fff640 1 --2- 172.21.15.120:0/1297795112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbf81001e0 0x7fdbf8195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.070+0000 7fdbf77fe640 1 --2- 172.21.15.120:0/1297795112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbf81013d0 0x7fdbf8196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.071+0000 7fdbf77fe640 1 -- 172.21.15.120:0/1297795112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbf81001e0 msgr2=0x7fdbf8195b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.071+0000 7fdbf77fe640 1 --2- 172.21.15.120:0/1297795112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbf81001e0 0x7fdbf8195b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.071+0000 7fdbf77fe640 1 -- 172.21.15.120:0/1297795112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbf8101d90 msgr2=0x7fdbf819d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.071+0000 7fdbf77fe640 1 --2- 172.21.15.120:0/1297795112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbf8101d90 0x7fdbf819d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.071+0000 7fdbf77fe640 1 -- 172.21.15.120:0/1297795112 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf819d770 con 0x7fdbf81013d0 2024-09-19T09:03:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.071+0000 7fdbf7fff640 1 --2- 172.21.15.120:0/1297795112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbf81001e0 0x7fdbf8195b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:03:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.071+0000 7fdbfc8ce640 1 --2- 172.21.15.120:0/1297795112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbf8101d90 0x7fdbf819d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:03:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.071+0000 7fdbf77fe640 1 --2- 172.21.15.120:0/1297795112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbf81013d0 0x7fdbf8196090 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00e9f0 tx=0x7fdbec00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.071+0000 7fdbf57fa640 1 -- 172.21.15.120:0/1297795112 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec00ccc0 con 0x7fdbf81013d0 2024-09-19T09:03:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.071+0000 7fdbfe331640 1 -- 172.21.15.120:0/1297795112 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf819da60 con 0x7fdbf81013d0 2024-09-19T09:03:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.071+0000 7fdbf57fa640 1 -- 172.21.15.120:0/1297795112 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbec005020 con 0x7fdbf81013d0 2024-09-19T09:03:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.071+0000 7fdbf57fa640 1 -- 172.21.15.120:0/1297795112 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec01c070 con 0x7fdbf81013d0 2024-09-19T09:03:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.072+0000 7fdbfe331640 1 -- 172.21.15.120:0/1297795112 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf819df50 con 0x7fdbf81013d0 2024-09-19T09:03:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.075+0000 7fdbf57fa640 1 -- 172.21.15.120:0/1297795112 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7fdbec005540 con 0x7fdbf81013d0 2024-09-19T09:03:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.075+0000 7fdc010ca740 1 Processor -- start 2024-09-19T09:03:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.075+0000 7fdc010ca740 1 -- start start 2024-09-19T09:03:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.075+0000 7fdc010ca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db75f5bdb0 0x55db75f5c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.075+0000 7fdc010ca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db75f5c6f0 0x55db75f611b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.075+0000 7fdbf7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db75f5c6f0 0x55db75f611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.075+0000 7fdbf7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db75f5c6f0 0x55db75f611b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59748/0 (socket says 172.21.15.120:59748) 2024-09-19T09:03:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.075+0000 7fdc010ca740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db75f61780 0x55db75f63c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.075+0000 7fdbf7fff640 1 -- 172.21.15.120:0/1945234500 learned_addr learned my addr 172.21.15.120:0/1945234500 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.075+0000 7fdc010ca740 1 -- 172.21.15.120:0/1945234500 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55db75ebb2f0 con 0x55db75f5bdb0 2024-09-19T09:03:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.075+0000 7fdc010ca740 1 -- 172.21.15.120:0/1945234500 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55db75e63270 con 0x55db75f5c6f0 2024-09-19T09:03:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.075+0000 7fdc010ca740 1 -- 172.21.15.120:0/1945234500 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55db75eb72b0 con 0x55db75f61780 2024-09-19T09:03:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.075+0000 7fdbfc8ce640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db75f5bdb0 0x55db75f5c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.076+0000 7fdbf77fe640 1 --2- 172.21.15.120:0/1945234500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db75f61780 0x55db75f63c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.076+0000 7fdbf57fa640 1 --2- 172.21.15.120:0/1297795112 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdbe00a3410 0x7fdbe00a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.076+0000 7fdbfc8ce640 1 -- 172.21.15.120:0/1945234500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db75f61780 msgr2=0x55db75f63c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.076+0000 7fdbfc8ce640 1 --2- 172.21.15.120:0/1945234500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db75f61780 0x55db75f63c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.076+0000 7fdbfc8ce640 1 -- 172.21.15.120:0/1945234500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db75f5c6f0 msgr2=0x55db75f611b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.076+0000 7fdbfc8ce640 1 --2- 172.21.15.120:0/1945234500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db75f5c6f0 0x55db75f611b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.076+0000 7fdbfc8ce640 1 -- 172.21.15.120:0/1945234500 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db75e62340 con 0x55db75f5bdb0 2024-09-19T09:03:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.076+0000 7fdbf57fa640 1 -- 172.21.15.120:0/1297795112 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fdbec0cf9d0 con 0x7fdbf81013d0 2024-09-19T09:03:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.076+0000 7fdbfc8ce640 1 --2- 172.21.15.120:0/1945234500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db75f5bdb0 0x55db75f5c1b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0006010 tx=0x7fdbf0005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ac562575e868ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.076+0000 7fdbfc8ce640 1 --2- 172.21.15.120:0/1297795112 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdbe00a3410 0x7fdbe00a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.077+0000 7fdbfc8ce640 1 --2- 172.21.15.120:0/1297795112 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdbe00a3410 0x7fdbe00a58d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0045000 tx=0x7fdbf003d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.077+0000 7fdbde7fc640 1 -- 172.21.15.120:0/1945234500 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf0004710 con 0x55db75f5bdb0 2024-09-19T09:03:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.077+0000 7fdbde7fc640 1 -- 172.21.15.120:0/1945234500 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbf0036050 con 0x55db75f5bdb0 2024-09-19T09:03:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.077+0000 7fdbde7fc640 1 -- 172.21.15.120:0/1945234500 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf00305c0 con 0x55db75f5bdb0 2024-09-19T09:03:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.078+0000 7fdc010ca740 1 -- 172.21.15.120:0/1945234500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db75f5bdb0 msgr2=0x55db75f5c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.078+0000 7fdc010ca740 1 --2- 172.21.15.120:0/1945234500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db75f5bdb0 0x55db75f5c1b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0006010 tx=0x7fdbf0005fa0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.078+0000 7fdc010ca740 1 -- 172.21.15.120:0/1945234500 shutdown_connections 2024-09-19T09:03:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.078+0000 7fdc010ca740 1 --2- 172.21.15.120:0/1945234500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db75f61780 0x55db75f63c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.078+0000 7fdc010ca740 1 --2- 172.21.15.120:0/1945234500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db75f5c6f0 0x55db75f611b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.078+0000 7fdc010ca740 1 --2- 172.21.15.120:0/1945234500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db75f5bdb0 0x55db75f5c1b0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.078+0000 7fdc010ca740 1 -- 172.21.15.120:0/1945234500 >> 172.21.15.120:0/1945234500 conn(0x55db75f57420 msgr2=0x55db75f57820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.078+0000 7fdc010ca740 1 -- 172.21.15.120:0/1945234500 shutdown_connections 2024-09-19T09:03:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.078+0000 7fdc010ca740 1 -- 172.21.15.120:0/1945234500 wait complete. 2024-09-19T09:03:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.079+0000 7fdc010ca740 1 -- start start 2024-09-19T09:03:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.079+0000 7fdc010ca740 1 Processor -- start 2024-09-19T09:03:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.079+0000 7fdc010ca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db75f5bdb0 0x55db75f5ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.079+0000 7fdc010ca740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db75f5c6f0 0x55db75f5f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.079+0000 7fdc010ca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db75f61780 0x55db75f60190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.079+0000 7fdc010ca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55db75f52a50 con 0x55db75f5bdb0 2024-09-19T09:03:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.079+0000 7fdc010ca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55db75e44290 con 0x55db75f61780 2024-09-19T09:03:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.079+0000 7fdc010ca740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55db75f52bd0 con 0x55db75f5c6f0 2024-09-19T09:03:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.079+0000 7fdbf7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db75f5bdb0 0x55db75f5ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.079+0000 7fdc010ca740 10 client.0 _ll_register_callbacks cb 0x55db75e75140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.079+0000 7fdbf7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db75f5bdb0 0x55db75f5ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51520/0 (socket says 172.21.15.120:51520) 2024-09-19T09:03:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.079+0000 7fdbf7fff640 1 -- 172.21.15.120:0/3401103928 learned_addr learned my addr 172.21.15.120:0/3401103928 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.080+0000 7fdbfc8ce640 1 --2- 172.21.15.120:0/3401103928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db75f61780 0x55db75f60190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.080+0000 7fdbf77fe640 1 --2- 172.21.15.120:0/3401103928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db75f5c6f0 0x55db75f5f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.080+0000 7fdbf7fff640 1 -- 172.21.15.120:0/3401103928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db75f5c6f0 msgr2=0x55db75f5f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.080+0000 7fdbf7fff640 1 --2- 172.21.15.120:0/3401103928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db75f5c6f0 0x55db75f5f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.080+0000 7fdbf7fff640 1 -- 172.21.15.120:0/3401103928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db75f61780 msgr2=0x55db75f60190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.080+0000 7fdbf7fff640 1 --2- 172.21.15.120:0/3401103928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db75f61780 0x55db75f60190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.080+0000 7fdbf7fff640 1 -- 172.21.15.120:0/3401103928 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db75bce210 con 0x55db75f5bdb0 2024-09-19T09:03:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.080+0000 7fdbf7fff640 1 --2- 172.21.15.120:0/3401103928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db75f5bdb0 0x55db75f5ecb0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800adf0 tx=0x7fdbe800d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c63a3a3f18c013e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.080+0000 7fdbd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.080+0000 7fdbd77fe640 1 -- 172.21.15.120:0/3401103928 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe80207d0 con 0x55db75f5bdb0 2024-09-19T09:03:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.080+0000 7fdbd77fe640 1 -- 172.21.15.120:0/3401103928 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fdbe8020970 con 0x55db75f5bdb0 2024-09-19T09:03:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.080+0000 7fdc010ca740 1 -- 172.21.15.120:0/3401103928 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbf0036050 con 0x55db75f5bdb0 2024-09-19T09:03:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.080+0000 7fdbd77fe640 1 -- 172.21.15.120:0/3401103928 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdbe8020b70 con 0x55db75f5bdb0 2024-09-19T09:03:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.080+0000 7fdc010ca740 10 client.15657 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.080+0000 7fdc010ca740 1 -- 172.21.15.120:0/3401103928 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55db75c9a620 con 0x55db75f5bdb0 2024-09-19T09:03:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.081+0000 7fdbd77fe640 1 -- 172.21.15.120:0/3401103928 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbf0036050 con 0x55db75f5bdb0 2024-09-19T09:03:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.081+0000 7fdbd77fe640 1 -- 172.21.15.120:0/3401103928 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdbe800b040 con 0x55db75f5bdb0 2024-09-19T09:03:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.081+0000 7fdc010ca740 10 client.15657 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.081+0000 7fdc010ca740 10 client.15657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:03:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.081+0000 7fdc010ca740 20 client.15657 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.081+0000 7fdc010ca740 1 --2- 172.21.15.120:0/3401103928 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55db75f6d780 0x55db75f6fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.081+0000 7fdc010ca740 4 client.15657 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:03:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.081+0000 7fdc010ca740 1 -- 172.21.15.120:0/3401103928 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db75ec8a20 con 0x55db75f6d780 2024-09-19T09:03:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.081+0000 7fdbf77fe640 1 --2- 172.21.15.120:0/3401103928 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55db75f6d780 0x55db75f6fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.082+0000 7fdbd77fe640 10 client.15657 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.082+0000 7fdbf77fe640 1 --2- 172.21.15.120:0/3401103928 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55db75f6d780 0x55db75f6fbb0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d636eeb4080a050 server_cookie=693d73dc5e0f076f in_seq=0 out_seq=0 2024-09-19T09:03:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.083+0000 7fdbd77fe640 1 -- 172.21.15.120:0/3401103928 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55db75ec8a20 con 0x55db75f6d780 2024-09-19T09:03:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.083+0000 7fdbd77fe640 10 client.15657 handle_command_reply: tid=0 2024-09-19T09:03:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.083+0000 7fdc010ca740 1 client.15657 shutdown 2024-09-19T09:03:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.083+0000 7fdc010ca740 1 -- 172.21.15.120:0/3401103928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db75f5bdb0 msgr2=0x55db75f5ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.083+0000 7fdc010ca740 1 --2- 172.21.15.120:0/3401103928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db75f5bdb0 0x55db75f5ecb0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800adf0 tx=0x7fdbe800d7b0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.083+0000 7fdc010ca740 1 -- 172.21.15.120:0/3401103928 shutdown_connections 2024-09-19T09:03:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.083+0000 7fdc010ca740 1 --2- 172.21.15.120:0/3401103928 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55db75f6d780 0x55db75f6fbb0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.083+0000 7fdc010ca740 1 --2- 172.21.15.120:0/3401103928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db75f61780 0x55db75f60190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.083+0000 7fdc010ca740 1 --2- 172.21.15.120:0/3401103928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db75f5c6f0 0x55db75f5f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.083+0000 7fdc010ca740 1 --2- 172.21.15.120:0/3401103928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db75f5bdb0 0x55db75f5ecb0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.083+0000 7fdc010ca740 1 -- 172.21.15.120:0/3401103928 >> 172.21.15.120:0/3401103928 conn(0x55db75f57420 msgr2=0x55db75f521c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.083+0000 7fdbd77fe640 0 client.15657 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.083+0000 7fdc010ca740 1 -- 172.21.15.120:0/3401103928 shutdown_connections 2024-09-19T09:03:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.083+0000 7fdc010ca740 1 -- 172.21.15.120:0/3401103928 wait complete. 2024-09-19T09:03:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.084+0000 7fdc010ca740 20 client.15657 trim_cache size 0 max 16384 2024-09-19T09:03:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.111+0000 7fdc010ca740 1 Processor -- start 2024-09-19T09:03:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.111+0000 7fdc010ca740 1 -- start start 2024-09-19T09:03:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.111+0000 7fdc010ca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db75f5c6f0 0x55db75f5ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.111+0000 7fdc010ca740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db75f61780 0x55db75f5efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.111+0000 7fdc010ca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db75f57f10 0x55db75f583b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.111+0000 7fdc010ca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf81a6d30 con 0x55db75f57f10 2024-09-19T09:03:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.111+0000 7fdc010ca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf81a8870 con 0x55db75f5c6f0 2024-09-19T09:03:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.111+0000 7fdc010ca740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf81a7400 con 0x55db75f61780 2024-09-19T09:03:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.112+0000 7fdbfc8ce640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db75f57f10 0x55db75f583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.112+0000 7fdbf7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db75f5c6f0 0x55db75f5ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.112+0000 7fdbfc8ce640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db75f57f10 0x55db75f583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51532/0 (socket says 172.21.15.120:51532) 2024-09-19T09:03:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.112+0000 7fdbf7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db75f5c6f0 0x55db75f5ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59768/0 (socket says 172.21.15.120:59768) 2024-09-19T09:03:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.112+0000 7fdbfc8ce640 1 -- 172.21.15.120:0/2198109604 learned_addr learned my addr 172.21.15.120:0/2198109604 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.112+0000 7fdbf77fe640 1 --2- 172.21.15.120:0/2198109604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db75f61780 0x55db75f5efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.112+0000 7fdbf7fff640 1 -- 172.21.15.120:0/2198109604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db75f61780 msgr2=0x55db75f5efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.112+0000 7fdbf7fff640 1 --2- 172.21.15.120:0/2198109604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db75f61780 0x55db75f5efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.112+0000 7fdbf7fff640 1 -- 172.21.15.120:0/2198109604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db75f57f10 msgr2=0x55db75f583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.112+0000 7fdbf7fff640 1 --2- 172.21.15.120:0/2198109604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db75f57f10 0x55db75f583b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.112+0000 7fdbf7fff640 1 -- 172.21.15.120:0/2198109604 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db75f57240 con 0x55db75f5c6f0 2024-09-19T09:03:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.112+0000 7fdbf77fe640 1 --2- 172.21.15.120:0/2198109604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db75f61780 0x55db75f5efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:03:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.112+0000 7fdbfc8ce640 1 --2- 172.21.15.120:0/2198109604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db75f57f10 0x55db75f583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:03:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.112+0000 7fdbf7fff640 1 --2- 172.21.15.120:0/2198109604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db75f5c6f0 0x55db75f5ea60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdbe801f000 tx=0x7fdbe8005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52ff3f9d9b5757c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.112+0000 7fdbddffb640 1 -- 172.21.15.120:0/2198109604 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe800c070 con 0x55db75f5c6f0 2024-09-19T09:03:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.113+0000 7fdbddffb640 1 -- 172.21.15.120:0/2198109604 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbe8026070 con 0x55db75f5c6f0 2024-09-19T09:03:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.113+0000 7fdbddffb640 1 -- 172.21.15.120:0/2198109604 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8020990 con 0x55db75f5c6f0 2024-09-19T09:03:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.113+0000 7fdc010ca740 1 -- 172.21.15.120:0/2198109604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db75f5c6f0 msgr2=0x55db75f5ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.114+0000 7fdc010ca740 1 --2- 172.21.15.120:0/2198109604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db75f5c6f0 0x55db75f5ea60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdbe801f000 tx=0x7fdbe8005c50 comp rx=0 tx=0).stop 2024-09-19T09:03:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.114+0000 7fdc010ca740 1 -- 172.21.15.120:0/2198109604 shutdown_connections 2024-09-19T09:03:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.114+0000 7fdc010ca740 1 --2- 172.21.15.120:0/2198109604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db75f57f10 0x55db75f583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.114+0000 7fdc010ca740 1 --2- 172.21.15.120:0/2198109604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db75f61780 0x55db75f5efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.114+0000 7fdc010ca740 1 --2- 172.21.15.120:0/2198109604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db75f5c6f0 0x55db75f5ea60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.114+0000 7fdc010ca740 1 -- 172.21.15.120:0/2198109604 >> 172.21.15.120:0/2198109604 conn(0x55db75f57420 msgr2=0x55db75f57800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.114+0000 7fdc010ca740 1 -- 172.21.15.120:0/2198109604 shutdown_connections 2024-09-19T09:03:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.114+0000 7fdc010ca740 1 -- 172.21.15.120:0/2198109604 wait complete. 2024-09-19T09:03:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.114+0000 7fdc010ca740 1 -- start start 2024-09-19T09:03:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.114+0000 7fdc010ca740 1 Processor -- start 2024-09-19T09:03:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.115+0000 7fdc010ca740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db75f57f10 0x55db75f67700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.115+0000 7fdc010ca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db75f5c6f0 0x55db75f67c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.115+0000 7fdc010ca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db75f61780 0x55db75f68c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.115+0000 7fdbf77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db75f5c6f0 0x55db75f67c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.115+0000 7fdbf77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db75f5c6f0 0x55db75f67c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51544/0 (socket says 172.21.15.120:51544) 2024-09-19T09:03:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.115+0000 7fdbf77fe640 1 -- 172.21.15.120:0/609028375 learned_addr learned my addr 172.21.15.120:0/609028375 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.115+0000 7fdc010ca740 1 -- 172.21.15.120:0/609028375 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf81a81a0 con 0x55db75f5c6f0 2024-09-19T09:03:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.115+0000 7fdc010ca740 1 -- 172.21.15.120:0/609028375 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf81a7ad0 con 0x55db75f61780 2024-09-19T09:03:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.115+0000 7fdc010ca740 1 -- 172.21.15.120:0/609028375 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf81a4cf0 con 0x55db75f57f10 2024-09-19T09:03:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.115+0000 7fdbf7fff640 1 --2- 172.21.15.120:0/609028375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db75f57f10 0x55db75f67700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.115+0000 7fdc010ca740 10 client.0 _ll_register_callbacks cb 0x7fdbe800ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.115+0000 7fdbfc8ce640 1 --2- 172.21.15.120:0/609028375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db75f61780 0x55db75f68c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.115+0000 7fdbf7fff640 1 -- 172.21.15.120:0/609028375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db75f61780 msgr2=0x55db75f68c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.115+0000 7fdbf7fff640 1 --2- 172.21.15.120:0/609028375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db75f61780 0x55db75f68c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.115+0000 7fdbf7fff640 1 -- 172.21.15.120:0/609028375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db75f5c6f0 msgr2=0x55db75f67c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.115+0000 7fdbf7fff640 1 --2- 172.21.15.120:0/609028375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db75f5c6f0 0x55db75f67c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.115+0000 7fdbf7fff640 1 -- 172.21.15.120:0/609028375 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db75f69260 con 0x55db75f57f10 2024-09-19T09:03:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.115+0000 7fdbfc8ce640 1 --2- 172.21.15.120:0/609028375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db75f61780 0x55db75f68c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:03:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.115+0000 7fdbf77fe640 1 --2- 172.21.15.120:0/609028375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db75f5c6f0 0x55db75f67c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:03:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.116+0000 7fdbf7fff640 1 --2- 172.21.15.120:0/609028375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db75f57f10 0x55db75f67700 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8000c00 tx=0x7fdbe800ad80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7580cf56e15734ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.116+0000 7fdbd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:03:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.116+0000 7fdbd77fe640 1 -- 172.21.15.120:0/609028375 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8005110 con 0x55db75f57f10 2024-09-19T09:03:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.116+0000 7fdc010ca740 1 -- 172.21.15.120:0/609028375 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbe8026070 con 0x55db75f57f10 2024-09-19T09:03:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.116+0000 7fdbd77fe640 1 -- 172.21.15.120:0/609028375 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fdbe80052b0 con 0x55db75f57f10 2024-09-19T09:03:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.116+0000 7fdbd77fe640 1 -- 172.21.15.120:0/609028375 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdbe802a9a0 con 0x55db75f57f10 2024-09-19T09:03:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.116+0000 7fdc010ca740 10 client.25078 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.116+0000 7fdc010ca740 1 -- 172.21.15.120:0/609028375 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55db75fd86d0 con 0x55db75f57f10 2024-09-19T09:03:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.116+0000 7fdbd77fe640 1 -- 172.21.15.120:0/609028375 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbe8026070 con 0x55db75f57f10 2024-09-19T09:03:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.116+0000 7fdbd77fe640 1 -- 172.21.15.120:0/609028375 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdbe8047030 con 0x55db75f57f10 2024-09-19T09:03:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.117+0000 7fdc010ca740 10 client.25078 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.117+0000 7fdc010ca740 10 client.25078 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:03:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.117+0000 7fdc010ca740 20 client.25078 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.117+0000 7fdc010ca740 1 --2- 172.21.15.120:0/609028375 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55db75fd9080 0x55db75fdb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.117+0000 7fdc010ca740 4 client.25078 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:03:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.117+0000 7fdc010ca740 1 -- 172.21.15.120:0/609028375 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55db75ec2d70 con 0x55db75fd9080 2024-09-19T09:03:42.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.117+0000 7fdbf77fe640 1 --2- 172.21.15.120:0/609028375 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55db75fd9080 0x55db75fdb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:42.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.117+0000 7fdbd77fe640 10 client.25078 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:42.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.117+0000 7fdbf77fe640 1 --2- 172.21.15.120:0/609028375 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55db75fd9080 0x55db75fdb4a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c096aac4c4f34056 server_cookie=42df26e50569b863 in_seq=0 out_seq=0 2024-09-19T09:03:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.124+0000 7fdbd77fe640 1 -- 172.21.15.120:0/609028375 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55db75ec2d70 con 0x55db75fd9080 2024-09-19T09:03:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.124+0000 7fdbd77fe640 10 client.25078 handle_command_reply: tid=0 2024-09-19T09:03:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.125+0000 7fdc010ca740 1 client.25078 shutdown 2024-09-19T09:03:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.125+0000 7fdc010ca740 1 -- 172.21.15.120:0/609028375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db75f57f10 msgr2=0x55db75f67700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.125+0000 7fdc010ca740 1 --2- 172.21.15.120:0/609028375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db75f57f10 0x55db75f67700 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8000c00 tx=0x7fdbe800ad80 comp rx=0 tx=0).stop 2024-09-19T09:03:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.125+0000 7fdc010ca740 1 -- 172.21.15.120:0/609028375 shutdown_connections 2024-09-19T09:03:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.125+0000 7fdc010ca740 1 --2- 172.21.15.120:0/609028375 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55db75fd9080 0x55db75fdb4a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.125+0000 7fdc010ca740 1 --2- 172.21.15.120:0/609028375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db75f61780 0x55db75f68c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.125+0000 7fdbd77fe640 0 client.25078 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.125+0000 7fdc010ca740 1 --2- 172.21.15.120:0/609028375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db75f5c6f0 0x55db75f67c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.125+0000 7fdc010ca740 1 --2- 172.21.15.120:0/609028375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db75f57f10 0x55db75f67700 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.125+0000 7fdc010ca740 1 -- 172.21.15.120:0/609028375 >> 172.21.15.120:0/609028375 conn(0x55db75f57420 msgr2=0x55db75f5ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.125+0000 7fdc010ca740 1 -- 172.21.15.120:0/609028375 shutdown_connections 2024-09-19T09:03:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.125+0000 7fdc010ca740 1 -- 172.21.15.120:0/609028375 wait complete. 2024-09-19T09:03:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.125+0000 7fdc010ca740 20 client.25078 trim_cache size 0 max 16384 2024-09-19T09:03:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:03:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:03:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:03:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:03:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.126+0000 7fdbfe331640 1 -- 172.21.15.120:0/1297795112 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdbe00a3410 msgr2=0x7fdbe00a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.126+0000 7fdbfe331640 1 --2- 172.21.15.120:0/1297795112 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdbe00a3410 0x7fdbe00a58d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0045000 tx=0x7fdbf003d000 comp rx=0 tx=0).stop 2024-09-19T09:03:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.126+0000 7fdbfe331640 1 -- 172.21.15.120:0/1297795112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbf81013d0 msgr2=0x7fdbf8196090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.126+0000 7fdbfe331640 1 --2- 172.21.15.120:0/1297795112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbf81013d0 0x7fdbf8196090 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00e9f0 tx=0x7fdbec00eeb0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.126+0000 7fdbfe331640 1 -- 172.21.15.120:0/1297795112 shutdown_connections 2024-09-19T09:03:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.126+0000 7fdbfe331640 1 --2- 172.21.15.120:0/1297795112 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdbe00a3410 0x7fdbe00a58d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.126+0000 7fdbfe331640 1 --2- 172.21.15.120:0/1297795112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbf8101d90 0x7fdbf819d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.126+0000 7fdbfe331640 1 --2- 172.21.15.120:0/1297795112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbf81013d0 0x7fdbf8196090 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.126+0000 7fdbfe331640 1 --2- 172.21.15.120:0/1297795112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbf81001e0 0x7fdbf8195b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.126+0000 7fdbfe331640 1 -- 172.21.15.120:0/1297795112 >> 172.21.15.120:0/1297795112 conn(0x7fdbf80fb940 msgr2=0x7fdbf8104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.126+0000 7fdbfe331640 1 -- 172.21.15.120:0/1297795112 shutdown_connections 2024-09-19T09:03:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:42.126+0000 7fdbfe331640 1 -- 172.21.15.120:0/1297795112 wait complete. 2024-09-19T09:03:42.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:41 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1146357580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:03:42.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:41 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1146357580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:03:42.567 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db --set-id 27390328 --release --await' 2024-09-19T09:03:42.568 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db --set-id 27390328 --release --await 2024-09-19T09:03:42.724 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:42 smithi120 ceph-mon[31150]: pgmap v366: 97 pgs: 97 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-19T09:03:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:42 smithi196 ceph-mon[35150]: pgmap v366: 97 pgs: 97 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-19T09:03:43.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:42 smithi153 ceph-mon[35071]: pgmap v366: 97 pgs: 97 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-19T09:03:43.352 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:03:43.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.954+0000 7fd8238f8640 1 -- 172.21.15.120:0/4250330104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd81c100c80 msgr2=0x7fd81c110890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:43.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.954+0000 7fd8238f8640 1 --2- 172.21.15.120:0/4250330104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd81c100c80 0x7fd81c110890 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd814009ca0 tx=0x7fd81402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:03:43.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.954+0000 7fd8238f8640 1 -- 172.21.15.120:0/4250330104 shutdown_connections 2024-09-19T09:03:43.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.954+0000 7fd8238f8640 1 --2- 172.21.15.120:0/4250330104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd81c100c80 0x7fd81c110890 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:43.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.954+0000 7fd8238f8640 1 --2- 172.21.15.120:0/4250330104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd81c100100 0x7fd81c100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:43.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.954+0000 7fd8238f8640 1 --2- 172.21.15.120:0/4250330104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd81c102a50 0x7fd81c102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:43.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.954+0000 7fd8238f8640 1 -- 172.21.15.120:0/4250330104 >> 172.21.15.120:0/4250330104 conn(0x7fd81c077230 msgr2=0x7fd81c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:43.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.954+0000 7fd8238f8640 1 -- 172.21.15.120:0/4250330104 shutdown_connections 2024-09-19T09:03:43.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.954+0000 7fd8238f8640 1 -- 172.21.15.120:0/4250330104 wait complete. 2024-09-19T09:03:43.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.955+0000 7fd8238f8640 1 Processor -- start 2024-09-19T09:03:43.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.955+0000 7fd8238f8640 1 -- start start 2024-09-19T09:03:43.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.956+0000 7fd8238f8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd81c100100 0x7fd81c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:43.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.956+0000 7fd8238f8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd81c100c80 0x7fd81c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:43.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.956+0000 7fd8238f8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd81c102a50 0x7fd81c19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:43.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.956+0000 7fd8238f8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c114120 con 0x7fd81c102a50 2024-09-19T09:03:43.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.956+0000 7fd8238f8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c113fa0 con 0x7fd81c100100 2024-09-19T09:03:43.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.956+0000 7fd8238f8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c1142a0 con 0x7fd81c100c80 2024-09-19T09:03:43.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.956+0000 7fd821e95640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd81c102a50 0x7fd81c19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:43.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.956+0000 7fd821e95640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd81c102a50 0x7fd81c19eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51550/0 (socket says 172.21.15.120:51550) 2024-09-19T09:03:43.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.956+0000 7fd821e95640 1 -- 172.21.15.120:0/813749918 learned_addr learned my addr 172.21.15.120:0/813749918 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:43.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.956+0000 7fd821694640 1 --2- 172.21.15.120:0/813749918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd81c100100 0x7fd81c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:43.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.956+0000 7fd820e93640 1 --2- 172.21.15.120:0/813749918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd81c100c80 0x7fd81c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:43.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.956+0000 7fd821694640 1 -- 172.21.15.120:0/813749918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd81c100c80 msgr2=0x7fd81c19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:43.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.956+0000 7fd821694640 1 --2- 172.21.15.120:0/813749918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd81c100c80 0x7fd81c19ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:43.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.956+0000 7fd821694640 1 -- 172.21.15.120:0/813749918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd81c102a50 msgr2=0x7fd81c19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:43.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.956+0000 7fd821694640 1 --2- 172.21.15.120:0/813749918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd81c102a50 0x7fd81c19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:43.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.956+0000 7fd821694640 1 -- 172.21.15.120:0/813749918 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81c19f590 con 0x7fd81c100100 2024-09-19T09:03:43.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.957+0000 7fd821694640 1 --2- 172.21.15.120:0/813749918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd81c100100 0x7fd81c19a5e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd81800e9f0 tx=0x7fd81800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:43.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.957+0000 7fd8127fc640 1 -- 172.21.15.120:0/813749918 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81800ccc0 con 0x7fd81c100100 2024-09-19T09:03:43.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.957+0000 7fd8127fc640 1 -- 172.21.15.120:0/813749918 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd818005020 con 0x7fd81c100100 2024-09-19T09:03:43.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.957+0000 7fd8238f8640 1 -- 172.21.15.120:0/813749918 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd81c19f880 con 0x7fd81c100100 2024-09-19T09:03:43.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.958+0000 7fd8127fc640 1 -- 172.21.15.120:0/813749918 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8180049e0 con 0x7fd81c100100 2024-09-19T09:03:43.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.958+0000 7fd8238f8640 1 -- 172.21.15.120:0/813749918 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd81c1a7160 con 0x7fd81c100100 2024-09-19T09:03:43.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.960+0000 7fd8127fc640 1 -- 172.21.15.120:0/813749918 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7fd818005560 con 0x7fd81c100100 2024-09-19T09:03:43.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.961+0000 7fd825cd1740 1 Processor -- start 2024-09-19T09:03:43.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.961+0000 7fd825cd1740 1 -- start start 2024-09-19T09:03:43.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.962+0000 7fd8127fc640 1 --2- 172.21.15.120:0/813749918 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd8080a33d0 0x7fd8080a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:43.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.962+0000 7fd825cd1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564644297060 0x564644297440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:43.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.962+0000 7fd825cd1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564644297980 0x5646442a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:43.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.962+0000 7fd825cd1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646442a2240 0x5646442a4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:43.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.962+0000 7fd825cd1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5646441fc620 con 0x564644297980 2024-09-19T09:03:43.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.962+0000 7fd825cd1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5646441a3e50 con 0x5646442a2240 2024-09-19T09:03:43.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.962+0000 7fd825cd1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5646441f85e0 con 0x564644297060 2024-09-19T09:03:43.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.962+0000 7fd8127fc640 1 -- 172.21.15.120:0/813749918 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fd8180cfa10 con 0x7fd81c100100 2024-09-19T09:03:43.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.962+0000 7fd821694640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564644297060 0x564644297440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:43.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.962+0000 7fd821694640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564644297060 0x564644297440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:50604/0 (socket says 172.21.15.120:50604) 2024-09-19T09:03:43.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.962+0000 7fd821694640 1 -- 172.21.15.120:0/474164900 learned_addr learned my addr 172.21.15.120:0/474164900 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:43.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.962+0000 7fd820e93640 1 --2- 172.21.15.120:0/474164900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564644297980 0x5646442a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:43.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.962+0000 7fd821694640 1 -- 172.21.15.120:0/474164900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646442a2240 msgr2=0x5646442a4630 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:03:43.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.962+0000 7fd821694640 1 --2- 172.21.15.120:0/474164900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646442a2240 0x5646442a4630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:43.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.962+0000 7fd821694640 1 -- 172.21.15.120:0/474164900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564644297980 msgr2=0x5646442a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:43.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.962+0000 7fd821694640 1 --2- 172.21.15.120:0/474164900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564644297980 0x5646442a1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:43.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.962+0000 7fd821694640 1 -- 172.21.15.120:0/474164900 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564644194d40 con 0x564644297060 2024-09-19T09:03:43.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.963+0000 7fd821694640 1 --2- 172.21.15.120:0/474164900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564644297060 0x564644297440 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd8180a45d0 tx=0x7fd8180051d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45b920e3e3593a60 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:43.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.963+0000 7fd821e95640 1 --2- 172.21.15.120:0/813749918 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd8080a33d0 0x7fd8080a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:43.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.963+0000 7fd8077fe640 1 -- 172.21.15.120:0/474164900 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81801c040 con 0x564644297060 2024-09-19T09:03:43.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.963+0000 7fd8077fe640 1 -- 172.21.15.120:0/474164900 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd81808be50 con 0x564644297060 2024-09-19T09:03:43.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.963+0000 7fd8077fe640 1 -- 172.21.15.120:0/474164900 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8180cd5d0 con 0x564644297060 2024-09-19T09:03:43.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.964+0000 7fd825cd1740 1 -- 172.21.15.120:0/474164900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564644297060 msgr2=0x564644297440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:43.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.964+0000 7fd821e95640 1 --2- 172.21.15.120:0/813749918 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd8080a33d0 0x7fd8080a5890 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x564644294970 tx=0x7fd814004a80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:43.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.964+0000 7fd825cd1740 1 --2- 172.21.15.120:0/474164900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564644297060 0x564644297440 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd8180a45d0 tx=0x7fd8180051d0 comp rx=0 tx=0).stop 2024-09-19T09:03:43.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.964+0000 7fd825cd1740 1 -- 172.21.15.120:0/474164900 shutdown_connections 2024-09-19T09:03:43.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.964+0000 7fd825cd1740 1 --2- 172.21.15.120:0/474164900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646442a2240 0x5646442a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:43.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.964+0000 7fd825cd1740 1 --2- 172.21.15.120:0/474164900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564644297980 0x5646442a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:43.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.964+0000 7fd825cd1740 1 --2- 172.21.15.120:0/474164900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564644297060 0x564644297440 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:43.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.964+0000 7fd825cd1740 1 -- 172.21.15.120:0/474164900 >> 172.21.15.120:0/474164900 conn(0x5646442935f0 msgr2=0x5646442939d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:43.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.964+0000 7fd825cd1740 1 -- 172.21.15.120:0/474164900 shutdown_connections 2024-09-19T09:03:43.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.964+0000 7fd825cd1740 1 -- 172.21.15.120:0/474164900 wait complete. 2024-09-19T09:03:43.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.965+0000 7fd825cd1740 1 -- start start 2024-09-19T09:03:43.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.965+0000 7fd825cd1740 1 Processor -- start 2024-09-19T09:03:43.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.965+0000 7fd825cd1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564644297060 0x56464429f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:43.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.965+0000 7fd825cd1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564644297980 0x56464429f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:43.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.965+0000 7fd825cd1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646442a2240 0x56464429fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:43.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.965+0000 7fd825cd1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5646442a4e10 con 0x5646442a2240 2024-09-19T09:03:43.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.965+0000 7fd825cd1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564644184f10 con 0x564644297980 2024-09-19T09:03:43.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.965+0000 7fd825cd1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5646442a4f90 con 0x564644297060 2024-09-19T09:03:43.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.966+0000 7fd825cd1740 10 client.0 _ll_register_callbacks cb 0x5646441a8db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:43.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.966+0000 7fd820e93640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564644297980 0x56464429f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:43.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.966+0000 7fd821694640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564644297060 0x56464429f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:43.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.966+0000 7fd821694640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564644297060 0x56464429f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:50618/0 (socket says 172.21.15.120:50618) 2024-09-19T09:03:43.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.966+0000 7fd821694640 1 -- 172.21.15.120:0/3134469723 learned_addr learned my addr 172.21.15.120:0/3134469723 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:43.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.966+0000 7fd820e93640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564644297980 0x56464429f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59820/0 (socket says 172.21.15.120:59820) 2024-09-19T09:03:43.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.966+0000 7fd821e95640 1 --2- 172.21.15.120:0/3134469723 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646442a2240 0x56464429fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:43.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.966+0000 7fd821694640 1 -- 172.21.15.120:0/3134469723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564644297980 msgr2=0x56464429f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:43.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.966+0000 7fd821694640 1 --2- 172.21.15.120:0/3134469723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564644297980 0x56464429f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:43.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.966+0000 7fd821694640 1 -- 172.21.15.120:0/3134469723 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646442a2240 msgr2=0x56464429fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:43.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.966+0000 7fd821694640 1 --2- 172.21.15.120:0/3134469723 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646442a2240 0x56464429fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:43.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.966+0000 7fd821694640 1 -- 172.21.15.120:0/3134469723 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564643f0b580 con 0x564644297060 2024-09-19T09:03:43.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.966+0000 7fd820e93640 1 --2- 172.21.15.120:0/3134469723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564644297980 0x56464429f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:03:43.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.966+0000 7fd821e95640 1 --2- 172.21.15.120:0/3134469723 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646442a2240 0x56464429fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:03:43.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.966+0000 7fd821694640 1 --2- 172.21.15.120:0/3134469723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564644297060 0x56464429f1c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd818004c50 tx=0x7fd818098500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c88ed758917fe74b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:43.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.966+0000 7fd804ff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:03:43.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.966+0000 7fd804ff9640 1 -- 172.21.15.120:0/3134469723 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81801c070 con 0x564644297060 2024-09-19T09:03:43.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.967+0000 7fd804ff9640 1 -- 172.21.15.120:0/3134469723 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fd8180a4870 con 0x564644297060 2024-09-19T09:03:43.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.967+0000 7fd825cd1740 1 -- 172.21.15.120:0/3134469723 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd81808be50 con 0x564644297060 2024-09-19T09:03:43.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.967+0000 7fd804ff9640 1 -- 172.21.15.120:0/3134469723 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd8180a0240 con 0x564644297060 2024-09-19T09:03:43.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.967+0000 7fd825cd1740 10 client.25090 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:43.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.967+0000 7fd825cd1740 1 -- 172.21.15.120:0/3134469723 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564643fd5960 con 0x564644297060 2024-09-19T09:03:43.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.967+0000 7fd804ff9640 1 -- 172.21.15.120:0/3134469723 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd81808be50 con 0x564644297060 2024-09-19T09:03:43.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.967+0000 7fd804ff9640 1 -- 172.21.15.120:0/3134469723 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd8180baad0 con 0x564644297060 2024-09-19T09:03:43.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.968+0000 7fd825cd1740 10 client.25090 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:43.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.968+0000 7fd825cd1740 10 client.25090 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:03:43.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.968+0000 7fd825cd1740 20 client.25090 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:43.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.968+0000 7fd825cd1740 1 --2- 172.21.15.120:0/3134469723 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5646442b1cc0 0x5646442b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:43.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.968+0000 7fd825cd1740 4 client.25090 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:03:43.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.968+0000 7fd825cd1740 1 -- 172.21.15.120:0/3134469723 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564644209b90 con 0x5646442b1cc0 2024-09-19T09:03:43.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.968+0000 7fd820e93640 1 --2- 172.21.15.120:0/3134469723 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5646442b1cc0 0x5646442b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:43.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.969+0000 7fd804ff9640 10 client.25090 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:43.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.969+0000 7fd820e93640 1 --2- 172.21.15.120:0/3134469723 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5646442b1cc0 0x5646442b40b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4109dfcdda622336 server_cookie=3ef38051cd0dd193 in_seq=0 out_seq=0 2024-09-19T09:03:43.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.970+0000 7fd804ff9640 1 -- 172.21.15.120:0/3134469723 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564644209b90 con 0x5646442b1cc0 2024-09-19T09:03:43.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.970+0000 7fd804ff9640 10 client.25090 handle_command_reply: tid=0 2024-09-19T09:03:43.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.970+0000 7fd825cd1740 1 client.25090 shutdown 2024-09-19T09:03:43.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.970+0000 7fd825cd1740 1 -- 172.21.15.120:0/3134469723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564644297060 msgr2=0x56464429f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:43.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.970+0000 7fd825cd1740 1 --2- 172.21.15.120:0/3134469723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564644297060 0x56464429f1c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd818004c50 tx=0x7fd818098500 comp rx=0 tx=0).stop 2024-09-19T09:03:43.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.970+0000 7fd825cd1740 1 -- 172.21.15.120:0/3134469723 shutdown_connections 2024-09-19T09:03:43.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.970+0000 7fd825cd1740 1 --2- 172.21.15.120:0/3134469723 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5646442b1cc0 0x5646442b40b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:43.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.970+0000 7fd825cd1740 1 --2- 172.21.15.120:0/3134469723 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646442a2240 0x56464429fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:43.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.970+0000 7fd825cd1740 1 --2- 172.21.15.120:0/3134469723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564644297980 0x56464429f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:43.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.970+0000 7fd804ff9640 0 client.25090 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:43.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.970+0000 7fd825cd1740 1 --2- 172.21.15.120:0/3134469723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564644297060 0x56464429f1c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:43.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.970+0000 7fd825cd1740 1 -- 172.21.15.120:0/3134469723 >> 172.21.15.120:0/3134469723 conn(0x5646442935f0 msgr2=0x564644299320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:43.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.970+0000 7fd825cd1740 1 -- 172.21.15.120:0/3134469723 shutdown_connections 2024-09-19T09:03:43.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.970+0000 7fd825cd1740 1 -- 172.21.15.120:0/3134469723 wait complete. 2024-09-19T09:03:43.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.970+0000 7fd825cd1740 20 client.25090 trim_cache size 0 max 16384 2024-09-19T09:03:43.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.988+0000 7fd825cd1740 1 Processor -- start 2024-09-19T09:03:43.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.988+0000 7fd825cd1740 1 -- start start 2024-09-19T09:03:43.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.988+0000 7fd825cd1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564644297060 0x5646442ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:43.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.988+0000 7fd825cd1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564644297980 0x56464429c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:43.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.988+0000 7fd825cd1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646442a2240 0x56464429c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:43.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.988+0000 7fd825cd1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c1af790 con 0x5646442a2240 2024-09-19T09:03:43.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.988+0000 7fd825cd1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c1b12d0 con 0x564644297980 2024-09-19T09:03:43.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.988+0000 7fd825cd1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c1afe60 con 0x564644297060 2024-09-19T09:03:43.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.988+0000 7fd821e95640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646442a2240 0x56464429c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:43.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.988+0000 7fd821e95640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646442a2240 0x56464429c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51568/0 (socket says 172.21.15.120:51568) 2024-09-19T09:03:43.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.988+0000 7fd821e95640 1 -- 172.21.15.120:0/3487323875 learned_addr learned my addr 172.21.15.120:0/3487323875 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:43.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.988+0000 7fd820e93640 1 --2- 172.21.15.120:0/3487323875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564644297980 0x56464429c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:43.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.988+0000 7fd821694640 1 --2- 172.21.15.120:0/3487323875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564644297060 0x5646442ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:43.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.989+0000 7fd821e95640 1 -- 172.21.15.120:0/3487323875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564644297060 msgr2=0x5646442ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:43.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.989+0000 7fd821e95640 1 --2- 172.21.15.120:0/3487323875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564644297060 0x5646442ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:43.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.989+0000 7fd821e95640 1 -- 172.21.15.120:0/3487323875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564644297980 msgr2=0x56464429c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:43.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.989+0000 7fd821e95640 1 --2- 172.21.15.120:0/3487323875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564644297980 0x56464429c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:43.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.989+0000 7fd821e95640 1 -- 172.21.15.120:0/3487323875 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646442946a0 con 0x5646442a2240 2024-09-19T09:03:43.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.989+0000 7fd821e95640 1 --2- 172.21.15.120:0/3487323875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646442a2240 0x56464429c9c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd814031ea0 tx=0x7fd81403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac75ced0231f6e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:43.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.989+0000 7fd806ffd640 1 -- 172.21.15.120:0/3487323875 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8140319d0 con 0x5646442a2240 2024-09-19T09:03:43.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.989+0000 7fd806ffd640 1 -- 172.21.15.120:0/3487323875 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd814036040 con 0x5646442a2240 2024-09-19T09:03:43.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.989+0000 7fd806ffd640 1 -- 172.21.15.120:0/3487323875 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81402fb50 con 0x5646442a2240 2024-09-19T09:03:43.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.990+0000 7fd825cd1740 1 -- 172.21.15.120:0/3487323875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646442a2240 msgr2=0x56464429c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:43.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.990+0000 7fd825cd1740 1 --2- 172.21.15.120:0/3487323875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646442a2240 0x56464429c9c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd814031ea0 tx=0x7fd81403f000 comp rx=0 tx=0).stop 2024-09-19T09:03:43.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.990+0000 7fd825cd1740 1 -- 172.21.15.120:0/3487323875 shutdown_connections 2024-09-19T09:03:43.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.990+0000 7fd825cd1740 1 --2- 172.21.15.120:0/3487323875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646442a2240 0x56464429c9c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:43.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.990+0000 7fd825cd1740 1 --2- 172.21.15.120:0/3487323875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564644297980 0x56464429c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:43.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.990+0000 7fd825cd1740 1 --2- 172.21.15.120:0/3487323875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564644297060 0x5646442ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:43.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.990+0000 7fd825cd1740 1 -- 172.21.15.120:0/3487323875 >> 172.21.15.120:0/3487323875 conn(0x5646442935f0 msgr2=0x5646442ad1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:43.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.990+0000 7fd825cd1740 1 -- 172.21.15.120:0/3487323875 shutdown_connections 2024-09-19T09:03:43.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.990+0000 7fd825cd1740 1 -- 172.21.15.120:0/3487323875 wait complete. 2024-09-19T09:03:43.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.991+0000 7fd825cd1740 1 -- start start 2024-09-19T09:03:43.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.991+0000 7fd825cd1740 1 Processor -- start 2024-09-19T09:03:43.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.991+0000 7fd825cd1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564644297060 0x5646442b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:43.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.991+0000 7fd825cd1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564644297980 0x5646442b5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:43.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.991+0000 7fd825cd1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646442a2240 0x56464431b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:43.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.991+0000 7fd825cd1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c1b0c00 con 0x564644297980 2024-09-19T09:03:43.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.991+0000 7fd825cd1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c1b0530 con 0x564644297060 2024-09-19T09:03:43.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.991+0000 7fd825cd1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c1ad730 con 0x5646442a2240 2024-09-19T09:03:43.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.992+0000 7fd825cd1740 10 client.0 _ll_register_callbacks cb 0x7fd8180986e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:43.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.992+0000 7fd821694640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564644297060 0x5646442b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:43.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.992+0000 7fd820e93640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564644297980 0x5646442b5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:43.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.992+0000 7fd821694640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564644297060 0x5646442b5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59828/0 (socket says 172.21.15.120:59828) 2024-09-19T09:03:43.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.992+0000 7fd821694640 1 -- 172.21.15.120:0/1833397809 learned_addr learned my addr 172.21.15.120:0/1833397809 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:43.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.992+0000 7fd821e95640 1 --2- 172.21.15.120:0/1833397809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646442a2240 0x56464431b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:43.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.992+0000 7fd821e95640 1 -- 172.21.15.120:0/1833397809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564644297060 msgr2=0x5646442b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:43.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.992+0000 7fd821e95640 1 --2- 172.21.15.120:0/1833397809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564644297060 0x5646442b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:43.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.992+0000 7fd821e95640 1 -- 172.21.15.120:0/1833397809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564644297980 msgr2=0x5646442b5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:43.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.992+0000 7fd821e95640 1 --2- 172.21.15.120:0/1833397809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564644297980 0x5646442b5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:43.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.992+0000 7fd821e95640 1 -- 172.21.15.120:0/1833397809 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564644294280 con 0x5646442a2240 2024-09-19T09:03:43.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.992+0000 7fd820e93640 1 --2- 172.21.15.120:0/1833397809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564644297980 0x5646442b5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:03:43.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.993+0000 7fd821e95640 1 --2- 172.21.15.120:0/1833397809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646442a2240 0x56464431b600 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd814005e30 tx=0x7fd81402ffd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aab6f342f77c8c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:43.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.993+0000 7fd804ff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:03:43.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.993+0000 7fd804ff9640 1 -- 172.21.15.120:0/1833397809 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd814057020 con 0x5646442a2240 2024-09-19T09:03:43.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.993+0000 7fd804ff9640 1 -- 172.21.15.120:0/1833397809 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fd814004150 con 0x5646442a2240 2024-09-19T09:03:43.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.993+0000 7fd825cd1740 1 -- 172.21.15.120:0/1833397809 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd814036040 con 0x5646442a2240 2024-09-19T09:03:43.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.993+0000 7fd804ff9640 1 -- 172.21.15.120:0/1833397809 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd8140319d0 con 0x5646442a2240 2024-09-19T09:03:43.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.993+0000 7fd825cd1740 10 client.25096 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:43.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.993+0000 7fd825cd1740 1 -- 172.21.15.120:0/1833397809 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56464431cd90 con 0x5646442a2240 2024-09-19T09:03:43.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.994+0000 7fd804ff9640 1 -- 172.21.15.120:0/1833397809 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd814036040 con 0x5646442a2240 2024-09-19T09:03:43.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.995+0000 7fd804ff9640 1 -- 172.21.15.120:0/1833397809 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd814043040 con 0x5646442a2240 2024-09-19T09:03:43.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.995+0000 7fd825cd1740 10 client.25096 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:43.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.995+0000 7fd825cd1740 10 client.25096 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:03:43.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.995+0000 7fd825cd1740 20 client.25096 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:43.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.995+0000 7fd825cd1740 1 --2- 172.21.15.120:0/1833397809 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5646442b1cc0 0x56464431f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:43.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.995+0000 7fd825cd1740 4 client.25096 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "27390328", "release": true, "await": true}] 2024-09-19T09:03:43.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.995+0000 7fd825cd1740 1 -- 172.21.15.120:0/1833397809 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "set_id": "27390328", "release": true, "await": true}) -- 0x5646442040a0 con 0x5646442b1cc0 2024-09-19T09:03:43.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.995+0000 7fd821694640 1 --2- 172.21.15.120:0/1833397809 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5646442b1cc0 0x56464431f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:43.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.996+0000 7fd804ff9640 10 client.25096 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:43.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.996+0000 7fd821694640 1 --2- 172.21.15.120:0/1833397809 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5646442b1cc0 0x56464431f410 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c518dca132198433 server_cookie=c1857fb6c0f4a53f in_seq=0 out_seq=0 2024-09-19T09:03:44.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.997+0000 7fd804ff9640 1 -- 172.21.15.120:0/1833397809 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5646442040a0 con 0x5646442b1cc0 2024-09-19T09:03:44.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.997+0000 7fd804ff9640 10 client.25096 handle_command_reply: tid=0 2024-09-19T09:03:44.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.998+0000 7fd825cd1740 1 client.25096 shutdown 2024-09-19T09:03:44.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.998+0000 7fd825cd1740 1 -- 172.21.15.120:0/1833397809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646442a2240 msgr2=0x56464431b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:44.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.998+0000 7fd825cd1740 1 --2- 172.21.15.120:0/1833397809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646442a2240 0x56464431b600 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd814005e30 tx=0x7fd81402ffd0 comp rx=0 tx=0).stop 2024-09-19T09:03:44.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.998+0000 7fd825cd1740 1 -- 172.21.15.120:0/1833397809 shutdown_connections 2024-09-19T09:03:44.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.998+0000 7fd825cd1740 1 --2- 172.21.15.120:0/1833397809 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5646442b1cc0 0x56464431f410 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:44.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.998+0000 7fd825cd1740 1 --2- 172.21.15.120:0/1833397809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646442a2240 0x56464431b600 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:44.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.998+0000 7fd825cd1740 1 --2- 172.21.15.120:0/1833397809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564644297980 0x5646442b5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:44.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.998+0000 7fd825cd1740 1 --2- 172.21.15.120:0/1833397809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564644297060 0x5646442b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:44.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.998+0000 7fd825cd1740 1 -- 172.21.15.120:0/1833397809 >> 172.21.15.120:0/1833397809 conn(0x5646442935f0 msgr2=0x564644292750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:44.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.998+0000 7fd804ff9640 0 client.25096 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:44.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.998+0000 7fd825cd1740 1 -- 172.21.15.120:0/1833397809 shutdown_connections 2024-09-19T09:03:44.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.998+0000 7fd825cd1740 1 -- 172.21.15.120:0/1833397809 wait complete. 2024-09-19T09:03:44.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.998+0000 7fd825cd1740 20 client.25096 trim_cache size 0 max 16384 2024-09-19T09:03:44.001 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:03:44.001 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 28, 2024-09-19T09:03:44.001 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:03:44.001 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 4, 2024-09-19T09:03:44.001 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:03:44.001 INFO:teuthology.orchestra.run.smithi120.stdout: "27390328": { 2024-09-19T09:03:44.001 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 4, 2024-09-19T09:03:44.002 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:03:44.002 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:03:44.002 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:03:44.002 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:03:44.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:03:44.002 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:03:44.002 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 131.2, 2024-09-19T09:03:44.002 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:03:44.002 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:03:44.002 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:03:44.002 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:03:44.002 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:03:44.002 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:03:44.003 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:03:44.003 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:03:44.003 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:03:44.003 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:03:44.003 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:03:44.003 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:03:44.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.999+0000 7fd8238f8640 1 -- 172.21.15.120:0/813749918 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd8080a33d0 msgr2=0x7fd8080a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:44.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.999+0000 7fd8238f8640 1 --2- 172.21.15.120:0/813749918 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd8080a33d0 0x7fd8080a5890 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x564644294970 tx=0x7fd814004a80 comp rx=0 tx=0).stop 2024-09-19T09:03:44.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.999+0000 7fd8238f8640 1 -- 172.21.15.120:0/813749918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd81c100100 msgr2=0x7fd81c19a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:44.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.999+0000 7fd8238f8640 1 --2- 172.21.15.120:0/813749918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd81c100100 0x7fd81c19a5e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd81800e9f0 tx=0x7fd81800eeb0 comp rx=0 tx=0).stop 2024-09-19T09:03:44.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.999+0000 7fd8238f8640 1 -- 172.21.15.120:0/813749918 shutdown_connections 2024-09-19T09:03:44.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.999+0000 7fd8238f8640 1 --2- 172.21.15.120:0/813749918 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd8080a33d0 0x7fd8080a5890 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:44.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.999+0000 7fd8238f8640 1 --2- 172.21.15.120:0/813749918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd81c102a50 0x7fd81c19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:44.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.999+0000 7fd8238f8640 1 --2- 172.21.15.120:0/813749918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd81c100c80 0x7fd81c19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:44.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.999+0000 7fd8238f8640 1 --2- 172.21.15.120:0/813749918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd81c100100 0x7fd81c19a5e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:44.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:43.999+0000 7fd8238f8640 1 -- 172.21.15.120:0/813749918 >> 172.21.15.120:0/813749918 conn(0x7fd81c077230 msgr2=0x7fd81c0fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:44.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:44.000+0000 7fd8238f8640 1 -- 172.21.15.120:0/813749918 shutdown_connections 2024-09-19T09:03:44.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:44.000+0000 7fd8238f8640 1 -- 172.21.15.120:0/813749918 wait complete. 2024-09-19T09:03:44.269 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:03:44.270 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:03:44.822 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '27390328', total seconds elapsed: 16.3 2024-09-19T09:03:44.958 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:03:45.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:44 smithi120 ceph-mon[31150]: pgmap v367: 97 pgs: 97 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-19T09:03:45.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:44 smithi196 ceph-mon[35150]: pgmap v367: 97 pgs: 97 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-19T09:03:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:44 smithi153 ceph-mon[35071]: pgmap v367: 97 pgs: 97 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-19T09:03:45.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.521+0000 7fde20b94640 1 -- 172.21.15.120:0/2367793657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde1c074e30 msgr2=0x7fde1c10df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:45.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.521+0000 7fde20b94640 1 --2- 172.21.15.120:0/2367793657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde1c074e30 0x7fde1c10df10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fde10009900 tx=0x7fde1002a480 comp rx=0 tx=0).stop 2024-09-19T09:03:45.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.522+0000 7fde20b94640 1 -- 172.21.15.120:0/2367793657 shutdown_connections 2024-09-19T09:03:45.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.522+0000 7fde20b94640 1 --2- 172.21.15.120:0/2367793657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde1c10e450 0x7fde1c110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.522+0000 7fde20b94640 1 --2- 172.21.15.120:0/2367793657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde1c074e30 0x7fde1c10df10 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.522+0000 7fde20b94640 1 --2- 172.21.15.120:0/2367793657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde1c074510 0x7fde1c0748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.522+0000 7fde20b94640 1 -- 172.21.15.120:0/2367793657 >> 172.21.15.120:0/2367793657 conn(0x7fde1c0fd8f0 msgr2=0x7fde1c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:45.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.523+0000 7fde20b94640 1 -- 172.21.15.120:0/2367793657 shutdown_connections 2024-09-19T09:03:45.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.523+0000 7fde20b94640 1 -- 172.21.15.120:0/2367793657 wait complete. 2024-09-19T09:03:45.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.523+0000 7fde20b94640 1 Processor -- start 2024-09-19T09:03:45.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.523+0000 7fde20b94640 1 -- start start 2024-09-19T09:03:45.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.523+0000 7fde20b94640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde1c074510 0x7fde1c19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:45.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.524+0000 7fde20b94640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde1c074e30 0x7fde1c19ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:45.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.524+0000 7fde20b94640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde1c10e450 0x7fde1c19dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:45.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.524+0000 7fde20b94640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c1141c0 con 0x7fde1c10e450 2024-09-19T09:03:45.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.524+0000 7fde20b94640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c114040 con 0x7fde1c074510 2024-09-19T09:03:45.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.524+0000 7fde20b94640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c114340 con 0x7fde1c074e30 2024-09-19T09:03:45.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.524+0000 7fde1a59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde1c074510 0x7fde1c19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:45.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.524+0000 7fde1a59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde1c074510 0x7fde1c19a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59848/0 (socket says 172.21.15.120:59848) 2024-09-19T09:03:45.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.524+0000 7fde1a59c640 1 -- 172.21.15.120:0/1096003686 learned_addr learned my addr 172.21.15.120:0/1096003686 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:45.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.524+0000 7fde19d9b640 1 --2- 172.21.15.120:0/1096003686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde1c074e30 0x7fde1c19ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:45.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.524+0000 7fde1a59c640 1 -- 172.21.15.120:0/1096003686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde1c074e30 msgr2=0x7fde1c19ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:45.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.524+0000 7fde1a59c640 1 --2- 172.21.15.120:0/1096003686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde1c074e30 0x7fde1c19ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.524+0000 7fde1a59c640 1 -- 172.21.15.120:0/1096003686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde1c10e450 msgr2=0x7fde1c19dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:45.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.524+0000 7fde1a59c640 1 --2- 172.21.15.120:0/1096003686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde1c10e450 0x7fde1c19dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.524+0000 7fde1a59c640 1 -- 172.21.15.120:0/1096003686 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde1c19e710 con 0x7fde1c074510 2024-09-19T09:03:45.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.524+0000 7fde1a59c640 1 --2- 172.21.15.120:0/1096003686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde1c074510 0x7fde1c19a7c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00d770 tx=0x7fde0c00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:45.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.524+0000 7fde0b7fe640 1 -- 172.21.15.120:0/1096003686 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c017070 con 0x7fde1c074510 2024-09-19T09:03:45.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.525+0000 7fde20b94640 1 -- 172.21.15.120:0/1096003686 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde1c19ea00 con 0x7fde1c074510 2024-09-19T09:03:45.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.525+0000 7fde0b7fe640 1 -- 172.21.15.120:0/1096003686 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde0c005500 con 0x7fde1c074510 2024-09-19T09:03:45.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.525+0000 7fde0b7fe640 1 -- 172.21.15.120:0/1096003686 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c004e40 con 0x7fde1c074510 2024-09-19T09:03:45.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.525+0000 7fde20b94640 1 -- 172.21.15.120:0/1096003686 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde1c1a7370 con 0x7fde1c074510 2024-09-19T09:03:45.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.528+0000 7fde0b7fe640 1 -- 172.21.15.120:0/1096003686 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7fde0c009c30 con 0x7fde1c074510 2024-09-19T09:03:45.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.528+0000 7fde22f6d740 1 Processor -- start 2024-09-19T09:03:45.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.528+0000 7fde22f6d740 1 -- start start 2024-09-19T09:03:45.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.528+0000 7fde22f6d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d61066b60 0x557d61066f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:45.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.528+0000 7fde22f6d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d61067480 0x557d61071800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:45.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.528+0000 7fde22f6d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d61071d40 0x557d61074130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:45.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.529+0000 7fde1ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d61066b60 0x557d61066f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:45.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.529+0000 7fde1ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d61066b60 0x557d61066f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51596/0 (socket says 172.21.15.120:51596) 2024-09-19T09:03:45.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.529+0000 7fde1a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d61067480 0x557d61071800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:45.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.529+0000 7fde1a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d61067480 0x557d61071800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:50666/0 (socket says 172.21.15.120:50666) 2024-09-19T09:03:45.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.529+0000 7fde1ad9d640 1 -- 172.21.15.120:0/3547898028 learned_addr learned my addr 172.21.15.120:0/3547898028 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:45.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.529+0000 7fde22f6d740 1 -- 172.21.15.120:0/3547898028 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557d60fcc280 con 0x557d61066b60 2024-09-19T09:03:45.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.529+0000 7fde22f6d740 1 -- 172.21.15.120:0/3547898028 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557d60f73b10 con 0x557d61071d40 2024-09-19T09:03:45.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.529+0000 7fde22f6d740 1 -- 172.21.15.120:0/3547898028 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557d60fc8240 con 0x557d61067480 2024-09-19T09:03:45.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.529+0000 7fde0b7fe640 1 --2- 172.21.15.120:0/1096003686 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fde040a3350 0x7fde040a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:45.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.530+0000 7fde19d9b640 1 --2- 172.21.15.120:0/3547898028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d61071d40 0x557d61074130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:45.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.530+0000 7fde1a59c640 1 -- 172.21.15.120:0/3547898028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d61071d40 msgr2=0x557d61074130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:45.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.530+0000 7fde1a59c640 1 --2- 172.21.15.120:0/3547898028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d61071d40 0x557d61074130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.530+0000 7fde1a59c640 1 -- 172.21.15.120:0/3547898028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d61066b60 msgr2=0x557d61066f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:45.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.530+0000 7fde1a59c640 1 --2- 172.21.15.120:0/3547898028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d61066b60 0x557d61066f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.530+0000 7fde1ad9d640 1 --2- 172.21.15.120:0/1096003686 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fde040a3350 0x7fde040a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:45.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.530+0000 7fde1a59c640 1 -- 172.21.15.120:0/3547898028 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d60f801f0 con 0x557d61067480 2024-09-19T09:03:45.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.530+0000 7fde1a59c640 1 --2- 172.21.15.120:0/3547898028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d61067480 0x557d61071800 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0ccef0 tx=0x7fde0c0d5ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d86d1979e39639c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:45.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.530+0000 7fde08ff9640 1 -- 172.21.15.120:0/3547898028 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c0e2070 con 0x557d61067480 2024-09-19T09:03:45.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.531+0000 7fde08ff9640 1 -- 172.21.15.120:0/3547898028 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde0c0046c0 con 0x557d61067480 2024-09-19T09:03:45.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.531+0000 7fde08ff9640 1 -- 172.21.15.120:0/3547898028 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c014d30 con 0x557d61067480 2024-09-19T09:03:45.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.531+0000 7fde0b7fe640 1 -- 172.21.15.120:0/1096003686 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fde0c0cf540 con 0x7fde1c074510 2024-09-19T09:03:45.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.531+0000 7fde1ad9d640 1 --2- 172.21.15.120:0/1096003686 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fde040a3350 0x7fde040a5810 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fde14005c40 tx=0x7fde14005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:45.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.531+0000 7fde22f6d740 1 -- 172.21.15.120:0/3547898028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d61067480 msgr2=0x557d61071800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:45.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.531+0000 7fde22f6d740 1 --2- 172.21.15.120:0/3547898028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d61067480 0x557d61071800 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0ccef0 tx=0x7fde0c0d5ba0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.532+0000 7fde22f6d740 1 -- 172.21.15.120:0/3547898028 shutdown_connections 2024-09-19T09:03:45.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.532+0000 7fde22f6d740 1 --2- 172.21.15.120:0/3547898028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d61071d40 0x557d61074130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.532+0000 7fde22f6d740 1 --2- 172.21.15.120:0/3547898028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d61067480 0x557d61071800 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.532+0000 7fde22f6d740 1 --2- 172.21.15.120:0/3547898028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d61066b60 0x557d61066f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.532+0000 7fde22f6d740 1 -- 172.21.15.120:0/3547898028 >> 172.21.15.120:0/3547898028 conn(0x557d610630f0 msgr2=0x557d610634d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:45.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.532+0000 7fde22f6d740 1 -- 172.21.15.120:0/3547898028 shutdown_connections 2024-09-19T09:03:45.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.532+0000 7fde22f6d740 1 -- 172.21.15.120:0/3547898028 wait complete. 2024-09-19T09:03:45.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.533+0000 7fde22f6d740 1 -- start start 2024-09-19T09:03:45.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.533+0000 7fde22f6d740 1 Processor -- start 2024-09-19T09:03:45.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.533+0000 7fde22f6d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d61066b60 0x557d6106eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:45.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.533+0000 7fde22f6d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d61067480 0x557d6106f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:45.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.534+0000 7fde22f6d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d61071d40 0x557d6106f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:45.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.534+0000 7fde1a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d61066b60 0x557d6106eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:45.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.534+0000 7fde1a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d61066b60 0x557d6106eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:50676/0 (socket says 172.21.15.120:50676) 2024-09-19T09:03:45.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.534+0000 7fde1a59c640 1 -- 172.21.15.120:0/2256688302 learned_addr learned my addr 172.21.15.120:0/2256688302 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:45.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.534+0000 7fde1ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d61071d40 0x557d6106f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:45.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.534+0000 7fde19d9b640 1 --2- 172.21.15.120:0/2256688302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d61067480 0x557d6106f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:45.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.534+0000 7fde22f6d740 1 -- 172.21.15.120:0/2256688302 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557d61074910 con 0x557d61071d40 2024-09-19T09:03:45.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.534+0000 7fde22f6d740 1 -- 172.21.15.120:0/2256688302 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557d60f86f10 con 0x557d61067480 2024-09-19T09:03:45.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.534+0000 7fde22f6d740 1 -- 172.21.15.120:0/2256688302 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557d61074a90 con 0x557d61066b60 2024-09-19T09:03:45.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.534+0000 7fde22f6d740 10 client.0 _ll_register_callbacks cb 0x557d60f4fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:45.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.534+0000 7fde1ad9d640 1 -- 172.21.15.120:0/2256688302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d61066b60 msgr2=0x557d6106eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:45.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.534+0000 7fde1ad9d640 1 --2- 172.21.15.120:0/2256688302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d61066b60 0x557d6106eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.534+0000 7fde1ad9d640 1 -- 172.21.15.120:0/2256688302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d61067480 msgr2=0x557d6106f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:45.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.534+0000 7fde1ad9d640 1 --2- 172.21.15.120:0/2256688302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d61067480 0x557d6106f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.534+0000 7fde1ad9d640 1 -- 172.21.15.120:0/2256688302 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d60d3d6a0 con 0x557d61071d40 2024-09-19T09:03:45.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.535+0000 7fde22f6d740 1 -- 172.21.15.120:0/2256688302 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557d60fd97f0 con 0x557d61071d40 2024-09-19T09:03:45.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.535+0000 7fde1ad9d640 1 --2- 172.21.15.120:0/2256688302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d61071d40 0x557d6106f620 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fde14009e10 tx=0x7fde14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b7862678b42f50 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:45.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.535+0000 7fde01ffb640 10 client.15684 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:45.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.535+0000 7fde01ffb640 1 -- 172.21.15.120:0/2256688302 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde14025020 con 0x557d61071d40 2024-09-19T09:03:45.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.535+0000 7fde01ffb640 1 -- 172.21.15.120:0/2256688302 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fde14004d20 con 0x557d61071d40 2024-09-19T09:03:45.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.535+0000 7fde01ffb640 1 -- 172.21.15.120:0/2256688302 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fde14010910 con 0x557d61071d40 2024-09-19T09:03:45.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.535+0000 7fde22f6d740 10 client.15684 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:45.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.535+0000 7fde22f6d740 1 -- 172.21.15.120:0/2256688302 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({fsmap=28}) -- 0x557d61080380 con 0x557d61071d40 2024-09-19T09:03:45.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.536+0000 7fde01ffb640 1 -- 172.21.15.120:0/2256688302 <== mon.0 v2:172.21.15.120:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fde1400f040 con 0x557d61071d40 2024-09-19T09:03:45.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.536+0000 7fde22f6d740 10 client.15684 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:45.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.536+0000 7fde22f6d740 10 client.15684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:03:45.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.536+0000 7fde22f6d740 20 client.15684 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:45.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.536+0000 7fde22f6d740 1 --2- 172.21.15.120:0/2256688302 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557d61081b90 0x557d61083f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:45.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.536+0000 7fde22f6d740 4 client.15684 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:03:45.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.536+0000 7fde22f6d740 1 -- 172.21.15.120:0/2256688302 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d60fd3d00 con 0x557d61081b90 2024-09-19T09:03:45.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.537+0000 7fde1a59c640 1 --2- 172.21.15.120:0/2256688302 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557d61081b90 0x557d61083f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:45.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.537+0000 7fde1a59c640 1 --2- 172.21.15.120:0/2256688302 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557d61081b90 0x557d61083f80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dac543442e9a1f4 server_cookie=2d3d91ec1578a3ec in_seq=0 out_seq=0 2024-09-19T09:03:45.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.537+0000 7fde01ffb640 10 client.15684 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:45.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.538+0000 7fde01ffb640 1 -- 172.21.15.120:0/2256688302 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d60fd3d00 con 0x557d61081b90 2024-09-19T09:03:45.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.538+0000 7fde01ffb640 10 client.15684 handle_command_reply: tid=0 2024-09-19T09:03:45.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.538+0000 7fde22f6d740 1 client.15684 shutdown 2024-09-19T09:03:45.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.538+0000 7fde22f6d740 1 -- 172.21.15.120:0/2256688302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d61071d40 msgr2=0x557d6106f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:45.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.538+0000 7fde22f6d740 1 --2- 172.21.15.120:0/2256688302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d61071d40 0x557d6106f620 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fde14009e10 tx=0x7fde14004a40 comp rx=0 tx=0).stop 2024-09-19T09:03:45.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.538+0000 7fde22f6d740 1 -- 172.21.15.120:0/2256688302 shutdown_connections 2024-09-19T09:03:45.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.538+0000 7fde22f6d740 1 --2- 172.21.15.120:0/2256688302 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557d61081b90 0x557d61083f80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.538+0000 7fde22f6d740 1 --2- 172.21.15.120:0/2256688302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d61071d40 0x557d6106f620 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.538+0000 7fde22f6d740 1 --2- 172.21.15.120:0/2256688302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d61067480 0x557d6106f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.538+0000 7fde22f6d740 1 --2- 172.21.15.120:0/2256688302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d61066b60 0x557d6106eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.538+0000 7fde22f6d740 1 -- 172.21.15.120:0/2256688302 >> 172.21.15.120:0/2256688302 conn(0x557d610630f0 msgr2=0x557d61068e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:45.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.538+0000 7fde01ffb640 0 client.15684 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:45.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.538+0000 7fde22f6d740 1 -- 172.21.15.120:0/2256688302 shutdown_connections 2024-09-19T09:03:45.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.538+0000 7fde22f6d740 1 -- 172.21.15.120:0/2256688302 wait complete. 2024-09-19T09:03:45.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.538+0000 7fde22f6d740 20 client.15684 trim_cache size 0 max 16384 2024-09-19T09:03:45.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.556+0000 7fde22f6d740 1 Processor -- start 2024-09-19T09:03:45.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.556+0000 7fde22f6d740 1 -- start start 2024-09-19T09:03:45.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.556+0000 7fde22f6d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d61066b60 0x557d610695c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:45.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.556+0000 7fde22f6d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d61067480 0x557d6107d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:45.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.556+0000 7fde22f6d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d61071d40 0x557d61086bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:45.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.556+0000 7fde22f6d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c1af9a0 con 0x557d61066b60 2024-09-19T09:03:45.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.556+0000 7fde22f6d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c1b14e0 con 0x557d61071d40 2024-09-19T09:03:45.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.556+0000 7fde22f6d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c1b0070 con 0x557d61067480 2024-09-19T09:03:45.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.557+0000 7fde19d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d61067480 0x557d6107d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:45.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.557+0000 7fde19d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d61067480 0x557d6107d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:50680/0 (socket says 172.21.15.120:50680) 2024-09-19T09:03:45.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.557+0000 7fde19d9b640 1 -- 172.21.15.120:0/2035810066 learned_addr learned my addr 172.21.15.120:0/2035810066 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:45.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.557+0000 7fde1ad9d640 1 --2- 172.21.15.120:0/2035810066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d61071d40 0x557d61086bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:45.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.557+0000 7fde1a59c640 1 --2- 172.21.15.120:0/2035810066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d61066b60 0x557d610695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:45.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.557+0000 7fde1a59c640 1 -- 172.21.15.120:0/2035810066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d61067480 msgr2=0x557d6107d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:45.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.557+0000 7fde1a59c640 1 --2- 172.21.15.120:0/2035810066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d61067480 0x557d6107d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.557+0000 7fde1a59c640 1 -- 172.21.15.120:0/2035810066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d61071d40 msgr2=0x557d61086bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:45.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.557+0000 7fde1a59c640 1 --2- 172.21.15.120:0/2035810066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d61071d40 0x557d61086bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.557+0000 7fde1a59c640 1 -- 172.21.15.120:0/2035810066 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d61062f20 con 0x557d61066b60 2024-09-19T09:03:45.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.558+0000 7fde1a59c640 1 --2- 172.21.15.120:0/2035810066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d61066b60 0x557d610695c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0a4140 tx=0x7fde0c086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d66753a0d9c493a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:45.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.558+0000 7fde03fff640 1 -- 172.21.15.120:0/2035810066 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c09b7d0 con 0x557d61066b60 2024-09-19T09:03:45.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.558+0000 7fde03fff640 1 -- 172.21.15.120:0/2035810066 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde0c09c100 con 0x557d61066b60 2024-09-19T09:03:45.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.558+0000 7fde03fff640 1 -- 172.21.15.120:0/2035810066 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c09b540 con 0x557d61066b60 2024-09-19T09:03:45.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.559+0000 7fde22f6d740 1 -- 172.21.15.120:0/2035810066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d61066b60 msgr2=0x557d610695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:45.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.559+0000 7fde22f6d740 1 --2- 172.21.15.120:0/2035810066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d61066b60 0x557d610695c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0a4140 tx=0x7fde0c086040 comp rx=0 tx=0).stop 2024-09-19T09:03:45.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.559+0000 7fde22f6d740 1 -- 172.21.15.120:0/2035810066 shutdown_connections 2024-09-19T09:03:45.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.559+0000 7fde22f6d740 1 --2- 172.21.15.120:0/2035810066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d61071d40 0x557d61086bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.559+0000 7fde22f6d740 1 --2- 172.21.15.120:0/2035810066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d61067480 0x557d6107d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.559+0000 7fde22f6d740 1 --2- 172.21.15.120:0/2035810066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d61066b60 0x557d610695c0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.559+0000 7fde22f6d740 1 -- 172.21.15.120:0/2035810066 >> 172.21.15.120:0/2035810066 conn(0x557d610630f0 msgr2=0x557d6107cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:45.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.559+0000 7fde22f6d740 1 -- 172.21.15.120:0/2035810066 shutdown_connections 2024-09-19T09:03:45.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.559+0000 7fde22f6d740 1 -- 172.21.15.120:0/2035810066 wait complete. 2024-09-19T09:03:45.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.559+0000 7fde22f6d740 1 -- start start 2024-09-19T09:03:45.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.560+0000 7fde22f6d740 1 Processor -- start 2024-09-19T09:03:45.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.560+0000 7fde22f6d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d61067480 0x557d61084fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:45.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.560+0000 7fde22f6d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d61071d40 0x557d61085500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:45.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.560+0000 7fde22f6d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d610f1cd0 0x557d610f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:45.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.560+0000 7fde22f6d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c1b0e10 con 0x557d61071d40 2024-09-19T09:03:45.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.560+0000 7fde22f6d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c1b0740 con 0x557d610f1cd0 2024-09-19T09:03:45.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.560+0000 7fde22f6d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c1ad960 con 0x557d61067480 2024-09-19T09:03:45.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.560+0000 7fde22f6d740 10 client.0 _ll_register_callbacks cb 0x7fde14009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:45.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.560+0000 7fde19d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d61071d40 0x557d61085500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:45.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.560+0000 7fde19d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d61071d40 0x557d61085500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51628/0 (socket says 172.21.15.120:51628) 2024-09-19T09:03:45.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.560+0000 7fde19d9b640 1 -- 172.21.15.120:0/722867080 learned_addr learned my addr 172.21.15.120:0/722867080 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:45.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.560+0000 7fde1a59c640 1 --2- 172.21.15.120:0/722867080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d61067480 0x557d61084fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:45.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.560+0000 7fde1ad9d640 1 --2- 172.21.15.120:0/722867080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d610f1cd0 0x557d610f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:45.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.560+0000 7fde19d9b640 1 -- 172.21.15.120:0/722867080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d61067480 msgr2=0x557d61084fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:45.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.560+0000 7fde19d9b640 1 --2- 172.21.15.120:0/722867080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d61067480 0x557d61084fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.561+0000 7fde19d9b640 1 -- 172.21.15.120:0/722867080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d610f1cd0 msgr2=0x557d610f40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:45.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.561+0000 7fde19d9b640 1 --2- 172.21.15.120:0/722867080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d610f1cd0 0x557d610f40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.561+0000 7fde19d9b640 1 -- 172.21.15.120:0/722867080 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d610f4600 con 0x557d61071d40 2024-09-19T09:03:45.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.561+0000 7fde19d9b640 1 --2- 172.21.15.120:0/722867080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d61071d40 0x557d61085500 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fde10009230 tx=0x7fde10005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c686b506d1960381 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:45.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.561+0000 7fde01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:45.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.561+0000 7fde01ffb640 1 -- 172.21.15.120:0/722867080 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde10035040 con 0x557d61071d40 2024-09-19T09:03:45.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.561+0000 7fde01ffb640 1 -- 172.21.15.120:0/722867080 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fde100054f0 con 0x557d61071d40 2024-09-19T09:03:45.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.561+0000 7fde22f6d740 1 -- 172.21.15.120:0/722867080 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde0c09c100 con 0x557d61071d40 2024-09-19T09:03:45.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.561+0000 7fde01ffb640 1 -- 172.21.15.120:0/722867080 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fde100046c0 con 0x557d61071d40 2024-09-19T09:03:45.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.561+0000 7fde22f6d740 10 client.15690 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:45.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.561+0000 7fde22f6d740 1 -- 172.21.15.120:0/722867080 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557d610f5910 con 0x557d61071d40 2024-09-19T09:03:45.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.562+0000 7fde01ffb640 1 -- 172.21.15.120:0/722867080 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde0c09c100 con 0x557d61071d40 2024-09-19T09:03:45.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.562+0000 7fde01ffb640 1 -- 172.21.15.120:0/722867080 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fde10031ad0 con 0x557d61071d40 2024-09-19T09:03:45.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.562+0000 7fde22f6d740 10 client.15690 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:45.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.562+0000 7fde22f6d740 10 client.15690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:03:45.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.562+0000 7fde22f6d740 20 client.15690 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:45.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.562+0000 7fde22f6d740 1 --2- 172.21.15.120:0/722867080 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557d610f6010 0x557d610f8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:45.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.562+0000 7fde22f6d740 4 client.15690 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:03:45.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.562+0000 7fde22f6d740 1 -- 172.21.15.120:0/722867080 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557d60f50110 con 0x557d610f6010 2024-09-19T09:03:45.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.562+0000 7fde1a59c640 1 --2- 172.21.15.120:0/722867080 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557d610f6010 0x557d610f8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:45.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.563+0000 7fde01ffb640 10 client.15690 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:45.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.563+0000 7fde1a59c640 1 --2- 172.21.15.120:0/722867080 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557d610f6010 0x557d610f8430 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23875a6c08660ef6 server_cookie=89d606705d2bc6f3 in_seq=0 out_seq=0 2024-09-19T09:03:45.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.564+0000 7fde01ffb640 1 -- 172.21.15.120:0/722867080 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557d60f50110 con 0x557d610f6010 2024-09-19T09:03:45.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.564+0000 7fde01ffb640 10 client.15690 handle_command_reply: tid=0 2024-09-19T09:03:45.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.564+0000 7fde22f6d740 1 client.15690 shutdown 2024-09-19T09:03:45.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.564+0000 7fde22f6d740 1 -- 172.21.15.120:0/722867080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d61071d40 msgr2=0x557d61085500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:45.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.564+0000 7fde22f6d740 1 --2- 172.21.15.120:0/722867080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d61071d40 0x557d61085500 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fde10009230 tx=0x7fde10005270 comp rx=0 tx=0).stop 2024-09-19T09:03:45.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.565+0000 7fde22f6d740 1 -- 172.21.15.120:0/722867080 shutdown_connections 2024-09-19T09:03:45.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.565+0000 7fde22f6d740 1 --2- 172.21.15.120:0/722867080 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557d610f6010 0x557d610f8430 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.565+0000 7fde22f6d740 1 --2- 172.21.15.120:0/722867080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d610f1cd0 0x557d610f40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.565+0000 7fde22f6d740 1 --2- 172.21.15.120:0/722867080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d61071d40 0x557d61085500 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.565+0000 7fde22f6d740 1 --2- 172.21.15.120:0/722867080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d61067480 0x557d61084fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.565+0000 7fde22f6d740 1 -- 172.21.15.120:0/722867080 >> 172.21.15.120:0/722867080 conn(0x557d610630f0 msgr2=0x557d61062270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:45.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.565+0000 7fde01ffb640 0 client.15690 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:45.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.565+0000 7fde22f6d740 1 -- 172.21.15.120:0/722867080 shutdown_connections 2024-09-19T09:03:45.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.565+0000 7fde22f6d740 1 -- 172.21.15.120:0/722867080 wait complete. 2024-09-19T09:03:45.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.566+0000 7fde22f6d740 20 client.15690 trim_cache size 0 max 16384 2024-09-19T09:03:45.572 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:03:45.572 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:03:45.572 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "56e704ee-8828-40ca-8d0d-c9ecdb8b4627", 2024-09-19T09:03:45.573 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:03:45.573 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:03:45.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.566+0000 7fde20b94640 1 -- 172.21.15.120:0/1096003686 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fde040a3350 msgr2=0x7fde040a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:45.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.566+0000 7fde20b94640 1 --2- 172.21.15.120:0/1096003686 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fde040a3350 0x7fde040a5810 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fde14005c40 tx=0x7fde14005bd0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.566+0000 7fde20b94640 1 -- 172.21.15.120:0/1096003686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde1c074510 msgr2=0x7fde1c19a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:45.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.566+0000 7fde20b94640 1 --2- 172.21.15.120:0/1096003686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde1c074510 0x7fde1c19a7c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00d770 tx=0x7fde0c00dc30 comp rx=0 tx=0).stop 2024-09-19T09:03:45.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.566+0000 7fde20b94640 1 -- 172.21.15.120:0/1096003686 shutdown_connections 2024-09-19T09:03:45.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.566+0000 7fde20b94640 1 --2- 172.21.15.120:0/1096003686 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fde040a3350 0x7fde040a5810 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.566+0000 7fde20b94640 1 --2- 172.21.15.120:0/1096003686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde1c10e450 0x7fde1c19dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.566+0000 7fde20b94640 1 --2- 172.21.15.120:0/1096003686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde1c074e30 0x7fde1c19ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.566+0000 7fde20b94640 1 --2- 172.21.15.120:0/1096003686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde1c074510 0x7fde1c19a7c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:45.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.566+0000 7fde20b94640 1 -- 172.21.15.120:0/1096003686 >> 172.21.15.120:0/1096003686 conn(0x7fde1c0fd8f0 msgr2=0x7fde1c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:45.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.566+0000 7fde20b94640 1 -- 172.21.15.120:0/1096003686 shutdown_connections 2024-09-19T09:03:45.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:45.566+0000 7fde20b94640 1 -- 172.21.15.120:0/1096003686 wait complete. 2024-09-19T09:03:46.430 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:03:46.704 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:03:46.979 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:46 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:03:46.979 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:46 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:03:46.979 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:46 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:03:46.979 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:46 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:03:46.979 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:46 smithi120 ceph-mon[31150]: pgmap v368: 97 pgs: 97 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-19T09:03:47.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:46 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:03:47.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:46 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:03:47.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:46 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:03:47.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:46 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:03:47.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:46 smithi196 ceph-mon[35150]: pgmap v368: 97 pgs: 97 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-19T09:03:47.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:46 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:03:47.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:46 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:03:47.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:46 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:03:47.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:46 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:03:47.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:46 smithi153 ceph-mon[35071]: pgmap v368: 97 pgs: 97 active+clean; 674 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-19T09:03:47.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.288+0000 7f4c6f61b640 1 -- 172.21.15.120:0/966591513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c68102570 msgr2=0x7f4c681029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:47.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.288+0000 7f4c6f61b640 1 --2- 172.21.15.120:0/966591513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c68102570 0x7f4c681029b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009900 tx=0x7f4c5802a480 comp rx=0 tx=0).stop 2024-09-19T09:03:47.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.288+0000 7f4c6f61b640 1 -- 172.21.15.120:0/966591513 shutdown_connections 2024-09-19T09:03:47.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.288+0000 7f4c6f61b640 1 --2- 172.21.15.120:0/966591513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c68102ef0 0x7f4c6810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.288+0000 7f4c6f61b640 1 --2- 172.21.15.120:0/966591513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c68102570 0x7f4c681029b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.288+0000 7f4c6f61b640 1 --2- 172.21.15.120:0/966591513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c68100e20 0x7f4c68101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.288+0000 7f4c6f61b640 1 -- 172.21.15.120:0/966591513 >> 172.21.15.120:0/966591513 conn(0x7f4c680fc7d0 msgr2=0x7f4c680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:47.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.289+0000 7f4c6f61b640 1 -- 172.21.15.120:0/966591513 shutdown_connections 2024-09-19T09:03:47.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.289+0000 7f4c6f61b640 1 -- 172.21.15.120:0/966591513 wait complete. 2024-09-19T09:03:47.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.289+0000 7f4c6f61b640 1 Processor -- start 2024-09-19T09:03:47.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.289+0000 7f4c6f61b640 1 -- start start 2024-09-19T09:03:47.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.290+0000 7f4c6f61b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c68100e20 0x7f4c681986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:47.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.290+0000 7f4c6f61b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c68102ef0 0x7f4c68198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:47.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.290+0000 7f4c6f61b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c6819cfb0 0x7f4c6819d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:47.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.290+0000 7f4c6f61b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681120a0 con 0x7f4c68102ef0 2024-09-19T09:03:47.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.290+0000 7f4c6f61b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4c68111f20 con 0x7f4c68100e20 2024-09-19T09:03:47.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.290+0000 7f4c6f61b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4c68112220 con 0x7f4c6819cfb0 2024-09-19T09:03:47.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.290+0000 7f4c6d3b7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c68100e20 0x7f4c681986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:47.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.290+0000 7f4c6cbb6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c68102ef0 0x7f4c68198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:47.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.290+0000 7f4c6d3b7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c68100e20 0x7f4c681986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59904/0 (socket says 172.21.15.120:59904) 2024-09-19T09:03:47.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.290+0000 7f4c6d3b7640 1 -- 172.21.15.120:0/2218082481 learned_addr learned my addr 172.21.15.120:0/2218082481 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:47.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.290+0000 7f4c6dbb8640 1 --2- 172.21.15.120:0/2218082481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c6819cfb0 0x7f4c6819d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:47.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.290+0000 7f4c6d3b7640 1 -- 172.21.15.120:0/2218082481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c6819cfb0 msgr2=0x7f4c6819d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:47.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.290+0000 7f4c6d3b7640 1 --2- 172.21.15.120:0/2218082481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c6819cfb0 0x7f4c6819d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.290+0000 7f4c6d3b7640 1 -- 172.21.15.120:0/2218082481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c68102ef0 msgr2=0x7f4c68198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:47.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.290+0000 7f4c6d3b7640 1 --2- 172.21.15.120:0/2218082481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c68102ef0 0x7f4c68198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.290+0000 7f4c6d3b7640 1 -- 172.21.15.120:0/2218082481 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c6819da70 con 0x7f4c68100e20 2024-09-19T09:03:47.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.290+0000 7f4c6cbb6640 1 --2- 172.21.15.120:0/2218082481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c68102ef0 0x7f4c68198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:03:47.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.291+0000 7f4c6d3b7640 1 --2- 172.21.15.120:0/2218082481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c68100e20 0x7f4c681986e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4c6400d950 tx=0x7f4c6400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:47.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.291+0000 7f4c5e7fc640 1 -- 172.21.15.120:0/2218082481 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c64017070 con 0x7f4c68100e20 2024-09-19T09:03:47.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.291+0000 7f4c5e7fc640 1 -- 172.21.15.120:0/2218082481 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c64009c30 con 0x7f4c68100e20 2024-09-19T09:03:47.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.291+0000 7f4c6f61b640 1 -- 172.21.15.120:0/2218082481 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c6819dd60 con 0x7f4c68100e20 2024-09-19T09:03:47.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.292+0000 7f4c5e7fc640 1 -- 172.21.15.120:0/2218082481 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6400f6b0 con 0x7f4c68100e20 2024-09-19T09:03:47.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.292+0000 7f4c6f61b640 1 -- 172.21.15.120:0/2218082481 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c6810e620 con 0x7f4c68100e20 2024-09-19T09:03:47.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.294+0000 7f4c5e7fc640 1 -- 172.21.15.120:0/2218082481 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f4c6400f850 con 0x7f4c68100e20 2024-09-19T09:03:47.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.295+0000 7f4c719f4740 1 Processor -- start 2024-09-19T09:03:47.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.296+0000 7f4c719f4740 1 -- start start 2024-09-19T09:03:47.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.296+0000 7f4c719f4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e0b3414a0 0x558e0b341880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:47.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.296+0000 7f4c719f4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e0b341dc0 0x558e0b34c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:47.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.296+0000 7f4c719f4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e0b34c680 0x558e0b34ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:47.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.296+0000 7f4c719f4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558e0b2a72e0 con 0x558e0b34c680 2024-09-19T09:03:47.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.296+0000 7f4c719f4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558e0b24ece0 con 0x558e0b3414a0 2024-09-19T09:03:47.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.296+0000 7f4c719f4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558e0b2a32a0 con 0x558e0b341dc0 2024-09-19T09:03:47.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.296+0000 7f4c6dbb8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e0b3414a0 0x558e0b341880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:47.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.296+0000 7f4c6dbb8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e0b3414a0 0x558e0b341880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59906/0 (socket says 172.21.15.120:59906) 2024-09-19T09:03:47.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.296+0000 7f4c6dbb8640 1 -- 172.21.15.120:0/3523128960 learned_addr learned my addr 172.21.15.120:0/3523128960 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:47.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.297+0000 7f4c6d3b7640 1 --2- 172.21.15.120:0/3523128960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e0b341dc0 0x558e0b34c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:47.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.297+0000 7f4c6cbb6640 1 --2- 172.21.15.120:0/3523128960 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e0b34c680 0x558e0b34ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:47.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.297+0000 7f4c5e7fc640 1 --2- 172.21.15.120:0/2218082481 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4c540a3230 0x7f4c540a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:47.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.297+0000 7f4c6d3b7640 1 -- 172.21.15.120:0/3523128960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e0b3414a0 msgr2=0x558e0b341880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:47.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.297+0000 7f4c6d3b7640 1 --2- 172.21.15.120:0/3523128960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e0b3414a0 0x558e0b341880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.297+0000 7f4c6d3b7640 1 -- 172.21.15.120:0/3523128960 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e0b34c680 msgr2=0x558e0b34ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:47.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.297+0000 7f4c6d3b7640 1 --2- 172.21.15.120:0/3523128960 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e0b34c680 0x558e0b34ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.297+0000 7f4c6d3b7640 1 -- 172.21.15.120:0/3523128960 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e0b233440 con 0x558e0b341dc0 2024-09-19T09:03:47.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.297+0000 7f4c6dbb8640 1 --2- 172.21.15.120:0/3523128960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e0b3414a0 0x558e0b341880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:03:47.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.298+0000 7f4c6dbb8640 1 --2- 172.21.15.120:0/2218082481 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4c540a3230 0x7f4c540a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:47.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.298+0000 7f4c6d3b7640 1 --2- 172.21.15.120:0/3523128960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e0b341dc0 0x558e0b34c140 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4c640ccc80 tx=0x7f4c640af920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dbd32ca49b0aaf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:47.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.298+0000 7f4c5e7fc640 1 -- 172.21.15.120:0/2218082481 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f4c640cf340 con 0x7f4c68100e20 2024-09-19T09:03:47.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.298+0000 7f4c6dbb8640 1 --2- 172.21.15.120:0/2218082481 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4c540a3230 0x7f4c540a56f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4c60005c40 tx=0x7f4c60004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:47.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.298+0000 7f4c537fe640 1 -- 172.21.15.120:0/3523128960 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c64020070 con 0x558e0b341dc0 2024-09-19T09:03:47.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.298+0000 7f4c537fe640 1 -- 172.21.15.120:0/3523128960 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c640c7770 con 0x558e0b341dc0 2024-09-19T09:03:47.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.299+0000 7f4c719f4740 1 -- 172.21.15.120:0/3523128960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e0b341dc0 msgr2=0x558e0b34c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:47.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.299+0000 7f4c719f4740 1 --2- 172.21.15.120:0/3523128960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e0b341dc0 0x558e0b34c140 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4c640ccc80 tx=0x7f4c640af920 comp rx=0 tx=0).stop 2024-09-19T09:03:47.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.299+0000 7f4c719f4740 1 -- 172.21.15.120:0/3523128960 shutdown_connections 2024-09-19T09:03:47.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.299+0000 7f4c719f4740 1 --2- 172.21.15.120:0/3523128960 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e0b34c680 0x558e0b34ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.299+0000 7f4c719f4740 1 --2- 172.21.15.120:0/3523128960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e0b341dc0 0x558e0b34c140 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.299+0000 7f4c719f4740 1 --2- 172.21.15.120:0/3523128960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e0b3414a0 0x558e0b341880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.299+0000 7f4c719f4740 1 -- 172.21.15.120:0/3523128960 >> 172.21.15.120:0/3523128960 conn(0x558e0b33da30 msgr2=0x558e0b33de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:47.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.300+0000 7f4c719f4740 1 -- 172.21.15.120:0/3523128960 shutdown_connections 2024-09-19T09:03:47.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.300+0000 7f4c719f4740 1 -- 172.21.15.120:0/3523128960 wait complete. 2024-09-19T09:03:47.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.300+0000 7f4c719f4740 1 -- start start 2024-09-19T09:03:47.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.300+0000 7f4c719f4740 1 Processor -- start 2024-09-19T09:03:47.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.301+0000 7f4c719f4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e0b3414a0 0x558e0b34c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:47.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.301+0000 7f4c719f4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e0b34c680 0x558e0b3493c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:47.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.301+0000 7f4c719f4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e0b349ab0 0x558e0b3586c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:47.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.302+0000 7f4c6d3b7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e0b3414a0 0x558e0b34c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:47.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.302+0000 7f4c6d3b7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e0b3414a0 0x558e0b34c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51682/0 (socket says 172.21.15.120:51682) 2024-09-19T09:03:47.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.302+0000 7f4c6dbb8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e0b349ab0 0x558e0b3586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:47.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.302+0000 7f4c6cbb6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e0b34c680 0x558e0b3493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:47.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.302+0000 7f4c6cbb6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e0b34c680 0x558e0b3493c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59912/0 (socket says 172.21.15.120:59912) 2024-09-19T09:03:47.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.302+0000 7f4c6dbb8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e0b349ab0 0x558e0b3586c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:50700/0 (socket says 172.21.15.120:50700) 2024-09-19T09:03:47.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.302+0000 7f4c6d3b7640 1 -- 172.21.15.120:0/1859220863 learned_addr learned my addr 172.21.15.120:0/1859220863 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:47.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.302+0000 7f4c719f4740 1 -- 172.21.15.120:0/1859220863 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558e0b34f250 con 0x558e0b3414a0 2024-09-19T09:03:47.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.302+0000 7f4c719f4740 1 -- 172.21.15.120:0/1859220863 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558e0b262050 con 0x558e0b34c680 2024-09-19T09:03:47.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.302+0000 7f4c719f4740 1 -- 172.21.15.120:0/1859220863 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558e0b34f3d0 con 0x558e0b349ab0 2024-09-19T09:03:47.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.302+0000 7f4c6d3b7640 1 -- 172.21.15.120:0/1859220863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e0b349ab0 msgr2=0x558e0b3586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:47.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.302+0000 7f4c6d3b7640 1 --2- 172.21.15.120:0/1859220863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e0b349ab0 0x558e0b3586c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.303+0000 7f4c6d3b7640 1 -- 172.21.15.120:0/1859220863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e0b34c680 msgr2=0x558e0b3493c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:47.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.303+0000 7f4c6d3b7640 1 --2- 172.21.15.120:0/1859220863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e0b34c680 0x558e0b3493c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.303+0000 7f4c6d3b7640 1 -- 172.21.15.120:0/1859220863 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e0b0183c0 con 0x558e0b3414a0 2024-09-19T09:03:47.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.302+0000 7f4c719f4740 10 client.0 _ll_register_callbacks cb 0x558e0b2a2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:47.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.303+0000 7f4c6d3b7640 1 --2- 172.21.15.120:0/1859220863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e0b3414a0 0x558e0b34c110 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4c64010fd0 tx=0x7f4c64005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92704595c83c912 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:47.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.303+0000 7f4c719f4740 1 -- 172.21.15.120:0/1859220863 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c640c7770 con 0x558e0b3414a0 2024-09-19T09:03:47.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.303+0000 7f4c50ff9640 10 client.15702 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:47.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.303+0000 7f4c50ff9640 1 -- 172.21.15.120:0/1859220863 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c64020040 con 0x558e0b3414a0 2024-09-19T09:03:47.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.303+0000 7f4c50ff9640 1 -- 172.21.15.120:0/1859220863 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f4c64097e30 con 0x558e0b3414a0 2024-09-19T09:03:47.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.303+0000 7f4c50ff9640 1 -- 172.21.15.120:0/1859220863 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4c6409b590 con 0x558e0b3414a0 2024-09-19T09:03:47.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.303+0000 7f4c719f4740 10 client.15702 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:47.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.303+0000 7f4c719f4740 1 -- 172.21.15.120:0/1859220863 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({fsmap=28}) -- 0x558e0b35af80 con 0x558e0b3414a0 2024-09-19T09:03:47.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.304+0000 7f4c50ff9640 1 -- 172.21.15.120:0/1859220863 <== mon.0 v2:172.21.15.120:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4c64023060 con 0x558e0b3414a0 2024-09-19T09:03:47.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.304+0000 7f4c719f4740 10 client.15702 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:47.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.304+0000 7f4c719f4740 10 client.15702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:03:47.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.304+0000 7f4c719f4740 20 client.15702 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:47.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.304+0000 7f4c719f4740 1 --2- 172.21.15.120:0/1859220863 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558e0b35c2d0 0x558e0b35e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:47.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.304+0000 7f4c719f4740 4 client.15702 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:03:47.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.304+0000 7f4c719f4740 1 -- 172.21.15.120:0/1859220863 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e0b2b4850 con 0x558e0b35c2d0 2024-09-19T09:03:47.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.304+0000 7f4c6cbb6640 1 --2- 172.21.15.120:0/1859220863 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558e0b35c2d0 0x558e0b35e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:47.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.305+0000 7f4c50ff9640 10 client.15702 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:47.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.305+0000 7f4c6cbb6640 1 --2- 172.21.15.120:0/1859220863 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558e0b35c2d0 0x558e0b35e6c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b42e316bb8a7533b server_cookie=a2e9ce56bc2e9ccd in_seq=0 out_seq=0 2024-09-19T09:03:47.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.306+0000 7f4c50ff9640 1 -- 172.21.15.120:0/1859220863 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e0b2b4850 con 0x558e0b35c2d0 2024-09-19T09:03:47.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.306+0000 7f4c50ff9640 10 client.15702 handle_command_reply: tid=0 2024-09-19T09:03:47.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.306+0000 7f4c719f4740 1 client.15702 shutdown 2024-09-19T09:03:47.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.306+0000 7f4c719f4740 1 -- 172.21.15.120:0/1859220863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e0b3414a0 msgr2=0x558e0b34c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:47.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.306+0000 7f4c719f4740 1 --2- 172.21.15.120:0/1859220863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e0b3414a0 0x558e0b34c110 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4c64010fd0 tx=0x7f4c64005880 comp rx=0 tx=0).stop 2024-09-19T09:03:47.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.306+0000 7f4c719f4740 1 -- 172.21.15.120:0/1859220863 shutdown_connections 2024-09-19T09:03:47.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.306+0000 7f4c719f4740 1 --2- 172.21.15.120:0/1859220863 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558e0b35c2d0 0x558e0b35e6c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.306+0000 7f4c719f4740 1 --2- 172.21.15.120:0/1859220863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e0b349ab0 0x558e0b3586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.306+0000 7f4c50ff9640 0 client.15702 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:47.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.306+0000 7f4c719f4740 1 --2- 172.21.15.120:0/1859220863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e0b34c680 0x558e0b3493c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.306+0000 7f4c719f4740 1 --2- 172.21.15.120:0/1859220863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e0b3414a0 0x558e0b34c110 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.306+0000 7f4c719f4740 1 -- 172.21.15.120:0/1859220863 >> 172.21.15.120:0/1859220863 conn(0x558e0b33da30 msgr2=0x558e0b33cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:47.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.307+0000 7f4c719f4740 1 -- 172.21.15.120:0/1859220863 shutdown_connections 2024-09-19T09:03:47.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.307+0000 7f4c719f4740 1 -- 172.21.15.120:0/1859220863 wait complete. 2024-09-19T09:03:47.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.307+0000 7f4c719f4740 20 client.15702 trim_cache size 0 max 16384 2024-09-19T09:03:47.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.340+0000 7f4c719f4740 1 Processor -- start 2024-09-19T09:03:47.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.340+0000 7f4c719f4740 1 -- start start 2024-09-19T09:03:47.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.341+0000 7f4c719f4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e0b3414a0 0x558e0b3616f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:47.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.341+0000 7f4c719f4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e0b349ab0 0x558e0b361c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:47.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.341+0000 7f4c719f4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e0b34c680 0x558e0b346620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:47.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.341+0000 7f4c719f4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681ab790 con 0x558e0b349ab0 2024-09-19T09:03:47.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.341+0000 7f4c719f4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681ad2d0 con 0x558e0b3414a0 2024-09-19T09:03:47.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.341+0000 7f4c719f4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681abe60 con 0x558e0b34c680 2024-09-19T09:03:47.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.341+0000 7f4c6cbb6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e0b349ab0 0x558e0b361c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:47.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.341+0000 7f4c6cbb6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e0b349ab0 0x558e0b361c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51686/0 (socket says 172.21.15.120:51686) 2024-09-19T09:03:47.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.341+0000 7f4c6cbb6640 1 -- 172.21.15.120:0/2274484842 learned_addr learned my addr 172.21.15.120:0/2274484842 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:47.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.341+0000 7f4c6dbb8640 1 --2- 172.21.15.120:0/2274484842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e0b34c680 0x558e0b346620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:47.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.341+0000 7f4c6d3b7640 1 --2- 172.21.15.120:0/2274484842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e0b3414a0 0x558e0b3616f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:47.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.341+0000 7f4c6cbb6640 1 -- 172.21.15.120:0/2274484842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e0b34c680 msgr2=0x558e0b346620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:47.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.341+0000 7f4c6cbb6640 1 --2- 172.21.15.120:0/2274484842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e0b34c680 0x558e0b346620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.341+0000 7f4c6cbb6640 1 -- 172.21.15.120:0/2274484842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e0b3414a0 msgr2=0x558e0b3616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:47.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.341+0000 7f4c6cbb6640 1 --2- 172.21.15.120:0/2274484842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e0b3414a0 0x558e0b3616f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.341+0000 7f4c6cbb6640 1 -- 172.21.15.120:0/2274484842 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e0b33d2e0 con 0x558e0b349ab0 2024-09-19T09:03:47.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.342+0000 7f4c6cbb6640 1 --2- 172.21.15.120:0/2274484842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e0b349ab0 0x558e0b361c30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4c580314e0 tx=0x7f4c580058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa43c06de310a5a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:47.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.342+0000 7f4c52ffd640 1 -- 172.21.15.120:0/2274484842 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c58035040 con 0x558e0b349ab0 2024-09-19T09:03:47.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.342+0000 7f4c52ffd640 1 -- 172.21.15.120:0/2274484842 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c58004750 con 0x558e0b349ab0 2024-09-19T09:03:47.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.342+0000 7f4c52ffd640 1 -- 172.21.15.120:0/2274484842 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c58003850 con 0x558e0b349ab0 2024-09-19T09:03:47.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.343+0000 7f4c719f4740 1 -- 172.21.15.120:0/2274484842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e0b349ab0 msgr2=0x558e0b361c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:47.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.343+0000 7f4c719f4740 1 --2- 172.21.15.120:0/2274484842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e0b349ab0 0x558e0b361c30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4c580314e0 tx=0x7f4c580058d0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.343+0000 7f4c719f4740 1 -- 172.21.15.120:0/2274484842 shutdown_connections 2024-09-19T09:03:47.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.343+0000 7f4c719f4740 1 --2- 172.21.15.120:0/2274484842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e0b34c680 0x558e0b346620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.343+0000 7f4c719f4740 1 --2- 172.21.15.120:0/2274484842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e0b349ab0 0x558e0b361c30 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.343+0000 7f4c719f4740 1 --2- 172.21.15.120:0/2274484842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e0b3414a0 0x558e0b3616f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.343+0000 7f4c719f4740 1 -- 172.21.15.120:0/2274484842 >> 172.21.15.120:0/2274484842 conn(0x558e0b33da30 msgr2=0x558e0b33cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:47.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.343+0000 7f4c719f4740 1 -- 172.21.15.120:0/2274484842 shutdown_connections 2024-09-19T09:03:47.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.344+0000 7f4c719f4740 1 -- 172.21.15.120:0/2274484842 wait complete. 2024-09-19T09:03:47.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.344+0000 7f4c719f4740 1 -- start start 2024-09-19T09:03:47.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.344+0000 7f4c719f4740 1 Processor -- start 2024-09-19T09:03:47.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.345+0000 7f4c719f4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e0b3414a0 0x558e0b3baba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:47.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.345+0000 7f4c719f4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e0b349ab0 0x558e0b3bb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:47.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.345+0000 7f4c719f4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e0b34c680 0x558e0b3c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:47.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.345+0000 7f4c719f4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681acc00 con 0x558e0b349ab0 2024-09-19T09:03:47.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.345+0000 7f4c719f4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681ac530 con 0x558e0b34c680 2024-09-19T09:03:47.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.345+0000 7f4c719f4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681a9750 con 0x558e0b3414a0 2024-09-19T09:03:47.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.345+0000 7f4c719f4740 10 client.0 _ll_register_callbacks cb 0x7f4c60009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:47.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.345+0000 7f4c6cbb6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e0b349ab0 0x558e0b3bb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:47.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.345+0000 7f4c6cbb6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e0b349ab0 0x558e0b3bb0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51692/0 (socket says 172.21.15.120:51692) 2024-09-19T09:03:47.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.345+0000 7f4c6cbb6640 1 -- 172.21.15.120:0/619881327 learned_addr learned my addr 172.21.15.120:0/619881327 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:47.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.345+0000 7f4c6d3b7640 1 --2- 172.21.15.120:0/619881327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e0b3414a0 0x558e0b3baba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:47.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.345+0000 7f4c6dbb8640 1 --2- 172.21.15.120:0/619881327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e0b34c680 0x558e0b3c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:47.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.345+0000 7f4c6cbb6640 1 -- 172.21.15.120:0/619881327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e0b3414a0 msgr2=0x558e0b3baba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:47.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.345+0000 7f4c6cbb6640 1 --2- 172.21.15.120:0/619881327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e0b3414a0 0x558e0b3baba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.345+0000 7f4c6cbb6640 1 -- 172.21.15.120:0/619881327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e0b34c680 msgr2=0x558e0b3c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:47.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.345+0000 7f4c6cbb6640 1 --2- 172.21.15.120:0/619881327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e0b34c680 0x558e0b3c5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.345+0000 7f4c6cbb6640 1 -- 172.21.15.120:0/619881327 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e0b3c6160 con 0x558e0b349ab0 2024-09-19T09:03:47.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.345+0000 7f4c6d3b7640 1 --2- 172.21.15.120:0/619881327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e0b3414a0 0x558e0b3baba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:03:47.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.346+0000 7f4c6cbb6640 1 --2- 172.21.15.120:0/619881327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e0b349ab0 0x558e0b3bb0e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009850 tx=0x7f4c58041c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb8fd3b530ec9515 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:47.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.346+0000 7f4c50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:47.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.346+0000 7f4c50ff9640 1 -- 172.21.15.120:0/619881327 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c58064060 con 0x558e0b349ab0 2024-09-19T09:03:47.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.346+0000 7f4c50ff9640 1 -- 172.21.15.120:0/619881327 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f4c58004b90 con 0x558e0b349ab0 2024-09-19T09:03:47.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.346+0000 7f4c719f4740 1 -- 172.21.15.120:0/619881327 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c58004750 con 0x558e0b349ab0 2024-09-19T09:03:47.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.346+0000 7f4c50ff9640 1 -- 172.21.15.120:0/619881327 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4c58035050 con 0x558e0b349ab0 2024-09-19T09:03:47.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.346+0000 7f4c719f4740 10 client.15708 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:47.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.346+0000 7f4c719f4740 1 -- 172.21.15.120:0/619881327 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558e0b3c73b0 con 0x558e0b349ab0 2024-09-19T09:03:47.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.346+0000 7f4c50ff9640 1 -- 172.21.15.120:0/619881327 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c58004750 con 0x558e0b349ab0 2024-09-19T09:03:47.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.347+0000 7f4c50ff9640 1 -- 172.21.15.120:0/619881327 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4c58069070 con 0x558e0b349ab0 2024-09-19T09:03:47.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.347+0000 7f4c719f4740 10 client.15708 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:47.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.347+0000 7f4c719f4740 10 client.15708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:03:47.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.347+0000 7f4c719f4740 20 client.15708 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:47.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.347+0000 7f4c719f4740 1 --2- 172.21.15.120:0/619881327 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558e0b35c2d0 0x558e0b3c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:47.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.347+0000 7f4c719f4740 4 client.15708 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:03:47.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.347+0000 7f4c719f4740 1 -- 172.21.15.120:0/619881327 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558e0b2aed60 con 0x558e0b35c2d0 2024-09-19T09:03:47.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.347+0000 7f4c6d3b7640 1 --2- 172.21.15.120:0/619881327 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558e0b35c2d0 0x558e0b3c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:47.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.348+0000 7f4c50ff9640 10 client.15708 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:47.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.348+0000 7f4c6d3b7640 1 --2- 172.21.15.120:0/619881327 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558e0b35c2d0 0x558e0b3c9b10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=596d9ff0f289f0fe server_cookie=5c9f59bbd0fc89f3 in_seq=0 out_seq=0 2024-09-19T09:03:47.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.348+0000 7f4c50ff9640 1 -- 172.21.15.120:0/619881327 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558e0b2aed60 con 0x558e0b35c2d0 2024-09-19T09:03:47.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.348+0000 7f4c50ff9640 10 client.15708 handle_command_reply: tid=0 2024-09-19T09:03:47.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.348+0000 7f4c719f4740 1 client.15708 shutdown 2024-09-19T09:03:47.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.348+0000 7f4c719f4740 1 -- 172.21.15.120:0/619881327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e0b349ab0 msgr2=0x558e0b3bb0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:47.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.348+0000 7f4c719f4740 1 --2- 172.21.15.120:0/619881327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e0b349ab0 0x558e0b3bb0e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009850 tx=0x7f4c58041c70 comp rx=0 tx=0).stop 2024-09-19T09:03:47.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.348+0000 7f4c719f4740 1 -- 172.21.15.120:0/619881327 shutdown_connections 2024-09-19T09:03:47.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.348+0000 7f4c719f4740 1 --2- 172.21.15.120:0/619881327 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558e0b35c2d0 0x558e0b3c9b10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.348+0000 7f4c719f4740 1 --2- 172.21.15.120:0/619881327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e0b34c680 0x558e0b3c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.352 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:03:47.352 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:03:47.352 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-19T09:03:47.353 INFO:teuthology.orchestra.run.smithi120.stdout: "56e704ee-8828-40ca-8d0d-c9ecdb8b4627": { 2024-09-19T09:03:47.353 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-19T09:03:47.353 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "56e704ee-8828-40ca-8d0d-c9ecdb8b4627", 2024-09-19T09:03:47.353 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-19T09:03:47.353 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:03:47.353 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:03:47.353 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:03:47.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.348+0000 7f4c719f4740 1 --2- 172.21.15.120:0/619881327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e0b349ab0 0x558e0b3bb0e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.348+0000 7f4c50ff9640 0 client.15708 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:47.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.348+0000 7f4c719f4740 1 --2- 172.21.15.120:0/619881327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e0b3414a0 0x558e0b3baba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.348+0000 7f4c719f4740 1 -- 172.21.15.120:0/619881327 >> 172.21.15.120:0/619881327 conn(0x558e0b33da30 msgr2=0x558e0b34ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:47.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.349+0000 7f4c719f4740 1 -- 172.21.15.120:0/619881327 shutdown_connections 2024-09-19T09:03:47.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.349+0000 7f4c719f4740 1 -- 172.21.15.120:0/619881327 wait complete. 2024-09-19T09:03:47.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.349+0000 7f4c719f4740 20 client.15708 trim_cache size 0 max 16384 2024-09-19T09:03:47.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.349+0000 7f4c6f61b640 1 -- 172.21.15.120:0/2218082481 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4c540a3230 msgr2=0x7f4c540a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:47.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.349+0000 7f4c6f61b640 1 --2- 172.21.15.120:0/2218082481 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4c540a3230 0x7f4c540a56f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4c60005c40 tx=0x7f4c60004a40 comp rx=0 tx=0).stop 2024-09-19T09:03:47.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.349+0000 7f4c6f61b640 1 -- 172.21.15.120:0/2218082481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c68100e20 msgr2=0x7f4c681986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:47.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.349+0000 7f4c6f61b640 1 --2- 172.21.15.120:0/2218082481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c68100e20 0x7f4c681986e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4c6400d950 tx=0x7f4c6400de10 comp rx=0 tx=0).stop 2024-09-19T09:03:47.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.349+0000 7f4c6f61b640 1 -- 172.21.15.120:0/2218082481 shutdown_connections 2024-09-19T09:03:47.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.350+0000 7f4c6f61b640 1 --2- 172.21.15.120:0/2218082481 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4c540a3230 0x7f4c540a56f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.350+0000 7f4c6f61b640 1 --2- 172.21.15.120:0/2218082481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c6819cfb0 0x7f4c6819d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.350+0000 7f4c6f61b640 1 --2- 172.21.15.120:0/2218082481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c68102ef0 0x7f4c68198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.350+0000 7f4c6f61b640 1 --2- 172.21.15.120:0/2218082481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c68100e20 0x7f4c681986e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:47.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.350+0000 7f4c6f61b640 1 -- 172.21.15.120:0/2218082481 >> 172.21.15.120:0/2218082481 conn(0x7f4c680fc7d0 msgr2=0x7f4c680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:47.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.350+0000 7f4c6f61b640 1 -- 172.21.15.120:0/2218082481 shutdown_connections 2024-09-19T09:03:47.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:03:47.350+0000 7f4c6f61b640 1 -- 172.21.15.120:0/2218082481 wait complete. 2024-09-19T09:03:48.124 INFO:tasks.cephfs.filesystem:scrub status for tag:56e704ee-8828-40ca-8d0d-c9ecdb8b4627 - {'path': '/', 'tag': '56e704ee-8828-40ca-8d0d-c9ecdb8b4627', 'options': 'recursive,force'} 2024-09-19T09:03:49.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:48 smithi120 ceph-mon[31150]: pgmap v369: 97 pgs: 97 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-09-19T09:03:49.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:48 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:03:49.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:48 smithi196 ceph-mon[35150]: pgmap v369: 97 pgs: 97 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-09-19T09:03:49.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:48 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:03:49.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:48 smithi153 ceph-mon[35071]: pgmap v369: 97 pgs: 97 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-09-19T09:03:49.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:48 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:03:51.060 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:50 smithi120 ceph-mon[31150]: pgmap v370: 97 pgs: 97 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-19T09:03:51.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:50 smithi196 ceph-mon[35150]: pgmap v370: 97 pgs: 97 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-19T09:03:51.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:50 smithi153 ceph-mon[35071]: pgmap v370: 97 pgs: 97 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-19T09:03:53.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:52 smithi120 ceph-mon[31150]: pgmap v371: 97 pgs: 97 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 853 B/s wr, 14 op/s 2024-09-19T09:03:53.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:52 smithi120 ceph-mon[31150]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-19T09:03:53.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:52 smithi196 ceph-mon[35150]: pgmap v371: 97 pgs: 97 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 853 B/s wr, 14 op/s 2024-09-19T09:03:53.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:52 smithi196 ceph-mon[35150]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-19T09:03:53.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:52 smithi153 ceph-mon[35071]: pgmap v371: 97 pgs: 97 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 853 B/s wr, 14 op/s 2024-09-19T09:03:53.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:52 smithi153 ceph-mon[35071]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-19T09:03:55.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:54 smithi120 ceph-mon[31150]: pgmap v372: 97 pgs: 97 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-19T09:03:55.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:54 smithi196 ceph-mon[35150]: pgmap v372: 97 pgs: 97 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-19T09:03:55.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:54 smithi153 ceph-mon[35071]: pgmap v372: 97 pgs: 97 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-19T09:03:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.248+0000 7f3c66fc4640 1 -- 172.21.15.120:0/2461955324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c60101b60 msgr2=0x7f3c60101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.248+0000 7f3c66fc4640 1 --2- 172.21.15.120:0/2461955324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c60101b60 0x7f3c60101f60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3c54009900 tx=0x7f3c5402a410 comp rx=0 tx=0).stop 2024-09-19T09:03:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.248+0000 7f3c66fc4640 1 -- 172.21.15.120:0/2461955324 shutdown_connections 2024-09-19T09:03:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.248+0000 7f3c66fc4640 1 --2- 172.21.15.120:0/2461955324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c60103710 0x7f3c60109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.248+0000 7f3c66fc4640 1 --2- 172.21.15.120:0/2461955324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c60102d50 0x7f3c601031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.248+0000 7f3c66fc4640 1 --2- 172.21.15.120:0/2461955324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c60101b60 0x7f3c60101f60 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.248+0000 7f3c66fc4640 1 -- 172.21.15.120:0/2461955324 >> 172.21.15.120:0/2461955324 conn(0x7f3c600fd2e0 msgr2=0x7f3c600ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.248+0000 7f3c66fc4640 1 -- 172.21.15.120:0/2461955324 shutdown_connections 2024-09-19T09:03:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.248+0000 7f3c66fc4640 1 -- 172.21.15.120:0/2461955324 wait complete. 2024-09-19T09:03:55.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.249+0000 7f3c66fc4640 1 Processor -- start 2024-09-19T09:03:55.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.249+0000 7f3c66fc4640 1 -- start start 2024-09-19T09:03:55.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.249+0000 7f3c66fc4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c60101b60 0x7f3c60197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:55.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.249+0000 7f3c66fc4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c60102d50 0x7f3c60198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:55.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.249+0000 7f3c66fc4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c60103710 0x7f3c6019f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:55.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.250+0000 7f3c66fc4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6010d7e0 con 0x7f3c60101b60 2024-09-19T09:03:55.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.250+0000 7f3c66fc4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6010d660 con 0x7f3c60102d50 2024-09-19T09:03:55.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.250+0000 7f3c64d60640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c60101b60 0x7f3c60197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:55.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.250+0000 7f3c66fc4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6010d960 con 0x7f3c60103710 2024-09-19T09:03:55.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.250+0000 7f3c64d60640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c60101b60 0x7f3c60197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37734/0 (socket says 172.21.15.120:37734) 2024-09-19T09:03:55.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.250+0000 7f3c64d60640 1 -- 172.21.15.120:0/3289000420 learned_addr learned my addr 172.21.15.120:0/3289000420 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.250+0000 7f3c5ffff640 1 --2- 172.21.15.120:0/3289000420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c60102d50 0x7f3c60198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.250+0000 7f3c65561640 1 --2- 172.21.15.120:0/3289000420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c60103710 0x7f3c6019f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.250+0000 7f3c64d60640 1 -- 172.21.15.120:0/3289000420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c60103710 msgr2=0x7f3c6019f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.250+0000 7f3c64d60640 1 --2- 172.21.15.120:0/3289000420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c60103710 0x7f3c6019f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.250+0000 7f3c64d60640 1 -- 172.21.15.120:0/3289000420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c60102d50 msgr2=0x7f3c60198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.250+0000 7f3c64d60640 1 --2- 172.21.15.120:0/3289000420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c60102d50 0x7f3c60198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.250+0000 7f3c64d60640 1 -- 172.21.15.120:0/3289000420 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c6019fa20 con 0x7f3c60101b60 2024-09-19T09:03:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.250+0000 7f3c64d60640 1 --2- 172.21.15.120:0/3289000420 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c60101b60 0x7f3c60197e00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3c54031610 tx=0x7f3c540043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.250+0000 7f3c5dffb640 1 -- 172.21.15.120:0/3289000420 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5403f030 con 0x7f3c60101b60 2024-09-19T09:03:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.251+0000 7f3c66fc4640 1 -- 172.21.15.120:0/3289000420 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c60074420 con 0x7f3c60101b60 2024-09-19T09:03:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.251+0000 7f3c5dffb640 1 -- 172.21.15.120:0/3289000420 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c54003ce0 con 0x7f3c60101b60 2024-09-19T09:03:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.251+0000 7f3c5dffb640 1 -- 172.21.15.120:0/3289000420 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c54035070 con 0x7f3c60101b60 2024-09-19T09:03:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.251+0000 7f3c66fc4640 1 -- 172.21.15.120:0/3289000420 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c60074960 con 0x7f3c60101b60 2024-09-19T09:03:55.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.254+0000 7f3c69e40740 1 Processor -- start 2024-09-19T09:03:55.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.254+0000 7f3c69e40740 1 -- start start 2024-09-19T09:03:55.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.254+0000 7f3c5dffb640 1 -- 172.21.15.120:0/3289000420 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f3c54042030 con 0x7f3c60101b60 2024-09-19T09:03:55.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.254+0000 7f3c69e40740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5068a19a0 0x55d5068a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:55.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.254+0000 7f3c69e40740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5068a22e0 0x55d5068a6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:55.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.254+0000 7f3c69e40740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5068a7370 0x55d5068a9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:55.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.254+0000 7f3c69e40740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d5068052f0 con 0x55d5068a22e0 2024-09-19T09:03:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.254+0000 7f3c69e40740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d5067ad270 con 0x55d5068a19a0 2024-09-19T09:03:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.254+0000 7f3c69e40740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d5068012b0 con 0x55d5068a7370 2024-09-19T09:03:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.254+0000 7f3c65561640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5068a19a0 0x55d5068a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.255+0000 7f3c5ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5068a7370 0x55d5068a9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.255+0000 7f3c64d60640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5068a22e0 0x55d5068a6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.255+0000 7f3c65561640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5068a19a0 0x55d5068a1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:49692/0 (socket says 172.21.15.120:49692) 2024-09-19T09:03:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.255+0000 7f3c65561640 1 -- 172.21.15.120:0/2450041791 learned_addr learned my addr 172.21.15.120:0/2450041791 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.255+0000 7f3c64d60640 1 -- 172.21.15.120:0/2450041791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5068a7370 msgr2=0x55d5068a9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.255+0000 7f3c64d60640 1 --2- 172.21.15.120:0/2450041791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5068a7370 0x55d5068a9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.255+0000 7f3c64d60640 1 -- 172.21.15.120:0/2450041791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5068a19a0 msgr2=0x55d5068a1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.255+0000 7f3c64d60640 1 --2- 172.21.15.120:0/2450041791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5068a19a0 0x55d5068a1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.255+0000 7f3c64d60640 1 -- 172.21.15.120:0/2450041791 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5067ac340 con 0x55d5068a22e0 2024-09-19T09:03:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.255+0000 7f3c65561640 1 --2- 172.21.15.120:0/2450041791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5068a19a0 0x55d5068a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:03:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.255+0000 7f3c64d60640 1 --2- 172.21.15.120:0/2450041791 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5068a22e0 0x55d5068a6da0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3c540ed570 tx=0x7f3c540f3c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a301a5a1763e3994 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.255+0000 7f3c4affd640 1 -- 172.21.15.120:0/2450041791 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c54103070 con 0x55d5068a22e0 2024-09-19T09:03:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.255+0000 7f3c4affd640 1 -- 172.21.15.120:0/2450041791 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c540f8050 con 0x55d5068a22e0 2024-09-19T09:03:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.255+0000 7f3c4affd640 1 -- 172.21.15.120:0/2450041791 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c540fca60 con 0x55d5068a22e0 2024-09-19T09:03:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.256+0000 7f3c5dffb640 1 --2- 172.21.15.120:0/3289000420 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3c500a32f0 0x7f3c500a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.256+0000 7f3c65561640 1 --2- 172.21.15.120:0/3289000420 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3c500a32f0 0x7f3c500a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.256+0000 7f3c5dffb640 1 -- 172.21.15.120:0/3289000420 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f3c540efad0 con 0x7f3c60101b60 2024-09-19T09:03:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.256+0000 7f3c69e40740 1 -- 172.21.15.120:0/2450041791 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5068a22e0 msgr2=0x55d5068a6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.256+0000 7f3c69e40740 1 --2- 172.21.15.120:0/2450041791 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5068a22e0 0x55d5068a6da0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3c540ed570 tx=0x7f3c540f3c20 comp rx=0 tx=0).stop 2024-09-19T09:03:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.256+0000 7f3c65561640 1 --2- 172.21.15.120:0/3289000420 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3c500a32f0 0x7f3c500a57b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3c58005f40 tx=0x7f3c58012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.256+0000 7f3c69e40740 1 -- 172.21.15.120:0/2450041791 shutdown_connections 2024-09-19T09:03:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.256+0000 7f3c69e40740 1 --2- 172.21.15.120:0/2450041791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5068a7370 0x55d5068a9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.256+0000 7f3c69e40740 1 --2- 172.21.15.120:0/2450041791 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5068a22e0 0x55d5068a6da0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.256+0000 7f3c69e40740 1 --2- 172.21.15.120:0/2450041791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5068a19a0 0x55d5068a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.256+0000 7f3c69e40740 1 -- 172.21.15.120:0/2450041791 >> 172.21.15.120:0/2450041791 conn(0x55d50689d010 msgr2=0x55d50689d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.257+0000 7f3c69e40740 1 -- 172.21.15.120:0/2450041791 shutdown_connections 2024-09-19T09:03:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.257+0000 7f3c69e40740 1 -- 172.21.15.120:0/2450041791 wait complete. 2024-09-19T09:03:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.257+0000 7f3c69e40740 1 -- start start 2024-09-19T09:03:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.257+0000 7f3c69e40740 1 Processor -- start 2024-09-19T09:03:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.257+0000 7f3c69e40740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5068a19a0 0x55d5068ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.257+0000 7f3c69e40740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5068a22e0 0x55d5068abe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.257+0000 7f3c69e40740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5068a7370 0x55d5068b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.258+0000 7f3c69e40740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d5068aa100 con 0x55d5068a22e0 2024-09-19T09:03:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.258+0000 7f3c69e40740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d50678e290 con 0x55d5068a19a0 2024-09-19T09:03:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.258+0000 7f3c69e40740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d5068aa280 con 0x55d5068a7370 2024-09-19T09:03:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.258+0000 7f3c5ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5068a22e0 0x55d5068abe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.258+0000 7f3c69e40740 10 client.0 _ll_register_callbacks cb 0x55d5067bf140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.258+0000 7f3c5ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5068a22e0 0x55d5068abe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37752/0 (socket says 172.21.15.120:37752) 2024-09-19T09:03:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.258+0000 7f3c5ffff640 1 -- 172.21.15.120:0/3315553097 learned_addr learned my addr 172.21.15.120:0/3315553097 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.258+0000 7f3c64d60640 1 --2- 172.21.15.120:0/3315553097 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5068a19a0 0x55d5068ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.258+0000 7f3c65561640 1 --2- 172.21.15.120:0/3315553097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5068a7370 0x55d5068b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.258+0000 7f3c5ffff640 1 -- 172.21.15.120:0/3315553097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5068a7370 msgr2=0x55d5068b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.258+0000 7f3c5ffff640 1 --2- 172.21.15.120:0/3315553097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5068a7370 0x55d5068b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.258+0000 7f3c5ffff640 1 -- 172.21.15.120:0/3315553097 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5068a19a0 msgr2=0x55d5068ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.258+0000 7f3c5ffff640 1 --2- 172.21.15.120:0/3315553097 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5068a19a0 0x55d5068ab8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.258+0000 7f3c5ffff640 1 -- 172.21.15.120:0/3315553097 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d506518210 con 0x55d5068a22e0 2024-09-19T09:03:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.258+0000 7f3c64d60640 1 --2- 172.21.15.120:0/3315553097 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5068a19a0 0x55d5068ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:03:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.258+0000 7f3c5ffff640 1 --2- 172.21.15.120:0/3315553097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5068a22e0 0x55d5068abe20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c010820 tx=0x7f3c4c010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d7133837018a23 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.258+0000 7f3c3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.258+0000 7f3c3ffff640 1 -- 172.21.15.120:0/3315553097 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c00fb90 con 0x55d5068a22e0 2024-09-19T09:03:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.258+0000 7f3c3ffff640 1 -- 172.21.15.120:0/3315553097 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f3c4c00fd30 con 0x55d5068a22e0 2024-09-19T09:03:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.258+0000 7f3c69e40740 1 -- 172.21.15.120:0/3315553097 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c540f8050 con 0x55d5068a22e0 2024-09-19T09:03:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.259+0000 7f3c3ffff640 1 -- 172.21.15.120:0/3315553097 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3c4c005020 con 0x55d5068a22e0 2024-09-19T09:03:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.259+0000 7f3c69e40740 10 client.15720 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.259+0000 7f3c69e40740 1 -- 172.21.15.120:0/3315553097 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d5065e4620 con 0x55d5068a22e0 2024-09-19T09:03:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.259+0000 7f3c3ffff640 1 -- 172.21.15.120:0/3315553097 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c540f8050 con 0x55d5068a22e0 2024-09-19T09:03:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.259+0000 7f3c3ffff640 1 -- 172.21.15.120:0/3315553097 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3c4c019410 con 0x55d5068a22e0 2024-09-19T09:03:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.259+0000 7f3c69e40740 10 client.15720 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.259+0000 7f3c69e40740 10 client.15720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:03:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.259+0000 7f3c69e40740 20 client.15720 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.259+0000 7f3c69e40740 1 --2- 172.21.15.120:0/3315553097 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d5068b75b0 0x55d5068b9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.259+0000 7f3c69e40740 4 client.15720 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:03:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.259+0000 7f3c69e40740 1 -- 172.21.15.120:0/3315553097 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d506812a20 con 0x55d5068b75b0 2024-09-19T09:03:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.260+0000 7f3c64d60640 1 --2- 172.21.15.120:0/3315553097 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d5068b75b0 0x55d5068b9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.260+0000 7f3c3ffff640 10 client.15720 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.260+0000 7f3c64d60640 1 --2- 172.21.15.120:0/3315553097 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d5068b75b0 0x55d5068b9a00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe33d9c3836fd3e5 server_cookie=4b8a747aafa1aefe in_seq=0 out_seq=0 2024-09-19T09:03:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.261+0000 7f3c3ffff640 1 -- 172.21.15.120:0/3315553097 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d506812a20 con 0x55d5068b75b0 2024-09-19T09:03:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.261+0000 7f3c3ffff640 10 client.15720 handle_command_reply: tid=0 2024-09-19T09:03:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.261+0000 7f3c69e40740 1 client.15720 shutdown 2024-09-19T09:03:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.261+0000 7f3c69e40740 1 -- 172.21.15.120:0/3315553097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5068a22e0 msgr2=0x55d5068abe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.261+0000 7f3c69e40740 1 --2- 172.21.15.120:0/3315553097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5068a22e0 0x55d5068abe20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c010820 tx=0x7f3c4c010ce0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.261+0000 7f3c69e40740 1 -- 172.21.15.120:0/3315553097 shutdown_connections 2024-09-19T09:03:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.261+0000 7f3c69e40740 1 --2- 172.21.15.120:0/3315553097 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d5068b75b0 0x55d5068b9a00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.261+0000 7f3c69e40740 1 --2- 172.21.15.120:0/3315553097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5068a7370 0x55d5068b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.261+0000 7f3c69e40740 1 --2- 172.21.15.120:0/3315553097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5068a22e0 0x55d5068abe20 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.261+0000 7f3c69e40740 1 --2- 172.21.15.120:0/3315553097 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5068a19a0 0x55d5068ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.261+0000 7f3c69e40740 1 -- 172.21.15.120:0/3315553097 >> 172.21.15.120:0/3315553097 conn(0x55d50689d010 msgr2=0x55d50689e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.261+0000 7f3c3ffff640 0 client.15720 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.261+0000 7f3c69e40740 1 -- 172.21.15.120:0/3315553097 shutdown_connections 2024-09-19T09:03:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.261+0000 7f3c69e40740 1 -- 172.21.15.120:0/3315553097 wait complete. 2024-09-19T09:03:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.261+0000 7f3c69e40740 20 client.15720 trim_cache size 0 max 16384 2024-09-19T09:03:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.288+0000 7f3c69e40740 1 Processor -- start 2024-09-19T09:03:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.288+0000 7f3c69e40740 1 -- start start 2024-09-19T09:03:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.288+0000 7f3c69e40740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5068a19a0 0x55d50689d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.288+0000 7f3c69e40740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5068a22e0 0x55d5068b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.288+0000 7f3c69e40740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5068a7370 0x55d50689d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.288+0000 7f3c69e40740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3c601a6e50 con 0x55d5068a7370 2024-09-19T09:03:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.288+0000 7f3c69e40740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c601a8990 con 0x55d5068a22e0 2024-09-19T09:03:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.288+0000 7f3c69e40740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3c601a7520 con 0x55d5068a19a0 2024-09-19T09:03:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.289+0000 7f3c65561640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5068a7370 0x55d50689d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.289+0000 7f3c65561640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5068a7370 0x55d50689d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37762/0 (socket says 172.21.15.120:37762) 2024-09-19T09:03:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.289+0000 7f3c65561640 1 -- 172.21.15.120:0/1245096690 learned_addr learned my addr 172.21.15.120:0/1245096690 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.289+0000 7f3c64d60640 1 --2- 172.21.15.120:0/1245096690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5068a19a0 0x55d50689d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.289+0000 7f3c5ffff640 1 --2- 172.21.15.120:0/1245096690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5068a22e0 0x55d5068b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.289+0000 7f3c65561640 1 -- 172.21.15.120:0/1245096690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5068a19a0 msgr2=0x55d50689d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.289+0000 7f3c65561640 1 --2- 172.21.15.120:0/1245096690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5068a19a0 0x55d50689d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.289+0000 7f3c65561640 1 -- 172.21.15.120:0/1245096690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5068a22e0 msgr2=0x55d5068b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.289+0000 7f3c65561640 1 --2- 172.21.15.120:0/1245096690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5068a22e0 0x55d5068b2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.289+0000 7f3c65561640 1 -- 172.21.15.120:0/1245096690 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5068aa380 con 0x55d5068a7370 2024-09-19T09:03:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.289+0000 7f3c5ffff640 1 --2- 172.21.15.120:0/1245096690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5068a22e0 0x55d5068b2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:03:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.289+0000 7f3c65561640 1 --2- 172.21.15.120:0/1245096690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5068a7370 0x55d50689d930 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3c58003cc0 tx=0x7f3c5801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b6695d6ca602cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.289+0000 7f3c4a7fc640 1 -- 172.21.15.120:0/1245096690 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c58008db0 con 0x55d5068a7370 2024-09-19T09:03:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.289+0000 7f3c4a7fc640 1 -- 172.21.15.120:0/1245096690 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c5802e070 con 0x55d5068a7370 2024-09-19T09:03:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.289+0000 7f3c4a7fc640 1 -- 172.21.15.120:0/1245096690 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c58029a00 con 0x55d5068a7370 2024-09-19T09:03:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.290+0000 7f3c69e40740 1 -- 172.21.15.120:0/1245096690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5068a7370 msgr2=0x55d50689d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.290+0000 7f3c69e40740 1 --2- 172.21.15.120:0/1245096690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5068a7370 0x55d50689d930 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3c58003cc0 tx=0x7f3c5801a000 comp rx=0 tx=0).stop 2024-09-19T09:03:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.290+0000 7f3c69e40740 1 -- 172.21.15.120:0/1245096690 shutdown_connections 2024-09-19T09:03:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.290+0000 7f3c69e40740 1 --2- 172.21.15.120:0/1245096690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5068a7370 0x55d50689d930 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.290+0000 7f3c69e40740 1 --2- 172.21.15.120:0/1245096690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5068a22e0 0x55d5068b2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.290+0000 7f3c69e40740 1 --2- 172.21.15.120:0/1245096690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5068a19a0 0x55d50689d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.290+0000 7f3c69e40740 1 -- 172.21.15.120:0/1245096690 >> 172.21.15.120:0/1245096690 conn(0x55d50689d010 msgr2=0x55d5068ab8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.290+0000 7f3c69e40740 1 -- 172.21.15.120:0/1245096690 shutdown_connections 2024-09-19T09:03:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.291+0000 7f3c69e40740 1 -- 172.21.15.120:0/1245096690 wait complete. 2024-09-19T09:03:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.291+0000 7f3c69e40740 1 -- start start 2024-09-19T09:03:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.291+0000 7f3c69e40740 1 Processor -- start 2024-09-19T09:03:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.291+0000 7f3c69e40740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5068a19a0 0x55d5068afd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.291+0000 7f3c69e40740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5068a22e0 0x55d5068b0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.292+0000 7f3c69e40740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5068a7370 0x55d506920280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.292+0000 7f3c69e40740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3c601a82c0 con 0x55d5068a7370 2024-09-19T09:03:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.292+0000 7f3c69e40740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c601a7bf0 con 0x55d5068a22e0 2024-09-19T09:03:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.292+0000 7f3c69e40740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3c601a4e10 con 0x55d5068a19a0 2024-09-19T09:03:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.292+0000 7f3c65561640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5068a7370 0x55d506920280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.292+0000 7f3c64d60640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5068a19a0 0x55d5068afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.292+0000 7f3c65561640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5068a7370 0x55d506920280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37774/0 (socket says 172.21.15.120:37774) 2024-09-19T09:03:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.292+0000 7f3c64d60640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5068a19a0 0x55d5068afd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:60224/0 (socket says 172.21.15.120:60224) 2024-09-19T09:03:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.292+0000 7f3c65561640 1 -- 172.21.15.120:0/584872293 learned_addr learned my addr 172.21.15.120:0/584872293 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:03:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.292+0000 7f3c69e40740 10 client.0 _ll_register_callbacks cb 0x7f3c4c00a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:03:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.292+0000 7f3c5ffff640 1 --2- 172.21.15.120:0/584872293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5068a22e0 0x55d5068b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.292+0000 7f3c65561640 1 -- 172.21.15.120:0/584872293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5068a19a0 msgr2=0x55d5068afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.292+0000 7f3c65561640 1 --2- 172.21.15.120:0/584872293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5068a19a0 0x55d5068afd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.292+0000 7f3c65561640 1 -- 172.21.15.120:0/584872293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5068a22e0 msgr2=0x55d5068b0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:03:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.292+0000 7f3c65561640 1 --2- 172.21.15.120:0/584872293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5068a22e0 0x55d5068b0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.292+0000 7f3c65561640 1 -- 172.21.15.120:0/584872293 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5068a2f70 con 0x55d5068a7370 2024-09-19T09:03:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.292+0000 7f3c65561640 1 --2- 172.21.15.120:0/584872293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5068a7370 0x55d506920280 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3c58008600 tx=0x7f3c58004f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1e4daeac83c394 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:03:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.292+0000 7f3c3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:03:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.292+0000 7f3c3ffff640 1 -- 172.21.15.120:0/584872293 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c58004200 con 0x55d5068a7370 2024-09-19T09:03:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.292+0000 7f3c3ffff640 1 -- 172.21.15.120:0/584872293 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f3c580043a0 con 0x55d5068a7370 2024-09-19T09:03:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.293+0000 7f3c69e40740 1 -- 172.21.15.120:0/584872293 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c5802e070 con 0x55d5068a7370 2024-09-19T09:03:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.293+0000 7f3c3ffff640 1 -- 172.21.15.120:0/584872293 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3c58004620 con 0x55d5068a7370 2024-09-19T09:03:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.293+0000 7f3c69e40740 10 client.15726 fetch_fsmap learned FSMap version 28 2024-09-19T09:03:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.293+0000 7f3c69e40740 1 -- 172.21.15.120:0/584872293 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d506921910 con 0x55d5068a7370 2024-09-19T09:03:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.293+0000 7f3c3ffff640 1 -- 172.21.15.120:0/584872293 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c5802e070 con 0x55d5068a7370 2024-09-19T09:03:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.293+0000 7f3c3ffff640 1 -- 172.21.15.120:0/584872293 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3c5803c820 con 0x55d5068a7370 2024-09-19T09:03:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.293+0000 7f3c69e40740 10 client.15726 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:03:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.293+0000 7f3c69e40740 10 client.15726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:03:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.293+0000 7f3c69e40740 20 client.15726 populate_metadata read hostname 'smithi120' 2024-09-19T09:03:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.293+0000 7f3c69e40740 1 --2- 172.21.15.120:0/584872293 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d506922270 0x55d506924690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:03:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.293+0000 7f3c69e40740 4 client.15726 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:03:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.293+0000 7f3c69e40740 1 -- 172.21.15.120:0/584872293 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d50680cd70 con 0x55d506922270 2024-09-19T09:03:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.294+0000 7f3c64d60640 1 --2- 172.21.15.120:0/584872293 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d506922270 0x55d506924690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:03:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.294+0000 7f3c64d60640 1 --2- 172.21.15.120:0/584872293 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d506922270 0x55d506924690 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7906b13768a064ee server_cookie=fcb420d338e18cfc in_seq=0 out_seq=0 2024-09-19T09:03:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.294+0000 7f3c3ffff640 10 client.15726 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.300+0000 7f3c3ffff640 1 -- 172.21.15.120:0/584872293 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d50680cd70 con 0x55d506922270 2024-09-19T09:03:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.300+0000 7f3c3ffff640 10 client.15726 handle_command_reply: tid=0 2024-09-19T09:03:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.300+0000 7f3c69e40740 1 client.15726 shutdown 2024-09-19T09:03:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.300+0000 7f3c69e40740 1 -- 172.21.15.120:0/584872293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5068a7370 msgr2=0x55d506920280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.300+0000 7f3c69e40740 1 --2- 172.21.15.120:0/584872293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5068a7370 0x55d506920280 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3c58008600 tx=0x7f3c58004f40 comp rx=0 tx=0).stop 2024-09-19T09:03:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.300+0000 7f3c69e40740 1 -- 172.21.15.120:0/584872293 shutdown_connections 2024-09-19T09:03:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.300+0000 7f3c69e40740 1 --2- 172.21.15.120:0/584872293 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d506922270 0x55d506924690 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.300+0000 7f3c69e40740 1 --2- 172.21.15.120:0/584872293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5068a7370 0x55d506920280 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.300+0000 7f3c3ffff640 0 client.15726 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:03:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.300+0000 7f3c69e40740 1 --2- 172.21.15.120:0/584872293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5068a22e0 0x55d5068b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.300+0000 7f3c69e40740 1 --2- 172.21.15.120:0/584872293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5068a19a0 0x55d5068afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.300+0000 7f3c69e40740 1 -- 172.21.15.120:0/584872293 >> 172.21.15.120:0/584872293 conn(0x55d50689d010 msgr2=0x55d50689b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.300+0000 7f3c69e40740 1 -- 172.21.15.120:0/584872293 shutdown_connections 2024-09-19T09:03:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.300+0000 7f3c69e40740 1 -- 172.21.15.120:0/584872293 wait complete. 2024-09-19T09:03:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.300+0000 7f3c69e40740 20 client.15726 trim_cache size 0 max 16384 2024-09-19T09:03:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:03:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:03:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:03:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:03:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.301+0000 7f3c66fc4640 1 -- 172.21.15.120:0/3289000420 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3c500a32f0 msgr2=0x7f3c500a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.301+0000 7f3c66fc4640 1 --2- 172.21.15.120:0/3289000420 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3c500a32f0 0x7f3c500a57b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3c58005f40 tx=0x7f3c58012040 comp rx=0 tx=0).stop 2024-09-19T09:03:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.301+0000 7f3c66fc4640 1 -- 172.21.15.120:0/3289000420 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c60101b60 msgr2=0x7f3c60197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:03:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.301+0000 7f3c66fc4640 1 --2- 172.21.15.120:0/3289000420 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c60101b60 0x7f3c60197e00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3c54031610 tx=0x7f3c540043b0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.301+0000 7f3c66fc4640 1 -- 172.21.15.120:0/3289000420 shutdown_connections 2024-09-19T09:03:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.301+0000 7f3c66fc4640 1 --2- 172.21.15.120:0/3289000420 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3c500a32f0 0x7f3c500a57b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.301+0000 7f3c66fc4640 1 --2- 172.21.15.120:0/3289000420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c60103710 0x7f3c6019f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.301+0000 7f3c66fc4640 1 --2- 172.21.15.120:0/3289000420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c60102d50 0x7f3c60198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.301+0000 7f3c66fc4640 1 --2- 172.21.15.120:0/3289000420 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c60101b60 0x7f3c60197e00 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:03:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.301+0000 7f3c66fc4640 1 -- 172.21.15.120:0/3289000420 >> 172.21.15.120:0/3289000420 conn(0x7f3c600fd2e0 msgr2=0x7f3c601060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:03:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.301+0000 7f3c66fc4640 1 -- 172.21.15.120:0/3289000420 shutdown_connections 2024-09-19T09:03:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:03:55.301+0000 7f3c66fc4640 1 -- 172.21.15.120:0/3289000420 wait complete. 2024-09-19T09:03:57.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:56 smithi120 ceph-mon[31150]: pgmap v373: 97 pgs: 97 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-19T09:03:57.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:56 smithi196 ceph-mon[35150]: pgmap v373: 97 pgs: 97 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-19T09:03:57.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:56 smithi153 ceph-mon[35071]: pgmap v373: 97 pgs: 97 active+clean; 675 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-19T09:03:59.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:58 smithi120 ceph-mon[31150]: pgmap v374: 97 pgs: 97 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-19T09:03:59.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:58 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:03:59.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:58 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:03:59.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:58 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:03:59.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:58 smithi196 ceph-mon[35150]: pgmap v374: 97 pgs: 97 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-19T09:03:59.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:58 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:03:59.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:58 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:03:59.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:58 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:03:59.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:58 smithi153 ceph-mon[35071]: pgmap v374: 97 pgs: 97 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-19T09:03:59.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:58 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:03:59.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:58 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:03:59.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:58 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:04:00.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:59 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:04:00.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:59 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:04:00.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:03:59 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:04:00.087 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:59 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:04:00.087 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:59 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:04:00.087 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:03:59 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:04:00.088 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:59 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:04:00.088 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:59 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:04:00.088 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:03:59 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:04:01.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:00 smithi120 ceph-mon[31150]: pgmap v375: 97 pgs: 97 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T09:04:01.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:00 smithi196 ceph-mon[35150]: pgmap v375: 97 pgs: 97 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T09:04:01.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:00 smithi153 ceph-mon[35071]: pgmap v375: 97 pgs: 97 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T09:04:03.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:02 smithi120 ceph-mon[31150]: pgmap v376: 97 pgs: 97 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-19T09:04:03.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:02 smithi196 ceph-mon[35150]: pgmap v376: 97 pgs: 97 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-19T09:04:03.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:02 smithi153 ceph-mon[35071]: pgmap v376: 97 pgs: 97 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-19T09:04:03.873 INFO:tasks.workunit.client.0.smithi120.stderr:Note: switching to '6fad5dd73c93545e52f57f681e140a9c3529c11e'. 2024-09-19T09:04:03.873 INFO:tasks.workunit.client.0.smithi120.stderr: 2024-09-19T09:04:03.873 INFO:tasks.workunit.client.0.smithi120.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-19T09:04:03.873 INFO:tasks.workunit.client.0.smithi120.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-19T09:04:03.873 INFO:tasks.workunit.client.0.smithi120.stderr:state without impacting any branches by switching back to a branch. 2024-09-19T09:04:03.874 INFO:tasks.workunit.client.0.smithi120.stderr: 2024-09-19T09:04:03.874 INFO:tasks.workunit.client.0.smithi120.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-19T09:04:03.874 INFO:tasks.workunit.client.0.smithi120.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-19T09:04:03.874 INFO:tasks.workunit.client.0.smithi120.stderr: 2024-09-19T09:04:03.874 INFO:tasks.workunit.client.0.smithi120.stderr: git switch -c 2024-09-19T09:04:03.874 INFO:tasks.workunit.client.0.smithi120.stderr: 2024-09-19T09:04:03.874 INFO:tasks.workunit.client.0.smithi120.stderr:Or undo this operation with: 2024-09-19T09:04:03.874 INFO:tasks.workunit.client.0.smithi120.stderr: 2024-09-19T09:04:03.874 INFO:tasks.workunit.client.0.smithi120.stderr: git switch - 2024-09-19T09:04:03.874 INFO:tasks.workunit.client.0.smithi120.stderr: 2024-09-19T09:04:03.874 INFO:tasks.workunit.client.0.smithi120.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-19T09:04:03.874 INFO:tasks.workunit.client.0.smithi120.stderr: 2024-09-19T09:04:03.874 INFO:tasks.workunit.client.0.smithi120.stderr:HEAD is now at 6fad5dd73c9 Merge pull request #59812 from neha-ojha/wip-rtd-squid 2024-09-19T09:04:03.878 DEBUG:teuthology.orchestra.run.smithi120:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-19T09:04:03.947 INFO:tasks.workunit.client.0.smithi120.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-19T09:04:03.949 INFO:tasks.workunit.client.0.smithi120.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T09:04:03.950 INFO:tasks.workunit.client.0.smithi120.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-19T09:04:04.646 INFO:tasks.workunit.client.0.smithi120.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-19T09:04:04.716 INFO:tasks.workunit.client.0.smithi120.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-19T09:04:04.757 INFO:tasks.workunit.client.0.smithi120.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T09:04:04.758 INFO:tasks.workunit.client.0.smithi120.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T09:04:04.758 INFO:tasks.workunit.client.0.smithi120.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-19T09:04:04.797 INFO:tasks.workunit.client.0.smithi120.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T09:04:04.800 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-19T09:04:04.800 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-19T09:04:04.857 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-09-19T09:04:04.858 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-09-19T09:04:04.858 DEBUG:teuthology.orchestra.run.smithi120:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-09-19T09:04:04.919 INFO:tasks.workunit.client.0.smithi120.stderr:+ dbench 1 2024-09-19T09:04:04.924 INFO:tasks.workunit.client.0.smithi120.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-19T09:04:04.924 INFO:tasks.workunit.client.0.smithi120.stdout: 2024-09-19T09:04:04.924 INFO:tasks.workunit.client.0.smithi120.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-19T09:04:04.933 INFO:tasks.workunit.client.0.smithi120.stdout:0 of 1 processes prepared for launch 0 sec 2024-09-19T09:04:05.033 INFO:tasks.workunit.client.0.smithi120.stdout:1 of 1 processes prepared for launch 0 sec 2024-09-19T09:04:05.033 INFO:tasks.workunit.client.0.smithi120.stdout:releasing clients 2024-09-19T09:04:05.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:04 smithi120 ceph-mon[31150]: pgmap v377: 97 pgs: 97 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T09:04:05.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:04 smithi196 ceph-mon[35150]: pgmap v377: 97 pgs: 97 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T09:04:05.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:04 smithi153 ceph-mon[35071]: pgmap v377: 97 pgs: 97 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T09:04:05.623 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T09:04:05.623 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.35 2024-09-19T09:04:05.624 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:04:05.887 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:04:06.033 INFO:tasks.workunit.client.0.smithi120.stdout: 1 3944 40.07 MB/sec warmup 1 sec latency 6.682 ms 2024-09-19T09:04:06.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.457+0000 7f4e34570640 1 -- 172.21.15.120:0/3196962225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e2c102570 msgr2=0x7f4e2c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:06.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.457+0000 7f4e34570640 1 --2- 172.21.15.120:0/3196962225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e2c102570 0x7f4e2c1029b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009900 tx=0x7f4e1c02a480 comp rx=0 tx=0).stop 2024-09-19T09:04:06.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.457+0000 7f4e34570640 1 -- 172.21.15.120:0/3196962225 shutdown_connections 2024-09-19T09:04:06.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.457+0000 7f4e34570640 1 --2- 172.21.15.120:0/3196962225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e2c102ef0 0x7f4e2c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:06.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.457+0000 7f4e34570640 1 --2- 172.21.15.120:0/3196962225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e2c102570 0x7f4e2c1029b0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:06.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.457+0000 7f4e34570640 1 --2- 172.21.15.120:0/3196962225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e2c100e20 0x7f4e2c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:06.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.457+0000 7f4e34570640 1 -- 172.21.15.120:0/3196962225 >> 172.21.15.120:0/3196962225 conn(0x7f4e2c0fc7d0 msgr2=0x7f4e2c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:06.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.457+0000 7f4e34570640 1 -- 172.21.15.120:0/3196962225 shutdown_connections 2024-09-19T09:04:06.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.457+0000 7f4e34570640 1 -- 172.21.15.120:0/3196962225 wait complete. 2024-09-19T09:04:06.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.458+0000 7f4e34570640 1 Processor -- start 2024-09-19T09:04:06.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.458+0000 7f4e34570640 1 -- start start 2024-09-19T09:04:06.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.458+0000 7f4e34570640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e2c100e20 0x7f4e2c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:06.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.458+0000 7f4e34570640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e2c102570 0x7f4e2c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:06.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.458+0000 7f4e34570640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e2c102ef0 0x7f4e2c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:06.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.458+0000 7f4e34570640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c110f80 con 0x7f4e2c102570 2024-09-19T09:04:06.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.458+0000 7f4e34570640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c110e00 con 0x7f4e2c102ef0 2024-09-19T09:04:06.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.458+0000 7f4e34570640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c111100 con 0x7f4e2c100e20 2024-09-19T09:04:06.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.458+0000 7f4e31b0b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e2c102570 0x7f4e2c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:06.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.458+0000 7f4e31b0b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e2c102570 0x7f4e2c0fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45058/0 (socket says 172.21.15.120:45058) 2024-09-19T09:04:06.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.458+0000 7f4e31b0b640 1 -- 172.21.15.120:0/1572807313 learned_addr learned my addr 172.21.15.120:0/1572807313 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:06.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.458+0000 7f4e32b0d640 1 --2- 172.21.15.120:0/1572807313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e2c102ef0 0x7f4e2c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:06.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.458+0000 7f4e3230c640 1 --2- 172.21.15.120:0/1572807313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e2c100e20 0x7f4e2c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:06.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.458+0000 7f4e31b0b640 1 -- 172.21.15.120:0/1572807313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e2c100e20 msgr2=0x7f4e2c0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:06.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.458+0000 7f4e31b0b640 1 --2- 172.21.15.120:0/1572807313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e2c100e20 0x7f4e2c0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:06.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.458+0000 7f4e31b0b640 1 -- 172.21.15.120:0/1572807313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e2c102ef0 msgr2=0x7f4e2c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:06.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.458+0000 7f4e31b0b640 1 --2- 172.21.15.120:0/1572807313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e2c102ef0 0x7f4e2c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:06.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.458+0000 7f4e31b0b640 1 -- 172.21.15.120:0/1572807313 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2c1a6970 con 0x7f4e2c102570 2024-09-19T09:04:06.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.458+0000 7f4e32b0d640 1 --2- 172.21.15.120:0/1572807313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e2c102ef0 0x7f4e2c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:04:06.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.459+0000 7f4e31b0b640 1 --2- 172.21.15.120:0/1572807313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e2c102570 0x7f4e2c0fd1c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c006010 tx=0x7f4e1c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:06.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.459+0000 7f4e237fe640 1 -- 172.21.15.120:0/1572807313 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c03f070 con 0x7f4e2c102570 2024-09-19T09:04:06.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.459+0000 7f4e34570640 1 -- 172.21.15.120:0/1572807313 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e2c1a6c00 con 0x7f4e2c102570 2024-09-19T09:04:06.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.459+0000 7f4e237fe640 1 -- 172.21.15.120:0/1572807313 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e1c031c30 con 0x7f4e2c102570 2024-09-19T09:04:06.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.459+0000 7f4e237fe640 1 -- 172.21.15.120:0/1572807313 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c035030 con 0x7f4e2c102570 2024-09-19T09:04:06.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.459+0000 7f4e34570640 1 -- 172.21.15.120:0/1572807313 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e2c1a7130 con 0x7f4e2c102570 2024-09-19T09:04:06.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.460+0000 7f4e34570640 1 -- 172.21.15.120:0/1572807313 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e2c101200 con 0x7f4e2c102570 2024-09-19T09:04:06.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.461+0000 7f4e237fe640 1 -- 172.21.15.120:0/1572807313 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f4e1c042070 con 0x7f4e2c102570 2024-09-19T09:04:06.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.462+0000 7f4e237fe640 1 --2- 172.21.15.120:0/1572807313 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4e180a3240 0x7f4e180a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:06.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.462+0000 7f4e3230c640 1 --2- 172.21.15.120:0/1572807313 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4e180a3240 0x7f4e180a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:06.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.462+0000 7f4e237fe640 1 -- 172.21.15.120:0/1572807313 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f4e1c0efca0 con 0x7f4e2c102570 2024-09-19T09:04:06.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.462+0000 7f4e3230c640 1 --2- 172.21.15.120:0/1572807313 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4e180a3240 0x7f4e180a5700 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009b60 tx=0x7f4e28005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:06.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.466+0000 7f4e237fe640 1 -- 172.21.15.120:0/1572807313 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4e1c031de0 con 0x7f4e2c102570 2024-09-19T09:04:06.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.682+0000 7f4e34570640 1 -- 172.21.15.120:0/1572807313 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e2c10a0a0 con 0x7f4e2c102570 2024-09-19T09:04:06.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.683+0000 7f4e237fe640 1 -- 172.21.15.120:0/1572807313 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f4e1c0bc9c0 con 0x7f4e2c102570 2024-09-19T09:04:06.683 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:04:06.684 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":28,"btime":"2024-09-19T09:02:41:696728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:02:41.207357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:04:06.684 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 28 2024-09-19T09:04:06.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.686+0000 7f4e34570640 1 -- 172.21.15.120:0/1572807313 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4e180a3240 msgr2=0x7f4e180a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:06.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.686+0000 7f4e34570640 1 --2- 172.21.15.120:0/1572807313 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4e180a3240 0x7f4e180a5700 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009b60 tx=0x7f4e28005f90 comp rx=0 tx=0).stop 2024-09-19T09:04:06.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.686+0000 7f4e34570640 1 -- 172.21.15.120:0/1572807313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e2c102570 msgr2=0x7f4e2c0fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:06.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.686+0000 7f4e34570640 1 --2- 172.21.15.120:0/1572807313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e2c102570 0x7f4e2c0fd1c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c006010 tx=0x7f4e1c003c10 comp rx=0 tx=0).stop 2024-09-19T09:04:06.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.686+0000 7f4e34570640 1 -- 172.21.15.120:0/1572807313 shutdown_connections 2024-09-19T09:04:06.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.686+0000 7f4e34570640 1 --2- 172.21.15.120:0/1572807313 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4e180a3240 0x7f4e180a5700 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:06.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.686+0000 7f4e34570640 1 --2- 172.21.15.120:0/1572807313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e2c102ef0 0x7f4e2c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:06.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.686+0000 7f4e34570640 1 --2- 172.21.15.120:0/1572807313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e2c102570 0x7f4e2c0fd1c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:06.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.686+0000 7f4e34570640 1 --2- 172.21.15.120:0/1572807313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e2c100e20 0x7f4e2c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:06.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.686+0000 7f4e34570640 1 -- 172.21.15.120:0/1572807313 >> 172.21.15.120:0/1572807313 conn(0x7f4e2c0fc7d0 msgr2=0x7f4e2c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:06.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.686+0000 7f4e34570640 1 -- 172.21.15.120:0/1572807313 shutdown_connections 2024-09-19T09:04:06.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:06.686+0000 7f4e34570640 1 -- 172.21.15.120:0/1572807313 wait complete. 2024-09-19T09:04:06.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:06 smithi120 ceph-mon[31150]: pgmap v378: 97 pgs: 97 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T09:04:06.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:06 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1572807313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:04:07.033 INFO:tasks.workunit.client.0.smithi120.stdout: 1 7503 29.59 MB/sec warmup 2 sec latency 15.859 ms 2024-09-19T09:04:07.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:06 smithi196 ceph-mon[35150]: pgmap v378: 97 pgs: 97 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T09:04:07.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:06 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1572807313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:04:07.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:06 smithi153 ceph-mon[35071]: pgmap v378: 97 pgs: 97 active+clean; 676 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T09:04:07.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:06 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1572807313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:04:07.569 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0' 2024-09-19T09:04:07.569 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0 2024-09-19T09:04:07.830 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:04:08.033 INFO:tasks.workunit.client.0.smithi120.stdout: 1 11092 26.10 MB/sec warmup 3 sec latency 12.956 ms 2024-09-19T09:04:08.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.354+0000 7f115c3e8640 1 -- 172.21.15.120:0/3674364726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1154100e20 msgr2=0x7f1154101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:08.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.354+0000 7f115c3e8640 1 --2- 172.21.15.120:0/3674364726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1154100e20 0x7f1154101200 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1150009ca0 tx=0x7f115002a950 comp rx=0 tx=0).stop 2024-09-19T09:04:08.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.354+0000 7f115c3e8640 1 -- 172.21.15.120:0/3674364726 shutdown_connections 2024-09-19T09:04:08.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.354+0000 7f115c3e8640 1 --2- 172.21.15.120:0/3674364726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1154102ef0 0x7f115410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.354+0000 7f115c3e8640 1 --2- 172.21.15.120:0/3674364726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1154102570 0x7f11541029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.354+0000 7f115c3e8640 1 --2- 172.21.15.120:0/3674364726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1154100e20 0x7f1154101200 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.354+0000 7f115c3e8640 1 -- 172.21.15.120:0/3674364726 >> 172.21.15.120:0/3674364726 conn(0x7f11540fc7d0 msgr2=0x7f11540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.354+0000 7f115c3e8640 1 -- 172.21.15.120:0/3674364726 shutdown_connections 2024-09-19T09:04:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.354+0000 7f115c3e8640 1 -- 172.21.15.120:0/3674364726 wait complete. 2024-09-19T09:04:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.354+0000 7f115c3e8640 1 Processor -- start 2024-09-19T09:04:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.354+0000 7f115c3e8640 1 -- start start 2024-09-19T09:04:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.355+0000 7f115c3e8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1154100e20 0x7f1154199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.355+0000 7f115c3e8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1154102570 0x7f11541998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.355+0000 7f115c3e8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1154102ef0 0x7f11541a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.355+0000 7f115c3e8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1154110f80 con 0x7f1154100e20 2024-09-19T09:04:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.355+0000 7f115c3e8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1154110e00 con 0x7f1154102ef0 2024-09-19T09:04:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.355+0000 7f115c3e8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1154111100 con 0x7f1154102570 2024-09-19T09:04:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.355+0000 7f115a184640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1154100e20 0x7f1154199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.355+0000 7f115a184640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1154100e20 0x7f1154199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45080/0 (socket says 172.21.15.120:45080) 2024-09-19T09:04:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.355+0000 7f115a985640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1154102ef0 0x7f11541a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.355+0000 7f115a184640 1 -- 172.21.15.120:0/1206047931 learned_addr learned my addr 172.21.15.120:0/1206047931 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.355+0000 7f1159983640 1 --2- 172.21.15.120:0/1206047931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1154102570 0x7f11541998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.355+0000 7f115a184640 1 -- 172.21.15.120:0/1206047931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1154102570 msgr2=0x7f11541998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.355+0000 7f115a184640 1 --2- 172.21.15.120:0/1206047931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1154102570 0x7f11541998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.355+0000 7f115a184640 1 -- 172.21.15.120:0/1206047931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1154102ef0 msgr2=0x7f11541a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.355+0000 7f115a184640 1 --2- 172.21.15.120:0/1206047931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1154102ef0 0x7f11541a4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.355+0000 7f115a184640 1 -- 172.21.15.120:0/1206047931 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11541a4950 con 0x7f1154100e20 2024-09-19T09:04:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.355+0000 7f115a184640 1 --2- 172.21.15.120:0/1206047931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1154100e20 0x7f1154199390 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1150005cc0 tx=0x7f115002f630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.355+0000 7f114b7fe640 1 -- 172.21.15.120:0/1206047931 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1150034040 con 0x7f1154100e20 2024-09-19T09:04:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.355+0000 7f114b7fe640 1 -- 172.21.15.120:0/1206047931 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1150030920 con 0x7f1154100e20 2024-09-19T09:04:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.355+0000 7f115c3e8640 1 -- 172.21.15.120:0/1206047931 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11541a4b20 con 0x7f1154100e20 2024-09-19T09:04:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.355+0000 7f114b7fe640 1 -- 172.21.15.120:0/1206047931 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1150004790 con 0x7f1154100e20 2024-09-19T09:04:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.355+0000 7f115c3e8640 1 -- 172.21.15.120:0/1206047931 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11541a4fd0 con 0x7f1154100e20 2024-09-19T09:04:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.358+0000 7f115e7c1740 1 Processor -- start 2024-09-19T09:04:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.359+0000 7f115e7c1740 1 -- start start 2024-09-19T09:04:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.359+0000 7f115e7c1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e07df850 0x5575e07dfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.359+0000 7f114b7fe640 1 -- 172.21.15.120:0/1206047931 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f1150037020 con 0x7f1154100e20 2024-09-19T09:04:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.359+0000 7f115e7c1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575e07e0170 0x5575e07ea4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.359+0000 7f115e7c1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575e07eaa30 0x5575e07ece20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.359+0000 7f115e7c1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5575e0744160 con 0x5575e07eaa30 2024-09-19T09:04:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.359+0000 7f115e7c1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5575e06eb720 con 0x5575e07df850 2024-09-19T09:04:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.359+0000 7f115e7c1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5575e0740120 con 0x5575e07e0170 2024-09-19T09:04:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.359+0000 7f115a985640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e07df850 0x5575e07dfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.359+0000 7f1159983640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575e07eaa30 0x5575e07ece20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.359+0000 7f1159983640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575e07eaa30 0x5575e07ece20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45088/0 (socket says 172.21.15.120:45088) 2024-09-19T09:04:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.359+0000 7f1159983640 1 -- 172.21.15.120:0/1556350821 learned_addr learned my addr 172.21.15.120:0/1556350821 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.359+0000 7f115a985640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e07df850 0x5575e07dfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:34940/0 (socket says 172.21.15.120:34940) 2024-09-19T09:04:08.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.359+0000 7f115a184640 1 --2- 172.21.15.120:0/1556350821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575e07e0170 0x5575e07ea4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.359+0000 7f1159983640 1 -- 172.21.15.120:0/1556350821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575e07e0170 msgr2=0x5575e07ea4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:08.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.359+0000 7f1159983640 1 --2- 172.21.15.120:0/1556350821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575e07e0170 0x5575e07ea4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.359+0000 7f1159983640 1 -- 172.21.15.120:0/1556350821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e07df850 msgr2=0x5575e07dfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:08.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.359+0000 7f1159983640 1 --2- 172.21.15.120:0/1556350821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e07df850 0x5575e07dfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.359+0000 7f1159983640 1 -- 172.21.15.120:0/1556350821 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575e04b5ba0 con 0x5575e07eaa30 2024-09-19T09:04:08.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.359+0000 7f115a985640 1 --2- 172.21.15.120:0/1556350821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e07df850 0x5575e07dfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:04:08.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.359+0000 7f115a184640 1 --2- 172.21.15.120:0/1556350821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575e07e0170 0x5575e07ea4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:04:08.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.360+0000 7f1159983640 1 --2- 172.21.15.120:0/1556350821 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575e07eaa30 0x5575e07ece20 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1144009ca0 tx=0x7f1144011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7857c3f632a78c71 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:08.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.360+0000 7f114b7fe640 1 --2- 172.21.15.120:0/1206047931 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f11400a3240 0x7f11400a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.360+0000 7f1148dea640 1 -- 172.21.15.120:0/1556350821 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114401e070 con 0x5575e07eaa30 2024-09-19T09:04:08.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.360+0000 7f1148dea640 1 -- 172.21.15.120:0/1556350821 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1144019b10 con 0x5575e07eaa30 2024-09-19T09:04:08.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.360+0000 7f114b7fe640 1 -- 172.21.15.120:0/1206047931 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f11500f0820 con 0x7f1154100e20 2024-09-19T09:04:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.360+0000 7f1148dea640 1 -- 172.21.15.120:0/1556350821 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114400eae0 con 0x5575e07eaa30 2024-09-19T09:04:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.360+0000 7f115a985640 1 --2- 172.21.15.120:0/1206047931 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f11400a3240 0x7f11400a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.361+0000 7f115e7c1740 1 -- 172.21.15.120:0/1556350821 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575e07eaa30 msgr2=0x5575e07ece20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.361+0000 7f115a985640 1 --2- 172.21.15.120:0/1206047931 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f11400a3240 0x7f11400a5700 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f114c005fd0 tx=0x7f114c005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.361+0000 7f115e7c1740 1 --2- 172.21.15.120:0/1556350821 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575e07eaa30 0x5575e07ece20 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1144009ca0 tx=0x7f1144011810 comp rx=0 tx=0).stop 2024-09-19T09:04:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.361+0000 7f115e7c1740 1 -- 172.21.15.120:0/1556350821 shutdown_connections 2024-09-19T09:04:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.361+0000 7f115e7c1740 1 --2- 172.21.15.120:0/1556350821 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575e07eaa30 0x5575e07ece20 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.361+0000 7f115e7c1740 1 --2- 172.21.15.120:0/1556350821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575e07e0170 0x5575e07ea4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.361+0000 7f115e7c1740 1 --2- 172.21.15.120:0/1556350821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e07df850 0x5575e07dfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.361+0000 7f115e7c1740 1 -- 172.21.15.120:0/1556350821 >> 172.21.15.120:0/1556350821 conn(0x5575e07dbde0 msgr2=0x5575e07dc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.361+0000 7f115e7c1740 1 -- 172.21.15.120:0/1556350821 shutdown_connections 2024-09-19T09:04:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.361+0000 7f115e7c1740 1 -- 172.21.15.120:0/1556350821 wait complete. 2024-09-19T09:04:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.361+0000 7f115e7c1740 1 -- start start 2024-09-19T09:04:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.361+0000 7f115e7c1740 1 Processor -- start 2024-09-19T09:04:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.362+0000 7f115e7c1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e07df850 0x5575e07e7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.362+0000 7f115e7c1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575e07e0170 0x5575e07e7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.362+0000 7f115e7c1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575e07eaa30 0x5575e07e83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.362+0000 7f115e7c1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5575e07ed600 con 0x5575e07e0170 2024-09-19T09:04:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.362+0000 7f115e7c1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5575e06cc900 con 0x5575e07df850 2024-09-19T09:04:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.362+0000 7f115e7c1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5575e07ed780 con 0x5575e07eaa30 2024-09-19T09:04:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.362+0000 7f115e7c1740 10 client.0 _ll_register_callbacks cb 0x5575e06ca780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.362+0000 7f1159983640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575e07e0170 0x5575e07e7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.362+0000 7f1159983640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575e07e0170 0x5575e07e7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45098/0 (socket says 172.21.15.120:45098) 2024-09-19T09:04:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.362+0000 7f1159983640 1 -- 172.21.15.120:0/2611618555 learned_addr learned my addr 172.21.15.120:0/2611618555 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.362+0000 7f115a184640 1 --2- 172.21.15.120:0/2611618555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e07df850 0x5575e07e7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.362+0000 7f115a985640 1 --2- 172.21.15.120:0/2611618555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575e07eaa30 0x5575e07e83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.362+0000 7f1159983640 1 -- 172.21.15.120:0/2611618555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575e07eaa30 msgr2=0x5575e07e83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.362+0000 7f1159983640 1 --2- 172.21.15.120:0/2611618555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575e07eaa30 0x5575e07e83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.362+0000 7f1159983640 1 -- 172.21.15.120:0/2611618555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e07df850 msgr2=0x5575e07e7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:08.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.362+0000 7f1159983640 1 --2- 172.21.15.120:0/2611618555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e07df850 0x5575e07e7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.362+0000 7f1159983640 1 -- 172.21.15.120:0/2611618555 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575e070cc50 con 0x5575e07e0170 2024-09-19T09:04:08.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.362+0000 7f1159983640 1 --2- 172.21.15.120:0/2611618555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575e07e0170 0x5575e07e7e90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1144009c70 tx=0x7f1144019a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa2c93412047931 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:08.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.362+0000 7f113dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:04:08.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.362+0000 7f113dffb640 1 -- 172.21.15.120:0/2611618555 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114401e070 con 0x5575e07e0170 2024-09-19T09:04:08.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.362+0000 7f113dffb640 1 -- 172.21.15.120:0/2611618555 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f1144005440 con 0x5575e07e0170 2024-09-19T09:04:08.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.362+0000 7f115e7c1740 1 -- 172.21.15.120:0/2611618555 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1144019b10 con 0x5575e07e0170 2024-09-19T09:04:08.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.363+0000 7f113dffb640 1 -- 172.21.15.120:0/2611618555 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1144005640 con 0x5575e07e0170 2024-09-19T09:04:08.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.363+0000 7f115e7c1740 10 client.15744 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:08.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.363+0000 7f115e7c1740 1 -- 172.21.15.120:0/2611618555 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5575e07f9340 con 0x5575e07e0170 2024-09-19T09:04:08.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.363+0000 7f113dffb640 1 -- 172.21.15.120:0/2611618555 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1144019b10 con 0x5575e07e0170 2024-09-19T09:04:08.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.363+0000 7f113dffb640 1 -- 172.21.15.120:0/2611618555 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f11440048f0 con 0x5575e07e0170 2024-09-19T09:04:08.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.363+0000 7f115e7c1740 10 client.15744 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:08.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.363+0000 7f115e7c1740 10 client.15744 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:04:08.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.363+0000 7f115e7c1740 20 client.15744 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:08.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.363+0000 7f115e7c1740 1 --2- 172.21.15.120:0/2611618555 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5575e07fa800 0x5575e07fcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.363+0000 7f115e7c1740 4 client.15744 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:04:08.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.363+0000 7f115e7c1740 1 -- 172.21.15.120:0/2611618555 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575e0751890 con 0x5575e07fa800 2024-09-19T09:04:08.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.363+0000 7f115a184640 1 --2- 172.21.15.120:0/2611618555 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5575e07fa800 0x5575e07fcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.364+0000 7f113dffb640 10 client.15744 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:08.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.364+0000 7f115a184640 1 --2- 172.21.15.120:0/2611618555 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5575e07fa800 0x5575e07fcbf0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd4d5f4d50250445 server_cookie=94d25b56298dcd0e in_seq=0 out_seq=0 2024-09-19T09:04:08.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.366+0000 7f113dffb640 1 -- 172.21.15.120:0/2611618555 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5575e0751890 con 0x5575e07fa800 2024-09-19T09:04:08.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.366+0000 7f113dffb640 10 client.15744 handle_command_reply: tid=0 2024-09-19T09:04:08.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.366+0000 7f115e7c1740 1 client.15744 shutdown 2024-09-19T09:04:08.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.366+0000 7f115e7c1740 1 -- 172.21.15.120:0/2611618555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575e07e0170 msgr2=0x5575e07e7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:08.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.366+0000 7f115e7c1740 1 --2- 172.21.15.120:0/2611618555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575e07e0170 0x5575e07e7e90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1144009c70 tx=0x7f1144019a20 comp rx=0 tx=0).stop 2024-09-19T09:04:08.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.366+0000 7f115e7c1740 1 -- 172.21.15.120:0/2611618555 shutdown_connections 2024-09-19T09:04:08.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.366+0000 7f115e7c1740 1 --2- 172.21.15.120:0/2611618555 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5575e07fa800 0x5575e07fcbf0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.366+0000 7f115e7c1740 1 --2- 172.21.15.120:0/2611618555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575e07eaa30 0x5575e07e83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.366+0000 7f115e7c1740 1 --2- 172.21.15.120:0/2611618555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575e07e0170 0x5575e07e7e90 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.366+0000 7f115e7c1740 1 --2- 172.21.15.120:0/2611618555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e07df850 0x5575e07e7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.366+0000 7f113dffb640 0 client.15744 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:08.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.366+0000 7f115e7c1740 1 -- 172.21.15.120:0/2611618555 >> 172.21.15.120:0/2611618555 conn(0x5575e07dbde0 msgr2=0x5575e07e1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:08.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.366+0000 7f115e7c1740 1 -- 172.21.15.120:0/2611618555 shutdown_connections 2024-09-19T09:04:08.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.366+0000 7f115e7c1740 1 -- 172.21.15.120:0/2611618555 wait complete. 2024-09-19T09:04:08.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.366+0000 7f115e7c1740 20 client.15744 trim_cache size 0 max 16384 2024-09-19T09:04:08.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.385+0000 7f115e7c1740 1 Processor -- start 2024-09-19T09:04:08.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.385+0000 7f115e7c1740 1 -- start start 2024-09-19T09:04:08.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.385+0000 7f115e7c1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575e07df850 0x5575e07dc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.385+0000 7f115e7c1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575e07e0170 0x5575e0802380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.385+0000 7f115e7c1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e07eaa30 0x5575e08028c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.385+0000 7f115e7c1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f11541ad4e0 con 0x5575e07df850 2024-09-19T09:04:08.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.385+0000 7f115e7c1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f11541af020 con 0x5575e07eaa30 2024-09-19T09:04:08.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.385+0000 7f115e7c1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f11541adbb0 con 0x5575e07e0170 2024-09-19T09:04:08.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.385+0000 7f115a184640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575e07df850 0x5575e07dc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.385+0000 7f115a184640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575e07df850 0x5575e07dc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45114/0 (socket says 172.21.15.120:45114) 2024-09-19T09:04:08.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.385+0000 7f115a985640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e07eaa30 0x5575e08028c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.385+0000 7f115a184640 1 -- 172.21.15.120:0/2411253907 learned_addr learned my addr 172.21.15.120:0/2411253907 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:08.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.385+0000 7f1159983640 1 --2- 172.21.15.120:0/2411253907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575e07e0170 0x5575e0802380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.385+0000 7f1159983640 1 -- 172.21.15.120:0/2411253907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e07eaa30 msgr2=0x5575e08028c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:08.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.385+0000 7f1159983640 1 --2- 172.21.15.120:0/2411253907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e07eaa30 0x5575e08028c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.385+0000 7f1159983640 1 -- 172.21.15.120:0/2411253907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575e07df850 msgr2=0x5575e07dc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:08.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.385+0000 7f1159983640 1 --2- 172.21.15.120:0/2411253907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575e07df850 0x5575e07dc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.385+0000 7f1159983640 1 -- 172.21.15.120:0/2411253907 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575e07e32b0 con 0x5575e07e0170 2024-09-19T09:04:08.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.386+0000 7f1159983640 1 --2- 172.21.15.120:0/2411253907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575e07e0170 0x5575e0802380 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f114400e610 tx=0x7f1144001e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7561bb26f141887 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:08.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.386+0000 7f113ffff640 1 -- 172.21.15.120:0/2411253907 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114400e3b0 con 0x5575e07e0170 2024-09-19T09:04:08.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.386+0000 7f113ffff640 1 -- 172.21.15.120:0/2411253907 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1144028070 con 0x5575e07e0170 2024-09-19T09:04:08.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.386+0000 7f113ffff640 1 -- 172.21.15.120:0/2411253907 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114400ec30 con 0x5575e07e0170 2024-09-19T09:04:08.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.387+0000 7f115e7c1740 1 -- 172.21.15.120:0/2411253907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575e07e0170 msgr2=0x5575e0802380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:08.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.387+0000 7f115e7c1740 1 --2- 172.21.15.120:0/2411253907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575e07e0170 0x5575e0802380 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f114400e610 tx=0x7f1144001e50 comp rx=0 tx=0).stop 2024-09-19T09:04:08.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.387+0000 7f115e7c1740 1 -- 172.21.15.120:0/2411253907 shutdown_connections 2024-09-19T09:04:08.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.387+0000 7f115e7c1740 1 --2- 172.21.15.120:0/2411253907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e07eaa30 0x5575e08028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.387+0000 7f115e7c1740 1 --2- 172.21.15.120:0/2411253907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575e07e0170 0x5575e0802380 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.387+0000 7f115e7c1740 1 --2- 172.21.15.120:0/2411253907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575e07df850 0x5575e07dc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.387+0000 7f115e7c1740 1 -- 172.21.15.120:0/2411253907 >> 172.21.15.120:0/2411253907 conn(0x5575e07dbde0 msgr2=0x5575e07e99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:08.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.387+0000 7f115e7c1740 1 -- 172.21.15.120:0/2411253907 shutdown_connections 2024-09-19T09:04:08.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.387+0000 7f115e7c1740 1 -- 172.21.15.120:0/2411253907 wait complete. 2024-09-19T09:04:08.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.387+0000 7f115e7c1740 1 -- start start 2024-09-19T09:04:08.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.387+0000 7f115e7c1740 1 Processor -- start 2024-09-19T09:04:08.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.387+0000 7f115e7c1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575e07df850 0x5575e07ee790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.387+0000 7f115e7c1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e07e0170 0x5575e07eecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.387+0000 7f115e7c1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575e07eaa30 0x5575e07ef210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.387+0000 7f115e7c1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f11541ae950 con 0x5575e07df850 2024-09-19T09:04:08.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.387+0000 7f115e7c1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f11541ae280 con 0x5575e07e0170 2024-09-19T09:04:08.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.387+0000 7f115e7c1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f11541ab4a0 con 0x5575e07eaa30 2024-09-19T09:04:08.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.387+0000 7f115e7c1740 10 client.0 _ll_register_callbacks cb 0x7f114401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:08.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.388+0000 7f115a985640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575e07eaa30 0x5575e07ef210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.388+0000 7f1159983640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e07e0170 0x5575e07eecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.388+0000 7f115a184640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575e07df850 0x5575e07ee790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.388+0000 7f115a985640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575e07eaa30 0x5575e07ef210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:46268/0 (socket says 172.21.15.120:46268) 2024-09-19T09:04:08.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.388+0000 7f115a985640 1 -- 172.21.15.120:0/1582082272 learned_addr learned my addr 172.21.15.120:0/1582082272 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:08.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.388+0000 7f1159983640 1 -- 172.21.15.120:0/1582082272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575e07eaa30 msgr2=0x5575e07ef210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:08.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.388+0000 7f1159983640 1 --2- 172.21.15.120:0/1582082272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575e07eaa30 0x5575e07ef210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.388+0000 7f1159983640 1 -- 172.21.15.120:0/1582082272 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575e07df850 msgr2=0x5575e07ee790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:08.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.388+0000 7f1159983640 1 --2- 172.21.15.120:0/1582082272 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575e07df850 0x5575e07ee790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.388+0000 7f1159983640 1 -- 172.21.15.120:0/1582082272 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575e07dffa0 con 0x5575e07e0170 2024-09-19T09:04:08.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.388+0000 7f1159983640 1 --2- 172.21.15.120:0/1582082272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e07e0170 0x5575e07eecd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x5575e078ac50 tx=0x7f114400e640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7d7913b93a4cb2d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:08.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.388+0000 7f113dffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:04:08.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.388+0000 7f113dffb640 1 -- 172.21.15.120:0/1582082272 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1144005440 con 0x5575e07e0170 2024-09-19T09:04:08.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.388+0000 7f113dffb640 1 -- 172.21.15.120:0/1582082272 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f11440055e0 con 0x5575e07e0170 2024-09-19T09:04:08.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.388+0000 7f115e7c1740 1 -- 172.21.15.120:0/1582082272 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1144028070 con 0x5575e07e0170 2024-09-19T09:04:08.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.388+0000 7f113dffb640 1 -- 172.21.15.120:0/1582082272 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1144046050 con 0x5575e07e0170 2024-09-19T09:04:08.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.388+0000 7f115e7c1740 10 client.25169 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:08.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.388+0000 7f115e7c1740 1 -- 172.21.15.120:0/1582082272 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5575e0865440 con 0x5575e07e0170 2024-09-19T09:04:08.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.389+0000 7f113dffb640 1 -- 172.21.15.120:0/1582082272 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1144028070 con 0x5575e07e0170 2024-09-19T09:04:08.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.389+0000 7f113dffb640 1 -- 172.21.15.120:0/1582082272 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1144003680 con 0x5575e07e0170 2024-09-19T09:04:08.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.389+0000 7f115e7c1740 10 client.25169 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:08.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.389+0000 7f115e7c1740 10 client.25169 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:04:08.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.389+0000 7f115e7c1740 20 client.25169 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:08.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.389+0000 7f115e7c1740 1 --2- 172.21.15.120:0/1582082272 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5575e0865b40 0x5575e0867f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.389+0000 7f115e7c1740 4 client.25169 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}] 2024-09-19T09:04:08.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.389+0000 7f115e7c1740 1 -- 172.21.15.120:0/1582082272 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}) -- 0x5575e074bbe0 con 0x5575e0865b40 2024-09-19T09:04:08.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.389+0000 7f115a184640 1 --2- 172.21.15.120:0/1582082272 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5575e0865b40 0x5575e0867f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.389+0000 7f115a184640 1 --2- 172.21.15.120:0/1582082272 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5575e0865b40 0x5575e0867f60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29a305d9751eab61 server_cookie=91d6316ac337301 in_seq=0 out_seq=0 2024-09-19T09:04:08.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.389+0000 7f113dffb640 10 client.25169 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.399+0000 7f88006e4640 1 -- 172.21.15.120:0/4176976338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87f8102d50 msgr2=0x7f87f81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.399+0000 7f88006e4640 1 --2- 172.21.15.120:0/4176976338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87f8102d50 0x7f87f81031d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009900 tx=0x7f87e802a410 comp rx=0 tx=0).stop 2024-09-19T09:04:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.399+0000 7f88006e4640 1 -- 172.21.15.120:0/4176976338 shutdown_connections 2024-09-19T09:04:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.399+0000 7f88006e4640 1 --2- 172.21.15.120:0/4176976338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87f8103710 0x7f87f8109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.399+0000 7f88006e4640 1 --2- 172.21.15.120:0/4176976338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87f8102d50 0x7f87f81031d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.399+0000 7f88006e4640 1 --2- 172.21.15.120:0/4176976338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87f8101b60 0x7f87f8101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.399+0000 7f88006e4640 1 -- 172.21.15.120:0/4176976338 >> 172.21.15.120:0/4176976338 conn(0x7f87f80fd2e0 msgr2=0x7f87f80ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.399+0000 7f88006e4640 1 -- 172.21.15.120:0/4176976338 shutdown_connections 2024-09-19T09:04:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.399+0000 7f88006e4640 1 -- 172.21.15.120:0/4176976338 wait complete. 2024-09-19T09:04:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.400+0000 7f88006e4640 1 Processor -- start 2024-09-19T09:04:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.400+0000 7f88006e4640 1 -- start start 2024-09-19T09:04:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.400+0000 7f88006e4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87f8101b60 0x7f87f8075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.400+0000 7f88006e4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87f8102d50 0x7f87f80745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.400+0000 7f88006e4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87f8103710 0x7f87f8074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.400+0000 7f88006e4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f87f810d7c0 con 0x7f87f8103710 2024-09-19T09:04:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.400+0000 7f88006e4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f87f810d640 con 0x7f87f8102d50 2024-09-19T09:04:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.400+0000 7f88006e4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f87f810d940 con 0x7f87f8101b60 2024-09-19T09:04:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.400+0000 7f87fec81640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87f8103710 0x7f87f8074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.400+0000 7f87fec81640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87f8103710 0x7f87f8074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45144/0 (socket says 172.21.15.120:45144) 2024-09-19T09:04:08.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.400+0000 7f87fe480640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87f8101b60 0x7f87f8075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.400+0000 7f87fec81640 1 -- 172.21.15.120:0/733237212 learned_addr learned my addr 172.21.15.120:0/733237212 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:08.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.400+0000 7f87fdc7f640 1 --2- 172.21.15.120:0/733237212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87f8102d50 0x7f87f80745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.400+0000 7f87fec81640 1 -- 172.21.15.120:0/733237212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87f8101b60 msgr2=0x7f87f8075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:08.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.400+0000 7f87fec81640 1 --2- 172.21.15.120:0/733237212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87f8101b60 0x7f87f8075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.400+0000 7f87fec81640 1 -- 172.21.15.120:0/733237212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87f8102d50 msgr2=0x7f87f80745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:08.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.400+0000 7f87fec81640 1 --2- 172.21.15.120:0/733237212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87f8102d50 0x7f87f80745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.400+0000 7f87fec81640 1 -- 172.21.15.120:0/733237212 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f80753b0 con 0x7f87f8103710 2024-09-19T09:04:08.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.401+0000 7f87fec81640 1 --2- 172.21.15.120:0/733237212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87f8103710 0x7f87f8074b20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f87f000efd0 tx=0x7f87f000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:08.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.401+0000 7f87ef7fe640 1 -- 172.21.15.120:0/733237212 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f0003b70 con 0x7f87f8103710 2024-09-19T09:04:08.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.401+0000 7f87ef7fe640 1 -- 172.21.15.120:0/733237212 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87f0003d10 con 0x7f87f8103710 2024-09-19T09:04:08.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.401+0000 7f88006e4640 1 -- 172.21.15.120:0/733237212 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f81a1ee0 con 0x7f87f8103710 2024-09-19T09:04:08.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.401+0000 7f87ef7fe640 1 -- 172.21.15.120:0/733237212 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f00109f0 con 0x7f87f8103710 2024-09-19T09:04:08.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.401+0000 7f88006e4640 1 -- 172.21.15.120:0/733237212 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f81a2340 con 0x7f87f8103710 2024-09-19T09:04:08.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.403+0000 7f87ef7fe640 1 -- 172.21.15.120:0/733237212 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f87f0004020 con 0x7f87f8103710 2024-09-19T09:04:08.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.403+0000 7f8803483740 1 Processor -- start 2024-09-19T09:04:08.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.404+0000 7f8803483740 1 -- start start 2024-09-19T09:04:08.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.404+0000 7f8803483740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4fe2969a0 0x55b4fe296da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.404+0000 7f8803483740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4fe2972e0 0x55b4fe29bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.404+0000 7f8803483740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4fe29c370 0x55b4fe29e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.404+0000 7f8803483740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b4fe1fa2f0 con 0x55b4fe2969a0 2024-09-19T09:04:08.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.404+0000 7f8803483740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b4fe1a2270 con 0x55b4fe29c370 2024-09-19T09:04:08.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.404+0000 7f8803483740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b4fe1f62b0 con 0x55b4fe2972e0 2024-09-19T09:04:08.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.404+0000 7f87fdc7f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4fe2969a0 0x55b4fe296da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.404+0000 7f87fdc7f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4fe2969a0 0x55b4fe296da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45154/0 (socket says 172.21.15.120:45154) 2024-09-19T09:04:08.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.404+0000 7f87fdc7f640 1 -- 172.21.15.120:0/3494826212 learned_addr learned my addr 172.21.15.120:0/3494826212 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:08.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.404+0000 7f87fdc7f640 1 --2- 172.21.15.120:0/3494826212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4fe29c370 0x55b4fe29e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.404+0000 7f87fe480640 1 --2- 172.21.15.120:0/3494826212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4fe2972e0 0x55b4fe29bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.404+0000 7f87fdc7f640 1 -- 172.21.15.120:0/3494826212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4fe2972e0 msgr2=0x55b4fe29bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.404+0000 7f87fdc7f640 1 --2- 172.21.15.120:0/3494826212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4fe2972e0 0x55b4fe29bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.404+0000 7f87fdc7f640 1 -- 172.21.15.120:0/3494826212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4fe29c370 msgr2=0x55b4fe29e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.404+0000 7f87fdc7f640 1 --2- 172.21.15.120:0/3494826212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4fe29c370 0x55b4fe29e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.404+0000 7f87fdc7f640 1 -- 172.21.15.120:0/3494826212 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4fe1a1340 con 0x55b4fe2969a0 2024-09-19T09:04:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.404+0000 7f87ef7fe640 1 --2- 172.21.15.120:0/733237212 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f87e40a3370 0x7f87e40a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.404+0000 7f87fdc7f640 1 --2- 172.21.15.120:0/3494826212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4fe2969a0 0x55b4fe296da0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f87e802a930 tx=0x7f87e8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69fae1de841bdac5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.404+0000 7f87fe480640 1 --2- 172.21.15.120:0/733237212 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f87e40a3370 0x7f87e40a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.405+0000 7f87ef7fe640 1 -- 172.21.15.120:0/733237212 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f87f00d0050 con 0x7f87f8103710 2024-09-19T09:04:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.405+0000 7f87ecff9640 1 -- 172.21.15.120:0/3494826212 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e8044020 con 0x55b4fe2969a0 2024-09-19T09:04:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.405+0000 7f87ecff9640 1 -- 172.21.15.120:0/3494826212 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87e80318b0 con 0x55b4fe2969a0 2024-09-19T09:04:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.405+0000 7f87ecff9640 1 -- 172.21.15.120:0/3494826212 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e8035040 con 0x55b4fe2969a0 2024-09-19T09:04:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.405+0000 7f8803483740 1 -- 172.21.15.120:0/3494826212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4fe2969a0 msgr2=0x55b4fe296da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.405+0000 7f8803483740 1 --2- 172.21.15.120:0/3494826212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4fe2969a0 0x55b4fe296da0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f87e802a930 tx=0x7f87e8005fd0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.405+0000 7f87fe480640 1 --2- 172.21.15.120:0/733237212 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f87e40a3370 0x7f87e40a5830 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55b4fe297d10 tx=0x7f87f4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.405+0000 7f8803483740 1 -- 172.21.15.120:0/3494826212 shutdown_connections 2024-09-19T09:04:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.405+0000 7f8803483740 1 --2- 172.21.15.120:0/3494826212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4fe29c370 0x55b4fe29e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.405+0000 7f8803483740 1 --2- 172.21.15.120:0/3494826212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4fe2972e0 0x55b4fe29bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.405+0000 7f8803483740 1 --2- 172.21.15.120:0/3494826212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4fe2969a0 0x55b4fe296da0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.405+0000 7f8803483740 1 -- 172.21.15.120:0/3494826212 >> 172.21.15.120:0/3494826212 conn(0x55b4fe292010 msgr2=0x55b4fe292410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.406+0000 7f8803483740 1 -- 172.21.15.120:0/3494826212 shutdown_connections 2024-09-19T09:04:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.406+0000 7f8803483740 1 -- 172.21.15.120:0/3494826212 wait complete. 2024-09-19T09:04:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.406+0000 7f8803483740 1 -- start start 2024-09-19T09:04:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.406+0000 7f8803483740 1 Processor -- start 2024-09-19T09:04:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.406+0000 7f8803483740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4fe2969a0 0x55b4fe2a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.406+0000 7f8803483740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4fe2972e0 0x55b4fe2a1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.406+0000 7f8803483740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4fe29c370 0x55b4fe2a81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.406+0000 7f8803483740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b4fe29f2e0 con 0x55b4fe29c370 2024-09-19T09:04:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.406+0000 7f8803483740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b4fe183290 con 0x55b4fe2969a0 2024-09-19T09:04:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.406+0000 7f8803483740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b4fe29f460 con 0x55b4fe2972e0 2024-09-19T09:04:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.406+0000 7f8803483740 10 client.0 _ll_register_callbacks cb 0x55b4fe1b4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.406+0000 7f87fec81640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4fe29c370 0x55b4fe2a81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.406+0000 7f87fec81640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4fe29c370 0x55b4fe2a81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45160/0 (socket says 172.21.15.120:45160) 2024-09-19T09:04:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.406+0000 7f87fec81640 1 -- 172.21.15.120:0/1957401733 learned_addr learned my addr 172.21.15.120:0/1957401733 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.406+0000 7f87fe480640 1 --2- 172.21.15.120:0/1957401733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4fe2969a0 0x55b4fe2a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.406+0000 7f87fdc7f640 1 --2- 172.21.15.120:0/1957401733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4fe2972e0 0x55b4fe2a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.406+0000 7f87fec81640 1 -- 172.21.15.120:0/1957401733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4fe2972e0 msgr2=0x55b4fe2a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.407+0000 7f87fec81640 1 --2- 172.21.15.120:0/1957401733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4fe2972e0 0x55b4fe2a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.407+0000 7f87fec81640 1 -- 172.21.15.120:0/1957401733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4fe2969a0 msgr2=0x55b4fe2a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.407+0000 7f87fec81640 1 --2- 172.21.15.120:0/1957401733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4fe2969a0 0x55b4fe2a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.407+0000 7f87fec81640 1 -- 172.21.15.120:0/1957401733 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4fdf0d210 con 0x55b4fe29c370 2024-09-19T09:04:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.407+0000 7f87fe480640 1 --2- 172.21.15.120:0/1957401733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4fe2969a0 0x55b4fe2a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:04:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.407+0000 7f87fdc7f640 1 --2- 172.21.15.120:0/1957401733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4fe2972e0 0x55b4fe2a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:04:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.407+0000 7f87fec81640 1 --2- 172.21.15.120:0/1957401733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4fe29c370 0x55b4fe2a81a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f87f00a4db0 tx=0x7f87f0009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=888f3bbec4fc182f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.407+0000 7f87e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:04:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.407+0000 7f87e1ffb640 1 -- 172.21.15.120:0/1957401733 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f00b4020 con 0x55b4fe29c370 2024-09-19T09:04:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.407+0000 7f87e1ffb640 1 -- 172.21.15.120:0/1957401733 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f87f0004310 con 0x55b4fe29c370 2024-09-19T09:04:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.407+0000 7f8803483740 1 -- 172.21.15.120:0/1957401733 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87e80318b0 con 0x55b4fe29c370 2024-09-19T09:04:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.407+0000 7f87e1ffb640 1 -- 172.21.15.120:0/1957401733 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f87f00a0a20 con 0x55b4fe29c370 2024-09-19T09:04:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.407+0000 7f8803483740 10 client.15762 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.407+0000 7f8803483740 1 -- 172.21.15.120:0/1957401733 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b4fdfd9620 con 0x55b4fe29c370 2024-09-19T09:04:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.407+0000 7f87e1ffb640 1 -- 172.21.15.120:0/1957401733 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87e80318b0 con 0x55b4fe29c370 2024-09-19T09:04:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.407+0000 7f87e1ffb640 1 -- 172.21.15.120:0/1957401733 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f87f00b0b90 con 0x55b4fe29c370 2024-09-19T09:04:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.407+0000 7f8803483740 10 client.15762 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.407+0000 7f8803483740 10 client.15762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:04:08.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.407+0000 7f8803483740 20 client.15762 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:08.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.407+0000 7f8803483740 1 --2- 172.21.15.120:0/1957401733 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b4fe2ac900 0x55b4fe2aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.407+0000 7f8803483740 4 client.15762 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:04:08.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.407+0000 7f8803483740 1 -- 172.21.15.120:0/1957401733 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4fe207a20 con 0x55b4fe2ac900 2024-09-19T09:04:08.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.408+0000 7f87fe480640 1 --2- 172.21.15.120:0/1957401733 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b4fe2ac900 0x55b4fe2aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.408+0000 7f87e1ffb640 10 client.15762 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:08.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.408+0000 7f87fe480640 1 --2- 172.21.15.120:0/1957401733 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b4fe2ac900 0x55b4fe2aecf0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc13e6c72cdbec58 server_cookie=48c905f0e8275bec in_seq=0 out_seq=0 2024-09-19T09:04:08.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.410+0000 7f87e1ffb640 1 -- 172.21.15.120:0/1957401733 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b4fe207a20 con 0x55b4fe2ac900 2024-09-19T09:04:08.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.410+0000 7f87e1ffb640 10 client.15762 handle_command_reply: tid=0 2024-09-19T09:04:08.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.410+0000 7f8803483740 1 client.15762 shutdown 2024-09-19T09:04:08.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.410+0000 7f8803483740 1 -- 172.21.15.120:0/1957401733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4fe29c370 msgr2=0x55b4fe2a81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:08.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.410+0000 7f8803483740 1 --2- 172.21.15.120:0/1957401733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4fe29c370 0x55b4fe2a81a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f87f00a4db0 tx=0x7f87f0009820 comp rx=0 tx=0).stop 2024-09-19T09:04:08.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.410+0000 7f8803483740 1 -- 172.21.15.120:0/1957401733 shutdown_connections 2024-09-19T09:04:08.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.410+0000 7f8803483740 1 --2- 172.21.15.120:0/1957401733 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b4fe2ac900 0x55b4fe2aecf0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.410+0000 7f87e1ffb640 0 client.15762 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:08.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.410+0000 7f8803483740 1 --2- 172.21.15.120:0/1957401733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4fe29c370 0x55b4fe2a81a0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.410+0000 7f8803483740 1 --2- 172.21.15.120:0/1957401733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4fe2972e0 0x55b4fe2a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.410+0000 7f8803483740 1 --2- 172.21.15.120:0/1957401733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4fe2969a0 0x55b4fe2a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.410+0000 7f8803483740 1 -- 172.21.15.120:0/1957401733 >> 172.21.15.120:0/1957401733 conn(0x55b4fe292010 msgr2=0x55b4fe293820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:08.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.410+0000 7f8803483740 1 -- 172.21.15.120:0/1957401733 shutdown_connections 2024-09-19T09:04:08.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.410+0000 7f8803483740 1 -- 172.21.15.120:0/1957401733 wait complete. 2024-09-19T09:04:08.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.410+0000 7f8803483740 20 client.15762 trim_cache size 0 max 16384 2024-09-19T09:04:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.424+0000 7f8803483740 1 Processor -- start 2024-09-19T09:04:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.424+0000 7f8803483740 1 -- start start 2024-09-19T09:04:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.424+0000 7f8803483740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4fe2969a0 0x55b4fe292ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.424+0000 7f8803483740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4fe2972e0 0x55b4fe2a8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.424+0000 7f8803483740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4fe29c370 0x55b4fe293400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.424+0000 7f8803483740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f87f81ab0a0 con 0x55b4fe29c370 2024-09-19T09:04:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.424+0000 7f8803483740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f87f81acbe0 con 0x55b4fe2969a0 2024-09-19T09:04:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.424+0000 7f8803483740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f87f81ab770 con 0x55b4fe2972e0 2024-09-19T09:04:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.424+0000 7f87fec81640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4fe29c370 0x55b4fe293400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.424+0000 7f87fec81640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4fe29c370 0x55b4fe293400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45162/0 (socket says 172.21.15.120:45162) 2024-09-19T09:04:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.424+0000 7f87fec81640 1 -- 172.21.15.120:0/1850627184 learned_addr learned my addr 172.21.15.120:0/1850627184 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.424+0000 7f87fe480640 1 --2- 172.21.15.120:0/1850627184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4fe2969a0 0x55b4fe292ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.425+0000 7f87fec81640 1 -- 172.21.15.120:0/1850627184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4fe2972e0 msgr2=0x55b4fe2a8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.425+0000 7f87fdc7f640 1 --2- 172.21.15.120:0/1850627184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4fe2972e0 0x55b4fe2a8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.425+0000 7f87fec81640 1 --2- 172.21.15.120:0/1850627184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4fe2972e0 0x55b4fe2a8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.425+0000 7f87fec81640 1 -- 172.21.15.120:0/1850627184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4fe2969a0 msgr2=0x55b4fe292ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.425+0000 7f87fec81640 1 --2- 172.21.15.120:0/1850627184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4fe2969a0 0x55b4fe292ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.425+0000 7f87fec81640 1 -- 172.21.15.120:0/1850627184 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4fe2af060 con 0x55b4fe29c370 2024-09-19T09:04:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.425+0000 7f87fec81640 1 --2- 172.21.15.120:0/1850627184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4fe29c370 0x55b4fe293400 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f87f0099040 tx=0x7f87f009ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c475b38684e71263 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.425+0000 7f87e3fff640 1 -- 172.21.15.120:0/1850627184 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f00b4030 con 0x55b4fe29c370 2024-09-19T09:04:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.425+0000 7f87e3fff640 1 -- 172.21.15.120:0/1850627184 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87f0083020 con 0x55b4fe29c370 2024-09-19T09:04:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.425+0000 7f87e3fff640 1 -- 172.21.15.120:0/1850627184 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f0004430 con 0x55b4fe29c370 2024-09-19T09:04:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.425+0000 7f8803483740 1 -- 172.21.15.120:0/1850627184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4fe29c370 msgr2=0x55b4fe293400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.425+0000 7f8803483740 1 --2- 172.21.15.120:0/1850627184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4fe29c370 0x55b4fe293400 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f87f0099040 tx=0x7f87f009ab80 comp rx=0 tx=0).stop 2024-09-19T09:04:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.425+0000 7f8803483740 1 -- 172.21.15.120:0/1850627184 shutdown_connections 2024-09-19T09:04:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.425+0000 7f8803483740 1 --2- 172.21.15.120:0/1850627184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4fe29c370 0x55b4fe293400 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.425+0000 7f8803483740 1 --2- 172.21.15.120:0/1850627184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4fe2972e0 0x55b4fe2a8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.425+0000 7f8803483740 1 --2- 172.21.15.120:0/1850627184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4fe2969a0 0x55b4fe292ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.425+0000 7f8803483740 1 -- 172.21.15.120:0/1850627184 >> 172.21.15.120:0/1850627184 conn(0x55b4fe292010 msgr2=0x55b4fe2a0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.426+0000 7f8803483740 1 -- 172.21.15.120:0/1850627184 shutdown_connections 2024-09-19T09:04:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.426+0000 7f8803483740 1 -- 172.21.15.120:0/1850627184 wait complete. 2024-09-19T09:04:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.426+0000 7f8803483740 1 -- start start 2024-09-19T09:04:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.426+0000 7f8803483740 1 Processor -- start 2024-09-19T09:04:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.426+0000 7f8803483740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4fe2969a0 0x55b4fe2a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.426+0000 7f8803483740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4fe2972e0 0x55b4fe2a5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.426+0000 7f8803483740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4fe29c370 0x55b4fe2a6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.426+0000 7f8803483740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f87f81ac510 con 0x55b4fe2969a0 2024-09-19T09:04:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.426+0000 7f8803483740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f87f81abe40 con 0x55b4fe29c370 2024-09-19T09:04:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.426+0000 7f8803483740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f87f81a9060 con 0x55b4fe2972e0 2024-09-19T09:04:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.426+0000 7f87fe480640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4fe2969a0 0x55b4fe2a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.426+0000 7f87fe480640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4fe2969a0 0x55b4fe2a5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45168/0 (socket says 172.21.15.120:45168) 2024-09-19T09:04:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.426+0000 7f87fe480640 1 -- 172.21.15.120:0/2400878992 learned_addr learned my addr 172.21.15.120:0/2400878992 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.426+0000 7f87fec81640 1 --2- 172.21.15.120:0/2400878992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4fe29c370 0x55b4fe2a6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.426+0000 7f87fdc7f640 1 --2- 172.21.15.120:0/2400878992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4fe2972e0 0x55b4fe2a5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.426+0000 7f8803483740 10 client.0 _ll_register_callbacks cb 0x7f87f0009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.426+0000 7f87fe480640 1 -- 172.21.15.120:0/2400878992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4fe2972e0 msgr2=0x55b4fe2a5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.426+0000 7f87fe480640 1 --2- 172.21.15.120:0/2400878992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4fe2972e0 0x55b4fe2a5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.427+0000 7f87fe480640 1 -- 172.21.15.120:0/2400878992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4fe29c370 msgr2=0x55b4fe2a6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.427+0000 7f87fe480640 1 --2- 172.21.15.120:0/2400878992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4fe29c370 0x55b4fe2a6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.427+0000 7f87fe480640 1 -- 172.21.15.120:0/2400878992 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4fe2a7580 con 0x55b4fe2969a0 2024-09-19T09:04:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.427+0000 7f87fe480640 1 --2- 172.21.15.120:0/2400878992 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4fe2969a0 0x55b4fe2a5a70 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x55b4fe292e90 tx=0x7f87f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca5ce2b407057b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.427+0000 7f87e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:04:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.427+0000 7f87e1ffb640 1 -- 172.21.15.120:0/2400878992 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4004920 con 0x55b4fe2969a0 2024-09-19T09:04:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.427+0000 7f87e1ffb640 1 -- 172.21.15.120:0/2400878992 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f87f4004ac0 con 0x55b4fe2969a0 2024-09-19T09:04:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.427+0000 7f8803483740 1 -- 172.21.15.120:0/2400878992 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87f0083020 con 0x55b4fe2969a0 2024-09-19T09:04:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.427+0000 7f87e1ffb640 1 -- 172.21.15.120:0/2400878992 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f87f4004cc0 con 0x55b4fe2969a0 2024-09-19T09:04:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.427+0000 7f8803483740 10 client.15768 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.427+0000 7f8803483740 1 -- 172.21.15.120:0/2400878992 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b4fe317860 con 0x55b4fe2969a0 2024-09-19T09:04:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.427+0000 7f87e1ffb640 1 -- 172.21.15.120:0/2400878992 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87f0083020 con 0x55b4fe2969a0 2024-09-19T09:04:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.427+0000 7f87e1ffb640 1 -- 172.21.15.120:0/2400878992 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f87f401d5d0 con 0x55b4fe2969a0 2024-09-19T09:04:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.427+0000 7f8803483740 10 client.15768 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.427+0000 7f8803483740 10 client.15768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:04:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.427+0000 7f8803483740 20 client.15768 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.427+0000 7f8803483740 1 --2- 172.21.15.120:0/2400878992 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b4fe2ac900 0x55b4fe31a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.427+0000 7f8803483740 4 client.15768 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:04:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.427+0000 7f8803483740 1 -- 172.21.15.120:0/2400878992 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b4fe201d70 con 0x55b4fe2ac900 2024-09-19T09:04:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.428+0000 7f87fdc7f640 1 --2- 172.21.15.120:0/2400878992 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b4fe2ac900 0x55b4fe31a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.428+0000 7f87e1ffb640 10 client.15768 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.428+0000 7f87fdc7f640 1 --2- 172.21.15.120:0/2400878992 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b4fe2ac900 0x55b4fe31a230 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aed4a013576300a9 server_cookie=1a1985209a80baf7 in_seq=0 out_seq=0 2024-09-19T09:04:08.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.570+0000 7f87e1ffb640 1 -- 172.21.15.120:0/2400878992 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b4fe201d70 con 0x55b4fe2ac900 2024-09-19T09:04:08.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.570+0000 7f87e1ffb640 10 client.15768 handle_command_reply: tid=0 2024-09-19T09:04:08.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.570+0000 7f8803483740 1 client.15768 shutdown 2024-09-19T09:04:08.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.570+0000 7f8803483740 1 -- 172.21.15.120:0/2400878992 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4fe2969a0 msgr2=0x55b4fe2a5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:08.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.570+0000 7f8803483740 1 --2- 172.21.15.120:0/2400878992 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4fe2969a0 0x55b4fe2a5a70 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x55b4fe292e90 tx=0x7f87f40046c0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.570+0000 7f113dffb640 1 -- 172.21.15.120:0/1582082272 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5575e074bbe0 con 0x5575e0865b40 2024-09-19T09:04:08.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.570+0000 7f113dffb640 10 client.25169 handle_command_reply: tid=0 2024-09-19T09:04:08.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.570+0000 7f115e7c1740 1 client.25169 shutdown 2024-09-19T09:04:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.570+0000 7f8803483740 1 -- 172.21.15.120:0/2400878992 shutdown_connections 2024-09-19T09:04:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.570+0000 7f8803483740 1 --2- 172.21.15.120:0/2400878992 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b4fe2ac900 0x55b4fe31a230 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.570+0000 7f8803483740 1 --2- 172.21.15.120:0/2400878992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4fe29c370 0x55b4fe2a6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.570+0000 7f8803483740 1 --2- 172.21.15.120:0/2400878992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4fe2972e0 0x55b4fe2a5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.570+0000 7f8803483740 1 --2- 172.21.15.120:0/2400878992 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4fe2969a0 0x55b4fe2a5a70 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.571+0000 7f8803483740 1 -- 172.21.15.120:0/2400878992 >> 172.21.15.120:0/2400878992 conn(0x55b4fe292010 msgr2=0x55b4fe2909a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.571+0000 7f87e1ffb640 0 client.15768 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:08.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.570+0000 7f115e7c1740 1 -- 172.21.15.120:0/1582082272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e07e0170 msgr2=0x5575e07eecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:08.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.570+0000 7f115e7c1740 1 --2- 172.21.15.120:0/1582082272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e07e0170 0x5575e07eecd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x5575e078ac50 tx=0x7f114400e640 comp rx=0 tx=0).stop 2024-09-19T09:04:08.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.571+0000 7f115e7c1740 1 -- 172.21.15.120:0/1582082272 shutdown_connections 2024-09-19T09:04:08.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.571+0000 7f115e7c1740 1 --2- 172.21.15.120:0/1582082272 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5575e0865b40 0x5575e0867f60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.571+0000 7f8803483740 1 -- 172.21.15.120:0/2400878992 shutdown_connections 2024-09-19T09:04:08.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.571+0000 7f115e7c1740 1 --2- 172.21.15.120:0/1582082272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575e07eaa30 0x5575e07ef210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.571+0000 7f115e7c1740 1 --2- 172.21.15.120:0/1582082272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5575e07e0170 0x5575e07eecd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.571+0000 7f115e7c1740 1 --2- 172.21.15.120:0/1582082272 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575e07df850 0x5575e07ee790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.571+0000 7f115e7c1740 1 -- 172.21.15.120:0/1582082272 >> 172.21.15.120:0/1582082272 conn(0x5575e07dbde0 msgr2=0x5575e07e91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.571+0000 7f8803483740 1 -- 172.21.15.120:0/2400878992 wait complete. 2024-09-19T09:04:08.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.571+0000 7f113dffb640 0 client.25169 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:08.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.571+0000 7f115e7c1740 1 -- 172.21.15.120:0/1582082272 shutdown_connections 2024-09-19T09:04:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.571+0000 7f8803483740 20 client.15768 trim_cache size 0 max 16384 2024-09-19T09:04:08.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.571+0000 7f115e7c1740 1 -- 172.21.15.120:0/1582082272 wait complete. 2024-09-19T09:04:08.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.571+0000 7f115e7c1740 20 client.25169 trim_cache size 0 max 16384 2024-09-19T09:04:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:04:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:04:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:04:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:04:08.576 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:04:08.576 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 28, 2024-09-19T09:04:08.577 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:04:08.577 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 6, 2024-09-19T09:04:08.577 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:04:08.577 INFO:teuthology.orchestra.run.smithi120.stdout: "5c43545b": { 2024-09-19T09:04:08.577 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 6, 2024-09-19T09:04:08.577 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:04:08.577 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:04:08.577 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:04:08.577 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:04:08.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:04:08.577 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:04:08.577 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 140.3, 2024-09-19T09:04:08.577 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:04:08.577 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:04:08.578 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:04:08.578 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:04:08.578 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:04:08.578 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:04:08.578 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:04:08.578 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:04:08.578 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:04:08.578 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:04:08.578 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:04:08.578 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:04:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.572+0000 7f88006e4640 1 -- 172.21.15.120:0/733237212 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f87e40a3370 msgr2=0x7f87e40a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.572+0000 7f88006e4640 1 --2- 172.21.15.120:0/733237212 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f87e40a3370 0x7f87e40a5830 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55b4fe297d10 tx=0x7f87f4005f90 comp rx=0 tx=0).stop 2024-09-19T09:04:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.572+0000 7f88006e4640 1 -- 172.21.15.120:0/733237212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87f8103710 msgr2=0x7f87f8074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.572+0000 7f88006e4640 1 --2- 172.21.15.120:0/733237212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87f8103710 0x7f87f8074b20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f87f000efd0 tx=0x7f87f000c6a0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.572+0000 7f88006e4640 1 -- 172.21.15.120:0/733237212 shutdown_connections 2024-09-19T09:04:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.572+0000 7f88006e4640 1 --2- 172.21.15.120:0/733237212 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f87e40a3370 0x7f87e40a5830 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.572+0000 7f88006e4640 1 --2- 172.21.15.120:0/733237212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87f8103710 0x7f87f8074b20 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.572+0000 7f88006e4640 1 --2- 172.21.15.120:0/733237212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87f8102d50 0x7f87f80745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.572+0000 7f88006e4640 1 --2- 172.21.15.120:0/733237212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87f8101b60 0x7f87f8075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.572+0000 7f88006e4640 1 -- 172.21.15.120:0/733237212 >> 172.21.15.120:0/733237212 conn(0x7f87f80fd2e0 msgr2=0x7f87f81060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:08.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.572+0000 7f115c3e8640 1 -- 172.21.15.120:0/1206047931 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f11400a3240 msgr2=0x7f11400a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:08.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.572+0000 7f115c3e8640 1 --2- 172.21.15.120:0/1206047931 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f11400a3240 0x7f11400a5700 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f114c005fd0 tx=0x7f114c005cc0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.572+0000 7f115c3e8640 1 -- 172.21.15.120:0/1206047931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1154100e20 msgr2=0x7f1154199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:08.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.572+0000 7f115c3e8640 1 --2- 172.21.15.120:0/1206047931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1154100e20 0x7f1154199390 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1150005cc0 tx=0x7f115002f630 comp rx=0 tx=0).stop 2024-09-19T09:04:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.572+0000 7f88006e4640 1 -- 172.21.15.120:0/733237212 shutdown_connections 2024-09-19T09:04:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:08.572+0000 7f88006e4640 1 -- 172.21.15.120:0/733237212 wait complete. 2024-09-19T09:04:08.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.572+0000 7f115c3e8640 1 -- 172.21.15.120:0/1206047931 shutdown_connections 2024-09-19T09:04:08.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.572+0000 7f115c3e8640 1 --2- 172.21.15.120:0/1206047931 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f11400a3240 0x7f11400a5700 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.572+0000 7f115c3e8640 1 --2- 172.21.15.120:0/1206047931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1154102ef0 0x7f11541a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.572+0000 7f115c3e8640 1 --2- 172.21.15.120:0/1206047931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1154102570 0x7f11541998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.572+0000 7f115c3e8640 1 --2- 172.21.15.120:0/1206047931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1154100e20 0x7f1154199390 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:08.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.572+0000 7f115c3e8640 1 -- 172.21.15.120:0/1206047931 >> 172.21.15.120:0/1206047931 conn(0x7f11540fc7d0 msgr2=0x7f11540fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:08.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.572+0000 7f115c3e8640 1 -- 172.21.15.120:0/1206047931 shutdown_connections 2024-09-19T09:04:08.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:08.572+0000 7f115c3e8640 1 -- 172.21.15.120:0/1206047931 wait complete. 2024-09-19T09:04:08.824 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:08 smithi120 ceph-mon[31150]: pgmap v379: 97 pgs: 97 active+clean; 1.8 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 119 KiB/s wr, 29 op/s 2024-09-19T09:04:09.033 INFO:tasks.workunit.client.0.smithi120.stdout: 1 11836 20.09 MB/sec warmup 4 sec latency 498.268 ms 2024-09-19T09:04:09.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:08 smithi196 ceph-mon[35150]: pgmap v379: 97 pgs: 97 active+clean; 1.8 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 119 KiB/s wr, 29 op/s 2024-09-19T09:04:09.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:08 smithi153 ceph-mon[35071]: pgmap v379: 97 pgs: 97 active+clean; 1.8 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 119 KiB/s wr, 29 op/s 2024-09-19T09:04:09.300 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '5c43545b', quiesce took 3.7 seconds. Will release after: 16.650000000000002 2024-09-19T09:04:10.034 INFO:tasks.workunit.client.0.smithi120.stdout: 1 11836 16.07 MB/sec warmup 5 sec latency 1498.328 ms 2024-09-19T09:04:11.034 INFO:tasks.workunit.client.0.smithi120.stdout: 1 11836 13.40 MB/sec warmup 6 sec latency 2498.395 ms 2024-09-19T09:04:11.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:10 smithi120 ceph-mon[31150]: pgmap v380: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 236 KiB/s wr, 33 op/s 2024-09-19T09:04:11.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:10 smithi196 ceph-mon[35150]: pgmap v380: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 236 KiB/s wr, 33 op/s 2024-09-19T09:04:11.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:10 smithi153 ceph-mon[35071]: pgmap v380: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 236 KiB/s wr, 33 op/s 2024-09-19T09:04:12.034 INFO:tasks.workunit.client.0.smithi120.stdout: 1 11836 11.48 MB/sec warmup 7 sec latency 3498.481 ms 2024-09-19T09:04:13.034 INFO:tasks.workunit.client.0.smithi120.stdout: 1 11836 10.05 MB/sec warmup 8 sec latency 4498.576 ms 2024-09-19T09:04:13.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:12 smithi120 ceph-mon[31150]: pgmap v381: 97 pgs: 97 active+clean; 6.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 609 KiB/s wr, 112 op/s 2024-09-19T09:04:13.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:12 smithi196 ceph-mon[35150]: pgmap v381: 97 pgs: 97 active+clean; 6.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 609 KiB/s wr, 112 op/s 2024-09-19T09:04:13.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:12 smithi153 ceph-mon[35071]: pgmap v381: 97 pgs: 97 active+clean; 6.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 609 KiB/s wr, 112 op/s 2024-09-19T09:04:14.034 INFO:tasks.workunit.client.0.smithi120.stdout: 1 11836 8.93 MB/sec warmup 9 sec latency 5498.685 ms 2024-09-19T09:04:15.034 INFO:tasks.workunit.client.0.smithi120.stdout: 1 11836 8.04 MB/sec warmup 10 sec latency 6498.779 ms 2024-09-19T09:04:15.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:14 smithi120 ceph-mon[31150]: pgmap v382: 97 pgs: 97 active+clean; 6.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 647 KiB/s wr, 147 op/s 2024-09-19T09:04:15.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:14 smithi196 ceph-mon[35150]: pgmap v382: 97 pgs: 97 active+clean; 6.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 647 KiB/s wr, 147 op/s 2024-09-19T09:04:15.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:14 smithi153 ceph-mon[35071]: pgmap v382: 97 pgs: 97 active+clean; 6.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 647 KiB/s wr, 147 op/s 2024-09-19T09:04:16.034 INFO:tasks.workunit.client.0.smithi120.stdout: 1 11836 7.31 MB/sec warmup 11 sec latency 7498.878 ms 2024-09-19T09:04:17.034 INFO:tasks.workunit.client.0.smithi120.stdout: 1 11836 6.70 MB/sec warmup 12 sec latency 8498.930 ms 2024-09-19T09:04:17.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:16 smithi120 ceph-mon[31150]: pgmap v383: 97 pgs: 97 active+clean; 6.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 647 KiB/s wr, 145 op/s 2024-09-19T09:04:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:16 smithi196 ceph-mon[35150]: pgmap v383: 97 pgs: 97 active+clean; 6.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 647 KiB/s wr, 145 op/s 2024-09-19T09:04:17.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:16 smithi153 ceph-mon[35071]: pgmap v383: 97 pgs: 97 active+clean; 6.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 647 KiB/s wr, 145 op/s 2024-09-19T09:04:18.034 INFO:tasks.workunit.client.0.smithi120.stdout: 1 11836 6.18 MB/sec warmup 13 sec latency 9499.027 ms 2024-09-19T09:04:18.125 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:04:18.383 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:04:18.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.913+0000 7fa4cb83c640 1 -- 172.21.15.120:0/90149446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4c41082e0 msgr2=0x7fa4c410ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:18.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.913+0000 7fa4cb83c640 1 --2- 172.21.15.120:0/90149446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4c41082e0 0x7fa4c410ec50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009cd0 tx=0x7fa4bc02a910 comp rx=0 tx=0).stop 2024-09-19T09:04:18.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.913+0000 7fa4cb83c640 1 -- 172.21.15.120:0/90149446 shutdown_connections 2024-09-19T09:04:18.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.913+0000 7fa4cb83c640 1 --2- 172.21.15.120:0/90149446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4c41082e0 0x7fa4c410ec50 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.913+0000 7fa4cb83c640 1 --2- 172.21.15.120:0/90149446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4c4103760 0x7fa4c4107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.913+0000 7fa4cb83c640 1 --2- 172.21.15.120:0/90149446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c4102db0 0x7fa4c4103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.913+0000 7fa4cb83c640 1 -- 172.21.15.120:0/90149446 >> 172.21.15.120:0/90149446 conn(0x7fa4c40fc7d0 msgr2=0x7fa4c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:18.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.913+0000 7fa4cb83c640 1 -- 172.21.15.120:0/90149446 shutdown_connections 2024-09-19T09:04:18.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.913+0000 7fa4cb83c640 1 -- 172.21.15.120:0/90149446 wait complete. 2024-09-19T09:04:18.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.914+0000 7fa4cb83c640 1 Processor -- start 2024-09-19T09:04:18.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.914+0000 7fa4cb83c640 1 -- start start 2024-09-19T09:04:18.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.914+0000 7fa4cb83c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c4102db0 0x7fa4c4198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:18.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.914+0000 7fa4cb83c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4c4103760 0x7fa4c4198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:18.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.914+0000 7fa4cb83c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4c41082e0 0x7fa4c419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:18.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.914+0000 7fa4cb83c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c41120f0 con 0x7fa4c41082e0 2024-09-19T09:04:18.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.914+0000 7fa4cb83c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c4111f70 con 0x7fa4c4103760 2024-09-19T09:04:18.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.914+0000 7fa4cb83c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c4112270 con 0x7fa4c4102db0 2024-09-19T09:04:18.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.914+0000 7fa4c9dd9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4c41082e0 0x7fa4c419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:18.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.915+0000 7fa4c95d8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c4102db0 0x7fa4c4198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:18.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.915+0000 7fa4c95d8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c4102db0 0x7fa4c4198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:56392/0 (socket says 172.21.15.120:56392) 2024-09-19T09:04:18.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.915+0000 7fa4c9dd9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4c41082e0 0x7fa4c419ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45118/0 (socket says 172.21.15.120:45118) 2024-09-19T09:04:18.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.915+0000 7fa4c95d8640 1 -- 172.21.15.120:0/3081485338 learned_addr learned my addr 172.21.15.120:0/3081485338 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:18.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.915+0000 7fa4c8dd7640 1 --2- 172.21.15.120:0/3081485338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4c4103760 0x7fa4c4198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:18.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.915+0000 7fa4c9dd9640 1 -- 172.21.15.120:0/3081485338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c4102db0 msgr2=0x7fa4c4198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:18.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.915+0000 7fa4c9dd9640 1 --2- 172.21.15.120:0/3081485338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c4102db0 0x7fa4c4198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.915+0000 7fa4c9dd9640 1 -- 172.21.15.120:0/3081485338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4c4103760 msgr2=0x7fa4c4198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:18.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.915+0000 7fa4c9dd9640 1 --2- 172.21.15.120:0/3081485338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4c4103760 0x7fa4c4198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.915+0000 7fa4c9dd9640 1 -- 172.21.15.120:0/3081485338 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c419d520 con 0x7fa4c41082e0 2024-09-19T09:04:18.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.915+0000 7fa4c9dd9640 1 --2- 172.21.15.120:0/3081485338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4c41082e0 0x7fa4c419ce40 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009ca0 tx=0x7fa4bc031460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:18.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.915+0000 7fa4ba7fc640 1 -- 172.21.15.120:0/3081485338 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc031850 con 0x7fa4c41082e0 2024-09-19T09:04:18.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.915+0000 7fa4cb83c640 1 -- 172.21.15.120:0/3081485338 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4c419d7b0 con 0x7fa4c41082e0 2024-09-19T09:04:18.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.915+0000 7fa4ba7fc640 1 -- 172.21.15.120:0/3081485338 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4bc004520 con 0x7fa4c41082e0 2024-09-19T09:04:18.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.915+0000 7fa4ba7fc640 1 -- 172.21.15.120:0/3081485338 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc02fd90 con 0x7fa4c41082e0 2024-09-19T09:04:18.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.915+0000 7fa4cb83c640 1 -- 172.21.15.120:0/3081485338 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4c41a50f0 con 0x7fa4c41082e0 2024-09-19T09:04:18.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.918+0000 7fa4ba7fc640 1 -- 172.21.15.120:0/3081485338 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa4bc0046d0 con 0x7fa4c41082e0 2024-09-19T09:04:18.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.918+0000 7fa4ba7fc640 1 --2- 172.21.15.120:0/3081485338 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa4b00a3240 0x7fa4b00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:18.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.918+0000 7fa4cdc15740 1 Processor -- start 2024-09-19T09:04:18.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.919+0000 7fa4cdc15740 1 -- start start 2024-09-19T09:04:18.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.919+0000 7fa4c95d8640 1 --2- 172.21.15.120:0/3081485338 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa4b00a3240 0x7fa4b00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:18.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.919+0000 7fa4ba7fc640 1 -- 172.21.15.120:0/3081485338 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa4bc0f0cf0 con 0x7fa4c41082e0 2024-09-19T09:04:18.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.919+0000 7fa4cdc15740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c6133d4a0 0x557c6133d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:18.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.919+0000 7fa4cdc15740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c6133ddc0 0x557c61348140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:18.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.919+0000 7fa4cdc15740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c61348680 0x557c6134aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:18.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.919+0000 7fa4cdc15740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557c612a32e0 con 0x557c6133d4a0 2024-09-19T09:04:18.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.919+0000 7fa4cdc15740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557c6124ace0 con 0x557c61348680 2024-09-19T09:04:18.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.919+0000 7fa4cdc15740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557c6129f2a0 con 0x557c6133ddc0 2024-09-19T09:04:18.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.919+0000 7fa4c95d8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c6133d4a0 0x557c6133d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:18.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.919+0000 7fa4c95d8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c6133d4a0 0x557c6133d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45126/0 (socket says 172.21.15.120:45126) 2024-09-19T09:04:18.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.919+0000 7fa4c95d8640 1 -- 172.21.15.120:0/1787183961 learned_addr learned my addr 172.21.15.120:0/1787183961 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:18.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.919+0000 7fa4c95d8640 1 --2- 172.21.15.120:0/3081485338 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa4b00a3240 0x7fa4b00a5700 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa4c00136d0 tx=0x7fa4c000af90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:18.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.919+0000 7fa4c8dd7640 1 --2- 172.21.15.120:0/1787183961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c6133ddc0 0x557c61348140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:18.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.919+0000 7fa4c9dd9640 1 --2- 172.21.15.120:0/1787183961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c61348680 0x557c6134aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:18.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.919+0000 7fa4c95d8640 1 -- 172.21.15.120:0/1787183961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c6133ddc0 msgr2=0x557c61348140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:18.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.919+0000 7fa4c95d8640 1 --2- 172.21.15.120:0/1787183961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c6133ddc0 0x557c61348140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.919+0000 7fa4c95d8640 1 -- 172.21.15.120:0/1787183961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c61348680 msgr2=0x557c6134aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:18.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.919+0000 7fa4c95d8640 1 --2- 172.21.15.120:0/1787183961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c61348680 0x557c6134aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.919+0000 7fa4c95d8640 1 -- 172.21.15.120:0/1787183961 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c6122f440 con 0x557c6133d4a0 2024-09-19T09:04:18.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.920+0000 7fa4c95d8640 1 --2- 172.21.15.120:0/1787183961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c6133d4a0 0x557c6133d880 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0013f60 tx=0x7fa4c001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a9761c846cd5164 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:18.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.920+0000 7fa4af7fe640 1 -- 172.21.15.120:0/1787183961 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c001ea30 con 0x557c6133d4a0 2024-09-19T09:04:18.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.920+0000 7fa4af7fe640 1 -- 172.21.15.120:0/1787183961 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4c001ebd0 con 0x557c6133d4a0 2024-09-19T09:04:18.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.920+0000 7fa4af7fe640 1 -- 172.21.15.120:0/1787183961 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c000e610 con 0x557c6133d4a0 2024-09-19T09:04:18.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.921+0000 7fa4cdc15740 1 -- 172.21.15.120:0/1787183961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c6133d4a0 msgr2=0x557c6133d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:18.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.921+0000 7fa4cdc15740 1 --2- 172.21.15.120:0/1787183961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c6133d4a0 0x557c6133d880 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0013f60 tx=0x7fa4c001e000 comp rx=0 tx=0).stop 2024-09-19T09:04:18.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.921+0000 7fa4cdc15740 1 -- 172.21.15.120:0/1787183961 shutdown_connections 2024-09-19T09:04:18.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.921+0000 7fa4cdc15740 1 --2- 172.21.15.120:0/1787183961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c61348680 0x557c6134aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.921+0000 7fa4cdc15740 1 --2- 172.21.15.120:0/1787183961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c6133ddc0 0x557c61348140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.921+0000 7fa4cdc15740 1 --2- 172.21.15.120:0/1787183961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c6133d4a0 0x557c6133d880 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.921+0000 7fa4cdc15740 1 -- 172.21.15.120:0/1787183961 >> 172.21.15.120:0/1787183961 conn(0x557c61339a30 msgr2=0x557c61339e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:18.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.921+0000 7fa4cdc15740 1 -- 172.21.15.120:0/1787183961 shutdown_connections 2024-09-19T09:04:18.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.921+0000 7fa4cdc15740 1 -- 172.21.15.120:0/1787183961 wait complete. 2024-09-19T09:04:18.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.921+0000 7fa4cdc15740 1 -- start start 2024-09-19T09:04:18.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.921+0000 7fa4cdc15740 1 Processor -- start 2024-09-19T09:04:18.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.922+0000 7fa4cdc15740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c6133d4a0 0x557c61347f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:18.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.922+0000 7fa4cdc15740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c6133ddc0 0x557c613451f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:18.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.922+0000 7fa4cdc15740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c61348680 0x557c61345730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:18.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.922+0000 7fa4cdc15740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557c6134b250 con 0x557c6133ddc0 2024-09-19T09:04:18.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.922+0000 7fa4cdc15740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557c6125e050 con 0x557c6133d4a0 2024-09-19T09:04:18.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.922+0000 7fa4cdc15740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557c6134b3d0 con 0x557c61348680 2024-09-19T09:04:18.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.922+0000 7fa4cdc15740 10 client.0 _ll_register_callbacks cb 0x557c6129e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:18.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.922+0000 7fa4c8dd7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c6133ddc0 0x557c613451f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:18.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.922+0000 7fa4c8dd7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c6133ddc0 0x557c613451f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45132/0 (socket says 172.21.15.120:45132) 2024-09-19T09:04:18.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.922+0000 7fa4c8dd7640 1 -- 172.21.15.120:0/3020188221 learned_addr learned my addr 172.21.15.120:0/3020188221 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:18.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.922+0000 7fa4c95d8640 1 --2- 172.21.15.120:0/3020188221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c6133d4a0 0x557c61347f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:18.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.922+0000 7fa4c9dd9640 1 --2- 172.21.15.120:0/3020188221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c61348680 0x557c61345730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:18.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.922+0000 7fa4c8dd7640 1 -- 172.21.15.120:0/3020188221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c61348680 msgr2=0x557c61345730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:18.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.922+0000 7fa4c8dd7640 1 --2- 172.21.15.120:0/3020188221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c61348680 0x557c61345730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.922+0000 7fa4c8dd7640 1 -- 172.21.15.120:0/3020188221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c6133d4a0 msgr2=0x557c61347f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:18.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.922+0000 7fa4c8dd7640 1 --2- 172.21.15.120:0/3020188221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c6133d4a0 0x557c61347f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.922+0000 7fa4c8dd7640 1 -- 172.21.15.120:0/3020188221 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c610143c0 con 0x557c6133ddc0 2024-09-19T09:04:18.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.923+0000 7fa4c8dd7640 1 --2- 172.21.15.120:0/3020188221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c6133ddc0 0x557c613451f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa4b400bf30 tx=0x7fa4b400f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf99348dc830acc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:18.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.923+0000 7fa4acff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:04:18.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.923+0000 7fa4acff9640 1 -- 172.21.15.120:0/3020188221 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4018020 con 0x557c6133ddc0 2024-09-19T09:04:18.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.923+0000 7fa4acff9640 1 -- 172.21.15.120:0/3020188221 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa4b4014670 con 0x557c6133ddc0 2024-09-19T09:04:18.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.923+0000 7fa4cdc15740 1 -- 172.21.15.120:0/3020188221 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4c001ebd0 con 0x557c6133ddc0 2024-09-19T09:04:18.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.923+0000 7fa4acff9640 1 -- 172.21.15.120:0/3020188221 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa4b4014870 con 0x557c6133ddc0 2024-09-19T09:04:18.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.923+0000 7fa4cdc15740 10 client.15780 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:18.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.923+0000 7fa4cdc15740 1 -- 172.21.15.120:0/3020188221 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557c61356c00 con 0x557c6133ddc0 2024-09-19T09:04:18.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.923+0000 7fa4acff9640 1 -- 172.21.15.120:0/3020188221 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4c001ebd0 con 0x557c6133ddc0 2024-09-19T09:04:18.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.923+0000 7fa4acff9640 1 -- 172.21.15.120:0/3020188221 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa4b401fc70 con 0x557c6133ddc0 2024-09-19T09:04:18.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.924+0000 7fa4cdc15740 10 client.15780 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:18.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.924+0000 7fa4cdc15740 10 client.15780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:04:18.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.924+0000 7fa4cdc15740 20 client.15780 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:18.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.924+0000 7fa4cdc15740 1 --2- 172.21.15.120:0/3020188221 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557c613580c0 0x557c6135a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:18.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.924+0000 7fa4cdc15740 4 client.15780 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:04:18.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.924+0000 7fa4cdc15740 1 -- 172.21.15.120:0/3020188221 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c612b0850 con 0x557c613580c0 2024-09-19T09:04:18.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.924+0000 7fa4c95d8640 1 --2- 172.21.15.120:0/3020188221 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557c613580c0 0x557c6135a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:18.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.924+0000 7fa4c95d8640 1 --2- 172.21.15.120:0/3020188221 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557c613580c0 0x557c6135a560 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77a0a9ddcb5f7720 server_cookie=34d091217e8ed1a8 in_seq=0 out_seq=0 2024-09-19T09:04:18.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.924+0000 7fa4acff9640 10 client.15780 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:18.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.925+0000 7fa4acff9640 1 -- 172.21.15.120:0/3020188221 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557c612b0850 con 0x557c613580c0 2024-09-19T09:04:18.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.925+0000 7fa4acff9640 10 client.15780 handle_command_reply: tid=0 2024-09-19T09:04:18.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.925+0000 7fa4cdc15740 1 client.15780 shutdown 2024-09-19T09:04:18.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.925+0000 7fa4cdc15740 1 -- 172.21.15.120:0/3020188221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c6133ddc0 msgr2=0x557c613451f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:18.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.925+0000 7fa4cdc15740 1 --2- 172.21.15.120:0/3020188221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c6133ddc0 0x557c613451f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa4b400bf30 tx=0x7fa4b400f670 comp rx=0 tx=0).stop 2024-09-19T09:04:18.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.925+0000 7fa4cdc15740 1 -- 172.21.15.120:0/3020188221 shutdown_connections 2024-09-19T09:04:18.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.925+0000 7fa4cdc15740 1 --2- 172.21.15.120:0/3020188221 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557c613580c0 0x557c6135a560 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.925+0000 7fa4cdc15740 1 --2- 172.21.15.120:0/3020188221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c61348680 0x557c61345730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.925+0000 7fa4cdc15740 1 --2- 172.21.15.120:0/3020188221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c6133ddc0 0x557c613451f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.925+0000 7fa4cdc15740 1 --2- 172.21.15.120:0/3020188221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c6133d4a0 0x557c61347f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.925+0000 7fa4cdc15740 1 -- 172.21.15.120:0/3020188221 >> 172.21.15.120:0/3020188221 conn(0x557c61339a30 msgr2=0x557c6133f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:18.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.925+0000 7fa4acff9640 0 client.15780 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:18.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.925+0000 7fa4cdc15740 1 -- 172.21.15.120:0/3020188221 shutdown_connections 2024-09-19T09:04:18.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.925+0000 7fa4cdc15740 1 -- 172.21.15.120:0/3020188221 wait complete. 2024-09-19T09:04:18.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.926+0000 7fa4cdc15740 20 client.15780 trim_cache size 0 max 16384 2024-09-19T09:04:18.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.959+0000 7fa4cdc15740 1 Processor -- start 2024-09-19T09:04:18.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.959+0000 7fa4cdc15740 1 -- start start 2024-09-19T09:04:18.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.960+0000 7fa4cdc15740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c6133d4a0 0x557c613544b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:18.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.960+0000 7fa4cdc15740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c6133ddc0 0x557c6135fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:18.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.960+0000 7fa4cdc15740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c61348680 0x557c61360370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:18.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.960+0000 7fa4cdc15740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c41ad590 con 0x557c6133d4a0 2024-09-19T09:04:18.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.960+0000 7fa4cdc15740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c41af0d0 con 0x557c61348680 2024-09-19T09:04:18.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.960+0000 7fa4cdc15740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c41adc60 con 0x557c6133ddc0 2024-09-19T09:04:18.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.960+0000 7fa4c95d8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c6133d4a0 0x557c613544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:18.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.960+0000 7fa4c95d8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c6133d4a0 0x557c613544b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45134/0 (socket says 172.21.15.120:45134) 2024-09-19T09:04:18.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.960+0000 7fa4c95d8640 1 -- 172.21.15.120:0/3520309017 learned_addr learned my addr 172.21.15.120:0/3520309017 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:18.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.960+0000 7fa4c8dd7640 1 --2- 172.21.15.120:0/3520309017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c6133ddc0 0x557c6135fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:18.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.960+0000 7fa4c9dd9640 1 --2- 172.21.15.120:0/3520309017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c61348680 0x557c61360370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:18.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.960+0000 7fa4c95d8640 1 -- 172.21.15.120:0/3520309017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c6133ddc0 msgr2=0x557c6135fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:18.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.960+0000 7fa4c95d8640 1 --2- 172.21.15.120:0/3520309017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c6133ddc0 0x557c6135fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.960+0000 7fa4c95d8640 1 -- 172.21.15.120:0/3520309017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c61348680 msgr2=0x557c61360370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:18.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.960+0000 7fa4c95d8640 1 --2- 172.21.15.120:0/3520309017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c61348680 0x557c61360370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.960+0000 7fa4c95d8640 1 -- 172.21.15.120:0/3520309017 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c6133aae0 con 0x557c6133d4a0 2024-09-19T09:04:18.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.960+0000 7fa4c95d8640 1 --2- 172.21.15.120:0/3520309017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c6133d4a0 0x557c613544b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0005f20 tx=0x7fa4c0021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c969f2226f8b0490 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:18.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.960+0000 7fa4aeffd640 1 -- 172.21.15.120:0/3520309017 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0013040 con 0x557c6133d4a0 2024-09-19T09:04:18.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.961+0000 7fa4aeffd640 1 -- 172.21.15.120:0/3520309017 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4c00131e0 con 0x557c6133d4a0 2024-09-19T09:04:18.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.961+0000 7fa4aeffd640 1 -- 172.21.15.120:0/3520309017 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c00055e0 con 0x557c6133d4a0 2024-09-19T09:04:18.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.961+0000 7fa4cdc15740 1 -- 172.21.15.120:0/3520309017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c6133d4a0 msgr2=0x557c613544b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:18.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.961+0000 7fa4cdc15740 1 --2- 172.21.15.120:0/3520309017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c6133d4a0 0x557c613544b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0005f20 tx=0x7fa4c0021040 comp rx=0 tx=0).stop 2024-09-19T09:04:18.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.961+0000 7fa4cdc15740 1 -- 172.21.15.120:0/3520309017 shutdown_connections 2024-09-19T09:04:18.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.961+0000 7fa4cdc15740 1 --2- 172.21.15.120:0/3520309017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c61348680 0x557c61360370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.961+0000 7fa4cdc15740 1 --2- 172.21.15.120:0/3520309017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c6133ddc0 0x557c6135fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.961+0000 7fa4cdc15740 1 --2- 172.21.15.120:0/3520309017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c6133d4a0 0x557c613544b0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.961+0000 7fa4cdc15740 1 -- 172.21.15.120:0/3520309017 >> 172.21.15.120:0/3520309017 conn(0x557c61339a30 msgr2=0x557c613442e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:18.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.961+0000 7fa4cdc15740 1 -- 172.21.15.120:0/3520309017 shutdown_connections 2024-09-19T09:04:18.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.961+0000 7fa4cdc15740 1 -- 172.21.15.120:0/3520309017 wait complete. 2024-09-19T09:04:18.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.962+0000 7fa4cdc15740 1 -- start start 2024-09-19T09:04:18.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.962+0000 7fa4cdc15740 1 Processor -- start 2024-09-19T09:04:18.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.962+0000 7fa4cdc15740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c6133d4a0 0x557c6134d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:18.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.962+0000 7fa4cdc15740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c6133ddc0 0x557c6134dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:18.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.962+0000 7fa4cdc15740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c61348680 0x557c613c0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:18.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.962+0000 7fa4cdc15740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c41aea00 con 0x557c6133d4a0 2024-09-19T09:04:18.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.963+0000 7fa4c95d8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c6133d4a0 0x557c6134d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:18.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.963+0000 7fa4cdc15740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c41ae330 con 0x557c6133ddc0 2024-09-19T09:04:18.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.963+0000 7fa4cdc15740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c41ab550 con 0x557c61348680 2024-09-19T09:04:18.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.963+0000 7fa4c95d8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c6133d4a0 0x557c6134d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45146/0 (socket says 172.21.15.120:45146) 2024-09-19T09:04:18.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.963+0000 7fa4c95d8640 1 -- 172.21.15.120:0/3757129463 learned_addr learned my addr 172.21.15.120:0/3757129463 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:18.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.963+0000 7fa4c9dd9640 1 --2- 172.21.15.120:0/3757129463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c61348680 0x557c613c0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:18.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.963+0000 7fa4cdc15740 10 client.0 _ll_register_callbacks cb 0x7fa4b400b240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:18.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.963+0000 7fa4c8dd7640 1 --2- 172.21.15.120:0/3757129463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c6133ddc0 0x557c6134dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:18.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.963+0000 7fa4c95d8640 1 -- 172.21.15.120:0/3757129463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c61348680 msgr2=0x557c613c0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:18.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.963+0000 7fa4c95d8640 1 --2- 172.21.15.120:0/3757129463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c61348680 0x557c613c0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.963+0000 7fa4c95d8640 1 -- 172.21.15.120:0/3757129463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c6133ddc0 msgr2=0x557c6134dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:18.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.963+0000 7fa4c95d8640 1 --2- 172.21.15.120:0/3757129463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c6133ddc0 0x557c6134dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.963+0000 7fa4c95d8640 1 -- 172.21.15.120:0/3757129463 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c6134e4f0 con 0x557c6133d4a0 2024-09-19T09:04:18.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.963+0000 7fa4c95d8640 1 --2- 172.21.15.120:0/3757129463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c6133d4a0 0x557c6134d6f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0005080 tx=0x7fa4c000a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e01c80dd5e3c0f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:18.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.963+0000 7fa4acff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:04:18.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.963+0000 7fa4acff9640 1 -- 172.21.15.120:0/3757129463 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c000aa00 con 0x557c6133d4a0 2024-09-19T09:04:18.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.963+0000 7fa4acff9640 1 -- 172.21.15.120:0/3757129463 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa4c0005530 con 0x557c6133d4a0 2024-09-19T09:04:18.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.963+0000 7fa4cdc15740 1 -- 172.21.15.120:0/3757129463 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4c00131e0 con 0x557c6133d4a0 2024-09-19T09:04:18.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.964+0000 7fa4acff9640 1 -- 172.21.15.120:0/3757129463 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa4c001f450 con 0x557c6133d4a0 2024-09-19T09:04:18.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.964+0000 7fa4cdc15740 10 client.15786 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:18.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.964+0000 7fa4cdc15740 1 -- 172.21.15.120:0/3757129463 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557c613c2450 con 0x557c6133d4a0 2024-09-19T09:04:18.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.964+0000 7fa4acff9640 1 -- 172.21.15.120:0/3757129463 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4c00131e0 con 0x557c6133d4a0 2024-09-19T09:04:18.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.964+0000 7fa4acff9640 1 -- 172.21.15.120:0/3757129463 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa4c003f6d0 con 0x557c6133d4a0 2024-09-19T09:04:18.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.964+0000 7fa4cdc15740 10 client.15786 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:18.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.964+0000 7fa4cdc15740 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:04:18.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.964+0000 7fa4cdc15740 20 client.15786 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:18.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.964+0000 7fa4cdc15740 1 --2- 172.21.15.120:0/3757129463 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557c613c2970 0x557c613c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:18.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.964+0000 7fa4cdc15740 4 client.15786 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:04:18.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.964+0000 7fa4cdc15740 1 -- 172.21.15.120:0/3757129463 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557c612aad60 con 0x557c613c2970 2024-09-19T09:04:18.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.964+0000 7fa4c8dd7640 1 --2- 172.21.15.120:0/3757129463 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557c613c2970 0x557c613c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:18.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.965+0000 7fa4c8dd7640 1 --2- 172.21.15.120:0/3757129463 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557c613c2970 0x557c613c4d60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a7c831d9f18068f server_cookie=67a429fc76f20b0b in_seq=0 out_seq=0 2024-09-19T09:04:18.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.965+0000 7fa4acff9640 10 client.15786 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:18.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.965+0000 7fa4acff9640 1 -- 172.21.15.120:0/3757129463 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557c612aad60 con 0x557c613c2970 2024-09-19T09:04:18.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.965+0000 7fa4acff9640 10 client.15786 handle_command_reply: tid=0 2024-09-19T09:04:18.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.965+0000 7fa4cdc15740 1 client.15786 shutdown 2024-09-19T09:04:18.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.965+0000 7fa4cdc15740 1 -- 172.21.15.120:0/3757129463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c6133d4a0 msgr2=0x557c6134d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:18.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.965+0000 7fa4cdc15740 1 --2- 172.21.15.120:0/3757129463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c6133d4a0 0x557c6134d6f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0005080 tx=0x7fa4c000a400 comp rx=0 tx=0).stop 2024-09-19T09:04:18.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.965+0000 7fa4cdc15740 1 -- 172.21.15.120:0/3757129463 shutdown_connections 2024-09-19T09:04:18.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.965+0000 7fa4cdc15740 1 --2- 172.21.15.120:0/3757129463 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557c613c2970 0x557c613c4d60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.965+0000 7fa4cdc15740 1 --2- 172.21.15.120:0/3757129463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557c61348680 0x557c613c0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.965+0000 7fa4cdc15740 1 --2- 172.21.15.120:0/3757129463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c6133ddc0 0x557c6134dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.965+0000 7fa4acff9640 0 client.15786 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:18.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.965+0000 7fa4cdc15740 1 --2- 172.21.15.120:0/3757129463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c6133d4a0 0x557c6134d6f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.965+0000 7fa4cdc15740 1 -- 172.21.15.120:0/3757129463 >> 172.21.15.120:0/3757129463 conn(0x557c61339a30 msgr2=0x557c61346a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:18.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.965+0000 7fa4cdc15740 1 -- 172.21.15.120:0/3757129463 shutdown_connections 2024-09-19T09:04:18.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.965+0000 7fa4cdc15740 1 -- 172.21.15.120:0/3757129463 wait complete. 2024-09-19T09:04:18.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.965+0000 7fa4cdc15740 20 client.15786 trim_cache size 0 max 16384 2024-09-19T09:04:18.971 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:04:18.971 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:04:18.971 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:04:18.971 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:04:18.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.966+0000 7fa4cb83c640 1 -- 172.21.15.120:0/3081485338 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa4b00a3240 msgr2=0x7fa4b00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:18.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.966+0000 7fa4cb83c640 1 --2- 172.21.15.120:0/3081485338 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa4b00a3240 0x7fa4b00a5700 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa4c00136d0 tx=0x7fa4c000af90 comp rx=0 tx=0).stop 2024-09-19T09:04:18.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.966+0000 7fa4cb83c640 1 -- 172.21.15.120:0/3081485338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4c41082e0 msgr2=0x7fa4c419ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:18.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.966+0000 7fa4cb83c640 1 --2- 172.21.15.120:0/3081485338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4c41082e0 0x7fa4c419ce40 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009ca0 tx=0x7fa4bc031460 comp rx=0 tx=0).stop 2024-09-19T09:04:18.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.966+0000 7fa4cb83c640 1 -- 172.21.15.120:0/3081485338 shutdown_connections 2024-09-19T09:04:18.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.966+0000 7fa4cb83c640 1 --2- 172.21.15.120:0/3081485338 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa4b00a3240 0x7fa4b00a5700 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.966+0000 7fa4cb83c640 1 --2- 172.21.15.120:0/3081485338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4c41082e0 0x7fa4c419ce40 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.966+0000 7fa4cb83c640 1 --2- 172.21.15.120:0/3081485338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4c4103760 0x7fa4c4198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.966+0000 7fa4cb83c640 1 --2- 172.21.15.120:0/3081485338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c4102db0 0x7fa4c4198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:18.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.966+0000 7fa4cb83c640 1 -- 172.21.15.120:0/3081485338 >> 172.21.15.120:0/3081485338 conn(0x7fa4c40fc7d0 msgr2=0x7fa4c40fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:18.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.966+0000 7fa4cb83c640 1 -- 172.21.15.120:0/3081485338 shutdown_connections 2024-09-19T09:04:18.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:18.966+0000 7fa4cb83c640 1 -- 172.21.15.120:0/3081485338 wait complete. 2024-09-19T09:04:19.034 INFO:tasks.workunit.client.0.smithi120.stdout: 1 11836 5.74 MB/sec warmup 14 sec latency 10499.113 ms 2024-09-19T09:04:19.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:18 smithi120 ceph-mon[31150]: pgmap v384: 97 pgs: 97 active+clean; 6.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 651 KiB/s wr, 148 op/s 2024-09-19T09:04:19.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:18 smithi196 ceph-mon[35150]: pgmap v384: 97 pgs: 97 active+clean; 6.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 651 KiB/s wr, 148 op/s 2024-09-19T09:04:19.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:18 smithi153 ceph-mon[35071]: pgmap v384: 97 pgs: 97 active+clean; 6.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 651 KiB/s wr, 148 op/s 2024-09-19T09:04:19.787 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:04:19.787 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:04:20.034 INFO:tasks.workunit.client.0.smithi120.stdout: 1 11836 5.36 MB/sec warmup 15 sec latency 11499.137 ms 2024-09-19T09:04:20.039 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:04:20.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.619+0000 7f1f2bc96640 1 -- 172.21.15.120:0/2214757511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f24106f10 msgr2=0x7f1f2410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:20.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.619+0000 7f1f2bc96640 1 --2- 172.21.15.120:0/2214757511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f24106f10 0x7f1f2410ea90 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c009ca0 tx=0x7f1f1c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:04:20.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.619+0000 7f1f2bc96640 1 -- 172.21.15.120:0/2214757511 shutdown_connections 2024-09-19T09:04:20.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.619+0000 7f1f2bc96640 1 --2- 172.21.15.120:0/2214757511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f24106f10 0x7f1f2410ea90 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:20.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.619+0000 7f1f2bc96640 1 --2- 172.21.15.120:0/2214757511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f24102790 0x7f1f241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:20.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.619+0000 7f1f2bc96640 1 --2- 172.21.15.120:0/2214757511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f24101de0 0x7f1f241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:20.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.619+0000 7f1f2bc96640 1 -- 172.21.15.120:0/2214757511 >> 172.21.15.120:0/2214757511 conn(0x7f1f240fd8c0 msgr2=0x7f1f240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:20.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.619+0000 7f1f2bc96640 1 -- 172.21.15.120:0/2214757511 shutdown_connections 2024-09-19T09:04:20.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.619+0000 7f1f2bc96640 1 -- 172.21.15.120:0/2214757511 wait complete. 2024-09-19T09:04:20.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.620+0000 7f1f2bc96640 1 Processor -- start 2024-09-19T09:04:20.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.620+0000 7f1f2bc96640 1 -- start start 2024-09-19T09:04:20.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.620+0000 7f1f2bc96640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f24101de0 0x7f1f241983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:20.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.620+0000 7f1f2bc96640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f24102790 0x7f1f24198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:20.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.620+0000 7f1f2bc96640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f24106f10 0x7f1f2419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:20.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.620+0000 7f1f2bc96640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1f24111f70 con 0x7f1f24101de0 2024-09-19T09:04:20.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.620+0000 7f1f2bc96640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1f24111df0 con 0x7f1f24106f10 2024-09-19T09:04:20.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.620+0000 7f1f2bc96640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1f241120f0 con 0x7f1f24102790 2024-09-19T09:04:20.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.620+0000 7f1f29a32640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f24101de0 0x7f1f241983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:20.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.620+0000 7f1f29a32640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f24101de0 0x7f1f241983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45170/0 (socket says 172.21.15.120:45170) 2024-09-19T09:04:20.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.620+0000 7f1f29a32640 1 -- 172.21.15.120:0/2420153137 learned_addr learned my addr 172.21.15.120:0/2420153137 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:20.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.620+0000 7f1f2a233640 1 --2- 172.21.15.120:0/2420153137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f24106f10 0x7f1f2419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:20.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.620+0000 7f1f29231640 1 --2- 172.21.15.120:0/2420153137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f24102790 0x7f1f24198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:20.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.620+0000 7f1f29a32640 1 -- 172.21.15.120:0/2420153137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f24102790 msgr2=0x7f1f24198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:20.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.620+0000 7f1f29a32640 1 --2- 172.21.15.120:0/2420153137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f24102790 0x7f1f24198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:20.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.620+0000 7f1f29a32640 1 -- 172.21.15.120:0/2420153137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f24106f10 msgr2=0x7f1f2419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:20.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.620+0000 7f1f29a32640 1 --2- 172.21.15.120:0/2420153137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f24106f10 0x7f1f2419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:20.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.620+0000 7f1f29a32640 1 -- 172.21.15.120:0/2420153137 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f2419d380 con 0x7f1f24101de0 2024-09-19T09:04:20.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.621+0000 7f1f29a32640 1 --2- 172.21.15.120:0/2420153137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f24101de0 0x7f1f241983d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000d770 tx=0x7f1f2000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:20.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.621+0000 7f1f1affd640 1 -- 172.21.15.120:0/2420153137 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f20017070 con 0x7f1f24101de0 2024-09-19T09:04:20.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.621+0000 7f1f1affd640 1 -- 172.21.15.120:0/2420153137 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f20005500 con 0x7f1f24101de0 2024-09-19T09:04:20.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.621+0000 7f1f2bc96640 1 -- 172.21.15.120:0/2420153137 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f2419d670 con 0x7f1f24101de0 2024-09-19T09:04:20.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.621+0000 7f1f1affd640 1 -- 172.21.15.120:0/2420153137 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f20004e00 con 0x7f1f24101de0 2024-09-19T09:04:20.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.621+0000 7f1f2bc96640 1 -- 172.21.15.120:0/2420153137 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f241a4f50 con 0x7f1f24101de0 2024-09-19T09:04:20.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.622+0000 7f1f2bc96640 1 -- 172.21.15.120:0/2420153137 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f241021c0 con 0x7f1f24101de0 2024-09-19T09:04:20.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.623+0000 7f1f1affd640 1 -- 172.21.15.120:0/2420153137 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f1f20004fa0 con 0x7f1f24101de0 2024-09-19T09:04:20.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.623+0000 7f1f1affd640 1 --2- 172.21.15.120:0/2420153137 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1f100a3240 0x7f1f100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:20.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.624+0000 7f1f29231640 1 --2- 172.21.15.120:0/2420153137 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1f100a3240 0x7f1f100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:20.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.624+0000 7f1f1affd640 1 -- 172.21.15.120:0/2420153137 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f1f200cf350 con 0x7f1f24101de0 2024-09-19T09:04:20.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.624+0000 7f1f29231640 1 --2- 172.21.15.120:0/2420153137 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1f100a3240 0x7f1f100a5700 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1f241088e0 tx=0x7f1f14005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:20.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.628+0000 7f1f1affd640 1 -- 172.21.15.120:0/2420153137 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1f2009c070 con 0x7f1f24101de0 2024-09-19T09:04:20.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:20 smithi120 ceph-mon[31150]: pgmap v385: 97 pgs: 97 active+clean; 6.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 533 KiB/s wr, 129 op/s 2024-09-19T09:04:20.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.832+0000 7f1f2bc96640 1 -- 172.21.15.120:0/2420153137 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f24199730 con 0x7f1f24101de0 2024-09-19T09:04:20.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.832+0000 7f1f1affd640 1 -- 172.21.15.120:0/2420153137 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f1f2009b970 con 0x7f1f24101de0 2024-09-19T09:04:20.833 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:04:20.833 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":28,"btime":"2024-09-19T09:02:41:696728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:02:41.207357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:04:20.833 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 28 2024-09-19T09:04:20.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.835+0000 7f1f2bc96640 1 -- 172.21.15.120:0/2420153137 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1f100a3240 msgr2=0x7f1f100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:20.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.835+0000 7f1f2bc96640 1 --2- 172.21.15.120:0/2420153137 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1f100a3240 0x7f1f100a5700 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1f241088e0 tx=0x7f1f14005f90 comp rx=0 tx=0).stop 2024-09-19T09:04:20.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.835+0000 7f1f2bc96640 1 -- 172.21.15.120:0/2420153137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f24101de0 msgr2=0x7f1f241983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:20.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.835+0000 7f1f2bc96640 1 --2- 172.21.15.120:0/2420153137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f24101de0 0x7f1f241983d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000d770 tx=0x7f1f2000dc30 comp rx=0 tx=0).stop 2024-09-19T09:04:20.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.835+0000 7f1f2bc96640 1 -- 172.21.15.120:0/2420153137 shutdown_connections 2024-09-19T09:04:20.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.835+0000 7f1f2bc96640 1 --2- 172.21.15.120:0/2420153137 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1f100a3240 0x7f1f100a5700 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:20.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.835+0000 7f1f2bc96640 1 --2- 172.21.15.120:0/2420153137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f24106f10 0x7f1f2419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:20.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.835+0000 7f1f2bc96640 1 --2- 172.21.15.120:0/2420153137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f24102790 0x7f1f24198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:20.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.835+0000 7f1f2bc96640 1 --2- 172.21.15.120:0/2420153137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f24101de0 0x7f1f241983d0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:20.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.835+0000 7f1f2bc96640 1 -- 172.21.15.120:0/2420153137 >> 172.21.15.120:0/2420153137 conn(0x7f1f240fd8c0 msgr2=0x7f1f240feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:20.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.835+0000 7f1f2bc96640 1 -- 172.21.15.120:0/2420153137 shutdown_connections 2024-09-19T09:04:20.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:20.835+0000 7f1f2bc96640 1 -- 172.21.15.120:0/2420153137 wait complete. 2024-09-19T09:04:21.035 INFO:tasks.workunit.client.0.smithi120.stdout: 1 11836 5.02 MB/sec warmup 16 sec latency 12499.218 ms 2024-09-19T09:04:21.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:20 smithi196 ceph-mon[35150]: pgmap v385: 97 pgs: 97 active+clean; 6.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 533 KiB/s wr, 129 op/s 2024-09-19T09:04:21.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:20 smithi153 ceph-mon[35071]: pgmap v385: 97 pgs: 97 active+clean; 6.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 533 KiB/s wr, 129 op/s 2024-09-19T09:04:21.555 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:04:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.665+0000 7f5837d9f640 1 -- 172.21.15.120:0/3553203189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58301003b0 msgr2=0x7f58301027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.665+0000 7f5837d9f640 1 --2- 172.21.15.120:0/3553203189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58301003b0 0x7f58301027c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f582c009900 tx=0x7f582c02a410 comp rx=0 tx=0).stop 2024-09-19T09:04:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.665+0000 7f5837d9f640 1 -- 172.21.15.120:0/3553203189 shutdown_connections 2024-09-19T09:04:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.665+0000 7f5837d9f640 1 --2- 172.21.15.120:0/3553203189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58301056b0 0x7f5830105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.665+0000 7f5837d9f640 1 --2- 172.21.15.120:0/3553203189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5830102d00 0x7f5830105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.665+0000 7f5837d9f640 1 --2- 172.21.15.120:0/3553203189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58301003b0 0x7f58301027c0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.665+0000 7f5837d9f640 1 -- 172.21.15.120:0/3553203189 >> 172.21.15.120:0/3553203189 conn(0x7f58300fc1e0 msgr2=0x7f58300fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.665+0000 7f5837d9f640 1 -- 172.21.15.120:0/3553203189 shutdown_connections 2024-09-19T09:04:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.665+0000 7f5837d9f640 1 -- 172.21.15.120:0/3553203189 wait complete. 2024-09-19T09:04:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.665+0000 7f5837d9f640 1 Processor -- start 2024-09-19T09:04:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.665+0000 7f5837d9f640 1 -- start start 2024-09-19T09:04:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.665+0000 7f5837d9f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58301003b0 0x7f583019e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.665+0000 7f5837d9f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5830102d00 0x7f583019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.666+0000 7f5837d9f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58301056b0 0x7f58301a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.666+0000 7f5837d9f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5830113dc0 con 0x7f58301056b0 2024-09-19T09:04:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.666+0000 7f5837d9f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5830113c40 con 0x7f5830102d00 2024-09-19T09:04:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.666+0000 7f5837d9f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5830113f40 con 0x7f58301003b0 2024-09-19T09:04:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.666+0000 7f583633c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58301056b0 0x7f58301a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.666+0000 7f583633c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58301056b0 0x7f58301a58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33654/0 (socket says 172.21.15.120:33654) 2024-09-19T09:04:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.666+0000 7f583633c640 1 -- 172.21.15.120:0/2490065895 learned_addr learned my addr 172.21.15.120:0/2490065895 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.666+0000 7f5835b3b640 1 --2- 172.21.15.120:0/2490065895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58301003b0 0x7f583019e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.666+0000 7f583533a640 1 --2- 172.21.15.120:0/2490065895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5830102d00 0x7f583019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.666+0000 7f583633c640 1 -- 172.21.15.120:0/2490065895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58301003b0 msgr2=0x7f583019e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.666+0000 7f583633c640 1 --2- 172.21.15.120:0/2490065895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58301003b0 0x7f583019e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.666+0000 7f583633c640 1 -- 172.21.15.120:0/2490065895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5830102d00 msgr2=0x7f583019e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.666+0000 7f583633c640 1 --2- 172.21.15.120:0/2490065895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5830102d00 0x7f583019e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.666+0000 7f583633c640 1 -- 172.21.15.120:0/2490065895 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58301a5f20 con 0x7f58301056b0 2024-09-19T09:04:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.666+0000 7f583633c640 1 --2- 172.21.15.120:0/2490065895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58301056b0 0x7f58301a58b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f582800cce0 tx=0x7f582800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.666+0000 7f5826ffd640 1 -- 172.21.15.120:0/2490065895 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582800f450 con 0x7f58301056b0 2024-09-19T09:04:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.666+0000 7f5837d9f640 1 -- 172.21.15.120:0/2490065895 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58301a6210 con 0x7f58301056b0 2024-09-19T09:04:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.666+0000 7f5826ffd640 1 -- 172.21.15.120:0/2490065895 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f582800fad0 con 0x7f58301056b0 2024-09-19T09:04:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.666+0000 7f5826ffd640 1 -- 172.21.15.120:0/2490065895 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5828018b60 con 0x7f58301056b0 2024-09-19T09:04:21.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.667+0000 7f5837d9f640 1 -- 172.21.15.120:0/2490065895 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58301a6760 con 0x7f58301056b0 2024-09-19T09:04:21.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.669+0000 7f583ac15740 1 Processor -- start 2024-09-19T09:04:21.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.669+0000 7f5826ffd640 1 -- 172.21.15.120:0/2490065895 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f582800fc80 con 0x7f58301056b0 2024-09-19T09:04:21.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.669+0000 7f583ac15740 1 -- start start 2024-09-19T09:04:21.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.669+0000 7f583ac15740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555af8ce49a0 0x555af8ce4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:21.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.670+0000 7f583ac15740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555af8ce52e0 0x555af8ce9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:21.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.670+0000 7f583ac15740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555af8cea370 0x555af8cec830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:21.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.670+0000 7f583ac15740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555af8c482f0 con 0x555af8cea370 2024-09-19T09:04:21.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.670+0000 7f583ac15740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555af8bf0270 con 0x555af8ce52e0 2024-09-19T09:04:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.670+0000 7f583ac15740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555af8c442b0 con 0x555af8ce49a0 2024-09-19T09:04:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.670+0000 7f583533a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555af8cea370 0x555af8cec830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.670+0000 7f583533a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555af8cea370 0x555af8cec830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33660/0 (socket says 172.21.15.120:33660) 2024-09-19T09:04:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.670+0000 7f583533a640 1 -- 172.21.15.120:0/1409794000 learned_addr learned my addr 172.21.15.120:0/1409794000 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.670+0000 7f5835b3b640 1 --2- 172.21.15.120:0/1409794000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555af8ce52e0 0x555af8ce9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.670+0000 7f583533a640 1 --2- 172.21.15.120:0/1409794000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555af8ce49a0 0x555af8ce4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.670+0000 7f5826ffd640 1 --2- 172.21.15.120:0/2490065895 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f581c0a32f0 0x7f581c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.670+0000 7f583533a640 1 -- 172.21.15.120:0/1409794000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555af8ce49a0 msgr2=0x555af8ce4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.670+0000 7f583533a640 1 --2- 172.21.15.120:0/1409794000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555af8ce49a0 0x555af8ce4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.670+0000 7f583533a640 1 -- 172.21.15.120:0/1409794000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555af8ce52e0 msgr2=0x555af8ce9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.670+0000 7f583633c640 1 --2- 172.21.15.120:0/2490065895 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f581c0a32f0 0x7f581c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.670+0000 7f583533a640 1 --2- 172.21.15.120:0/1409794000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555af8ce52e0 0x555af8ce9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.670+0000 7f583533a640 1 -- 172.21.15.120:0/1409794000 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555af8bef340 con 0x555af8cea370 2024-09-19T09:04:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.670+0000 7f5835b3b640 1 --2- 172.21.15.120:0/1409794000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555af8ce52e0 0x555af8ce9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:04:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.670+0000 7f5826ffd640 1 -- 172.21.15.120:0/2490065895 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f58280cfdc0 con 0x7f58301056b0 2024-09-19T09:04:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.670+0000 7f583533a640 1 --2- 172.21.15.120:0/1409794000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555af8cea370 0x555af8cec830 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f58200117e0 tx=0x7f582000a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785a2452eaa6bcb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.671+0000 7f583633c640 1 --2- 172.21.15.120:0/2490065895 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f581c0a32f0 0x7f581c0a57b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f582800ccb0 tx=0x7f5828098040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.671+0000 7f581bfff640 1 -- 172.21.15.120:0/1409794000 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5820011e20 con 0x555af8cea370 2024-09-19T09:04:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.671+0000 7f581bfff640 1 -- 172.21.15.120:0/1409794000 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5820005020 con 0x555af8cea370 2024-09-19T09:04:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.671+0000 7f581bfff640 1 -- 172.21.15.120:0/1409794000 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5820004b20 con 0x555af8cea370 2024-09-19T09:04:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.671+0000 7f583ac15740 1 -- 172.21.15.120:0/1409794000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555af8cea370 msgr2=0x555af8cec830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.671+0000 7f583ac15740 1 --2- 172.21.15.120:0/1409794000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555af8cea370 0x555af8cec830 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f58200117e0 tx=0x7f582000a760 comp rx=0 tx=0).stop 2024-09-19T09:04:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.671+0000 7f583ac15740 1 -- 172.21.15.120:0/1409794000 shutdown_connections 2024-09-19T09:04:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.671+0000 7f583ac15740 1 --2- 172.21.15.120:0/1409794000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555af8cea370 0x555af8cec830 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.671+0000 7f583ac15740 1 --2- 172.21.15.120:0/1409794000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555af8ce52e0 0x555af8ce9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.671+0000 7f583ac15740 1 --2- 172.21.15.120:0/1409794000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555af8ce49a0 0x555af8ce4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.671+0000 7f583ac15740 1 -- 172.21.15.120:0/1409794000 >> 172.21.15.120:0/1409794000 conn(0x555af8ce0010 msgr2=0x555af8ce0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.672+0000 7f583ac15740 1 -- 172.21.15.120:0/1409794000 shutdown_connections 2024-09-19T09:04:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.672+0000 7f583ac15740 1 -- 172.21.15.120:0/1409794000 wait complete. 2024-09-19T09:04:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.672+0000 7f583ac15740 1 -- start start 2024-09-19T09:04:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.672+0000 7f583ac15740 1 Processor -- start 2024-09-19T09:04:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.672+0000 7f583ac15740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555af8ce49a0 0x555af8ceec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.672+0000 7f583ac15740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555af8ce52e0 0x555af8cef1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.672+0000 7f583ac15740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555af8cea370 0x555af8cf6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.672+0000 7f583ac15740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555af8ced340 con 0x555af8ce52e0 2024-09-19T09:04:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.672+0000 7f583ac15740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555af8bd1290 con 0x555af8ce49a0 2024-09-19T09:04:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.672+0000 7f583ac15740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555af8ced4c0 con 0x555af8cea370 2024-09-19T09:04:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.672+0000 7f583ac15740 10 client.0 _ll_register_callbacks cb 0x555af8c02140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.672+0000 7f5835b3b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555af8ce52e0 0x555af8cef1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.672+0000 7f5835b3b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555af8ce52e0 0x555af8cef1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33662/0 (socket says 172.21.15.120:33662) 2024-09-19T09:04:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.672+0000 7f5835b3b640 1 -- 172.21.15.120:0/187720204 learned_addr learned my addr 172.21.15.120:0/187720204 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.672+0000 7f583533a640 1 --2- 172.21.15.120:0/187720204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555af8ce49a0 0x555af8ceec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.673+0000 7f583533a640 1 --2- 172.21.15.120:0/187720204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555af8cea370 0x555af8cf6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.673+0000 7f5835b3b640 1 -- 172.21.15.120:0/187720204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555af8cea370 msgr2=0x555af8cf6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.673+0000 7f5835b3b640 1 --2- 172.21.15.120:0/187720204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555af8cea370 0x555af8cf6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.673+0000 7f5835b3b640 1 -- 172.21.15.120:0/187720204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555af8ce49a0 msgr2=0x555af8ceec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.673+0000 7f5835b3b640 1 --2- 172.21.15.120:0/187720204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555af8ce49a0 0x555af8ceec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.673+0000 7f5835b3b640 1 -- 172.21.15.120:0/187720204 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555af895b210 con 0x555af8ce52e0 2024-09-19T09:04:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.673+0000 7f583533a640 1 --2- 172.21.15.120:0/187720204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555af8cea370 0x555af8cf6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:04:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.673+0000 7f5835b3b640 1 --2- 172.21.15.120:0/187720204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555af8ce52e0 0x555af8cef1c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f582c02a930 tx=0x7f582c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09d7f6965bd3779 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.673+0000 7f58197fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:04:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.673+0000 7f58197fa640 1 -- 172.21.15.120:0/187720204 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c035040 con 0x555af8ce52e0 2024-09-19T09:04:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.673+0000 7f58197fa640 1 -- 172.21.15.120:0/187720204 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f582c005020 con 0x555af8ce52e0 2024-09-19T09:04:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.673+0000 7f583ac15740 1 -- 172.21.15.120:0/187720204 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5820005020 con 0x555af8ce52e0 2024-09-19T09:04:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.673+0000 7f58197fa640 1 -- 172.21.15.120:0/187720204 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f582c0319e0 con 0x555af8ce52e0 2024-09-19T09:04:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.673+0000 7f583ac15740 10 client.15804 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.673+0000 7f583ac15740 1 -- 172.21.15.120:0/187720204 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555af8a27620 con 0x555af8ce52e0 2024-09-19T09:04:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.673+0000 7f58197fa640 1 -- 172.21.15.120:0/187720204 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5820005020 con 0x555af8ce52e0 2024-09-19T09:04:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.674+0000 7f58197fa640 1 -- 172.21.15.120:0/187720204 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f582c0035f0 con 0x555af8ce52e0 2024-09-19T09:04:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.674+0000 7f583ac15740 10 client.15804 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.674+0000 7f583ac15740 10 client.15804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:04:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.674+0000 7f583ac15740 20 client.15804 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.674+0000 7f583ac15740 1 --2- 172.21.15.120:0/187720204 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555af8cfa8f0 0x555af8cfcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.674+0000 7f583ac15740 4 client.15804 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:04:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.674+0000 7f583ac15740 1 -- 172.21.15.120:0/187720204 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555af8c55a20 con 0x555af8cfa8f0 2024-09-19T09:04:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.674+0000 7f583533a640 1 --2- 172.21.15.120:0/187720204 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555af8cfa8f0 0x555af8cfcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.674+0000 7f583533a640 1 --2- 172.21.15.120:0/187720204 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555af8cfa8f0 0x555af8cfcce0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fe6796828bff415 server_cookie=99f7c54a4b969083 in_seq=0 out_seq=0 2024-09-19T09:04:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.674+0000 7f58197fa640 10 client.15804 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.675+0000 7f58197fa640 1 -- 172.21.15.120:0/187720204 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555af8c55a20 con 0x555af8cfa8f0 2024-09-19T09:04:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.675+0000 7f58197fa640 10 client.15804 handle_command_reply: tid=0 2024-09-19T09:04:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.675+0000 7f583ac15740 1 client.15804 shutdown 2024-09-19T09:04:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.675+0000 7f583ac15740 1 -- 172.21.15.120:0/187720204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555af8ce52e0 msgr2=0x555af8cef1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.675+0000 7f583ac15740 1 --2- 172.21.15.120:0/187720204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555af8ce52e0 0x555af8cef1c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f582c02a930 tx=0x7f582c004270 comp rx=0 tx=0).stop 2024-09-19T09:04:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.675+0000 7f583ac15740 1 -- 172.21.15.120:0/187720204 shutdown_connections 2024-09-19T09:04:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.675+0000 7f583ac15740 1 --2- 172.21.15.120:0/187720204 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555af8cfa8f0 0x555af8cfcce0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.675+0000 7f583ac15740 1 --2- 172.21.15.120:0/187720204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555af8cea370 0x555af8cf6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.675+0000 7f583ac15740 1 --2- 172.21.15.120:0/187720204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555af8ce52e0 0x555af8cef1c0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.675+0000 7f58197fa640 0 client.15804 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.675+0000 7f583ac15740 1 --2- 172.21.15.120:0/187720204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555af8ce49a0 0x555af8ceec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.675+0000 7f583ac15740 1 -- 172.21.15.120:0/187720204 >> 172.21.15.120:0/187720204 conn(0x555af8ce0010 msgr2=0x555af8ce1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.675+0000 7f583ac15740 1 -- 172.21.15.120:0/187720204 shutdown_connections 2024-09-19T09:04:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.675+0000 7f583ac15740 1 -- 172.21.15.120:0/187720204 wait complete. 2024-09-19T09:04:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.675+0000 7f583ac15740 20 client.15804 trim_cache size 0 max 16384 2024-09-19T09:04:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.689+0000 7f583ac15740 1 Processor -- start 2024-09-19T09:04:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.690+0000 7f583ac15740 1 -- start start 2024-09-19T09:04:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.690+0000 7f583ac15740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555af8ce49a0 0x555af8cdf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.690+0000 7f583ac15740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555af8ce52e0 0x555af8cf61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.690+0000 7f583ac15740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555af8cea370 0x555af8ce1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.690+0000 7f583ac15740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f58301af470 con 0x555af8cea370 2024-09-19T09:04:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.690+0000 7f583ac15740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f58301b0fb0 con 0x555af8ce49a0 2024-09-19T09:04:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.690+0000 7f583ac15740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f58301afb40 con 0x555af8ce52e0 2024-09-19T09:04:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.690+0000 7f583533a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555af8cea370 0x555af8ce1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.690+0000 7f583533a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555af8cea370 0x555af8ce1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33674/0 (socket says 172.21.15.120:33674) 2024-09-19T09:04:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.690+0000 7f583533a640 1 -- 172.21.15.120:0/1486714597 learned_addr learned my addr 172.21.15.120:0/1486714597 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.690+0000 7f583533a640 1 --2- 172.21.15.120:0/1486714597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555af8ce49a0 0x555af8cdf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.690+0000 7f5835b3b640 1 --2- 172.21.15.120:0/1486714597 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555af8ce52e0 0x555af8cf61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.690+0000 7f583533a640 1 -- 172.21.15.120:0/1486714597 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555af8ce52e0 msgr2=0x555af8cf61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.690+0000 7f583533a640 1 --2- 172.21.15.120:0/1486714597 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555af8ce52e0 0x555af8cf61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.690+0000 7f583533a640 1 -- 172.21.15.120:0/1486714597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555af8ce49a0 msgr2=0x555af8cdf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.690+0000 7f583533a640 1 --2- 172.21.15.120:0/1486714597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555af8ce49a0 0x555af8cdf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.690+0000 7f583533a640 1 -- 172.21.15.120:0/1486714597 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555af8cfd360 con 0x555af8cea370 2024-09-19T09:04:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.690+0000 7f583533a640 1 --2- 172.21.15.120:0/1486714597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555af8cea370 0x555af8ce1400 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5820010010 tx=0x7f582001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae463e685262e24 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.690+0000 7f581b7fe640 1 -- 172.21.15.120:0/1486714597 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582000fb30 con 0x555af8cea370 2024-09-19T09:04:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.690+0000 7f581b7fe640 1 -- 172.21.15.120:0/1486714597 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f582000fcd0 con 0x555af8cea370 2024-09-19T09:04:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.690+0000 7f581b7fe640 1 -- 172.21.15.120:0/1486714597 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582002dca0 con 0x555af8cea370 2024-09-19T09:04:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.691+0000 7f583ac15740 1 -- 172.21.15.120:0/1486714597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555af8cea370 msgr2=0x555af8ce1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.691+0000 7f583ac15740 1 --2- 172.21.15.120:0/1486714597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555af8cea370 0x555af8ce1400 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5820010010 tx=0x7f582001a040 comp rx=0 tx=0).stop 2024-09-19T09:04:21.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.691+0000 7f583ac15740 1 -- 172.21.15.120:0/1486714597 shutdown_connections 2024-09-19T09:04:21.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.691+0000 7f583ac15740 1 --2- 172.21.15.120:0/1486714597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555af8cea370 0x555af8ce1400 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.691+0000 7f583ac15740 1 --2- 172.21.15.120:0/1486714597 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555af8ce52e0 0x555af8cf61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.691+0000 7f583ac15740 1 --2- 172.21.15.120:0/1486714597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555af8ce49a0 0x555af8cdf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.691+0000 7f583ac15740 1 -- 172.21.15.120:0/1486714597 >> 172.21.15.120:0/1486714597 conn(0x555af8ce0010 msgr2=0x555af8cde9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:21.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.691+0000 7f583ac15740 1 -- 172.21.15.120:0/1486714597 shutdown_connections 2024-09-19T09:04:21.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.691+0000 7f583ac15740 1 -- 172.21.15.120:0/1486714597 wait complete. 2024-09-19T09:04:21.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.692+0000 7f583ac15740 1 -- start start 2024-09-19T09:04:21.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.692+0000 7f583ac15740 1 Processor -- start 2024-09-19T09:04:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.692+0000 7f583ac15740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555af8ce49a0 0x555af8cf3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.692+0000 7f583ac15740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555af8ce52e0 0x555af8cf3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.692+0000 7f583ac15740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555af8cea370 0x555af8d635b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.692+0000 7f583ac15740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f58301b08e0 con 0x555af8ce49a0 2024-09-19T09:04:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.692+0000 7f583ac15740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f58301b0210 con 0x555af8ce52e0 2024-09-19T09:04:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.692+0000 7f583ac15740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f58301ad430 con 0x555af8cea370 2024-09-19T09:04:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.692+0000 7f583ac15740 10 client.0 _ll_register_callbacks cb 0x7f582c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.692+0000 7f583533a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555af8ce49a0 0x555af8cf3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.692+0000 7f583533a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555af8ce49a0 0x555af8cf3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33676/0 (socket says 172.21.15.120:33676) 2024-09-19T09:04:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.692+0000 7f583533a640 1 -- 172.21.15.120:0/141164353 learned_addr learned my addr 172.21.15.120:0/141164353 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.692+0000 7f5835b3b640 1 --2- 172.21.15.120:0/141164353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555af8ce52e0 0x555af8cf3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.692+0000 7f583533a640 1 -- 172.21.15.120:0/141164353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555af8cea370 msgr2=0x555af8d635b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.692+0000 7f583533a640 1 --2- 172.21.15.120:0/141164353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555af8cea370 0x555af8d635b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.692+0000 7f583533a640 1 -- 172.21.15.120:0/141164353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555af8ce52e0 msgr2=0x555af8cf3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.692+0000 7f583533a640 1 --2- 172.21.15.120:0/141164353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555af8ce52e0 0x555af8cf3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.692+0000 7f583533a640 1 -- 172.21.15.120:0/141164353 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555af8d63af0 con 0x555af8ce49a0 2024-09-19T09:04:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.692+0000 7f5835b3b640 1 --2- 172.21.15.120:0/141164353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555af8ce52e0 0x555af8cf3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:04:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.693+0000 7f583533a640 1 --2- 172.21.15.120:0/141164353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555af8ce49a0 0x555af8cf3020 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f582000fc90 tx=0x7f582000ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ba56bcf0418582 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.693+0000 7f58197fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:04:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.693+0000 7f58197fa640 1 -- 172.21.15.120:0/141164353 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582003b070 con 0x555af8ce49a0 2024-09-19T09:04:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.693+0000 7f58197fa640 1 -- 172.21.15.120:0/141164353 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f5820011040 con 0x555af8ce49a0 2024-09-19T09:04:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.693+0000 7f583ac15740 1 -- 172.21.15.120:0/141164353 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f582000fcd0 con 0x555af8ce49a0 2024-09-19T09:04:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.693+0000 7f58197fa640 1 -- 172.21.15.120:0/141164353 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5820011240 con 0x555af8ce49a0 2024-09-19T09:04:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.693+0000 7f583ac15740 10 client.15810 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.693+0000 7f583ac15740 1 -- 172.21.15.120:0/141164353 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555af8d64d10 con 0x555af8ce49a0 2024-09-19T09:04:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.693+0000 7f58197fa640 1 -- 172.21.15.120:0/141164353 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f582000fcd0 con 0x555af8ce49a0 2024-09-19T09:04:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.693+0000 7f58197fa640 1 -- 172.21.15.120:0/141164353 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f582001ea00 con 0x555af8ce49a0 2024-09-19T09:04:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.693+0000 7f583ac15740 10 client.15810 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.693+0000 7f583ac15740 10 client.15810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:04:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.693+0000 7f583ac15740 20 client.15810 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.693+0000 7f583ac15740 1 --2- 172.21.15.120:0/141164353 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555af8d653d0 0x555af8d677f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.693+0000 7f583ac15740 4 client.15810 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:04:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.693+0000 7f583ac15740 1 -- 172.21.15.120:0/141164353 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555af8c4fd70 con 0x555af8d653d0 2024-09-19T09:04:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.694+0000 7f5835b3b640 1 --2- 172.21.15.120:0/141164353 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555af8d653d0 0x555af8d677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.694+0000 7f58197fa640 10 client.15810 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.694+0000 7f5835b3b640 1 --2- 172.21.15.120:0/141164353 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555af8d653d0 0x555af8d677f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cbe3d1b8845185c server_cookie=5bd2620132f87db8 in_seq=0 out_seq=0 2024-09-19T09:04:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.749+0000 7f58197fa640 1 -- 172.21.15.120:0/141164353 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555af8c4fd70 con 0x555af8d653d0 2024-09-19T09:04:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.749+0000 7f58197fa640 10 client.15810 handle_command_reply: tid=0 2024-09-19T09:04:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.749+0000 7f583ac15740 1 client.15810 shutdown 2024-09-19T09:04:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.749+0000 7f583ac15740 1 -- 172.21.15.120:0/141164353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555af8ce49a0 msgr2=0x555af8cf3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.749+0000 7f583ac15740 1 --2- 172.21.15.120:0/141164353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555af8ce49a0 0x555af8cf3020 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f582000fc90 tx=0x7f582000ff90 comp rx=0 tx=0).stop 2024-09-19T09:04:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.749+0000 7f583ac15740 1 -- 172.21.15.120:0/141164353 shutdown_connections 2024-09-19T09:04:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.750+0000 7f583ac15740 1 --2- 172.21.15.120:0/141164353 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555af8d653d0 0x555af8d677f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.750+0000 7f583ac15740 1 --2- 172.21.15.120:0/141164353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555af8cea370 0x555af8d635b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.750+0000 7f58197fa640 0 client.15810 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.750+0000 7f583ac15740 1 --2- 172.21.15.120:0/141164353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555af8ce52e0 0x555af8cf3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.750+0000 7f583ac15740 1 --2- 172.21.15.120:0/141164353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555af8ce49a0 0x555af8cf3020 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.750+0000 7f583ac15740 1 -- 172.21.15.120:0/141164353 >> 172.21.15.120:0/141164353 conn(0x555af8ce0010 msgr2=0x555af8cdf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.750+0000 7f583ac15740 1 -- 172.21.15.120:0/141164353 shutdown_connections 2024-09-19T09:04:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.750+0000 7f583ac15740 1 -- 172.21.15.120:0/141164353 wait complete. 2024-09-19T09:04:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.750+0000 7f583ac15740 20 client.15810 trim_cache size 0 max 16384 2024-09-19T09:04:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:04:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:04:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:04:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:04:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.750+0000 7f5837d9f640 1 -- 172.21.15.120:0/2490065895 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f581c0a32f0 msgr2=0x7f581c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.750+0000 7f5837d9f640 1 --2- 172.21.15.120:0/2490065895 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f581c0a32f0 0x7f581c0a57b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f582800ccb0 tx=0x7f5828098040 comp rx=0 tx=0).stop 2024-09-19T09:04:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.750+0000 7f5837d9f640 1 -- 172.21.15.120:0/2490065895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58301056b0 msgr2=0x7f58301a58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.750+0000 7f5837d9f640 1 --2- 172.21.15.120:0/2490065895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58301056b0 0x7f58301a58b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f582800cce0 tx=0x7f582800a620 comp rx=0 tx=0).stop 2024-09-19T09:04:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.750+0000 7f5837d9f640 1 -- 172.21.15.120:0/2490065895 shutdown_connections 2024-09-19T09:04:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.750+0000 7f5837d9f640 1 --2- 172.21.15.120:0/2490065895 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f581c0a32f0 0x7f581c0a57b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.750+0000 7f5837d9f640 1 --2- 172.21.15.120:0/2490065895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58301056b0 0x7f58301a58b0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.750+0000 7f5837d9f640 1 --2- 172.21.15.120:0/2490065895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5830102d00 0x7f583019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.750+0000 7f5837d9f640 1 --2- 172.21.15.120:0/2490065895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58301003b0 0x7f583019e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.750+0000 7f5837d9f640 1 -- 172.21.15.120:0/2490065895 >> 172.21.15.120:0/2490065895 conn(0x7f58300fc1e0 msgr2=0x7f5830103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.750+0000 7f5837d9f640 1 -- 172.21.15.120:0/2490065895 shutdown_connections 2024-09-19T09:04:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:21.750+0000 7f5837d9f640 1 -- 172.21.15.120:0/2490065895 wait complete. 2024-09-19T09:04:21.813 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:04:22.035 INFO:tasks.workunit.client.0.smithi120.stdout: 1 11836 4.73 MB/sec warmup 17 sec latency 13499.267 ms 2024-09-19T09:04:22.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:21 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2420153137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:04:22.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:21 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2420153137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:04:22.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:21 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2420153137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:04:22.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.629+0000 7f7b20d32640 1 -- 172.21.15.120:0/1046698809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b1c101420 msgr2=0x7f7b1c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:22.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.629+0000 7f7b20d32640 1 --2- 172.21.15.120:0/1046698809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b1c101420 0x7f7b1c10d950 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7b14009cd0 tx=0x7f7b1402a8e0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.629+0000 7f7b20d32640 1 -- 172.21.15.120:0/1046698809 shutdown_connections 2024-09-19T09:04:22.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.629+0000 7f7b20d32640 1 --2- 172.21.15.120:0/1046698809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b1c101420 0x7f7b1c10d950 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.629+0000 7f7b20d32640 1 --2- 172.21.15.120:0/1046698809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b1c100a80 0x7f7b1c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.629+0000 7f7b20d32640 1 --2- 172.21.15.120:0/1046698809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b1c106a80 0x7f7b1c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.629+0000 7f7b20d32640 1 -- 172.21.15.120:0/1046698809 >> 172.21.15.120:0/1046698809 conn(0x7f7b1c0fc7a0 msgr2=0x7f7b1c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:22.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.629+0000 7f7b20d32640 1 -- 172.21.15.120:0/1046698809 shutdown_connections 2024-09-19T09:04:22.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.630+0000 7f7b20d32640 1 -- 172.21.15.120:0/1046698809 wait complete. 2024-09-19T09:04:22.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.630+0000 7f7b20d32640 1 Processor -- start 2024-09-19T09:04:22.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.630+0000 7f7b20d32640 1 -- start start 2024-09-19T09:04:22.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.630+0000 7f7b20d32640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b1c100a80 0x7f7b1c197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:22.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.630+0000 7f7b20d32640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b1c101420 0x7f7b1c197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:22.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.630+0000 7f7b20d32640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b1c106a80 0x7f7b1c19bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:22.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.630+0000 7f7b20d32640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c110e80 con 0x7f7b1c101420 2024-09-19T09:04:22.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.630+0000 7f7b20d32640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c110d00 con 0x7f7b1c100a80 2024-09-19T09:04:22.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.630+0000 7f7b20d32640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c111000 con 0x7f7b1c106a80 2024-09-19T09:04:22.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.631+0000 7f7b19d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b1c101420 0x7f7b1c197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:22.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.631+0000 7f7b19d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b1c101420 0x7f7b1c197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33690/0 (socket says 172.21.15.120:33690) 2024-09-19T09:04:22.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.631+0000 7f7b19d9b640 1 -- 172.21.15.120:0/591064937 learned_addr learned my addr 172.21.15.120:0/591064937 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:22.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.631+0000 7f7b1a59c640 1 --2- 172.21.15.120:0/591064937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b1c100a80 0x7f7b1c197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:22.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.631+0000 7f7b1ad9d640 1 --2- 172.21.15.120:0/591064937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b1c106a80 0x7f7b1c19bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:22.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.631+0000 7f7b19d9b640 1 -- 172.21.15.120:0/591064937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b1c106a80 msgr2=0x7f7b1c19bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:22.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.631+0000 7f7b19d9b640 1 --2- 172.21.15.120:0/591064937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b1c106a80 0x7f7b1c19bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.631+0000 7f7b19d9b640 1 -- 172.21.15.120:0/591064937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b1c100a80 msgr2=0x7f7b1c197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:22.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.631+0000 7f7b19d9b640 1 --2- 172.21.15.120:0/591064937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b1c100a80 0x7f7b1c197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.631+0000 7f7b19d9b640 1 -- 172.21.15.120:0/591064937 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b1c19c2d0 con 0x7f7b1c101420 2024-09-19T09:04:22.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.631+0000 7f7b19d9b640 1 --2- 172.21.15.120:0/591064937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b1c101420 0x7f7b1c197860 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7b1000d950 tx=0x7f7b1000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:22.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.631+0000 7f7b0b7fe640 1 -- 172.21.15.120:0/591064937 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b10017030 con 0x7f7b1c101420 2024-09-19T09:04:22.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.631+0000 7f7b0b7fe640 1 -- 172.21.15.120:0/591064937 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b10004020 con 0x7f7b1c101420 2024-09-19T09:04:22.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.631+0000 7f7b20d32640 1 -- 172.21.15.120:0/591064937 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b1c19c5c0 con 0x7f7b1c101420 2024-09-19T09:04:22.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.631+0000 7f7b0b7fe640 1 -- 172.21.15.120:0/591064937 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b100047a0 con 0x7f7b1c101420 2024-09-19T09:04:22.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.631+0000 7f7b20d32640 1 -- 172.21.15.120:0/591064937 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b1c1a3ea0 con 0x7f7b1c101420 2024-09-19T09:04:22.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.634+0000 7f7b0b7fe640 1 -- 172.21.15.120:0/591064937 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f7b100041d0 con 0x7f7b1c101420 2024-09-19T09:04:22.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.634+0000 7f7b2310b740 1 Processor -- start 2024-09-19T09:04:22.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.634+0000 7f7b2310b740 1 -- start start 2024-09-19T09:04:22.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.634+0000 7f7b2310b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7aceb8d50 0x55c7aceb72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:22.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.634+0000 7f7b0b7fe640 1 --2- 172.21.15.120:0/591064937 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7b040a3240 0x7f7b040a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:22.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.634+0000 7f7b2310b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7aceb77e0 0x55c7aceb7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:22.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.635+0000 7f7b2310b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7acec2580 0x55c7acec4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:22.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.635+0000 7f7b2310b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c7ace1b2f0 con 0x55c7aceb8d50 2024-09-19T09:04:22.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.635+0000 7f7b2310b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c7acdc2cf0 con 0x55c7acec2580 2024-09-19T09:04:22.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.635+0000 7f7b2310b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c7ace172b0 con 0x55c7aceb77e0 2024-09-19T09:04:22.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.635+0000 7f7b1a59c640 1 --2- 172.21.15.120:0/591064937 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7b040a3240 0x7f7b040a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:22.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.635+0000 7f7b1ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7aceb8d50 0x55c7aceb72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:22.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.635+0000 7f7b1ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7aceb8d50 0x55c7aceb72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33698/0 (socket says 172.21.15.120:33698) 2024-09-19T09:04:22.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.635+0000 7f7b1ad9d640 1 -- 172.21.15.120:0/1246710833 learned_addr learned my addr 172.21.15.120:0/1246710833 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:22.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.635+0000 7f7b0b7fe640 1 -- 172.21.15.120:0/591064937 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f7b100cf2f0 con 0x7f7b1c101420 2024-09-19T09:04:22.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.635+0000 7f7b1ad9d640 1 --2- 172.21.15.120:0/1246710833 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7acec2580 0x55c7acec4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:22.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.635+0000 7f7b19d9b640 1 --2- 172.21.15.120:0/1246710833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7aceb77e0 0x55c7aceb7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:22.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.635+0000 7f7b1ad9d640 1 -- 172.21.15.120:0/1246710833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7aceb77e0 msgr2=0x55c7aceb7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:22.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.635+0000 7f7b1ad9d640 1 --2- 172.21.15.120:0/1246710833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7aceb77e0 0x55c7aceb7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.635+0000 7f7b1ad9d640 1 -- 172.21.15.120:0/1246710833 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7acec2580 msgr2=0x55c7acec4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:22.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.635+0000 7f7b1ad9d640 1 --2- 172.21.15.120:0/1246710833 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7acec2580 0x55c7acec4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.635+0000 7f7b1ad9d640 1 -- 172.21.15.120:0/1246710833 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7acda73a0 con 0x55c7aceb8d50 2024-09-19T09:04:22.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.635+0000 7f7b1a59c640 1 --2- 172.21.15.120:0/591064937 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7b040a3240 0x7f7b040a5700 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c005fd0 tx=0x7f7b0c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:22.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.635+0000 7f7b1ad9d640 1 --2- 172.21.15.120:0/1246710833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7aceb8d50 0x55c7aceb72a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7b14004af0 tx=0x7f7b1402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ce56fe262cda34 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:22.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.635+0000 7f7b08ff9640 1 -- 172.21.15.120:0/1246710833 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b140318c0 con 0x55c7aceb8d50 2024-09-19T09:04:22.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.635+0000 7f7b08ff9640 1 -- 172.21.15.120:0/1246710833 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b14036040 con 0x55c7aceb8d50 2024-09-19T09:04:22.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.636+0000 7f7b08ff9640 1 -- 172.21.15.120:0/1246710833 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b14004390 con 0x55c7aceb8d50 2024-09-19T09:04:22.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.636+0000 7f7b2310b740 1 -- 172.21.15.120:0/1246710833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7aceb8d50 msgr2=0x55c7aceb72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:22.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.636+0000 7f7b2310b740 1 --2- 172.21.15.120:0/1246710833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7aceb8d50 0x55c7aceb72a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7b14004af0 tx=0x7f7b1402a8e0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.636+0000 7f7b2310b740 1 -- 172.21.15.120:0/1246710833 shutdown_connections 2024-09-19T09:04:22.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.636+0000 7f7b2310b740 1 --2- 172.21.15.120:0/1246710833 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7acec2580 0x55c7acec4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.636+0000 7f7b2310b740 1 --2- 172.21.15.120:0/1246710833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7aceb77e0 0x55c7aceb7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.636+0000 7f7b2310b740 1 --2- 172.21.15.120:0/1246710833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7aceb8d50 0x55c7aceb72a0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.636+0000 7f7b2310b740 1 -- 172.21.15.120:0/1246710833 >> 172.21.15.120:0/1246710833 conn(0x55c7ace7e310 msgr2=0x55c7aceb3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:22.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.636+0000 7f7b2310b740 1 -- 172.21.15.120:0/1246710833 shutdown_connections 2024-09-19T09:04:22.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.636+0000 7f7b2310b740 1 -- 172.21.15.120:0/1246710833 wait complete. 2024-09-19T09:04:22.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.637+0000 7f7b2310b740 1 -- start start 2024-09-19T09:04:22.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.637+0000 7f7b2310b740 1 Processor -- start 2024-09-19T09:04:22.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.637+0000 7f7b2310b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7aceb77e0 0x55c7acec1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:22.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.637+0000 7f7b2310b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7aceb8d50 0x55c7acebf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:22.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.637+0000 7f7b2310b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7acec2580 0x55c7acebf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:22.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.637+0000 7f7b2310b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c7acec5220 con 0x55c7aceb8d50 2024-09-19T09:04:22.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.637+0000 7f7b2310b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c7acdd6060 con 0x55c7acec2580 2024-09-19T09:04:22.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.637+0000 7f7b2310b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c7acec53a0 con 0x55c7aceb77e0 2024-09-19T09:04:22.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.637+0000 7f7b2310b740 10 client.0 _ll_register_callbacks cb 0x55c7acb298c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:22.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.637+0000 7f7b19d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7aceb8d50 0x55c7acebf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:22.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.637+0000 7f7b19d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7aceb8d50 0x55c7acebf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33712/0 (socket says 172.21.15.120:33712) 2024-09-19T09:04:22.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.637+0000 7f7b19d9b640 1 -- 172.21.15.120:0/3144601428 learned_addr learned my addr 172.21.15.120:0/3144601428 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:22.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.637+0000 7f7b19d9b640 1 -- 172.21.15.120:0/3144601428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7aceb77e0 msgr2=0x55c7acec1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:22.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.637+0000 7f7b19d9b640 1 --2- 172.21.15.120:0/3144601428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7aceb77e0 0x55c7acec1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.637+0000 7f7b19d9b640 1 -- 172.21.15.120:0/3144601428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7acec2580 msgr2=0x55c7acebf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:22.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.637+0000 7f7b19d9b640 1 --2- 172.21.15.120:0/3144601428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7acec2580 0x55c7acebf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.637+0000 7f7b19d9b640 1 -- 172.21.15.120:0/3144601428 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7acb8c3c0 con 0x55c7aceb8d50 2024-09-19T09:04:22.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.637+0000 7f7b19d9b640 1 --2- 172.21.15.120:0/3144601428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7aceb8d50 0x55c7acebf210 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7b100ccdf0 tx=0x7f7b1009c140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57702b34ac89fc9c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:22.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.637+0000 7f7b01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:04:22.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.638+0000 7f7b01ffb640 1 -- 172.21.15.120:0/3144601428 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1009b910 con 0x55c7aceb8d50 2024-09-19T09:04:22.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.638+0000 7f7b01ffb640 1 -- 172.21.15.120:0/3144601428 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f7b100ab2d0 con 0x55c7aceb8d50 2024-09-19T09:04:22.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.638+0000 7f7b2310b740 1 -- 172.21.15.120:0/3144601428 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b14036040 con 0x55c7aceb8d50 2024-09-19T09:04:22.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.638+0000 7f7b01ffb640 1 -- 172.21.15.120:0/3144601428 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7b1009fcc0 con 0x55c7aceb8d50 2024-09-19T09:04:22.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.638+0000 7f7b2310b740 10 client.15822 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:22.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.638+0000 7f7b2310b740 1 -- 172.21.15.120:0/3144601428 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c7aced0d00 con 0x55c7aceb8d50 2024-09-19T09:04:22.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.638+0000 7f7b01ffb640 1 -- 172.21.15.120:0/3144601428 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b14036040 con 0x55c7aceb8d50 2024-09-19T09:04:22.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.638+0000 7f7b01ffb640 1 -- 172.21.15.120:0/3144601428 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7b1009adf0 con 0x55c7aceb8d50 2024-09-19T09:04:22.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.638+0000 7f7b2310b740 10 client.15822 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:22.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.638+0000 7f7b2310b740 10 client.15822 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:04:22.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.638+0000 7f7b2310b740 20 client.15822 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:22.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.638+0000 7f7b2310b740 1 --2- 172.21.15.120:0/3144601428 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55c7aced2160 0x55c7aced45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:22.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.638+0000 7f7b2310b740 4 client.15822 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:04:22.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.638+0000 7f7b2310b740 1 -- 172.21.15.120:0/3144601428 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7ace28860 con 0x55c7aced2160 2024-09-19T09:04:22.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.638+0000 7f7b1a59c640 1 --2- 172.21.15.120:0/3144601428 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55c7aced2160 0x55c7aced45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:22.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.639+0000 7f7b01ffb640 10 client.15822 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:04:22.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.639+0000 7f7b1a59c640 1 --2- 172.21.15.120:0/3144601428 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55c7aced2160 0x55c7aced45d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ed093499ba978625 server_cookie=d4d82a9ea44ed821 in_seq=0 out_seq=0 2024-09-19T09:04:22.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.640+0000 7f7b01ffb640 1 -- 172.21.15.120:0/3144601428 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c7ace28860 con 0x55c7aced2160 2024-09-19T09:04:22.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.640+0000 7f7b01ffb640 10 client.15822 handle_command_reply: tid=0 2024-09-19T09:04:22.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.640+0000 7f7b2310b740 1 client.15822 shutdown 2024-09-19T09:04:22.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.640+0000 7f7b2310b740 1 -- 172.21.15.120:0/3144601428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7aceb8d50 msgr2=0x55c7acebf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:22.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.640+0000 7f7b2310b740 1 --2- 172.21.15.120:0/3144601428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7aceb8d50 0x55c7acebf210 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7b100ccdf0 tx=0x7f7b1009c140 comp rx=0 tx=0).stop 2024-09-19T09:04:22.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.640+0000 7f7b2310b740 1 -- 172.21.15.120:0/3144601428 shutdown_connections 2024-09-19T09:04:22.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.640+0000 7f7b2310b740 1 --2- 172.21.15.120:0/3144601428 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55c7aced2160 0x55c7aced45d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.640+0000 7f7b2310b740 1 --2- 172.21.15.120:0/3144601428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7acec2580 0x55c7acebf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.640+0000 7f7b2310b740 1 --2- 172.21.15.120:0/3144601428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7aceb8d50 0x55c7acebf210 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.640+0000 7f7b2310b740 1 --2- 172.21.15.120:0/3144601428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7aceb77e0 0x55c7acec1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.640+0000 7f7b2310b740 1 -- 172.21.15.120:0/3144601428 >> 172.21.15.120:0/3144601428 conn(0x55c7ace7e310 msgr2=0x55c7aceb9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:22.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.640+0000 7f7b01ffb640 0 client.15822 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:04:22.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.640+0000 7f7b2310b740 1 -- 172.21.15.120:0/3144601428 shutdown_connections 2024-09-19T09:04:22.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.640+0000 7f7b2310b740 1 -- 172.21.15.120:0/3144601428 wait complete. 2024-09-19T09:04:22.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.640+0000 7f7b2310b740 20 client.15822 trim_cache size 0 max 16384 2024-09-19T09:04:22.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.653+0000 7f7b2310b740 1 Processor -- start 2024-09-19T09:04:22.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.653+0000 7f7b2310b740 1 -- start start 2024-09-19T09:04:22.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.653+0000 7f7b2310b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7aceb77e0 0x55c7aceba330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:22.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.653+0000 7f7b2310b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7aceb8d50 0x55c7aceba870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:22.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.653+0000 7f7b2310b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7acec2580 0x55c7aced9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:22.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.653+0000 7f7b2310b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c1ab450 con 0x55c7aceb8d50 2024-09-19T09:04:22.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.653+0000 7f7b2310b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c1acf90 con 0x55c7acec2580 2024-09-19T09:04:22.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.653+0000 7f7b2310b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c1abb20 con 0x55c7aceb77e0 2024-09-19T09:04:22.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.653+0000 7f7b19d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7aceb8d50 0x55c7aceba870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:22.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.653+0000 7f7b19d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7aceb8d50 0x55c7aceba870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33720/0 (socket says 172.21.15.120:33720) 2024-09-19T09:04:22.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.653+0000 7f7b19d9b640 1 -- 172.21.15.120:0/3485040315 learned_addr learned my addr 172.21.15.120:0/3485040315 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:22.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.653+0000 7f7b1ad9d640 1 --2- 172.21.15.120:0/3485040315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7acec2580 0x55c7aced9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:22.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.653+0000 7f7b1a59c640 1 --2- 172.21.15.120:0/3485040315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7aceb77e0 0x55c7aceba330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:22.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.653+0000 7f7b19d9b640 1 -- 172.21.15.120:0/3485040315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7aceb77e0 msgr2=0x55c7aceba330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:22.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.653+0000 7f7b19d9b640 1 --2- 172.21.15.120:0/3485040315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7aceb77e0 0x55c7aceba330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.653+0000 7f7b19d9b640 1 -- 172.21.15.120:0/3485040315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7acec2580 msgr2=0x55c7aced9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:22.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.653+0000 7f7b19d9b640 1 --2- 172.21.15.120:0/3485040315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7acec2580 0x55c7aced9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.653+0000 7f7b19d9b640 1 -- 172.21.15.120:0/3485040315 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7acebfb50 con 0x55c7aceb8d50 2024-09-19T09:04:22.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.653+0000 7f7b1ad9d640 1 --2- 172.21.15.120:0/3485040315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7acec2580 0x55c7aced9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:04:22.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.654+0000 7f7b19d9b640 1 --2- 172.21.15.120:0/3485040315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7aceb8d50 0x55c7aceba870 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7b1009b610 tx=0x7f7b10099e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeddd11c8e22c84c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:22.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.654+0000 7f7b03fff640 1 -- 172.21.15.120:0/3485040315 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1009b910 con 0x55c7aceb8d50 2024-09-19T09:04:22.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.654+0000 7f7b03fff640 1 -- 172.21.15.120:0/3485040315 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b100a4580 con 0x55c7aceb8d50 2024-09-19T09:04:22.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.654+0000 7f7b03fff640 1 -- 172.21.15.120:0/3485040315 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1009ad60 con 0x55c7aceb8d50 2024-09-19T09:04:22.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.654+0000 7f7b2310b740 1 -- 172.21.15.120:0/3485040315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7aceb8d50 msgr2=0x55c7aceba870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:22.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.654+0000 7f7b2310b740 1 --2- 172.21.15.120:0/3485040315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7aceb8d50 0x55c7aceba870 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7b1009b610 tx=0x7f7b10099e20 comp rx=0 tx=0).stop 2024-09-19T09:04:22.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.654+0000 7f7b2310b740 1 -- 172.21.15.120:0/3485040315 shutdown_connections 2024-09-19T09:04:22.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.654+0000 7f7b2310b740 1 --2- 172.21.15.120:0/3485040315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7acec2580 0x55c7aced9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.654+0000 7f7b2310b740 1 --2- 172.21.15.120:0/3485040315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7aceb8d50 0x55c7aceba870 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.654+0000 7f7b2310b740 1 --2- 172.21.15.120:0/3485040315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7aceb77e0 0x55c7aceba330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.654+0000 7f7b2310b740 1 -- 172.21.15.120:0/3485040315 >> 172.21.15.120:0/3485040315 conn(0x55c7ace7e310 msgr2=0x55c7acec1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:22.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.655+0000 7f7b2310b740 1 -- 172.21.15.120:0/3485040315 shutdown_connections 2024-09-19T09:04:22.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.655+0000 7f7b2310b740 1 -- 172.21.15.120:0/3485040315 wait complete. 2024-09-19T09:04:22.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.655+0000 7f7b2310b740 1 -- start start 2024-09-19T09:04:22.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.655+0000 7f7b2310b740 1 Processor -- start 2024-09-19T09:04:22.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.655+0000 7f7b2310b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7aceb77e0 0x55c7acec6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:22.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.655+0000 7f7b2310b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7aceb8d50 0x55c7acec6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:22.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.655+0000 7f7b2310b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7acec2580 0x55c7acec6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:22.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.655+0000 7f7b2310b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c1ac8c0 con 0x55c7acec2580 2024-09-19T09:04:22.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.655+0000 7f7b2310b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c1ac1f0 con 0x55c7aceb77e0 2024-09-19T09:04:22.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.655+0000 7f7b2310b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c1a9410 con 0x55c7aceb8d50 2024-09-19T09:04:22.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.655+0000 7f7b2310b740 10 client.0 _ll_register_callbacks cb 0x7f7b1009b6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:22.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.655+0000 7f7b1ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7acec2580 0x55c7acec6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:22.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.655+0000 7f7b1ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7acec2580 0x55c7acec6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33724/0 (socket says 172.21.15.120:33724) 2024-09-19T09:04:22.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.655+0000 7f7b1ad9d640 1 -- 172.21.15.120:0/3890035898 learned_addr learned my addr 172.21.15.120:0/3890035898 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:22.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.655+0000 7f7b19d9b640 1 --2- 172.21.15.120:0/3890035898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7aceb8d50 0x55c7acec6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:22.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.656+0000 7f7b1a59c640 1 --2- 172.21.15.120:0/3890035898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7aceb77e0 0x55c7acec6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:22.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.656+0000 7f7b1ad9d640 1 -- 172.21.15.120:0/3890035898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7aceb8d50 msgr2=0x55c7acec6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:22.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.656+0000 7f7b1ad9d640 1 --2- 172.21.15.120:0/3890035898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7aceb8d50 0x55c7acec6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.656+0000 7f7b1ad9d640 1 -- 172.21.15.120:0/3890035898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7aceb77e0 msgr2=0x55c7acec6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:22.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.656+0000 7f7b1ad9d640 1 --2- 172.21.15.120:0/3890035898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7aceb77e0 0x55c7acec6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.656+0000 7f7b1ad9d640 1 -- 172.21.15.120:0/3890035898 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7acf3c070 con 0x55c7acec2580 2024-09-19T09:04:22.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.656+0000 7f7b1ad9d640 1 --2- 172.21.15.120:0/3890035898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7acec2580 0x55c7acec6fd0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7b14031b20 tx=0x7f7b14031b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e36e479f2f2f8b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:22.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.656+0000 7f7b01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:04:22.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.656+0000 7f7b01ffb640 1 -- 172.21.15.120:0/3890035898 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1402af20 con 0x55c7acec2580 2024-09-19T09:04:22.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.656+0000 7f7b01ffb640 1 -- 172.21.15.120:0/3890035898 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f7b14003b70 con 0x55c7acec2580 2024-09-19T09:04:22.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.656+0000 7f7b2310b740 1 -- 172.21.15.120:0/3890035898 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b100a4580 con 0x55c7acec2580 2024-09-19T09:04:22.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.656+0000 7f7b01ffb640 1 -- 172.21.15.120:0/3890035898 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7b14003d70 con 0x55c7acec2580 2024-09-19T09:04:22.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.656+0000 7f7b2310b740 10 client.15828 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:22.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.656+0000 7f7b2310b740 1 -- 172.21.15.120:0/3890035898 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c7acf3d310 con 0x55c7acec2580 2024-09-19T09:04:22.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.657+0000 7f7b01ffb640 1 -- 172.21.15.120:0/3890035898 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b100a4580 con 0x55c7acec2580 2024-09-19T09:04:22.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.657+0000 7f7b01ffb640 1 -- 172.21.15.120:0/3890035898 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7b14030070 con 0x55c7acec2580 2024-09-19T09:04:22.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.657+0000 7f7b2310b740 10 client.15828 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:22.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.657+0000 7f7b2310b740 10 client.15828 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:04:22.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.657+0000 7f7b2310b740 20 client.15828 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:22.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.657+0000 7f7b2310b740 1 --2- 172.21.15.120:0/3890035898 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55c7acf3d530 0x55c7acf3f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:22.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.657+0000 7f7b2310b740 4 client.15828 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:04:22.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.657+0000 7f7b2310b740 1 -- 172.21.15.120:0/3890035898 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c7ace22d70 con 0x55c7acf3d530 2024-09-19T09:04:22.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.657+0000 7f7b1a59c640 1 --2- 172.21.15.120:0/3890035898 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55c7acf3d530 0x55c7acf3f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:22.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.658+0000 7f7b01ffb640 10 client.15828 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:04:22.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.658+0000 7f7b1a59c640 1 --2- 172.21.15.120:0/3890035898 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55c7acf3d530 0x55c7acf3f920 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cec797ebe8728f9 server_cookie=53960b08f10627f0 in_seq=0 out_seq=0 2024-09-19T09:04:22.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.658+0000 7f7b01ffb640 1 -- 172.21.15.120:0/3890035898 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c7ace22d70 con 0x55c7acf3d530 2024-09-19T09:04:22.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.658+0000 7f7b01ffb640 10 client.15828 handle_command_reply: tid=0 2024-09-19T09:04:22.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.658+0000 7f7b2310b740 1 client.15828 shutdown 2024-09-19T09:04:22.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.658+0000 7f7b2310b740 1 -- 172.21.15.120:0/3890035898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7acec2580 msgr2=0x55c7acec6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:22.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.658+0000 7f7b2310b740 1 --2- 172.21.15.120:0/3890035898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7acec2580 0x55c7acec6fd0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7b14031b20 tx=0x7f7b14031b50 comp rx=0 tx=0).stop 2024-09-19T09:04:22.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.658+0000 7f7b2310b740 1 -- 172.21.15.120:0/3890035898 shutdown_connections 2024-09-19T09:04:22.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.658+0000 7f7b2310b740 1 --2- 172.21.15.120:0/3890035898 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55c7acf3d530 0x55c7acf3f920 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.658+0000 7f7b2310b740 1 --2- 172.21.15.120:0/3890035898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7acec2580 0x55c7acec6fd0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.658+0000 7f7b2310b740 1 --2- 172.21.15.120:0/3890035898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7aceb8d50 0x55c7acec6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.658+0000 7f7b2310b740 1 --2- 172.21.15.120:0/3890035898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7aceb77e0 0x55c7acec6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.658+0000 7f7b2310b740 1 -- 172.21.15.120:0/3890035898 >> 172.21.15.120:0/3890035898 conn(0x55c7ace7e310 msgr2=0x55c7acec1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:22.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.658+0000 7f7b01ffb640 0 client.15828 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:04:22.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.658+0000 7f7b2310b740 1 -- 172.21.15.120:0/3890035898 shutdown_connections 2024-09-19T09:04:22.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.658+0000 7f7b2310b740 1 -- 172.21.15.120:0/3890035898 wait complete. 2024-09-19T09:04:22.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.658+0000 7f7b2310b740 20 client.15828 trim_cache size 0 max 16384 2024-09-19T09:04:22.672 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:04:22.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.659+0000 7f7b20d32640 1 -- 172.21.15.120:0/591064937 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7b040a3240 msgr2=0x7f7b040a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:22.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.659+0000 7f7b20d32640 1 --2- 172.21.15.120:0/591064937 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7b040a3240 0x7f7b040a5700 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c005fd0 tx=0x7f7b0c00b390 comp rx=0 tx=0).stop 2024-09-19T09:04:22.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.659+0000 7f7b20d32640 1 -- 172.21.15.120:0/591064937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b1c101420 msgr2=0x7f7b1c197860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:22.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.659+0000 7f7b20d32640 1 --2- 172.21.15.120:0/591064937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b1c101420 0x7f7b1c197860 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7b1000d950 tx=0x7f7b1000de10 comp rx=0 tx=0).stop 2024-09-19T09:04:22.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.659+0000 7f7b20d32640 1 -- 172.21.15.120:0/591064937 shutdown_connections 2024-09-19T09:04:22.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.659+0000 7f7b20d32640 1 --2- 172.21.15.120:0/591064937 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7b040a3240 0x7f7b040a5700 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.659+0000 7f7b20d32640 1 --2- 172.21.15.120:0/591064937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b1c106a80 0x7f7b1c19bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.659+0000 7f7b20d32640 1 --2- 172.21.15.120:0/591064937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b1c101420 0x7f7b1c197860 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.659+0000 7f7b20d32640 1 --2- 172.21.15.120:0/591064937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b1c100a80 0x7f7b1c197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:22.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.659+0000 7f7b20d32640 1 -- 172.21.15.120:0/591064937 >> 172.21.15.120:0/591064937 conn(0x7f7b1c0fc7a0 msgr2=0x7f7b1c0fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:22.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.659+0000 7f7b20d32640 1 -- 172.21.15.120:0/591064937 shutdown_connections 2024-09-19T09:04:22.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:22.659+0000 7f7b20d32640 1 -- 172.21.15.120:0/591064937 wait complete. 2024-09-19T09:04:23.035 INFO:tasks.workunit.client.0.smithi120.stdout: 1 11836 4.46 MB/sec warmup 18 sec latency 14499.310 ms 2024-09-19T09:04:23.036 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:22 smithi120 ceph-mon[31150]: pgmap v386: 97 pgs: 97 active+clean; 12 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 894 KiB/s wr, 128 op/s 2024-09-19T09:04:23.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:22 smithi196 ceph-mon[35150]: pgmap v386: 97 pgs: 97 active+clean; 12 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 894 KiB/s wr, 128 op/s 2024-09-19T09:04:23.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:22 smithi153 ceph-mon[35071]: pgmap v386: 97 pgs: 97 active+clean; 12 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 894 KiB/s wr, 128 op/s 2024-09-19T09:04:23.372 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:04:23.616 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:04:24.035 INFO:tasks.workunit.client.0.smithi120.stdout: 1 11836 4.23 MB/sec warmup 19 sec latency 15499.393 ms 2024-09-19T09:04:24.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.147+0000 7f369703e640 1 -- 172.21.15.120:0/1329014396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3690106a80 msgr2=0x7f3690106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:24.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.147+0000 7f369703e640 1 --2- 172.21.15.120:0/1329014396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3690106a80 0x7f3690106e60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3684009930 tx=0x7f368402a420 comp rx=0 tx=0).stop 2024-09-19T09:04:24.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.147+0000 7f369703e640 1 -- 172.21.15.120:0/1329014396 shutdown_connections 2024-09-19T09:04:24.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.147+0000 7f369703e640 1 --2- 172.21.15.120:0/1329014396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3690101420 0x7f369010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.147+0000 7f369703e640 1 --2- 172.21.15.120:0/1329014396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3690100a80 0x7f3690100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.147+0000 7f369703e640 1 --2- 172.21.15.120:0/1329014396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3690106a80 0x7f3690106e60 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.147+0000 7f369703e640 1 -- 172.21.15.120:0/1329014396 >> 172.21.15.120:0/1329014396 conn(0x7f36900fc7a0 msgr2=0x7f36900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:24.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.147+0000 7f369703e640 1 -- 172.21.15.120:0/1329014396 shutdown_connections 2024-09-19T09:04:24.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.147+0000 7f369703e640 1 -- 172.21.15.120:0/1329014396 wait complete. 2024-09-19T09:04:24.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.148+0000 7f369703e640 1 Processor -- start 2024-09-19T09:04:24.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.148+0000 7f369703e640 1 -- start start 2024-09-19T09:04:24.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.148+0000 7f369703e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3690100a80 0x7f36901973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:24.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.148+0000 7f369703e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3690101420 0x7f3690197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:24.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.149+0000 7f369703e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3690106a80 0x7f369019bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:24.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.149+0000 7f369703e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3690110ea0 con 0x7f3690101420 2024-09-19T09:04:24.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.149+0000 7f369703e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3690110d20 con 0x7f3690106a80 2024-09-19T09:04:24.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.149+0000 7f369703e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3690111020 con 0x7f3690100a80 2024-09-19T09:04:24.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.149+0000 7f368ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3690101420 0x7f3690197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:24.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.149+0000 7f368ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3690101420 0x7f3690197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33730/0 (socket says 172.21.15.120:33730) 2024-09-19T09:04:24.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.149+0000 7f368ffff640 1 -- 172.21.15.120:0/195355696 learned_addr learned my addr 172.21.15.120:0/195355696 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:24.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.149+0000 7f3694dda640 1 --2- 172.21.15.120:0/195355696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3690100a80 0x7f36901973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:24.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.149+0000 7f36955db640 1 --2- 172.21.15.120:0/195355696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3690106a80 0x7f369019bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:24.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.149+0000 7f368ffff640 1 -- 172.21.15.120:0/195355696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3690100a80 msgr2=0x7f36901973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:24.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.149+0000 7f368ffff640 1 --2- 172.21.15.120:0/195355696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3690100a80 0x7f36901973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.149+0000 7f368ffff640 1 -- 172.21.15.120:0/195355696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3690106a80 msgr2=0x7f369019bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:24.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.149+0000 7f368ffff640 1 --2- 172.21.15.120:0/195355696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3690106a80 0x7f369019bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.149+0000 7f368ffff640 1 -- 172.21.15.120:0/195355696 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f369019c390 con 0x7f3690101420 2024-09-19T09:04:24.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.149+0000 7f368ffff640 1 --2- 172.21.15.120:0/195355696 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3690101420 0x7f3690197920 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f367c00b4a0 tx=0x7f367c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:24.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.149+0000 7f368dffb640 1 -- 172.21.15.120:0/195355696 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c014070 con 0x7f3690101420 2024-09-19T09:04:24.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.149+0000 7f368dffb640 1 -- 172.21.15.120:0/195355696 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f367c010af0 con 0x7f3690101420 2024-09-19T09:04:24.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.149+0000 7f369703e640 1 -- 172.21.15.120:0/195355696 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f369019c680 con 0x7f3690101420 2024-09-19T09:04:24.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.149+0000 7f368dffb640 1 -- 172.21.15.120:0/195355696 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c017d10 con 0x7f3690101420 2024-09-19T09:04:24.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.149+0000 7f369703e640 1 -- 172.21.15.120:0/195355696 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36901a3f60 con 0x7f3690101420 2024-09-19T09:04:24.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.152+0000 7f368dffb640 1 -- 172.21.15.120:0/195355696 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f367c017070 con 0x7f3690101420 2024-09-19T09:04:24.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.152+0000 7f3699417740 1 Processor -- start 2024-09-19T09:04:24.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.152+0000 7f3699417740 1 -- start start 2024-09-19T09:04:24.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.152+0000 7f368dffb640 1 --2- 172.21.15.120:0/195355696 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f36800a3240 0x7f36800a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:24.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.152+0000 7f3699417740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e77472c30 0x558e774712a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:24.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.152+0000 7f3699417740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e774717e0 0x558e77471c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:24.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.152+0000 7f3699417740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e7747e970 0x558e77480d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:24.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.152+0000 7f3699417740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558e773d52f0 con 0x558e7747e970 2024-09-19T09:04:24.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.152+0000 7f3699417740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558e7737ccf0 con 0x558e774717e0 2024-09-19T09:04:24.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.152+0000 7f3699417740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558e773d12b0 con 0x558e77472c30 2024-09-19T09:04:24.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.152+0000 7f36955db640 1 --2- 172.21.15.120:0/195355696 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f36800a3240 0x7f36800a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:24.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.153+0000 7f368dffb640 1 -- 172.21.15.120:0/195355696 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f367c0ced30 con 0x7f3690101420 2024-09-19T09:04:24.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.153+0000 7f3694dda640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e77472c30 0x558e774712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:24.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.153+0000 7f36955db640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e7747e970 0x558e77480d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:24.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.153+0000 7f3694dda640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e77472c30 0x558e774712a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:38874/0 (socket says 172.21.15.120:38874) 2024-09-19T09:04:24.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.153+0000 7f3694dda640 1 -- 172.21.15.120:0/3050165920 learned_addr learned my addr 172.21.15.120:0/3050165920 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:24.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.153+0000 7f368ffff640 1 --2- 172.21.15.120:0/3050165920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e774717e0 0x558e77471c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:24.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.153+0000 7f36955db640 1 -- 172.21.15.120:0/3050165920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e77472c30 msgr2=0x558e774712a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:24.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.153+0000 7f36955db640 1 --2- 172.21.15.120:0/3050165920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e77472c30 0x558e774712a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.153+0000 7f36955db640 1 -- 172.21.15.120:0/3050165920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e774717e0 msgr2=0x558e77471c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:24.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.153+0000 7f36955db640 1 --2- 172.21.15.120:0/3050165920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e774717e0 0x558e77471c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.153+0000 7f36955db640 1 -- 172.21.15.120:0/3050165920 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e773613a0 con 0x558e7747e970 2024-09-19T09:04:24.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.153+0000 7f36955db640 1 --2- 172.21.15.120:0/195355696 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f36800a3240 0x7f36800a5700 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3688005fd0 tx=0x7f36880146d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:24.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.153+0000 7f36955db640 1 --2- 172.21.15.120:0/3050165920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e7747e970 0x558e77480d60 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3688012cf0 tx=0x7f368801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9384b0698c02b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:24.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.153+0000 7f367affd640 1 -- 172.21.15.120:0/3050165920 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36880147c0 con 0x558e7747e970 2024-09-19T09:04:24.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.153+0000 7f367affd640 1 -- 172.21.15.120:0/3050165920 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3688026070 con 0x558e7747e970 2024-09-19T09:04:24.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.153+0000 7f367affd640 1 -- 172.21.15.120:0/3050165920 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3688021b90 con 0x558e7747e970 2024-09-19T09:04:24.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.154+0000 7f3699417740 1 -- 172.21.15.120:0/3050165920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e7747e970 msgr2=0x558e77480d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:24.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.154+0000 7f3699417740 1 --2- 172.21.15.120:0/3050165920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e7747e970 0x558e77480d60 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3688012cf0 tx=0x7f368801c000 comp rx=0 tx=0).stop 2024-09-19T09:04:24.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.154+0000 7f3699417740 1 -- 172.21.15.120:0/3050165920 shutdown_connections 2024-09-19T09:04:24.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.154+0000 7f3699417740 1 --2- 172.21.15.120:0/3050165920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e7747e970 0x558e77480d60 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.154+0000 7f3699417740 1 --2- 172.21.15.120:0/3050165920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e774717e0 0x558e77471c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.154+0000 7f3699417740 1 --2- 172.21.15.120:0/3050165920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e77472c30 0x558e774712a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.154+0000 7f3699417740 1 -- 172.21.15.120:0/3050165920 >> 172.21.15.120:0/3050165920 conn(0x558e77438310 msgr2=0x558e7746de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:24.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.154+0000 7f3699417740 1 -- 172.21.15.120:0/3050165920 shutdown_connections 2024-09-19T09:04:24.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.154+0000 7f3699417740 1 -- 172.21.15.120:0/3050165920 wait complete. 2024-09-19T09:04:24.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.154+0000 7f3699417740 1 -- start start 2024-09-19T09:04:24.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.154+0000 7f3699417740 1 Processor -- start 2024-09-19T09:04:24.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.155+0000 7f3699417740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e774717e0 0x558e7747e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:24.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.155+0000 7f3699417740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e77472c30 0x558e7747b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:24.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.155+0000 7f3699417740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e7747e970 0x558e7747bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:24.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.155+0000 7f3699417740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558e774814e0 con 0x558e77472c30 2024-09-19T09:04:24.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.155+0000 7f3699417740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558e77390060 con 0x558e7747e970 2024-09-19T09:04:24.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.155+0000 7f3699417740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558e77481660 con 0x558e774717e0 2024-09-19T09:04:24.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.155+0000 7f3699417740 10 client.0 _ll_register_callbacks cb 0x558e770e38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:24.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.155+0000 7f368ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e77472c30 0x558e7747b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:24.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.155+0000 7f368ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e77472c30 0x558e7747b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33744/0 (socket says 172.21.15.120:33744) 2024-09-19T09:04:24.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.155+0000 7f368ffff640 1 -- 172.21.15.120:0/2568987465 learned_addr learned my addr 172.21.15.120:0/2568987465 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:24.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.155+0000 7f3694dda640 1 --2- 172.21.15.120:0/2568987465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e774717e0 0x558e7747e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:24.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.155+0000 7f36955db640 1 --2- 172.21.15.120:0/2568987465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e7747e970 0x558e7747bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:24.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.155+0000 7f368ffff640 1 -- 172.21.15.120:0/2568987465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e774717e0 msgr2=0x558e7747e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:24.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.155+0000 7f368ffff640 1 --2- 172.21.15.120:0/2568987465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e774717e0 0x558e7747e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.155+0000 7f368ffff640 1 -- 172.21.15.120:0/2568987465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e7747e970 msgr2=0x558e7747bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:24.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.155+0000 7f368ffff640 1 --2- 172.21.15.120:0/2568987465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e7747e970 0x558e7747bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.155+0000 7f368ffff640 1 -- 172.21.15.120:0/2568987465 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e771463c0 con 0x558e77472c30 2024-09-19T09:04:24.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.155+0000 7f36955db640 1 --2- 172.21.15.120:0/2568987465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e7747e970 0x558e7747bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:04:24.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.155+0000 7f368ffff640 1 --2- 172.21.15.120:0/2568987465 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e77472c30 0x558e7747b580 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f367c00be60 tx=0x7f367c09ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860a2d897440db48 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:24.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.155+0000 7f366ffff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:04:24.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.155+0000 7f366ffff640 1 -- 172.21.15.120:0/2568987465 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c01e070 con 0x558e77472c30 2024-09-19T09:04:24.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.156+0000 7f366ffff640 1 -- 172.21.15.120:0/2568987465 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f367c017360 con 0x558e77472c30 2024-09-19T09:04:24.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.156+0000 7f3699417740 1 -- 172.21.15.120:0/2568987465 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3688026070 con 0x558e77472c30 2024-09-19T09:04:24.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.156+0000 7f366ffff640 1 -- 172.21.15.120:0/2568987465 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f367c09f700 con 0x558e77472c30 2024-09-19T09:04:24.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.156+0000 7f3699417740 10 client.15840 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:24.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.156+0000 7f3699417740 1 -- 172.21.15.120:0/2568987465 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558e7748cef0 con 0x558e77472c30 2024-09-19T09:04:24.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.156+0000 7f366ffff640 1 -- 172.21.15.120:0/2568987465 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3688026070 con 0x558e77472c30 2024-09-19T09:04:24.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.156+0000 7f366ffff640 1 -- 172.21.15.120:0/2568987465 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f367c0979c0 con 0x558e77472c30 2024-09-19T09:04:24.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.156+0000 7f3699417740 10 client.15840 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:24.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.156+0000 7f3699417740 10 client.15840 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:04:24.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.156+0000 7f3699417740 20 client.15840 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:24.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.156+0000 7f3699417740 1 --2- 172.21.15.120:0/2568987465 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558e7748e350 0x558e77490760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:24.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.156+0000 7f3699417740 4 client.15840 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:04:24.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.156+0000 7f3699417740 1 -- 172.21.15.120:0/2568987465 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e773e2860 con 0x558e7748e350 2024-09-19T09:04:24.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.156+0000 7f3694dda640 1 --2- 172.21.15.120:0/2568987465 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558e7748e350 0x558e77490760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:24.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.157+0000 7f366ffff640 10 client.15840 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:04:24.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.157+0000 7f3694dda640 1 --2- 172.21.15.120:0/2568987465 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558e7748e350 0x558e77490760 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3020a1019bd30399 server_cookie=33f2ce4443bd65e3 in_seq=0 out_seq=0 2024-09-19T09:04:24.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.158+0000 7f366ffff640 1 -- 172.21.15.120:0/2568987465 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e773e2860 con 0x558e7748e350 2024-09-19T09:04:24.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.158+0000 7f366ffff640 10 client.15840 handle_command_reply: tid=0 2024-09-19T09:04:24.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.158+0000 7f3699417740 1 client.15840 shutdown 2024-09-19T09:04:24.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.158+0000 7f3699417740 1 -- 172.21.15.120:0/2568987465 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e77472c30 msgr2=0x558e7747b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:24.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.158+0000 7f3699417740 1 --2- 172.21.15.120:0/2568987465 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e77472c30 0x558e7747b580 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f367c00be60 tx=0x7f367c09ad70 comp rx=0 tx=0).stop 2024-09-19T09:04:24.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.158+0000 7f3699417740 1 -- 172.21.15.120:0/2568987465 shutdown_connections 2024-09-19T09:04:24.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.158+0000 7f3699417740 1 --2- 172.21.15.120:0/2568987465 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558e7748e350 0x558e77490760 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.158+0000 7f3699417740 1 --2- 172.21.15.120:0/2568987465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e7747e970 0x558e7747bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.158+0000 7f3699417740 1 --2- 172.21.15.120:0/2568987465 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e77472c30 0x558e7747b580 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.158+0000 7f3699417740 1 --2- 172.21.15.120:0/2568987465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e774717e0 0x558e7747e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.158+0000 7f366ffff640 0 client.15840 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:04:24.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.158+0000 7f3699417740 1 -- 172.21.15.120:0/2568987465 >> 172.21.15.120:0/2568987465 conn(0x558e77438310 msgr2=0x558e77473bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:24.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.158+0000 7f3699417740 1 -- 172.21.15.120:0/2568987465 shutdown_connections 2024-09-19T09:04:24.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.158+0000 7f3699417740 1 -- 172.21.15.120:0/2568987465 wait complete. 2024-09-19T09:04:24.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.158+0000 7f3699417740 20 client.15840 trim_cache size 0 max 16384 2024-09-19T09:04:24.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.171+0000 7f3699417740 1 Processor -- start 2024-09-19T09:04:24.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.171+0000 7f3699417740 1 -- start start 2024-09-19T09:04:24.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.171+0000 7f3699417740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e774717e0 0x558e77479150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:24.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.171+0000 7f3699417740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e77472c30 0x558e7748a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:24.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.171+0000 7f3699417740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e7747e970 0x558e7747d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:24.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.171+0000 7f3699417740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f36901ab420 con 0x558e77472c30 2024-09-19T09:04:24.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.171+0000 7f3699417740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f36901acf60 con 0x558e774717e0 2024-09-19T09:04:24.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.171+0000 7f3699417740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f36901abaf0 con 0x558e7747e970 2024-09-19T09:04:24.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.171+0000 7f368ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e77472c30 0x558e7748a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:24.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.171+0000 7f368ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e77472c30 0x558e7748a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33758/0 (socket says 172.21.15.120:33758) 2024-09-19T09:04:24.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.171+0000 7f368ffff640 1 -- 172.21.15.120:0/3640233908 learned_addr learned my addr 172.21.15.120:0/3640233908 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:24.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.171+0000 7f3694dda640 1 --2- 172.21.15.120:0/3640233908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e774717e0 0x558e77479150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:24.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.171+0000 7f36955db640 1 --2- 172.21.15.120:0/3640233908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e7747e970 0x558e7747d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:24.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.171+0000 7f368ffff640 1 -- 172.21.15.120:0/3640233908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e7747e970 msgr2=0x558e7747d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:24.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.171+0000 7f368ffff640 1 --2- 172.21.15.120:0/3640233908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e7747e970 0x558e7747d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.171+0000 7f368ffff640 1 -- 172.21.15.120:0/3640233908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e774717e0 msgr2=0x558e77479150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:24.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.171+0000 7f368ffff640 1 --2- 172.21.15.120:0/3640233908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e774717e0 0x558e77479150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.171+0000 7f368ffff640 1 -- 172.21.15.120:0/3640233908 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e77490de0 con 0x558e77472c30 2024-09-19T09:04:24.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.171+0000 7f36955db640 1 --2- 172.21.15.120:0/3640233908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e7747e970 0x558e7747d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:04:24.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.171+0000 7f368ffff640 1 --2- 172.21.15.120:0/3640233908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e77472c30 0x558e7748a740 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f367c09f8c0 tx=0x7f367c099860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0631f4735ac283c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:24.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.172+0000 7f367a7fc640 1 -- 172.21.15.120:0/3640233908 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c0b9070 con 0x558e77472c30 2024-09-19T09:04:24.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.172+0000 7f367a7fc640 1 -- 172.21.15.120:0/3640233908 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f367c017360 con 0x558e77472c30 2024-09-19T09:04:24.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.172+0000 7f367a7fc640 1 -- 172.21.15.120:0/3640233908 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c01e070 con 0x558e77472c30 2024-09-19T09:04:24.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.172+0000 7f3699417740 1 -- 172.21.15.120:0/3640233908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e77472c30 msgr2=0x558e7748a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:24.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.172+0000 7f3699417740 1 --2- 172.21.15.120:0/3640233908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e77472c30 0x558e7748a740 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f367c09f8c0 tx=0x7f367c099860 comp rx=0 tx=0).stop 2024-09-19T09:04:24.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.172+0000 7f3699417740 1 -- 172.21.15.120:0/3640233908 shutdown_connections 2024-09-19T09:04:24.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.172+0000 7f3699417740 1 --2- 172.21.15.120:0/3640233908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e7747e970 0x558e7747d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.172+0000 7f3699417740 1 --2- 172.21.15.120:0/3640233908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e77472c30 0x558e7748a740 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.172+0000 7f3699417740 1 --2- 172.21.15.120:0/3640233908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e774717e0 0x558e77479150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.172+0000 7f3699417740 1 -- 172.21.15.120:0/3640233908 >> 172.21.15.120:0/3640233908 conn(0x558e77438310 msgr2=0x558e7747cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:24.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.172+0000 7f3699417740 1 -- 172.21.15.120:0/3640233908 shutdown_connections 2024-09-19T09:04:24.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.172+0000 7f3699417740 1 -- 172.21.15.120:0/3640233908 wait complete. 2024-09-19T09:04:24.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.173+0000 7f3699417740 1 -- start start 2024-09-19T09:04:24.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.173+0000 7f3699417740 1 Processor -- start 2024-09-19T09:04:24.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.173+0000 7f3699417740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e774717e0 0x558e77486a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:24.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.173+0000 7f3699417740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e77472c30 0x558e77486fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:24.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.173+0000 7f3699417740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e7747e970 0x558e77483880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:24.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.173+0000 7f3699417740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f36901ac890 con 0x558e774717e0 2024-09-19T09:04:24.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.173+0000 7f3699417740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f36901ac1c0 con 0x558e7747e970 2024-09-19T09:04:24.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.173+0000 7f3699417740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f36901a93e0 con 0x558e77472c30 2024-09-19T09:04:24.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.173+0000 7f3699417740 10 client.0 _ll_register_callbacks cb 0x7f367c00b3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:24.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.173+0000 7f3694dda640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e774717e0 0x558e77486a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:24.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.173+0000 7f3694dda640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e774717e0 0x558e77486a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33764/0 (socket says 172.21.15.120:33764) 2024-09-19T09:04:24.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.173+0000 7f3694dda640 1 -- 172.21.15.120:0/1257448551 learned_addr learned my addr 172.21.15.120:0/1257448551 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:24.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.174+0000 7f368ffff640 1 --2- 172.21.15.120:0/1257448551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e77472c30 0x558e77486fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:24.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.174+0000 7f36955db640 1 --2- 172.21.15.120:0/1257448551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e7747e970 0x558e77483880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:24.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.174+0000 7f3694dda640 1 -- 172.21.15.120:0/1257448551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e77472c30 msgr2=0x558e77486fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:24.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.174+0000 7f3694dda640 1 --2- 172.21.15.120:0/1257448551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e77472c30 0x558e77486fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.174+0000 7f3694dda640 1 -- 172.21.15.120:0/1257448551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e7747e970 msgr2=0x558e77483880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:24.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.174+0000 7f3694dda640 1 --2- 172.21.15.120:0/1257448551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e7747e970 0x558e77483880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.174+0000 7f3694dda640 1 -- 172.21.15.120:0/1257448551 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e77483f90 con 0x558e774717e0 2024-09-19T09:04:24.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.174+0000 7f36955db640 1 --2- 172.21.15.120:0/1257448551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e7747e970 0x558e77483880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:04:24.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.174+0000 7f3694dda640 1 --2- 172.21.15.120:0/1257448551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e774717e0 0x558e77486a90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3684009610 tx=0x7f3684004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4f16143b28f64fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:24.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.174+0000 7f366ffff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:04:24.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.174+0000 7f366ffff640 1 -- 172.21.15.120:0/1257448551 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3684035040 con 0x558e774717e0 2024-09-19T09:04:24.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.174+0000 7f366ffff640 1 -- 172.21.15.120:0/1257448551 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f3684005240 con 0x558e774717e0 2024-09-19T09:04:24.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.174+0000 7f3699417740 1 -- 172.21.15.120:0/1257448551 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f367c017360 con 0x558e774717e0 2024-09-19T09:04:24.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.174+0000 7f366ffff640 1 -- 172.21.15.120:0/1257448551 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3684005440 con 0x558e774717e0 2024-09-19T09:04:24.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.174+0000 7f3699417740 10 client.15846 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:24.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.174+0000 7f3699417740 1 -- 172.21.15.120:0/1257448551 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558e774f99e0 con 0x558e774717e0 2024-09-19T09:04:24.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.174+0000 7f366ffff640 1 -- 172.21.15.120:0/1257448551 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f367c017360 con 0x558e774717e0 2024-09-19T09:04:24.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.175+0000 7f366ffff640 1 -- 172.21.15.120:0/1257448551 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3684031950 con 0x558e774717e0 2024-09-19T09:04:24.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.175+0000 7f3699417740 10 client.15846 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:24.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.175+0000 7f3699417740 10 client.15846 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:04:24.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.175+0000 7f3699417740 20 client.15846 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:24.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.175+0000 7f3699417740 1 --2- 172.21.15.120:0/1257448551 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558e774fa1b0 0x558e774fc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:24.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.175+0000 7f3699417740 4 client.15846 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:04:24.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.175+0000 7f3699417740 1 -- 172.21.15.120:0/1257448551 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558e773dcd70 con 0x558e774fa1b0 2024-09-19T09:04:24.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.175+0000 7f368ffff640 1 --2- 172.21.15.120:0/1257448551 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558e774fa1b0 0x558e774fc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:24.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.175+0000 7f366ffff640 10 client.15846 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:04:24.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.175+0000 7f368ffff640 1 --2- 172.21.15.120:0/1257448551 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558e774fa1b0 0x558e774fc5d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=625fb7a3eaa56ce9 server_cookie=7fbb18c929c61203 in_seq=0 out_seq=0 2024-09-19T09:04:24.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.176+0000 7f366ffff640 1 -- 172.21.15.120:0/1257448551 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558e773dcd70 con 0x558e774fa1b0 2024-09-19T09:04:24.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.176+0000 7f366ffff640 10 client.15846 handle_command_reply: tid=0 2024-09-19T09:04:24.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.176+0000 7f3699417740 1 client.15846 shutdown 2024-09-19T09:04:24.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.176+0000 7f3699417740 1 -- 172.21.15.120:0/1257448551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e774717e0 msgr2=0x558e77486a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:24.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.176+0000 7f3699417740 1 --2- 172.21.15.120:0/1257448551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e774717e0 0x558e77486a90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3684009610 tx=0x7f3684004270 comp rx=0 tx=0).stop 2024-09-19T09:04:24.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.176+0000 7f3699417740 1 -- 172.21.15.120:0/1257448551 shutdown_connections 2024-09-19T09:04:24.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.176+0000 7f3699417740 1 --2- 172.21.15.120:0/1257448551 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558e774fa1b0 0x558e774fc5d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.176+0000 7f3699417740 1 --2- 172.21.15.120:0/1257448551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e7747e970 0x558e77483880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.176+0000 7f3699417740 1 --2- 172.21.15.120:0/1257448551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e77472c30 0x558e77486fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.176+0000 7f3699417740 1 --2- 172.21.15.120:0/1257448551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e774717e0 0x558e77486a90 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.176+0000 7f3699417740 1 -- 172.21.15.120:0/1257448551 >> 172.21.15.120:0/1257448551 conn(0x558e77438310 msgr2=0x558e77473b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:24.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.176+0000 7f366ffff640 0 client.15846 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:04:24.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.176+0000 7f3699417740 1 -- 172.21.15.120:0/1257448551 shutdown_connections 2024-09-19T09:04:24.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.176+0000 7f3699417740 1 -- 172.21.15.120:0/1257448551 wait complete. 2024-09-19T09:04:24.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.176+0000 7f3699417740 20 client.15846 trim_cache size 0 max 16384 2024-09-19T09:04:24.187 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:04:24.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.176+0000 7f369703e640 1 -- 172.21.15.120:0/195355696 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f36800a3240 msgr2=0x7f36800a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:24.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.176+0000 7f369703e640 1 --2- 172.21.15.120:0/195355696 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f36800a3240 0x7f36800a5700 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3688005fd0 tx=0x7f36880146d0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.176+0000 7f369703e640 1 -- 172.21.15.120:0/195355696 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3690101420 msgr2=0x7f3690197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:24.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.176+0000 7f369703e640 1 --2- 172.21.15.120:0/195355696 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3690101420 0x7f3690197920 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f367c00b4a0 tx=0x7f367c00b960 comp rx=0 tx=0).stop 2024-09-19T09:04:24.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.177+0000 7f369703e640 1 -- 172.21.15.120:0/195355696 shutdown_connections 2024-09-19T09:04:24.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.177+0000 7f369703e640 1 --2- 172.21.15.120:0/195355696 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f36800a3240 0x7f36800a5700 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.177+0000 7f369703e640 1 --2- 172.21.15.120:0/195355696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3690106a80 0x7f369019bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.177+0000 7f369703e640 1 --2- 172.21.15.120:0/195355696 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3690101420 0x7f3690197920 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.177+0000 7f369703e640 1 --2- 172.21.15.120:0/195355696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3690100a80 0x7f36901973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:24.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.177+0000 7f369703e640 1 -- 172.21.15.120:0/195355696 >> 172.21.15.120:0/195355696 conn(0x7f36900fc7a0 msgr2=0x7f36900fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:24.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.177+0000 7f369703e640 1 -- 172.21.15.120:0/195355696 shutdown_connections 2024-09-19T09:04:24.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:24.177+0000 7f369703e640 1 -- 172.21.15.120:0/195355696 wait complete. 2024-09-19T09:04:24.955 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:24 smithi120 ceph-mon[31150]: pgmap v387: 97 pgs: 97 active+clean; 23 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.9 MiB/s wr, 68 op/s 2024-09-19T09:04:24.985 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:04:25.035 INFO:tasks.workunit.client.0.smithi120.stdout: 1 11836 4.02 MB/sec warmup 20 sec latency 16499.449 ms 2024-09-19T09:04:25.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:24 smithi196 ceph-mon[35150]: pgmap v387: 97 pgs: 97 active+clean; 23 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.9 MiB/s wr, 68 op/s 2024-09-19T09:04:25.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:24 smithi153 ceph-mon[35071]: pgmap v387: 97 pgs: 97 active+clean; 23 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.9 MiB/s wr, 68 op/s 2024-09-19T09:04:25.231 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:04:25.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.731+0000 7fc171754640 1 -- 172.21.15.120:0/936699861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc16c106a80 msgr2=0x7fc16c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:25.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.731+0000 7fc171754640 1 --2- 172.21.15.120:0/936699861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc16c106a80 0x7fc16c106e60 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc15c009930 tx=0x7fc15c02a420 comp rx=0 tx=0).stop 2024-09-19T09:04:25.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.731+0000 7fc171754640 1 -- 172.21.15.120:0/936699861 shutdown_connections 2024-09-19T09:04:25.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.731+0000 7fc171754640 1 --2- 172.21.15.120:0/936699861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc16c101420 0x7fc16c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.731+0000 7fc171754640 1 --2- 172.21.15.120:0/936699861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc16c100a80 0x7fc16c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.731+0000 7fc171754640 1 --2- 172.21.15.120:0/936699861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc16c106a80 0x7fc16c106e60 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.731+0000 7fc171754640 1 -- 172.21.15.120:0/936699861 >> 172.21.15.120:0/936699861 conn(0x7fc16c0fc7a0 msgr2=0x7fc16c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:25.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.731+0000 7fc171754640 1 -- 172.21.15.120:0/936699861 shutdown_connections 2024-09-19T09:04:25.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.731+0000 7fc171754640 1 -- 172.21.15.120:0/936699861 wait complete. 2024-09-19T09:04:25.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.732+0000 7fc171754640 1 Processor -- start 2024-09-19T09:04:25.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.732+0000 7fc171754640 1 -- start start 2024-09-19T09:04:25.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.732+0000 7fc171754640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc16c100a80 0x7fc16c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:25.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.732+0000 7fc171754640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc16c101420 0x7fc16c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:25.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.732+0000 7fc171754640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc16c106a80 0x7fc16c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:25.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.732+0000 7fc171754640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c110e80 con 0x7fc16c101420 2024-09-19T09:04:25.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.732+0000 7fc171754640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c110d00 con 0x7fc16c106a80 2024-09-19T09:04:25.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.732+0000 7fc171754640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c111000 con 0x7fc16c100a80 2024-09-19T09:04:25.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.732+0000 7fc163fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc16c101420 0x7fc16c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:25.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.732+0000 7fc163fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc16c101420 0x7fc16c1989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33780/0 (socket says 172.21.15.120:33780) 2024-09-19T09:04:25.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.732+0000 7fc163fff640 1 -- 172.21.15.120:0/1073660762 learned_addr learned my addr 172.21.15.120:0/1073660762 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:25.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.732+0000 7fc16affd640 1 --2- 172.21.15.120:0/1073660762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc16c100a80 0x7fc16c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:25.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.732+0000 7fc163fff640 1 -- 172.21.15.120:0/1073660762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc16c100a80 msgr2=0x7fc16c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:25.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.732+0000 7fc16b7fe640 1 --2- 172.21.15.120:0/1073660762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc16c106a80 0x7fc16c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:25.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.732+0000 7fc163fff640 1 --2- 172.21.15.120:0/1073660762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc16c100a80 0x7fc16c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.732+0000 7fc163fff640 1 -- 172.21.15.120:0/1073660762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc16c106a80 msgr2=0x7fc16c19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:25.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.732+0000 7fc163fff640 1 --2- 172.21.15.120:0/1073660762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc16c106a80 0x7fc16c19cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.732+0000 7fc163fff640 1 -- 172.21.15.120:0/1073660762 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc16c19d410 con 0x7fc16c101420 2024-09-19T09:04:25.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.733+0000 7fc163fff640 1 --2- 172.21.15.120:0/1073660762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc16c101420 0x7fc16c1989a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc15400d950 tx=0x7fc15400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:25.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.733+0000 7fc168ff9640 1 -- 172.21.15.120:0/1073660762 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc154017070 con 0x7fc16c101420 2024-09-19T09:04:25.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.733+0000 7fc171754640 1 -- 172.21.15.120:0/1073660762 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc16c19d700 con 0x7fc16c101420 2024-09-19T09:04:25.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.733+0000 7fc168ff9640 1 -- 172.21.15.120:0/1073660762 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc154009c30 con 0x7fc16c101420 2024-09-19T09:04:25.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.733+0000 7fc168ff9640 1 -- 172.21.15.120:0/1073660762 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15400f6b0 con 0x7fc16c101420 2024-09-19T09:04:25.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.733+0000 7fc171754640 1 -- 172.21.15.120:0/1073660762 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc16c1a4fe0 con 0x7fc16c101420 2024-09-19T09:04:25.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.735+0000 7fc168ff9640 1 -- 172.21.15.120:0/1073660762 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fc1540109d0 con 0x7fc16c101420 2024-09-19T09:04:25.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.735+0000 7fc173b2d740 1 Processor -- start 2024-09-19T09:04:25.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.735+0000 7fc173b2d740 1 -- start start 2024-09-19T09:04:25.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.736+0000 7fc173b2d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556082e4db40 0x556082e4c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:25.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.736+0000 7fc173b2d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556082e4c5d0 0x556082e4ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:25.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.736+0000 7fc173b2d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556082e57370 0x556082e59760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:25.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.736+0000 7fc173b2d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556082db22f0 con 0x556082e4db40 2024-09-19T09:04:25.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.736+0000 7fc173b2d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556082d59cf0 con 0x556082e4c5d0 2024-09-19T09:04:25.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.736+0000 7fc173b2d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556082dae2b0 con 0x556082e57370 2024-09-19T09:04:25.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.736+0000 7fc168ff9640 1 --2- 172.21.15.120:0/1073660762 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc1580a3240 0x7fc1580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:25.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.736+0000 7fc16b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556082e4db40 0x556082e4c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:25.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.736+0000 7fc16b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556082e4db40 0x556082e4c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33788/0 (socket says 172.21.15.120:33788) 2024-09-19T09:04:25.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.736+0000 7fc16b7fe640 1 -- 172.21.15.120:0/3094618779 learned_addr learned my addr 172.21.15.120:0/3094618779 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:25.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.736+0000 7fc16affd640 1 --2- 172.21.15.120:0/3094618779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556082e4c5d0 0x556082e4ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:25.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.736+0000 7fc16b7fe640 1 --2- 172.21.15.120:0/1073660762 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc1580a3240 0x7fc1580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:25.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.736+0000 7fc168ff9640 1 -- 172.21.15.120:0/1073660762 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fc1540cf650 con 0x7fc16c101420 2024-09-19T09:04:25.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.736+0000 7fc163fff640 1 --2- 172.21.15.120:0/3094618779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556082e57370 0x556082e59760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:25.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.736+0000 7fc16b7fe640 1 -- 172.21.15.120:0/3094618779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556082e57370 msgr2=0x556082e59760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:25.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.736+0000 7fc16b7fe640 1 --2- 172.21.15.120:0/3094618779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556082e57370 0x556082e59760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.736+0000 7fc16b7fe640 1 -- 172.21.15.120:0/3094618779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556082e4c5d0 msgr2=0x556082e4ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:25.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.736+0000 7fc16b7fe640 1 --2- 172.21.15.120:0/3094618779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556082e4c5d0 0x556082e4ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.736+0000 7fc16b7fe640 1 -- 172.21.15.120:0/3094618779 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556082d3e3a0 con 0x556082e4db40 2024-09-19T09:04:25.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.736+0000 7fc163fff640 1 --2- 172.21.15.120:0/3094618779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556082e57370 0x556082e59760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:04:25.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.736+0000 7fc16affd640 1 --2- 172.21.15.120:0/3094618779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556082e4c5d0 0x556082e4ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:04:25.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.737+0000 7fc16b7fe640 1 --2- 172.21.15.120:0/3094618779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556082e4db40 0x556082e4c090 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc164014cf0 tx=0x7fc164011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0cc8126a137c15a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:25.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.737+0000 7fc1617fa640 1 -- 172.21.15.120:0/3094618779 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc164023070 con 0x556082e4db40 2024-09-19T09:04:25.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.737+0000 7fc1617fa640 1 -- 172.21.15.120:0/3094618779 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc164009ab0 con 0x556082e4db40 2024-09-19T09:04:25.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.737+0000 7fc1617fa640 1 -- 172.21.15.120:0/3094618779 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16401e990 con 0x556082e4db40 2024-09-19T09:04:25.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.738+0000 7fc173b2d740 1 -- 172.21.15.120:0/3094618779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556082e4db40 msgr2=0x556082e4c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:25.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.738+0000 7fc173b2d740 1 --2- 172.21.15.120:0/3094618779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556082e4db40 0x556082e4c090 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc164014cf0 tx=0x7fc164011750 comp rx=0 tx=0).stop 2024-09-19T09:04:25.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.738+0000 7fc16b7fe640 1 --2- 172.21.15.120:0/1073660762 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc1580a3240 0x7fc1580a5700 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc164009fd0 tx=0x7fc164016000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:25.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.738+0000 7fc173b2d740 1 -- 172.21.15.120:0/3094618779 shutdown_connections 2024-09-19T09:04:25.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.738+0000 7fc173b2d740 1 --2- 172.21.15.120:0/3094618779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556082e57370 0x556082e59760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.738+0000 7fc173b2d740 1 --2- 172.21.15.120:0/3094618779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556082e4c5d0 0x556082e4ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.738+0000 7fc173b2d740 1 --2- 172.21.15.120:0/3094618779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556082e4db40 0x556082e4c090 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.738+0000 7fc173b2d740 1 -- 172.21.15.120:0/3094618779 >> 172.21.15.120:0/3094618779 conn(0x556082e15310 msgr2=0x556082e48c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:25.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.738+0000 7fc173b2d740 1 -- 172.21.15.120:0/3094618779 shutdown_connections 2024-09-19T09:04:25.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.738+0000 7fc173b2d740 1 -- 172.21.15.120:0/3094618779 wait complete. 2024-09-19T09:04:25.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.738+0000 7fc173b2d740 1 -- start start 2024-09-19T09:04:25.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.738+0000 7fc173b2d740 1 Processor -- start 2024-09-19T09:04:25.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.738+0000 7fc173b2d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556082e4c5d0 0x556082e56d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:25.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.738+0000 7fc173b2d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556082e4db40 0x556082e54040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:25.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.738+0000 7fc173b2d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556082e57370 0x556082e54650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:25.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.738+0000 7fc173b2d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556082e5a010 con 0x556082e4db40 2024-09-19T09:04:25.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.738+0000 7fc173b2d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556082d6d060 con 0x556082e4c5d0 2024-09-19T09:04:25.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.738+0000 7fc173b2d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556082e5a190 con 0x556082e57370 2024-09-19T09:04:25.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.738+0000 7fc173b2d740 10 client.0 _ll_register_callbacks cb 0x556082ac08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:25.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.739+0000 7fc163fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556082e4db40 0x556082e54040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:25.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.739+0000 7fc163fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556082e4db40 0x556082e54040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33800/0 (socket says 172.21.15.120:33800) 2024-09-19T09:04:25.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.739+0000 7fc163fff640 1 -- 172.21.15.120:0/2813803500 learned_addr learned my addr 172.21.15.120:0/2813803500 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:25.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.739+0000 7fc16b7fe640 1 --2- 172.21.15.120:0/2813803500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556082e57370 0x556082e54650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:25.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.739+0000 7fc16affd640 1 --2- 172.21.15.120:0/2813803500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556082e4c5d0 0x556082e56d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:25.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.739+0000 7fc163fff640 1 -- 172.21.15.120:0/2813803500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556082e57370 msgr2=0x556082e54650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:25.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.739+0000 7fc163fff640 1 --2- 172.21.15.120:0/2813803500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556082e57370 0x556082e54650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.739+0000 7fc163fff640 1 -- 172.21.15.120:0/2813803500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556082e4c5d0 msgr2=0x556082e56d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:25.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.739+0000 7fc163fff640 1 --2- 172.21.15.120:0/2813803500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556082e4c5d0 0x556082e56d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.739+0000 7fc163fff640 1 -- 172.21.15.120:0/2813803500 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556082b233c0 con 0x556082e4db40 2024-09-19T09:04:25.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.739+0000 7fc16b7fe640 1 --2- 172.21.15.120:0/2813803500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556082e57370 0x556082e54650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:04:25.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.739+0000 7fc16affd640 1 --2- 172.21.15.120:0/2813803500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556082e4c5d0 0x556082e56d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:04:25.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.739+0000 7fc163fff640 1 --2- 172.21.15.120:0/2813803500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556082e4db40 0x556082e54040 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc1540980d0 tx=0x7fc154000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7005461a94678e4f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:25.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.739+0000 7fc1527fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:04:25.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.739+0000 7fc1527fc640 1 -- 172.21.15.120:0/2813803500 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc154020030 con 0x556082e4db40 2024-09-19T09:04:25.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.739+0000 7fc1527fc640 1 -- 172.21.15.120:0/2813803500 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fc15409b0c0 con 0x556082e4db40 2024-09-19T09:04:25.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.739+0000 7fc173b2d740 1 -- 172.21.15.120:0/2813803500 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc164009ab0 con 0x556082e4db40 2024-09-19T09:04:25.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.739+0000 7fc1527fc640 1 -- 172.21.15.120:0/2813803500 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc1540b9050 con 0x556082e4db40 2024-09-19T09:04:25.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.739+0000 7fc173b2d740 10 client.15858 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:25.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.739+0000 7fc173b2d740 1 -- 172.21.15.120:0/2813803500 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556082e65a50 con 0x556082e4db40 2024-09-19T09:04:25.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.739+0000 7fc1527fc640 1 -- 172.21.15.120:0/2813803500 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc164009ab0 con 0x556082e4db40 2024-09-19T09:04:25.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.740+0000 7fc1527fc640 1 -- 172.21.15.120:0/2813803500 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc1540cf110 con 0x556082e4db40 2024-09-19T09:04:25.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.740+0000 7fc173b2d740 10 client.15858 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:25.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.740+0000 7fc173b2d740 10 client.15858 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:04:25.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.740+0000 7fc173b2d740 20 client.15858 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:25.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.740+0000 7fc173b2d740 1 --2- 172.21.15.120:0/2813803500 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x556082e66e60 0x556082e69250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:25.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.740+0000 7fc173b2d740 4 client.15858 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:04:25.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.740+0000 7fc173b2d740 1 -- 172.21.15.120:0/2813803500 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556082dbf860 con 0x556082e66e60 2024-09-19T09:04:25.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.740+0000 7fc16affd640 1 --2- 172.21.15.120:0/2813803500 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x556082e66e60 0x556082e69250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:25.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.740+0000 7fc16affd640 1 --2- 172.21.15.120:0/2813803500 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x556082e66e60 0x556082e69250 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fa2f0f8ee6947f19 server_cookie=fb67d891cfdb2438 in_seq=0 out_seq=0 2024-09-19T09:04:25.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.740+0000 7fc1527fc640 10 client.15858 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:04:25.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.741+0000 7fc1527fc640 1 -- 172.21.15.120:0/2813803500 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556082dbf860 con 0x556082e66e60 2024-09-19T09:04:25.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.741+0000 7fc1527fc640 10 client.15858 handle_command_reply: tid=0 2024-09-19T09:04:25.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.741+0000 7fc173b2d740 1 client.15858 shutdown 2024-09-19T09:04:25.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.741+0000 7fc173b2d740 1 -- 172.21.15.120:0/2813803500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556082e4db40 msgr2=0x556082e54040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:25.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.741+0000 7fc173b2d740 1 --2- 172.21.15.120:0/2813803500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556082e4db40 0x556082e54040 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc1540980d0 tx=0x7fc154000bc0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.741+0000 7fc173b2d740 1 -- 172.21.15.120:0/2813803500 shutdown_connections 2024-09-19T09:04:25.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.741+0000 7fc173b2d740 1 --2- 172.21.15.120:0/2813803500 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x556082e66e60 0x556082e69250 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.741+0000 7fc173b2d740 1 --2- 172.21.15.120:0/2813803500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556082e57370 0x556082e54650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.741+0000 7fc173b2d740 1 --2- 172.21.15.120:0/2813803500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556082e4db40 0x556082e54040 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.741+0000 7fc1527fc640 0 client.15858 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:04:25.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.741+0000 7fc173b2d740 1 --2- 172.21.15.120:0/2813803500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556082e4c5d0 0x556082e56d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.741+0000 7fc173b2d740 1 -- 172.21.15.120:0/2813803500 >> 172.21.15.120:0/2813803500 conn(0x556082e15310 msgr2=0x556082e4e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:25.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.741+0000 7fc173b2d740 1 -- 172.21.15.120:0/2813803500 shutdown_connections 2024-09-19T09:04:25.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.741+0000 7fc173b2d740 1 -- 172.21.15.120:0/2813803500 wait complete. 2024-09-19T09:04:25.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.741+0000 7fc173b2d740 20 client.15858 trim_cache size 0 max 16384 2024-09-19T09:04:25.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.754+0000 7fc173b2d740 1 Processor -- start 2024-09-19T09:04:25.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.754+0000 7fc173b2d740 1 -- start start 2024-09-19T09:04:25.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.754+0000 7fc173b2d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556082e4c5d0 0x556082e53900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:25.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.754+0000 7fc173b2d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556082e4db40 0x556082e55fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:25.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.754+0000 7fc173b2d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556082e57370 0x556082e4f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:25.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.754+0000 7fc173b2d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c1ad6b0 con 0x556082e4c5d0 2024-09-19T09:04:25.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.754+0000 7fc173b2d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c1af1f0 con 0x556082e4db40 2024-09-19T09:04:25.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.754+0000 7fc173b2d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c1add80 con 0x556082e57370 2024-09-19T09:04:25.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.754+0000 7fc16affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556082e4c5d0 0x556082e53900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:25.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.755+0000 7fc16affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556082e4c5d0 0x556082e53900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33816/0 (socket says 172.21.15.120:33816) 2024-09-19T09:04:25.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.755+0000 7fc16affd640 1 -- 172.21.15.120:0/2552479508 learned_addr learned my addr 172.21.15.120:0/2552479508 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:25.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.755+0000 7fc163fff640 1 --2- 172.21.15.120:0/2552479508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556082e4db40 0x556082e55fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:25.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.755+0000 7fc16affd640 1 -- 172.21.15.120:0/2552479508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556082e57370 msgr2=0x556082e4f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:25.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.755+0000 7fc16b7fe640 1 --2- 172.21.15.120:0/2552479508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556082e57370 0x556082e4f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:25.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.755+0000 7fc16affd640 1 --2- 172.21.15.120:0/2552479508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556082e57370 0x556082e4f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.755+0000 7fc16affd640 1 -- 172.21.15.120:0/2552479508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556082e4db40 msgr2=0x556082e55fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:25.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.755+0000 7fc16affd640 1 --2- 172.21.15.120:0/2552479508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556082e4db40 0x556082e55fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.755+0000 7fc16affd640 1 -- 172.21.15.120:0/2552479508 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556082e698a0 con 0x556082e4c5d0 2024-09-19T09:04:25.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.755+0000 7fc163fff640 1 --2- 172.21.15.120:0/2552479508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556082e4db40 0x556082e55fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:04:25.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.755+0000 7fc16affd640 1 --2- 172.21.15.120:0/2552479508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556082e4c5d0 0x556082e53900 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc15c02a940 tx=0x7fc15c0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4b523ba4dce328 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:25.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.755+0000 7fc160ff9640 1 -- 172.21.15.120:0/2552479508 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c035040 con 0x556082e4c5d0 2024-09-19T09:04:25.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.755+0000 7fc160ff9640 1 -- 172.21.15.120:0/2552479508 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc15c048070 con 0x556082e4c5d0 2024-09-19T09:04:25.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.755+0000 7fc160ff9640 1 -- 172.21.15.120:0/2552479508 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c0049b0 con 0x556082e4c5d0 2024-09-19T09:04:25.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.756+0000 7fc173b2d740 1 -- 172.21.15.120:0/2552479508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556082e4c5d0 msgr2=0x556082e53900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:25.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.756+0000 7fc173b2d740 1 --2- 172.21.15.120:0/2552479508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556082e4c5d0 0x556082e53900 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc15c02a940 tx=0x7fc15c0058b0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.756+0000 7fc173b2d740 1 -- 172.21.15.120:0/2552479508 shutdown_connections 2024-09-19T09:04:25.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.756+0000 7fc173b2d740 1 --2- 172.21.15.120:0/2552479508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556082e57370 0x556082e4f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.756+0000 7fc173b2d740 1 --2- 172.21.15.120:0/2552479508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556082e4db40 0x556082e55fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.756+0000 7fc173b2d740 1 --2- 172.21.15.120:0/2552479508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556082e4c5d0 0x556082e53900 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.756+0000 7fc173b2d740 1 -- 172.21.15.120:0/2552479508 >> 172.21.15.120:0/2552479508 conn(0x556082e15310 msgr2=0x556082e558a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:25.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.756+0000 7fc173b2d740 1 -- 172.21.15.120:0/2552479508 shutdown_connections 2024-09-19T09:04:25.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.756+0000 7fc173b2d740 1 -- 172.21.15.120:0/2552479508 wait complete. 2024-09-19T09:04:25.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.756+0000 7fc173b2d740 1 -- start start 2024-09-19T09:04:25.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.756+0000 7fc173b2d740 1 Processor -- start 2024-09-19T09:04:25.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.756+0000 7fc173b2d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556082e4c5d0 0x556082e5f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:25.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.756+0000 7fc173b2d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556082e4db40 0x556082e5ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:25.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.756+0000 7fc173b2d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556082e57370 0x556082e5b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:25.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.756+0000 7fc173b2d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c1aeb20 con 0x556082e4db40 2024-09-19T09:04:25.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.756+0000 7fc173b2d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c1ae450 con 0x556082e57370 2024-09-19T09:04:25.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.756+0000 7fc173b2d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c1ab670 con 0x556082e4c5d0 2024-09-19T09:04:25.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.757+0000 7fc173b2d740 10 client.0 _ll_register_callbacks cb 0x7fc154097f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:25.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.757+0000 7fc163fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556082e4db40 0x556082e5ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:25.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.757+0000 7fc163fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556082e4db40 0x556082e5ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33822/0 (socket says 172.21.15.120:33822) 2024-09-19T09:04:25.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.757+0000 7fc163fff640 1 -- 172.21.15.120:0/1363922790 learned_addr learned my addr 172.21.15.120:0/1363922790 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:25.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.757+0000 7fc16b7fe640 1 --2- 172.21.15.120:0/1363922790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556082e57370 0x556082e5b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:25.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.757+0000 7fc16affd640 1 --2- 172.21.15.120:0/1363922790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556082e4c5d0 0x556082e5f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:25.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.757+0000 7fc163fff640 1 -- 172.21.15.120:0/1363922790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556082e4c5d0 msgr2=0x556082e5f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:25.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.757+0000 7fc163fff640 1 --2- 172.21.15.120:0/1363922790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556082e4c5d0 0x556082e5f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.757+0000 7fc163fff640 1 -- 172.21.15.120:0/1363922790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556082e57370 msgr2=0x556082e5b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:25.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.757+0000 7fc163fff640 1 --2- 172.21.15.120:0/1363922790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556082e57370 0x556082e5b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.757+0000 7fc163fff640 1 -- 172.21.15.120:0/1363922790 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556082e5b5d0 con 0x556082e4db40 2024-09-19T09:04:25.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.757+0000 7fc163fff640 1 --2- 172.21.15.120:0/1363922790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556082e4db40 0x556082e5ab50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc154098080 tx=0x7fc154005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b089fb45a38371e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:25.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.757+0000 7fc1527fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:04:25.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.757+0000 7fc1527fc640 1 -- 172.21.15.120:0/1363922790 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc154020070 con 0x556082e4db40 2024-09-19T09:04:25.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.757+0000 7fc1527fc640 1 -- 172.21.15.120:0/1363922790 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fc15409b2c0 con 0x556082e4db40 2024-09-19T09:04:25.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.757+0000 7fc173b2d740 1 -- 172.21.15.120:0/1363922790 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc15c048070 con 0x556082e4db40 2024-09-19T09:04:25.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.757+0000 7fc1527fc640 1 -- 172.21.15.120:0/1363922790 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc15409b5d0 con 0x556082e4db40 2024-09-19T09:04:25.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.757+0000 7fc173b2d740 10 client.15864 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:25.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.757+0000 7fc173b2d740 1 -- 172.21.15.120:0/1363922790 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556082ed1770 con 0x556082e4db40 2024-09-19T09:04:25.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.757+0000 7fc1527fc640 1 -- 172.21.15.120:0/1363922790 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc15c048070 con 0x556082e4db40 2024-09-19T09:04:25.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.758+0000 7fc1527fc640 1 -- 172.21.15.120:0/1363922790 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc15409ad30 con 0x556082e4db40 2024-09-19T09:04:25.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.758+0000 7fc173b2d740 10 client.15864 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:25.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.758+0000 7fc173b2d740 10 client.15864 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:04:25.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.758+0000 7fc173b2d740 20 client.15864 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:25.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.758+0000 7fc173b2d740 1 --2- 172.21.15.120:0/1363922790 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x556082ed1db0 0x556082ed41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:25.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.758+0000 7fc173b2d740 4 client.15864 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:04:25.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.758+0000 7fc173b2d740 1 -- 172.21.15.120:0/1363922790 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556082db9d70 con 0x556082ed1db0 2024-09-19T09:04:25.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.758+0000 7fc16affd640 1 --2- 172.21.15.120:0/1363922790 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x556082ed1db0 0x556082ed41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:25.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.758+0000 7fc16affd640 1 --2- 172.21.15.120:0/1363922790 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x556082ed1db0 0x556082ed41d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b515c788fd698f8a server_cookie=82cf86cf81fb7834 in_seq=0 out_seq=0 2024-09-19T09:04:25.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.758+0000 7fc1527fc640 10 client.15864 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:04:25.785 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:04:25.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.759+0000 7fc1527fc640 1 -- 172.21.15.120:0/1363922790 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556082db9d70 con 0x556082ed1db0 2024-09-19T09:04:25.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.759+0000 7fc1527fc640 10 client.15864 handle_command_reply: tid=0 2024-09-19T09:04:25.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.759+0000 7fc173b2d740 1 client.15864 shutdown 2024-09-19T09:04:25.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.759+0000 7fc173b2d740 1 -- 172.21.15.120:0/1363922790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556082e4db40 msgr2=0x556082e5ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:25.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.759+0000 7fc173b2d740 1 --2- 172.21.15.120:0/1363922790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556082e4db40 0x556082e5ab50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc154098080 tx=0x7fc154005880 comp rx=0 tx=0).stop 2024-09-19T09:04:25.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.759+0000 7fc173b2d740 1 -- 172.21.15.120:0/1363922790 shutdown_connections 2024-09-19T09:04:25.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.759+0000 7fc173b2d740 1 --2- 172.21.15.120:0/1363922790 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x556082ed1db0 0x556082ed41d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.759+0000 7fc173b2d740 1 --2- 172.21.15.120:0/1363922790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556082e57370 0x556082e5b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.759+0000 7fc173b2d740 1 --2- 172.21.15.120:0/1363922790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556082e4db40 0x556082e5ab50 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.759+0000 7fc1527fc640 0 client.15864 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:04:25.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.759+0000 7fc173b2d740 1 --2- 172.21.15.120:0/1363922790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556082e4c5d0 0x556082e5f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.759+0000 7fc173b2d740 1 -- 172.21.15.120:0/1363922790 >> 172.21.15.120:0/1363922790 conn(0x556082e15310 msgr2=0x556082e622a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:25.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.759+0000 7fc173b2d740 1 -- 172.21.15.120:0/1363922790 shutdown_connections 2024-09-19T09:04:25.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.759+0000 7fc173b2d740 1 -- 172.21.15.120:0/1363922790 wait complete. 2024-09-19T09:04:25.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.759+0000 7fc173b2d740 20 client.15864 trim_cache size 0 max 16384 2024-09-19T09:04:25.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.760+0000 7fc171754640 1 -- 172.21.15.120:0/1073660762 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc1580a3240 msgr2=0x7fc1580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:25.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.760+0000 7fc171754640 1 --2- 172.21.15.120:0/1073660762 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc1580a3240 0x7fc1580a5700 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc164009fd0 tx=0x7fc164016000 comp rx=0 tx=0).stop 2024-09-19T09:04:25.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.760+0000 7fc171754640 1 -- 172.21.15.120:0/1073660762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc16c101420 msgr2=0x7fc16c1989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:25.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.760+0000 7fc171754640 1 --2- 172.21.15.120:0/1073660762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc16c101420 0x7fc16c1989a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc15400d950 tx=0x7fc15400de10 comp rx=0 tx=0).stop 2024-09-19T09:04:25.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.760+0000 7fc171754640 1 -- 172.21.15.120:0/1073660762 shutdown_connections 2024-09-19T09:04:25.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.760+0000 7fc171754640 1 --2- 172.21.15.120:0/1073660762 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc1580a3240 0x7fc1580a5700 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.760+0000 7fc171754640 1 --2- 172.21.15.120:0/1073660762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc16c106a80 0x7fc16c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.760+0000 7fc171754640 1 --2- 172.21.15.120:0/1073660762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc16c101420 0x7fc16c1989a0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.760+0000 7fc171754640 1 --2- 172.21.15.120:0/1073660762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc16c100a80 0x7fc16c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:25.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.760+0000 7fc171754640 1 -- 172.21.15.120:0/1073660762 >> 172.21.15.120:0/1073660762 conn(0x7fc16c0fc7a0 msgr2=0x7fc16c06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:25.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.760+0000 7fc171754640 1 -- 172.21.15.120:0/1073660762 shutdown_connections 2024-09-19T09:04:25.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:25.760+0000 7fc171754640 1 -- 172.21.15.120:0/1073660762 wait complete. 2024-09-19T09:04:25.951 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:04:26.035 INFO:tasks.workunit.client.0.smithi120.stdout: 1 11836 3.83 MB/sec warmup 21 sec latency 17499.478 ms 2024-09-19T09:04:26.475 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:04:26.533 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:04:26.929 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:26 smithi120 ceph-mon[31150]: pgmap v388: 97 pgs: 97 active+clean; 26 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.3 MiB/s wr, 40 op/s 2024-09-19T09:04:27.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.008+0000 7fc8faebd640 1 -- 172.21.15.120:0/2719668336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8f4103700 msgr2=0x7fc8f410bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:27.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.008+0000 7fc8faebd640 1 --2- 172.21.15.120:0/2719668336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8f4103700 0x7fc8f410bbf0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009900 tx=0x7fc8e002a480 comp rx=0 tx=0).stop 2024-09-19T09:04:27.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.008+0000 7fc8faebd640 1 -- 172.21.15.120:0/2719668336 shutdown_connections 2024-09-19T09:04:27.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.008+0000 7fc8faebd640 1 --2- 172.21.15.120:0/2719668336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8f410c180 0x7fc8f410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:27.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.008+0000 7fc8faebd640 1 --2- 172.21.15.120:0/2719668336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8f4103700 0x7fc8f410bbf0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:27.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.008+0000 7fc8faebd640 1 --2- 172.21.15.120:0/2719668336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8f4102de0 0x7fc8f41031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:27.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.008+0000 7fc8faebd640 1 -- 172.21.15.120:0/2719668336 >> 172.21.15.120:0/2719668336 conn(0x7fc8f40fc7a0 msgr2=0x7fc8f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:27.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.008+0000 7fc8faebd640 1 -- 172.21.15.120:0/2719668336 shutdown_connections 2024-09-19T09:04:27.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.009+0000 7fc8faebd640 1 -- 172.21.15.120:0/2719668336 wait complete. 2024-09-19T09:04:27.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.009+0000 7fc8faebd640 1 Processor -- start 2024-09-19T09:04:27.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.009+0000 7fc8faebd640 1 -- start start 2024-09-19T09:04:27.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.010+0000 7fc8faebd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8f4102de0 0x7fc8f41984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:27.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.010+0000 7fc8faebd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8f4103700 0x7fc8f41989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:27.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.010+0000 7fc8faebd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8f410c180 0x7fc8f419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:27.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.010+0000 7fc8faebd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f4111f70 con 0x7fc8f4103700 2024-09-19T09:04:27.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.010+0000 7fc8faebd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f4111df0 con 0x7fc8f410c180 2024-09-19T09:04:27.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.010+0000 7fc8faebd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f41120f0 con 0x7fc8f4102de0 2024-09-19T09:04:27.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.010+0000 7fc8f3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8f4103700 0x7fc8f41989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:27.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.010+0000 7fc8f3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8f4103700 0x7fc8f41989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33826/0 (socket says 172.21.15.120:33826) 2024-09-19T09:04:27.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.010+0000 7fc8f3fff640 1 -- 172.21.15.120:0/43151373 learned_addr learned my addr 172.21.15.120:0/43151373 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:27.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.010+0000 7fc8f945a640 1 --2- 172.21.15.120:0/43151373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8f410c180 0x7fc8f419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:27.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.010+0000 7fc8f8c59640 1 --2- 172.21.15.120:0/43151373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8f4102de0 0x7fc8f41984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:27.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.010+0000 7fc8f3fff640 1 -- 172.21.15.120:0/43151373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8f4102de0 msgr2=0x7fc8f41984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:27.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.010+0000 7fc8f3fff640 1 --2- 172.21.15.120:0/43151373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8f4102de0 0x7fc8f41984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:27.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.010+0000 7fc8f3fff640 1 -- 172.21.15.120:0/43151373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8f410c180 msgr2=0x7fc8f419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:27.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.010+0000 7fc8f3fff640 1 --2- 172.21.15.120:0/43151373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8f410c180 0x7fc8f419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:27.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.010+0000 7fc8f3fff640 1 -- 172.21.15.120:0/43151373 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f419d460 con 0x7fc8f4103700 2024-09-19T09:04:27.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.010+0000 7fc8f3fff640 1 --2- 172.21.15.120:0/43151373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8f4103700 0x7fc8f41989f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0006010 tx=0x7fc8e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:27.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.010+0000 7fc8f1ffb640 1 -- 172.21.15.120:0/43151373 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e003f030 con 0x7fc8f4103700 2024-09-19T09:04:27.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.010+0000 7fc8f1ffb640 1 -- 172.21.15.120:0/43151373 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8e0003ce0 con 0x7fc8f4103700 2024-09-19T09:04:27.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.010+0000 7fc8f1ffb640 1 -- 172.21.15.120:0/43151373 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0035070 con 0x7fc8f4103700 2024-09-19T09:04:27.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.010+0000 7fc8faebd640 1 -- 172.21.15.120:0/43151373 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f419d6f0 con 0x7fc8f4103700 2024-09-19T09:04:27.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.011+0000 7fc8faebd640 1 -- 172.21.15.120:0/43151373 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f41a5030 con 0x7fc8f4103700 2024-09-19T09:04:27.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.012+0000 7fc8faebd640 1 -- 172.21.15.120:0/43151373 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8f41031c0 con 0x7fc8f4103700 2024-09-19T09:04:27.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.013+0000 7fc8f1ffb640 1 -- 172.21.15.120:0/43151373 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fc8e0042030 con 0x7fc8f4103700 2024-09-19T09:04:27.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.013+0000 7fc8f1ffb640 1 --2- 172.21.15.120:0/43151373 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc8e40a32c0 0x7fc8e40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:27.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.013+0000 7fc8f8c59640 1 --2- 172.21.15.120:0/43151373 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc8e40a32c0 0x7fc8e40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:27.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.013+0000 7fc8f1ffb640 1 -- 172.21.15.120:0/43151373 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fc8e00efdf0 con 0x7fc8f4103700 2024-09-19T09:04:27.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.014+0000 7fc8f8c59640 1 --2- 172.21.15.120:0/43151373 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc8e40a32c0 0x7fc8e40a5780 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800bbe0 tx=0x7fc8e800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:27.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.017+0000 7fc8f1ffb640 1 -- 172.21.15.120:0/43151373 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc8e00bc580 con 0x7fc8f4103700 2024-09-19T09:04:27.035 INFO:tasks.workunit.client.0.smithi120.stdout: 1 11836 3.65 MB/sec warmup 22 sec latency 18499.507 ms 2024-09-19T09:04:27.065 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:04:27.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:26 smithi196 ceph-mon[35150]: pgmap v388: 97 pgs: 97 active+clean; 26 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.3 MiB/s wr, 40 op/s 2024-09-19T09:04:27.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:26 smithi153 ceph-mon[35071]: pgmap v388: 97 pgs: 97 active+clean; 26 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.3 MiB/s wr, 40 op/s 2024-09-19T09:04:27.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.231+0000 7fc8faebd640 1 -- 172.21.15.120:0/43151373 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8f4199810 con 0x7fc8f4103700 2024-09-19T09:04:27.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.231+0000 7fc8f1ffb640 1 -- 172.21.15.120:0/43151373 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7fc8f4199810 con 0x7fc8f4103700 2024-09-19T09:04:27.232 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:04:27.232 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":28,"btime":"2024-09-19T09:02:41:696728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:02:41.207357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:04:27.232 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 28 2024-09-19T09:04:27.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.234+0000 7fc8faebd640 1 -- 172.21.15.120:0/43151373 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc8e40a32c0 msgr2=0x7fc8e40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:27.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.234+0000 7fc8faebd640 1 --2- 172.21.15.120:0/43151373 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc8e40a32c0 0x7fc8e40a5780 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800bbe0 tx=0x7fc8e800b390 comp rx=0 tx=0).stop 2024-09-19T09:04:27.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.234+0000 7fc8faebd640 1 -- 172.21.15.120:0/43151373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8f4103700 msgr2=0x7fc8f41989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:27.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.234+0000 7fc8faebd640 1 --2- 172.21.15.120:0/43151373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8f4103700 0x7fc8f41989f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0006010 tx=0x7fc8e00042f0 comp rx=0 tx=0).stop 2024-09-19T09:04:27.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.234+0000 7fc8faebd640 1 -- 172.21.15.120:0/43151373 shutdown_connections 2024-09-19T09:04:27.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.234+0000 7fc8faebd640 1 --2- 172.21.15.120:0/43151373 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc8e40a32c0 0x7fc8e40a5780 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:27.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.234+0000 7fc8faebd640 1 --2- 172.21.15.120:0/43151373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8f410c180 0x7fc8f419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:27.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.234+0000 7fc8faebd640 1 --2- 172.21.15.120:0/43151373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8f4103700 0x7fc8f41989f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:27.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.234+0000 7fc8faebd640 1 --2- 172.21.15.120:0/43151373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8f4102de0 0x7fc8f41984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:27.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.234+0000 7fc8faebd640 1 -- 172.21.15.120:0/43151373 >> 172.21.15.120:0/43151373 conn(0x7fc8f40fc7a0 msgr2=0x7fc8f40fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:27.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.234+0000 7fc8faebd640 1 -- 172.21.15.120:0/43151373 shutdown_connections 2024-09-19T09:04:27.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.234+0000 7fc8faebd640 1 -- 172.21.15.120:0/43151373 wait complete. 2024-09-19T09:04:27.870 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:27 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/43151373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:04:28.035 INFO:tasks.workunit.client.0.smithi120.stdout: 1 11836 3.49 MB/sec warmup 23 sec latency 19499.549 ms 2024-09-19T09:04:28.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:27 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/43151373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:04:28.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:27 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/43151373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:04:28.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.955+0000 7f1239cdf640 1 -- 172.21.15.120:0/2357240929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1234101de0 msgr2=0x7f12341021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:28.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.955+0000 7f1239cdf640 1 --2- 172.21.15.120:0/2357240929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1234101de0 0x7f12341021c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1224009900 tx=0x7f122402a410 comp rx=0 tx=0).stop 2024-09-19T09:04:28.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.955+0000 7f1239cdf640 1 -- 172.21.15.120:0/2357240929 shutdown_connections 2024-09-19T09:04:28.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.955+0000 7f1239cdf640 1 --2- 172.21.15.120:0/2357240929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1234106f10 0x7f123410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.955+0000 7f1239cdf640 1 --2- 172.21.15.120:0/2357240929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1234102790 0x7f12341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.955+0000 7f1239cdf640 1 --2- 172.21.15.120:0/2357240929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1234101de0 0x7f12341021c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.955+0000 7f1239cdf640 1 -- 172.21.15.120:0/2357240929 >> 172.21.15.120:0/2357240929 conn(0x7f12340fd8c0 msgr2=0x7f12340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:28.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.955+0000 7f1239cdf640 1 -- 172.21.15.120:0/2357240929 shutdown_connections 2024-09-19T09:04:28.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.955+0000 7f1239cdf640 1 -- 172.21.15.120:0/2357240929 wait complete. 2024-09-19T09:04:28.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.956+0000 7f1239cdf640 1 Processor -- start 2024-09-19T09:04:28.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.956+0000 7f1239cdf640 1 -- start start 2024-09-19T09:04:28.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.956+0000 7f1239cdf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1234101de0 0x7f12340744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:28.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.956+0000 7f1239cdf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1234102790 0x7f1234074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:28.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.956+0000 7f1239cdf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1234106f10 0x7f1234074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:28.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.956+0000 7f1239cdf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1234112010 con 0x7f1234101de0 2024-09-19T09:04:28.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.956+0000 7f1239cdf640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1234111e90 con 0x7f1234102790 2024-09-19T09:04:28.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.956+0000 7f1239cdf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1234112190 con 0x7f1234106f10 2024-09-19T09:04:28.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.956+0000 7f12337fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1234101de0 0x7f12340744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:28.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.956+0000 7f12337fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1234101de0 0x7f12340744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33842/0 (socket says 172.21.15.120:33842) 2024-09-19T09:04:28.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.956+0000 7f12337fe640 1 -- 172.21.15.120:0/3222452605 learned_addr learned my addr 172.21.15.120:0/3222452605 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:28.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.956+0000 7f122bfff640 1 --2- 172.21.15.120:0/3222452605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1234102790 0x7f1234074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:28.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.956+0000 7f12337fe640 1 -- 172.21.15.120:0/3222452605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1234106f10 msgr2=0x7f1234074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:28.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.956+0000 7f12337fe640 1 --2- 172.21.15.120:0/3222452605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1234106f10 0x7f1234074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.956+0000 7f12337fe640 1 -- 172.21.15.120:0/3222452605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1234102790 msgr2=0x7f1234074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:28.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.956+0000 7f12337fe640 1 --2- 172.21.15.120:0/3222452605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1234102790 0x7f1234074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.956+0000 7f12337fe640 1 -- 172.21.15.120:0/3222452605 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12341a6a70 con 0x7f1234101de0 2024-09-19T09:04:28.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.956+0000 7f122bfff640 1 --2- 172.21.15.120:0/3222452605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1234102790 0x7f1234074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:04:28.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.957+0000 7f12337fe640 1 --2- 172.21.15.120:0/3222452605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1234101de0 0x7f12340744f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1224005ab0 tx=0x7f12240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:28.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.957+0000 7f12317fa640 1 -- 172.21.15.120:0/3222452605 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122403f030 con 0x7f1234101de0 2024-09-19T09:04:28.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.957+0000 7f1239cdf640 1 -- 172.21.15.120:0/3222452605 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12341a6d00 con 0x7f1234101de0 2024-09-19T09:04:28.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.957+0000 7f12317fa640 1 -- 172.21.15.120:0/3222452605 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1224003ce0 con 0x7f1234101de0 2024-09-19T09:04:28.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.957+0000 7f12317fa640 1 -- 172.21.15.120:0/3222452605 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224035070 con 0x7f1234101de0 2024-09-19T09:04:28.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.957+0000 7f1239cdf640 1 -- 172.21.15.120:0/3222452605 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12341a7190 con 0x7f1234101de0 2024-09-19T09:04:28.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.959+0000 7f12317fa640 1 -- 172.21.15.120:0/3222452605 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f1224042030 con 0x7f1234101de0 2024-09-19T09:04:28.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.959+0000 7f123c0b8740 1 Processor -- start 2024-09-19T09:04:28.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.960+0000 7f123c0b8740 1 -- start start 2024-09-19T09:04:28.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.960+0000 7f12317fa640 1 --2- 172.21.15.120:0/3222452605 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f12200a3240 0x7f12200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:28.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.960+0000 7f123c0b8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643fb090b40 0x5643fb08f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:28.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.960+0000 7f123c0b8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fb08f5d0 0x5643fb08fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:28.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.960+0000 7f123c0b8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643fb09a370 0x5643fb09c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:28.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.960+0000 7f123c0b8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5643faff52f0 con 0x5643fb09a370 2024-09-19T09:04:28.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.960+0000 7f123c0b8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5643faf9ccf0 con 0x5643fb08f5d0 2024-09-19T09:04:28.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.960+0000 7f123c0b8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5643faff12b0 con 0x5643fb090b40 2024-09-19T09:04:28.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.960+0000 7f12317fa640 1 -- 172.21.15.120:0/3222452605 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f12240f0310 con 0x7f1234101de0 2024-09-19T09:04:28.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.960+0000 7f1233fff640 1 --2- 172.21.15.120:0/3222452605 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f12200a3240 0x7f12200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:28.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.960+0000 7f1233fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643fb09a370 0x5643fb09c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:28.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.960+0000 7f122bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fb08f5d0 0x5643fb08fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:28.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.960+0000 7f1233fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643fb09a370 0x5643fb09c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33848/0 (socket says 172.21.15.120:33848) 2024-09-19T09:04:28.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.960+0000 7f122bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fb08f5d0 0x5643fb08fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38090/0 (socket says 172.21.15.120:38090) 2024-09-19T09:04:28.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.960+0000 7f1233fff640 1 -- 172.21.15.120:0/2304818634 learned_addr learned my addr 172.21.15.120:0/2304818634 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:28.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.960+0000 7f12337fe640 1 --2- 172.21.15.120:0/2304818634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643fb090b40 0x5643fb08f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:28.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.960+0000 7f1233fff640 1 -- 172.21.15.120:0/2304818634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643fb090b40 msgr2=0x5643fb08f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:28.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.960+0000 7f1233fff640 1 --2- 172.21.15.120:0/2304818634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643fb090b40 0x5643fb08f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.960+0000 7f1233fff640 1 -- 172.21.15.120:0/2304818634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fb08f5d0 msgr2=0x5643fb08fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:28.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.960+0000 7f1233fff640 1 --2- 172.21.15.120:0/2304818634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fb08f5d0 0x5643fb08fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.960+0000 7f1233fff640 1 -- 172.21.15.120:0/2304818634 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643faf813a0 con 0x5643fb09a370 2024-09-19T09:04:28.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.960+0000 7f122bfff640 1 --2- 172.21.15.120:0/2304818634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fb08f5d0 0x5643fb08fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:04:28.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.961+0000 7f1233fff640 1 --2- 172.21.15.120:0/3222452605 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f12200a3240 0x7f12200a5700 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f122c011800 tx=0x7f122c00ca00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:28.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.961+0000 7f1233fff640 1 --2- 172.21.15.120:0/2304818634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643fb09a370 0x5643fb09c760 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f122c009040 tx=0x7f122c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d621e769bf3b8ec7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:28.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.961+0000 7f1229ffb640 1 -- 172.21.15.120:0/2304818634 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c01b8e0 con 0x5643fb09a370 2024-09-19T09:04:28.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.961+0000 7f1229ffb640 1 -- 172.21.15.120:0/2304818634 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f122c023070 con 0x5643fb09a370 2024-09-19T09:04:28.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.961+0000 7f1229ffb640 1 -- 172.21.15.120:0/2304818634 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c006770 con 0x5643fb09a370 2024-09-19T09:04:28.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.961+0000 7f123c0b8740 1 -- 172.21.15.120:0/2304818634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643fb09a370 msgr2=0x5643fb09c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:28.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.961+0000 7f123c0b8740 1 --2- 172.21.15.120:0/2304818634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643fb09a370 0x5643fb09c760 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f122c009040 tx=0x7f122c018000 comp rx=0 tx=0).stop 2024-09-19T09:04:28.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.962+0000 7f123c0b8740 1 -- 172.21.15.120:0/2304818634 shutdown_connections 2024-09-19T09:04:28.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.962+0000 7f123c0b8740 1 --2- 172.21.15.120:0/2304818634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643fb09a370 0x5643fb09c760 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.962+0000 7f123c0b8740 1 --2- 172.21.15.120:0/2304818634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fb08f5d0 0x5643fb08fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.962+0000 7f123c0b8740 1 --2- 172.21.15.120:0/2304818634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643fb090b40 0x5643fb08f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.962+0000 7f123c0b8740 1 -- 172.21.15.120:0/2304818634 >> 172.21.15.120:0/2304818634 conn(0x5643fb058310 msgr2=0x5643fb08bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:28.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.962+0000 7f123c0b8740 1 -- 172.21.15.120:0/2304818634 shutdown_connections 2024-09-19T09:04:28.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.962+0000 7f123c0b8740 1 -- 172.21.15.120:0/2304818634 wait complete. 2024-09-19T09:04:28.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.962+0000 7f123c0b8740 1 -- start start 2024-09-19T09:04:28.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.962+0000 7f123c0b8740 1 Processor -- start 2024-09-19T09:04:28.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.962+0000 7f123c0b8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fb08f5d0 0x5643fb099d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:28.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.962+0000 7f123c0b8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643fb090b40 0x5643fb097000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:28.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.962+0000 7f123c0b8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643fb09a370 0x5643fb0975d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:28.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.962+0000 7f123c0b8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5643fb09d010 con 0x5643fb09a370 2024-09-19T09:04:28.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.962+0000 7f123c0b8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5643fafb0060 con 0x5643fb08f5d0 2024-09-19T09:04:28.595 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:04:28.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.962+0000 7f123c0b8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5643fb09d190 con 0x5643fb090b40 2024-09-19T09:04:28.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.962+0000 7f123c0b8740 10 client.0 _ll_register_callbacks cb 0x5643fad038c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:28.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.963+0000 7f1233fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643fb09a370 0x5643fb0975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:28.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.963+0000 7f1233fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643fb09a370 0x5643fb0975d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33858/0 (socket says 172.21.15.120:33858) 2024-09-19T09:04:28.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.963+0000 7f1233fff640 1 -- 172.21.15.120:0/596722148 learned_addr learned my addr 172.21.15.120:0/596722148 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:28.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.963+0000 7f122bfff640 1 --2- 172.21.15.120:0/596722148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643fb090b40 0x5643fb097000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:28.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.963+0000 7f12337fe640 1 --2- 172.21.15.120:0/596722148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fb08f5d0 0x5643fb099d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:28.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.963+0000 7f1233fff640 1 -- 172.21.15.120:0/596722148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643fb090b40 msgr2=0x5643fb097000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:28.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.963+0000 7f1233fff640 1 --2- 172.21.15.120:0/596722148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643fb090b40 0x5643fb097000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.963+0000 7f1233fff640 1 -- 172.21.15.120:0/596722148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fb08f5d0 msgr2=0x5643fb099d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:28.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.963+0000 7f1233fff640 1 --2- 172.21.15.120:0/596722148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fb08f5d0 0x5643fb099d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.963+0000 7f1233fff640 1 -- 172.21.15.120:0/596722148 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643fad663c0 con 0x5643fb09a370 2024-09-19T09:04:28.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.963+0000 7f12337fe640 1 --2- 172.21.15.120:0/596722148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fb08f5d0 0x5643fb099d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:04:28.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.963+0000 7f1233fff640 1 --2- 172.21.15.120:0/596722148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643fb09a370 0x5643fb0975d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f122c012010 tx=0x7f122c011550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886f77fc7aa13bfb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:28.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.963+0000 7f1212ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:04:28.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.963+0000 7f1212ffd640 1 -- 172.21.15.120:0/596722148 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c02e070 con 0x5643fb09a370 2024-09-19T09:04:28.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.963+0000 7f1212ffd640 1 -- 172.21.15.120:0/596722148 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f122c0068c0 con 0x5643fb09a370 2024-09-19T09:04:28.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.963+0000 7f123c0b8740 1 -- 172.21.15.120:0/596722148 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f122c023070 con 0x5643fb09a370 2024-09-19T09:04:28.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.963+0000 7f1212ffd640 1 -- 172.21.15.120:0/596722148 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f122c00c710 con 0x5643fb09a370 2024-09-19T09:04:28.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.963+0000 7f123c0b8740 10 client.15882 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:28.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.963+0000 7f123c0b8740 1 -- 172.21.15.120:0/596722148 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5643fb0a8b10 con 0x5643fb09a370 2024-09-19T09:04:28.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.964+0000 7f1212ffd640 1 -- 172.21.15.120:0/596722148 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f122c023070 con 0x5643fb09a370 2024-09-19T09:04:28.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.964+0000 7f1212ffd640 1 -- 172.21.15.120:0/596722148 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f122c01e510 con 0x5643fb09a370 2024-09-19T09:04:28.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.964+0000 7f123c0b8740 10 client.15882 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:28.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.964+0000 7f123c0b8740 10 client.15882 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:04:28.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.964+0000 7f123c0b8740 20 client.15882 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:28.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.964+0000 7f123c0b8740 1 --2- 172.21.15.120:0/596722148 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5643fb0a9ec0 0x5643fb0ac2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:28.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.964+0000 7f123c0b8740 4 client.15882 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:04:28.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.964+0000 7f123c0b8740 1 -- 172.21.15.120:0/596722148 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643fb002860 con 0x5643fb0a9ec0 2024-09-19T09:04:28.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.964+0000 7f12337fe640 1 --2- 172.21.15.120:0/596722148 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5643fb0a9ec0 0x5643fb0ac2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:28.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.964+0000 7f1212ffd640 10 client.15882 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:04:28.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.964+0000 7f12337fe640 1 --2- 172.21.15.120:0/596722148 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5643fb0a9ec0 0x5643fb0ac2b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ddf9d5a802902ebe server_cookie=1e4500ea8f6ae5a2 in_seq=0 out_seq=0 2024-09-19T09:04:28.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.965+0000 7f1212ffd640 1 -- 172.21.15.120:0/596722148 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5643fb002860 con 0x5643fb0a9ec0 2024-09-19T09:04:28.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.965+0000 7f1212ffd640 10 client.15882 handle_command_reply: tid=0 2024-09-19T09:04:28.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.965+0000 7f123c0b8740 1 client.15882 shutdown 2024-09-19T09:04:28.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.965+0000 7f123c0b8740 1 -- 172.21.15.120:0/596722148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643fb09a370 msgr2=0x5643fb0975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:28.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.965+0000 7f123c0b8740 1 --2- 172.21.15.120:0/596722148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643fb09a370 0x5643fb0975d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f122c012010 tx=0x7f122c011550 comp rx=0 tx=0).stop 2024-09-19T09:04:28.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.965+0000 7f123c0b8740 1 -- 172.21.15.120:0/596722148 shutdown_connections 2024-09-19T09:04:28.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.965+0000 7f123c0b8740 1 --2- 172.21.15.120:0/596722148 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5643fb0a9ec0 0x5643fb0ac2b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.965+0000 7f123c0b8740 1 --2- 172.21.15.120:0/596722148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643fb09a370 0x5643fb0975d0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.965+0000 7f123c0b8740 1 --2- 172.21.15.120:0/596722148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643fb090b40 0x5643fb097000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.965+0000 7f1212ffd640 0 client.15882 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:04:28.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.965+0000 7f123c0b8740 1 --2- 172.21.15.120:0/596722148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fb08f5d0 0x5643fb099d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.965+0000 7f123c0b8740 1 -- 172.21.15.120:0/596722148 >> 172.21.15.120:0/596722148 conn(0x5643fb058310 msgr2=0x5643fb091960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:28.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.965+0000 7f123c0b8740 1 -- 172.21.15.120:0/596722148 shutdown_connections 2024-09-19T09:04:28.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.965+0000 7f123c0b8740 1 -- 172.21.15.120:0/596722148 wait complete. 2024-09-19T09:04:28.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.966+0000 7f123c0b8740 20 client.15882 trim_cache size 0 max 16384 2024-09-19T09:04:28.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.978+0000 7f123c0b8740 1 Processor -- start 2024-09-19T09:04:28.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.978+0000 7f123c0b8740 1 -- start start 2024-09-19T09:04:28.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.978+0000 7f123c0b8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fb08f5d0 0x5643fb0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:28.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.978+0000 7f123c0b8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643fb090b40 0x5643fb08b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:28.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.978+0000 7f123c0b8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643fb09a370 0x5643fb08bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:28.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.978+0000 7f123c0b8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f12341af900 con 0x5643fb090b40 2024-09-19T09:04:28.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.978+0000 7f123c0b8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f12341b1440 con 0x5643fb08f5d0 2024-09-19T09:04:28.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.978+0000 7f123c0b8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f12341affd0 con 0x5643fb09a370 2024-09-19T09:04:28.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.979+0000 7f122bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643fb090b40 0x5643fb08b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:28.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.979+0000 7f122bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643fb090b40 0x5643fb08b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33866/0 (socket says 172.21.15.120:33866) 2024-09-19T09:04:28.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.979+0000 7f122bfff640 1 -- 172.21.15.120:0/3925092829 learned_addr learned my addr 172.21.15.120:0/3925092829 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:28.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.979+0000 7f12337fe640 1 --2- 172.21.15.120:0/3925092829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fb08f5d0 0x5643fb0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:28.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.979+0000 7f122bfff640 1 -- 172.21.15.120:0/3925092829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643fb09a370 msgr2=0x5643fb08bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:28.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.979+0000 7f122bfff640 1 --2- 172.21.15.120:0/3925092829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643fb09a370 0x5643fb08bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.979+0000 7f122bfff640 1 -- 172.21.15.120:0/3925092829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fb08f5d0 msgr2=0x5643fb0a6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:28.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.979+0000 7f122bfff640 1 --2- 172.21.15.120:0/3925092829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fb08f5d0 0x5643fb0a6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.979+0000 7f122bfff640 1 -- 172.21.15.120:0/3925092829 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643fb08c180 con 0x5643fb090b40 2024-09-19T09:04:28.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.979+0000 7f122bfff640 1 --2- 172.21.15.120:0/3925092829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643fb090b40 0x5643fb08b700 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f121c00aa00 tx=0x7f121c00aec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9846990624f4759 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:28.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.979+0000 7f12297fa640 1 -- 172.21.15.120:0/3925092829 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c012dc0 con 0x5643fb090b40 2024-09-19T09:04:28.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.979+0000 7f12297fa640 1 -- 172.21.15.120:0/3925092829 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f121c01c900 con 0x5643fb090b40 2024-09-19T09:04:28.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.979+0000 7f12297fa640 1 -- 172.21.15.120:0/3925092829 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c0149b0 con 0x5643fb090b40 2024-09-19T09:04:28.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.980+0000 7f123c0b8740 1 -- 172.21.15.120:0/3925092829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643fb090b40 msgr2=0x5643fb08b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:28.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.980+0000 7f123c0b8740 1 --2- 172.21.15.120:0/3925092829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643fb090b40 0x5643fb08b700 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f121c00aa00 tx=0x7f121c00aec0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.980+0000 7f123c0b8740 1 -- 172.21.15.120:0/3925092829 shutdown_connections 2024-09-19T09:04:28.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.980+0000 7f123c0b8740 1 --2- 172.21.15.120:0/3925092829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643fb09a370 0x5643fb08bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.980+0000 7f123c0b8740 1 --2- 172.21.15.120:0/3925092829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643fb090b40 0x5643fb08b700 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.980+0000 7f123c0b8740 1 --2- 172.21.15.120:0/3925092829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fb08f5d0 0x5643fb0a6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.980+0000 7f123c0b8740 1 -- 172.21.15.120:0/3925092829 >> 172.21.15.120:0/3925092829 conn(0x5643fb058310 msgr2=0x5643fb092260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:28.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.980+0000 7f123c0b8740 1 -- 172.21.15.120:0/3925092829 shutdown_connections 2024-09-19T09:04:28.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.980+0000 7f123c0b8740 1 -- 172.21.15.120:0/3925092829 wait complete. 2024-09-19T09:04:28.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.980+0000 7f123c0b8740 1 -- start start 2024-09-19T09:04:28.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.980+0000 7f123c0b8740 1 Processor -- start 2024-09-19T09:04:28.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.980+0000 7f123c0b8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fb08f5d0 0x5643fb0a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:28.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.980+0000 7f123c0b8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643fb090b40 0x5643fb093c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:28.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.980+0000 7f123c0b8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643fb09a370 0x5643fb0941c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:28.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.980+0000 7f123c0b8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f12341b0d70 con 0x5643fb09a370 2024-09-19T09:04:28.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.980+0000 7f123c0b8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f12341b06a0 con 0x5643fb08f5d0 2024-09-19T09:04:28.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.980+0000 7f123c0b8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f12341ad8c0 con 0x5643fb090b40 2024-09-19T09:04:28.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.981+0000 7f123c0b8740 10 client.0 _ll_register_callbacks cb 0x7f121c00a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:28.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.981+0000 7f1233fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643fb09a370 0x5643fb0941c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:28.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.981+0000 7f1233fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643fb09a370 0x5643fb0941c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33872/0 (socket says 172.21.15.120:33872) 2024-09-19T09:04:28.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.981+0000 7f1233fff640 1 -- 172.21.15.120:0/3081947189 learned_addr learned my addr 172.21.15.120:0/3081947189 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:28.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.981+0000 7f122bfff640 1 --2- 172.21.15.120:0/3081947189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643fb090b40 0x5643fb093c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:28.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.981+0000 7f12337fe640 1 --2- 172.21.15.120:0/3081947189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fb08f5d0 0x5643fb0a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:28.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.981+0000 7f1233fff640 1 -- 172.21.15.120:0/3081947189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643fb090b40 msgr2=0x5643fb093c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:28.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.981+0000 7f1233fff640 1 --2- 172.21.15.120:0/3081947189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643fb090b40 0x5643fb093c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.981+0000 7f1233fff640 1 -- 172.21.15.120:0/3081947189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fb08f5d0 msgr2=0x5643fb0a2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:28.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.981+0000 7f1233fff640 1 --2- 172.21.15.120:0/3081947189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fb08f5d0 0x5643fb0a2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.981+0000 7f1233fff640 1 -- 172.21.15.120:0/3081947189 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643fb0949a0 con 0x5643fb09a370 2024-09-19T09:04:28.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.981+0000 7f12337fe640 1 --2- 172.21.15.120:0/3081947189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fb08f5d0 0x5643fb0a2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:04:28.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.981+0000 7f122bfff640 1 --2- 172.21.15.120:0/3081947189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643fb090b40 0x5643fb093c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:04:28.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.981+0000 7f1233fff640 1 --2- 172.21.15.120:0/3081947189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643fb09a370 0x5643fb0941c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f122c004be0 tx=0x7f122c00e880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7016b06804aa75 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:28.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.981+0000 7f1212ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:04:28.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.981+0000 7f1212ffd640 1 -- 172.21.15.120:0/3081947189 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c02e040 con 0x5643fb09a370 2024-09-19T09:04:28.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.981+0000 7f1212ffd640 1 -- 172.21.15.120:0/3081947189 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f122c01e680 con 0x5643fb09a370 2024-09-19T09:04:28.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.981+0000 7f123c0b8740 1 -- 172.21.15.120:0/3081947189 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f121c01c900 con 0x5643fb09a370 2024-09-19T09:04:28.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.981+0000 7f1212ffd640 1 -- 172.21.15.120:0/3081947189 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f122c044050 con 0x5643fb09a370 2024-09-19T09:04:28.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.981+0000 7f123c0b8740 10 client.15888 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:28.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.981+0000 7f123c0b8740 1 -- 172.21.15.120:0/3081947189 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5643fb1147f0 con 0x5643fb09a370 2024-09-19T09:04:28.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.981+0000 7f1212ffd640 1 -- 172.21.15.120:0/3081947189 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f121c01c900 con 0x5643fb09a370 2024-09-19T09:04:28.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.982+0000 7f1212ffd640 1 -- 172.21.15.120:0/3081947189 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f122c01bba0 con 0x5643fb09a370 2024-09-19T09:04:28.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.982+0000 7f123c0b8740 10 client.15888 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:28.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.982+0000 7f123c0b8740 10 client.15888 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:04:28.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.982+0000 7f123c0b8740 20 client.15888 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:28.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.982+0000 7f123c0b8740 1 --2- 172.21.15.120:0/3081947189 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5643fb114e80 0x5643fb1172a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:28.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.982+0000 7f123c0b8740 4 client.15888 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:04:28.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.982+0000 7f123c0b8740 1 -- 172.21.15.120:0/3081947189 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5643faffcd70 con 0x5643fb114e80 2024-09-19T09:04:28.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.982+0000 7f12337fe640 1 --2- 172.21.15.120:0/3081947189 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5643fb114e80 0x5643fb1172a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:28.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.982+0000 7f1212ffd640 10 client.15888 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:04:28.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.982+0000 7f12337fe640 1 --2- 172.21.15.120:0/3081947189 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5643fb114e80 0x5643fb1172a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8abb42d2ed55a3d7 server_cookie=9453b1d2b432c3af in_seq=0 out_seq=0 2024-09-19T09:04:28.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.982+0000 7f1212ffd640 1 -- 172.21.15.120:0/3081947189 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5643faffcd70 con 0x5643fb114e80 2024-09-19T09:04:28.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.982+0000 7f1212ffd640 10 client.15888 handle_command_reply: tid=0 2024-09-19T09:04:28.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.982+0000 7f123c0b8740 1 client.15888 shutdown 2024-09-19T09:04:28.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.983+0000 7f123c0b8740 1 -- 172.21.15.120:0/3081947189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643fb09a370 msgr2=0x5643fb0941c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:28.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.983+0000 7f123c0b8740 1 --2- 172.21.15.120:0/3081947189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643fb09a370 0x5643fb0941c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f122c004be0 tx=0x7f122c00e880 comp rx=0 tx=0).stop 2024-09-19T09:04:28.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.983+0000 7f123c0b8740 1 -- 172.21.15.120:0/3081947189 shutdown_connections 2024-09-19T09:04:28.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.983+0000 7f123c0b8740 1 --2- 172.21.15.120:0/3081947189 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5643fb114e80 0x5643fb1172a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.983+0000 7f123c0b8740 1 --2- 172.21.15.120:0/3081947189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643fb09a370 0x5643fb0941c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.983+0000 7f123c0b8740 1 --2- 172.21.15.120:0/3081947189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643fb090b40 0x5643fb093c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.983+0000 7f1212ffd640 0 client.15888 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:04:28.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.983+0000 7f123c0b8740 1 --2- 172.21.15.120:0/3081947189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643fb08f5d0 0x5643fb0a2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.983+0000 7f123c0b8740 1 -- 172.21.15.120:0/3081947189 >> 172.21.15.120:0/3081947189 conn(0x5643fb058310 msgr2=0x5643fb0954a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:28.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.983+0000 7f123c0b8740 1 -- 172.21.15.120:0/3081947189 shutdown_connections 2024-09-19T09:04:28.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.983+0000 7f123c0b8740 1 -- 172.21.15.120:0/3081947189 wait complete. 2024-09-19T09:04:28.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.983+0000 7f123c0b8740 20 client.15888 trim_cache size 0 max 16384 2024-09-19T09:04:28.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.983+0000 7f1239cdf640 1 -- 172.21.15.120:0/3222452605 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f12200a3240 msgr2=0x7f12200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:28.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.983+0000 7f1239cdf640 1 --2- 172.21.15.120:0/3222452605 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f12200a3240 0x7f12200a5700 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f122c011800 tx=0x7f122c00ca00 comp rx=0 tx=0).stop 2024-09-19T09:04:28.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.983+0000 7f1239cdf640 1 -- 172.21.15.120:0/3222452605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1234101de0 msgr2=0x7f12340744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:28.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.983+0000 7f1239cdf640 1 --2- 172.21.15.120:0/3222452605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1234101de0 0x7f12340744f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1224005ab0 tx=0x7f12240042f0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.983+0000 7f1239cdf640 1 -- 172.21.15.120:0/3222452605 shutdown_connections 2024-09-19T09:04:28.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.983+0000 7f1239cdf640 1 --2- 172.21.15.120:0/3222452605 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f12200a3240 0x7f12200a5700 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.983+0000 7f1239cdf640 1 --2- 172.21.15.120:0/3222452605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1234106f10 0x7f1234074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.983+0000 7f1239cdf640 1 --2- 172.21.15.120:0/3222452605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1234102790 0x7f1234074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.983+0000 7f1239cdf640 1 --2- 172.21.15.120:0/3222452605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1234101de0 0x7f12340744f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:28.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.983+0000 7f1239cdf640 1 -- 172.21.15.120:0/3222452605 >> 172.21.15.120:0/3222452605 conn(0x7f12340fd8c0 msgr2=0x7f12340fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:28.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.984+0000 7f1239cdf640 1 -- 172.21.15.120:0/3222452605 shutdown_connections 2024-09-19T09:04:28.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:27.984+0000 7f1239cdf640 1 -- 172.21.15.120:0/3222452605 wait complete. 2024-09-19T09:04:29.035 INFO:tasks.workunit.client.0.smithi120.stdout: 1 11836 3.35 MB/sec warmup 24 sec latency 20499.650 ms 2024-09-19T09:04:29.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:28 smithi120 ceph-mon[31150]: pgmap v389: 97 pgs: 97 active+clean; 25 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 2.4 MiB/s wr, 49 op/s 2024-09-19T09:04:29.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:28 smithi196 ceph-mon[35150]: pgmap v389: 97 pgs: 97 active+clean; 25 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 2.4 MiB/s wr, 49 op/s 2024-09-19T09:04:29.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:28 smithi153 ceph-mon[35071]: pgmap v389: 97 pgs: 97 active+clean; 25 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 2.4 MiB/s wr, 49 op/s 2024-09-19T09:04:30.035 INFO:tasks.workunit.client.0.smithi120.stdout: 1 11836 3.21 MB/sec warmup 25 sec latency 21499.789 ms 2024-09-19T09:04:30.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:30 smithi120 ceph-mon[31150]: pgmap v390: 97 pgs: 97 active+clean; 25 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 2.4 MiB/s wr, 48 op/s 2024-09-19T09:04:30.922 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:04:31.035 INFO:tasks.workunit.client.0.smithi120.stdout: 1 11836 3.09 MB/sec warmup 26 sec latency 22499.855 ms 2024-09-19T09:04:31.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:30 smithi196 ceph-mon[35150]: pgmap v390: 97 pgs: 97 active+clean; 25 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 2.4 MiB/s wr, 48 op/s 2024-09-19T09:04:31.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:30 smithi153 ceph-mon[35071]: pgmap v390: 97 pgs: 97 active+clean; 25 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 2.4 MiB/s wr, 48 op/s 2024-09-19T09:04:31.417 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db --set-id 5c43545b --release --await' 2024-09-19T09:04:31.417 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db --set-id 5c43545b --release --await 2024-09-19T09:04:31.468 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:04:31.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.960+0000 7f342cebe640 1 -- 172.21.15.120:0/4100466803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3428101df0 msgr2=0x7f34281021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:31.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.960+0000 7f342cebe640 1 --2- 172.21.15.120:0/4100466803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3428101df0 0x7f34281021d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f3420009ca0 tx=0x7f342002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:04:31.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.960+0000 7f342cebe640 1 -- 172.21.15.120:0/4100466803 shutdown_connections 2024-09-19T09:04:31.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.960+0000 7f342cebe640 1 --2- 172.21.15.120:0/4100466803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3428106f20 0x7f342810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:31.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.960+0000 7f342cebe640 1 --2- 172.21.15.120:0/4100466803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34281027a0 0x7f34281067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:31.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.960+0000 7f342cebe640 1 --2- 172.21.15.120:0/4100466803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3428101df0 0x7f34281021d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:31.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.960+0000 7f342cebe640 1 -- 172.21.15.120:0/4100466803 >> 172.21.15.120:0/4100466803 conn(0x7f34280fd8f0 msgr2=0x7f34280ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:31.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.960+0000 7f342cebe640 1 -- 172.21.15.120:0/4100466803 shutdown_connections 2024-09-19T09:04:31.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.960+0000 7f342cebe640 1 -- 172.21.15.120:0/4100466803 wait complete. 2024-09-19T09:04:31.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.961+0000 7f342cebe640 1 Processor -- start 2024-09-19T09:04:31.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.961+0000 7f342cebe640 1 -- start start 2024-09-19T09:04:31.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.961+0000 7f342cebe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3428101df0 0x7f3428198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:31.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.961+0000 7f342cebe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34281027a0 0x7f34281989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:31.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.961+0000 7f342cebe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3428106f20 0x7f342819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:31.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.961+0000 7f342cebe640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3428111f80 con 0x7f3428106f20 2024-09-19T09:04:31.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.961+0000 7f342cebe640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3428111e00 con 0x7f34281027a0 2024-09-19T09:04:31.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.961+0000 7f342cebe640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3428112100 con 0x7f3428101df0 2024-09-19T09:04:31.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.961+0000 7f3426d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3428106f20 0x7f342819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:31.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.961+0000 7f3426d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3428106f20 0x7f342819cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60502/0 (socket says 172.21.15.120:60502) 2024-09-19T09:04:31.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.961+0000 7f3426d9d640 1 -- 172.21.15.120:0/3358743611 learned_addr learned my addr 172.21.15.120:0/3358743611 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:31.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.961+0000 7f342659c640 1 --2- 172.21.15.120:0/3358743611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3428101df0 0x7f3428198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:31.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.961+0000 7f3426d9d640 1 -- 172.21.15.120:0/3358743611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3428101df0 msgr2=0x7f3428198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:31.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.961+0000 7f3426d9d640 1 --2- 172.21.15.120:0/3358743611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3428101df0 0x7f3428198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:31.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.961+0000 7f3426d9d640 1 -- 172.21.15.120:0/3358743611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34281027a0 msgr2=0x7f34281989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:31.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.961+0000 7f3425d9b640 1 --2- 172.21.15.120:0/3358743611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34281027a0 0x7f34281989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:31.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.962+0000 7f3426d9d640 1 --2- 172.21.15.120:0/3358743611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34281027a0 0x7f34281989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:31.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.962+0000 7f3426d9d640 1 -- 172.21.15.120:0/3358743611 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342819d410 con 0x7f3428106f20 2024-09-19T09:04:31.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.962+0000 7f342659c640 1 --2- 172.21.15.120:0/3358743611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3428101df0 0x7f3428198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:04:31.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.962+0000 7f3426d9d640 1 --2- 172.21.15.120:0/3358743611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3428106f20 0x7f342819cd30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f341800c970 tx=0x7f341800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:31.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.962+0000 7f341f7fe640 1 -- 172.21.15.120:0/3358743611 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418007bb0 con 0x7f3428106f20 2024-09-19T09:04:31.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.962+0000 7f341f7fe640 1 -- 172.21.15.120:0/3358743611 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3418007d50 con 0x7f3428106f20 2024-09-19T09:04:31.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.962+0000 7f341f7fe640 1 -- 172.21.15.120:0/3358743611 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341800f450 con 0x7f3428106f20 2024-09-19T09:04:31.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.962+0000 7f342cebe640 1 -- 172.21.15.120:0/3358743611 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342819d700 con 0x7f3428106f20 2024-09-19T09:04:31.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.962+0000 7f342cebe640 1 -- 172.21.15.120:0/3358743611 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34281a4fe0 con 0x7f3428106f20 2024-09-19T09:04:31.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.964+0000 7f341f7fe640 1 -- 172.21.15.120:0/3358743611 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f3418004f60 con 0x7f3428106f20 2024-09-19T09:04:31.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.965+0000 7f342f297740 1 Processor -- start 2024-09-19T09:04:31.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.965+0000 7f342f297740 1 -- start start 2024-09-19T09:04:31.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.965+0000 7f342f297740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558098abeb40 0x558098abd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:31.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.965+0000 7f342f297740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558098abd5d0 0x558098abda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:31.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.965+0000 7f342f297740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558098ac8370 0x558098aca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:31.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.965+0000 7f342f297740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558098a232f0 con 0x558098abeb40 2024-09-19T09:04:31.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.965+0000 7f342f297740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5580989cacf0 con 0x558098ac8370 2024-09-19T09:04:31.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.965+0000 7f342f297740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558098a1f2b0 con 0x558098abd5d0 2024-09-19T09:04:31.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.965+0000 7f341f7fe640 1 --2- 172.21.15.120:0/3358743611 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f34140a3240 0x7f34140a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:31.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.965+0000 7f3425d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558098abeb40 0x558098abd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:31.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.965+0000 7f3425d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558098abeb40 0x558098abd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60512/0 (socket says 172.21.15.120:60512) 2024-09-19T09:04:31.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.965+0000 7f3425d9b640 1 -- 172.21.15.120:0/3440999886 learned_addr learned my addr 172.21.15.120:0/3440999886 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:31.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.965+0000 7f3426d9d640 1 --2- 172.21.15.120:0/3358743611 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f34140a3240 0x7f34140a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:31.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.965+0000 7f3425d9b640 1 --2- 172.21.15.120:0/3440999886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558098ac8370 0x558098aca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:31.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.965+0000 7f342659c640 1 --2- 172.21.15.120:0/3440999886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558098abd5d0 0x558098abda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:31.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.965+0000 7f341f7fe640 1 -- 172.21.15.120:0/3358743611 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f34180cf670 con 0x7f3428106f20 2024-09-19T09:04:31.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.965+0000 7f3425d9b640 1 -- 172.21.15.120:0/3440999886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558098abd5d0 msgr2=0x558098abda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:31.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.965+0000 7f3425d9b640 1 --2- 172.21.15.120:0/3440999886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558098abd5d0 0x558098abda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:31.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.965+0000 7f3425d9b640 1 -- 172.21.15.120:0/3440999886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558098ac8370 msgr2=0x558098aca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:31.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.965+0000 7f3425d9b640 1 --2- 172.21.15.120:0/3440999886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558098ac8370 0x558098aca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:31.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.965+0000 7f3425d9b640 1 -- 172.21.15.120:0/3440999886 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580989af3a0 con 0x558098abeb40 2024-09-19T09:04:31.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.966+0000 7f3425d9b640 1 --2- 172.21.15.120:0/3440999886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558098abeb40 0x558098abd090 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3410005d00 tx=0x7f3410009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=220f7d46c595f01d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:31.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.966+0000 7f3426d9d640 1 --2- 172.21.15.120:0/3358743611 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f34140a3240 0x7f34140a5700 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3418007590 tx=0x7f3418091170 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:31.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.966+0000 7f341cff9640 1 -- 172.21.15.120:0/3440999886 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341001b070 con 0x558098abeb40 2024-09-19T09:04:31.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.966+0000 7f341cff9640 1 -- 172.21.15.120:0/3440999886 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34100139a0 con 0x558098abeb40 2024-09-19T09:04:31.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.966+0000 7f341cff9640 1 -- 172.21.15.120:0/3440999886 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3410010940 con 0x558098abeb40 2024-09-19T09:04:31.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.966+0000 7f342f297740 1 -- 172.21.15.120:0/3440999886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558098abeb40 msgr2=0x558098abd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:31.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.966+0000 7f342f297740 1 --2- 172.21.15.120:0/3440999886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558098abeb40 0x558098abd090 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3410005d00 tx=0x7f3410009590 comp rx=0 tx=0).stop 2024-09-19T09:04:31.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.967+0000 7f342f297740 1 -- 172.21.15.120:0/3440999886 shutdown_connections 2024-09-19T09:04:31.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.967+0000 7f342f297740 1 --2- 172.21.15.120:0/3440999886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558098ac8370 0x558098aca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:31.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.967+0000 7f342f297740 1 --2- 172.21.15.120:0/3440999886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558098abd5d0 0x558098abda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:31.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.967+0000 7f342f297740 1 --2- 172.21.15.120:0/3440999886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558098abeb40 0x558098abd090 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:31.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.967+0000 7f342f297740 1 -- 172.21.15.120:0/3440999886 >> 172.21.15.120:0/3440999886 conn(0x558098a86310 msgr2=0x558098ab9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:31.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.967+0000 7f342f297740 1 -- 172.21.15.120:0/3440999886 shutdown_connections 2024-09-19T09:04:31.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.967+0000 7f342f297740 1 -- 172.21.15.120:0/3440999886 wait complete. 2024-09-19T09:04:31.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.967+0000 7f342f297740 1 -- start start 2024-09-19T09:04:31.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.967+0000 7f342f297740 1 Processor -- start 2024-09-19T09:04:31.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.967+0000 7f342f297740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558098abd5d0 0x558098ac7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:31.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.967+0000 7f342f297740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558098abeb40 0x558098ac5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:31.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.967+0000 7f342f297740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558098ac8370 0x558098ac5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:31.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.967+0000 7f342f297740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558098acb070 con 0x558098ac8370 2024-09-19T09:04:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.967+0000 7f342f297740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5580989de060 con 0x558098abd5d0 2024-09-19T09:04:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.967+0000 7f342f297740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558098acb1f0 con 0x558098abeb40 2024-09-19T09:04:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.967+0000 7f342f297740 10 client.0 _ll_register_callbacks cb 0x5580987318c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.967+0000 7f3425d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558098ac8370 0x558098ac5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.967+0000 7f3425d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558098ac8370 0x558098ac5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60524/0 (socket says 172.21.15.120:60524) 2024-09-19T09:04:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.967+0000 7f3425d9b640 1 -- 172.21.15.120:0/2052195706 learned_addr learned my addr 172.21.15.120:0/2052195706 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.967+0000 7f3425d9b640 1 --2- 172.21.15.120:0/2052195706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558098abd5d0 0x558098ac7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.968+0000 7f342659c640 1 --2- 172.21.15.120:0/2052195706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558098abeb40 0x558098ac5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.968+0000 7f3425d9b640 1 -- 172.21.15.120:0/2052195706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558098abeb40 msgr2=0x558098ac5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.968+0000 7f3425d9b640 1 --2- 172.21.15.120:0/2052195706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558098abeb40 0x558098ac5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.968+0000 7f3425d9b640 1 -- 172.21.15.120:0/2052195706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558098abd5d0 msgr2=0x558098ac7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.968+0000 7f3425d9b640 1 --2- 172.21.15.120:0/2052195706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558098abd5d0 0x558098ac7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.968+0000 7f3425d9b640 1 -- 172.21.15.120:0/2052195706 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580987943c0 con 0x558098ac8370 2024-09-19T09:04:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.968+0000 7f342659c640 1 --2- 172.21.15.120:0/2052195706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558098abeb40 0x558098ac5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:04:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.968+0000 7f3425d9b640 1 --2- 172.21.15.120:0/2052195706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558098ac8370 0x558098ac5810 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3410010620 tx=0x7f3410010480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f885d91458e79 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.968+0000 7f340dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:04:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.968+0000 7f340dffb640 1 -- 172.21.15.120:0/2052195706 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341002a060 con 0x558098ac8370 2024-09-19T09:04:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.968+0000 7f340dffb640 1 -- 172.21.15.120:0/2052195706 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f3410013d50 con 0x558098ac8370 2024-09-19T09:04:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.968+0000 7f342f297740 1 -- 172.21.15.120:0/2052195706 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34100139a0 con 0x558098ac8370 2024-09-19T09:04:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.968+0000 7f340dffb640 1 -- 172.21.15.120:0/2052195706 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f341001b040 con 0x558098ac8370 2024-09-19T09:04:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.968+0000 7f342f297740 10 client.15900 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.968+0000 7f342f297740 1 -- 172.21.15.120:0/2052195706 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558098ad6bc0 con 0x558098ac8370 2024-09-19T09:04:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.968+0000 7f340dffb640 1 -- 172.21.15.120:0/2052195706 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34100139a0 con 0x558098ac8370 2024-09-19T09:04:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.969+0000 7f340dffb640 1 -- 172.21.15.120:0/2052195706 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f341002f070 con 0x558098ac8370 2024-09-19T09:04:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.969+0000 7f342f297740 10 client.15900 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.969+0000 7f342f297740 10 client.15900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:04:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.969+0000 7f342f297740 20 client.15900 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.969+0000 7f342f297740 1 --2- 172.21.15.120:0/2052195706 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558098ad7f70 0x558098ada360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.969+0000 7f342f297740 4 client.15900 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:04:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.969+0000 7f342f297740 1 -- 172.21.15.120:0/2052195706 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558098a30860 con 0x558098ad7f70 2024-09-19T09:04:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.969+0000 7f342659c640 1 --2- 172.21.15.120:0/2052195706 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558098ad7f70 0x558098ada360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.969+0000 7f340dffb640 10 client.15900 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.969+0000 7f342659c640 1 --2- 172.21.15.120:0/2052195706 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558098ad7f70 0x558098ada360 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe686425f5703c14 server_cookie=55117b3e891ff96c in_seq=0 out_seq=0 2024-09-19T09:04:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.970+0000 7f340dffb640 1 -- 172.21.15.120:0/2052195706 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558098a30860 con 0x558098ad7f70 2024-09-19T09:04:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.970+0000 7f340dffb640 10 client.15900 handle_command_reply: tid=0 2024-09-19T09:04:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.970+0000 7f342f297740 1 client.15900 shutdown 2024-09-19T09:04:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.970+0000 7f342f297740 1 -- 172.21.15.120:0/2052195706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558098ac8370 msgr2=0x558098ac5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.970+0000 7f342f297740 1 --2- 172.21.15.120:0/2052195706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558098ac8370 0x558098ac5810 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3410010620 tx=0x7f3410010480 comp rx=0 tx=0).stop 2024-09-19T09:04:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.970+0000 7f342f297740 1 -- 172.21.15.120:0/2052195706 shutdown_connections 2024-09-19T09:04:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.970+0000 7f342f297740 1 --2- 172.21.15.120:0/2052195706 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558098ad7f70 0x558098ada360 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.970+0000 7f342f297740 1 --2- 172.21.15.120:0/2052195706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558098ac8370 0x558098ac5810 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.970+0000 7f342f297740 1 --2- 172.21.15.120:0/2052195706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558098abeb40 0x558098ac5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.970+0000 7f340dffb640 0 client.15900 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.970+0000 7f342f297740 1 --2- 172.21.15.120:0/2052195706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558098abd5d0 0x558098ac7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.970+0000 7f342f297740 1 -- 172.21.15.120:0/2052195706 >> 172.21.15.120:0/2052195706 conn(0x558098a86310 msgr2=0x558098abf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.970+0000 7f342f297740 1 -- 172.21.15.120:0/2052195706 shutdown_connections 2024-09-19T09:04:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.970+0000 7f342f297740 1 -- 172.21.15.120:0/2052195706 wait complete. 2024-09-19T09:04:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.970+0000 7f342f297740 20 client.15900 trim_cache size 0 max 16384 2024-09-19T09:04:31.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.986+0000 7f342f297740 1 Processor -- start 2024-09-19T09:04:31.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.986+0000 7f342f297740 1 -- start start 2024-09-19T09:04:31.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.986+0000 7f342f297740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558098abd5d0 0x558098ac0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:31.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.986+0000 7f342f297740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558098abeb40 0x558098ac0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:31.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.986+0000 7f342f297740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558098ac8370 0x558098add380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:31.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.986+0000 7f342f297740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f34281ad620 con 0x558098abeb40 2024-09-19T09:04:31.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.986+0000 7f342f297740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f34281af160 con 0x558098abd5d0 2024-09-19T09:04:31.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.986+0000 7f342f297740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f34281adcf0 con 0x558098ac8370 2024-09-19T09:04:31.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.986+0000 7f342659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558098abeb40 0x558098ac0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:31.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.986+0000 7f342659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558098abeb40 0x558098ac0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60532/0 (socket says 172.21.15.120:60532) 2024-09-19T09:04:31.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.986+0000 7f342659c640 1 -- 172.21.15.120:0/3971130533 learned_addr learned my addr 172.21.15.120:0/3971130533 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:31.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.986+0000 7f3425d9b640 1 --2- 172.21.15.120:0/3971130533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558098abd5d0 0x558098ac0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:31.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.986+0000 7f3425d9b640 1 --2- 172.21.15.120:0/3971130533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558098ac8370 0x558098add380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:31.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.986+0000 7f342659c640 1 -- 172.21.15.120:0/3971130533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558098ac8370 msgr2=0x558098add380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:31.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.986+0000 7f342659c640 1 --2- 172.21.15.120:0/3971130533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558098ac8370 0x558098add380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:31.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.986+0000 7f342659c640 1 -- 172.21.15.120:0/3971130533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558098abd5d0 msgr2=0x558098ac0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:31.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.986+0000 7f342659c640 1 --2- 172.21.15.120:0/3971130533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558098abd5d0 0x558098ac0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:31.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.986+0000 7f342659c640 1 -- 172.21.15.120:0/3971130533 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558098acb110 con 0x558098abeb40 2024-09-19T09:04:31.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.987+0000 7f342659c640 1 --2- 172.21.15.120:0/3971130533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558098abeb40 0x558098ac0660 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3420009a60 tx=0x7f3420031db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb7c8934ec53c49 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:31.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.987+0000 7f340ffff640 1 -- 172.21.15.120:0/3971130533 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3420049030 con 0x558098abeb40 2024-09-19T09:04:31.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.987+0000 7f340ffff640 1 -- 172.21.15.120:0/3971130533 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3420036040 con 0x558098abeb40 2024-09-19T09:04:31.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.987+0000 7f340ffff640 1 -- 172.21.15.120:0/3971130533 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34200042a0 con 0x558098abeb40 2024-09-19T09:04:31.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.988+0000 7f342f297740 1 -- 172.21.15.120:0/3971130533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558098abeb40 msgr2=0x558098ac0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:31.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.988+0000 7f342f297740 1 --2- 172.21.15.120:0/3971130533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558098abeb40 0x558098ac0660 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3420009a60 tx=0x7f3420031db0 comp rx=0 tx=0).stop 2024-09-19T09:04:31.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.988+0000 7f342f297740 1 -- 172.21.15.120:0/3971130533 shutdown_connections 2024-09-19T09:04:31.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.988+0000 7f342f297740 1 --2- 172.21.15.120:0/3971130533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558098ac8370 0x558098add380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:31.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.988+0000 7f342f297740 1 --2- 172.21.15.120:0/3971130533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558098abeb40 0x558098ac0660 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:31.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.988+0000 7f342f297740 1 --2- 172.21.15.120:0/3971130533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558098abd5d0 0x558098ac0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:31.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.988+0000 7f342f297740 1 -- 172.21.15.120:0/3971130533 >> 172.21.15.120:0/3971130533 conn(0x558098a86310 msgr2=0x558098ac4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:31.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.988+0000 7f342f297740 1 -- 172.21.15.120:0/3971130533 shutdown_connections 2024-09-19T09:04:31.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.988+0000 7f342f297740 1 -- 172.21.15.120:0/3971130533 wait complete. 2024-09-19T09:04:31.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.988+0000 7f342f297740 1 -- start start 2024-09-19T09:04:31.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.988+0000 7f342f297740 1 Processor -- start 2024-09-19T09:04:31.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.988+0000 7f342f297740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558098abd5d0 0x558098addbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:31.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.988+0000 7f342f297740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558098abeb40 0x558098adafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:31.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.988+0000 7f342f297740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558098ac8370 0x558098adb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:31.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.988+0000 7f342f297740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f34281aea90 con 0x558098abd5d0 2024-09-19T09:04:31.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.988+0000 7f342f297740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f34281ae3c0 con 0x558098ac8370 2024-09-19T09:04:31.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.988+0000 7f342f297740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f34281ab5e0 con 0x558098abeb40 2024-09-19T09:04:31.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.988+0000 7f342f297740 10 client.0 _ll_register_callbacks cb 0x7f3410019070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:31.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.988+0000 7f3425d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558098abd5d0 0x558098addbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:31.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.989+0000 7f342659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558098abeb40 0x558098adafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:31.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.989+0000 7f3425d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558098abd5d0 0x558098addbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60536/0 (socket says 172.21.15.120:60536) 2024-09-19T09:04:31.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.989+0000 7f3425d9b640 1 -- 172.21.15.120:0/3455484783 learned_addr learned my addr 172.21.15.120:0/3455484783 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:31.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.989+0000 7f3425d9b640 1 --2- 172.21.15.120:0/3455484783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558098ac8370 0x558098adb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:31.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.989+0000 7f3425d9b640 1 -- 172.21.15.120:0/3455484783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558098abeb40 msgr2=0x558098adafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:31.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.989+0000 7f3425d9b640 1 --2- 172.21.15.120:0/3455484783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558098abeb40 0x558098adafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:31.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.989+0000 7f3425d9b640 1 -- 172.21.15.120:0/3455484783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558098ac8370 msgr2=0x558098adb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:31.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.989+0000 7f3425d9b640 1 --2- 172.21.15.120:0/3455484783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558098ac8370 0x558098adb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:31.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.989+0000 7f3425d9b640 1 -- 172.21.15.120:0/3455484783 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558098adbdd0 con 0x558098abd5d0 2024-09-19T09:04:31.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.989+0000 7f3425d9b640 1 --2- 172.21.15.120:0/3455484783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558098abd5d0 0x558098addbe0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3410010ca0 tx=0x7f3410005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847b990a0b0cae54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:31.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.989+0000 7f340dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:04:31.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.989+0000 7f340dffb640 1 -- 172.21.15.120:0/3455484783 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341002a070 con 0x558098abd5d0 2024-09-19T09:04:31.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.989+0000 7f340dffb640 1 -- 172.21.15.120:0/3455484783 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f3410010650 con 0x558098abd5d0 2024-09-19T09:04:31.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.989+0000 7f342f297740 1 -- 172.21.15.120:0/3455484783 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3420036040 con 0x558098abd5d0 2024-09-19T09:04:31.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.989+0000 7f340dffb640 1 -- 172.21.15.120:0/3455484783 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3410035040 con 0x558098abd5d0 2024-09-19T09:04:31.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.989+0000 7f342f297740 10 client.15906 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:31.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.989+0000 7f342f297740 1 -- 172.21.15.120:0/3455484783 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558098b429b0 con 0x558098abd5d0 2024-09-19T09:04:31.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.990+0000 7f340dffb640 1 -- 172.21.15.120:0/3455484783 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3420036040 con 0x558098abd5d0 2024-09-19T09:04:31.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.990+0000 7f340dffb640 1 -- 172.21.15.120:0/3455484783 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f341002f040 con 0x558098abd5d0 2024-09-19T09:04:31.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.990+0000 7f342f297740 10 client.15906 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:31.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.990+0000 7f342f297740 10 client.15906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:04:31.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.990+0000 7f342f297740 20 client.15906 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:31.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.990+0000 7f342f297740 1 --2- 172.21.15.120:0/3455484783 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558098b43080 0x558098b45470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:31.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.990+0000 7f342f297740 4 client.15906 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:04:31.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.990+0000 7f342f297740 1 -- 172.21.15.120:0/3455484783 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558098a2ad70 con 0x558098b43080 2024-09-19T09:04:31.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.990+0000 7f342659c640 1 --2- 172.21.15.120:0/3455484783 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558098b43080 0x558098b45470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:31.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.991+0000 7f342659c640 1 --2- 172.21.15.120:0/3455484783 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558098b43080 0x558098b45470 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c91260e086e8900 server_cookie=e6037df4467a029a in_seq=0 out_seq=0 2024-09-19T09:04:31.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.991+0000 7f340dffb640 10 client.15906 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:31.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.991+0000 7f340dffb640 1 -- 172.21.15.120:0/3455484783 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558098a2ad70 con 0x558098b43080 2024-09-19T09:04:31.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.991+0000 7f340dffb640 10 client.15906 handle_command_reply: tid=0 2024-09-19T09:04:31.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.991+0000 7f342f297740 1 client.15906 shutdown 2024-09-19T09:04:31.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.991+0000 7f342f297740 1 -- 172.21.15.120:0/3455484783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558098abd5d0 msgr2=0x558098addbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:31.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.991+0000 7f342f297740 1 --2- 172.21.15.120:0/3455484783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558098abd5d0 0x558098addbe0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3410010ca0 tx=0x7f3410005f30 comp rx=0 tx=0).stop 2024-09-19T09:04:31.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.991+0000 7f342f297740 1 -- 172.21.15.120:0/3455484783 shutdown_connections 2024-09-19T09:04:31.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.991+0000 7f342f297740 1 --2- 172.21.15.120:0/3455484783 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558098b43080 0x558098b45470 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:31.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.991+0000 7f342f297740 1 --2- 172.21.15.120:0/3455484783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558098ac8370 0x558098adb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:31.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.991+0000 7f342f297740 1 --2- 172.21.15.120:0/3455484783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558098abeb40 0x558098adafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:31.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.991+0000 7f342f297740 1 --2- 172.21.15.120:0/3455484783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558098abd5d0 0x558098addbe0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:31.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.991+0000 7f342f297740 1 -- 172.21.15.120:0/3455484783 >> 172.21.15.120:0/3455484783 conn(0x558098a86310 msgr2=0x558098ad34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:31.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.991+0000 7f340dffb640 0 client.15906 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:31.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.991+0000 7f342f297740 1 -- 172.21.15.120:0/3455484783 shutdown_connections 2024-09-19T09:04:31.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.991+0000 7f342f297740 1 -- 172.21.15.120:0/3455484783 wait complete. 2024-09-19T09:04:31.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.991+0000 7f342f297740 20 client.15906 trim_cache size 0 max 16384 2024-09-19T09:04:31.999 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:04:32.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.992+0000 7f342cebe640 1 -- 172.21.15.120:0/3358743611 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f34140a3240 msgr2=0x7f34140a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:32.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.992+0000 7f342cebe640 1 --2- 172.21.15.120:0/3358743611 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f34140a3240 0x7f34140a5700 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3418007590 tx=0x7f3418091170 comp rx=0 tx=0).stop 2024-09-19T09:04:32.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.992+0000 7f342cebe640 1 -- 172.21.15.120:0/3358743611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3428106f20 msgr2=0x7f342819cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:32.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.992+0000 7f342cebe640 1 --2- 172.21.15.120:0/3358743611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3428106f20 0x7f342819cd30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f341800c970 tx=0x7f341800ce30 comp rx=0 tx=0).stop 2024-09-19T09:04:32.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.992+0000 7f342cebe640 1 -- 172.21.15.120:0/3358743611 shutdown_connections 2024-09-19T09:04:32.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.992+0000 7f342cebe640 1 --2- 172.21.15.120:0/3358743611 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f34140a3240 0x7f34140a5700 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:32.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.992+0000 7f342cebe640 1 --2- 172.21.15.120:0/3358743611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3428106f20 0x7f342819cd30 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:32.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.992+0000 7f342cebe640 1 --2- 172.21.15.120:0/3358743611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34281027a0 0x7f34281989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:32.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.992+0000 7f342cebe640 1 --2- 172.21.15.120:0/3358743611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3428101df0 0x7f3428198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:32.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.992+0000 7f342cebe640 1 -- 172.21.15.120:0/3358743611 >> 172.21.15.120:0/3358743611 conn(0x7f34280fd8f0 msgr2=0x7f34280fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:32.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.992+0000 7f342cebe640 1 -- 172.21.15.120:0/3358743611 shutdown_connections 2024-09-19T09:04:32.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:31.992+0000 7f342cebe640 1 -- 172.21.15.120:0/3358743611 wait complete. 2024-09-19T09:04:32.035 INFO:tasks.workunit.client.0.smithi120.stdout: 1 11836 2.98 MB/sec warmup 27 sec latency 23499.886 ms 2024-09-19T09:04:32.347 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:04:32.899 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:32 smithi120 ceph-mon[31150]: pgmap v391: 97 pgs: 97 active+clean; 25 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 2.4 MiB/s wr, 50 op/s 2024-09-19T09:04:33.035 INFO:tasks.workunit.client.0.smithi120.stdout: 1 11836 2.87 MB/sec warmup 28 sec latency 24499.926 ms 2024-09-19T09:04:33.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:32 smithi196 ceph-mon[35150]: pgmap v391: 97 pgs: 97 active+clean; 25 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 2.4 MiB/s wr, 50 op/s 2024-09-19T09:04:33.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:32 smithi153 ceph-mon[35071]: pgmap v391: 97 pgs: 97 active+clean; 25 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 2.4 MiB/s wr, 50 op/s 2024-09-19T09:04:33.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.269+0000 7f985d7f8640 1 -- 172.21.15.120:0/3062015607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9858102ef0 msgr2=0x7f985810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:33.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.269+0000 7f985d7f8640 1 --2- 172.21.15.120:0/3062015607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9858102ef0 0x7f985810da40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9850009ca0 tx=0x7f985002a950 comp rx=0 tx=0).stop 2024-09-19T09:04:33.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.269+0000 7f985d7f8640 1 -- 172.21.15.120:0/3062015607 shutdown_connections 2024-09-19T09:04:33.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.269+0000 7f985d7f8640 1 --2- 172.21.15.120:0/3062015607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9858102ef0 0x7f985810da40 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.269+0000 7f985d7f8640 1 --2- 172.21.15.120:0/3062015607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9858102570 0x7f98581029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.269+0000 7f985d7f8640 1 --2- 172.21.15.120:0/3062015607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9858100e20 0x7f9858101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.269+0000 7f985d7f8640 1 -- 172.21.15.120:0/3062015607 >> 172.21.15.120:0/3062015607 conn(0x7f98580fc7d0 msgr2=0x7f98580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:33.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.269+0000 7f985d7f8640 1 -- 172.21.15.120:0/3062015607 shutdown_connections 2024-09-19T09:04:33.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.269+0000 7f985d7f8640 1 -- 172.21.15.120:0/3062015607 wait complete. 2024-09-19T09:04:33.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.270+0000 7f985d7f8640 1 Processor -- start 2024-09-19T09:04:33.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.270+0000 7f985d7f8640 1 -- start start 2024-09-19T09:04:33.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.270+0000 7f985d7f8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9858100e20 0x7f9858199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:33.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.270+0000 7f985d7f8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9858102570 0x7f98581998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:33.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.270+0000 7f985d7f8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9858102ef0 0x7f98581a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:33.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.270+0000 7f985d7f8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9858110f80 con 0x7f9858100e20 2024-09-19T09:04:33.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.270+0000 7f985d7f8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9858110e00 con 0x7f9858102570 2024-09-19T09:04:33.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.270+0000 7f985d7f8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9858111100 con 0x7f9858102ef0 2024-09-19T09:04:33.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.270+0000 7f9856ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9858100e20 0x7f9858199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:33.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.270+0000 7f9856ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9858100e20 0x7f9858199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60556/0 (socket says 172.21.15.120:60556) 2024-09-19T09:04:33.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.270+0000 7f9856ffd640 1 -- 172.21.15.120:0/2681375288 learned_addr learned my addr 172.21.15.120:0/2681375288 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:33.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.270+0000 7f98567fc640 1 --2- 172.21.15.120:0/2681375288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9858102570 0x7f98581998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:33.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.270+0000 7f9856ffd640 1 -- 172.21.15.120:0/2681375288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9858102ef0 msgr2=0x7f98581a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:33.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.270+0000 7f9856ffd640 1 --2- 172.21.15.120:0/2681375288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9858102ef0 0x7f98581a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.270+0000 7f9856ffd640 1 -- 172.21.15.120:0/2681375288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9858102570 msgr2=0x7f98581998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:33.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.270+0000 7f9856ffd640 1 --2- 172.21.15.120:0/2681375288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9858102570 0x7f98581998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.270+0000 7f9856ffd640 1 -- 172.21.15.120:0/2681375288 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98581a4930 con 0x7f9858100e20 2024-09-19T09:04:33.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.270+0000 7f98567fc640 1 --2- 172.21.15.120:0/2681375288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9858102570 0x7f98581998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:04:33.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.271+0000 7f9856ffd640 1 --2- 172.21.15.120:0/2681375288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9858100e20 0x7f9858199370 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f984800d950 tx=0x7f984800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:33.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.271+0000 7f983ffff640 1 -- 172.21.15.120:0/2681375288 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9848017070 con 0x7f9858100e20 2024-09-19T09:04:33.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.271+0000 7f985d7f8640 1 -- 172.21.15.120:0/2681375288 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98581a4b00 con 0x7f9858100e20 2024-09-19T09:04:33.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.271+0000 7f983ffff640 1 -- 172.21.15.120:0/2681375288 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9848009c30 con 0x7f9858100e20 2024-09-19T09:04:33.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.271+0000 7f983ffff640 1 -- 172.21.15.120:0/2681375288 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984800f6b0 con 0x7f9858100e20 2024-09-19T09:04:33.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.271+0000 7f985d7f8640 1 -- 172.21.15.120:0/2681375288 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98581a5010 con 0x7f9858100e20 2024-09-19T09:04:33.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.273+0000 7f983ffff640 1 -- 172.21.15.120:0/2681375288 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f98480109d0 con 0x7f9858100e20 2024-09-19T09:04:33.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.273+0000 7f985fbd1740 1 Processor -- start 2024-09-19T09:04:33.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.273+0000 7f985fbd1740 1 -- start start 2024-09-19T09:04:33.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.274+0000 7f985fbd1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565f6b44060 0x5565f6b44440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:33.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.274+0000 7f985fbd1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565f6b44980 0x5565f6b4ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:33.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.274+0000 7f985fbd1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f6b4f240 0x5565f6b51630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:33.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.274+0000 7f985fbd1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5565f6aa9620 con 0x5565f6b44980 2024-09-19T09:04:33.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.274+0000 7f985fbd1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5565f6a50e50 con 0x5565f6b44060 2024-09-19T09:04:33.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.274+0000 7f985fbd1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5565f6aa55e0 con 0x5565f6b4f240 2024-09-19T09:04:33.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.274+0000 7f983ffff640 1 --2- 172.21.15.120:0/2681375288 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f98400a3240 0x7f98400a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:33.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.274+0000 7f9856ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565f6b44980 0x5565f6b4ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:33.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.274+0000 7f9856ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565f6b44980 0x5565f6b4ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60566/0 (socket says 172.21.15.120:60566) 2024-09-19T09:04:33.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.274+0000 7f9856ffd640 1 -- 172.21.15.120:0/3354003671 learned_addr learned my addr 172.21.15.120:0/3354003671 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:33.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.274+0000 7f98577fe640 1 --2- 172.21.15.120:0/3354003671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565f6b44060 0x5565f6b44440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:33.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.274+0000 7f98567fc640 1 --2- 172.21.15.120:0/3354003671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f6b4f240 0x5565f6b51630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:33.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.274+0000 7f98577fe640 1 --2- 172.21.15.120:0/2681375288 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f98400a3240 0x7f98400a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:33.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.274+0000 7f983ffff640 1 -- 172.21.15.120:0/2681375288 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f98480cf5d0 con 0x7f9858100e20 2024-09-19T09:04:33.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.274+0000 7f9856ffd640 1 -- 172.21.15.120:0/3354003671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f6b4f240 msgr2=0x5565f6b51630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:33.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.274+0000 7f9856ffd640 1 --2- 172.21.15.120:0/3354003671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f6b4f240 0x5565f6b51630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.274+0000 7f9856ffd640 1 -- 172.21.15.120:0/3354003671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565f6b44060 msgr2=0x5565f6b44440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:33.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.274+0000 7f9856ffd640 1 --2- 172.21.15.120:0/3354003671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565f6b44060 0x5565f6b44440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.274+0000 7f9856ffd640 1 -- 172.21.15.120:0/3354003671 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565f6a41d40 con 0x5565f6b44980 2024-09-19T09:04:33.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.274+0000 7f9856ffd640 1 --2- 172.21.15.120:0/3354003671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565f6b44980 0x5565f6b4ed00 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f984806ea40 tx=0x7f984806ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cce062520b511ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:33.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.274+0000 7f98577fe640 1 --2- 172.21.15.120:0/2681375288 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f98400a3240 0x7f98400a5700 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9850030040 tx=0x7f98500093a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:33.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.275+0000 7f983d7fa640 1 -- 172.21.15.120:0/3354003671 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9848020070 con 0x5565f6b44980 2024-09-19T09:04:33.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.275+0000 7f983d7fa640 1 -- 172.21.15.120:0/3354003671 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9848066a90 con 0x5565f6b44980 2024-09-19T09:04:33.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.275+0000 7f983d7fa640 1 -- 172.21.15.120:0/3354003671 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98480a3320 con 0x5565f6b44980 2024-09-19T09:04:33.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.275+0000 7f985fbd1740 1 -- 172.21.15.120:0/3354003671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565f6b44980 msgr2=0x5565f6b4ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:33.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.275+0000 7f985fbd1740 1 --2- 172.21.15.120:0/3354003671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565f6b44980 0x5565f6b4ed00 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f984806ea40 tx=0x7f984806ea70 comp rx=0 tx=0).stop 2024-09-19T09:04:33.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.275+0000 7f985fbd1740 1 -- 172.21.15.120:0/3354003671 shutdown_connections 2024-09-19T09:04:33.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.275+0000 7f985fbd1740 1 --2- 172.21.15.120:0/3354003671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f6b4f240 0x5565f6b51630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.275+0000 7f985fbd1740 1 --2- 172.21.15.120:0/3354003671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565f6b44980 0x5565f6b4ed00 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.275+0000 7f985fbd1740 1 --2- 172.21.15.120:0/3354003671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565f6b44060 0x5565f6b44440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.275+0000 7f985fbd1740 1 -- 172.21.15.120:0/3354003671 >> 172.21.15.120:0/3354003671 conn(0x5565f6b405f0 msgr2=0x5565f6b409d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:33.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.275+0000 7f985fbd1740 1 -- 172.21.15.120:0/3354003671 shutdown_connections 2024-09-19T09:04:33.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.275+0000 7f985fbd1740 1 -- 172.21.15.120:0/3354003671 wait complete. 2024-09-19T09:04:33.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.276+0000 7f985fbd1740 1 -- start start 2024-09-19T09:04:33.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.276+0000 7f985fbd1740 1 Processor -- start 2024-09-19T09:04:33.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.276+0000 7f985fbd1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f6b44060 0x5565f6b4c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:33.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.276+0000 7f985fbd1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565f6b44980 0x5565f6b4c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:33.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.276+0000 7f985fbd1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565f6b4f240 0x5565f6b4cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:33.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.276+0000 7f985fbd1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5565f6b51e10 con 0x5565f6b44980 2024-09-19T09:04:33.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.276+0000 7f985fbd1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5565f6a31f10 con 0x5565f6b4f240 2024-09-19T09:04:33.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.276+0000 7f985fbd1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5565f6b51f90 con 0x5565f6b44060 2024-09-19T09:04:33.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.276+0000 7f985fbd1740 10 client.0 _ll_register_callbacks cb 0x5565f6a55db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:33.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.276+0000 7f98567fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565f6b44980 0x5565f6b4c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:33.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.276+0000 7f98567fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565f6b44980 0x5565f6b4c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60570/0 (socket says 172.21.15.120:60570) 2024-09-19T09:04:33.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.276+0000 7f98567fc640 1 -- 172.21.15.120:0/911470755 learned_addr learned my addr 172.21.15.120:0/911470755 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:33.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.276+0000 7f98577fe640 1 --2- 172.21.15.120:0/911470755 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565f6b4f240 0x5565f6b4cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:33.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.276+0000 7f9856ffd640 1 --2- 172.21.15.120:0/911470755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f6b44060 0x5565f6b4c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:33.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.276+0000 7f98567fc640 1 -- 172.21.15.120:0/911470755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f6b44060 msgr2=0x5565f6b4c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:33.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.276+0000 7f98567fc640 1 --2- 172.21.15.120:0/911470755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f6b44060 0x5565f6b4c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.276+0000 7f98567fc640 1 -- 172.21.15.120:0/911470755 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565f6b4f240 msgr2=0x5565f6b4cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:33.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.276+0000 7f98567fc640 1 --2- 172.21.15.120:0/911470755 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565f6b4f240 0x5565f6b4cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.276+0000 7f98567fc640 1 -- 172.21.15.120:0/911470755 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565f67b8580 con 0x5565f6b44980 2024-09-19T09:04:33.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.276+0000 7f98567fc640 1 --2- 172.21.15.120:0/911470755 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565f6b44980 0x5565f6b4c620 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f984c011840 tx=0x7f984c011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6e6e63ee6cdb9d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:33.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.276+0000 7f983aff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:04:33.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.277+0000 7f983aff5640 1 -- 172.21.15.120:0/911470755 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c00fe70 con 0x5565f6b44980 2024-09-19T09:04:33.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.277+0000 7f983aff5640 1 -- 172.21.15.120:0/911470755 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f984c005020 con 0x5565f6b44980 2024-09-19T09:04:33.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.277+0000 7f985fbd1740 1 -- 172.21.15.120:0/911470755 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9848066a90 con 0x5565f6b44980 2024-09-19T09:04:33.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.277+0000 7f983aff5640 1 -- 172.21.15.120:0/911470755 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f984c005220 con 0x5565f6b44980 2024-09-19T09:04:33.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.277+0000 7f985fbd1740 10 client.15918 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:33.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.277+0000 7f985fbd1740 1 -- 172.21.15.120:0/911470755 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5565f6882960 con 0x5565f6b44980 2024-09-19T09:04:33.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.277+0000 7f983aff5640 1 -- 172.21.15.120:0/911470755 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9848066a90 con 0x5565f6b44980 2024-09-19T09:04:33.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.277+0000 7f983aff5640 1 -- 172.21.15.120:0/911470755 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f984c018030 con 0x5565f6b44980 2024-09-19T09:04:33.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.277+0000 7f985fbd1740 10 client.15918 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:33.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.277+0000 7f985fbd1740 10 client.15918 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:04:33.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.277+0000 7f985fbd1740 20 client.15918 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:33.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.277+0000 7f985fbd1740 1 --2- 172.21.15.120:0/911470755 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5565f6b5ee90 0x5565f6b612d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:33.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.277+0000 7f985fbd1740 4 client.15918 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:04:33.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.277+0000 7f985fbd1740 1 -- 172.21.15.120:0/911470755 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565f6ab6b90 con 0x5565f6b5ee90 2024-09-19T09:04:33.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.277+0000 7f9856ffd640 1 --2- 172.21.15.120:0/911470755 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5565f6b5ee90 0x5565f6b612d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:33.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.278+0000 7f983aff5640 10 client.15918 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:33.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.278+0000 7f9856ffd640 1 --2- 172.21.15.120:0/911470755 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5565f6b5ee90 0x5565f6b612d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cab812c0caf056 server_cookie=5cc0988fddbf2f5d in_seq=0 out_seq=0 2024-09-19T09:04:33.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.279+0000 7f983aff5640 1 -- 172.21.15.120:0/911470755 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5565f6ab6b90 con 0x5565f6b5ee90 2024-09-19T09:04:33.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.279+0000 7f983aff5640 10 client.15918 handle_command_reply: tid=0 2024-09-19T09:04:33.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.279+0000 7f985fbd1740 1 client.15918 shutdown 2024-09-19T09:04:33.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.279+0000 7f985fbd1740 1 -- 172.21.15.120:0/911470755 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565f6b44980 msgr2=0x5565f6b4c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:33.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.279+0000 7f985fbd1740 1 --2- 172.21.15.120:0/911470755 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565f6b44980 0x5565f6b4c620 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f984c011840 tx=0x7f984c011d00 comp rx=0 tx=0).stop 2024-09-19T09:04:33.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.279+0000 7f985fbd1740 1 -- 172.21.15.120:0/911470755 shutdown_connections 2024-09-19T09:04:33.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.279+0000 7f985fbd1740 1 --2- 172.21.15.120:0/911470755 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5565f6b5ee90 0x5565f6b612d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.279+0000 7f985fbd1740 1 --2- 172.21.15.120:0/911470755 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565f6b4f240 0x5565f6b4cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.279+0000 7f985fbd1740 1 --2- 172.21.15.120:0/911470755 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565f6b44980 0x5565f6b4c620 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.279+0000 7f985fbd1740 1 --2- 172.21.15.120:0/911470755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f6b44060 0x5565f6b4c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.279+0000 7f985fbd1740 1 -- 172.21.15.120:0/911470755 >> 172.21.15.120:0/911470755 conn(0x5565f6b405f0 msgr2=0x5565f6b3f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:33.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.279+0000 7f983aff5640 0 client.15918 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:33.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.279+0000 7f985fbd1740 1 -- 172.21.15.120:0/911470755 shutdown_connections 2024-09-19T09:04:33.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.279+0000 7f985fbd1740 1 -- 172.21.15.120:0/911470755 wait complete. 2024-09-19T09:04:33.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.279+0000 7f985fbd1740 20 client.15918 trim_cache size 0 max 16384 2024-09-19T09:04:33.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.295+0000 7f985fbd1740 1 Processor -- start 2024-09-19T09:04:33.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.295+0000 7f985fbd1740 1 -- start start 2024-09-19T09:04:33.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.295+0000 7f985fbd1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f6b44060 0x5565f6b46ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:33.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.295+0000 7f985fbd1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565f6b44980 0x5565f6b3f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:33.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.295+0000 7f985fbd1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565f6b4f240 0x5565f6b66ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:33.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.295+0000 7f985fbd1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f98581ad700 con 0x5565f6b4f240 2024-09-19T09:04:33.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.295+0000 7f985fbd1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f98581af240 con 0x5565f6b44980 2024-09-19T09:04:33.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.295+0000 7f985fbd1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f98581addd0 con 0x5565f6b44060 2024-09-19T09:04:33.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.295+0000 7f98577fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565f6b4f240 0x5565f6b66ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:33.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.296+0000 7f98577fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565f6b4f240 0x5565f6b66ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60586/0 (socket says 172.21.15.120:60586) 2024-09-19T09:04:33.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.296+0000 7f98577fe640 1 -- 172.21.15.120:0/2271607462 learned_addr learned my addr 172.21.15.120:0/2271607462 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:33.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.296+0000 7f98567fc640 1 --2- 172.21.15.120:0/2271607462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565f6b44980 0x5565f6b3f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:33.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.296+0000 7f9856ffd640 1 --2- 172.21.15.120:0/2271607462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f6b44060 0x5565f6b46ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:33.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.296+0000 7f98577fe640 1 -- 172.21.15.120:0/2271607462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f6b44060 msgr2=0x5565f6b46ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:33.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.296+0000 7f98577fe640 1 --2- 172.21.15.120:0/2271607462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f6b44060 0x5565f6b46ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.296+0000 7f98577fe640 1 -- 172.21.15.120:0/2271607462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565f6b44980 msgr2=0x5565f6b3f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:33.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.296+0000 7f98577fe640 1 --2- 172.21.15.120:0/2271607462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565f6b44980 0x5565f6b3f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.296+0000 7f98577fe640 1 -- 172.21.15.120:0/2271607462 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565f6b447b0 con 0x5565f6b4f240 2024-09-19T09:04:33.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.296+0000 7f98567fc640 1 --2- 172.21.15.120:0/2271607462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565f6b44980 0x5565f6b3f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:04:33.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.296+0000 7f98577fe640 1 --2- 172.21.15.120:0/2271607462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565f6b4f240 0x5565f6b66ba0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9850009950 tx=0x7f9850034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adbdae66c915de2e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:33.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.296+0000 7f983cff9640 1 -- 172.21.15.120:0/2271607462 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9850034070 con 0x5565f6b4f240 2024-09-19T09:04:33.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.296+0000 7f983cff9640 1 -- 172.21.15.120:0/2271607462 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9850003e50 con 0x5565f6b4f240 2024-09-19T09:04:33.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.296+0000 7f983cff9640 1 -- 172.21.15.120:0/2271607462 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98500536a0 con 0x5565f6b4f240 2024-09-19T09:04:33.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.297+0000 7f985fbd1740 1 -- 172.21.15.120:0/2271607462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565f6b4f240 msgr2=0x5565f6b66ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:33.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.297+0000 7f985fbd1740 1 --2- 172.21.15.120:0/2271607462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565f6b4f240 0x5565f6b66ba0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9850009950 tx=0x7f9850034000 comp rx=0 tx=0).stop 2024-09-19T09:04:33.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.297+0000 7f985fbd1740 1 -- 172.21.15.120:0/2271607462 shutdown_connections 2024-09-19T09:04:33.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.297+0000 7f985fbd1740 1 --2- 172.21.15.120:0/2271607462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565f6b4f240 0x5565f6b66ba0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.297+0000 7f985fbd1740 1 --2- 172.21.15.120:0/2271607462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565f6b44980 0x5565f6b3f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.297+0000 7f985fbd1740 1 --2- 172.21.15.120:0/2271607462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f6b44060 0x5565f6b46ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.297+0000 7f985fbd1740 1 -- 172.21.15.120:0/2271607462 >> 172.21.15.120:0/2271607462 conn(0x5565f6b405f0 msgr2=0x5565f6b3ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:33.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.297+0000 7f985fbd1740 1 -- 172.21.15.120:0/2271607462 shutdown_connections 2024-09-19T09:04:33.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.297+0000 7f985fbd1740 1 -- 172.21.15.120:0/2271607462 wait complete. 2024-09-19T09:04:33.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.297+0000 7f985fbd1740 1 -- start start 2024-09-19T09:04:33.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.297+0000 7f985fbd1740 1 Processor -- start 2024-09-19T09:04:33.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.297+0000 7f985fbd1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565f6b44060 0x5565f6b532e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:33.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.297+0000 7f985fbd1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565f6b44980 0x5565f6b53820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:33.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.297+0000 7f985fbd1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f6b4f240 0x5565f6bc8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:33.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.297+0000 7f985fbd1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f98581aeb70 con 0x5565f6b44980 2024-09-19T09:04:33.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.297+0000 7f985fbd1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f98581ae4a0 con 0x5565f6b44060 2024-09-19T09:04:33.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.297+0000 7f985fbd1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f98581ab6c0 con 0x5565f6b4f240 2024-09-19T09:04:33.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.297+0000 7f985fbd1740 10 client.0 _ll_register_callbacks cb 0x7f984c011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:33.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.297+0000 7f98567fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565f6b44980 0x5565f6b53820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:33.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.297+0000 7f98567fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565f6b44980 0x5565f6b53820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60602/0 (socket says 172.21.15.120:60602) 2024-09-19T09:04:33.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.297+0000 7f98567fc640 1 -- 172.21.15.120:0/3584361695 learned_addr learned my addr 172.21.15.120:0/3584361695 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:33.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.298+0000 7f98577fe640 1 --2- 172.21.15.120:0/3584361695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f6b4f240 0x5565f6bc8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:33.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.298+0000 7f98567fc640 1 -- 172.21.15.120:0/3584361695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f6b4f240 msgr2=0x5565f6bc8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:33.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.298+0000 7f9856ffd640 1 --2- 172.21.15.120:0/3584361695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565f6b44060 0x5565f6b532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:33.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.298+0000 7f98567fc640 1 --2- 172.21.15.120:0/3584361695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f6b4f240 0x5565f6bc8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.298+0000 7f98567fc640 1 -- 172.21.15.120:0/3584361695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565f6b44060 msgr2=0x5565f6b532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:33.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.298+0000 7f98567fc640 1 --2- 172.21.15.120:0/3584361695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565f6b44060 0x5565f6b532e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.298+0000 7f98567fc640 1 -- 172.21.15.120:0/3584361695 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565f6b54100 con 0x5565f6b44980 2024-09-19T09:04:33.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.298+0000 7f98577fe640 1 --2- 172.21.15.120:0/3584361695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f6b4f240 0x5565f6bc8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:04:33.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.298+0000 7f9856ffd640 1 --2- 172.21.15.120:0/3584361695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565f6b44060 0x5565f6b532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:04:33.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.298+0000 7f98567fc640 1 --2- 172.21.15.120:0/3584361695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565f6b44980 0x5565f6b53820 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x5565f6b4d720 tx=0x7f984c00fea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aaf9886f1d185d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:33.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.298+0000 7f983aff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:04:33.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.298+0000 7f983aff5640 1 -- 172.21.15.120:0/3584361695 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c028070 con 0x5565f6b44980 2024-09-19T09:04:33.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.298+0000 7f983aff5640 1 -- 172.21.15.120:0/3584361695 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f984c004220 con 0x5565f6b44980 2024-09-19T09:04:33.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.298+0000 7f985fbd1740 1 -- 172.21.15.120:0/3584361695 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9850003e50 con 0x5565f6b44980 2024-09-19T09:04:33.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.298+0000 7f983aff5640 1 -- 172.21.15.120:0/3584361695 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f984c00f790 con 0x5565f6b44980 2024-09-19T09:04:33.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.298+0000 7f985fbd1740 10 client.15924 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:33.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.298+0000 7f985fbd1740 1 -- 172.21.15.120:0/3584361695 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5565f6bca170 con 0x5565f6b44980 2024-09-19T09:04:33.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.298+0000 7f983aff5640 1 -- 172.21.15.120:0/3584361695 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9850003e50 con 0x5565f6b44980 2024-09-19T09:04:33.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.299+0000 7f983aff5640 1 -- 172.21.15.120:0/3584361695 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f984c018040 con 0x5565f6b44980 2024-09-19T09:04:33.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.299+0000 7f985fbd1740 10 client.15924 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:33.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.299+0000 7f985fbd1740 10 client.15924 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:04:33.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.299+0000 7f985fbd1740 20 client.15924 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:33.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.299+0000 7f985fbd1740 1 --2- 172.21.15.120:0/3584361695 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5565f6bca910 0x5565f6bccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:33.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.299+0000 7f985fbd1740 4 client.15924 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "5c43545b", "release": true, "await": true}] 2024-09-19T09:04:33.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.299+0000 7f985fbd1740 1 -- 172.21.15.120:0/3584361695 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "set_id": "5c43545b", "release": true, "await": true}) -- 0x5565f6ab10a0 con 0x5565f6bca910 2024-09-19T09:04:33.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.299+0000 7f9856ffd640 1 --2- 172.21.15.120:0/3584361695 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5565f6bca910 0x5565f6bccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:33.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.299+0000 7f983aff5640 10 client.15924 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:33.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.299+0000 7f9856ffd640 1 --2- 172.21.15.120:0/3584361695 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5565f6bca910 0x5565f6bccd30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef726860427f5525 server_cookie=dc36f6c3482cd163 in_seq=0 out_seq=0 2024-09-19T09:04:33.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.310+0000 7f983aff5640 1 -- 172.21.15.120:0/3584361695 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5565f6ab10a0 con 0x5565f6bca910 2024-09-19T09:04:33.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.311+0000 7f983aff5640 10 client.15924 handle_command_reply: tid=0 2024-09-19T09:04:33.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.311+0000 7f985fbd1740 1 client.15924 shutdown 2024-09-19T09:04:33.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.311+0000 7f985fbd1740 1 -- 172.21.15.120:0/3584361695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565f6b44980 msgr2=0x5565f6b53820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:33.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.311+0000 7f985fbd1740 1 --2- 172.21.15.120:0/3584361695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565f6b44980 0x5565f6b53820 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x5565f6b4d720 tx=0x7f984c00fea0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.311+0000 7f985fbd1740 1 -- 172.21.15.120:0/3584361695 shutdown_connections 2024-09-19T09:04:33.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.311+0000 7f985fbd1740 1 --2- 172.21.15.120:0/3584361695 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5565f6bca910 0x5565f6bccd30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.311+0000 7f985fbd1740 1 --2- 172.21.15.120:0/3584361695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f6b4f240 0x5565f6bc8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.311+0000 7f985fbd1740 1 --2- 172.21.15.120:0/3584361695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5565f6b44980 0x5565f6b53820 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.311+0000 7f983aff5640 0 client.15924 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:33.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.311+0000 7f985fbd1740 1 --2- 172.21.15.120:0/3584361695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565f6b44060 0x5565f6b532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.311+0000 7f985fbd1740 1 -- 172.21.15.120:0/3584361695 >> 172.21.15.120:0/3584361695 conn(0x5565f6b405f0 msgr2=0x5565f6b3f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:33.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.311+0000 7f985fbd1740 1 -- 172.21.15.120:0/3584361695 shutdown_connections 2024-09-19T09:04:33.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.311+0000 7f985fbd1740 1 -- 172.21.15.120:0/3584361695 wait complete. 2024-09-19T09:04:33.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.311+0000 7f985fbd1740 20 client.15924 trim_cache size 0 max 16384 2024-09-19T09:04:33.332 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:04:33.332 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 28, 2024-09-19T09:04:33.332 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:04:33.332 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 8, 2024-09-19T09:04:33.332 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:04:33.332 INFO:teuthology.orchestra.run.smithi120.stdout: "5c43545b": { 2024-09-19T09:04:33.332 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 8, 2024-09-19T09:04:33.332 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:04:33.332 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:04:33.333 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:04:33.333 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:04:33.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:04:33.333 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:04:33.333 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 140.3, 2024-09-19T09:04:33.333 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:04:33.333 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:04:33.333 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:04:33.333 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:04:33.333 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:04:33.333 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:04:33.333 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:04:33.333 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:04:33.334 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:04:33.334 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:04:33.334 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:04:33.334 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:04:33.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.312+0000 7f985d7f8640 1 -- 172.21.15.120:0/2681375288 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f98400a3240 msgr2=0x7f98400a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:33.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.312+0000 7f985d7f8640 1 --2- 172.21.15.120:0/2681375288 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f98400a3240 0x7f98400a5700 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9850030040 tx=0x7f98500093a0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.312+0000 7f985d7f8640 1 -- 172.21.15.120:0/2681375288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9858100e20 msgr2=0x7f9858199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:33.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.312+0000 7f985d7f8640 1 --2- 172.21.15.120:0/2681375288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9858100e20 0x7f9858199370 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f984800d950 tx=0x7f984800de10 comp rx=0 tx=0).stop 2024-09-19T09:04:33.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.312+0000 7f985d7f8640 1 -- 172.21.15.120:0/2681375288 shutdown_connections 2024-09-19T09:04:33.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.312+0000 7f985d7f8640 1 --2- 172.21.15.120:0/2681375288 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f98400a3240 0x7f98400a5700 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.312+0000 7f985d7f8640 1 --2- 172.21.15.120:0/2681375288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9858102ef0 0x7f98581a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.312+0000 7f985d7f8640 1 --2- 172.21.15.120:0/2681375288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9858102570 0x7f98581998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.312+0000 7f985d7f8640 1 --2- 172.21.15.120:0/2681375288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9858100e20 0x7f9858199370 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:33.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.312+0000 7f985d7f8640 1 -- 172.21.15.120:0/2681375288 >> 172.21.15.120:0/2681375288 conn(0x7f98580fc7d0 msgr2=0x7f9858105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:33.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.312+0000 7f985d7f8640 1 -- 172.21.15.120:0/2681375288 shutdown_connections 2024-09-19T09:04:33.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:33.312+0000 7f985d7f8640 1 -- 172.21.15.120:0/2681375288 wait complete. 2024-09-19T09:04:33.808 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:04:33.809 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:04:34.035 INFO:tasks.workunit.client.0.smithi120.stdout: 1 14524 3.35 MB/sec warmup 29 sec latency 24769.606 ms 2024-09-19T09:04:34.085 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '5c43545b', total seconds elapsed: 28.5 2024-09-19T09:04:34.085 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.35 2024-09-19T09:04:34.085 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:04:34.149 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:04:34.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.655+0000 7f68d66e0640 1 -- 172.21.15.120:0/2038917456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68d0106f10 msgr2=0x7f68d010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:34.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.655+0000 7f68d66e0640 1 --2- 172.21.15.120:0/2038917456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68d0106f10 0x7f68d010ea90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f68c8009ca0 tx=0x7f68c802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.655+0000 7f68d66e0640 1 -- 172.21.15.120:0/2038917456 shutdown_connections 2024-09-19T09:04:34.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.655+0000 7f68d66e0640 1 --2- 172.21.15.120:0/2038917456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68d0106f10 0x7f68d010ea90 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.655+0000 7f68d66e0640 1 --2- 172.21.15.120:0/2038917456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68d0102790 0x7f68d01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.655+0000 7f68d66e0640 1 --2- 172.21.15.120:0/2038917456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68d0101de0 0x7f68d01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.655+0000 7f68d66e0640 1 -- 172.21.15.120:0/2038917456 >> 172.21.15.120:0/2038917456 conn(0x7f68d00fd8c0 msgr2=0x7f68d00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:34.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.655+0000 7f68d66e0640 1 -- 172.21.15.120:0/2038917456 shutdown_connections 2024-09-19T09:04:34.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.655+0000 7f68d66e0640 1 -- 172.21.15.120:0/2038917456 wait complete. 2024-09-19T09:04:34.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.656+0000 7f68d66e0640 1 Processor -- start 2024-09-19T09:04:34.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.656+0000 7f68d66e0640 1 -- start start 2024-09-19T09:04:34.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.656+0000 7f68d66e0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68d0101de0 0x7f68d01983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.656+0000 7f68d66e0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68d0102790 0x7f68d0198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.656+0000 7f68d66e0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68d0106f10 0x7f68d019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.656+0000 7f68d66e0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f68d0111f70 con 0x7f68d0102790 2024-09-19T09:04:34.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.656+0000 7f68d66e0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f68d0111df0 con 0x7f68d0106f10 2024-09-19T09:04:34.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.656+0000 7f68d66e0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f68d01120f0 con 0x7f68d0101de0 2024-09-19T09:04:34.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.656+0000 7f68c7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68d0102790 0x7f68d0198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.656+0000 7f68cffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68d0101de0 0x7f68d01983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.656+0000 7f68c7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68d0102790 0x7f68d0198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60608/0 (socket says 172.21.15.120:60608) 2024-09-19T09:04:34.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.656+0000 7f68c7fff640 1 -- 172.21.15.120:0/1629396431 learned_addr learned my addr 172.21.15.120:0/1629396431 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:34.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.656+0000 7f68d4c7d640 1 --2- 172.21.15.120:0/1629396431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68d0106f10 0x7f68d019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.657+0000 7f68cffff640 1 -- 172.21.15.120:0/1629396431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68d0106f10 msgr2=0x7f68d019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:34.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.657+0000 7f68cffff640 1 --2- 172.21.15.120:0/1629396431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68d0106f10 0x7f68d019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.657+0000 7f68cffff640 1 -- 172.21.15.120:0/1629396431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68d0102790 msgr2=0x7f68d0198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:34.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.657+0000 7f68cffff640 1 --2- 172.21.15.120:0/1629396431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68d0102790 0x7f68d0198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.657+0000 7f68cffff640 1 -- 172.21.15.120:0/1629396431 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d019d3a0 con 0x7f68d0101de0 2024-09-19T09:04:34.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.657+0000 7f68c7fff640 1 --2- 172.21.15.120:0/1629396431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68d0102790 0x7f68d0198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:04:34.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.657+0000 7f68cffff640 1 --2- 172.21.15.120:0/1629396431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68d0101de0 0x7f68d01983f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f68c000e9f0 tx=0x7f68c000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:34.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.657+0000 7f68cdffb640 1 -- 172.21.15.120:0/1629396431 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c000ccc0 con 0x7f68d0101de0 2024-09-19T09:04:34.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.657+0000 7f68d66e0640 1 -- 172.21.15.120:0/1629396431 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68d019d690 con 0x7f68d0101de0 2024-09-19T09:04:34.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.657+0000 7f68cdffb640 1 -- 172.21.15.120:0/1629396431 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68c0010910 con 0x7f68d0101de0 2024-09-19T09:04:34.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.657+0000 7f68cdffb640 1 -- 172.21.15.120:0/1629396431 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c0018960 con 0x7f68d0101de0 2024-09-19T09:04:34.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.657+0000 7f68d66e0640 1 -- 172.21.15.120:0/1629396431 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68d01a4f70 con 0x7f68d0101de0 2024-09-19T09:04:34.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.659+0000 7f68cdffb640 1 -- 172.21.15.120:0/1629396431 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f68c0010ac0 con 0x7f68d0101de0 2024-09-19T09:04:34.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.659+0000 7f68d8ab9740 1 Processor -- start 2024-09-19T09:04:34.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.660+0000 7f68d8ab9740 1 -- start start 2024-09-19T09:04:34.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.660+0000 7f68d8ab9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1970bcf70 0x55a1970bd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.660+0000 7f68d8ab9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1970bd890 0x55a1970c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.660+0000 7f68d8ab9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1970c8150 0x55a1970ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.660+0000 7f68d8ab9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a19701e280 con 0x55a1970c8150 2024-09-19T09:04:34.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.660+0000 7f68d8ab9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a196fc5b10 con 0x55a1970bd890 2024-09-19T09:04:34.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.660+0000 7f68d8ab9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a19701a240 con 0x55a1970bcf70 2024-09-19T09:04:34.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.660+0000 7f68c7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1970c8150 0x55a1970ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.660+0000 7f68d4c7d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1970bcf70 0x55a1970bd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.660+0000 7f68cffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1970bd890 0x55a1970c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.660+0000 7f68d4c7d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1970bcf70 0x55a1970bd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:40640/0 (socket says 172.21.15.120:40640) 2024-09-19T09:04:34.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.660+0000 7f68d4c7d640 1 -- 172.21.15.120:0/2505773933 learned_addr learned my addr 172.21.15.120:0/2505773933 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:34.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.660+0000 7f68c7fff640 1 -- 172.21.15.120:0/2505773933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1970bcf70 msgr2=0x55a1970bd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:34.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.660+0000 7f68c7fff640 1 --2- 172.21.15.120:0/2505773933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1970bcf70 0x55a1970bd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.660+0000 7f68c7fff640 1 -- 172.21.15.120:0/2505773933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1970bd890 msgr2=0x55a1970c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:34.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.660+0000 7f68c7fff640 1 --2- 172.21.15.120:0/2505773933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1970bd890 0x55a1970c7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.660+0000 7f68c7fff640 1 -- 172.21.15.120:0/2505773933 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a196fd21f0 con 0x55a1970c8150 2024-09-19T09:04:34.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.660+0000 7f68cdffb640 1 --2- 172.21.15.120:0/1629396431 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f68bc0a3240 0x7f68bc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.660+0000 7f68d4c7d640 1 --2- 172.21.15.120:0/2505773933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1970bcf70 0x55a1970bd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:04:34.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.660+0000 7f68d4c7d640 1 --2- 172.21.15.120:0/1629396431 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f68bc0a3240 0x7f68bc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.660+0000 7f68cdffb640 1 -- 172.21.15.120:0/1629396431 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f68c00cfca0 con 0x7f68d0101de0 2024-09-19T09:04:34.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.660+0000 7f68c7fff640 1 --2- 172.21.15.120:0/2505773933 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1970c8150 0x55a1970ca540 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f68d00fdce0 tx=0x7f68b80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d156b105dfbca59 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:34.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.661+0000 7f68c67fc640 1 -- 172.21.15.120:0/2505773933 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b801e070 con 0x55a1970c8150 2024-09-19T09:04:34.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.661+0000 7f68c67fc640 1 -- 172.21.15.120:0/2505773933 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68b80035f0 con 0x55a1970c8150 2024-09-19T09:04:34.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.661+0000 7f68c67fc640 1 -- 172.21.15.120:0/2505773933 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b800f800 con 0x55a1970c8150 2024-09-19T09:04:34.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.662+0000 7f68d8ab9740 1 -- 172.21.15.120:0/2505773933 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1970c8150 msgr2=0x55a1970ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:34.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.662+0000 7f68d4c7d640 1 --2- 172.21.15.120:0/1629396431 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f68bc0a3240 0x7f68bc0a5700 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f68c8004af0 tx=0x7f68c8004a80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:34.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.662+0000 7f68d8ab9740 1 --2- 172.21.15.120:0/2505773933 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1970c8150 0x55a1970ca540 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f68d00fdce0 tx=0x7f68b80118d0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.662+0000 7f68d8ab9740 1 -- 172.21.15.120:0/2505773933 shutdown_connections 2024-09-19T09:04:34.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.662+0000 7f68d8ab9740 1 --2- 172.21.15.120:0/2505773933 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1970c8150 0x55a1970ca540 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.662+0000 7f68d8ab9740 1 --2- 172.21.15.120:0/2505773933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1970bd890 0x55a1970c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.662+0000 7f68d8ab9740 1 --2- 172.21.15.120:0/2505773933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1970bcf70 0x55a1970bd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.662+0000 7f68d8ab9740 1 -- 172.21.15.120:0/2505773933 >> 172.21.15.120:0/2505773933 conn(0x55a1970b9500 msgr2=0x55a1970b98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:34.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.662+0000 7f68d8ab9740 1 -- 172.21.15.120:0/2505773933 shutdown_connections 2024-09-19T09:04:34.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.662+0000 7f68d8ab9740 1 -- 172.21.15.120:0/2505773933 wait complete. 2024-09-19T09:04:34.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.662+0000 7f68d8ab9740 1 -- start start 2024-09-19T09:04:34.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.662+0000 7f68d8ab9740 1 Processor -- start 2024-09-19T09:04:34.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.662+0000 7f68d8ab9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1970bcf70 0x55a1970c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.663+0000 7f68d8ab9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1970bd890 0x55a1970c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.663+0000 7f68d8ab9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1970c8150 0x55a1970c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.663+0000 7f68d8ab9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a1970cad20 con 0x55a1970bcf70 2024-09-19T09:04:34.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.663+0000 7f68d8ab9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a196fd8f10 con 0x55a1970bd890 2024-09-19T09:04:34.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.663+0000 7f68d8ab9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a1970caea0 con 0x55a1970c8150 2024-09-19T09:04:34.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.663+0000 7f68d8ab9740 10 client.0 _ll_register_callbacks cb 0x55a196fa1a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:34.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.663+0000 7f68cffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1970bcf70 0x55a1970c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.663+0000 7f68cffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1970bcf70 0x55a1970c50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60638/0 (socket says 172.21.15.120:60638) 2024-09-19T09:04:34.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.663+0000 7f68cffff640 1 -- 172.21.15.120:0/2717714215 learned_addr learned my addr 172.21.15.120:0/2717714215 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:34.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.663+0000 7f68c7fff640 1 --2- 172.21.15.120:0/2717714215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1970bd890 0x55a1970c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.663+0000 7f68d4c7d640 1 --2- 172.21.15.120:0/2717714215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1970c8150 0x55a1970c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.663+0000 7f68cffff640 1 -- 172.21.15.120:0/2717714215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1970c8150 msgr2=0x55a1970c5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:34.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.663+0000 7f68cffff640 1 --2- 172.21.15.120:0/2717714215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1970c8150 0x55a1970c5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.663+0000 7f68cffff640 1 -- 172.21.15.120:0/2717714215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1970bd890 msgr2=0x55a1970c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:34.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.663+0000 7f68cffff640 1 --2- 172.21.15.120:0/2717714215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1970bd890 0x55a1970c55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.663+0000 7f68cffff640 1 -- 172.21.15.120:0/2717714215 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a196d8f6a0 con 0x55a1970bcf70 2024-09-19T09:04:34.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.663+0000 7f68cffff640 1 --2- 172.21.15.120:0/2717714215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1970bcf70 0x55a1970c50b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f68c0098d40 tx=0x7f68c00cff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42a97413fba3b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:34.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.663+0000 7f68af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:04:34.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.663+0000 7f68af7fe640 1 -- 172.21.15.120:0/2717714215 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c009beb0 con 0x55a1970bcf70 2024-09-19T09:04:34.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.663+0000 7f68af7fe640 1 -- 172.21.15.120:0/2717714215 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f68c00bc0b0 con 0x55a1970bcf70 2024-09-19T09:04:34.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.663+0000 7f68d8ab9740 1 -- 172.21.15.120:0/2717714215 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68b80035f0 con 0x55a1970bcf70 2024-09-19T09:04:34.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.663+0000 7f68af7fe640 1 -- 172.21.15.120:0/2717714215 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f68c009bcd0 con 0x55a1970bcf70 2024-09-19T09:04:34.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.663+0000 7f68d8ab9740 10 client.15936 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:34.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.663+0000 7f68d8ab9740 1 -- 172.21.15.120:0/2717714215 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a1970d69a0 con 0x55a1970bcf70 2024-09-19T09:04:34.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.664+0000 7f68af7fe640 1 -- 172.21.15.120:0/2717714215 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68b80035f0 con 0x55a1970bcf70 2024-09-19T09:04:34.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.664+0000 7f68af7fe640 1 -- 172.21.15.120:0/2717714215 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f68c009b7c0 con 0x55a1970bcf70 2024-09-19T09:04:34.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.664+0000 7f68d8ab9740 10 client.15936 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:34.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.664+0000 7f68d8ab9740 10 client.15936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:04:34.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.664+0000 7f68d8ab9740 20 client.15936 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:34.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.664+0000 7f68d8ab9740 1 --2- 172.21.15.120:0/2717714215 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a1970d7e60 0x55a1970da270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.664+0000 7f68d8ab9740 4 client.15936 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:04:34.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.664+0000 7f68d8ab9740 1 -- 172.21.15.120:0/2717714215 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a19702b7f0 con 0x55a1970d7e60 2024-09-19T09:04:34.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.664+0000 7f68c7fff640 1 --2- 172.21.15.120:0/2717714215 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a1970d7e60 0x55a1970da270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.664+0000 7f68af7fe640 10 client.15936 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:34.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.664+0000 7f68c7fff640 1 --2- 172.21.15.120:0/2717714215 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a1970d7e60 0x55a1970da270 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2ba12f4a18000d1 server_cookie=1205e12c37ed3081 in_seq=0 out_seq=0 2024-09-19T09:04:34.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.666+0000 7f68af7fe640 1 -- 172.21.15.120:0/2717714215 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a19702b7f0 con 0x55a1970d7e60 2024-09-19T09:04:34.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.666+0000 7f68af7fe640 10 client.15936 handle_command_reply: tid=0 2024-09-19T09:04:34.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.666+0000 7f68d8ab9740 1 client.15936 shutdown 2024-09-19T09:04:34.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.666+0000 7f68d8ab9740 1 -- 172.21.15.120:0/2717714215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1970bcf70 msgr2=0x55a1970c50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:34.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.666+0000 7f68d8ab9740 1 --2- 172.21.15.120:0/2717714215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1970bcf70 0x55a1970c50b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f68c0098d40 tx=0x7f68c00cff20 comp rx=0 tx=0).stop 2024-09-19T09:04:34.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.666+0000 7f68d8ab9740 1 -- 172.21.15.120:0/2717714215 shutdown_connections 2024-09-19T09:04:34.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.666+0000 7f68d8ab9740 1 --2- 172.21.15.120:0/2717714215 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a1970d7e60 0x55a1970da270 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.666+0000 7f68d8ab9740 1 --2- 172.21.15.120:0/2717714215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1970c8150 0x55a1970c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.666+0000 7f68d8ab9740 1 --2- 172.21.15.120:0/2717714215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1970bd890 0x55a1970c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.666+0000 7f68d8ab9740 1 --2- 172.21.15.120:0/2717714215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1970bcf70 0x55a1970c50b0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.666+0000 7f68d8ab9740 1 -- 172.21.15.120:0/2717714215 >> 172.21.15.120:0/2717714215 conn(0x55a1970b9500 msgr2=0x55a1970b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:34.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.666+0000 7f68af7fe640 0 client.15936 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:34.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.666+0000 7f68d8ab9740 1 -- 172.21.15.120:0/2717714215 shutdown_connections 2024-09-19T09:04:34.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.666+0000 7f68d8ab9740 1 -- 172.21.15.120:0/2717714215 wait complete. 2024-09-19T09:04:34.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.666+0000 7f68d8ab9740 20 client.15936 trim_cache size 0 max 16384 2024-09-19T09:04:34.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.684+0000 7f68d8ab9740 1 Processor -- start 2024-09-19T09:04:34.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.684+0000 7f68d8ab9740 1 -- start start 2024-09-19T09:04:34.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.684+0000 7f68d8ab9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1970bcf70 0x55a1970bf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.684+0000 7f68d8ab9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1970bd890 0x55a1970dfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.684+0000 7f68d8ab9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1970c8150 0x55a1970e0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.684+0000 7f68d8ab9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f68d01ab470 con 0x55a1970bcf70 2024-09-19T09:04:34.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.684+0000 7f68d8ab9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f68d01acfb0 con 0x55a1970bd890 2024-09-19T09:04:34.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.684+0000 7f68d8ab9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f68d01abb40 con 0x55a1970c8150 2024-09-19T09:04:34.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.685+0000 7f68cffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1970bcf70 0x55a1970bf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.685+0000 7f68d4c7d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1970c8150 0x55a1970e0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.685+0000 7f68d4c7d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1970c8150 0x55a1970e0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:40646/0 (socket says 172.21.15.120:40646) 2024-09-19T09:04:34.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.685+0000 7f68c7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1970bd890 0x55a1970dfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.685+0000 7f68d4c7d640 1 -- 172.21.15.120:0/3275910874 learned_addr learned my addr 172.21.15.120:0/3275910874 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:34.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.685+0000 7f68d4c7d640 1 -- 172.21.15.120:0/3275910874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1970bd890 msgr2=0x55a1970dfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:34.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.685+0000 7f68d4c7d640 1 --2- 172.21.15.120:0/3275910874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1970bd890 0x55a1970dfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.685+0000 7f68d4c7d640 1 -- 172.21.15.120:0/3275910874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1970bcf70 msgr2=0x55a1970bf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:34.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.685+0000 7f68d4c7d640 1 --2- 172.21.15.120:0/3275910874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1970bcf70 0x55a1970bf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.685+0000 7f68d4c7d640 1 -- 172.21.15.120:0/3275910874 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1970c4060 con 0x55a1970c8150 2024-09-19T09:04:34.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.685+0000 7f68cffff640 1 --2- 172.21.15.120:0/3275910874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1970bcf70 0x55a1970bf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:04:34.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.685+0000 7f68d4c7d640 1 --2- 172.21.15.120:0/3275910874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1970c8150 0x55a1970e0080 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f68c802f540 tx=0x7f68c8041000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e789c0426b375a5c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:34.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.685+0000 7f68c5ffb640 1 -- 172.21.15.120:0/3275910874 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c8031920 con 0x55a1970c8150 2024-09-19T09:04:34.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.686+0000 7f68c5ffb640 1 -- 172.21.15.120:0/3275910874 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68c8036040 con 0x55a1970c8150 2024-09-19T09:04:34.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.686+0000 7f68c5ffb640 1 -- 172.21.15.120:0/3275910874 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c802f950 con 0x55a1970c8150 2024-09-19T09:04:34.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.686+0000 7f68d8ab9740 1 -- 172.21.15.120:0/3275910874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1970c8150 msgr2=0x55a1970e0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:34.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.686+0000 7f68d8ab9740 1 --2- 172.21.15.120:0/3275910874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1970c8150 0x55a1970e0080 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f68c802f540 tx=0x7f68c8041000 comp rx=0 tx=0).stop 2024-09-19T09:04:34.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.686+0000 7f68d8ab9740 1 -- 172.21.15.120:0/3275910874 shutdown_connections 2024-09-19T09:04:34.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.686+0000 7f68d8ab9740 1 --2- 172.21.15.120:0/3275910874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1970c8150 0x55a1970e0080 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.686+0000 7f68d8ab9740 1 --2- 172.21.15.120:0/3275910874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1970bd890 0x55a1970dfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.686+0000 7f68d8ab9740 1 --2- 172.21.15.120:0/3275910874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1970bcf70 0x55a1970bf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.686+0000 7f68d8ab9740 1 -- 172.21.15.120:0/3275910874 >> 172.21.15.120:0/3275910874 conn(0x55a1970b9500 msgr2=0x55a1970be9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:34.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.687+0000 7f68d8ab9740 1 -- 172.21.15.120:0/3275910874 shutdown_connections 2024-09-19T09:04:34.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.687+0000 7f68d8ab9740 1 -- 172.21.15.120:0/3275910874 wait complete. 2024-09-19T09:04:34.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.687+0000 7f68d8ab9740 1 -- start start 2024-09-19T09:04:34.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.687+0000 7f68d8ab9740 1 Processor -- start 2024-09-19T09:04:34.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.688+0000 7f68d8ab9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1970bcf70 0x55a1970cc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.688+0000 7f68d8ab9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1970bd890 0x55a1970ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.688+0000 7f68d8ab9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1970c8150 0x55a197141b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.688+0000 7f68d8ab9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f68d01ac8e0 con 0x55a1970bd890 2024-09-19T09:04:34.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.688+0000 7f68d8ab9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f68d01ac210 con 0x55a1970bcf70 2024-09-19T09:04:34.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.688+0000 7f68d8ab9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f68d01a9430 con 0x55a1970c8150 2024-09-19T09:04:34.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.688+0000 7f68d8ab9740 10 client.0 _ll_register_callbacks cb 0x7f68c000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:34.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.688+0000 7f68cffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1970bcf70 0x55a1970cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.688+0000 7f68cffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1970bcf70 0x55a1970cc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:46674/0 (socket says 172.21.15.120:46674) 2024-09-19T09:04:34.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.688+0000 7f68cffff640 1 -- 172.21.15.120:0/1878897181 learned_addr learned my addr 172.21.15.120:0/1878897181 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:34.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.688+0000 7f68c7fff640 1 --2- 172.21.15.120:0/1878897181 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1970bd890 0x55a1970ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.688+0000 7f68d4c7d640 1 --2- 172.21.15.120:0/1878897181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1970c8150 0x55a197141b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.688+0000 7f68cffff640 1 -- 172.21.15.120:0/1878897181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1970c8150 msgr2=0x55a197141b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:34.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.688+0000 7f68cffff640 1 --2- 172.21.15.120:0/1878897181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1970c8150 0x55a197141b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.688+0000 7f68cffff640 1 -- 172.21.15.120:0/1878897181 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1970bd890 msgr2=0x55a1970ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:34.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.688+0000 7f68cffff640 1 --2- 172.21.15.120:0/1878897181 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1970bd890 0x55a1970ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.688+0000 7f68cffff640 1 -- 172.21.15.120:0/1878897181 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1970b0830 con 0x55a1970bcf70 2024-09-19T09:04:34.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.689+0000 7f68cffff640 1 --2- 172.21.15.120:0/1878897181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1970bcf70 0x55a1970cc580 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f68c00987d0 tx=0x7f68c000c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ca41db6a2a7560f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:34.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.689+0000 7f68af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:04:34.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.689+0000 7f68af7fe640 1 -- 172.21.15.120:0/1878897181 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c009beb0 con 0x55a1970bcf70 2024-09-19T09:04:34.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.689+0000 7f68af7fe640 1 -- 172.21.15.120:0/1878897181 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f68c0010430 con 0x55a1970bcf70 2024-09-19T09:04:34.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.689+0000 7f68d8ab9740 1 -- 172.21.15.120:0/1878897181 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68c8036040 con 0x55a1970bcf70 2024-09-19T09:04:34.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.689+0000 7f68af7fe640 1 -- 172.21.15.120:0/1878897181 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f68c00d31f0 con 0x55a1970bcf70 2024-09-19T09:04:34.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.689+0000 7f68d8ab9740 10 client.25316 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:34.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.689+0000 7f68d8ab9740 1 -- 172.21.15.120:0/1878897181 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a1970b0830 con 0x55a1970bcf70 2024-09-19T09:04:34.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.689+0000 7f68af7fe640 1 -- 172.21.15.120:0/1878897181 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68c8036040 con 0x55a1970bcf70 2024-09-19T09:04:34.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.690+0000 7f68af7fe640 1 -- 172.21.15.120:0/1878897181 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f68c009b230 con 0x55a1970bcf70 2024-09-19T09:04:34.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.690+0000 7f68d8ab9740 10 client.25316 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:34.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.690+0000 7f68d8ab9740 10 client.25316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:04:34.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.690+0000 7f68d8ab9740 20 client.25316 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:34.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.690+0000 7f68d8ab9740 1 --2- 172.21.15.120:0/1878897181 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a1971435d0 0x55a1971459c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.690+0000 7f68d8ab9740 4 client.25316 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:04:34.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.690+0000 7f68d8ab9740 1 -- 172.21.15.120:0/1878897181 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a197025d00 con 0x55a1971435d0 2024-09-19T09:04:34.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.690+0000 7f68c7fff640 1 --2- 172.21.15.120:0/1878897181 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a1971435d0 0x55a1971459c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.690+0000 7f68c7fff640 1 --2- 172.21.15.120:0/1878897181 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a1971435d0 0x55a1971459c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd80d693d50f2611 server_cookie=9f0f60c058f6e7d8 in_seq=0 out_seq=0 2024-09-19T09:04:34.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.690+0000 7f68af7fe640 10 client.25316 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:34.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.691+0000 7f68af7fe640 1 -- 172.21.15.120:0/1878897181 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a197025d00 con 0x55a1971435d0 2024-09-19T09:04:34.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.691+0000 7f68af7fe640 10 client.25316 handle_command_reply: tid=0 2024-09-19T09:04:34.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.691+0000 7f68d8ab9740 1 client.25316 shutdown 2024-09-19T09:04:34.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.691+0000 7f68d8ab9740 1 -- 172.21.15.120:0/1878897181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1970bcf70 msgr2=0x55a1970cc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:34.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.691+0000 7f68d8ab9740 1 --2- 172.21.15.120:0/1878897181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1970bcf70 0x55a1970cc580 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f68c00987d0 tx=0x7f68c000c840 comp rx=0 tx=0).stop 2024-09-19T09:04:34.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.691+0000 7f68d8ab9740 1 -- 172.21.15.120:0/1878897181 shutdown_connections 2024-09-19T09:04:34.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.691+0000 7f68d8ab9740 1 --2- 172.21.15.120:0/1878897181 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a1971435d0 0x55a1971459c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.692+0000 7f68d8ab9740 1 --2- 172.21.15.120:0/1878897181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1970c8150 0x55a197141b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.692+0000 7f68d8ab9740 1 --2- 172.21.15.120:0/1878897181 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1970bd890 0x55a1970ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.692+0000 7f68d8ab9740 1 --2- 172.21.15.120:0/1878897181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1970bcf70 0x55a1970cc580 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.692+0000 7f68d8ab9740 1 -- 172.21.15.120:0/1878897181 >> 172.21.15.120:0/1878897181 conn(0x55a1970b9500 msgr2=0x55a1970c5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:34.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.692+0000 7f68af7fe640 0 client.25316 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:34.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.692+0000 7f68d8ab9740 1 -- 172.21.15.120:0/1878897181 shutdown_connections 2024-09-19T09:04:34.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.692+0000 7f68d8ab9740 1 -- 172.21.15.120:0/1878897181 wait complete. 2024-09-19T09:04:34.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.692+0000 7f68d8ab9740 20 client.25316 trim_cache size 0 max 16384 2024-09-19T09:04:34.697 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:04:34.697 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:04:34.697 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "7e98f4c4-fa37-4a87-9a31-a6b946fc787d", 2024-09-19T09:04:34.697 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:04:34.697 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:04:34.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.692+0000 7f68d66e0640 1 -- 172.21.15.120:0/1629396431 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f68bc0a3240 msgr2=0x7f68bc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:34.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.692+0000 7f68d66e0640 1 --2- 172.21.15.120:0/1629396431 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f68bc0a3240 0x7f68bc0a5700 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f68c8004af0 tx=0x7f68c8004a80 comp rx=0 tx=0).stop 2024-09-19T09:04:34.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.692+0000 7f68d66e0640 1 -- 172.21.15.120:0/1629396431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68d0101de0 msgr2=0x7f68d01983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:34.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.692+0000 7f68d66e0640 1 --2- 172.21.15.120:0/1629396431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68d0101de0 0x7f68d01983f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f68c000e9f0 tx=0x7f68c000eeb0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.693+0000 7f68d66e0640 1 -- 172.21.15.120:0/1629396431 shutdown_connections 2024-09-19T09:04:34.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.693+0000 7f68d66e0640 1 --2- 172.21.15.120:0/1629396431 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f68bc0a3240 0x7f68bc0a5700 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.693+0000 7f68d66e0640 1 --2- 172.21.15.120:0/1629396431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68d0106f10 0x7f68d019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.693+0000 7f68d66e0640 1 --2- 172.21.15.120:0/1629396431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68d0102790 0x7f68d0198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.693+0000 7f68d66e0640 1 --2- 172.21.15.120:0/1629396431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68d0101de0 0x7f68d01983f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.693+0000 7f68d66e0640 1 -- 172.21.15.120:0/1629396431 >> 172.21.15.120:0/1629396431 conn(0x7f68d00fd8c0 msgr2=0x7f68d00feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:34.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.693+0000 7f68d66e0640 1 -- 172.21.15.120:0/1629396431 shutdown_connections 2024-09-19T09:04:34.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:34.693+0000 7f68d66e0640 1 -- 172.21.15.120:0/1629396431 wait complete. 2024-09-19T09:04:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.841+0000 7f7412b86640 1 -- 172.21.15.120:0/443658011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f740c1025f0 msgr2=0x7f740c108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.841+0000 7f7412b86640 1 --2- 172.21.15.120:0/443658011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f740c1025f0 0x7f740c108e80 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7404009ca0 tx=0x7f740402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.841+0000 7f7412b86640 1 -- 172.21.15.120:0/443658011 shutdown_connections 2024-09-19T09:04:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.841+0000 7f7412b86640 1 --2- 172.21.15.120:0/443658011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f740c1025f0 0x7f740c108e80 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.841+0000 7f7412b86640 1 --2- 172.21.15.120:0/443658011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f740c101c30 0x7f740c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.841+0000 7f7412b86640 1 --2- 172.21.15.120:0/443658011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c100a40 0x7f740c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.841+0000 7f7412b86640 1 -- 172.21.15.120:0/443658011 >> 172.21.15.120:0/443658011 conn(0x7f740c0fc1e0 msgr2=0x7f740c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.842+0000 7f7412b86640 1 -- 172.21.15.120:0/443658011 shutdown_connections 2024-09-19T09:04:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.842+0000 7f7412b86640 1 -- 172.21.15.120:0/443658011 wait complete. 2024-09-19T09:04:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.842+0000 7f7412b86640 1 Processor -- start 2024-09-19T09:04:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.842+0000 7f7412b86640 1 -- start start 2024-09-19T09:04:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.842+0000 7f7412b86640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c100a40 0x7f740c06a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.842+0000 7f7412b86640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f740c101c30 0x7f740c19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.842+0000 7f7412b86640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f740c1025f0 0x7f740c19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.842+0000 7f7412b86640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f740c10c6c0 con 0x7f740c1025f0 2024-09-19T09:04:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.842+0000 7f7412b86640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f740c10c540 con 0x7f740c101c30 2024-09-19T09:04:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.842+0000 7f7412b86640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f740c10c840 con 0x7f740c100a40 2024-09-19T09:04:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.842+0000 7f7411123640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f740c1025f0 0x7f740c19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.842+0000 7f7410922640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c100a40 0x7f740c06a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.842+0000 7f7411123640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f740c1025f0 0x7f740c19f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60672/0 (socket says 172.21.15.120:60672) 2024-09-19T09:04:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.842+0000 7f7411123640 1 -- 172.21.15.120:0/3607008798 learned_addr learned my addr 172.21.15.120:0/3607008798 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.842+0000 7f740bfff640 1 --2- 172.21.15.120:0/3607008798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f740c101c30 0x7f740c19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.843+0000 7f7410922640 1 -- 172.21.15.120:0/3607008798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f740c101c30 msgr2=0x7f740c19cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.843+0000 7f7410922640 1 --2- 172.21.15.120:0/3607008798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f740c101c30 0x7f740c19cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.843+0000 7f7410922640 1 -- 172.21.15.120:0/3607008798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f740c1025f0 msgr2=0x7f740c19f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.843+0000 7f7410922640 1 --2- 172.21.15.120:0/3607008798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f740c1025f0 0x7f740c19f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.843+0000 7f7410922640 1 -- 172.21.15.120:0/3607008798 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740c19fa10 con 0x7f740c100a40 2024-09-19T09:04:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.843+0000 7f7410922640 1 --2- 172.21.15.120:0/3607008798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c100a40 0x7f740c06a410 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f740000e9c0 tx=0x7f740000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.843+0000 7f7409ffb640 1 -- 172.21.15.120:0/3607008798 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740000ccc0 con 0x7f740c100a40 2024-09-19T09:04:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.843+0000 7f7412b86640 1 -- 172.21.15.120:0/3607008798 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f740c19fca0 con 0x7f740c100a40 2024-09-19T09:04:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.843+0000 7f7409ffb640 1 -- 172.21.15.120:0/3607008798 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74000035f0 con 0x7f740c100a40 2024-09-19T09:04:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.843+0000 7f7409ffb640 1 -- 172.21.15.120:0/3607008798 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740000ce60 con 0x7f740c100a40 2024-09-19T09:04:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.843+0000 7f7412b86640 1 -- 172.21.15.120:0/3607008798 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f740c1a01a0 con 0x7f740c100a40 2024-09-19T09:04:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.845+0000 7f7409ffb640 1 -- 172.21.15.120:0/3607008798 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f74000037a0 con 0x7f740c100a40 2024-09-19T09:04:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.846+0000 7f74159f7740 1 Processor -- start 2024-09-19T09:04:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.846+0000 7f74159f7740 1 -- start start 2024-09-19T09:04:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.846+0000 7f7409ffb640 1 --2- 172.21.15.120:0/3607008798 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f73fc0a32f0 0x7f73fc0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.846+0000 7f74159f7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3999439a0 0x55a399943da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.846+0000 7f74159f7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3999442e0 0x55a399948da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.846+0000 7f74159f7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a399949370 0x55a39994b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.846+0000 7f74159f7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a3998a72f0 con 0x55a3999439a0 2024-09-19T09:04:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.846+0000 7f74159f7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a39984f270 con 0x55a399949370 2024-09-19T09:04:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.846+0000 7f74159f7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a3998a32b0 con 0x55a3999442e0 2024-09-19T09:04:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.846+0000 7f7410922640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3999439a0 0x55a399943da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.846+0000 7f7411123640 1 --2- 172.21.15.120:0/3607008798 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f73fc0a32f0 0x7f73fc0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.846+0000 7f740bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3999442e0 0x55a399948da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.846+0000 7f740bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3999442e0 0x55a399948da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:40688/0 (socket says 172.21.15.120:40688) 2024-09-19T09:04:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.846+0000 7f740bfff640 1 -- 172.21.15.120:0/2495943273 learned_addr learned my addr 172.21.15.120:0/2495943273 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.846+0000 7f7409ffb640 1 -- 172.21.15.120:0/3607008798 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f74000cfe10 con 0x7f740c100a40 2024-09-19T09:04:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.846+0000 7f7411123640 1 --2- 172.21.15.120:0/2495943273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a399949370 0x55a39994b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.846+0000 7f7410922640 1 -- 172.21.15.120:0/2495943273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3999442e0 msgr2=0x55a399948da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.846+0000 7f7410922640 1 --2- 172.21.15.120:0/2495943273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3999442e0 0x55a399948da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.846+0000 7f7410922640 1 -- 172.21.15.120:0/2495943273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a399949370 msgr2=0x55a39994b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.846+0000 7f7410922640 1 --2- 172.21.15.120:0/2495943273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a399949370 0x55a39994b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.846+0000 7f7410922640 1 -- 172.21.15.120:0/2495943273 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a39984e340 con 0x55a3999439a0 2024-09-19T09:04:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.846+0000 7f740bfff640 1 --2- 172.21.15.120:0/2495943273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3999442e0 0x55a399948da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:04:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.846+0000 7f7411123640 1 --2- 172.21.15.120:0/2495943273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a399949370 0x55a39994b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:04:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.847+0000 7f7410922640 1 --2- 172.21.15.120:0/2495943273 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3999439a0 0x55a399943da0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7400098890 tx=0x7f7400091910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee400335a029147c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.847+0000 7f7411123640 1 --2- 172.21.15.120:0/3607008798 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f73fc0a32f0 0x7f73fc0a57b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f740402f540 tx=0x7f7404005df0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.847+0000 7f73f6ffd640 1 -- 172.21.15.120:0/2495943273 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740009a9f0 con 0x55a3999439a0 2024-09-19T09:04:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.847+0000 7f73f6ffd640 1 -- 172.21.15.120:0/2495943273 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f740009cb30 con 0x55a3999439a0 2024-09-19T09:04:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.847+0000 7f73f6ffd640 1 -- 172.21.15.120:0/2495943273 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740009a570 con 0x55a3999439a0 2024-09-19T09:04:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.847+0000 7f74159f7740 1 -- 172.21.15.120:0/2495943273 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3999439a0 msgr2=0x55a399943da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.847+0000 7f74159f7740 1 --2- 172.21.15.120:0/2495943273 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3999439a0 0x55a399943da0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7400098890 tx=0x7f7400091910 comp rx=0 tx=0).stop 2024-09-19T09:04:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.847+0000 7f74159f7740 1 -- 172.21.15.120:0/2495943273 shutdown_connections 2024-09-19T09:04:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.847+0000 7f74159f7740 1 --2- 172.21.15.120:0/2495943273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a399949370 0x55a39994b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.847+0000 7f74159f7740 1 --2- 172.21.15.120:0/2495943273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3999442e0 0x55a399948da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.847+0000 7f74159f7740 1 --2- 172.21.15.120:0/2495943273 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3999439a0 0x55a399943da0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.847+0000 7f74159f7740 1 -- 172.21.15.120:0/2495943273 >> 172.21.15.120:0/2495943273 conn(0x55a39993f010 msgr2=0x55a39993f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.848+0000 7f74159f7740 1 -- 172.21.15.120:0/2495943273 shutdown_connections 2024-09-19T09:04:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.848+0000 7f74159f7740 1 -- 172.21.15.120:0/2495943273 wait complete. 2024-09-19T09:04:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.848+0000 7f74159f7740 1 -- start start 2024-09-19T09:04:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.848+0000 7f74159f7740 1 Processor -- start 2024-09-19T09:04:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.848+0000 7f74159f7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3999439a0 0x55a39994dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.848+0000 7f74159f7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3999442e0 0x55a39994e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.848+0000 7f74159f7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a399949370 0x55a399955230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.848+0000 7f74159f7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a39994c340 con 0x55a3999442e0 2024-09-19T09:04:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.848+0000 7f74159f7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a399830290 con 0x55a399949370 2024-09-19T09:04:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.848+0000 7f74159f7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a39994c4c0 con 0x55a3999439a0 2024-09-19T09:04:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.848+0000 7f74159f7740 10 client.0 _ll_register_callbacks cb 0x55a399861140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.848+0000 7f740bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3999442e0 0x55a39994e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.848+0000 7f7410922640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3999439a0 0x55a39994dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.848+0000 7f740bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3999442e0 0x55a39994e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60702/0 (socket says 172.21.15.120:60702) 2024-09-19T09:04:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.848+0000 7f740bfff640 1 -- 172.21.15.120:0/4194614297 learned_addr learned my addr 172.21.15.120:0/4194614297 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.848+0000 7f7411123640 1 --2- 172.21.15.120:0/4194614297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a399949370 0x55a399955230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.849+0000 7f740bfff640 1 -- 172.21.15.120:0/4194614297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3999439a0 msgr2=0x55a39994dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.849+0000 7f740bfff640 1 --2- 172.21.15.120:0/4194614297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3999439a0 0x55a39994dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.849+0000 7f740bfff640 1 -- 172.21.15.120:0/4194614297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a399949370 msgr2=0x55a399955230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.849+0000 7f740bfff640 1 --2- 172.21.15.120:0/4194614297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a399949370 0x55a399955230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.849+0000 7f740bfff640 1 -- 172.21.15.120:0/4194614297 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3995ba210 con 0x55a3999442e0 2024-09-19T09:04:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.849+0000 7f740bfff640 1 --2- 172.21.15.120:0/4194614297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3999442e0 0x55a39994e1f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f73f8012720 tx=0x7f73f8012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41cbd9c1d55c4085 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.849+0000 7f73ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:04:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.849+0000 7f73ebfff640 1 -- 172.21.15.120:0/4194614297 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8004050 con 0x55a3999442e0 2024-09-19T09:04:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.849+0000 7f73ebfff640 1 -- 172.21.15.120:0/4194614297 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f73f80041f0 con 0x55a3999442e0 2024-09-19T09:04:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.849+0000 7f74159f7740 1 -- 172.21.15.120:0/4194614297 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f740009cb30 con 0x55a3999442e0 2024-09-19T09:04:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.849+0000 7f73ebfff640 1 -- 172.21.15.120:0/4194614297 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f73f8004430 con 0x55a3999442e0 2024-09-19T09:04:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.849+0000 7f74159f7740 10 client.15954 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.849+0000 7f74159f7740 1 -- 172.21.15.120:0/4194614297 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a399686620 con 0x55a3999442e0 2024-09-19T09:04:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.849+0000 7f73ebfff640 1 -- 172.21.15.120:0/4194614297 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f740009cb30 con 0x55a3999442e0 2024-09-19T09:04:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.849+0000 7f73ebfff640 1 -- 172.21.15.120:0/4194614297 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f73f8003c50 con 0x55a3999442e0 2024-09-19T09:04:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.850+0000 7f74159f7740 10 client.15954 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.850+0000 7f74159f7740 10 client.15954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:04:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.850+0000 7f74159f7740 20 client.15954 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.850+0000 7f74159f7740 1 --2- 172.21.15.120:0/4194614297 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a399959920 0x55a39995bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.850+0000 7f74159f7740 4 client.15954 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:04:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.850+0000 7f74159f7740 1 -- 172.21.15.120:0/4194614297 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3998b4a20 con 0x55a399959920 2024-09-19T09:04:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.850+0000 7f7410922640 1 --2- 172.21.15.120:0/4194614297 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a399959920 0x55a39995bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.850+0000 7f73ebfff640 10 client.15954 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.850+0000 7f7410922640 1 --2- 172.21.15.120:0/4194614297 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a399959920 0x55a39995bd50 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=204f870589157c88 server_cookie=cd587d7823cc0fcf in_seq=0 out_seq=0 2024-09-19T09:04:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.851+0000 7f73ebfff640 1 -- 172.21.15.120:0/4194614297 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a3998b4a20 con 0x55a399959920 2024-09-19T09:04:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.851+0000 7f73ebfff640 10 client.15954 handle_command_reply: tid=0 2024-09-19T09:04:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.851+0000 7f74159f7740 1 client.15954 shutdown 2024-09-19T09:04:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.851+0000 7f74159f7740 1 -- 172.21.15.120:0/4194614297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3999442e0 msgr2=0x55a39994e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.851+0000 7f74159f7740 1 --2- 172.21.15.120:0/4194614297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3999442e0 0x55a39994e1f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f73f8012720 tx=0x7f73f8012be0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.851+0000 7f74159f7740 1 -- 172.21.15.120:0/4194614297 shutdown_connections 2024-09-19T09:04:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.851+0000 7f74159f7740 1 --2- 172.21.15.120:0/4194614297 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a399959920 0x55a39995bd50 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.851+0000 7f74159f7740 1 --2- 172.21.15.120:0/4194614297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a399949370 0x55a399955230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.851+0000 7f74159f7740 1 --2- 172.21.15.120:0/4194614297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3999442e0 0x55a39994e1f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.851+0000 7f74159f7740 1 --2- 172.21.15.120:0/4194614297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3999439a0 0x55a39994dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.851+0000 7f74159f7740 1 -- 172.21.15.120:0/4194614297 >> 172.21.15.120:0/4194614297 conn(0x55a39993f010 msgr2=0x55a399940820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.851+0000 7f73ebfff640 0 client.15954 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.851+0000 7f74159f7740 1 -- 172.21.15.120:0/4194614297 shutdown_connections 2024-09-19T09:04:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.851+0000 7f74159f7740 1 -- 172.21.15.120:0/4194614297 wait complete. 2024-09-19T09:04:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.851+0000 7f74159f7740 20 client.15954 trim_cache size 0 max 16384 2024-09-19T09:04:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.866+0000 7f74159f7740 1 Processor -- start 2024-09-19T09:04:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.866+0000 7f74159f7740 1 -- start start 2024-09-19T09:04:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.866+0000 7f74159f7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3999439a0 0x55a39993f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.866+0000 7f74159f7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3999442e0 0x55a3999551f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.866+0000 7f74159f7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a399949370 0x55a39993f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.866+0000 7f74159f7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f740c1a8f70 con 0x55a3999439a0 2024-09-19T09:04:34.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.866+0000 7f74159f7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f740c1aaab0 con 0x55a399949370 2024-09-19T09:04:34.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.866+0000 7f74159f7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f740c1a9640 con 0x55a3999442e0 2024-09-19T09:04:34.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.866+0000 7f7410922640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3999439a0 0x55a39993f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.866+0000 7f7410922640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3999439a0 0x55a39993f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60708/0 (socket says 172.21.15.120:60708) 2024-09-19T09:04:34.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.866+0000 7f7410922640 1 -- 172.21.15.120:0/2794550401 learned_addr learned my addr 172.21.15.120:0/2794550401 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:34.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.866+0000 7f7411123640 1 --2- 172.21.15.120:0/2794550401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a399949370 0x55a39993f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.867+0000 7f7410922640 1 -- 172.21.15.120:0/2794550401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3999442e0 msgr2=0x55a3999551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:34.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.867+0000 7f7410922640 1 --2- 172.21.15.120:0/2794550401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3999442e0 0x55a3999551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.867+0000 7f7410922640 1 -- 172.21.15.120:0/2794550401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a399949370 msgr2=0x55a39993f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:34.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.867+0000 7f7410922640 1 --2- 172.21.15.120:0/2794550401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a399949370 0x55a39993f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.867+0000 7f7410922640 1 -- 172.21.15.120:0/2794550401 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a39993ee30 con 0x55a3999439a0 2024-09-19T09:04:34.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.867+0000 7f7410922640 1 --2- 172.21.15.120:0/2794550401 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3999439a0 0x55a39993f3f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f740009c130 tx=0x7f7400066e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e866f271dbd5e9d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:34.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.867+0000 7f73f67fc640 1 -- 172.21.15.120:0/2794550401 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740009a7d0 con 0x55a3999439a0 2024-09-19T09:04:34.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.867+0000 7f73f67fc640 1 -- 172.21.15.120:0/2794550401 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74000a07e0 con 0x55a3999439a0 2024-09-19T09:04:34.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.867+0000 7f73f67fc640 1 -- 172.21.15.120:0/2794550401 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74000a4e70 con 0x55a3999439a0 2024-09-19T09:04:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.868+0000 7f74159f7740 1 -- 172.21.15.120:0/2794550401 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3999439a0 msgr2=0x55a39993f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.868+0000 7f74159f7740 1 --2- 172.21.15.120:0/2794550401 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3999439a0 0x55a39993f3f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f740009c130 tx=0x7f7400066e70 comp rx=0 tx=0).stop 2024-09-19T09:04:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.868+0000 7f74159f7740 1 -- 172.21.15.120:0/2794550401 shutdown_connections 2024-09-19T09:04:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.868+0000 7f74159f7740 1 --2- 172.21.15.120:0/2794550401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a399949370 0x55a39993f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.868+0000 7f74159f7740 1 --2- 172.21.15.120:0/2794550401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3999442e0 0x55a3999551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.868+0000 7f74159f7740 1 --2- 172.21.15.120:0/2794550401 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3999439a0 0x55a39993f3f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.868+0000 7f74159f7740 1 -- 172.21.15.120:0/2794550401 >> 172.21.15.120:0/2794550401 conn(0x55a39993f010 msgr2=0x55a39994d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.868+0000 7f74159f7740 1 -- 172.21.15.120:0/2794550401 shutdown_connections 2024-09-19T09:04:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.868+0000 7f74159f7740 1 -- 172.21.15.120:0/2794550401 wait complete. 2024-09-19T09:04:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.868+0000 7f74159f7740 1 -- start start 2024-09-19T09:04:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.868+0000 7f74159f7740 1 Processor -- start 2024-09-19T09:04:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.868+0000 7f74159f7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3999439a0 0x55a3999523f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.868+0000 7f74159f7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3999442e0 0x55a399952930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.868+0000 7f74159f7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a399949370 0x55a399953940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.868+0000 7f74159f7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f740c1aa3e0 con 0x55a3999442e0 2024-09-19T09:04:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.868+0000 7f74159f7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f740c1a9d10 con 0x55a3999439a0 2024-09-19T09:04:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.868+0000 7f74159f7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f740c1a6f30 con 0x55a399949370 2024-09-19T09:04:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.868+0000 7f74159f7740 10 client.0 _ll_register_callbacks cb 0x7f73f8009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.868+0000 7f740bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3999442e0 0x55a399952930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.868+0000 7f740bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3999442e0 0x55a399952930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60714/0 (socket says 172.21.15.120:60714) 2024-09-19T09:04:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.868+0000 7f740bfff640 1 -- 172.21.15.120:0/3873597668 learned_addr learned my addr 172.21.15.120:0/3873597668 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.869+0000 7f7410922640 1 --2- 172.21.15.120:0/3873597668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3999439a0 0x55a3999523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.869+0000 7f7411123640 1 --2- 172.21.15.120:0/3873597668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a399949370 0x55a399953940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.869+0000 7f740bfff640 1 -- 172.21.15.120:0/3873597668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a399949370 msgr2=0x55a399953940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.869+0000 7f740bfff640 1 --2- 172.21.15.120:0/3873597668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a399949370 0x55a399953940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.869+0000 7f740bfff640 1 -- 172.21.15.120:0/3873597668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3999439a0 msgr2=0x55a3999523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.869+0000 7f740bfff640 1 --2- 172.21.15.120:0/3873597668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3999439a0 0x55a3999523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.869+0000 7f740bfff640 1 -- 172.21.15.120:0/3873597668 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a399953f50 con 0x55a3999442e0 2024-09-19T09:04:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.869+0000 7f740bfff640 1 --2- 172.21.15.120:0/3873597668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3999442e0 0x55a399952930 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f73f8010670 tx=0x7f73f8005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89afb95eb8a94997 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.869+0000 7f73ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:04:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.869+0000 7f73ebfff640 1 -- 172.21.15.120:0/3873597668 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f802c070 con 0x55a3999442e0 2024-09-19T09:04:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.869+0000 7f74159f7740 1 -- 172.21.15.120:0/3873597668 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74000a07e0 con 0x55a3999442e0 2024-09-19T09:04:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.869+0000 7f73ebfff640 1 -- 172.21.15.120:0/3873597668 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f73f8010cd0 con 0x55a3999442e0 2024-09-19T09:04:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.869+0000 7f73ebfff640 1 -- 172.21.15.120:0/3873597668 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f73f800f920 con 0x55a3999442e0 2024-09-19T09:04:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.869+0000 7f74159f7740 10 client.15960 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.869+0000 7f74159f7740 1 -- 172.21.15.120:0/3873597668 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a3999552c0 con 0x55a3999442e0 2024-09-19T09:04:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.869+0000 7f73ebfff640 1 -- 172.21.15.120:0/3873597668 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74000a07e0 con 0x55a3999442e0 2024-09-19T09:04:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.869+0000 7f73ebfff640 1 -- 172.21.15.120:0/3873597668 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f73f80045d0 con 0x55a3999442e0 2024-09-19T09:04:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.870+0000 7f74159f7740 10 client.15960 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.870+0000 7f74159f7740 10 client.15960 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:04:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.870+0000 7f74159f7740 20 client.15960 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.870+0000 7f74159f7740 1 --2- 172.21.15.120:0/3873597668 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a3999c4b30 0x55a3999c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.870+0000 7f74159f7740 4 client.15960 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:04:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.870+0000 7f74159f7740 1 -- 172.21.15.120:0/3873597668 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a3998aed70 con 0x55a3999c4b30 2024-09-19T09:04:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.870+0000 7f7410922640 1 --2- 172.21.15.120:0/3873597668 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a3999c4b30 0x55a3999c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.870+0000 7f73ebfff640 10 client.15960 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.870+0000 7f7410922640 1 --2- 172.21.15.120:0/3873597668 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a3999c4b30 0x55a3999c6f50 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a093ba0b02ea1a15 server_cookie=f3d662db3092708e in_seq=0 out_seq=0 2024-09-19T09:04:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.921+0000 7f73ebfff640 1 -- 172.21.15.120:0/3873597668 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a3998aed70 con 0x55a3999c4b30 2024-09-19T09:04:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.921+0000 7f73ebfff640 10 client.15960 handle_command_reply: tid=0 2024-09-19T09:04:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.921+0000 7f74159f7740 1 client.15960 shutdown 2024-09-19T09:04:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.921+0000 7f74159f7740 1 -- 172.21.15.120:0/3873597668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3999442e0 msgr2=0x55a399952930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.921+0000 7f74159f7740 1 --2- 172.21.15.120:0/3873597668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3999442e0 0x55a399952930 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f73f8010670 tx=0x7f73f8005d40 comp rx=0 tx=0).stop 2024-09-19T09:04:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.921+0000 7f74159f7740 1 -- 172.21.15.120:0/3873597668 shutdown_connections 2024-09-19T09:04:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.921+0000 7f74159f7740 1 --2- 172.21.15.120:0/3873597668 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a3999c4b30 0x55a3999c6f50 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.921+0000 7f74159f7740 1 --2- 172.21.15.120:0/3873597668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a399949370 0x55a399953940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.921+0000 7f74159f7740 1 --2- 172.21.15.120:0/3873597668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3999442e0 0x55a399952930 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.921+0000 7f73ebfff640 0 client.15960 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.921+0000 7f74159f7740 1 --2- 172.21.15.120:0/3873597668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3999439a0 0x55a3999523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.921+0000 7f74159f7740 1 -- 172.21.15.120:0/3873597668 >> 172.21.15.120:0/3873597668 conn(0x55a39993f010 msgr2=0x55a39993d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.921+0000 7f74159f7740 1 -- 172.21.15.120:0/3873597668 shutdown_connections 2024-09-19T09:04:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.921+0000 7f74159f7740 1 -- 172.21.15.120:0/3873597668 wait complete. 2024-09-19T09:04:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.921+0000 7f74159f7740 20 client.15960 trim_cache size 0 max 16384 2024-09-19T09:04:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:04:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:04:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:04:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:04:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.922+0000 7f7412b86640 1 -- 172.21.15.120:0/3607008798 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f73fc0a32f0 msgr2=0x7f73fc0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.922+0000 7f7412b86640 1 --2- 172.21.15.120:0/3607008798 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f73fc0a32f0 0x7f73fc0a57b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f740402f540 tx=0x7f7404005df0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.922+0000 7f7412b86640 1 -- 172.21.15.120:0/3607008798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c100a40 msgr2=0x7f740c06a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.922+0000 7f7412b86640 1 --2- 172.21.15.120:0/3607008798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c100a40 0x7f740c06a410 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f740000e9c0 tx=0x7f740000ee80 comp rx=0 tx=0).stop 2024-09-19T09:04:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.922+0000 7f7412b86640 1 -- 172.21.15.120:0/3607008798 shutdown_connections 2024-09-19T09:04:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.922+0000 7f7412b86640 1 --2- 172.21.15.120:0/3607008798 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f73fc0a32f0 0x7f73fc0a57b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.922+0000 7f7412b86640 1 --2- 172.21.15.120:0/3607008798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f740c1025f0 0x7f740c19f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.922+0000 7f7412b86640 1 --2- 172.21.15.120:0/3607008798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f740c101c30 0x7f740c19cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.922+0000 7f7412b86640 1 --2- 172.21.15.120:0/3607008798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c100a40 0x7f740c06a410 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.922+0000 7f7412b86640 1 -- 172.21.15.120:0/3607008798 >> 172.21.15.120:0/3607008798 conn(0x7f740c0fc1e0 msgr2=0x7f740c104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.922+0000 7f7412b86640 1 -- 172.21.15.120:0/3607008798 shutdown_connections 2024-09-19T09:04:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:34.922+0000 7f7412b86640 1 -- 172.21.15.120:0/3607008798 wait complete. 2024-09-19T09:04:35.035 INFO:tasks.workunit.client.0.smithi120.stdout: 1 17859 3.84 MB/sec warmup 30 sec latency 14.049 ms 2024-09-19T09:04:35.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:34 smithi120 ceph-mon[31150]: pgmap v392: 97 pgs: 97 active+clean; 25 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 1.9 MiB/s wr, 45 op/s 2024-09-19T09:04:35.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:34 smithi196 ceph-mon[35150]: pgmap v392: 97 pgs: 97 active+clean; 25 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 1.9 MiB/s wr, 45 op/s 2024-09-19T09:04:35.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:34 smithi153 ceph-mon[35071]: pgmap v392: 97 pgs: 97 active+clean; 25 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 1.9 MiB/s wr, 45 op/s 2024-09-19T09:04:35.415 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:04:35.440 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:04:35.969 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:35 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:04:35.969 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:35 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:04:35.970 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:35 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:04:35.970 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:35 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:04:36.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.968+0000 7f55f1d06640 1 -- 172.21.15.120:0/1877552837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55ec102ef0 msgr2=0x7f55ec10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:36.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.968+0000 7f55f1d06640 1 --2- 172.21.15.120:0/1877552837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55ec102ef0 0x7f55ec10da40 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f55e4009ca0 tx=0x7f55e402a950 comp rx=0 tx=0).stop 2024-09-19T09:04:36.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.968+0000 7f55f1d06640 1 -- 172.21.15.120:0/1877552837 shutdown_connections 2024-09-19T09:04:36.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.968+0000 7f55f1d06640 1 --2- 172.21.15.120:0/1877552837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55ec102ef0 0x7f55ec10da40 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.968+0000 7f55f1d06640 1 --2- 172.21.15.120:0/1877552837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55ec102570 0x7f55ec1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.968+0000 7f55f1d06640 1 --2- 172.21.15.120:0/1877552837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55ec100e20 0x7f55ec101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.968+0000 7f55f1d06640 1 -- 172.21.15.120:0/1877552837 >> 172.21.15.120:0/1877552837 conn(0x7f55ec0fc7d0 msgr2=0x7f55ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:36.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.968+0000 7f55f1d06640 1 -- 172.21.15.120:0/1877552837 shutdown_connections 2024-09-19T09:04:36.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.968+0000 7f55f1d06640 1 -- 172.21.15.120:0/1877552837 wait complete. 2024-09-19T09:04:36.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.969+0000 7f55f1d06640 1 Processor -- start 2024-09-19T09:04:36.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.969+0000 7f55f1d06640 1 -- start start 2024-09-19T09:04:36.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.969+0000 7f55f1d06640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55ec100e20 0x7f55ec19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:36.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.969+0000 7f55f1d06640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55ec102570 0x7f55ec19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:36.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.969+0000 7f55f1d06640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55ec102ef0 0x7f55ec19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:36.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.969+0000 7f55f1d06640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec110f80 con 0x7f55ec102ef0 2024-09-19T09:04:36.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.969+0000 7f55f1d06640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec110e00 con 0x7f55ec102570 2024-09-19T09:04:36.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.969+0000 7f55f1d06640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec111100 con 0x7f55ec100e20 2024-09-19T09:04:36.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.969+0000 7f55ebfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55ec102ef0 0x7f55ec19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:36.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.969+0000 7f55eaffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55ec102570 0x7f55ec19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:36.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.969+0000 7f55ebfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55ec102ef0 0x7f55ec19ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60736/0 (socket says 172.21.15.120:60736) 2024-09-19T09:04:36.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.969+0000 7f55ebfff640 1 -- 172.21.15.120:0/3926401252 learned_addr learned my addr 172.21.15.120:0/3926401252 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:36.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.969+0000 7f55eb7fe640 1 --2- 172.21.15.120:0/3926401252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55ec100e20 0x7f55ec19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:36.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.970+0000 7f55ebfff640 1 -- 172.21.15.120:0/3926401252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55ec100e20 msgr2=0x7f55ec19a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:36.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.970+0000 7f55ebfff640 1 --2- 172.21.15.120:0/3926401252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55ec100e20 0x7f55ec19a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.970+0000 7f55ebfff640 1 -- 172.21.15.120:0/3926401252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55ec102570 msgr2=0x7f55ec19abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:36.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.970+0000 7f55ebfff640 1 --2- 172.21.15.120:0/3926401252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55ec102570 0x7f55ec19abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.970+0000 7f55ebfff640 1 -- 172.21.15.120:0/3926401252 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55ec19f650 con 0x7f55ec102ef0 2024-09-19T09:04:36.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.970+0000 7f55eaffd640 1 --2- 172.21.15.120:0/3926401252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55ec102570 0x7f55ec19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:04:36.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.970+0000 7f55ebfff640 1 --2- 172.21.15.120:0/3926401252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55ec102ef0 0x7f55ec19ef70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f55e4004af0 tx=0x7f55e402f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:36.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.970+0000 7f55e8ff9640 1 -- 172.21.15.120:0/3926401252 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e4034040 con 0x7f55ec102ef0 2024-09-19T09:04:36.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.970+0000 7f55e8ff9640 1 -- 172.21.15.120:0/3926401252 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55e4030920 con 0x7f55ec102ef0 2024-09-19T09:04:36.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.970+0000 7f55e8ff9640 1 -- 172.21.15.120:0/3926401252 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e4004630 con 0x7f55ec102ef0 2024-09-19T09:04:36.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.970+0000 7f55f1d06640 1 -- 172.21.15.120:0/3926401252 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55ec19f8e0 con 0x7f55ec102ef0 2024-09-19T09:04:36.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.970+0000 7f55f1d06640 1 -- 172.21.15.120:0/3926401252 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55ec1a7220 con 0x7f55ec102ef0 2024-09-19T09:04:36.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.971+0000 7f55f1d06640 1 -- 172.21.15.120:0/3926401252 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55ec1021b0 con 0x7f55ec102ef0 2024-09-19T09:04:36.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.972+0000 7f55e8ff9640 1 -- 172.21.15.120:0/3926401252 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f55e4037020 con 0x7f55ec102ef0 2024-09-19T09:04:36.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.973+0000 7f55e8ff9640 1 --2- 172.21.15.120:0/3926401252 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f55d40a3240 0x7f55d40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:36.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.973+0000 7f55eb7fe640 1 --2- 172.21.15.120:0/3926401252 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f55d40a3240 0x7f55d40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:36.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.974+0000 7f55e8ff9640 1 -- 172.21.15.120:0/3926401252 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f55e40f0850 con 0x7f55ec102ef0 2024-09-19T09:04:36.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.974+0000 7f55eb7fe640 1 --2- 172.21.15.120:0/3926401252 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f55d40a3240 0x7f55d40a5700 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009bd0 tx=0x7f55dc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:36.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:35.977+0000 7f55e8ff9640 1 -- 172.21.15.120:0/3926401252 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f55e4081210 con 0x7f55ec102ef0 2024-09-19T09:04:36.035 INFO:tasks.workunit.client.0.smithi120.stdout: 1 21138 4.23 MB/sec warmup 31 sec latency 37.467 ms 2024-09-19T09:04:36.114 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:04:36.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:35 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:04:36.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:35 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:04:36.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:35 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:04:36.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:35 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:04:36.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:35 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:04:36.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:35 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:04:36.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:35 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:04:36.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:35 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:04:36.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.211+0000 7f55f1d06640 1 -- 172.21.15.120:0/3926401252 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55ec1a73f0 con 0x7f55ec102ef0 2024-09-19T09:04:36.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.212+0000 7f55e8ff9640 1 -- 172.21.15.120:0/3926401252 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f55e40bd530 con 0x7f55ec102ef0 2024-09-19T09:04:36.214 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:04:36.214 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":28,"btime":"2024-09-19T09:02:41:696728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:02:41.207357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:04:36.214 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 28 2024-09-19T09:04:36.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.214+0000 7f55f1d06640 1 -- 172.21.15.120:0/3926401252 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f55d40a3240 msgr2=0x7f55d40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:36.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.214+0000 7f55f1d06640 1 --2- 172.21.15.120:0/3926401252 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f55d40a3240 0x7f55d40a5700 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009bd0 tx=0x7f55dc005f90 comp rx=0 tx=0).stop 2024-09-19T09:04:36.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.214+0000 7f55f1d06640 1 -- 172.21.15.120:0/3926401252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55ec102ef0 msgr2=0x7f55ec19ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:36.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.214+0000 7f55f1d06640 1 --2- 172.21.15.120:0/3926401252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55ec102ef0 0x7f55ec19ef70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f55e4004af0 tx=0x7f55e402f3a0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.214+0000 7f55f1d06640 1 -- 172.21.15.120:0/3926401252 shutdown_connections 2024-09-19T09:04:36.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.214+0000 7f55f1d06640 1 --2- 172.21.15.120:0/3926401252 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f55d40a3240 0x7f55d40a5700 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.214+0000 7f55f1d06640 1 --2- 172.21.15.120:0/3926401252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55ec102ef0 0x7f55ec19ef70 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.214+0000 7f55f1d06640 1 --2- 172.21.15.120:0/3926401252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55ec102570 0x7f55ec19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.214+0000 7f55f1d06640 1 --2- 172.21.15.120:0/3926401252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55ec100e20 0x7f55ec19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.214+0000 7f55f1d06640 1 -- 172.21.15.120:0/3926401252 >> 172.21.15.120:0/3926401252 conn(0x7f55ec0fc7d0 msgr2=0x7f55ec06a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:36.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.214+0000 7f55f1d06640 1 -- 172.21.15.120:0/3926401252 shutdown_connections 2024-09-19T09:04:36.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.214+0000 7f55f1d06640 1 -- 172.21.15.120:0/3926401252 wait complete. 2024-09-19T09:04:36.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.798+0000 7fc249db6640 1 -- 172.21.15.120:0/2505643678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc244102ef0 msgr2=0x7fc24410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:36.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.798+0000 7fc249db6640 1 --2- 172.21.15.120:0/2505643678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc244102ef0 0x7fc24410da40 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009ca0 tx=0x7fc23c02a950 comp rx=0 tx=0).stop 2024-09-19T09:04:36.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.798+0000 7fc249db6640 1 -- 172.21.15.120:0/2505643678 shutdown_connections 2024-09-19T09:04:36.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.798+0000 7fc249db6640 1 --2- 172.21.15.120:0/2505643678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc244102ef0 0x7fc24410da40 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.798+0000 7fc249db6640 1 --2- 172.21.15.120:0/2505643678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc244102570 0x7fc2441029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.798+0000 7fc249db6640 1 --2- 172.21.15.120:0/2505643678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc244100e20 0x7fc244101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.798+0000 7fc249db6640 1 -- 172.21.15.120:0/2505643678 >> 172.21.15.120:0/2505643678 conn(0x7fc2440fc7d0 msgr2=0x7fc2440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:36.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.798+0000 7fc249db6640 1 -- 172.21.15.120:0/2505643678 shutdown_connections 2024-09-19T09:04:36.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.798+0000 7fc249db6640 1 -- 172.21.15.120:0/2505643678 wait complete. 2024-09-19T09:04:36.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.798+0000 7fc249db6640 1 Processor -- start 2024-09-19T09:04:36.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.798+0000 7fc249db6640 1 -- start start 2024-09-19T09:04:36.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.799+0000 7fc249db6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc244100e20 0x7fc24419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:36.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.799+0000 7fc249db6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc244102570 0x7fc24419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:36.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.799+0000 7fc249db6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc244102ef0 0x7fc24419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:36.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.799+0000 7fc249db6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc244110f80 con 0x7fc244102ef0 2024-09-19T09:04:36.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.799+0000 7fc249db6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc244110e00 con 0x7fc244102570 2024-09-19T09:04:36.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.799+0000 7fc249db6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc244111100 con 0x7fc244100e20 2024-09-19T09:04:36.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.799+0000 7fc243fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc244102ef0 0x7fc24419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:36.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.799+0000 7fc2437fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc244100e20 0x7fc24419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:36.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.799+0000 7fc243fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc244102ef0 0x7fc24419ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60766/0 (socket says 172.21.15.120:60766) 2024-09-19T09:04:36.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.799+0000 7fc243fff640 1 -- 172.21.15.120:0/236067403 learned_addr learned my addr 172.21.15.120:0/236067403 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:36.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.799+0000 7fc242ffd640 1 --2- 172.21.15.120:0/236067403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc244102570 0x7fc24419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:36.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.799+0000 7fc2437fe640 1 -- 172.21.15.120:0/236067403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc244102570 msgr2=0x7fc24419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:36.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.799+0000 7fc2437fe640 1 --2- 172.21.15.120:0/236067403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc244102570 0x7fc24419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.799+0000 7fc2437fe640 1 -- 172.21.15.120:0/236067403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc244102ef0 msgr2=0x7fc24419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:36.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.799+0000 7fc2437fe640 1 --2- 172.21.15.120:0/236067403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc244102ef0 0x7fc24419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.799+0000 7fc2437fe640 1 -- 172.21.15.120:0/236067403 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc24419f600 con 0x7fc244100e20 2024-09-19T09:04:36.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.799+0000 7fc243fff640 1 --2- 172.21.15.120:0/236067403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc244102ef0 0x7fc24419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:04:36.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.799+0000 7fc2437fe640 1 --2- 172.21.15.120:0/236067403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc244100e20 0x7fc24419a650 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc23400e9f0 tx=0x7fc23400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:36.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.800+0000 7fc240ff9640 1 -- 172.21.15.120:0/236067403 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23400ccc0 con 0x7fc244100e20 2024-09-19T09:04:36.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.800+0000 7fc249db6640 1 -- 172.21.15.120:0/236067403 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc24419f8f0 con 0x7fc244100e20 2024-09-19T09:04:36.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.800+0000 7fc240ff9640 1 -- 172.21.15.120:0/236067403 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc234010910 con 0x7fc244100e20 2024-09-19T09:04:36.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.800+0000 7fc240ff9640 1 -- 172.21.15.120:0/236067403 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc234018960 con 0x7fc244100e20 2024-09-19T09:04:36.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.800+0000 7fc249db6640 1 -- 172.21.15.120:0/236067403 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2441a71d0 con 0x7fc244100e20 2024-09-19T09:04:36.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.802+0000 7fc24c18f740 1 Processor -- start 2024-09-19T09:04:36.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.802+0000 7fc24c18f740 1 -- start start 2024-09-19T09:04:36.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.802+0000 7fc240ff9640 1 -- 172.21.15.120:0/236067403 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fc234010ac0 con 0x7fc244100e20 2024-09-19T09:04:36.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.802+0000 7fc24c18f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584d2c934a0 0x5584d2c93880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:36.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.802+0000 7fc24c18f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d2c93dc0 0x5584d2c9e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:36.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.802+0000 7fc24c18f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584d2c9e680 0x5584d2ca0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:36.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.802+0000 7fc24c18f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5584d2bf92e0 con 0x5584d2c93dc0 2024-09-19T09:04:36.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.802+0000 7fc24c18f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5584d2ba0ce0 con 0x5584d2c9e680 2024-09-19T09:04:36.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.802+0000 7fc24c18f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5584d2bf52a0 con 0x5584d2c934a0 2024-09-19T09:04:36.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.802+0000 7fc243fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584d2c934a0 0x5584d2c93880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:36.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.802+0000 7fc2437fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d2c93dc0 0x5584d2c9e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:36.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.802+0000 7fc242ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584d2c9e680 0x5584d2ca0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:36.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.802+0000 7fc243fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584d2c934a0 0x5584d2c93880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:40754/0 (socket says 172.21.15.120:40754) 2024-09-19T09:04:36.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.802+0000 7fc243fff640 1 -- 172.21.15.120:0/3794225433 learned_addr learned my addr 172.21.15.120:0/3794225433 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:36.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.803+0000 7fc242ffd640 1 -- 172.21.15.120:0/3794225433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584d2c934a0 msgr2=0x5584d2c93880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:36.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.803+0000 7fc242ffd640 1 --2- 172.21.15.120:0/3794225433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584d2c934a0 0x5584d2c93880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.803+0000 7fc242ffd640 1 -- 172.21.15.120:0/3794225433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d2c93dc0 msgr2=0x5584d2c9e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:36.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.803+0000 7fc242ffd640 1 --2- 172.21.15.120:0/3794225433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d2c93dc0 0x5584d2c9e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.803+0000 7fc242ffd640 1 -- 172.21.15.120:0/3794225433 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584d2b85440 con 0x5584d2c9e680 2024-09-19T09:04:36.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.803+0000 7fc242ffd640 1 --2- 172.21.15.120:0/3794225433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584d2c9e680 0x5584d2ca0a70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc22c009bf0 tx=0x7fc22c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbcf73f232500d8f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:36.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.803+0000 7fc240ff9640 1 --2- 172.21.15.120:0/236067403 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc2300a3240 0x7fc2300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:36.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.803+0000 7fc239ffb640 1 -- 172.21.15.120:0/3794225433 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c004050 con 0x5584d2c9e680 2024-09-19T09:04:36.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.803+0000 7fc239ffb640 1 -- 172.21.15.120:0/3794225433 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc22c005500 con 0x5584d2c9e680 2024-09-19T09:04:36.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.803+0000 7fc239ffb640 1 -- 172.21.15.120:0/3794225433 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c0035f0 con 0x5584d2c9e680 2024-09-19T09:04:36.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.803+0000 7fc243fff640 1 --2- 172.21.15.120:0/236067403 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc2300a3240 0x7fc2300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:36.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.803+0000 7fc240ff9640 1 -- 172.21.15.120:0/236067403 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fc2340cfce0 con 0x7fc244100e20 2024-09-19T09:04:36.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.804+0000 7fc24c18f740 1 -- 172.21.15.120:0/3794225433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584d2c9e680 msgr2=0x5584d2ca0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:36.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.804+0000 7fc243fff640 1 --2- 172.21.15.120:0/236067403 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc2300a3240 0x7fc2300a5700 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc23c02fd80 tx=0x7fc23c009860 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:36.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.804+0000 7fc24c18f740 1 --2- 172.21.15.120:0/3794225433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584d2c9e680 0x5584d2ca0a70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc22c009bf0 tx=0x7fc22c011810 comp rx=0 tx=0).stop 2024-09-19T09:04:36.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.804+0000 7fc24c18f740 1 -- 172.21.15.120:0/3794225433 shutdown_connections 2024-09-19T09:04:36.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.804+0000 7fc24c18f740 1 --2- 172.21.15.120:0/3794225433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584d2c9e680 0x5584d2ca0a70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.804+0000 7fc24c18f740 1 --2- 172.21.15.120:0/3794225433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d2c93dc0 0x5584d2c9e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.804+0000 7fc24c18f740 1 --2- 172.21.15.120:0/3794225433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584d2c934a0 0x5584d2c93880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.804+0000 7fc24c18f740 1 -- 172.21.15.120:0/3794225433 >> 172.21.15.120:0/3794225433 conn(0x5584d2c8fa30 msgr2=0x5584d2c8fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:36.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.804+0000 7fc24c18f740 1 -- 172.21.15.120:0/3794225433 shutdown_connections 2024-09-19T09:04:36.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.804+0000 7fc24c18f740 1 -- 172.21.15.120:0/3794225433 wait complete. 2024-09-19T09:04:36.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.804+0000 7fc24c18f740 1 -- start start 2024-09-19T09:04:36.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.805+0000 7fc24c18f740 1 Processor -- start 2024-09-19T09:04:36.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.805+0000 7fc24c18f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584d2c934a0 0x5584d2c9df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:36.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.805+0000 7fc24c18f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d2c93dc0 0x5584d2c9b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:36.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.805+0000 7fc24c18f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584d2c9e680 0x5584d2c9b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:36.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.805+0000 7fc24c18f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5584d2ca1250 con 0x5584d2c93dc0 2024-09-19T09:04:36.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.805+0000 7fc24c18f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5584d2bb4050 con 0x5584d2c934a0 2024-09-19T09:04:36.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.805+0000 7fc24c18f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5584d2ca13d0 con 0x5584d2c9e680 2024-09-19T09:04:36.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.805+0000 7fc24c18f740 10 client.0 _ll_register_callbacks cb 0x5584d2bf4160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:36.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.805+0000 7fc242ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d2c93dc0 0x5584d2c9b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:36.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.805+0000 7fc2437fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584d2c934a0 0x5584d2c9df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:36.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.805+0000 7fc242ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d2c93dc0 0x5584d2c9b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60770/0 (socket says 172.21.15.120:60770) 2024-09-19T09:04:36.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.805+0000 7fc242ffd640 1 -- 172.21.15.120:0/1584224211 learned_addr learned my addr 172.21.15.120:0/1584224211 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:36.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.805+0000 7fc243fff640 1 --2- 172.21.15.120:0/1584224211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584d2c9e680 0x5584d2c9b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:36.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.805+0000 7fc242ffd640 1 -- 172.21.15.120:0/1584224211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584d2c9e680 msgr2=0x5584d2c9b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:36.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.805+0000 7fc242ffd640 1 --2- 172.21.15.120:0/1584224211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584d2c9e680 0x5584d2c9b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.805+0000 7fc242ffd640 1 -- 172.21.15.120:0/1584224211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584d2c934a0 msgr2=0x5584d2c9df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:36.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.805+0000 7fc242ffd640 1 --2- 172.21.15.120:0/1584224211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584d2c934a0 0x5584d2c9df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.805+0000 7fc242ffd640 1 -- 172.21.15.120:0/1584224211 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584d296a3c0 con 0x5584d2c93dc0 2024-09-19T09:04:36.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.806+0000 7fc242ffd640 1 --2- 172.21.15.120:0/1584224211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d2c93dc0 0x5584d2c9b1b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc22c004a20 tx=0x7fc22c009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb8466ab750f45f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:36.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.806+0000 7fc222ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:04:36.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.806+0000 7fc222ffd640 1 -- 172.21.15.120:0/1584224211 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c005030 con 0x5584d2c93dc0 2024-09-19T09:04:36.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.806+0000 7fc222ffd640 1 -- 172.21.15.120:0/1584224211 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fc22c0051d0 con 0x5584d2c93dc0 2024-09-19T09:04:36.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.806+0000 7fc24c18f740 1 -- 172.21.15.120:0/1584224211 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc22c005500 con 0x5584d2c93dc0 2024-09-19T09:04:36.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.806+0000 7fc222ffd640 1 -- 172.21.15.120:0/1584224211 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc22c0035f0 con 0x5584d2c93dc0 2024-09-19T09:04:36.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.806+0000 7fc24c18f740 10 client.15978 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:36.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.806+0000 7fc24c18f740 1 -- 172.21.15.120:0/1584224211 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5584d2cacb60 con 0x5584d2c93dc0 2024-09-19T09:04:36.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.806+0000 7fc222ffd640 1 -- 172.21.15.120:0/1584224211 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc22c005500 con 0x5584d2c93dc0 2024-09-19T09:04:36.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.806+0000 7fc222ffd640 1 -- 172.21.15.120:0/1584224211 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc22c018070 con 0x5584d2c93dc0 2024-09-19T09:04:36.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.806+0000 7fc24c18f740 10 client.15978 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:36.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.806+0000 7fc24c18f740 10 client.15978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:04:36.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.806+0000 7fc24c18f740 20 client.15978 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:36.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.806+0000 7fc24c18f740 1 --2- 172.21.15.120:0/1584224211 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5584d2cadfc0 0x5584d2cb03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:36.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.806+0000 7fc24c18f740 4 client.15978 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:04:36.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.806+0000 7fc24c18f740 1 -- 172.21.15.120:0/1584224211 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584d2c06850 con 0x5584d2cadfc0 2024-09-19T09:04:36.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.807+0000 7fc2437fe640 1 --2- 172.21.15.120:0/1584224211 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5584d2cadfc0 0x5584d2cb03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:36.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.807+0000 7fc2437fe640 1 --2- 172.21.15.120:0/1584224211 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5584d2cadfc0 0x5584d2cb03b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1bd078c1b8395f9 server_cookie=d1e2d6363f333709 in_seq=0 out_seq=0 2024-09-19T09:04:36.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.807+0000 7fc222ffd640 10 client.15978 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:36.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.809+0000 7fc222ffd640 1 -- 172.21.15.120:0/1584224211 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5584d2c06850 con 0x5584d2cadfc0 2024-09-19T09:04:36.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.809+0000 7fc222ffd640 10 client.15978 handle_command_reply: tid=0 2024-09-19T09:04:36.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.809+0000 7fc24c18f740 1 client.15978 shutdown 2024-09-19T09:04:36.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.810+0000 7fc24c18f740 1 -- 172.21.15.120:0/1584224211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d2c93dc0 msgr2=0x5584d2c9b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:36.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.810+0000 7fc24c18f740 1 --2- 172.21.15.120:0/1584224211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d2c93dc0 0x5584d2c9b1b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc22c004a20 tx=0x7fc22c009fd0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.810+0000 7fc24c18f740 1 -- 172.21.15.120:0/1584224211 shutdown_connections 2024-09-19T09:04:36.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.810+0000 7fc24c18f740 1 --2- 172.21.15.120:0/1584224211 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5584d2cadfc0 0x5584d2cb03b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.810+0000 7fc24c18f740 1 --2- 172.21.15.120:0/1584224211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584d2c9e680 0x5584d2c9b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.810+0000 7fc24c18f740 1 --2- 172.21.15.120:0/1584224211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d2c93dc0 0x5584d2c9b1b0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.810+0000 7fc24c18f740 1 --2- 172.21.15.120:0/1584224211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584d2c934a0 0x5584d2c9df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.810+0000 7fc222ffd640 0 client.15978 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:36.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.810+0000 7fc24c18f740 1 -- 172.21.15.120:0/1584224211 >> 172.21.15.120:0/1584224211 conn(0x5584d2c8fa30 msgr2=0x5584d2c95760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:36.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.810+0000 7fc24c18f740 1 -- 172.21.15.120:0/1584224211 shutdown_connections 2024-09-19T09:04:36.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.810+0000 7fc24c18f740 1 -- 172.21.15.120:0/1584224211 wait complete. 2024-09-19T09:04:36.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.810+0000 7fc24c18f740 20 client.15978 trim_cache size 0 max 16384 2024-09-19T09:04:36.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.827+0000 7fc24c18f740 1 Processor -- start 2024-09-19T09:04:36.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.827+0000 7fc24c18f740 1 -- start start 2024-09-19T09:04:36.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.827+0000 7fc24c18f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d2c8fa30 0x5584d2caa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:36.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.827+0000 7fc24c18f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584d2c934a0 0x5584d2cb5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:36.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.827+0000 7fc24c18f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584d2c93dc0 0x5584d2cb6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:36.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.827+0000 7fc24c18f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc2441af750 con 0x5584d2c8fa30 2024-09-19T09:04:36.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.827+0000 7fc24c18f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc2441b1290 con 0x5584d2c934a0 2024-09-19T09:04:36.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.827+0000 7fc24c18f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc2441afe20 con 0x5584d2c93dc0 2024-09-19T09:04:36.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.828+0000 7fc243fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584d2c93dc0 0x5584d2cb6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:36.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.828+0000 7fc243fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584d2c93dc0 0x5584d2cb6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:40768/0 (socket says 172.21.15.120:40768) 2024-09-19T09:04:36.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.828+0000 7fc243fff640 1 -- 172.21.15.120:0/3872670642 learned_addr learned my addr 172.21.15.120:0/3872670642 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:36.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.828+0000 7fc242ffd640 1 --2- 172.21.15.120:0/3872670642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584d2c934a0 0x5584d2cb5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:36.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.828+0000 7fc243fff640 1 -- 172.21.15.120:0/3872670642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584d2c934a0 msgr2=0x5584d2cb5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:36.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.828+0000 7fc243fff640 1 --2- 172.21.15.120:0/3872670642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584d2c934a0 0x5584d2cb5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.828+0000 7fc243fff640 1 -- 172.21.15.120:0/3872670642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d2c8fa30 msgr2=0x5584d2caa4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:04:36.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.828+0000 7fc2437fe640 1 --2- 172.21.15.120:0/3872670642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d2c8fa30 0x5584d2caa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:36.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.828+0000 7fc243fff640 1 --2- 172.21.15.120:0/3872670642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d2c8fa30 0x5584d2caa4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.828+0000 7fc243fff640 1 -- 172.21.15.120:0/3872670642 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584d2cb09e0 con 0x5584d2c93dc0 2024-09-19T09:04:36.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.828+0000 7fc2437fe640 1 --2- 172.21.15.120:0/3872670642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d2c8fa30 0x5584d2caa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:04:36.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.828+0000 7fc243fff640 1 --2- 172.21.15.120:0/3872670642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584d2c93dc0 0x5584d2cb6150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc23c02f710 tx=0x7fc23c045000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c98c6ebc9f45d737 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:36.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.831+0000 7fc2397fa640 1 -- 172.21.15.120:0/3872670642 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c056070 con 0x5584d2c93dc0 2024-09-19T09:04:36.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.831+0000 7fc2397fa640 1 -- 172.21.15.120:0/3872670642 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc23c004720 con 0x5584d2c93dc0 2024-09-19T09:04:36.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.831+0000 7fc2397fa640 1 -- 172.21.15.120:0/3872670642 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c034040 con 0x5584d2c93dc0 2024-09-19T09:04:36.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.832+0000 7fc24c18f740 1 -- 172.21.15.120:0/3872670642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584d2c93dc0 msgr2=0x5584d2cb6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:36.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.832+0000 7fc24c18f740 1 --2- 172.21.15.120:0/3872670642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584d2c93dc0 0x5584d2cb6150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc23c02f710 tx=0x7fc23c045000 comp rx=0 tx=0).stop 2024-09-19T09:04:36.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.832+0000 7fc24c18f740 1 -- 172.21.15.120:0/3872670642 shutdown_connections 2024-09-19T09:04:36.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.832+0000 7fc24c18f740 1 --2- 172.21.15.120:0/3872670642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584d2c93dc0 0x5584d2cb6150 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.832+0000 7fc24c18f740 1 --2- 172.21.15.120:0/3872670642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584d2c934a0 0x5584d2cb5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.832+0000 7fc24c18f740 1 --2- 172.21.15.120:0/3872670642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d2c8fa30 0x5584d2caa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.832+0000 7fc24c18f740 1 -- 172.21.15.120:0/3872670642 >> 172.21.15.120:0/3872670642 conn(0x5584d2c9b850 msgr2=0x5584d2c9a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:36.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.832+0000 7fc24c18f740 1 -- 172.21.15.120:0/3872670642 shutdown_connections 2024-09-19T09:04:36.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.832+0000 7fc24c18f740 1 -- 172.21.15.120:0/3872670642 wait complete. 2024-09-19T09:04:36.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.832+0000 7fc24c18f740 1 -- start start 2024-09-19T09:04:36.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.833+0000 7fc24c18f740 1 Processor -- start 2024-09-19T09:04:36.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.833+0000 7fc24c18f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584d2c8fa30 0x5584d2ca2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:36.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.833+0000 7fc24c18f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584d2c934a0 0x5584d2ca2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:36.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.833+0000 7fc24c18f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d2c93dc0 0x5584d2d17be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:36.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.833+0000 7fc24c18f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc2441b0bc0 con 0x5584d2c93dc0 2024-09-19T09:04:36.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.833+0000 7fc24c18f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc2441b04f0 con 0x5584d2c934a0 2024-09-19T09:04:36.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.833+0000 7fc24c18f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc2441ad710 con 0x5584d2c8fa30 2024-09-19T09:04:36.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.833+0000 7fc24c18f740 10 client.0 _ll_register_callbacks cb 0x7fc2340d35f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:36.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.833+0000 7fc242ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584d2c934a0 0x5584d2ca2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:36.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.833+0000 7fc2437fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584d2c8fa30 0x5584d2ca2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:36.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.833+0000 7fc243fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d2c93dc0 0x5584d2d17be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:36.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.833+0000 7fc243fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d2c93dc0 0x5584d2d17be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60786/0 (socket says 172.21.15.120:60786) 2024-09-19T09:04:36.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.833+0000 7fc242ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584d2c934a0 0x5584d2ca2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:46800/0 (socket says 172.21.15.120:46800) 2024-09-19T09:04:36.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.833+0000 7fc2437fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584d2c8fa30 0x5584d2ca2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:40782/0 (socket says 172.21.15.120:40782) 2024-09-19T09:04:36.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.833+0000 7fc243fff640 1 -- 172.21.15.120:0/292900262 learned_addr learned my addr 172.21.15.120:0/292900262 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:36.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.833+0000 7fc243fff640 1 -- 172.21.15.120:0/292900262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584d2c8fa30 msgr2=0x5584d2ca2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:36.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.833+0000 7fc243fff640 1 --2- 172.21.15.120:0/292900262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584d2c8fa30 0x5584d2ca2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.833+0000 7fc243fff640 1 -- 172.21.15.120:0/292900262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584d2c934a0 msgr2=0x5584d2ca2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:36.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.833+0000 7fc243fff640 1 --2- 172.21.15.120:0/292900262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584d2c934a0 0x5584d2ca2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.833+0000 7fc243fff640 1 -- 172.21.15.120:0/292900262 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584d2c8f860 con 0x5584d2c93dc0 2024-09-19T09:04:36.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.834+0000 7fc243fff640 1 --2- 172.21.15.120:0/292900262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d2c93dc0 0x5584d2d17be0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc23c004af0 tx=0x7fc23c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab1d1787597f5d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:36.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.834+0000 7fc222ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:04:36.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.834+0000 7fc222ffd640 1 -- 172.21.15.120:0/292900262 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c056040 con 0x5584d2c93dc0 2024-09-19T09:04:36.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.834+0000 7fc222ffd640 1 -- 172.21.15.120:0/292900262 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fc23c003f30 con 0x5584d2c93dc0 2024-09-19T09:04:36.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.834+0000 7fc24c18f740 1 -- 172.21.15.120:0/292900262 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc23c004720 con 0x5584d2c93dc0 2024-09-19T09:04:36.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.834+0000 7fc222ffd640 1 -- 172.21.15.120:0/292900262 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc23c004130 con 0x5584d2c93dc0 2024-09-19T09:04:36.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.834+0000 7fc24c18f740 10 client.15981 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:36.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.834+0000 7fc24c18f740 1 -- 172.21.15.120:0/292900262 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5584d2d19150 con 0x5584d2c93dc0 2024-09-19T09:04:36.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.834+0000 7fc222ffd640 1 -- 172.21.15.120:0/292900262 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc23c004720 con 0x5584d2c93dc0 2024-09-19T09:04:36.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.834+0000 7fc222ffd640 1 -- 172.21.15.120:0/292900262 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc23c063030 con 0x5584d2c93dc0 2024-09-19T09:04:36.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.834+0000 7fc24c18f740 10 client.15981 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:36.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.834+0000 7fc24c18f740 10 client.15981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:04:36.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.834+0000 7fc24c18f740 20 client.15981 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:36.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.834+0000 7fc24c18f740 1 --2- 172.21.15.120:0/292900262 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5584d2c9e680 0x5584d2d1b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:36.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.834+0000 7fc24c18f740 4 client.15981 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:04:36.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.834+0000 7fc24c18f740 1 -- 172.21.15.120:0/292900262 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5584d2c00d60 con 0x5584d2c9e680 2024-09-19T09:04:36.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.835+0000 7fc2437fe640 1 --2- 172.21.15.120:0/292900262 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5584d2c9e680 0x5584d2d1b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:36.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.835+0000 7fc2437fe640 1 --2- 172.21.15.120:0/292900262 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5584d2c9e680 0x5584d2d1b7c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b4a9c009167c47a server_cookie=a45760d337d0abc1 in_seq=0 out_seq=0 2024-09-19T09:04:36.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.835+0000 7fc222ffd640 10 client.15981 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:36.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.835+0000 7fc222ffd640 1 -- 172.21.15.120:0/292900262 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5584d2c00d60 con 0x5584d2c9e680 2024-09-19T09:04:36.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.835+0000 7fc222ffd640 10 client.15981 handle_command_reply: tid=0 2024-09-19T09:04:36.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.835+0000 7fc24c18f740 1 client.15981 shutdown 2024-09-19T09:04:36.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.836+0000 7fc24c18f740 1 -- 172.21.15.120:0/292900262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d2c93dc0 msgr2=0x5584d2d17be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:36.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.836+0000 7fc24c18f740 1 --2- 172.21.15.120:0/292900262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d2c93dc0 0x5584d2d17be0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc23c004af0 tx=0x7fc23c030d80 comp rx=0 tx=0).stop 2024-09-19T09:04:36.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.836+0000 7fc24c18f740 1 -- 172.21.15.120:0/292900262 shutdown_connections 2024-09-19T09:04:36.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.836+0000 7fc24c18f740 1 --2- 172.21.15.120:0/292900262 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5584d2c9e680 0x5584d2d1b7c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.836+0000 7fc24c18f740 1 --2- 172.21.15.120:0/292900262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d2c93dc0 0x5584d2d17be0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.836+0000 7fc222ffd640 0 client.15981 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:36.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.836+0000 7fc24c18f740 1 --2- 172.21.15.120:0/292900262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584d2c934a0 0x5584d2ca2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.836+0000 7fc24c18f740 1 --2- 172.21.15.120:0/292900262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584d2c8fa30 0x5584d2ca2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.836+0000 7fc24c18f740 1 -- 172.21.15.120:0/292900262 >> 172.21.15.120:0/292900262 conn(0x5584d2c9b850 msgr2=0x5584d2c9ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:36.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.836+0000 7fc24c18f740 1 -- 172.21.15.120:0/292900262 shutdown_connections 2024-09-19T09:04:36.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.836+0000 7fc24c18f740 1 -- 172.21.15.120:0/292900262 wait complete. 2024-09-19T09:04:36.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.836+0000 7fc24c18f740 20 client.15981 trim_cache size 0 max 16384 2024-09-19T09:04:36.842 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:04:36.842 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:04:36.842 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-19T09:04:36.842 INFO:teuthology.orchestra.run.smithi120.stdout: "7e98f4c4-fa37-4a87-9a31-a6b946fc787d": { 2024-09-19T09:04:36.842 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-19T09:04:36.842 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "7e98f4c4-fa37-4a87-9a31-a6b946fc787d", 2024-09-19T09:04:36.842 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-19T09:04:36.843 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:04:36.843 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:04:36.843 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:04:36.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.837+0000 7fc249db6640 1 -- 172.21.15.120:0/236067403 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc2300a3240 msgr2=0x7fc2300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:36.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.837+0000 7fc249db6640 1 --2- 172.21.15.120:0/236067403 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc2300a3240 0x7fc2300a5700 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc23c02fd80 tx=0x7fc23c009860 comp rx=0 tx=0).stop 2024-09-19T09:04:36.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.837+0000 7fc249db6640 1 -- 172.21.15.120:0/236067403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc244100e20 msgr2=0x7fc24419a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:36.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.837+0000 7fc249db6640 1 --2- 172.21.15.120:0/236067403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc244100e20 0x7fc24419a650 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc23400e9f0 tx=0x7fc23400eeb0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.837+0000 7fc249db6640 1 -- 172.21.15.120:0/236067403 shutdown_connections 2024-09-19T09:04:36.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.837+0000 7fc249db6640 1 --2- 172.21.15.120:0/236067403 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc2300a3240 0x7fc2300a5700 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.837+0000 7fc249db6640 1 --2- 172.21.15.120:0/236067403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc244102ef0 0x7fc24419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.837+0000 7fc249db6640 1 --2- 172.21.15.120:0/236067403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc244102570 0x7fc24419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.837+0000 7fc249db6640 1 --2- 172.21.15.120:0/236067403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc244100e20 0x7fc24419a650 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:36.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.837+0000 7fc249db6640 1 -- 172.21.15.120:0/236067403 >> 172.21.15.120:0/236067403 conn(0x7fc2440fc7d0 msgr2=0x7fc24406a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:36.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.837+0000 7fc249db6640 1 -- 172.21.15.120:0/236067403 shutdown_connections 2024-09-19T09:04:36.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:36.837+0000 7fc249db6640 1 -- 172.21.15.120:0/236067403 wait complete. 2024-09-19T09:04:37.035 INFO:tasks.workunit.client.0.smithi120.stdout: 1 23927 4.49 MB/sec warmup 32 sec latency 21.290 ms 2024-09-19T09:04:37.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:36 smithi120 ceph-mon[31150]: pgmap v393: 97 pgs: 97 active+clean; 25 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 619 KiB/s wr, 31 op/s 2024-09-19T09:04:37.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:36 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3926401252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:04:37.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:36 smithi196 ceph-mon[35150]: pgmap v393: 97 pgs: 97 active+clean; 25 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 619 KiB/s wr, 31 op/s 2024-09-19T09:04:37.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:36 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3926401252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:04:37.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:36 smithi153 ceph-mon[35071]: pgmap v393: 97 pgs: 97 active+clean; 25 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 619 KiB/s wr, 31 op/s 2024-09-19T09:04:37.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:36 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3926401252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:04:37.349 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0' 2024-09-19T09:04:37.350 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0 2024-09-19T09:04:38.035 INFO:tasks.workunit.client.0.smithi120.stdout: 1 27322 4.91 MB/sec warmup 33 sec latency 23.542 ms 2024-09-19T09:04:38.048 INFO:tasks.cephfs.filesystem:scrub status for tag:7e98f4c4-fa37-4a87-9a31-a6b946fc787d - {'path': '/', 'tag': '7e98f4c4-fa37-4a87-9a31-a6b946fc787d', 'options': 'recursive,force'} 2024-09-19T09:04:38.073 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:04:38.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.581+0000 7fa63487b640 1 -- 172.21.15.120:0/3915713443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa630106f40 msgr2=0x7fa63010eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:38.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.581+0000 7fa63487b640 1 --2- 172.21.15.120:0/3915713443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa630106f40 0x7fa63010eac0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa628009ca0 tx=0x7fa62802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.581+0000 7fa63487b640 1 -- 172.21.15.120:0/3915713443 shutdown_connections 2024-09-19T09:04:38.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.581+0000 7fa63487b640 1 --2- 172.21.15.120:0/3915713443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa630106f40 0x7fa63010eac0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.581+0000 7fa63487b640 1 --2- 172.21.15.120:0/3915713443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6301027c0 0x7fa630106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.581+0000 7fa63487b640 1 --2- 172.21.15.120:0/3915713443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa630101e10 0x7fa6301021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.581+0000 7fa63487b640 1 -- 172.21.15.120:0/3915713443 >> 172.21.15.120:0/3915713443 conn(0x7fa6300fd8d0 msgr2=0x7fa6300ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:38.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.581+0000 7fa63487b640 1 -- 172.21.15.120:0/3915713443 shutdown_connections 2024-09-19T09:04:38.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.581+0000 7fa63487b640 1 -- 172.21.15.120:0/3915713443 wait complete. 2024-09-19T09:04:38.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.582+0000 7fa63487b640 1 Processor -- start 2024-09-19T09:04:38.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.582+0000 7fa63487b640 1 -- start start 2024-09-19T09:04:38.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.582+0000 7fa63487b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa630101e10 0x7fa63019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:38.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.582+0000 7fa63487b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6301027c0 0x7fa63019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:38.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.582+0000 7fa63487b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa630106f40 0x7fa63019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:38.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.582+0000 7fa63487b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa630111fa0 con 0x7fa6301027c0 2024-09-19T09:04:38.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.582+0000 7fa63487b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa630111e20 con 0x7fa630101e10 2024-09-19T09:04:38.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.582+0000 7fa63487b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa630112120 con 0x7fa630106f40 2024-09-19T09:04:38.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.582+0000 7fa627fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6301027c0 0x7fa63019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:38.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.582+0000 7fa627fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6301027c0 0x7fa63019ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60798/0 (socket says 172.21.15.120:60798) 2024-09-19T09:04:38.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.582+0000 7fa62ed9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa630106f40 0x7fa63019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:38.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.582+0000 7fa627fff640 1 -- 172.21.15.120:0/976293126 learned_addr learned my addr 172.21.15.120:0/976293126 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:38.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.582+0000 7fa62e59c640 1 --2- 172.21.15.120:0/976293126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa630101e10 0x7fa63019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:38.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.583+0000 7fa627fff640 1 -- 172.21.15.120:0/976293126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa630106f40 msgr2=0x7fa63019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:38.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.583+0000 7fa627fff640 1 --2- 172.21.15.120:0/976293126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa630106f40 0x7fa63019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.583+0000 7fa627fff640 1 -- 172.21.15.120:0/976293126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa630101e10 msgr2=0x7fa63019a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:38.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.583+0000 7fa627fff640 1 --2- 172.21.15.120:0/976293126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa630101e10 0x7fa63019a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.583+0000 7fa627fff640 1 -- 172.21.15.120:0/976293126 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa63019f5b0 con 0x7fa6301027c0 2024-09-19T09:04:38.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.583+0000 7fa627fff640 1 --2- 172.21.15.120:0/976293126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6301027c0 0x7fa63019ab40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa61800c970 tx=0x7fa61800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:38.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.583+0000 7fa6277fe640 1 -- 172.21.15.120:0/976293126 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa618007bb0 con 0x7fa6301027c0 2024-09-19T09:04:38.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.583+0000 7fa6277fe640 1 -- 172.21.15.120:0/976293126 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa618007d50 con 0x7fa6301027c0 2024-09-19T09:04:38.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.583+0000 7fa63487b640 1 -- 172.21.15.120:0/976293126 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa63019f8a0 con 0x7fa6301027c0 2024-09-19T09:04:38.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.583+0000 7fa6277fe640 1 -- 172.21.15.120:0/976293126 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61800f450 con 0x7fa6301027c0 2024-09-19T09:04:38.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.583+0000 7fa63487b640 1 -- 172.21.15.120:0/976293126 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6301a7180 con 0x7fa6301027c0 2024-09-19T09:04:38.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.585+0000 7fa6277fe640 1 -- 172.21.15.120:0/976293126 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa618005020 con 0x7fa6301027c0 2024-09-19T09:04:38.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.586+0000 7fa636c54740 1 Processor -- start 2024-09-19T09:04:38.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.586+0000 7fa636c54740 1 -- start start 2024-09-19T09:04:38.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.586+0000 7fa636c54740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb252ff850 0x55cb252ffc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:38.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.586+0000 7fa636c54740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb25300170 0x55cb2530a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:38.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.586+0000 7fa636c54740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb2530aa30 0x55cb2530ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:38.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.586+0000 7fa636c54740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cb25264160 con 0x55cb2530aa30 2024-09-19T09:04:38.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.586+0000 7fa636c54740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cb2520b720 con 0x55cb252ff850 2024-09-19T09:04:38.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.586+0000 7fa636c54740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cb25260120 con 0x55cb25300170 2024-09-19T09:04:38.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.586+0000 7fa6277fe640 1 --2- 172.21.15.120:0/976293126 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa61c0a32c0 0x7fa61c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:38.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.586+0000 7fa62e59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb25300170 0x55cb2530a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:38.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.586+0000 7fa62ed9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb252ff850 0x55cb252ffc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:38.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.587+0000 7fa62e59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb25300170 0x55cb2530a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:40810/0 (socket says 172.21.15.120:40810) 2024-09-19T09:04:38.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.587+0000 7fa627fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb2530aa30 0x55cb2530ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:38.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.587+0000 7fa62ed9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb252ff850 0x55cb252ffc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:46814/0 (socket says 172.21.15.120:46814) 2024-09-19T09:04:38.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.587+0000 7fa62e59c640 1 -- 172.21.15.120:0/1190442391 learned_addr learned my addr 172.21.15.120:0/1190442391 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:38.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.587+0000 7fa62ed9d640 1 --2- 172.21.15.120:0/976293126 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa61c0a32c0 0x7fa61c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:38.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.587+0000 7fa6277fe640 1 -- 172.21.15.120:0/976293126 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa6180cf550 con 0x7fa6301027c0 2024-09-19T09:04:38.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.587+0000 7fa62ed9d640 1 -- 172.21.15.120:0/1190442391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb25300170 msgr2=0x55cb2530a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:38.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.587+0000 7fa62ed9d640 1 --2- 172.21.15.120:0/1190442391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb25300170 0x55cb2530a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.587+0000 7fa62ed9d640 1 -- 172.21.15.120:0/1190442391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb2530aa30 msgr2=0x55cb2530ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:38.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.587+0000 7fa62ed9d640 1 --2- 172.21.15.120:0/1190442391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb2530aa30 0x55cb2530ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.587+0000 7fa62ed9d640 1 -- 172.21.15.120:0/1190442391 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb24fd5ba0 con 0x55cb252ff850 2024-09-19T09:04:38.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.587+0000 7fa62e59c640 1 --2- 172.21.15.120:0/1190442391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb25300170 0x55cb2530a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:04:38.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.587+0000 7fa62ed9d640 1 --2- 172.21.15.120:0/1190442391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb252ff850 0x55cb252ffc30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa628006010 tx=0x7fa628031690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f25a3f423c791b74 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:38.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.587+0000 7fa62ed9d640 1 --2- 172.21.15.120:0/976293126 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa61c0a32c0 0x7fa61c0a5780 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa628009bf0 tx=0x7fa628036000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:38.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.587+0000 7fa624ff9640 1 -- 172.21.15.120:0/1190442391 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628056070 con 0x55cb252ff850 2024-09-19T09:04:38.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.587+0000 7fa624ff9640 1 -- 172.21.15.120:0/1190442391 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa628036070 con 0x55cb252ff850 2024-09-19T09:04:38.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.587+0000 7fa624ff9640 1 -- 172.21.15.120:0/1190442391 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628051450 con 0x55cb252ff850 2024-09-19T09:04:38.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.588+0000 7fa636c54740 1 -- 172.21.15.120:0/1190442391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb252ff850 msgr2=0x55cb252ffc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:38.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.588+0000 7fa636c54740 1 --2- 172.21.15.120:0/1190442391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb252ff850 0x55cb252ffc30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa628006010 tx=0x7fa628031690 comp rx=0 tx=0).stop 2024-09-19T09:04:38.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.588+0000 7fa636c54740 1 -- 172.21.15.120:0/1190442391 shutdown_connections 2024-09-19T09:04:38.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.588+0000 7fa636c54740 1 --2- 172.21.15.120:0/1190442391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb2530aa30 0x55cb2530ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.588+0000 7fa636c54740 1 --2- 172.21.15.120:0/1190442391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb25300170 0x55cb2530a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.588+0000 7fa636c54740 1 --2- 172.21.15.120:0/1190442391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb252ff850 0x55cb252ffc30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.588+0000 7fa636c54740 1 -- 172.21.15.120:0/1190442391 >> 172.21.15.120:0/1190442391 conn(0x55cb252fbde0 msgr2=0x55cb252fc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:38.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.588+0000 7fa636c54740 1 -- 172.21.15.120:0/1190442391 shutdown_connections 2024-09-19T09:04:38.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.588+0000 7fa636c54740 1 -- 172.21.15.120:0/1190442391 wait complete. 2024-09-19T09:04:38.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.589+0000 7fa636c54740 1 -- start start 2024-09-19T09:04:38.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.589+0000 7fa636c54740 1 Processor -- start 2024-09-19T09:04:38.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.589+0000 7fa636c54740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb252ff850 0x55cb25307940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:38.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.589+0000 7fa636c54740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb25300170 0x55cb25307e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:38.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.589+0000 7fa636c54740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb2530aa30 0x55cb253083c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:38.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.589+0000 7fa636c54740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cb2530d480 con 0x55cb252ff850 2024-09-19T09:04:38.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.589+0000 7fa636c54740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cb251ec900 con 0x55cb2530aa30 2024-09-19T09:04:38.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.589+0000 7fa636c54740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cb2530d600 con 0x55cb25300170 2024-09-19T09:04:38.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.589+0000 7fa636c54740 10 client.0 _ll_register_callbacks cb 0x55cb251ea780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:38.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.589+0000 7fa62e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb252ff850 0x55cb25307940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:38.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.589+0000 7fa62e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb252ff850 0x55cb25307940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60810/0 (socket says 172.21.15.120:60810) 2024-09-19T09:04:38.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.589+0000 7fa62e59c640 1 -- 172.21.15.120:0/1825634918 learned_addr learned my addr 172.21.15.120:0/1825634918 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:38.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.589+0000 7fa627fff640 1 --2- 172.21.15.120:0/1825634918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb25300170 0x55cb25307e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:38.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.589+0000 7fa62ed9d640 1 --2- 172.21.15.120:0/1825634918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb2530aa30 0x55cb253083c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:38.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.589+0000 7fa62e59c640 1 -- 172.21.15.120:0/1825634918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb25300170 msgr2=0x55cb25307e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:38.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.589+0000 7fa62e59c640 1 --2- 172.21.15.120:0/1825634918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb25300170 0x55cb25307e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.589+0000 7fa62e59c640 1 -- 172.21.15.120:0/1825634918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb2530aa30 msgr2=0x55cb253083c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:38.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.589+0000 7fa62e59c640 1 --2- 172.21.15.120:0/1825634918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb2530aa30 0x55cb253083c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.589+0000 7fa62e59c640 1 -- 172.21.15.120:0/1825634918 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb2522cc50 con 0x55cb252ff850 2024-09-19T09:04:38.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.589+0000 7fa627fff640 1 --2- 172.21.15.120:0/1825634918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb25300170 0x55cb25307e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:04:38.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.590+0000 7fa62e59c640 1 --2- 172.21.15.120:0/1825634918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb252ff850 0x55cb25307940 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa6200126e0 tx=0x7fa620012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb314b8c3617947 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:38.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.590+0000 7fa615ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:04:38.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.590+0000 7fa615ffb640 1 -- 172.21.15.120:0/1825634918 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6200121d0 con 0x55cb252ff850 2024-09-19T09:04:38.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.590+0000 7fa615ffb640 1 -- 172.21.15.120:0/1825634918 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa620004050 con 0x55cb252ff850 2024-09-19T09:04:38.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.590+0000 7fa636c54740 1 -- 172.21.15.120:0/1825634918 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa628036070 con 0x55cb252ff850 2024-09-19T09:04:38.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.590+0000 7fa615ffb640 1 -- 172.21.15.120:0/1825634918 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa620004250 con 0x55cb252ff850 2024-09-19T09:04:38.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.590+0000 7fa636c54740 10 client.15993 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:38.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.590+0000 7fa636c54740 1 -- 172.21.15.120:0/1825634918 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cb253191c0 con 0x55cb252ff850 2024-09-19T09:04:38.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.590+0000 7fa615ffb640 1 -- 172.21.15.120:0/1825634918 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa628036070 con 0x55cb252ff850 2024-09-19T09:04:38.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.590+0000 7fa615ffb640 1 -- 172.21.15.120:0/1825634918 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa620003a00 con 0x55cb252ff850 2024-09-19T09:04:38.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.590+0000 7fa636c54740 10 client.15993 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:38.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.590+0000 7fa636c54740 10 client.15993 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:04:38.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.590+0000 7fa636c54740 20 client.15993 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:38.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.590+0000 7fa636c54740 1 --2- 172.21.15.120:0/1825634918 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cb2531a680 0x55cb2531caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:38.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.590+0000 7fa636c54740 4 client.15993 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:04:38.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.590+0000 7fa636c54740 1 -- 172.21.15.120:0/1825634918 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb25271890 con 0x55cb2531a680 2024-09-19T09:04:38.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.590+0000 7fa627fff640 1 --2- 172.21.15.120:0/1825634918 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cb2531a680 0x55cb2531caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:38.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.591+0000 7fa615ffb640 10 client.15993 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:38.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.591+0000 7fa627fff640 1 --2- 172.21.15.120:0/1825634918 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cb2531a680 0x55cb2531caf0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2974ad80b0839677 server_cookie=274e1eaed648d58e in_seq=0 out_seq=0 2024-09-19T09:04:38.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.593+0000 7fa615ffb640 1 -- 172.21.15.120:0/1825634918 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cb25271890 con 0x55cb2531a680 2024-09-19T09:04:38.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.593+0000 7fa615ffb640 10 client.15993 handle_command_reply: tid=0 2024-09-19T09:04:38.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.593+0000 7fa636c54740 1 client.15993 shutdown 2024-09-19T09:04:38.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.593+0000 7fa636c54740 1 -- 172.21.15.120:0/1825634918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb252ff850 msgr2=0x55cb25307940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:38.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.593+0000 7fa636c54740 1 --2- 172.21.15.120:0/1825634918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb252ff850 0x55cb25307940 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa6200126e0 tx=0x7fa620012ba0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.593+0000 7fa636c54740 1 -- 172.21.15.120:0/1825634918 shutdown_connections 2024-09-19T09:04:38.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.593+0000 7fa636c54740 1 --2- 172.21.15.120:0/1825634918 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cb2531a680 0x55cb2531caf0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.593+0000 7fa636c54740 1 --2- 172.21.15.120:0/1825634918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb2530aa30 0x55cb253083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.593+0000 7fa636c54740 1 --2- 172.21.15.120:0/1825634918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb25300170 0x55cb25307e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.593+0000 7fa636c54740 1 --2- 172.21.15.120:0/1825634918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb252ff850 0x55cb25307940 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.593+0000 7fa636c54740 1 -- 172.21.15.120:0/1825634918 >> 172.21.15.120:0/1825634918 conn(0x55cb252fbde0 msgr2=0x55cb25301b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:38.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.593+0000 7fa615ffb640 0 client.15993 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:38.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.593+0000 7fa636c54740 1 -- 172.21.15.120:0/1825634918 shutdown_connections 2024-09-19T09:04:38.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.593+0000 7fa636c54740 1 -- 172.21.15.120:0/1825634918 wait complete. 2024-09-19T09:04:38.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.593+0000 7fa636c54740 20 client.15993 trim_cache size 0 max 16384 2024-09-19T09:04:38.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.610+0000 7fa636c54740 1 Processor -- start 2024-09-19T09:04:38.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.610+0000 7fa636c54740 1 -- start start 2024-09-19T09:04:38.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.610+0000 7fa636c54740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb252ff850 0x55cb25301a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:38.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.611+0000 7fa636c54740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb25300170 0x55cb253223c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:38.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.611+0000 7fa636c54740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb2530aa30 0x55cb25322900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:38.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.611+0000 7fa636c54740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa6301af7d0 con 0x55cb25300170 2024-09-19T09:04:38.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.611+0000 7fa636c54740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa6301b1310 con 0x55cb252ff850 2024-09-19T09:04:38.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.611+0000 7fa636c54740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa6301afea0 con 0x55cb2530aa30 2024-09-19T09:04:38.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.611+0000 7fa627fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb25300170 0x55cb253223c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:38.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.611+0000 7fa62ed9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb2530aa30 0x55cb25322900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:38.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.611+0000 7fa62ed9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb2530aa30 0x55cb25322900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:40826/0 (socket says 172.21.15.120:40826) 2024-09-19T09:04:38.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.611+0000 7fa62ed9d640 1 -- 172.21.15.120:0/4135221309 learned_addr learned my addr 172.21.15.120:0/4135221309 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:38.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.611+0000 7fa62e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb252ff850 0x55cb25301a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:38.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.611+0000 7fa62ed9d640 1 -- 172.21.15.120:0/4135221309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb252ff850 msgr2=0x55cb25301a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:38.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.611+0000 7fa62ed9d640 1 --2- 172.21.15.120:0/4135221309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb252ff850 0x55cb25301a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.611+0000 7fa62ed9d640 1 -- 172.21.15.120:0/4135221309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb25300170 msgr2=0x55cb253223c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:38.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.611+0000 7fa62ed9d640 1 --2- 172.21.15.120:0/4135221309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb25300170 0x55cb253223c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.611+0000 7fa62ed9d640 1 -- 172.21.15.120:0/4135221309 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb252fbc10 con 0x55cb2530aa30 2024-09-19T09:04:38.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.611+0000 7fa62ed9d640 1 --2- 172.21.15.120:0/4135221309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb2530aa30 0x55cb25322900 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa628030f20 tx=0x7fa628033040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6535db2ad0b9c9d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:38.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.611+0000 7fa617fff640 1 -- 172.21.15.120:0/4135221309 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628061070 con 0x55cb2530aa30 2024-09-19T09:04:38.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.611+0000 7fa617fff640 1 -- 172.21.15.120:0/4135221309 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa628030aa0 con 0x55cb2530aa30 2024-09-19T09:04:38.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.612+0000 7fa617fff640 1 -- 172.21.15.120:0/4135221309 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628056040 con 0x55cb2530aa30 2024-09-19T09:04:38.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.612+0000 7fa636c54740 1 -- 172.21.15.120:0/4135221309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb2530aa30 msgr2=0x55cb25322900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:38.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.612+0000 7fa636c54740 1 --2- 172.21.15.120:0/4135221309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb2530aa30 0x55cb25322900 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa628030f20 tx=0x7fa628033040 comp rx=0 tx=0).stop 2024-09-19T09:04:38.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.613+0000 7fa636c54740 1 -- 172.21.15.120:0/4135221309 shutdown_connections 2024-09-19T09:04:38.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.613+0000 7fa636c54740 1 --2- 172.21.15.120:0/4135221309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb2530aa30 0x55cb25322900 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.613+0000 7fa636c54740 1 --2- 172.21.15.120:0/4135221309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb25300170 0x55cb253223c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.613+0000 7fa636c54740 1 --2- 172.21.15.120:0/4135221309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb252ff850 0x55cb25301a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.613+0000 7fa636c54740 1 -- 172.21.15.120:0/4135221309 >> 172.21.15.120:0/4135221309 conn(0x55cb252fbde0 msgr2=0x55cb253012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:38.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.613+0000 7fa636c54740 1 -- 172.21.15.120:0/4135221309 shutdown_connections 2024-09-19T09:04:38.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.613+0000 7fa636c54740 1 -- 172.21.15.120:0/4135221309 wait complete. 2024-09-19T09:04:38.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.613+0000 7fa636c54740 1 -- start start 2024-09-19T09:04:38.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.613+0000 7fa636c54740 1 Processor -- start 2024-09-19T09:04:38.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.613+0000 7fa636c54740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb252ff850 0x55cb2530eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:38.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.613+0000 7fa636c54740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb25300170 0x55cb2530f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:38.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.613+0000 7fa636c54740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb2530aa30 0x55cb2537e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:38.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.613+0000 7fa636c54740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa6301b0c40 con 0x55cb2530aa30 2024-09-19T09:04:38.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.613+0000 7fa636c54740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa6301b0570 con 0x55cb252ff850 2024-09-19T09:04:38.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.613+0000 7fa636c54740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa6301ad790 con 0x55cb25300170 2024-09-19T09:04:38.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.613+0000 7fa636c54740 10 client.0 _ll_register_callbacks cb 0x7fa620009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:38.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.614+0000 7fa62ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb2530aa30 0x55cb2537e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:38.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.614+0000 7fa62e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb252ff850 0x55cb2530eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:38.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.614+0000 7fa62ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb2530aa30 0x55cb2537e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60832/0 (socket says 172.21.15.120:60832) 2024-09-19T09:04:38.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.614+0000 7fa62ed9d640 1 -- 172.21.15.120:0/527153131 learned_addr learned my addr 172.21.15.120:0/527153131 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:38.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.614+0000 7fa62e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb252ff850 0x55cb2530eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:46826/0 (socket says 172.21.15.120:46826) 2024-09-19T09:04:38.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.614+0000 7fa627fff640 1 --2- 172.21.15.120:0/527153131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb25300170 0x55cb2530f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:38.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.614+0000 7fa62e59c640 1 -- 172.21.15.120:0/527153131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb25300170 msgr2=0x55cb2530f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:38.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.614+0000 7fa62e59c640 1 --2- 172.21.15.120:0/527153131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb25300170 0x55cb2530f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.614+0000 7fa62e59c640 1 -- 172.21.15.120:0/527153131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb2530aa30 msgr2=0x55cb2537e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:38.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.614+0000 7fa62e59c640 1 --2- 172.21.15.120:0/527153131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb2530aa30 0x55cb2537e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.614+0000 7fa62e59c640 1 -- 172.21.15.120:0/527153131 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb252fce90 con 0x55cb252ff850 2024-09-19T09:04:38.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.614+0000 7fa62ed9d640 1 --2- 172.21.15.120:0/527153131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb2530aa30 0x55cb2537e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:04:38.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.614+0000 7fa627fff640 1 --2- 172.21.15.120:0/527153131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb25300170 0x55cb2530f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:04:38.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.614+0000 7fa62e59c640 1 --2- 172.21.15.120:0/527153131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb252ff850 0x55cb2530eec0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa6200126b0 tx=0x7fa6200096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=789ec0c69a8b10d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:38.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.614+0000 7fa615ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:04:38.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.614+0000 7fa615ffb640 1 -- 172.21.15.120:0/527153131 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa620003f90 con 0x55cb252ff850 2024-09-19T09:04:38.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.614+0000 7fa615ffb640 1 -- 172.21.15.120:0/527153131 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa620018040 con 0x55cb252ff850 2024-09-19T09:04:38.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.614+0000 7fa636c54740 1 -- 172.21.15.120:0/527153131 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa628030aa0 con 0x55cb252ff850 2024-09-19T09:04:38.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.615+0000 7fa615ffb640 1 -- 172.21.15.120:0/527153131 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa620038050 con 0x55cb252ff850 2024-09-19T09:04:38.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.615+0000 7fa636c54740 10 client.25376 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:38.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.615+0000 7fa636c54740 1 -- 172.21.15.120:0/527153131 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cb25385b40 con 0x55cb252ff850 2024-09-19T09:04:38.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.615+0000 7fa615ffb640 1 -- 172.21.15.120:0/527153131 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa628030aa0 con 0x55cb252ff850 2024-09-19T09:04:38.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.615+0000 7fa615ffb640 1 -- 172.21.15.120:0/527153131 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa6200052a0 con 0x55cb252ff850 2024-09-19T09:04:38.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.615+0000 7fa636c54740 10 client.25376 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:38.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.615+0000 7fa636c54740 10 client.25376 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:04:38.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.615+0000 7fa636c54740 20 client.25376 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:38.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.615+0000 7fa636c54740 1 --2- 172.21.15.120:0/527153131 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cb25386330 0x55cb25388750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:38.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.615+0000 7fa636c54740 4 client.25376 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}] 2024-09-19T09:04:38.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.615+0000 7fa636c54740 1 -- 172.21.15.120:0/527153131 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}) -- 0x55cb252f7410 con 0x55cb25386330 2024-09-19T09:04:38.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.615+0000 7fa627fff640 1 --2- 172.21.15.120:0/527153131 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cb25386330 0x55cb25388750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:38.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.616+0000 7fa615ffb640 10 client.25376 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:38.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.616+0000 7fa627fff640 1 --2- 172.21.15.120:0/527153131 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cb25386330 0x55cb25388750 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5479d6c4c4204132 server_cookie=6afd6a879ac5d7a8 in_seq=0 out_seq=0 2024-09-19T09:04:38.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.720+0000 7fa615ffb640 1 -- 172.21.15.120:0/527153131 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55cb252f7410 con 0x55cb25386330 2024-09-19T09:04:38.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.720+0000 7fa615ffb640 10 client.25376 handle_command_reply: tid=0 2024-09-19T09:04:38.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.720+0000 7fa636c54740 1 client.25376 shutdown 2024-09-19T09:04:38.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.720+0000 7fa636c54740 1 -- 172.21.15.120:0/527153131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb252ff850 msgr2=0x55cb2530eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:38.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.720+0000 7fa636c54740 1 --2- 172.21.15.120:0/527153131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb252ff850 0x55cb2530eec0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa6200126b0 tx=0x7fa6200096c0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.720+0000 7fa636c54740 1 -- 172.21.15.120:0/527153131 shutdown_connections 2024-09-19T09:04:38.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.720+0000 7fa636c54740 1 --2- 172.21.15.120:0/527153131 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cb25386330 0x55cb25388750 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.720+0000 7fa636c54740 1 --2- 172.21.15.120:0/527153131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb2530aa30 0x55cb2537e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.720+0000 7fa615ffb640 0 client.25376 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:38.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.720+0000 7fa636c54740 1 --2- 172.21.15.120:0/527153131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb25300170 0x55cb2530f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.720+0000 7fa636c54740 1 --2- 172.21.15.120:0/527153131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb252ff850 0x55cb2530eec0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.720+0000 7fa636c54740 1 -- 172.21.15.120:0/527153131 >> 172.21.15.120:0/527153131 conn(0x55cb252fbde0 msgr2=0x55cb252faec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:38.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.720+0000 7fa636c54740 1 -- 172.21.15.120:0/527153131 shutdown_connections 2024-09-19T09:04:38.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.720+0000 7fa636c54740 1 -- 172.21.15.120:0/527153131 wait complete. 2024-09-19T09:04:38.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.720+0000 7fa636c54740 20 client.25376 trim_cache size 0 max 16384 2024-09-19T09:04:38.722 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:04:38.722 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 28, 2024-09-19T09:04:38.722 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:04:38.722 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 10, 2024-09-19T09:04:38.722 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:04:38.722 INFO:teuthology.orchestra.run.smithi120.stdout: "d6f3c5f1": { 2024-09-19T09:04:38.722 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 10, 2024-09-19T09:04:38.722 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:04:38.722 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:04:38.723 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:04:38.723 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:04:38.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:04:38.723 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:04:38.723 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 140.3, 2024-09-19T09:04:38.723 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:04:38.723 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:04:38.723 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:04:38.723 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:04:38.723 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:04:38.724 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:04:38.724 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:04:38.724 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:04:38.724 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:04:38.724 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:04:38.724 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:04:38.724 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:04:38.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.721+0000 7fa63487b640 1 -- 172.21.15.120:0/976293126 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa61c0a32c0 msgr2=0x7fa61c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:38.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.721+0000 7fa63487b640 1 --2- 172.21.15.120:0/976293126 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa61c0a32c0 0x7fa61c0a5780 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa628009bf0 tx=0x7fa628036000 comp rx=0 tx=0).stop 2024-09-19T09:04:38.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.721+0000 7fa63487b640 1 -- 172.21.15.120:0/976293126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6301027c0 msgr2=0x7fa63019ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:38.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.721+0000 7fa63487b640 1 --2- 172.21.15.120:0/976293126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6301027c0 0x7fa63019ab40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa61800c970 tx=0x7fa61800ce30 comp rx=0 tx=0).stop 2024-09-19T09:04:38.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.721+0000 7fa63487b640 1 -- 172.21.15.120:0/976293126 shutdown_connections 2024-09-19T09:04:38.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.721+0000 7fa63487b640 1 --2- 172.21.15.120:0/976293126 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa61c0a32c0 0x7fa61c0a5780 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.721+0000 7fa63487b640 1 --2- 172.21.15.120:0/976293126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa630106f40 0x7fa63019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.721+0000 7fa63487b640 1 --2- 172.21.15.120:0/976293126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6301027c0 0x7fa63019ab40 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.721+0000 7fa63487b640 1 --2- 172.21.15.120:0/976293126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa630101e10 0x7fa63019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:38.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.721+0000 7fa63487b640 1 -- 172.21.15.120:0/976293126 >> 172.21.15.120:0/976293126 conn(0x7fa6300fd8d0 msgr2=0x7fa6300febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:38.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.721+0000 7fa63487b640 1 -- 172.21.15.120:0/976293126 shutdown_connections 2024-09-19T09:04:38.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:38.721+0000 7fa63487b640 1 -- 172.21.15.120:0/976293126 wait complete. 2024-09-19T09:04:39.035 INFO:tasks.workunit.client.0.smithi120.stdout: 1 28960 5.12 MB/sec warmup 34 sec latency 394.387 ms 2024-09-19T09:04:39.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:38 smithi120 ceph-mon[31150]: pgmap v394: 97 pgs: 97 active+clean; 22 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 196 KiB/s wr, 47 op/s 2024-09-19T09:04:39.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:38 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:04:39.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:38 smithi196 ceph-mon[35150]: pgmap v394: 97 pgs: 97 active+clean; 22 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 196 KiB/s wr, 47 op/s 2024-09-19T09:04:39.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:38 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:04:39.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:38 smithi153 ceph-mon[35071]: pgmap v394: 97 pgs: 97 active+clean; 22 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 196 KiB/s wr, 47 op/s 2024-09-19T09:04:39.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:38 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:04:39.449 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd6f3c5f1', quiesce took 5.4 seconds. Will release after: 14.950000000000001 2024-09-19T09:04:40.036 INFO:tasks.workunit.client.0.smithi120.stdout: 1 28960 4.97 MB/sec warmup 35 sec latency 1394.479 ms 2024-09-19T09:04:41.036 INFO:tasks.workunit.client.0.smithi120.stdout: 1 28960 4.83 MB/sec warmup 36 sec latency 2394.578 ms 2024-09-19T09:04:41.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:40 smithi196 ceph-mon[35150]: pgmap v395: 97 pgs: 97 active+clean; 27 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 667 KiB/s wr, 137 op/s 2024-09-19T09:04:41.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:40 smithi153 ceph-mon[35071]: pgmap v395: 97 pgs: 97 active+clean; 27 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 667 KiB/s wr, 137 op/s 2024-09-19T09:04:41.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:40 smithi120 ceph-mon[31150]: pgmap v395: 97 pgs: 97 active+clean; 27 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 667 KiB/s wr, 137 op/s 2024-09-19T09:04:42.036 INFO:tasks.workunit.client.0.smithi120.stdout: 1 28960 4.70 MB/sec warmup 37 sec latency 3394.637 ms 2024-09-19T09:04:43.036 INFO:tasks.workunit.client.0.smithi120.stdout: 1 28960 4.58 MB/sec warmup 38 sec latency 4394.714 ms 2024-09-19T09:04:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:42 smithi196 ceph-mon[35150]: pgmap v396: 97 pgs: 97 active+clean; 28 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 832 KiB/s wr, 169 op/s 2024-09-19T09:04:43.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:42 smithi153 ceph-mon[35071]: pgmap v396: 97 pgs: 97 active+clean; 28 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 832 KiB/s wr, 169 op/s 2024-09-19T09:04:43.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:42 smithi120 ceph-mon[31150]: pgmap v396: 97 pgs: 97 active+clean; 28 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 832 KiB/s wr, 169 op/s 2024-09-19T09:04:44.036 INFO:tasks.workunit.client.0.smithi120.stdout: 1 28960 4.46 MB/sec warmup 39 sec latency 5394.808 ms 2024-09-19T09:04:45.036 INFO:tasks.workunit.client.0.smithi120.stdout: 1 28960 4.35 MB/sec warmup 40 sec latency 6394.891 ms 2024-09-19T09:04:45.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:44 smithi196 ceph-mon[35150]: pgmap v397: 97 pgs: 97 active+clean; 28 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 852 KiB/s wr, 171 op/s 2024-09-19T09:04:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:44 smithi153 ceph-mon[35071]: pgmap v397: 97 pgs: 97 active+clean; 28 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 852 KiB/s wr, 171 op/s 2024-09-19T09:04:45.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:44 smithi120 ceph-mon[31150]: pgmap v397: 97 pgs: 97 active+clean; 28 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 852 KiB/s wr, 171 op/s 2024-09-19T09:04:46.036 INFO:tasks.workunit.client.0.smithi120.stdout: 1 28960 4.24 MB/sec warmup 41 sec latency 7394.955 ms 2024-09-19T09:04:47.036 INFO:tasks.workunit.client.0.smithi120.stdout: 1 28960 4.14 MB/sec warmup 42 sec latency 8395.018 ms 2024-09-19T09:04:47.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:46 smithi196 ceph-mon[35150]: pgmap v398: 97 pgs: 97 active+clean; 29 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 859 KiB/s wr, 171 op/s 2024-09-19T09:04:47.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:46 smithi153 ceph-mon[35071]: pgmap v398: 97 pgs: 97 active+clean; 29 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 859 KiB/s wr, 171 op/s 2024-09-19T09:04:47.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:46 smithi120 ceph-mon[31150]: pgmap v398: 97 pgs: 97 active+clean; 29 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 859 KiB/s wr, 171 op/s 2024-09-19T09:04:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.016+0000 7facddd8d640 1 -- 172.21.15.120:0/30545621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facd8102600 msgr2=0x7facd8108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.016+0000 7facddd8d640 1 --2- 172.21.15.120:0/30545621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facd8102600 0x7facd8108e90 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7facd0009ca0 tx=0x7facd002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.017+0000 7facddd8d640 1 -- 172.21.15.120:0/30545621 shutdown_connections 2024-09-19T09:04:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.017+0000 7facddd8d640 1 --2- 172.21.15.120:0/30545621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facd8102600 0x7facd8108e90 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.017+0000 7facddd8d640 1 --2- 172.21.15.120:0/30545621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facd8101c40 0x7facd81020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.017+0000 7facddd8d640 1 --2- 172.21.15.120:0/30545621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facd8100a50 0x7facd8100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.017+0000 7facddd8d640 1 -- 172.21.15.120:0/30545621 >> 172.21.15.120:0/30545621 conn(0x7facd80fc1b0 msgr2=0x7facd80fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.017+0000 7facddd8d640 1 -- 172.21.15.120:0/30545621 shutdown_connections 2024-09-19T09:04:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.017+0000 7facddd8d640 1 -- 172.21.15.120:0/30545621 wait complete. 2024-09-19T09:04:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.017+0000 7facddd8d640 1 Processor -- start 2024-09-19T09:04:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.017+0000 7facddd8d640 1 -- start start 2024-09-19T09:04:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.018+0000 7facddd8d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facd8100a50 0x7facd806ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.018+0000 7facddd8d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facd8101c40 0x7facd819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.018+0000 7facddd8d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facd8102600 0x7facd819f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.018+0000 7facddd8d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7facd810c6b0 con 0x7facd8102600 2024-09-19T09:04:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.018+0000 7facddd8d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7facd810c530 con 0x7facd8100a50 2024-09-19T09:04:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.018+0000 7facddd8d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7facd810c830 con 0x7facd8101c40 2024-09-19T09:04:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.018+0000 7facd7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facd8102600 0x7facd819f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.018+0000 7facd7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facd8102600 0x7facd819f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52558/0 (socket says 172.21.15.120:52558) 2024-09-19T09:04:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.018+0000 7facd7fff640 1 -- 172.21.15.120:0/89651123 learned_addr learned my addr 172.21.15.120:0/89651123 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.018+0000 7facd77fe640 1 --2- 172.21.15.120:0/89651123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facd8100a50 0x7facd806ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.018+0000 7facd6ffd640 1 --2- 172.21.15.120:0/89651123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facd8101c40 0x7facd819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.018+0000 7facd7fff640 1 -- 172.21.15.120:0/89651123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facd8101c40 msgr2=0x7facd819cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.018+0000 7facd7fff640 1 --2- 172.21.15.120:0/89651123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facd8101c40 0x7facd819cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.018+0000 7facd7fff640 1 -- 172.21.15.120:0/89651123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facd8100a50 msgr2=0x7facd806ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.018+0000 7facd7fff640 1 --2- 172.21.15.120:0/89651123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facd8100a50 0x7facd806ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.018+0000 7facd7fff640 1 -- 172.21.15.120:0/89651123 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd819f9f0 con 0x7facd8102600 2024-09-19T09:04:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.018+0000 7facd6ffd640 1 --2- 172.21.15.120:0/89651123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facd8101c40 0x7facd819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:04:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.018+0000 7facd7fff640 1 --2- 172.21.15.120:0/89651123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facd8102600 0x7facd819f0e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7facd0005e30 tx=0x7facd0031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.018+0000 7facd4ff9640 1 -- 172.21.15.120:0/89651123 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd0003c30 con 0x7facd8102600 2024-09-19T09:04:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.018+0000 7facd4ff9640 1 -- 172.21.15.120:0/89651123 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facd0003dd0 con 0x7facd8102600 2024-09-19T09:04:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.018+0000 7facddd8d640 1 -- 172.21.15.120:0/89651123 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facd819fc20 con 0x7facd8102600 2024-09-19T09:04:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.018+0000 7facd4ff9640 1 -- 172.21.15.120:0/89651123 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd002fde0 con 0x7facd8102600 2024-09-19T09:04:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.018+0000 7facddd8d640 1 -- 172.21.15.120:0/89651123 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facd81a0170 con 0x7facd8102600 2024-09-19T09:04:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.021+0000 7facd4ff9640 1 -- 172.21.15.120:0/89651123 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7facd0004050 con 0x7facd8102600 2024-09-19T09:04:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.021+0000 7face0c00740 1 Processor -- start 2024-09-19T09:04:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.021+0000 7face0c00740 1 -- start start 2024-09-19T09:04:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.021+0000 7face0c00740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560523d6fdb0 0x560523d701b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.021+0000 7face0c00740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560523d706f0 0x560523d751b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.021+0000 7face0c00740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560523d75780 0x560523d77c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.021+0000 7face0c00740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560523ccf2f0 con 0x560523d6fdb0 2024-09-19T09:04:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.021+0000 7face0c00740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560523c77270 con 0x560523d706f0 2024-09-19T09:04:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.021+0000 7face0c00740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560523ccb2b0 con 0x560523d75780 2024-09-19T09:04:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.022+0000 7facd6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560523d6fdb0 0x560523d701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.022+0000 7facd4ff9640 1 --2- 172.21.15.120:0/89651123 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7facc00a32f0 0x7facc00a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.022+0000 7facd6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560523d6fdb0 0x560523d701b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52560/0 (socket says 172.21.15.120:52560) 2024-09-19T09:04:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.022+0000 7facd6ffd640 1 -- 172.21.15.120:0/3457715158 learned_addr learned my addr 172.21.15.120:0/3457715158 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.022+0000 7facd77fe640 1 --2- 172.21.15.120:0/3457715158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560523d706f0 0x560523d751b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.022+0000 7facd7fff640 1 --2- 172.21.15.120:0/89651123 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7facc00a32f0 0x7facc00a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.022+0000 7facd6ffd640 1 --2- 172.21.15.120:0/3457715158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560523d75780 0x560523d77c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.022+0000 7facd4ff9640 1 -- 172.21.15.120:0/89651123 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7facd00f0e30 con 0x7facd8102600 2024-09-19T09:04:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.022+0000 7facd6ffd640 1 -- 172.21.15.120:0/3457715158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560523d75780 msgr2=0x560523d77c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.022+0000 7facd6ffd640 1 --2- 172.21.15.120:0/3457715158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560523d75780 0x560523d77c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.022+0000 7facd6ffd640 1 -- 172.21.15.120:0/3457715158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560523d706f0 msgr2=0x560523d751b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.022+0000 7facd6ffd640 1 --2- 172.21.15.120:0/3457715158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560523d706f0 0x560523d751b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.022+0000 7facd6ffd640 1 -- 172.21.15.120:0/3457715158 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560523c76340 con 0x560523d6fdb0 2024-09-19T09:04:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.022+0000 7facd6ffd640 1 --2- 172.21.15.120:0/3457715158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560523d6fdb0 0x560523d701b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7faccc012440 tx=0x7faccc0085c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3deafeb7a6468c00 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.022+0000 7facbdffb640 1 -- 172.21.15.120:0/3457715158 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faccc018030 con 0x560523d6fdb0 2024-09-19T09:04:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.022+0000 7facd7fff640 1 --2- 172.21.15.120:0/89651123 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7facc00a32f0 0x7facc00a57b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7facd00904e0 tx=0x7facd00e8220 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.022+0000 7facbdffb640 1 -- 172.21.15.120:0/3457715158 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faccc005500 con 0x560523d6fdb0 2024-09-19T09:04:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.022+0000 7facbdffb640 1 -- 172.21.15.120:0/3457715158 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faccc00c7b0 con 0x560523d6fdb0 2024-09-19T09:04:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.023+0000 7face0c00740 1 -- 172.21.15.120:0/3457715158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560523d6fdb0 msgr2=0x560523d701b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.023+0000 7face0c00740 1 --2- 172.21.15.120:0/3457715158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560523d6fdb0 0x560523d701b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7faccc012440 tx=0x7faccc0085c0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.023+0000 7face0c00740 1 -- 172.21.15.120:0/3457715158 shutdown_connections 2024-09-19T09:04:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.023+0000 7face0c00740 1 --2- 172.21.15.120:0/3457715158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560523d75780 0x560523d77c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.023+0000 7face0c00740 1 --2- 172.21.15.120:0/3457715158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560523d706f0 0x560523d751b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.023+0000 7face0c00740 1 --2- 172.21.15.120:0/3457715158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560523d6fdb0 0x560523d701b0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.023+0000 7face0c00740 1 -- 172.21.15.120:0/3457715158 >> 172.21.15.120:0/3457715158 conn(0x560523d67010 msgr2=0x560523d67410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.023+0000 7face0c00740 1 -- 172.21.15.120:0/3457715158 shutdown_connections 2024-09-19T09:04:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.023+0000 7face0c00740 1 -- 172.21.15.120:0/3457715158 wait complete. 2024-09-19T09:04:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.023+0000 7face0c00740 1 -- start start 2024-09-19T09:04:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.024+0000 7face0c00740 1 Processor -- start 2024-09-19T09:04:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.024+0000 7face0c00740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560523d6fdb0 0x560523d79fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.024+0000 7face0c00740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560523d706f0 0x560523d7a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.024+0000 7face0c00740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560523d75780 0x560523d81550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.024+0000 7face0c00740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560523d78750 con 0x560523d6fdb0 2024-09-19T09:04:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.024+0000 7face0c00740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560523c58290 con 0x560523d706f0 2024-09-19T09:04:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.024+0000 7face0c00740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560523d788d0 con 0x560523d75780 2024-09-19T09:04:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.024+0000 7face0c00740 10 client.0 _ll_register_callbacks cb 0x560523c89140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.024+0000 7facd6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560523d6fdb0 0x560523d79fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.024+0000 7facd6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560523d6fdb0 0x560523d79fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52562/0 (socket says 172.21.15.120:52562) 2024-09-19T09:04:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.024+0000 7facd6ffd640 1 -- 172.21.15.120:0/3667003732 learned_addr learned my addr 172.21.15.120:0/3667003732 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.024+0000 7facd6ffd640 1 -- 172.21.15.120:0/3667003732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560523d75780 msgr2=0x560523d81550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.024+0000 7facd6ffd640 1 --2- 172.21.15.120:0/3667003732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560523d75780 0x560523d81550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.024+0000 7facd6ffd640 1 -- 172.21.15.120:0/3667003732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560523d706f0 msgr2=0x560523d7a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.024+0000 7facd6ffd640 1 --2- 172.21.15.120:0/3667003732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560523d706f0 0x560523d7a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.024+0000 7facd6ffd640 1 -- 172.21.15.120:0/3667003732 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605239e2210 con 0x560523d6fdb0 2024-09-19T09:04:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.025+0000 7facd6ffd640 1 --2- 172.21.15.120:0/3667003732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560523d6fdb0 0x560523d79fd0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7faccc0083f0 tx=0x7faccc004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda2eaecef52a357 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.025+0000 7facb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:04:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.025+0000 7facb6ffd640 1 -- 172.21.15.120:0/3667003732 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faccc00c550 con 0x560523d6fdb0 2024-09-19T09:04:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.025+0000 7facb6ffd640 1 -- 172.21.15.120:0/3667003732 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7faccc00c6f0 con 0x560523d6fdb0 2024-09-19T09:04:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.025+0000 7face0c00740 1 -- 172.21.15.120:0/3667003732 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faccc005500 con 0x560523d6fdb0 2024-09-19T09:04:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.025+0000 7facb6ffd640 1 -- 172.21.15.120:0/3667003732 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faccc00c8f0 con 0x560523d6fdb0 2024-09-19T09:04:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.025+0000 7face0c00740 10 client.16011 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.025+0000 7face0c00740 1 -- 172.21.15.120:0/3667003732 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560523aae620 con 0x560523d6fdb0 2024-09-19T09:04:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.025+0000 7facb6ffd640 1 -- 172.21.15.120:0/3667003732 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faccc005500 con 0x560523d6fdb0 2024-09-19T09:04:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.025+0000 7facb6ffd640 1 -- 172.21.15.120:0/3667003732 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faccc01db70 con 0x560523d6fdb0 2024-09-19T09:04:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.025+0000 7face0c00740 10 client.16011 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.025+0000 7face0c00740 10 client.16011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:04:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.025+0000 7face0c00740 20 client.16011 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.025+0000 7face0c00740 1 --2- 172.21.15.120:0/3667003732 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560523d85a40 0x560523d87e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.025+0000 7face0c00740 4 client.16011 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:04:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.025+0000 7face0c00740 1 -- 172.21.15.120:0/3667003732 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560523cdca20 con 0x560523d85a40 2024-09-19T09:04:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.026+0000 7facd77fe640 1 --2- 172.21.15.120:0/3667003732 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560523d85a40 0x560523d87e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.026+0000 7facb6ffd640 10 client.16011 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.026+0000 7facd77fe640 1 --2- 172.21.15.120:0/3667003732 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560523d85a40 0x560523d87e30 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e67b3e8bb0b51ba server_cookie=ade19da536e5c03e in_seq=0 out_seq=0 2024-09-19T09:04:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.027+0000 7facb6ffd640 1 -- 172.21.15.120:0/3667003732 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560523cdca20 con 0x560523d85a40 2024-09-19T09:04:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.027+0000 7facb6ffd640 10 client.16011 handle_command_reply: tid=0 2024-09-19T09:04:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.027+0000 7face0c00740 1 client.16011 shutdown 2024-09-19T09:04:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.027+0000 7face0c00740 1 -- 172.21.15.120:0/3667003732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560523d6fdb0 msgr2=0x560523d79fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.027+0000 7face0c00740 1 --2- 172.21.15.120:0/3667003732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560523d6fdb0 0x560523d79fd0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7faccc0083f0 tx=0x7faccc004b80 comp rx=0 tx=0).stop 2024-09-19T09:04:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.027+0000 7face0c00740 1 -- 172.21.15.120:0/3667003732 shutdown_connections 2024-09-19T09:04:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.027+0000 7face0c00740 1 --2- 172.21.15.120:0/3667003732 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560523d85a40 0x560523d87e30 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.027+0000 7face0c00740 1 --2- 172.21.15.120:0/3667003732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560523d75780 0x560523d81550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.027+0000 7face0c00740 1 --2- 172.21.15.120:0/3667003732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560523d706f0 0x560523d7a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.027+0000 7face0c00740 1 --2- 172.21.15.120:0/3667003732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560523d6fdb0 0x560523d79fd0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.027+0000 7facb6ffd640 0 client.16011 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.027+0000 7face0c00740 1 -- 172.21.15.120:0/3667003732 >> 172.21.15.120:0/3667003732 conn(0x560523d67010 msgr2=0x560523d68820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.027+0000 7face0c00740 1 -- 172.21.15.120:0/3667003732 shutdown_connections 2024-09-19T09:04:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.027+0000 7face0c00740 1 -- 172.21.15.120:0/3667003732 wait complete. 2024-09-19T09:04:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.027+0000 7face0c00740 20 client.16011 trim_cache size 0 max 16384 2024-09-19T09:04:48.036 INFO:tasks.workunit.client.0.smithi120.stdout: 1 28960 4.05 MB/sec warmup 43 sec latency 9395.050 ms 2024-09-19T09:04:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.042+0000 7face0c00740 1 Processor -- start 2024-09-19T09:04:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.042+0000 7face0c00740 1 -- start start 2024-09-19T09:04:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.042+0000 7face0c00740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560523d6fdb0 0x560523d7a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.042+0000 7face0c00740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560523d706f0 0x560523d81510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.042+0000 7face0c00740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560523d75780 0x560523d659a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.042+0000 7face0c00740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7facd81a6de0 con 0x560523d6fdb0 2024-09-19T09:04:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.042+0000 7face0c00740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7facd81a8920 con 0x560523d75780 2024-09-19T09:04:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.042+0000 7face0c00740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7facd81a74b0 con 0x560523d706f0 2024-09-19T09:04:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.042+0000 7facd6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560523d6fdb0 0x560523d7a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.042+0000 7facd6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560523d6fdb0 0x560523d7a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52574/0 (socket says 172.21.15.120:52574) 2024-09-19T09:04:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.042+0000 7facd6ffd640 1 -- 172.21.15.120:0/2451209381 learned_addr learned my addr 172.21.15.120:0/2451209381 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.042+0000 7facd77fe640 1 --2- 172.21.15.120:0/2451209381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560523d706f0 0x560523d81510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.042+0000 7facd6ffd640 1 -- 172.21.15.120:0/2451209381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560523d706f0 msgr2=0x560523d81510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.042+0000 7facd6ffd640 1 --2- 172.21.15.120:0/2451209381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560523d706f0 0x560523d81510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.042+0000 7facd6ffd640 1 -- 172.21.15.120:0/2451209381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560523d75780 msgr2=0x560523d659a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.042+0000 7facd6ffd640 1 --2- 172.21.15.120:0/2451209381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560523d75780 0x560523d659a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.042+0000 7facd6ffd640 1 -- 172.21.15.120:0/2451209381 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560523d79b10 con 0x560523d6fdb0 2024-09-19T09:04:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.042+0000 7facd77fe640 1 --2- 172.21.15.120:0/2451209381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560523d706f0 0x560523d81510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:04:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.042+0000 7facd6ffd640 1 --2- 172.21.15.120:0/2451209381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560523d6fdb0 0x560523d7a490 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7faccc000c00 tx=0x7faccc008520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed7aab5e6b641bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.043+0000 7facbd7fa640 1 -- 172.21.15.120:0/2451209381 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faccc018070 con 0x560523d6fdb0 2024-09-19T09:04:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.043+0000 7facbd7fa640 1 -- 172.21.15.120:0/2451209381 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faccc012150 con 0x560523d6fdb0 2024-09-19T09:04:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.043+0000 7facbd7fa640 1 -- 172.21.15.120:0/2451209381 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faccc00c750 con 0x560523d6fdb0 2024-09-19T09:04:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.043+0000 7face0c00740 1 -- 172.21.15.120:0/2451209381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560523d6fdb0 msgr2=0x560523d7a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.043+0000 7face0c00740 1 --2- 172.21.15.120:0/2451209381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560523d6fdb0 0x560523d7a490 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7faccc000c00 tx=0x7faccc008520 comp rx=0 tx=0).stop 2024-09-19T09:04:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.043+0000 7face0c00740 1 -- 172.21.15.120:0/2451209381 shutdown_connections 2024-09-19T09:04:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.043+0000 7face0c00740 1 --2- 172.21.15.120:0/2451209381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560523d75780 0x560523d659a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.043+0000 7face0c00740 1 --2- 172.21.15.120:0/2451209381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560523d706f0 0x560523d81510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.043+0000 7face0c00740 1 --2- 172.21.15.120:0/2451209381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560523d6fdb0 0x560523d7a490 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.043+0000 7face0c00740 1 -- 172.21.15.120:0/2451209381 >> 172.21.15.120:0/2451209381 conn(0x560523d67010 msgr2=0x560523d79d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.043+0000 7face0c00740 1 -- 172.21.15.120:0/2451209381 shutdown_connections 2024-09-19T09:04:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.043+0000 7face0c00740 1 -- 172.21.15.120:0/2451209381 wait complete. 2024-09-19T09:04:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.044+0000 7face0c00740 1 -- start start 2024-09-19T09:04:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.044+0000 7face0c00740 1 Processor -- start 2024-09-19T09:04:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.044+0000 7face0c00740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560523d6fdb0 0x560523d7e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.044+0000 7face0c00740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560523d706f0 0x560523d7eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.044+0000 7face0c00740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560523d75780 0x560523d7fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.044+0000 7face0c00740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7facd81a8250 con 0x560523d706f0 2024-09-19T09:04:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.044+0000 7face0c00740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7facd81a7b80 con 0x560523d6fdb0 2024-09-19T09:04:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.044+0000 7face0c00740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7facd81a4da0 con 0x560523d75780 2024-09-19T09:04:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.044+0000 7face0c00740 10 client.0 _ll_register_callbacks cb 0x7facc800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.044+0000 7facd77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560523d706f0 0x560523d7eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.044+0000 7facd77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560523d706f0 0x560523d7eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52576/0 (socket says 172.21.15.120:52576) 2024-09-19T09:04:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.044+0000 7facd77fe640 1 -- 172.21.15.120:0/521097220 learned_addr learned my addr 172.21.15.120:0/521097220 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.044+0000 7facd6ffd640 1 --2- 172.21.15.120:0/521097220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560523d6fdb0 0x560523d7e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.044+0000 7facd77fe640 1 -- 172.21.15.120:0/521097220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560523d75780 msgr2=0x560523d7fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.044+0000 7facd77fe640 1 --2- 172.21.15.120:0/521097220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560523d75780 0x560523d7fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.044+0000 7facd77fe640 1 -- 172.21.15.120:0/521097220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560523d6fdb0 msgr2=0x560523d7e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.044+0000 7facd77fe640 1 --2- 172.21.15.120:0/521097220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560523d6fdb0 0x560523d7e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.045+0000 7facd77fe640 1 -- 172.21.15.120:0/521097220 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560523d71380 con 0x560523d706f0 2024-09-19T09:04:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.045+0000 7facd77fe640 1 --2- 172.21.15.120:0/521097220 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560523d706f0 0x560523d7eeb0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7facc8004430 tx=0x7facc800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee4226b6b90bffa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.045+0000 7facb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:04:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.045+0000 7facb6ffd640 1 -- 172.21.15.120:0/521097220 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc8003eb0 con 0x560523d706f0 2024-09-19T09:04:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.045+0000 7facb6ffd640 1 -- 172.21.15.120:0/521097220 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7facc8011440 con 0x560523d706f0 2024-09-19T09:04:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.045+0000 7face0c00740 1 -- 172.21.15.120:0/521097220 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faccc012150 con 0x560523d706f0 2024-09-19T09:04:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.045+0000 7facb6ffd640 1 -- 172.21.15.120:0/521097220 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7facc8011640 con 0x560523d706f0 2024-09-19T09:04:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.045+0000 7face0c00740 10 client.16017 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.045+0000 7face0c00740 1 -- 172.21.15.120:0/521097220 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560523d88490 con 0x560523d706f0 2024-09-19T09:04:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.045+0000 7facb6ffd640 1 -- 172.21.15.120:0/521097220 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faccc012150 con 0x560523d706f0 2024-09-19T09:04:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.045+0000 7facb6ffd640 1 -- 172.21.15.120:0/521097220 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7facc80258e0 con 0x560523d706f0 2024-09-19T09:04:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.045+0000 7face0c00740 10 client.16017 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:48.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.045+0000 7face0c00740 10 client.16017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:04:48.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.045+0000 7face0c00740 20 client.16017 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:48.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.045+0000 7face0c00740 1 --2- 172.21.15.120:0/521097220 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560523df0ab0 0x560523df2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:48.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.045+0000 7face0c00740 4 client.16017 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:04:48.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.045+0000 7face0c00740 1 -- 172.21.15.120:0/521097220 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560523cd6d70 con 0x560523df0ab0 2024-09-19T09:04:48.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.046+0000 7facd6ffd640 1 --2- 172.21.15.120:0/521097220 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560523df0ab0 0x560523df2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:48.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.046+0000 7facb6ffd640 10 client.16017 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:48.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.046+0000 7facd6ffd640 1 --2- 172.21.15.120:0/521097220 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560523df0ab0 0x560523df2ed0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c06add924e4fc2f server_cookie=115de9dd93d245fe in_seq=0 out_seq=0 2024-09-19T09:04:48.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.096+0000 7facb6ffd640 1 -- 172.21.15.120:0/521097220 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560523cd6d70 con 0x560523df0ab0 2024-09-19T09:04:48.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.096+0000 7facb6ffd640 10 client.16017 handle_command_reply: tid=0 2024-09-19T09:04:48.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.096+0000 7face0c00740 1 client.16017 shutdown 2024-09-19T09:04:48.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.096+0000 7face0c00740 1 -- 172.21.15.120:0/521097220 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560523d706f0 msgr2=0x560523d7eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:48.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.096+0000 7face0c00740 1 --2- 172.21.15.120:0/521097220 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560523d706f0 0x560523d7eeb0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7facc8004430 tx=0x7facc800bea0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.096+0000 7face0c00740 1 -- 172.21.15.120:0/521097220 shutdown_connections 2024-09-19T09:04:48.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.096+0000 7face0c00740 1 --2- 172.21.15.120:0/521097220 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560523df0ab0 0x560523df2ed0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.096+0000 7face0c00740 1 --2- 172.21.15.120:0/521097220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560523d75780 0x560523d7fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.096+0000 7facb6ffd640 0 client.16017 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:48.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.096+0000 7face0c00740 1 --2- 172.21.15.120:0/521097220 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560523d706f0 0x560523d7eeb0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.096+0000 7face0c00740 1 --2- 172.21.15.120:0/521097220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560523d6fdb0 0x560523d7e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.096+0000 7face0c00740 1 -- 172.21.15.120:0/521097220 >> 172.21.15.120:0/521097220 conn(0x560523d67010 msgr2=0x560523d67ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.096+0000 7face0c00740 1 -- 172.21.15.120:0/521097220 shutdown_connections 2024-09-19T09:04:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.096+0000 7face0c00740 1 -- 172.21.15.120:0/521097220 wait complete. 2024-09-19T09:04:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.096+0000 7face0c00740 20 client.16017 trim_cache size 0 max 16384 2024-09-19T09:04:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:04:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:04:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:04:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:04:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.097+0000 7facddd8d640 1 -- 172.21.15.120:0/89651123 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7facc00a32f0 msgr2=0x7facc00a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.097+0000 7facddd8d640 1 --2- 172.21.15.120:0/89651123 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7facc00a32f0 0x7facc00a57b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7facd00904e0 tx=0x7facd00e8220 comp rx=0 tx=0).stop 2024-09-19T09:04:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.097+0000 7facddd8d640 1 -- 172.21.15.120:0/89651123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facd8102600 msgr2=0x7facd819f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.097+0000 7facddd8d640 1 --2- 172.21.15.120:0/89651123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facd8102600 0x7facd819f0e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7facd0005e30 tx=0x7facd0031ee0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.097+0000 7facddd8d640 1 -- 172.21.15.120:0/89651123 shutdown_connections 2024-09-19T09:04:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.097+0000 7facddd8d640 1 --2- 172.21.15.120:0/89651123 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7facc00a32f0 0x7facc00a57b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.097+0000 7facddd8d640 1 --2- 172.21.15.120:0/89651123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facd8102600 0x7facd819f0e0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.097+0000 7facddd8d640 1 --2- 172.21.15.120:0/89651123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facd8101c40 0x7facd819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.097+0000 7facddd8d640 1 --2- 172.21.15.120:0/89651123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facd8100a50 0x7facd806ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.097+0000 7facddd8d640 1 -- 172.21.15.120:0/89651123 >> 172.21.15.120:0/89651123 conn(0x7facd80fc1b0 msgr2=0x7facd8104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.097+0000 7facddd8d640 1 -- 172.21.15.120:0/89651123 shutdown_connections 2024-09-19T09:04:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:04:48.097+0000 7facddd8d640 1 -- 172.21.15.120:0/89651123 wait complete. 2024-09-19T09:04:49.036 INFO:tasks.workunit.client.0.smithi120.stdout: 1 28960 3.95 MB/sec warmup 44 sec latency 10395.146 ms 2024-09-19T09:04:49.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:48 smithi196 ceph-mon[35150]: pgmap v399: 97 pgs: 97 active+clean; 29 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 832 KiB/s wr, 166 op/s 2024-09-19T09:04:49.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:48 smithi153 ceph-mon[35071]: pgmap v399: 97 pgs: 97 active+clean; 29 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 832 KiB/s wr, 166 op/s 2024-09-19T09:04:49.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:48 smithi120 ceph-mon[31150]: pgmap v399: 97 pgs: 97 active+clean; 29 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 832 KiB/s wr, 166 op/s 2024-09-19T09:04:50.037 INFO:tasks.workunit.client.0.smithi120.stdout: 1 28960 3.87 MB/sec warmup 45 sec latency 11395.232 ms 2024-09-19T09:04:51.037 INFO:tasks.workunit.client.0.smithi120.stdout: 1 28960 3.78 MB/sec warmup 46 sec latency 12395.330 ms 2024-09-19T09:04:51.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:50 smithi196 ceph-mon[35150]: pgmap v400: 97 pgs: 97 active+clean; 25 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 722 KiB/s wr, 149 op/s 2024-09-19T09:04:51.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:50 smithi153 ceph-mon[35071]: pgmap v400: 97 pgs: 97 active+clean; 25 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 722 KiB/s wr, 149 op/s 2024-09-19T09:04:51.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:50 smithi120 ceph-mon[31150]: pgmap v400: 97 pgs: 97 active+clean; 25 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 722 KiB/s wr, 149 op/s 2024-09-19T09:04:52.037 INFO:tasks.workunit.client.0.smithi120.stdout: 1 28960 3.70 MB/sec warmup 47 sec latency 13395.386 ms 2024-09-19T09:04:53.037 INFO:tasks.workunit.client.0.smithi120.stdout: 1 28960 3.62 MB/sec warmup 48 sec latency 14395.511 ms 2024-09-19T09:04:53.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:52 smithi196 ceph-mon[35150]: pgmap v401: 97 pgs: 97 active+clean; 25 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 208 KiB/s wr, 69 op/s 2024-09-19T09:04:53.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:52 smithi153 ceph-mon[35071]: pgmap v401: 97 pgs: 97 active+clean; 25 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 208 KiB/s wr, 69 op/s 2024-09-19T09:04:53.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:52 smithi120 ceph-mon[31150]: pgmap v401: 97 pgs: 97 active+clean; 25 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 208 KiB/s wr, 69 op/s 2024-09-19T09:04:54.037 INFO:tasks.workunit.client.0.smithi120.stdout: 1 28960 3.55 MB/sec warmup 49 sec latency 15395.638 ms 2024-09-19T09:04:54.400 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:04:54.647 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:04:55.037 INFO:tasks.workunit.client.0.smithi120.stdout: 1 28960 3.48 MB/sec warmup 50 sec latency 16395.731 ms 2024-09-19T09:04:55.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:54 smithi120 ceph-mon[31150]: pgmap v402: 97 pgs: 97 active+clean; 25 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 44 KiB/s wr, 52 op/s 2024-09-19T09:04:55.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:54 smithi196 ceph-mon[35150]: pgmap v402: 97 pgs: 97 active+clean; 25 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 44 KiB/s wr, 52 op/s 2024-09-19T09:04:55.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:54 smithi153 ceph-mon[35071]: pgmap v402: 97 pgs: 97 active+clean; 25 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 44 KiB/s wr, 52 op/s 2024-09-19T09:04:55.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.190+0000 7f9ca3baa640 1 -- 172.21.15.120:0/3451155353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c9c101420 msgr2=0x7f9c9c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:55.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.190+0000 7f9ca3baa640 1 --2- 172.21.15.120:0/3451155353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c9c101420 0x7f9c9c10d950 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009cd0 tx=0x7f9c9402a8e0 comp rx=0 tx=0).stop 2024-09-19T09:04:55.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.190+0000 7f9ca3baa640 1 -- 172.21.15.120:0/3451155353 shutdown_connections 2024-09-19T09:04:55.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.190+0000 7f9ca3baa640 1 --2- 172.21.15.120:0/3451155353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c9c101420 0x7f9c9c10d950 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:55.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.190+0000 7f9ca3baa640 1 --2- 172.21.15.120:0/3451155353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c9c100a80 0x7f9c9c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:55.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.190+0000 7f9ca3baa640 1 --2- 172.21.15.120:0/3451155353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c9c106a80 0x7f9c9c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:55.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.190+0000 7f9ca3baa640 1 -- 172.21.15.120:0/3451155353 >> 172.21.15.120:0/3451155353 conn(0x7f9c9c0fc7a0 msgr2=0x7f9c9c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:55.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.190+0000 7f9ca3baa640 1 -- 172.21.15.120:0/3451155353 shutdown_connections 2024-09-19T09:04:55.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.190+0000 7f9ca3baa640 1 -- 172.21.15.120:0/3451155353 wait complete. 2024-09-19T09:04:55.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.191+0000 7f9ca3baa640 1 Processor -- start 2024-09-19T09:04:55.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.191+0000 7f9ca3baa640 1 -- start start 2024-09-19T09:04:55.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.191+0000 7f9ca3baa640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c9c100a80 0x7f9c9c1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:55.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.191+0000 7f9ca3baa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c9c101420 0x7f9c9c199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:55.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.191+0000 7f9ca3baa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c9c106a80 0x7f9c9c1a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:55.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.191+0000 7f9ca3baa640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9c110ea0 con 0x7f9c9c101420 2024-09-19T09:04:55.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.191+0000 7f9ca3baa640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9c110d20 con 0x7f9c9c100a80 2024-09-19T09:04:55.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.191+0000 7f9ca3baa640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9c111020 con 0x7f9c9c106a80 2024-09-19T09:04:55.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.191+0000 7f9ca1145640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c9c101420 0x7f9c9c199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:55.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.191+0000 7f9ca1145640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c9c101420 0x7f9c9c199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35788/0 (socket says 172.21.15.120:35788) 2024-09-19T09:04:55.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.191+0000 7f9ca1145640 1 -- 172.21.15.120:0/3368669578 learned_addr learned my addr 172.21.15.120:0/3368669578 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:55.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.191+0000 7f9ca2147640 1 --2- 172.21.15.120:0/3368669578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c9c106a80 0x7f9c9c1a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:55.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.191+0000 7f9ca1946640 1 --2- 172.21.15.120:0/3368669578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c9c100a80 0x7f9c9c1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:55.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.191+0000 7f9ca1145640 1 -- 172.21.15.120:0/3368669578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c9c106a80 msgr2=0x7f9c9c1a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:55.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.191+0000 7f9ca1145640 1 --2- 172.21.15.120:0/3368669578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c9c106a80 0x7f9c9c1a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:55.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.191+0000 7f9ca1145640 1 -- 172.21.15.120:0/3368669578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c9c100a80 msgr2=0x7f9c9c1992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:55.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.191+0000 7f9ca1145640 1 --2- 172.21.15.120:0/3368669578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c9c100a80 0x7f9c9c1992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:55.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.191+0000 7f9ca1145640 1 -- 172.21.15.120:0/3368669578 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c9c1a4890 con 0x7f9c9c101420 2024-09-19T09:04:55.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.191+0000 7f9ca1145640 1 --2- 172.21.15.120:0/3368669578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c9c101420 0x7f9c9c199810 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800c890 tx=0x7f9c9800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:55.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.192+0000 7f9c8effd640 1 -- 172.21.15.120:0/3368669578 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98004220 con 0x7f9c9c101420 2024-09-19T09:04:55.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.192+0000 7f9c8effd640 1 -- 172.21.15.120:0/3368669578 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c980043c0 con 0x7f9c9c101420 2024-09-19T09:04:55.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.192+0000 7f9ca3baa640 1 -- 172.21.15.120:0/3368669578 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c9c1a4a60 con 0x7f9c9c101420 2024-09-19T09:04:55.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.192+0000 7f9c8effd640 1 -- 172.21.15.120:0/3368669578 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c980049e0 con 0x7f9c9c101420 2024-09-19T09:04:55.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.192+0000 7f9ca3baa640 1 -- 172.21.15.120:0/3368669578 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c9c1a4f70 con 0x7f9c9c101420 2024-09-19T09:04:55.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.193+0000 7f9ca3baa640 1 -- 172.21.15.120:0/3368669578 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c9c100ee0 con 0x7f9c9c101420 2024-09-19T09:04:55.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.194+0000 7f9c8effd640 1 -- 172.21.15.120:0/3368669578 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f9c980079b0 con 0x7f9c9c101420 2024-09-19T09:04:55.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.194+0000 7f9c8effd640 1 --2- 172.21.15.120:0/3368669578 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9c880a3240 0x7f9c880a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:55.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.195+0000 7f9c8effd640 1 -- 172.21.15.120:0/3368669578 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f9c980cfa50 con 0x7f9c9c101420 2024-09-19T09:04:55.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.195+0000 7f9ca1946640 1 --2- 172.21.15.120:0/3368669578 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9c880a3240 0x7f9c880a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:55.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.195+0000 7f9ca1946640 1 --2- 172.21.15.120:0/3368669578 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9c880a3240 0x7f9c880a5700 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9c90009890 tx=0x7f9c90008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:55.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.198+0000 7f9c8effd640 1 -- 172.21.15.120:0/3368669578 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9c98014030 con 0x7f9c9c101420 2024-09-19T09:04:55.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.401+0000 7f9ca3baa640 1 -- 172.21.15.120:0/3368669578 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c9c06ad80 con 0x7f9c9c101420 2024-09-19T09:04:55.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.402+0000 7f9c8effd640 1 -- 172.21.15.120:0/3368669578 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f9c9c06ad80 con 0x7f9c9c101420 2024-09-19T09:04:55.403 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:04:55.403 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":28,"btime":"2024-09-19T09:02:41:696728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:02:41.207357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:04:55.403 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 28 2024-09-19T09:04:55.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.404+0000 7f9ca3baa640 1 -- 172.21.15.120:0/3368669578 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9c880a3240 msgr2=0x7f9c880a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:55.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.404+0000 7f9ca3baa640 1 --2- 172.21.15.120:0/3368669578 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9c880a3240 0x7f9c880a5700 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9c90009890 tx=0x7f9c90008040 comp rx=0 tx=0).stop 2024-09-19T09:04:55.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.404+0000 7f9ca3baa640 1 -- 172.21.15.120:0/3368669578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c9c101420 msgr2=0x7f9c9c199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:55.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.404+0000 7f9ca3baa640 1 --2- 172.21.15.120:0/3368669578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c9c101420 0x7f9c9c199810 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800c890 tx=0x7f9c9800cd50 comp rx=0 tx=0).stop 2024-09-19T09:04:55.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.404+0000 7f9ca3baa640 1 -- 172.21.15.120:0/3368669578 shutdown_connections 2024-09-19T09:04:55.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.404+0000 7f9ca3baa640 1 --2- 172.21.15.120:0/3368669578 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9c880a3240 0x7f9c880a5700 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:55.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.404+0000 7f9ca3baa640 1 --2- 172.21.15.120:0/3368669578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c9c106a80 0x7f9c9c1a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:55.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.404+0000 7f9ca3baa640 1 --2- 172.21.15.120:0/3368669578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c9c101420 0x7f9c9c199810 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:55.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.404+0000 7f9ca3baa640 1 --2- 172.21.15.120:0/3368669578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c9c100a80 0x7f9c9c1992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:55.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.404+0000 7f9ca3baa640 1 -- 172.21.15.120:0/3368669578 >> 172.21.15.120:0/3368669578 conn(0x7f9c9c0fc7a0 msgr2=0x7f9c9c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:55.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.404+0000 7f9ca3baa640 1 -- 172.21.15.120:0/3368669578 shutdown_connections 2024-09-19T09:04:55.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:55.404+0000 7f9ca3baa640 1 -- 172.21.15.120:0/3368669578 wait complete. 2024-09-19T09:04:56.037 INFO:tasks.workunit.client.0.smithi120.stdout: 1 28960 3.41 MB/sec warmup 51 sec latency 17395.797 ms 2024-09-19T09:04:56.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:55 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3368669578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:04:56.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:55 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3368669578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:04:56.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:55 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3368669578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:04:56.391 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db --set-id d6f3c5f1 --release --await' 2024-09-19T09:04:56.392 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db --set-id d6f3c5f1 --release --await 2024-09-19T09:04:56.642 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:04:57.037 INFO:tasks.workunit.client.0.smithi120.stdout: 1 28960 3.35 MB/sec warmup 52 sec latency 18395.848 ms 2024-09-19T09:04:57.072 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:56 smithi120 ceph-mon[31150]: pgmap v403: 97 pgs: 97 active+clean; 25 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 90 KiB/s wr, 51 op/s 2024-09-19T09:04:57.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.158+0000 7fa44167c640 1 -- 172.21.15.120:0/780395204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa43c106a80 msgr2=0x7fa43c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:57.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.158+0000 7fa44167c640 1 --2- 172.21.15.120:0/780395204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa43c106a80 0x7fa43c106e60 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009cd0 tx=0x7fa42c02a8e0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.158+0000 7fa44167c640 1 -- 172.21.15.120:0/780395204 shutdown_connections 2024-09-19T09:04:57.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.158+0000 7fa44167c640 1 --2- 172.21.15.120:0/780395204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa43c101420 0x7fa43c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.158+0000 7fa44167c640 1 --2- 172.21.15.120:0/780395204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa43c100a80 0x7fa43c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.158+0000 7fa44167c640 1 --2- 172.21.15.120:0/780395204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa43c106a80 0x7fa43c106e60 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.158+0000 7fa44167c640 1 -- 172.21.15.120:0/780395204 >> 172.21.15.120:0/780395204 conn(0x7fa43c0fc7a0 msgr2=0x7fa43c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:57.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.158+0000 7fa44167c640 1 -- 172.21.15.120:0/780395204 shutdown_connections 2024-09-19T09:04:57.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.158+0000 7fa44167c640 1 -- 172.21.15.120:0/780395204 wait complete. 2024-09-19T09:04:57.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.158+0000 7fa44167c640 1 Processor -- start 2024-09-19T09:04:57.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.159+0000 7fa44167c640 1 -- start start 2024-09-19T09:04:57.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.159+0000 7fa44167c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa43c100a80 0x7fa43c199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:57.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.159+0000 7fa44167c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa43c101420 0x7fa43c1998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:57.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.159+0000 7fa44167c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa43c106a80 0x7fa43c1a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:57.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.159+0000 7fa44167c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c110ec0 con 0x7fa43c106a80 2024-09-19T09:04:57.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.159+0000 7fa44167c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c110d40 con 0x7fa43c101420 2024-09-19T09:04:57.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.159+0000 7fa44167c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c111040 con 0x7fa43c100a80 2024-09-19T09:04:57.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.159+0000 7fa43b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa43c106a80 0x7fa43c1a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:57.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.159+0000 7fa43b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa43c106a80 0x7fa43c1a4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35814/0 (socket says 172.21.15.120:35814) 2024-09-19T09:04:57.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.159+0000 7fa43b7fe640 1 -- 172.21.15.120:0/2151403798 learned_addr learned my addr 172.21.15.120:0/2151403798 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:57.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.159+0000 7fa43a7fc640 1 --2- 172.21.15.120:0/2151403798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa43c101420 0x7fa43c1998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:57.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.159+0000 7fa43affd640 1 --2- 172.21.15.120:0/2151403798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa43c100a80 0x7fa43c199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:57.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.159+0000 7fa43b7fe640 1 -- 172.21.15.120:0/2151403798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa43c100a80 msgr2=0x7fa43c199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:57.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.159+0000 7fa43b7fe640 1 --2- 172.21.15.120:0/2151403798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa43c100a80 0x7fa43c199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.159+0000 7fa43b7fe640 1 -- 172.21.15.120:0/2151403798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa43c101420 msgr2=0x7fa43c1998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:57.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.159+0000 7fa43b7fe640 1 --2- 172.21.15.120:0/2151403798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa43c101420 0x7fa43c1998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.159+0000 7fa43b7fe640 1 -- 172.21.15.120:0/2151403798 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa43c1a4950 con 0x7fa43c106a80 2024-09-19T09:04:57.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.159+0000 7fa43b7fe640 1 --2- 172.21.15.120:0/2151403798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa43c106a80 0x7fa43c1a4410 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa434009fd0 tx=0x7fa43400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:57.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.160+0000 7fa433fff640 1 -- 172.21.15.120:0/2151403798 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa434011830 con 0x7fa43c106a80 2024-09-19T09:04:57.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.160+0000 7fa44167c640 1 -- 172.21.15.120:0/2151403798 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa43c1a4b20 con 0x7fa43c106a80 2024-09-19T09:04:57.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.160+0000 7fa433fff640 1 -- 172.21.15.120:0/2151403798 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa434016070 con 0x7fa43c106a80 2024-09-19T09:04:57.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.160+0000 7fa433fff640 1 -- 172.21.15.120:0/2151403798 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43400f2e0 con 0x7fa43c106a80 2024-09-19T09:04:57.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.160+0000 7fa44167c640 1 -- 172.21.15.120:0/2151403798 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa43c1a4fe0 con 0x7fa43c106a80 2024-09-19T09:04:57.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.162+0000 7fa433fff640 1 -- 172.21.15.120:0/2151403798 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa4340119d0 con 0x7fa43c106a80 2024-09-19T09:04:57.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.162+0000 7fa443a55740 1 Processor -- start 2024-09-19T09:04:57.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.163+0000 7fa443a55740 1 -- start start 2024-09-19T09:04:57.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.163+0000 7fa443a55740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561afd521060 0x561afd521440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:57.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.163+0000 7fa443a55740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561afd521980 0x561afd52bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:57.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.163+0000 7fa443a55740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561afd52c240 0x561afd52e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:57.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.163+0000 7fa443a55740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561afd486620 con 0x561afd521060 2024-09-19T09:04:57.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.163+0000 7fa443a55740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561afd42de50 con 0x561afd521980 2024-09-19T09:04:57.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.163+0000 7fa443a55740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561afd4825e0 con 0x561afd52c240 2024-09-19T09:04:57.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.163+0000 7fa43a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561afd521060 0x561afd521440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:57.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.163+0000 7fa433fff640 1 --2- 172.21.15.120:0/2151403798 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa4280a3240 0x7fa4280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:57.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.163+0000 7fa43a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561afd521060 0x561afd521440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35826/0 (socket says 172.21.15.120:35826) 2024-09-19T09:04:57.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.163+0000 7fa43a7fc640 1 -- 172.21.15.120:0/643316949 learned_addr learned my addr 172.21.15.120:0/643316949 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:57.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.163+0000 7fa43affd640 1 --2- 172.21.15.120:0/643316949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561afd521980 0x561afd52bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:57.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.163+0000 7fa43b7fe640 1 --2- 172.21.15.120:0/2151403798 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa4280a3240 0x7fa4280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:57.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.163+0000 7fa43a7fc640 1 -- 172.21.15.120:0/643316949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561afd52c240 msgr2=0x561afd52e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:57.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.163+0000 7fa43a7fc640 1 --2- 172.21.15.120:0/643316949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561afd52c240 0x561afd52e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.163+0000 7fa43a7fc640 1 -- 172.21.15.120:0/643316949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561afd521980 msgr2=0x561afd52bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:57.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.163+0000 7fa43a7fc640 1 --2- 172.21.15.120:0/643316949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561afd521980 0x561afd52bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.163+0000 7fa433fff640 1 -- 172.21.15.120:0/2151403798 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa4340cf070 con 0x7fa43c106a80 2024-09-19T09:04:57.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.163+0000 7fa43a7fc640 1 -- 172.21.15.120:0/643316949 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561afd41ed40 con 0x561afd521060 2024-09-19T09:04:57.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.163+0000 7fa43affd640 1 --2- 172.21.15.120:0/643316949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561afd521980 0x561afd52bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:04:57.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.164+0000 7fa43a7fc640 1 --2- 172.21.15.120:0/643316949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561afd521060 0x561afd521440 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa42400cc20 tx=0x7fa424010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff1926479bf799f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:57.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.164+0000 7fa43b7fe640 1 --2- 172.21.15.120:0/2151403798 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa4280a3240 0x7fa4280a5700 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa434097af0 tx=0x7fa43406d460 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:57.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.164+0000 7fa4317fa640 1 -- 172.21.15.120:0/643316949 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa424016030 con 0x561afd521060 2024-09-19T09:04:57.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.164+0000 7fa4317fa640 1 -- 172.21.15.120:0/643316949 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa424012960 con 0x561afd521060 2024-09-19T09:04:57.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.164+0000 7fa4317fa640 1 -- 172.21.15.120:0/643316949 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa424011910 con 0x561afd521060 2024-09-19T09:04:57.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.165+0000 7fa443a55740 1 -- 172.21.15.120:0/643316949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561afd521060 msgr2=0x561afd521440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:57.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.165+0000 7fa443a55740 1 --2- 172.21.15.120:0/643316949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561afd521060 0x561afd521440 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa42400cc20 tx=0x7fa424010740 comp rx=0 tx=0).stop 2024-09-19T09:04:57.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.165+0000 7fa443a55740 1 -- 172.21.15.120:0/643316949 shutdown_connections 2024-09-19T09:04:57.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.165+0000 7fa443a55740 1 --2- 172.21.15.120:0/643316949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561afd52c240 0x561afd52e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.165+0000 7fa443a55740 1 --2- 172.21.15.120:0/643316949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561afd521980 0x561afd52bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.165+0000 7fa443a55740 1 --2- 172.21.15.120:0/643316949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561afd521060 0x561afd521440 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.165+0000 7fa443a55740 1 -- 172.21.15.120:0/643316949 >> 172.21.15.120:0/643316949 conn(0x561afd51d5f0 msgr2=0x561afd51d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:57.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.165+0000 7fa443a55740 1 -- 172.21.15.120:0/643316949 shutdown_connections 2024-09-19T09:04:57.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.165+0000 7fa443a55740 1 -- 172.21.15.120:0/643316949 wait complete. 2024-09-19T09:04:57.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.165+0000 7fa443a55740 1 -- start start 2024-09-19T09:04:57.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.165+0000 7fa443a55740 1 Processor -- start 2024-09-19T09:04:57.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.165+0000 7fa443a55740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561afd521060 0x561afd529180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:57.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.165+0000 7fa443a55740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561afd521980 0x561afd5296c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:57.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.165+0000 7fa443a55740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561afd52c240 0x561afd529c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:57.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.165+0000 7fa443a55740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561afd52ee10 con 0x561afd521980 2024-09-19T09:04:57.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.165+0000 7fa443a55740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561afd40ef10 con 0x561afd521060 2024-09-19T09:04:57.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.165+0000 7fa443a55740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561afd52ef90 con 0x561afd52c240 2024-09-19T09:04:57.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.165+0000 7fa443a55740 10 client.0 _ll_register_callbacks cb 0x561afd432db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:57.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.166+0000 7fa43affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561afd521980 0x561afd5296c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:57.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.166+0000 7fa43affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561afd521980 0x561afd5296c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35828/0 (socket says 172.21.15.120:35828) 2024-09-19T09:04:57.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.166+0000 7fa43affd640 1 -- 172.21.15.120:0/3698445429 learned_addr learned my addr 172.21.15.120:0/3698445429 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:57.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.166+0000 7fa43a7fc640 1 --2- 172.21.15.120:0/3698445429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561afd52c240 0x561afd529c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:57.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.166+0000 7fa43affd640 1 -- 172.21.15.120:0/3698445429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561afd52c240 msgr2=0x561afd529c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:57.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.166+0000 7fa43affd640 1 --2- 172.21.15.120:0/3698445429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561afd52c240 0x561afd529c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.166+0000 7fa43affd640 1 -- 172.21.15.120:0/3698445429 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561afd521060 msgr2=0x561afd529180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:57.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.166+0000 7fa43a7fc640 1 --2- 172.21.15.120:0/3698445429 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561afd521060 0x561afd529180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:57.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.166+0000 7fa43affd640 1 --2- 172.21.15.120:0/3698445429 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561afd521060 0x561afd529180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.166+0000 7fa43affd640 1 -- 172.21.15.120:0/3698445429 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561afd195580 con 0x561afd521980 2024-09-19T09:04:57.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.166+0000 7fa43a7fc640 1 --2- 172.21.15.120:0/3698445429 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561afd521060 0x561afd529180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:04:57.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.166+0000 7fa43affd640 1 --2- 172.21.15.120:0/3698445429 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561afd521980 0x561afd5296c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa42c02ae00 tx=0x7fa42c031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf71a4f5fe4662ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:57.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.166+0000 7fa4227fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:04:57.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.166+0000 7fa4227fc640 1 -- 172.21.15.120:0/3698445429 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c047020 con 0x561afd521980 2024-09-19T09:04:57.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.166+0000 7fa443a55740 1 -- 172.21.15.120:0/3698445429 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa424012960 con 0x561afd521980 2024-09-19T09:04:57.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.166+0000 7fa4227fc640 1 -- 172.21.15.120:0/3698445429 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa42c003b80 con 0x561afd521980 2024-09-19T09:04:57.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.166+0000 7fa4227fc640 1 -- 172.21.15.120:0/3698445429 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa42c003d80 con 0x561afd521980 2024-09-19T09:04:57.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.166+0000 7fa443a55740 10 client.16035 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:57.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.166+0000 7fa443a55740 1 -- 172.21.15.120:0/3698445429 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561afd25f960 con 0x561afd521980 2024-09-19T09:04:57.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.167+0000 7fa4227fc640 1 -- 172.21.15.120:0/3698445429 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa424012960 con 0x561afd521980 2024-09-19T09:04:57.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.167+0000 7fa4227fc640 1 -- 172.21.15.120:0/3698445429 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa42c04c070 con 0x561afd521980 2024-09-19T09:04:57.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.167+0000 7fa443a55740 10 client.16035 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:57.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.167+0000 7fa443a55740 10 client.16035 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:04:57.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.167+0000 7fa443a55740 20 client.16035 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:57.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.167+0000 7fa443a55740 1 --2- 172.21.15.120:0/3698445429 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561afd53bda0 0x561afd53e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:57.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.167+0000 7fa443a55740 4 client.16035 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:04:57.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.167+0000 7fa443a55740 1 -- 172.21.15.120:0/3698445429 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561afd493b90 con 0x561afd53bda0 2024-09-19T09:04:57.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.167+0000 7fa43a7fc640 1 --2- 172.21.15.120:0/3698445429 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561afd53bda0 0x561afd53e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:57.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.168+0000 7fa4227fc640 10 client.16035 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:57.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.168+0000 7fa43a7fc640 1 --2- 172.21.15.120:0/3698445429 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561afd53bda0 0x561afd53e1f0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3ea363d7b787fba server_cookie=c8dc04c2a60c87f in_seq=0 out_seq=0 2024-09-19T09:04:57.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.168+0000 7fa4227fc640 1 -- 172.21.15.120:0/3698445429 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561afd493b90 con 0x561afd53bda0 2024-09-19T09:04:57.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.168+0000 7fa4227fc640 10 client.16035 handle_command_reply: tid=0 2024-09-19T09:04:57.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.168+0000 7fa443a55740 1 client.16035 shutdown 2024-09-19T09:04:57.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.168+0000 7fa443a55740 1 -- 172.21.15.120:0/3698445429 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561afd521980 msgr2=0x561afd5296c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:57.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.168+0000 7fa443a55740 1 --2- 172.21.15.120:0/3698445429 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561afd521980 0x561afd5296c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa42c02ae00 tx=0x7fa42c031b60 comp rx=0 tx=0).stop 2024-09-19T09:04:57.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.169+0000 7fa443a55740 1 -- 172.21.15.120:0/3698445429 shutdown_connections 2024-09-19T09:04:57.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.169+0000 7fa443a55740 1 --2- 172.21.15.120:0/3698445429 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561afd53bda0 0x561afd53e1f0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.169+0000 7fa443a55740 1 --2- 172.21.15.120:0/3698445429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561afd52c240 0x561afd529c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.169+0000 7fa4227fc640 0 client.16035 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:57.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.169+0000 7fa443a55740 1 --2- 172.21.15.120:0/3698445429 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561afd521980 0x561afd5296c0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.169+0000 7fa443a55740 1 --2- 172.21.15.120:0/3698445429 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561afd521060 0x561afd529180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.169+0000 7fa443a55740 1 -- 172.21.15.120:0/3698445429 >> 172.21.15.120:0/3698445429 conn(0x561afd51d5f0 msgr2=0x561afd523320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:57.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.169+0000 7fa443a55740 1 -- 172.21.15.120:0/3698445429 shutdown_connections 2024-09-19T09:04:57.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.169+0000 7fa443a55740 1 -- 172.21.15.120:0/3698445429 wait complete. 2024-09-19T09:04:57.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.169+0000 7fa443a55740 20 client.16035 trim_cache size 0 max 16384 2024-09-19T09:04:57.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.185+0000 7fa443a55740 1 Processor -- start 2024-09-19T09:04:57.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.185+0000 7fa443a55740 1 -- start start 2024-09-19T09:04:57.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:56 smithi196 ceph-mon[35150]: pgmap v403: 97 pgs: 97 active+clean; 25 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 90 KiB/s wr, 51 op/s 2024-09-19T09:04:57.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.186+0000 7fa443a55740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561afd521060 0x561afd52b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:57.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.186+0000 7fa443a55740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561afd521980 0x561afd523ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:57.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.186+0000 7fa443a55740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561afd52c240 0x561afd543ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:57.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.186+0000 7fa443a55740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c1ad670 con 0x561afd521980 2024-09-19T09:04:57.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.186+0000 7fa443a55740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c1af1b0 con 0x561afd521060 2024-09-19T09:04:57.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.186+0000 7fa443a55740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c1add40 con 0x561afd52c240 2024-09-19T09:04:57.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.186+0000 7fa43affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561afd521980 0x561afd523ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:57.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.186+0000 7fa43affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561afd521980 0x561afd523ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35834/0 (socket says 172.21.15.120:35834) 2024-09-19T09:04:57.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.186+0000 7fa43affd640 1 -- 172.21.15.120:0/3722613340 learned_addr learned my addr 172.21.15.120:0/3722613340 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:57.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.186+0000 7fa43a7fc640 1 --2- 172.21.15.120:0/3722613340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561afd52c240 0x561afd543ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:57.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.186+0000 7fa43a7fc640 1 --2- 172.21.15.120:0/3722613340 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561afd521060 0x561afd52b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:57.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.186+0000 7fa43affd640 1 -- 172.21.15.120:0/3722613340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561afd52c240 msgr2=0x561afd543ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:57.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.186+0000 7fa43affd640 1 --2- 172.21.15.120:0/3722613340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561afd52c240 0x561afd543ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.186+0000 7fa43affd640 1 -- 172.21.15.120:0/3722613340 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561afd521060 msgr2=0x561afd52b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:57.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.186+0000 7fa43affd640 1 --2- 172.21.15.120:0/3722613340 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561afd521060 0x561afd52b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.186+0000 7fa43affd640 1 -- 172.21.15.120:0/3722613340 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561afd51e6a0 con 0x561afd521980 2024-09-19T09:04:57.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.186+0000 7fa43affd640 1 --2- 172.21.15.120:0/3722613340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561afd521980 0x561afd523ac0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa42c030810 tx=0x7fa42c02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770ce3f0025f75bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:57.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.186+0000 7fa430ff9640 1 -- 172.21.15.120:0/3722613340 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c052070 con 0x561afd521980 2024-09-19T09:04:57.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.186+0000 7fa430ff9640 1 -- 172.21.15.120:0/3722613340 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa42c036040 con 0x561afd521980 2024-09-19T09:04:57.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.186+0000 7fa430ff9640 1 -- 172.21.15.120:0/3722613340 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c047070 con 0x561afd521980 2024-09-19T09:04:57.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.187+0000 7fa443a55740 1 -- 172.21.15.120:0/3722613340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561afd521980 msgr2=0x561afd523ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:57.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.187+0000 7fa443a55740 1 --2- 172.21.15.120:0/3722613340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561afd521980 0x561afd523ac0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa42c030810 tx=0x7fa42c02a8e0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.187+0000 7fa443a55740 1 -- 172.21.15.120:0/3722613340 shutdown_connections 2024-09-19T09:04:57.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.187+0000 7fa443a55740 1 --2- 172.21.15.120:0/3722613340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561afd52c240 0x561afd543ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.187+0000 7fa443a55740 1 --2- 172.21.15.120:0/3722613340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561afd521980 0x561afd523ac0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.187+0000 7fa443a55740 1 --2- 172.21.15.120:0/3722613340 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561afd521060 0x561afd52b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.187+0000 7fa443a55740 1 -- 172.21.15.120:0/3722613340 >> 172.21.15.120:0/3722613340 conn(0x561afd51d5f0 msgr2=0x561afd51c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:57.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.187+0000 7fa443a55740 1 -- 172.21.15.120:0/3722613340 shutdown_connections 2024-09-19T09:04:57.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.187+0000 7fa443a55740 1 -- 172.21.15.120:0/3722613340 wait complete. 2024-09-19T09:04:57.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.188+0000 7fa443a55740 1 -- start start 2024-09-19T09:04:57.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.188+0000 7fa443a55740 1 Processor -- start 2024-09-19T09:04:57.190 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:56 smithi153 ceph-mon[35071]: pgmap v403: 97 pgs: 97 active+clean; 25 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 90 KiB/s wr, 51 op/s 2024-09-19T09:04:57.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.188+0000 7fa443a55740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561afd521060 0x561afd5343c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:57.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.188+0000 7fa443a55740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561afd521980 0x561afd534900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:57.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.188+0000 7fa443a55740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561afd52c240 0x561afd5311b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:57.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.188+0000 7fa443a55740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c1aeae0 con 0x561afd52c240 2024-09-19T09:04:57.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.188+0000 7fa443a55740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c1ae410 con 0x561afd521060 2024-09-19T09:04:57.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.188+0000 7fa443a55740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c1ab630 con 0x561afd521980 2024-09-19T09:04:57.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.188+0000 7fa443a55740 10 client.0 _ll_register_callbacks cb 0x7fa42c009980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:04:57.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.188+0000 7fa43a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561afd52c240 0x561afd5311b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:57.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.188+0000 7fa43a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561afd52c240 0x561afd5311b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35848/0 (socket says 172.21.15.120:35848) 2024-09-19T09:04:57.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.188+0000 7fa43a7fc640 1 -- 172.21.15.120:0/4240033109 learned_addr learned my addr 172.21.15.120:0/4240033109 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:04:57.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.188+0000 7fa43affd640 1 --2- 172.21.15.120:0/4240033109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561afd521980 0x561afd534900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:57.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.188+0000 7fa43a7fc640 1 --2- 172.21.15.120:0/4240033109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561afd521060 0x561afd5343c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:57.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.188+0000 7fa43a7fc640 1 -- 172.21.15.120:0/4240033109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561afd521980 msgr2=0x561afd534900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:57.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.188+0000 7fa43a7fc640 1 --2- 172.21.15.120:0/4240033109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561afd521980 0x561afd534900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.188+0000 7fa43a7fc640 1 -- 172.21.15.120:0/4240033109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561afd521060 msgr2=0x561afd5343c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:04:57.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.188+0000 7fa43a7fc640 1 --2- 172.21.15.120:0/4240033109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561afd521060 0x561afd5343c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.188+0000 7fa43a7fc640 1 -- 172.21.15.120:0/4240033109 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561afd531870 con 0x561afd52c240 2024-09-19T09:04:57.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.188+0000 7fa43affd640 1 --2- 172.21.15.120:0/4240033109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561afd521980 0x561afd534900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:04:57.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.189+0000 7fa43a7fc640 1 --2- 172.21.15.120:0/4240033109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561afd52c240 0x561afd5311b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa4240058c0 tx=0x7fa42401d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44748549e9e4708f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:04:57.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.189+0000 7fa4227fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:04:57.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.189+0000 7fa4227fc640 1 -- 172.21.15.120:0/4240033109 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa424016070 con 0x561afd52c240 2024-09-19T09:04:57.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.189+0000 7fa4227fc640 1 -- 172.21.15.120:0/4240033109 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa424004020 con 0x561afd52c240 2024-09-19T09:04:57.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.189+0000 7fa443a55740 1 -- 172.21.15.120:0/4240033109 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa42c036040 con 0x561afd52c240 2024-09-19T09:04:57.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.189+0000 7fa4227fc640 1 -- 172.21.15.120:0/4240033109 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa42403b050 con 0x561afd52c240 2024-09-19T09:04:57.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.189+0000 7fa443a55740 10 client.16041 fetch_fsmap learned FSMap version 28 2024-09-19T09:04:57.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.189+0000 7fa443a55740 1 -- 172.21.15.120:0/4240033109 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561afd5a74a0 con 0x561afd52c240 2024-09-19T09:04:57.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.189+0000 7fa4227fc640 1 -- 172.21.15.120:0/4240033109 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa42c036040 con 0x561afd52c240 2024-09-19T09:04:57.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.189+0000 7fa4227fc640 1 -- 172.21.15.120:0/4240033109 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa4240035f0 con 0x561afd52c240 2024-09-19T09:04:57.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.189+0000 7fa443a55740 10 client.16041 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:04:57.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.189+0000 7fa443a55740 10 client.16041 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:04:57.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.189+0000 7fa443a55740 20 client.16041 populate_metadata read hostname 'smithi120' 2024-09-19T09:04:57.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.189+0000 7fa443a55740 1 --2- 172.21.15.120:0/4240033109 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561afd5a7970 0x561afd5a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:04:57.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.189+0000 7fa443a55740 4 client.16041 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d6f3c5f1", "release": true, "await": true}] 2024-09-19T09:04:57.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.189+0000 7fa443a55740 1 -- 172.21.15.120:0/4240033109 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d6f3c5f1", "release": true, "await": true}) -- 0x561afd48e0a0 con 0x561afd5a7970 2024-09-19T09:04:57.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.190+0000 7fa43affd640 1 --2- 172.21.15.120:0/4240033109 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561afd5a7970 0x561afd5a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:04:57.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.190+0000 7fa43affd640 1 --2- 172.21.15.120:0/4240033109 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561afd5a7970 0x561afd5a9d90 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6226b5ac2e6265 server_cookie=1bc704c3def80acd in_seq=0 out_seq=0 2024-09-19T09:04:57.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.190+0000 7fa4227fc640 10 client.16041 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:57.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.203+0000 7fa4227fc640 1 -- 172.21.15.120:0/4240033109 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561afd48e0a0 con 0x561afd5a7970 2024-09-19T09:04:57.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.203+0000 7fa4227fc640 10 client.16041 handle_command_reply: tid=0 2024-09-19T09:04:57.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.203+0000 7fa443a55740 1 client.16041 shutdown 2024-09-19T09:04:57.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.203+0000 7fa443a55740 1 -- 172.21.15.120:0/4240033109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561afd52c240 msgr2=0x561afd5311b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:57.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.203+0000 7fa443a55740 1 --2- 172.21.15.120:0/4240033109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561afd52c240 0x561afd5311b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa4240058c0 tx=0x7fa42401d040 comp rx=0 tx=0).stop 2024-09-19T09:04:57.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.203+0000 7fa443a55740 1 -- 172.21.15.120:0/4240033109 shutdown_connections 2024-09-19T09:04:57.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.203+0000 7fa443a55740 1 --2- 172.21.15.120:0/4240033109 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561afd5a7970 0x561afd5a9d90 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.203+0000 7fa443a55740 1 --2- 172.21.15.120:0/4240033109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561afd52c240 0x561afd5311b0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.203+0000 7fa443a55740 1 --2- 172.21.15.120:0/4240033109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561afd521980 0x561afd534900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.203+0000 7fa4227fc640 0 client.16041 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:04:57.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.203+0000 7fa443a55740 1 --2- 172.21.15.120:0/4240033109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561afd521060 0x561afd5343c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.203+0000 7fa443a55740 1 -- 172.21.15.120:0/4240033109 >> 172.21.15.120:0/4240033109 conn(0x561afd51d5f0 msgr2=0x561afd51bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:57.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.203+0000 7fa443a55740 1 -- 172.21.15.120:0/4240033109 shutdown_connections 2024-09-19T09:04:57.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.203+0000 7fa443a55740 1 -- 172.21.15.120:0/4240033109 wait complete. 2024-09-19T09:04:57.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.203+0000 7fa443a55740 20 client.16041 trim_cache size 0 max 16384 2024-09-19T09:04:57.206 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:04:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 28, 2024-09-19T09:04:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:04:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 12, 2024-09-19T09:04:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:04:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: "d6f3c5f1": { 2024-09-19T09:04:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 12, 2024-09-19T09:04:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:04:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:04:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:04:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:04:57.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:04:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:04:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 140.3, 2024-09-19T09:04:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:04:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:04:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:04:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:04:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:04:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:04:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:04:57.207 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:04:57.208 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:04:57.208 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:04:57.208 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:04:57.208 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:04:57.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.204+0000 7fa44167c640 1 -- 172.21.15.120:0/2151403798 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa4280a3240 msgr2=0x7fa4280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:57.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.204+0000 7fa44167c640 1 --2- 172.21.15.120:0/2151403798 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa4280a3240 0x7fa4280a5700 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa434097af0 tx=0x7fa43406d460 comp rx=0 tx=0).stop 2024-09-19T09:04:57.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.204+0000 7fa44167c640 1 -- 172.21.15.120:0/2151403798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa43c106a80 msgr2=0x7fa43c1a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:04:57.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.204+0000 7fa44167c640 1 --2- 172.21.15.120:0/2151403798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa43c106a80 0x7fa43c1a4410 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa434009fd0 tx=0x7fa43400b8e0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.204+0000 7fa44167c640 1 -- 172.21.15.120:0/2151403798 shutdown_connections 2024-09-19T09:04:57.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.204+0000 7fa44167c640 1 --2- 172.21.15.120:0/2151403798 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa4280a3240 0x7fa4280a5700 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.204+0000 7fa44167c640 1 --2- 172.21.15.120:0/2151403798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa43c106a80 0x7fa43c1a4410 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.204+0000 7fa44167c640 1 --2- 172.21.15.120:0/2151403798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa43c101420 0x7fa43c1998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.204+0000 7fa44167c640 1 --2- 172.21.15.120:0/2151403798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa43c100a80 0x7fa43c199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:04:57.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.204+0000 7fa44167c640 1 -- 172.21.15.120:0/2151403798 >> 172.21.15.120:0/2151403798 conn(0x7fa43c0fc7a0 msgr2=0x7fa43c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:04:57.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.204+0000 7fa44167c640 1 -- 172.21.15.120:0/2151403798 shutdown_connections 2024-09-19T09:04:57.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:04:57.204+0000 7fa44167c640 1 -- 172.21.15.120:0/2151403798 wait complete. 2024-09-19T09:04:58.037 INFO:tasks.workunit.client.0.smithi120.stdout: 1 32035 3.62 MB/sec warmup 53 sec latency 18563.978 ms 2024-09-19T09:04:58.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:57 smithi120 ceph-mon[31150]: pgmap v404: 97 pgs: 97 active+clean; 29 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 479 KiB/s wr, 59 op/s 2024-09-19T09:04:58.118 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd6f3c5f1', total seconds elapsed: 24.0 2024-09-19T09:04:58.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:57 smithi196 ceph-mon[35150]: pgmap v404: 97 pgs: 97 active+clean; 29 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 479 KiB/s wr, 59 op/s 2024-09-19T09:04:58.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:57 smithi153 ceph-mon[35071]: pgmap v404: 97 pgs: 97 active+clean; 29 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 479 KiB/s wr, 59 op/s 2024-09-19T09:04:59.037 INFO:tasks.workunit.client.0.smithi120.stdout: 1 35139 3.82 MB/sec warmup 54 sec latency 19.185 ms 2024-09-19T09:04:59.574 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:04:59 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:04:59.685 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:04:59 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:04:59.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:04:59 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:05:00.037 INFO:tasks.workunit.client.0.smithi120.stdout: 1 38266 4.06 MB/sec warmup 55 sec latency 11.765 ms 2024-09-19T09:05:00.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:05:00.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:05:00.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:05:00.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:05:00.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:00 smithi120 ceph-mon[31150]: pgmap v405: 97 pgs: 97 active+clean; 36 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.3 MiB/s wr, 75 op/s 2024-09-19T09:05:00.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:05:00.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:05:00.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:05:00.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:05:00.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:05:00.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:05:00.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:05:00.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:00 smithi196 ceph-mon[35150]: pgmap v405: 97 pgs: 97 active+clean; 36 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.3 MiB/s wr, 75 op/s 2024-09-19T09:05:00.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:05:00.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:05:00.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:05:00.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:05:00.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:05:00.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:05:00.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:05:00.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:00 smithi153 ceph-mon[35071]: pgmap v405: 97 pgs: 97 active+clean; 36 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.3 MiB/s wr, 75 op/s 2024-09-19T09:05:00.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:05:00.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:05:00.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:05:01.037 INFO:tasks.workunit.client.0.smithi120.stdout: 1 41526 4.27 MB/sec warmup 56 sec latency 11.702 ms 2024-09-19T09:05:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.199+0000 7fee638e6640 1 -- 172.21.15.120:0/2593360799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee5c107850 msgr2=0x7fee5c109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.199+0000 7fee638e6640 1 --2- 172.21.15.120:0/2593360799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee5c107850 0x7fee5c109cc0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fee54009ca0 tx=0x7fee5402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.199+0000 7fee638e6640 1 -- 172.21.15.120:0/2593360799 shutdown_connections 2024-09-19T09:05:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.199+0000 7fee638e6640 1 --2- 172.21.15.120:0/2593360799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee5c107850 0x7fee5c109cc0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.199+0000 7fee638e6640 1 --2- 172.21.15.120:0/2593360799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee5c104f20 0x7fee5c107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.199+0000 7fee638e6640 1 --2- 172.21.15.120:0/2593360799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee5c1025f0 0x7fee5c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.199+0000 7fee638e6640 1 -- 172.21.15.120:0/2593360799 >> 172.21.15.120:0/2593360799 conn(0x7fee5c0fc1b0 msgr2=0x7fee5c0fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.199+0000 7fee638e6640 1 -- 172.21.15.120:0/2593360799 shutdown_connections 2024-09-19T09:05:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.199+0000 7fee638e6640 1 -- 172.21.15.120:0/2593360799 wait complete. 2024-09-19T09:05:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.200+0000 7fee638e6640 1 Processor -- start 2024-09-19T09:05:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.200+0000 7fee638e6640 1 -- start start 2024-09-19T09:05:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.200+0000 7fee638e6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee5c1025f0 0x7fee5c101f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.200+0000 7fee638e6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee5c104f20 0x7fee5c1005a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.200+0000 7fee638e6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee5c107850 0x7fee5c100ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.200+0000 7fee638e6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c10d810 con 0x7fee5c1025f0 2024-09-19T09:05:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.200+0000 7fee638e6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c10d690 con 0x7fee5c107850 2024-09-19T09:05:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.200+0000 7fee638e6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c10d990 con 0x7fee5c104f20 2024-09-19T09:05:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.200+0000 7fee61682640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee5c1025f0 0x7fee5c101f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.200+0000 7fee61682640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee5c1025f0 0x7fee5c101f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35880/0 (socket says 172.21.15.120:35880) 2024-09-19T09:05:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.200+0000 7fee61682640 1 -- 172.21.15.120:0/1112158793 learned_addr learned my addr 172.21.15.120:0/1112158793 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.200+0000 7fee60e81640 1 --2- 172.21.15.120:0/1112158793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee5c104f20 0x7fee5c1005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.200+0000 7fee61e83640 1 --2- 172.21.15.120:0/1112158793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee5c107850 0x7fee5c100ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.200+0000 7fee61682640 1 -- 172.21.15.120:0/1112158793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee5c104f20 msgr2=0x7fee5c1005a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.200+0000 7fee61682640 1 --2- 172.21.15.120:0/1112158793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee5c104f20 0x7fee5c1005a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.200+0000 7fee61682640 1 -- 172.21.15.120:0/1112158793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee5c107850 msgr2=0x7fee5c100ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.200+0000 7fee61682640 1 --2- 172.21.15.120:0/1112158793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee5c107850 0x7fee5c100ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.200+0000 7fee61682640 1 -- 172.21.15.120:0/1112158793 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5c101370 con 0x7fee5c1025f0 2024-09-19T09:05:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.201+0000 7fee61682640 1 --2- 172.21.15.120:0/1112158793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee5c1025f0 0x7fee5c101f60 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fee5800e9f0 tx=0x7fee5800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.201+0000 7fee527fc640 1 -- 172.21.15.120:0/1112158793 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5800cd80 con 0x7fee5c1025f0 2024-09-19T09:05:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.201+0000 7fee527fc640 1 -- 172.21.15.120:0/1112158793 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee58010910 con 0x7fee5c1025f0 2024-09-19T09:05:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.201+0000 7fee527fc640 1 -- 172.21.15.120:0/1112158793 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee58018990 con 0x7fee5c1025f0 2024-09-19T09:05:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.201+0000 7fee638e6640 1 -- 172.21.15.120:0/1112158793 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee5c1a1f00 con 0x7fee5c1025f0 2024-09-19T09:05:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.201+0000 7fee638e6640 1 -- 172.21.15.120:0/1112158793 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee5c1a2310 con 0x7fee5c1025f0 2024-09-19T09:05:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.204+0000 7fee527fc640 1 -- 172.21.15.120:0/1112158793 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fee58010ac0 con 0x7fee5c1025f0 2024-09-19T09:05:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.204+0000 7fee66680740 1 Processor -- start 2024-09-19T09:05:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.204+0000 7fee66680740 1 -- start start 2024-09-19T09:05:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.204+0000 7fee66680740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172011e9a0 0x56172011eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.204+0000 7fee66680740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172011f2e0 0x561720123da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.204+0000 7fee66680740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561720124370 0x561720126830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.204+0000 7fee66680740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5617200822f0 con 0x56172011f2e0 2024-09-19T09:05:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.204+0000 7fee66680740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56172002a270 con 0x56172011e9a0 2024-09-19T09:05:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.204+0000 7fee66680740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56172007e2b0 con 0x561720124370 2024-09-19T09:05:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.204+0000 7fee61682640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172011f2e0 0x561720123da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.204+0000 7fee61e83640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172011e9a0 0x56172011eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.204+0000 7fee61e83640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172011e9a0 0x56172011eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:60536/0 (socket says 172.21.15.120:60536) 2024-09-19T09:05:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.204+0000 7fee60e81640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561720124370 0x561720126830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.204+0000 7fee61682640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172011f2e0 0x561720123da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35896/0 (socket says 172.21.15.120:35896) 2024-09-19T09:05:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.204+0000 7fee61e83640 1 -- 172.21.15.120:0/3425498837 learned_addr learned my addr 172.21.15.120:0/3425498837 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.204+0000 7fee61e83640 1 -- 172.21.15.120:0/3425498837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561720124370 msgr2=0x561720126830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.204+0000 7fee61e83640 1 --2- 172.21.15.120:0/3425498837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561720124370 0x561720126830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.204+0000 7fee61e83640 1 -- 172.21.15.120:0/3425498837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172011f2e0 msgr2=0x561720123da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.204+0000 7fee61e83640 1 --2- 172.21.15.120:0/3425498837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172011f2e0 0x561720123da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.204+0000 7fee527fc640 1 --2- 172.21.15.120:0/1112158793 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fee480a32f0 0x7fee480a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.204+0000 7fee61e83640 1 -- 172.21.15.120:0/3425498837 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561720029340 con 0x56172011e9a0 2024-09-19T09:05:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.205+0000 7fee60e81640 1 --2- 172.21.15.120:0/1112158793 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fee480a32f0 0x7fee480a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.205+0000 7fee527fc640 1 -- 172.21.15.120:0/1112158793 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fee580cfba0 con 0x7fee5c1025f0 2024-09-19T09:05:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.205+0000 7fee61e83640 1 --2- 172.21.15.120:0/3425498837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172011e9a0 0x56172011eda0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fee5c06b380 tx=0x7fee54004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40928bc027e0ee55 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.205+0000 7fee477fe640 1 -- 172.21.15.120:0/3425498837 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee540098c0 con 0x56172011e9a0 2024-09-19T09:05:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.205+0000 7fee477fe640 1 -- 172.21.15.120:0/3425498837 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee54036040 con 0x56172011e9a0 2024-09-19T09:05:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.205+0000 7fee477fe640 1 -- 172.21.15.120:0/3425498837 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee54004290 con 0x56172011e9a0 2024-09-19T09:05:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.206+0000 7fee66680740 1 -- 172.21.15.120:0/3425498837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172011e9a0 msgr2=0x56172011eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.206+0000 7fee60e81640 1 --2- 172.21.15.120:0/1112158793 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fee480a32f0 0x7fee480a57b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fee5c101cd0 tx=0x7fee4c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.206+0000 7fee66680740 1 --2- 172.21.15.120:0/3425498837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172011e9a0 0x56172011eda0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fee5c06b380 tx=0x7fee54004a40 comp rx=0 tx=0).stop 2024-09-19T09:05:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.206+0000 7fee66680740 1 -- 172.21.15.120:0/3425498837 shutdown_connections 2024-09-19T09:05:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.206+0000 7fee66680740 1 --2- 172.21.15.120:0/3425498837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561720124370 0x561720126830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.206+0000 7fee66680740 1 --2- 172.21.15.120:0/3425498837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172011f2e0 0x561720123da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.206+0000 7fee66680740 1 --2- 172.21.15.120:0/3425498837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172011e9a0 0x56172011eda0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.206+0000 7fee66680740 1 -- 172.21.15.120:0/3425498837 >> 172.21.15.120:0/3425498837 conn(0x56172011a010 msgr2=0x56172011a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.206+0000 7fee66680740 1 -- 172.21.15.120:0/3425498837 shutdown_connections 2024-09-19T09:05:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.206+0000 7fee66680740 1 -- 172.21.15.120:0/3425498837 wait complete. 2024-09-19T09:05:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.206+0000 7fee66680740 1 -- start start 2024-09-19T09:05:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.206+0000 7fee66680740 1 Processor -- start 2024-09-19T09:05:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.207+0000 7fee66680740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172011e9a0 0x561720128920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.207+0000 7fee66680740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172011f2e0 0x561720128e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.207+0000 7fee66680740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561720124370 0x56172012fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.207+0000 7fee66680740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561720127100 con 0x561720124370 2024-09-19T09:05:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.207+0000 7fee66680740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56172000b290 con 0x56172011e9a0 2024-09-19T09:05:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.207+0000 7fee66680740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561720127280 con 0x56172011f2e0 2024-09-19T09:05:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.207+0000 7fee66680740 10 client.0 _ll_register_callbacks cb 0x56172003c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.207+0000 7fee60e81640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172011f2e0 0x561720128e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.207+0000 7fee61682640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172011e9a0 0x561720128920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.207+0000 7fee61e83640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561720124370 0x56172012fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.207+0000 7fee60e81640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172011f2e0 0x561720128e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:52880/0 (socket says 172.21.15.120:52880) 2024-09-19T09:05:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.207+0000 7fee60e81640 1 -- 172.21.15.120:0/2883711363 learned_addr learned my addr 172.21.15.120:0/2883711363 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.207+0000 7fee61e83640 1 -- 172.21.15.120:0/2883711363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172011f2e0 msgr2=0x561720128e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.207+0000 7fee61e83640 1 --2- 172.21.15.120:0/2883711363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172011f2e0 0x561720128e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.207+0000 7fee61e83640 1 -- 172.21.15.120:0/2883711363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172011e9a0 msgr2=0x561720128920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.207+0000 7fee61e83640 1 --2- 172.21.15.120:0/2883711363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172011e9a0 0x561720128920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.207+0000 7fee61e83640 1 -- 172.21.15.120:0/2883711363 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56171fd95210 con 0x561720124370 2024-09-19T09:05:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.207+0000 7fee61682640 1 --2- 172.21.15.120:0/2883711363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172011e9a0 0x561720128920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.207+0000 7fee60e81640 1 --2- 172.21.15.120:0/2883711363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172011f2e0 0x561720128e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.207+0000 7fee61e83640 1 --2- 172.21.15.120:0/2883711363 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561720124370 0x56172012fea0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fee54009ca0 tx=0x7fee5402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b07d984b4ab50d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.207+0000 7fee44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.207+0000 7fee44ff9640 1 -- 172.21.15.120:0/2883711363 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5402f610 con 0x561720124370 2024-09-19T09:05:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.207+0000 7fee44ff9640 1 -- 172.21.15.120:0/2883711363 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fee5402f7b0 con 0x561720124370 2024-09-19T09:05:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.208+0000 7fee66680740 1 -- 172.21.15.120:0/2883711363 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee54036040 con 0x561720124370 2024-09-19T09:05:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.208+0000 7fee44ff9640 1 -- 172.21.15.120:0/2883711363 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fee54004b40 con 0x561720124370 2024-09-19T09:05:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.208+0000 7fee66680740 10 client.16053 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.208+0000 7fee66680740 1 -- 172.21.15.120:0/2883711363 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56171fe61620 con 0x561720124370 2024-09-19T09:05:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.208+0000 7fee44ff9640 1 -- 172.21.15.120:0/2883711363 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee54036040 con 0x561720124370 2024-09-19T09:05:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.208+0000 7fee44ff9640 1 -- 172.21.15.120:0/2883711363 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fee5404d070 con 0x561720124370 2024-09-19T09:05:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.208+0000 7fee66680740 10 client.16053 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.208+0000 7fee66680740 10 client.16053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.208+0000 7fee66680740 20 client.16053 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.208+0000 7fee66680740 1 --2- 172.21.15.120:0/2883711363 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5617201342b0 0x5617201366a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.208+0000 7fee66680740 4 client.16053 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:05:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.208+0000 7fee66680740 1 -- 172.21.15.120:0/2883711363 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56172008fa20 con 0x5617201342b0 2024-09-19T09:05:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.208+0000 7fee61682640 1 --2- 172.21.15.120:0/2883711363 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5617201342b0 0x5617201366a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.209+0000 7fee44ff9640 10 client.16053 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.209+0000 7fee61682640 1 --2- 172.21.15.120:0/2883711363 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5617201342b0 0x5617201366a0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=668b4aa06b756553 server_cookie=305bb02de5a92c33 in_seq=0 out_seq=0 2024-09-19T09:05:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.210+0000 7fee44ff9640 1 -- 172.21.15.120:0/2883711363 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56172008fa20 con 0x5617201342b0 2024-09-19T09:05:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.210+0000 7fee44ff9640 10 client.16053 handle_command_reply: tid=0 2024-09-19T09:05:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.210+0000 7fee66680740 1 client.16053 shutdown 2024-09-19T09:05:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.210+0000 7fee66680740 1 -- 172.21.15.120:0/2883711363 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561720124370 msgr2=0x56172012fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.210+0000 7fee66680740 1 --2- 172.21.15.120:0/2883711363 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561720124370 0x56172012fea0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fee54009ca0 tx=0x7fee5402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.210+0000 7fee66680740 1 -- 172.21.15.120:0/2883711363 shutdown_connections 2024-09-19T09:05:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.210+0000 7fee66680740 1 --2- 172.21.15.120:0/2883711363 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5617201342b0 0x5617201366a0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.210+0000 7fee66680740 1 --2- 172.21.15.120:0/2883711363 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561720124370 0x56172012fea0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.210+0000 7fee66680740 1 --2- 172.21.15.120:0/2883711363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172011f2e0 0x561720128e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.210+0000 7fee66680740 1 --2- 172.21.15.120:0/2883711363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172011e9a0 0x561720128920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.210+0000 7fee44ff9640 0 client.16053 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.210+0000 7fee66680740 1 -- 172.21.15.120:0/2883711363 >> 172.21.15.120:0/2883711363 conn(0x56172011a010 msgr2=0x56172011b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.210+0000 7fee66680740 1 -- 172.21.15.120:0/2883711363 shutdown_connections 2024-09-19T09:05:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.210+0000 7fee66680740 1 -- 172.21.15.120:0/2883711363 wait complete. 2024-09-19T09:05:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.210+0000 7fee66680740 20 client.16053 trim_cache size 0 max 16384 2024-09-19T09:05:01.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.225+0000 7fee66680740 1 Processor -- start 2024-09-19T09:05:01.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.225+0000 7fee66680740 1 -- start start 2024-09-19T09:05:01.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.225+0000 7fee66680740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172011e9a0 0x56172011ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.225+0000 7fee66680740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172011f2e0 0x56172012fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.225+0000 7fee66680740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561720124370 0x56172011ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.225+0000 7fee66680740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c1ab1b0 con 0x56172011e9a0 2024-09-19T09:05:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.225+0000 7fee66680740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c1accf0 con 0x561720124370 2024-09-19T09:05:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.225+0000 7fee66680740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c1ab880 con 0x56172011f2e0 2024-09-19T09:05:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.225+0000 7fee61682640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172011e9a0 0x56172011ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.225+0000 7fee60e81640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172011f2e0 0x56172012fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.225+0000 7fee61682640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172011e9a0 0x56172011ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36804/0 (socket says 172.21.15.120:36804) 2024-09-19T09:05:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.225+0000 7fee61682640 1 -- 172.21.15.120:0/3199384604 learned_addr learned my addr 172.21.15.120:0/3199384604 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.226+0000 7fee61e83640 1 --2- 172.21.15.120:0/3199384604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561720124370 0x56172011ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.226+0000 7fee61682640 1 -- 172.21.15.120:0/3199384604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172011f2e0 msgr2=0x56172012fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.226+0000 7fee61682640 1 --2- 172.21.15.120:0/3199384604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172011f2e0 0x56172012fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.226+0000 7fee61682640 1 -- 172.21.15.120:0/3199384604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561720124370 msgr2=0x56172011ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.226+0000 7fee61682640 1 --2- 172.21.15.120:0/3199384604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561720124370 0x56172011ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.226+0000 7fee61682640 1 -- 172.21.15.120:0/3199384604 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56172011ff70 con 0x56172011e9a0 2024-09-19T09:05:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.226+0000 7fee61682640 1 --2- 172.21.15.120:0/3199384604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172011e9a0 0x56172011ed80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fee58000c00 tx=0x7fee580cd570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a54e4534d6179c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.226+0000 7fee46ffd640 1 -- 172.21.15.120:0/3199384604 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5809bbd0 con 0x56172011e9a0 2024-09-19T09:05:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.226+0000 7fee46ffd640 1 -- 172.21.15.120:0/3199384604 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee5809c270 con 0x56172011e9a0 2024-09-19T09:05:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.226+0000 7fee46ffd640 1 -- 172.21.15.120:0/3199384604 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee580bc0b0 con 0x56172011e9a0 2024-09-19T09:05:01.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.227+0000 7fee66680740 1 -- 172.21.15.120:0/3199384604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172011e9a0 msgr2=0x56172011ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:01.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.227+0000 7fee66680740 1 --2- 172.21.15.120:0/3199384604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172011e9a0 0x56172011ed80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fee58000c00 tx=0x7fee580cd570 comp rx=0 tx=0).stop 2024-09-19T09:05:01.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.227+0000 7fee66680740 1 -- 172.21.15.120:0/3199384604 shutdown_connections 2024-09-19T09:05:01.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.227+0000 7fee66680740 1 --2- 172.21.15.120:0/3199384604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561720124370 0x56172011ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.227+0000 7fee66680740 1 --2- 172.21.15.120:0/3199384604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172011f2e0 0x56172012fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.227+0000 7fee66680740 1 --2- 172.21.15.120:0/3199384604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172011e9a0 0x56172011ed80 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.227+0000 7fee66680740 1 -- 172.21.15.120:0/3199384604 >> 172.21.15.120:0/3199384604 conn(0x56172011a010 msgr2=0x561720128920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.227+0000 7fee66680740 1 -- 172.21.15.120:0/3199384604 shutdown_connections 2024-09-19T09:05:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.227+0000 7fee66680740 1 -- 172.21.15.120:0/3199384604 wait complete. 2024-09-19T09:05:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.227+0000 7fee66680740 1 -- start start 2024-09-19T09:05:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.227+0000 7fee66680740 1 Processor -- start 2024-09-19T09:05:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.228+0000 7fee66680740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172011e9a0 0x56172012cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.228+0000 7fee66680740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172011f2e0 0x56172012d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.228+0000 7fee66680740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561720124370 0x56172019cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.228+0000 7fee66680740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c1ac620 con 0x56172011e9a0 2024-09-19T09:05:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.228+0000 7fee66680740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c1abf50 con 0x561720124370 2024-09-19T09:05:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.228+0000 7fee66680740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c1a9170 con 0x56172011f2e0 2024-09-19T09:05:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.228+0000 7fee66680740 10 client.0 _ll_register_callbacks cb 0x7fee580d34f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.228+0000 7fee61682640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172011e9a0 0x56172012cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.228+0000 7fee61682640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172011e9a0 0x56172012cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36812/0 (socket says 172.21.15.120:36812) 2024-09-19T09:05:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.228+0000 7fee61682640 1 -- 172.21.15.120:0/1775051001 learned_addr learned my addr 172.21.15.120:0/1775051001 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.228+0000 7fee61e83640 1 --2- 172.21.15.120:0/1775051001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561720124370 0x56172019cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.228+0000 7fee60e81640 1 --2- 172.21.15.120:0/1775051001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172011f2e0 0x56172012d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.228+0000 7fee61682640 1 -- 172.21.15.120:0/1775051001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172011f2e0 msgr2=0x56172012d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.228+0000 7fee61682640 1 --2- 172.21.15.120:0/1775051001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172011f2e0 0x56172012d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.228+0000 7fee61682640 1 -- 172.21.15.120:0/1775051001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561720124370 msgr2=0x56172019cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.228+0000 7fee61682640 1 --2- 172.21.15.120:0/1775051001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561720124370 0x56172019cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.228+0000 7fee61682640 1 -- 172.21.15.120:0/1775051001 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56172019d490 con 0x56172011e9a0 2024-09-19T09:05:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.228+0000 7fee61682640 1 --2- 172.21.15.120:0/1775051001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172011e9a0 0x56172012cdf0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fee580a4900 tx=0x7fee58005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418308a27d4cf045 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.228+0000 7fee44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.228+0000 7fee44ff9640 1 -- 172.21.15.120:0/1775051001 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee580a4e30 con 0x56172011e9a0 2024-09-19T09:05:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.228+0000 7fee44ff9640 1 -- 172.21.15.120:0/1775051001 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fee580cd720 con 0x56172011e9a0 2024-09-19T09:05:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.228+0000 7fee66680740 1 -- 172.21.15.120:0/1775051001 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee5809c270 con 0x56172011e9a0 2024-09-19T09:05:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.229+0000 7fee44ff9640 1 -- 172.21.15.120:0/1775051001 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fee580a0570 con 0x56172011e9a0 2024-09-19T09:05:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.229+0000 7fee66680740 10 client.16059 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.229+0000 7fee66680740 1 -- 172.21.15.120:0/1775051001 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56172019e790 con 0x56172011e9a0 2024-09-19T09:05:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.229+0000 7fee44ff9640 1 -- 172.21.15.120:0/1775051001 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee5809c270 con 0x56172011e9a0 2024-09-19T09:05:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.229+0000 7fee44ff9640 1 -- 172.21.15.120:0/1775051001 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fee580b0220 con 0x56172011e9a0 2024-09-19T09:05:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.229+0000 7fee66680740 10 client.16059 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.229+0000 7fee66680740 10 client.16059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.229+0000 7fee66680740 20 client.16059 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.229+0000 7fee66680740 1 --2- 172.21.15.120:0/1775051001 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56172019f200 0x5617201a1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.229+0000 7fee66680740 4 client.16059 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:05:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.229+0000 7fee66680740 1 -- 172.21.15.120:0/1775051001 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561720089d70 con 0x56172019f200 2024-09-19T09:05:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.229+0000 7fee60e81640 1 --2- 172.21.15.120:0/1775051001 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56172019f200 0x5617201a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.230+0000 7fee44ff9640 10 client.16059 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.230+0000 7fee60e81640 1 --2- 172.21.15.120:0/1775051001 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56172019f200 0x5617201a1620 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea8d26c607a849f6 server_cookie=e3b145cd87fca22 in_seq=0 out_seq=0 2024-09-19T09:05:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.284+0000 7fee44ff9640 1 -- 172.21.15.120:0/1775051001 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561720089d70 con 0x56172019f200 2024-09-19T09:05:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.284+0000 7fee44ff9640 10 client.16059 handle_command_reply: tid=0 2024-09-19T09:05:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.284+0000 7fee66680740 1 client.16059 shutdown 2024-09-19T09:05:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.285+0000 7fee66680740 1 -- 172.21.15.120:0/1775051001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172011e9a0 msgr2=0x56172012cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.285+0000 7fee66680740 1 --2- 172.21.15.120:0/1775051001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172011e9a0 0x56172012cdf0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fee580a4900 tx=0x7fee58005c70 comp rx=0 tx=0).stop 2024-09-19T09:05:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.285+0000 7fee66680740 1 -- 172.21.15.120:0/1775051001 shutdown_connections 2024-09-19T09:05:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.285+0000 7fee66680740 1 --2- 172.21.15.120:0/1775051001 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56172019f200 0x5617201a1620 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.285+0000 7fee44ff9640 0 client.16059 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.285+0000 7fee66680740 1 --2- 172.21.15.120:0/1775051001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561720124370 0x56172019cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.285+0000 7fee66680740 1 --2- 172.21.15.120:0/1775051001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172011f2e0 0x56172012d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.285+0000 7fee66680740 1 --2- 172.21.15.120:0/1775051001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172011e9a0 0x56172012cdf0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.285+0000 7fee66680740 1 -- 172.21.15.120:0/1775051001 >> 172.21.15.120:0/1775051001 conn(0x56172011a010 msgr2=0x5617201189a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.285+0000 7fee66680740 1 -- 172.21.15.120:0/1775051001 shutdown_connections 2024-09-19T09:05:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.285+0000 7fee66680740 1 -- 172.21.15.120:0/1775051001 wait complete. 2024-09-19T09:05:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.285+0000 7fee66680740 20 client.16059 trim_cache size 0 max 16384 2024-09-19T09:05:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:05:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:05:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:05:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:05:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.285+0000 7fee638e6640 1 -- 172.21.15.120:0/1112158793 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fee480a32f0 msgr2=0x7fee480a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.285+0000 7fee638e6640 1 --2- 172.21.15.120:0/1112158793 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fee480a32f0 0x7fee480a57b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fee5c101cd0 tx=0x7fee4c005f90 comp rx=0 tx=0).stop 2024-09-19T09:05:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.285+0000 7fee638e6640 1 -- 172.21.15.120:0/1112158793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee5c1025f0 msgr2=0x7fee5c101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.285+0000 7fee638e6640 1 --2- 172.21.15.120:0/1112158793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee5c1025f0 0x7fee5c101f60 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fee5800e9f0 tx=0x7fee5800eeb0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.285+0000 7fee638e6640 1 -- 172.21.15.120:0/1112158793 shutdown_connections 2024-09-19T09:05:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.285+0000 7fee638e6640 1 --2- 172.21.15.120:0/1112158793 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fee480a32f0 0x7fee480a57b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.285+0000 7fee638e6640 1 --2- 172.21.15.120:0/1112158793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee5c107850 0x7fee5c100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.285+0000 7fee638e6640 1 --2- 172.21.15.120:0/1112158793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee5c104f20 0x7fee5c1005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.285+0000 7fee638e6640 1 --2- 172.21.15.120:0/1112158793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee5c1025f0 0x7fee5c101f60 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.286+0000 7fee638e6640 1 -- 172.21.15.120:0/1112158793 >> 172.21.15.120:0/1112158793 conn(0x7fee5c0fc1b0 msgr2=0x7fee5c102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:01.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.286+0000 7fee638e6640 1 -- 172.21.15.120:0/1112158793 shutdown_connections 2024-09-19T09:05:01.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:01.286+0000 7fee638e6640 1 -- 172.21.15.120:0/1112158793 wait complete. 2024-09-19T09:05:02.037 INFO:tasks.workunit.client.0.smithi120.stdout: 1 44361 4.49 MB/sec warmup 57 sec latency 14.732 ms 2024-09-19T09:05:03.037 INFO:tasks.workunit.client.0.smithi120.stdout: 1 47549 4.73 MB/sec warmup 58 sec latency 12.852 ms 2024-09-19T09:05:03.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:02 smithi120 ceph-mon[31150]: pgmap v406: 97 pgs: 97 active+clean; 38 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 1.8 MiB/s wr, 158 op/s 2024-09-19T09:05:03.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:02 smithi196 ceph-mon[35150]: pgmap v406: 97 pgs: 97 active+clean; 38 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 1.8 MiB/s wr, 158 op/s 2024-09-19T09:05:03.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:02 smithi153 ceph-mon[35071]: pgmap v406: 97 pgs: 97 active+clean; 38 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 1.8 MiB/s wr, 158 op/s 2024-09-19T09:05:04.037 INFO:tasks.workunit.client.0.smithi120.stdout: 1 50575 4.90 MB/sec warmup 59 sec latency 12.017 ms 2024-09-19T09:05:05.037 INFO:tasks.workunit.client.0.smithi120.stdout: 1 53960 5.13 MB/sec warmup 60 sec latency 12.400 ms 2024-09-19T09:05:05.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:04 smithi120 ceph-mon[31150]: pgmap v407: 97 pgs: 97 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 191 op/s 2024-09-19T09:05:05.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:04 smithi196 ceph-mon[35150]: pgmap v407: 97 pgs: 97 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 191 op/s 2024-09-19T09:05:05.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:04 smithi153 ceph-mon[35071]: pgmap v407: 97 pgs: 97 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 191 op/s 2024-09-19T09:05:06.037 INFO:tasks.workunit.client.0.smithi120.stdout: 1 57329 5.36 MB/sec warmup 61 sec latency 11.201 ms 2024-09-19T09:05:07.037 INFO:tasks.workunit.client.0.smithi120.stdout: 1 60750 5.58 MB/sec warmup 62 sec latency 15.128 ms 2024-09-19T09:05:07.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:06 smithi120 ceph-mon[31150]: pgmap v408: 97 pgs: 97 active+clean; 26 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 190 op/s 2024-09-19T09:05:07.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:06 smithi196 ceph-mon[35150]: pgmap v408: 97 pgs: 97 active+clean; 26 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 190 op/s 2024-09-19T09:05:07.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:06 smithi153 ceph-mon[35071]: pgmap v408: 97 pgs: 97 active+clean; 26 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 190 op/s 2024-09-19T09:05:08.037 INFO:tasks.workunit.client.0.smithi120.stdout: 1 64237 5.80 MB/sec warmup 63 sec latency 20.354 ms 2024-09-19T09:05:08.049 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:05:08.314 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:05:08.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.845+0000 7f91ad57c640 1 -- 172.21.15.120:0/406355732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91a8101270 msgr2=0x7f91a810d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:08.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.845+0000 7f91ad57c640 1 --2- 172.21.15.120:0/406355732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91a8101270 0x7f91a810d7a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009cd0 tx=0x7f91a002a8e0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.845+0000 7f91ad57c640 1 -- 172.21.15.120:0/406355732 shutdown_connections 2024-09-19T09:05:08.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.845+0000 7f91ad57c640 1 --2- 172.21.15.120:0/406355732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91a8101270 0x7f91a810d7a0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.845+0000 7f91ad57c640 1 --2- 172.21.15.120:0/406355732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91a81008d0 0x7f91a8100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.845+0000 7f91ad57c640 1 --2- 172.21.15.120:0/406355732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91a81068d0 0x7f91a8106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.845+0000 7f91ad57c640 1 -- 172.21.15.120:0/406355732 >> 172.21.15.120:0/406355732 conn(0x7f91a80fc610 msgr2=0x7f91a80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:08.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.845+0000 7f91ad57c640 1 -- 172.21.15.120:0/406355732 shutdown_connections 2024-09-19T09:05:08.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.845+0000 7f91ad57c640 1 -- 172.21.15.120:0/406355732 wait complete. 2024-09-19T09:05:08.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.846+0000 7f91ad57c640 1 Processor -- start 2024-09-19T09:05:08.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.846+0000 7f91ad57c640 1 -- start start 2024-09-19T09:05:08.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.846+0000 7f91ad57c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91a81008d0 0x7f91a8198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:08.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.846+0000 7f91ad57c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91a8101270 0x7f91a81987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:08.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.846+0000 7f91ad57c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91a81068d0 0x7f91a819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:08.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.846+0000 7f91ad57c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f91a8110cd0 con 0x7f91a81068d0 2024-09-19T09:05:08.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.846+0000 7f91ad57c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f91a8110b50 con 0x7f91a81008d0 2024-09-19T09:05:08.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.846+0000 7f91ad57c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f91a8110e50 con 0x7f91a8101270 2024-09-19T09:05:08.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.846+0000 7f91a7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91a81008d0 0x7f91a8198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:08.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.846+0000 7f91a7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91a81008d0 0x7f91a8198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:60568/0 (socket says 172.21.15.120:60568) 2024-09-19T09:05:08.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.846+0000 7f91acd7b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91a81068d0 0x7f91a819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:08.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.846+0000 7f91a7fff640 1 -- 172.21.15.120:0/3155896324 learned_addr learned my addr 172.21.15.120:0/3155896324 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:08.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.846+0000 7f919ffff640 1 --2- 172.21.15.120:0/3155896324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91a8101270 0x7f91a81987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:08.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.847+0000 7f91a7fff640 1 -- 172.21.15.120:0/3155896324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91a8101270 msgr2=0x7f91a81987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:08.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.847+0000 7f91a7fff640 1 --2- 172.21.15.120:0/3155896324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91a8101270 0x7f91a81987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.847+0000 7f91a7fff640 1 -- 172.21.15.120:0/3155896324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91a81068d0 msgr2=0x7f91a819cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:08.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.847+0000 7f91a7fff640 1 --2- 172.21.15.120:0/3155896324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91a81068d0 0x7f91a819cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.847+0000 7f91a7fff640 1 -- 172.21.15.120:0/3155896324 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a819d230 con 0x7f91a81008d0 2024-09-19T09:05:08.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.847+0000 7f919ffff640 1 --2- 172.21.15.120:0/3155896324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91a8101270 0x7f91a81987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:08.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.847+0000 7f91a7fff640 1 --2- 172.21.15.120:0/3155896324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91a81008d0 0x7f91a8198280 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f919800e9f0 tx=0x7f919800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:08.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.847+0000 7f91a5ffb640 1 -- 172.21.15.120:0/3155896324 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919800cd80 con 0x7f91a81008d0 2024-09-19T09:05:08.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.847+0000 7f91a5ffb640 1 -- 172.21.15.120:0/3155896324 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9198005500 con 0x7f91a81008d0 2024-09-19T09:05:08.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.847+0000 7f91ad57c640 1 -- 172.21.15.120:0/3155896324 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a819d520 con 0x7f91a81008d0 2024-09-19T09:05:08.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.847+0000 7f91a5ffb640 1 -- 172.21.15.120:0/3155896324 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9198010690 con 0x7f91a81008d0 2024-09-19T09:05:08.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.847+0000 7f91ad57c640 1 -- 172.21.15.120:0/3155896324 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a81023d0 con 0x7f91a81008d0 2024-09-19T09:05:08.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.849+0000 7f91a5ffb640 1 -- 172.21.15.120:0/3155896324 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f91980046c0 con 0x7f91a81008d0 2024-09-19T09:05:08.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.850+0000 7f91b0bb7740 1 Processor -- start 2024-09-19T09:05:08.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.850+0000 7f91b0bb7740 1 -- start start 2024-09-19T09:05:08.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.850+0000 7f91b0bb7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56324b73b670 0x56324b73ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:08.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.850+0000 7f91b0bb7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56324b73bf90 0x56324b746310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:08.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.850+0000 7f91acd7b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56324b73b670 0x56324b73ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:08.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.850+0000 7f91acd7b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56324b73b670 0x56324b73ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36852/0 (socket says 172.21.15.120:36852) 2024-09-19T09:05:08.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.850+0000 7f91b0bb7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56324b746850 0x56324b748c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:08.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.850+0000 7f91b0bb7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56324b6a12e0 con 0x56324b73b670 2024-09-19T09:05:08.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.850+0000 7f91b0bb7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56324b648ce0 con 0x56324b73bf90 2024-09-19T09:05:08.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.850+0000 7f91b0bb7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56324b69d2a0 con 0x56324b746850 2024-09-19T09:05:08.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.850+0000 7f91acd7b640 1 -- 172.21.15.120:0/1986642829 learned_addr learned my addr 172.21.15.120:0/1986642829 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:08.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.850+0000 7f91a7fff640 1 --2- 172.21.15.120:0/1986642829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56324b73bf90 0x56324b746310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:08.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.851+0000 7f919ffff640 1 --2- 172.21.15.120:0/1986642829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56324b746850 0x56324b748c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:08.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.851+0000 7f91acd7b640 1 -- 172.21.15.120:0/1986642829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56324b746850 msgr2=0x56324b748c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:08.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.851+0000 7f91acd7b640 1 --2- 172.21.15.120:0/1986642829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56324b746850 0x56324b748c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.851+0000 7f91acd7b640 1 -- 172.21.15.120:0/1986642829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56324b73bf90 msgr2=0x56324b746310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:08.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.851+0000 7f91acd7b640 1 --2- 172.21.15.120:0/1986642829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56324b73bf90 0x56324b746310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.851+0000 7f91acd7b640 1 -- 172.21.15.120:0/1986642829 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56324b62d440 con 0x56324b73b670 2024-09-19T09:05:08.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.851+0000 7f91a5ffb640 1 --2- 172.21.15.120:0/3155896324 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f91940a32c0 0x7f91940a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:08.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.851+0000 7f91acd7b640 1 --2- 172.21.15.120:0/1986642829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56324b73b670 0x56324b73ba50 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009cd0 tx=0x7f91a0009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22dcea9e1feb52bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:08.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.851+0000 7f919e7fc640 1 -- 172.21.15.120:0/1986642829 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a0048070 con 0x56324b73b670 2024-09-19T09:05:08.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.851+0000 7f91a7fff640 1 --2- 172.21.15.120:0/3155896324 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f91940a32c0 0x7f91940a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:08.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.851+0000 7f919e7fc640 1 -- 172.21.15.120:0/1986642829 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91a0036070 con 0x56324b73b670 2024-09-19T09:05:08.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.851+0000 7f919e7fc640 1 -- 172.21.15.120:0/1986642829 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a0004400 con 0x56324b73b670 2024-09-19T09:05:08.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.852+0000 7f91a5ffb640 1 -- 172.21.15.120:0/3155896324 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f91980cfc60 con 0x7f91a81008d0 2024-09-19T09:05:08.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.852+0000 7f91b0bb7740 1 -- 172.21.15.120:0/1986642829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56324b73b670 msgr2=0x56324b73ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:08.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.852+0000 7f91a7fff640 1 --2- 172.21.15.120:0/3155896324 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f91940a32c0 0x7f91940a5780 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f919809c210 tx=0x7f91980d36e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:08.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.852+0000 7f91b0bb7740 1 --2- 172.21.15.120:0/1986642829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56324b73b670 0x56324b73ba50 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009cd0 tx=0x7f91a0009a20 comp rx=0 tx=0).stop 2024-09-19T09:05:08.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.852+0000 7f91b0bb7740 1 -- 172.21.15.120:0/1986642829 shutdown_connections 2024-09-19T09:05:08.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.852+0000 7f91b0bb7740 1 --2- 172.21.15.120:0/1986642829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56324b746850 0x56324b748c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.852+0000 7f91b0bb7740 1 --2- 172.21.15.120:0/1986642829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56324b73bf90 0x56324b746310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.852+0000 7f91b0bb7740 1 --2- 172.21.15.120:0/1986642829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56324b73b670 0x56324b73ba50 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.852+0000 7f91b0bb7740 1 -- 172.21.15.120:0/1986642829 >> 172.21.15.120:0/1986642829 conn(0x56324b737c00 msgr2=0x56324b737fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:08.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.852+0000 7f91b0bb7740 1 -- 172.21.15.120:0/1986642829 shutdown_connections 2024-09-19T09:05:08.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.852+0000 7f91b0bb7740 1 -- 172.21.15.120:0/1986642829 wait complete. 2024-09-19T09:05:08.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.853+0000 7f91b0bb7740 1 -- start start 2024-09-19T09:05:08.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.853+0000 7f91b0bb7740 1 Processor -- start 2024-09-19T09:05:08.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.853+0000 7f91b0bb7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56324b73b670 0x56324b746220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:08.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.853+0000 7f91b0bb7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56324b73bf90 0x56324b7434d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:08.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.853+0000 7f91b0bb7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56324b746850 0x56324b743ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:08.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.853+0000 7f91b0bb7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56324b749420 con 0x56324b73bf90 2024-09-19T09:05:08.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.853+0000 7f91b0bb7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56324b65c050 con 0x56324b73b670 2024-09-19T09:05:08.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.853+0000 7f91b0bb7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56324b7495a0 con 0x56324b746850 2024-09-19T09:05:08.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.853+0000 7f91b0bb7740 10 client.0 _ll_register_callbacks cb 0x56324b69c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:08.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.853+0000 7f919ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56324b73bf90 0x56324b7434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:08.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.853+0000 7f919ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56324b73bf90 0x56324b7434d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36866/0 (socket says 172.21.15.120:36866) 2024-09-19T09:05:08.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.853+0000 7f919ffff640 1 -- 172.21.15.120:0/1344714184 learned_addr learned my addr 172.21.15.120:0/1344714184 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:08.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.853+0000 7f91acd7b640 1 --2- 172.21.15.120:0/1344714184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56324b73b670 0x56324b746220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:08.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.854+0000 7f91acd7b640 1 --2- 172.21.15.120:0/1344714184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56324b746850 0x56324b743ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:08.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.854+0000 7f919ffff640 1 -- 172.21.15.120:0/1344714184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56324b746850 msgr2=0x56324b743ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:08.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.854+0000 7f919ffff640 1 --2- 172.21.15.120:0/1344714184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56324b746850 0x56324b743ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.854+0000 7f919ffff640 1 -- 172.21.15.120:0/1344714184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56324b73b670 msgr2=0x56324b746220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:08.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.854+0000 7f919ffff640 1 --2- 172.21.15.120:0/1344714184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56324b73b670 0x56324b746220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.854+0000 7f919ffff640 1 -- 172.21.15.120:0/1344714184 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56324b4123c0 con 0x56324b73bf90 2024-09-19T09:05:08.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.854+0000 7f919ffff640 1 --2- 172.21.15.120:0/1344714184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56324b73bf90 0x56324b7434d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9190009f00 tx=0x7f919000b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3b9b52d5ba31ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:08.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.854+0000 7f91877fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:08.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.854+0000 7f91877fe640 1 -- 172.21.15.120:0/1344714184 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9190004460 con 0x56324b73bf90 2024-09-19T09:05:08.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.854+0000 7f91877fe640 1 -- 172.21.15.120:0/1344714184 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f91900035f0 con 0x56324b73bf90 2024-09-19T09:05:08.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.854+0000 7f91b0bb7740 1 -- 172.21.15.120:0/1344714184 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91a0036070 con 0x56324b73bf90 2024-09-19T09:05:08.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.854+0000 7f91877fe640 1 -- 172.21.15.120:0/1344714184 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f91900037f0 con 0x56324b73bf90 2024-09-19T09:05:08.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.854+0000 7f91b0bb7740 10 client.16071 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:08.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.854+0000 7f91b0bb7740 1 -- 172.21.15.120:0/1344714184 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56324b754e90 con 0x56324b73bf90 2024-09-19T09:05:08.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.854+0000 7f91877fe640 1 -- 172.21.15.120:0/1344714184 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91a0036070 con 0x56324b73bf90 2024-09-19T09:05:08.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.855+0000 7f91877fe640 1 -- 172.21.15.120:0/1344714184 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f919001b960 con 0x56324b73bf90 2024-09-19T09:05:08.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.855+0000 7f91b0bb7740 10 client.16071 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:08.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.855+0000 7f91b0bb7740 10 client.16071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:08.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.855+0000 7f91b0bb7740 20 client.16071 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:08.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.855+0000 7f91b0bb7740 1 --2- 172.21.15.120:0/1344714184 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56324b7563f0 0x56324b758830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:08.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.855+0000 7f91b0bb7740 4 client.16071 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:05:08.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.855+0000 7f91b0bb7740 1 -- 172.21.15.120:0/1344714184 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56324b6ae850 con 0x56324b7563f0 2024-09-19T09:05:08.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.855+0000 7f91acd7b640 1 --2- 172.21.15.120:0/1344714184 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56324b7563f0 0x56324b758830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:08.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.855+0000 7f91877fe640 10 client.16071 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:08.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.855+0000 7f91acd7b640 1 --2- 172.21.15.120:0/1344714184 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56324b7563f0 0x56324b758830 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4826fc08ce4156c server_cookie=eb5a502f3f26fe22 in_seq=0 out_seq=0 2024-09-19T09:05:08.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.856+0000 7f91877fe640 1 -- 172.21.15.120:0/1344714184 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56324b6ae850 con 0x56324b7563f0 2024-09-19T09:05:08.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.856+0000 7f91877fe640 10 client.16071 handle_command_reply: tid=0 2024-09-19T09:05:08.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.856+0000 7f91b0bb7740 1 client.16071 shutdown 2024-09-19T09:05:08.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.857+0000 7f91b0bb7740 1 -- 172.21.15.120:0/1344714184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56324b73bf90 msgr2=0x56324b7434d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:08.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.857+0000 7f91b0bb7740 1 --2- 172.21.15.120:0/1344714184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56324b73bf90 0x56324b7434d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9190009f00 tx=0x7f919000b690 comp rx=0 tx=0).stop 2024-09-19T09:05:08.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.857+0000 7f91b0bb7740 1 -- 172.21.15.120:0/1344714184 shutdown_connections 2024-09-19T09:05:08.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.857+0000 7f91b0bb7740 1 --2- 172.21.15.120:0/1344714184 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56324b7563f0 0x56324b758830 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.857+0000 7f91b0bb7740 1 --2- 172.21.15.120:0/1344714184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56324b746850 0x56324b743ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.857+0000 7f91b0bb7740 1 --2- 172.21.15.120:0/1344714184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56324b73bf90 0x56324b7434d0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.857+0000 7f91877fe640 0 client.16071 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:08.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.857+0000 7f91b0bb7740 1 --2- 172.21.15.120:0/1344714184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56324b73b670 0x56324b746220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.857+0000 7f91b0bb7740 1 -- 172.21.15.120:0/1344714184 >> 172.21.15.120:0/1344714184 conn(0x56324b737c00 msgr2=0x56324b73d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:08.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.857+0000 7f91b0bb7740 1 -- 172.21.15.120:0/1344714184 shutdown_connections 2024-09-19T09:05:08.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.857+0000 7f91b0bb7740 1 -- 172.21.15.120:0/1344714184 wait complete. 2024-09-19T09:05:08.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.857+0000 7f91b0bb7740 20 client.16071 trim_cache size 0 max 16384 2024-09-19T09:05:08.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.876+0000 7f91b0bb7740 1 Processor -- start 2024-09-19T09:05:08.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.876+0000 7f91b0bb7740 1 -- start start 2024-09-19T09:05:08.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.876+0000 7f91b0bb7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56324b73b670 0x56324b741de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:08.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.876+0000 7f91b0bb7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56324b73bf90 0x56324b75e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:08.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.876+0000 7f91b0bb7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56324b746850 0x56324b75e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:08.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.876+0000 7f91b0bb7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f91a81ab200 con 0x56324b73bf90 2024-09-19T09:05:08.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.876+0000 7f91b0bb7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f91a81acd40 con 0x56324b746850 2024-09-19T09:05:08.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.876+0000 7f91b0bb7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f91a81ab8d0 con 0x56324b73b670 2024-09-19T09:05:08.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.876+0000 7f919ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56324b73bf90 0x56324b75e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:08.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.876+0000 7f919ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56324b73bf90 0x56324b75e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36874/0 (socket says 172.21.15.120:36874) 2024-09-19T09:05:08.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.876+0000 7f919ffff640 1 -- 172.21.15.120:0/3662529022 learned_addr learned my addr 172.21.15.120:0/3662529022 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:08.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.876+0000 7f91acd7b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56324b73b670 0x56324b741de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:08.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.876+0000 7f91acd7b640 1 --2- 172.21.15.120:0/3662529022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56324b746850 0x56324b75e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:08.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.876+0000 7f919ffff640 1 -- 172.21.15.120:0/3662529022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56324b73b670 msgr2=0x56324b741de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:08.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.876+0000 7f919ffff640 1 --2- 172.21.15.120:0/3662529022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56324b73b670 0x56324b741de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.876+0000 7f919ffff640 1 -- 172.21.15.120:0/3662529022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56324b746850 msgr2=0x56324b75e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:08.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.876+0000 7f919ffff640 1 --2- 172.21.15.120:0/3662529022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56324b746850 0x56324b75e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.876+0000 7f919ffff640 1 -- 172.21.15.120:0/3662529022 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56324b7424c0 con 0x56324b73bf90 2024-09-19T09:05:08.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.877+0000 7f919ffff640 1 --2- 172.21.15.120:0/3662529022 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56324b73bf90 0x56324b75e100 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9190012010 tx=0x7f919000bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b82a990d0bfaa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:08.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.877+0000 7f919dffb640 1 -- 172.21.15.120:0/3662529022 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9190005300 con 0x56324b73bf90 2024-09-19T09:05:08.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.877+0000 7f919dffb640 1 -- 172.21.15.120:0/3662529022 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9190035070 con 0x56324b73bf90 2024-09-19T09:05:08.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.877+0000 7f919dffb640 1 -- 172.21.15.120:0/3662529022 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919001ba00 con 0x56324b73bf90 2024-09-19T09:05:08.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.877+0000 7f91b0bb7740 1 -- 172.21.15.120:0/3662529022 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56324b73bf90 msgr2=0x56324b75e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:08.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.877+0000 7f91b0bb7740 1 --2- 172.21.15.120:0/3662529022 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56324b73bf90 0x56324b75e100 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9190012010 tx=0x7f919000bbd0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.877+0000 7f91b0bb7740 1 -- 172.21.15.120:0/3662529022 shutdown_connections 2024-09-19T09:05:08.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.877+0000 7f91b0bb7740 1 --2- 172.21.15.120:0/3662529022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56324b746850 0x56324b75e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.877+0000 7f91b0bb7740 1 --2- 172.21.15.120:0/3662529022 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56324b73bf90 0x56324b75e100 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.877+0000 7f91b0bb7740 1 --2- 172.21.15.120:0/3662529022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56324b73b670 0x56324b741de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.877+0000 7f91b0bb7740 1 -- 172.21.15.120:0/3662529022 >> 172.21.15.120:0/3662529022 conn(0x56324b737c00 msgr2=0x56324b745550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:08.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.878+0000 7f91b0bb7740 1 -- 172.21.15.120:0/3662529022 shutdown_connections 2024-09-19T09:05:08.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.878+0000 7f91b0bb7740 1 -- 172.21.15.120:0/3662529022 wait complete. 2024-09-19T09:05:08.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.878+0000 7f91b0bb7740 1 -- start start 2024-09-19T09:05:08.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.878+0000 7f91b0bb7740 1 Processor -- start 2024-09-19T09:05:08.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.878+0000 7f91b0bb7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56324b73b670 0x56324b74c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:08.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.878+0000 7f91b0bb7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56324b73bf90 0x56324b74cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:08.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.878+0000 7f91b0bb7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56324b746850 0x56324b74d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:08.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.878+0000 7f91b0bb7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f91a81ac670 con 0x56324b73b670 2024-09-19T09:05:08.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.878+0000 7f91b0bb7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f91a81abfa0 con 0x56324b746850 2024-09-19T09:05:08.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.878+0000 7f91b0bb7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f91a81a91e0 con 0x56324b73bf90 2024-09-19T09:05:08.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.878+0000 7f91b0bb7740 10 client.0 _ll_register_callbacks cb 0x7f9190009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:08.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.878+0000 7f91acd7b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56324b73b670 0x56324b74c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:08.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.878+0000 7f919ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56324b73bf90 0x56324b74cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:08.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.878+0000 7f91acd7b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56324b73b670 0x56324b74c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36890/0 (socket says 172.21.15.120:36890) 2024-09-19T09:05:08.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.879+0000 7f91acd7b640 1 -- 172.21.15.120:0/1092506485 learned_addr learned my addr 172.21.15.120:0/1092506485 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:08.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.878+0000 7f919ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56324b73bf90 0x56324b74cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:35456/0 (socket says 172.21.15.120:35456) 2024-09-19T09:05:08.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.879+0000 7f91acd7b640 1 --2- 172.21.15.120:0/1092506485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56324b746850 0x56324b74d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:08.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.879+0000 7f91acd7b640 1 -- 172.21.15.120:0/1092506485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56324b73bf90 msgr2=0x56324b74cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:08.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.879+0000 7f91acd7b640 1 --2- 172.21.15.120:0/1092506485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56324b73bf90 0x56324b74cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.879+0000 7f91acd7b640 1 -- 172.21.15.120:0/1092506485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56324b746850 msgr2=0x56324b74d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:08.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.879+0000 7f91acd7b640 1 --2- 172.21.15.120:0/1092506485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56324b746850 0x56324b74d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.879+0000 7f91acd7b640 1 -- 172.21.15.120:0/1092506485 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56324b743e30 con 0x56324b73b670 2024-09-19T09:05:08.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.879+0000 7f919ffff640 1 --2- 172.21.15.120:0/1092506485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56324b73bf90 0x56324b74cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:08.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.879+0000 7f91acd7b640 1 --2- 172.21.15.120:0/1092506485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56324b73b670 0x56324b74c6f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f91a0005e30 tx=0x7f9190005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1188cffa13f95fe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:08.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.879+0000 7f91877fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:08.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.879+0000 7f91877fe640 1 -- 172.21.15.120:0/1092506485 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a0055070 con 0x56324b73b670 2024-09-19T09:05:08.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.879+0000 7f91877fe640 1 -- 172.21.15.120:0/1092506485 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f91a0031c90 con 0x56324b73b670 2024-09-19T09:05:08.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.879+0000 7f91b0bb7740 1 -- 172.21.15.120:0/1092506485 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9190035070 con 0x56324b73b670 2024-09-19T09:05:08.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.879+0000 7f91877fe640 1 -- 172.21.15.120:0/1092506485 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f91a0048040 con 0x56324b73b670 2024-09-19T09:05:08.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.879+0000 7f91b0bb7740 10 client.16077 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:08.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.879+0000 7f91b0bb7740 1 -- 172.21.15.120:0/1092506485 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56324b7c1320 con 0x56324b73b670 2024-09-19T09:05:08.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.879+0000 7f91877fe640 1 -- 172.21.15.120:0/1092506485 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9190035070 con 0x56324b73b670 2024-09-19T09:05:08.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.880+0000 7f91877fe640 1 -- 172.21.15.120:0/1092506485 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f91a004e040 con 0x56324b73b670 2024-09-19T09:05:08.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.880+0000 7f91b0bb7740 10 client.16077 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:08.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.880+0000 7f91b0bb7740 10 client.16077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:08.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.880+0000 7f91b0bb7740 20 client.16077 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:08.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.880+0000 7f91b0bb7740 1 --2- 172.21.15.120:0/1092506485 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56324b7c1550 0x56324b7c3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:08.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.880+0000 7f91b0bb7740 4 client.16077 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:05:08.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.880+0000 7f91b0bb7740 1 -- 172.21.15.120:0/1092506485 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56324b6a8d60 con 0x56324b7c1550 2024-09-19T09:05:08.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.880+0000 7f919ffff640 1 --2- 172.21.15.120:0/1092506485 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56324b7c1550 0x56324b7c3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:08.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.880+0000 7f91877fe640 10 client.16077 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:08.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.880+0000 7f919ffff640 1 --2- 172.21.15.120:0/1092506485 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56324b7c1550 0x56324b7c3940 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da80869bee0c2db6 server_cookie=9ca226bdc0b4d922 in_seq=0 out_seq=0 2024-09-19T09:05:08.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.881+0000 7f91877fe640 1 -- 172.21.15.120:0/1092506485 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56324b6a8d60 con 0x56324b7c1550 2024-09-19T09:05:08.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.881+0000 7f91877fe640 10 client.16077 handle_command_reply: tid=0 2024-09-19T09:05:08.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.881+0000 7f91b0bb7740 1 client.16077 shutdown 2024-09-19T09:05:08.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.881+0000 7f91b0bb7740 1 -- 172.21.15.120:0/1092506485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56324b73b670 msgr2=0x56324b74c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:08.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.881+0000 7f91b0bb7740 1 --2- 172.21.15.120:0/1092506485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56324b73b670 0x56324b74c6f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f91a0005e30 tx=0x7f9190005f90 comp rx=0 tx=0).stop 2024-09-19T09:05:08.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.881+0000 7f91b0bb7740 1 -- 172.21.15.120:0/1092506485 shutdown_connections 2024-09-19T09:05:08.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.881+0000 7f91b0bb7740 1 --2- 172.21.15.120:0/1092506485 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56324b7c1550 0x56324b7c3940 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.881+0000 7f91b0bb7740 1 --2- 172.21.15.120:0/1092506485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56324b746850 0x56324b74d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.881+0000 7f91877fe640 0 client.16077 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:08.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.881+0000 7f91b0bb7740 1 --2- 172.21.15.120:0/1092506485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56324b73bf90 0x56324b74cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.881+0000 7f91b0bb7740 1 --2- 172.21.15.120:0/1092506485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56324b73b670 0x56324b74c6f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.881+0000 7f91b0bb7740 1 -- 172.21.15.120:0/1092506485 >> 172.21.15.120:0/1092506485 conn(0x56324b737c00 msgr2=0x56324b736500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:08.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.882+0000 7f91b0bb7740 1 -- 172.21.15.120:0/1092506485 shutdown_connections 2024-09-19T09:05:08.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.882+0000 7f91b0bb7740 1 -- 172.21.15.120:0/1092506485 wait complete. 2024-09-19T09:05:08.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.882+0000 7f91b0bb7740 20 client.16077 trim_cache size 0 max 16384 2024-09-19T09:05:08.894 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:05:08.894 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:05:08.895 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:05:08.895 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:05:08.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.882+0000 7f91ad57c640 1 -- 172.21.15.120:0/3155896324 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f91940a32c0 msgr2=0x7f91940a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:08.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.882+0000 7f91ad57c640 1 --2- 172.21.15.120:0/3155896324 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f91940a32c0 0x7f91940a5780 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f919809c210 tx=0x7f91980d36e0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.882+0000 7f91ad57c640 1 -- 172.21.15.120:0/3155896324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91a81008d0 msgr2=0x7f91a8198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:08.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.882+0000 7f91ad57c640 1 --2- 172.21.15.120:0/3155896324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91a81008d0 0x7f91a8198280 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f919800e9f0 tx=0x7f919800edf0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.882+0000 7f91ad57c640 1 -- 172.21.15.120:0/3155896324 shutdown_connections 2024-09-19T09:05:08.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.882+0000 7f91ad57c640 1 --2- 172.21.15.120:0/3155896324 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f91940a32c0 0x7f91940a5780 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.882+0000 7f91ad57c640 1 --2- 172.21.15.120:0/3155896324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91a81068d0 0x7f91a819cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.882+0000 7f91ad57c640 1 --2- 172.21.15.120:0/3155896324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91a8101270 0x7f91a81987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.882+0000 7f91ad57c640 1 --2- 172.21.15.120:0/3155896324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91a81008d0 0x7f91a8198280 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:08.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.882+0000 7f91ad57c640 1 -- 172.21.15.120:0/3155896324 >> 172.21.15.120:0/3155896324 conn(0x7f91a80fc610 msgr2=0x7f91a806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:08.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.883+0000 7f91ad57c640 1 -- 172.21.15.120:0/3155896324 shutdown_connections 2024-09-19T09:05:08.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:08.883+0000 7f91ad57c640 1 -- 172.21.15.120:0/3155896324 wait complete. 2024-09-19T09:05:09.037 INFO:tasks.workunit.client.0.smithi120.stdout: 1 67399 5.98 MB/sec warmup 64 sec latency 23.786 ms 2024-09-19T09:05:09.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:08 smithi120 ceph-mon[31150]: pgmap v409: 97 pgs: 97 active+clean; 31 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 265 op/s 2024-09-19T09:05:09.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:08 smithi196 ceph-mon[35150]: pgmap v409: 97 pgs: 97 active+clean; 31 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 265 op/s 2024-09-19T09:05:09.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:08 smithi153 ceph-mon[35071]: pgmap v409: 97 pgs: 97 active+clean; 31 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 265 op/s 2024-09-19T09:05:09.967 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:05:09.967 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:05:10.038 INFO:tasks.workunit.client.0.smithi120.stdout: 1 70298 6.09 MB/sec warmup 65 sec latency 19.704 ms 2024-09-19T09:05:10.239 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:05:10.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.731+0000 7ff2fc6c0640 1 -- 172.21.15.120:0/2534698534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2f4102ef0 msgr2=0x7ff2f410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:10.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.731+0000 7ff2fc6c0640 1 --2- 172.21.15.120:0/2534698534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2f4102ef0 0x7ff2f410da40 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009ca0 tx=0x7ff2ec02a950 comp rx=0 tx=0).stop 2024-09-19T09:05:10.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.731+0000 7ff2fc6c0640 1 -- 172.21.15.120:0/2534698534 shutdown_connections 2024-09-19T09:05:10.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.731+0000 7ff2fc6c0640 1 --2- 172.21.15.120:0/2534698534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2f4102ef0 0x7ff2f410da40 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:10.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.731+0000 7ff2fc6c0640 1 --2- 172.21.15.120:0/2534698534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2f4102570 0x7ff2f41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:10.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.731+0000 7ff2fc6c0640 1 --2- 172.21.15.120:0/2534698534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2f4100e20 0x7ff2f4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:10.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.731+0000 7ff2fc6c0640 1 -- 172.21.15.120:0/2534698534 >> 172.21.15.120:0/2534698534 conn(0x7ff2f40fc7d0 msgr2=0x7ff2f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:10.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.732+0000 7ff2fc6c0640 1 -- 172.21.15.120:0/2534698534 shutdown_connections 2024-09-19T09:05:10.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.732+0000 7ff2fc6c0640 1 -- 172.21.15.120:0/2534698534 wait complete. 2024-09-19T09:05:10.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.732+0000 7ff2fc6c0640 1 Processor -- start 2024-09-19T09:05:10.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.732+0000 7ff2fc6c0640 1 -- start start 2024-09-19T09:05:10.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.732+0000 7ff2fc6c0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2f4100e20 0x7ff2f40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:10.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.732+0000 7ff2fc6c0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2f4102570 0x7ff2f40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:10.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.732+0000 7ff2fc6c0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2f4102ef0 0x7ff2f40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:10.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.732+0000 7ff2fc6c0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f4110fd0 con 0x7ff2f4100e20 2024-09-19T09:05:10.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.732+0000 7ff2fc6c0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f4110e50 con 0x7ff2f4102ef0 2024-09-19T09:05:10.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.732+0000 7ff2fc6c0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f4111150 con 0x7ff2f4102570 2024-09-19T09:05:10.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.733+0000 7ff2fa45c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2f4100e20 0x7ff2f40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:10.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.733+0000 7ff2fa45c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2f4100e20 0x7ff2f40fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36908/0 (socket says 172.21.15.120:36908) 2024-09-19T09:05:10.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.733+0000 7ff2fa45c640 1 -- 172.21.15.120:0/394132877 learned_addr learned my addr 172.21.15.120:0/394132877 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:10.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.733+0000 7ff2fac5d640 1 --2- 172.21.15.120:0/394132877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2f4102ef0 0x7ff2f40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:10.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.733+0000 7ff2f9c5b640 1 --2- 172.21.15.120:0/394132877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2f4102570 0x7ff2f40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:10.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.733+0000 7ff2fa45c640 1 -- 172.21.15.120:0/394132877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2f4102570 msgr2=0x7ff2f40fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:10.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.733+0000 7ff2fa45c640 1 --2- 172.21.15.120:0/394132877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2f4102570 0x7ff2f40fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:10.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.733+0000 7ff2fa45c640 1 -- 172.21.15.120:0/394132877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2f4102ef0 msgr2=0x7ff2f40fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:10.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.733+0000 7ff2fa45c640 1 --2- 172.21.15.120:0/394132877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2f4102ef0 0x7ff2f40fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:10.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.733+0000 7ff2fa45c640 1 -- 172.21.15.120:0/394132877 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f41a6940 con 0x7ff2f4100e20 2024-09-19T09:05:10.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.733+0000 7ff2fa45c640 1 --2- 172.21.15.120:0/394132877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2f4100e20 0x7ff2f40fcc80 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000e9f0 tx=0x7ff2f000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:10.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.733+0000 7ff2eb7fe640 1 -- 172.21.15.120:0/394132877 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f000cd80 con 0x7ff2f4100e20 2024-09-19T09:05:10.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.733+0000 7ff2eb7fe640 1 -- 172.21.15.120:0/394132877 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f0005500 con 0x7ff2f4100e20 2024-09-19T09:05:10.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.733+0000 7ff2eb7fe640 1 -- 172.21.15.120:0/394132877 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0010690 con 0x7ff2f4100e20 2024-09-19T09:05:10.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.733+0000 7ff2fc6c0640 1 -- 172.21.15.120:0/394132877 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f41a6c30 con 0x7ff2f4100e20 2024-09-19T09:05:10.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.734+0000 7ff2fc6c0640 1 -- 172.21.15.120:0/394132877 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f41a7120 con 0x7ff2f4100e20 2024-09-19T09:05:10.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.735+0000 7ff2fc6c0640 1 -- 172.21.15.120:0/394132877 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2f4101200 con 0x7ff2f4100e20 2024-09-19T09:05:10.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.737+0000 7ff2eb7fe640 1 -- 172.21.15.120:0/394132877 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7ff2f00046c0 con 0x7ff2f4100e20 2024-09-19T09:05:10.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.737+0000 7ff2eb7fe640 1 --2- 172.21.15.120:0/394132877 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff2e00a3240 0x7ff2e00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:10.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.737+0000 7ff2f9c5b640 1 --2- 172.21.15.120:0/394132877 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff2e00a3240 0x7ff2e00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:10.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.737+0000 7ff2eb7fe640 1 -- 172.21.15.120:0/394132877 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7ff2f00cff70 con 0x7ff2f4100e20 2024-09-19T09:05:10.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.738+0000 7ff2f9c5b640 1 --2- 172.21.15.120:0/394132877 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff2e00a3240 0x7ff2e00a5700 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40fe710 tx=0x7ff2e4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:10.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.741+0000 7ff2eb7fe640 1 -- 172.21.15.120:0/394132877 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff2f0054210 con 0x7ff2f4100e20 2024-09-19T09:05:10.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.965+0000 7ff2fc6c0640 1 -- 172.21.15.120:0/394132877 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2f40fe1b0 con 0x7ff2f4100e20 2024-09-19T09:05:10.966 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:10 smithi120 ceph-mon[31150]: pgmap v410: 97 pgs: 97 active+clean; 32 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 271 op/s 2024-09-19T09:05:10.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.966+0000 7ff2eb7fe640 1 -- 172.21.15.120:0/394132877 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7ff2f40fe1b0 con 0x7ff2f4100e20 2024-09-19T09:05:10.968 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:05:10.968 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":28,"btime":"2024-09-19T09:02:41:696728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:02:41.207357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:05:10.968 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 28 2024-09-19T09:05:10.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.969+0000 7ff2fc6c0640 1 -- 172.21.15.120:0/394132877 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff2e00a3240 msgr2=0x7ff2e00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:10.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.969+0000 7ff2fc6c0640 1 --2- 172.21.15.120:0/394132877 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff2e00a3240 0x7ff2e00a5700 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40fe710 tx=0x7ff2e4005f90 comp rx=0 tx=0).stop 2024-09-19T09:05:10.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.969+0000 7ff2fc6c0640 1 -- 172.21.15.120:0/394132877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2f4100e20 msgr2=0x7ff2f40fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:10.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.969+0000 7ff2fc6c0640 1 --2- 172.21.15.120:0/394132877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2f4100e20 0x7ff2f40fcc80 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000e9f0 tx=0x7ff2f000edf0 comp rx=0 tx=0).stop 2024-09-19T09:05:10.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.969+0000 7ff2fc6c0640 1 -- 172.21.15.120:0/394132877 shutdown_connections 2024-09-19T09:05:10.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.969+0000 7ff2fc6c0640 1 --2- 172.21.15.120:0/394132877 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff2e00a3240 0x7ff2e00a5700 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:10.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.969+0000 7ff2fc6c0640 1 --2- 172.21.15.120:0/394132877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2f4102ef0 0x7ff2f40fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:10.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.969+0000 7ff2fc6c0640 1 --2- 172.21.15.120:0/394132877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2f4102570 0x7ff2f40fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:10.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.969+0000 7ff2fc6c0640 1 --2- 172.21.15.120:0/394132877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2f4100e20 0x7ff2f40fcc80 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:10.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.969+0000 7ff2fc6c0640 1 -- 172.21.15.120:0/394132877 >> 172.21.15.120:0/394132877 conn(0x7ff2f40fc7d0 msgr2=0x7ff2f406a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:10.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.969+0000 7ff2fc6c0640 1 -- 172.21.15.120:0/394132877 shutdown_connections 2024-09-19T09:05:10.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:10.969+0000 7ff2fc6c0640 1 -- 172.21.15.120:0/394132877 wait complete. 2024-09-19T09:05:11.038 INFO:tasks.workunit.client.0.smithi120.stdout: 1 73434 6.22 MB/sec warmup 66 sec latency 12.560 ms 2024-09-19T09:05:11.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:10 smithi196 ceph-mon[35150]: pgmap v410: 97 pgs: 97 active+clean; 32 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 271 op/s 2024-09-19T09:05:11.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:10 smithi153 ceph-mon[35071]: pgmap v410: 97 pgs: 97 active+clean; 32 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 271 op/s 2024-09-19T09:05:12.038 INFO:tasks.workunit.client.0.smithi120.stdout: 1 76319 6.39 MB/sec warmup 67 sec latency 12.462 ms 2024-09-19T09:05:12.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:11 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/394132877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:05:12.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:11 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/394132877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:05:12.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:11 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/394132877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:05:12.406 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:05:12.671 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:05:13.038 INFO:tasks.workunit.client.0.smithi120.stdout: 1 79146 6.52 MB/sec warmup 68 sec latency 13.245 ms 2024-09-19T09:05:13.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:12 smithi120 ceph-mon[31150]: pgmap v411: 97 pgs: 97 active+clean; 38 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.1 MiB/s wr, 353 op/s 2024-09-19T09:05:13.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:12 smithi196 ceph-mon[35150]: pgmap v411: 97 pgs: 97 active+clean; 38 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.1 MiB/s wr, 353 op/s 2024-09-19T09:05:13.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:12 smithi153 ceph-mon[35071]: pgmap v411: 97 pgs: 97 active+clean; 38 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.1 MiB/s wr, 353 op/s 2024-09-19T09:05:13.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.169+0000 7fb773aa5640 1 -- 172.21.15.120:0/518480945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb76c102ef0 msgr2=0x7fb76c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:13.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.169+0000 7fb773aa5640 1 --2- 172.21.15.120:0/518480945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb76c102ef0 0x7fb76c10da40 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb764009860 tx=0x7fb76402a3d0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.169+0000 7fb773aa5640 1 -- 172.21.15.120:0/518480945 shutdown_connections 2024-09-19T09:05:13.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.169+0000 7fb773aa5640 1 --2- 172.21.15.120:0/518480945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb76c102ef0 0x7fb76c10da40 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.169+0000 7fb773aa5640 1 --2- 172.21.15.120:0/518480945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb76c102570 0x7fb76c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.169+0000 7fb773aa5640 1 --2- 172.21.15.120:0/518480945 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb76c100e20 0x7fb76c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.169+0000 7fb773aa5640 1 -- 172.21.15.120:0/518480945 >> 172.21.15.120:0/518480945 conn(0x7fb76c0fc7d0 msgr2=0x7fb76c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:13.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.170+0000 7fb773aa5640 1 -- 172.21.15.120:0/518480945 shutdown_connections 2024-09-19T09:05:13.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.170+0000 7fb773aa5640 1 -- 172.21.15.120:0/518480945 wait complete. 2024-09-19T09:05:13.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.171+0000 7fb773aa5640 1 Processor -- start 2024-09-19T09:05:13.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.171+0000 7fb773aa5640 1 -- start start 2024-09-19T09:05:13.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.171+0000 7fb773aa5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb76c100e20 0x7fb76c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:13.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.171+0000 7fb773aa5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb76c102570 0x7fb76c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:13.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.171+0000 7fb773aa5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb76c102ef0 0x7fb76c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:13.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.171+0000 7fb773aa5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c110f80 con 0x7fb76c102ef0 2024-09-19T09:05:13.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.171+0000 7fb773aa5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c110e00 con 0x7fb76c100e20 2024-09-19T09:05:13.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.171+0000 7fb773aa5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c111100 con 0x7fb76c102570 2024-09-19T09:05:13.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.171+0000 7fb771841640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb76c100e20 0x7fb76c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:13.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.171+0000 7fb771841640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb76c100e20 0x7fb76c198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38174/0 (socket says 172.21.15.120:38174) 2024-09-19T09:05:13.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.171+0000 7fb771841640 1 -- 172.21.15.120:0/2024340679 learned_addr learned my addr 172.21.15.120:0/2024340679 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:13.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.172+0000 7fb771040640 1 --2- 172.21.15.120:0/2024340679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb76c102570 0x7fb76c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:13.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.172+0000 7fb772042640 1 --2- 172.21.15.120:0/2024340679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb76c102ef0 0x7fb76c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:13.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.172+0000 7fb771841640 1 -- 172.21.15.120:0/2024340679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb76c102570 msgr2=0x7fb76c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:13.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.172+0000 7fb771841640 1 --2- 172.21.15.120:0/2024340679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb76c102570 0x7fb76c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.172+0000 7fb771841640 1 -- 172.21.15.120:0/2024340679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb76c102ef0 msgr2=0x7fb76c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:13.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.172+0000 7fb771841640 1 --2- 172.21.15.120:0/2024340679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb76c102ef0 0x7fb76c19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.172+0000 7fb771841640 1 -- 172.21.15.120:0/2024340679 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76c19d440 con 0x7fb76c100e20 2024-09-19T09:05:13.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.172+0000 7fb771040640 1 --2- 172.21.15.120:0/2024340679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb76c102570 0x7fb76c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:13.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.172+0000 7fb771841640 1 --2- 172.21.15.120:0/2024340679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb76c100e20 0x7fb76c198490 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb76800c420 tx=0x7fb768009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:13.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.172+0000 7fb762ffd640 1 -- 172.21.15.120:0/2024340679 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb768003970 con 0x7fb76c100e20 2024-09-19T09:05:13.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.172+0000 7fb773aa5640 1 -- 172.21.15.120:0/2024340679 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb76c19d730 con 0x7fb76c100e20 2024-09-19T09:05:13.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.173+0000 7fb773aa5640 1 -- 172.21.15.120:0/2024340679 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb76c107a10 con 0x7fb76c100e20 2024-09-19T09:05:13.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.173+0000 7fb762ffd640 1 -- 172.21.15.120:0/2024340679 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb768004020 con 0x7fb76c100e20 2024-09-19T09:05:13.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.173+0000 7fb762ffd640 1 -- 172.21.15.120:0/2024340679 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76800bad0 con 0x7fb76c100e20 2024-09-19T09:05:13.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.175+0000 7fb762ffd640 1 -- 172.21.15.120:0/2024340679 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fb76801d020 con 0x7fb76c100e20 2024-09-19T09:05:13.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.175+0000 7fb775e7e740 1 Processor -- start 2024-09-19T09:05:13.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.176+0000 7fb775e7e740 1 -- start start 2024-09-19T09:05:13.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.176+0000 7fb775e7e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634e09a9b40 0x5634e09a8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:13.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.176+0000 7fb775e7e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634e09a85d0 0x5634e09a8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:13.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.176+0000 7fb775e7e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634e09b3370 0x5634e09b5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:13.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.176+0000 7fb775e7e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5634e090e2f0 con 0x5634e09b3370 2024-09-19T09:05:13.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.176+0000 7fb775e7e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5634e08b5cf0 con 0x5634e09a9b40 2024-09-19T09:05:13.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.176+0000 7fb775e7e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5634e090a2b0 con 0x5634e09a85d0 2024-09-19T09:05:13.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.176+0000 7fb771040640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634e09b3370 0x5634e09b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:13.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.176+0000 7fb771841640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634e09a85d0 0x5634e09a8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:13.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.176+0000 7fb772042640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634e09a9b40 0x5634e09a8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:13.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.176+0000 7fb762ffd640 1 --2- 172.21.15.120:0/2024340679 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb7580a3300 0x7fb7580a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:13.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.176+0000 7fb771040640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634e09b3370 0x5634e09b5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34104/0 (socket says 172.21.15.120:34104) 2024-09-19T09:05:13.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.176+0000 7fb771040640 1 -- 172.21.15.120:0/1247142775 learned_addr learned my addr 172.21.15.120:0/1247142775 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:13.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.176+0000 7fb772042640 1 --2- 172.21.15.120:0/2024340679 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb7580a3300 0x7fb7580a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:13.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.176+0000 7fb771040640 1 -- 172.21.15.120:0/1247142775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634e09a85d0 msgr2=0x5634e09a8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:13.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.176+0000 7fb771040640 1 --2- 172.21.15.120:0/1247142775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634e09a85d0 0x5634e09a8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.176+0000 7fb771040640 1 -- 172.21.15.120:0/1247142775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634e09a9b40 msgr2=0x5634e09a8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:13.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.176+0000 7fb771040640 1 --2- 172.21.15.120:0/1247142775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634e09a9b40 0x5634e09a8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.176+0000 7fb771040640 1 -- 172.21.15.120:0/1247142775 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634e089a3a0 con 0x5634e09b3370 2024-09-19T09:05:13.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.176+0000 7fb762ffd640 1 -- 172.21.15.120:0/2024340679 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb7680cf590 con 0x7fb76c100e20 2024-09-19T09:05:13.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.177+0000 7fb771040640 1 --2- 172.21.15.120:0/1247142775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634e09b3370 0x5634e09b5760 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0096f0 tx=0x7fb75c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38dbb588dd685239 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:13.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.177+0000 7fb757fff640 1 -- 172.21.15.120:0/1247142775 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c004030 con 0x5634e09b3370 2024-09-19T09:05:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.177+0000 7fb772042640 1 --2- 172.21.15.120:0/2024340679 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb7580a3300 0x7fb7580a57c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb7640095f0 tx=0x7fb76402a3d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.177+0000 7fb757fff640 1 -- 172.21.15.120:0/1247142775 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb75c00bd80 con 0x5634e09b3370 2024-09-19T09:05:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.177+0000 7fb757fff640 1 -- 172.21.15.120:0/1247142775 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c003c30 con 0x5634e09b3370 2024-09-19T09:05:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.178+0000 7fb775e7e740 1 -- 172.21.15.120:0/1247142775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634e09b3370 msgr2=0x5634e09b5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.178+0000 7fb775e7e740 1 --2- 172.21.15.120:0/1247142775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634e09b3370 0x5634e09b5760 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0096f0 tx=0x7fb75c005f90 comp rx=0 tx=0).stop 2024-09-19T09:05:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.178+0000 7fb775e7e740 1 -- 172.21.15.120:0/1247142775 shutdown_connections 2024-09-19T09:05:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.178+0000 7fb775e7e740 1 --2- 172.21.15.120:0/1247142775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634e09b3370 0x5634e09b5760 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.178+0000 7fb775e7e740 1 --2- 172.21.15.120:0/1247142775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634e09a85d0 0x5634e09a8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.178+0000 7fb775e7e740 1 --2- 172.21.15.120:0/1247142775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634e09a9b40 0x5634e09a8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.178+0000 7fb775e7e740 1 -- 172.21.15.120:0/1247142775 >> 172.21.15.120:0/1247142775 conn(0x5634e0971310 msgr2=0x5634e09a4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.178+0000 7fb775e7e740 1 -- 172.21.15.120:0/1247142775 shutdown_connections 2024-09-19T09:05:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.178+0000 7fb775e7e740 1 -- 172.21.15.120:0/1247142775 wait complete. 2024-09-19T09:05:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.178+0000 7fb775e7e740 1 -- start start 2024-09-19T09:05:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.178+0000 7fb775e7e740 1 Processor -- start 2024-09-19T09:05:13.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.179+0000 7fb775e7e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634e09a85d0 0x5634e09b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:13.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.179+0000 7fb775e7e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634e09a9b40 0x5634e09aff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:13.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.179+0000 7fb775e7e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634e09b3370 0x5634e09b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:13.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.179+0000 7fb775e7e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5634e09b60c0 con 0x5634e09a9b40 2024-09-19T09:05:13.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.179+0000 7fb775e7e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5634e08c9060 con 0x5634e09b3370 2024-09-19T09:05:13.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.179+0000 7fb775e7e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5634e09b6240 con 0x5634e09a85d0 2024-09-19T09:05:13.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.179+0000 7fb775e7e740 10 client.0 _ll_register_callbacks cb 0x5634e061c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:13.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.179+0000 7fb771040640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634e09a9b40 0x5634e09aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:13.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.179+0000 7fb771040640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634e09a9b40 0x5634e09aff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34114/0 (socket says 172.21.15.120:34114) 2024-09-19T09:05:13.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.179+0000 7fb771841640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634e09a85d0 0x5634e09b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:13.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.179+0000 7fb771040640 1 -- 172.21.15.120:0/2132457713 learned_addr learned my addr 172.21.15.120:0/2132457713 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:13.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.179+0000 7fb772042640 1 --2- 172.21.15.120:0/2132457713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634e09b3370 0x5634e09b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:13.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.179+0000 7fb771040640 1 -- 172.21.15.120:0/2132457713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634e09a85d0 msgr2=0x5634e09b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:13.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.179+0000 7fb771040640 1 --2- 172.21.15.120:0/2132457713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634e09a85d0 0x5634e09b2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.179+0000 7fb771040640 1 -- 172.21.15.120:0/2132457713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634e09b3370 msgr2=0x5634e09b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:13.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.179+0000 7fb771040640 1 --2- 172.21.15.120:0/2132457713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634e09b3370 0x5634e09b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.179+0000 7fb771040640 1 -- 172.21.15.120:0/2132457713 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634e067f3c0 con 0x5634e09a9b40 2024-09-19T09:05:13.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.179+0000 7fb771040640 1 --2- 172.21.15.120:0/2132457713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634e09a9b40 0x5634e09aff20 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0037d0 tx=0x7fb75c009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a6b54636f37a8ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:13.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.179+0000 7fb7557fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:13.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.179+0000 7fb7557fa640 1 -- 172.21.15.120:0/2132457713 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c025860 con 0x5634e09a9b40 2024-09-19T09:05:13.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.179+0000 7fb7557fa640 1 -- 172.21.15.120:0/2132457713 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb75c025a00 con 0x5634e09a9b40 2024-09-19T09:05:13.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.179+0000 7fb775e7e740 1 -- 172.21.15.120:0/2132457713 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb75c00bd80 con 0x5634e09a9b40 2024-09-19T09:05:13.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.180+0000 7fb7557fa640 1 -- 172.21.15.120:0/2132457713 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb75c025c00 con 0x5634e09a9b40 2024-09-19T09:05:13.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.180+0000 7fb775e7e740 10 client.16095 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:13.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.180+0000 7fb775e7e740 1 -- 172.21.15.120:0/2132457713 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5634e09c1a70 con 0x5634e09a9b40 2024-09-19T09:05:13.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.180+0000 7fb7557fa640 1 -- 172.21.15.120:0/2132457713 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb75c00bd80 con 0x5634e09a9b40 2024-09-19T09:05:13.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.180+0000 7fb7557fa640 1 -- 172.21.15.120:0/2132457713 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb75c024470 con 0x5634e09a9b40 2024-09-19T09:05:13.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.180+0000 7fb775e7e740 10 client.16095 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:13.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.180+0000 7fb775e7e740 10 client.16095 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:05:13.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.180+0000 7fb775e7e740 20 client.16095 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:13.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.180+0000 7fb775e7e740 1 --2- 172.21.15.120:0/2132457713 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5634e09c2e70 0x5634e09c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:13.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.180+0000 7fb775e7e740 4 client.16095 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:05:13.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.180+0000 7fb775e7e740 1 -- 172.21.15.120:0/2132457713 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634e091b860 con 0x5634e09c2e70 2024-09-19T09:05:13.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.180+0000 7fb771841640 1 --2- 172.21.15.120:0/2132457713 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5634e09c2e70 0x5634e09c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:13.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.181+0000 7fb7557fa640 10 client.16095 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:05:13.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.181+0000 7fb771841640 1 --2- 172.21.15.120:0/2132457713 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5634e09c2e70 0x5634e09c5260 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=48052c42f72664d7 server_cookie=ada039f9adfe2de8 in_seq=0 out_seq=0 2024-09-19T09:05:13.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.181+0000 7fb7557fa640 1 -- 172.21.15.120:0/2132457713 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5634e091b860 con 0x5634e09c2e70 2024-09-19T09:05:13.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.181+0000 7fb7557fa640 10 client.16095 handle_command_reply: tid=0 2024-09-19T09:05:13.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.181+0000 7fb775e7e740 1 client.16095 shutdown 2024-09-19T09:05:13.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.181+0000 7fb775e7e740 1 -- 172.21.15.120:0/2132457713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634e09a9b40 msgr2=0x5634e09aff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:13.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.181+0000 7fb775e7e740 1 --2- 172.21.15.120:0/2132457713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634e09a9b40 0x5634e09aff20 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0037d0 tx=0x7fb75c009fa0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.182+0000 7fb775e7e740 1 -- 172.21.15.120:0/2132457713 shutdown_connections 2024-09-19T09:05:13.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.182+0000 7fb775e7e740 1 --2- 172.21.15.120:0/2132457713 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5634e09c2e70 0x5634e09c5260 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.182+0000 7fb775e7e740 1 --2- 172.21.15.120:0/2132457713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634e09b3370 0x5634e09b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.182+0000 7fb775e7e740 1 --2- 172.21.15.120:0/2132457713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634e09a9b40 0x5634e09aff20 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.182+0000 7fb775e7e740 1 --2- 172.21.15.120:0/2132457713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634e09a85d0 0x5634e09b2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.182+0000 7fb7557fa640 0 client.16095 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:05:13.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.182+0000 7fb775e7e740 1 -- 172.21.15.120:0/2132457713 >> 172.21.15.120:0/2132457713 conn(0x5634e0971310 msgr2=0x5634e09aa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:13.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.182+0000 7fb775e7e740 1 -- 172.21.15.120:0/2132457713 shutdown_connections 2024-09-19T09:05:13.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.182+0000 7fb775e7e740 1 -- 172.21.15.120:0/2132457713 wait complete. 2024-09-19T09:05:13.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.182+0000 7fb775e7e740 20 client.16095 trim_cache size 0 max 16384 2024-09-19T09:05:13.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.195+0000 7fb775e7e740 1 Processor -- start 2024-09-19T09:05:13.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.195+0000 7fb775e7e740 1 -- start start 2024-09-19T09:05:13.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.195+0000 7fb775e7e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634e09a85d0 0x5634e09bf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:13.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.195+0000 7fb775e7e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634e09a9b40 0x5634e09a7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:13.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.195+0000 7fb775e7e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634e09b3370 0x5634e09aef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:13.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.195+0000 7fb775e7e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c1ab3e0 con 0x5634e09a85d0 2024-09-19T09:05:13.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.195+0000 7fb775e7e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c1acf20 con 0x5634e09a9b40 2024-09-19T09:05:13.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.195+0000 7fb775e7e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c1abab0 con 0x5634e09b3370 2024-09-19T09:05:13.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.195+0000 7fb771841640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634e09a85d0 0x5634e09bf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:13.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.195+0000 7fb771841640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634e09a85d0 0x5634e09bf360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34124/0 (socket says 172.21.15.120:34124) 2024-09-19T09:05:13.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.195+0000 7fb771841640 1 -- 172.21.15.120:0/2071395873 learned_addr learned my addr 172.21.15.120:0/2071395873 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:13.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.195+0000 7fb771040640 1 --2- 172.21.15.120:0/2071395873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634e09a9b40 0x5634e09a7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:13.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.195+0000 7fb772042640 1 --2- 172.21.15.120:0/2071395873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634e09b3370 0x5634e09aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:13.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.196+0000 7fb771841640 1 -- 172.21.15.120:0/2071395873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634e09b3370 msgr2=0x5634e09aef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:13.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.196+0000 7fb771841640 1 --2- 172.21.15.120:0/2071395873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634e09b3370 0x5634e09aef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.196+0000 7fb771841640 1 -- 172.21.15.120:0/2071395873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634e09a9b40 msgr2=0x5634e09a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:13.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.196+0000 7fb771841640 1 --2- 172.21.15.120:0/2071395873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634e09a9b40 0x5634e09a7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.196+0000 7fb771841640 1 -- 172.21.15.120:0/2071395873 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634e09a5960 con 0x5634e09a85d0 2024-09-19T09:05:13.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.196+0000 7fb771841640 1 --2- 172.21.15.120:0/2071395873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634e09a85d0 0x5634e09bf360 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb76809bb40 tx=0x7fb7680d1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4db21070965624 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:13.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.196+0000 7fb7577fe640 1 -- 172.21.15.120:0/2071395873 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76809bbb0 con 0x5634e09a85d0 2024-09-19T09:05:13.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.196+0000 7fb7577fe640 1 -- 172.21.15.120:0/2071395873 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb76809ff60 con 0x5634e09a85d0 2024-09-19T09:05:13.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.196+0000 7fb7577fe640 1 -- 172.21.15.120:0/2071395873 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7680041d0 con 0x5634e09a85d0 2024-09-19T09:05:13.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.197+0000 7fb775e7e740 1 -- 172.21.15.120:0/2071395873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634e09a85d0 msgr2=0x5634e09bf360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:13.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.197+0000 7fb775e7e740 1 --2- 172.21.15.120:0/2071395873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634e09a85d0 0x5634e09bf360 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb76809bb40 tx=0x7fb7680d1040 comp rx=0 tx=0).stop 2024-09-19T09:05:13.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.197+0000 7fb775e7e740 1 -- 172.21.15.120:0/2071395873 shutdown_connections 2024-09-19T09:05:13.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.197+0000 7fb775e7e740 1 --2- 172.21.15.120:0/2071395873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634e09b3370 0x5634e09aef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.197+0000 7fb775e7e740 1 --2- 172.21.15.120:0/2071395873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634e09a9b40 0x5634e09a7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.197+0000 7fb775e7e740 1 --2- 172.21.15.120:0/2071395873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634e09a85d0 0x5634e09bf360 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.197+0000 7fb775e7e740 1 -- 172.21.15.120:0/2071395873 >> 172.21.15.120:0/2071395873 conn(0x5634e0971310 msgr2=0x5634e09ae7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:13.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.197+0000 7fb775e7e740 1 -- 172.21.15.120:0/2071395873 shutdown_connections 2024-09-19T09:05:13.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.197+0000 7fb775e7e740 1 -- 172.21.15.120:0/2071395873 wait complete. 2024-09-19T09:05:13.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.197+0000 7fb775e7e740 1 -- start start 2024-09-19T09:05:13.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.197+0000 7fb775e7e740 1 Processor -- start 2024-09-19T09:05:13.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.198+0000 7fb775e7e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634e09a85d0 0x5634e09bb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:13.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.198+0000 7fb775e7e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634e09a9b40 0x5634e09bba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:13.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.198+0000 7fb775e7e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634e09b3370 0x5634e09ac190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:13.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.198+0000 7fb775e7e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c1ac850 con 0x5634e09b3370 2024-09-19T09:05:13.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.198+0000 7fb775e7e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c1ac180 con 0x5634e09a9b40 2024-09-19T09:05:13.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.198+0000 7fb775e7e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c1a93a0 con 0x5634e09a85d0 2024-09-19T09:05:13.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.198+0000 7fb775e7e740 10 client.0 _ll_register_callbacks cb 0x7fb75c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:13.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.198+0000 7fb772042640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634e09b3370 0x5634e09ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:13.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.198+0000 7fb772042640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634e09b3370 0x5634e09ac190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34138/0 (socket says 172.21.15.120:34138) 2024-09-19T09:05:13.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.198+0000 7fb772042640 1 -- 172.21.15.120:0/930188163 learned_addr learned my addr 172.21.15.120:0/930188163 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:13.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.198+0000 7fb771040640 1 --2- 172.21.15.120:0/930188163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634e09a9b40 0x5634e09bba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:13.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.198+0000 7fb771841640 1 --2- 172.21.15.120:0/930188163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634e09a85d0 0x5634e09bb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:13.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.198+0000 7fb772042640 1 -- 172.21.15.120:0/930188163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634e09a85d0 msgr2=0x5634e09bb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:13.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.198+0000 7fb772042640 1 --2- 172.21.15.120:0/930188163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634e09a85d0 0x5634e09bb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.198+0000 7fb772042640 1 -- 172.21.15.120:0/930188163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634e09a9b40 msgr2=0x5634e09bba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:13.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.198+0000 7fb772042640 1 --2- 172.21.15.120:0/930188163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634e09a9b40 0x5634e09bba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.198+0000 7fb772042640 1 -- 172.21.15.120:0/930188163 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634e09ab120 con 0x5634e09b3370 2024-09-19T09:05:13.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.198+0000 7fb772042640 1 --2- 172.21.15.120:0/930188163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634e09b3370 0x5634e09ac190 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb764009780 tx=0x7fb764005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d5d94894039eba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:13.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.198+0000 7fb7557fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:13.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.198+0000 7fb7557fa640 1 -- 172.21.15.120:0/930188163 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb764053030 con 0x5634e09b3370 2024-09-19T09:05:13.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.198+0000 7fb7557fa640 1 -- 172.21.15.120:0/930188163 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb764049040 con 0x5634e09b3370 2024-09-19T09:05:13.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.198+0000 7fb775e7e740 1 -- 172.21.15.120:0/930188163 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb76809ff60 con 0x5634e09b3370 2024-09-19T09:05:13.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.199+0000 7fb7557fa640 1 -- 172.21.15.120:0/930188163 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb7640048d0 con 0x5634e09b3370 2024-09-19T09:05:13.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.199+0000 7fb775e7e740 10 client.16101 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:13.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.199+0000 7fb775e7e740 1 -- 172.21.15.120:0/930188163 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5634e09ad4c0 con 0x5634e09b3370 2024-09-19T09:05:13.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.199+0000 7fb7557fa640 1 -- 172.21.15.120:0/930188163 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb76809ff60 con 0x5634e09b3370 2024-09-19T09:05:13.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.199+0000 7fb7557fa640 1 -- 172.21.15.120:0/930188163 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb76403cb50 con 0x5634e09b3370 2024-09-19T09:05:13.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.199+0000 7fb775e7e740 10 client.16101 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:13.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.199+0000 7fb775e7e740 10 client.16101 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:05:13.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.199+0000 7fb775e7e740 20 client.16101 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:13.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.199+0000 7fb775e7e740 1 --2- 172.21.15.120:0/930188163 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5634e09adc40 0x5634e0a309e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:13.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.199+0000 7fb775e7e740 4 client.16101 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:05:13.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.199+0000 7fb775e7e740 1 -- 172.21.15.120:0/930188163 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5634e0915d70 con 0x5634e09adc40 2024-09-19T09:05:13.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.199+0000 7fb771841640 1 --2- 172.21.15.120:0/930188163 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5634e09adc40 0x5634e0a309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:13.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.200+0000 7fb7557fa640 10 client.16101 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:05:13.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.200+0000 7fb771841640 1 --2- 172.21.15.120:0/930188163 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5634e09adc40 0x5634e0a309e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8509855d540177e3 server_cookie=db2f4ae28f00fe4a in_seq=0 out_seq=0 2024-09-19T09:05:13.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.200+0000 7fb7557fa640 1 -- 172.21.15.120:0/930188163 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5634e0915d70 con 0x5634e09adc40 2024-09-19T09:05:13.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.200+0000 7fb7557fa640 10 client.16101 handle_command_reply: tid=0 2024-09-19T09:05:13.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.200+0000 7fb775e7e740 1 client.16101 shutdown 2024-09-19T09:05:13.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.200+0000 7fb775e7e740 1 -- 172.21.15.120:0/930188163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634e09b3370 msgr2=0x5634e09ac190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:13.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.200+0000 7fb775e7e740 1 --2- 172.21.15.120:0/930188163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634e09b3370 0x5634e09ac190 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb764009780 tx=0x7fb764005f90 comp rx=0 tx=0).stop 2024-09-19T09:05:13.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.200+0000 7fb775e7e740 1 -- 172.21.15.120:0/930188163 shutdown_connections 2024-09-19T09:05:13.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.200+0000 7fb775e7e740 1 --2- 172.21.15.120:0/930188163 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5634e09adc40 0x5634e0a309e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.200+0000 7fb775e7e740 1 --2- 172.21.15.120:0/930188163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634e09b3370 0x5634e09ac190 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.200+0000 7fb775e7e740 1 --2- 172.21.15.120:0/930188163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634e09a9b40 0x5634e09bba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.200+0000 7fb775e7e740 1 --2- 172.21.15.120:0/930188163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634e09a85d0 0x5634e09bb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.200+0000 7fb775e7e740 1 -- 172.21.15.120:0/930188163 >> 172.21.15.120:0/930188163 conn(0x5634e0971310 msgr2=0x5634e09af010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:13.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.200+0000 7fb7557fa640 0 client.16101 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:05:13.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.200+0000 7fb775e7e740 1 -- 172.21.15.120:0/930188163 shutdown_connections 2024-09-19T09:05:13.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.200+0000 7fb775e7e740 1 -- 172.21.15.120:0/930188163 wait complete. 2024-09-19T09:05:13.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.200+0000 7fb775e7e740 20 client.16101 trim_cache size 0 max 16384 2024-09-19T09:05:13.211 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:05:13.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.201+0000 7fb773aa5640 1 -- 172.21.15.120:0/2024340679 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb7580a3300 msgr2=0x7fb7580a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:13.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.201+0000 7fb773aa5640 1 --2- 172.21.15.120:0/2024340679 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb7580a3300 0x7fb7580a57c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb7640095f0 tx=0x7fb76402a3d0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.201+0000 7fb773aa5640 1 -- 172.21.15.120:0/2024340679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb76c100e20 msgr2=0x7fb76c198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:13.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.201+0000 7fb773aa5640 1 --2- 172.21.15.120:0/2024340679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb76c100e20 0x7fb76c198490 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb76800c420 tx=0x7fb768009f90 comp rx=0 tx=0).stop 2024-09-19T09:05:13.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.201+0000 7fb773aa5640 1 -- 172.21.15.120:0/2024340679 shutdown_connections 2024-09-19T09:05:13.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.201+0000 7fb773aa5640 1 --2- 172.21.15.120:0/2024340679 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb7580a3300 0x7fb7580a57c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.201+0000 7fb773aa5640 1 --2- 172.21.15.120:0/2024340679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb76c102ef0 0x7fb76c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.201+0000 7fb773aa5640 1 --2- 172.21.15.120:0/2024340679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb76c102570 0x7fb76c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.201+0000 7fb773aa5640 1 --2- 172.21.15.120:0/2024340679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb76c100e20 0x7fb76c198490 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:13.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.201+0000 7fb773aa5640 1 -- 172.21.15.120:0/2024340679 >> 172.21.15.120:0/2024340679 conn(0x7fb76c0fc7d0 msgr2=0x7fb76c103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:13.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.201+0000 7fb773aa5640 1 -- 172.21.15.120:0/2024340679 shutdown_connections 2024-09-19T09:05:13.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:13.201+0000 7fb773aa5640 1 -- 172.21.15.120:0/2024340679 wait complete. 2024-09-19T09:05:14.038 INFO:tasks.workunit.client.0.smithi120.stdout: 1 82416 6.68 MB/sec warmup 69 sec latency 12.054 ms 2024-09-19T09:05:14.043 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:05:14.305 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:05:14.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.376+0000 7fe7021bd640 1 -- 172.21.15.120:0/2463603912 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6fc1025f0 msgr2=0x7fe6fc108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:14.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.376+0000 7fe7021bd640 1 --2- 172.21.15.120:0/2463603912 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6fc1025f0 0x7fe6fc108e80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009be0 tx=0x7fe6f402a6c0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.376+0000 7fe7021bd640 1 -- 172.21.15.120:0/2463603912 shutdown_connections 2024-09-19T09:05:14.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.376+0000 7fe7021bd640 1 --2- 172.21.15.120:0/2463603912 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6fc1025f0 0x7fe6fc108e80 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.376+0000 7fe7021bd640 1 --2- 172.21.15.120:0/2463603912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6fc101c30 0x7fe6fc1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.376+0000 7fe7021bd640 1 --2- 172.21.15.120:0/2463603912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6fc100a40 0x7fe6fc100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.376+0000 7fe7021bd640 1 -- 172.21.15.120:0/2463603912 >> 172.21.15.120:0/2463603912 conn(0x7fe6fc0fc1e0 msgr2=0x7fe6fc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.376+0000 7fe7021bd640 1 -- 172.21.15.120:0/2463603912 shutdown_connections 2024-09-19T09:05:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.376+0000 7fe7021bd640 1 -- 172.21.15.120:0/2463603912 wait complete. 2024-09-19T09:05:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.377+0000 7fe7021bd640 1 Processor -- start 2024-09-19T09:05:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.377+0000 7fe7021bd640 1 -- start start 2024-09-19T09:05:14.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.377+0000 7fe7021bd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6fc100a40 0x7fe6fc06a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.377+0000 7fe7021bd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6fc101c30 0x7fe6fc19ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.377+0000 7fe7021bd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6fc1025f0 0x7fe6fc19f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.377+0000 7fe7021bd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc10c650 con 0x7fe6fc100a40 2024-09-19T09:05:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.377+0000 7fe7021bd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc10c4d0 con 0x7fe6fc101c30 2024-09-19T09:05:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.377+0000 7fe7021bd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc10c7d0 con 0x7fe6fc1025f0 2024-09-19T09:05:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.377+0000 7fe6fb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6fc100a40 0x7fe6fc06a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.377+0000 7fe6fb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6fc100a40 0x7fe6fc06a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34152/0 (socket says 172.21.15.120:34152) 2024-09-19T09:05:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.377+0000 7fe6fb7fe640 1 -- 172.21.15.120:0/1284236310 learned_addr learned my addr 172.21.15.120:0/1284236310 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.377+0000 7fe6faffd640 1 --2- 172.21.15.120:0/1284236310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6fc101c30 0x7fe6fc19ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.377+0000 7fe6fbfff640 1 --2- 172.21.15.120:0/1284236310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6fc1025f0 0x7fe6fc19f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.377+0000 7fe6fb7fe640 1 -- 172.21.15.120:0/1284236310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6fc1025f0 msgr2=0x7fe6fc19f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:14.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.377+0000 7fe6fb7fe640 1 --2- 172.21.15.120:0/1284236310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6fc1025f0 0x7fe6fc19f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.377+0000 7fe6fb7fe640 1 -- 172.21.15.120:0/1284236310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6fc101c30 msgr2=0x7fe6fc19ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:14.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.377+0000 7fe6fb7fe640 1 --2- 172.21.15.120:0/1284236310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6fc101c30 0x7fe6fc19ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.378+0000 7fe6fb7fe640 1 -- 172.21.15.120:0/1284236310 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6fc19f9c0 con 0x7fe6fc100a40 2024-09-19T09:05:14.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.378+0000 7fe6faffd640 1 --2- 172.21.15.120:0/1284236310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6fc101c30 0x7fe6fc19ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:14.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.378+0000 7fe6fb7fe640 1 --2- 172.21.15.120:0/1284236310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6fc100a40 0x7fe6fc06a410 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00e9a0 tx=0x7fe6ec00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:14.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.378+0000 7fe6f8ff9640 1 -- 172.21.15.120:0/1284236310 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec00cd80 con 0x7fe6fc100a40 2024-09-19T09:05:14.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.378+0000 7fe6f8ff9640 1 -- 172.21.15.120:0/1284236310 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6ec010910 con 0x7fe6fc100a40 2024-09-19T09:05:14.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.378+0000 7fe7021bd640 1 -- 172.21.15.120:0/1284236310 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6fc19fc50 con 0x7fe6fc100a40 2024-09-19T09:05:14.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.378+0000 7fe6f8ff9640 1 -- 172.21.15.120:0/1284236310 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec018a30 con 0x7fe6fc100a40 2024-09-19T09:05:14.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.378+0000 7fe7021bd640 1 -- 172.21.15.120:0/1284236310 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6fc1a01a0 con 0x7fe6fc100a40 2024-09-19T09:05:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.380+0000 7fe6f8ff9640 1 -- 172.21.15.120:0/1284236310 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe6ec010ac0 con 0x7fe6fc100a40 2024-09-19T09:05:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.380+0000 7fe705035740 1 Processor -- start 2024-09-19T09:05:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.381+0000 7fe705035740 1 -- start start 2024-09-19T09:05:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.381+0000 7fe705035740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af3f8f8db0 0x55af3f8f91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.381+0000 7fe705035740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af3f8f96f0 0x55af3f8fe1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.381+0000 7fe705035740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af3f8fe780 0x55af3f900c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.381+0000 7fe705035740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55af3f8582f0 con 0x55af3f8f96f0 2024-09-19T09:05:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.381+0000 7fe705035740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55af3f800270 con 0x55af3f8f8db0 2024-09-19T09:05:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.381+0000 7fe705035740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55af3f8542b0 con 0x55af3f8fe780 2024-09-19T09:05:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.381+0000 7fe6fb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af3f8f96f0 0x55af3f8fe1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.381+0000 7fe6fbfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af3f8f8db0 0x55af3f8f91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.381+0000 7fe6fbfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af3f8f8db0 0x55af3f8f91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38238/0 (socket says 172.21.15.120:38238) 2024-09-19T09:05:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.381+0000 7fe6fb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af3f8f96f0 0x55af3f8fe1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34166/0 (socket says 172.21.15.120:34166) 2024-09-19T09:05:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.381+0000 7fe6faffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af3f8fe780 0x55af3f900c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.381+0000 7fe6fbfff640 1 -- 172.21.15.120:0/2926217741 learned_addr learned my addr 172.21.15.120:0/2926217741 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.381+0000 7fe6fbfff640 1 -- 172.21.15.120:0/2926217741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af3f8fe780 msgr2=0x55af3f900c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.381+0000 7fe6fbfff640 1 --2- 172.21.15.120:0/2926217741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af3f8fe780 0x55af3f900c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.381+0000 7fe6fbfff640 1 -- 172.21.15.120:0/2926217741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af3f8f96f0 msgr2=0x55af3f8fe1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.381+0000 7fe6fbfff640 1 --2- 172.21.15.120:0/2926217741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af3f8f96f0 0x55af3f8fe1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.381+0000 7fe6f8ff9640 1 --2- 172.21.15.120:0/1284236310 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe6e40a32f0 0x7fe6e40a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.381+0000 7fe6fbfff640 1 -- 172.21.15.120:0/2926217741 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af3f7ff340 con 0x55af3f8f8db0 2024-09-19T09:05:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.381+0000 7fe6faffd640 1 --2- 172.21.15.120:0/2926217741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af3f8fe780 0x55af3f900c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.381+0000 7fe6f8ff9640 1 -- 172.21.15.120:0/1284236310 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe6ec0cfc80 con 0x7fe6fc100a40 2024-09-19T09:05:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.381+0000 7fe6faffd640 1 --2- 172.21.15.120:0/1284236310 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe6e40a32f0 0x7fe6e40a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.381+0000 7fe6fbfff640 1 --2- 172.21.15.120:0/2926217741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af3f8f8db0 0x55af3f8f91b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009be0 tx=0x7fe6f4005ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5dff4230cae390c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.382+0000 7fe6e1ffb640 1 -- 172.21.15.120:0/2926217741 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4042030 con 0x55af3f8f8db0 2024-09-19T09:05:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.382+0000 7fe6e1ffb640 1 -- 172.21.15.120:0/2926217741 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6f4037040 con 0x55af3f8f8db0 2024-09-19T09:05:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.382+0000 7fe6e1ffb640 1 -- 172.21.15.120:0/2926217741 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4036070 con 0x55af3f8f8db0 2024-09-19T09:05:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.382+0000 7fe705035740 1 -- 172.21.15.120:0/2926217741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af3f8f8db0 msgr2=0x55af3f8f91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.382+0000 7fe6faffd640 1 --2- 172.21.15.120:0/1284236310 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe6e40a32f0 0x7fe6e40a57b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0009fd0 tx=0x7fe6f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.382+0000 7fe705035740 1 --2- 172.21.15.120:0/2926217741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af3f8f8db0 0x55af3f8f91b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009be0 tx=0x7fe6f4005ec0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.382+0000 7fe705035740 1 -- 172.21.15.120:0/2926217741 shutdown_connections 2024-09-19T09:05:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.382+0000 7fe705035740 1 --2- 172.21.15.120:0/2926217741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af3f8fe780 0x55af3f900c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.382+0000 7fe705035740 1 --2- 172.21.15.120:0/2926217741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af3f8f96f0 0x55af3f8fe1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.382+0000 7fe705035740 1 --2- 172.21.15.120:0/2926217741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af3f8f8db0 0x55af3f8f91b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.382+0000 7fe705035740 1 -- 172.21.15.120:0/2926217741 >> 172.21.15.120:0/2926217741 conn(0x55af3f8f0010 msgr2=0x55af3f8f0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.382+0000 7fe705035740 1 -- 172.21.15.120:0/2926217741 shutdown_connections 2024-09-19T09:05:14.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.382+0000 7fe705035740 1 -- 172.21.15.120:0/2926217741 wait complete. 2024-09-19T09:05:14.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.383+0000 7fe705035740 1 -- start start 2024-09-19T09:05:14.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.383+0000 7fe705035740 1 Processor -- start 2024-09-19T09:05:14.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.383+0000 7fe705035740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af3f8f8db0 0x55af3f902f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.383+0000 7fe705035740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af3f8f96f0 0x55af3f903460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.383+0000 7fe705035740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af3f8fe780 0x55af3f90a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.383+0000 7fe705035740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55af3f9016f0 con 0x55af3f8f96f0 2024-09-19T09:05:14.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.383+0000 7fe705035740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55af3f7e1290 con 0x55af3f8f8db0 2024-09-19T09:05:14.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.383+0000 7fe705035740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55af3f901870 con 0x55af3f8fe780 2024-09-19T09:05:14.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.383+0000 7fe705035740 10 client.0 _ll_register_callbacks cb 0x55af3f812140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:14.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.383+0000 7fe6faffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af3f8f96f0 0x55af3f903460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.383+0000 7fe6fb7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af3f8f8db0 0x55af3f902f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.383+0000 7fe6faffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af3f8f96f0 0x55af3f903460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34180/0 (socket says 172.21.15.120:34180) 2024-09-19T09:05:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.383+0000 7fe6fb7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af3f8f8db0 0x55af3f902f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38248/0 (socket says 172.21.15.120:38248) 2024-09-19T09:05:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.383+0000 7fe6faffd640 1 -- 172.21.15.120:0/3676255862 learned_addr learned my addr 172.21.15.120:0/3676255862 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.383+0000 7fe6fbfff640 1 --2- 172.21.15.120:0/3676255862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af3f8fe780 0x55af3f90a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.384+0000 7fe6faffd640 1 -- 172.21.15.120:0/3676255862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af3f8fe780 msgr2=0x55af3f90a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.384+0000 7fe6faffd640 1 --2- 172.21.15.120:0/3676255862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af3f8fe780 0x55af3f90a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.384+0000 7fe6faffd640 1 -- 172.21.15.120:0/3676255862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af3f8f8db0 msgr2=0x55af3f902f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.384+0000 7fe6faffd640 1 --2- 172.21.15.120:0/3676255862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af3f8f8db0 0x55af3f902f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.384+0000 7fe6faffd640 1 -- 172.21.15.120:0/3676255862 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af3f56b210 con 0x55af3f8f96f0 2024-09-19T09:05:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.384+0000 7fe6fb7fe640 1 --2- 172.21.15.120:0/3676255862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af3f8f8db0 0x55af3f902f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.384+0000 7fe6faffd640 1 --2- 172.21.15.120:0/3676255862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af3f8f96f0 0x55af3f903460 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000f850 tx=0x7fe6f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ef18b0d8c11903 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.384+0000 7fe6daffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.384+0000 7fe6daffd640 1 -- 172.21.15.120:0/3676255862 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0004260 con 0x55af3f8f96f0 2024-09-19T09:05:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.384+0000 7fe6daffd640 1 -- 172.21.15.120:0/3676255862 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe6f0004400 con 0x55af3f8f96f0 2024-09-19T09:05:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.384+0000 7fe705035740 1 -- 172.21.15.120:0/3676255862 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6f4037040 con 0x55af3f8f96f0 2024-09-19T09:05:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.384+0000 7fe6daffd640 1 -- 172.21.15.120:0/3676255862 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe6f0011940 con 0x55af3f8f96f0 2024-09-19T09:05:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.384+0000 7fe705035740 10 client.16113 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.384+0000 7fe705035740 1 -- 172.21.15.120:0/3676255862 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55af3f637620 con 0x55af3f8f96f0 2024-09-19T09:05:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.384+0000 7fe6daffd640 1 -- 172.21.15.120:0/3676255862 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6f4037040 con 0x55af3f8f96f0 2024-09-19T09:05:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.385+0000 7fe6daffd640 1 -- 172.21.15.120:0/3676255862 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe6f0027020 con 0x55af3f8f96f0 2024-09-19T09:05:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.385+0000 7fe705035740 10 client.16113 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.385+0000 7fe705035740 10 client.16113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.385+0000 7fe705035740 20 client.16113 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.385+0000 7fe705035740 1 --2- 172.21.15.120:0/3676255862 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55af3f90ec40 0x55af3f911050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.385+0000 7fe705035740 4 client.16113 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:05:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.385+0000 7fe705035740 1 -- 172.21.15.120:0/3676255862 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af3f865a20 con 0x55af3f90ec40 2024-09-19T09:05:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.385+0000 7fe6fb7fe640 1 --2- 172.21.15.120:0/3676255862 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55af3f90ec40 0x55af3f911050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.385+0000 7fe6fb7fe640 1 --2- 172.21.15.120:0/3676255862 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55af3f90ec40 0x55af3f911050 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17d54cce09f5f7a5 server_cookie=3229bc6cdb7cf543 in_seq=0 out_seq=0 2024-09-19T09:05:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.385+0000 7fe6daffd640 10 client.16113 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.387+0000 7fe6daffd640 1 -- 172.21.15.120:0/3676255862 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af3f865a20 con 0x55af3f90ec40 2024-09-19T09:05:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.387+0000 7fe6daffd640 10 client.16113 handle_command_reply: tid=0 2024-09-19T09:05:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.387+0000 7fe705035740 1 client.16113 shutdown 2024-09-19T09:05:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.387+0000 7fe705035740 1 -- 172.21.15.120:0/3676255862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af3f8f96f0 msgr2=0x55af3f903460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.387+0000 7fe705035740 1 --2- 172.21.15.120:0/3676255862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af3f8f96f0 0x55af3f903460 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000f850 tx=0x7fe6f0004730 comp rx=0 tx=0).stop 2024-09-19T09:05:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.387+0000 7fe705035740 1 -- 172.21.15.120:0/3676255862 shutdown_connections 2024-09-19T09:05:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.387+0000 7fe705035740 1 --2- 172.21.15.120:0/3676255862 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55af3f90ec40 0x55af3f911050 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.387+0000 7fe705035740 1 --2- 172.21.15.120:0/3676255862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af3f8fe780 0x55af3f90a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.387+0000 7fe705035740 1 --2- 172.21.15.120:0/3676255862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af3f8f96f0 0x55af3f903460 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.387+0000 7fe6daffd640 0 client.16113 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.387+0000 7fe705035740 1 --2- 172.21.15.120:0/3676255862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af3f8f8db0 0x55af3f902f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.387+0000 7fe705035740 1 -- 172.21.15.120:0/3676255862 >> 172.21.15.120:0/3676255862 conn(0x55af3f8f0010 msgr2=0x55af3f8f1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.387+0000 7fe705035740 1 -- 172.21.15.120:0/3676255862 shutdown_connections 2024-09-19T09:05:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.387+0000 7fe705035740 1 -- 172.21.15.120:0/3676255862 wait complete. 2024-09-19T09:05:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.387+0000 7fe705035740 20 client.16113 trim_cache size 0 max 16384 2024-09-19T09:05:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.401+0000 7fe705035740 1 Processor -- start 2024-09-19T09:05:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.401+0000 7fe705035740 1 -- start start 2024-09-19T09:05:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.402+0000 7fe705035740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af3f8f8db0 0x55af3f8f03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.402+0000 7fe705035740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af3f8f96f0 0x55af3f90a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.402+0000 7fe705035740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af3f8fe780 0x55af3f8f0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.402+0000 7fe705035740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc1a6cf0 con 0x55af3f8fe780 2024-09-19T09:05:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.402+0000 7fe705035740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc1a8830 con 0x55af3f8f96f0 2024-09-19T09:05:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.402+0000 7fe705035740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc1a73c0 con 0x55af3f8f8db0 2024-09-19T09:05:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.402+0000 7fe6fbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af3f8fe780 0x55af3f8f0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.402+0000 7fe6fbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af3f8fe780 0x55af3f8f0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34196/0 (socket says 172.21.15.120:34196) 2024-09-19T09:05:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.402+0000 7fe6fbfff640 1 -- 172.21.15.120:0/119117312 learned_addr learned my addr 172.21.15.120:0/119117312 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.402+0000 7fe6faffd640 1 --2- 172.21.15.120:0/119117312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af3f8f96f0 0x55af3f90a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.402+0000 7fe6fb7fe640 1 --2- 172.21.15.120:0/119117312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af3f8f8db0 0x55af3f8f03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.402+0000 7fe6fbfff640 1 -- 172.21.15.120:0/119117312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af3f8f8db0 msgr2=0x55af3f8f03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.402+0000 7fe6fbfff640 1 --2- 172.21.15.120:0/119117312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af3f8f8db0 0x55af3f8f03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.402+0000 7fe6fbfff640 1 -- 172.21.15.120:0/119117312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af3f8f96f0 msgr2=0x55af3f90a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.402+0000 7fe6fbfff640 1 --2- 172.21.15.120:0/119117312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af3f8f96f0 0x55af3f90a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.402+0000 7fe6fbfff640 1 -- 172.21.15.120:0/119117312 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af3f8fa380 con 0x55af3f8fe780 2024-09-19T09:05:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.402+0000 7fe6fbfff640 1 --2- 172.21.15.120:0/119117312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af3f8fe780 0x55af3f8f0930 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4004ad0 tx=0x7fe6f402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41090a2decd382f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.402+0000 7fe6e17fa640 1 -- 172.21.15.120:0/119117312 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4042040 con 0x55af3f8fe780 2024-09-19T09:05:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.402+0000 7fe6e17fa640 1 -- 172.21.15.120:0/119117312 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6f404b070 con 0x55af3f8fe780 2024-09-19T09:05:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.402+0000 7fe6e17fa640 1 -- 172.21.15.120:0/119117312 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4036040 con 0x55af3f8fe780 2024-09-19T09:05:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.403+0000 7fe705035740 1 -- 172.21.15.120:0/119117312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af3f8fe780 msgr2=0x55af3f8f0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.403+0000 7fe705035740 1 --2- 172.21.15.120:0/119117312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af3f8fe780 0x55af3f8f0930 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4004ad0 tx=0x7fe6f402a6c0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.403+0000 7fe705035740 1 -- 172.21.15.120:0/119117312 shutdown_connections 2024-09-19T09:05:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.403+0000 7fe705035740 1 --2- 172.21.15.120:0/119117312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af3f8fe780 0x55af3f8f0930 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.403+0000 7fe705035740 1 --2- 172.21.15.120:0/119117312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af3f8f96f0 0x55af3f90a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.403+0000 7fe705035740 1 --2- 172.21.15.120:0/119117312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af3f8f8db0 0x55af3f8f03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.403+0000 7fe705035740 1 -- 172.21.15.120:0/119117312 >> 172.21.15.120:0/119117312 conn(0x55af3f8f0010 msgr2=0x55af3f8fa890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.403+0000 7fe705035740 1 -- 172.21.15.120:0/119117312 shutdown_connections 2024-09-19T09:05:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.403+0000 7fe705035740 1 -- 172.21.15.120:0/119117312 wait complete. 2024-09-19T09:05:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.403+0000 7fe705035740 1 -- start start 2024-09-19T09:05:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.404+0000 7fe705035740 1 Processor -- start 2024-09-19T09:05:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.404+0000 7fe705035740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af3f8f8db0 0x55af3f90a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.404+0000 7fe705035740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af3f8f96f0 0x55af3f906e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.404+0000 7fe705035740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af3f8fe780 0x55af3f9073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.404+0000 7fe705035740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc1a8160 con 0x55af3f8f96f0 2024-09-19T09:05:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.404+0000 7fe705035740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc1a7a90 con 0x55af3f8f8db0 2024-09-19T09:05:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.404+0000 7fe705035740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc1a4cb0 con 0x55af3f8fe780 2024-09-19T09:05:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.404+0000 7fe705035740 10 client.0 _ll_register_callbacks cb 0x7fe6f0009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.404+0000 7fe6faffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af3f8f96f0 0x55af3f906e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.404+0000 7fe6faffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af3f8f96f0 0x55af3f906e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34206/0 (socket says 172.21.15.120:34206) 2024-09-19T09:05:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.404+0000 7fe6faffd640 1 -- 172.21.15.120:0/862334209 learned_addr learned my addr 172.21.15.120:0/862334209 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.404+0000 7fe6fb7fe640 1 --2- 172.21.15.120:0/862334209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af3f8f8db0 0x55af3f90a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.404+0000 7fe6fbfff640 1 --2- 172.21.15.120:0/862334209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af3f8fe780 0x55af3f9073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.404+0000 7fe6faffd640 1 -- 172.21.15.120:0/862334209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af3f8fe780 msgr2=0x55af3f9073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.404+0000 7fe6faffd640 1 --2- 172.21.15.120:0/862334209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af3f8fe780 0x55af3f9073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.404+0000 7fe6faffd640 1 -- 172.21.15.120:0/862334209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af3f8f8db0 msgr2=0x55af3f90a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.404+0000 7fe6faffd640 1 --2- 172.21.15.120:0/862334209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af3f8f8db0 0x55af3f90a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.404+0000 7fe6faffd640 1 -- 172.21.15.120:0/862334209 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af3f977920 con 0x55af3f8f96f0 2024-09-19T09:05:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.404+0000 7fe6faffd640 1 --2- 172.21.15.120:0/862334209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af3f8f96f0 0x55af3f906e60 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x55af3f904400 tx=0x7fe6f0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1805a06dd2e0c3a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.404+0000 7fe6daffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.405+0000 7fe6daffd640 1 -- 172.21.15.120:0/862334209 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0011ef0 con 0x55af3f8f96f0 2024-09-19T09:05:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.405+0000 7fe705035740 1 -- 172.21.15.120:0/862334209 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6f404b070 con 0x55af3f8f96f0 2024-09-19T09:05:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.405+0000 7fe6daffd640 1 -- 172.21.15.120:0/862334209 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe6f00035f0 con 0x55af3f8f96f0 2024-09-19T09:05:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.405+0000 7fe6daffd640 1 -- 172.21.15.120:0/862334209 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe6f0044050 con 0x55af3f8f96f0 2024-09-19T09:05:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.405+0000 7fe705035740 10 client.16119 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.405+0000 7fe705035740 1 -- 172.21.15.120:0/862334209 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55af3f978c40 con 0x55af3f8f96f0 2024-09-19T09:05:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.405+0000 7fe6daffd640 1 -- 172.21.15.120:0/862334209 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6f404b070 con 0x55af3f8f96f0 2024-09-19T09:05:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.405+0000 7fe6daffd640 1 -- 172.21.15.120:0/862334209 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe6f0027040 con 0x55af3f8f96f0 2024-09-19T09:05:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.405+0000 7fe705035740 10 client.16119 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.405+0000 7fe705035740 10 client.16119 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.405+0000 7fe705035740 20 client.16119 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.405+0000 7fe705035740 1 --2- 172.21.15.120:0/862334209 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55af3f979620 0x55af3f97ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.405+0000 7fe705035740 4 client.16119 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:05:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.405+0000 7fe705035740 1 -- 172.21.15.120:0/862334209 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55af3f85fd70 con 0x55af3f979620 2024-09-19T09:05:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.405+0000 7fe6fb7fe640 1 --2- 172.21.15.120:0/862334209 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55af3f979620 0x55af3f97ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.406+0000 7fe6daffd640 10 client.16119 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.406+0000 7fe6fb7fe640 1 --2- 172.21.15.120:0/862334209 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55af3f979620 0x55af3f97ba40 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d211ae11c53ebb0 server_cookie=6883e9f636dc368f in_seq=0 out_seq=0 2024-09-19T09:05:14.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.520+0000 7fe6daffd640 1 -- 172.21.15.120:0/862334209 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55af3f85fd70 con 0x55af3f979620 2024-09-19T09:05:14.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.520+0000 7fe6daffd640 10 client.16119 handle_command_reply: tid=0 2024-09-19T09:05:14.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.520+0000 7fe705035740 1 client.16119 shutdown 2024-09-19T09:05:14.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.520+0000 7fe705035740 1 -- 172.21.15.120:0/862334209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af3f8f96f0 msgr2=0x55af3f906e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:14.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.520+0000 7fe705035740 1 --2- 172.21.15.120:0/862334209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af3f8f96f0 0x55af3f906e60 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x55af3f904400 tx=0x7fe6f0011810 comp rx=0 tx=0).stop 2024-09-19T09:05:14.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.520+0000 7fe705035740 1 -- 172.21.15.120:0/862334209 shutdown_connections 2024-09-19T09:05:14.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.520+0000 7fe705035740 1 --2- 172.21.15.120:0/862334209 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55af3f979620 0x55af3f97ba40 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.520+0000 7fe705035740 1 --2- 172.21.15.120:0/862334209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af3f8fe780 0x55af3f9073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.520+0000 7fe6daffd640 0 client.16119 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:14.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.520+0000 7fe705035740 1 --2- 172.21.15.120:0/862334209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af3f8f96f0 0x55af3f906e60 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.520+0000 7fe705035740 1 --2- 172.21.15.120:0/862334209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af3f8f8db0 0x55af3f90a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.520+0000 7fe705035740 1 -- 172.21.15.120:0/862334209 >> 172.21.15.120:0/862334209 conn(0x55af3f8f0010 msgr2=0x55af3f8ee9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:14.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.520+0000 7fe705035740 1 -- 172.21.15.120:0/862334209 shutdown_connections 2024-09-19T09:05:14.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.520+0000 7fe705035740 1 -- 172.21.15.120:0/862334209 wait complete. 2024-09-19T09:05:14.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.520+0000 7fe705035740 20 client.16119 trim_cache size 0 max 16384 2024-09-19T09:05:14.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:05:14.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:05:14.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:05:14.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:05:14.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.521+0000 7fe7021bd640 1 -- 172.21.15.120:0/1284236310 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe6e40a32f0 msgr2=0x7fe6e40a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:14.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.521+0000 7fe7021bd640 1 --2- 172.21.15.120:0/1284236310 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe6e40a32f0 0x7fe6e40a57b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0009fd0 tx=0x7fe6f0005f90 comp rx=0 tx=0).stop 2024-09-19T09:05:14.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.521+0000 7fe7021bd640 1 -- 172.21.15.120:0/1284236310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6fc100a40 msgr2=0x7fe6fc06a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.521+0000 7fe7021bd640 1 --2- 172.21.15.120:0/1284236310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6fc100a40 0x7fe6fc06a410 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00e9a0 tx=0x7fe6ec00ee60 comp rx=0 tx=0).stop 2024-09-19T09:05:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.522+0000 7fe7021bd640 1 -- 172.21.15.120:0/1284236310 shutdown_connections 2024-09-19T09:05:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.522+0000 7fe7021bd640 1 --2- 172.21.15.120:0/1284236310 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe6e40a32f0 0x7fe6e40a57b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.522+0000 7fe7021bd640 1 --2- 172.21.15.120:0/1284236310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6fc1025f0 0x7fe6fc19f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.522+0000 7fe7021bd640 1 --2- 172.21.15.120:0/1284236310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6fc101c30 0x7fe6fc19ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.522+0000 7fe7021bd640 1 --2- 172.21.15.120:0/1284236310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6fc100a40 0x7fe6fc06a410 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.522+0000 7fe7021bd640 1 -- 172.21.15.120:0/1284236310 >> 172.21.15.120:0/1284236310 conn(0x7fe6fc0fc1e0 msgr2=0x7fe6fc104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:14.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.526+0000 7fe7021bd640 1 -- 172.21.15.120:0/1284236310 shutdown_connections 2024-09-19T09:05:14.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:14.526+0000 7fe7021bd640 1 -- 172.21.15.120:0/1284236310 wait complete. 2024-09-19T09:05:14.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.778+0000 7f3d89244640 1 -- 172.21.15.120:0/2551217898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d84102790 msgr2=0x7f3d841067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:14.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.778+0000 7f3d89244640 1 --2- 172.21.15.120:0/2551217898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d84102790 0x7f3d841067e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3d78009900 tx=0x7f3d7802a410 comp rx=0 tx=0).stop 2024-09-19T09:05:14.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.778+0000 7f3d89244640 1 -- 172.21.15.120:0/2551217898 shutdown_connections 2024-09-19T09:05:14.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.778+0000 7f3d89244640 1 --2- 172.21.15.120:0/2551217898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d84106f10 0x7f3d8410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.778+0000 7f3d89244640 1 --2- 172.21.15.120:0/2551217898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d84102790 0x7f3d841067e0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.778+0000 7f3d89244640 1 --2- 172.21.15.120:0/2551217898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d84101de0 0x7f3d841021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.778+0000 7f3d89244640 1 -- 172.21.15.120:0/2551217898 >> 172.21.15.120:0/2551217898 conn(0x7f3d840fd8e0 msgr2=0x7f3d840ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:14.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.778+0000 7f3d89244640 1 -- 172.21.15.120:0/2551217898 shutdown_connections 2024-09-19T09:05:14.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.778+0000 7f3d89244640 1 -- 172.21.15.120:0/2551217898 wait complete. 2024-09-19T09:05:14.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.778+0000 7f3d89244640 1 Processor -- start 2024-09-19T09:05:14.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.779+0000 7f3d89244640 1 -- start start 2024-09-19T09:05:14.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.779+0000 7f3d89244640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d84101de0 0x7f3d841984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.779+0000 7f3d89244640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d84102790 0x7f3d84198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.779+0000 7f3d89244640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d84106f10 0x7f3d8419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.779+0000 7f3d89244640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3d84111f70 con 0x7f3d84106f10 2024-09-19T09:05:14.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.779+0000 7f3d89244640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3d84111df0 con 0x7f3d84101de0 2024-09-19T09:05:14.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.779+0000 7f3d89244640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3d841120f0 con 0x7f3d84102790 2024-09-19T09:05:14.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.779+0000 7f3d8259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d84102790 0x7f3d84198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.779+0000 7f3d8359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d84106f10 0x7f3d8419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.779+0000 7f3d82d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d84101de0 0x7f3d841984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.779+0000 7f3d8259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d84102790 0x7f3d84198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:58752/0 (socket says 172.21.15.120:58752) 2024-09-19T09:05:14.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.779+0000 7f3d8259c640 1 -- 172.21.15.120:0/541106172 learned_addr learned my addr 172.21.15.120:0/541106172 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:14.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.779+0000 7f3d8359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d84106f10 0x7f3d8419cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34232/0 (socket says 172.21.15.120:34232) 2024-09-19T09:05:14.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.779+0000 7f3d82d9d640 1 -- 172.21.15.120:0/541106172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d84102790 msgr2=0x7f3d84198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:14.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.779+0000 7f3d82d9d640 1 --2- 172.21.15.120:0/541106172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d84102790 0x7f3d84198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.779+0000 7f3d82d9d640 1 -- 172.21.15.120:0/541106172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d84106f10 msgr2=0x7f3d8419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:14.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.779+0000 7f3d82d9d640 1 --2- 172.21.15.120:0/541106172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d84106f10 0x7f3d8419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.779+0000 7f3d82d9d640 1 -- 172.21.15.120:0/541106172 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8419d480 con 0x7f3d84101de0 2024-09-19T09:05:14.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.780+0000 7f3d82d9d640 1 --2- 172.21.15.120:0/541106172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d84101de0 0x7f3d841984d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400e9f0 tx=0x7f3d7400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:14.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.780+0000 7f3d6bfff640 1 -- 172.21.15.120:0/541106172 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7400ccc0 con 0x7f3d84101de0 2024-09-19T09:05:14.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.780+0000 7f3d89244640 1 -- 172.21.15.120:0/541106172 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d8419d770 con 0x7f3d84101de0 2024-09-19T09:05:14.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.780+0000 7f3d6bfff640 1 -- 172.21.15.120:0/541106172 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d74010910 con 0x7f3d84101de0 2024-09-19T09:05:14.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.780+0000 7f3d6bfff640 1 -- 172.21.15.120:0/541106172 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d74018960 con 0x7f3d84101de0 2024-09-19T09:05:14.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.780+0000 7f3d89244640 1 -- 172.21.15.120:0/541106172 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d841a5050 con 0x7f3d84101de0 2024-09-19T09:05:14.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.782+0000 7f3d8b61d740 1 Processor -- start 2024-09-19T09:05:14.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.782+0000 7f3d8b61d740 1 -- start start 2024-09-19T09:05:14.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.782+0000 7f3d6bfff640 1 -- 172.21.15.120:0/541106172 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f3d74010ac0 con 0x7f3d84101de0 2024-09-19T09:05:14.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.782+0000 7f3d8b61d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56330cb8eb40 0x56330cb8d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.782+0000 7f3d8b61d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56330cb8d5d0 0x56330cb8da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.782+0000 7f3d8b61d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56330cb9c780 0x56330cb9eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.782+0000 7f3d8b61d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56330caf32f0 con 0x56330cb8d5d0 2024-09-19T09:05:14.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.782+0000 7f3d8b61d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56330ca9acf0 con 0x56330cb8eb40 2024-09-19T09:05:14.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.782+0000 7f3d8b61d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56330caef2b0 con 0x56330cb9c780 2024-09-19T09:05:14.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.783+0000 7f3d82d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56330cb8d5d0 0x56330cb8da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.783+0000 7f3d8359e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56330cb8eb40 0x56330cb8d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.783+0000 7f3d8359e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56330cb8eb40 0x56330cb8d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38300/0 (socket says 172.21.15.120:38300) 2024-09-19T09:05:14.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.783+0000 7f3d82d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56330cb8d5d0 0x56330cb8da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34234/0 (socket says 172.21.15.120:34234) 2024-09-19T09:05:14.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.783+0000 7f3d8359e640 1 -- 172.21.15.120:0/816014053 learned_addr learned my addr 172.21.15.120:0/816014053 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:14.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.783+0000 7f3d8259c640 1 --2- 172.21.15.120:0/816014053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56330cb9c780 0x56330cb9eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.783+0000 7f3d8359e640 1 -- 172.21.15.120:0/816014053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56330cb9c780 msgr2=0x56330cb9eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:14.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.783+0000 7f3d8359e640 1 --2- 172.21.15.120:0/816014053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56330cb9c780 0x56330cb9eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.783+0000 7f3d8359e640 1 -- 172.21.15.120:0/816014053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56330cb8d5d0 msgr2=0x56330cb8da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:14.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.783+0000 7f3d8359e640 1 --2- 172.21.15.120:0/816014053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56330cb8d5d0 0x56330cb8da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.783+0000 7f3d8359e640 1 -- 172.21.15.120:0/816014053 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56330ca7f3a0 con 0x56330cb8eb40 2024-09-19T09:05:14.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.783+0000 7f3d82d9d640 1 --2- 172.21.15.120:0/816014053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56330cb8d5d0 0x56330cb8da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:14.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.783+0000 7f3d6bfff640 1 --2- 172.21.15.120:0/541106172 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3d6c0a3240 0x7f3d6c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.783+0000 7f3d82d9d640 1 --2- 172.21.15.120:0/541106172 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3d6c0a3240 0x7f3d6c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.783+0000 7f3d8359e640 1 --2- 172.21.15.120:0/816014053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56330cb8eb40 0x56330cb8d090 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c009f90 tx=0x7f3d7c010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e2075d1f1ad3751 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:14.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.783+0000 7f3d6bfff640 1 -- 172.21.15.120:0/541106172 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f3d740cfca0 con 0x7f3d84101de0 2024-09-19T09:05:14.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.783+0000 7f3d697fa640 1 -- 172.21.15.120:0/816014053 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c011700 con 0x56330cb8eb40 2024-09-19T09:05:14.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.783+0000 7f3d697fa640 1 -- 172.21.15.120:0/816014053 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d7c011d80 con 0x56330cb8eb40 2024-09-19T09:05:14.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.783+0000 7f3d697fa640 1 -- 172.21.15.120:0/816014053 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c019e40 con 0x56330cb8eb40 2024-09-19T09:05:14.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.784+0000 7f3d8b61d740 1 -- 172.21.15.120:0/816014053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56330cb8eb40 msgr2=0x56330cb8d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:14.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.784+0000 7f3d8b61d740 1 --2- 172.21.15.120:0/816014053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56330cb8eb40 0x56330cb8d090 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c009f90 tx=0x7f3d7c010860 comp rx=0 tx=0).stop 2024-09-19T09:05:14.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.784+0000 7f3d82d9d640 1 --2- 172.21.15.120:0/541106172 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3d6c0a3240 0x7f3d6c0a5700 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3d740a4a00 tx=0x7f3d740d3720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:14.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.784+0000 7f3d8b61d740 1 -- 172.21.15.120:0/816014053 shutdown_connections 2024-09-19T09:05:14.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.784+0000 7f3d8b61d740 1 --2- 172.21.15.120:0/816014053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56330cb9c780 0x56330cb9eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.784+0000 7f3d8b61d740 1 --2- 172.21.15.120:0/816014053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56330cb8d5d0 0x56330cb8da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.784+0000 7f3d8b61d740 1 --2- 172.21.15.120:0/816014053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56330cb8eb40 0x56330cb8d090 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.784+0000 7f3d8b61d740 1 -- 172.21.15.120:0/816014053 >> 172.21.15.120:0/816014053 conn(0x56330cb56310 msgr2=0x56330cb89c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:14.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.784+0000 7f3d8b61d740 1 -- 172.21.15.120:0/816014053 shutdown_connections 2024-09-19T09:05:14.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.784+0000 7f3d8b61d740 1 -- 172.21.15.120:0/816014053 wait complete. 2024-09-19T09:05:14.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.785+0000 7f3d8b61d740 1 -- start start 2024-09-19T09:05:14.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.785+0000 7f3d8b61d740 1 Processor -- start 2024-09-19T09:05:14.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.785+0000 7f3d8b61d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56330cb8d5d0 0x56330cb9c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.785+0000 7f3d8b61d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56330cb8eb40 0x56330cb993b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.785+0000 7f3d8b61d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56330cb9c780 0x56330cb99930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.785+0000 7f3d8b61d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56330cb9f480 con 0x56330cb9c780 2024-09-19T09:05:14.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.785+0000 7f3d8b61d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56330caae060 con 0x56330cb8d5d0 2024-09-19T09:05:14.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.785+0000 7f3d8b61d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56330cb9f600 con 0x56330cb8eb40 2024-09-19T09:05:14.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.785+0000 7f3d8b61d740 10 client.0 _ll_register_callbacks cb 0x56330c8018c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:14.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.785+0000 7f3d8359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56330cb9c780 0x56330cb99930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.785+0000 7f3d8359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56330cb9c780 0x56330cb99930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34246/0 (socket says 172.21.15.120:34246) 2024-09-19T09:05:14.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.785+0000 7f3d8359e640 1 -- 172.21.15.120:0/1357836776 learned_addr learned my addr 172.21.15.120:0/1357836776 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:14.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.785+0000 7f3d8259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56330cb8eb40 0x56330cb993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.785+0000 7f3d8359e640 1 --2- 172.21.15.120:0/1357836776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56330cb8d5d0 0x56330cb9c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.785+0000 7f3d8359e640 1 -- 172.21.15.120:0/1357836776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56330cb8eb40 msgr2=0x56330cb993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:14.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.785+0000 7f3d8359e640 1 --2- 172.21.15.120:0/1357836776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56330cb8eb40 0x56330cb993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.785+0000 7f3d8359e640 1 -- 172.21.15.120:0/1357836776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56330cb8d5d0 msgr2=0x56330cb9c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:14.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.785+0000 7f3d8359e640 1 --2- 172.21.15.120:0/1357836776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56330cb8d5d0 0x56330cb9c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.785+0000 7f3d8359e640 1 -- 172.21.15.120:0/1357836776 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56330c8643c0 con 0x56330cb9c780 2024-09-19T09:05:14.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.785+0000 7f3d8259c640 1 --2- 172.21.15.120:0/1357836776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56330cb8eb40 0x56330cb993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:14.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.786+0000 7f3d8359e640 1 --2- 172.21.15.120:0/1357836776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56330cb9c780 0x56330cb99930 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c009d50 tx=0x7f3d7c013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be0dceef00e4ee39 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:14.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.786+0000 7f3d66ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:14.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.786+0000 7f3d66ff5640 1 -- 172.21.15.120:0/1357836776 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c003970 con 0x56330cb9c780 2024-09-19T09:05:14.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.786+0000 7f3d66ff5640 1 -- 172.21.15.120:0/1357836776 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f3d7c003b10 con 0x56330cb9c780 2024-09-19T09:05:14.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.786+0000 7f3d8b61d740 1 -- 172.21.15.120:0/1357836776 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d7c011d80 con 0x56330cb9c780 2024-09-19T09:05:14.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.786+0000 7f3d66ff5640 1 -- 172.21.15.120:0/1357836776 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3d7c003d10 con 0x56330cb9c780 2024-09-19T09:05:14.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.786+0000 7f3d8b61d740 10 client.16131 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:14.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.786+0000 7f3d8b61d740 1 -- 172.21.15.120:0/1357836776 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56330cbaaeb0 con 0x56330cb9c780 2024-09-19T09:05:14.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.786+0000 7f3d66ff5640 1 -- 172.21.15.120:0/1357836776 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d7c011d80 con 0x56330cb9c780 2024-09-19T09:05:14.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.786+0000 7f3d66ff5640 1 -- 172.21.15.120:0/1357836776 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3d7c032050 con 0x56330cb9c780 2024-09-19T09:05:14.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.786+0000 7f3d8b61d740 10 client.16131 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:14.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.786+0000 7f3d8b61d740 10 client.16131 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:05:14.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.786+0000 7f3d8b61d740 20 client.16131 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:14.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.786+0000 7f3d8b61d740 1 --2- 172.21.15.120:0/1357836776 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56330cbac260 0x56330cbae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.786+0000 7f3d8b61d740 4 client.16131 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:05:14.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.786+0000 7f3d8b61d740 1 -- 172.21.15.120:0/1357836776 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56330cb00860 con 0x56330cbac260 2024-09-19T09:05:14.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.787+0000 7f3d8259c640 1 --2- 172.21.15.120:0/1357836776 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56330cbac260 0x56330cbae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.787+0000 7f3d8259c640 1 --2- 172.21.15.120:0/1357836776 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56330cbac260 0x56330cbae650 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a14d18d394001a48 server_cookie=d498b17896f760e4 in_seq=0 out_seq=0 2024-09-19T09:05:14.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.787+0000 7f3d66ff5640 10 client.16131 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:05:14.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.788+0000 7f3d66ff5640 1 -- 172.21.15.120:0/1357836776 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56330cb00860 con 0x56330cbac260 2024-09-19T09:05:14.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.788+0000 7f3d66ff5640 10 client.16131 handle_command_reply: tid=0 2024-09-19T09:05:14.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.788+0000 7f3d8b61d740 1 client.16131 shutdown 2024-09-19T09:05:14.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.788+0000 7f3d8b61d740 1 -- 172.21.15.120:0/1357836776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56330cb9c780 msgr2=0x56330cb99930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:14.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.788+0000 7f3d8b61d740 1 --2- 172.21.15.120:0/1357836776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56330cb9c780 0x56330cb99930 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c009d50 tx=0x7f3d7c013040 comp rx=0 tx=0).stop 2024-09-19T09:05:14.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.788+0000 7f3d8b61d740 1 -- 172.21.15.120:0/1357836776 shutdown_connections 2024-09-19T09:05:14.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.788+0000 7f3d8b61d740 1 --2- 172.21.15.120:0/1357836776 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56330cbac260 0x56330cbae650 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.788+0000 7f3d8b61d740 1 --2- 172.21.15.120:0/1357836776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56330cb9c780 0x56330cb99930 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.788+0000 7f3d8b61d740 1 --2- 172.21.15.120:0/1357836776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56330cb8eb40 0x56330cb993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.788+0000 7f3d8b61d740 1 --2- 172.21.15.120:0/1357836776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56330cb8d5d0 0x56330cb9c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.788+0000 7f3d66ff5640 0 client.16131 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:05:14.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.788+0000 7f3d8b61d740 1 -- 172.21.15.120:0/1357836776 >> 172.21.15.120:0/1357836776 conn(0x56330cb56310 msgr2=0x56330cb8f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:14.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.788+0000 7f3d8b61d740 1 -- 172.21.15.120:0/1357836776 shutdown_connections 2024-09-19T09:05:14.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.788+0000 7f3d8b61d740 1 -- 172.21.15.120:0/1357836776 wait complete. 2024-09-19T09:05:14.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.788+0000 7f3d8b61d740 20 client.16131 trim_cache size 0 max 16384 2024-09-19T09:05:14.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.801+0000 7f3d8b61d740 1 Processor -- start 2024-09-19T09:05:14.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.801+0000 7f3d8b61d740 1 -- start start 2024-09-19T09:05:14.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.801+0000 7f3d8b61d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56330cb8d5d0 0x56330cba8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.801+0000 7f3d8b61d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56330cb8eb40 0x56330cb98390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.801+0000 7f3d8b61d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56330cb9c780 0x56330cbb3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.801+0000 7f3d8b61d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3d841ab3f0 con 0x56330cb8eb40 2024-09-19T09:05:14.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.801+0000 7f3d8b61d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3d841acf30 con 0x56330cb9c780 2024-09-19T09:05:14.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.801+0000 7f3d8b61d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3d841abac0 con 0x56330cb8d5d0 2024-09-19T09:05:14.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.801+0000 7f3d8259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56330cb8eb40 0x56330cb98390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.801+0000 7f3d8259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56330cb8eb40 0x56330cb98390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34248/0 (socket says 172.21.15.120:34248) 2024-09-19T09:05:14.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.801+0000 7f3d8259c640 1 -- 172.21.15.120:0/2376264780 learned_addr learned my addr 172.21.15.120:0/2376264780 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:14.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.801+0000 7f3d8359e640 1 --2- 172.21.15.120:0/2376264780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56330cb8d5d0 0x56330cba8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.801+0000 7f3d8359e640 1 --2- 172.21.15.120:0/2376264780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56330cb9c780 0x56330cbb3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.802+0000 7f3d8259c640 1 -- 172.21.15.120:0/2376264780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56330cb8d5d0 msgr2=0x56330cba8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:14.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.802+0000 7f3d8259c640 1 --2- 172.21.15.120:0/2376264780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56330cb8d5d0 0x56330cba8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.802+0000 7f3d8259c640 1 -- 172.21.15.120:0/2376264780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56330cb9c780 msgr2=0x56330cbb3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:14.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.802+0000 7f3d8259c640 1 --2- 172.21.15.120:0/2376264780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56330cb9c780 0x56330cbb3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.802+0000 7f3d8259c640 1 -- 172.21.15.120:0/2376264780 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56330cb8a730 con 0x56330cb8eb40 2024-09-19T09:05:14.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.802+0000 7f3d8359e640 1 --2- 172.21.15.120:0/2376264780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56330cb9c780 0x56330cbb3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:05:14.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.802+0000 7f3d8259c640 1 --2- 172.21.15.120:0/2376264780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56330cb8eb40 0x56330cb98390 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3d78031550 tx=0x7f3d78005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c16c16843b89a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:14.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.802+0000 7f3d68ff9640 1 -- 172.21.15.120:0/2376264780 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7804b070 con 0x56330cb8eb40 2024-09-19T09:05:14.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.802+0000 7f3d68ff9640 1 -- 172.21.15.120:0/2376264780 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d78005520 con 0x56330cb8eb40 2024-09-19T09:05:14.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.802+0000 7f3d68ff9640 1 -- 172.21.15.120:0/2376264780 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d78035040 con 0x56330cb8eb40 2024-09-19T09:05:14.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.803+0000 7f3d8b61d740 1 -- 172.21.15.120:0/2376264780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56330cb8eb40 msgr2=0x56330cb98390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:14.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.803+0000 7f3d8b61d740 1 --2- 172.21.15.120:0/2376264780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56330cb8eb40 0x56330cb98390 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3d78031550 tx=0x7f3d78005a20 comp rx=0 tx=0).stop 2024-09-19T09:05:14.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.803+0000 7f3d8b61d740 1 -- 172.21.15.120:0/2376264780 shutdown_connections 2024-09-19T09:05:14.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.803+0000 7f3d8b61d740 1 --2- 172.21.15.120:0/2376264780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56330cb9c780 0x56330cbb3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.803+0000 7f3d8b61d740 1 --2- 172.21.15.120:0/2376264780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56330cb8eb40 0x56330cb98390 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.803+0000 7f3d8b61d740 1 --2- 172.21.15.120:0/2376264780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56330cb8d5d0 0x56330cba8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.803+0000 7f3d8b61d740 1 -- 172.21.15.120:0/2376264780 >> 172.21.15.120:0/2376264780 conn(0x56330cb56310 msgr2=0x56330cb97c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:14.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.803+0000 7f3d8b61d740 1 -- 172.21.15.120:0/2376264780 shutdown_connections 2024-09-19T09:05:14.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.803+0000 7f3d8b61d740 1 -- 172.21.15.120:0/2376264780 wait complete. 2024-09-19T09:05:14.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.803+0000 7f3d8b61d740 1 -- start start 2024-09-19T09:05:14.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.803+0000 7f3d8b61d740 1 Processor -- start 2024-09-19T09:05:14.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.804+0000 7f3d8b61d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56330cb8d5d0 0x56330cba5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.804+0000 7f3d8b61d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56330cb8eb40 0x56330cba0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.804+0000 7f3d8b61d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56330cb9c780 0x56330cba08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.804+0000 7f3d8b61d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3d841ac860 con 0x56330cb9c780 2024-09-19T09:05:14.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.804+0000 7f3d8b61d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3d841ac190 con 0x56330cb8eb40 2024-09-19T09:05:14.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.804+0000 7f3d8b61d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3d841a93b0 con 0x56330cb8d5d0 2024-09-19T09:05:14.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.804+0000 7f3d8b61d740 10 client.0 _ll_register_callbacks cb 0x7f3d740d35f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:14.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.804+0000 7f3d8359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56330cb9c780 0x56330cba08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.804+0000 7f3d8259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56330cb8eb40 0x56330cba0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.804+0000 7f3d8359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56330cb9c780 0x56330cba08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34260/0 (socket says 172.21.15.120:34260) 2024-09-19T09:05:14.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.804+0000 7f3d8259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56330cb8eb40 0x56330cba0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38326/0 (socket says 172.21.15.120:38326) 2024-09-19T09:05:14.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.804+0000 7f3d8359e640 1 -- 172.21.15.120:0/432501284 learned_addr learned my addr 172.21.15.120:0/432501284 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:14.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.804+0000 7f3d8359e640 1 --2- 172.21.15.120:0/432501284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56330cb8d5d0 0x56330cba5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.804+0000 7f3d8259c640 1 -- 172.21.15.120:0/432501284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56330cb8d5d0 msgr2=0x56330cba5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:14.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.804+0000 7f3d8259c640 1 --2- 172.21.15.120:0/432501284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56330cb8d5d0 0x56330cba5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.804+0000 7f3d8259c640 1 -- 172.21.15.120:0/432501284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56330cb9c780 msgr2=0x56330cba08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:14.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.804+0000 7f3d8259c640 1 --2- 172.21.15.120:0/432501284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56330cb9c780 0x56330cba08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.804+0000 7f3d8259c640 1 -- 172.21.15.120:0/432501284 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56330cba10d0 con 0x56330cb8eb40 2024-09-19T09:05:14.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.804+0000 7f3d8359e640 1 --2- 172.21.15.120:0/432501284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56330cb9c780 0x56330cba08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:14.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.804+0000 7f3d8259c640 1 --2- 172.21.15.120:0/432501284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56330cb8eb40 0x56330cba0360 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3d78009230 tx=0x7f3d78004060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb96110e241e795d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:14.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.805+0000 7f3d66ff5640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:05:14.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.805+0000 7f3d66ff5640 1 -- 172.21.15.120:0/432501284 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d78054060 con 0x56330cb8eb40 2024-09-19T09:05:14.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.805+0000 7f3d66ff5640 1 -- 172.21.15.120:0/432501284 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f3d78004940 con 0x56330cb8eb40 2024-09-19T09:05:14.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.805+0000 7f3d8b61d740 1 -- 172.21.15.120:0/432501284 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d78005520 con 0x56330cb8eb40 2024-09-19T09:05:14.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.805+0000 7f3d66ff5640 1 -- 172.21.15.120:0/432501284 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3d7804b040 con 0x56330cb8eb40 2024-09-19T09:05:14.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.805+0000 7f3d8b61d740 10 client.25499 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:14.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.805+0000 7f3d8b61d740 1 -- 172.21.15.120:0/432501284 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56330cc16c20 con 0x56330cb8eb40 2024-09-19T09:05:14.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.805+0000 7f3d66ff5640 1 -- 172.21.15.120:0/432501284 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d78005520 con 0x56330cb8eb40 2024-09-19T09:05:14.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.805+0000 7f3d66ff5640 1 -- 172.21.15.120:0/432501284 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3d78059070 con 0x56330cb8eb40 2024-09-19T09:05:14.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.805+0000 7f3d8b61d740 10 client.25499 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:14.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.805+0000 7f3d8b61d740 10 client.25499 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:05:14.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.805+0000 7f3d8b61d740 20 client.25499 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:14.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.805+0000 7f3d8b61d740 1 --2- 172.21.15.120:0/432501284 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56330cc17240 0x56330cc19660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:14.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.805+0000 7f3d8b61d740 4 client.25499 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:05:14.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.805+0000 7f3d8b61d740 1 -- 172.21.15.120:0/432501284 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56330cafad70 con 0x56330cc17240 2024-09-19T09:05:14.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.806+0000 7f3d8359e640 1 --2- 172.21.15.120:0/432501284 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56330cc17240 0x56330cc19660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:14.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.806+0000 7f3d8359e640 1 --2- 172.21.15.120:0/432501284 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56330cc17240 0x56330cc19660 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5a5f2205cdea4b08 server_cookie=5e08dbe137155abd in_seq=0 out_seq=0 2024-09-19T09:05:14.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.806+0000 7f3d66ff5640 10 client.25499 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:05:14.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.806+0000 7f3d66ff5640 1 -- 172.21.15.120:0/432501284 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56330cafad70 con 0x56330cc17240 2024-09-19T09:05:14.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.806+0000 7f3d66ff5640 10 client.25499 handle_command_reply: tid=0 2024-09-19T09:05:14.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.806+0000 7f3d8b61d740 1 client.25499 shutdown 2024-09-19T09:05:14.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.806+0000 7f3d8b61d740 1 -- 172.21.15.120:0/432501284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56330cb8eb40 msgr2=0x56330cba0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:14.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.806+0000 7f3d8b61d740 1 --2- 172.21.15.120:0/432501284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56330cb8eb40 0x56330cba0360 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3d78009230 tx=0x7f3d78004060 comp rx=0 tx=0).stop 2024-09-19T09:05:14.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.806+0000 7f3d8b61d740 1 -- 172.21.15.120:0/432501284 shutdown_connections 2024-09-19T09:05:14.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.806+0000 7f3d8b61d740 1 --2- 172.21.15.120:0/432501284 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56330cc17240 0x56330cc19660 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.806+0000 7f3d8b61d740 1 --2- 172.21.15.120:0/432501284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56330cb9c780 0x56330cba08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.806+0000 7f3d66ff5640 0 client.25499 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:05:14.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.806+0000 7f3d8b61d740 1 --2- 172.21.15.120:0/432501284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56330cb8eb40 0x56330cba0360 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.806+0000 7f3d8b61d740 1 --2- 172.21.15.120:0/432501284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56330cb8d5d0 0x56330cba5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.806+0000 7f3d8b61d740 1 -- 172.21.15.120:0/432501284 >> 172.21.15.120:0/432501284 conn(0x56330cb56310 msgr2=0x56330cb984a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:14.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.806+0000 7f3d8b61d740 1 -- 172.21.15.120:0/432501284 shutdown_connections 2024-09-19T09:05:14.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.806+0000 7f3d8b61d740 1 -- 172.21.15.120:0/432501284 wait complete. 2024-09-19T09:05:14.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.807+0000 7f3d8b61d740 20 client.25499 trim_cache size 0 max 16384 2024-09-19T09:05:14.823 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:05:14.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.807+0000 7f3d89244640 1 -- 172.21.15.120:0/541106172 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3d6c0a3240 msgr2=0x7f3d6c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:14.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.807+0000 7f3d89244640 1 --2- 172.21.15.120:0/541106172 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3d6c0a3240 0x7f3d6c0a5700 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3d740a4a00 tx=0x7f3d740d3720 comp rx=0 tx=0).stop 2024-09-19T09:05:14.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.807+0000 7f3d89244640 1 -- 172.21.15.120:0/541106172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d84101de0 msgr2=0x7f3d841984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:14.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.807+0000 7f3d89244640 1 --2- 172.21.15.120:0/541106172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d84101de0 0x7f3d841984d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400e9f0 tx=0x7f3d7400eeb0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.807+0000 7f3d89244640 1 -- 172.21.15.120:0/541106172 shutdown_connections 2024-09-19T09:05:14.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.807+0000 7f3d89244640 1 --2- 172.21.15.120:0/541106172 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3d6c0a3240 0x7f3d6c0a5700 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.807+0000 7f3d89244640 1 --2- 172.21.15.120:0/541106172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d84106f10 0x7f3d8419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.807+0000 7f3d89244640 1 --2- 172.21.15.120:0/541106172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d84102790 0x7f3d84198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.807+0000 7f3d89244640 1 --2- 172.21.15.120:0/541106172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d84101de0 0x7f3d841984d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:14.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.807+0000 7f3d89244640 1 -- 172.21.15.120:0/541106172 >> 172.21.15.120:0/541106172 conn(0x7f3d840fd8e0 msgr2=0x7f3d840fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:14.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.807+0000 7f3d89244640 1 -- 172.21.15.120:0/541106172 shutdown_connections 2024-09-19T09:05:14.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:14.807+0000 7f3d89244640 1 -- 172.21.15.120:0/541106172 wait complete. 2024-09-19T09:05:15.038 INFO:tasks.workunit.client.0.smithi120.stdout: 1 85556 6.86 MB/sec warmup 70 sec latency 65.597 ms 2024-09-19T09:05:15.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:14 smithi120 ceph-mon[31150]: pgmap v412: 97 pgs: 97 active+clean; 40 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.8 MiB/s wr, 278 op/s 2024-09-19T09:05:15.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:14 smithi196 ceph-mon[35150]: pgmap v412: 97 pgs: 97 active+clean; 40 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.8 MiB/s wr, 278 op/s 2024-09-19T09:05:15.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:14 smithi153 ceph-mon[35071]: pgmap v412: 97 pgs: 97 active+clean; 40 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.8 MiB/s wr, 278 op/s 2024-09-19T09:05:15.730 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:05:15.983 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:05:16.038 INFO:tasks.workunit.client.0.smithi120.stdout: 1 88874 7.01 MB/sec warmup 71 sec latency 24.637 ms 2024-09-19T09:05:16.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.460+0000 7f774cc2e640 1 -- 172.21.15.120:0/280780098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7748102dd0 msgr2=0x7f77481031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:16.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.460+0000 7f774cc2e640 1 --2- 172.21.15.120:0/280780098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7748102dd0 0x7f77481031b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7738009930 tx=0x7f773802a420 comp rx=0 tx=0).stop 2024-09-19T09:05:16.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.460+0000 7f774cc2e640 1 -- 172.21.15.120:0/280780098 shutdown_connections 2024-09-19T09:05:16.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.460+0000 7f774cc2e640 1 --2- 172.21.15.120:0/280780098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7748108300 0x7f774810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.460+0000 7f774cc2e640 1 --2- 172.21.15.120:0/280780098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7748103780 0x7f7748107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.460+0000 7f774cc2e640 1 --2- 172.21.15.120:0/280780098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7748102dd0 0x7f77481031b0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.460+0000 7f774cc2e640 1 -- 172.21.15.120:0/280780098 >> 172.21.15.120:0/280780098 conn(0x7f77480fc7d0 msgr2=0x7f77480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:16.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.460+0000 7f774cc2e640 1 -- 172.21.15.120:0/280780098 shutdown_connections 2024-09-19T09:05:16.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.460+0000 7f774cc2e640 1 -- 172.21.15.120:0/280780098 wait complete. 2024-09-19T09:05:16.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.461+0000 7f774cc2e640 1 Processor -- start 2024-09-19T09:05:16.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.461+0000 7f774cc2e640 1 -- start start 2024-09-19T09:05:16.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.461+0000 7f774cc2e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7748102dd0 0x7f7748198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:16.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.461+0000 7f774cc2e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7748103780 0x7f7748198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:16.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.461+0000 7f774cc2e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7748108300 0x7f774819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:16.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.461+0000 7f774cc2e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f77481120b0 con 0x7f7748103780 2024-09-19T09:05:16.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.461+0000 7f774cc2e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7748111f30 con 0x7f7748108300 2024-09-19T09:05:16.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.461+0000 7f774cc2e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7748112230 con 0x7f7748102dd0 2024-09-19T09:05:16.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.461+0000 7f773ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7748103780 0x7f7748198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:16.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.461+0000 7f773ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7748103780 0x7f7748198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34266/0 (socket says 172.21.15.120:34266) 2024-09-19T09:05:16.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.461+0000 7f773ffff640 1 -- 172.21.15.120:0/2589444497 learned_addr learned my addr 172.21.15.120:0/2589444497 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:16.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.461+0000 7f774659c640 1 --2- 172.21.15.120:0/2589444497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7748102dd0 0x7f7748198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:16.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.461+0000 7f7746d9d640 1 --2- 172.21.15.120:0/2589444497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7748108300 0x7f774819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:16.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.461+0000 7f773ffff640 1 -- 172.21.15.120:0/2589444497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7748102dd0 msgr2=0x7f7748198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:16.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.461+0000 7f773ffff640 1 --2- 172.21.15.120:0/2589444497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7748102dd0 0x7f7748198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.461+0000 7f773ffff640 1 -- 172.21.15.120:0/2589444497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7748108300 msgr2=0x7f774819cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:16.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.461+0000 7f773ffff640 1 --2- 172.21.15.120:0/2589444497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7748108300 0x7f774819cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.461+0000 7f773ffff640 1 -- 172.21.15.120:0/2589444497 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774819d620 con 0x7f7748103780 2024-09-19T09:05:16.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.462+0000 7f773ffff640 1 --2- 172.21.15.120:0/2589444497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7748103780 0x7f7748198bb0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f773000e9f0 tx=0x7f773000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:16.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.462+0000 7f773f7fe640 1 -- 172.21.15.120:0/2589444497 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773000ccc0 con 0x7f7748103780 2024-09-19T09:05:16.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.462+0000 7f774cc2e640 1 -- 172.21.15.120:0/2589444497 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f774819d910 con 0x7f7748103780 2024-09-19T09:05:16.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.462+0000 7f773f7fe640 1 -- 172.21.15.120:0/2589444497 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7730010910 con 0x7f7748103780 2024-09-19T09:05:16.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.462+0000 7f773f7fe640 1 -- 172.21.15.120:0/2589444497 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7730018920 con 0x7f7748103780 2024-09-19T09:05:16.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.462+0000 7f774cc2e640 1 -- 172.21.15.120:0/2589444497 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77481a51f0 con 0x7f7748103780 2024-09-19T09:05:16.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.464+0000 7f773f7fe640 1 -- 172.21.15.120:0/2589444497 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f7730010ac0 con 0x7f7748103780 2024-09-19T09:05:16.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.464+0000 7f774f007740 1 Processor -- start 2024-09-19T09:05:16.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.465+0000 7f774f007740 1 -- start start 2024-09-19T09:05:16.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.465+0000 7f774f007740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b10ef27b40 0x55b10ef26090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:16.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.465+0000 7f774f007740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10ef265d0 0x55b10ef26a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:16.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.465+0000 7f773f7fe640 1 --2- 172.21.15.120:0/2589444497 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f77340a3240 0x7f77340a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:16.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.465+0000 7f774f007740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b10ef31370 0x55b10ef33760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:16.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.465+0000 7f774f007740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b10ee8c2f0 con 0x55b10ef31370 2024-09-19T09:05:16.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.465+0000 7f774f007740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b10ee33cf0 con 0x55b10ef27b40 2024-09-19T09:05:16.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.465+0000 7f774f007740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b10ee882b0 con 0x55b10ef265d0 2024-09-19T09:05:16.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.465+0000 7f7746d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b10ef27b40 0x55b10ef26090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:16.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.465+0000 7f7746d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b10ef31370 0x55b10ef33760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:16.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.465+0000 7f774659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10ef265d0 0x55b10ef26a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:16.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.465+0000 7f7746d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b10ef27b40 0x55b10ef26090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38366/0 (socket says 172.21.15.120:38366) 2024-09-19T09:05:16.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.465+0000 7f7746d9d640 1 -- 172.21.15.120:0/1423858635 learned_addr learned my addr 172.21.15.120:0/1423858635 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:16.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.465+0000 7f773ffff640 1 --2- 172.21.15.120:0/2589444497 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f77340a3240 0x7f77340a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:16.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.465+0000 7f773f7fe640 1 -- 172.21.15.120:0/2589444497 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f77300cfba0 con 0x7f7748103780 2024-09-19T09:05:16.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.465+0000 7f774659c640 1 -- 172.21.15.120:0/1423858635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b10ef27b40 msgr2=0x55b10ef26090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:16.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.465+0000 7f774659c640 1 --2- 172.21.15.120:0/1423858635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b10ef27b40 0x55b10ef26090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.465+0000 7f774659c640 1 -- 172.21.15.120:0/1423858635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b10ef31370 msgr2=0x55b10ef33760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:16.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.465+0000 7f774659c640 1 --2- 172.21.15.120:0/1423858635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b10ef31370 0x55b10ef33760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.465+0000 7f774659c640 1 -- 172.21.15.120:0/1423858635 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b10ee183a0 con 0x55b10ef265d0 2024-09-19T09:05:16.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.466+0000 7f773ffff640 1 --2- 172.21.15.120:0/2589444497 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f77340a3240 0x7f77340a5700 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f773000c6a0 tx=0x7f7730091750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:16.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.466+0000 7f774659c640 1 --2- 172.21.15.120:0/1423858635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10ef265d0 0x55b10ef26a10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7738005940 tx=0x7f7738005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b91f9f22b1abc20 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:16.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.466+0000 7f773cff9640 1 -- 172.21.15.120:0/1423858635 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7738040070 con 0x55b10ef265d0 2024-09-19T09:05:16.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.466+0000 7f773cff9640 1 -- 172.21.15.120:0/1423858635 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77380318f0 con 0x55b10ef265d0 2024-09-19T09:05:16.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.466+0000 7f773cff9640 1 -- 172.21.15.120:0/1423858635 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7738035040 con 0x55b10ef265d0 2024-09-19T09:05:16.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.467+0000 7f774f007740 1 -- 172.21.15.120:0/1423858635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10ef265d0 msgr2=0x55b10ef26a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:16.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.467+0000 7f774f007740 1 --2- 172.21.15.120:0/1423858635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10ef265d0 0x55b10ef26a10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7738005940 tx=0x7f7738005c00 comp rx=0 tx=0).stop 2024-09-19T09:05:16.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.467+0000 7f774f007740 1 -- 172.21.15.120:0/1423858635 shutdown_connections 2024-09-19T09:05:16.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.467+0000 7f774f007740 1 --2- 172.21.15.120:0/1423858635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b10ef31370 0x55b10ef33760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.467+0000 7f774f007740 1 --2- 172.21.15.120:0/1423858635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10ef265d0 0x55b10ef26a10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.467+0000 7f774f007740 1 --2- 172.21.15.120:0/1423858635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b10ef27b40 0x55b10ef26090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.467+0000 7f774f007740 1 -- 172.21.15.120:0/1423858635 >> 172.21.15.120:0/1423858635 conn(0x55b10eeef310 msgr2=0x55b10ef22c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:16.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.467+0000 7f774f007740 1 -- 172.21.15.120:0/1423858635 shutdown_connections 2024-09-19T09:05:16.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.467+0000 7f774f007740 1 -- 172.21.15.120:0/1423858635 wait complete. 2024-09-19T09:05:16.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.467+0000 7f774f007740 1 -- start start 2024-09-19T09:05:16.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.467+0000 7f774f007740 1 Processor -- start 2024-09-19T09:05:16.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.467+0000 7f774f007740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10ef265d0 0x55b10ef30d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:16.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.467+0000 7f774f007740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b10ef27b40 0x55b10ef2dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:16.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.467+0000 7f774f007740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b10ef31370 0x55b10ef2e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:16.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.467+0000 7f774f007740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b10ef34010 con 0x55b10ef31370 2024-09-19T09:05:16.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.467+0000 7f774f007740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b10ee47060 con 0x55b10ef27b40 2024-09-19T09:05:16.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.467+0000 7f774f007740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b10ef34190 con 0x55b10ef265d0 2024-09-19T09:05:16.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.468+0000 7f774f007740 10 client.0 _ll_register_callbacks cb 0x55b10eb9a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:16.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.468+0000 7f7746d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b10ef31370 0x55b10ef2e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:16.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.468+0000 7f774659c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b10ef27b40 0x55b10ef2dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:16.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.468+0000 7f7746d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b10ef31370 0x55b10ef2e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34272/0 (socket says 172.21.15.120:34272) 2024-09-19T09:05:16.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.468+0000 7f7746d9d640 1 -- 172.21.15.120:0/2465621691 learned_addr learned my addr 172.21.15.120:0/2465621691 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:16.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.468+0000 7f7746d9d640 1 --2- 172.21.15.120:0/2465621691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10ef265d0 0x55b10ef30d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:16.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.468+0000 7f7746d9d640 1 -- 172.21.15.120:0/2465621691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10ef265d0 msgr2=0x55b10ef30d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:16.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.468+0000 7f7746d9d640 1 --2- 172.21.15.120:0/2465621691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10ef265d0 0x55b10ef30d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.468+0000 7f7746d9d640 1 -- 172.21.15.120:0/2465621691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b10ef27b40 msgr2=0x55b10ef2dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:16.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.468+0000 7f7746d9d640 1 --2- 172.21.15.120:0/2465621691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b10ef27b40 0x55b10ef2dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.468+0000 7f7746d9d640 1 -- 172.21.15.120:0/2465621691 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b10ebfd3c0 con 0x55b10ef31370 2024-09-19T09:05:16.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.468+0000 7f7746d9d640 1 --2- 172.21.15.120:0/2465621691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b10ef31370 0x55b10ef2e570 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7740005f40 tx=0x7f7740009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0368aa258b7fc6a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:16.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.468+0000 7f772dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:16.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.468+0000 7f772dffb640 1 -- 172.21.15.120:0/2465621691 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7740004b20 con 0x55b10ef31370 2024-09-19T09:05:16.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.468+0000 7f772dffb640 1 -- 172.21.15.120:0/2465621691 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f7740004cc0 con 0x55b10ef31370 2024-09-19T09:05:16.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.468+0000 7f774f007740 1 -- 172.21.15.120:0/2465621691 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77380318f0 con 0x55b10ef31370 2024-09-19T09:05:16.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.469+0000 7f772dffb640 1 -- 172.21.15.120:0/2465621691 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f774002a050 con 0x55b10ef31370 2024-09-19T09:05:16.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.469+0000 7f774f007740 10 client.16149 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:16.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.469+0000 7f774f007740 1 -- 172.21.15.120:0/2465621691 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b10ef3fb10 con 0x55b10ef31370 2024-09-19T09:05:16.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.469+0000 7f772dffb640 1 -- 172.21.15.120:0/2465621691 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77380318f0 con 0x55b10ef31370 2024-09-19T09:05:16.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.469+0000 7f772dffb640 1 -- 172.21.15.120:0/2465621691 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7740021040 con 0x55b10ef31370 2024-09-19T09:05:16.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.469+0000 7f774f007740 10 client.16149 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:16.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.469+0000 7f774f007740 10 client.16149 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:05:16.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.469+0000 7f774f007740 20 client.16149 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:16.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.469+0000 7f774f007740 1 --2- 172.21.15.120:0/2465621691 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55b10ef40f20 0x55b10ef43310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:16.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.469+0000 7f774f007740 4 client.16149 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:05:16.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.469+0000 7f774f007740 1 -- 172.21.15.120:0/2465621691 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b10ee99860 con 0x55b10ef40f20 2024-09-19T09:05:16.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.469+0000 7f774659c640 1 --2- 172.21.15.120:0/2465621691 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55b10ef40f20 0x55b10ef43310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:16.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.470+0000 7f772dffb640 10 client.16149 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:05:16.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.470+0000 7f774659c640 1 --2- 172.21.15.120:0/2465621691 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55b10ef40f20 0x55b10ef43310 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=46ad520a9f1665a1 server_cookie=fed278c65ca85466 in_seq=0 out_seq=0 2024-09-19T09:05:16.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.470+0000 7f772dffb640 1 -- 172.21.15.120:0/2465621691 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b10ee99860 con 0x55b10ef40f20 2024-09-19T09:05:16.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.470+0000 7f772dffb640 10 client.16149 handle_command_reply: tid=0 2024-09-19T09:05:16.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.470+0000 7f774f007740 1 client.16149 shutdown 2024-09-19T09:05:16.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.470+0000 7f774f007740 1 -- 172.21.15.120:0/2465621691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b10ef31370 msgr2=0x55b10ef2e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:16.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.470+0000 7f774f007740 1 --2- 172.21.15.120:0/2465621691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b10ef31370 0x55b10ef2e570 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7740005f40 tx=0x7f7740009ea0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.471+0000 7f774f007740 1 -- 172.21.15.120:0/2465621691 shutdown_connections 2024-09-19T09:05:16.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.471+0000 7f774f007740 1 --2- 172.21.15.120:0/2465621691 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55b10ef40f20 0x55b10ef43310 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.471+0000 7f772dffb640 0 client.16149 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:05:16.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.471+0000 7f774f007740 1 --2- 172.21.15.120:0/2465621691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b10ef31370 0x55b10ef2e570 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.471+0000 7f774f007740 1 --2- 172.21.15.120:0/2465621691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b10ef27b40 0x55b10ef2dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.471+0000 7f774f007740 1 --2- 172.21.15.120:0/2465621691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10ef265d0 0x55b10ef30d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.471+0000 7f774f007740 1 -- 172.21.15.120:0/2465621691 >> 172.21.15.120:0/2465621691 conn(0x55b10eeef310 msgr2=0x55b10ef28960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:16.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.471+0000 7f774f007740 1 -- 172.21.15.120:0/2465621691 shutdown_connections 2024-09-19T09:05:16.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.471+0000 7f774f007740 1 -- 172.21.15.120:0/2465621691 wait complete. 2024-09-19T09:05:16.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.471+0000 7f774f007740 20 client.16149 trim_cache size 0 max 16384 2024-09-19T09:05:16.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.484+0000 7f774f007740 1 Processor -- start 2024-09-19T09:05:16.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.484+0000 7f774f007740 1 -- start start 2024-09-19T09:05:16.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.485+0000 7f774f007740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b10ef265d0 0x55b10ef3d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:16.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.485+0000 7f774f007740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b10ef27b40 0x55b10ef32760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:16.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.485+0000 7f774f007740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10ef31370 0x55b10ef32ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:16.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.485+0000 7f774f007740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f77481ad660 con 0x55b10ef265d0 2024-09-19T09:05:16.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.485+0000 7f774f007740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f77481af1a0 con 0x55b10ef27b40 2024-09-19T09:05:16.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.485+0000 7f774f007740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f77481add30 con 0x55b10ef31370 2024-09-19T09:05:16.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.485+0000 7f7746d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b10ef265d0 0x55b10ef3d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:16.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.485+0000 7f7746d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b10ef265d0 0x55b10ef3d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34284/0 (socket says 172.21.15.120:34284) 2024-09-19T09:05:16.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.485+0000 7f7746d9d640 1 -- 172.21.15.120:0/1563168409 learned_addr learned my addr 172.21.15.120:0/1563168409 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:16.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.485+0000 7f774659c640 1 --2- 172.21.15.120:0/1563168409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b10ef27b40 0x55b10ef32760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:16.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.485+0000 7f7746d9d640 1 --2- 172.21.15.120:0/1563168409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10ef31370 0x55b10ef32ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:16.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.485+0000 7f7746d9d640 1 -- 172.21.15.120:0/1563168409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10ef31370 msgr2=0x55b10ef32ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:16.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.485+0000 7f7746d9d640 1 --2- 172.21.15.120:0/1563168409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10ef31370 0x55b10ef32ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.485+0000 7f7746d9d640 1 -- 172.21.15.120:0/1563168409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b10ef27b40 msgr2=0x55b10ef32760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:16.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.485+0000 7f7746d9d640 1 --2- 172.21.15.120:0/1563168409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b10ef27b40 0x55b10ef32760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.485+0000 7f7746d9d640 1 -- 172.21.15.120:0/1563168409 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b10ef30940 con 0x55b10ef265d0 2024-09-19T09:05:16.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.485+0000 7f774659c640 1 --2- 172.21.15.120:0/1563168409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b10ef27b40 0x55b10ef32760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:16.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.485+0000 7f7746d9d640 1 --2- 172.21.15.120:0/1563168409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b10ef265d0 0x55b10ef3d2b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x55b10ef2f4d0 tx=0x7f77400104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92a7921c24634f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:16.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.486+0000 7f772ffff640 1 -- 172.21.15.120:0/1563168409 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774000f730 con 0x55b10ef265d0 2024-09-19T09:05:16.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.486+0000 7f772ffff640 1 -- 172.21.15.120:0/1563168409 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7740003e50 con 0x55b10ef265d0 2024-09-19T09:05:16.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.486+0000 7f772ffff640 1 -- 172.21.15.120:0/1563168409 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7740005260 con 0x55b10ef265d0 2024-09-19T09:05:16.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.486+0000 7f774f007740 1 -- 172.21.15.120:0/1563168409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b10ef265d0 msgr2=0x55b10ef3d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:16.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.486+0000 7f774f007740 1 --2- 172.21.15.120:0/1563168409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b10ef265d0 0x55b10ef3d2b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x55b10ef2f4d0 tx=0x7f77400104a0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.486+0000 7f774f007740 1 -- 172.21.15.120:0/1563168409 shutdown_connections 2024-09-19T09:05:16.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.486+0000 7f774f007740 1 --2- 172.21.15.120:0/1563168409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10ef31370 0x55b10ef32ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.486+0000 7f774f007740 1 --2- 172.21.15.120:0/1563168409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b10ef27b40 0x55b10ef32760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.486+0000 7f774f007740 1 --2- 172.21.15.120:0/1563168409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b10ef265d0 0x55b10ef3d2b0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.486+0000 7f774f007740 1 -- 172.21.15.120:0/1563168409 >> 172.21.15.120:0/1563168409 conn(0x55b10eeef310 msgr2=0x55b10ef2df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:16.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.486+0000 7f774f007740 1 -- 172.21.15.120:0/1563168409 shutdown_connections 2024-09-19T09:05:16.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.486+0000 7f774f007740 1 -- 172.21.15.120:0/1563168409 wait complete. 2024-09-19T09:05:16.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.487+0000 7f774f007740 1 -- start start 2024-09-19T09:05:16.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.487+0000 7f774f007740 1 Processor -- start 2024-09-19T09:05:16.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.487+0000 7f774f007740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b10ef265d0 0x55b10ef39710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:16.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.487+0000 7f774f007740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10ef27b40 0x55b10ef39c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:16.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.487+0000 7f774f007740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b10ef31370 0x55b10ef48290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:16.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.487+0000 7f774f007740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f77481aead0 con 0x55b10ef31370 2024-09-19T09:05:16.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.487+0000 7f774f007740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f77481ae400 con 0x55b10ef265d0 2024-09-19T09:05:16.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.487+0000 7f774f007740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f77481ab620 con 0x55b10ef27b40 2024-09-19T09:05:16.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.487+0000 7f774f007740 10 client.0 _ll_register_callbacks cb 0x7f77400103f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:16.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.487+0000 7f7746d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b10ef31370 0x55b10ef48290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:16.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.487+0000 7f7746d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b10ef31370 0x55b10ef48290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34298/0 (socket says 172.21.15.120:34298) 2024-09-19T09:05:16.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.487+0000 7f7746d9d640 1 -- 172.21.15.120:0/1753059265 learned_addr learned my addr 172.21.15.120:0/1753059265 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:16.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.487+0000 7f7746d9d640 1 --2- 172.21.15.120:0/1753059265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b10ef265d0 0x55b10ef39710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:16.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.487+0000 7f774659c640 1 --2- 172.21.15.120:0/1753059265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10ef27b40 0x55b10ef39c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:16.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.488+0000 7f7746d9d640 1 -- 172.21.15.120:0/1753059265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10ef27b40 msgr2=0x55b10ef39c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:16.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.488+0000 7f7746d9d640 1 --2- 172.21.15.120:0/1753059265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10ef27b40 0x55b10ef39c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.488+0000 7f7746d9d640 1 -- 172.21.15.120:0/1753059265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b10ef265d0 msgr2=0x55b10ef39710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:16.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.488+0000 7f7746d9d640 1 --2- 172.21.15.120:0/1753059265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b10ef265d0 0x55b10ef39710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.488+0000 7f7746d9d640 1 -- 172.21.15.120:0/1753059265 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b10ef48ac0 con 0x55b10ef31370 2024-09-19T09:05:16.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.488+0000 7f7746d9d640 1 --2- 172.21.15.120:0/1753059265 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b10ef31370 0x55b10ef48290 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7740012f50 tx=0x7f774000fb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=433603569153860b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:16.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.488+0000 7f772dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:16.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.488+0000 7f772dffb640 1 -- 172.21.15.120:0/1753059265 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7740037040 con 0x55b10ef31370 2024-09-19T09:05:16.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.488+0000 7f772dffb640 1 -- 172.21.15.120:0/1753059265 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f7740005000 con 0x55b10ef31370 2024-09-19T09:05:16.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.488+0000 7f774f007740 1 -- 172.21.15.120:0/1753059265 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7740003e50 con 0x55b10ef31370 2024-09-19T09:05:16.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.488+0000 7f772dffb640 1 -- 172.21.15.120:0/1753059265 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f774000fde0 con 0x55b10ef31370 2024-09-19T09:05:16.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.488+0000 7f774f007740 10 client.16155 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:16.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.488+0000 7f774f007740 1 -- 172.21.15.120:0/1753059265 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b10ef390b0 con 0x55b10ef31370 2024-09-19T09:05:16.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.488+0000 7f772dffb640 1 -- 172.21.15.120:0/1753059265 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7740003e50 con 0x55b10ef31370 2024-09-19T09:05:16.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.488+0000 7f772dffb640 1 -- 172.21.15.120:0/1753059265 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f774003d070 con 0x55b10ef31370 2024-09-19T09:05:16.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.488+0000 7f774f007740 10 client.16155 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:16.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.488+0000 7f774f007740 10 client.16155 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:05:16.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.488+0000 7f774f007740 20 client.16155 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:16.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.488+0000 7f774f007740 1 --2- 172.21.15.120:0/1753059265 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55b10efabee0 0x55b10efae300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:16.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.489+0000 7f774f007740 4 client.16155 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:05:16.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.489+0000 7f774f007740 1 -- 172.21.15.120:0/1753059265 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b10ee93d70 con 0x55b10efabee0 2024-09-19T09:05:16.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.489+0000 7f774659c640 1 --2- 172.21.15.120:0/1753059265 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55b10efabee0 0x55b10efae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:16.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.489+0000 7f772dffb640 10 client.16155 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:05:16.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.489+0000 7f774659c640 1 --2- 172.21.15.120:0/1753059265 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55b10efabee0 0x55b10efae300 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c003439aac8cef9b server_cookie=c3f0f88372d679fd in_seq=0 out_seq=0 2024-09-19T09:05:16.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.489+0000 7f772dffb640 1 -- 172.21.15.120:0/1753059265 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b10ee93d70 con 0x55b10efabee0 2024-09-19T09:05:16.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.489+0000 7f772dffb640 10 client.16155 handle_command_reply: tid=0 2024-09-19T09:05:16.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.489+0000 7f774f007740 1 client.16155 shutdown 2024-09-19T09:05:16.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.489+0000 7f774f007740 1 -- 172.21.15.120:0/1753059265 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b10ef31370 msgr2=0x55b10ef48290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:16.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.489+0000 7f774f007740 1 --2- 172.21.15.120:0/1753059265 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b10ef31370 0x55b10ef48290 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7740012f50 tx=0x7f774000fb80 comp rx=0 tx=0).stop 2024-09-19T09:05:16.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.489+0000 7f774f007740 1 -- 172.21.15.120:0/1753059265 shutdown_connections 2024-09-19T09:05:16.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.489+0000 7f774f007740 1 --2- 172.21.15.120:0/1753059265 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55b10efabee0 0x55b10efae300 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.489+0000 7f774f007740 1 --2- 172.21.15.120:0/1753059265 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b10ef31370 0x55b10ef48290 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.489+0000 7f774f007740 1 --2- 172.21.15.120:0/1753059265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10ef27b40 0x55b10ef39c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.489+0000 7f774f007740 1 --2- 172.21.15.120:0/1753059265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b10ef265d0 0x55b10ef39710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.489+0000 7f774f007740 1 -- 172.21.15.120:0/1753059265 >> 172.21.15.120:0/1753059265 conn(0x55b10eeef310 msgr2=0x55b10ef2f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:16.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.489+0000 7f772dffb640 0 client.16155 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:05:16.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.490+0000 7f774f007740 1 -- 172.21.15.120:0/1753059265 shutdown_connections 2024-09-19T09:05:16.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.490+0000 7f774f007740 1 -- 172.21.15.120:0/1753059265 wait complete. 2024-09-19T09:05:16.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.490+0000 7f774f007740 20 client.16155 trim_cache size 0 max 16384 2024-09-19T09:05:16.505 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:05:16.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.490+0000 7f774cc2e640 1 -- 172.21.15.120:0/2589444497 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f77340a3240 msgr2=0x7f77340a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:16.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.490+0000 7f774cc2e640 1 --2- 172.21.15.120:0/2589444497 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f77340a3240 0x7f77340a5700 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f773000c6a0 tx=0x7f7730091750 comp rx=0 tx=0).stop 2024-09-19T09:05:16.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.490+0000 7f774cc2e640 1 -- 172.21.15.120:0/2589444497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7748103780 msgr2=0x7f7748198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:16.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.490+0000 7f774cc2e640 1 --2- 172.21.15.120:0/2589444497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7748103780 0x7f7748198bb0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f773000e9f0 tx=0x7f773000eeb0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.490+0000 7f774cc2e640 1 -- 172.21.15.120:0/2589444497 shutdown_connections 2024-09-19T09:05:16.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.490+0000 7f774cc2e640 1 --2- 172.21.15.120:0/2589444497 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f77340a3240 0x7f77340a5700 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.490+0000 7f774cc2e640 1 --2- 172.21.15.120:0/2589444497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7748108300 0x7f774819cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.490+0000 7f774cc2e640 1 --2- 172.21.15.120:0/2589444497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7748103780 0x7f7748198bb0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.490+0000 7f774cc2e640 1 --2- 172.21.15.120:0/2589444497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7748102dd0 0x7f7748198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:16.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.490+0000 7f774cc2e640 1 -- 172.21.15.120:0/2589444497 >> 172.21.15.120:0/2589444497 conn(0x7f77480fc7d0 msgr2=0x7f7748105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:16.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.490+0000 7f774cc2e640 1 -- 172.21.15.120:0/2589444497 shutdown_connections 2024-09-19T09:05:16.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:16.490+0000 7f774cc2e640 1 -- 172.21.15.120:0/2589444497 wait complete. 2024-09-19T09:05:17.036 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:16 smithi120 ceph-mon[31150]: pgmap v413: 97 pgs: 97 active+clean; 42 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 284 op/s 2024-09-19T09:05:17.038 INFO:tasks.workunit.client.0.smithi120.stdout: 1 92248 7.17 MB/sec warmup 72 sec latency 12.542 ms 2024-09-19T09:05:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:16 smithi196 ceph-mon[35150]: pgmap v413: 97 pgs: 97 active+clean; 42 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 284 op/s 2024-09-19T09:05:17.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:16 smithi153 ceph-mon[35071]: pgmap v413: 97 pgs: 97 active+clean; 42 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 284 op/s 2024-09-19T09:05:17.753 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:05:18.010 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:05:18.038 INFO:tasks.workunit.client.0.smithi120.stdout: 1 95193 7.27 MB/sec warmup 73 sec latency 15.262 ms 2024-09-19T09:05:18.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.512+0000 7fa20539f640 1 -- 172.21.15.120:0/1890773144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa20010e4d0 msgr2=0x7fa2001108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:18.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.512+0000 7fa20539f640 1 --2- 172.21.15.120:0/1890773144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa20010e4d0 0x7fa2001108e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009ca0 tx=0x7fa1f802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.512+0000 7fa20539f640 1 -- 172.21.15.120:0/1890773144 shutdown_connections 2024-09-19T09:05:18.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.512+0000 7fa20539f640 1 --2- 172.21.15.120:0/1890773144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa20010e4d0 0x7fa2001108e0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.512+0000 7fa20539f640 1 --2- 172.21.15.120:0/1890773144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa200101360 0x7fa20010df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.512+0000 7fa20539f640 1 --2- 172.21.15.120:0/1890773144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2001009b0 0x7fa200100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.512+0000 7fa20539f640 1 -- 172.21.15.120:0/1890773144 >> 172.21.15.120:0/1890773144 conn(0x7fa2000fc7d0 msgr2=0x7fa2000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:18.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.512+0000 7fa20539f640 1 -- 172.21.15.120:0/1890773144 shutdown_connections 2024-09-19T09:05:18.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.512+0000 7fa20539f640 1 -- 172.21.15.120:0/1890773144 wait complete. 2024-09-19T09:05:18.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.513+0000 7fa20539f640 1 Processor -- start 2024-09-19T09:05:18.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.513+0000 7fa20539f640 1 -- start start 2024-09-19T09:05:18.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.513+0000 7fa20539f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2001009b0 0x7fa20019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:18.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.513+0000 7fa20539f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa200101360 0x7fa20019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:18.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.513+0000 7fa20539f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa20010e4d0 0x7fa20019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:18.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.513+0000 7fa20539f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa200114220 con 0x7fa20010e4d0 2024-09-19T09:05:18.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.513+0000 7fa20539f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa2001140a0 con 0x7fa200101360 2024-09-19T09:05:18.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.513+0000 7fa20539f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa2001143a0 con 0x7fa2001009b0 2024-09-19T09:05:18.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.513+0000 7fa1ff7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa20010e4d0 0x7fa20019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:18.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.513+0000 7fa1fe7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa200101360 0x7fa20019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:18.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.513+0000 7fa1feffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2001009b0 0x7fa20019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:18.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.513+0000 7fa1fe7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa200101360 0x7fa20019ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38406/0 (socket says 172.21.15.120:38406) 2024-09-19T09:05:18.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.513+0000 7fa1ff7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa20010e4d0 0x7fa20019efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34314/0 (socket says 172.21.15.120:34314) 2024-09-19T09:05:18.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.513+0000 7fa1fe7fc640 1 -- 172.21.15.120:0/2964813149 learned_addr learned my addr 172.21.15.120:0/2964813149 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:18.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.513+0000 7fa1feffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2001009b0 0x7fa20019a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:58868/0 (socket says 172.21.15.120:58868) 2024-09-19T09:05:18.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.514+0000 7fa1fe7fc640 1 -- 172.21.15.120:0/2964813149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2001009b0 msgr2=0x7fa20019a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:18.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.514+0000 7fa1fe7fc640 1 --2- 172.21.15.120:0/2964813149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2001009b0 0x7fa20019a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.514+0000 7fa1fe7fc640 1 -- 172.21.15.120:0/2964813149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa20010e4d0 msgr2=0x7fa20019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:18.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.514+0000 7fa1fe7fc640 1 --2- 172.21.15.120:0/2964813149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa20010e4d0 0x7fa20019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.514+0000 7fa1fe7fc640 1 -- 172.21.15.120:0/2964813149 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20019f6b0 con 0x7fa200101360 2024-09-19T09:05:18.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.514+0000 7fa1ff7fe640 1 --2- 172.21.15.120:0/2964813149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa20010e4d0 0x7fa20019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:18.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.514+0000 7fa1fe7fc640 1 --2- 172.21.15.120:0/2964813149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa200101360 0x7fa20019ac40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800e9f0 tx=0x7fa1e800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:18.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.514+0000 7fa1f7fff640 1 -- 172.21.15.120:0/2964813149 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e800cd80 con 0x7fa200101360 2024-09-19T09:05:18.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.514+0000 7fa20539f640 1 -- 172.21.15.120:0/2964813149 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa20019f9a0 con 0x7fa200101360 2024-09-19T09:05:18.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.514+0000 7fa1f7fff640 1 -- 172.21.15.120:0/2964813149 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1e8005500 con 0x7fa200101360 2024-09-19T09:05:18.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.514+0000 7fa1f7fff640 1 -- 172.21.15.120:0/2964813149 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8010690 con 0x7fa200101360 2024-09-19T09:05:18.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.514+0000 7fa20539f640 1 -- 172.21.15.120:0/2964813149 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2001a7280 con 0x7fa200101360 2024-09-19T09:05:18.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.516+0000 7fa207778740 1 Processor -- start 2024-09-19T09:05:18.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.517+0000 7fa207778740 1 -- start start 2024-09-19T09:05:18.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.517+0000 7fa207778740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aafae9b40 0x557aafae8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:18.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.517+0000 7fa207778740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aafae85d0 0x557aafae8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:18.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.517+0000 7fa207778740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aafaf3370 0x557aafaf5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:18.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.517+0000 7fa207778740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557aafa4e2f0 con 0x557aafae9b40 2024-09-19T09:05:18.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.517+0000 7fa207778740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557aaf9f5cf0 con 0x557aafae85d0 2024-09-19T09:05:18.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.517+0000 7fa207778740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557aafa4a2b0 con 0x557aafaf3370 2024-09-19T09:05:18.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.517+0000 7fa1ff7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aafae9b40 0x557aafae8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:18.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.517+0000 7fa1ff7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aafae9b40 0x557aafae8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34324/0 (socket says 172.21.15.120:34324) 2024-09-19T09:05:18.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.517+0000 7fa1ff7fe640 1 -- 172.21.15.120:0/3515703862 learned_addr learned my addr 172.21.15.120:0/3515703862 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:18.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.517+0000 7fa1feffd640 1 --2- 172.21.15.120:0/3515703862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aafae85d0 0x557aafae8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:18.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.517+0000 7fa1f7fff640 1 -- 172.21.15.120:0/2964813149 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa1e8010830 con 0x7fa200101360 2024-09-19T09:05:18.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.517+0000 7fa1fe7fc640 1 --2- 172.21.15.120:0/3515703862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aafaf3370 0x557aafaf5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:18.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.517+0000 7fa1ff7fe640 1 -- 172.21.15.120:0/3515703862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aafaf3370 msgr2=0x557aafaf5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:18.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.517+0000 7fa1ff7fe640 1 --2- 172.21.15.120:0/3515703862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aafaf3370 0x557aafaf5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.517+0000 7fa1ff7fe640 1 -- 172.21.15.120:0/3515703862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aafae85d0 msgr2=0x557aafae8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:18.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.517+0000 7fa1ff7fe640 1 --2- 172.21.15.120:0/3515703862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aafae85d0 0x557aafae8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.517+0000 7fa1ff7fe640 1 -- 172.21.15.120:0/3515703862 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557aaf9da3a0 con 0x557aafae9b40 2024-09-19T09:05:18.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.517+0000 7fa1feffd640 1 --2- 172.21.15.120:0/3515703862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aafae85d0 0x557aafae8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:18.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.518+0000 7fa1ff7fe640 1 --2- 172.21.15.120:0/3515703862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aafae9b40 0x557aafae8090 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8031f80 tx=0x7fa1f8005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151145d9edbd1647 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:18.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.518+0000 7fa1f57fa640 1 -- 172.21.15.120:0/3515703862 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8003b70 con 0x557aafae9b40 2024-09-19T09:05:18.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.518+0000 7fa1f57fa640 1 -- 172.21.15.120:0/3515703862 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1f8036040 con 0x557aafae9b40 2024-09-19T09:05:18.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.518+0000 7fa1f57fa640 1 -- 172.21.15.120:0/3515703862 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8004590 con 0x557aafae9b40 2024-09-19T09:05:18.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.518+0000 7fa1f7fff640 1 --2- 172.21.15.120:0/2964813149 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa1ec0a3240 0x7fa1ec0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:18.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.518+0000 7fa1feffd640 1 --2- 172.21.15.120:0/2964813149 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa1ec0a3240 0x7fa1ec0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:18.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.518+0000 7fa1f7fff640 1 -- 172.21.15.120:0/2964813149 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa1e80cfc80 con 0x7fa200101360 2024-09-19T09:05:18.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.519+0000 7fa207778740 1 -- 172.21.15.120:0/3515703862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aafae9b40 msgr2=0x557aafae8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:18.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.519+0000 7fa1feffd640 1 --2- 172.21.15.120:0/2964813149 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa1ec0a3240 0x7fa1ec0a5700 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x557aafae5ac0 tx=0x7fa1f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:18.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.519+0000 7fa207778740 1 --2- 172.21.15.120:0/3515703862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aafae9b40 0x557aafae8090 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8031f80 tx=0x7fa1f8005ef0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.519+0000 7fa207778740 1 -- 172.21.15.120:0/3515703862 shutdown_connections 2024-09-19T09:05:18.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.519+0000 7fa207778740 1 --2- 172.21.15.120:0/3515703862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aafaf3370 0x557aafaf5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.519+0000 7fa207778740 1 --2- 172.21.15.120:0/3515703862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aafae85d0 0x557aafae8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.519+0000 7fa207778740 1 --2- 172.21.15.120:0/3515703862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aafae9b40 0x557aafae8090 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.519+0000 7fa207778740 1 -- 172.21.15.120:0/3515703862 >> 172.21.15.120:0/3515703862 conn(0x557aafab1310 msgr2=0x557aafae4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:18.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.519+0000 7fa207778740 1 -- 172.21.15.120:0/3515703862 shutdown_connections 2024-09-19T09:05:18.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.519+0000 7fa207778740 1 -- 172.21.15.120:0/3515703862 wait complete. 2024-09-19T09:05:18.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.519+0000 7fa207778740 1 -- start start 2024-09-19T09:05:18.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.520+0000 7fa207778740 1 Processor -- start 2024-09-19T09:05:18.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.520+0000 7fa207778740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aafae85d0 0x557aafaf2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:18.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.520+0000 7fa207778740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aafae9b40 0x557aafaeffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:18.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.520+0000 7fa207778740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aafaf3370 0x557aafaf0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:18.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.520+0000 7fa207778740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557aafaf5e90 con 0x557aafae85d0 2024-09-19T09:05:18.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.520+0000 7fa207778740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557aafa09060 con 0x557aafae9b40 2024-09-19T09:05:18.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.520+0000 7fa207778740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557aafaf6010 con 0x557aafaf3370 2024-09-19T09:05:18.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.520+0000 7fa207778740 10 client.0 _ll_register_callbacks cb 0x557aaf75c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:18.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.520+0000 7fa1feffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aafae85d0 0x557aafaf2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:18.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.520+0000 7fa1fe7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aafae9b40 0x557aafaeffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:18.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.520+0000 7fa1feffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aafae85d0 0x557aafaf2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34326/0 (socket says 172.21.15.120:34326) 2024-09-19T09:05:18.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.520+0000 7fa1fe7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aafae9b40 0x557aafaeffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38412/0 (socket says 172.21.15.120:38412) 2024-09-19T09:05:18.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.520+0000 7fa1feffd640 1 -- 172.21.15.120:0/1113969580 learned_addr learned my addr 172.21.15.120:0/1113969580 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:18.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.520+0000 7fa1ff7fe640 1 --2- 172.21.15.120:0/1113969580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aafaf3370 0x557aafaf0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:18.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.520+0000 7fa1fe7fc640 1 -- 172.21.15.120:0/1113969580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aafaf3370 msgr2=0x557aafaf0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:18.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.520+0000 7fa1fe7fc640 1 --2- 172.21.15.120:0/1113969580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aafaf3370 0x557aafaf0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.520+0000 7fa1fe7fc640 1 -- 172.21.15.120:0/1113969580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aafae85d0 msgr2=0x557aafaf2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:18.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.520+0000 7fa1fe7fc640 1 --2- 172.21.15.120:0/1113969580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aafae85d0 0x557aafaf2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.520+0000 7fa1fe7fc640 1 -- 172.21.15.120:0/1113969580 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557aaf7bf3c0 con 0x557aafae9b40 2024-09-19T09:05:18.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.520+0000 7fa1feffd640 1 --2- 172.21.15.120:0/1113969580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aafae85d0 0x557aafaf2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:18.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.521+0000 7fa1fe7fc640 1 --2- 172.21.15.120:0/1113969580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aafae9b40 0x557aafaeffc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x557aafae9070 tx=0x7fa1e809c070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fed734e7ca4bda9a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:18.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.521+0000 7fa1e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:05:18.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.521+0000 7fa1e67fc640 1 -- 172.21.15.120:0/1113969580 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e809c9a0 con 0x557aafae9b40 2024-09-19T09:05:18.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.521+0000 7fa1e67fc640 1 -- 172.21.15.120:0/1113969580 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa1e80b5190 con 0x557aafae9b40 2024-09-19T09:05:18.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.521+0000 7fa207778740 1 -- 172.21.15.120:0/1113969580 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1f8036040 con 0x557aafae9b40 2024-09-19T09:05:18.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.521+0000 7fa1e67fc640 1 -- 172.21.15.120:0/1113969580 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa1e809c2a0 con 0x557aafae9b40 2024-09-19T09:05:18.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.521+0000 7fa207778740 10 client.25529 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:18.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.521+0000 7fa207778740 1 -- 172.21.15.120:0/1113969580 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557aafb018e0 con 0x557aafae9b40 2024-09-19T09:05:18.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.521+0000 7fa1e67fc640 1 -- 172.21.15.120:0/1113969580 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1f8036040 con 0x557aafae9b40 2024-09-19T09:05:18.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.521+0000 7fa1e67fc640 1 -- 172.21.15.120:0/1113969580 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa1e80bc070 con 0x557aafae9b40 2024-09-19T09:05:18.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.521+0000 7fa207778740 10 client.25529 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:18.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.521+0000 7fa207778740 10 client.25529 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:05:18.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.521+0000 7fa207778740 20 client.25529 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:18.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.521+0000 7fa207778740 1 --2- 172.21.15.120:0/1113969580 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x557aafb02d40 0x557aafb05150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:18.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.521+0000 7fa207778740 4 client.25529 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:05:18.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.521+0000 7fa207778740 1 -- 172.21.15.120:0/1113969580 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557aafa5b860 con 0x557aafb02d40 2024-09-19T09:05:18.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.522+0000 7fa1feffd640 1 --2- 172.21.15.120:0/1113969580 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x557aafb02d40 0x557aafb05150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:18.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.522+0000 7fa1e67fc640 10 client.25529 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:05:18.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.522+0000 7fa1feffd640 1 --2- 172.21.15.120:0/1113969580 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x557aafb02d40 0x557aafb05150 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bd0181aee7ced8a6 server_cookie=b9b5379269d4d387 in_seq=0 out_seq=0 2024-09-19T09:05:18.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.522+0000 7fa1e67fc640 1 -- 172.21.15.120:0/1113969580 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557aafa5b860 con 0x557aafb02d40 2024-09-19T09:05:18.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.523+0000 7fa1e67fc640 10 client.25529 handle_command_reply: tid=0 2024-09-19T09:05:18.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.523+0000 7fa207778740 1 client.25529 shutdown 2024-09-19T09:05:18.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.523+0000 7fa207778740 1 -- 172.21.15.120:0/1113969580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aafae9b40 msgr2=0x557aafaeffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:18.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.523+0000 7fa207778740 1 --2- 172.21.15.120:0/1113969580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aafae9b40 0x557aafaeffc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x557aafae9070 tx=0x7fa1e809c070 comp rx=0 tx=0).stop 2024-09-19T09:05:18.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.523+0000 7fa207778740 1 -- 172.21.15.120:0/1113969580 shutdown_connections 2024-09-19T09:05:18.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.523+0000 7fa207778740 1 --2- 172.21.15.120:0/1113969580 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x557aafb02d40 0x557aafb05150 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.523+0000 7fa207778740 1 --2- 172.21.15.120:0/1113969580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aafaf3370 0x557aafaf0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.523+0000 7fa207778740 1 --2- 172.21.15.120:0/1113969580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aafae9b40 0x557aafaeffc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.523+0000 7fa1e67fc640 0 client.25529 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:05:18.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.523+0000 7fa207778740 1 --2- 172.21.15.120:0/1113969580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aafae85d0 0x557aafaf2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.523+0000 7fa207778740 1 -- 172.21.15.120:0/1113969580 >> 172.21.15.120:0/1113969580 conn(0x557aafab1310 msgr2=0x557aafaea930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:18.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.523+0000 7fa207778740 1 -- 172.21.15.120:0/1113969580 shutdown_connections 2024-09-19T09:05:18.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.523+0000 7fa207778740 1 -- 172.21.15.120:0/1113969580 wait complete. 2024-09-19T09:05:18.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.523+0000 7fa207778740 20 client.25529 trim_cache size 0 max 16384 2024-09-19T09:05:18.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.537+0000 7fa207778740 1 Processor -- start 2024-09-19T09:05:18.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.537+0000 7fa207778740 1 -- start start 2024-09-19T09:05:18.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.537+0000 7fa207778740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aafae85d0 0x557aafaff130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:18.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.537+0000 7fa207778740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aafae9b40 0x557aafb0aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:18.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.537+0000 7fa207778740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aafaf3370 0x557aafb0af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:18.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.537+0000 7fa207778740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa2001af720 con 0x557aafae85d0 2024-09-19T09:05:18.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.537+0000 7fa207778740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa2001b1260 con 0x557aafaf3370 2024-09-19T09:05:18.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.537+0000 7fa207778740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa2001afdf0 con 0x557aafae9b40 2024-09-19T09:05:18.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.537+0000 7fa1feffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aafae85d0 0x557aafaff130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:18.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.537+0000 7fa1feffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aafae85d0 0x557aafaff130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34342/0 (socket says 172.21.15.120:34342) 2024-09-19T09:05:18.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.537+0000 7fa1feffd640 1 -- 172.21.15.120:0/1788620525 learned_addr learned my addr 172.21.15.120:0/1788620525 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:18.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.537+0000 7fa1ff7fe640 1 --2- 172.21.15.120:0/1788620525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aafaf3370 0x557aafb0af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:18.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.537+0000 7fa1fe7fc640 1 --2- 172.21.15.120:0/1788620525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aafae9b40 0x557aafb0aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:18.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.538+0000 7fa1feffd640 1 -- 172.21.15.120:0/1788620525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aafae9b40 msgr2=0x557aafb0aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:18.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.538+0000 7fa1feffd640 1 --2- 172.21.15.120:0/1788620525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aafae9b40 0x557aafb0aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.538+0000 7fa1feffd640 1 -- 172.21.15.120:0/1788620525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aafaf3370 msgr2=0x557aafb0af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:18.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.538+0000 7fa1feffd640 1 --2- 172.21.15.120:0/1788620525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aafaf3370 0x557aafb0af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.538+0000 7fa1feffd640 1 -- 172.21.15.120:0/1788620525 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557aafaf08d0 con 0x557aafae85d0 2024-09-19T09:05:18.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.538+0000 7fa1fe7fc640 1 --2- 172.21.15.120:0/1788620525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aafae9b40 0x557aafb0aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:18.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.538+0000 7fa1feffd640 1 --2- 172.21.15.120:0/1788620525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aafae85d0 0x557aafaff130 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0012cb0 tx=0x7fa1f001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7c312f09130c85 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:18.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.538+0000 7fa1f4ff9640 1 -- 172.21.15.120:0/1788620525 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0005020 con 0x557aafae85d0 2024-09-19T09:05:18.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.538+0000 7fa1f4ff9640 1 -- 172.21.15.120:0/1788620525 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1f00047a0 con 0x557aafae85d0 2024-09-19T09:05:18.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.538+0000 7fa1f4ff9640 1 -- 172.21.15.120:0/1788620525 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f002b7b0 con 0x557aafae85d0 2024-09-19T09:05:18.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.539+0000 7fa207778740 1 -- 172.21.15.120:0/1788620525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aafae85d0 msgr2=0x557aafaff130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:18.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.539+0000 7fa207778740 1 --2- 172.21.15.120:0/1788620525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aafae85d0 0x557aafaff130 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0012cb0 tx=0x7fa1f001b000 comp rx=0 tx=0).stop 2024-09-19T09:05:18.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.539+0000 7fa207778740 1 -- 172.21.15.120:0/1788620525 shutdown_connections 2024-09-19T09:05:18.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.539+0000 7fa207778740 1 --2- 172.21.15.120:0/1788620525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aafaf3370 0x557aafb0af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.539+0000 7fa207778740 1 --2- 172.21.15.120:0/1788620525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aafae9b40 0x557aafb0aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.539+0000 7fa207778740 1 --2- 172.21.15.120:0/1788620525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aafae85d0 0x557aafaff130 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.539+0000 7fa207778740 1 -- 172.21.15.120:0/1788620525 >> 172.21.15.120:0/1788620525 conn(0x557aafab1310 msgr2=0x557aafaee890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:18.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.539+0000 7fa207778740 1 -- 172.21.15.120:0/1788620525 shutdown_connections 2024-09-19T09:05:18.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.539+0000 7fa207778740 1 -- 172.21.15.120:0/1788620525 wait complete. 2024-09-19T09:05:18.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.539+0000 7fa207778740 1 -- start start 2024-09-19T09:05:18.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.539+0000 7fa207778740 1 Processor -- start 2024-09-19T09:05:18.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.540+0000 7fa207778740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aafae85d0 0x557aafafbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:18.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.540+0000 7fa207778740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aafae9b40 0x557aafaece10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:18.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.540+0000 7fa207778740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aafaf3370 0x557aafaed420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:18.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.540+0000 7fa207778740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa2001b0b90 con 0x557aafae85d0 2024-09-19T09:05:18.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.540+0000 7fa207778740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa2001b04c0 con 0x557aafaf3370 2024-09-19T09:05:18.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.540+0000 7fa207778740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa2001ad6e0 con 0x557aafae9b40 2024-09-19T09:05:18.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.540+0000 7fa207778740 10 client.0 _ll_register_callbacks cb 0x7fa1e800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:18.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.540+0000 7fa1feffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aafae85d0 0x557aafafbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:18.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.540+0000 7fa1ff7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aafaf3370 0x557aafaed420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:18.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.540+0000 7fa1feffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aafae85d0 0x557aafafbb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34354/0 (socket says 172.21.15.120:34354) 2024-09-19T09:05:18.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.540+0000 7fa1feffd640 1 -- 172.21.15.120:0/3791631371 learned_addr learned my addr 172.21.15.120:0/3791631371 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:18.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.540+0000 7fa1fe7fc640 1 --2- 172.21.15.120:0/3791631371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aafae9b40 0x557aafaece10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:18.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.540+0000 7fa1feffd640 1 -- 172.21.15.120:0/3791631371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aafae9b40 msgr2=0x557aafaece10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:18.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.540+0000 7fa1feffd640 1 --2- 172.21.15.120:0/3791631371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aafae9b40 0x557aafaece10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.540+0000 7fa1feffd640 1 -- 172.21.15.120:0/3791631371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aafaf3370 msgr2=0x557aafaed420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:18.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.540+0000 7fa1feffd640 1 --2- 172.21.15.120:0/3791631371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aafaf3370 0x557aafaed420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.540+0000 7fa1feffd640 1 -- 172.21.15.120:0/3791631371 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557aafaedc50 con 0x557aafae85d0 2024-09-19T09:05:18.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.540+0000 7fa1ff7fe640 1 --2- 172.21.15.120:0/3791631371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aafaf3370 0x557aafaed420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:18.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.540+0000 7fa1feffd640 1 --2- 172.21.15.120:0/3791631371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aafae85d0 0x557aafafbb20 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0005890 tx=0x7fa1f00043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f556b03e475b59a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:18.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.540+0000 7fa1e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:18.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.540+0000 7fa1e67fc640 1 -- 172.21.15.120:0/3791631371 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f002b860 con 0x557aafae85d0 2024-09-19T09:05:18.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.541+0000 7fa1e67fc640 1 -- 172.21.15.120:0/3791631371 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa1f002ba00 con 0x557aafae85d0 2024-09-19T09:05:18.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.541+0000 7fa207778740 1 -- 172.21.15.120:0/3791631371 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1f00047a0 con 0x557aafae85d0 2024-09-19T09:05:18.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.541+0000 7fa1e67fc640 1 -- 172.21.15.120:0/3791631371 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa1f002bc00 con 0x557aafae85d0 2024-09-19T09:05:18.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.541+0000 7fa207778740 10 client.16173 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:18.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.541+0000 7fa207778740 1 -- 172.21.15.120:0/3791631371 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557aafb6db20 con 0x557aafae85d0 2024-09-19T09:05:18.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.541+0000 7fa1e67fc640 1 -- 172.21.15.120:0/3791631371 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1f00047a0 con 0x557aafae85d0 2024-09-19T09:05:18.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.541+0000 7fa1e67fc640 1 -- 172.21.15.120:0/3791631371 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa1f0040740 con 0x557aafae85d0 2024-09-19T09:05:18.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.541+0000 7fa207778740 10 client.16173 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:18.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.541+0000 7fa207778740 10 client.16173 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:05:18.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.541+0000 7fa207778740 20 client.16173 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:18.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.541+0000 7fa207778740 1 --2- 172.21.15.120:0/3791631371 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x557aafb6e180 0x557aafb705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:18.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.541+0000 7fa207778740 4 client.16173 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:05:18.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.541+0000 7fa207778740 1 -- 172.21.15.120:0/3791631371 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557aafa55d70 con 0x557aafb6e180 2024-09-19T09:05:18.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.541+0000 7fa1fe7fc640 1 --2- 172.21.15.120:0/3791631371 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x557aafb6e180 0x557aafb705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:18.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.542+0000 7fa1fe7fc640 1 --2- 172.21.15.120:0/3791631371 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x557aafb6e180 0x557aafb705a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3ad8d0117525d938 server_cookie=a4dd221b5062631a in_seq=0 out_seq=0 2024-09-19T09:05:18.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.542+0000 7fa1e67fc640 10 client.16173 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:05:18.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.542+0000 7fa1e67fc640 1 -- 172.21.15.120:0/3791631371 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557aafa55d70 con 0x557aafb6e180 2024-09-19T09:05:18.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.542+0000 7fa1e67fc640 10 client.16173 handle_command_reply: tid=0 2024-09-19T09:05:18.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.542+0000 7fa207778740 1 client.16173 shutdown 2024-09-19T09:05:18.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.542+0000 7fa207778740 1 -- 172.21.15.120:0/3791631371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aafae85d0 msgr2=0x557aafafbb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:18.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.542+0000 7fa207778740 1 --2- 172.21.15.120:0/3791631371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aafae85d0 0x557aafafbb20 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0005890 tx=0x7fa1f00043d0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.542+0000 7fa207778740 1 -- 172.21.15.120:0/3791631371 shutdown_connections 2024-09-19T09:05:18.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.542+0000 7fa207778740 1 --2- 172.21.15.120:0/3791631371 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x557aafb6e180 0x557aafb705a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.542+0000 7fa207778740 1 --2- 172.21.15.120:0/3791631371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aafaf3370 0x557aafaed420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.542+0000 7fa207778740 1 --2- 172.21.15.120:0/3791631371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aafae9b40 0x557aafaece10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.542+0000 7fa207778740 1 --2- 172.21.15.120:0/3791631371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aafae85d0 0x557aafafbb20 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.542+0000 7fa207778740 1 -- 172.21.15.120:0/3791631371 >> 172.21.15.120:0/3791631371 conn(0x557aafab1310 msgr2=0x557aafae3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:18.566 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:05:18.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.542+0000 7fa1e67fc640 0 client.16173 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:05:18.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.542+0000 7fa207778740 1 -- 172.21.15.120:0/3791631371 shutdown_connections 2024-09-19T09:05:18.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.542+0000 7fa207778740 1 -- 172.21.15.120:0/3791631371 wait complete. 2024-09-19T09:05:18.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.542+0000 7fa207778740 20 client.16173 trim_cache size 0 max 16384 2024-09-19T09:05:18.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.543+0000 7fa20539f640 1 -- 172.21.15.120:0/2964813149 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa1ec0a3240 msgr2=0x7fa1ec0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:18.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.543+0000 7fa20539f640 1 --2- 172.21.15.120:0/2964813149 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa1ec0a3240 0x7fa1ec0a5700 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x557aafae5ac0 tx=0x7fa1f0005f90 comp rx=0 tx=0).stop 2024-09-19T09:05:18.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.543+0000 7fa20539f640 1 -- 172.21.15.120:0/2964813149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa200101360 msgr2=0x7fa20019ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:18.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.543+0000 7fa20539f640 1 --2- 172.21.15.120:0/2964813149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa200101360 0x7fa20019ac40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800e9f0 tx=0x7fa1e800edf0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.543+0000 7fa20539f640 1 -- 172.21.15.120:0/2964813149 shutdown_connections 2024-09-19T09:05:18.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.543+0000 7fa20539f640 1 --2- 172.21.15.120:0/2964813149 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa1ec0a3240 0x7fa1ec0a5700 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.543+0000 7fa20539f640 1 --2- 172.21.15.120:0/2964813149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa20010e4d0 0x7fa20019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.543+0000 7fa20539f640 1 --2- 172.21.15.120:0/2964813149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa200101360 0x7fa20019ac40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.543+0000 7fa20539f640 1 --2- 172.21.15.120:0/2964813149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2001009b0 0x7fa20019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:18.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.543+0000 7fa20539f640 1 -- 172.21.15.120:0/2964813149 >> 172.21.15.120:0/2964813149 conn(0x7fa2000fc7d0 msgr2=0x7fa2000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:18.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.543+0000 7fa20539f640 1 -- 172.21.15.120:0/2964813149 shutdown_connections 2024-09-19T09:05:18.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:18.543+0000 7fa20539f640 1 -- 172.21.15.120:0/2964813149 wait complete. 2024-09-19T09:05:19.038 INFO:tasks.workunit.client.0.smithi120.stdout: 1 98017 7.34 MB/sec warmup 74 sec latency 23.739 ms 2024-09-19T09:05:19.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:18 smithi120 ceph-mon[31150]: pgmap v414: 97 pgs: 97 active+clean; 32 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.0 MiB/s wr, 337 op/s 2024-09-19T09:05:19.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:18 smithi196 ceph-mon[35150]: pgmap v414: 97 pgs: 97 active+clean; 32 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.0 MiB/s wr, 337 op/s 2024-09-19T09:05:19.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:18 smithi153 ceph-mon[35071]: pgmap v414: 97 pgs: 97 active+clean; 32 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.0 MiB/s wr, 337 op/s 2024-09-19T09:05:19.367 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:05:19.625 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:05:20.038 INFO:tasks.workunit.client.0.smithi120.stdout: 1 100516 7.45 MB/sec warmup 75 sec latency 24.321 ms 2024-09-19T09:05:20.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.135+0000 7f66230f5640 1 -- 172.21.15.120:0/3011003834 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f661c102570 msgr2=0x7f661c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:20.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.135+0000 7f66230f5640 1 --2- 172.21.15.120:0/3011003834 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f661c102570 0x7f661c1029b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f660c009900 tx=0x7f660c02a480 comp rx=0 tx=0).stop 2024-09-19T09:05:20.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.135+0000 7f66230f5640 1 -- 172.21.15.120:0/3011003834 shutdown_connections 2024-09-19T09:05:20.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.135+0000 7f66230f5640 1 --2- 172.21.15.120:0/3011003834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f661c102ef0 0x7f661c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.135+0000 7f66230f5640 1 --2- 172.21.15.120:0/3011003834 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f661c102570 0x7f661c1029b0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.135+0000 7f66230f5640 1 --2- 172.21.15.120:0/3011003834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f661c100e20 0x7f661c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.135+0000 7f66230f5640 1 -- 172.21.15.120:0/3011003834 >> 172.21.15.120:0/3011003834 conn(0x7f661c0fc7d0 msgr2=0x7f661c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:20.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.135+0000 7f66230f5640 1 -- 172.21.15.120:0/3011003834 shutdown_connections 2024-09-19T09:05:20.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.135+0000 7f66230f5640 1 -- 172.21.15.120:0/3011003834 wait complete. 2024-09-19T09:05:20.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.136+0000 7f66230f5640 1 Processor -- start 2024-09-19T09:05:20.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.136+0000 7f66230f5640 1 -- start start 2024-09-19T09:05:20.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.136+0000 7f66230f5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f661c100e20 0x7f661c1973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:20.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.136+0000 7f66230f5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f661c102570 0x7f661c197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:20.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.136+0000 7f66230f5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f661c102ef0 0x7f661c19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:20.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.136+0000 7f66230f5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f661c110f80 con 0x7f661c102570 2024-09-19T09:05:20.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.136+0000 7f66230f5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f661c110e00 con 0x7f661c100e20 2024-09-19T09:05:20.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.136+0000 7f66230f5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f661c111100 con 0x7f661c102ef0 2024-09-19T09:05:20.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.136+0000 7f661bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f661c102570 0x7f661c197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:20.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.136+0000 7f661bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f661c102570 0x7f661c197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34364/0 (socket says 172.21.15.120:34364) 2024-09-19T09:05:20.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.136+0000 7f661bfff640 1 -- 172.21.15.120:0/209656810 learned_addr learned my addr 172.21.15.120:0/209656810 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:20.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.136+0000 7f6620e91640 1 --2- 172.21.15.120:0/209656810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f661c100e20 0x7f661c1973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:20.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.136+0000 7f6621692640 1 --2- 172.21.15.120:0/209656810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f661c102ef0 0x7f661c19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:20.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.136+0000 7f661bfff640 1 -- 172.21.15.120:0/209656810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f661c102ef0 msgr2=0x7f661c19bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:20.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.136+0000 7f661bfff640 1 --2- 172.21.15.120:0/209656810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f661c102ef0 0x7f661c19bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.136+0000 7f661bfff640 1 -- 172.21.15.120:0/209656810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f661c100e20 msgr2=0x7f661c1973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:20.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.136+0000 7f661bfff640 1 --2- 172.21.15.120:0/209656810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f661c100e20 0x7f661c1973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.136+0000 7f661bfff640 1 -- 172.21.15.120:0/209656810 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661c19c390 con 0x7f661c102570 2024-09-19T09:05:20.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.136+0000 7f6620e91640 1 --2- 172.21.15.120:0/209656810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f661c100e20 0x7f661c1973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:20.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.136+0000 7f661bfff640 1 --2- 172.21.15.120:0/209656810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f661c102570 0x7f661c197920 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f660c006010 tx=0x7f660c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:20.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.137+0000 7f6619ffb640 1 -- 172.21.15.120:0/209656810 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c03f070 con 0x7f661c102570 2024-09-19T09:05:20.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.137+0000 7f6619ffb640 1 -- 172.21.15.120:0/209656810 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f660c031c30 con 0x7f661c102570 2024-09-19T09:05:20.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.137+0000 7f66230f5640 1 -- 172.21.15.120:0/209656810 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f661c19c620 con 0x7f661c102570 2024-09-19T09:05:20.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.137+0000 7f6619ffb640 1 -- 172.21.15.120:0/209656810 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c035030 con 0x7f661c102570 2024-09-19T09:05:20.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.137+0000 7f66230f5640 1 -- 172.21.15.120:0/209656810 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f661c1a3f60 con 0x7f661c102570 2024-09-19T09:05:20.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.139+0000 7f6619ffb640 1 -- 172.21.15.120:0/209656810 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f660c042070 con 0x7f661c102570 2024-09-19T09:05:20.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.140+0000 7f66254ce740 1 Processor -- start 2024-09-19T09:05:20.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.140+0000 7f6619ffb640 1 --2- 172.21.15.120:0/209656810 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f66080a32c0 0x7f66080a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:20.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.140+0000 7f66254ce740 1 -- start start 2024-09-19T09:05:20.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.140+0000 7f6621692640 1 --2- 172.21.15.120:0/209656810 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f66080a32c0 0x7f66080a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:20.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.140+0000 7f6619ffb640 1 -- 172.21.15.120:0/209656810 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f660c0eff30 con 0x7f661c102570 2024-09-19T09:05:20.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.140+0000 7f66254ce740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab82a6cb40 0x55ab82a6b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:20.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.140+0000 7f66254ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab82a6b5d0 0x55ab82a6ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:20.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.140+0000 7f66254ce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab82a78580 0x55ab82a7a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:20.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.140+0000 7f66254ce740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ab829d12f0 con 0x55ab82a6cb40 2024-09-19T09:05:20.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.140+0000 7f66254ce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ab82978cf0 con 0x55ab82a78580 2024-09-19T09:05:20.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.140+0000 7f66254ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ab829cd2b0 con 0x55ab82a6b5d0 2024-09-19T09:05:20.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.140+0000 7f6620e91640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab82a6cb40 0x55ab82a6b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:20.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.140+0000 7f6620e91640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab82a6cb40 0x55ab82a6b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34372/0 (socket says 172.21.15.120:34372) 2024-09-19T09:05:20.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.140+0000 7f6620e91640 1 -- 172.21.15.120:0/603945735 learned_addr learned my addr 172.21.15.120:0/603945735 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:20.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.140+0000 7f6621692640 1 --2- 172.21.15.120:0/603945735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab82a78580 0x55ab82a7a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:20.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.140+0000 7f661bfff640 1 --2- 172.21.15.120:0/603945735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab82a6b5d0 0x55ab82a6ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:20.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.141+0000 7f6621692640 1 --2- 172.21.15.120:0/209656810 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f66080a32c0 0x7f66080a5780 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6604004650 tx=0x7f660400be60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:20.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.141+0000 7f6620e91640 1 -- 172.21.15.120:0/603945735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab82a6b5d0 msgr2=0x55ab82a6ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:20.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.141+0000 7f6620e91640 1 --2- 172.21.15.120:0/603945735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab82a6b5d0 0x55ab82a6ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.141+0000 7f6620e91640 1 -- 172.21.15.120:0/603945735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab82a78580 msgr2=0x55ab82a7a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:20.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.141+0000 7f6620e91640 1 --2- 172.21.15.120:0/603945735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab82a78580 0x55ab82a7a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.141+0000 7f6620e91640 1 -- 172.21.15.120:0/603945735 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab8295d3a0 con 0x55ab82a6cb40 2024-09-19T09:05:20.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.141+0000 7f6621692640 1 --2- 172.21.15.120:0/603945735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab82a78580 0x55ab82a7a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:20.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.141+0000 7f661bfff640 1 --2- 172.21.15.120:0/603945735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab82a6b5d0 0x55ab82a6ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:20.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.141+0000 7f6620e91640 1 --2- 172.21.15.120:0/603945735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab82a6cb40 0x55ab82a6b090 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6614005f40 tx=0x7f6614011c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ed3858b26bd05 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:20.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.141+0000 7f6612ffd640 1 -- 172.21.15.120:0/603945735 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6614011e60 con 0x55ab82a6cb40 2024-09-19T09:05:20.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.141+0000 7f6612ffd640 1 -- 172.21.15.120:0/603945735 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f661400fe80 con 0x55ab82a6cb40 2024-09-19T09:05:20.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.141+0000 7f6612ffd640 1 -- 172.21.15.120:0/603945735 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6614010e50 con 0x55ab82a6cb40 2024-09-19T09:05:20.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.142+0000 7f66254ce740 1 -- 172.21.15.120:0/603945735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab82a6cb40 msgr2=0x55ab82a6b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:20.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.142+0000 7f66254ce740 1 --2- 172.21.15.120:0/603945735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab82a6cb40 0x55ab82a6b090 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6614005f40 tx=0x7f6614011c70 comp rx=0 tx=0).stop 2024-09-19T09:05:20.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.142+0000 7f66254ce740 1 -- 172.21.15.120:0/603945735 shutdown_connections 2024-09-19T09:05:20.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.142+0000 7f66254ce740 1 --2- 172.21.15.120:0/603945735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab82a78580 0x55ab82a7a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.142+0000 7f66254ce740 1 --2- 172.21.15.120:0/603945735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab82a6b5d0 0x55ab82a6ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.142+0000 7f66254ce740 1 --2- 172.21.15.120:0/603945735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab82a6cb40 0x55ab82a6b090 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.142+0000 7f66254ce740 1 -- 172.21.15.120:0/603945735 >> 172.21.15.120:0/603945735 conn(0x55ab82a34310 msgr2=0x55ab82a67c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:20.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.142+0000 7f66254ce740 1 -- 172.21.15.120:0/603945735 shutdown_connections 2024-09-19T09:05:20.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.142+0000 7f66254ce740 1 -- 172.21.15.120:0/603945735 wait complete. 2024-09-19T09:05:20.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.142+0000 7f66254ce740 1 -- start start 2024-09-19T09:05:20.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.143+0000 7f66254ce740 1 Processor -- start 2024-09-19T09:05:20.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.143+0000 7f66254ce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab82a6b5d0 0x55ab82a73660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:20.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.143+0000 7f66254ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab82a6cb40 0x55ab82a73ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:20.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.143+0000 7f66254ce740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab82a78580 0x55ab82a740e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:20.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.143+0000 7f66254ce740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ab82a6e550 con 0x55ab82a78580 2024-09-19T09:05:20.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.143+0000 7f66254ce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ab8298c060 con 0x55ab82a6b5d0 2024-09-19T09:05:20.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.143+0000 7f66254ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ab82a6e6d0 con 0x55ab82a6cb40 2024-09-19T09:05:20.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.143+0000 7f66254ce740 10 client.0 _ll_register_callbacks cb 0x55ab826df8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:20.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.143+0000 7f6621692640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab82a78580 0x55ab82a740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:20.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.143+0000 7f661bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab82a6cb40 0x55ab82a73ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:20.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.143+0000 7f6620e91640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab82a6b5d0 0x55ab82a73660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:20.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.143+0000 7f6621692640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab82a78580 0x55ab82a740e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34382/0 (socket says 172.21.15.120:34382) 2024-09-19T09:05:20.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.143+0000 7f661bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab82a6cb40 0x55ab82a73ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:58958/0 (socket says 172.21.15.120:58958) 2024-09-19T09:05:20.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.143+0000 7f6621692640 1 -- 172.21.15.120:0/3994581513 learned_addr learned my addr 172.21.15.120:0/3994581513 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:20.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.144+0000 7f6620e91640 1 -- 172.21.15.120:0/3994581513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab82a6cb40 msgr2=0x55ab82a73ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:20.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.144+0000 7f6620e91640 1 --2- 172.21.15.120:0/3994581513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab82a6cb40 0x55ab82a73ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.144+0000 7f6620e91640 1 -- 172.21.15.120:0/3994581513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab82a78580 msgr2=0x55ab82a740e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:20.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.144+0000 7f6620e91640 1 --2- 172.21.15.120:0/3994581513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab82a78580 0x55ab82a740e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.144+0000 7f6620e91640 1 -- 172.21.15.120:0/3994581513 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab827423c0 con 0x55ab82a6b5d0 2024-09-19T09:05:20.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.144+0000 7f6621692640 1 --2- 172.21.15.120:0/3994581513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab82a78580 0x55ab82a740e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:20.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.144+0000 7f6620e91640 1 --2- 172.21.15.120:0/3994581513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab82a6b5d0 0x55ab82a73660 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6614006010 tx=0x7f6614011590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=270b3eb18a559f28 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:20.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.144+0000 7f65fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:05:20.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.144+0000 7f65fbfff640 1 -- 172.21.15.120:0/3994581513 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6614028020 con 0x55ab82a6b5d0 2024-09-19T09:05:20.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.144+0000 7f66254ce740 1 -- 172.21.15.120:0/3994581513 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ab829de860 con 0x55ab82a6b5d0 2024-09-19T09:05:20.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.144+0000 7f65fbfff640 1 -- 172.21.15.120:0/3994581513 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f6614010de0 con 0x55ab82a6b5d0 2024-09-19T09:05:20.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.144+0000 7f65fbfff640 1 -- 172.21.15.120:0/3994581513 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f661400f9a0 con 0x55ab82a6b5d0 2024-09-19T09:05:20.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.144+0000 7f66254ce740 10 client.25547 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:20.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.144+0000 7f66254ce740 1 -- 172.21.15.120:0/3994581513 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ab82a88ef0 con 0x55ab82a6b5d0 2024-09-19T09:05:20.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.144+0000 7f65fbfff640 1 -- 172.21.15.120:0/3994581513 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ab829de860 con 0x55ab82a6b5d0 2024-09-19T09:05:20.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.145+0000 7f65fbfff640 1 -- 172.21.15.120:0/3994581513 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6614018070 con 0x55ab82a6b5d0 2024-09-19T09:05:20.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.145+0000 7f66254ce740 10 client.25547 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:20.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.145+0000 7f66254ce740 10 client.25547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:20.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.145+0000 7f66254ce740 20 client.25547 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:20.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.145+0000 7f66254ce740 1 --2- 172.21.15.120:0/3994581513 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ab82a8a2a0 0x55ab82a8c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:20.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.145+0000 7f66254ce740 4 client.25547 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:05:20.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.145+0000 7f66254ce740 1 -- 172.21.15.120:0/3994581513 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab829d8d70 con 0x55ab82a8a2a0 2024-09-19T09:05:20.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.145+0000 7f661bfff640 1 --2- 172.21.15.120:0/3994581513 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ab82a8a2a0 0x55ab82a8c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:20.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.145+0000 7f65fbfff640 10 client.25547 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:20.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.145+0000 7f661bfff640 1 --2- 172.21.15.120:0/3994581513 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ab82a8a2a0 0x55ab82a8c690 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c515ec35305dae6 server_cookie=f8a0b2d319f97b28 in_seq=0 out_seq=0 2024-09-19T09:05:20.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.146+0000 7f65fbfff640 1 -- 172.21.15.120:0/3994581513 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ab829d8d70 con 0x55ab82a8a2a0 2024-09-19T09:05:20.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.146+0000 7f65fbfff640 10 client.25547 handle_command_reply: tid=0 2024-09-19T09:05:20.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.146+0000 7f66254ce740 1 client.25547 shutdown 2024-09-19T09:05:20.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.147+0000 7f66254ce740 1 -- 172.21.15.120:0/3994581513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab82a6b5d0 msgr2=0x55ab82a73660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:20.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.147+0000 7f66254ce740 1 --2- 172.21.15.120:0/3994581513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab82a6b5d0 0x55ab82a73660 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6614006010 tx=0x7f6614011590 comp rx=0 tx=0).stop 2024-09-19T09:05:20.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.147+0000 7f66254ce740 1 -- 172.21.15.120:0/3994581513 shutdown_connections 2024-09-19T09:05:20.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.147+0000 7f66254ce740 1 --2- 172.21.15.120:0/3994581513 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ab82a8a2a0 0x55ab82a8c690 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.147+0000 7f66254ce740 1 --2- 172.21.15.120:0/3994581513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab82a78580 0x55ab82a740e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.147+0000 7f66254ce740 1 --2- 172.21.15.120:0/3994581513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab82a6cb40 0x55ab82a73ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.147+0000 7f65fbfff640 0 client.25547 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:20.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.147+0000 7f66254ce740 1 --2- 172.21.15.120:0/3994581513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab82a6b5d0 0x55ab82a73660 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.147+0000 7f66254ce740 1 -- 172.21.15.120:0/3994581513 >> 172.21.15.120:0/3994581513 conn(0x55ab82a34310 msgr2=0x55ab82a6d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:20.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.147+0000 7f66254ce740 1 -- 172.21.15.120:0/3994581513 shutdown_connections 2024-09-19T09:05:20.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.147+0000 7f66254ce740 1 -- 172.21.15.120:0/3994581513 wait complete. 2024-09-19T09:05:20.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.147+0000 7f66254ce740 20 client.25547 trim_cache size 0 max 16384 2024-09-19T09:05:20.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.163+0000 7f66254ce740 1 Processor -- start 2024-09-19T09:05:20.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.163+0000 7f66254ce740 1 -- start start 2024-09-19T09:05:20.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.163+0000 7f66254ce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab82a6b5d0 0x55ab82a91e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:20.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.163+0000 7f66254ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab82a6cb40 0x55ab82a92340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:20.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.163+0000 7f66254ce740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab82a78580 0x55ab82a708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:20.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.163+0000 7f66254ce740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f661c1ab250 con 0x55ab82a78580 2024-09-19T09:05:20.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.163+0000 7f66254ce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f661c1acd90 con 0x55ab82a6b5d0 2024-09-19T09:05:20.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.163+0000 7f66254ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f661c1ab920 con 0x55ab82a6cb40 2024-09-19T09:05:20.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.163+0000 7f6620e91640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab82a6b5d0 0x55ab82a91e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:20.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.163+0000 7f661bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab82a6cb40 0x55ab82a92340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:20.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.163+0000 7f6621692640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab82a78580 0x55ab82a708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:20.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.163+0000 7f6620e91640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab82a6b5d0 0x55ab82a91e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38490/0 (socket says 172.21.15.120:38490) 2024-09-19T09:05:20.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.163+0000 7f6620e91640 1 -- 172.21.15.120:0/4027500482 learned_addr learned my addr 172.21.15.120:0/4027500482 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:20.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.163+0000 7f661bfff640 1 -- 172.21.15.120:0/4027500482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab82a6b5d0 msgr2=0x55ab82a91e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:20.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.163+0000 7f661bfff640 1 --2- 172.21.15.120:0/4027500482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab82a6b5d0 0x55ab82a91e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.163+0000 7f661bfff640 1 -- 172.21.15.120:0/4027500482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab82a78580 msgr2=0x55ab82a708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:20.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.163+0000 7f661bfff640 1 --2- 172.21.15.120:0/4027500482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab82a78580 0x55ab82a708c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.164+0000 7f661bfff640 1 -- 172.21.15.120:0/4027500482 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab82a68960 con 0x55ab82a6cb40 2024-09-19T09:05:20.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.164+0000 7f661bfff640 1 --2- 172.21.15.120:0/4027500482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab82a6cb40 0x55ab82a92340 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f660c0ed9f0 tx=0x7f660c03b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=892bad2e80413330 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:20.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.164+0000 7f66127fc640 1 -- 172.21.15.120:0/4027500482 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c049040 con 0x55ab82a6cb40 2024-09-19T09:05:20.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.164+0000 7f66127fc640 1 -- 172.21.15.120:0/4027500482 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f660c0c5180 con 0x55ab82a6cb40 2024-09-19T09:05:20.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.164+0000 7f66127fc640 1 -- 172.21.15.120:0/4027500482 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c03b580 con 0x55ab82a6cb40 2024-09-19T09:05:20.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.165+0000 7f66254ce740 1 -- 172.21.15.120:0/4027500482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab82a6cb40 msgr2=0x55ab82a92340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:20.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.165+0000 7f66254ce740 1 --2- 172.21.15.120:0/4027500482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab82a6cb40 0x55ab82a92340 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f660c0ed9f0 tx=0x7f660c03b790 comp rx=0 tx=0).stop 2024-09-19T09:05:20.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.165+0000 7f66254ce740 1 -- 172.21.15.120:0/4027500482 shutdown_connections 2024-09-19T09:05:20.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.165+0000 7f66254ce740 1 --2- 172.21.15.120:0/4027500482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab82a78580 0x55ab82a708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.165+0000 7f66254ce740 1 --2- 172.21.15.120:0/4027500482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab82a6cb40 0x55ab82a92340 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.165+0000 7f66254ce740 1 --2- 172.21.15.120:0/4027500482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab82a6b5d0 0x55ab82a91e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.165+0000 7f66254ce740 1 -- 172.21.15.120:0/4027500482 >> 172.21.15.120:0/4027500482 conn(0x55ab82a34310 msgr2=0x55ab82a66d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:20.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.165+0000 7f66254ce740 1 -- 172.21.15.120:0/4027500482 shutdown_connections 2024-09-19T09:05:20.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.165+0000 7f66254ce740 1 -- 172.21.15.120:0/4027500482 wait complete. 2024-09-19T09:05:20.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.166+0000 7f66254ce740 1 -- start start 2024-09-19T09:05:20.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.166+0000 7f66254ce740 1 Processor -- start 2024-09-19T09:05:20.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.166+0000 7f66254ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab82a6b5d0 0x55ab82a85b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:20.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.166+0000 7f66254ce740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab82a6cb40 0x55ab82a80ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:20.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.166+0000 7f66254ce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab82a78580 0x55ab82a81420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:20.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.166+0000 7f66254ce740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f661c1ac6c0 con 0x55ab82a6cb40 2024-09-19T09:05:20.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.166+0000 7f66254ce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f661c1abff0 con 0x55ab82a78580 2024-09-19T09:05:20.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.166+0000 7f66254ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f661c1a9210 con 0x55ab82a6b5d0 2024-09-19T09:05:20.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.166+0000 7f66254ce740 10 client.0 _ll_register_callbacks cb 0x7f660c005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:20.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.167+0000 7f661bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab82a6cb40 0x55ab82a80ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:20.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.167+0000 7f661bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab82a6cb40 0x55ab82a80ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34386/0 (socket says 172.21.15.120:34386) 2024-09-19T09:05:20.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.167+0000 7f661bfff640 1 -- 172.21.15.120:0/2315230297 learned_addr learned my addr 172.21.15.120:0/2315230297 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:20.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.167+0000 7f6621692640 1 --2- 172.21.15.120:0/2315230297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab82a78580 0x55ab82a81420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:20.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.167+0000 7f6620e91640 1 --2- 172.21.15.120:0/2315230297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab82a6b5d0 0x55ab82a85b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:20.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.167+0000 7f661bfff640 1 -- 172.21.15.120:0/2315230297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab82a6b5d0 msgr2=0x55ab82a85b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:20.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.167+0000 7f661bfff640 1 --2- 172.21.15.120:0/2315230297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab82a6b5d0 0x55ab82a85b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.167+0000 7f661bfff640 1 -- 172.21.15.120:0/2315230297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab82a78580 msgr2=0x55ab82a81420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:20.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.167+0000 7f661bfff640 1 --2- 172.21.15.120:0/2315230297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab82a78580 0x55ab82a81420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.167+0000 7f661bfff640 1 -- 172.21.15.120:0/2315230297 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab82a81a10 con 0x55ab82a6cb40 2024-09-19T09:05:20.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.167+0000 7f661bfff640 1 --2- 172.21.15.120:0/2315230297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab82a6cb40 0x55ab82a80ee0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f660c0c4c50 tx=0x7f660c0ba6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d0c97193709864 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:20.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.167+0000 7f65fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:20.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.168+0000 7f65fbfff640 1 -- 172.21.15.120:0/2315230297 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c049070 con 0x55ab82a6cb40 2024-09-19T09:05:20.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.168+0000 7f65fbfff640 1 -- 172.21.15.120:0/2315230297 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f660c0b8c30 con 0x55ab82a6cb40 2024-09-19T09:05:20.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.168+0000 7f66254ce740 1 -- 172.21.15.120:0/2315230297 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f660c0c5180 con 0x55ab82a6cb40 2024-09-19T09:05:20.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.168+0000 7f65fbfff640 1 -- 172.21.15.120:0/2315230297 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f660c0da050 con 0x55ab82a6cb40 2024-09-19T09:05:20.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.168+0000 7f66254ce740 10 client.16191 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:20.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.168+0000 7f66254ce740 1 -- 172.21.15.120:0/2315230297 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ab82a7b040 con 0x55ab82a6cb40 2024-09-19T09:05:20.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.168+0000 7f65fbfff640 1 -- 172.21.15.120:0/2315230297 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f660c0c5180 con 0x55ab82a6cb40 2024-09-19T09:05:20.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.168+0000 7f65fbfff640 1 -- 172.21.15.120:0/2315230297 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f660c0bba10 con 0x55ab82a6cb40 2024-09-19T09:05:20.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.168+0000 7f66254ce740 10 client.16191 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:20.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.168+0000 7f66254ce740 10 client.16191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:20.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.168+0000 7f66254ce740 20 client.16191 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:20.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.168+0000 7f66254ce740 1 --2- 172.21.15.120:0/2315230297 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ab82a7b6f0 0x55ab82a7db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:20.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.168+0000 7f66254ce740 4 client.16191 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:05:20.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.168+0000 7f66254ce740 1 -- 172.21.15.120:0/2315230297 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab82955170 con 0x55ab82a7b6f0 2024-09-19T09:05:20.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.169+0000 7f6620e91640 1 --2- 172.21.15.120:0/2315230297 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ab82a7b6f0 0x55ab82a7db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:20.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.170+0000 7f65fbfff640 10 client.16191 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:20.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.170+0000 7f6620e91640 1 --2- 172.21.15.120:0/2315230297 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ab82a7b6f0 0x55ab82a7db10 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ae88969458b1f70 server_cookie=bd01db1f95ad8651 in_seq=0 out_seq=0 2024-09-19T09:05:20.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.171+0000 7f65fbfff640 1 -- 172.21.15.120:0/2315230297 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab82955170 con 0x55ab82a7b6f0 2024-09-19T09:05:20.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.171+0000 7f65fbfff640 10 client.16191 handle_command_reply: tid=0 2024-09-19T09:05:20.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.171+0000 7f66254ce740 1 client.16191 shutdown 2024-09-19T09:05:20.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.171+0000 7f66254ce740 1 -- 172.21.15.120:0/2315230297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab82a6cb40 msgr2=0x55ab82a80ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:20.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.171+0000 7f66254ce740 1 --2- 172.21.15.120:0/2315230297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab82a6cb40 0x55ab82a80ee0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f660c0c4c50 tx=0x7f660c0ba6d0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.171+0000 7f66254ce740 1 -- 172.21.15.120:0/2315230297 shutdown_connections 2024-09-19T09:05:20.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.171+0000 7f66254ce740 1 --2- 172.21.15.120:0/2315230297 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ab82a7b6f0 0x55ab82a7db10 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.171+0000 7f66254ce740 1 --2- 172.21.15.120:0/2315230297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab82a78580 0x55ab82a81420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.171+0000 7f66254ce740 1 --2- 172.21.15.120:0/2315230297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab82a6cb40 0x55ab82a80ee0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.171+0000 7f66254ce740 1 --2- 172.21.15.120:0/2315230297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab82a6b5d0 0x55ab82a85b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.171+0000 7f65fbfff640 0 client.16191 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:20.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.171+0000 7f66254ce740 1 -- 172.21.15.120:0/2315230297 >> 172.21.15.120:0/2315230297 conn(0x55ab82a34310 msgr2=0x55ab82a6d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:20.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.172+0000 7f66254ce740 1 -- 172.21.15.120:0/2315230297 shutdown_connections 2024-09-19T09:05:20.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.172+0000 7f66254ce740 1 -- 172.21.15.120:0/2315230297 wait complete. 2024-09-19T09:05:20.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.172+0000 7f66254ce740 20 client.16191 trim_cache size 0 max 16384 2024-09-19T09:05:20.173 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:05:20.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.173+0000 7f66230f5640 1 -- 172.21.15.120:0/209656810 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f66080a32c0 msgr2=0x7f66080a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:20.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.173+0000 7f66230f5640 1 --2- 172.21.15.120:0/209656810 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f66080a32c0 0x7f66080a5780 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6604004650 tx=0x7f660400be60 comp rx=0 tx=0).stop 2024-09-19T09:05:20.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.173+0000 7f66230f5640 1 -- 172.21.15.120:0/209656810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f661c102570 msgr2=0x7f661c197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:20.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.173+0000 7f66230f5640 1 --2- 172.21.15.120:0/209656810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f661c102570 0x7f661c197920 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f660c006010 tx=0x7f660c003c10 comp rx=0 tx=0).stop 2024-09-19T09:05:20.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.173+0000 7f66230f5640 1 -- 172.21.15.120:0/209656810 shutdown_connections 2024-09-19T09:05:20.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.173+0000 7f66230f5640 1 --2- 172.21.15.120:0/209656810 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f66080a32c0 0x7f66080a5780 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.173+0000 7f66230f5640 1 --2- 172.21.15.120:0/209656810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f661c102ef0 0x7f661c19bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.173+0000 7f66230f5640 1 --2- 172.21.15.120:0/209656810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f661c102570 0x7f661c197920 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.173+0000 7f66230f5640 1 --2- 172.21.15.120:0/209656810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f661c100e20 0x7f661c1973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:20.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.173+0000 7f66230f5640 1 -- 172.21.15.120:0/209656810 >> 172.21.15.120:0/209656810 conn(0x7f661c0fc7d0 msgr2=0x7f661c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:20.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.173+0000 7f66230f5640 1 -- 172.21.15.120:0/209656810 shutdown_connections 2024-09-19T09:05:20.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:20.173+0000 7f66230f5640 1 -- 172.21.15.120:0/209656810 wait complete. 2024-09-19T09:05:20.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:20 smithi120 ceph-mon[31150]: pgmap v415: 97 pgs: 97 active+clean; 33 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.6 MiB/s wr, 327 op/s 2024-09-19T09:05:21.038 INFO:tasks.workunit.client.0.smithi120.stdout: 1 103529 7.57 MB/sec warmup 76 sec latency 10.239 ms 2024-09-19T09:05:21.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:20 smithi196 ceph-mon[35150]: pgmap v415: 97 pgs: 97 active+clean; 33 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.6 MiB/s wr, 327 op/s 2024-09-19T09:05:21.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:20 smithi153 ceph-mon[35071]: pgmap v415: 97 pgs: 97 active+clean; 33 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.6 MiB/s wr, 327 op/s 2024-09-19T09:05:21.893 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:05:21.893 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:05:22.038 INFO:tasks.workunit.client.0.smithi120.stdout: 1 106775 7.72 MB/sec warmup 77 sec latency 15.425 ms 2024-09-19T09:05:22.146 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:05:22.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.692+0000 7f1c3009c640 1 -- 172.21.15.120:0/3948369340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c28101de0 msgr2=0x7f1c281021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:22.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.692+0000 7f1c3009c640 1 --2- 172.21.15.120:0/3948369340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c28101de0 0x7f1c281021c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1c24009900 tx=0x7f1c2402a410 comp rx=0 tx=0).stop 2024-09-19T09:05:22.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.692+0000 7f1c3009c640 1 -- 172.21.15.120:0/3948369340 shutdown_connections 2024-09-19T09:05:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.692+0000 7f1c3009c640 1 --2- 172.21.15.120:0/3948369340 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c28106f10 0x7f1c2810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.692+0000 7f1c3009c640 1 --2- 172.21.15.120:0/3948369340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c28102790 0x7f1c281067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.692+0000 7f1c3009c640 1 --2- 172.21.15.120:0/3948369340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c28101de0 0x7f1c281021c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.692+0000 7f1c3009c640 1 -- 172.21.15.120:0/3948369340 >> 172.21.15.120:0/3948369340 conn(0x7f1c280fd8c0 msgr2=0x7f1c280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.692+0000 7f1c3009c640 1 -- 172.21.15.120:0/3948369340 shutdown_connections 2024-09-19T09:05:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.692+0000 7f1c3009c640 1 -- 172.21.15.120:0/3948369340 wait complete. 2024-09-19T09:05:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.693+0000 7f1c3009c640 1 Processor -- start 2024-09-19T09:05:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.693+0000 7f1c3009c640 1 -- start start 2024-09-19T09:05:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.693+0000 7f1c3009c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c28101de0 0x7f1c281984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.693+0000 7f1c3009c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c28102790 0x7f1c281989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.693+0000 7f1c3009c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c28106f10 0x7f1c2819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.693+0000 7f1c3009c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1c28111f70 con 0x7f1c28102790 2024-09-19T09:05:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.693+0000 7f1c3009c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1c28111df0 con 0x7f1c28106f10 2024-09-19T09:05:22.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.693+0000 7f1c3009c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1c281120f0 con 0x7f1c28101de0 2024-09-19T09:05:22.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.693+0000 7f1c2de38640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c28101de0 0x7f1c281984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:22.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.693+0000 7f1c2de38640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c28101de0 0x7f1c281984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:59644/0 (socket says 172.21.15.120:59644) 2024-09-19T09:05:22.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.693+0000 7f1c2de38640 1 -- 172.21.15.120:0/442393530 learned_addr learned my addr 172.21.15.120:0/442393530 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:22.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.693+0000 7f1c2d637640 1 --2- 172.21.15.120:0/442393530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c28102790 0x7f1c281989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:22.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.694+0000 7f1c2e639640 1 --2- 172.21.15.120:0/442393530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c28106f10 0x7f1c2819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:22.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.694+0000 7f1c2de38640 1 -- 172.21.15.120:0/442393530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c28106f10 msgr2=0x7f1c2819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:22.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.694+0000 7f1c2de38640 1 --2- 172.21.15.120:0/442393530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c28106f10 0x7f1c2819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.694+0000 7f1c2de38640 1 -- 172.21.15.120:0/442393530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c28102790 msgr2=0x7f1c281989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:22.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.694+0000 7f1c2de38640 1 --2- 172.21.15.120:0/442393530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c28102790 0x7f1c281989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.694+0000 7f1c2de38640 1 -- 172.21.15.120:0/442393530 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2819d460 con 0x7f1c28101de0 2024-09-19T09:05:22.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.694+0000 7f1c2de38640 1 --2- 172.21.15.120:0/442393530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c28101de0 0x7f1c281984b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1c24006010 tx=0x7f1c24004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:22.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.694+0000 7f1c1effd640 1 -- 172.21.15.120:0/442393530 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2403f030 con 0x7f1c28101de0 2024-09-19T09:05:22.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.694+0000 7f1c1effd640 1 -- 172.21.15.120:0/442393530 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c24004440 con 0x7f1c28101de0 2024-09-19T09:05:22.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.694+0000 7f1c1effd640 1 -- 172.21.15.120:0/442393530 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24035070 con 0x7f1c28101de0 2024-09-19T09:05:22.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.694+0000 7f1c3009c640 1 -- 172.21.15.120:0/442393530 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c2819d6f0 con 0x7f1c28101de0 2024-09-19T09:05:22.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.694+0000 7f1c3009c640 1 -- 172.21.15.120:0/442393530 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c281a5030 con 0x7f1c28101de0 2024-09-19T09:05:22.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.696+0000 7f1c1effd640 1 -- 172.21.15.120:0/442393530 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f1c24042030 con 0x7f1c28101de0 2024-09-19T09:05:22.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.697+0000 7f1c32475740 1 Processor -- start 2024-09-19T09:05:22.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.697+0000 7f1c32475740 1 -- start start 2024-09-19T09:05:22.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.697+0000 7f1c32475740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56230ef39b60 0x56230ef39f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:22.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.697+0000 7f1c32475740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56230ef3a480 0x56230ef44800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:22.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.697+0000 7f1c32475740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56230ef44d40 0x56230ef47130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:22.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.697+0000 7f1c32475740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56230ee9f280 con 0x56230ef39b60 2024-09-19T09:05:22.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.697+0000 7f1c32475740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56230ee46b10 con 0x56230ef44d40 2024-09-19T09:05:22.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.697+0000 7f1c32475740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56230ee9b240 con 0x56230ef3a480 2024-09-19T09:05:22.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.697+0000 7f1c2e639640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56230ef39b60 0x56230ef39f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:22.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.697+0000 7f1c1effd640 1 --2- 172.21.15.120:0/442393530 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1c140a3240 0x7f1c140a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:22.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.697+0000 7f1c2d637640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56230ef44d40 0x56230ef47130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:22.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.697+0000 7f1c2d637640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56230ef44d40 0x56230ef47130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:48466/0 (socket says 172.21.15.120:48466) 2024-09-19T09:05:22.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.697+0000 7f1c2d637640 1 -- 172.21.15.120:0/3364674236 learned_addr learned my addr 172.21.15.120:0/3364674236 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:22.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.697+0000 7f1c2de38640 1 --2- 172.21.15.120:0/3364674236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56230ef3a480 0x56230ef44800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:22.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.698+0000 7f1c2e639640 1 --2- 172.21.15.120:0/442393530 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1c140a3240 0x7f1c140a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:22.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.698+0000 7f1c2d637640 1 -- 172.21.15.120:0/3364674236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56230ef3a480 msgr2=0x56230ef44800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:22.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.698+0000 7f1c2d637640 1 --2- 172.21.15.120:0/3364674236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56230ef3a480 0x56230ef44800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.698+0000 7f1c2d637640 1 -- 172.21.15.120:0/3364674236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56230ef39b60 msgr2=0x56230ef39f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:22.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.698+0000 7f1c2d637640 1 --2- 172.21.15.120:0/3364674236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56230ef39b60 0x56230ef39f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.698+0000 7f1c1effd640 1 -- 172.21.15.120:0/442393530 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f1c240efca0 con 0x7f1c28101de0 2024-09-19T09:05:22.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.698+0000 7f1c2d637640 1 -- 172.21.15.120:0/3364674236 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56230ee531f0 con 0x56230ef44d40 2024-09-19T09:05:22.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.698+0000 7f1c2d637640 1 --2- 172.21.15.120:0/3364674236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56230ef44d40 0x56230ef47130 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009ca0 tx=0x7f1c18011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4a26a54335f74f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:22.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.698+0000 7f1c2e639640 1 --2- 172.21.15.120:0/442393530 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1c140a3240 0x7f1c140a5700 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009fd0 tx=0x7f1c20004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:22.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.698+0000 7f1c13fff640 1 -- 172.21.15.120:0/3364674236 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1801e070 con 0x56230ef44d40 2024-09-19T09:05:22.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.698+0000 7f1c13fff640 1 -- 172.21.15.120:0/3364674236 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c18019b10 con 0x56230ef44d40 2024-09-19T09:05:22.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.698+0000 7f1c13fff640 1 -- 172.21.15.120:0/3364674236 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1800ea30 con 0x56230ef44d40 2024-09-19T09:05:22.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.699+0000 7f1c32475740 1 -- 172.21.15.120:0/3364674236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56230ef44d40 msgr2=0x56230ef47130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:22.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.699+0000 7f1c32475740 1 --2- 172.21.15.120:0/3364674236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56230ef44d40 0x56230ef47130 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009ca0 tx=0x7f1c18011810 comp rx=0 tx=0).stop 2024-09-19T09:05:22.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.699+0000 7f1c32475740 1 -- 172.21.15.120:0/3364674236 shutdown_connections 2024-09-19T09:05:22.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.699+0000 7f1c32475740 1 --2- 172.21.15.120:0/3364674236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56230ef44d40 0x56230ef47130 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.699+0000 7f1c32475740 1 --2- 172.21.15.120:0/3364674236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56230ef3a480 0x56230ef44800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.699+0000 7f1c32475740 1 --2- 172.21.15.120:0/3364674236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56230ef39b60 0x56230ef39f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.699+0000 7f1c32475740 1 -- 172.21.15.120:0/3364674236 >> 172.21.15.120:0/3364674236 conn(0x56230ef360f0 msgr2=0x56230ef364d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:22.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.699+0000 7f1c32475740 1 -- 172.21.15.120:0/3364674236 shutdown_connections 2024-09-19T09:05:22.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.699+0000 7f1c32475740 1 -- 172.21.15.120:0/3364674236 wait complete. 2024-09-19T09:05:22.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.699+0000 7f1c32475740 1 -- start start 2024-09-19T09:05:22.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.699+0000 7f1c32475740 1 Processor -- start 2024-09-19T09:05:22.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.700+0000 7f1c32475740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56230ef39b60 0x56230ef41bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:22.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.700+0000 7f1c32475740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56230ef3a480 0x56230ef42100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:22.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.700+0000 7f1c32475740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56230ef44d40 0x56230ef42640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:22.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.700+0000 7f1c32475740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56230ef47910 con 0x56230ef44d40 2024-09-19T09:05:22.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.700+0000 7f1c32475740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56230ee59f10 con 0x56230ef39b60 2024-09-19T09:05:22.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.700+0000 7f1c32475740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56230ef47a90 con 0x56230ef3a480 2024-09-19T09:05:22.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.700+0000 7f1c32475740 10 client.0 _ll_register_callbacks cb 0x56230ee22a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:22.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.700+0000 7f1c2e639640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56230ef44d40 0x56230ef42640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:22.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.700+0000 7f1c2de38640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56230ef39b60 0x56230ef41bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:22.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.700+0000 7f1c2de38640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56230ef39b60 0x56230ef41bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:48472/0 (socket says 172.21.15.120:48472) 2024-09-19T09:05:22.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.700+0000 7f1c2d637640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56230ef3a480 0x56230ef42100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:22.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.700+0000 7f1c2e639640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56230ef44d40 0x56230ef42640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36122/0 (socket says 172.21.15.120:36122) 2024-09-19T09:05:22.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.700+0000 7f1c2de38640 1 -- 172.21.15.120:0/1803945179 learned_addr learned my addr 172.21.15.120:0/1803945179 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:22.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.700+0000 7f1c2de38640 1 -- 172.21.15.120:0/1803945179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56230ef3a480 msgr2=0x56230ef42100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:22.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.700+0000 7f1c2de38640 1 --2- 172.21.15.120:0/1803945179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56230ef3a480 0x56230ef42100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.700+0000 7f1c2de38640 1 -- 172.21.15.120:0/1803945179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56230ef44d40 msgr2=0x56230ef42640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:22.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.700+0000 7f1c2de38640 1 --2- 172.21.15.120:0/1803945179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56230ef44d40 0x56230ef42640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.700+0000 7f1c2de38640 1 -- 172.21.15.120:0/1803945179 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56230ec106a0 con 0x56230ef39b60 2024-09-19T09:05:22.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.700+0000 7f1c2de38640 1 --2- 172.21.15.120:0/1803945179 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56230ef39b60 0x56230ef41bc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1c240c49c0 tx=0x7f1c2408b070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88816480abd72635 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:22.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.700+0000 7f1c117fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:05:22.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.701+0000 7f1c117fa640 1 -- 172.21.15.120:0/1803945179 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24049070 con 0x56230ef39b60 2024-09-19T09:05:22.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.701+0000 7f1c117fa640 1 -- 172.21.15.120:0/1803945179 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f1c24003cd0 con 0x56230ef39b60 2024-09-19T09:05:22.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.701+0000 7f1c32475740 1 -- 172.21.15.120:0/1803945179 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c18019b10 con 0x56230ef39b60 2024-09-19T09:05:22.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.701+0000 7f1c117fa640 1 -- 172.21.15.120:0/1803945179 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1c240bbc90 con 0x56230ef39b60 2024-09-19T09:05:22.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.701+0000 7f1c32475740 10 client.25565 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:22.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.701+0000 7f1c32475740 1 -- 172.21.15.120:0/1803945179 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56230ef534c0 con 0x56230ef39b60 2024-09-19T09:05:22.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.701+0000 7f1c117fa640 1 -- 172.21.15.120:0/1803945179 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c18019b10 con 0x56230ef39b60 2024-09-19T09:05:22.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.701+0000 7f1c117fa640 1 -- 172.21.15.120:0/1803945179 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1c240b89a0 con 0x56230ef39b60 2024-09-19T09:05:22.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.701+0000 7f1c32475740 10 client.25565 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:22.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.701+0000 7f1c32475740 10 client.25565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:22.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.701+0000 7f1c32475740 20 client.25565 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:22.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.701+0000 7f1c32475740 1 --2- 172.21.15.120:0/1803945179 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56230ef54970 0x56230ef56de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:22.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.701+0000 7f1c32475740 4 client.25565 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:05:22.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.701+0000 7f1c32475740 1 -- 172.21.15.120:0/1803945179 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56230eeac7f0 con 0x56230ef54970 2024-09-19T09:05:22.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.701+0000 7f1c2d637640 1 --2- 172.21.15.120:0/1803945179 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56230ef54970 0x56230ef56de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:22.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.702+0000 7f1c117fa640 10 client.25565 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:22.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.702+0000 7f1c2d637640 1 --2- 172.21.15.120:0/1803945179 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56230ef54970 0x56230ef56de0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=679c1293d0a08ba8 server_cookie=ab5b8dbce2016c98 in_seq=0 out_seq=0 2024-09-19T09:05:22.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.703+0000 7f1c117fa640 1 -- 172.21.15.120:0/1803945179 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56230eeac7f0 con 0x56230ef54970 2024-09-19T09:05:22.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.703+0000 7f1c117fa640 10 client.25565 handle_command_reply: tid=0 2024-09-19T09:05:22.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.703+0000 7f1c32475740 1 client.25565 shutdown 2024-09-19T09:05:22.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.703+0000 7f1c32475740 1 -- 172.21.15.120:0/1803945179 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56230ef39b60 msgr2=0x56230ef41bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:22.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.703+0000 7f1c32475740 1 --2- 172.21.15.120:0/1803945179 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56230ef39b60 0x56230ef41bc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1c240c49c0 tx=0x7f1c2408b070 comp rx=0 tx=0).stop 2024-09-19T09:05:22.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.703+0000 7f1c32475740 1 -- 172.21.15.120:0/1803945179 shutdown_connections 2024-09-19T09:05:22.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.703+0000 7f1c32475740 1 --2- 172.21.15.120:0/1803945179 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56230ef54970 0x56230ef56de0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.703+0000 7f1c32475740 1 --2- 172.21.15.120:0/1803945179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56230ef44d40 0x56230ef42640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.703+0000 7f1c32475740 1 --2- 172.21.15.120:0/1803945179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56230ef3a480 0x56230ef42100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.703+0000 7f1c32475740 1 --2- 172.21.15.120:0/1803945179 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56230ef39b60 0x56230ef41bc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.703+0000 7f1c117fa640 0 client.25565 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:22.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.703+0000 7f1c32475740 1 -- 172.21.15.120:0/1803945179 >> 172.21.15.120:0/1803945179 conn(0x56230ef360f0 msgr2=0x56230ef3be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:22.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.703+0000 7f1c32475740 1 -- 172.21.15.120:0/1803945179 shutdown_connections 2024-09-19T09:05:22.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.703+0000 7f1c32475740 1 -- 172.21.15.120:0/1803945179 wait complete. 2024-09-19T09:05:22.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.703+0000 7f1c32475740 20 client.25565 trim_cache size 0 max 16384 2024-09-19T09:05:22.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.721+0000 7f1c32475740 1 Processor -- start 2024-09-19T09:05:22.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.721+0000 7f1c32475740 1 -- start start 2024-09-19T09:05:22.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.722+0000 7f1c32475740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56230ef39b60 0x56230ef3bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:22.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.722+0000 7f1c32475740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56230ef3a480 0x56230ef5c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:22.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.722+0000 7f1c32475740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56230ef44d40 0x56230ef5cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:22.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.722+0000 7f1c32475740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1c281ad540 con 0x56230ef39b60 2024-09-19T09:05:22.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.722+0000 7f1c32475740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1c281af080 con 0x56230ef3a480 2024-09-19T09:05:22.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.722+0000 7f1c32475740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1c281adc10 con 0x56230ef44d40 2024-09-19T09:05:22.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.722+0000 7f1c2de38640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56230ef39b60 0x56230ef3bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:22.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.722+0000 7f1c2d637640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56230ef3a480 0x56230ef5c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:22.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.722+0000 7f1c2de38640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56230ef39b60 0x56230ef3bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36130/0 (socket says 172.21.15.120:36130) 2024-09-19T09:05:22.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.722+0000 7f1c2de38640 1 -- 172.21.15.120:0/1480177259 learned_addr learned my addr 172.21.15.120:0/1480177259 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:22.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.722+0000 7f1c2e639640 1 --2- 172.21.15.120:0/1480177259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56230ef44d40 0x56230ef5cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:22.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.722+0000 7f1c2de38640 1 -- 172.21.15.120:0/1480177259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56230ef44d40 msgr2=0x56230ef5cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:22.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.722+0000 7f1c2de38640 1 --2- 172.21.15.120:0/1480177259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56230ef44d40 0x56230ef5cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.722+0000 7f1c2de38640 1 -- 172.21.15.120:0/1480177259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56230ef3a480 msgr2=0x56230ef5c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:22.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.722+0000 7f1c2de38640 1 --2- 172.21.15.120:0/1480177259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56230ef3a480 0x56230ef5c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.722+0000 7f1c2de38640 1 -- 172.21.15.120:0/1480177259 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56230ef35f20 con 0x56230ef39b60 2024-09-19T09:05:22.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.722+0000 7f1c2d637640 1 --2- 172.21.15.120:0/1480177259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56230ef3a480 0x56230ef5c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:22.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.722+0000 7f1c2de38640 1 --2- 172.21.15.120:0/1480177259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56230ef39b60 0x56230ef3bcc0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1c2403d040 tx=0x7f1c240b8690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdaa8c779931d906 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:22.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.722+0000 7f1c137fe640 1 -- 172.21.15.120:0/1480177259 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24049070 con 0x56230ef39b60 2024-09-19T09:05:22.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.723+0000 7f1c137fe640 1 -- 172.21.15.120:0/1480177259 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c2408d030 con 0x56230ef39b60 2024-09-19T09:05:22.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.723+0000 7f1c137fe640 1 -- 172.21.15.120:0/1480177259 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c240b89a0 con 0x56230ef39b60 2024-09-19T09:05:22.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.723+0000 7f1c32475740 1 -- 172.21.15.120:0/1480177259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56230ef39b60 msgr2=0x56230ef3bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:22.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.723+0000 7f1c32475740 1 --2- 172.21.15.120:0/1480177259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56230ef39b60 0x56230ef3bcc0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1c2403d040 tx=0x7f1c240b8690 comp rx=0 tx=0).stop 2024-09-19T09:05:22.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.723+0000 7f1c32475740 1 -- 172.21.15.120:0/1480177259 shutdown_connections 2024-09-19T09:05:22.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.723+0000 7f1c32475740 1 --2- 172.21.15.120:0/1480177259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56230ef44d40 0x56230ef5cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.723+0000 7f1c32475740 1 --2- 172.21.15.120:0/1480177259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56230ef3a480 0x56230ef5c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.723+0000 7f1c32475740 1 --2- 172.21.15.120:0/1480177259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56230ef39b60 0x56230ef3bcc0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.723+0000 7f1c32475740 1 -- 172.21.15.120:0/1480177259 >> 172.21.15.120:0/1480177259 conn(0x56230ef360f0 msgr2=0x56230ef3b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:22.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.723+0000 7f1c32475740 1 -- 172.21.15.120:0/1480177259 shutdown_connections 2024-09-19T09:05:22.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.723+0000 7f1c32475740 1 -- 172.21.15.120:0/1480177259 wait complete. 2024-09-19T09:05:22.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.724+0000 7f1c32475740 1 -- start start 2024-09-19T09:05:22.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.724+0000 7f1c32475740 1 Processor -- start 2024-09-19T09:05:22.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.724+0000 7f1c32475740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56230ef39b60 0x56230ef4cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:22.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.724+0000 7f1c32475740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56230ef3a480 0x56230ef4d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:22.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.724+0000 7f1c32475740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56230ef44d40 0x56230ef49cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:22.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.724+0000 7f1c32475740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1c281ae9b0 con 0x56230ef39b60 2024-09-19T09:05:22.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.724+0000 7f1c32475740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1c281ae2e0 con 0x56230ef44d40 2024-09-19T09:05:22.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.724+0000 7f1c32475740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1c281ab500 con 0x56230ef3a480 2024-09-19T09:05:22.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.724+0000 7f1c32475740 10 client.0 _ll_register_callbacks cb 0x7f1c24009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:22.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.724+0000 7f1c2de38640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56230ef39b60 0x56230ef4cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:22.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.725+0000 7f1c2de38640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56230ef39b60 0x56230ef4cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36132/0 (socket says 172.21.15.120:36132) 2024-09-19T09:05:22.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.725+0000 7f1c2de38640 1 -- 172.21.15.120:0/1306999072 learned_addr learned my addr 172.21.15.120:0/1306999072 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:22.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.725+0000 7f1c2d637640 1 --2- 172.21.15.120:0/1306999072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56230ef3a480 0x56230ef4d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:22.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.725+0000 7f1c2e639640 1 --2- 172.21.15.120:0/1306999072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56230ef44d40 0x56230ef49cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:22.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.725+0000 7f1c2de38640 1 -- 172.21.15.120:0/1306999072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56230ef3a480 msgr2=0x56230ef4d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:22.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.725+0000 7f1c2de38640 1 --2- 172.21.15.120:0/1306999072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56230ef3a480 0x56230ef4d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.725+0000 7f1c2de38640 1 -- 172.21.15.120:0/1306999072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56230ef44d40 msgr2=0x56230ef49cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:22.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.725+0000 7f1c2de38640 1 --2- 172.21.15.120:0/1306999072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56230ef44d40 0x56230ef49cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.725+0000 7f1c2de38640 1 -- 172.21.15.120:0/1306999072 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56230ef4a3c0 con 0x56230ef39b60 2024-09-19T09:05:22.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.725+0000 7f1c2d637640 1 --2- 172.21.15.120:0/1306999072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56230ef3a480 0x56230ef4d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:22.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.725+0000 7f1c2de38640 1 --2- 172.21.15.120:0/1306999072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56230ef39b60 0x56230ef4cec0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f1c240bc030 tx=0x7f1c240bc6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af9fcb52093e7d0a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:22.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.725+0000 7f1c117fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:22.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.725+0000 7f1c117fa640 1 -- 172.21.15.120:0/1306999072 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24049070 con 0x56230ef39b60 2024-09-19T09:05:22.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.725+0000 7f1c117fa640 1 -- 172.21.15.120:0/1306999072 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f1c240dc2e0 con 0x56230ef39b60 2024-09-19T09:05:22.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.725+0000 7f1c32475740 1 -- 172.21.15.120:0/1306999072 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c2408d030 con 0x56230ef39b60 2024-09-19T09:05:22.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.725+0000 7f1c117fa640 1 -- 172.21.15.120:0/1306999072 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1c240c4ef0 con 0x56230ef39b60 2024-09-19T09:05:22.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.725+0000 7f1c32475740 10 client.16209 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:22.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.725+0000 7f1c32475740 1 -- 172.21.15.120:0/1306999072 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56230ef4a3c0 con 0x56230ef39b60 2024-09-19T09:05:22.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.726+0000 7f1c117fa640 1 -- 172.21.15.120:0/1306999072 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c2408d030 con 0x56230ef39b60 2024-09-19T09:05:22.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.726+0000 7f1c117fa640 1 -- 172.21.15.120:0/1306999072 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1c240c0220 con 0x56230ef39b60 2024-09-19T09:05:22.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.726+0000 7f1c32475740 10 client.16209 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:22.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.726+0000 7f1c32475740 10 client.16209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:22.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.726+0000 7f1c32475740 20 client.16209 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:22.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.726+0000 7f1c32475740 1 --2- 172.21.15.120:0/1306999072 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56230efc46d0 0x56230efc6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:22.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.726+0000 7f1c32475740 4 client.16209 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:05:22.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.726+0000 7f1c32475740 1 -- 172.21.15.120:0/1306999072 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56230eea6d00 con 0x56230efc46d0 2024-09-19T09:05:22.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.726+0000 7f1c2d637640 1 --2- 172.21.15.120:0/1306999072 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56230efc46d0 0x56230efc6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:22.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.726+0000 7f1c117fa640 10 client.16209 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:22.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.726+0000 7f1c2d637640 1 --2- 172.21.15.120:0/1306999072 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56230efc46d0 0x56230efc6ac0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54061c496a3f2688 server_cookie=f80f2500f45405df in_seq=0 out_seq=0 2024-09-19T09:05:22.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.728+0000 7f1c117fa640 1 -- 172.21.15.120:0/1306999072 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56230eea6d00 con 0x56230efc46d0 2024-09-19T09:05:22.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.728+0000 7f1c117fa640 10 client.16209 handle_command_reply: tid=0 2024-09-19T09:05:22.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.728+0000 7f1c32475740 1 client.16209 shutdown 2024-09-19T09:05:22.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.728+0000 7f1c32475740 1 -- 172.21.15.120:0/1306999072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56230ef39b60 msgr2=0x56230ef4cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:22.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.728+0000 7f1c32475740 1 --2- 172.21.15.120:0/1306999072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56230ef39b60 0x56230ef4cec0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f1c240bc030 tx=0x7f1c240bc6b0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.728+0000 7f1c32475740 1 -- 172.21.15.120:0/1306999072 shutdown_connections 2024-09-19T09:05:22.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.728+0000 7f1c32475740 1 --2- 172.21.15.120:0/1306999072 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56230efc46d0 0x56230efc6ac0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.728+0000 7f1c32475740 1 --2- 172.21.15.120:0/1306999072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56230ef44d40 0x56230ef49cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.728+0000 7f1c32475740 1 --2- 172.21.15.120:0/1306999072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56230ef3a480 0x56230ef4d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.728+0000 7f1c32475740 1 --2- 172.21.15.120:0/1306999072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56230ef39b60 0x56230ef4cec0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.728+0000 7f1c117fa640 0 client.16209 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:22.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.728+0000 7f1c32475740 1 -- 172.21.15.120:0/1306999072 >> 172.21.15.120:0/1306999072 conn(0x56230ef360f0 msgr2=0x56230ef35270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:22.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.729+0000 7f1c32475740 1 -- 172.21.15.120:0/1306999072 shutdown_connections 2024-09-19T09:05:22.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.729+0000 7f1c32475740 1 -- 172.21.15.120:0/1306999072 wait complete. 2024-09-19T09:05:22.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.729+0000 7f1c32475740 20 client.16209 trim_cache size 0 max 16384 2024-09-19T09:05:22.733 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:05:22.734 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:05:22.734 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "1f8a79b2-4945-4de2-952d-d85d117e972d", 2024-09-19T09:05:22.734 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:05:22.734 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:05:22.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.729+0000 7f1c3009c640 1 -- 172.21.15.120:0/442393530 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1c140a3240 msgr2=0x7f1c140a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:22.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.729+0000 7f1c3009c640 1 --2- 172.21.15.120:0/442393530 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1c140a3240 0x7f1c140a5700 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009fd0 tx=0x7f1c20004ab0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.729+0000 7f1c3009c640 1 -- 172.21.15.120:0/442393530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c28101de0 msgr2=0x7f1c281984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:22.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.729+0000 7f1c3009c640 1 --2- 172.21.15.120:0/442393530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c28101de0 0x7f1c281984b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1c24006010 tx=0x7f1c24004040 comp rx=0 tx=0).stop 2024-09-19T09:05:22.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.729+0000 7f1c3009c640 1 -- 172.21.15.120:0/442393530 shutdown_connections 2024-09-19T09:05:22.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.729+0000 7f1c3009c640 1 --2- 172.21.15.120:0/442393530 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1c140a3240 0x7f1c140a5700 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.729+0000 7f1c3009c640 1 --2- 172.21.15.120:0/442393530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c28106f10 0x7f1c2819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.729+0000 7f1c3009c640 1 --2- 172.21.15.120:0/442393530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c28102790 0x7f1c281989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.729+0000 7f1c3009c640 1 --2- 172.21.15.120:0/442393530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c28101de0 0x7f1c281984b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:22.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.729+0000 7f1c3009c640 1 -- 172.21.15.120:0/442393530 >> 172.21.15.120:0/442393530 conn(0x7f1c280fd8c0 msgr2=0x7f1c280fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:22.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.729+0000 7f1c3009c640 1 -- 172.21.15.120:0/442393530 shutdown_connections 2024-09-19T09:05:22.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:22.729+0000 7f1c3009c640 1 -- 172.21.15.120:0/442393530 wait complete. 2024-09-19T09:05:23.038 INFO:tasks.workunit.client.0.smithi120.stdout: 1 109827 7.81 MB/sec warmup 78 sec latency 40.335 ms 2024-09-19T09:05:23.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:22 smithi120 ceph-mon[31150]: pgmap v416: 97 pgs: 97 active+clean; 39 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.1 MiB/s wr, 398 op/s 2024-09-19T09:05:23.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:22 smithi196 ceph-mon[35150]: pgmap v416: 97 pgs: 97 active+clean; 39 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.1 MiB/s wr, 398 op/s 2024-09-19T09:05:23.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:22 smithi153 ceph-mon[35071]: pgmap v416: 97 pgs: 97 active+clean; 39 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.1 MiB/s wr, 398 op/s 2024-09-19T09:05:23.615 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:05:23.872 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:05:23.992 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:23 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:05:23.992 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:23 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:05:23.992 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:23 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:05:23.992 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:23 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:05:23.992 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:23 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:05:24.038 INFO:tasks.workunit.client.0.smithi120.stdout: 1 112601 7.87 MB/sec warmup 79 sec latency 11.246 ms 2024-09-19T09:05:24.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:23 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:05:24.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:23 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:05:24.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:23 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:05:24.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:23 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:05:24.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:23 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:05:24.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:23 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:05:24.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:23 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:05:24.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:23 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:05:24.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:23 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:05:24.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:23 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:05:24.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.394+0000 7f7f2fd5f640 1 -- 172.21.15.120:0/2706839728 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f28101420 msgr2=0x7f7f2810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:24.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.394+0000 7f7f2fd5f640 1 --2- 172.21.15.120:0/2706839728 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f28101420 0x7f7f2810d950 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7f20009cd0 tx=0x7f7f2002a8e0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.394+0000 7f7f2fd5f640 1 -- 172.21.15.120:0/2706839728 shutdown_connections 2024-09-19T09:05:24.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.394+0000 7f7f2fd5f640 1 --2- 172.21.15.120:0/2706839728 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f28101420 0x7f7f2810d950 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.394+0000 7f7f2fd5f640 1 --2- 172.21.15.120:0/2706839728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f28100a80 0x7f7f28100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.394+0000 7f7f2fd5f640 1 --2- 172.21.15.120:0/2706839728 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f28106a80 0x7f7f28106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.394+0000 7f7f2fd5f640 1 -- 172.21.15.120:0/2706839728 >> 172.21.15.120:0/2706839728 conn(0x7f7f280fc7a0 msgr2=0x7f7f280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:24.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.394+0000 7f7f2fd5f640 1 -- 172.21.15.120:0/2706839728 shutdown_connections 2024-09-19T09:05:24.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.394+0000 7f7f2fd5f640 1 -- 172.21.15.120:0/2706839728 wait complete. 2024-09-19T09:05:24.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.395+0000 7f7f2fd5f640 1 Processor -- start 2024-09-19T09:05:24.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.395+0000 7f7f2fd5f640 1 -- start start 2024-09-19T09:05:24.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.395+0000 7f7f2fd5f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f28100a80 0x7f7f28198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:24.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.395+0000 7f7f2fd5f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f28101420 0x7f7f28198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:24.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.395+0000 7f7f2fd5f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f28106a80 0x7f7f2819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:24.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.395+0000 7f7f2fd5f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7f28111fa0 con 0x7f7f28100a80 2024-09-19T09:05:24.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.395+0000 7f7f2fd5f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7f28111e20 con 0x7f7f28101420 2024-09-19T09:05:24.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.395+0000 7f7f2fd5f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7f28112120 con 0x7f7f28106a80 2024-09-19T09:05:24.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.395+0000 7f7f2dafb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f28100a80 0x7f7f28198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:24.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.395+0000 7f7f2dafb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f28100a80 0x7f7f28198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36158/0 (socket says 172.21.15.120:36158) 2024-09-19T09:05:24.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.395+0000 7f7f2dafb640 1 -- 172.21.15.120:0/2635567165 learned_addr learned my addr 172.21.15.120:0/2635567165 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:24.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.395+0000 7f7f2e2fc640 1 --2- 172.21.15.120:0/2635567165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f28106a80 0x7f7f2819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:24.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.395+0000 7f7f2d2fa640 1 --2- 172.21.15.120:0/2635567165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f28101420 0x7f7f28198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:24.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.395+0000 7f7f2dafb640 1 -- 172.21.15.120:0/2635567165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f28106a80 msgr2=0x7f7f2819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:24.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.395+0000 7f7f2dafb640 1 --2- 172.21.15.120:0/2635567165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f28106a80 0x7f7f2819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.395+0000 7f7f2dafb640 1 -- 172.21.15.120:0/2635567165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f28101420 msgr2=0x7f7f28198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:24.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.395+0000 7f7f2dafb640 1 --2- 172.21.15.120:0/2635567165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f28101420 0x7f7f28198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.395+0000 7f7f2dafb640 1 -- 172.21.15.120:0/2635567165 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2819d3f0 con 0x7f7f28100a80 2024-09-19T09:05:24.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.396+0000 7f7f2dafb640 1 --2- 172.21.15.120:0/2635567165 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f28100a80 0x7f7f28198440 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7f2400e9f0 tx=0x7f7f2400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:24.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.396+0000 7f7f1effd640 1 -- 172.21.15.120:0/2635567165 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2400cd80 con 0x7f7f28100a80 2024-09-19T09:05:24.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.396+0000 7f7f2fd5f640 1 -- 172.21.15.120:0/2635567165 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f2819d6e0 con 0x7f7f28100a80 2024-09-19T09:05:24.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.396+0000 7f7f1effd640 1 -- 172.21.15.120:0/2635567165 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f24010910 con 0x7f7f28100a80 2024-09-19T09:05:24.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.396+0000 7f7f1effd640 1 -- 172.21.15.120:0/2635567165 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f24018990 con 0x7f7f28100a80 2024-09-19T09:05:24.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.396+0000 7f7f2fd5f640 1 -- 172.21.15.120:0/2635567165 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f281a4fc0 con 0x7f7f28100a80 2024-09-19T09:05:24.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.398+0000 7f7f1effd640 1 -- 172.21.15.120:0/2635567165 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f7f24010ac0 con 0x7f7f28100a80 2024-09-19T09:05:24.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.399+0000 7f7f1effd640 1 --2- 172.21.15.120:0/2635567165 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7f140a3240 0x7f7f140a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:24.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.399+0000 7f7f32138740 1 Processor -- start 2024-09-19T09:05:24.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.399+0000 7f7f2d2fa640 1 --2- 172.21.15.120:0/2635567165 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7f140a3240 0x7f7f140a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:24.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.399+0000 7f7f1effd640 1 -- 172.21.15.120:0/2635567165 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f7f240cfba0 con 0x7f7f28100a80 2024-09-19T09:05:24.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.399+0000 7f7f32138740 1 -- start start 2024-09-19T09:05:24.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.399+0000 7f7f32138740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b594d64a0 0x555b594d6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:24.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.399+0000 7f7f32138740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b594d6dc0 0x555b594e1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:24.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.399+0000 7f7f32138740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b594e1680 0x555b594e3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:24.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.399+0000 7f7f32138740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555b5943c2e0 con 0x555b594e1680 2024-09-19T09:05:24.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.399+0000 7f7f32138740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555b593e3ce0 con 0x555b594d64a0 2024-09-19T09:05:24.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.399+0000 7f7f32138740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555b594382a0 con 0x555b594d6dc0 2024-09-19T09:05:24.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.400+0000 7f7f2d2fa640 1 --2- 172.21.15.120:0/2635567165 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7f140a3240 0x7f7f140a5700 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7f280fed80 tx=0x7f7f1800a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:24.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.400+0000 7f7f2d2fa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b594d6dc0 0x555b594e1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:24.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.400+0000 7f7f2dafb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b594d64a0 0x555b594d6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:24.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.400+0000 7f7f2e2fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b594e1680 0x555b594e3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:24.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.400+0000 7f7f2dafb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b594d64a0 0x555b594d6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:48522/0 (socket says 172.21.15.120:48522) 2024-09-19T09:05:24.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.400+0000 7f7f2d2fa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b594d6dc0 0x555b594e1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:59700/0 (socket says 172.21.15.120:59700) 2024-09-19T09:05:24.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.400+0000 7f7f2dafb640 1 -- 172.21.15.120:0/3472834177 learned_addr learned my addr 172.21.15.120:0/3472834177 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:24.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.400+0000 7f7f2e2fc640 1 -- 172.21.15.120:0/3472834177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b594d6dc0 msgr2=0x555b594e1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:24.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.400+0000 7f7f2e2fc640 1 --2- 172.21.15.120:0/3472834177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b594d6dc0 0x555b594e1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.400+0000 7f7f2e2fc640 1 -- 172.21.15.120:0/3472834177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b594d64a0 msgr2=0x555b594d6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:24.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.400+0000 7f7f2e2fc640 1 --2- 172.21.15.120:0/3472834177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b594d64a0 0x555b594d6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.400+0000 7f7f2e2fc640 1 -- 172.21.15.120:0/3472834177 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b593c8440 con 0x555b594e1680 2024-09-19T09:05:24.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.400+0000 7f7f2dafb640 1 --2- 172.21.15.120:0/3472834177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b594d64a0 0x555b594d6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:24.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.400+0000 7f7f2e2fc640 1 --2- 172.21.15.120:0/3472834177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b594e1680 0x555b594e3a70 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7f20009ca0 tx=0x7f7f20009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad81299cbfa3dbe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:24.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.400+0000 7f7f13fff640 1 -- 172.21.15.120:0/3472834177 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20048070 con 0x555b594e1680 2024-09-19T09:05:24.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.400+0000 7f7f13fff640 1 -- 172.21.15.120:0/3472834177 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f20036070 con 0x555b594e1680 2024-09-19T09:05:24.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.400+0000 7f7f13fff640 1 -- 172.21.15.120:0/3472834177 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20004480 con 0x555b594e1680 2024-09-19T09:05:24.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.401+0000 7f7f32138740 1 -- 172.21.15.120:0/3472834177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b594e1680 msgr2=0x555b594e3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:24.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.401+0000 7f7f32138740 1 --2- 172.21.15.120:0/3472834177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b594e1680 0x555b594e3a70 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7f20009ca0 tx=0x7f7f20009a20 comp rx=0 tx=0).stop 2024-09-19T09:05:24.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.401+0000 7f7f32138740 1 -- 172.21.15.120:0/3472834177 shutdown_connections 2024-09-19T09:05:24.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.401+0000 7f7f32138740 1 --2- 172.21.15.120:0/3472834177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b594e1680 0x555b594e3a70 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.401+0000 7f7f32138740 1 --2- 172.21.15.120:0/3472834177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b594d6dc0 0x555b594e1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.401+0000 7f7f32138740 1 --2- 172.21.15.120:0/3472834177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b594d64a0 0x555b594d6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.401+0000 7f7f32138740 1 -- 172.21.15.120:0/3472834177 >> 172.21.15.120:0/3472834177 conn(0x555b594d2a30 msgr2=0x555b594d2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:24.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.401+0000 7f7f32138740 1 -- 172.21.15.120:0/3472834177 shutdown_connections 2024-09-19T09:05:24.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.401+0000 7f7f32138740 1 -- 172.21.15.120:0/3472834177 wait complete. 2024-09-19T09:05:24.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.401+0000 7f7f32138740 1 -- start start 2024-09-19T09:05:24.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.401+0000 7f7f32138740 1 Processor -- start 2024-09-19T09:05:24.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.402+0000 7f7f32138740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b594d64a0 0x555b594e0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:24.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.402+0000 7f7f32138740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b594d6dc0 0x555b594de230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:24.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.402+0000 7f7f32138740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b594e1680 0x555b594de770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:24.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.402+0000 7f7f32138740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555b594e4250 con 0x555b594d6dc0 2024-09-19T09:05:24.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.402+0000 7f7f32138740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555b593f7050 con 0x555b594d64a0 2024-09-19T09:05:24.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.402+0000 7f7f32138740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555b594e43d0 con 0x555b594e1680 2024-09-19T09:05:24.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.402+0000 7f7f32138740 10 client.0 _ll_register_callbacks cb 0x555b59437160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:24.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.402+0000 7f7f2d2fa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b594d6dc0 0x555b594de230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:24.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.402+0000 7f7f2d2fa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b594d6dc0 0x555b594de230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36174/0 (socket says 172.21.15.120:36174) 2024-09-19T09:05:24.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.402+0000 7f7f2dafb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b594d64a0 0x555b594e0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:24.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.402+0000 7f7f2d2fa640 1 -- 172.21.15.120:0/4241692716 learned_addr learned my addr 172.21.15.120:0/4241692716 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:24.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.402+0000 7f7f2e2fc640 1 --2- 172.21.15.120:0/4241692716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b594e1680 0x555b594de770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:24.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.402+0000 7f7f2dafb640 1 -- 172.21.15.120:0/4241692716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b594e1680 msgr2=0x555b594de770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:24.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.402+0000 7f7f2dafb640 1 --2- 172.21.15.120:0/4241692716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b594e1680 0x555b594de770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.402+0000 7f7f2dafb640 1 -- 172.21.15.120:0/4241692716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b594d6dc0 msgr2=0x555b594de230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:24.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.402+0000 7f7f2dafb640 1 --2- 172.21.15.120:0/4241692716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b594d6dc0 0x555b594de230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.402+0000 7f7f2dafb640 1 -- 172.21.15.120:0/4241692716 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b591ad3c0 con 0x555b594d64a0 2024-09-19T09:05:24.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.402+0000 7f7f2d2fa640 1 --2- 172.21.15.120:0/4241692716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b594d6dc0 0x555b594de230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:24.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.402+0000 7f7f2dafb640 1 --2- 172.21.15.120:0/4241692716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b594d64a0 0x555b594e0f80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7f240a4900 tx=0x7f7f2409c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a5f112685436e65 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:24.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.402+0000 7f7f117fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:05:24.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.403+0000 7f7f117fa640 1 -- 172.21.15.120:0/4241692716 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f240b5060 con 0x555b594d64a0 2024-09-19T09:05:24.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.403+0000 7f7f117fa640 1 -- 172.21.15.120:0/4241692716 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f7f240a0170 con 0x555b594d64a0 2024-09-19T09:05:24.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.403+0000 7f7f32138740 1 -- 172.21.15.120:0/4241692716 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f20036070 con 0x555b594d64a0 2024-09-19T09:05:24.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.403+0000 7f7f117fa640 1 -- 172.21.15.120:0/4241692716 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7f240bc2c0 con 0x555b594d64a0 2024-09-19T09:05:24.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.403+0000 7f7f32138740 10 client.25583 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:24.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.403+0000 7f7f32138740 1 -- 172.21.15.120:0/4241692716 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555b594efc20 con 0x555b594d64a0 2024-09-19T09:05:24.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.403+0000 7f7f117fa640 1 -- 172.21.15.120:0/4241692716 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f20036070 con 0x555b594d64a0 2024-09-19T09:05:24.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.403+0000 7f7f117fa640 1 -- 172.21.15.120:0/4241692716 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7f24018c30 con 0x555b594d64a0 2024-09-19T09:05:24.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.403+0000 7f7f32138740 10 client.25583 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:24.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.403+0000 7f7f32138740 10 client.25583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:24.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.403+0000 7f7f32138740 20 client.25583 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:24.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.403+0000 7f7f32138740 1 --2- 172.21.15.120:0/4241692716 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555b594f10e0 0x555b594f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:24.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.403+0000 7f7f32138740 4 client.25583 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:05:24.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.403+0000 7f7f32138740 1 -- 172.21.15.120:0/4241692716 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b59449850 con 0x555b594f10e0 2024-09-19T09:05:24.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.403+0000 7f7f2d2fa640 1 --2- 172.21.15.120:0/4241692716 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555b594f10e0 0x555b594f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:24.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.404+0000 7f7f117fa640 10 client.25583 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:24.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.404+0000 7f7f2d2fa640 1 --2- 172.21.15.120:0/4241692716 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555b594f10e0 0x555b594f34f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5c80cfe0a7d16c9 server_cookie=d4cb90e1adfef218 in_seq=0 out_seq=0 2024-09-19T09:05:24.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.405+0000 7f7f117fa640 1 -- 172.21.15.120:0/4241692716 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b59449850 con 0x555b594f10e0 2024-09-19T09:05:24.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.405+0000 7f7f117fa640 10 client.25583 handle_command_reply: tid=0 2024-09-19T09:05:24.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.405+0000 7f7f32138740 1 client.25583 shutdown 2024-09-19T09:05:24.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.405+0000 7f7f32138740 1 -- 172.21.15.120:0/4241692716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b594d64a0 msgr2=0x555b594e0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:24.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.405+0000 7f7f32138740 1 --2- 172.21.15.120:0/4241692716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b594d64a0 0x555b594e0f80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7f240a4900 tx=0x7f7f2409c3b0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.405+0000 7f7f32138740 1 -- 172.21.15.120:0/4241692716 shutdown_connections 2024-09-19T09:05:24.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.405+0000 7f7f32138740 1 --2- 172.21.15.120:0/4241692716 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555b594f10e0 0x555b594f34f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.405+0000 7f7f32138740 1 --2- 172.21.15.120:0/4241692716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b594e1680 0x555b594de770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.405+0000 7f7f32138740 1 --2- 172.21.15.120:0/4241692716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b594d6dc0 0x555b594de230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.405+0000 7f7f32138740 1 --2- 172.21.15.120:0/4241692716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b594d64a0 0x555b594e0f80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.405+0000 7f7f32138740 1 -- 172.21.15.120:0/4241692716 >> 172.21.15.120:0/4241692716 conn(0x555b594d2a30 msgr2=0x555b594d8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:24.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.405+0000 7f7f117fa640 0 client.25583 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:24.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.405+0000 7f7f32138740 1 -- 172.21.15.120:0/4241692716 shutdown_connections 2024-09-19T09:05:24.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.405+0000 7f7f32138740 1 -- 172.21.15.120:0/4241692716 wait complete. 2024-09-19T09:05:24.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.405+0000 7f7f32138740 20 client.25583 trim_cache size 0 max 16384 2024-09-19T09:05:24.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.422+0000 7f7f32138740 1 Processor -- start 2024-09-19T09:05:24.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.422+0000 7f7f32138740 1 -- start start 2024-09-19T09:05:24.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.422+0000 7f7f32138740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b594d2a30 0x555b594ed4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:24.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.422+0000 7f7f32138740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b594d64a0 0x555b594f8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:24.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.422+0000 7f7f32138740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b594d6dc0 0x555b594f9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:24.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.422+0000 7f7f32138740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7f281ad650 con 0x555b594d64a0 2024-09-19T09:05:24.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.422+0000 7f7f32138740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7f281af190 con 0x555b594d2a30 2024-09-19T09:05:24.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.422+0000 7f7f32138740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7f281add20 con 0x555b594d6dc0 2024-09-19T09:05:24.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.423+0000 7f7f2d2fa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b594d64a0 0x555b594f8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:24.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.423+0000 7f7f2dafb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b594d2a30 0x555b594ed4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:24.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.423+0000 7f7f2e2fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b594d6dc0 0x555b594f9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:24.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.423+0000 7f7f2dafb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b594d2a30 0x555b594ed4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:48546/0 (socket says 172.21.15.120:48546) 2024-09-19T09:05:24.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.423+0000 7f7f2dafb640 1 -- 172.21.15.120:0/4276554210 learned_addr learned my addr 172.21.15.120:0/4276554210 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:24.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.423+0000 7f7f2d2fa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b594d64a0 0x555b594f8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36180/0 (socket says 172.21.15.120:36180) 2024-09-19T09:05:24.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.423+0000 7f7f2d2fa640 1 -- 172.21.15.120:0/4276554210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b594d6dc0 msgr2=0x555b594f9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:24.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.423+0000 7f7f2d2fa640 1 --2- 172.21.15.120:0/4276554210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b594d6dc0 0x555b594f9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.423+0000 7f7f2d2fa640 1 -- 172.21.15.120:0/4276554210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b594d2a30 msgr2=0x555b594ed4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:24.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.423+0000 7f7f2d2fa640 1 --2- 172.21.15.120:0/4276554210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b594d2a30 0x555b594ed4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.423+0000 7f7f2d2fa640 1 -- 172.21.15.120:0/4276554210 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b594e12f0 con 0x555b594d64a0 2024-09-19T09:05:24.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.423+0000 7f7f2dafb640 1 --2- 172.21.15.120:0/4276554210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b594d2a30 0x555b594ed4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:24.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.423+0000 7f7f2d2fa640 1 --2- 172.21.15.120:0/4276554210 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b594d64a0 0x555b594f8dc0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x555b594f3d00 tx=0x7f7f18012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbf129a35c2ece6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:24.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.423+0000 7f7f137fe640 1 -- 172.21.15.120:0/4276554210 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f18016070 con 0x555b594d64a0 2024-09-19T09:05:24.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.423+0000 7f7f137fe640 1 -- 172.21.15.120:0/4276554210 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f180047a0 con 0x555b594d64a0 2024-09-19T09:05:24.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.423+0000 7f7f137fe640 1 -- 172.21.15.120:0/4276554210 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1802a7e0 con 0x555b594d64a0 2024-09-19T09:05:24.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.424+0000 7f7f32138740 1 -- 172.21.15.120:0/4276554210 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b594d64a0 msgr2=0x555b594f8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:24.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.424+0000 7f7f32138740 1 --2- 172.21.15.120:0/4276554210 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b594d64a0 0x555b594f8dc0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x555b594f3d00 tx=0x7f7f18012000 comp rx=0 tx=0).stop 2024-09-19T09:05:24.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.424+0000 7f7f32138740 1 -- 172.21.15.120:0/4276554210 shutdown_connections 2024-09-19T09:05:24.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.424+0000 7f7f32138740 1 --2- 172.21.15.120:0/4276554210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b594d6dc0 0x555b594f9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.424+0000 7f7f32138740 1 --2- 172.21.15.120:0/4276554210 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b594d64a0 0x555b594f8dc0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.424+0000 7f7f32138740 1 --2- 172.21.15.120:0/4276554210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b594d2a30 0x555b594ed4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.424+0000 7f7f32138740 1 -- 172.21.15.120:0/4276554210 >> 172.21.15.120:0/4276554210 conn(0x555b594de8d0 msgr2=0x555b594dcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:24.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.424+0000 7f7f32138740 1 -- 172.21.15.120:0/4276554210 shutdown_connections 2024-09-19T09:05:24.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.424+0000 7f7f32138740 1 -- 172.21.15.120:0/4276554210 wait complete. 2024-09-19T09:05:24.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.424+0000 7f7f32138740 1 -- start start 2024-09-19T09:05:24.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.425+0000 7f7f32138740 1 Processor -- start 2024-09-19T09:05:24.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.425+0000 7f7f32138740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b594d2a30 0x555b594e9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:24.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.425+0000 7f7f32138740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b594d64a0 0x555b594e5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:24.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.425+0000 7f7f32138740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b594d6dc0 0x555b594e56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:24.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.425+0000 7f7f32138740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7f281aeac0 con 0x555b594d6dc0 2024-09-19T09:05:24.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.425+0000 7f7f32138740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7f281ae3f0 con 0x555b594d64a0 2024-09-19T09:05:24.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.425+0000 7f7f32138740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7f281ab610 con 0x555b594d2a30 2024-09-19T09:05:24.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.425+0000 7f7f32138740 10 client.0 _ll_register_callbacks cb 0x7f7f240cd5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:24.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.425+0000 7f7f2dafb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b594d2a30 0x555b594e9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:24.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.425+0000 7f7f2e2fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b594d6dc0 0x555b594e56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:24.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.425+0000 7f7f2dafb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b594d2a30 0x555b594e9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:59724/0 (socket says 172.21.15.120:59724) 2024-09-19T09:05:24.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.425+0000 7f7f2dafb640 1 -- 172.21.15.120:0/998064124 learned_addr learned my addr 172.21.15.120:0/998064124 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:24.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.425+0000 7f7f2d2fa640 1 --2- 172.21.15.120:0/998064124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b594d64a0 0x555b594e5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:24.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.425+0000 7f7f2dafb640 1 -- 172.21.15.120:0/998064124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b594d64a0 msgr2=0x555b594e5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:24.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.425+0000 7f7f2dafb640 1 --2- 172.21.15.120:0/998064124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b594d64a0 0x555b594e5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.425+0000 7f7f2dafb640 1 -- 172.21.15.120:0/998064124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b594d6dc0 msgr2=0x555b594e56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:24.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.425+0000 7f7f2dafb640 1 --2- 172.21.15.120:0/998064124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b594d6dc0 0x555b594e56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.425+0000 7f7f2dafb640 1 -- 172.21.15.120:0/998064124 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b594e5f00 con 0x555b594d2a30 2024-09-19T09:05:24.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.425+0000 7f7f2e2fc640 1 --2- 172.21.15.120:0/998064124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b594d6dc0 0x555b594e56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:24.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.426+0000 7f7f2dafb640 1 --2- 172.21.15.120:0/998064124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b594d2a30 0x555b594e9e20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7f24018fd0 tx=0x7f7f240b5270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c16872225aa15f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:24.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.426+0000 7f7f117fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:05:24.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.426+0000 7f7f117fa640 1 -- 172.21.15.120:0/998064124 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f240a0370 con 0x555b594d2a30 2024-09-19T09:05:24.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.426+0000 7f7f117fa640 1 -- 172.21.15.120:0/998064124 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f7f240a0170 con 0x555b594d2a30 2024-09-19T09:05:24.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.426+0000 7f7f32138740 1 -- 172.21.15.120:0/998064124 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f180047a0 con 0x555b594d2a30 2024-09-19T09:05:24.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.426+0000 7f7f117fa640 1 -- 172.21.15.120:0/998064124 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7f2409a630 con 0x555b594d2a30 2024-09-19T09:05:24.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.426+0000 7f7f32138740 10 client.25564 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:24.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.426+0000 7f7f32138740 1 -- 172.21.15.120:0/998064124 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555b5955be40 con 0x555b594d2a30 2024-09-19T09:05:24.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.426+0000 7f7f117fa640 1 -- 172.21.15.120:0/998064124 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f180047a0 con 0x555b594d2a30 2024-09-19T09:05:24.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.426+0000 7f7f117fa640 1 -- 172.21.15.120:0/998064124 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7f240ab070 con 0x555b594d2a30 2024-09-19T09:05:24.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.426+0000 7f7f32138740 10 client.25564 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:24.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.426+0000 7f7f32138740 10 client.25564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:24.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.426+0000 7f7f32138740 20 client.25564 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:24.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.426+0000 7f7f32138740 1 --2- 172.21.15.120:0/998064124 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555b594e1680 0x555b5955e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:24.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.426+0000 7f7f32138740 4 client.25564 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:05:24.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.426+0000 7f7f32138740 1 -- 172.21.15.120:0/998064124 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555b59443d60 con 0x555b594e1680 2024-09-19T09:05:24.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.426+0000 7f7f2d2fa640 1 --2- 172.21.15.120:0/998064124 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555b594e1680 0x555b5955e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:24.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.427+0000 7f7f117fa640 10 client.25564 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:24.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.427+0000 7f7f2d2fa640 1 --2- 172.21.15.120:0/998064124 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555b594e1680 0x555b5955e570 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dd782984820de61 server_cookie=bbd40edb4ee96bce in_seq=0 out_seq=0 2024-09-19T09:05:24.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.427+0000 7f7f117fa640 1 -- 172.21.15.120:0/998064124 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555b59443d60 con 0x555b594e1680 2024-09-19T09:05:24.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.427+0000 7f7f117fa640 10 client.25564 handle_command_reply: tid=0 2024-09-19T09:05:24.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.427+0000 7f7f32138740 1 client.25564 shutdown 2024-09-19T09:05:24.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.427+0000 7f7f32138740 1 -- 172.21.15.120:0/998064124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b594d2a30 msgr2=0x555b594e9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:24.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.427+0000 7f7f32138740 1 --2- 172.21.15.120:0/998064124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b594d2a30 0x555b594e9e20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7f24018fd0 tx=0x7f7f240b5270 comp rx=0 tx=0).stop 2024-09-19T09:05:24.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.427+0000 7f7f32138740 1 -- 172.21.15.120:0/998064124 shutdown_connections 2024-09-19T09:05:24.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.427+0000 7f7f32138740 1 --2- 172.21.15.120:0/998064124 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555b594e1680 0x555b5955e570 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.427+0000 7f7f32138740 1 --2- 172.21.15.120:0/998064124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b594d6dc0 0x555b594e56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.427+0000 7f7f32138740 1 --2- 172.21.15.120:0/998064124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555b594d64a0 0x555b594e5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.427+0000 7f7f117fa640 0 client.25564 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:24.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.427+0000 7f7f32138740 1 --2- 172.21.15.120:0/998064124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b594d2a30 0x555b594e9e20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.427+0000 7f7f32138740 1 -- 172.21.15.120:0/998064124 >> 172.21.15.120:0/998064124 conn(0x555b594de8d0 msgr2=0x555b594dfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:24.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.427+0000 7f7f32138740 1 -- 172.21.15.120:0/998064124 shutdown_connections 2024-09-19T09:05:24.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.427+0000 7f7f32138740 1 -- 172.21.15.120:0/998064124 wait complete. 2024-09-19T09:05:24.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.427+0000 7f7f32138740 20 client.25564 trim_cache size 0 max 16384 2024-09-19T09:05:24.442 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:05:24.442 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:05:24.442 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:05:24.442 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:05:24.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.428+0000 7f7f2fd5f640 1 -- 172.21.15.120:0/2635567165 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7f140a3240 msgr2=0x7f7f140a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:24.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.428+0000 7f7f2fd5f640 1 --2- 172.21.15.120:0/2635567165 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7f140a3240 0x7f7f140a5700 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7f280fed80 tx=0x7f7f1800a250 comp rx=0 tx=0).stop 2024-09-19T09:05:24.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.428+0000 7f7f2fd5f640 1 -- 172.21.15.120:0/2635567165 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f28100a80 msgr2=0x7f7f28198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:24.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.428+0000 7f7f2fd5f640 1 --2- 172.21.15.120:0/2635567165 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f28100a80 0x7f7f28198440 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7f2400e9f0 tx=0x7f7f2400eeb0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.428+0000 7f7f2fd5f640 1 -- 172.21.15.120:0/2635567165 shutdown_connections 2024-09-19T09:05:24.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.428+0000 7f7f2fd5f640 1 --2- 172.21.15.120:0/2635567165 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7f140a3240 0x7f7f140a5700 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.428+0000 7f7f2fd5f640 1 --2- 172.21.15.120:0/2635567165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f28106a80 0x7f7f2819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.428+0000 7f7f2fd5f640 1 --2- 172.21.15.120:0/2635567165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f28101420 0x7f7f28198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.428+0000 7f7f2fd5f640 1 --2- 172.21.15.120:0/2635567165 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f28100a80 0x7f7f28198440 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:24.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.428+0000 7f7f2fd5f640 1 -- 172.21.15.120:0/2635567165 >> 172.21.15.120:0/2635567165 conn(0x7f7f280fc7a0 msgr2=0x7f7f280fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:24.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.428+0000 7f7f2fd5f640 1 -- 172.21.15.120:0/2635567165 shutdown_connections 2024-09-19T09:05:24.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:24.428+0000 7f7f2fd5f640 1 -- 172.21.15.120:0/2635567165 wait complete. 2024-09-19T09:05:25.038 INFO:tasks.workunit.client.0.smithi120.stdout: 1 115248 7.98 MB/sec warmup 80 sec latency 13.683 ms 2024-09-19T09:05:25.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:24 smithi120 ceph-mon[31150]: pgmap v417: 97 pgs: 97 active+clean; 40 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.6 MiB/s wr, 321 op/s 2024-09-19T09:05:25.154 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:05:25.154 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:05:25.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:24 smithi196 ceph-mon[35150]: pgmap v417: 97 pgs: 97 active+clean; 40 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.6 MiB/s wr, 321 op/s 2024-09-19T09:05:25.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:24 smithi153 ceph-mon[35071]: pgmap v417: 97 pgs: 97 active+clean; 40 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.6 MiB/s wr, 321 op/s 2024-09-19T09:05:25.420 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:05:25.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.947+0000 7f9808341640 1 -- 172.21.15.120:0/3320332955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9800100e20 msgr2=0x7f9800101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:25.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.947+0000 7f9808341640 1 --2- 172.21.15.120:0/3320332955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9800100e20 0x7f9800101200 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f97f0009ca0 tx=0x7f97f002a950 comp rx=0 tx=0).stop 2024-09-19T09:05:25.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.948+0000 7f9808341640 1 -- 172.21.15.120:0/3320332955 shutdown_connections 2024-09-19T09:05:25.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.948+0000 7f9808341640 1 --2- 172.21.15.120:0/3320332955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9800102ef0 0x7f980010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:25.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.948+0000 7f9808341640 1 --2- 172.21.15.120:0/3320332955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9800102570 0x7f98001029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:25.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.948+0000 7f9808341640 1 --2- 172.21.15.120:0/3320332955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9800100e20 0x7f9800101200 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:25.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.948+0000 7f9808341640 1 -- 172.21.15.120:0/3320332955 >> 172.21.15.120:0/3320332955 conn(0x7f98000fc7d0 msgr2=0x7f98000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:25.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.948+0000 7f9808341640 1 -- 172.21.15.120:0/3320332955 shutdown_connections 2024-09-19T09:05:25.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.948+0000 7f9808341640 1 -- 172.21.15.120:0/3320332955 wait complete. 2024-09-19T09:05:25.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.948+0000 7f9808341640 1 Processor -- start 2024-09-19T09:05:25.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.948+0000 7f9808341640 1 -- start start 2024-09-19T09:05:25.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.948+0000 7f9808341640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9800100e20 0x7f98001985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:25.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.948+0000 7f9808341640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9800102570 0x7f9800198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:25.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.948+0000 7f9808341640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9800102ef0 0x7f980019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:25.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.948+0000 7f9808341640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f98001120a0 con 0x7f9800102ef0 2024-09-19T09:05:25.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.948+0000 7f9808341640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9800111f20 con 0x7f9800102570 2024-09-19T09:05:25.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.948+0000 7f9808341640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9800112220 con 0x7f9800100e20 2024-09-19T09:05:25.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.949+0000 7f98060dd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9800100e20 0x7f98001985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:25.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.949+0000 7f98068de640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9800102ef0 0x7f980019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:25.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.949+0000 7f98058dc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9800102570 0x7f9800198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:25.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.949+0000 7f98068de640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9800102ef0 0x7f980019ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36192/0 (socket says 172.21.15.120:36192) 2024-09-19T09:05:25.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.949+0000 7f98068de640 1 -- 172.21.15.120:0/2507995930 learned_addr learned my addr 172.21.15.120:0/2507995930 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:25.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.949+0000 7f98060dd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9800100e20 0x7f98001985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:59742/0 (socket says 172.21.15.120:59742) 2024-09-19T09:05:25.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.949+0000 7f98068de640 1 -- 172.21.15.120:0/2507995930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9800100e20 msgr2=0x7f98001985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:25.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.949+0000 7f98068de640 1 --2- 172.21.15.120:0/2507995930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9800100e20 0x7f98001985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:25.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.949+0000 7f98068de640 1 -- 172.21.15.120:0/2507995930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9800102570 msgr2=0x7f9800198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:25.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.949+0000 7f98068de640 1 --2- 172.21.15.120:0/2507995930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9800102570 0x7f9800198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:25.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.949+0000 7f98068de640 1 -- 172.21.15.120:0/2507995930 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f980019d550 con 0x7f9800102ef0 2024-09-19T09:05:25.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.949+0000 7f98068de640 1 --2- 172.21.15.120:0/2507995930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9800102ef0 0x7f980019ce70 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f97f800e970 tx=0x7f97f800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:25.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.949+0000 7f97ff7fe640 1 -- 172.21.15.120:0/2507995930 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f800ccc0 con 0x7f9800102ef0 2024-09-19T09:05:25.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.949+0000 7f97ff7fe640 1 -- 172.21.15.120:0/2507995930 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97f800ce60 con 0x7f9800102ef0 2024-09-19T09:05:25.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.949+0000 7f9808341640 1 -- 172.21.15.120:0/2507995930 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f980019d840 con 0x7f9800102ef0 2024-09-19T09:05:25.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.949+0000 7f97ff7fe640 1 -- 172.21.15.120:0/2507995930 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f8010490 con 0x7f9800102ef0 2024-09-19T09:05:25.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.949+0000 7f9808341640 1 -- 172.21.15.120:0/2507995930 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98001a5120 con 0x7f9800102ef0 2024-09-19T09:05:25.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.950+0000 7f9808341640 1 -- 172.21.15.120:0/2507995930 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9800101200 con 0x7f9800102ef0 2024-09-19T09:05:25.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.952+0000 7f97ff7fe640 1 -- 172.21.15.120:0/2507995930 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f97f8005000 con 0x7f9800102ef0 2024-09-19T09:05:25.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.952+0000 7f97ff7fe640 1 --2- 172.21.15.120:0/2507995930 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f97e80a3240 0x7f97e80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:25.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.952+0000 7f97ff7fe640 1 -- 172.21.15.120:0/2507995930 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f97f80cff00 con 0x7f9800102ef0 2024-09-19T09:05:25.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.952+0000 7f98060dd640 1 --2- 172.21.15.120:0/2507995930 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f97e80a3240 0x7f97e80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:25.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.953+0000 7f98060dd640 1 --2- 172.21.15.120:0/2507995930 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f97e80a3240 0x7f97e80a5700 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f97f0004ad0 tx=0x7f97f0004a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:25.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:25.955+0000 7f97ff7fe640 1 -- 172.21.15.120:0/2507995930 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f97f8054210 con 0x7f9800102ef0 2024-09-19T09:05:26.038 INFO:tasks.workunit.client.0.smithi120.stdout: 1 118135 8.07 MB/sec warmup 81 sec latency 27.086 ms 2024-09-19T09:05:26.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:26.169+0000 7f9808341640 1 -- 172.21.15.120:0/2507995930 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9800199900 con 0x7f9800102ef0 2024-09-19T09:05:26.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:26.170+0000 7f97ff7fe640 1 -- 172.21.15.120:0/2507995930 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f9800199900 con 0x7f9800102ef0 2024-09-19T09:05:26.170 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:05:26.170 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":28,"btime":"2024-09-19T09:02:41:696728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:02:41.207357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:05:26.171 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 28 2024-09-19T09:05:26.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:26.172+0000 7f9808341640 1 -- 172.21.15.120:0/2507995930 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f97e80a3240 msgr2=0x7f97e80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:26.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:26.172+0000 7f9808341640 1 --2- 172.21.15.120:0/2507995930 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f97e80a3240 0x7f97e80a5700 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f97f0004ad0 tx=0x7f97f0004a60 comp rx=0 tx=0).stop 2024-09-19T09:05:26.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:26.172+0000 7f9808341640 1 -- 172.21.15.120:0/2507995930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9800102ef0 msgr2=0x7f980019ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:26.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:26.172+0000 7f9808341640 1 --2- 172.21.15.120:0/2507995930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9800102ef0 0x7f980019ce70 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f97f800e970 tx=0x7f97f800ed70 comp rx=0 tx=0).stop 2024-09-19T09:05:26.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:26.172+0000 7f9808341640 1 -- 172.21.15.120:0/2507995930 shutdown_connections 2024-09-19T09:05:26.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:26.172+0000 7f9808341640 1 --2- 172.21.15.120:0/2507995930 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f97e80a3240 0x7f97e80a5700 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:26.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:26.172+0000 7f9808341640 1 --2- 172.21.15.120:0/2507995930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9800102ef0 0x7f980019ce70 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:26.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:26.172+0000 7f9808341640 1 --2- 172.21.15.120:0/2507995930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9800102570 0x7f9800198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:26.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:26.172+0000 7f9808341640 1 --2- 172.21.15.120:0/2507995930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9800100e20 0x7f98001985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:26.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:26.172+0000 7f9808341640 1 -- 172.21.15.120:0/2507995930 >> 172.21.15.120:0/2507995930 conn(0x7f98000fc7d0 msgr2=0x7f98000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:26.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:26.172+0000 7f9808341640 1 -- 172.21.15.120:0/2507995930 shutdown_connections 2024-09-19T09:05:26.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:26.172+0000 7f9808341640 1 -- 172.21.15.120:0/2507995930 wait complete. 2024-09-19T09:05:26.931 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:26 smithi120 ceph-mon[31150]: pgmap v418: 97 pgs: 97 active+clean; 45 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 376 op/s 2024-09-19T09:05:26.931 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:26 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2507995930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:05:26.960 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:05:27.038 INFO:tasks.workunit.client.0.smithi120.stdout: 1 121300 8.19 MB/sec warmup 82 sec latency 12.703 ms 2024-09-19T09:05:27.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:26 smithi196 ceph-mon[35150]: pgmap v418: 97 pgs: 97 active+clean; 45 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 376 op/s 2024-09-19T09:05:27.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:26 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2507995930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:05:27.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:26 smithi153 ceph-mon[35071]: pgmap v418: 97 pgs: 97 active+clean; 45 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 376 op/s 2024-09-19T09:05:27.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:26 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2507995930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:05:27.225 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:05:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.620+0000 7f83e28e4640 1 -- 172.21.15.120:0/2268205080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83dc1025f0 msgr2=0x7f83dc108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.620+0000 7f83e28e4640 1 --2- 172.21.15.120:0/2268205080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83dc1025f0 0x7f83dc108e80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f83d4009ca0 tx=0x7f83d402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.620+0000 7f83e28e4640 1 -- 172.21.15.120:0/2268205080 shutdown_connections 2024-09-19T09:05:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.620+0000 7f83e28e4640 1 --2- 172.21.15.120:0/2268205080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83dc1025f0 0x7f83dc108e80 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.620+0000 7f83e28e4640 1 --2- 172.21.15.120:0/2268205080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83dc101c30 0x7f83dc1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.620+0000 7f83e28e4640 1 --2- 172.21.15.120:0/2268205080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83dc100a40 0x7f83dc100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.620+0000 7f83e28e4640 1 -- 172.21.15.120:0/2268205080 >> 172.21.15.120:0/2268205080 conn(0x7f83dc0fc1e0 msgr2=0x7f83dc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.620+0000 7f83e28e4640 1 -- 172.21.15.120:0/2268205080 shutdown_connections 2024-09-19T09:05:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.620+0000 7f83e28e4640 1 -- 172.21.15.120:0/2268205080 wait complete. 2024-09-19T09:05:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.620+0000 7f83e28e4640 1 Processor -- start 2024-09-19T09:05:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.620+0000 7f83e28e4640 1 -- start start 2024-09-19T09:05:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.621+0000 7f83e28e4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83dc100a40 0x7f83dc195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.621+0000 7f83e28e4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83dc101c30 0x7f83dc1960a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.621+0000 7f83e28e4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83dc1025f0 0x7f83dc19d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.621+0000 7f83e28e4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc10b660 con 0x7f83dc101c30 2024-09-19T09:05:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.621+0000 7f83e28e4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc10b4e0 con 0x7f83dc1025f0 2024-09-19T09:05:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.621+0000 7f83e28e4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc10b7e0 con 0x7f83dc100a40 2024-09-19T09:05:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.621+0000 7f83db7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83dc101c30 0x7f83dc1960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.621+0000 7f83dbfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83dc100a40 0x7f83dc195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.621+0000 7f83db7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83dc101c30 0x7f83dc1960a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36206/0 (socket says 172.21.15.120:36206) 2024-09-19T09:05:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.621+0000 7f83db7fe640 1 -- 172.21.15.120:0/1598112642 learned_addr learned my addr 172.21.15.120:0/1598112642 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.621+0000 7f83e0e81640 1 --2- 172.21.15.120:0/1598112642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83dc1025f0 0x7f83dc19d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.621+0000 7f83db7fe640 1 -- 172.21.15.120:0/1598112642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83dc100a40 msgr2=0x7f83dc195b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.621+0000 7f83db7fe640 1 --2- 172.21.15.120:0/1598112642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83dc100a40 0x7f83dc195b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.621+0000 7f83db7fe640 1 -- 172.21.15.120:0/1598112642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83dc1025f0 msgr2=0x7f83dc19d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.621+0000 7f83db7fe640 1 --2- 172.21.15.120:0/1598112642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83dc1025f0 0x7f83dc19d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.621+0000 7f83db7fe640 1 -- 172.21.15.120:0/1598112642 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83dc19d780 con 0x7f83dc101c30 2024-09-19T09:05:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.621+0000 7f83db7fe640 1 --2- 172.21.15.120:0/1598112642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83dc101c30 0x7f83dc1960a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f83d0012fc0 tx=0x7f83d0010490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.621+0000 7f83d97fa640 1 -- 172.21.15.120:0/1598112642 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d0010db0 con 0x7f83dc101c30 2024-09-19T09:05:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.621+0000 7f83e28e4640 1 -- 172.21.15.120:0/1598112642 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83dc19da10 con 0x7f83dc101c30 2024-09-19T09:05:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.621+0000 7f83d97fa640 1 -- 172.21.15.120:0/1598112642 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83d0005020 con 0x7f83dc101c30 2024-09-19T09:05:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.621+0000 7f83d97fa640 1 -- 172.21.15.120:0/1598112642 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d0004a90 con 0x7f83dc101c30 2024-09-19T09:05:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.622+0000 7f83e28e4640 1 -- 172.21.15.120:0/1598112642 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83dc19df10 con 0x7f83dc101c30 2024-09-19T09:05:27.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.624+0000 7f83d97fa640 1 -- 172.21.15.120:0/1598112642 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f83d001f020 con 0x7f83dc101c30 2024-09-19T09:05:27.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.624+0000 7f83e5683740 1 Processor -- start 2024-09-19T09:05:27.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.624+0000 7f83e5683740 1 -- start start 2024-09-19T09:05:27.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.624+0000 7f83e5683740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69db59a0 0x55ef69db5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.624+0000 7f83e5683740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef69db62e0 0x55ef69dbada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.624+0000 7f83e5683740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef69dbb370 0x55ef69dbd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.624+0000 7f83e5683740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ef69d192f0 con 0x55ef69db59a0 2024-09-19T09:05:27.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.624+0000 7f83e5683740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ef69cc1270 con 0x55ef69dbb370 2024-09-19T09:05:27.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.624+0000 7f83e5683740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ef69d152b0 con 0x55ef69db62e0 2024-09-19T09:05:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.625+0000 7f83d97fa640 1 --2- 172.21.15.120:0/1598112642 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83c40a32f0 0x7f83c40a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.625+0000 7f83dbfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef69db62e0 0x55ef69dbada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.625+0000 7f83db7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef69dbb370 0x55ef69dbd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.625+0000 7f83dbfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef69db62e0 0x55ef69dbada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:59766/0 (socket says 172.21.15.120:59766) 2024-09-19T09:05:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.625+0000 7f83dbfff640 1 -- 172.21.15.120:0/3725425493 learned_addr learned my addr 172.21.15.120:0/3725425493 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.625+0000 7f83e0e81640 1 --2- 172.21.15.120:0/3725425493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69db59a0 0x55ef69db5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.625+0000 7f83e0e81640 1 --2- 172.21.15.120:0/1598112642 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83c40a32f0 0x7f83c40a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.625+0000 7f83dbfff640 1 -- 172.21.15.120:0/3725425493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef69dbb370 msgr2=0x55ef69dbd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.625+0000 7f83dbfff640 1 --2- 172.21.15.120:0/3725425493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef69dbb370 0x55ef69dbd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.625+0000 7f83dbfff640 1 -- 172.21.15.120:0/3725425493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69db59a0 msgr2=0x55ef69db5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.625+0000 7f83dbfff640 1 --2- 172.21.15.120:0/3725425493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69db59a0 0x55ef69db5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.625+0000 7f83dbfff640 1 -- 172.21.15.120:0/3725425493 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef69cc0340 con 0x55ef69db62e0 2024-09-19T09:05:27.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.625+0000 7f83d97fa640 1 -- 172.21.15.120:0/1598112642 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f83d00d3030 con 0x7f83dc101c30 2024-09-19T09:05:27.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.625+0000 7f83dbfff640 1 --2- 172.21.15.120:0/3725425493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef69db62e0 0x55ef69dbada0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f83cc009ca0 tx=0x7f83cc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26f27654d4a41d2c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:27.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.625+0000 7f83e0e81640 1 --2- 172.21.15.120:0/1598112642 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83c40a32f0 0x7f83c40a57b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f83d4005e30 tx=0x7f83d4031bb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:27.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.625+0000 7f83c27fc640 1 -- 172.21.15.120:0/3725425493 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc004220 con 0x55ef69db62e0 2024-09-19T09:05:27.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.625+0000 7f83c27fc640 1 -- 172.21.15.120:0/3725425493 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83cc0043c0 con 0x55ef69db62e0 2024-09-19T09:05:27.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.625+0000 7f83c27fc640 1 -- 172.21.15.120:0/3725425493 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc019890 con 0x55ef69db62e0 2024-09-19T09:05:27.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.626+0000 7f83e5683740 1 -- 172.21.15.120:0/3725425493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef69db62e0 msgr2=0x55ef69dbada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:27.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.626+0000 7f83e5683740 1 --2- 172.21.15.120:0/3725425493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef69db62e0 0x55ef69dbada0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f83cc009ca0 tx=0x7f83cc011810 comp rx=0 tx=0).stop 2024-09-19T09:05:27.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.626+0000 7f83e5683740 1 -- 172.21.15.120:0/3725425493 shutdown_connections 2024-09-19T09:05:27.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.626+0000 7f83e5683740 1 --2- 172.21.15.120:0/3725425493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef69dbb370 0x55ef69dbd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.626+0000 7f83e5683740 1 --2- 172.21.15.120:0/3725425493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef69db62e0 0x55ef69dbada0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.626+0000 7f83e5683740 1 --2- 172.21.15.120:0/3725425493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69db59a0 0x55ef69db5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.626+0000 7f83e5683740 1 -- 172.21.15.120:0/3725425493 >> 172.21.15.120:0/3725425493 conn(0x55ef69db1010 msgr2=0x55ef69db1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:27.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.626+0000 7f83e5683740 1 -- 172.21.15.120:0/3725425493 shutdown_connections 2024-09-19T09:05:27.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.626+0000 7f83e5683740 1 -- 172.21.15.120:0/3725425493 wait complete. 2024-09-19T09:05:27.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.626+0000 7f83e5683740 1 -- start start 2024-09-19T09:05:27.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.627+0000 7f83e5683740 1 Processor -- start 2024-09-19T09:05:27.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.627+0000 7f83e5683740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef69db59a0 0x55ef69dbfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.627+0000 7f83e5683740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69db62e0 0x55ef69dc0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.627+0000 7f83e5683740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef69dbb370 0x55ef69dc70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.627+0000 7f83e5683740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ef69dbe340 con 0x55ef69db62e0 2024-09-19T09:05:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.627+0000 7f83e5683740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ef69ca2290 con 0x55ef69db59a0 2024-09-19T09:05:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.627+0000 7f83e5683740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ef69dbe4c0 con 0x55ef69dbb370 2024-09-19T09:05:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.627+0000 7f83e5683740 10 client.0 _ll_register_callbacks cb 0x55ef69cd3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.627+0000 7f83db7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69db62e0 0x55ef69dc0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.627+0000 7f83dbfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef69db59a0 0x55ef69dbfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.627+0000 7f83e0e81640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef69dbb370 0x55ef69dc70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.627+0000 7f83db7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69db62e0 0x55ef69dc0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36226/0 (socket says 172.21.15.120:36226) 2024-09-19T09:05:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.627+0000 7f83db7fe640 1 -- 172.21.15.120:0/1326663162 learned_addr learned my addr 172.21.15.120:0/1326663162 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.627+0000 7f83dbfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef69db59a0 0x55ef69dbfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:48608/0 (socket says 172.21.15.120:48608) 2024-09-19T09:05:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.627+0000 7f83e0e81640 1 -- 172.21.15.120:0/1326663162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef69db59a0 msgr2=0x55ef69dbfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.627+0000 7f83e0e81640 1 --2- 172.21.15.120:0/1326663162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef69db59a0 0x55ef69dbfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.627+0000 7f83e0e81640 1 -- 172.21.15.120:0/1326663162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69db62e0 msgr2=0x55ef69dc0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.627+0000 7f83e0e81640 1 --2- 172.21.15.120:0/1326663162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69db62e0 0x55ef69dc0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.627+0000 7f83e0e81640 1 -- 172.21.15.120:0/1326663162 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef69a2c210 con 0x55ef69dbb370 2024-09-19T09:05:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.627+0000 7f83dbfff640 1 --2- 172.21.15.120:0/1326663162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef69db59a0 0x55ef69dbfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.627+0000 7f83db7fe640 1 --2- 172.21.15.120:0/1326663162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69db62e0 0x55ef69dc0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.628+0000 7f83e0e81640 1 --2- 172.21.15.120:0/1326663162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef69dbb370 0x55ef69dc70c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f83d402ade0 tx=0x7f83d40095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e2c1780c8c7357c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.628+0000 7f83bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:05:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.628+0000 7f83bb7fe640 1 -- 172.21.15.120:0/1326663162 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d4052070 con 0x55ef69dbb370 2024-09-19T09:05:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.628+0000 7f83e5683740 1 -- 172.21.15.120:0/1326663162 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83cc0043c0 con 0x55ef69dbb370 2024-09-19T09:05:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.628+0000 7f83bb7fe640 1 -- 172.21.15.120:0/1326663162 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f83d4004240 con 0x55ef69dbb370 2024-09-19T09:05:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.628+0000 7f83bb7fe640 1 -- 172.21.15.120:0/1326663162 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f83d405e050 con 0x55ef69dbb370 2024-09-19T09:05:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.628+0000 7f83e5683740 10 client.25582 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.628+0000 7f83e5683740 1 -- 172.21.15.120:0/1326663162 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ef69af8620 con 0x55ef69dbb370 2024-09-19T09:05:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.628+0000 7f83bb7fe640 1 -- 172.21.15.120:0/1326663162 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83cc0043c0 con 0x55ef69dbb370 2024-09-19T09:05:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.628+0000 7f83bb7fe640 1 -- 172.21.15.120:0/1326663162 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f83d402fcb0 con 0x55ef69dbb370 2024-09-19T09:05:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.628+0000 7f83e5683740 10 client.25582 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.628+0000 7f83e5683740 10 client.25582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.628+0000 7f83e5683740 20 client.25582 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.628+0000 7f83e5683740 1 --2- 172.21.15.120:0/1326663162 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ef69dcb800 0x55ef69dcdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.628+0000 7f83e5683740 4 client.25582 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:05:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.628+0000 7f83e5683740 1 -- 172.21.15.120:0/1326663162 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef69d26a20 con 0x55ef69dcb800 2024-09-19T09:05:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.629+0000 7f83dbfff640 1 --2- 172.21.15.120:0/1326663162 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ef69dcb800 0x55ef69dcdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.629+0000 7f83dbfff640 1 --2- 172.21.15.120:0/1326663162 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ef69dcb800 0x55ef69dcdbf0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb9619d9e8a20c8 server_cookie=b06fd767bb90fc96 in_seq=0 out_seq=0 2024-09-19T09:05:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.629+0000 7f83bb7fe640 10 client.25582 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.631+0000 7f83bb7fe640 1 -- 172.21.15.120:0/1326663162 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ef69d26a20 con 0x55ef69dcb800 2024-09-19T09:05:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.631+0000 7f83bb7fe640 10 client.25582 handle_command_reply: tid=0 2024-09-19T09:05:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.631+0000 7f83e5683740 1 client.25582 shutdown 2024-09-19T09:05:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.631+0000 7f83e5683740 1 -- 172.21.15.120:0/1326663162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef69dbb370 msgr2=0x55ef69dc70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.631+0000 7f83e5683740 1 --2- 172.21.15.120:0/1326663162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef69dbb370 0x55ef69dc70c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f83d402ade0 tx=0x7f83d40095c0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.631+0000 7f83e5683740 1 -- 172.21.15.120:0/1326663162 shutdown_connections 2024-09-19T09:05:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.631+0000 7f83e5683740 1 --2- 172.21.15.120:0/1326663162 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ef69dcb800 0x55ef69dcdbf0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.631+0000 7f83e5683740 1 --2- 172.21.15.120:0/1326663162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef69dbb370 0x55ef69dc70c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.631+0000 7f83e5683740 1 --2- 172.21.15.120:0/1326663162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69db62e0 0x55ef69dc0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.631+0000 7f83bb7fe640 0 client.25582 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.631+0000 7f83e5683740 1 --2- 172.21.15.120:0/1326663162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef69db59a0 0x55ef69dbfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.631+0000 7f83e5683740 1 -- 172.21.15.120:0/1326663162 >> 172.21.15.120:0/1326663162 conn(0x55ef69db1010 msgr2=0x55ef69db2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.631+0000 7f83e5683740 1 -- 172.21.15.120:0/1326663162 shutdown_connections 2024-09-19T09:05:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.631+0000 7f83e5683740 1 -- 172.21.15.120:0/1326663162 wait complete. 2024-09-19T09:05:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.631+0000 7f83e5683740 20 client.25582 trim_cache size 0 max 16384 2024-09-19T09:05:27.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.647+0000 7f83e5683740 1 Processor -- start 2024-09-19T09:05:27.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.647+0000 7f83e5683740 1 -- start start 2024-09-19T09:05:27.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.647+0000 7f83e5683740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69db59a0 0x55ef69db1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.647+0000 7f83e5683740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef69db62e0 0x55ef69dc7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.647+0000 7f83e5683740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef69dbb370 0x55ef69db2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.647+0000 7f83e5683740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc1a6d00 con 0x55ef69db59a0 2024-09-19T09:05:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.647+0000 7f83e5683740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc1a8840 con 0x55ef69dbb370 2024-09-19T09:05:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.647+0000 7f83e5683740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc1a73d0 con 0x55ef69db62e0 2024-09-19T09:05:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.647+0000 7f83dbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69db59a0 0x55ef69db1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.647+0000 7f83dbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69db59a0 0x55ef69db1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36240/0 (socket says 172.21.15.120:36240) 2024-09-19T09:05:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.647+0000 7f83dbfff640 1 -- 172.21.15.120:0/834978268 learned_addr learned my addr 172.21.15.120:0/834978268 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.647+0000 7f83e0e81640 1 --2- 172.21.15.120:0/834978268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef69dbb370 0x55ef69db2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.647+0000 7f83db7fe640 1 --2- 172.21.15.120:0/834978268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef69db62e0 0x55ef69dc7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.647+0000 7f83dbfff640 1 -- 172.21.15.120:0/834978268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef69db62e0 msgr2=0x55ef69dc7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.647+0000 7f83dbfff640 1 --2- 172.21.15.120:0/834978268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef69db62e0 0x55ef69dc7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.647+0000 7f83dbfff640 1 -- 172.21.15.120:0/834978268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef69dbb370 msgr2=0x55ef69db2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.647+0000 7f83dbfff640 1 --2- 172.21.15.120:0/834978268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef69dbb370 0x55ef69db2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.647+0000 7f83dbfff640 1 -- 172.21.15.120:0/834978268 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef69db0e30 con 0x55ef69db59a0 2024-09-19T09:05:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.648+0000 7f83dbfff640 1 --2- 172.21.15.120:0/834978268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69db59a0 0x55ef69db1ec0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f83cc003da0 tx=0x7f83cc003800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e2269e6f261097 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.648+0000 7f83c1ffb640 1 -- 172.21.15.120:0/834978268 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc034070 con 0x55ef69db59a0 2024-09-19T09:05:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.648+0000 7f83c1ffb640 1 -- 172.21.15.120:0/834978268 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83cc005040 con 0x55ef69db59a0 2024-09-19T09:05:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.648+0000 7f83c1ffb640 1 -- 172.21.15.120:0/834978268 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc019430 con 0x55ef69db59a0 2024-09-19T09:05:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.648+0000 7f83e5683740 1 -- 172.21.15.120:0/834978268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69db59a0 msgr2=0x55ef69db1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.648+0000 7f83e5683740 1 --2- 172.21.15.120:0/834978268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69db59a0 0x55ef69db1ec0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f83cc003da0 tx=0x7f83cc003800 comp rx=0 tx=0).stop 2024-09-19T09:05:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.648+0000 7f83e5683740 1 -- 172.21.15.120:0/834978268 shutdown_connections 2024-09-19T09:05:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.649+0000 7f83e5683740 1 --2- 172.21.15.120:0/834978268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef69dbb370 0x55ef69db2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.649+0000 7f83e5683740 1 --2- 172.21.15.120:0/834978268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef69db62e0 0x55ef69dc7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.649+0000 7f83e5683740 1 --2- 172.21.15.120:0/834978268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69db59a0 0x55ef69db1ec0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.649+0000 7f83e5683740 1 -- 172.21.15.120:0/834978268 >> 172.21.15.120:0/834978268 conn(0x55ef69db1010 msgr2=0x55ef69dbfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.649+0000 7f83e5683740 1 -- 172.21.15.120:0/834978268 shutdown_connections 2024-09-19T09:05:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.649+0000 7f83e5683740 1 -- 172.21.15.120:0/834978268 wait complete. 2024-09-19T09:05:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.649+0000 7f83e5683740 1 -- start start 2024-09-19T09:05:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.649+0000 7f83e5683740 1 Processor -- start 2024-09-19T09:05:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.649+0000 7f83e5683740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69db59a0 0x55ef69dc4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.649+0000 7f83e5683740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef69db62e0 0x55ef69dc4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.649+0000 7f83e5683740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef69dbb370 0x55ef69dc5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.649+0000 7f83e5683740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc1a8170 con 0x55ef69db59a0 2024-09-19T09:05:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.649+0000 7f83e5683740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc1a7aa0 con 0x55ef69db62e0 2024-09-19T09:05:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.649+0000 7f83e5683740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc1a4cc0 con 0x55ef69dbb370 2024-09-19T09:05:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.649+0000 7f83e5683740 10 client.0 _ll_register_callbacks cb 0x7f83d4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.649+0000 7f83dbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69db59a0 0x55ef69dc4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.649+0000 7f83dbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69db59a0 0x55ef69dc4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36254/0 (socket says 172.21.15.120:36254) 2024-09-19T09:05:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.649+0000 7f83dbfff640 1 -- 172.21.15.120:0/744431052 learned_addr learned my addr 172.21.15.120:0/744431052 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.649+0000 7f83db7fe640 1 --2- 172.21.15.120:0/744431052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef69db62e0 0x55ef69dc4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.649+0000 7f83e0e81640 1 --2- 172.21.15.120:0/744431052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef69dbb370 0x55ef69dc5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.650+0000 7f83dbfff640 1 -- 172.21.15.120:0/744431052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef69dbb370 msgr2=0x55ef69dc5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.650+0000 7f83dbfff640 1 --2- 172.21.15.120:0/744431052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef69dbb370 0x55ef69dc5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.650+0000 7f83dbfff640 1 -- 172.21.15.120:0/744431052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef69db62e0 msgr2=0x55ef69dc4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.650+0000 7f83dbfff640 1 --2- 172.21.15.120:0/744431052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef69db62e0 0x55ef69dc4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.650+0000 7f83dbfff640 1 -- 172.21.15.120:0/744431052 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef69dc5f70 con 0x55ef69db59a0 2024-09-19T09:05:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.650+0000 7f83dbfff640 1 --2- 172.21.15.120:0/744431052 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69db59a0 0x55ef69dc4410 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f83cc009380 tx=0x7f83cc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d54a83c8c909ad8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.650+0000 7f83bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.650+0000 7f83bb7fe640 1 -- 172.21.15.120:0/744431052 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc03d060 con 0x55ef69db59a0 2024-09-19T09:05:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.650+0000 7f83e5683740 1 -- 172.21.15.120:0/744431052 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83cc005040 con 0x55ef69db59a0 2024-09-19T09:05:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.650+0000 7f83bb7fe640 1 -- 172.21.15.120:0/744431052 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f83cc02c050 con 0x55ef69db59a0 2024-09-19T09:05:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.650+0000 7f83bb7fe640 1 -- 172.21.15.120:0/744431052 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f83cc034030 con 0x55ef69db59a0 2024-09-19T09:05:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.650+0000 7f83e5683740 10 client.16251 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.650+0000 7f83e5683740 1 -- 172.21.15.120:0/744431052 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ef69dc7280 con 0x55ef69db59a0 2024-09-19T09:05:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.650+0000 7f83bb7fe640 1 -- 172.21.15.120:0/744431052 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83cc005040 con 0x55ef69db59a0 2024-09-19T09:05:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.650+0000 7f83bb7fe640 1 -- 172.21.15.120:0/744431052 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f83cc0197d0 con 0x55ef69db59a0 2024-09-19T09:05:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.651+0000 7f83e5683740 10 client.16251 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.651+0000 7f83e5683740 10 client.16251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.651+0000 7f83e5683740 20 client.16251 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.651+0000 7f83e5683740 1 --2- 172.21.15.120:0/744431052 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ef69e367f0 0x55ef69e38c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.651+0000 7f83e5683740 4 client.16251 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:05:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.651+0000 7f83e5683740 1 -- 172.21.15.120:0/744431052 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef69d20d70 con 0x55ef69e367f0 2024-09-19T09:05:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.651+0000 7f83db7fe640 1 --2- 172.21.15.120:0/744431052 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ef69e367f0 0x55ef69e38c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.651+0000 7f83db7fe640 1 --2- 172.21.15.120:0/744431052 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ef69e367f0 0x55ef69e38c10 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fe9a5e3a96eb6d4 server_cookie=46b363d63bf2acfa in_seq=0 out_seq=0 2024-09-19T09:05:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.651+0000 7f83bb7fe640 10 client.16251 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.772+0000 7f83bb7fe640 1 -- 172.21.15.120:0/744431052 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ef69d20d70 con 0x55ef69e367f0 2024-09-19T09:05:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.772+0000 7f83bb7fe640 10 client.16251 handle_command_reply: tid=0 2024-09-19T09:05:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.772+0000 7f83e5683740 1 client.16251 shutdown 2024-09-19T09:05:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.772+0000 7f83e5683740 1 -- 172.21.15.120:0/744431052 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69db59a0 msgr2=0x55ef69dc4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.772+0000 7f83e5683740 1 --2- 172.21.15.120:0/744431052 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69db59a0 0x55ef69dc4410 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f83cc009380 tx=0x7f83cc005f90 comp rx=0 tx=0).stop 2024-09-19T09:05:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.772+0000 7f83e5683740 1 -- 172.21.15.120:0/744431052 shutdown_connections 2024-09-19T09:05:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.772+0000 7f83e5683740 1 --2- 172.21.15.120:0/744431052 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ef69e367f0 0x55ef69e38c10 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.773+0000 7f83e5683740 1 --2- 172.21.15.120:0/744431052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef69dbb370 0x55ef69dc5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.773+0000 7f83e5683740 1 --2- 172.21.15.120:0/744431052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef69db62e0 0x55ef69dc4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.773+0000 7f83e5683740 1 --2- 172.21.15.120:0/744431052 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef69db59a0 0x55ef69dc4410 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.773+0000 7f83bb7fe640 0 client.16251 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.773+0000 7f83e5683740 1 -- 172.21.15.120:0/744431052 >> 172.21.15.120:0/744431052 conn(0x55ef69db1010 msgr2=0x55ef69daf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.773+0000 7f83e5683740 1 -- 172.21.15.120:0/744431052 shutdown_connections 2024-09-19T09:05:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.773+0000 7f83e5683740 1 -- 172.21.15.120:0/744431052 wait complete. 2024-09-19T09:05:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.773+0000 7f83e5683740 20 client.16251 trim_cache size 0 max 16384 2024-09-19T09:05:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:05:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:05:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:05:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:05:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.773+0000 7f83e28e4640 1 -- 172.21.15.120:0/1598112642 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83c40a32f0 msgr2=0x7f83c40a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.773+0000 7f83e28e4640 1 --2- 172.21.15.120:0/1598112642 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83c40a32f0 0x7f83c40a57b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f83d4005e30 tx=0x7f83d4031bb0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.773+0000 7f83e28e4640 1 -- 172.21.15.120:0/1598112642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83dc101c30 msgr2=0x7f83dc1960a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.773+0000 7f83e28e4640 1 --2- 172.21.15.120:0/1598112642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83dc101c30 0x7f83dc1960a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f83d0012fc0 tx=0x7f83d0010490 comp rx=0 tx=0).stop 2024-09-19T09:05:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.774+0000 7f83e28e4640 1 -- 172.21.15.120:0/1598112642 shutdown_connections 2024-09-19T09:05:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.774+0000 7f83e28e4640 1 --2- 172.21.15.120:0/1598112642 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83c40a32f0 0x7f83c40a57b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.774+0000 7f83e28e4640 1 --2- 172.21.15.120:0/1598112642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83dc1025f0 0x7f83dc19d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.774+0000 7f83e28e4640 1 --2- 172.21.15.120:0/1598112642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83dc101c30 0x7f83dc1960a0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.774+0000 7f83e28e4640 1 --2- 172.21.15.120:0/1598112642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83dc100a40 0x7f83dc195b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.774+0000 7f83e28e4640 1 -- 172.21.15.120:0/1598112642 >> 172.21.15.120:0/1598112642 conn(0x7f83dc0fc1e0 msgr2=0x7f83dc104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.774+0000 7f83e28e4640 1 -- 172.21.15.120:0/1598112642 shutdown_connections 2024-09-19T09:05:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:27.774+0000 7f83e28e4640 1 -- 172.21.15.120:0/1598112642 wait complete. 2024-09-19T09:05:27.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.837+0000 7fc70a3bc640 1 -- 172.21.15.120:0/1827999589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc704102ef0 msgr2=0x7fc70410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:27.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.837+0000 7fc70a3bc640 1 --2- 172.21.15.120:0/1827999589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc704102ef0 0x7fc70410da40 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc009ca0 tx=0x7fc6fc02a950 comp rx=0 tx=0).stop 2024-09-19T09:05:27.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.837+0000 7fc70a3bc640 1 -- 172.21.15.120:0/1827999589 shutdown_connections 2024-09-19T09:05:27.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.837+0000 7fc70a3bc640 1 --2- 172.21.15.120:0/1827999589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc704102ef0 0x7fc70410da40 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.837+0000 7fc70a3bc640 1 --2- 172.21.15.120:0/1827999589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc704102570 0x7fc7041029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.837+0000 7fc70a3bc640 1 --2- 172.21.15.120:0/1827999589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc704100e20 0x7fc704101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.837+0000 7fc70a3bc640 1 -- 172.21.15.120:0/1827999589 >> 172.21.15.120:0/1827999589 conn(0x7fc7040fc7d0 msgr2=0x7fc7040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:27.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.837+0000 7fc70a3bc640 1 -- 172.21.15.120:0/1827999589 shutdown_connections 2024-09-19T09:05:27.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.837+0000 7fc70a3bc640 1 -- 172.21.15.120:0/1827999589 wait complete. 2024-09-19T09:05:27.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.837+0000 7fc70a3bc640 1 Processor -- start 2024-09-19T09:05:27.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.837+0000 7fc70a3bc640 1 -- start start 2024-09-19T09:05:27.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.838+0000 7fc70a3bc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc704100e20 0x7fc704198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.838+0000 7fc70a3bc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc704102570 0x7fc7041989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.838+0000 7fc70a3bc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc704102ef0 0x7fc70419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.838+0000 7fc70a3bc640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc704110fd0 con 0x7fc704100e20 2024-09-19T09:05:27.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.838+0000 7fc70a3bc640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc704110e50 con 0x7fc704102ef0 2024-09-19T09:05:27.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.838+0000 7fc70a3bc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc704111150 con 0x7fc704102570 2024-09-19T09:05:27.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.838+0000 7fc703fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc704100e20 0x7fc704198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.838+0000 7fc703fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc704100e20 0x7fc704198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36278/0 (socket says 172.21.15.120:36278) 2024-09-19T09:05:27.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.838+0000 7fc703fff640 1 -- 172.21.15.120:0/2054231933 learned_addr learned my addr 172.21.15.120:0/2054231933 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:27.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.838+0000 7fc708959640 1 --2- 172.21.15.120:0/2054231933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc704102ef0 0x7fc70419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.838+0000 7fc7037fe640 1 --2- 172.21.15.120:0/2054231933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc704102570 0x7fc7041989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.838+0000 7fc703fff640 1 -- 172.21.15.120:0/2054231933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc704102570 msgr2=0x7fc7041989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:27.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.838+0000 7fc703fff640 1 --2- 172.21.15.120:0/2054231933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc704102570 0x7fc7041989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.838+0000 7fc703fff640 1 -- 172.21.15.120:0/2054231933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc704102ef0 msgr2=0x7fc70419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:27.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.838+0000 7fc703fff640 1 --2- 172.21.15.120:0/2054231933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc704102ef0 0x7fc70419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.838+0000 7fc703fff640 1 -- 172.21.15.120:0/2054231933 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70419d430 con 0x7fc704100e20 2024-09-19T09:05:27.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.838+0000 7fc708959640 1 --2- 172.21.15.120:0/2054231933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc704102ef0 0x7fc70419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:27.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.838+0000 7fc7037fe640 1 --2- 172.21.15.120:0/2054231933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc704102570 0x7fc7041989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:27.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.839+0000 7fc703fff640 1 --2- 172.21.15.120:0/2054231933 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc704100e20 0x7fc704198480 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc6f400e9f0 tx=0x7fc6f400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:27.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.839+0000 7fc7017fa640 1 -- 172.21.15.120:0/2054231933 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f400cd80 con 0x7fc704100e20 2024-09-19T09:05:27.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.839+0000 7fc7017fa640 1 -- 172.21.15.120:0/2054231933 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6f4005500 con 0x7fc704100e20 2024-09-19T09:05:27.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.839+0000 7fc70a3bc640 1 -- 172.21.15.120:0/2054231933 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc70419d720 con 0x7fc704100e20 2024-09-19T09:05:27.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.839+0000 7fc7017fa640 1 -- 172.21.15.120:0/2054231933 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4010690 con 0x7fc704100e20 2024-09-19T09:05:27.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.839+0000 7fc70a3bc640 1 -- 172.21.15.120:0/2054231933 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7041a5000 con 0x7fc704100e20 2024-09-19T09:05:27.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.841+0000 7fc7017fa640 1 -- 172.21.15.120:0/2054231933 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fc6f40046c0 con 0x7fc704100e20 2024-09-19T09:05:27.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.841+0000 7fc70c795740 1 Processor -- start 2024-09-19T09:05:27.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.842+0000 7fc70c795740 1 -- start start 2024-09-19T09:05:27.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.842+0000 7fc70c795740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f6b53b40 0x5615f6b52090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.842+0000 7fc70c795740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f6b525d0 0x5615f6b52a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.842+0000 7fc70c795740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615f6b5d370 0x5615f6b5f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.842+0000 7fc70c795740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5615f6ab82f0 con 0x5615f6b53b40 2024-09-19T09:05:27.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.842+0000 7fc70c795740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5615f6a5fcf0 con 0x5615f6b5d370 2024-09-19T09:05:27.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.842+0000 7fc70c795740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5615f6ab42b0 con 0x5615f6b525d0 2024-09-19T09:05:27.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.842+0000 7fc7017fa640 1 --2- 172.21.15.120:0/2054231933 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc6ec0a3240 0x7fc6ec0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.842+0000 7fc708959640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f6b53b40 0x5615f6b52090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.842+0000 7fc708959640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f6b53b40 0x5615f6b52090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36294/0 (socket says 172.21.15.120:36294) 2024-09-19T09:05:27.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.842+0000 7fc708959640 1 -- 172.21.15.120:0/1674816049 learned_addr learned my addr 172.21.15.120:0/1674816049 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:27.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.842+0000 7fc7037fe640 1 --2- 172.21.15.120:0/1674816049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615f6b5d370 0x5615f6b5f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.842+0000 7fc7017fa640 1 -- 172.21.15.120:0/2054231933 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fc6f40cfcf0 con 0x7fc704100e20 2024-09-19T09:05:27.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.842+0000 7fc703fff640 1 --2- 172.21.15.120:0/1674816049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f6b525d0 0x5615f6b52a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.842+0000 7fc708959640 1 -- 172.21.15.120:0/1674816049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f6b525d0 msgr2=0x5615f6b52a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:27.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.842+0000 7fc708959640 1 --2- 172.21.15.120:0/1674816049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f6b525d0 0x5615f6b52a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.842+0000 7fc708959640 1 -- 172.21.15.120:0/1674816049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615f6b5d370 msgr2=0x5615f6b5f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:27.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.842+0000 7fc708959640 1 --2- 172.21.15.120:0/1674816049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615f6b5d370 0x5615f6b5f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.842+0000 7fc708959640 1 -- 172.21.15.120:0/1674816049 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615f6a443a0 con 0x5615f6b53b40 2024-09-19T09:05:27.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.842+0000 7fc7037fe640 1 --2- 172.21.15.120:0/1674816049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615f6b5d370 0x5615f6b5f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:27.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.843+0000 7fc703fff640 1 --2- 172.21.15.120:0/1674816049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f6b525d0 0x5615f6b52a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:27.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.843+0000 7fc708959640 1 --2- 172.21.15.120:0/2054231933 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc6ec0a3240 0x7fc6ec0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.843+0000 7fc708959640 1 --2- 172.21.15.120:0/1674816049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f6b53b40 0x5615f6b52090 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc02ae70 tx=0x7fc6fc02f7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5959f706dda65a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:27.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.843+0000 7fc6ea7fc640 1 -- 172.21.15.120:0/1674816049 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc034040 con 0x5615f6b53b40 2024-09-19T09:05:27.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.843+0000 7fc6ea7fc640 1 -- 172.21.15.120:0/1674816049 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6fc02fe90 con 0x5615f6b53b40 2024-09-19T09:05:27.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.843+0000 7fc6ea7fc640 1 -- 172.21.15.120:0/1674816049 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc046750 con 0x5615f6b53b40 2024-09-19T09:05:27.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.844+0000 7fc70c795740 1 -- 172.21.15.120:0/1674816049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f6b53b40 msgr2=0x5615f6b52090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:27.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.844+0000 7fc70c795740 1 --2- 172.21.15.120:0/1674816049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f6b53b40 0x5615f6b52090 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc02ae70 tx=0x7fc6fc02f7e0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.844+0000 7fc708959640 1 --2- 172.21.15.120:0/2054231933 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc6ec0a3240 0x7fc6ec0a5700 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc04b000 tx=0x7fc6fc046a30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:27.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.844+0000 7fc70c795740 1 -- 172.21.15.120:0/1674816049 shutdown_connections 2024-09-19T09:05:27.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.844+0000 7fc70c795740 1 --2- 172.21.15.120:0/1674816049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615f6b5d370 0x5615f6b5f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.844+0000 7fc70c795740 1 --2- 172.21.15.120:0/1674816049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f6b525d0 0x5615f6b52a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.844+0000 7fc70c795740 1 --2- 172.21.15.120:0/1674816049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f6b53b40 0x5615f6b52090 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.844+0000 7fc70c795740 1 -- 172.21.15.120:0/1674816049 >> 172.21.15.120:0/1674816049 conn(0x5615f6b1b310 msgr2=0x5615f6b4ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:27.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.844+0000 7fc70c795740 1 -- 172.21.15.120:0/1674816049 shutdown_connections 2024-09-19T09:05:27.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.844+0000 7fc70c795740 1 -- 172.21.15.120:0/1674816049 wait complete. 2024-09-19T09:05:27.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.844+0000 7fc70c795740 1 -- start start 2024-09-19T09:05:27.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.845+0000 7fc70c795740 1 Processor -- start 2024-09-19T09:05:27.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.845+0000 7fc70c795740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f6b525d0 0x5615f6b5cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.845+0000 7fc70c795740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f6b53b40 0x5615f6b59fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.845+0000 7fc70c795740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615f6b5d370 0x5615f6b5a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.845+0000 7fc70c795740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5615f6b60010 con 0x5615f6b525d0 2024-09-19T09:05:27.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.845+0000 7fc70c795740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5615f6a73060 con 0x5615f6b5d370 2024-09-19T09:05:27.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.845+0000 7fc70c795740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5615f6b60190 con 0x5615f6b53b40 2024-09-19T09:05:27.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.845+0000 7fc70c795740 10 client.0 _ll_register_callbacks cb 0x5615f67c68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:27.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.845+0000 7fc7037fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f6b53b40 0x5615f6b59fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.845+0000 7fc708959640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615f6b5d370 0x5615f6b5a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.845+0000 7fc708959640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615f6b5d370 0x5615f6b5a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:48662/0 (socket says 172.21.15.120:48662) 2024-09-19T09:05:27.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.845+0000 7fc708959640 1 -- 172.21.15.120:0/1596832438 learned_addr learned my addr 172.21.15.120:0/1596832438 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:27.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.845+0000 7fc7037fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f6b53b40 0x5615f6b59fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:59834/0 (socket says 172.21.15.120:59834) 2024-09-19T09:05:27.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.845+0000 7fc703fff640 1 --2- 172.21.15.120:0/1596832438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f6b525d0 0x5615f6b5cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.846+0000 7fc708959640 1 -- 172.21.15.120:0/1596832438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f6b53b40 msgr2=0x5615f6b59fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:27.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.846+0000 7fc708959640 1 --2- 172.21.15.120:0/1596832438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f6b53b40 0x5615f6b59fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.846+0000 7fc708959640 1 -- 172.21.15.120:0/1596832438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f6b525d0 msgr2=0x5615f6b5cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:27.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.846+0000 7fc708959640 1 --2- 172.21.15.120:0/1596832438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f6b525d0 0x5615f6b5cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.846+0000 7fc708959640 1 -- 172.21.15.120:0/1596832438 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615f68293c0 con 0x5615f6b5d370 2024-09-19T09:05:27.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.846+0000 7fc7037fe640 1 --2- 172.21.15.120:0/1596832438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f6b53b40 0x5615f6b59fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:27.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.846+0000 7fc708959640 1 --2- 172.21.15.120:0/1596832438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615f6b5d370 0x5615f6b5a570 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc02af40 tx=0x7fc6fc03e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9db8167a4ab99c3c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:27.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.846+0000 7fc6e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:05:27.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.846+0000 7fc6e37fe640 1 -- 172.21.15.120:0/1596832438 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc061060 con 0x5615f6b5d370 2024-09-19T09:05:27.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.846+0000 7fc70c795740 1 -- 172.21.15.120:0/1596832438 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6fc02fe90 con 0x5615f6b5d370 2024-09-19T09:05:27.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.846+0000 7fc6e37fe640 1 -- 172.21.15.120:0/1596832438 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fc6fc004040 con 0x5615f6b5d370 2024-09-19T09:05:27.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.846+0000 7fc6e37fe640 1 -- 172.21.15.120:0/1596832438 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc6fc004240 con 0x5615f6b5d370 2024-09-19T09:05:27.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.846+0000 7fc70c795740 10 client.25625 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:27.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.846+0000 7fc70c795740 1 -- 172.21.15.120:0/1596832438 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5615f6b60010 con 0x5615f6b5d370 2024-09-19T09:05:27.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.847+0000 7fc6e37fe640 1 -- 172.21.15.120:0/1596832438 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6fc02fe90 con 0x5615f6b5d370 2024-09-19T09:05:27.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.847+0000 7fc6e37fe640 1 -- 172.21.15.120:0/1596832438 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc6fc066070 con 0x5615f6b5d370 2024-09-19T09:05:27.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.847+0000 7fc70c795740 10 client.25625 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:27.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.847+0000 7fc70c795740 10 client.25625 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:05:27.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.847+0000 7fc70c795740 20 client.25625 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:27.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.847+0000 7fc70c795740 1 --2- 172.21.15.120:0/1596832438 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5615f6b6cbe0 0x5615f6b6efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.847+0000 7fc70c795740 4 client.25625 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:05:27.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.847+0000 7fc70c795740 1 -- 172.21.15.120:0/1596832438 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615f6ac5860 con 0x5615f6b6cbe0 2024-09-19T09:05:27.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.847+0000 7fc703fff640 1 --2- 172.21.15.120:0/1596832438 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5615f6b6cbe0 0x5615f6b6efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.847+0000 7fc6e37fe640 10 client.25625 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:05:27.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.847+0000 7fc703fff640 1 --2- 172.21.15.120:0/1596832438 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5615f6b6cbe0 0x5615f6b6efd0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ce8fce314c18532e server_cookie=128486022ae21739 in_seq=0 out_seq=0 2024-09-19T09:05:27.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.848+0000 7fc6e37fe640 1 -- 172.21.15.120:0/1596832438 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5615f6ac5860 con 0x5615f6b6cbe0 2024-09-19T09:05:27.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.848+0000 7fc6e37fe640 10 client.25625 handle_command_reply: tid=0 2024-09-19T09:05:27.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.848+0000 7fc70c795740 1 client.25625 shutdown 2024-09-19T09:05:27.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.848+0000 7fc70c795740 1 -- 172.21.15.120:0/1596832438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615f6b5d370 msgr2=0x5615f6b5a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:27.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.848+0000 7fc70c795740 1 --2- 172.21.15.120:0/1596832438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615f6b5d370 0x5615f6b5a570 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc02af40 tx=0x7fc6fc03e000 comp rx=0 tx=0).stop 2024-09-19T09:05:27.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.848+0000 7fc70c795740 1 -- 172.21.15.120:0/1596832438 shutdown_connections 2024-09-19T09:05:27.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.848+0000 7fc70c795740 1 --2- 172.21.15.120:0/1596832438 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5615f6b6cbe0 0x5615f6b6efd0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.848+0000 7fc70c795740 1 --2- 172.21.15.120:0/1596832438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615f6b5d370 0x5615f6b5a570 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.848+0000 7fc70c795740 1 --2- 172.21.15.120:0/1596832438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f6b53b40 0x5615f6b59fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.848+0000 7fc70c795740 1 --2- 172.21.15.120:0/1596832438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f6b525d0 0x5615f6b5cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.848+0000 7fc70c795740 1 -- 172.21.15.120:0/1596832438 >> 172.21.15.120:0/1596832438 conn(0x5615f6b1b310 msgr2=0x5615f6b54960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:27.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.849+0000 7fc6e37fe640 0 client.25625 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:05:27.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.849+0000 7fc70c795740 1 -- 172.21.15.120:0/1596832438 shutdown_connections 2024-09-19T09:05:27.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.849+0000 7fc70c795740 1 -- 172.21.15.120:0/1596832438 wait complete. 2024-09-19T09:05:27.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.849+0000 7fc70c795740 20 client.25625 trim_cache size 0 max 16384 2024-09-19T09:05:27.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.862+0000 7fc70c795740 1 Processor -- start 2024-09-19T09:05:27.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.862+0000 7fc70c795740 1 -- start start 2024-09-19T09:05:27.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.862+0000 7fc70c795740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f6b525d0 0x5615f6b692b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.862+0000 7fc70c795740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f6b53b40 0x5615f6b747e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.862+0000 7fc70c795740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615f6b5d370 0x5615f6b74d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.862+0000 7fc70c795740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc7041ad5f0 con 0x5615f6b525d0 2024-09-19T09:05:27.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.862+0000 7fc70c795740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc7041af130 con 0x5615f6b5d370 2024-09-19T09:05:27.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.862+0000 7fc70c795740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc7041adcc0 con 0x5615f6b53b40 2024-09-19T09:05:27.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.862+0000 7fc703fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f6b525d0 0x5615f6b692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.863+0000 7fc703fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f6b525d0 0x5615f6b692b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36314/0 (socket says 172.21.15.120:36314) 2024-09-19T09:05:27.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.863+0000 7fc703fff640 1 -- 172.21.15.120:0/161457997 learned_addr learned my addr 172.21.15.120:0/161457997 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:27.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.863+0000 7fc7037fe640 1 --2- 172.21.15.120:0/161457997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f6b53b40 0x5615f6b747e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.863+0000 7fc708959640 1 --2- 172.21.15.120:0/161457997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615f6b5d370 0x5615f6b74d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.863+0000 7fc703fff640 1 -- 172.21.15.120:0/161457997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f6b53b40 msgr2=0x5615f6b747e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:27.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.863+0000 7fc703fff640 1 --2- 172.21.15.120:0/161457997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f6b53b40 0x5615f6b747e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.863+0000 7fc703fff640 1 -- 172.21.15.120:0/161457997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615f6b5d370 msgr2=0x5615f6b74d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:27.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.863+0000 7fc703fff640 1 --2- 172.21.15.120:0/161457997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615f6b5d370 0x5615f6b74d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.863+0000 7fc703fff640 1 -- 172.21.15.120:0/161457997 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615f6b5a7b0 con 0x5615f6b525d0 2024-09-19T09:05:27.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.863+0000 7fc7037fe640 1 --2- 172.21.15.120:0/161457997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f6b53b40 0x5615f6b747e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:27.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.863+0000 7fc703fff640 1 --2- 172.21.15.120:0/161457997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f6b525d0 0x5615f6b692b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc6f409c2a0 tx=0x7fc6f40bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e75087b112cbba1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:27.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.863+0000 7fc6e9ffb640 1 -- 172.21.15.120:0/161457997 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f40b5070 con 0x5615f6b525d0 2024-09-19T09:05:27.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.863+0000 7fc6e9ffb640 1 -- 172.21.15.120:0/161457997 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6f40af070 con 0x5615f6b525d0 2024-09-19T09:05:27.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.863+0000 7fc6e9ffb640 1 -- 172.21.15.120:0/161457997 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f409a440 con 0x5615f6b525d0 2024-09-19T09:05:27.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.864+0000 7fc70c795740 1 -- 172.21.15.120:0/161457997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f6b525d0 msgr2=0x5615f6b692b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:27.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.864+0000 7fc70c795740 1 --2- 172.21.15.120:0/161457997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f6b525d0 0x5615f6b692b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc6f409c2a0 tx=0x7fc6f40bc040 comp rx=0 tx=0).stop 2024-09-19T09:05:27.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.864+0000 7fc70c795740 1 -- 172.21.15.120:0/161457997 shutdown_connections 2024-09-19T09:05:27.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.864+0000 7fc70c795740 1 --2- 172.21.15.120:0/161457997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615f6b5d370 0x5615f6b74d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.864+0000 7fc70c795740 1 --2- 172.21.15.120:0/161457997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f6b53b40 0x5615f6b747e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.864+0000 7fc70c795740 1 --2- 172.21.15.120:0/161457997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f6b525d0 0x5615f6b692b0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.864+0000 7fc70c795740 1 -- 172.21.15.120:0/161457997 >> 172.21.15.120:0/161457997 conn(0x5615f6b1b310 msgr2=0x5615f6b6ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:27.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.864+0000 7fc70c795740 1 -- 172.21.15.120:0/161457997 shutdown_connections 2024-09-19T09:05:27.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.864+0000 7fc70c795740 1 -- 172.21.15.120:0/161457997 wait complete. 2024-09-19T09:05:27.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.864+0000 7fc70c795740 1 -- start start 2024-09-19T09:05:27.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.865+0000 7fc70c795740 1 Processor -- start 2024-09-19T09:05:27.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.865+0000 7fc70c795740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f6b525d0 0x5615f6b65750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.865+0000 7fc70c795740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f6b53b40 0x5615f6b65c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.865+0000 7fc70c795740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615f6b5d370 0x5615f6b60bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.865+0000 7fc70c795740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc7041aea60 con 0x5615f6b525d0 2024-09-19T09:05:27.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.865+0000 7fc70c795740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc7041ae390 con 0x5615f6b5d370 2024-09-19T09:05:27.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.865+0000 7fc70c795740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc7041ab5b0 con 0x5615f6b53b40 2024-09-19T09:05:27.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.865+0000 7fc70c795740 10 client.0 _ll_register_callbacks cb 0x7fc6f8009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:27.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.865+0000 7fc703fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f6b525d0 0x5615f6b65750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.865+0000 7fc708959640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615f6b5d370 0x5615f6b60bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.865+0000 7fc7037fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f6b53b40 0x5615f6b65c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.865+0000 7fc703fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f6b525d0 0x5615f6b65750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36326/0 (socket says 172.21.15.120:36326) 2024-09-19T09:05:27.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.865+0000 7fc703fff640 1 -- 172.21.15.120:0/2049253101 learned_addr learned my addr 172.21.15.120:0/2049253101 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:27.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.865+0000 7fc7037fe640 1 -- 172.21.15.120:0/2049253101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615f6b5d370 msgr2=0x5615f6b60bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:27.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.865+0000 7fc7037fe640 1 --2- 172.21.15.120:0/2049253101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615f6b5d370 0x5615f6b60bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.865+0000 7fc7037fe640 1 -- 172.21.15.120:0/2049253101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f6b525d0 msgr2=0x5615f6b65750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:27.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.865+0000 7fc7037fe640 1 --2- 172.21.15.120:0/2049253101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f6b525d0 0x5615f6b65750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.865+0000 7fc7037fe640 1 -- 172.21.15.120:0/2049253101 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615f6b61400 con 0x5615f6b53b40 2024-09-19T09:05:27.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.866+0000 7fc7037fe640 1 --2- 172.21.15.120:0/2049253101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f6b53b40 0x5615f6b65c90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x5615f6b5b580 tx=0x7fc6f800f850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d88d313b59051c51 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:27.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.866+0000 7fc6e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:05:27.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.866+0000 7fc6e37fe640 1 -- 172.21.15.120:0/2049253101 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8004250 con 0x5615f6b53b40 2024-09-19T09:05:27.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.866+0000 7fc6e37fe640 1 -- 172.21.15.120:0/2049253101 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fc6f80043f0 con 0x5615f6b53b40 2024-09-19T09:05:27.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.866+0000 7fc70c795740 1 -- 172.21.15.120:0/2049253101 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6f40af070 con 0x5615f6b53b40 2024-09-19T09:05:27.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.866+0000 7fc6e37fe640 1 -- 172.21.15.120:0/2049253101 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc6f80035f0 con 0x5615f6b53b40 2024-09-19T09:05:27.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.866+0000 7fc70c795740 10 client.25606 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:27.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.866+0000 7fc70c795740 1 -- 172.21.15.120:0/2049253101 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5615f6bd7390 con 0x5615f6b53b40 2024-09-19T09:05:27.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.866+0000 7fc6e37fe640 1 -- 172.21.15.120:0/2049253101 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6f40af070 con 0x5615f6b53b40 2024-09-19T09:05:27.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.866+0000 7fc6e37fe640 1 -- 172.21.15.120:0/2049253101 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc6f801d880 con 0x5615f6b53b40 2024-09-19T09:05:27.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.866+0000 7fc70c795740 10 client.25606 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:27.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.866+0000 7fc70c795740 10 client.25606 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:05:27.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.866+0000 7fc70c795740 20 client.25606 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:27.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.866+0000 7fc70c795740 1 --2- 172.21.15.120:0/2049253101 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5615f6bd7b60 0x5615f6bd9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:27.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.866+0000 7fc70c795740 4 client.25606 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:05:27.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.866+0000 7fc70c795740 1 -- 172.21.15.120:0/2049253101 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5615f6abfd70 con 0x5615f6bd7b60 2024-09-19T09:05:27.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.867+0000 7fc703fff640 1 --2- 172.21.15.120:0/2049253101 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5615f6bd7b60 0x5615f6bd9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:27.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.867+0000 7fc6e37fe640 10 client.25606 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:05:27.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.867+0000 7fc703fff640 1 --2- 172.21.15.120:0/2049253101 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5615f6bd7b60 0x5615f6bd9f80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1bed81329bc71ef server_cookie=8626bd1519f6bc38 in_seq=0 out_seq=0 2024-09-19T09:05:27.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.867+0000 7fc6e37fe640 1 -- 172.21.15.120:0/2049253101 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5615f6abfd70 con 0x5615f6bd7b60 2024-09-19T09:05:27.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.867+0000 7fc6e37fe640 10 client.25606 handle_command_reply: tid=0 2024-09-19T09:05:27.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.867+0000 7fc70c795740 1 client.25606 shutdown 2024-09-19T09:05:27.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.867+0000 7fc70c795740 1 -- 172.21.15.120:0/2049253101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f6b53b40 msgr2=0x5615f6b65c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:27.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.867+0000 7fc70c795740 1 --2- 172.21.15.120:0/2049253101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f6b53b40 0x5615f6b65c90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x5615f6b5b580 tx=0x7fc6f800f850 comp rx=0 tx=0).stop 2024-09-19T09:05:27.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.867+0000 7fc70c795740 1 -- 172.21.15.120:0/2049253101 shutdown_connections 2024-09-19T09:05:27.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.867+0000 7fc70c795740 1 --2- 172.21.15.120:0/2049253101 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5615f6bd7b60 0x5615f6bd9f80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.867+0000 7fc70c795740 1 --2- 172.21.15.120:0/2049253101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615f6b5d370 0x5615f6b60bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.867+0000 7fc70c795740 1 --2- 172.21.15.120:0/2049253101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f6b53b40 0x5615f6b65c90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.867+0000 7fc70c795740 1 --2- 172.21.15.120:0/2049253101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615f6b525d0 0x5615f6b65750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.867+0000 7fc6e37fe640 0 client.25606 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:05:27.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.867+0000 7fc70c795740 1 -- 172.21.15.120:0/2049253101 >> 172.21.15.120:0/2049253101 conn(0x5615f6b1b310 msgr2=0x5615f6b588a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:27.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.867+0000 7fc70c795740 1 -- 172.21.15.120:0/2049253101 shutdown_connections 2024-09-19T09:05:27.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.867+0000 7fc70c795740 1 -- 172.21.15.120:0/2049253101 wait complete. 2024-09-19T09:05:27.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.867+0000 7fc70c795740 20 client.25606 trim_cache size 0 max 16384 2024-09-19T09:05:27.877 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:05:27.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.868+0000 7fc70a3bc640 1 -- 172.21.15.120:0/2054231933 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc6ec0a3240 msgr2=0x7fc6ec0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:27.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.868+0000 7fc70a3bc640 1 --2- 172.21.15.120:0/2054231933 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc6ec0a3240 0x7fc6ec0a5700 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc04b000 tx=0x7fc6fc046a30 comp rx=0 tx=0).stop 2024-09-19T09:05:27.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.868+0000 7fc70a3bc640 1 -- 172.21.15.120:0/2054231933 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc704100e20 msgr2=0x7fc704198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:27.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.868+0000 7fc70a3bc640 1 --2- 172.21.15.120:0/2054231933 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc704100e20 0x7fc704198480 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc6f400e9f0 tx=0x7fc6f400edf0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.868+0000 7fc70a3bc640 1 -- 172.21.15.120:0/2054231933 shutdown_connections 2024-09-19T09:05:27.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.868+0000 7fc70a3bc640 1 --2- 172.21.15.120:0/2054231933 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc6ec0a3240 0x7fc6ec0a5700 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.868+0000 7fc70a3bc640 1 --2- 172.21.15.120:0/2054231933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc704102ef0 0x7fc70419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.868+0000 7fc70a3bc640 1 --2- 172.21.15.120:0/2054231933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc704102570 0x7fc7041989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.868+0000 7fc70a3bc640 1 --2- 172.21.15.120:0/2054231933 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc704100e20 0x7fc704198480 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:27.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.868+0000 7fc70a3bc640 1 -- 172.21.15.120:0/2054231933 >> 172.21.15.120:0/2054231933 conn(0x7fc7040fc7d0 msgr2=0x7fc70406a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:27.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.868+0000 7fc70a3bc640 1 -- 172.21.15.120:0/2054231933 shutdown_connections 2024-09-19T09:05:27.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:27.868+0000 7fc70a3bc640 1 -- 172.21.15.120:0/2054231933 wait complete. 2024-09-19T09:05:28.039 INFO:tasks.workunit.client.0.smithi120.stdout: 1 124413 8.31 MB/sec warmup 83 sec latency 52.282 ms 2024-09-19T09:05:28.539 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:05:28.796 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:05:29.038 INFO:tasks.workunit.client.0.smithi120.stdout: 1 127165 8.37 MB/sec warmup 84 sec latency 31.520 ms 2024-09-19T09:05:29.064 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:28 smithi120 ceph-mon[31150]: pgmap v419: 97 pgs: 97 active+clean; 46 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.7 MiB/s wr, 330 op/s 2024-09-19T09:05:29.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:28 smithi196 ceph-mon[35150]: pgmap v419: 97 pgs: 97 active+clean; 46 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.7 MiB/s wr, 330 op/s 2024-09-19T09:05:29.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:28 smithi153 ceph-mon[35071]: pgmap v419: 97 pgs: 97 active+clean; 46 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.7 MiB/s wr, 330 op/s 2024-09-19T09:05:29.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.295+0000 7f23c938d640 1 -- 172.21.15.120:0/2540391984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23c4102570 msgr2=0x7f23c41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:29.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.295+0000 7f23c938d640 1 --2- 172.21.15.120:0/2540391984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23c4102570 0x7f23c41029b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f23ac009900 tx=0x7f23ac02a480 comp rx=0 tx=0).stop 2024-09-19T09:05:29.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.295+0000 7f23c938d640 1 -- 172.21.15.120:0/2540391984 shutdown_connections 2024-09-19T09:05:29.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.295+0000 7f23c938d640 1 --2- 172.21.15.120:0/2540391984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23c4102ef0 0x7f23c410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.295+0000 7f23c938d640 1 --2- 172.21.15.120:0/2540391984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23c4102570 0x7f23c41029b0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.295+0000 7f23c938d640 1 --2- 172.21.15.120:0/2540391984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23c4100e20 0x7f23c4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.295+0000 7f23c938d640 1 -- 172.21.15.120:0/2540391984 >> 172.21.15.120:0/2540391984 conn(0x7f23c40fc7d0 msgr2=0x7f23c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:29.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.295+0000 7f23c938d640 1 -- 172.21.15.120:0/2540391984 shutdown_connections 2024-09-19T09:05:29.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.295+0000 7f23c938d640 1 -- 172.21.15.120:0/2540391984 wait complete. 2024-09-19T09:05:29.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.296+0000 7f23c938d640 1 Processor -- start 2024-09-19T09:05:29.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.296+0000 7f23c938d640 1 -- start start 2024-09-19T09:05:29.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.296+0000 7f23c938d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23c4100e20 0x7f23c4199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:29.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.296+0000 7f23c938d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23c4102570 0x7f23c4199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:29.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.296+0000 7f23c938d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23c4102ef0 0x7f23c41a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:29.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.296+0000 7f23c938d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f23c4110f80 con 0x7f23c4102570 2024-09-19T09:05:29.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.296+0000 7f23c938d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f23c4110e00 con 0x7f23c4100e20 2024-09-19T09:05:29.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.296+0000 7f23c938d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f23c4111100 con 0x7f23c4102ef0 2024-09-19T09:05:29.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.296+0000 7f23c2ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23c4100e20 0x7f23c4199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:29.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.296+0000 7f23c27fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23c4102570 0x7f23c4199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:29.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.296+0000 7f23c2ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23c4100e20 0x7f23c4199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:48690/0 (socket says 172.21.15.120:48690) 2024-09-19T09:05:29.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.296+0000 7f23c27fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23c4102570 0x7f23c4199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36350/0 (socket says 172.21.15.120:36350) 2024-09-19T09:05:29.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.296+0000 7f23c2ffd640 1 -- 172.21.15.120:0/559078370 learned_addr learned my addr 172.21.15.120:0/559078370 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:29.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.296+0000 7f23c37fe640 1 --2- 172.21.15.120:0/559078370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23c4102ef0 0x7f23c41a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:29.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.297+0000 7f23c2ffd640 1 -- 172.21.15.120:0/559078370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23c4102ef0 msgr2=0x7f23c41a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:29.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.297+0000 7f23c2ffd640 1 --2- 172.21.15.120:0/559078370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23c4102ef0 0x7f23c41a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.297+0000 7f23c2ffd640 1 -- 172.21.15.120:0/559078370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23c4102570 msgr2=0x7f23c4199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:29.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.297+0000 7f23c2ffd640 1 --2- 172.21.15.120:0/559078370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23c4102570 0x7f23c4199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.297+0000 7f23c2ffd640 1 -- 172.21.15.120:0/559078370 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c41a48d0 con 0x7f23c4100e20 2024-09-19T09:05:29.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.297+0000 7f23c27fc640 1 --2- 172.21.15.120:0/559078370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23c4102570 0x7f23c4199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:29.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.297+0000 7f23c37fe640 1 --2- 172.21.15.120:0/559078370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23c4102ef0 0x7f23c41a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:29.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.297+0000 7f23c2ffd640 1 --2- 172.21.15.120:0/559078370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23c4100e20 0x7f23c4199310 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f23b400e9f0 tx=0x7f23b400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:29.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.297+0000 7f23bbfff640 1 -- 172.21.15.120:0/559078370 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b400cd80 con 0x7f23c4100e20 2024-09-19T09:05:29.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.297+0000 7f23c938d640 1 -- 172.21.15.120:0/559078370 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c41a4aa0 con 0x7f23c4100e20 2024-09-19T09:05:29.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.297+0000 7f23bbfff640 1 -- 172.21.15.120:0/559078370 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23b4010910 con 0x7f23c4100e20 2024-09-19T09:05:29.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.297+0000 7f23bbfff640 1 -- 172.21.15.120:0/559078370 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b4018990 con 0x7f23c4100e20 2024-09-19T09:05:29.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.297+0000 7f23c938d640 1 -- 172.21.15.120:0/559078370 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c41a4f60 con 0x7f23c4100e20 2024-09-19T09:05:29.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.300+0000 7f23cb766740 1 Processor -- start 2024-09-19T09:05:29.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.300+0000 7f23bbfff640 1 -- 172.21.15.120:0/559078370 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f23b4010ac0 con 0x7f23c4100e20 2024-09-19T09:05:29.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.300+0000 7f23cb766740 1 -- start start 2024-09-19T09:05:29.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.301+0000 7f23cb766740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642578a1b40 0x5642578a0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:29.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.301+0000 7f23cb766740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642578a05d0 0x5642578a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:29.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.301+0000 7f23cb766740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642578af780 0x5642578b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:29.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.301+0000 7f23cb766740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5642578062f0 con 0x5642578a1b40 2024-09-19T09:05:29.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.301+0000 7f23cb766740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5642577adcf0 con 0x5642578af780 2024-09-19T09:05:29.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.301+0000 7f23cb766740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5642578022b0 con 0x5642578a05d0 2024-09-19T09:05:29.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.301+0000 7f23c2ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642578a05d0 0x5642578a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:29.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.301+0000 7f23c37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642578a1b40 0x5642578a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:29.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.301+0000 7f23c37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642578a1b40 0x5642578a0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36366/0 (socket says 172.21.15.120:36366) 2024-09-19T09:05:29.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.301+0000 7f23c37fe640 1 -- 172.21.15.120:0/3688664167 learned_addr learned my addr 172.21.15.120:0/3688664167 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:29.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.301+0000 7f23c27fc640 1 --2- 172.21.15.120:0/3688664167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642578af780 0x5642578b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:29.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.301+0000 7f23c37fe640 1 -- 172.21.15.120:0/3688664167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642578a05d0 msgr2=0x5642578a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:29.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.301+0000 7f23c37fe640 1 --2- 172.21.15.120:0/3688664167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642578a05d0 0x5642578a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.301+0000 7f23c37fe640 1 -- 172.21.15.120:0/3688664167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642578af780 msgr2=0x5642578b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:29.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.301+0000 7f23c37fe640 1 --2- 172.21.15.120:0/3688664167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642578af780 0x5642578b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.301+0000 7f23c37fe640 1 -- 172.21.15.120:0/3688664167 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642577923a0 con 0x5642578a1b40 2024-09-19T09:05:29.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.301+0000 7f23c27fc640 1 --2- 172.21.15.120:0/3688664167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642578af780 0x5642578b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:29.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.301+0000 7f23bbfff640 1 --2- 172.21.15.120:0/559078370 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f23b00a3240 0x7f23b00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:29.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.301+0000 7f23c37fe640 1 --2- 172.21.15.120:0/3688664167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642578a1b40 0x5642578a0090 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f23bc011790 tx=0x7f23bc011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d80349d61fd9701 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:29.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.301+0000 7f23c2ffd640 1 --2- 172.21.15.120:0/559078370 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f23b00a3240 0x7f23b00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:29.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.302+0000 7f23bbfff640 1 -- 172.21.15.120:0/559078370 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f23b40cfc00 con 0x7f23c4100e20 2024-09-19T09:05:29.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.302+0000 7f23b97fa640 1 -- 172.21.15.120:0/3688664167 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc003c30 con 0x5642578a1b40 2024-09-19T09:05:29.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.302+0000 7f23b97fa640 1 -- 172.21.15.120:0/3688664167 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23bc00fe80 con 0x5642578a1b40 2024-09-19T09:05:29.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.302+0000 7f23b97fa640 1 -- 172.21.15.120:0/3688664167 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc0193e0 con 0x5642578a1b40 2024-09-19T09:05:29.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.302+0000 7f23cb766740 1 -- 172.21.15.120:0/3688664167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642578a1b40 msgr2=0x5642578a0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:29.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.302+0000 7f23c2ffd640 1 --2- 172.21.15.120:0/559078370 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f23b00a3240 0x7f23b00a5700 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f23b4098ca0 tx=0x7f23b40d3680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:29.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.302+0000 7f23cb766740 1 --2- 172.21.15.120:0/3688664167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642578a1b40 0x5642578a0090 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f23bc011790 tx=0x7f23bc011c50 comp rx=0 tx=0).stop 2024-09-19T09:05:29.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.303+0000 7f23cb766740 1 -- 172.21.15.120:0/3688664167 shutdown_connections 2024-09-19T09:05:29.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.303+0000 7f23cb766740 1 --2- 172.21.15.120:0/3688664167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642578af780 0x5642578b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.303+0000 7f23cb766740 1 --2- 172.21.15.120:0/3688664167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642578a05d0 0x5642578a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.303+0000 7f23cb766740 1 --2- 172.21.15.120:0/3688664167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642578a1b40 0x5642578a0090 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.303+0000 7f23cb766740 1 -- 172.21.15.120:0/3688664167 >> 172.21.15.120:0/3688664167 conn(0x564257869310 msgr2=0x56425789cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:29.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.303+0000 7f23cb766740 1 -- 172.21.15.120:0/3688664167 shutdown_connections 2024-09-19T09:05:29.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.303+0000 7f23cb766740 1 -- 172.21.15.120:0/3688664167 wait complete. 2024-09-19T09:05:29.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.303+0000 7f23cb766740 1 -- start start 2024-09-19T09:05:29.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.303+0000 7f23cb766740 1 Processor -- start 2024-09-19T09:05:29.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.303+0000 7f23cb766740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642578a05d0 0x5642578af0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:29.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.303+0000 7f23cb766740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642578a1b40 0x5642578ac3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:29.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.303+0000 7f23cb766740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642578af780 0x5642578ac900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:29.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.303+0000 7f23cb766740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5642578b2480 con 0x5642578a05d0 2024-09-19T09:05:29.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.303+0000 7f23cb766740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5642577c1060 con 0x5642578a1b40 2024-09-19T09:05:29.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.303+0000 7f23cb766740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5642578b2600 con 0x5642578af780 2024-09-19T09:05:29.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.303+0000 7f23cb766740 10 client.0 _ll_register_callbacks cb 0x5642575148c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:29.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.304+0000 7f23c37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642578a05d0 0x5642578af0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:29.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.304+0000 7f23c37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642578a05d0 0x5642578af0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36378/0 (socket says 172.21.15.120:36378) 2024-09-19T09:05:29.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.304+0000 7f23c37fe640 1 -- 172.21.15.120:0/184837344 learned_addr learned my addr 172.21.15.120:0/184837344 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:29.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.304+0000 7f23c27fc640 1 --2- 172.21.15.120:0/184837344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642578a1b40 0x5642578ac3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:29.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.304+0000 7f23c37fe640 1 --2- 172.21.15.120:0/184837344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642578af780 0x5642578ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:29.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.304+0000 7f23c37fe640 1 -- 172.21.15.120:0/184837344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642578af780 msgr2=0x5642578ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:29.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.304+0000 7f23c37fe640 1 --2- 172.21.15.120:0/184837344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642578af780 0x5642578ac900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.304+0000 7f23c37fe640 1 -- 172.21.15.120:0/184837344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642578a1b40 msgr2=0x5642578ac3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:29.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.304+0000 7f23c37fe640 1 --2- 172.21.15.120:0/184837344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642578a1b40 0x5642578ac3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.304+0000 7f23c37fe640 1 -- 172.21.15.120:0/184837344 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642575773c0 con 0x5642578a05d0 2024-09-19T09:05:29.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.304+0000 7f23c27fc640 1 --2- 172.21.15.120:0/184837344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642578a1b40 0x5642578ac3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:29.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.304+0000 7f23c37fe640 1 --2- 172.21.15.120:0/184837344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642578a05d0 0x5642578af0f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f23bc005f40 tx=0x7f23bc005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f608b43192d44db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:29.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.304+0000 7f23aa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:29.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.304+0000 7f23aa7fc640 1 -- 172.21.15.120:0/184837344 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc02e060 con 0x5642578a05d0 2024-09-19T09:05:29.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.304+0000 7f23aa7fc640 1 -- 172.21.15.120:0/184837344 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f23bc011240 con 0x5642578a05d0 2024-09-19T09:05:29.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.304+0000 7f23cb766740 1 -- 172.21.15.120:0/184837344 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564257813860 con 0x5642578a05d0 2024-09-19T09:05:29.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.304+0000 7f23aa7fc640 1 -- 172.21.15.120:0/184837344 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f23bc01f040 con 0x5642578a05d0 2024-09-19T09:05:29.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.304+0000 7f23cb766740 10 client.16281 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:29.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.304+0000 7f23cb766740 1 -- 172.21.15.120:0/184837344 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5642578bde90 con 0x5642578a05d0 2024-09-19T09:05:29.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.304+0000 7f23aa7fc640 1 -- 172.21.15.120:0/184837344 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564257813860 con 0x5642578a05d0 2024-09-19T09:05:29.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.305+0000 7f23aa7fc640 1 -- 172.21.15.120:0/184837344 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f23bc033070 con 0x5642578a05d0 2024-09-19T09:05:29.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.305+0000 7f23cb766740 10 client.16281 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:29.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.305+0000 7f23cb766740 10 client.16281 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:05:29.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.305+0000 7f23cb766740 20 client.16281 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:29.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.305+0000 7f23cb766740 1 --2- 172.21.15.120:0/184837344 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5642578bf2f0 0x5642578c16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:29.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.305+0000 7f23cb766740 4 client.16281 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:05:29.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.305+0000 7f23cb766740 1 -- 172.21.15.120:0/184837344 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56425780dd70 con 0x5642578bf2f0 2024-09-19T09:05:29.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.305+0000 7f23c27fc640 1 --2- 172.21.15.120:0/184837344 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5642578bf2f0 0x5642578c16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:29.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.305+0000 7f23c27fc640 1 --2- 172.21.15.120:0/184837344 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5642578bf2f0 0x5642578c16e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=32535ac0b3897410 server_cookie=afed5262cbc53b44 in_seq=0 out_seq=0 2024-09-19T09:05:29.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.305+0000 7f23aa7fc640 10 client.16281 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:05:29.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.306+0000 7f23aa7fc640 1 -- 172.21.15.120:0/184837344 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56425780dd70 con 0x5642578bf2f0 2024-09-19T09:05:29.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.306+0000 7f23aa7fc640 10 client.16281 handle_command_reply: tid=0 2024-09-19T09:05:29.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.306+0000 7f23cb766740 1 client.16281 shutdown 2024-09-19T09:05:29.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.306+0000 7f23cb766740 1 -- 172.21.15.120:0/184837344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642578a05d0 msgr2=0x5642578af0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:29.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.306+0000 7f23cb766740 1 --2- 172.21.15.120:0/184837344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642578a05d0 0x5642578af0f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f23bc005f40 tx=0x7f23bc005a50 comp rx=0 tx=0).stop 2024-09-19T09:05:29.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.306+0000 7f23cb766740 1 -- 172.21.15.120:0/184837344 shutdown_connections 2024-09-19T09:05:29.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.306+0000 7f23cb766740 1 --2- 172.21.15.120:0/184837344 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5642578bf2f0 0x5642578c16e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.306+0000 7f23cb766740 1 --2- 172.21.15.120:0/184837344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642578af780 0x5642578ac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.306+0000 7f23cb766740 1 --2- 172.21.15.120:0/184837344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642578a1b40 0x5642578ac3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.306+0000 7f23cb766740 1 --2- 172.21.15.120:0/184837344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642578a05d0 0x5642578af0f0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.306+0000 7f23cb766740 1 -- 172.21.15.120:0/184837344 >> 172.21.15.120:0/184837344 conn(0x564257869310 msgr2=0x5642578a2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:29.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.306+0000 7f23aa7fc640 0 client.16281 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:05:29.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.306+0000 7f23cb766740 1 -- 172.21.15.120:0/184837344 shutdown_connections 2024-09-19T09:05:29.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.306+0000 7f23cb766740 1 -- 172.21.15.120:0/184837344 wait complete. 2024-09-19T09:05:29.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.306+0000 7f23cb766740 20 client.16281 trim_cache size 0 max 16384 2024-09-19T09:05:29.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.319+0000 7f23cb766740 1 Processor -- start 2024-09-19T09:05:29.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.319+0000 7f23cb766740 1 -- start start 2024-09-19T09:05:29.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.320+0000 7f23cb766740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642578a05d0 0x5642578bb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:29.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.320+0000 7f23cb766740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642578a1b40 0x5642578c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:29.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.320+0000 7f23cb766740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642578af780 0x5642578c7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:29.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.320+0000 7f23cb766740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f23c41ab3c0 con 0x5642578a05d0 2024-09-19T09:05:29.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.320+0000 7f23cb766740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f23c41acf00 con 0x5642578a1b40 2024-09-19T09:05:29.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.320+0000 7f23cb766740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f23c41aba90 con 0x5642578af780 2024-09-19T09:05:29.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.320+0000 7f23c37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642578a05d0 0x5642578bb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:29.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.320+0000 7f23c27fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642578a1b40 0x5642578c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:29.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.320+0000 7f23c37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642578a05d0 0x5642578bb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36380/0 (socket says 172.21.15.120:36380) 2024-09-19T09:05:29.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.320+0000 7f23c37fe640 1 -- 172.21.15.120:0/2386259928 learned_addr learned my addr 172.21.15.120:0/2386259928 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:29.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.320+0000 7f23c37fe640 1 --2- 172.21.15.120:0/2386259928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642578af780 0x5642578c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:29.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.320+0000 7f23c27fc640 1 -- 172.21.15.120:0/2386259928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642578af780 msgr2=0x5642578c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:29.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.320+0000 7f23c27fc640 1 --2- 172.21.15.120:0/2386259928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642578af780 0x5642578c7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.320+0000 7f23c27fc640 1 -- 172.21.15.120:0/2386259928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642578a05d0 msgr2=0x5642578bb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:29.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.320+0000 7f23c27fc640 1 --2- 172.21.15.120:0/2386259928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642578a05d0 0x5642578bb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.320+0000 7f23c27fc640 1 -- 172.21.15.120:0/2386259928 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642578a3120 con 0x5642578a1b40 2024-09-19T09:05:29.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.320+0000 7f23c37fe640 1 --2- 172.21.15.120:0/2386259928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642578a05d0 0x5642578bb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:29.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.320+0000 7f23c27fc640 1 --2- 172.21.15.120:0/2386259928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642578a1b40 0x5642578c6dd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f23ac005940 tx=0x7f23ac0058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f266783c8132bd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:29.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.321+0000 7f23b8ff9640 1 -- 172.21.15.120:0/2386259928 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac04c030 con 0x5642578a1b40 2024-09-19T09:05:29.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.321+0000 7f23b8ff9640 1 -- 172.21.15.120:0/2386259928 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23ac005500 con 0x5642578a1b40 2024-09-19T09:05:29.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.321+0000 7f23b8ff9640 1 -- 172.21.15.120:0/2386259928 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac035040 con 0x5642578a1b40 2024-09-19T09:05:29.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.321+0000 7f23cb766740 1 -- 172.21.15.120:0/2386259928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642578a1b40 msgr2=0x5642578c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:29.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.321+0000 7f23cb766740 1 --2- 172.21.15.120:0/2386259928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642578a1b40 0x5642578c6dd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f23ac005940 tx=0x7f23ac0058d0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.322+0000 7f23cb766740 1 -- 172.21.15.120:0/2386259928 shutdown_connections 2024-09-19T09:05:29.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.322+0000 7f23cb766740 1 --2- 172.21.15.120:0/2386259928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642578af780 0x5642578c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.322+0000 7f23cb766740 1 --2- 172.21.15.120:0/2386259928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642578a1b40 0x5642578c6dd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.322+0000 7f23cb766740 1 --2- 172.21.15.120:0/2386259928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642578a05d0 0x5642578bb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.322+0000 7f23cb766740 1 -- 172.21.15.120:0/2386259928 >> 172.21.15.120:0/2386259928 conn(0x564257869310 msgr2=0x56425789bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:29.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.322+0000 7f23cb766740 1 -- 172.21.15.120:0/2386259928 shutdown_connections 2024-09-19T09:05:29.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.322+0000 7f23cb766740 1 -- 172.21.15.120:0/2386259928 wait complete. 2024-09-19T09:05:29.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.322+0000 7f23cb766740 1 -- start start 2024-09-19T09:05:29.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.322+0000 7f23cb766740 1 Processor -- start 2024-09-19T09:05:29.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.322+0000 7f23cb766740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642578a05d0 0x5642578b7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:29.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.322+0000 7f23cb766740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642578a1b40 0x5642578b7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:29.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.322+0000 7f23cb766740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642578af780 0x5642578a8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:29.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.322+0000 7f23cb766740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f23c41ac830 con 0x5642578a05d0 2024-09-19T09:05:29.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.322+0000 7f23cb766740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f23c41ac160 con 0x5642578af780 2024-09-19T09:05:29.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.322+0000 7f23cb766740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f23c41a9380 con 0x5642578a1b40 2024-09-19T09:05:29.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.322+0000 7f23cb766740 10 client.0 _ll_register_callbacks cb 0x7f23bc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:29.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.322+0000 7f23c37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642578a05d0 0x5642578b7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:29.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.322+0000 7f23c37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642578a05d0 0x5642578b7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36384/0 (socket says 172.21.15.120:36384) 2024-09-19T09:05:29.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.322+0000 7f23c37fe640 1 -- 172.21.15.120:0/486031700 learned_addr learned my addr 172.21.15.120:0/486031700 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:29.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.322+0000 7f23c27fc640 1 --2- 172.21.15.120:0/486031700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642578a1b40 0x5642578b7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:29.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.323+0000 7f23c37fe640 1 --2- 172.21.15.120:0/486031700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642578af780 0x5642578a8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:29.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.323+0000 7f23c37fe640 1 -- 172.21.15.120:0/486031700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642578a1b40 msgr2=0x5642578b7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:29.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.323+0000 7f23c37fe640 1 --2- 172.21.15.120:0/486031700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642578a1b40 0x5642578b7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.323+0000 7f23c37fe640 1 -- 172.21.15.120:0/486031700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642578af780 msgr2=0x5642578a8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:29.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.323+0000 7f23c37fe640 1 --2- 172.21.15.120:0/486031700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642578af780 0x5642578a8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.323+0000 7f23c37fe640 1 -- 172.21.15.120:0/486031700 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642578b84c0 con 0x5642578a05d0 2024-09-19T09:05:29.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.323+0000 7f23c37fe640 1 --2- 172.21.15.120:0/486031700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642578a05d0 0x5642578b7890 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f23bc006010 tx=0x7f23bc005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7772da684d22f02 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:29.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.323+0000 7f23aa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:29.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.323+0000 7f23aa7fc640 1 -- 172.21.15.120:0/486031700 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc02e050 con 0x5642578a05d0 2024-09-19T09:05:29.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.323+0000 7f23aa7fc640 1 -- 172.21.15.120:0/486031700 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f23bc004c00 con 0x5642578a05d0 2024-09-19T09:05:29.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.323+0000 7f23cb766740 1 -- 172.21.15.120:0/486031700 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23ac005500 con 0x5642578a05d0 2024-09-19T09:05:29.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.323+0000 7f23aa7fc640 1 -- 172.21.15.120:0/486031700 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f23bc039040 con 0x5642578a05d0 2024-09-19T09:05:29.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.323+0000 7f23cb766740 10 client.16287 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:29.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.323+0000 7f23cb766740 1 -- 172.21.15.120:0/486031700 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564257929330 con 0x5642578a05d0 2024-09-19T09:05:29.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.323+0000 7f23aa7fc640 1 -- 172.21.15.120:0/486031700 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23ac005500 con 0x5642578a05d0 2024-09-19T09:05:29.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.324+0000 7f23aa7fc640 1 -- 172.21.15.120:0/486031700 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f23bc033040 con 0x5642578a05d0 2024-09-19T09:05:29.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.324+0000 7f23cb766740 10 client.16287 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:29.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.324+0000 7f23cb766740 10 client.16287 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:05:29.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.324+0000 7f23cb766740 20 client.16287 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:29.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.324+0000 7f23cb766740 1 --2- 172.21.15.120:0/486031700 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5642579299a0 0x56425792bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:29.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.324+0000 7f23cb766740 4 client.16287 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:05:29.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.324+0000 7f23cb766740 1 -- 172.21.15.120:0/486031700 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56425778a170 con 0x5642579299a0 2024-09-19T09:05:29.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.324+0000 7f23c27fc640 1 --2- 172.21.15.120:0/486031700 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5642579299a0 0x56425792bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:29.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.325+0000 7f23c27fc640 1 --2- 172.21.15.120:0/486031700 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5642579299a0 0x56425792bdc0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7c337708bccbcb3f server_cookie=20177df94c417931 in_seq=0 out_seq=0 2024-09-19T09:05:29.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.325+0000 7f23aa7fc640 10 client.16287 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:05:29.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.325+0000 7f23aa7fc640 1 -- 172.21.15.120:0/486031700 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56425778a170 con 0x5642579299a0 2024-09-19T09:05:29.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.325+0000 7f23aa7fc640 10 client.16287 handle_command_reply: tid=0 2024-09-19T09:05:29.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.325+0000 7f23cb766740 1 client.16287 shutdown 2024-09-19T09:05:29.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.325+0000 7f23cb766740 1 -- 172.21.15.120:0/486031700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642578a05d0 msgr2=0x5642578b7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:29.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.325+0000 7f23cb766740 1 --2- 172.21.15.120:0/486031700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642578a05d0 0x5642578b7890 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f23bc006010 tx=0x7f23bc005cf0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.325+0000 7f23cb766740 1 -- 172.21.15.120:0/486031700 shutdown_connections 2024-09-19T09:05:29.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.325+0000 7f23cb766740 1 --2- 172.21.15.120:0/486031700 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5642579299a0 0x56425792bdc0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.325+0000 7f23cb766740 1 --2- 172.21.15.120:0/486031700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642578af780 0x5642578a8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.325+0000 7f23cb766740 1 --2- 172.21.15.120:0/486031700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642578a1b40 0x5642578b7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.325+0000 7f23cb766740 1 --2- 172.21.15.120:0/486031700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642578a05d0 0x5642578b7890 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.325+0000 7f23cb766740 1 -- 172.21.15.120:0/486031700 >> 172.21.15.120:0/486031700 conn(0x564257869310 msgr2=0x5642578aab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:29.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.325+0000 7f23aa7fc640 0 client.16287 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:05:29.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.325+0000 7f23cb766740 1 -- 172.21.15.120:0/486031700 shutdown_connections 2024-09-19T09:05:29.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.325+0000 7f23cb766740 1 -- 172.21.15.120:0/486031700 wait complete. 2024-09-19T09:05:29.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.325+0000 7f23cb766740 20 client.16287 trim_cache size 0 max 16384 2024-09-19T09:05:29.343 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:05:29.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.326+0000 7f23c938d640 1 -- 172.21.15.120:0/559078370 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f23b00a3240 msgr2=0x7f23b00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:29.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.326+0000 7f23c938d640 1 --2- 172.21.15.120:0/559078370 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f23b00a3240 0x7f23b00a5700 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f23b4098ca0 tx=0x7f23b40d3680 comp rx=0 tx=0).stop 2024-09-19T09:05:29.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.326+0000 7f23c938d640 1 -- 172.21.15.120:0/559078370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23c4100e20 msgr2=0x7f23c4199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:29.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.326+0000 7f23c938d640 1 --2- 172.21.15.120:0/559078370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23c4100e20 0x7f23c4199310 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f23b400e9f0 tx=0x7f23b400eeb0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.326+0000 7f23c938d640 1 -- 172.21.15.120:0/559078370 shutdown_connections 2024-09-19T09:05:29.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.326+0000 7f23c938d640 1 --2- 172.21.15.120:0/559078370 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f23b00a3240 0x7f23b00a5700 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.326+0000 7f23c938d640 1 --2- 172.21.15.120:0/559078370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23c4102ef0 0x7f23c41a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.326+0000 7f23c938d640 1 --2- 172.21.15.120:0/559078370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23c4102570 0x7f23c4199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.326+0000 7f23c938d640 1 --2- 172.21.15.120:0/559078370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23c4100e20 0x7f23c4199310 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:29.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.326+0000 7f23c938d640 1 -- 172.21.15.120:0/559078370 >> 172.21.15.120:0/559078370 conn(0x7f23c40fc7d0 msgr2=0x7f23c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:29.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.326+0000 7f23c938d640 1 -- 172.21.15.120:0/559078370 shutdown_connections 2024-09-19T09:05:29.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:29.327+0000 7f23c938d640 1 -- 172.21.15.120:0/559078370 wait complete. 2024-09-19T09:05:30.039 INFO:tasks.workunit.client.0.smithi120.stdout: 1 129931 8.41 MB/sec warmup 85 sec latency 17.583 ms 2024-09-19T09:05:30.368 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:05:30.626 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:05:31.020 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:30 smithi120 ceph-mon[31150]: pgmap v420: 97 pgs: 97 active+clean; 48 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.8 MiB/s wr, 340 op/s 2024-09-19T09:05:31.039 INFO:tasks.workunit.client.0.smithi120.stdout: 1 132256 8.48 MB/sec warmup 86 sec latency 37.752 ms 2024-09-19T09:05:31.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.106+0000 7f6f79b9f640 1 -- 172.21.15.120:0/1860068917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f74101270 msgr2=0x7f6f7410d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:31.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.106+0000 7f6f79b9f640 1 --2- 172.21.15.120:0/1860068917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f74101270 0x7f6f7410d7a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c009cd0 tx=0x7f6f6c02a8e0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.106+0000 7f6f79b9f640 1 -- 172.21.15.120:0/1860068917 shutdown_connections 2024-09-19T09:05:31.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.106+0000 7f6f79b9f640 1 --2- 172.21.15.120:0/1860068917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f74101270 0x7f6f7410d7a0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.106+0000 7f6f79b9f640 1 --2- 172.21.15.120:0/1860068917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f741008d0 0x7f6f74100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.106+0000 7f6f79b9f640 1 --2- 172.21.15.120:0/1860068917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f741068d0 0x7f6f74106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.106+0000 7f6f79b9f640 1 -- 172.21.15.120:0/1860068917 >> 172.21.15.120:0/1860068917 conn(0x7f6f740fc610 msgr2=0x7f6f740fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:31.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.106+0000 7f6f79b9f640 1 -- 172.21.15.120:0/1860068917 shutdown_connections 2024-09-19T09:05:31.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.106+0000 7f6f79b9f640 1 -- 172.21.15.120:0/1860068917 wait complete. 2024-09-19T09:05:31.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.107+0000 7f6f79b9f640 1 Processor -- start 2024-09-19T09:05:31.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.107+0000 7f6f79b9f640 1 -- start start 2024-09-19T09:05:31.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.107+0000 7f6f79b9f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f741008d0 0x7f6f741990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:31.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.107+0000 7f6f79b9f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f74101270 0x7f6f74199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:31.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.107+0000 7f6f79b9f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f741068d0 0x7f6f741a4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:31.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.107+0000 7f6f79b9f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6f74110cd0 con 0x7f6f741008d0 2024-09-19T09:05:31.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.107+0000 7f6f79b9f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6f74110b50 con 0x7f6f741068d0 2024-09-19T09:05:31.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.107+0000 7f6f79b9f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6f74110e50 con 0x7f6f74101270 2024-09-19T09:05:31.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.107+0000 7f6f78b9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f741008d0 0x7f6f741990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:31.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.107+0000 7f6f78b9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f741008d0 0x7f6f741990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36400/0 (socket says 172.21.15.120:36400) 2024-09-19T09:05:31.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.107+0000 7f6f78b9d640 1 -- 172.21.15.120:0/97741143 learned_addr learned my addr 172.21.15.120:0/97741143 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:31.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.107+0000 7f6f7939e640 1 --2- 172.21.15.120:0/97741143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f741068d0 0x7f6f741a4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:31.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.107+0000 7f6f73fff640 1 --2- 172.21.15.120:0/97741143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f74101270 0x7f6f74199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:31.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.107+0000 7f6f78b9d640 1 -- 172.21.15.120:0/97741143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f74101270 msgr2=0x7f6f74199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:31.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.107+0000 7f6f78b9d640 1 --2- 172.21.15.120:0/97741143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f74101270 0x7f6f74199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.107+0000 7f6f78b9d640 1 -- 172.21.15.120:0/97741143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f741068d0 msgr2=0x7f6f741a4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:31.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.107+0000 7f6f78b9d640 1 --2- 172.21.15.120:0/97741143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f741068d0 0x7f6f741a4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.107+0000 7f6f78b9d640 1 -- 172.21.15.120:0/97741143 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f741a4690 con 0x7f6f741008d0 2024-09-19T09:05:31.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.107+0000 7f6f73fff640 1 --2- 172.21.15.120:0/97741143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f74101270 0x7f6f74199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:05:31.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.107+0000 7f6f78b9d640 1 --2- 172.21.15.120:0/97741143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f741008d0 0x7f6f741990d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6f6800e9f0 tx=0x7f6f6800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:31.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.108+0000 7f6f71ffb640 1 -- 172.21.15.120:0/97741143 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6800cd80 con 0x7f6f741008d0 2024-09-19T09:05:31.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.108+0000 7f6f71ffb640 1 -- 172.21.15.120:0/97741143 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f68010910 con 0x7f6f741008d0 2024-09-19T09:05:31.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.108+0000 7f6f71ffb640 1 -- 172.21.15.120:0/97741143 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f68018990 con 0x7f6f741008d0 2024-09-19T09:05:31.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.108+0000 7f6f79b9f640 1 -- 172.21.15.120:0/97741143 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f741a4860 con 0x7f6f741008d0 2024-09-19T09:05:31.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.108+0000 7f6f79b9f640 1 -- 172.21.15.120:0/97741143 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f741a4da0 con 0x7f6f741008d0 2024-09-19T09:05:31.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.110+0000 7f6f71ffb640 1 -- 172.21.15.120:0/97741143 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f6f68010ac0 con 0x7f6f741008d0 2024-09-19T09:05:31.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.110+0000 7f6f7d1da740 1 Processor -- start 2024-09-19T09:05:31.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.111+0000 7f6f7d1da740 1 -- start start 2024-09-19T09:05:31.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.111+0000 7f6f7d1da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4dc5b4d20 0x55b4dc5b3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:31.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.111+0000 7f6f7d1da740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4dc5b37b0 0x55b4dc5b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:31.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.111+0000 7f6f7d1da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4dc5be550 0x55b4dc5c0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:31.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.111+0000 7f6f7d1da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b4dc5192f0 con 0x55b4dc5be550 2024-09-19T09:05:31.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.111+0000 7f6f7d1da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b4dc4c0cf0 con 0x55b4dc5b4d20 2024-09-19T09:05:31.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.111+0000 7f6f71ffb640 1 --2- 172.21.15.120:0/97741143 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6f640a32c0 0x7f6f640a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:31.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.111+0000 7f6f7d1da740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b4dc5152b0 con 0x55b4dc5b37b0 2024-09-19T09:05:31.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.111+0000 7f6f7939e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4dc5b4d20 0x55b4dc5b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:31.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.111+0000 7f6f7939e640 1 --2- 172.21.15.120:0/97741143 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6f640a32c0 0x7f6f640a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:31.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.111+0000 7f6f78b9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4dc5b37b0 0x55b4dc5b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:31.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.111+0000 7f6f7939e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4dc5b4d20 0x55b4dc5b3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:48748/0 (socket says 172.21.15.120:48748) 2024-09-19T09:05:31.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.111+0000 7f6f7939e640 1 -- 172.21.15.120:0/713026460 learned_addr learned my addr 172.21.15.120:0/713026460 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:31.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.111+0000 7f6f73fff640 1 --2- 172.21.15.120:0/713026460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4dc5be550 0x55b4dc5c0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:31.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.111+0000 7f6f71ffb640 1 -- 172.21.15.120:0/97741143 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f6f680cfb90 con 0x7f6f741008d0 2024-09-19T09:05:31.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.111+0000 7f6f73fff640 1 -- 172.21.15.120:0/713026460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4dc5b37b0 msgr2=0x55b4dc5b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:31.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.111+0000 7f6f73fff640 1 --2- 172.21.15.120:0/713026460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4dc5b37b0 0x55b4dc5b3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.111+0000 7f6f73fff640 1 -- 172.21.15.120:0/713026460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4dc5b4d20 msgr2=0x55b4dc5b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:31.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.111+0000 7f6f73fff640 1 --2- 172.21.15.120:0/713026460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4dc5b4d20 0x55b4dc5b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.111+0000 7f6f73fff640 1 -- 172.21.15.120:0/713026460 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4dc4a53a0 con 0x55b4dc5be550 2024-09-19T09:05:31.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.111+0000 7f6f78b9d640 1 --2- 172.21.15.120:0/713026460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4dc5b37b0 0x55b4dc5b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:05:31.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.112+0000 7f6f7939e640 1 --2- 172.21.15.120:0/97741143 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6f640a32c0 0x7f6f640a5780 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c02ae00 tx=0x7f6f6c0093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:31.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.112+0000 7f6f73fff640 1 --2- 172.21.15.120:0/713026460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4dc5be550 0x55b4dc5c0940 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6f7406aac0 tx=0x7f6f6000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c5e1360572618d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:31.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.112+0000 7f6f5effd640 1 -- 172.21.15.120:0/713026460 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60004030 con 0x55b4dc5be550 2024-09-19T09:05:31.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.112+0000 7f6f5effd640 1 -- 172.21.15.120:0/713026460 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f60005500 con 0x55b4dc5be550 2024-09-19T09:05:31.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.112+0000 7f6f5effd640 1 -- 172.21.15.120:0/713026460 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60004bc0 con 0x55b4dc5be550 2024-09-19T09:05:31.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.112+0000 7f6f7d1da740 1 -- 172.21.15.120:0/713026460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4dc5be550 msgr2=0x55b4dc5c0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:31.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.112+0000 7f6f7d1da740 1 --2- 172.21.15.120:0/713026460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4dc5be550 0x55b4dc5c0940 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6f7406aac0 tx=0x7f6f6000f730 comp rx=0 tx=0).stop 2024-09-19T09:05:31.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.113+0000 7f6f7d1da740 1 -- 172.21.15.120:0/713026460 shutdown_connections 2024-09-19T09:05:31.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.113+0000 7f6f7d1da740 1 --2- 172.21.15.120:0/713026460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4dc5be550 0x55b4dc5c0940 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.113+0000 7f6f7d1da740 1 --2- 172.21.15.120:0/713026460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4dc5b37b0 0x55b4dc5b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.113+0000 7f6f7d1da740 1 --2- 172.21.15.120:0/713026460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4dc5b4d20 0x55b4dc5b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.113+0000 7f6f7d1da740 1 -- 172.21.15.120:0/713026460 >> 172.21.15.120:0/713026460 conn(0x55b4dc57c310 msgr2=0x55b4dc5afe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:31.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.113+0000 7f6f7d1da740 1 -- 172.21.15.120:0/713026460 shutdown_connections 2024-09-19T09:05:31.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.113+0000 7f6f7d1da740 1 -- 172.21.15.120:0/713026460 wait complete. 2024-09-19T09:05:31.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.113+0000 7f6f7d1da740 1 -- start start 2024-09-19T09:05:31.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.113+0000 7f6f7d1da740 1 Processor -- start 2024-09-19T09:05:31.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.113+0000 7f6f7d1da740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4dc5b37b0 0x55b4dc5bde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:31.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.113+0000 7f6f7d1da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4dc5b4d20 0x55b4dc5bb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:31.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.113+0000 7f6f7d1da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4dc5be550 0x55b4dc5bb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:31.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.113+0000 7f6f7d1da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b4dc5c1250 con 0x55b4dc5be550 2024-09-19T09:05:31.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.113+0000 7f6f7d1da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b4dc4d4060 con 0x55b4dc5b4d20 2024-09-19T09:05:31.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.113+0000 7f6f7d1da740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b4dc5c13d0 con 0x55b4dc5b37b0 2024-09-19T09:05:31.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.114+0000 7f6f7d1da740 10 client.0 _ll_register_callbacks cb 0x55b4dc2278c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:31.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.114+0000 7f6f7939e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4dc5be550 0x55b4dc5bb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:31.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.114+0000 7f6f7939e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4dc5be550 0x55b4dc5bb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36410/0 (socket says 172.21.15.120:36410) 2024-09-19T09:05:31.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.114+0000 7f6f7939e640 1 -- 172.21.15.120:0/3667481471 learned_addr learned my addr 172.21.15.120:0/3667481471 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:31.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.114+0000 7f6f78b9d640 1 --2- 172.21.15.120:0/3667481471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4dc5b37b0 0x55b4dc5bde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:31.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.114+0000 7f6f73fff640 1 --2- 172.21.15.120:0/3667481471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4dc5b4d20 0x55b4dc5bb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:31.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.114+0000 7f6f7939e640 1 -- 172.21.15.120:0/3667481471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4dc5b37b0 msgr2=0x55b4dc5bde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:31.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.114+0000 7f6f7939e640 1 --2- 172.21.15.120:0/3667481471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4dc5b37b0 0x55b4dc5bde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.114+0000 7f6f7939e640 1 -- 172.21.15.120:0/3667481471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4dc5b4d20 msgr2=0x55b4dc5bb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:31.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.114+0000 7f6f7939e640 1 --2- 172.21.15.120:0/3667481471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4dc5b4d20 0x55b4dc5bb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.114+0000 7f6f7939e640 1 -- 172.21.15.120:0/3667481471 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4dc28a3c0 con 0x55b4dc5be550 2024-09-19T09:05:31.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.114+0000 7f6f7939e640 1 --2- 172.21.15.120:0/3667481471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4dc5be550 0x55b4dc5bb660 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c031860 tx=0x7f6f6c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ce8691fc920169 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:31.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.114+0000 7f6f53fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:31.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.114+0000 7f6f53fff640 1 -- 172.21.15.120:0/3667481471 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c041070 con 0x55b4dc5be550 2024-09-19T09:05:31.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.114+0000 7f6f53fff640 1 -- 172.21.15.120:0/3667481471 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f6f6c0047a0 con 0x55b4dc5be550 2024-09-19T09:05:31.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.114+0000 7f6f7d1da740 1 -- 172.21.15.120:0/3667481471 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f60005500 con 0x55b4dc5be550 2024-09-19T09:05:31.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.114+0000 7f6f53fff640 1 -- 172.21.15.120:0/3667481471 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6f6c044040 con 0x55b4dc5be550 2024-09-19T09:05:31.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.114+0000 7f6f7d1da740 10 client.16299 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:31.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.114+0000 7f6f7d1da740 1 -- 172.21.15.120:0/3667481471 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b4dc5ccbb0 con 0x55b4dc5be550 2024-09-19T09:05:31.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.115+0000 7f6f53fff640 1 -- 172.21.15.120:0/3667481471 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f60005500 con 0x55b4dc5be550 2024-09-19T09:05:31.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.115+0000 7f6f53fff640 1 -- 172.21.15.120:0/3667481471 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6f6c05d020 con 0x55b4dc5be550 2024-09-19T09:05:31.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.115+0000 7f6f7d1da740 10 client.16299 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:31.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.115+0000 7f6f7d1da740 10 client.16299 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:05:31.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.115+0000 7f6f7d1da740 20 client.16299 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:31.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.115+0000 7f6f7d1da740 1 --2- 172.21.15.120:0/3667481471 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55b4dc5cdfc0 0x55b4dc5d0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:31.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.115+0000 7f6f7d1da740 4 client.16299 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:05:31.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.115+0000 7f6f7d1da740 1 -- 172.21.15.120:0/3667481471 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4dc526860 con 0x55b4dc5cdfc0 2024-09-19T09:05:31.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.115+0000 7f6f78b9d640 1 --2- 172.21.15.120:0/3667481471 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55b4dc5cdfc0 0x55b4dc5d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:31.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.116+0000 7f6f53fff640 10 client.16299 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:05:31.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.116+0000 7f6f78b9d640 1 --2- 172.21.15.120:0/3667481471 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55b4dc5cdfc0 0x55b4dc5d0450 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=88f1f76e67916975 server_cookie=ebdfc73031a0e7f5 in_seq=0 out_seq=0 2024-09-19T09:05:31.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.116+0000 7f6f53fff640 1 -- 172.21.15.120:0/3667481471 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b4dc526860 con 0x55b4dc5cdfc0 2024-09-19T09:05:31.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.116+0000 7f6f53fff640 10 client.16299 handle_command_reply: tid=0 2024-09-19T09:05:31.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.116+0000 7f6f7d1da740 1 client.16299 shutdown 2024-09-19T09:05:31.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.116+0000 7f6f7d1da740 1 -- 172.21.15.120:0/3667481471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4dc5be550 msgr2=0x55b4dc5bb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:31.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.116+0000 7f6f7d1da740 1 --2- 172.21.15.120:0/3667481471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4dc5be550 0x55b4dc5bb660 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c031860 tx=0x7f6f6c009360 comp rx=0 tx=0).stop 2024-09-19T09:05:31.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.116+0000 7f6f7d1da740 1 -- 172.21.15.120:0/3667481471 shutdown_connections 2024-09-19T09:05:31.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.116+0000 7f6f7d1da740 1 --2- 172.21.15.120:0/3667481471 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55b4dc5cdfc0 0x55b4dc5d0450 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.117+0000 7f6f7d1da740 1 --2- 172.21.15.120:0/3667481471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4dc5be550 0x55b4dc5bb660 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.117+0000 7f6f7d1da740 1 --2- 172.21.15.120:0/3667481471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4dc5b4d20 0x55b4dc5bb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.117+0000 7f6f7d1da740 1 --2- 172.21.15.120:0/3667481471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4dc5b37b0 0x55b4dc5bde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.117+0000 7f6f53fff640 0 client.16299 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:05:31.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.117+0000 7f6f7d1da740 1 -- 172.21.15.120:0/3667481471 >> 172.21.15.120:0/3667481471 conn(0x55b4dc57c310 msgr2=0x55b4dc5b5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:31.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.117+0000 7f6f7d1da740 1 -- 172.21.15.120:0/3667481471 shutdown_connections 2024-09-19T09:05:31.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.117+0000 7f6f7d1da740 1 -- 172.21.15.120:0/3667481471 wait complete. 2024-09-19T09:05:31.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.117+0000 7f6f7d1da740 20 client.16299 trim_cache size 0 max 16384 2024-09-19T09:05:31.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.130+0000 7f6f7d1da740 1 Processor -- start 2024-09-19T09:05:31.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.130+0000 7f6f7d1da740 1 -- start start 2024-09-19T09:05:31.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.130+0000 7f6f7d1da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4dc5b37b0 0x55b4dc5ca520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:31.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.130+0000 7f6f7d1da740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4dc5b4d20 0x55b4dc5bd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:31.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.130+0000 7f6f7d1da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4dc5be550 0x55b4dc5d5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:31.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.130+0000 7f6f7d1da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6f741ad500 con 0x55b4dc5be550 2024-09-19T09:05:31.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.130+0000 7f6f7d1da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6f741af040 con 0x55b4dc5b37b0 2024-09-19T09:05:31.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.130+0000 7f6f7d1da740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6f741adbd0 con 0x55b4dc5b4d20 2024-09-19T09:05:31.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.131+0000 7f6f7939e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4dc5be550 0x55b4dc5d5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:31.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.131+0000 7f6f7939e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4dc5be550 0x55b4dc5d5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36416/0 (socket says 172.21.15.120:36416) 2024-09-19T09:05:31.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.131+0000 7f6f7939e640 1 -- 172.21.15.120:0/1513007455 learned_addr learned my addr 172.21.15.120:0/1513007455 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:31.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.131+0000 7f6f78b9d640 1 --2- 172.21.15.120:0/1513007455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4dc5b37b0 0x55b4dc5ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:31.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.131+0000 7f6f73fff640 1 --2- 172.21.15.120:0/1513007455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4dc5b4d20 0x55b4dc5bd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:31.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.131+0000 7f6f7939e640 1 -- 172.21.15.120:0/1513007455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4dc5b4d20 msgr2=0x55b4dc5bd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:31.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.131+0000 7f6f7939e640 1 --2- 172.21.15.120:0/1513007455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4dc5b4d20 0x55b4dc5bd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.131+0000 7f6f7939e640 1 -- 172.21.15.120:0/1513007455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4dc5b37b0 msgr2=0x55b4dc5ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:31.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.131+0000 7f6f7939e640 1 --2- 172.21.15.120:0/1513007455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4dc5b37b0 0x55b4dc5ca520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.131+0000 7f6f7939e640 1 -- 172.21.15.120:0/1513007455 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4dc5b3f30 con 0x55b4dc5be550 2024-09-19T09:05:31.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.131+0000 7f6f73fff640 1 --2- 172.21.15.120:0/1513007455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4dc5b4d20 0x55b4dc5bd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:05:31.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.131+0000 7f6f7939e640 1 --2- 172.21.15.120:0/1513007455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4dc5be550 0x55b4dc5d5d20 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c009cd0 tx=0x7f6f6c005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8131b6bdc90a92a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:31.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.131+0000 7f6f5e7fc640 1 -- 172.21.15.120:0/1513007455 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c041040 con 0x55b4dc5be550 2024-09-19T09:05:31.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.131+0000 7f6f5e7fc640 1 -- 172.21.15.120:0/1513007455 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f6c064070 con 0x55b4dc5be550 2024-09-19T09:05:31.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.131+0000 7f6f5e7fc640 1 -- 172.21.15.120:0/1513007455 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c05fb60 con 0x55b4dc5be550 2024-09-19T09:05:31.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.132+0000 7f6f7d1da740 1 -- 172.21.15.120:0/1513007455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4dc5be550 msgr2=0x55b4dc5d5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:31.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.132+0000 7f6f7d1da740 1 --2- 172.21.15.120:0/1513007455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4dc5be550 0x55b4dc5d5d20 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c009cd0 tx=0x7f6f6c005f50 comp rx=0 tx=0).stop 2024-09-19T09:05:31.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.132+0000 7f6f7d1da740 1 -- 172.21.15.120:0/1513007455 shutdown_connections 2024-09-19T09:05:31.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.132+0000 7f6f7d1da740 1 --2- 172.21.15.120:0/1513007455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4dc5be550 0x55b4dc5d5d20 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.132+0000 7f6f7d1da740 1 --2- 172.21.15.120:0/1513007455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4dc5b4d20 0x55b4dc5bd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.132+0000 7f6f7d1da740 1 --2- 172.21.15.120:0/1513007455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4dc5b37b0 0x55b4dc5ca520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.132+0000 7f6f7d1da740 1 -- 172.21.15.120:0/1513007455 >> 172.21.15.120:0/1513007455 conn(0x55b4dc57c310 msgr2=0x55b4dc5bc980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:31.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.132+0000 7f6f7d1da740 1 -- 172.21.15.120:0/1513007455 shutdown_connections 2024-09-19T09:05:31.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.132+0000 7f6f7d1da740 1 -- 172.21.15.120:0/1513007455 wait complete. 2024-09-19T09:05:31.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.132+0000 7f6f7d1da740 1 -- start start 2024-09-19T09:05:31.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.133+0000 7f6f7d1da740 1 Processor -- start 2024-09-19T09:05:31.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.133+0000 7f6f7d1da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4dc5b37b0 0x55b4dc5c23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:31.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.133+0000 7f6f7d1da740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4dc5b4d20 0x55b4dc5c2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:31.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.133+0000 7f6f7d1da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4dc5be550 0x55b4dc5c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:31.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.133+0000 7f6f7d1da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6f741ae970 con 0x55b4dc5b37b0 2024-09-19T09:05:31.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.133+0000 7f6f7d1da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6f741ae2a0 con 0x55b4dc5be550 2024-09-19T09:05:31.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.133+0000 7f6f7d1da740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6f741ab4c0 con 0x55b4dc5b4d20 2024-09-19T09:05:31.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.133+0000 7f6f7d1da740 10 client.0 _ll_register_callbacks cb 0x7f6f6c032f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:31.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.133+0000 7f6f78b9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4dc5b37b0 0x55b4dc5c23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:31.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.133+0000 7f6f78b9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4dc5b37b0 0x55b4dc5c23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36430/0 (socket says 172.21.15.120:36430) 2024-09-19T09:05:31.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.133+0000 7f6f78b9d640 1 -- 172.21.15.120:0/2067401869 learned_addr learned my addr 172.21.15.120:0/2067401869 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:31.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.133+0000 7f6f7939e640 1 --2- 172.21.15.120:0/2067401869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4dc5be550 0x55b4dc5c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:31.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.133+0000 7f6f73fff640 1 --2- 172.21.15.120:0/2067401869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4dc5b4d20 0x55b4dc5c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:31.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.133+0000 7f6f78b9d640 1 -- 172.21.15.120:0/2067401869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4dc5b4d20 msgr2=0x55b4dc5c2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:31.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.134+0000 7f6f78b9d640 1 --2- 172.21.15.120:0/2067401869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4dc5b4d20 0x55b4dc5c2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.134+0000 7f6f78b9d640 1 -- 172.21.15.120:0/2067401869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4dc5be550 msgr2=0x55b4dc5c2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:31.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.134+0000 7f6f78b9d640 1 --2- 172.21.15.120:0/2067401869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4dc5be550 0x55b4dc5c2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.134+0000 7f6f78b9d640 1 -- 172.21.15.120:0/2067401869 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4dc637cf0 con 0x55b4dc5b37b0 2024-09-19T09:05:31.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.134+0000 7f6f78b9d640 1 --2- 172.21.15.120:0/2067401869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4dc5b37b0 0x55b4dc5c23d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f6f6809bb80 tx=0x7f6f68025360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dfe70ff3f6183e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:31.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.134+0000 7f6f53fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:31.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.134+0000 7f6f53fff640 1 -- 172.21.15.120:0/2067401869 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f68025170 con 0x55b4dc5b37b0 2024-09-19T09:05:31.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.134+0000 7f6f53fff640 1 -- 172.21.15.120:0/2067401869 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f6f680cd7a0 con 0x55b4dc5b37b0 2024-09-19T09:05:31.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.134+0000 7f6f7d1da740 1 -- 172.21.15.120:0/2067401869 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f6c064070 con 0x55b4dc5b37b0 2024-09-19T09:05:31.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.134+0000 7f6f53fff640 1 -- 172.21.15.120:0/2067401869 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6f68098820 con 0x55b4dc5b37b0 2024-09-19T09:05:31.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.134+0000 7f6f7d1da740 10 client.16305 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:31.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.134+0000 7f6f7d1da740 1 -- 172.21.15.120:0/2067401869 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b4dc638f80 con 0x55b4dc5b37b0 2024-09-19T09:05:31.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.134+0000 7f6f53fff640 1 -- 172.21.15.120:0/2067401869 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f6c064070 con 0x55b4dc5b37b0 2024-09-19T09:05:31.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.134+0000 7f6f53fff640 1 -- 172.21.15.120:0/2067401869 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6f680a0160 con 0x55b4dc5b37b0 2024-09-19T09:05:31.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.134+0000 7f6f7d1da740 10 client.16305 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:31.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.134+0000 7f6f7d1da740 10 client.16305 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:05:31.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.135+0000 7f6f7d1da740 20 client.16305 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:31.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.135+0000 7f6f7d1da740 1 --2- 172.21.15.120:0/2067401869 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55b4dc639650 0x55b4dc63ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:31.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.135+0000 7f6f7d1da740 4 client.16305 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:05:31.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.135+0000 7f6f7d1da740 1 -- 172.21.15.120:0/2067401869 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b4dc520d70 con 0x55b4dc639650 2024-09-19T09:05:31.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.135+0000 7f6f73fff640 1 --2- 172.21.15.120:0/2067401869 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55b4dc639650 0x55b4dc63ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:31.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.135+0000 7f6f53fff640 10 client.16305 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:05:31.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.135+0000 7f6f73fff640 1 --2- 172.21.15.120:0/2067401869 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55b4dc639650 0x55b4dc63ba70 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d51875399d11ba31 server_cookie=53f267514ef1eec4 in_seq=0 out_seq=0 2024-09-19T09:05:31.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.135+0000 7f6f53fff640 1 -- 172.21.15.120:0/2067401869 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b4dc520d70 con 0x55b4dc639650 2024-09-19T09:05:31.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.135+0000 7f6f53fff640 10 client.16305 handle_command_reply: tid=0 2024-09-19T09:05:31.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.135+0000 7f6f7d1da740 1 client.16305 shutdown 2024-09-19T09:05:31.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.135+0000 7f6f7d1da740 1 -- 172.21.15.120:0/2067401869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4dc5b37b0 msgr2=0x55b4dc5c23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:31.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.135+0000 7f6f7d1da740 1 --2- 172.21.15.120:0/2067401869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4dc5b37b0 0x55b4dc5c23d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f6f6809bb80 tx=0x7f6f68025360 comp rx=0 tx=0).stop 2024-09-19T09:05:31.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.135+0000 7f6f7d1da740 1 -- 172.21.15.120:0/2067401869 shutdown_connections 2024-09-19T09:05:31.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.135+0000 7f6f7d1da740 1 --2- 172.21.15.120:0/2067401869 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55b4dc639650 0x55b4dc63ba70 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.135+0000 7f6f7d1da740 1 --2- 172.21.15.120:0/2067401869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b4dc5be550 0x55b4dc5c2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.135+0000 7f6f7d1da740 1 --2- 172.21.15.120:0/2067401869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b4dc5b4d20 0x55b4dc5c2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.135+0000 7f6f53fff640 0 client.16305 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:05:31.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.135+0000 7f6f7d1da740 1 --2- 172.21.15.120:0/2067401869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4dc5b37b0 0x55b4dc5c23d0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.135+0000 7f6f7d1da740 1 -- 172.21.15.120:0/2067401869 >> 172.21.15.120:0/2067401869 conn(0x55b4dc57c310 msgr2=0x55b4dc5ae780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:31.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.136+0000 7f6f7d1da740 1 -- 172.21.15.120:0/2067401869 shutdown_connections 2024-09-19T09:05:31.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.136+0000 7f6f7d1da740 1 -- 172.21.15.120:0/2067401869 wait complete. 2024-09-19T09:05:31.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.136+0000 7f6f7d1da740 20 client.16305 trim_cache size 0 max 16384 2024-09-19T09:05:31.155 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:05:31.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.136+0000 7f6f79b9f640 1 -- 172.21.15.120:0/97741143 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6f640a32c0 msgr2=0x7f6f640a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:31.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.136+0000 7f6f79b9f640 1 --2- 172.21.15.120:0/97741143 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6f640a32c0 0x7f6f640a5780 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c02ae00 tx=0x7f6f6c0093f0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.136+0000 7f6f79b9f640 1 -- 172.21.15.120:0/97741143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f741008d0 msgr2=0x7f6f741990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:31.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.136+0000 7f6f79b9f640 1 --2- 172.21.15.120:0/97741143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f741008d0 0x7f6f741990d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6f6800e9f0 tx=0x7f6f6800eeb0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.137+0000 7f6f79b9f640 1 -- 172.21.15.120:0/97741143 shutdown_connections 2024-09-19T09:05:31.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.137+0000 7f6f79b9f640 1 --2- 172.21.15.120:0/97741143 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6f640a32c0 0x7f6f640a5780 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.137+0000 7f6f79b9f640 1 --2- 172.21.15.120:0/97741143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f741068d0 0x7f6f741a4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.137+0000 7f6f79b9f640 1 --2- 172.21.15.120:0/97741143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f74101270 0x7f6f74199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.137+0000 7f6f79b9f640 1 --2- 172.21.15.120:0/97741143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f741008d0 0x7f6f741990d0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:31.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.137+0000 7f6f79b9f640 1 -- 172.21.15.120:0/97741143 >> 172.21.15.120:0/97741143 conn(0x7f6f740fc610 msgr2=0x7f6f740fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:31.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.137+0000 7f6f79b9f640 1 -- 172.21.15.120:0/97741143 shutdown_connections 2024-09-19T09:05:31.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:31.137+0000 7f6f79b9f640 1 -- 172.21.15.120:0/97741143 wait complete. 2024-09-19T09:05:31.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:30 smithi196 ceph-mon[35150]: pgmap v420: 97 pgs: 97 active+clean; 48 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.8 MiB/s wr, 340 op/s 2024-09-19T09:05:31.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:30 smithi153 ceph-mon[35071]: pgmap v420: 97 pgs: 97 active+clean; 48 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.8 MiB/s wr, 340 op/s 2024-09-19T09:05:31.831 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:05:32.039 INFO:tasks.workunit.client.0.smithi120.stdout: 1 135487 8.60 MB/sec warmup 87 sec latency 10.932 ms 2024-09-19T09:05:32.090 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:05:32.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.614+0000 7f0068962640 1 -- 172.21.15.120:0/2911861833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0064100e20 msgr2=0x7f0064101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:32.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.614+0000 7f0068962640 1 --2- 172.21.15.120:0/2911861833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0064100e20 0x7f0064101200 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0054009900 tx=0x7f005402a480 comp rx=0 tx=0).stop 2024-09-19T09:05:32.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.614+0000 7f0068962640 1 -- 172.21.15.120:0/2911861833 shutdown_connections 2024-09-19T09:05:32.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.614+0000 7f0068962640 1 --2- 172.21.15.120:0/2911861833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0064102ef0 0x7f006410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.614+0000 7f0068962640 1 --2- 172.21.15.120:0/2911861833 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0064102570 0x7f00641029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.614+0000 7f0068962640 1 --2- 172.21.15.120:0/2911861833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0064100e20 0x7f0064101200 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.614+0000 7f0068962640 1 -- 172.21.15.120:0/2911861833 >> 172.21.15.120:0/2911861833 conn(0x7f00640fc7d0 msgr2=0x7f00640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:32.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.614+0000 7f0068962640 1 -- 172.21.15.120:0/2911861833 shutdown_connections 2024-09-19T09:05:32.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.614+0000 7f0068962640 1 -- 172.21.15.120:0/2911861833 wait complete. 2024-09-19T09:05:32.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.615+0000 7f0068962640 1 Processor -- start 2024-09-19T09:05:32.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.615+0000 7f0068962640 1 -- start start 2024-09-19T09:05:32.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.616+0000 7f0068962640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0064100e20 0x7f00641984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:32.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.616+0000 7f0068962640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0064102570 0x7f0064198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:32.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.616+0000 7f0068962640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0064102ef0 0x7f006419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:32.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.616+0000 7f0068962640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0064110f80 con 0x7f0064102570 2024-09-19T09:05:32.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.616+0000 7f0068962640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0064110e00 con 0x7f0064100e20 2024-09-19T09:05:32.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.616+0000 7f0068962640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0064111100 con 0x7f0064102ef0 2024-09-19T09:05:32.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.616+0000 7f0061d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0064102570 0x7f0064198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:32.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.616+0000 7f006259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0064100e20 0x7f00641984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:32.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.616+0000 7f006259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0064100e20 0x7f00641984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:54988/0 (socket says 172.21.15.120:54988) 2024-09-19T09:05:32.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.616+0000 7f006259c640 1 -- 172.21.15.120:0/2528243686 learned_addr learned my addr 172.21.15.120:0/2528243686 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:32.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.616+0000 7f0062d9d640 1 --2- 172.21.15.120:0/2528243686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0064102ef0 0x7f006419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:32.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.616+0000 7f006259c640 1 -- 172.21.15.120:0/2528243686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0064102ef0 msgr2=0x7f006419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:32.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.616+0000 7f006259c640 1 --2- 172.21.15.120:0/2528243686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0064102ef0 0x7f006419cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.616+0000 7f006259c640 1 -- 172.21.15.120:0/2528243686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0064102570 msgr2=0x7f0064198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:32.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.616+0000 7f006259c640 1 --2- 172.21.15.120:0/2528243686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0064102570 0x7f0064198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.616+0000 7f006259c640 1 -- 172.21.15.120:0/2528243686 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006419d470 con 0x7f0064100e20 2024-09-19T09:05:32.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.616+0000 7f006259c640 1 --2- 172.21.15.120:0/2528243686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0064100e20 0x7f00641984c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0054006010 tx=0x7f0054003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:32.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.616+0000 7f00537fe640 1 -- 172.21.15.120:0/2528243686 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005403f070 con 0x7f0064100e20 2024-09-19T09:05:32.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.617+0000 7f0068962640 1 -- 172.21.15.120:0/2528243686 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f006419d700 con 0x7f0064100e20 2024-09-19T09:05:32.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.617+0000 7f00537fe640 1 -- 172.21.15.120:0/2528243686 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0054031c30 con 0x7f0064100e20 2024-09-19T09:05:32.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.617+0000 7f0068962640 1 -- 172.21.15.120:0/2528243686 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00641a5040 con 0x7f0064100e20 2024-09-19T09:05:32.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.617+0000 7f00537fe640 1 -- 172.21.15.120:0/2528243686 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054035030 con 0x7f0064100e20 2024-09-19T09:05:32.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.619+0000 7f006ad3b740 1 Processor -- start 2024-09-19T09:05:32.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.619+0000 7f006ad3b740 1 -- start start 2024-09-19T09:05:32.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.619+0000 7f006ad3b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd20edbb40 0x55cd20eda090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:32.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.619+0000 7f006ad3b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd20eda5d0 0x55cd20edaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:32.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.619+0000 7f006ad3b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd20ee5370 0x55cd20ee7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:32.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.619+0000 7f006ad3b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cd20e402f0 con 0x55cd20ee5370 2024-09-19T09:05:32.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.619+0000 7f006ad3b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cd20de7cf0 con 0x55cd20eda5d0 2024-09-19T09:05:32.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.619+0000 7f006ad3b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cd20e3c2b0 con 0x55cd20edbb40 2024-09-19T09:05:32.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.619+0000 7f00537fe640 1 -- 172.21.15.120:0/2528243686 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f0054042070 con 0x7f0064100e20 2024-09-19T09:05:32.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.620+0000 7f0062d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd20edbb40 0x55cd20eda090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:32.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.620+0000 7f0061d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd20ee5370 0x55cd20ee7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:32.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.620+0000 7f0062d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd20edbb40 0x55cd20eda090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:45006/0 (socket says 172.21.15.120:45006) 2024-09-19T09:05:32.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.620+0000 7f0062d9d640 1 -- 172.21.15.120:0/738627015 learned_addr learned my addr 172.21.15.120:0/738627015 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:32.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.620+0000 7f006259c640 1 --2- 172.21.15.120:0/738627015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd20eda5d0 0x55cd20edaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:32.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.620+0000 7f0062d9d640 1 -- 172.21.15.120:0/738627015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd20eda5d0 msgr2=0x55cd20edaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:32.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.620+0000 7f0062d9d640 1 --2- 172.21.15.120:0/738627015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd20eda5d0 0x55cd20edaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.620+0000 7f0062d9d640 1 -- 172.21.15.120:0/738627015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd20ee5370 msgr2=0x55cd20ee7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:32.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.620+0000 7f0062d9d640 1 --2- 172.21.15.120:0/738627015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd20ee5370 0x55cd20ee7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.620+0000 7f0062d9d640 1 -- 172.21.15.120:0/738627015 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd20dcc3a0 con 0x55cd20edbb40 2024-09-19T09:05:32.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.620+0000 7f006259c640 1 --2- 172.21.15.120:0/738627015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd20eda5d0 0x55cd20edaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:05:32.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.620+0000 7f0062d9d640 1 --2- 172.21.15.120:0/738627015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd20edbb40 0x55cd20eda090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f005c010370 tx=0x7f005c010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=947f61969bc49cde server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:32.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.620+0000 7f0050ff9640 1 -- 172.21.15.120:0/738627015 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c016070 con 0x55cd20edbb40 2024-09-19T09:05:32.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.620+0000 7f0050ff9640 1 -- 172.21.15.120:0/738627015 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f005c004d20 con 0x55cd20edbb40 2024-09-19T09:05:32.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.620+0000 7f0050ff9640 1 -- 172.21.15.120:0/738627015 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c005420 con 0x55cd20edbb40 2024-09-19T09:05:32.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.620+0000 7f00537fe640 1 --2- 172.21.15.120:0/2528243686 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f004c0a32c0 0x7f004c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:32.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.621+0000 7f006259c640 1 --2- 172.21.15.120:0/2528243686 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f004c0a32c0 0x7f004c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:32.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.621+0000 7f00537fe640 1 -- 172.21.15.120:0/2528243686 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f00540efef0 con 0x7f0064100e20 2024-09-19T09:05:32.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.621+0000 7f006ad3b740 1 -- 172.21.15.120:0/738627015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd20edbb40 msgr2=0x55cd20eda090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:32.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.621+0000 7f006259c640 1 --2- 172.21.15.120:0/2528243686 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f004c0a32c0 0x7f004c0a5780 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f00540c4c50 tx=0x7f005402a480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:32.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.621+0000 7f006ad3b740 1 --2- 172.21.15.120:0/738627015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd20edbb40 0x55cd20eda090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f005c010370 tx=0x7f005c010830 comp rx=0 tx=0).stop 2024-09-19T09:05:32.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.621+0000 7f006ad3b740 1 -- 172.21.15.120:0/738627015 shutdown_connections 2024-09-19T09:05:32.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.621+0000 7f006ad3b740 1 --2- 172.21.15.120:0/738627015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd20ee5370 0x55cd20ee7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.621+0000 7f006ad3b740 1 --2- 172.21.15.120:0/738627015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd20eda5d0 0x55cd20edaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.621+0000 7f006ad3b740 1 --2- 172.21.15.120:0/738627015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd20edbb40 0x55cd20eda090 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.621+0000 7f006ad3b740 1 -- 172.21.15.120:0/738627015 >> 172.21.15.120:0/738627015 conn(0x55cd20ea3310 msgr2=0x55cd20ed6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:32.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.621+0000 7f006ad3b740 1 -- 172.21.15.120:0/738627015 shutdown_connections 2024-09-19T09:05:32.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.621+0000 7f006ad3b740 1 -- 172.21.15.120:0/738627015 wait complete. 2024-09-19T09:05:32.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.622+0000 7f006ad3b740 1 -- start start 2024-09-19T09:05:32.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.622+0000 7f006ad3b740 1 Processor -- start 2024-09-19T09:05:32.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.622+0000 7f006ad3b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd20eda5d0 0x55cd20ee4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:32.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.622+0000 7f006ad3b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd20edbb40 0x55cd20ee2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:32.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.622+0000 7f006ad3b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd20ee5370 0x55cd20ee25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:32.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.622+0000 7f006ad3b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cd20ee8070 con 0x55cd20eda5d0 2024-09-19T09:05:32.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.622+0000 7f006ad3b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cd20dfb060 con 0x55cd20ee5370 2024-09-19T09:05:32.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.622+0000 7f006ad3b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cd20ee81f0 con 0x55cd20edbb40 2024-09-19T09:05:32.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.622+0000 7f006ad3b740 10 client.0 _ll_register_callbacks cb 0x55cd20b4e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:32.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.622+0000 7f0062d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd20eda5d0 0x55cd20ee4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:32.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.622+0000 7f0062d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd20eda5d0 0x55cd20ee4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58384/0 (socket says 172.21.15.120:58384) 2024-09-19T09:05:32.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.622+0000 7f0062d9d640 1 -- 172.21.15.120:0/4022245527 learned_addr learned my addr 172.21.15.120:0/4022245527 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:32.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.622+0000 7f0062d9d640 1 --2- 172.21.15.120:0/4022245527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd20ee5370 0x55cd20ee25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:32.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.622+0000 7f0061d9b640 1 --2- 172.21.15.120:0/4022245527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd20edbb40 0x55cd20ee2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:32.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.623+0000 7f0062d9d640 1 -- 172.21.15.120:0/4022245527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd20edbb40 msgr2=0x55cd20ee2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:32.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.623+0000 7f0062d9d640 1 --2- 172.21.15.120:0/4022245527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd20edbb40 0x55cd20ee2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.623+0000 7f0062d9d640 1 -- 172.21.15.120:0/4022245527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd20ee5370 msgr2=0x55cd20ee25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:32.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.623+0000 7f0062d9d640 1 --2- 172.21.15.120:0/4022245527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd20ee5370 0x55cd20ee25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.623+0000 7f0062d9d640 1 -- 172.21.15.120:0/4022245527 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd20bb13c0 con 0x55cd20eda5d0 2024-09-19T09:05:32.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.623+0000 7f0062d9d640 1 --2- 172.21.15.120:0/4022245527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd20eda5d0 0x55cd20ee4d50 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f005c009bf0 tx=0x7f005c009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f87898558d25f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:32.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.623+0000 7f0049ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:32.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.623+0000 7f0049ffb640 1 -- 172.21.15.120:0/4022245527 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c016070 con 0x55cd20eda5d0 2024-09-19T09:05:32.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.623+0000 7f0049ffb640 1 -- 172.21.15.120:0/4022245527 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f005c004450 con 0x55cd20eda5d0 2024-09-19T09:05:32.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.623+0000 7f006ad3b740 1 -- 172.21.15.120:0/4022245527 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f005c004d20 con 0x55cd20eda5d0 2024-09-19T09:05:32.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.623+0000 7f0049ffb640 1 -- 172.21.15.120:0/4022245527 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f005c004650 con 0x55cd20eda5d0 2024-09-19T09:05:32.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.623+0000 7f006ad3b740 10 client.16317 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:32.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.623+0000 7f006ad3b740 1 -- 172.21.15.120:0/4022245527 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cd20ef3af0 con 0x55cd20eda5d0 2024-09-19T09:05:32.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.623+0000 7f0049ffb640 1 -- 172.21.15.120:0/4022245527 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f005c004d20 con 0x55cd20eda5d0 2024-09-19T09:05:32.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.623+0000 7f0049ffb640 1 -- 172.21.15.120:0/4022245527 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f005c00f040 con 0x55cd20eda5d0 2024-09-19T09:05:32.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.624+0000 7f006ad3b740 10 client.16317 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:32.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.624+0000 7f006ad3b740 10 client.16317 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:05:32.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.624+0000 7f006ad3b740 20 client.16317 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:32.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.624+0000 7f006ad3b740 1 --2- 172.21.15.120:0/4022245527 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55cd20ef4e40 0x55cd20ef7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:32.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.624+0000 7f006ad3b740 4 client.16317 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:05:32.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.624+0000 7f006ad3b740 1 -- 172.21.15.120:0/4022245527 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd20e4d860 con 0x55cd20ef4e40 2024-09-19T09:05:32.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.624+0000 7f0061d9b640 1 --2- 172.21.15.120:0/4022245527 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55cd20ef4e40 0x55cd20ef7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:32.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.624+0000 7f0061d9b640 1 --2- 172.21.15.120:0/4022245527 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55cd20ef4e40 0x55cd20ef7230 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1a7fb4e3827b1118 server_cookie=531a2ca3ea6a2f3f in_seq=0 out_seq=0 2024-09-19T09:05:32.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.624+0000 7f0049ffb640 10 client.16317 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:05:32.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.625+0000 7f0049ffb640 1 -- 172.21.15.120:0/4022245527 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cd20e4d860 con 0x55cd20ef4e40 2024-09-19T09:05:32.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.625+0000 7f0049ffb640 10 client.16317 handle_command_reply: tid=0 2024-09-19T09:05:32.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.625+0000 7f006ad3b740 1 client.16317 shutdown 2024-09-19T09:05:32.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.625+0000 7f006ad3b740 1 -- 172.21.15.120:0/4022245527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd20eda5d0 msgr2=0x55cd20ee4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:32.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.625+0000 7f006ad3b740 1 --2- 172.21.15.120:0/4022245527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd20eda5d0 0x55cd20ee4d50 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f005c009bf0 tx=0x7f005c009c50 comp rx=0 tx=0).stop 2024-09-19T09:05:32.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.625+0000 7f006ad3b740 1 -- 172.21.15.120:0/4022245527 shutdown_connections 2024-09-19T09:05:32.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.625+0000 7f006ad3b740 1 --2- 172.21.15.120:0/4022245527 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55cd20ef4e40 0x55cd20ef7230 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.625+0000 7f006ad3b740 1 --2- 172.21.15.120:0/4022245527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd20ee5370 0x55cd20ee25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.625+0000 7f006ad3b740 1 --2- 172.21.15.120:0/4022245527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd20edbb40 0x55cd20ee2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.625+0000 7f0049ffb640 0 client.16317 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:05:32.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.625+0000 7f006ad3b740 1 --2- 172.21.15.120:0/4022245527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd20eda5d0 0x55cd20ee4d50 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.625+0000 7f006ad3b740 1 -- 172.21.15.120:0/4022245527 >> 172.21.15.120:0/4022245527 conn(0x55cd20ea3310 msgr2=0x55cd20edc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:32.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.625+0000 7f006ad3b740 1 -- 172.21.15.120:0/4022245527 shutdown_connections 2024-09-19T09:05:32.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.625+0000 7f006ad3b740 1 -- 172.21.15.120:0/4022245527 wait complete. 2024-09-19T09:05:32.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.625+0000 7f006ad3b740 20 client.16317 trim_cache size 0 max 16384 2024-09-19T09:05:32.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.639+0000 7f006ad3b740 1 Processor -- start 2024-09-19T09:05:32.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.639+0000 7f006ad3b740 1 -- start start 2024-09-19T09:05:32.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.639+0000 7f006ad3b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd20eda5d0 0x55cd20ef1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:32.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.639+0000 7f006ad3b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd20edbb40 0x55cd20efc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:32.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.639+0000 7f006ad3b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd20ee5370 0x55cd20efceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:32.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.639+0000 7f006ad3b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f00641ab3e0 con 0x55cd20edbb40 2024-09-19T09:05:32.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.639+0000 7f006ad3b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f00641acf20 con 0x55cd20eda5d0 2024-09-19T09:05:32.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.639+0000 7f006ad3b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f00641abab0 con 0x55cd20ee5370 2024-09-19T09:05:32.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.639+0000 7f0061d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd20edbb40 0x55cd20efc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:32.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.639+0000 7f0061d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd20edbb40 0x55cd20efc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58398/0 (socket says 172.21.15.120:58398) 2024-09-19T09:05:32.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.639+0000 7f0061d9b640 1 -- 172.21.15.120:0/2382741040 learned_addr learned my addr 172.21.15.120:0/2382741040 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:32.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.640+0000 7f0062d9d640 1 --2- 172.21.15.120:0/2382741040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd20eda5d0 0x55cd20ef1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:32.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.640+0000 7f0062d9d640 1 --2- 172.21.15.120:0/2382741040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd20ee5370 0x55cd20efceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:32.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.640+0000 7f0061d9b640 1 -- 172.21.15.120:0/2382741040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd20ee5370 msgr2=0x55cd20efceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:32.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.640+0000 7f0061d9b640 1 --2- 172.21.15.120:0/2382741040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd20ee5370 0x55cd20efceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.640+0000 7f0061d9b640 1 -- 172.21.15.120:0/2382741040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd20eda5d0 msgr2=0x55cd20ef1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:32.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.640+0000 7f0061d9b640 1 --2- 172.21.15.120:0/2382741040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd20eda5d0 0x55cd20ef1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.640+0000 7f0061d9b640 1 -- 172.21.15.120:0/2382741040 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd20ee4870 con 0x55cd20edbb40 2024-09-19T09:05:32.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.640+0000 7f0061d9b640 1 --2- 172.21.15.120:0/2382741040 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd20edbb40 0x55cd20efc970 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f0058013c10 tx=0x7f0058005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4202b649f811d9b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:32.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.640+0000 7f004bfff640 1 -- 172.21.15.120:0/2382741040 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0058005020 con 0x55cd20edbb40 2024-09-19T09:05:32.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.640+0000 7f004bfff640 1 -- 172.21.15.120:0/2382741040 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00580047a0 con 0x55cd20edbb40 2024-09-19T09:05:32.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.640+0000 7f004bfff640 1 -- 172.21.15.120:0/2382741040 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0058003830 con 0x55cd20edbb40 2024-09-19T09:05:32.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.641+0000 7f006ad3b740 1 -- 172.21.15.120:0/2382741040 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd20edbb40 msgr2=0x55cd20efc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:32.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.641+0000 7f006ad3b740 1 --2- 172.21.15.120:0/2382741040 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd20edbb40 0x55cd20efc970 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f0058013c10 tx=0x7f0058005f90 comp rx=0 tx=0).stop 2024-09-19T09:05:32.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.641+0000 7f006ad3b740 1 -- 172.21.15.120:0/2382741040 shutdown_connections 2024-09-19T09:05:32.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.641+0000 7f006ad3b740 1 --2- 172.21.15.120:0/2382741040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd20ee5370 0x55cd20efceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.641+0000 7f006ad3b740 1 --2- 172.21.15.120:0/2382741040 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd20edbb40 0x55cd20efc970 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.641+0000 7f006ad3b740 1 --2- 172.21.15.120:0/2382741040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd20eda5d0 0x55cd20ef1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.641+0000 7f006ad3b740 1 -- 172.21.15.120:0/2382741040 >> 172.21.15.120:0/2382741040 conn(0x55cd20ea3310 msgr2=0x55cd20ee10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:32.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.641+0000 7f006ad3b740 1 -- 172.21.15.120:0/2382741040 shutdown_connections 2024-09-19T09:05:32.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.641+0000 7f006ad3b740 1 -- 172.21.15.120:0/2382741040 wait complete. 2024-09-19T09:05:32.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.641+0000 7f006ad3b740 1 -- start start 2024-09-19T09:05:32.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.642+0000 7f006ad3b740 1 Processor -- start 2024-09-19T09:05:32.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.642+0000 7f006ad3b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd20eda5d0 0x55cd20ee93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:32.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.642+0000 7f006ad3b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd20edbb40 0x55cd20ee9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:32.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.642+0000 7f006ad3b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd20ee5370 0x55cd20ee9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:32.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.642+0000 7f006ad3b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f00641ac850 con 0x55cd20edbb40 2024-09-19T09:05:32.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.642+0000 7f006ad3b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f00641ac180 con 0x55cd20eda5d0 2024-09-19T09:05:32.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.642+0000 7f006ad3b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f00641a93a0 con 0x55cd20ee5370 2024-09-19T09:05:32.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.642+0000 7f006ad3b740 10 client.0 _ll_register_callbacks cb 0x7f00580137d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:32.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.642+0000 7f0061d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd20edbb40 0x55cd20ee9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:32.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.642+0000 7f0061d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd20edbb40 0x55cd20ee9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58400/0 (socket says 172.21.15.120:58400) 2024-09-19T09:05:32.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.642+0000 7f0061d9b640 1 -- 172.21.15.120:0/315343805 learned_addr learned my addr 172.21.15.120:0/315343805 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:32.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.642+0000 7f0062d9d640 1 --2- 172.21.15.120:0/315343805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd20eda5d0 0x55cd20ee93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:32.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.642+0000 7f0062d9d640 1 --2- 172.21.15.120:0/315343805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd20ee5370 0x55cd20ee9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:32.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.642+0000 7f0061d9b640 1 -- 172.21.15.120:0/315343805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd20ee5370 msgr2=0x55cd20ee9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:32.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.642+0000 7f0061d9b640 1 --2- 172.21.15.120:0/315343805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd20ee5370 0x55cd20ee9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.642+0000 7f0061d9b640 1 -- 172.21.15.120:0/315343805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd20eda5d0 msgr2=0x55cd20ee93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:32.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.642+0000 7f0061d9b640 1 --2- 172.21.15.120:0/315343805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd20eda5d0 0x55cd20ee93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.642+0000 7f0061d9b640 1 -- 172.21.15.120:0/315343805 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd20f5eb40 con 0x55cd20edbb40 2024-09-19T09:05:32.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.642+0000 7f0061d9b640 1 --2- 172.21.15.120:0/315343805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd20edbb40 0x55cd20ee9930 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0058000c00 tx=0x7f0058004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c597442e85a0dc1c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:32.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.642+0000 7f0049ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:32.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.643+0000 7f0049ffb640 1 -- 172.21.15.120:0/315343805 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0058005230 con 0x55cd20edbb40 2024-09-19T09:05:32.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.643+0000 7f0049ffb640 1 -- 172.21.15.120:0/315343805 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f00580053d0 con 0x55cd20edbb40 2024-09-19T09:05:32.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.643+0000 7f006ad3b740 1 -- 172.21.15.120:0/315343805 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00580047a0 con 0x55cd20edbb40 2024-09-19T09:05:32.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.643+0000 7f0049ffb640 1 -- 172.21.15.120:0/315343805 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0058004950 con 0x55cd20edbb40 2024-09-19T09:05:32.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.643+0000 7f006ad3b740 10 client.16323 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:32.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.643+0000 7f006ad3b740 1 -- 172.21.15.120:0/315343805 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cd20f5fde0 con 0x55cd20edbb40 2024-09-19T09:05:32.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.643+0000 7f0049ffb640 1 -- 172.21.15.120:0/315343805 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00580047a0 con 0x55cd20edbb40 2024-09-19T09:05:32.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.643+0000 7f0049ffb640 1 -- 172.21.15.120:0/315343805 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f00580364d0 con 0x55cd20edbb40 2024-09-19T09:05:32.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.643+0000 7f006ad3b740 10 client.16323 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:32.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.643+0000 7f006ad3b740 10 client.16323 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:05:32.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.643+0000 7f006ad3b740 20 client.16323 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:32.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.643+0000 7f006ad3b740 1 --2- 172.21.15.120:0/315343805 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55cd20f603f0 0x55cd20f627e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:32.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.643+0000 7f006ad3b740 4 client.16323 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:05:32.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.643+0000 7f006ad3b740 1 -- 172.21.15.120:0/315343805 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cd20e47d70 con 0x55cd20f603f0 2024-09-19T09:05:32.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.643+0000 7f0062d9d640 1 --2- 172.21.15.120:0/315343805 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55cd20f603f0 0x55cd20f627e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:32.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.644+0000 7f0049ffb640 10 client.16323 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:05:32.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.644+0000 7f0062d9d640 1 --2- 172.21.15.120:0/315343805 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55cd20f603f0 0x55cd20f627e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3a4b7ab8799ea1ed server_cookie=f0a441cf9d2bfd74 in_seq=0 out_seq=0 2024-09-19T09:05:32.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.644+0000 7f0049ffb640 1 -- 172.21.15.120:0/315343805 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cd20e47d70 con 0x55cd20f603f0 2024-09-19T09:05:32.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.644+0000 7f0049ffb640 10 client.16323 handle_command_reply: tid=0 2024-09-19T09:05:32.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.644+0000 7f006ad3b740 1 client.16323 shutdown 2024-09-19T09:05:32.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.644+0000 7f006ad3b740 1 -- 172.21.15.120:0/315343805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd20edbb40 msgr2=0x55cd20ee9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:32.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.644+0000 7f006ad3b740 1 --2- 172.21.15.120:0/315343805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd20edbb40 0x55cd20ee9930 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0058000c00 tx=0x7f0058004040 comp rx=0 tx=0).stop 2024-09-19T09:05:32.665 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:05:32.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.644+0000 7f006ad3b740 1 -- 172.21.15.120:0/315343805 shutdown_connections 2024-09-19T09:05:32.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.644+0000 7f006ad3b740 1 --2- 172.21.15.120:0/315343805 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55cd20f603f0 0x55cd20f627e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.644+0000 7f006ad3b740 1 --2- 172.21.15.120:0/315343805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd20ee5370 0x55cd20ee9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.644+0000 7f006ad3b740 1 --2- 172.21.15.120:0/315343805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd20edbb40 0x55cd20ee9930 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.644+0000 7f0049ffb640 0 client.16323 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:05:32.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.644+0000 7f006ad3b740 1 --2- 172.21.15.120:0/315343805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd20eda5d0 0x55cd20ee93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.644+0000 7f006ad3b740 1 -- 172.21.15.120:0/315343805 >> 172.21.15.120:0/315343805 conn(0x55cd20ea3310 msgr2=0x55cd20ee3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:32.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.644+0000 7f006ad3b740 1 -- 172.21.15.120:0/315343805 shutdown_connections 2024-09-19T09:05:32.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.644+0000 7f006ad3b740 1 -- 172.21.15.120:0/315343805 wait complete. 2024-09-19T09:05:32.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.644+0000 7f006ad3b740 20 client.16323 trim_cache size 0 max 16384 2024-09-19T09:05:32.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.645+0000 7f0068962640 1 -- 172.21.15.120:0/2528243686 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f004c0a32c0 msgr2=0x7f004c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:32.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.645+0000 7f0068962640 1 --2- 172.21.15.120:0/2528243686 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f004c0a32c0 0x7f004c0a5780 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f00540c4c50 tx=0x7f005402a480 comp rx=0 tx=0).stop 2024-09-19T09:05:32.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.645+0000 7f0068962640 1 -- 172.21.15.120:0/2528243686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0064100e20 msgr2=0x7f00641984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:32.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.645+0000 7f0068962640 1 --2- 172.21.15.120:0/2528243686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0064100e20 0x7f00641984c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0054006010 tx=0x7f0054003c10 comp rx=0 tx=0).stop 2024-09-19T09:05:32.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.645+0000 7f0068962640 1 -- 172.21.15.120:0/2528243686 shutdown_connections 2024-09-19T09:05:32.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.645+0000 7f0068962640 1 --2- 172.21.15.120:0/2528243686 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f004c0a32c0 0x7f004c0a5780 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.645+0000 7f0068962640 1 --2- 172.21.15.120:0/2528243686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0064102ef0 0x7f006419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.645+0000 7f0068962640 1 --2- 172.21.15.120:0/2528243686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0064102570 0x7f0064198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.645+0000 7f0068962640 1 --2- 172.21.15.120:0/2528243686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0064100e20 0x7f00641984c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:32.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.645+0000 7f0068962640 1 -- 172.21.15.120:0/2528243686 >> 172.21.15.120:0/2528243686 conn(0x7f00640fc7d0 msgr2=0x7f0064105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:32.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.645+0000 7f0068962640 1 -- 172.21.15.120:0/2528243686 shutdown_connections 2024-09-19T09:05:32.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:32.645+0000 7f0068962640 1 -- 172.21.15.120:0/2528243686 wait complete. 2024-09-19T09:05:32.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:32 smithi120 ceph-mon[31150]: pgmap v421: 97 pgs: 97 active+clean; 38 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 364 op/s 2024-09-19T09:05:33.040 INFO:tasks.workunit.client.0.smithi120.stdout: 1 138710 8.71 MB/sec warmup 88 sec latency 14.408 ms 2024-09-19T09:05:33.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:32 smithi196 ceph-mon[35150]: pgmap v421: 97 pgs: 97 active+clean; 38 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 364 op/s 2024-09-19T09:05:33.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:32 smithi153 ceph-mon[35071]: pgmap v421: 97 pgs: 97 active+clean; 38 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 364 op/s 2024-09-19T09:05:33.662 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:05:33.926 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:05:34.039 INFO:tasks.workunit.client.0.smithi120.stdout: 1 141535 8.76 MB/sec warmup 89 sec latency 53.966 ms 2024-09-19T09:05:34.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.455+0000 7fafe1a89640 1 -- 172.21.15.120:0/453299646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafdc102ef0 msgr2=0x7fafdc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:34.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.455+0000 7fafe1a89640 1 --2- 172.21.15.120:0/453299646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafdc102ef0 0x7fafdc10da40 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fafd4009ca0 tx=0x7fafd402a950 comp rx=0 tx=0).stop 2024-09-19T09:05:34.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.455+0000 7fafe1a89640 1 -- 172.21.15.120:0/453299646 shutdown_connections 2024-09-19T09:05:34.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.455+0000 7fafe1a89640 1 --2- 172.21.15.120:0/453299646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafdc102ef0 0x7fafdc10da40 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.455+0000 7fafe1a89640 1 --2- 172.21.15.120:0/453299646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafdc102570 0x7fafdc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.455+0000 7fafe1a89640 1 --2- 172.21.15.120:0/453299646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafdc100e20 0x7fafdc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.455+0000 7fafe1a89640 1 -- 172.21.15.120:0/453299646 >> 172.21.15.120:0/453299646 conn(0x7fafdc0fc7d0 msgr2=0x7fafdc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:34.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.455+0000 7fafe1a89640 1 -- 172.21.15.120:0/453299646 shutdown_connections 2024-09-19T09:05:34.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.455+0000 7fafe1a89640 1 -- 172.21.15.120:0/453299646 wait complete. 2024-09-19T09:05:34.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.456+0000 7fafe1a89640 1 Processor -- start 2024-09-19T09:05:34.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.456+0000 7fafe1a89640 1 -- start start 2024-09-19T09:05:34.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.456+0000 7fafe1a89640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafdc100e20 0x7fafdc198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:34.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.456+0000 7fafe1a89640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafdc102570 0x7fafdc198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:34.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.456+0000 7fafe1a89640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafdc102ef0 0x7fafdc19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:34.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.456+0000 7fafe1a89640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc110f80 con 0x7fafdc102ef0 2024-09-19T09:05:34.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.456+0000 7fafe1a89640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc110e00 con 0x7fafdc102570 2024-09-19T09:05:34.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.456+0000 7fafe1a89640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc111100 con 0x7fafdc100e20 2024-09-19T09:05:34.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.457+0000 7fafdbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafdc102ef0 0x7fafdc19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:34.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.457+0000 7fafdb7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafdc100e20 0x7fafdc198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:34.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.457+0000 7fafdbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafdc102ef0 0x7fafdc19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58430/0 (socket says 172.21.15.120:58430) 2024-09-19T09:05:34.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.457+0000 7fafdbfff640 1 -- 172.21.15.120:0/1621491818 learned_addr learned my addr 172.21.15.120:0/1621491818 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:34.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.457+0000 7fafdb7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafdc100e20 0x7fafdc198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:45056/0 (socket says 172.21.15.120:45056) 2024-09-19T09:05:34.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.457+0000 7fafdaffd640 1 --2- 172.21.15.120:0/1621491818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafdc102570 0x7fafdc198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:34.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.457+0000 7fafdbfff640 1 -- 172.21.15.120:0/1621491818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafdc100e20 msgr2=0x7fafdc198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:34.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.457+0000 7fafdbfff640 1 --2- 172.21.15.120:0/1621491818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafdc100e20 0x7fafdc198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.457+0000 7fafdbfff640 1 -- 172.21.15.120:0/1621491818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafdc102570 msgr2=0x7fafdc198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:34.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.457+0000 7fafdbfff640 1 --2- 172.21.15.120:0/1621491818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafdc102570 0x7fafdc198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.457+0000 7fafdbfff640 1 -- 172.21.15.120:0/1621491818 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafdc19d400 con 0x7fafdc102ef0 2024-09-19T09:05:34.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.457+0000 7fafdbfff640 1 --2- 172.21.15.120:0/1621491818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafdc102ef0 0x7fafdc19cd20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fafd4004af0 tx=0x7fafd402f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:34.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.457+0000 7fafd8ff9640 1 -- 172.21.15.120:0/1621491818 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd4034040 con 0x7fafdc102ef0 2024-09-19T09:05:34.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.457+0000 7fafe1a89640 1 -- 172.21.15.120:0/1621491818 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafdc19d690 con 0x7fafdc102ef0 2024-09-19T09:05:34.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.457+0000 7fafd8ff9640 1 -- 172.21.15.120:0/1621491818 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafd4030920 con 0x7fafdc102ef0 2024-09-19T09:05:34.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.457+0000 7fafd8ff9640 1 -- 172.21.15.120:0/1621491818 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd4004820 con 0x7fafdc102ef0 2024-09-19T09:05:34.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.457+0000 7fafe1a89640 1 -- 172.21.15.120:0/1621491818 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafdc1a4fd0 con 0x7fafdc102ef0 2024-09-19T09:05:34.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.460+0000 7fafd8ff9640 1 -- 172.21.15.120:0/1621491818 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fafd4037020 con 0x7fafdc102ef0 2024-09-19T09:05:34.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.460+0000 7fafe3e62740 1 Processor -- start 2024-09-19T09:05:34.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.460+0000 7fafe3e62740 1 -- start start 2024-09-19T09:05:34.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.460+0000 7fafd8ff9640 1 --2- 172.21.15.120:0/1621491818 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fafc80a3240 0x7fafc80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:34.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.460+0000 7fafe3e62740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f8709fb40 0x559f8709e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:34.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.460+0000 7fafe3e62740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f8709e5d0 0x559f8709ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:34.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.460+0000 7fafe3e62740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f870a9370 0x559f870ab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:34.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.460+0000 7fafe3e62740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559f870042f0 con 0x559f870a9370 2024-09-19T09:05:34.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.460+0000 7fafe3e62740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559f86fabcf0 con 0x559f8709fb40 2024-09-19T09:05:34.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.460+0000 7fafe3e62740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559f870002b0 con 0x559f8709e5d0 2024-09-19T09:05:34.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.460+0000 7fafdaffd640 1 --2- 172.21.15.120:0/1621491818 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fafc80a3240 0x7fafc80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:34.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.460+0000 7fafd8ff9640 1 -- 172.21.15.120:0/1621491818 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fafd40f0730 con 0x7fafdc102ef0 2024-09-19T09:05:34.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.461+0000 7fafdb7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f8709fb40 0x559f8709e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:34.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.461+0000 7fafdaffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f8709e5d0 0x559f8709ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:34.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.461+0000 7fafdb7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f8709fb40 0x559f8709e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:55042/0 (socket says 172.21.15.120:55042) 2024-09-19T09:05:34.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.461+0000 7fafdb7fe640 1 -- 172.21.15.120:0/2664955463 learned_addr learned my addr 172.21.15.120:0/2664955463 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:34.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.461+0000 7fafdbfff640 1 --2- 172.21.15.120:0/2664955463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f870a9370 0x559f870ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:34.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.461+0000 7fafdb7fe640 1 -- 172.21.15.120:0/2664955463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f8709e5d0 msgr2=0x559f8709ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:34.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.461+0000 7fafdb7fe640 1 --2- 172.21.15.120:0/2664955463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f8709e5d0 0x559f8709ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.461+0000 7fafdb7fe640 1 -- 172.21.15.120:0/2664955463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f870a9370 msgr2=0x559f870ab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:34.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.461+0000 7fafdb7fe640 1 --2- 172.21.15.120:0/2664955463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f870a9370 0x559f870ab760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.461+0000 7fafdb7fe640 1 -- 172.21.15.120:0/2664955463 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f86f903a0 con 0x559f8709fb40 2024-09-19T09:05:34.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.461+0000 7fafdaffd640 1 --2- 172.21.15.120:0/1621491818 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fafc80a3240 0x7fafc80a5700 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fafdc0ff0e0 tx=0x7fafc400a760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:34.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.461+0000 7fafdb7fe640 1 --2- 172.21.15.120:0/2664955463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f8709fb40 0x559f8709e090 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009cf0 tx=0x7fafcc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=726142a455392e9f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:34.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.461+0000 7fafd1ffb640 1 -- 172.21.15.120:0/2664955463 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc01e070 con 0x559f8709fb40 2024-09-19T09:05:34.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.461+0000 7fafd1ffb640 1 -- 172.21.15.120:0/2664955463 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafcc009de0 con 0x559f8709fb40 2024-09-19T09:05:34.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.461+0000 7fafd1ffb640 1 -- 172.21.15.120:0/2664955463 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc0196b0 con 0x559f8709fb40 2024-09-19T09:05:34.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.462+0000 7fafe3e62740 1 -- 172.21.15.120:0/2664955463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f8709fb40 msgr2=0x559f8709e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:34.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.462+0000 7fafe3e62740 1 --2- 172.21.15.120:0/2664955463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f8709fb40 0x559f8709e090 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009cf0 tx=0x7fafcc011810 comp rx=0 tx=0).stop 2024-09-19T09:05:34.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.462+0000 7fafe3e62740 1 -- 172.21.15.120:0/2664955463 shutdown_connections 2024-09-19T09:05:34.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.462+0000 7fafe3e62740 1 --2- 172.21.15.120:0/2664955463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f870a9370 0x559f870ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.462+0000 7fafe3e62740 1 --2- 172.21.15.120:0/2664955463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f8709e5d0 0x559f8709ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.462+0000 7fafe3e62740 1 --2- 172.21.15.120:0/2664955463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f8709fb40 0x559f8709e090 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.462+0000 7fafe3e62740 1 -- 172.21.15.120:0/2664955463 >> 172.21.15.120:0/2664955463 conn(0x559f87067310 msgr2=0x559f8709ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:34.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.462+0000 7fafe3e62740 1 -- 172.21.15.120:0/2664955463 shutdown_connections 2024-09-19T09:05:34.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.462+0000 7fafe3e62740 1 -- 172.21.15.120:0/2664955463 wait complete. 2024-09-19T09:05:34.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.463+0000 7fafe3e62740 1 -- start start 2024-09-19T09:05:34.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.463+0000 7fafe3e62740 1 Processor -- start 2024-09-19T09:05:34.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.463+0000 7fafe3e62740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f8709e5d0 0x559f870a8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:34.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.463+0000 7fafe3e62740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f8709fb40 0x559f870a5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:34.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.463+0000 7fafe3e62740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f870a9370 0x559f870a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:34.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.463+0000 7fafe3e62740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559f870ac070 con 0x559f8709e5d0 2024-09-19T09:05:34.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.463+0000 7fafe3e62740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559f86fbf060 con 0x559f870a9370 2024-09-19T09:05:34.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.463+0000 7fafe3e62740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559f870ac1f0 con 0x559f8709fb40 2024-09-19T09:05:34.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.463+0000 7fafe3e62740 10 client.0 _ll_register_callbacks cb 0x559f86d128c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:34.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.463+0000 7fafdb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f8709e5d0 0x559f870a8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:34.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.463+0000 7fafdbfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f870a9370 0x559f870a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:34.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.463+0000 7fafdb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f8709e5d0 0x559f870a8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58450/0 (socket says 172.21.15.120:58450) 2024-09-19T09:05:34.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.463+0000 7fafdb7fe640 1 -- 172.21.15.120:0/2648078541 learned_addr learned my addr 172.21.15.120:0/2648078541 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:34.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.463+0000 7fafdaffd640 1 --2- 172.21.15.120:0/2648078541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f8709fb40 0x559f870a5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:34.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.464+0000 7fafdb7fe640 1 -- 172.21.15.120:0/2648078541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f8709fb40 msgr2=0x559f870a5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:34.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.464+0000 7fafdb7fe640 1 --2- 172.21.15.120:0/2648078541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f8709fb40 0x559f870a5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.464+0000 7fafdb7fe640 1 -- 172.21.15.120:0/2648078541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f870a9370 msgr2=0x559f870a64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:34.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.464+0000 7fafdb7fe640 1 --2- 172.21.15.120:0/2648078541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f870a9370 0x559f870a64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.464+0000 7fafdb7fe640 1 -- 172.21.15.120:0/2648078541 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f86d753c0 con 0x559f8709e5d0 2024-09-19T09:05:34.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.464+0000 7fafdaffd640 1 --2- 172.21.15.120:0/2648078541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f8709fb40 0x559f870a5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:34.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.464+0000 7fafdbfff640 1 --2- 172.21.15.120:0/2648078541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f870a9370 0x559f870a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:34.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.464+0000 7fafdb7fe640 1 --2- 172.21.15.120:0/2648078541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f8709e5d0 0x559f870a8ce0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fafcc003d30 tx=0x7fafcc0052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee10462b24d7323 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:34.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.464+0000 7fafbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:34.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.464+0000 7fafbaffd640 1 -- 172.21.15.120:0/2648078541 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc01e040 con 0x559f8709e5d0 2024-09-19T09:05:34.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.464+0000 7fafbaffd640 1 -- 172.21.15.120:0/2648078541 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fafcc016070 con 0x559f8709e5d0 2024-09-19T09:05:34.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.464+0000 7fafe3e62740 1 -- 172.21.15.120:0/2648078541 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafcc009de0 con 0x559f8709e5d0 2024-09-19T09:05:34.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.464+0000 7fafbaffd640 1 -- 172.21.15.120:0/2648078541 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fafcc0054e0 con 0x559f8709e5d0 2024-09-19T09:05:34.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.464+0000 7fafe3e62740 10 client.16335 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:34.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.464+0000 7fafe3e62740 1 -- 172.21.15.120:0/2648078541 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559f870b7a80 con 0x559f8709e5d0 2024-09-19T09:05:34.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.464+0000 7fafbaffd640 1 -- 172.21.15.120:0/2648078541 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafcc009de0 con 0x559f8709e5d0 2024-09-19T09:05:34.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.465+0000 7fafbaffd640 1 -- 172.21.15.120:0/2648078541 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fafcc02b030 con 0x559f8709e5d0 2024-09-19T09:05:34.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.465+0000 7fafe3e62740 10 client.16335 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:34.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.465+0000 7fafe3e62740 10 client.16335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:34.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.465+0000 7fafe3e62740 20 client.16335 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:34.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.465+0000 7fafe3e62740 1 --2- 172.21.15.120:0/2648078541 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559f870b8ee0 0x559f870bb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:34.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.465+0000 7fafe3e62740 4 client.16335 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:05:34.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.465+0000 7fafe3e62740 1 -- 172.21.15.120:0/2648078541 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f87011860 con 0x559f870b8ee0 2024-09-19T09:05:34.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.465+0000 7fafdaffd640 1 --2- 172.21.15.120:0/2648078541 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559f870b8ee0 0x559f870bb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:34.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.465+0000 7fafbaffd640 10 client.16335 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:34.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.465+0000 7fafdaffd640 1 --2- 172.21.15.120:0/2648078541 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559f870b8ee0 0x559f870bb2d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99e4b7c506295758 server_cookie=c57adebcf58274b0 in_seq=0 out_seq=0 2024-09-19T09:05:34.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.466+0000 7fafbaffd640 1 -- 172.21.15.120:0/2648078541 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559f87011860 con 0x559f870b8ee0 2024-09-19T09:05:34.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.466+0000 7fafbaffd640 10 client.16335 handle_command_reply: tid=0 2024-09-19T09:05:34.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.466+0000 7fafe3e62740 1 client.16335 shutdown 2024-09-19T09:05:34.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.466+0000 7fafe3e62740 1 -- 172.21.15.120:0/2648078541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f8709e5d0 msgr2=0x559f870a8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:34.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.466+0000 7fafe3e62740 1 --2- 172.21.15.120:0/2648078541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f8709e5d0 0x559f870a8ce0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fafcc003d30 tx=0x7fafcc0052f0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.466+0000 7fafe3e62740 1 -- 172.21.15.120:0/2648078541 shutdown_connections 2024-09-19T09:05:34.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.466+0000 7fafe3e62740 1 --2- 172.21.15.120:0/2648078541 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559f870b8ee0 0x559f870bb2d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.466+0000 7fafe3e62740 1 --2- 172.21.15.120:0/2648078541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f870a9370 0x559f870a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.466+0000 7fafe3e62740 1 --2- 172.21.15.120:0/2648078541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f8709fb40 0x559f870a5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.466+0000 7fafe3e62740 1 --2- 172.21.15.120:0/2648078541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f8709e5d0 0x559f870a8ce0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.466+0000 7fafe3e62740 1 -- 172.21.15.120:0/2648078541 >> 172.21.15.120:0/2648078541 conn(0x559f87067310 msgr2=0x559f870a0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:34.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.466+0000 7fafbaffd640 0 client.16335 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:34.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.466+0000 7fafe3e62740 1 -- 172.21.15.120:0/2648078541 shutdown_connections 2024-09-19T09:05:34.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.466+0000 7fafe3e62740 1 -- 172.21.15.120:0/2648078541 wait complete. 2024-09-19T09:05:34.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.466+0000 7fafe3e62740 20 client.16335 trim_cache size 0 max 16384 2024-09-19T09:05:34.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.480+0000 7fafe3e62740 1 Processor -- start 2024-09-19T09:05:34.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.480+0000 7fafe3e62740 1 -- start start 2024-09-19T09:05:34.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.480+0000 7fafe3e62740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f8709e5d0 0x559f870b5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:34.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.480+0000 7fafe3e62740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f8709fb40 0x559f870a5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:34.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.480+0000 7fafe3e62740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f870a9370 0x559f870a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:34.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.480+0000 7fafe3e62740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc1ad4c0 con 0x559f870a9370 2024-09-19T09:05:34.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.480+0000 7fafe3e62740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc1af000 con 0x559f8709e5d0 2024-09-19T09:05:34.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.480+0000 7fafe3e62740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc1adb90 con 0x559f8709fb40 2024-09-19T09:05:34.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.480+0000 7fafdbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f870a9370 0x559f870a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:34.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.480+0000 7fafdb7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f8709e5d0 0x559f870b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:34.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.480+0000 7fafdbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f870a9370 0x559f870a4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58460/0 (socket says 172.21.15.120:58460) 2024-09-19T09:05:34.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.480+0000 7fafdbfff640 1 -- 172.21.15.120:0/1939698595 learned_addr learned my addr 172.21.15.120:0/1939698595 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:34.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.481+0000 7fafdaffd640 1 --2- 172.21.15.120:0/1939698595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f8709fb40 0x559f870a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:34.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.481+0000 7fafdbfff640 1 -- 172.21.15.120:0/1939698595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f8709fb40 msgr2=0x559f870a5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:34.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.481+0000 7fafdbfff640 1 --2- 172.21.15.120:0/1939698595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f8709fb40 0x559f870a5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.481+0000 7fafdbfff640 1 -- 172.21.15.120:0/1939698595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f8709e5d0 msgr2=0x559f870b5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:34.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.481+0000 7fafdbfff640 1 --2- 172.21.15.120:0/1939698595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f8709e5d0 0x559f870b5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.481+0000 7fafdbfff640 1 -- 172.21.15.120:0/1939698595 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f870a8900 con 0x559f870a9370 2024-09-19T09:05:34.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.481+0000 7fafdb7fe640 1 --2- 172.21.15.120:0/1939698595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f8709e5d0 0x559f870b5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:34.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.481+0000 7fafdbfff640 1 --2- 172.21.15.120:0/1939698595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f870a9370 0x559f870a4f70 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fafd4003f50 tx=0x7fafd40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394497c5a77eef47 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:34.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.481+0000 7fafd17fa640 1 -- 172.21.15.120:0/1939698595 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd402c040 con 0x559f870a9370 2024-09-19T09:05:34.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.481+0000 7fafd17fa640 1 -- 172.21.15.120:0/1939698595 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafd40098d0 con 0x559f870a9370 2024-09-19T09:05:34.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.481+0000 7fafd17fa640 1 -- 172.21.15.120:0/1939698595 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd40c5530 con 0x559f870a9370 2024-09-19T09:05:34.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.482+0000 7fafe3e62740 1 -- 172.21.15.120:0/1939698595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f870a9370 msgr2=0x559f870a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:34.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.482+0000 7fafe3e62740 1 --2- 172.21.15.120:0/1939698595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f870a9370 0x559f870a4f70 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fafd4003f50 tx=0x7fafd40095c0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.482+0000 7fafe3e62740 1 -- 172.21.15.120:0/1939698595 shutdown_connections 2024-09-19T09:05:34.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.482+0000 7fafe3e62740 1 --2- 172.21.15.120:0/1939698595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f870a9370 0x559f870a4f70 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.482+0000 7fafe3e62740 1 --2- 172.21.15.120:0/1939698595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f8709fb40 0x559f870a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.482+0000 7fafe3e62740 1 --2- 172.21.15.120:0/1939698595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f8709e5d0 0x559f870b5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.482+0000 7fafe3e62740 1 -- 172.21.15.120:0/1939698595 >> 172.21.15.120:0/1939698595 conn(0x559f87067310 msgr2=0x559f870a4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:34.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.482+0000 7fafe3e62740 1 -- 172.21.15.120:0/1939698595 shutdown_connections 2024-09-19T09:05:34.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.482+0000 7fafe3e62740 1 -- 172.21.15.120:0/1939698595 wait complete. 2024-09-19T09:05:34.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.482+0000 7fafe3e62740 1 -- start start 2024-09-19T09:05:34.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.482+0000 7fafe3e62740 1 Processor -- start 2024-09-19T09:05:34.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.483+0000 7fafe3e62740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f8709e5d0 0x559f870b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:34.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.483+0000 7fafe3e62740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f8709fb40 0x559f870a2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:34.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.483+0000 7fafe3e62740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f870a9370 0x559f870a2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:34.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.483+0000 7fafe3e62740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc1ae930 con 0x559f8709fb40 2024-09-19T09:05:34.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.483+0000 7fafe3e62740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc1ae260 con 0x559f8709e5d0 2024-09-19T09:05:34.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.483+0000 7fafe3e62740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc1ab480 con 0x559f870a9370 2024-09-19T09:05:34.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.483+0000 7fafe3e62740 10 client.0 _ll_register_callbacks cb 0x7fafcc01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:34.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.483+0000 7fafdaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f8709fb40 0x559f870a2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:34.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.483+0000 7fafdb7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f8709e5d0 0x559f870b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:34.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.483+0000 7fafdb7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f8709e5d0 0x559f870b1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:55084/0 (socket says 172.21.15.120:55084) 2024-09-19T09:05:34.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.483+0000 7fafdaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f8709fb40 0x559f870a2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58476/0 (socket says 172.21.15.120:58476) 2024-09-19T09:05:34.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.483+0000 7fafdb7fe640 1 -- 172.21.15.120:0/28688416 learned_addr learned my addr 172.21.15.120:0/28688416 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:34.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.483+0000 7fafdbfff640 1 --2- 172.21.15.120:0/28688416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f870a9370 0x559f870a2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:34.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.483+0000 7fafdb7fe640 1 -- 172.21.15.120:0/28688416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f870a9370 msgr2=0x559f870a2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:34.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.483+0000 7fafdb7fe640 1 --2- 172.21.15.120:0/28688416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f870a9370 0x559f870a2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.483+0000 7fafdb7fe640 1 -- 172.21.15.120:0/28688416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f8709fb40 msgr2=0x559f870a2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:34.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.483+0000 7fafdb7fe640 1 --2- 172.21.15.120:0/28688416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f8709fb40 0x559f870a2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.483+0000 7fafdb7fe640 1 -- 172.21.15.120:0/28688416 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f870a2cb0 con 0x559f8709e5d0 2024-09-19T09:05:34.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.483+0000 7fafdb7fe640 1 --2- 172.21.15.120:0/28688416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f8709e5d0 0x559f870b1900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fafcc0049d0 tx=0x7fafcc004370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5b0458a788d6537 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:34.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.483+0000 7fafbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:05:34.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.484+0000 7fafbaffd640 1 -- 172.21.15.120:0/28688416 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc01e050 con 0x559f8709e5d0 2024-09-19T09:05:34.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.484+0000 7fafbaffd640 1 -- 172.21.15.120:0/28688416 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fafcc009de0 con 0x559f8709e5d0 2024-09-19T09:05:34.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.484+0000 7fafe3e62740 1 -- 172.21.15.120:0/28688416 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559f8700bd70 con 0x559f8709e5d0 2024-09-19T09:05:34.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.484+0000 7fafbaffd640 1 -- 172.21.15.120:0/28688416 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fafcc03d050 con 0x559f8709e5d0 2024-09-19T09:05:34.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.484+0000 7fafe3e62740 10 client.25703 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:34.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.484+0000 7fafe3e62740 1 -- 172.21.15.120:0/28688416 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559f870a3f70 con 0x559f8709e5d0 2024-09-19T09:05:34.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.484+0000 7fafbaffd640 1 -- 172.21.15.120:0/28688416 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559f8700bd70 con 0x559f8709e5d0 2024-09-19T09:05:34.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.484+0000 7fafbaffd640 1 -- 172.21.15.120:0/28688416 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fafcc034070 con 0x559f8709e5d0 2024-09-19T09:05:34.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.484+0000 7fafe3e62740 10 client.25703 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:34.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.484+0000 7fafe3e62740 10 client.25703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:34.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.484+0000 7fafe3e62740 20 client.25703 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:34.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.484+0000 7fafe3e62740 1 --2- 172.21.15.120:0/28688416 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559f87122ea0 0x559f871252c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:34.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.484+0000 7fafe3e62740 4 client.25703 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:05:34.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.484+0000 7fafe3e62740 1 -- 172.21.15.120:0/28688416 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559f86f88170 con 0x559f87122ea0 2024-09-19T09:05:34.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.484+0000 7fafdaffd640 1 --2- 172.21.15.120:0/28688416 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559f87122ea0 0x559f871252c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:34.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.485+0000 7fafbaffd640 10 client.25703 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:34.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.485+0000 7fafdaffd640 1 --2- 172.21.15.120:0/28688416 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559f87122ea0 0x559f871252c0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb757576cb12a6e0 server_cookie=c4b6b1d4d9d27f31 in_seq=0 out_seq=0 2024-09-19T09:05:34.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.485+0000 7fafbaffd640 1 -- 172.21.15.120:0/28688416 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559f86f88170 con 0x559f87122ea0 2024-09-19T09:05:34.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.485+0000 7fafbaffd640 10 client.25703 handle_command_reply: tid=0 2024-09-19T09:05:34.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.485+0000 7fafe3e62740 1 client.25703 shutdown 2024-09-19T09:05:34.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.485+0000 7fafe3e62740 1 -- 172.21.15.120:0/28688416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f8709e5d0 msgr2=0x559f870b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:34.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.485+0000 7fafe3e62740 1 --2- 172.21.15.120:0/28688416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f8709e5d0 0x559f870b1900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fafcc0049d0 tx=0x7fafcc004370 comp rx=0 tx=0).stop 2024-09-19T09:05:34.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.485+0000 7fafe3e62740 1 -- 172.21.15.120:0/28688416 shutdown_connections 2024-09-19T09:05:34.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.485+0000 7fafe3e62740 1 --2- 172.21.15.120:0/28688416 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559f87122ea0 0x559f871252c0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.485+0000 7fafe3e62740 1 --2- 172.21.15.120:0/28688416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f870a9370 0x559f870a2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.485+0000 7fafe3e62740 1 --2- 172.21.15.120:0/28688416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f8709fb40 0x559f870a2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.485+0000 7fafbaffd640 0 client.25703 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:34.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.485+0000 7fafe3e62740 1 --2- 172.21.15.120:0/28688416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f8709e5d0 0x559f870b1900 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.485+0000 7fafe3e62740 1 -- 172.21.15.120:0/28688416 >> 172.21.15.120:0/28688416 conn(0x559f87067310 msgr2=0x559f870a5080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:34.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.485+0000 7fafe3e62740 1 -- 172.21.15.120:0/28688416 shutdown_connections 2024-09-19T09:05:34.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.485+0000 7fafe3e62740 1 -- 172.21.15.120:0/28688416 wait complete. 2024-09-19T09:05:34.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.485+0000 7fafe3e62740 20 client.25703 trim_cache size 0 max 16384 2024-09-19T09:05:34.502 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:05:34.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.486+0000 7fafe1a89640 1 -- 172.21.15.120:0/1621491818 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fafc80a3240 msgr2=0x7fafc80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:34.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.486+0000 7fafe1a89640 1 --2- 172.21.15.120:0/1621491818 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fafc80a3240 0x7fafc80a5700 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fafdc0ff0e0 tx=0x7fafc400a760 comp rx=0 tx=0).stop 2024-09-19T09:05:34.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.486+0000 7fafe1a89640 1 -- 172.21.15.120:0/1621491818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafdc102ef0 msgr2=0x7fafdc19cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:34.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.486+0000 7fafe1a89640 1 --2- 172.21.15.120:0/1621491818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafdc102ef0 0x7fafdc19cd20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fafd4004af0 tx=0x7fafd402f5b0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.486+0000 7fafe1a89640 1 -- 172.21.15.120:0/1621491818 shutdown_connections 2024-09-19T09:05:34.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.486+0000 7fafe1a89640 1 --2- 172.21.15.120:0/1621491818 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fafc80a3240 0x7fafc80a5700 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.486+0000 7fafe1a89640 1 --2- 172.21.15.120:0/1621491818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafdc102ef0 0x7fafdc19cd20 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.486+0000 7fafe1a89640 1 --2- 172.21.15.120:0/1621491818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafdc102570 0x7fafdc198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.486+0000 7fafe1a89640 1 --2- 172.21.15.120:0/1621491818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafdc100e20 0x7fafdc198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:34.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.486+0000 7fafe1a89640 1 -- 172.21.15.120:0/1621491818 >> 172.21.15.120:0/1621491818 conn(0x7fafdc0fc7d0 msgr2=0x7fafdc06a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:34.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.486+0000 7fafe1a89640 1 -- 172.21.15.120:0/1621491818 shutdown_connections 2024-09-19T09:05:34.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:34.486+0000 7fafe1a89640 1 -- 172.21.15.120:0/1621491818 wait complete. 2024-09-19T09:05:35.039 INFO:tasks.workunit.client.0.smithi120.stdout: 1 143719 8.79 MB/sec warmup 90 sec latency 54.456 ms 2024-09-19T09:05:35.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:34 smithi120 ceph-mon[31150]: pgmap v422: 97 pgs: 97 active+clean; 45 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.3 MiB/s wr, 373 op/s 2024-09-19T09:05:35.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:34 smithi196 ceph-mon[35150]: pgmap v422: 97 pgs: 97 active+clean; 45 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.3 MiB/s wr, 373 op/s 2024-09-19T09:05:35.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:34 smithi153 ceph-mon[35071]: pgmap v422: 97 pgs: 97 active+clean; 45 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.3 MiB/s wr, 373 op/s 2024-09-19T09:05:36.039 INFO:tasks.workunit.client.0.smithi120.stdout: 1 146518 8.87 MB/sec warmup 91 sec latency 13.097 ms 2024-09-19T09:05:36.466 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:05:36.466 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:05:36.726 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:05:37.039 INFO:tasks.workunit.client.0.smithi120.stdout: 1 149596 8.96 MB/sec warmup 92 sec latency 10.993 ms 2024-09-19T09:05:37.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:36 smithi120 ceph-mon[31150]: pgmap v423: 97 pgs: 97 active+clean; 49 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.7 MiB/s wr, 415 op/s 2024-09-19T09:05:37.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:36 smithi196 ceph-mon[35150]: pgmap v423: 97 pgs: 97 active+clean; 49 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.7 MiB/s wr, 415 op/s 2024-09-19T09:05:37.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:36 smithi153 ceph-mon[35071]: pgmap v423: 97 pgs: 97 active+clean; 49 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.7 MiB/s wr, 415 op/s 2024-09-19T09:05:37.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.316+0000 7f8a802d1640 1 -- 172.21.15.120:0/3416763475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a78102570 msgr2=0x7f8a781029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:37.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.316+0000 7f8a802d1640 1 --2- 172.21.15.120:0/3416763475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a78102570 0x7f8a781029b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8a68009900 tx=0x7f8a6802a480 comp rx=0 tx=0).stop 2024-09-19T09:05:37.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.316+0000 7f8a802d1640 1 -- 172.21.15.120:0/3416763475 shutdown_connections 2024-09-19T09:05:37.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.316+0000 7f8a802d1640 1 --2- 172.21.15.120:0/3416763475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a78102ef0 0x7f8a7810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.316+0000 7f8a802d1640 1 --2- 172.21.15.120:0/3416763475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a78102570 0x7f8a781029b0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.316+0000 7f8a802d1640 1 --2- 172.21.15.120:0/3416763475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a78100e20 0x7f8a78101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.316+0000 7f8a802d1640 1 -- 172.21.15.120:0/3416763475 >> 172.21.15.120:0/3416763475 conn(0x7f8a780fc7d0 msgr2=0x7f8a780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:37.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.316+0000 7f8a802d1640 1 -- 172.21.15.120:0/3416763475 shutdown_connections 2024-09-19T09:05:37.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.316+0000 7f8a802d1640 1 -- 172.21.15.120:0/3416763475 wait complete. 2024-09-19T09:05:37.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.317+0000 7f8a802d1640 1 Processor -- start 2024-09-19T09:05:37.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.317+0000 7f8a802d1640 1 -- start start 2024-09-19T09:05:37.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.317+0000 7f8a802d1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a78100e20 0x7f8a78197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:37.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.317+0000 7f8a802d1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a78102570 0x7f8a78197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:37.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.317+0000 7f8a802d1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a78102ef0 0x7f8a7819bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:37.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.317+0000 7f8a802d1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8a78110fa0 con 0x7f8a78102ef0 2024-09-19T09:05:37.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.317+0000 7f8a802d1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8a78110e20 con 0x7f8a78102570 2024-09-19T09:05:37.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.317+0000 7f8a802d1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8a78111120 con 0x7f8a78100e20 2024-09-19T09:05:37.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.317+0000 7f8a7e86e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a78102ef0 0x7f8a7819bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:37.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.317+0000 7f8a7d86c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a78102570 0x7f8a78197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:37.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.317+0000 7f8a7e86e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a78102ef0 0x7f8a7819bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58494/0 (socket says 172.21.15.120:58494) 2024-09-19T09:05:37.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.317+0000 7f8a7e06d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a78100e20 0x7f8a78197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:37.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.317+0000 7f8a7e86e640 1 -- 172.21.15.120:0/2270999444 learned_addr learned my addr 172.21.15.120:0/2270999444 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:37.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.318+0000 7f8a7e86e640 1 -- 172.21.15.120:0/2270999444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a78100e20 msgr2=0x7f8a78197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:37.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.318+0000 7f8a7e86e640 1 --2- 172.21.15.120:0/2270999444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a78100e20 0x7f8a78197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.318+0000 7f8a7e86e640 1 -- 172.21.15.120:0/2270999444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a78102570 msgr2=0x7f8a78197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:37.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.318+0000 7f8a7e86e640 1 --2- 172.21.15.120:0/2270999444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a78102570 0x7f8a78197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.318+0000 7f8a7e86e640 1 -- 172.21.15.120:0/2270999444 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a7819c3f0 con 0x7f8a78102ef0 2024-09-19T09:05:37.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.318+0000 7f8a7e86e640 1 --2- 172.21.15.120:0/2270999444 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a78102ef0 0x7f8a7819bd10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f8a7000b970 tx=0x7f8a7000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:37.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.318+0000 7f8a6f7fe640 1 -- 172.21.15.120:0/2270999444 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a70019070 con 0x7f8a78102ef0 2024-09-19T09:05:37.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.318+0000 7f8a6f7fe640 1 -- 172.21.15.120:0/2270999444 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a7000cb40 con 0x7f8a78102ef0 2024-09-19T09:05:37.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.318+0000 7f8a802d1640 1 -- 172.21.15.120:0/2270999444 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a7819c6e0 con 0x7f8a78102ef0 2024-09-19T09:05:37.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.318+0000 7f8a6f7fe640 1 -- 172.21.15.120:0/2270999444 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a70007650 con 0x7f8a78102ef0 2024-09-19T09:05:37.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.318+0000 7f8a802d1640 1 -- 172.21.15.120:0/2270999444 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a78069f70 con 0x7f8a78102ef0 2024-09-19T09:05:37.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.320+0000 7f8a6f7fe640 1 -- 172.21.15.120:0/2270999444 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f8a7001c070 con 0x7f8a78102ef0 2024-09-19T09:05:37.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.320+0000 7f8a826aa740 1 Processor -- start 2024-09-19T09:05:37.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.321+0000 7f8a826aa740 1 -- start start 2024-09-19T09:05:37.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.321+0000 7f8a826aa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621e12fab60 0x5621e12faf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:37.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.321+0000 7f8a6f7fe640 1 --2- 172.21.15.120:0/2270999444 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a640a3240 0x7f8a640a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:37.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.321+0000 7f8a826aa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621e12fb480 0x5621e1305800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:37.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.321+0000 7f8a826aa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e1305d40 0x5621e1308130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:37.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.321+0000 7f8a826aa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5621e1260280 con 0x5621e12fab60 2024-09-19T09:05:37.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.321+0000 7f8a826aa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5621e1207b10 con 0x5621e12fb480 2024-09-19T09:05:37.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.321+0000 7f8a826aa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5621e125c240 con 0x5621e1305d40 2024-09-19T09:05:37.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.321+0000 7f8a7d86c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621e12fab60 0x5621e12faf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:37.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.321+0000 7f8a7e86e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e1305d40 0x5621e1308130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:37.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.321+0000 7f8a7d86c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621e12fab60 0x5621e12faf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58508/0 (socket says 172.21.15.120:58508) 2024-09-19T09:05:37.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.321+0000 7f8a7d86c640 1 -- 172.21.15.120:0/4267583727 learned_addr learned my addr 172.21.15.120:0/4267583727 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:37.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.321+0000 7f8a6f7fe640 1 -- 172.21.15.120:0/2270999444 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f8a700cef00 con 0x7f8a78102ef0 2024-09-19T09:05:37.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.321+0000 7f8a7e06d640 1 --2- 172.21.15.120:0/2270999444 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a640a3240 0x7f8a640a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:37.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.321+0000 7f8a7d86c640 1 --2- 172.21.15.120:0/4267583727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621e12fb480 0x5621e1305800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:37.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.321+0000 7f8a7e86e640 1 -- 172.21.15.120:0/4267583727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621e12fb480 msgr2=0x5621e1305800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:37.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.321+0000 7f8a7e86e640 1 --2- 172.21.15.120:0/4267583727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621e12fb480 0x5621e1305800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.321+0000 7f8a7e86e640 1 -- 172.21.15.120:0/4267583727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621e12fab60 msgr2=0x5621e12faf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:37.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.321+0000 7f8a7e86e640 1 --2- 172.21.15.120:0/4267583727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621e12fab60 0x5621e12faf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.321+0000 7f8a7e86e640 1 -- 172.21.15.120:0/4267583727 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621e12141f0 con 0x5621e1305d40 2024-09-19T09:05:37.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.322+0000 7f8a7e86e640 1 --2- 172.21.15.120:0/4267583727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e1305d40 0x5621e1308130 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8a700cca00 tx=0x7f8a70021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcf5a99e54d2f8ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:37.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.322+0000 7f8a7e06d640 1 --2- 172.21.15.120:0/2270999444 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a640a3240 0x7f8a640a5700 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8a74005d00 tx=0x7f8a7400d620 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:37.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.322+0000 7f8a6cdea640 1 -- 172.21.15.120:0/4267583727 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7009b5d0 con 0x5621e1305d40 2024-09-19T09:05:37.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.322+0000 7f8a6cdea640 1 -- 172.21.15.120:0/4267583727 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a7009f8d0 con 0x5621e1305d40 2024-09-19T09:05:37.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.322+0000 7f8a6cdea640 1 -- 172.21.15.120:0/4267583727 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a70075b50 con 0x5621e1305d40 2024-09-19T09:05:37.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.323+0000 7f8a826aa740 1 -- 172.21.15.120:0/4267583727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e1305d40 msgr2=0x5621e1308130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:37.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.323+0000 7f8a826aa740 1 --2- 172.21.15.120:0/4267583727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e1305d40 0x5621e1308130 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8a700cca00 tx=0x7f8a70021000 comp rx=0 tx=0).stop 2024-09-19T09:05:37.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.323+0000 7f8a826aa740 1 -- 172.21.15.120:0/4267583727 shutdown_connections 2024-09-19T09:05:37.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.323+0000 7f8a826aa740 1 --2- 172.21.15.120:0/4267583727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e1305d40 0x5621e1308130 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.323+0000 7f8a826aa740 1 --2- 172.21.15.120:0/4267583727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621e12fb480 0x5621e1305800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.323+0000 7f8a826aa740 1 --2- 172.21.15.120:0/4267583727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621e12fab60 0x5621e12faf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.323+0000 7f8a826aa740 1 -- 172.21.15.120:0/4267583727 >> 172.21.15.120:0/4267583727 conn(0x5621e12f70f0 msgr2=0x5621e12f74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:37.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.323+0000 7f8a826aa740 1 -- 172.21.15.120:0/4267583727 shutdown_connections 2024-09-19T09:05:37.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.323+0000 7f8a826aa740 1 -- 172.21.15.120:0/4267583727 wait complete. 2024-09-19T09:05:37.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.323+0000 7f8a826aa740 1 -- start start 2024-09-19T09:05:37.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.323+0000 7f8a826aa740 1 Processor -- start 2024-09-19T09:05:37.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.323+0000 7f8a826aa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621e12fab60 0x5621e1302c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:37.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.323+0000 7f8a826aa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e12fb480 0x5621e13031c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:37.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.323+0000 7f8a826aa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621e1305d40 0x5621e1303700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:37.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.323+0000 7f8a826aa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5621e1308910 con 0x5621e12fab60 2024-09-19T09:05:37.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.323+0000 7f8a826aa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5621e121af10 con 0x5621e1305d40 2024-09-19T09:05:37.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.323+0000 7f8a826aa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5621e1308a90 con 0x5621e12fb480 2024-09-19T09:05:37.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.323+0000 7f8a826aa740 10 client.0 _ll_register_callbacks cb 0x5621e11e3a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:37.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.323+0000 7f8a7e06d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621e12fab60 0x5621e1302c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:37.358 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:05:37.359 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:05:37.359 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "a10f1156-e44b-4b0d-a22f-ce478c7e674b", 2024-09-19T09:05:37.359 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:05:37.359 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:05:37.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.323+0000 7f8a7e06d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621e12fab60 0x5621e1302c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58524/0 (socket says 172.21.15.120:58524) 2024-09-19T09:05:37.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.323+0000 7f8a7e06d640 1 -- 172.21.15.120:0/2172597262 learned_addr learned my addr 172.21.15.120:0/2172597262 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:37.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.323+0000 7f8a7d86c640 1 --2- 172.21.15.120:0/2172597262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e12fb480 0x5621e13031c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:37.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.324+0000 7f8a7e86e640 1 --2- 172.21.15.120:0/2172597262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621e1305d40 0x5621e1303700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:37.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.324+0000 7f8a7e06d640 1 -- 172.21.15.120:0/2172597262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e12fb480 msgr2=0x5621e13031c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:37.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.324+0000 7f8a7e06d640 1 --2- 172.21.15.120:0/2172597262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e12fb480 0x5621e13031c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.324+0000 7f8a7e06d640 1 -- 172.21.15.120:0/2172597262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621e1305d40 msgr2=0x5621e1303700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:37.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.324+0000 7f8a7e06d640 1 --2- 172.21.15.120:0/2172597262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621e1305d40 0x5621e1303700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.324+0000 7f8a7e06d640 1 -- 172.21.15.120:0/2172597262 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621e0fd16a0 con 0x5621e12fab60 2024-09-19T09:05:37.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.324+0000 7f8a7e06d640 1 --2- 172.21.15.120:0/2172597262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621e12fab60 0x5621e1302c80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f8a74019b70 tx=0x7f8a74004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30aa81f75202092 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:37.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.324+0000 7f8a61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:37.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.324+0000 7f8a61ffb640 1 -- 172.21.15.120:0/2172597262 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a74029060 con 0x5621e12fab60 2024-09-19T09:05:37.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.324+0000 7f8a61ffb640 1 -- 172.21.15.120:0/2172597262 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f8a74017d60 con 0x5621e12fab60 2024-09-19T09:05:37.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.324+0000 7f8a826aa740 1 -- 172.21.15.120:0/2172597262 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a7009f8d0 con 0x5621e12fab60 2024-09-19T09:05:37.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.324+0000 7f8a61ffb640 1 -- 172.21.15.120:0/2172597262 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8a74024400 con 0x5621e12fab60 2024-09-19T09:05:37.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.324+0000 7f8a826aa740 10 client.16353 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:37.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.324+0000 7f8a826aa740 1 -- 172.21.15.120:0/2172597262 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5621e13145d0 con 0x5621e12fab60 2024-09-19T09:05:37.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.325+0000 7f8a61ffb640 1 -- 172.21.15.120:0/2172597262 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a7009f8d0 con 0x5621e12fab60 2024-09-19T09:05:37.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.325+0000 7f8a61ffb640 1 -- 172.21.15.120:0/2172597262 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8a7402e070 con 0x5621e12fab60 2024-09-19T09:05:37.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.325+0000 7f8a826aa740 10 client.16353 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:37.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.325+0000 7f8a826aa740 10 client.16353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:37.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.325+0000 7f8a826aa740 20 client.16353 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:37.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.325+0000 7f8a826aa740 1 --2- 172.21.15.120:0/2172597262 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5621e1315a90 0x5621e1317e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:37.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.325+0000 7f8a826aa740 4 client.16353 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:05:37.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.325+0000 7f8a826aa740 1 -- 172.21.15.120:0/2172597262 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621e126d7f0 con 0x5621e1315a90 2024-09-19T09:05:37.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.325+0000 7f8a7d86c640 1 --2- 172.21.15.120:0/2172597262 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5621e1315a90 0x5621e1317e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:37.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.325+0000 7f8a61ffb640 10 client.16353 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:37.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.325+0000 7f8a7d86c640 1 --2- 172.21.15.120:0/2172597262 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5621e1315a90 0x5621e1317e80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a449f978580d3a65 server_cookie=c578af90d6bd0c50 in_seq=0 out_seq=0 2024-09-19T09:05:37.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.326+0000 7f8a61ffb640 1 -- 172.21.15.120:0/2172597262 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5621e126d7f0 con 0x5621e1315a90 2024-09-19T09:05:37.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.326+0000 7f8a61ffb640 10 client.16353 handle_command_reply: tid=0 2024-09-19T09:05:37.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.326+0000 7f8a826aa740 1 client.16353 shutdown 2024-09-19T09:05:37.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.326+0000 7f8a826aa740 1 -- 172.21.15.120:0/2172597262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621e12fab60 msgr2=0x5621e1302c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:37.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.326+0000 7f8a826aa740 1 --2- 172.21.15.120:0/2172597262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621e12fab60 0x5621e1302c80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f8a74019b70 tx=0x7f8a74004730 comp rx=0 tx=0).stop 2024-09-19T09:05:37.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.326+0000 7f8a826aa740 1 -- 172.21.15.120:0/2172597262 shutdown_connections 2024-09-19T09:05:37.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.326+0000 7f8a826aa740 1 --2- 172.21.15.120:0/2172597262 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5621e1315a90 0x5621e1317e80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.326+0000 7f8a826aa740 1 --2- 172.21.15.120:0/2172597262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621e1305d40 0x5621e1303700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.326+0000 7f8a826aa740 1 --2- 172.21.15.120:0/2172597262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e12fb480 0x5621e13031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.326+0000 7f8a826aa740 1 --2- 172.21.15.120:0/2172597262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621e12fab60 0x5621e1302c80 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.326+0000 7f8a61ffb640 0 client.16353 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:37.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.326+0000 7f8a826aa740 1 -- 172.21.15.120:0/2172597262 >> 172.21.15.120:0/2172597262 conn(0x5621e12f70f0 msgr2=0x5621e12f6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:37.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.326+0000 7f8a826aa740 1 -- 172.21.15.120:0/2172597262 shutdown_connections 2024-09-19T09:05:37.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.326+0000 7f8a826aa740 1 -- 172.21.15.120:0/2172597262 wait complete. 2024-09-19T09:05:37.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.326+0000 7f8a826aa740 20 client.16353 trim_cache size 0 max 16384 2024-09-19T09:05:37.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.343+0000 7f8a826aa740 1 Processor -- start 2024-09-19T09:05:37.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.344+0000 7f8a826aa740 1 -- start start 2024-09-19T09:05:37.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.344+0000 7f8a826aa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e12fab60 0x5621e131d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:37.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.344+0000 7f8a826aa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621e12fb480 0x5621e131dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:37.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.344+0000 7f8a826aa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621e1305d40 0x5621e12ffee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:37.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.344+0000 7f8a826aa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8a781ab400 con 0x5621e12fb480 2024-09-19T09:05:37.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.344+0000 7f8a826aa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8a781acf40 con 0x5621e1305d40 2024-09-19T09:05:37.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.344+0000 7f8a826aa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8a781abad0 con 0x5621e12fab60 2024-09-19T09:05:37.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.344+0000 7f8a7d86c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621e12fb480 0x5621e131dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:37.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.344+0000 7f8a7e06d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e12fab60 0x5621e131d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:37.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.344+0000 7f8a7d86c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621e12fb480 0x5621e131dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58536/0 (socket says 172.21.15.120:58536) 2024-09-19T09:05:37.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.344+0000 7f8a7d86c640 1 -- 172.21.15.120:0/3382463944 learned_addr learned my addr 172.21.15.120:0/3382463944 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:37.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.344+0000 7f8a7e06d640 1 --2- 172.21.15.120:0/3382463944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e12fab60 0x5621e131d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:45138/0 (socket says 172.21.15.120:45138) 2024-09-19T09:05:37.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.344+0000 7f8a7e86e640 1 --2- 172.21.15.120:0/3382463944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621e1305d40 0x5621e12ffee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:37.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.344+0000 7f8a7d86c640 1 -- 172.21.15.120:0/3382463944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e12fab60 msgr2=0x5621e131d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:37.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.344+0000 7f8a7d86c640 1 --2- 172.21.15.120:0/3382463944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e12fab60 0x5621e131d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.344+0000 7f8a7d86c640 1 -- 172.21.15.120:0/3382463944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621e1305d40 msgr2=0x5621e12ffee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:37.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.344+0000 7f8a7d86c640 1 --2- 172.21.15.120:0/3382463944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621e1305d40 0x5621e12ffee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.344+0000 7f8a7d86c640 1 -- 172.21.15.120:0/3382463944 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621e12f2830 con 0x5621e12fb480 2024-09-19T09:05:37.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.344+0000 7f8a7e06d640 1 --2- 172.21.15.120:0/3382463944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e12fab60 0x5621e131d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:37.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.344+0000 7f8a7e86e640 1 --2- 172.21.15.120:0/3382463944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621e1305d40 0x5621e12ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:05:37.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.344+0000 7f8a7d86c640 1 --2- 172.21.15.120:0/3382463944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621e12fb480 0x5621e131dc20 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f8a68005940 tx=0x7f8a6803f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651f7bb6d769d639 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:37.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.345+0000 7f8a63fff640 1 -- 172.21.15.120:0/3382463944 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a68035040 con 0x5621e12fb480 2024-09-19T09:05:37.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.345+0000 7f8a63fff640 1 -- 172.21.15.120:0/3382463944 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a6804f020 con 0x5621e12fb480 2024-09-19T09:05:37.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.345+0000 7f8a63fff640 1 -- 172.21.15.120:0/3382463944 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a68003710 con 0x5621e12fb480 2024-09-19T09:05:37.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.345+0000 7f8a826aa740 1 -- 172.21.15.120:0/3382463944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621e12fb480 msgr2=0x5621e131dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:37.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.345+0000 7f8a826aa740 1 --2- 172.21.15.120:0/3382463944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621e12fb480 0x5621e131dc20 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f8a68005940 tx=0x7f8a6803f040 comp rx=0 tx=0).stop 2024-09-19T09:05:37.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.345+0000 7f8a826aa740 1 -- 172.21.15.120:0/3382463944 shutdown_connections 2024-09-19T09:05:37.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.345+0000 7f8a826aa740 1 --2- 172.21.15.120:0/3382463944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621e1305d40 0x5621e12ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.345+0000 7f8a826aa740 1 --2- 172.21.15.120:0/3382463944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621e12fb480 0x5621e131dc20 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.345+0000 7f8a826aa740 1 --2- 172.21.15.120:0/3382463944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e12fab60 0x5621e131d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.345+0000 7f8a826aa740 1 -- 172.21.15.120:0/3382463944 >> 172.21.15.120:0/3382463944 conn(0x5621e12f70f0 msgr2=0x5621e1304bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:37.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.345+0000 7f8a826aa740 1 -- 172.21.15.120:0/3382463944 shutdown_connections 2024-09-19T09:05:37.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.345+0000 7f8a826aa740 1 -- 172.21.15.120:0/3382463944 wait complete. 2024-09-19T09:05:37.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.346+0000 7f8a826aa740 1 -- start start 2024-09-19T09:05:37.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.346+0000 7f8a826aa740 1 Processor -- start 2024-09-19T09:05:37.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.346+0000 7f8a826aa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e12fab60 0x5621e1309e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:37.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.346+0000 7f8a826aa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621e12fb480 0x5621e130a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:37.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.346+0000 7f8a826aa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621e1305d40 0x5621e137f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:37.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.346+0000 7f8a826aa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8a781ac870 con 0x5621e1305d40 2024-09-19T09:05:37.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.346+0000 7f8a826aa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8a781ac1a0 con 0x5621e12fb480 2024-09-19T09:05:37.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.346+0000 7f8a826aa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8a781a93c0 con 0x5621e12fab60 2024-09-19T09:05:37.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.346+0000 7f8a826aa740 10 client.0 _ll_register_callbacks cb 0x7f8a74009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:37.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.346+0000 7f8a7e86e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621e1305d40 0x5621e137f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:37.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.346+0000 7f8a7e86e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621e1305d40 0x5621e137f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58540/0 (socket says 172.21.15.120:58540) 2024-09-19T09:05:37.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.346+0000 7f8a7d86c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621e12fb480 0x5621e130a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:37.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.346+0000 7f8a7e86e640 1 -- 172.21.15.120:0/3523437240 learned_addr learned my addr 172.21.15.120:0/3523437240 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:37.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.346+0000 7f8a7e06d640 1 --2- 172.21.15.120:0/3523437240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e12fab60 0x5621e1309e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:37.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.346+0000 7f8a7e86e640 1 -- 172.21.15.120:0/3523437240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e12fab60 msgr2=0x5621e1309e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:37.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.346+0000 7f8a7e86e640 1 --2- 172.21.15.120:0/3523437240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e12fab60 0x5621e1309e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.346+0000 7f8a7e86e640 1 -- 172.21.15.120:0/3523437240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621e12fb480 msgr2=0x5621e130a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:37.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.346+0000 7f8a7e86e640 1 --2- 172.21.15.120:0/3523437240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621e12fb480 0x5621e130a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.346+0000 7f8a7e86e640 1 -- 172.21.15.120:0/3523437240 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621e137fbf0 con 0x5621e1305d40 2024-09-19T09:05:37.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.346+0000 7f8a7d86c640 1 --2- 172.21.15.120:0/3523437240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621e12fb480 0x5621e130a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:37.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.347+0000 7f8a7e86e640 1 --2- 172.21.15.120:0/3523437240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621e1305d40 0x5621e137f6b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8a70097fa0 tx=0x7f8a7000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b0cc19718cb7e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:37.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.347+0000 7f8a61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:37.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.347+0000 7f8a61ffb640 1 -- 172.21.15.120:0/3523437240 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a70079020 con 0x5621e1305d40 2024-09-19T09:05:37.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.347+0000 7f8a61ffb640 1 -- 172.21.15.120:0/3523437240 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f8a700bb3f0 con 0x5621e1305d40 2024-09-19T09:05:37.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.347+0000 7f8a826aa740 1 -- 172.21.15.120:0/3523437240 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a6804f020 con 0x5621e1305d40 2024-09-19T09:05:37.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.347+0000 7f8a61ffb640 1 -- 172.21.15.120:0/3523437240 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8a700bb5f0 con 0x5621e1305d40 2024-09-19T09:05:37.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.347+0000 7f8a826aa740 10 client.16359 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:37.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.347+0000 7f8a826aa740 1 -- 172.21.15.120:0/3523437240 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5621e1380e10 con 0x5621e1305d40 2024-09-19T09:05:37.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.347+0000 7f8a61ffb640 1 -- 172.21.15.120:0/3523437240 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a6804f020 con 0x5621e1305d40 2024-09-19T09:05:37.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.347+0000 7f8a61ffb640 1 -- 172.21.15.120:0/3523437240 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8a70073bc0 con 0x5621e1305d40 2024-09-19T09:05:37.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.347+0000 7f8a826aa740 10 client.16359 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:37.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.347+0000 7f8a826aa740 10 client.16359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:37.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.347+0000 7f8a826aa740 20 client.16359 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:37.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.347+0000 7f8a826aa740 1 --2- 172.21.15.120:0/3523437240 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5621e1381150 0x5621e1383540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:37.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.347+0000 7f8a826aa740 4 client.16359 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:05:37.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.347+0000 7f8a826aa740 1 -- 172.21.15.120:0/3523437240 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5621e1267d00 con 0x5621e1381150 2024-09-19T09:05:37.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.347+0000 7f8a7e06d640 1 --2- 172.21.15.120:0/3523437240 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5621e1381150 0x5621e1383540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:37.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.348+0000 7f8a61ffb640 10 client.16359 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:37.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.348+0000 7f8a7e06d640 1 --2- 172.21.15.120:0/3523437240 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5621e1381150 0x5621e1383540 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62bbd773a98e0f68 server_cookie=c79f58fb3e9b86ce in_seq=0 out_seq=0 2024-09-19T09:05:37.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.349+0000 7f8a61ffb640 1 -- 172.21.15.120:0/3523437240 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5621e1267d00 con 0x5621e1381150 2024-09-19T09:05:37.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.349+0000 7f8a61ffb640 10 client.16359 handle_command_reply: tid=0 2024-09-19T09:05:37.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.349+0000 7f8a826aa740 1 client.16359 shutdown 2024-09-19T09:05:37.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.349+0000 7f8a826aa740 1 -- 172.21.15.120:0/3523437240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621e1305d40 msgr2=0x5621e137f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:37.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.349+0000 7f8a826aa740 1 --2- 172.21.15.120:0/3523437240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621e1305d40 0x5621e137f6b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8a70097fa0 tx=0x7f8a7000bf60 comp rx=0 tx=0).stop 2024-09-19T09:05:37.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.349+0000 7f8a826aa740 1 -- 172.21.15.120:0/3523437240 shutdown_connections 2024-09-19T09:05:37.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.349+0000 7f8a826aa740 1 --2- 172.21.15.120:0/3523437240 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5621e1381150 0x5621e1383540 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.349+0000 7f8a61ffb640 0 client.16359 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:37.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.349+0000 7f8a826aa740 1 --2- 172.21.15.120:0/3523437240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621e1305d40 0x5621e137f6b0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.349+0000 7f8a826aa740 1 --2- 172.21.15.120:0/3523437240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621e12fb480 0x5621e130a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.349+0000 7f8a826aa740 1 --2- 172.21.15.120:0/3523437240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e12fab60 0x5621e1309e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.349+0000 7f8a826aa740 1 -- 172.21.15.120:0/3523437240 >> 172.21.15.120:0/3523437240 conn(0x5621e12f70f0 msgr2=0x5621e12fc5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:37.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.349+0000 7f8a826aa740 1 -- 172.21.15.120:0/3523437240 shutdown_connections 2024-09-19T09:05:37.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.349+0000 7f8a826aa740 1 -- 172.21.15.120:0/3523437240 wait complete. 2024-09-19T09:05:37.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.349+0000 7f8a826aa740 20 client.16359 trim_cache size 0 max 16384 2024-09-19T09:05:37.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.350+0000 7f8a802d1640 1 -- 172.21.15.120:0/2270999444 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a640a3240 msgr2=0x7f8a640a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:37.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.350+0000 7f8a802d1640 1 --2- 172.21.15.120:0/2270999444 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a640a3240 0x7f8a640a5700 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8a74005d00 tx=0x7f8a7400d620 comp rx=0 tx=0).stop 2024-09-19T09:05:37.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.350+0000 7f8a802d1640 1 -- 172.21.15.120:0/2270999444 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a78102ef0 msgr2=0x7f8a7819bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:37.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.350+0000 7f8a802d1640 1 --2- 172.21.15.120:0/2270999444 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a78102ef0 0x7f8a7819bd10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f8a7000b970 tx=0x7f8a7000be30 comp rx=0 tx=0).stop 2024-09-19T09:05:37.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.350+0000 7f8a802d1640 1 -- 172.21.15.120:0/2270999444 shutdown_connections 2024-09-19T09:05:37.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.350+0000 7f8a802d1640 1 --2- 172.21.15.120:0/2270999444 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a640a3240 0x7f8a640a5700 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.350+0000 7f8a802d1640 1 --2- 172.21.15.120:0/2270999444 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a78102ef0 0x7f8a7819bd10 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.350+0000 7f8a802d1640 1 --2- 172.21.15.120:0/2270999444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a78102570 0x7f8a78197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.350+0000 7f8a802d1640 1 --2- 172.21.15.120:0/2270999444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a78100e20 0x7f8a78197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:37.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.350+0000 7f8a802d1640 1 -- 172.21.15.120:0/2270999444 >> 172.21.15.120:0/2270999444 conn(0x7f8a780fc7d0 msgr2=0x7f8a780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:37.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.350+0000 7f8a802d1640 1 -- 172.21.15.120:0/2270999444 shutdown_connections 2024-09-19T09:05:37.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:37.350+0000 7f8a802d1640 1 -- 172.21.15.120:0/2270999444 wait complete. 2024-09-19T09:05:38.039 INFO:tasks.workunit.client.0.smithi120.stdout: 1 152791 9.06 MB/sec warmup 93 sec latency 50.593 ms 2024-09-19T09:05:38.152 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:05:38.432 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:05:38.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:38 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:05:38.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:38 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:05:38.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:38 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:05:38.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:38 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:05:38.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:38 smithi120 ceph-mon[31150]: pgmap v424: 97 pgs: 97 active+clean; 49 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.6 MiB/s wr, 367 op/s 2024-09-19T09:05:38.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:38 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:05:39.039 INFO:tasks.workunit.client.0.smithi120.stdout: 1 155427 9.09 MB/sec warmup 94 sec latency 56.984 ms 2024-09-19T09:05:39.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.073+0000 7fa50dc18640 1 -- 172.21.15.120:0/628253757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa508102d30 msgr2=0x7fa508103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:39.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.073+0000 7fa50dc18640 1 --2- 172.21.15.120:0/628253757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa508102d30 0x7fa508103110 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8009900 tx=0x7fa4f802a480 comp rx=0 tx=0).stop 2024-09-19T09:05:39.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.073+0000 7fa50dc18640 1 -- 172.21.15.120:0/628253757 shutdown_connections 2024-09-19T09:05:39.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.073+0000 7fa50dc18640 1 --2- 172.21.15.120:0/628253757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa50810c160 0x7fa50810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.073+0000 7fa50dc18640 1 --2- 172.21.15.120:0/628253757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa508103650 0x7fa50810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.073+0000 7fa50dc18640 1 --2- 172.21.15.120:0/628253757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa508102d30 0x7fa508103110 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.073+0000 7fa50dc18640 1 -- 172.21.15.120:0/628253757 >> 172.21.15.120:0/628253757 conn(0x7fa5080fc790 msgr2=0x7fa5080febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:39.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.073+0000 7fa50dc18640 1 -- 172.21.15.120:0/628253757 shutdown_connections 2024-09-19T09:05:39.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.073+0000 7fa50dc18640 1 -- 172.21.15.120:0/628253757 wait complete. 2024-09-19T09:05:39.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.073+0000 7fa50dc18640 1 Processor -- start 2024-09-19T09:05:39.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.074+0000 7fa50dc18640 1 -- start start 2024-09-19T09:05:39.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.074+0000 7fa50dc18640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa508102d30 0x7fa508100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:39.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.074+0000 7fa50dc18640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa508103650 0x7fa508100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:39.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.074+0000 7fa50dc18640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa50810c160 0x7fa508101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:39.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.074+0000 7fa50dc18640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa508111fb0 con 0x7fa508103650 2024-09-19T09:05:39.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.074+0000 7fa50dc18640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa508111e30 con 0x7fa50810c160 2024-09-19T09:05:39.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.074+0000 7fa50dc18640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa508112130 con 0x7fa508102d30 2024-09-19T09:05:39.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.074+0000 7fa4fffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa508103650 0x7fa508100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:39.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.074+0000 7fa4fffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa508103650 0x7fa508100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58548/0 (socket says 172.21.15.120:58548) 2024-09-19T09:05:39.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.074+0000 7fa5077fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa508102d30 0x7fa508100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:39.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.074+0000 7fa4fffff640 1 -- 172.21.15.120:0/1551814191 learned_addr learned my addr 172.21.15.120:0/1551814191 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:39.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.074+0000 7fa507fff640 1 --2- 172.21.15.120:0/1551814191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa50810c160 0x7fa508101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:39.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.075+0000 7fa4fffff640 1 -- 172.21.15.120:0/1551814191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa508102d30 msgr2=0x7fa508100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:39.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.075+0000 7fa4fffff640 1 --2- 172.21.15.120:0/1551814191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa508102d30 0x7fa508100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.075+0000 7fa4fffff640 1 -- 172.21.15.120:0/1551814191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa50810c160 msgr2=0x7fa508101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:39.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.075+0000 7fa4fffff640 1 --2- 172.21.15.120:0/1551814191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa50810c160 0x7fa508101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.075+0000 7fa4fffff640 1 -- 172.21.15.120:0/1551814191 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5081a6900 con 0x7fa508103650 2024-09-19T09:05:39.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.075+0000 7fa507fff640 1 --2- 172.21.15.120:0/1551814191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa50810c160 0x7fa508101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:39.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.075+0000 7fa4fffff640 1 --2- 172.21.15.120:0/1551814191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa508103650 0x7fa508100ed0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000e9f0 tx=0x7fa4f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:39.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.075+0000 7fa5057fa640 1 -- 172.21.15.120:0/1551814191 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f000ccc0 con 0x7fa508103650 2024-09-19T09:05:39.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.075+0000 7fa5057fa640 1 -- 172.21.15.120:0/1551814191 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f0010910 con 0x7fa508103650 2024-09-19T09:05:39.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.075+0000 7fa50dc18640 1 -- 172.21.15.120:0/1551814191 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5081a6bf0 con 0x7fa508103650 2024-09-19T09:05:39.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.075+0000 7fa5057fa640 1 -- 172.21.15.120:0/1551814191 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0018920 con 0x7fa508103650 2024-09-19T09:05:39.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.075+0000 7fa50dc18640 1 -- 172.21.15.120:0/1551814191 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5081a7180 con 0x7fa508103650 2024-09-19T09:05:39.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.077+0000 7fa5057fa640 1 -- 172.21.15.120:0/1551814191 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa4f0010430 con 0x7fa508103650 2024-09-19T09:05:39.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.078+0000 7fa50fff1740 1 Processor -- start 2024-09-19T09:05:39.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.078+0000 7fa50fff1740 1 -- start start 2024-09-19T09:05:39.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.078+0000 7fa50fff1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2b4c774a0 0x55b2b4c77880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:39.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.078+0000 7fa50fff1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b4c77dc0 0x55b2b4c82140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:39.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.078+0000 7fa50fff1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2b4c82680 0x55b2b4c84a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:39.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.078+0000 7fa50fff1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b2b4bdd2e0 con 0x55b2b4c82680 2024-09-19T09:05:39.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.078+0000 7fa50fff1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b2b4b84ce0 con 0x55b2b4c774a0 2024-09-19T09:05:39.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.078+0000 7fa50fff1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b2b4bd92a0 con 0x55b2b4c77dc0 2024-09-19T09:05:39.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.078+0000 7fa507fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2b4c774a0 0x55b2b4c77880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:39.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.078+0000 7fa5077fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b4c77dc0 0x55b2b4c82140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:39.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.078+0000 7fa4fffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2b4c82680 0x55b2b4c84a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:39.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.078+0000 7fa5077fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b4c77dc0 0x55b2b4c82140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:45174/0 (socket says 172.21.15.120:45174) 2024-09-19T09:05:39.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.078+0000 7fa5077fe640 1 -- 172.21.15.120:0/1160881360 learned_addr learned my addr 172.21.15.120:0/1160881360 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:39.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.078+0000 7fa5057fa640 1 --2- 172.21.15.120:0/1551814191 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa4f40a32c0 0x7fa4f40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:39.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.078+0000 7fa5077fe640 1 -- 172.21.15.120:0/1160881360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2b4c774a0 msgr2=0x55b2b4c77880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:39.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.078+0000 7fa5077fe640 1 --2- 172.21.15.120:0/1160881360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2b4c774a0 0x55b2b4c77880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.079+0000 7fa5077fe640 1 -- 172.21.15.120:0/1160881360 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2b4c82680 msgr2=0x55b2b4c84a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:39.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.079+0000 7fa507fff640 1 --2- 172.21.15.120:0/1551814191 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa4f40a32c0 0x7fa4f40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:39.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.079+0000 7fa5057fa640 1 -- 172.21.15.120:0/1551814191 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa4f00cfab0 con 0x7fa508103650 2024-09-19T09:05:39.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.079+0000 7fa5077fe640 1 --2- 172.21.15.120:0/1160881360 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2b4c82680 0x55b2b4c84a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.079+0000 7fa5077fe640 1 -- 172.21.15.120:0/1160881360 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b4b69440 con 0x55b2b4c77dc0 2024-09-19T09:05:39.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.079+0000 7fa4fffff640 1 --2- 172.21.15.120:0/1160881360 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2b4c82680 0x55b2b4c84a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:39.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.079+0000 7fa5077fe640 1 --2- 172.21.15.120:0/1160881360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b4c77dc0 0x55b2b4c82140 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8009900 tx=0x7fa4f8005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce6dc85959451a9b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:39.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.079+0000 7fa507fff640 1 --2- 172.21.15.120:0/1551814191 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa4f40a32c0 0x7fa4f40a5780 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa500005c40 tx=0x7fa500004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:39.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.079+0000 7fa4fdffb640 1 -- 172.21.15.120:0/1160881360 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8035040 con 0x55b2b4c77dc0 2024-09-19T09:05:39.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.079+0000 7fa4fdffb640 1 -- 172.21.15.120:0/1160881360 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f8004400 con 0x55b2b4c77dc0 2024-09-19T09:05:39.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.079+0000 7fa4fdffb640 1 -- 172.21.15.120:0/1160881360 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f80035f0 con 0x55b2b4c77dc0 2024-09-19T09:05:39.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.080+0000 7fa50fff1740 1 -- 172.21.15.120:0/1160881360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b4c77dc0 msgr2=0x55b2b4c82140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:39.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.080+0000 7fa50fff1740 1 --2- 172.21.15.120:0/1160881360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b4c77dc0 0x55b2b4c82140 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8009900 tx=0x7fa4f8005de0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.080+0000 7fa50fff1740 1 -- 172.21.15.120:0/1160881360 shutdown_connections 2024-09-19T09:05:39.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.080+0000 7fa50fff1740 1 --2- 172.21.15.120:0/1160881360 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2b4c82680 0x55b2b4c84a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.080+0000 7fa50fff1740 1 --2- 172.21.15.120:0/1160881360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b4c77dc0 0x55b2b4c82140 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.080+0000 7fa50fff1740 1 --2- 172.21.15.120:0/1160881360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2b4c774a0 0x55b2b4c77880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.080+0000 7fa50fff1740 1 -- 172.21.15.120:0/1160881360 >> 172.21.15.120:0/1160881360 conn(0x55b2b4c73a30 msgr2=0x55b2b4c73e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:39.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.080+0000 7fa50fff1740 1 -- 172.21.15.120:0/1160881360 shutdown_connections 2024-09-19T09:05:39.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.080+0000 7fa50fff1740 1 -- 172.21.15.120:0/1160881360 wait complete. 2024-09-19T09:05:39.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.080+0000 7fa50fff1740 1 -- start start 2024-09-19T09:05:39.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.081+0000 7fa50fff1740 1 Processor -- start 2024-09-19T09:05:39.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.081+0000 7fa50fff1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2b4c774a0 0x55b2b4c81fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:39.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.081+0000 7fa50fff1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2b4c77dc0 0x55b2b4c7f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:39.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.081+0000 7fa50fff1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b4c82680 0x55b2b4c7f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:39.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.081+0000 7fa50fff1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b2b4c85250 con 0x55b2b4c774a0 2024-09-19T09:05:39.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.081+0000 7fa50fff1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b2b4b98050 con 0x55b2b4c77dc0 2024-09-19T09:05:39.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.081+0000 7fa50fff1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b2b4c853d0 con 0x55b2b4c82680 2024-09-19T09:05:39.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.081+0000 7fa50fff1740 10 client.0 _ll_register_callbacks cb 0x55b2b4bd8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:39.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.081+0000 7fa5077fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2b4c774a0 0x55b2b4c81fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:39.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.081+0000 7fa5077fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2b4c774a0 0x55b2b4c81fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58578/0 (socket says 172.21.15.120:58578) 2024-09-19T09:05:39.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.081+0000 7fa5077fe640 1 -- 172.21.15.120:0/2738002466 learned_addr learned my addr 172.21.15.120:0/2738002466 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:39.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.081+0000 7fa4fffff640 1 --2- 172.21.15.120:0/2738002466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2b4c77dc0 0x55b2b4c7f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:39.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.081+0000 7fa507fff640 1 --2- 172.21.15.120:0/2738002466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b4c82680 0x55b2b4c7f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:39.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.081+0000 7fa5077fe640 1 -- 172.21.15.120:0/2738002466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b4c82680 msgr2=0x55b2b4c7f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:39.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.081+0000 7fa5077fe640 1 --2- 172.21.15.120:0/2738002466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b4c82680 0x55b2b4c7f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.081+0000 7fa5077fe640 1 -- 172.21.15.120:0/2738002466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2b4c77dc0 msgr2=0x55b2b4c7f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:39.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.081+0000 7fa5077fe640 1 --2- 172.21.15.120:0/2738002466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2b4c77dc0 0x55b2b4c7f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.081+0000 7fa5077fe640 1 -- 172.21.15.120:0/2738002466 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2b494e3c0 con 0x55b2b4c774a0 2024-09-19T09:05:39.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.081+0000 7fa4fffff640 1 --2- 172.21.15.120:0/2738002466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2b4c77dc0 0x55b2b4c7f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:39.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.082+0000 7fa5077fe640 1 --2- 172.21.15.120:0/2738002466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2b4c774a0 0x55b2b4c81fc0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa4f80048f0 tx=0x7fa4f802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f054b865d2dd4824 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:39.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.082+0000 7fa4e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:39.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.082+0000 7fa4e6ffd640 1 -- 172.21.15.120:0/2738002466 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8052060 con 0x55b2b4c774a0 2024-09-19T09:05:39.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.082+0000 7fa4e6ffd640 1 -- 172.21.15.120:0/2738002466 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa4f8004200 con 0x55b2b4c774a0 2024-09-19T09:05:39.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.082+0000 7fa50fff1740 1 -- 172.21.15.120:0/2738002466 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4f8004400 con 0x55b2b4c774a0 2024-09-19T09:05:39.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.082+0000 7fa4e6ffd640 1 -- 172.21.15.120:0/2738002466 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa4f8035070 con 0x55b2b4c774a0 2024-09-19T09:05:39.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.082+0000 7fa50fff1740 10 client.16371 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:39.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.082+0000 7fa50fff1740 1 -- 172.21.15.120:0/2738002466 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b2b4c90c60 con 0x55b2b4c774a0 2024-09-19T09:05:39.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.082+0000 7fa4e6ffd640 1 -- 172.21.15.120:0/2738002466 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f8004400 con 0x55b2b4c774a0 2024-09-19T09:05:39.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.082+0000 7fa4e6ffd640 1 -- 172.21.15.120:0/2738002466 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa4f8031980 con 0x55b2b4c774a0 2024-09-19T09:05:39.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.082+0000 7fa50fff1740 10 client.16371 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:39.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.082+0000 7fa50fff1740 10 client.16371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:39.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.082+0000 7fa50fff1740 20 client.16371 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:39.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.082+0000 7fa50fff1740 1 --2- 172.21.15.120:0/2738002466 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b2b4c91fb0 0x55b2b4c943a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:39.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.082+0000 7fa50fff1740 4 client.16371 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:05:39.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.082+0000 7fa50fff1740 1 -- 172.21.15.120:0/2738002466 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2b4bea850 con 0x55b2b4c91fb0 2024-09-19T09:05:39.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.083+0000 7fa4fffff640 1 --2- 172.21.15.120:0/2738002466 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b2b4c91fb0 0x55b2b4c943a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:39.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.083+0000 7fa4e6ffd640 10 client.16371 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:39.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.083+0000 7fa4fffff640 1 --2- 172.21.15.120:0/2738002466 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b2b4c91fb0 0x55b2b4c943a0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=583f58a56cadb175 server_cookie=3bb4672112ecb4ba in_seq=0 out_seq=0 2024-09-19T09:05:39.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.084+0000 7fa4e6ffd640 1 -- 172.21.15.120:0/2738002466 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b2b4bea850 con 0x55b2b4c91fb0 2024-09-19T09:05:39.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.084+0000 7fa4e6ffd640 10 client.16371 handle_command_reply: tid=0 2024-09-19T09:05:39.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.084+0000 7fa50fff1740 1 client.16371 shutdown 2024-09-19T09:05:39.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.085+0000 7fa50fff1740 1 -- 172.21.15.120:0/2738002466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2b4c774a0 msgr2=0x55b2b4c81fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:39.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.085+0000 7fa50fff1740 1 --2- 172.21.15.120:0/2738002466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2b4c774a0 0x55b2b4c81fc0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa4f80048f0 tx=0x7fa4f802a480 comp rx=0 tx=0).stop 2024-09-19T09:05:39.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.085+0000 7fa50fff1740 1 -- 172.21.15.120:0/2738002466 shutdown_connections 2024-09-19T09:05:39.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.085+0000 7fa50fff1740 1 --2- 172.21.15.120:0/2738002466 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b2b4c91fb0 0x55b2b4c943a0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.085+0000 7fa4e6ffd640 0 client.16371 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:39.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.085+0000 7fa50fff1740 1 --2- 172.21.15.120:0/2738002466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b4c82680 0x55b2b4c7f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.085+0000 7fa50fff1740 1 --2- 172.21.15.120:0/2738002466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2b4c77dc0 0x55b2b4c7f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.085+0000 7fa50fff1740 1 --2- 172.21.15.120:0/2738002466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2b4c774a0 0x55b2b4c81fc0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.085+0000 7fa50fff1740 1 -- 172.21.15.120:0/2738002466 >> 172.21.15.120:0/2738002466 conn(0x55b2b4c73a30 msgr2=0x55b2b4c79710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:39.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.085+0000 7fa50fff1740 1 -- 172.21.15.120:0/2738002466 shutdown_connections 2024-09-19T09:05:39.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.085+0000 7fa50fff1740 1 -- 172.21.15.120:0/2738002466 wait complete. 2024-09-19T09:05:39.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.085+0000 7fa50fff1740 20 client.16371 trim_cache size 0 max 16384 2024-09-19T09:05:39.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.102+0000 7fa50fff1740 1 Processor -- start 2024-09-19T09:05:39.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.102+0000 7fa50fff1740 1 -- start start 2024-09-19T09:05:39.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.102+0000 7fa50fff1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2b4c774a0 0x55b2b4c7db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:39.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.102+0000 7fa50fff1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b4c77dc0 0x55b2b4c973d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:39.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.102+0000 7fa50fff1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2b4c82680 0x55b2b4c97910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:39.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.102+0000 7fa50fff1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa5081af830 con 0x55b2b4c774a0 2024-09-19T09:05:39.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.102+0000 7fa50fff1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa5081b1370 con 0x55b2b4c82680 2024-09-19T09:05:39.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.102+0000 7fa50fff1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa5081aff00 con 0x55b2b4c77dc0 2024-09-19T09:05:39.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.103+0000 7fa5077fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2b4c774a0 0x55b2b4c7db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:39.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.103+0000 7fa5077fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2b4c774a0 0x55b2b4c7db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58590/0 (socket says 172.21.15.120:58590) 2024-09-19T09:05:39.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.103+0000 7fa5077fe640 1 -- 172.21.15.120:0/4110822719 learned_addr learned my addr 172.21.15.120:0/4110822719 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:39.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.103+0000 7fa507fff640 1 --2- 172.21.15.120:0/4110822719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2b4c82680 0x55b2b4c97910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:39.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.103+0000 7fa4fffff640 1 --2- 172.21.15.120:0/4110822719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b4c77dc0 0x55b2b4c973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:39.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.103+0000 7fa5077fe640 1 -- 172.21.15.120:0/4110822719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b4c77dc0 msgr2=0x55b2b4c973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:39.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.103+0000 7fa5077fe640 1 --2- 172.21.15.120:0/4110822719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b4c77dc0 0x55b2b4c973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.103+0000 7fa5077fe640 1 -- 172.21.15.120:0/4110822719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2b4c82680 msgr2=0x55b2b4c97910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:39.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.103+0000 7fa5077fe640 1 --2- 172.21.15.120:0/4110822719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2b4c82680 0x55b2b4c97910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.103+0000 7fa5077fe640 1 -- 172.21.15.120:0/4110822719 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b4c7e220 con 0x55b2b4c774a0 2024-09-19T09:05:39.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.103+0000 7fa5077fe640 1 --2- 172.21.15.120:0/4110822719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2b4c774a0 0x55b2b4c7db40 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8031440 tx=0x7fa4f8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6db385fe93b2ab0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:39.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.103+0000 7fa4fd7fa640 1 -- 172.21.15.120:0/4110822719 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f805c020 con 0x55b2b4c774a0 2024-09-19T09:05:39.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.103+0000 7fa4fd7fa640 1 -- 172.21.15.120:0/4110822719 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f8031a90 con 0x55b2b4c774a0 2024-09-19T09:05:39.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.103+0000 7fa4fd7fa640 1 -- 172.21.15.120:0/4110822719 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8052030 con 0x55b2b4c774a0 2024-09-19T09:05:39.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.104+0000 7fa50fff1740 1 -- 172.21.15.120:0/4110822719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2b4c774a0 msgr2=0x55b2b4c7db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:39.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.104+0000 7fa50fff1740 1 --2- 172.21.15.120:0/4110822719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2b4c774a0 0x55b2b4c7db40 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8031440 tx=0x7fa4f8001e50 comp rx=0 tx=0).stop 2024-09-19T09:05:39.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.104+0000 7fa50fff1740 1 -- 172.21.15.120:0/4110822719 shutdown_connections 2024-09-19T09:05:39.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.104+0000 7fa50fff1740 1 --2- 172.21.15.120:0/4110822719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2b4c82680 0x55b2b4c97910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.104+0000 7fa50fff1740 1 --2- 172.21.15.120:0/4110822719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b4c77dc0 0x55b2b4c973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.104+0000 7fa50fff1740 1 --2- 172.21.15.120:0/4110822719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2b4c774a0 0x55b2b4c7db40 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.104+0000 7fa50fff1740 1 -- 172.21.15.120:0/4110822719 >> 172.21.15.120:0/4110822719 conn(0x55b2b4c73a30 msgr2=0x55b2b4c72b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:39.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.104+0000 7fa50fff1740 1 -- 172.21.15.120:0/4110822719 shutdown_connections 2024-09-19T09:05:39.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.104+0000 7fa50fff1740 1 -- 172.21.15.120:0/4110822719 wait complete. 2024-09-19T09:05:39.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.104+0000 7fa50fff1740 1 -- start start 2024-09-19T09:05:39.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.105+0000 7fa50fff1740 1 Processor -- start 2024-09-19T09:05:39.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.105+0000 7fa50fff1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2b4c774a0 0x55b2b4c97050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:39.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.105+0000 7fa50fff1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b4c77dc0 0x55b2b4c97590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:39.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.105+0000 7fa50fff1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2b4c82680 0x55b2b4cf3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:39.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.105+0000 7fa50fff1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa5081b0ca0 con 0x55b2b4c82680 2024-09-19T09:05:39.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.105+0000 7fa50fff1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa5081b05d0 con 0x55b2b4c774a0 2024-09-19T09:05:39.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.105+0000 7fa50fff1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa5081ad7f0 con 0x55b2b4c77dc0 2024-09-19T09:05:39.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.105+0000 7fa50fff1740 10 client.0 _ll_register_callbacks cb 0x7fa4f00d3400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:39.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.105+0000 7fa507fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2b4c82680 0x55b2b4cf3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:39.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.105+0000 7fa5077fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2b4c774a0 0x55b2b4c97050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:39.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.105+0000 7fa507fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2b4c82680 0x55b2b4cf3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58600/0 (socket says 172.21.15.120:58600) 2024-09-19T09:05:39.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.105+0000 7fa507fff640 1 -- 172.21.15.120:0/116471463 learned_addr learned my addr 172.21.15.120:0/116471463 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:39.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.105+0000 7fa4fffff640 1 --2- 172.21.15.120:0/116471463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b4c77dc0 0x55b2b4c97590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:39.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.105+0000 7fa507fff640 1 -- 172.21.15.120:0/116471463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b4c77dc0 msgr2=0x55b2b4c97590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:39.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.105+0000 7fa507fff640 1 --2- 172.21.15.120:0/116471463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b4c77dc0 0x55b2b4c97590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.105+0000 7fa507fff640 1 -- 172.21.15.120:0/116471463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2b4c774a0 msgr2=0x55b2b4c97050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:39.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.105+0000 7fa507fff640 1 --2- 172.21.15.120:0/116471463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2b4c774a0 0x55b2b4c97050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.105+0000 7fa507fff640 1 -- 172.21.15.120:0/116471463 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2b4cf4340 con 0x55b2b4c82680 2024-09-19T09:05:39.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.105+0000 7fa5077fe640 1 --2- 172.21.15.120:0/116471463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2b4c774a0 0x55b2b4c97050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:39.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.106+0000 7fa507fff640 1 --2- 172.21.15.120:0/116471463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2b4c82680 0x55b2b4cf3ca0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa50001f720 tx=0x7fa500004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d315c759296303e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:39.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.106+0000 7fa4e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:39.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.106+0000 7fa4e6ffd640 1 -- 172.21.15.120:0/116471463 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50002f030 con 0x55b2b4c82680 2024-09-19T09:05:39.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.106+0000 7fa4e6ffd640 1 -- 172.21.15.120:0/116471463 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa500004b20 con 0x55b2b4c82680 2024-09-19T09:05:39.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.106+0000 7fa50fff1740 1 -- 172.21.15.120:0/116471463 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4f8031a90 con 0x55b2b4c82680 2024-09-19T09:05:39.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.106+0000 7fa4e6ffd640 1 -- 172.21.15.120:0/116471463 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa500004d20 con 0x55b2b4c82680 2024-09-19T09:05:39.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.106+0000 7fa50fff1740 10 client.16377 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:39.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.106+0000 7fa50fff1740 1 -- 172.21.15.120:0/116471463 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b2b4cfc940 con 0x55b2b4c82680 2024-09-19T09:05:39.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.106+0000 7fa4e6ffd640 1 -- 172.21.15.120:0/116471463 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f8031a90 con 0x55b2b4c82680 2024-09-19T09:05:39.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.106+0000 7fa4e6ffd640 1 -- 172.21.15.120:0/116471463 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa50002b720 con 0x55b2b4c82680 2024-09-19T09:05:39.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.106+0000 7fa50fff1740 10 client.16377 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:39.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.106+0000 7fa50fff1740 10 client.16377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:39.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.106+0000 7fa50fff1740 20 client.16377 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:39.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.106+0000 7fa50fff1740 1 --2- 172.21.15.120:0/116471463 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b2b4c91fb0 0x55b2b4cff180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:39.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.106+0000 7fa50fff1740 4 client.16377 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:05:39.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.106+0000 7fa50fff1740 1 -- 172.21.15.120:0/116471463 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b2b4be4d60 con 0x55b2b4c91fb0 2024-09-19T09:05:39.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.107+0000 7fa5077fe640 1 --2- 172.21.15.120:0/116471463 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b2b4c91fb0 0x55b2b4cff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:39.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.107+0000 7fa5077fe640 1 --2- 172.21.15.120:0/116471463 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b2b4c91fb0 0x55b2b4cff180 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e837cf782f502105 server_cookie=4ebf9074ca4cf832 in_seq=0 out_seq=0 2024-09-19T09:05:39.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.107+0000 7fa4e6ffd640 10 client.16377 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:39.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.108+0000 7fa4e6ffd640 1 -- 172.21.15.120:0/116471463 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b2b4be4d60 con 0x55b2b4c91fb0 2024-09-19T09:05:39.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.108+0000 7fa4e6ffd640 10 client.16377 handle_command_reply: tid=0 2024-09-19T09:05:39.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.108+0000 7fa50fff1740 1 client.16377 shutdown 2024-09-19T09:05:39.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.108+0000 7fa50fff1740 1 -- 172.21.15.120:0/116471463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2b4c82680 msgr2=0x55b2b4cf3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:39.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.108+0000 7fa50fff1740 1 --2- 172.21.15.120:0/116471463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2b4c82680 0x55b2b4cf3ca0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa50001f720 tx=0x7fa500004ab0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.108+0000 7fa50fff1740 1 -- 172.21.15.120:0/116471463 shutdown_connections 2024-09-19T09:05:39.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.108+0000 7fa50fff1740 1 --2- 172.21.15.120:0/116471463 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b2b4c91fb0 0x55b2b4cff180 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.108+0000 7fa50fff1740 1 --2- 172.21.15.120:0/116471463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b2b4c82680 0x55b2b4cf3ca0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.108+0000 7fa50fff1740 1 --2- 172.21.15.120:0/116471463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b4c77dc0 0x55b2b4c97590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.108+0000 7fa4e6ffd640 0 client.16377 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:39.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.108+0000 7fa50fff1740 1 --2- 172.21.15.120:0/116471463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b2b4c774a0 0x55b2b4c97050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.108+0000 7fa50fff1740 1 -- 172.21.15.120:0/116471463 >> 172.21.15.120:0/116471463 conn(0x55b2b4c73a30 msgr2=0x55b2b4c72b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:39.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.108+0000 7fa50fff1740 1 -- 172.21.15.120:0/116471463 shutdown_connections 2024-09-19T09:05:39.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.108+0000 7fa50fff1740 1 -- 172.21.15.120:0/116471463 wait complete. 2024-09-19T09:05:39.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.108+0000 7fa50fff1740 20 client.16377 trim_cache size 0 max 16384 2024-09-19T09:05:39.114 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:05:39.115 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:05:39.115 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:05:39.115 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:05:39.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.109+0000 7fa50dc18640 1 -- 172.21.15.120:0/1551814191 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa4f40a32c0 msgr2=0x7fa4f40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:39.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.109+0000 7fa50dc18640 1 --2- 172.21.15.120:0/1551814191 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa4f40a32c0 0x7fa4f40a5780 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa500005c40 tx=0x7fa500004a40 comp rx=0 tx=0).stop 2024-09-19T09:05:39.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.109+0000 7fa50dc18640 1 -- 172.21.15.120:0/1551814191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa508103650 msgr2=0x7fa508100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:39.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.109+0000 7fa50dc18640 1 --2- 172.21.15.120:0/1551814191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa508103650 0x7fa508100ed0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000e9f0 tx=0x7fa4f000eeb0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.109+0000 7fa50dc18640 1 -- 172.21.15.120:0/1551814191 shutdown_connections 2024-09-19T09:05:39.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.109+0000 7fa50dc18640 1 --2- 172.21.15.120:0/1551814191 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa4f40a32c0 0x7fa4f40a5780 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.109+0000 7fa50dc18640 1 --2- 172.21.15.120:0/1551814191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa50810c160 0x7fa508101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.109+0000 7fa50dc18640 1 --2- 172.21.15.120:0/1551814191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa508103650 0x7fa508100ed0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.109+0000 7fa50dc18640 1 --2- 172.21.15.120:0/1551814191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa508102d30 0x7fa508100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:39.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.109+0000 7fa50dc18640 1 -- 172.21.15.120:0/1551814191 >> 172.21.15.120:0/1551814191 conn(0x7fa5080fc790 msgr2=0x7fa5080fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:39.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.109+0000 7fa50dc18640 1 -- 172.21.15.120:0/1551814191 shutdown_connections 2024-09-19T09:05:39.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:39.109+0000 7fa50dc18640 1 -- 172.21.15.120:0/1551814191 wait complete. 2024-09-19T09:05:39.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:38 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:05:39.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:38 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:05:39.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:38 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:05:39.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:38 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:05:39.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:38 smithi196 ceph-mon[35150]: pgmap v424: 97 pgs: 97 active+clean; 49 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.6 MiB/s wr, 367 op/s 2024-09-19T09:05:39.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:38 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:05:39.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:38 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:05:39.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:38 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:05:39.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:38 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:05:39.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:38 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:05:39.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:38 smithi153 ceph-mon[35071]: pgmap v424: 97 pgs: 97 active+clean; 49 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.6 MiB/s wr, 367 op/s 2024-09-19T09:05:39.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:38 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:05:39.958 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:05:39.959 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:05:40.039 INFO:tasks.workunit.client.0.smithi120.stdout: 1 157761 9.13 MB/sec warmup 95 sec latency 13.658 ms 2024-09-19T09:05:40.213 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:05:40.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.783+0000 7fd611193640 1 -- 172.21.15.120:0/668832227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60c102570 msgr2=0x7fd60c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:40.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.783+0000 7fd611193640 1 --2- 172.21.15.120:0/668832227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60c102570 0x7fd60c1029b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd600009900 tx=0x7fd60002a480 comp rx=0 tx=0).stop 2024-09-19T09:05:40.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.783+0000 7fd611193640 1 -- 172.21.15.120:0/668832227 shutdown_connections 2024-09-19T09:05:40.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.784+0000 7fd611193640 1 --2- 172.21.15.120:0/668832227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd60c102ef0 0x7fd60c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.784+0000 7fd611193640 1 --2- 172.21.15.120:0/668832227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60c102570 0x7fd60c1029b0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.784+0000 7fd611193640 1 --2- 172.21.15.120:0/668832227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd60c100e20 0x7fd60c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.784+0000 7fd611193640 1 -- 172.21.15.120:0/668832227 >> 172.21.15.120:0/668832227 conn(0x7fd60c0fc7d0 msgr2=0x7fd60c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:40.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.784+0000 7fd611193640 1 -- 172.21.15.120:0/668832227 shutdown_connections 2024-09-19T09:05:40.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.784+0000 7fd611193640 1 -- 172.21.15.120:0/668832227 wait complete. 2024-09-19T09:05:40.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.785+0000 7fd611193640 1 Processor -- start 2024-09-19T09:05:40.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.785+0000 7fd611193640 1 -- start start 2024-09-19T09:05:40.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.785+0000 7fd611193640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd60c100e20 0x7fd60c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:40.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.785+0000 7fd611193640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd60c102570 0x7fd60c198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:40.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.785+0000 7fd611193640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60c102ef0 0x7fd60c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:40.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.785+0000 7fd611193640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c110f80 con 0x7fd60c102ef0 2024-09-19T09:05:40.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.785+0000 7fd611193640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c110e00 con 0x7fd60c102570 2024-09-19T09:05:40.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.785+0000 7fd611193640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c111100 con 0x7fd60c100e20 2024-09-19T09:05:40.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.785+0000 7fd60b59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60c102ef0 0x7fd60c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:40.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.785+0000 7fd60ad9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd60c100e20 0x7fd60c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:40.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.785+0000 7fd60a59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd60c102570 0x7fd60c198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:40.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.785+0000 7fd60b59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60c102ef0 0x7fd60c19cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58618/0 (socket says 172.21.15.120:58618) 2024-09-19T09:05:40.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.785+0000 7fd60ad9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd60c100e20 0x7fd60c1984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:45216/0 (socket says 172.21.15.120:45216) 2024-09-19T09:05:40.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.785+0000 7fd60b59e640 1 -- 172.21.15.120:0/3861781386 learned_addr learned my addr 172.21.15.120:0/3861781386 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:40.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.785+0000 7fd60a59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd60c102570 0x7fd60c198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:55168/0 (socket says 172.21.15.120:55168) 2024-09-19T09:05:40.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.786+0000 7fd60b59e640 1 -- 172.21.15.120:0/3861781386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd60c100e20 msgr2=0x7fd60c1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:40.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.786+0000 7fd60b59e640 1 --2- 172.21.15.120:0/3861781386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd60c100e20 0x7fd60c1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.786+0000 7fd60b59e640 1 -- 172.21.15.120:0/3861781386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd60c102570 msgr2=0x7fd60c198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:40.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.786+0000 7fd60b59e640 1 --2- 172.21.15.120:0/3861781386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd60c102570 0x7fd60c198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.786+0000 7fd60b59e640 1 -- 172.21.15.120:0/3861781386 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60c19d480 con 0x7fd60c102ef0 2024-09-19T09:05:40.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.786+0000 7fd60ad9d640 1 --2- 172.21.15.120:0/3861781386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd60c100e20 0x7fd60c1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:40.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.786+0000 7fd60b59e640 1 --2- 172.21.15.120:0/3861781386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60c102ef0 0x7fd60c19cda0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd60400eb10 tx=0x7fd60400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:40.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.787+0000 7fd5f3fff640 1 -- 172.21.15.120:0/3861781386 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604004b20 con 0x7fd60c102ef0 2024-09-19T09:05:40.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.787+0000 7fd5f3fff640 1 -- 172.21.15.120:0/3861781386 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd604004cc0 con 0x7fd60c102ef0 2024-09-19T09:05:40.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.787+0000 7fd5f3fff640 1 -- 172.21.15.120:0/3861781386 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604010760 con 0x7fd60c102ef0 2024-09-19T09:05:40.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.787+0000 7fd611193640 1 -- 172.21.15.120:0/3861781386 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd60c19d770 con 0x7fd60c102ef0 2024-09-19T09:05:40.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.787+0000 7fd611193640 1 -- 172.21.15.120:0/3861781386 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd60c1a5050 con 0x7fd60c102ef0 2024-09-19T09:05:40.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.788+0000 7fd611193640 1 -- 172.21.15.120:0/3861781386 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd60c101200 con 0x7fd60c102ef0 2024-09-19T09:05:40.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.789+0000 7fd5f3fff640 1 -- 172.21.15.120:0/3861781386 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd604005000 con 0x7fd60c102ef0 2024-09-19T09:05:40.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.790+0000 7fd5f3fff640 1 --2- 172.21.15.120:0/3861781386 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd5f40a32c0 0x7fd5f40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:40.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.790+0000 7fd60ad9d640 1 --2- 172.21.15.120:0/3861781386 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd5f40a32c0 0x7fd5f40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:40.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.790+0000 7fd5f3fff640 1 -- 172.21.15.120:0/3861781386 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fd6040d0070 con 0x7fd60c102ef0 2024-09-19T09:05:40.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.791+0000 7fd60ad9d640 1 --2- 172.21.15.120:0/3861781386 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd5f40a32c0 0x7fd5f40a5780 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc005fd0 tx=0x7fd5fc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:40.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:40.793+0000 7fd5f3fff640 1 -- 172.21.15.120:0/3861781386 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd604054210 con 0x7fd60c102ef0 2024-09-19T09:05:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.869+0000 7f2592fc0640 1 -- 172.21.15.120:0/3092554269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f258c100a40 msgr2=0x7f258c100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.869+0000 7f2592fc0640 1 --2- 172.21.15.120:0/3092554269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f258c100a40 0x7f258c100e40 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f2580009ca0 tx=0x7f258002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.869+0000 7f2592fc0640 1 -- 172.21.15.120:0/3092554269 shutdown_connections 2024-09-19T09:05:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.869+0000 7f2592fc0640 1 --2- 172.21.15.120:0/3092554269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f258c1025f0 0x7f258c108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.869+0000 7f2592fc0640 1 --2- 172.21.15.120:0/3092554269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f258c101c30 0x7f258c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.869+0000 7f2592fc0640 1 --2- 172.21.15.120:0/3092554269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f258c100a40 0x7f258c100e40 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.869+0000 7f2592fc0640 1 -- 172.21.15.120:0/3092554269 >> 172.21.15.120:0/3092554269 conn(0x7f258c0fc1e0 msgr2=0x7f258c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.869+0000 7f2592fc0640 1 -- 172.21.15.120:0/3092554269 shutdown_connections 2024-09-19T09:05:40.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.870+0000 7f2592fc0640 1 -- 172.21.15.120:0/3092554269 wait complete. 2024-09-19T09:05:40.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.870+0000 7f2592fc0640 1 Processor -- start 2024-09-19T09:05:40.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.870+0000 7f2592fc0640 1 -- start start 2024-09-19T09:05:40.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.870+0000 7f2592fc0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f258c100a40 0x7f258c10b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:40.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.870+0000 7f2592fc0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f258c101c30 0x7f258c109a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:40.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.870+0000 7f2592fc0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f258c1025f0 0x7f258c109f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:40.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.870+0000 7f2592fc0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f258c10d870 con 0x7f258c100a40 2024-09-19T09:05:40.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.870+0000 7f2592fc0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f258c10d6f0 con 0x7f258c1025f0 2024-09-19T09:05:40.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.870+0000 7f2592fc0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f258c10d9f0 con 0x7f258c101c30 2024-09-19T09:05:40.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.870+0000 7f2590d5c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f258c100a40 0x7f258c10b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:40.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.870+0000 7f2590d5c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f258c100a40 0x7f258c10b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58634/0 (socket says 172.21.15.120:58634) 2024-09-19T09:05:40.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.870+0000 7f259155d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f258c1025f0 0x7f258c109f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:40.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.871+0000 7f2590d5c640 1 -- 172.21.15.120:0/2956668290 learned_addr learned my addr 172.21.15.120:0/2956668290 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:40.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.871+0000 7f258bfff640 1 --2- 172.21.15.120:0/2956668290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f258c101c30 0x7f258c109a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:40.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.871+0000 7f259155d640 1 -- 172.21.15.120:0/2956668290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f258c101c30 msgr2=0x7f258c109a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:40.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.871+0000 7f259155d640 1 --2- 172.21.15.120:0/2956668290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f258c101c30 0x7f258c109a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.871+0000 7f259155d640 1 -- 172.21.15.120:0/2956668290 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f258c100a40 msgr2=0x7f258c10b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:40.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.871+0000 7f259155d640 1 --2- 172.21.15.120:0/2956668290 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f258c100a40 0x7f258c10b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.871+0000 7f259155d640 1 -- 172.21.15.120:0/2956668290 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f258c10a7d0 con 0x7f258c1025f0 2024-09-19T09:05:40.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.871+0000 7f258bfff640 1 --2- 172.21.15.120:0/2956668290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f258c101c30 0x7f258c109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:40.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.871+0000 7f259155d640 1 --2- 172.21.15.120:0/2956668290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f258c1025f0 0x7f258c109f40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f258400e970 tx=0x7f258400ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:40.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.871+0000 7f2589ffb640 1 -- 172.21.15.120:0/2956668290 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258400ccc0 con 0x7f258c1025f0 2024-09-19T09:05:40.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.871+0000 7f2592fc0640 1 -- 172.21.15.120:0/2956668290 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f258c1a1f30 con 0x7f258c1025f0 2024-09-19T09:05:40.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.871+0000 7f2589ffb640 1 -- 172.21.15.120:0/2956668290 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2584005020 con 0x7f258c1025f0 2024-09-19T09:05:40.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.871+0000 7f2589ffb640 1 -- 172.21.15.120:0/2956668290 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2584004a00 con 0x7f258c1025f0 2024-09-19T09:05:40.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.871+0000 7f2592fc0640 1 -- 172.21.15.120:0/2956668290 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f258c1a2390 con 0x7f258c1025f0 2024-09-19T09:05:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.874+0000 7f2589ffb640 1 -- 172.21.15.120:0/2956668290 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f25840051d0 con 0x7f258c1025f0 2024-09-19T09:05:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.874+0000 7f2595e2e740 1 Processor -- start 2024-09-19T09:05:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.874+0000 7f2595e2e740 1 -- start start 2024-09-19T09:05:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.874+0000 7f2595e2e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590723389a0 0x559072338da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.874+0000 7f2595e2e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590723392e0 0x55907233dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.875+0000 7f2595e2e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55907233e370 0x559072340830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.875+0000 7f2595e2e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55907229c2f0 con 0x5590723389a0 2024-09-19T09:05:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.875+0000 7f2595e2e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559072244270 con 0x55907233e370 2024-09-19T09:05:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.875+0000 7f2595e2e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5590722982b0 con 0x5590723392e0 2024-09-19T09:05:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.875+0000 7f2589ffb640 1 --2- 172.21.15.120:0/2956668290 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f257c0a32f0 0x7f257c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.875+0000 7f258bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590723389a0 0x559072338da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.875+0000 7f258bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590723389a0 0x559072338da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58650/0 (socket says 172.21.15.120:58650) 2024-09-19T09:05:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.875+0000 7f258bfff640 1 -- 172.21.15.120:0/1675101229 learned_addr learned my addr 172.21.15.120:0/1675101229 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.875+0000 7f258bfff640 1 --2- 172.21.15.120:0/1675101229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55907233e370 0x559072340830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.875+0000 7f2590d5c640 1 --2- 172.21.15.120:0/1675101229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590723392e0 0x55907233dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.875+0000 7f259155d640 1 --2- 172.21.15.120:0/2956668290 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f257c0a32f0 0x7f257c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.875+0000 7f2589ffb640 1 -- 172.21.15.120:0/2956668290 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f25840cf880 con 0x7f258c1025f0 2024-09-19T09:05:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.875+0000 7f258bfff640 1 -- 172.21.15.120:0/1675101229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590723392e0 msgr2=0x55907233dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.875+0000 7f258bfff640 1 --2- 172.21.15.120:0/1675101229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590723392e0 0x55907233dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.875+0000 7f258bfff640 1 -- 172.21.15.120:0/1675101229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55907233e370 msgr2=0x559072340830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.875+0000 7f258bfff640 1 --2- 172.21.15.120:0/1675101229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55907233e370 0x559072340830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.875+0000 7f258bfff640 1 -- 172.21.15.120:0/1675101229 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559072243340 con 0x5590723389a0 2024-09-19T09:05:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.875+0000 7f258bfff640 1 --2- 172.21.15.120:0/1675101229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590723389a0 0x559072338da0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2578011bd0 tx=0x7f2578009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111b162c4e2d8190 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.875+0000 7f259155d640 1 --2- 172.21.15.120:0/2956668290 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f257c0a32f0 0x7f257c0a57b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f258409be30 tx=0x7f2584066d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.875+0000 7f2576ffd640 1 -- 172.21.15.120:0/1675101229 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257800fc50 con 0x5590723389a0 2024-09-19T09:05:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.876+0000 7f2576ffd640 1 -- 172.21.15.120:0/1675101229 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f257800fdf0 con 0x5590723389a0 2024-09-19T09:05:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.876+0000 7f2576ffd640 1 -- 172.21.15.120:0/1675101229 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257800e3b0 con 0x5590723389a0 2024-09-19T09:05:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.876+0000 7f2595e2e740 1 -- 172.21.15.120:0/1675101229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590723389a0 msgr2=0x559072338da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.876+0000 7f2595e2e740 1 --2- 172.21.15.120:0/1675101229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590723389a0 0x559072338da0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2578011bd0 tx=0x7f2578009b50 comp rx=0 tx=0).stop 2024-09-19T09:05:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.876+0000 7f2595e2e740 1 -- 172.21.15.120:0/1675101229 shutdown_connections 2024-09-19T09:05:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.876+0000 7f2595e2e740 1 --2- 172.21.15.120:0/1675101229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55907233e370 0x559072340830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.876+0000 7f2595e2e740 1 --2- 172.21.15.120:0/1675101229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590723392e0 0x55907233dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.876+0000 7f2595e2e740 1 --2- 172.21.15.120:0/1675101229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590723389a0 0x559072338da0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.876+0000 7f2595e2e740 1 -- 172.21.15.120:0/1675101229 >> 172.21.15.120:0/1675101229 conn(0x559072334010 msgr2=0x559072334410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.876+0000 7f2595e2e740 1 -- 172.21.15.120:0/1675101229 shutdown_connections 2024-09-19T09:05:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.876+0000 7f2595e2e740 1 -- 172.21.15.120:0/1675101229 wait complete. 2024-09-19T09:05:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.877+0000 7f2595e2e740 1 -- start start 2024-09-19T09:05:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.877+0000 7f2595e2e740 1 Processor -- start 2024-09-19T09:05:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.877+0000 7f2595e2e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590723389a0 0x559072342ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.877+0000 7f2595e2e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5590723392e0 0x5590723430e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.877+0000 7f2595e2e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55907233e370 0x55907234a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.877+0000 7f2595e2e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559072341340 con 0x5590723389a0 2024-09-19T09:05:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.877+0000 7f2595e2e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559072225290 con 0x5590723392e0 2024-09-19T09:05:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.877+0000 7f2595e2e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5590723414c0 con 0x55907233e370 2024-09-19T09:05:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.877+0000 7f2595e2e740 10 client.0 _ll_register_callbacks cb 0x559072256140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.877+0000 7f258bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590723389a0 0x559072342ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.877+0000 7f2590d5c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5590723392e0 0x5590723430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.877+0000 7f258bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590723389a0 0x559072342ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58652/0 (socket says 172.21.15.120:58652) 2024-09-19T09:05:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.877+0000 7f258bfff640 1 -- 172.21.15.120:0/149925565 learned_addr learned my addr 172.21.15.120:0/149925565 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.877+0000 7f258bfff640 1 --2- 172.21.15.120:0/149925565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55907233e370 0x55907234a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.877+0000 7f2590d5c640 1 -- 172.21.15.120:0/149925565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55907233e370 msgr2=0x55907234a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.877+0000 7f2590d5c640 1 --2- 172.21.15.120:0/149925565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55907233e370 0x55907234a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.877+0000 7f2590d5c640 1 -- 172.21.15.120:0/149925565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590723389a0 msgr2=0x559072342ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.877+0000 7f2590d5c640 1 --2- 172.21.15.120:0/149925565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590723389a0 0x559072342ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.878+0000 7f2590d5c640 1 -- 172.21.15.120:0/149925565 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559071faf210 con 0x5590723392e0 2024-09-19T09:05:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.878+0000 7f2590d5c640 1 --2- 172.21.15.120:0/149925565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5590723392e0 0x5590723430e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2580006010 tx=0x7f2580009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7ca5e1d8eac0cec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.878+0000 7f256bfff640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:05:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.878+0000 7f256bfff640 1 -- 172.21.15.120:0/149925565 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580003b80 con 0x5590723392e0 2024-09-19T09:05:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.878+0000 7f256bfff640 1 -- 172.21.15.120:0/149925565 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f2580003d20 con 0x5590723392e0 2024-09-19T09:05:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.878+0000 7f2595e2e740 1 -- 172.21.15.120:0/149925565 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f257800fdf0 con 0x5590723392e0 2024-09-19T09:05:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.878+0000 7f256bfff640 1 -- 172.21.15.120:0/149925565 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2580004040 con 0x5590723392e0 2024-09-19T09:05:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.878+0000 7f2595e2e740 10 client.25757 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.878+0000 7f2595e2e740 1 -- 172.21.15.120:0/149925565 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55907207b620 con 0x5590723392e0 2024-09-19T09:05:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.878+0000 7f256bfff640 1 -- 172.21.15.120:0/149925565 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f257800fdf0 con 0x5590723392e0 2024-09-19T09:05:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.878+0000 7f256bfff640 1 -- 172.21.15.120:0/149925565 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f258004e050 con 0x5590723392e0 2024-09-19T09:05:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.879+0000 7f2595e2e740 10 client.25757 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.879+0000 7f2595e2e740 10 client.25757 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.879+0000 7f2595e2e740 20 client.25757 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.879+0000 7f2595e2e740 1 --2- 172.21.15.120:0/149925565 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55907234e7c0 0x559072350bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.879+0000 7f2595e2e740 4 client.25757 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:05:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.879+0000 7f2595e2e740 1 -- 172.21.15.120:0/149925565 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590722a9a20 con 0x55907234e7c0 2024-09-19T09:05:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.879+0000 7f258bfff640 1 --2- 172.21.15.120:0/149925565 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55907234e7c0 0x559072350bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.879+0000 7f256bfff640 10 client.25757 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.879+0000 7f258bfff640 1 --2- 172.21.15.120:0/149925565 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55907234e7c0 0x559072350bd0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e97ec362dca3e543 server_cookie=1603156f70dc98f6 in_seq=0 out_seq=0 2024-09-19T09:05:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.880+0000 7f256bfff640 1 -- 172.21.15.120:0/149925565 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5590722a9a20 con 0x55907234e7c0 2024-09-19T09:05:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.880+0000 7f256bfff640 10 client.25757 handle_command_reply: tid=0 2024-09-19T09:05:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.880+0000 7f2595e2e740 1 client.25757 shutdown 2024-09-19T09:05:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.880+0000 7f2595e2e740 1 -- 172.21.15.120:0/149925565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5590723392e0 msgr2=0x5590723430e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.880+0000 7f2595e2e740 1 --2- 172.21.15.120:0/149925565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5590723392e0 0x5590723430e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2580006010 tx=0x7f2580009800 comp rx=0 tx=0).stop 2024-09-19T09:05:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.880+0000 7f2595e2e740 1 -- 172.21.15.120:0/149925565 shutdown_connections 2024-09-19T09:05:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.880+0000 7f2595e2e740 1 --2- 172.21.15.120:0/149925565 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55907234e7c0 0x559072350bd0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.880+0000 7f2595e2e740 1 --2- 172.21.15.120:0/149925565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55907233e370 0x55907234a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.880+0000 7f256bfff640 0 client.25757 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.880+0000 7f2595e2e740 1 --2- 172.21.15.120:0/149925565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5590723392e0 0x5590723430e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.880+0000 7f2595e2e740 1 --2- 172.21.15.120:0/149925565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590723389a0 0x559072342ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.880+0000 7f2595e2e740 1 -- 172.21.15.120:0/149925565 >> 172.21.15.120:0/149925565 conn(0x559072334010 msgr2=0x559072335820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.880+0000 7f2595e2e740 1 -- 172.21.15.120:0/149925565 shutdown_connections 2024-09-19T09:05:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.880+0000 7f2595e2e740 1 -- 172.21.15.120:0/149925565 wait complete. 2024-09-19T09:05:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.880+0000 7f2595e2e740 20 client.25757 trim_cache size 0 max 16384 2024-09-19T09:05:40.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.894+0000 7f2595e2e740 1 Processor -- start 2024-09-19T09:05:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.894+0000 7f2595e2e740 1 -- start start 2024-09-19T09:05:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.894+0000 7f2595e2e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590723389a0 0x559072334f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.894+0000 7f2595e2e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590723392e0 0x55907234a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.894+0000 7f2595e2e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55907233e370 0x559072335460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.894+0000 7f2595e2e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f258c1ab1c0 con 0x5590723389a0 2024-09-19T09:05:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.894+0000 7f2595e2e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f258c1acd00 con 0x55907233e370 2024-09-19T09:05:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.894+0000 7f2595e2e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f258c1ab890 con 0x5590723392e0 2024-09-19T09:05:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.895+0000 7f258bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590723389a0 0x559072334f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.895+0000 7f258bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590723389a0 0x559072334f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58664/0 (socket says 172.21.15.120:58664) 2024-09-19T09:05:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.895+0000 7f258bfff640 1 -- 172.21.15.120:0/2199513335 learned_addr learned my addr 172.21.15.120:0/2199513335 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.895+0000 7f2590d5c640 1 --2- 172.21.15.120:0/2199513335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590723392e0 0x55907234a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.895+0000 7f258bfff640 1 --2- 172.21.15.120:0/2199513335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55907233e370 0x559072335460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.895+0000 7f2590d5c640 1 -- 172.21.15.120:0/2199513335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55907233e370 msgr2=0x559072335460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.895+0000 7f2590d5c640 1 --2- 172.21.15.120:0/2199513335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55907233e370 0x559072335460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.895+0000 7f2590d5c640 1 -- 172.21.15.120:0/2199513335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590723389a0 msgr2=0x559072334f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.895+0000 7f2590d5c640 1 --2- 172.21.15.120:0/2199513335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590723389a0 0x559072334f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.895+0000 7f2590d5c640 1 -- 172.21.15.120:0/2199513335 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55907233d760 con 0x5590723392e0 2024-09-19T09:05:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.895+0000 7f258bfff640 1 --2- 172.21.15.120:0/2199513335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590723389a0 0x559072334f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.895+0000 7f2590d5c640 1 --2- 172.21.15.120:0/2199513335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590723392e0 0x55907234a0e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f258002f540 tx=0x7f258002a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5d60793f5c2aaac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.895+0000 7f25767fc640 1 -- 172.21.15.120:0/2199513335 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580004b40 con 0x5590723392e0 2024-09-19T09:05:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.895+0000 7f25767fc640 1 -- 172.21.15.120:0/2199513335 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2580036050 con 0x5590723392e0 2024-09-19T09:05:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.895+0000 7f25767fc640 1 -- 172.21.15.120:0/2199513335 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580004ce0 con 0x5590723392e0 2024-09-19T09:05:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.896+0000 7f2595e2e740 1 -- 172.21.15.120:0/2199513335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590723392e0 msgr2=0x55907234a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:40.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.896+0000 7f2595e2e740 1 --2- 172.21.15.120:0/2199513335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590723392e0 0x55907234a0e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f258002f540 tx=0x7f258002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.896+0000 7f2595e2e740 1 -- 172.21.15.120:0/2199513335 shutdown_connections 2024-09-19T09:05:40.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.896+0000 7f2595e2e740 1 --2- 172.21.15.120:0/2199513335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55907233e370 0x559072335460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.896+0000 7f2595e2e740 1 --2- 172.21.15.120:0/2199513335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590723392e0 0x55907234a0e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.896+0000 7f2595e2e740 1 --2- 172.21.15.120:0/2199513335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590723389a0 0x559072334f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.896+0000 7f2595e2e740 1 -- 172.21.15.120:0/2199513335 >> 172.21.15.120:0/2199513335 conn(0x559072334010 msgr2=0x559072342ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:40.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.896+0000 7f2595e2e740 1 -- 172.21.15.120:0/2199513335 shutdown_connections 2024-09-19T09:05:40.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.896+0000 7f2595e2e740 1 -- 172.21.15.120:0/2199513335 wait complete. 2024-09-19T09:05:40.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.896+0000 7f2595e2e740 1 -- start start 2024-09-19T09:05:40.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.896+0000 7f2595e2e740 1 Processor -- start 2024-09-19T09:05:40.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.897+0000 7f2595e2e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590723389a0 0x5590723477d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:40.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.897+0000 7f2595e2e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590723392e0 0x559072347d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:40.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.897+0000 7f2595e2e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55907233e370 0x559072348d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:40.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.897+0000 7f2595e2e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f258c1ac630 con 0x5590723392e0 2024-09-19T09:05:40.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.897+0000 7f2595e2e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f258c1abf60 con 0x55907233e370 2024-09-19T09:05:40.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.897+0000 7f2595e2e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f258c1a9180 con 0x5590723389a0 2024-09-19T09:05:40.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.897+0000 7f2595e2e740 10 client.0 _ll_register_callbacks cb 0x7f2580009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:40.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.897+0000 7f2590d5c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590723392e0 0x559072347d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:40.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.897+0000 7f2590d5c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590723392e0 0x559072347d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58666/0 (socket says 172.21.15.120:58666) 2024-09-19T09:05:40.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.897+0000 7f258bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590723389a0 0x5590723477d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:40.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.897+0000 7f2590d5c640 1 -- 172.21.15.120:0/3027536269 learned_addr learned my addr 172.21.15.120:0/3027536269 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:40.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.897+0000 7f258bfff640 1 --2- 172.21.15.120:0/3027536269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55907233e370 0x559072348d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:40.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.897+0000 7f2590d5c640 1 -- 172.21.15.120:0/3027536269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590723389a0 msgr2=0x5590723477d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:40.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.897+0000 7f2590d5c640 1 --2- 172.21.15.120:0/3027536269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590723389a0 0x5590723477d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.897+0000 7f2590d5c640 1 -- 172.21.15.120:0/3027536269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55907233e370 msgr2=0x559072348d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:40.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.897+0000 7f2590d5c640 1 --2- 172.21.15.120:0/3027536269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55907233e370 0x559072348d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:40.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.897+0000 7f2590d5c640 1 -- 172.21.15.120:0/3027536269 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559072349330 con 0x5590723392e0 2024-09-19T09:05:40.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.897+0000 7f2590d5c640 1 --2- 172.21.15.120:0/3027536269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590723392e0 0x559072347d10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f25800096c0 tx=0x7f2580031bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9ac76e6ecf05a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:40.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.897+0000 7f256bfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:40.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.898+0000 7f256bfff640 1 -- 172.21.15.120:0/3027536269 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580054040 con 0x5590723392e0 2024-09-19T09:05:40.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.898+0000 7f256bfff640 1 -- 172.21.15.120:0/3027536269 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f2580004b40 con 0x5590723392e0 2024-09-19T09:05:40.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.898+0000 7f2595e2e740 1 -- 172.21.15.120:0/3027536269 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2580036050 con 0x5590723392e0 2024-09-19T09:05:40.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.898+0000 7f256bfff640 1 -- 172.21.15.120:0/3027536269 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2580004d40 con 0x5590723392e0 2024-09-19T09:05:40.905 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.898+0000 7f2595e2e740 10 client.16401 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:40.905 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.898+0000 7f2595e2e740 1 -- 172.21.15.120:0/3027536269 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5590723b94b0 con 0x5590723392e0 2024-09-19T09:05:40.905 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.898+0000 7f256bfff640 1 -- 172.21.15.120:0/3027536269 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2580036050 con 0x5590723392e0 2024-09-19T09:05:40.905 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.898+0000 7f256bfff640 1 -- 172.21.15.120:0/3027536269 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f258004e070 con 0x5590723392e0 2024-09-19T09:05:40.905 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.898+0000 7f2595e2e740 10 client.16401 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:40.905 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.898+0000 7f2595e2e740 10 client.16401 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:40.906 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.898+0000 7f2595e2e740 20 client.16401 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:40.906 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.898+0000 7f2595e2e740 1 --2- 172.21.15.120:0/3027536269 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5590723b9c40 0x5590723bc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:40.906 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.898+0000 7f2595e2e740 4 client.16401 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:05:40.906 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.898+0000 7f2595e2e740 1 -- 172.21.15.120:0/3027536269 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5590722a3d70 con 0x5590723b9c40 2024-09-19T09:05:40.906 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.898+0000 7f258bfff640 1 --2- 172.21.15.120:0/3027536269 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5590723b9c40 0x5590723bc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:40.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.899+0000 7f256bfff640 10 client.16401 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:40.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:40.899+0000 7f258bfff640 1 --2- 172.21.15.120:0/3027536269 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5590723b9c40 0x5590723bc030 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc95d5086f57a90 server_cookie=9eb652212cbd5087 in_seq=0 out_seq=0 2024-09-19T09:05:41.039 INFO:tasks.workunit.client.0.smithi120.stdout: 1 159892 9.18 MB/sec warmup 96 sec latency 292.257 ms 2024-09-19T09:05:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:41.041+0000 7f256bfff640 1 -- 172.21.15.120:0/3027536269 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5590722a3d70 con 0x5590723b9c40 2024-09-19T09:05:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:41.041+0000 7f256bfff640 10 client.16401 handle_command_reply: tid=0 2024-09-19T09:05:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:41.041+0000 7f2595e2e740 1 client.16401 shutdown 2024-09-19T09:05:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:41.041+0000 7f2595e2e740 1 -- 172.21.15.120:0/3027536269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590723392e0 msgr2=0x559072347d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:41.041+0000 7f2595e2e740 1 --2- 172.21.15.120:0/3027536269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590723392e0 0x559072347d10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f25800096c0 tx=0x7f2580031bb0 comp rx=0 tx=0).stop 2024-09-19T09:05:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:41.041+0000 7f2595e2e740 1 -- 172.21.15.120:0/3027536269 shutdown_connections 2024-09-19T09:05:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:41.041+0000 7f2595e2e740 1 --2- 172.21.15.120:0/3027536269 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5590723b9c40 0x5590723bc030 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:41.041+0000 7f2595e2e740 1 --2- 172.21.15.120:0/3027536269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55907233e370 0x559072348d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:41.041+0000 7f2595e2e740 1 --2- 172.21.15.120:0/3027536269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590723392e0 0x559072347d10 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:41.041+0000 7f256bfff640 0 client.16401 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:41.041+0000 7f2595e2e740 1 --2- 172.21.15.120:0/3027536269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590723389a0 0x5590723477d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:41.041+0000 7f2595e2e740 1 -- 172.21.15.120:0/3027536269 >> 172.21.15.120:0/3027536269 conn(0x559072334010 msgr2=0x5590723329a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:41.042+0000 7f2595e2e740 1 -- 172.21.15.120:0/3027536269 shutdown_connections 2024-09-19T09:05:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:41.042+0000 7f2595e2e740 1 -- 172.21.15.120:0/3027536269 wait complete. 2024-09-19T09:05:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:41.042+0000 7f2595e2e740 20 client.16401 trim_cache size 0 max 16384 2024-09-19T09:05:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:05:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:05:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:05:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:05:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:41.042+0000 7f2592fc0640 1 -- 172.21.15.120:0/2956668290 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f257c0a32f0 msgr2=0x7f257c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:41.042+0000 7f2592fc0640 1 --2- 172.21.15.120:0/2956668290 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f257c0a32f0 0x7f257c0a57b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f258409be30 tx=0x7f2584066d80 comp rx=0 tx=0).stop 2024-09-19T09:05:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:41.042+0000 7f2592fc0640 1 -- 172.21.15.120:0/2956668290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f258c1025f0 msgr2=0x7f258c109f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:41.042+0000 7f2592fc0640 1 --2- 172.21.15.120:0/2956668290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f258c1025f0 0x7f258c109f40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f258400e970 tx=0x7f258400ee30 comp rx=0 tx=0).stop 2024-09-19T09:05:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:41.042+0000 7f2592fc0640 1 -- 172.21.15.120:0/2956668290 shutdown_connections 2024-09-19T09:05:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:41.042+0000 7f2592fc0640 1 --2- 172.21.15.120:0/2956668290 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f257c0a32f0 0x7f257c0a57b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:41.042+0000 7f2592fc0640 1 --2- 172.21.15.120:0/2956668290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f258c1025f0 0x7f258c109f40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:41.042+0000 7f2592fc0640 1 --2- 172.21.15.120:0/2956668290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f258c101c30 0x7f258c109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:41.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:41.042+0000 7f2592fc0640 1 --2- 172.21.15.120:0/2956668290 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f258c100a40 0x7f258c10b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:41.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:41.042+0000 7f2592fc0640 1 -- 172.21.15.120:0/2956668290 >> 172.21.15.120:0/2956668290 conn(0x7f258c0fc1e0 msgr2=0x7f258c104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:41.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:41.042+0000 7f2592fc0640 1 -- 172.21.15.120:0/2956668290 shutdown_connections 2024-09-19T09:05:41.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:41.042+0000 7f2592fc0640 1 -- 172.21.15.120:0/2956668290 wait complete. 2024-09-19T09:05:41.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:41.056+0000 7fd611193640 1 -- 172.21.15.120:0/3861781386 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd60c199830 con 0x7fd60c102ef0 2024-09-19T09:05:41.057 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:40 smithi120 ceph-mon[31150]: pgmap v425: 97 pgs: 97 active+clean; 47 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.0 MiB/s wr, 391 op/s 2024-09-19T09:05:41.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:41.057+0000 7fd5f3fff640 1 -- 172.21.15.120:0/3861781386 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7fd60c199830 con 0x7fd60c102ef0 2024-09-19T09:05:41.058 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:05:41.058 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":28,"btime":"2024-09-19T09:02:41:696728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:02:41.207357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:05:41.059 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 28 2024-09-19T09:05:41.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:41.060+0000 7fd611193640 1 -- 172.21.15.120:0/3861781386 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd5f40a32c0 msgr2=0x7fd5f40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:41.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:41.060+0000 7fd611193640 1 --2- 172.21.15.120:0/3861781386 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd5f40a32c0 0x7fd5f40a5780 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc005fd0 tx=0x7fd5fc00b390 comp rx=0 tx=0).stop 2024-09-19T09:05:41.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:41.060+0000 7fd611193640 1 -- 172.21.15.120:0/3861781386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60c102ef0 msgr2=0x7fd60c19cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:41.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:41.060+0000 7fd611193640 1 --2- 172.21.15.120:0/3861781386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60c102ef0 0x7fd60c19cda0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd60400eb10 tx=0x7fd60400c2d0 comp rx=0 tx=0).stop 2024-09-19T09:05:41.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:41.060+0000 7fd611193640 1 -- 172.21.15.120:0/3861781386 shutdown_connections 2024-09-19T09:05:41.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:41.061+0000 7fd611193640 1 --2- 172.21.15.120:0/3861781386 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd5f40a32c0 0x7fd5f40a5780 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:41.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:41.061+0000 7fd611193640 1 --2- 172.21.15.120:0/3861781386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60c102ef0 0x7fd60c19cda0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:41.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:41.061+0000 7fd611193640 1 --2- 172.21.15.120:0/3861781386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd60c102570 0x7fd60c198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:41.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:41.061+0000 7fd611193640 1 --2- 172.21.15.120:0/3861781386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd60c100e20 0x7fd60c1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:41.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:41.061+0000 7fd611193640 1 -- 172.21.15.120:0/3861781386 >> 172.21.15.120:0/3861781386 conn(0x7fd60c0fc7d0 msgr2=0x7fd60c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:41.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:41.061+0000 7fd611193640 1 -- 172.21.15.120:0/3861781386 shutdown_connections 2024-09-19T09:05:41.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:41.061+0000 7fd611193640 1 -- 172.21.15.120:0/3861781386 wait complete. 2024-09-19T09:05:41.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:40 smithi196 ceph-mon[35150]: pgmap v425: 97 pgs: 97 active+clean; 47 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.0 MiB/s wr, 391 op/s 2024-09-19T09:05:41.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:40 smithi153 ceph-mon[35071]: pgmap v425: 97 pgs: 97 active+clean; 47 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.0 MiB/s wr, 391 op/s 2024-09-19T09:05:41.872 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:05:42.039 INFO:tasks.workunit.client.0.smithi120.stdout: 1 162828 9.22 MB/sec warmup 97 sec latency 17.050 ms 2024-09-19T09:05:42.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:41 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3861781386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:05:42.130 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:05:42.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:41 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3861781386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:05:42.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:41 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3861781386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:05:42.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.662+0000 7fa281644640 1 -- 172.21.15.120:0/4280777123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa27c101df0 msgr2=0x7fa27c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:42.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.662+0000 7fa281644640 1 --2- 172.21.15.120:0/4280777123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa27c101df0 0x7fa27c1021d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa274009ca0 tx=0x7fa27402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.662+0000 7fa281644640 1 -- 172.21.15.120:0/4280777123 shutdown_connections 2024-09-19T09:05:42.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.662+0000 7fa281644640 1 --2- 172.21.15.120:0/4280777123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa27c106f20 0x7fa27c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.662+0000 7fa281644640 1 --2- 172.21.15.120:0/4280777123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa27c1027a0 0x7fa27c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.662+0000 7fa281644640 1 --2- 172.21.15.120:0/4280777123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa27c101df0 0x7fa27c1021d0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.662+0000 7fa281644640 1 -- 172.21.15.120:0/4280777123 >> 172.21.15.120:0/4280777123 conn(0x7fa27c0fd8f0 msgr2=0x7fa27c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:42.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.662+0000 7fa281644640 1 -- 172.21.15.120:0/4280777123 shutdown_connections 2024-09-19T09:05:42.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.662+0000 7fa281644640 1 -- 172.21.15.120:0/4280777123 wait complete. 2024-09-19T09:05:42.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.662+0000 7fa281644640 1 Processor -- start 2024-09-19T09:05:42.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.663+0000 7fa281644640 1 -- start start 2024-09-19T09:05:42.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.663+0000 7fa281644640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa27c101df0 0x7fa27c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:42.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.663+0000 7fa281644640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa27c1027a0 0x7fa27c198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:42.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.663+0000 7fa281644640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa27c106f20 0x7fa27c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:42.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.663+0000 7fa281644640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c111f40 con 0x7fa27c1027a0 2024-09-19T09:05:42.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.663+0000 7fa281644640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c111dc0 con 0x7fa27c106f20 2024-09-19T09:05:42.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.663+0000 7fa281644640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c1120c0 con 0x7fa27c101df0 2024-09-19T09:05:42.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.663+0000 7fa27a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa27c1027a0 0x7fa27c198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:42.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.663+0000 7fa27a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa27c1027a0 0x7fa27c198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52554/0 (socket says 172.21.15.120:52554) 2024-09-19T09:05:42.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.663+0000 7fa27a7fc640 1 -- 172.21.15.120:0/2576168658 learned_addr learned my addr 172.21.15.120:0/2576168658 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:42.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.663+0000 7fa27b7fe640 1 --2- 172.21.15.120:0/2576168658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa27c106f20 0x7fa27c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:42.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.663+0000 7fa27affd640 1 --2- 172.21.15.120:0/2576168658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa27c101df0 0x7fa27c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:42.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.663+0000 7fa27a7fc640 1 -- 172.21.15.120:0/2576168658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa27c101df0 msgr2=0x7fa27c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:42.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.663+0000 7fa27a7fc640 1 --2- 172.21.15.120:0/2576168658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa27c101df0 0x7fa27c1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.663+0000 7fa27a7fc640 1 -- 172.21.15.120:0/2576168658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa27c106f20 msgr2=0x7fa27c19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:42.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.663+0000 7fa27a7fc640 1 --2- 172.21.15.120:0/2576168658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa27c106f20 0x7fa27c19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.663+0000 7fa27a7fc640 1 -- 172.21.15.120:0/2576168658 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27c19d390 con 0x7fa27c1027a0 2024-09-19T09:05:42.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.663+0000 7fa27b7fe640 1 --2- 172.21.15.120:0/2576168658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa27c106f20 0x7fa27c19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:42.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.664+0000 7fa27a7fc640 1 --2- 172.21.15.120:0/2576168658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa27c1027a0 0x7fa27c198920 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa26400e9f0 tx=0x7fa26400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:42.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.664+0000 7fa273fff640 1 -- 172.21.15.120:0/2576168658 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26400cd80 con 0x7fa27c1027a0 2024-09-19T09:05:42.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.664+0000 7fa281644640 1 -- 172.21.15.120:0/2576168658 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa27c19d680 con 0x7fa27c1027a0 2024-09-19T09:05:42.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.664+0000 7fa273fff640 1 -- 172.21.15.120:0/2576168658 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa264010910 con 0x7fa27c1027a0 2024-09-19T09:05:42.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.664+0000 7fa273fff640 1 -- 172.21.15.120:0/2576168658 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2640188f0 con 0x7fa27c1027a0 2024-09-19T09:05:42.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.664+0000 7fa281644640 1 -- 172.21.15.120:0/2576168658 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa27c1a4f60 con 0x7fa27c1027a0 2024-09-19T09:05:42.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.666+0000 7fa273fff640 1 -- 172.21.15.120:0/2576168658 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa264010ac0 con 0x7fa27c1027a0 2024-09-19T09:05:42.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.667+0000 7fa283a1d740 1 Processor -- start 2024-09-19T09:05:42.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.667+0000 7fa283a1d740 1 -- start start 2024-09-19T09:05:42.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.667+0000 7fa283a1d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56474bbe2b40 0x56474bbe1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:42.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.667+0000 7fa283a1d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56474bbe15d0 0x56474bbe1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:42.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.667+0000 7fa283a1d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56474bbec370 0x56474bbee760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:42.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.667+0000 7fa283a1d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56474bb472f0 con 0x56474bbec370 2024-09-19T09:05:42.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.667+0000 7fa273fff640 1 --2- 172.21.15.120:0/2576168658 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa2680a3240 0x7fa2680a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:42.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.667+0000 7fa283a1d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56474baeecf0 con 0x56474bbe15d0 2024-09-19T09:05:42.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.667+0000 7fa283a1d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56474bb432b0 con 0x56474bbe2b40 2024-09-19T09:05:42.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.667+0000 7fa27affd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56474bbe15d0 0x56474bbe1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:42.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.667+0000 7fa27b7fe640 1 --2- 172.21.15.120:0/2576168658 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa2680a3240 0x7fa2680a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:42.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.667+0000 7fa27a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56474bbec370 0x56474bbee760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:42.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.667+0000 7fa27b7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56474bbe2b40 0x56474bbe1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:42.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.667+0000 7fa27affd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56474bbe15d0 0x56474bbe1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:45880/0 (socket says 172.21.15.120:45880) 2024-09-19T09:05:42.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.667+0000 7fa27affd640 1 -- 172.21.15.120:0/2584166083 learned_addr learned my addr 172.21.15.120:0/2584166083 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:42.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.667+0000 7fa27b7fe640 1 --2- 172.21.15.120:0/2584166083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56474bbe2b40 0x56474bbe1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:54804/0 (socket says 172.21.15.120:54804) 2024-09-19T09:05:42.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.667+0000 7fa273fff640 1 -- 172.21.15.120:0/2576168658 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa2640cfcd0 con 0x7fa27c1027a0 2024-09-19T09:05:42.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.668+0000 7fa27affd640 1 -- 172.21.15.120:0/2584166083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56474bbe2b40 msgr2=0x56474bbe1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:42.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.668+0000 7fa27affd640 1 --2- 172.21.15.120:0/2584166083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56474bbe2b40 0x56474bbe1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.668+0000 7fa27affd640 1 -- 172.21.15.120:0/2584166083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56474bbec370 msgr2=0x56474bbee760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:42.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.668+0000 7fa27affd640 1 --2- 172.21.15.120:0/2584166083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56474bbec370 0x56474bbee760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.668+0000 7fa27affd640 1 -- 172.21.15.120:0/2584166083 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56474bad33a0 con 0x56474bbe15d0 2024-09-19T09:05:42.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.668+0000 7fa27b7fe640 1 --2- 172.21.15.120:0/2584166083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56474bbe2b40 0x56474bbe1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:42.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.668+0000 7fa27b7fe640 1 --2- 172.21.15.120:0/2576168658 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa2680a3240 0x7fa2680a5700 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa26c010cd0 tx=0x7fa26c004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:42.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.668+0000 7fa27affd640 1 --2- 172.21.15.120:0/2584166083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56474bbe15d0 0x56474bbe1a10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa274009bf0 tx=0x7fa274005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=898e741e289b4350 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:42.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.668+0000 7fa2717fa640 1 -- 172.21.15.120:0/2584166083 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa274049070 con 0x56474bbe15d0 2024-09-19T09:05:42.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.668+0000 7fa2717fa640 1 -- 172.21.15.120:0/2584166083 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa274036040 con 0x56474bbe15d0 2024-09-19T09:05:42.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.668+0000 7fa2717fa640 1 -- 172.21.15.120:0/2584166083 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27402fa70 con 0x56474bbe15d0 2024-09-19T09:05:42.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.669+0000 7fa283a1d740 1 -- 172.21.15.120:0/2584166083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56474bbe15d0 msgr2=0x56474bbe1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:42.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.669+0000 7fa283a1d740 1 --2- 172.21.15.120:0/2584166083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56474bbe15d0 0x56474bbe1a10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa274009bf0 tx=0x7fa274005ef0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.669+0000 7fa283a1d740 1 -- 172.21.15.120:0/2584166083 shutdown_connections 2024-09-19T09:05:42.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.669+0000 7fa283a1d740 1 --2- 172.21.15.120:0/2584166083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56474bbec370 0x56474bbee760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.669+0000 7fa283a1d740 1 --2- 172.21.15.120:0/2584166083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56474bbe15d0 0x56474bbe1a10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.669+0000 7fa283a1d740 1 --2- 172.21.15.120:0/2584166083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56474bbe2b40 0x56474bbe1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.669+0000 7fa283a1d740 1 -- 172.21.15.120:0/2584166083 >> 172.21.15.120:0/2584166083 conn(0x56474bbaa310 msgr2=0x56474bbddc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:42.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.669+0000 7fa283a1d740 1 -- 172.21.15.120:0/2584166083 shutdown_connections 2024-09-19T09:05:42.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.669+0000 7fa283a1d740 1 -- 172.21.15.120:0/2584166083 wait complete. 2024-09-19T09:05:42.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.669+0000 7fa283a1d740 1 -- start start 2024-09-19T09:05:42.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.669+0000 7fa283a1d740 1 Processor -- start 2024-09-19T09:05:42.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.670+0000 7fa283a1d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56474bbe15d0 0x56474bbebd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:42.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.670+0000 7fa283a1d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56474bbe2b40 0x56474bbe9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:42.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.670+0000 7fa283a1d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56474bbec370 0x56474bbe9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:42.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.670+0000 7fa283a1d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56474bbeee90 con 0x56474bbe15d0 2024-09-19T09:05:42.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.670+0000 7fa283a1d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56474bb02060 con 0x56474bbe2b40 2024-09-19T09:05:42.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.670+0000 7fa283a1d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56474bbef010 con 0x56474bbec370 2024-09-19T09:05:42.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.670+0000 7fa283a1d740 10 client.0 _ll_register_callbacks cb 0x56474b8558c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:42.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.670+0000 7fa27affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56474bbe15d0 0x56474bbebd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:42.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.670+0000 7fa27a7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56474bbe2b40 0x56474bbe9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:42.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.670+0000 7fa27affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56474bbe15d0 0x56474bbebd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52578/0 (socket says 172.21.15.120:52578) 2024-09-19T09:05:42.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.670+0000 7fa27affd640 1 -- 172.21.15.120:0/1039115824 learned_addr learned my addr 172.21.15.120:0/1039115824 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:42.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.670+0000 7fa27b7fe640 1 --2- 172.21.15.120:0/1039115824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56474bbec370 0x56474bbe9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:42.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.670+0000 7fa27a7fc640 1 -- 172.21.15.120:0/1039115824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56474bbec370 msgr2=0x56474bbe9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:42.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.670+0000 7fa27a7fc640 1 --2- 172.21.15.120:0/1039115824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56474bbec370 0x56474bbe9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.670+0000 7fa27a7fc640 1 -- 172.21.15.120:0/1039115824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56474bbe15d0 msgr2=0x56474bbebd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:42.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.670+0000 7fa27a7fc640 1 --2- 172.21.15.120:0/1039115824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56474bbe15d0 0x56474bbebd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.670+0000 7fa27a7fc640 1 -- 172.21.15.120:0/1039115824 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56474b8b83c0 con 0x56474bbe2b40 2024-09-19T09:05:42.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.670+0000 7fa27b7fe640 1 --2- 172.21.15.120:0/1039115824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56474bbec370 0x56474bbe9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:05:42.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.670+0000 7fa27affd640 1 --2- 172.21.15.120:0/1039115824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56474bbe15d0 0x56474bbebd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:42.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.670+0000 7fa27a7fc640 1 --2- 172.21.15.120:0/1039115824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56474bbe2b40 0x56474bbe9010 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa26409c0a0 tx=0x7fa26409a8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb450eab587bc38a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:42.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.670+0000 7fa2627fc640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:05:42.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.671+0000 7fa2627fc640 1 -- 172.21.15.120:0/1039115824 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa264026040 con 0x56474bbe2b40 2024-09-19T09:05:42.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.671+0000 7fa283a1d740 1 -- 172.21.15.120:0/1039115824 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa274036040 con 0x56474bbe2b40 2024-09-19T09:05:42.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.671+0000 7fa2627fc640 1 -- 172.21.15.120:0/1039115824 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa2640b5190 con 0x56474bbe2b40 2024-09-19T09:05:42.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.671+0000 7fa2627fc640 1 -- 172.21.15.120:0/1039115824 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa26409d620 con 0x56474bbe2b40 2024-09-19T09:05:42.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.671+0000 7fa283a1d740 10 client.25775 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:42.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.671+0000 7fa283a1d740 1 -- 172.21.15.120:0/1039115824 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56474bbfa930 con 0x56474bbe2b40 2024-09-19T09:05:42.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.671+0000 7fa2627fc640 1 -- 172.21.15.120:0/1039115824 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa274036040 con 0x56474bbe2b40 2024-09-19T09:05:42.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.671+0000 7fa2627fc640 1 -- 172.21.15.120:0/1039115824 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa26409ceb0 con 0x56474bbe2b40 2024-09-19T09:05:42.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.671+0000 7fa283a1d740 10 client.25775 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:42.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.671+0000 7fa283a1d740 10 client.25775 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:05:42.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.671+0000 7fa283a1d740 20 client.25775 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:42.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.671+0000 7fa283a1d740 1 --2- 172.21.15.120:0/1039115824 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x56474bbfbd40 0x56474bbfe170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:42.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.671+0000 7fa283a1d740 4 client.25775 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:05:42.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.671+0000 7fa283a1d740 1 -- 172.21.15.120:0/1039115824 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56474bb54860 con 0x56474bbfbd40 2024-09-19T09:05:42.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.671+0000 7fa27affd640 1 --2- 172.21.15.120:0/1039115824 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x56474bbfbd40 0x56474bbfe170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:42.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.672+0000 7fa2627fc640 10 client.25775 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:05:42.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.672+0000 7fa27affd640 1 --2- 172.21.15.120:0/1039115824 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x56474bbfbd40 0x56474bbfe170 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4716c85c351e4e1c server_cookie=4c01b98bda739b08 in_seq=0 out_seq=0 2024-09-19T09:05:42.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.672+0000 7fa2627fc640 1 -- 172.21.15.120:0/1039115824 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56474bb54860 con 0x56474bbfbd40 2024-09-19T09:05:42.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.672+0000 7fa2627fc640 10 client.25775 handle_command_reply: tid=0 2024-09-19T09:05:42.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.673+0000 7fa283a1d740 1 client.25775 shutdown 2024-09-19T09:05:42.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.673+0000 7fa283a1d740 1 -- 172.21.15.120:0/1039115824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56474bbe2b40 msgr2=0x56474bbe9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:42.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.673+0000 7fa283a1d740 1 --2- 172.21.15.120:0/1039115824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56474bbe2b40 0x56474bbe9010 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa26409c0a0 tx=0x7fa26409a8b0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.673+0000 7fa283a1d740 1 -- 172.21.15.120:0/1039115824 shutdown_connections 2024-09-19T09:05:42.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.673+0000 7fa283a1d740 1 --2- 172.21.15.120:0/1039115824 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x56474bbfbd40 0x56474bbfe170 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.673+0000 7fa283a1d740 1 --2- 172.21.15.120:0/1039115824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56474bbec370 0x56474bbe9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.673+0000 7fa2627fc640 0 client.25775 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:05:42.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.673+0000 7fa283a1d740 1 --2- 172.21.15.120:0/1039115824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56474bbe2b40 0x56474bbe9010 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.673+0000 7fa283a1d740 1 --2- 172.21.15.120:0/1039115824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56474bbe15d0 0x56474bbebd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.673+0000 7fa283a1d740 1 -- 172.21.15.120:0/1039115824 >> 172.21.15.120:0/1039115824 conn(0x56474bbaa310 msgr2=0x56474bbe3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:42.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.673+0000 7fa283a1d740 1 -- 172.21.15.120:0/1039115824 shutdown_connections 2024-09-19T09:05:42.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.673+0000 7fa283a1d740 1 -- 172.21.15.120:0/1039115824 wait complete. 2024-09-19T09:05:42.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.673+0000 7fa283a1d740 20 client.25775 trim_cache size 0 max 16384 2024-09-19T09:05:42.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.687+0000 7fa283a1d740 1 Processor -- start 2024-09-19T09:05:42.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.687+0000 7fa283a1d740 1 -- start start 2024-09-19T09:05:42.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.688+0000 7fa283a1d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56474bbe15d0 0x56474bbf8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:42.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.688+0000 7fa283a1d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56474bbe2b40 0x56474bbe7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:42.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.688+0000 7fa283a1d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56474bbec370 0x56474bc03a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:42.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.688+0000 7fa283a1d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c1ad4e0 con 0x56474bbe2b40 2024-09-19T09:05:42.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.688+0000 7fa283a1d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c1af020 con 0x56474bbe15d0 2024-09-19T09:05:42.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.688+0000 7fa283a1d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c1adbb0 con 0x56474bbec370 2024-09-19T09:05:42.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.688+0000 7fa27a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56474bbe2b40 0x56474bbe7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:42.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.688+0000 7fa27a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56474bbe2b40 0x56474bbe7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52590/0 (socket says 172.21.15.120:52590) 2024-09-19T09:05:42.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.688+0000 7fa27a7fc640 1 -- 172.21.15.120:0/458382180 learned_addr learned my addr 172.21.15.120:0/458382180 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:42.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.688+0000 7fa27affd640 1 --2- 172.21.15.120:0/458382180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56474bbe15d0 0x56474bbf8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:42.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.688+0000 7fa27b7fe640 1 --2- 172.21.15.120:0/458382180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56474bbec370 0x56474bc03a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:42.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.688+0000 7fa27a7fc640 1 -- 172.21.15.120:0/458382180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56474bbec370 msgr2=0x56474bc03a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:42.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.688+0000 7fa27a7fc640 1 --2- 172.21.15.120:0/458382180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56474bbec370 0x56474bc03a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.688+0000 7fa27a7fc640 1 -- 172.21.15.120:0/458382180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56474bbe15d0 msgr2=0x56474bbf8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:42.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.688+0000 7fa27a7fc640 1 --2- 172.21.15.120:0/458382180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56474bbe15d0 0x56474bbf8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.688+0000 7fa27a7fc640 1 -- 172.21.15.120:0/458382180 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56474bbe7560 con 0x56474bbe2b40 2024-09-19T09:05:42.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.688+0000 7fa27a7fc640 1 --2- 172.21.15.120:0/458382180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56474bbe2b40 0x56474bbe7ff0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa26409a650 tx=0x7fa26409bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41fe8d685626b2b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:42.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.689+0000 7fa270ff9640 1 -- 172.21.15.120:0/458382180 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2640a4f60 con 0x56474bbe2b40 2024-09-19T09:05:42.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.689+0000 7fa270ff9640 1 -- 172.21.15.120:0/458382180 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2640cd850 con 0x56474bbe2b40 2024-09-19T09:05:42.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.689+0000 7fa270ff9640 1 -- 172.21.15.120:0/458382180 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2640bc330 con 0x56474bbe2b40 2024-09-19T09:05:42.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.689+0000 7fa283a1d740 1 -- 172.21.15.120:0/458382180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56474bbe2b40 msgr2=0x56474bbe7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:42.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.689+0000 7fa283a1d740 1 --2- 172.21.15.120:0/458382180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56474bbe2b40 0x56474bbe7ff0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa26409a650 tx=0x7fa26409bfb0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.689+0000 7fa283a1d740 1 -- 172.21.15.120:0/458382180 shutdown_connections 2024-09-19T09:05:42.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.689+0000 7fa283a1d740 1 --2- 172.21.15.120:0/458382180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56474bbec370 0x56474bc03a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.689+0000 7fa283a1d740 1 --2- 172.21.15.120:0/458382180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56474bbe2b40 0x56474bbe7ff0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.689+0000 7fa283a1d740 1 --2- 172.21.15.120:0/458382180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56474bbe15d0 0x56474bbf8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.690+0000 7fa283a1d740 1 -- 172.21.15.120:0/458382180 >> 172.21.15.120:0/458382180 conn(0x56474bbaa310 msgr2=0x56474bbe78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:42.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.690+0000 7fa283a1d740 1 -- 172.21.15.120:0/458382180 shutdown_connections 2024-09-19T09:05:42.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.690+0000 7fa283a1d740 1 -- 172.21.15.120:0/458382180 wait complete. 2024-09-19T09:05:42.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.690+0000 7fa283a1d740 1 -- start start 2024-09-19T09:05:42.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.690+0000 7fa283a1d740 1 Processor -- start 2024-09-19T09:05:42.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.690+0000 7fa283a1d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56474bbe15d0 0x56474bbe5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:42.712 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:05:42.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.690+0000 7fa283a1d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56474bbe2b40 0x56474bbe64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:42.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.690+0000 7fa283a1d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56474bbec370 0x56474bbe69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:42.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.690+0000 7fa283a1d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c1ae950 con 0x56474bbe15d0 2024-09-19T09:05:42.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.690+0000 7fa283a1d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c1ae280 con 0x56474bbec370 2024-09-19T09:05:42.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.690+0000 7fa283a1d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c1ab4a0 con 0x56474bbe2b40 2024-09-19T09:05:42.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.690+0000 7fa283a1d740 10 client.0 _ll_register_callbacks cb 0x7fa26409c170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:42.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.690+0000 7fa27a7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56474bbe2b40 0x56474bbe64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:42.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.690+0000 7fa27affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56474bbe15d0 0x56474bbe5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:42.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.690+0000 7fa27b7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56474bbec370 0x56474bbe69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:42.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.690+0000 7fa27b7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56474bbec370 0x56474bbe69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:45910/0 (socket says 172.21.15.120:45910) 2024-09-19T09:05:42.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.690+0000 7fa27affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56474bbe15d0 0x56474bbe5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52606/0 (socket says 172.21.15.120:52606) 2024-09-19T09:05:42.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.690+0000 7fa27b7fe640 1 -- 172.21.15.120:0/491531532 learned_addr learned my addr 172.21.15.120:0/491531532 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:42.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.691+0000 7fa27b7fe640 1 -- 172.21.15.120:0/491531532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56474bbe2b40 msgr2=0x56474bbe64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:42.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.691+0000 7fa27b7fe640 1 --2- 172.21.15.120:0/491531532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56474bbe2b40 0x56474bbe64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.691+0000 7fa27b7fe640 1 -- 172.21.15.120:0/491531532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56474bbe15d0 msgr2=0x56474bbe5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:42.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.691+0000 7fa27b7fe640 1 --2- 172.21.15.120:0/491531532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56474bbe15d0 0x56474bbe5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.691+0000 7fa27b7fe640 1 -- 172.21.15.120:0/491531532 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56474bc65a10 con 0x56474bbec370 2024-09-19T09:05:42.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.691+0000 7fa27a7fc640 1 --2- 172.21.15.120:0/491531532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56474bbe2b40 0x56474bbe64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:42.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.691+0000 7fa27affd640 1 --2- 172.21.15.120:0/491531532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56474bbe15d0 0x56474bbe5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:42.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.691+0000 7fa27b7fe640 1 --2- 172.21.15.120:0/491531532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56474bbec370 0x56474bbe69e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa26c026730 tx=0x7fa26c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87cf63b0e1f8bb04 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:42.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.691+0000 7fa2627fc640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:05:42.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.691+0000 7fa2627fc640 1 -- 172.21.15.120:0/491531532 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c019040 con 0x56474bbec370 2024-09-19T09:05:42.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.691+0000 7fa283a1d740 1 -- 172.21.15.120:0/491531532 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2640cd850 con 0x56474bbec370 2024-09-19T09:05:42.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.691+0000 7fa2627fc640 1 -- 172.21.15.120:0/491531532 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa26c004020 con 0x56474bbec370 2024-09-19T09:05:42.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.691+0000 7fa2627fc640 1 -- 172.21.15.120:0/491531532 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa26c004220 con 0x56474bbec370 2024-09-19T09:05:42.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.691+0000 7fa283a1d740 10 client.25781 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:42.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.691+0000 7fa283a1d740 1 -- 172.21.15.120:0/491531532 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56474bc66ca0 con 0x56474bbec370 2024-09-19T09:05:42.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.692+0000 7fa2627fc640 1 -- 172.21.15.120:0/491531532 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2640cd850 con 0x56474bbec370 2024-09-19T09:05:42.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.692+0000 7fa2627fc640 1 -- 172.21.15.120:0/491531532 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa26c003890 con 0x56474bbec370 2024-09-19T09:05:42.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.692+0000 7fa283a1d740 10 client.25781 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:42.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.692+0000 7fa283a1d740 10 client.25781 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:05:42.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.692+0000 7fa283a1d740 20 client.25781 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:42.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.692+0000 7fa283a1d740 1 --2- 172.21.15.120:0/491531532 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x56474bc673d0 0x56474bc697f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:42.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.692+0000 7fa283a1d740 4 client.25781 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:05:42.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.692+0000 7fa283a1d740 1 -- 172.21.15.120:0/491531532 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56474bb4ed70 con 0x56474bc673d0 2024-09-19T09:05:42.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.692+0000 7fa27affd640 1 --2- 172.21.15.120:0/491531532 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x56474bc673d0 0x56474bc697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:42.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.692+0000 7fa2627fc640 10 client.25781 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:05:42.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.692+0000 7fa27affd640 1 --2- 172.21.15.120:0/491531532 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x56474bc673d0 0x56474bc697f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=119ae183582bce39 server_cookie=79312c4cb1e3f9d1 in_seq=0 out_seq=0 2024-09-19T09:05:42.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.693+0000 7fa2627fc640 1 -- 172.21.15.120:0/491531532 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56474bb4ed70 con 0x56474bc673d0 2024-09-19T09:05:42.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.693+0000 7fa2627fc640 10 client.25781 handle_command_reply: tid=0 2024-09-19T09:05:42.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.693+0000 7fa283a1d740 1 client.25781 shutdown 2024-09-19T09:05:42.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.693+0000 7fa283a1d740 1 -- 172.21.15.120:0/491531532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56474bbec370 msgr2=0x56474bbe69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:42.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.693+0000 7fa283a1d740 1 --2- 172.21.15.120:0/491531532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56474bbec370 0x56474bbe69e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa26c026730 tx=0x7fa26c0046c0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.693+0000 7fa283a1d740 1 -- 172.21.15.120:0/491531532 shutdown_connections 2024-09-19T09:05:42.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.693+0000 7fa283a1d740 1 --2- 172.21.15.120:0/491531532 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x56474bc673d0 0x56474bc697f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.693+0000 7fa283a1d740 1 --2- 172.21.15.120:0/491531532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56474bbec370 0x56474bbe69e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.693+0000 7fa283a1d740 1 --2- 172.21.15.120:0/491531532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56474bbe2b40 0x56474bbe64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.693+0000 7fa2627fc640 0 client.25781 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:05:42.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.693+0000 7fa283a1d740 1 --2- 172.21.15.120:0/491531532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56474bbe15d0 0x56474bbe5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.693+0000 7fa283a1d740 1 -- 172.21.15.120:0/491531532 >> 172.21.15.120:0/491531532 conn(0x56474bbaa310 msgr2=0x56474bbe8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:42.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.693+0000 7fa283a1d740 1 -- 172.21.15.120:0/491531532 shutdown_connections 2024-09-19T09:05:42.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.693+0000 7fa283a1d740 1 -- 172.21.15.120:0/491531532 wait complete. 2024-09-19T09:05:42.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.693+0000 7fa283a1d740 20 client.25781 trim_cache size 0 max 16384 2024-09-19T09:05:42.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.694+0000 7fa281644640 1 -- 172.21.15.120:0/2576168658 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa2680a3240 msgr2=0x7fa2680a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:42.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.694+0000 7fa281644640 1 --2- 172.21.15.120:0/2576168658 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa2680a3240 0x7fa2680a5700 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa26c010cd0 tx=0x7fa26c004730 comp rx=0 tx=0).stop 2024-09-19T09:05:42.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.694+0000 7fa281644640 1 -- 172.21.15.120:0/2576168658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa27c1027a0 msgr2=0x7fa27c198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:42.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.694+0000 7fa281644640 1 --2- 172.21.15.120:0/2576168658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa27c1027a0 0x7fa27c198920 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa26400e9f0 tx=0x7fa26400eeb0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.694+0000 7fa281644640 1 -- 172.21.15.120:0/2576168658 shutdown_connections 2024-09-19T09:05:42.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.694+0000 7fa281644640 1 --2- 172.21.15.120:0/2576168658 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa2680a3240 0x7fa2680a5700 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.694+0000 7fa281644640 1 --2- 172.21.15.120:0/2576168658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa27c106f20 0x7fa27c19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.694+0000 7fa281644640 1 --2- 172.21.15.120:0/2576168658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa27c1027a0 0x7fa27c198920 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.694+0000 7fa281644640 1 --2- 172.21.15.120:0/2576168658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa27c101df0 0x7fa27c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:42.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.694+0000 7fa281644640 1 -- 172.21.15.120:0/2576168658 >> 172.21.15.120:0/2576168658 conn(0x7fa27c0fd8f0 msgr2=0x7fa27c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:42.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.694+0000 7fa281644640 1 -- 172.21.15.120:0/2576168658 shutdown_connections 2024-09-19T09:05:42.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:42.694+0000 7fa281644640 1 -- 172.21.15.120:0/2576168658 wait complete. 2024-09-19T09:05:42.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:42 smithi120 ceph-mon[31150]: pgmap v426: 97 pgs: 97 active+clean; 50 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.6 MiB/s wr, 430 op/s 2024-09-19T09:05:43.039 INFO:tasks.workunit.client.0.smithi120.stdout: 1 165637 9.26 MB/sec warmup 98 sec latency 14.325 ms 2024-09-19T09:05:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:42 smithi196 ceph-mon[35150]: pgmap v426: 97 pgs: 97 active+clean; 50 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.6 MiB/s wr, 430 op/s 2024-09-19T09:05:43.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:42 smithi153 ceph-mon[35071]: pgmap v426: 97 pgs: 97 active+clean; 50 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.6 MiB/s wr, 430 op/s 2024-09-19T09:05:44.026 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:05:44.039 INFO:tasks.workunit.client.0.smithi120.stdout: 1 167669 9.30 MB/sec warmup 99 sec latency 123.601 ms 2024-09-19T09:05:44.282 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:05:44.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.797+0000 7fb659c32640 1 -- 172.21.15.120:0/1356293330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb654108290 msgr2=0x7fb65410ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:44.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.797+0000 7fb659c32640 1 --2- 172.21.15.120:0/1356293330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb654108290 0x7fb65410ec30 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb64c009cd0 tx=0x7fb64c02a8e0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.797+0000 7fb659c32640 1 -- 172.21.15.120:0/1356293330 shutdown_connections 2024-09-19T09:05:44.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.797+0000 7fb659c32640 1 --2- 172.21.15.120:0/1356293330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb654108290 0x7fb65410ec30 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.797+0000 7fb659c32640 1 --2- 172.21.15.120:0/1356293330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb654103740 0x7fb654107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.797+0000 7fb659c32640 1 --2- 172.21.15.120:0/1356293330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb654102d90 0x7fb654103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.797+0000 7fb659c32640 1 -- 172.21.15.120:0/1356293330 >> 172.21.15.120:0/1356293330 conn(0x7fb6540fc7d0 msgr2=0x7fb6540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:44.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.797+0000 7fb659c32640 1 -- 172.21.15.120:0/1356293330 shutdown_connections 2024-09-19T09:05:44.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.797+0000 7fb659c32640 1 -- 172.21.15.120:0/1356293330 wait complete. 2024-09-19T09:05:44.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.797+0000 7fb659c32640 1 Processor -- start 2024-09-19T09:05:44.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.798+0000 7fb659c32640 1 -- start start 2024-09-19T09:05:44.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.798+0000 7fb659c32640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb654102d90 0x7fb6541009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:44.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.798+0000 7fb659c32640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb654103740 0x7fb654100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:44.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.798+0000 7fb659c32640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb654108290 0x7fb654101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:44.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.798+0000 7fb659c32640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb654112090 con 0x7fb654103740 2024-09-19T09:05:44.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.798+0000 7fb659c32640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb654111f10 con 0x7fb654102d90 2024-09-19T09:05:44.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.798+0000 7fb659c32640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb654112210 con 0x7fb654108290 2024-09-19T09:05:44.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.798+0000 7fb652ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb654103740 0x7fb654100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:44.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.798+0000 7fb652ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb654103740 0x7fb654100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52630/0 (socket says 172.21.15.120:52630) 2024-09-19T09:05:44.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.798+0000 7fb652ffd640 1 -- 172.21.15.120:0/631287143 learned_addr learned my addr 172.21.15.120:0/631287143 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:44.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.798+0000 7fb6537fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb654102d90 0x7fb6541009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:44.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.798+0000 7fb653fff640 1 --2- 172.21.15.120:0/631287143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb654108290 0x7fb654101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:44.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.798+0000 7fb652ffd640 1 -- 172.21.15.120:0/631287143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb654108290 msgr2=0x7fb654101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:44.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.798+0000 7fb652ffd640 1 --2- 172.21.15.120:0/631287143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb654108290 0x7fb654101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.798+0000 7fb652ffd640 1 -- 172.21.15.120:0/631287143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb654102d90 msgr2=0x7fb6541009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:44.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.798+0000 7fb652ffd640 1 --2- 172.21.15.120:0/631287143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb654102d90 0x7fb6541009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.798+0000 7fb652ffd640 1 -- 172.21.15.120:0/631287143 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6541a6aa0 con 0x7fb654103740 2024-09-19T09:05:44.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.798+0000 7fb652ffd640 1 --2- 172.21.15.120:0/631287143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb654103740 0x7fb654100f30 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb64800e9c0 tx=0x7fb64800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:44.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.799+0000 7fb650ff9640 1 -- 172.21.15.120:0/631287143 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64800cd80 con 0x7fb654103740 2024-09-19T09:05:44.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.799+0000 7fb659c32640 1 -- 172.21.15.120:0/631287143 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6541a6d90 con 0x7fb654103740 2024-09-19T09:05:44.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.799+0000 7fb650ff9640 1 -- 172.21.15.120:0/631287143 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb648005500 con 0x7fb654103740 2024-09-19T09:05:44.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.799+0000 7fb650ff9640 1 -- 172.21.15.120:0/631287143 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648010690 con 0x7fb654103740 2024-09-19T09:05:44.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.799+0000 7fb659c32640 1 -- 172.21.15.120:0/631287143 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6541a7250 con 0x7fb654103740 2024-09-19T09:05:44.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.801+0000 7fb650ff9640 1 -- 172.21.15.120:0/631287143 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fb6480046c0 con 0x7fb654103740 2024-09-19T09:05:44.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.801+0000 7fb65c00b740 1 Processor -- start 2024-09-19T09:05:44.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.802+0000 7fb65c00b740 1 -- start start 2024-09-19T09:05:44.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.802+0000 7fb650ff9640 1 --2- 172.21.15.120:0/631287143 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb63c0a3240 0x7fb63c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:44.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.802+0000 7fb653fff640 1 --2- 172.21.15.120:0/631287143 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb63c0a3240 0x7fb63c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:44.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.802+0000 7fb65c00b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3c907b40 0x55da3c906090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:44.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.802+0000 7fb65c00b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da3c9065d0 0x55da3c906a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:44.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.802+0000 7fb650ff9640 1 -- 172.21.15.120:0/631287143 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb6480cfbe0 con 0x7fb654103740 2024-09-19T09:05:44.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.802+0000 7fb65c00b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da3c911370 0x55da3c913760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:44.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.802+0000 7fb65c00b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55da3c86c2f0 con 0x55da3c911370 2024-09-19T09:05:44.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.802+0000 7fb65c00b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55da3c813cf0 con 0x55da3c9065d0 2024-09-19T09:05:44.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.802+0000 7fb65c00b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55da3c8682b0 con 0x55da3c907b40 2024-09-19T09:05:44.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.802+0000 7fb652ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da3c9065d0 0x55da3c906a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:44.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.802+0000 7fb653fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da3c911370 0x55da3c913760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:44.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.802+0000 7fb652ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da3c9065d0 0x55da3c906a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:45936/0 (socket says 172.21.15.120:45936) 2024-09-19T09:05:44.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.802+0000 7fb652ffd640 1 -- 172.21.15.120:0/3448664872 learned_addr learned my addr 172.21.15.120:0/3448664872 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:44.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.802+0000 7fb6537fe640 1 --2- 172.21.15.120:0/3448664872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3c907b40 0x55da3c906090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:44.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.802+0000 7fb652ffd640 1 -- 172.21.15.120:0/3448664872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3c907b40 msgr2=0x55da3c906090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:44.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.802+0000 7fb652ffd640 1 --2- 172.21.15.120:0/3448664872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3c907b40 0x55da3c906090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.802+0000 7fb652ffd640 1 -- 172.21.15.120:0/3448664872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da3c911370 msgr2=0x55da3c913760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:44.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.802+0000 7fb652ffd640 1 --2- 172.21.15.120:0/3448664872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da3c911370 0x55da3c913760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.802+0000 7fb653fff640 1 --2- 172.21.15.120:0/631287143 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb63c0a3240 0x7fb63c0a5700 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb64c004ab0 tx=0x7fb64c0316c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:44.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.802+0000 7fb652ffd640 1 -- 172.21.15.120:0/3448664872 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da3c7f83a0 con 0x55da3c9065d0 2024-09-19T09:05:44.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.803+0000 7fb652ffd640 1 --2- 172.21.15.120:0/3448664872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da3c9065d0 0x55da3c906a10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb648098bd0 tx=0x7fb648009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0cdf13d9dfd8a89 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:44.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.803+0000 7fb639ffb640 1 -- 172.21.15.120:0/3448664872 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64809c200 con 0x55da3c9065d0 2024-09-19T09:05:44.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.803+0000 7fb639ffb640 1 -- 172.21.15.120:0/3448664872 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6480a05b0 con 0x55da3c9065d0 2024-09-19T09:05:44.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.803+0000 7fb639ffb640 1 -- 172.21.15.120:0/3448664872 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6480a0170 con 0x55da3c9065d0 2024-09-19T09:05:44.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.804+0000 7fb65c00b740 1 -- 172.21.15.120:0/3448664872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da3c9065d0 msgr2=0x55da3c906a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:44.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.804+0000 7fb65c00b740 1 --2- 172.21.15.120:0/3448664872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da3c9065d0 0x55da3c906a10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb648098bd0 tx=0x7fb648009700 comp rx=0 tx=0).stop 2024-09-19T09:05:44.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.804+0000 7fb65c00b740 1 -- 172.21.15.120:0/3448664872 shutdown_connections 2024-09-19T09:05:44.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.804+0000 7fb65c00b740 1 --2- 172.21.15.120:0/3448664872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da3c911370 0x55da3c913760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.804+0000 7fb65c00b740 1 --2- 172.21.15.120:0/3448664872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da3c9065d0 0x55da3c906a10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.804+0000 7fb65c00b740 1 --2- 172.21.15.120:0/3448664872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3c907b40 0x55da3c906090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.804+0000 7fb65c00b740 1 -- 172.21.15.120:0/3448664872 >> 172.21.15.120:0/3448664872 conn(0x55da3c8cf310 msgr2=0x55da3c902c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:44.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.804+0000 7fb65c00b740 1 -- 172.21.15.120:0/3448664872 shutdown_connections 2024-09-19T09:05:44.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.804+0000 7fb65c00b740 1 -- 172.21.15.120:0/3448664872 wait complete. 2024-09-19T09:05:44.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.804+0000 7fb65c00b740 1 -- start start 2024-09-19T09:05:44.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.804+0000 7fb65c00b740 1 Processor -- start 2024-09-19T09:05:44.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.805+0000 7fb65c00b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da3c9065d0 0x55da3c910d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:44.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.805+0000 7fb65c00b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3c907b40 0x55da3c90e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:44.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.805+0000 7fb65c00b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da3c911370 0x55da3c90e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:44.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.805+0000 7fb65c00b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55da3c914070 con 0x55da3c911370 2024-09-19T09:05:44.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.805+0000 7fb65c00b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55da3c827060 con 0x55da3c9065d0 2024-09-19T09:05:44.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.805+0000 7fb65c00b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55da3c9141f0 con 0x55da3c907b40 2024-09-19T09:05:44.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.805+0000 7fb65c00b740 10 client.0 _ll_register_callbacks cb 0x55da3c57a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:44.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.805+0000 7fb653fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da3c911370 0x55da3c90e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:44.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.805+0000 7fb652ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3c907b40 0x55da3c90e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:44.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.805+0000 7fb653fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da3c911370 0x55da3c90e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52654/0 (socket says 172.21.15.120:52654) 2024-09-19T09:05:44.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.805+0000 7fb652ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3c907b40 0x55da3c90e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:54846/0 (socket says 172.21.15.120:54846) 2024-09-19T09:05:44.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.805+0000 7fb653fff640 1 -- 172.21.15.120:0/990023837 learned_addr learned my addr 172.21.15.120:0/990023837 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:44.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.805+0000 7fb6537fe640 1 --2- 172.21.15.120:0/990023837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da3c9065d0 0x55da3c910d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:44.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.805+0000 7fb652ffd640 1 -- 172.21.15.120:0/990023837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da3c9065d0 msgr2=0x55da3c910d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:44.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.805+0000 7fb652ffd640 1 --2- 172.21.15.120:0/990023837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da3c9065d0 0x55da3c910d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.805+0000 7fb652ffd640 1 -- 172.21.15.120:0/990023837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da3c911370 msgr2=0x55da3c90e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:44.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.805+0000 7fb652ffd640 1 --2- 172.21.15.120:0/990023837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da3c911370 0x55da3c90e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.805+0000 7fb652ffd640 1 -- 172.21.15.120:0/990023837 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da3c5dd3c0 con 0x55da3c907b40 2024-09-19T09:05:44.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.805+0000 7fb653fff640 1 --2- 172.21.15.120:0/990023837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da3c911370 0x55da3c90e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:44.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.806+0000 7fb652ffd640 1 --2- 172.21.15.120:0/990023837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3c907b40 0x55da3c90e030 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb64809c190 tx=0x7fb64809bbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=981d7ce98f73ff92 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:44.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.806+0000 7fb632ffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:05:44.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.806+0000 7fb632ffd640 1 -- 172.21.15.120:0/990023837 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64809c200 con 0x55da3c907b40 2024-09-19T09:05:44.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.806+0000 7fb632ffd640 1 -- 172.21.15.120:0/990023837 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb6480a0170 con 0x55da3c907b40 2024-09-19T09:05:44.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.806+0000 7fb65c00b740 1 -- 172.21.15.120:0/990023837 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6480a05b0 con 0x55da3c907b40 2024-09-19T09:05:44.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.806+0000 7fb632ffd640 1 -- 172.21.15.120:0/990023837 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb6480a0370 con 0x55da3c907b40 2024-09-19T09:05:44.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.806+0000 7fb65c00b740 10 client.25756 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:44.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.806+0000 7fb65c00b740 1 -- 172.21.15.120:0/990023837 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55da3c914070 con 0x55da3c907b40 2024-09-19T09:05:44.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.806+0000 7fb632ffd640 1 -- 172.21.15.120:0/990023837 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6480a05b0 con 0x55da3c907b40 2024-09-19T09:05:44.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.806+0000 7fb632ffd640 1 -- 172.21.15.120:0/990023837 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb6480cdba0 con 0x55da3c907b40 2024-09-19T09:05:44.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.806+0000 7fb65c00b740 10 client.25756 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:44.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.806+0000 7fb65c00b740 10 client.25756 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:05:44.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.806+0000 7fb65c00b740 20 client.25756 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:44.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.806+0000 7fb65c00b740 1 --2- 172.21.15.120:0/990023837 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55da3c920cc0 0x55da3c9230b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:44.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.806+0000 7fb65c00b740 4 client.25756 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:05:44.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.806+0000 7fb65c00b740 1 -- 172.21.15.120:0/990023837 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da3c879860 con 0x55da3c920cc0 2024-09-19T09:05:44.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.807+0000 7fb6537fe640 1 --2- 172.21.15.120:0/990023837 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55da3c920cc0 0x55da3c9230b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:44.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.807+0000 7fb632ffd640 10 client.25756 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:05:44.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.807+0000 7fb6537fe640 1 --2- 172.21.15.120:0/990023837 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55da3c920cc0 0x55da3c9230b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e5767207f09acbd2 server_cookie=aceea0dc194dd22b in_seq=0 out_seq=0 2024-09-19T09:05:44.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.808+0000 7fb632ffd640 1 -- 172.21.15.120:0/990023837 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55da3c879860 con 0x55da3c920cc0 2024-09-19T09:05:44.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.808+0000 7fb632ffd640 10 client.25756 handle_command_reply: tid=0 2024-09-19T09:05:44.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.808+0000 7fb65c00b740 1 client.25756 shutdown 2024-09-19T09:05:44.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.808+0000 7fb65c00b740 1 -- 172.21.15.120:0/990023837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3c907b40 msgr2=0x55da3c90e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:44.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.808+0000 7fb65c00b740 1 --2- 172.21.15.120:0/990023837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3c907b40 0x55da3c90e030 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb64809c190 tx=0x7fb64809bbe0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.808+0000 7fb65c00b740 1 -- 172.21.15.120:0/990023837 shutdown_connections 2024-09-19T09:05:44.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.808+0000 7fb65c00b740 1 --2- 172.21.15.120:0/990023837 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55da3c920cc0 0x55da3c9230b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.808+0000 7fb65c00b740 1 --2- 172.21.15.120:0/990023837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da3c911370 0x55da3c90e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.808+0000 7fb65c00b740 1 --2- 172.21.15.120:0/990023837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3c907b40 0x55da3c90e030 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.808+0000 7fb65c00b740 1 --2- 172.21.15.120:0/990023837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da3c9065d0 0x55da3c910d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.808+0000 7fb65c00b740 1 -- 172.21.15.120:0/990023837 >> 172.21.15.120:0/990023837 conn(0x55da3c8cf310 msgr2=0x55da3c9089c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:44.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.808+0000 7fb632ffd640 0 client.25756 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:05:44.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.808+0000 7fb65c00b740 1 -- 172.21.15.120:0/990023837 shutdown_connections 2024-09-19T09:05:44.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.808+0000 7fb65c00b740 1 -- 172.21.15.120:0/990023837 wait complete. 2024-09-19T09:05:44.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.808+0000 7fb65c00b740 20 client.25756 trim_cache size 0 max 16384 2024-09-19T09:05:44.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.821+0000 7fb65c00b740 1 Processor -- start 2024-09-19T09:05:44.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.821+0000 7fb65c00b740 1 -- start start 2024-09-19T09:05:44.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.821+0000 7fb65c00b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da3c9065d0 0x55da3c90df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:44.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.821+0000 7fb65c00b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da3c907b40 0x55da3c91d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:44.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.821+0000 7fb65c00b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3c911370 0x55da3c90cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:44.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.821+0000 7fb65c00b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb6541af820 con 0x55da3c907b40 2024-09-19T09:05:44.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.821+0000 7fb65c00b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb6541b1360 con 0x55da3c9065d0 2024-09-19T09:05:44.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.821+0000 7fb65c00b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb6541afef0 con 0x55da3c911370 2024-09-19T09:05:44.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.821+0000 7fb652ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da3c907b40 0x55da3c91d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:44.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.821+0000 7fb652ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da3c907b40 0x55da3c91d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52670/0 (socket says 172.21.15.120:52670) 2024-09-19T09:05:44.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.821+0000 7fb652ffd640 1 -- 172.21.15.120:0/326298112 learned_addr learned my addr 172.21.15.120:0/326298112 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:44.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.821+0000 7fb6537fe640 1 --2- 172.21.15.120:0/326298112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da3c9065d0 0x55da3c90df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:44.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.822+0000 7fb653fff640 1 --2- 172.21.15.120:0/326298112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3c911370 0x55da3c90cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:44.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.822+0000 7fb652ffd640 1 -- 172.21.15.120:0/326298112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3c911370 msgr2=0x55da3c90cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:44.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.822+0000 7fb652ffd640 1 --2- 172.21.15.120:0/326298112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3c911370 0x55da3c90cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.822+0000 7fb652ffd640 1 -- 172.21.15.120:0/326298112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da3c9065d0 msgr2=0x55da3c90df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:44.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.822+0000 7fb652ffd640 1 --2- 172.21.15.120:0/326298112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da3c9065d0 0x55da3c90df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.822+0000 7fb652ffd640 1 -- 172.21.15.120:0/326298112 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da3c90c5c0 con 0x55da3c907b40 2024-09-19T09:05:44.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.822+0000 7fb652ffd640 1 --2- 172.21.15.120:0/326298112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da3c907b40 0x55da3c91d310 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb6480cfe80 tx=0x7fb64809b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d49c4f6f73a0a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:44.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.822+0000 7fb6397fa640 1 -- 172.21.15.120:0/326298112 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6480a05b0 con 0x55da3c907b40 2024-09-19T09:05:44.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.822+0000 7fb6397fa640 1 -- 172.21.15.120:0/326298112 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6480ba070 con 0x55da3c907b40 2024-09-19T09:05:44.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.822+0000 7fb6397fa640 1 -- 172.21.15.120:0/326298112 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648098870 con 0x55da3c907b40 2024-09-19T09:05:44.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.823+0000 7fb65c00b740 1 -- 172.21.15.120:0/326298112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da3c907b40 msgr2=0x55da3c91d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:44.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.823+0000 7fb65c00b740 1 --2- 172.21.15.120:0/326298112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da3c907b40 0x55da3c91d310 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb6480cfe80 tx=0x7fb64809b8c0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.823+0000 7fb65c00b740 1 -- 172.21.15.120:0/326298112 shutdown_connections 2024-09-19T09:05:44.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.823+0000 7fb65c00b740 1 --2- 172.21.15.120:0/326298112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3c911370 0x55da3c90cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.823+0000 7fb65c00b740 1 --2- 172.21.15.120:0/326298112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da3c907b40 0x55da3c91d310 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.823+0000 7fb65c00b740 1 --2- 172.21.15.120:0/326298112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da3c9065d0 0x55da3c90df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.823+0000 7fb65c00b740 1 -- 172.21.15.120:0/326298112 >> 172.21.15.120:0/326298112 conn(0x55da3c8cf310 msgr2=0x55da3c90c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:44.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.823+0000 7fb65c00b740 1 -- 172.21.15.120:0/326298112 shutdown_connections 2024-09-19T09:05:44.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.823+0000 7fb65c00b740 1 -- 172.21.15.120:0/326298112 wait complete. 2024-09-19T09:05:44.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.823+0000 7fb65c00b740 1 -- start start 2024-09-19T09:05:44.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.823+0000 7fb65c00b740 1 Processor -- start 2024-09-19T09:05:44.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.823+0000 7fb65c00b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da3c9065d0 0x55da3c919480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:44.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.823+0000 7fb65c00b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3c907b40 0x55da3c9199c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:44.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.823+0000 7fb65c00b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da3c911370 0x55da3c90a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:44.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.824+0000 7fb65c00b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb6541b0c90 con 0x55da3c9065d0 2024-09-19T09:05:44.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.824+0000 7fb65c00b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb6541b05c0 con 0x55da3c911370 2024-09-19T09:05:44.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.824+0000 7fb65c00b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb6541ad7e0 con 0x55da3c907b40 2024-09-19T09:05:44.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.824+0000 7fb65c00b740 10 client.0 _ll_register_callbacks cb 0x7fb64c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:44.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.824+0000 7fb6537fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da3c9065d0 0x55da3c919480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:44.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.824+0000 7fb6537fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da3c9065d0 0x55da3c919480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52676/0 (socket says 172.21.15.120:52676) 2024-09-19T09:05:44.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.824+0000 7fb6537fe640 1 -- 172.21.15.120:0/2778383681 learned_addr learned my addr 172.21.15.120:0/2778383681 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:44.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.824+0000 7fb652ffd640 1 --2- 172.21.15.120:0/2778383681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3c907b40 0x55da3c9199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:44.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.824+0000 7fb653fff640 1 --2- 172.21.15.120:0/2778383681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da3c911370 0x55da3c90a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:44.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.824+0000 7fb6537fe640 1 -- 172.21.15.120:0/2778383681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3c907b40 msgr2=0x55da3c9199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:44.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.824+0000 7fb6537fe640 1 --2- 172.21.15.120:0/2778383681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3c907b40 0x55da3c9199c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.824+0000 7fb6537fe640 1 -- 172.21.15.120:0/2778383681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da3c911370 msgr2=0x55da3c90a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:44.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.824+0000 7fb6537fe640 1 --2- 172.21.15.120:0/2778383681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da3c911370 0x55da3c90a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.824+0000 7fb6537fe640 1 -- 172.21.15.120:0/2778383681 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da3c90bed0 con 0x55da3c9065d0 2024-09-19T09:05:44.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.824+0000 7fb652ffd640 1 --2- 172.21.15.120:0/2778383681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3c907b40 0x55da3c9199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:44.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.824+0000 7fb6537fe640 1 --2- 172.21.15.120:0/2778383681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da3c9065d0 0x55da3c919480 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x55da3c923620 tx=0x7fb644012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a05c3f458fe223 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:44.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.824+0000 7fb632ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:44.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.824+0000 7fb632ffd640 1 -- 172.21.15.120:0/2778383681 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb644004220 con 0x55da3c9065d0 2024-09-19T09:05:44.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.824+0000 7fb632ffd640 1 -- 172.21.15.120:0/2778383681 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb6440043c0 con 0x55da3c9065d0 2024-09-19T09:05:44.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.824+0000 7fb65c00b740 1 -- 172.21.15.120:0/2778383681 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6480ba070 con 0x55da3c9065d0 2024-09-19T09:05:44.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.825+0000 7fb632ffd640 1 -- 172.21.15.120:0/2778383681 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb644005020 con 0x55da3c9065d0 2024-09-19T09:05:44.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.825+0000 7fb65c00b740 10 client.16437 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:44.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.825+0000 7fb65c00b740 1 -- 172.21.15.120:0/2778383681 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55da3c98aab0 con 0x55da3c9065d0 2024-09-19T09:05:44.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.825+0000 7fb632ffd640 1 -- 172.21.15.120:0/2778383681 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6480ba070 con 0x55da3c9065d0 2024-09-19T09:05:44.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.825+0000 7fb632ffd640 1 -- 172.21.15.120:0/2778383681 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb64402f020 con 0x55da3c9065d0 2024-09-19T09:05:44.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.825+0000 7fb65c00b740 10 client.16437 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:44.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.825+0000 7fb65c00b740 10 client.16437 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:05:44.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.825+0000 7fb65c00b740 20 client.16437 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:44.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.825+0000 7fb65c00b740 1 --2- 172.21.15.120:0/2778383681 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55da3c98b210 0x55da3c98d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:44.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.825+0000 7fb65c00b740 4 client.16437 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:05:44.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.825+0000 7fb65c00b740 1 -- 172.21.15.120:0/2778383681 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55da3c873d70 con 0x55da3c98b210 2024-09-19T09:05:44.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.825+0000 7fb652ffd640 1 --2- 172.21.15.120:0/2778383681 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55da3c98b210 0x55da3c98d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:44.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.826+0000 7fb632ffd640 10 client.16437 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:05:44.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.826+0000 7fb652ffd640 1 --2- 172.21.15.120:0/2778383681 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55da3c98b210 0x55da3c98d630 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2760e109752994f0 server_cookie=ed27fd3184234a35 in_seq=0 out_seq=0 2024-09-19T09:05:44.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.826+0000 7fb632ffd640 1 -- 172.21.15.120:0/2778383681 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55da3c873d70 con 0x55da3c98b210 2024-09-19T09:05:44.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.826+0000 7fb632ffd640 10 client.16437 handle_command_reply: tid=0 2024-09-19T09:05:44.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.826+0000 7fb65c00b740 1 client.16437 shutdown 2024-09-19T09:05:44.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.826+0000 7fb65c00b740 1 -- 172.21.15.120:0/2778383681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da3c9065d0 msgr2=0x55da3c919480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:44.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.826+0000 7fb65c00b740 1 --2- 172.21.15.120:0/2778383681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da3c9065d0 0x55da3c919480 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x55da3c923620 tx=0x7fb644012820 comp rx=0 tx=0).stop 2024-09-19T09:05:44.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.826+0000 7fb65c00b740 1 -- 172.21.15.120:0/2778383681 shutdown_connections 2024-09-19T09:05:44.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.826+0000 7fb65c00b740 1 --2- 172.21.15.120:0/2778383681 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55da3c98b210 0x55da3c98d630 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.826+0000 7fb65c00b740 1 --2- 172.21.15.120:0/2778383681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da3c911370 0x55da3c90a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.826+0000 7fb65c00b740 1 --2- 172.21.15.120:0/2778383681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3c907b40 0x55da3c9199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.826+0000 7fb65c00b740 1 --2- 172.21.15.120:0/2778383681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da3c9065d0 0x55da3c919480 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.826+0000 7fb65c00b740 1 -- 172.21.15.120:0/2778383681 >> 172.21.15.120:0/2778383681 conn(0x55da3c8cf310 msgr2=0x55da3c902c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:44.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.826+0000 7fb632ffd640 0 client.16437 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:05:44.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.826+0000 7fb65c00b740 1 -- 172.21.15.120:0/2778383681 shutdown_connections 2024-09-19T09:05:44.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.826+0000 7fb65c00b740 1 -- 172.21.15.120:0/2778383681 wait complete. 2024-09-19T09:05:44.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.826+0000 7fb65c00b740 20 client.16437 trim_cache size 0 max 16384 2024-09-19T09:05:44.832 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:05:44.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.827+0000 7fb659c32640 1 -- 172.21.15.120:0/631287143 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb63c0a3240 msgr2=0x7fb63c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:44.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.827+0000 7fb659c32640 1 --2- 172.21.15.120:0/631287143 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb63c0a3240 0x7fb63c0a5700 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb64c004ab0 tx=0x7fb64c0316c0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.827+0000 7fb659c32640 1 -- 172.21.15.120:0/631287143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb654103740 msgr2=0x7fb654100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:44.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.827+0000 7fb659c32640 1 --2- 172.21.15.120:0/631287143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb654103740 0x7fb654100f30 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb64800e9c0 tx=0x7fb64800ee80 comp rx=0 tx=0).stop 2024-09-19T09:05:44.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.827+0000 7fb659c32640 1 -- 172.21.15.120:0/631287143 shutdown_connections 2024-09-19T09:05:44.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.827+0000 7fb659c32640 1 --2- 172.21.15.120:0/631287143 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb63c0a3240 0x7fb63c0a5700 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.827+0000 7fb659c32640 1 --2- 172.21.15.120:0/631287143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb654108290 0x7fb654101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.827+0000 7fb659c32640 1 --2- 172.21.15.120:0/631287143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb654103740 0x7fb654100f30 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.827+0000 7fb659c32640 1 --2- 172.21.15.120:0/631287143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb654102d90 0x7fb6541009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:44.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.827+0000 7fb659c32640 1 -- 172.21.15.120:0/631287143 >> 172.21.15.120:0/631287143 conn(0x7fb6540fc7d0 msgr2=0x7fb654105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:44.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.827+0000 7fb659c32640 1 -- 172.21.15.120:0/631287143 shutdown_connections 2024-09-19T09:05:44.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:44.827+0000 7fb659c32640 1 -- 172.21.15.120:0/631287143 wait complete. 2024-09-19T09:05:45.039 INFO:tasks.workunit.client.0.smithi120.stdout: 1 170815 9.39 MB/sec warmup 100 sec latency 33.359 ms 2024-09-19T09:05:45.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:44 smithi120 ceph-mon[31150]: pgmap v427: 97 pgs: 97 active+clean; 48 MiB data, 591 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.4 MiB/s wr, 487 op/s 2024-09-19T09:05:45.188 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:44 smithi196 ceph-mon[35150]: pgmap v427: 97 pgs: 97 active+clean; 48 MiB data, 591 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.4 MiB/s wr, 487 op/s 2024-09-19T09:05:45.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:44 smithi153 ceph-mon[35071]: pgmap v427: 97 pgs: 97 active+clean; 48 MiB data, 591 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.4 MiB/s wr, 487 op/s 2024-09-19T09:05:45.664 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:05:45.957 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:05:46.039 INFO:tasks.workunit.client.0.smithi120.stdout: 1 174068 9.48 MB/sec warmup 101 sec latency 11.823 ms 2024-09-19T09:05:46.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.501+0000 7f7729f48640 1 -- 172.21.15.120:0/3713944870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7724100a80 msgr2=0x7f7724100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:46.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.501+0000 7f7729f48640 1 --2- 172.21.15.120:0/3713944870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7724100a80 0x7f7724100ee0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f770c009930 tx=0x7f770c02a420 comp rx=0 tx=0).stop 2024-09-19T09:05:46.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.501+0000 7f7729f48640 1 -- 172.21.15.120:0/3713944870 shutdown_connections 2024-09-19T09:05:46.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.501+0000 7f7729f48640 1 --2- 172.21.15.120:0/3713944870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7724101420 0x7f772410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.501+0000 7f7729f48640 1 --2- 172.21.15.120:0/3713944870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7724100a80 0x7f7724100ee0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.501+0000 7f7729f48640 1 --2- 172.21.15.120:0/3713944870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7724106a80 0x7f7724106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.501+0000 7f7729f48640 1 -- 172.21.15.120:0/3713944870 >> 172.21.15.120:0/3713944870 conn(0x7f77240fc7a0 msgr2=0x7f77240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:46.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.502+0000 7f7729f48640 1 -- 172.21.15.120:0/3713944870 shutdown_connections 2024-09-19T09:05:46.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.502+0000 7f7729f48640 1 -- 172.21.15.120:0/3713944870 wait complete. 2024-09-19T09:05:46.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.502+0000 7f7729f48640 1 Processor -- start 2024-09-19T09:05:46.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.502+0000 7f7729f48640 1 -- start start 2024-09-19T09:05:46.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.502+0000 7f7729f48640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7724100a80 0x7f77241983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:46.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.502+0000 7f7729f48640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7724101420 0x7f7724198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:46.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.502+0000 7f7729f48640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7724106a80 0x7f772419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:46.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.502+0000 7f7729f48640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7724110e40 con 0x7f7724106a80 2024-09-19T09:05:46.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.502+0000 7f7729f48640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7724110cc0 con 0x7f7724100a80 2024-09-19T09:05:46.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.502+0000 7f7729f48640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7724110fc0 con 0x7f7724101420 2024-09-19T09:05:46.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.502+0000 7f7723fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7724106a80 0x7f772419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:46.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.503+0000 7f7723fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7724106a80 0x7f772419ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52698/0 (socket says 172.21.15.120:52698) 2024-09-19T09:05:46.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.503+0000 7f77237fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7724100a80 0x7f77241983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:46.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.503+0000 7f7723fff640 1 -- 172.21.15.120:0/1716445561 learned_addr learned my addr 172.21.15.120:0/1716445561 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:46.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.503+0000 7f771bfff640 1 --2- 172.21.15.120:0/1716445561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7724101420 0x7f7724198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:46.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.503+0000 7f7723fff640 1 -- 172.21.15.120:0/1716445561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7724101420 msgr2=0x7f7724198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:46.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.503+0000 7f7723fff640 1 --2- 172.21.15.120:0/1716445561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7724101420 0x7f7724198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.503+0000 7f7723fff640 1 -- 172.21.15.120:0/1716445561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7724100a80 msgr2=0x7f77241983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:46.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.503+0000 7f7723fff640 1 --2- 172.21.15.120:0/1716445561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7724100a80 0x7f77241983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.503+0000 7f7723fff640 1 -- 172.21.15.120:0/1716445561 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f772419d3a0 con 0x7f7724106a80 2024-09-19T09:05:46.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.503+0000 7f7723fff640 1 --2- 172.21.15.120:0/1716445561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7724106a80 0x7f772419ccc0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f771c00eb10 tx=0x7f771c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:46.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.503+0000 7f77217fa640 1 -- 172.21.15.120:0/1716445561 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c004b20 con 0x7f7724106a80 2024-09-19T09:05:46.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.503+0000 7f7729f48640 1 -- 172.21.15.120:0/1716445561 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f772419d690 con 0x7f7724106a80 2024-09-19T09:05:46.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.503+0000 7f77217fa640 1 -- 172.21.15.120:0/1716445561 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f771c004cc0 con 0x7f7724106a80 2024-09-19T09:05:46.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.503+0000 7f77217fa640 1 -- 172.21.15.120:0/1716445561 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c010760 con 0x7f7724106a80 2024-09-19T09:05:46.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.503+0000 7f7729f48640 1 -- 172.21.15.120:0/1716445561 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77241a4f70 con 0x7f7724106a80 2024-09-19T09:05:46.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.505+0000 7f77217fa640 1 -- 172.21.15.120:0/1716445561 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f771c005020 con 0x7f7724106a80 2024-09-19T09:05:46.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.506+0000 7f772c321740 1 Processor -- start 2024-09-19T09:05:46.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.506+0000 7f772c321740 1 -- start start 2024-09-19T09:05:46.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.506+0000 7f772c321740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f47987fb40 0x55f47987e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:46.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.506+0000 7f77217fa640 1 --2- 172.21.15.120:0/1716445561 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f77100a3240 0x7f77100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:46.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.506+0000 7f772c321740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f47987e5d0 0x55f47987ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:46.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.506+0000 7f772c321740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f479889370 0x55f47988b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:46.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.506+0000 7f772c321740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f4797e42f0 con 0x55f47987fb40 2024-09-19T09:05:46.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.506+0000 7f772c321740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f47978bcf0 con 0x55f479889370 2024-09-19T09:05:46.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.506+0000 7f772c321740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f4797e02b0 con 0x55f47987e5d0 2024-09-19T09:05:46.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.506+0000 7f771bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f47987fb40 0x55f47987e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:46.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.506+0000 7f77237fe640 1 --2- 172.21.15.120:0/1716445561 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f77100a3240 0x7f77100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:46.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.506+0000 7f771bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f47987fb40 0x55f47987e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52708/0 (socket says 172.21.15.120:52708) 2024-09-19T09:05:46.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.506+0000 7f771bfff640 1 -- 172.21.15.120:0/3950942053 learned_addr learned my addr 172.21.15.120:0/3950942053 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:46.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.506+0000 7f7723fff640 1 --2- 172.21.15.120:0/3950942053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f479889370 0x55f47988b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:46.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.506+0000 7f771bfff640 1 --2- 172.21.15.120:0/3950942053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f47987e5d0 0x55f47987ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:46.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.507+0000 7f77217fa640 1 -- 172.21.15.120:0/1716445561 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f771c0cfec0 con 0x7f7724106a80 2024-09-19T09:05:46.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.507+0000 7f7723fff640 1 -- 172.21.15.120:0/3950942053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f47987e5d0 msgr2=0x55f47987ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:46.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.507+0000 7f7723fff640 1 --2- 172.21.15.120:0/3950942053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f47987e5d0 0x55f47987ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.507+0000 7f7723fff640 1 -- 172.21.15.120:0/3950942053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f47987fb40 msgr2=0x55f47987e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:46.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.507+0000 7f7723fff640 1 --2- 172.21.15.120:0/3950942053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f47987fb40 0x55f47987e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.507+0000 7f7723fff640 1 -- 172.21.15.120:0/3950942053 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4797703a0 con 0x55f479889370 2024-09-19T09:05:46.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.507+0000 7f77237fe640 1 --2- 172.21.15.120:0/1716445561 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f77100a3240 0x7f77100a5700 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7714009b50 tx=0x7f7714005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:46.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.507+0000 7f7723fff640 1 --2- 172.21.15.120:0/3950942053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f479889370 0x55f47988b760 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f771c020000 tx=0x7f771c0980e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b20cd2f207a993c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:46.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.507+0000 7f7719ffb640 1 -- 172.21.15.120:0/3950942053 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c09be40 con 0x55f479889370 2024-09-19T09:05:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.507+0000 7f7719ffb640 1 -- 172.21.15.120:0/3950942053 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f771c09cbe0 con 0x55f479889370 2024-09-19T09:05:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.507+0000 7f7719ffb640 1 -- 172.21.15.120:0/3950942053 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c098b50 con 0x55f479889370 2024-09-19T09:05:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.508+0000 7f772c321740 1 -- 172.21.15.120:0/3950942053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f479889370 msgr2=0x55f47988b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.508+0000 7f772c321740 1 --2- 172.21.15.120:0/3950942053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f479889370 0x55f47988b760 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f771c020000 tx=0x7f771c0980e0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.508+0000 7f772c321740 1 -- 172.21.15.120:0/3950942053 shutdown_connections 2024-09-19T09:05:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.508+0000 7f772c321740 1 --2- 172.21.15.120:0/3950942053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f479889370 0x55f47988b760 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.508+0000 7f772c321740 1 --2- 172.21.15.120:0/3950942053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f47987e5d0 0x55f47987ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.508+0000 7f772c321740 1 --2- 172.21.15.120:0/3950942053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f47987fb40 0x55f47987e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.508+0000 7f772c321740 1 -- 172.21.15.120:0/3950942053 >> 172.21.15.120:0/3950942053 conn(0x55f479847310 msgr2=0x55f47987ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.508+0000 7f772c321740 1 -- 172.21.15.120:0/3950942053 shutdown_connections 2024-09-19T09:05:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.508+0000 7f772c321740 1 -- 172.21.15.120:0/3950942053 wait complete. 2024-09-19T09:05:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.508+0000 7f772c321740 1 -- start start 2024-09-19T09:05:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.509+0000 7f772c321740 1 Processor -- start 2024-09-19T09:05:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.509+0000 7f772c321740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f47987e5d0 0x55f479888d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.509+0000 7f772c321740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f47987fb40 0x55f479886040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.509+0000 7f772c321740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f479889370 0x55f4798866a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.509+0000 7f772c321740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f47988c070 con 0x55f47987fb40 2024-09-19T09:05:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.509+0000 7f772c321740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f47979f060 con 0x55f47987e5d0 2024-09-19T09:05:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.509+0000 7f772c321740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f47988c1f0 con 0x55f479889370 2024-09-19T09:05:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.509+0000 7f772c321740 10 client.0 _ll_register_callbacks cb 0x55f4794f28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.509+0000 7f77237fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f47987e5d0 0x55f479888d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.509+0000 7f771bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f47987fb40 0x55f479886040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.509+0000 7f77237fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f47987e5d0 0x55f479888d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:45982/0 (socket says 172.21.15.120:45982) 2024-09-19T09:05:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.509+0000 7f77237fe640 1 -- 172.21.15.120:0/1902946407 learned_addr learned my addr 172.21.15.120:0/1902946407 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.509+0000 7f7723fff640 1 --2- 172.21.15.120:0/1902946407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f479889370 0x55f4798866a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.509+0000 7f7723fff640 1 -- 172.21.15.120:0/1902946407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f47987e5d0 msgr2=0x55f479888d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.509+0000 7f7723fff640 1 --2- 172.21.15.120:0/1902946407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f47987e5d0 0x55f479888d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.509+0000 7f7723fff640 1 -- 172.21.15.120:0/1902946407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f47987fb40 msgr2=0x55f479886040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.509+0000 7f7723fff640 1 --2- 172.21.15.120:0/1902946407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f47987fb40 0x55f479886040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.509+0000 7f7723fff640 1 -- 172.21.15.120:0/1902946407 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4795553c0 con 0x55f479889370 2024-09-19T09:05:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.509+0000 7f77237fe640 1 --2- 172.21.15.120:0/1902946407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f47987e5d0 0x55f479888d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.509+0000 7f771bfff640 1 --2- 172.21.15.120:0/1902946407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f47987fb40 0x55f479886040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.510+0000 7f7723fff640 1 --2- 172.21.15.120:0/1902946407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f479889370 0x55f4798866a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55f47987f070 tx=0x7f771c09c150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c66aa33d6c5f044b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.510+0000 7f7702ffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:05:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.510+0000 7f7702ffd640 1 -- 172.21.15.120:0/1902946407 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c09be40 con 0x55f479889370 2024-09-19T09:05:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.510+0000 7f7702ffd640 1 -- 172.21.15.120:0/1902946407 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f771c0a4ec0 con 0x55f479889370 2024-09-19T09:05:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.510+0000 7f772c321740 1 -- 172.21.15.120:0/1902946407 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f771c09cbe0 con 0x55f479889370 2024-09-19T09:05:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.510+0000 7f7702ffd640 1 -- 172.21.15.120:0/1902946407 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f771c09c4e0 con 0x55f479889370 2024-09-19T09:05:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.510+0000 7f772c321740 10 client.25774 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.510+0000 7f772c321740 1 -- 172.21.15.120:0/1902946407 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f47988c070 con 0x55f479889370 2024-09-19T09:05:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.510+0000 7f7702ffd640 1 -- 172.21.15.120:0/1902946407 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f771c09cbe0 con 0x55f479889370 2024-09-19T09:05:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.510+0000 7f7702ffd640 1 -- 172.21.15.120:0/1902946407 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f771c098230 con 0x55f479889370 2024-09-19T09:05:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.510+0000 7f772c321740 10 client.25774 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.510+0000 7f772c321740 10 client.25774 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:05:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.510+0000 7f772c321740 20 client.25774 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.510+0000 7f772c321740 1 --2- 172.21.15.120:0/1902946407 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55f479898ca0 0x55f47989b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.511+0000 7f772c321740 4 client.25774 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:05:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.511+0000 7f772c321740 1 -- 172.21.15.120:0/1902946407 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4797f1860 con 0x55f479898ca0 2024-09-19T09:05:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.511+0000 7f77237fe640 1 --2- 172.21.15.120:0/1902946407 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55f479898ca0 0x55f47989b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.511+0000 7f77237fe640 1 --2- 172.21.15.120:0/1902946407 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55f479898ca0 0x55f47989b090 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a8681da275457268 server_cookie=7204732ebc3c15c9 in_seq=0 out_seq=0 2024-09-19T09:05:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.511+0000 7f7702ffd640 10 client.25774 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:05:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.512+0000 7f7702ffd640 1 -- 172.21.15.120:0/1902946407 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4797f1860 con 0x55f479898ca0 2024-09-19T09:05:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.512+0000 7f7702ffd640 10 client.25774 handle_command_reply: tid=0 2024-09-19T09:05:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.512+0000 7f772c321740 1 client.25774 shutdown 2024-09-19T09:05:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.512+0000 7f772c321740 1 -- 172.21.15.120:0/1902946407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f479889370 msgr2=0x55f4798866a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.512+0000 7f772c321740 1 --2- 172.21.15.120:0/1902946407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f479889370 0x55f4798866a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55f47987f070 tx=0x7f771c09c150 comp rx=0 tx=0).stop 2024-09-19T09:05:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.512+0000 7f772c321740 1 -- 172.21.15.120:0/1902946407 shutdown_connections 2024-09-19T09:05:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.512+0000 7f772c321740 1 --2- 172.21.15.120:0/1902946407 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55f479898ca0 0x55f47989b090 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.512+0000 7f772c321740 1 --2- 172.21.15.120:0/1902946407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f479889370 0x55f4798866a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.512+0000 7f772c321740 1 --2- 172.21.15.120:0/1902946407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f47987fb40 0x55f479886040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.512+0000 7f772c321740 1 --2- 172.21.15.120:0/1902946407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f47987e5d0 0x55f479888d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.512+0000 7f7702ffd640 0 client.25774 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:05:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.512+0000 7f772c321740 1 -- 172.21.15.120:0/1902946407 >> 172.21.15.120:0/1902946407 conn(0x55f479847310 msgr2=0x55f4798809c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.512+0000 7f772c321740 1 -- 172.21.15.120:0/1902946407 shutdown_connections 2024-09-19T09:05:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.512+0000 7f772c321740 1 -- 172.21.15.120:0/1902946407 wait complete. 2024-09-19T09:05:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.512+0000 7f772c321740 20 client.25774 trim_cache size 0 max 16384 2024-09-19T09:05:46.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.526+0000 7f772c321740 1 Processor -- start 2024-09-19T09:05:46.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.526+0000 7f772c321740 1 -- start start 2024-09-19T09:05:46.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.526+0000 7f772c321740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f47987e5d0 0x55f479895310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:46.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.526+0000 7f772c321740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f47987fb40 0x55f479885020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:46.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.526+0000 7f772c321740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f479889370 0x55f4798a0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:46.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.526+0000 7f772c321740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f77241ad5e0 con 0x55f479889370 2024-09-19T09:05:46.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.526+0000 7f772c321740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f77241af120 con 0x55f47987e5d0 2024-09-19T09:05:46.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.526+0000 7f772c321740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f77241adcb0 con 0x55f47987fb40 2024-09-19T09:05:46.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.526+0000 7f7723fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f479889370 0x55f4798a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:46.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.526+0000 7f7723fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f479889370 0x55f4798a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52730/0 (socket says 172.21.15.120:52730) 2024-09-19T09:05:46.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.526+0000 7f7723fff640 1 -- 172.21.15.120:0/3012149313 learned_addr learned my addr 172.21.15.120:0/3012149313 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:46.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.526+0000 7f77237fe640 1 --2- 172.21.15.120:0/3012149313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f47987e5d0 0x55f479895310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:46.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.526+0000 7f771bfff640 1 --2- 172.21.15.120:0/3012149313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f47987fb40 0x55f479885020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:46.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.527+0000 7f7723fff640 1 -- 172.21.15.120:0/3012149313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f47987fb40 msgr2=0x55f479885020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:46.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.527+0000 7f7723fff640 1 --2- 172.21.15.120:0/3012149313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f47987fb40 0x55f479885020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.527+0000 7f7723fff640 1 -- 172.21.15.120:0/3012149313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f47987e5d0 msgr2=0x55f479895310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:46.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.527+0000 7f7723fff640 1 --2- 172.21.15.120:0/3012149313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f47987e5d0 0x55f479895310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.527+0000 7f7723fff640 1 -- 172.21.15.120:0/3012149313 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f47987bd80 con 0x55f479889370 2024-09-19T09:05:46.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.527+0000 7f7723fff640 1 --2- 172.21.15.120:0/3012149313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f479889370 0x55f4798a0720 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f771c09c470 tx=0x7f771c00efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575720cb54f8fd83 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:46.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.527+0000 7f77197fa640 1 -- 172.21.15.120:0/3012149313 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c09be40 con 0x55f479889370 2024-09-19T09:05:46.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.527+0000 7f77197fa640 1 -- 172.21.15.120:0/3012149313 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f771c09cbe0 con 0x55f479889370 2024-09-19T09:05:46.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.527+0000 7f77197fa640 1 -- 172.21.15.120:0/3012149313 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c0a4cc0 con 0x55f479889370 2024-09-19T09:05:46.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.528+0000 7f772c321740 1 -- 172.21.15.120:0/3012149313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f479889370 msgr2=0x55f4798a0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:46.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.528+0000 7f772c321740 1 --2- 172.21.15.120:0/3012149313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f479889370 0x55f4798a0720 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f771c09c470 tx=0x7f771c00efd0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.528+0000 7f772c321740 1 -- 172.21.15.120:0/3012149313 shutdown_connections 2024-09-19T09:05:46.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.528+0000 7f772c321740 1 --2- 172.21.15.120:0/3012149313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f479889370 0x55f4798a0720 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.528+0000 7f772c321740 1 --2- 172.21.15.120:0/3012149313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f47987fb40 0x55f479885020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.528+0000 7f772c321740 1 --2- 172.21.15.120:0/3012149313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f47987e5d0 0x55f479895310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.528+0000 7f772c321740 1 -- 172.21.15.120:0/3012149313 >> 172.21.15.120:0/3012149313 conn(0x55f479847310 msgr2=0x55f479884910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:46.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.528+0000 7f772c321740 1 -- 172.21.15.120:0/3012149313 shutdown_connections 2024-09-19T09:05:46.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.528+0000 7f772c321740 1 -- 172.21.15.120:0/3012149313 wait complete. 2024-09-19T09:05:46.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.528+0000 7f772c321740 1 -- start start 2024-09-19T09:05:46.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.528+0000 7f772c321740 1 Processor -- start 2024-09-19T09:05:46.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.529+0000 7f772c321740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f47987e5d0 0x55f47988cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:46.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.529+0000 7f772c321740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f47987fb40 0x55f47988d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:46.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.529+0000 7f772c321740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f479889370 0x55f4799016e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:46.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.529+0000 7f772c321740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f77241aea50 con 0x55f479889370 2024-09-19T09:05:46.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.529+0000 7f772c321740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f77241ae380 con 0x55f47987fb40 2024-09-19T09:05:46.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.529+0000 7f772c321740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f77241ab5a0 con 0x55f47987e5d0 2024-09-19T09:05:46.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.529+0000 7f772c321740 10 client.0 _ll_register_callbacks cb 0x7f7714005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:46.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.529+0000 7f7723fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f479889370 0x55f4799016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:46.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.529+0000 7f77237fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f47987e5d0 0x55f47988cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:46.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.529+0000 7f771bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f47987fb40 0x55f47988d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:46.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.529+0000 7f7723fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f479889370 0x55f4799016e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52734/0 (socket says 172.21.15.120:52734) 2024-09-19T09:05:46.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.529+0000 7f7723fff640 1 -- 172.21.15.120:0/2790727607 learned_addr learned my addr 172.21.15.120:0/2790727607 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:46.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.529+0000 7f7723fff640 1 -- 172.21.15.120:0/2790727607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f47987e5d0 msgr2=0x55f47988cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:46.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.529+0000 7f7723fff640 1 --2- 172.21.15.120:0/2790727607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f47987e5d0 0x55f47988cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.529+0000 7f7723fff640 1 -- 172.21.15.120:0/2790727607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f47987fb40 msgr2=0x55f47988d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:46.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.529+0000 7f7723fff640 1 --2- 172.21.15.120:0/2790727607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f47987fb40 0x55f47988d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.529+0000 7f7723fff640 1 -- 172.21.15.120:0/2790727607 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f479901d50 con 0x55f479889370 2024-09-19T09:05:46.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.529+0000 7f7723fff640 1 --2- 172.21.15.120:0/2790727607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f479889370 0x55f4799016e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f771c098e60 tx=0x7f771c0cdc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fff15c0ffde4a54b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:46.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.529+0000 7f7702ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:46.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.529+0000 7f7702ffd640 1 -- 172.21.15.120:0/2790727607 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c09be40 con 0x55f479889370 2024-09-19T09:05:46.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.529+0000 7f7702ffd640 1 -- 172.21.15.120:0/2790727607 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f771c0a4cc0 con 0x55f479889370 2024-09-19T09:05:46.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.529+0000 7f772c321740 1 -- 172.21.15.120:0/2790727607 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f771c09cbe0 con 0x55f479889370 2024-09-19T09:05:46.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.530+0000 7f7702ffd640 1 -- 172.21.15.120:0/2790727607 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f771c0a5150 con 0x55f479889370 2024-09-19T09:05:46.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.530+0000 7f772c321740 10 client.16455 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:46.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.530+0000 7f772c321740 1 -- 172.21.15.120:0/2790727607 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f479907440 con 0x55f479889370 2024-09-19T09:05:46.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.530+0000 7f7702ffd640 1 -- 172.21.15.120:0/2790727607 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f771c09cbe0 con 0x55f479889370 2024-09-19T09:05:46.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.530+0000 7f7702ffd640 1 -- 172.21.15.120:0/2790727607 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f771c09ba50 con 0x55f479889370 2024-09-19T09:05:46.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.530+0000 7f772c321740 10 client.16455 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:46.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.530+0000 7f772c321740 10 client.16455 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:05:46.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.530+0000 7f772c321740 20 client.16455 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:46.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.530+0000 7f772c321740 1 --2- 172.21.15.120:0/2790727607 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55f479907870 0x55f479909c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:46.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.530+0000 7f772c321740 4 client.16455 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:05:46.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.530+0000 7f772c321740 1 -- 172.21.15.120:0/2790727607 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f4797ebd70 con 0x55f479907870 2024-09-19T09:05:46.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.530+0000 7f77237fe640 1 --2- 172.21.15.120:0/2790727607 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55f479907870 0x55f479909c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:46.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.531+0000 7f7702ffd640 10 client.16455 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:05:46.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.531+0000 7f77237fe640 1 --2- 172.21.15.120:0/2790727607 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55f479907870 0x55f479909c60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f722bbcf3629481e server_cookie=21b459ae08069c2b in_seq=0 out_seq=0 2024-09-19T09:05:46.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.531+0000 7f7702ffd640 1 -- 172.21.15.120:0/2790727607 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f4797ebd70 con 0x55f479907870 2024-09-19T09:05:46.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.531+0000 7f7702ffd640 10 client.16455 handle_command_reply: tid=0 2024-09-19T09:05:46.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.531+0000 7f772c321740 1 client.16455 shutdown 2024-09-19T09:05:46.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.531+0000 7f772c321740 1 -- 172.21.15.120:0/2790727607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f479889370 msgr2=0x55f4799016e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:46.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.531+0000 7f772c321740 1 --2- 172.21.15.120:0/2790727607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f479889370 0x55f4799016e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f771c098e60 tx=0x7f771c0cdc50 comp rx=0 tx=0).stop 2024-09-19T09:05:46.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.531+0000 7f772c321740 1 -- 172.21.15.120:0/2790727607 shutdown_connections 2024-09-19T09:05:46.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.531+0000 7f772c321740 1 --2- 172.21.15.120:0/2790727607 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55f479907870 0x55f479909c60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.531+0000 7f772c321740 1 --2- 172.21.15.120:0/2790727607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f479889370 0x55f4799016e0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.531+0000 7f772c321740 1 --2- 172.21.15.120:0/2790727607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f47987fb40 0x55f47988d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.531+0000 7f772c321740 1 --2- 172.21.15.120:0/2790727607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f47987e5d0 0x55f47988cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.531+0000 7f772c321740 1 -- 172.21.15.120:0/2790727607 >> 172.21.15.120:0/2790727607 conn(0x55f479847310 msgr2=0x55f479885130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:46.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.531+0000 7f7702ffd640 0 client.16455 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:05:46.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.531+0000 7f772c321740 1 -- 172.21.15.120:0/2790727607 shutdown_connections 2024-09-19T09:05:46.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.531+0000 7f772c321740 1 -- 172.21.15.120:0/2790727607 wait complete. 2024-09-19T09:05:46.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.531+0000 7f772c321740 20 client.16455 trim_cache size 0 max 16384 2024-09-19T09:05:46.541 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:05:46.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.532+0000 7f7729f48640 1 -- 172.21.15.120:0/1716445561 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f77100a3240 msgr2=0x7f77100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:46.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.532+0000 7f7729f48640 1 --2- 172.21.15.120:0/1716445561 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f77100a3240 0x7f77100a5700 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7714009b50 tx=0x7f7714005f90 comp rx=0 tx=0).stop 2024-09-19T09:05:46.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.532+0000 7f7729f48640 1 -- 172.21.15.120:0/1716445561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7724106a80 msgr2=0x7f772419ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:46.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.532+0000 7f7729f48640 1 --2- 172.21.15.120:0/1716445561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7724106a80 0x7f772419ccc0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f771c00eb10 tx=0x7f771c00c2d0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.532+0000 7f7729f48640 1 -- 172.21.15.120:0/1716445561 shutdown_connections 2024-09-19T09:05:46.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.532+0000 7f7729f48640 1 --2- 172.21.15.120:0/1716445561 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f77100a3240 0x7f77100a5700 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.532+0000 7f7729f48640 1 --2- 172.21.15.120:0/1716445561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7724106a80 0x7f772419ccc0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.532+0000 7f7729f48640 1 --2- 172.21.15.120:0/1716445561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7724101420 0x7f7724198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.532+0000 7f7729f48640 1 --2- 172.21.15.120:0/1716445561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7724100a80 0x7f77241983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:46.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.532+0000 7f7729f48640 1 -- 172.21.15.120:0/1716445561 >> 172.21.15.120:0/1716445561 conn(0x7f77240fc7a0 msgr2=0x7f772406aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:46.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.532+0000 7f7729f48640 1 -- 172.21.15.120:0/1716445561 shutdown_connections 2024-09-19T09:05:46.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:46.532+0000 7f7729f48640 1 -- 172.21.15.120:0/1716445561 wait complete. 2024-09-19T09:05:46.795 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:46 smithi120 ceph-mon[31150]: pgmap v428: 97 pgs: 97 active+clean; 40 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 433 op/s 2024-09-19T09:05:47.039 INFO:tasks.workunit.client.0.smithi120.stdout: 1 177197 9.53 MB/sec warmup 102 sec latency 17.015 ms 2024-09-19T09:05:47.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:46 smithi196 ceph-mon[35150]: pgmap v428: 97 pgs: 97 active+clean; 40 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 433 op/s 2024-09-19T09:05:47.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:46 smithi153 ceph-mon[35071]: pgmap v428: 97 pgs: 97 active+clean; 40 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 433 op/s 2024-09-19T09:05:47.386 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:05:47.635 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:05:48.039 INFO:tasks.workunit.client.0.smithi120.stdout: 1 180124 9.59 MB/sec warmup 103 sec latency 17.378 ms 2024-09-19T09:05:48.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.166+0000 7f231ad4d640 1 -- 172.21.15.120:0/107227894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2314102ef0 msgr2=0x7f231410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:48.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.166+0000 7f231ad4d640 1 --2- 172.21.15.120:0/107227894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2314102ef0 0x7f231410da40 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f230c009900 tx=0x7f230c02a480 comp rx=0 tx=0).stop 2024-09-19T09:05:48.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.166+0000 7f231ad4d640 1 -- 172.21.15.120:0/107227894 shutdown_connections 2024-09-19T09:05:48.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.166+0000 7f231ad4d640 1 --2- 172.21.15.120:0/107227894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2314102ef0 0x7f231410da40 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.166+0000 7f231ad4d640 1 --2- 172.21.15.120:0/107227894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2314102570 0x7f23141029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.166+0000 7f231ad4d640 1 --2- 172.21.15.120:0/107227894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2314100e20 0x7f2314101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.166+0000 7f231ad4d640 1 -- 172.21.15.120:0/107227894 >> 172.21.15.120:0/107227894 conn(0x7f23140fc7d0 msgr2=0x7f23140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:48.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.166+0000 7f231ad4d640 1 -- 172.21.15.120:0/107227894 shutdown_connections 2024-09-19T09:05:48.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.166+0000 7f231ad4d640 1 -- 172.21.15.120:0/107227894 wait complete. 2024-09-19T09:05:48.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.167+0000 7f231ad4d640 1 Processor -- start 2024-09-19T09:05:48.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.167+0000 7f231ad4d640 1 -- start start 2024-09-19T09:05:48.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.167+0000 7f231ad4d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2314100e20 0x7f2314197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:48.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.167+0000 7f231ad4d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2314102570 0x7f2314197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:48.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.167+0000 7f231ad4d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2314102ef0 0x7f231419bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:48.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.167+0000 7f231ad4d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2314110f80 con 0x7f2314102570 2024-09-19T09:05:48.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.167+0000 7f231ad4d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2314110e00 con 0x7f2314100e20 2024-09-19T09:05:48.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.167+0000 7f231ad4d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2314111100 con 0x7f2314102ef0 2024-09-19T09:05:48.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.167+0000 7f2313fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2314102570 0x7f2314197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:48.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.167+0000 7f2318ae9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2314100e20 0x7f2314197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:48.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.167+0000 7f2313fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2314102570 0x7f2314197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52738/0 (socket says 172.21.15.120:52738) 2024-09-19T09:05:48.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.167+0000 7f2313fff640 1 -- 172.21.15.120:0/1901585012 learned_addr learned my addr 172.21.15.120:0/1901585012 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:48.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.167+0000 7f23192ea640 1 --2- 172.21.15.120:0/1901585012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2314102ef0 0x7f231419bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:48.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.168+0000 7f2313fff640 1 -- 172.21.15.120:0/1901585012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2314102ef0 msgr2=0x7f231419bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:48.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.168+0000 7f2313fff640 1 --2- 172.21.15.120:0/1901585012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2314102ef0 0x7f231419bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.168+0000 7f2313fff640 1 -- 172.21.15.120:0/1901585012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2314100e20 msgr2=0x7f2314197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:48.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.168+0000 7f2313fff640 1 --2- 172.21.15.120:0/1901585012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2314100e20 0x7f2314197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.168+0000 7f2313fff640 1 -- 172.21.15.120:0/1901585012 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f231419c3f0 con 0x7f2314102570 2024-09-19T09:05:48.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.168+0000 7f2313fff640 1 --2- 172.21.15.120:0/1901585012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2314102570 0x7f2314197980 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f230800ebc0 tx=0x7f230800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:48.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.168+0000 7f2311ffb640 1 -- 172.21.15.120:0/1901585012 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308010630 con 0x7f2314102570 2024-09-19T09:05:48.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.168+0000 7f231ad4d640 1 -- 172.21.15.120:0/1901585012 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f231419c6e0 con 0x7f2314102570 2024-09-19T09:05:48.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.168+0000 7f2311ffb640 1 -- 172.21.15.120:0/1901585012 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2308010cb0 con 0x7f2314102570 2024-09-19T09:05:48.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.168+0000 7f2311ffb640 1 -- 172.21.15.120:0/1901585012 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308018bd0 con 0x7f2314102570 2024-09-19T09:05:48.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.168+0000 7f231ad4d640 1 -- 172.21.15.120:0/1901585012 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2314069f70 con 0x7f2314102570 2024-09-19T09:05:48.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.171+0000 7f2311ffb640 1 -- 172.21.15.120:0/1901585012 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f23080107d0 con 0x7f2314102570 2024-09-19T09:05:48.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.172+0000 7f231d126740 1 Processor -- start 2024-09-19T09:05:48.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.172+0000 7f231d126740 1 -- start start 2024-09-19T09:05:48.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.172+0000 7f231d126740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c50430b40 0x559c5042f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:48.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.172+0000 7f231d126740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c5042f5d0 0x559c5042fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:48.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.172+0000 7f231d126740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c5043e780 0x559c50440b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:48.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.172+0000 7f231d126740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559c503952f0 con 0x559c5043e780 2024-09-19T09:05:48.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.172+0000 7f231d126740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559c5033ccf0 con 0x559c50430b40 2024-09-19T09:05:48.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.172+0000 7f231d126740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559c503912b0 con 0x559c5042f5d0 2024-09-19T09:05:48.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.172+0000 7f2313fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c5043e780 0x559c50440b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:48.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.172+0000 7f2318ae9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c5042f5d0 0x559c5042fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:48.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.172+0000 7f2313fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c5043e780 0x559c50440b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52748/0 (socket says 172.21.15.120:52748) 2024-09-19T09:05:48.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.172+0000 7f2313fff640 1 -- 172.21.15.120:0/4234628072 learned_addr learned my addr 172.21.15.120:0/4234628072 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:48.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.172+0000 7f2311ffb640 1 --2- 172.21.15.120:0/1901585012 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f22fc0a3240 0x7f22fc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:48.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.172+0000 7f23192ea640 1 --2- 172.21.15.120:0/4234628072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c50430b40 0x559c5042f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:48.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.173+0000 7f23192ea640 1 --2- 172.21.15.120:0/1901585012 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f22fc0a3240 0x7f22fc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:48.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.173+0000 7f2311ffb640 1 -- 172.21.15.120:0/1901585012 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f23080cfe70 con 0x7f2314102570 2024-09-19T09:05:48.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.173+0000 7f2313fff640 1 -- 172.21.15.120:0/4234628072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c5042f5d0 msgr2=0x559c5042fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:48.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.173+0000 7f2313fff640 1 --2- 172.21.15.120:0/4234628072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c5042f5d0 0x559c5042fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.173+0000 7f2313fff640 1 -- 172.21.15.120:0/4234628072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c50430b40 msgr2=0x559c5042f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:48.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.173+0000 7f2313fff640 1 --2- 172.21.15.120:0/4234628072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c50430b40 0x559c5042f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.173+0000 7f2313fff640 1 -- 172.21.15.120:0/4234628072 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c503213a0 con 0x559c5043e780 2024-09-19T09:05:48.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.173+0000 7f2313fff640 1 --2- 172.21.15.120:0/4234628072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c5043e780 0x559c50440b70 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f230809c240 tx=0x7f230800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d718660c2949523d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:48.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.173+0000 7f23192ea640 1 --2- 172.21.15.120:0/1901585012 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f22fc0a3240 0x7f22fc0a5700 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f230c0096c0 tx=0x7f230c02a480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:48.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.173+0000 7f22faffd640 1 -- 172.21.15.120:0/4234628072 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308010e60 con 0x559c5043e780 2024-09-19T09:05:48.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.173+0000 7f22faffd640 1 -- 172.21.15.120:0/4234628072 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f230809c540 con 0x559c5043e780 2024-09-19T09:05:48.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.173+0000 7f22faffd640 1 -- 172.21.15.120:0/4234628072 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23080cda80 con 0x559c5043e780 2024-09-19T09:05:48.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.174+0000 7f231d126740 1 -- 172.21.15.120:0/4234628072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c5043e780 msgr2=0x559c50440b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:48.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.174+0000 7f231d126740 1 --2- 172.21.15.120:0/4234628072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c5043e780 0x559c50440b70 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f230809c240 tx=0x7f230800cf90 comp rx=0 tx=0).stop 2024-09-19T09:05:48.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.174+0000 7f231d126740 1 -- 172.21.15.120:0/4234628072 shutdown_connections 2024-09-19T09:05:48.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.174+0000 7f231d126740 1 --2- 172.21.15.120:0/4234628072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c5043e780 0x559c50440b70 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.174+0000 7f231d126740 1 --2- 172.21.15.120:0/4234628072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c5042f5d0 0x559c5042fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.174+0000 7f231d126740 1 --2- 172.21.15.120:0/4234628072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c50430b40 0x559c5042f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.174+0000 7f231d126740 1 -- 172.21.15.120:0/4234628072 >> 172.21.15.120:0/4234628072 conn(0x559c503f8310 msgr2=0x559c5042bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:48.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.174+0000 7f231d126740 1 -- 172.21.15.120:0/4234628072 shutdown_connections 2024-09-19T09:05:48.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.174+0000 7f231d126740 1 -- 172.21.15.120:0/4234628072 wait complete. 2024-09-19T09:05:48.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.175+0000 7f231d126740 1 -- start start 2024-09-19T09:05:48.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.175+0000 7f231d126740 1 Processor -- start 2024-09-19T09:05:48.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.175+0000 7f231d126740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c5042f5d0 0x559c5043e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:48.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.175+0000 7f231d126740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c50430b40 0x559c5043b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:48.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.175+0000 7f231d126740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c5043e780 0x559c5043bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:48.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.175+0000 7f231d126740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559c50441420 con 0x559c5043e780 2024-09-19T09:05:48.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.175+0000 7f231d126740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559c50350060 con 0x559c5042f5d0 2024-09-19T09:05:48.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.175+0000 7f231d126740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559c504415a0 con 0x559c50430b40 2024-09-19T09:05:48.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.175+0000 7f231d126740 10 client.0 _ll_register_callbacks cb 0x559c500a38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:48.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.175+0000 7f2318ae9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c5042f5d0 0x559c5043e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:48.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.175+0000 7f23192ea640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c5043e780 0x559c5043bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:48.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.175+0000 7f2318ae9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c5042f5d0 0x559c5043e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:46020/0 (socket says 172.21.15.120:46020) 2024-09-19T09:05:48.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.175+0000 7f2318ae9640 1 -- 172.21.15.120:0/681172094 learned_addr learned my addr 172.21.15.120:0/681172094 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:48.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.175+0000 7f2313fff640 1 --2- 172.21.15.120:0/681172094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c50430b40 0x559c5043b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:48.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.176+0000 7f2318ae9640 1 -- 172.21.15.120:0/681172094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c50430b40 msgr2=0x559c5043b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:48.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.176+0000 7f2318ae9640 1 --2- 172.21.15.120:0/681172094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c50430b40 0x559c5043b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.176+0000 7f2318ae9640 1 -- 172.21.15.120:0/681172094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c5043e780 msgr2=0x559c5043bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:48.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.176+0000 7f2318ae9640 1 --2- 172.21.15.120:0/681172094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c5043e780 0x559c5043bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.176+0000 7f2318ae9640 1 -- 172.21.15.120:0/681172094 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c501063c0 con 0x559c5042f5d0 2024-09-19T09:05:48.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.176+0000 7f2313fff640 1 --2- 172.21.15.120:0/681172094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c50430b40 0x559c5043b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:05:48.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.176+0000 7f2318ae9640 1 --2- 172.21.15.120:0/681172094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c5042f5d0 0x559c5043e230 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f23040126e0 tx=0x7f2304012ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa7027a148b6ec35 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:48.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.176+0000 7f22f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:05:48.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.176+0000 7f22f3fff640 1 -- 172.21.15.120:0/681172094 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304012190 con 0x559c5042f5d0 2024-09-19T09:05:48.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.176+0000 7f22f3fff640 1 -- 172.21.15.120:0/681172094 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f230400f920 con 0x559c5042f5d0 2024-09-19T09:05:48.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.176+0000 7f231d126740 1 -- 172.21.15.120:0/681172094 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f230809c540 con 0x559c5042f5d0 2024-09-19T09:05:48.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.176+0000 7f22f3fff640 1 -- 172.21.15.120:0/681172094 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f230400fb20 con 0x559c5042f5d0 2024-09-19T09:05:48.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.176+0000 7f231d126740 10 client.25829 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:48.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.176+0000 7f231d126740 1 -- 172.21.15.120:0/681172094 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559c5044cff0 con 0x559c5042f5d0 2024-09-19T09:05:48.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.177+0000 7f22f3fff640 1 -- 172.21.15.120:0/681172094 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f230809c540 con 0x559c5042f5d0 2024-09-19T09:05:48.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.177+0000 7f22f3fff640 1 -- 172.21.15.120:0/681172094 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f230401c420 con 0x559c5042f5d0 2024-09-19T09:05:48.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.177+0000 7f231d126740 10 client.25829 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:48.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.177+0000 7f231d126740 10 client.25829 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:05:48.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.177+0000 7f231d126740 20 client.25829 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:48.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.177+0000 7f231d126740 1 --2- 172.21.15.120:0/681172094 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x559c5044e400 0x559c504508a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:48.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.177+0000 7f231d126740 4 client.25829 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:05:48.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.177+0000 7f231d126740 1 -- 172.21.15.120:0/681172094 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c503a2860 con 0x559c5044e400 2024-09-19T09:05:48.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.177+0000 7f2313fff640 1 --2- 172.21.15.120:0/681172094 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x559c5044e400 0x559c504508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:48.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.178+0000 7f2313fff640 1 --2- 172.21.15.120:0/681172094 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x559c5044e400 0x559c504508a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=214bbaba6071cd37 server_cookie=cc726bfe2659939f in_seq=0 out_seq=0 2024-09-19T09:05:48.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.178+0000 7f22f3fff640 10 client.25829 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:05:48.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.178+0000 7f22f3fff640 1 -- 172.21.15.120:0/681172094 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559c503a2860 con 0x559c5044e400 2024-09-19T09:05:48.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.178+0000 7f22f3fff640 10 client.25829 handle_command_reply: tid=0 2024-09-19T09:05:48.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.178+0000 7f231d126740 1 client.25829 shutdown 2024-09-19T09:05:48.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.179+0000 7f231d126740 1 -- 172.21.15.120:0/681172094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c5042f5d0 msgr2=0x559c5043e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:48.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.179+0000 7f231d126740 1 --2- 172.21.15.120:0/681172094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c5042f5d0 0x559c5043e230 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f23040126e0 tx=0x7f2304012ba0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.179+0000 7f231d126740 1 -- 172.21.15.120:0/681172094 shutdown_connections 2024-09-19T09:05:48.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.179+0000 7f231d126740 1 --2- 172.21.15.120:0/681172094 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x559c5044e400 0x559c504508a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.179+0000 7f231d126740 1 --2- 172.21.15.120:0/681172094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c5043e780 0x559c5043bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.179+0000 7f231d126740 1 --2- 172.21.15.120:0/681172094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c50430b40 0x559c5043b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.179+0000 7f22f3fff640 0 client.25829 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:05:48.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.179+0000 7f231d126740 1 --2- 172.21.15.120:0/681172094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c5042f5d0 0x559c5043e230 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.179+0000 7f231d126740 1 -- 172.21.15.120:0/681172094 >> 172.21.15.120:0/681172094 conn(0x559c503f8310 msgr2=0x559c50431960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:48.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.179+0000 7f231d126740 1 -- 172.21.15.120:0/681172094 shutdown_connections 2024-09-19T09:05:48.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.179+0000 7f231d126740 1 -- 172.21.15.120:0/681172094 wait complete. 2024-09-19T09:05:48.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.179+0000 7f231d126740 20 client.25829 trim_cache size 0 max 16384 2024-09-19T09:05:48.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.194+0000 7f231d126740 1 Processor -- start 2024-09-19T09:05:48.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.194+0000 7f231d126740 1 -- start start 2024-09-19T09:05:48.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.194+0000 7f231d126740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c5042f5d0 0x559c50430300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:48.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.194+0000 7f231d126740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c50430b40 0x559c5042b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:48.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.194+0000 7f231d126740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c5043e780 0x559c5042bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:48.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.194+0000 7f231d126740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f23141ab3e0 con 0x559c50430b40 2024-09-19T09:05:48.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.194+0000 7f231d126740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f23141acf20 con 0x559c5043e780 2024-09-19T09:05:48.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.194+0000 7f231d126740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f23141abab0 con 0x559c5042f5d0 2024-09-19T09:05:48.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.194+0000 7f2313fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c50430b40 0x559c5042b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:48.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.194+0000 7f2313fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c50430b40 0x559c5042b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52772/0 (socket says 172.21.15.120:52772) 2024-09-19T09:05:48.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.194+0000 7f2313fff640 1 -- 172.21.15.120:0/98622792 learned_addr learned my addr 172.21.15.120:0/98622792 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:48.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.194+0000 7f23192ea640 1 --2- 172.21.15.120:0/98622792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c5043e780 0x559c5042bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:48.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.194+0000 7f2318ae9640 1 --2- 172.21.15.120:0/98622792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c5042f5d0 0x559c50430300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:48.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.194+0000 7f2313fff640 1 -- 172.21.15.120:0/98622792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c5042f5d0 msgr2=0x559c50430300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:48.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.194+0000 7f2313fff640 1 --2- 172.21.15.120:0/98622792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c5042f5d0 0x559c50430300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.194+0000 7f2313fff640 1 -- 172.21.15.120:0/98622792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c5043e780 msgr2=0x559c5042bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:48.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.194+0000 7f2313fff640 1 --2- 172.21.15.120:0/98622792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c5043e780 0x559c5042bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.194+0000 7f2313fff640 1 -- 172.21.15.120:0/98622792 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c50450f20 con 0x559c50430b40 2024-09-19T09:05:48.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.195+0000 7f2313fff640 1 --2- 172.21.15.120:0/98622792 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c50430b40 0x559c5042b700 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f230809c4d0 tx=0x7f2308018670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd97965d5fca7b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:48.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.195+0000 7f22fa7fc640 1 -- 172.21.15.120:0/98622792 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308081e80 con 0x559c50430b40 2024-09-19T09:05:48.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.195+0000 7f22fa7fc640 1 -- 172.21.15.120:0/98622792 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23080a0840 con 0x559c50430b40 2024-09-19T09:05:48.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.195+0000 7f22fa7fc640 1 -- 172.21.15.120:0/98622792 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308010e60 con 0x559c50430b40 2024-09-19T09:05:48.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.196+0000 7f231d126740 1 -- 172.21.15.120:0/98622792 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c50430b40 msgr2=0x559c5042b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:48.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.196+0000 7f231d126740 1 --2- 172.21.15.120:0/98622792 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c50430b40 0x559c5042b700 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f230809c4d0 tx=0x7f2308018670 comp rx=0 tx=0).stop 2024-09-19T09:05:48.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.196+0000 7f231d126740 1 -- 172.21.15.120:0/98622792 shutdown_connections 2024-09-19T09:05:48.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.196+0000 7f231d126740 1 --2- 172.21.15.120:0/98622792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c5043e780 0x559c5042bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.196+0000 7f231d126740 1 --2- 172.21.15.120:0/98622792 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c50430b40 0x559c5042b700 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.196+0000 7f231d126740 1 --2- 172.21.15.120:0/98622792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c5042f5d0 0x559c50430300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.196+0000 7f231d126740 1 -- 172.21.15.120:0/98622792 >> 172.21.15.120:0/98622792 conn(0x559c503f8310 msgr2=0x559c50426a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:48.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.196+0000 7f231d126740 1 -- 172.21.15.120:0/98622792 shutdown_connections 2024-09-19T09:05:48.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.196+0000 7f231d126740 1 -- 172.21.15.120:0/98622792 wait complete. 2024-09-19T09:05:48.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.196+0000 7f231d126740 1 -- start start 2024-09-19T09:05:48.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.196+0000 7f231d126740 1 Processor -- start 2024-09-19T09:05:48.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.196+0000 7f231d126740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c5042f5d0 0x559c50446d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:48.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.196+0000 7f231d126740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c50430b40 0x559c50442050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:48.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.197+0000 7f231d126740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c5043e780 0x559c50442590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:48.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.197+0000 7f231d126740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f23141ac850 con 0x559c5043e780 2024-09-19T09:05:48.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.197+0000 7f231d126740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f23141ac180 con 0x559c50430b40 2024-09-19T09:05:48.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.197+0000 7f231d126740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f23141a93a0 con 0x559c5042f5d0 2024-09-19T09:05:48.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.197+0000 7f231d126740 10 client.0 _ll_register_callbacks cb 0x7f2304009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:48.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.197+0000 7f2318ae9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c5042f5d0 0x559c50446d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:48.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.197+0000 7f23192ea640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c5043e780 0x559c50442590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:48.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.197+0000 7f2318ae9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c5042f5d0 0x559c50446d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:54960/0 (socket says 172.21.15.120:54960) 2024-09-19T09:05:48.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.197+0000 7f2318ae9640 1 -- 172.21.15.120:0/29470145 learned_addr learned my addr 172.21.15.120:0/29470145 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:48.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.197+0000 7f2313fff640 1 --2- 172.21.15.120:0/29470145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c50430b40 0x559c50442050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:48.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.197+0000 7f2318ae9640 1 -- 172.21.15.120:0/29470145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c50430b40 msgr2=0x559c50442050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:48.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.197+0000 7f2318ae9640 1 --2- 172.21.15.120:0/29470145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c50430b40 0x559c50442050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.197+0000 7f2318ae9640 1 -- 172.21.15.120:0/29470145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c5043e780 msgr2=0x559c50442590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:48.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.197+0000 7f2318ae9640 1 --2- 172.21.15.120:0/29470145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c5043e780 0x559c50442590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.197+0000 7f2318ae9640 1 -- 172.21.15.120:0/29470145 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c50442b50 con 0x559c5042f5d0 2024-09-19T09:05:48.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.197+0000 7f23192ea640 1 --2- 172.21.15.120:0/29470145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c5043e780 0x559c50442590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:48.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.197+0000 7f2318ae9640 1 --2- 172.21.15.120:0/29470145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c5042f5d0 0x559c50446d20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f230401ca30 tx=0x7f2304005cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca4631b763bea584 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:48.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.197+0000 7f22f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:05:48.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.197+0000 7f22f3fff640 1 -- 172.21.15.120:0/29470145 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230402b030 con 0x559c5042f5d0 2024-09-19T09:05:48.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.197+0000 7f22f3fff640 1 -- 172.21.15.120:0/29470145 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f2304018040 con 0x559c5042f5d0 2024-09-19T09:05:48.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.197+0000 7f231d126740 1 -- 172.21.15.120:0/29470145 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23080a0840 con 0x559c5042f5d0 2024-09-19T09:05:48.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.198+0000 7f22f3fff640 1 -- 172.21.15.120:0/29470145 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f230401c420 con 0x559c5042f5d0 2024-09-19T09:05:48.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.198+0000 7f231d126740 10 client.25798 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:48.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.198+0000 7f231d126740 1 -- 172.21.15.120:0/29470145 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559c504b8f20 con 0x559c5042f5d0 2024-09-19T09:05:48.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.198+0000 7f22f3fff640 1 -- 172.21.15.120:0/29470145 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23080a0840 con 0x559c5042f5d0 2024-09-19T09:05:48.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.198+0000 7f22f3fff640 1 -- 172.21.15.120:0/29470145 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2304005440 con 0x559c5042f5d0 2024-09-19T09:05:48.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.198+0000 7f231d126740 10 client.25798 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:48.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.198+0000 7f231d126740 10 client.25798 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:05:48.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.198+0000 7f231d126740 20 client.25798 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:48.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.198+0000 7f231d126740 1 --2- 172.21.15.120:0/29470145 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x559c504b9630 0x559c504bba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:48.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.198+0000 7f231d126740 4 client.25798 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:05:48.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.198+0000 7f231d126740 1 -- 172.21.15.120:0/29470145 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559c5039cd70 con 0x559c504b9630 2024-09-19T09:05:48.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.198+0000 7f2313fff640 1 --2- 172.21.15.120:0/29470145 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x559c504b9630 0x559c504bba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:48.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.199+0000 7f22f3fff640 10 client.25798 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:05:48.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.199+0000 7f2313fff640 1 --2- 172.21.15.120:0/29470145 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x559c504b9630 0x559c504bba50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=87c83169b1299bd1 server_cookie=42d030ced9291252 in_seq=0 out_seq=0 2024-09-19T09:05:48.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.199+0000 7f22f3fff640 1 -- 172.21.15.120:0/29470145 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559c5039cd70 con 0x559c504b9630 2024-09-19T09:05:48.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.199+0000 7f22f3fff640 10 client.25798 handle_command_reply: tid=0 2024-09-19T09:05:48.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.199+0000 7f231d126740 1 client.25798 shutdown 2024-09-19T09:05:48.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.199+0000 7f231d126740 1 -- 172.21.15.120:0/29470145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c5042f5d0 msgr2=0x559c50446d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:48.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.199+0000 7f231d126740 1 --2- 172.21.15.120:0/29470145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c5042f5d0 0x559c50446d20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f230401ca30 tx=0x7f2304005cc0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.199+0000 7f231d126740 1 -- 172.21.15.120:0/29470145 shutdown_connections 2024-09-19T09:05:48.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.199+0000 7f231d126740 1 --2- 172.21.15.120:0/29470145 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x559c504b9630 0x559c504bba50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.199+0000 7f231d126740 1 --2- 172.21.15.120:0/29470145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c5043e780 0x559c50442590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.199+0000 7f231d126740 1 --2- 172.21.15.120:0/29470145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c50430b40 0x559c50442050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.199+0000 7f231d126740 1 --2- 172.21.15.120:0/29470145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c5042f5d0 0x559c50446d20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.199+0000 7f231d126740 1 -- 172.21.15.120:0/29470145 >> 172.21.15.120:0/29470145 conn(0x559c503f8310 msgr2=0x559c5042b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:48.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.199+0000 7f22f3fff640 0 client.25798 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:05:48.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.199+0000 7f231d126740 1 -- 172.21.15.120:0/29470145 shutdown_connections 2024-09-19T09:05:48.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.199+0000 7f231d126740 1 -- 172.21.15.120:0/29470145 wait complete. 2024-09-19T09:05:48.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.199+0000 7f231d126740 20 client.25798 trim_cache size 0 max 16384 2024-09-19T09:05:48.206 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:05:48.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.200+0000 7f231ad4d640 1 -- 172.21.15.120:0/1901585012 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f22fc0a3240 msgr2=0x7f22fc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:48.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.200+0000 7f231ad4d640 1 --2- 172.21.15.120:0/1901585012 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f22fc0a3240 0x7f22fc0a5700 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f230c0096c0 tx=0x7f230c02a480 comp rx=0 tx=0).stop 2024-09-19T09:05:48.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.200+0000 7f231ad4d640 1 -- 172.21.15.120:0/1901585012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2314102570 msgr2=0x7f2314197980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:48.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.200+0000 7f231ad4d640 1 --2- 172.21.15.120:0/1901585012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2314102570 0x7f2314197980 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f230800ebc0 tx=0x7f230800c6a0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.200+0000 7f231ad4d640 1 -- 172.21.15.120:0/1901585012 shutdown_connections 2024-09-19T09:05:48.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.200+0000 7f231ad4d640 1 --2- 172.21.15.120:0/1901585012 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f22fc0a3240 0x7f22fc0a5700 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.200+0000 7f231ad4d640 1 --2- 172.21.15.120:0/1901585012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2314102ef0 0x7f231419bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.200+0000 7f231ad4d640 1 --2- 172.21.15.120:0/1901585012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2314102570 0x7f2314197980 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.200+0000 7f231ad4d640 1 --2- 172.21.15.120:0/1901585012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2314100e20 0x7f2314197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:48.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.200+0000 7f231ad4d640 1 -- 172.21.15.120:0/1901585012 >> 172.21.15.120:0/1901585012 conn(0x7f23140fc7d0 msgr2=0x7f23141068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:48.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.200+0000 7f231ad4d640 1 -- 172.21.15.120:0/1901585012 shutdown_connections 2024-09-19T09:05:48.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:48.200+0000 7f231ad4d640 1 -- 172.21.15.120:0/1901585012 wait complete. 2024-09-19T09:05:49.040 INFO:tasks.workunit.client.0.smithi120.stdout: 1 182144 9.60 MB/sec warmup 104 sec latency 57.621 ms 2024-09-19T09:05:49.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:48 smithi120 ceph-mon[31150]: pgmap v429: 97 pgs: 97 active+clean; 40 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 400 op/s 2024-09-19T09:05:49.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:48 smithi196 ceph-mon[35150]: pgmap v429: 97 pgs: 97 active+clean; 40 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 400 op/s 2024-09-19T09:05:49.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:48 smithi153 ceph-mon[35071]: pgmap v429: 97 pgs: 97 active+clean; 40 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 400 op/s 2024-09-19T09:05:49.193 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:05:49.496 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:05:50.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.994+0000 7f563b0db640 1 -- 172.21.15.120:0/2775359437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5634101400 msgr2=0x7f563410d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:50.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.994+0000 7f563b0db640 1 --2- 172.21.15.120:0/2775359437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5634101400 0x7f563410d930 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f562c009c10 tx=0x7f562c02a6e0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.994+0000 7f563b0db640 1 -- 172.21.15.120:0/2775359437 shutdown_connections 2024-09-19T09:05:50.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.994+0000 7f563b0db640 1 --2- 172.21.15.120:0/2775359437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5634101400 0x7f563410d930 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.994+0000 7f563b0db640 1 --2- 172.21.15.120:0/2775359437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5634100a60 0x7f5634100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.994+0000 7f563b0db640 1 --2- 172.21.15.120:0/2775359437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5634106a60 0x7f5634106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.994+0000 7f563b0db640 1 -- 172.21.15.120:0/2775359437 >> 172.21.15.120:0/2775359437 conn(0x7f56340fc780 msgr2=0x7f56340feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:50.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.994+0000 7f563b0db640 1 -- 172.21.15.120:0/2775359437 shutdown_connections 2024-09-19T09:05:50.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.994+0000 7f563b0db640 1 -- 172.21.15.120:0/2775359437 wait complete. 2024-09-19T09:05:50.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.994+0000 7f563b0db640 1 Processor -- start 2024-09-19T09:05:50.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.994+0000 7f563b0db640 1 -- start start 2024-09-19T09:05:50.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.995+0000 7f563b0db640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5634100a60 0x7f5634198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:50.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.995+0000 7f563b0db640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5634101400 0x7f56341989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:50.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.995+0000 7f563b0db640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5634106a60 0x7f563419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:50.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.995+0000 7f563b0db640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5634111f20 con 0x7f5634106a60 2024-09-19T09:05:50.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.995+0000 7f563b0db640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5634111da0 con 0x7f5634100a60 2024-09-19T09:05:50.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.995+0000 7f563b0db640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56341120a0 con 0x7f5634101400 2024-09-19T09:05:50.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.995+0000 7f5639678640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5634106a60 0x7f563419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:50.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.995+0000 7f5639678640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5634106a60 0x7f563419cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52798/0 (socket says 172.21.15.120:52798) 2024-09-19T09:05:50.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.995+0000 7f5639678640 1 -- 172.21.15.120:0/622238716 learned_addr learned my addr 172.21.15.120:0/622238716 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:50.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.995+0000 7f5638e77640 1 --2- 172.21.15.120:0/622238716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5634100a60 0x7f5634198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:50.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.995+0000 7f5633fff640 1 --2- 172.21.15.120:0/622238716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5634101400 0x7f56341989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:50.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.995+0000 7f5639678640 1 -- 172.21.15.120:0/622238716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5634101400 msgr2=0x7f56341989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:50.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.995+0000 7f5639678640 1 --2- 172.21.15.120:0/622238716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5634101400 0x7f56341989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.995+0000 7f5639678640 1 -- 172.21.15.120:0/622238716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5634100a60 msgr2=0x7f5634198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:50.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.995+0000 7f5639678640 1 --2- 172.21.15.120:0/622238716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5634100a60 0x7f5634198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.995+0000 7f5639678640 1 -- 172.21.15.120:0/622238716 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563419d410 con 0x7f5634106a60 2024-09-19T09:05:50.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.995+0000 7f5638e77640 1 --2- 172.21.15.120:0/622238716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5634100a60 0x7f5634198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:50.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.995+0000 7f5639678640 1 --2- 172.21.15.120:0/622238716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5634106a60 0x7f563419cd30 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f562c0305d0 tx=0x7f562c030600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:50.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.995+0000 7f5631ffb640 1 -- 172.21.15.120:0/622238716 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c041020 con 0x7f5634106a60 2024-09-19T09:05:50.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.996+0000 7f563b0db640 1 -- 172.21.15.120:0/622238716 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f563419d6a0 con 0x7f5634106a60 2024-09-19T09:05:50.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.996+0000 7f5631ffb640 1 -- 172.21.15.120:0/622238716 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f562c037040 con 0x7f5634106a60 2024-09-19T09:05:50.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.996+0000 7f5631ffb640 1 -- 172.21.15.120:0/622238716 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c036070 con 0x7f5634106a60 2024-09-19T09:05:50.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.996+0000 7f563b0db640 1 -- 172.21.15.120:0/622238716 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56341a4fe0 con 0x7f5634106a60 2024-09-19T09:05:50.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.998+0000 7f5631ffb640 1 -- 172.21.15.120:0/622238716 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f562c044020 con 0x7f5634106a60 2024-09-19T09:05:50.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.998+0000 7f563d4b4740 1 Processor -- start 2024-09-19T09:05:50.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.998+0000 7f563d4b4740 1 -- start start 2024-09-19T09:05:50.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.999+0000 7f563d4b4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ffad5cb40 0x555ffad5b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:50.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.999+0000 7f563d4b4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffad5b5d0 0x555ffad5ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:50.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.999+0000 7f563d4b4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ffad66370 0x555ffad68760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:50.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.999+0000 7f563d4b4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555ffacc12f0 con 0x555ffad5b5d0 2024-09-19T09:05:50.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.999+0000 7f563d4b4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555ffac68cf0 con 0x555ffad5cb40 2024-09-19T09:05:50.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.999+0000 7f563d4b4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555ffacbd2b0 con 0x555ffad66370 2024-09-19T09:05:50.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.999+0000 7f5638e77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffad5b5d0 0x555ffad5ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:50.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.999+0000 7f5631ffb640 1 --2- 172.21.15.120:0/622238716 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f56240a3240 0x7f56240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:50.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.999+0000 7f5638e77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffad5b5d0 0x555ffad5ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52808/0 (socket says 172.21.15.120:52808) 2024-09-19T09:05:50.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.999+0000 7f5638e77640 1 -- 172.21.15.120:0/839475041 learned_addr learned my addr 172.21.15.120:0/839475041 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:50.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.999+0000 7f5633fff640 1 --2- 172.21.15.120:0/839475041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ffad5cb40 0x555ffad5b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:50.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.999+0000 7f5633fff640 1 --2- 172.21.15.120:0/839475041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ffad66370 0x555ffad68760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:50.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.999+0000 7f5639678640 1 --2- 172.21.15.120:0/622238716 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f56240a3240 0x7f56240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:50.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.999+0000 7f5631ffb640 1 -- 172.21.15.120:0/622238716 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f562c0f0410 con 0x7f5634106a60 2024-09-19T09:05:50.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.999+0000 7f5638e77640 1 -- 172.21.15.120:0/839475041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ffad66370 msgr2=0x555ffad68760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:50.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.999+0000 7f5638e77640 1 --2- 172.21.15.120:0/839475041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ffad66370 0x555ffad68760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.999+0000 7f5638e77640 1 -- 172.21.15.120:0/839475041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ffad5cb40 msgr2=0x555ffad5b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:50.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.999+0000 7f5638e77640 1 --2- 172.21.15.120:0/839475041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ffad5cb40 0x555ffad5b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.999+0000 7f5638e77640 1 -- 172.21.15.120:0/839475041 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ffac4d3a0 con 0x555ffad5b5d0 2024-09-19T09:05:50.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.999+0000 7f5633fff640 1 --2- 172.21.15.120:0/839475041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ffad66370 0x555ffad68760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:05:50.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:49.999+0000 7f5638e77640 1 --2- 172.21.15.120:0/839475041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffad5b5d0 0x555ffad5ba10 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5628011470 tx=0x7f5628011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff884a6d1b7f6467 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:50.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.000+0000 7f5639678640 1 --2- 172.21.15.120:0/622238716 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f56240a3240 0x7f56240a5700 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f562c004af0 tx=0x7f562c087430 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:50.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.000+0000 7f561effd640 1 -- 172.21.15.120:0/839475041 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5628005020 con 0x555ffad5b5d0 2024-09-19T09:05:50.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.000+0000 7f561effd640 1 -- 172.21.15.120:0/839475041 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56280047a0 con 0x555ffad5b5d0 2024-09-19T09:05:50.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.000+0000 7f561effd640 1 -- 172.21.15.120:0/839475041 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562800f9a0 con 0x555ffad5b5d0 2024-09-19T09:05:50.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.000+0000 7f563d4b4740 1 -- 172.21.15.120:0/839475041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffad5b5d0 msgr2=0x555ffad5ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:50.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.000+0000 7f563d4b4740 1 --2- 172.21.15.120:0/839475041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffad5b5d0 0x555ffad5ba10 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5628011470 tx=0x7f5628011930 comp rx=0 tx=0).stop 2024-09-19T09:05:50.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.000+0000 7f563d4b4740 1 -- 172.21.15.120:0/839475041 shutdown_connections 2024-09-19T09:05:50.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.000+0000 7f563d4b4740 1 --2- 172.21.15.120:0/839475041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ffad66370 0x555ffad68760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.000+0000 7f563d4b4740 1 --2- 172.21.15.120:0/839475041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffad5b5d0 0x555ffad5ba10 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.000+0000 7f563d4b4740 1 --2- 172.21.15.120:0/839475041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ffad5cb40 0x555ffad5b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.000+0000 7f563d4b4740 1 -- 172.21.15.120:0/839475041 >> 172.21.15.120:0/839475041 conn(0x555ffad24310 msgr2=0x555ffad57c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:50.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.000+0000 7f563d4b4740 1 -- 172.21.15.120:0/839475041 shutdown_connections 2024-09-19T09:05:50.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.000+0000 7f563d4b4740 1 -- 172.21.15.120:0/839475041 wait complete. 2024-09-19T09:05:50.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.001+0000 7f563d4b4740 1 -- start start 2024-09-19T09:05:50.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.001+0000 7f563d4b4740 1 Processor -- start 2024-09-19T09:05:50.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.001+0000 7f563d4b4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffad5b5d0 0x555ffad65cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:50.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.001+0000 7f563d4b4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ffad5cb40 0x555ffad62f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:50.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.001+0000 7f563d4b4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ffad66370 0x555ffad634d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:50.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.001+0000 7f563d4b4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555ffad69070 con 0x555ffad5b5d0 2024-09-19T09:05:50.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.001+0000 7f563d4b4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555ffac7c060 con 0x555ffad66370 2024-09-19T09:05:50.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.001+0000 7f563d4b4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555ffad691f0 con 0x555ffad5cb40 2024-09-19T09:05:50.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.001+0000 7f563d4b4740 10 client.0 _ll_register_callbacks cb 0x555ffa9cf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:50.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.001+0000 7f5633fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffad5b5d0 0x555ffad65cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:50.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.001+0000 7f5633fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffad5b5d0 0x555ffad65cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52822/0 (socket says 172.21.15.120:52822) 2024-09-19T09:05:50.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.001+0000 7f5633fff640 1 -- 172.21.15.120:0/1160191955 learned_addr learned my addr 172.21.15.120:0/1160191955 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:50.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.001+0000 7f5638e77640 1 --2- 172.21.15.120:0/1160191955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ffad5cb40 0x555ffad62f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:50.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.001+0000 7f5633fff640 1 --2- 172.21.15.120:0/1160191955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ffad66370 0x555ffad634d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:50.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.002+0000 7f5633fff640 1 -- 172.21.15.120:0/1160191955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ffad5cb40 msgr2=0x555ffad62f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:50.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.002+0000 7f5633fff640 1 --2- 172.21.15.120:0/1160191955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ffad5cb40 0x555ffad62f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.002+0000 7f5633fff640 1 -- 172.21.15.120:0/1160191955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ffad66370 msgr2=0x555ffad634d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:50.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.002+0000 7f5633fff640 1 --2- 172.21.15.120:0/1160191955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ffad66370 0x555ffad634d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.002+0000 7f5633fff640 1 -- 172.21.15.120:0/1160191955 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ffaa323c0 con 0x555ffad5b5d0 2024-09-19T09:05:50.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.002+0000 7f5633fff640 1 --2- 172.21.15.120:0/1160191955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffad5b5d0 0x555ffad65cc0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5620011e60 tx=0x7f5620009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6119ffecdf7babb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:50.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.002+0000 7f5613fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:50.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.002+0000 7f5613fff640 1 -- 172.21.15.120:0/1160191955 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5620011020 con 0x555ffad5b5d0 2024-09-19T09:05:50.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.002+0000 7f5613fff640 1 -- 172.21.15.120:0/1160191955 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f5620009040 con 0x555ffad5b5d0 2024-09-19T09:05:50.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.002+0000 7f563d4b4740 1 -- 172.21.15.120:0/1160191955 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56280047a0 con 0x555ffad5b5d0 2024-09-19T09:05:50.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.002+0000 7f5613fff640 1 -- 172.21.15.120:0/1160191955 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f56200111c0 con 0x555ffad5b5d0 2024-09-19T09:05:50.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.002+0000 7f563d4b4740 10 client.16485 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:50.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.002+0000 7f563d4b4740 1 -- 172.21.15.120:0/1160191955 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555ffad74b10 con 0x555ffad5b5d0 2024-09-19T09:05:50.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.002+0000 7f5613fff640 1 -- 172.21.15.120:0/1160191955 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56280047a0 con 0x555ffad5b5d0 2024-09-19T09:05:50.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.002+0000 7f5613fff640 1 -- 172.21.15.120:0/1160191955 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5620003a30 con 0x555ffad5b5d0 2024-09-19T09:05:50.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.003+0000 7f563d4b4740 10 client.16485 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:50.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.003+0000 7f563d4b4740 10 client.16485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:50.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.003+0000 7f563d4b4740 20 client.16485 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:50.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.003+0000 7f563d4b4740 1 --2- 172.21.15.120:0/1160191955 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555ffad75fd0 0x555ffad783c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:50.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.003+0000 7f563d4b4740 4 client.16485 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:05:50.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.003+0000 7f563d4b4740 1 -- 172.21.15.120:0/1160191955 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ffacce860 con 0x555ffad75fd0 2024-09-19T09:05:50.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.003+0000 7f5638e77640 1 --2- 172.21.15.120:0/1160191955 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555ffad75fd0 0x555ffad783c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:50.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.003+0000 7f5638e77640 1 --2- 172.21.15.120:0/1160191955 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555ffad75fd0 0x555ffad783c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b5baecb72f45fb1 server_cookie=181bf9f448feb88f in_seq=0 out_seq=0 2024-09-19T09:05:50.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.003+0000 7f5613fff640 10 client.16485 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:50.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.004+0000 7f5613fff640 1 -- 172.21.15.120:0/1160191955 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555ffacce860 con 0x555ffad75fd0 2024-09-19T09:05:50.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.004+0000 7f5613fff640 10 client.16485 handle_command_reply: tid=0 2024-09-19T09:05:50.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.005+0000 7f563d4b4740 1 client.16485 shutdown 2024-09-19T09:05:50.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.005+0000 7f563d4b4740 1 -- 172.21.15.120:0/1160191955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffad5b5d0 msgr2=0x555ffad65cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:50.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.005+0000 7f563d4b4740 1 --2- 172.21.15.120:0/1160191955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffad5b5d0 0x555ffad65cc0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5620011e60 tx=0x7f5620009ac0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.005+0000 7f563d4b4740 1 -- 172.21.15.120:0/1160191955 shutdown_connections 2024-09-19T09:05:50.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.005+0000 7f563d4b4740 1 --2- 172.21.15.120:0/1160191955 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555ffad75fd0 0x555ffad783c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.005+0000 7f563d4b4740 1 --2- 172.21.15.120:0/1160191955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ffad66370 0x555ffad634d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.005+0000 7f5613fff640 0 client.16485 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:50.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.005+0000 7f563d4b4740 1 --2- 172.21.15.120:0/1160191955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ffad5cb40 0x555ffad62f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.005+0000 7f563d4b4740 1 --2- 172.21.15.120:0/1160191955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffad5b5d0 0x555ffad65cc0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.005+0000 7f563d4b4740 1 -- 172.21.15.120:0/1160191955 >> 172.21.15.120:0/1160191955 conn(0x555ffad24310 msgr2=0x555ffad5d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:50.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.005+0000 7f563d4b4740 1 -- 172.21.15.120:0/1160191955 shutdown_connections 2024-09-19T09:05:50.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.005+0000 7f563d4b4740 1 -- 172.21.15.120:0/1160191955 wait complete. 2024-09-19T09:05:50.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.005+0000 7f563d4b4740 20 client.16485 trim_cache size 0 max 16384 2024-09-19T09:05:50.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.018+0000 7f563d4b4740 1 Processor -- start 2024-09-19T09:05:50.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.018+0000 7f563d4b4740 1 -- start start 2024-09-19T09:05:50.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.018+0000 7f563d4b4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ffad5b5d0 0x555ffad56c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:50.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.018+0000 7f563d4b4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffad5cb40 0x555ffad5e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:50.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.018+0000 7f563d4b4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ffad66370 0x555ffad5e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:50.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.018+0000 7f563d4b4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f56341ad680 con 0x555ffad5cb40 2024-09-19T09:05:50.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.018+0000 7f563d4b4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f56341af1c0 con 0x555ffad5b5d0 2024-09-19T09:05:50.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.018+0000 7f563d4b4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56341add50 con 0x555ffad66370 2024-09-19T09:05:50.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.019+0000 7f5638e77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffad5cb40 0x555ffad5e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:50.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.019+0000 7f5638e77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffad5cb40 0x555ffad5e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52826/0 (socket says 172.21.15.120:52826) 2024-09-19T09:05:50.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.019+0000 7f5638e77640 1 -- 172.21.15.120:0/831905073 learned_addr learned my addr 172.21.15.120:0/831905073 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:50.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.019+0000 7f5633fff640 1 --2- 172.21.15.120:0/831905073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ffad5b5d0 0x555ffad56c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:50.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.019+0000 7f5633fff640 1 --2- 172.21.15.120:0/831905073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ffad66370 0x555ffad5e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:50.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.019+0000 7f5633fff640 1 -- 172.21.15.120:0/831905073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ffad66370 msgr2=0x555ffad5e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:50.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.019+0000 7f5633fff640 1 --2- 172.21.15.120:0/831905073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ffad66370 0x555ffad5e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.019+0000 7f5633fff640 1 -- 172.21.15.120:0/831905073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffad5cb40 msgr2=0x555ffad5e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:50.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.019+0000 7f5633fff640 1 --2- 172.21.15.120:0/831905073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffad5cb40 0x555ffad5e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.019+0000 7f5633fff640 1 -- 172.21.15.120:0/831905073 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ffad5eba0 con 0x555ffad5b5d0 2024-09-19T09:05:50.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.019+0000 7f5633fff640 1 --2- 172.21.15.120:0/831905073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ffad5b5d0 0x555ffad56c10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5620009c30 tx=0x7f5620009e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aeb9e7972341f710 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:50.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.019+0000 7f561e7fc640 1 -- 172.21.15.120:0/831905073 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5620030070 con 0x555ffad5b5d0 2024-09-19T09:05:50.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.020+0000 7f561e7fc640 1 -- 172.21.15.120:0/831905073 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5620004480 con 0x555ffad5b5d0 2024-09-19T09:05:50.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.020+0000 7f561e7fc640 1 -- 172.21.15.120:0/831905073 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5620005070 con 0x555ffad5b5d0 2024-09-19T09:05:50.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.020+0000 7f563d4b4740 1 -- 172.21.15.120:0/831905073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ffad5b5d0 msgr2=0x555ffad56c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:50.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.020+0000 7f563d4b4740 1 --2- 172.21.15.120:0/831905073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ffad5b5d0 0x555ffad56c10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5620009c30 tx=0x7f5620009e70 comp rx=0 tx=0).stop 2024-09-19T09:05:50.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.020+0000 7f563d4b4740 1 -- 172.21.15.120:0/831905073 shutdown_connections 2024-09-19T09:05:50.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.020+0000 7f563d4b4740 1 --2- 172.21.15.120:0/831905073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ffad66370 0x555ffad5e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.020+0000 7f563d4b4740 1 --2- 172.21.15.120:0/831905073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffad5cb40 0x555ffad5e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.020+0000 7f563d4b4740 1 --2- 172.21.15.120:0/831905073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ffad5b5d0 0x555ffad56c10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.020+0000 7f563d4b4740 1 -- 172.21.15.120:0/831905073 >> 172.21.15.120:0/831905073 conn(0x555ffad24310 msgr2=0x555ffad56500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:50.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.020+0000 7f563d4b4740 1 -- 172.21.15.120:0/831905073 shutdown_connections 2024-09-19T09:05:50.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.021+0000 7f563d4b4740 1 -- 172.21.15.120:0/831905073 wait complete. 2024-09-19T09:05:50.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.021+0000 7f563d4b4740 1 -- start start 2024-09-19T09:05:50.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.021+0000 7f563d4b4740 1 Processor -- start 2024-09-19T09:05:50.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.021+0000 7f563d4b4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ffad5b5d0 0x555ffad6ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:50.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.022+0000 7f563d4b4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ffad5cb40 0x555ffad5fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:50.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.022+0000 7f563d4b4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffad66370 0x555ffad602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:50.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.022+0000 7f563d4b4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f56341aeaf0 con 0x555ffad66370 2024-09-19T09:05:50.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.022+0000 7f563d4b4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f56341ae420 con 0x555ffad5cb40 2024-09-19T09:05:50.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.022+0000 7f563d4b4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56341ab640 con 0x555ffad5b5d0 2024-09-19T09:05:50.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.022+0000 7f563d4b4740 10 client.0 _ll_register_callbacks cb 0x7f5620009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:50.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.022+0000 7f5633fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffad66370 0x555ffad602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:50.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.022+0000 7f5633fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ffad5b5d0 0x555ffad6ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:50.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.022+0000 7f5638e77640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ffad5cb40 0x555ffad5fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:50.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.022+0000 7f5633fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ffad5b5d0 0x555ffad6ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:55028/0 (socket says 172.21.15.120:55028) 2024-09-19T09:05:50.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.022+0000 7f5633fff640 1 -- 172.21.15.120:0/2975913708 learned_addr learned my addr 172.21.15.120:0/2975913708 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:50.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.022+0000 7f5633fff640 1 -- 172.21.15.120:0/2975913708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ffad5b5d0 msgr2=0x555ffad6ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:50.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.022+0000 7f5633fff640 1 --2- 172.21.15.120:0/2975913708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ffad5b5d0 0x555ffad6ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.022+0000 7f5633fff640 1 -- 172.21.15.120:0/2975913708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ffad5cb40 msgr2=0x555ffad5fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:50.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.022+0000 7f5633fff640 1 --2- 172.21.15.120:0/2975913708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ffad5cb40 0x555ffad5fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.022+0000 7f5633fff640 1 -- 172.21.15.120:0/2975913708 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ffad60ad0 con 0x555ffad66370 2024-09-19T09:05:50.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.022+0000 7f5638e77640 1 --2- 172.21.15.120:0/2975913708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ffad5cb40 0x555ffad5fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:50.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.022+0000 7f5633fff640 1 --2- 172.21.15.120:0/2975913708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffad66370 0x555ffad602a0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5620010700 tx=0x7f5620011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2801c49b781f13c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:50.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.022+0000 7f5613fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:50.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.022+0000 7f5613fff640 1 -- 172.21.15.120:0/2975913708 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5620030070 con 0x555ffad66370 2024-09-19T09:05:50.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.022+0000 7f5613fff640 1 -- 172.21.15.120:0/2975913708 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f56200035f0 con 0x555ffad66370 2024-09-19T09:05:50.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.022+0000 7f563d4b4740 1 -- 172.21.15.120:0/2975913708 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5620004480 con 0x555ffad66370 2024-09-19T09:05:50.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.023+0000 7f5613fff640 1 -- 172.21.15.120:0/2975913708 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5620040050 con 0x555ffad66370 2024-09-19T09:05:50.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.023+0000 7f563d4b4740 10 client.16491 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:50.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.023+0000 7f563d4b4740 1 -- 172.21.15.120:0/2975913708 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555ffade0c70 con 0x555ffad66370 2024-09-19T09:05:50.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.023+0000 7f5613fff640 1 -- 172.21.15.120:0/2975913708 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5620004480 con 0x555ffad66370 2024-09-19T09:05:50.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.023+0000 7f5613fff640 1 -- 172.21.15.120:0/2975913708 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5620038070 con 0x555ffad66370 2024-09-19T09:05:50.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.023+0000 7f563d4b4740 10 client.16491 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:50.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.023+0000 7f563d4b4740 10 client.16491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:50.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.023+0000 7f563d4b4740 20 client.16491 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:50.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.023+0000 7f563d4b4740 1 --2- 172.21.15.120:0/2975913708 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555ffad75fd0 0x555ffade33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:50.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.023+0000 7f563d4b4740 4 client.16491 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:05:50.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.023+0000 7f563d4b4740 1 -- 172.21.15.120:0/2975913708 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555ffacc8d70 con 0x555ffad75fd0 2024-09-19T09:05:50.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.023+0000 7f5638e77640 1 --2- 172.21.15.120:0/2975913708 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555ffad75fd0 0x555ffade33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:50.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.024+0000 7f5613fff640 10 client.16491 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:50.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.024+0000 7f5638e77640 1 --2- 172.21.15.120:0/2975913708 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555ffad75fd0 0x555ffade33c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e7a1192264927e4 server_cookie=5c0210afa72447da in_seq=0 out_seq=0 2024-09-19T09:05:50.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.025+0000 7f5613fff640 1 -- 172.21.15.120:0/2975913708 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555ffacc8d70 con 0x555ffad75fd0 2024-09-19T09:05:50.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.025+0000 7f5613fff640 10 client.16491 handle_command_reply: tid=0 2024-09-19T09:05:50.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.025+0000 7f563d4b4740 1 client.16491 shutdown 2024-09-19T09:05:50.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.025+0000 7f563d4b4740 1 -- 172.21.15.120:0/2975913708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffad66370 msgr2=0x555ffad602a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:50.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.025+0000 7f563d4b4740 1 --2- 172.21.15.120:0/2975913708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffad66370 0x555ffad602a0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5620010700 tx=0x7f5620011b40 comp rx=0 tx=0).stop 2024-09-19T09:05:50.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.025+0000 7f563d4b4740 1 -- 172.21.15.120:0/2975913708 shutdown_connections 2024-09-19T09:05:50.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.025+0000 7f563d4b4740 1 --2- 172.21.15.120:0/2975913708 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555ffad75fd0 0x555ffade33c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.025+0000 7f5613fff640 0 client.16491 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:50.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.025+0000 7f563d4b4740 1 --2- 172.21.15.120:0/2975913708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555ffad66370 0x555ffad602a0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.025+0000 7f563d4b4740 1 --2- 172.21.15.120:0/2975913708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ffad5cb40 0x555ffad5fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.025+0000 7f563d4b4740 1 --2- 172.21.15.120:0/2975913708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ffad5b5d0 0x555ffad6ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.025+0000 7f563d4b4740 1 -- 172.21.15.120:0/2975913708 >> 172.21.15.120:0/2975913708 conn(0x555ffad24310 msgr2=0x555ffad56d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:50.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.025+0000 7f563d4b4740 1 -- 172.21.15.120:0/2975913708 shutdown_connections 2024-09-19T09:05:50.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.025+0000 7f563d4b4740 1 -- 172.21.15.120:0/2975913708 wait complete. 2024-09-19T09:05:50.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.025+0000 7f563d4b4740 20 client.16491 trim_cache size 0 max 16384 2024-09-19T09:05:50.039 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:05:50.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.026+0000 7f563b0db640 1 -- 172.21.15.120:0/622238716 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f56240a3240 msgr2=0x7f56240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:50.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.026+0000 7f563b0db640 1 --2- 172.21.15.120:0/622238716 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f56240a3240 0x7f56240a5700 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f562c004af0 tx=0x7f562c087430 comp rx=0 tx=0).stop 2024-09-19T09:05:50.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.026+0000 7f563b0db640 1 -- 172.21.15.120:0/622238716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5634106a60 msgr2=0x7f563419cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:50.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.026+0000 7f563b0db640 1 --2- 172.21.15.120:0/622238716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5634106a60 0x7f563419cd30 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f562c0305d0 tx=0x7f562c030600 comp rx=0 tx=0).stop 2024-09-19T09:05:50.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.026+0000 7f563b0db640 1 -- 172.21.15.120:0/622238716 shutdown_connections 2024-09-19T09:05:50.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.026+0000 7f563b0db640 1 --2- 172.21.15.120:0/622238716 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f56240a3240 0x7f56240a5700 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.026+0000 7f563b0db640 1 --2- 172.21.15.120:0/622238716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5634106a60 0x7f563419cd30 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.026+0000 7f563b0db640 1 --2- 172.21.15.120:0/622238716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5634101400 0x7f56341989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.026+0000 7f563b0db640 1 --2- 172.21.15.120:0/622238716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5634100a60 0x7f5634198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:50.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.026+0000 7f563b0db640 1 -- 172.21.15.120:0/622238716 >> 172.21.15.120:0/622238716 conn(0x7f56340fc780 msgr2=0x7f56340fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:50.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.026+0000 7f563b0db640 1 -- 172.21.15.120:0/622238716 shutdown_connections 2024-09-19T09:05:50.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:50.026+0000 7f563b0db640 1 -- 172.21.15.120:0/622238716 wait complete. 2024-09-19T09:05:50.042 INFO:tasks.workunit.client.0.smithi120.stdout: 1 185122 9.67 MB/sec warmup 105 sec latency 19.070 ms 2024-09-19T09:05:51.040 INFO:tasks.workunit.client.0.smithi120.stdout: 1 188397 9.76 MB/sec warmup 106 sec latency 12.399 ms 2024-09-19T09:05:51.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:50 smithi120 ceph-mon[31150]: pgmap v430: 97 pgs: 97 active+clean; 44 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.3 MiB/s wr, 486 op/s 2024-09-19T09:05:51.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:50 smithi196 ceph-mon[35150]: pgmap v430: 97 pgs: 97 active+clean; 44 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.3 MiB/s wr, 486 op/s 2024-09-19T09:05:51.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:50 smithi153 ceph-mon[35071]: pgmap v430: 97 pgs: 97 active+clean; 44 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.3 MiB/s wr, 486 op/s 2024-09-19T09:05:51.846 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:05:51.870 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:05:52.040 INFO:tasks.workunit.client.0.smithi120.stdout: 1 191694 9.83 MB/sec warmup 107 sec latency 11.402 ms 2024-09-19T09:05:52.127 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:05:52.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.733+0000 7fafad570640 1 -- 172.21.15.120:0/4143333990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafa8100e20 msgr2=0x7fafa8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:52.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.733+0000 7fafad570640 1 --2- 172.21.15.120:0/4143333990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafa8100e20 0x7fafa8101200 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7faf98009900 tx=0x7faf9802a480 comp rx=0 tx=0).stop 2024-09-19T09:05:52.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.733+0000 7fafad570640 1 -- 172.21.15.120:0/4143333990 shutdown_connections 2024-09-19T09:05:52.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.733+0000 7fafad570640 1 --2- 172.21.15.120:0/4143333990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafa8102ef0 0x7fafa810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.733+0000 7fafad570640 1 --2- 172.21.15.120:0/4143333990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafa8102570 0x7fafa81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.733+0000 7fafad570640 1 --2- 172.21.15.120:0/4143333990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafa8100e20 0x7fafa8101200 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.733+0000 7fafad570640 1 -- 172.21.15.120:0/4143333990 >> 172.21.15.120:0/4143333990 conn(0x7fafa80fc7d0 msgr2=0x7fafa80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:52.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.733+0000 7fafad570640 1 -- 172.21.15.120:0/4143333990 shutdown_connections 2024-09-19T09:05:52.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.733+0000 7fafad570640 1 -- 172.21.15.120:0/4143333990 wait complete. 2024-09-19T09:05:52.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.734+0000 7fafad570640 1 Processor -- start 2024-09-19T09:05:52.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.734+0000 7fafad570640 1 -- start start 2024-09-19T09:05:52.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.734+0000 7fafad570640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafa8100e20 0x7fafa80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:52.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.734+0000 7fafad570640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafa8102570 0x7fafa80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:52.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.734+0000 7fafad570640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafa8102ef0 0x7fafa80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:52.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.734+0000 7fafad570640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fafa8110f50 con 0x7fafa8102ef0 2024-09-19T09:05:52.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.734+0000 7fafad570640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fafa8110dd0 con 0x7fafa8102570 2024-09-19T09:05:52.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.734+0000 7fafad570640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fafa81110d0 con 0x7fafa8100e20 2024-09-19T09:05:52.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.734+0000 7fafa77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafa8102ef0 0x7fafa80fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:52.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.734+0000 7fafa77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafa8102ef0 0x7fafa80fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55680/0 (socket says 172.21.15.120:55680) 2024-09-19T09:05:52.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.734+0000 7fafa77fe640 1 -- 172.21.15.120:0/2718934357 learned_addr learned my addr 172.21.15.120:0/2718934357 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:52.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.734+0000 7faf9e7fc640 1 --2- 172.21.15.120:0/2718934357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafa8102570 0x7fafa80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:52.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.734+0000 7fafa6ffd640 1 --2- 172.21.15.120:0/2718934357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafa8100e20 0x7fafa80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:52.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.734+0000 7fafa77fe640 1 -- 172.21.15.120:0/2718934357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafa8100e20 msgr2=0x7fafa80fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:52.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.734+0000 7fafa77fe640 1 --2- 172.21.15.120:0/2718934357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafa8100e20 0x7fafa80fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.734+0000 7fafa77fe640 1 -- 172.21.15.120:0/2718934357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafa8102570 msgr2=0x7fafa80fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:52.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.734+0000 7fafa77fe640 1 --2- 172.21.15.120:0/2718934357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafa8102570 0x7fafa80fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.734+0000 7fafa77fe640 1 -- 172.21.15.120:0/2718934357 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa81a6a10 con 0x7fafa8102ef0 2024-09-19T09:05:52.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.735+0000 7fafa77fe640 1 --2- 172.21.15.120:0/2718934357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafa8102ef0 0x7fafa80fd700 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fafa000eb10 tx=0x7fafa000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:52.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.735+0000 7fafa4ff9640 1 -- 172.21.15.120:0/2718934357 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa0004b20 con 0x7fafa8102ef0 2024-09-19T09:05:52.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.735+0000 7fafad570640 1 -- 172.21.15.120:0/2718934357 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa81a6d00 con 0x7fafa8102ef0 2024-09-19T09:05:52.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.735+0000 7fafa4ff9640 1 -- 172.21.15.120:0/2718934357 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafa0004cc0 con 0x7fafa8102ef0 2024-09-19T09:05:52.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.735+0000 7fafa4ff9640 1 -- 172.21.15.120:0/2718934357 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa0010760 con 0x7fafa8102ef0 2024-09-19T09:05:52.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.735+0000 7fafad570640 1 -- 172.21.15.120:0/2718934357 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa81a71f0 con 0x7fafa8102ef0 2024-09-19T09:05:52.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.738+0000 7fafaf949740 1 Processor -- start 2024-09-19T09:05:52.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.738+0000 7fafa4ff9640 1 -- 172.21.15.120:0/2718934357 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fafa0010900 con 0x7fafa8102ef0 2024-09-19T09:05:52.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.738+0000 7fafaf949740 1 -- start start 2024-09-19T09:05:52.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.738+0000 7fafaf949740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2d0543b60 0x55f2d0543f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:52.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.738+0000 7fafaf949740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d0544480 0x55f2d054e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:52.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.738+0000 7fafaf949740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f2d054ed40 0x55f2d0551130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:52.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.738+0000 7fafaf949740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f2d04a9280 con 0x55f2d054ed40 2024-09-19T09:05:52.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.738+0000 7fafaf949740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f2d0450b10 con 0x55f2d0543b60 2024-09-19T09:05:52.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.738+0000 7fafaf949740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f2d04a5240 con 0x55f2d0544480 2024-09-19T09:05:52.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.739+0000 7fafa6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d0544480 0x55f2d054e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:52.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.739+0000 7faf9e7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2d0543b60 0x55f2d0543f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:52.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.739+0000 7fafa6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d0544480 0x55f2d054e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:54312/0 (socket says 172.21.15.120:54312) 2024-09-19T09:05:52.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.739+0000 7fafa6ffd640 1 -- 172.21.15.120:0/2586914922 learned_addr learned my addr 172.21.15.120:0/2586914922 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:52.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.739+0000 7faf9e7fc640 1 --2- 172.21.15.120:0/2586914922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f2d054ed40 0x55f2d0551130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:52.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.739+0000 7fafa6ffd640 1 -- 172.21.15.120:0/2586914922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2d0543b60 msgr2=0x55f2d0543f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:52.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.739+0000 7fafa6ffd640 1 --2- 172.21.15.120:0/2586914922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2d0543b60 0x55f2d0543f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.739+0000 7fafa6ffd640 1 -- 172.21.15.120:0/2586914922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f2d054ed40 msgr2=0x55f2d0551130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:52.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.739+0000 7fafa6ffd640 1 --2- 172.21.15.120:0/2586914922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f2d054ed40 0x55f2d0551130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.739+0000 7fafa6ffd640 1 -- 172.21.15.120:0/2586914922 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2d045d1f0 con 0x55f2d0544480 2024-09-19T09:05:52.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.739+0000 7faf9e7fc640 1 --2- 172.21.15.120:0/2586914922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2d0543b60 0x55f2d0543f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:52.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.739+0000 7fafa4ff9640 1 --2- 172.21.15.120:0/2718934357 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faf940a3240 0x7faf940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:52.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.739+0000 7faf9e7fc640 1 --2- 172.21.15.120:0/2718934357 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faf940a3240 0x7faf940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:52.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.739+0000 7fafa6ffd640 1 --2- 172.21.15.120:0/2586914922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d0544480 0x55f2d054e800 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faf9802a9a0 tx=0x7faf98005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff3cc34bdf842fea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:52.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.739+0000 7fafa4ff9640 1 -- 172.21.15.120:0/2718934357 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fafa00cfe30 con 0x7fafa8102ef0 2024-09-19T09:05:52.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.739+0000 7faf9d7fa640 1 -- 172.21.15.120:0/2586914922 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf98035030 con 0x55f2d0544480 2024-09-19T09:05:52.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.739+0000 7faf9e7fc640 1 --2- 172.21.15.120:0/2718934357 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faf940a3240 0x7faf940a5700 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x55f2d0541470 tx=0x7faf90004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:52.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.739+0000 7faf9d7fa640 1 -- 172.21.15.120:0/2586914922 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf98040070 con 0x55f2d0544480 2024-09-19T09:05:52.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.739+0000 7faf9d7fa640 1 -- 172.21.15.120:0/2586914922 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf98031880 con 0x55f2d0544480 2024-09-19T09:05:52.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.740+0000 7fafaf949740 1 -- 172.21.15.120:0/2586914922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d0544480 msgr2=0x55f2d054e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:52.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.740+0000 7fafaf949740 1 --2- 172.21.15.120:0/2586914922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d0544480 0x55f2d054e800 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faf9802a9a0 tx=0x7faf98005e80 comp rx=0 tx=0).stop 2024-09-19T09:05:52.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.740+0000 7fafaf949740 1 -- 172.21.15.120:0/2586914922 shutdown_connections 2024-09-19T09:05:52.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.740+0000 7fafaf949740 1 --2- 172.21.15.120:0/2586914922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f2d054ed40 0x55f2d0551130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.740+0000 7fafaf949740 1 --2- 172.21.15.120:0/2586914922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d0544480 0x55f2d054e800 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.740+0000 7fafaf949740 1 --2- 172.21.15.120:0/2586914922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2d0543b60 0x55f2d0543f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.740+0000 7fafaf949740 1 -- 172.21.15.120:0/2586914922 >> 172.21.15.120:0/2586914922 conn(0x55f2d05400f0 msgr2=0x55f2d05404d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:52.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.741+0000 7fafaf949740 1 -- 172.21.15.120:0/2586914922 shutdown_connections 2024-09-19T09:05:52.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.741+0000 7fafaf949740 1 -- 172.21.15.120:0/2586914922 wait complete. 2024-09-19T09:05:52.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.741+0000 7fafaf949740 1 -- start start 2024-09-19T09:05:52.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.741+0000 7fafaf949740 1 Processor -- start 2024-09-19T09:05:52.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.741+0000 7fafaf949740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d0543b60 0x55f2d054bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:52.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.741+0000 7fafaf949740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2d0544480 0x55f2d054c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:52.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.741+0000 7fafaf949740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f2d054ed40 0x55f2d054c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:52.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.741+0000 7fafaf949740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f2d0551910 con 0x55f2d054ed40 2024-09-19T09:05:52.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.741+0000 7fafaf949740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f2d0463f10 con 0x55f2d0544480 2024-09-19T09:05:52.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.741+0000 7fafaf949740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f2d0551a90 con 0x55f2d0543b60 2024-09-19T09:05:52.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.741+0000 7fafaf949740 10 client.0 _ll_register_callbacks cb 0x55f2d042ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:52.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.741+0000 7faf9e7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2d0544480 0x55f2d054c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:52.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.741+0000 7fafa6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d0543b60 0x55f2d054bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:52.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.741+0000 7faf9e7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2d0544480 0x55f2d054c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:54776/0 (socket says 172.21.15.120:54776) 2024-09-19T09:05:52.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.741+0000 7faf9e7fc640 1 -- 172.21.15.120:0/1851417140 learned_addr learned my addr 172.21.15.120:0/1851417140 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:52.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.741+0000 7fafa77fe640 1 --2- 172.21.15.120:0/1851417140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f2d054ed40 0x55f2d054c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:52.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.742+0000 7faf9e7fc640 1 -- 172.21.15.120:0/1851417140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d0543b60 msgr2=0x55f2d054bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:52.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.742+0000 7faf9e7fc640 1 --2- 172.21.15.120:0/1851417140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d0543b60 0x55f2d054bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.742+0000 7faf9e7fc640 1 -- 172.21.15.120:0/1851417140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f2d054ed40 msgr2=0x55f2d054c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:52.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.742+0000 7faf9e7fc640 1 --2- 172.21.15.120:0/1851417140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f2d054ed40 0x55f2d054c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.742+0000 7faf9e7fc640 1 -- 172.21.15.120:0/1851417140 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2d021a6a0 con 0x55f2d0544480 2024-09-19T09:05:52.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.742+0000 7faf9e7fc640 1 --2- 172.21.15.120:0/1851417140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2d0544480 0x55f2d054c100 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7faf90012950 tx=0x7faf900046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6962dd120ddf9e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:52.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.742+0000 7faf8e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:05:52.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.742+0000 7faf8e7fc640 1 -- 172.21.15.120:0/1851417140 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9002d060 con 0x55f2d0544480 2024-09-19T09:05:52.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.742+0000 7faf8e7fc640 1 -- 172.21.15.120:0/1851417140 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7faf9001e640 con 0x55f2d0544480 2024-09-19T09:05:52.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.742+0000 7fafaf949740 1 -- 172.21.15.120:0/1851417140 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf98040070 con 0x55f2d0544480 2024-09-19T09:05:52.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.743+0000 7faf8e7fc640 1 -- 172.21.15.120:0/1851417140 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faf9001e840 con 0x55f2d0544480 2024-09-19T09:05:52.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.743+0000 7fafaf949740 10 client.25865 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:52.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.743+0000 7fafaf949740 1 -- 172.21.15.120:0/1851417140 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f2d055d4f0 con 0x55f2d0544480 2024-09-19T09:05:52.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.743+0000 7faf8e7fc640 1 -- 172.21.15.120:0/1851417140 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf98040070 con 0x55f2d0544480 2024-09-19T09:05:52.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.743+0000 7faf8e7fc640 1 -- 172.21.15.120:0/1851417140 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faf90027030 con 0x55f2d0544480 2024-09-19T09:05:52.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.743+0000 7fafaf949740 10 client.25865 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:52.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.743+0000 7fafaf949740 10 client.25865 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:52.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.743+0000 7fafaf949740 20 client.25865 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:52.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.743+0000 7fafaf949740 1 --2- 172.21.15.120:0/1851417140 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f2d055e9c0 0x55f2d0560e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:52.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.743+0000 7fafaf949740 4 client.25865 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:05:52.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.743+0000 7fafaf949740 1 -- 172.21.15.120:0/1851417140 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2d04b67f0 con 0x55f2d055e9c0 2024-09-19T09:05:52.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.743+0000 7fafa6ffd640 1 --2- 172.21.15.120:0/1851417140 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f2d055e9c0 0x55f2d0560e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:52.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.744+0000 7fafa6ffd640 1 --2- 172.21.15.120:0/1851417140 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f2d055e9c0 0x55f2d0560e00 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b97f47cbc0b0e47 server_cookie=4acee9b8016a0a85 in_seq=0 out_seq=0 2024-09-19T09:05:52.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.744+0000 7faf8e7fc640 10 client.25865 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:52.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.746+0000 7faf8e7fc640 1 -- 172.21.15.120:0/1851417140 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f2d04b67f0 con 0x55f2d055e9c0 2024-09-19T09:05:52.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.746+0000 7faf8e7fc640 10 client.25865 handle_command_reply: tid=0 2024-09-19T09:05:52.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.746+0000 7fafaf949740 1 client.25865 shutdown 2024-09-19T09:05:52.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.746+0000 7fafaf949740 1 -- 172.21.15.120:0/1851417140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2d0544480 msgr2=0x55f2d054c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:52.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.746+0000 7fafaf949740 1 --2- 172.21.15.120:0/1851417140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2d0544480 0x55f2d054c100 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7faf90012950 tx=0x7faf900046c0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.746+0000 7fafaf949740 1 -- 172.21.15.120:0/1851417140 shutdown_connections 2024-09-19T09:05:52.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.746+0000 7fafaf949740 1 --2- 172.21.15.120:0/1851417140 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f2d055e9c0 0x55f2d0560e00 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.746+0000 7fafaf949740 1 --2- 172.21.15.120:0/1851417140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f2d054ed40 0x55f2d054c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.746+0000 7fafaf949740 1 --2- 172.21.15.120:0/1851417140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2d0544480 0x55f2d054c100 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.746+0000 7faf8e7fc640 0 client.25865 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:52.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.746+0000 7fafaf949740 1 --2- 172.21.15.120:0/1851417140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d0543b60 0x55f2d054bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.746+0000 7fafaf949740 1 -- 172.21.15.120:0/1851417140 >> 172.21.15.120:0/1851417140 conn(0x55f2d05400f0 msgr2=0x55f2d0545dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:52.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.746+0000 7fafaf949740 1 -- 172.21.15.120:0/1851417140 shutdown_connections 2024-09-19T09:05:52.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.747+0000 7fafaf949740 1 -- 172.21.15.120:0/1851417140 wait complete. 2024-09-19T09:05:52.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.747+0000 7fafaf949740 20 client.25865 trim_cache size 0 max 16384 2024-09-19T09:05:52.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.766+0000 7fafaf949740 1 Processor -- start 2024-09-19T09:05:52.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.766+0000 7fafaf949740 1 -- start start 2024-09-19T09:05:52.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.766+0000 7fafaf949740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2d0543b60 0x55f2d0546080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:52.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.766+0000 7fafaf949740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d0544480 0x55f2d05666d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:52.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.766+0000 7fafaf949740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f2d054ed40 0x55f2d0566c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:52.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.766+0000 7fafaf949740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fafa81af810 con 0x55f2d054ed40 2024-09-19T09:05:52.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.766+0000 7fafaf949740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fafa81b1350 con 0x55f2d0543b60 2024-09-19T09:05:52.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.766+0000 7fafaf949740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fafa81afee0 con 0x55f2d0544480 2024-09-19T09:05:52.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.767+0000 7faf9e7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d0544480 0x55f2d05666d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:52.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.767+0000 7faf9e7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d0544480 0x55f2d05666d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:54320/0 (socket says 172.21.15.120:54320) 2024-09-19T09:05:52.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.767+0000 7fafa6ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2d0543b60 0x55f2d0546080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:52.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.767+0000 7faf9e7fc640 1 -- 172.21.15.120:0/1641188087 learned_addr learned my addr 172.21.15.120:0/1641188087 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:52.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.767+0000 7fafa77fe640 1 --2- 172.21.15.120:0/1641188087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f2d054ed40 0x55f2d0566c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:52.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.767+0000 7faf9e7fc640 1 -- 172.21.15.120:0/1641188087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2d0543b60 msgr2=0x55f2d0546080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:52.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.767+0000 7faf9e7fc640 1 --2- 172.21.15.120:0/1641188087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2d0543b60 0x55f2d0546080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.767+0000 7faf9e7fc640 1 -- 172.21.15.120:0/1641188087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f2d054ed40 msgr2=0x55f2d0566c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:52.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.767+0000 7faf9e7fc640 1 --2- 172.21.15.120:0/1641188087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f2d054ed40 0x55f2d0566c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.767+0000 7faf9e7fc640 1 -- 172.21.15.120:0/1641188087 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2d054e490 con 0x55f2d0544480 2024-09-19T09:05:52.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.767+0000 7fafa77fe640 1 --2- 172.21.15.120:0/1641188087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f2d054ed40 0x55f2d0566c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:52.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.767+0000 7fafa6ffd640 1 --2- 172.21.15.120:0/1641188087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2d0543b60 0x55f2d0546080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:52.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.767+0000 7faf9e7fc640 1 --2- 172.21.15.120:0/1641188087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d0544480 0x55f2d05666d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faf90012920 tx=0x7faf9000bb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f53d18b52764ccb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:52.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.767+0000 7faf9cff9640 1 -- 172.21.15.120:0/1641188087 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9002d040 con 0x55f2d0544480 2024-09-19T09:05:52.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.767+0000 7faf9cff9640 1 -- 172.21.15.120:0/1641188087 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf90012d50 con 0x55f2d0544480 2024-09-19T09:05:52.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.767+0000 7faf9cff9640 1 -- 172.21.15.120:0/1641188087 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf90037080 con 0x55f2d0544480 2024-09-19T09:05:52.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.768+0000 7fafaf949740 1 -- 172.21.15.120:0/1641188087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d0544480 msgr2=0x55f2d05666d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:52.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.768+0000 7fafaf949740 1 --2- 172.21.15.120:0/1641188087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d0544480 0x55f2d05666d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faf90012920 tx=0x7faf9000bb90 comp rx=0 tx=0).stop 2024-09-19T09:05:52.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.768+0000 7fafaf949740 1 -- 172.21.15.120:0/1641188087 shutdown_connections 2024-09-19T09:05:52.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.768+0000 7fafaf949740 1 --2- 172.21.15.120:0/1641188087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f2d054ed40 0x55f2d0566c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.768+0000 7fafaf949740 1 --2- 172.21.15.120:0/1641188087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d0544480 0x55f2d05666d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.768+0000 7fafaf949740 1 --2- 172.21.15.120:0/1641188087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2d0543b60 0x55f2d0546080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.768+0000 7fafaf949740 1 -- 172.21.15.120:0/1641188087 >> 172.21.15.120:0/1641188087 conn(0x55f2d05400f0 msgr2=0x55f2d054db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:52.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.768+0000 7fafaf949740 1 -- 172.21.15.120:0/1641188087 shutdown_connections 2024-09-19T09:05:52.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.768+0000 7fafaf949740 1 -- 172.21.15.120:0/1641188087 wait complete. 2024-09-19T09:05:52.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.769+0000 7fafaf949740 1 -- start start 2024-09-19T09:05:52.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.769+0000 7fafaf949740 1 Processor -- start 2024-09-19T09:05:52.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.769+0000 7fafaf949740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d0543b60 0x55f2d0552ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:52.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.769+0000 7fafaf949740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2d0544480 0x55f2d05531e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:52.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.769+0000 7fafaf949740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f2d054ed40 0x55f2d0553720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:52.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.769+0000 7fafaf949740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fafa81b0c80 con 0x55f2d054ed40 2024-09-19T09:05:52.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.769+0000 7fafaf949740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fafa81b05b0 con 0x55f2d0544480 2024-09-19T09:05:52.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.769+0000 7fafaf949740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fafa81ad7d0 con 0x55f2d0543b60 2024-09-19T09:05:52.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.769+0000 7fafa77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f2d054ed40 0x55f2d0553720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:52.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.769+0000 7faf9e7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2d0544480 0x55f2d05531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:52.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.769+0000 7fafa6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d0543b60 0x55f2d0552ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:52.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.769+0000 7fafa77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f2d054ed40 0x55f2d0553720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55720/0 (socket says 172.21.15.120:55720) 2024-09-19T09:05:52.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.769+0000 7fafa77fe640 1 -- 172.21.15.120:0/1559097860 learned_addr learned my addr 172.21.15.120:0/1559097860 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:52.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.769+0000 7fafaf949740 10 client.0 _ll_register_callbacks cb 0x7faf90012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:52.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.770+0000 7faf9e7fc640 1 -- 172.21.15.120:0/1559097860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d0543b60 msgr2=0x55f2d0552ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:52.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.770+0000 7faf9e7fc640 1 --2- 172.21.15.120:0/1559097860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d0543b60 0x55f2d0552ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.770+0000 7faf9e7fc640 1 -- 172.21.15.120:0/1559097860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f2d054ed40 msgr2=0x55f2d0553720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:52.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.770+0000 7faf9e7fc640 1 --2- 172.21.15.120:0/1559097860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f2d054ed40 0x55f2d0553720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.770+0000 7faf9e7fc640 1 -- 172.21.15.120:0/1559097860 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2d05d10b0 con 0x55f2d0544480 2024-09-19T09:05:52.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.770+0000 7faf9e7fc640 1 --2- 172.21.15.120:0/1559097860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2d0544480 0x55f2d05531e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faf9000bff0 tx=0x7faf9001ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59f5a6d3b5621828 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:52.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.770+0000 7faf8e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:05:52.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.770+0000 7faf8e7fc640 1 -- 172.21.15.120:0/1559097860 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9003f070 con 0x55f2d0544480 2024-09-19T09:05:52.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.770+0000 7faf8e7fc640 1 -- 172.21.15.120:0/1559097860 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7faf90010f90 con 0x55f2d0544480 2024-09-19T09:05:52.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.770+0000 7fafaf949740 1 -- 172.21.15.120:0/1559097860 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf90012d50 con 0x55f2d0544480 2024-09-19T09:05:52.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.770+0000 7faf8e7fc640 1 -- 172.21.15.120:0/1559097860 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faf9004b050 con 0x55f2d0544480 2024-09-19T09:05:52.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.770+0000 7fafaf949740 10 client.25871 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:52.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.770+0000 7fafaf949740 1 -- 172.21.15.120:0/1559097860 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f2d05d23b0 con 0x55f2d0544480 2024-09-19T09:05:52.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.771+0000 7faf8e7fc640 1 -- 172.21.15.120:0/1559097860 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf90012d50 con 0x55f2d0544480 2024-09-19T09:05:52.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.771+0000 7faf8e7fc640 1 -- 172.21.15.120:0/1559097860 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faf90027040 con 0x55f2d0544480 2024-09-19T09:05:52.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.771+0000 7fafaf949740 10 client.25871 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:52.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.771+0000 7fafaf949740 10 client.25871 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:52.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.771+0000 7fafaf949740 20 client.25871 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:52.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.771+0000 7fafaf949740 1 --2- 172.21.15.120:0/1559097860 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f2d05d2a20 0x55f2d05d4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:52.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.771+0000 7fafaf949740 4 client.25871 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:05:52.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.771+0000 7fafaf949740 1 -- 172.21.15.120:0/1559097860 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f2d04b0d00 con 0x55f2d05d2a20 2024-09-19T09:05:52.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.771+0000 7fafa6ffd640 1 --2- 172.21.15.120:0/1559097860 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f2d05d2a20 0x55f2d05d4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:52.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.771+0000 7faf8e7fc640 10 client.25871 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:52.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.771+0000 7fafa6ffd640 1 --2- 172.21.15.120:0/1559097860 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f2d05d2a20 0x55f2d05d4e40 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=953c955923f3e6f server_cookie=7ae99ec993cac911 in_seq=0 out_seq=0 2024-09-19T09:05:52.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.772+0000 7faf8e7fc640 1 -- 172.21.15.120:0/1559097860 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f2d04b0d00 con 0x55f2d05d2a20 2024-09-19T09:05:52.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.772+0000 7faf8e7fc640 10 client.25871 handle_command_reply: tid=0 2024-09-19T09:05:52.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.772+0000 7fafaf949740 1 client.25871 shutdown 2024-09-19T09:05:52.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.772+0000 7fafaf949740 1 -- 172.21.15.120:0/1559097860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2d0544480 msgr2=0x55f2d05531e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:52.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.772+0000 7fafaf949740 1 --2- 172.21.15.120:0/1559097860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2d0544480 0x55f2d05531e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faf9000bff0 tx=0x7faf9001ec90 comp rx=0 tx=0).stop 2024-09-19T09:05:52.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.773+0000 7fafaf949740 1 -- 172.21.15.120:0/1559097860 shutdown_connections 2024-09-19T09:05:52.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.773+0000 7fafaf949740 1 --2- 172.21.15.120:0/1559097860 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f2d05d2a20 0x55f2d05d4e40 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.773+0000 7fafaf949740 1 --2- 172.21.15.120:0/1559097860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f2d054ed40 0x55f2d0553720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.773+0000 7fafaf949740 1 --2- 172.21.15.120:0/1559097860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f2d0544480 0x55f2d05531e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.773+0000 7fafaf949740 1 --2- 172.21.15.120:0/1559097860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2d0543b60 0x55f2d0552ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.773+0000 7fafaf949740 1 -- 172.21.15.120:0/1559097860 >> 172.21.15.120:0/1559097860 conn(0x55f2d05400f0 msgr2=0x55f2d053ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:52.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.773+0000 7faf8e7fc640 0 client.25871 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:52.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.773+0000 7fafaf949740 1 -- 172.21.15.120:0/1559097860 shutdown_connections 2024-09-19T09:05:52.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.773+0000 7fafaf949740 1 -- 172.21.15.120:0/1559097860 wait complete. 2024-09-19T09:05:52.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.773+0000 7fafaf949740 20 client.25871 trim_cache size 0 max 16384 2024-09-19T09:05:52.779 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:05:52.779 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:05:52.779 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "cd93dc07-856c-41b5-b483-b1c4cc9f57c0", 2024-09-19T09:05:52.779 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:05:52.779 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:05:52.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.773+0000 7fafad570640 1 -- 172.21.15.120:0/2718934357 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faf940a3240 msgr2=0x7faf940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:52.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.773+0000 7fafad570640 1 --2- 172.21.15.120:0/2718934357 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faf940a3240 0x7faf940a5700 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x55f2d0541470 tx=0x7faf90004730 comp rx=0 tx=0).stop 2024-09-19T09:05:52.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.774+0000 7fafad570640 1 -- 172.21.15.120:0/2718934357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafa8102ef0 msgr2=0x7fafa80fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:52.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.774+0000 7fafad570640 1 --2- 172.21.15.120:0/2718934357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafa8102ef0 0x7fafa80fd700 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fafa000eb10 tx=0x7fafa000c2d0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.774+0000 7fafad570640 1 -- 172.21.15.120:0/2718934357 shutdown_connections 2024-09-19T09:05:52.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.774+0000 7fafad570640 1 --2- 172.21.15.120:0/2718934357 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faf940a3240 0x7faf940a5700 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.774+0000 7fafad570640 1 --2- 172.21.15.120:0/2718934357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafa8102ef0 0x7fafa80fd700 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.774+0000 7fafad570640 1 --2- 172.21.15.120:0/2718934357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafa8102570 0x7fafa80fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.774+0000 7fafad570640 1 --2- 172.21.15.120:0/2718934357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafa8100e20 0x7fafa80fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:52.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.774+0000 7fafad570640 1 -- 172.21.15.120:0/2718934357 >> 172.21.15.120:0/2718934357 conn(0x7fafa80fc7d0 msgr2=0x7fafa806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:52.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.774+0000 7fafad570640 1 -- 172.21.15.120:0/2718934357 shutdown_connections 2024-09-19T09:05:52.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:52.774+0000 7fafad570640 1 -- 172.21.15.120:0/2718934357 wait complete. 2024-09-19T09:05:53.040 INFO:tasks.workunit.client.0.smithi120.stdout: 1 194516 9.87 MB/sec warmup 108 sec latency 18.803 ms 2024-09-19T09:05:53.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:52 smithi120 ceph-mon[31150]: pgmap v431: 97 pgs: 97 active+clean; 45 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.4 MiB/s wr, 477 op/s 2024-09-19T09:05:53.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:52 smithi196 ceph-mon[35150]: pgmap v431: 97 pgs: 97 active+clean; 45 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.4 MiB/s wr, 477 op/s 2024-09-19T09:05:53.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:52 smithi153 ceph-mon[35071]: pgmap v431: 97 pgs: 97 active+clean; 45 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.4 MiB/s wr, 477 op/s 2024-09-19T09:05:53.716 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:05:53.980 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:05:54.040 INFO:tasks.workunit.client.0.smithi120.stdout: 1 196800 9.89 MB/sec warmup 109 sec latency 86.143 ms 2024-09-19T09:05:54.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:53 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:05:54.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:53 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:05:54.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:53 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:05:54.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:53 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:05:54.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:53 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:05:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.136+0000 7f636f28d640 1 -- 172.21.15.120:0/1281758878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6368101c30 msgr2=0x7f63681020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.136+0000 7f636f28d640 1 --2- 172.21.15.120:0/1281758878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6368101c30 0x7f63681020b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f6358009900 tx=0x7f635802a410 comp rx=0 tx=0).stop 2024-09-19T09:05:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.136+0000 7f636f28d640 1 -- 172.21.15.120:0/1281758878 shutdown_connections 2024-09-19T09:05:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.136+0000 7f636f28d640 1 --2- 172.21.15.120:0/1281758878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63681025f0 0x7f6368108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.136+0000 7f636f28d640 1 --2- 172.21.15.120:0/1281758878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6368101c30 0x7f63681020b0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.136+0000 7f636f28d640 1 --2- 172.21.15.120:0/1281758878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6368100a40 0x7f6368100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.136+0000 7f636f28d640 1 -- 172.21.15.120:0/1281758878 >> 172.21.15.120:0/1281758878 conn(0x7f63680fc1e0 msgr2=0x7f63680fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.136+0000 7f636f28d640 1 -- 172.21.15.120:0/1281758878 shutdown_connections 2024-09-19T09:05:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.136+0000 7f636f28d640 1 -- 172.21.15.120:0/1281758878 wait complete. 2024-09-19T09:05:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.137+0000 7f636f28d640 1 Processor -- start 2024-09-19T09:05:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.137+0000 7f636f28d640 1 -- start start 2024-09-19T09:05:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.137+0000 7f636f28d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6368100a40 0x7f636806a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.137+0000 7f636f28d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6368101c30 0x7f636819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.137+0000 7f636f28d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63681025f0 0x7f636819f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.137+0000 7f636f28d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f636810c6c0 con 0x7f63681025f0 2024-09-19T09:05:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.137+0000 7f636f28d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f636810c540 con 0x7f6368101c30 2024-09-19T09:05:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.137+0000 7f636f28d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f636810c840 con 0x7f6368100a40 2024-09-19T09:05:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.137+0000 7f636d82a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63681025f0 0x7f636819f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.137+0000 7f636d82a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63681025f0 0x7f636819f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55738/0 (socket says 172.21.15.120:55738) 2024-09-19T09:05:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.137+0000 7f636d82a640 1 -- 172.21.15.120:0/974747882 learned_addr learned my addr 172.21.15.120:0/974747882 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.137+0000 7f636c828640 1 --2- 172.21.15.120:0/974747882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6368101c30 0x7f636819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.137+0000 7f636d029640 1 --2- 172.21.15.120:0/974747882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6368100a40 0x7f636806a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.138+0000 7f636d82a640 1 -- 172.21.15.120:0/974747882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6368100a40 msgr2=0x7f636806a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.138+0000 7f636d82a640 1 --2- 172.21.15.120:0/974747882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6368100a40 0x7f636806a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.138+0000 7f636d82a640 1 -- 172.21.15.120:0/974747882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6368101c30 msgr2=0x7f636819cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.138+0000 7f636d82a640 1 --2- 172.21.15.120:0/974747882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6368101c30 0x7f636819cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.138+0000 7f636d82a640 1 -- 172.21.15.120:0/974747882 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636819fa90 con 0x7f63681025f0 2024-09-19T09:05:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.138+0000 7f636d82a640 1 --2- 172.21.15.120:0/974747882 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63681025f0 0x7f636819f180 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f636000efd0 tx=0x7f636000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.138+0000 7f635e7fc640 1 -- 172.21.15.120:0/974747882 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360003b70 con 0x7f63681025f0 2024-09-19T09:05:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.138+0000 7f635e7fc640 1 -- 172.21.15.120:0/974747882 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6360003d10 con 0x7f63681025f0 2024-09-19T09:05:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.138+0000 7f636f28d640 1 -- 172.21.15.120:0/974747882 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f636819fd20 con 0x7f63681025f0 2024-09-19T09:05:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.138+0000 7f635e7fc640 1 -- 172.21.15.120:0/974747882 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63600109f0 con 0x7f63681025f0 2024-09-19T09:05:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.138+0000 7f636f28d640 1 -- 172.21.15.120:0/974747882 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63681a01e0 con 0x7f63681025f0 2024-09-19T09:05:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.140+0000 7f635e7fc640 1 -- 172.21.15.120:0/974747882 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f6360004020 con 0x7f63681025f0 2024-09-19T09:05:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.141+0000 7f6372029740 1 Processor -- start 2024-09-19T09:05:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.141+0000 7f6372029740 1 -- start start 2024-09-19T09:05:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.141+0000 7f635e7fc640 1 --2- 172.21.15.120:0/974747882 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f63540a3370 0x7f63540a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.141+0000 7f6372029740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56065aff39a0 0x56065aff3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.141+0000 7f6372029740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56065aff42e0 0x56065aff8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.141+0000 7f6372029740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56065aff9370 0x56065affb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.141+0000 7f6372029740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56065af572f0 con 0x56065aff9370 2024-09-19T09:05:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.141+0000 7f6372029740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56065aeff270 con 0x56065aff42e0 2024-09-19T09:05:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.141+0000 7f6372029740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56065af532b0 con 0x56065aff39a0 2024-09-19T09:05:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.141+0000 7f636c828640 1 --2- 172.21.15.120:0/974747882 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f63540a3370 0x7f63540a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.141+0000 7f635e7fc640 1 -- 172.21.15.120:0/974747882 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f63600cfca0 con 0x7f63681025f0 2024-09-19T09:05:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.141+0000 7f636d029640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56065aff39a0 0x56065aff3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.141+0000 7f636d029640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56065aff39a0 0x56065aff3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:54352/0 (socket says 172.21.15.120:54352) 2024-09-19T09:05:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.141+0000 7f636d82a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56065aff9370 0x56065affb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.141+0000 7f636d029640 1 -- 172.21.15.120:0/3503659795 learned_addr learned my addr 172.21.15.120:0/3503659795 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.141+0000 7f636c828640 1 --2- 172.21.15.120:0/3503659795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56065aff42e0 0x56065aff8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.142+0000 7f636d029640 1 -- 172.21.15.120:0/3503659795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56065aff42e0 msgr2=0x56065aff8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.142+0000 7f636d029640 1 --2- 172.21.15.120:0/3503659795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56065aff42e0 0x56065aff8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.142+0000 7f636d029640 1 -- 172.21.15.120:0/3503659795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56065aff9370 msgr2=0x56065affb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.142+0000 7f636d029640 1 --2- 172.21.15.120:0/3503659795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56065aff9370 0x56065affb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.142+0000 7f636d029640 1 -- 172.21.15.120:0/3503659795 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56065aefe340 con 0x56065aff39a0 2024-09-19T09:05:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.142+0000 7f636c828640 1 --2- 172.21.15.120:0/974747882 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f63540a3370 0x7f63540a5830 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f63580314e0 tx=0x7f6358005e00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.142+0000 7f636d029640 1 --2- 172.21.15.120:0/3503659795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56065aff39a0 0x56065aff3da0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6364009ca0 tx=0x7f6364011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16f262512a607940 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.142+0000 7f63537fe640 1 -- 172.21.15.120:0/3503659795 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364004220 con 0x56065aff39a0 2024-09-19T09:05:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.142+0000 7f63537fe640 1 -- 172.21.15.120:0/3503659795 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63640043c0 con 0x56065aff39a0 2024-09-19T09:05:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.142+0000 7f63537fe640 1 -- 172.21.15.120:0/3503659795 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364018870 con 0x56065aff39a0 2024-09-19T09:05:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.143+0000 7f6372029740 1 -- 172.21.15.120:0/3503659795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56065aff39a0 msgr2=0x56065aff3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.143+0000 7f6372029740 1 --2- 172.21.15.120:0/3503659795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56065aff39a0 0x56065aff3da0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6364009ca0 tx=0x7f6364011810 comp rx=0 tx=0).stop 2024-09-19T09:05:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.143+0000 7f6372029740 1 -- 172.21.15.120:0/3503659795 shutdown_connections 2024-09-19T09:05:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.143+0000 7f6372029740 1 --2- 172.21.15.120:0/3503659795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56065aff9370 0x56065affb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.143+0000 7f6372029740 1 --2- 172.21.15.120:0/3503659795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56065aff42e0 0x56065aff8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.143+0000 7f6372029740 1 --2- 172.21.15.120:0/3503659795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56065aff39a0 0x56065aff3da0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.143+0000 7f6372029740 1 -- 172.21.15.120:0/3503659795 >> 172.21.15.120:0/3503659795 conn(0x56065afef010 msgr2=0x56065afef410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.143+0000 7f6372029740 1 -- 172.21.15.120:0/3503659795 shutdown_connections 2024-09-19T09:05:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.143+0000 7f6372029740 1 -- 172.21.15.120:0/3503659795 wait complete. 2024-09-19T09:05:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.143+0000 7f6372029740 1 -- start start 2024-09-19T09:05:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.143+0000 7f6372029740 1 Processor -- start 2024-09-19T09:05:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.144+0000 7f6372029740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56065aff39a0 0x56065affdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.144+0000 7f6372029740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56065aff42e0 0x56065affe170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.144+0000 7f6372029740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56065aff9370 0x56065b0051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.144+0000 7f6372029740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56065affc340 con 0x56065aff9370 2024-09-19T09:05:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.144+0000 7f6372029740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56065aee0290 con 0x56065aff39a0 2024-09-19T09:05:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.144+0000 7f6372029740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56065affc4c0 con 0x56065aff42e0 2024-09-19T09:05:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.144+0000 7f6372029740 10 client.0 _ll_register_callbacks cb 0x56065af11140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.144+0000 7f636d82a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56065aff9370 0x56065b0051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.144+0000 7f636d82a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56065aff9370 0x56065b0051b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55760/0 (socket says 172.21.15.120:55760) 2024-09-19T09:05:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.144+0000 7f636d82a640 1 -- 172.21.15.120:0/4286255184 learned_addr learned my addr 172.21.15.120:0/4286255184 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.144+0000 7f636c828640 1 --2- 172.21.15.120:0/4286255184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56065aff42e0 0x56065affe170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.144+0000 7f636d029640 1 --2- 172.21.15.120:0/4286255184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56065aff39a0 0x56065affdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.144+0000 7f636d82a640 1 -- 172.21.15.120:0/4286255184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56065aff42e0 msgr2=0x56065affe170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.144+0000 7f636d82a640 1 --2- 172.21.15.120:0/4286255184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56065aff42e0 0x56065affe170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.144+0000 7f636d82a640 1 -- 172.21.15.120:0/4286255184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56065aff39a0 msgr2=0x56065affdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.144+0000 7f636d82a640 1 --2- 172.21.15.120:0/4286255184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56065aff39a0 0x56065affdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.144+0000 7f636d82a640 1 -- 172.21.15.120:0/4286255184 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56065ac6a210 con 0x56065aff9370 2024-09-19T09:05:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.144+0000 7f636d82a640 1 --2- 172.21.15.120:0/4286255184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56065aff9370 0x56065b0051b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f63600cd7a0 tx=0x7f636009bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5094228b1882006 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.144+0000 7f6350ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.144+0000 7f6350ff9640 1 -- 172.21.15.120:0/4286255184 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636009c2c0 con 0x56065aff9370 2024-09-19T09:05:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.144+0000 7f6350ff9640 1 -- 172.21.15.120:0/4286255184 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f6360004310 con 0x56065aff9370 2024-09-19T09:05:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.144+0000 7f6372029740 1 -- 172.21.15.120:0/4286255184 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63640043c0 con 0x56065aff9370 2024-09-19T09:05:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.145+0000 7f6350ff9640 1 -- 172.21.15.120:0/4286255184 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f63600a0670 con 0x56065aff9370 2024-09-19T09:05:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.145+0000 7f6372029740 10 client.16521 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.145+0000 7f6372029740 1 -- 172.21.15.120:0/4286255184 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56065ad36620 con 0x56065aff9370 2024-09-19T09:05:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.145+0000 7f6350ff9640 1 -- 172.21.15.120:0/4286255184 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63640043c0 con 0x56065aff9370 2024-09-19T09:05:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.145+0000 7f6350ff9640 1 -- 172.21.15.120:0/4286255184 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f636001f020 con 0x56065aff9370 2024-09-19T09:05:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.145+0000 7f6372029740 10 client.16521 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.145+0000 7f6372029740 10 client.16521 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.145+0000 7f6372029740 20 client.16521 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.145+0000 7f6372029740 1 --2- 172.21.15.120:0/4286255184 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56065b009870 0x56065b00bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.145+0000 7f6372029740 4 client.16521 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:05:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.145+0000 7f6372029740 1 -- 172.21.15.120:0/4286255184 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56065af64a20 con 0x56065b009870 2024-09-19T09:05:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.145+0000 7f636d029640 1 --2- 172.21.15.120:0/4286255184 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56065b009870 0x56065b00bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.146+0000 7f6350ff9640 10 client.16521 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.146+0000 7f636d029640 1 --2- 172.21.15.120:0/4286255184 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56065b009870 0x56065b00bc60 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6008e4244f520928 server_cookie=c76c41fa8b119ef5 in_seq=0 out_seq=0 2024-09-19T09:05:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.146+0000 7f6350ff9640 1 -- 172.21.15.120:0/4286255184 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56065af64a20 con 0x56065b009870 2024-09-19T09:05:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.146+0000 7f6350ff9640 10 client.16521 handle_command_reply: tid=0 2024-09-19T09:05:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.146+0000 7f6372029740 1 client.16521 shutdown 2024-09-19T09:05:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.146+0000 7f6372029740 1 -- 172.21.15.120:0/4286255184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56065aff9370 msgr2=0x56065b0051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.146+0000 7f6372029740 1 --2- 172.21.15.120:0/4286255184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56065aff9370 0x56065b0051b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f63600cd7a0 tx=0x7f636009bec0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.146+0000 7f6372029740 1 -- 172.21.15.120:0/4286255184 shutdown_connections 2024-09-19T09:05:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.146+0000 7f6372029740 1 --2- 172.21.15.120:0/4286255184 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56065b009870 0x56065b00bc60 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.146+0000 7f6372029740 1 --2- 172.21.15.120:0/4286255184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56065aff9370 0x56065b0051b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.146+0000 7f6372029740 1 --2- 172.21.15.120:0/4286255184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56065aff42e0 0x56065affe170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.146+0000 7f6350ff9640 0 client.16521 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.146+0000 7f6372029740 1 --2- 172.21.15.120:0/4286255184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56065aff39a0 0x56065affdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.146+0000 7f6372029740 1 -- 172.21.15.120:0/4286255184 >> 172.21.15.120:0/4286255184 conn(0x56065afef010 msgr2=0x56065aff0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.146+0000 7f6372029740 1 -- 172.21.15.120:0/4286255184 shutdown_connections 2024-09-19T09:05:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.146+0000 7f6372029740 1 -- 172.21.15.120:0/4286255184 wait complete. 2024-09-19T09:05:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.147+0000 7f6372029740 20 client.16521 trim_cache size 0 max 16384 2024-09-19T09:05:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.163+0000 7f6372029740 1 Processor -- start 2024-09-19T09:05:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.163+0000 7f6372029740 1 -- start start 2024-09-19T09:05:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.163+0000 7f6372029740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56065aff39a0 0x56065afefec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.163+0000 7f6372029740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56065aff42e0 0x56065b005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.163+0000 7f6372029740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56065aff9370 0x56065aff0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.163+0000 7f6372029740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f63681a8f80 con 0x56065aff42e0 2024-09-19T09:05:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.163+0000 7f6372029740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f63681aaac0 con 0x56065aff9370 2024-09-19T09:05:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.163+0000 7f6372029740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f63681a9650 con 0x56065aff39a0 2024-09-19T09:05:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.164+0000 7f636c828640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56065aff42e0 0x56065b005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.164+0000 7f636c828640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56065aff42e0 0x56065b005170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55764/0 (socket says 172.21.15.120:55764) 2024-09-19T09:05:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.164+0000 7f636c828640 1 -- 172.21.15.120:0/2634460527 learned_addr learned my addr 172.21.15.120:0/2634460527 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.164+0000 7f636d029640 1 --2- 172.21.15.120:0/2634460527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56065aff39a0 0x56065afefec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.164+0000 7f636d82a640 1 --2- 172.21.15.120:0/2634460527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56065aff9370 0x56065aff0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.164+0000 7f636c828640 1 -- 172.21.15.120:0/2634460527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56065aff39a0 msgr2=0x56065afefec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.164+0000 7f636c828640 1 --2- 172.21.15.120:0/2634460527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56065aff39a0 0x56065afefec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.164+0000 7f636c828640 1 -- 172.21.15.120:0/2634460527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56065aff9370 msgr2=0x56065aff0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.164+0000 7f636c828640 1 --2- 172.21.15.120:0/2634460527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56065aff9370 0x56065aff0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.164+0000 7f636c828640 1 -- 172.21.15.120:0/2634460527 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56065afefca0 con 0x56065aff42e0 2024-09-19T09:05:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.164+0000 7f636c828640 1 --2- 172.21.15.120:0/2634460527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56065aff42e0 0x56065b005170 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f63580096c0 tx=0x7f6358049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28f025e0b9031a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.164+0000 7f6352ffd640 1 -- 172.21.15.120:0/2634460527 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6358004780 con 0x56065aff42e0 2024-09-19T09:05:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.164+0000 7f6352ffd640 1 -- 172.21.15.120:0/2634460527 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6358004e00 con 0x56065aff42e0 2024-09-19T09:05:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.164+0000 7f6352ffd640 1 -- 172.21.15.120:0/2634460527 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6358003850 con 0x56065aff42e0 2024-09-19T09:05:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.165+0000 7f6372029740 1 -- 172.21.15.120:0/2634460527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56065aff42e0 msgr2=0x56065b005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.165+0000 7f6372029740 1 --2- 172.21.15.120:0/2634460527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56065aff42e0 0x56065b005170 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f63580096c0 tx=0x7f6358049000 comp rx=0 tx=0).stop 2024-09-19T09:05:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.165+0000 7f6372029740 1 -- 172.21.15.120:0/2634460527 shutdown_connections 2024-09-19T09:05:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.165+0000 7f6372029740 1 --2- 172.21.15.120:0/2634460527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56065aff9370 0x56065aff0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.165+0000 7f6372029740 1 --2- 172.21.15.120:0/2634460527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56065aff42e0 0x56065b005170 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.165+0000 7f6372029740 1 --2- 172.21.15.120:0/2634460527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56065aff39a0 0x56065afefec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.165+0000 7f6372029740 1 -- 172.21.15.120:0/2634460527 >> 172.21.15.120:0/2634460527 conn(0x56065afef010 msgr2=0x56065affdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.165+0000 7f6372029740 1 -- 172.21.15.120:0/2634460527 shutdown_connections 2024-09-19T09:05:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.165+0000 7f6372029740 1 -- 172.21.15.120:0/2634460527 wait complete. 2024-09-19T09:05:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.165+0000 7f6372029740 1 -- start start 2024-09-19T09:05:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.165+0000 7f6372029740 1 Processor -- start 2024-09-19T09:05:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.166+0000 7f6372029740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56065aff39a0 0x56065b002a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.166+0000 7f6372029740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56065aff42e0 0x56065b002fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.166+0000 7f6372029740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56065aff9370 0x56065b003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.166+0000 7f6372029740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f63681aa3f0 con 0x56065aff9370 2024-09-19T09:05:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.166+0000 7f6372029740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f63681a9d20 con 0x56065aff42e0 2024-09-19T09:05:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.166+0000 7f6372029740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f63681a6f40 con 0x56065aff39a0 2024-09-19T09:05:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.166+0000 7f6372029740 10 client.0 _ll_register_callbacks cb 0x7f6360009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.166+0000 7f636d82a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56065aff9370 0x56065b003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.166+0000 7f636c828640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56065aff42e0 0x56065b002fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.166+0000 7f636d82a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56065aff9370 0x56065b003f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55778/0 (socket says 172.21.15.120:55778) 2024-09-19T09:05:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.166+0000 7f636d82a640 1 -- 172.21.15.120:0/2104216589 learned_addr learned my addr 172.21.15.120:0/2104216589 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.166+0000 7f636d029640 1 --2- 172.21.15.120:0/2104216589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56065aff39a0 0x56065b002a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.166+0000 7f636d82a640 1 -- 172.21.15.120:0/2104216589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56065aff39a0 msgr2=0x56065b002a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.166+0000 7f636d82a640 1 --2- 172.21.15.120:0/2104216589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56065aff39a0 0x56065b002a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.166+0000 7f636d82a640 1 -- 172.21.15.120:0/2104216589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56065aff42e0 msgr2=0x56065b002fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.166+0000 7f636d82a640 1 --2- 172.21.15.120:0/2104216589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56065aff42e0 0x56065b002fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.166+0000 7f636d82a640 1 -- 172.21.15.120:0/2104216589 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56065b004570 con 0x56065aff9370 2024-09-19T09:05:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.166+0000 7f636c828640 1 --2- 172.21.15.120:0/2104216589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56065aff42e0 0x56065b002fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.166+0000 7f636d82a640 1 --2- 172.21.15.120:0/2104216589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56065aff9370 0x56065b003f60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f636009bfc0 tx=0x7f6360052ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873c0f3fd32c3d1d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.166+0000 7f6350ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.166+0000 7f6350ff9640 1 -- 172.21.15.120:0/2104216589 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636009c2c0 con 0x56065aff9370 2024-09-19T09:05:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.166+0000 7f6350ff9640 1 -- 172.21.15.120:0/2104216589 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f63600cdc60 con 0x56065aff9370 2024-09-19T09:05:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.166+0000 7f6372029740 1 -- 172.21.15.120:0/2104216589 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6358004e00 con 0x56065aff9370 2024-09-19T09:05:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.167+0000 7f6350ff9640 1 -- 172.21.15.120:0/2104216589 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f63600cde60 con 0x56065aff9370 2024-09-19T09:05:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.167+0000 7f6372029740 10 client.16527 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.167+0000 7f6372029740 1 -- 172.21.15.120:0/2104216589 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56065b074680 con 0x56065aff9370 2024-09-19T09:05:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.167+0000 7f6350ff9640 1 -- 172.21.15.120:0/2104216589 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6358004e00 con 0x56065aff9370 2024-09-19T09:05:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.167+0000 7f6350ff9640 1 -- 172.21.15.120:0/2104216589 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f636001f030 con 0x56065aff9370 2024-09-19T09:05:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.167+0000 7f6372029740 10 client.16527 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.167+0000 7f6372029740 10 client.16527 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.167+0000 7f6372029740 20 client.16527 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.167+0000 7f6372029740 1 --2- 172.21.15.120:0/2104216589 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56065b074e20 0x56065b077240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.167+0000 7f6372029740 4 client.16527 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:05:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.167+0000 7f6372029740 1 -- 172.21.15.120:0/2104216589 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56065af5ed70 con 0x56065b074e20 2024-09-19T09:05:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.167+0000 7f636d029640 1 --2- 172.21.15.120:0/2104216589 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56065b074e20 0x56065b077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.167+0000 7f6350ff9640 10 client.16527 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.167+0000 7f636d029640 1 --2- 172.21.15.120:0/2104216589 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56065b074e20 0x56065b077240 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f7a09618c1544dd server_cookie=1b6f77257830ff22 in_seq=0 out_seq=0 2024-09-19T09:05:54.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:53 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:05:54.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:53 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:05:54.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:53 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:05:54.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:53 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:05:54.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:53 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:05:54.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:53 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:05:54.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:53 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:05:54.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:53 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:05:54.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:53 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:05:54.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:53 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:05:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.319+0000 7f6350ff9640 1 -- 172.21.15.120:0/2104216589 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56065af5ed70 con 0x56065b074e20 2024-09-19T09:05:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.319+0000 7f6350ff9640 10 client.16527 handle_command_reply: tid=0 2024-09-19T09:05:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.319+0000 7f6372029740 1 client.16527 shutdown 2024-09-19T09:05:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.319+0000 7f6372029740 1 -- 172.21.15.120:0/2104216589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56065aff9370 msgr2=0x56065b003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.319+0000 7f6372029740 1 --2- 172.21.15.120:0/2104216589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56065aff9370 0x56065b003f60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f636009bfc0 tx=0x7f6360052ec0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.319+0000 7f6372029740 1 -- 172.21.15.120:0/2104216589 shutdown_connections 2024-09-19T09:05:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.319+0000 7f6372029740 1 --2- 172.21.15.120:0/2104216589 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56065b074e20 0x56065b077240 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.319+0000 7f6350ff9640 0 client.16527 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.319+0000 7f6372029740 1 --2- 172.21.15.120:0/2104216589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56065aff9370 0x56065b003f60 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.319+0000 7f6372029740 1 --2- 172.21.15.120:0/2104216589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56065aff42e0 0x56065b002fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.319+0000 7f6372029740 1 --2- 172.21.15.120:0/2104216589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56065aff39a0 0x56065b002a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.319+0000 7f6372029740 1 -- 172.21.15.120:0/2104216589 >> 172.21.15.120:0/2104216589 conn(0x56065afef010 msgr2=0x56065afed9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.319+0000 7f6372029740 1 -- 172.21.15.120:0/2104216589 shutdown_connections 2024-09-19T09:05:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.319+0000 7f6372029740 1 -- 172.21.15.120:0/2104216589 wait complete. 2024-09-19T09:05:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.319+0000 7f6372029740 20 client.16527 trim_cache size 0 max 16384 2024-09-19T09:05:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:05:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:05:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:05:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:05:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.320+0000 7f636f28d640 1 -- 172.21.15.120:0/974747882 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f63540a3370 msgr2=0x7f63540a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.320+0000 7f636f28d640 1 --2- 172.21.15.120:0/974747882 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f63540a3370 0x7f63540a5830 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f63580314e0 tx=0x7f6358005e00 comp rx=0 tx=0).stop 2024-09-19T09:05:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.320+0000 7f636f28d640 1 -- 172.21.15.120:0/974747882 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63681025f0 msgr2=0x7f636819f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.320+0000 7f636f28d640 1 --2- 172.21.15.120:0/974747882 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63681025f0 0x7f636819f180 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f636000efd0 tx=0x7f636000c6a0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.321+0000 7f636f28d640 1 -- 172.21.15.120:0/974747882 shutdown_connections 2024-09-19T09:05:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.321+0000 7f636f28d640 1 --2- 172.21.15.120:0/974747882 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f63540a3370 0x7f63540a5830 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.321+0000 7f636f28d640 1 --2- 172.21.15.120:0/974747882 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63681025f0 0x7f636819f180 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.321+0000 7f636f28d640 1 --2- 172.21.15.120:0/974747882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6368101c30 0x7f636819cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.321+0000 7f636f28d640 1 --2- 172.21.15.120:0/974747882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6368100a40 0x7f636806a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.321+0000 7f636f28d640 1 -- 172.21.15.120:0/974747882 >> 172.21.15.120:0/974747882 conn(0x7f63680fc1e0 msgr2=0x7f6368104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.321+0000 7f636f28d640 1 -- 172.21.15.120:0/974747882 shutdown_connections 2024-09-19T09:05:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:05:54.321+0000 7f636f28d640 1 -- 172.21.15.120:0/974747882 wait complete. 2024-09-19T09:05:54.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.519+0000 7fe66d997640 1 -- 172.21.15.120:0/3531066510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe668102ef0 msgr2=0x7fe66810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:54.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.519+0000 7fe66d997640 1 --2- 172.21.15.120:0/3531066510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe668102ef0 0x7fe66810da40 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe660009ca0 tx=0x7fe66002a950 comp rx=0 tx=0).stop 2024-09-19T09:05:54.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.519+0000 7fe66d997640 1 -- 172.21.15.120:0/3531066510 shutdown_connections 2024-09-19T09:05:54.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.519+0000 7fe66d997640 1 --2- 172.21.15.120:0/3531066510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe668102ef0 0x7fe66810da40 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.519+0000 7fe66d997640 1 --2- 172.21.15.120:0/3531066510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe668102570 0x7fe6681029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.519+0000 7fe66d997640 1 --2- 172.21.15.120:0/3531066510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe668100e20 0x7fe668101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.519+0000 7fe66d997640 1 -- 172.21.15.120:0/3531066510 >> 172.21.15.120:0/3531066510 conn(0x7fe6680fc7d0 msgr2=0x7fe6680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:54.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.519+0000 7fe66d997640 1 -- 172.21.15.120:0/3531066510 shutdown_connections 2024-09-19T09:05:54.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.520+0000 7fe66d997640 1 -- 172.21.15.120:0/3531066510 wait complete. 2024-09-19T09:05:54.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.520+0000 7fe66d997640 1 Processor -- start 2024-09-19T09:05:54.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.520+0000 7fe66d997640 1 -- start start 2024-09-19T09:05:54.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.520+0000 7fe66d997640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe668100e20 0x7fe6681984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.520+0000 7fe66d997640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe668102570 0x7fe668198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.520+0000 7fe66d997640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe668102ef0 0x7fe66819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.520+0000 7fe66d997640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe668110f80 con 0x7fe668100e20 2024-09-19T09:05:54.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.520+0000 7fe66d997640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe668110e00 con 0x7fe668102570 2024-09-19T09:05:54.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.520+0000 7fe66d997640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe668111100 con 0x7fe668102ef0 2024-09-19T09:05:54.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.521+0000 7fe666ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe668100e20 0x7fe6681984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.521+0000 7fe666ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe668100e20 0x7fe6681984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55804/0 (socket says 172.21.15.120:55804) 2024-09-19T09:05:54.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.521+0000 7fe666ffd640 1 -- 172.21.15.120:0/1436629996 learned_addr learned my addr 172.21.15.120:0/1436629996 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:54.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.521+0000 7fe65e7fc640 1 --2- 172.21.15.120:0/1436629996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe668102570 0x7fe668198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.521+0000 7fe6677fe640 1 --2- 172.21.15.120:0/1436629996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe668102ef0 0x7fe66819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.521+0000 7fe666ffd640 1 -- 172.21.15.120:0/1436629996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe668102ef0 msgr2=0x7fe66819cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:54.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.521+0000 7fe666ffd640 1 --2- 172.21.15.120:0/1436629996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe668102ef0 0x7fe66819cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.521+0000 7fe666ffd640 1 -- 172.21.15.120:0/1436629996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe668102570 msgr2=0x7fe668198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:54.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.521+0000 7fe666ffd640 1 --2- 172.21.15.120:0/1436629996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe668102570 0x7fe668198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.521+0000 7fe666ffd640 1 -- 172.21.15.120:0/1436629996 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66819d470 con 0x7fe668100e20 2024-09-19T09:05:54.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.521+0000 7fe65e7fc640 1 --2- 172.21.15.120:0/1436629996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe668102570 0x7fe668198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:54.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.521+0000 7fe666ffd640 1 --2- 172.21.15.120:0/1436629996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe668100e20 0x7fe6681984c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe65800e9f0 tx=0x7fe65800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:54.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.521+0000 7fe66c995640 1 -- 172.21.15.120:0/1436629996 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65800cd80 con 0x7fe668100e20 2024-09-19T09:05:54.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.521+0000 7fe66d997640 1 -- 172.21.15.120:0/1436629996 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe66819d760 con 0x7fe668100e20 2024-09-19T09:05:54.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.521+0000 7fe66c995640 1 -- 172.21.15.120:0/1436629996 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe658010910 con 0x7fe668100e20 2024-09-19T09:05:54.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.521+0000 7fe66c995640 1 -- 172.21.15.120:0/1436629996 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6580188f0 con 0x7fe668100e20 2024-09-19T09:05:54.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.521+0000 7fe66d997640 1 -- 172.21.15.120:0/1436629996 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6681a5040 con 0x7fe668100e20 2024-09-19T09:05:54.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.524+0000 7fe66fd70740 1 Processor -- start 2024-09-19T09:05:54.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.524+0000 7fe66fd70740 1 -- start start 2024-09-19T09:05:54.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.524+0000 7fe66c995640 1 -- 172.21.15.120:0/1436629996 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe658010ac0 con 0x7fe668100e20 2024-09-19T09:05:54.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.524+0000 7fe66fd70740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a6631e4a0 0x558a6631e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.525+0000 7fe66fd70740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a6631edc0 0x558a66329140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.525+0000 7fe66fd70740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a66329680 0x558a6632ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.525+0000 7fe66fd70740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558a662842e0 con 0x558a6631e4a0 2024-09-19T09:05:54.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.525+0000 7fe66fd70740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558a6622bce0 con 0x558a6631edc0 2024-09-19T09:05:54.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.525+0000 7fe66fd70740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558a662802a0 con 0x558a66329680 2024-09-19T09:05:54.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.525+0000 7fe6677fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a6631e4a0 0x558a6631e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.525+0000 7fe6677fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a6631e4a0 0x558a6631e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55816/0 (socket says 172.21.15.120:55816) 2024-09-19T09:05:54.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.525+0000 7fe6677fe640 1 -- 172.21.15.120:0/2647318857 learned_addr learned my addr 172.21.15.120:0/2647318857 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:54.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.525+0000 7fe65e7fc640 1 --2- 172.21.15.120:0/2647318857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a66329680 0x558a6632ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.525+0000 7fe666ffd640 1 --2- 172.21.15.120:0/2647318857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a6631edc0 0x558a66329140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.525+0000 7fe6677fe640 1 -- 172.21.15.120:0/2647318857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a66329680 msgr2=0x558a6632ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:54.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.525+0000 7fe6677fe640 1 --2- 172.21.15.120:0/2647318857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a66329680 0x558a6632ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.525+0000 7fe6677fe640 1 -- 172.21.15.120:0/2647318857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a6631edc0 msgr2=0x558a66329140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:54.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.525+0000 7fe6677fe640 1 --2- 172.21.15.120:0/2647318857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a6631edc0 0x558a66329140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.525+0000 7fe6677fe640 1 -- 172.21.15.120:0/2647318857 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a66210440 con 0x558a6631e4a0 2024-09-19T09:05:54.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.525+0000 7fe66c995640 1 --2- 172.21.15.120:0/1436629996 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe6540a3240 0x7fe6540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.525+0000 7fe6677fe640 1 --2- 172.21.15.120:0/2647318857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a6631e4a0 0x558a6631e880 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe660035000 tx=0x7fe6600093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3009323b175d6aa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:54.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.525+0000 7fe666ffd640 1 --2- 172.21.15.120:0/1436629996 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe6540a3240 0x7fe6540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.525+0000 7fe65dffb640 1 -- 172.21.15.120:0/2647318857 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe660048070 con 0x558a6631e4a0 2024-09-19T09:05:54.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.525+0000 7fe66c995640 1 -- 172.21.15.120:0/1436629996 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe6580cfca0 con 0x7fe668100e20 2024-09-19T09:05:54.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.526+0000 7fe65dffb640 1 -- 172.21.15.120:0/2647318857 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe66002f930 con 0x558a6631e4a0 2024-09-19T09:05:54.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.526+0000 7fe65dffb640 1 -- 172.21.15.120:0/2647318857 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe660034040 con 0x558a6631e4a0 2024-09-19T09:05:54.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.526+0000 7fe66fd70740 1 -- 172.21.15.120:0/2647318857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a6631e4a0 msgr2=0x558a6631e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:54.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.526+0000 7fe66fd70740 1 --2- 172.21.15.120:0/2647318857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a6631e4a0 0x558a6631e880 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe660035000 tx=0x7fe6600093e0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.526+0000 7fe666ffd640 1 --2- 172.21.15.120:0/1436629996 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe6540a3240 0x7fe6540a5700 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe658098d40 tx=0x7fe6580d3720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:54.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.526+0000 7fe66fd70740 1 -- 172.21.15.120:0/2647318857 shutdown_connections 2024-09-19T09:05:54.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.526+0000 7fe66fd70740 1 --2- 172.21.15.120:0/2647318857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a66329680 0x558a6632ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.526+0000 7fe66fd70740 1 --2- 172.21.15.120:0/2647318857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a6631edc0 0x558a66329140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.526+0000 7fe66fd70740 1 --2- 172.21.15.120:0/2647318857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a6631e4a0 0x558a6631e880 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.526+0000 7fe66fd70740 1 -- 172.21.15.120:0/2647318857 >> 172.21.15.120:0/2647318857 conn(0x558a6631aa30 msgr2=0x558a6631ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:54.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.526+0000 7fe66fd70740 1 -- 172.21.15.120:0/2647318857 shutdown_connections 2024-09-19T09:05:54.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.526+0000 7fe66fd70740 1 -- 172.21.15.120:0/2647318857 wait complete. 2024-09-19T09:05:54.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.527+0000 7fe66fd70740 1 -- start start 2024-09-19T09:05:54.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.527+0000 7fe66fd70740 1 Processor -- start 2024-09-19T09:05:54.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.527+0000 7fe66fd70740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a6631e4a0 0x558a66328fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.527+0000 7fe66fd70740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a6631edc0 0x558a66326280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.527+0000 7fe66fd70740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a66329680 0x558a663267e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.527+0000 7fe66fd70740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558a6632c250 con 0x558a66329680 2024-09-19T09:05:54.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.527+0000 7fe66fd70740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558a6623f050 con 0x558a6631edc0 2024-09-19T09:05:54.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.527+0000 7fe66fd70740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558a6632c3d0 con 0x558a6631e4a0 2024-09-19T09:05:54.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.527+0000 7fe66fd70740 10 client.0 _ll_register_callbacks cb 0x558a6627f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:54.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.527+0000 7fe6677fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a6631e4a0 0x558a66328fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.528+0000 7fe65e7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a6631edc0 0x558a66326280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.528+0000 7fe6677fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a6631e4a0 0x558a66328fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:54436/0 (socket says 172.21.15.120:54436) 2024-09-19T09:05:54.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.528+0000 7fe6677fe640 1 -- 172.21.15.120:0/682894619 learned_addr learned my addr 172.21.15.120:0/682894619 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:54.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.528+0000 7fe6677fe640 1 --2- 172.21.15.120:0/682894619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a66329680 0x558a663267e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.528+0000 7fe65e7fc640 1 -- 172.21.15.120:0/682894619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a6631e4a0 msgr2=0x558a66328fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:54.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.528+0000 7fe65e7fc640 1 --2- 172.21.15.120:0/682894619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a6631e4a0 0x558a66328fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.528+0000 7fe65e7fc640 1 -- 172.21.15.120:0/682894619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a66329680 msgr2=0x558a663267e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:54.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.528+0000 7fe65e7fc640 1 --2- 172.21.15.120:0/682894619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a66329680 0x558a663267e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.528+0000 7fe65e7fc640 1 -- 172.21.15.120:0/682894619 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a65ff53c0 con 0x558a6631edc0 2024-09-19T09:05:54.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.528+0000 7fe6677fe640 1 --2- 172.21.15.120:0/682894619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a66329680 0x558a663267e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:54.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.528+0000 7fe65e7fc640 1 --2- 172.21.15.120:0/682894619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a6631edc0 0x558a66326280 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe6680fee90 tx=0x7fe65000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1de9f13b5c4aff5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:54.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.528+0000 7fe646ffd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:05:54.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.528+0000 7fe646ffd640 1 -- 172.21.15.120:0/682894619 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe650012110 con 0x558a6631edc0 2024-09-19T09:05:54.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.528+0000 7fe66fd70740 1 -- 172.21.15.120:0/682894619 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe66002f930 con 0x558a6631edc0 2024-09-19T09:05:54.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.528+0000 7fe646ffd640 1 -- 172.21.15.120:0/682894619 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe650004050 con 0x558a6631edc0 2024-09-19T09:05:54.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.528+0000 7fe646ffd640 1 -- 172.21.15.120:0/682894619 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe6500122b0 con 0x558a6631edc0 2024-09-19T09:05:54.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.528+0000 7fe66fd70740 10 client.25901 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:54.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.529+0000 7fe66fd70740 1 -- 172.21.15.120:0/682894619 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558a66337c80 con 0x558a6631edc0 2024-09-19T09:05:54.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.529+0000 7fe646ffd640 1 -- 172.21.15.120:0/682894619 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe66002f930 con 0x558a6631edc0 2024-09-19T09:05:54.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.529+0000 7fe646ffd640 1 -- 172.21.15.120:0/682894619 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe6500104c0 con 0x558a6631edc0 2024-09-19T09:05:54.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.529+0000 7fe66fd70740 10 client.25901 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:54.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.529+0000 7fe66fd70740 10 client.25901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:54.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.529+0000 7fe66fd70740 20 client.25901 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:54.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.529+0000 7fe66fd70740 1 --2- 172.21.15.120:0/682894619 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558a66339140 0x558a6633b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.529+0000 7fe66fd70740 4 client.25901 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:05:54.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.529+0000 7fe66fd70740 1 -- 172.21.15.120:0/682894619 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a66291850 con 0x558a66339140 2024-09-19T09:05:54.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.529+0000 7fe6677fe640 1 --2- 172.21.15.120:0/682894619 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558a66339140 0x558a6633b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.529+0000 7fe6677fe640 1 --2- 172.21.15.120:0/682894619 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558a66339140 0x558a6633b560 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5add73b0963867f8 server_cookie=666e7ef826eb4d00 in_seq=0 out_seq=0 2024-09-19T09:05:54.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.529+0000 7fe646ffd640 10 client.25901 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:54.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.530+0000 7fe646ffd640 1 -- 172.21.15.120:0/682894619 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a66291850 con 0x558a66339140 2024-09-19T09:05:54.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.530+0000 7fe646ffd640 10 client.25901 handle_command_reply: tid=0 2024-09-19T09:05:54.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.530+0000 7fe66fd70740 1 client.25901 shutdown 2024-09-19T09:05:54.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.530+0000 7fe66fd70740 1 -- 172.21.15.120:0/682894619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a6631edc0 msgr2=0x558a66326280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:54.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.530+0000 7fe66fd70740 1 --2- 172.21.15.120:0/682894619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a6631edc0 0x558a66326280 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe6680fee90 tx=0x7fe65000f790 comp rx=0 tx=0).stop 2024-09-19T09:05:54.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.530+0000 7fe66fd70740 1 -- 172.21.15.120:0/682894619 shutdown_connections 2024-09-19T09:05:54.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.530+0000 7fe66fd70740 1 --2- 172.21.15.120:0/682894619 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558a66339140 0x558a6633b560 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.530+0000 7fe66fd70740 1 --2- 172.21.15.120:0/682894619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a66329680 0x558a663267e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.530+0000 7fe66fd70740 1 --2- 172.21.15.120:0/682894619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a6631edc0 0x558a66326280 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.530+0000 7fe646ffd640 0 client.25901 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:54.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.530+0000 7fe66fd70740 1 --2- 172.21.15.120:0/682894619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a6631e4a0 0x558a66328fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.530+0000 7fe66fd70740 1 -- 172.21.15.120:0/682894619 >> 172.21.15.120:0/682894619 conn(0x558a6631aa30 msgr2=0x558a66320760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:54.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.530+0000 7fe66fd70740 1 -- 172.21.15.120:0/682894619 shutdown_connections 2024-09-19T09:05:54.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.530+0000 7fe66fd70740 1 -- 172.21.15.120:0/682894619 wait complete. 2024-09-19T09:05:54.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.531+0000 7fe66fd70740 20 client.25901 trim_cache size 0 max 16384 2024-09-19T09:05:54.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.549+0000 7fe66fd70740 1 Processor -- start 2024-09-19T09:05:54.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.549+0000 7fe66fd70740 1 -- start start 2024-09-19T09:05:54.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.549+0000 7fe66fd70740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a6631e4a0 0x558a663354b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.549+0000 7fe66fd70740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a6631edc0 0x558a663217f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.549+0000 7fe66fd70740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a66329680 0x558a66340e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.549+0000 7fe66fd70740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe6681ad570 con 0x558a6631e4a0 2024-09-19T09:05:54.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.549+0000 7fe66fd70740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe6681af0b0 con 0x558a6631edc0 2024-09-19T09:05:54.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.549+0000 7fe66fd70740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6681adc40 con 0x558a66329680 2024-09-19T09:05:54.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.549+0000 7fe6677fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a6631e4a0 0x558a663354b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.549+0000 7fe6677fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a6631e4a0 0x558a663354b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55832/0 (socket says 172.21.15.120:55832) 2024-09-19T09:05:54.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.549+0000 7fe6677fe640 1 -- 172.21.15.120:0/1809021284 learned_addr learned my addr 172.21.15.120:0/1809021284 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:54.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.549+0000 7fe6677fe640 1 --2- 172.21.15.120:0/1809021284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a66329680 0x558a66340e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.549+0000 7fe65e7fc640 1 --2- 172.21.15.120:0/1809021284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a6631edc0 0x558a663217f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.549+0000 7fe6677fe640 1 -- 172.21.15.120:0/1809021284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a66329680 msgr2=0x558a66340e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:54.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.549+0000 7fe6677fe640 1 --2- 172.21.15.120:0/1809021284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a66329680 0x558a66340e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.549+0000 7fe6677fe640 1 -- 172.21.15.120:0/1809021284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a6631edc0 msgr2=0x558a663217f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:54.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.549+0000 7fe6677fe640 1 --2- 172.21.15.120:0/1809021284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a6631edc0 0x558a663217f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.550+0000 7fe6677fe640 1 -- 172.21.15.120:0/1809021284 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a6631b6c0 con 0x558a6631e4a0 2024-09-19T09:05:54.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.550+0000 7fe6677fe640 1 --2- 172.21.15.120:0/1809021284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a6631e4a0 0x558a663354b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe660009ca0 tx=0x7fe66002a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa020d4b517f5ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:54.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.550+0000 7fe65d7fa640 1 -- 172.21.15.120:0/1809021284 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe660048040 con 0x558a6631e4a0 2024-09-19T09:05:54.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.550+0000 7fe65d7fa640 1 -- 172.21.15.120:0/1809021284 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe660049040 con 0x558a6631e4a0 2024-09-19T09:05:54.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.550+0000 7fe65d7fa640 1 -- 172.21.15.120:0/1809021284 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6600045d0 con 0x558a6631e4a0 2024-09-19T09:05:54.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.551+0000 7fe66fd70740 1 -- 172.21.15.120:0/1809021284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a6631e4a0 msgr2=0x558a663354b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:54.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.551+0000 7fe66fd70740 1 --2- 172.21.15.120:0/1809021284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a6631e4a0 0x558a663354b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe660009ca0 tx=0x7fe66002a950 comp rx=0 tx=0).stop 2024-09-19T09:05:54.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.551+0000 7fe66fd70740 1 -- 172.21.15.120:0/1809021284 shutdown_connections 2024-09-19T09:05:54.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.551+0000 7fe66fd70740 1 --2- 172.21.15.120:0/1809021284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a66329680 0x558a66340e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.551+0000 7fe66fd70740 1 --2- 172.21.15.120:0/1809021284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a6631edc0 0x558a663217f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.551+0000 7fe66fd70740 1 --2- 172.21.15.120:0/1809021284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a6631e4a0 0x558a663354b0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.551+0000 7fe66fd70740 1 -- 172.21.15.120:0/1809021284 >> 172.21.15.120:0/1809021284 conn(0x558a6631aa30 msgr2=0x558a663210e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:54.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.551+0000 7fe66fd70740 1 -- 172.21.15.120:0/1809021284 shutdown_connections 2024-09-19T09:05:54.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.551+0000 7fe66fd70740 1 -- 172.21.15.120:0/1809021284 wait complete. 2024-09-19T09:05:54.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.551+0000 7fe66fd70740 1 -- start start 2024-09-19T09:05:54.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.551+0000 7fe66fd70740 1 Processor -- start 2024-09-19T09:05:54.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.552+0000 7fe66fd70740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a6631e4a0 0x558a66331e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.552+0000 7fe66fd70740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a6631edc0 0x558a6632d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.552+0000 7fe66fd70740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a66329680 0x558a6632d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.552+0000 7fe66fd70740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe6681ae9e0 con 0x558a6631edc0 2024-09-19T09:05:54.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.552+0000 7fe66fd70740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe6681ae310 con 0x558a66329680 2024-09-19T09:05:54.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.552+0000 7fe66fd70740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6681ab530 con 0x558a6631e4a0 2024-09-19T09:05:54.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.552+0000 7fe66fd70740 10 client.0 _ll_register_callbacks cb 0x7fe650009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:54.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.552+0000 7fe65e7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a6631edc0 0x558a6632d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.552+0000 7fe6677fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a6631e4a0 0x558a66331e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.552+0000 7fe65e7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a6631edc0 0x558a6632d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55834/0 (socket says 172.21.15.120:55834) 2024-09-19T09:05:54.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.552+0000 7fe65e7fc640 1 -- 172.21.15.120:0/1784474580 learned_addr learned my addr 172.21.15.120:0/1784474580 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:54.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.552+0000 7fe6677fe640 1 --2- 172.21.15.120:0/1784474580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a66329680 0x558a6632d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.552+0000 7fe65e7fc640 1 -- 172.21.15.120:0/1784474580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a6631e4a0 msgr2=0x558a66331e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:54.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.552+0000 7fe65e7fc640 1 --2- 172.21.15.120:0/1784474580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a6631e4a0 0x558a66331e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.552+0000 7fe65e7fc640 1 -- 172.21.15.120:0/1784474580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a66329680 msgr2=0x558a6632d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:54.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.552+0000 7fe65e7fc640 1 --2- 172.21.15.120:0/1784474580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a66329680 0x558a6632d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.552+0000 7fe65e7fc640 1 -- 172.21.15.120:0/1784474580 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a6632dfd0 con 0x558a6631edc0 2024-09-19T09:05:54.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.552+0000 7fe6677fe640 1 --2- 172.21.15.120:0/1784474580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a6631e4a0 0x558a66331e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:54.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.552+0000 7fe65e7fc640 1 --2- 172.21.15.120:0/1784474580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a6631edc0 0x558a6632d200 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe650000c00 tx=0x7fe650012110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166a67476d9bc9d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:54.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.552+0000 7fe646ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:54.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.552+0000 7fe646ffd640 1 -- 172.21.15.120:0/1784474580 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6500106c0 con 0x558a6631edc0 2024-09-19T09:05:54.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.552+0000 7fe646ffd640 1 -- 172.21.15.120:0/1784474580 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe650005110 con 0x558a6631edc0 2024-09-19T09:05:54.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.552+0000 7fe66fd70740 1 -- 172.21.15.120:0/1784474580 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe660049040 con 0x558a6631edc0 2024-09-19T09:05:54.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.553+0000 7fe646ffd640 1 -- 172.21.15.120:0/1784474580 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe650005350 con 0x558a6631edc0 2024-09-19T09:05:54.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.553+0000 7fe66fd70740 10 client.16545 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:54.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.553+0000 7fe66fd70740 1 -- 172.21.15.120:0/1784474580 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558a663a3ef0 con 0x558a6631edc0 2024-09-19T09:05:54.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.553+0000 7fe646ffd640 1 -- 172.21.15.120:0/1784474580 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe660049040 con 0x558a6631edc0 2024-09-19T09:05:54.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.553+0000 7fe646ffd640 1 -- 172.21.15.120:0/1784474580 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe650027460 con 0x558a6631edc0 2024-09-19T09:05:54.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.553+0000 7fe66fd70740 10 client.16545 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:54.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.553+0000 7fe66fd70740 10 client.16545 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:05:54.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.553+0000 7fe66fd70740 20 client.16545 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:54.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.553+0000 7fe66fd70740 1 --2- 172.21.15.120:0/1784474580 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558a663a46a0 0x558a663a6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:54.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.553+0000 7fe66fd70740 4 client.16545 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:05:54.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.553+0000 7fe66fd70740 1 -- 172.21.15.120:0/1784474580 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558a6628bd60 con 0x558a663a46a0 2024-09-19T09:05:54.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.553+0000 7fe6677fe640 1 --2- 172.21.15.120:0/1784474580 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558a663a46a0 0x558a663a6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:54.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.554+0000 7fe646ffd640 10 client.16545 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:54.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.554+0000 7fe6677fe640 1 --2- 172.21.15.120:0/1784474580 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558a663a46a0 0x558a663a6ac0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed8ba8b321b5e0f server_cookie=53714bdc1892ddc8 in_seq=0 out_seq=0 2024-09-19T09:05:54.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.554+0000 7fe646ffd640 1 -- 172.21.15.120:0/1784474580 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558a6628bd60 con 0x558a663a46a0 2024-09-19T09:05:54.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.554+0000 7fe646ffd640 10 client.16545 handle_command_reply: tid=0 2024-09-19T09:05:54.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.554+0000 7fe66fd70740 1 client.16545 shutdown 2024-09-19T09:05:54.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.554+0000 7fe66fd70740 1 -- 172.21.15.120:0/1784474580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a6631edc0 msgr2=0x558a6632d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:54.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.554+0000 7fe66fd70740 1 --2- 172.21.15.120:0/1784474580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a6631edc0 0x558a6632d200 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe650000c00 tx=0x7fe650012110 comp rx=0 tx=0).stop 2024-09-19T09:05:54.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.554+0000 7fe66fd70740 1 -- 172.21.15.120:0/1784474580 shutdown_connections 2024-09-19T09:05:54.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.554+0000 7fe66fd70740 1 --2- 172.21.15.120:0/1784474580 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558a663a46a0 0x558a663a6ac0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.554+0000 7fe66fd70740 1 --2- 172.21.15.120:0/1784474580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a66329680 0x558a6632d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.554+0000 7fe66fd70740 1 --2- 172.21.15.120:0/1784474580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a6631edc0 0x558a6632d200 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.554+0000 7fe66fd70740 1 --2- 172.21.15.120:0/1784474580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a6631e4a0 0x558a66331e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.554+0000 7fe66fd70740 1 -- 172.21.15.120:0/1784474580 >> 172.21.15.120:0/1784474580 conn(0x558a6631aa30 msgr2=0x558a66328300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:54.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.554+0000 7fe646ffd640 0 client.16545 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:05:54.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.555+0000 7fe66fd70740 1 -- 172.21.15.120:0/1784474580 shutdown_connections 2024-09-19T09:05:54.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.555+0000 7fe66fd70740 1 -- 172.21.15.120:0/1784474580 wait complete. 2024-09-19T09:05:54.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.555+0000 7fe66fd70740 20 client.16545 trim_cache size 0 max 16384 2024-09-19T09:05:54.566 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:05:54.566 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:05:54.567 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:05:54.567 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:05:54.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.555+0000 7fe66d997640 1 -- 172.21.15.120:0/1436629996 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe6540a3240 msgr2=0x7fe6540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:54.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.555+0000 7fe66d997640 1 --2- 172.21.15.120:0/1436629996 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe6540a3240 0x7fe6540a5700 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe658098d40 tx=0x7fe6580d3720 comp rx=0 tx=0).stop 2024-09-19T09:05:54.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.555+0000 7fe66d997640 1 -- 172.21.15.120:0/1436629996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe668100e20 msgr2=0x7fe6681984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:54.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.555+0000 7fe66d997640 1 --2- 172.21.15.120:0/1436629996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe668100e20 0x7fe6681984c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe65800e9f0 tx=0x7fe65800eeb0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.555+0000 7fe66d997640 1 -- 172.21.15.120:0/1436629996 shutdown_connections 2024-09-19T09:05:54.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.555+0000 7fe66d997640 1 --2- 172.21.15.120:0/1436629996 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe6540a3240 0x7fe6540a5700 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.555+0000 7fe66d997640 1 --2- 172.21.15.120:0/1436629996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe668102ef0 0x7fe66819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.555+0000 7fe66d997640 1 --2- 172.21.15.120:0/1436629996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe668102570 0x7fe668198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.555+0000 7fe66d997640 1 --2- 172.21.15.120:0/1436629996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe668100e20 0x7fe6681984c0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:54.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.555+0000 7fe66d997640 1 -- 172.21.15.120:0/1436629996 >> 172.21.15.120:0/1436629996 conn(0x7fe6680fc7d0 msgr2=0x7fe66806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:54.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.555+0000 7fe66d997640 1 -- 172.21.15.120:0/1436629996 shutdown_connections 2024-09-19T09:05:54.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:54.555+0000 7fe66d997640 1 -- 172.21.15.120:0/1436629996 wait complete. 2024-09-19T09:05:55.040 INFO:tasks.workunit.client.0.smithi120.stdout: 1 199258 9.93 MB/sec warmup 110 sec latency 51.741 ms 2024-09-19T09:05:55.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:54 smithi120 ceph-mon[31150]: pgmap v432: 97 pgs: 97 active+clean; 50 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.8 MiB/s wr, 472 op/s 2024-09-19T09:05:55.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:54 smithi196 ceph-mon[35150]: pgmap v432: 97 pgs: 97 active+clean; 50 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.8 MiB/s wr, 472 op/s 2024-09-19T09:05:55.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:54 smithi153 ceph-mon[35071]: pgmap v432: 97 pgs: 97 active+clean; 50 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.8 MiB/s wr, 472 op/s 2024-09-19T09:05:55.416 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:05:55.417 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:05:55.674 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:05:56.040 INFO:tasks.workunit.client.0.smithi120.stdout: 1 202447 10.01 MB/sec warmup 111 sec latency 57.188 ms 2024-09-19T09:05:56.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.442+0000 7f412ae38640 1 -- 172.21.15.120:0/2227965229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41241036e0 msgr2=0x7f4124107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:56.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.442+0000 7f412ae38640 1 --2- 172.21.15.120:0/2227965229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41241036e0 0x7f4124107b90 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f4110009930 tx=0x7f411002a420 comp rx=0 tx=0).stop 2024-09-19T09:05:56.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.442+0000 7f412ae38640 1 -- 172.21.15.120:0/2227965229 shutdown_connections 2024-09-19T09:05:56.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.442+0000 7f412ae38640 1 --2- 172.21.15.120:0/2227965229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41241082a0 0x7f412410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:56.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.442+0000 7f412ae38640 1 --2- 172.21.15.120:0/2227965229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41241036e0 0x7f4124107b90 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:56.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.442+0000 7f412ae38640 1 --2- 172.21.15.120:0/2227965229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4124102d30 0x7f4124103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:56.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.442+0000 7f412ae38640 1 -- 172.21.15.120:0/2227965229 >> 172.21.15.120:0/2227965229 conn(0x7f41240fc7d0 msgr2=0x7f41240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:56.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.442+0000 7f412ae38640 1 -- 172.21.15.120:0/2227965229 shutdown_connections 2024-09-19T09:05:56.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.442+0000 7f412ae38640 1 -- 172.21.15.120:0/2227965229 wait complete. 2024-09-19T09:05:56.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.443+0000 7f412ae38640 1 Processor -- start 2024-09-19T09:05:56.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.443+0000 7f412ae38640 1 -- start start 2024-09-19T09:05:56.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.443+0000 7f412ae38640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4124102d30 0x7f41241a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:56.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.443+0000 7f412ae38640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41241036e0 0x7f41241a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:56.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.443+0000 7f412ae38640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41241082a0 0x7f412419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:56.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.443+0000 7f412ae38640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f41241120a0 con 0x7f41241036e0 2024-09-19T09:05:56.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.443+0000 7f412ae38640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4124111f20 con 0x7f4124102d30 2024-09-19T09:05:56.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.443+0000 7f412ae38640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4124112220 con 0x7f41241082a0 2024-09-19T09:05:56.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.443+0000 7f4128bd4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4124102d30 0x7f41241a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:56.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.443+0000 7f4123fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41241036e0 0x7f41241a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:56.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.443+0000 7f4128bd4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4124102d30 0x7f41241a0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:54934/0 (socket says 172.21.15.120:54934) 2024-09-19T09:05:56.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.443+0000 7f4128bd4640 1 -- 172.21.15.120:0/2928729949 learned_addr learned my addr 172.21.15.120:0/2928729949 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:56.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.443+0000 7f41293d5640 1 --2- 172.21.15.120:0/2928729949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41241082a0 0x7f412419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:56.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.443+0000 7f41293d5640 1 -- 172.21.15.120:0/2928729949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4124102d30 msgr2=0x7f41241a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:56.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.444+0000 7f41293d5640 1 --2- 172.21.15.120:0/2928729949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4124102d30 0x7f41241a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:56.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.444+0000 7f41293d5640 1 -- 172.21.15.120:0/2928729949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41241036e0 msgr2=0x7f41241a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:56.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.444+0000 7f41293d5640 1 --2- 172.21.15.120:0/2928729949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41241036e0 0x7f41241a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:56.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.444+0000 7f41293d5640 1 -- 172.21.15.120:0/2928729949 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f412419ad40 con 0x7f41241082a0 2024-09-19T09:05:56.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.444+0000 7f4128bd4640 1 --2- 172.21.15.120:0/2928729949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4124102d30 0x7f41241a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:56.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.444+0000 7f4123fff640 1 --2- 172.21.15.120:0/2928729949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41241036e0 0x7f41241a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:56.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.444+0000 7f41293d5640 1 --2- 172.21.15.120:0/2928729949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41241082a0 0x7f412419a610 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f411c00e940 tx=0x7f411c00ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:56.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.444+0000 7f4121ffb640 1 -- 172.21.15.120:0/2928729949 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c004d20 con 0x7f41241082a0 2024-09-19T09:05:56.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.444+0000 7f412ae38640 1 -- 172.21.15.120:0/2928729949 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f412419b030 con 0x7f41241082a0 2024-09-19T09:05:56.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.444+0000 7f4121ffb640 1 -- 172.21.15.120:0/2928729949 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f411c005020 con 0x7f41241082a0 2024-09-19T09:05:56.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.444+0000 7f4121ffb640 1 -- 172.21.15.120:0/2928729949 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c0116b0 con 0x7f41241082a0 2024-09-19T09:05:56.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.444+0000 7f412ae38640 1 -- 172.21.15.120:0/2928729949 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41241a7320 con 0x7f41241082a0 2024-09-19T09:05:56.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.445+0000 7f412ae38640 1 -- 172.21.15.120:0/2928729949 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4124103110 con 0x7f41241082a0 2024-09-19T09:05:56.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.446+0000 7f4121ffb640 1 -- 172.21.15.120:0/2928729949 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f411c0054d0 con 0x7f41241082a0 2024-09-19T09:05:56.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.447+0000 7f4121ffb640 1 --2- 172.21.15.120:0/2928729949 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f41140a3240 0x7f41140a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:56.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.447+0000 7f4128bd4640 1 --2- 172.21.15.120:0/2928729949 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f41140a3240 0x7f41140a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:56.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.447+0000 7f4121ffb640 1 -- 172.21.15.120:0/2928729949 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f411c0cf920 con 0x7f41241082a0 2024-09-19T09:05:56.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.447+0000 7f4128bd4640 1 --2- 172.21.15.120:0/2928729949 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f41140a3240 0x7f41140a5700 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4118011500 tx=0x7f4118005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:56.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.452+0000 7f4121ffb640 1 -- 172.21.15.120:0/2928729949 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f411c016030 con 0x7f41241082a0 2024-09-19T09:05:56.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.662+0000 7f412ae38640 1 -- 172.21.15.120:0/2928729949 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f412410c2c0 con 0x7f41241082a0 2024-09-19T09:05:56.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.663+0000 7f4121ffb640 1 -- 172.21.15.120:0/2928729949 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f411c09bff0 con 0x7f41241082a0 2024-09-19T09:05:56.665 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:05:56.665 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":28,"btime":"2024-09-19T09:02:41:696728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:02:41.207357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:05:56.665 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 28 2024-09-19T09:05:56.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.665+0000 7f412ae38640 1 -- 172.21.15.120:0/2928729949 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f41140a3240 msgr2=0x7f41140a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:56.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.665+0000 7f412ae38640 1 --2- 172.21.15.120:0/2928729949 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f41140a3240 0x7f41140a5700 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4118011500 tx=0x7f4118005f90 comp rx=0 tx=0).stop 2024-09-19T09:05:56.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.665+0000 7f412ae38640 1 -- 172.21.15.120:0/2928729949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41241082a0 msgr2=0x7f412419a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:56.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.665+0000 7f412ae38640 1 --2- 172.21.15.120:0/2928729949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41241082a0 0x7f412419a610 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f411c00e940 tx=0x7f411c00ee00 comp rx=0 tx=0).stop 2024-09-19T09:05:56.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.665+0000 7f412ae38640 1 -- 172.21.15.120:0/2928729949 shutdown_connections 2024-09-19T09:05:56.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.665+0000 7f412ae38640 1 --2- 172.21.15.120:0/2928729949 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f41140a3240 0x7f41140a5700 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:56.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.665+0000 7f412ae38640 1 --2- 172.21.15.120:0/2928729949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41241082a0 0x7f412419a610 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:56.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.665+0000 7f412ae38640 1 --2- 172.21.15.120:0/2928729949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41241036e0 0x7f41241a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:56.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.665+0000 7f412ae38640 1 --2- 172.21.15.120:0/2928729949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4124102d30 0x7f41241a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:56.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.665+0000 7f412ae38640 1 -- 172.21.15.120:0/2928729949 >> 172.21.15.120:0/2928729949 conn(0x7f41240fc7d0 msgr2=0x7f4124100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:56.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.665+0000 7f412ae38640 1 -- 172.21.15.120:0/2928729949 shutdown_connections 2024-09-19T09:05:56.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:56.665+0000 7f412ae38640 1 -- 172.21.15.120:0/2928729949 wait complete. 2024-09-19T09:05:57.040 INFO:tasks.workunit.client.0.smithi120.stdout: 1 205688 10.07 MB/sec warmup 112 sec latency 10.672 ms 2024-09-19T09:05:57.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:56 smithi120 ceph-mon[31150]: pgmap v433: 97 pgs: 97 active+clean; 49 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.8 MiB/s wr, 363 op/s 2024-09-19T09:05:57.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:56 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2928729949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:05:57.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:56 smithi196 ceph-mon[35150]: pgmap v433: 97 pgs: 97 active+clean; 49 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.8 MiB/s wr, 363 op/s 2024-09-19T09:05:57.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:56 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2928729949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:05:57.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:56 smithi153 ceph-mon[35071]: pgmap v433: 97 pgs: 97 active+clean; 49 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.8 MiB/s wr, 363 op/s 2024-09-19T09:05:57.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:56 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2928729949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:05:57.502 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:05:57.764 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:05:58.040 INFO:tasks.workunit.client.0.smithi120.stdout: 1 208319 10.07 MB/sec warmup 113 sec latency 15.758 ms 2024-09-19T09:05:58.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.272+0000 7fa3840d2640 1 -- 172.21.15.120:0/3239122428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa37c100e20 msgr2=0x7fa37c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:58.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.272+0000 7fa3840d2640 1 --2- 172.21.15.120:0/3239122428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa37c100e20 0x7fa37c101200 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa378009900 tx=0x7fa37802a480 comp rx=0 tx=0).stop 2024-09-19T09:05:58.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.272+0000 7fa3840d2640 1 -- 172.21.15.120:0/3239122428 shutdown_connections 2024-09-19T09:05:58.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.272+0000 7fa3840d2640 1 --2- 172.21.15.120:0/3239122428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa37c102ef0 0x7fa37c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.272+0000 7fa3840d2640 1 --2- 172.21.15.120:0/3239122428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa37c102570 0x7fa37c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.272+0000 7fa3840d2640 1 --2- 172.21.15.120:0/3239122428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa37c100e20 0x7fa37c101200 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.272+0000 7fa3840d2640 1 -- 172.21.15.120:0/3239122428 >> 172.21.15.120:0/3239122428 conn(0x7fa37c0fc7d0 msgr2=0x7fa37c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:58.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.272+0000 7fa3840d2640 1 -- 172.21.15.120:0/3239122428 shutdown_connections 2024-09-19T09:05:58.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.272+0000 7fa3840d2640 1 -- 172.21.15.120:0/3239122428 wait complete. 2024-09-19T09:05:58.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.273+0000 7fa3840d2640 1 Processor -- start 2024-09-19T09:05:58.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.273+0000 7fa3840d2640 1 -- start start 2024-09-19T09:05:58.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.273+0000 7fa3840d2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa37c100e20 0x7fa37c197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:58.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.273+0000 7fa3840d2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa37c102570 0x7fa37c197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:58.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.273+0000 7fa3840d2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa37c102ef0 0x7fa37c19bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:58.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.273+0000 7fa3840d2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c110f50 con 0x7fa37c102570 2024-09-19T09:05:58.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.273+0000 7fa3840d2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c110dd0 con 0x7fa37c100e20 2024-09-19T09:05:58.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.273+0000 7fa3840d2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c1110d0 con 0x7fa37c102ef0 2024-09-19T09:05:58.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.273+0000 7fa38166d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa37c102570 0x7fa37c197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:58.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.273+0000 7fa38166d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa37c102570 0x7fa37c197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55862/0 (socket says 172.21.15.120:55862) 2024-09-19T09:05:58.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.273+0000 7fa38166d640 1 -- 172.21.15.120:0/2753170886 learned_addr learned my addr 172.21.15.120:0/2753170886 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:58.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.273+0000 7fa381e6e640 1 --2- 172.21.15.120:0/2753170886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa37c100e20 0x7fa37c197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:58.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.273+0000 7fa38266f640 1 --2- 172.21.15.120:0/2753170886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa37c102ef0 0x7fa37c19bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:58.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.273+0000 7fa38166d640 1 -- 172.21.15.120:0/2753170886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa37c102ef0 msgr2=0x7fa37c19bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:58.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.273+0000 7fa38166d640 1 --2- 172.21.15.120:0/2753170886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa37c102ef0 0x7fa37c19bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.273+0000 7fa38166d640 1 -- 172.21.15.120:0/2753170886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa37c100e20 msgr2=0x7fa37c197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:58.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.273+0000 7fa38166d640 1 --2- 172.21.15.120:0/2753170886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa37c100e20 0x7fa37c197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.273+0000 7fa38166d640 1 -- 172.21.15.120:0/2753170886 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37c19c400 con 0x7fa37c102570 2024-09-19T09:05:58.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.273+0000 7fa38266f640 1 --2- 172.21.15.120:0/2753170886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa37c102ef0 0x7fa37c19bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:05:58.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.274+0000 7fa38166d640 1 --2- 172.21.15.120:0/2753170886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa37c102570 0x7fa37c197990 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa36c00fb60 tx=0x7fa36c009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:58.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.274+0000 7fa372ffd640 1 -- 172.21.15.120:0/2753170886 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c00fc50 con 0x7fa37c102570 2024-09-19T09:05:58.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.274+0000 7fa372ffd640 1 -- 172.21.15.120:0/2753170886 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa36c00fdf0 con 0x7fa37c102570 2024-09-19T09:05:58.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.274+0000 7fa3840d2640 1 -- 172.21.15.120:0/2753170886 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa37c19c6f0 con 0x7fa37c102570 2024-09-19T09:05:58.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.274+0000 7fa372ffd640 1 -- 172.21.15.120:0/2753170886 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c004b60 con 0x7fa37c102570 2024-09-19T09:05:58.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.274+0000 7fa3840d2640 1 -- 172.21.15.120:0/2753170886 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa37c069f70 con 0x7fa37c102570 2024-09-19T09:05:58.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.276+0000 7fa372ffd640 1 -- 172.21.15.120:0/2753170886 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa36c004d00 con 0x7fa37c102570 2024-09-19T09:05:58.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.276+0000 7fa3864ab740 1 Processor -- start 2024-09-19T09:05:58.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.277+0000 7fa3864ab740 1 -- start start 2024-09-19T09:05:58.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.277+0000 7fa3864ab740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1f9833b40 0x55b1f9832090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:58.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.277+0000 7fa3864ab740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1f98325d0 0x55b1f9832a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:58.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.277+0000 7fa3864ab740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1f983d370 0x55b1f983f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:58.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.277+0000 7fa3864ab740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b1f97982f0 con 0x55b1f983d370 2024-09-19T09:05:58.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.277+0000 7fa3864ab740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b1f973fcf0 con 0x55b1f9833b40 2024-09-19T09:05:58.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.277+0000 7fa3864ab740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b1f97942b0 con 0x55b1f98325d0 2024-09-19T09:05:58.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.277+0000 7fa372ffd640 1 --2- 172.21.15.120:0/2753170886 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa3680a3240 0x7fa3680a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:58.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.277+0000 7fa381e6e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1f98325d0 0x55b1f9832a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:58.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.277+0000 7fa381e6e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1f98325d0 0x55b1f9832a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:54484/0 (socket says 172.21.15.120:54484) 2024-09-19T09:05:58.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.277+0000 7fa381e6e640 1 -- 172.21.15.120:0/3513821057 learned_addr learned my addr 172.21.15.120:0/3513821057 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:58.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.277+0000 7fa38166d640 1 --2- 172.21.15.120:0/3513821057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1f983d370 0x55b1f983f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:58.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.277+0000 7fa38266f640 1 --2- 172.21.15.120:0/2753170886 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa3680a3240 0x7fa3680a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:58.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.277+0000 7fa372ffd640 1 -- 172.21.15.120:0/2753170886 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa36c0d12e0 con 0x7fa37c102570 2024-09-19T09:05:58.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.277+0000 7fa38266f640 1 --2- 172.21.15.120:0/3513821057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1f9833b40 0x55b1f9832090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:58.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.277+0000 7fa381e6e640 1 -- 172.21.15.120:0/3513821057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1f9833b40 msgr2=0x55b1f9832090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:58.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.277+0000 7fa381e6e640 1 --2- 172.21.15.120:0/3513821057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1f9833b40 0x55b1f9832090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.277+0000 7fa381e6e640 1 -- 172.21.15.120:0/3513821057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1f983d370 msgr2=0x55b1f983f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:58.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.277+0000 7fa381e6e640 1 --2- 172.21.15.120:0/3513821057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1f983d370 0x55b1f983f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.278+0000 7fa381e6e640 1 -- 172.21.15.120:0/3513821057 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1f97243a0 con 0x55b1f98325d0 2024-09-19T09:05:58.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.278+0000 7fa38266f640 1 --2- 172.21.15.120:0/2753170886 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa3680a3240 0x7fa3680a5700 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa374009270 tx=0x7fa3740091e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:58.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.278+0000 7fa381e6e640 1 --2- 172.21.15.120:0/3513821057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1f98325d0 0x55b1f9832a10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa37802a9a0 tx=0x7fa378005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f91b34beedf8311 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:58.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.278+0000 7fa367fff640 1 -- 172.21.15.120:0/3513821057 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa378035030 con 0x55b1f98325d0 2024-09-19T09:05:58.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.278+0000 7fa367fff640 1 -- 172.21.15.120:0/3513821057 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa378040070 con 0x55b1f98325d0 2024-09-19T09:05:58.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.278+0000 7fa367fff640 1 -- 172.21.15.120:0/3513821057 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa378031930 con 0x55b1f98325d0 2024-09-19T09:05:58.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.279+0000 7fa3864ab740 1 -- 172.21.15.120:0/3513821057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1f98325d0 msgr2=0x55b1f9832a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:58.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.279+0000 7fa3864ab740 1 --2- 172.21.15.120:0/3513821057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1f98325d0 0x55b1f9832a10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa37802a9a0 tx=0x7fa378005e80 comp rx=0 tx=0).stop 2024-09-19T09:05:58.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.279+0000 7fa3864ab740 1 -- 172.21.15.120:0/3513821057 shutdown_connections 2024-09-19T09:05:58.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.279+0000 7fa3864ab740 1 --2- 172.21.15.120:0/3513821057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1f983d370 0x55b1f983f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.279+0000 7fa3864ab740 1 --2- 172.21.15.120:0/3513821057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1f98325d0 0x55b1f9832a10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.279+0000 7fa3864ab740 1 --2- 172.21.15.120:0/3513821057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1f9833b40 0x55b1f9832090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.279+0000 7fa3864ab740 1 -- 172.21.15.120:0/3513821057 >> 172.21.15.120:0/3513821057 conn(0x55b1f97fb310 msgr2=0x55b1f982ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:58.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.279+0000 7fa3864ab740 1 -- 172.21.15.120:0/3513821057 shutdown_connections 2024-09-19T09:05:58.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.279+0000 7fa3864ab740 1 -- 172.21.15.120:0/3513821057 wait complete. 2024-09-19T09:05:58.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.279+0000 7fa3864ab740 1 -- start start 2024-09-19T09:05:58.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.279+0000 7fa3864ab740 1 Processor -- start 2024-09-19T09:05:58.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.280+0000 7fa3864ab740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1f98325d0 0x55b1f983cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:58.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.280+0000 7fa3864ab740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1f9833b40 0x55b1f983a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:58.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.280+0000 7fa3864ab740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1f983d370 0x55b1f983a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:58.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.280+0000 7fa3864ab740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b1f9840010 con 0x55b1f98325d0 2024-09-19T09:05:58.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.280+0000 7fa381e6e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1f98325d0 0x55b1f983cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:58.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.280+0000 7fa3864ab740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b1f9753060 con 0x55b1f9833b40 2024-09-19T09:05:58.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.280+0000 7fa3864ab740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b1f9840190 con 0x55b1f983d370 2024-09-19T09:05:58.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.280+0000 7fa381e6e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1f98325d0 0x55b1f983cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55876/0 (socket says 172.21.15.120:55876) 2024-09-19T09:05:58.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.280+0000 7fa381e6e640 1 -- 172.21.15.120:0/3004257836 learned_addr learned my addr 172.21.15.120:0/3004257836 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:58.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.280+0000 7fa38166d640 1 --2- 172.21.15.120:0/3004257836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1f9833b40 0x55b1f983a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:58.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.280+0000 7fa38266f640 1 --2- 172.21.15.120:0/3004257836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1f983d370 0x55b1f983a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:58.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.280+0000 7fa3864ab740 10 client.0 _ll_register_callbacks cb 0x55b1f94a68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:58.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.280+0000 7fa381e6e640 1 -- 172.21.15.120:0/3004257836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1f983d370 msgr2=0x55b1f983a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:58.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.280+0000 7fa381e6e640 1 --2- 172.21.15.120:0/3004257836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1f983d370 0x55b1f983a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.280+0000 7fa381e6e640 1 -- 172.21.15.120:0/3004257836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1f9833b40 msgr2=0x55b1f983a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:58.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.280+0000 7fa381e6e640 1 --2- 172.21.15.120:0/3004257836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1f9833b40 0x55b1f983a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.280+0000 7fa381e6e640 1 -- 172.21.15.120:0/3004257836 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1f95093c0 con 0x55b1f98325d0 2024-09-19T09:05:58.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.280+0000 7fa38166d640 1 --2- 172.21.15.120:0/3004257836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1f9833b40 0x55b1f983a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:58.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.280+0000 7fa381e6e640 1 --2- 172.21.15.120:0/3004257836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1f98325d0 0x55b1f983cd70 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa378000c00 tx=0x7fa3780044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb05379e5b0e487 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:58.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.280+0000 7fa3657fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:58.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.281+0000 7fa3657fa640 1 -- 172.21.15.120:0/3004257836 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa378035070 con 0x55b1f98325d0 2024-09-19T09:05:58.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.281+0000 7fa3657fa640 1 -- 172.21.15.120:0/3004257836 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa3780318c0 con 0x55b1f98325d0 2024-09-19T09:05:58.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.281+0000 7fa3864ab740 1 -- 172.21.15.120:0/3004257836 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa378040070 con 0x55b1f98325d0 2024-09-19T09:05:58.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.281+0000 7fa3657fa640 1 -- 172.21.15.120:0/3004257836 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa378031ac0 con 0x55b1f98325d0 2024-09-19T09:05:58.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.281+0000 7fa3864ab740 10 client.16563 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:58.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.281+0000 7fa3864ab740 1 -- 172.21.15.120:0/3004257836 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b1f984ba90 con 0x55b1f98325d0 2024-09-19T09:05:58.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.281+0000 7fa3657fa640 1 -- 172.21.15.120:0/3004257836 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa378040070 con 0x55b1f98325d0 2024-09-19T09:05:58.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.281+0000 7fa3657fa640 1 -- 172.21.15.120:0/3004257836 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa378052020 con 0x55b1f98325d0 2024-09-19T09:05:58.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.281+0000 7fa3864ab740 10 client.16563 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:58.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.281+0000 7fa3864ab740 10 client.16563 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:05:58.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.281+0000 7fa3864ab740 20 client.16563 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:58.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.281+0000 7fa3864ab740 1 --2- 172.21.15.120:0/3004257836 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55b1f984cde0 0x55b1f984f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:58.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.281+0000 7fa3864ab740 4 client.16563 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:05:58.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.281+0000 7fa3864ab740 1 -- 172.21.15.120:0/3004257836 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1f97a5860 con 0x55b1f984cde0 2024-09-19T09:05:58.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.281+0000 7fa38166d640 1 --2- 172.21.15.120:0/3004257836 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55b1f984cde0 0x55b1f984f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:58.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.282+0000 7fa38166d640 1 --2- 172.21.15.120:0/3004257836 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55b1f984cde0 0x55b1f984f1d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4236a8491beb4a9e server_cookie=5cdb42920a735a0c in_seq=0 out_seq=0 2024-09-19T09:05:58.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.282+0000 7fa3657fa640 10 client.16563 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:05:58.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.282+0000 7fa3657fa640 1 -- 172.21.15.120:0/3004257836 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b1f97a5860 con 0x55b1f984cde0 2024-09-19T09:05:58.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.282+0000 7fa3657fa640 10 client.16563 handle_command_reply: tid=0 2024-09-19T09:05:58.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.282+0000 7fa3864ab740 1 client.16563 shutdown 2024-09-19T09:05:58.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.283+0000 7fa3864ab740 1 -- 172.21.15.120:0/3004257836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1f98325d0 msgr2=0x55b1f983cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:58.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.283+0000 7fa3864ab740 1 --2- 172.21.15.120:0/3004257836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1f98325d0 0x55b1f983cd70 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa378000c00 tx=0x7fa3780044b0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.283+0000 7fa3864ab740 1 -- 172.21.15.120:0/3004257836 shutdown_connections 2024-09-19T09:05:58.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.283+0000 7fa3864ab740 1 --2- 172.21.15.120:0/3004257836 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55b1f984cde0 0x55b1f984f1d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.283+0000 7fa3864ab740 1 --2- 172.21.15.120:0/3004257836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1f983d370 0x55b1f983a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.283+0000 7fa3864ab740 1 --2- 172.21.15.120:0/3004257836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1f9833b40 0x55b1f983a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.283+0000 7fa3657fa640 0 client.16563 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:05:58.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.283+0000 7fa3864ab740 1 --2- 172.21.15.120:0/3004257836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1f98325d0 0x55b1f983cd70 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.283+0000 7fa3864ab740 1 -- 172.21.15.120:0/3004257836 >> 172.21.15.120:0/3004257836 conn(0x55b1f97fb310 msgr2=0x55b1f9834960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:58.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.283+0000 7fa3864ab740 1 -- 172.21.15.120:0/3004257836 shutdown_connections 2024-09-19T09:05:58.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.283+0000 7fa3864ab740 1 -- 172.21.15.120:0/3004257836 wait complete. 2024-09-19T09:05:58.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.283+0000 7fa3864ab740 20 client.16563 trim_cache size 0 max 16384 2024-09-19T09:05:58.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.296+0000 7fa3864ab740 1 Processor -- start 2024-09-19T09:05:58.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.296+0000 7fa3864ab740 1 -- start start 2024-09-19T09:05:58.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.297+0000 7fa3864ab740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1f98325d0 0x55b1f9833410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:58.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.297+0000 7fa3864ab740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1f9833b40 0x55b1f98521f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:58.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.297+0000 7fa3864ab740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1f983d370 0x55b1f9852730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:58.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.297+0000 7fa3864ab740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c1ab400 con 0x55b1f9833b40 2024-09-19T09:05:58.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.297+0000 7fa3864ab740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c1acf40 con 0x55b1f98325d0 2024-09-19T09:05:58.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.297+0000 7fa3864ab740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c1abad0 con 0x55b1f983d370 2024-09-19T09:05:58.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.297+0000 7fa38166d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1f9833b40 0x55b1f98521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:58.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.297+0000 7fa38266f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1f983d370 0x55b1f9852730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:58.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.297+0000 7fa38166d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1f9833b40 0x55b1f98521f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55880/0 (socket says 172.21.15.120:55880) 2024-09-19T09:05:58.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.297+0000 7fa38166d640 1 -- 172.21.15.120:0/3684661337 learned_addr learned my addr 172.21.15.120:0/3684661337 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:58.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.297+0000 7fa381e6e640 1 --2- 172.21.15.120:0/3684661337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1f98325d0 0x55b1f9833410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:58.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.297+0000 7fa38166d640 1 -- 172.21.15.120:0/3684661337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1f983d370 msgr2=0x55b1f9852730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:58.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.297+0000 7fa38166d640 1 --2- 172.21.15.120:0/3684661337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1f983d370 0x55b1f9852730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.297+0000 7fa38166d640 1 -- 172.21.15.120:0/3684661337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1f98325d0 msgr2=0x55b1f9833410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:58.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.297+0000 7fa38166d640 1 --2- 172.21.15.120:0/3684661337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1f98325d0 0x55b1f9833410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.297+0000 7fa38166d640 1 -- 172.21.15.120:0/3684661337 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1f982e4f0 con 0x55b1f9833b40 2024-09-19T09:05:58.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.297+0000 7fa38266f640 1 --2- 172.21.15.120:0/3684661337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1f983d370 0x55b1f9852730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:05:58.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.297+0000 7fa38166d640 1 --2- 172.21.15.120:0/3684661337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1f9833b40 0x55b1f98521f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa36c09d850 tx=0x7fa36c070740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c84fe7d04abbd015 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:58.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.297+0000 7fa3677fe640 1 -- 172.21.15.120:0/3684661337 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c01f040 con 0x55b1f9833b40 2024-09-19T09:05:58.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.298+0000 7fa3677fe640 1 -- 172.21.15.120:0/3684661337 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa36c09dfc0 con 0x55b1f9833b40 2024-09-19T09:05:58.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.298+0000 7fa3677fe640 1 -- 172.21.15.120:0/3684661337 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c0051b0 con 0x55b1f9833b40 2024-09-19T09:05:58.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.298+0000 7fa3864ab740 1 -- 172.21.15.120:0/3684661337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1f9833b40 msgr2=0x55b1f98521f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:58.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.298+0000 7fa3864ab740 1 --2- 172.21.15.120:0/3684661337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1f9833b40 0x55b1f98521f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa36c09d850 tx=0x7fa36c070740 comp rx=0 tx=0).stop 2024-09-19T09:05:58.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.298+0000 7fa3864ab740 1 -- 172.21.15.120:0/3684661337 shutdown_connections 2024-09-19T09:05:58.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.298+0000 7fa3864ab740 1 --2- 172.21.15.120:0/3684661337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1f983d370 0x55b1f9852730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.298+0000 7fa3864ab740 1 --2- 172.21.15.120:0/3684661337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1f9833b40 0x55b1f98521f0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.298+0000 7fa3864ab740 1 --2- 172.21.15.120:0/3684661337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1f98325d0 0x55b1f9833410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.298+0000 7fa3864ab740 1 -- 172.21.15.120:0/3684661337 >> 172.21.15.120:0/3684661337 conn(0x55b1f97fb310 msgr2=0x55b1f98388f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:58.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.298+0000 7fa3864ab740 1 -- 172.21.15.120:0/3684661337 shutdown_connections 2024-09-19T09:05:58.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.298+0000 7fa3864ab740 1 -- 172.21.15.120:0/3684661337 wait complete. 2024-09-19T09:05:58.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.299+0000 7fa3864ab740 1 -- start start 2024-09-19T09:05:58.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.299+0000 7fa3864ab740 1 Processor -- start 2024-09-19T09:05:58.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.299+0000 7fa3864ab740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1f98325d0 0x55b1f984f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:58.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.299+0000 7fa3864ab740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1f9833b40 0x55b1f984f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:58.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.299+0000 7fa3864ab740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1f983d370 0x55b1f98b5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:58.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.299+0000 7fa3864ab740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c1ac870 con 0x55b1f983d370 2024-09-19T09:05:58.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.299+0000 7fa3864ab740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c1ac1a0 con 0x55b1f9833b40 2024-09-19T09:05:58.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.299+0000 7fa3864ab740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c1a93c0 con 0x55b1f98325d0 2024-09-19T09:05:58.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.299+0000 7fa381e6e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1f98325d0 0x55b1f984f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:58.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.299+0000 7fa3864ab740 10 client.0 _ll_register_callbacks cb 0x7fa36c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:05:58.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.299+0000 7fa38266f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1f983d370 0x55b1f98b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:58.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.299+0000 7fa381e6e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1f98325d0 0x55b1f984f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:54502/0 (socket says 172.21.15.120:54502) 2024-09-19T09:05:58.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.299+0000 7fa381e6e640 1 -- 172.21.15.120:0/3981818088 learned_addr learned my addr 172.21.15.120:0/3981818088 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:05:58.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.299+0000 7fa38166d640 1 --2- 172.21.15.120:0/3981818088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1f9833b40 0x55b1f984f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:58.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.300+0000 7fa38266f640 1 -- 172.21.15.120:0/3981818088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1f98325d0 msgr2=0x55b1f984f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:58.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.300+0000 7fa38266f640 1 --2- 172.21.15.120:0/3981818088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1f98325d0 0x55b1f984f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.300+0000 7fa38266f640 1 -- 172.21.15.120:0/3981818088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1f9833b40 msgr2=0x55b1f984f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:05:58.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.300+0000 7fa38266f640 1 --2- 172.21.15.120:0/3981818088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1f9833b40 0x55b1f984f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.300+0000 7fa38266f640 1 -- 172.21.15.120:0/3981818088 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1f98b5c50 con 0x55b1f983d370 2024-09-19T09:05:58.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.300+0000 7fa38266f640 1 --2- 172.21.15.120:0/3981818088 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1f983d370 0x55b1f98b5710 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x55b1f983d210 tx=0x7fa374004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92bcf1b89a51aae4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:05:58.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.300+0000 7fa3657fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:05:58.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.300+0000 7fa3657fa640 1 -- 172.21.15.120:0/3981818088 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37402e020 con 0x55b1f983d370 2024-09-19T09:05:58.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.300+0000 7fa3657fa640 1 -- 172.21.15.120:0/3981818088 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa37401a070 con 0x55b1f983d370 2024-09-19T09:05:58.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.300+0000 7fa3864ab740 1 -- 172.21.15.120:0/3981818088 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa36c09dfc0 con 0x55b1f983d370 2024-09-19T09:05:58.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.300+0000 7fa3657fa640 1 -- 172.21.15.120:0/3981818088 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa374029650 con 0x55b1f983d370 2024-09-19T09:05:58.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.300+0000 7fa3864ab740 10 client.16569 fetch_fsmap learned FSMap version 28 2024-09-19T09:05:58.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.301+0000 7fa3864ab740 1 -- 172.21.15.120:0/3981818088 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b1f98b6e70 con 0x55b1f983d370 2024-09-19T09:05:58.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.301+0000 7fa3657fa640 1 -- 172.21.15.120:0/3981818088 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa36c09dfc0 con 0x55b1f983d370 2024-09-19T09:05:58.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.301+0000 7fa3657fa640 1 -- 172.21.15.120:0/3981818088 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa374033970 con 0x55b1f983d370 2024-09-19T09:05:58.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.301+0000 7fa3864ab740 10 client.16569 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:05:58.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.301+0000 7fa3864ab740 10 client.16569 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:05:58.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.301+0000 7fa3864ab740 20 client.16569 populate_metadata read hostname 'smithi120' 2024-09-19T09:05:58.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.301+0000 7fa3864ab740 1 --2- 172.21.15.120:0/3981818088 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55b1f98b7590 0x55b1f98b99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:05:58.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.301+0000 7fa3864ab740 4 client.16569 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:05:58.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.301+0000 7fa3864ab740 1 -- 172.21.15.120:0/3981818088 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b1f979fd70 con 0x55b1f98b7590 2024-09-19T09:05:58.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.301+0000 7fa381e6e640 1 --2- 172.21.15.120:0/3981818088 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55b1f98b7590 0x55b1f98b99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:05:58.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.302+0000 7fa3657fa640 10 client.16569 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:05:58.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.302+0000 7fa381e6e640 1 --2- 172.21.15.120:0/3981818088 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55b1f98b7590 0x55b1f98b99b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8212282863ce6486 server_cookie=248c903f7e72d9a5 in_seq=0 out_seq=0 2024-09-19T09:05:58.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.302+0000 7fa3657fa640 1 -- 172.21.15.120:0/3981818088 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b1f979fd70 con 0x55b1f98b7590 2024-09-19T09:05:58.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.302+0000 7fa3657fa640 10 client.16569 handle_command_reply: tid=0 2024-09-19T09:05:58.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.302+0000 7fa3864ab740 1 client.16569 shutdown 2024-09-19T09:05:58.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.302+0000 7fa3864ab740 1 -- 172.21.15.120:0/3981818088 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1f983d370 msgr2=0x55b1f98b5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:58.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.302+0000 7fa3864ab740 1 --2- 172.21.15.120:0/3981818088 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1f983d370 0x55b1f98b5710 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x55b1f983d210 tx=0x7fa374004a40 comp rx=0 tx=0).stop 2024-09-19T09:05:58.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.302+0000 7fa3864ab740 1 -- 172.21.15.120:0/3981818088 shutdown_connections 2024-09-19T09:05:58.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.302+0000 7fa3864ab740 1 --2- 172.21.15.120:0/3981818088 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55b1f98b7590 0x55b1f98b99b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.302+0000 7fa3864ab740 1 --2- 172.21.15.120:0/3981818088 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1f983d370 0x55b1f98b5710 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.302+0000 7fa3864ab740 1 --2- 172.21.15.120:0/3981818088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1f9833b40 0x55b1f984f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.302+0000 7fa3864ab740 1 --2- 172.21.15.120:0/3981818088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1f98325d0 0x55b1f984f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.302+0000 7fa3864ab740 1 -- 172.21.15.120:0/3981818088 >> 172.21.15.120:0/3981818088 conn(0x55b1f97fb310 msgr2=0x55b1f982dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:58.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.302+0000 7fa3657fa640 0 client.16569 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:05:58.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.302+0000 7fa3864ab740 1 -- 172.21.15.120:0/3981818088 shutdown_connections 2024-09-19T09:05:58.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.302+0000 7fa3864ab740 1 -- 172.21.15.120:0/3981818088 wait complete. 2024-09-19T09:05:58.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.302+0000 7fa3864ab740 20 client.16569 trim_cache size 0 max 16384 2024-09-19T09:05:58.310 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:05:58.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.303+0000 7fa3840d2640 1 -- 172.21.15.120:0/2753170886 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa3680a3240 msgr2=0x7fa3680a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:58.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.303+0000 7fa3840d2640 1 --2- 172.21.15.120:0/2753170886 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa3680a3240 0x7fa3680a5700 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa374009270 tx=0x7fa3740091e0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.303+0000 7fa3840d2640 1 -- 172.21.15.120:0/2753170886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa37c102570 msgr2=0x7fa37c197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:05:58.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.303+0000 7fa3840d2640 1 --2- 172.21.15.120:0/2753170886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa37c102570 0x7fa37c197990 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa36c00fb60 tx=0x7fa36c009610 comp rx=0 tx=0).stop 2024-09-19T09:05:58.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.303+0000 7fa3840d2640 1 -- 172.21.15.120:0/2753170886 shutdown_connections 2024-09-19T09:05:58.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.303+0000 7fa3840d2640 1 --2- 172.21.15.120:0/2753170886 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa3680a3240 0x7fa3680a5700 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.303+0000 7fa3840d2640 1 --2- 172.21.15.120:0/2753170886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa37c102ef0 0x7fa37c19bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.303+0000 7fa3840d2640 1 --2- 172.21.15.120:0/2753170886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa37c102570 0x7fa37c197990 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.303+0000 7fa3840d2640 1 --2- 172.21.15.120:0/2753170886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa37c100e20 0x7fa37c197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:05:58.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.303+0000 7fa3840d2640 1 -- 172.21.15.120:0/2753170886 >> 172.21.15.120:0/2753170886 conn(0x7fa37c0fc7d0 msgr2=0x7fa37c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:05:58.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.303+0000 7fa3840d2640 1 -- 172.21.15.120:0/2753170886 shutdown_connections 2024-09-19T09:05:58.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:05:58.303+0000 7fa3840d2640 1 -- 172.21.15.120:0/2753170886 wait complete. 2024-09-19T09:05:59.040 INFO:tasks.workunit.client.0.smithi120.stdout: 1 210448 10.10 MB/sec warmup 114 sec latency 106.882 ms 2024-09-19T09:05:59.045 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:05:58 smithi120 ceph-mon[31150]: pgmap v434: 97 pgs: 97 active+clean; 49 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.0 MiB/s wr, 388 op/s 2024-09-19T09:05:59.074 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:05:59.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:05:58 smithi196 ceph-mon[35150]: pgmap v434: 97 pgs: 97 active+clean; 49 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.0 MiB/s wr, 388 op/s 2024-09-19T09:05:59.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:05:58 smithi153 ceph-mon[35071]: pgmap v434: 97 pgs: 97 active+clean; 49 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.0 MiB/s wr, 388 op/s 2024-09-19T09:05:59.334 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:06:00.040 INFO:tasks.workunit.client.0.smithi120.stdout: 1 213265 10.16 MB/sec warmup 115 sec latency 54.567 ms 2024-09-19T09:06:00.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.453+0000 7f12849ed640 1 -- 172.21.15.120:0/252867104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1280102d40 msgr2=0x7f1280103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:00.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.453+0000 7f12849ed640 1 --2- 172.21.15.120:0/252867104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1280102d40 0x7f1280103120 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f1270009900 tx=0x7f127002a480 comp rx=0 tx=0).stop 2024-09-19T09:06:00.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.453+0000 7f12849ed640 1 -- 172.21.15.120:0/252867104 shutdown_connections 2024-09-19T09:06:00.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.453+0000 7f12849ed640 1 --2- 172.21.15.120:0/252867104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f128010c170 0x7f128010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.453+0000 7f12849ed640 1 --2- 172.21.15.120:0/252867104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1280103660 0x7f128010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.453+0000 7f12849ed640 1 --2- 172.21.15.120:0/252867104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1280102d40 0x7f1280103120 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.453+0000 7f12849ed640 1 -- 172.21.15.120:0/252867104 >> 172.21.15.120:0/252867104 conn(0x7f12800fc7a0 msgr2=0x7f12800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:00.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.453+0000 7f12849ed640 1 -- 172.21.15.120:0/252867104 shutdown_connections 2024-09-19T09:06:00.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.453+0000 7f12849ed640 1 -- 172.21.15.120:0/252867104 wait complete. 2024-09-19T09:06:00.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.454+0000 7f12849ed640 1 Processor -- start 2024-09-19T09:06:00.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.454+0000 7f12849ed640 1 -- start start 2024-09-19T09:06:00.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.454+0000 7f12849ed640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1280102d40 0x7f1280198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:00.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.454+0000 7f12849ed640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1280103660 0x7f1280198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:00.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.454+0000 7f12849ed640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f128010c170 0x7f128019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:00.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.454+0000 7f12849ed640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1280111f70 con 0x7f128010c170 2024-09-19T09:06:00.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.454+0000 7f12849ed640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1280111df0 con 0x7f1280102d40 2024-09-19T09:06:00.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.454+0000 7f12849ed640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f12801120f0 con 0x7f1280103660 2024-09-19T09:06:00.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.454+0000 7f127ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f128010c170 0x7f128019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:00.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.454+0000 7f1277fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1280103660 0x7f1280198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:00.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.454+0000 7f127e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1280102d40 0x7f1280198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:00.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.454+0000 7f127ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f128010c170 0x7f128019cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55900/0 (socket says 172.21.15.120:55900) 2024-09-19T09:06:00.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.454+0000 7f127ed9d640 1 -- 172.21.15.120:0/3892138807 learned_addr learned my addr 172.21.15.120:0/3892138807 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:00.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.455+0000 7f127ed9d640 1 -- 172.21.15.120:0/3892138807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1280103660 msgr2=0x7f1280198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:00.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.455+0000 7f127ed9d640 1 --2- 172.21.15.120:0/3892138807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1280103660 0x7f1280198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.455+0000 7f127ed9d640 1 -- 172.21.15.120:0/3892138807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1280102d40 msgr2=0x7f1280198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:00.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.455+0000 7f127ed9d640 1 --2- 172.21.15.120:0/3892138807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1280102d40 0x7f1280198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.455+0000 7f127ed9d640 1 -- 172.21.15.120:0/3892138807 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f128019d3c0 con 0x7f128010c170 2024-09-19T09:06:00.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.455+0000 7f127e59c640 1 --2- 172.21.15.120:0/3892138807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1280102d40 0x7f1280198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:00.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.455+0000 7f127ed9d640 1 --2- 172.21.15.120:0/3892138807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f128010c170 0x7f128019cce0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f127800eaa0 tx=0x7f127800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:00.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.455+0000 7f12777fe640 1 -- 172.21.15.120:0/3892138807 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127800cdb0 con 0x7f128010c170 2024-09-19T09:06:00.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.455+0000 7f12777fe640 1 -- 172.21.15.120:0/3892138807 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1278005020 con 0x7f128010c170 2024-09-19T09:06:00.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.455+0000 7f12849ed640 1 -- 172.21.15.120:0/3892138807 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f128019d6b0 con 0x7f128010c170 2024-09-19T09:06:00.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.455+0000 7f12777fe640 1 -- 172.21.15.120:0/3892138807 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12780116b0 con 0x7f128010c170 2024-09-19T09:06:00.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.455+0000 7f12849ed640 1 -- 172.21.15.120:0/3892138807 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12801a4f90 con 0x7f128010c170 2024-09-19T09:06:00.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.457+0000 7f12777fe640 1 -- 172.21.15.120:0/3892138807 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f1278005560 con 0x7f128010c170 2024-09-19T09:06:00.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.458+0000 7f1286dc6740 1 Processor -- start 2024-09-19T09:06:00.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.458+0000 7f1286dc6740 1 -- start start 2024-09-19T09:06:00.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.458+0000 7f12777fe640 1 --2- 172.21.15.120:0/3892138807 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f126c0a32c0 0x7f126c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:00.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.458+0000 7f1286dc6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc66a5b40 0x55abc66a4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:00.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.458+0000 7f1286dc6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc66a45d0 0x55abc66a4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:00.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.458+0000 7f1286dc6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc66af370 0x55abc66b1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:00.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.458+0000 7f1286dc6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55abc660a2f0 con 0x55abc66af370 2024-09-19T09:06:00.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.458+0000 7f1286dc6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55abc65b1cf0 con 0x55abc66a5b40 2024-09-19T09:06:00.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.458+0000 7f1286dc6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55abc66062b0 con 0x55abc66a45d0 2024-09-19T09:06:00.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.459+0000 7f12777fe640 1 -- 172.21.15.120:0/3892138807 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f12780cf990 con 0x7f128010c170 2024-09-19T09:06:00.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.459+0000 7f127e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc66a5b40 0x55abc66a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:00.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.459+0000 7f127ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc66af370 0x55abc66b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:00.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.459+0000 7f127e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc66a5b40 0x55abc66a4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:55004/0 (socket says 172.21.15.120:55004) 2024-09-19T09:06:00.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.459+0000 7f127e59c640 1 -- 172.21.15.120:0/4077612500 learned_addr learned my addr 172.21.15.120:0/4077612500 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:00.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.459+0000 7f1277fff640 1 --2- 172.21.15.120:0/3892138807 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f126c0a32c0 0x7f126c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:00.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.459+0000 7f1277fff640 1 --2- 172.21.15.120:0/4077612500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc66a45d0 0x55abc66a4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:00.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.459+0000 7f127ed9d640 1 -- 172.21.15.120:0/4077612500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc66a45d0 msgr2=0x55abc66a4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:00.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.459+0000 7f127ed9d640 1 --2- 172.21.15.120:0/4077612500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc66a45d0 0x55abc66a4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.459+0000 7f127ed9d640 1 -- 172.21.15.120:0/4077612500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc66a5b40 msgr2=0x55abc66a4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:00.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.459+0000 7f127ed9d640 1 --2- 172.21.15.120:0/4077612500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc66a5b40 0x55abc66a4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.459+0000 7f127ed9d640 1 -- 172.21.15.120:0/4077612500 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abc65963a0 con 0x55abc66af370 2024-09-19T09:06:00.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.459+0000 7f127e59c640 1 --2- 172.21.15.120:0/4077612500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc66a5b40 0x55abc66a4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:00.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.459+0000 7f1277fff640 1 --2- 172.21.15.120:0/3892138807 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f126c0a32c0 0x7f126c0a5780 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f128010d5b0 tx=0x7f126800b830 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:00.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.459+0000 7f127ed9d640 1 --2- 172.21.15.120:0/4077612500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc66af370 0x55abc66b1760 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f127800efd0 tx=0x7f1278079670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=465bd651d538f6d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:00.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.459+0000 7f1274ff9640 1 -- 172.21.15.120:0/4077612500 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127801f030 con 0x55abc66af370 2024-09-19T09:06:00.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.459+0000 7f1274ff9640 1 -- 172.21.15.120:0/4077612500 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1278079080 con 0x55abc66af370 2024-09-19T09:06:00.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.459+0000 7f1274ff9640 1 -- 172.21.15.120:0/4077612500 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12780cd5a0 con 0x55abc66af370 2024-09-19T09:06:00.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.460+0000 7f1286dc6740 1 -- 172.21.15.120:0/4077612500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc66af370 msgr2=0x55abc66b1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:00.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.460+0000 7f1286dc6740 1 --2- 172.21.15.120:0/4077612500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc66af370 0x55abc66b1760 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f127800efd0 tx=0x7f1278079670 comp rx=0 tx=0).stop 2024-09-19T09:06:00.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.460+0000 7f1286dc6740 1 -- 172.21.15.120:0/4077612500 shutdown_connections 2024-09-19T09:06:00.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.460+0000 7f1286dc6740 1 --2- 172.21.15.120:0/4077612500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc66af370 0x55abc66b1760 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.460+0000 7f1286dc6740 1 --2- 172.21.15.120:0/4077612500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc66a45d0 0x55abc66a4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.460+0000 7f1286dc6740 1 --2- 172.21.15.120:0/4077612500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc66a5b40 0x55abc66a4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.460+0000 7f1286dc6740 1 -- 172.21.15.120:0/4077612500 >> 172.21.15.120:0/4077612500 conn(0x55abc666d310 msgr2=0x55abc66a0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:00.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.460+0000 7f1286dc6740 1 -- 172.21.15.120:0/4077612500 shutdown_connections 2024-09-19T09:06:00.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.460+0000 7f1286dc6740 1 -- 172.21.15.120:0/4077612500 wait complete. 2024-09-19T09:06:00.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.461+0000 7f1286dc6740 1 -- start start 2024-09-19T09:06:00.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.461+0000 7f1286dc6740 1 Processor -- start 2024-09-19T09:06:00.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.461+0000 7f1286dc6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc66a45d0 0x55abc66aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:00.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.461+0000 7f1286dc6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc66a5b40 0x55abc66ac080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:00.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.461+0000 7f1286dc6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc66af370 0x55abc66ac810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:00.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.461+0000 7f1286dc6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55abc66b2010 con 0x55abc66a5b40 2024-09-19T09:06:00.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.461+0000 7f1286dc6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55abc65c5060 con 0x55abc66a45d0 2024-09-19T09:06:00.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.461+0000 7f1286dc6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55abc66b2190 con 0x55abc66af370 2024-09-19T09:06:00.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.461+0000 7f1286dc6740 10 client.0 _ll_register_callbacks cb 0x55abc63188c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:00.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.461+0000 7f1277fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc66a5b40 0x55abc66ac080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:00.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.461+0000 7f127e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc66a45d0 0x55abc66aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:00.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.461+0000 7f1277fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc66a5b40 0x55abc66ac080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55924/0 (socket says 172.21.15.120:55924) 2024-09-19T09:06:00.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.461+0000 7f1277fff640 1 -- 172.21.15.120:0/3778088964 learned_addr learned my addr 172.21.15.120:0/3778088964 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:00.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.461+0000 7f127ed9d640 1 --2- 172.21.15.120:0/3778088964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc66af370 0x55abc66ac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:00.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.461+0000 7f1277fff640 1 -- 172.21.15.120:0/3778088964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc66af370 msgr2=0x55abc66ac810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:00.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.461+0000 7f1277fff640 1 --2- 172.21.15.120:0/3778088964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc66af370 0x55abc66ac810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.461+0000 7f1277fff640 1 -- 172.21.15.120:0/3778088964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc66a45d0 msgr2=0x55abc66aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:00.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.461+0000 7f1277fff640 1 --2- 172.21.15.120:0/3778088964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc66a45d0 0x55abc66aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.461+0000 7f1277fff640 1 -- 172.21.15.120:0/3778088964 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abc637b3c0 con 0x55abc66a5b40 2024-09-19T09:06:00.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.462+0000 7f1277fff640 1 --2- 172.21.15.120:0/3778088964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc66a5b40 0x55abc66ac080 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f126800ebd0 tx=0x7f1268004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f13c57639c4304a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:00.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.462+0000 7f1265ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:06:00.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.462+0000 7f1265ffb640 1 -- 172.21.15.120:0/3778088964 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126801e7b0 con 0x55abc66a5b40 2024-09-19T09:06:00.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.462+0000 7f1265ffb640 1 -- 172.21.15.120:0/3778088964 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f126801e950 con 0x55abc66a5b40 2024-09-19T09:06:00.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.462+0000 7f1286dc6740 1 -- 172.21.15.120:0/3778088964 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1278079080 con 0x55abc66a5b40 2024-09-19T09:06:00.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.462+0000 7f1265ffb640 1 -- 172.21.15.120:0/3778088964 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f126801eb50 con 0x55abc66a5b40 2024-09-19T09:06:00.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.462+0000 7f1286dc6740 10 client.16581 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:00.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.462+0000 7f1286dc6740 1 -- 172.21.15.120:0/3778088964 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55abc66bdb30 con 0x55abc66a5b40 2024-09-19T09:06:00.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.462+0000 7f1265ffb640 1 -- 172.21.15.120:0/3778088964 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1278079080 con 0x55abc66a5b40 2024-09-19T09:06:00.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.462+0000 7f1265ffb640 1 -- 172.21.15.120:0/3778088964 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f126801f070 con 0x55abc66a5b40 2024-09-19T09:06:00.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.462+0000 7f1286dc6740 10 client.16581 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:00.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.462+0000 7f1286dc6740 10 client.16581 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:06:00.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.462+0000 7f1286dc6740 20 client.16581 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:00.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.462+0000 7f1286dc6740 1 --2- 172.21.15.120:0/3778088964 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55abc66beff0 0x55abc66c13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:00.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.462+0000 7f1286dc6740 4 client.16581 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:06:00.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.462+0000 7f1286dc6740 1 -- 172.21.15.120:0/3778088964 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abc6617860 con 0x55abc66beff0 2024-09-19T09:06:00.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.463+0000 7f127e59c640 1 --2- 172.21.15.120:0/3778088964 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55abc66beff0 0x55abc66c13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:00.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.463+0000 7f127e59c640 1 --2- 172.21.15.120:0/3778088964 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55abc66beff0 0x55abc66c13e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=517cfc94036218f7 server_cookie=7a89e4f1fc05c9e4 in_seq=0 out_seq=0 2024-09-19T09:06:00.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.463+0000 7f1265ffb640 10 client.16581 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:06:00.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.464+0000 7f1265ffb640 1 -- 172.21.15.120:0/3778088964 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55abc6617860 con 0x55abc66beff0 2024-09-19T09:06:00.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.464+0000 7f1265ffb640 10 client.16581 handle_command_reply: tid=0 2024-09-19T09:06:00.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.464+0000 7f1286dc6740 1 client.16581 shutdown 2024-09-19T09:06:00.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.464+0000 7f1286dc6740 1 -- 172.21.15.120:0/3778088964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc66a5b40 msgr2=0x55abc66ac080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:00.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.464+0000 7f1286dc6740 1 --2- 172.21.15.120:0/3778088964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc66a5b40 0x55abc66ac080 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f126800ebd0 tx=0x7f1268004730 comp rx=0 tx=0).stop 2024-09-19T09:06:00.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.464+0000 7f1286dc6740 1 -- 172.21.15.120:0/3778088964 shutdown_connections 2024-09-19T09:06:00.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.464+0000 7f1286dc6740 1 --2- 172.21.15.120:0/3778088964 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55abc66beff0 0x55abc66c13e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.464+0000 7f1286dc6740 1 --2- 172.21.15.120:0/3778088964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc66af370 0x55abc66ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.464+0000 7f1265ffb640 0 client.16581 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:06:00.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.464+0000 7f1286dc6740 1 --2- 172.21.15.120:0/3778088964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc66a5b40 0x55abc66ac080 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.464+0000 7f1286dc6740 1 --2- 172.21.15.120:0/3778088964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc66a45d0 0x55abc66aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.464+0000 7f1286dc6740 1 -- 172.21.15.120:0/3778088964 >> 172.21.15.120:0/3778088964 conn(0x55abc666d310 msgr2=0x55abc66a6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:00.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.464+0000 7f1286dc6740 1 -- 172.21.15.120:0/3778088964 shutdown_connections 2024-09-19T09:06:00.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.464+0000 7f1286dc6740 1 -- 172.21.15.120:0/3778088964 wait complete. 2024-09-19T09:06:00.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.464+0000 7f1286dc6740 20 client.16581 trim_cache size 0 max 16384 2024-09-19T09:06:00.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.478+0000 7f1286dc6740 1 Processor -- start 2024-09-19T09:06:00.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.478+0000 7f1286dc6740 1 -- start start 2024-09-19T09:06:00.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.479+0000 7f1286dc6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc66a45d0 0x55abc66ab940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:00.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.479+0000 7f1286dc6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc66a5b40 0x55abc669fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:00.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.479+0000 7f1286dc6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc66af370 0x55abc66a7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:00.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.479+0000 7f1286dc6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f12801ad600 con 0x55abc66a5b40 2024-09-19T09:06:00.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.479+0000 7f1286dc6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f12801af140 con 0x55abc66a45d0 2024-09-19T09:06:00.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.479+0000 7f1286dc6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f12801adcd0 con 0x55abc66af370 2024-09-19T09:06:00.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.479+0000 7f1277fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc66a5b40 0x55abc669fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:00.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.479+0000 7f1277fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc66a5b40 0x55abc669fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55926/0 (socket says 172.21.15.120:55926) 2024-09-19T09:06:00.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.479+0000 7f127e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc66a45d0 0x55abc66ab940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:00.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.479+0000 7f1277fff640 1 -- 172.21.15.120:0/965042086 learned_addr learned my addr 172.21.15.120:0/965042086 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:00.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.479+0000 7f127ed9d640 1 --2- 172.21.15.120:0/965042086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc66af370 0x55abc66a7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:00.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.479+0000 7f1277fff640 1 -- 172.21.15.120:0/965042086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc66af370 msgr2=0x55abc66a7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:00.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.479+0000 7f1277fff640 1 --2- 172.21.15.120:0/965042086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc66af370 0x55abc66a7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.479+0000 7f1277fff640 1 -- 172.21.15.120:0/965042086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc66a45d0 msgr2=0x55abc66ab940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:00.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.479+0000 7f1277fff640 1 --2- 172.21.15.120:0/965042086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc66a45d0 0x55abc66ab940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.479+0000 7f1277fff640 1 -- 172.21.15.120:0/965042086 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abc66aa5d0 con 0x55abc66a5b40 2024-09-19T09:06:00.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.479+0000 7f127e59c640 1 --2- 172.21.15.120:0/965042086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc66a45d0 0x55abc66ab940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:00.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.479+0000 7f1277fff640 1 --2- 172.21.15.120:0/965042086 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc66a5b40 0x55abc669fbf0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f12680204d0 tx=0x7f12680046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c1b4c30b806b72 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:00.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.480+0000 7f1267fff640 1 -- 172.21.15.120:0/965042086 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126800bb80 con 0x55abc66a5b40 2024-09-19T09:06:00.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.480+0000 7f1267fff640 1 -- 172.21.15.120:0/965042086 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1268025070 con 0x55abc66a5b40 2024-09-19T09:06:00.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.480+0000 7f1267fff640 1 -- 172.21.15.120:0/965042086 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1268010b80 con 0x55abc66a5b40 2024-09-19T09:06:00.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.480+0000 7f1286dc6740 1 -- 172.21.15.120:0/965042086 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc66a5b40 msgr2=0x55abc669fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:00.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.480+0000 7f1286dc6740 1 --2- 172.21.15.120:0/965042086 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc66a5b40 0x55abc669fbf0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f12680204d0 tx=0x7f12680046a0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.480+0000 7f1286dc6740 1 -- 172.21.15.120:0/965042086 shutdown_connections 2024-09-19T09:06:00.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.480+0000 7f1286dc6740 1 --2- 172.21.15.120:0/965042086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc66af370 0x55abc66a7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.480+0000 7f1286dc6740 1 --2- 172.21.15.120:0/965042086 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc66a5b40 0x55abc669fbf0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.480+0000 7f1286dc6740 1 --2- 172.21.15.120:0/965042086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc66a45d0 0x55abc66ab940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.480+0000 7f1286dc6740 1 -- 172.21.15.120:0/965042086 >> 172.21.15.120:0/965042086 conn(0x55abc666d310 msgr2=0x55abc669f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:00.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.480+0000 7f1286dc6740 1 -- 172.21.15.120:0/965042086 shutdown_connections 2024-09-19T09:06:00.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.480+0000 7f1286dc6740 1 -- 172.21.15.120:0/965042086 wait complete. 2024-09-19T09:06:00.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.481+0000 7f1286dc6740 1 -- start start 2024-09-19T09:06:00.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.481+0000 7f1286dc6740 1 Processor -- start 2024-09-19T09:06:00.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.481+0000 7f1286dc6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc66a45d0 0x55abc66b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:00.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.481+0000 7f1286dc6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc66a5b40 0x55abc66b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:00.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.481+0000 7f1286dc6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc66af370 0x55abc66b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:00.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.481+0000 7f1286dc6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f12801aea70 con 0x55abc66af370 2024-09-19T09:06:00.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.481+0000 7f1286dc6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f12801ae3a0 con 0x55abc66a45d0 2024-09-19T09:06:00.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.481+0000 7f1286dc6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f12801ab5c0 con 0x55abc66a5b40 2024-09-19T09:06:00.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.481+0000 7f1286dc6740 10 client.0 _ll_register_callbacks cb 0x7f1268012ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:00.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.481+0000 7f127ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc66af370 0x55abc66b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:00.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.481+0000 7f127e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc66a45d0 0x55abc66b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:00.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.481+0000 7f1277fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc66a5b40 0x55abc66b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:00.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.481+0000 7f127e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc66a45d0 0x55abc66b7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:55014/0 (socket says 172.21.15.120:55014) 2024-09-19T09:06:00.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.481+0000 7f127e59c640 1 -- 172.21.15.120:0/1745015844 learned_addr learned my addr 172.21.15.120:0/1745015844 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:00.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.481+0000 7f127ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc66af370 0x55abc66b32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55932/0 (socket says 172.21.15.120:55932) 2024-09-19T09:06:00.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.482+0000 7f127ed9d640 1 -- 172.21.15.120:0/1745015844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc66a5b40 msgr2=0x55abc66b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:00.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.482+0000 7f127ed9d640 1 --2- 172.21.15.120:0/1745015844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc66a5b40 0x55abc66b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.482+0000 7f127ed9d640 1 -- 172.21.15.120:0/1745015844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc66a45d0 msgr2=0x55abc66b7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:00.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.482+0000 7f127ed9d640 1 --2- 172.21.15.120:0/1745015844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc66a45d0 0x55abc66b7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.482+0000 7f127ed9d640 1 -- 172.21.15.120:0/1745015844 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abc66b3b10 con 0x55abc66af370 2024-09-19T09:06:00.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.482+0000 7f127ed9d640 1 --2- 172.21.15.120:0/1745015844 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc66af370 0x55abc66b32e0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f12780a0520 tx=0x7f127809bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f841179611e66424 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:00.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.482+0000 7f1265ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:06:00.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.482+0000 7f1265ffb640 1 -- 172.21.15.120:0/1745015844 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127801f040 con 0x55abc66af370 2024-09-19T09:06:00.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.482+0000 7f1265ffb640 1 -- 172.21.15.120:0/1745015844 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f12780986d0 con 0x55abc66af370 2024-09-19T09:06:00.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.482+0000 7f1286dc6740 1 -- 172.21.15.120:0/1745015844 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1268025070 con 0x55abc66af370 2024-09-19T09:06:00.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.482+0000 7f1265ffb640 1 -- 172.21.15.120:0/1745015844 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1278079080 con 0x55abc66af370 2024-09-19T09:06:00.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.482+0000 7f1286dc6740 10 client.16587 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:00.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.482+0000 7f1286dc6740 1 -- 172.21.15.120:0/1745015844 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55abc6729bc0 con 0x55abc66af370 2024-09-19T09:06:00.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.482+0000 7f1265ffb640 1 -- 172.21.15.120:0/1745015844 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1268025070 con 0x55abc66af370 2024-09-19T09:06:00.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.482+0000 7f1265ffb640 1 -- 172.21.15.120:0/1745015844 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f12780b8070 con 0x55abc66af370 2024-09-19T09:06:00.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.482+0000 7f1286dc6740 10 client.16587 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:00.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.482+0000 7f1286dc6740 10 client.16587 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:06:00.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.482+0000 7f1286dc6740 20 client.16587 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:00.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.482+0000 7f1286dc6740 1 --2- 172.21.15.120:0/1745015844 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55abc6729e40 0x55abc672c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:00.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.483+0000 7f1286dc6740 4 client.16587 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:06:00.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.483+0000 7f1286dc6740 1 -- 172.21.15.120:0/1745015844 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55abc6611d70 con 0x55abc6729e40 2024-09-19T09:06:00.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.483+0000 7f127e59c640 1 --2- 172.21.15.120:0/1745015844 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55abc6729e40 0x55abc672c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:00.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.483+0000 7f1265ffb640 10 client.16587 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:06:00.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.483+0000 7f127e59c640 1 --2- 172.21.15.120:0/1745015844 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55abc6729e40 0x55abc672c230 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=23d270540a501c53 server_cookie=7c161b932e823aaa in_seq=0 out_seq=0 2024-09-19T09:06:00.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.483+0000 7f1265ffb640 1 -- 172.21.15.120:0/1745015844 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55abc6611d70 con 0x55abc6729e40 2024-09-19T09:06:00.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.483+0000 7f1265ffb640 10 client.16587 handle_command_reply: tid=0 2024-09-19T09:06:00.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.483+0000 7f1286dc6740 1 client.16587 shutdown 2024-09-19T09:06:00.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.483+0000 7f1286dc6740 1 -- 172.21.15.120:0/1745015844 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc66af370 msgr2=0x55abc66b32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:00.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.483+0000 7f1286dc6740 1 --2- 172.21.15.120:0/1745015844 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc66af370 0x55abc66b32e0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f12780a0520 tx=0x7f127809bcd0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.483+0000 7f1286dc6740 1 -- 172.21.15.120:0/1745015844 shutdown_connections 2024-09-19T09:06:00.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.483+0000 7f1286dc6740 1 --2- 172.21.15.120:0/1745015844 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55abc6729e40 0x55abc672c230 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.484+0000 7f1265ffb640 0 client.16587 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:06:00.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.484+0000 7f1286dc6740 1 --2- 172.21.15.120:0/1745015844 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc66af370 0x55abc66b32e0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.484+0000 7f1286dc6740 1 --2- 172.21.15.120:0/1745015844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc66a5b40 0x55abc66b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.484+0000 7f1286dc6740 1 --2- 172.21.15.120:0/1745015844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc66a45d0 0x55abc66b7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.484+0000 7f1286dc6740 1 -- 172.21.15.120:0/1745015844 >> 172.21.15.120:0/1745015844 conn(0x55abc666d310 msgr2=0x55abc669f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:00.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.484+0000 7f1286dc6740 1 -- 172.21.15.120:0/1745015844 shutdown_connections 2024-09-19T09:06:00.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.484+0000 7f1286dc6740 1 -- 172.21.15.120:0/1745015844 wait complete. 2024-09-19T09:06:00.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.484+0000 7f1286dc6740 20 client.16587 trim_cache size 0 max 16384 2024-09-19T09:06:00.502 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:06:00.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.484+0000 7f12849ed640 1 -- 172.21.15.120:0/3892138807 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f126c0a32c0 msgr2=0x7f126c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:00.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.484+0000 7f12849ed640 1 --2- 172.21.15.120:0/3892138807 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f126c0a32c0 0x7f126c0a5780 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f128010d5b0 tx=0x7f126800b830 comp rx=0 tx=0).stop 2024-09-19T09:06:00.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.484+0000 7f12849ed640 1 -- 172.21.15.120:0/3892138807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f128010c170 msgr2=0x7f128019cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:00.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.484+0000 7f12849ed640 1 --2- 172.21.15.120:0/3892138807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f128010c170 0x7f128019cce0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f127800eaa0 tx=0x7f127800ef60 comp rx=0 tx=0).stop 2024-09-19T09:06:00.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.484+0000 7f12849ed640 1 -- 172.21.15.120:0/3892138807 shutdown_connections 2024-09-19T09:06:00.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.484+0000 7f12849ed640 1 --2- 172.21.15.120:0/3892138807 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f126c0a32c0 0x7f126c0a5780 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.485+0000 7f12849ed640 1 --2- 172.21.15.120:0/3892138807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f128010c170 0x7f128019cce0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.485+0000 7f12849ed640 1 --2- 172.21.15.120:0/3892138807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1280103660 0x7f1280198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.485+0000 7f12849ed640 1 --2- 172.21.15.120:0/3892138807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1280102d40 0x7f1280198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:00.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.485+0000 7f12849ed640 1 -- 172.21.15.120:0/3892138807 >> 172.21.15.120:0/3892138807 conn(0x7f12800fc7a0 msgr2=0x7f12800fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:00.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.485+0000 7f12849ed640 1 -- 172.21.15.120:0/3892138807 shutdown_connections 2024-09-19T09:06:00.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:00.485+0000 7f12849ed640 1 -- 172.21.15.120:0/3892138807 wait complete. 2024-09-19T09:06:01.040 INFO:tasks.workunit.client.0.smithi120.stdout: 1 216525 10.23 MB/sec warmup 116 sec latency 21.126 ms 2024-09-19T09:06:01.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:00 smithi120 ceph-mon[31150]: pgmap v435: 97 pgs: 97 active+clean; 41 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.5 MiB/s wr, 501 op/s 2024-09-19T09:06:01.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:06:01.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:06:01.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:06:01.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:06:01.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:00 smithi196 ceph-mon[35150]: pgmap v435: 97 pgs: 97 active+clean; 41 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.5 MiB/s wr, 501 op/s 2024-09-19T09:06:01.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:06:01.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:06:01.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:06:01.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:06:01.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:00 smithi153 ceph-mon[35071]: pgmap v435: 97 pgs: 97 active+clean; 41 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.5 MiB/s wr, 501 op/s 2024-09-19T09:06:01.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:06:01.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:06:01.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:06:01.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:06:01.470 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:06:01.727 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:06:02.040 INFO:tasks.workunit.client.0.smithi120.stdout: 1 219207 10.24 MB/sec warmup 117 sec latency 70.291 ms 2024-09-19T09:06:02.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.225+0000 7f1c0e39f640 1 -- 172.21.15.120:0/503163103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c08101de0 msgr2=0x7f1c081021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:02.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.225+0000 7f1c0e39f640 1 --2- 172.21.15.120:0/503163103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c08101de0 0x7f1c081021c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8009900 tx=0x7f1bf802a410 comp rx=0 tx=0).stop 2024-09-19T09:06:02.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.225+0000 7f1c0e39f640 1 -- 172.21.15.120:0/503163103 shutdown_connections 2024-09-19T09:06:02.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.225+0000 7f1c0e39f640 1 --2- 172.21.15.120:0/503163103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c08106f10 0x7f1c0810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.225+0000 7f1c0e39f640 1 --2- 172.21.15.120:0/503163103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c08102790 0x7f1c081067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.225+0000 7f1c0e39f640 1 --2- 172.21.15.120:0/503163103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c08101de0 0x7f1c081021c0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.225+0000 7f1c0e39f640 1 -- 172.21.15.120:0/503163103 >> 172.21.15.120:0/503163103 conn(0x7f1c080fd8c0 msgr2=0x7f1c080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:02.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.225+0000 7f1c0e39f640 1 -- 172.21.15.120:0/503163103 shutdown_connections 2024-09-19T09:06:02.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.225+0000 7f1c0e39f640 1 -- 172.21.15.120:0/503163103 wait complete. 2024-09-19T09:06:02.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.226+0000 7f1c0e39f640 1 Processor -- start 2024-09-19T09:06:02.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.226+0000 7f1c0e39f640 1 -- start start 2024-09-19T09:06:02.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.226+0000 7f1c0e39f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c08101de0 0x7f1c080744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:02.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.226+0000 7f1c0e39f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c08102790 0x7f1c08074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:02.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.226+0000 7f1c0e39f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c08106f10 0x7f1c08074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:02.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.226+0000 7f1c0e39f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1c08111f70 con 0x7f1c08106f10 2024-09-19T09:06:02.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.226+0000 7f1c0e39f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1c08111df0 con 0x7f1c08101de0 2024-09-19T09:06:02.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.226+0000 7f1c0e39f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1c081120f0 con 0x7f1c08102790 2024-09-19T09:06:02.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.226+0000 7f1c0c93c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c08106f10 0x7f1c08074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:02.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.226+0000 7f1c0c93c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c08106f10 0x7f1c08074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50260/0 (socket says 172.21.15.120:50260) 2024-09-19T09:06:02.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.226+0000 7f1c0c93c640 1 -- 172.21.15.120:0/3776392503 learned_addr learned my addr 172.21.15.120:0/3776392503 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:02.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.226+0000 7f1c077fe640 1 --2- 172.21.15.120:0/3776392503 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c08102790 0x7f1c08074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:02.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.226+0000 7f1c07fff640 1 --2- 172.21.15.120:0/3776392503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c08101de0 0x7f1c080744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:02.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.227+0000 7f1c0c93c640 1 -- 172.21.15.120:0/3776392503 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c08102790 msgr2=0x7f1c08074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:02.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.227+0000 7f1c0c93c640 1 --2- 172.21.15.120:0/3776392503 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c08102790 0x7f1c08074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.227+0000 7f1c0c93c640 1 -- 172.21.15.120:0/3776392503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c08101de0 msgr2=0x7f1c080744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:02.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.227+0000 7f1c0c93c640 1 --2- 172.21.15.120:0/3776392503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c08101de0 0x7f1c080744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.227+0000 7f1c0c93c640 1 -- 172.21.15.120:0/3776392503 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c081a68d0 con 0x7f1c08106f10 2024-09-19T09:06:02.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.227+0000 7f1c0c93c640 1 --2- 172.21.15.120:0/3776392503 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c08106f10 0x7f1c08074f70 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000eb10 tx=0x7f1c0000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:02.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.227+0000 7f1c057fa640 1 -- 172.21.15.120:0/3776392503 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00004b20 con 0x7f1c08106f10 2024-09-19T09:06:02.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.227+0000 7f1c057fa640 1 -- 172.21.15.120:0/3776392503 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c00004cc0 con 0x7f1c08106f10 2024-09-19T09:06:02.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.227+0000 7f1c0e39f640 1 -- 172.21.15.120:0/3776392503 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c081a6bc0 con 0x7f1c08106f10 2024-09-19T09:06:02.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.227+0000 7f1c057fa640 1 -- 172.21.15.120:0/3776392503 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00010760 con 0x7f1c08106f10 2024-09-19T09:06:02.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.227+0000 7f1c0e39f640 1 -- 172.21.15.120:0/3776392503 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c081a7100 con 0x7f1c08106f10 2024-09-19T09:06:02.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.230+0000 7f1c057fa640 1 -- 172.21.15.120:0/3776392503 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f1c00005020 con 0x7f1c08106f10 2024-09-19T09:06:02.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.230+0000 7f1c10778740 1 Processor -- start 2024-09-19T09:06:02.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.230+0000 7f1c10778740 1 -- start start 2024-09-19T09:06:02.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.230+0000 7f1c10778740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563313a59b40 0x563313a58090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:02.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.230+0000 7f1c10778740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563313a585d0 0x563313a58a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:02.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.230+0000 7f1c10778740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563313a63370 0x563313a65760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:02.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.230+0000 7f1c10778740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5633139be2f0 con 0x563313a585d0 2024-09-19T09:06:02.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.230+0000 7f1c10778740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563313965cf0 con 0x563313a59b40 2024-09-19T09:06:02.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.230+0000 7f1c10778740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5633139ba2b0 con 0x563313a63370 2024-09-19T09:06:02.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.230+0000 7f1c07fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563313a585d0 0x563313a58a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:02.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.230+0000 7f1c07fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563313a585d0 0x563313a58a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50268/0 (socket says 172.21.15.120:50268) 2024-09-19T09:06:02.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.230+0000 7f1c07fff640 1 -- 172.21.15.120:0/33362944 learned_addr learned my addr 172.21.15.120:0/33362944 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:02.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.230+0000 7f1c077fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563313a59b40 0x563313a58090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:02.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.231+0000 7f1c077fe640 1 --2- 172.21.15.120:0/33362944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563313a63370 0x563313a65760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:02.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.231+0000 7f1c057fa640 1 --2- 172.21.15.120:0/3776392503 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1bf00a3240 0x7f1bf00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:02.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.231+0000 7f1c07fff640 1 -- 172.21.15.120:0/33362944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563313a63370 msgr2=0x563313a65760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:02.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.231+0000 7f1c07fff640 1 --2- 172.21.15.120:0/33362944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563313a63370 0x563313a65760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.231+0000 7f1c07fff640 1 -- 172.21.15.120:0/33362944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563313a59b40 msgr2=0x563313a58090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:02.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.231+0000 7f1c07fff640 1 --2- 172.21.15.120:0/33362944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563313a59b40 0x563313a58090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.231+0000 7f1c07fff640 1 -- 172.21.15.120:0/33362944 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56331394a3a0 con 0x563313a585d0 2024-09-19T09:06:02.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.231+0000 7f1c077fe640 1 --2- 172.21.15.120:0/33362944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563313a59b40 0x563313a58090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:02.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.231+0000 7f1c0c93c640 1 --2- 172.21.15.120:0/3776392503 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1bf00a3240 0x7f1bf00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:02.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.231+0000 7f1c057fa640 1 -- 172.21.15.120:0/3776392503 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f1c000cfff0 con 0x7f1c08106f10 2024-09-19T09:06:02.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.231+0000 7f1c07fff640 1 --2- 172.21.15.120:0/33362944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563313a585d0 0x563313a58a10 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8009900 tx=0x7f1bf8005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58de1d1659184dc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:02.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.231+0000 7f1bee7fc640 1 -- 172.21.15.120:0/33362944 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf803f070 con 0x563313a585d0 2024-09-19T09:06:02.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.231+0000 7f1bee7fc640 1 -- 172.21.15.120:0/33362944 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bf8004030 con 0x563313a585d0 2024-09-19T09:06:02.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.231+0000 7f1bee7fc640 1 -- 172.21.15.120:0/33362944 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8035070 con 0x563313a585d0 2024-09-19T09:06:02.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.232+0000 7f1c10778740 1 -- 172.21.15.120:0/33362944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563313a585d0 msgr2=0x563313a58a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:02.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.232+0000 7f1c10778740 1 --2- 172.21.15.120:0/33362944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563313a585d0 0x563313a58a10 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8009900 tx=0x7f1bf8005dd0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.232+0000 7f1c0c93c640 1 --2- 172.21.15.120:0/3776392503 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1bf00a3240 0x7f1bf00a5700 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1c000049d0 tx=0x7f1c00091af0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:02.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.232+0000 7f1c10778740 1 -- 172.21.15.120:0/33362944 shutdown_connections 2024-09-19T09:06:02.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.232+0000 7f1c10778740 1 --2- 172.21.15.120:0/33362944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563313a63370 0x563313a65760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.232+0000 7f1c10778740 1 --2- 172.21.15.120:0/33362944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563313a585d0 0x563313a58a10 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.232+0000 7f1c10778740 1 --2- 172.21.15.120:0/33362944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563313a59b40 0x563313a58090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.232+0000 7f1c10778740 1 -- 172.21.15.120:0/33362944 >> 172.21.15.120:0/33362944 conn(0x563313a21310 msgr2=0x563313a54c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:02.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.232+0000 7f1c10778740 1 -- 172.21.15.120:0/33362944 shutdown_connections 2024-09-19T09:06:02.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.232+0000 7f1c10778740 1 -- 172.21.15.120:0/33362944 wait complete. 2024-09-19T09:06:02.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.233+0000 7f1c10778740 1 -- start start 2024-09-19T09:06:02.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.233+0000 7f1c10778740 1 Processor -- start 2024-09-19T09:06:02.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.233+0000 7f1c10778740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563313a585d0 0x563313a62d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:02.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.233+0000 7f1c10778740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563313a59b40 0x563313a5ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:02.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.233+0000 7f1c10778740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563313a63370 0x563313a60560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:02.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.233+0000 7f1c10778740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563313a66010 con 0x563313a59b40 2024-09-19T09:06:02.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.233+0000 7f1c10778740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563313979060 con 0x563313a585d0 2024-09-19T09:06:02.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.233+0000 7f1c10778740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563313a66190 con 0x563313a63370 2024-09-19T09:06:02.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.233+0000 7f1c10778740 10 client.0 _ll_register_callbacks cb 0x5633136cc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:02.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.233+0000 7f1c07fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563313a59b40 0x563313a5ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:02.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.233+0000 7f1c07fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563313a59b40 0x563313a5ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50280/0 (socket says 172.21.15.120:50280) 2024-09-19T09:06:02.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.233+0000 7f1c07fff640 1 -- 172.21.15.120:0/1061805327 learned_addr learned my addr 172.21.15.120:0/1061805327 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:02.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.233+0000 7f1c077fe640 1 --2- 172.21.15.120:0/1061805327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563313a63370 0x563313a60560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:02.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.233+0000 7f1c077fe640 1 --2- 172.21.15.120:0/1061805327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563313a585d0 0x563313a62d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:02.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.233+0000 7f1c07fff640 1 -- 172.21.15.120:0/1061805327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563313a63370 msgr2=0x563313a60560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:02.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.233+0000 7f1c07fff640 1 --2- 172.21.15.120:0/1061805327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563313a63370 0x563313a60560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.233+0000 7f1c07fff640 1 -- 172.21.15.120:0/1061805327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563313a585d0 msgr2=0x563313a62d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:02.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.233+0000 7f1c07fff640 1 --2- 172.21.15.120:0/1061805327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563313a585d0 0x563313a62d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.234+0000 7f1c07fff640 1 -- 172.21.15.120:0/1061805327 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56331372f3c0 con 0x563313a59b40 2024-09-19T09:06:02.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.234+0000 7f1c07fff640 1 --2- 172.21.15.120:0/1061805327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563313a59b40 0x563313a5ffc0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8006010 tx=0x7f1bf80045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c255f0c9a7120e93 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:02.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.234+0000 7f1be77fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:06:02.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.234+0000 7f1be77fe640 1 -- 172.21.15.120:0/1061805327 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8035070 con 0x563313a59b40 2024-09-19T09:06:02.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.234+0000 7f1be77fe640 1 -- 172.21.15.120:0/1061805327 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f1bf80315b0 con 0x563313a59b40 2024-09-19T09:06:02.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.234+0000 7f1c10778740 1 -- 172.21.15.120:0/1061805327 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bf8004030 con 0x563313a59b40 2024-09-19T09:06:02.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.234+0000 7f1be77fe640 1 -- 172.21.15.120:0/1061805327 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1bf803f040 con 0x563313a59b40 2024-09-19T09:06:02.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.234+0000 7f1c10778740 10 client.16599 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:02.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.234+0000 7f1c10778740 1 -- 172.21.15.120:0/1061805327 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563313a71a90 con 0x563313a59b40 2024-09-19T09:06:02.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.234+0000 7f1be77fe640 1 -- 172.21.15.120:0/1061805327 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bf8004030 con 0x563313a59b40 2024-09-19T09:06:02.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.234+0000 7f1be77fe640 1 -- 172.21.15.120:0/1061805327 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1bf80035f0 con 0x563313a59b40 2024-09-19T09:06:02.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.234+0000 7f1c10778740 10 client.16599 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:02.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.234+0000 7f1c10778740 10 client.16599 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:06:02.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.234+0000 7f1c10778740 20 client.16599 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:02.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.235+0000 7f1c10778740 1 --2- 172.21.15.120:0/1061805327 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x563313a72de0 0x563313a751d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:02.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.235+0000 7f1c10778740 4 client.16599 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:06:02.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.235+0000 7f1c10778740 1 -- 172.21.15.120:0/1061805327 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633139cb860 con 0x563313a72de0 2024-09-19T09:06:02.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.235+0000 7f1c077fe640 1 --2- 172.21.15.120:0/1061805327 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x563313a72de0 0x563313a751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:02.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.235+0000 7f1be77fe640 10 client.16599 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:06:02.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.235+0000 7f1c077fe640 1 --2- 172.21.15.120:0/1061805327 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x563313a72de0 0x563313a751d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a79f7d71914e1070 server_cookie=13b239f5652db752 in_seq=0 out_seq=0 2024-09-19T09:06:02.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.236+0000 7f1be77fe640 1 -- 172.21.15.120:0/1061805327 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5633139cb860 con 0x563313a72de0 2024-09-19T09:06:02.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.236+0000 7f1be77fe640 10 client.16599 handle_command_reply: tid=0 2024-09-19T09:06:02.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.236+0000 7f1c10778740 1 client.16599 shutdown 2024-09-19T09:06:02.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.236+0000 7f1c10778740 1 -- 172.21.15.120:0/1061805327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563313a59b40 msgr2=0x563313a5ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:02.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.236+0000 7f1c10778740 1 --2- 172.21.15.120:0/1061805327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563313a59b40 0x563313a5ffc0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8006010 tx=0x7f1bf80045a0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.236+0000 7f1c10778740 1 -- 172.21.15.120:0/1061805327 shutdown_connections 2024-09-19T09:06:02.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.236+0000 7f1c10778740 1 --2- 172.21.15.120:0/1061805327 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x563313a72de0 0x563313a751d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.236+0000 7f1c10778740 1 --2- 172.21.15.120:0/1061805327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563313a63370 0x563313a60560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.236+0000 7f1c10778740 1 --2- 172.21.15.120:0/1061805327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563313a59b40 0x563313a5ffc0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.236+0000 7f1c10778740 1 --2- 172.21.15.120:0/1061805327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563313a585d0 0x563313a62d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.236+0000 7f1c10778740 1 -- 172.21.15.120:0/1061805327 >> 172.21.15.120:0/1061805327 conn(0x563313a21310 msgr2=0x563313a5a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:02.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.236+0000 7f1be77fe640 0 client.16599 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:06:02.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.236+0000 7f1c10778740 1 -- 172.21.15.120:0/1061805327 shutdown_connections 2024-09-19T09:06:02.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.236+0000 7f1c10778740 1 -- 172.21.15.120:0/1061805327 wait complete. 2024-09-19T09:06:02.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.236+0000 7f1c10778740 20 client.16599 trim_cache size 0 max 16384 2024-09-19T09:06:02.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.249+0000 7f1c10778740 1 Processor -- start 2024-09-19T09:06:02.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.249+0000 7f1c10778740 1 -- start start 2024-09-19T09:06:02.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.249+0000 7f1c10778740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563313a585d0 0x563313a54e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:02.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.249+0000 7f1c10778740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563313a59b40 0x563313a6f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:02.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.249+0000 7f1c10778740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563313a63370 0x563313a7a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:02.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.249+0000 7f1c10778740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1c081af7f0 con 0x563313a585d0 2024-09-19T09:06:02.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.249+0000 7f1c10778740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1c081b1330 con 0x563313a59b40 2024-09-19T09:06:02.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.249+0000 7f1c10778740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1c081afec0 con 0x563313a63370 2024-09-19T09:06:02.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.250+0000 7f1c077fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563313a585d0 0x563313a54e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:02.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.250+0000 7f1c077fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563313a585d0 0x563313a54e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50290/0 (socket says 172.21.15.120:50290) 2024-09-19T09:06:02.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.250+0000 7f1c077fe640 1 -- 172.21.15.120:0/1894419410 learned_addr learned my addr 172.21.15.120:0/1894419410 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:02.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.250+0000 7f1c07fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563313a59b40 0x563313a6f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:02.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.250+0000 7f1c077fe640 1 --2- 172.21.15.120:0/1894419410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563313a63370 0x563313a7a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:02.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.250+0000 7f1c07fff640 1 -- 172.21.15.120:0/1894419410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563313a63370 msgr2=0x563313a7a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:02.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.250+0000 7f1c07fff640 1 --2- 172.21.15.120:0/1894419410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563313a63370 0x563313a7a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.250+0000 7f1c07fff640 1 -- 172.21.15.120:0/1894419410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563313a585d0 msgr2=0x563313a54e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:02.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.250+0000 7f1c07fff640 1 --2- 172.21.15.120:0/1894419410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563313a585d0 0x563313a54e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.250+0000 7f1c07fff640 1 -- 172.21.15.120:0/1894419410 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563313a55960 con 0x563313a59b40 2024-09-19T09:06:02.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.250+0000 7f1c07fff640 1 --2- 172.21.15.120:0/1894419410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563313a59b40 0x563313a6f2e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1bf80039c0 tx=0x7f1bf8005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=830b929dcd614bfd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:02.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.250+0000 7f1bedffb640 1 -- 172.21.15.120:0/1894419410 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8035040 con 0x563313a59b40 2024-09-19T09:06:02.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.250+0000 7f1bedffb640 1 -- 172.21.15.120:0/1894419410 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bf80035f0 con 0x563313a59b40 2024-09-19T09:06:02.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.250+0000 7f1bedffb640 1 -- 172.21.15.120:0/1894419410 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8004e20 con 0x563313a59b40 2024-09-19T09:06:02.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.251+0000 7f1c10778740 1 -- 172.21.15.120:0/1894419410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563313a59b40 msgr2=0x563313a6f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:02.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.251+0000 7f1c10778740 1 --2- 172.21.15.120:0/1894419410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563313a59b40 0x563313a6f2e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1bf80039c0 tx=0x7f1bf8005dd0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.251+0000 7f1c10778740 1 -- 172.21.15.120:0/1894419410 shutdown_connections 2024-09-19T09:06:02.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.251+0000 7f1c10778740 1 --2- 172.21.15.120:0/1894419410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563313a63370 0x563313a7a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.251+0000 7f1c10778740 1 --2- 172.21.15.120:0/1894419410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563313a59b40 0x563313a6f2e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.251+0000 7f1c10778740 1 --2- 172.21.15.120:0/1894419410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563313a585d0 0x563313a54e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.251+0000 7f1c10778740 1 -- 172.21.15.120:0/1894419410 >> 172.21.15.120:0/1894419410 conn(0x563313a21310 msgr2=0x563313a54700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:02.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.251+0000 7f1c10778740 1 -- 172.21.15.120:0/1894419410 shutdown_connections 2024-09-19T09:06:02.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.251+0000 7f1c10778740 1 -- 172.21.15.120:0/1894419410 wait complete. 2024-09-19T09:06:02.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.252+0000 7f1c10778740 1 -- start start 2024-09-19T09:06:02.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.252+0000 7f1c10778740 1 Processor -- start 2024-09-19T09:06:02.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.252+0000 7f1c10778740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563313a585d0 0x563313a6bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:02.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.252+0000 7f1c10778740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563313a59b40 0x563313a5cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:02.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.252+0000 7f1c10778740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563313a63370 0x563313a5d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:02.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.252+0000 7f1c10778740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1c081b0c60 con 0x563313a63370 2024-09-19T09:06:02.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.252+0000 7f1c10778740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1c081b0590 con 0x563313a59b40 2024-09-19T09:06:02.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.252+0000 7f1c10778740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1c081ad7b0 con 0x563313a585d0 2024-09-19T09:06:02.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.252+0000 7f1c10778740 10 client.0 _ll_register_callbacks cb 0x7f1bfc009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:02.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.252+0000 7f1c077fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563313a63370 0x563313a5d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:02.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.252+0000 7f1c07fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563313a59b40 0x563313a5cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:02.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.252+0000 7f1c07fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563313a59b40 0x563313a5cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37818/0 (socket says 172.21.15.120:37818) 2024-09-19T09:06:02.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.252+0000 7f1c07fff640 1 -- 172.21.15.120:0/331494400 learned_addr learned my addr 172.21.15.120:0/331494400 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:02.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.252+0000 7f1c077fe640 1 --2- 172.21.15.120:0/331494400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563313a585d0 0x563313a6bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:02.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.253+0000 7f1c07fff640 1 -- 172.21.15.120:0/331494400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563313a585d0 msgr2=0x563313a6bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:02.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.253+0000 7f1c07fff640 1 --2- 172.21.15.120:0/331494400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563313a585d0 0x563313a6bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.253+0000 7f1c07fff640 1 -- 172.21.15.120:0/331494400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563313a63370 msgr2=0x563313a5d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:02.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.253+0000 7f1c07fff640 1 --2- 172.21.15.120:0/331494400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563313a63370 0x563313a5d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.253+0000 7f1c07fff640 1 -- 172.21.15.120:0/331494400 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563313a54510 con 0x563313a59b40 2024-09-19T09:06:02.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.253+0000 7f1c077fe640 1 --2- 172.21.15.120:0/331494400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563313a63370 0x563313a5d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:02.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.253+0000 7f1c07fff640 1 --2- 172.21.15.120:0/331494400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563313a59b40 0x563313a5cc50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8033000 tx=0x7f1bf8004570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e45f345bbb53b156 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:02.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.253+0000 7f1be77fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:06:02.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.253+0000 7f1be77fe640 1 -- 172.21.15.120:0/331494400 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8035040 con 0x563313a59b40 2024-09-19T09:06:02.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.253+0000 7f1c10778740 1 -- 172.21.15.120:0/331494400 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bf80035f0 con 0x563313a59b40 2024-09-19T09:06:02.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.253+0000 7f1be77fe640 1 -- 172.21.15.120:0/331494400 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f1bf80046c0 con 0x563313a59b40 2024-09-19T09:06:02.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.253+0000 7f1be77fe640 1 -- 172.21.15.120:0/331494400 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1bf80048c0 con 0x563313a59b40 2024-09-19T09:06:02.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.253+0000 7f1c10778740 10 client.25967 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:02.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.253+0000 7f1c10778740 1 -- 172.21.15.120:0/331494400 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563313add570 con 0x563313a59b40 2024-09-19T09:06:02.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.253+0000 7f1be77fe640 1 -- 172.21.15.120:0/331494400 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bf80035f0 con 0x563313a59b40 2024-09-19T09:06:02.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.254+0000 7f1be77fe640 1 -- 172.21.15.120:0/331494400 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1bf80319a0 con 0x563313a59b40 2024-09-19T09:06:02.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.254+0000 7f1c10778740 10 client.25967 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:02.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.254+0000 7f1c10778740 10 client.25967 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:06:02.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.254+0000 7f1c10778740 20 client.25967 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:02.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.254+0000 7f1c10778740 1 --2- 172.21.15.120:0/331494400 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x563313addba0 0x563313adffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:02.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.254+0000 7f1c10778740 4 client.25967 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:06:02.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.254+0000 7f1c10778740 1 -- 172.21.15.120:0/331494400 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5633139c5d70 con 0x563313addba0 2024-09-19T09:06:02.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.254+0000 7f1c077fe640 1 --2- 172.21.15.120:0/331494400 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x563313addba0 0x563313adffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:02.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.254+0000 7f1c077fe640 1 --2- 172.21.15.120:0/331494400 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x563313addba0 0x563313adffc0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=78bf74325fcf3233 server_cookie=2ec05864f0d5ef73 in_seq=0 out_seq=0 2024-09-19T09:06:02.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.254+0000 7f1be77fe640 10 client.25967 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:06:02.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.254+0000 7f1be77fe640 1 -- 172.21.15.120:0/331494400 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5633139c5d70 con 0x563313addba0 2024-09-19T09:06:02.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.254+0000 7f1be77fe640 10 client.25967 handle_command_reply: tid=0 2024-09-19T09:06:02.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.254+0000 7f1c10778740 1 client.25967 shutdown 2024-09-19T09:06:02.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.255+0000 7f1c10778740 1 -- 172.21.15.120:0/331494400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563313a59b40 msgr2=0x563313a5cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:02.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.255+0000 7f1c10778740 1 --2- 172.21.15.120:0/331494400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563313a59b40 0x563313a5cc50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8033000 tx=0x7f1bf8004570 comp rx=0 tx=0).stop 2024-09-19T09:06:02.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.255+0000 7f1c10778740 1 -- 172.21.15.120:0/331494400 shutdown_connections 2024-09-19T09:06:02.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.255+0000 7f1c10778740 1 --2- 172.21.15.120:0/331494400 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x563313addba0 0x563313adffc0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.255+0000 7f1c10778740 1 --2- 172.21.15.120:0/331494400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563313a63370 0x563313a5d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.255+0000 7f1c10778740 1 --2- 172.21.15.120:0/331494400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563313a59b40 0x563313a5cc50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.255+0000 7f1c10778740 1 --2- 172.21.15.120:0/331494400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563313a585d0 0x563313a6bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.255+0000 7f1c10778740 1 -- 172.21.15.120:0/331494400 >> 172.21.15.120:0/331494400 conn(0x563313a21310 msgr2=0x563313a53d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:02.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.255+0000 7f1be77fe640 0 client.25967 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:06:02.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.255+0000 7f1c10778740 1 -- 172.21.15.120:0/331494400 shutdown_connections 2024-09-19T09:06:02.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.255+0000 7f1c10778740 1 -- 172.21.15.120:0/331494400 wait complete. 2024-09-19T09:06:02.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.255+0000 7f1c10778740 20 client.25967 trim_cache size 0 max 16384 2024-09-19T09:06:02.261 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:06:02.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.256+0000 7f1c0e39f640 1 -- 172.21.15.120:0/3776392503 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1bf00a3240 msgr2=0x7f1bf00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:02.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.256+0000 7f1c0e39f640 1 --2- 172.21.15.120:0/3776392503 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1bf00a3240 0x7f1bf00a5700 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1c000049d0 tx=0x7f1c00091af0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.256+0000 7f1c0e39f640 1 -- 172.21.15.120:0/3776392503 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c08106f10 msgr2=0x7f1c08074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:02.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.256+0000 7f1c0e39f640 1 --2- 172.21.15.120:0/3776392503 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c08106f10 0x7f1c08074f70 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000eb10 tx=0x7f1c0000c2d0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.256+0000 7f1c0e39f640 1 -- 172.21.15.120:0/3776392503 shutdown_connections 2024-09-19T09:06:02.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.256+0000 7f1c0e39f640 1 --2- 172.21.15.120:0/3776392503 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1bf00a3240 0x7f1bf00a5700 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.256+0000 7f1c0e39f640 1 --2- 172.21.15.120:0/3776392503 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c08106f10 0x7f1c08074f70 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.256+0000 7f1c0e39f640 1 --2- 172.21.15.120:0/3776392503 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c08102790 0x7f1c08074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.256+0000 7f1c0e39f640 1 --2- 172.21.15.120:0/3776392503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c08101de0 0x7f1c080744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:02.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.256+0000 7f1c0e39f640 1 -- 172.21.15.120:0/3776392503 >> 172.21.15.120:0/3776392503 conn(0x7f1c080fd8c0 msgr2=0x7f1c080fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:02.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.256+0000 7f1c0e39f640 1 -- 172.21.15.120:0/3776392503 shutdown_connections 2024-09-19T09:06:02.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:02.256+0000 7f1c0e39f640 1 -- 172.21.15.120:0/3776392503 wait complete. 2024-09-19T09:06:03.040 INFO:tasks.workunit.client.0.smithi120.stdout: 1 221826 10.27 MB/sec warmup 118 sec latency 33.108 ms 2024-09-19T09:06:03.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:02 smithi120 ceph-mon[31150]: pgmap v436: 97 pgs: 97 active+clean; 44 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.4 MiB/s wr, 463 op/s 2024-09-19T09:06:03.126 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:06:03.189 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:02 smithi196 ceph-mon[35150]: pgmap v436: 97 pgs: 97 active+clean; 44 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.4 MiB/s wr, 463 op/s 2024-09-19T09:06:03.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:02 smithi153 ceph-mon[35071]: pgmap v436: 97 pgs: 97 active+clean; 44 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.4 MiB/s wr, 463 op/s 2024-09-19T09:06:03.385 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:06:03.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.911+0000 7f99d63cc640 1 -- 172.21.15.120:0/3326304604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99d0109830 msgr2=0x7f99d0109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:03.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.911+0000 7f99d63cc640 1 --2- 172.21.15.120:0/3326304604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99d0109830 0x7f99d0109c90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f99c4009900 tx=0x7f99c402a400 comp rx=0 tx=0).stop 2024-09-19T09:06:03.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.912+0000 7f99d63cc640 1 -- 172.21.15.120:0/3326304604 shutdown_connections 2024-09-19T09:06:03.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.912+0000 7f99d63cc640 1 --2- 172.21.15.120:0/3326304604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99d011b140 0x7f99d011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.912+0000 7f99d63cc640 1 --2- 172.21.15.120:0/3326304604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99d0109830 0x7f99d0109c90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.912+0000 7f99d63cc640 1 --2- 172.21.15.120:0/3326304604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99d0109450 0x7f99d0110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.912+0000 7f99d63cc640 1 -- 172.21.15.120:0/3326304604 >> 172.21.15.120:0/3326304604 conn(0x7f99d006e140 msgr2=0x7f99d006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:03.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.913+0000 7f99d63cc640 1 -- 172.21.15.120:0/3326304604 shutdown_connections 2024-09-19T09:06:03.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.913+0000 7f99d63cc640 1 -- 172.21.15.120:0/3326304604 wait complete. 2024-09-19T09:06:03.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.914+0000 7f99d63cc640 1 Processor -- start 2024-09-19T09:06:03.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.914+0000 7f99d63cc640 1 -- start start 2024-09-19T09:06:03.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.914+0000 7f99d63cc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99d0109450 0x7f99d01186b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:03.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.915+0000 7f99d63cc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99d0109830 0x7f99d0118bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:03.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.914+0000 7f99cffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99d0109450 0x7f99d01186b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:03.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.916+0000 7f99cf7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99d0109830 0x7f99d0118bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:03.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.916+0000 7f99cf7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99d0109830 0x7f99d0118bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:38908/0 (socket says 172.21.15.120:38908) 2024-09-19T09:06:03.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.916+0000 7f99d63cc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99d011b140 0x7f99d0111920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:03.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.916+0000 7f99d63cc640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f99d0120df0 con 0x7f99d0109450 2024-09-19T09:06:03.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.916+0000 7f99d63cc640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f99d0120c70 con 0x7f99d011b140 2024-09-19T09:06:03.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.916+0000 7f99d63cc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f99d0120f70 con 0x7f99d0109830 2024-09-19T09:06:03.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.916+0000 7f99cffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99d0109450 0x7f99d01186b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50314/0 (socket says 172.21.15.120:50314) 2024-09-19T09:06:03.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.916+0000 7f99cffff640 1 -- 172.21.15.120:0/2006946436 learned_addr learned my addr 172.21.15.120:0/2006946436 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:03.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.917+0000 7f99d4969640 1 --2- 172.21.15.120:0/2006946436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99d011b140 0x7f99d0111920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:03.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.917+0000 7f99d4969640 1 -- 172.21.15.120:0/2006946436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99d0109830 msgr2=0x7f99d0118bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:03.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.917+0000 7f99d4969640 1 --2- 172.21.15.120:0/2006946436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99d0109830 0x7f99d0118bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.917+0000 7f99d4969640 1 -- 172.21.15.120:0/2006946436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99d0109450 msgr2=0x7f99d01186b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:03.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.918+0000 7f99d4969640 1 --2- 172.21.15.120:0/2006946436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99d0109450 0x7f99d01186b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.918+0000 7f99d4969640 1 -- 172.21.15.120:0/2006946436 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d01121e0 con 0x7f99d011b140 2024-09-19T09:06:03.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.918+0000 7f99d4969640 1 --2- 172.21.15.120:0/2006946436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99d011b140 0x7f99d0111920 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f99c800ea80 tx=0x7f99c800ef40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:03.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.918+0000 7f99cd7fa640 1 -- 172.21.15.120:0/2006946436 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c8004b20 con 0x7f99d011b140 2024-09-19T09:06:03.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.918+0000 7f99d63cc640 1 -- 172.21.15.120:0/2006946436 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99d01124d0 con 0x7f99d011b140 2024-09-19T09:06:03.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.918+0000 7f99d63cc640 1 -- 172.21.15.120:0/2006946436 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99d0116470 con 0x7f99d011b140 2024-09-19T09:06:03.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.918+0000 7f99cd7fa640 1 -- 172.21.15.120:0/2006946436 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99c8004cc0 con 0x7f99d011b140 2024-09-19T09:06:03.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.919+0000 7f99cd7fa640 1 -- 172.21.15.120:0/2006946436 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c8010720 con 0x7f99d011b140 2024-09-19T09:06:03.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.923+0000 7f99cd7fa640 1 -- 172.21.15.120:0/2006946436 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f99c80108c0 con 0x7f99d011b140 2024-09-19T09:06:03.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.924+0000 7f99cd7fa640 1 --2- 172.21.15.120:0/2006946436 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f99b80a3300 0x7f99b80a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:03.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.924+0000 7f99cffff640 1 --2- 172.21.15.120:0/2006946436 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f99b80a3300 0x7f99b80a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:03.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.924+0000 7f99cd7fa640 1 -- 172.21.15.120:0/2006946436 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f99c80d0040 con 0x7f99d011b140 2024-09-19T09:06:03.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.924+0000 7f99d87a5740 1 Processor -- start 2024-09-19T09:06:03.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.924+0000 7f99d87a5740 1 -- start start 2024-09-19T09:06:03.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.924+0000 7f99cffff640 1 --2- 172.21.15.120:0/2006946436 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f99b80a3300 0x7f99b80a57c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f99c00058c0 tx=0x7f99c000a580 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:03.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.924+0000 7f99d87a5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c0e2b8b40 0x564c0e2b7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:03.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.924+0000 7f99d87a5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564c0e2b75d0 0x564c0e2b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:03.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.924+0000 7f99d87a5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c0e2c2370 0x564c0e2c4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:03.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.924+0000 7f99d87a5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564c0e21d2f0 con 0x564c0e2b75d0 2024-09-19T09:06:03.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.924+0000 7f99d87a5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564c0e1c4cf0 con 0x564c0e2b8b40 2024-09-19T09:06:03.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.924+0000 7f99d87a5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564c0e2192b0 con 0x564c0e2c2370 2024-09-19T09:06:03.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.925+0000 7f99cffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c0e2b8b40 0x564c0e2b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:03.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.925+0000 7f99cf7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564c0e2b75d0 0x564c0e2b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:03.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.925+0000 7f99cffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c0e2b8b40 0x564c0e2b7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37852/0 (socket says 172.21.15.120:37852) 2024-09-19T09:06:03.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.925+0000 7f99cffff640 1 -- 172.21.15.120:0/1792082333 learned_addr learned my addr 172.21.15.120:0/1792082333 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:03.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.925+0000 7f99d4969640 1 --2- 172.21.15.120:0/1792082333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c0e2c2370 0x564c0e2c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:03.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.925+0000 7f99cf7fe640 1 -- 172.21.15.120:0/1792082333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c0e2c2370 msgr2=0x564c0e2c4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:03.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.925+0000 7f99cf7fe640 1 --2- 172.21.15.120:0/1792082333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c0e2c2370 0x564c0e2c4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.925+0000 7f99cf7fe640 1 -- 172.21.15.120:0/1792082333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c0e2b8b40 msgr2=0x564c0e2b7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:03.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.925+0000 7f99cf7fe640 1 --2- 172.21.15.120:0/1792082333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c0e2b8b40 0x564c0e2b7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.925+0000 7f99cf7fe640 1 -- 172.21.15.120:0/1792082333 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c0e1a93a0 con 0x564c0e2b75d0 2024-09-19T09:06:03.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.925+0000 7f99cffff640 1 --2- 172.21.15.120:0/1792082333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c0e2b8b40 0x564c0e2b7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:03.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.925+0000 7f99cf7fe640 1 --2- 172.21.15.120:0/1792082333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564c0e2b75d0 0x564c0e2b7a10 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f99c402a9f0 tx=0x7f99c4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41bc3e80549c31f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:03.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.925+0000 7f99b67fc640 1 -- 172.21.15.120:0/1792082333 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c4035030 con 0x564c0e2b75d0 2024-09-19T09:06:03.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.925+0000 7f99b67fc640 1 -- 172.21.15.120:0/1792082333 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99c4005520 con 0x564c0e2b75d0 2024-09-19T09:06:03.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.926+0000 7f99b67fc640 1 -- 172.21.15.120:0/1792082333 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c40035f0 con 0x564c0e2b75d0 2024-09-19T09:06:03.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.926+0000 7f99d87a5740 1 -- 172.21.15.120:0/1792082333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564c0e2b75d0 msgr2=0x564c0e2b7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:03.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.926+0000 7f99d87a5740 1 --2- 172.21.15.120:0/1792082333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564c0e2b75d0 0x564c0e2b7a10 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f99c402a9f0 tx=0x7f99c4004040 comp rx=0 tx=0).stop 2024-09-19T09:06:03.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.926+0000 7f99d87a5740 1 -- 172.21.15.120:0/1792082333 shutdown_connections 2024-09-19T09:06:03.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.926+0000 7f99d87a5740 1 --2- 172.21.15.120:0/1792082333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c0e2c2370 0x564c0e2c4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.926+0000 7f99d87a5740 1 --2- 172.21.15.120:0/1792082333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564c0e2b75d0 0x564c0e2b7a10 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.926+0000 7f99d87a5740 1 --2- 172.21.15.120:0/1792082333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c0e2b8b40 0x564c0e2b7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.926+0000 7f99d87a5740 1 -- 172.21.15.120:0/1792082333 >> 172.21.15.120:0/1792082333 conn(0x564c0e280310 msgr2=0x564c0e2b3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:03.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.926+0000 7f99d87a5740 1 -- 172.21.15.120:0/1792082333 shutdown_connections 2024-09-19T09:06:03.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.926+0000 7f99d87a5740 1 -- 172.21.15.120:0/1792082333 wait complete. 2024-09-19T09:06:03.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.927+0000 7f99d87a5740 1 -- start start 2024-09-19T09:06:03.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.927+0000 7f99d87a5740 1 Processor -- start 2024-09-19T09:06:03.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.927+0000 7f99d87a5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564c0e2b75d0 0x564c0e2c1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:03.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.927+0000 7f99d87a5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c0e2b8b40 0x564c0e2befa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:03.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.927+0000 7f99d87a5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c0e2c2370 0x564c0e2bf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:03.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.927+0000 7f99d87a5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564c0e2c5060 con 0x564c0e2b75d0 2024-09-19T09:06:03.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.927+0000 7f99d87a5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564c0e1d8060 con 0x564c0e2c2370 2024-09-19T09:06:03.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.927+0000 7f99cffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564c0e2b75d0 0x564c0e2c1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:03.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.927+0000 7f99d87a5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564c0e2c51e0 con 0x564c0e2b8b40 2024-09-19T09:06:03.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.927+0000 7f99cffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564c0e2b75d0 0x564c0e2c1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50332/0 (socket says 172.21.15.120:50332) 2024-09-19T09:06:03.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.927+0000 7f99cffff640 1 -- 172.21.15.120:0/151265761 learned_addr learned my addr 172.21.15.120:0/151265761 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:03.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.927+0000 7f99cf7fe640 1 --2- 172.21.15.120:0/151265761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c0e2b8b40 0x564c0e2befa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:03.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.927+0000 7f99d4969640 1 --2- 172.21.15.120:0/151265761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c0e2c2370 0x564c0e2bf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:03.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.927+0000 7f99d87a5740 10 client.0 _ll_register_callbacks cb 0x564c0df2b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:03.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.928+0000 7f99d4969640 1 -- 172.21.15.120:0/151265761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c0e2b8b40 msgr2=0x564c0e2befa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:03.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.928+0000 7f99d4969640 1 --2- 172.21.15.120:0/151265761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c0e2b8b40 0x564c0e2befa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.928+0000 7f99d4969640 1 -- 172.21.15.120:0/151265761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564c0e2b75d0 msgr2=0x564c0e2c1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:03.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.928+0000 7f99d4969640 1 --2- 172.21.15.120:0/151265761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564c0e2b75d0 0x564c0e2c1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.928+0000 7f99d4969640 1 -- 172.21.15.120:0/151265761 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c0df8e3c0 con 0x564c0e2c2370 2024-09-19T09:06:03.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.928+0000 7f99cf7fe640 1 --2- 172.21.15.120:0/151265761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c0e2b8b40 0x564c0e2befa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:03.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.928+0000 7f99cffff640 1 --2- 172.21.15.120:0/151265761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564c0e2b75d0 0x564c0e2c1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:03.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.928+0000 7f99d4969640 1 --2- 172.21.15.120:0/151265761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c0e2c2370 0x564c0e2bf520 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f99c800c760 tx=0x7f99c809d2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73f479981ee5ad1f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:03.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.928+0000 7f99af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:06:03.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.928+0000 7f99af7fe640 1 -- 172.21.15.120:0/151265761 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c80a1a60 con 0x564c0e2c2370 2024-09-19T09:06:03.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.928+0000 7f99af7fe640 1 -- 172.21.15.120:0/151265761 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f99c80b1bd0 con 0x564c0e2c2370 2024-09-19T09:06:03.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.928+0000 7f99d87a5740 1 -- 172.21.15.120:0/151265761 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99c4005520 con 0x564c0e2c2370 2024-09-19T09:06:03.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.928+0000 7f99af7fe640 1 -- 172.21.15.120:0/151265761 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f99c80a6320 con 0x564c0e2c2370 2024-09-19T09:06:03.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.928+0000 7f99d87a5740 10 client.25979 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:03.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.928+0000 7f99d87a5740 1 -- 172.21.15.120:0/151265761 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564c0e2d0aa0 con 0x564c0e2c2370 2024-09-19T09:06:03.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.928+0000 7f99af7fe640 1 -- 172.21.15.120:0/151265761 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99c4005520 con 0x564c0e2c2370 2024-09-19T09:06:03.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.929+0000 7f99af7fe640 1 -- 172.21.15.120:0/151265761 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f99c80bd040 con 0x564c0e2c2370 2024-09-19T09:06:03.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.929+0000 7f99d87a5740 10 client.25979 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:03.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.929+0000 7f99d87a5740 10 client.25979 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:06:03.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.929+0000 7f99d87a5740 20 client.25979 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:03.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.929+0000 7f99d87a5740 1 --2- 172.21.15.120:0/151265761 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x564c0e2d1fb0 0x564c0e2d4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:03.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.929+0000 7f99d87a5740 4 client.25979 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:06:03.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.929+0000 7f99d87a5740 1 -- 172.21.15.120:0/151265761 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c0e22a860 con 0x564c0e2d1fb0 2024-09-19T09:06:03.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.929+0000 7f99cffff640 1 --2- 172.21.15.120:0/151265761 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x564c0e2d1fb0 0x564c0e2d4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:03.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.929+0000 7f99cffff640 1 --2- 172.21.15.120:0/151265761 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x564c0e2d1fb0 0x564c0e2d4450 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aae6d8d787b3b149 server_cookie=caccfb3183ce5152 in_seq=0 out_seq=0 2024-09-19T09:06:03.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.929+0000 7f99af7fe640 10 client.25979 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:06:03.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.930+0000 7f99af7fe640 1 -- 172.21.15.120:0/151265761 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564c0e22a860 con 0x564c0e2d1fb0 2024-09-19T09:06:03.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.930+0000 7f99af7fe640 10 client.25979 handle_command_reply: tid=0 2024-09-19T09:06:03.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.930+0000 7f99d87a5740 1 client.25979 shutdown 2024-09-19T09:06:03.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.931+0000 7f99d87a5740 1 -- 172.21.15.120:0/151265761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c0e2c2370 msgr2=0x564c0e2bf520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:03.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.931+0000 7f99d87a5740 1 --2- 172.21.15.120:0/151265761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c0e2c2370 0x564c0e2bf520 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f99c800c760 tx=0x7f99c809d2b0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.931+0000 7f99d87a5740 1 -- 172.21.15.120:0/151265761 shutdown_connections 2024-09-19T09:06:03.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.931+0000 7f99d87a5740 1 --2- 172.21.15.120:0/151265761 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x564c0e2d1fb0 0x564c0e2d4450 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.931+0000 7f99af7fe640 0 client.25979 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:06:03.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.931+0000 7f99d87a5740 1 --2- 172.21.15.120:0/151265761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c0e2c2370 0x564c0e2bf520 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.931+0000 7f99d87a5740 1 --2- 172.21.15.120:0/151265761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c0e2b8b40 0x564c0e2befa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.931+0000 7f99d87a5740 1 --2- 172.21.15.120:0/151265761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564c0e2b75d0 0x564c0e2c1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.931+0000 7f99d87a5740 1 -- 172.21.15.120:0/151265761 >> 172.21.15.120:0/151265761 conn(0x564c0e280310 msgr2=0x564c0e2b9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:03.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.931+0000 7f99d87a5740 1 -- 172.21.15.120:0/151265761 shutdown_connections 2024-09-19T09:06:03.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.931+0000 7f99d87a5740 1 -- 172.21.15.120:0/151265761 wait complete. 2024-09-19T09:06:03.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.931+0000 7f99d87a5740 20 client.25979 trim_cache size 0 max 16384 2024-09-19T09:06:03.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.943+0000 7f99d87a5740 1 Processor -- start 2024-09-19T09:06:03.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.943+0000 7f99d87a5740 1 -- start start 2024-09-19T09:06:03.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.944+0000 7f99d87a5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564c0e2b75d0 0x564c0e2ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:03.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.944+0000 7f99d87a5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c0e2b8b40 0x564c0e2b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:03.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.944+0000 7f99d87a5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c0e2c2370 0x564c0e2b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:03.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.944+0000 7f99d87a5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc00e4a0 con 0x564c0e2b75d0 2024-09-19T09:06:03.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.944+0000 7f99d87a5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc00ffe0 con 0x564c0e2b8b40 2024-09-19T09:06:03.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.944+0000 7f99d87a5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc00eb70 con 0x564c0e2c2370 2024-09-19T09:06:03.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.944+0000 7f99cffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564c0e2b75d0 0x564c0e2ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:03.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.944+0000 7f99cf7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c0e2b8b40 0x564c0e2b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:03.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.944+0000 7f99cffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564c0e2b75d0 0x564c0e2ce330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50342/0 (socket says 172.21.15.120:50342) 2024-09-19T09:06:03.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.944+0000 7f99cf7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c0e2b8b40 0x564c0e2b3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37878/0 (socket says 172.21.15.120:37878) 2024-09-19T09:06:03.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.944+0000 7f99cffff640 1 -- 172.21.15.120:0/2742409625 learned_addr learned my addr 172.21.15.120:0/2742409625 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:03.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.944+0000 7f99d4969640 1 --2- 172.21.15.120:0/2742409625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c0e2c2370 0x564c0e2b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:03.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.944+0000 7f99cffff640 1 -- 172.21.15.120:0/2742409625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c0e2c2370 msgr2=0x564c0e2b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:03.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.944+0000 7f99cffff640 1 --2- 172.21.15.120:0/2742409625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c0e2c2370 0x564c0e2b3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.944+0000 7f99cffff640 1 -- 172.21.15.120:0/2742409625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c0e2b8b40 msgr2=0x564c0e2b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:03.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.944+0000 7f99cffff640 1 --2- 172.21.15.120:0/2742409625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c0e2b8b40 0x564c0e2b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.944+0000 7f99cffff640 1 -- 172.21.15.120:0/2742409625 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c0e2c2060 con 0x564c0e2b75d0 2024-09-19T09:06:03.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.944+0000 7f99cffff640 1 --2- 172.21.15.120:0/2742409625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564c0e2b75d0 0x564c0e2ce330 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f99c000a4c0 tx=0x7f99c0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc09494ddb358cbd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:03.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.944+0000 7f99b5ffb640 1 -- 172.21.15.120:0/2742409625 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c0010900 con 0x564c0e2b75d0 2024-09-19T09:06:03.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.944+0000 7f99b5ffb640 1 -- 172.21.15.120:0/2742409625 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99c00200d0 con 0x564c0e2b75d0 2024-09-19T09:06:03.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.945+0000 7f99b5ffb640 1 -- 172.21.15.120:0/2742409625 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c001eca0 con 0x564c0e2b75d0 2024-09-19T09:06:03.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.945+0000 7f99d87a5740 1 -- 172.21.15.120:0/2742409625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564c0e2b75d0 msgr2=0x564c0e2ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:03.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.945+0000 7f99d87a5740 1 --2- 172.21.15.120:0/2742409625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564c0e2b75d0 0x564c0e2ce330 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f99c000a4c0 tx=0x7f99c0011000 comp rx=0 tx=0).stop 2024-09-19T09:06:03.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.945+0000 7f99d87a5740 1 -- 172.21.15.120:0/2742409625 shutdown_connections 2024-09-19T09:06:03.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.945+0000 7f99d87a5740 1 --2- 172.21.15.120:0/2742409625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c0e2c2370 0x564c0e2b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.945+0000 7f99d87a5740 1 --2- 172.21.15.120:0/2742409625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c0e2b8b40 0x564c0e2b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.945+0000 7f99d87a5740 1 --2- 172.21.15.120:0/2742409625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564c0e2b75d0 0x564c0e2ce330 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.945+0000 7f99d87a5740 1 -- 172.21.15.120:0/2742409625 >> 172.21.15.120:0/2742409625 conn(0x564c0e280310 msgr2=0x564c0e2bd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:03.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.945+0000 7f99d87a5740 1 -- 172.21.15.120:0/2742409625 shutdown_connections 2024-09-19T09:06:03.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.945+0000 7f99d87a5740 1 -- 172.21.15.120:0/2742409625 wait complete. 2024-09-19T09:06:03.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.946+0000 7f99d87a5740 1 -- start start 2024-09-19T09:06:03.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.946+0000 7f99d87a5740 1 Processor -- start 2024-09-19T09:06:03.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.946+0000 7f99d87a5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564c0e2b75d0 0x564c0e2ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:03.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.946+0000 7f99d87a5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c0e2b8b40 0x564c0e2cad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:03.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.946+0000 7f99d87a5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c0e2c2370 0x564c0e2bb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:03.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.946+0000 7f99d87a5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc00f910 con 0x564c0e2b75d0 2024-09-19T09:06:03.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.946+0000 7f99d87a5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc00f240 con 0x564c0e2c2370 2024-09-19T09:06:03.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.946+0000 7f99d87a5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc00c460 con 0x564c0e2b8b40 2024-09-19T09:06:03.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.946+0000 7f99d87a5740 10 client.0 _ll_register_callbacks cb 0x7f99c809d480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:03.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.946+0000 7f99cffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564c0e2b75d0 0x564c0e2ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:03.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.946+0000 7f99cffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564c0e2b75d0 0x564c0e2ca7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50358/0 (socket says 172.21.15.120:50358) 2024-09-19T09:06:03.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.946+0000 7f99cffff640 1 -- 172.21.15.120:0/2098578380 learned_addr learned my addr 172.21.15.120:0/2098578380 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:03.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.946+0000 7f99d4969640 1 --2- 172.21.15.120:0/2098578380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c0e2c2370 0x564c0e2bb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:03.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.946+0000 7f99cf7fe640 1 --2- 172.21.15.120:0/2098578380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c0e2b8b40 0x564c0e2cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:03.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.947+0000 7f99cffff640 1 -- 172.21.15.120:0/2098578380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c0e2b8b40 msgr2=0x564c0e2cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:03.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.947+0000 7f99cffff640 1 --2- 172.21.15.120:0/2098578380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c0e2b8b40 0x564c0e2cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.947+0000 7f99cffff640 1 -- 172.21.15.120:0/2098578380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c0e2c2370 msgr2=0x564c0e2bb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:03.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.947+0000 7f99cffff640 1 --2- 172.21.15.120:0/2098578380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c0e2c2370 0x564c0e2bb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.947+0000 7f99cffff640 1 -- 172.21.15.120:0/2098578380 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c0e2bc170 con 0x564c0e2b75d0 2024-09-19T09:06:03.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.947+0000 7f99d4969640 1 --2- 172.21.15.120:0/2098578380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c0e2c2370 0x564c0e2bb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:03.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.947+0000 7f99cffff640 1 --2- 172.21.15.120:0/2098578380 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564c0e2b75d0 0x564c0e2ca7d0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f99c001e980 tx=0x7f99c001fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd1c76343aead2c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:03.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.947+0000 7f99af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:06:03.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.947+0000 7f99af7fe640 1 -- 172.21.15.120:0/2098578380 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c0040060 con 0x564c0e2b75d0 2024-09-19T09:06:03.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.947+0000 7f99af7fe640 1 -- 172.21.15.120:0/2098578380 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f99c002e050 con 0x564c0e2b75d0 2024-09-19T09:06:03.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.947+0000 7f99d87a5740 1 -- 172.21.15.120:0/2098578380 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99c00200d0 con 0x564c0e2b75d0 2024-09-19T09:06:03.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.947+0000 7f99af7fe640 1 -- 172.21.15.120:0/2098578380 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f99c0010b10 con 0x564c0e2b75d0 2024-09-19T09:06:03.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.947+0000 7f99d87a5740 10 client.16623 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:03.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.947+0000 7f99d87a5740 1 -- 172.21.15.120:0/2098578380 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564c0e33c8f0 con 0x564c0e2b75d0 2024-09-19T09:06:03.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.947+0000 7f99af7fe640 1 -- 172.21.15.120:0/2098578380 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99c00200d0 con 0x564c0e2b75d0 2024-09-19T09:06:03.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.948+0000 7f99af7fe640 1 -- 172.21.15.120:0/2098578380 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f99c0005370 con 0x564c0e2b75d0 2024-09-19T09:06:03.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.948+0000 7f99d87a5740 10 client.16623 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:03.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.948+0000 7f99d87a5740 10 client.16623 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:06:03.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.948+0000 7f99d87a5740 20 client.16623 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:03.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.948+0000 7f99d87a5740 1 --2- 172.21.15.120:0/2098578380 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x564c0e2d1fb0 0x564c0e33ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:03.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.948+0000 7f99d87a5740 4 client.16623 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:06:03.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.948+0000 7f99d87a5740 1 -- 172.21.15.120:0/2098578380 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564c0e224d70 con 0x564c0e2d1fb0 2024-09-19T09:06:03.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.948+0000 7f99cf7fe640 1 --2- 172.21.15.120:0/2098578380 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x564c0e2d1fb0 0x564c0e33ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:03.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.948+0000 7f99cf7fe640 1 --2- 172.21.15.120:0/2098578380 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x564c0e2d1fb0 0x564c0e33ef40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f1871f54761b15cf server_cookie=bdbbce8172f8c5be in_seq=0 out_seq=0 2024-09-19T09:06:03.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.948+0000 7f99af7fe640 10 client.16623 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:06:03.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.949+0000 7f99af7fe640 1 -- 172.21.15.120:0/2098578380 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564c0e224d70 con 0x564c0e2d1fb0 2024-09-19T09:06:03.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.949+0000 7f99af7fe640 10 client.16623 handle_command_reply: tid=0 2024-09-19T09:06:03.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.949+0000 7f99d87a5740 1 client.16623 shutdown 2024-09-19T09:06:03.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.949+0000 7f99d87a5740 1 -- 172.21.15.120:0/2098578380 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564c0e2b75d0 msgr2=0x564c0e2ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:03.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.949+0000 7f99d87a5740 1 --2- 172.21.15.120:0/2098578380 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564c0e2b75d0 0x564c0e2ca7d0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f99c001e980 tx=0x7f99c001fda0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.949+0000 7f99d87a5740 1 -- 172.21.15.120:0/2098578380 shutdown_connections 2024-09-19T09:06:03.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.949+0000 7f99d87a5740 1 --2- 172.21.15.120:0/2098578380 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x564c0e2d1fb0 0x564c0e33ef40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.949+0000 7f99af7fe640 0 client.16623 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:06:03.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.949+0000 7f99d87a5740 1 --2- 172.21.15.120:0/2098578380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564c0e2c2370 0x564c0e2bb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.949+0000 7f99d87a5740 1 --2- 172.21.15.120:0/2098578380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c0e2b8b40 0x564c0e2cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.949+0000 7f99d87a5740 1 --2- 172.21.15.120:0/2098578380 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564c0e2b75d0 0x564c0e2ca7d0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.949+0000 7f99d87a5740 1 -- 172.21.15.120:0/2098578380 >> 172.21.15.120:0/2098578380 conn(0x564c0e280310 msgr2=0x564c0e2b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:03.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.949+0000 7f99d87a5740 1 -- 172.21.15.120:0/2098578380 shutdown_connections 2024-09-19T09:06:03.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.949+0000 7f99d87a5740 1 -- 172.21.15.120:0/2098578380 wait complete. 2024-09-19T09:06:03.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.949+0000 7f99d87a5740 20 client.16623 trim_cache size 0 max 16384 2024-09-19T09:06:03.955 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:06:03.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.950+0000 7f99d63cc640 1 -- 172.21.15.120:0/2006946436 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f99b80a3300 msgr2=0x7f99b80a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:03.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.950+0000 7f99d63cc640 1 --2- 172.21.15.120:0/2006946436 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f99b80a3300 0x7f99b80a57c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f99c00058c0 tx=0x7f99c000a580 comp rx=0 tx=0).stop 2024-09-19T09:06:03.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.950+0000 7f99d63cc640 1 -- 172.21.15.120:0/2006946436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99d011b140 msgr2=0x7f99d0111920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:03.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.950+0000 7f99d63cc640 1 --2- 172.21.15.120:0/2006946436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99d011b140 0x7f99d0111920 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f99c800ea80 tx=0x7f99c800ef40 comp rx=0 tx=0).stop 2024-09-19T09:06:03.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.950+0000 7f99d63cc640 1 -- 172.21.15.120:0/2006946436 shutdown_connections 2024-09-19T09:06:03.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.950+0000 7f99d63cc640 1 --2- 172.21.15.120:0/2006946436 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f99b80a3300 0x7f99b80a57c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.950+0000 7f99d63cc640 1 --2- 172.21.15.120:0/2006946436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99d011b140 0x7f99d0111920 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.950+0000 7f99d63cc640 1 --2- 172.21.15.120:0/2006946436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99d0109830 0x7f99d0118bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.950+0000 7f99d63cc640 1 --2- 172.21.15.120:0/2006946436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99d0109450 0x7f99d01186b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:03.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.950+0000 7f99d63cc640 1 -- 172.21.15.120:0/2006946436 >> 172.21.15.120:0/2006946436 conn(0x7f99d006e140 msgr2=0x7f99d011c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:03.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.950+0000 7f99d63cc640 1 -- 172.21.15.120:0/2006946436 shutdown_connections 2024-09-19T09:06:03.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:03.950+0000 7f99d63cc640 1 -- 172.21.15.120:0/2006946436 wait complete. 2024-09-19T09:06:04.041 INFO:tasks.workunit.client.0.smithi120.stdout: 1 224195 10.30 MB/sec warmup 119 sec latency 57.445 ms 2024-09-19T09:06:04.791 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:06:05.045 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:06:05.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:04 smithi120 ceph-mon[31150]: pgmap v437: 97 pgs: 97 active+clean; 45 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.6 MiB/s wr, 499 op/s 2024-09-19T09:06:05.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:04 smithi196 ceph-mon[35150]: pgmap v437: 97 pgs: 97 active+clean; 45 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.6 MiB/s wr, 499 op/s 2024-09-19T09:06:05.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:04 smithi153 ceph-mon[35071]: pgmap v437: 97 pgs: 97 active+clean; 45 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.6 MiB/s wr, 499 op/s 2024-09-19T09:06:05.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.603+0000 7fb277b7b640 1 -- 172.21.15.120:0/2813696343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb270100e20 msgr2=0x7fb270101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:05.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.603+0000 7fb277b7b640 1 --2- 172.21.15.120:0/2813696343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb270100e20 0x7fb270101200 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb26c009900 tx=0x7fb26c02a480 comp rx=0 tx=0).stop 2024-09-19T09:06:05.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.603+0000 7fb277b7b640 1 -- 172.21.15.120:0/2813696343 shutdown_connections 2024-09-19T09:06:05.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.603+0000 7fb277b7b640 1 --2- 172.21.15.120:0/2813696343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb270102ef0 0x7fb27010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.603+0000 7fb277b7b640 1 --2- 172.21.15.120:0/2813696343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb270102570 0x7fb2701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.603+0000 7fb277b7b640 1 --2- 172.21.15.120:0/2813696343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb270100e20 0x7fb270101200 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.603+0000 7fb277b7b640 1 -- 172.21.15.120:0/2813696343 >> 172.21.15.120:0/2813696343 conn(0x7fb2700fc7d0 msgr2=0x7fb2700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:05.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.603+0000 7fb277b7b640 1 -- 172.21.15.120:0/2813696343 shutdown_connections 2024-09-19T09:06:05.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.604+0000 7fb277b7b640 1 -- 172.21.15.120:0/2813696343 wait complete. 2024-09-19T09:06:05.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.604+0000 7fb277b7b640 1 Processor -- start 2024-09-19T09:06:05.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.604+0000 7fb277b7b640 1 -- start start 2024-09-19T09:06:05.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.605+0000 7fb277b7b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb270100e20 0x7fb27019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:05.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.605+0000 7fb277b7b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb270102570 0x7fb27019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:05.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.605+0000 7fb277b7b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb270102ef0 0x7fb27019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:05.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.605+0000 7fb277b7b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb270110f80 con 0x7fb270100e20 2024-09-19T09:06:05.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.605+0000 7fb277b7b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb270110e00 con 0x7fb270102ef0 2024-09-19T09:06:05.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.605+0000 7fb277b7b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb270111100 con 0x7fb270102570 2024-09-19T09:06:05.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.605+0000 7fb275917640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb270100e20 0x7fb27019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:05.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.605+0000 7fb275917640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb270100e20 0x7fb27019a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50366/0 (socket says 172.21.15.120:50366) 2024-09-19T09:06:05.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.605+0000 7fb275917640 1 -- 172.21.15.120:0/3685927432 learned_addr learned my addr 172.21.15.120:0/3685927432 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:05.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.605+0000 7fb275116640 1 --2- 172.21.15.120:0/3685927432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb270102570 0x7fb27019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:05.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.605+0000 7fb276118640 1 --2- 172.21.15.120:0/3685927432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb270102ef0 0x7fb27019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:05.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.605+0000 7fb275917640 1 -- 172.21.15.120:0/3685927432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb270102570 msgr2=0x7fb27019ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:05.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.605+0000 7fb275917640 1 --2- 172.21.15.120:0/3685927432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb270102570 0x7fb27019ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.605+0000 7fb275917640 1 -- 172.21.15.120:0/3685927432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb270102ef0 msgr2=0x7fb27019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:05.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.605+0000 7fb275917640 1 --2- 172.21.15.120:0/3685927432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb270102ef0 0x7fb27019eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.605+0000 7fb275917640 1 -- 172.21.15.120:0/3685927432 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27019f6d0 con 0x7fb270100e20 2024-09-19T09:06:05.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.605+0000 7fb276118640 1 --2- 172.21.15.120:0/3685927432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb270102ef0 0x7fb27019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:06:05.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.605+0000 7fb275116640 1 --2- 172.21.15.120:0/3685927432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb270102570 0x7fb27019ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:05.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.606+0000 7fb275917640 1 --2- 172.21.15.120:0/3685927432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb270100e20 0x7fb27019a720 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fb26c006010 tx=0x7fb26c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:05.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.606+0000 7fb266ffd640 1 -- 172.21.15.120:0/3685927432 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26c03f070 con 0x7fb270100e20 2024-09-19T09:06:05.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.606+0000 7fb277b7b640 1 -- 172.21.15.120:0/3685927432 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb27019f960 con 0x7fb270100e20 2024-09-19T09:06:05.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.606+0000 7fb266ffd640 1 -- 172.21.15.120:0/3685927432 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb26c031c30 con 0x7fb270100e20 2024-09-19T09:06:05.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.606+0000 7fb266ffd640 1 -- 172.21.15.120:0/3685927432 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26c035030 con 0x7fb270100e20 2024-09-19T09:06:05.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.606+0000 7fb277b7b640 1 -- 172.21.15.120:0/3685927432 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2701a72a0 con 0x7fb270100e20 2024-09-19T09:06:05.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.609+0000 7fb266ffd640 1 -- 172.21.15.120:0/3685927432 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fb26c042070 con 0x7fb270100e20 2024-09-19T09:06:05.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.610+0000 7fb279f54740 1 Processor -- start 2024-09-19T09:06:05.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.611+0000 7fb279f54740 1 -- start start 2024-09-19T09:06:05.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.611+0000 7fb279f54740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c63a272b40 0x55c63a271090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:05.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.611+0000 7fb279f54740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c63a2715d0 0x55c63a271a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:05.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.611+0000 7fb279f54740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c63a27c370 0x55c63a27e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:05.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.611+0000 7fb279f54740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c63a1d72f0 con 0x55c63a27c370 2024-09-19T09:06:05.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.611+0000 7fb279f54740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c63a17ecf0 con 0x55c63a2715d0 2024-09-19T09:06:05.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.611+0000 7fb279f54740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c63a1d32b0 con 0x55c63a272b40 2024-09-19T09:06:05.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.611+0000 7fb266ffd640 1 --2- 172.21.15.120:0/3685927432 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb25c0a3240 0x7fb25c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:05.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.611+0000 7fb275116640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c63a27c370 0x55c63a27e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:05.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.611+0000 7fb276118640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c63a272b40 0x55c63a271090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:05.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.611+0000 7fb276118640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c63a272b40 0x55c63a271090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:38966/0 (socket says 172.21.15.120:38966) 2024-09-19T09:06:05.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.611+0000 7fb276118640 1 -- 172.21.15.120:0/2057497330 learned_addr learned my addr 172.21.15.120:0/2057497330 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:05.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.611+0000 7fb276118640 1 --2- 172.21.15.120:0/3685927432 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb25c0a3240 0x7fb25c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:05.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.611+0000 7fb275917640 1 --2- 172.21.15.120:0/2057497330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c63a2715d0 0x55c63a271a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:05.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.612+0000 7fb276118640 1 -- 172.21.15.120:0/2057497330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c63a2715d0 msgr2=0x55c63a271a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:05.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.612+0000 7fb276118640 1 --2- 172.21.15.120:0/2057497330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c63a2715d0 0x55c63a271a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.612+0000 7fb276118640 1 -- 172.21.15.120:0/2057497330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c63a27c370 msgr2=0x55c63a27e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:05.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.612+0000 7fb276118640 1 --2- 172.21.15.120:0/2057497330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c63a27c370 0x55c63a27e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.612+0000 7fb276118640 1 -- 172.21.15.120:0/2057497330 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c63a1633a0 con 0x55c63a272b40 2024-09-19T09:06:05.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.612+0000 7fb266ffd640 1 -- 172.21.15.120:0/3685927432 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb26c0efc90 con 0x7fb270100e20 2024-09-19T09:06:05.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.612+0000 7fb275116640 1 --2- 172.21.15.120:0/2057497330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c63a27c370 0x55c63a27e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:05.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.612+0000 7fb276118640 1 --2- 172.21.15.120:0/2057497330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c63a272b40 0x55c63a271090 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb268014f30 tx=0x7fb268011870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d44be570345d969 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:05.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.612+0000 7fb276118640 1 --2- 172.21.15.120:0/3685927432 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb25c0a3240 0x7fb25c0a5700 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb268005c40 tx=0x7fb26801c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:05.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.612+0000 7fb25bfff640 1 -- 172.21.15.120:0/2057497330 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb268022070 con 0x55c63a272b40 2024-09-19T09:06:05.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.612+0000 7fb25bfff640 1 -- 172.21.15.120:0/2057497330 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb268003e50 con 0x55c63a272b40 2024-09-19T09:06:05.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.612+0000 7fb25bfff640 1 -- 172.21.15.120:0/2057497330 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2680046d0 con 0x55c63a272b40 2024-09-19T09:06:05.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.613+0000 7fb279f54740 1 -- 172.21.15.120:0/2057497330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c63a272b40 msgr2=0x55c63a271090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:05.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.613+0000 7fb279f54740 1 --2- 172.21.15.120:0/2057497330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c63a272b40 0x55c63a271090 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb268014f30 tx=0x7fb268011870 comp rx=0 tx=0).stop 2024-09-19T09:06:05.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.613+0000 7fb279f54740 1 -- 172.21.15.120:0/2057497330 shutdown_connections 2024-09-19T09:06:05.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.613+0000 7fb279f54740 1 --2- 172.21.15.120:0/2057497330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c63a27c370 0x55c63a27e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.613+0000 7fb279f54740 1 --2- 172.21.15.120:0/2057497330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c63a2715d0 0x55c63a271a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.613+0000 7fb279f54740 1 --2- 172.21.15.120:0/2057497330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c63a272b40 0x55c63a271090 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.613+0000 7fb279f54740 1 -- 172.21.15.120:0/2057497330 >> 172.21.15.120:0/2057497330 conn(0x55c63a23a310 msgr2=0x55c63a26dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:05.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.614+0000 7fb279f54740 1 -- 172.21.15.120:0/2057497330 shutdown_connections 2024-09-19T09:06:05.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.614+0000 7fb279f54740 1 -- 172.21.15.120:0/2057497330 wait complete. 2024-09-19T09:06:05.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.614+0000 7fb279f54740 1 -- start start 2024-09-19T09:06:05.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.614+0000 7fb279f54740 1 Processor -- start 2024-09-19T09:06:05.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.615+0000 7fb279f54740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c63a2715d0 0x55c63a27bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:05.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.615+0000 7fb279f54740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c63a272b40 0x55c63a279080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:05.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.615+0000 7fb279f54740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c63a27c370 0x55c63a279810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:05.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.615+0000 7fb275116640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c63a272b40 0x55c63a279080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:05.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.615+0000 7fb279f54740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c63a27f010 con 0x55c63a27c370 2024-09-19T09:06:05.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.615+0000 7fb279f54740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c63a192060 con 0x55c63a2715d0 2024-09-19T09:06:05.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.615+0000 7fb279f54740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c63a27f190 con 0x55c63a272b40 2024-09-19T09:06:05.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.615+0000 7fb275116640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c63a272b40 0x55c63a279080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:38972/0 (socket says 172.21.15.120:38972) 2024-09-19T09:06:05.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.615+0000 7fb275116640 1 -- 172.21.15.120:0/3518463109 learned_addr learned my addr 172.21.15.120:0/3518463109 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:05.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.615+0000 7fb279f54740 10 client.0 _ll_register_callbacks cb 0x55c639ee58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:05.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.615+0000 7fb275917640 1 --2- 172.21.15.120:0/3518463109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c63a2715d0 0x55c63a27bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:05.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.615+0000 7fb276118640 1 --2- 172.21.15.120:0/3518463109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c63a27c370 0x55c63a279810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:05.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.615+0000 7fb275116640 1 -- 172.21.15.120:0/3518463109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c63a2715d0 msgr2=0x55c63a27bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:05.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.615+0000 7fb275116640 1 --2- 172.21.15.120:0/3518463109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c63a2715d0 0x55c63a27bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.615+0000 7fb275116640 1 -- 172.21.15.120:0/3518463109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c63a27c370 msgr2=0x55c63a279810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:05.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.615+0000 7fb275116640 1 --2- 172.21.15.120:0/3518463109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c63a27c370 0x55c63a279810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.615+0000 7fb275116640 1 -- 172.21.15.120:0/3518463109 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c639f483c0 con 0x55c63a272b40 2024-09-19T09:06:05.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.615+0000 7fb276118640 1 --2- 172.21.15.120:0/3518463109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c63a27c370 0x55c63a279810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:06:05.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.615+0000 7fb275116640 1 --2- 172.21.15.120:0/3518463109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c63a272b40 0x55c63a279080 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb260012b10 tx=0x7fb26000f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16a61e790d9d12dc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:05.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.615+0000 7fb2597fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:06:05.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.616+0000 7fb2597fa640 1 -- 172.21.15.120:0/3518463109 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26000fb40 con 0x55c63a272b40 2024-09-19T09:06:05.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.616+0000 7fb2597fa640 1 -- 172.21.15.120:0/3518463109 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb26000fce0 con 0x55c63a272b40 2024-09-19T09:06:05.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.616+0000 7fb279f54740 1 -- 172.21.15.120:0/3518463109 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb268003e50 con 0x55c63a272b40 2024-09-19T09:06:05.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.616+0000 7fb2597fa640 1 -- 172.21.15.120:0/3518463109 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb260004050 con 0x55c63a272b40 2024-09-19T09:06:05.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.616+0000 7fb279f54740 10 client.25954 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:05.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.616+0000 7fb279f54740 1 -- 172.21.15.120:0/3518463109 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c63a28ab70 con 0x55c63a272b40 2024-09-19T09:06:05.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.616+0000 7fb2597fa640 1 -- 172.21.15.120:0/3518463109 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb268003e50 con 0x55c63a272b40 2024-09-19T09:06:05.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.616+0000 7fb2597fa640 1 -- 172.21.15.120:0/3518463109 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb260010430 con 0x55c63a272b40 2024-09-19T09:06:05.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.616+0000 7fb279f54740 10 client.25954 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:05.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.616+0000 7fb279f54740 10 client.25954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:06:05.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.616+0000 7fb279f54740 20 client.25954 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:05.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.616+0000 7fb279f54740 1 --2- 172.21.15.120:0/3518463109 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c63a28c080 0x55c63a28e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:05.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.616+0000 7fb279f54740 4 client.25954 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:06:05.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.616+0000 7fb279f54740 1 -- 172.21.15.120:0/3518463109 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c63a1e4860 con 0x55c63a28c080 2024-09-19T09:06:05.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.616+0000 7fb275917640 1 --2- 172.21.15.120:0/3518463109 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c63a28c080 0x55c63a28e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:05.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.617+0000 7fb2597fa640 10 client.25954 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:05.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.617+0000 7fb275917640 1 --2- 172.21.15.120:0/3518463109 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c63a28c080 0x55c63a28e4c0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1efda54ac12243a server_cookie=54c12cba59bbbaf2 in_seq=0 out_seq=0 2024-09-19T09:06:05.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.618+0000 7fb2597fa640 1 -- 172.21.15.120:0/3518463109 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c63a1e4860 con 0x55c63a28c080 2024-09-19T09:06:05.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.618+0000 7fb2597fa640 10 client.25954 handle_command_reply: tid=0 2024-09-19T09:06:05.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.618+0000 7fb279f54740 1 client.25954 shutdown 2024-09-19T09:06:05.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.618+0000 7fb279f54740 1 -- 172.21.15.120:0/3518463109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c63a272b40 msgr2=0x55c63a279080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:05.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.618+0000 7fb279f54740 1 --2- 172.21.15.120:0/3518463109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c63a272b40 0x55c63a279080 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb260012b10 tx=0x7fb26000f3c0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.618+0000 7fb279f54740 1 -- 172.21.15.120:0/3518463109 shutdown_connections 2024-09-19T09:06:05.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.618+0000 7fb279f54740 1 --2- 172.21.15.120:0/3518463109 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c63a28c080 0x55c63a28e4c0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.618+0000 7fb279f54740 1 --2- 172.21.15.120:0/3518463109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c63a27c370 0x55c63a279810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.618+0000 7fb279f54740 1 --2- 172.21.15.120:0/3518463109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c63a272b40 0x55c63a279080 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.618+0000 7fb279f54740 1 --2- 172.21.15.120:0/3518463109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c63a2715d0 0x55c63a27bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.618+0000 7fb279f54740 1 -- 172.21.15.120:0/3518463109 >> 172.21.15.120:0/3518463109 conn(0x55c63a23a310 msgr2=0x55c63a273960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:05.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.618+0000 7fb2597fa640 0 client.25954 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:05.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.618+0000 7fb279f54740 1 -- 172.21.15.120:0/3518463109 shutdown_connections 2024-09-19T09:06:05.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.618+0000 7fb279f54740 1 -- 172.21.15.120:0/3518463109 wait complete. 2024-09-19T09:06:05.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.619+0000 7fb279f54740 20 client.25954 trim_cache size 0 max 16384 2024-09-19T09:06:05.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.632+0000 7fb279f54740 1 Processor -- start 2024-09-19T09:06:05.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.632+0000 7fb279f54740 1 -- start start 2024-09-19T09:06:05.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.633+0000 7fb279f54740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c63a2715d0 0x55c63a272300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:05.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.633+0000 7fb279f54740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c63a272b40 0x55c63a293d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:05.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.633+0000 7fb279f54740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c63a27c370 0x55c63a2942d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:05.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.633+0000 7fb279f54740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb2701af770 con 0x55c63a272b40 2024-09-19T09:06:05.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.633+0000 7fb279f54740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb2701b12b0 con 0x55c63a2715d0 2024-09-19T09:06:05.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.633+0000 7fb279f54740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb2701afe40 con 0x55c63a27c370 2024-09-19T09:06:05.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.633+0000 7fb275116640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c63a272b40 0x55c63a293d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:05.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.633+0000 7fb275116640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c63a272b40 0x55c63a293d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50394/0 (socket says 172.21.15.120:50394) 2024-09-19T09:06:05.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.633+0000 7fb275917640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c63a2715d0 0x55c63a272300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:05.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.633+0000 7fb276118640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c63a27c370 0x55c63a2942d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:05.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.633+0000 7fb275116640 1 -- 172.21.15.120:0/1804889893 learned_addr learned my addr 172.21.15.120:0/1804889893 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:05.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.633+0000 7fb275116640 1 -- 172.21.15.120:0/1804889893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c63a27c370 msgr2=0x55c63a2942d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:05.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.633+0000 7fb275116640 1 --2- 172.21.15.120:0/1804889893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c63a27c370 0x55c63a2942d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.633+0000 7fb275116640 1 -- 172.21.15.120:0/1804889893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c63a2715d0 msgr2=0x55c63a272300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:05.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.633+0000 7fb275116640 1 --2- 172.21.15.120:0/1804889893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c63a2715d0 0x55c63a272300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.633+0000 7fb275116640 1 -- 172.21.15.120:0/1804889893 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c63a2775d0 con 0x55c63a272b40 2024-09-19T09:06:05.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.633+0000 7fb275917640 1 --2- 172.21.15.120:0/1804889893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c63a2715d0 0x55c63a272300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:05.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.633+0000 7fb276118640 1 --2- 172.21.15.120:0/1804889893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c63a27c370 0x55c63a2942d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:05.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.633+0000 7fb275116640 1 --2- 172.21.15.120:0/1804889893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c63a272b40 0x55c63a293d90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fb260014000 tx=0x7fb260012990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed89d30f30db7520 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:05.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.634+0000 7fb25b7fe640 1 -- 172.21.15.120:0/1804889893 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26002b070 con 0x55c63a272b40 2024-09-19T09:06:05.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.634+0000 7fb25b7fe640 1 -- 172.21.15.120:0/1804889893 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb260010b40 con 0x55c63a272b40 2024-09-19T09:06:05.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.634+0000 7fb25b7fe640 1 -- 172.21.15.120:0/1804889893 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2600035f0 con 0x55c63a272b40 2024-09-19T09:06:05.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.634+0000 7fb279f54740 1 -- 172.21.15.120:0/1804889893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c63a272b40 msgr2=0x55c63a293d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:05.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.634+0000 7fb279f54740 1 --2- 172.21.15.120:0/1804889893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c63a272b40 0x55c63a293d90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fb260014000 tx=0x7fb260012990 comp rx=0 tx=0).stop 2024-09-19T09:06:05.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.634+0000 7fb279f54740 1 -- 172.21.15.120:0/1804889893 shutdown_connections 2024-09-19T09:06:05.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.634+0000 7fb279f54740 1 --2- 172.21.15.120:0/1804889893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c63a27c370 0x55c63a2942d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.634+0000 7fb279f54740 1 --2- 172.21.15.120:0/1804889893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c63a272b40 0x55c63a293d90 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.634+0000 7fb279f54740 1 --2- 172.21.15.120:0/1804889893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c63a2715d0 0x55c63a272300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.634+0000 7fb279f54740 1 -- 172.21.15.120:0/1804889893 >> 172.21.15.120:0/1804889893 conn(0x55c63a23a310 msgr2=0x55c63a268a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:05.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.635+0000 7fb279f54740 1 -- 172.21.15.120:0/1804889893 shutdown_connections 2024-09-19T09:06:05.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.635+0000 7fb279f54740 1 -- 172.21.15.120:0/1804889893 wait complete. 2024-09-19T09:06:05.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.635+0000 7fb279f54740 1 -- start start 2024-09-19T09:06:05.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.635+0000 7fb279f54740 1 Processor -- start 2024-09-19T09:06:05.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.636+0000 7fb279f54740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c63a2715d0 0x55c63a2ef050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:05.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.636+0000 7fb279f54740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c63a272b40 0x55c63a2ef590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:05.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.636+0000 7fb279f54740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c63a27c370 0x55c63a27f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:05.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.636+0000 7fb279f54740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb2701b0be0 con 0x55c63a27c370 2024-09-19T09:06:05.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.636+0000 7fb279f54740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb2701b0510 con 0x55c63a2715d0 2024-09-19T09:06:05.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.636+0000 7fb279f54740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb2701ad730 con 0x55c63a272b40 2024-09-19T09:06:05.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.636+0000 7fb279f54740 10 client.0 _ll_register_callbacks cb 0x7fb260009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:05.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.636+0000 7fb275917640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c63a2715d0 0x55c63a2ef050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:05.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.636+0000 7fb276118640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c63a27c370 0x55c63a27f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:05.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.636+0000 7fb275917640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c63a2715d0 0x55c63a2ef050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37928/0 (socket says 172.21.15.120:37928) 2024-09-19T09:06:05.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.636+0000 7fb275917640 1 -- 172.21.15.120:0/2719393103 learned_addr learned my addr 172.21.15.120:0/2719393103 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:05.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.636+0000 7fb275116640 1 --2- 172.21.15.120:0/2719393103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c63a272b40 0x55c63a2ef590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:05.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.636+0000 7fb275917640 1 -- 172.21.15.120:0/2719393103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c63a272b40 msgr2=0x55c63a2ef590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:05.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.636+0000 7fb275917640 1 --2- 172.21.15.120:0/2719393103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c63a272b40 0x55c63a2ef590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.636+0000 7fb275917640 1 -- 172.21.15.120:0/2719393103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c63a27c370 msgr2=0x55c63a27f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:05.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.636+0000 7fb275917640 1 --2- 172.21.15.120:0/2719393103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c63a27c370 0x55c63a27f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.636+0000 7fb275917640 1 -- 172.21.15.120:0/2719393103 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c63a280090 con 0x55c63a2715d0 2024-09-19T09:06:05.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.636+0000 7fb275116640 1 --2- 172.21.15.120:0/2719393103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c63a272b40 0x55c63a2ef590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:05.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.636+0000 7fb276118640 1 --2- 172.21.15.120:0/2719393103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c63a27c370 0x55c63a27f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:05.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.636+0000 7fb275917640 1 --2- 172.21.15.120:0/2719393103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c63a2715d0 0x55c63a2ef050 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb26c0ed790 tx=0x7fb26c0dc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fea378e942244754 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:05.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.636+0000 7fb2597fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:06:05.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.636+0000 7fb2597fa640 1 -- 172.21.15.120:0/2719393103 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26c049040 con 0x55c63a2715d0 2024-09-19T09:06:05.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.636+0000 7fb279f54740 1 -- 172.21.15.120:0/2719393103 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb260010b40 con 0x55c63a2715d0 2024-09-19T09:06:05.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.636+0000 7fb2597fa640 1 -- 172.21.15.120:0/2719393103 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb26c03b460 con 0x55c63a2715d0 2024-09-19T09:06:05.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.637+0000 7fb2597fa640 1 -- 172.21.15.120:0/2719393103 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb26c0bc2b0 con 0x55c63a2715d0 2024-09-19T09:06:05.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.637+0000 7fb279f54740 10 client.26000 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:05.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.637+0000 7fb279f54740 1 -- 172.21.15.120:0/2719393103 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c63a2f6880 con 0x55c63a2715d0 2024-09-19T09:06:05.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.637+0000 7fb2597fa640 1 -- 172.21.15.120:0/2719393103 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb260010b40 con 0x55c63a2715d0 2024-09-19T09:06:05.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.637+0000 7fb2597fa640 1 -- 172.21.15.120:0/2719393103 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb26c04c060 con 0x55c63a2715d0 2024-09-19T09:06:05.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.637+0000 7fb279f54740 10 client.26000 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:05.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.637+0000 7fb279f54740 10 client.26000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:06:05.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.637+0000 7fb279f54740 20 client.26000 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:05.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.637+0000 7fb279f54740 1 --2- 172.21.15.120:0/2719393103 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c63a2f6fa0 0x55c63a2f9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:05.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.637+0000 7fb279f54740 4 client.26000 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:06:05.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.637+0000 7fb279f54740 1 -- 172.21.15.120:0/2719393103 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c63a1ded70 con 0x55c63a2f6fa0 2024-09-19T09:06:05.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.637+0000 7fb275116640 1 --2- 172.21.15.120:0/2719393103 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c63a2f6fa0 0x55c63a2f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:05.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.638+0000 7fb275116640 1 --2- 172.21.15.120:0/2719393103 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c63a2f6fa0 0x55c63a2f9390 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cc34bc7c03b9ef8 server_cookie=c9791f73381e4f7 in_seq=0 out_seq=0 2024-09-19T09:06:05.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.638+0000 7fb2597fa640 10 client.26000 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:05.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.639+0000 7fb2597fa640 1 -- 172.21.15.120:0/2719393103 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c63a1ded70 con 0x55c63a2f6fa0 2024-09-19T09:06:05.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.639+0000 7fb2597fa640 10 client.26000 handle_command_reply: tid=0 2024-09-19T09:06:05.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.639+0000 7fb279f54740 1 client.26000 shutdown 2024-09-19T09:06:05.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.639+0000 7fb279f54740 1 -- 172.21.15.120:0/2719393103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c63a2715d0 msgr2=0x55c63a2ef050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:05.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.639+0000 7fb279f54740 1 --2- 172.21.15.120:0/2719393103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c63a2715d0 0x55c63a2ef050 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb26c0ed790 tx=0x7fb26c0dc040 comp rx=0 tx=0).stop 2024-09-19T09:06:05.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.639+0000 7fb279f54740 1 -- 172.21.15.120:0/2719393103 shutdown_connections 2024-09-19T09:06:05.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.639+0000 7fb279f54740 1 --2- 172.21.15.120:0/2719393103 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c63a2f6fa0 0x55c63a2f9390 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.639+0000 7fb2597fa640 0 client.26000 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:05.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.639+0000 7fb279f54740 1 --2- 172.21.15.120:0/2719393103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c63a27c370 0x55c63a27f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.639+0000 7fb279f54740 1 --2- 172.21.15.120:0/2719393103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c63a272b40 0x55c63a2ef590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.639+0000 7fb279f54740 1 --2- 172.21.15.120:0/2719393103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c63a2715d0 0x55c63a2ef050 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.639+0000 7fb279f54740 1 -- 172.21.15.120:0/2719393103 >> 172.21.15.120:0/2719393103 conn(0x55c63a23a310 msgr2=0x55c63a268a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:05.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.639+0000 7fb279f54740 1 -- 172.21.15.120:0/2719393103 shutdown_connections 2024-09-19T09:06:05.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.639+0000 7fb279f54740 1 -- 172.21.15.120:0/2719393103 wait complete. 2024-09-19T09:06:05.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.639+0000 7fb279f54740 20 client.26000 trim_cache size 0 max 16384 2024-09-19T09:06:05.647 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:06:05.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.640+0000 7fb277b7b640 1 -- 172.21.15.120:0/3685927432 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb25c0a3240 msgr2=0x7fb25c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:05.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.640+0000 7fb277b7b640 1 --2- 172.21.15.120:0/3685927432 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb25c0a3240 0x7fb25c0a5700 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb268005c40 tx=0x7fb26801c000 comp rx=0 tx=0).stop 2024-09-19T09:06:05.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.640+0000 7fb277b7b640 1 -- 172.21.15.120:0/3685927432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb270100e20 msgr2=0x7fb27019a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:05.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.640+0000 7fb277b7b640 1 --2- 172.21.15.120:0/3685927432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb270100e20 0x7fb27019a720 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fb26c006010 tx=0x7fb26c003c10 comp rx=0 tx=0).stop 2024-09-19T09:06:05.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.640+0000 7fb277b7b640 1 -- 172.21.15.120:0/3685927432 shutdown_connections 2024-09-19T09:06:05.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.640+0000 7fb277b7b640 1 --2- 172.21.15.120:0/3685927432 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb25c0a3240 0x7fb25c0a5700 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.640+0000 7fb277b7b640 1 --2- 172.21.15.120:0/3685927432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb270102ef0 0x7fb27019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.640+0000 7fb277b7b640 1 --2- 172.21.15.120:0/3685927432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb270102570 0x7fb27019ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.640+0000 7fb277b7b640 1 --2- 172.21.15.120:0/3685927432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb270100e20 0x7fb27019a720 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:05.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.640+0000 7fb277b7b640 1 -- 172.21.15.120:0/3685927432 >> 172.21.15.120:0/3685927432 conn(0x7fb2700fc7d0 msgr2=0x7fb2700fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:05.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.640+0000 7fb277b7b640 1 -- 172.21.15.120:0/3685927432 shutdown_connections 2024-09-19T09:06:05.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:05.640+0000 7fb277b7b640 1 -- 172.21.15.120:0/3685927432 wait complete. 2024-09-19T09:06:06.041 INFO:tasks.workunit.client.0.smithi120.stdout: 1 230565 17.72 MB/sec execute 1 sec latency 44.988 ms 2024-09-19T09:06:07.041 INFO:tasks.workunit.client.0.smithi120.stdout: 1 233007 13.22 MB/sec execute 2 sec latency 14.216 ms 2024-09-19T09:06:07.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:06 smithi120 ceph-mon[31150]: pgmap v438: 97 pgs: 97 active+clean; 46 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 417 op/s 2024-09-19T09:06:07.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:06 smithi196 ceph-mon[35150]: pgmap v438: 97 pgs: 97 active+clean; 46 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 417 op/s 2024-09-19T09:06:07.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:06 smithi153 ceph-mon[35071]: pgmap v438: 97 pgs: 97 active+clean; 46 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 417 op/s 2024-09-19T09:06:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.423+0000 7faad49fe640 1 -- 172.21.15.120:0/1811924708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faad0100840 msgr2=0x7faad0100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.423+0000 7faad49fe640 1 --2- 172.21.15.120:0/1811924708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faad0100840 0x7faad0100c40 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7faac0009900 tx=0x7faac002a410 comp rx=0 tx=0).stop 2024-09-19T09:06:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.423+0000 7faad49fe640 1 -- 172.21.15.120:0/1811924708 shutdown_connections 2024-09-19T09:06:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.423+0000 7faad49fe640 1 --2- 172.21.15.120:0/1811924708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faad01023f0 0x7faad0108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.423+0000 7faad49fe640 1 --2- 172.21.15.120:0/1811924708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faad0101a30 0x7faad0101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.423+0000 7faad49fe640 1 --2- 172.21.15.120:0/1811924708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faad0100840 0x7faad0100c40 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.423+0000 7faad49fe640 1 -- 172.21.15.120:0/1811924708 >> 172.21.15.120:0/1811924708 conn(0x7faad00fbfa0 msgr2=0x7faad00fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.423+0000 7faad49fe640 1 -- 172.21.15.120:0/1811924708 shutdown_connections 2024-09-19T09:06:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.423+0000 7faad49fe640 1 -- 172.21.15.120:0/1811924708 wait complete. 2024-09-19T09:06:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.423+0000 7faad49fe640 1 Processor -- start 2024-09-19T09:06:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.424+0000 7faad49fe640 1 -- start start 2024-09-19T09:06:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.424+0000 7faad49fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faad0100840 0x7faad0197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.424+0000 7faad49fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faad0101a30 0x7faad0198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.424+0000 7faad49fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faad01023f0 0x7faad019f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.424+0000 7faad49fe640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faad010c4c0 con 0x7faad0101a30 2024-09-19T09:06:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.424+0000 7faad49fe640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faad010c340 con 0x7faad0100840 2024-09-19T09:06:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.424+0000 7faad49fe640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faad010c640 con 0x7faad01023f0 2024-09-19T09:06:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.424+0000 7faace59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faad0101a30 0x7faad0198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.424+0000 7faacf59e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faad01023f0 0x7faad019f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.424+0000 7faaced9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faad0100840 0x7faad0197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.424+0000 7faace59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faad0101a30 0x7faad0198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50420/0 (socket says 172.21.15.120:50420) 2024-09-19T09:06:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.424+0000 7faaced9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faad0100840 0x7faad0197cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37946/0 (socket says 172.21.15.120:37946) 2024-09-19T09:06:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.424+0000 7faacf59e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faad01023f0 0x7faad019f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:39010/0 (socket says 172.21.15.120:39010) 2024-09-19T09:06:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.424+0000 7faace59c640 1 -- 172.21.15.120:0/484103540 learned_addr learned my addr 172.21.15.120:0/484103540 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.424+0000 7faace59c640 1 -- 172.21.15.120:0/484103540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faad01023f0 msgr2=0x7faad019f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.424+0000 7faace59c640 1 --2- 172.21.15.120:0/484103540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faad01023f0 0x7faad019f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.424+0000 7faace59c640 1 -- 172.21.15.120:0/484103540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faad0100840 msgr2=0x7faad0197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.424+0000 7faace59c640 1 --2- 172.21.15.120:0/484103540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faad0100840 0x7faad0197cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.424+0000 7faace59c640 1 -- 172.21.15.120:0/484103540 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad019f900 con 0x7faad0101a30 2024-09-19T09:06:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.425+0000 7faace59c640 1 --2- 172.21.15.120:0/484103540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faad0101a30 0x7faad0198200 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7faac400e9f0 tx=0x7faac400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.425+0000 7faab7fff640 1 -- 172.21.15.120:0/484103540 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac400cd80 con 0x7faad0101a30 2024-09-19T09:06:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.425+0000 7faab7fff640 1 -- 172.21.15.120:0/484103540 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faac4010910 con 0x7faad0101a30 2024-09-19T09:06:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.425+0000 7faad49fe640 1 -- 172.21.15.120:0/484103540 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faad019fbf0 con 0x7faad0101a30 2024-09-19T09:06:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.425+0000 7faab7fff640 1 -- 172.21.15.120:0/484103540 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac4018990 con 0x7faad0101a30 2024-09-19T09:06:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.425+0000 7faad49fe640 1 -- 172.21.15.120:0/484103540 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faad01a0110 con 0x7faad0101a30 2024-09-19T09:06:07.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.427+0000 7faab7fff640 1 -- 172.21.15.120:0/484103540 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7faac4010ac0 con 0x7faad0101a30 2024-09-19T09:06:07.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.427+0000 7faad806f740 1 Processor -- start 2024-09-19T09:06:07.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.427+0000 7faad806f740 1 -- start start 2024-09-19T09:06:07.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.427+0000 7faad806f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae09d4fb70 0x55ae09d4ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:07.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.427+0000 7faad806f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae09d504b0 0x55ae09d54f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:07.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.427+0000 7faad806f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae09d55540 0x55ae09d57a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:07.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.427+0000 7faad806f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ae09cb32f0 con 0x55ae09d4fb70 2024-09-19T09:06:07.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.427+0000 7faad806f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ae09c5b270 con 0x55ae09d504b0 2024-09-19T09:06:07.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.427+0000 7faad806f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ae09caf2b0 con 0x55ae09d55540 2024-09-19T09:06:07.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.428+0000 7faacf59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae09d4fb70 0x55ae09d4ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:07.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.428+0000 7faacf59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae09d4fb70 0x55ae09d4ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50422/0 (socket says 172.21.15.120:50422) 2024-09-19T09:06:07.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.428+0000 7faacf59e640 1 -- 172.21.15.120:0/1822813029 learned_addr learned my addr 172.21.15.120:0/1822813029 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:07.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.428+0000 7faace59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae09d55540 0x55ae09d57a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:07.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.428+0000 7faaced9d640 1 --2- 172.21.15.120:0/1822813029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae09d504b0 0x55ae09d54f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:07.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.428+0000 7faab7fff640 1 --2- 172.21.15.120:0/484103540 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faab80a3240 0x7faab80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:07.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.428+0000 7faacf59e640 1 --2- 172.21.15.120:0/484103540 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faab80a3240 0x7faab80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.428+0000 7faaced9d640 1 -- 172.21.15.120:0/1822813029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae09d55540 msgr2=0x55ae09d57a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.428+0000 7faab7fff640 1 -- 172.21.15.120:0/484103540 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7faac40cfc00 con 0x7faad0101a30 2024-09-19T09:06:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.428+0000 7faaced9d640 1 --2- 172.21.15.120:0/1822813029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae09d55540 0x55ae09d57a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.428+0000 7faaced9d640 1 -- 172.21.15.120:0/1822813029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae09d4fb70 msgr2=0x55ae09d4ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.428+0000 7faaced9d640 1 --2- 172.21.15.120:0/1822813029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae09d4fb70 0x55ae09d4ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.428+0000 7faaced9d640 1 -- 172.21.15.120:0/1822813029 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae09c5a340 con 0x55ae09d504b0 2024-09-19T09:06:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.428+0000 7faace59c640 1 --2- 172.21.15.120:0/1822813029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae09d55540 0x55ae09d57a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.428+0000 7faacf59e640 1 --2- 172.21.15.120:0/1822813029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae09d4fb70 0x55ae09d4ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.428+0000 7faaced9d640 1 --2- 172.21.15.120:0/1822813029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae09d504b0 0x55ae09d54f70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7faac0009230 tx=0x7faac0032c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad5b34ddc86d5f89 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.428+0000 7faacf59e640 1 --2- 172.21.15.120:0/484103540 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faab80a3240 0x7faab80a5700 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7faac8009fd0 tx=0x7faac8004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.428+0000 7faab57fa640 1 -- 172.21.15.120:0/1822813029 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac0040070 con 0x55ae09d504b0 2024-09-19T09:06:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.428+0000 7faab57fa640 1 -- 172.21.15.120:0/1822813029 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faac0032e20 con 0x55ae09d504b0 2024-09-19T09:06:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.428+0000 7faab57fa640 1 -- 172.21.15.120:0/1822813029 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac0035040 con 0x55ae09d504b0 2024-09-19T09:06:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.429+0000 7faad806f740 1 -- 172.21.15.120:0/1822813029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae09d504b0 msgr2=0x55ae09d54f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.429+0000 7faad806f740 1 --2- 172.21.15.120:0/1822813029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae09d504b0 0x55ae09d54f70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7faac0009230 tx=0x7faac0032c30 comp rx=0 tx=0).stop 2024-09-19T09:06:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.429+0000 7faad806f740 1 -- 172.21.15.120:0/1822813029 shutdown_connections 2024-09-19T09:06:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.429+0000 7faad806f740 1 --2- 172.21.15.120:0/1822813029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae09d55540 0x55ae09d57a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.429+0000 7faad806f740 1 --2- 172.21.15.120:0/1822813029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae09d504b0 0x55ae09d54f70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.429+0000 7faad806f740 1 --2- 172.21.15.120:0/1822813029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae09d4fb70 0x55ae09d4ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.429+0000 7faad806f740 1 -- 172.21.15.120:0/1822813029 >> 172.21.15.120:0/1822813029 conn(0x55ae09d4b1e0 msgr2=0x55ae09d4b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.429+0000 7faad806f740 1 -- 172.21.15.120:0/1822813029 shutdown_connections 2024-09-19T09:06:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.429+0000 7faad806f740 1 -- 172.21.15.120:0/1822813029 wait complete. 2024-09-19T09:06:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.429+0000 7faad806f740 1 -- start start 2024-09-19T09:06:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.430+0000 7faad806f740 1 Processor -- start 2024-09-19T09:06:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.430+0000 7faad806f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae09d4fb70 0x55ae09d5e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.430+0000 7faad806f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae09d504b0 0x55ae09d5e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.430+0000 7faad806f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae09d55540 0x55ae09d656a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.430+0000 7faad806f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ae09d584b0 con 0x55ae09d504b0 2024-09-19T09:06:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.430+0000 7faad806f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ae09c3c290 con 0x55ae09d55540 2024-09-19T09:06:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.430+0000 7faad806f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ae09d58630 con 0x55ae09d4fb70 2024-09-19T09:06:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.430+0000 7faad806f740 10 client.0 _ll_register_callbacks cb 0x55ae09c6d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.430+0000 7faace59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae09d504b0 0x55ae09d5e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.430+0000 7faacf59e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae09d55540 0x55ae09d656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.430+0000 7faace59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae09d504b0 0x55ae09d5e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50428/0 (socket says 172.21.15.120:50428) 2024-09-19T09:06:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.430+0000 7faace59c640 1 -- 172.21.15.120:0/247824153 learned_addr learned my addr 172.21.15.120:0/247824153 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.430+0000 7faaced9d640 1 --2- 172.21.15.120:0/247824153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae09d4fb70 0x55ae09d5e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.430+0000 7faace59c640 1 -- 172.21.15.120:0/247824153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae09d4fb70 msgr2=0x55ae09d5e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.430+0000 7faace59c640 1 --2- 172.21.15.120:0/247824153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae09d4fb70 0x55ae09d5e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.430+0000 7faace59c640 1 -- 172.21.15.120:0/247824153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae09d55540 msgr2=0x55ae09d656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.430+0000 7faace59c640 1 --2- 172.21.15.120:0/247824153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae09d55540 0x55ae09d656a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.430+0000 7faace59c640 1 -- 172.21.15.120:0/247824153 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae099c6210 con 0x55ae09d504b0 2024-09-19T09:06:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.430+0000 7faace59c640 1 --2- 172.21.15.120:0/247824153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae09d504b0 0x55ae09d5e660 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7faac400c6a0 tx=0x7faac4009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784ddbf21e102d99 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.430+0000 7faab2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:06:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.431+0000 7faab2ff5640 1 -- 172.21.15.120:0/247824153 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac40bae60 con 0x55ae09d504b0 2024-09-19T09:06:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.431+0000 7faab2ff5640 1 -- 172.21.15.120:0/247824153 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7faac40bc0b0 con 0x55ae09d504b0 2024-09-19T09:06:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.431+0000 7faad806f740 1 -- 172.21.15.120:0/247824153 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faac0032e20 con 0x55ae09d504b0 2024-09-19T09:06:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.431+0000 7faab2ff5640 1 -- 172.21.15.120:0/247824153 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faac409c2d0 con 0x55ae09d504b0 2024-09-19T09:06:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.431+0000 7faad806f740 10 client.16653 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.431+0000 7faad806f740 1 -- 172.21.15.120:0/247824153 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ae09a92620 con 0x55ae09d504b0 2024-09-19T09:06:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.431+0000 7faab2ff5640 1 -- 172.21.15.120:0/247824153 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faac0032e20 con 0x55ae09d504b0 2024-09-19T09:06:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.431+0000 7faab2ff5640 1 -- 172.21.15.120:0/247824153 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faac40cdbc0 con 0x55ae09d504b0 2024-09-19T09:06:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.431+0000 7faad806f740 10 client.16653 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.431+0000 7faad806f740 10 client.16653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:06:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.431+0000 7faad806f740 20 client.16653 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.431+0000 7faad806f740 1 --2- 172.21.15.120:0/247824153 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ae09d69d90 0x55ae09d6c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.431+0000 7faad806f740 4 client.16653 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:06:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.431+0000 7faad806f740 1 -- 172.21.15.120:0/247824153 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae09cc0a20 con 0x55ae09d69d90 2024-09-19T09:06:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.431+0000 7faaced9d640 1 --2- 172.21.15.120:0/247824153 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ae09d69d90 0x55ae09d6c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.432+0000 7faab2ff5640 10 client.16653 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.432+0000 7faaced9d640 1 --2- 172.21.15.120:0/247824153 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ae09d69d90 0x55ae09d6c180 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a787934d4d93b44a server_cookie=f3fd43f1a0fe04b7 in_seq=0 out_seq=0 2024-09-19T09:06:07.442 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:06:07.442 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:06:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.433+0000 7faab2ff5640 1 -- 172.21.15.120:0/247824153 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae09cc0a20 con 0x55ae09d69d90 2024-09-19T09:06:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.433+0000 7faab2ff5640 10 client.16653 handle_command_reply: tid=0 2024-09-19T09:06:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.433+0000 7faad806f740 1 client.16653 shutdown 2024-09-19T09:06:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.433+0000 7faad806f740 1 -- 172.21.15.120:0/247824153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae09d504b0 msgr2=0x55ae09d5e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.433+0000 7faad806f740 1 --2- 172.21.15.120:0/247824153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae09d504b0 0x55ae09d5e660 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7faac400c6a0 tx=0x7faac4009700 comp rx=0 tx=0).stop 2024-09-19T09:06:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.433+0000 7faad806f740 1 -- 172.21.15.120:0/247824153 shutdown_connections 2024-09-19T09:06:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.433+0000 7faad806f740 1 --2- 172.21.15.120:0/247824153 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ae09d69d90 0x55ae09d6c180 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.433+0000 7faad806f740 1 --2- 172.21.15.120:0/247824153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae09d55540 0x55ae09d656a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.433+0000 7faab2ff5640 0 client.16653 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.433+0000 7faad806f740 1 --2- 172.21.15.120:0/247824153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae09d504b0 0x55ae09d5e660 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.433+0000 7faad806f740 1 --2- 172.21.15.120:0/247824153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae09d4fb70 0x55ae09d5e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.433+0000 7faad806f740 1 -- 172.21.15.120:0/247824153 >> 172.21.15.120:0/247824153 conn(0x55ae09d4b1e0 msgr2=0x55ae09d4c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.433+0000 7faad806f740 1 -- 172.21.15.120:0/247824153 shutdown_connections 2024-09-19T09:06:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.433+0000 7faad806f740 1 -- 172.21.15.120:0/247824153 wait complete. 2024-09-19T09:06:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.434+0000 7faad806f740 20 client.16653 trim_cache size 0 max 16384 2024-09-19T09:06:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.449+0000 7faad806f740 1 Processor -- start 2024-09-19T09:06:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.449+0000 7faad806f740 1 -- start start 2024-09-19T09:06:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.449+0000 7faad806f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae09d4fb70 0x55ae09d4c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.449+0000 7faad806f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae09d504b0 0x55ae09d4cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.449+0000 7faad806f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae09d55540 0x55ae09d65660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.449+0000 7faad806f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faad01a6c60 con 0x55ae09d4fb70 2024-09-19T09:06:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.449+0000 7faad806f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faad01a87a0 con 0x55ae09d504b0 2024-09-19T09:06:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.449+0000 7faad806f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faad01a7330 con 0x55ae09d55540 2024-09-19T09:06:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.449+0000 7faace59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae09d504b0 0x55ae09d4cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.449+0000 7faaced9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae09d4fb70 0x55ae09d4c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.449+0000 7faace59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae09d504b0 0x55ae09d4cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37974/0 (socket says 172.21.15.120:37974) 2024-09-19T09:06:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.449+0000 7faaced9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae09d4fb70 0x55ae09d4c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50444/0 (socket says 172.21.15.120:50444) 2024-09-19T09:06:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.449+0000 7faace59c640 1 -- 172.21.15.120:0/1899014201 learned_addr learned my addr 172.21.15.120:0/1899014201 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.450+0000 7faacf59e640 1 --2- 172.21.15.120:0/1899014201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae09d55540 0x55ae09d65660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.450+0000 7faace59c640 1 -- 172.21.15.120:0/1899014201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae09d55540 msgr2=0x55ae09d65660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.450+0000 7faace59c640 1 --2- 172.21.15.120:0/1899014201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae09d55540 0x55ae09d65660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.450+0000 7faace59c640 1 -- 172.21.15.120:0/1899014201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae09d4fb70 msgr2=0x55ae09d4c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.450+0000 7faace59c640 1 --2- 172.21.15.120:0/1899014201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae09d4fb70 0x55ae09d4c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.450+0000 7faace59c640 1 -- 172.21.15.120:0/1899014201 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae09d4b000 con 0x55ae09d504b0 2024-09-19T09:06:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.450+0000 7faaced9d640 1 --2- 172.21.15.120:0/1899014201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae09d4fb70 0x55ae09d4c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.450+0000 7faace59c640 1 --2- 172.21.15.120:0/1899014201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae09d504b0 0x55ae09d4cce0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7faac40a4960 tx=0x7faac4018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3485685373142e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.450+0000 7faab4ff9640 1 -- 172.21.15.120:0/1899014201 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac40bae60 con 0x55ae09d504b0 2024-09-19T09:06:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.450+0000 7faab4ff9640 1 -- 172.21.15.120:0/1899014201 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faac40a4e90 con 0x55ae09d504b0 2024-09-19T09:06:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.450+0000 7faab4ff9640 1 -- 172.21.15.120:0/1899014201 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac40a03a0 con 0x55ae09d504b0 2024-09-19T09:06:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.451+0000 7faad806f740 1 -- 172.21.15.120:0/1899014201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae09d504b0 msgr2=0x55ae09d4cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.451+0000 7faad806f740 1 --2- 172.21.15.120:0/1899014201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae09d504b0 0x55ae09d4cce0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7faac40a4960 tx=0x7faac4018420 comp rx=0 tx=0).stop 2024-09-19T09:06:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.451+0000 7faad806f740 1 -- 172.21.15.120:0/1899014201 shutdown_connections 2024-09-19T09:06:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.451+0000 7faad806f740 1 --2- 172.21.15.120:0/1899014201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae09d55540 0x55ae09d65660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.451+0000 7faad806f740 1 --2- 172.21.15.120:0/1899014201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae09d504b0 0x55ae09d4cce0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.451+0000 7faad806f740 1 --2- 172.21.15.120:0/1899014201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae09d4fb70 0x55ae09d4c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.451+0000 7faad806f740 1 -- 172.21.15.120:0/1899014201 >> 172.21.15.120:0/1899014201 conn(0x55ae09d4b1e0 msgr2=0x55ae09d4c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.451+0000 7faad806f740 1 -- 172.21.15.120:0/1899014201 shutdown_connections 2024-09-19T09:06:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.451+0000 7faad806f740 1 -- 172.21.15.120:0/1899014201 wait complete. 2024-09-19T09:06:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.451+0000 7faad806f740 1 -- start start 2024-09-19T09:06:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.451+0000 7faad806f740 1 Processor -- start 2024-09-19T09:06:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.451+0000 7faad806f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae09d4fb70 0x55ae09d62a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.451+0000 7faad806f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae09d504b0 0x55ae09d62f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.451+0000 7faad806f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae09d55540 0x55ae09d63f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.451+0000 7faad806f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faad01a80d0 con 0x55ae09d55540 2024-09-19T09:06:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.451+0000 7faad806f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faad01a7a00 con 0x55ae09d504b0 2024-09-19T09:06:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.451+0000 7faad806f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faad01a4c20 con 0x55ae09d4fb70 2024-09-19T09:06:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.452+0000 7faad806f740 10 client.0 _ll_register_callbacks cb 0x7faac400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.452+0000 7faacf59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae09d55540 0x55ae09d63f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.452+0000 7faace59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae09d504b0 0x55ae09d62f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.452+0000 7faace59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae09d504b0 0x55ae09d62f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37978/0 (socket says 172.21.15.120:37978) 2024-09-19T09:06:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.452+0000 7faaced9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae09d4fb70 0x55ae09d62a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.452+0000 7faace59c640 1 -- 172.21.15.120:0/3529379443 learned_addr learned my addr 172.21.15.120:0/3529379443 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.452+0000 7faace59c640 1 -- 172.21.15.120:0/3529379443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae09d4fb70 msgr2=0x55ae09d62a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.452+0000 7faace59c640 1 --2- 172.21.15.120:0/3529379443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae09d4fb70 0x55ae09d62a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.452+0000 7faace59c640 1 -- 172.21.15.120:0/3529379443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae09d55540 msgr2=0x55ae09d63f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.452+0000 7faace59c640 1 --2- 172.21.15.120:0/3529379443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae09d55540 0x55ae09d63f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.452+0000 7faace59c640 1 -- 172.21.15.120:0/3529379443 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae09d645a0 con 0x55ae09d504b0 2024-09-19T09:06:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.452+0000 7faacf59e640 1 --2- 172.21.15.120:0/3529379443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae09d55540 0x55ae09d63f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.452+0000 7faace59c640 1 --2- 172.21.15.120:0/3529379443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae09d504b0 0x55ae09d62f80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7faac409a580 tx=0x7faac40a06c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3523fba0c62ded3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.452+0000 7faab2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:06:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.452+0000 7faab2ff5640 1 -- 172.21.15.120:0/3529379443 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac40bae60 con 0x55ae09d504b0 2024-09-19T09:06:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.452+0000 7faad806f740 1 -- 172.21.15.120:0/3529379443 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faac40a4e90 con 0x55ae09d504b0 2024-09-19T09:06:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.452+0000 7faab2ff5640 1 -- 172.21.15.120:0/3529379443 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7faac40a0360 con 0x55ae09d504b0 2024-09-19T09:06:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.452+0000 7faab2ff5640 1 -- 172.21.15.120:0/3529379443 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faac4010430 con 0x55ae09d504b0 2024-09-19T09:06:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.452+0000 7faad806f740 10 client.26018 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.452+0000 7faad806f740 1 -- 172.21.15.120:0/3529379443 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ae09d658b0 con 0x55ae09d504b0 2024-09-19T09:06:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.453+0000 7faab2ff5640 1 -- 172.21.15.120:0/3529379443 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faac40a4e90 con 0x55ae09d504b0 2024-09-19T09:06:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.453+0000 7faab2ff5640 1 -- 172.21.15.120:0/3529379443 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faac40a1760 con 0x55ae09d504b0 2024-09-19T09:06:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.453+0000 7faad806f740 10 client.26018 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.453+0000 7faad806f740 10 client.26018 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:06:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.453+0000 7faad806f740 20 client.26018 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.453+0000 7faad806f740 1 --2- 172.21.15.120:0/3529379443 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ae09dd4ef0 0x55ae09dd7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.453+0000 7faad806f740 4 client.26018 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:06:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.453+0000 7faad806f740 1 -- 172.21.15.120:0/3529379443 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ae09cbad70 con 0x55ae09dd4ef0 2024-09-19T09:06:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.453+0000 7faaced9d640 1 --2- 172.21.15.120:0/3529379443 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ae09dd4ef0 0x55ae09dd7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.453+0000 7faab2ff5640 10 client.26018 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.453+0000 7faaced9d640 1 --2- 172.21.15.120:0/3529379443 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ae09dd4ef0 0x55ae09dd7310 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a22d097dece2aa51 server_cookie=927665456c0e385e in_seq=0 out_seq=0 2024-09-19T09:06:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.587+0000 7faab2ff5640 1 -- 172.21.15.120:0/3529379443 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ae09cbad70 con 0x55ae09dd4ef0 2024-09-19T09:06:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.588+0000 7faab2ff5640 10 client.26018 handle_command_reply: tid=0 2024-09-19T09:06:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.588+0000 7faad806f740 1 client.26018 shutdown 2024-09-19T09:06:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.588+0000 7faad806f740 1 -- 172.21.15.120:0/3529379443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae09d504b0 msgr2=0x55ae09d62f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.588+0000 7faad806f740 1 --2- 172.21.15.120:0/3529379443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae09d504b0 0x55ae09d62f80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7faac409a580 tx=0x7faac40a06c0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.588+0000 7faad806f740 1 -- 172.21.15.120:0/3529379443 shutdown_connections 2024-09-19T09:06:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.588+0000 7faad806f740 1 --2- 172.21.15.120:0/3529379443 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ae09dd4ef0 0x55ae09dd7310 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.588+0000 7faad806f740 1 --2- 172.21.15.120:0/3529379443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae09d55540 0x55ae09d63f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.588+0000 7faab2ff5640 0 client.26018 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.588+0000 7faad806f740 1 --2- 172.21.15.120:0/3529379443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae09d504b0 0x55ae09d62f80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.588+0000 7faad806f740 1 --2- 172.21.15.120:0/3529379443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae09d4fb70 0x55ae09d62a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.588+0000 7faad806f740 1 -- 172.21.15.120:0/3529379443 >> 172.21.15.120:0/3529379443 conn(0x55ae09d4b1e0 msgr2=0x55ae09d4c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.588+0000 7faad806f740 1 -- 172.21.15.120:0/3529379443 shutdown_connections 2024-09-19T09:06:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.588+0000 7faad806f740 1 -- 172.21.15.120:0/3529379443 wait complete. 2024-09-19T09:06:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.588+0000 7faad806f740 20 client.26018 trim_cache size 0 max 16384 2024-09-19T09:06:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:06:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:06:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:06:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:06:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.589+0000 7faad49fe640 1 -- 172.21.15.120:0/484103540 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faab80a3240 msgr2=0x7faab80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.589+0000 7faad49fe640 1 --2- 172.21.15.120:0/484103540 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faab80a3240 0x7faab80a5700 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7faac8009fd0 tx=0x7faac8004ab0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.589+0000 7faad49fe640 1 -- 172.21.15.120:0/484103540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faad0101a30 msgr2=0x7faad0198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.589+0000 7faad49fe640 1 --2- 172.21.15.120:0/484103540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faad0101a30 0x7faad0198200 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7faac400e9f0 tx=0x7faac400eeb0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.589+0000 7faad49fe640 1 -- 172.21.15.120:0/484103540 shutdown_connections 2024-09-19T09:06:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.589+0000 7faad49fe640 1 --2- 172.21.15.120:0/484103540 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faab80a3240 0x7faab80a5700 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.589+0000 7faad49fe640 1 --2- 172.21.15.120:0/484103540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faad01023f0 0x7faad019f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.589+0000 7faad49fe640 1 --2- 172.21.15.120:0/484103540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faad0101a30 0x7faad0198200 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.589+0000 7faad49fe640 1 --2- 172.21.15.120:0/484103540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faad0100840 0x7faad0197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.589+0000 7faad49fe640 1 -- 172.21.15.120:0/484103540 >> 172.21.15.120:0/484103540 conn(0x7faad00fbfa0 msgr2=0x7faad006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.589+0000 7faad49fe640 1 -- 172.21.15.120:0/484103540 shutdown_connections 2024-09-19T09:06:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:07.589+0000 7faad49fe640 1 -- 172.21.15.120:0/484103540 wait complete. 2024-09-19T09:06:07.744 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:06:08.041 INFO:tasks.workunit.client.0.smithi120.stdout: 1 235074 13.12 MB/sec execute 3 sec latency 81.861 ms 2024-09-19T09:06:08.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.314+0000 7f5f6e7fe640 1 -- 172.21.15.120:0/2310764135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f681036e0 msgr2=0x7f5f68107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:08.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.314+0000 7f5f6e7fe640 1 --2- 172.21.15.120:0/2310764135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f681036e0 0x7f5f68107b90 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5f50009930 tx=0x7f5f5002a420 comp rx=0 tx=0).stop 2024-09-19T09:06:08.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.314+0000 7f5f6e7fe640 1 -- 172.21.15.120:0/2310764135 shutdown_connections 2024-09-19T09:06:08.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.314+0000 7f5f6e7fe640 1 --2- 172.21.15.120:0/2310764135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f681082a0 0x7f5f6810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.314+0000 7f5f6e7fe640 1 --2- 172.21.15.120:0/2310764135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f681036e0 0x7f5f68107b90 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.314+0000 7f5f6e7fe640 1 --2- 172.21.15.120:0/2310764135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f68102d30 0x7f5f68103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.314+0000 7f5f6e7fe640 1 -- 172.21.15.120:0/2310764135 >> 172.21.15.120:0/2310764135 conn(0x7f5f680fc7d0 msgr2=0x7f5f680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:08.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.314+0000 7f5f6e7fe640 1 -- 172.21.15.120:0/2310764135 shutdown_connections 2024-09-19T09:06:08.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.314+0000 7f5f6e7fe640 1 -- 172.21.15.120:0/2310764135 wait complete. 2024-09-19T09:06:08.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.315+0000 7f5f6e7fe640 1 Processor -- start 2024-09-19T09:06:08.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.315+0000 7f5f6e7fe640 1 -- start start 2024-09-19T09:06:08.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.315+0000 7f5f6e7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f68102d30 0x7f5f68198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:08.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.315+0000 7f5f6e7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f681036e0 0x7f5f68198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:08.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.315+0000 7f5f6e7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f681082a0 0x7f5f6819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:08.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.315+0000 7f5f6e7fe640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5f681120a0 con 0x7f5f681036e0 2024-09-19T09:06:08.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.315+0000 7f5f6e7fe640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5f68111f20 con 0x7f5f68102d30 2024-09-19T09:06:08.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.315+0000 7f5f6e7fe640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5f68112220 con 0x7f5f681082a0 2024-09-19T09:06:08.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.315+0000 7f5f5ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f681036e0 0x7f5f68198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:08.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.315+0000 7f5f67fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f68102d30 0x7f5f68198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:08.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.315+0000 7f5f67fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f68102d30 0x7f5f68198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37996/0 (socket says 172.21.15.120:37996) 2024-09-19T09:06:08.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.315+0000 7f5f67fff640 1 -- 172.21.15.120:0/3122795568 learned_addr learned my addr 172.21.15.120:0/3122795568 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:08.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.315+0000 7f5f5ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f681036e0 0x7f5f68198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50464/0 (socket says 172.21.15.120:50464) 2024-09-19T09:06:08.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.315+0000 7f5f6cd9b640 1 --2- 172.21.15.120:0/3122795568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f681082a0 0x7f5f6819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:08.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.315+0000 7f5f67fff640 1 -- 172.21.15.120:0/3122795568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f681082a0 msgr2=0x7f5f6819cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:08.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.316+0000 7f5f67fff640 1 --2- 172.21.15.120:0/3122795568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f681082a0 0x7f5f6819cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.316+0000 7f5f67fff640 1 -- 172.21.15.120:0/3122795568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f681036e0 msgr2=0x7f5f68198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:08.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.316+0000 7f5f67fff640 1 --2- 172.21.15.120:0/3122795568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f681036e0 0x7f5f68198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.316+0000 7f5f67fff640 1 -- 172.21.15.120:0/3122795568 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f6819d5f0 con 0x7f5f68102d30 2024-09-19T09:06:08.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.316+0000 7f5f67fff640 1 --2- 172.21.15.120:0/3122795568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f68102d30 0x7f5f68198640 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800e9f0 tx=0x7f5f5800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:08.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.316+0000 7f5f65ffb640 1 -- 172.21.15.120:0/3122795568 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5800cd80 con 0x7f5f68102d30 2024-09-19T09:06:08.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.316+0000 7f5f65ffb640 1 -- 172.21.15.120:0/3122795568 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f58010910 con 0x7f5f68102d30 2024-09-19T09:06:08.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.316+0000 7f5f65ffb640 1 -- 172.21.15.120:0/3122795568 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f58018990 con 0x7f5f68102d30 2024-09-19T09:06:08.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.316+0000 7f5f6e7fe640 1 -- 172.21.15.120:0/3122795568 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f6819d8e0 con 0x7f5f68102d30 2024-09-19T09:06:08.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.316+0000 7f5f6e7fe640 1 -- 172.21.15.120:0/3122795568 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f681008c0 con 0x7f5f68102d30 2024-09-19T09:06:08.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.319+0000 7f5f65ffb640 1 -- 172.21.15.120:0/3122795568 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5f58010ac0 con 0x7f5f68102d30 2024-09-19T09:06:08.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.319+0000 7f5f70bd7740 1 Processor -- start 2024-09-19T09:06:08.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.319+0000 7f5f70bd7740 1 -- start start 2024-09-19T09:06:08.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.319+0000 7f5f70bd7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de53f41b60 0x55de53f41f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:08.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.319+0000 7f5f70bd7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de53f42480 0x55de53f4c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:08.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.319+0000 7f5f70bd7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de53f4cd40 0x55de53f4f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:08.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.319+0000 7f5f70bd7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55de53ea7280 con 0x55de53f42480 2024-09-19T09:06:08.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.319+0000 7f5f70bd7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55de53e4eb10 con 0x55de53f41b60 2024-09-19T09:06:08.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.319+0000 7f5f70bd7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55de53ea3240 con 0x55de53f4cd40 2024-09-19T09:06:08.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.319+0000 7f5f65ffb640 1 --2- 172.21.15.120:0/3122795568 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5f540a3240 0x7f5f540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:08.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.320+0000 7f5f6cd9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de53f41b60 0x55de53f41f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:08.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.320+0000 7f5f67fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de53f42480 0x55de53f4c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:08.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.320+0000 7f5f6cd9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de53f41b60 0x55de53f41f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38008/0 (socket says 172.21.15.120:38008) 2024-09-19T09:06:08.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.320+0000 7f5f6cd9b640 1 -- 172.21.15.120:0/3860655729 learned_addr learned my addr 172.21.15.120:0/3860655729 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:08.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.320+0000 7f5f5ffff640 1 --2- 172.21.15.120:0/3860655729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de53f4cd40 0x55de53f4f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:08.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.320+0000 7f5f6cd9b640 1 --2- 172.21.15.120:0/3122795568 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5f540a3240 0x7f5f540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:08.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.320+0000 7f5f65ffb640 1 -- 172.21.15.120:0/3122795568 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f5f580cfcf0 con 0x7f5f68102d30 2024-09-19T09:06:08.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.320+0000 7f5f6cd9b640 1 -- 172.21.15.120:0/3860655729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de53f4cd40 msgr2=0x55de53f4f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:08.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.320+0000 7f5f6cd9b640 1 --2- 172.21.15.120:0/3860655729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de53f4cd40 0x55de53f4f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.320+0000 7f5f6cd9b640 1 -- 172.21.15.120:0/3860655729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de53f42480 msgr2=0x55de53f4c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:08.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.320+0000 7f5f6cd9b640 1 --2- 172.21.15.120:0/3860655729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de53f42480 0x55de53f4c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.320+0000 7f5f6cd9b640 1 -- 172.21.15.120:0/3860655729 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de53e5b1f0 con 0x55de53f41b60 2024-09-19T09:06:08.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.320+0000 7f5f6cd9b640 1 --2- 172.21.15.120:0/3860655729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de53f41b60 0x55de53f41f40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5f600129f0 tx=0x7f5f60012eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b178b65e124e185 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:08.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.320+0000 7f5f6cd9b640 1 --2- 172.21.15.120:0/3122795568 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5f540a3240 0x7f5f540a5700 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5f6001c000 tx=0x7f5f60020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:08.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.320+0000 7f5f5e7fc640 1 -- 172.21.15.120:0/3860655729 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f600120d0 con 0x55de53f41b60 2024-09-19T09:06:08.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.320+0000 7f5f5e7fc640 1 -- 172.21.15.120:0/3860655729 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f60023020 con 0x55de53f41b60 2024-09-19T09:06:08.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.320+0000 7f5f5e7fc640 1 -- 172.21.15.120:0/3860655729 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f60004030 con 0x55de53f41b60 2024-09-19T09:06:08.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.321+0000 7f5f70bd7740 1 -- 172.21.15.120:0/3860655729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de53f41b60 msgr2=0x55de53f41f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:08.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.321+0000 7f5f70bd7740 1 --2- 172.21.15.120:0/3860655729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de53f41b60 0x55de53f41f40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5f600129f0 tx=0x7f5f60012eb0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.321+0000 7f5f70bd7740 1 -- 172.21.15.120:0/3860655729 shutdown_connections 2024-09-19T09:06:08.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.321+0000 7f5f70bd7740 1 --2- 172.21.15.120:0/3860655729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de53f4cd40 0x55de53f4f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.321+0000 7f5f70bd7740 1 --2- 172.21.15.120:0/3860655729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de53f42480 0x55de53f4c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.321+0000 7f5f70bd7740 1 --2- 172.21.15.120:0/3860655729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de53f41b60 0x55de53f41f40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.321+0000 7f5f70bd7740 1 -- 172.21.15.120:0/3860655729 >> 172.21.15.120:0/3860655729 conn(0x55de53f3e0f0 msgr2=0x55de53f3e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:08.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.321+0000 7f5f70bd7740 1 -- 172.21.15.120:0/3860655729 shutdown_connections 2024-09-19T09:06:08.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.321+0000 7f5f70bd7740 1 -- 172.21.15.120:0/3860655729 wait complete. 2024-09-19T09:06:08.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.322+0000 7f5f70bd7740 1 -- start start 2024-09-19T09:06:08.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.322+0000 7f5f70bd7740 1 Processor -- start 2024-09-19T09:06:08.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.322+0000 7f5f70bd7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de53f41b60 0x55de53f49c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:08.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.322+0000 7f5f70bd7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de53f42480 0x55de53f4a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:08.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.322+0000 7f5f70bd7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de53f4cd40 0x55de53f4a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:08.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.322+0000 7f5f70bd7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55de53f4f8b0 con 0x55de53f41b60 2024-09-19T09:06:08.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.322+0000 7f5f70bd7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55de53e61f10 con 0x55de53f4cd40 2024-09-19T09:06:08.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.322+0000 7f5f70bd7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55de53f4fa30 con 0x55de53f42480 2024-09-19T09:06:08.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.322+0000 7f5f70bd7740 10 client.0 _ll_register_callbacks cb 0x55de53e2aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:08.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.323+0000 7f5f67fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de53f41b60 0x55de53f49c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:08.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.323+0000 7f5f5ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de53f42480 0x55de53f4a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:08.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.323+0000 7f5f67fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de53f41b60 0x55de53f49c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50494/0 (socket says 172.21.15.120:50494) 2024-09-19T09:06:08.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.323+0000 7f5f67fff640 1 -- 172.21.15.120:0/2893273958 learned_addr learned my addr 172.21.15.120:0/2893273958 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:08.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.323+0000 7f5f6cd9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de53f4cd40 0x55de53f4a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:08.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.323+0000 7f5f67fff640 1 -- 172.21.15.120:0/2893273958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de53f42480 msgr2=0x55de53f4a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:08.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.323+0000 7f5f67fff640 1 --2- 172.21.15.120:0/2893273958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de53f42480 0x55de53f4a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.323+0000 7f5f67fff640 1 -- 172.21.15.120:0/2893273958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de53f4cd40 msgr2=0x55de53f4a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:08.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.323+0000 7f5f67fff640 1 --2- 172.21.15.120:0/2893273958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de53f4cd40 0x55de53f4a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.323+0000 7f5f67fff640 1 -- 172.21.15.120:0/2893273958 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de53c186a0 con 0x55de53f41b60 2024-09-19T09:06:08.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.323+0000 7f5f67fff640 1 --2- 172.21.15.120:0/2893273958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de53f41b60 0x55de53f49c40 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800e9c0 tx=0x7f5f5809bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05b44fd4eb2bdee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:08.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.323+0000 7f5f477fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:06:08.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.323+0000 7f5f477fe640 1 -- 172.21.15.120:0/2893273958 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f58026040 con 0x55de53f41b60 2024-09-19T09:06:08.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.323+0000 7f5f477fe640 1 -- 172.21.15.120:0/2893273958 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f5f58018b30 con 0x55de53f41b60 2024-09-19T09:06:08.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.323+0000 7f5f70bd7740 1 -- 172.21.15.120:0/2893273958 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f60023020 con 0x55de53f41b60 2024-09-19T09:06:08.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.323+0000 7f5f477fe640 1 -- 172.21.15.120:0/2893273958 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5f580a06c0 con 0x55de53f41b60 2024-09-19T09:06:08.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.323+0000 7f5f70bd7740 10 client.16671 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:08.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.323+0000 7f5f70bd7740 1 -- 172.21.15.120:0/2893273958 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55de53f5b4e0 con 0x55de53f41b60 2024-09-19T09:06:08.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.324+0000 7f5f477fe640 1 -- 172.21.15.120:0/2893273958 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f60023020 con 0x55de53f41b60 2024-09-19T09:06:08.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.324+0000 7f5f477fe640 1 -- 172.21.15.120:0/2893273958 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5f5809b760 con 0x55de53f41b60 2024-09-19T09:06:08.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.324+0000 7f5f70bd7740 10 client.16671 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:08.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.324+0000 7f5f70bd7740 10 client.16671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:06:08.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.324+0000 7f5f70bd7740 20 client.16671 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:08.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.324+0000 7f5f70bd7740 1 --2- 172.21.15.120:0/2893273958 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55de53f5c9f0 0x55de53f5ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:08.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.324+0000 7f5f70bd7740 4 client.16671 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:06:08.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.324+0000 7f5f70bd7740 1 -- 172.21.15.120:0/2893273958 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de53eb47f0 con 0x55de53f5c9f0 2024-09-19T09:06:08.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.324+0000 7f5f5ffff640 1 --2- 172.21.15.120:0/2893273958 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55de53f5c9f0 0x55de53f5ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:08.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.324+0000 7f5f477fe640 10 client.16671 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:08.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.324+0000 7f5f5ffff640 1 --2- 172.21.15.120:0/2893273958 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55de53f5c9f0 0x55de53f5ee60 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a46ec53e370af7b server_cookie=2842dde3dae03adb in_seq=0 out_seq=0 2024-09-19T09:06:08.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.325+0000 7f5f477fe640 1 -- 172.21.15.120:0/2893273958 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55de53eb47f0 con 0x55de53f5c9f0 2024-09-19T09:06:08.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.325+0000 7f5f477fe640 10 client.16671 handle_command_reply: tid=0 2024-09-19T09:06:08.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.325+0000 7f5f70bd7740 1 client.16671 shutdown 2024-09-19T09:06:08.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.326+0000 7f5f70bd7740 1 -- 172.21.15.120:0/2893273958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de53f41b60 msgr2=0x55de53f49c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:08.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.326+0000 7f5f70bd7740 1 --2- 172.21.15.120:0/2893273958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de53f41b60 0x55de53f49c40 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800e9c0 tx=0x7f5f5809bc70 comp rx=0 tx=0).stop 2024-09-19T09:06:08.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.326+0000 7f5f70bd7740 1 -- 172.21.15.120:0/2893273958 shutdown_connections 2024-09-19T09:06:08.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.326+0000 7f5f70bd7740 1 --2- 172.21.15.120:0/2893273958 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55de53f5c9f0 0x55de53f5ee60 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.326+0000 7f5f477fe640 0 client.16671 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:08.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.326+0000 7f5f70bd7740 1 --2- 172.21.15.120:0/2893273958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de53f4cd40 0x55de53f4a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.326+0000 7f5f70bd7740 1 --2- 172.21.15.120:0/2893273958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de53f42480 0x55de53f4a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.326+0000 7f5f70bd7740 1 --2- 172.21.15.120:0/2893273958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de53f41b60 0x55de53f49c40 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.326+0000 7f5f70bd7740 1 -- 172.21.15.120:0/2893273958 >> 172.21.15.120:0/2893273958 conn(0x55de53f3e0f0 msgr2=0x55de53f3d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:08.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.326+0000 7f5f70bd7740 1 -- 172.21.15.120:0/2893273958 shutdown_connections 2024-09-19T09:06:08.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.326+0000 7f5f70bd7740 1 -- 172.21.15.120:0/2893273958 wait complete. 2024-09-19T09:06:08.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.326+0000 7f5f70bd7740 20 client.16671 trim_cache size 0 max 16384 2024-09-19T09:06:08.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.344+0000 7f5f70bd7740 1 Processor -- start 2024-09-19T09:06:08.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.344+0000 7f5f70bd7740 1 -- start start 2024-09-19T09:06:08.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.344+0000 7f5f70bd7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de53f41b60 0x55de53f445c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:08.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.344+0000 7f5f70bd7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de53f42480 0x55de53f4baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:08.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.344+0000 7f5f70bd7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de53f4cd40 0x55de53f64730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:08.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.344+0000 7f5f70bd7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5f681af9b0 con 0x55de53f4cd40 2024-09-19T09:06:08.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.344+0000 7f5f70bd7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5f681b14f0 con 0x55de53f42480 2024-09-19T09:06:08.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.344+0000 7f5f70bd7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5f681b0080 con 0x55de53f41b60 2024-09-19T09:06:08.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.344+0000 7f5f6cd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de53f4cd40 0x55de53f64730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:08.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.344+0000 7f5f5ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de53f42480 0x55de53f4baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:08.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.344+0000 7f5f6cd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de53f4cd40 0x55de53f64730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50500/0 (socket says 172.21.15.120:50500) 2024-09-19T09:06:08.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.344+0000 7f5f6cd9b640 1 -- 172.21.15.120:0/452405538 learned_addr learned my addr 172.21.15.120:0/452405538 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:08.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.344+0000 7f5f67fff640 1 --2- 172.21.15.120:0/452405538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de53f41b60 0x55de53f445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:08.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.344+0000 7f5f6cd9b640 1 -- 172.21.15.120:0/452405538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de53f41b60 msgr2=0x55de53f445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:08.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.344+0000 7f5f6cd9b640 1 --2- 172.21.15.120:0/452405538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de53f41b60 0x55de53f445c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.345+0000 7f5f6cd9b640 1 -- 172.21.15.120:0/452405538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de53f42480 msgr2=0x55de53f4baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:08.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.345+0000 7f5f6cd9b640 1 --2- 172.21.15.120:0/452405538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de53f42480 0x55de53f4baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.345+0000 7f5f6cd9b640 1 -- 172.21.15.120:0/452405538 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de53f4c510 con 0x55de53f4cd40 2024-09-19T09:06:08.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.345+0000 7f5f67fff640 1 --2- 172.21.15.120:0/452405538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de53f41b60 0x55de53f445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:08.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.345+0000 7f5f5ffff640 1 --2- 172.21.15.120:0/452405538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de53f42480 0x55de53f4baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:08.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.345+0000 7f5f6cd9b640 1 --2- 172.21.15.120:0/452405538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de53f4cd40 0x55de53f64730 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f5f60003a80 tx=0x7f5f60003b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5a7633d2a47d65 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:08.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.345+0000 7f5f5dffb640 1 -- 172.21.15.120:0/452405538 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f60003bc0 con 0x55de53f4cd40 2024-09-19T09:06:08.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.345+0000 7f5f5dffb640 1 -- 172.21.15.120:0/452405538 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f60023040 con 0x55de53f4cd40 2024-09-19T09:06:08.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.345+0000 7f5f5dffb640 1 -- 172.21.15.120:0/452405538 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f60005300 con 0x55de53f4cd40 2024-09-19T09:06:08.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.346+0000 7f5f70bd7740 1 -- 172.21.15.120:0/452405538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de53f4cd40 msgr2=0x55de53f64730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:08.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.346+0000 7f5f70bd7740 1 --2- 172.21.15.120:0/452405538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de53f4cd40 0x55de53f64730 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f5f60003a80 tx=0x7f5f60003b50 comp rx=0 tx=0).stop 2024-09-19T09:06:08.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.346+0000 7f5f70bd7740 1 -- 172.21.15.120:0/452405538 shutdown_connections 2024-09-19T09:06:08.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.346+0000 7f5f70bd7740 1 --2- 172.21.15.120:0/452405538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de53f4cd40 0x55de53f64730 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.346+0000 7f5f70bd7740 1 --2- 172.21.15.120:0/452405538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de53f42480 0x55de53f4baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.346+0000 7f5f70bd7740 1 --2- 172.21.15.120:0/452405538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de53f41b60 0x55de53f445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.346+0000 7f5f70bd7740 1 -- 172.21.15.120:0/452405538 >> 172.21.15.120:0/452405538 conn(0x55de53f3e0f0 msgr2=0x55de53f4b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:08.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.346+0000 7f5f70bd7740 1 -- 172.21.15.120:0/452405538 shutdown_connections 2024-09-19T09:06:08.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.346+0000 7f5f70bd7740 1 -- 172.21.15.120:0/452405538 wait complete. 2024-09-19T09:06:08.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.346+0000 7f5f70bd7740 1 -- start start 2024-09-19T09:06:08.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.346+0000 7f5f70bd7740 1 Processor -- start 2024-09-19T09:06:08.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.347+0000 7f5f70bd7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de53f41b60 0x55de53f510c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:08.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.347+0000 7f5f70bd7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de53f42480 0x55de53f51600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:08.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.347+0000 7f5f70bd7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de53f4cd40 0x55de53fc6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:08.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.347+0000 7f5f70bd7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5f681b0e20 con 0x55de53f42480 2024-09-19T09:06:08.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.347+0000 7f5f70bd7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5f681b0750 con 0x55de53f41b60 2024-09-19T09:06:08.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.347+0000 7f5f70bd7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5f681ad970 con 0x55de53f4cd40 2024-09-19T09:06:08.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.347+0000 7f5f70bd7740 10 client.0 _ll_register_callbacks cb 0x7f5f58097e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:08.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.347+0000 7f5f5ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de53f42480 0x55de53f51600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.347+0000 7f5f5ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de53f42480 0x55de53f51600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50516/0 (socket says 172.21.15.120:50516) 2024-09-19T09:06:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.347+0000 7f5f5ffff640 1 -- 172.21.15.120:0/1701014724 learned_addr learned my addr 172.21.15.120:0/1701014724 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.347+0000 7f5f67fff640 1 --2- 172.21.15.120:0/1701014724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de53f41b60 0x55de53f510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.347+0000 7f5f5ffff640 1 -- 172.21.15.120:0/1701014724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de53f4cd40 msgr2=0x55de53fc6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.347+0000 7f5f6cd9b640 1 --2- 172.21.15.120:0/1701014724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de53f4cd40 0x55de53fc6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.347+0000 7f5f5ffff640 1 --2- 172.21.15.120:0/1701014724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de53f4cd40 0x55de53fc6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.347+0000 7f5f5ffff640 1 -- 172.21.15.120:0/1701014724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de53f41b60 msgr2=0x55de53f510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.347+0000 7f5f5ffff640 1 --2- 172.21.15.120:0/1701014724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de53f41b60 0x55de53f510c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.347+0000 7f5f5ffff640 1 -- 172.21.15.120:0/1701014724 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de53fc6c40 con 0x55de53f42480 2024-09-19T09:06:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.347+0000 7f5f67fff640 1 --2- 172.21.15.120:0/1701014724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de53f41b60 0x55de53f510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:06:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.347+0000 7f5f6cd9b640 1 --2- 172.21.15.120:0/1701014724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de53f4cd40 0x55de53fc6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:06:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.347+0000 7f5f5ffff640 1 --2- 172.21.15.120:0/1701014724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de53f42480 0x55de53f51600 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5f50005940 tx=0x7f5f5002a9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce8cf417d5cd9ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.347+0000 7f5f477fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:06:08.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.347+0000 7f5f477fe640 1 -- 172.21.15.120:0/1701014724 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f50035040 con 0x55de53f42480 2024-09-19T09:06:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.347+0000 7f5f477fe640 1 -- 172.21.15.120:0/1701014724 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f5f50005040 con 0x55de53f42480 2024-09-19T09:06:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.347+0000 7f5f70bd7740 1 -- 172.21.15.120:0/1701014724 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f60023040 con 0x55de53f42480 2024-09-19T09:06:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.348+0000 7f5f477fe640 1 -- 172.21.15.120:0/1701014724 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5f50005240 con 0x55de53f42480 2024-09-19T09:06:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.348+0000 7f5f70bd7740 10 client.16677 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.348+0000 7f5f70bd7740 1 -- 172.21.15.120:0/1701014724 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55de53fc7f40 con 0x55de53f42480 2024-09-19T09:06:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.348+0000 7f5f477fe640 1 -- 172.21.15.120:0/1701014724 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f60023040 con 0x55de53f42480 2024-09-19T09:06:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.348+0000 7f5f477fe640 1 -- 172.21.15.120:0/1701014724 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5f50003710 con 0x55de53f42480 2024-09-19T09:06:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.348+0000 7f5f70bd7740 10 client.16677 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.348+0000 7f5f70bd7740 10 client.16677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:06:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.348+0000 7f5f70bd7740 20 client.16677 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.348+0000 7f5f70bd7740 1 --2- 172.21.15.120:0/1701014724 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55de53f5c9f0 0x55de53fca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.348+0000 7f5f70bd7740 4 client.16677 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:06:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.348+0000 7f5f70bd7740 1 -- 172.21.15.120:0/1701014724 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55de53eaed00 con 0x55de53f5c9f0 2024-09-19T09:06:08.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.348+0000 7f5f67fff640 1 --2- 172.21.15.120:0/1701014724 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55de53f5c9f0 0x55de53fca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:08.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.349+0000 7f5f67fff640 1 --2- 172.21.15.120:0/1701014724 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55de53f5c9f0 0x55de53fca6b0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=403be1b5b5d06773 server_cookie=7a87091004ad948d in_seq=0 out_seq=0 2024-09-19T09:06:08.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.349+0000 7f5f477fe640 10 client.16677 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:08.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.351+0000 7f5f477fe640 1 -- 172.21.15.120:0/1701014724 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55de53eaed00 con 0x55de53f5c9f0 2024-09-19T09:06:08.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.351+0000 7f5f477fe640 10 client.16677 handle_command_reply: tid=0 2024-09-19T09:06:08.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.351+0000 7f5f70bd7740 1 client.16677 shutdown 2024-09-19T09:06:08.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.351+0000 7f5f70bd7740 1 -- 172.21.15.120:0/1701014724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de53f42480 msgr2=0x55de53f51600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:08.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.351+0000 7f5f70bd7740 1 --2- 172.21.15.120:0/1701014724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de53f42480 0x55de53f51600 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5f50005940 tx=0x7f5f5002a9c0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.351+0000 7f5f70bd7740 1 -- 172.21.15.120:0/1701014724 shutdown_connections 2024-09-19T09:06:08.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.351+0000 7f5f70bd7740 1 --2- 172.21.15.120:0/1701014724 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55de53f5c9f0 0x55de53fca6b0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.351+0000 7f5f70bd7740 1 --2- 172.21.15.120:0/1701014724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de53f4cd40 0x55de53fc6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.351+0000 7f5f70bd7740 1 --2- 172.21.15.120:0/1701014724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de53f42480 0x55de53f51600 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.351+0000 7f5f477fe640 0 client.16677 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:08.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.351+0000 7f5f70bd7740 1 --2- 172.21.15.120:0/1701014724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de53f41b60 0x55de53f510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.351+0000 7f5f70bd7740 1 -- 172.21.15.120:0/1701014724 >> 172.21.15.120:0/1701014724 conn(0x55de53f3e0f0 msgr2=0x55de53f3d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:08.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.352+0000 7f5f70bd7740 1 -- 172.21.15.120:0/1701014724 shutdown_connections 2024-09-19T09:06:08.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.352+0000 7f5f70bd7740 1 -- 172.21.15.120:0/1701014724 wait complete. 2024-09-19T09:06:08.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.352+0000 7f5f70bd7740 20 client.16677 trim_cache size 0 max 16384 2024-09-19T09:06:08.356 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:06:08.356 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:06:08.356 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "48a62073-f66c-498c-b1d3-611b2faee032", 2024-09-19T09:06:08.356 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:06:08.356 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:06:08.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.352+0000 7f5f6e7fe640 1 -- 172.21.15.120:0/3122795568 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5f540a3240 msgr2=0x7f5f540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:08.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.352+0000 7f5f6e7fe640 1 --2- 172.21.15.120:0/3122795568 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5f540a3240 0x7f5f540a5700 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5f6001c000 tx=0x7f5f60020000 comp rx=0 tx=0).stop 2024-09-19T09:06:08.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.352+0000 7f5f6e7fe640 1 -- 172.21.15.120:0/3122795568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f68102d30 msgr2=0x7f5f68198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:08.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.352+0000 7f5f6e7fe640 1 --2- 172.21.15.120:0/3122795568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f68102d30 0x7f5f68198640 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800e9f0 tx=0x7f5f5800eeb0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.352+0000 7f5f6e7fe640 1 -- 172.21.15.120:0/3122795568 shutdown_connections 2024-09-19T09:06:08.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.352+0000 7f5f6e7fe640 1 --2- 172.21.15.120:0/3122795568 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5f540a3240 0x7f5f540a5700 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.352+0000 7f5f6e7fe640 1 --2- 172.21.15.120:0/3122795568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f681082a0 0x7f5f6819cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.352+0000 7f5f6e7fe640 1 --2- 172.21.15.120:0/3122795568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f681036e0 0x7f5f68198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.352+0000 7f5f6e7fe640 1 --2- 172.21.15.120:0/3122795568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f68102d30 0x7f5f68198640 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:08.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.352+0000 7f5f6e7fe640 1 -- 172.21.15.120:0/3122795568 >> 172.21.15.120:0/3122795568 conn(0x7f5f680fc7d0 msgr2=0x7f5f68106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:08.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.353+0000 7f5f6e7fe640 1 -- 172.21.15.120:0/3122795568 shutdown_connections 2024-09-19T09:06:08.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:08.353+0000 7f5f6e7fe640 1 -- 172.21.15.120:0/3122795568 wait complete. 2024-09-19T09:06:09.041 INFO:tasks.workunit.client.0.smithi120.stdout: 1 238165 14.42 MB/sec execute 4 sec latency 40.697 ms 2024-09-19T09:06:09.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:08 smithi120 ceph-mon[31150]: pgmap v439: 97 pgs: 97 active+clean; 46 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 399 op/s 2024-09-19T09:06:09.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:08 smithi196 ceph-mon[35150]: pgmap v439: 97 pgs: 97 active+clean; 46 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 399 op/s 2024-09-19T09:06:09.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:08 smithi153 ceph-mon[35071]: pgmap v439: 97 pgs: 97 active+clean; 46 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 399 op/s 2024-09-19T09:06:09.266 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:06:09.592 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:06:09.998 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:09 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:06:09.998 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:09 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:06:09.998 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:09 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:06:09.998 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:09 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:06:10.041 INFO:tasks.workunit.client.0.smithi120.stdout: 1 241317 15.07 MB/sec execute 5 sec latency 56.992 ms 2024-09-19T09:06:10.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.083+0000 7f00a5fba640 1 -- 172.21.15.120:0/285529959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00a0101df0 msgr2=0x7f00a01021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:10.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.083+0000 7f00a5fba640 1 --2- 172.21.15.120:0/285529959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00a0101df0 0x7f00a01021d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f0090009900 tx=0x7f009002a410 comp rx=0 tx=0).stop 2024-09-19T09:06:10.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.083+0000 7f00a5fba640 1 -- 172.21.15.120:0/285529959 shutdown_connections 2024-09-19T09:06:10.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.083+0000 7f00a5fba640 1 --2- 172.21.15.120:0/285529959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00a0106f20 0x7f00a010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.083+0000 7f00a5fba640 1 --2- 172.21.15.120:0/285529959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00a01027a0 0x7f00a01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.083+0000 7f00a5fba640 1 --2- 172.21.15.120:0/285529959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00a0101df0 0x7f00a01021d0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.083+0000 7f00a5fba640 1 -- 172.21.15.120:0/285529959 >> 172.21.15.120:0/285529959 conn(0x7f00a00fd8f0 msgr2=0x7f00a00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:10.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.083+0000 7f00a5fba640 1 -- 172.21.15.120:0/285529959 shutdown_connections 2024-09-19T09:06:10.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.083+0000 7f00a5fba640 1 -- 172.21.15.120:0/285529959 wait complete. 2024-09-19T09:06:10.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.084+0000 7f00a5fba640 1 Processor -- start 2024-09-19T09:06:10.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.084+0000 7f00a5fba640 1 -- start start 2024-09-19T09:06:10.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.084+0000 7f00a5fba640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00a0101df0 0x7f00a019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:10.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.084+0000 7f00a5fba640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00a01027a0 0x7f00a019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:10.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.084+0000 7f00a5fba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00a0106f20 0x7f00a019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:10.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.084+0000 7f00a5fba640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f00a0111f80 con 0x7f00a0101df0 2024-09-19T09:06:10.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.084+0000 7f00a5fba640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f00a0111e00 con 0x7f00a01027a0 2024-09-19T09:06:10.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.084+0000 7f00a5fba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f00a0112100 con 0x7f00a0106f20 2024-09-19T09:06:10.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.084+0000 7f009f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00a0101df0 0x7f00a019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:10.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.084+0000 7f009f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00a0101df0 0x7f00a019a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50540/0 (socket says 172.21.15.120:50540) 2024-09-19T09:06:10.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.084+0000 7f009f7fe640 1 -- 172.21.15.120:0/2481028638 learned_addr learned my addr 172.21.15.120:0/2481028638 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:10.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.084+0000 7f009effd640 1 --2- 172.21.15.120:0/2481028638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00a01027a0 0x7f00a019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:10.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.084+0000 7f009ffff640 1 --2- 172.21.15.120:0/2481028638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00a0106f20 0x7f00a019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:10.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.085+0000 7f009f7fe640 1 -- 172.21.15.120:0/2481028638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00a0106f20 msgr2=0x7f00a019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:10.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.085+0000 7f009f7fe640 1 --2- 172.21.15.120:0/2481028638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00a0106f20 0x7f00a019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.085+0000 7f009f7fe640 1 -- 172.21.15.120:0/2481028638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00a01027a0 msgr2=0x7f00a019ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:10.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.085+0000 7f009f7fe640 1 --2- 172.21.15.120:0/2481028638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00a01027a0 0x7f00a019ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.085+0000 7f009f7fe640 1 -- 172.21.15.120:0/2481028638 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a019f670 con 0x7f00a0101df0 2024-09-19T09:06:10.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.085+0000 7f009f7fe640 1 --2- 172.21.15.120:0/2481028638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00a0101df0 0x7f00a019a6c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0090006010 tx=0x7f00900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:10.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.085+0000 7f009cff9640 1 -- 172.21.15.120:0/2481028638 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009003f030 con 0x7f00a0101df0 2024-09-19T09:06:10.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.085+0000 7f009cff9640 1 -- 172.21.15.120:0/2481028638 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0090003ce0 con 0x7f00a0101df0 2024-09-19T09:06:10.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.085+0000 7f00a5fba640 1 -- 172.21.15.120:0/2481028638 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00a019f900 con 0x7f00a0101df0 2024-09-19T09:06:10.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.085+0000 7f009cff9640 1 -- 172.21.15.120:0/2481028638 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0090035070 con 0x7f00a0101df0 2024-09-19T09:06:10.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.085+0000 7f00a5fba640 1 -- 172.21.15.120:0/2481028638 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00a01a7240 con 0x7f00a0101df0 2024-09-19T09:06:10.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.087+0000 7f009cff9640 1 -- 172.21.15.120:0/2481028638 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f0090042030 con 0x7f00a0101df0 2024-09-19T09:06:10.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.088+0000 7f00a8393740 1 Processor -- start 2024-09-19T09:06:10.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.088+0000 7f00a8393740 1 -- start start 2024-09-19T09:06:10.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.088+0000 7f00a8393740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e0b7334a0 0x564e0b733880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:10.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.088+0000 7f00a8393740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e0b733dc0 0x564e0b73e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:10.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.088+0000 7f009cff9640 1 --2- 172.21.15.120:0/2481028638 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f00880a32c0 0x7f00880a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:10.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.088+0000 7f00a8393740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e0b73e680 0x564e0b740a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:10.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.088+0000 7f00a8393740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564e0b6992e0 con 0x564e0b73e680 2024-09-19T09:06:10.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.088+0000 7f00a8393740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564e0b640ce0 con 0x564e0b733dc0 2024-09-19T09:06:10.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.088+0000 7f00a8393740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564e0b6952a0 con 0x564e0b7334a0 2024-09-19T09:06:10.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.088+0000 7f009ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e0b7334a0 0x564e0b733880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:10.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.088+0000 7f009f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e0b733dc0 0x564e0b73e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:10.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.088+0000 7f009ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e0b7334a0 0x564e0b733880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:39156/0 (socket says 172.21.15.120:39156) 2024-09-19T09:06:10.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.088+0000 7f009ffff640 1 -- 172.21.15.120:0/3182165342 learned_addr learned my addr 172.21.15.120:0/3182165342 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:10.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.088+0000 7f009ffff640 1 --2- 172.21.15.120:0/3182165342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e0b73e680 0x564e0b740a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:10.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.088+0000 7f009effd640 1 --2- 172.21.15.120:0/2481028638 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f00880a32c0 0x7f00880a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:10.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.088+0000 7f009cff9640 1 -- 172.21.15.120:0/2481028638 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f00900efdc0 con 0x7f00a0101df0 2024-09-19T09:06:10.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.089+0000 7f009ffff640 1 -- 172.21.15.120:0/3182165342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e0b733dc0 msgr2=0x564e0b73e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:10.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.089+0000 7f009ffff640 1 --2- 172.21.15.120:0/3182165342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e0b733dc0 0x564e0b73e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.089+0000 7f009ffff640 1 -- 172.21.15.120:0/3182165342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e0b73e680 msgr2=0x564e0b740a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:10.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.089+0000 7f009ffff640 1 --2- 172.21.15.120:0/3182165342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e0b73e680 0x564e0b740a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.089+0000 7f009ffff640 1 -- 172.21.15.120:0/3182165342 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e0b625440 con 0x564e0b7334a0 2024-09-19T09:06:10.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.089+0000 7f009ffff640 1 --2- 172.21.15.120:0/3182165342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e0b7334a0 0x564e0b733880 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0098005c40 tx=0x7f0098009a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2304d41297eb6c11 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:10.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.089+0000 7f009effd640 1 --2- 172.21.15.120:0/2481028638 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f00880a32c0 0x7f00880a5780 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f00a0076510 tx=0x7f0094005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:10.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.089+0000 7f0085ffb640 1 -- 172.21.15.120:0/3182165342 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0098004b20 con 0x564e0b7334a0 2024-09-19T09:06:10.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.089+0000 7f0085ffb640 1 -- 172.21.15.120:0/3182165342 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0098003e50 con 0x564e0b7334a0 2024-09-19T09:06:10.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.089+0000 7f0085ffb640 1 -- 172.21.15.120:0/3182165342 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0098004030 con 0x564e0b7334a0 2024-09-19T09:06:10.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.090+0000 7f00a8393740 1 -- 172.21.15.120:0/3182165342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e0b7334a0 msgr2=0x564e0b733880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:10.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.090+0000 7f00a8393740 1 --2- 172.21.15.120:0/3182165342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e0b7334a0 0x564e0b733880 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0098005c40 tx=0x7f0098009a50 comp rx=0 tx=0).stop 2024-09-19T09:06:10.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.090+0000 7f00a8393740 1 -- 172.21.15.120:0/3182165342 shutdown_connections 2024-09-19T09:06:10.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.090+0000 7f00a8393740 1 --2- 172.21.15.120:0/3182165342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e0b73e680 0x564e0b740a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.090+0000 7f00a8393740 1 --2- 172.21.15.120:0/3182165342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e0b733dc0 0x564e0b73e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.090+0000 7f00a8393740 1 --2- 172.21.15.120:0/3182165342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e0b7334a0 0x564e0b733880 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.090+0000 7f00a8393740 1 -- 172.21.15.120:0/3182165342 >> 172.21.15.120:0/3182165342 conn(0x564e0b72fa30 msgr2=0x564e0b72fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:10.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.090+0000 7f00a8393740 1 -- 172.21.15.120:0/3182165342 shutdown_connections 2024-09-19T09:06:10.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.090+0000 7f00a8393740 1 -- 172.21.15.120:0/3182165342 wait complete. 2024-09-19T09:06:10.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.090+0000 7f00a8393740 1 -- start start 2024-09-19T09:06:10.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.090+0000 7f00a8393740 1 Processor -- start 2024-09-19T09:06:10.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.091+0000 7f00a8393740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e0b7334a0 0x564e0b73dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:10.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.091+0000 7f00a8393740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e0b733dc0 0x564e0b73b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:10.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.091+0000 7f00a8393740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e0b73e680 0x564e0b73b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:10.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.091+0000 7f00a8393740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564e0b741250 con 0x564e0b73e680 2024-09-19T09:06:10.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.091+0000 7f00a8393740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564e0b654050 con 0x564e0b7334a0 2024-09-19T09:06:10.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.091+0000 7f00a8393740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564e0b7413d0 con 0x564e0b733dc0 2024-09-19T09:06:10.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.091+0000 7f00a8393740 10 client.0 _ll_register_callbacks cb 0x564e0b694160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:10.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.091+0000 7f009ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e0b73e680 0x564e0b73b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:10.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.091+0000 7f009f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e0b7334a0 0x564e0b73dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:10.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.091+0000 7f009effd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e0b733dc0 0x564e0b73b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:10.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.091+0000 7f009effd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e0b733dc0 0x564e0b73b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:39162/0 (socket says 172.21.15.120:39162) 2024-09-19T09:06:10.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.091+0000 7f009f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e0b7334a0 0x564e0b73dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38086/0 (socket says 172.21.15.120:38086) 2024-09-19T09:06:10.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.091+0000 7f009ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e0b73e680 0x564e0b73b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50558/0 (socket says 172.21.15.120:50558) 2024-09-19T09:06:10.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.091+0000 7f009effd640 1 -- 172.21.15.120:0/2690715702 learned_addr learned my addr 172.21.15.120:0/2690715702 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:10.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.091+0000 7f009effd640 1 -- 172.21.15.120:0/2690715702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e0b7334a0 msgr2=0x564e0b73dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:10.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.091+0000 7f009effd640 1 --2- 172.21.15.120:0/2690715702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e0b7334a0 0x564e0b73dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.091+0000 7f009effd640 1 -- 172.21.15.120:0/2690715702 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e0b73e680 msgr2=0x564e0b73b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:10.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.091+0000 7f009effd640 1 --2- 172.21.15.120:0/2690715702 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e0b73e680 0x564e0b73b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.091+0000 7f009effd640 1 -- 172.21.15.120:0/2690715702 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e0b40a3c0 con 0x564e0b733dc0 2024-09-19T09:06:10.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.091+0000 7f009ffff640 1 --2- 172.21.15.120:0/2690715702 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e0b73e680 0x564e0b73b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:10.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.091+0000 7f009effd640 1 --2- 172.21.15.120:0/2690715702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e0b733dc0 0x564e0b73b280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0094003940 tx=0x7f0094004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7df09f7b892c5746 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:10.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.091+0000 7f007effd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:06:10.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.091+0000 7f007effd640 1 -- 172.21.15.120:0/2690715702 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009400ecc0 con 0x564e0b733dc0 2024-09-19T09:06:10.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.091+0000 7f00a8393740 1 -- 172.21.15.120:0/2690715702 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0098003e50 con 0x564e0b733dc0 2024-09-19T09:06:10.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.092+0000 7f007effd640 1 -- 172.21.15.120:0/2690715702 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f0094005020 con 0x564e0b733dc0 2024-09-19T09:06:10.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.092+0000 7f007effd640 1 -- 172.21.15.120:0/2690715702 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f009400ee60 con 0x564e0b733dc0 2024-09-19T09:06:10.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.092+0000 7f00a8393740 10 client.26005 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:10.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.092+0000 7f00a8393740 1 -- 172.21.15.120:0/2690715702 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564e0b74ccd0 con 0x564e0b733dc0 2024-09-19T09:06:10.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.092+0000 7f007effd640 1 -- 172.21.15.120:0/2690715702 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0098003e50 con 0x564e0b733dc0 2024-09-19T09:06:10.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.092+0000 7f007effd640 1 -- 172.21.15.120:0/2690715702 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f009401e420 con 0x564e0b733dc0 2024-09-19T09:06:10.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.092+0000 7f00a8393740 10 client.26005 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:10.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.092+0000 7f00a8393740 10 client.26005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:06:10.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.092+0000 7f00a8393740 20 client.26005 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:10.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.092+0000 7f00a8393740 1 --2- 172.21.15.120:0/2690715702 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564e0b74e130 0x564e0b750550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:10.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.092+0000 7f00a8393740 4 client.26005 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:06:10.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.092+0000 7f00a8393740 1 -- 172.21.15.120:0/2690715702 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e0b6a6850 con 0x564e0b74e130 2024-09-19T09:06:10.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.092+0000 7f009f7fe640 1 --2- 172.21.15.120:0/2690715702 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564e0b74e130 0x564e0b750550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:10.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.093+0000 7f007effd640 10 client.26005 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:10.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.093+0000 7f009f7fe640 1 --2- 172.21.15.120:0/2690715702 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564e0b74e130 0x564e0b750550 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63607d4f9a0595a9 server_cookie=27e0a910fd753e1e in_seq=0 out_seq=0 2024-09-19T09:06:10.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.094+0000 7f007effd640 1 -- 172.21.15.120:0/2690715702 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e0b6a6850 con 0x564e0b74e130 2024-09-19T09:06:10.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.094+0000 7f007effd640 10 client.26005 handle_command_reply: tid=0 2024-09-19T09:06:10.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.094+0000 7f00a8393740 1 client.26005 shutdown 2024-09-19T09:06:10.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.094+0000 7f00a8393740 1 -- 172.21.15.120:0/2690715702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e0b733dc0 msgr2=0x564e0b73b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:10.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.094+0000 7f00a8393740 1 --2- 172.21.15.120:0/2690715702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e0b733dc0 0x564e0b73b280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0094003940 tx=0x7f0094004730 comp rx=0 tx=0).stop 2024-09-19T09:06:10.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.095+0000 7f00a8393740 1 -- 172.21.15.120:0/2690715702 shutdown_connections 2024-09-19T09:06:10.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.095+0000 7f00a8393740 1 --2- 172.21.15.120:0/2690715702 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564e0b74e130 0x564e0b750550 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.095+0000 7f00a8393740 1 --2- 172.21.15.120:0/2690715702 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e0b73e680 0x564e0b73b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.095+0000 7f00a8393740 1 --2- 172.21.15.120:0/2690715702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e0b733dc0 0x564e0b73b280 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.095+0000 7f007effd640 0 client.26005 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:10.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.095+0000 7f00a8393740 1 --2- 172.21.15.120:0/2690715702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e0b7334a0 0x564e0b73dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.095+0000 7f00a8393740 1 -- 172.21.15.120:0/2690715702 >> 172.21.15.120:0/2690715702 conn(0x564e0b72fa30 msgr2=0x564e0b735710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:10.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.095+0000 7f00a8393740 1 -- 172.21.15.120:0/2690715702 shutdown_connections 2024-09-19T09:06:10.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.095+0000 7f00a8393740 1 -- 172.21.15.120:0/2690715702 wait complete. 2024-09-19T09:06:10.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.095+0000 7f00a8393740 20 client.26005 trim_cache size 0 max 16384 2024-09-19T09:06:10.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.112+0000 7f00a8393740 1 Processor -- start 2024-09-19T09:06:10.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.112+0000 7f00a8393740 1 -- start start 2024-09-19T09:06:10.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.112+0000 7f00a8393740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e0b7334a0 0x564e0b73b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:10.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.112+0000 7f00a8393740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e0b733dc0 0x564e0b74a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:10.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.112+0000 7f00a8393740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e0b73e680 0x564e0b755e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:10.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.112+0000 7f00a8393740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f00a01af7f0 con 0x564e0b73e680 2024-09-19T09:06:10.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.112+0000 7f00a8393740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f00a01b1330 con 0x564e0b7334a0 2024-09-19T09:06:10.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.112+0000 7f00a8393740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f00a01afec0 con 0x564e0b733dc0 2024-09-19T09:06:10.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.112+0000 7f009ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e0b73e680 0x564e0b755e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:10.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.112+0000 7f009ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e0b73e680 0x564e0b755e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50574/0 (socket says 172.21.15.120:50574) 2024-09-19T09:06:10.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.113+0000 7f009ffff640 1 -- 172.21.15.120:0/835138759 learned_addr learned my addr 172.21.15.120:0/835138759 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:10.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.113+0000 7f009effd640 1 --2- 172.21.15.120:0/835138759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e0b733dc0 0x564e0b74a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:10.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.113+0000 7f009f7fe640 1 --2- 172.21.15.120:0/835138759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e0b7334a0 0x564e0b73b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:10.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.113+0000 7f009ffff640 1 -- 172.21.15.120:0/835138759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e0b733dc0 msgr2=0x564e0b74a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:10.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.113+0000 7f009ffff640 1 --2- 172.21.15.120:0/835138759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e0b733dc0 0x564e0b74a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.113+0000 7f009ffff640 1 -- 172.21.15.120:0/835138759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e0b7334a0 msgr2=0x564e0b73b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:10.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.113+0000 7f009ffff640 1 --2- 172.21.15.120:0/835138759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e0b7334a0 0x564e0b73b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.113+0000 7f009ffff640 1 -- 172.21.15.120:0/835138759 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e0b73e340 con 0x564e0b73e680 2024-09-19T09:06:10.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.113+0000 7f009ffff640 1 --2- 172.21.15.120:0/835138759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e0b73e680 0x564e0b755e20 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0098009a20 tx=0x7f009801b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5c3050fd7809 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:10.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.113+0000 7f00857fa640 1 -- 172.21.15.120:0/835138759 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0098010950 con 0x564e0b73e680 2024-09-19T09:06:10.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.113+0000 7f00857fa640 1 -- 172.21.15.120:0/835138759 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0098016040 con 0x564e0b73e680 2024-09-19T09:06:10.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.113+0000 7f00857fa640 1 -- 172.21.15.120:0/835138759 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0098005400 con 0x564e0b73e680 2024-09-19T09:06:10.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.114+0000 7f00a8393740 1 -- 172.21.15.120:0/835138759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e0b73e680 msgr2=0x564e0b755e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:10.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.114+0000 7f00a8393740 1 --2- 172.21.15.120:0/835138759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e0b73e680 0x564e0b755e20 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0098009a20 tx=0x7f009801b040 comp rx=0 tx=0).stop 2024-09-19T09:06:10.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.114+0000 7f00a8393740 1 -- 172.21.15.120:0/835138759 shutdown_connections 2024-09-19T09:06:10.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.114+0000 7f00a8393740 1 --2- 172.21.15.120:0/835138759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e0b73e680 0x564e0b755e20 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.114+0000 7f00a8393740 1 --2- 172.21.15.120:0/835138759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e0b733dc0 0x564e0b74a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.114+0000 7f00a8393740 1 --2- 172.21.15.120:0/835138759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e0b7334a0 0x564e0b73b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.114+0000 7f00a8393740 1 -- 172.21.15.120:0/835138759 >> 172.21.15.120:0/835138759 conn(0x564e0b72fa30 msgr2=0x564e0b73ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:10.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.114+0000 7f00a8393740 1 -- 172.21.15.120:0/835138759 shutdown_connections 2024-09-19T09:06:10.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.114+0000 7f00a8393740 1 -- 172.21.15.120:0/835138759 wait complete. 2024-09-19T09:06:10.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.114+0000 7f00a8393740 1 -- start start 2024-09-19T09:06:10.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.115+0000 7f00a8393740 1 Processor -- start 2024-09-19T09:06:10.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.115+0000 7f00a8393740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e0b7334a0 0x564e0b7424e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:10.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.115+0000 7f00a8393740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e0b733dc0 0x564e0b742a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:10.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.115+0000 7f00a8393740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e0b73e680 0x564e0b742f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:10.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.115+0000 7f00a8393740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f00a01b0c60 con 0x564e0b7334a0 2024-09-19T09:06:10.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.115+0000 7f00a8393740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f00a01b0590 con 0x564e0b73e680 2024-09-19T09:06:10.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.115+0000 7f00a8393740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f00a01ad7b0 con 0x564e0b733dc0 2024-09-19T09:06:10.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.115+0000 7f00a8393740 10 client.0 _ll_register_callbacks cb 0x7f0094004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:10.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.115+0000 7f009f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e0b7334a0 0x564e0b7424e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:10.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.115+0000 7f009f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e0b7334a0 0x564e0b7424e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50590/0 (socket says 172.21.15.120:50590) 2024-09-19T09:06:10.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.115+0000 7f009f7fe640 1 -- 172.21.15.120:0/1203760291 learned_addr learned my addr 172.21.15.120:0/1203760291 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:10.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.115+0000 7f009effd640 1 --2- 172.21.15.120:0/1203760291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e0b733dc0 0x564e0b742a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:10.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.115+0000 7f009ffff640 1 --2- 172.21.15.120:0/1203760291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e0b73e680 0x564e0b742f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:10.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.115+0000 7f009f7fe640 1 -- 172.21.15.120:0/1203760291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e0b733dc0 msgr2=0x564e0b742a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:10.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.115+0000 7f009f7fe640 1 --2- 172.21.15.120:0/1203760291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e0b733dc0 0x564e0b742a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.115+0000 7f009f7fe640 1 -- 172.21.15.120:0/1203760291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e0b73e680 msgr2=0x564e0b742f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:10.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.115+0000 7f009f7fe640 1 --2- 172.21.15.120:0/1203760291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e0b73e680 0x564e0b742f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.115+0000 7f009f7fe640 1 -- 172.21.15.120:0/1203760291 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e0b7b7f60 con 0x564e0b7334a0 2024-09-19T09:06:10.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.115+0000 7f009f7fe640 1 --2- 172.21.15.120:0/1203760291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e0b7334a0 0x564e0b7424e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f00900096c0 tx=0x7f009008b350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9a7c815c621834 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:10.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.115+0000 7f007effd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:06:10.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.115+0000 7f007effd640 1 -- 172.21.15.120:0/1203760291 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0090049040 con 0x564e0b7334a0 2024-09-19T09:06:10.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.115+0000 7f007effd640 1 -- 172.21.15.120:0/1203760291 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f00900ed9d0 con 0x564e0b7334a0 2024-09-19T09:06:10.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.115+0000 7f00a8393740 1 -- 172.21.15.120:0/1203760291 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0098016040 con 0x564e0b7334a0 2024-09-19T09:06:10.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.116+0000 7f007effd640 1 -- 172.21.15.120:0/1203760291 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f00900bbeb0 con 0x564e0b7334a0 2024-09-19T09:06:10.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.116+0000 7f00a8393740 10 client.16695 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:10.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.116+0000 7f00a8393740 1 -- 172.21.15.120:0/1203760291 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564e0b7b9290 con 0x564e0b7334a0 2024-09-19T09:06:10.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.116+0000 7f007effd640 1 -- 172.21.15.120:0/1203760291 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0098016040 con 0x564e0b7334a0 2024-09-19T09:06:10.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.116+0000 7f007effd640 1 -- 172.21.15.120:0/1203760291 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f00900cf3f0 con 0x564e0b7334a0 2024-09-19T09:06:10.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.116+0000 7f00a8393740 10 client.16695 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:10.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.116+0000 7f00a8393740 10 client.16695 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:06:10.157 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:06:10.157 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:06:10.157 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-19T09:06:10.157 INFO:teuthology.orchestra.run.smithi120.stdout: "48a62073-f66c-498c-b1d3-611b2faee032": { 2024-09-19T09:06:10.157 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-19T09:06:10.157 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "48a62073-f66c-498c-b1d3-611b2faee032", 2024-09-19T09:06:10.158 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-19T09:06:10.158 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:06:10.158 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:06:10.158 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:06:10.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.116+0000 7f00a8393740 20 client.16695 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:10.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.116+0000 7f00a8393740 1 --2- 172.21.15.120:0/1203760291 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564e0b7b99b0 0x564e0b7bbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:10.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.116+0000 7f00a8393740 4 client.16695 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:06:10.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.116+0000 7f00a8393740 1 -- 172.21.15.120:0/1203760291 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564e0b6a0d60 con 0x564e0b7b99b0 2024-09-19T09:06:10.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.116+0000 7f009effd640 1 --2- 172.21.15.120:0/1203760291 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564e0b7b99b0 0x564e0b7bbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:10.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.117+0000 7f007effd640 10 client.16695 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:10.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.117+0000 7f009effd640 1 --2- 172.21.15.120:0/1203760291 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564e0b7b99b0 0x564e0b7bbda0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dce2680d34b46f77 server_cookie=f9072013217d11b7 in_seq=0 out_seq=0 2024-09-19T09:06:10.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.118+0000 7f007effd640 1 -- 172.21.15.120:0/1203760291 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564e0b6a0d60 con 0x564e0b7b99b0 2024-09-19T09:06:10.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.118+0000 7f007effd640 10 client.16695 handle_command_reply: tid=0 2024-09-19T09:06:10.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.118+0000 7f00a8393740 1 client.16695 shutdown 2024-09-19T09:06:10.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.118+0000 7f00a8393740 1 -- 172.21.15.120:0/1203760291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e0b7334a0 msgr2=0x564e0b7424e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:10.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.118+0000 7f00a8393740 1 --2- 172.21.15.120:0/1203760291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e0b7334a0 0x564e0b7424e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f00900096c0 tx=0x7f009008b350 comp rx=0 tx=0).stop 2024-09-19T09:06:10.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.118+0000 7f00a8393740 1 -- 172.21.15.120:0/1203760291 shutdown_connections 2024-09-19T09:06:10.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.118+0000 7f00a8393740 1 --2- 172.21.15.120:0/1203760291 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564e0b7b99b0 0x564e0b7bbda0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.118+0000 7f00a8393740 1 --2- 172.21.15.120:0/1203760291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e0b73e680 0x564e0b742f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.118+0000 7f00a8393740 1 --2- 172.21.15.120:0/1203760291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e0b733dc0 0x564e0b742a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.118+0000 7f00a8393740 1 --2- 172.21.15.120:0/1203760291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e0b7334a0 0x564e0b7424e0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.118+0000 7f007effd640 0 client.16695 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:10.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.118+0000 7f00a8393740 1 -- 172.21.15.120:0/1203760291 >> 172.21.15.120:0/1203760291 conn(0x564e0b72fa30 msgr2=0x564e0b73d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:10.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.118+0000 7f00a8393740 1 -- 172.21.15.120:0/1203760291 shutdown_connections 2024-09-19T09:06:10.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.118+0000 7f00a8393740 1 -- 172.21.15.120:0/1203760291 wait complete. 2024-09-19T09:06:10.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.118+0000 7f00a8393740 20 client.16695 trim_cache size 0 max 16384 2024-09-19T09:06:10.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.119+0000 7f00a5fba640 1 -- 172.21.15.120:0/2481028638 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f00880a32c0 msgr2=0x7f00880a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:10.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.119+0000 7f00a5fba640 1 --2- 172.21.15.120:0/2481028638 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f00880a32c0 0x7f00880a5780 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f00a0076510 tx=0x7f0094005f90 comp rx=0 tx=0).stop 2024-09-19T09:06:10.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.119+0000 7f00a5fba640 1 -- 172.21.15.120:0/2481028638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00a0101df0 msgr2=0x7f00a019a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:10.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.119+0000 7f00a5fba640 1 --2- 172.21.15.120:0/2481028638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00a0101df0 0x7f00a019a6c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0090006010 tx=0x7f00900042f0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.119+0000 7f00a5fba640 1 -- 172.21.15.120:0/2481028638 shutdown_connections 2024-09-19T09:06:10.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.119+0000 7f00a5fba640 1 --2- 172.21.15.120:0/2481028638 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f00880a32c0 0x7f00880a5780 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.119+0000 7f00a5fba640 1 --2- 172.21.15.120:0/2481028638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00a0106f20 0x7f00a019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.119+0000 7f00a5fba640 1 --2- 172.21.15.120:0/2481028638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00a01027a0 0x7f00a019ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.119+0000 7f00a5fba640 1 --2- 172.21.15.120:0/2481028638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00a0101df0 0x7f00a019a6c0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:10.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.119+0000 7f00a5fba640 1 -- 172.21.15.120:0/2481028638 >> 172.21.15.120:0/2481028638 conn(0x7f00a00fd8f0 msgr2=0x7f00a00fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:10.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.119+0000 7f00a5fba640 1 -- 172.21.15.120:0/2481028638 shutdown_connections 2024-09-19T09:06:10.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:10.119+0000 7f00a5fba640 1 -- 172.21.15.120:0/2481028638 wait complete. 2024-09-19T09:06:10.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:09 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:06:10.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:09 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:06:10.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:09 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:06:10.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:09 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:06:10.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:09 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:06:10.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:09 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:06:10.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:09 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:06:10.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:09 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:06:10.941 INFO:tasks.cephfs.filesystem:scrub status for tag:48a62073-f66c-498c-b1d3-611b2faee032 - {'path': '/', 'tag': '48a62073-f66c-498c-b1d3-611b2faee032', 'options': 'recursive,force'} 2024-09-19T09:06:11.041 INFO:tasks.workunit.client.0.smithi120.stdout: 1 244236 14.90 MB/sec execute 6 sec latency 15.506 ms 2024-09-19T09:06:11.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:10 smithi196 ceph-mon[35150]: pgmap v440: 97 pgs: 97 active+clean; 43 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.6 MiB/s wr, 485 op/s 2024-09-19T09:06:11.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:10 smithi153 ceph-mon[35071]: pgmap v440: 97 pgs: 97 active+clean; 43 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.6 MiB/s wr, 485 op/s 2024-09-19T09:06:11.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:10 smithi120 ceph-mon[31150]: pgmap v440: 97 pgs: 97 active+clean; 43 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.6 MiB/s wr, 485 op/s 2024-09-19T09:06:12.041 INFO:tasks.workunit.client.0.smithi120.stdout: 1 247155 14.69 MB/sec execute 7 sec latency 11.633 ms 2024-09-19T09:06:13.041 INFO:tasks.workunit.client.0.smithi120.stdout: 1 249561 14.64 MB/sec execute 8 sec latency 10.561 ms 2024-09-19T09:06:13.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:12 smithi196 ceph-mon[35150]: pgmap v441: 97 pgs: 97 active+clean; 39 MiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 426 op/s 2024-09-19T09:06:13.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:12 smithi153 ceph-mon[35071]: pgmap v441: 97 pgs: 97 active+clean; 39 MiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 426 op/s 2024-09-19T09:06:13.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:12 smithi120 ceph-mon[31150]: pgmap v441: 97 pgs: 97 active+clean; 39 MiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 426 op/s 2024-09-19T09:06:13.705 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 16.6 2024-09-19T09:06:13.706 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:06:13.963 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:06:14.041 INFO:tasks.workunit.client.0.smithi120.stdout: 1 252634 14.95 MB/sec execute 9 sec latency 10.255 ms 2024-09-19T09:06:14.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:13 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:06:14.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:13 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:06:14.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:13 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:06:14.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.614+0000 7f4dfdbcc640 1 -- 172.21.15.120:0/3435305273 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4df8101de0 msgr2=0x7f4df81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:14.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.614+0000 7f4dfdbcc640 1 --2- 172.21.15.120:0/3435305273 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4df8101de0 0x7f4df81021c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009900 tx=0x7f4de802a410 comp rx=0 tx=0).stop 2024-09-19T09:06:14.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.614+0000 7f4dfdbcc640 1 -- 172.21.15.120:0/3435305273 shutdown_connections 2024-09-19T09:06:14.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.614+0000 7f4dfdbcc640 1 --2- 172.21.15.120:0/3435305273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4df8106f10 0x7f4df810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:14.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.614+0000 7f4dfdbcc640 1 --2- 172.21.15.120:0/3435305273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4df8102790 0x7f4df81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:14.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.614+0000 7f4dfdbcc640 1 --2- 172.21.15.120:0/3435305273 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4df8101de0 0x7f4df81021c0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:14.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.614+0000 7f4dfdbcc640 1 -- 172.21.15.120:0/3435305273 >> 172.21.15.120:0/3435305273 conn(0x7f4df80fd8e0 msgr2=0x7f4df80ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:14.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.614+0000 7f4dfdbcc640 1 -- 172.21.15.120:0/3435305273 shutdown_connections 2024-09-19T09:06:14.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.614+0000 7f4dfdbcc640 1 -- 172.21.15.120:0/3435305273 wait complete. 2024-09-19T09:06:14.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.615+0000 7f4dfdbcc640 1 Processor -- start 2024-09-19T09:06:14.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.615+0000 7f4dfdbcc640 1 -- start start 2024-09-19T09:06:14.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.615+0000 7f4dfdbcc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4df8101de0 0x7f4df8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:14.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.615+0000 7f4dfdbcc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4df8102790 0x7f4df8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:14.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.615+0000 7f4dfdbcc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4df8106f10 0x7f4df819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:14.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.615+0000 7f4dfdbcc640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4df8111f70 con 0x7f4df8106f10 2024-09-19T09:06:14.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.615+0000 7f4dfdbcc640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4df8111df0 con 0x7f4df8101de0 2024-09-19T09:06:14.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.615+0000 7f4dfdbcc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4df81120f0 con 0x7f4df8102790 2024-09-19T09:06:14.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.615+0000 7f4df7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4df8106f10 0x7f4df819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:14.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.615+0000 7f4df77fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4df8101de0 0x7f4df8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:14.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.615+0000 7f4df7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4df8106f10 0x7f4df819cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40852/0 (socket says 172.21.15.120:40852) 2024-09-19T09:06:14.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.615+0000 7f4df7fff640 1 -- 172.21.15.120:0/2956323956 learned_addr learned my addr 172.21.15.120:0/2956323956 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:14.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.615+0000 7f4deffff640 1 --2- 172.21.15.120:0/2956323956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4df8102790 0x7f4df8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:14.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.616+0000 7f4df7fff640 1 -- 172.21.15.120:0/2956323956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4df8102790 msgr2=0x7f4df8198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:14.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.616+0000 7f4df7fff640 1 --2- 172.21.15.120:0/2956323956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4df8102790 0x7f4df8198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:14.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.616+0000 7f4df7fff640 1 -- 172.21.15.120:0/2956323956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4df8101de0 msgr2=0x7f4df8198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:14.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.616+0000 7f4df7fff640 1 --2- 172.21.15.120:0/2956323956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4df8101de0 0x7f4df8198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:14.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.616+0000 7f4df7fff640 1 -- 172.21.15.120:0/2956323956 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df819d3e0 con 0x7f4df8106f10 2024-09-19T09:06:14.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.616+0000 7f4df77fe640 1 --2- 172.21.15.120:0/2956323956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4df8101de0 0x7f4df8198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:14.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.616+0000 7f4df7fff640 1 --2- 172.21.15.120:0/2956323956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4df8106f10 0x7f4df819cd00 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4df000eb10 tx=0x7f4df000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:14.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.616+0000 7f4df57fa640 1 -- 172.21.15.120:0/2956323956 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df0004b20 con 0x7f4df8106f10 2024-09-19T09:06:14.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.616+0000 7f4df57fa640 1 -- 172.21.15.120:0/2956323956 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4df0004cc0 con 0x7f4df8106f10 2024-09-19T09:06:14.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.616+0000 7f4dfdbcc640 1 -- 172.21.15.120:0/2956323956 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4df819d6d0 con 0x7f4df8106f10 2024-09-19T09:06:14.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.616+0000 7f4df57fa640 1 -- 172.21.15.120:0/2956323956 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df0010760 con 0x7f4df8106f10 2024-09-19T09:06:14.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.616+0000 7f4dfdbcc640 1 -- 172.21.15.120:0/2956323956 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4df8074420 con 0x7f4df8106f10 2024-09-19T09:06:14.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.617+0000 7f4dfdbcc640 1 -- 172.21.15.120:0/2956323956 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4df81032c0 con 0x7f4df8106f10 2024-09-19T09:06:14.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.618+0000 7f4df57fa640 1 -- 172.21.15.120:0/2956323956 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f4df0005020 con 0x7f4df8106f10 2024-09-19T09:06:14.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.619+0000 7f4df57fa640 1 --2- 172.21.15.120:0/2956323956 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4de40a3240 0x7f4de40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:14.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.619+0000 7f4df77fe640 1 --2- 172.21.15.120:0/2956323956 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4de40a3240 0x7f4de40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:14.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.619+0000 7f4df57fa640 1 -- 172.21.15.120:0/2956323956 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f4df00cfe00 con 0x7f4df8106f10 2024-09-19T09:06:14.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.619+0000 7f4df77fe640 1 --2- 172.21.15.120:0/2956323956 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4de40a3240 0x7f4de40a5700 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009230 tx=0x7f4de80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:14.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.623+0000 7f4df57fa640 1 -- 172.21.15.120:0/2956323956 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4df009cb20 con 0x7f4df8106f10 2024-09-19T09:06:14.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.838+0000 7f4dfdbcc640 1 -- 172.21.15.120:0/2956323956 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4df810a0f0 con 0x7f4df8106f10 2024-09-19T09:06:14.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.839+0000 7f4df57fa640 1 -- 172.21.15.120:0/2956323956 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f4df009c420 con 0x7f4df8106f10 2024-09-19T09:06:14.842 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:06:14.842 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":28,"btime":"2024-09-19T09:02:41:696728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:02:41.207357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:06:14.843 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 28 2024-09-19T09:06:14.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.841+0000 7f4dfdbcc640 1 -- 172.21.15.120:0/2956323956 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4de40a3240 msgr2=0x7f4de40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:14.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.841+0000 7f4dfdbcc640 1 --2- 172.21.15.120:0/2956323956 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4de40a3240 0x7f4de40a5700 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009230 tx=0x7f4de80058d0 comp rx=0 tx=0).stop 2024-09-19T09:06:14.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.841+0000 7f4dfdbcc640 1 -- 172.21.15.120:0/2956323956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4df8106f10 msgr2=0x7f4df819cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:14.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.841+0000 7f4dfdbcc640 1 --2- 172.21.15.120:0/2956323956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4df8106f10 0x7f4df819cd00 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4df000eb10 tx=0x7f4df000c2d0 comp rx=0 tx=0).stop 2024-09-19T09:06:14.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.841+0000 7f4dfdbcc640 1 -- 172.21.15.120:0/2956323956 shutdown_connections 2024-09-19T09:06:14.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.841+0000 7f4dfdbcc640 1 --2- 172.21.15.120:0/2956323956 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4de40a3240 0x7f4de40a5700 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:14.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.841+0000 7f4dfdbcc640 1 --2- 172.21.15.120:0/2956323956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4df8106f10 0x7f4df819cd00 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:14.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.841+0000 7f4dfdbcc640 1 --2- 172.21.15.120:0/2956323956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4df8102790 0x7f4df8198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:14.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.841+0000 7f4dfdbcc640 1 --2- 172.21.15.120:0/2956323956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4df8101de0 0x7f4df8198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:14.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.841+0000 7f4dfdbcc640 1 -- 172.21.15.120:0/2956323956 >> 172.21.15.120:0/2956323956 conn(0x7f4df80fd8e0 msgr2=0x7f4df80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:14.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.841+0000 7f4dfdbcc640 1 -- 172.21.15.120:0/2956323956 shutdown_connections 2024-09-19T09:06:14.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:14.841+0000 7f4dfdbcc640 1 -- 172.21.15.120:0/2956323956 wait complete. 2024-09-19T09:06:15.041 INFO:tasks.workunit.client.0.smithi120.stdout: 1 255537 15.19 MB/sec execute 10 sec latency 70.331 ms 2024-09-19T09:06:15.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:14 smithi120 ceph-mon[31150]: pgmap v442: 97 pgs: 97 active+clean; 42 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.2 MiB/s wr, 450 op/s 2024-09-19T09:06:15.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:14 smithi196 ceph-mon[35150]: pgmap v442: 97 pgs: 97 active+clean; 42 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.2 MiB/s wr, 450 op/s 2024-09-19T09:06:15.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:14 smithi153 ceph-mon[35071]: pgmap v442: 97 pgs: 97 active+clean; 42 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.2 MiB/s wr, 450 op/s 2024-09-19T09:06:15.668 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0' 2024-09-19T09:06:15.668 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0 2024-09-19T09:06:15.923 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:06:16.041 INFO:tasks.workunit.client.0.smithi120.stdout: 1 258394 15.08 MB/sec execute 11 sec latency 12.362 ms 2024-09-19T09:06:16.043 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:15 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2956323956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:06:16.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:15 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2956323956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:06:16.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:15 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2956323956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:06:16.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.428+0000 7f44e6511640 1 -- 172.21.15.120:0/3838944517 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44e0106f00 msgr2=0x7f44e010ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:16.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.428+0000 7f44e6511640 1 --2- 172.21.15.120:0/3838944517 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44e0106f00 0x7f44e010ea80 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009ca0 tx=0x7f44d802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.428+0000 7f44e6511640 1 -- 172.21.15.120:0/3838944517 shutdown_connections 2024-09-19T09:06:16.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.428+0000 7f44e6511640 1 --2- 172.21.15.120:0/3838944517 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44e0106f00 0x7f44e010ea80 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.428+0000 7f44e6511640 1 --2- 172.21.15.120:0/3838944517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44e0102780 0x7f44e01067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.428+0000 7f44e6511640 1 --2- 172.21.15.120:0/3838944517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44e0101dd0 0x7f44e01021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.428+0000 7f44e6511640 1 -- 172.21.15.120:0/3838944517 >> 172.21.15.120:0/3838944517 conn(0x7f44e00fd890 msgr2=0x7f44e00ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:16.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.428+0000 7f44e6511640 1 -- 172.21.15.120:0/3838944517 shutdown_connections 2024-09-19T09:06:16.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.428+0000 7f44e6511640 1 -- 172.21.15.120:0/3838944517 wait complete. 2024-09-19T09:06:16.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.429+0000 7f44e6511640 1 Processor -- start 2024-09-19T09:06:16.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.429+0000 7f44e6511640 1 -- start start 2024-09-19T09:06:16.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.429+0000 7f44e6511640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44e0101dd0 0x7f44e01983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:16.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.429+0000 7f44e6511640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44e0102780 0x7f44e0198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:16.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.429+0000 7f44e6511640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44e0106f00 0x7f44e019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:16.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.429+0000 7f44e6511640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f44e0111f60 con 0x7f44e0101dd0 2024-09-19T09:06:16.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.429+0000 7f44e6511640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f44e0111de0 con 0x7f44e0106f00 2024-09-19T09:06:16.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.429+0000 7f44e6511640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f44e01120e0 con 0x7f44e0102780 2024-09-19T09:06:16.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.429+0000 7f44dffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44e0101dd0 0x7f44e01983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:16.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.430+0000 7f44dffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44e0101dd0 0x7f44e01983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40872/0 (socket says 172.21.15.120:40872) 2024-09-19T09:06:16.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.430+0000 7f44dffff640 1 -- 172.21.15.120:0/3461311221 learned_addr learned my addr 172.21.15.120:0/3461311221 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:16.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.430+0000 7f44df7fe640 1 --2- 172.21.15.120:0/3461311221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44e0102780 0x7f44e0198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:16.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.430+0000 7f44e4aae640 1 --2- 172.21.15.120:0/3461311221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44e0106f00 0x7f44e019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:16.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.430+0000 7f44dffff640 1 -- 172.21.15.120:0/3461311221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44e0102780 msgr2=0x7f44e0198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:16.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.430+0000 7f44dffff640 1 --2- 172.21.15.120:0/3461311221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44e0102780 0x7f44e0198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.430+0000 7f44dffff640 1 -- 172.21.15.120:0/3461311221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44e0106f00 msgr2=0x7f44e019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:16.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.430+0000 7f44dffff640 1 --2- 172.21.15.120:0/3461311221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44e0106f00 0x7f44e019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.430+0000 7f44dffff640 1 -- 172.21.15.120:0/3461311221 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e019d380 con 0x7f44e0101dd0 2024-09-19T09:06:16.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.430+0000 7f44dffff640 1 --2- 172.21.15.120:0/3461311221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44e0101dd0 0x7f44e01983d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f44d000e9f0 tx=0x7f44d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:16.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.430+0000 7f44dd7fa640 1 -- 172.21.15.120:0/3461311221 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d000cd80 con 0x7f44e0101dd0 2024-09-19T09:06:16.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.430+0000 7f44dd7fa640 1 -- 172.21.15.120:0/3461311221 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44d0010910 con 0x7f44e0101dd0 2024-09-19T09:06:16.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.430+0000 7f44e6511640 1 -- 172.21.15.120:0/3461311221 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e019d670 con 0x7f44e0101dd0 2024-09-19T09:06:16.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.430+0000 7f44dd7fa640 1 -- 172.21.15.120:0/3461311221 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d0018990 con 0x7f44e0101dd0 2024-09-19T09:06:16.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.430+0000 7f44e6511640 1 -- 172.21.15.120:0/3461311221 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e01a4f50 con 0x7f44e0101dd0 2024-09-19T09:06:16.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.432+0000 7f44dd7fa640 1 -- 172.21.15.120:0/3461311221 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f44d0010ac0 con 0x7f44e0101dd0 2024-09-19T09:06:16.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.433+0000 7f44e88ea740 1 Processor -- start 2024-09-19T09:06:16.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.433+0000 7f44e88ea740 1 -- start start 2024-09-19T09:06:16.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.433+0000 7f44e88ea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56174ec40b70 0x56174ec40f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:16.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.433+0000 7f44e88ea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56174ec41490 0x56174ec4b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:16.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.433+0000 7f44e88ea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56174ec4bd50 0x56174ec4e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:16.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.433+0000 7f44e88ea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56174eba22a0 con 0x56174ec4bd50 2024-09-19T09:06:16.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.433+0000 7f44e88ea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56174eb49b10 con 0x56174ec40b70 2024-09-19T09:06:16.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.433+0000 7f44e88ea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56174eb9e260 con 0x56174ec41490 2024-09-19T09:06:16.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.433+0000 7f44e4aae640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56174ec40b70 0x56174ec40f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:16.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.433+0000 7f44dffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56174ec41490 0x56174ec4b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:16.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.433+0000 7f44e4aae640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56174ec40b70 0x56174ec40f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:56190/0 (socket says 172.21.15.120:56190) 2024-09-19T09:06:16.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.433+0000 7f44e4aae640 1 -- 172.21.15.120:0/4065336164 learned_addr learned my addr 172.21.15.120:0/4065336164 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:16.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.433+0000 7f44df7fe640 1 --2- 172.21.15.120:0/4065336164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56174ec4bd50 0x56174ec4e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:16.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.433+0000 7f44dd7fa640 1 --2- 172.21.15.120:0/3461311221 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f44c80a3240 0x7f44c80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:16.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.433+0000 7f44dffff640 1 -- 172.21.15.120:0/4065336164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56174ec40b70 msgr2=0x56174ec40f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:16.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.433+0000 7f44dffff640 1 --2- 172.21.15.120:0/4065336164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56174ec40b70 0x56174ec40f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.433+0000 7f44dffff640 1 -- 172.21.15.120:0/4065336164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56174ec4bd50 msgr2=0x56174ec4e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:16.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.433+0000 7f44dffff640 1 --2- 172.21.15.120:0/4065336164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56174ec4bd50 0x56174ec4e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.433+0000 7f44dffff640 1 -- 172.21.15.120:0/4065336164 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56174e913b90 con 0x56174ec41490 2024-09-19T09:06:16.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.434+0000 7f44e4aae640 1 --2- 172.21.15.120:0/4065336164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56174ec40b70 0x56174ec40f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:16.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.434+0000 7f44df7fe640 1 --2- 172.21.15.120:0/4065336164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56174ec4bd50 0x56174ec4e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:16.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.434+0000 7f44e4aae640 1 --2- 172.21.15.120:0/3461311221 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f44c80a3240 0x7f44c80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:16.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.434+0000 7f44dd7fa640 1 -- 172.21.15.120:0/3461311221 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f44d00cfc00 con 0x7f44e0101dd0 2024-09-19T09:06:16.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.434+0000 7f44dffff640 1 --2- 172.21.15.120:0/4065336164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56174ec41490 0x56174ec4b810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f44d00cd6a0 tx=0x7f44d0066dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87f1d0e809cf720d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:16.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.434+0000 7f44c67fc640 1 -- 172.21.15.120:0/4065336164 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d00bae60 con 0x56174ec41490 2024-09-19T09:06:16.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.434+0000 7f44c67fc640 1 -- 172.21.15.120:0/4065336164 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44d006aee0 con 0x56174ec41490 2024-09-19T09:06:16.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.434+0000 7f44c67fc640 1 -- 172.21.15.120:0/4065336164 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d009c2d0 con 0x56174ec41490 2024-09-19T09:06:16.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.435+0000 7f44e88ea740 1 -- 172.21.15.120:0/4065336164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56174ec41490 msgr2=0x56174ec4b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:16.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.435+0000 7f44e88ea740 1 --2- 172.21.15.120:0/4065336164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56174ec41490 0x56174ec4b810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f44d00cd6a0 tx=0x7f44d0066dd0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.435+0000 7f44e4aae640 1 --2- 172.21.15.120:0/3461311221 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f44c80a3240 0x7f44c80a5700 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f44d802f540 tx=0x7f44d80095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:16.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.435+0000 7f44e88ea740 1 -- 172.21.15.120:0/4065336164 shutdown_connections 2024-09-19T09:06:16.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.435+0000 7f44e88ea740 1 --2- 172.21.15.120:0/4065336164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56174ec4bd50 0x56174ec4e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.435+0000 7f44e88ea740 1 --2- 172.21.15.120:0/4065336164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56174ec41490 0x56174ec4b810 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.435+0000 7f44e88ea740 1 --2- 172.21.15.120:0/4065336164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56174ec40b70 0x56174ec40f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.435+0000 7f44e88ea740 1 -- 172.21.15.120:0/4065336164 >> 172.21.15.120:0/4065336164 conn(0x56174ec3d100 msgr2=0x56174ec3d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:16.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.435+0000 7f44e88ea740 1 -- 172.21.15.120:0/4065336164 shutdown_connections 2024-09-19T09:06:16.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.435+0000 7f44e88ea740 1 -- 172.21.15.120:0/4065336164 wait complete. 2024-09-19T09:06:16.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.435+0000 7f44e88ea740 1 -- start start 2024-09-19T09:06:16.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.435+0000 7f44e88ea740 1 Processor -- start 2024-09-19T09:06:16.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.436+0000 7f44e88ea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56174ec40b70 0x56174ec48c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:16.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.436+0000 7f44e88ea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56174ec41490 0x56174ec49140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:16.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.436+0000 7f44e88ea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56174ec4bd50 0x56174ec49680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:16.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.436+0000 7f44e88ea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56174ec4e8c0 con 0x56174ec40b70 2024-09-19T09:06:16.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.436+0000 7f44e88ea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56174eb2a6e0 con 0x56174ec41490 2024-09-19T09:06:16.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.436+0000 7f44e88ea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56174ec4ea40 con 0x56174ec4bd50 2024-09-19T09:06:16.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.436+0000 7f44e88ea740 10 client.0 _ll_register_callbacks cb 0x56174eb4ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:16.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.436+0000 7f44dffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56174ec40b70 0x56174ec48c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:16.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.436+0000 7f44df7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56174ec41490 0x56174ec49140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:16.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.436+0000 7f44dffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56174ec40b70 0x56174ec48c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40902/0 (socket says 172.21.15.120:40902) 2024-09-19T09:06:16.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.436+0000 7f44dffff640 1 -- 172.21.15.120:0/3738990619 learned_addr learned my addr 172.21.15.120:0/3738990619 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:16.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.436+0000 7f44e4aae640 1 --2- 172.21.15.120:0/3738990619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56174ec4bd50 0x56174ec49680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:16.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.436+0000 7f44dffff640 1 -- 172.21.15.120:0/3738990619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56174ec4bd50 msgr2=0x56174ec49680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:16.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.436+0000 7f44dffff640 1 --2- 172.21.15.120:0/3738990619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56174ec4bd50 0x56174ec49680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.436+0000 7f44dffff640 1 -- 172.21.15.120:0/3738990619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56174ec41490 msgr2=0x56174ec49140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:16.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.436+0000 7f44dffff640 1 --2- 172.21.15.120:0/3738990619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56174ec41490 0x56174ec49140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.436+0000 7f44dffff640 1 -- 172.21.15.120:0/3738990619 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56174ec57d60 con 0x56174ec40b70 2024-09-19T09:06:16.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.436+0000 7f44dffff640 1 --2- 172.21.15.120:0/3738990619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56174ec40b70 0x56174ec48c00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f44d00a4960 tx=0x7f44d00cfea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396e8774a944c508 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:16.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.436+0000 7f44bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:06:16.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.436+0000 7f44bf7fe640 1 -- 172.21.15.120:0/3738990619 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d00bae60 con 0x56174ec40b70 2024-09-19T09:06:16.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.436+0000 7f44bf7fe640 1 -- 172.21.15.120:0/3738990619 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f44d009a5d0 con 0x56174ec40b70 2024-09-19T09:06:16.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.436+0000 7f44e88ea740 1 -- 172.21.15.120:0/3738990619 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44d006aee0 con 0x56174ec40b70 2024-09-19T09:06:16.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.437+0000 7f44bf7fe640 1 -- 172.21.15.120:0/3738990619 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f44d009be10 con 0x56174ec40b70 2024-09-19T09:06:16.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.437+0000 7f44e88ea740 10 client.16713 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:16.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.437+0000 7f44e88ea740 1 -- 172.21.15.120:0/3738990619 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56174ec4e8c0 con 0x56174ec40b70 2024-09-19T09:06:16.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.437+0000 7f44bf7fe640 1 -- 172.21.15.120:0/3738990619 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44d006aee0 con 0x56174ec40b70 2024-09-19T09:06:16.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.437+0000 7f44bf7fe640 1 -- 172.21.15.120:0/3738990619 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f44d00d31d0 con 0x56174ec40b70 2024-09-19T09:06:16.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.437+0000 7f44e88ea740 10 client.16713 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:16.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.437+0000 7f44e88ea740 10 client.16713 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:06:16.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.437+0000 7f44e88ea740 20 client.16713 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:16.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.437+0000 7f44e88ea740 1 --2- 172.21.15.120:0/3738990619 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56174ec5b870 0x56174ec5dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:16.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.437+0000 7f44e88ea740 4 client.16713 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:06:16.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.437+0000 7f44e88ea740 1 -- 172.21.15.120:0/3738990619 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56174ebaf9d0 con 0x56174ec5b870 2024-09-19T09:06:16.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.437+0000 7f44df7fe640 1 --2- 172.21.15.120:0/3738990619 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56174ec5b870 0x56174ec5dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:16.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.438+0000 7f44bf7fe640 10 client.16713 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:16.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.438+0000 7f44df7fe640 1 --2- 172.21.15.120:0/3738990619 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56174ec5b870 0x56174ec5dc60 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b921ada5bf9aef13 server_cookie=6ea4b1b9c28f05db in_seq=0 out_seq=0 2024-09-19T09:06:16.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.440+0000 7f44bf7fe640 1 -- 172.21.15.120:0/3738990619 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56174ebaf9d0 con 0x56174ec5b870 2024-09-19T09:06:16.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.440+0000 7f44bf7fe640 10 client.16713 handle_command_reply: tid=0 2024-09-19T09:06:16.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.440+0000 7f44e88ea740 1 client.16713 shutdown 2024-09-19T09:06:16.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.440+0000 7f44e88ea740 1 -- 172.21.15.120:0/3738990619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56174ec40b70 msgr2=0x56174ec48c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:16.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.440+0000 7f44e88ea740 1 --2- 172.21.15.120:0/3738990619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56174ec40b70 0x56174ec48c00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f44d00a4960 tx=0x7f44d00cfea0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.440+0000 7f44e88ea740 1 -- 172.21.15.120:0/3738990619 shutdown_connections 2024-09-19T09:06:16.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.440+0000 7f44e88ea740 1 --2- 172.21.15.120:0/3738990619 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56174ec5b870 0x56174ec5dc60 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.440+0000 7f44e88ea740 1 --2- 172.21.15.120:0/3738990619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56174ec4bd50 0x56174ec49680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.440+0000 7f44e88ea740 1 --2- 172.21.15.120:0/3738990619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56174ec41490 0x56174ec49140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.440+0000 7f44bf7fe640 0 client.16713 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:16.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.440+0000 7f44e88ea740 1 --2- 172.21.15.120:0/3738990619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56174ec40b70 0x56174ec48c00 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.440+0000 7f44e88ea740 1 -- 172.21.15.120:0/3738990619 >> 172.21.15.120:0/3738990619 conn(0x56174ec3d100 msgr2=0x56174ec37e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:16.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.440+0000 7f44e88ea740 1 -- 172.21.15.120:0/3738990619 shutdown_connections 2024-09-19T09:06:16.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.440+0000 7f44e88ea740 1 -- 172.21.15.120:0/3738990619 wait complete. 2024-09-19T09:06:16.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.440+0000 7f44e88ea740 20 client.16713 trim_cache size 0 max 16384 2024-09-19T09:06:16.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.457+0000 7f44e88ea740 1 Processor -- start 2024-09-19T09:06:16.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.457+0000 7f44e88ea740 1 -- start start 2024-09-19T09:06:16.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.457+0000 7f44e88ea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56174ec40b70 0x56174ec435d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:16.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.457+0000 7f44e88ea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56174ec41490 0x56174ec4aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:16.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.457+0000 7f44e88ea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56174ec4bd50 0x56174ec633d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:16.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.457+0000 7f44e88ea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f44e01ab3f0 con 0x56174ec4bd50 2024-09-19T09:06:16.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.457+0000 7f44e88ea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f44e01acf30 con 0x56174ec40b70 2024-09-19T09:06:16.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.457+0000 7f44e88ea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f44e01abac0 con 0x56174ec41490 2024-09-19T09:06:16.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.457+0000 7f44e4aae640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56174ec4bd50 0x56174ec633d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:16.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.457+0000 7f44e4aae640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56174ec4bd50 0x56174ec633d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40906/0 (socket says 172.21.15.120:40906) 2024-09-19T09:06:16.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.457+0000 7f44e4aae640 1 -- 172.21.15.120:0/2334069395 learned_addr learned my addr 172.21.15.120:0/2334069395 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:16.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.457+0000 7f44dffff640 1 --2- 172.21.15.120:0/2334069395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56174ec40b70 0x56174ec435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:16.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.457+0000 7f44df7fe640 1 --2- 172.21.15.120:0/2334069395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56174ec41490 0x56174ec4aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:16.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.457+0000 7f44e4aae640 1 -- 172.21.15.120:0/2334069395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56174ec41490 msgr2=0x56174ec4aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:16.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.457+0000 7f44e4aae640 1 --2- 172.21.15.120:0/2334069395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56174ec41490 0x56174ec4aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.457+0000 7f44e4aae640 1 -- 172.21.15.120:0/2334069395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56174ec40b70 msgr2=0x56174ec435d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:16.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.457+0000 7f44e4aae640 1 --2- 172.21.15.120:0/2334069395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56174ec40b70 0x56174ec435d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.457+0000 7f44e4aae640 1 -- 172.21.15.120:0/2334069395 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56174ec3cf30 con 0x56174ec4bd50 2024-09-19T09:06:16.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.458+0000 7f44e4aae640 1 --2- 172.21.15.120:0/2334069395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56174ec4bd50 0x56174ec633d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f44d8006010 tx=0x7f44d8043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107c14ffc29d6490 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:16.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.458+0000 7f44c5ffb640 1 -- 172.21.15.120:0/2334069395 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d802fa60 con 0x56174ec4bd50 2024-09-19T09:06:16.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.458+0000 7f44c5ffb640 1 -- 172.21.15.120:0/2334069395 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44d8036040 con 0x56174ec4bd50 2024-09-19T09:06:16.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.458+0000 7f44c5ffb640 1 -- 172.21.15.120:0/2334069395 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d8004580 con 0x56174ec4bd50 2024-09-19T09:06:16.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.458+0000 7f44e88ea740 1 -- 172.21.15.120:0/2334069395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56174ec4bd50 msgr2=0x56174ec633d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:16.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.458+0000 7f44e88ea740 1 --2- 172.21.15.120:0/2334069395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56174ec4bd50 0x56174ec633d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f44d8006010 tx=0x7f44d8043000 comp rx=0 tx=0).stop 2024-09-19T09:06:16.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.458+0000 7f44e88ea740 1 -- 172.21.15.120:0/2334069395 shutdown_connections 2024-09-19T09:06:16.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.458+0000 7f44e88ea740 1 --2- 172.21.15.120:0/2334069395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56174ec4bd50 0x56174ec633d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.458+0000 7f44e88ea740 1 --2- 172.21.15.120:0/2334069395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56174ec41490 0x56174ec4aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.458+0000 7f44e88ea740 1 --2- 172.21.15.120:0/2334069395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56174ec40b70 0x56174ec435d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.458+0000 7f44e88ea740 1 -- 172.21.15.120:0/2334069395 >> 172.21.15.120:0/2334069395 conn(0x56174ec3d100 msgr2=0x56174ec37e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:16.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.459+0000 7f44e88ea740 1 -- 172.21.15.120:0/2334069395 shutdown_connections 2024-09-19T09:06:16.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.459+0000 7f44e88ea740 1 -- 172.21.15.120:0/2334069395 wait complete. 2024-09-19T09:06:16.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.459+0000 7f44e88ea740 1 -- start start 2024-09-19T09:06:16.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.459+0000 7f44e88ea740 1 Processor -- start 2024-09-19T09:06:16.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.459+0000 7f44e88ea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56174ec40b70 0x56174ec50ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:16.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.459+0000 7f44e88ea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56174ec41490 0x56174ec511e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:16.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.459+0000 7f44e88ea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56174ec4bd50 0x56174ec51720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:16.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.459+0000 7f44e88ea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f44e01ac860 con 0x56174ec4bd50 2024-09-19T09:06:16.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.459+0000 7f44e88ea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f44e01ac190 con 0x56174ec41490 2024-09-19T09:06:16.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.459+0000 7f44e88ea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f44e01a93b0 con 0x56174ec40b70 2024-09-19T09:06:16.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.460+0000 7f44e88ea740 10 client.0 _ll_register_callbacks cb 0x7f44d0098430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:16.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.460+0000 7f44e4aae640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56174ec4bd50 0x56174ec51720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:16.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.460+0000 7f44dffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56174ec40b70 0x56174ec50ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:16.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.460+0000 7f44e4aae640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56174ec4bd50 0x56174ec51720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40920/0 (socket says 172.21.15.120:40920) 2024-09-19T09:06:16.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.460+0000 7f44dffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56174ec40b70 0x56174ec50ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:54800/0 (socket says 172.21.15.120:54800) 2024-09-19T09:06:16.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.460+0000 7f44e4aae640 1 -- 172.21.15.120:0/4225657928 learned_addr learned my addr 172.21.15.120:0/4225657928 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:16.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.460+0000 7f44df7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56174ec41490 0x56174ec511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:16.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.460+0000 7f44e4aae640 1 -- 172.21.15.120:0/4225657928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56174ec40b70 msgr2=0x56174ec50ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:16.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.460+0000 7f44e4aae640 1 --2- 172.21.15.120:0/4225657928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56174ec40b70 0x56174ec50ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.460+0000 7f44e4aae640 1 -- 172.21.15.120:0/4225657928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56174ec41490 msgr2=0x56174ec511e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:16.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.460+0000 7f44e4aae640 1 --2- 172.21.15.120:0/4225657928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56174ec41490 0x56174ec511e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.460+0000 7f44e4aae640 1 -- 172.21.15.120:0/4225657928 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56174ec34260 con 0x56174ec4bd50 2024-09-19T09:06:16.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.460+0000 7f44e4aae640 1 --2- 172.21.15.120:0/4225657928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56174ec4bd50 0x56174ec51720 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009bf0 tx=0x7f44d8004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77d385b1e0cab4d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:16.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.460+0000 7f44bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:06:16.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.460+0000 7f44bf7fe640 1 -- 172.21.15.120:0/4225657928 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d802f310 con 0x56174ec4bd50 2024-09-19T09:06:16.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.460+0000 7f44bf7fe640 1 -- 172.21.15.120:0/4225657928 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f44d8003b80 con 0x56174ec4bd50 2024-09-19T09:06:16.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.460+0000 7f44e88ea740 1 -- 172.21.15.120:0/4225657928 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44d8036040 con 0x56174ec4bd50 2024-09-19T09:06:16.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.460+0000 7f44bf7fe640 1 -- 172.21.15.120:0/4225657928 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f44d8003d80 con 0x56174ec4bd50 2024-09-19T09:06:16.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.460+0000 7f44e88ea740 10 client.16719 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:16.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.460+0000 7f44e88ea740 1 -- 172.21.15.120:0/4225657928 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56174ecc7740 con 0x56174ec4bd50 2024-09-19T09:06:16.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.461+0000 7f44bf7fe640 1 -- 172.21.15.120:0/4225657928 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44d8036040 con 0x56174ec4bd50 2024-09-19T09:06:16.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.461+0000 7f44bf7fe640 1 -- 172.21.15.120:0/4225657928 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f44d80308e0 con 0x56174ec4bd50 2024-09-19T09:06:16.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.461+0000 7f44e88ea740 10 client.16719 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:16.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.461+0000 7f44e88ea740 10 client.16719 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:06:16.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.461+0000 7f44e88ea740 20 client.16719 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:16.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.461+0000 7f44e88ea740 1 --2- 172.21.15.120:0/4225657928 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56174ecc7e10 0x56174ecca230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:16.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.461+0000 7f44e88ea740 4 client.16719 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}] 2024-09-19T09:06:16.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.461+0000 7f44e88ea740 1 -- 172.21.15.120:0/4225657928 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}) -- 0x56174eba9d20 con 0x56174ecc7e10 2024-09-19T09:06:16.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.461+0000 7f44dffff640 1 --2- 172.21.15.120:0/4225657928 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56174ecc7e10 0x56174ecca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:16.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.461+0000 7f44bf7fe640 10 client.16719 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:16.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.461+0000 7f44dffff640 1 --2- 172.21.15.120:0/4225657928 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56174ecc7e10 0x56174ecca230 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af8fab514434fb55 server_cookie=92034283dffd2779 in_seq=0 out_seq=0 2024-09-19T09:06:16.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.563+0000 7f44bf7fe640 1 -- 172.21.15.120:0/4225657928 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56174eba9d20 con 0x56174ecc7e10 2024-09-19T09:06:16.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.563+0000 7f44bf7fe640 10 client.16719 handle_command_reply: tid=0 2024-09-19T09:06:16.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.563+0000 7f44e88ea740 1 client.16719 shutdown 2024-09-19T09:06:16.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.563+0000 7f44e88ea740 1 -- 172.21.15.120:0/4225657928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56174ec4bd50 msgr2=0x56174ec51720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:16.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.563+0000 7f44e88ea740 1 --2- 172.21.15.120:0/4225657928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56174ec4bd50 0x56174ec51720 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009bf0 tx=0x7f44d8004200 comp rx=0 tx=0).stop 2024-09-19T09:06:16.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.563+0000 7f44e88ea740 1 -- 172.21.15.120:0/4225657928 shutdown_connections 2024-09-19T09:06:16.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.563+0000 7f44e88ea740 1 --2- 172.21.15.120:0/4225657928 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56174ecc7e10 0x56174ecca230 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.563+0000 7f44e88ea740 1 --2- 172.21.15.120:0/4225657928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56174ec4bd50 0x56174ec51720 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.563+0000 7f44e88ea740 1 --2- 172.21.15.120:0/4225657928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56174ec41490 0x56174ec511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.563+0000 7f44e88ea740 1 --2- 172.21.15.120:0/4225657928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56174ec40b70 0x56174ec50ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.563+0000 7f44e88ea740 1 -- 172.21.15.120:0/4225657928 >> 172.21.15.120:0/4225657928 conn(0x56174ec3d100 msgr2=0x56174ec37e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:16.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.563+0000 7f44bf7fe640 0 client.16719 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:16.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.563+0000 7f44e88ea740 1 -- 172.21.15.120:0/4225657928 shutdown_connections 2024-09-19T09:06:16.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.563+0000 7f44e88ea740 1 -- 172.21.15.120:0/4225657928 wait complete. 2024-09-19T09:06:16.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.563+0000 7f44e88ea740 20 client.16719 trim_cache size 0 max 16384 2024-09-19T09:06:16.565 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:06:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 28, 2024-09-19T09:06:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:06:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 14, 2024-09-19T09:06:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:06:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "b11bef3b": { 2024-09-19T09:06:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 14, 2024-09-19T09:06:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:06:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:06:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:06:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:06:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:06:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:06:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 136.6, 2024-09-19T09:06:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:06:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:06:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:06:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:06:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:06:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:06:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:06:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:06:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:06:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:06:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:06:16.567 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:06:16.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.564+0000 7f44e6511640 1 -- 172.21.15.120:0/3461311221 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f44c80a3240 msgr2=0x7f44c80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:16.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.564+0000 7f44e6511640 1 --2- 172.21.15.120:0/3461311221 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f44c80a3240 0x7f44c80a5700 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f44d802f540 tx=0x7f44d80095c0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.564+0000 7f44e6511640 1 -- 172.21.15.120:0/3461311221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44e0101dd0 msgr2=0x7f44e01983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:16.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.564+0000 7f44e6511640 1 --2- 172.21.15.120:0/3461311221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44e0101dd0 0x7f44e01983d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f44d000e9f0 tx=0x7f44d000eeb0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.564+0000 7f44e6511640 1 -- 172.21.15.120:0/3461311221 shutdown_connections 2024-09-19T09:06:16.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.564+0000 7f44e6511640 1 --2- 172.21.15.120:0/3461311221 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f44c80a3240 0x7f44c80a5700 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.564+0000 7f44e6511640 1 --2- 172.21.15.120:0/3461311221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44e0106f00 0x7f44e019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.564+0000 7f44e6511640 1 --2- 172.21.15.120:0/3461311221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44e0102780 0x7f44e0198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.564+0000 7f44e6511640 1 --2- 172.21.15.120:0/3461311221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44e0101dd0 0x7f44e01983d0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:16.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.564+0000 7f44e6511640 1 -- 172.21.15.120:0/3461311221 >> 172.21.15.120:0/3461311221 conn(0x7f44e00fd890 msgr2=0x7f44e00fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:16.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.564+0000 7f44e6511640 1 -- 172.21.15.120:0/3461311221 shutdown_connections 2024-09-19T09:06:16.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:16.564+0000 7f44e6511640 1 -- 172.21.15.120:0/3461311221 wait complete. 2024-09-19T09:06:17.041 INFO:tasks.workunit.client.0.smithi120.stdout: 1 259365 14.37 MB/sec execute 12 sec latency 534.046 ms 2024-09-19T09:06:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:16 smithi196 ceph-mon[35150]: pgmap v443: 97 pgs: 97 active+clean; 43 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 405 op/s 2024-09-19T09:06:17.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:16 smithi153 ceph-mon[35071]: pgmap v443: 97 pgs: 97 active+clean; 43 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 405 op/s 2024-09-19T09:06:17.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:16 smithi120 ceph-mon[31150]: pgmap v443: 97 pgs: 97 active+clean; 43 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 405 op/s 2024-09-19T09:06:17.471 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b11bef3b', quiesce took 3.8 seconds. Will release after: 12.8 2024-09-19T09:06:18.041 INFO:tasks.workunit.client.0.smithi120.stdout: 1 259365 13.26 MB/sec execute 13 sec latency 1534.074 ms 2024-09-19T09:06:19.041 INFO:tasks.workunit.client.0.smithi120.stdout: 1 259365 12.31 MB/sec execute 14 sec latency 2534.128 ms 2024-09-19T09:06:19.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:18 smithi120 ceph-mon[31150]: pgmap v444: 97 pgs: 97 active+clean; 45 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 430 op/s 2024-09-19T09:06:19.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:18 smithi196 ceph-mon[35150]: pgmap v444: 97 pgs: 97 active+clean; 45 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 430 op/s 2024-09-19T09:06:19.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:18 smithi153 ceph-mon[35071]: pgmap v444: 97 pgs: 97 active+clean; 45 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 430 op/s 2024-09-19T09:06:20.041 INFO:tasks.workunit.client.0.smithi120.stdout: 1 259365 11.49 MB/sec execute 15 sec latency 3534.436 ms 2024-09-19T09:06:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.680+0000 7fb955f6d640 1 -- 172.21.15.120:0/3019129967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9501025f0 msgr2=0x7fb950108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.680+0000 7fb955f6d640 1 --2- 172.21.15.120:0/3019129967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9501025f0 0x7fb950108e80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fb948009ca0 tx=0x7fb94802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.680+0000 7fb955f6d640 1 -- 172.21.15.120:0/3019129967 shutdown_connections 2024-09-19T09:06:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.680+0000 7fb955f6d640 1 --2- 172.21.15.120:0/3019129967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9501025f0 0x7fb950108e80 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.680+0000 7fb955f6d640 1 --2- 172.21.15.120:0/3019129967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb950101c30 0x7fb9501020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.680+0000 7fb955f6d640 1 --2- 172.21.15.120:0/3019129967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb950100a40 0x7fb950100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.680+0000 7fb955f6d640 1 -- 172.21.15.120:0/3019129967 >> 172.21.15.120:0/3019129967 conn(0x7fb9500fc1e0 msgr2=0x7fb9500fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.680+0000 7fb955f6d640 1 -- 172.21.15.120:0/3019129967 shutdown_connections 2024-09-19T09:06:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.680+0000 7fb955f6d640 1 -- 172.21.15.120:0/3019129967 wait complete. 2024-09-19T09:06:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.680+0000 7fb955f6d640 1 Processor -- start 2024-09-19T09:06:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.681+0000 7fb955f6d640 1 -- start start 2024-09-19T09:06:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.681+0000 7fb955f6d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb950100a40 0x7fb950197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.681+0000 7fb955f6d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb950101c30 0x7fb9501982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.681+0000 7fb955f6d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9501025f0 0x7fb95019f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.681+0000 7fb955f6d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb95010c770 con 0x7fb9501025f0 2024-09-19T09:06:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.681+0000 7fb955f6d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb95010c5f0 con 0x7fb950100a40 2024-09-19T09:06:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.681+0000 7fb955f6d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb95010c8f0 con 0x7fb950101c30 2024-09-19T09:06:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.681+0000 7fb94ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9501025f0 0x7fb95019f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.681+0000 7fb94ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9501025f0 0x7fb95019f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40946/0 (socket says 172.21.15.120:40946) 2024-09-19T09:06:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.681+0000 7fb94ffff640 1 -- 172.21.15.120:0/4192971253 learned_addr learned my addr 172.21.15.120:0/4192971253 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.681+0000 7fb94f7fe640 1 --2- 172.21.15.120:0/4192971253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb950100a40 0x7fb950197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.681+0000 7fb94effd640 1 --2- 172.21.15.120:0/4192971253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb950101c30 0x7fb9501982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.681+0000 7fb94ffff640 1 -- 172.21.15.120:0/4192971253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb950101c30 msgr2=0x7fb9501982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.681+0000 7fb94ffff640 1 --2- 172.21.15.120:0/4192971253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb950101c30 0x7fb9501982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.681+0000 7fb94ffff640 1 -- 172.21.15.120:0/4192971253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb950100a40 msgr2=0x7fb950197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.681+0000 7fb94ffff640 1 --2- 172.21.15.120:0/4192971253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb950100a40 0x7fb950197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.681+0000 7fb94ffff640 1 -- 172.21.15.120:0/4192971253 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95019f9b0 con 0x7fb9501025f0 2024-09-19T09:06:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.682+0000 7fb94ffff640 1 --2- 172.21.15.120:0/4192971253 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9501025f0 0x7fb95019f310 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb948003ba0 tx=0x7fb948003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.682+0000 7fb94cff9640 1 -- 172.21.15.120:0/4192971253 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb948004040 con 0x7fb9501025f0 2024-09-19T09:06:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.682+0000 7fb94cff9640 1 -- 172.21.15.120:0/4192971253 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9480046c0 con 0x7fb9501025f0 2024-09-19T09:06:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.682+0000 7fb955f6d640 1 -- 172.21.15.120:0/4192971253 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb95019fc40 con 0x7fb9501025f0 2024-09-19T09:06:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.682+0000 7fb94cff9640 1 -- 172.21.15.120:0/4192971253 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94802fea0 con 0x7fb9501025f0 2024-09-19T09:06:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.682+0000 7fb955f6d640 1 -- 172.21.15.120:0/4192971253 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9501a0190 con 0x7fb9501025f0 2024-09-19T09:06:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.685+0000 7fb94cff9640 1 -- 172.21.15.120:0/4192971253 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fb94802f610 con 0x7fb9501025f0 2024-09-19T09:06:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.685+0000 7fb958de9740 1 Processor -- start 2024-09-19T09:06:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.685+0000 7fb958de9740 1 -- start start 2024-09-19T09:06:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.685+0000 7fb958de9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db270e19a0 0x55db270e1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.685+0000 7fb958de9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db270e22e0 0x55db270e6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.685+0000 7fb958de9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db270e7370 0x55db270e9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.685+0000 7fb958de9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55db270452f0 con 0x55db270e22e0 2024-09-19T09:06:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.685+0000 7fb958de9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55db26fed270 con 0x55db270e7370 2024-09-19T09:06:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.685+0000 7fb958de9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55db270412b0 con 0x55db270e19a0 2024-09-19T09:06:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.685+0000 7fb94f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db270e22e0 0x55db270e6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.685+0000 7fb94f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db270e22e0 0x55db270e6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40948/0 (socket says 172.21.15.120:40948) 2024-09-19T09:06:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.685+0000 7fb94f7fe640 1 -- 172.21.15.120:0/2337239909 learned_addr learned my addr 172.21.15.120:0/2337239909 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.685+0000 7fb94effd640 1 --2- 172.21.15.120:0/2337239909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db270e19a0 0x55db270e1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.685+0000 7fb94cff9640 1 --2- 172.21.15.120:0/4192971253 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb9380a3370 0x7fb9380a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.685+0000 7fb94f7fe640 1 -- 172.21.15.120:0/2337239909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db270e19a0 msgr2=0x55db270e1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:20.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.685+0000 7fb94f7fe640 1 --2- 172.21.15.120:0/2337239909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db270e19a0 0x55db270e1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.685+0000 7fb94effd640 1 --2- 172.21.15.120:0/2337239909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db270e7370 0x55db270e9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:20.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.685+0000 7fb94f7fe640 1 -- 172.21.15.120:0/2337239909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db270e7370 msgr2=0x55db270e9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:20.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.685+0000 7fb94f7fe640 1 --2- 172.21.15.120:0/2337239909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db270e7370 0x55db270e9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.685+0000 7fb94f7fe640 1 -- 172.21.15.120:0/2337239909 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db26fec340 con 0x55db270e22e0 2024-09-19T09:06:20.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.685+0000 7fb94effd640 1 --2- 172.21.15.120:0/2337239909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db270e7370 0x55db270e9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:06:20.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.685+0000 7fb94ffff640 1 --2- 172.21.15.120:0/4192971253 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb9380a3370 0x7fb9380a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:20.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.686+0000 7fb94f7fe640 1 --2- 172.21.15.120:0/2337239909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db270e22e0 0x55db270e6da0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb94000cc00 tx=0x7fb94000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6768b7e27a97a8b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:20.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.686+0000 7fb94cff9640 1 -- 172.21.15.120:0/4192971253 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb9480f12f0 con 0x7fb9501025f0 2024-09-19T09:06:20.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.686+0000 7fb935ffb640 1 -- 172.21.15.120:0/2337239909 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb940005220 con 0x55db270e22e0 2024-09-19T09:06:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.686+0000 7fb935ffb640 1 -- 172.21.15.120:0/2337239909 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb940016070 con 0x55db270e22e0 2024-09-19T09:06:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.686+0000 7fb935ffb640 1 -- 172.21.15.120:0/2337239909 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9400106e0 con 0x55db270e22e0 2024-09-19T09:06:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.686+0000 7fb958de9740 1 -- 172.21.15.120:0/2337239909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db270e22e0 msgr2=0x55db270e6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.686+0000 7fb94ffff640 1 --2- 172.21.15.120:0/4192971253 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb9380a3370 0x7fb9380a5830 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb9480b9ce0 tx=0x7fb94802f940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.686+0000 7fb958de9740 1 --2- 172.21.15.120:0/2337239909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db270e22e0 0x55db270e6da0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb94000cc00 tx=0x7fb94000e720 comp rx=0 tx=0).stop 2024-09-19T09:06:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.686+0000 7fb958de9740 1 -- 172.21.15.120:0/2337239909 shutdown_connections 2024-09-19T09:06:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.686+0000 7fb958de9740 1 --2- 172.21.15.120:0/2337239909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db270e7370 0x55db270e9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.686+0000 7fb958de9740 1 --2- 172.21.15.120:0/2337239909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db270e22e0 0x55db270e6da0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.686+0000 7fb958de9740 1 --2- 172.21.15.120:0/2337239909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db270e19a0 0x55db270e1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.686+0000 7fb958de9740 1 -- 172.21.15.120:0/2337239909 >> 172.21.15.120:0/2337239909 conn(0x55db270dd010 msgr2=0x55db270dd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.687+0000 7fb958de9740 1 -- 172.21.15.120:0/2337239909 shutdown_connections 2024-09-19T09:06:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.687+0000 7fb958de9740 1 -- 172.21.15.120:0/2337239909 wait complete. 2024-09-19T09:06:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.687+0000 7fb958de9740 1 -- start start 2024-09-19T09:06:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.687+0000 7fb958de9740 1 Processor -- start 2024-09-19T09:06:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.687+0000 7fb958de9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db270e19a0 0x55db270ebb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.687+0000 7fb958de9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db270e22e0 0x55db270ec0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.687+0000 7fb958de9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db270e7370 0x55db270f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.687+0000 7fb958de9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55db270ea340 con 0x55db270e22e0 2024-09-19T09:06:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.687+0000 7fb958de9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55db26fce290 con 0x55db270e7370 2024-09-19T09:06:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.687+0000 7fb958de9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55db270ea4c0 con 0x55db270e19a0 2024-09-19T09:06:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.687+0000 7fb958de9740 10 client.0 _ll_register_callbacks cb 0x55db26fff140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.687+0000 7fb94f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db270e22e0 0x55db270ec0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.687+0000 7fb94f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db270e22e0 0x55db270ec0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40956/0 (socket says 172.21.15.120:40956) 2024-09-19T09:06:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.687+0000 7fb94f7fe640 1 -- 172.21.15.120:0/2771964077 learned_addr learned my addr 172.21.15.120:0/2771964077 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.687+0000 7fb94effd640 1 --2- 172.21.15.120:0/2771964077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db270e19a0 0x55db270ebb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.688+0000 7fb94f7fe640 1 -- 172.21.15.120:0/2771964077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db270e19a0 msgr2=0x55db270ebb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.688+0000 7fb94f7fe640 1 --2- 172.21.15.120:0/2771964077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db270e19a0 0x55db270ebb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.688+0000 7fb94f7fe640 1 -- 172.21.15.120:0/2771964077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db270e7370 msgr2=0x55db270f30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.688+0000 7fb94f7fe640 1 --2- 172.21.15.120:0/2771964077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db270e7370 0x55db270f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.688+0000 7fb94f7fe640 1 -- 172.21.15.120:0/2771964077 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db26d58210 con 0x55db270e22e0 2024-09-19T09:06:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.688+0000 7fb94effd640 1 --2- 172.21.15.120:0/2771964077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db270e19a0 0x55db270ebb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:06:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.688+0000 7fb94f7fe640 1 --2- 172.21.15.120:0/2771964077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db270e22e0 0x55db270ec0a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb940000c00 tx=0x7fb94000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fbef286234eb9a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.688+0000 7fb92effd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:06:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.688+0000 7fb92effd640 1 -- 172.21.15.120:0/2771964077 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb940029060 con 0x55db270e22e0 2024-09-19T09:06:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.688+0000 7fb92effd640 1 -- 172.21.15.120:0/2771964077 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb940004300 con 0x55db270e22e0 2024-09-19T09:06:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.688+0000 7fb958de9740 1 -- 172.21.15.120:0/2771964077 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb940016070 con 0x55db270e22e0 2024-09-19T09:06:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.688+0000 7fb92effd640 1 -- 172.21.15.120:0/2771964077 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb940004740 con 0x55db270e22e0 2024-09-19T09:06:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.688+0000 7fb958de9740 10 client.16731 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.688+0000 7fb958de9740 1 -- 172.21.15.120:0/2771964077 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55db26e24620 con 0x55db270e22e0 2024-09-19T09:06:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.688+0000 7fb92effd640 1 -- 172.21.15.120:0/2771964077 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb940016070 con 0x55db270e22e0 2024-09-19T09:06:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.688+0000 7fb92effd640 1 -- 172.21.15.120:0/2771964077 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb94002e070 con 0x55db270e22e0 2024-09-19T09:06:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.688+0000 7fb958de9740 10 client.16731 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.688+0000 7fb958de9740 10 client.16731 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:06:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.688+0000 7fb958de9740 20 client.16731 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.688+0000 7fb958de9740 1 --2- 172.21.15.120:0/2771964077 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55db270f7460 0x55db270f9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.689+0000 7fb958de9740 4 client.16731 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:06:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.689+0000 7fb958de9740 1 -- 172.21.15.120:0/2771964077 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db27052a20 con 0x55db270f7460 2024-09-19T09:06:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.689+0000 7fb94effd640 1 --2- 172.21.15.120:0/2771964077 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55db270f7460 0x55db270f9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.689+0000 7fb92effd640 10 client.16731 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.689+0000 7fb94effd640 1 --2- 172.21.15.120:0/2771964077 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55db270f7460 0x55db270f9880 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=731d04bc9fd1924f server_cookie=82a6e692ac3def68 in_seq=0 out_seq=0 2024-09-19T09:06:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.690+0000 7fb92effd640 1 -- 172.21.15.120:0/2771964077 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55db27052a20 con 0x55db270f7460 2024-09-19T09:06:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.690+0000 7fb92effd640 10 client.16731 handle_command_reply: tid=0 2024-09-19T09:06:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.690+0000 7fb958de9740 1 client.16731 shutdown 2024-09-19T09:06:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.690+0000 7fb958de9740 1 -- 172.21.15.120:0/2771964077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db270e22e0 msgr2=0x55db270ec0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.690+0000 7fb958de9740 1 --2- 172.21.15.120:0/2771964077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db270e22e0 0x55db270ec0a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb940000c00 tx=0x7fb94000c940 comp rx=0 tx=0).stop 2024-09-19T09:06:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.690+0000 7fb958de9740 1 -- 172.21.15.120:0/2771964077 shutdown_connections 2024-09-19T09:06:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.690+0000 7fb958de9740 1 --2- 172.21.15.120:0/2771964077 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55db270f7460 0x55db270f9880 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.690+0000 7fb958de9740 1 --2- 172.21.15.120:0/2771964077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db270e7370 0x55db270f30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.690+0000 7fb92effd640 0 client.16731 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.690+0000 7fb958de9740 1 --2- 172.21.15.120:0/2771964077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db270e22e0 0x55db270ec0a0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.690+0000 7fb958de9740 1 --2- 172.21.15.120:0/2771964077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db270e19a0 0x55db270ebb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.690+0000 7fb958de9740 1 -- 172.21.15.120:0/2771964077 >> 172.21.15.120:0/2771964077 conn(0x55db270dd010 msgr2=0x55db270de820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.690+0000 7fb958de9740 1 -- 172.21.15.120:0/2771964077 shutdown_connections 2024-09-19T09:06:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.690+0000 7fb958de9740 1 -- 172.21.15.120:0/2771964077 wait complete. 2024-09-19T09:06:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.690+0000 7fb958de9740 20 client.16731 trim_cache size 0 max 16384 2024-09-19T09:06:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.704+0000 7fb958de9740 1 Processor -- start 2024-09-19T09:06:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.704+0000 7fb958de9740 1 -- start start 2024-09-19T09:06:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.704+0000 7fb958de9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db270e19a0 0x55db270dc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.704+0000 7fb958de9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db270e22e0 0x55db270f30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.704+0000 7fb958de9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db270e7370 0x55db270de400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.704+0000 7fb958de9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501a9070 con 0x55db270e7370 2024-09-19T09:06:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.704+0000 7fb958de9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501aabb0 con 0x55db270e22e0 2024-09-19T09:06:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.704+0000 7fb958de9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501a9740 con 0x55db270e19a0 2024-09-19T09:06:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.705+0000 7fb94effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db270e7370 0x55db270de400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.705+0000 7fb94effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db270e7370 0x55db270de400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40966/0 (socket says 172.21.15.120:40966) 2024-09-19T09:06:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.705+0000 7fb94effd640 1 -- 172.21.15.120:0/7370705 learned_addr learned my addr 172.21.15.120:0/7370705 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.705+0000 7fb94effd640 1 --2- 172.21.15.120:0/7370705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db270e19a0 0x55db270dc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.705+0000 7fb94f7fe640 1 --2- 172.21.15.120:0/7370705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db270e22e0 0x55db270f30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.705+0000 7fb94effd640 1 -- 172.21.15.120:0/7370705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db270e19a0 msgr2=0x55db270dc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.705+0000 7fb94effd640 1 --2- 172.21.15.120:0/7370705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db270e19a0 0x55db270dc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.705+0000 7fb94effd640 1 -- 172.21.15.120:0/7370705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db270e22e0 msgr2=0x55db270f30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.705+0000 7fb94effd640 1 --2- 172.21.15.120:0/7370705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db270e22e0 0x55db270f30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.705+0000 7fb94effd640 1 -- 172.21.15.120:0/7370705 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db270ea620 con 0x55db270e7370 2024-09-19T09:06:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.705+0000 7fb94effd640 1 --2- 172.21.15.120:0/7370705 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db270e7370 0x55db270de400 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb944000c00 tx=0x7fb944006b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e64de5ba6f3247d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.705+0000 7fb9357fa640 1 -- 172.21.15.120:0/7370705 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb944027070 con 0x55db270e7370 2024-09-19T09:06:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.705+0000 7fb9357fa640 1 -- 172.21.15.120:0/7370705 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb944020960 con 0x55db270e7370 2024-09-19T09:06:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.705+0000 7fb9357fa640 1 -- 172.21.15.120:0/7370705 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94400f040 con 0x55db270e7370 2024-09-19T09:06:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.706+0000 7fb958de9740 1 -- 172.21.15.120:0/7370705 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db270e7370 msgr2=0x55db270de400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.706+0000 7fb958de9740 1 --2- 172.21.15.120:0/7370705 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db270e7370 0x55db270de400 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb944000c00 tx=0x7fb944006b20 comp rx=0 tx=0).stop 2024-09-19T09:06:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.706+0000 7fb958de9740 1 -- 172.21.15.120:0/7370705 shutdown_connections 2024-09-19T09:06:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.706+0000 7fb958de9740 1 --2- 172.21.15.120:0/7370705 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db270e7370 0x55db270de400 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.706+0000 7fb958de9740 1 --2- 172.21.15.120:0/7370705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db270e22e0 0x55db270f30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.706+0000 7fb958de9740 1 --2- 172.21.15.120:0/7370705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db270e19a0 0x55db270dc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.706+0000 7fb958de9740 1 -- 172.21.15.120:0/7370705 >> 172.21.15.120:0/7370705 conn(0x55db270dd010 msgr2=0x55db270db9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.706+0000 7fb958de9740 1 -- 172.21.15.120:0/7370705 shutdown_connections 2024-09-19T09:06:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.706+0000 7fb958de9740 1 -- 172.21.15.120:0/7370705 wait complete. 2024-09-19T09:06:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.706+0000 7fb958de9740 1 -- start start 2024-09-19T09:06:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.706+0000 7fb958de9740 1 Processor -- start 2024-09-19T09:06:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.707+0000 7fb958de9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db270e19a0 0x55db270f09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.707+0000 7fb958de9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db270e22e0 0x55db270f0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.707+0000 7fb958de9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db270e7370 0x55db270f1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.707+0000 7fb958de9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501aa4e0 con 0x55db270e19a0 2024-09-19T09:06:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.707+0000 7fb958de9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501a9e10 con 0x55db270e7370 2024-09-19T09:06:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.707+0000 7fb958de9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501a7030 con 0x55db270e22e0 2024-09-19T09:06:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.707+0000 7fb958de9740 10 client.0 _ll_register_callbacks cb 0x7fb940014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.707+0000 7fb94effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db270e19a0 0x55db270f09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.707+0000 7fb94effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db270e19a0 0x55db270f09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40980/0 (socket says 172.21.15.120:40980) 2024-09-19T09:06:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.707+0000 7fb94effd640 1 -- 172.21.15.120:0/4174765430 learned_addr learned my addr 172.21.15.120:0/4174765430 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.707+0000 7fb94effd640 1 --2- 172.21.15.120:0/4174765430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db270e7370 0x55db270f1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.707+0000 7fb94f7fe640 1 --2- 172.21.15.120:0/4174765430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db270e22e0 0x55db270f0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.707+0000 7fb94effd640 1 -- 172.21.15.120:0/4174765430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db270e22e0 msgr2=0x55db270f0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.707+0000 7fb94effd640 1 --2- 172.21.15.120:0/4174765430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db270e22e0 0x55db270f0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.707+0000 7fb94effd640 1 -- 172.21.15.120:0/4174765430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db270e7370 msgr2=0x55db270f1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.707+0000 7fb94effd640 1 --2- 172.21.15.120:0/4174765430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db270e7370 0x55db270f1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.707+0000 7fb94effd640 1 -- 172.21.15.120:0/4174765430 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db270f2510 con 0x55db270e19a0 2024-09-19T09:06:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.707+0000 7fb94f7fe640 1 --2- 172.21.15.120:0/4174765430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db270e22e0 0x55db270f0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:06:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.707+0000 7fb94effd640 1 --2- 172.21.15.120:0/4174765430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db270e19a0 0x55db270f09b0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb944004810 tx=0x7fb944004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33c63f736a057ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.707+0000 7fb92effd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:06:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.707+0000 7fb92effd640 1 -- 172.21.15.120:0/4174765430 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb944032060 con 0x55db270e19a0 2024-09-19T09:06:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.707+0000 7fb92effd640 1 -- 172.21.15.120:0/4174765430 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb944020680 con 0x55db270e19a0 2024-09-19T09:06:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.707+0000 7fb958de9740 1 -- 172.21.15.120:0/4174765430 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb944020960 con 0x55db270e19a0 2024-09-19T09:06:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.708+0000 7fb92effd640 1 -- 172.21.15.120:0/4174765430 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb944027040 con 0x55db270e19a0 2024-09-19T09:06:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.708+0000 7fb958de9740 10 client.16737 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.708+0000 7fb958de9740 1 -- 172.21.15.120:0/4174765430 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55db271622f0 con 0x55db270e19a0 2024-09-19T09:06:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.708+0000 7fb92effd640 1 -- 172.21.15.120:0/4174765430 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb944020960 con 0x55db270e19a0 2024-09-19T09:06:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.708+0000 7fb92effd640 1 -- 172.21.15.120:0/4174765430 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb944037070 con 0x55db270e19a0 2024-09-19T09:06:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.708+0000 7fb958de9740 10 client.16737 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.708+0000 7fb958de9740 10 client.16737 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:06:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.708+0000 7fb958de9740 20 client.16737 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.708+0000 7fb958de9740 1 --2- 172.21.15.120:0/4174765430 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55db27162bd0 0x55db27164ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.708+0000 7fb958de9740 4 client.16737 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:06:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.708+0000 7fb958de9740 1 -- 172.21.15.120:0/4174765430 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55db2704cd70 con 0x55db27162bd0 2024-09-19T09:06:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.708+0000 7fb94f7fe640 1 --2- 172.21.15.120:0/4174765430 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55db27162bd0 0x55db27164ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.709+0000 7fb92effd640 10 client.16737 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.709+0000 7fb94f7fe640 1 --2- 172.21.15.120:0/4174765430 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55db27162bd0 0x55db27164ff0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4452782ff383af8 server_cookie=1a58e5219af2fe82 in_seq=0 out_seq=0 2024-09-19T09:06:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.843+0000 7fb92effd640 1 -- 172.21.15.120:0/4174765430 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55db2704cd70 con 0x55db27162bd0 2024-09-19T09:06:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.843+0000 7fb92effd640 10 client.16737 handle_command_reply: tid=0 2024-09-19T09:06:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.843+0000 7fb958de9740 1 client.16737 shutdown 2024-09-19T09:06:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.843+0000 7fb958de9740 1 -- 172.21.15.120:0/4174765430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db270e19a0 msgr2=0x55db270f09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.843+0000 7fb958de9740 1 --2- 172.21.15.120:0/4174765430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db270e19a0 0x55db270f09b0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb944004810 tx=0x7fb944004680 comp rx=0 tx=0).stop 2024-09-19T09:06:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.843+0000 7fb958de9740 1 -- 172.21.15.120:0/4174765430 shutdown_connections 2024-09-19T09:06:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.843+0000 7fb958de9740 1 --2- 172.21.15.120:0/4174765430 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55db27162bd0 0x55db27164ff0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.843+0000 7fb958de9740 1 --2- 172.21.15.120:0/4174765430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db270e7370 0x55db270f1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.843+0000 7fb958de9740 1 --2- 172.21.15.120:0/4174765430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db270e22e0 0x55db270f0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.843+0000 7fb958de9740 1 --2- 172.21.15.120:0/4174765430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db270e19a0 0x55db270f09b0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.843+0000 7fb92effd640 0 client.16737 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.843+0000 7fb958de9740 1 -- 172.21.15.120:0/4174765430 >> 172.21.15.120:0/4174765430 conn(0x55db270dd010 msgr2=0x55db270dc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.844+0000 7fb958de9740 1 -- 172.21.15.120:0/4174765430 shutdown_connections 2024-09-19T09:06:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.844+0000 7fb958de9740 1 -- 172.21.15.120:0/4174765430 wait complete. 2024-09-19T09:06:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.844+0000 7fb958de9740 20 client.16737 trim_cache size 0 max 16384 2024-09-19T09:06:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:06:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:06:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:06:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:06:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.844+0000 7fb955f6d640 1 -- 172.21.15.120:0/4192971253 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb9380a3370 msgr2=0x7fb9380a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.844+0000 7fb955f6d640 1 --2- 172.21.15.120:0/4192971253 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb9380a3370 0x7fb9380a5830 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb9480b9ce0 tx=0x7fb94802f940 comp rx=0 tx=0).stop 2024-09-19T09:06:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.844+0000 7fb955f6d640 1 -- 172.21.15.120:0/4192971253 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9501025f0 msgr2=0x7fb95019f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.844+0000 7fb955f6d640 1 --2- 172.21.15.120:0/4192971253 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9501025f0 0x7fb95019f310 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb948003ba0 tx=0x7fb948003bd0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.844+0000 7fb955f6d640 1 -- 172.21.15.120:0/4192971253 shutdown_connections 2024-09-19T09:06:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.844+0000 7fb955f6d640 1 --2- 172.21.15.120:0/4192971253 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb9380a3370 0x7fb9380a5830 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.844+0000 7fb955f6d640 1 --2- 172.21.15.120:0/4192971253 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9501025f0 0x7fb95019f310 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.844+0000 7fb955f6d640 1 --2- 172.21.15.120:0/4192971253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb950101c30 0x7fb9501982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.844+0000 7fb955f6d640 1 --2- 172.21.15.120:0/4192971253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb950100a40 0x7fb950197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.844+0000 7fb955f6d640 1 -- 172.21.15.120:0/4192971253 >> 172.21.15.120:0/4192971253 conn(0x7fb9500fc1e0 msgr2=0x7fb95006a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.844+0000 7fb955f6d640 1 -- 172.21.15.120:0/4192971253 shutdown_connections 2024-09-19T09:06:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:20.844+0000 7fb955f6d640 1 -- 172.21.15.120:0/4192971253 wait complete. 2024-09-19T09:06:21.041 INFO:tasks.workunit.client.0.smithi120.stdout: 1 259365 10.77 MB/sec execute 16 sec latency 4534.502 ms 2024-09-19T09:06:21.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:20 smithi196 ceph-mon[35150]: pgmap v445: 97 pgs: 97 active+clean; 45 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 433 op/s 2024-09-19T09:06:21.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:20 smithi153 ceph-mon[35071]: pgmap v445: 97 pgs: 97 active+clean; 45 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 433 op/s 2024-09-19T09:06:21.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:20 smithi120 ceph-mon[31150]: pgmap v445: 97 pgs: 97 active+clean; 45 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 433 op/s 2024-09-19T09:06:22.042 INFO:tasks.workunit.client.0.smithi120.stdout: 1 259365 10.14 MB/sec execute 17 sec latency 5534.570 ms 2024-09-19T09:06:23.042 INFO:tasks.workunit.client.0.smithi120.stdout: 1 259365 9.58 MB/sec execute 18 sec latency 6534.649 ms 2024-09-19T09:06:23.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:22 smithi196 ceph-mon[35150]: pgmap v446: 97 pgs: 97 active+clean; 50 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 338 op/s 2024-09-19T09:06:23.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:22 smithi153 ceph-mon[35071]: pgmap v446: 97 pgs: 97 active+clean; 50 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 338 op/s 2024-09-19T09:06:23.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:22 smithi120 ceph-mon[31150]: pgmap v446: 97 pgs: 97 active+clean; 50 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 338 op/s 2024-09-19T09:06:24.042 INFO:tasks.workunit.client.0.smithi120.stdout: 1 259365 9.07 MB/sec execute 19 sec latency 7534.737 ms 2024-09-19T09:06:25.042 INFO:tasks.workunit.client.0.smithi120.stdout: 1 259365 8.62 MB/sec execute 20 sec latency 8534.863 ms 2024-09-19T09:06:25.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:24 smithi196 ceph-mon[35150]: pgmap v447: 97 pgs: 97 active+clean; 40 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 325 op/s 2024-09-19T09:06:25.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:24 smithi153 ceph-mon[35071]: pgmap v447: 97 pgs: 97 active+clean; 40 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 325 op/s 2024-09-19T09:06:25.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:24 smithi120 ceph-mon[31150]: pgmap v447: 97 pgs: 97 active+clean; 40 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 325 op/s 2024-09-19T09:06:26.042 INFO:tasks.workunit.client.0.smithi120.stdout: 1 259365 8.21 MB/sec execute 21 sec latency 9534.961 ms 2024-09-19T09:06:27.042 INFO:tasks.workunit.client.0.smithi120.stdout: 1 259365 7.84 MB/sec execute 22 sec latency 10535.088 ms 2024-09-19T09:06:27.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:26 smithi196 ceph-mon[35150]: pgmap v448: 97 pgs: 97 active+clean; 42 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 261 op/s 2024-09-19T09:06:27.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:26 smithi153 ceph-mon[35071]: pgmap v448: 97 pgs: 97 active+clean; 42 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 261 op/s 2024-09-19T09:06:27.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:26 smithi120 ceph-mon[31150]: pgmap v448: 97 pgs: 97 active+clean; 42 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 261 op/s 2024-09-19T09:06:28.042 INFO:tasks.workunit.client.0.smithi120.stdout: 1 259365 7.49 MB/sec execute 23 sec latency 11535.175 ms 2024-09-19T09:06:29.042 INFO:tasks.workunit.client.0.smithi120.stdout: 1 259365 7.18 MB/sec execute 24 sec latency 12535.285 ms 2024-09-19T09:06:29.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:28 smithi196 ceph-mon[35150]: pgmap v449: 97 pgs: 97 active+clean; 44 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 251 op/s 2024-09-19T09:06:29.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:28 smithi153 ceph-mon[35071]: pgmap v449: 97 pgs: 97 active+clean; 44 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 251 op/s 2024-09-19T09:06:29.209 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:28 smithi120 ceph-mon[31150]: pgmap v449: 97 pgs: 97 active+clean; 44 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 251 op/s 2024-09-19T09:06:30.042 INFO:tasks.workunit.client.0.smithi120.stdout: 1 259365 6.90 MB/sec execute 25 sec latency 13535.370 ms 2024-09-19T09:06:30.272 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:06:30.523 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:06:30.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:30 smithi120 ceph-mon[31150]: pgmap v450: 97 pgs: 97 active+clean; 48 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 208 op/s 2024-09-19T09:06:31.042 INFO:tasks.workunit.client.0.smithi120.stdout: 1 259365 6.63 MB/sec execute 26 sec latency 14535.409 ms 2024-09-19T09:06:31.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:30 smithi196 ceph-mon[35150]: pgmap v450: 97 pgs: 97 active+clean; 48 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 208 op/s 2024-09-19T09:06:31.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:30 smithi153 ceph-mon[35071]: pgmap v450: 97 pgs: 97 active+clean; 48 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 208 op/s 2024-09-19T09:06:31.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.031+0000 7f04c5200640 1 -- 172.21.15.120:0/487745171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04c0106a80 msgr2=0x7f04c0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:31.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.031+0000 7f04c5200640 1 --2- 172.21.15.120:0/487745171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04c0106a80 0x7f04c0106e60 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009930 tx=0x7f04b002a420 comp rx=0 tx=0).stop 2024-09-19T09:06:31.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.031+0000 7f04c5200640 1 -- 172.21.15.120:0/487745171 shutdown_connections 2024-09-19T09:06:31.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.031+0000 7f04c5200640 1 --2- 172.21.15.120:0/487745171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04c0101420 0x7f04c010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:31.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.031+0000 7f04c5200640 1 --2- 172.21.15.120:0/487745171 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04c0100a80 0x7f04c0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:31.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.031+0000 7f04c5200640 1 --2- 172.21.15.120:0/487745171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04c0106a80 0x7f04c0106e60 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:31.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.031+0000 7f04c5200640 1 -- 172.21.15.120:0/487745171 >> 172.21.15.120:0/487745171 conn(0x7f04c00fc7a0 msgr2=0x7f04c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:31.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.031+0000 7f04c5200640 1 -- 172.21.15.120:0/487745171 shutdown_connections 2024-09-19T09:06:31.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.031+0000 7f04c5200640 1 -- 172.21.15.120:0/487745171 wait complete. 2024-09-19T09:06:31.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.032+0000 7f04c5200640 1 Processor -- start 2024-09-19T09:06:31.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.032+0000 7f04c5200640 1 -- start start 2024-09-19T09:06:31.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.032+0000 7f04c5200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04c0100a80 0x7f04c0198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:31.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.032+0000 7f04c5200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04c0101420 0x7f04c01988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:31.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.033+0000 7f04c5200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04c0106a80 0x7f04c019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:31.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.033+0000 7f04c5200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f04c0110e40 con 0x7f04c0101420 2024-09-19T09:06:31.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.033+0000 7f04c5200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f04c0110cc0 con 0x7f04c0106a80 2024-09-19T09:06:31.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.033+0000 7f04c5200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f04c0110fc0 con 0x7f04c0100a80 2024-09-19T09:06:31.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.033+0000 7f04be59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04c0101420 0x7f04c01988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:31.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.033+0000 7f04be59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04c0101420 0x7f04c01988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33928/0 (socket says 172.21.15.120:33928) 2024-09-19T09:06:31.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.033+0000 7f04be59c640 1 -- 172.21.15.120:0/1226891628 learned_addr learned my addr 172.21.15.120:0/1226891628 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:31.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.033+0000 7f04bf59e640 1 --2- 172.21.15.120:0/1226891628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04c0106a80 0x7f04c019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:31.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.033+0000 7f04bed9d640 1 --2- 172.21.15.120:0/1226891628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04c0100a80 0x7f04c0198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:31.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.033+0000 7f04be59c640 1 -- 172.21.15.120:0/1226891628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04c0100a80 msgr2=0x7f04c0198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:31.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.033+0000 7f04be59c640 1 --2- 172.21.15.120:0/1226891628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04c0100a80 0x7f04c0198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:31.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.033+0000 7f04be59c640 1 -- 172.21.15.120:0/1226891628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04c0106a80 msgr2=0x7f04c019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:31.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.033+0000 7f04be59c640 1 --2- 172.21.15.120:0/1226891628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04c0106a80 0x7f04c019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:31.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.033+0000 7f04be59c640 1 -- 172.21.15.120:0/1226891628 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c019d330 con 0x7f04c0101420 2024-09-19T09:06:31.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.033+0000 7f04bed9d640 1 --2- 172.21.15.120:0/1226891628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04c0100a80 0x7f04c0198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:06:31.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.033+0000 7f04be59c640 1 --2- 172.21.15.120:0/1226891628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04c0101420 0x7f04c01988c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f04b400c920 tx=0x7f04b400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:31.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.033+0000 7f04a7fff640 1 -- 172.21.15.120:0/1226891628 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b4007c70 con 0x7f04c0101420 2024-09-19T09:06:31.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.033+0000 7f04c5200640 1 -- 172.21.15.120:0/1226891628 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c019d620 con 0x7f04c0101420 2024-09-19T09:06:31.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.033+0000 7f04a7fff640 1 -- 172.21.15.120:0/1226891628 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04b4007e10 con 0x7f04c0101420 2024-09-19T09:06:31.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.033+0000 7f04c5200640 1 -- 172.21.15.120:0/1226891628 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c01a4f00 con 0x7f04c0101420 2024-09-19T09:06:31.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.033+0000 7f04a7fff640 1 -- 172.21.15.120:0/1226891628 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b4018890 con 0x7f04c0101420 2024-09-19T09:06:31.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.034+0000 7f04c5200640 1 -- 172.21.15.120:0/1226891628 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04c0100ee0 con 0x7f04c0101420 2024-09-19T09:06:31.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.036+0000 7f04a7fff640 1 -- 172.21.15.120:0/1226891628 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f04b4018b10 con 0x7f04c0101420 2024-09-19T09:06:31.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.036+0000 7f04a7fff640 1 --2- 172.21.15.120:0/1226891628 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f04a80a3240 0x7f04a80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:31.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.036+0000 7f04a7fff640 1 -- 172.21.15.120:0/1226891628 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f04b40cfcf0 con 0x7f04c0101420 2024-09-19T09:06:31.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.036+0000 7f04bed9d640 1 --2- 172.21.15.120:0/1226891628 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f04a80a3240 0x7f04a80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:31.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.037+0000 7f04bed9d640 1 --2- 172.21.15.120:0/1226891628 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f04a80a3240 0x7f04a80a5700 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f04b0000c00 tx=0x7f04b0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:31.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.040+0000 7f04a7fff640 1 -- 172.21.15.120:0/1226891628 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f04b4014030 con 0x7f04c0101420 2024-09-19T09:06:31.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.244+0000 7f04c5200640 1 -- 172.21.15.120:0/1226891628 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04c002d7c0 con 0x7f04c0101420 2024-09-19T09:06:31.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.244+0000 7f04a7fff640 1 -- 172.21.15.120:0/1226891628 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f04b409caa0 con 0x7f04c0101420 2024-09-19T09:06:31.245 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:06:31.245 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":28,"btime":"2024-09-19T09:02:41:696728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:02:41.207357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:06:31.245 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 28 2024-09-19T09:06:31.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.246+0000 7f04c5200640 1 -- 172.21.15.120:0/1226891628 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f04a80a3240 msgr2=0x7f04a80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:31.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.246+0000 7f04c5200640 1 --2- 172.21.15.120:0/1226891628 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f04a80a3240 0x7f04a80a5700 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f04b0000c00 tx=0x7f04b0005a20 comp rx=0 tx=0).stop 2024-09-19T09:06:31.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.246+0000 7f04c5200640 1 -- 172.21.15.120:0/1226891628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04c0101420 msgr2=0x7f04c01988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:31.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.246+0000 7f04c5200640 1 --2- 172.21.15.120:0/1226891628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04c0101420 0x7f04c01988c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f04b400c920 tx=0x7f04b400cde0 comp rx=0 tx=0).stop 2024-09-19T09:06:31.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.246+0000 7f04c5200640 1 -- 172.21.15.120:0/1226891628 shutdown_connections 2024-09-19T09:06:31.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.246+0000 7f04c5200640 1 --2- 172.21.15.120:0/1226891628 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f04a80a3240 0x7f04a80a5700 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:31.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.246+0000 7f04c5200640 1 --2- 172.21.15.120:0/1226891628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04c0106a80 0x7f04c019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:31.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.246+0000 7f04c5200640 1 --2- 172.21.15.120:0/1226891628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04c0101420 0x7f04c01988c0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:31.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.246+0000 7f04c5200640 1 --2- 172.21.15.120:0/1226891628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04c0100a80 0x7f04c0198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:31.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.246+0000 7f04c5200640 1 -- 172.21.15.120:0/1226891628 >> 172.21.15.120:0/1226891628 conn(0x7f04c00fc7a0 msgr2=0x7f04c006a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:31.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.247+0000 7f04c5200640 1 -- 172.21.15.120:0/1226891628 shutdown_connections 2024-09-19T09:06:31.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:31.247+0000 7f04c5200640 1 -- 172.21.15.120:0/1226891628 wait complete. 2024-09-19T09:06:32.043 INFO:tasks.workunit.client.0.smithi120.stdout: 1 259365 6.38 MB/sec execute 27 sec latency 15535.501 ms 2024-09-19T09:06:32.179 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:31 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1226891628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:06:32.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:31 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1226891628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:06:32.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:31 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1226891628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:06:32.210 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db --set-id b11bef3b --release --await' 2024-09-19T09:06:32.211 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db --set-id b11bef3b --release --await 2024-09-19T09:06:32.459 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:06:33.043 INFO:tasks.workunit.client.0.smithi120.stdout: 1 259365 6.16 MB/sec execute 28 sec latency 16535.564 ms 2024-09-19T09:06:33.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.084+0000 7effed03c640 1 -- 172.21.15.120:0/558804381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe81036e0 msgr2=0x7effe8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:33.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.084+0000 7effed03c640 1 --2- 172.21.15.120:0/558804381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe81036e0 0x7effe8107b90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7effdc009930 tx=0x7effdc02a420 comp rx=0 tx=0).stop 2024-09-19T09:06:33.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.084+0000 7effed03c640 1 -- 172.21.15.120:0/558804381 shutdown_connections 2024-09-19T09:06:33.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.084+0000 7effed03c640 1 --2- 172.21.15.120:0/558804381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effe81082a0 0x7effe810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.084+0000 7effed03c640 1 --2- 172.21.15.120:0/558804381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe81036e0 0x7effe8107b90 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.084+0000 7effed03c640 1 --2- 172.21.15.120:0/558804381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effe8102d30 0x7effe8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.084+0000 7effed03c640 1 -- 172.21.15.120:0/558804381 >> 172.21.15.120:0/558804381 conn(0x7effe80fc7d0 msgr2=0x7effe80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:33.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.084+0000 7effed03c640 1 -- 172.21.15.120:0/558804381 shutdown_connections 2024-09-19T09:06:33.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.084+0000 7effed03c640 1 -- 172.21.15.120:0/558804381 wait complete. 2024-09-19T09:06:33.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.085+0000 7effed03c640 1 Processor -- start 2024-09-19T09:06:33.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.085+0000 7effed03c640 1 -- start start 2024-09-19T09:06:33.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.085+0000 7effed03c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe8102d30 0x7effe81985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.085+0000 7effed03c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effe81036e0 0x7effe8198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.085+0000 7effed03c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effe81082a0 0x7effe819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.085+0000 7effed03c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7effe81120a0 con 0x7effe8102d30 2024-09-19T09:06:33.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.085+0000 7effed03c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7effe8111f20 con 0x7effe81082a0 2024-09-19T09:06:33.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.085+0000 7effed03c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7effe8112220 con 0x7effe81036e0 2024-09-19T09:06:33.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.086+0000 7effe6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe8102d30 0x7effe81985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.086+0000 7effe6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe8102d30 0x7effe81985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40238/0 (socket says 172.21.15.120:40238) 2024-09-19T09:06:33.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.086+0000 7effe6d9d640 1 -- 172.21.15.120:0/1227232131 learned_addr learned my addr 172.21.15.120:0/1227232131 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:33.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.086+0000 7effe659c640 1 --2- 172.21.15.120:0/1227232131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effe81036e0 0x7effe8198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.086+0000 7effe759e640 1 --2- 172.21.15.120:0/1227232131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effe81082a0 0x7effe819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.086+0000 7effe6d9d640 1 -- 172.21.15.120:0/1227232131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effe81036e0 msgr2=0x7effe8198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:33.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.086+0000 7effe6d9d640 1 --2- 172.21.15.120:0/1227232131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effe81036e0 0x7effe8198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.086+0000 7effe6d9d640 1 -- 172.21.15.120:0/1227232131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effe81082a0 msgr2=0x7effe819ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:33.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.086+0000 7effe6d9d640 1 --2- 172.21.15.120:0/1227232131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effe81082a0 0x7effe819ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.086+0000 7effe6d9d640 1 -- 172.21.15.120:0/1227232131 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe819d590 con 0x7effe8102d30 2024-09-19T09:06:33.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.086+0000 7effe659c640 1 --2- 172.21.15.120:0/1227232131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effe81036e0 0x7effe8198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:06:33.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.086+0000 7effe6d9d640 1 --2- 172.21.15.120:0/1227232131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe8102d30 0x7effe81985e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7effd800c970 tx=0x7effd800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:33.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.086+0000 7effcffff640 1 -- 172.21.15.120:0/1227232131 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd8007bb0 con 0x7effe8102d30 2024-09-19T09:06:33.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.086+0000 7effcffff640 1 -- 172.21.15.120:0/1227232131 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effd8007d50 con 0x7effe8102d30 2024-09-19T09:06:33.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.086+0000 7effed03c640 1 -- 172.21.15.120:0/1227232131 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effe819d880 con 0x7effe8102d30 2024-09-19T09:06:33.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.086+0000 7effcffff640 1 -- 172.21.15.120:0/1227232131 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd800f450 con 0x7effe8102d30 2024-09-19T09:06:33.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.086+0000 7effed03c640 1 -- 172.21.15.120:0/1227232131 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effe81a5160 con 0x7effe8102d30 2024-09-19T09:06:33.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.088+0000 7effcffff640 1 -- 172.21.15.120:0/1227232131 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7effd8005020 con 0x7effe8102d30 2024-09-19T09:06:33.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.089+0000 7effef415740 1 Processor -- start 2024-09-19T09:06:33.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.089+0000 7effef415740 1 -- start start 2024-09-19T09:06:33.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.089+0000 7effef415740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650c272e060 0x5650c272e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.089+0000 7effef415740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650c272e980 0x5650c2738d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.089+0000 7effef415740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650c2739240 0x5650c273b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.089+0000 7effef415740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5650c2693620 con 0x5650c272e060 2024-09-19T09:06:33.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.089+0000 7effef415740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5650c263ae50 con 0x5650c272e980 2024-09-19T09:06:33.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.089+0000 7effef415740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5650c268f5e0 con 0x5650c2739240 2024-09-19T09:06:33.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.089+0000 7effcffff640 1 --2- 172.21.15.120:0/1227232131 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7effd00a3240 0x7effd00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.089+0000 7effe759e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650c272e060 0x5650c272e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.089+0000 7effe759e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650c272e060 0x5650c272e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40240/0 (socket says 172.21.15.120:40240) 2024-09-19T09:06:33.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.089+0000 7effe759e640 1 -- 172.21.15.120:0/2502087362 learned_addr learned my addr 172.21.15.120:0/2502087362 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:33.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.089+0000 7effe759e640 1 --2- 172.21.15.120:0/1227232131 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7effd00a3240 0x7effd00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.089+0000 7effe6d9d640 1 --2- 172.21.15.120:0/2502087362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650c272e980 0x5650c2738d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.089+0000 7effcffff640 1 -- 172.21.15.120:0/1227232131 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7effd80cf7a0 con 0x7effe8102d30 2024-09-19T09:06:33.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.089+0000 7effe659c640 1 --2- 172.21.15.120:0/2502087362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650c2739240 0x5650c273b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.089+0000 7effe759e640 1 -- 172.21.15.120:0/2502087362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650c2739240 msgr2=0x5650c273b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:33.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.090+0000 7effe759e640 1 --2- 172.21.15.120:0/2502087362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650c2739240 0x5650c273b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.090+0000 7effe759e640 1 -- 172.21.15.120:0/2502087362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650c272e980 msgr2=0x5650c2738d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:33.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.090+0000 7effe759e640 1 --2- 172.21.15.120:0/2502087362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650c272e980 0x5650c2738d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.090+0000 7effe759e640 1 -- 172.21.15.120:0/2502087362 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650c262bd40 con 0x5650c272e060 2024-09-19T09:06:33.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.090+0000 7effe659c640 1 --2- 172.21.15.120:0/2502087362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650c2739240 0x5650c273b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:06:33.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.090+0000 7effe6d9d640 1 --2- 172.21.15.120:0/2502087362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650c272e980 0x5650c2738d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:06:33.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.090+0000 7effe759e640 1 --2- 172.21.15.120:0/2502087362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650c272e060 0x5650c272e440 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7effe0012c40 tx=0x7effe0010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e94d802e1bef68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:33.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.090+0000 7effcd7fa640 1 -- 172.21.15.120:0/2502087362 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe0009c10 con 0x5650c272e060 2024-09-19T09:06:33.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.090+0000 7effe759e640 1 --2- 172.21.15.120:0/1227232131 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7effd00a3240 0x7effd00a5700 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7effe000f000 tx=0x7effe001e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:33.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.090+0000 7effcd7fa640 1 -- 172.21.15.120:0/2502087362 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effe0012df0 con 0x5650c272e060 2024-09-19T09:06:33.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.090+0000 7effcd7fa640 1 -- 172.21.15.120:0/2502087362 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe001ebf0 con 0x5650c272e060 2024-09-19T09:06:33.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.091+0000 7effef415740 1 -- 172.21.15.120:0/2502087362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650c272e060 msgr2=0x5650c272e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:33.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.091+0000 7effef415740 1 --2- 172.21.15.120:0/2502087362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650c272e060 0x5650c272e440 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7effe0012c40 tx=0x7effe0010740 comp rx=0 tx=0).stop 2024-09-19T09:06:33.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.091+0000 7effef415740 1 -- 172.21.15.120:0/2502087362 shutdown_connections 2024-09-19T09:06:33.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.091+0000 7effef415740 1 --2- 172.21.15.120:0/2502087362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650c2739240 0x5650c273b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.091+0000 7effef415740 1 --2- 172.21.15.120:0/2502087362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650c272e980 0x5650c2738d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.091+0000 7effef415740 1 --2- 172.21.15.120:0/2502087362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650c272e060 0x5650c272e440 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.091+0000 7effef415740 1 -- 172.21.15.120:0/2502087362 >> 172.21.15.120:0/2502087362 conn(0x5650c272a5f0 msgr2=0x5650c272a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:33.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.091+0000 7effef415740 1 -- 172.21.15.120:0/2502087362 shutdown_connections 2024-09-19T09:06:33.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.091+0000 7effef415740 1 -- 172.21.15.120:0/2502087362 wait complete. 2024-09-19T09:06:33.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.091+0000 7effef415740 1 -- start start 2024-09-19T09:06:33.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.091+0000 7effef415740 1 Processor -- start 2024-09-19T09:06:33.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.091+0000 7effef415740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650c272e060 0x5650c27361e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.091+0000 7effef415740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650c272e980 0x5650c2736720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.091+0000 7effef415740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650c2739240 0x5650c2736c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.091+0000 7effef415740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5650c273bdb0 con 0x5650c2739240 2024-09-19T09:06:33.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.091+0000 7effef415740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5650c261bf10 con 0x5650c272e980 2024-09-19T09:06:33.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.091+0000 7effef415740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5650c273bf30 con 0x5650c272e060 2024-09-19T09:06:33.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.092+0000 7effef415740 10 client.0 _ll_register_callbacks cb 0x5650c263fdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:33.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.092+0000 7effe759e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650c2739240 0x5650c2736c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.092+0000 7effe759e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650c2739240 0x5650c2736c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40244/0 (socket says 172.21.15.120:40244) 2024-09-19T09:06:33.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.092+0000 7effe759e640 1 -- 172.21.15.120:0/2174267491 learned_addr learned my addr 172.21.15.120:0/2174267491 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:33.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.092+0000 7effe6d9d640 1 --2- 172.21.15.120:0/2174267491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650c272e060 0x5650c27361e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.092+0000 7effe659c640 1 --2- 172.21.15.120:0/2174267491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650c272e980 0x5650c2736720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.092+0000 7effe759e640 1 -- 172.21.15.120:0/2174267491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650c272e060 msgr2=0x5650c27361e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:33.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.092+0000 7effe759e640 1 --2- 172.21.15.120:0/2174267491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650c272e060 0x5650c27361e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.092+0000 7effe759e640 1 -- 172.21.15.120:0/2174267491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650c272e980 msgr2=0x5650c2736720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:33.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.092+0000 7effe759e640 1 --2- 172.21.15.120:0/2174267491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650c272e980 0x5650c2736720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.092+0000 7effe759e640 1 -- 172.21.15.120:0/2174267491 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650c23a2580 con 0x5650c2739240 2024-09-19T09:06:33.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.092+0000 7effe659c640 1 --2- 172.21.15.120:0/2174267491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650c272e980 0x5650c2736720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:06:33.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.092+0000 7effe759e640 1 --2- 172.21.15.120:0/2174267491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650c2739240 0x5650c2736c60 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7effe0010c40 tx=0x7effe0009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ff1d9d85e7ec62 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:33.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.092+0000 7effcaff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:06:33.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.092+0000 7effcaff5640 1 -- 172.21.15.120:0/2174267491 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe002c030 con 0x5650c2739240 2024-09-19T09:06:33.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.092+0000 7effcaff5640 1 -- 172.21.15.120:0/2174267491 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7effe001eb80 con 0x5650c2739240 2024-09-19T09:06:33.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.092+0000 7effef415740 1 -- 172.21.15.120:0/2174267491 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effe0012df0 con 0x5650c2739240 2024-09-19T09:06:33.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.092+0000 7effcaff5640 1 -- 172.21.15.120:0/2174267491 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7effe001b040 con 0x5650c2739240 2024-09-19T09:06:33.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.092+0000 7effef415740 10 client.16755 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:33.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.092+0000 7effef415740 1 -- 172.21.15.120:0/2174267491 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5650c246c960 con 0x5650c2739240 2024-09-19T09:06:33.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.093+0000 7effcaff5640 1 -- 172.21.15.120:0/2174267491 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effe0012df0 con 0x5650c2739240 2024-09-19T09:06:33.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.093+0000 7effcaff5640 1 -- 172.21.15.120:0/2174267491 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7effe00050d0 con 0x5650c2739240 2024-09-19T09:06:33.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.093+0000 7effef415740 10 client.16755 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:33.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.093+0000 7effef415740 10 client.16755 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:06:33.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.093+0000 7effef415740 20 client.16755 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:33.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.093+0000 7effef415740 1 --2- 172.21.15.120:0/2174267491 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5650c2748cb0 0x5650c274b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.093+0000 7effef415740 4 client.16755 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:06:33.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.093+0000 7effef415740 1 -- 172.21.15.120:0/2174267491 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650c26a0b90 con 0x5650c2748cb0 2024-09-19T09:06:33.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.093+0000 7effe6d9d640 1 --2- 172.21.15.120:0/2174267491 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5650c2748cb0 0x5650c274b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.093+0000 7effcaff5640 10 client.16755 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:33.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.093+0000 7effe6d9d640 1 --2- 172.21.15.120:0/2174267491 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5650c2748cb0 0x5650c274b0a0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=781f6e0bce67b6ee server_cookie=dedf62bd3c1cf861 in_seq=0 out_seq=0 2024-09-19T09:06:33.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.094+0000 7effcaff5640 1 -- 172.21.15.120:0/2174267491 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5650c26a0b90 con 0x5650c2748cb0 2024-09-19T09:06:33.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.094+0000 7effcaff5640 10 client.16755 handle_command_reply: tid=0 2024-09-19T09:06:33.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.094+0000 7effef415740 1 client.16755 shutdown 2024-09-19T09:06:33.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.094+0000 7effef415740 1 -- 172.21.15.120:0/2174267491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650c2739240 msgr2=0x5650c2736c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:33.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.094+0000 7effef415740 1 --2- 172.21.15.120:0/2174267491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650c2739240 0x5650c2736c60 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7effe0010c40 tx=0x7effe0009c70 comp rx=0 tx=0).stop 2024-09-19T09:06:33.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.094+0000 7effef415740 1 -- 172.21.15.120:0/2174267491 shutdown_connections 2024-09-19T09:06:33.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.094+0000 7effef415740 1 --2- 172.21.15.120:0/2174267491 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5650c2748cb0 0x5650c274b0a0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.094+0000 7effef415740 1 --2- 172.21.15.120:0/2174267491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650c2739240 0x5650c2736c60 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.094+0000 7effef415740 1 --2- 172.21.15.120:0/2174267491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650c272e980 0x5650c2736720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.094+0000 7effcaff5640 0 client.16755 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:33.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.094+0000 7effef415740 1 --2- 172.21.15.120:0/2174267491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650c272e060 0x5650c27361e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.094+0000 7effef415740 1 -- 172.21.15.120:0/2174267491 >> 172.21.15.120:0/2174267491 conn(0x5650c272a5f0 msgr2=0x5650c2729750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:33.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.094+0000 7effef415740 1 -- 172.21.15.120:0/2174267491 shutdown_connections 2024-09-19T09:06:33.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.094+0000 7effef415740 1 -- 172.21.15.120:0/2174267491 wait complete. 2024-09-19T09:06:33.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.094+0000 7effef415740 20 client.16755 trim_cache size 0 max 16384 2024-09-19T09:06:33.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.111+0000 7effef415740 1 Processor -- start 2024-09-19T09:06:33.112 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:32 smithi120 ceph-mon[31150]: pgmap v451: 97 pgs: 97 active+clean; 46 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 2.6 MiB/s wr, 199 op/s 2024-09-19T09:06:33.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.111+0000 7effef415740 1 -- start start 2024-09-19T09:06:33.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.111+0000 7effef415740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650c272e060 0x5650c2730b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.111+0000 7effef415740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650c272e980 0x5650c2738040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.111+0000 7effef415740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650c2739240 0x5650c2750860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.111+0000 7effef415740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7effe81ad6e0 con 0x5650c272e980 2024-09-19T09:06:33.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.111+0000 7effef415740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7effe81af220 con 0x5650c272e060 2024-09-19T09:06:33.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.111+0000 7effef415740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7effe81addb0 con 0x5650c2739240 2024-09-19T09:06:33.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.111+0000 7effe659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650c272e980 0x5650c2738040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.111+0000 7effe659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650c272e980 0x5650c2738040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40248/0 (socket says 172.21.15.120:40248) 2024-09-19T09:06:33.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.111+0000 7effe659c640 1 -- 172.21.15.120:0/3214282651 learned_addr learned my addr 172.21.15.120:0/3214282651 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:33.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.111+0000 7effe759e640 1 --2- 172.21.15.120:0/3214282651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650c2739240 0x5650c2750860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.111+0000 7effe6d9d640 1 --2- 172.21.15.120:0/3214282651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650c272e060 0x5650c2730b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.112+0000 7effe659c640 1 -- 172.21.15.120:0/3214282651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650c2739240 msgr2=0x5650c2750860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:33.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.112+0000 7effe659c640 1 --2- 172.21.15.120:0/3214282651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650c2739240 0x5650c2750860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.112+0000 7effe659c640 1 -- 172.21.15.120:0/3214282651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650c272e060 msgr2=0x5650c2730b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:33.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.112+0000 7effe659c640 1 --2- 172.21.15.120:0/3214282651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650c272e060 0x5650c2730b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.112+0000 7effe659c640 1 -- 172.21.15.120:0/3214282651 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650c274b630 con 0x5650c272e980 2024-09-19T09:06:33.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.112+0000 7effe6d9d640 1 --2- 172.21.15.120:0/3214282651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650c272e060 0x5650c2730b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:06:33.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.112+0000 7effe659c640 1 --2- 172.21.15.120:0/3214282651 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650c272e980 0x5650c2738040 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7effe80697d0 tx=0x7effdc032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41861be4110b05b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:33.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.112+0000 7effccff9640 1 -- 172.21.15.120:0/3214282651 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effdc03f070 con 0x5650c272e980 2024-09-19T09:06:33.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.112+0000 7effccff9640 1 -- 172.21.15.120:0/3214282651 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effdc0317b0 con 0x5650c272e980 2024-09-19T09:06:33.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.112+0000 7effccff9640 1 -- 172.21.15.120:0/3214282651 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effdc035070 con 0x5650c272e980 2024-09-19T09:06:33.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.113+0000 7effef415740 1 -- 172.21.15.120:0/3214282651 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650c272e980 msgr2=0x5650c2738040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:33.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.113+0000 7effef415740 1 --2- 172.21.15.120:0/3214282651 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650c272e980 0x5650c2738040 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7effe80697d0 tx=0x7effdc032ce0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.113+0000 7effef415740 1 -- 172.21.15.120:0/3214282651 shutdown_connections 2024-09-19T09:06:33.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.113+0000 7effef415740 1 --2- 172.21.15.120:0/3214282651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650c2739240 0x5650c2750860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.113+0000 7effef415740 1 --2- 172.21.15.120:0/3214282651 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650c272e980 0x5650c2738040 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.113+0000 7effef415740 1 --2- 172.21.15.120:0/3214282651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650c272e060 0x5650c2730b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.113+0000 7effef415740 1 -- 172.21.15.120:0/3214282651 >> 172.21.15.120:0/3214282651 conn(0x5650c272a5f0 msgr2=0x5650c2737930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:33.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.113+0000 7effef415740 1 -- 172.21.15.120:0/3214282651 shutdown_connections 2024-09-19T09:06:33.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.113+0000 7effef415740 1 -- 172.21.15.120:0/3214282651 wait complete. 2024-09-19T09:06:33.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.113+0000 7effef415740 1 -- start start 2024-09-19T09:06:33.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.113+0000 7effef415740 1 Processor -- start 2024-09-19T09:06:33.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.113+0000 7effef415740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650c272e060 0x5650c2741360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.113+0000 7effef415740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650c272e980 0x5650c27418a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.113+0000 7effef415740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650c2739240 0x5650c273e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.113+0000 7effef415740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7effe81aeb50 con 0x5650c2739240 2024-09-19T09:06:33.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.113+0000 7effef415740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7effe81ae480 con 0x5650c272e980 2024-09-19T09:06:33.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.113+0000 7effef415740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7effe81ab6a0 con 0x5650c272e060 2024-09-19T09:06:33.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.113+0000 7effef415740 10 client.0 _ll_register_callbacks cb 0x7effe00095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:33.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.114+0000 7effe759e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650c2739240 0x5650c273e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.114+0000 7effe6d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650c272e060 0x5650c2741360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.114+0000 7effe6d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650c272e060 0x5650c2741360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:47160/0 (socket says 172.21.15.120:47160) 2024-09-19T09:06:33.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.114+0000 7effe6d9d640 1 -- 172.21.15.120:0/3486546862 learned_addr learned my addr 172.21.15.120:0/3486546862 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:33.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.114+0000 7effe659c640 1 --2- 172.21.15.120:0/3486546862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650c272e980 0x5650c27418a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.114+0000 7effe759e640 1 -- 172.21.15.120:0/3486546862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650c272e060 msgr2=0x5650c2741360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:33.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.114+0000 7effe759e640 1 --2- 172.21.15.120:0/3486546862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650c272e060 0x5650c2741360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.114+0000 7effe759e640 1 -- 172.21.15.120:0/3486546862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650c272e980 msgr2=0x5650c27418a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:33.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.114+0000 7effe759e640 1 --2- 172.21.15.120:0/3486546862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650c272e980 0x5650c27418a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.114+0000 7effe759e640 1 -- 172.21.15.120:0/3486546862 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650c2731050 con 0x5650c2739240 2024-09-19T09:06:33.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.114+0000 7effe759e640 1 --2- 172.21.15.120:0/3486546862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650c2739240 0x5650c273e150 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7effe00125d0 tx=0x7effe0009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eccaa1932344b284 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:33.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.114+0000 7effcaff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:06:33.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.114+0000 7effcaff5640 1 -- 172.21.15.120:0/3486546862 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe002c040 con 0x5650c2739240 2024-09-19T09:06:33.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.114+0000 7effcaff5640 1 -- 172.21.15.120:0/3486546862 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7effe0005710 con 0x5650c2739240 2024-09-19T09:06:33.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.114+0000 7effef415740 1 -- 172.21.15.120:0/3486546862 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effdc0317b0 con 0x5650c2739240 2024-09-19T09:06:33.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.114+0000 7effcaff5640 1 -- 172.21.15.120:0/3486546862 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7effe0005bd0 con 0x5650c2739240 2024-09-19T09:06:33.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.114+0000 7effef415740 10 client.16761 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:33.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.114+0000 7effef415740 1 -- 172.21.15.120:0/3486546862 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5650c27b40d0 con 0x5650c2739240 2024-09-19T09:06:33.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.115+0000 7effcaff5640 1 -- 172.21.15.120:0/3486546862 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effdc0317b0 con 0x5650c2739240 2024-09-19T09:06:33.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.115+0000 7effcaff5640 1 -- 172.21.15.120:0/3486546862 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7effe0004a70 con 0x5650c2739240 2024-09-19T09:06:33.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.115+0000 7effef415740 10 client.16761 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:33.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.115+0000 7effef415740 10 client.16761 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:06:33.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.115+0000 7effef415740 20 client.16761 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:33.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.115+0000 7effef415740 1 --2- 172.21.15.120:0/3486546862 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5650c27b4720 0x5650c27b6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.115+0000 7effef415740 4 client.16761 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b11bef3b", "release": true, "await": true}] 2024-09-19T09:06:33.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.115+0000 7effef415740 1 -- 172.21.15.120:0/3486546862 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b11bef3b", "release": true, "await": true}) -- 0x5650c269b0a0 con 0x5650c27b4720 2024-09-19T09:06:33.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.115+0000 7effe6d9d640 1 --2- 172.21.15.120:0/3486546862 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5650c27b4720 0x5650c27b6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.115+0000 7effcaff5640 10 client.16761 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:33.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.115+0000 7effe6d9d640 1 --2- 172.21.15.120:0/3486546862 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5650c27b4720 0x5650c27b6b10 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0eaa32c9555a29d server_cookie=33263f455b6f925b in_seq=0 out_seq=0 2024-09-19T09:06:33.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.127+0000 7effcaff5640 1 -- 172.21.15.120:0/3486546862 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5650c269b0a0 con 0x5650c27b4720 2024-09-19T09:06:33.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.127+0000 7effcaff5640 10 client.16761 handle_command_reply: tid=0 2024-09-19T09:06:33.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.127+0000 7effef415740 1 client.16761 shutdown 2024-09-19T09:06:33.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.127+0000 7effef415740 1 -- 172.21.15.120:0/3486546862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650c2739240 msgr2=0x5650c273e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:33.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.127+0000 7effef415740 1 --2- 172.21.15.120:0/3486546862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650c2739240 0x5650c273e150 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7effe00125d0 tx=0x7effe0009440 comp rx=0 tx=0).stop 2024-09-19T09:06:33.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.127+0000 7effef415740 1 -- 172.21.15.120:0/3486546862 shutdown_connections 2024-09-19T09:06:33.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.127+0000 7effef415740 1 --2- 172.21.15.120:0/3486546862 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5650c27b4720 0x5650c27b6b10 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.127+0000 7effef415740 1 --2- 172.21.15.120:0/3486546862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650c2739240 0x5650c273e150 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.127+0000 7effef415740 1 --2- 172.21.15.120:0/3486546862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650c272e980 0x5650c27418a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.127+0000 7effef415740 1 --2- 172.21.15.120:0/3486546862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650c272e060 0x5650c2741360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.127+0000 7effef415740 1 -- 172.21.15.120:0/3486546862 >> 172.21.15.120:0/3486546862 conn(0x5650c272a5f0 msgr2=0x5650c2738040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:33.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.127+0000 7effcaff5640 0 client.16761 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:33.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.127+0000 7effef415740 1 -- 172.21.15.120:0/3486546862 shutdown_connections 2024-09-19T09:06:33.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.127+0000 7effef415740 1 -- 172.21.15.120:0/3486546862 wait complete. 2024-09-19T09:06:33.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.128+0000 7effef415740 20 client.16761 trim_cache size 0 max 16384 2024-09-19T09:06:33.130 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:06:33.130 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 28, 2024-09-19T09:06:33.130 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:06:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 16, 2024-09-19T09:06:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:06:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: "b11bef3b": { 2024-09-19T09:06:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 16, 2024-09-19T09:06:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:06:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:06:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:06:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:06:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:06:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:06:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 136.6, 2024-09-19T09:06:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:06:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:06:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:06:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:06:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:06:33.132 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:06:33.132 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:06:33.132 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:06:33.132 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:06:33.132 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:06:33.132 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:06:33.132 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:06:33.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.128+0000 7effed03c640 1 -- 172.21.15.120:0/1227232131 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7effd00a3240 msgr2=0x7effd00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:33.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.128+0000 7effed03c640 1 --2- 172.21.15.120:0/1227232131 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7effd00a3240 0x7effd00a5700 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7effe000f000 tx=0x7effe001e000 comp rx=0 tx=0).stop 2024-09-19T09:06:33.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.128+0000 7effed03c640 1 -- 172.21.15.120:0/1227232131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe8102d30 msgr2=0x7effe81985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:33.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.128+0000 7effed03c640 1 --2- 172.21.15.120:0/1227232131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe8102d30 0x7effe81985e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7effd800c970 tx=0x7effd800ce30 comp rx=0 tx=0).stop 2024-09-19T09:06:33.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.128+0000 7effed03c640 1 -- 172.21.15.120:0/1227232131 shutdown_connections 2024-09-19T09:06:33.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.128+0000 7effed03c640 1 --2- 172.21.15.120:0/1227232131 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7effd00a3240 0x7effd00a5700 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.128+0000 7effed03c640 1 --2- 172.21.15.120:0/1227232131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7effe81082a0 0x7effe819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.128+0000 7effed03c640 1 --2- 172.21.15.120:0/1227232131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effe81036e0 0x7effe8198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.128+0000 7effed03c640 1 --2- 172.21.15.120:0/1227232131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe8102d30 0x7effe81985e0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.128+0000 7effed03c640 1 -- 172.21.15.120:0/1227232131 >> 172.21.15.120:0/1227232131 conn(0x7effe80fc7d0 msgr2=0x7effe8105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:33.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.128+0000 7effed03c640 1 -- 172.21.15.120:0/1227232131 shutdown_connections 2024-09-19T09:06:33.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:33.128+0000 7effed03c640 1 -- 172.21.15.120:0/1227232131 wait complete. 2024-09-19T09:06:33.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:32 smithi196 ceph-mon[35150]: pgmap v451: 97 pgs: 97 active+clean; 46 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 2.6 MiB/s wr, 199 op/s 2024-09-19T09:06:33.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:32 smithi153 ceph-mon[35071]: pgmap v451: 97 pgs: 97 active+clean; 46 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 2.6 MiB/s wr, 199 op/s 2024-09-19T09:06:33.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.932+0000 7f56fac77640 1 -- 172.21.15.120:0/3217119012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f56f4101c30 msgr2=0x7f56f41020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:33.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.932+0000 7f56fac77640 1 --2- 172.21.15.120:0/3217119012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f56f4101c30 0x7f56f41020b0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f56e0009900 tx=0x7f56e002a410 comp rx=0 tx=0).stop 2024-09-19T09:06:33.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.932+0000 7f56fac77640 1 -- 172.21.15.120:0/3217119012 shutdown_connections 2024-09-19T09:06:33.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.932+0000 7f56fac77640 1 --2- 172.21.15.120:0/3217119012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56f41025f0 0x7f56f4108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.932+0000 7f56fac77640 1 --2- 172.21.15.120:0/3217119012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f56f4101c30 0x7f56f41020b0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.932+0000 7f56fac77640 1 --2- 172.21.15.120:0/3217119012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56f4100a40 0x7f56f4100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.932+0000 7f56fac77640 1 -- 172.21.15.120:0/3217119012 >> 172.21.15.120:0/3217119012 conn(0x7f56f40fc1e0 msgr2=0x7f56f40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.932+0000 7f56fac77640 1 -- 172.21.15.120:0/3217119012 shutdown_connections 2024-09-19T09:06:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.932+0000 7f56fac77640 1 -- 172.21.15.120:0/3217119012 wait complete. 2024-09-19T09:06:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.932+0000 7f56fac77640 1 Processor -- start 2024-09-19T09:06:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.933+0000 7f56fac77640 1 -- start start 2024-09-19T09:06:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.933+0000 7f56fac77640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56f4100a40 0x7f56f4196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.933+0000 7f56fac77640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56f4101c30 0x7f56f41971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.933+0000 7f56fac77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f56f41025f0 0x7f56f419e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.933+0000 7f56fac77640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f56f410c6c0 con 0x7f56f41025f0 2024-09-19T09:06:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.933+0000 7f56fac77640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f56f410c540 con 0x7f56f4100a40 2024-09-19T09:06:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.933+0000 7f56fac77640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56f410c840 con 0x7f56f4101c30 2024-09-19T09:06:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.933+0000 7f56f9214640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f56f41025f0 0x7f56f419e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.933+0000 7f56f8a13640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56f4100a40 0x7f56f4196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.933+0000 7f56f9214640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f56f41025f0 0x7f56f419e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40278/0 (socket says 172.21.15.120:40278) 2024-09-19T09:06:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.933+0000 7f56f9214640 1 -- 172.21.15.120:0/98727872 learned_addr learned my addr 172.21.15.120:0/98727872 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.933+0000 7f56f3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56f4101c30 0x7f56f41971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.933+0000 7f56f9214640 1 -- 172.21.15.120:0/98727872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56f4101c30 msgr2=0x7f56f41971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.933+0000 7f56f9214640 1 --2- 172.21.15.120:0/98727872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56f4101c30 0x7f56f41971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.933+0000 7f56f9214640 1 -- 172.21.15.120:0/98727872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56f4100a40 msgr2=0x7f56f4196cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.933+0000 7f56f9214640 1 --2- 172.21.15.120:0/98727872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56f4100a40 0x7f56f4196cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.933+0000 7f56f9214640 1 -- 172.21.15.120:0/98727872 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f419e8f0 con 0x7f56f41025f0 2024-09-19T09:06:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.933+0000 7f56f9214640 1 --2- 172.21.15.120:0/98727872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f56f41025f0 0x7f56f419e250 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00eca0 tx=0x7f56ec00b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.933+0000 7f56f1ffb640 1 -- 172.21.15.120:0/98727872 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec004b20 con 0x7f56f41025f0 2024-09-19T09:06:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.933+0000 7f56fac77640 1 -- 172.21.15.120:0/98727872 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f4069900 con 0x7f56f41025f0 2024-09-19T09:06:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.933+0000 7f56f1ffb640 1 -- 172.21.15.120:0/98727872 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56ec004cc0 con 0x7f56f41025f0 2024-09-19T09:06:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.933+0000 7f56f1ffb640 1 -- 172.21.15.120:0/98727872 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec017670 con 0x7f56f41025f0 2024-09-19T09:06:33.938 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.934+0000 7f56fac77640 1 -- 172.21.15.120:0/98727872 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f4069e20 con 0x7f56f41025f0 2024-09-19T09:06:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.936+0000 7f56f1ffb640 1 -- 172.21.15.120:0/98727872 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f56ec005020 con 0x7f56f41025f0 2024-09-19T09:06:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.936+0000 7f56fda13740 1 Processor -- start 2024-09-19T09:06:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.936+0000 7f56fda13740 1 -- start start 2024-09-19T09:06:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.936+0000 7f56fda13740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1f84ee9a0 0x55a1f84eeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.936+0000 7f56fda13740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1f84ef2e0 0x55a1f84f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.936+0000 7f56fda13740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f84f4370 0x55a1f84f6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.936+0000 7f56fda13740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a1f84522f0 con 0x55a1f84f4370 2024-09-19T09:06:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.936+0000 7f56fda13740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a1f83fa270 con 0x55a1f84ee9a0 2024-09-19T09:06:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.936+0000 7f56fda13740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a1f844e2b0 con 0x55a1f84ef2e0 2024-09-19T09:06:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.936+0000 7f56f3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f84f4370 0x55a1f84f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.936+0000 7f56f1ffb640 1 --2- 172.21.15.120:0/98727872 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f56e40a32f0 0x7f56e40a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.937+0000 7f56f3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f84f4370 0x55a1f84f6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40284/0 (socket says 172.21.15.120:40284) 2024-09-19T09:06:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.937+0000 7f56f3fff640 1 -- 172.21.15.120:0/588831160 learned_addr learned my addr 172.21.15.120:0/588831160 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.937+0000 7f56f8a13640 1 --2- 172.21.15.120:0/588831160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1f84ef2e0 0x55a1f84f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.937+0000 7f56f3fff640 1 --2- 172.21.15.120:0/588831160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1f84ee9a0 0x55a1f84eeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.937+0000 7f56f9214640 1 --2- 172.21.15.120:0/98727872 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f56e40a32f0 0x7f56e40a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.937+0000 7f56f1ffb640 1 -- 172.21.15.120:0/98727872 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f56ec0d1310 con 0x7f56f41025f0 2024-09-19T09:06:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.937+0000 7f56f8a13640 1 -- 172.21.15.120:0/588831160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1f84ee9a0 msgr2=0x55a1f84eeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.937+0000 7f56f8a13640 1 --2- 172.21.15.120:0/588831160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1f84ee9a0 0x55a1f84eeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.937+0000 7f56f8a13640 1 -- 172.21.15.120:0/588831160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f84f4370 msgr2=0x55a1f84f6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.937+0000 7f56f8a13640 1 --2- 172.21.15.120:0/588831160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f84f4370 0x55a1f84f6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.937+0000 7f56f8a13640 1 -- 172.21.15.120:0/588831160 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1f83f9340 con 0x55a1f84ef2e0 2024-09-19T09:06:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.937+0000 7f56f3fff640 1 --2- 172.21.15.120:0/588831160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f84f4370 0x55a1f84f6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.937+0000 7f56f9214640 1 --2- 172.21.15.120:0/98727872 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f56e40a32f0 0x7f56e40a57b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00ffd0 tx=0x7f56ec00ef70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.937+0000 7f56f8a13640 1 --2- 172.21.15.120:0/588831160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1f84ef2e0 0x55a1f84f3da0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f56f4101a90 tx=0x7f56e8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3130451ceb528723 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.937+0000 7f56deffd640 1 -- 172.21.15.120:0/588831160 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e8009d70 con 0x55a1f84ef2e0 2024-09-19T09:06:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.937+0000 7f56deffd640 1 -- 172.21.15.120:0/588831160 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56e80035f0 con 0x55a1f84ef2e0 2024-09-19T09:06:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.937+0000 7f56deffd640 1 -- 172.21.15.120:0/588831160 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e8019670 con 0x55a1f84ef2e0 2024-09-19T09:06:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.938+0000 7f56fda13740 1 -- 172.21.15.120:0/588831160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1f84ef2e0 msgr2=0x55a1f84f3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.938+0000 7f56fda13740 1 --2- 172.21.15.120:0/588831160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1f84ef2e0 0x55a1f84f3da0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f56f4101a90 tx=0x7f56e8011810 comp rx=0 tx=0).stop 2024-09-19T09:06:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.938+0000 7f56fda13740 1 -- 172.21.15.120:0/588831160 shutdown_connections 2024-09-19T09:06:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.938+0000 7f56fda13740 1 --2- 172.21.15.120:0/588831160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f84f4370 0x55a1f84f6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.938+0000 7f56fda13740 1 --2- 172.21.15.120:0/588831160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1f84ef2e0 0x55a1f84f3da0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.938+0000 7f56fda13740 1 --2- 172.21.15.120:0/588831160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1f84ee9a0 0x55a1f84eeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.938+0000 7f56fda13740 1 -- 172.21.15.120:0/588831160 >> 172.21.15.120:0/588831160 conn(0x55a1f84ea010 msgr2=0x55a1f84ea410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.938+0000 7f56fda13740 1 -- 172.21.15.120:0/588831160 shutdown_connections 2024-09-19T09:06:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.938+0000 7f56fda13740 1 -- 172.21.15.120:0/588831160 wait complete. 2024-09-19T09:06:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.938+0000 7f56fda13740 1 -- start start 2024-09-19T09:06:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.938+0000 7f56fda13740 1 Processor -- start 2024-09-19T09:06:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.939+0000 7f56fda13740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1f84ee9a0 0x55a1f84f8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.939+0000 7f56fda13740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1f84ef2e0 0x55a1f84f90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.939+0000 7f56fda13740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f84f4370 0x55a1f8500110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.939+0000 7f56fda13740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a1f84f7340 con 0x55a1f84f4370 2024-09-19T09:06:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.939+0000 7f56fda13740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a1f83db290 con 0x55a1f84ee9a0 2024-09-19T09:06:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.939+0000 7f56fda13740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a1f84f74c0 con 0x55a1f84ef2e0 2024-09-19T09:06:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.939+0000 7f56fda13740 10 client.0 _ll_register_callbacks cb 0x55a1f840c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.939+0000 7f56f3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f84f4370 0x55a1f8500110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.939+0000 7f56f3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f84f4370 0x55a1f8500110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40294/0 (socket says 172.21.15.120:40294) 2024-09-19T09:06:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.939+0000 7f56f3fff640 1 -- 172.21.15.120:0/1796677654 learned_addr learned my addr 172.21.15.120:0/1796677654 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.939+0000 7f56f8a13640 1 --2- 172.21.15.120:0/1796677654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1f84ef2e0 0x55a1f84f90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.939+0000 7f56f3fff640 1 --2- 172.21.15.120:0/1796677654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1f84ee9a0 0x55a1f84f8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.939+0000 7f56f3fff640 1 -- 172.21.15.120:0/1796677654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1f84ef2e0 msgr2=0x55a1f84f90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.939+0000 7f56f3fff640 1 --2- 172.21.15.120:0/1796677654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1f84ef2e0 0x55a1f84f90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.939+0000 7f56f3fff640 1 -- 172.21.15.120:0/1796677654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1f84ee9a0 msgr2=0x55a1f84f8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.939+0000 7f56f3fff640 1 --2- 172.21.15.120:0/1796677654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1f84ee9a0 0x55a1f84f8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.939+0000 7f56f3fff640 1 -- 172.21.15.120:0/1796677654 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1f8165210 con 0x55a1f84f4370 2024-09-19T09:06:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.939+0000 7f56f3fff640 1 --2- 172.21.15.120:0/1796677654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f84f4370 0x55a1f8500110 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x55a1f84f80c0 tx=0x7f56e002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3677f19b20e931 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.939+0000 7f56d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:06:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.940+0000 7f56d3fff640 1 -- 172.21.15.120:0/1796677654 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e0035040 con 0x55a1f84f4370 2024-09-19T09:06:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.940+0000 7f56d3fff640 1 -- 172.21.15.120:0/1796677654 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f56e003e8a0 con 0x55a1f84f4370 2024-09-19T09:06:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.940+0000 7f56fda13740 1 -- 172.21.15.120:0/1796677654 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56e80035f0 con 0x55a1f84f4370 2024-09-19T09:06:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.940+0000 7f56d3fff640 1 -- 172.21.15.120:0/1796677654 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f56e003eaa0 con 0x55a1f84f4370 2024-09-19T09:06:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.940+0000 7f56fda13740 10 client.16773 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.940+0000 7f56fda13740 1 -- 172.21.15.120:0/1796677654 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a1f8231620 con 0x55a1f84f4370 2024-09-19T09:06:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.940+0000 7f56d3fff640 1 -- 172.21.15.120:0/1796677654 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56e80035f0 con 0x55a1f84f4370 2024-09-19T09:06:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.940+0000 7f56d3fff640 1 -- 172.21.15.120:0/1796677654 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f56e00048a0 con 0x55a1f84f4370 2024-09-19T09:06:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.940+0000 7f56fda13740 10 client.16773 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.940+0000 7f56fda13740 10 client.16773 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:06:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.940+0000 7f56fda13740 20 client.16773 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.940+0000 7f56fda13740 1 --2- 172.21.15.120:0/1796677654 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a1f8504770 0x55a1f8506c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.940+0000 7f56fda13740 4 client.16773 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:06:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.940+0000 7f56fda13740 1 -- 172.21.15.120:0/1796677654 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1f845fa20 con 0x55a1f8504770 2024-09-19T09:06:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.940+0000 7f56f8a13640 1 --2- 172.21.15.120:0/1796677654 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a1f8504770 0x55a1f8506c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.941+0000 7f56f8a13640 1 --2- 172.21.15.120:0/1796677654 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a1f8504770 0x55a1f8506c00 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a0abce345c3c319 server_cookie=9ffa95f3792d6287 in_seq=0 out_seq=0 2024-09-19T09:06:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.941+0000 7f56d3fff640 10 client.16773 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:33.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.942+0000 7f56d3fff640 1 -- 172.21.15.120:0/1796677654 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a1f845fa20 con 0x55a1f8504770 2024-09-19T09:06:33.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.942+0000 7f56d3fff640 10 client.16773 handle_command_reply: tid=0 2024-09-19T09:06:33.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.942+0000 7f56fda13740 1 client.16773 shutdown 2024-09-19T09:06:33.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.942+0000 7f56fda13740 1 -- 172.21.15.120:0/1796677654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f84f4370 msgr2=0x55a1f8500110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:33.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.942+0000 7f56fda13740 1 --2- 172.21.15.120:0/1796677654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f84f4370 0x55a1f8500110 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x55a1f84f80c0 tx=0x7f56e002a410 comp rx=0 tx=0).stop 2024-09-19T09:06:33.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.942+0000 7f56fda13740 1 -- 172.21.15.120:0/1796677654 shutdown_connections 2024-09-19T09:06:33.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.942+0000 7f56fda13740 1 --2- 172.21.15.120:0/1796677654 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a1f8504770 0x55a1f8506c00 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.942+0000 7f56fda13740 1 --2- 172.21.15.120:0/1796677654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f84f4370 0x55a1f8500110 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.942+0000 7f56fda13740 1 --2- 172.21.15.120:0/1796677654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1f84ef2e0 0x55a1f84f90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.942+0000 7f56fda13740 1 --2- 172.21.15.120:0/1796677654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1f84ee9a0 0x55a1f84f8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.942+0000 7f56d3fff640 0 client.16773 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:33.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.942+0000 7f56fda13740 1 -- 172.21.15.120:0/1796677654 >> 172.21.15.120:0/1796677654 conn(0x55a1f84ea010 msgr2=0x55a1f84eb820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:33.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.942+0000 7f56fda13740 1 -- 172.21.15.120:0/1796677654 shutdown_connections 2024-09-19T09:06:33.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.942+0000 7f56fda13740 1 -- 172.21.15.120:0/1796677654 wait complete. 2024-09-19T09:06:33.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.942+0000 7f56fda13740 20 client.16773 trim_cache size 0 max 16384 2024-09-19T09:06:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.960+0000 7f56fda13740 1 Processor -- start 2024-09-19T09:06:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.960+0000 7f56fda13740 1 -- start start 2024-09-19T09:06:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.960+0000 7f56fda13740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f84ee9a0 0x55a1f84e90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.960+0000 7f56fda13740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1f84ef2e0 0x55a1f85000d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.960+0000 7f56fda13740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1f84f4370 0x55a1f850b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.960+0000 7f56fda13740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f56f41a6e30 con 0x55a1f84ee9a0 2024-09-19T09:06:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.960+0000 7f56fda13740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f56f41a8970 con 0x55a1f84ef2e0 2024-09-19T09:06:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.960+0000 7f56fda13740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56f41a7500 con 0x55a1f84f4370 2024-09-19T09:06:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.960+0000 7f56f3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f84ee9a0 0x55a1f84e90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.960+0000 7f56f8a13640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1f84ef2e0 0x55a1f85000d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.960+0000 7f56f3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f84ee9a0 0x55a1f84e90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40298/0 (socket says 172.21.15.120:40298) 2024-09-19T09:06:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.960+0000 7f56f3fff640 1 -- 172.21.15.120:0/3529293843 learned_addr learned my addr 172.21.15.120:0/3529293843 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.960+0000 7f56f3fff640 1 --2- 172.21.15.120:0/3529293843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1f84f4370 0x55a1f850b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.960+0000 7f56f8a13640 1 -- 172.21.15.120:0/3529293843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1f84f4370 msgr2=0x55a1f850b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.960+0000 7f56f8a13640 1 --2- 172.21.15.120:0/3529293843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1f84f4370 0x55a1f850b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.960+0000 7f56f8a13640 1 -- 172.21.15.120:0/3529293843 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f84ee9a0 msgr2=0x55a1f84e90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.960+0000 7f56f8a13640 1 --2- 172.21.15.120:0/3529293843 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f84ee9a0 0x55a1f84e90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.960+0000 7f56f8a13640 1 -- 172.21.15.120:0/3529293843 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1f84eaca0 con 0x55a1f84ef2e0 2024-09-19T09:06:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.961+0000 7f56f8a13640 1 --2- 172.21.15.120:0/3529293843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1f84ef2e0 0x55a1f85000d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f56e8011d10 tx=0x7f56e80037c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b54e9c86a2d4d98b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.961+0000 7f56de7fc640 1 -- 172.21.15.120:0/3529293843 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e8032070 con 0x55a1f84ef2e0 2024-09-19T09:06:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.961+0000 7f56de7fc640 1 -- 172.21.15.120:0/3529293843 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56e802a070 con 0x55a1f84ef2e0 2024-09-19T09:06:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.961+0000 7f56de7fc640 1 -- 172.21.15.120:0/3529293843 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e802c320 con 0x55a1f84ef2e0 2024-09-19T09:06:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.962+0000 7f56fda13740 1 -- 172.21.15.120:0/3529293843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1f84ef2e0 msgr2=0x55a1f85000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.962+0000 7f56fda13740 1 --2- 172.21.15.120:0/3529293843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1f84ef2e0 0x55a1f85000d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f56e8011d10 tx=0x7f56e80037c0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.962+0000 7f56fda13740 1 -- 172.21.15.120:0/3529293843 shutdown_connections 2024-09-19T09:06:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.962+0000 7f56fda13740 1 --2- 172.21.15.120:0/3529293843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1f84f4370 0x55a1f850b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.962+0000 7f56fda13740 1 --2- 172.21.15.120:0/3529293843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1f84ef2e0 0x55a1f85000d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.962+0000 7f56fda13740 1 --2- 172.21.15.120:0/3529293843 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f84ee9a0 0x55a1f84e90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.962+0000 7f56fda13740 1 -- 172.21.15.120:0/3529293843 >> 172.21.15.120:0/3529293843 conn(0x55a1f84ea010 msgr2=0x55a1f84e89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.962+0000 7f56fda13740 1 -- 172.21.15.120:0/3529293843 shutdown_connections 2024-09-19T09:06:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.962+0000 7f56fda13740 1 -- 172.21.15.120:0/3529293843 wait complete. 2024-09-19T09:06:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.962+0000 7f56fda13740 1 -- start start 2024-09-19T09:06:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.962+0000 7f56fda13740 1 Processor -- start 2024-09-19T09:06:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.962+0000 7f56fda13740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1f84ee9a0 0x55a1f84fe000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.962+0000 7f56fda13740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1f84ef2e0 0x55a1f84fe540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.962+0000 7f56fda13740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f84f4370 0x55a1f84ff440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.962+0000 7f56fda13740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f56f41a82a0 con 0x55a1f84f4370 2024-09-19T09:06:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.962+0000 7f56fda13740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f56f41a7bd0 con 0x55a1f84ef2e0 2024-09-19T09:06:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.962+0000 7f56fda13740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56f41a4df0 con 0x55a1f84ee9a0 2024-09-19T09:06:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.963+0000 7f56fda13740 10 client.0 _ll_register_callbacks cb 0x7f56e0031970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.963+0000 7f56f8a13640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1f84ef2e0 0x55a1f84fe540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.963+0000 7f56f8a13640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1f84ef2e0 0x55a1f84fe540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35616/0 (socket says 172.21.15.120:35616) 2024-09-19T09:06:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.963+0000 7f56f8a13640 1 -- 172.21.15.120:0/1471785680 learned_addr learned my addr 172.21.15.120:0/1471785680 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.963+0000 7f56f3fff640 1 --2- 172.21.15.120:0/1471785680 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f84f4370 0x55a1f84ff440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.963+0000 7f56f3fff640 1 --2- 172.21.15.120:0/1471785680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1f84ee9a0 0x55a1f84fe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.963+0000 7f56f8a13640 1 -- 172.21.15.120:0/1471785680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1f84ee9a0 msgr2=0x55a1f84fe000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.963+0000 7f56f8a13640 1 --2- 172.21.15.120:0/1471785680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1f84ee9a0 0x55a1f84fe000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.963+0000 7f56f8a13640 1 -- 172.21.15.120:0/1471785680 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f84f4370 msgr2=0x55a1f84ff440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.963+0000 7f56f8a13640 1 --2- 172.21.15.120:0/1471785680 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f84f4370 0x55a1f84ff440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.963+0000 7f56f8a13640 1 -- 172.21.15.120:0/1471785680 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1f84ffa50 con 0x55a1f84ef2e0 2024-09-19T09:06:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.963+0000 7f56f3fff640 1 --2- 172.21.15.120:0/1471785680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1f84ee9a0 0x55a1f84fe000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:06:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.963+0000 7f56f8a13640 1 --2- 172.21.15.120:0/1471785680 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1f84ef2e0 0x55a1f84fe540 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f56e8005fd0 tx=0x7f56e8009d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8faa250298215921 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.963+0000 7f56d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:06:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.963+0000 7f56d3fff640 1 -- 172.21.15.120:0/1471785680 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e803b060 con 0x55a1f84ef2e0 2024-09-19T09:06:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.963+0000 7f56fda13740 1 -- 172.21.15.120:0/1471785680 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56e802a070 con 0x55a1f84ef2e0 2024-09-19T09:06:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.963+0000 7f56d3fff640 1 -- 172.21.15.120:0/1471785680 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f56e80050c0 con 0x55a1f84ef2e0 2024-09-19T09:06:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.964+0000 7f56d3fff640 1 -- 172.21.15.120:0/1471785680 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f56e8032070 con 0x55a1f84ef2e0 2024-09-19T09:06:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.964+0000 7f56fda13740 10 client.26120 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.964+0000 7f56fda13740 1 -- 172.21.15.120:0/1471785680 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a1f856fbf0 con 0x55a1f84ef2e0 2024-09-19T09:06:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.964+0000 7f56d3fff640 1 -- 172.21.15.120:0/1471785680 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56e802a070 con 0x55a1f84ef2e0 2024-09-19T09:06:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.964+0000 7f56d3fff640 1 -- 172.21.15.120:0/1471785680 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f56e8036450 con 0x55a1f84ef2e0 2024-09-19T09:06:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.964+0000 7f56fda13740 10 client.26120 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.964+0000 7f56fda13740 10 client.26120 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:06:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.964+0000 7f56fda13740 20 client.26120 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.964+0000 7f56fda13740 1 --2- 172.21.15.120:0/1471785680 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a1f8570440 0x55a1f8572860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.964+0000 7f56fda13740 4 client.26120 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:06:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.964+0000 7f56fda13740 1 -- 172.21.15.120:0/1471785680 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a1f8459d70 con 0x55a1f8570440 2024-09-19T09:06:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.964+0000 7f56f3fff640 1 --2- 172.21.15.120:0/1471785680 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a1f8570440 0x55a1f8572860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.965+0000 7f56d3fff640 10 client.26120 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:33.965+0000 7f56f3fff640 1 --2- 172.21.15.120:0/1471785680 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a1f8570440 0x55a1f8572860 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32e574f3212cdebf server_cookie=49d9322f7d7b781 in_seq=0 out_seq=0 2024-09-19T09:06:33.987 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b11bef3b', total seconds elapsed: 20.3 2024-09-19T09:06:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:34.018+0000 7f56d3fff640 1 -- 172.21.15.120:0/1471785680 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a1f8459d70 con 0x55a1f8570440 2024-09-19T09:06:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:34.018+0000 7f56d3fff640 10 client.26120 handle_command_reply: tid=0 2024-09-19T09:06:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:34.018+0000 7f56fda13740 1 client.26120 shutdown 2024-09-19T09:06:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:34.018+0000 7f56fda13740 1 -- 172.21.15.120:0/1471785680 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1f84ef2e0 msgr2=0x55a1f84fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:34.018+0000 7f56fda13740 1 --2- 172.21.15.120:0/1471785680 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1f84ef2e0 0x55a1f84fe540 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f56e8005fd0 tx=0x7f56e8009d70 comp rx=0 tx=0).stop 2024-09-19T09:06:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:34.019+0000 7f56fda13740 1 -- 172.21.15.120:0/1471785680 shutdown_connections 2024-09-19T09:06:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:34.019+0000 7f56fda13740 1 --2- 172.21.15.120:0/1471785680 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a1f8570440 0x55a1f8572860 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:34.019+0000 7f56fda13740 1 --2- 172.21.15.120:0/1471785680 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1f84f4370 0x55a1f84ff440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:34.019+0000 7f56fda13740 1 --2- 172.21.15.120:0/1471785680 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1f84ef2e0 0x55a1f84fe540 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:34.019+0000 7f56d3fff640 0 client.26120 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:34.019+0000 7f56fda13740 1 --2- 172.21.15.120:0/1471785680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1f84ee9a0 0x55a1f84fe000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:34.019+0000 7f56fda13740 1 -- 172.21.15.120:0/1471785680 >> 172.21.15.120:0/1471785680 conn(0x55a1f84ea010 msgr2=0x55a1f84e91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:34.019+0000 7f56fda13740 1 -- 172.21.15.120:0/1471785680 shutdown_connections 2024-09-19T09:06:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:34.019+0000 7f56fda13740 1 -- 172.21.15.120:0/1471785680 wait complete. 2024-09-19T09:06:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:34.019+0000 7f56fda13740 20 client.26120 trim_cache size 0 max 16384 2024-09-19T09:06:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:06:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:06:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:06:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:06:34.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:34.019+0000 7f56fac77640 1 -- 172.21.15.120:0/98727872 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f56e40a32f0 msgr2=0x7f56e40a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:34.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:34.019+0000 7f56fac77640 1 --2- 172.21.15.120:0/98727872 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f56e40a32f0 0x7f56e40a57b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00ffd0 tx=0x7f56ec00ef70 comp rx=0 tx=0).stop 2024-09-19T09:06:34.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:34.019+0000 7f56fac77640 1 -- 172.21.15.120:0/98727872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f56f41025f0 msgr2=0x7f56f419e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:34.019+0000 7f56fac77640 1 --2- 172.21.15.120:0/98727872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f56f41025f0 0x7f56f419e250 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00eca0 tx=0x7f56ec00b690 comp rx=0 tx=0).stop 2024-09-19T09:06:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:34.019+0000 7f56fac77640 1 -- 172.21.15.120:0/98727872 shutdown_connections 2024-09-19T09:06:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:34.019+0000 7f56fac77640 1 --2- 172.21.15.120:0/98727872 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f56e40a32f0 0x7f56e40a57b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:34.019+0000 7f56fac77640 1 --2- 172.21.15.120:0/98727872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f56f41025f0 0x7f56f419e250 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:34.019+0000 7f56fac77640 1 --2- 172.21.15.120:0/98727872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56f4101c30 0x7f56f41971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:34.019+0000 7f56fac77640 1 --2- 172.21.15.120:0/98727872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56f4100a40 0x7f56f4196cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:34.019+0000 7f56fac77640 1 -- 172.21.15.120:0/98727872 >> 172.21.15.120:0/98727872 conn(0x7f56f40fc1e0 msgr2=0x7f56f4104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:34.019+0000 7f56fac77640 1 -- 172.21.15.120:0/98727872 shutdown_connections 2024-09-19T09:06:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:34.020+0000 7f56fac77640 1 -- 172.21.15.120:0/98727872 wait complete. 2024-09-19T09:06:34.043 INFO:tasks.workunit.client.0.smithi120.stdout: 1 262327 6.44 MB/sec execute 29 sec latency 16624.599 ms 2024-09-19T09:06:35.043 INFO:tasks.workunit.client.0.smithi120.stdout: 1 264872 6.61 MB/sec execute 30 sec latency 46.790 ms 2024-09-19T09:06:35.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:34 smithi196 ceph-mon[35150]: pgmap v452: 97 pgs: 97 active+clean; 46 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.8 MiB/s wr, 165 op/s 2024-09-19T09:06:35.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:34 smithi153 ceph-mon[35071]: pgmap v452: 97 pgs: 97 active+clean; 46 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.8 MiB/s wr, 165 op/s 2024-09-19T09:06:35.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:34 smithi120 ceph-mon[31150]: pgmap v452: 97 pgs: 97 active+clean; 46 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.8 MiB/s wr, 165 op/s 2024-09-19T09:06:36.043 INFO:tasks.workunit.client.0.smithi120.stdout: 1 267346 6.86 MB/sec execute 31 sec latency 11.841 ms 2024-09-19T09:06:37.043 INFO:tasks.workunit.client.0.smithi120.stdout: 1 270405 7.20 MB/sec execute 32 sec latency 12.501 ms 2024-09-19T09:06:37.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:36 smithi196 ceph-mon[35150]: pgmap v453: 97 pgs: 97 active+clean; 45 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 2.6 MiB/s wr, 141 op/s 2024-09-19T09:06:37.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:36 smithi153 ceph-mon[35071]: pgmap v453: 97 pgs: 97 active+clean; 45 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 2.6 MiB/s wr, 141 op/s 2024-09-19T09:06:37.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:36 smithi120 ceph-mon[31150]: pgmap v453: 97 pgs: 97 active+clean; 45 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 2.6 MiB/s wr, 141 op/s 2024-09-19T09:06:38.043 INFO:tasks.workunit.client.0.smithi120.stdout: 1 273619 7.54 MB/sec execute 33 sec latency 7.374 ms 2024-09-19T09:06:39.043 INFO:tasks.workunit.client.0.smithi120.stdout: 1 276945 7.87 MB/sec execute 34 sec latency 10.227 ms 2024-09-19T09:06:39.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:38 smithi196 ceph-mon[35150]: pgmap v454: 97 pgs: 97 active+clean; 40 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 2.7 MiB/s wr, 161 op/s 2024-09-19T09:06:39.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:38 smithi153 ceph-mon[35071]: pgmap v454: 97 pgs: 97 active+clean; 40 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 2.7 MiB/s wr, 161 op/s 2024-09-19T09:06:39.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:38 smithi120 ceph-mon[31150]: pgmap v454: 97 pgs: 97 active+clean; 40 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 2.7 MiB/s wr, 161 op/s 2024-09-19T09:06:40.043 INFO:tasks.workunit.client.0.smithi120.stdout: 1 279703 8.02 MB/sec execute 35 sec latency 59.289 ms 2024-09-19T09:06:40.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:39 smithi196 ceph-mon[35150]: pgmap v455: 97 pgs: 97 active+clean; 34 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 198 op/s 2024-09-19T09:06:40.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:39 smithi153 ceph-mon[35071]: pgmap v455: 97 pgs: 97 active+clean; 34 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 198 op/s 2024-09-19T09:06:40.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:39 smithi120 ceph-mon[31150]: pgmap v455: 97 pgs: 97 active+clean; 34 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 198 op/s 2024-09-19T09:06:40.942 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:06:41.043 INFO:tasks.workunit.client.0.smithi120.stdout: 1 282442 8.14 MB/sec execute 36 sec latency 15.661 ms 2024-09-19T09:06:41.205 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:06:41.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.869+0000 7f2dd21ad640 1 -- 172.21.15.120:0/1760341534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dcc102ef0 msgr2=0x7f2dcc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:41.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.869+0000 7f2dd21ad640 1 --2- 172.21.15.120:0/1760341534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dcc102ef0 0x7f2dcc10da40 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4009ca0 tx=0x7f2dc402a950 comp rx=0 tx=0).stop 2024-09-19T09:06:41.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dd21ad640 1 -- 172.21.15.120:0/1760341534 shutdown_connections 2024-09-19T09:06:41.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dd21ad640 1 --2- 172.21.15.120:0/1760341534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dcc102ef0 0x7f2dcc10da40 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dd21ad640 1 --2- 172.21.15.120:0/1760341534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dcc102570 0x7f2dcc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dd21ad640 1 --2- 172.21.15.120:0/1760341534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2dcc100e20 0x7f2dcc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dd21ad640 1 -- 172.21.15.120:0/1760341534 >> 172.21.15.120:0/1760341534 conn(0x7f2dcc0fc7d0 msgr2=0x7f2dcc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:41.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dd21ad640 1 -- 172.21.15.120:0/1760341534 shutdown_connections 2024-09-19T09:06:41.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dd21ad640 1 -- 172.21.15.120:0/1760341534 wait complete. 2024-09-19T09:06:41.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dd21ad640 1 Processor -- start 2024-09-19T09:06:41.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dd21ad640 1 -- start start 2024-09-19T09:06:41.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dd21ad640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2dcc100e20 0x7f2dcc19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:41.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dd21ad640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dcc102570 0x7f2dcc19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:41.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dd21ad640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dcc102ef0 0x7f2dcc19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:41.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dd21ad640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2dcc110fd0 con 0x7f2dcc102570 2024-09-19T09:06:41.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dd21ad640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2dcc110e50 con 0x7f2dcc102ef0 2024-09-19T09:06:41.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dd21ad640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2dcc111150 con 0x7f2dcc100e20 2024-09-19T09:06:41.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dc3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dcc102570 0x7f2dcc19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:41.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dc3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dcc102570 0x7f2dcc19abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37078/0 (socket says 172.21.15.120:37078) 2024-09-19T09:06:41.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dcb7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2dcc100e20 0x7f2dcc19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:41.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dc3fff640 1 -- 172.21.15.120:0/1368568161 learned_addr learned my addr 172.21.15.120:0/1368568161 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:41.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dcbfff640 1 --2- 172.21.15.120:0/1368568161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dcc102ef0 0x7f2dcc19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:41.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dc3fff640 1 -- 172.21.15.120:0/1368568161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2dcc100e20 msgr2=0x7f2dcc19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:41.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dc3fff640 1 --2- 172.21.15.120:0/1368568161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2dcc100e20 0x7f2dcc19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dc3fff640 1 -- 172.21.15.120:0/1368568161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dcc102ef0 msgr2=0x7f2dcc19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:41.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dc3fff640 1 --2- 172.21.15.120:0/1368568161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dcc102ef0 0x7f2dcc19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dc3fff640 1 -- 172.21.15.120:0/1368568161 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dcc19f620 con 0x7f2dcc102570 2024-09-19T09:06:41.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dc3fff640 1 --2- 172.21.15.120:0/1368568161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dcc102570 0x7f2dcc19abb0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2db400d950 tx=0x7f2db400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:41.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dc97fa640 1 -- 172.21.15.120:0/1368568161 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db4017030 con 0x7f2dcc102570 2024-09-19T09:06:41.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dd21ad640 1 -- 172.21.15.120:0/1368568161 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dcc19f910 con 0x7f2dcc102570 2024-09-19T09:06:41.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dc97fa640 1 -- 172.21.15.120:0/1368568161 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2db4004020 con 0x7f2dcc102570 2024-09-19T09:06:41.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dc97fa640 1 -- 172.21.15.120:0/1368568161 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db40047a0 con 0x7f2dcc102570 2024-09-19T09:06:41.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.871+0000 7f2dd21ad640 1 -- 172.21.15.120:0/1368568161 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dcc1a71f0 con 0x7f2dcc102570 2024-09-19T09:06:41.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.873+0000 7f2dc97fa640 1 -- 172.21.15.120:0/1368568161 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f2db4004940 con 0x7f2dcc102570 2024-09-19T09:06:41.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.874+0000 7f2dd4586740 1 Processor -- start 2024-09-19T09:06:41.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.874+0000 7f2dd4586740 1 -- start start 2024-09-19T09:06:41.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.874+0000 7f2dd4586740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aac85f4a0 0x557aac85f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:41.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.874+0000 7f2dd4586740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aac85fdc0 0x557aac86a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:41.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.874+0000 7f2dd4586740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aac86a680 0x557aac86ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:41.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.874+0000 7f2dd4586740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557aac7c52e0 con 0x557aac85f4a0 2024-09-19T09:06:41.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.874+0000 7f2dd4586740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557aac76cce0 con 0x557aac86a680 2024-09-19T09:06:41.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.874+0000 7f2dd4586740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557aac7c12a0 con 0x557aac85fdc0 2024-09-19T09:06:41.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.874+0000 7f2dcbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aac85f4a0 0x557aac85f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:41.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.874+0000 7f2dcbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aac85f4a0 0x557aac85f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37080/0 (socket says 172.21.15.120:37080) 2024-09-19T09:06:41.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.874+0000 7f2dcbfff640 1 -- 172.21.15.120:0/1182839693 learned_addr learned my addr 172.21.15.120:0/1182839693 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:41.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.874+0000 7f2dcb7fe640 1 --2- 172.21.15.120:0/1182839693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aac85fdc0 0x557aac86a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:41.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.874+0000 7f2dc3fff640 1 --2- 172.21.15.120:0/1182839693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aac86a680 0x557aac86ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:41.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.874+0000 7f2dcbfff640 1 -- 172.21.15.120:0/1182839693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aac85fdc0 msgr2=0x557aac86a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.874+0000 7f2dcbfff640 1 --2- 172.21.15.120:0/1182839693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aac85fdc0 0x557aac86a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.874+0000 7f2dcbfff640 1 -- 172.21.15.120:0/1182839693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aac86a680 msgr2=0x557aac86ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.874+0000 7f2dcbfff640 1 --2- 172.21.15.120:0/1182839693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aac86a680 0x557aac86ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.874+0000 7f2dcbfff640 1 -- 172.21.15.120:0/1182839693 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557aac751440 con 0x557aac85f4a0 2024-09-19T09:06:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.874+0000 7f2dc3fff640 1 --2- 172.21.15.120:0/1182839693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aac86a680 0x557aac86ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:06:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.874+0000 7f2dc97fa640 1 --2- 172.21.15.120:0/1368568161 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2db80a3240 0x7f2db80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.875+0000 7f2dcbfff640 1 --2- 172.21.15.120:0/1182839693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aac85f4a0 0x557aac85f880 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4009ca0 tx=0x7f2dc402f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137b33b8ad14b277 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.875+0000 7f2dcb7fe640 1 --2- 172.21.15.120:0/1368568161 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2db80a3240 0x7f2db80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.875+0000 7f2dc97fa640 1 -- 172.21.15.120:0/1368568161 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f2db40cf2c0 con 0x7f2dcc102570 2024-09-19T09:06:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.875+0000 7f2dc1ffb640 1 -- 172.21.15.120:0/1182839693 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4048070 con 0x557aac85f4a0 2024-09-19T09:06:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.875+0000 7f2dc1ffb640 1 -- 172.21.15.120:0/1182839693 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dc4003e50 con 0x557aac85f4a0 2024-09-19T09:06:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.875+0000 7f2dc1ffb640 1 -- 172.21.15.120:0/1182839693 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4034040 con 0x557aac85f4a0 2024-09-19T09:06:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dd4586740 1 -- 172.21.15.120:0/1182839693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aac85f4a0 msgr2=0x557aac85f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dcb7fe640 1 --2- 172.21.15.120:0/1368568161 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2db80a3240 0x7f2db80a5700 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x557aac85c890 tx=0x7f2dbc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dd4586740 1 --2- 172.21.15.120:0/1182839693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aac85f4a0 0x557aac85f880 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4009ca0 tx=0x7f2dc402f450 comp rx=0 tx=0).stop 2024-09-19T09:06:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dd4586740 1 -- 172.21.15.120:0/1182839693 shutdown_connections 2024-09-19T09:06:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dd4586740 1 --2- 172.21.15.120:0/1182839693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aac86a680 0x557aac86ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dd4586740 1 --2- 172.21.15.120:0/1182839693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aac85fdc0 0x557aac86a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dd4586740 1 --2- 172.21.15.120:0/1182839693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aac85f4a0 0x557aac85f880 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dd4586740 1 -- 172.21.15.120:0/1182839693 >> 172.21.15.120:0/1182839693 conn(0x557aac85ba30 msgr2=0x557aac85be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dd4586740 1 -- 172.21.15.120:0/1182839693 shutdown_connections 2024-09-19T09:06:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dd4586740 1 -- 172.21.15.120:0/1182839693 wait complete. 2024-09-19T09:06:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dd4586740 1 -- start start 2024-09-19T09:06:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dd4586740 1 Processor -- start 2024-09-19T09:06:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dd4586740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aac85f4a0 0x557aac869fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dd4586740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aac85fdc0 0x557aac867270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dd4586740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aac86a680 0x557aac8677b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dd4586740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557aac86d250 con 0x557aac85fdc0 2024-09-19T09:06:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dd4586740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557aac780050 con 0x557aac85f4a0 2024-09-19T09:06:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dd4586740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557aac86d3d0 con 0x557aac86a680 2024-09-19T09:06:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dd4586740 10 client.0 _ll_register_callbacks cb 0x557aac7c0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dc3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aac85fdc0 0x557aac867270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dcb7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aac85f4a0 0x557aac869fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dc3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aac85fdc0 0x557aac867270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37086/0 (socket says 172.21.15.120:37086) 2024-09-19T09:06:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dc3fff640 1 -- 172.21.15.120:0/3294006474 learned_addr learned my addr 172.21.15.120:0/3294006474 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dcbfff640 1 --2- 172.21.15.120:0/3294006474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aac86a680 0x557aac8677b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dcb7fe640 1 -- 172.21.15.120:0/3294006474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aac86a680 msgr2=0x557aac8677b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dcb7fe640 1 --2- 172.21.15.120:0/3294006474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aac86a680 0x557aac8677b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dcb7fe640 1 -- 172.21.15.120:0/3294006474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aac85fdc0 msgr2=0x557aac867270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:41.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dcb7fe640 1 --2- 172.21.15.120:0/3294006474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aac85fdc0 0x557aac867270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dcb7fe640 1 -- 172.21.15.120:0/3294006474 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557aac5363c0 con 0x557aac85f4a0 2024-09-19T09:06:41.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dc3fff640 1 --2- 172.21.15.120:0/3294006474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aac85fdc0 0x557aac867270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:41.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dcbfff640 1 --2- 172.21.15.120:0/3294006474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aac86a680 0x557aac8677b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:06:41.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dcb7fe640 1 --2- 172.21.15.120:0/3294006474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aac85f4a0 0x557aac869fc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc012cb0 tx=0x7f2dbc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b756628eb36a7396 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:41.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2daaffd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:06:41.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2daaffd640 1 -- 172.21.15.120:0/3294006474 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc009e60 con 0x557aac85f4a0 2024-09-19T09:06:41.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2daaffd640 1 -- 172.21.15.120:0/3294006474 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f2dbc005020 con 0x557aac85f4a0 2024-09-19T09:06:41.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.877+0000 7f2dd4586740 1 -- 172.21.15.120:0/3294006474 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dc4003e50 con 0x557aac85f4a0 2024-09-19T09:06:41.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.878+0000 7f2daaffd640 1 -- 172.21.15.120:0/3294006474 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2dbc005220 con 0x557aac85f4a0 2024-09-19T09:06:41.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.878+0000 7f2dd4586740 10 client.26132 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:41.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.878+0000 7f2dd4586740 1 -- 172.21.15.120:0/3294006474 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557aac878c60 con 0x557aac85f4a0 2024-09-19T09:06:41.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.878+0000 7f2daaffd640 1 -- 172.21.15.120:0/3294006474 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dc4003e50 con 0x557aac85f4a0 2024-09-19T09:06:41.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.878+0000 7f2daaffd640 1 -- 172.21.15.120:0/3294006474 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2dbc027070 con 0x557aac85f4a0 2024-09-19T09:06:41.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.878+0000 7f2dd4586740 10 client.26132 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:41.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.878+0000 7f2dd4586740 10 client.26132 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:06:41.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.878+0000 7f2dd4586740 20 client.26132 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:41.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.878+0000 7f2dd4586740 1 --2- 172.21.15.120:0/3294006474 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557aac87a070 0x557aac87c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:41.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.878+0000 7f2dd4586740 4 client.26132 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:06:41.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.878+0000 7f2dd4586740 1 -- 172.21.15.120:0/3294006474 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557aac7d2850 con 0x557aac87a070 2024-09-19T09:06:41.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.878+0000 7f2dc3fff640 1 --2- 172.21.15.120:0/3294006474 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557aac87a070 0x557aac87c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:41.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.879+0000 7f2daaffd640 10 client.26132 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:41.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.879+0000 7f2dc3fff640 1 --2- 172.21.15.120:0/3294006474 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557aac87a070 0x557aac87c510 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd8b04c2ada29d1b server_cookie=a4ae0f5cac2f3ac in_seq=0 out_seq=0 2024-09-19T09:06:41.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.879+0000 7f2daaffd640 1 -- 172.21.15.120:0/3294006474 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557aac7d2850 con 0x557aac87a070 2024-09-19T09:06:41.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.879+0000 7f2daaffd640 10 client.26132 handle_command_reply: tid=0 2024-09-19T09:06:41.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.879+0000 7f2dd4586740 1 client.26132 shutdown 2024-09-19T09:06:41.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.880+0000 7f2dd4586740 1 -- 172.21.15.120:0/3294006474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aac85f4a0 msgr2=0x557aac869fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:41.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.880+0000 7f2dd4586740 1 --2- 172.21.15.120:0/3294006474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aac85f4a0 0x557aac869fc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc012cb0 tx=0x7f2dbc004730 comp rx=0 tx=0).stop 2024-09-19T09:06:41.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.880+0000 7f2dd4586740 1 -- 172.21.15.120:0/3294006474 shutdown_connections 2024-09-19T09:06:41.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.880+0000 7f2dd4586740 1 --2- 172.21.15.120:0/3294006474 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557aac87a070 0x557aac87c510 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.880+0000 7f2dd4586740 1 --2- 172.21.15.120:0/3294006474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aac86a680 0x557aac8677b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.880+0000 7f2dd4586740 1 --2- 172.21.15.120:0/3294006474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aac85fdc0 0x557aac867270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.880+0000 7f2dd4586740 1 --2- 172.21.15.120:0/3294006474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aac85f4a0 0x557aac869fc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.880+0000 7f2daaffd640 0 client.26132 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:41.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.880+0000 7f2dd4586740 1 -- 172.21.15.120:0/3294006474 >> 172.21.15.120:0/3294006474 conn(0x557aac85ba30 msgr2=0x557aac861710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:41.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.880+0000 7f2dd4586740 1 -- 172.21.15.120:0/3294006474 shutdown_connections 2024-09-19T09:06:41.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.880+0000 7f2dd4586740 1 -- 172.21.15.120:0/3294006474 wait complete. 2024-09-19T09:06:41.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.880+0000 7f2dd4586740 20 client.26132 trim_cache size 0 max 16384 2024-09-19T09:06:41.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.899+0000 7f2dd4586740 1 Processor -- start 2024-09-19T09:06:41.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.899+0000 7f2dd4586740 1 -- start start 2024-09-19T09:06:41.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.899+0000 7f2dd4586740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aac85ba30 0x557aac85aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:41.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.899+0000 7f2dd4586740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aac85f4a0 0x557aac881e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:41.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.899+0000 7f2dd4586740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aac85fdc0 0x557aac882370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:41.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.899+0000 7f2dd4586740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2dcc1af700 con 0x557aac85f4a0 2024-09-19T09:06:41.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.899+0000 7f2dd4586740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2dcc1b1240 con 0x557aac85fdc0 2024-09-19T09:06:41.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.899+0000 7f2dd4586740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2dcc1afdd0 con 0x557aac85ba30 2024-09-19T09:06:41.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.899+0000 7f2dc3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aac85f4a0 0x557aac881e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:41.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.899+0000 7f2dcbfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aac85fdc0 0x557aac882370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.899+0000 7f2dc3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aac85f4a0 0x557aac881e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37092/0 (socket says 172.21.15.120:37092) 2024-09-19T09:06:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.899+0000 7f2dc3fff640 1 -- 172.21.15.120:0/1643287801 learned_addr learned my addr 172.21.15.120:0/1643287801 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.899+0000 7f2dcb7fe640 1 --2- 172.21.15.120:0/1643287801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aac85ba30 0x557aac85aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.900+0000 7f2dc3fff640 1 -- 172.21.15.120:0/1643287801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aac85ba30 msgr2=0x557aac85aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.900+0000 7f2dc3fff640 1 --2- 172.21.15.120:0/1643287801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aac85ba30 0x557aac85aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.900+0000 7f2dc3fff640 1 -- 172.21.15.120:0/1643287801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aac85fdc0 msgr2=0x557aac882370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.900+0000 7f2dc3fff640 1 --2- 172.21.15.120:0/1643287801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aac85fdc0 0x557aac882370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.900+0000 7f2dc3fff640 1 -- 172.21.15.120:0/1643287801 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557aac86a330 con 0x557aac85f4a0 2024-09-19T09:06:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.900+0000 7f2dcbfff640 1 --2- 172.21.15.120:0/1643287801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aac85fdc0 0x557aac882370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.900+0000 7f2dcb7fe640 1 --2- 172.21.15.120:0/1643287801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aac85ba30 0x557aac85aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.900+0000 7f2dc3fff640 1 --2- 172.21.15.120:0/1643287801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aac85f4a0 0x557aac881e30 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2db4009fd0 tx=0x7f2db40ccc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e05276290ced65 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.900+0000 7f2dc17fa640 1 -- 172.21.15.120:0/1643287801 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db409b240 con 0x557aac85f4a0 2024-09-19T09:06:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.900+0000 7f2dc17fa640 1 -- 172.21.15.120:0/1643287801 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2db409bfe0 con 0x557aac85f4a0 2024-09-19T09:06:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.900+0000 7f2dc17fa640 1 -- 172.21.15.120:0/1643287801 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db40cce40 con 0x557aac85f4a0 2024-09-19T09:06:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.901+0000 7f2dd4586740 1 -- 172.21.15.120:0/1643287801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aac85f4a0 msgr2=0x557aac881e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.901+0000 7f2dd4586740 1 --2- 172.21.15.120:0/1643287801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aac85f4a0 0x557aac881e30 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2db4009fd0 tx=0x7f2db40ccc90 comp rx=0 tx=0).stop 2024-09-19T09:06:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.901+0000 7f2dd4586740 1 -- 172.21.15.120:0/1643287801 shutdown_connections 2024-09-19T09:06:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.901+0000 7f2dd4586740 1 --2- 172.21.15.120:0/1643287801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aac85fdc0 0x557aac882370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.901+0000 7f2dd4586740 1 --2- 172.21.15.120:0/1643287801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aac85f4a0 0x557aac881e30 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.901+0000 7f2dd4586740 1 --2- 172.21.15.120:0/1643287801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aac85ba30 0x557aac85aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.901+0000 7f2dd4586740 1 -- 172.21.15.120:0/1643287801 >> 172.21.15.120:0/1643287801 conn(0x557aac867910 msgr2=0x557aac85a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.901+0000 7f2dd4586740 1 -- 172.21.15.120:0/1643287801 shutdown_connections 2024-09-19T09:06:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.901+0000 7f2dd4586740 1 -- 172.21.15.120:0/1643287801 wait complete. 2024-09-19T09:06:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.901+0000 7f2dd4586740 1 -- start start 2024-09-19T09:06:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.902+0000 7f2dd4586740 1 Processor -- start 2024-09-19T09:06:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.902+0000 7f2dd4586740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aac85ba30 0x557aac872ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.902+0000 7f2dd4586740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aac85f4a0 0x557aac86fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.902+0000 7f2dd4586740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aac85fdc0 0x557aac870370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.902+0000 7f2dcb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aac85ba30 0x557aac872ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.902+0000 7f2dcb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aac85ba30 0x557aac872ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37096/0 (socket says 172.21.15.120:37096) 2024-09-19T09:06:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.902+0000 7f2dcb7fe640 1 -- 172.21.15.120:0/47489671 learned_addr learned my addr 172.21.15.120:0/47489671 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.902+0000 7f2dcbfff640 1 --2- 172.21.15.120:0/47489671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aac85fdc0 0x557aac870370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.902+0000 7f2dc3fff640 1 --2- 172.21.15.120:0/47489671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aac85f4a0 0x557aac86fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.902+0000 7f2dd4586740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2dcc1b0b70 con 0x557aac85ba30 2024-09-19T09:06:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.902+0000 7f2dd4586740 1 -- 172.21.15.120:0/47489671 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2dcc1b04a0 con 0x557aac85fdc0 2024-09-19T09:06:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.902+0000 7f2dd4586740 1 -- 172.21.15.120:0/47489671 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2dcc1ad6c0 con 0x557aac85f4a0 2024-09-19T09:06:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.903+0000 7f2dd4586740 10 client.0 _ll_register_callbacks cb 0x7f2dbc009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.903+0000 7f2dcb7fe640 1 -- 172.21.15.120:0/47489671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aac85f4a0 msgr2=0x557aac86fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.903+0000 7f2dcb7fe640 1 --2- 172.21.15.120:0/47489671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aac85f4a0 0x557aac86fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.903+0000 7f2dcb7fe640 1 -- 172.21.15.120:0/47489671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aac85fdc0 msgr2=0x557aac870370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.903+0000 7f2dcb7fe640 1 --2- 172.21.15.120:0/47489671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aac85fdc0 0x557aac870370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.903+0000 7f2dcb7fe640 1 -- 172.21.15.120:0/47489671 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557aac8708e0 con 0x557aac85ba30 2024-09-19T09:06:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.903+0000 7f2dc3fff640 1 --2- 172.21.15.120:0/47489671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aac85f4a0 0x557aac86fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.903+0000 7f2dcb7fe640 1 --2- 172.21.15.120:0/47489671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aac85ba30 0x557aac872ae0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc012cb0 tx=0x7f2dbc004450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c33e69a7625dcc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.903+0000 7f2daaffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:06:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.903+0000 7f2daaffd640 1 -- 172.21.15.120:0/47489671 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc037030 con 0x557aac85ba30 2024-09-19T09:06:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.903+0000 7f2daaffd640 1 -- 172.21.15.120:0/47489671 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f2dbc018040 con 0x557aac85ba30 2024-09-19T09:06:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.903+0000 7f2dd4586740 1 -- 172.21.15.120:0/47489671 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2db409bfe0 con 0x557aac85ba30 2024-09-19T09:06:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.903+0000 7f2daaffd640 1 -- 172.21.15.120:0/47489671 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2dbc01ed20 con 0x557aac85ba30 2024-09-19T09:06:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.903+0000 7f2dd4586740 10 client.16797 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:41.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.903+0000 7f2dd4586740 1 -- 172.21.15.120:0/47489671 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557aac8e4b60 con 0x557aac85ba30 2024-09-19T09:06:41.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.904+0000 7f2daaffd640 1 -- 172.21.15.120:0/47489671 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2db409bfe0 con 0x557aac85ba30 2024-09-19T09:06:41.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.904+0000 7f2daaffd640 1 -- 172.21.15.120:0/47489671 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2dbc027070 con 0x557aac85ba30 2024-09-19T09:06:41.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.904+0000 7f2dd4586740 10 client.16797 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:41.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.904+0000 7f2dd4586740 10 client.16797 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:06:41.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.904+0000 7f2dd4586740 20 client.16797 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:41.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.904+0000 7f2dd4586740 1 --2- 172.21.15.120:0/47489671 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557aac86a680 0x557aac8e7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:41.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.904+0000 7f2dd4586740 4 client.16797 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:06:41.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.904+0000 7f2dd4586740 1 -- 172.21.15.120:0/47489671 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557aac7ccd60 con 0x557aac86a680 2024-09-19T09:06:41.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.904+0000 7f2dc3fff640 1 --2- 172.21.15.120:0/47489671 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557aac86a680 0x557aac8e7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:41.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.904+0000 7f2daaffd640 10 client.16797 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:41.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.904+0000 7f2dc3fff640 1 --2- 172.21.15.120:0/47489671 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557aac86a680 0x557aac8e7370 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b83a2bf38551190c server_cookie=603ee62f01fa418e in_seq=0 out_seq=0 2024-09-19T09:06:41.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.905+0000 7f2daaffd640 1 -- 172.21.15.120:0/47489671 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557aac7ccd60 con 0x557aac86a680 2024-09-19T09:06:41.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.905+0000 7f2daaffd640 10 client.16797 handle_command_reply: tid=0 2024-09-19T09:06:41.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.905+0000 7f2dd4586740 1 client.16797 shutdown 2024-09-19T09:06:41.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.905+0000 7f2dd4586740 1 -- 172.21.15.120:0/47489671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aac85ba30 msgr2=0x557aac872ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:41.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.905+0000 7f2dd4586740 1 --2- 172.21.15.120:0/47489671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aac85ba30 0x557aac872ae0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc012cb0 tx=0x7f2dbc004450 comp rx=0 tx=0).stop 2024-09-19T09:06:41.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.905+0000 7f2dd4586740 1 -- 172.21.15.120:0/47489671 shutdown_connections 2024-09-19T09:06:41.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.905+0000 7f2dd4586740 1 --2- 172.21.15.120:0/47489671 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557aac86a680 0x557aac8e7370 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.906+0000 7f2dd4586740 1 --2- 172.21.15.120:0/47489671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aac85fdc0 0x557aac870370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.906+0000 7f2dd4586740 1 --2- 172.21.15.120:0/47489671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aac85f4a0 0x557aac86fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.906+0000 7f2dd4586740 1 --2- 172.21.15.120:0/47489671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aac85ba30 0x557aac872ae0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.906+0000 7f2daaffd640 0 client.16797 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:41.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.906+0000 7f2dd4586740 1 -- 172.21.15.120:0/47489671 >> 172.21.15.120:0/47489671 conn(0x557aac867910 msgr2=0x557aac85a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:41.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.906+0000 7f2dd4586740 1 -- 172.21.15.120:0/47489671 shutdown_connections 2024-09-19T09:06:41.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.906+0000 7f2dd4586740 1 -- 172.21.15.120:0/47489671 wait complete. 2024-09-19T09:06:41.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.906+0000 7f2dd4586740 20 client.16797 trim_cache size 0 max 16384 2024-09-19T09:06:41.918 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:06:41.918 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:06:41.918 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:06:41.918 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:06:41.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.907+0000 7f2dd21ad640 1 -- 172.21.15.120:0/1368568161 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2db80a3240 msgr2=0x7f2db80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:41.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.907+0000 7f2dd21ad640 1 --2- 172.21.15.120:0/1368568161 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2db80a3240 0x7f2db80a5700 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x557aac85c890 tx=0x7f2dbc005f90 comp rx=0 tx=0).stop 2024-09-19T09:06:41.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.907+0000 7f2dd21ad640 1 -- 172.21.15.120:0/1368568161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dcc102570 msgr2=0x7f2dcc19abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:41.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.907+0000 7f2dd21ad640 1 --2- 172.21.15.120:0/1368568161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dcc102570 0x7f2dcc19abb0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2db400d950 tx=0x7f2db400de10 comp rx=0 tx=0).stop 2024-09-19T09:06:41.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.907+0000 7f2dd21ad640 1 -- 172.21.15.120:0/1368568161 shutdown_connections 2024-09-19T09:06:41.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.907+0000 7f2dd21ad640 1 --2- 172.21.15.120:0/1368568161 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2db80a3240 0x7f2db80a5700 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.907+0000 7f2dd21ad640 1 --2- 172.21.15.120:0/1368568161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dcc102ef0 0x7f2dcc19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.907+0000 7f2dd21ad640 1 --2- 172.21.15.120:0/1368568161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dcc102570 0x7f2dcc19abb0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.907+0000 7f2dd21ad640 1 --2- 172.21.15.120:0/1368568161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2dcc100e20 0x7f2dcc19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:41.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.907+0000 7f2dd21ad640 1 -- 172.21.15.120:0/1368568161 >> 172.21.15.120:0/1368568161 conn(0x7f2dcc0fc7d0 msgr2=0x7f2dcc06a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:41.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.907+0000 7f2dd21ad640 1 -- 172.21.15.120:0/1368568161 shutdown_connections 2024-09-19T09:06:41.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:41.907+0000 7f2dd21ad640 1 -- 172.21.15.120:0/1368568161 wait complete. 2024-09-19T09:06:42.043 INFO:tasks.workunit.client.0.smithi120.stdout: 1 284926 8.33 MB/sec execute 37 sec latency 28.655 ms 2024-09-19T09:06:42.713 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:06:42.713 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:06:42.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:42 smithi120 ceph-mon[31150]: pgmap v456: 97 pgs: 97 active+clean; 34 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 240 op/s 2024-09-19T09:06:42.970 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:06:43.043 INFO:tasks.workunit.client.0.smithi120.stdout: 1 287952 8.56 MB/sec execute 38 sec latency 14.318 ms 2024-09-19T09:06:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:42 smithi196 ceph-mon[35150]: pgmap v456: 97 pgs: 97 active+clean; 34 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 240 op/s 2024-09-19T09:06:43.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:42 smithi153 ceph-mon[35071]: pgmap v456: 97 pgs: 97 active+clean; 34 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 240 op/s 2024-09-19T09:06:43.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.493+0000 7f162c768640 1 -- 172.21.15.120:0/1717362134 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16241027c0 msgr2=0x7f1624106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:43.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.493+0000 7f162c768640 1 --2- 172.21.15.120:0/1717362134 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16241027c0 0x7f1624106810 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1614009900 tx=0x7f161402a410 comp rx=0 tx=0).stop 2024-09-19T09:06:43.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.493+0000 7f162c768640 1 -- 172.21.15.120:0/1717362134 shutdown_connections 2024-09-19T09:06:43.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.493+0000 7f162c768640 1 --2- 172.21.15.120:0/1717362134 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1624106f40 0x7f162410eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:43.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.493+0000 7f162c768640 1 --2- 172.21.15.120:0/1717362134 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16241027c0 0x7f1624106810 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:43.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.493+0000 7f162c768640 1 --2- 172.21.15.120:0/1717362134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1624101e10 0x7f16241021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:43.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.493+0000 7f162c768640 1 -- 172.21.15.120:0/1717362134 >> 172.21.15.120:0/1717362134 conn(0x7f16240fd8d0 msgr2=0x7f16240ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:43.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.493+0000 7f162c768640 1 -- 172.21.15.120:0/1717362134 shutdown_connections 2024-09-19T09:06:43.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.493+0000 7f162c768640 1 -- 172.21.15.120:0/1717362134 wait complete. 2024-09-19T09:06:43.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.493+0000 7f162c768640 1 Processor -- start 2024-09-19T09:06:43.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.493+0000 7f162c768640 1 -- start start 2024-09-19T09:06:43.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.493+0000 7f162c768640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1624101e10 0x7f16241984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:43.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.493+0000 7f162c768640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16241027c0 0x7f1624198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:43.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.494+0000 7f162c768640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1624106f40 0x7f162419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:43.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.494+0000 7f162c768640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1624111fc0 con 0x7f1624106f40 2024-09-19T09:06:43.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.494+0000 7f162c768640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1624111e40 con 0x7f1624101e10 2024-09-19T09:06:43.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.494+0000 7f162c768640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1624112140 con 0x7f16241027c0 2024-09-19T09:06:43.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.494+0000 7f162a504640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1624101e10 0x7f16241984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:43.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.494+0000 7f162a504640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1624101e10 0x7f16241984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:45502/0 (socket says 172.21.15.120:45502) 2024-09-19T09:06:43.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.494+0000 7f162a504640 1 -- 172.21.15.120:0/17491939 learned_addr learned my addr 172.21.15.120:0/17491939 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:43.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.494+0000 7f1629d03640 1 --2- 172.21.15.120:0/17491939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16241027c0 0x7f1624198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:43.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.494+0000 7f162a504640 1 -- 172.21.15.120:0/17491939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16241027c0 msgr2=0x7f1624198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:43.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.494+0000 7f162a504640 1 --2- 172.21.15.120:0/17491939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16241027c0 0x7f1624198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:43.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.494+0000 7f162a504640 1 -- 172.21.15.120:0/17491939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1624106f40 msgr2=0x7f162419cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:43.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.494+0000 7f162a504640 1 --2- 172.21.15.120:0/17491939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1624106f40 0x7f162419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:43.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.494+0000 7f162a504640 1 -- 172.21.15.120:0/17491939 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162419d490 con 0x7f1624101e10 2024-09-19T09:06:43.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.494+0000 7f162a504640 1 --2- 172.21.15.120:0/17491939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1624101e10 0x7f16241984e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f162000b4a0 tx=0x7f162000b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:43.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.494+0000 7f161b7fe640 1 -- 172.21.15.120:0/17491939 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1620014070 con 0x7f1624101e10 2024-09-19T09:06:43.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.494+0000 7f162c768640 1 -- 172.21.15.120:0/17491939 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f162419d780 con 0x7f1624101e10 2024-09-19T09:06:43.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.494+0000 7f161b7fe640 1 -- 172.21.15.120:0/17491939 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1620010af0 con 0x7f1624101e10 2024-09-19T09:06:43.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.494+0000 7f161b7fe640 1 -- 172.21.15.120:0/17491939 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1620017d10 con 0x7f1624101e10 2024-09-19T09:06:43.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.494+0000 7f162c768640 1 -- 172.21.15.120:0/17491939 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16241a5060 con 0x7f1624101e10 2024-09-19T09:06:43.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.495+0000 7f162c768640 1 -- 172.21.15.120:0/17491939 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16241021f0 con 0x7f1624101e10 2024-09-19T09:06:43.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.497+0000 7f161b7fe640 1 -- 172.21.15.120:0/17491939 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f1620017070 con 0x7f1624101e10 2024-09-19T09:06:43.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.497+0000 7f161b7fe640 1 --2- 172.21.15.120:0/17491939 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16100a3240 0x7f16100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:43.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.497+0000 7f1629d03640 1 --2- 172.21.15.120:0/17491939 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16100a3240 0x7f16100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:43.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.497+0000 7f161b7fe640 1 -- 172.21.15.120:0/17491939 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f16200cee90 con 0x7f1624101e10 2024-09-19T09:06:43.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.498+0000 7f1629d03640 1 --2- 172.21.15.120:0/17491939 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16100a3240 0x7f16100a5700 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1614009230 tx=0x7f16140058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:43.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.501+0000 7f161b7fe640 1 -- 172.21.15.120:0/17491939 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f162009bb70 con 0x7f1624101e10 2024-09-19T09:06:43.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.717+0000 7f162c768640 1 -- 172.21.15.120:0/17491939 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f162402d7c0 con 0x7f1624101e10 2024-09-19T09:06:43.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.718+0000 7f161b7fe640 1 -- 172.21.15.120:0/17491939 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f162009b470 con 0x7f1624101e10 2024-09-19T09:06:43.718 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:06:43.718 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":28,"btime":"2024-09-19T09:02:41:696728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:02:41.207357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:06:43.719 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 28 2024-09-19T09:06:43.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.720+0000 7f162c768640 1 -- 172.21.15.120:0/17491939 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16100a3240 msgr2=0x7f16100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:43.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.720+0000 7f162c768640 1 --2- 172.21.15.120:0/17491939 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16100a3240 0x7f16100a5700 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1614009230 tx=0x7f16140058d0 comp rx=0 tx=0).stop 2024-09-19T09:06:43.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.720+0000 7f162c768640 1 -- 172.21.15.120:0/17491939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1624101e10 msgr2=0x7f16241984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:43.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.720+0000 7f162c768640 1 --2- 172.21.15.120:0/17491939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1624101e10 0x7f16241984e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f162000b4a0 tx=0x7f162000b960 comp rx=0 tx=0).stop 2024-09-19T09:06:43.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.720+0000 7f162c768640 1 -- 172.21.15.120:0/17491939 shutdown_connections 2024-09-19T09:06:43.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.720+0000 7f162c768640 1 --2- 172.21.15.120:0/17491939 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16100a3240 0x7f16100a5700 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:43.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.720+0000 7f162c768640 1 --2- 172.21.15.120:0/17491939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1624106f40 0x7f162419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:43.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.720+0000 7f162c768640 1 --2- 172.21.15.120:0/17491939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16241027c0 0x7f1624198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:43.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.720+0000 7f162c768640 1 --2- 172.21.15.120:0/17491939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1624101e10 0x7f16241984e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:43.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.720+0000 7f162c768640 1 -- 172.21.15.120:0/17491939 >> 172.21.15.120:0/17491939 conn(0x7f16240fd8d0 msgr2=0x7f16240fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:43.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.720+0000 7f162c768640 1 -- 172.21.15.120:0/17491939 shutdown_connections 2024-09-19T09:06:43.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:43.720+0000 7f162c768640 1 -- 172.21.15.120:0/17491939 wait complete. 2024-09-19T09:06:44.043 INFO:tasks.workunit.client.0.smithi120.stdout: 1 291183 8.82 MB/sec execute 39 sec latency 18.343 ms 2024-09-19T09:06:44.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:43 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/17491939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:06:44.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:43 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/17491939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:06:44.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:43 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/17491939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:06:44.566 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:06:44.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:44 smithi120 ceph-mon[31150]: pgmap v457: 97 pgs: 97 active+clean; 38 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 310 op/s 2024-09-19T09:06:44.829 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:06:45.043 INFO:tasks.workunit.client.0.smithi120.stdout: 1 293884 8.93 MB/sec execute 40 sec latency 62.108 ms 2024-09-19T09:06:45.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:44 smithi196 ceph-mon[35150]: pgmap v457: 97 pgs: 97 active+clean; 38 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 310 op/s 2024-09-19T09:06:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:44 smithi153 ceph-mon[35071]: pgmap v457: 97 pgs: 97 active+clean; 38 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 310 op/s 2024-09-19T09:06:45.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.336+0000 7f9e742c8640 1 -- 172.21.15.120:0/2215336037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e6c102570 msgr2=0x7f9e6c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:45.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.336+0000 7f9e742c8640 1 --2- 172.21.15.120:0/2215336037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e6c102570 0x7f9e6c1029b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c009900 tx=0x7f9e5c02a480 comp rx=0 tx=0).stop 2024-09-19T09:06:45.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.336+0000 7f9e742c8640 1 -- 172.21.15.120:0/2215336037 shutdown_connections 2024-09-19T09:06:45.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.336+0000 7f9e742c8640 1 --2- 172.21.15.120:0/2215336037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e6c102ef0 0x7f9e6c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.336+0000 7f9e742c8640 1 --2- 172.21.15.120:0/2215336037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e6c102570 0x7f9e6c1029b0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.336+0000 7f9e742c8640 1 --2- 172.21.15.120:0/2215336037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e6c100e20 0x7f9e6c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.336+0000 7f9e742c8640 1 -- 172.21.15.120:0/2215336037 >> 172.21.15.120:0/2215336037 conn(0x7f9e6c0fc7d0 msgr2=0x7f9e6c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:45.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.336+0000 7f9e742c8640 1 -- 172.21.15.120:0/2215336037 shutdown_connections 2024-09-19T09:06:45.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.336+0000 7f9e742c8640 1 -- 172.21.15.120:0/2215336037 wait complete. 2024-09-19T09:06:45.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.337+0000 7f9e742c8640 1 Processor -- start 2024-09-19T09:06:45.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.337+0000 7f9e742c8640 1 -- start start 2024-09-19T09:06:45.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.337+0000 7f9e742c8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e6c100e20 0x7f9e6c1973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:45.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.337+0000 7f9e742c8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e6c102570 0x7f9e6c1978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:45.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.337+0000 7f9e742c8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e6c102ef0 0x7f9e6c19bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:45.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.337+0000 7f9e742c8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c110f80 con 0x7f9e6c100e20 2024-09-19T09:06:45.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.337+0000 7f9e742c8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c110e00 con 0x7f9e6c102570 2024-09-19T09:06:45.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.337+0000 7f9e742c8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c111100 con 0x7f9e6c102ef0 2024-09-19T09:06:45.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.337+0000 7f9e72064640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e6c100e20 0x7f9e6c1973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:45.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.338+0000 7f9e72064640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e6c100e20 0x7f9e6c1973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37130/0 (socket says 172.21.15.120:37130) 2024-09-19T09:06:45.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.338+0000 7f9e71863640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e6c102570 0x7f9e6c1978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:45.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.338+0000 7f9e72064640 1 -- 172.21.15.120:0/1494636832 learned_addr learned my addr 172.21.15.120:0/1494636832 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:45.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.338+0000 7f9e72865640 1 --2- 172.21.15.120:0/1494636832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e6c102ef0 0x7f9e6c19bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:45.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.338+0000 7f9e72064640 1 -- 172.21.15.120:0/1494636832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e6c102ef0 msgr2=0x7f9e6c19bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:45.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.338+0000 7f9e72064640 1 --2- 172.21.15.120:0/1494636832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e6c102ef0 0x7f9e6c19bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.338+0000 7f9e72064640 1 -- 172.21.15.120:0/1494636832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e6c102570 msgr2=0x7f9e6c1978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:45.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.338+0000 7f9e72064640 1 --2- 172.21.15.120:0/1494636832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e6c102570 0x7f9e6c1978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.338+0000 7f9e72064640 1 -- 172.21.15.120:0/1494636832 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e6c19c350 con 0x7f9e6c100e20 2024-09-19T09:06:45.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.338+0000 7f9e71863640 1 --2- 172.21.15.120:0/1494636832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e6c102570 0x7f9e6c1978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:45.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.338+0000 7f9e72064640 1 --2- 172.21.15.120:0/1494636832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e6c100e20 0x7f9e6c1973a0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800eb00 tx=0x7f9e68009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:45.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.338+0000 7f9e637fe640 1 -- 172.21.15.120:0/1494636832 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6801c070 con 0x7f9e6c100e20 2024-09-19T09:06:45.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.338+0000 7f9e742c8640 1 -- 172.21.15.120:0/1494636832 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e6c19c640 con 0x7f9e6c100e20 2024-09-19T09:06:45.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.338+0000 7f9e637fe640 1 -- 172.21.15.120:0/1494636832 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e6800bc30 con 0x7f9e6c100e20 2024-09-19T09:06:45.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.338+0000 7f9e637fe640 1 -- 172.21.15.120:0/1494636832 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e680166a0 con 0x7f9e6c100e20 2024-09-19T09:06:45.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.338+0000 7f9e742c8640 1 -- 172.21.15.120:0/1494636832 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e6c069f70 con 0x7f9e6c100e20 2024-09-19T09:06:45.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.341+0000 7f9e637fe640 1 -- 172.21.15.120:0/1494636832 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f9e6801f070 con 0x7f9e6c100e20 2024-09-19T09:06:45.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.341+0000 7f9e766a1740 1 Processor -- start 2024-09-19T09:06:45.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.341+0000 7f9e766a1740 1 -- start start 2024-09-19T09:06:45.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.341+0000 7f9e766a1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b615a5b40 0x560b615a4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:45.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.341+0000 7f9e766a1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b615a45d0 0x560b615a4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:45.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.341+0000 7f9e766a1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b615af370 0x560b615b1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:45.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.341+0000 7f9e766a1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560b6150a2f0 con 0x560b615a45d0 2024-09-19T09:06:45.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.341+0000 7f9e766a1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560b614b1cf0 con 0x560b615af370 2024-09-19T09:06:45.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.341+0000 7f9e766a1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560b615062b0 con 0x560b615a5b40 2024-09-19T09:06:45.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.341+0000 7f9e637fe640 1 --2- 172.21.15.120:0/1494636832 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9e580a3240 0x7f9e580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:45.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.342+0000 7f9e72865640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b615a5b40 0x560b615a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:45.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.342+0000 7f9e72064640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b615a45d0 0x560b615a4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:45.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.342+0000 7f9e72865640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b615a5b40 0x560b615a4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:45418/0 (socket says 172.21.15.120:45418) 2024-09-19T09:06:45.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.342+0000 7f9e72865640 1 -- 172.21.15.120:0/1687559651 learned_addr learned my addr 172.21.15.120:0/1687559651 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:45.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.342+0000 7f9e72865640 1 --2- 172.21.15.120:0/1494636832 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9e580a3240 0x7f9e580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:45.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.342+0000 7f9e71863640 1 --2- 172.21.15.120:0/1687559651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b615af370 0x560b615b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:45.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.342+0000 7f9e637fe640 1 -- 172.21.15.120:0/1494636832 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f9e680d0ec0 con 0x7f9e6c100e20 2024-09-19T09:06:45.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.342+0000 7f9e72865640 1 -- 172.21.15.120:0/1687559651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b615af370 msgr2=0x560b615b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:45.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.342+0000 7f9e72865640 1 --2- 172.21.15.120:0/1687559651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b615af370 0x560b615b1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.342+0000 7f9e72865640 1 -- 172.21.15.120:0/1687559651 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b615a45d0 msgr2=0x560b615a4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:45.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.342+0000 7f9e72865640 1 --2- 172.21.15.120:0/1687559651 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b615a45d0 0x560b615a4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.342+0000 7f9e72865640 1 -- 172.21.15.120:0/1687559651 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b614963a0 con 0x560b615a5b40 2024-09-19T09:06:45.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.342+0000 7f9e72865640 1 --2- 172.21.15.120:0/1687559651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b615a5b40 0x560b615a4090 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9e64010570 tx=0x7f9e64016f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=747bb4f6f95e7420 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:45.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.342+0000 7f9e72865640 1 --2- 172.21.15.120:0/1494636832 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9e580a3240 0x7f9e580a5700 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9e64010ec0 tx=0x7f9e6401b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:45.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.343+0000 7f9e60dea640 1 -- 172.21.15.120:0/1687559651 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e640169b0 con 0x560b615a5b40 2024-09-19T09:06:45.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.343+0000 7f9e60dea640 1 -- 172.21.15.120:0/1687559651 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e64009da0 con 0x560b615a5b40 2024-09-19T09:06:45.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.343+0000 7f9e60dea640 1 -- 172.21.15.120:0/1687559651 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e64005790 con 0x560b615a5b40 2024-09-19T09:06:45.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.343+0000 7f9e766a1740 1 -- 172.21.15.120:0/1687559651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b615a5b40 msgr2=0x560b615a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:45.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.343+0000 7f9e766a1740 1 --2- 172.21.15.120:0/1687559651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b615a5b40 0x560b615a4090 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9e64010570 tx=0x7f9e64016f90 comp rx=0 tx=0).stop 2024-09-19T09:06:45.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.343+0000 7f9e766a1740 1 -- 172.21.15.120:0/1687559651 shutdown_connections 2024-09-19T09:06:45.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.343+0000 7f9e766a1740 1 --2- 172.21.15.120:0/1687559651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b615af370 0x560b615b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.343+0000 7f9e766a1740 1 --2- 172.21.15.120:0/1687559651 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b615a45d0 0x560b615a4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.343+0000 7f9e766a1740 1 --2- 172.21.15.120:0/1687559651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b615a5b40 0x560b615a4090 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.343+0000 7f9e766a1740 1 -- 172.21.15.120:0/1687559651 >> 172.21.15.120:0/1687559651 conn(0x560b6156d310 msgr2=0x560b615a0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:45.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.343+0000 7f9e766a1740 1 -- 172.21.15.120:0/1687559651 shutdown_connections 2024-09-19T09:06:45.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.343+0000 7f9e766a1740 1 -- 172.21.15.120:0/1687559651 wait complete. 2024-09-19T09:06:45.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.344+0000 7f9e766a1740 1 -- start start 2024-09-19T09:06:45.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.344+0000 7f9e766a1740 1 Processor -- start 2024-09-19T09:06:45.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.344+0000 7f9e766a1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b615a45d0 0x560b615aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:45.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.344+0000 7f9e766a1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b615a5b40 0x560b615ac080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:45.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.344+0000 7f9e766a1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b615af370 0x560b615ac810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:45.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.344+0000 7f9e766a1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560b615b2010 con 0x560b615af370 2024-09-19T09:06:45.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.344+0000 7f9e766a1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560b614c5060 con 0x560b615a45d0 2024-09-19T09:06:45.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.344+0000 7f9e766a1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560b615b2190 con 0x560b615a5b40 2024-09-19T09:06:45.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.344+0000 7f9e766a1740 10 client.0 _ll_register_callbacks cb 0x560b612188c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:45.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.344+0000 7f9e72064640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b615a45d0 0x560b615aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:45.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.344+0000 7f9e71863640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b615a5b40 0x560b615ac080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:45.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.344+0000 7f9e72064640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b615a45d0 0x560b615aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:45532/0 (socket says 172.21.15.120:45532) 2024-09-19T09:06:45.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.344+0000 7f9e72064640 1 -- 172.21.15.120:0/1037477263 learned_addr learned my addr 172.21.15.120:0/1037477263 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:45.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.344+0000 7f9e72865640 1 --2- 172.21.15.120:0/1037477263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b615af370 0x560b615ac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:45.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.344+0000 7f9e71863640 1 -- 172.21.15.120:0/1037477263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b615a45d0 msgr2=0x560b615aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:45.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.345+0000 7f9e71863640 1 --2- 172.21.15.120:0/1037477263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b615a45d0 0x560b615aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.345+0000 7f9e71863640 1 -- 172.21.15.120:0/1037477263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b615af370 msgr2=0x560b615ac810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:45.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.345+0000 7f9e71863640 1 --2- 172.21.15.120:0/1037477263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b615af370 0x560b615ac810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.345+0000 7f9e71863640 1 -- 172.21.15.120:0/1037477263 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b6127b3c0 con 0x560b615a5b40 2024-09-19T09:06:45.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.345+0000 7f9e72865640 1 --2- 172.21.15.120:0/1037477263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b615af370 0x560b615ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:45.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.345+0000 7f9e72064640 1 --2- 172.21.15.120:0/1037477263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b615a45d0 0x560b615aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:45.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.345+0000 7f9e55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:06:45.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.345+0000 7f9e71863640 1 --2- 172.21.15.120:0/1037477263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b615a5b40 0x560b615ac080 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c005940 tx=0x7f9e5c003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fccf57932dd9ae6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:45.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.345+0000 7f9e55ffb640 1 -- 172.21.15.120:0/1037477263 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c035070 con 0x560b615a5b40 2024-09-19T09:06:45.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.345+0000 7f9e766a1740 1 -- 172.21.15.120:0/1037477263 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560b61517860 con 0x560b615a5b40 2024-09-19T09:06:45.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.345+0000 7f9e55ffb640 1 -- 172.21.15.120:0/1037477263 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f9e5c003ba0 con 0x560b615a5b40 2024-09-19T09:06:45.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.345+0000 7f9e55ffb640 1 -- 172.21.15.120:0/1037477263 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9e5c003da0 con 0x560b615a5b40 2024-09-19T09:06:45.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.345+0000 7f9e766a1740 10 client.26101 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:45.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.345+0000 7f9e766a1740 1 -- 172.21.15.120:0/1037477263 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560b615bdb80 con 0x560b615a5b40 2024-09-19T09:06:45.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.345+0000 7f9e55ffb640 1 -- 172.21.15.120:0/1037477263 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560b61517860 con 0x560b615a5b40 2024-09-19T09:06:45.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.345+0000 7f9e55ffb640 1 -- 172.21.15.120:0/1037477263 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9e5c031540 con 0x560b615a5b40 2024-09-19T09:06:45.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.345+0000 7f9e766a1740 10 client.26101 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:45.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.345+0000 7f9e766a1740 10 client.26101 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:06:45.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.346+0000 7f9e766a1740 20 client.26101 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:45.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.346+0000 7f9e766a1740 1 --2- 172.21.15.120:0/1037477263 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560b615bef90 0x560b615c13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:45.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.346+0000 7f9e766a1740 4 client.26101 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:06:45.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.346+0000 7f9e766a1740 1 -- 172.21.15.120:0/1037477263 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b61511d70 con 0x560b615bef90 2024-09-19T09:06:45.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.346+0000 7f9e72064640 1 --2- 172.21.15.120:0/1037477263 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560b615bef90 0x560b615c13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:45.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.346+0000 7f9e72064640 1 --2- 172.21.15.120:0/1037477263 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560b615bef90 0x560b615c13a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=85fabb96887a91fa server_cookie=95cafb292b353bb1 in_seq=0 out_seq=0 2024-09-19T09:06:45.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.346+0000 7f9e55ffb640 10 client.26101 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:06:45.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.347+0000 7f9e55ffb640 1 -- 172.21.15.120:0/1037477263 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b61511d70 con 0x560b615bef90 2024-09-19T09:06:45.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.347+0000 7f9e55ffb640 10 client.26101 handle_command_reply: tid=0 2024-09-19T09:06:45.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.347+0000 7f9e766a1740 1 client.26101 shutdown 2024-09-19T09:06:45.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.347+0000 7f9e766a1740 1 -- 172.21.15.120:0/1037477263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b615a5b40 msgr2=0x560b615ac080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:45.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.347+0000 7f9e766a1740 1 --2- 172.21.15.120:0/1037477263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b615a5b40 0x560b615ac080 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c005940 tx=0x7f9e5c003820 comp rx=0 tx=0).stop 2024-09-19T09:06:45.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.347+0000 7f9e766a1740 1 -- 172.21.15.120:0/1037477263 shutdown_connections 2024-09-19T09:06:45.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.347+0000 7f9e766a1740 1 --2- 172.21.15.120:0/1037477263 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560b615bef90 0x560b615c13a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.347+0000 7f9e766a1740 1 --2- 172.21.15.120:0/1037477263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b615af370 0x560b615ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.347+0000 7f9e766a1740 1 --2- 172.21.15.120:0/1037477263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b615a5b40 0x560b615ac080 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.347+0000 7f9e55ffb640 0 client.26101 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:06:45.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.347+0000 7f9e766a1740 1 --2- 172.21.15.120:0/1037477263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b615a45d0 0x560b615aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.347+0000 7f9e766a1740 1 -- 172.21.15.120:0/1037477263 >> 172.21.15.120:0/1037477263 conn(0x560b6156d310 msgr2=0x560b615a6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:45.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.347+0000 7f9e766a1740 1 -- 172.21.15.120:0/1037477263 shutdown_connections 2024-09-19T09:06:45.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.347+0000 7f9e766a1740 1 -- 172.21.15.120:0/1037477263 wait complete. 2024-09-19T09:06:45.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.347+0000 7f9e766a1740 20 client.26101 trim_cache size 0 max 16384 2024-09-19T09:06:45.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.361+0000 7f9e766a1740 1 Processor -- start 2024-09-19T09:06:45.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.361+0000 7f9e766a1740 1 -- start start 2024-09-19T09:06:45.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.361+0000 7f9e766a1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b615a45d0 0x560b615a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:45.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.361+0000 7f9e766a1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b615a5b40 0x560b615ab060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:45.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.361+0000 7f9e766a1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b615af370 0x560b615c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:45.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.361+0000 7f9e766a1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c1ab2f0 con 0x560b615af370 2024-09-19T09:06:45.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.361+0000 7f9e766a1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c1ace30 con 0x560b615a45d0 2024-09-19T09:06:45.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.361+0000 7f9e766a1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c1ab9c0 con 0x560b615a5b40 2024-09-19T09:06:45.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.361+0000 7f9e72865640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b615af370 0x560b615c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:45.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.361+0000 7f9e72064640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b615a45d0 0x560b615a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:45.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.361+0000 7f9e72865640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b615af370 0x560b615c6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37166/0 (socket says 172.21.15.120:37166) 2024-09-19T09:06:45.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.361+0000 7f9e72865640 1 -- 172.21.15.120:0/3203065040 learned_addr learned my addr 172.21.15.120:0/3203065040 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:45.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.361+0000 7f9e72064640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b615a45d0 0x560b615a3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:45534/0 (socket says 172.21.15.120:45534) 2024-09-19T09:06:45.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.361+0000 7f9e71863640 1 --2- 172.21.15.120:0/3203065040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b615a5b40 0x560b615ab060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:45.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.362+0000 7f9e72865640 1 -- 172.21.15.120:0/3203065040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b615a5b40 msgr2=0x560b615ab060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:45.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.362+0000 7f9e72865640 1 --2- 172.21.15.120:0/3203065040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b615a5b40 0x560b615ab060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.362+0000 7f9e72865640 1 -- 172.21.15.120:0/3203065040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b615a45d0 msgr2=0x560b615a3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:45.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.362+0000 7f9e72865640 1 --2- 172.21.15.120:0/3203065040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b615a45d0 0x560b615a3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.362+0000 7f9e72865640 1 -- 172.21.15.120:0/3203065040 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b615a0510 con 0x560b615af370 2024-09-19T09:06:45.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.362+0000 7f9e72064640 1 --2- 172.21.15.120:0/3203065040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b615a45d0 0x560b615a3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:45.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.362+0000 7f9e72865640 1 --2- 172.21.15.120:0/3203065040 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b615af370 0x560b615c6c70 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f9e64000f80 tx=0x7f9e64005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58848389659ec3a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:45.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.362+0000 7f9e57fff640 1 -- 172.21.15.120:0/3203065040 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e64032070 con 0x560b615af370 2024-09-19T09:06:45.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.362+0000 7f9e57fff640 1 -- 172.21.15.120:0/3203065040 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e640169b0 con 0x560b615af370 2024-09-19T09:06:45.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.362+0000 7f9e57fff640 1 -- 172.21.15.120:0/3203065040 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e64015d20 con 0x560b615af370 2024-09-19T09:06:45.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.363+0000 7f9e766a1740 1 -- 172.21.15.120:0/3203065040 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b615af370 msgr2=0x560b615c6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:45.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.363+0000 7f9e766a1740 1 --2- 172.21.15.120:0/3203065040 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b615af370 0x560b615c6c70 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f9e64000f80 tx=0x7f9e64005f80 comp rx=0 tx=0).stop 2024-09-19T09:06:45.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.363+0000 7f9e766a1740 1 -- 172.21.15.120:0/3203065040 shutdown_connections 2024-09-19T09:06:45.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.363+0000 7f9e766a1740 1 --2- 172.21.15.120:0/3203065040 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b615af370 0x560b615c6c70 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.363+0000 7f9e766a1740 1 --2- 172.21.15.120:0/3203065040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b615a5b40 0x560b615ab060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.363+0000 7f9e766a1740 1 --2- 172.21.15.120:0/3203065040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b615a45d0 0x560b615a3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.363+0000 7f9e766a1740 1 -- 172.21.15.120:0/3203065040 >> 172.21.15.120:0/3203065040 conn(0x560b6156d310 msgr2=0x560b615aa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:45.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.363+0000 7f9e766a1740 1 -- 172.21.15.120:0/3203065040 shutdown_connections 2024-09-19T09:06:45.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.363+0000 7f9e766a1740 1 -- 172.21.15.120:0/3203065040 wait complete. 2024-09-19T09:06:45.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.363+0000 7f9e766a1740 1 -- start start 2024-09-19T09:06:45.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.363+0000 7f9e766a1740 1 Processor -- start 2024-09-19T09:06:45.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.363+0000 7f9e766a1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b615a45d0 0x560b615b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:45.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.364+0000 7f9e766a1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b615a5b40 0x560b615b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:45.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.364+0000 7f9e766a1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b615af370 0x560b61627c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:45.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.364+0000 7f9e766a1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c1ac760 con 0x560b615af370 2024-09-19T09:06:45.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.364+0000 7f9e766a1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c1ac090 con 0x560b615a5b40 2024-09-19T09:06:45.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.364+0000 7f9e766a1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c1a92b0 con 0x560b615a45d0 2024-09-19T09:06:45.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.364+0000 7f9e766a1740 10 client.0 _ll_register_callbacks cb 0x7f9e5c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:45.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.364+0000 7f9e72865640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b615af370 0x560b61627c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:45.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.364+0000 7f9e72064640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b615a45d0 0x560b615b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:45.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.364+0000 7f9e72865640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b615af370 0x560b61627c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37168/0 (socket says 172.21.15.120:37168) 2024-09-19T09:06:45.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.364+0000 7f9e72865640 1 -- 172.21.15.120:0/631426832 learned_addr learned my addr 172.21.15.120:0/631426832 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:45.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.364+0000 7f9e71863640 1 --2- 172.21.15.120:0/631426832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b615a5b40 0x560b615b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:45.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.364+0000 7f9e72865640 1 -- 172.21.15.120:0/631426832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b615a45d0 msgr2=0x560b615b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:45.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.364+0000 7f9e72865640 1 --2- 172.21.15.120:0/631426832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b615a45d0 0x560b615b25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.364+0000 7f9e72865640 1 -- 172.21.15.120:0/631426832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b615a5b40 msgr2=0x560b615b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:45.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.364+0000 7f9e72865640 1 --2- 172.21.15.120:0/631426832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b615a5b40 0x560b615b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.364+0000 7f9e72865640 1 -- 172.21.15.120:0/631426832 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b61628170 con 0x560b615af370 2024-09-19T09:06:45.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.364+0000 7f9e72865640 1 --2- 172.21.15.120:0/631426832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b615af370 0x560b61627c30 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f9e64005bf0 tx=0x7f9e640052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a324a416e8ab09c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:45.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.364+0000 7f9e55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:06:45.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.364+0000 7f9e55ffb640 1 -- 172.21.15.120:0/631426832 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e64043060 con 0x560b615af370 2024-09-19T09:06:45.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.364+0000 7f9e55ffb640 1 -- 172.21.15.120:0/631426832 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f9e64009620 con 0x560b615af370 2024-09-19T09:06:45.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.364+0000 7f9e766a1740 1 -- 172.21.15.120:0/631426832 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e640169b0 con 0x560b615af370 2024-09-19T09:06:45.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.365+0000 7f9e55ffb640 1 -- 172.21.15.120:0/631426832 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9e64004e30 con 0x560b615af370 2024-09-19T09:06:45.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.365+0000 7f9e766a1740 10 client.16818 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:45.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.365+0000 7f9e766a1740 1 -- 172.21.15.120:0/631426832 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560b61629330 con 0x560b615af370 2024-09-19T09:06:45.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.365+0000 7f9e55ffb640 1 -- 172.21.15.120:0/631426832 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e640169b0 con 0x560b615af370 2024-09-19T09:06:45.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.365+0000 7f9e55ffb640 1 -- 172.21.15.120:0/631426832 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9e6400f040 con 0x560b615af370 2024-09-19T09:06:45.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.365+0000 7f9e766a1740 10 client.16818 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:45.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.365+0000 7f9e766a1740 10 client.16818 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:06:45.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.365+0000 7f9e766a1740 20 client.16818 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:45.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.365+0000 7f9e766a1740 1 --2- 172.21.15.120:0/631426832 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560b61629850 0x560b6162bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:45.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.365+0000 7f9e766a1740 4 client.16818 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:06:45.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.365+0000 7f9e766a1740 1 -- 172.21.15.120:0/631426832 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b6148e170 con 0x560b61629850 2024-09-19T09:06:45.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.365+0000 7f9e72064640 1 --2- 172.21.15.120:0/631426832 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560b61629850 0x560b6162bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:45.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.366+0000 7f9e55ffb640 10 client.16818 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:06:45.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.366+0000 7f9e72064640 1 --2- 172.21.15.120:0/631426832 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560b61629850 0x560b6162bc40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=40967b282c865a69 server_cookie=b82b53fbe88304f2 in_seq=0 out_seq=0 2024-09-19T09:06:45.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.366+0000 7f9e55ffb640 1 -- 172.21.15.120:0/631426832 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b6148e170 con 0x560b61629850 2024-09-19T09:06:45.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.366+0000 7f9e55ffb640 10 client.16818 handle_command_reply: tid=0 2024-09-19T09:06:45.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.366+0000 7f9e766a1740 1 client.16818 shutdown 2024-09-19T09:06:45.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.366+0000 7f9e766a1740 1 -- 172.21.15.120:0/631426832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b615af370 msgr2=0x560b61627c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:45.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.366+0000 7f9e766a1740 1 --2- 172.21.15.120:0/631426832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b615af370 0x560b61627c30 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f9e64005bf0 tx=0x7f9e640052f0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.366+0000 7f9e766a1740 1 -- 172.21.15.120:0/631426832 shutdown_connections 2024-09-19T09:06:45.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.366+0000 7f9e766a1740 1 --2- 172.21.15.120:0/631426832 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560b61629850 0x560b6162bc40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.366+0000 7f9e55ffb640 0 client.16818 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:06:45.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.366+0000 7f9e766a1740 1 --2- 172.21.15.120:0/631426832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560b615af370 0x560b61627c30 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.366+0000 7f9e766a1740 1 --2- 172.21.15.120:0/631426832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560b615a5b40 0x560b615b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.366+0000 7f9e766a1740 1 --2- 172.21.15.120:0/631426832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b615a45d0 0x560b615b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.366+0000 7f9e766a1740 1 -- 172.21.15.120:0/631426832 >> 172.21.15.120:0/631426832 conn(0x560b6156d310 msgr2=0x560b615ab170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:45.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.366+0000 7f9e766a1740 1 -- 172.21.15.120:0/631426832 shutdown_connections 2024-09-19T09:06:45.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.366+0000 7f9e766a1740 1 -- 172.21.15.120:0/631426832 wait complete. 2024-09-19T09:06:45.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.366+0000 7f9e766a1740 20 client.16818 trim_cache size 0 max 16384 2024-09-19T09:06:45.374 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:06:45.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.367+0000 7f9e742c8640 1 -- 172.21.15.120:0/1494636832 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9e580a3240 msgr2=0x7f9e580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:45.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.367+0000 7f9e742c8640 1 --2- 172.21.15.120:0/1494636832 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9e580a3240 0x7f9e580a5700 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9e64010ec0 tx=0x7f9e6401b000 comp rx=0 tx=0).stop 2024-09-19T09:06:45.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.367+0000 7f9e742c8640 1 -- 172.21.15.120:0/1494636832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e6c100e20 msgr2=0x7f9e6c1973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:45.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.367+0000 7f9e742c8640 1 --2- 172.21.15.120:0/1494636832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e6c100e20 0x7f9e6c1973a0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800eb00 tx=0x7f9e68009f20 comp rx=0 tx=0).stop 2024-09-19T09:06:45.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.367+0000 7f9e742c8640 1 -- 172.21.15.120:0/1494636832 shutdown_connections 2024-09-19T09:06:45.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.367+0000 7f9e742c8640 1 --2- 172.21.15.120:0/1494636832 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9e580a3240 0x7f9e580a5700 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.367+0000 7f9e742c8640 1 --2- 172.21.15.120:0/1494636832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e6c102ef0 0x7f9e6c19bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.367+0000 7f9e742c8640 1 --2- 172.21.15.120:0/1494636832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e6c102570 0x7f9e6c1978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.367+0000 7f9e742c8640 1 --2- 172.21.15.120:0/1494636832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e6c100e20 0x7f9e6c1973a0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:45.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.367+0000 7f9e742c8640 1 -- 172.21.15.120:0/1494636832 >> 172.21.15.120:0/1494636832 conn(0x7f9e6c0fc7d0 msgr2=0x7f9e6c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:45.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.367+0000 7f9e742c8640 1 -- 172.21.15.120:0/1494636832 shutdown_connections 2024-09-19T09:06:45.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:45.367+0000 7f9e742c8640 1 -- 172.21.15.120:0/1494636832 wait complete. 2024-09-19T09:06:46.043 INFO:tasks.workunit.client.0.smithi120.stdout: 1 296404 9.00 MB/sec execute 41 sec latency 17.767 ms 2024-09-19T09:06:46.085 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:06:46.335 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:06:46.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.844+0000 7f289c492640 1 -- 172.21.15.120:0/1923689955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2894106f10 msgr2=0x7f289410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:46.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.844+0000 7f289c492640 1 --2- 172.21.15.120:0/1923689955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2894106f10 0x7f289410ea90 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f288c009ca0 tx=0x7f288c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.844+0000 7f289c492640 1 -- 172.21.15.120:0/1923689955 shutdown_connections 2024-09-19T09:06:46.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.844+0000 7f289c492640 1 --2- 172.21.15.120:0/1923689955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2894106f10 0x7f289410ea90 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.844+0000 7f289c492640 1 --2- 172.21.15.120:0/1923689955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2894102790 0x7f28941067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.844+0000 7f289c492640 1 --2- 172.21.15.120:0/1923689955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2894101de0 0x7f28941021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.844+0000 7f289c492640 1 -- 172.21.15.120:0/1923689955 >> 172.21.15.120:0/1923689955 conn(0x7f28940fd8c0 msgr2=0x7f28940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:46.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.844+0000 7f289c492640 1 -- 172.21.15.120:0/1923689955 shutdown_connections 2024-09-19T09:06:46.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.844+0000 7f289c492640 1 -- 172.21.15.120:0/1923689955 wait complete. 2024-09-19T09:06:46.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.845+0000 7f289c492640 1 Processor -- start 2024-09-19T09:06:46.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.845+0000 7f289c492640 1 -- start start 2024-09-19T09:06:46.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.845+0000 7f289c492640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2894101de0 0x7f28941983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:46.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.845+0000 7f289c492640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2894102790 0x7f2894198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:46.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.845+0000 7f289c492640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2894106f10 0x7f289419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:46.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.845+0000 7f289c492640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2894111f70 con 0x7f2894106f10 2024-09-19T09:06:46.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.845+0000 7f289c492640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2894111df0 con 0x7f2894101de0 2024-09-19T09:06:46.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.845+0000 7f289c492640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f28941120f0 con 0x7f2894102790 2024-09-19T09:06:46.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.845+0000 7f289aa2f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2894106f10 0x7f289419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:46.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.845+0000 7f289aa2f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2894106f10 0x7f289419cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37186/0 (socket says 172.21.15.120:37186) 2024-09-19T09:06:46.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.845+0000 7f289aa2f640 1 -- 172.21.15.120:0/3736435421 learned_addr learned my addr 172.21.15.120:0/3736435421 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:46.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.845+0000 7f289a22e640 1 --2- 172.21.15.120:0/3736435421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2894101de0 0x7f28941983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:46.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.845+0000 7f2899a2d640 1 --2- 172.21.15.120:0/3736435421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2894102790 0x7f2894198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:46.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.845+0000 7f289aa2f640 1 -- 172.21.15.120:0/3736435421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2894102790 msgr2=0x7f2894198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:46.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.845+0000 7f289aa2f640 1 --2- 172.21.15.120:0/3736435421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2894102790 0x7f2894198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.845+0000 7f289aa2f640 1 -- 172.21.15.120:0/3736435421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2894101de0 msgr2=0x7f28941983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:46.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.845+0000 7f289aa2f640 1 --2- 172.21.15.120:0/3736435421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2894101de0 0x7f28941983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.845+0000 7f289aa2f640 1 -- 172.21.15.120:0/3736435421 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f289419d380 con 0x7f2894106f10 2024-09-19T09:06:46.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.845+0000 7f2899a2d640 1 --2- 172.21.15.120:0/3736435421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2894102790 0x7f2894198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:06:46.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.846+0000 7f289aa2f640 1 --2- 172.21.15.120:0/3736435421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2894106f10 0x7f289419cca0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f288c031a20 tx=0x7f288c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:46.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.846+0000 7f288b7fe640 1 -- 172.21.15.120:0/3736435421 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c003c30 con 0x7f2894106f10 2024-09-19T09:06:46.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.846+0000 7f289c492640 1 -- 172.21.15.120:0/3736435421 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f289419d610 con 0x7f2894106f10 2024-09-19T09:06:46.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.846+0000 7f288b7fe640 1 -- 172.21.15.120:0/3736435421 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f288c003dd0 con 0x7f2894106f10 2024-09-19T09:06:46.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.846+0000 7f288b7fe640 1 -- 172.21.15.120:0/3736435421 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c02fcf0 con 0x7f2894106f10 2024-09-19T09:06:46.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.846+0000 7f289c492640 1 -- 172.21.15.120:0/3736435421 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28941a4f50 con 0x7f2894106f10 2024-09-19T09:06:46.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.848+0000 7f288b7fe640 1 -- 172.21.15.120:0/3736435421 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f288c004050 con 0x7f2894106f10 2024-09-19T09:06:46.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.849+0000 7f289e86b740 1 Processor -- start 2024-09-19T09:06:46.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.849+0000 7f289e86b740 1 -- start start 2024-09-19T09:06:46.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.849+0000 7f288b7fe640 1 --2- 172.21.15.120:0/3736435421 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f28800a3240 0x7f28800a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:46.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.849+0000 7f289e86b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ae9f6bf50 0x558ae9f6a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:46.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.849+0000 7f289e86b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ae9f6a9e0 0x558ae9f6ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:46.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.849+0000 7f289e86b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae9f75780 0x558ae9f77b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:46.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.849+0000 7f289e86b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558ae9ecc2f0 con 0x558ae9f75780 2024-09-19T09:06:46.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.849+0000 7f289e86b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558ae9e73cf0 con 0x558ae9f6a9e0 2024-09-19T09:06:46.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.849+0000 7f289e86b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558ae9ec82b0 con 0x558ae9f6bf50 2024-09-19T09:06:46.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.849+0000 7f288b7fe640 1 -- 172.21.15.120:0/3736435421 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f288c0f0c60 con 0x7f2894106f10 2024-09-19T09:06:46.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.849+0000 7f2899a2d640 1 --2- 172.21.15.120:0/3736435421 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f28800a3240 0x7f28800a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:46.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.849+0000 7f289aa2f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae9f75780 0x558ae9f77b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:46.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.849+0000 7f289aa2f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae9f75780 0x558ae9f77b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37190/0 (socket says 172.21.15.120:37190) 2024-09-19T09:06:46.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.849+0000 7f289aa2f640 1 -- 172.21.15.120:0/1021631017 learned_addr learned my addr 172.21.15.120:0/1021631017 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:46.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.849+0000 7f2899a2d640 1 --2- 172.21.15.120:0/1021631017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ae9f6a9e0 0x558ae9f6ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:46.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.849+0000 7f289a22e640 1 --2- 172.21.15.120:0/1021631017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ae9f6bf50 0x558ae9f6a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:46.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.850+0000 7f289aa2f640 1 -- 172.21.15.120:0/1021631017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ae9f6bf50 msgr2=0x558ae9f6a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:46.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.850+0000 7f289aa2f640 1 --2- 172.21.15.120:0/1021631017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ae9f6bf50 0x558ae9f6a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.850+0000 7f289aa2f640 1 -- 172.21.15.120:0/1021631017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ae9f6a9e0 msgr2=0x558ae9f6ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:46.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.850+0000 7f289aa2f640 1 --2- 172.21.15.120:0/1021631017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ae9f6a9e0 0x558ae9f6ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.850+0000 7f289aa2f640 1 -- 172.21.15.120:0/1021631017 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ae9e583a0 con 0x558ae9f75780 2024-09-19T09:06:46.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.850+0000 7f2899a2d640 1 --2- 172.21.15.120:0/1021631017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ae9f6a9e0 0x558ae9f6ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:46.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.850+0000 7f2899a2d640 1 --2- 172.21.15.120:0/3736435421 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f28800a3240 0x7f28800a5700 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f28941088e0 tx=0x7f289000bb00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:46.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.850+0000 7f289aa2f640 1 --2- 172.21.15.120:0/1021631017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae9f75780 0x558ae9f77b70 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f288c0bd2a0 tx=0x7f288c005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9ee3d11e84dec1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:46.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.850+0000 7f2888ff9640 1 -- 172.21.15.120:0/1021631017 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c035180 con 0x558ae9f75780 2024-09-19T09:06:46.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.850+0000 7f2888ff9640 1 -- 172.21.15.120:0/1021631017 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f288c0bda10 con 0x558ae9f75780 2024-09-19T09:06:46.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.850+0000 7f2888ff9640 1 -- 172.21.15.120:0/1021631017 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c0bb3b0 con 0x558ae9f75780 2024-09-19T09:06:46.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.851+0000 7f289e86b740 1 -- 172.21.15.120:0/1021631017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae9f75780 msgr2=0x558ae9f77b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:46.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.851+0000 7f289e86b740 1 --2- 172.21.15.120:0/1021631017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae9f75780 0x558ae9f77b70 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f288c0bd2a0 tx=0x7f288c005d80 comp rx=0 tx=0).stop 2024-09-19T09:06:46.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.851+0000 7f289e86b740 1 -- 172.21.15.120:0/1021631017 shutdown_connections 2024-09-19T09:06:46.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.851+0000 7f289e86b740 1 --2- 172.21.15.120:0/1021631017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae9f75780 0x558ae9f77b70 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.851+0000 7f289e86b740 1 --2- 172.21.15.120:0/1021631017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ae9f6a9e0 0x558ae9f6ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.851+0000 7f289e86b740 1 --2- 172.21.15.120:0/1021631017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ae9f6bf50 0x558ae9f6a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.851+0000 7f289e86b740 1 -- 172.21.15.120:0/1021631017 >> 172.21.15.120:0/1021631017 conn(0x558ae9f2f310 msgr2=0x558ae9f67070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:46.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.851+0000 7f289e86b740 1 -- 172.21.15.120:0/1021631017 shutdown_connections 2024-09-19T09:06:46.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.851+0000 7f289e86b740 1 -- 172.21.15.120:0/1021631017 wait complete. 2024-09-19T09:06:46.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.851+0000 7f289e86b740 1 -- start start 2024-09-19T09:06:46.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.851+0000 7f289e86b740 1 Processor -- start 2024-09-19T09:06:46.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.852+0000 7f289e86b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ae9f6a9e0 0x558ae9f751a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:46.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.852+0000 7f289e86b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ae9f6bf50 0x558ae9f72450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:46.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.852+0000 7f289e86b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae9f75780 0x558ae9f72a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:46.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.852+0000 7f289e86b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558ae9ecc2f0 con 0x558ae9f75780 2024-09-19T09:06:46.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.852+0000 7f289e86b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558ae9e87060 con 0x558ae9f6bf50 2024-09-19T09:06:46.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.852+0000 7f289e86b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558ae9f78240 con 0x558ae9f6a9e0 2024-09-19T09:06:46.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.852+0000 7f289e86b740 10 client.0 _ll_register_callbacks cb 0x558ae9bda8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:46.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.852+0000 7f289aa2f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae9f75780 0x558ae9f72a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:46.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.852+0000 7f2899a2d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ae9f6bf50 0x558ae9f72450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:46.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.852+0000 7f289aa2f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae9f75780 0x558ae9f72a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37194/0 (socket says 172.21.15.120:37194) 2024-09-19T09:06:46.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.852+0000 7f289aa2f640 1 -- 172.21.15.120:0/399968752 learned_addr learned my addr 172.21.15.120:0/399968752 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:46.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.852+0000 7f289a22e640 1 --2- 172.21.15.120:0/399968752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ae9f6a9e0 0x558ae9f751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:46.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.852+0000 7f289aa2f640 1 -- 172.21.15.120:0/399968752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ae9f6a9e0 msgr2=0x558ae9f751a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:46.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.852+0000 7f289aa2f640 1 --2- 172.21.15.120:0/399968752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ae9f6a9e0 0x558ae9f751a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.852+0000 7f289aa2f640 1 -- 172.21.15.120:0/399968752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ae9f6bf50 msgr2=0x558ae9f72450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:46.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.852+0000 7f289aa2f640 1 --2- 172.21.15.120:0/399968752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ae9f6bf50 0x558ae9f72450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.852+0000 7f289aa2f640 1 -- 172.21.15.120:0/399968752 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ae9c3d3c0 con 0x558ae9f75780 2024-09-19T09:06:46.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.852+0000 7f2899a2d640 1 --2- 172.21.15.120:0/399968752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ae9f6bf50 0x558ae9f72450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:46.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.852+0000 7f289aa2f640 1 --2- 172.21.15.120:0/399968752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae9f75780 0x558ae9f72a60 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f288c006010 tx=0x7f288c009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cabb72e0860d28a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:46.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.852+0000 7f287dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:06:46.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.852+0000 7f287dffb640 1 -- 172.21.15.120:0/399968752 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c035180 con 0x558ae9f75780 2024-09-19T09:06:46.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.852+0000 7f287dffb640 1 -- 172.21.15.120:0/399968752 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f288c0ee7e0 con 0x558ae9f75780 2024-09-19T09:06:46.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.852+0000 7f289e86b740 1 -- 172.21.15.120:0/399968752 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f288c0bda10 con 0x558ae9f75780 2024-09-19T09:06:46.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.852+0000 7f287dffb640 1 -- 172.21.15.120:0/399968752 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f288c0bd310 con 0x558ae9f75780 2024-09-19T09:06:46.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.852+0000 7f289e86b740 10 client.16830 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:46.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.852+0000 7f289e86b740 1 -- 172.21.15.120:0/399968752 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558ae9f83ad0 con 0x558ae9f75780 2024-09-19T09:06:46.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.853+0000 7f287dffb640 1 -- 172.21.15.120:0/399968752 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f288c0bda10 con 0x558ae9f75780 2024-09-19T09:06:46.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.853+0000 7f287dffb640 1 -- 172.21.15.120:0/399968752 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f288c00a2f0 con 0x558ae9f75780 2024-09-19T09:06:46.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.853+0000 7f289e86b740 10 client.16830 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:46.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.853+0000 7f289e86b740 10 client.16830 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:06:46.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.853+0000 7f289e86b740 20 client.16830 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:46.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.853+0000 7f289e86b740 1 --2- 172.21.15.120:0/399968752 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558ae9f84e20 0x558ae9f87240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:46.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.853+0000 7f289e86b740 4 client.16830 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:06:46.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.853+0000 7f289e86b740 1 -- 172.21.15.120:0/399968752 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ae9ed9860 con 0x558ae9f84e20 2024-09-19T09:06:46.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.853+0000 7f289a22e640 1 --2- 172.21.15.120:0/399968752 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558ae9f84e20 0x558ae9f87240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:46.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.853+0000 7f287dffb640 10 client.16830 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:06:46.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.853+0000 7f289a22e640 1 --2- 172.21.15.120:0/399968752 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558ae9f84e20 0x558ae9f87240 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=56d49d28e5e2c884 server_cookie=5f42cdffb8baf450 in_seq=0 out_seq=0 2024-09-19T09:06:46.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.854+0000 7f287dffb640 1 -- 172.21.15.120:0/399968752 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558ae9ed9860 con 0x558ae9f84e20 2024-09-19T09:06:46.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.854+0000 7f287dffb640 10 client.16830 handle_command_reply: tid=0 2024-09-19T09:06:46.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.854+0000 7f289e86b740 1 client.16830 shutdown 2024-09-19T09:06:46.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.854+0000 7f289e86b740 1 -- 172.21.15.120:0/399968752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae9f75780 msgr2=0x558ae9f72a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:46.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.854+0000 7f289e86b740 1 --2- 172.21.15.120:0/399968752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae9f75780 0x558ae9f72a60 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f288c006010 tx=0x7f288c009990 comp rx=0 tx=0).stop 2024-09-19T09:06:46.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.854+0000 7f289e86b740 1 -- 172.21.15.120:0/399968752 shutdown_connections 2024-09-19T09:06:46.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.854+0000 7f289e86b740 1 --2- 172.21.15.120:0/399968752 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558ae9f84e20 0x558ae9f87240 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.854+0000 7f289e86b740 1 --2- 172.21.15.120:0/399968752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae9f75780 0x558ae9f72a60 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.854+0000 7f289e86b740 1 --2- 172.21.15.120:0/399968752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ae9f6bf50 0x558ae9f72450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.854+0000 7f287dffb640 0 client.16830 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:06:46.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.854+0000 7f289e86b740 1 --2- 172.21.15.120:0/399968752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ae9f6a9e0 0x558ae9f751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.854+0000 7f289e86b740 1 -- 172.21.15.120:0/399968752 >> 172.21.15.120:0/399968752 conn(0x558ae9f2f310 msgr2=0x558ae9f6cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:46.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.854+0000 7f289e86b740 1 -- 172.21.15.120:0/399968752 shutdown_connections 2024-09-19T09:06:46.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.854+0000 7f289e86b740 1 -- 172.21.15.120:0/399968752 wait complete. 2024-09-19T09:06:46.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.854+0000 7f289e86b740 20 client.16830 trim_cache size 0 max 16384 2024-09-19T09:06:46.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.868+0000 7f289e86b740 1 Processor -- start 2024-09-19T09:06:46.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.868+0000 7f289e86b740 1 -- start start 2024-09-19T09:06:46.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.868+0000 7f289e86b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ae9f6a9e0 0x558ae9f67220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:46.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.868+0000 7f289e86b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ae9f6bf50 0x558ae9f71d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:46.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.868+0000 7f289e86b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae9f75780 0x558ae9f81320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:46.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.868+0000 7f289e86b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f28941ab3a0 con 0x558ae9f75780 2024-09-19T09:06:46.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.868+0000 7f289e86b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f28941acee0 con 0x558ae9f6a9e0 2024-09-19T09:06:46.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.868+0000 7f289e86b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f28941aba70 con 0x558ae9f6bf50 2024-09-19T09:06:46.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.868+0000 7f289aa2f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae9f75780 0x558ae9f81320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:46.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.868+0000 7f289aa2f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae9f75780 0x558ae9f81320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37202/0 (socket says 172.21.15.120:37202) 2024-09-19T09:06:46.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.868+0000 7f289aa2f640 1 -- 172.21.15.120:0/2457646499 learned_addr learned my addr 172.21.15.120:0/2457646499 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:46.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.868+0000 7f289a22e640 1 --2- 172.21.15.120:0/2457646499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ae9f6a9e0 0x558ae9f67220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:46.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.868+0000 7f289aa2f640 1 -- 172.21.15.120:0/2457646499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ae9f6bf50 msgr2=0x558ae9f71d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:46.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.868+0000 7f289aa2f640 1 --2- 172.21.15.120:0/2457646499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ae9f6bf50 0x558ae9f71d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.868+0000 7f289aa2f640 1 -- 172.21.15.120:0/2457646499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ae9f6a9e0 msgr2=0x558ae9f67220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:46.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.868+0000 7f289aa2f640 1 --2- 172.21.15.120:0/2457646499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ae9f6a9e0 0x558ae9f67220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.868+0000 7f289aa2f640 1 -- 172.21.15.120:0/2457646499 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ae9f71400 con 0x558ae9f75780 2024-09-19T09:06:46.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.868+0000 7f289aa2f640 1 --2- 172.21.15.120:0/2457646499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae9f75780 0x558ae9f81320 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f288c0bd2a0 tx=0x7f288c0bb820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0fa8a3dfd68018 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:46.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.868+0000 7f287ffff640 1 -- 172.21.15.120:0/2457646499 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c0bda10 con 0x558ae9f75780 2024-09-19T09:06:46.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.868+0000 7f287ffff640 1 -- 172.21.15.120:0/2457646499 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f288c02fe90 con 0x558ae9f75780 2024-09-19T09:06:46.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.868+0000 7f287ffff640 1 -- 172.21.15.120:0/2457646499 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c0c5ef0 con 0x558ae9f75780 2024-09-19T09:06:46.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.869+0000 7f289e86b740 1 -- 172.21.15.120:0/2457646499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae9f75780 msgr2=0x558ae9f81320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:46.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.869+0000 7f289e86b740 1 --2- 172.21.15.120:0/2457646499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae9f75780 0x558ae9f81320 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f288c0bd2a0 tx=0x7f288c0bb820 comp rx=0 tx=0).stop 2024-09-19T09:06:46.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.869+0000 7f289e86b740 1 -- 172.21.15.120:0/2457646499 shutdown_connections 2024-09-19T09:06:46.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.869+0000 7f289e86b740 1 --2- 172.21.15.120:0/2457646499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae9f75780 0x558ae9f81320 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.869+0000 7f289e86b740 1 --2- 172.21.15.120:0/2457646499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ae9f6bf50 0x558ae9f71d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.869+0000 7f289e86b740 1 --2- 172.21.15.120:0/2457646499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ae9f6a9e0 0x558ae9f67220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.869+0000 7f289e86b740 1 -- 172.21.15.120:0/2457646499 >> 172.21.15.120:0/2457646499 conn(0x558ae9f2f310 msgr2=0x558ae9f66b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:46.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.869+0000 7f289e86b740 1 -- 172.21.15.120:0/2457646499 shutdown_connections 2024-09-19T09:06:46.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.869+0000 7f289e86b740 1 -- 172.21.15.120:0/2457646499 wait complete. 2024-09-19T09:06:46.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.869+0000 7f289e86b740 1 -- start start 2024-09-19T09:06:46.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.870+0000 7f289e86b740 1 Processor -- start 2024-09-19T09:06:46.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.870+0000 7f289e86b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ae9f6a9e0 0x558ae9f7d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:46.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.870+0000 7f289e86b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae9f6bf50 0x558ae9f7dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:46.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.870+0000 7f289e86b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ae9f75780 0x558ae9f78a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:46.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.870+0000 7f289e86b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f28941ac810 con 0x558ae9f6bf50 2024-09-19T09:06:46.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.870+0000 7f289e86b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f28941ac140 con 0x558ae9f75780 2024-09-19T09:06:46.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.870+0000 7f289e86b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f28941a9360 con 0x558ae9f6a9e0 2024-09-19T09:06:46.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.870+0000 7f289e86b740 10 client.0 _ll_register_callbacks cb 0x7f2890014470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:46.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.870+0000 7f2899a2d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae9f6bf50 0x558ae9f7dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:46.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.870+0000 7f2899a2d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae9f6bf50 0x558ae9f7dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37204/0 (socket says 172.21.15.120:37204) 2024-09-19T09:06:46.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.870+0000 7f2899a2d640 1 -- 172.21.15.120:0/168682366 learned_addr learned my addr 172.21.15.120:0/168682366 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:46.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.870+0000 7f289a22e640 1 --2- 172.21.15.120:0/168682366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ae9f6a9e0 0x558ae9f7d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:46.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.870+0000 7f289aa2f640 1 --2- 172.21.15.120:0/168682366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ae9f75780 0x558ae9f78a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:46.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.870+0000 7f2899a2d640 1 -- 172.21.15.120:0/168682366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ae9f6a9e0 msgr2=0x558ae9f7d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:46.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.870+0000 7f2899a2d640 1 --2- 172.21.15.120:0/168682366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ae9f6a9e0 0x558ae9f7d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.870+0000 7f2899a2d640 1 -- 172.21.15.120:0/168682366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ae9f75780 msgr2=0x558ae9f78a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:46.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.870+0000 7f2899a2d640 1 --2- 172.21.15.120:0/168682366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ae9f75780 0x558ae9f78a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.870+0000 7f2899a2d640 1 -- 172.21.15.120:0/168682366 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ae9f79260 con 0x558ae9f6bf50 2024-09-19T09:06:46.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.870+0000 7f289a22e640 1 --2- 172.21.15.120:0/168682366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ae9f6a9e0 0x558ae9f7d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:46.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.871+0000 7f2899a2d640 1 --2- 172.21.15.120:0/168682366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae9f6bf50 0x558ae9f7dbb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f289000bfd0 tx=0x7f28900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d69860f181ab06eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:46.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.871+0000 7f287dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:06:46.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.871+0000 7f287dffb640 1 -- 172.21.15.120:0/168682366 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2890003cd0 con 0x558ae9f6bf50 2024-09-19T09:06:46.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.871+0000 7f289e86b740 1 -- 172.21.15.120:0/168682366 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f288c02fe90 con 0x558ae9f6bf50 2024-09-19T09:06:46.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.871+0000 7f287dffb640 1 -- 172.21.15.120:0/168682366 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f2890005020 con 0x558ae9f6bf50 2024-09-19T09:06:46.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.871+0000 7f287dffb640 1 -- 172.21.15.120:0/168682366 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2890005260 con 0x558ae9f6bf50 2024-09-19T09:06:46.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.871+0000 7f289e86b740 10 client.16836 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:46.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.871+0000 7f289e86b740 1 -- 172.21.15.120:0/168682366 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558ae9fef6a0 con 0x558ae9f6bf50 2024-09-19T09:06:46.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.871+0000 7f287dffb640 1 -- 172.21.15.120:0/168682366 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f288c02fe90 con 0x558ae9f6bf50 2024-09-19T09:06:46.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.871+0000 7f287dffb640 1 -- 172.21.15.120:0/168682366 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2890028070 con 0x558ae9f6bf50 2024-09-19T09:06:46.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.871+0000 7f289e86b740 10 client.16836 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:46.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.871+0000 7f289e86b740 10 client.16836 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:06:46.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.871+0000 7f289e86b740 20 client.16836 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:46.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.871+0000 7f289e86b740 1 --2- 172.21.15.120:0/168682366 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558ae9fefd70 0x558ae9ff2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:46.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.871+0000 7f289e86b740 4 client.16836 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:06:46.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.871+0000 7f289e86b740 1 -- 172.21.15.120:0/168682366 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ae9ed3d70 con 0x558ae9fefd70 2024-09-19T09:06:46.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.872+0000 7f289a22e640 1 --2- 172.21.15.120:0/168682366 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558ae9fefd70 0x558ae9ff2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:46.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.872+0000 7f287dffb640 10 client.16836 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:06:46.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.872+0000 7f289a22e640 1 --2- 172.21.15.120:0/168682366 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558ae9fefd70 0x558ae9ff2160 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9f4cd8875f065c9d server_cookie=ed72c4525da6bd0c in_seq=0 out_seq=0 2024-09-19T09:06:46.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.872+0000 7f287dffb640 1 -- 172.21.15.120:0/168682366 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ae9ed3d70 con 0x558ae9fefd70 2024-09-19T09:06:46.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.872+0000 7f287dffb640 10 client.16836 handle_command_reply: tid=0 2024-09-19T09:06:46.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.872+0000 7f289e86b740 1 client.16836 shutdown 2024-09-19T09:06:46.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.872+0000 7f289e86b740 1 -- 172.21.15.120:0/168682366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae9f6bf50 msgr2=0x558ae9f7dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:46.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.872+0000 7f289e86b740 1 --2- 172.21.15.120:0/168682366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae9f6bf50 0x558ae9f7dbb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f289000bfd0 tx=0x7f28900046c0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.872+0000 7f289e86b740 1 -- 172.21.15.120:0/168682366 shutdown_connections 2024-09-19T09:06:46.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.872+0000 7f289e86b740 1 --2- 172.21.15.120:0/168682366 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558ae9fefd70 0x558ae9ff2160 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.872+0000 7f289e86b740 1 --2- 172.21.15.120:0/168682366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ae9f75780 0x558ae9f78a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.872+0000 7f289e86b740 1 --2- 172.21.15.120:0/168682366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ae9f6bf50 0x558ae9f7dbb0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.872+0000 7f287dffb640 0 client.16836 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:06:46.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.872+0000 7f289e86b740 1 --2- 172.21.15.120:0/168682366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ae9f6a9e0 0x558ae9f7d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.872+0000 7f289e86b740 1 -- 172.21.15.120:0/168682366 >> 172.21.15.120:0/168682366 conn(0x558ae9f2f310 msgr2=0x558ae9f73cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:46.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.872+0000 7f289e86b740 1 -- 172.21.15.120:0/168682366 shutdown_connections 2024-09-19T09:06:46.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.872+0000 7f289e86b740 1 -- 172.21.15.120:0/168682366 wait complete. 2024-09-19T09:06:46.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.873+0000 7f289e86b740 20 client.16836 trim_cache size 0 max 16384 2024-09-19T09:06:46.906 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:06:46.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.873+0000 7f289c492640 1 -- 172.21.15.120:0/3736435421 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f28800a3240 msgr2=0x7f28800a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:46.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.873+0000 7f289c492640 1 --2- 172.21.15.120:0/3736435421 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f28800a3240 0x7f28800a5700 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f28941088e0 tx=0x7f289000bb00 comp rx=0 tx=0).stop 2024-09-19T09:06:46.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.873+0000 7f289c492640 1 -- 172.21.15.120:0/3736435421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2894106f10 msgr2=0x7f289419cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:46.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.873+0000 7f289c492640 1 --2- 172.21.15.120:0/3736435421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2894106f10 0x7f289419cca0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f288c031a20 tx=0x7f288c031a50 comp rx=0 tx=0).stop 2024-09-19T09:06:46.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.873+0000 7f289c492640 1 -- 172.21.15.120:0/3736435421 shutdown_connections 2024-09-19T09:06:46.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.873+0000 7f289c492640 1 --2- 172.21.15.120:0/3736435421 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f28800a3240 0x7f28800a5700 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.873+0000 7f289c492640 1 --2- 172.21.15.120:0/3736435421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2894106f10 0x7f289419cca0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.873+0000 7f289c492640 1 --2- 172.21.15.120:0/3736435421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2894102790 0x7f2894198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.873+0000 7f289c492640 1 --2- 172.21.15.120:0/3736435421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2894101de0 0x7f28941983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:46.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.873+0000 7f289c492640 1 -- 172.21.15.120:0/3736435421 >> 172.21.15.120:0/3736435421 conn(0x7f28940fd8c0 msgr2=0x7f28940feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:46.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.873+0000 7f289c492640 1 -- 172.21.15.120:0/3736435421 shutdown_connections 2024-09-19T09:06:46.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:46.873+0000 7f289c492640 1 -- 172.21.15.120:0/3736435421 wait complete. 2024-09-19T09:06:47.044 INFO:tasks.workunit.client.0.smithi120.stdout: 1 298963 9.13 MB/sec execute 42 sec latency 25.722 ms 2024-09-19T09:06:47.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:46 smithi120 ceph-mon[31150]: pgmap v458: 97 pgs: 97 active+clean; 38 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 300 op/s 2024-09-19T09:06:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.110+0000 7f937e423640 1 -- 172.21.15.120:0/2423029778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9378104f50 msgr2=0x7f9378107340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.110+0000 7f937e423640 1 --2- 172.21.15.120:0/2423029778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9378104f50 0x7f9378107340 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f936c009900 tx=0x7f936c02a410 comp rx=0 tx=0).stop 2024-09-19T09:06:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.110+0000 7f937e423640 1 -- 172.21.15.120:0/2423029778 shutdown_connections 2024-09-19T09:06:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.110+0000 7f937e423640 1 --2- 172.21.15.120:0/2423029778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9378107880 0x7f9378109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.110+0000 7f937e423640 1 --2- 172.21.15.120:0/2423029778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9378104f50 0x7f9378107340 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.110+0000 7f937e423640 1 --2- 172.21.15.120:0/2423029778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9378102620 0x7f9378104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.110+0000 7f937e423640 1 -- 172.21.15.120:0/2423029778 >> 172.21.15.120:0/2423029778 conn(0x7f93780fc1e0 msgr2=0x7f93780fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.110+0000 7f937e423640 1 -- 172.21.15.120:0/2423029778 shutdown_connections 2024-09-19T09:06:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.110+0000 7f937e423640 1 -- 172.21.15.120:0/2423029778 wait complete. 2024-09-19T09:06:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.111+0000 7f937e423640 1 Processor -- start 2024-09-19T09:06:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.111+0000 7f937e423640 1 -- start start 2024-09-19T09:06:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.111+0000 7f937e423640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9378102620 0x7f9378197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.111+0000 7f937e423640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9378104f50 0x7f93781983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.111+0000 7f937e423640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9378107880 0x7f937819f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.111+0000 7f937e423640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f937810d7f0 con 0x7f9378102620 2024-09-19T09:06:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.111+0000 7f937e423640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f937810d670 con 0x7f9378104f50 2024-09-19T09:06:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.111+0000 7f937e423640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f937810d970 con 0x7f9378107880 2024-09-19T09:06:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.111+0000 7f9377fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9378102620 0x7f9378197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.111+0000 7f9377fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9378102620 0x7f9378197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37216/0 (socket says 172.21.15.120:37216) 2024-09-19T09:06:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.111+0000 7f9377fff640 1 -- 172.21.15.120:0/1251208137 learned_addr learned my addr 172.21.15.120:0/1251208137 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.111+0000 7f93777fe640 1 --2- 172.21.15.120:0/1251208137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9378104f50 0x7f93781983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.111+0000 7f937c9c0640 1 --2- 172.21.15.120:0/1251208137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9378107880 0x7f937819f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.111+0000 7f9377fff640 1 -- 172.21.15.120:0/1251208137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9378107880 msgr2=0x7f937819f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.111+0000 7f9377fff640 1 --2- 172.21.15.120:0/1251208137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9378107880 0x7f937819f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.111+0000 7f9377fff640 1 -- 172.21.15.120:0/1251208137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9378104f50 msgr2=0x7f93781983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.111+0000 7f9377fff640 1 --2- 172.21.15.120:0/1251208137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9378104f50 0x7f93781983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.111+0000 7f9377fff640 1 -- 172.21.15.120:0/1251208137 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937819fa90 con 0x7f9378102620 2024-09-19T09:06:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.112+0000 7f9377fff640 1 --2- 172.21.15.120:0/1251208137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9378102620 0x7f9378197e70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f936800e9f0 tx=0x7f936800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.112+0000 7f93757fa640 1 -- 172.21.15.120:0/1251208137 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936800cd80 con 0x7f9378102620 2024-09-19T09:06:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.112+0000 7f937e423640 1 -- 172.21.15.120:0/1251208137 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f937819fd80 con 0x7f9378102620 2024-09-19T09:06:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.112+0000 7f93757fa640 1 -- 172.21.15.120:0/1251208137 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9368010910 con 0x7f9378102620 2024-09-19T09:06:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.112+0000 7f93757fa640 1 -- 172.21.15.120:0/1251208137 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93680188f0 con 0x7f9378102620 2024-09-19T09:06:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.112+0000 7f937e423640 1 -- 172.21.15.120:0/1251208137 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93781a01e0 con 0x7f9378102620 2024-09-19T09:06:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.114+0000 7f93757fa640 1 -- 172.21.15.120:0/1251208137 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f9368010ac0 con 0x7f9378102620 2024-09-19T09:06:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.114+0000 7f93811bc740 1 Processor -- start 2024-09-19T09:06:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.114+0000 7f93811bc740 1 -- start start 2024-09-19T09:06:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.115+0000 7f93811bc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de833249a0 0x55de83324da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.115+0000 7f93811bc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de833252e0 0x55de83329da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.115+0000 7f93811bc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de8332a370 0x55de8332c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.115+0000 7f93811bc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55de832882f0 con 0x55de8332a370 2024-09-19T09:06:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.115+0000 7f93811bc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55de83230270 con 0x55de833252e0 2024-09-19T09:06:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.115+0000 7f93811bc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55de832842b0 con 0x55de833249a0 2024-09-19T09:06:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.115+0000 7f93777fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de8332a370 0x55de8332c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.115+0000 7f937c9c0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de833249a0 0x55de83324da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.115+0000 7f937c9c0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de833249a0 0x55de83324da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:45534/0 (socket says 172.21.15.120:45534) 2024-09-19T09:06:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.115+0000 7f9377fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de833252e0 0x55de83329da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.115+0000 7f937c9c0640 1 -- 172.21.15.120:0/3065890481 learned_addr learned my addr 172.21.15.120:0/3065890481 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.115+0000 7f93757fa640 1 --2- 172.21.15.120:0/1251208137 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93600a32f0 0x7f93600a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.115+0000 7f937c9c0640 1 --2- 172.21.15.120:0/1251208137 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93600a32f0 0x7f93600a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.115+0000 7f93777fe640 1 -- 172.21.15.120:0/3065890481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de833249a0 msgr2=0x55de83324da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.115+0000 7f93777fe640 1 --2- 172.21.15.120:0/3065890481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de833249a0 0x55de83324da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.115+0000 7f93777fe640 1 -- 172.21.15.120:0/3065890481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de833252e0 msgr2=0x55de83329da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.115+0000 7f93757fa640 1 -- 172.21.15.120:0/1251208137 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f93680cfd00 con 0x7f9378102620 2024-09-19T09:06:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.115+0000 7f93777fe640 1 --2- 172.21.15.120:0/3065890481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de833252e0 0x55de83329da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.115+0000 7f93777fe640 1 -- 172.21.15.120:0/3065890481 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de8322f340 con 0x55de8332a370 2024-09-19T09:06:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.115+0000 7f9377fff640 1 --2- 172.21.15.120:0/3065890481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de833252e0 0x55de83329da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.115+0000 7f93777fe640 1 --2- 172.21.15.120:0/3065890481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de8332a370 0x55de8332c830 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f936c009230 tx=0x7f936c005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe026a6c292c1f8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.115+0000 7f937c9c0640 1 --2- 172.21.15.120:0/1251208137 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93600a32f0 0x7f93600a57b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9370009fd0 tx=0x7f9370004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.115+0000 7f935e7fc640 1 -- 172.21.15.120:0/3065890481 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c040070 con 0x55de8332a370 2024-09-19T09:06:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.116+0000 7f935e7fc640 1 -- 172.21.15.120:0/3065890481 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f936c004030 con 0x55de8332a370 2024-09-19T09:06:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.116+0000 7f935e7fc640 1 -- 172.21.15.120:0/3065890481 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c035040 con 0x55de8332a370 2024-09-19T09:06:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.116+0000 7f93811bc740 1 -- 172.21.15.120:0/3065890481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de8332a370 msgr2=0x55de8332c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.116+0000 7f93811bc740 1 --2- 172.21.15.120:0/3065890481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de8332a370 0x55de8332c830 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f936c009230 tx=0x7f936c005e20 comp rx=0 tx=0).stop 2024-09-19T09:06:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.116+0000 7f93811bc740 1 -- 172.21.15.120:0/3065890481 shutdown_connections 2024-09-19T09:06:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.116+0000 7f93811bc740 1 --2- 172.21.15.120:0/3065890481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de8332a370 0x55de8332c830 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.116+0000 7f93811bc740 1 --2- 172.21.15.120:0/3065890481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de833252e0 0x55de83329da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.116+0000 7f93811bc740 1 --2- 172.21.15.120:0/3065890481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de833249a0 0x55de83324da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.116+0000 7f93811bc740 1 -- 172.21.15.120:0/3065890481 >> 172.21.15.120:0/3065890481 conn(0x55de83320010 msgr2=0x55de83320410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.116+0000 7f93811bc740 1 -- 172.21.15.120:0/3065890481 shutdown_connections 2024-09-19T09:06:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.116+0000 7f93811bc740 1 -- 172.21.15.120:0/3065890481 wait complete. 2024-09-19T09:06:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.117+0000 7f93811bc740 1 -- start start 2024-09-19T09:06:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.117+0000 7f93811bc740 1 Processor -- start 2024-09-19T09:06:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.117+0000 7f93811bc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de833249a0 0x55de8332ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.117+0000 7f93811bc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de833252e0 0x55de8332f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.117+0000 7f93811bc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de8332a370 0x55de83336140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.117+0000 7f93811bc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55de8332d2e0 con 0x55de8332a370 2024-09-19T09:06:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.117+0000 7f93811bc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55de83211290 con 0x55de833249a0 2024-09-19T09:06:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.117+0000 7f93811bc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55de8332d460 con 0x55de833252e0 2024-09-19T09:06:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.117+0000 7f93811bc740 10 client.0 _ll_register_callbacks cb 0x55de83242140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.117+0000 7f937c9c0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de8332a370 0x55de83336140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.117+0000 7f9377fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de833249a0 0x55de8332ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.117+0000 7f937c9c0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de8332a370 0x55de83336140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37232/0 (socket says 172.21.15.120:37232) 2024-09-19T09:06:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.117+0000 7f937c9c0640 1 -- 172.21.15.120:0/2956248685 learned_addr learned my addr 172.21.15.120:0/2956248685 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.117+0000 7f93777fe640 1 --2- 172.21.15.120:0/2956248685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de833252e0 0x55de8332f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.117+0000 7f937c9c0640 1 -- 172.21.15.120:0/2956248685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de833252e0 msgr2=0x55de8332f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.117+0000 7f937c9c0640 1 --2- 172.21.15.120:0/2956248685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de833252e0 0x55de8332f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.117+0000 7f937c9c0640 1 -- 172.21.15.120:0/2956248685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de833249a0 msgr2=0x55de8332ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.117+0000 7f937c9c0640 1 --2- 172.21.15.120:0/2956248685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de833249a0 0x55de8332ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.117+0000 7f937c9c0640 1 -- 172.21.15.120:0/2956248685 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de82f9b210 con 0x55de8332a370 2024-09-19T09:06:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.117+0000 7f93777fe640 1 --2- 172.21.15.120:0/2956248685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de833252e0 0x55de8332f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:06:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.118+0000 7f937c9c0640 1 --2- 172.21.15.120:0/2956248685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de8332a370 0x55de83336140 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f937000f900 tx=0x7f9370004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99af9f7d4b9a9cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.118+0000 7f93577fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:06:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.118+0000 7f93577fe640 1 -- 172.21.15.120:0/2956248685 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9370025070 con 0x55de8332a370 2024-09-19T09:06:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.118+0000 7f93577fe640 1 -- 172.21.15.120:0/2956248685 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f9370010dc0 con 0x55de8332a370 2024-09-19T09:06:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.118+0000 7f93811bc740 1 -- 172.21.15.120:0/2956248685 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f936c004030 con 0x55de8332a370 2024-09-19T09:06:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.118+0000 7f93577fe640 1 -- 172.21.15.120:0/2956248685 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f937002a460 con 0x55de8332a370 2024-09-19T09:06:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.118+0000 7f93811bc740 10 client.16848 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.118+0000 7f93811bc740 1 -- 172.21.15.120:0/2956248685 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55de83067620 con 0x55de8332a370 2024-09-19T09:06:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.118+0000 7f93577fe640 1 -- 172.21.15.120:0/2956248685 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f936c004030 con 0x55de8332a370 2024-09-19T09:06:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.118+0000 7f93577fe640 1 -- 172.21.15.120:0/2956248685 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9370021850 con 0x55de8332a370 2024-09-19T09:06:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.118+0000 7f93811bc740 10 client.16848 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.118+0000 7f93811bc740 10 client.16848 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:06:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.118+0000 7f93811bc740 20 client.16848 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.118+0000 7f93811bc740 1 --2- 172.21.15.120:0/2956248685 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55de8333a890 0x55de8333ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.119+0000 7f93811bc740 4 client.16848 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:06:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.119+0000 7f93811bc740 1 -- 172.21.15.120:0/2956248685 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de83295a20 con 0x55de8333a890 2024-09-19T09:06:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.119+0000 7f9377fff640 1 --2- 172.21.15.120:0/2956248685 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55de8333a890 0x55de8333ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.119+0000 7f9377fff640 1 --2- 172.21.15.120:0/2956248685 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55de8333a890 0x55de8333ccc0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=825807533933e83c server_cookie=fe162a784a27c719 in_seq=0 out_seq=0 2024-09-19T09:06:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.119+0000 7f93577fe640 10 client.16848 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.120+0000 7f93577fe640 1 -- 172.21.15.120:0/2956248685 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55de83295a20 con 0x55de8333a890 2024-09-19T09:06:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.120+0000 7f93577fe640 10 client.16848 handle_command_reply: tid=0 2024-09-19T09:06:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.120+0000 7f93811bc740 1 client.16848 shutdown 2024-09-19T09:06:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.120+0000 7f93811bc740 1 -- 172.21.15.120:0/2956248685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de8332a370 msgr2=0x55de83336140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.120+0000 7f93811bc740 1 --2- 172.21.15.120:0/2956248685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de8332a370 0x55de83336140 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f937000f900 tx=0x7f9370004a40 comp rx=0 tx=0).stop 2024-09-19T09:06:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.120+0000 7f93811bc740 1 -- 172.21.15.120:0/2956248685 shutdown_connections 2024-09-19T09:06:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.120+0000 7f93811bc740 1 --2- 172.21.15.120:0/2956248685 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55de8333a890 0x55de8333ccc0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.120+0000 7f93811bc740 1 --2- 172.21.15.120:0/2956248685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de8332a370 0x55de83336140 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.120+0000 7f93811bc740 1 --2- 172.21.15.120:0/2956248685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de833252e0 0x55de8332f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.120+0000 7f93811bc740 1 --2- 172.21.15.120:0/2956248685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de833249a0 0x55de8332ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.120+0000 7f93811bc740 1 -- 172.21.15.120:0/2956248685 >> 172.21.15.120:0/2956248685 conn(0x55de83320010 msgr2=0x55de83321820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.120+0000 7f93577fe640 0 client.16848 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.121+0000 7f93811bc740 1 -- 172.21.15.120:0/2956248685 shutdown_connections 2024-09-19T09:06:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.121+0000 7f93811bc740 1 -- 172.21.15.120:0/2956248685 wait complete. 2024-09-19T09:06:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.121+0000 7f93811bc740 20 client.16848 trim_cache size 0 max 16384 2024-09-19T09:06:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.141+0000 7f93811bc740 1 Processor -- start 2024-09-19T09:06:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.141+0000 7f93811bc740 1 -- start start 2024-09-19T09:06:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.141+0000 7f93811bc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de833249a0 0x55de8331f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.141+0000 7f93811bc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de833252e0 0x55de83336100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.141+0000 7f93811bc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de8332a370 0x55de833415c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.141+0000 7f93811bc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f93781a8ff0 con 0x55de833249a0 2024-09-19T09:06:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.141+0000 7f93811bc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f93781aab30 con 0x55de8332a370 2024-09-19T09:06:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.141+0000 7f93811bc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f93781a96c0 con 0x55de833252e0 2024-09-19T09:06:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.141+0000 7f9377fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de833249a0 0x55de8331f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.141+0000 7f93777fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de833252e0 0x55de83336100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.141+0000 7f93777fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de833252e0 0x55de83336100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:45554/0 (socket says 172.21.15.120:45554) 2024-09-19T09:06:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.141+0000 7f937c9c0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de8332a370 0x55de833415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.141+0000 7f93777fe640 1 -- 172.21.15.120:0/1731751748 learned_addr learned my addr 172.21.15.120:0/1731751748 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.141+0000 7f9377fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de833249a0 0x55de8331f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37244/0 (socket says 172.21.15.120:37244) 2024-09-19T09:06:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.141+0000 7f93777fe640 1 -- 172.21.15.120:0/1731751748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de8332a370 msgr2=0x55de833415c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.141+0000 7f93777fe640 1 --2- 172.21.15.120:0/1731751748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de8332a370 0x55de833415c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.141+0000 7f93777fe640 1 -- 172.21.15.120:0/1731751748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de833249a0 msgr2=0x55de8331f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.141+0000 7f93777fe640 1 --2- 172.21.15.120:0/1731751748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de833249a0 0x55de8331f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.142+0000 7f93777fe640 1 -- 172.21.15.120:0/1731751748 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de8331fe30 con 0x55de833252e0 2024-09-19T09:06:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.142+0000 7f9377fff640 1 --2- 172.21.15.120:0/1731751748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de833249a0 0x55de8331f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.142+0000 7f93777fe640 1 --2- 172.21.15.120:0/1731751748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de833252e0 0x55de83336100 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f936c009230 tx=0x7f936c005300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eed25e982f3ae70b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.142+0000 7f935dffb640 1 -- 172.21.15.120:0/1731751748 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c040070 con 0x55de833252e0 2024-09-19T09:06:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.142+0000 7f935dffb640 1 -- 172.21.15.120:0/1731751748 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f936c04d070 con 0x55de833252e0 2024-09-19T09:06:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.142+0000 7f935dffb640 1 -- 172.21.15.120:0/1731751748 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c035040 con 0x55de833252e0 2024-09-19T09:06:47.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.143+0000 7f93811bc740 1 -- 172.21.15.120:0/1731751748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de833252e0 msgr2=0x55de83336100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.143+0000 7f93811bc740 1 --2- 172.21.15.120:0/1731751748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de833252e0 0x55de83336100 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f936c009230 tx=0x7f936c005300 comp rx=0 tx=0).stop 2024-09-19T09:06:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.143+0000 7f93811bc740 1 -- 172.21.15.120:0/1731751748 shutdown_connections 2024-09-19T09:06:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.143+0000 7f93811bc740 1 --2- 172.21.15.120:0/1731751748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de8332a370 0x55de833415c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.143+0000 7f93811bc740 1 --2- 172.21.15.120:0/1731751748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de833252e0 0x55de83336100 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.143+0000 7f93811bc740 1 --2- 172.21.15.120:0/1731751748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de833249a0 0x55de8331f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.143+0000 7f93811bc740 1 -- 172.21.15.120:0/1731751748 >> 172.21.15.120:0/1731751748 conn(0x55de83320010 msgr2=0x55de8331e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.143+0000 7f93811bc740 1 -- 172.21.15.120:0/1731751748 shutdown_connections 2024-09-19T09:06:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.143+0000 7f93811bc740 1 -- 172.21.15.120:0/1731751748 wait complete. 2024-09-19T09:06:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.143+0000 7f93811bc740 1 -- start start 2024-09-19T09:06:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.143+0000 7f93811bc740 1 Processor -- start 2024-09-19T09:06:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.144+0000 7f93811bc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de833249a0 0x55de83333c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.144+0000 7f93811bc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de833252e0 0x55de83334170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.144+0000 7f93811bc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de8332a370 0x55de83335180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.144+0000 7f93811bc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f93781aa460 con 0x55de833252e0 2024-09-19T09:06:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.144+0000 7f93811bc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f93781a9d90 con 0x55de8332a370 2024-09-19T09:06:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.144+0000 7f93811bc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f93781a6fb0 con 0x55de833249a0 2024-09-19T09:06:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.144+0000 7f93811bc740 10 client.0 _ll_register_callbacks cb 0x7f9370013f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.144+0000 7f93777fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de833252e0 0x55de83334170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.144+0000 7f9377fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de833249a0 0x55de83333c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.144+0000 7f937c9c0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de8332a370 0x55de83335180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.144+0000 7f93777fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de833252e0 0x55de83334170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37256/0 (socket says 172.21.15.120:37256) 2024-09-19T09:06:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.144+0000 7f93777fe640 1 -- 172.21.15.120:0/2668325794 learned_addr learned my addr 172.21.15.120:0/2668325794 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.144+0000 7f9377fff640 1 -- 172.21.15.120:0/2668325794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de8332a370 msgr2=0x55de83335180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.144+0000 7f9377fff640 1 --2- 172.21.15.120:0/2668325794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de8332a370 0x55de83335180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.144+0000 7f9377fff640 1 -- 172.21.15.120:0/2668325794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de833252e0 msgr2=0x55de83334170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.144+0000 7f9377fff640 1 --2- 172.21.15.120:0/2668325794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de833252e0 0x55de83334170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.144+0000 7f9377fff640 1 -- 172.21.15.120:0/2668325794 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de83335790 con 0x55de833249a0 2024-09-19T09:06:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.144+0000 7f93777fe640 1 --2- 172.21.15.120:0/2668325794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de833252e0 0x55de83334170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.144+0000 7f937c9c0640 1 --2- 172.21.15.120:0/2668325794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de8332a370 0x55de83335180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.144+0000 7f9377fff640 1 --2- 172.21.15.120:0/2668325794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de833249a0 0x55de83333c30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f936809c090 tx=0x7f936809c290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a509154bda8880cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.144+0000 7f93577fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:06:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.144+0000 7f93577fe640 1 -- 172.21.15.120:0/2668325794 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9368077040 con 0x55de833249a0 2024-09-19T09:06:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.145+0000 7f93811bc740 1 -- 172.21.15.120:0/2668325794 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f936c04d070 con 0x55de833249a0 2024-09-19T09:06:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.145+0000 7f93577fe640 1 -- 172.21.15.120:0/2668325794 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f9368098a00 con 0x55de833249a0 2024-09-19T09:06:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.145+0000 7f93577fe640 1 -- 172.21.15.120:0/2668325794 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f93680b5140 con 0x55de833249a0 2024-09-19T09:06:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.145+0000 7f93811bc740 10 client.26140 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.145+0000 7f93811bc740 1 -- 172.21.15.120:0/2668325794 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55de833a59b0 con 0x55de833249a0 2024-09-19T09:06:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.145+0000 7f93577fe640 1 -- 172.21.15.120:0/2668325794 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f936c04d070 con 0x55de833249a0 2024-09-19T09:06:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.145+0000 7f93577fe640 1 -- 172.21.15.120:0/2668325794 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f93680d6040 con 0x55de833249a0 2024-09-19T09:06:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.145+0000 7f93811bc740 10 client.26140 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.145+0000 7f93811bc740 10 client.26140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:06:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.145+0000 7f93811bc740 20 client.26140 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.145+0000 7f93811bc740 1 --2- 172.21.15.120:0/2668325794 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55de833a6130 0x55de833a8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.145+0000 7f93811bc740 4 client.26140 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:06:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.145+0000 7f93811bc740 1 -- 172.21.15.120:0/2668325794 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55de8328fd70 con 0x55de833a6130 2024-09-19T09:06:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.145+0000 7f93777fe640 1 --2- 172.21.15.120:0/2668325794 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55de833a6130 0x55de833a8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.146+0000 7f93777fe640 1 --2- 172.21.15.120:0/2668325794 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55de833a6130 0x55de833a8550 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ee7d8c992f676ef server_cookie=11e1d51bbb570db in_seq=0 out_seq=0 2024-09-19T09:06:47.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.146+0000 7f93577fe640 10 client.26140 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:47.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:46 smithi196 ceph-mon[35150]: pgmap v458: 97 pgs: 97 active+clean; 38 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 300 op/s 2024-09-19T09:06:47.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:46 smithi153 ceph-mon[35071]: pgmap v458: 97 pgs: 97 active+clean; 38 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 300 op/s 2024-09-19T09:06:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.274+0000 7f93577fe640 1 -- 172.21.15.120:0/2668325794 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55de8328fd70 con 0x55de833a6130 2024-09-19T09:06:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.274+0000 7f93577fe640 10 client.26140 handle_command_reply: tid=0 2024-09-19T09:06:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.274+0000 7f93811bc740 1 client.26140 shutdown 2024-09-19T09:06:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.274+0000 7f93811bc740 1 -- 172.21.15.120:0/2668325794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de833249a0 msgr2=0x55de83333c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.274+0000 7f93811bc740 1 --2- 172.21.15.120:0/2668325794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de833249a0 0x55de83333c30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f936809c090 tx=0x7f936809c290 comp rx=0 tx=0).stop 2024-09-19T09:06:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.275+0000 7f93811bc740 1 -- 172.21.15.120:0/2668325794 shutdown_connections 2024-09-19T09:06:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.275+0000 7f93811bc740 1 --2- 172.21.15.120:0/2668325794 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55de833a6130 0x55de833a8550 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.275+0000 7f93811bc740 1 --2- 172.21.15.120:0/2668325794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de8332a370 0x55de83335180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.275+0000 7f93811bc740 1 --2- 172.21.15.120:0/2668325794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de833252e0 0x55de83334170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.275+0000 7f93811bc740 1 --2- 172.21.15.120:0/2668325794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de833249a0 0x55de83333c30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.275+0000 7f93577fe640 0 client.26140 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.275+0000 7f93811bc740 1 -- 172.21.15.120:0/2668325794 >> 172.21.15.120:0/2668325794 conn(0x55de83320010 msgr2=0x55de8331f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.275+0000 7f93811bc740 1 -- 172.21.15.120:0/2668325794 shutdown_connections 2024-09-19T09:06:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.275+0000 7f93811bc740 1 -- 172.21.15.120:0/2668325794 wait complete. 2024-09-19T09:06:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.275+0000 7f93811bc740 20 client.26140 trim_cache size 0 max 16384 2024-09-19T09:06:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:06:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:06:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:06:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:06:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.275+0000 7f937e423640 1 -- 172.21.15.120:0/1251208137 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93600a32f0 msgr2=0x7f93600a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.275+0000 7f937e423640 1 --2- 172.21.15.120:0/1251208137 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93600a32f0 0x7f93600a57b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9370009fd0 tx=0x7f9370004ab0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.275+0000 7f937e423640 1 -- 172.21.15.120:0/1251208137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9378102620 msgr2=0x7f9378197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.275+0000 7f937e423640 1 --2- 172.21.15.120:0/1251208137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9378102620 0x7f9378197e70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f936800e9f0 tx=0x7f936800eeb0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.276+0000 7f937e423640 1 -- 172.21.15.120:0/1251208137 shutdown_connections 2024-09-19T09:06:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.276+0000 7f937e423640 1 --2- 172.21.15.120:0/1251208137 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93600a32f0 0x7f93600a57b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.276+0000 7f937e423640 1 --2- 172.21.15.120:0/1251208137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9378107880 0x7f937819f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.276+0000 7f937e423640 1 --2- 172.21.15.120:0/1251208137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9378104f50 0x7f93781983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.276+0000 7f937e423640 1 --2- 172.21.15.120:0/1251208137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9378102620 0x7f9378197e70 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.276+0000 7f937e423640 1 -- 172.21.15.120:0/1251208137 >> 172.21.15.120:0/1251208137 conn(0x7f93780fc1e0 msgr2=0x7f9378102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:47.279 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.276+0000 7f937e423640 1 -- 172.21.15.120:0/1251208137 shutdown_connections 2024-09-19T09:06:47.279 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:06:47.276+0000 7f937e423640 1 -- 172.21.15.120:0/1251208137 wait complete. 2024-09-19T09:06:47.745 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:06:47.994 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:06:48.043 INFO:tasks.workunit.client.0.smithi120.stdout: 1 301931 9.34 MB/sec execute 43 sec latency 76.536 ms 2024-09-19T09:06:48.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.608+0000 7f94e1af1640 1 -- 172.21.15.120:0/337733143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94dc106a70 msgr2=0x7f94dc106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:48.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.608+0000 7f94e1af1640 1 --2- 172.21.15.120:0/337733143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94dc106a70 0x7f94dc106e50 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009930 tx=0x7f94cc02a420 comp rx=0 tx=0).stop 2024-09-19T09:06:48.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.608+0000 7f94e1af1640 1 -- 172.21.15.120:0/337733143 shutdown_connections 2024-09-19T09:06:48.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.608+0000 7f94e1af1640 1 --2- 172.21.15.120:0/337733143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94dc101410 0x7f94dc10d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.608+0000 7f94e1af1640 1 --2- 172.21.15.120:0/337733143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94dc100a70 0x7f94dc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.608+0000 7f94e1af1640 1 --2- 172.21.15.120:0/337733143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94dc106a70 0x7f94dc106e50 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.608+0000 7f94e1af1640 1 -- 172.21.15.120:0/337733143 >> 172.21.15.120:0/337733143 conn(0x7f94dc0fc7d0 msgr2=0x7f94dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:48.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.608+0000 7f94e1af1640 1 -- 172.21.15.120:0/337733143 shutdown_connections 2024-09-19T09:06:48.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.608+0000 7f94e1af1640 1 -- 172.21.15.120:0/337733143 wait complete. 2024-09-19T09:06:48.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.609+0000 7f94e1af1640 1 Processor -- start 2024-09-19T09:06:48.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.609+0000 7f94e1af1640 1 -- start start 2024-09-19T09:06:48.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.610+0000 7f94e1af1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94dc100a70 0x7f94dc1973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:48.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.610+0000 7f94e1af1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94dc101410 0x7f94dc1978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:48.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.610+0000 7f94e1af1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94dc106a70 0x7f94dc19bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:48.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.610+0000 7f94e1af1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc110e70 con 0x7f94dc101410 2024-09-19T09:06:48.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.610+0000 7f94e1af1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc110cf0 con 0x7f94dc106a70 2024-09-19T09:06:48.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.610+0000 7f94e1af1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc110ff0 con 0x7f94dc100a70 2024-09-19T09:06:48.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.610+0000 7f94daffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94dc101410 0x7f94dc1978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:48.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.610+0000 7f94db7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94dc100a70 0x7f94dc1973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:48.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.610+0000 7f94daffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94dc101410 0x7f94dc1978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37278/0 (socket says 172.21.15.120:37278) 2024-09-19T09:06:48.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.610+0000 7f94db7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94dc100a70 0x7f94dc1973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:45576/0 (socket says 172.21.15.120:45576) 2024-09-19T09:06:48.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.610+0000 7f94daffd640 1 -- 172.21.15.120:0/3289962841 learned_addr learned my addr 172.21.15.120:0/3289962841 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:48.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.610+0000 7f94dbfff640 1 --2- 172.21.15.120:0/3289962841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94dc106a70 0x7f94dc19bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:48.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.610+0000 7f94daffd640 1 -- 172.21.15.120:0/3289962841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94dc100a70 msgr2=0x7f94dc1973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:48.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.610+0000 7f94daffd640 1 --2- 172.21.15.120:0/3289962841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94dc100a70 0x7f94dc1973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.610+0000 7f94daffd640 1 -- 172.21.15.120:0/3289962841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94dc106a70 msgr2=0x7f94dc19bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:48.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.610+0000 7f94daffd640 1 --2- 172.21.15.120:0/3289962841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94dc106a70 0x7f94dc19bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.610+0000 7f94daffd640 1 -- 172.21.15.120:0/3289962841 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94dc19c350 con 0x7f94dc101410 2024-09-19T09:06:48.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.610+0000 7f94daffd640 1 --2- 172.21.15.120:0/3289962841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94dc101410 0x7f94dc1978e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f94c400e9f0 tx=0x7f94c400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:48.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.611+0000 7f94d8ff9640 1 -- 172.21.15.120:0/3289962841 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c400cd80 con 0x7f94dc101410 2024-09-19T09:06:48.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.611+0000 7f94e1af1640 1 -- 172.21.15.120:0/3289962841 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94dc19c640 con 0x7f94dc101410 2024-09-19T09:06:48.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.611+0000 7f94d8ff9640 1 -- 172.21.15.120:0/3289962841 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94c4005500 con 0x7f94dc101410 2024-09-19T09:06:48.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.611+0000 7f94d8ff9640 1 -- 172.21.15.120:0/3289962841 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c4010690 con 0x7f94dc101410 2024-09-19T09:06:48.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.611+0000 7f94e1af1640 1 -- 172.21.15.120:0/3289962841 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94dc1a3f20 con 0x7f94dc101410 2024-09-19T09:06:48.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.613+0000 7f94d8ff9640 1 -- 172.21.15.120:0/3289962841 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f94c40046c0 con 0x7f94dc101410 2024-09-19T09:06:48.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.613+0000 7f94e3eca740 1 Processor -- start 2024-09-19T09:06:48.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.613+0000 7f94e3eca740 1 -- start start 2024-09-19T09:06:48.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.614+0000 7f94e3eca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639d8f70b40 0x5639d8f6f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:48.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.614+0000 7f94e3eca740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639d8f6f5d0 0x5639d8f6fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:48.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.614+0000 7f94e3eca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639d8f7c580 0x5639d8f7e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:48.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.614+0000 7f94e3eca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5639d8ed52f0 con 0x5639d8f7c580 2024-09-19T09:06:48.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.614+0000 7f94e3eca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5639d8e7ccf0 con 0x5639d8f70b40 2024-09-19T09:06:48.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.614+0000 7f94e3eca740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5639d8ed12b0 con 0x5639d8f6f5d0 2024-09-19T09:06:48.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.614+0000 7f94dbfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639d8f70b40 0x5639d8f6f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:48.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.614+0000 7f94daffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639d8f7c580 0x5639d8f7e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:48.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.614+0000 7f94dbfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639d8f70b40 0x5639d8f6f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:45652/0 (socket says 172.21.15.120:45652) 2024-09-19T09:06:48.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.614+0000 7f94dbfff640 1 -- 172.21.15.120:0/615609292 learned_addr learned my addr 172.21.15.120:0/615609292 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:48.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.614+0000 7f94db7fe640 1 --2- 172.21.15.120:0/615609292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639d8f6f5d0 0x5639d8f6fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:48.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.614+0000 7f94dbfff640 1 -- 172.21.15.120:0/615609292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639d8f6f5d0 msgr2=0x5639d8f6fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:48.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.614+0000 7f94dbfff640 1 --2- 172.21.15.120:0/615609292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639d8f6f5d0 0x5639d8f6fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.614+0000 7f94dbfff640 1 -- 172.21.15.120:0/615609292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639d8f7c580 msgr2=0x5639d8f7e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:48.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.614+0000 7f94dbfff640 1 --2- 172.21.15.120:0/615609292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639d8f7c580 0x5639d8f7e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.614+0000 7f94dbfff640 1 -- 172.21.15.120:0/615609292 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639d8e613a0 con 0x5639d8f70b40 2024-09-19T09:06:48.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.614+0000 7f94db7fe640 1 --2- 172.21.15.120:0/615609292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639d8f6f5d0 0x5639d8f6fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:48.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.614+0000 7f94dbfff640 1 --2- 172.21.15.120:0/615609292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639d8f70b40 0x5639d8f6f090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f94d4009f50 tx=0x7f94d40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec9f3303dd239002 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:48.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.615+0000 7f94d8ff9640 1 --2- 172.21.15.120:0/3289962841 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f94c80a3240 0x7f94c80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:48.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.615+0000 7f94d1ffb640 1 -- 172.21.15.120:0/615609292 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d4004d20 con 0x5639d8f70b40 2024-09-19T09:06:48.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.615+0000 7f94d1ffb640 1 -- 172.21.15.120:0/615609292 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94d4004050 con 0x5639d8f70b40 2024-09-19T09:06:48.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.615+0000 7f94db7fe640 1 --2- 172.21.15.120:0/3289962841 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f94c80a3240 0x7f94c80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:48.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.615+0000 7f94d8ff9640 1 -- 172.21.15.120:0/3289962841 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f94c40cfd30 con 0x7f94dc101410 2024-09-19T09:06:48.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.615+0000 7f94d1ffb640 1 -- 172.21.15.120:0/615609292 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d4005280 con 0x5639d8f70b40 2024-09-19T09:06:48.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.616+0000 7f94e3eca740 1 -- 172.21.15.120:0/615609292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639d8f70b40 msgr2=0x5639d8f6f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:48.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.616+0000 7f94db7fe640 1 --2- 172.21.15.120:0/3289962841 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f94c80a3240 0x7f94c80a5700 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009900 tx=0x7f94cc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:48.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.616+0000 7f94e3eca740 1 --2- 172.21.15.120:0/615609292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639d8f70b40 0x5639d8f6f090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f94d4009f50 tx=0x7f94d40118d0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.616+0000 7f94e3eca740 1 -- 172.21.15.120:0/615609292 shutdown_connections 2024-09-19T09:06:48.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.616+0000 7f94e3eca740 1 --2- 172.21.15.120:0/615609292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639d8f7c580 0x5639d8f7e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.616+0000 7f94e3eca740 1 --2- 172.21.15.120:0/615609292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639d8f6f5d0 0x5639d8f6fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.616+0000 7f94e3eca740 1 --2- 172.21.15.120:0/615609292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639d8f70b40 0x5639d8f6f090 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.616+0000 7f94e3eca740 1 -- 172.21.15.120:0/615609292 >> 172.21.15.120:0/615609292 conn(0x5639d8f38310 msgr2=0x5639d8f6bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:48.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.616+0000 7f94e3eca740 1 -- 172.21.15.120:0/615609292 shutdown_connections 2024-09-19T09:06:48.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.616+0000 7f94e3eca740 1 -- 172.21.15.120:0/615609292 wait complete. 2024-09-19T09:06:48.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.616+0000 7f94e3eca740 1 -- start start 2024-09-19T09:06:48.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.617+0000 7f94e3eca740 1 Processor -- start 2024-09-19T09:06:48.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.617+0000 7f94e3eca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639d8f6f5d0 0x5639d8f77780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:48.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.617+0000 7f94e3eca740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639d8f70b40 0x5639d8f77cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:48.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.617+0000 7f94e3eca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639d8f7c580 0x5639d8f8a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:48.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.617+0000 7f94e3eca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5639d8f72730 con 0x5639d8f7c580 2024-09-19T09:06:48.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.617+0000 7f94e3eca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5639d8e90060 con 0x5639d8f6f5d0 2024-09-19T09:06:48.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.617+0000 7f94e3eca740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5639d8f728b0 con 0x5639d8f70b40 2024-09-19T09:06:48.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.617+0000 7f94e3eca740 10 client.0 _ll_register_callbacks cb 0x5639d8be38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:48.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.617+0000 7f94dbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639d8f7c580 0x5639d8f8a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:48.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.617+0000 7f94db7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639d8f6f5d0 0x5639d8f77780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:48.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.617+0000 7f94daffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639d8f70b40 0x5639d8f77cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:48.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.617+0000 7f94db7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639d8f6f5d0 0x5639d8f77780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:45668/0 (socket says 172.21.15.120:45668) 2024-09-19T09:06:48.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.617+0000 7f94db7fe640 1 -- 172.21.15.120:0/344166942 learned_addr learned my addr 172.21.15.120:0/344166942 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:48.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.617+0000 7f94dbfff640 1 -- 172.21.15.120:0/344166942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639d8f70b40 msgr2=0x5639d8f77cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:48.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.617+0000 7f94dbfff640 1 --2- 172.21.15.120:0/344166942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639d8f70b40 0x5639d8f77cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.617+0000 7f94dbfff640 1 -- 172.21.15.120:0/344166942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639d8f6f5d0 msgr2=0x5639d8f77780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:48.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.617+0000 7f94dbfff640 1 --2- 172.21.15.120:0/344166942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639d8f6f5d0 0x5639d8f77780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.617+0000 7f94dbfff640 1 -- 172.21.15.120:0/344166942 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639d8c463c0 con 0x5639d8f7c580 2024-09-19T09:06:48.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.617+0000 7f94daffd640 1 --2- 172.21.15.120:0/344166942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639d8f70b40 0x5639d8f77cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:48.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.618+0000 7f94dbfff640 1 --2- 172.21.15.120:0/344166942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639d8f7c580 0x5639d8f8a350 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f94d4012fd0 tx=0x7f94d4004cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26fb5359e087c5e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:48.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.618+0000 7f94baffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:06:48.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.618+0000 7f94baffd640 1 -- 172.21.15.120:0/344166942 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d4009be0 con 0x5639d8f7c580 2024-09-19T09:06:48.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.618+0000 7f94e3eca740 1 -- 172.21.15.120:0/344166942 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94d4004050 con 0x5639d8f7c580 2024-09-19T09:06:48.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.618+0000 7f94baffd640 1 -- 172.21.15.120:0/344166942 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f94d400f480 con 0x5639d8f7c580 2024-09-19T09:06:48.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.618+0000 7f94baffd640 1 -- 172.21.15.120:0/344166942 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f94d4003e40 con 0x5639d8f7c580 2024-09-19T09:06:48.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.618+0000 7f94e3eca740 10 client.16866 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:48.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.618+0000 7f94e3eca740 1 -- 172.21.15.120:0/344166942 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5639d8f8cec0 con 0x5639d8f7c580 2024-09-19T09:06:48.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.618+0000 7f94baffd640 1 -- 172.21.15.120:0/344166942 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94d4004050 con 0x5639d8f7c580 2024-09-19T09:06:48.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.618+0000 7f94baffd640 1 -- 172.21.15.120:0/344166942 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f94d402f770 con 0x5639d8f7c580 2024-09-19T09:06:48.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.618+0000 7f94e3eca740 10 client.16866 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:48.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.618+0000 7f94e3eca740 10 client.16866 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:06:48.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.618+0000 7f94e3eca740 20 client.16866 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:48.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.619+0000 7f94e3eca740 1 --2- 172.21.15.120:0/344166942 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5639d8f8e2c0 0x5639d8f906b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:48.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.619+0000 7f94e3eca740 4 client.16866 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:06:48.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.619+0000 7f94e3eca740 1 -- 172.21.15.120:0/344166942 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639d8ee2860 con 0x5639d8f8e2c0 2024-09-19T09:06:48.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.619+0000 7f94db7fe640 1 --2- 172.21.15.120:0/344166942 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5639d8f8e2c0 0x5639d8f906b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:48.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.619+0000 7f94db7fe640 1 --2- 172.21.15.120:0/344166942 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5639d8f8e2c0 0x5639d8f906b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1476fad0ccbdca0f server_cookie=b0accedd7fd1e6f1 in_seq=0 out_seq=0 2024-09-19T09:06:48.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.619+0000 7f94baffd640 10 client.16866 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:06:48.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.620+0000 7f94baffd640 1 -- 172.21.15.120:0/344166942 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5639d8ee2860 con 0x5639d8f8e2c0 2024-09-19T09:06:48.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.620+0000 7f94baffd640 10 client.16866 handle_command_reply: tid=0 2024-09-19T09:06:48.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.620+0000 7f94e3eca740 1 client.16866 shutdown 2024-09-19T09:06:48.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.620+0000 7f94e3eca740 1 -- 172.21.15.120:0/344166942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639d8f7c580 msgr2=0x5639d8f8a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:48.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.620+0000 7f94e3eca740 1 --2- 172.21.15.120:0/344166942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639d8f7c580 0x5639d8f8a350 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f94d4012fd0 tx=0x7f94d4004cb0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.620+0000 7f94e3eca740 1 -- 172.21.15.120:0/344166942 shutdown_connections 2024-09-19T09:06:48.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.620+0000 7f94e3eca740 1 --2- 172.21.15.120:0/344166942 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5639d8f8e2c0 0x5639d8f906b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.620+0000 7f94e3eca740 1 --2- 172.21.15.120:0/344166942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639d8f7c580 0x5639d8f8a350 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.620+0000 7f94baffd640 0 client.16866 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:06:48.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.620+0000 7f94e3eca740 1 --2- 172.21.15.120:0/344166942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639d8f70b40 0x5639d8f77cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.620+0000 7f94e3eca740 1 --2- 172.21.15.120:0/344166942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639d8f6f5d0 0x5639d8f77780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.620+0000 7f94e3eca740 1 -- 172.21.15.120:0/344166942 >> 172.21.15.120:0/344166942 conn(0x5639d8f38310 msgr2=0x5639d8f71930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:48.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.620+0000 7f94e3eca740 1 -- 172.21.15.120:0/344166942 shutdown_connections 2024-09-19T09:06:48.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.620+0000 7f94e3eca740 1 -- 172.21.15.120:0/344166942 wait complete. 2024-09-19T09:06:48.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.620+0000 7f94e3eca740 20 client.16866 trim_cache size 0 max 16384 2024-09-19T09:06:48.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.634+0000 7f94e3eca740 1 Processor -- start 2024-09-19T09:06:48.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.634+0000 7f94e3eca740 1 -- start start 2024-09-19T09:06:48.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.634+0000 7f94e3eca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639d8f6f5d0 0x5639d8f76760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:48.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.634+0000 7f94e3eca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639d8f70b40 0x5639d8f8a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:48.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.634+0000 7f94e3eca740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639d8f7c580 0x5639d8f95e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:48.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.634+0000 7f94e3eca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc1ab440 con 0x5639d8f6f5d0 2024-09-19T09:06:48.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.634+0000 7f94e3eca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc1acf80 con 0x5639d8f70b40 2024-09-19T09:06:48.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.634+0000 7f94e3eca740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc1abb10 con 0x5639d8f7c580 2024-09-19T09:06:48.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.634+0000 7f94db7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639d8f6f5d0 0x5639d8f76760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:48.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.634+0000 7f94daffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639d8f70b40 0x5639d8f8a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:48.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.634+0000 7f94dbfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639d8f7c580 0x5639d8f95e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:48.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.634+0000 7f94db7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639d8f6f5d0 0x5639d8f76760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37300/0 (socket says 172.21.15.120:37300) 2024-09-19T09:06:48.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.634+0000 7f94db7fe640 1 -- 172.21.15.120:0/502721815 learned_addr learned my addr 172.21.15.120:0/502721815 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:48.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.634+0000 7f94dbfff640 1 -- 172.21.15.120:0/502721815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639d8f70b40 msgr2=0x5639d8f8a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:48.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.634+0000 7f94dbfff640 1 --2- 172.21.15.120:0/502721815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639d8f70b40 0x5639d8f8a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.635+0000 7f94dbfff640 1 -- 172.21.15.120:0/502721815 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639d8f6f5d0 msgr2=0x5639d8f76760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:48.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.635+0000 7f94dbfff640 1 --2- 172.21.15.120:0/502721815 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639d8f6f5d0 0x5639d8f76760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.635+0000 7f94dbfff640 1 -- 172.21.15.120:0/502721815 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639d8f72950 con 0x5639d8f7c580 2024-09-19T09:06:48.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.635+0000 7f94daffd640 1 --2- 172.21.15.120:0/502721815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639d8f70b40 0x5639d8f8a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:48.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.635+0000 7f94dbfff640 1 --2- 172.21.15.120:0/502721815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639d8f7c580 0x5639d8f95e20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f94d4011910 tx=0x7f94d4005cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec9063fac4e9dab3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:48.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.635+0000 7f94d17fa640 1 -- 172.21.15.120:0/502721815 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d40047f0 con 0x5639d8f7c580 2024-09-19T09:06:48.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.635+0000 7f94d17fa640 1 -- 172.21.15.120:0/502721815 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94d4003fe0 con 0x5639d8f7c580 2024-09-19T09:06:48.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.635+0000 7f94d17fa640 1 -- 172.21.15.120:0/502721815 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d40199e0 con 0x5639d8f7c580 2024-09-19T09:06:48.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.636+0000 7f94e3eca740 1 -- 172.21.15.120:0/502721815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639d8f7c580 msgr2=0x5639d8f95e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:48.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.636+0000 7f94e3eca740 1 --2- 172.21.15.120:0/502721815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639d8f7c580 0x5639d8f95e20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f94d4011910 tx=0x7f94d4005cf0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.636+0000 7f94e3eca740 1 -- 172.21.15.120:0/502721815 shutdown_connections 2024-09-19T09:06:48.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.636+0000 7f94e3eca740 1 --2- 172.21.15.120:0/502721815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639d8f7c580 0x5639d8f95e20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.636+0000 7f94e3eca740 1 --2- 172.21.15.120:0/502721815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639d8f70b40 0x5639d8f8a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.636+0000 7f94e3eca740 1 --2- 172.21.15.120:0/502721815 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639d8f6f5d0 0x5639d8f76760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.636+0000 7f94e3eca740 1 -- 172.21.15.120:0/502721815 >> 172.21.15.120:0/502721815 conn(0x5639d8f38310 msgr2=0x5639d8f76050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:48.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.636+0000 7f94e3eca740 1 -- 172.21.15.120:0/502721815 shutdown_connections 2024-09-19T09:06:48.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.636+0000 7f94e3eca740 1 -- 172.21.15.120:0/502721815 wait complete. 2024-09-19T09:06:48.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.636+0000 7f94e3eca740 1 -- start start 2024-09-19T09:06:48.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.637+0000 7f94e3eca740 1 Processor -- start 2024-09-19T09:06:48.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.637+0000 7f94e3eca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639d8f6f5d0 0x5639d8f8a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:48.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.637+0000 7f94e3eca740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639d8f70b40 0x5639d8f853e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:48.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.637+0000 7f94e3eca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639d8f7c580 0x5639d8f85920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:48.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.637+0000 7f94e3eca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc1ac8b0 con 0x5639d8f7c580 2024-09-19T09:06:48.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.637+0000 7f94e3eca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc1ac1e0 con 0x5639d8f6f5d0 2024-09-19T09:06:48.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.637+0000 7f94e3eca740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc1a9400 con 0x5639d8f70b40 2024-09-19T09:06:48.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.637+0000 7f94e3eca740 10 client.0 _ll_register_callbacks cb 0x7f94c40d3640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:48.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.637+0000 7f94db7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639d8f6f5d0 0x5639d8f8a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:48.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.637+0000 7f94dbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639d8f7c580 0x5639d8f85920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:48.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.637+0000 7f94db7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639d8f6f5d0 0x5639d8f8a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:45688/0 (socket says 172.21.15.120:45688) 2024-09-19T09:06:48.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.637+0000 7f94db7fe640 1 -- 172.21.15.120:0/519565563 learned_addr learned my addr 172.21.15.120:0/519565563 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:48.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.637+0000 7f94daffd640 1 --2- 172.21.15.120:0/519565563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639d8f70b40 0x5639d8f853e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:48.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.638+0000 7f94db7fe640 1 -- 172.21.15.120:0/519565563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639d8f70b40 msgr2=0x5639d8f853e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:48.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.638+0000 7f94db7fe640 1 --2- 172.21.15.120:0/519565563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639d8f70b40 0x5639d8f853e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.638+0000 7f94db7fe640 1 -- 172.21.15.120:0/519565563 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639d8f7c580 msgr2=0x5639d8f85920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:48.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.638+0000 7f94db7fe640 1 --2- 172.21.15.120:0/519565563 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639d8f7c580 0x5639d8f85920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.638+0000 7f94db7fe640 1 -- 172.21.15.120:0/519565563 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639d8f86100 con 0x5639d8f6f5d0 2024-09-19T09:06:48.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.638+0000 7f94db7fe640 1 --2- 172.21.15.120:0/519565563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639d8f6f5d0 0x5639d8f8a0b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f94cc006010 tx=0x7f94cc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faeb7939871d4aa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:48.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.638+0000 7f94baffd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:06:48.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.638+0000 7f94baffd640 1 -- 172.21.15.120:0/519565563 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc035040 con 0x5639d8f6f5d0 2024-09-19T09:06:48.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.638+0000 7f94e3eca740 1 -- 172.21.15.120:0/519565563 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94d4003fe0 con 0x5639d8f6f5d0 2024-09-19T09:06:48.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.638+0000 7f94baffd640 1 -- 172.21.15.120:0/519565563 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f94cc003a00 con 0x5639d8f6f5d0 2024-09-19T09:06:48.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.638+0000 7f94baffd640 1 -- 172.21.15.120:0/519565563 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f94cc003c00 con 0x5639d8f6f5d0 2024-09-19T09:06:48.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.638+0000 7f94e3eca740 10 client.26216 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:48.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.638+0000 7f94e3eca740 1 -- 172.21.15.120:0/519565563 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5639d8ff8da0 con 0x5639d8f6f5d0 2024-09-19T09:06:48.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.639+0000 7f94baffd640 1 -- 172.21.15.120:0/519565563 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94d4003fe0 con 0x5639d8f6f5d0 2024-09-19T09:06:48.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.639+0000 7f94baffd640 1 -- 172.21.15.120:0/519565563 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f94cc05d500 con 0x5639d8f6f5d0 2024-09-19T09:06:48.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.639+0000 7f94e3eca740 10 client.26216 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:48.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.639+0000 7f94e3eca740 10 client.26216 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:06:48.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.639+0000 7f94e3eca740 20 client.26216 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:48.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.639+0000 7f94e3eca740 1 --2- 172.21.15.120:0/519565563 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5639d8f8e2c0 0x5639d8ffb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:48.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.639+0000 7f94e3eca740 4 client.26216 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:06:48.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.639+0000 7f94e3eca740 1 -- 172.21.15.120:0/519565563 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5639d8edcd70 con 0x5639d8f8e2c0 2024-09-19T09:06:48.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.639+0000 7f94daffd640 1 --2- 172.21.15.120:0/519565563 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5639d8f8e2c0 0x5639d8ffb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:48.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.639+0000 7f94baffd640 10 client.26216 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:06:48.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.639+0000 7f94daffd640 1 --2- 172.21.15.120:0/519565563 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5639d8f8e2c0 0x5639d8ffb490 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=94d9024292e512f2 server_cookie=562c7ff02e566c03 in_seq=0 out_seq=0 2024-09-19T09:06:48.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.640+0000 7f94baffd640 1 -- 172.21.15.120:0/519565563 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5639d8edcd70 con 0x5639d8f8e2c0 2024-09-19T09:06:48.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.640+0000 7f94baffd640 10 client.26216 handle_command_reply: tid=0 2024-09-19T09:06:48.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.640+0000 7f94e3eca740 1 client.26216 shutdown 2024-09-19T09:06:48.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.640+0000 7f94e3eca740 1 -- 172.21.15.120:0/519565563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639d8f6f5d0 msgr2=0x5639d8f8a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:48.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.640+0000 7f94e3eca740 1 --2- 172.21.15.120:0/519565563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639d8f6f5d0 0x5639d8f8a0b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f94cc006010 tx=0x7f94cc009260 comp rx=0 tx=0).stop 2024-09-19T09:06:48.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.640+0000 7f94e3eca740 1 -- 172.21.15.120:0/519565563 shutdown_connections 2024-09-19T09:06:48.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.640+0000 7f94e3eca740 1 --2- 172.21.15.120:0/519565563 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5639d8f8e2c0 0x5639d8ffb490 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.640+0000 7f94e3eca740 1 --2- 172.21.15.120:0/519565563 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639d8f7c580 0x5639d8f85920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.640+0000 7f94baffd640 0 client.26216 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:06:48.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.640+0000 7f94e3eca740 1 --2- 172.21.15.120:0/519565563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639d8f70b40 0x5639d8f853e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.640+0000 7f94e3eca740 1 --2- 172.21.15.120:0/519565563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639d8f6f5d0 0x5639d8f8a0b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.640+0000 7f94e3eca740 1 -- 172.21.15.120:0/519565563 >> 172.21.15.120:0/519565563 conn(0x5639d8f38310 msgr2=0x5639d8f76050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:48.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.640+0000 7f94e3eca740 1 -- 172.21.15.120:0/519565563 shutdown_connections 2024-09-19T09:06:48.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.640+0000 7f94e3eca740 1 -- 172.21.15.120:0/519565563 wait complete. 2024-09-19T09:06:48.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.640+0000 7f94e3eca740 20 client.26216 trim_cache size 0 max 16384 2024-09-19T09:06:48.645 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:06:48.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.641+0000 7f94e1af1640 1 -- 172.21.15.120:0/3289962841 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f94c80a3240 msgr2=0x7f94c80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:48.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.641+0000 7f94e1af1640 1 --2- 172.21.15.120:0/3289962841 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f94c80a3240 0x7f94c80a5700 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009900 tx=0x7f94cc0058d0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.641+0000 7f94e1af1640 1 -- 172.21.15.120:0/3289962841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94dc101410 msgr2=0x7f94dc1978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:48.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.641+0000 7f94e1af1640 1 --2- 172.21.15.120:0/3289962841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94dc101410 0x7f94dc1978e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f94c400e9f0 tx=0x7f94c400edf0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.641+0000 7f94e1af1640 1 -- 172.21.15.120:0/3289962841 shutdown_connections 2024-09-19T09:06:48.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.641+0000 7f94e1af1640 1 --2- 172.21.15.120:0/3289962841 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f94c80a3240 0x7f94c80a5700 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.641+0000 7f94e1af1640 1 --2- 172.21.15.120:0/3289962841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94dc106a70 0x7f94dc19bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.641+0000 7f94e1af1640 1 --2- 172.21.15.120:0/3289962841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94dc101410 0x7f94dc1978e0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.641+0000 7f94e1af1640 1 --2- 172.21.15.120:0/3289962841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94dc100a70 0x7f94dc1973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:48.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.641+0000 7f94e1af1640 1 -- 172.21.15.120:0/3289962841 >> 172.21.15.120:0/3289962841 conn(0x7f94dc0fc7d0 msgr2=0x7f94dc0fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:48.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.641+0000 7f94e1af1640 1 -- 172.21.15.120:0/3289962841 shutdown_connections 2024-09-19T09:06:48.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:48.641+0000 7f94e1af1640 1 -- 172.21.15.120:0/3289962841 wait complete. 2024-09-19T09:06:49.044 INFO:tasks.workunit.client.0.smithi120.stdout: 1 305153 9.53 MB/sec execute 44 sec latency 39.523 ms 2024-09-19T09:06:49.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:48 smithi120 ceph-mon[31150]: pgmap v459: 97 pgs: 97 active+clean; 38 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 339 op/s 2024-09-19T09:06:49.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:48 smithi196 ceph-mon[35150]: pgmap v459: 97 pgs: 97 active+clean; 38 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 339 op/s 2024-09-19T09:06:49.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:48 smithi153 ceph-mon[35071]: pgmap v459: 97 pgs: 97 active+clean; 38 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 339 op/s 2024-09-19T09:06:49.425 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:06:49.693 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:06:50.044 INFO:tasks.workunit.client.0.smithi120.stdout: 1 307943 9.60 MB/sec execute 45 sec latency 23.462 ms 2024-09-19T09:06:50.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.218+0000 7f8c22d0d640 1 -- 172.21.15.120:0/2443345830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c1c102790 msgr2=0x7f8c1c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:50.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.218+0000 7f8c22d0d640 1 --2- 172.21.15.120:0/2443345830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c1c102790 0x7f8c1c1067e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8c08009900 tx=0x7f8c0802a410 comp rx=0 tx=0).stop 2024-09-19T09:06:50.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.219+0000 7f8c22d0d640 1 -- 172.21.15.120:0/2443345830 shutdown_connections 2024-09-19T09:06:50.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.219+0000 7f8c22d0d640 1 --2- 172.21.15.120:0/2443345830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c1c106f10 0x7f8c1c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.219+0000 7f8c22d0d640 1 --2- 172.21.15.120:0/2443345830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c1c102790 0x7f8c1c1067e0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.219+0000 7f8c22d0d640 1 --2- 172.21.15.120:0/2443345830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c1c101de0 0x7f8c1c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.219+0000 7f8c22d0d640 1 -- 172.21.15.120:0/2443345830 >> 172.21.15.120:0/2443345830 conn(0x7f8c1c0fd8c0 msgr2=0x7f8c1c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:50.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.219+0000 7f8c22d0d640 1 -- 172.21.15.120:0/2443345830 shutdown_connections 2024-09-19T09:06:50.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.219+0000 7f8c22d0d640 1 -- 172.21.15.120:0/2443345830 wait complete. 2024-09-19T09:06:50.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.219+0000 7f8c22d0d640 1 Processor -- start 2024-09-19T09:06:50.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.219+0000 7f8c22d0d640 1 -- start start 2024-09-19T09:06:50.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.219+0000 7f8c22d0d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c1c101de0 0x7f8c1c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:50.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.220+0000 7f8c22d0d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c1c102790 0x7f8c1c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:50.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.220+0000 7f8c22d0d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c1c106f10 0x7f8c1c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:50.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.220+0000 7f8c22d0d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c111f70 con 0x7f8c1c102790 2024-09-19T09:06:50.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.220+0000 7f8c22d0d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c111df0 con 0x7f8c1c101de0 2024-09-19T09:06:50.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.220+0000 7f8c22d0d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1120f0 con 0x7f8c1c106f10 2024-09-19T09:06:50.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.220+0000 7f8c1bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c1c102790 0x7f8c1c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:50.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.220+0000 7f8c1bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c1c102790 0x7f8c1c1988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37336/0 (socket says 172.21.15.120:37336) 2024-09-19T09:06:50.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.220+0000 7f8c20aa9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c1c101de0 0x7f8c1c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:50.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.220+0000 7f8c1bfff640 1 -- 172.21.15.120:0/1812959202 learned_addr learned my addr 172.21.15.120:0/1812959202 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:50.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.220+0000 7f8c212aa640 1 --2- 172.21.15.120:0/1812959202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c1c106f10 0x7f8c1c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:50.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.220+0000 7f8c20aa9640 1 -- 172.21.15.120:0/1812959202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c1c106f10 msgr2=0x7f8c1c19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:50.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.220+0000 7f8c20aa9640 1 --2- 172.21.15.120:0/1812959202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c1c106f10 0x7f8c1c19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.220+0000 7f8c20aa9640 1 -- 172.21.15.120:0/1812959202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c1c102790 msgr2=0x7f8c1c1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:50.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.220+0000 7f8c20aa9640 1 --2- 172.21.15.120:0/1812959202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c1c102790 0x7f8c1c1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.220+0000 7f8c20aa9640 1 -- 172.21.15.120:0/1812959202 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1c19d360 con 0x7f8c1c101de0 2024-09-19T09:06:50.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.220+0000 7f8c1bfff640 1 --2- 172.21.15.120:0/1812959202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c1c102790 0x7f8c1c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:50.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.220+0000 7f8c212aa640 1 --2- 172.21.15.120:0/1812959202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c1c106f10 0x7f8c1c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:50.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.220+0000 7f8c20aa9640 1 --2- 172.21.15.120:0/1812959202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c1c101de0 0x7f8c1c1983b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8c1000e9f0 tx=0x7f8c1000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:50.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.220+0000 7f8c19ffb640 1 -- 172.21.15.120:0/1812959202 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1000ccc0 con 0x7f8c1c101de0 2024-09-19T09:06:50.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.221+0000 7f8c22d0d640 1 -- 172.21.15.120:0/1812959202 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c1c19d650 con 0x7f8c1c101de0 2024-09-19T09:06:50.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.221+0000 7f8c19ffb640 1 -- 172.21.15.120:0/1812959202 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c10010910 con 0x7f8c1c101de0 2024-09-19T09:06:50.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.221+0000 7f8c19ffb640 1 -- 172.21.15.120:0/1812959202 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c10018960 con 0x7f8c1c101de0 2024-09-19T09:06:50.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.221+0000 7f8c22d0d640 1 -- 172.21.15.120:0/1812959202 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c1c1a4f30 con 0x7f8c1c101de0 2024-09-19T09:06:50.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.224+0000 7f8c19ffb640 1 -- 172.21.15.120:0/1812959202 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f8c10010ac0 con 0x7f8c1c101de0 2024-09-19T09:06:50.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.224+0000 7f8c250e6740 1 Processor -- start 2024-09-19T09:06:50.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.224+0000 7f8c250e6740 1 -- start start 2024-09-19T09:06:50.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.224+0000 7f8c250e6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd08f84b40 0x55bd08f83090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:50.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.224+0000 7f8c250e6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd08f835d0 0x55bd08f83a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:50.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.224+0000 7f8c250e6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd08f8e370 0x55bd08f90760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:50.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.224+0000 7f8c250e6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bd08ee92f0 con 0x55bd08f835d0 2024-09-19T09:06:50.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.224+0000 7f8c250e6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bd08e90cf0 con 0x55bd08f8e370 2024-09-19T09:06:50.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.224+0000 7f8c250e6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bd08ee52b0 con 0x55bd08f84b40 2024-09-19T09:06:50.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.224+0000 7f8c212aa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd08f84b40 0x55bd08f83090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:50.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.224+0000 7f8c20aa9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd08f835d0 0x55bd08f83a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:50.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.224+0000 7f8c212aa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd08f84b40 0x55bd08f83090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:45648/0 (socket says 172.21.15.120:45648) 2024-09-19T09:06:50.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.224+0000 7f8c212aa640 1 -- 172.21.15.120:0/597288233 learned_addr learned my addr 172.21.15.120:0/597288233 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:50.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.224+0000 7f8c1bfff640 1 --2- 172.21.15.120:0/597288233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd08f8e370 0x55bd08f90760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:50.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.224+0000 7f8c212aa640 1 -- 172.21.15.120:0/597288233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd08f8e370 msgr2=0x55bd08f90760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:50.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.224+0000 7f8c212aa640 1 --2- 172.21.15.120:0/597288233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd08f8e370 0x55bd08f90760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.224+0000 7f8c212aa640 1 -- 172.21.15.120:0/597288233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd08f835d0 msgr2=0x55bd08f83a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:50.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.224+0000 7f8c212aa640 1 --2- 172.21.15.120:0/597288233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd08f835d0 0x55bd08f83a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.224+0000 7f8c212aa640 1 -- 172.21.15.120:0/597288233 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd08e753a0 con 0x55bd08f84b40 2024-09-19T09:06:50.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.224+0000 7f8c19ffb640 1 --2- 172.21.15.120:0/1812959202 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8c0c0a3240 0x7f8c0c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:50.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.225+0000 7f8c212aa640 1 --2- 172.21.15.120:0/597288233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd08f84b40 0x55bd08f83090 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8c1400f790 tx=0x7f8c1400fc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24de0ff24a5a1b11 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:50.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.225+0000 7f8c20aa9640 1 --2- 172.21.15.120:0/1812959202 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8c0c0a3240 0x7f8c0c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:50.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.225+0000 7f8c06ffd640 1 -- 172.21.15.120:0/597288233 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c14004d20 con 0x55bd08f84b40 2024-09-19T09:06:50.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.225+0000 7f8c06ffd640 1 -- 172.21.15.120:0/597288233 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c1400de00 con 0x55bd08f84b40 2024-09-19T09:06:50.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.225+0000 7f8c06ffd640 1 -- 172.21.15.120:0/597288233 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c14010430 con 0x55bd08f84b40 2024-09-19T09:06:50.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.225+0000 7f8c19ffb640 1 -- 172.21.15.120:0/1812959202 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f8c100cfca0 con 0x7f8c1c101de0 2024-09-19T09:06:50.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.225+0000 7f8c250e6740 1 -- 172.21.15.120:0/597288233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd08f84b40 msgr2=0x55bd08f83090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:50.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.225+0000 7f8c20aa9640 1 --2- 172.21.15.120:0/1812959202 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8c0c0a3240 0x7f8c0c0a5700 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8c1001e000 tx=0x7f8c100d3720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:50.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.225+0000 7f8c250e6740 1 --2- 172.21.15.120:0/597288233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd08f84b40 0x55bd08f83090 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8c1400f790 tx=0x7f8c1400fc50 comp rx=0 tx=0).stop 2024-09-19T09:06:50.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.226+0000 7f8c250e6740 1 -- 172.21.15.120:0/597288233 shutdown_connections 2024-09-19T09:06:50.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.226+0000 7f8c250e6740 1 --2- 172.21.15.120:0/597288233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd08f8e370 0x55bd08f90760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.226+0000 7f8c250e6740 1 --2- 172.21.15.120:0/597288233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd08f835d0 0x55bd08f83a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.226+0000 7f8c250e6740 1 --2- 172.21.15.120:0/597288233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd08f84b40 0x55bd08f83090 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.226+0000 7f8c250e6740 1 -- 172.21.15.120:0/597288233 >> 172.21.15.120:0/597288233 conn(0x55bd08f4c310 msgr2=0x55bd08f7fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:50.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.226+0000 7f8c250e6740 1 -- 172.21.15.120:0/597288233 shutdown_connections 2024-09-19T09:06:50.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.226+0000 7f8c250e6740 1 -- 172.21.15.120:0/597288233 wait complete. 2024-09-19T09:06:50.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.226+0000 7f8c250e6740 1 -- start start 2024-09-19T09:06:50.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.226+0000 7f8c250e6740 1 Processor -- start 2024-09-19T09:06:50.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.226+0000 7f8c250e6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd08f835d0 0x55bd08f8dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:50.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.226+0000 7f8c250e6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd08f84b40 0x55bd08f8afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:50.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.226+0000 7f8c250e6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd08f8e370 0x55bd08f8b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:50.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.226+0000 7f8c250e6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bd08f91070 con 0x55bd08f8e370 2024-09-19T09:06:50.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.226+0000 7f8c250e6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bd08ea4060 con 0x55bd08f84b40 2024-09-19T09:06:50.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.226+0000 7f8c250e6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bd08f911f0 con 0x55bd08f835d0 2024-09-19T09:06:50.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.227+0000 7f8c250e6740 10 client.0 _ll_register_callbacks cb 0x55bd08bf78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:50.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.227+0000 7f8c212aa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd08f8e370 0x55bd08f8b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:50.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.227+0000 7f8c1bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd08f84b40 0x55bd08f8afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:50.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.227+0000 7f8c212aa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd08f8e370 0x55bd08f8b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37348/0 (socket says 172.21.15.120:37348) 2024-09-19T09:06:50.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.227+0000 7f8c212aa640 1 -- 172.21.15.120:0/2112397545 learned_addr learned my addr 172.21.15.120:0/2112397545 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:50.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.227+0000 7f8c212aa640 1 --2- 172.21.15.120:0/2112397545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd08f835d0 0x55bd08f8dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:50.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.227+0000 7f8c212aa640 1 -- 172.21.15.120:0/2112397545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd08f835d0 msgr2=0x55bd08f8dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:50.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.227+0000 7f8c212aa640 1 --2- 172.21.15.120:0/2112397545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd08f835d0 0x55bd08f8dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.227+0000 7f8c212aa640 1 -- 172.21.15.120:0/2112397545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd08f84b40 msgr2=0x55bd08f8afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:50.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.227+0000 7f8c212aa640 1 --2- 172.21.15.120:0/2112397545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd08f84b40 0x55bd08f8afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.227+0000 7f8c212aa640 1 -- 172.21.15.120:0/2112397545 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd08c5a3c0 con 0x55bd08f8e370 2024-09-19T09:06:50.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.227+0000 7f8c1bfff640 1 --2- 172.21.15.120:0/2112397545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd08f84b40 0x55bd08f8afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:50.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.227+0000 7f8c212aa640 1 --2- 172.21.15.120:0/2112397545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd08f8e370 0x55bd08f8b530 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f8c140054b0 tx=0x7f8c14005650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c0077b15587365 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:50.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.227+0000 7f8bfbfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:06:50.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.227+0000 7f8bfbfff640 1 -- 172.21.15.120:0/2112397545 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c14004c50 con 0x55bd08f8e370 2024-09-19T09:06:50.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.227+0000 7f8bfbfff640 1 -- 172.21.15.120:0/2112397545 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f8c14004df0 con 0x55bd08f8e370 2024-09-19T09:06:50.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.227+0000 7f8c250e6740 1 -- 172.21.15.120:0/2112397545 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c1400de00 con 0x55bd08f8e370 2024-09-19T09:06:50.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.227+0000 7f8bfbfff640 1 -- 172.21.15.120:0/2112397545 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8c14004ff0 con 0x55bd08f8e370 2024-09-19T09:06:50.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.227+0000 7f8c250e6740 10 client.16884 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:50.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.227+0000 7f8c250e6740 1 -- 172.21.15.120:0/2112397545 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bd08f9ca80 con 0x55bd08f8e370 2024-09-19T09:06:50.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.228+0000 7f8bfbfff640 1 -- 172.21.15.120:0/2112397545 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c1400de00 con 0x55bd08f8e370 2024-09-19T09:06:50.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.228+0000 7f8bfbfff640 1 -- 172.21.15.120:0/2112397545 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8c14004400 con 0x55bd08f8e370 2024-09-19T09:06:50.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.228+0000 7f8c250e6740 10 client.16884 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:50.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.228+0000 7f8c250e6740 10 client.16884 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:06:50.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.228+0000 7f8c250e6740 20 client.16884 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:50.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.228+0000 7f8c250e6740 1 --2- 172.21.15.120:0/2112397545 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55bd08f9de80 0x55bd08fa0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:50.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.228+0000 7f8c250e6740 4 client.16884 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:06:50.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.228+0000 7f8c250e6740 1 -- 172.21.15.120:0/2112397545 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd08ef6860 con 0x55bd08f9de80 2024-09-19T09:06:50.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.228+0000 7f8c1bfff640 1 --2- 172.21.15.120:0/2112397545 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55bd08f9de80 0x55bd08fa0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:50.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.228+0000 7f8bfbfff640 10 client.16884 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:06:50.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.228+0000 7f8c1bfff640 1 --2- 172.21.15.120:0/2112397545 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55bd08f9de80 0x55bd08fa0270 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=55f2309d46e404c9 server_cookie=3ca345e913f9737f in_seq=0 out_seq=0 2024-09-19T09:06:50.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.229+0000 7f8bfbfff640 1 -- 172.21.15.120:0/2112397545 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bd08ef6860 con 0x55bd08f9de80 2024-09-19T09:06:50.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.229+0000 7f8bfbfff640 10 client.16884 handle_command_reply: tid=0 2024-09-19T09:06:50.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.229+0000 7f8c250e6740 1 client.16884 shutdown 2024-09-19T09:06:50.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.229+0000 7f8c250e6740 1 -- 172.21.15.120:0/2112397545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd08f8e370 msgr2=0x55bd08f8b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:50.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.229+0000 7f8c250e6740 1 --2- 172.21.15.120:0/2112397545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd08f8e370 0x55bd08f8b530 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f8c140054b0 tx=0x7f8c14005650 comp rx=0 tx=0).stop 2024-09-19T09:06:50.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.229+0000 7f8c250e6740 1 -- 172.21.15.120:0/2112397545 shutdown_connections 2024-09-19T09:06:50.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.229+0000 7f8c250e6740 1 --2- 172.21.15.120:0/2112397545 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55bd08f9de80 0x55bd08fa0270 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.229+0000 7f8c250e6740 1 --2- 172.21.15.120:0/2112397545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd08f8e370 0x55bd08f8b530 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.229+0000 7f8c250e6740 1 --2- 172.21.15.120:0/2112397545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd08f84b40 0x55bd08f8afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.229+0000 7f8c250e6740 1 --2- 172.21.15.120:0/2112397545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd08f835d0 0x55bd08f8dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.229+0000 7f8bfbfff640 0 client.16884 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:06:50.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.229+0000 7f8c250e6740 1 -- 172.21.15.120:0/2112397545 >> 172.21.15.120:0/2112397545 conn(0x55bd08f4c310 msgr2=0x55bd08f85960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:50.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.230+0000 7f8c250e6740 1 -- 172.21.15.120:0/2112397545 shutdown_connections 2024-09-19T09:06:50.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.230+0000 7f8c250e6740 1 -- 172.21.15.120:0/2112397545 wait complete. 2024-09-19T09:06:50.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.230+0000 7f8c250e6740 20 client.16884 trim_cache size 0 max 16384 2024-09-19T09:06:50.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.242+0000 7f8c250e6740 1 Processor -- start 2024-09-19T09:06:50.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.242+0000 7f8c250e6740 1 -- start start 2024-09-19T09:06:50.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.242+0000 7f8c250e6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd08f835d0 0x55bd08f9a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:50.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.242+0000 7f8c250e6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd08f84b40 0x55bd08f86120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:50.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.242+0000 7f8c250e6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd08f8e370 0x55bd08f86660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:50.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.242+0000 7f8c250e6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1ad5f0 con 0x55bd08f835d0 2024-09-19T09:06:50.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.242+0000 7f8c250e6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1af130 con 0x55bd08f8e370 2024-09-19T09:06:50.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.242+0000 7f8c250e6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1adcc0 con 0x55bd08f84b40 2024-09-19T09:06:50.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.243+0000 7f8c212aa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd08f835d0 0x55bd08f9a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:50.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.243+0000 7f8c212aa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd08f835d0 0x55bd08f9a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37352/0 (socket says 172.21.15.120:37352) 2024-09-19T09:06:50.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.243+0000 7f8c212aa640 1 -- 172.21.15.120:0/2753210742 learned_addr learned my addr 172.21.15.120:0/2753210742 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:50.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.243+0000 7f8c1bfff640 1 --2- 172.21.15.120:0/2753210742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd08f84b40 0x55bd08f86120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:50.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.243+0000 7f8c212aa640 1 --2- 172.21.15.120:0/2753210742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd08f8e370 0x55bd08f86660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:50.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.243+0000 7f8c212aa640 1 -- 172.21.15.120:0/2753210742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd08f84b40 msgr2=0x55bd08f86120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:50.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.243+0000 7f8c212aa640 1 --2- 172.21.15.120:0/2753210742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd08f84b40 0x55bd08f86120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.243+0000 7f8c212aa640 1 -- 172.21.15.120:0/2753210742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd08f8e370 msgr2=0x55bd08f86660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:50.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.243+0000 7f8c212aa640 1 --2- 172.21.15.120:0/2753210742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd08f8e370 0x55bd08f86660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.243+0000 7f8c212aa640 1 -- 172.21.15.120:0/2753210742 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd08f8e070 con 0x55bd08f835d0 2024-09-19T09:06:50.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.243+0000 7f8c212aa640 1 --2- 172.21.15.120:0/2753210742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd08f835d0 0x55bd08f9a310 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8c140077f0 tx=0x7f8c14007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587f5ea2c241af7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:50.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.243+0000 7f8c067fc640 1 -- 172.21.15.120:0/2753210742 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1400d710 con 0x55bd08f835d0 2024-09-19T09:06:50.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.243+0000 7f8c067fc640 1 -- 172.21.15.120:0/2753210742 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c14005210 con 0x55bd08f835d0 2024-09-19T09:06:50.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.243+0000 7f8c067fc640 1 -- 172.21.15.120:0/2753210742 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c14010b40 con 0x55bd08f835d0 2024-09-19T09:06:50.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.244+0000 7f8c250e6740 1 -- 172.21.15.120:0/2753210742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd08f835d0 msgr2=0x55bd08f9a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:50.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.244+0000 7f8c250e6740 1 --2- 172.21.15.120:0/2753210742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd08f835d0 0x55bd08f9a310 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8c140077f0 tx=0x7f8c14007780 comp rx=0 tx=0).stop 2024-09-19T09:06:50.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.244+0000 7f8c250e6740 1 -- 172.21.15.120:0/2753210742 shutdown_connections 2024-09-19T09:06:50.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.244+0000 7f8c250e6740 1 --2- 172.21.15.120:0/2753210742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd08f8e370 0x55bd08f86660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.244+0000 7f8c250e6740 1 --2- 172.21.15.120:0/2753210742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd08f84b40 0x55bd08f86120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.244+0000 7f8c250e6740 1 --2- 172.21.15.120:0/2753210742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd08f835d0 0x55bd08f9a310 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.244+0000 7f8c250e6740 1 -- 172.21.15.120:0/2753210742 >> 172.21.15.120:0/2753210742 conn(0x55bd08f4c310 msgr2=0x55bd08f8a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:50.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.244+0000 7f8c250e6740 1 -- 172.21.15.120:0/2753210742 shutdown_connections 2024-09-19T09:06:50.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.244+0000 7f8c250e6740 1 -- 172.21.15.120:0/2753210742 wait complete. 2024-09-19T09:06:50.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.244+0000 7f8c250e6740 1 -- start start 2024-09-19T09:06:50.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.244+0000 7f8c250e6740 1 Processor -- start 2024-09-19T09:06:50.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.245+0000 7f8c250e6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd08f835d0 0x55bd08f91800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:50.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.245+0000 7f8c250e6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd08f84b40 0x55bd08f91d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:50.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.245+0000 7f8c250e6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd08f8e370 0x55bd09006950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:50.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.245+0000 7f8c250e6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1aea60 con 0x55bd08f835d0 2024-09-19T09:06:50.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.245+0000 7f8c250e6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1ae390 con 0x55bd08f8e370 2024-09-19T09:06:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.245+0000 7f8c250e6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1ab5b0 con 0x55bd08f84b40 2024-09-19T09:06:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.245+0000 7f8c250e6740 10 client.0 _ll_register_callbacks cb 0x7f8c100d35f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.245+0000 7f8c212aa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd08f835d0 0x55bd08f91800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.245+0000 7f8c212aa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd08f835d0 0x55bd08f91800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37360/0 (socket says 172.21.15.120:37360) 2024-09-19T09:06:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.245+0000 7f8c212aa640 1 -- 172.21.15.120:0/528505538 learned_addr learned my addr 172.21.15.120:0/528505538 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.245+0000 7f8c1bfff640 1 --2- 172.21.15.120:0/528505538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd08f84b40 0x55bd08f91d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.245+0000 7f8c212aa640 1 --2- 172.21.15.120:0/528505538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd08f8e370 0x55bd09006950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.245+0000 7f8c212aa640 1 -- 172.21.15.120:0/528505538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd08f84b40 msgr2=0x55bd08f91d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.245+0000 7f8c212aa640 1 --2- 172.21.15.120:0/528505538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd08f84b40 0x55bd08f91d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.245+0000 7f8c212aa640 1 -- 172.21.15.120:0/528505538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd08f8e370 msgr2=0x55bd09006950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.245+0000 7f8c212aa640 1 --2- 172.21.15.120:0/528505538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd08f8e370 0x55bd09006950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.271 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:06:50.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.245+0000 7f8c212aa640 1 -- 172.21.15.120:0/528505538 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd09006e90 con 0x55bd08f835d0 2024-09-19T09:06:50.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.245+0000 7f8c212aa640 1 --2- 172.21.15.120:0/528505538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd08f835d0 0x55bd08f91800 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f8c14004e80 tx=0x7f8c14004ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a10fc0c99e59a47 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:50.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.245+0000 7f8bfbfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:06:50.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.245+0000 7f8bfbfff640 1 -- 172.21.15.120:0/528505538 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c14004c50 con 0x55bd08f835d0 2024-09-19T09:06:50.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.245+0000 7f8bfbfff640 1 -- 172.21.15.120:0/528505538 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f8c14010660 con 0x55bd08f835d0 2024-09-19T09:06:50.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.245+0000 7f8c250e6740 1 -- 172.21.15.120:0/528505538 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c14005210 con 0x55bd08f835d0 2024-09-19T09:06:50.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.246+0000 7f8bfbfff640 1 -- 172.21.15.120:0/528505538 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8c1400de80 con 0x55bd08f835d0 2024-09-19T09:06:50.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.246+0000 7f8c250e6740 10 client.16890 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:50.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.246+0000 7f8c250e6740 1 -- 172.21.15.120:0/528505538 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bd090080f0 con 0x55bd08f835d0 2024-09-19T09:06:50.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.246+0000 7f8bfbfff640 1 -- 172.21.15.120:0/528505538 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c14005210 con 0x55bd08f835d0 2024-09-19T09:06:50.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.246+0000 7f8bfbfff640 1 -- 172.21.15.120:0/528505538 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8c1404b7f0 con 0x55bd08f835d0 2024-09-19T09:06:50.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.246+0000 7f8c250e6740 10 client.16890 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:50.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.246+0000 7f8c250e6740 10 client.16890 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:06:50.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.246+0000 7f8c250e6740 20 client.16890 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:50.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.246+0000 7f8c250e6740 1 --2- 172.21.15.120:0/528505538 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55bd09008750 0x55bd0900ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:50.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.246+0000 7f8c250e6740 4 client.16890 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:06:50.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.246+0000 7f8c250e6740 1 -- 172.21.15.120:0/528505538 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bd08ef0d70 con 0x55bd09008750 2024-09-19T09:06:50.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.246+0000 7f8c1bfff640 1 --2- 172.21.15.120:0/528505538 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55bd09008750 0x55bd0900ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:50.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.247+0000 7f8bfbfff640 10 client.16890 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:06:50.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.247+0000 7f8c1bfff640 1 --2- 172.21.15.120:0/528505538 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55bd09008750 0x55bd0900ab40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c18846b4adabf0d server_cookie=e14ed294768d8f2 in_seq=0 out_seq=0 2024-09-19T09:06:50.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.247+0000 7f8bfbfff640 1 -- 172.21.15.120:0/528505538 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bd08ef0d70 con 0x55bd09008750 2024-09-19T09:06:50.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.247+0000 7f8bfbfff640 10 client.16890 handle_command_reply: tid=0 2024-09-19T09:06:50.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.247+0000 7f8c250e6740 1 client.16890 shutdown 2024-09-19T09:06:50.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.247+0000 7f8c250e6740 1 -- 172.21.15.120:0/528505538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd08f835d0 msgr2=0x55bd08f91800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:50.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.247+0000 7f8c250e6740 1 --2- 172.21.15.120:0/528505538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd08f835d0 0x55bd08f91800 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f8c14004e80 tx=0x7f8c14004ee0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.247+0000 7f8c250e6740 1 -- 172.21.15.120:0/528505538 shutdown_connections 2024-09-19T09:06:50.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.247+0000 7f8c250e6740 1 --2- 172.21.15.120:0/528505538 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55bd09008750 0x55bd0900ab40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.247+0000 7f8c250e6740 1 --2- 172.21.15.120:0/528505538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd08f8e370 0x55bd09006950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.247+0000 7f8c250e6740 1 --2- 172.21.15.120:0/528505538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd08f84b40 0x55bd08f91d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.247+0000 7f8bfbfff640 0 client.16890 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:06:50.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.247+0000 7f8c250e6740 1 --2- 172.21.15.120:0/528505538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd08f835d0 0x55bd08f91800 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.247+0000 7f8c250e6740 1 -- 172.21.15.120:0/528505538 >> 172.21.15.120:0/528505538 conn(0x55bd08f4c310 msgr2=0x55bd08f8c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:50.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.247+0000 7f8c250e6740 1 -- 172.21.15.120:0/528505538 shutdown_connections 2024-09-19T09:06:50.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.247+0000 7f8c250e6740 1 -- 172.21.15.120:0/528505538 wait complete. 2024-09-19T09:06:50.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.247+0000 7f8c250e6740 20 client.16890 trim_cache size 0 max 16384 2024-09-19T09:06:50.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.248+0000 7f8c22d0d640 1 -- 172.21.15.120:0/1812959202 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8c0c0a3240 msgr2=0x7f8c0c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:50.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.248+0000 7f8c22d0d640 1 --2- 172.21.15.120:0/1812959202 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8c0c0a3240 0x7f8c0c0a5700 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8c1001e000 tx=0x7f8c100d3720 comp rx=0 tx=0).stop 2024-09-19T09:06:50.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.248+0000 7f8c22d0d640 1 -- 172.21.15.120:0/1812959202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c1c101de0 msgr2=0x7f8c1c1983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:50.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.248+0000 7f8c22d0d640 1 --2- 172.21.15.120:0/1812959202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c1c101de0 0x7f8c1c1983b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8c1000e9f0 tx=0x7f8c1000eeb0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.248+0000 7f8c22d0d640 1 -- 172.21.15.120:0/1812959202 shutdown_connections 2024-09-19T09:06:50.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.248+0000 7f8c22d0d640 1 --2- 172.21.15.120:0/1812959202 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8c0c0a3240 0x7f8c0c0a5700 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.248+0000 7f8c22d0d640 1 --2- 172.21.15.120:0/1812959202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c1c106f10 0x7f8c1c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.248+0000 7f8c22d0d640 1 --2- 172.21.15.120:0/1812959202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c1c102790 0x7f8c1c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.248+0000 7f8c22d0d640 1 --2- 172.21.15.120:0/1812959202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c1c101de0 0x7f8c1c1983b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:50.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.248+0000 7f8c22d0d640 1 -- 172.21.15.120:0/1812959202 >> 172.21.15.120:0/1812959202 conn(0x7f8c1c0fd8c0 msgr2=0x7f8c1c0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:50.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.248+0000 7f8c22d0d640 1 -- 172.21.15.120:0/1812959202 shutdown_connections 2024-09-19T09:06:50.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:50.248+0000 7f8c22d0d640 1 -- 172.21.15.120:0/1812959202 wait complete. 2024-09-19T09:06:51.044 INFO:tasks.workunit.client.0.smithi120.stdout: 1 310508 9.68 MB/sec execute 46 sec latency 12.491 ms 2024-09-19T09:06:51.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:50 smithi120 ceph-mon[31150]: pgmap v460: 97 pgs: 97 active+clean; 38 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 365 op/s 2024-09-19T09:06:51.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:50 smithi196 ceph-mon[35150]: pgmap v460: 97 pgs: 97 active+clean; 38 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 365 op/s 2024-09-19T09:06:51.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:50 smithi153 ceph-mon[35071]: pgmap v460: 97 pgs: 97 active+clean; 38 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 365 op/s 2024-09-19T09:06:51.195 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:06:51.458 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:06:51.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.972+0000 7f482d8a7640 1 -- 172.21.15.120:0/1009506052 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4828106f10 msgr2=0x7f482810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:51.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.972+0000 7f482d8a7640 1 --2- 172.21.15.120:0/1009506052 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4828106f10 0x7f482810ea90 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f4820009ca0 tx=0x7f482002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:06:51.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.973+0000 7f482d8a7640 1 -- 172.21.15.120:0/1009506052 shutdown_connections 2024-09-19T09:06:51.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.973+0000 7f482d8a7640 1 --2- 172.21.15.120:0/1009506052 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4828106f10 0x7f482810ea90 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:51.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.973+0000 7f482d8a7640 1 --2- 172.21.15.120:0/1009506052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4828102790 0x7f48281067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:51.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.973+0000 7f482d8a7640 1 --2- 172.21.15.120:0/1009506052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4828101de0 0x7f48281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:51.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.973+0000 7f482d8a7640 1 -- 172.21.15.120:0/1009506052 >> 172.21.15.120:0/1009506052 conn(0x7f48280fd8c0 msgr2=0x7f48280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:51.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.973+0000 7f482d8a7640 1 -- 172.21.15.120:0/1009506052 shutdown_connections 2024-09-19T09:06:51.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.973+0000 7f482d8a7640 1 -- 172.21.15.120:0/1009506052 wait complete. 2024-09-19T09:06:51.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.973+0000 7f482d8a7640 1 Processor -- start 2024-09-19T09:06:51.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.973+0000 7f482d8a7640 1 -- start start 2024-09-19T09:06:51.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.974+0000 7f482d8a7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4828101de0 0x7f482819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:51.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.974+0000 7f482d8a7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4828102790 0x7f482819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:51.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.974+0000 7f482d8a7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4828106f10 0x7f482819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:51.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.974+0000 7f482d8a7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4828111f70 con 0x7f4828106f10 2024-09-19T09:06:51.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.974+0000 7f482d8a7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4828111df0 con 0x7f4828102790 2024-09-19T09:06:51.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.974+0000 7f482d8a7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f48281120f0 con 0x7f4828101de0 2024-09-19T09:06:51.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.974+0000 7f48277fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4828106f10 0x7f482819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:51.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.974+0000 7f4826ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4828101de0 0x7f482819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:51.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.974+0000 7f48277fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4828106f10 0x7f482819eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42430/0 (socket says 172.21.15.120:42430) 2024-09-19T09:06:51.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.974+0000 7f48267fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4828102790 0x7f482819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:51.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.974+0000 7f4826ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4828101de0 0x7f482819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:37270/0 (socket says 172.21.15.120:37270) 2024-09-19T09:06:51.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.974+0000 7f48277fe640 1 -- 172.21.15.120:0/312811137 learned_addr learned my addr 172.21.15.120:0/312811137 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:51.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.974+0000 7f48277fe640 1 -- 172.21.15.120:0/312811137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4828101de0 msgr2=0x7f482819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:51.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.974+0000 7f48277fe640 1 --2- 172.21.15.120:0/312811137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4828101de0 0x7f482819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:51.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.974+0000 7f48277fe640 1 -- 172.21.15.120:0/312811137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4828102790 msgr2=0x7f482819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:51.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.974+0000 7f48277fe640 1 --2- 172.21.15.120:0/312811137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4828102790 0x7f482819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:51.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.974+0000 7f48277fe640 1 -- 172.21.15.120:0/312811137 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482819f5a0 con 0x7f4828106f10 2024-09-19T09:06:51.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.974+0000 7f48267fc640 1 --2- 172.21.15.120:0/312811137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4828102790 0x7f482819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:51.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.975+0000 7f48277fe640 1 --2- 172.21.15.120:0/312811137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4828106f10 0x7f482819eec0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f4820031a20 tx=0x7f4820031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:51.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.975+0000 7f482c8a5640 1 -- 172.21.15.120:0/312811137 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820003c30 con 0x7f4828106f10 2024-09-19T09:06:51.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.975+0000 7f482d8a7640 1 -- 172.21.15.120:0/312811137 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f482819f830 con 0x7f4828106f10 2024-09-19T09:06:51.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.975+0000 7f482c8a5640 1 -- 172.21.15.120:0/312811137 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4820003dd0 con 0x7f4828106f10 2024-09-19T09:06:51.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.975+0000 7f482c8a5640 1 -- 172.21.15.120:0/312811137 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482002fc40 con 0x7f4828106f10 2024-09-19T09:06:51.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.975+0000 7f482d8a7640 1 -- 172.21.15.120:0/312811137 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48281a7170 con 0x7f4828106f10 2024-09-19T09:06:51.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.978+0000 7f482c8a5640 1 -- 172.21.15.120:0/312811137 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f4820004050 con 0x7f4828106f10 2024-09-19T09:06:51.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.978+0000 7f482fc80740 1 Processor -- start 2024-09-19T09:06:51.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.978+0000 7f482fc80740 1 -- start start 2024-09-19T09:06:51.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.978+0000 7f482fc80740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c280283b40 0x55c280282090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:51.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.978+0000 7f482fc80740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2802825d0 0x55c280282a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:51.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.978+0000 7f482fc80740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c28028d370 0x55c28028f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:51.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.978+0000 7f482fc80740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c2801e82f0 con 0x55c28028d370 2024-09-19T09:06:51.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.978+0000 7f482fc80740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c28018fcf0 con 0x55c2802825d0 2024-09-19T09:06:51.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.978+0000 7f482fc80740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c2801e42b0 con 0x55c280283b40 2024-09-19T09:06:51.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.978+0000 7f4826ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2802825d0 0x55c280282a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:52.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.978+0000 7f4826ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2802825d0 0x55c280282a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:51574/0 (socket says 172.21.15.120:51574) 2024-09-19T09:06:52.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.978+0000 7f4826ffd640 1 -- 172.21.15.120:0/1492497642 learned_addr learned my addr 172.21.15.120:0/1492497642 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:52.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.978+0000 7f48267fc640 1 --2- 172.21.15.120:0/1492497642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c280283b40 0x55c280282090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:52.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.979+0000 7f48267fc640 1 --2- 172.21.15.120:0/1492497642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c28028d370 0x55c28028f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:52.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.979+0000 7f482c8a5640 1 --2- 172.21.15.120:0/312811137 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f48100a3240 0x7f48100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:52.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.979+0000 7f4826ffd640 1 -- 172.21.15.120:0/1492497642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c280283b40 msgr2=0x55c280282090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:52.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.979+0000 7f4826ffd640 1 --2- 172.21.15.120:0/1492497642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c280283b40 0x55c280282090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:52.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.979+0000 7f4826ffd640 1 -- 172.21.15.120:0/1492497642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c28028d370 msgr2=0x55c28028f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:52.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.979+0000 7f482c8a5640 1 -- 172.21.15.120:0/312811137 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f48200f0a90 con 0x7f4828106f10 2024-09-19T09:06:52.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.979+0000 7f48277fe640 1 --2- 172.21.15.120:0/312811137 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f48100a3240 0x7f48100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:52.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.979+0000 7f4826ffd640 1 --2- 172.21.15.120:0/1492497642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c28028d370 0x55c28028f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:52.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.979+0000 7f4826ffd640 1 -- 172.21.15.120:0/1492497642 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2801743a0 con 0x55c2802825d0 2024-09-19T09:06:52.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.979+0000 7f48267fc640 1 --2- 172.21.15.120:0/1492497642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c28028d370 0x55c28028f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:52.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.979+0000 7f4826ffd640 1 --2- 172.21.15.120:0/1492497642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2802825d0 0x55c280282a10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4818009cf0 tx=0x7f4818011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efad849b9307af6f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:52.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.979+0000 7f48277fe640 1 --2- 172.21.15.120:0/312811137 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f48100a3240 0x7f48100a5700 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f482002f540 tx=0x7f482002a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:52.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.979+0000 7f480dffb640 1 -- 172.21.15.120:0/1492497642 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4818004230 con 0x55c2802825d0 2024-09-19T09:06:52.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.980+0000 7f480dffb640 1 -- 172.21.15.120:0/1492497642 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48180043d0 con 0x55c2802825d0 2024-09-19T09:06:52.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.980+0000 7f480dffb640 1 -- 172.21.15.120:0/1492497642 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48180197e0 con 0x55c2802825d0 2024-09-19T09:06:52.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.980+0000 7f482fc80740 1 -- 172.21.15.120:0/1492497642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2802825d0 msgr2=0x55c280282a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:52.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.980+0000 7f482fc80740 1 --2- 172.21.15.120:0/1492497642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2802825d0 0x55c280282a10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4818009cf0 tx=0x7f4818011810 comp rx=0 tx=0).stop 2024-09-19T09:06:52.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.980+0000 7f482fc80740 1 -- 172.21.15.120:0/1492497642 shutdown_connections 2024-09-19T09:06:52.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.980+0000 7f482fc80740 1 --2- 172.21.15.120:0/1492497642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c28028d370 0x55c28028f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:52.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.980+0000 7f482fc80740 1 --2- 172.21.15.120:0/1492497642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2802825d0 0x55c280282a10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:52.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.980+0000 7f482fc80740 1 --2- 172.21.15.120:0/1492497642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c280283b40 0x55c280282090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:52.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.980+0000 7f482fc80740 1 -- 172.21.15.120:0/1492497642 >> 172.21.15.120:0/1492497642 conn(0x55c28024b310 msgr2=0x55c28027ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:52.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.980+0000 7f482fc80740 1 -- 172.21.15.120:0/1492497642 shutdown_connections 2024-09-19T09:06:52.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.980+0000 7f482fc80740 1 -- 172.21.15.120:0/1492497642 wait complete. 2024-09-19T09:06:52.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.981+0000 7f482fc80740 1 -- start start 2024-09-19T09:06:52.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.981+0000 7f482fc80740 1 Processor -- start 2024-09-19T09:06:52.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.981+0000 7f482fc80740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2802825d0 0x55c28028cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:52.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.981+0000 7f482fc80740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280283b40 0x55c280289fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:52.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.981+0000 7f482fc80740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c28028d370 0x55c28028a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:52.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.981+0000 7f482fc80740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c280290070 con 0x55c28028d370 2024-09-19T09:06:52.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.981+0000 7f482fc80740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c2801a3060 con 0x55c280283b40 2024-09-19T09:06:52.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.981+0000 7f482fc80740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c2802901f0 con 0x55c2802825d0 2024-09-19T09:06:52.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.981+0000 7f482fc80740 10 client.0 _ll_register_callbacks cb 0x55c27fef68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:52.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.982+0000 7f48267fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2802825d0 0x55c28028cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:52.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.982+0000 7f4826ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280283b40 0x55c280289fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:52.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.982+0000 7f48267fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2802825d0 0x55c28028cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:37288/0 (socket says 172.21.15.120:37288) 2024-09-19T09:06:52.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.982+0000 7f4826ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280283b40 0x55c280289fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:51582/0 (socket says 172.21.15.120:51582) 2024-09-19T09:06:52.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.982+0000 7f48267fc640 1 -- 172.21.15.120:0/2989128427 learned_addr learned my addr 172.21.15.120:0/2989128427 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:52.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.982+0000 7f48267fc640 1 --2- 172.21.15.120:0/2989128427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c28028d370 0x55c28028a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:52.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.982+0000 7f48267fc640 1 -- 172.21.15.120:0/2989128427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280283b40 msgr2=0x55c280289fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:52.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.982+0000 7f48267fc640 1 --2- 172.21.15.120:0/2989128427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280283b40 0x55c280289fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:52.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.982+0000 7f48267fc640 1 -- 172.21.15.120:0/2989128427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c28028d370 msgr2=0x55c28028a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:52.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.982+0000 7f48267fc640 1 --2- 172.21.15.120:0/2989128427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c28028d370 0x55c28028a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:52.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.982+0000 7f48267fc640 1 -- 172.21.15.120:0/2989128427 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c27ff593c0 con 0x55c2802825d0 2024-09-19T09:06:52.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.982+0000 7f48267fc640 1 --2- 172.21.15.120:0/2989128427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2802825d0 0x55c28028cd00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f481c013f10 tx=0x7f481c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75c43713f2e8e8d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:52.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.982+0000 7f4806ffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:06:52.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.982+0000 7f4806ffd640 1 -- 172.21.15.120:0/2989128427 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481c005020 con 0x55c2802825d0 2024-09-19T09:06:52.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.982+0000 7f482fc80740 1 -- 172.21.15.120:0/2989128427 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48180043d0 con 0x55c2802825d0 2024-09-19T09:06:52.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.982+0000 7f4806ffd640 1 -- 172.21.15.120:0/2989128427 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f481c0051c0 con 0x55c2802825d0 2024-09-19T09:06:52.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.983+0000 7f4806ffd640 1 -- 172.21.15.120:0/2989128427 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f481c0053c0 con 0x55c2802825d0 2024-09-19T09:06:52.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.983+0000 7f482fc80740 10 client.26185 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:52.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.983+0000 7f482fc80740 1 -- 172.21.15.120:0/2989128427 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c28029bb00 con 0x55c2802825d0 2024-09-19T09:06:52.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.983+0000 7f4806ffd640 1 -- 172.21.15.120:0/2989128427 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48180043d0 con 0x55c2802825d0 2024-09-19T09:06:52.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.983+0000 7f4806ffd640 1 -- 172.21.15.120:0/2989128427 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f481c0049d0 con 0x55c2802825d0 2024-09-19T09:06:52.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.983+0000 7f482fc80740 10 client.26185 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:52.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.983+0000 7f482fc80740 10 client.26185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:06:52.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.983+0000 7f482fc80740 20 client.26185 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:52.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.983+0000 7f482fc80740 1 --2- 172.21.15.120:0/2989128427 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c28029cfc0 0x55c28029f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:52.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.983+0000 7f482fc80740 4 client.26185 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:06:52.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.983+0000 7f482fc80740 1 -- 172.21.15.120:0/2989128427 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2801f5860 con 0x55c28029cfc0 2024-09-19T09:06:52.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.983+0000 7f4826ffd640 1 --2- 172.21.15.120:0/2989128427 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c28029cfc0 0x55c28029f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:52.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.984+0000 7f4806ffd640 10 client.26185 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:52.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.984+0000 7f4826ffd640 1 --2- 172.21.15.120:0/2989128427 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c28029cfc0 0x55c28029f3b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eec1829089c3764f server_cookie=24b0230ebcdb369f in_seq=0 out_seq=0 2024-09-19T09:06:52.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.985+0000 7f4806ffd640 1 -- 172.21.15.120:0/2989128427 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c2801f5860 con 0x55c28029cfc0 2024-09-19T09:06:52.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.985+0000 7f4806ffd640 10 client.26185 handle_command_reply: tid=0 2024-09-19T09:06:52.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.985+0000 7f482fc80740 1 client.26185 shutdown 2024-09-19T09:06:52.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.985+0000 7f482fc80740 1 -- 172.21.15.120:0/2989128427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2802825d0 msgr2=0x55c28028cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:52.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.985+0000 7f482fc80740 1 --2- 172.21.15.120:0/2989128427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2802825d0 0x55c28028cd00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f481c013f10 tx=0x7f481c004730 comp rx=0 tx=0).stop 2024-09-19T09:06:52.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.985+0000 7f482fc80740 1 -- 172.21.15.120:0/2989128427 shutdown_connections 2024-09-19T09:06:52.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.985+0000 7f482fc80740 1 --2- 172.21.15.120:0/2989128427 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c28029cfc0 0x55c28029f3b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:52.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.985+0000 7f482fc80740 1 --2- 172.21.15.120:0/2989128427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c28028d370 0x55c28028a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:52.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.985+0000 7f482fc80740 1 --2- 172.21.15.120:0/2989128427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280283b40 0x55c280289fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:52.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.985+0000 7f482fc80740 1 --2- 172.21.15.120:0/2989128427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2802825d0 0x55c28028cd00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:52.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.985+0000 7f4806ffd640 0 client.26185 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:52.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.985+0000 7f482fc80740 1 -- 172.21.15.120:0/2989128427 >> 172.21.15.120:0/2989128427 conn(0x55c28024b310 msgr2=0x55c280284960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:52.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.985+0000 7f482fc80740 1 -- 172.21.15.120:0/2989128427 shutdown_connections 2024-09-19T09:06:52.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.985+0000 7f482fc80740 1 -- 172.21.15.120:0/2989128427 wait complete. 2024-09-19T09:06:52.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.985+0000 7f482fc80740 20 client.26185 trim_cache size 0 max 16384 2024-09-19T09:06:52.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.998+0000 7f482fc80740 1 Processor -- start 2024-09-19T09:06:52.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.999+0000 7f482fc80740 1 -- start start 2024-09-19T09:06:52.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.999+0000 7f482fc80740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2802825d0 0x55c28027ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:52.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.999+0000 7f482fc80740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280283b40 0x55c280299310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:52.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.999+0000 7f482fc80740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c28028d370 0x55c280285120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:52.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.999+0000 7f482fc80740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f48281af7b0 con 0x55c28028d370 2024-09-19T09:06:52.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.999+0000 7f482fc80740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f48281b12f0 con 0x55c280283b40 2024-09-19T09:06:52.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.999+0000 7f482fc80740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f48281afe80 con 0x55c2802825d0 2024-09-19T09:06:52.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.999+0000 7f48267fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c28028d370 0x55c280285120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:52.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.999+0000 7f4826ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280283b40 0x55c280299310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:52.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.999+0000 7f48267fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c28028d370 0x55c280285120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42438/0 (socket says 172.21.15.120:42438) 2024-09-19T09:06:52.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.999+0000 7f48267fc640 1 -- 172.21.15.120:0/2967302532 learned_addr learned my addr 172.21.15.120:0/2967302532 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:52.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.999+0000 7f48267fc640 1 --2- 172.21.15.120:0/2967302532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2802825d0 0x55c28027ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:52.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.999+0000 7f48267fc640 1 -- 172.21.15.120:0/2967302532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2802825d0 msgr2=0x55c28027ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:52.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.999+0000 7f48267fc640 1 --2- 172.21.15.120:0/2967302532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2802825d0 0x55c28027ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:52.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.999+0000 7f48267fc640 1 -- 172.21.15.120:0/2967302532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280283b40 msgr2=0x55c280299310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:52.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.999+0000 7f48267fc640 1 --2- 172.21.15.120:0/2967302532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280283b40 0x55c280299310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:52.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.999+0000 7f48267fc640 1 -- 172.21.15.120:0/2967302532 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c28028c920 con 0x55c28028d370 2024-09-19T09:06:52.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:51.999+0000 7f48267fc640 1 --2- 172.21.15.120:0/2967302532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c28028d370 0x55c280285120 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f481c013ca0 tx=0x7f481c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d29f5f4e71ff2b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:52.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.000+0000 7f480d7fa640 1 -- 172.21.15.120:0/2967302532 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481c0056f0 con 0x55c28028d370 2024-09-19T09:06:52.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.000+0000 7f480d7fa640 1 -- 172.21.15.120:0/2967302532 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f481c017030 con 0x55c28028d370 2024-09-19T09:06:52.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.000+0000 7f480d7fa640 1 -- 172.21.15.120:0/2967302532 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481c013000 con 0x55c28028d370 2024-09-19T09:06:52.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.000+0000 7f482fc80740 1 -- 172.21.15.120:0/2967302532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c28028d370 msgr2=0x55c280285120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:52.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.000+0000 7f482fc80740 1 --2- 172.21.15.120:0/2967302532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c28028d370 0x55c280285120 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f481c013ca0 tx=0x7f481c0046c0 comp rx=0 tx=0).stop 2024-09-19T09:06:52.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.000+0000 7f482fc80740 1 -- 172.21.15.120:0/2967302532 shutdown_connections 2024-09-19T09:06:52.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.001+0000 7f482fc80740 1 --2- 172.21.15.120:0/2967302532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c28028d370 0x55c280285120 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:52.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.001+0000 7f482fc80740 1 --2- 172.21.15.120:0/2967302532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280283b40 0x55c280299310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:52.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.001+0000 7f482fc80740 1 --2- 172.21.15.120:0/2967302532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2802825d0 0x55c28027ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:52.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.001+0000 7f482fc80740 1 -- 172.21.15.120:0/2967302532 >> 172.21.15.120:0/2967302532 conn(0x55c28024b310 msgr2=0x55c28027e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:52.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.001+0000 7f482fc80740 1 -- 172.21.15.120:0/2967302532 shutdown_connections 2024-09-19T09:06:52.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.001+0000 7f482fc80740 1 -- 172.21.15.120:0/2967302532 wait complete. 2024-09-19T09:06:52.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.001+0000 7f482fc80740 1 -- start start 2024-09-19T09:06:52.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.001+0000 7f482fc80740 1 Processor -- start 2024-09-19T09:06:52.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.001+0000 7f482fc80740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2802825d0 0x55c280295480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:52.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.002+0000 7f482fc80740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280283b40 0x55c2802959c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:52.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.002+0000 7f482fc80740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c28028d370 0x55c280286620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:52.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.002+0000 7f482fc80740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f48281b0c20 con 0x55c28028d370 2024-09-19T09:06:52.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.002+0000 7f482fc80740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f48281b0550 con 0x55c280283b40 2024-09-19T09:06:52.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.002+0000 7f482fc80740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f48281ad770 con 0x55c2802825d0 2024-09-19T09:06:52.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.002+0000 7f482fc80740 10 client.0 _ll_register_callbacks cb 0x7f481c00bf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:52.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.002+0000 7f48267fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2802825d0 0x55c280295480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:52.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.002+0000 7f4826ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280283b40 0x55c2802959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:52.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.002+0000 7f48267fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2802825d0 0x55c280295480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:37310/0 (socket says 172.21.15.120:37310) 2024-09-19T09:06:52.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.002+0000 7f48267fc640 1 -- 172.21.15.120:0/351163802 learned_addr learned my addr 172.21.15.120:0/351163802 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:52.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.002+0000 7f48267fc640 1 --2- 172.21.15.120:0/351163802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c28028d370 0x55c280286620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:52.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.002+0000 7f4826ffd640 1 -- 172.21.15.120:0/351163802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2802825d0 msgr2=0x55c280295480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:52.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.002+0000 7f4826ffd640 1 --2- 172.21.15.120:0/351163802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2802825d0 0x55c280295480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:52.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.002+0000 7f4826ffd640 1 -- 172.21.15.120:0/351163802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c28028d370 msgr2=0x55c280286620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:52.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.002+0000 7f4826ffd640 1 --2- 172.21.15.120:0/351163802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c28028d370 0x55c280286620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:52.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.002+0000 7f4826ffd640 1 -- 172.21.15.120:0/351163802 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c280286b60 con 0x55c280283b40 2024-09-19T09:06:52.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.002+0000 7f48267fc640 1 --2- 172.21.15.120:0/351163802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2802825d0 0x55c280295480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:52.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.002+0000 7f4826ffd640 1 --2- 172.21.15.120:0/351163802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280283b40 0x55c2802959c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f48180096e0 tx=0x7f48180194c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e305684363f92e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:52.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.002+0000 7f4806ffd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:06:52.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.002+0000 7f4806ffd640 1 -- 172.21.15.120:0/351163802 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481802f070 con 0x55c280283b40 2024-09-19T09:06:52.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.002+0000 7f4806ffd640 1 -- 172.21.15.120:0/351163802 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f4818005450 con 0x55c280283b40 2024-09-19T09:06:52.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.002+0000 7f482fc80740 1 -- 172.21.15.120:0/351163802 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f481c017030 con 0x55c280283b40 2024-09-19T09:06:52.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.003+0000 7f4806ffd640 1 -- 172.21.15.120:0/351163802 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f481803b050 con 0x55c280283b40 2024-09-19T09:06:52.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.003+0000 7f482fc80740 10 client.26252 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:52.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.003+0000 7f482fc80740 1 -- 172.21.15.120:0/351163802 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c280307440 con 0x55c280283b40 2024-09-19T09:06:52.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.003+0000 7f4806ffd640 1 -- 172.21.15.120:0/351163802 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f481c017030 con 0x55c280283b40 2024-09-19T09:06:52.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.003+0000 7f4806ffd640 1 -- 172.21.15.120:0/351163802 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4818019860 con 0x55c280283b40 2024-09-19T09:06:52.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.003+0000 7f482fc80740 10 client.26252 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:52.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.003+0000 7f482fc80740 10 client.26252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:06:52.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.003+0000 7f482fc80740 20 client.26252 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:52.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.003+0000 7f482fc80740 1 --2- 172.21.15.120:0/351163802 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c280307a30 0x55c280309e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:52.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.003+0000 7f482fc80740 4 client.26252 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:06:52.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.003+0000 7f482fc80740 1 -- 172.21.15.120:0/351163802 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c2801efd70 con 0x55c280307a30 2024-09-19T09:06:52.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.003+0000 7f48267fc640 1 --2- 172.21.15.120:0/351163802 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c280307a30 0x55c280309e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:52.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.004+0000 7f48267fc640 1 --2- 172.21.15.120:0/351163802 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c280307a30 0x55c280309e50 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a0d68b08474498c server_cookie=b20535009fa35770 in_seq=0 out_seq=0 2024-09-19T09:06:52.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.004+0000 7f4806ffd640 10 client.26252 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:52.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.004+0000 7f4806ffd640 1 -- 172.21.15.120:0/351163802 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c2801efd70 con 0x55c280307a30 2024-09-19T09:06:52.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.004+0000 7f4806ffd640 10 client.26252 handle_command_reply: tid=0 2024-09-19T09:06:52.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.004+0000 7f482fc80740 1 client.26252 shutdown 2024-09-19T09:06:52.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.004+0000 7f482fc80740 1 -- 172.21.15.120:0/351163802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280283b40 msgr2=0x55c2802959c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:52.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.004+0000 7f482fc80740 1 --2- 172.21.15.120:0/351163802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280283b40 0x55c2802959c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f48180096e0 tx=0x7f48180194c0 comp rx=0 tx=0).stop 2024-09-19T09:06:52.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.004+0000 7f482fc80740 1 -- 172.21.15.120:0/351163802 shutdown_connections 2024-09-19T09:06:52.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.004+0000 7f482fc80740 1 --2- 172.21.15.120:0/351163802 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c280307a30 0x55c280309e50 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:52.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.004+0000 7f482fc80740 1 --2- 172.21.15.120:0/351163802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c28028d370 0x55c280286620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:52.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.004+0000 7f482fc80740 1 --2- 172.21.15.120:0/351163802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c280283b40 0x55c2802959c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:52.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.004+0000 7f482fc80740 1 --2- 172.21.15.120:0/351163802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2802825d0 0x55c280295480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:52.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.004+0000 7f482fc80740 1 -- 172.21.15.120:0/351163802 >> 172.21.15.120:0/351163802 conn(0x55c28024b310 msgr2=0x55c28027d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:52.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.004+0000 7f4806ffd640 0 client.26252 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:52.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.004+0000 7f482fc80740 1 -- 172.21.15.120:0/351163802 shutdown_connections 2024-09-19T09:06:52.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.004+0000 7f482fc80740 1 -- 172.21.15.120:0/351163802 wait complete. 2024-09-19T09:06:52.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.005+0000 7f482fc80740 20 client.26252 trim_cache size 0 max 16384 2024-09-19T09:06:52.017 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:06:52.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.005+0000 7f482d8a7640 1 -- 172.21.15.120:0/312811137 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f48100a3240 msgr2=0x7f48100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:52.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.005+0000 7f482d8a7640 1 --2- 172.21.15.120:0/312811137 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f48100a3240 0x7f48100a5700 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f482002f540 tx=0x7f482002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:06:52.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.005+0000 7f482d8a7640 1 -- 172.21.15.120:0/312811137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4828106f10 msgr2=0x7f482819eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:52.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.005+0000 7f482d8a7640 1 --2- 172.21.15.120:0/312811137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4828106f10 0x7f482819eec0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f4820031a20 tx=0x7f4820031a50 comp rx=0 tx=0).stop 2024-09-19T09:06:52.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.005+0000 7f482d8a7640 1 -- 172.21.15.120:0/312811137 shutdown_connections 2024-09-19T09:06:52.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.005+0000 7f482d8a7640 1 --2- 172.21.15.120:0/312811137 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f48100a3240 0x7f48100a5700 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:52.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.005+0000 7f482d8a7640 1 --2- 172.21.15.120:0/312811137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4828106f10 0x7f482819eec0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:52.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.005+0000 7f482d8a7640 1 --2- 172.21.15.120:0/312811137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4828102790 0x7f482819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:52.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.005+0000 7f482d8a7640 1 --2- 172.21.15.120:0/312811137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4828101de0 0x7f482819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:52.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.005+0000 7f482d8a7640 1 -- 172.21.15.120:0/312811137 >> 172.21.15.120:0/312811137 conn(0x7f48280fd8c0 msgr2=0x7f48280ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:52.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.005+0000 7f482d8a7640 1 -- 172.21.15.120:0/312811137 shutdown_connections 2024-09-19T09:06:52.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:52.005+0000 7f482d8a7640 1 -- 172.21.15.120:0/312811137 wait complete. 2024-09-19T09:06:52.044 INFO:tasks.workunit.client.0.smithi120.stdout: 1 313134 9.79 MB/sec execute 47 sec latency 18.425 ms 2024-09-19T09:06:53.044 INFO:tasks.workunit.client.0.smithi120.stdout: 1 316261 9.96 MB/sec execute 48 sec latency 10.334 ms 2024-09-19T09:06:53.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:52 smithi120 ceph-mon[31150]: pgmap v461: 97 pgs: 97 active+clean; 35 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.1 MiB/s wr, 466 op/s 2024-09-19T09:06:53.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:52 smithi196 ceph-mon[35150]: pgmap v461: 97 pgs: 97 active+clean; 35 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.1 MiB/s wr, 466 op/s 2024-09-19T09:06:53.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:52 smithi153 ceph-mon[35071]: pgmap v461: 97 pgs: 97 active+clean; 35 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.1 MiB/s wr, 466 op/s 2024-09-19T09:06:54.044 INFO:tasks.workunit.client.0.smithi120.stdout: 1 319543 10.15 MB/sec execute 49 sec latency 10.344 ms 2024-09-19T09:06:54.240 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:06:54.241 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:06:54.498 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:06:54.941 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:54 smithi120 ceph-mon[31150]: pgmap v462: 97 pgs: 97 active+clean; 36 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.0 MiB/s wr, 451 op/s 2024-09-19T09:06:55.044 INFO:tasks.workunit.client.0.smithi120.stdout: 1 322801 10.30 MB/sec execute 50 sec latency 11.430 ms 2024-09-19T09:06:55.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.032+0000 7ff4b1fa4640 1 -- 172.21.15.120:0/635262235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4ac103780 msgr2=0x7ff4ac107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:55.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.032+0000 7ff4b1fa4640 1 --2- 172.21.15.120:0/635262235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4ac103780 0x7ff4ac107bc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009890 tx=0x7ff4a002a380 comp rx=0 tx=0).stop 2024-09-19T09:06:55.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.032+0000 7ff4b1fa4640 1 -- 172.21.15.120:0/635262235 shutdown_connections 2024-09-19T09:06:55.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.032+0000 7ff4b1fa4640 1 --2- 172.21.15.120:0/635262235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4ac108300 0x7ff4ac10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.032+0000 7ff4b1fa4640 1 --2- 172.21.15.120:0/635262235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4ac103780 0x7ff4ac107bc0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.032+0000 7ff4b1fa4640 1 --2- 172.21.15.120:0/635262235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4ac102dd0 0x7ff4ac1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.032+0000 7ff4b1fa4640 1 -- 172.21.15.120:0/635262235 >> 172.21.15.120:0/635262235 conn(0x7ff4ac0fc7d0 msgr2=0x7ff4ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:55.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.032+0000 7ff4b1fa4640 1 -- 172.21.15.120:0/635262235 shutdown_connections 2024-09-19T09:06:55.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.032+0000 7ff4b1fa4640 1 -- 172.21.15.120:0/635262235 wait complete. 2024-09-19T09:06:55.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.033+0000 7ff4b1fa4640 1 Processor -- start 2024-09-19T09:06:55.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.033+0000 7ff4b1fa4640 1 -- start start 2024-09-19T09:06:55.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.033+0000 7ff4b1fa4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4ac102dd0 0x7ff4ac198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:55.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.033+0000 7ff4b1fa4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4ac103780 0x7ff4ac198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:55.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.033+0000 7ff4b1fa4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4ac108300 0x7ff4ac19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:55.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.033+0000 7ff4b1fa4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac112070 con 0x7ff4ac102dd0 2024-09-19T09:06:55.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.033+0000 7ff4b1fa4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac111ef0 con 0x7ff4ac103780 2024-09-19T09:06:55.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.033+0000 7ff4b1fa4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac1121f0 con 0x7ff4ac108300 2024-09-19T09:06:55.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.034+0000 7ff4ab7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4ac102dd0 0x7ff4ac198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:55.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.034+0000 7ff4ab7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4ac102dd0 0x7ff4ac198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42472/0 (socket says 172.21.15.120:42472) 2024-09-19T09:06:55.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.034+0000 7ff4ab7fe640 1 -- 172.21.15.120:0/3196950209 learned_addr learned my addr 172.21.15.120:0/3196950209 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:55.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.034+0000 7ff4abfff640 1 --2- 172.21.15.120:0/3196950209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4ac108300 0x7ff4ac19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:55.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.034+0000 7ff4aaffd640 1 --2- 172.21.15.120:0/3196950209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4ac103780 0x7ff4ac198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:55.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.034+0000 7ff4ab7fe640 1 -- 172.21.15.120:0/3196950209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4ac108300 msgr2=0x7ff4ac19cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:55.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.034+0000 7ff4ab7fe640 1 --2- 172.21.15.120:0/3196950209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4ac108300 0x7ff4ac19cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.034+0000 7ff4ab7fe640 1 -- 172.21.15.120:0/3196950209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4ac103780 msgr2=0x7ff4ac198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:55.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.034+0000 7ff4ab7fe640 1 --2- 172.21.15.120:0/3196950209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4ac103780 0x7ff4ac198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.034+0000 7ff4ab7fe640 1 -- 172.21.15.120:0/3196950209 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ac19d5f0 con 0x7ff4ac102dd0 2024-09-19T09:06:55.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.034+0000 7ff4ab7fe640 1 --2- 172.21.15.120:0/3196950209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4ac102dd0 0x7ff4ac198640 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00e9f0 tx=0x7ff49c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:55.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.034+0000 7ff4a8ff9640 1 -- 172.21.15.120:0/3196950209 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49c00cd80 con 0x7ff4ac102dd0 2024-09-19T09:06:55.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.034+0000 7ff4b1fa4640 1 -- 172.21.15.120:0/3196950209 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4ac19d8e0 con 0x7ff4ac102dd0 2024-09-19T09:06:55.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.034+0000 7ff4a8ff9640 1 -- 172.21.15.120:0/3196950209 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff49c005500 con 0x7ff4ac102dd0 2024-09-19T09:06:55.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.034+0000 7ff4a8ff9640 1 -- 172.21.15.120:0/3196950209 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49c010690 con 0x7ff4ac102dd0 2024-09-19T09:06:55.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.034+0000 7ff4b1fa4640 1 -- 172.21.15.120:0/3196950209 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4ac1a51c0 con 0x7ff4ac102dd0 2024-09-19T09:06:55.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.037+0000 7ff4a8ff9640 1 -- 172.21.15.120:0/3196950209 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7ff49c0046c0 con 0x7ff4ac102dd0 2024-09-19T09:06:55.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.037+0000 7ff4b437d740 1 Processor -- start 2024-09-19T09:06:55.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.037+0000 7ff4b437d740 1 -- start start 2024-09-19T09:06:55.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.037+0000 7ff4b437d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e409898a40 0x55e409898e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:55.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.038+0000 7ff4b437d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e409899360 0x55e4098a7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:55.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.038+0000 7ff4b437d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4098a8150 0x55e4098aa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:55.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.038+0000 7ff4b437d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e4097fe280 con 0x55e409899360 2024-09-19T09:06:55.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.038+0000 7ff4b437d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e4097a5b10 con 0x55e4098a8150 2024-09-19T09:06:55.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.038+0000 7ff4b437d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e4097fa240 con 0x55e409898a40 2024-09-19T09:06:55.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.038+0000 7ff4ab7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e409899360 0x55e4098a7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:55.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.038+0000 7ff4ab7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e409899360 0x55e4098a7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42478/0 (socket says 172.21.15.120:42478) 2024-09-19T09:06:55.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.038+0000 7ff4aaffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4098a8150 0x55e4098aa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:55.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.038+0000 7ff4ab7fe640 1 -- 172.21.15.120:0/3610149168 learned_addr learned my addr 172.21.15.120:0/3610149168 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:55.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.038+0000 7ff4abfff640 1 --2- 172.21.15.120:0/3610149168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e409898a40 0x55e409898e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:55.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.038+0000 7ff4a8ff9640 1 --2- 172.21.15.120:0/3196950209 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff4940a32c0 0x7ff4940a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:55.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.038+0000 7ff4abfff640 1 --2- 172.21.15.120:0/3196950209 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff4940a32c0 0x7ff4940a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:55.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.038+0000 7ff4aaffd640 1 -- 172.21.15.120:0/3610149168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e409898a40 msgr2=0x55e409898e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:55.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.038+0000 7ff4aaffd640 1 --2- 172.21.15.120:0/3610149168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e409898a40 0x55e409898e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.038+0000 7ff4aaffd640 1 -- 172.21.15.120:0/3610149168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e409899360 msgr2=0x55e4098a7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:55.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.038+0000 7ff4aaffd640 1 --2- 172.21.15.120:0/3610149168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e409899360 0x55e4098a7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.038+0000 7ff4aaffd640 1 -- 172.21.15.120:0/3610149168 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4097b21f0 con 0x55e4098a8150 2024-09-19T09:06:55.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.038+0000 7ff4ab7fe640 1 --2- 172.21.15.120:0/3610149168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e409899360 0x55e4098a7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:55.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.038+0000 7ff4a8ff9640 1 -- 172.21.15.120:0/3196950209 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7ff49c0cfc90 con 0x7ff4ac102dd0 2024-09-19T09:06:55.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.038+0000 7ff4aaffd640 1 --2- 172.21.15.120:0/3610149168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4098a8150 0x55e4098aa540 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff4a002a8a0 tx=0x7ff4a0009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeb68fdaec666d4d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:55.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.038+0000 7ff4abfff640 1 --2- 172.21.15.120:0/3196950209 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff4940a32c0 0x7ff4940a5780 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff4a40049d0 tx=0x7ff4a4004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:55.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.039+0000 7ff491ffb640 1 -- 172.21.15.120:0/3610149168 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0037040 con 0x55e4098a8150 2024-09-19T09:06:55.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.039+0000 7ff491ffb640 1 -- 172.21.15.120:0/3610149168 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a0038040 con 0x55e4098a8150 2024-09-19T09:06:55.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.039+0000 7ff491ffb640 1 -- 172.21.15.120:0/3610149168 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a00035f0 con 0x55e4098a8150 2024-09-19T09:06:55.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.039+0000 7ff4b437d740 1 -- 172.21.15.120:0/3610149168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4098a8150 msgr2=0x55e4098aa540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:55.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.039+0000 7ff4b437d740 1 --2- 172.21.15.120:0/3610149168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4098a8150 0x55e4098aa540 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff4a002a8a0 tx=0x7ff4a0009490 comp rx=0 tx=0).stop 2024-09-19T09:06:55.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.039+0000 7ff4b437d740 1 -- 172.21.15.120:0/3610149168 shutdown_connections 2024-09-19T09:06:55.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.039+0000 7ff4b437d740 1 --2- 172.21.15.120:0/3610149168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4098a8150 0x55e4098aa540 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.039+0000 7ff4b437d740 1 --2- 172.21.15.120:0/3610149168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e409899360 0x55e4098a7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.039+0000 7ff4b437d740 1 --2- 172.21.15.120:0/3610149168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e409898a40 0x55e409898e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.039+0000 7ff4b437d740 1 -- 172.21.15.120:0/3610149168 >> 172.21.15.120:0/3610149168 conn(0x55e4098950f0 msgr2=0x55e4098954d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:55.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.039+0000 7ff4b437d740 1 -- 172.21.15.120:0/3610149168 shutdown_connections 2024-09-19T09:06:55.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.039+0000 7ff4b437d740 1 -- 172.21.15.120:0/3610149168 wait complete. 2024-09-19T09:06:55.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.040+0000 7ff4b437d740 1 -- start start 2024-09-19T09:06:55.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.040+0000 7ff4b437d740 1 Processor -- start 2024-09-19T09:06:55.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.040+0000 7ff4b437d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e409898a40 0x55e4098a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:55.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.040+0000 7ff4b437d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e409899360 0x55e4098a5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:55.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.040+0000 7ff4b437d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4098a8150 0x55e4098a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:55.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.040+0000 7ff4b437d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e4098aad20 con 0x55e409899360 2024-09-19T09:06:55.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.040+0000 7ff4b437d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e4097b8f10 con 0x55e4098a8150 2024-09-19T09:06:55.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.040+0000 7ff4b437d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e4098aaea0 con 0x55e409898a40 2024-09-19T09:06:55.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.040+0000 7ff4b437d740 10 client.0 _ll_register_callbacks cb 0x55e409781a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:55.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.040+0000 7ff4abfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4098a8150 0x55e4098a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:55.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.040+0000 7ff4aaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e409899360 0x55e4098a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:55.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.041+0000 7ff4abfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4098a8150 0x55e4098a5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:51650/0 (socket says 172.21.15.120:51650) 2024-09-19T09:06:55.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.041+0000 7ff4abfff640 1 -- 172.21.15.120:0/4229541814 learned_addr learned my addr 172.21.15.120:0/4229541814 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:55.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.041+0000 7ff4ab7fe640 1 --2- 172.21.15.120:0/4229541814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e409898a40 0x55e4098a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:55.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.041+0000 7ff4aaffd640 1 -- 172.21.15.120:0/4229541814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e409898a40 msgr2=0x55e4098a50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:55.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.041+0000 7ff4aaffd640 1 --2- 172.21.15.120:0/4229541814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e409898a40 0x55e4098a50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.041+0000 7ff4aaffd640 1 -- 172.21.15.120:0/4229541814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4098a8150 msgr2=0x55e4098a5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:55.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.041+0000 7ff4aaffd640 1 --2- 172.21.15.120:0/4229541814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4098a8150 0x55e4098a5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.041+0000 7ff4aaffd640 1 -- 172.21.15.120:0/4229541814 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e40956f6a0 con 0x55e409899360 2024-09-19T09:06:55.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.041+0000 7ff4abfff640 1 --2- 172.21.15.120:0/4229541814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4098a8150 0x55e4098a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:55.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.041+0000 7ff4aaffd640 1 --2- 172.21.15.120:0/4229541814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e409899360 0x55e4098a5620 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x55e409896470 tx=0x7ff4a002aa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ad4c6b3c357fad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:55.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.041+0000 7ff48affd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:06:55.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.041+0000 7ff48affd640 1 -- 172.21.15.120:0/4229541814 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a004d070 con 0x55e409899360 2024-09-19T09:06:55.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.041+0000 7ff48affd640 1 -- 172.21.15.120:0/4229541814 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7ff4a0004040 con 0x55e409899360 2024-09-19T09:06:55.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.041+0000 7ff4b437d740 1 -- 172.21.15.120:0/4229541814 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4a0038040 con 0x55e409899360 2024-09-19T09:06:55.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.042+0000 7ff48affd640 1 -- 172.21.15.120:0/4229541814 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff4a0058050 con 0x55e409899360 2024-09-19T09:06:55.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.042+0000 7ff4b437d740 10 client.16920 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:55.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.042+0000 7ff4b437d740 1 -- 172.21.15.120:0/4229541814 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e4098b6a10 con 0x55e409899360 2024-09-19T09:06:55.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.042+0000 7ff48affd640 1 -- 172.21.15.120:0/4229541814 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a0038040 con 0x55e409899360 2024-09-19T09:06:55.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.042+0000 7ff48affd640 1 -- 172.21.15.120:0/4229541814 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff4a0032440 con 0x55e409899360 2024-09-19T09:06:55.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.042+0000 7ff4b437d740 10 client.16920 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:55.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.042+0000 7ff4b437d740 10 client.16920 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:06:55.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.042+0000 7ff4b437d740 20 client.16920 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:55.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.042+0000 7ff4b437d740 1 --2- 172.21.15.120:0/4229541814 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e4098b7d60 0x55e4098ba150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:55.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.042+0000 7ff4b437d740 4 client.16920 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:06:55.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.042+0000 7ff4b437d740 1 -- 172.21.15.120:0/4229541814 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e40980b7f0 con 0x55e4098b7d60 2024-09-19T09:06:55.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.042+0000 7ff4ab7fe640 1 --2- 172.21.15.120:0/4229541814 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e4098b7d60 0x55e4098ba150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:55.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.042+0000 7ff48affd640 10 client.16920 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:55.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.043+0000 7ff4ab7fe640 1 --2- 172.21.15.120:0/4229541814 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e4098b7d60 0x55e4098ba150 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aedea63e7b70ca21 server_cookie=163f8d29f39ba49d in_seq=0 out_seq=0 2024-09-19T09:06:55.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.044+0000 7ff48affd640 1 -- 172.21.15.120:0/4229541814 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e40980b7f0 con 0x55e4098b7d60 2024-09-19T09:06:55.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.044+0000 7ff48affd640 10 client.16920 handle_command_reply: tid=0 2024-09-19T09:06:55.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.044+0000 7ff4b437d740 1 client.16920 shutdown 2024-09-19T09:06:55.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.044+0000 7ff4b437d740 1 -- 172.21.15.120:0/4229541814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e409899360 msgr2=0x55e4098a5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:55.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.044+0000 7ff4b437d740 1 --2- 172.21.15.120:0/4229541814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e409899360 0x55e4098a5620 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x55e409896470 tx=0x7ff4a002aa40 comp rx=0 tx=0).stop 2024-09-19T09:06:55.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.044+0000 7ff4b437d740 1 -- 172.21.15.120:0/4229541814 shutdown_connections 2024-09-19T09:06:55.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.044+0000 7ff4b437d740 1 --2- 172.21.15.120:0/4229541814 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e4098b7d60 0x55e4098ba150 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.044+0000 7ff4b437d740 1 --2- 172.21.15.120:0/4229541814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4098a8150 0x55e4098a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.044+0000 7ff4b437d740 1 --2- 172.21.15.120:0/4229541814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e409899360 0x55e4098a5620 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.044+0000 7ff4b437d740 1 --2- 172.21.15.120:0/4229541814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e409898a40 0x55e4098a50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.044+0000 7ff48affd640 0 client.16920 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:55.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.044+0000 7ff4b437d740 1 -- 172.21.15.120:0/4229541814 >> 172.21.15.120:0/4229541814 conn(0x55e4098950f0 msgr2=0x55e40989ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:55.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.044+0000 7ff4b437d740 1 -- 172.21.15.120:0/4229541814 shutdown_connections 2024-09-19T09:06:55.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.044+0000 7ff4b437d740 1 -- 172.21.15.120:0/4229541814 wait complete. 2024-09-19T09:06:55.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.044+0000 7ff4b437d740 20 client.16920 trim_cache size 0 max 16384 2024-09-19T09:06:55.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.063+0000 7ff4b437d740 1 Processor -- start 2024-09-19T09:06:55.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.063+0000 7ff4b437d740 1 -- start start 2024-09-19T09:06:55.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.063+0000 7ff4b437d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e409898a40 0x55e4098a6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:55.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.063+0000 7ff4b437d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e409899360 0x55e4098bd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:55.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.063+0000 7ff4b437d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4098a8150 0x55e4098bd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:55.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.063+0000 7ff4b437d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac1ab360 con 0x55e409898a40 2024-09-19T09:06:55.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.063+0000 7ff4b437d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac1acea0 con 0x55e4098a8150 2024-09-19T09:06:55.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.063+0000 7ff4b437d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac1aba30 con 0x55e409899360 2024-09-19T09:06:55.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.063+0000 7ff4ab7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e409898a40 0x55e4098a6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:55.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.063+0000 7ff4ab7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e409898a40 0x55e4098a6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42492/0 (socket says 172.21.15.120:42492) 2024-09-19T09:06:55.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.063+0000 7ff4ab7fe640 1 -- 172.21.15.120:0/1751483472 learned_addr learned my addr 172.21.15.120:0/1751483472 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:55.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.063+0000 7ff4aaffd640 1 --2- 172.21.15.120:0/1751483472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e409899360 0x55e4098bd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:55.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.063+0000 7ff4abfff640 1 --2- 172.21.15.120:0/1751483472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4098a8150 0x55e4098bd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:55.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.064+0000 7ff4aaffd640 1 -- 172.21.15.120:0/1751483472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4098a8150 msgr2=0x55e4098bd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:55.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.064+0000 7ff4aaffd640 1 --2- 172.21.15.120:0/1751483472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4098a8150 0x55e4098bd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.064+0000 7ff4aaffd640 1 -- 172.21.15.120:0/1751483472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e409898a40 msgr2=0x55e4098a6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:55.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.064+0000 7ff4aaffd640 1 --2- 172.21.15.120:0/1751483472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e409898a40 0x55e4098a6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.064+0000 7ff4aaffd640 1 -- 172.21.15.120:0/1751483472 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e40989b560 con 0x55e409899360 2024-09-19T09:06:55.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.064+0000 7ff4aaffd640 1 --2- 172.21.15.120:0/1751483472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e409899360 0x55e4098bd170 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff4a00097b0 tx=0x7ff4a0009450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a123717db7697ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:55.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.064+0000 7ff4917fa640 1 -- 172.21.15.120:0/1751483472 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a004d050 con 0x55e409899360 2024-09-19T09:06:55.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.064+0000 7ff4917fa640 1 -- 172.21.15.120:0/1751483472 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a00057c0 con 0x55e409899360 2024-09-19T09:06:55.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.064+0000 7ff4917fa640 1 -- 172.21.15.120:0/1751483472 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0037040 con 0x55e409899360 2024-09-19T09:06:55.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.065+0000 7ff4b437d740 1 -- 172.21.15.120:0/1751483472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e409899360 msgr2=0x55e4098bd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:55.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.065+0000 7ff4b437d740 1 --2- 172.21.15.120:0/1751483472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e409899360 0x55e4098bd170 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff4a00097b0 tx=0x7ff4a0009450 comp rx=0 tx=0).stop 2024-09-19T09:06:55.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.065+0000 7ff4b437d740 1 -- 172.21.15.120:0/1751483472 shutdown_connections 2024-09-19T09:06:55.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.065+0000 7ff4b437d740 1 --2- 172.21.15.120:0/1751483472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4098a8150 0x55e4098bd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.065+0000 7ff4b437d740 1 --2- 172.21.15.120:0/1751483472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e409899360 0x55e4098bd170 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.065+0000 7ff4b437d740 1 --2- 172.21.15.120:0/1751483472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e409898a40 0x55e4098a6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.065+0000 7ff4b437d740 1 -- 172.21.15.120:0/1751483472 >> 172.21.15.120:0/1751483472 conn(0x55e4098950f0 msgr2=0x55e4098a6830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:55.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.065+0000 7ff4b437d740 1 -- 172.21.15.120:0/1751483472 shutdown_connections 2024-09-19T09:06:55.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.065+0000 7ff4b437d740 1 -- 172.21.15.120:0/1751483472 wait complete. 2024-09-19T09:06:55.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.065+0000 7ff4b437d740 1 -- start start 2024-09-19T09:06:55.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.066+0000 7ff4b437d740 1 Processor -- start 2024-09-19T09:06:55.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.066+0000 7ff4b437d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e409898a40 0x55e4098bb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:55.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.066+0000 7ff4b437d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e409899360 0x55e4098bbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:55.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.066+0000 7ff4b437d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4098a8150 0x55e4099216a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:55.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.066+0000 7ff4b437d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac1ac7d0 con 0x55e4098a8150 2024-09-19T09:06:55.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.066+0000 7ff4b437d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac1ac100 con 0x55e409898a40 2024-09-19T09:06:55.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.066+0000 7ff4b437d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac1a9320 con 0x55e409899360 2024-09-19T09:06:55.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.066+0000 7ff4b437d740 10 client.0 _ll_register_callbacks cb 0x7ff49c0d35e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:55.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.066+0000 7ff4abfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4098a8150 0x55e4099216a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:55.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.066+0000 7ff4aaffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e409899360 0x55e4098bbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:55.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.066+0000 7ff4aaffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e409899360 0x55e4098bbb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:37342/0 (socket says 172.21.15.120:37342) 2024-09-19T09:06:55.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.066+0000 7ff4abfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4098a8150 0x55e4099216a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42500/0 (socket says 172.21.15.120:42500) 2024-09-19T09:06:55.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.066+0000 7ff4aaffd640 1 -- 172.21.15.120:0/1291412802 learned_addr learned my addr 172.21.15.120:0/1291412802 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:55.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.066+0000 7ff4ab7fe640 1 --2- 172.21.15.120:0/1291412802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e409898a40 0x55e4098bb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:55.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.067+0000 7ff4aaffd640 1 -- 172.21.15.120:0/1291412802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e409898a40 msgr2=0x55e4098bb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:55.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.067+0000 7ff4aaffd640 1 --2- 172.21.15.120:0/1291412802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e409898a40 0x55e4098bb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.067+0000 7ff4aaffd640 1 -- 172.21.15.120:0/1291412802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4098a8150 msgr2=0x55e4099216a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:55.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.067+0000 7ff4aaffd640 1 --2- 172.21.15.120:0/1291412802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4098a8150 0x55e4099216a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.067+0000 7ff4aaffd640 1 -- 172.21.15.120:0/1291412802 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e409921be0 con 0x55e409899360 2024-09-19T09:06:55.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.067+0000 7ff4aaffd640 1 --2- 172.21.15.120:0/1291412802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e409899360 0x55e4098bbb10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0005d10 tx=0x7ff4a0030b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d77bff253d6b5edd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:55.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.067+0000 7ff48affd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:06:55.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.067+0000 7ff48affd640 1 -- 172.21.15.120:0/1291412802 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0037070 con 0x55e409899360 2024-09-19T09:06:55.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.067+0000 7ff4b437d740 1 -- 172.21.15.120:0/1291412802 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4a00057c0 con 0x55e409899360 2024-09-19T09:06:55.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.067+0000 7ff48affd640 1 -- 172.21.15.120:0/1291412802 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7ff4a0004340 con 0x55e409899360 2024-09-19T09:06:55.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.067+0000 7ff48affd640 1 -- 172.21.15.120:0/1291412802 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff4a004d040 con 0x55e409899360 2024-09-19T09:06:55.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.067+0000 7ff4b437d740 10 client.26209 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:55.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.067+0000 7ff4b437d740 1 -- 172.21.15.120:0/1291412802 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e409922ea0 con 0x55e409899360 2024-09-19T09:06:55.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.068+0000 7ff48affd640 1 -- 172.21.15.120:0/1291412802 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a00057c0 con 0x55e409899360 2024-09-19T09:06:55.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.068+0000 7ff48affd640 1 -- 172.21.15.120:0/1291412802 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff4a0003a60 con 0x55e409899360 2024-09-19T09:06:55.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.068+0000 7ff4b437d740 10 client.26209 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:55.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.068+0000 7ff4b437d740 10 client.26209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:06:55.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.068+0000 7ff4b437d740 20 client.26209 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:55.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.068+0000 7ff4b437d740 1 --2- 172.21.15.120:0/1291412802 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e409923550 0x55e409925940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:55.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.068+0000 7ff4b437d740 4 client.26209 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:06:55.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.068+0000 7ff4b437d740 1 -- 172.21.15.120:0/1291412802 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e409805d00 con 0x55e409923550 2024-09-19T09:06:55.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.068+0000 7ff4ab7fe640 1 --2- 172.21.15.120:0/1291412802 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e409923550 0x55e409925940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:55.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.068+0000 7ff48affd640 10 client.26209 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:55.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.068+0000 7ff4ab7fe640 1 --2- 172.21.15.120:0/1291412802 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e409923550 0x55e409925940 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83b50bd9b7680c03 server_cookie=38e84dcbd2f69ea4 in_seq=0 out_seq=0 2024-09-19T09:06:55.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.070+0000 7ff48affd640 1 -- 172.21.15.120:0/1291412802 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e409805d00 con 0x55e409923550 2024-09-19T09:06:55.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.070+0000 7ff48affd640 10 client.26209 handle_command_reply: tid=0 2024-09-19T09:06:55.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.070+0000 7ff4b437d740 1 client.26209 shutdown 2024-09-19T09:06:55.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.070+0000 7ff4b437d740 1 -- 172.21.15.120:0/1291412802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e409899360 msgr2=0x55e4098bbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:55.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.070+0000 7ff4b437d740 1 --2- 172.21.15.120:0/1291412802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e409899360 0x55e4098bbb10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0005d10 tx=0x7ff4a0030b40 comp rx=0 tx=0).stop 2024-09-19T09:06:55.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.070+0000 7ff4b437d740 1 -- 172.21.15.120:0/1291412802 shutdown_connections 2024-09-19T09:06:55.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.070+0000 7ff4b437d740 1 --2- 172.21.15.120:0/1291412802 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e409923550 0x55e409925940 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.070+0000 7ff4b437d740 1 --2- 172.21.15.120:0/1291412802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4098a8150 0x55e4099216a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.070+0000 7ff4b437d740 1 --2- 172.21.15.120:0/1291412802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e409899360 0x55e4098bbb10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.070+0000 7ff48affd640 0 client.26209 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:55.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.070+0000 7ff4b437d740 1 --2- 172.21.15.120:0/1291412802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e409898a40 0x55e4098bb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.070+0000 7ff4b437d740 1 -- 172.21.15.120:0/1291412802 >> 172.21.15.120:0/1291412802 conn(0x55e4098950f0 msgr2=0x55e409894270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:55.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.070+0000 7ff4b437d740 1 -- 172.21.15.120:0/1291412802 shutdown_connections 2024-09-19T09:06:55.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.070+0000 7ff4b437d740 1 -- 172.21.15.120:0/1291412802 wait complete. 2024-09-19T09:06:55.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.070+0000 7ff4b437d740 20 client.26209 trim_cache size 0 max 16384 2024-09-19T09:06:55.080 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:06:55.080 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:06:55.080 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "b9be3b32-d4e7-4c55-91b2-8c5dac30c678", 2024-09-19T09:06:55.080 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:06:55.080 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:06:55.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.071+0000 7ff4b1fa4640 1 -- 172.21.15.120:0/3196950209 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff4940a32c0 msgr2=0x7ff4940a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:55.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.071+0000 7ff4b1fa4640 1 --2- 172.21.15.120:0/3196950209 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff4940a32c0 0x7ff4940a5780 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff4a40049d0 tx=0x7ff4a4004ab0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.071+0000 7ff4b1fa4640 1 -- 172.21.15.120:0/3196950209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4ac102dd0 msgr2=0x7ff4ac198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:55.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.071+0000 7ff4b1fa4640 1 --2- 172.21.15.120:0/3196950209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4ac102dd0 0x7ff4ac198640 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00e9f0 tx=0x7ff49c00edf0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.071+0000 7ff4b1fa4640 1 -- 172.21.15.120:0/3196950209 shutdown_connections 2024-09-19T09:06:55.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.071+0000 7ff4b1fa4640 1 --2- 172.21.15.120:0/3196950209 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff4940a32c0 0x7ff4940a5780 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.071+0000 7ff4b1fa4640 1 --2- 172.21.15.120:0/3196950209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4ac108300 0x7ff4ac19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.071+0000 7ff4b1fa4640 1 --2- 172.21.15.120:0/3196950209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4ac103780 0x7ff4ac198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.071+0000 7ff4b1fa4640 1 --2- 172.21.15.120:0/3196950209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4ac102dd0 0x7ff4ac198640 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:55.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.071+0000 7ff4b1fa4640 1 -- 172.21.15.120:0/3196950209 >> 172.21.15.120:0/3196950209 conn(0x7ff4ac0fc7d0 msgr2=0x7ff4ac105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:55.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.071+0000 7ff4b1fa4640 1 -- 172.21.15.120:0/3196950209 shutdown_connections 2024-09-19T09:06:55.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:55.071+0000 7ff4b1fa4640 1 -- 172.21.15.120:0/3196950209 wait complete. 2024-09-19T09:06:55.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:54 smithi196 ceph-mon[35150]: pgmap v462: 97 pgs: 97 active+clean; 36 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.0 MiB/s wr, 451 op/s 2024-09-19T09:06:55.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:54 smithi153 ceph-mon[35071]: pgmap v462: 97 pgs: 97 active+clean; 36 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.0 MiB/s wr, 451 op/s 2024-09-19T09:06:55.928 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:06:56.044 INFO:tasks.workunit.client.0.smithi120.stdout: 1 325688 10.36 MB/sec execute 51 sec latency 14.434 ms 2024-09-19T09:06:56.191 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:06:56.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.782+0000 7f39a61be640 1 -- 172.21.15.120:0/3827686408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39a0100e20 msgr2=0x7f39a0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:56.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.782+0000 7f39a61be640 1 --2- 172.21.15.120:0/3827686408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39a0100e20 0x7f39a0101200 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f3990009900 tx=0x7f399002a480 comp rx=0 tx=0).stop 2024-09-19T09:06:56.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.782+0000 7f39a61be640 1 -- 172.21.15.120:0/3827686408 shutdown_connections 2024-09-19T09:06:56.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.782+0000 7f39a61be640 1 --2- 172.21.15.120:0/3827686408 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39a0102ef0 0x7f39a010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.782+0000 7f39a61be640 1 --2- 172.21.15.120:0/3827686408 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39a0102570 0x7f39a01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.782+0000 7f39a61be640 1 --2- 172.21.15.120:0/3827686408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39a0100e20 0x7f39a0101200 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.783+0000 7f39a61be640 1 -- 172.21.15.120:0/3827686408 >> 172.21.15.120:0/3827686408 conn(0x7f39a00fc7d0 msgr2=0x7f39a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:56.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.783+0000 7f39a61be640 1 -- 172.21.15.120:0/3827686408 shutdown_connections 2024-09-19T09:06:56.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.783+0000 7f39a61be640 1 -- 172.21.15.120:0/3827686408 wait complete. 2024-09-19T09:06:56.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.783+0000 7f39a61be640 1 Processor -- start 2024-09-19T09:06:56.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.783+0000 7f39a61be640 1 -- start start 2024-09-19T09:06:56.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.784+0000 7f39a61be640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39a0100e20 0x7f39a0198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.784+0000 7f39a61be640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39a0102570 0x7f39a0198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.784+0000 7f39a61be640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39a0102ef0 0x7f39a019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.784+0000 7f39a61be640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f39a0110f80 con 0x7f39a0102570 2024-09-19T09:06:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.784+0000 7f39a61be640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f39a0110e00 con 0x7f39a0100e20 2024-09-19T09:06:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.784+0000 7f39a61be640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f39a0111100 con 0x7f39a0102ef0 2024-09-19T09:06:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.784+0000 7f399f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39a0100e20 0x7f39a0198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.784+0000 7f399effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39a0102570 0x7f39a0198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.784+0000 7f399f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39a0100e20 0x7f39a0198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:51682/0 (socket says 172.21.15.120:51682) 2024-09-19T09:06:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.784+0000 7f399f7fe640 1 -- 172.21.15.120:0/572800177 learned_addr learned my addr 172.21.15.120:0/572800177 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.784+0000 7f399effd640 1 -- 172.21.15.120:0/572800177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39a0102ef0 msgr2=0x7f39a019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.784+0000 7f399effd640 1 --2- 172.21.15.120:0/572800177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39a0102ef0 0x7f39a019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.784+0000 7f399effd640 1 -- 172.21.15.120:0/572800177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39a0100e20 msgr2=0x7f39a0198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.784+0000 7f399effd640 1 --2- 172.21.15.120:0/572800177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39a0100e20 0x7f39a0198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.784+0000 7f399effd640 1 -- 172.21.15.120:0/572800177 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a019d4b0 con 0x7f39a0102570 2024-09-19T09:06:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.785+0000 7f399effd640 1 --2- 172.21.15.120:0/572800177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39a0102570 0x7f39a0198a40 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f3994009bc0 tx=0x7f399400b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.787+0000 7f399cff9640 1 -- 172.21.15.120:0/572800177 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399401e070 con 0x7f39a0102570 2024-09-19T09:06:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.787+0000 7f399cff9640 1 -- 172.21.15.120:0/572800177 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39940198f0 con 0x7f39a0102570 2024-09-19T09:06:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.787+0000 7f399cff9640 1 -- 172.21.15.120:0/572800177 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39940187e0 con 0x7f39a0102570 2024-09-19T09:06:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.788+0000 7f39a61be640 1 -- 172.21.15.120:0/572800177 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a019d7a0 con 0x7f39a0102570 2024-09-19T09:06:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.788+0000 7f39a61be640 1 -- 172.21.15.120:0/572800177 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a00fcbb0 con 0x7f39a0102570 2024-09-19T09:06:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.791+0000 7f39a8597740 1 Processor -- start 2024-09-19T09:06:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.791+0000 7f39a8597740 1 -- start start 2024-09-19T09:06:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.791+0000 7f399cff9640 1 -- 172.21.15.120:0/572800177 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f3994021070 con 0x7f39a0102570 2024-09-19T09:06:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.791+0000 7f39a8597740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609256554a0 0x560925655880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.791+0000 7f39a8597740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560925655dc0 0x560925660140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.791+0000 7f39a8597740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560925660680 0x560925662a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.791+0000 7f39a8597740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5609255bb2e0 con 0x5609256554a0 2024-09-19T09:06:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.791+0000 7f39a8597740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560925562ce0 con 0x560925655dc0 2024-09-19T09:06:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.791+0000 7f39a8597740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5609255b72a0 con 0x560925660680 2024-09-19T09:06:56.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.791+0000 7f399ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609256554a0 0x560925655880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:56.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.791+0000 7f399ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609256554a0 0x560925655880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42534/0 (socket says 172.21.15.120:42534) 2024-09-19T09:06:56.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.791+0000 7f399ffff640 1 -- 172.21.15.120:0/12393390 learned_addr learned my addr 172.21.15.120:0/12393390 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:56.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.791+0000 7f399f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560925655dc0 0x560925660140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:56.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.791+0000 7f399effd640 1 --2- 172.21.15.120:0/12393390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560925660680 0x560925662a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:56.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.791+0000 7f399ffff640 1 -- 172.21.15.120:0/12393390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560925660680 msgr2=0x560925662a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:56.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.791+0000 7f399ffff640 1 --2- 172.21.15.120:0/12393390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560925660680 0x560925662a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.791+0000 7f399ffff640 1 -- 172.21.15.120:0/12393390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560925655dc0 msgr2=0x560925660140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:56.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.791+0000 7f399ffff640 1 --2- 172.21.15.120:0/12393390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560925655dc0 0x560925660140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.791+0000 7f399ffff640 1 -- 172.21.15.120:0/12393390 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560925547440 con 0x5609256554a0 2024-09-19T09:06:56.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.791+0000 7f399f7fe640 1 --2- 172.21.15.120:0/12393390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560925655dc0 0x560925660140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:56.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.792+0000 7f399ffff640 1 --2- 172.21.15.120:0/12393390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609256554a0 0x560925655880 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f3998009d80 tx=0x7f399800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70134e7646f8082 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:56.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.792+0000 7f3985ffb640 1 -- 172.21.15.120:0/12393390 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998016070 con 0x5609256554a0 2024-09-19T09:06:56.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.792+0000 7f399cff9640 1 --2- 172.21.15.120:0/572800177 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f39880a32c0 0x7f39880a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:56.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.792+0000 7f3985ffb640 1 -- 172.21.15.120:0/12393390 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3998005020 con 0x5609256554a0 2024-09-19T09:06:56.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.792+0000 7f3985ffb640 1 -- 172.21.15.120:0/12393390 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39980116b0 con 0x5609256554a0 2024-09-19T09:06:56.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.792+0000 7f399f7fe640 1 --2- 172.21.15.120:0/572800177 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f39880a32c0 0x7f39880a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:56.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.792+0000 7f399cff9640 1 -- 172.21.15.120:0/572800177 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f39940d2fb0 con 0x7f39a0102570 2024-09-19T09:06:56.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.793+0000 7f39a8597740 1 -- 172.21.15.120:0/12393390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609256554a0 msgr2=0x560925655880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:56.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.793+0000 7f39a8597740 1 --2- 172.21.15.120:0/12393390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609256554a0 0x560925655880 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f3998009d80 tx=0x7f399800e6c0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.793+0000 7f399f7fe640 1 --2- 172.21.15.120:0/572800177 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f39880a32c0 0x7f39880a5780 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x560925652890 tx=0x7f3990005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:56.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.793+0000 7f39a8597740 1 -- 172.21.15.120:0/12393390 shutdown_connections 2024-09-19T09:06:56.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.793+0000 7f39a8597740 1 --2- 172.21.15.120:0/12393390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560925660680 0x560925662a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.793+0000 7f39a8597740 1 --2- 172.21.15.120:0/12393390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560925655dc0 0x560925660140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.793+0000 7f39a8597740 1 --2- 172.21.15.120:0/12393390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609256554a0 0x560925655880 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.793+0000 7f39a8597740 1 -- 172.21.15.120:0/12393390 >> 172.21.15.120:0/12393390 conn(0x560925651a30 msgr2=0x560925651e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:56.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.793+0000 7f39a8597740 1 -- 172.21.15.120:0/12393390 shutdown_connections 2024-09-19T09:06:56.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.793+0000 7f39a8597740 1 -- 172.21.15.120:0/12393390 wait complete. 2024-09-19T09:06:56.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.793+0000 7f39a8597740 1 -- start start 2024-09-19T09:06:56.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.794+0000 7f39a8597740 1 Processor -- start 2024-09-19T09:06:56.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.794+0000 7f39a8597740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609256554a0 0x56092565fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:56.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.794+0000 7f39a8597740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560925655dc0 0x56092565d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:56.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.794+0000 7f39a8597740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560925660680 0x56092565d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:56.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.794+0000 7f399f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609256554a0 0x56092565fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:56.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.794+0000 7f399effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560925655dc0 0x56092565d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:56.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.794+0000 7f399f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609256554a0 0x56092565fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:37382/0 (socket says 172.21.15.120:37382) 2024-09-19T09:06:56.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.794+0000 7f399f7fe640 1 -- 172.21.15.120:0/1478399433 learned_addr learned my addr 172.21.15.120:0/1478399433 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:56.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.794+0000 7f399ffff640 1 --2- 172.21.15.120:0/1478399433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560925660680 0x56092565d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:56.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.794+0000 7f39a8597740 1 -- 172.21.15.120:0/1478399433 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560925663250 con 0x560925655dc0 2024-09-19T09:06:56.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.794+0000 7f39a8597740 1 -- 172.21.15.120:0/1478399433 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560925576050 con 0x560925660680 2024-09-19T09:06:56.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.794+0000 7f39a8597740 1 -- 172.21.15.120:0/1478399433 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5609256633d0 con 0x5609256554a0 2024-09-19T09:06:56.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.794+0000 7f39a8597740 10 client.0 _ll_register_callbacks cb 0x5609255b6160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:56.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.795+0000 7f399ffff640 1 -- 172.21.15.120:0/1478399433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609256554a0 msgr2=0x56092565fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:56.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.795+0000 7f399ffff640 1 --2- 172.21.15.120:0/1478399433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609256554a0 0x56092565fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.795+0000 7f399ffff640 1 -- 172.21.15.120:0/1478399433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560925655dc0 msgr2=0x56092565d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:56.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.795+0000 7f399ffff640 1 --2- 172.21.15.120:0/1478399433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560925655dc0 0x56092565d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.795+0000 7f399ffff640 1 -- 172.21.15.120:0/1478399433 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56092532c3c0 con 0x560925660680 2024-09-19T09:06:56.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.795+0000 7f39a8597740 1 -- 172.21.15.120:0/1478399433 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3998005020 con 0x560925660680 2024-09-19T09:06:56.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.795+0000 7f399effd640 1 --2- 172.21.15.120:0/1478399433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560925655dc0 0x56092565d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:06:56.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.795+0000 7f399ffff640 1 --2- 172.21.15.120:0/1478399433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560925660680 0x56092565d6e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f399800a010 tx=0x7f3998004e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaefbfc89873c5fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:56.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.795+0000 7f397effd640 10 client.26282 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:06:56.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.795+0000 7f397effd640 1 -- 172.21.15.120:0/1478399433 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998016050 con 0x560925660680 2024-09-19T09:06:56.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.795+0000 7f397effd640 1 -- 172.21.15.120:0/1478399433 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f3998003d80 con 0x560925660680 2024-09-19T09:06:56.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.795+0000 7f397effd640 1 -- 172.21.15.120:0/1478399433 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f39980051d0 con 0x560925660680 2024-09-19T09:06:56.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.795+0000 7f39a8597740 10 client.26282 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:56.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.795+0000 7f39a8597740 1 -- 172.21.15.120:0/1478399433 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({fsmap=28}) -- 0x56092566ea70 con 0x560925660680 2024-09-19T09:06:56.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.796+0000 7f397effd640 1 -- 172.21.15.120:0/1478399433 <== mon.2 v2:172.21.15.153:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f399802a030 con 0x560925660680 2024-09-19T09:06:56.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.796+0000 7f39a8597740 10 client.26282 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:56.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.796+0000 7f39a8597740 10 client.26282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:06:56.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.796+0000 7f39a8597740 20 client.26282 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:56.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.796+0000 7f39a8597740 1 --2- 172.21.15.120:0/1478399433 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56092566fe20 0x560925672210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:56.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.796+0000 7f39a8597740 4 client.26282 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:06:56.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.796+0000 7f39a8597740 1 -- 172.21.15.120:0/1478399433 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609255c8850 con 0x56092566fe20 2024-09-19T09:06:56.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.796+0000 7f399f7fe640 1 --2- 172.21.15.120:0/1478399433 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56092566fe20 0x560925672210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:56.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.797+0000 7f399f7fe640 1 --2- 172.21.15.120:0/1478399433 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56092566fe20 0x560925672210 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34680981e2f0fea2 server_cookie=79ba921328aff93f in_seq=0 out_seq=0 2024-09-19T09:06:56.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.797+0000 7f397effd640 10 client.26282 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:56.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.798+0000 7f397effd640 1 -- 172.21.15.120:0/1478399433 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5609255c8850 con 0x56092566fe20 2024-09-19T09:06:56.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.798+0000 7f397effd640 10 client.26282 handle_command_reply: tid=0 2024-09-19T09:06:56.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.798+0000 7f39a8597740 1 client.26282 shutdown 2024-09-19T09:06:56.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.798+0000 7f39a8597740 1 -- 172.21.15.120:0/1478399433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560925660680 msgr2=0x56092565d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:56.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.798+0000 7f39a8597740 1 --2- 172.21.15.120:0/1478399433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560925660680 0x56092565d6e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f399800a010 tx=0x7f3998004e90 comp rx=0 tx=0).stop 2024-09-19T09:06:56.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.798+0000 7f39a8597740 1 -- 172.21.15.120:0/1478399433 shutdown_connections 2024-09-19T09:06:56.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.798+0000 7f39a8597740 1 --2- 172.21.15.120:0/1478399433 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56092566fe20 0x560925672210 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.798+0000 7f39a8597740 1 --2- 172.21.15.120:0/1478399433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560925660680 0x56092565d6e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.798+0000 7f397effd640 0 client.26282 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:56.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.798+0000 7f39a8597740 1 --2- 172.21.15.120:0/1478399433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560925655dc0 0x56092565d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.798+0000 7f39a8597740 1 --2- 172.21.15.120:0/1478399433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609256554a0 0x56092565fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.798+0000 7f39a8597740 1 -- 172.21.15.120:0/1478399433 >> 172.21.15.120:0/1478399433 conn(0x560925651a30 msgr2=0x560925650b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:56.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.798+0000 7f39a8597740 1 -- 172.21.15.120:0/1478399433 shutdown_connections 2024-09-19T09:06:56.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.798+0000 7f39a8597740 1 -- 172.21.15.120:0/1478399433 wait complete. 2024-09-19T09:06:56.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.798+0000 7f39a8597740 20 client.26282 trim_cache size 0 max 16384 2024-09-19T09:06:56.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.817+0000 7f39a8597740 1 Processor -- start 2024-09-19T09:06:56.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.817+0000 7f39a8597740 1 -- start start 2024-09-19T09:06:56.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.817+0000 7f39a8597740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609256554a0 0x5609256588c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:56.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.817+0000 7f39a8597740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560925655dc0 0x56092565c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:56.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.817+0000 7f39a8597740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560925660680 0x5609256779a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:56.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.817+0000 7f39a8597740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f39a01ab350 con 0x5609256554a0 2024-09-19T09:06:56.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.817+0000 7f39a8597740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f39a01ace90 con 0x560925660680 2024-09-19T09:06:56.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.817+0000 7f39a8597740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f39a01aba20 con 0x560925655dc0 2024-09-19T09:06:56.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.817+0000 7f399f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609256554a0 0x5609256588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:56.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.818+0000 7f399f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609256554a0 0x5609256588c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42550/0 (socket says 172.21.15.120:42550) 2024-09-19T09:06:56.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.818+0000 7f399f7fe640 1 -- 172.21.15.120:0/797333856 learned_addr learned my addr 172.21.15.120:0/797333856 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:56.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.818+0000 7f399ffff640 1 --2- 172.21.15.120:0/797333856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560925660680 0x5609256779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.818+0000 7f399effd640 1 --2- 172.21.15.120:0/797333856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560925655dc0 0x56092565c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.818+0000 7f399f7fe640 1 -- 172.21.15.120:0/797333856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560925655dc0 msgr2=0x56092565c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.818+0000 7f399f7fe640 1 --2- 172.21.15.120:0/797333856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560925655dc0 0x56092565c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.818+0000 7f399f7fe640 1 -- 172.21.15.120:0/797333856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560925660680 msgr2=0x5609256779a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.818+0000 7f399f7fe640 1 --2- 172.21.15.120:0/797333856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560925660680 0x5609256779a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.818+0000 7f399f7fe640 1 -- 172.21.15.120:0/797333856 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560925652ae0 con 0x5609256554a0 2024-09-19T09:06:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.818+0000 7f399f7fe640 1 --2- 172.21.15.120:0/797333856 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609256554a0 0x5609256588c0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f3990005ab0 tx=0x7f399003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf51b7903e9124c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.818+0000 7f39857fa640 1 -- 172.21.15.120:0/797333856 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3990035040 con 0x5609256554a0 2024-09-19T09:06:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.818+0000 7f39857fa640 1 -- 172.21.15.120:0/797333856 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3990059070 con 0x5609256554a0 2024-09-19T09:06:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.818+0000 7f39857fa640 1 -- 172.21.15.120:0/797333856 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3990054800 con 0x5609256554a0 2024-09-19T09:06:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.819+0000 7f39a8597740 1 -- 172.21.15.120:0/797333856 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609256554a0 msgr2=0x5609256588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.819+0000 7f39a8597740 1 --2- 172.21.15.120:0/797333856 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609256554a0 0x5609256588c0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f3990005ab0 tx=0x7f399003f000 comp rx=0 tx=0).stop 2024-09-19T09:06:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.819+0000 7f39a8597740 1 -- 172.21.15.120:0/797333856 shutdown_connections 2024-09-19T09:06:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.819+0000 7f39a8597740 1 --2- 172.21.15.120:0/797333856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560925660680 0x5609256779a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.819+0000 7f39a8597740 1 --2- 172.21.15.120:0/797333856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560925655dc0 0x56092565c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.819+0000 7f39a8597740 1 --2- 172.21.15.120:0/797333856 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609256554a0 0x5609256588c0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.819+0000 7f39a8597740 1 -- 172.21.15.120:0/797333856 >> 172.21.15.120:0/797333856 conn(0x560925651a30 msgr2=0x56092565ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:56.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.819+0000 7f39a8597740 1 -- 172.21.15.120:0/797333856 shutdown_connections 2024-09-19T09:06:56.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.819+0000 7f39a8597740 1 -- 172.21.15.120:0/797333856 wait complete. 2024-09-19T09:06:56.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.819+0000 7f39a8597740 1 -- start start 2024-09-19T09:06:56.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.820+0000 7f39a8597740 1 Processor -- start 2024-09-19T09:06:56.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.820+0000 7f39a8597740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609256554a0 0x5609256688e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:56.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.820+0000 7f39a8597740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560925655dc0 0x560925668e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:56.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.820+0000 7f39a8597740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560925660680 0x560925659af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:56.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.820+0000 7f39a8597740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f39a01ac7c0 con 0x560925660680 2024-09-19T09:06:56.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.820+0000 7f39a8597740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f39a01ac0f0 con 0x560925655dc0 2024-09-19T09:06:56.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.820+0000 7f39a8597740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f39a01a9330 con 0x5609256554a0 2024-09-19T09:06:56.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.820+0000 7f39a8597740 10 client.0 _ll_register_callbacks cb 0x7f3990009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:06:56.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.820+0000 7f399ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560925660680 0x560925659af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:56.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.820+0000 7f399f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609256554a0 0x5609256688e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:56.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.820+0000 7f399effd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560925655dc0 0x560925668e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:56.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.820+0000 7f399ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560925660680 0x560925659af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42564/0 (socket says 172.21.15.120:42564) 2024-09-19T09:06:56.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.820+0000 7f399ffff640 1 -- 172.21.15.120:0/3444209457 learned_addr learned my addr 172.21.15.120:0/3444209457 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:06:56.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.820+0000 7f399f7fe640 1 -- 172.21.15.120:0/3444209457 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560925655dc0 msgr2=0x560925668e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:56.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.820+0000 7f399f7fe640 1 --2- 172.21.15.120:0/3444209457 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560925655dc0 0x560925668e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.820+0000 7f399f7fe640 1 -- 172.21.15.120:0/3444209457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560925660680 msgr2=0x560925659af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:06:56.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.820+0000 7f399f7fe640 1 --2- 172.21.15.120:0/3444209457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560925660680 0x560925659af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.820+0000 7f399f7fe640 1 -- 172.21.15.120:0/3444209457 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56092565a320 con 0x5609256554a0 2024-09-19T09:06:56.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.821+0000 7f399f7fe640 1 --2- 172.21.15.120:0/3444209457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609256554a0 0x5609256688e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3990054fd0 tx=0x7f3990003d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=550f48bde8a59ec5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:06:56.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.821+0000 7f397effd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:06:56.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.821+0000 7f397effd640 1 -- 172.21.15.120:0/3444209457 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3990062070 con 0x5609256554a0 2024-09-19T09:06:56.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.821+0000 7f39a8597740 1 -- 172.21.15.120:0/3444209457 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3990059070 con 0x5609256554a0 2024-09-19T09:06:56.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.821+0000 7f397effd640 1 -- 172.21.15.120:0/3444209457 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f3990031950 con 0x5609256554a0 2024-09-19T09:06:56.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.821+0000 7f397effd640 1 -- 172.21.15.120:0/3444209457 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3990031b50 con 0x5609256554a0 2024-09-19T09:06:56.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.821+0000 7f39a8597740 10 client.26224 fetch_fsmap learned FSMap version 28 2024-09-19T09:06:56.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.821+0000 7f39a8597740 1 -- 172.21.15.120:0/3444209457 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5609256da500 con 0x5609256554a0 2024-09-19T09:06:56.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.821+0000 7f397effd640 1 -- 172.21.15.120:0/3444209457 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3990059070 con 0x5609256554a0 2024-09-19T09:06:56.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.821+0000 7f397effd640 1 -- 172.21.15.120:0/3444209457 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f39900674f0 con 0x5609256554a0 2024-09-19T09:06:56.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.821+0000 7f39a8597740 10 client.26224 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:06:56.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.821+0000 7f39a8597740 10 client.26224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:06:56.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.821+0000 7f39a8597740 20 client.26224 populate_metadata read hostname 'smithi120' 2024-09-19T09:06:56.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.821+0000 7f39a8597740 1 --2- 172.21.15.120:0/3444209457 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5609256dabd0 0x5609256dcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:06:56.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.821+0000 7f39a8597740 4 client.26224 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:06:56.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.821+0000 7f39a8597740 1 -- 172.21.15.120:0/3444209457 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56092565da70 con 0x5609256dabd0 2024-09-19T09:06:56.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.822+0000 7f399effd640 1 --2- 172.21.15.120:0/3444209457 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5609256dabd0 0x5609256dcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:06:56.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.822+0000 7f397effd640 10 client.26224 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:56.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.822+0000 7f399effd640 1 --2- 172.21.15.120:0/3444209457 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5609256dabd0 0x5609256dcfc0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c375d5f7cc93ab3 server_cookie=d301374e9daa911e in_seq=0 out_seq=0 2024-09-19T09:06:56.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.822+0000 7f397effd640 1 -- 172.21.15.120:0/3444209457 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56092565da70 con 0x5609256dabd0 2024-09-19T09:06:56.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.822+0000 7f397effd640 10 client.26224 handle_command_reply: tid=0 2024-09-19T09:06:56.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.822+0000 7f39a8597740 1 client.26224 shutdown 2024-09-19T09:06:56.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.822+0000 7f39a8597740 1 -- 172.21.15.120:0/3444209457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609256554a0 msgr2=0x5609256688e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:56.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.822+0000 7f39a8597740 1 --2- 172.21.15.120:0/3444209457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609256554a0 0x5609256688e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3990054fd0 tx=0x7f3990003d80 comp rx=0 tx=0).stop 2024-09-19T09:06:56.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.823+0000 7f39a8597740 1 -- 172.21.15.120:0/3444209457 shutdown_connections 2024-09-19T09:06:56.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.823+0000 7f39a8597740 1 --2- 172.21.15.120:0/3444209457 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5609256dabd0 0x5609256dcfc0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.823+0000 7f39a8597740 1 --2- 172.21.15.120:0/3444209457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560925660680 0x560925659af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.823+0000 7f397effd640 0 client.26224 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:06:56.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.823+0000 7f39a8597740 1 --2- 172.21.15.120:0/3444209457 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560925655dc0 0x560925668e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.823+0000 7f39a8597740 1 --2- 172.21.15.120:0/3444209457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609256554a0 0x5609256688e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.823+0000 7f39a8597740 1 -- 172.21.15.120:0/3444209457 >> 172.21.15.120:0/3444209457 conn(0x560925651a30 msgr2=0x56092565ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:56.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.823+0000 7f39a8597740 1 -- 172.21.15.120:0/3444209457 shutdown_connections 2024-09-19T09:06:56.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.823+0000 7f39a8597740 1 -- 172.21.15.120:0/3444209457 wait complete. 2024-09-19T09:06:56.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.823+0000 7f39a8597740 20 client.26224 trim_cache size 0 max 16384 2024-09-19T09:06:56.835 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:06:56.835 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:06:56.835 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-19T09:06:56.835 INFO:teuthology.orchestra.run.smithi120.stdout: "b9be3b32-d4e7-4c55-91b2-8c5dac30c678": { 2024-09-19T09:06:56.835 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-19T09:06:56.835 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "b9be3b32-d4e7-4c55-91b2-8c5dac30c678", 2024-09-19T09:06:56.835 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-19T09:06:56.836 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:06:56.836 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:06:56.836 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:06:56.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.823+0000 7f39a61be640 1 -- 172.21.15.120:0/572800177 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f39880a32c0 msgr2=0x7f39880a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:56.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.823+0000 7f39a61be640 1 --2- 172.21.15.120:0/572800177 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f39880a32c0 0x7f39880a5780 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x560925652890 tx=0x7f3990005a20 comp rx=0 tx=0).stop 2024-09-19T09:06:56.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.823+0000 7f39a61be640 1 -- 172.21.15.120:0/572800177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39a0102570 msgr2=0x7f39a0198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:06:56.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.823+0000 7f39a61be640 1 --2- 172.21.15.120:0/572800177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39a0102570 0x7f39a0198a40 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f3994009bc0 tx=0x7f399400b690 comp rx=0 tx=0).stop 2024-09-19T09:06:56.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.823+0000 7f39a61be640 1 -- 172.21.15.120:0/572800177 shutdown_connections 2024-09-19T09:06:56.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.823+0000 7f39a61be640 1 --2- 172.21.15.120:0/572800177 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f39880a32c0 0x7f39880a5780 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.823+0000 7f39a61be640 1 --2- 172.21.15.120:0/572800177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39a0102ef0 0x7f39a019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.823+0000 7f39a61be640 1 --2- 172.21.15.120:0/572800177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39a0102570 0x7f39a0198a40 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.823+0000 7f39a61be640 1 --2- 172.21.15.120:0/572800177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39a0100e20 0x7f39a0198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:06:56.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.823+0000 7f39a61be640 1 -- 172.21.15.120:0/572800177 >> 172.21.15.120:0/572800177 conn(0x7f39a00fc7d0 msgr2=0x7f39a006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:06:56.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.824+0000 7f39a61be640 1 -- 172.21.15.120:0/572800177 shutdown_connections 2024-09-19T09:06:56.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:06:56.824+0000 7f39a61be640 1 -- 172.21.15.120:0/572800177 wait complete. 2024-09-19T09:06:57.044 INFO:tasks.workunit.client.0.smithi120.stdout: 1 327996 10.40 MB/sec execute 52 sec latency 23.726 ms 2024-09-19T09:06:57.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:56 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:06:57.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:56 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:06:57.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:56 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:06:57.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:56 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:06:57.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:56 smithi120 ceph-mon[31150]: pgmap v463: 97 pgs: 97 active+clean; 36 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.0 MiB/s wr, 374 op/s 2024-09-19T09:06:57.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:56 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:06:57.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:56 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:06:57.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:56 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:06:57.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:56 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:06:57.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:56 smithi196 ceph-mon[35150]: pgmap v463: 97 pgs: 97 active+clean; 36 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.0 MiB/s wr, 374 op/s 2024-09-19T09:06:57.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:56 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:06:57.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:56 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:06:57.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:56 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:06:57.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:56 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:06:57.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:56 smithi153 ceph-mon[35071]: pgmap v463: 97 pgs: 97 active+clean; 36 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.0 MiB/s wr, 374 op/s 2024-09-19T09:06:57.663 INFO:tasks.cephfs.filesystem:scrub status for tag:b9be3b32-d4e7-4c55-91b2-8c5dac30c678 - {'path': '/', 'tag': 'b9be3b32-d4e7-4c55-91b2-8c5dac30c678', 'options': 'recursive,force'} 2024-09-19T09:06:58.044 INFO:tasks.workunit.client.0.smithi120.stdout: 1 330935 10.50 MB/sec execute 53 sec latency 11.266 ms 2024-09-19T09:06:59.044 INFO:tasks.workunit.client.0.smithi120.stdout: 1 334112 10.63 MB/sec execute 54 sec latency 14.857 ms 2024-09-19T09:06:59.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:58 smithi120 ceph-mon[31150]: pgmap v464: 97 pgs: 97 active+clean; 40 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.2 MiB/s wr, 430 op/s 2024-09-19T09:06:59.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:06:58 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:06:59.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:58 smithi196 ceph-mon[35150]: pgmap v464: 97 pgs: 97 active+clean; 40 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.2 MiB/s wr, 430 op/s 2024-09-19T09:06:59.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:06:58 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:06:59.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:58 smithi153 ceph-mon[35071]: pgmap v464: 97 pgs: 97 active+clean; 40 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.2 MiB/s wr, 430 op/s 2024-09-19T09:06:59.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:06:58 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:07:00.044 INFO:tasks.workunit.client.0.smithi120.stdout: 1 337364 10.78 MB/sec execute 55 sec latency 11.009 ms 2024-09-19T09:07:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.365+0000 7fa5bc931640 1 -- 172.21.15.120:0/2131898439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5b8101c40 msgr2=0x7fa5b81020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.365+0000 7fa5bc931640 1 --2- 172.21.15.120:0/2131898439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5b8101c40 0x7fa5b81020c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac009900 tx=0x7fa5ac02a410 comp rx=0 tx=0).stop 2024-09-19T09:07:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.365+0000 7fa5bc931640 1 -- 172.21.15.120:0/2131898439 shutdown_connections 2024-09-19T09:07:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.365+0000 7fa5bc931640 1 --2- 172.21.15.120:0/2131898439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5b8102600 0x7fa5b8108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.365+0000 7fa5bc931640 1 --2- 172.21.15.120:0/2131898439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5b8101c40 0x7fa5b81020c0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.365+0000 7fa5bc931640 1 --2- 172.21.15.120:0/2131898439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5b8100a50 0x7fa5b8100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.365+0000 7fa5bc931640 1 -- 172.21.15.120:0/2131898439 >> 172.21.15.120:0/2131898439 conn(0x7fa5b80fc1b0 msgr2=0x7fa5b80fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.365+0000 7fa5bc931640 1 -- 172.21.15.120:0/2131898439 shutdown_connections 2024-09-19T09:07:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.365+0000 7fa5bc931640 1 -- 172.21.15.120:0/2131898439 wait complete. 2024-09-19T09:07:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.366+0000 7fa5bc931640 1 Processor -- start 2024-09-19T09:07:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.366+0000 7fa5bc931640 1 -- start start 2024-09-19T09:07:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.366+0000 7fa5bc931640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5b8100a50 0x7fa5b806aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.366+0000 7fa5bc931640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5b8101c40 0x7fa5b819cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.366+0000 7fa5bc931640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5b8102600 0x7fa5b819f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.366+0000 7fa5bc931640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b810c6d0 con 0x7fa5b8100a50 2024-09-19T09:07:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.366+0000 7fa5bc931640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b810c550 con 0x7fa5b8102600 2024-09-19T09:07:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.366+0000 7fa5bc931640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b810c850 con 0x7fa5b8101c40 2024-09-19T09:07:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.366+0000 7fa5b659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5b8100a50 0x7fa5b806aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.366+0000 7fa5b659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5b8100a50 0x7fa5b806aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42584/0 (socket says 172.21.15.120:42584) 2024-09-19T09:07:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.366+0000 7fa5b659c640 1 -- 172.21.15.120:0/2955952715 learned_addr learned my addr 172.21.15.120:0/2955952715 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.366+0000 7fa5b5d9b640 1 --2- 172.21.15.120:0/2955952715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5b8101c40 0x7fa5b819cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.367+0000 7fa5b6d9d640 1 --2- 172.21.15.120:0/2955952715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5b8102600 0x7fa5b819f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.367+0000 7fa5b659c640 1 -- 172.21.15.120:0/2955952715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5b8101c40 msgr2=0x7fa5b819cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.367+0000 7fa5b659c640 1 --2- 172.21.15.120:0/2955952715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5b8101c40 0x7fa5b819cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.367+0000 7fa5b659c640 1 -- 172.21.15.120:0/2955952715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5b8102600 msgr2=0x7fa5b819f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.367+0000 7fa5b659c640 1 --2- 172.21.15.120:0/2955952715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5b8102600 0x7fa5b819f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.367+0000 7fa5b659c640 1 -- 172.21.15.120:0/2955952715 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b819fa40 con 0x7fa5b8100a50 2024-09-19T09:07:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.367+0000 7fa5b659c640 1 --2- 172.21.15.120:0/2955952715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5b8100a50 0x7fa5b806aeb0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa5a800e9c0 tx=0x7fa5a800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.367+0000 7fa5a77fe640 1 -- 172.21.15.120:0/2955952715 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a800cd80 con 0x7fa5b8100a50 2024-09-19T09:07:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.367+0000 7fa5a77fe640 1 -- 172.21.15.120:0/2955952715 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a80035f0 con 0x7fa5b8100a50 2024-09-19T09:07:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.367+0000 7fa5bc931640 1 -- 172.21.15.120:0/2955952715 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5b819fcd0 con 0x7fa5b8100a50 2024-09-19T09:07:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.367+0000 7fa5a77fe640 1 -- 172.21.15.120:0/2955952715 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8010690 con 0x7fa5b8100a50 2024-09-19T09:07:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.367+0000 7fa5bc931640 1 -- 172.21.15.120:0/2955952715 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5b81a01a0 con 0x7fa5b8100a50 2024-09-19T09:07:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.369+0000 7fa5a77fe640 1 -- 172.21.15.120:0/2955952715 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa5a80037a0 con 0x7fa5b8100a50 2024-09-19T09:07:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.369+0000 7fa5bf6d2740 1 Processor -- start 2024-09-19T09:07:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.370+0000 7fa5bf6d2740 1 -- start start 2024-09-19T09:07:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.370+0000 7fa5bf6d2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fb25899a0 0x556fb2589da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.370+0000 7fa5bf6d2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fb258a2e0 0x556fb258eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.370+0000 7fa5bf6d2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556fb258f370 0x556fb2591830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.370+0000 7fa5bf6d2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556fb24ed2f0 con 0x556fb25899a0 2024-09-19T09:07:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.370+0000 7fa5bf6d2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556fb2495270 con 0x556fb258a2e0 2024-09-19T09:07:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.370+0000 7fa5bf6d2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556fb24e92b0 con 0x556fb258f370 2024-09-19T09:07:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.370+0000 7fa5b6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fb25899a0 0x556fb2589da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.370+0000 7fa5b6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fb25899a0 0x556fb2589da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42590/0 (socket says 172.21.15.120:42590) 2024-09-19T09:07:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.370+0000 7fa5b6d9d640 1 -- 172.21.15.120:0/1144248197 learned_addr learned my addr 172.21.15.120:0/1144248197 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.370+0000 7fa5b659c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fb258a2e0 0x556fb258eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.370+0000 7fa5b5d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556fb258f370 0x556fb2591830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.370+0000 7fa5a77fe640 1 --2- 172.21.15.120:0/2955952715 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa5a00a32f0 0x7fa5a00a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.370+0000 7fa5b6d9d640 1 --2- 172.21.15.120:0/2955952715 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa5a00a32f0 0x7fa5a00a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.370+0000 7fa5b6d9d640 1 -- 172.21.15.120:0/1144248197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556fb258f370 msgr2=0x556fb2591830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.370+0000 7fa5b6d9d640 1 --2- 172.21.15.120:0/1144248197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556fb258f370 0x556fb2591830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.370+0000 7fa5b6d9d640 1 -- 172.21.15.120:0/1144248197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fb258a2e0 msgr2=0x556fb258eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.370+0000 7fa5b6d9d640 1 --2- 172.21.15.120:0/1144248197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fb258a2e0 0x556fb258eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.370+0000 7fa5a77fe640 1 -- 172.21.15.120:0/2955952715 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa5a80cfc60 con 0x7fa5b8100a50 2024-09-19T09:07:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.370+0000 7fa5b6d9d640 1 -- 172.21.15.120:0/1144248197 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fb2494340 con 0x556fb25899a0 2024-09-19T09:07:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.370+0000 7fa5b5d9b640 1 --2- 172.21.15.120:0/1144248197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556fb258f370 0x556fb2591830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:07:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.371+0000 7fa5b6d9d640 1 --2- 172.21.15.120:0/1144248197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fb25899a0 0x556fb2589da0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0005f40 tx=0x7fa5b0014f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3b194411057611 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.371+0000 7fa5a4ff9640 1 -- 172.21.15.120:0/1144248197 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0012d90 con 0x556fb25899a0 2024-09-19T09:07:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.371+0000 7fa5a4ff9640 1 -- 172.21.15.120:0/1144248197 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b0023070 con 0x556fb25899a0 2024-09-19T09:07:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.371+0000 7fa5a4ff9640 1 -- 172.21.15.120:0/1144248197 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0005750 con 0x556fb25899a0 2024-09-19T09:07:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.372+0000 7fa5b6d9d640 1 --2- 172.21.15.120:0/2955952715 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa5a00a32f0 0x7fa5a00a57b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0022000 tx=0x7fa5b0020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.372+0000 7fa5bf6d2740 1 -- 172.21.15.120:0/1144248197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fb25899a0 msgr2=0x556fb2589da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.372+0000 7fa5bf6d2740 1 --2- 172.21.15.120:0/1144248197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fb25899a0 0x556fb2589da0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0005f40 tx=0x7fa5b0014f70 comp rx=0 tx=0).stop 2024-09-19T09:07:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.372+0000 7fa5bf6d2740 1 -- 172.21.15.120:0/1144248197 shutdown_connections 2024-09-19T09:07:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.372+0000 7fa5bf6d2740 1 --2- 172.21.15.120:0/1144248197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556fb258f370 0x556fb2591830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.372+0000 7fa5bf6d2740 1 --2- 172.21.15.120:0/1144248197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fb258a2e0 0x556fb258eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.372+0000 7fa5bf6d2740 1 --2- 172.21.15.120:0/1144248197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fb25899a0 0x556fb2589da0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.372+0000 7fa5bf6d2740 1 -- 172.21.15.120:0/1144248197 >> 172.21.15.120:0/1144248197 conn(0x556fb2585010 msgr2=0x556fb2585410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.372+0000 7fa5bf6d2740 1 -- 172.21.15.120:0/1144248197 shutdown_connections 2024-09-19T09:07:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.372+0000 7fa5bf6d2740 1 -- 172.21.15.120:0/1144248197 wait complete. 2024-09-19T09:07:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.372+0000 7fa5bf6d2740 1 -- start start 2024-09-19T09:07:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.372+0000 7fa5bf6d2740 1 Processor -- start 2024-09-19T09:07:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.372+0000 7fa5bf6d2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fb25899a0 0x556fb2593c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.372+0000 7fa5bf6d2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fb258a2e0 0x556fb2594140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.372+0000 7fa5bf6d2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556fb258f370 0x556fb259b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.372+0000 7fa5bf6d2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556fb2592340 con 0x556fb258a2e0 2024-09-19T09:07:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.372+0000 7fa5bf6d2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556fb2476290 con 0x556fb25899a0 2024-09-19T09:07:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.372+0000 7fa5bf6d2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556fb25924c0 con 0x556fb258f370 2024-09-19T09:07:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.372+0000 7fa5bf6d2740 10 client.0 _ll_register_callbacks cb 0x556fb24a7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.372+0000 7fa5b5d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fb258a2e0 0x556fb2594140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.373+0000 7fa5b5d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fb258a2e0 0x556fb2594140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42600/0 (socket says 172.21.15.120:42600) 2024-09-19T09:07:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.373+0000 7fa5b5d9b640 1 -- 172.21.15.120:0/796296078 learned_addr learned my addr 172.21.15.120:0/796296078 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.373+0000 7fa5b659c640 1 --2- 172.21.15.120:0/796296078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fb25899a0 0x556fb2593c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.373+0000 7fa5b6d9d640 1 --2- 172.21.15.120:0/796296078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556fb258f370 0x556fb259b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.373+0000 7fa5b5d9b640 1 -- 172.21.15.120:0/796296078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556fb258f370 msgr2=0x556fb259b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.373+0000 7fa5b5d9b640 1 --2- 172.21.15.120:0/796296078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556fb258f370 0x556fb259b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.373+0000 7fa5b5d9b640 1 -- 172.21.15.120:0/796296078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fb25899a0 msgr2=0x556fb2593c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.373+0000 7fa5b5d9b640 1 --2- 172.21.15.120:0/796296078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fb25899a0 0x556fb2593c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.373+0000 7fa5b5d9b640 1 -- 172.21.15.120:0/796296078 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fb2200210 con 0x556fb258a2e0 2024-09-19T09:07:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.373+0000 7fa5b5d9b640 1 --2- 172.21.15.120:0/796296078 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fb258a2e0 0x556fb2594140 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac02a930 tx=0x7fa5ac0318e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc0e73a84178149 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.373+0000 7fa59dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:07:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.373+0000 7fa59dffb640 1 -- 172.21.15.120:0/796296078 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac035040 con 0x556fb258a2e0 2024-09-19T09:07:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.373+0000 7fa5bf6d2740 1 -- 172.21.15.120:0/796296078 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5b0023070 con 0x556fb258a2e0 2024-09-19T09:07:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.373+0000 7fa59dffb640 1 -- 172.21.15.120:0/796296078 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa5ac004040 con 0x556fb258a2e0 2024-09-19T09:07:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.373+0000 7fa59dffb640 1 -- 172.21.15.120:0/796296078 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa5ac004240 con 0x556fb258a2e0 2024-09-19T09:07:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.373+0000 7fa5bf6d2740 10 client.16956 fetch_fsmap learned FSMap version 28 2024-09-19T09:07:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.373+0000 7fa5bf6d2740 1 -- 172.21.15.120:0/796296078 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556fb22cc620 con 0x556fb258a2e0 2024-09-19T09:07:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.373+0000 7fa59dffb640 1 -- 172.21.15.120:0/796296078 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b0023070 con 0x556fb258a2e0 2024-09-19T09:07:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.374+0000 7fa59dffb640 1 -- 172.21.15.120:0/796296078 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa5ac004ae0 con 0x556fb258a2e0 2024-09-19T09:07:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.374+0000 7fa5bf6d2740 10 client.16956 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:07:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.374+0000 7fa5bf6d2740 10 client.16956 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:07:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.374+0000 7fa5bf6d2740 20 client.16956 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.374+0000 7fa5bf6d2740 1 --2- 172.21.15.120:0/796296078 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556fb259f7b0 0x556fb25a1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.374+0000 7fa5bf6d2740 4 client.16956 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:07:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.374+0000 7fa5bf6d2740 1 -- 172.21.15.120:0/796296078 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fb24faa20 con 0x556fb259f7b0 2024-09-19T09:07:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.374+0000 7fa5b659c640 1 --2- 172.21.15.120:0/796296078 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556fb259f7b0 0x556fb25a1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.374+0000 7fa59dffb640 10 client.16956 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.374+0000 7fa5b659c640 1 --2- 172.21.15.120:0/796296078 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556fb259f7b0 0x556fb25a1ba0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87a9e357267392d9 server_cookie=7a90e342122e3d0b in_seq=0 out_seq=0 2024-09-19T09:07:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.376+0000 7fa59dffb640 1 -- 172.21.15.120:0/796296078 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556fb24faa20 con 0x556fb259f7b0 2024-09-19T09:07:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.376+0000 7fa59dffb640 10 client.16956 handle_command_reply: tid=0 2024-09-19T09:07:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.376+0000 7fa5bf6d2740 1 client.16956 shutdown 2024-09-19T09:07:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.376+0000 7fa5bf6d2740 1 -- 172.21.15.120:0/796296078 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fb258a2e0 msgr2=0x556fb2594140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.376+0000 7fa5bf6d2740 1 --2- 172.21.15.120:0/796296078 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fb258a2e0 0x556fb2594140 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac02a930 tx=0x7fa5ac0318e0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.376+0000 7fa5bf6d2740 1 -- 172.21.15.120:0/796296078 shutdown_connections 2024-09-19T09:07:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.376+0000 7fa5bf6d2740 1 --2- 172.21.15.120:0/796296078 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556fb259f7b0 0x556fb25a1ba0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.376+0000 7fa5bf6d2740 1 --2- 172.21.15.120:0/796296078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556fb258f370 0x556fb259b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.376+0000 7fa59dffb640 0 client.16956 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.376+0000 7fa5bf6d2740 1 --2- 172.21.15.120:0/796296078 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fb258a2e0 0x556fb2594140 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.376+0000 7fa5bf6d2740 1 --2- 172.21.15.120:0/796296078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fb25899a0 0x556fb2593c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.376+0000 7fa5bf6d2740 1 -- 172.21.15.120:0/796296078 >> 172.21.15.120:0/796296078 conn(0x556fb2585010 msgr2=0x556fb2586820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.376+0000 7fa5bf6d2740 1 -- 172.21.15.120:0/796296078 shutdown_connections 2024-09-19T09:07:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.376+0000 7fa5bf6d2740 1 -- 172.21.15.120:0/796296078 wait complete. 2024-09-19T09:07:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.376+0000 7fa5bf6d2740 20 client.16956 trim_cache size 0 max 16384 2024-09-19T09:07:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.391+0000 7fa5bf6d2740 1 Processor -- start 2024-09-19T09:07:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.391+0000 7fa5bf6d2740 1 -- start start 2024-09-19T09:07:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.391+0000 7fa5bf6d2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fb25899a0 0x556fb2589d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.391+0000 7fa5bf6d2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fb258a2e0 0x556fb259b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.391+0000 7fa5bf6d2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556fb258f370 0x556fb25840b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.391+0000 7fa5bf6d2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b81a8f20 con 0x556fb25899a0 2024-09-19T09:07:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.391+0000 7fa5bf6d2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b81aaa60 con 0x556fb258a2e0 2024-09-19T09:07:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.391+0000 7fa5bf6d2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b81a95f0 con 0x556fb258f370 2024-09-19T09:07:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.391+0000 7fa5b659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fb25899a0 0x556fb2589d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.391+0000 7fa5b659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fb25899a0 0x556fb2589d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42614/0 (socket says 172.21.15.120:42614) 2024-09-19T09:07:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.391+0000 7fa5b659c640 1 -- 172.21.15.120:0/3821196782 learned_addr learned my addr 172.21.15.120:0/3821196782 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.391+0000 7fa5b5d9b640 1 --2- 172.21.15.120:0/3821196782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fb258a2e0 0x556fb259b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.391+0000 7fa5b659c640 1 -- 172.21.15.120:0/3821196782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556fb258f370 msgr2=0x556fb25840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.391+0000 7fa5b659c640 1 --2- 172.21.15.120:0/3821196782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556fb258f370 0x556fb25840b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.391+0000 7fa5b659c640 1 -- 172.21.15.120:0/3821196782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fb258a2e0 msgr2=0x556fb259b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.391+0000 7fa5b659c640 1 --2- 172.21.15.120:0/3821196782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fb258a2e0 0x556fb259b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.391+0000 7fa5b659c640 1 -- 172.21.15.120:0/3821196782 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fb2584e30 con 0x556fb25899a0 2024-09-19T09:07:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.391+0000 7fa5b659c640 1 --2- 172.21.15.120:0/3821196782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fb25899a0 0x556fb2589d80 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80986e0 tx=0x7fa5a8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bee93d0c89b1c21 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.391+0000 7fa59ffff640 1 -- 172.21.15.120:0/3821196782 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a809c980 con 0x556fb25899a0 2024-09-19T09:07:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.392+0000 7fa59ffff640 1 -- 172.21.15.120:0/3821196782 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a809c280 con 0x556fb25899a0 2024-09-19T09:07:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.392+0000 7fa59ffff640 1 -- 172.21.15.120:0/3821196782 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a80a4a60 con 0x556fb25899a0 2024-09-19T09:07:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.392+0000 7fa5bf6d2740 1 -- 172.21.15.120:0/3821196782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fb25899a0 msgr2=0x556fb2589d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.392+0000 7fa5bf6d2740 1 --2- 172.21.15.120:0/3821196782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fb25899a0 0x556fb2589d80 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80986e0 tx=0x7fa5a8003f10 comp rx=0 tx=0).stop 2024-09-19T09:07:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.392+0000 7fa5bf6d2740 1 -- 172.21.15.120:0/3821196782 shutdown_connections 2024-09-19T09:07:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.392+0000 7fa5bf6d2740 1 --2- 172.21.15.120:0/3821196782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556fb258f370 0x556fb25840b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.392+0000 7fa5bf6d2740 1 --2- 172.21.15.120:0/3821196782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fb258a2e0 0x556fb259b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.392+0000 7fa5bf6d2740 1 --2- 172.21.15.120:0/3821196782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fb25899a0 0x556fb2589d80 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.392+0000 7fa5bf6d2740 1 -- 172.21.15.120:0/3821196782 >> 172.21.15.120:0/3821196782 conn(0x556fb2585010 msgr2=0x556fb25839a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.392+0000 7fa5bf6d2740 1 -- 172.21.15.120:0/3821196782 shutdown_connections 2024-09-19T09:07:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.392+0000 7fa5bf6d2740 1 -- 172.21.15.120:0/3821196782 wait complete. 2024-09-19T09:07:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.392+0000 7fa5bf6d2740 1 -- start start 2024-09-19T09:07:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.393+0000 7fa5bf6d2740 1 Processor -- start 2024-09-19T09:07:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.393+0000 7fa5bf6d2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fb25899a0 0x556fb25982e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.393+0000 7fa5bf6d2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556fb258a2e0 0x556fb2598820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.393+0000 7fa5bf6d2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fb258f370 0x556fb25997e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.393+0000 7fa5bf6d2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b81aa390 con 0x556fb25899a0 2024-09-19T09:07:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.393+0000 7fa5bf6d2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b81a9cc0 con 0x556fb258f370 2024-09-19T09:07:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.393+0000 7fa5bf6d2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b81a6ee0 con 0x556fb258a2e0 2024-09-19T09:07:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.393+0000 7fa5bf6d2740 10 client.0 _ll_register_callbacks cb 0x7fa5ac009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.393+0000 7fa5b659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fb25899a0 0x556fb25982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.393+0000 7fa5b659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fb25899a0 0x556fb25982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42626/0 (socket says 172.21.15.120:42626) 2024-09-19T09:07:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.393+0000 7fa5b659c640 1 -- 172.21.15.120:0/739801440 learned_addr learned my addr 172.21.15.120:0/739801440 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.393+0000 7fa5b6d9d640 1 --2- 172.21.15.120:0/739801440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fb258f370 0x556fb25997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.393+0000 7fa5b5d9b640 1 --2- 172.21.15.120:0/739801440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556fb258a2e0 0x556fb2598820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.393+0000 7fa5b659c640 1 -- 172.21.15.120:0/739801440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556fb258a2e0 msgr2=0x556fb2598820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.393+0000 7fa5b659c640 1 --2- 172.21.15.120:0/739801440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556fb258a2e0 0x556fb2598820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.393+0000 7fa5b659c640 1 -- 172.21.15.120:0/739801440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fb258f370 msgr2=0x556fb25997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.393+0000 7fa5b659c640 1 --2- 172.21.15.120:0/739801440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fb258f370 0x556fb25997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.393+0000 7fa5b659c640 1 -- 172.21.15.120:0/739801440 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fb2599df0 con 0x556fb25899a0 2024-09-19T09:07:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.393+0000 7fa5b5d9b640 1 --2- 172.21.15.120:0/739801440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556fb258a2e0 0x556fb2598820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:07:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.393+0000 7fa5b659c640 1 --2- 172.21.15.120:0/739801440 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fb25899a0 0x556fb25982e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80a49c0 tx=0x7fa5a80a4fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6cb6d42cfb95500 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.393+0000 7fa59dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:07:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.393+0000 7fa59dffb640 1 -- 172.21.15.120:0/739801440 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a80b4070 con 0x556fb25899a0 2024-09-19T09:07:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.394+0000 7fa59dffb640 1 -- 172.21.15.120:0/739801440 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa5a80cd870 con 0x556fb25899a0 2024-09-19T09:07:00.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.394+0000 7fa5bf6d2740 1 -- 172.21.15.120:0/739801440 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5a809c280 con 0x556fb25899a0 2024-09-19T09:07:00.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.394+0000 7fa59dffb640 1 -- 172.21.15.120:0/739801440 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa5a80b5260 con 0x556fb25899a0 2024-09-19T09:07:00.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.394+0000 7fa5bf6d2740 10 client.16962 fetch_fsmap learned FSMap version 28 2024-09-19T09:07:00.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.394+0000 7fa5bf6d2740 1 -- 172.21.15.120:0/739801440 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556fb259b100 con 0x556fb25899a0 2024-09-19T09:07:00.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.394+0000 7fa59dffb640 1 -- 172.21.15.120:0/739801440 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a809c280 con 0x556fb25899a0 2024-09-19T09:07:00.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.394+0000 7fa59dffb640 1 -- 172.21.15.120:0/739801440 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa5a80bc080 con 0x556fb25899a0 2024-09-19T09:07:00.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.394+0000 7fa5bf6d2740 10 client.16962 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:07:00.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.394+0000 7fa5bf6d2740 10 client.16962 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:07:00.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.394+0000 7fa5bf6d2740 20 client.16962 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:00.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.394+0000 7fa5bf6d2740 1 --2- 172.21.15.120:0/739801440 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556fb260a8e0 0x556fb260cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:00.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.394+0000 7fa5bf6d2740 4 client.16962 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:07:00.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.394+0000 7fa5bf6d2740 1 -- 172.21.15.120:0/739801440 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556fb24f4d70 con 0x556fb260a8e0 2024-09-19T09:07:00.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.394+0000 7fa5b5d9b640 1 --2- 172.21.15.120:0/739801440 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556fb260a8e0 0x556fb260cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.395+0000 7fa59dffb640 10 client.16962 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.395+0000 7fa5b5d9b640 1 --2- 172.21.15.120:0/739801440 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556fb260a8e0 0x556fb260cd00 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4384c848f6e08bf5 server_cookie=16b62ba5c92cd1 in_seq=0 out_seq=0 2024-09-19T09:07:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.542+0000 7fa59dffb640 1 -- 172.21.15.120:0/739801440 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556fb24f4d70 con 0x556fb260a8e0 2024-09-19T09:07:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.542+0000 7fa59dffb640 10 client.16962 handle_command_reply: tid=0 2024-09-19T09:07:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.542+0000 7fa5bf6d2740 1 client.16962 shutdown 2024-09-19T09:07:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.542+0000 7fa5bf6d2740 1 -- 172.21.15.120:0/739801440 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fb25899a0 msgr2=0x556fb25982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.542+0000 7fa5bf6d2740 1 --2- 172.21.15.120:0/739801440 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fb25899a0 0x556fb25982e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80a49c0 tx=0x7fa5a80a4fe0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.542+0000 7fa5bf6d2740 1 -- 172.21.15.120:0/739801440 shutdown_connections 2024-09-19T09:07:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.542+0000 7fa5bf6d2740 1 --2- 172.21.15.120:0/739801440 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556fb260a8e0 0x556fb260cd00 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.542+0000 7fa5bf6d2740 1 --2- 172.21.15.120:0/739801440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fb258f370 0x556fb25997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.542+0000 7fa5bf6d2740 1 --2- 172.21.15.120:0/739801440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556fb258a2e0 0x556fb2598820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.542+0000 7fa59dffb640 0 client.16962 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.542+0000 7fa5bf6d2740 1 --2- 172.21.15.120:0/739801440 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fb25899a0 0x556fb25982e0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.542+0000 7fa5bf6d2740 1 -- 172.21.15.120:0/739801440 >> 172.21.15.120:0/739801440 conn(0x556fb2585010 msgr2=0x556fb25841c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.542+0000 7fa5bf6d2740 1 -- 172.21.15.120:0/739801440 shutdown_connections 2024-09-19T09:07:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.542+0000 7fa5bf6d2740 1 -- 172.21.15.120:0/739801440 wait complete. 2024-09-19T09:07:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.542+0000 7fa5bf6d2740 20 client.16962 trim_cache size 0 max 16384 2024-09-19T09:07:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:07:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:07:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:07:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:07:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.543+0000 7fa5bc931640 1 -- 172.21.15.120:0/2955952715 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa5a00a32f0 msgr2=0x7fa5a00a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.543+0000 7fa5bc931640 1 --2- 172.21.15.120:0/2955952715 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa5a00a32f0 0x7fa5a00a57b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0022000 tx=0x7fa5b0020000 comp rx=0 tx=0).stop 2024-09-19T09:07:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.543+0000 7fa5bc931640 1 -- 172.21.15.120:0/2955952715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5b8100a50 msgr2=0x7fa5b806aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.543+0000 7fa5bc931640 1 --2- 172.21.15.120:0/2955952715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5b8100a50 0x7fa5b806aeb0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa5a800e9c0 tx=0x7fa5a800ee80 comp rx=0 tx=0).stop 2024-09-19T09:07:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.543+0000 7fa5bc931640 1 -- 172.21.15.120:0/2955952715 shutdown_connections 2024-09-19T09:07:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.543+0000 7fa5bc931640 1 --2- 172.21.15.120:0/2955952715 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa5a00a32f0 0x7fa5a00a57b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.543+0000 7fa5bc931640 1 --2- 172.21.15.120:0/2955952715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5b8102600 0x7fa5b819f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.543+0000 7fa5bc931640 1 --2- 172.21.15.120:0/2955952715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5b8101c40 0x7fa5b819cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.543+0000 7fa5bc931640 1 --2- 172.21.15.120:0/2955952715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5b8100a50 0x7fa5b806aeb0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.543+0000 7fa5bc931640 1 -- 172.21.15.120:0/2955952715 >> 172.21.15.120:0/2955952715 conn(0x7fa5b80fc1b0 msgr2=0x7fa5b8104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.543+0000 7fa5bc931640 1 -- 172.21.15.120:0/2955952715 shutdown_connections 2024-09-19T09:07:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:00.543+0000 7fa5bc931640 1 -- 172.21.15.120:0/2955952715 wait complete. 2024-09-19T09:07:01.044 INFO:tasks.workunit.client.0.smithi120.stdout: 1 340324 10.85 MB/sec execute 56 sec latency 98.424 ms 2024-09-19T09:07:01.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:00 smithi120 ceph-mon[31150]: pgmap v465: 97 pgs: 97 active+clean; 44 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.6 MiB/s wr, 412 op/s 2024-09-19T09:07:01.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:00 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:07:01.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:00 smithi196 ceph-mon[35150]: pgmap v465: 97 pgs: 97 active+clean; 44 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.6 MiB/s wr, 412 op/s 2024-09-19T09:07:01.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:00 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:07:01.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:00 smithi153 ceph-mon[35071]: pgmap v465: 97 pgs: 97 active+clean; 44 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.6 MiB/s wr, 412 op/s 2024-09-19T09:07:01.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:00 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:07:02.044 INFO:tasks.workunit.client.0.smithi120.stdout: 1 342887 10.86 MB/sec execute 57 sec latency 40.823 ms 2024-09-19T09:07:02.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:01 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:07:02.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:01 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:07:02.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:01 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:07:02.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:01 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:07:02.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:01 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:07:02.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:01 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:07:02.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:01 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:07:02.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:01 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:07:02.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:01 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:07:03.044 INFO:tasks.workunit.client.0.smithi120.stdout: 1 345555 10.97 MB/sec execute 58 sec latency 11.397 ms 2024-09-19T09:07:03.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:02 smithi120 ceph-mon[31150]: pgmap v466: 97 pgs: 97 active+clean; 42 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.4 MiB/s wr, 389 op/s 2024-09-19T09:07:03.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:02 smithi196 ceph-mon[35150]: pgmap v466: 97 pgs: 97 active+clean; 42 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.4 MiB/s wr, 389 op/s 2024-09-19T09:07:03.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:02 smithi153 ceph-mon[35071]: pgmap v466: 97 pgs: 97 active+clean; 42 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.4 MiB/s wr, 389 op/s 2024-09-19T09:07:04.044 INFO:tasks.workunit.client.0.smithi120.stdout: 1 348523 11.04 MB/sec execute 59 sec latency 9.715 ms 2024-09-19T09:07:04.817 INFO:tasks.quiescer.fs.[cephfs]:Total duration (59.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T09:07:04.817 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.8 2024-09-19T09:07:04.817 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:07:05.044 INFO:tasks.workunit.client.0.smithi120.stdout: 1 351733 11.16 MB/sec execute 60 sec latency 9.312 ms 2024-09-19T09:07:05.074 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:07:05.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:04 smithi120 ceph-mon[31150]: pgmap v467: 97 pgs: 97 active+clean; 40 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.8 MiB/s wr, 348 op/s 2024-09-19T09:07:05.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:04 smithi196 ceph-mon[35150]: pgmap v467: 97 pgs: 97 active+clean; 40 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.8 MiB/s wr, 348 op/s 2024-09-19T09:07:05.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:04 smithi153 ceph-mon[35071]: pgmap v467: 97 pgs: 97 active+clean; 40 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.8 MiB/s wr, 348 op/s 2024-09-19T09:07:05.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.862+0000 7f64ab20a640 1 -- 172.21.15.120:0/2091067560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64a410ae80 msgr2=0x7f64a4113370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:05.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.862+0000 7f64ab20a640 1 --2- 172.21.15.120:0/2091067560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64a410ae80 0x7f64a4113370 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f6490009900 tx=0x7f649002e800 comp rx=0 tx=0).stop 2024-09-19T09:07:05.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.862+0000 7f64ab20a640 1 -- 172.21.15.120:0/2091067560 shutdown_connections 2024-09-19T09:07:05.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.862+0000 7f64ab20a640 1 --2- 172.21.15.120:0/2091067560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64a41139e0 0x7f64a4115e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:05.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.862+0000 7f64ab20a640 1 --2- 172.21.15.120:0/2091067560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64a410ae80 0x7f64a4113370 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:05.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.862+0000 7f64ab20a640 1 --2- 172.21.15.120:0/2091067560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64a410a560 0x7f64a410a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:05.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.862+0000 7f64ab20a640 1 -- 172.21.15.120:0/2091067560 >> 172.21.15.120:0/2091067560 conn(0x7f64a406e2d0 msgr2=0x7f64a406e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:05.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.862+0000 7f64ab20a640 1 -- 172.21.15.120:0/2091067560 shutdown_connections 2024-09-19T09:07:05.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.862+0000 7f64ab20a640 1 -- 172.21.15.120:0/2091067560 wait complete. 2024-09-19T09:07:05.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.863+0000 7f64ab20a640 1 Processor -- start 2024-09-19T09:07:05.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.863+0000 7f64ab20a640 1 -- start start 2024-09-19T09:07:05.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.863+0000 7f64ab20a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64a410a560 0x7f64a41a47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:05.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.863+0000 7f64ab20a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64a410ae80 0x7f64a41a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:05.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.863+0000 7f64ab20a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64a41139e0 0x7f64a419e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:05.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.863+0000 7f64ab20a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f64a41185e0 con 0x7f64a410a560 2024-09-19T09:07:05.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.863+0000 7f64ab20a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f64a4118460 con 0x7f64a41139e0 2024-09-19T09:07:05.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.863+0000 7f64ab20a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f64a4118760 con 0x7f64a410ae80 2024-09-19T09:07:05.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.863+0000 7f64a8fa6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64a410a560 0x7f64a41a47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:05.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.863+0000 7f64a8fa6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64a410a560 0x7f64a41a47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33784/0 (socket says 172.21.15.120:33784) 2024-09-19T09:07:05.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.863+0000 7f64a8fa6640 1 -- 172.21.15.120:0/2300603844 learned_addr learned my addr 172.21.15.120:0/2300603844 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:05.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.863+0000 7f64a3fff640 1 --2- 172.21.15.120:0/2300603844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64a410ae80 0x7f64a41a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:05.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.863+0000 7f64a97a7640 1 --2- 172.21.15.120:0/2300603844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64a41139e0 0x7f64a419e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:05.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.863+0000 7f64a8fa6640 1 -- 172.21.15.120:0/2300603844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64a410ae80 msgr2=0x7f64a41a4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:05.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.863+0000 7f64a8fa6640 1 --2- 172.21.15.120:0/2300603844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64a410ae80 0x7f64a41a4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:05.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.863+0000 7f64a8fa6640 1 -- 172.21.15.120:0/2300603844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64a41139e0 msgr2=0x7f64a419e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:05.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.863+0000 7f64a8fa6640 1 --2- 172.21.15.120:0/2300603844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64a41139e0 0x7f64a419e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:05.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.863+0000 7f64a8fa6640 1 -- 172.21.15.120:0/2300603844 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a419f0d0 con 0x7f64a410a560 2024-09-19T09:07:05.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.863+0000 7f64a3fff640 1 --2- 172.21.15.120:0/2300603844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64a410ae80 0x7f64a41a4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:07:05.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.863+0000 7f64a97a7640 1 --2- 172.21.15.120:0/2300603844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64a41139e0 0x7f64a419e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:07:05.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.863+0000 7f64a8fa6640 1 --2- 172.21.15.120:0/2300603844 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64a410a560 0x7f64a41a47e0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f6498009990 tx=0x7f6498009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:05.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.864+0000 7f64a1ffb640 1 -- 172.21.15.120:0/2300603844 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649801b070 con 0x7f64a410a560 2024-09-19T09:07:05.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.864+0000 7f64a1ffb640 1 -- 172.21.15.120:0/2300603844 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6498016410 con 0x7f64a410a560 2024-09-19T09:07:05.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.864+0000 7f64ab20a640 1 -- 172.21.15.120:0/2300603844 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64a419f3c0 con 0x7f64a410a560 2024-09-19T09:07:05.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.864+0000 7f64a1ffb640 1 -- 172.21.15.120:0/2300603844 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6498015610 con 0x7f64a410a560 2024-09-19T09:07:05.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.864+0000 7f64ab20a640 1 -- 172.21.15.120:0/2300603844 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64a41ab6b0 con 0x7f64a410a560 2024-09-19T09:07:05.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.865+0000 7f64ab20a640 1 -- 172.21.15.120:0/2300603844 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64a410a940 con 0x7f64a410a560 2024-09-19T09:07:05.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.867+0000 7f64a1ffb640 1 -- 172.21.15.120:0/2300603844 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f649801e070 con 0x7f64a410a560 2024-09-19T09:07:05.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.867+0000 7f64a1ffb640 1 --2- 172.21.15.120:0/2300603844 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f64940a3240 0x7f64940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:05.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.868+0000 7f64a3fff640 1 --2- 172.21.15.120:0/2300603844 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f64940a3240 0x7f64940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:05.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.868+0000 7f64a1ffb640 1 -- 172.21.15.120:0/2300603844 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f64980ceff0 con 0x7f64a410a560 2024-09-19T09:07:05.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.868+0000 7f64a3fff640 1 --2- 172.21.15.120:0/2300603844 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f64940a3240 0x7f64940a5700 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f649002edf0 tx=0x7f6490005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:05.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:05.872+0000 7f64a1ffb640 1 -- 172.21.15.120:0/2300603844 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f649809b6c0 con 0x7f64a410a560 2024-09-19T09:07:06.044 INFO:tasks.workunit.client.0.smithi120.stdout: 1 354790 11.27 MB/sec execute 61 sec latency 60.917 ms 2024-09-19T09:07:06.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:06.123+0000 7f64ab20a640 1 -- 172.21.15.120:0/2300603844 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64a410ea60 con 0x7f64a410a560 2024-09-19T09:07:06.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:06.124+0000 7f64a1ffb640 1 -- 172.21.15.120:0/2300603844 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f649809f9c0 con 0x7f64a410a560 2024-09-19T09:07:06.125 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:07:06.125 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":28,"btime":"2024-09-19T09:02:41:696728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:02:41.207357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3103515283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3103515283},{"type":"v1","addr":"172.21.15.153:6835","nonce":3103515283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:07:06.125 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 28 2024-09-19T09:07:06.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:06.126+0000 7f64ab20a640 1 -- 172.21.15.120:0/2300603844 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f64940a3240 msgr2=0x7f64940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:06.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:06.126+0000 7f64ab20a640 1 --2- 172.21.15.120:0/2300603844 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f64940a3240 0x7f64940a5700 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f649002edf0 tx=0x7f6490005a20 comp rx=0 tx=0).stop 2024-09-19T09:07:06.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:06.126+0000 7f64ab20a640 1 -- 172.21.15.120:0/2300603844 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64a410a560 msgr2=0x7f64a41a47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:06.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:06.126+0000 7f64ab20a640 1 --2- 172.21.15.120:0/2300603844 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64a410a560 0x7f64a41a47e0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f6498009990 tx=0x7f6498009e50 comp rx=0 tx=0).stop 2024-09-19T09:07:06.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:06.126+0000 7f64ab20a640 1 -- 172.21.15.120:0/2300603844 shutdown_connections 2024-09-19T09:07:06.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:06.126+0000 7f64ab20a640 1 --2- 172.21.15.120:0/2300603844 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f64940a3240 0x7f64940a5700 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:06.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:06.126+0000 7f64ab20a640 1 --2- 172.21.15.120:0/2300603844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64a41139e0 0x7f64a419e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:06.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:06.126+0000 7f64ab20a640 1 --2- 172.21.15.120:0/2300603844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64a410ae80 0x7f64a41a4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:06.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:06.126+0000 7f64ab20a640 1 --2- 172.21.15.120:0/2300603844 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64a410a560 0x7f64a41a47e0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:06.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:06.126+0000 7f64ab20a640 1 -- 172.21.15.120:0/2300603844 >> 172.21.15.120:0/2300603844 conn(0x7f64a406e2d0 msgr2=0x7f64a4071df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:06.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:06.126+0000 7f64ab20a640 1 -- 172.21.15.120:0/2300603844 shutdown_connections 2024-09-19T09:07:06.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:06.126+0000 7f64ab20a640 1 -- 172.21.15.120:0/2300603844 wait complete. 2024-09-19T09:07:06.985 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0' 2024-09-19T09:07:06.985 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0 2024-09-19T09:07:07.044 INFO:tasks.workunit.client.0.smithi120.stdout: 1 357698 11.31 MB/sec execute 62 sec latency 9.943 ms 2024-09-19T09:07:07.074 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:06 smithi120 ceph-mon[31150]: pgmap v468: 97 pgs: 97 active+clean; 41 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 417 op/s 2024-09-19T09:07:07.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:06 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2300603844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:07:07.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:06 smithi196 ceph-mon[35150]: pgmap v468: 97 pgs: 97 active+clean; 41 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 417 op/s 2024-09-19T09:07:07.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:06 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2300603844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:07:07.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:06 smithi153 ceph-mon[35071]: pgmap v468: 97 pgs: 97 active+clean; 41 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 417 op/s 2024-09-19T09:07:07.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:06 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2300603844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:07:07.255 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:07:07.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.840+0000 7faac50c6640 1 -- 172.21.15.120:0/569605669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faac01036e0 msgr2=0x7faac0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:07.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.840+0000 7faac50c6640 1 --2- 172.21.15.120:0/569605669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faac01036e0 0x7faac0107b90 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7faaa8009930 tx=0x7faaa802a420 comp rx=0 tx=0).stop 2024-09-19T09:07:07.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.840+0000 7faac50c6640 1 -- 172.21.15.120:0/569605669 shutdown_connections 2024-09-19T09:07:07.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.840+0000 7faac50c6640 1 --2- 172.21.15.120:0/569605669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faac01082a0 0x7faac010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.840+0000 7faac50c6640 1 --2- 172.21.15.120:0/569605669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faac01036e0 0x7faac0107b90 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.840+0000 7faac50c6640 1 --2- 172.21.15.120:0/569605669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faac0102d30 0x7faac0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.841+0000 7faac50c6640 1 -- 172.21.15.120:0/569605669 >> 172.21.15.120:0/569605669 conn(0x7faac00fc7d0 msgr2=0x7faac00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:07.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.841+0000 7faac50c6640 1 -- 172.21.15.120:0/569605669 shutdown_connections 2024-09-19T09:07:07.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.841+0000 7faac50c6640 1 -- 172.21.15.120:0/569605669 wait complete. 2024-09-19T09:07:07.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.841+0000 7faac50c6640 1 Processor -- start 2024-09-19T09:07:07.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.841+0000 7faac50c6640 1 -- start start 2024-09-19T09:07:07.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.842+0000 7faac50c6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faac0102d30 0x7faac0198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:07.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.842+0000 7faac50c6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faac01036e0 0x7faac0198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:07.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.842+0000 7faac50c6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faac01082a0 0x7faac019ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:07.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.842+0000 7faac50c6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faac01120a0 con 0x7faac01082a0 2024-09-19T09:07:07.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.842+0000 7faac50c6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faac0111f20 con 0x7faac0102d30 2024-09-19T09:07:07.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.842+0000 7faac50c6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faac0112220 con 0x7faac01036e0 2024-09-19T09:07:07.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.842+0000 7faabf59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faac01082a0 0x7faac019ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:07.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.842+0000 7faabed9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faac0102d30 0x7faac0198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:07.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.842+0000 7faabf59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faac01082a0 0x7faac019ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33790/0 (socket says 172.21.15.120:33790) 2024-09-19T09:07:07.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.842+0000 7faabf59e640 1 -- 172.21.15.120:0/2729099192 learned_addr learned my addr 172.21.15.120:0/2729099192 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:07.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.842+0000 7faab7fff640 1 --2- 172.21.15.120:0/2729099192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faac01036e0 0x7faac0198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:07.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.842+0000 7faabf59e640 1 -- 172.21.15.120:0/2729099192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faac01036e0 msgr2=0x7faac0198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:07.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.842+0000 7faabf59e640 1 --2- 172.21.15.120:0/2729099192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faac01036e0 0x7faac0198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.842+0000 7faabf59e640 1 -- 172.21.15.120:0/2729099192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faac0102d30 msgr2=0x7faac0198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:07.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.842+0000 7faabf59e640 1 --2- 172.21.15.120:0/2729099192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faac0102d30 0x7faac0198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.842+0000 7faabf59e640 1 -- 172.21.15.120:0/2729099192 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac019d5b0 con 0x7faac01082a0 2024-09-19T09:07:07.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.842+0000 7faab7fff640 1 --2- 172.21.15.120:0/2729099192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faac01036e0 0x7faac0198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:07:07.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.842+0000 7faabed9d640 1 --2- 172.21.15.120:0/2729099192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faac0102d30 0x7faac0198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:07:07.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.842+0000 7faabf59e640 1 --2- 172.21.15.120:0/2729099192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faac01082a0 0x7faac019ced0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7faab8009fd0 tx=0x7faab800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:07.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.842+0000 7faabcd99640 1 -- 172.21.15.120:0/2729099192 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab8017070 con 0x7faac01082a0 2024-09-19T09:07:07.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.842+0000 7faabcd99640 1 -- 172.21.15.120:0/2729099192 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faab8003e50 con 0x7faac01082a0 2024-09-19T09:07:07.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.842+0000 7faac50c6640 1 -- 172.21.15.120:0/2729099192 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faac019d8a0 con 0x7faac01082a0 2024-09-19T09:07:07.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.842+0000 7faabcd99640 1 -- 172.21.15.120:0/2729099192 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab80106c0 con 0x7faac01082a0 2024-09-19T09:07:07.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.842+0000 7faac50c6640 1 -- 172.21.15.120:0/2729099192 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faac01a5180 con 0x7faac01082a0 2024-09-19T09:07:07.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.845+0000 7faabcd99640 1 -- 172.21.15.120:0/2729099192 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7faab8003970 con 0x7faac01082a0 2024-09-19T09:07:07.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.845+0000 7faac749f740 1 Processor -- start 2024-09-19T09:07:07.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.845+0000 7faac749f740 1 -- start start 2024-09-19T09:07:07.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.846+0000 7faac749f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635fb6db760 0x5635fb6dbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:07.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.846+0000 7faac749f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635fb6dc080 0x5635fb6e6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:07.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.846+0000 7faac749f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635fb6e6940 0x5635fb6e8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:07.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.846+0000 7faab7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635fb6db760 0x5635fb6dbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:07.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.846+0000 7faac749f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5635fb6412a0 con 0x5635fb6db760 2024-09-19T09:07:07.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.846+0000 7faac749f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5635fb5e8b10 con 0x5635fb6e6940 2024-09-19T09:07:07.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.846+0000 7faac749f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5635fb63d260 con 0x5635fb6dc080 2024-09-19T09:07:07.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.846+0000 7faab7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635fb6db760 0x5635fb6dbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33804/0 (socket says 172.21.15.120:33804) 2024-09-19T09:07:07.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.846+0000 7faab7fff640 1 -- 172.21.15.120:0/2125532316 learned_addr learned my addr 172.21.15.120:0/2125532316 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:07.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.846+0000 7faabcd99640 1 --2- 172.21.15.120:0/2729099192 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faaac0a3240 0x7faaac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:07.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.846+0000 7faabed9d640 1 --2- 172.21.15.120:0/2125532316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635fb6dc080 0x5635fb6e6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:07.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.846+0000 7faab7fff640 1 --2- 172.21.15.120:0/2125532316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635fb6e6940 0x5635fb6e8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:07.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.846+0000 7faabcd99640 1 -- 172.21.15.120:0/2729099192 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7faab80cf6c0 con 0x7faac01082a0 2024-09-19T09:07:07.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.846+0000 7faabf59e640 1 --2- 172.21.15.120:0/2729099192 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faaac0a3240 0x7faaac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:07.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.847+0000 7faabed9d640 1 -- 172.21.15.120:0/2125532316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635fb6e6940 msgr2=0x5635fb6e8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:07.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.847+0000 7faabf59e640 1 --2- 172.21.15.120:0/2729099192 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faaac0a3240 0x7faaac0a5700 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faab8098100 tx=0x7faab80044b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:07.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.847+0000 7faabed9d640 1 --2- 172.21.15.120:0/2125532316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635fb6e6940 0x5635fb6e8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.847+0000 7faabed9d640 1 -- 172.21.15.120:0/2125532316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635fb6db760 msgr2=0x5635fb6dbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:07.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.847+0000 7faabed9d640 1 --2- 172.21.15.120:0/2125532316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635fb6db760 0x5635fb6dbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.847+0000 7faabed9d640 1 -- 172.21.15.120:0/2125532316 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635fb3b2b90 con 0x5635fb6dc080 2024-09-19T09:07:07.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.847+0000 7faab7fff640 1 --2- 172.21.15.120:0/2125532316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635fb6db760 0x5635fb6dbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:07:07.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.847+0000 7faabed9d640 1 --2- 172.21.15.120:0/2125532316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635fb6dc080 0x5635fb6e6400 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7faab0009fd0 tx=0x7faab00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45acdc6098be6376 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:07.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.847+0000 7faab57fa640 1 -- 172.21.15.120:0/2125532316 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab0004030 con 0x5635fb6dc080 2024-09-19T09:07:07.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.847+0000 7faab57fa640 1 -- 172.21.15.120:0/2125532316 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faab0005500 con 0x5635fb6dc080 2024-09-19T09:07:07.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.847+0000 7faab57fa640 1 -- 172.21.15.120:0/2125532316 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab0004d40 con 0x5635fb6dc080 2024-09-19T09:07:07.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.848+0000 7faac749f740 1 -- 172.21.15.120:0/2125532316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635fb6dc080 msgr2=0x5635fb6e6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:07.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.848+0000 7faac749f740 1 --2- 172.21.15.120:0/2125532316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635fb6dc080 0x5635fb6e6400 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7faab0009fd0 tx=0x7faab00118d0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.848+0000 7faac749f740 1 -- 172.21.15.120:0/2125532316 shutdown_connections 2024-09-19T09:07:07.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.848+0000 7faac749f740 1 --2- 172.21.15.120:0/2125532316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635fb6e6940 0x5635fb6e8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.848+0000 7faac749f740 1 --2- 172.21.15.120:0/2125532316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635fb6dc080 0x5635fb6e6400 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.848+0000 7faac749f740 1 --2- 172.21.15.120:0/2125532316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635fb6db760 0x5635fb6dbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.848+0000 7faac749f740 1 -- 172.21.15.120:0/2125532316 >> 172.21.15.120:0/2125532316 conn(0x5635fb6d7cf0 msgr2=0x5635fb6d80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:07.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.848+0000 7faac749f740 1 -- 172.21.15.120:0/2125532316 shutdown_connections 2024-09-19T09:07:07.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.848+0000 7faac749f740 1 -- 172.21.15.120:0/2125532316 wait complete. 2024-09-19T09:07:07.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.848+0000 7faac749f740 1 -- start start 2024-09-19T09:07:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.848+0000 7faac749f740 1 Processor -- start 2024-09-19T09:07:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.849+0000 7faac749f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635fb6db760 0x5635fb6df9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.849+0000 7faac749f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635fb6dc080 0x5635fb6dfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.849+0000 7faac749f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635fb6e6940 0x5635fb6e0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.849+0000 7faac749f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5635fb6e94b0 con 0x5635fb6e6940 2024-09-19T09:07:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.849+0000 7faac749f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5635fb5c96e0 con 0x5635fb6dc080 2024-09-19T09:07:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.849+0000 7faac749f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5635fb6e9630 con 0x5635fb6db760 2024-09-19T09:07:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.849+0000 7faac749f740 10 client.0 _ll_register_callbacks cb 0x5635fb5eda80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.849+0000 7faab7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635fb6e6940 0x5635fb6e0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.849+0000 7faabed9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635fb6dc080 0x5635fb6dfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.849+0000 7faab7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635fb6e6940 0x5635fb6e0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33816/0 (socket says 172.21.15.120:33816) 2024-09-19T09:07:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.849+0000 7faab7fff640 1 -- 172.21.15.120:0/4152166606 learned_addr learned my addr 172.21.15.120:0/4152166606 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.849+0000 7faab7fff640 1 --2- 172.21.15.120:0/4152166606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635fb6db760 0x5635fb6df9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.849+0000 7faab7fff640 1 -- 172.21.15.120:0/4152166606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635fb6db760 msgr2=0x5635fb6df9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.849+0000 7faab7fff640 1 --2- 172.21.15.120:0/4152166606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635fb6db760 0x5635fb6df9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.849+0000 7faab7fff640 1 -- 172.21.15.120:0/4152166606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635fb6dc080 msgr2=0x5635fb6dfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.849+0000 7faab7fff640 1 --2- 172.21.15.120:0/4152166606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635fb6dc080 0x5635fb6dfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.849+0000 7faab7fff640 1 -- 172.21.15.120:0/4152166606 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635fb6e44c0 con 0x5635fb6e6940 2024-09-19T09:07:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.849+0000 7faabed9d640 1 --2- 172.21.15.120:0/4152166606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635fb6dc080 0x5635fb6dfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:07:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.849+0000 7faab7fff640 1 --2- 172.21.15.120:0/4152166606 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635fb6e6940 0x5635fb6e0430 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7faaa8006010 tx=0x7faaa80058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa8d973ae18bdda server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.849+0000 7faaa67fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:07:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.850+0000 7faaa67fc640 1 -- 172.21.15.120:0/4152166606 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8055060 con 0x5635fb6e6940 2024-09-19T09:07:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.850+0000 7faac749f740 1 -- 172.21.15.120:0/4152166606 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faab0005500 con 0x5635fb6e6940 2024-09-19T09:07:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.850+0000 7faaa67fc640 1 -- 172.21.15.120:0/4152166606 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7faaa8005480 con 0x5635fb6e6940 2024-09-19T09:07:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.850+0000 7faaa67fc640 1 -- 172.21.15.120:0/4152166606 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faaa8048070 con 0x5635fb6e6940 2024-09-19T09:07:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.850+0000 7faac749f740 10 client.16980 fetch_fsmap learned FSMap version 28 2024-09-19T09:07:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.850+0000 7faac749f740 1 -- 172.21.15.120:0/4152166606 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5635fb6f5330 con 0x5635fb6e6940 2024-09-19T09:07:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.850+0000 7faaa67fc640 1 -- 172.21.15.120:0/4152166606 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faab0005500 con 0x5635fb6e6940 2024-09-19T09:07:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.850+0000 7faaa67fc640 1 -- 172.21.15.120:0/4152166606 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faaa805a070 con 0x5635fb6e6940 2024-09-19T09:07:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.850+0000 7faac749f740 10 client.16980 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:07:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.850+0000 7faac749f740 10 client.16980 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:07:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.850+0000 7faac749f740 20 client.16980 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.850+0000 7faac749f740 1 --2- 172.21.15.120:0/4152166606 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5635fb6f67f0 0x5635fb6f8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.850+0000 7faac749f740 4 client.16980 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:07:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.850+0000 7faac749f740 1 -- 172.21.15.120:0/4152166606 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635fb64e9d0 con 0x5635fb6f67f0 2024-09-19T09:07:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.851+0000 7faabed9d640 1 --2- 172.21.15.120:0/4152166606 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5635fb6f67f0 0x5635fb6f8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.851+0000 7faabed9d640 1 --2- 172.21.15.120:0/4152166606 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5635fb6f67f0 0x5635fb6f8be0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e24899540f760887 server_cookie=f2b2483762829b9 in_seq=0 out_seq=0 2024-09-19T09:07:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.851+0000 7faaa67fc640 10 client.16980 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.852+0000 7faaa67fc640 1 -- 172.21.15.120:0/4152166606 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5635fb64e9d0 con 0x5635fb6f67f0 2024-09-19T09:07:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.852+0000 7faaa67fc640 10 client.16980 handle_command_reply: tid=0 2024-09-19T09:07:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.852+0000 7faac749f740 1 client.16980 shutdown 2024-09-19T09:07:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.853+0000 7faac749f740 1 -- 172.21.15.120:0/4152166606 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635fb6e6940 msgr2=0x5635fb6e0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.853+0000 7faac749f740 1 --2- 172.21.15.120:0/4152166606 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635fb6e6940 0x5635fb6e0430 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7faaa8006010 tx=0x7faaa80058b0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.853+0000 7faac749f740 1 -- 172.21.15.120:0/4152166606 shutdown_connections 2024-09-19T09:07:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.853+0000 7faac749f740 1 --2- 172.21.15.120:0/4152166606 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5635fb6f67f0 0x5635fb6f8be0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.853+0000 7faac749f740 1 --2- 172.21.15.120:0/4152166606 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635fb6e6940 0x5635fb6e0430 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.853+0000 7faaa67fc640 0 client.16980 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.853+0000 7faac749f740 1 --2- 172.21.15.120:0/4152166606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635fb6dc080 0x5635fb6dfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.853+0000 7faac749f740 1 --2- 172.21.15.120:0/4152166606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635fb6db760 0x5635fb6df9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.853+0000 7faac749f740 1 -- 172.21.15.120:0/4152166606 >> 172.21.15.120:0/4152166606 conn(0x5635fb6d7cf0 msgr2=0x5635fb6d6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.853+0000 7faac749f740 1 -- 172.21.15.120:0/4152166606 shutdown_connections 2024-09-19T09:07:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.853+0000 7faac749f740 1 -- 172.21.15.120:0/4152166606 wait complete. 2024-09-19T09:07:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.853+0000 7faac749f740 20 client.16980 trim_cache size 0 max 16384 2024-09-19T09:07:07.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.871+0000 7faac749f740 1 Processor -- start 2024-09-19T09:07:07.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.871+0000 7faac749f740 1 -- start start 2024-09-19T09:07:07.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.872+0000 7faac749f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635fb6db760 0x5635fb6e3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:07.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.872+0000 7faac749f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635fb6dc080 0x5635fb6fe410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:07.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.872+0000 7faac749f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635fb6e6940 0x5635fb6fe950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:07.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.872+0000 7faac749f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faac01ab420 con 0x5635fb6e6940 2024-09-19T09:07:07.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.872+0000 7faac749f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faac01acf60 con 0x5635fb6db760 2024-09-19T09:07:07.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.872+0000 7faac749f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faac01abaf0 con 0x5635fb6dc080 2024-09-19T09:07:07.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.872+0000 7faab7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635fb6e6940 0x5635fb6fe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:07.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.872+0000 7faab7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635fb6e6940 0x5635fb6fe950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33818/0 (socket says 172.21.15.120:33818) 2024-09-19T09:07:07.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.872+0000 7faab7fff640 1 -- 172.21.15.120:0/2387285199 learned_addr learned my addr 172.21.15.120:0/2387285199 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:07.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.872+0000 7faabed9d640 1 --2- 172.21.15.120:0/2387285199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635fb6dc080 0x5635fb6fe410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:07.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.872+0000 7faab7fff640 1 --2- 172.21.15.120:0/2387285199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635fb6db760 0x5635fb6e3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:07.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.872+0000 7faab7fff640 1 -- 172.21.15.120:0/2387285199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635fb6dc080 msgr2=0x5635fb6fe410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:07.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.872+0000 7faab7fff640 1 --2- 172.21.15.120:0/2387285199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635fb6dc080 0x5635fb6fe410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.872+0000 7faab7fff640 1 -- 172.21.15.120:0/2387285199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635fb6db760 msgr2=0x5635fb6e3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:07.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.872+0000 7faab7fff640 1 --2- 172.21.15.120:0/2387285199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635fb6db760 0x5635fb6e3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.872+0000 7faab7fff640 1 -- 172.21.15.120:0/2387285199 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635fb6d7b20 con 0x5635fb6e6940 2024-09-19T09:07:07.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.872+0000 7faab7fff640 1 --2- 172.21.15.120:0/2387285199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635fb6e6940 0x5635fb6fe950 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7faaa8001eb0 tx=0x7faaa802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc5e6d2945324a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:07.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.872+0000 7faab4ff9640 1 -- 172.21.15.120:0/2387285199 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8060070 con 0x5635fb6e6940 2024-09-19T09:07:07.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.872+0000 7faab4ff9640 1 -- 172.21.15.120:0/2387285199 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaa8061040 con 0x5635fb6e6940 2024-09-19T09:07:07.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.872+0000 7faab4ff9640 1 -- 172.21.15.120:0/2387285199 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8055070 con 0x5635fb6e6940 2024-09-19T09:07:07.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.873+0000 7faac749f740 1 -- 172.21.15.120:0/2387285199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635fb6e6940 msgr2=0x5635fb6fe950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:07.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.873+0000 7faac749f740 1 --2- 172.21.15.120:0/2387285199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635fb6e6940 0x5635fb6fe950 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7faaa8001eb0 tx=0x7faaa802a420 comp rx=0 tx=0).stop 2024-09-19T09:07:07.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.873+0000 7faac749f740 1 -- 172.21.15.120:0/2387285199 shutdown_connections 2024-09-19T09:07:07.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.873+0000 7faac749f740 1 --2- 172.21.15.120:0/2387285199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635fb6e6940 0x5635fb6fe950 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.873+0000 7faac749f740 1 --2- 172.21.15.120:0/2387285199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635fb6dc080 0x5635fb6fe410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.873+0000 7faac749f740 1 --2- 172.21.15.120:0/2387285199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635fb6db760 0x5635fb6e3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.873+0000 7faac749f740 1 -- 172.21.15.120:0/2387285199 >> 172.21.15.120:0/2387285199 conn(0x5635fb6d7cf0 msgr2=0x5635fb6dd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:07.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.873+0000 7faac749f740 1 -- 172.21.15.120:0/2387285199 shutdown_connections 2024-09-19T09:07:07.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.873+0000 7faac749f740 1 -- 172.21.15.120:0/2387285199 wait complete. 2024-09-19T09:07:07.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.874+0000 7faac749f740 1 -- start start 2024-09-19T09:07:07.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.874+0000 7faac749f740 1 Processor -- start 2024-09-19T09:07:07.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.874+0000 7faac749f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635fb6db760 0x5635fb6f0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:07.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.874+0000 7faac749f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635fb6dc080 0x5635fb6f0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:07.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.874+0000 7faac749f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635fb6e6940 0x5635fb6f1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:07.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.874+0000 7faac749f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faac01ac890 con 0x5635fb6dc080 2024-09-19T09:07:07.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.874+0000 7faac749f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faac01ac1c0 con 0x5635fb6e6940 2024-09-19T09:07:07.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.874+0000 7faac749f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faac01a93e0 con 0x5635fb6db760 2024-09-19T09:07:07.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.874+0000 7faac749f740 10 client.0 _ll_register_callbacks cb 0x7faaa8031950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:07.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.874+0000 7faabed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635fb6dc080 0x5635fb6f0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:07.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.874+0000 7faabed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635fb6dc080 0x5635fb6f0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33834/0 (socket says 172.21.15.120:33834) 2024-09-19T09:07:07.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.874+0000 7faabed9d640 1 -- 172.21.15.120:0/1444379814 learned_addr learned my addr 172.21.15.120:0/1444379814 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:07.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.874+0000 7faab7fff640 1 --2- 172.21.15.120:0/1444379814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635fb6db760 0x5635fb6f0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:07.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.874+0000 7faab7fff640 1 --2- 172.21.15.120:0/1444379814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635fb6e6940 0x5635fb6f1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:07.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.874+0000 7faabed9d640 1 -- 172.21.15.120:0/1444379814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635fb6db760 msgr2=0x5635fb6f0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:07.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.874+0000 7faabed9d640 1 --2- 172.21.15.120:0/1444379814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635fb6db760 0x5635fb6f0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.874+0000 7faabed9d640 1 -- 172.21.15.120:0/1444379814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635fb6e6940 msgr2=0x5635fb6f1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:07.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.874+0000 7faabed9d640 1 --2- 172.21.15.120:0/1444379814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635fb6e6940 0x5635fb6f1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.874+0000 7faabed9d640 1 -- 172.21.15.120:0/1444379814 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635fb6d3260 con 0x5635fb6dc080 2024-09-19T09:07:07.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.874+0000 7faab7fff640 1 --2- 172.21.15.120:0/1444379814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635fb6e6940 0x5635fb6f1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:07:07.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.875+0000 7faabed9d640 1 --2- 172.21.15.120:0/1444379814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635fb6dc080 0x5635fb6f0ec0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7faab0009ef0 tx=0x7faab0005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d2c14f9bdaecf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:07.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.875+0000 7faaa67fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:07:07.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.875+0000 7faaa67fc640 1 -- 172.21.15.120:0/1444379814 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab0004040 con 0x5635fb6dc080 2024-09-19T09:07:07.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.875+0000 7faaa67fc640 1 -- 172.21.15.120:0/1444379814 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7faab0016040 con 0x5635fb6dc080 2024-09-19T09:07:07.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.875+0000 7faac749f740 1 -- 172.21.15.120:0/1444379814 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaa8061040 con 0x5635fb6dc080 2024-09-19T09:07:07.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.875+0000 7faaa67fc640 1 -- 172.21.15.120:0/1444379814 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faab00041e0 con 0x5635fb6dc080 2024-09-19T09:07:07.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.875+0000 7faac749f740 10 client.16986 fetch_fsmap learned FSMap version 28 2024-09-19T09:07:07.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.875+0000 7faac749f740 1 -- 172.21.15.120:0/1444379814 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5635fb761710 con 0x5635fb6dc080 2024-09-19T09:07:07.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.875+0000 7faaa67fc640 1 -- 172.21.15.120:0/1444379814 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaa8061040 con 0x5635fb6dc080 2024-09-19T09:07:07.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.875+0000 7faaa67fc640 1 -- 172.21.15.120:0/1444379814 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faab0003710 con 0x5635fb6dc080 2024-09-19T09:07:07.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.875+0000 7faac749f740 10 client.16986 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:07:07.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.875+0000 7faac749f740 10 client.16986 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:07:07.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.876+0000 7faac749f740 20 client.16986 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:07.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.876+0000 7faac749f740 1 --2- 172.21.15.120:0/1444379814 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5635fb761d90 0x5635fb7641b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:07.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.876+0000 7faac749f740 4 client.16986 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}] 2024-09-19T09:07:07.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.876+0000 7faac749f740 1 -- 172.21.15.120:0/1444379814 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}) -- 0x5635fb648d20 con 0x5635fb761d90 2024-09-19T09:07:07.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.876+0000 7faab7fff640 1 --2- 172.21.15.120:0/1444379814 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5635fb761d90 0x5635fb7641b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:07.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.876+0000 7faab7fff640 1 --2- 172.21.15.120:0/1444379814 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5635fb761d90 0x5635fb7641b0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee9749a1626cb40e server_cookie=7decc36d0e734f91 in_seq=0 out_seq=0 2024-09-19T09:07:07.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.876+0000 7faaa67fc640 10 client.16986 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:07.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.961+0000 7faaa67fc640 1 -- 172.21.15.120:0/1444379814 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5635fb648d20 con 0x5635fb761d90 2024-09-19T09:07:07.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.961+0000 7faaa67fc640 10 client.16986 handle_command_reply: tid=0 2024-09-19T09:07:07.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.961+0000 7faac749f740 1 client.16986 shutdown 2024-09-19T09:07:07.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.961+0000 7faac749f740 1 -- 172.21.15.120:0/1444379814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635fb6dc080 msgr2=0x5635fb6f0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:07.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.961+0000 7faac749f740 1 --2- 172.21.15.120:0/1444379814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635fb6dc080 0x5635fb6f0ec0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7faab0009ef0 tx=0x7faab0005870 comp rx=0 tx=0).stop 2024-09-19T09:07:07.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.961+0000 7faac749f740 1 -- 172.21.15.120:0/1444379814 shutdown_connections 2024-09-19T09:07:07.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.961+0000 7faac749f740 1 --2- 172.21.15.120:0/1444379814 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5635fb761d90 0x5635fb7641b0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.961+0000 7faac749f740 1 --2- 172.21.15.120:0/1444379814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5635fb6e6940 0x5635fb6f1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.961+0000 7faac749f740 1 --2- 172.21.15.120:0/1444379814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5635fb6dc080 0x5635fb6f0ec0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.961+0000 7faaa67fc640 0 client.16986 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:07.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.961+0000 7faac749f740 1 --2- 172.21.15.120:0/1444379814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635fb6db760 0x5635fb6f0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.961+0000 7faac749f740 1 -- 172.21.15.120:0/1444379814 >> 172.21.15.120:0/1444379814 conn(0x5635fb6d7cf0 msgr2=0x5635fb6d6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:07.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.962+0000 7faac749f740 1 -- 172.21.15.120:0/1444379814 shutdown_connections 2024-09-19T09:07:07.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.962+0000 7faac749f740 1 -- 172.21.15.120:0/1444379814 wait complete. 2024-09-19T09:07:07.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.962+0000 7faac749f740 20 client.16986 trim_cache size 0 max 16384 2024-09-19T09:07:07.964 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:07:07.964 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 28, 2024-09-19T09:07:07.964 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:07:07.964 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 18, 2024-09-19T09:07:07.964 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:07:07.964 INFO:teuthology.orchestra.run.smithi120.stdout: "27b8e0": { 2024-09-19T09:07:07.964 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 18, 2024-09-19T09:07:07.964 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:07:07.965 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:07:07.965 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:07:07.965 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:07:07.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:07:07.965 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:07:07.965 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 149.8, 2024-09-19T09:07:07.965 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:07:07.965 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:07:07.965 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:07:07.965 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:07:07.965 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:07:07.965 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:07:07.965 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:07:07.965 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:07:07.966 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:07:07.966 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:07:07.966 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:07:07.966 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:07:07.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.962+0000 7faac50c6640 1 -- 172.21.15.120:0/2729099192 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faaac0a3240 msgr2=0x7faaac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:07.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.962+0000 7faac50c6640 1 --2- 172.21.15.120:0/2729099192 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faaac0a3240 0x7faaac0a5700 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faab8098100 tx=0x7faab80044b0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.962+0000 7faac50c6640 1 -- 172.21.15.120:0/2729099192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faac01082a0 msgr2=0x7faac019ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:07.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.962+0000 7faac50c6640 1 --2- 172.21.15.120:0/2729099192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faac01082a0 0x7faac019ced0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7faab8009fd0 tx=0x7faab800b260 comp rx=0 tx=0).stop 2024-09-19T09:07:07.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.962+0000 7faac50c6640 1 -- 172.21.15.120:0/2729099192 shutdown_connections 2024-09-19T09:07:07.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.962+0000 7faac50c6640 1 --2- 172.21.15.120:0/2729099192 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faaac0a3240 0x7faaac0a5700 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.962+0000 7faac50c6640 1 --2- 172.21.15.120:0/2729099192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faac01082a0 0x7faac019ced0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.962+0000 7faac50c6640 1 --2- 172.21.15.120:0/2729099192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faac01036e0 0x7faac0198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.962+0000 7faac50c6640 1 --2- 172.21.15.120:0/2729099192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faac0102d30 0x7faac0198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:07.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.962+0000 7faac50c6640 1 -- 172.21.15.120:0/2729099192 >> 172.21.15.120:0/2729099192 conn(0x7faac00fc7d0 msgr2=0x7faac0105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:07.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.962+0000 7faac50c6640 1 -- 172.21.15.120:0/2729099192 shutdown_connections 2024-09-19T09:07:07.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:07.962+0000 7faac50c6640 1 -- 172.21.15.120:0/2729099192 wait complete. 2024-09-19T09:07:08.044 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 11.31 MB/sec execute 63 sec latency 83.205 ms 2024-09-19T09:07:08.895 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '27b8e0', quiesce took 4.1 seconds. Will release after: 25.700000000000003 2024-09-19T09:07:09.044 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 11.14 MB/sec execute 64 sec latency 1083.284 ms 2024-09-19T09:07:09.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:08 smithi120 ceph-mon[31150]: pgmap v469: 97 pgs: 97 active+clean; 41 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.1 MiB/s wr, 424 op/s 2024-09-19T09:07:09.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:08 smithi196 ceph-mon[35150]: pgmap v469: 97 pgs: 97 active+clean; 41 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.1 MiB/s wr, 424 op/s 2024-09-19T09:07:09.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:08 smithi153 ceph-mon[35071]: pgmap v469: 97 pgs: 97 active+clean; 41 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.1 MiB/s wr, 424 op/s 2024-09-19T09:07:10.045 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 10.97 MB/sec execute 65 sec latency 2083.382 ms 2024-09-19T09:07:11.045 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 10.80 MB/sec execute 66 sec latency 3083.477 ms 2024-09-19T09:07:11.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:10 smithi120 ceph-mon[31150]: pgmap v470: 97 pgs: 97 active+clean; 42 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 375 op/s 2024-09-19T09:07:11.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:10 smithi196 ceph-mon[35150]: pgmap v470: 97 pgs: 97 active+clean; 42 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 375 op/s 2024-09-19T09:07:11.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:10 smithi153 ceph-mon[35071]: pgmap v470: 97 pgs: 97 active+clean; 42 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 375 op/s 2024-09-19T09:07:12.045 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 10.64 MB/sec execute 67 sec latency 4083.557 ms 2024-09-19T09:07:13.045 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 10.48 MB/sec execute 68 sec latency 5083.704 ms 2024-09-19T09:07:13.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:12 smithi120 ceph-mon[31150]: pgmap v471: 97 pgs: 97 active+clean; 48 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.3 MiB/s wr, 360 op/s 2024-09-19T09:07:13.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:12 smithi196 ceph-mon[35150]: pgmap v471: 97 pgs: 97 active+clean; 48 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.3 MiB/s wr, 360 op/s 2024-09-19T09:07:13.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:12 smithi153 ceph-mon[35071]: pgmap v471: 97 pgs: 97 active+clean; 48 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.3 MiB/s wr, 360 op/s 2024-09-19T09:07:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.656+0000 7fcd2096d640 1 -- 172.21.15.120:0/1628412287 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd1c100a40 msgr2=0x7fcd1c100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.656+0000 7fcd2096d640 1 --2- 172.21.15.120:0/1628412287 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd1c100a40 0x7fcd1c100e40 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009900 tx=0x7fcd0c02a410 comp rx=0 tx=0).stop 2024-09-19T09:07:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.656+0000 7fcd2096d640 1 -- 172.21.15.120:0/1628412287 shutdown_connections 2024-09-19T09:07:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.656+0000 7fcd2096d640 1 --2- 172.21.15.120:0/1628412287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd1c1025f0 0x7fcd1c108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.656+0000 7fcd2096d640 1 --2- 172.21.15.120:0/1628412287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd1c101c30 0x7fcd1c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.656+0000 7fcd2096d640 1 --2- 172.21.15.120:0/1628412287 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd1c100a40 0x7fcd1c100e40 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.656+0000 7fcd2096d640 1 -- 172.21.15.120:0/1628412287 >> 172.21.15.120:0/1628412287 conn(0x7fcd1c0fc1e0 msgr2=0x7fcd1c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.656+0000 7fcd2096d640 1 -- 172.21.15.120:0/1628412287 shutdown_connections 2024-09-19T09:07:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.656+0000 7fcd2096d640 1 -- 172.21.15.120:0/1628412287 wait complete. 2024-09-19T09:07:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.657+0000 7fcd2096d640 1 Processor -- start 2024-09-19T09:07:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.657+0000 7fcd2096d640 1 -- start start 2024-09-19T09:07:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.657+0000 7fcd2096d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd1c100a40 0x7fcd1c196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.657+0000 7fcd2096d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd1c101c30 0x7fcd1c197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.657+0000 7fcd2096d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd1c1025f0 0x7fcd1c19e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.657+0000 7fcd2096d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c10c6c0 con 0x7fcd1c100a40 2024-09-19T09:07:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.657+0000 7fcd2096d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c10c540 con 0x7fcd1c101c30 2024-09-19T09:07:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.657+0000 7fcd2096d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c10c840 con 0x7fcd1c1025f0 2024-09-19T09:07:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.657+0000 7fcd1a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd1c100a40 0x7fcd1c196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.657+0000 7fcd1a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd1c100a40 0x7fcd1c196d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53700/0 (socket says 172.21.15.120:53700) 2024-09-19T09:07:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.657+0000 7fcd1a59c640 1 -- 172.21.15.120:0/968313448 learned_addr learned my addr 172.21.15.120:0/968313448 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.657+0000 7fcd19d9b640 1 --2- 172.21.15.120:0/968313448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd1c101c30 0x7fcd1c197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.657+0000 7fcd1ad9d640 1 --2- 172.21.15.120:0/968313448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd1c1025f0 0x7fcd1c19e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.657+0000 7fcd1a59c640 1 -- 172.21.15.120:0/968313448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd1c1025f0 msgr2=0x7fcd1c19e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.657+0000 7fcd1a59c640 1 --2- 172.21.15.120:0/968313448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd1c1025f0 0x7fcd1c19e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.657+0000 7fcd1a59c640 1 -- 172.21.15.120:0/968313448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd1c101c30 msgr2=0x7fcd1c197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.657+0000 7fcd1a59c640 1 --2- 172.21.15.120:0/968313448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd1c101c30 0x7fcd1c197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.657+0000 7fcd1a59c640 1 -- 172.21.15.120:0/968313448 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1c19e910 con 0x7fcd1c100a40 2024-09-19T09:07:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.658+0000 7fcd1a59c640 1 --2- 172.21.15.120:0/968313448 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd1c100a40 0x7fcd1c196d00 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c031610 tx=0x7fcd0c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.658+0000 7fcd137fe640 1 -- 172.21.15.120:0/968313448 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c03f030 con 0x7fcd1c100a40 2024-09-19T09:07:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.658+0000 7fcd137fe640 1 -- 172.21.15.120:0/968313448 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd0c003ce0 con 0x7fcd1c100a40 2024-09-19T09:07:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.658+0000 7fcd2096d640 1 -- 172.21.15.120:0/968313448 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd1c19eba0 con 0x7fcd1c100a40 2024-09-19T09:07:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.658+0000 7fcd137fe640 1 -- 172.21.15.120:0/968313448 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c035070 con 0x7fcd1c100a40 2024-09-19T09:07:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.658+0000 7fcd2096d640 1 -- 172.21.15.120:0/968313448 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd1c19f0e0 con 0x7fcd1c100a40 2024-09-19T09:07:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.661+0000 7fcd137fe640 1 -- 172.21.15.120:0/968313448 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fcd0c042030 con 0x7fcd1c100a40 2024-09-19T09:07:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.661+0000 7fcd237e3740 1 Processor -- start 2024-09-19T09:07:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.661+0000 7fcd237e3740 1 -- start start 2024-09-19T09:07:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.661+0000 7fcd237e3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ce092bb0 0x55c6ce092fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.661+0000 7fcd237e3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6ce0934f0 0x55c6ce097fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.661+0000 7fcd237e3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c6ce098580 0x55c6ce09aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.661+0000 7fcd237e3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c6cdff42f0 con 0x55c6ce092bb0 2024-09-19T09:07:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.661+0000 7fcd237e3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c6cdf9c270 con 0x55c6ce098580 2024-09-19T09:07:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.661+0000 7fcd237e3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c6cdff02b0 con 0x55c6ce0934f0 2024-09-19T09:07:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.661+0000 7fcd1ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ce092bb0 0x55c6ce092fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.661+0000 7fcd1ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ce092bb0 0x55c6ce092fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53702/0 (socket says 172.21.15.120:53702) 2024-09-19T09:07:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.661+0000 7fcd1ad9d640 1 -- 172.21.15.120:0/1970445116 learned_addr learned my addr 172.21.15.120:0/1970445116 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.662+0000 7fcd1a59c640 1 --2- 172.21.15.120:0/1970445116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6ce0934f0 0x55c6ce097fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.662+0000 7fcd19d9b640 1 --2- 172.21.15.120:0/1970445116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c6ce098580 0x55c6ce09aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.662+0000 7fcd137fe640 1 --2- 172.21.15.120:0/968313448 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcd080a3370 0x7fcd080a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.662+0000 7fcd1ad9d640 1 -- 172.21.15.120:0/1970445116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6ce0934f0 msgr2=0x55c6ce097fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.662+0000 7fcd1ad9d640 1 --2- 172.21.15.120:0/1970445116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6ce0934f0 0x55c6ce097fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.662+0000 7fcd1ad9d640 1 -- 172.21.15.120:0/1970445116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c6ce098580 msgr2=0x55c6ce09aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.662+0000 7fcd1ad9d640 1 --2- 172.21.15.120:0/1970445116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c6ce098580 0x55c6ce09aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.662+0000 7fcd1ad9d640 1 -- 172.21.15.120:0/1970445116 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6cdf9b340 con 0x55c6ce092bb0 2024-09-19T09:07:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.662+0000 7fcd137fe640 1 -- 172.21.15.120:0/968313448 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fcd0c0efae0 con 0x7fcd1c100a40 2024-09-19T09:07:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.662+0000 7fcd1ad9d640 1 --2- 172.21.15.120:0/968313448 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcd080a3370 0x7fcd080a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.662+0000 7fcd1ad9d640 1 --2- 172.21.15.120:0/1970445116 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ce092bb0 0x55c6ce092fb0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fcd14010370 tx=0x7fcd14010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce02d213fe545996 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.662+0000 7fcd10ff9640 1 -- 172.21.15.120:0/1970445116 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd14022070 con 0x55c6ce092bb0 2024-09-19T09:07:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.662+0000 7fcd10ff9640 1 -- 172.21.15.120:0/1970445116 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd140113e0 con 0x55c6ce092bb0 2024-09-19T09:07:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.662+0000 7fcd10ff9640 1 -- 172.21.15.120:0/1970445116 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd14016070 con 0x55c6ce092bb0 2024-09-19T09:07:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.663+0000 7fcd237e3740 1 -- 172.21.15.120:0/1970445116 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ce092bb0 msgr2=0x55c6ce092fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.663+0000 7fcd1ad9d640 1 --2- 172.21.15.120:0/968313448 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcd080a3370 0x7fcd080a5830 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcd1401f000 tx=0x7fcd14022000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.663+0000 7fcd237e3740 1 --2- 172.21.15.120:0/1970445116 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ce092bb0 0x55c6ce092fb0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fcd14010370 tx=0x7fcd14010830 comp rx=0 tx=0).stop 2024-09-19T09:07:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.663+0000 7fcd237e3740 1 -- 172.21.15.120:0/1970445116 shutdown_connections 2024-09-19T09:07:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.663+0000 7fcd237e3740 1 --2- 172.21.15.120:0/1970445116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c6ce098580 0x55c6ce09aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.663+0000 7fcd237e3740 1 --2- 172.21.15.120:0/1970445116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6ce0934f0 0x55c6ce097fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.663+0000 7fcd237e3740 1 --2- 172.21.15.120:0/1970445116 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ce092bb0 0x55c6ce092fb0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.663+0000 7fcd237e3740 1 -- 172.21.15.120:0/1970445116 >> 172.21.15.120:0/1970445116 conn(0x55c6ce08c010 msgr2=0x55c6ce08c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.663+0000 7fcd237e3740 1 -- 172.21.15.120:0/1970445116 shutdown_connections 2024-09-19T09:07:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.663+0000 7fcd237e3740 1 -- 172.21.15.120:0/1970445116 wait complete. 2024-09-19T09:07:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.664+0000 7fcd237e3740 1 -- start start 2024-09-19T09:07:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.664+0000 7fcd237e3740 1 Processor -- start 2024-09-19T09:07:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.664+0000 7fcd237e3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ce092bb0 0x55c6ce09ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.664+0000 7fcd237e3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c6ce0934f0 0x55c6ce09d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.664+0000 7fcd237e3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6ce098580 0x55c6ce0a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.664+0000 7fcd237e3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c6ce09b550 con 0x55c6ce092bb0 2024-09-19T09:07:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.664+0000 7fcd237e3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c6cdf7d290 con 0x55c6ce0934f0 2024-09-19T09:07:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.664+0000 7fcd237e3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c6ce09b6d0 con 0x55c6ce098580 2024-09-19T09:07:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.664+0000 7fcd237e3740 10 client.0 _ll_register_callbacks cb 0x55c6cdfae140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.664+0000 7fcd1a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ce092bb0 0x55c6ce09ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.664+0000 7fcd1a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ce092bb0 0x55c6ce09ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53706/0 (socket says 172.21.15.120:53706) 2024-09-19T09:07:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.664+0000 7fcd1a59c640 1 -- 172.21.15.120:0/2548539976 learned_addr learned my addr 172.21.15.120:0/2548539976 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.664+0000 7fcd19d9b640 1 --2- 172.21.15.120:0/2548539976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c6ce0934f0 0x55c6ce09d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.664+0000 7fcd1ad9d640 1 --2- 172.21.15.120:0/2548539976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6ce098580 0x55c6ce0a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.664+0000 7fcd1a59c640 1 -- 172.21.15.120:0/2548539976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6ce098580 msgr2=0x55c6ce0a43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.664+0000 7fcd1a59c640 1 --2- 172.21.15.120:0/2548539976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6ce098580 0x55c6ce0a43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.664+0000 7fcd1a59c640 1 -- 172.21.15.120:0/2548539976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c6ce0934f0 msgr2=0x55c6ce09d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.664+0000 7fcd1a59c640 1 --2- 172.21.15.120:0/2548539976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c6ce0934f0 0x55c6ce09d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.664+0000 7fcd1a59c640 1 -- 172.21.15.120:0/2548539976 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6cdd07210 con 0x55c6ce092bb0 2024-09-19T09:07:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.664+0000 7fcd1a59c640 1 --2- 172.21.15.120:0/2548539976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ce092bb0 0x55c6ce09ce50 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c000c00 tx=0x7fcd0c0f3090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9b2e2c197d81e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.664+0000 7fcd01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:07:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.664+0000 7fcd01ffb640 1 -- 172.21.15.120:0/2548539976 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c049070 con 0x55c6ce092bb0 2024-09-19T09:07:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.665+0000 7fcd01ffb640 1 -- 172.21.15.120:0/2548539976 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fcd0c0b8790 con 0x55c6ce092bb0 2024-09-19T09:07:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.665+0000 7fcd237e3740 1 -- 172.21.15.120:0/2548539976 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd140113e0 con 0x55c6ce092bb0 2024-09-19T09:07:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.665+0000 7fcd01ffb640 1 -- 172.21.15.120:0/2548539976 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcd0c0b83c0 con 0x55c6ce092bb0 2024-09-19T09:07:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.665+0000 7fcd237e3740 10 client.16998 fetch_fsmap learned FSMap version 28 2024-09-19T09:07:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.665+0000 7fcd237e3740 1 -- 172.21.15.120:0/2548539976 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c6cddd3620 con 0x55c6ce092bb0 2024-09-19T09:07:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.665+0000 7fcd01ffb640 1 -- 172.21.15.120:0/2548539976 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd140113e0 con 0x55c6ce092bb0 2024-09-19T09:07:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.665+0000 7fcd01ffb640 1 -- 172.21.15.120:0/2548539976 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcd0c03b460 con 0x55c6ce092bb0 2024-09-19T09:07:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.665+0000 7fcd237e3740 10 client.16998 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:07:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.665+0000 7fcd237e3740 10 client.16998 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:07:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.665+0000 7fcd237e3740 20 client.16998 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.665+0000 7fcd237e3740 1 --2- 172.21.15.120:0/2548539976 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c6ce0a8ae0 0x55c6ce0aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.665+0000 7fcd237e3740 4 client.16998 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:07:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.665+0000 7fcd237e3740 1 -- 172.21.15.120:0/2548539976 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6ce001a20 con 0x55c6ce0a8ae0 2024-09-19T09:07:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.665+0000 7fcd19d9b640 1 --2- 172.21.15.120:0/2548539976 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c6ce0a8ae0 0x55c6ce0aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.666+0000 7fcd19d9b640 1 --2- 172.21.15.120:0/2548539976 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c6ce0a8ae0 0x55c6ce0aaed0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43390f9aa17bddb6 server_cookie=a5372f0a7f08af32 in_seq=0 out_seq=0 2024-09-19T09:07:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.666+0000 7fcd01ffb640 10 client.16998 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.666+0000 7fcd01ffb640 1 -- 172.21.15.120:0/2548539976 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c6ce001a20 con 0x55c6ce0a8ae0 2024-09-19T09:07:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.666+0000 7fcd01ffb640 10 client.16998 handle_command_reply: tid=0 2024-09-19T09:07:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.666+0000 7fcd237e3740 1 client.16998 shutdown 2024-09-19T09:07:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.666+0000 7fcd237e3740 1 -- 172.21.15.120:0/2548539976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ce092bb0 msgr2=0x55c6ce09ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.666+0000 7fcd237e3740 1 --2- 172.21.15.120:0/2548539976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ce092bb0 0x55c6ce09ce50 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c000c00 tx=0x7fcd0c0f3090 comp rx=0 tx=0).stop 2024-09-19T09:07:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.666+0000 7fcd237e3740 1 -- 172.21.15.120:0/2548539976 shutdown_connections 2024-09-19T09:07:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.666+0000 7fcd237e3740 1 --2- 172.21.15.120:0/2548539976 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c6ce0a8ae0 0x55c6ce0aaed0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.666+0000 7fcd237e3740 1 --2- 172.21.15.120:0/2548539976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6ce098580 0x55c6ce0a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.666+0000 7fcd01ffb640 0 client.16998 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.666+0000 7fcd237e3740 1 --2- 172.21.15.120:0/2548539976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c6ce0934f0 0x55c6ce09d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.666+0000 7fcd237e3740 1 --2- 172.21.15.120:0/2548539976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ce092bb0 0x55c6ce09ce50 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.666+0000 7fcd237e3740 1 -- 172.21.15.120:0/2548539976 >> 172.21.15.120:0/2548539976 conn(0x55c6ce08c010 msgr2=0x55c6ce08d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.667+0000 7fcd237e3740 1 -- 172.21.15.120:0/2548539976 shutdown_connections 2024-09-19T09:07:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.667+0000 7fcd237e3740 1 -- 172.21.15.120:0/2548539976 wait complete. 2024-09-19T09:07:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.667+0000 7fcd237e3740 20 client.16998 trim_cache size 0 max 16384 2024-09-19T09:07:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.681+0000 7fcd237e3740 1 Processor -- start 2024-09-19T09:07:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.681+0000 7fcd237e3740 1 -- start start 2024-09-19T09:07:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.681+0000 7fcd237e3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6ce092bb0 0x55c6ce09d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.681+0000 7fcd237e3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ce0934f0 0x55c6ce0a4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.681+0000 7fcd237e3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c6ce098580 0x55c6ce0af780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.681+0000 7fcd237e3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c1a6d90 con 0x55c6ce0934f0 2024-09-19T09:07:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.681+0000 7fcd237e3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c1a88d0 con 0x55c6ce098580 2024-09-19T09:07:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.681+0000 7fcd237e3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c1a7460 con 0x55c6ce092bb0 2024-09-19T09:07:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.681+0000 7fcd19d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ce0934f0 0x55c6ce0a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.681+0000 7fcd19d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ce0934f0 0x55c6ce0a4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53714/0 (socket says 172.21.15.120:53714) 2024-09-19T09:07:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.681+0000 7fcd19d9b640 1 -- 172.21.15.120:0/2392141603 learned_addr learned my addr 172.21.15.120:0/2392141603 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.681+0000 7fcd1a59c640 1 --2- 172.21.15.120:0/2392141603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6ce092bb0 0x55c6ce09d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.681+0000 7fcd1ad9d640 1 --2- 172.21.15.120:0/2392141603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c6ce098580 0x55c6ce0af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.681+0000 7fcd19d9b640 1 -- 172.21.15.120:0/2392141603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6ce092bb0 msgr2=0x55c6ce09d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.681+0000 7fcd19d9b640 1 --2- 172.21.15.120:0/2392141603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6ce092bb0 0x55c6ce09d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.681+0000 7fcd19d9b640 1 -- 172.21.15.120:0/2392141603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c6ce098580 msgr2=0x55c6ce0af780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.681+0000 7fcd19d9b640 1 --2- 172.21.15.120:0/2392141603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c6ce098580 0x55c6ce0af780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.681+0000 7fcd19d9b640 1 -- 172.21.15.120:0/2392141603 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6ce0ab500 con 0x55c6ce0934f0 2024-09-19T09:07:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.682+0000 7fcd19d9b640 1 --2- 172.21.15.120:0/2392141603 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ce0934f0 0x55c6ce0a4390 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x55c6ce08ce90 tx=0x7fcd0400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6428c3c1d01f513 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.682+0000 7fcd03fff640 1 -- 172.21.15.120:0/2392141603 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04027470 con 0x55c6ce0934f0 2024-09-19T09:07:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.682+0000 7fcd03fff640 1 -- 172.21.15.120:0/2392141603 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd04027af0 con 0x55c6ce0934f0 2024-09-19T09:07:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.682+0000 7fcd03fff640 1 -- 172.21.15.120:0/2392141603 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04011a50 con 0x55c6ce0934f0 2024-09-19T09:07:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.682+0000 7fcd237e3740 1 -- 172.21.15.120:0/2392141603 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ce0934f0 msgr2=0x55c6ce0a4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.682+0000 7fcd237e3740 1 --2- 172.21.15.120:0/2392141603 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ce0934f0 0x55c6ce0a4390 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x55c6ce08ce90 tx=0x7fcd0400b390 comp rx=0 tx=0).stop 2024-09-19T09:07:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.682+0000 7fcd237e3740 1 -- 172.21.15.120:0/2392141603 shutdown_connections 2024-09-19T09:07:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.682+0000 7fcd237e3740 1 --2- 172.21.15.120:0/2392141603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c6ce098580 0x55c6ce0af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.682+0000 7fcd237e3740 1 --2- 172.21.15.120:0/2392141603 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ce0934f0 0x55c6ce0a4390 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.682+0000 7fcd237e3740 1 --2- 172.21.15.120:0/2392141603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6ce092bb0 0x55c6ce09d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.682+0000 7fcd237e3740 1 -- 172.21.15.120:0/2392141603 >> 172.21.15.120:0/2392141603 conn(0x55c6ce08c010 msgr2=0x55c6ce08d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.682+0000 7fcd237e3740 1 -- 172.21.15.120:0/2392141603 shutdown_connections 2024-09-19T09:07:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.682+0000 7fcd237e3740 1 -- 172.21.15.120:0/2392141603 wait complete. 2024-09-19T09:07:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.683+0000 7fcd237e3740 1 -- start start 2024-09-19T09:07:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.683+0000 7fcd237e3740 1 Processor -- start 2024-09-19T09:07:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.683+0000 7fcd237e3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6ce092bb0 0x55c6ce0a2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.683+0000 7fcd237e3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c6ce0934f0 0x55c6ce0a2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.683+0000 7fcd237e3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ce098580 0x55c6ce0a3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.683+0000 7fcd237e3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c1a8200 con 0x55c6ce098580 2024-09-19T09:07:13.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.683+0000 7fcd237e3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c1a7b30 con 0x55c6ce0934f0 2024-09-19T09:07:13.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.683+0000 7fcd237e3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c1a4d50 con 0x55c6ce092bb0 2024-09-19T09:07:13.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.683+0000 7fcd237e3740 10 client.0 _ll_register_callbacks cb 0x7fcd0c0f3250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:13.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.683+0000 7fcd1ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ce098580 0x55c6ce0a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:13.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.683+0000 7fcd1ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ce098580 0x55c6ce0a3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53720/0 (socket says 172.21.15.120:53720) 2024-09-19T09:07:13.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.683+0000 7fcd1ad9d640 1 -- 172.21.15.120:0/3015372633 learned_addr learned my addr 172.21.15.120:0/3015372633 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:13.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.683+0000 7fcd1a59c640 1 --2- 172.21.15.120:0/3015372633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6ce092bb0 0x55c6ce0a2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:13.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.683+0000 7fcd19d9b640 1 --2- 172.21.15.120:0/3015372633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c6ce0934f0 0x55c6ce0a2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:13.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.683+0000 7fcd1ad9d640 1 -- 172.21.15.120:0/3015372633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6ce092bb0 msgr2=0x55c6ce0a2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:13.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.683+0000 7fcd1ad9d640 1 --2- 172.21.15.120:0/3015372633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6ce092bb0 0x55c6ce0a2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.683+0000 7fcd1ad9d640 1 -- 172.21.15.120:0/3015372633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c6ce0934f0 msgr2=0x55c6ce0a2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:13.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.683+0000 7fcd1ad9d640 1 --2- 172.21.15.120:0/3015372633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c6ce0934f0 0x55c6ce0a2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.683+0000 7fcd1ad9d640 1 -- 172.21.15.120:0/3015372633 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6ce0a3d70 con 0x55c6ce098580 2024-09-19T09:07:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.683+0000 7fcd19d9b640 1 --2- 172.21.15.120:0/3015372633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c6ce0934f0 0x55c6ce0a2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:07:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.683+0000 7fcd1ad9d640 1 --2- 172.21.15.120:0/3015372633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ce098580 0x55c6ce0a3760 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fcd14012a20 tx=0x7fcd14012a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eecf13b6fef61efe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.683+0000 7fcd01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:07:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.684+0000 7fcd01ffb640 1 -- 172.21.15.120:0/3015372633 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd14016040 con 0x55c6ce098580 2024-09-19T09:07:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.684+0000 7fcd01ffb640 1 -- 172.21.15.120:0/3015372633 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fcd14003970 con 0x55c6ce098580 2024-09-19T09:07:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.684+0000 7fcd237e3740 1 -- 172.21.15.120:0/3015372633 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd04027af0 con 0x55c6ce098580 2024-09-19T09:07:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.684+0000 7fcd01ffb640 1 -- 172.21.15.120:0/3015372633 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcd14003bb0 con 0x55c6ce098580 2024-09-19T09:07:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.684+0000 7fcd237e3740 10 client.17004 fetch_fsmap learned FSMap version 28 2024-09-19T09:07:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.684+0000 7fcd237e3740 1 -- 172.21.15.120:0/3015372633 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c6ce113e90 con 0x55c6ce098580 2024-09-19T09:07:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.684+0000 7fcd01ffb640 1 -- 172.21.15.120:0/3015372633 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd04027af0 con 0x55c6ce098580 2024-09-19T09:07:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.684+0000 7fcd01ffb640 1 -- 172.21.15.120:0/3015372633 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcd14032030 con 0x55c6ce098580 2024-09-19T09:07:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.684+0000 7fcd237e3740 10 client.17004 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:07:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.684+0000 7fcd237e3740 10 client.17004 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:07:13.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.684+0000 7fcd237e3740 20 client.17004 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:13.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.684+0000 7fcd237e3740 1 --2- 172.21.15.120:0/3015372633 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c6ce1145f0 0x55c6ce116a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:13.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.684+0000 7fcd237e3740 4 client.17004 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:07:13.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.684+0000 7fcd237e3740 1 -- 172.21.15.120:0/3015372633 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c6cdffbd70 con 0x55c6ce1145f0 2024-09-19T09:07:13.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.684+0000 7fcd1a59c640 1 --2- 172.21.15.120:0/3015372633 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c6ce1145f0 0x55c6ce116a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:13.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.685+0000 7fcd01ffb640 10 client.17004 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:13.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.685+0000 7fcd1a59c640 1 --2- 172.21.15.120:0/3015372633 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c6ce1145f0 0x55c6ce116a10 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d7d798edb7b08e4 server_cookie=af9898905830cc94 in_seq=0 out_seq=0 2024-09-19T09:07:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.770+0000 7fcd01ffb640 1 -- 172.21.15.120:0/3015372633 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c6cdffbd70 con 0x55c6ce1145f0 2024-09-19T09:07:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.770+0000 7fcd01ffb640 10 client.17004 handle_command_reply: tid=0 2024-09-19T09:07:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.770+0000 7fcd237e3740 1 client.17004 shutdown 2024-09-19T09:07:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.770+0000 7fcd237e3740 1 -- 172.21.15.120:0/3015372633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ce098580 msgr2=0x55c6ce0a3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.770+0000 7fcd237e3740 1 --2- 172.21.15.120:0/3015372633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ce098580 0x55c6ce0a3760 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fcd14012a20 tx=0x7fcd14012a50 comp rx=0 tx=0).stop 2024-09-19T09:07:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.771+0000 7fcd237e3740 1 -- 172.21.15.120:0/3015372633 shutdown_connections 2024-09-19T09:07:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.771+0000 7fcd237e3740 1 --2- 172.21.15.120:0/3015372633 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c6ce1145f0 0x55c6ce116a10 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.771+0000 7fcd01ffb640 0 client.17004 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.771+0000 7fcd237e3740 1 --2- 172.21.15.120:0/3015372633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c6ce098580 0x55c6ce0a3760 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.771+0000 7fcd237e3740 1 --2- 172.21.15.120:0/3015372633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c6ce0934f0 0x55c6ce0a2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.771+0000 7fcd237e3740 1 --2- 172.21.15.120:0/3015372633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6ce092bb0 0x55c6ce0a2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.771+0000 7fcd237e3740 1 -- 172.21.15.120:0/3015372633 >> 172.21.15.120:0/3015372633 conn(0x55c6ce08c010 msgr2=0x55c6ce08d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.771+0000 7fcd237e3740 1 -- 172.21.15.120:0/3015372633 shutdown_connections 2024-09-19T09:07:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.771+0000 7fcd237e3740 1 -- 172.21.15.120:0/3015372633 wait complete. 2024-09-19T09:07:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.771+0000 7fcd237e3740 20 client.17004 trim_cache size 0 max 16384 2024-09-19T09:07:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:07:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:07:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:07:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:07:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.771+0000 7fcd2096d640 1 -- 172.21.15.120:0/968313448 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcd080a3370 msgr2=0x7fcd080a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.771+0000 7fcd2096d640 1 --2- 172.21.15.120:0/968313448 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcd080a3370 0x7fcd080a5830 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcd1401f000 tx=0x7fcd14022000 comp rx=0 tx=0).stop 2024-09-19T09:07:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.771+0000 7fcd2096d640 1 -- 172.21.15.120:0/968313448 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd1c100a40 msgr2=0x7fcd1c196d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.771+0000 7fcd2096d640 1 --2- 172.21.15.120:0/968313448 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd1c100a40 0x7fcd1c196d00 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c031610 tx=0x7fcd0c0043b0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.772+0000 7fcd2096d640 1 -- 172.21.15.120:0/968313448 shutdown_connections 2024-09-19T09:07:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.772+0000 7fcd2096d640 1 --2- 172.21.15.120:0/968313448 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcd080a3370 0x7fcd080a5830 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.772+0000 7fcd2096d640 1 --2- 172.21.15.120:0/968313448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd1c1025f0 0x7fcd1c19e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.772+0000 7fcd2096d640 1 --2- 172.21.15.120:0/968313448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd1c101c30 0x7fcd1c197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.772+0000 7fcd2096d640 1 --2- 172.21.15.120:0/968313448 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd1c100a40 0x7fcd1c196d00 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.772+0000 7fcd2096d640 1 -- 172.21.15.120:0/968313448 >> 172.21.15.120:0/968313448 conn(0x7fcd1c0fc1e0 msgr2=0x7fcd1c104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.772+0000 7fcd2096d640 1 -- 172.21.15.120:0/968313448 shutdown_connections 2024-09-19T09:07:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:13.772+0000 7fcd2096d640 1 -- 172.21.15.120:0/968313448 wait complete. 2024-09-19T09:07:14.045 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 10.33 MB/sec execute 69 sec latency 6083.790 ms 2024-09-19T09:07:14.687 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:07:14 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-e[63911]: ignoring --setuser ceph since I am not root 2024-09-19T09:07:14.688 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:07:14 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-e[63911]: ignoring --setgroup ceph since I am not root 2024-09-19T09:07:14.688 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:07:14 smithi153 ceph-mds[63915]: -- 172.21.15.153:0/3677300771 <== mon.2 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5559920ad860 con 0x55599208b000 2024-09-19T09:07:14.688 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:07:14 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-e[63911]: starting mds.e at 2024-09-19T09:07:15.045 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 10.18 MB/sec execute 70 sec latency 7083.881 ms 2024-09-19T09:07:15.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:14 smithi120 ceph-mon[31150]: pgmap v472: 97 pgs: 97 active+clean; 53 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 318 op/s 2024-09-19T09:07:15.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:14 smithi120 ceph-mon[31150]: daemon mds.e restarted 2024-09-19T09:07:15.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:14 smithi196 ceph-mon[35150]: pgmap v472: 97 pgs: 97 active+clean; 53 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 318 op/s 2024-09-19T09:07:15.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:14 smithi196 ceph-mon[35150]: daemon mds.e restarted 2024-09-19T09:07:15.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:14 smithi153 ceph-mon[35071]: pgmap v472: 97 pgs: 97 active+clean; 53 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 318 op/s 2024-09-19T09:07:15.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:14 smithi153 ceph-mon[35071]: daemon mds.e restarted 2024-09-19T09:07:16.045 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 10.04 MB/sec execute 71 sec latency 8084.002 ms 2024-09-19T09:07:16.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:15 smithi120 ceph-mon[31150]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T09:07:16.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:15 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.153:6834/364941779,v1:172.21.15.153:6835/364941779] up:boot 2024-09-19T09:07:16.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:15 smithi120 ceph-mon[31150]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-19T09:07:16.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:15 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:07:16.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:15 smithi120 ceph-mon[31150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:07:16.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:15 smithi196 ceph-mon[35150]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T09:07:16.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:15 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.153:6834/364941779,v1:172.21.15.153:6835/364941779] up:boot 2024-09-19T09:07:16.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:15 smithi196 ceph-mon[35150]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-19T09:07:16.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:15 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:07:16.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:15 smithi196 ceph-mon[35150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:07:16.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:15 smithi153 ceph-mon[35071]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T09:07:16.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:15 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.153:6834/364941779,v1:172.21.15.153:6835/364941779] up:boot 2024-09-19T09:07:16.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:15 smithi153 ceph-mon[35071]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-19T09:07:16.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:15 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:07:16.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:15 smithi153 ceph-mon[35071]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:07:17.045 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 9.90 MB/sec execute 72 sec latency 9084.053 ms 2024-09-19T09:07:17.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:16 smithi120 ceph-mon[31150]: pgmap v474: 97 pgs: 97 active+clean; 45 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 155 op/s 2024-09-19T09:07:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:16 smithi196 ceph-mon[35150]: pgmap v474: 97 pgs: 97 active+clean; 45 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 155 op/s 2024-09-19T09:07:17.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:16 smithi153 ceph-mon[35071]: pgmap v474: 97 pgs: 97 active+clean; 45 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 155 op/s 2024-09-19T09:07:18.045 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 9.76 MB/sec execute 73 sec latency 10084.114 ms 2024-09-19T09:07:19.045 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 9.63 MB/sec execute 74 sec latency 11084.186 ms 2024-09-19T09:07:19.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:18 smithi120 ceph-mon[31150]: pgmap v475: 97 pgs: 97 active+clean; 43 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 185 op/s 2024-09-19T09:07:19.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:18 smithi196 ceph-mon[35150]: pgmap v475: 97 pgs: 97 active+clean; 43 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 185 op/s 2024-09-19T09:07:19.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:18 smithi153 ceph-mon[35071]: pgmap v475: 97 pgs: 97 active+clean; 43 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 185 op/s 2024-09-19T09:07:20.046 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 9.50 MB/sec execute 75 sec latency 12084.290 ms 2024-09-19T09:07:21.046 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 9.38 MB/sec execute 76 sec latency 13084.371 ms 2024-09-19T09:07:21.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:20 smithi196 ceph-mon[35150]: pgmap v476: 97 pgs: 97 active+clean; 44 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 2.6 MiB/s wr, 215 op/s 2024-09-19T09:07:21.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:20 smithi153 ceph-mon[35071]: pgmap v476: 97 pgs: 97 active+clean; 44 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 2.6 MiB/s wr, 215 op/s 2024-09-19T09:07:21.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:20 smithi120 ceph-mon[31150]: pgmap v476: 97 pgs: 97 active+clean; 44 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 2.6 MiB/s wr, 215 op/s 2024-09-19T09:07:22.046 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 9.26 MB/sec execute 77 sec latency 14084.452 ms 2024-09-19T09:07:23.046 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 9.14 MB/sec execute 78 sec latency 15084.578 ms 2024-09-19T09:07:23.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:22 smithi196 ceph-mon[35150]: pgmap v477: 97 pgs: 97 active+clean; 46 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 2.4 MiB/s wr, 170 op/s 2024-09-19T09:07:23.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:22 smithi153 ceph-mon[35071]: pgmap v477: 97 pgs: 97 active+clean; 46 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 2.4 MiB/s wr, 170 op/s 2024-09-19T09:07:23.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:22 smithi120 ceph-mon[31150]: pgmap v477: 97 pgs: 97 active+clean; 46 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 2.4 MiB/s wr, 170 op/s 2024-09-19T09:07:24.046 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 9.02 MB/sec execute 79 sec latency 16084.675 ms 2024-09-19T09:07:25.046 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 8.91 MB/sec execute 80 sec latency 17084.774 ms 2024-09-19T09:07:25.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:24 smithi196 ceph-mon[35150]: pgmap v478: 97 pgs: 97 active+clean; 42 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 2.4 MiB/s wr, 180 op/s 2024-09-19T09:07:25.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:24 smithi153 ceph-mon[35071]: pgmap v478: 97 pgs: 97 active+clean; 42 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 2.4 MiB/s wr, 180 op/s 2024-09-19T09:07:25.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:24 smithi120 ceph-mon[31150]: pgmap v478: 97 pgs: 97 active+clean; 42 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 2.4 MiB/s wr, 180 op/s 2024-09-19T09:07:26.046 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 8.80 MB/sec execute 81 sec latency 18084.863 ms 2024-09-19T09:07:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.867+0000 7f9e96bb9640 1 -- 172.21.15.120:0/305691190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e90100a60 msgr2=0x7f9e90100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.867+0000 7f9e96bb9640 1 --2- 172.21.15.120:0/305691190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e90100a60 0x7f9e90100e60 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f9e84009900 tx=0x7f9e8402a3f0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.867+0000 7f9e96bb9640 1 -- 172.21.15.120:0/305691190 shutdown_connections 2024-09-19T09:07:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.867+0000 7f9e96bb9640 1 --2- 172.21.15.120:0/305691190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e90102610 0x7f9e90108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.867+0000 7f9e96bb9640 1 --2- 172.21.15.120:0/305691190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e90101c50 0x7f9e901020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.867+0000 7f9e96bb9640 1 --2- 172.21.15.120:0/305691190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e90100a60 0x7f9e90100e60 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.867+0000 7f9e96bb9640 1 -- 172.21.15.120:0/305691190 >> 172.21.15.120:0/305691190 conn(0x7f9e900fc1e0 msgr2=0x7f9e900fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.867+0000 7f9e96bb9640 1 -- 172.21.15.120:0/305691190 shutdown_connections 2024-09-19T09:07:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.867+0000 7f9e96bb9640 1 -- 172.21.15.120:0/305691190 wait complete. 2024-09-19T09:07:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.868+0000 7f9e96bb9640 1 Processor -- start 2024-09-19T09:07:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.868+0000 7f9e96bb9640 1 -- start start 2024-09-19T09:07:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.868+0000 7f9e96bb9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e90100a60 0x7f9e9010b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.868+0000 7f9e96bb9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e90101c50 0x7f9e90109a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.868+0000 7f9e96bb9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e90102610 0x7f9e90109f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.868+0000 7f9e96bb9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9010d890 con 0x7f9e90101c50 2024-09-19T09:07:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.868+0000 7f9e96bb9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9010d710 con 0x7f9e90100a60 2024-09-19T09:07:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.868+0000 7f9e96bb9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9010da10 con 0x7f9e90102610 2024-09-19T09:07:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.868+0000 7f9e8ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e90101c50 0x7f9e90109a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.868+0000 7f9e8ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e90101c50 0x7f9e90109a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42684/0 (socket says 172.21.15.120:42684) 2024-09-19T09:07:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.868+0000 7f9e8ffff640 1 -- 172.21.15.120:0/3818340368 learned_addr learned my addr 172.21.15.120:0/3818340368 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.868+0000 7f9e94955640 1 --2- 172.21.15.120:0/3818340368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e90100a60 0x7f9e9010b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.868+0000 7f9e95156640 1 --2- 172.21.15.120:0/3818340368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e90102610 0x7f9e90109f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.868+0000 7f9e8ffff640 1 -- 172.21.15.120:0/3818340368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e90102610 msgr2=0x7f9e90109f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.868+0000 7f9e8ffff640 1 --2- 172.21.15.120:0/3818340368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e90102610 0x7f9e90109f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.868+0000 7f9e8ffff640 1 -- 172.21.15.120:0/3818340368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e90100a60 msgr2=0x7f9e9010b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.868+0000 7f9e8ffff640 1 --2- 172.21.15.120:0/3818340368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e90100a60 0x7f9e9010b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.868+0000 7f9e8ffff640 1 -- 172.21.15.120:0/3818340368 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e9010a7f0 con 0x7f9e90101c50 2024-09-19T09:07:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.868+0000 7f9e94955640 1 --2- 172.21.15.120:0/3818340368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e90100a60 0x7f9e9010b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:07:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.868+0000 7f9e8ffff640 1 --2- 172.21.15.120:0/3818340368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e90101c50 0x7f9e90109a20 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00b4a0 tx=0x7f9e7c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.869+0000 7f9e8dffb640 1 -- 172.21.15.120:0/3818340368 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c014070 con 0x7f9e90101c50 2024-09-19T09:07:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.869+0000 7f9e8dffb640 1 -- 172.21.15.120:0/3818340368 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e7c010af0 con 0x7f9e90101c50 2024-09-19T09:07:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.869+0000 7f9e8dffb640 1 -- 172.21.15.120:0/3818340368 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c017d10 con 0x7f9e90101c50 2024-09-19T09:07:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.869+0000 7f9e96bb9640 1 -- 172.21.15.120:0/3818340368 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e901a1f20 con 0x7f9e90101c50 2024-09-19T09:07:26.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.869+0000 7f9e96bb9640 1 -- 172.21.15.120:0/3818340368 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e901a2400 con 0x7f9e90101c50 2024-09-19T09:07:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.871+0000 7f9e8dffb640 1 -- 172.21.15.120:0/3818340368 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f9e7c017070 con 0x7f9e90101c50 2024-09-19T09:07:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.872+0000 7f9e99a27740 1 Processor -- start 2024-09-19T09:07:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.872+0000 7f9e99a27740 1 -- start start 2024-09-19T09:07:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.872+0000 7f9e99a27740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16b469a0 0x562f16b46da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.872+0000 7f9e99a27740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f16b472e0 0x562f16b4bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.872+0000 7f9e8dffb640 1 --2- 172.21.15.120:0/3818340368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9e800a3370 0x7f9e800a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.872+0000 7f9e99a27740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f16b4c370 0x562f16b4e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.872+0000 7f9e99a27740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562f16aaa2f0 con 0x562f16b469a0 2024-09-19T09:07:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.872+0000 7f9e99a27740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562f16a52270 con 0x562f16b4c370 2024-09-19T09:07:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.872+0000 7f9e99a27740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562f16aa62b0 con 0x562f16b472e0 2024-09-19T09:07:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.872+0000 7f9e95156640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16b469a0 0x562f16b46da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.872+0000 7f9e8ffff640 1 --2- 172.21.15.120:0/3818340368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9e800a3370 0x7f9e800a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.872+0000 7f9e95156640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16b469a0 0x562f16b46da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42694/0 (socket says 172.21.15.120:42694) 2024-09-19T09:07:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.872+0000 7f9e95156640 1 -- 172.21.15.120:0/2041333615 learned_addr learned my addr 172.21.15.120:0/2041333615 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.872+0000 7f9e94955640 1 --2- 172.21.15.120:0/2041333615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f16b472e0 0x562f16b4bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.872+0000 7f9e8dffb640 1 -- 172.21.15.120:0/3818340368 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f9e7c0ced10 con 0x7f9e90101c50 2024-09-19T09:07:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.872+0000 7f9e95156640 1 --2- 172.21.15.120:0/2041333615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f16b4c370 0x562f16b4e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.872+0000 7f9e95156640 1 -- 172.21.15.120:0/2041333615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f16b472e0 msgr2=0x562f16b4bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.872+0000 7f9e95156640 1 --2- 172.21.15.120:0/2041333615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f16b472e0 0x562f16b4bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.872+0000 7f9e95156640 1 -- 172.21.15.120:0/2041333615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f16b4c370 msgr2=0x562f16b4e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.872+0000 7f9e95156640 1 --2- 172.21.15.120:0/2041333615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f16b4c370 0x562f16b4e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.872+0000 7f9e95156640 1 -- 172.21.15.120:0/2041333615 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f16a51340 con 0x562f16b469a0 2024-09-19T09:07:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.873+0000 7f9e8ffff640 1 --2- 172.21.15.120:0/3818340368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9e800a3370 0x7f9e800a5830 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c097790 tx=0x7f9e7c090810 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.873+0000 7f9e95156640 1 --2- 172.21.15.120:0/2041333615 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16b469a0 0x562f16b46da0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f9e88012730 tx=0x7f9e88007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af824839e6a14ffc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.873+0000 7f9e7affd640 1 -- 172.21.15.120:0/2041333615 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e88004b20 con 0x562f16b469a0 2024-09-19T09:07:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.873+0000 7f9e7affd640 1 -- 172.21.15.120:0/2041333615 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e88003e50 con 0x562f16b469a0 2024-09-19T09:07:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.873+0000 7f9e7affd640 1 -- 172.21.15.120:0/2041333615 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e88004250 con 0x562f16b469a0 2024-09-19T09:07:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.873+0000 7f9e99a27740 1 -- 172.21.15.120:0/2041333615 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16b469a0 msgr2=0x562f16b46da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.873+0000 7f9e99a27740 1 --2- 172.21.15.120:0/2041333615 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16b469a0 0x562f16b46da0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f9e88012730 tx=0x7f9e88007860 comp rx=0 tx=0).stop 2024-09-19T09:07:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.873+0000 7f9e99a27740 1 -- 172.21.15.120:0/2041333615 shutdown_connections 2024-09-19T09:07:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.873+0000 7f9e99a27740 1 --2- 172.21.15.120:0/2041333615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f16b4c370 0x562f16b4e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.873+0000 7f9e99a27740 1 --2- 172.21.15.120:0/2041333615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f16b472e0 0x562f16b4bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.873+0000 7f9e99a27740 1 --2- 172.21.15.120:0/2041333615 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16b469a0 0x562f16b46da0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.873+0000 7f9e99a27740 1 -- 172.21.15.120:0/2041333615 >> 172.21.15.120:0/2041333615 conn(0x562f16b42010 msgr2=0x562f16b42410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.874+0000 7f9e99a27740 1 -- 172.21.15.120:0/2041333615 shutdown_connections 2024-09-19T09:07:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.874+0000 7f9e99a27740 1 -- 172.21.15.120:0/2041333615 wait complete. 2024-09-19T09:07:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.874+0000 7f9e99a27740 1 -- start start 2024-09-19T09:07:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.874+0000 7f9e99a27740 1 Processor -- start 2024-09-19T09:07:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.874+0000 7f9e99a27740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16b469a0 0x562f16b50b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.874+0000 7f9e99a27740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f16b472e0 0x562f16b510b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.874+0000 7f9e99a27740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f16b4c370 0x562f16b580f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.874+0000 7f9e99a27740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562f16b4f340 con 0x562f16b469a0 2024-09-19T09:07:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.874+0000 7f9e99a27740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562f16a33290 con 0x562f16b472e0 2024-09-19T09:07:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.874+0000 7f9e99a27740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562f16b4f4c0 con 0x562f16b4c370 2024-09-19T09:07:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.874+0000 7f9e99a27740 10 client.0 _ll_register_callbacks cb 0x562f16a64140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.874+0000 7f9e95156640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16b469a0 0x562f16b50b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.875+0000 7f9e95156640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16b469a0 0x562f16b50b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42706/0 (socket says 172.21.15.120:42706) 2024-09-19T09:07:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.875+0000 7f9e95156640 1 -- 172.21.15.120:0/1081350835 learned_addr learned my addr 172.21.15.120:0/1081350835 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.875+0000 7f9e95156640 1 --2- 172.21.15.120:0/1081350835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f16b4c370 0x562f16b580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.875+0000 7f9e94955640 1 --2- 172.21.15.120:0/1081350835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f16b472e0 0x562f16b510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.875+0000 7f9e95156640 1 -- 172.21.15.120:0/1081350835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f16b4c370 msgr2=0x562f16b580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.875+0000 7f9e95156640 1 --2- 172.21.15.120:0/1081350835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f16b4c370 0x562f16b580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.875+0000 7f9e95156640 1 -- 172.21.15.120:0/1081350835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f16b472e0 msgr2=0x562f16b510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.875+0000 7f9e95156640 1 --2- 172.21.15.120:0/1081350835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f16b472e0 0x562f16b510b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.875+0000 7f9e95156640 1 -- 172.21.15.120:0/1081350835 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f167bd210 con 0x562f16b469a0 2024-09-19T09:07:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.875+0000 7f9e95156640 1 --2- 172.21.15.120:0/1081350835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16b469a0 0x562f16b50b70 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f9e88012550 tx=0x7f9e88005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2f224e42319c06 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.875+0000 7f9e6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:07:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.875+0000 7f9e6ffff640 1 -- 172.21.15.120:0/1081350835 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e88021070 con 0x562f16b469a0 2024-09-19T09:07:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.875+0000 7f9e6ffff640 1 -- 172.21.15.120:0/1081350835 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f9e88004200 con 0x562f16b469a0 2024-09-19T09:07:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.875+0000 7f9e99a27740 1 -- 172.21.15.120:0/1081350835 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e88003e50 con 0x562f16b469a0 2024-09-19T09:07:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.875+0000 7f9e6ffff640 1 -- 172.21.15.120:0/1081350835 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9e88004400 con 0x562f16b469a0 2024-09-19T09:07:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.875+0000 7f9e99a27740 10 client.17022 fetch_fsmap learned FSMap version 30 2024-09-19T09:07:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.875+0000 7f9e99a27740 1 -- 172.21.15.120:0/1081350835 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562f16889620 con 0x562f16b469a0 2024-09-19T09:07:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.875+0000 7f9e6ffff640 1 -- 172.21.15.120:0/1081350835 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e88003e50 con 0x562f16b469a0 2024-09-19T09:07:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.876+0000 7f9e6ffff640 1 -- 172.21.15.120:0/1081350835 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9e8802b060 con 0x562f16b469a0 2024-09-19T09:07:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.876+0000 7f9e99a27740 10 client.17022 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:07:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.876+0000 7f9e99a27740 10 client.17022 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:07:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.876+0000 7f9e99a27740 20 client.17022 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.876+0000 7f9e99a27740 1 --2- 172.21.15.120:0/1081350835 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562f16b5c570 0x562f16b5e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.876+0000 7f9e99a27740 4 client.17022 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:07:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.876+0000 7f9e99a27740 1 -- 172.21.15.120:0/1081350835 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f16ab7a20 con 0x562f16b5c570 2024-09-19T09:07:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.876+0000 7f9e94955640 1 --2- 172.21.15.120:0/1081350835 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562f16b5c570 0x562f16b5e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.876+0000 7f9e94955640 1 --2- 172.21.15.120:0/1081350835 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562f16b5c570 0x562f16b5e9b0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eac651cbd932cbed server_cookie=25b3079882a3e58d in_seq=0 out_seq=0 2024-09-19T09:07:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.876+0000 7f9e6ffff640 10 client.17022 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.877+0000 7f9e6ffff640 1 -- 172.21.15.120:0/1081350835 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562f16ab7a20 con 0x562f16b5c570 2024-09-19T09:07:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.877+0000 7f9e6ffff640 10 client.17022 handle_command_reply: tid=0 2024-09-19T09:07:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.877+0000 7f9e99a27740 1 client.17022 shutdown 2024-09-19T09:07:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.877+0000 7f9e99a27740 1 -- 172.21.15.120:0/1081350835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16b469a0 msgr2=0x562f16b50b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.877+0000 7f9e99a27740 1 --2- 172.21.15.120:0/1081350835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16b469a0 0x562f16b50b70 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f9e88012550 tx=0x7f9e88005bf0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.877+0000 7f9e99a27740 1 -- 172.21.15.120:0/1081350835 shutdown_connections 2024-09-19T09:07:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.877+0000 7f9e99a27740 1 --2- 172.21.15.120:0/1081350835 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562f16b5c570 0x562f16b5e9b0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.877+0000 7f9e99a27740 1 --2- 172.21.15.120:0/1081350835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f16b4c370 0x562f16b580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.877+0000 7f9e6ffff640 0 client.17022 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.877+0000 7f9e99a27740 1 --2- 172.21.15.120:0/1081350835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f16b472e0 0x562f16b510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.877+0000 7f9e99a27740 1 --2- 172.21.15.120:0/1081350835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16b469a0 0x562f16b50b70 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.877+0000 7f9e99a27740 1 -- 172.21.15.120:0/1081350835 >> 172.21.15.120:0/1081350835 conn(0x562f16b42010 msgr2=0x562f16b43820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.878+0000 7f9e99a27740 1 -- 172.21.15.120:0/1081350835 shutdown_connections 2024-09-19T09:07:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.878+0000 7f9e99a27740 1 -- 172.21.15.120:0/1081350835 wait complete. 2024-09-19T09:07:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.878+0000 7f9e99a27740 20 client.17022 trim_cache size 0 max 16384 2024-09-19T09:07:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.892+0000 7f9e99a27740 1 Processor -- start 2024-09-19T09:07:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.892+0000 7f9e99a27740 1 -- start start 2024-09-19T09:07:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.893+0000 7f9e99a27740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16b469a0 0x562f16b419b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.893+0000 7f9e99a27740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f16b472e0 0x562f16b580b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.893+0000 7f9e99a27740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f16b4c370 0x562f16b50f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.893+0000 7f9e99a27740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9e901ab290 con 0x562f16b469a0 2024-09-19T09:07:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.893+0000 7f9e99a27740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9e901acdd0 con 0x562f16b472e0 2024-09-19T09:07:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.893+0000 7f9e99a27740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9e901ab960 con 0x562f16b4c370 2024-09-19T09:07:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.893+0000 7f9e95156640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16b469a0 0x562f16b419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.893+0000 7f9e95156640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16b469a0 0x562f16b419b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42712/0 (socket says 172.21.15.120:42712) 2024-09-19T09:07:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.893+0000 7f9e95156640 1 -- 172.21.15.120:0/3237479150 learned_addr learned my addr 172.21.15.120:0/3237479150 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.893+0000 7f9e95156640 1 -- 172.21.15.120:0/3237479150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f16b4c370 msgr2=0x562f16b50f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.893+0000 7f9e95156640 1 --2- 172.21.15.120:0/3237479150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f16b4c370 0x562f16b50f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.893+0000 7f9e95156640 1 -- 172.21.15.120:0/3237479150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f16b472e0 msgr2=0x562f16b580b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.893+0000 7f9e95156640 1 --2- 172.21.15.120:0/3237479150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f16b472e0 0x562f16b580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.893+0000 7f9e95156640 1 -- 172.21.15.120:0/3237479150 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f16b4f220 con 0x562f16b469a0 2024-09-19T09:07:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.893+0000 7f9e95156640 1 --2- 172.21.15.120:0/3237479150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16b469a0 0x562f16b419b0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f9e88002230 tx=0x7f9e88007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4620abf2fda949c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.893+0000 7f9e7a7fc640 1 -- 172.21.15.120:0/3237479150 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e88021070 con 0x562f16b469a0 2024-09-19T09:07:26.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.893+0000 7f9e7a7fc640 1 -- 172.21.15.120:0/3237479150 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e88022040 con 0x562f16b469a0 2024-09-19T09:07:26.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.893+0000 7f9e7a7fc640 1 -- 172.21.15.120:0/3237479150 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8802d7b0 con 0x562f16b469a0 2024-09-19T09:07:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.894+0000 7f9e99a27740 1 -- 172.21.15.120:0/3237479150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16b469a0 msgr2=0x562f16b419b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.894+0000 7f9e99a27740 1 --2- 172.21.15.120:0/3237479150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16b469a0 0x562f16b419b0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f9e88002230 tx=0x7f9e88007d30 comp rx=0 tx=0).stop 2024-09-19T09:07:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.894+0000 7f9e99a27740 1 -- 172.21.15.120:0/3237479150 shutdown_connections 2024-09-19T09:07:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.894+0000 7f9e99a27740 1 --2- 172.21.15.120:0/3237479150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f16b4c370 0x562f16b50f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.894+0000 7f9e99a27740 1 --2- 172.21.15.120:0/3237479150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f16b472e0 0x562f16b580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.894+0000 7f9e99a27740 1 --2- 172.21.15.120:0/3237479150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16b469a0 0x562f16b419b0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.894+0000 7f9e99a27740 1 -- 172.21.15.120:0/3237479150 >> 172.21.15.120:0/3237479150 conn(0x562f16b42010 msgr2=0x562f16b43740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.894+0000 7f9e99a27740 1 -- 172.21.15.120:0/3237479150 shutdown_connections 2024-09-19T09:07:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.894+0000 7f9e99a27740 1 -- 172.21.15.120:0/3237479150 wait complete. 2024-09-19T09:07:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.894+0000 7f9e99a27740 1 -- start start 2024-09-19T09:07:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.895+0000 7f9e99a27740 1 Processor -- start 2024-09-19T09:07:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.895+0000 7f9e99a27740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f16b469a0 0x562f16b55880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.895+0000 7f9e99a27740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16b472e0 0x562f16b55dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.895+0000 7f9e99a27740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f16b4c370 0x562f16b56dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.895+0000 7f9e99a27740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9e901ac700 con 0x562f16b472e0 2024-09-19T09:07:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.895+0000 7f9e99a27740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9e901ac030 con 0x562f16b469a0 2024-09-19T09:07:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.895+0000 7f9e99a27740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9e901a9250 con 0x562f16b4c370 2024-09-19T09:07:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.895+0000 7f9e99a27740 10 client.0 _ll_register_callbacks cb 0x7f9e84009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:26.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.895+0000 7f9e94955640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16b472e0 0x562f16b55dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:26.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.895+0000 7f9e94955640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16b472e0 0x562f16b55dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42722/0 (socket says 172.21.15.120:42722) 2024-09-19T09:07:26.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.895+0000 7f9e94955640 1 -- 172.21.15.120:0/1935682653 learned_addr learned my addr 172.21.15.120:0/1935682653 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:26.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.895+0000 7f9e95156640 1 --2- 172.21.15.120:0/1935682653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f16b469a0 0x562f16b55880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:26.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.895+0000 7f9e95156640 1 --2- 172.21.15.120:0/1935682653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f16b4c370 0x562f16b56dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:26.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.895+0000 7f9e94955640 1 -- 172.21.15.120:0/1935682653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f16b4c370 msgr2=0x562f16b56dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:26.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.895+0000 7f9e94955640 1 --2- 172.21.15.120:0/1935682653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f16b4c370 0x562f16b56dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.895+0000 7f9e94955640 1 -- 172.21.15.120:0/1935682653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f16b469a0 msgr2=0x562f16b55880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:26.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.895+0000 7f9e94955640 1 --2- 172.21.15.120:0/1935682653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f16b469a0 0x562f16b55880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.895+0000 7f9e94955640 1 -- 172.21.15.120:0/1935682653 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f16b573e0 con 0x562f16b472e0 2024-09-19T09:07:26.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.895+0000 7f9e95156640 1 --2- 172.21.15.120:0/1935682653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f16b4c370 0x562f16b56dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:07:26.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.895+0000 7f9e94955640 1 --2- 172.21.15.120:0/1935682653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16b472e0 0x562f16b55dc0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f9e84005e00 tx=0x7f9e8402aa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ed4725f7ff5a1a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:26.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.895+0000 7f9e6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:07:26.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.896+0000 7f9e6ffff640 1 -- 172.21.15.120:0/1935682653 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e84048030 con 0x562f16b472e0 2024-09-19T09:07:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.896+0000 7f9e6ffff640 1 -- 172.21.15.120:0/1935682653 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f9e84031a00 con 0x562f16b472e0 2024-09-19T09:07:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.896+0000 7f9e99a27740 1 -- 172.21.15.120:0/1935682653 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e88022040 con 0x562f16b472e0 2024-09-19T09:07:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.896+0000 7f9e6ffff640 1 -- 172.21.15.120:0/1935682653 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9e84035040 con 0x562f16b472e0 2024-09-19T09:07:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.896+0000 7f9e99a27740 10 client.17028 fetch_fsmap learned FSMap version 30 2024-09-19T09:07:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.896+0000 7f9e99a27740 1 -- 172.21.15.120:0/1935682653 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562f16bc7380 con 0x562f16b472e0 2024-09-19T09:07:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.896+0000 7f9e6ffff640 1 -- 172.21.15.120:0/1935682653 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e88022040 con 0x562f16b472e0 2024-09-19T09:07:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.896+0000 7f9e6ffff640 1 -- 172.21.15.120:0/1935682653 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9e84004a40 con 0x562f16b472e0 2024-09-19T09:07:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.896+0000 7f9e99a27740 10 client.17028 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:07:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.896+0000 7f9e99a27740 10 client.17028 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:07:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.896+0000 7f9e99a27740 20 client.17028 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.896+0000 7f9e99a27740 1 --2- 172.21.15.120:0/1935682653 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562f16bc7d60 0x562f16bca180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.896+0000 7f9e99a27740 4 client.17028 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:07:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.896+0000 7f9e99a27740 1 -- 172.21.15.120:0/1935682653 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562f16ab1d70 con 0x562f16bc7d60 2024-09-19T09:07:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.896+0000 7f9e95156640 1 --2- 172.21.15.120:0/1935682653 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562f16bc7d60 0x562f16bca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.897+0000 7f9e95156640 1 --2- 172.21.15.120:0/1935682653 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562f16bc7d60 0x562f16bca180 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=855fe733210a4218 server_cookie=191371ad32cb0007 in_seq=0 out_seq=0 2024-09-19T09:07:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.897+0000 7f9e6ffff640 10 client.17028 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:26.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.977+0000 7f9e6ffff640 1 -- 172.21.15.120:0/1935682653 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562f16ab1d70 con 0x562f16bc7d60 2024-09-19T09:07:26.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.977+0000 7f9e6ffff640 10 client.17028 handle_command_reply: tid=0 2024-09-19T09:07:26.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.977+0000 7f9e99a27740 1 client.17028 shutdown 2024-09-19T09:07:26.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.977+0000 7f9e99a27740 1 -- 172.21.15.120:0/1935682653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16b472e0 msgr2=0x562f16b55dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:26.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.977+0000 7f9e99a27740 1 --2- 172.21.15.120:0/1935682653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16b472e0 0x562f16b55dc0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f9e84005e00 tx=0x7f9e8402aa70 comp rx=0 tx=0).stop 2024-09-19T09:07:26.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.977+0000 7f9e99a27740 1 -- 172.21.15.120:0/1935682653 shutdown_connections 2024-09-19T09:07:26.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.977+0000 7f9e99a27740 1 --2- 172.21.15.120:0/1935682653 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562f16bc7d60 0x562f16bca180 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.977+0000 7f9e99a27740 1 --2- 172.21.15.120:0/1935682653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f16b4c370 0x562f16b56dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.977+0000 7f9e99a27740 1 --2- 172.21.15.120:0/1935682653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16b472e0 0x562f16b55dc0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.977+0000 7f9e6ffff640 0 client.17028 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.977+0000 7f9e99a27740 1 --2- 172.21.15.120:0/1935682653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f16b469a0 0x562f16b55880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.977+0000 7f9e99a27740 1 -- 172.21.15.120:0/1935682653 >> 172.21.15.120:0/1935682653 conn(0x562f16b42010 msgr2=0x562f16b409a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.978+0000 7f9e99a27740 1 -- 172.21.15.120:0/1935682653 shutdown_connections 2024-09-19T09:07:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.978+0000 7f9e99a27740 1 -- 172.21.15.120:0/1935682653 wait complete. 2024-09-19T09:07:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.978+0000 7f9e99a27740 20 client.17028 trim_cache size 0 max 16384 2024-09-19T09:07:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:07:26.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:07:26.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:07:26.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:07:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.978+0000 7f9e96bb9640 1 -- 172.21.15.120:0/3818340368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9e800a3370 msgr2=0x7f9e800a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.978+0000 7f9e96bb9640 1 --2- 172.21.15.120:0/3818340368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9e800a3370 0x7f9e800a5830 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c097790 tx=0x7f9e7c090810 comp rx=0 tx=0).stop 2024-09-19T09:07:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.978+0000 7f9e96bb9640 1 -- 172.21.15.120:0/3818340368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e90101c50 msgr2=0x7f9e90109a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.978+0000 7f9e96bb9640 1 --2- 172.21.15.120:0/3818340368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e90101c50 0x7f9e90109a20 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00b4a0 tx=0x7f9e7c00b960 comp rx=0 tx=0).stop 2024-09-19T09:07:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.978+0000 7f9e96bb9640 1 -- 172.21.15.120:0/3818340368 shutdown_connections 2024-09-19T09:07:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.978+0000 7f9e96bb9640 1 --2- 172.21.15.120:0/3818340368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9e800a3370 0x7f9e800a5830 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.978+0000 7f9e96bb9640 1 --2- 172.21.15.120:0/3818340368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e90102610 0x7f9e90109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.978+0000 7f9e96bb9640 1 --2- 172.21.15.120:0/3818340368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e90101c50 0x7f9e90109a20 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.978+0000 7f9e96bb9640 1 --2- 172.21.15.120:0/3818340368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e90100a60 0x7f9e9010b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.978+0000 7f9e96bb9640 1 -- 172.21.15.120:0/3818340368 >> 172.21.15.120:0/3818340368 conn(0x7f9e900fc1e0 msgr2=0x7f9e90104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.978+0000 7f9e96bb9640 1 -- 172.21.15.120:0/3818340368 shutdown_connections 2024-09-19T09:07:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:26.978+0000 7f9e96bb9640 1 -- 172.21.15.120:0/3818340368 wait complete. 2024-09-19T09:07:27.046 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 8.69 MB/sec execute 82 sec latency 19084.934 ms 2024-09-19T09:07:27.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:26 smithi196 ceph-mon[35150]: pgmap v479: 97 pgs: 97 active+clean; 41 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 2.2 MiB/s wr, 158 op/s 2024-09-19T09:07:27.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:26 smithi153 ceph-mon[35071]: pgmap v479: 97 pgs: 97 active+clean; 41 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 2.2 MiB/s wr, 158 op/s 2024-09-19T09:07:27.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:26 smithi120 ceph-mon[31150]: pgmap v479: 97 pgs: 97 active+clean; 41 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 2.2 MiB/s wr, 158 op/s 2024-09-19T09:07:27.664 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:07:27.917 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:07:28.046 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 8.59 MB/sec execute 83 sec latency 20084.968 ms 2024-09-19T09:07:28.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.438+0000 7f31780c7640 1 -- 172.21.15.120:0/1010117960 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3170102ef0 msgr2=0x7f317010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:28.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.438+0000 7f31780c7640 1 --2- 172.21.15.120:0/1010117960 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3170102ef0 0x7f317010da40 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f3168009ca0 tx=0x7f316802a950 comp rx=0 tx=0).stop 2024-09-19T09:07:28.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.438+0000 7f31780c7640 1 -- 172.21.15.120:0/1010117960 shutdown_connections 2024-09-19T09:07:28.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.438+0000 7f31780c7640 1 --2- 172.21.15.120:0/1010117960 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3170102ef0 0x7f317010da40 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.438+0000 7f31780c7640 1 --2- 172.21.15.120:0/1010117960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3170102570 0x7f31701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.438+0000 7f31780c7640 1 --2- 172.21.15.120:0/1010117960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3170100e20 0x7f3170101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.438+0000 7f31780c7640 1 -- 172.21.15.120:0/1010117960 >> 172.21.15.120:0/1010117960 conn(0x7f31700fc7d0 msgr2=0x7f31700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:28.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.438+0000 7f31780c7640 1 -- 172.21.15.120:0/1010117960 shutdown_connections 2024-09-19T09:07:28.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.438+0000 7f31780c7640 1 -- 172.21.15.120:0/1010117960 wait complete. 2024-09-19T09:07:28.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.439+0000 7f31780c7640 1 Processor -- start 2024-09-19T09:07:28.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.439+0000 7f31780c7640 1 -- start start 2024-09-19T09:07:28.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.439+0000 7f31780c7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3170100e20 0x7f3170198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:28.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.439+0000 7f31780c7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3170102570 0x7f3170198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:28.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.439+0000 7f31780c7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3170102ef0 0x7f317019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:28.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.439+0000 7f31780c7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3170110fc0 con 0x7f3170102570 2024-09-19T09:07:28.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.439+0000 7f31780c7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3170110e40 con 0x7f3170102ef0 2024-09-19T09:07:28.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.439+0000 7f31780c7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3170111140 con 0x7f3170100e20 2024-09-19T09:07:28.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.439+0000 7f3175662640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3170102570 0x7f3170198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:28.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.439+0000 7f3175662640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3170102570 0x7f3170198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42742/0 (socket says 172.21.15.120:42742) 2024-09-19T09:07:28.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.439+0000 7f3175662640 1 -- 172.21.15.120:0/3909154232 learned_addr learned my addr 172.21.15.120:0/3909154232 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:28.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.439+0000 7f3176664640 1 --2- 172.21.15.120:0/3909154232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3170102ef0 0x7f317019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:28.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.439+0000 7f3175e63640 1 --2- 172.21.15.120:0/3909154232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3170100e20 0x7f3170198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:28.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.439+0000 7f3175662640 1 -- 172.21.15.120:0/3909154232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3170100e20 msgr2=0x7f3170198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:28.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.439+0000 7f3175662640 1 --2- 172.21.15.120:0/3909154232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3170100e20 0x7f3170198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.439+0000 7f3175662640 1 -- 172.21.15.120:0/3909154232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3170102ef0 msgr2=0x7f317019ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:28.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.439+0000 7f3175662640 1 --2- 172.21.15.120:0/3909154232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3170102ef0 0x7f317019ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.439+0000 7f3175662640 1 -- 172.21.15.120:0/3909154232 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f317019d4e0 con 0x7f3170102570 2024-09-19T09:07:28.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.439+0000 7f3175e63640 1 --2- 172.21.15.120:0/3909154232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3170100e20 0x7f3170198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:07:28.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.439+0000 7f3176664640 1 --2- 172.21.15.120:0/3909154232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3170102ef0 0x7f317019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:07:28.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.439+0000 7f3175662640 1 --2- 172.21.15.120:0/3909154232 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3170102570 0x7f3170198a70 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f316400d770 tx=0x7f316400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:28.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.440+0000 7f3162ffd640 1 -- 172.21.15.120:0/3909154232 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3164017070 con 0x7f3170102570 2024-09-19T09:07:28.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.440+0000 7f3162ffd640 1 -- 172.21.15.120:0/3909154232 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3164005500 con 0x7f3170102570 2024-09-19T09:07:28.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.440+0000 7f31780c7640 1 -- 172.21.15.120:0/3909154232 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f317019d7d0 con 0x7f3170102570 2024-09-19T09:07:28.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.440+0000 7f3162ffd640 1 -- 172.21.15.120:0/3909154232 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3164004e00 con 0x7f3170102570 2024-09-19T09:07:28.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.440+0000 7f31780c7640 1 -- 172.21.15.120:0/3909154232 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31701a50b0 con 0x7f3170102570 2024-09-19T09:07:28.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.442+0000 7f3162ffd640 1 -- 172.21.15.120:0/3909154232 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f3164004fa0 con 0x7f3170102570 2024-09-19T09:07:28.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.442+0000 7f317a4a0740 1 Processor -- start 2024-09-19T09:07:28.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.442+0000 7f317a4a0740 1 -- start start 2024-09-19T09:07:28.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.443+0000 7f317a4a0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42c0b54a0 0x55f42c0b5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:28.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.443+0000 7f317a4a0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42c0b5dc0 0x55f42c0c0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:28.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.443+0000 7f3162ffd640 1 --2- 172.21.15.120:0/3909154232 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f315c0a3240 0x7f315c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:28.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.443+0000 7f317a4a0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42c0c0680 0x55f42c0c2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:28.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.443+0000 7f317a4a0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f42c01b2e0 con 0x55f42c0b5dc0 2024-09-19T09:07:28.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.443+0000 7f317a4a0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f42bfc2ce0 con 0x55f42c0b54a0 2024-09-19T09:07:28.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.443+0000 7f317a4a0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f42c0172a0 con 0x55f42c0c0680 2024-09-19T09:07:28.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.443+0000 7f3175e63640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42c0b5dc0 0x55f42c0c0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:28.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.443+0000 7f3175e63640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42c0b5dc0 0x55f42c0c0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42752/0 (socket says 172.21.15.120:42752) 2024-09-19T09:07:28.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.443+0000 7f3175e63640 1 -- 172.21.15.120:0/2376341235 learned_addr learned my addr 172.21.15.120:0/2376341235 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:28.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.443+0000 7f3175662640 1 --2- 172.21.15.120:0/3909154232 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f315c0a3240 0x7f315c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:28.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.443+0000 7f3162ffd640 1 -- 172.21.15.120:0/3909154232 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f31640cf440 con 0x7f3170102570 2024-09-19T09:07:28.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.443+0000 7f3176664640 1 --2- 172.21.15.120:0/2376341235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42c0b54a0 0x55f42c0b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:28.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.443+0000 7f3176664640 1 --2- 172.21.15.120:0/2376341235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42c0c0680 0x55f42c0c2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:28.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.443+0000 7f3175e63640 1 -- 172.21.15.120:0/2376341235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42c0c0680 msgr2=0x55f42c0c2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:28.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.443+0000 7f3175e63640 1 --2- 172.21.15.120:0/2376341235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42c0c0680 0x55f42c0c2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.443+0000 7f3175e63640 1 -- 172.21.15.120:0/2376341235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42c0b54a0 msgr2=0x55f42c0b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:28.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.443+0000 7f3175e63640 1 --2- 172.21.15.120:0/2376341235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42c0b54a0 0x55f42c0b5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.443+0000 7f3175e63640 1 -- 172.21.15.120:0/2376341235 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f42bfa7440 con 0x55f42c0b5dc0 2024-09-19T09:07:28.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.443+0000 7f3176664640 1 --2- 172.21.15.120:0/2376341235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42c0c0680 0x55f42c0c2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:07:28.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.443+0000 7f3175662640 1 --2- 172.21.15.120:0/3909154232 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f315c0a3240 0x7f315c0a5700 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3164098430 tx=0x7f3164009c30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:28.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.444+0000 7f3175e63640 1 --2- 172.21.15.120:0/2376341235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42c0b5dc0 0x55f42c0c0140 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f316c00aa10 tx=0x7f316c00aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c66a44b3bbf85b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:28.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.444+0000 7f315bfff640 1 -- 172.21.15.120:0/2376341235 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c015070 con 0x55f42c0b5dc0 2024-09-19T09:07:28.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.444+0000 7f315bfff640 1 -- 172.21.15.120:0/2376341235 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f316c00cc60 con 0x55f42c0b5dc0 2024-09-19T09:07:28.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.444+0000 7f315bfff640 1 -- 172.21.15.120:0/2376341235 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c0106c0 con 0x55f42c0b5dc0 2024-09-19T09:07:28.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.444+0000 7f317a4a0740 1 -- 172.21.15.120:0/2376341235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42c0b5dc0 msgr2=0x55f42c0c0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:28.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.444+0000 7f317a4a0740 1 --2- 172.21.15.120:0/2376341235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42c0b5dc0 0x55f42c0c0140 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f316c00aa10 tx=0x7f316c00aed0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.444+0000 7f317a4a0740 1 -- 172.21.15.120:0/2376341235 shutdown_connections 2024-09-19T09:07:28.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.444+0000 7f317a4a0740 1 --2- 172.21.15.120:0/2376341235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42c0c0680 0x55f42c0c2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.444+0000 7f317a4a0740 1 --2- 172.21.15.120:0/2376341235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42c0b5dc0 0x55f42c0c0140 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.444+0000 7f317a4a0740 1 --2- 172.21.15.120:0/2376341235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42c0b54a0 0x55f42c0b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.444+0000 7f317a4a0740 1 -- 172.21.15.120:0/2376341235 >> 172.21.15.120:0/2376341235 conn(0x55f42c0b1a30 msgr2=0x55f42c0b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:28.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.445+0000 7f317a4a0740 1 -- 172.21.15.120:0/2376341235 shutdown_connections 2024-09-19T09:07:28.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.445+0000 7f317a4a0740 1 -- 172.21.15.120:0/2376341235 wait complete. 2024-09-19T09:07:28.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.445+0000 7f317a4a0740 1 -- start start 2024-09-19T09:07:28.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.445+0000 7f317a4a0740 1 Processor -- start 2024-09-19T09:07:28.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.445+0000 7f317a4a0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42c0b54a0 0x55f42c0bff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:28.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.445+0000 7f317a4a0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42c0b5dc0 0x55f42c0bd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:28.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.445+0000 7f317a4a0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42c0c0680 0x55f42c0bd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:28.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.446+0000 7f317a4a0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f42c0c3250 con 0x55f42c0b54a0 2024-09-19T09:07:28.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.446+0000 7f317a4a0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f42bfd6050 con 0x55f42c0b5dc0 2024-09-19T09:07:28.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.446+0000 7f317a4a0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f42c0c33d0 con 0x55f42c0c0680 2024-09-19T09:07:28.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.446+0000 7f317a4a0740 10 client.0 _ll_register_callbacks cb 0x55f42c016160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:28.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.446+0000 7f3176664640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42c0b54a0 0x55f42c0bff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:28.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.446+0000 7f3176664640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42c0b54a0 0x55f42c0bff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42766/0 (socket says 172.21.15.120:42766) 2024-09-19T09:07:28.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.446+0000 7f3176664640 1 -- 172.21.15.120:0/4212213915 learned_addr learned my addr 172.21.15.120:0/4212213915 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:28.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.446+0000 7f3175e63640 1 --2- 172.21.15.120:0/4212213915 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42c0b5dc0 0x55f42c0bd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:28.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.446+0000 7f3176664640 1 -- 172.21.15.120:0/4212213915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42c0c0680 msgr2=0x55f42c0bd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:28.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.446+0000 7f3176664640 1 --2- 172.21.15.120:0/4212213915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42c0c0680 0x55f42c0bd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.446+0000 7f3176664640 1 -- 172.21.15.120:0/4212213915 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42c0b5dc0 msgr2=0x55f42c0bd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:28.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.446+0000 7f3176664640 1 --2- 172.21.15.120:0/4212213915 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42c0b5dc0 0x55f42c0bd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.446+0000 7f3176664640 1 -- 172.21.15.120:0/4212213915 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f42bd8c3c0 con 0x55f42c0b54a0 2024-09-19T09:07:28.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.446+0000 7f3175e63640 1 --2- 172.21.15.120:0/4212213915 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42c0b5dc0 0x55f42c0bd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:07:28.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.446+0000 7f3176664640 1 --2- 172.21.15.120:0/4212213915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42c0b54a0 0x55f42c0bff20 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f3168035000 tx=0x7f316802ff40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d431347a04170369 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:28.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.446+0000 7f31597fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:07:28.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.446+0000 7f31597fa640 1 -- 172.21.15.120:0/4212213915 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316804c060 con 0x55f42c0b54a0 2024-09-19T09:07:28.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.446+0000 7f31597fa640 1 -- 172.21.15.120:0/4212213915 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f3168003a30 con 0x55f42c0b54a0 2024-09-19T09:07:28.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.446+0000 7f317a4a0740 1 -- 172.21.15.120:0/4212213915 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f316c00cc60 con 0x55f42c0b54a0 2024-09-19T09:07:28.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.446+0000 7f31597fa640 1 -- 172.21.15.120:0/4212213915 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3168034040 con 0x55f42c0b54a0 2024-09-19T09:07:28.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.446+0000 7f317a4a0740 10 client.17040 fetch_fsmap learned FSMap version 30 2024-09-19T09:07:28.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.446+0000 7f317a4a0740 1 -- 172.21.15.120:0/4212213915 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f42c0ceb70 con 0x55f42c0b54a0 2024-09-19T09:07:28.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.447+0000 7f31597fa640 1 -- 172.21.15.120:0/4212213915 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f316c00cc60 con 0x55f42c0b54a0 2024-09-19T09:07:28.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.447+0000 7f31597fa640 1 -- 172.21.15.120:0/4212213915 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3168051070 con 0x55f42c0b54a0 2024-09-19T09:07:28.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.447+0000 7f317a4a0740 10 client.17040 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:07:28.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.447+0000 7f317a4a0740 10 client.17040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:07:28.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.447+0000 7f317a4a0740 20 client.17040 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:28.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.447+0000 7f317a4a0740 1 --2- 172.21.15.120:0/4212213915 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f42c0d0030 0x55f42c0d2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:28.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.447+0000 7f317a4a0740 4 client.17040 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:07:28.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.447+0000 7f317a4a0740 1 -- 172.21.15.120:0/4212213915 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f42c028850 con 0x55f42c0d0030 2024-09-19T09:07:28.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.447+0000 7f3175e63640 1 --2- 172.21.15.120:0/4212213915 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f42c0d0030 0x55f42c0d2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:28.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.447+0000 7f31597fa640 10 client.17040 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:28.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.447+0000 7f3175e63640 1 --2- 172.21.15.120:0/4212213915 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f42c0d0030 0x55f42c0d2420 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85a2c0e369c280e2 server_cookie=e830d5629b52d647 in_seq=0 out_seq=0 2024-09-19T09:07:28.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.448+0000 7f31597fa640 1 -- 172.21.15.120:0/4212213915 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f42c028850 con 0x55f42c0d0030 2024-09-19T09:07:28.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.448+0000 7f31597fa640 10 client.17040 handle_command_reply: tid=0 2024-09-19T09:07:28.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.448+0000 7f317a4a0740 1 client.17040 shutdown 2024-09-19T09:07:28.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.449+0000 7f317a4a0740 1 -- 172.21.15.120:0/4212213915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42c0b54a0 msgr2=0x55f42c0bff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:28.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.449+0000 7f317a4a0740 1 --2- 172.21.15.120:0/4212213915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42c0b54a0 0x55f42c0bff20 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f3168035000 tx=0x7f316802ff40 comp rx=0 tx=0).stop 2024-09-19T09:07:28.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.449+0000 7f317a4a0740 1 -- 172.21.15.120:0/4212213915 shutdown_connections 2024-09-19T09:07:28.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.449+0000 7f317a4a0740 1 --2- 172.21.15.120:0/4212213915 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f42c0d0030 0x55f42c0d2420 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.449+0000 7f317a4a0740 1 --2- 172.21.15.120:0/4212213915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42c0c0680 0x55f42c0bd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.449+0000 7f317a4a0740 1 --2- 172.21.15.120:0/4212213915 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42c0b5dc0 0x55f42c0bd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.449+0000 7f31597fa640 0 client.17040 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:28.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.449+0000 7f317a4a0740 1 --2- 172.21.15.120:0/4212213915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42c0b54a0 0x55f42c0bff20 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.449+0000 7f317a4a0740 1 -- 172.21.15.120:0/4212213915 >> 172.21.15.120:0/4212213915 conn(0x55f42c0b1a30 msgr2=0x55f42c0b0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:28.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.449+0000 7f317a4a0740 1 -- 172.21.15.120:0/4212213915 shutdown_connections 2024-09-19T09:07:28.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.449+0000 7f317a4a0740 1 -- 172.21.15.120:0/4212213915 wait complete. 2024-09-19T09:07:28.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.449+0000 7f317a4a0740 20 client.17040 trim_cache size 0 max 16384 2024-09-19T09:07:28.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.466+0000 7f317a4a0740 1 Processor -- start 2024-09-19T09:07:28.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.466+0000 7f317a4a0740 1 -- start start 2024-09-19T09:07:28.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.466+0000 7f317a4a0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42c0b54a0 0x55f42c0cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:28.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.466+0000 7f317a4a0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42c0b5dc0 0x55f42c0b7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:28.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.466+0000 7f317a4a0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42c0c0680 0x55f42c0c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:28.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.466+0000 7f317a4a0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f31701ad5e0 con 0x55f42c0c0680 2024-09-19T09:07:28.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.466+0000 7f317a4a0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f31701af120 con 0x55f42c0b54a0 2024-09-19T09:07:28.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.466+0000 7f317a4a0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f31701adcb0 con 0x55f42c0b5dc0 2024-09-19T09:07:28.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.466+0000 7f3176664640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42c0c0680 0x55f42c0c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:28.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.466+0000 7f3176664640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42c0c0680 0x55f42c0c8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42772/0 (socket says 172.21.15.120:42772) 2024-09-19T09:07:28.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.466+0000 7f3176664640 1 -- 172.21.15.120:0/2284702423 learned_addr learned my addr 172.21.15.120:0/2284702423 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:28.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.466+0000 7f3176664640 1 --2- 172.21.15.120:0/2284702423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42c0b54a0 0x55f42c0cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:28.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.466+0000 7f3175e63640 1 --2- 172.21.15.120:0/2284702423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42c0b5dc0 0x55f42c0b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:28.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.466+0000 7f3176664640 1 -- 172.21.15.120:0/2284702423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42c0b5dc0 msgr2=0x55f42c0b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:28.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.466+0000 7f3176664640 1 --2- 172.21.15.120:0/2284702423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42c0b5dc0 0x55f42c0b7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.466+0000 7f3176664640 1 -- 172.21.15.120:0/2284702423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42c0b54a0 msgr2=0x55f42c0cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:28.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.466+0000 7f3176664640 1 --2- 172.21.15.120:0/2284702423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42c0b54a0 0x55f42c0cc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.466+0000 7f3176664640 1 -- 172.21.15.120:0/2284702423 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f42c0b14d0 con 0x55f42c0c0680 2024-09-19T09:07:28.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.466+0000 7f3175e63640 1 --2- 172.21.15.120:0/2284702423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42c0b5dc0 0x55f42c0b7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:07:28.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.466+0000 7f3176664640 1 --2- 172.21.15.120:0/2284702423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42c0c0680 0x55f42c0c8a00 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f316802ffb0 tx=0x7f3168005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9549aa53dd5df5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:28.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.466+0000 7f315b7fe640 1 -- 172.21.15.120:0/2284702423 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316804c070 con 0x55f42c0c0680 2024-09-19T09:07:28.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.467+0000 7f315b7fe640 1 -- 172.21.15.120:0/2284702423 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31680043a0 con 0x55f42c0c0680 2024-09-19T09:07:28.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.467+0000 7f315b7fe640 1 -- 172.21.15.120:0/2284702423 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3168003c90 con 0x55f42c0c0680 2024-09-19T09:07:28.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.467+0000 7f317a4a0740 1 -- 172.21.15.120:0/2284702423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42c0c0680 msgr2=0x55f42c0c8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:28.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.467+0000 7f317a4a0740 1 --2- 172.21.15.120:0/2284702423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42c0c0680 0x55f42c0c8a00 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f316802ffb0 tx=0x7f3168005ef0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.467+0000 7f317a4a0740 1 -- 172.21.15.120:0/2284702423 shutdown_connections 2024-09-19T09:07:28.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.467+0000 7f317a4a0740 1 --2- 172.21.15.120:0/2284702423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42c0c0680 0x55f42c0c8a00 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.467+0000 7f317a4a0740 1 --2- 172.21.15.120:0/2284702423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42c0b5dc0 0x55f42c0b7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.467+0000 7f317a4a0740 1 --2- 172.21.15.120:0/2284702423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42c0b54a0 0x55f42c0cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.467+0000 7f317a4a0740 1 -- 172.21.15.120:0/2284702423 >> 172.21.15.120:0/2284702423 conn(0x55f42c0b1a30 msgr2=0x55f42c0b6ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:28.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.467+0000 7f317a4a0740 1 -- 172.21.15.120:0/2284702423 shutdown_connections 2024-09-19T09:07:28.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.467+0000 7f317a4a0740 1 -- 172.21.15.120:0/2284702423 wait complete. 2024-09-19T09:07:28.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.467+0000 7f317a4a0740 1 -- start start 2024-09-19T09:07:28.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.468+0000 7f317a4a0740 1 Processor -- start 2024-09-19T09:07:28.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.468+0000 7f317a4a0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42c0b54a0 0x55f42c132e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:28.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.468+0000 7f317a4a0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42c0b5dc0 0x55f42c133380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:28.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.468+0000 7f317a4a0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42c0c0680 0x55f42c136d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:28.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.468+0000 7f317a4a0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f31701aea50 con 0x55f42c0c0680 2024-09-19T09:07:28.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.468+0000 7f317a4a0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f31701ae380 con 0x55f42c0b54a0 2024-09-19T09:07:28.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.468+0000 7f317a4a0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f31701ab5a0 con 0x55f42c0b5dc0 2024-09-19T09:07:28.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.468+0000 7f317a4a0740 10 client.0 _ll_register_callbacks cb 0x7f3168009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:28.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.468+0000 7f3176664640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42c0c0680 0x55f42c136d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:28.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.468+0000 7f3176664640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42c0c0680 0x55f42c136d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42782/0 (socket says 172.21.15.120:42782) 2024-09-19T09:07:28.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.468+0000 7f3176664640 1 -- 172.21.15.120:0/394862395 learned_addr learned my addr 172.21.15.120:0/394862395 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:28.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.468+0000 7f3176664640 1 --2- 172.21.15.120:0/394862395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42c0b54a0 0x55f42c132e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:28.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.468+0000 7f3175e63640 1 --2- 172.21.15.120:0/394862395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42c0b5dc0 0x55f42c133380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:28.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.468+0000 7f3176664640 1 -- 172.21.15.120:0/394862395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42c0b5dc0 msgr2=0x55f42c133380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:28.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.468+0000 7f3176664640 1 --2- 172.21.15.120:0/394862395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42c0b5dc0 0x55f42c133380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.468+0000 7f3176664640 1 -- 172.21.15.120:0/394862395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42c0b54a0 msgr2=0x55f42c132e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:28.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.468+0000 7f3176664640 1 --2- 172.21.15.120:0/394862395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42c0b54a0 0x55f42c132e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.468+0000 7f3176664640 1 -- 172.21.15.120:0/394862395 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f42c137420 con 0x55f42c0c0680 2024-09-19T09:07:28.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.468+0000 7f3175e63640 1 --2- 172.21.15.120:0/394862395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42c0b5dc0 0x55f42c133380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:07:28.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.469+0000 7f3176664640 1 --2- 172.21.15.120:0/394862395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42c0c0680 0x55f42c136d80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f316802ae70 tx=0x7f31680047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5442fcc6a2b801e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:28.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.469+0000 7f31597fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:07:28.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.469+0000 7f31597fa640 1 -- 172.21.15.120:0/394862395 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3168057040 con 0x55f42c0c0680 2024-09-19T09:07:28.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.469+0000 7f31597fa640 1 -- 172.21.15.120:0/394862395 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f3168030d30 con 0x55f42c0c0680 2024-09-19T09:07:28.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.469+0000 7f317a4a0740 1 -- 172.21.15.120:0/394862395 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31680043a0 con 0x55f42c0c0680 2024-09-19T09:07:28.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.469+0000 7f31597fa640 1 -- 172.21.15.120:0/394862395 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f316804c040 con 0x55f42c0c0680 2024-09-19T09:07:28.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.469+0000 7f317a4a0740 10 client.17046 fetch_fsmap learned FSMap version 30 2024-09-19T09:07:28.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.469+0000 7f317a4a0740 1 -- 172.21.15.120:0/394862395 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f42c13f930 con 0x55f42c0c0680 2024-09-19T09:07:28.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.469+0000 7f31597fa640 1 -- 172.21.15.120:0/394862395 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31680043a0 con 0x55f42c0c0680 2024-09-19T09:07:28.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.469+0000 7f31597fa640 1 -- 172.21.15.120:0/394862395 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3168051040 con 0x55f42c0c0680 2024-09-19T09:07:28.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.469+0000 7f317a4a0740 10 client.17046 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:07:28.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.469+0000 7f317a4a0740 10 client.17046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:07:28.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.469+0000 7f317a4a0740 20 client.17046 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:28.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.469+0000 7f317a4a0740 1 --2- 172.21.15.120:0/394862395 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f42c13ff30 0x55f42c142320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:28.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.469+0000 7f317a4a0740 4 client.17046 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:07:28.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.469+0000 7f317a4a0740 1 -- 172.21.15.120:0/394862395 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f42c022d60 con 0x55f42c13ff30 2024-09-19T09:07:28.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.470+0000 7f3175e63640 1 --2- 172.21.15.120:0/394862395 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f42c13ff30 0x55f42c142320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:28.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.470+0000 7f31597fa640 10 client.17046 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:28.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.470+0000 7f3175e63640 1 --2- 172.21.15.120:0/394862395 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f42c13ff30 0x55f42c142320 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caab8b9b5e0a4dd4 server_cookie=dd90948d07d28b84 in_seq=0 out_seq=0 2024-09-19T09:07:28.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.470+0000 7f31597fa640 1 -- 172.21.15.120:0/394862395 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f42c022d60 con 0x55f42c13ff30 2024-09-19T09:07:28.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.470+0000 7f31597fa640 10 client.17046 handle_command_reply: tid=0 2024-09-19T09:07:28.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.470+0000 7f317a4a0740 1 client.17046 shutdown 2024-09-19T09:07:28.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.470+0000 7f317a4a0740 1 -- 172.21.15.120:0/394862395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42c0c0680 msgr2=0x55f42c136d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:28.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.470+0000 7f317a4a0740 1 --2- 172.21.15.120:0/394862395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42c0c0680 0x55f42c136d80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f316802ae70 tx=0x7f31680047e0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.471+0000 7f317a4a0740 1 -- 172.21.15.120:0/394862395 shutdown_connections 2024-09-19T09:07:28.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.471+0000 7f317a4a0740 1 --2- 172.21.15.120:0/394862395 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f42c13ff30 0x55f42c142320 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.471+0000 7f317a4a0740 1 --2- 172.21.15.120:0/394862395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42c0c0680 0x55f42c136d80 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.471+0000 7f31597fa640 0 client.17046 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:28.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.471+0000 7f317a4a0740 1 --2- 172.21.15.120:0/394862395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42c0b5dc0 0x55f42c133380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.471+0000 7f317a4a0740 1 --2- 172.21.15.120:0/394862395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42c0b54a0 0x55f42c132e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.471+0000 7f317a4a0740 1 -- 172.21.15.120:0/394862395 >> 172.21.15.120:0/394862395 conn(0x55f42c0b1a30 msgr2=0x55f42c0b7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:28.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.471+0000 7f317a4a0740 1 -- 172.21.15.120:0/394862395 shutdown_connections 2024-09-19T09:07:28.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.471+0000 7f317a4a0740 1 -- 172.21.15.120:0/394862395 wait complete. 2024-09-19T09:07:28.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.471+0000 7f317a4a0740 20 client.17046 trim_cache size 0 max 16384 2024-09-19T09:07:28.489 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:07:28.489 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:07:28.489 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:07:28.489 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:07:28.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.471+0000 7f31780c7640 1 -- 172.21.15.120:0/3909154232 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f315c0a3240 msgr2=0x7f315c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:28.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.471+0000 7f31780c7640 1 --2- 172.21.15.120:0/3909154232 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f315c0a3240 0x7f315c0a5700 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3164098430 tx=0x7f3164009c30 comp rx=0 tx=0).stop 2024-09-19T09:07:28.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.471+0000 7f31780c7640 1 -- 172.21.15.120:0/3909154232 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3170102570 msgr2=0x7f3170198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:28.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.471+0000 7f31780c7640 1 --2- 172.21.15.120:0/3909154232 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3170102570 0x7f3170198a70 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f316400d770 tx=0x7f316400dc30 comp rx=0 tx=0).stop 2024-09-19T09:07:28.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.471+0000 7f31780c7640 1 -- 172.21.15.120:0/3909154232 shutdown_connections 2024-09-19T09:07:28.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.471+0000 7f31780c7640 1 --2- 172.21.15.120:0/3909154232 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f315c0a3240 0x7f315c0a5700 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.471+0000 7f31780c7640 1 --2- 172.21.15.120:0/3909154232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3170102ef0 0x7f317019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.471+0000 7f31780c7640 1 --2- 172.21.15.120:0/3909154232 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3170102570 0x7f3170198a70 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.471+0000 7f31780c7640 1 --2- 172.21.15.120:0/3909154232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3170100e20 0x7f3170198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:28.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.471+0000 7f31780c7640 1 -- 172.21.15.120:0/3909154232 >> 172.21.15.120:0/3909154232 conn(0x7f31700fc7d0 msgr2=0x7f317006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:28.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.472+0000 7f31780c7640 1 -- 172.21.15.120:0/3909154232 shutdown_connections 2024-09-19T09:07:28.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:28.472+0000 7f31780c7640 1 -- 172.21.15.120:0/3909154232 wait complete. 2024-09-19T09:07:29.046 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 8.49 MB/sec execute 84 sec latency 21085.039 ms 2024-09-19T09:07:29.104 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:28 smithi120 ceph-mon[31150]: pgmap v480: 97 pgs: 97 active+clean; 38 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 2.2 MiB/s wr, 152 op/s 2024-09-19T09:07:29.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:28 smithi196 ceph-mon[35150]: pgmap v480: 97 pgs: 97 active+clean; 38 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 2.2 MiB/s wr, 152 op/s 2024-09-19T09:07:29.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:28 smithi153 ceph-mon[35071]: pgmap v480: 97 pgs: 97 active+clean; 38 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 2.2 MiB/s wr, 152 op/s 2024-09-19T09:07:29.409 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:07:29.409 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:07:29.655 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:07:30.046 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 8.39 MB/sec execute 85 sec latency 22085.113 ms 2024-09-19T09:07:30.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.162+0000 7f0d29068640 1 -- 172.21.15.120:0/2446348601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d24101420 msgr2=0x7f0d2410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:30.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.162+0000 7f0d29068640 1 --2- 172.21.15.120:0/2446348601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d24101420 0x7f0d2410d950 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c009cd0 tx=0x7f0d1c02a8e0 comp rx=0 tx=0).stop 2024-09-19T09:07:30.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.162+0000 7f0d29068640 1 -- 172.21.15.120:0/2446348601 shutdown_connections 2024-09-19T09:07:30.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.162+0000 7f0d29068640 1 --2- 172.21.15.120:0/2446348601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d24101420 0x7f0d2410d950 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:30.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.162+0000 7f0d29068640 1 --2- 172.21.15.120:0/2446348601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d24100a80 0x7f0d24100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:30.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.162+0000 7f0d29068640 1 --2- 172.21.15.120:0/2446348601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d24106a80 0x7f0d24106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:30.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.162+0000 7f0d29068640 1 -- 172.21.15.120:0/2446348601 >> 172.21.15.120:0/2446348601 conn(0x7f0d240fc7a0 msgr2=0x7f0d240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:30.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.162+0000 7f0d29068640 1 -- 172.21.15.120:0/2446348601 shutdown_connections 2024-09-19T09:07:30.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.162+0000 7f0d29068640 1 -- 172.21.15.120:0/2446348601 wait complete. 2024-09-19T09:07:30.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.163+0000 7f0d29068640 1 Processor -- start 2024-09-19T09:07:30.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.163+0000 7f0d29068640 1 -- start start 2024-09-19T09:07:30.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.163+0000 7f0d29068640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d24100a80 0x7f0d241972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:30.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.163+0000 7f0d29068640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d24101420 0x7f0d24197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:30.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.163+0000 7f0d29068640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d24106a80 0x7f0d2419bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:30.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.163+0000 7f0d29068640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0d24110ea0 con 0x7f0d24100a80 2024-09-19T09:07:30.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.163+0000 7f0d29068640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0d24110d20 con 0x7f0d24106a80 2024-09-19T09:07:30.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.163+0000 7f0d29068640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0d24111020 con 0x7f0d24101420 2024-09-19T09:07:30.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.163+0000 7f0d22d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d24100a80 0x7f0d241972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:30.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.163+0000 7f0d22d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d24100a80 0x7f0d241972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42814/0 (socket says 172.21.15.120:42814) 2024-09-19T09:07:30.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.163+0000 7f0d22d9d640 1 -- 172.21.15.120:0/2848297451 learned_addr learned my addr 172.21.15.120:0/2848297451 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:30.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.163+0000 7f0d2259c640 1 --2- 172.21.15.120:0/2848297451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d24101420 0x7f0d24197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:30.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.164+0000 7f0d2359e640 1 --2- 172.21.15.120:0/2848297451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d24106a80 0x7f0d2419bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:30.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.164+0000 7f0d22d9d640 1 -- 172.21.15.120:0/2848297451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d24101420 msgr2=0x7f0d24197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:30.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.164+0000 7f0d22d9d640 1 --2- 172.21.15.120:0/2848297451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d24101420 0x7f0d24197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:30.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.164+0000 7f0d22d9d640 1 -- 172.21.15.120:0/2848297451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d24106a80 msgr2=0x7f0d2419bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:30.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.164+0000 7f0d22d9d640 1 --2- 172.21.15.120:0/2848297451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d24106a80 0x7f0d2419bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:30.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.164+0000 7f0d22d9d640 1 -- 172.21.15.120:0/2848297451 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d2419c290 con 0x7f0d24100a80 2024-09-19T09:07:30.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.164+0000 7f0d2259c640 1 --2- 172.21.15.120:0/2848297451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d24101420 0x7f0d24197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:07:30.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.164+0000 7f0d2359e640 1 --2- 172.21.15.120:0/2848297451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d24106a80 0x7f0d2419bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:07:30.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.164+0000 7f0d22d9d640 1 --2- 172.21.15.120:0/2848297451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d24100a80 0x7f0d241972e0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f0d1400d990 tx=0x7f0d1400de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:30.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.164+0000 7f0d0bfff640 1 -- 172.21.15.120:0/2848297451 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1401b070 con 0x7f0d24100a80 2024-09-19T09:07:30.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.164+0000 7f0d0bfff640 1 -- 172.21.15.120:0/2848297451 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d1400edf0 con 0x7f0d24100a80 2024-09-19T09:07:30.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.164+0000 7f0d29068640 1 -- 172.21.15.120:0/2848297451 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d2419c580 con 0x7f0d24100a80 2024-09-19T09:07:30.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.164+0000 7f0d0bfff640 1 -- 172.21.15.120:0/2848297451 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d14014870 con 0x7f0d24100a80 2024-09-19T09:07:30.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.164+0000 7f0d29068640 1 -- 172.21.15.120:0/2848297451 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d2406a790 con 0x7f0d24100a80 2024-09-19T09:07:30.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.165+0000 7f0d29068640 1 -- 172.21.15.120:0/2848297451 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d241021c0 con 0x7f0d24100a80 2024-09-19T09:07:30.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.166+0000 7f0d0bfff640 1 -- 172.21.15.120:0/2848297451 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f0d1401e070 con 0x7f0d24100a80 2024-09-19T09:07:30.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.167+0000 7f0d0bfff640 1 --2- 172.21.15.120:0/2848297451 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0d0c0a3240 0x7f0d0c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:30.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.167+0000 7f0d2259c640 1 --2- 172.21.15.120:0/2848297451 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0d0c0a3240 0x7f0d0c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:30.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.167+0000 7f0d0bfff640 1 -- 172.21.15.120:0/2848297451 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f0d140d1580 con 0x7f0d24100a80 2024-09-19T09:07:30.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.167+0000 7f0d2259c640 1 --2- 172.21.15.120:0/2848297451 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0d0c0a3240 0x7f0d0c0a5700 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0d240fec60 tx=0x7f0d1800a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:30.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.170+0000 7f0d0bfff640 1 -- 172.21.15.120:0/2848297451 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0d1409e2a0 con 0x7f0d24100a80 2024-09-19T09:07:30.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.372+0000 7f0d29068640 1 -- 172.21.15.120:0/2848297451 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d24100ee0 con 0x7f0d24100a80 2024-09-19T09:07:30.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.373+0000 7f0d0bfff640 1 -- 172.21.15.120:0/2848297451 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11985 (secure 0 0 0) 0x7f0d1409dba0 con 0x7f0d24100a80 2024-09-19T09:07:30.373 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:07:30.374 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":30,"btime":"2024-09-19T09:07:14:812065+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:07:14.812046+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26351":{"gid":26351,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/364941779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":364941779},{"type":"v1","addr":"172.21.15.153:6835","nonce":364941779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:07:30.374 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 30 2024-09-19T09:07:30.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.375+0000 7f0d29068640 1 -- 172.21.15.120:0/2848297451 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0d0c0a3240 msgr2=0x7f0d0c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:30.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.375+0000 7f0d29068640 1 --2- 172.21.15.120:0/2848297451 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0d0c0a3240 0x7f0d0c0a5700 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0d240fec60 tx=0x7f0d1800a5c0 comp rx=0 tx=0).stop 2024-09-19T09:07:30.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.375+0000 7f0d29068640 1 -- 172.21.15.120:0/2848297451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d24100a80 msgr2=0x7f0d241972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:30.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.375+0000 7f0d29068640 1 --2- 172.21.15.120:0/2848297451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d24100a80 0x7f0d241972e0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f0d1400d990 tx=0x7f0d1400de50 comp rx=0 tx=0).stop 2024-09-19T09:07:30.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.375+0000 7f0d29068640 1 -- 172.21.15.120:0/2848297451 shutdown_connections 2024-09-19T09:07:30.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.375+0000 7f0d29068640 1 --2- 172.21.15.120:0/2848297451 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0d0c0a3240 0x7f0d0c0a5700 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:30.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.375+0000 7f0d29068640 1 --2- 172.21.15.120:0/2848297451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d24106a80 0x7f0d2419bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:30.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.375+0000 7f0d29068640 1 --2- 172.21.15.120:0/2848297451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d24101420 0x7f0d24197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:30.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.375+0000 7f0d29068640 1 --2- 172.21.15.120:0/2848297451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d24100a80 0x7f0d241972e0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:30.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.375+0000 7f0d29068640 1 -- 172.21.15.120:0/2848297451 >> 172.21.15.120:0/2848297451 conn(0x7f0d240fc7a0 msgr2=0x7f0d240fcd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:30.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.375+0000 7f0d29068640 1 -- 172.21.15.120:0/2848297451 shutdown_connections 2024-09-19T09:07:30.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:30.375+0000 7f0d29068640 1 -- 172.21.15.120:0/2848297451 wait complete. 2024-09-19T09:07:31.047 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 8.29 MB/sec execute 86 sec latency 23085.204 ms 2024-09-19T09:07:31.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:30 smithi120 ceph-mon[31150]: pgmap v481: 97 pgs: 97 active+clean; 32 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 2.3 MiB/s wr, 140 op/s 2024-09-19T09:07:31.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:30 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2848297451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:07:31.106 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:07:31.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:30 smithi196 ceph-mon[35150]: pgmap v481: 97 pgs: 97 active+clean; 32 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 2.3 MiB/s wr, 140 op/s 2024-09-19T09:07:31.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:30 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2848297451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:07:31.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:30 smithi153 ceph-mon[35071]: pgmap v481: 97 pgs: 97 active+clean; 32 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 2.3 MiB/s wr, 140 op/s 2024-09-19T09:07:31.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:30 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2848297451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:07:31.350 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:07:31.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.965+0000 7fb98895f640 1 -- 172.21.15.120:0/3787210716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb984100e30 msgr2=0x7fb984104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:31.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.965+0000 7fb98895f640 1 --2- 172.21.15.120:0/3787210716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb984100e30 0x7fb984104e80 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009900 tx=0x7fb96c02a410 comp rx=0 tx=0).stop 2024-09-19T09:07:31.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.965+0000 7fb98895f640 1 -- 172.21.15.120:0/3787210716 shutdown_connections 2024-09-19T09:07:31.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.965+0000 7fb98895f640 1 --2- 172.21.15.120:0/3787210716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9841055b0 0x7fb98410d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:31.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.965+0000 7fb98895f640 1 --2- 172.21.15.120:0/3787210716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb984100e30 0x7fb984104e80 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:31.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.965+0000 7fb98895f640 1 --2- 172.21.15.120:0/3787210716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb984100480 0x7fb984100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:31.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.965+0000 7fb98895f640 1 -- 172.21.15.120:0/3787210716 >> 172.21.15.120:0/3787210716 conn(0x7fb984077210 msgr2=0x7fb9840fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:31.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.965+0000 7fb98895f640 1 -- 172.21.15.120:0/3787210716 shutdown_connections 2024-09-19T09:07:31.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.965+0000 7fb98895f640 1 -- 172.21.15.120:0/3787210716 wait complete. 2024-09-19T09:07:31.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.966+0000 7fb98895f640 1 Processor -- start 2024-09-19T09:07:31.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.966+0000 7fb98895f640 1 -- start start 2024-09-19T09:07:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.966+0000 7fb98895f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb984100480 0x7fb984198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.966+0000 7fb98895f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb984100e30 0x7fb984198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.966+0000 7fb98895f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9841055b0 0x7fb98419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.966+0000 7fb98895f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb984111f90 con 0x7fb984100480 2024-09-19T09:07:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.966+0000 7fb98895f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb984111e10 con 0x7fb984100e30 2024-09-19T09:07:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.966+0000 7fb98895f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb984112110 con 0x7fb9841055b0 2024-09-19T09:07:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.966+0000 7fb98259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb984100480 0x7fb984198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.966+0000 7fb98259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb984100480 0x7fb984198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56438/0 (socket says 172.21.15.120:56438) 2024-09-19T09:07:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.966+0000 7fb98259c640 1 -- 172.21.15.120:0/4234846212 learned_addr learned my addr 172.21.15.120:0/4234846212 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.966+0000 7fb982d9d640 1 --2- 172.21.15.120:0/4234846212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9841055b0 0x7fb98419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.966+0000 7fb981d9b640 1 --2- 172.21.15.120:0/4234846212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb984100e30 0x7fb984198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.967+0000 7fb98259c640 1 -- 172.21.15.120:0/4234846212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9841055b0 msgr2=0x7fb98419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:31.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.967+0000 7fb98259c640 1 --2- 172.21.15.120:0/4234846212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9841055b0 0x7fb98419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.967+0000 7fb98259c640 1 -- 172.21.15.120:0/4234846212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb984100e30 msgr2=0x7fb984198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.967+0000 7fb98259c640 1 --2- 172.21.15.120:0/4234846212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb984100e30 0x7fb984198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.967+0000 7fb98259c640 1 -- 172.21.15.120:0/4234846212 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98419d400 con 0x7fb984100480 2024-09-19T09:07:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.967+0000 7fb98259c640 1 --2- 172.21.15.120:0/4234846212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb984100480 0x7fb984198450 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fb97400e9f0 tx=0x7fb97400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.967+0000 7fb97b7fe640 1 -- 172.21.15.120:0/4234846212 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97400cd80 con 0x7fb984100480 2024-09-19T09:07:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.967+0000 7fb98895f640 1 -- 172.21.15.120:0/4234846212 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb98419d6f0 con 0x7fb984100480 2024-09-19T09:07:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.967+0000 7fb97b7fe640 1 -- 172.21.15.120:0/4234846212 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb974005500 con 0x7fb984100480 2024-09-19T09:07:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.967+0000 7fb97b7fe640 1 -- 172.21.15.120:0/4234846212 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb974018630 con 0x7fb984100480 2024-09-19T09:07:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.967+0000 7fb98895f640 1 -- 172.21.15.120:0/4234846212 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9841a4fd0 con 0x7fb984100480 2024-09-19T09:07:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.969+0000 7fb97b7fe640 1 -- 172.21.15.120:0/4234846212 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fb9740046c0 con 0x7fb984100480 2024-09-19T09:07:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.969+0000 7fb98ad38740 1 Processor -- start 2024-09-19T09:07:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.970+0000 7fb98ad38740 1 -- start start 2024-09-19T09:07:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.970+0000 7fb98ad38740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5620aaf0ab40 0x5620aaf09090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:31.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.970+0000 7fb98ad38740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620aaf095d0 0x5620aaf09a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.970+0000 7fb98ad38740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620aaf14370 0x5620aaf16760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.970+0000 7fb98ad38740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5620aae6f2f0 con 0x5620aaf095d0 2024-09-19T09:07:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.970+0000 7fb98ad38740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5620aae16cf0 con 0x5620aaf14370 2024-09-19T09:07:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.970+0000 7fb98ad38740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5620aae6b2b0 con 0x5620aaf0ab40 2024-09-19T09:07:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.970+0000 7fb98259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620aaf095d0 0x5620aaf09a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.970+0000 7fb98259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620aaf095d0 0x5620aaf09a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56446/0 (socket says 172.21.15.120:56446) 2024-09-19T09:07:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.970+0000 7fb98259c640 1 -- 172.21.15.120:0/3379313552 learned_addr learned my addr 172.21.15.120:0/3379313552 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.970+0000 7fb982d9d640 1 --2- 172.21.15.120:0/3379313552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5620aaf0ab40 0x5620aaf09090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.970+0000 7fb981d9b640 1 --2- 172.21.15.120:0/3379313552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620aaf14370 0x5620aaf16760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.970+0000 7fb97b7fe640 1 --2- 172.21.15.120:0/4234846212 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb9700a3240 0x7fb9700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.970+0000 7fb982d9d640 1 --2- 172.21.15.120:0/4234846212 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb9700a3240 0x7fb9700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.970+0000 7fb98259c640 1 -- 172.21.15.120:0/3379313552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5620aaf0ab40 msgr2=0x5620aaf09090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:31.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.970+0000 7fb98259c640 1 --2- 172.21.15.120:0/3379313552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5620aaf0ab40 0x5620aaf09090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.971+0000 7fb98259c640 1 -- 172.21.15.120:0/3379313552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620aaf14370 msgr2=0x5620aaf16760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.971+0000 7fb98259c640 1 --2- 172.21.15.120:0/3379313552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620aaf14370 0x5620aaf16760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.971+0000 7fb98259c640 1 -- 172.21.15.120:0/3379313552 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620aadfb3a0 con 0x5620aaf095d0 2024-09-19T09:07:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.971+0000 7fb981d9b640 1 --2- 172.21.15.120:0/3379313552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620aaf14370 0x5620aaf16760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:07:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.971+0000 7fb97b7fe640 1 -- 172.21.15.120:0/4234846212 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fb9740cfdf0 con 0x7fb984100480 2024-09-19T09:07:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.971+0000 7fb98259c640 1 --2- 172.21.15.120:0/3379313552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620aaf095d0 0x5620aaf09a10 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb9740cd8f0 tx=0x7fb97409c080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11275c3019c092 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.971+0000 7fb982d9d640 1 --2- 172.21.15.120:0/4234846212 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb9700a3240 0x7fb9700a5700 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb97c009fd0 tx=0x7fb97c004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.971+0000 7fb978ff9640 1 -- 172.21.15.120:0/3379313552 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9740490d0 con 0x5620aaf095d0 2024-09-19T09:07:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.971+0000 7fb978ff9640 1 -- 172.21.15.120:0/3379313552 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb97409cb10 con 0x5620aaf095d0 2024-09-19T09:07:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.971+0000 7fb978ff9640 1 -- 172.21.15.120:0/3379313552 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb974098a80 con 0x5620aaf095d0 2024-09-19T09:07:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.972+0000 7fb98ad38740 1 -- 172.21.15.120:0/3379313552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620aaf095d0 msgr2=0x5620aaf09a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:31.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.972+0000 7fb98ad38740 1 --2- 172.21.15.120:0/3379313552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620aaf095d0 0x5620aaf09a10 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb9740cd8f0 tx=0x7fb97409c080 comp rx=0 tx=0).stop 2024-09-19T09:07:31.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.972+0000 7fb98ad38740 1 -- 172.21.15.120:0/3379313552 shutdown_connections 2024-09-19T09:07:31.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.972+0000 7fb98ad38740 1 --2- 172.21.15.120:0/3379313552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620aaf14370 0x5620aaf16760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:31.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.972+0000 7fb98ad38740 1 --2- 172.21.15.120:0/3379313552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620aaf095d0 0x5620aaf09a10 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:31.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.972+0000 7fb98ad38740 1 --2- 172.21.15.120:0/3379313552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5620aaf0ab40 0x5620aaf09090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:31.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.972+0000 7fb98ad38740 1 -- 172.21.15.120:0/3379313552 >> 172.21.15.120:0/3379313552 conn(0x5620aaed2310 msgr2=0x5620aaf05c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:31.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.972+0000 7fb98ad38740 1 -- 172.21.15.120:0/3379313552 shutdown_connections 2024-09-19T09:07:31.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.972+0000 7fb98ad38740 1 -- 172.21.15.120:0/3379313552 wait complete. 2024-09-19T09:07:31.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.972+0000 7fb98ad38740 1 -- start start 2024-09-19T09:07:31.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.972+0000 7fb98ad38740 1 Processor -- start 2024-09-19T09:07:31.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.973+0000 7fb98ad38740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620aaf095d0 0x5620aaf13d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:31.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.973+0000 7fb98ad38740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5620aaf0ab40 0x5620aaf10ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:31.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.973+0000 7fb98ad38740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620aaf14370 0x5620aaf115e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:31.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.973+0000 7fb98ad38740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5620aaf16fb0 con 0x5620aaf095d0 2024-09-19T09:07:31.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.973+0000 7fb98ad38740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5620aae2a060 con 0x5620aaf14370 2024-09-19T09:07:31.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.973+0000 7fb98ad38740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5620aaf17130 con 0x5620aaf0ab40 2024-09-19T09:07:31.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.973+0000 7fb98ad38740 10 client.0 _ll_register_callbacks cb 0x5620aab7d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:31.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.973+0000 7fb98259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620aaf095d0 0x5620aaf13d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:31.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.973+0000 7fb98259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620aaf095d0 0x5620aaf13d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56462/0 (socket says 172.21.15.120:56462) 2024-09-19T09:07:31.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.973+0000 7fb98259c640 1 -- 172.21.15.120:0/2603092917 learned_addr learned my addr 172.21.15.120:0/2603092917 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:31.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.973+0000 7fb982d9d640 1 --2- 172.21.15.120:0/2603092917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620aaf14370 0x5620aaf115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:31.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.973+0000 7fb981d9b640 1 --2- 172.21.15.120:0/2603092917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5620aaf0ab40 0x5620aaf10ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:31.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.973+0000 7fb98259c640 1 -- 172.21.15.120:0/2603092917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5620aaf0ab40 msgr2=0x5620aaf10ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:31.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.973+0000 7fb98259c640 1 --2- 172.21.15.120:0/2603092917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5620aaf0ab40 0x5620aaf10ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:31.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.973+0000 7fb98259c640 1 -- 172.21.15.120:0/2603092917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620aaf14370 msgr2=0x5620aaf115e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:31.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.973+0000 7fb98259c640 1 --2- 172.21.15.120:0/2603092917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620aaf14370 0x5620aaf115e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:31.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.973+0000 7fb98259c640 1 -- 172.21.15.120:0/2603092917 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620aabe03c0 con 0x5620aaf095d0 2024-09-19T09:07:31.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.973+0000 7fb981d9b640 1 --2- 172.21.15.120:0/2603092917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5620aaf0ab40 0x5620aaf10ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:07:31.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.973+0000 7fb98259c640 1 --2- 172.21.15.120:0/2603092917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620aaf095d0 0x5620aaf13d40 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fb9740cdbe0 tx=0x7fb974004ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6fb582be2c97e23 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:31.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.973+0000 7fb969ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:07:31.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.973+0000 7fb969ffb640 1 -- 172.21.15.120:0/2603092917 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9740b5060 con 0x5620aaf095d0 2024-09-19T09:07:31.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.973+0000 7fb969ffb640 1 -- 172.21.15.120:0/2603092917 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fb9740a4bf0 con 0x5620aaf095d0 2024-09-19T09:07:31.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.973+0000 7fb98ad38740 1 -- 172.21.15.120:0/2603092917 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb97409cb10 con 0x5620aaf095d0 2024-09-19T09:07:31.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.974+0000 7fb969ffb640 1 -- 172.21.15.120:0/2603092917 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb9740b5400 con 0x5620aaf095d0 2024-09-19T09:07:31.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.974+0000 7fb98ad38740 10 client.17064 fetch_fsmap learned FSMap version 30 2024-09-19T09:07:31.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.974+0000 7fb98ad38740 1 -- 172.21.15.120:0/2603092917 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5620aaf22a20 con 0x5620aaf095d0 2024-09-19T09:07:31.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.974+0000 7fb969ffb640 1 -- 172.21.15.120:0/2603092917 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb97409cb10 con 0x5620aaf095d0 2024-09-19T09:07:31.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.974+0000 7fb969ffb640 1 -- 172.21.15.120:0/2603092917 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb9740d3070 con 0x5620aaf095d0 2024-09-19T09:07:31.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.974+0000 7fb98ad38740 10 client.17064 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:07:31.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.974+0000 7fb98ad38740 10 client.17064 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:07:31.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.974+0000 7fb98ad38740 20 client.17064 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:31.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.974+0000 7fb98ad38740 1 --2- 172.21.15.120:0/2603092917 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5620aaf23d70 0x5620aaf26160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:31.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.974+0000 7fb98ad38740 4 client.17064 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:07:31.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.974+0000 7fb98ad38740 1 -- 172.21.15.120:0/2603092917 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620aae7c860 con 0x5620aaf23d70 2024-09-19T09:07:31.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.974+0000 7fb981d9b640 1 --2- 172.21.15.120:0/2603092917 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5620aaf23d70 0x5620aaf26160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:31.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.975+0000 7fb969ffb640 10 client.17064 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:07:31.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.975+0000 7fb981d9b640 1 --2- 172.21.15.120:0/2603092917 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5620aaf23d70 0x5620aaf26160 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1bd5946c56b0cb11 server_cookie=1052f9a52dbbeb17 in_seq=0 out_seq=0 2024-09-19T09:07:31.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.975+0000 7fb969ffb640 1 -- 172.21.15.120:0/2603092917 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5620aae7c860 con 0x5620aaf23d70 2024-09-19T09:07:31.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.975+0000 7fb969ffb640 10 client.17064 handle_command_reply: tid=0 2024-09-19T09:07:31.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.975+0000 7fb98ad38740 1 client.17064 shutdown 2024-09-19T09:07:31.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.975+0000 7fb98ad38740 1 -- 172.21.15.120:0/2603092917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620aaf095d0 msgr2=0x5620aaf13d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:31.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.975+0000 7fb98ad38740 1 --2- 172.21.15.120:0/2603092917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620aaf095d0 0x5620aaf13d40 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fb9740cdbe0 tx=0x7fb974004ba0 comp rx=0 tx=0).stop 2024-09-19T09:07:31.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.975+0000 7fb98ad38740 1 -- 172.21.15.120:0/2603092917 shutdown_connections 2024-09-19T09:07:31.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.975+0000 7fb98ad38740 1 --2- 172.21.15.120:0/2603092917 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5620aaf23d70 0x5620aaf26160 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:31.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.975+0000 7fb98ad38740 1 --2- 172.21.15.120:0/2603092917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620aaf14370 0x5620aaf115e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:31.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.975+0000 7fb969ffb640 0 client.17064 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:07:31.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.975+0000 7fb98ad38740 1 --2- 172.21.15.120:0/2603092917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5620aaf0ab40 0x5620aaf10ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:31.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.975+0000 7fb98ad38740 1 --2- 172.21.15.120:0/2603092917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620aaf095d0 0x5620aaf13d40 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:31.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.975+0000 7fb98ad38740 1 -- 172.21.15.120:0/2603092917 >> 172.21.15.120:0/2603092917 conn(0x5620aaed2310 msgr2=0x5620aaf0b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:31.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.976+0000 7fb98ad38740 1 -- 172.21.15.120:0/2603092917 shutdown_connections 2024-09-19T09:07:31.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.976+0000 7fb98ad38740 1 -- 172.21.15.120:0/2603092917 wait complete. 2024-09-19T09:07:31.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.976+0000 7fb98ad38740 20 client.17064 trim_cache size 0 max 16384 2024-09-19T09:07:31.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.991+0000 7fb98ad38740 1 Processor -- start 2024-09-19T09:07:31.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.991+0000 7fb98ad38740 1 -- start start 2024-09-19T09:07:31.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.991+0000 7fb98ad38740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620aaf095d0 0x5620aaf0c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:31.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.991+0000 7fb98ad38740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620aaf0ab40 0x5620aaf0c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:31.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.991+0000 7fb98ad38740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5620aaf14370 0x5620aaf0e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:31.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.991+0000 7fb98ad38740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb9841ad690 con 0x5620aaf0ab40 2024-09-19T09:07:31.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.991+0000 7fb98ad38740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb9841af1d0 con 0x5620aaf095d0 2024-09-19T09:07:31.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.991+0000 7fb98ad38740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb9841add60 con 0x5620aaf14370 2024-09-19T09:07:31.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.991+0000 7fb981d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620aaf0ab40 0x5620aaf0c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:31.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.991+0000 7fb981d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620aaf0ab40 0x5620aaf0c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56474/0 (socket says 172.21.15.120:56474) 2024-09-19T09:07:32.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.991+0000 7fb981d9b640 1 -- 172.21.15.120:0/2022361257 learned_addr learned my addr 172.21.15.120:0/2022361257 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:32.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.991+0000 7fb982d9d640 1 --2- 172.21.15.120:0/2022361257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5620aaf14370 0x5620aaf0e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:32.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.991+0000 7fb98259c640 1 --2- 172.21.15.120:0/2022361257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620aaf095d0 0x5620aaf0c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:32.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.992+0000 7fb981d9b640 1 -- 172.21.15.120:0/2022361257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5620aaf14370 msgr2=0x5620aaf0e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:32.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.992+0000 7fb981d9b640 1 --2- 172.21.15.120:0/2022361257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5620aaf14370 0x5620aaf0e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:32.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.992+0000 7fb981d9b640 1 -- 172.21.15.120:0/2022361257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620aaf095d0 msgr2=0x5620aaf0c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:32.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.992+0000 7fb981d9b640 1 --2- 172.21.15.120:0/2022361257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620aaf095d0 0x5620aaf0c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:32.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.992+0000 7fb981d9b640 1 -- 172.21.15.120:0/2022361257 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620aaf13960 con 0x5620aaf0ab40 2024-09-19T09:07:32.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.992+0000 7fb98259c640 1 --2- 172.21.15.120:0/2022361257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620aaf095d0 0x5620aaf0c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:07:32.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.992+0000 7fb981d9b640 1 --2- 172.21.15.120:0/2022361257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620aaf0ab40 0x5620aaf0c660 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009230 tx=0x7fb96c0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d68767204cecdd70 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:32.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.992+0000 7fb96bfff640 1 -- 172.21.15.120:0/2022361257 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c04a070 con 0x5620aaf0ab40 2024-09-19T09:07:32.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.992+0000 7fb96bfff640 1 -- 172.21.15.120:0/2022361257 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb96c005500 con 0x5620aaf0ab40 2024-09-19T09:07:32.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.992+0000 7fb96bfff640 1 -- 172.21.15.120:0/2022361257 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c035040 con 0x5620aaf0ab40 2024-09-19T09:07:32.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.993+0000 7fb98ad38740 1 -- 172.21.15.120:0/2022361257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620aaf0ab40 msgr2=0x5620aaf0c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:32.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.993+0000 7fb98ad38740 1 --2- 172.21.15.120:0/2022361257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620aaf0ab40 0x5620aaf0c660 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009230 tx=0x7fb96c0058d0 comp rx=0 tx=0).stop 2024-09-19T09:07:32.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.993+0000 7fb98ad38740 1 -- 172.21.15.120:0/2022361257 shutdown_connections 2024-09-19T09:07:32.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.993+0000 7fb98ad38740 1 --2- 172.21.15.120:0/2022361257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5620aaf14370 0x5620aaf0e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:32.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.993+0000 7fb98ad38740 1 --2- 172.21.15.120:0/2022361257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620aaf0ab40 0x5620aaf0c660 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:32.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.993+0000 7fb98ad38740 1 --2- 172.21.15.120:0/2022361257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620aaf095d0 0x5620aaf0c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:32.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.993+0000 7fb98ad38740 1 -- 172.21.15.120:0/2022361257 >> 172.21.15.120:0/2022361257 conn(0x5620aaed2310 msgr2=0x5620aaf13070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:32.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.993+0000 7fb98ad38740 1 -- 172.21.15.120:0/2022361257 shutdown_connections 2024-09-19T09:07:32.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.993+0000 7fb98ad38740 1 -- 172.21.15.120:0/2022361257 wait complete. 2024-09-19T09:07:32.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.993+0000 7fb98ad38740 1 -- start start 2024-09-19T09:07:32.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.993+0000 7fb98ad38740 1 Processor -- start 2024-09-19T09:07:32.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.994+0000 7fb98ad38740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620aaf095d0 0x5620aaf27800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:32.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.994+0000 7fb98ad38740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620aaf0ab40 0x5620aaf27d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:32.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.994+0000 7fb98ad38740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5620aaf14370 0x5620aaf8bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:32.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.994+0000 7fb98ad38740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb9841aeb00 con 0x5620aaf0ab40 2024-09-19T09:07:32.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.994+0000 7fb98ad38740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb9841ae430 con 0x5620aaf095d0 2024-09-19T09:07:32.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.994+0000 7fb98ad38740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb9841ab650 con 0x5620aaf14370 2024-09-19T09:07:32.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.994+0000 7fb98ad38740 10 client.0 _ll_register_callbacks cb 0x7fb974005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:32.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.994+0000 7fb981d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620aaf0ab40 0x5620aaf27d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:32.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.994+0000 7fb98259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620aaf095d0 0x5620aaf27800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:32.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.994+0000 7fb981d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620aaf0ab40 0x5620aaf27d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56478/0 (socket says 172.21.15.120:56478) 2024-09-19T09:07:32.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.994+0000 7fb981d9b640 1 -- 172.21.15.120:0/3509383799 learned_addr learned my addr 172.21.15.120:0/3509383799 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:32.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.994+0000 7fb982d9d640 1 --2- 172.21.15.120:0/3509383799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5620aaf14370 0x5620aaf8bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:32.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.994+0000 7fb98259c640 1 -- 172.21.15.120:0/3509383799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5620aaf14370 msgr2=0x5620aaf8bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:32.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.994+0000 7fb98259c640 1 --2- 172.21.15.120:0/3509383799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5620aaf14370 0x5620aaf8bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:32.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.994+0000 7fb98259c640 1 -- 172.21.15.120:0/3509383799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620aaf0ab40 msgr2=0x5620aaf27d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:32.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.994+0000 7fb98259c640 1 --2- 172.21.15.120:0/3509383799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620aaf0ab40 0x5620aaf27d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:32.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.994+0000 7fb98259c640 1 -- 172.21.15.120:0/3509383799 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620aaf8c1b0 con 0x5620aaf095d0 2024-09-19T09:07:32.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.994+0000 7fb981d9b640 1 --2- 172.21.15.120:0/3509383799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620aaf0ab40 0x5620aaf27d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:07:32.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.994+0000 7fb98259c640 1 --2- 172.21.15.120:0/3509383799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620aaf095d0 0x5620aaf27800 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb9740cdac0 tx=0x7fb974098080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f64736a1a194e8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:32.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.994+0000 7fb969ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:07:32.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.994+0000 7fb969ffb640 1 -- 172.21.15.120:0/3509383799 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9740986b0 con 0x5620aaf095d0 2024-09-19T09:07:32.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.994+0000 7fb98ad38740 1 -- 172.21.15.120:0/3509383799 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb96c005500 con 0x5620aaf095d0 2024-09-19T09:07:32.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.994+0000 7fb969ffb640 1 -- 172.21.15.120:0/3509383799 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fb9740a4df0 con 0x5620aaf095d0 2024-09-19T09:07:32.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.995+0000 7fb969ffb640 1 -- 172.21.15.120:0/3509383799 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb97409bf50 con 0x5620aaf095d0 2024-09-19T09:07:32.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.995+0000 7fb98ad38740 10 client.26393 fetch_fsmap learned FSMap version 30 2024-09-19T09:07:32.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.995+0000 7fb98ad38740 1 -- 172.21.15.120:0/3509383799 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5620aaf93540 con 0x5620aaf095d0 2024-09-19T09:07:32.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.995+0000 7fb969ffb640 1 -- 172.21.15.120:0/3509383799 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb96c005500 con 0x5620aaf095d0 2024-09-19T09:07:32.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.995+0000 7fb969ffb640 1 -- 172.21.15.120:0/3509383799 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb9740a1950 con 0x5620aaf095d0 2024-09-19T09:07:32.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.995+0000 7fb98ad38740 10 client.26393 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:07:32.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.995+0000 7fb98ad38740 10 client.26393 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:07:32.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.995+0000 7fb98ad38740 20 client.26393 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:32.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.995+0000 7fb98ad38740 1 --2- 172.21.15.120:0/3509383799 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5620aaf93a70 0x5620aaf95e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:32.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.995+0000 7fb98ad38740 4 client.26393 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:07:32.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.995+0000 7fb98ad38740 1 -- 172.21.15.120:0/3509383799 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5620aae76d70 con 0x5620aaf93a70 2024-09-19T09:07:32.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.995+0000 7fb981d9b640 1 --2- 172.21.15.120:0/3509383799 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5620aaf93a70 0x5620aaf95e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:32.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.996+0000 7fb981d9b640 1 --2- 172.21.15.120:0/3509383799 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5620aaf93a70 0x5620aaf95e60 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9b789fde2aaf7cd6 server_cookie=a1f80050b9dbfc19 in_seq=0 out_seq=0 2024-09-19T09:07:32.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.996+0000 7fb969ffb640 10 client.26393 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:07:32.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.996+0000 7fb969ffb640 1 -- 172.21.15.120:0/3509383799 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5620aae76d70 con 0x5620aaf93a70 2024-09-19T09:07:32.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.996+0000 7fb969ffb640 10 client.26393 handle_command_reply: tid=0 2024-09-19T09:07:32.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.996+0000 7fb98ad38740 1 client.26393 shutdown 2024-09-19T09:07:32.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.996+0000 7fb98ad38740 1 -- 172.21.15.120:0/3509383799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620aaf095d0 msgr2=0x5620aaf27800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:32.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.996+0000 7fb98ad38740 1 --2- 172.21.15.120:0/3509383799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620aaf095d0 0x5620aaf27800 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb9740cdac0 tx=0x7fb974098080 comp rx=0 tx=0).stop 2024-09-19T09:07:32.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.996+0000 7fb98ad38740 1 -- 172.21.15.120:0/3509383799 shutdown_connections 2024-09-19T09:07:32.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.996+0000 7fb98ad38740 1 --2- 172.21.15.120:0/3509383799 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5620aaf93a70 0x5620aaf95e60 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:32.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.996+0000 7fb98ad38740 1 --2- 172.21.15.120:0/3509383799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5620aaf14370 0x5620aaf8bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:32.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.996+0000 7fb969ffb640 0 client.26393 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:07:32.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.996+0000 7fb98ad38740 1 --2- 172.21.15.120:0/3509383799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620aaf0ab40 0x5620aaf27d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:32.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.996+0000 7fb98ad38740 1 --2- 172.21.15.120:0/3509383799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620aaf095d0 0x5620aaf27800 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:32.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.996+0000 7fb98ad38740 1 -- 172.21.15.120:0/3509383799 >> 172.21.15.120:0/3509383799 conn(0x5620aaed2310 msgr2=0x5620aaf044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:32.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.996+0000 7fb98ad38740 1 -- 172.21.15.120:0/3509383799 shutdown_connections 2024-09-19T09:07:32.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.996+0000 7fb98ad38740 1 -- 172.21.15.120:0/3509383799 wait complete. 2024-09-19T09:07:32.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.996+0000 7fb98ad38740 20 client.26393 trim_cache size 0 max 16384 2024-09-19T09:07:32.005 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:07:32.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.997+0000 7fb98895f640 1 -- 172.21.15.120:0/4234846212 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb9700a3240 msgr2=0x7fb9700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:32.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.997+0000 7fb98895f640 1 --2- 172.21.15.120:0/4234846212 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb9700a3240 0x7fb9700a5700 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb97c009fd0 tx=0x7fb97c004ab0 comp rx=0 tx=0).stop 2024-09-19T09:07:32.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.997+0000 7fb98895f640 1 -- 172.21.15.120:0/4234846212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb984100480 msgr2=0x7fb984198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:32.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.997+0000 7fb98895f640 1 --2- 172.21.15.120:0/4234846212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb984100480 0x7fb984198450 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fb97400e9f0 tx=0x7fb97400edf0 comp rx=0 tx=0).stop 2024-09-19T09:07:32.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.997+0000 7fb98895f640 1 -- 172.21.15.120:0/4234846212 shutdown_connections 2024-09-19T09:07:32.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.997+0000 7fb98895f640 1 --2- 172.21.15.120:0/4234846212 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb9700a3240 0x7fb9700a5700 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:32.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.997+0000 7fb98895f640 1 --2- 172.21.15.120:0/4234846212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9841055b0 0x7fb98419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:32.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.997+0000 7fb98895f640 1 --2- 172.21.15.120:0/4234846212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb984100e30 0x7fb984198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:32.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.997+0000 7fb98895f640 1 --2- 172.21.15.120:0/4234846212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb984100480 0x7fb984198450 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:32.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.997+0000 7fb98895f640 1 -- 172.21.15.120:0/4234846212 >> 172.21.15.120:0/4234846212 conn(0x7fb984077210 msgr2=0x7fb984103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:32.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.997+0000 7fb98895f640 1 -- 172.21.15.120:0/4234846212 shutdown_connections 2024-09-19T09:07:32.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:31.997+0000 7fb98895f640 1 -- 172.21.15.120:0/4234846212 wait complete. 2024-09-19T09:07:32.046 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 8.19 MB/sec execute 87 sec latency 24085.250 ms 2024-09-19T09:07:32.802 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:07:33.046 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 8.10 MB/sec execute 88 sec latency 25085.287 ms 2024-09-19T09:07:33.050 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:07:33.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:32 smithi120 ceph-mon[31150]: pgmap v482: 97 pgs: 97 active+clean; 35 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 2.4 MiB/s wr, 124 op/s 2024-09-19T09:07:33.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:32 smithi196 ceph-mon[35150]: pgmap v482: 97 pgs: 97 active+clean; 35 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 2.4 MiB/s wr, 124 op/s 2024-09-19T09:07:33.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:32 smithi153 ceph-mon[35071]: pgmap v482: 97 pgs: 97 active+clean; 35 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 2.4 MiB/s wr, 124 op/s 2024-09-19T09:07:33.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.499+0000 7fc12a511640 1 -- 172.21.15.120:0/3800218299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc124106f10 msgr2=0x7fc12410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:33.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.499+0000 7fc12a511640 1 --2- 172.21.15.120:0/3800218299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc124106f10 0x7fc12410ea90 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc11c009ca0 tx=0x7fc11c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.499+0000 7fc12a511640 1 -- 172.21.15.120:0/3800218299 shutdown_connections 2024-09-19T09:07:33.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.499+0000 7fc12a511640 1 --2- 172.21.15.120:0/3800218299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc124106f10 0x7fc12410ea90 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.499+0000 7fc12a511640 1 --2- 172.21.15.120:0/3800218299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc124102790 0x7fc1241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.499+0000 7fc12a511640 1 --2- 172.21.15.120:0/3800218299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc124101de0 0x7fc1241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.499+0000 7fc12a511640 1 -- 172.21.15.120:0/3800218299 >> 172.21.15.120:0/3800218299 conn(0x7fc1240fd8c0 msgr2=0x7fc1240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:33.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.499+0000 7fc12a511640 1 -- 172.21.15.120:0/3800218299 shutdown_connections 2024-09-19T09:07:33.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.499+0000 7fc12a511640 1 -- 172.21.15.120:0/3800218299 wait complete. 2024-09-19T09:07:33.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.500+0000 7fc12a511640 1 Processor -- start 2024-09-19T09:07:33.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.500+0000 7fc12a511640 1 -- start start 2024-09-19T09:07:33.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.500+0000 7fc12a511640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc124101de0 0x7fc1240744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:33.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.500+0000 7fc12a511640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc124102790 0x7fc124074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:33.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.500+0000 7fc12a511640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc124106f10 0x7fc124074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:33.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.500+0000 7fc12a511640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc124111f70 con 0x7fc124106f10 2024-09-19T09:07:33.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.500+0000 7fc12a511640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc124111df0 con 0x7fc124101de0 2024-09-19T09:07:33.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.500+0000 7fc12a511640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc1241120f0 con 0x7fc124102790 2024-09-19T09:07:33.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.500+0000 7fc128aae640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc124106f10 0x7fc124074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:33.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.500+0000 7fc128aae640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc124106f10 0x7fc124074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56498/0 (socket says 172.21.15.120:56498) 2024-09-19T09:07:33.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.500+0000 7fc128aae640 1 -- 172.21.15.120:0/1966445565 learned_addr learned my addr 172.21.15.120:0/1966445565 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:33.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.500+0000 7fc123fff640 1 --2- 172.21.15.120:0/1966445565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc124101de0 0x7fc1240744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:33.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.500+0000 7fc11bfff640 1 --2- 172.21.15.120:0/1966445565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc124102790 0x7fc124074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:33.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.501+0000 7fc128aae640 1 -- 172.21.15.120:0/1966445565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc124102790 msgr2=0x7fc124074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:33.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.501+0000 7fc128aae640 1 --2- 172.21.15.120:0/1966445565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc124102790 0x7fc124074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.501+0000 7fc128aae640 1 -- 172.21.15.120:0/1966445565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc124101de0 msgr2=0x7fc1240744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:33.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.501+0000 7fc128aae640 1 --2- 172.21.15.120:0/1966445565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc124101de0 0x7fc1240744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.501+0000 7fc128aae640 1 -- 172.21.15.120:0/1966445565 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1241a68b0 con 0x7fc124106f10 2024-09-19T09:07:33.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.501+0000 7fc128aae640 1 --2- 172.21.15.120:0/1966445565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc124106f10 0x7fc124074f70 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc11c031a20 tx=0x7fc11c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:33.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.501+0000 7fc121ffb640 1 -- 172.21.15.120:0/1966445565 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c003c30 con 0x7fc124106f10 2024-09-19T09:07:33.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.501+0000 7fc121ffb640 1 -- 172.21.15.120:0/1966445565 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc11c003dd0 con 0x7fc124106f10 2024-09-19T09:07:33.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.501+0000 7fc12a511640 1 -- 172.21.15.120:0/1966445565 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1241a6b40 con 0x7fc124106f10 2024-09-19T09:07:33.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.501+0000 7fc121ffb640 1 -- 172.21.15.120:0/1966445565 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c02fcf0 con 0x7fc124106f10 2024-09-19T09:07:33.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.501+0000 7fc12a511640 1 -- 172.21.15.120:0/1966445565 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1241a7080 con 0x7fc124106f10 2024-09-19T09:07:33.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.503+0000 7fc121ffb640 1 -- 172.21.15.120:0/1966445565 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fc11c004050 con 0x7fc124106f10 2024-09-19T09:07:33.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.504+0000 7fc12c8ea740 1 Processor -- start 2024-09-19T09:07:33.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.504+0000 7fc12c8ea740 1 -- start start 2024-09-19T09:07:33.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.504+0000 7fc12c8ea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4e6d72b40 0x55e4e6d71090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:33.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.504+0000 7fc12c8ea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4e6d715d0 0x55e4e6d71a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:33.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.504+0000 7fc12c8ea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4e6d7c370 0x55e4e6d7e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:33.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.504+0000 7fc121ffb640 1 --2- 172.21.15.120:0/1966445565 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc1100a3240 0x7fc1100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:33.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.504+0000 7fc12c8ea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e4e6cd72f0 con 0x55e4e6d72b40 2024-09-19T09:07:33.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.504+0000 7fc12c8ea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e4e6c7ecf0 con 0x55e4e6d7c370 2024-09-19T09:07:33.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.504+0000 7fc12c8ea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e4e6cd32b0 con 0x55e4e6d715d0 2024-09-19T09:07:33.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.504+0000 7fc11bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4e6d72b40 0x55e4e6d71090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:33.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.504+0000 7fc11bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4e6d72b40 0x55e4e6d71090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56510/0 (socket says 172.21.15.120:56510) 2024-09-19T09:07:33.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.504+0000 7fc11bfff640 1 -- 172.21.15.120:0/3941172175 learned_addr learned my addr 172.21.15.120:0/3941172175 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:33.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.504+0000 7fc128aae640 1 --2- 172.21.15.120:0/1966445565 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc1100a3240 0x7fc1100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:33.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.504+0000 7fc11bfff640 1 --2- 172.21.15.120:0/3941172175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4e6d7c370 0x55e4e6d7e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:33.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.504+0000 7fc123fff640 1 --2- 172.21.15.120:0/3941172175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4e6d715d0 0x55e4e6d71a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:33.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.504+0000 7fc11bfff640 1 -- 172.21.15.120:0/3941172175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4e6d715d0 msgr2=0x55e4e6d71a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:33.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.504+0000 7fc11bfff640 1 --2- 172.21.15.120:0/3941172175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4e6d715d0 0x55e4e6d71a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.504+0000 7fc121ffb640 1 -- 172.21.15.120:0/1966445565 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fc11c0f0ca0 con 0x7fc124106f10 2024-09-19T09:07:33.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.504+0000 7fc11bfff640 1 -- 172.21.15.120:0/3941172175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4e6d7c370 msgr2=0x55e4e6d7e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:33.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.504+0000 7fc11bfff640 1 --2- 172.21.15.120:0/3941172175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4e6d7c370 0x55e4e6d7e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.504+0000 7fc11bfff640 1 -- 172.21.15.120:0/3941172175 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4e6c633a0 con 0x55e4e6d72b40 2024-09-19T09:07:33.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.505+0000 7fc128aae640 1 --2- 172.21.15.120:0/1966445565 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc1100a3240 0x7fc1100a5700 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc11c0b8ff0 tx=0x7fc11c0c1040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:33.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.505+0000 7fc11bfff640 1 --2- 172.21.15.120:0/3941172175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4e6d72b40 0x55e4e6d71090 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc10c005fd0 tx=0x7fc10c00b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eebed658aadd061c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:33.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.505+0000 7fc11a7fc640 1 -- 172.21.15.120:0/3941172175 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c019030 con 0x55e4e6d72b40 2024-09-19T09:07:33.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.505+0000 7fc11a7fc640 1 -- 172.21.15.120:0/3941172175 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc10c005500 con 0x55e4e6d72b40 2024-09-19T09:07:33.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.505+0000 7fc11a7fc640 1 -- 172.21.15.120:0/3941172175 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c011440 con 0x55e4e6d72b40 2024-09-19T09:07:33.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.505+0000 7fc12c8ea740 1 -- 172.21.15.120:0/3941172175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4e6d72b40 msgr2=0x55e4e6d71090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:33.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.505+0000 7fc12c8ea740 1 --2- 172.21.15.120:0/3941172175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4e6d72b40 0x55e4e6d71090 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc10c005fd0 tx=0x7fc10c00b610 comp rx=0 tx=0).stop 2024-09-19T09:07:33.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.506+0000 7fc12c8ea740 1 -- 172.21.15.120:0/3941172175 shutdown_connections 2024-09-19T09:07:33.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.506+0000 7fc12c8ea740 1 --2- 172.21.15.120:0/3941172175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4e6d7c370 0x55e4e6d7e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.506+0000 7fc12c8ea740 1 --2- 172.21.15.120:0/3941172175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4e6d715d0 0x55e4e6d71a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.506+0000 7fc12c8ea740 1 --2- 172.21.15.120:0/3941172175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4e6d72b40 0x55e4e6d71090 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.506+0000 7fc12c8ea740 1 -- 172.21.15.120:0/3941172175 >> 172.21.15.120:0/3941172175 conn(0x55e4e6d3a310 msgr2=0x55e4e6d6dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:33.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.506+0000 7fc12c8ea740 1 -- 172.21.15.120:0/3941172175 shutdown_connections 2024-09-19T09:07:33.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.506+0000 7fc12c8ea740 1 -- 172.21.15.120:0/3941172175 wait complete. 2024-09-19T09:07:33.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.506+0000 7fc12c8ea740 1 -- start start 2024-09-19T09:07:33.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.506+0000 7fc12c8ea740 1 Processor -- start 2024-09-19T09:07:33.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.506+0000 7fc12c8ea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4e6d715d0 0x55e4e6d7bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:33.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.506+0000 7fc12c8ea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4e6d72b40 0x55e4e6d78f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:33.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.506+0000 7fc12c8ea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4e6d7c370 0x55e4e6d79480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:33.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.506+0000 7fc12c8ea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e4e6d7f070 con 0x55e4e6d7c370 2024-09-19T09:07:33.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.506+0000 7fc12c8ea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e4e6c92060 con 0x55e4e6d715d0 2024-09-19T09:07:33.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.506+0000 7fc12c8ea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e4e6d7f1f0 con 0x55e4e6d72b40 2024-09-19T09:07:33.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.506+0000 7fc12c8ea740 10 client.0 _ll_register_callbacks cb 0x55e4e69e58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:33.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.507+0000 7fc11bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4e6d7c370 0x55e4e6d79480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:33.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.507+0000 7fc11bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4e6d7c370 0x55e4e6d79480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56524/0 (socket says 172.21.15.120:56524) 2024-09-19T09:07:33.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.507+0000 7fc11bfff640 1 -- 172.21.15.120:0/3876887312 learned_addr learned my addr 172.21.15.120:0/3876887312 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:33.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.507+0000 7fc123fff640 1 --2- 172.21.15.120:0/3876887312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4e6d72b40 0x55e4e6d78f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:33.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.507+0000 7fc11bfff640 1 --2- 172.21.15.120:0/3876887312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4e6d715d0 0x55e4e6d7bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:33.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.507+0000 7fc11bfff640 1 -- 172.21.15.120:0/3876887312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4e6d72b40 msgr2=0x55e4e6d78f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:33.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.507+0000 7fc11bfff640 1 --2- 172.21.15.120:0/3876887312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4e6d72b40 0x55e4e6d78f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.507+0000 7fc11bfff640 1 -- 172.21.15.120:0/3876887312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4e6d715d0 msgr2=0x55e4e6d7bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:33.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.507+0000 7fc11bfff640 1 --2- 172.21.15.120:0/3876887312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4e6d715d0 0x55e4e6d7bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.507+0000 7fc11bfff640 1 -- 172.21.15.120:0/3876887312 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4e6a483c0 con 0x55e4e6d7c370 2024-09-19T09:07:33.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.507+0000 7fc11bfff640 1 --2- 172.21.15.120:0/3876887312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4e6d7c370 0x55e4e6d79480 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00bec0 tx=0x7fc10c004c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b52b110569e7ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:33.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.507+0000 7fc1037fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:07:33.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.507+0000 7fc1037fe640 1 -- 172.21.15.120:0/3876887312 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c019050 con 0x55e4e6d7c370 2024-09-19T09:07:33.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.507+0000 7fc1037fe640 1 -- 172.21.15.120:0/3876887312 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fc10c005000 con 0x55e4e6d7c370 2024-09-19T09:07:33.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.507+0000 7fc12c8ea740 1 -- 172.21.15.120:0/3876887312 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc10c005500 con 0x55e4e6d7c370 2024-09-19T09:07:33.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.507+0000 7fc1037fe640 1 -- 172.21.15.120:0/3876887312 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc10c035050 con 0x55e4e6d7c370 2024-09-19T09:07:33.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.507+0000 7fc12c8ea740 10 client.17082 fetch_fsmap learned FSMap version 30 2024-09-19T09:07:33.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.507+0000 7fc12c8ea740 1 -- 172.21.15.120:0/3876887312 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e4e6d8aa90 con 0x55e4e6d7c370 2024-09-19T09:07:33.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.508+0000 7fc1037fe640 1 -- 172.21.15.120:0/3876887312 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc10c005500 con 0x55e4e6d7c370 2024-09-19T09:07:33.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.508+0000 7fc1037fe640 1 -- 172.21.15.120:0/3876887312 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc10c01eba0 con 0x55e4e6d7c370 2024-09-19T09:07:33.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.508+0000 7fc12c8ea740 10 client.17082 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:07:33.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.508+0000 7fc12c8ea740 10 client.17082 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:07:33.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.508+0000 7fc12c8ea740 20 client.17082 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:33.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.508+0000 7fc12c8ea740 1 --2- 172.21.15.120:0/3876887312 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55e4e6d8bde0 0x55e4e6d8e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:33.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.508+0000 7fc12c8ea740 4 client.17082 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:07:33.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.508+0000 7fc12c8ea740 1 -- 172.21.15.120:0/3876887312 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4e6ce4860 con 0x55e4e6d8bde0 2024-09-19T09:07:33.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.508+0000 7fc123fff640 1 --2- 172.21.15.120:0/3876887312 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55e4e6d8bde0 0x55e4e6d8e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:33.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.508+0000 7fc1037fe640 10 client.17082 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:07:33.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.508+0000 7fc123fff640 1 --2- 172.21.15.120:0/3876887312 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55e4e6d8bde0 0x55e4e6d8e1d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=83c3819c21d89dcb server_cookie=30865c934836f236 in_seq=0 out_seq=0 2024-09-19T09:07:33.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.509+0000 7fc1037fe640 1 -- 172.21.15.120:0/3876887312 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e4e6ce4860 con 0x55e4e6d8bde0 2024-09-19T09:07:33.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.509+0000 7fc1037fe640 10 client.17082 handle_command_reply: tid=0 2024-09-19T09:07:33.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.509+0000 7fc12c8ea740 1 client.17082 shutdown 2024-09-19T09:07:33.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.509+0000 7fc12c8ea740 1 -- 172.21.15.120:0/3876887312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4e6d7c370 msgr2=0x55e4e6d79480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:33.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.509+0000 7fc12c8ea740 1 --2- 172.21.15.120:0/3876887312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4e6d7c370 0x55e4e6d79480 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00bec0 tx=0x7fc10c004c20 comp rx=0 tx=0).stop 2024-09-19T09:07:33.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.509+0000 7fc12c8ea740 1 -- 172.21.15.120:0/3876887312 shutdown_connections 2024-09-19T09:07:33.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.509+0000 7fc12c8ea740 1 --2- 172.21.15.120:0/3876887312 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55e4e6d8bde0 0x55e4e6d8e1d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.509+0000 7fc12c8ea740 1 --2- 172.21.15.120:0/3876887312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4e6d7c370 0x55e4e6d79480 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.509+0000 7fc12c8ea740 1 --2- 172.21.15.120:0/3876887312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4e6d72b40 0x55e4e6d78f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.509+0000 7fc1037fe640 0 client.17082 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:07:33.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.509+0000 7fc12c8ea740 1 --2- 172.21.15.120:0/3876887312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4e6d715d0 0x55e4e6d7bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.509+0000 7fc12c8ea740 1 -- 172.21.15.120:0/3876887312 >> 172.21.15.120:0/3876887312 conn(0x55e4e6d3a310 msgr2=0x55e4e6d73930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:33.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.509+0000 7fc12c8ea740 1 -- 172.21.15.120:0/3876887312 shutdown_connections 2024-09-19T09:07:33.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.509+0000 7fc12c8ea740 1 -- 172.21.15.120:0/3876887312 wait complete. 2024-09-19T09:07:33.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.510+0000 7fc12c8ea740 20 client.17082 trim_cache size 0 max 16384 2024-09-19T09:07:33.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.522+0000 7fc12c8ea740 1 Processor -- start 2024-09-19T09:07:33.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.522+0000 7fc12c8ea740 1 -- start start 2024-09-19T09:07:33.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.522+0000 7fc12c8ea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4e6d715d0 0x55e4e6d88370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:33.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.522+0000 7fc12c8ea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4e6d72b40 0x55e4e6d74120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:33.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.522+0000 7fc12c8ea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4e6d7c370 0x55e4e6d74660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:33.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.522+0000 7fc12c8ea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc1241af7c0 con 0x55e4e6d7c370 2024-09-19T09:07:33.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.522+0000 7fc12c8ea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc1241b1300 con 0x55e4e6d72b40 2024-09-19T09:07:33.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.522+0000 7fc12c8ea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc1241afe90 con 0x55e4e6d715d0 2024-09-19T09:07:33.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.522+0000 7fc11bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4e6d7c370 0x55e4e6d74660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:33.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.522+0000 7fc11bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4e6d7c370 0x55e4e6d74660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56530/0 (socket says 172.21.15.120:56530) 2024-09-19T09:07:33.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.522+0000 7fc11bfff640 1 -- 172.21.15.120:0/2594931320 learned_addr learned my addr 172.21.15.120:0/2594931320 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:33.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.523+0000 7fc11bfff640 1 --2- 172.21.15.120:0/2594931320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4e6d715d0 0x55e4e6d88370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:33.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.523+0000 7fc123fff640 1 --2- 172.21.15.120:0/2594931320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4e6d72b40 0x55e4e6d74120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:33.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.523+0000 7fc11bfff640 1 -- 172.21.15.120:0/2594931320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4e6d715d0 msgr2=0x55e4e6d88370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:33.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.523+0000 7fc11bfff640 1 --2- 172.21.15.120:0/2594931320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4e6d715d0 0x55e4e6d88370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.523+0000 7fc11bfff640 1 -- 172.21.15.120:0/2594931320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4e6d72b40 msgr2=0x55e4e6d74120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:33.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.523+0000 7fc11bfff640 1 --2- 172.21.15.120:0/2594931320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4e6d72b40 0x55e4e6d74120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.523+0000 7fc11bfff640 1 -- 172.21.15.120:0/2594931320 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4e6d71d50 con 0x55e4e6d7c370 2024-09-19T09:07:33.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.523+0000 7fc123fff640 1 --2- 172.21.15.120:0/2594931320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4e6d72b40 0x55e4e6d74120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:07:33.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.523+0000 7fc11bfff640 1 --2- 172.21.15.120:0/2594931320 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4e6d7c370 0x55e4e6d74660 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fc10c020040 tx=0x7fc10c015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611ccd03cf3db95e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:33.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.523+0000 7fc119ffb640 1 -- 172.21.15.120:0/2594931320 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c019070 con 0x55e4e6d7c370 2024-09-19T09:07:33.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.523+0000 7fc119ffb640 1 -- 172.21.15.120:0/2594931320 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc10c01a040 con 0x55e4e6d7c370 2024-09-19T09:07:33.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.523+0000 7fc119ffb640 1 -- 172.21.15.120:0/2594931320 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c000e60 con 0x55e4e6d7c370 2024-09-19T09:07:33.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.524+0000 7fc12c8ea740 1 -- 172.21.15.120:0/2594931320 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4e6d7c370 msgr2=0x55e4e6d74660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:33.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.524+0000 7fc12c8ea740 1 --2- 172.21.15.120:0/2594931320 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4e6d7c370 0x55e4e6d74660 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fc10c020040 tx=0x7fc10c015040 comp rx=0 tx=0).stop 2024-09-19T09:07:33.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.524+0000 7fc12c8ea740 1 -- 172.21.15.120:0/2594931320 shutdown_connections 2024-09-19T09:07:33.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.524+0000 7fc12c8ea740 1 --2- 172.21.15.120:0/2594931320 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4e6d7c370 0x55e4e6d74660 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.524+0000 7fc12c8ea740 1 --2- 172.21.15.120:0/2594931320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4e6d72b40 0x55e4e6d74120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.524+0000 7fc12c8ea740 1 --2- 172.21.15.120:0/2594931320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4e6d715d0 0x55e4e6d88370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.524+0000 7fc12c8ea740 1 -- 172.21.15.120:0/2594931320 >> 172.21.15.120:0/2594931320 conn(0x55e4e6d3a310 msgr2=0x55e4e6d78030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:33.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.524+0000 7fc12c8ea740 1 -- 172.21.15.120:0/2594931320 shutdown_connections 2024-09-19T09:07:33.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.524+0000 7fc12c8ea740 1 -- 172.21.15.120:0/2594931320 wait complete. 2024-09-19T09:07:33.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.524+0000 7fc12c8ea740 1 -- start start 2024-09-19T09:07:33.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.524+0000 7fc12c8ea740 1 Processor -- start 2024-09-19T09:07:33.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.524+0000 7fc12c8ea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4e6d715d0 0x55e4e6d84860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:33.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.524+0000 7fc12c8ea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4e6d72b40 0x55e4e6d84da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:33.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.524+0000 7fc12c8ea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4e6d7c370 0x55e4e6d7fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:33.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.525+0000 7fc12c8ea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc1241b0c30 con 0x55e4e6d7c370 2024-09-19T09:07:33.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.525+0000 7fc12c8ea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc1241b0560 con 0x55e4e6d72b40 2024-09-19T09:07:33.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.525+0000 7fc12c8ea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc1241ad780 con 0x55e4e6d715d0 2024-09-19T09:07:33.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.525+0000 7fc12c8ea740 10 client.0 _ll_register_callbacks cb 0x7fc10c013400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:33.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.525+0000 7fc11bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4e6d7c370 0x55e4e6d7fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:33.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.525+0000 7fc123fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4e6d72b40 0x55e4e6d84da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:33.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.525+0000 7fc11bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4e6d7c370 0x55e4e6d7fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56544/0 (socket says 172.21.15.120:56544) 2024-09-19T09:07:33.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.525+0000 7fc11bfff640 1 -- 172.21.15.120:0/1693297384 learned_addr learned my addr 172.21.15.120:0/1693297384 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:33.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.525+0000 7fc123fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4e6d72b40 0x55e4e6d84da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:42960/0 (socket says 172.21.15.120:42960) 2024-09-19T09:07:33.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.525+0000 7fc11bfff640 1 --2- 172.21.15.120:0/1693297384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4e6d715d0 0x55e4e6d84860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:33.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.525+0000 7fc123fff640 1 -- 172.21.15.120:0/1693297384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4e6d715d0 msgr2=0x55e4e6d84860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:33.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.525+0000 7fc123fff640 1 --2- 172.21.15.120:0/1693297384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4e6d715d0 0x55e4e6d84860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.525+0000 7fc123fff640 1 -- 172.21.15.120:0/1693297384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4e6d7c370 msgr2=0x55e4e6d7fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:33.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.525+0000 7fc123fff640 1 --2- 172.21.15.120:0/1693297384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4e6d7c370 0x55e4e6d7fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.525+0000 7fc123fff640 1 -- 172.21.15.120:0/1693297384 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4e6d80610 con 0x55e4e6d72b40 2024-09-19T09:07:33.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.525+0000 7fc11bfff640 1 --2- 172.21.15.120:0/1693297384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4e6d7c370 0x55e4e6d7fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:07:33.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.525+0000 7fc123fff640 1 --2- 172.21.15.120:0/1693297384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4e6d72b40 0x55e4e6d84da0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55e4e6d8e900 tx=0x7fc1140118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fda9d8e03d1768ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:33.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.525+0000 7fc1037fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:07:33.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.525+0000 7fc1037fe640 1 -- 172.21.15.120:0/1693297384 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11401d940 con 0x55e4e6d72b40 2024-09-19T09:07:33.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.525+0000 7fc12c8ea740 1 -- 172.21.15.120:0/1693297384 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc10c01a040 con 0x55e4e6d72b40 2024-09-19T09:07:33.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.525+0000 7fc1037fe640 1 -- 172.21.15.120:0/1693297384 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fc11401dae0 con 0x55e4e6d72b40 2024-09-19T09:07:33.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.526+0000 7fc1037fe640 1 -- 172.21.15.120:0/1693297384 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc11401dce0 con 0x55e4e6d72b40 2024-09-19T09:07:33.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.526+0000 7fc12c8ea740 10 client.26405 fetch_fsmap learned FSMap version 30 2024-09-19T09:07:33.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.526+0000 7fc12c8ea740 1 -- 172.21.15.120:0/1693297384 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e4e6df6560 con 0x55e4e6d72b40 2024-09-19T09:07:33.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.526+0000 7fc1037fe640 1 -- 172.21.15.120:0/1693297384 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc10c01a040 con 0x55e4e6d72b40 2024-09-19T09:07:33.551 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:07:33.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.526+0000 7fc1037fe640 1 -- 172.21.15.120:0/1693297384 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc11400f4e0 con 0x55e4e6d72b40 2024-09-19T09:07:33.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.526+0000 7fc12c8ea740 10 client.26405 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:07:33.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.526+0000 7fc12c8ea740 10 client.26405 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:07:33.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.526+0000 7fc12c8ea740 20 client.26405 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:33.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.526+0000 7fc12c8ea740 1 --2- 172.21.15.120:0/1693297384 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55e4e6df6cb0 0x55e4e6df90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:33.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.526+0000 7fc12c8ea740 4 client.26405 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:07:33.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.526+0000 7fc12c8ea740 1 -- 172.21.15.120:0/1693297384 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e4e6cded70 con 0x55e4e6df6cb0 2024-09-19T09:07:33.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.526+0000 7fc11bfff640 1 --2- 172.21.15.120:0/1693297384 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55e4e6df6cb0 0x55e4e6df90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:33.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.527+0000 7fc1037fe640 10 client.26405 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:07:33.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.527+0000 7fc11bfff640 1 --2- 172.21.15.120:0/1693297384 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55e4e6df6cb0 0x55e4e6df90d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=16d79cda6f977fbe server_cookie=6aea26ec399bba21 in_seq=0 out_seq=0 2024-09-19T09:07:33.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.527+0000 7fc1037fe640 1 -- 172.21.15.120:0/1693297384 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e4e6cded70 con 0x55e4e6df6cb0 2024-09-19T09:07:33.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.527+0000 7fc1037fe640 10 client.26405 handle_command_reply: tid=0 2024-09-19T09:07:33.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.527+0000 7fc12c8ea740 1 client.26405 shutdown 2024-09-19T09:07:33.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.527+0000 7fc12c8ea740 1 -- 172.21.15.120:0/1693297384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4e6d72b40 msgr2=0x55e4e6d84da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:33.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.527+0000 7fc12c8ea740 1 --2- 172.21.15.120:0/1693297384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4e6d72b40 0x55e4e6d84da0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55e4e6d8e900 tx=0x7fc1140118d0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.527+0000 7fc12c8ea740 1 -- 172.21.15.120:0/1693297384 shutdown_connections 2024-09-19T09:07:33.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.527+0000 7fc12c8ea740 1 --2- 172.21.15.120:0/1693297384 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55e4e6df6cb0 0x55e4e6df90d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.527+0000 7fc12c8ea740 1 --2- 172.21.15.120:0/1693297384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4e6d7c370 0x55e4e6d7fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.527+0000 7fc12c8ea740 1 --2- 172.21.15.120:0/1693297384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4e6d72b40 0x55e4e6d84da0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.527+0000 7fc1037fe640 0 client.26405 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:07:33.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.527+0000 7fc12c8ea740 1 --2- 172.21.15.120:0/1693297384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4e6d715d0 0x55e4e6d84860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.527+0000 7fc12c8ea740 1 -- 172.21.15.120:0/1693297384 >> 172.21.15.120:0/1693297384 conn(0x55e4e6d3a310 msgr2=0x55e4e6d6c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:33.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.527+0000 7fc12c8ea740 1 -- 172.21.15.120:0/1693297384 shutdown_connections 2024-09-19T09:07:33.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.527+0000 7fc12c8ea740 1 -- 172.21.15.120:0/1693297384 wait complete. 2024-09-19T09:07:33.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.527+0000 7fc12c8ea740 20 client.26405 trim_cache size 0 max 16384 2024-09-19T09:07:33.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.528+0000 7fc12a511640 1 -- 172.21.15.120:0/1966445565 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc1100a3240 msgr2=0x7fc1100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:33.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.528+0000 7fc12a511640 1 --2- 172.21.15.120:0/1966445565 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc1100a3240 0x7fc1100a5700 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc11c0b8ff0 tx=0x7fc11c0c1040 comp rx=0 tx=0).stop 2024-09-19T09:07:33.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.528+0000 7fc12a511640 1 -- 172.21.15.120:0/1966445565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc124106f10 msgr2=0x7fc124074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:33.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.528+0000 7fc12a511640 1 --2- 172.21.15.120:0/1966445565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc124106f10 0x7fc124074f70 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc11c031a20 tx=0x7fc11c031a50 comp rx=0 tx=0).stop 2024-09-19T09:07:33.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.528+0000 7fc12a511640 1 -- 172.21.15.120:0/1966445565 shutdown_connections 2024-09-19T09:07:33.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.528+0000 7fc12a511640 1 --2- 172.21.15.120:0/1966445565 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc1100a3240 0x7fc1100a5700 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.528+0000 7fc12a511640 1 --2- 172.21.15.120:0/1966445565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc124106f10 0x7fc124074f70 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.528+0000 7fc12a511640 1 --2- 172.21.15.120:0/1966445565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc124102790 0x7fc124074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.528+0000 7fc12a511640 1 --2- 172.21.15.120:0/1966445565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc124101de0 0x7fc1240744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:33.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.528+0000 7fc12a511640 1 -- 172.21.15.120:0/1966445565 >> 172.21.15.120:0/1966445565 conn(0x7fc1240fd8c0 msgr2=0x7fc1240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:33.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.528+0000 7fc12a511640 1 -- 172.21.15.120:0/1966445565 shutdown_connections 2024-09-19T09:07:33.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:33.528+0000 7fc12a511640 1 -- 172.21.15.120:0/1966445565 wait complete. 2024-09-19T09:07:34.047 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 8.01 MB/sec execute 89 sec latency 26085.326 ms 2024-09-19T09:07:34.465 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:07:34.595 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:07:34.712 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:07:35.047 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 7.92 MB/sec execute 90 sec latency 27085.413 ms 2024-09-19T09:07:35.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:34 smithi196 ceph-mon[35150]: pgmap v483: 97 pgs: 97 active+clean; 40 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 2.6 MiB/s wr, 123 op/s 2024-09-19T09:07:35.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:34 smithi153 ceph-mon[35071]: pgmap v483: 97 pgs: 97 active+clean; 40 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 2.6 MiB/s wr, 123 op/s 2024-09-19T09:07:35.267 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:34 smithi120 ceph-mon[31150]: pgmap v483: 97 pgs: 97 active+clean; 40 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 2.6 MiB/s wr, 123 op/s 2024-09-19T09:07:35.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.352+0000 7fcaa307c640 1 -- 172.21.15.120:0/4200175923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca9c102570 msgr2=0x7fca9c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:35.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.352+0000 7fcaa307c640 1 --2- 172.21.15.120:0/4200175923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca9c102570 0x7fca9c1029b0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fca88009900 tx=0x7fca8802a480 comp rx=0 tx=0).stop 2024-09-19T09:07:35.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.352+0000 7fcaa307c640 1 -- 172.21.15.120:0/4200175923 shutdown_connections 2024-09-19T09:07:35.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.352+0000 7fcaa307c640 1 --2- 172.21.15.120:0/4200175923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca9c102ef0 0x7fca9c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.352+0000 7fcaa307c640 1 --2- 172.21.15.120:0/4200175923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca9c102570 0x7fca9c1029b0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.352+0000 7fcaa307c640 1 --2- 172.21.15.120:0/4200175923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca9c100e20 0x7fca9c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.352+0000 7fcaa307c640 1 -- 172.21.15.120:0/4200175923 >> 172.21.15.120:0/4200175923 conn(0x7fca9c0fc7d0 msgr2=0x7fca9c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:35.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.352+0000 7fcaa307c640 1 -- 172.21.15.120:0/4200175923 shutdown_connections 2024-09-19T09:07:35.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.352+0000 7fcaa307c640 1 -- 172.21.15.120:0/4200175923 wait complete. 2024-09-19T09:07:35.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.353+0000 7fcaa307c640 1 Processor -- start 2024-09-19T09:07:35.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.353+0000 7fcaa307c640 1 -- start start 2024-09-19T09:07:35.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.353+0000 7fcaa307c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca9c100e20 0x7fca9c199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:35.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.353+0000 7fcaa307c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca9c102570 0x7fca9c1998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:35.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.353+0000 7fcaa307c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca9c102ef0 0x7fca9c1a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:35.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.353+0000 7fcaa307c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c110fa0 con 0x7fca9c100e20 2024-09-19T09:07:35.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.353+0000 7fcaa307c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c110e20 con 0x7fca9c102ef0 2024-09-19T09:07:35.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.353+0000 7fcaa307c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c111120 con 0x7fca9c102570 2024-09-19T09:07:35.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.353+0000 7fcaa0e18640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca9c100e20 0x7fca9c199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:35.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.353+0000 7fcaa0e18640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca9c100e20 0x7fca9c199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56550/0 (socket says 172.21.15.120:56550) 2024-09-19T09:07:35.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.353+0000 7fcaa0e18640 1 -- 172.21.15.120:0/1889098617 learned_addr learned my addr 172.21.15.120:0/1889098617 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:35.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.353+0000 7fca9bfff640 1 --2- 172.21.15.120:0/1889098617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca9c102570 0x7fca9c1998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:35.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.354+0000 7fcaa1619640 1 --2- 172.21.15.120:0/1889098617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca9c102ef0 0x7fca9c1a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:35.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.354+0000 7fcaa0e18640 1 -- 172.21.15.120:0/1889098617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca9c102570 msgr2=0x7fca9c1998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:35.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.354+0000 7fcaa0e18640 1 --2- 172.21.15.120:0/1889098617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca9c102570 0x7fca9c1998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.354+0000 7fcaa0e18640 1 -- 172.21.15.120:0/1889098617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca9c102ef0 msgr2=0x7fca9c1a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:35.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.354+0000 7fcaa0e18640 1 --2- 172.21.15.120:0/1889098617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca9c102ef0 0x7fca9c1a43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.354+0000 7fcaa0e18640 1 -- 172.21.15.120:0/1889098617 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca9c1a4930 con 0x7fca9c100e20 2024-09-19T09:07:35.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.354+0000 7fcaa1619640 1 --2- 172.21.15.120:0/1889098617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca9c102ef0 0x7fca9c1a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:07:35.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.354+0000 7fcaa0e18640 1 --2- 172.21.15.120:0/1889098617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca9c100e20 0x7fca9c199370 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fca9000b4a0 tx=0x7fca9000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:35.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.354+0000 7fca99ffb640 1 -- 172.21.15.120:0/1889098617 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90014070 con 0x7fca9c100e20 2024-09-19T09:07:35.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.354+0000 7fcaa307c640 1 -- 172.21.15.120:0/1889098617 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca9c1a4b00 con 0x7fca9c100e20 2024-09-19T09:07:35.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.354+0000 7fca99ffb640 1 -- 172.21.15.120:0/1889098617 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca90010af0 con 0x7fca9c100e20 2024-09-19T09:07:35.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.354+0000 7fca99ffb640 1 -- 172.21.15.120:0/1889098617 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90017d10 con 0x7fca9c100e20 2024-09-19T09:07:35.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.354+0000 7fcaa307c640 1 -- 172.21.15.120:0/1889098617 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca9c1a5010 con 0x7fca9c100e20 2024-09-19T09:07:35.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.357+0000 7fca99ffb640 1 -- 172.21.15.120:0/1889098617 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fca90017070 con 0x7fca9c100e20 2024-09-19T09:07:35.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.357+0000 7fcaa5455740 1 Processor -- start 2024-09-19T09:07:35.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.357+0000 7fcaa5455740 1 -- start start 2024-09-19T09:07:35.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.358+0000 7fcaa5455740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e49f1cb40 0x558e49f1b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:35.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.358+0000 7fcaa5455740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e49f1b5d0 0x558e49f1ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:35.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.358+0000 7fcaa5455740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e49f26370 0x558e49f28760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:35.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.358+0000 7fcaa5455740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558e49e812f0 con 0x558e49f1b5d0 2024-09-19T09:07:35.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.358+0000 7fcaa5455740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558e49e28cf0 con 0x558e49f26370 2024-09-19T09:07:35.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.358+0000 7fcaa5455740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558e49e7d2b0 con 0x558e49f1cb40 2024-09-19T09:07:35.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.358+0000 7fca99ffb640 1 --2- 172.21.15.120:0/1889098617 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fca8c0a3240 0x7fca8c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:35.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.358+0000 7fcaa0e18640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e49f1b5d0 0x558e49f1ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:35.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.358+0000 7fcaa1619640 1 --2- 172.21.15.120:0/1889098617 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fca8c0a3240 0x7fca8c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:35.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.358+0000 7fcaa0e18640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e49f1b5d0 0x558e49f1ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56562/0 (socket says 172.21.15.120:56562) 2024-09-19T09:07:35.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.358+0000 7fcaa0e18640 1 -- 172.21.15.120:0/1744817044 learned_addr learned my addr 172.21.15.120:0/1744817044 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:35.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.358+0000 7fcaa1619640 1 --2- 172.21.15.120:0/1744817044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e49f1cb40 0x558e49f1b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:35.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.358+0000 7fca9bfff640 1 --2- 172.21.15.120:0/1744817044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e49f26370 0x558e49f28760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:35.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.358+0000 7fca99ffb640 1 -- 172.21.15.120:0/1889098617 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fca900cec80 con 0x7fca9c100e20 2024-09-19T09:07:35.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.358+0000 7fcaa0e18640 1 -- 172.21.15.120:0/1744817044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e49f1cb40 msgr2=0x558e49f1b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:35.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.358+0000 7fcaa0e18640 1 --2- 172.21.15.120:0/1744817044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e49f1cb40 0x558e49f1b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.358+0000 7fcaa0e18640 1 -- 172.21.15.120:0/1744817044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e49f26370 msgr2=0x558e49f28760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:35.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.358+0000 7fcaa0e18640 1 --2- 172.21.15.120:0/1744817044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e49f26370 0x558e49f28760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.358+0000 7fcaa0e18640 1 -- 172.21.15.120:0/1744817044 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e49e0d3a0 con 0x558e49f1b5d0 2024-09-19T09:07:35.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.359+0000 7fcaa1619640 1 --2- 172.21.15.120:0/1744817044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e49f1cb40 0x558e49f1b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:07:35.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.359+0000 7fcaa1619640 1 --2- 172.21.15.120:0/1889098617 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fca8c0a3240 0x7fca8c0a5700 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fca940127a0 tx=0x7fca940079f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:35.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.359+0000 7fcaa0e18640 1 --2- 172.21.15.120:0/1744817044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e49f1b5d0 0x558e49f1ba10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fca900093d0 tx=0x7fca900047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b4e4193505640e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:35.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.359+0000 7fca86ffd640 1 -- 172.21.15.120:0/1744817044 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9001e070 con 0x558e49f1b5d0 2024-09-19T09:07:35.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.359+0000 7fca86ffd640 1 -- 172.21.15.120:0/1744817044 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca9009f650 con 0x558e49f1b5d0 2024-09-19T09:07:35.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.359+0000 7fca86ffd640 1 -- 172.21.15.120:0/1744817044 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90019450 con 0x558e49f1b5d0 2024-09-19T09:07:35.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.360+0000 7fcaa5455740 1 -- 172.21.15.120:0/1744817044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e49f1b5d0 msgr2=0x558e49f1ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:35.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.360+0000 7fcaa5455740 1 --2- 172.21.15.120:0/1744817044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e49f1b5d0 0x558e49f1ba10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fca900093d0 tx=0x7fca900047d0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.360+0000 7fcaa5455740 1 -- 172.21.15.120:0/1744817044 shutdown_connections 2024-09-19T09:07:35.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.360+0000 7fcaa5455740 1 --2- 172.21.15.120:0/1744817044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e49f26370 0x558e49f28760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.360+0000 7fcaa5455740 1 --2- 172.21.15.120:0/1744817044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e49f1b5d0 0x558e49f1ba10 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.360+0000 7fcaa5455740 1 --2- 172.21.15.120:0/1744817044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e49f1cb40 0x558e49f1b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.360+0000 7fcaa5455740 1 -- 172.21.15.120:0/1744817044 >> 172.21.15.120:0/1744817044 conn(0x558e49ee4310 msgr2=0x558e49f17c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:35.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.360+0000 7fcaa5455740 1 -- 172.21.15.120:0/1744817044 shutdown_connections 2024-09-19T09:07:35.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.360+0000 7fcaa5455740 1 -- 172.21.15.120:0/1744817044 wait complete. 2024-09-19T09:07:35.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.360+0000 7fcaa5455740 1 -- start start 2024-09-19T09:07:35.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.360+0000 7fcaa5455740 1 Processor -- start 2024-09-19T09:07:35.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.361+0000 7fcaa5455740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e49f1b5d0 0x558e49f25da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:35.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.361+0000 7fcaa5455740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e49f1cb40 0x558e49f23050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:35.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.361+0000 7fcaa5455740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e49f26370 0x558e49f237e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:35.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.361+0000 7fcaa5455740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558e49f29070 con 0x558e49f1cb40 2024-09-19T09:07:35.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.361+0000 7fcaa5455740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558e49e3c060 con 0x558e49f26370 2024-09-19T09:07:35.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.361+0000 7fcaa5455740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558e49f291f0 con 0x558e49f1b5d0 2024-09-19T09:07:35.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.361+0000 7fcaa5455740 10 client.0 _ll_register_callbacks cb 0x558e49b8f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:35.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.361+0000 7fca9bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e49f1cb40 0x558e49f23050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:35.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.361+0000 7fca9bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e49f1cb40 0x558e49f23050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56564/0 (socket says 172.21.15.120:56564) 2024-09-19T09:07:35.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.361+0000 7fca9bfff640 1 -- 172.21.15.120:0/267428203 learned_addr learned my addr 172.21.15.120:0/267428203 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:35.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.361+0000 7fcaa1619640 1 --2- 172.21.15.120:0/267428203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e49f26370 0x558e49f237e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:35.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.361+0000 7fcaa0e18640 1 --2- 172.21.15.120:0/267428203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e49f1b5d0 0x558e49f25da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:35.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.361+0000 7fca9bfff640 1 -- 172.21.15.120:0/267428203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e49f1b5d0 msgr2=0x558e49f25da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:35.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.361+0000 7fca9bfff640 1 --2- 172.21.15.120:0/267428203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e49f1b5d0 0x558e49f25da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.361+0000 7fca9bfff640 1 -- 172.21.15.120:0/267428203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e49f26370 msgr2=0x558e49f237e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:35.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.361+0000 7fca9bfff640 1 --2- 172.21.15.120:0/267428203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e49f26370 0x558e49f237e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.361+0000 7fca9bfff640 1 -- 172.21.15.120:0/267428203 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e49bf23c0 con 0x558e49f1cb40 2024-09-19T09:07:35.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.362+0000 7fca9bfff640 1 --2- 172.21.15.120:0/267428203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e49f1cb40 0x558e49f23050 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fca88009900 tx=0x7fca88004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c1a7e98ae868a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:35.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.362+0000 7fca7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:07:35.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.362+0000 7fca7bfff640 1 -- 172.21.15.120:0/267428203 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88035040 con 0x558e49f1cb40 2024-09-19T09:07:35.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.362+0000 7fca7bfff640 1 -- 172.21.15.120:0/267428203 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fca88003bc0 con 0x558e49f1cb40 2024-09-19T09:07:35.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.362+0000 7fcaa5455740 1 -- 172.21.15.120:0/267428203 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca9009f650 con 0x558e49f1cb40 2024-09-19T09:07:35.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.362+0000 7fca7bfff640 1 -- 172.21.15.120:0/267428203 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fca88003dc0 con 0x558e49f1cb40 2024-09-19T09:07:35.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.362+0000 7fcaa5455740 10 client.17100 fetch_fsmap learned FSMap version 30 2024-09-19T09:07:35.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.362+0000 7fcaa5455740 1 -- 172.21.15.120:0/267428203 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558e49f34b50 con 0x558e49f1cb40 2024-09-19T09:07:35.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.362+0000 7fca7bfff640 1 -- 172.21.15.120:0/267428203 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca9009f650 con 0x558e49f1cb40 2024-09-19T09:07:35.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.362+0000 7fca7bfff640 1 -- 172.21.15.120:0/267428203 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fca8803c6a0 con 0x558e49f1cb40 2024-09-19T09:07:35.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.362+0000 7fcaa5455740 10 client.17100 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:07:35.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.362+0000 7fcaa5455740 10 client.17100 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:07:35.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.362+0000 7fcaa5455740 20 client.17100 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:35.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.362+0000 7fcaa5455740 1 --2- 172.21.15.120:0/267428203 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x558e49f36010 0x558e49f38460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:35.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.362+0000 7fcaa5455740 4 client.17100 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:07:35.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.362+0000 7fcaa5455740 1 -- 172.21.15.120:0/267428203 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e49e8e860 con 0x558e49f36010 2024-09-19T09:07:35.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.363+0000 7fcaa0e18640 1 --2- 172.21.15.120:0/267428203 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x558e49f36010 0x558e49f38460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:35.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.363+0000 7fcaa0e18640 1 --2- 172.21.15.120:0/267428203 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x558e49f36010 0x558e49f38460 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3db0a80a3b0eb51d server_cookie=9925ea7cabf627c5 in_seq=0 out_seq=0 2024-09-19T09:07:35.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.363+0000 7fca7bfff640 10 client.17100 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:07:35.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.364+0000 7fca7bfff640 1 -- 172.21.15.120:0/267428203 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e49e8e860 con 0x558e49f36010 2024-09-19T09:07:35.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.364+0000 7fca7bfff640 10 client.17100 handle_command_reply: tid=0 2024-09-19T09:07:35.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.364+0000 7fcaa5455740 1 client.17100 shutdown 2024-09-19T09:07:35.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.364+0000 7fcaa5455740 1 -- 172.21.15.120:0/267428203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e49f1cb40 msgr2=0x558e49f23050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:35.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.364+0000 7fcaa5455740 1 --2- 172.21.15.120:0/267428203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e49f1cb40 0x558e49f23050 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fca88009900 tx=0x7fca88004580 comp rx=0 tx=0).stop 2024-09-19T09:07:35.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.364+0000 7fcaa5455740 1 -- 172.21.15.120:0/267428203 shutdown_connections 2024-09-19T09:07:35.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.364+0000 7fcaa5455740 1 --2- 172.21.15.120:0/267428203 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x558e49f36010 0x558e49f38460 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.364+0000 7fcaa5455740 1 --2- 172.21.15.120:0/267428203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e49f26370 0x558e49f237e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.364+0000 7fcaa5455740 1 --2- 172.21.15.120:0/267428203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e49f1cb40 0x558e49f23050 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.364+0000 7fca7bfff640 0 client.17100 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:07:35.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.364+0000 7fcaa5455740 1 --2- 172.21.15.120:0/267428203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e49f1b5d0 0x558e49f25da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.364+0000 7fcaa5455740 1 -- 172.21.15.120:0/267428203 >> 172.21.15.120:0/267428203 conn(0x558e49ee4310 msgr2=0x558e49f1d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:35.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.364+0000 7fcaa5455740 1 -- 172.21.15.120:0/267428203 shutdown_connections 2024-09-19T09:07:35.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.364+0000 7fcaa5455740 1 -- 172.21.15.120:0/267428203 wait complete. 2024-09-19T09:07:35.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.364+0000 7fcaa5455740 20 client.17100 trim_cache size 0 max 16384 2024-09-19T09:07:35.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.391+0000 7fcaa5455740 1 Processor -- start 2024-09-19T09:07:35.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.391+0000 7fcaa5455740 1 -- start start 2024-09-19T09:07:35.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.391+0000 7fcaa5455740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e49f1b5d0 0x558e49f24fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:35.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.391+0000 7fcaa5455740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e49f1cb40 0x558e49f1e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:35.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.391+0000 7fcaa5455740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e49f26370 0x558e49f1e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:35.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.391+0000 7fcaa5455740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c1ab450 con 0x558e49f1cb40 2024-09-19T09:07:35.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.391+0000 7fcaa5455740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c1acf90 con 0x558e49f26370 2024-09-19T09:07:35.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.391+0000 7fcaa5455740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c1abb20 con 0x558e49f1b5d0 2024-09-19T09:07:35.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.391+0000 7fca9bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e49f1cb40 0x558e49f1e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:35.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.391+0000 7fca9bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e49f1cb40 0x558e49f1e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56570/0 (socket says 172.21.15.120:56570) 2024-09-19T09:07:35.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.391+0000 7fca9bfff640 1 -- 172.21.15.120:0/386966262 learned_addr learned my addr 172.21.15.120:0/386966262 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:35.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.391+0000 7fcaa0e18640 1 --2- 172.21.15.120:0/386966262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e49f1b5d0 0x558e49f24fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:35.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.392+0000 7fcaa1619640 1 --2- 172.21.15.120:0/386966262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e49f26370 0x558e49f1e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:35.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.392+0000 7fca9bfff640 1 -- 172.21.15.120:0/386966262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e49f1b5d0 msgr2=0x558e49f24fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:35.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.392+0000 7fca9bfff640 1 --2- 172.21.15.120:0/386966262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e49f1b5d0 0x558e49f24fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.392+0000 7fca9bfff640 1 -- 172.21.15.120:0/386966262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e49f26370 msgr2=0x558e49f1e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:35.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.392+0000 7fca9bfff640 1 --2- 172.21.15.120:0/386966262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e49f26370 0x558e49f1e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.392+0000 7fca9bfff640 1 -- 172.21.15.120:0/386966262 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e49f327f0 con 0x558e49f1cb40 2024-09-19T09:07:35.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.392+0000 7fcaa1619640 1 --2- 172.21.15.120:0/386966262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e49f26370 0x558e49f1e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:07:35.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.392+0000 7fca9bfff640 1 --2- 172.21.15.120:0/386966262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e49f1cb40 0x558e49f1e180 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fca8803cb40 tx=0x7fca88005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d75e63138e1f9cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:35.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.392+0000 7fca867fc640 1 -- 172.21.15.120:0/386966262 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88035070 con 0x558e49f1cb40 2024-09-19T09:07:35.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.392+0000 7fca867fc640 1 -- 172.21.15.120:0/386966262 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca880482e0 con 0x558e49f1cb40 2024-09-19T09:07:35.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.392+0000 7fca867fc640 1 -- 172.21.15.120:0/386966262 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca880053a0 con 0x558e49f1cb40 2024-09-19T09:07:35.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.393+0000 7fcaa5455740 1 -- 172.21.15.120:0/386966262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e49f1cb40 msgr2=0x558e49f1e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:35.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.393+0000 7fcaa5455740 1 --2- 172.21.15.120:0/386966262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e49f1cb40 0x558e49f1e180 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fca8803cb40 tx=0x7fca88005c00 comp rx=0 tx=0).stop 2024-09-19T09:07:35.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.393+0000 7fcaa5455740 1 -- 172.21.15.120:0/386966262 shutdown_connections 2024-09-19T09:07:35.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.393+0000 7fcaa5455740 1 --2- 172.21.15.120:0/386966262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e49f26370 0x558e49f1e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.393+0000 7fcaa5455740 1 --2- 172.21.15.120:0/386966262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e49f1cb40 0x558e49f1e180 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.393+0000 7fcaa5455740 1 --2- 172.21.15.120:0/386966262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e49f1b5d0 0x558e49f24fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.393+0000 7fcaa5455740 1 -- 172.21.15.120:0/386966262 >> 172.21.15.120:0/386966262 conn(0x558e49ee4310 msgr2=0x558e49f248b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:35.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.393+0000 7fcaa5455740 1 -- 172.21.15.120:0/386966262 shutdown_connections 2024-09-19T09:07:35.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.393+0000 7fcaa5455740 1 -- 172.21.15.120:0/386966262 wait complete. 2024-09-19T09:07:35.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.394+0000 7fcaa5455740 1 -- start start 2024-09-19T09:07:35.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.394+0000 7fcaa5455740 1 Processor -- start 2024-09-19T09:07:35.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.394+0000 7fcaa5455740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e49f1b5d0 0x558e49f2a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:35.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.394+0000 7fcaa5455740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e49f1cb40 0x558e49f2aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:35.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.394+0000 7fcaa5455740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e49f26370 0x558e49fa8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:35.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.394+0000 7fcaa5455740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c1ac8c0 con 0x558e49f1cb40 2024-09-19T09:07:35.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.394+0000 7fcaa5455740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c1ac1f0 con 0x558e49f1b5d0 2024-09-19T09:07:35.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.394+0000 7fcaa5455740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c1a9410 con 0x558e49f26370 2024-09-19T09:07:35.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.394+0000 7fcaa5455740 10 client.0 _ll_register_callbacks cb 0x7fca88009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:35.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.394+0000 7fca9bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e49f1cb40 0x558e49f2aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:35.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.394+0000 7fca9bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e49f1cb40 0x558e49f2aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56576/0 (socket says 172.21.15.120:56576) 2024-09-19T09:07:35.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.395+0000 7fca9bfff640 1 -- 172.21.15.120:0/2837926112 learned_addr learned my addr 172.21.15.120:0/2837926112 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:35.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.395+0000 7fcaa0e18640 1 --2- 172.21.15.120:0/2837926112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e49f1b5d0 0x558e49f2a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:35.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.395+0000 7fcaa1619640 1 --2- 172.21.15.120:0/2837926112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e49f26370 0x558e49fa8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:35.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.395+0000 7fcaa0e18640 1 -- 172.21.15.120:0/2837926112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e49f26370 msgr2=0x558e49fa8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:35.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.395+0000 7fcaa0e18640 1 --2- 172.21.15.120:0/2837926112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e49f26370 0x558e49fa8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.395+0000 7fcaa0e18640 1 -- 172.21.15.120:0/2837926112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e49f1cb40 msgr2=0x558e49f2aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:35.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.395+0000 7fcaa0e18640 1 --2- 172.21.15.120:0/2837926112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e49f1cb40 0x558e49f2aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.395+0000 7fcaa0e18640 1 -- 172.21.15.120:0/2837926112 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e49fa8a90 con 0x558e49f1b5d0 2024-09-19T09:07:35.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.395+0000 7fcaa0e18640 1 --2- 172.21.15.120:0/2837926112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e49f1b5d0 0x558e49f2a590 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fca90009420 tx=0x7fca90017730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8459b6aa2a6a1e71 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:35.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.395+0000 7fca7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:07:35.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.395+0000 7fca7bfff640 1 -- 172.21.15.120:0/2837926112 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9001e040 con 0x558e49f1b5d0 2024-09-19T09:07:35.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.395+0000 7fcaa5455740 1 -- 172.21.15.120:0/2837926112 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca880482e0 con 0x558e49f1b5d0 2024-09-19T09:07:35.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.395+0000 7fca7bfff640 1 -- 172.21.15.120:0/2837926112 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fca90010ca0 con 0x558e49f1b5d0 2024-09-19T09:07:35.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.396+0000 7fca7bfff640 1 -- 172.21.15.120:0/2837926112 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fca900a3f10 con 0x558e49f1b5d0 2024-09-19T09:07:35.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.396+0000 7fcaa5455740 10 client.26420 fetch_fsmap learned FSMap version 30 2024-09-19T09:07:35.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.396+0000 7fcaa5455740 1 -- 172.21.15.120:0/2837926112 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558e49fa9be0 con 0x558e49f1b5d0 2024-09-19T09:07:35.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.396+0000 7fca7bfff640 1 -- 172.21.15.120:0/2837926112 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca880482e0 con 0x558e49f1b5d0 2024-09-19T09:07:35.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.396+0000 7fca7bfff640 1 -- 172.21.15.120:0/2837926112 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fca900bb0b0 con 0x558e49f1b5d0 2024-09-19T09:07:35.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.396+0000 7fcaa5455740 10 client.26420 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:07:35.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.396+0000 7fcaa5455740 10 client.26420 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:07:35.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.396+0000 7fcaa5455740 20 client.26420 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:35.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.396+0000 7fcaa5455740 1 --2- 172.21.15.120:0/2837926112 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x558e49f36010 0x558e49fabe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:35.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.396+0000 7fcaa5455740 4 client.26420 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:07:35.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.396+0000 7fcaa5455740 1 -- 172.21.15.120:0/2837926112 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558e49e88d70 con 0x558e49f36010 2024-09-19T09:07:35.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.396+0000 7fca9bfff640 1 --2- 172.21.15.120:0/2837926112 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x558e49f36010 0x558e49fabe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:35.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.397+0000 7fca9bfff640 1 --2- 172.21.15.120:0/2837926112 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x558e49f36010 0x558e49fabe20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ec58eba8d24f5e6c server_cookie=d045cfcaed8657ce in_seq=0 out_seq=0 2024-09-19T09:07:35.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.397+0000 7fca7bfff640 10 client.26420 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:07:35.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.397+0000 7fca7bfff640 1 -- 172.21.15.120:0/2837926112 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558e49e88d70 con 0x558e49f36010 2024-09-19T09:07:35.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.397+0000 7fca7bfff640 10 client.26420 handle_command_reply: tid=0 2024-09-19T09:07:35.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.397+0000 7fcaa5455740 1 client.26420 shutdown 2024-09-19T09:07:35.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.397+0000 7fcaa5455740 1 -- 172.21.15.120:0/2837926112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e49f1b5d0 msgr2=0x558e49f2a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:35.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.397+0000 7fcaa5455740 1 --2- 172.21.15.120:0/2837926112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e49f1b5d0 0x558e49f2a590 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fca90009420 tx=0x7fca90017730 comp rx=0 tx=0).stop 2024-09-19T09:07:35.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.397+0000 7fcaa5455740 1 -- 172.21.15.120:0/2837926112 shutdown_connections 2024-09-19T09:07:35.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.397+0000 7fcaa5455740 1 --2- 172.21.15.120:0/2837926112 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x558e49f36010 0x558e49fabe20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.397+0000 7fca7bfff640 0 client.26420 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:07:35.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.397+0000 7fcaa5455740 1 --2- 172.21.15.120:0/2837926112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e49f26370 0x558e49fa8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.397+0000 7fcaa5455740 1 --2- 172.21.15.120:0/2837926112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558e49f1cb40 0x558e49f2aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.397+0000 7fcaa5455740 1 --2- 172.21.15.120:0/2837926112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558e49f1b5d0 0x558e49f2a590 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.397+0000 7fcaa5455740 1 -- 172.21.15.120:0/2837926112 >> 172.21.15.120:0/2837926112 conn(0x558e49ee4310 msgr2=0x558e49f250d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:35.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.398+0000 7fcaa5455740 1 -- 172.21.15.120:0/2837926112 shutdown_connections 2024-09-19T09:07:35.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.398+0000 7fcaa5455740 1 -- 172.21.15.120:0/2837926112 wait complete. 2024-09-19T09:07:35.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.398+0000 7fcaa5455740 20 client.26420 trim_cache size 0 max 16384 2024-09-19T09:07:35.408 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:07:35.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.398+0000 7fcaa307c640 1 -- 172.21.15.120:0/1889098617 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fca8c0a3240 msgr2=0x7fca8c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:35.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.398+0000 7fcaa307c640 1 --2- 172.21.15.120:0/1889098617 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fca8c0a3240 0x7fca8c0a5700 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fca940127a0 tx=0x7fca940079f0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.398+0000 7fcaa307c640 1 -- 172.21.15.120:0/1889098617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca9c100e20 msgr2=0x7fca9c199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:35.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.398+0000 7fcaa307c640 1 --2- 172.21.15.120:0/1889098617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca9c100e20 0x7fca9c199370 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fca9000b4a0 tx=0x7fca9000b960 comp rx=0 tx=0).stop 2024-09-19T09:07:35.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.399+0000 7fcaa307c640 1 -- 172.21.15.120:0/1889098617 shutdown_connections 2024-09-19T09:07:35.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.399+0000 7fcaa307c640 1 --2- 172.21.15.120:0/1889098617 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fca8c0a3240 0x7fca8c0a5700 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.399+0000 7fcaa307c640 1 --2- 172.21.15.120:0/1889098617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca9c102ef0 0x7fca9c1a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.399+0000 7fcaa307c640 1 --2- 172.21.15.120:0/1889098617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca9c102570 0x7fca9c1998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.399+0000 7fcaa307c640 1 --2- 172.21.15.120:0/1889098617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca9c100e20 0x7fca9c199370 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:35.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.399+0000 7fcaa307c640 1 -- 172.21.15.120:0/1889098617 >> 172.21.15.120:0/1889098617 conn(0x7fca9c0fc7d0 msgr2=0x7fca9c1068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:35.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.399+0000 7fcaa307c640 1 -- 172.21.15.120:0/1889098617 shutdown_connections 2024-09-19T09:07:35.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:35.399+0000 7fcaa307c640 1 -- 172.21.15.120:0/1889098617 wait complete. 2024-09-19T09:07:35.745 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:07:36.047 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 7.83 MB/sec execute 91 sec latency 28085.458 ms 2024-09-19T09:07:36.220 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:07:36.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.744+0000 7f2eba1ce640 1 -- 172.21.15.120:0/2919052840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2eb4103720 msgr2=0x7f2eb410bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:36.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.744+0000 7f2eba1ce640 1 --2- 172.21.15.120:0/2919052840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2eb4103720 0x7f2eb410bc10 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c009900 tx=0x7f2e9c02a480 comp rx=0 tx=0).stop 2024-09-19T09:07:36.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.744+0000 7f2eba1ce640 1 -- 172.21.15.120:0/2919052840 shutdown_connections 2024-09-19T09:07:36.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.744+0000 7f2eba1ce640 1 --2- 172.21.15.120:0/2919052840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2eb410c1a0 0x7f2eb410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:36.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.744+0000 7f2eba1ce640 1 --2- 172.21.15.120:0/2919052840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2eb4103720 0x7f2eb410bc10 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:36.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.744+0000 7f2eba1ce640 1 --2- 172.21.15.120:0/2919052840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2eb4102e00 0x7f2eb41031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:36.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.744+0000 7f2eba1ce640 1 -- 172.21.15.120:0/2919052840 >> 172.21.15.120:0/2919052840 conn(0x7f2eb40fc7a0 msgr2=0x7f2eb40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:36.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.744+0000 7f2eba1ce640 1 -- 172.21.15.120:0/2919052840 shutdown_connections 2024-09-19T09:07:36.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.744+0000 7f2eba1ce640 1 -- 172.21.15.120:0/2919052840 wait complete. 2024-09-19T09:07:36.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.745+0000 7f2eba1ce640 1 Processor -- start 2024-09-19T09:07:36.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.745+0000 7f2eba1ce640 1 -- start start 2024-09-19T09:07:36.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.745+0000 7f2eba1ce640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2eb4102e00 0x7f2eb41a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:36.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.745+0000 7f2eba1ce640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2eb4103720 0x7f2eb41a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:36.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.745+0000 7f2eba1ce640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2eb410c1a0 0x7f2eb410ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:36.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.745+0000 7f2eba1ce640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb4111fb0 con 0x7f2eb4103720 2024-09-19T09:07:36.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.745+0000 7f2eba1ce640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb4111e30 con 0x7f2eb410c1a0 2024-09-19T09:07:36.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.745+0000 7f2eba1ce640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb4112130 con 0x7f2eb4102e00 2024-09-19T09:07:36.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.745+0000 7f2eabfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2eb4103720 0x7f2eb41a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:36.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.746+0000 7f2eabfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2eb4103720 0x7f2eb41a4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56590/0 (socket says 172.21.15.120:56590) 2024-09-19T09:07:36.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.746+0000 7f2eabfff640 1 -- 172.21.15.120:0/201975707 learned_addr learned my addr 172.21.15.120:0/201975707 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:36.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.746+0000 7f2eb37fe640 1 --2- 172.21.15.120:0/201975707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2eb4102e00 0x7f2eb41a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:36.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.746+0000 7f2eb3fff640 1 --2- 172.21.15.120:0/201975707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2eb410c1a0 0x7f2eb410ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:36.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.746+0000 7f2eabfff640 1 -- 172.21.15.120:0/201975707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2eb4102e00 msgr2=0x7f2eb41a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:36.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.746+0000 7f2eabfff640 1 --2- 172.21.15.120:0/201975707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2eb4102e00 0x7f2eb41a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:36.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.746+0000 7f2eabfff640 1 -- 172.21.15.120:0/201975707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2eb410c1a0 msgr2=0x7f2eb410ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:36.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.746+0000 7f2eabfff640 1 --2- 172.21.15.120:0/201975707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2eb410c1a0 0x7f2eb410ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:36.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.746+0000 7f2eabfff640 1 -- 172.21.15.120:0/201975707 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb410b480 con 0x7f2eb4103720 2024-09-19T09:07:36.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.746+0000 7f2eabfff640 1 --2- 172.21.15.120:0/201975707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2eb4103720 0x7f2eb41a4c10 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c006010 tx=0x7f2e9c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:36.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.746+0000 7f2eb17fa640 1 -- 172.21.15.120:0/201975707 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c03f030 con 0x7f2eb4103720 2024-09-19T09:07:36.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.746+0000 7f2eb17fa640 1 -- 172.21.15.120:0/201975707 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e9c003ce0 con 0x7f2eb4103720 2024-09-19T09:07:36.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.746+0000 7f2eba1ce640 1 -- 172.21.15.120:0/201975707 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eb410b710 con 0x7f2eb4103720 2024-09-19T09:07:36.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.746+0000 7f2eb17fa640 1 -- 172.21.15.120:0/201975707 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c035070 con 0x7f2eb4103720 2024-09-19T09:07:36.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.746+0000 7f2eba1ce640 1 -- 172.21.15.120:0/201975707 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eb41ab5a0 con 0x7f2eb4103720 2024-09-19T09:07:36.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.747+0000 7f2eba1ce640 1 -- 172.21.15.120:0/201975707 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eb41031e0 con 0x7f2eb4103720 2024-09-19T09:07:36.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.748+0000 7f2eb17fa640 1 -- 172.21.15.120:0/201975707 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f2e9c042030 con 0x7f2eb4103720 2024-09-19T09:07:36.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.749+0000 7f2eb17fa640 1 --2- 172.21.15.120:0/201975707 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2ea00a32c0 0x7f2ea00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:36.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.749+0000 7f2eb37fe640 1 --2- 172.21.15.120:0/201975707 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2ea00a32c0 0x7f2ea00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:36.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.749+0000 7f2eb17fa640 1 -- 172.21.15.120:0/201975707 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f2e9c0f0130 con 0x7f2eb4103720 2024-09-19T09:07:36.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.750+0000 7f2eb37fe640 1 --2- 172.21.15.120:0/201975707 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2ea00a32c0 0x7f2ea00a5780 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2eb410be90 tx=0x7f2ea400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:36.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.752+0000 7f2eb17fa640 1 -- 172.21.15.120:0/201975707 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2e9c0bc8c0 con 0x7f2eb4103720 2024-09-19T09:07:36.844 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:07:36.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.963+0000 7f2eba1ce640 1 -- 172.21.15.120:0/201975707 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2eb419fdc0 con 0x7f2eb4103720 2024-09-19T09:07:36.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.964+0000 7f2eb17fa640 1 -- 172.21.15.120:0/201975707 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11985 (secure 0 0 0) 0x7f2eb419fdc0 con 0x7f2eb4103720 2024-09-19T09:07:36.964 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:07:36.965 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":30,"btime":"2024-09-19T09:07:14:812065+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:07:14.812046+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26351":{"gid":26351,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/364941779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":364941779},{"type":"v1","addr":"172.21.15.153:6835","nonce":364941779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:07:36.965 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 30 2024-09-19T09:07:36.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.966+0000 7f2eba1ce640 1 -- 172.21.15.120:0/201975707 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2ea00a32c0 msgr2=0x7f2ea00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:36.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.966+0000 7f2eba1ce640 1 --2- 172.21.15.120:0/201975707 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2ea00a32c0 0x7f2ea00a5780 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2eb410be90 tx=0x7f2ea400b390 comp rx=0 tx=0).stop 2024-09-19T09:07:36.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.966+0000 7f2eba1ce640 1 -- 172.21.15.120:0/201975707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2eb4103720 msgr2=0x7f2eb41a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:36.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.966+0000 7f2eba1ce640 1 --2- 172.21.15.120:0/201975707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2eb4103720 0x7f2eb41a4c10 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c006010 tx=0x7f2e9c0042f0 comp rx=0 tx=0).stop 2024-09-19T09:07:36.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.966+0000 7f2eba1ce640 1 -- 172.21.15.120:0/201975707 shutdown_connections 2024-09-19T09:07:36.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.966+0000 7f2eba1ce640 1 --2- 172.21.15.120:0/201975707 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2ea00a32c0 0x7f2ea00a5780 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:36.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.966+0000 7f2eba1ce640 1 --2- 172.21.15.120:0/201975707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2eb410c1a0 0x7f2eb410ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:36.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.966+0000 7f2eba1ce640 1 --2- 172.21.15.120:0/201975707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2eb4103720 0x7f2eb41a4c10 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:36.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.966+0000 7f2eba1ce640 1 --2- 172.21.15.120:0/201975707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2eb4102e00 0x7f2eb41a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:36.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.966+0000 7f2eba1ce640 1 -- 172.21.15.120:0/201975707 >> 172.21.15.120:0/201975707 conn(0x7f2eb40fc7a0 msgr2=0x7f2eb4101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:36.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.966+0000 7f2eba1ce640 1 -- 172.21.15.120:0/201975707 shutdown_connections 2024-09-19T09:07:36.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:36.966+0000 7f2eba1ce640 1 -- 172.21.15.120:0/201975707 wait complete. 2024-09-19T09:07:37.023 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:36 smithi120 ceph-mon[31150]: pgmap v484: 97 pgs: 97 active+clean; 40 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 2.2 MiB/s wr, 109 op/s 2024-09-19T09:07:37.047 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 7.75 MB/sec execute 92 sec latency 29085.572 ms 2024-09-19T09:07:37.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:36 smithi196 ceph-mon[35150]: pgmap v484: 97 pgs: 97 active+clean; 40 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 2.2 MiB/s wr, 109 op/s 2024-09-19T09:07:37.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:36 smithi153 ceph-mon[35071]: pgmap v484: 97 pgs: 97 active+clean; 40 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 2.2 MiB/s wr, 109 op/s 2024-09-19T09:07:37.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.552+0000 7f1865022640 1 -- 172.21.15.120:0/1882486538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1860102790 msgr2=0x7f18601067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:37.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.552+0000 7f1865022640 1 --2- 172.21.15.120:0/1882486538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1860102790 0x7f18601067e0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f1854009900 tx=0x7f185402a410 comp rx=0 tx=0).stop 2024-09-19T09:07:37.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.552+0000 7f1865022640 1 -- 172.21.15.120:0/1882486538 shutdown_connections 2024-09-19T09:07:37.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.552+0000 7f1865022640 1 --2- 172.21.15.120:0/1882486538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1860106f10 0x7f186010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.552+0000 7f1865022640 1 --2- 172.21.15.120:0/1882486538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1860102790 0x7f18601067e0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.552+0000 7f1865022640 1 --2- 172.21.15.120:0/1882486538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1860101de0 0x7f18601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.552+0000 7f1865022640 1 -- 172.21.15.120:0/1882486538 >> 172.21.15.120:0/1882486538 conn(0x7f18600fd8c0 msgr2=0x7f18600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:37.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.552+0000 7f1865022640 1 -- 172.21.15.120:0/1882486538 shutdown_connections 2024-09-19T09:07:37.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.552+0000 7f1865022640 1 -- 172.21.15.120:0/1882486538 wait complete. 2024-09-19T09:07:37.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.553+0000 7f1865022640 1 Processor -- start 2024-09-19T09:07:37.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.553+0000 7f1865022640 1 -- start start 2024-09-19T09:07:37.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.553+0000 7f1865022640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1860101de0 0x7f186019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:37.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.553+0000 7f1865022640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1860102790 0x7f186019ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:37.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.553+0000 7f1865022640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1860106f10 0x7f186019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:37.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.553+0000 7f1865022640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1860111fe0 con 0x7f1860101de0 2024-09-19T09:07:37.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.553+0000 7f1865022640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1860111e60 con 0x7f1860102790 2024-09-19T09:07:37.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.553+0000 7f1865022640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1860112160 con 0x7f1860106f10 2024-09-19T09:07:37.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.553+0000 7f185ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1860101de0 0x7f186019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:37.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.553+0000 7f185ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1860101de0 0x7f186019a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56602/0 (socket says 172.21.15.120:56602) 2024-09-19T09:07:37.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.553+0000 7f185ed9d640 1 -- 172.21.15.120:0/1118738862 learned_addr learned my addr 172.21.15.120:0/1118738862 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:37.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.553+0000 7f185ed9d640 1 -- 172.21.15.120:0/1118738862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1860106f10 msgr2=0x7f186019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:37.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.553+0000 7f185f59e640 1 --2- 172.21.15.120:0/1118738862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1860106f10 0x7f186019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:37.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.553+0000 7f185e59c640 1 --2- 172.21.15.120:0/1118738862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1860102790 0x7f186019ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:37.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.553+0000 7f185ed9d640 1 --2- 172.21.15.120:0/1118738862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1860106f10 0x7f186019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.553+0000 7f185ed9d640 1 -- 172.21.15.120:0/1118738862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1860102790 msgr2=0x7f186019ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:37.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.553+0000 7f185ed9d640 1 --2- 172.21.15.120:0/1118738862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1860102790 0x7f186019ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.554+0000 7f185ed9d640 1 -- 172.21.15.120:0/1118738862 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186019f690 con 0x7f1860101de0 2024-09-19T09:07:37.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.554+0000 7f185f59e640 1 --2- 172.21.15.120:0/1118738862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1860106f10 0x7f186019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:07:37.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.554+0000 7f185e59c640 1 --2- 172.21.15.120:0/1118738862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1860102790 0x7f186019ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:07:37.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.554+0000 7f185ed9d640 1 --2- 172.21.15.120:0/1118738862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1860101de0 0x7f186019a6e0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f185000b4a0 tx=0x7f185000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:37.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.554+0000 7f1847fff640 1 -- 172.21.15.120:0/1118738862 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1850014070 con 0x7f1860101de0 2024-09-19T09:07:37.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.554+0000 7f1865022640 1 -- 172.21.15.120:0/1118738862 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f186019f980 con 0x7f1860101de0 2024-09-19T09:07:37.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.554+0000 7f1847fff640 1 -- 172.21.15.120:0/1118738862 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1850010af0 con 0x7f1860101de0 2024-09-19T09:07:37.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.554+0000 7f1847fff640 1 -- 172.21.15.120:0/1118738862 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1850017d10 con 0x7f1860101de0 2024-09-19T09:07:37.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.554+0000 7f1865022640 1 -- 172.21.15.120:0/1118738862 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18601a7260 con 0x7f1860101de0 2024-09-19T09:07:37.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.557+0000 7f1847fff640 1 -- 172.21.15.120:0/1118738862 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f1850017070 con 0x7f1860101de0 2024-09-19T09:07:37.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.557+0000 7f18673fb740 1 Processor -- start 2024-09-19T09:07:37.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.557+0000 7f18673fb740 1 -- start start 2024-09-19T09:07:37.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.557+0000 7f18673fb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42f5e7b40 0x55f42f5e6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:37.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.557+0000 7f18673fb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42f5e65d0 0x55f42f5e6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:37.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.557+0000 7f18673fb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42f5f1370 0x55f42f5f3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:37.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.557+0000 7f18673fb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f42f54c2f0 con 0x55f42f5e7b40 2024-09-19T09:07:37.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.557+0000 7f1847fff640 1 --2- 172.21.15.120:0/1118738862 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f18480a3240 0x7f18480a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:37.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.557+0000 7f18673fb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f42f4f3cf0 con 0x55f42f5e65d0 2024-09-19T09:07:37.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.557+0000 7f18673fb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f42f5482b0 con 0x55f42f5f1370 2024-09-19T09:07:37.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.557+0000 7f185f59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42f5e7b40 0x55f42f5e6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:37.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.558+0000 7f185f59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42f5e7b40 0x55f42f5e6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56612/0 (socket says 172.21.15.120:56612) 2024-09-19T09:07:37.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.558+0000 7f185f59e640 1 -- 172.21.15.120:0/556152628 learned_addr learned my addr 172.21.15.120:0/556152628 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:37.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.558+0000 7f1847fff640 1 -- 172.21.15.120:0/1118738862 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f18500ced90 con 0x7f1860101de0 2024-09-19T09:07:37.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.558+0000 7f185f59e640 1 --2- 172.21.15.120:0/1118738862 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f18480a3240 0x7f18480a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:37.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.558+0000 7f185e59c640 1 --2- 172.21.15.120:0/556152628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42f5f1370 0x55f42f5f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:37.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.558+0000 7f185ed9d640 1 --2- 172.21.15.120:0/556152628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42f5e65d0 0x55f42f5e6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:37.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.558+0000 7f185f59e640 1 -- 172.21.15.120:0/556152628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42f5f1370 msgr2=0x55f42f5f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:37.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.558+0000 7f185f59e640 1 --2- 172.21.15.120:0/556152628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42f5f1370 0x55f42f5f3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.558+0000 7f185f59e640 1 -- 172.21.15.120:0/556152628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42f5e65d0 msgr2=0x55f42f5e6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:37.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.558+0000 7f185f59e640 1 --2- 172.21.15.120:0/556152628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42f5e65d0 0x55f42f5e6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.558+0000 7f185f59e640 1 -- 172.21.15.120:0/556152628 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f42f4d83a0 con 0x55f42f5e7b40 2024-09-19T09:07:37.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.558+0000 7f185f59e640 1 --2- 172.21.15.120:0/556152628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42f5e7b40 0x55f42f5e6090 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f1858012e20 tx=0x7f185800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3573bb75afcab5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:37.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.558+0000 7f185f59e640 1 --2- 172.21.15.120:0/1118738862 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f18480a3240 0x7f18480a5700 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f185800afd0 tx=0x7f185801a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:37.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.558+0000 7f18457fa640 1 -- 172.21.15.120:0/556152628 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1858020070 con 0x55f42f5e7b40 2024-09-19T09:07:37.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.558+0000 7f18457fa640 1 -- 172.21.15.120:0/556152628 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f185800ac00 con 0x55f42f5e7b40 2024-09-19T09:07:37.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.558+0000 7f18457fa640 1 -- 172.21.15.120:0/556152628 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185801cc70 con 0x55f42f5e7b40 2024-09-19T09:07:37.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.559+0000 7f18673fb740 1 -- 172.21.15.120:0/556152628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42f5e7b40 msgr2=0x55f42f5e6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:37.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.559+0000 7f18673fb740 1 --2- 172.21.15.120:0/556152628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42f5e7b40 0x55f42f5e6090 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f1858012e20 tx=0x7f185800f730 comp rx=0 tx=0).stop 2024-09-19T09:07:37.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.559+0000 7f18673fb740 1 -- 172.21.15.120:0/556152628 shutdown_connections 2024-09-19T09:07:37.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.559+0000 7f18673fb740 1 --2- 172.21.15.120:0/556152628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42f5f1370 0x55f42f5f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.559+0000 7f18673fb740 1 --2- 172.21.15.120:0/556152628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42f5e65d0 0x55f42f5e6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.559+0000 7f18673fb740 1 --2- 172.21.15.120:0/556152628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42f5e7b40 0x55f42f5e6090 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.559+0000 7f18673fb740 1 -- 172.21.15.120:0/556152628 >> 172.21.15.120:0/556152628 conn(0x55f42f5af310 msgr2=0x55f42f5e2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:37.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.559+0000 7f18673fb740 1 -- 172.21.15.120:0/556152628 shutdown_connections 2024-09-19T09:07:37.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.559+0000 7f18673fb740 1 -- 172.21.15.120:0/556152628 wait complete. 2024-09-19T09:07:37.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.559+0000 7f18673fb740 1 -- start start 2024-09-19T09:07:37.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.559+0000 7f18673fb740 1 Processor -- start 2024-09-19T09:07:37.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.560+0000 7f18673fb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42f5e65d0 0x55f42f5f0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:37.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.560+0000 7f18673fb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42f5e7b40 0x55f42f5ee070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:37.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.560+0000 7f18673fb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42f5f1370 0x55f42f5ee800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:37.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.560+0000 7f18673fb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f42f5f4010 con 0x55f42f5e7b40 2024-09-19T09:07:37.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.560+0000 7f18673fb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f42f507060 con 0x55f42f5e65d0 2024-09-19T09:07:37.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.560+0000 7f18673fb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f42f5f4190 con 0x55f42f5f1370 2024-09-19T09:07:37.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.560+0000 7f18673fb740 10 client.0 _ll_register_callbacks cb 0x55f42f25a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:37.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.560+0000 7f185e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42f5e7b40 0x55f42f5ee070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:37.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.560+0000 7f185e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42f5e7b40 0x55f42f5ee070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56626/0 (socket says 172.21.15.120:56626) 2024-09-19T09:07:37.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.560+0000 7f185e59c640 1 -- 172.21.15.120:0/2391225702 learned_addr learned my addr 172.21.15.120:0/2391225702 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:37.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.560+0000 7f185ed9d640 1 --2- 172.21.15.120:0/2391225702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42f5e65d0 0x55f42f5f0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:37.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.560+0000 7f185f59e640 1 --2- 172.21.15.120:0/2391225702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42f5f1370 0x55f42f5ee800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:37.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.560+0000 7f185e59c640 1 -- 172.21.15.120:0/2391225702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42f5f1370 msgr2=0x55f42f5ee800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:37.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.560+0000 7f185e59c640 1 --2- 172.21.15.120:0/2391225702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42f5f1370 0x55f42f5ee800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.560+0000 7f185e59c640 1 -- 172.21.15.120:0/2391225702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42f5e65d0 msgr2=0x55f42f5f0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:37.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.560+0000 7f185e59c640 1 --2- 172.21.15.120:0/2391225702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42f5e65d0 0x55f42f5f0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.560+0000 7f185e59c640 1 -- 172.21.15.120:0/2391225702 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f42f2bd3c0 con 0x55f42f5e7b40 2024-09-19T09:07:37.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.560+0000 7f185ed9d640 1 --2- 172.21.15.120:0/2391225702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42f5e65d0 0x55f42f5f0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:07:37.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.560+0000 7f185e59c640 1 --2- 172.21.15.120:0/2391225702 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42f5e7b40 0x55f42f5ee070 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f18600fdce0 tx=0x7f18540044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e8f77d9a00298e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:37.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.560+0000 7f1842ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:07:37.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.560+0000 7f1842ff5640 1 -- 172.21.15.120:0/2391225702 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1854035040 con 0x55f42f5e7b40 2024-09-19T09:07:37.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.560+0000 7f1842ff5640 1 -- 172.21.15.120:0/2391225702 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f1854005590 con 0x55f42f5e7b40 2024-09-19T09:07:37.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.560+0000 7f18673fb740 1 -- 172.21.15.120:0/2391225702 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f185800ac00 con 0x55f42f5e7b40 2024-09-19T09:07:37.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.561+0000 7f1842ff5640 1 -- 172.21.15.120:0/2391225702 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f18540046c0 con 0x55f42f5e7b40 2024-09-19T09:07:37.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.561+0000 7f18673fb740 10 client.17124 fetch_fsmap learned FSMap version 30 2024-09-19T09:07:37.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.561+0000 7f18673fb740 1 -- 172.21.15.120:0/2391225702 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f42f5ffb20 con 0x55f42f5e7b40 2024-09-19T09:07:37.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.561+0000 7f1842ff5640 1 -- 172.21.15.120:0/2391225702 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f185800ac00 con 0x55f42f5e7b40 2024-09-19T09:07:37.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.561+0000 7f1842ff5640 1 -- 172.21.15.120:0/2391225702 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1854003bf0 con 0x55f42f5e7b40 2024-09-19T09:07:37.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.561+0000 7f18673fb740 10 client.17124 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:07:37.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.561+0000 7f18673fb740 10 client.17124 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:07:37.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.561+0000 7f18673fb740 20 client.17124 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:37.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.561+0000 7f18673fb740 1 --2- 172.21.15.120:0/2391225702 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55f42f600fe0 0x55f42f603410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:37.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.561+0000 7f18673fb740 4 client.17124 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:07:37.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.561+0000 7f18673fb740 1 -- 172.21.15.120:0/2391225702 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f42f559860 con 0x55f42f600fe0 2024-09-19T09:07:37.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.561+0000 7f185ed9d640 1 --2- 172.21.15.120:0/2391225702 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55f42f600fe0 0x55f42f603410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:37.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.562+0000 7f1842ff5640 10 client.17124 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:07:37.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.562+0000 7f185ed9d640 1 --2- 172.21.15.120:0/2391225702 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55f42f600fe0 0x55f42f603410 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1f9b4d0e1b5313c4 server_cookie=a26350f6c3263e83 in_seq=0 out_seq=0 2024-09-19T09:07:37.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.562+0000 7f1842ff5640 1 -- 172.21.15.120:0/2391225702 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f42f559860 con 0x55f42f600fe0 2024-09-19T09:07:37.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.562+0000 7f1842ff5640 10 client.17124 handle_command_reply: tid=0 2024-09-19T09:07:37.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.562+0000 7f18673fb740 1 client.17124 shutdown 2024-09-19T09:07:37.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.563+0000 7f18673fb740 1 -- 172.21.15.120:0/2391225702 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42f5e7b40 msgr2=0x55f42f5ee070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:37.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.563+0000 7f18673fb740 1 --2- 172.21.15.120:0/2391225702 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42f5e7b40 0x55f42f5ee070 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f18600fdce0 tx=0x7f18540044f0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.563+0000 7f18673fb740 1 -- 172.21.15.120:0/2391225702 shutdown_connections 2024-09-19T09:07:37.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.563+0000 7f18673fb740 1 --2- 172.21.15.120:0/2391225702 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55f42f600fe0 0x55f42f603410 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.563+0000 7f18673fb740 1 --2- 172.21.15.120:0/2391225702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42f5f1370 0x55f42f5ee800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.563+0000 7f18673fb740 1 --2- 172.21.15.120:0/2391225702 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42f5e7b40 0x55f42f5ee070 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.563+0000 7f18673fb740 1 --2- 172.21.15.120:0/2391225702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42f5e65d0 0x55f42f5f0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.563+0000 7f1842ff5640 0 client.17124 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:07:37.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.563+0000 7f18673fb740 1 -- 172.21.15.120:0/2391225702 >> 172.21.15.120:0/2391225702 conn(0x55f42f5af310 msgr2=0x55f42f5e8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:37.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.563+0000 7f18673fb740 1 -- 172.21.15.120:0/2391225702 shutdown_connections 2024-09-19T09:07:37.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.563+0000 7f18673fb740 1 -- 172.21.15.120:0/2391225702 wait complete. 2024-09-19T09:07:37.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.563+0000 7f18673fb740 20 client.17124 trim_cache size 0 max 16384 2024-09-19T09:07:37.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.576+0000 7f18673fb740 1 Processor -- start 2024-09-19T09:07:37.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.576+0000 7f18673fb740 1 -- start start 2024-09-19T09:07:37.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.576+0000 7f18673fb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42f5e65d0 0x55f42f5ed930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:37.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.576+0000 7f18673fb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42f5e7b40 0x55f42f5e2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:37.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.576+0000 7f18673fb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42f5f1370 0x55f42f5e2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:37.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.576+0000 7f18673fb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f18601af8d0 con 0x55f42f5f1370 2024-09-19T09:07:37.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.576+0000 7f18673fb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f18601b1410 con 0x55f42f5e65d0 2024-09-19T09:07:37.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.576+0000 7f18673fb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f18601affa0 con 0x55f42f5e7b40 2024-09-19T09:07:37.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.576+0000 7f185f59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42f5f1370 0x55f42f5e2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:37.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.576+0000 7f185f59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42f5f1370 0x55f42f5e2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56638/0 (socket says 172.21.15.120:56638) 2024-09-19T09:07:37.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.576+0000 7f185f59e640 1 -- 172.21.15.120:0/3675455327 learned_addr learned my addr 172.21.15.120:0/3675455327 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:37.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.576+0000 7f185ed9d640 1 --2- 172.21.15.120:0/3675455327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42f5e65d0 0x55f42f5ed930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:37.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.576+0000 7f185f59e640 1 -- 172.21.15.120:0/3675455327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42f5e7b40 msgr2=0x55f42f5e2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:37.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.576+0000 7f185f59e640 1 --2- 172.21.15.120:0/3675455327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42f5e7b40 0x55f42f5e2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.576+0000 7f185f59e640 1 -- 172.21.15.120:0/3675455327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42f5e65d0 msgr2=0x55f42f5ed930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:37.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.576+0000 7f185f59e640 1 --2- 172.21.15.120:0/3675455327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42f5e65d0 0x55f42f5ed930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.576+0000 7f185f59e640 1 -- 172.21.15.120:0/3675455327 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f42f5e2510 con 0x55f42f5f1370 2024-09-19T09:07:37.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.576+0000 7f185ed9d640 1 --2- 172.21.15.120:0/3675455327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42f5e65d0 0x55f42f5ed930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:07:37.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.576+0000 7f185f59e640 1 --2- 172.21.15.120:0/3675455327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42f5f1370 0x55f42f5e2c40 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f1858012500 tx=0x7f1858012530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ffe17a8a5906f73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:37.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.577+0000 7f1844ff9640 1 -- 172.21.15.120:0/3675455327 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1858020040 con 0x55f42f5f1370 2024-09-19T09:07:37.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.577+0000 7f1844ff9640 1 -- 172.21.15.120:0/3675455327 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1858029070 con 0x55f42f5f1370 2024-09-19T09:07:37.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.577+0000 7f1844ff9640 1 -- 172.21.15.120:0/3675455327 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185802ebb0 con 0x55f42f5f1370 2024-09-19T09:07:37.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.577+0000 7f18673fb740 1 -- 172.21.15.120:0/3675455327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42f5f1370 msgr2=0x55f42f5e2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:37.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.577+0000 7f18673fb740 1 --2- 172.21.15.120:0/3675455327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42f5f1370 0x55f42f5e2c40 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f1858012500 tx=0x7f1858012530 comp rx=0 tx=0).stop 2024-09-19T09:07:37.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.577+0000 7f18673fb740 1 -- 172.21.15.120:0/3675455327 shutdown_connections 2024-09-19T09:07:37.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.577+0000 7f18673fb740 1 --2- 172.21.15.120:0/3675455327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42f5f1370 0x55f42f5e2c40 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.577+0000 7f18673fb740 1 --2- 172.21.15.120:0/3675455327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42f5e7b40 0x55f42f5e2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.577+0000 7f18673fb740 1 --2- 172.21.15.120:0/3675455327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42f5e65d0 0x55f42f5ed930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.577+0000 7f18673fb740 1 -- 172.21.15.120:0/3675455327 >> 172.21.15.120:0/3675455327 conn(0x55f42f5af310 msgr2=0x55f42f5e9310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:37.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.577+0000 7f18673fb740 1 -- 172.21.15.120:0/3675455327 shutdown_connections 2024-09-19T09:07:37.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.577+0000 7f18673fb740 1 -- 172.21.15.120:0/3675455327 wait complete. 2024-09-19T09:07:37.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.577+0000 7f18673fb740 1 -- start start 2024-09-19T09:07:37.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.578+0000 7f18673fb740 1 Processor -- start 2024-09-19T09:07:37.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.578+0000 7f18673fb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42f5e65d0 0x55f42f5f9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:37.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.578+0000 7f18673fb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42f5e7b40 0x55f42f5f9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:37.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.578+0000 7f18673fb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42f5f1370 0x55f42f5eaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:37.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.578+0000 7f18673fb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f18601b0d40 con 0x55f42f5f1370 2024-09-19T09:07:37.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.578+0000 7f18673fb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f18601b0670 con 0x55f42f5e65d0 2024-09-19T09:07:37.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.578+0000 7f18673fb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f18601ad890 con 0x55f42f5e7b40 2024-09-19T09:07:37.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.578+0000 7f18673fb740 10 client.0 _ll_register_callbacks cb 0x7f1854005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:37.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.578+0000 7f185f59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42f5f1370 0x55f42f5eaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:37.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.578+0000 7f185f59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42f5f1370 0x55f42f5eaa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56644/0 (socket says 172.21.15.120:56644) 2024-09-19T09:07:37.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.578+0000 7f185f59e640 1 -- 172.21.15.120:0/3229863399 learned_addr learned my addr 172.21.15.120:0/3229863399 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:37.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.578+0000 7f185ed9d640 1 --2- 172.21.15.120:0/3229863399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42f5e65d0 0x55f42f5f9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:37.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.578+0000 7f185e59c640 1 --2- 172.21.15.120:0/3229863399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42f5e7b40 0x55f42f5f9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:37.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.578+0000 7f185f59e640 1 -- 172.21.15.120:0/3229863399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42f5e7b40 msgr2=0x55f42f5f9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:37.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.578+0000 7f185f59e640 1 --2- 172.21.15.120:0/3229863399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42f5e7b40 0x55f42f5f9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.578+0000 7f185f59e640 1 -- 172.21.15.120:0/3229863399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42f5e65d0 msgr2=0x55f42f5f9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:37.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.578+0000 7f185f59e640 1 --2- 172.21.15.120:0/3229863399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42f5e65d0 0x55f42f5f9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.578+0000 7f185f59e640 1 -- 172.21.15.120:0/3229863399 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f42f5eb290 con 0x55f42f5f1370 2024-09-19T09:07:37.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.578+0000 7f185e59c640 1 --2- 172.21.15.120:0/3229863399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42f5e7b40 0x55f42f5f9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:07:37.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.578+0000 7f185f59e640 1 --2- 172.21.15.120:0/3229863399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42f5f1370 0x55f42f5eaa90 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f185800a710 tx=0x7f18580129f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3704f2556f7088f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:37.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.578+0000 7f1842ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:07:37.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.579+0000 7f1842ff5640 1 -- 172.21.15.120:0/3229863399 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1858020050 con 0x55f42f5f1370 2024-09-19T09:07:37.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.579+0000 7f18673fb740 1 -- 172.21.15.120:0/3229863399 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1858029070 con 0x55f42f5f1370 2024-09-19T09:07:37.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.579+0000 7f1842ff5640 1 -- 172.21.15.120:0/3229863399 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f185802e470 con 0x55f42f5f1370 2024-09-19T09:07:37.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.579+0000 7f1842ff5640 1 -- 172.21.15.120:0/3229863399 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f185802e670 con 0x55f42f5f1370 2024-09-19T09:07:37.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.579+0000 7f18673fb740 10 client.17130 fetch_fsmap learned FSMap version 30 2024-09-19T09:07:37.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.579+0000 7f18673fb740 1 -- 172.21.15.120:0/3229863399 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f42f66b870 con 0x55f42f5f1370 2024-09-19T09:07:37.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.579+0000 7f1842ff5640 1 -- 172.21.15.120:0/3229863399 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1858029070 con 0x55f42f5f1370 2024-09-19T09:07:37.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.579+0000 7f1842ff5640 1 -- 172.21.15.120:0/3229863399 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f185801b070 con 0x55f42f5f1370 2024-09-19T09:07:37.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.579+0000 7f18673fb740 10 client.17130 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:07:37.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.579+0000 7f18673fb740 10 client.17130 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:07:37.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.579+0000 7f18673fb740 20 client.17130 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:37.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.579+0000 7f18673fb740 1 --2- 172.21.15.120:0/3229863399 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55f42f66be10 0x55f42f66e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:37.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.579+0000 7f18673fb740 4 client.17130 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:07:37.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.579+0000 7f18673fb740 1 -- 172.21.15.120:0/3229863399 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f42f553d70 con 0x55f42f66be10 2024-09-19T09:07:37.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.580+0000 7f185ed9d640 1 --2- 172.21.15.120:0/3229863399 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55f42f66be10 0x55f42f66e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:37.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.580+0000 7f1842ff5640 10 client.17130 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:07:37.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.580+0000 7f185ed9d640 1 --2- 172.21.15.120:0/3229863399 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55f42f66be10 0x55f42f66e230 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa38a3dbb03b6bbe server_cookie=6e5cf45677e89df4 in_seq=0 out_seq=0 2024-09-19T09:07:37.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.580+0000 7f1842ff5640 1 -- 172.21.15.120:0/3229863399 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f42f553d70 con 0x55f42f66be10 2024-09-19T09:07:37.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.580+0000 7f1842ff5640 10 client.17130 handle_command_reply: tid=0 2024-09-19T09:07:37.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.580+0000 7f18673fb740 1 client.17130 shutdown 2024-09-19T09:07:37.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.580+0000 7f18673fb740 1 -- 172.21.15.120:0/3229863399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42f5f1370 msgr2=0x55f42f5eaa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:37.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.580+0000 7f18673fb740 1 --2- 172.21.15.120:0/3229863399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42f5f1370 0x55f42f5eaa90 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f185800a710 tx=0x7f18580129f0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.581+0000 7f18673fb740 1 -- 172.21.15.120:0/3229863399 shutdown_connections 2024-09-19T09:07:37.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.581+0000 7f18673fb740 1 --2- 172.21.15.120:0/3229863399 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55f42f66be10 0x55f42f66e230 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.581+0000 7f18673fb740 1 --2- 172.21.15.120:0/3229863399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f42f5f1370 0x55f42f5eaa90 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.581+0000 7f18673fb740 1 --2- 172.21.15.120:0/3229863399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f42f5e7b40 0x55f42f5f9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.581+0000 7f1842ff5640 0 client.17130 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:07:37.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.581+0000 7f18673fb740 1 --2- 172.21.15.120:0/3229863399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f42f5e65d0 0x55f42f5f9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.581+0000 7f18673fb740 1 -- 172.21.15.120:0/3229863399 >> 172.21.15.120:0/3229863399 conn(0x55f42f5af310 msgr2=0x55f42f5ef8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:37.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.581+0000 7f18673fb740 1 -- 172.21.15.120:0/3229863399 shutdown_connections 2024-09-19T09:07:37.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.581+0000 7f18673fb740 1 -- 172.21.15.120:0/3229863399 wait complete. 2024-09-19T09:07:37.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.581+0000 7f18673fb740 20 client.17130 trim_cache size 0 max 16384 2024-09-19T09:07:37.599 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:07:37.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.581+0000 7f1865022640 1 -- 172.21.15.120:0/1118738862 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f18480a3240 msgr2=0x7f18480a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:37.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.581+0000 7f1865022640 1 --2- 172.21.15.120:0/1118738862 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f18480a3240 0x7f18480a5700 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f185800afd0 tx=0x7f185801a000 comp rx=0 tx=0).stop 2024-09-19T09:07:37.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.581+0000 7f1865022640 1 -- 172.21.15.120:0/1118738862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1860101de0 msgr2=0x7f186019a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:37.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.581+0000 7f1865022640 1 --2- 172.21.15.120:0/1118738862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1860101de0 0x7f186019a6e0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f185000b4a0 tx=0x7f185000b960 comp rx=0 tx=0).stop 2024-09-19T09:07:37.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.581+0000 7f1865022640 1 -- 172.21.15.120:0/1118738862 shutdown_connections 2024-09-19T09:07:37.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.581+0000 7f1865022640 1 --2- 172.21.15.120:0/1118738862 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f18480a3240 0x7f18480a5700 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.581+0000 7f1865022640 1 --2- 172.21.15.120:0/1118738862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1860106f10 0x7f186019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.581+0000 7f1865022640 1 --2- 172.21.15.120:0/1118738862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1860102790 0x7f186019ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.581+0000 7f1865022640 1 --2- 172.21.15.120:0/1118738862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1860101de0 0x7f186019a6e0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:37.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.581+0000 7f1865022640 1 -- 172.21.15.120:0/1118738862 >> 172.21.15.120:0/1118738862 conn(0x7f18600fd8c0 msgr2=0x7f18600febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:37.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.581+0000 7f1865022640 1 -- 172.21.15.120:0/1118738862 shutdown_connections 2024-09-19T09:07:37.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:37.581+0000 7f1865022640 1 -- 172.21.15.120:0/1118738862 wait complete. 2024-09-19T09:07:38.047 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 7.66 MB/sec execute 93 sec latency 30085.688 ms 2024-09-19T09:07:38.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:37 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/201975707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:07:38.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:37 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/201975707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:07:38.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:37 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/201975707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:07:38.450 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db --set-id 27b8e0 --release --await' 2024-09-19T09:07:38.450 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db --set-id 27b8e0 --release --await 2024-09-19T09:07:38.970 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:07:39.002 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:07:39.047 INFO:tasks.workunit.client.0.smithi120.stdout: 1 359675 7.58 MB/sec execute 94 sec latency 31085.715 ms 2024-09-19T09:07:39.131 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:38 smithi120 ceph-mon[31150]: pgmap v485: 97 pgs: 97 active+clean; 37 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 2.4 MiB/s wr, 119 op/s 2024-09-19T09:07:39.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:38 smithi196 ceph-mon[35150]: pgmap v485: 97 pgs: 97 active+clean; 37 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 2.4 MiB/s wr, 119 op/s 2024-09-19T09:07:39.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:38 smithi153 ceph-mon[35071]: pgmap v485: 97 pgs: 97 active+clean; 37 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 2.4 MiB/s wr, 119 op/s 2024-09-19T09:07:39.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.531+0000 7f17f5330640 1 -- 172.21.15.120:0/547202809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17f0106f10 msgr2=0x7f17f010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:39.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.531+0000 7f17f5330640 1 --2- 172.21.15.120:0/547202809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17f0106f10 0x7f17f010ea90 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f17e8009ca0 tx=0x7f17e802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.531+0000 7f17f5330640 1 -- 172.21.15.120:0/547202809 shutdown_connections 2024-09-19T09:07:39.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.531+0000 7f17f5330640 1 --2- 172.21.15.120:0/547202809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17f0106f10 0x7f17f010ea90 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.531+0000 7f17f5330640 1 --2- 172.21.15.120:0/547202809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17f0102790 0x7f17f01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.531+0000 7f17f5330640 1 --2- 172.21.15.120:0/547202809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17f0101de0 0x7f17f01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.531+0000 7f17f5330640 1 -- 172.21.15.120:0/547202809 >> 172.21.15.120:0/547202809 conn(0x7f17f00fd8c0 msgr2=0x7f17f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:39.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.531+0000 7f17f5330640 1 -- 172.21.15.120:0/547202809 shutdown_connections 2024-09-19T09:07:39.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.531+0000 7f17f5330640 1 -- 172.21.15.120:0/547202809 wait complete. 2024-09-19T09:07:39.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.532+0000 7f17f5330640 1 Processor -- start 2024-09-19T09:07:39.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.532+0000 7f17f5330640 1 -- start start 2024-09-19T09:07:39.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.532+0000 7f17f5330640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17f0101de0 0x7f17f0198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:39.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.532+0000 7f17f5330640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17f0102790 0x7f17f0198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:39.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.532+0000 7f17f5330640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17f0106f10 0x7f17f019ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:39.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.532+0000 7f17f5330640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f17f0112010 con 0x7f17f0102790 2024-09-19T09:07:39.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.532+0000 7f17f5330640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f17f0111e90 con 0x7f17f0101de0 2024-09-19T09:07:39.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.532+0000 7f17f5330640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17f0112190 con 0x7f17f0106f10 2024-09-19T09:07:39.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.532+0000 7f17ee7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17f0102790 0x7f17f0198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:39.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.532+0000 7f17ee7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17f0102790 0x7f17f0198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56660/0 (socket says 172.21.15.120:56660) 2024-09-19T09:07:39.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.532+0000 7f17ee7fc640 1 -- 172.21.15.120:0/2243834604 learned_addr learned my addr 172.21.15.120:0/2243834604 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:39.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.532+0000 7f17ef7fe640 1 --2- 172.21.15.120:0/2243834604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17f0106f10 0x7f17f019ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:39.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.532+0000 7f17ee7fc640 1 -- 172.21.15.120:0/2243834604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17f0106f10 msgr2=0x7f17f019ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:39.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.532+0000 7f17ee7fc640 1 --2- 172.21.15.120:0/2243834604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17f0106f10 0x7f17f019ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.532+0000 7f17ee7fc640 1 -- 172.21.15.120:0/2243834604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17f0101de0 msgr2=0x7f17f0198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:39.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.532+0000 7f17ee7fc640 1 --2- 172.21.15.120:0/2243834604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17f0101de0 0x7f17f0198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.532+0000 7f17ee7fc640 1 -- 172.21.15.120:0/2243834604 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f019d510 con 0x7f17f0102790 2024-09-19T09:07:39.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.533+0000 7f17ee7fc640 1 --2- 172.21.15.120:0/2243834604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17f0102790 0x7f17f0198aa0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f17e400b4a0 tx=0x7f17e400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:39.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.533+0000 7f17d7fff640 1 -- 172.21.15.120:0/2243834604 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e4014070 con 0x7f17f0102790 2024-09-19T09:07:39.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.533+0000 7f17d7fff640 1 -- 172.21.15.120:0/2243834604 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17e4010af0 con 0x7f17f0102790 2024-09-19T09:07:39.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.533+0000 7f17d7fff640 1 -- 172.21.15.120:0/2243834604 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e4017d10 con 0x7f17f0102790 2024-09-19T09:07:39.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.533+0000 7f17f5330640 1 -- 172.21.15.120:0/2243834604 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f019d800 con 0x7f17f0102790 2024-09-19T09:07:39.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.533+0000 7f17f5330640 1 -- 172.21.15.120:0/2243834604 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f01a50e0 con 0x7f17f0102790 2024-09-19T09:07:39.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.535+0000 7f17f7709740 1 Processor -- start 2024-09-19T09:07:39.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.535+0000 7f17d7fff640 1 -- 172.21.15.120:0/2243834604 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f17e4017070 con 0x7f17f0102790 2024-09-19T09:07:39.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.535+0000 7f17f7709740 1 -- start start 2024-09-19T09:07:39.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.535+0000 7f17f7709740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578e8085060 0x5578e8085440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:39.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.535+0000 7f17f7709740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578e8085980 0x5578e808fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:39.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.535+0000 7f17f7709740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578e8090240 0x5578e8092630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:39.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.535+0000 7f17f7709740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5578e7fea620 con 0x5578e8085060 2024-09-19T09:07:39.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.535+0000 7f17f7709740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5578e7f91e50 con 0x5578e8090240 2024-09-19T09:07:39.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.535+0000 7f17f7709740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5578e7fe65e0 con 0x5578e8085980 2024-09-19T09:07:39.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.536+0000 7f17ef7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578e8085060 0x5578e8085440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:39.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.536+0000 7f17ef7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578e8085060 0x5578e8085440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56670/0 (socket says 172.21.15.120:56670) 2024-09-19T09:07:39.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.536+0000 7f17ef7fe640 1 -- 172.21.15.120:0/2832739831 learned_addr learned my addr 172.21.15.120:0/2832739831 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:39.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.536+0000 7f17eeffd640 1 --2- 172.21.15.120:0/2832739831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578e8085980 0x5578e808fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:39.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.536+0000 7f17ee7fc640 1 --2- 172.21.15.120:0/2832739831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578e8090240 0x5578e8092630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:39.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.536+0000 7f17ef7fe640 1 -- 172.21.15.120:0/2832739831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578e8085980 msgr2=0x5578e808fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:39.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.536+0000 7f17ef7fe640 1 --2- 172.21.15.120:0/2832739831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578e8085980 0x5578e808fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.536+0000 7f17ef7fe640 1 -- 172.21.15.120:0/2832739831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578e8090240 msgr2=0x5578e8092630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:39.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.536+0000 7f17d7fff640 1 --2- 172.21.15.120:0/2243834604 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f17d80a3240 0x7f17d80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:39.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.536+0000 7f17ef7fe640 1 --2- 172.21.15.120:0/2832739831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578e8090240 0x5578e8092630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.536+0000 7f17ef7fe640 1 -- 172.21.15.120:0/2832739831 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578e7f82d40 con 0x5578e8085060 2024-09-19T09:07:39.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.536+0000 7f17ee7fc640 1 --2- 172.21.15.120:0/2832739831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578e8090240 0x5578e8092630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:07:39.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.536+0000 7f17d7fff640 1 -- 172.21.15.120:0/2243834604 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f17e40cee00 con 0x7f17f0102790 2024-09-19T09:07:39.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.536+0000 7f17ef7fe640 1 --2- 172.21.15.120:0/2243834604 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f17d80a3240 0x7f17d80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:39.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.536+0000 7f17ef7fe640 1 --2- 172.21.15.120:0/2832739831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578e8085060 0x5578e8085440 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f17e8005cc0 tx=0x7f17e8009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb262fab43a90d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:39.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.536+0000 7f17d57fa640 1 -- 172.21.15.120:0/2832739831 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8003a50 con 0x5578e8085060 2024-09-19T09:07:39.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.537+0000 7f17d57fa640 1 -- 172.21.15.120:0/2832739831 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17e8036050 con 0x5578e8085060 2024-09-19T09:07:39.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.537+0000 7f17d57fa640 1 -- 172.21.15.120:0/2832739831 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e802fcf0 con 0x5578e8085060 2024-09-19T09:07:39.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.537+0000 7f17f7709740 1 -- 172.21.15.120:0/2832739831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578e8085060 msgr2=0x5578e8085440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:39.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.537+0000 7f17f7709740 1 --2- 172.21.15.120:0/2832739831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578e8085060 0x5578e8085440 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f17e8005cc0 tx=0x7f17e8009930 comp rx=0 tx=0).stop 2024-09-19T09:07:39.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.537+0000 7f17ef7fe640 1 --2- 172.21.15.120:0/2243834604 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f17d80a3240 0x7f17d80a5700 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f17e8049000 tx=0x7f17e804a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:39.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.537+0000 7f17f7709740 1 -- 172.21.15.120:0/2832739831 shutdown_connections 2024-09-19T09:07:39.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.537+0000 7f17f7709740 1 --2- 172.21.15.120:0/2832739831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578e8090240 0x5578e8092630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.537+0000 7f17f7709740 1 --2- 172.21.15.120:0/2832739831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578e8085980 0x5578e808fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.537+0000 7f17f7709740 1 --2- 172.21.15.120:0/2832739831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578e8085060 0x5578e8085440 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.537+0000 7f17f7709740 1 -- 172.21.15.120:0/2832739831 >> 172.21.15.120:0/2832739831 conn(0x5578e80815f0 msgr2=0x5578e80819d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:39.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.537+0000 7f17f7709740 1 -- 172.21.15.120:0/2832739831 shutdown_connections 2024-09-19T09:07:39.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.537+0000 7f17f7709740 1 -- 172.21.15.120:0/2832739831 wait complete. 2024-09-19T09:07:39.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.538+0000 7f17f7709740 1 -- start start 2024-09-19T09:07:39.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.538+0000 7f17f7709740 1 Processor -- start 2024-09-19T09:07:39.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.538+0000 7f17f7709740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578e8085060 0x5578e808d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:39.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.538+0000 7f17f7709740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578e8085980 0x5578e808d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:39.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.538+0000 7f17f7709740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578e8090240 0x5578e808dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:39.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.538+0000 7f17f7709740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5578e8092c90 con 0x5578e8085980 2024-09-19T09:07:39.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.538+0000 7f17f7709740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5578e7f72f10 con 0x5578e8085060 2024-09-19T09:07:39.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.538+0000 7f17f7709740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5578e8092e10 con 0x5578e8090240 2024-09-19T09:07:39.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.538+0000 7f17f7709740 10 client.0 _ll_register_callbacks cb 0x5578e7f96db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:39.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.538+0000 7f17ee7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578e8085980 0x5578e808d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:39.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.538+0000 7f17ee7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578e8085980 0x5578e808d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56680/0 (socket says 172.21.15.120:56680) 2024-09-19T09:07:39.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.538+0000 7f17ee7fc640 1 -- 172.21.15.120:0/3597116046 learned_addr learned my addr 172.21.15.120:0/3597116046 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:39.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.538+0000 7f17eeffd640 1 --2- 172.21.15.120:0/3597116046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578e8085060 0x5578e808d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:39.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.538+0000 7f17ef7fe640 1 --2- 172.21.15.120:0/3597116046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578e8090240 0x5578e808dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:39.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.538+0000 7f17ee7fc640 1 -- 172.21.15.120:0/3597116046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578e8090240 msgr2=0x5578e808dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:39.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.538+0000 7f17ee7fc640 1 --2- 172.21.15.120:0/3597116046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578e8090240 0x5578e808dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.538+0000 7f17ee7fc640 1 -- 172.21.15.120:0/3597116046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578e8085060 msgr2=0x5578e808d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:39.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.538+0000 7f17ee7fc640 1 --2- 172.21.15.120:0/3597116046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578e8085060 0x5578e808d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.538+0000 7f17ee7fc640 1 -- 172.21.15.120:0/3597116046 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578e7cf9580 con 0x5578e8085980 2024-09-19T09:07:39.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.538+0000 7f17eeffd640 1 --2- 172.21.15.120:0/3597116046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578e8085060 0x5578e808d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:07:39.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.538+0000 7f17ef7fe640 1 --2- 172.21.15.120:0/3597116046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578e8090240 0x5578e808dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:07:39.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.538+0000 7f17ee7fc640 1 --2- 172.21.15.120:0/3597116046 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578e8085980 0x5578e808d680 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f17e4009420 tx=0x7f17e400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f28a1cd6cb8821 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:39.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.539+0000 7f17d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:07:39.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.539+0000 7f17d2ff5640 1 -- 172.21.15.120:0/3597116046 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e401e070 con 0x5578e8085980 2024-09-19T09:07:39.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.539+0000 7f17d2ff5640 1 -- 172.21.15.120:0/3597116046 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f17e4097a90 con 0x5578e8085980 2024-09-19T09:07:39.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.539+0000 7f17f7709740 1 -- 172.21.15.120:0/3597116046 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17e8036050 con 0x5578e8085980 2024-09-19T09:07:39.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.539+0000 7f17d2ff5640 1 -- 172.21.15.120:0/3597116046 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f17e409b420 con 0x5578e8085980 2024-09-19T09:07:39.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.539+0000 7f17f7709740 10 client.17142 fetch_fsmap learned FSMap version 30 2024-09-19T09:07:39.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.539+0000 7f17f7709740 1 -- 172.21.15.120:0/3597116046 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5578e7dc3960 con 0x5578e8085980 2024-09-19T09:07:39.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.539+0000 7f17d2ff5640 1 -- 172.21.15.120:0/3597116046 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17e8036050 con 0x5578e8085980 2024-09-19T09:07:39.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.539+0000 7f17d2ff5640 1 -- 172.21.15.120:0/3597116046 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f17e40b40f0 con 0x5578e8085980 2024-09-19T09:07:39.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.539+0000 7f17f7709740 10 client.17142 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:07:39.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.539+0000 7f17f7709740 10 client.17142 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:07:39.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.539+0000 7f17f7709740 20 client.17142 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:39.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.539+0000 7f17f7709740 1 --2- 172.21.15.120:0/3597116046 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5578e809fb90 0x5578e80a2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:39.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.539+0000 7f17f7709740 4 client.17142 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:07:39.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.539+0000 7f17f7709740 1 -- 172.21.15.120:0/3597116046 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578e7ff7b90 con 0x5578e809fb90 2024-09-19T09:07:39.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.540+0000 7f17eeffd640 1 --2- 172.21.15.120:0/3597116046 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5578e809fb90 0x5578e80a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:39.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.540+0000 7f17d2ff5640 10 client.17142 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:39.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.540+0000 7f17eeffd640 1 --2- 172.21.15.120:0/3597116046 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5578e809fb90 0x5578e80a2010 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c476a9db83510ed server_cookie=f2867168e6ea134 in_seq=0 out_seq=0 2024-09-19T09:07:39.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.541+0000 7f17d2ff5640 1 -- 172.21.15.120:0/3597116046 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5578e7ff7b90 con 0x5578e809fb90 2024-09-19T09:07:39.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.541+0000 7f17d2ff5640 10 client.17142 handle_command_reply: tid=0 2024-09-19T09:07:39.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.541+0000 7f17f7709740 1 client.17142 shutdown 2024-09-19T09:07:39.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.541+0000 7f17f7709740 1 -- 172.21.15.120:0/3597116046 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578e8085980 msgr2=0x5578e808d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:39.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.541+0000 7f17f7709740 1 --2- 172.21.15.120:0/3597116046 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578e8085980 0x5578e808d680 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f17e4009420 tx=0x7f17e400bf90 comp rx=0 tx=0).stop 2024-09-19T09:07:39.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.541+0000 7f17f7709740 1 -- 172.21.15.120:0/3597116046 shutdown_connections 2024-09-19T09:07:39.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.541+0000 7f17f7709740 1 --2- 172.21.15.120:0/3597116046 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5578e809fb90 0x5578e80a2010 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.541+0000 7f17f7709740 1 --2- 172.21.15.120:0/3597116046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578e8090240 0x5578e808dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.541+0000 7f17f7709740 1 --2- 172.21.15.120:0/3597116046 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578e8085980 0x5578e808d680 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.541+0000 7f17d2ff5640 0 client.17142 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:39.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.541+0000 7f17f7709740 1 --2- 172.21.15.120:0/3597116046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578e8085060 0x5578e808d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.541+0000 7f17f7709740 1 -- 172.21.15.120:0/3597116046 >> 172.21.15.120:0/3597116046 conn(0x5578e80815f0 msgr2=0x5578e80872d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:39.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.541+0000 7f17f7709740 1 -- 172.21.15.120:0/3597116046 shutdown_connections 2024-09-19T09:07:39.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.541+0000 7f17f7709740 1 -- 172.21.15.120:0/3597116046 wait complete. 2024-09-19T09:07:39.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.541+0000 7f17f7709740 20 client.17142 trim_cache size 0 max 16384 2024-09-19T09:07:39.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.558+0000 7f17f7709740 1 Processor -- start 2024-09-19T09:07:39.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.558+0000 7f17f7709740 1 -- start start 2024-09-19T09:07:39.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.558+0000 7f17f7709740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578e8085060 0x5578e8087ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:39.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.558+0000 7f17f7709740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578e8085980 0x5578e8080640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:39.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.558+0000 7f17f7709740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578e8090240 0x5578e80a78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:39.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.558+0000 7f17f7709740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01ad6c0 con 0x5578e8085980 2024-09-19T09:07:39.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.558+0000 7f17f7709740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01af200 con 0x5578e8090240 2024-09-19T09:07:39.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.558+0000 7f17f7709740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01add90 con 0x5578e8085060 2024-09-19T09:07:39.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.559+0000 7f17ee7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578e8085980 0x5578e8080640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:39.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.559+0000 7f17ee7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578e8085980 0x5578e8080640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56686/0 (socket says 172.21.15.120:56686) 2024-09-19T09:07:39.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.559+0000 7f17ee7fc640 1 -- 172.21.15.120:0/3434031274 learned_addr learned my addr 172.21.15.120:0/3434031274 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:39.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.559+0000 7f17eeffd640 1 --2- 172.21.15.120:0/3434031274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578e8085060 0x5578e8087ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:39.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.559+0000 7f17ef7fe640 1 --2- 172.21.15.120:0/3434031274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578e8090240 0x5578e80a78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:39.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.559+0000 7f17ee7fc640 1 -- 172.21.15.120:0/3434031274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578e8085060 msgr2=0x5578e8087ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:39.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.559+0000 7f17ee7fc640 1 --2- 172.21.15.120:0/3434031274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578e8085060 0x5578e8087ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.559+0000 7f17ee7fc640 1 -- 172.21.15.120:0/3434031274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578e8090240 msgr2=0x5578e80a78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:39.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.559+0000 7f17ee7fc640 1 --2- 172.21.15.120:0/3434031274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578e8090240 0x5578e80a78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.559+0000 7f17ee7fc640 1 -- 172.21.15.120:0/3434031274 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578e8082540 con 0x5578e8085980 2024-09-19T09:07:39.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.559+0000 7f17ef7fe640 1 --2- 172.21.15.120:0/3434031274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578e8090240 0x5578e80a78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:07:39.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.559+0000 7f17ee7fc640 1 --2- 172.21.15.120:0/3434031274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578e8085980 0x5578e8080640 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f17e40cf170 tx=0x7f17e40047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5710a737b0635e18 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:39.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.559+0000 7f17d4ff9640 1 -- 172.21.15.120:0/3434031274 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e401e070 con 0x5578e8085980 2024-09-19T09:07:39.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.559+0000 7f17d4ff9640 1 -- 172.21.15.120:0/3434031274 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17e409ab90 con 0x5578e8085980 2024-09-19T09:07:39.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.559+0000 7f17d4ff9640 1 -- 172.21.15.120:0/3434031274 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e40b40f0 con 0x5578e8085980 2024-09-19T09:07:39.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.560+0000 7f17f7709740 1 -- 172.21.15.120:0/3434031274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578e8085980 msgr2=0x5578e8080640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:39.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.560+0000 7f17f7709740 1 --2- 172.21.15.120:0/3434031274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578e8085980 0x5578e8080640 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f17e40cf170 tx=0x7f17e40047d0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.560+0000 7f17f7709740 1 -- 172.21.15.120:0/3434031274 shutdown_connections 2024-09-19T09:07:39.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.560+0000 7f17f7709740 1 --2- 172.21.15.120:0/3434031274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578e8090240 0x5578e80a78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.560+0000 7f17f7709740 1 --2- 172.21.15.120:0/3434031274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578e8085980 0x5578e8080640 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.560+0000 7f17f7709740 1 --2- 172.21.15.120:0/3434031274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578e8085060 0x5578e8087ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.560+0000 7f17f7709740 1 -- 172.21.15.120:0/3434031274 >> 172.21.15.120:0/3434031274 conn(0x5578e80815f0 msgr2=0x5578e807ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:39.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.561+0000 7f17f7709740 1 -- 172.21.15.120:0/3434031274 shutdown_connections 2024-09-19T09:07:39.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.561+0000 7f17f7709740 1 -- 172.21.15.120:0/3434031274 wait complete. 2024-09-19T09:07:39.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.561+0000 7f17f7709740 1 -- start start 2024-09-19T09:07:39.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.561+0000 7f17f7709740 1 Processor -- start 2024-09-19T09:07:39.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.561+0000 7f17f7709740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578e8085060 0x5578e8097060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:39.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.561+0000 7f17f7709740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578e8085980 0x5578e80975a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:39.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.561+0000 7f17f7709740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578e8090240 0x5578e8097c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:39.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.561+0000 7f17f7709740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01aeb30 con 0x5578e8090240 2024-09-19T09:07:39.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.561+0000 7f17f7709740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01ae460 con 0x5578e8085980 2024-09-19T09:07:39.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.561+0000 7f17f7709740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01ab680 con 0x5578e8085060 2024-09-19T09:07:39.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.561+0000 7f17f7709740 10 client.0 _ll_register_callbacks cb 0x7f17e40976c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:39.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.561+0000 7f17ef7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578e8090240 0x5578e8097c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:39.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.561+0000 7f17ef7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578e8090240 0x5578e8097c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56696/0 (socket says 172.21.15.120:56696) 2024-09-19T09:07:39.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.561+0000 7f17ef7fe640 1 -- 172.21.15.120:0/324971325 learned_addr learned my addr 172.21.15.120:0/324971325 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:39.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.562+0000 7f17ee7fc640 1 --2- 172.21.15.120:0/324971325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578e8085980 0x5578e80975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:39.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.562+0000 7f17eeffd640 1 --2- 172.21.15.120:0/324971325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578e8085060 0x5578e8097060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:39.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.562+0000 7f17ef7fe640 1 -- 172.21.15.120:0/324971325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578e8085060 msgr2=0x5578e8097060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:39.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.562+0000 7f17ef7fe640 1 --2- 172.21.15.120:0/324971325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578e8085060 0x5578e8097060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.562+0000 7f17ef7fe640 1 -- 172.21.15.120:0/324971325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578e8085980 msgr2=0x5578e80975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:39.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.562+0000 7f17ef7fe640 1 --2- 172.21.15.120:0/324971325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578e8085980 0x5578e80975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.562+0000 7f17ef7fe640 1 -- 172.21.15.120:0/324971325 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578e8098420 con 0x5578e8090240 2024-09-19T09:07:39.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.562+0000 7f17ef7fe640 1 --2- 172.21.15.120:0/324971325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578e8090240 0x5578e8097c20 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f17e8009430 tx=0x7f17e8004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97f0a8fe1242d36 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:39.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.562+0000 7f17d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:07:39.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.562+0000 7f17d2ff5640 1 -- 172.21.15.120:0/324971325 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8058070 con 0x5578e8090240 2024-09-19T09:07:39.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.562+0000 7f17d2ff5640 1 -- 172.21.15.120:0/324971325 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f17e8031bb0 con 0x5578e8090240 2024-09-19T09:07:39.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.562+0000 7f17f7709740 1 -- 172.21.15.120:0/324971325 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17e409ab90 con 0x5578e8090240 2024-09-19T09:07:39.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.562+0000 7f17d2ff5640 1 -- 172.21.15.120:0/324971325 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f17e8064050 con 0x5578e8090240 2024-09-19T09:07:39.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.562+0000 7f17f7709740 10 client.17148 fetch_fsmap learned FSMap version 30 2024-09-19T09:07:39.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.562+0000 7f17f7709740 1 -- 172.21.15.120:0/324971325 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5578e810f560 con 0x5578e8090240 2024-09-19T09:07:39.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.563+0000 7f17d2ff5640 1 -- 172.21.15.120:0/324971325 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17e409ab90 con 0x5578e8090240 2024-09-19T09:07:39.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.563+0000 7f17d2ff5640 1 -- 172.21.15.120:0/324971325 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f17e802fcb0 con 0x5578e8090240 2024-09-19T09:07:39.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.563+0000 7f17f7709740 10 client.17148 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:07:39.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.563+0000 7f17f7709740 10 client.17148 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:07:39.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.563+0000 7f17f7709740 20 client.17148 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:39.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.563+0000 7f17f7709740 1 --2- 172.21.15.120:0/324971325 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5578e809fb90 0x5578e8111a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:39.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.563+0000 7f17f7709740 4 client.17148 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "27b8e0", "release": true, "await": true}] 2024-09-19T09:07:39.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.563+0000 7f17f7709740 1 -- 172.21.15.120:0/324971325 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "set_id": "27b8e0", "release": true, "await": true}) -- 0x5578e7ff20a0 con 0x5578e809fb90 2024-09-19T09:07:39.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.563+0000 7f17eeffd640 1 --2- 172.21.15.120:0/324971325 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5578e809fb90 0x5578e8111a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:39.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.564+0000 7f17d2ff5640 10 client.17148 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:39.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.564+0000 7f17eeffd640 1 --2- 172.21.15.120:0/324971325 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5578e809fb90 0x5578e8111a40 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ed9f01570dcd533 server_cookie=1b67e40a95b9e9f2 in_seq=0 out_seq=0 2024-09-19T09:07:39.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.577+0000 7f17d2ff5640 1 -- 172.21.15.120:0/324971325 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5578e7ff20a0 con 0x5578e809fb90 2024-09-19T09:07:39.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.577+0000 7f17d2ff5640 10 client.17148 handle_command_reply: tid=0 2024-09-19T09:07:39.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.577+0000 7f17f7709740 1 client.17148 shutdown 2024-09-19T09:07:39.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.577+0000 7f17f7709740 1 -- 172.21.15.120:0/324971325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578e8090240 msgr2=0x5578e8097c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:39.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.577+0000 7f17f7709740 1 --2- 172.21.15.120:0/324971325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578e8090240 0x5578e8097c20 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f17e8009430 tx=0x7f17e8004d70 comp rx=0 tx=0).stop 2024-09-19T09:07:39.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.577+0000 7f17f7709740 1 -- 172.21.15.120:0/324971325 shutdown_connections 2024-09-19T09:07:39.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.577+0000 7f17f7709740 1 --2- 172.21.15.120:0/324971325 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5578e809fb90 0x5578e8111a40 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.577+0000 7f17f7709740 1 --2- 172.21.15.120:0/324971325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578e8090240 0x5578e8097c20 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.577+0000 7f17f7709740 1 --2- 172.21.15.120:0/324971325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578e8085980 0x5578e80975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.577+0000 7f17f7709740 1 --2- 172.21.15.120:0/324971325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578e8085060 0x5578e8097060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.577+0000 7f17f7709740 1 -- 172.21.15.120:0/324971325 >> 172.21.15.120:0/324971325 conn(0x5578e80815f0 msgr2=0x5578e8080750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:39.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.577+0000 7f17d2ff5640 0 client.17148 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:39.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.577+0000 7f17f7709740 1 -- 172.21.15.120:0/324971325 shutdown_connections 2024-09-19T09:07:39.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.577+0000 7f17f7709740 1 -- 172.21.15.120:0/324971325 wait complete. 2024-09-19T09:07:39.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.577+0000 7f17f7709740 20 client.17148 trim_cache size 0 max 16384 2024-09-19T09:07:39.579 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:07:39.579 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 30, 2024-09-19T09:07:39.579 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:07:39.579 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 20, 2024-09-19T09:07:39.579 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:07:39.579 INFO:teuthology.orchestra.run.smithi120.stdout: "27b8e0": { 2024-09-19T09:07:39.580 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 20, 2024-09-19T09:07:39.580 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:07:39.580 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:07:39.580 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:07:39.580 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:07:39.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:07:39.580 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:07:39.580 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 149.8, 2024-09-19T09:07:39.580 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:07:39.580 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:07:39.580 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:07:39.580 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:07:39.580 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:07:39.580 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:07:39.581 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:07:39.581 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:07:39.581 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:07:39.581 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:07:39.581 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:07:39.581 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:07:39.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.578+0000 7f17f5330640 1 -- 172.21.15.120:0/2243834604 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f17d80a3240 msgr2=0x7f17d80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:39.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.578+0000 7f17f5330640 1 --2- 172.21.15.120:0/2243834604 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f17d80a3240 0x7f17d80a5700 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f17e8049000 tx=0x7f17e804a000 comp rx=0 tx=0).stop 2024-09-19T09:07:39.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.578+0000 7f17f5330640 1 -- 172.21.15.120:0/2243834604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17f0102790 msgr2=0x7f17f0198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:39.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.578+0000 7f17f5330640 1 --2- 172.21.15.120:0/2243834604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17f0102790 0x7f17f0198aa0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f17e400b4a0 tx=0x7f17e400b960 comp rx=0 tx=0).stop 2024-09-19T09:07:39.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.578+0000 7f17f5330640 1 -- 172.21.15.120:0/2243834604 shutdown_connections 2024-09-19T09:07:39.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.578+0000 7f17f5330640 1 --2- 172.21.15.120:0/2243834604 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f17d80a3240 0x7f17d80a5700 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.578+0000 7f17f5330640 1 --2- 172.21.15.120:0/2243834604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17f0106f10 0x7f17f019ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.578+0000 7f17f5330640 1 --2- 172.21.15.120:0/2243834604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17f0102790 0x7f17f0198aa0 secure :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f17e400b4a0 tx=0x7f17e400b960 comp rx=0 tx=0).stop 2024-09-19T09:07:39.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.578+0000 7f17f5330640 1 --2- 172.21.15.120:0/2243834604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17f0101de0 0x7f17f0198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:39.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.578+0000 7f17f5330640 1 -- 172.21.15.120:0/2243834604 >> 172.21.15.120:0/2243834604 conn(0x7f17f00fd8c0 msgr2=0x7f17f00fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:39.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.578+0000 7f17f5330640 1 -- 172.21.15.120:0/2243834604 shutdown_connections 2024-09-19T09:07:39.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:39.579+0000 7f17f5330640 1 -- 172.21.15.120:0/2243834604 wait complete. 2024-09-19T09:07:39.592 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:07:40.047 INFO:tasks.workunit.client.0.smithi120.stdout: 1 361443 7.59 MB/sec execute 95 sec latency 31618.998 ms 2024-09-19T09:07:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.074+0000 7f5cc6d9f640 1 -- 172.21.15.120:0/2886037613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc01035c0 msgr2=0x7f5cc0109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.074+0000 7f5cc6d9f640 1 --2- 172.21.15.120:0/2886037613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc01035c0 0x7f5cc0109e50 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009ca0 tx=0x7f5cb802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.074+0000 7f5cc6d9f640 1 -- 172.21.15.120:0/2886037613 shutdown_connections 2024-09-19T09:07:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.074+0000 7f5cc6d9f640 1 --2- 172.21.15.120:0/2886037613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc01035c0 0x7f5cc0109e50 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.074+0000 7f5cc6d9f640 1 --2- 172.21.15.120:0/2886037613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cc0102c00 0x7f5cc0103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.074+0000 7f5cc6d9f640 1 --2- 172.21.15.120:0/2886037613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cc00ffc30 0x7f5cc0100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.074+0000 7f5cc6d9f640 1 -- 172.21.15.120:0/2886037613 >> 172.21.15.120:0/2886037613 conn(0x7f5cc00fb750 msgr2=0x7f5cc00fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.074+0000 7f5cc6d9f640 1 -- 172.21.15.120:0/2886037613 shutdown_connections 2024-09-19T09:07:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.074+0000 7f5cc6d9f640 1 -- 172.21.15.120:0/2886037613 wait complete. 2024-09-19T09:07:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.075+0000 7f5cc6d9f640 1 Processor -- start 2024-09-19T09:07:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.075+0000 7f5cc6d9f640 1 -- start start 2024-09-19T09:07:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.075+0000 7f5cc6d9f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cc00ffc30 0x7f5cc0197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.075+0000 7f5cc6d9f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc0102c00 0x7f5cc01981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.075+0000 7f5cc6d9f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cc01035c0 0x7f5cc019f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.075+0000 7f5cc6d9f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc010d600 con 0x7f5cc0102c00 2024-09-19T09:07:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.075+0000 7f5cc6d9f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc010d480 con 0x7f5cc01035c0 2024-09-19T09:07:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.075+0000 7f5cc6d9f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc010d780 con 0x7f5cc00ffc30 2024-09-19T09:07:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.075+0000 7f5cc559c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc0102c00 0x7f5cc01981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.075+0000 7f5cc559c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc0102c00 0x7f5cc01981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56712/0 (socket says 172.21.15.120:56712) 2024-09-19T09:07:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.075+0000 7f5cc5d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cc00ffc30 0x7f5cc0197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.075+0000 7f5cc559c640 1 -- 172.21.15.120:0/222843924 learned_addr learned my addr 172.21.15.120:0/222843924 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.075+0000 7f5cc659e640 1 --2- 172.21.15.120:0/222843924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cc01035c0 0x7f5cc019f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.075+0000 7f5cc559c640 1 -- 172.21.15.120:0/222843924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cc00ffc30 msgr2=0x7f5cc0197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.075+0000 7f5cc559c640 1 --2- 172.21.15.120:0/222843924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cc00ffc30 0x7f5cc0197c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.075+0000 7f5cc559c640 1 -- 172.21.15.120:0/222843924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cc01035c0 msgr2=0x7f5cc019f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.075+0000 7f5cc559c640 1 --2- 172.21.15.120:0/222843924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cc01035c0 0x7f5cc019f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.075+0000 7f5cc559c640 1 -- 172.21.15.120:0/222843924 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc019f870 con 0x7f5cc0102c00 2024-09-19T09:07:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.075+0000 7f5cc5d9d640 1 --2- 172.21.15.120:0/222843924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cc00ffc30 0x7f5cc0197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:07:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.075+0000 7f5cc559c640 1 --2- 172.21.15.120:0/222843924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc0102c00 0x7f5cc01981a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f5cb000e9f0 tx=0x7f5cb000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.076+0000 7f5cb6ffd640 1 -- 172.21.15.120:0/222843924 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb000cd80 con 0x7f5cc0102c00 2024-09-19T09:07:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.076+0000 7f5cc6d9f640 1 -- 172.21.15.120:0/222843924 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc019fb60 con 0x7f5cc0102c00 2024-09-19T09:07:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.076+0000 7f5cb6ffd640 1 -- 172.21.15.120:0/222843924 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cb0010910 con 0x7f5cc0102c00 2024-09-19T09:07:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.076+0000 7f5cb6ffd640 1 -- 172.21.15.120:0/222843924 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb00188f0 con 0x7f5cc0102c00 2024-09-19T09:07:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.076+0000 7f5cc6d9f640 1 -- 172.21.15.120:0/222843924 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc01a00b0 con 0x7f5cc0102c00 2024-09-19T09:07:40.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.078+0000 7f5cb6ffd640 1 -- 172.21.15.120:0/222843924 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5cb0010ac0 con 0x7f5cc0102c00 2024-09-19T09:07:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.078+0000 7f5ccae72740 1 Processor -- start 2024-09-19T09:07:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.079+0000 7f5ccae72740 1 -- start start 2024-09-19T09:07:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.079+0000 7f5ccae72740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaf303cb70 0x55aaf303cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.079+0000 7f5ccae72740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aaf303d4b0 0x55aaf3041f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.079+0000 7f5ccae72740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaf3042540 0x55aaf3044a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.079+0000 7f5ccae72740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55aaf2fa02f0 con 0x55aaf303d4b0 2024-09-19T09:07:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.079+0000 7f5ccae72740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55aaf2f48270 con 0x55aaf3042540 2024-09-19T09:07:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.079+0000 7f5ccae72740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55aaf2f9c2b0 con 0x55aaf303cb70 2024-09-19T09:07:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.079+0000 7f5cc5d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aaf303d4b0 0x55aaf3041f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.079+0000 7f5cc559c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaf3042540 0x55aaf3044a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.079+0000 7f5cc5d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aaf303d4b0 0x55aaf3041f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56722/0 (socket says 172.21.15.120:56722) 2024-09-19T09:07:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.079+0000 7f5cc5d9d640 1 -- 172.21.15.120:0/2735371002 learned_addr learned my addr 172.21.15.120:0/2735371002 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.079+0000 7f5cc659e640 1 --2- 172.21.15.120:0/2735371002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaf303cb70 0x55aaf303cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.079+0000 7f5cb6ffd640 1 --2- 172.21.15.120:0/222843924 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5cac0a3240 0x7f5cac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.079+0000 7f5cc5d9d640 1 -- 172.21.15.120:0/2735371002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaf303cb70 msgr2=0x55aaf303cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.079+0000 7f5cc5d9d640 1 --2- 172.21.15.120:0/2735371002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaf303cb70 0x55aaf303cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.079+0000 7f5cc5d9d640 1 -- 172.21.15.120:0/2735371002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaf3042540 msgr2=0x55aaf3044a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.079+0000 7f5cc5d9d640 1 --2- 172.21.15.120:0/2735371002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaf3042540 0x55aaf3044a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.079+0000 7f5cc5d9d640 1 -- 172.21.15.120:0/2735371002 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaf2f47340 con 0x55aaf303d4b0 2024-09-19T09:07:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.079+0000 7f5cc559c640 1 --2- 172.21.15.120:0/2735371002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaf3042540 0x55aaf3044a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:07:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.079+0000 7f5cc659e640 1 --2- 172.21.15.120:0/222843924 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5cac0a3240 0x7f5cac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.079+0000 7f5cb6ffd640 1 -- 172.21.15.120:0/222843924 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f5cb00cfd30 con 0x7f5cc0102c00 2024-09-19T09:07:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.079+0000 7f5cc5d9d640 1 --2- 172.21.15.120:0/2735371002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aaf303d4b0 0x55aaf3041f70 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc0114f0 tx=0x7f5cbc0119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e011376e945b496a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.080+0000 7f5cabfff640 1 -- 172.21.15.120:0/2735371002 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc01e070 con 0x55aaf303d4b0 2024-09-19T09:07:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.080+0000 7f5cabfff640 1 -- 172.21.15.120:0/2735371002 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cbc00fb00 con 0x55aaf303d4b0 2024-09-19T09:07:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.080+0000 7f5cabfff640 1 -- 172.21.15.120:0/2735371002 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc019bd0 con 0x55aaf303d4b0 2024-09-19T09:07:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.081+0000 7f5ccae72740 1 -- 172.21.15.120:0/2735371002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aaf303d4b0 msgr2=0x55aaf3041f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.081+0000 7f5cc659e640 1 --2- 172.21.15.120:0/222843924 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5cac0a3240 0x7f5cac0a5700 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8005e30 tx=0x7f5cb80095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.081+0000 7f5ccae72740 1 --2- 172.21.15.120:0/2735371002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aaf303d4b0 0x55aaf3041f70 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc0114f0 tx=0x7f5cbc0119b0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.081+0000 7f5ccae72740 1 -- 172.21.15.120:0/2735371002 shutdown_connections 2024-09-19T09:07:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.081+0000 7f5ccae72740 1 --2- 172.21.15.120:0/2735371002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaf3042540 0x55aaf3044a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.081+0000 7f5ccae72740 1 --2- 172.21.15.120:0/2735371002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aaf303d4b0 0x55aaf3041f70 secure :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc0114f0 tx=0x7f5cbc0119b0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.081+0000 7f5ccae72740 1 --2- 172.21.15.120:0/2735371002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaf303cb70 0x55aaf303cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.081+0000 7f5ccae72740 1 -- 172.21.15.120:0/2735371002 >> 172.21.15.120:0/2735371002 conn(0x55aaf30381e0 msgr2=0x55aaf30385e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.081+0000 7f5ccae72740 1 -- 172.21.15.120:0/2735371002 shutdown_connections 2024-09-19T09:07:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.081+0000 7f5ccae72740 1 -- 172.21.15.120:0/2735371002 wait complete. 2024-09-19T09:07:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.081+0000 7f5ccae72740 1 -- start start 2024-09-19T09:07:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.081+0000 7f5ccae72740 1 Processor -- start 2024-09-19T09:07:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.082+0000 7f5ccae72740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaf303cb70 0x55aaf3047090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.082+0000 7f5ccae72740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aaf3042540 0x55aaf30475d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.082+0000 7f5ccae72740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaf304c590 0x55aaf304e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.082+0000 7f5ccae72740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55aaf3045510 con 0x55aaf3042540 2024-09-19T09:07:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.082+0000 7f5ccae72740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55aaf2f29290 con 0x55aaf304c590 2024-09-19T09:07:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.082+0000 7f5ccae72740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55aaf3045690 con 0x55aaf303cb70 2024-09-19T09:07:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.082+0000 7f5ccae72740 10 client.0 _ll_register_callbacks cb 0x55aaf2f5a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.082+0000 7f5cc559c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aaf3042540 0x55aaf30475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.082+0000 7f5cc559c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aaf3042540 0x55aaf30475d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56738/0 (socket says 172.21.15.120:56738) 2024-09-19T09:07:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.082+0000 7f5cc659e640 1 --2- 172.21.15.120:0/933286730 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaf304c590 0x55aaf304e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.082+0000 7f5cc559c640 1 -- 172.21.15.120:0/933286730 learned_addr learned my addr 172.21.15.120:0/933286730 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.082+0000 7f5cc5d9d640 1 --2- 172.21.15.120:0/933286730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaf303cb70 0x55aaf3047090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.082+0000 7f5cc659e640 1 -- 172.21.15.120:0/933286730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaf303cb70 msgr2=0x55aaf3047090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.082+0000 7f5cc659e640 1 --2- 172.21.15.120:0/933286730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaf303cb70 0x55aaf3047090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.082+0000 7f5cc659e640 1 -- 172.21.15.120:0/933286730 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aaf3042540 msgr2=0x55aaf30475d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.082+0000 7f5cc659e640 1 --2- 172.21.15.120:0/933286730 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aaf3042540 0x55aaf30475d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.082+0000 7f5cc659e640 1 -- 172.21.15.120:0/933286730 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaf2cb3210 con 0x55aaf304c590 2024-09-19T09:07:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.082+0000 7f5cc559c640 1 --2- 172.21.15.120:0/933286730 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aaf3042540 0x55aaf30475d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:07:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.083+0000 7f5cc659e640 1 --2- 172.21.15.120:0/933286730 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaf304c590 0x55aaf304e980 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8031ae0 tx=0x7f5cb802a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d679f3e0670fcce2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.083+0000 7f5ca97fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:07:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.083+0000 7f5ca97fa640 1 -- 172.21.15.120:0/933286730 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb802feb0 con 0x55aaf304c590 2024-09-19T09:07:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.083+0000 7f5ccae72740 1 -- 172.21.15.120:0/933286730 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cbc00fb00 con 0x55aaf304c590 2024-09-19T09:07:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.083+0000 7f5ca97fa640 1 -- 172.21.15.120:0/933286730 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f5cb80307d0 con 0x55aaf304c590 2024-09-19T09:07:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.083+0000 7f5ca97fa640 1 -- 172.21.15.120:0/933286730 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5cb80309d0 con 0x55aaf304c590 2024-09-19T09:07:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.083+0000 7f5ccae72740 10 client.26459 fetch_fsmap learned FSMap version 30 2024-09-19T09:07:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.083+0000 7f5ccae72740 1 -- 172.21.15.120:0/933286730 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55aaf2d7f620 con 0x55aaf304c590 2024-09-19T09:07:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.083+0000 7f5ca97fa640 1 -- 172.21.15.120:0/933286730 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cbc00fb00 con 0x55aaf304c590 2024-09-19T09:07:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.083+0000 7f5ca97fa640 1 -- 172.21.15.120:0/933286730 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5cb8045070 con 0x55aaf304c590 2024-09-19T09:07:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.083+0000 7f5ccae72740 10 client.26459 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:07:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.083+0000 7f5ccae72740 10 client.26459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:07:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.083+0000 7f5ccae72740 20 client.26459 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.083+0000 7f5ccae72740 1 --2- 172.21.15.120:0/933286730 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55aaf30534f0 0x55aaf3055920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.084+0000 7f5ccae72740 4 client.26459 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:07:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.084+0000 7f5ccae72740 1 -- 172.21.15.120:0/933286730 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aaf2fada20 con 0x55aaf30534f0 2024-09-19T09:07:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.084+0000 7f5cc5d9d640 1 --2- 172.21.15.120:0/933286730 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55aaf30534f0 0x55aaf3055920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.084+0000 7f5cc5d9d640 1 --2- 172.21.15.120:0/933286730 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55aaf30534f0 0x55aaf3055920 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6e89a9f265fbd8a server_cookie=31935c8b3cf4d8fa in_seq=0 out_seq=0 2024-09-19T09:07:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.084+0000 7f5ca97fa640 10 client.26459 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.085+0000 7f5ca97fa640 1 -- 172.21.15.120:0/933286730 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aaf2fada20 con 0x55aaf30534f0 2024-09-19T09:07:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.085+0000 7f5ca97fa640 10 client.26459 handle_command_reply: tid=0 2024-09-19T09:07:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.085+0000 7f5ccae72740 1 client.26459 shutdown 2024-09-19T09:07:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.085+0000 7f5ccae72740 1 -- 172.21.15.120:0/933286730 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaf304c590 msgr2=0x55aaf304e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.085+0000 7f5ccae72740 1 --2- 172.21.15.120:0/933286730 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaf304c590 0x55aaf304e980 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8031ae0 tx=0x7f5cb802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.086+0000 7f5ccae72740 1 -- 172.21.15.120:0/933286730 shutdown_connections 2024-09-19T09:07:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.086+0000 7f5ccae72740 1 --2- 172.21.15.120:0/933286730 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55aaf30534f0 0x55aaf3055920 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.086+0000 7f5ccae72740 1 --2- 172.21.15.120:0/933286730 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaf304c590 0x55aaf304e980 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.086+0000 7f5ca97fa640 0 client.26459 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.086+0000 7f5ccae72740 1 --2- 172.21.15.120:0/933286730 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aaf3042540 0x55aaf30475d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.086+0000 7f5ccae72740 1 --2- 172.21.15.120:0/933286730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaf303cb70 0x55aaf3047090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.086+0000 7f5ccae72740 1 -- 172.21.15.120:0/933286730 >> 172.21.15.120:0/933286730 conn(0x55aaf30381e0 msgr2=0x55aaf30399f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.086+0000 7f5ccae72740 1 -- 172.21.15.120:0/933286730 shutdown_connections 2024-09-19T09:07:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.086+0000 7f5ccae72740 1 -- 172.21.15.120:0/933286730 wait complete. 2024-09-19T09:07:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.086+0000 7f5ccae72740 20 client.26459 trim_cache size 0 max 16384 2024-09-19T09:07:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.101+0000 7f5ccae72740 1 Processor -- start 2024-09-19T09:07:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.101+0000 7f5ccae72740 1 -- start start 2024-09-19T09:07:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.101+0000 7f5ccae72740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaf303cb70 0x55aaf303cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.101+0000 7f5ccae72740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaf3042540 0x55aaf3045a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.101+0000 7f5ccae72740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aaf304c590 0x55aaf30385c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.101+0000 7f5ccae72740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc01a8e00 con 0x55aaf304c590 2024-09-19T09:07:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.101+0000 7f5ccae72740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc01aa940 con 0x55aaf3042540 2024-09-19T09:07:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.101+0000 7f5ccae72740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc01a94d0 con 0x55aaf303cb70 2024-09-19T09:07:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.102+0000 7f5cc659e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aaf304c590 0x55aaf30385c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.102+0000 7f5cc659e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aaf304c590 0x55aaf30385c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56746/0 (socket says 172.21.15.120:56746) 2024-09-19T09:07:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.102+0000 7f5cc659e640 1 -- 172.21.15.120:0/3792339647 learned_addr learned my addr 172.21.15.120:0/3792339647 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.102+0000 7f5cc559c640 1 --2- 172.21.15.120:0/3792339647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaf3042540 0x55aaf3045a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.102+0000 7f5cc5d9d640 1 --2- 172.21.15.120:0/3792339647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaf303cb70 0x55aaf303cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.102+0000 7f5cc659e640 1 -- 172.21.15.120:0/3792339647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaf303cb70 msgr2=0x55aaf303cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.102+0000 7f5cc659e640 1 --2- 172.21.15.120:0/3792339647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaf303cb70 0x55aaf303cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.102+0000 7f5cc659e640 1 -- 172.21.15.120:0/3792339647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaf3042540 msgr2=0x55aaf3045a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.102+0000 7f5cc659e640 1 --2- 172.21.15.120:0/3792339647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaf3042540 0x55aaf3045a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.102+0000 7f5cc659e640 1 -- 172.21.15.120:0/3792339647 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaf3051a40 con 0x55aaf304c590 2024-09-19T09:07:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.102+0000 7f5cc5d9d640 1 --2- 172.21.15.120:0/3792339647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaf303cb70 0x55aaf303cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:07:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.102+0000 7f5cc659e640 1 --2- 172.21.15.120:0/3792339647 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aaf304c590 0x55aaf30385c0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8052ac0 tx=0x7f5cb8030960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e94f23ed7046ac9b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.102+0000 7f5cab7fe640 1 -- 172.21.15.120:0/3792339647 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb802fe30 con 0x55aaf304c590 2024-09-19T09:07:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.102+0000 7f5cab7fe640 1 -- 172.21.15.120:0/3792339647 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cb8061020 con 0x55aaf304c590 2024-09-19T09:07:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.102+0000 7f5cab7fe640 1 -- 172.21.15.120:0/3792339647 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb80526d0 con 0x55aaf304c590 2024-09-19T09:07:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.103+0000 7f5ccae72740 1 -- 172.21.15.120:0/3792339647 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aaf304c590 msgr2=0x55aaf30385c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.103+0000 7f5ccae72740 1 --2- 172.21.15.120:0/3792339647 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aaf304c590 0x55aaf30385c0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8052ac0 tx=0x7f5cb8030960 comp rx=0 tx=0).stop 2024-09-19T09:07:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.103+0000 7f5ccae72740 1 -- 172.21.15.120:0/3792339647 shutdown_connections 2024-09-19T09:07:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.103+0000 7f5ccae72740 1 --2- 172.21.15.120:0/3792339647 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aaf304c590 0x55aaf30385c0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.103+0000 7f5ccae72740 1 --2- 172.21.15.120:0/3792339647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaf3042540 0x55aaf3045a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.103+0000 7f5ccae72740 1 --2- 172.21.15.120:0/3792339647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaf303cb70 0x55aaf303cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.103+0000 7f5ccae72740 1 -- 172.21.15.120:0/3792339647 >> 172.21.15.120:0/3792339647 conn(0x55aaf30381e0 msgr2=0x55aaf30465f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.103+0000 7f5ccae72740 1 -- 172.21.15.120:0/3792339647 shutdown_connections 2024-09-19T09:07:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.103+0000 7f5ccae72740 1 -- 172.21.15.120:0/3792339647 wait complete. 2024-09-19T09:07:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.103+0000 7f5ccae72740 1 -- start start 2024-09-19T09:07:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.103+0000 7f5ccae72740 1 Processor -- start 2024-09-19T09:07:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.104+0000 7f5ccae72740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaf303cb70 0x55aaf30649a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.104+0000 7f5ccae72740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aaf3042540 0x55aaf3064ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.104+0000 7f5ccae72740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaf304c590 0x55aaf3065ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.104+0000 7f5ccae72740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc01aa270 con 0x55aaf3042540 2024-09-19T09:07:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.104+0000 7f5ccae72740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc01a9ba0 con 0x55aaf304c590 2024-09-19T09:07:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.104+0000 7f5ccae72740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc01a6dc0 con 0x55aaf303cb70 2024-09-19T09:07:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.104+0000 7f5ccae72740 10 client.0 _ll_register_callbacks cb 0x7f5cb8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.104+0000 7f5cc559c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aaf3042540 0x55aaf3064ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.104+0000 7f5cc559c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aaf3042540 0x55aaf3064ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56752/0 (socket says 172.21.15.120:56752) 2024-09-19T09:07:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.104+0000 7f5cc559c640 1 -- 172.21.15.120:0/3912741603 learned_addr learned my addr 172.21.15.120:0/3912741603 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.104+0000 7f5cc5d9d640 1 --2- 172.21.15.120:0/3912741603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaf303cb70 0x55aaf30649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.104+0000 7f5cc659e640 1 --2- 172.21.15.120:0/3912741603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaf304c590 0x55aaf3065ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.104+0000 7f5cc559c640 1 -- 172.21.15.120:0/3912741603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaf303cb70 msgr2=0x55aaf30649a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.104+0000 7f5cc559c640 1 --2- 172.21.15.120:0/3912741603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaf303cb70 0x55aaf30649a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.104+0000 7f5cc559c640 1 -- 172.21.15.120:0/3912741603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaf304c590 msgr2=0x55aaf3065ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.104+0000 7f5cc559c640 1 --2- 172.21.15.120:0/3912741603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaf304c590 0x55aaf3065ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.104+0000 7f5cc559c640 1 -- 172.21.15.120:0/3912741603 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaf3066500 con 0x55aaf3042540 2024-09-19T09:07:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.104+0000 7f5cc559c640 1 --2- 172.21.15.120:0/3912741603 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aaf3042540 0x55aaf3064ee0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f5cb00987d0 tx=0x7f5cb0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1207e3053fa0e66 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.104+0000 7f5ca97fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:07:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.104+0000 7f5ca97fa640 1 -- 172.21.15.120:0/3912741603 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb009c400 con 0x55aaf3042540 2024-09-19T09:07:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.104+0000 7f5ca97fa640 1 -- 172.21.15.120:0/3912741603 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f5cb0010430 con 0x55aaf3042540 2024-09-19T09:07:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.104+0000 7f5ccae72740 1 -- 172.21.15.120:0/3912741603 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cb8061020 con 0x55aaf3042540 2024-09-19T09:07:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.105+0000 7f5ca97fa640 1 -- 172.21.15.120:0/3912741603 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5cb00a0700 con 0x55aaf3042540 2024-09-19T09:07:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.105+0000 7f5ccae72740 10 client.17166 fetch_fsmap learned FSMap version 30 2024-09-19T09:07:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.105+0000 7f5ccae72740 1 -- 172.21.15.120:0/3912741603 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55aaf3067880 con 0x55aaf3042540 2024-09-19T09:07:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.105+0000 7f5ca97fa640 1 -- 172.21.15.120:0/3912741603 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cb8061020 con 0x55aaf3042540 2024-09-19T09:07:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.105+0000 7f5ca97fa640 1 -- 172.21.15.120:0/3912741603 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5cb00bc270 con 0x55aaf3042540 2024-09-19T09:07:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.105+0000 7f5ccae72740 10 client.17166 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:07:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.105+0000 7f5ccae72740 10 client.17166 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:07:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.105+0000 7f5ccae72740 20 client.17166 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.105+0000 7f5ccae72740 1 --2- 172.21.15.120:0/3912741603 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55aaf30be500 0x55aaf30c08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.105+0000 7f5ccae72740 4 client.17166 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:07:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.105+0000 7f5ccae72740 1 -- 172.21.15.120:0/3912741603 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aaf2fa7d70 con 0x55aaf30be500 2024-09-19T09:07:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.105+0000 7f5cc5d9d640 1 --2- 172.21.15.120:0/3912741603 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55aaf30be500 0x55aaf30c08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.106+0000 7f5ca97fa640 10 client.17166 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.106+0000 7f5cc5d9d640 1 --2- 172.21.15.120:0/3912741603 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55aaf30be500 0x55aaf30c08f0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c652498cef6fc5ad server_cookie=a74b797afafdc759 in_seq=0 out_seq=0 2024-09-19T09:07:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.160+0000 7f5ca97fa640 1 -- 172.21.15.120:0/3912741603 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aaf2fa7d70 con 0x55aaf30be500 2024-09-19T09:07:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.160+0000 7f5ca97fa640 10 client.17166 handle_command_reply: tid=0 2024-09-19T09:07:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.160+0000 7f5ccae72740 1 client.17166 shutdown 2024-09-19T09:07:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.161+0000 7f5ccae72740 1 -- 172.21.15.120:0/3912741603 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aaf3042540 msgr2=0x55aaf3064ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.161+0000 7f5ccae72740 1 --2- 172.21.15.120:0/3912741603 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aaf3042540 0x55aaf3064ee0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f5cb00987d0 tx=0x7f5cb0009310 comp rx=0 tx=0).stop 2024-09-19T09:07:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.161+0000 7f5ccae72740 1 -- 172.21.15.120:0/3912741603 shutdown_connections 2024-09-19T09:07:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.161+0000 7f5ccae72740 1 --2- 172.21.15.120:0/3912741603 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55aaf30be500 0x55aaf30c08f0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.161+0000 7f5ccae72740 1 --2- 172.21.15.120:0/3912741603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aaf304c590 0x55aaf3065ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.161+0000 7f5ccae72740 1 --2- 172.21.15.120:0/3912741603 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aaf3042540 0x55aaf3064ee0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.161+0000 7f5ccae72740 1 --2- 172.21.15.120:0/3912741603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aaf303cb70 0x55aaf30649a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.161+0000 7f5ca97fa640 0 client.17166 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.161+0000 7f5ccae72740 1 -- 172.21.15.120:0/3912741603 >> 172.21.15.120:0/3912741603 conn(0x55aaf30381e0 msgr2=0x55aaf30465c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.161+0000 7f5ccae72740 1 -- 172.21.15.120:0/3912741603 shutdown_connections 2024-09-19T09:07:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.161+0000 7f5ccae72740 1 -- 172.21.15.120:0/3912741603 wait complete. 2024-09-19T09:07:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.161+0000 7f5ccae72740 20 client.17166 trim_cache size 0 max 16384 2024-09-19T09:07:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:07:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:07:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:07:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:07:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.162+0000 7f5cc6d9f640 1 -- 172.21.15.120:0/222843924 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5cac0a3240 msgr2=0x7f5cac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.162+0000 7f5cc6d9f640 1 --2- 172.21.15.120:0/222843924 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5cac0a3240 0x7f5cac0a5700 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8005e30 tx=0x7f5cb80095c0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.162+0000 7f5cc6d9f640 1 -- 172.21.15.120:0/222843924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc0102c00 msgr2=0x7f5cc01981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.162+0000 7f5cc6d9f640 1 --2- 172.21.15.120:0/222843924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc0102c00 0x7f5cc01981a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f5cb000e9f0 tx=0x7f5cb000eeb0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.162+0000 7f5cc6d9f640 1 -- 172.21.15.120:0/222843924 shutdown_connections 2024-09-19T09:07:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.162+0000 7f5cc6d9f640 1 --2- 172.21.15.120:0/222843924 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5cac0a3240 0x7f5cac0a5700 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.162+0000 7f5cc6d9f640 1 --2- 172.21.15.120:0/222843924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cc01035c0 0x7f5cc019f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.162+0000 7f5cc6d9f640 1 --2- 172.21.15.120:0/222843924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc0102c00 0x7f5cc01981a0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.162+0000 7f5cc6d9f640 1 --2- 172.21.15.120:0/222843924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cc00ffc30 0x7f5cc0197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.162+0000 7f5cc6d9f640 1 -- 172.21.15.120:0/222843924 >> 172.21.15.120:0/222843924 conn(0x7f5cc00fb750 msgr2=0x7f5cc0105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.162+0000 7f5cc6d9f640 1 -- 172.21.15.120:0/222843924 shutdown_connections 2024-09-19T09:07:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:40.162+0000 7f5cc6d9f640 1 -- 172.21.15.120:0/222843924 wait complete. 2024-09-19T09:07:40.365 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '27b8e0', total seconds elapsed: 35.5 2024-09-19T09:07:40.365 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.8 2024-09-19T09:07:40.366 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:07:40.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.883+0000 7fe930850640 1 -- 172.21.15.120:0/3485339924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe92c102db0 msgr2=0x7fe92c103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:40.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.883+0000 7fe930850640 1 --2- 172.21.15.120:0/3485339924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe92c102db0 0x7fe92c103190 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fe924009930 tx=0x7fe92402a420 comp rx=0 tx=0).stop 2024-09-19T09:07:40.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.884+0000 7fe930850640 1 -- 172.21.15.120:0/3485339924 shutdown_connections 2024-09-19T09:07:40.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.884+0000 7fe930850640 1 --2- 172.21.15.120:0/3485339924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe92c1082e0 0x7fe92c10ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.884+0000 7fe930850640 1 --2- 172.21.15.120:0/3485339924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe92c103760 0x7fe92c107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.884+0000 7fe930850640 1 --2- 172.21.15.120:0/3485339924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe92c102db0 0x7fe92c103190 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.884+0000 7fe930850640 1 -- 172.21.15.120:0/3485339924 >> 172.21.15.120:0/3485339924 conn(0x7fe92c0fc7d0 msgr2=0x7fe92c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:40.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.884+0000 7fe930850640 1 -- 172.21.15.120:0/3485339924 shutdown_connections 2024-09-19T09:07:40.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.884+0000 7fe930850640 1 -- 172.21.15.120:0/3485339924 wait complete. 2024-09-19T09:07:40.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.884+0000 7fe930850640 1 Processor -- start 2024-09-19T09:07:40.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.884+0000 7fe930850640 1 -- start start 2024-09-19T09:07:40.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.885+0000 7fe930850640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe92c102db0 0x7fe92c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.885+0000 7fe930850640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe92c103760 0x7fe92c198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.885+0000 7fe930850640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe92c1082e0 0x7fe92c19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.885+0000 7fe930850640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c1120b0 con 0x7fe92c1082e0 2024-09-19T09:07:40.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.885+0000 7fe930850640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c111f30 con 0x7fe92c103760 2024-09-19T09:07:40.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.885+0000 7fe930850640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c112230 con 0x7fe92c102db0 2024-09-19T09:07:40.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.885+0000 7fe92ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe92c1082e0 0x7fe92c19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.885+0000 7fe92ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe92c1082e0 0x7fe92c19ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56772/0 (socket says 172.21.15.120:56772) 2024-09-19T09:07:40.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.885+0000 7fe92ad9d640 1 -- 172.21.15.120:0/2600197551 learned_addr learned my addr 172.21.15.120:0/2600197551 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:40.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.885+0000 7fe929d9b640 1 --2- 172.21.15.120:0/2600197551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe92c103760 0x7fe92c198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.885+0000 7fe92a59c640 1 --2- 172.21.15.120:0/2600197551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe92c102db0 0x7fe92c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.885+0000 7fe92ad9d640 1 -- 172.21.15.120:0/2600197551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe92c102db0 msgr2=0x7fe92c198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:40.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.885+0000 7fe92ad9d640 1 --2- 172.21.15.120:0/2600197551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe92c102db0 0x7fe92c198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.885+0000 7fe92ad9d640 1 -- 172.21.15.120:0/2600197551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe92c103760 msgr2=0x7fe92c198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:40.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.885+0000 7fe92ad9d640 1 --2- 172.21.15.120:0/2600197551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe92c103760 0x7fe92c198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.885+0000 7fe92ad9d640 1 -- 172.21.15.120:0/2600197551 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92c19d540 con 0x7fe92c1082e0 2024-09-19T09:07:40.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.885+0000 7fe929d9b640 1 --2- 172.21.15.120:0/2600197551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe92c103760 0x7fe92c198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:07:40.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.885+0000 7fe92ad9d640 1 --2- 172.21.15.120:0/2600197551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe92c1082e0 0x7fe92c19ce60 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00d750 tx=0x7fe91c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:40.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.885+0000 7fe91b7fe640 1 -- 172.21.15.120:0/2600197551 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c017070 con 0x7fe92c1082e0 2024-09-19T09:07:40.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.886+0000 7fe91b7fe640 1 -- 172.21.15.120:0/2600197551 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe91c005500 con 0x7fe92c1082e0 2024-09-19T09:07:40.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.886+0000 7fe930850640 1 -- 172.21.15.120:0/2600197551 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe92c19d830 con 0x7fe92c1082e0 2024-09-19T09:07:40.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.886+0000 7fe91b7fe640 1 -- 172.21.15.120:0/2600197551 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c004d90 con 0x7fe92c1082e0 2024-09-19T09:07:40.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.886+0000 7fe930850640 1 -- 172.21.15.120:0/2600197551 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe92c1a5110 con 0x7fe92c1082e0 2024-09-19T09:07:40.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.888+0000 7fe91b7fe640 1 -- 172.21.15.120:0/2600197551 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe91c004f30 con 0x7fe92c1082e0 2024-09-19T09:07:40.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.888+0000 7fe932c29740 1 Processor -- start 2024-09-19T09:07:40.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.888+0000 7fe932c29740 1 -- start start 2024-09-19T09:07:40.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.888+0000 7fe91b7fe640 1 --2- 172.21.15.120:0/2600197551 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9140a32c0 0x7fe9140a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.889+0000 7fe932c29740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562182c12b40 0x562182c11090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.889+0000 7fe932c29740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562182c115d0 0x562182c11a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.889+0000 7fe932c29740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562182c1c370 0x562182c1e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.889+0000 7fe932c29740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562182b772f0 con 0x562182c1c370 2024-09-19T09:07:40.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.889+0000 7fe932c29740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562182b1ecf0 con 0x562182c12b40 2024-09-19T09:07:40.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.889+0000 7fe932c29740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562182b732b0 con 0x562182c115d0 2024-09-19T09:07:40.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.889+0000 7fe91b7fe640 1 -- 172.21.15.120:0/2600197551 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fe91c0cf110 con 0x7fe92c1082e0 2024-09-19T09:07:40.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.889+0000 7fe929d9b640 1 --2- 172.21.15.120:0/2600197551 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9140a32c0 0x7fe9140a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.889+0000 7fe92a59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562182c12b40 0x562182c11090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.889+0000 7fe92a59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562182c12b40 0x562182c11090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:43198/0 (socket says 172.21.15.120:43198) 2024-09-19T09:07:40.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.889+0000 7fe92a59c640 1 -- 172.21.15.120:0/1938775166 learned_addr learned my addr 172.21.15.120:0/1938775166 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:40.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.889+0000 7fe929d9b640 1 --2- 172.21.15.120:0/1938775166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562182c115d0 0x562182c11a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.889+0000 7fe92ad9d640 1 --2- 172.21.15.120:0/1938775166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562182c1c370 0x562182c1e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.889+0000 7fe92a59c640 1 -- 172.21.15.120:0/1938775166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562182c115d0 msgr2=0x562182c11a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:40.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.889+0000 7fe92a59c640 1 --2- 172.21.15.120:0/1938775166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562182c115d0 0x562182c11a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.889+0000 7fe92a59c640 1 -- 172.21.15.120:0/1938775166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562182c1c370 msgr2=0x562182c1e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:40.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.889+0000 7fe92a59c640 1 --2- 172.21.15.120:0/1938775166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562182c1c370 0x562182c1e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.889+0000 7fe92a59c640 1 -- 172.21.15.120:0/1938775166 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562182b033a0 con 0x562182c12b40 2024-09-19T09:07:40.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.889+0000 7fe929d9b640 1 --2- 172.21.15.120:0/2600197551 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9140a32c0 0x7fe9140a5780 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x562182c0eac0 tx=0x7fe920009990 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:40.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.889+0000 7fe92a59c640 1 --2- 172.21.15.120:0/1938775166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562182c12b40 0x562182c11090 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe92402a940 tx=0x7fe924005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e984b7e46a9fc4a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:40.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.889+0000 7fe918ff9640 1 -- 172.21.15.120:0/1938775166 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924035030 con 0x562182c12b40 2024-09-19T09:07:40.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.890+0000 7fe918ff9640 1 -- 172.21.15.120:0/1938775166 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe924040070 con 0x562182c12b40 2024-09-19T09:07:40.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.890+0000 7fe918ff9640 1 -- 172.21.15.120:0/1938775166 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924031940 con 0x562182c12b40 2024-09-19T09:07:40.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.890+0000 7fe932c29740 1 -- 172.21.15.120:0/1938775166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562182c12b40 msgr2=0x562182c11090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:40.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.890+0000 7fe932c29740 1 --2- 172.21.15.120:0/1938775166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562182c12b40 0x562182c11090 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe92402a940 tx=0x7fe924005c00 comp rx=0 tx=0).stop 2024-09-19T09:07:40.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.890+0000 7fe932c29740 1 -- 172.21.15.120:0/1938775166 shutdown_connections 2024-09-19T09:07:40.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.890+0000 7fe932c29740 1 --2- 172.21.15.120:0/1938775166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562182c1c370 0x562182c1e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.890+0000 7fe932c29740 1 --2- 172.21.15.120:0/1938775166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562182c115d0 0x562182c11a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.890+0000 7fe932c29740 1 --2- 172.21.15.120:0/1938775166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562182c12b40 0x562182c11090 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.890+0000 7fe932c29740 1 -- 172.21.15.120:0/1938775166 >> 172.21.15.120:0/1938775166 conn(0x562182bda310 msgr2=0x562182c0dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:40.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.890+0000 7fe932c29740 1 -- 172.21.15.120:0/1938775166 shutdown_connections 2024-09-19T09:07:40.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.890+0000 7fe932c29740 1 -- 172.21.15.120:0/1938775166 wait complete. 2024-09-19T09:07:40.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.891+0000 7fe932c29740 1 -- start start 2024-09-19T09:07:40.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.891+0000 7fe932c29740 1 Processor -- start 2024-09-19T09:07:40.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.891+0000 7fe932c29740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562182c115d0 0x562182c1bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.891+0000 7fe932c29740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562182c12b40 0x562182c18f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.891+0000 7fe932c29740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562182c1c370 0x562182c194f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.891+0000 7fe932c29740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562182c1f010 con 0x562182c12b40 2024-09-19T09:07:40.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.891+0000 7fe932c29740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562182b32060 con 0x562182c115d0 2024-09-19T09:07:40.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.891+0000 7fe932c29740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562182c1f190 con 0x562182c1c370 2024-09-19T09:07:40.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.891+0000 7fe932c29740 10 client.0 _ll_register_callbacks cb 0x5621828858c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:40.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.891+0000 7fe929d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562182c12b40 0x562182c18f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.891+0000 7fe929d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562182c12b40 0x562182c18f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56792/0 (socket says 172.21.15.120:56792) 2024-09-19T09:07:40.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.891+0000 7fe929d9b640 1 -- 172.21.15.120:0/1460120898 learned_addr learned my addr 172.21.15.120:0/1460120898 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:40.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.891+0000 7fe92a59c640 1 --2- 172.21.15.120:0/1460120898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562182c115d0 0x562182c1bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.891+0000 7fe92ad9d640 1 --2- 172.21.15.120:0/1460120898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562182c1c370 0x562182c194f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.891+0000 7fe929d9b640 1 -- 172.21.15.120:0/1460120898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562182c1c370 msgr2=0x562182c194f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:40.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.891+0000 7fe929d9b640 1 --2- 172.21.15.120:0/1460120898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562182c1c370 0x562182c194f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.891+0000 7fe929d9b640 1 -- 172.21.15.120:0/1460120898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562182c115d0 msgr2=0x562182c1bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:40.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.892+0000 7fe929d9b640 1 --2- 172.21.15.120:0/1460120898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562182c115d0 0x562182c1bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.892+0000 7fe929d9b640 1 -- 172.21.15.120:0/1460120898 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621828e83c0 con 0x562182c12b40 2024-09-19T09:07:40.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.892+0000 7fe929d9b640 1 --2- 172.21.15.120:0/1460120898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562182c12b40 0x562182c18f90 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fe92001f8e0 tx=0x7fe920004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46cc542406200da0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:40.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.892+0000 7fe911ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:07:40.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.892+0000 7fe911ffb640 1 -- 172.21.15.120:0/1460120898 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe920026070 con 0x562182c12b40 2024-09-19T09:07:40.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.892+0000 7fe911ffb640 1 -- 172.21.15.120:0/1460120898 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fe92000e930 con 0x562182c12b40 2024-09-19T09:07:40.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.892+0000 7fe932c29740 1 -- 172.21.15.120:0/1460120898 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe924040070 con 0x562182c12b40 2024-09-19T09:07:40.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.892+0000 7fe911ffb640 1 -- 172.21.15.120:0/1460120898 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe920009e20 con 0x562182c12b40 2024-09-19T09:07:40.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.892+0000 7fe932c29740 10 client.17178 fetch_fsmap learned FSMap version 30 2024-09-19T09:07:40.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.892+0000 7fe932c29740 1 -- 172.21.15.120:0/1460120898 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562182c2aa20 con 0x562182c12b40 2024-09-19T09:07:40.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.892+0000 7fe911ffb640 1 -- 172.21.15.120:0/1460120898 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe924040070 con 0x562182c12b40 2024-09-19T09:07:40.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.892+0000 7fe911ffb640 1 -- 172.21.15.120:0/1460120898 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe92001d040 con 0x562182c12b40 2024-09-19T09:07:40.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.892+0000 7fe932c29740 10 client.17178 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:07:40.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.892+0000 7fe932c29740 10 client.17178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:07:40.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.892+0000 7fe932c29740 20 client.17178 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:40.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.892+0000 7fe932c29740 1 --2- 172.21.15.120:0/1460120898 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562182c2bf80 0x562182c2e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.893+0000 7fe932c29740 4 client.17178 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:07:40.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.893+0000 7fe932c29740 1 -- 172.21.15.120:0/1460120898 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562182b84860 con 0x562182c2bf80 2024-09-19T09:07:40.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.893+0000 7fe92a59c640 1 --2- 172.21.15.120:0/1460120898 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562182c2bf80 0x562182c2e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.893+0000 7fe92a59c640 1 --2- 172.21.15.120:0/1460120898 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562182c2bf80 0x562182c2e3e0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58b6988e14da24a4 server_cookie=fcccfb9077ff7eb7 in_seq=0 out_seq=0 2024-09-19T09:07:40.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.893+0000 7fe911ffb640 10 client.17178 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:40.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.895+0000 7fe911ffb640 1 -- 172.21.15.120:0/1460120898 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562182b84860 con 0x562182c2bf80 2024-09-19T09:07:40.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.895+0000 7fe911ffb640 10 client.17178 handle_command_reply: tid=0 2024-09-19T09:07:40.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.895+0000 7fe932c29740 1 client.17178 shutdown 2024-09-19T09:07:40.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.895+0000 7fe932c29740 1 -- 172.21.15.120:0/1460120898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562182c12b40 msgr2=0x562182c18f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:40.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.895+0000 7fe932c29740 1 --2- 172.21.15.120:0/1460120898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562182c12b40 0x562182c18f90 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fe92001f8e0 tx=0x7fe920004ab0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.895+0000 7fe932c29740 1 -- 172.21.15.120:0/1460120898 shutdown_connections 2024-09-19T09:07:40.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.895+0000 7fe932c29740 1 --2- 172.21.15.120:0/1460120898 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562182c2bf80 0x562182c2e3e0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.895+0000 7fe932c29740 1 --2- 172.21.15.120:0/1460120898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562182c1c370 0x562182c194f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.895+0000 7fe932c29740 1 --2- 172.21.15.120:0/1460120898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562182c12b40 0x562182c18f90 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.895+0000 7fe932c29740 1 --2- 172.21.15.120:0/1460120898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562182c115d0 0x562182c1bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.895+0000 7fe932c29740 1 -- 172.21.15.120:0/1460120898 >> 172.21.15.120:0/1460120898 conn(0x562182bda310 msgr2=0x562182c13960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:40.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.895+0000 7fe911ffb640 0 client.17178 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:40.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.895+0000 7fe932c29740 1 -- 172.21.15.120:0/1460120898 shutdown_connections 2024-09-19T09:07:40.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.895+0000 7fe932c29740 1 -- 172.21.15.120:0/1460120898 wait complete. 2024-09-19T09:07:40.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.895+0000 7fe932c29740 20 client.17178 trim_cache size 0 max 16384 2024-09-19T09:07:40.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.911+0000 7fe932c29740 1 Processor -- start 2024-09-19T09:07:40.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.911+0000 7fe932c29740 1 -- start start 2024-09-19T09:07:40.912 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:40 smithi120 ceph-mon[31150]: pgmap v486: 97 pgs: 97 active+clean; 37 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 2.7 MiB/s wr, 117 op/s 2024-09-19T09:07:40.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.912+0000 7fe932c29740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562182c115d0 0x562182c282b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.912+0000 7fe932c29740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562182c12b40 0x562182c14120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.912+0000 7fe932c29740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562182c1c370 0x562182c14660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.912+0000 7fe932c29740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c1ad680 con 0x562182c1c370 2024-09-19T09:07:40.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.912+0000 7fe932c29740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c1af1c0 con 0x562182c12b40 2024-09-19T09:07:40.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.912+0000 7fe932c29740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c1add50 con 0x562182c115d0 2024-09-19T09:07:40.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.912+0000 7fe92ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562182c1c370 0x562182c14660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.912+0000 7fe929d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562182c12b40 0x562182c14120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.912+0000 7fe92ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562182c1c370 0x562182c14660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56798/0 (socket says 172.21.15.120:56798) 2024-09-19T09:07:40.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.912+0000 7fe92ad9d640 1 -- 172.21.15.120:0/828638909 learned_addr learned my addr 172.21.15.120:0/828638909 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:40.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.912+0000 7fe929d9b640 1 --2- 172.21.15.120:0/828638909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562182c12b40 0x562182c14120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:43216/0 (socket says 172.21.15.120:43216) 2024-09-19T09:07:40.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.912+0000 7fe92a59c640 1 --2- 172.21.15.120:0/828638909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562182c115d0 0x562182c282b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.912+0000 7fe92ad9d640 1 -- 172.21.15.120:0/828638909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562182c115d0 msgr2=0x562182c282b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:40.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.912+0000 7fe92ad9d640 1 --2- 172.21.15.120:0/828638909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562182c115d0 0x562182c282b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.912+0000 7fe92ad9d640 1 -- 172.21.15.120:0/828638909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562182c12b40 msgr2=0x562182c14120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:40.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.912+0000 7fe92ad9d640 1 --2- 172.21.15.120:0/828638909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562182c12b40 0x562182c14120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.912+0000 7fe92ad9d640 1 -- 172.21.15.120:0/828638909 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562182c14ba0 con 0x562182c1c370 2024-09-19T09:07:40.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.912+0000 7fe92a59c640 1 --2- 172.21.15.120:0/828638909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562182c115d0 0x562182c282b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:07:40.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.912+0000 7fe92ad9d640 1 --2- 172.21.15.120:0/828638909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562182c1c370 0x562182c14660 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0981b0 tx=0x7fe91c086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fcb96db9f0cf455 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:40.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.913+0000 7fe913fff640 1 -- 172.21.15.120:0/828638909 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c0b8020 con 0x562182c1c370 2024-09-19T09:07:40.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.913+0000 7fe913fff640 1 -- 172.21.15.120:0/828638909 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe91c09fae0 con 0x562182c1c370 2024-09-19T09:07:40.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.913+0000 7fe913fff640 1 -- 172.21.15.120:0/828638909 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c020040 con 0x562182c1c370 2024-09-19T09:07:40.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.913+0000 7fe932c29740 1 -- 172.21.15.120:0/828638909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562182c1c370 msgr2=0x562182c14660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:40.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.914+0000 7fe932c29740 1 --2- 172.21.15.120:0/828638909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562182c1c370 0x562182c14660 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0981b0 tx=0x7fe91c086040 comp rx=0 tx=0).stop 2024-09-19T09:07:40.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.914+0000 7fe932c29740 1 -- 172.21.15.120:0/828638909 shutdown_connections 2024-09-19T09:07:40.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.914+0000 7fe932c29740 1 --2- 172.21.15.120:0/828638909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562182c1c370 0x562182c14660 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.914+0000 7fe932c29740 1 --2- 172.21.15.120:0/828638909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562182c12b40 0x562182c14120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.914+0000 7fe932c29740 1 --2- 172.21.15.120:0/828638909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562182c115d0 0x562182c282b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.914+0000 7fe932c29740 1 -- 172.21.15.120:0/828638909 >> 172.21.15.120:0/828638909 conn(0x562182bda310 msgr2=0x562182c18080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:40.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.914+0000 7fe932c29740 1 -- 172.21.15.120:0/828638909 shutdown_connections 2024-09-19T09:07:40.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.914+0000 7fe932c29740 1 -- 172.21.15.120:0/828638909 wait complete. 2024-09-19T09:07:40.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.914+0000 7fe932c29740 1 -- start start 2024-09-19T09:07:40.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.914+0000 7fe932c29740 1 Processor -- start 2024-09-19T09:07:40.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.914+0000 7fe932c29740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562182c115d0 0x562182c24600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.914+0000 7fe932c29740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562182c12b40 0x562182c24b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.914+0000 7fe932c29740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562182c1c370 0x562182c157a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.915+0000 7fe932c29740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c1aeaf0 con 0x562182c1c370 2024-09-19T09:07:40.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.915+0000 7fe932c29740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c1ae420 con 0x562182c115d0 2024-09-19T09:07:40.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.915+0000 7fe932c29740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c1ab640 con 0x562182c12b40 2024-09-19T09:07:40.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.915+0000 7fe932c29740 10 client.0 _ll_register_callbacks cb 0x7fe92000ff70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:40.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.915+0000 7fe92ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562182c1c370 0x562182c157a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.915+0000 7fe92ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562182c1c370 0x562182c157a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56800/0 (socket says 172.21.15.120:56800) 2024-09-19T09:07:40.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.915+0000 7fe92a59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562182c115d0 0x562182c24600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.915+0000 7fe929d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562182c12b40 0x562182c24b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.915+0000 7fe92ad9d640 1 -- 172.21.15.120:0/2710623871 learned_addr learned my addr 172.21.15.120:0/2710623871 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:40.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.915+0000 7fe929d9b640 1 -- 172.21.15.120:0/2710623871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562182c115d0 msgr2=0x562182c24600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:40.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.915+0000 7fe929d9b640 1 --2- 172.21.15.120:0/2710623871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562182c115d0 0x562182c24600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.915+0000 7fe929d9b640 1 -- 172.21.15.120:0/2710623871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562182c1c370 msgr2=0x562182c157a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:40.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.915+0000 7fe929d9b640 1 --2- 172.21.15.120:0/2710623871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562182c1c370 0x562182c157a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.915+0000 7fe929d9b640 1 -- 172.21.15.120:0/2710623871 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562182c15eb0 con 0x562182c12b40 2024-09-19T09:07:40.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.915+0000 7fe929d9b640 1 --2- 172.21.15.120:0/2710623871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562182c12b40 0x562182c24b40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x562182c1a540 tx=0x7fe92000ea30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df16ca80a63a019d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:40.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.915+0000 7fe911ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:07:40.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.916+0000 7fe911ffb640 1 -- 172.21.15.120:0/2710623871 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe920039070 con 0x562182c12b40 2024-09-19T09:07:40.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.916+0000 7fe932c29740 1 -- 172.21.15.120:0/2710623871 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe91c09fae0 con 0x562182c12b40 2024-09-19T09:07:40.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.916+0000 7fe911ffb640 1 -- 172.21.15.120:0/2710623871 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fe920004d20 con 0x562182c12b40 2024-09-19T09:07:40.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.916+0000 7fe911ffb640 1 -- 172.21.15.120:0/2710623871 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe920026040 con 0x562182c12b40 2024-09-19T09:07:40.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.916+0000 7fe932c29740 10 client.26404 fetch_fsmap learned FSMap version 30 2024-09-19T09:07:40.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.916+0000 7fe932c29740 1 -- 172.21.15.120:0/2710623871 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562182c96680 con 0x562182c12b40 2024-09-19T09:07:40.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.916+0000 7fe911ffb640 1 -- 172.21.15.120:0/2710623871 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe91c09fae0 con 0x562182c12b40 2024-09-19T09:07:40.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.917+0000 7fe911ffb640 1 -- 172.21.15.120:0/2710623871 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe92001d040 con 0x562182c12b40 2024-09-19T09:07:40.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.917+0000 7fe932c29740 10 client.26404 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:07:40.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.917+0000 7fe932c29740 10 client.26404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:07:40.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.917+0000 7fe932c29740 20 client.26404 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:40.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.917+0000 7fe932c29740 1 --2- 172.21.15.120:0/2710623871 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562182c96d10 0x562182c99130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:40.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.917+0000 7fe932c29740 4 client.26404 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:07:40.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.917+0000 7fe932c29740 1 -- 172.21.15.120:0/2710623871 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562182b7ed70 con 0x562182c96d10 2024-09-19T09:07:40.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.917+0000 7fe92a59c640 1 --2- 172.21.15.120:0/2710623871 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562182c96d10 0x562182c99130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:40.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.917+0000 7fe92a59c640 1 --2- 172.21.15.120:0/2710623871 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562182c96d10 0x562182c99130 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=591dafabf37c346 server_cookie=ee5512b36d2d08fe in_seq=0 out_seq=0 2024-09-19T09:07:40.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.917+0000 7fe911ffb640 10 client.26404 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:40.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.918+0000 7fe911ffb640 1 -- 172.21.15.120:0/2710623871 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562182b7ed70 con 0x562182c96d10 2024-09-19T09:07:40.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.918+0000 7fe911ffb640 10 client.26404 handle_command_reply: tid=0 2024-09-19T09:07:40.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.918+0000 7fe932c29740 1 client.26404 shutdown 2024-09-19T09:07:40.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.918+0000 7fe932c29740 1 -- 172.21.15.120:0/2710623871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562182c12b40 msgr2=0x562182c24b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:40.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.918+0000 7fe932c29740 1 --2- 172.21.15.120:0/2710623871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562182c12b40 0x562182c24b40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x562182c1a540 tx=0x7fe92000ea30 comp rx=0 tx=0).stop 2024-09-19T09:07:40.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.918+0000 7fe932c29740 1 -- 172.21.15.120:0/2710623871 shutdown_connections 2024-09-19T09:07:40.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.918+0000 7fe932c29740 1 --2- 172.21.15.120:0/2710623871 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562182c96d10 0x562182c99130 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.918+0000 7fe911ffb640 0 client.26404 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:40.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.918+0000 7fe932c29740 1 --2- 172.21.15.120:0/2710623871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562182c1c370 0x562182c157a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.918+0000 7fe932c29740 1 --2- 172.21.15.120:0/2710623871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562182c12b40 0x562182c24b40 secure :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0x562182c1a540 tx=0x7fe92000ea30 comp rx=0 tx=0).stop 2024-09-19T09:07:40.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.918+0000 7fe932c29740 1 --2- 172.21.15.120:0/2710623871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562182c115d0 0x562182c24600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.918+0000 7fe932c29740 1 -- 172.21.15.120:0/2710623871 >> 172.21.15.120:0/2710623871 conn(0x562182bda310 msgr2=0x562182c1a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:40.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.918+0000 7fe932c29740 1 -- 172.21.15.120:0/2710623871 shutdown_connections 2024-09-19T09:07:40.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.919+0000 7fe932c29740 1 -- 172.21.15.120:0/2710623871 wait complete. 2024-09-19T09:07:40.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.919+0000 7fe932c29740 20 client.26404 trim_cache size 0 max 16384 2024-09-19T09:07:40.923 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:07:40.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.919+0000 7fe930850640 1 -- 172.21.15.120:0/2600197551 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9140a32c0 msgr2=0x7fe9140a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:40.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.919+0000 7fe930850640 1 --2- 172.21.15.120:0/2600197551 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9140a32c0 0x7fe9140a5780 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x562182c0eac0 tx=0x7fe920009990 comp rx=0 tx=0).stop 2024-09-19T09:07:40.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.919+0000 7fe930850640 1 -- 172.21.15.120:0/2600197551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe92c1082e0 msgr2=0x7fe92c19ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:40.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.919+0000 7fe930850640 1 --2- 172.21.15.120:0/2600197551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe92c1082e0 0x7fe92c19ce60 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00d750 tx=0x7fe91c00dc10 comp rx=0 tx=0).stop 2024-09-19T09:07:40.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.919+0000 7fe930850640 1 -- 172.21.15.120:0/2600197551 shutdown_connections 2024-09-19T09:07:40.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.919+0000 7fe930850640 1 --2- 172.21.15.120:0/2600197551 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9140a32c0 0x7fe9140a5780 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.919+0000 7fe930850640 1 --2- 172.21.15.120:0/2600197551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe92c1082e0 0x7fe92c19ce60 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.919+0000 7fe930850640 1 --2- 172.21.15.120:0/2600197551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe92c103760 0x7fe92c198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.919+0000 7fe930850640 1 --2- 172.21.15.120:0/2600197551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe92c102db0 0x7fe92c198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:40.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.919+0000 7fe930850640 1 -- 172.21.15.120:0/2600197551 >> 172.21.15.120:0/2600197551 conn(0x7fe92c0fc7d0 msgr2=0x7fe92c105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:40.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.920+0000 7fe930850640 1 -- 172.21.15.120:0/2600197551 shutdown_connections 2024-09-19T09:07:40.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:40.920+0000 7fe930850640 1 -- 172.21.15.120:0/2600197551 wait complete. 2024-09-19T09:07:41.047 INFO:tasks.workunit.client.0.smithi120.stdout: 1 363567 7.63 MB/sec execute 96 sec latency 26.415 ms 2024-09-19T09:07:41.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:40 smithi196 ceph-mon[35150]: pgmap v486: 97 pgs: 97 active+clean; 37 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 2.7 MiB/s wr, 117 op/s 2024-09-19T09:07:41.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:40 smithi153 ceph-mon[35071]: pgmap v486: 97 pgs: 97 active+clean; 37 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 2.7 MiB/s wr, 117 op/s 2024-09-19T09:07:41.672 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:07:42.047 INFO:tasks.workunit.client.0.smithi120.stdout: 1 366050 7.69 MB/sec execute 97 sec latency 59.674 ms 2024-09-19T09:07:42.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.222+0000 7fc56fccc640 1 -- 172.21.15.120:0/3993227712 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5681036e0 msgr2=0x7fc568107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:42.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.222+0000 7fc56fccc640 1 --2- 172.21.15.120:0/3993227712 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5681036e0 0x7fc568107b90 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fc558009930 tx=0x7fc55802a420 comp rx=0 tx=0).stop 2024-09-19T09:07:42.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.222+0000 7fc56fccc640 1 -- 172.21.15.120:0/3993227712 shutdown_connections 2024-09-19T09:07:42.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.222+0000 7fc56fccc640 1 --2- 172.21.15.120:0/3993227712 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5681082a0 0x7fc56810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:42.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.222+0000 7fc56fccc640 1 --2- 172.21.15.120:0/3993227712 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5681036e0 0x7fc568107b90 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:42.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.222+0000 7fc56fccc640 1 --2- 172.21.15.120:0/3993227712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc568102d30 0x7fc568103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:42.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.222+0000 7fc56fccc640 1 -- 172.21.15.120:0/3993227712 >> 172.21.15.120:0/3993227712 conn(0x7fc5680fc7d0 msgr2=0x7fc5680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:42.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.222+0000 7fc56fccc640 1 -- 172.21.15.120:0/3993227712 shutdown_connections 2024-09-19T09:07:42.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.222+0000 7fc56fccc640 1 -- 172.21.15.120:0/3993227712 wait complete. 2024-09-19T09:07:42.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.223+0000 7fc56fccc640 1 Processor -- start 2024-09-19T09:07:42.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.223+0000 7fc56fccc640 1 -- start start 2024-09-19T09:07:42.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.223+0000 7fc56fccc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc568102d30 0x7fc5681a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:42.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.223+0000 7fc56fccc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5681036e0 0x7fc5681a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:42.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.223+0000 7fc56fccc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5681082a0 0x7fc56819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:42.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.223+0000 7fc56fccc640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc5681120a0 con 0x7fc5681036e0 2024-09-19T09:07:42.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.223+0000 7fc56fccc640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc568111f20 con 0x7fc568102d30 2024-09-19T09:07:42.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.223+0000 7fc56fccc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc568112220 con 0x7fc5681082a0 2024-09-19T09:07:42.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.224+0000 7fc56d267640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5681036e0 0x7fc5681a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:42.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.224+0000 7fc56d267640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5681036e0 0x7fc5681a0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56304/0 (socket says 172.21.15.120:56304) 2024-09-19T09:07:42.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.224+0000 7fc56d267640 1 -- 172.21.15.120:0/1839602109 learned_addr learned my addr 172.21.15.120:0/1839602109 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:42.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.224+0000 7fc56da68640 1 --2- 172.21.15.120:0/1839602109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc568102d30 0x7fc5681a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:42.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.224+0000 7fc56d267640 1 -- 172.21.15.120:0/1839602109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5681082a0 msgr2=0x7fc56819a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:42.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.224+0000 7fc56d267640 1 --2- 172.21.15.120:0/1839602109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5681082a0 0x7fc56819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:42.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.224+0000 7fc56d267640 1 -- 172.21.15.120:0/1839602109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc568102d30 msgr2=0x7fc5681a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:42.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.224+0000 7fc56d267640 1 --2- 172.21.15.120:0/1839602109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc568102d30 0x7fc5681a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:42.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.224+0000 7fc56d267640 1 -- 172.21.15.120:0/1839602109 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc56819ad40 con 0x7fc5681036e0 2024-09-19T09:07:42.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.224+0000 7fc56d267640 1 --2- 172.21.15.120:0/1839602109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5681036e0 0x7fc5681a0990 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fc558005940 tx=0x7fc558031b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:42.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.224+0000 7fc55effd640 1 -- 172.21.15.120:0/1839602109 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55803f030 con 0x7fc5681036e0 2024-09-19T09:07:42.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.224+0000 7fc55effd640 1 -- 172.21.15.120:0/1839602109 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5580043d0 con 0x7fc5681036e0 2024-09-19T09:07:42.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.224+0000 7fc55effd640 1 -- 172.21.15.120:0/1839602109 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc558035070 con 0x7fc5681036e0 2024-09-19T09:07:42.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.224+0000 7fc56fccc640 1 -- 172.21.15.120:0/1839602109 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc56819afd0 con 0x7fc5681036e0 2024-09-19T09:07:42.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.224+0000 7fc56fccc640 1 -- 172.21.15.120:0/1839602109 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5681a7320 con 0x7fc5681036e0 2024-09-19T09:07:42.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.225+0000 7fc56fccc640 1 -- 172.21.15.120:0/1839602109 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc568103110 con 0x7fc5681036e0 2024-09-19T09:07:42.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.226+0000 7fc55effd640 1 -- 172.21.15.120:0/1839602109 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fc558042030 con 0x7fc5681036e0 2024-09-19T09:07:42.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.227+0000 7fc55effd640 1 --2- 172.21.15.120:0/1839602109 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc5540a3240 0x7fc5540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:42.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.227+0000 7fc56da68640 1 --2- 172.21.15.120:0/1839602109 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc5540a3240 0x7fc5540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:42.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.227+0000 7fc55effd640 1 -- 172.21.15.120:0/1839602109 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fc5580eff00 con 0x7fc5681036e0 2024-09-19T09:07:42.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.228+0000 7fc56da68640 1 --2- 172.21.15.120:0/1839602109 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc5540a3240 0x7fc5540a5700 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc564009bd0 tx=0x7fc564005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:42.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.231+0000 7fc55effd640 1 -- 172.21.15.120:0/1839602109 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc5580bcbe0 con 0x7fc5681036e0 2024-09-19T09:07:42.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.445+0000 7fc56fccc640 1 -- 172.21.15.120:0/1839602109 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc56810c2c0 con 0x7fc5681036e0 2024-09-19T09:07:42.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.446+0000 7fc55effd640 1 -- 172.21.15.120:0/1839602109 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11985 (secure 0 0 0) 0x7fc5580bc4e0 con 0x7fc5681036e0 2024-09-19T09:07:42.446 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:07:42.447 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":30,"btime":"2024-09-19T09:07:14:812065+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:07:14.812046+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26351":{"gid":26351,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/364941779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":364941779},{"type":"v1","addr":"172.21.15.153:6835","nonce":364941779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:07:42.447 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 30 2024-09-19T09:07:42.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.448+0000 7fc56fccc640 1 -- 172.21.15.120:0/1839602109 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc5540a3240 msgr2=0x7fc5540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:42.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.448+0000 7fc56fccc640 1 --2- 172.21.15.120:0/1839602109 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc5540a3240 0x7fc5540a5700 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc564009bd0 tx=0x7fc564005f90 comp rx=0 tx=0).stop 2024-09-19T09:07:42.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.448+0000 7fc56fccc640 1 -- 172.21.15.120:0/1839602109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5681036e0 msgr2=0x7fc5681a0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:42.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.448+0000 7fc56fccc640 1 --2- 172.21.15.120:0/1839602109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5681036e0 0x7fc5681a0990 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fc558005940 tx=0x7fc558031b50 comp rx=0 tx=0).stop 2024-09-19T09:07:42.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.448+0000 7fc56fccc640 1 -- 172.21.15.120:0/1839602109 shutdown_connections 2024-09-19T09:07:42.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.448+0000 7fc56fccc640 1 --2- 172.21.15.120:0/1839602109 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc5540a3240 0x7fc5540a5700 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:42.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.448+0000 7fc56fccc640 1 --2- 172.21.15.120:0/1839602109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5681082a0 0x7fc56819a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:42.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.448+0000 7fc56fccc640 1 --2- 172.21.15.120:0/1839602109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5681036e0 0x7fc5681a0990 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:42.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.448+0000 7fc56fccc640 1 --2- 172.21.15.120:0/1839602109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc568102d30 0x7fc5681a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:42.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.448+0000 7fc56fccc640 1 -- 172.21.15.120:0/1839602109 >> 172.21.15.120:0/1839602109 conn(0x7fc5680fc7d0 msgr2=0x7fc568100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:42.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.448+0000 7fc56fccc640 1 -- 172.21.15.120:0/1839602109 shutdown_connections 2024-09-19T09:07:42.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:42.448+0000 7fc56fccc640 1 -- 172.21.15.120:0/1839602109 wait complete. 2024-09-19T09:07:42.673 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:07:42.674 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:07:43.047 INFO:tasks.workunit.client.0.smithi120.stdout: 1 369258 7.81 MB/sec execute 98 sec latency 10.379 ms 2024-09-19T09:07:43.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:42 smithi120 ceph-mon[31150]: pgmap v487: 97 pgs: 97 active+clean; 36 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 2.7 MiB/s wr, 131 op/s 2024-09-19T09:07:43.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:42 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1839602109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:07:43.164 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0' 2024-09-19T09:07:43.165 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0 2024-09-19T09:07:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:42 smithi196 ceph-mon[35150]: pgmap v487: 97 pgs: 97 active+clean; 36 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 2.7 MiB/s wr, 131 op/s 2024-09-19T09:07:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:42 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1839602109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:07:43.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:42 smithi153 ceph-mon[35071]: pgmap v487: 97 pgs: 97 active+clean; 36 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 2.7 MiB/s wr, 131 op/s 2024-09-19T09:07:43.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:42 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1839602109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:07:43.268 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:07:43.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.768+0000 7f538f4b4640 1 -- 172.21.15.120:0/165133564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5388106f10 msgr2=0x7f538810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:43.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.768+0000 7f538f4b4640 1 --2- 172.21.15.120:0/165133564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5388106f10 0x7f538810ea90 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f5380009ca0 tx=0x7f538002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.768+0000 7f538f4b4640 1 -- 172.21.15.120:0/165133564 shutdown_connections 2024-09-19T09:07:43.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.768+0000 7f538f4b4640 1 --2- 172.21.15.120:0/165133564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5388106f10 0x7f538810ea90 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.768+0000 7f538f4b4640 1 --2- 172.21.15.120:0/165133564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5388102790 0x7f53881067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.768+0000 7f538f4b4640 1 --2- 172.21.15.120:0/165133564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5388101de0 0x7f53881021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.768+0000 7f538f4b4640 1 -- 172.21.15.120:0/165133564 >> 172.21.15.120:0/165133564 conn(0x7f53880fd8c0 msgr2=0x7f53880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:43.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.768+0000 7f538f4b4640 1 -- 172.21.15.120:0/165133564 shutdown_connections 2024-09-19T09:07:43.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.768+0000 7f538f4b4640 1 -- 172.21.15.120:0/165133564 wait complete. 2024-09-19T09:07:43.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.769+0000 7f538f4b4640 1 Processor -- start 2024-09-19T09:07:43.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.769+0000 7f538f4b4640 1 -- start start 2024-09-19T09:07:43.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.769+0000 7f538f4b4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5388101de0 0x7f53880744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:43.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.769+0000 7f538f4b4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5388102790 0x7f5388074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:43.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.769+0000 7f538f4b4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5388106f10 0x7f5388074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:43.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.769+0000 7f538f4b4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5388111fc0 con 0x7f5388106f10 2024-09-19T09:07:43.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.769+0000 7f538f4b4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5388111e40 con 0x7f5388101de0 2024-09-19T09:07:43.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.769+0000 7f538f4b4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5388112140 con 0x7f5388102790 2024-09-19T09:07:43.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.769+0000 7f538da51640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5388106f10 0x7f5388074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:43.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.769+0000 7f538da51640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5388106f10 0x7f5388074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56312/0 (socket says 172.21.15.120:56312) 2024-09-19T09:07:43.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.769+0000 7f538da51640 1 -- 172.21.15.120:0/1544274263 learned_addr learned my addr 172.21.15.120:0/1544274263 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:43.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.769+0000 7f538ca4f640 1 --2- 172.21.15.120:0/1544274263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5388102790 0x7f5388074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:43.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.769+0000 7f538d250640 1 --2- 172.21.15.120:0/1544274263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5388101de0 0x7f53880744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:43.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.769+0000 7f538da51640 1 -- 172.21.15.120:0/1544274263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5388102790 msgr2=0x7f5388074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:43.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.769+0000 7f538da51640 1 --2- 172.21.15.120:0/1544274263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5388102790 0x7f5388074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.769+0000 7f538da51640 1 -- 172.21.15.120:0/1544274263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5388101de0 msgr2=0x7f53880744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:43.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.769+0000 7f538da51640 1 --2- 172.21.15.120:0/1544274263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5388101de0 0x7f53880744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.769+0000 7f538da51640 1 -- 172.21.15.120:0/1544274263 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53881a68f0 con 0x7f5388106f10 2024-09-19T09:07:43.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.769+0000 7f538da51640 1 --2- 172.21.15.120:0/1544274263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5388106f10 0x7f5388074f70 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f5380003f10 tx=0x7f5380003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:43.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.770+0000 7f537e7fc640 1 -- 172.21.15.120:0/1544274263 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5380031710 con 0x7f5388106f10 2024-09-19T09:07:43.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.770+0000 7f537e7fc640 1 -- 172.21.15.120:0/1544274263 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53800318b0 con 0x7f5388106f10 2024-09-19T09:07:43.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.770+0000 7f538f4b4640 1 -- 172.21.15.120:0/1544274263 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53881a6b80 con 0x7f5388106f10 2024-09-19T09:07:43.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.770+0000 7f537e7fc640 1 -- 172.21.15.120:0/1544274263 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538002fb50 con 0x7f5388106f10 2024-09-19T09:07:43.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.770+0000 7f538f4b4640 1 -- 172.21.15.120:0/1544274263 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53881a7110 con 0x7f5388106f10 2024-09-19T09:07:43.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.772+0000 7f537e7fc640 1 -- 172.21.15.120:0/1544274263 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f538002fd30 con 0x7f5388106f10 2024-09-19T09:07:43.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.772+0000 7f539188d740 1 Processor -- start 2024-09-19T09:07:43.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.772+0000 7f539188d740 1 -- start start 2024-09-19T09:07:43.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.773+0000 7f539188d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557524bddb60 0x557524bddf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:43.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.773+0000 7f539188d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557524bde480 0x557524be8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:43.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.773+0000 7f539188d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557524be8d40 0x557524beb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:43.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.773+0000 7f539188d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557524b43280 con 0x557524bde480 2024-09-19T09:07:43.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.773+0000 7f539188d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557524aeab10 con 0x557524be8d40 2024-09-19T09:07:43.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.773+0000 7f539188d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557524b3f240 con 0x557524bddb60 2024-09-19T09:07:43.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.773+0000 7f538d250640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557524bde480 0x557524be8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:43.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.773+0000 7f538d250640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557524bde480 0x557524be8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56322/0 (socket says 172.21.15.120:56322) 2024-09-19T09:07:43.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.773+0000 7f538d250640 1 -- 172.21.15.120:0/468993765 learned_addr learned my addr 172.21.15.120:0/468993765 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:43.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.773+0000 7f538ca4f640 1 --2- 172.21.15.120:0/468993765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557524be8d40 0x557524beb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:43.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.773+0000 7f538ca4f640 1 --2- 172.21.15.120:0/468993765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557524bddb60 0x557524bddf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:43.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.773+0000 7f537e7fc640 1 --2- 172.21.15.120:0/1544274263 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f53740a32c0 0x7f53740a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:43.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.773+0000 7f538d250640 1 -- 172.21.15.120:0/468993765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557524bddb60 msgr2=0x557524bddf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:43.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.773+0000 7f538d250640 1 --2- 172.21.15.120:0/468993765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557524bddb60 0x557524bddf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.773+0000 7f538d250640 1 -- 172.21.15.120:0/468993765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557524be8d40 msgr2=0x557524beb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:43.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.773+0000 7f538d250640 1 --2- 172.21.15.120:0/468993765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557524be8d40 0x557524beb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.773+0000 7f538d250640 1 -- 172.21.15.120:0/468993765 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557524af71f0 con 0x557524bde480 2024-09-19T09:07:43.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.773+0000 7f538da51640 1 --2- 172.21.15.120:0/1544274263 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f53740a32c0 0x7f53740a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:43.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.773+0000 7f538d250640 1 --2- 172.21.15.120:0/468993765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557524bde480 0x557524be8800 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f538400bcd0 tx=0x7f53840107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e946bc8ca0f22788 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:43.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.773+0000 7f537e7fc640 1 -- 172.21.15.120:0/1544274263 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f53800f0db0 con 0x7f5388106f10 2024-09-19T09:07:43.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.773+0000 7f53737fe640 1 -- 172.21.15.120:0/468993765 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384005020 con 0x557524bde480 2024-09-19T09:07:43.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.774+0000 7f53737fe640 1 -- 172.21.15.120:0/468993765 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53840047a0 con 0x557524bde480 2024-09-19T09:07:43.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.774+0000 7f53737fe640 1 -- 172.21.15.120:0/468993765 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384011740 con 0x557524bde480 2024-09-19T09:07:43.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.774+0000 7f539188d740 1 -- 172.21.15.120:0/468993765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557524bde480 msgr2=0x557524be8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:43.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.774+0000 7f539188d740 1 --2- 172.21.15.120:0/468993765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557524bde480 0x557524be8800 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f538400bcd0 tx=0x7f53840107a0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.774+0000 7f538da51640 1 --2- 172.21.15.120:0/1544274263 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f53740a32c0 0x7f53740a5780 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f53800b9d60 tx=0x7f538003e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:43.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.774+0000 7f539188d740 1 -- 172.21.15.120:0/468993765 shutdown_connections 2024-09-19T09:07:43.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.774+0000 7f539188d740 1 --2- 172.21.15.120:0/468993765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557524be8d40 0x557524beb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.774+0000 7f539188d740 1 --2- 172.21.15.120:0/468993765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557524bde480 0x557524be8800 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.774+0000 7f539188d740 1 --2- 172.21.15.120:0/468993765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557524bddb60 0x557524bddf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.774+0000 7f539188d740 1 -- 172.21.15.120:0/468993765 >> 172.21.15.120:0/468993765 conn(0x557524bda0f0 msgr2=0x557524bda4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:43.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.774+0000 7f539188d740 1 -- 172.21.15.120:0/468993765 shutdown_connections 2024-09-19T09:07:43.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.774+0000 7f539188d740 1 -- 172.21.15.120:0/468993765 wait complete. 2024-09-19T09:07:43.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.775+0000 7f539188d740 1 -- start start 2024-09-19T09:07:43.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.775+0000 7f539188d740 1 Processor -- start 2024-09-19T09:07:43.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.775+0000 7f539188d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557524bddb60 0x557524be5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:43.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.775+0000 7f539188d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557524bde480 0x557524be60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:43.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.775+0000 7f539188d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557524be8d40 0x557524be6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:43.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.775+0000 7f539188d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557524beb910 con 0x557524be8d40 2024-09-19T09:07:43.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.775+0000 7f539188d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557524afdf10 con 0x557524bddb60 2024-09-19T09:07:43.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.775+0000 7f539188d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557524beba90 con 0x557524bde480 2024-09-19T09:07:43.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.775+0000 7f539188d740 10 client.0 _ll_register_callbacks cb 0x557524ac6a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:43.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.775+0000 7f538ca4f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557524bddb60 0x557524be5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:43.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.775+0000 7f538d250640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557524bde480 0x557524be60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:43.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.775+0000 7f538d250640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557524bde480 0x557524be60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:48762/0 (socket says 172.21.15.120:48762) 2024-09-19T09:07:43.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.775+0000 7f538d250640 1 -- 172.21.15.120:0/2890467197 learned_addr learned my addr 172.21.15.120:0/2890467197 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:43.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.775+0000 7f538ca4f640 1 --2- 172.21.15.120:0/2890467197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557524be8d40 0x557524be6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:43.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.776+0000 7f538d250640 1 -- 172.21.15.120:0/2890467197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557524bddb60 msgr2=0x557524be5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:43.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.776+0000 7f538d250640 1 --2- 172.21.15.120:0/2890467197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557524bddb60 0x557524be5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.776+0000 7f538d250640 1 -- 172.21.15.120:0/2890467197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557524be8d40 msgr2=0x557524be6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:43.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.776+0000 7f538d250640 1 --2- 172.21.15.120:0/2890467197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557524be8d40 0x557524be6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.776+0000 7f538d250640 1 -- 172.21.15.120:0/2890467197 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575248b46a0 con 0x557524bde480 2024-09-19T09:07:43.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.776+0000 7f538ca4f640 1 --2- 172.21.15.120:0/2890467197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557524be8d40 0x557524be6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:07:43.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.776+0000 7f538d250640 1 --2- 172.21.15.120:0/2890467197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557524bde480 0x557524be60f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5384000c00 tx=0x7f538400b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92579d55a2d89cde server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:43.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.776+0000 7f5370ff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:07:43.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.776+0000 7f5370ff9640 1 -- 172.21.15.120:0/2890467197 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538402a060 con 0x557524bde480 2024-09-19T09:07:43.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.776+0000 7f5370ff9640 1 -- 172.21.15.120:0/2890467197 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f5384004c60 con 0x557524bde480 2024-09-19T09:07:43.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.776+0000 7f539188d740 1 -- 172.21.15.120:0/2890467197 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53840047a0 con 0x557524bde480 2024-09-19T09:07:43.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.776+0000 7f5370ff9640 1 -- 172.21.15.120:0/2890467197 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5384004f90 con 0x557524bde480 2024-09-19T09:07:43.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.776+0000 7f539188d740 10 client.26419 fetch_fsmap learned FSMap version 30 2024-09-19T09:07:43.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.776+0000 7f539188d740 1 -- 172.21.15.120:0/2890467197 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557524bf74d0 con 0x557524bde480 2024-09-19T09:07:43.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.776+0000 7f5370ff9640 1 -- 172.21.15.120:0/2890467197 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53840047a0 con 0x557524bde480 2024-09-19T09:07:43.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.777+0000 7f5370ff9640 1 -- 172.21.15.120:0/2890467197 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f538402f070 con 0x557524bde480 2024-09-19T09:07:43.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.777+0000 7f539188d740 10 client.26419 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:07:43.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.777+0000 7f539188d740 10 client.26419 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:07:43.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.777+0000 7f539188d740 20 client.26419 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:43.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.777+0000 7f539188d740 1 --2- 172.21.15.120:0/2890467197 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557524bf8990 0x557524bfad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:43.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.777+0000 7f539188d740 4 client.26419 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:07:43.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.777+0000 7f539188d740 1 -- 172.21.15.120:0/2890467197 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557524b507f0 con 0x557524bf8990 2024-09-19T09:07:43.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.777+0000 7f538ca4f640 1 --2- 172.21.15.120:0/2890467197 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557524bf8990 0x557524bfad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:43.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.777+0000 7f5370ff9640 10 client.26419 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:43.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.777+0000 7f538ca4f640 1 --2- 172.21.15.120:0/2890467197 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557524bf8990 0x557524bfad80 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e53bdaa9e44bbc9 server_cookie=cff8851f49f8ff6c in_seq=0 out_seq=0 2024-09-19T09:07:43.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.778+0000 7f5370ff9640 1 -- 172.21.15.120:0/2890467197 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557524b507f0 con 0x557524bf8990 2024-09-19T09:07:43.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.778+0000 7f5370ff9640 10 client.26419 handle_command_reply: tid=0 2024-09-19T09:07:43.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.778+0000 7f539188d740 1 client.26419 shutdown 2024-09-19T09:07:43.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.778+0000 7f539188d740 1 -- 172.21.15.120:0/2890467197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557524bde480 msgr2=0x557524be60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:43.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.778+0000 7f539188d740 1 --2- 172.21.15.120:0/2890467197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557524bde480 0x557524be60f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5384000c00 tx=0x7f538400b930 comp rx=0 tx=0).stop 2024-09-19T09:07:43.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.778+0000 7f539188d740 1 -- 172.21.15.120:0/2890467197 shutdown_connections 2024-09-19T09:07:43.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.778+0000 7f539188d740 1 --2- 172.21.15.120:0/2890467197 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557524bf8990 0x557524bfad80 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.778+0000 7f539188d740 1 --2- 172.21.15.120:0/2890467197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557524be8d40 0x557524be6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.778+0000 7f539188d740 1 --2- 172.21.15.120:0/2890467197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557524bde480 0x557524be60f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.778+0000 7f5370ff9640 0 client.26419 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:43.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.778+0000 7f539188d740 1 --2- 172.21.15.120:0/2890467197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557524bddb60 0x557524be5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.778+0000 7f539188d740 1 -- 172.21.15.120:0/2890467197 >> 172.21.15.120:0/2890467197 conn(0x557524bda0f0 msgr2=0x557524bd9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:43.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.778+0000 7f539188d740 1 -- 172.21.15.120:0/2890467197 shutdown_connections 2024-09-19T09:07:43.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.779+0000 7f539188d740 1 -- 172.21.15.120:0/2890467197 wait complete. 2024-09-19T09:07:43.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.779+0000 7f539188d740 20 client.26419 trim_cache size 0 max 16384 2024-09-19T09:07:43.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.797+0000 7f539188d740 1 Processor -- start 2024-09-19T09:07:43.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.797+0000 7f539188d740 1 -- start start 2024-09-19T09:07:43.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.797+0000 7f539188d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557524bddb60 0x557524be05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:43.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.797+0000 7f539188d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557524bde480 0x557524be7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:43.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.797+0000 7f539188d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557524be8d40 0x557524c00350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:43.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.797+0000 7f539188d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f53881af7a0 con 0x557524bde480 2024-09-19T09:07:43.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.797+0000 7f539188d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f53881b12e0 con 0x557524be8d40 2024-09-19T09:07:43.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.797+0000 7f539188d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f53881afe70 con 0x557524bddb60 2024-09-19T09:07:43.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.797+0000 7f538d250640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557524bde480 0x557524be7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:43.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.797+0000 7f538ca4f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557524bddb60 0x557524be05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:43.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.797+0000 7f538ca4f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557524bddb60 0x557524be05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:48772/0 (socket says 172.21.15.120:48772) 2024-09-19T09:07:43.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.797+0000 7f538d250640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557524bde480 0x557524be7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56336/0 (socket says 172.21.15.120:56336) 2024-09-19T09:07:43.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.797+0000 7f538ca4f640 1 -- 172.21.15.120:0/3817978095 learned_addr learned my addr 172.21.15.120:0/3817978095 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:43.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.797+0000 7f538ca4f640 1 --2- 172.21.15.120:0/3817978095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557524be8d40 0x557524c00350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:43.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.798+0000 7f538ca4f640 1 -- 172.21.15.120:0/3817978095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557524be8d40 msgr2=0x557524c00350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:43.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.798+0000 7f538ca4f640 1 --2- 172.21.15.120:0/3817978095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557524be8d40 0x557524c00350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.798+0000 7f538ca4f640 1 -- 172.21.15.120:0/3817978095 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557524bde480 msgr2=0x557524be7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:43.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.798+0000 7f538ca4f640 1 --2- 172.21.15.120:0/3817978095 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557524bde480 0x557524be7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.798+0000 7f538ca4f640 1 -- 172.21.15.120:0/3817978095 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557524be4100 con 0x557524bddb60 2024-09-19T09:07:43.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.798+0000 7f538d250640 1 --2- 172.21.15.120:0/3817978095 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557524bde480 0x557524be7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:07:43.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.798+0000 7f538ca4f640 1 --2- 172.21.15.120:0/3817978095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557524bddb60 0x557524be05c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f53780042e0 tx=0x7f5378004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=915cd42e056b3ff2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:43.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.798+0000 7f5372ffd640 1 -- 172.21.15.120:0/3817978095 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5378005020 con 0x557524bddb60 2024-09-19T09:07:43.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.798+0000 7f5372ffd640 1 -- 172.21.15.120:0/3817978095 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53780047a0 con 0x557524bddb60 2024-09-19T09:07:43.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.798+0000 7f5372ffd640 1 -- 172.21.15.120:0/3817978095 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53780039b0 con 0x557524bddb60 2024-09-19T09:07:43.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.799+0000 7f539188d740 1 -- 172.21.15.120:0/3817978095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557524bddb60 msgr2=0x557524be05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:43.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.799+0000 7f539188d740 1 --2- 172.21.15.120:0/3817978095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557524bddb60 0x557524be05c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f53780042e0 tx=0x7f5378004730 comp rx=0 tx=0).stop 2024-09-19T09:07:43.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.799+0000 7f539188d740 1 -- 172.21.15.120:0/3817978095 shutdown_connections 2024-09-19T09:07:43.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.799+0000 7f539188d740 1 --2- 172.21.15.120:0/3817978095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557524be8d40 0x557524c00350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.799+0000 7f539188d740 1 --2- 172.21.15.120:0/3817978095 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557524bde480 0x557524be7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.799+0000 7f539188d740 1 --2- 172.21.15.120:0/3817978095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557524bddb60 0x557524be05c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.799+0000 7f539188d740 1 -- 172.21.15.120:0/3817978095 >> 172.21.15.120:0/3817978095 conn(0x557524bda0f0 msgr2=0x557524be4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:43.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.799+0000 7f539188d740 1 -- 172.21.15.120:0/3817978095 shutdown_connections 2024-09-19T09:07:43.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.799+0000 7f539188d740 1 -- 172.21.15.120:0/3817978095 wait complete. 2024-09-19T09:07:43.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.799+0000 7f539188d740 1 -- start start 2024-09-19T09:07:43.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.800+0000 7f539188d740 1 Processor -- start 2024-09-19T09:07:43.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.800+0000 7f539188d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557524bddb60 0x557524bf1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:43.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.800+0000 7f539188d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557524bde480 0x557524bec830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:43.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.800+0000 7f539188d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557524be8d40 0x557524becdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:43.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.800+0000 7f539188d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f53881b0c10 con 0x557524bde480 2024-09-19T09:07:43.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.800+0000 7f539188d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f53881b0540 con 0x557524be8d40 2024-09-19T09:07:43.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.800+0000 7f539188d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f53881ad760 con 0x557524bddb60 2024-09-19T09:07:43.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.800+0000 7f539188d740 10 client.0 _ll_register_callbacks cb 0x7f537800b9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:43.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.800+0000 7f538d250640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557524bde480 0x557524bec830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:43.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.800+0000 7f538d250640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557524bde480 0x557524bec830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56346/0 (socket says 172.21.15.120:56346) 2024-09-19T09:07:43.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.800+0000 7f538d250640 1 -- 172.21.15.120:0/2537934864 learned_addr learned my addr 172.21.15.120:0/2537934864 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:43.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.800+0000 7f538ca4f640 1 --2- 172.21.15.120:0/2537934864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557524be8d40 0x557524becdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:43.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.800+0000 7f538ca4f640 1 --2- 172.21.15.120:0/2537934864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557524bddb60 0x557524bf1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:43.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.801+0000 7f538d250640 1 -- 172.21.15.120:0/2537934864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557524bddb60 msgr2=0x557524bf1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:43.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.801+0000 7f538d250640 1 --2- 172.21.15.120:0/2537934864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557524bddb60 0x557524bf1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.801+0000 7f538d250640 1 -- 172.21.15.120:0/2537934864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557524be8d40 msgr2=0x557524becdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:43.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.801+0000 7f538d250640 1 --2- 172.21.15.120:0/2537934864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557524be8d40 0x557524becdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.801+0000 7f538d250640 1 -- 172.21.15.120:0/2537934864 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557524bed5e0 con 0x557524bde480 2024-09-19T09:07:43.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.801+0000 7f538ca4f640 1 --2- 172.21.15.120:0/2537934864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557524be8d40 0x557524becdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:07:43.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.801+0000 7f538d250640 1 --2- 172.21.15.120:0/2537934864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557524bde480 0x557524bec830 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f5384010ca0 tx=0x7f53840107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b56b5e9063b4f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:43.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.801+0000 7f5370ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:07:43.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.801+0000 7f5370ff9640 1 -- 172.21.15.120:0/2537934864 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384035070 con 0x557524bde480 2024-09-19T09:07:43.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.801+0000 7f5370ff9640 1 -- 172.21.15.120:0/2537934864 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f5384005340 con 0x557524bde480 2024-09-19T09:07:43.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.801+0000 7f539188d740 1 -- 172.21.15.120:0/2537934864 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53780047a0 con 0x557524bde480 2024-09-19T09:07:43.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.801+0000 7f5370ff9640 1 -- 172.21.15.120:0/2537934864 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f538400b410 con 0x557524bde480 2024-09-19T09:07:43.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.801+0000 7f539188d740 10 client.17208 fetch_fsmap learned FSMap version 30 2024-09-19T09:07:43.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.801+0000 7f539188d740 1 -- 172.21.15.120:0/2537934864 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557524c63400 con 0x557524bde480 2024-09-19T09:07:43.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.801+0000 7f5370ff9640 1 -- 172.21.15.120:0/2537934864 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53780047a0 con 0x557524bde480 2024-09-19T09:07:43.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.802+0000 7f5370ff9640 1 -- 172.21.15.120:0/2537934864 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f538402f050 con 0x557524bde480 2024-09-19T09:07:43.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.802+0000 7f539188d740 10 client.17208 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:07:43.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.802+0000 7f539188d740 10 client.17208 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:07:43.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.802+0000 7f539188d740 20 client.17208 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:43.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.802+0000 7f539188d740 1 --2- 172.21.15.120:0/2537934864 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557524c63af0 0x557524c65f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:43.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.802+0000 7f539188d740 4 client.17208 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:07:43.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.802+0000 7f539188d740 1 -- 172.21.15.120:0/2537934864 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557524b4ad00 con 0x557524c63af0 2024-09-19T09:07:43.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.802+0000 7f538ca4f640 1 --2- 172.21.15.120:0/2537934864 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557524c63af0 0x557524c65f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:43.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.802+0000 7f5370ff9640 10 client.17208 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:43.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.802+0000 7f538ca4f640 1 --2- 172.21.15.120:0/2537934864 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557524c63af0 0x557524c65f10 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=265129fb9bf27fcd server_cookie=3d59b3f02302d740 in_seq=0 out_seq=0 2024-09-19T09:07:43.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.804+0000 7f5370ff9640 1 -- 172.21.15.120:0/2537934864 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557524b4ad00 con 0x557524c63af0 2024-09-19T09:07:43.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.804+0000 7f5370ff9640 10 client.17208 handle_command_reply: tid=0 2024-09-19T09:07:43.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.804+0000 7f539188d740 1 client.17208 shutdown 2024-09-19T09:07:43.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.805+0000 7f539188d740 1 -- 172.21.15.120:0/2537934864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557524bde480 msgr2=0x557524bec830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:43.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.805+0000 7f539188d740 1 --2- 172.21.15.120:0/2537934864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557524bde480 0x557524bec830 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f5384010ca0 tx=0x7f53840107a0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.805+0000 7f539188d740 1 -- 172.21.15.120:0/2537934864 shutdown_connections 2024-09-19T09:07:43.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.805+0000 7f539188d740 1 --2- 172.21.15.120:0/2537934864 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557524c63af0 0x557524c65f10 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.805+0000 7f539188d740 1 --2- 172.21.15.120:0/2537934864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557524be8d40 0x557524becdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.805+0000 7f539188d740 1 --2- 172.21.15.120:0/2537934864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557524bde480 0x557524bec830 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.805+0000 7f5370ff9640 0 client.17208 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:43.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.805+0000 7f539188d740 1 --2- 172.21.15.120:0/2537934864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557524bddb60 0x557524bf1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.805+0000 7f539188d740 1 -- 172.21.15.120:0/2537934864 >> 172.21.15.120:0/2537934864 conn(0x557524bda0f0 msgr2=0x557524be4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:43.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.805+0000 7f539188d740 1 -- 172.21.15.120:0/2537934864 shutdown_connections 2024-09-19T09:07:43.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.805+0000 7f539188d740 1 -- 172.21.15.120:0/2537934864 wait complete. 2024-09-19T09:07:43.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.805+0000 7f539188d740 20 client.17208 trim_cache size 0 max 16384 2024-09-19T09:07:43.816 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:07:43.816 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:07:43.816 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "c41431e6-c965-492c-9325-e798620ad87b", 2024-09-19T09:07:43.816 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:07:43.817 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:07:43.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.806+0000 7f538f4b4640 1 -- 172.21.15.120:0/1544274263 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f53740a32c0 msgr2=0x7f53740a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:43.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.806+0000 7f538f4b4640 1 --2- 172.21.15.120:0/1544274263 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f53740a32c0 0x7f53740a5780 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f53800b9d60 tx=0x7f538003e040 comp rx=0 tx=0).stop 2024-09-19T09:07:43.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.806+0000 7f538f4b4640 1 -- 172.21.15.120:0/1544274263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5388106f10 msgr2=0x7f5388074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:43.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.806+0000 7f538f4b4640 1 --2- 172.21.15.120:0/1544274263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5388106f10 0x7f5388074f70 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f5380003f10 tx=0x7f5380003f40 comp rx=0 tx=0).stop 2024-09-19T09:07:43.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.806+0000 7f538f4b4640 1 -- 172.21.15.120:0/1544274263 shutdown_connections 2024-09-19T09:07:43.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.806+0000 7f538f4b4640 1 --2- 172.21.15.120:0/1544274263 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f53740a32c0 0x7f53740a5780 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.806+0000 7f538f4b4640 1 --2- 172.21.15.120:0/1544274263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5388106f10 0x7f5388074f70 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.806+0000 7f538f4b4640 1 --2- 172.21.15.120:0/1544274263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5388102790 0x7f5388074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.806+0000 7f538f4b4640 1 --2- 172.21.15.120:0/1544274263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5388101de0 0x7f53880744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:43.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.806+0000 7f538f4b4640 1 -- 172.21.15.120:0/1544274263 >> 172.21.15.120:0/1544274263 conn(0x7f53880fd8c0 msgr2=0x7f53880febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:43.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.806+0000 7f538f4b4640 1 -- 172.21.15.120:0/1544274263 shutdown_connections 2024-09-19T09:07:43.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:43.806+0000 7f538f4b4640 1 -- 172.21.15.120:0/1544274263 wait complete. 2024-09-19T09:07:44.047 INFO:tasks.workunit.client.0.smithi120.stdout: 1 372259 7.87 MB/sec execute 99 sec latency 40.183 ms 2024-09-19T09:07:44.060 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:07:44.714 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:07:44.963 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:44 smithi120 ceph-mon[31150]: pgmap v488: 97 pgs: 97 active+clean; 28 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 146 op/s 2024-09-19T09:07:44.963 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:44 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:07:44.963 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:44 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:07:44.964 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:44 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:07:44.964 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:44 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:07:45.047 INFO:tasks.workunit.client.0.smithi120.stdout: 1 374839 7.91 MB/sec execute 100 sec latency 31.545 ms 2024-09-19T09:07:45.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:44 smithi196 ceph-mon[35150]: pgmap v488: 97 pgs: 97 active+clean; 28 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 146 op/s 2024-09-19T09:07:45.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:44 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:07:45.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:44 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:07:45.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:44 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:07:45.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:44 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:07:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:44 smithi153 ceph-mon[35071]: pgmap v488: 97 pgs: 97 active+clean; 28 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 146 op/s 2024-09-19T09:07:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:44 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:07:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:44 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:07:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:44 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:07:45.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:44 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:07:45.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.198+0000 7f815a2bc640 1 -- 172.21.15.120:0/1842112628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8154106a80 msgr2=0x7f8154106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:45.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.198+0000 7f815a2bc640 1 --2- 172.21.15.120:0/1842112628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8154106a80 0x7f8154106e60 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f8144009930 tx=0x7f814402a420 comp rx=0 tx=0).stop 2024-09-19T09:07:45.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.198+0000 7f815a2bc640 1 -- 172.21.15.120:0/1842112628 shutdown_connections 2024-09-19T09:07:45.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.198+0000 7f815a2bc640 1 --2- 172.21.15.120:0/1842112628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8154101420 0x7f815410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.198+0000 7f815a2bc640 1 --2- 172.21.15.120:0/1842112628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8154100a80 0x7f8154100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.198+0000 7f815a2bc640 1 --2- 172.21.15.120:0/1842112628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8154106a80 0x7f8154106e60 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.198+0000 7f815a2bc640 1 -- 172.21.15.120:0/1842112628 >> 172.21.15.120:0/1842112628 conn(0x7f81540fc7a0 msgr2=0x7f81540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:45.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.198+0000 7f815a2bc640 1 -- 172.21.15.120:0/1842112628 shutdown_connections 2024-09-19T09:07:45.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.198+0000 7f815a2bc640 1 -- 172.21.15.120:0/1842112628 wait complete. 2024-09-19T09:07:45.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.199+0000 7f815a2bc640 1 Processor -- start 2024-09-19T09:07:45.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.199+0000 7f815a2bc640 1 -- start start 2024-09-19T09:07:45.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.199+0000 7f815a2bc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8154100a80 0x7f8154197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.199+0000 7f815a2bc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8154101420 0x7f8154197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.199+0000 7f815a2bc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8154106a80 0x7f8154197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.199+0000 7f815a2bc640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8154110e80 con 0x7f8154106a80 2024-09-19T09:07:45.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.199+0000 7f815a2bc640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8154110d00 con 0x7f8154100a80 2024-09-19T09:07:45.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.199+0000 7f815a2bc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8154111000 con 0x7f8154101420 2024-09-19T09:07:45.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.200+0000 7f8153fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8154100a80 0x7f8154197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.200+0000 7f8158859640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8154106a80 0x7f8154197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.200+0000 7f8153fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8154100a80 0x7f8154197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:40396/0 (socket says 172.21.15.120:40396) 2024-09-19T09:07:45.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.200+0000 7f8153fff640 1 -- 172.21.15.120:0/474604368 learned_addr learned my addr 172.21.15.120:0/474604368 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:45.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.200+0000 7f81537fe640 1 --2- 172.21.15.120:0/474604368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8154101420 0x7f8154197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.200+0000 7f8153fff640 1 -- 172.21.15.120:0/474604368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8154101420 msgr2=0x7f8154197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:45.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.200+0000 7f8153fff640 1 --2- 172.21.15.120:0/474604368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8154101420 0x7f8154197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.200+0000 7f8153fff640 1 -- 172.21.15.120:0/474604368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8154106a80 msgr2=0x7f8154197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:45.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.200+0000 7f8153fff640 1 --2- 172.21.15.120:0/474604368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8154106a80 0x7f8154197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.200+0000 7f8153fff640 1 -- 172.21.15.120:0/474604368 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81541a6970 con 0x7f8154100a80 2024-09-19T09:07:45.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.200+0000 7f81537fe640 1 --2- 172.21.15.120:0/474604368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8154101420 0x7f8154197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:07:45.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.200+0000 7f8153fff640 1 --2- 172.21.15.120:0/474604368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8154100a80 0x7f8154197100 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8144009230 tx=0x7f81440040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:45.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.200+0000 7f81517fa640 1 -- 172.21.15.120:0/474604368 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814403f030 con 0x7f8154100a80 2024-09-19T09:07:45.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.200+0000 7f815a2bc640 1 -- 172.21.15.120:0/474604368 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81541a6c00 con 0x7f8154100a80 2024-09-19T09:07:45.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.200+0000 7f81517fa640 1 -- 172.21.15.120:0/474604368 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8144003ad0 con 0x7f8154100a80 2024-09-19T09:07:45.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.200+0000 7f81517fa640 1 -- 172.21.15.120:0/474604368 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8144035070 con 0x7f8154100a80 2024-09-19T09:07:45.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.200+0000 7f815a2bc640 1 -- 172.21.15.120:0/474604368 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81541a7130 con 0x7f8154100a80 2024-09-19T09:07:45.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.203+0000 7f815c695740 1 Processor -- start 2024-09-19T09:07:45.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.203+0000 7f81517fa640 1 -- 172.21.15.120:0/474604368 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f8144042030 con 0x7f8154100a80 2024-09-19T09:07:45.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.203+0000 7f815c695740 1 -- start start 2024-09-19T09:07:45.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.203+0000 7f815c695740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5602e726f760 0x5602e726fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.203+0000 7f815c695740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602e7270080 0x5602e727a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.203+0000 7f815c695740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5602e727a940 0x5602e727cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.203+0000 7f815c695740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5602e71d52a0 con 0x5602e7270080 2024-09-19T09:07:45.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.203+0000 7f815c695740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5602e717cb10 con 0x5602e727a940 2024-09-19T09:07:45.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.203+0000 7f815c695740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5602e71d1260 con 0x5602e726f760 2024-09-19T09:07:45.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.203+0000 7f8153fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602e7270080 0x5602e727a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.203+0000 7f8153fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602e7270080 0x5602e727a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56372/0 (socket says 172.21.15.120:56372) 2024-09-19T09:07:45.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.203+0000 7f8153fff640 1 -- 172.21.15.120:0/1646430327 learned_addr learned my addr 172.21.15.120:0/1646430327 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:45.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.203+0000 7f8158859640 1 --2- 172.21.15.120:0/1646430327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5602e726f760 0x5602e726fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.203+0000 7f81537fe640 1 --2- 172.21.15.120:0/1646430327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5602e727a940 0x5602e727cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.204+0000 7f8153fff640 1 -- 172.21.15.120:0/1646430327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5602e726f760 msgr2=0x5602e726fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:45.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.204+0000 7f8153fff640 1 --2- 172.21.15.120:0/1646430327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5602e726f760 0x5602e726fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.204+0000 7f8153fff640 1 -- 172.21.15.120:0/1646430327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5602e727a940 msgr2=0x5602e727cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:45.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.204+0000 7f8153fff640 1 --2- 172.21.15.120:0/1646430327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5602e727a940 0x5602e727cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.204+0000 7f8153fff640 1 -- 172.21.15.120:0/1646430327 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602e6f46b90 con 0x5602e7270080 2024-09-19T09:07:45.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.204+0000 7f81517fa640 1 --2- 172.21.15.120:0/474604368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f81400a3240 0x7f81400a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.204+0000 7f8153fff640 1 --2- 172.21.15.120:0/1646430327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602e7270080 0x5602e727a400 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f81440f3ef0 tx=0x7f81440f8440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d44aa7bd94a447 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:45.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.204+0000 7f8158859640 1 --2- 172.21.15.120:0/474604368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f81400a3240 0x7f81400a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.204+0000 7f81517fa640 1 -- 172.21.15.120:0/474604368 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f81440efd80 con 0x7f8154100a80 2024-09-19T09:07:45.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.204+0000 7f814a7fc640 1 -- 172.21.15.120:0/1646430327 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81440b8640 con 0x5602e7270080 2024-09-19T09:07:45.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.204+0000 7f814a7fc640 1 -- 172.21.15.120:0/1646430327 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81440bcaa0 con 0x5602e7270080 2024-09-19T09:07:45.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.204+0000 7f814a7fc640 1 -- 172.21.15.120:0/1646430327 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81440bbd00 con 0x5602e7270080 2024-09-19T09:07:45.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.205+0000 7f815c695740 1 -- 172.21.15.120:0/1646430327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602e7270080 msgr2=0x5602e727a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:45.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.205+0000 7f8158859640 1 --2- 172.21.15.120:0/474604368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f81400a3240 0x7f81400a5700 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f814c005fd0 tx=0x7f814c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:45.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.205+0000 7f815c695740 1 --2- 172.21.15.120:0/1646430327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602e7270080 0x5602e727a400 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f81440f3ef0 tx=0x7f81440f8440 comp rx=0 tx=0).stop 2024-09-19T09:07:45.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.205+0000 7f815c695740 1 -- 172.21.15.120:0/1646430327 shutdown_connections 2024-09-19T09:07:45.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.205+0000 7f815c695740 1 --2- 172.21.15.120:0/1646430327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5602e727a940 0x5602e727cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.205+0000 7f815c695740 1 --2- 172.21.15.120:0/1646430327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602e7270080 0x5602e727a400 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.205+0000 7f815c695740 1 --2- 172.21.15.120:0/1646430327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5602e726f760 0x5602e726fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.205+0000 7f815c695740 1 -- 172.21.15.120:0/1646430327 >> 172.21.15.120:0/1646430327 conn(0x5602e726bcf0 msgr2=0x5602e726c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:45.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.205+0000 7f815c695740 1 -- 172.21.15.120:0/1646430327 shutdown_connections 2024-09-19T09:07:45.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.205+0000 7f815c695740 1 -- 172.21.15.120:0/1646430327 wait complete. 2024-09-19T09:07:45.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.205+0000 7f815c695740 1 -- start start 2024-09-19T09:07:45.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.205+0000 7f815c695740 1 Processor -- start 2024-09-19T09:07:45.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.206+0000 7f815c695740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602e726f760 0x5602e7277740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.206+0000 7f815c695740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5602e7270080 0x5602e7277c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.206+0000 7f815c695740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5602e727a940 0x5602e72781c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.206+0000 7f815c695740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5602e727d2d0 con 0x5602e726f760 2024-09-19T09:07:45.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.206+0000 7f815c695740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5602e715d6e0 con 0x5602e727a940 2024-09-19T09:07:45.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.206+0000 7f815c695740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5602e727d450 con 0x5602e7270080 2024-09-19T09:07:45.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.206+0000 7f815c695740 10 client.0 _ll_register_callbacks cb 0x5602e7181a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:45.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.206+0000 7f8153fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602e726f760 0x5602e7277740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.206+0000 7f8153fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602e726f760 0x5602e7277740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56378/0 (socket says 172.21.15.120:56378) 2024-09-19T09:07:45.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.206+0000 7f8153fff640 1 -- 172.21.15.120:0/2991123404 learned_addr learned my addr 172.21.15.120:0/2991123404 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:45.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.206+0000 7f8158859640 1 --2- 172.21.15.120:0/2991123404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5602e727a940 0x5602e72781c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.206+0000 7f81537fe640 1 --2- 172.21.15.120:0/2991123404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5602e7270080 0x5602e7277c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.206+0000 7f8153fff640 1 -- 172.21.15.120:0/2991123404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5602e7270080 msgr2=0x5602e7277c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:45.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.206+0000 7f8153fff640 1 --2- 172.21.15.120:0/2991123404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5602e7270080 0x5602e7277c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.206+0000 7f8153fff640 1 -- 172.21.15.120:0/2991123404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5602e727a940 msgr2=0x5602e72781c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:45.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.206+0000 7f8153fff640 1 --2- 172.21.15.120:0/2991123404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5602e727a940 0x5602e72781c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.206+0000 7f8153fff640 1 -- 172.21.15.120:0/2991123404 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602e7286700 con 0x5602e726f760 2024-09-19T09:07:45.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.206+0000 7f8153fff640 1 --2- 172.21.15.120:0/2991123404 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602e726f760 0x5602e7277740 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f8144006010 tx=0x7f8144031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781b2ea8a0c00a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:45.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.206+0000 7f81337fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:07:45.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.206+0000 7f81337fe640 1 -- 172.21.15.120:0/2991123404 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81440bbee0 con 0x5602e726f760 2024-09-19T09:07:45.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.206+0000 7f81337fe640 1 -- 172.21.15.120:0/2991123404 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f814403b460 con 0x5602e726f760 2024-09-19T09:07:45.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.206+0000 7f815c695740 1 -- 172.21.15.120:0/2991123404 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81440bcaa0 con 0x5602e726f760 2024-09-19T09:07:45.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.207+0000 7f81337fe640 1 -- 172.21.15.120:0/2991123404 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f81440bc170 con 0x5602e726f760 2024-09-19T09:07:45.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.207+0000 7f815c695740 10 client.17220 fetch_fsmap learned FSMap version 30 2024-09-19T09:07:45.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.207+0000 7f815c695740 1 -- 172.21.15.120:0/2991123404 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5602e727d2d0 con 0x5602e726f760 2024-09-19T09:07:45.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.207+0000 7f81337fe640 1 -- 172.21.15.120:0/2991123404 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81440bcaa0 con 0x5602e726f760 2024-09-19T09:07:45.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.207+0000 7f81337fe640 1 -- 172.21.15.120:0/2991123404 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f814403bc50 con 0x5602e726f760 2024-09-19T09:07:45.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.207+0000 7f815c695740 10 client.17220 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:07:45.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.207+0000 7f815c695740 10 client.17220 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:07:45.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.207+0000 7f815c695740 20 client.17220 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:45.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.207+0000 7f815c695740 1 --2- 172.21.15.120:0/2991123404 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5602e728a250 0x5602e728c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.207+0000 7f815c695740 4 client.17220 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:07:45.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.207+0000 7f815c695740 1 -- 172.21.15.120:0/2991123404 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602e71e29d0 con 0x5602e728a250 2024-09-19T09:07:45.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.207+0000 7f81537fe640 1 --2- 172.21.15.120:0/2991123404 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5602e728a250 0x5602e728c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.208+0000 7f81537fe640 1 --2- 172.21.15.120:0/2991123404 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5602e728a250 0x5602e728c6b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e43a525f510628b3 server_cookie=a5ed99bd8c5d20ab in_seq=0 out_seq=0 2024-09-19T09:07:45.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.208+0000 7f81337fe640 10 client.17220 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:45.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.209+0000 7f81337fe640 1 -- 172.21.15.120:0/2991123404 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5602e71e29d0 con 0x5602e728a250 2024-09-19T09:07:45.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.209+0000 7f81337fe640 10 client.17220 handle_command_reply: tid=0 2024-09-19T09:07:45.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.209+0000 7f815c695740 1 client.17220 shutdown 2024-09-19T09:07:45.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.210+0000 7f815c695740 1 -- 172.21.15.120:0/2991123404 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602e726f760 msgr2=0x5602e7277740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:45.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.210+0000 7f815c695740 1 --2- 172.21.15.120:0/2991123404 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602e726f760 0x5602e7277740 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f8144006010 tx=0x7f8144031400 comp rx=0 tx=0).stop 2024-09-19T09:07:45.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.210+0000 7f815c695740 1 -- 172.21.15.120:0/2991123404 shutdown_connections 2024-09-19T09:07:45.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.210+0000 7f815c695740 1 --2- 172.21.15.120:0/2991123404 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5602e728a250 0x5602e728c6b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.210+0000 7f81337fe640 0 client.17220 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:45.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.210+0000 7f815c695740 1 --2- 172.21.15.120:0/2991123404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5602e727a940 0x5602e72781c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.210+0000 7f815c695740 1 --2- 172.21.15.120:0/2991123404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5602e7270080 0x5602e7277c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.210+0000 7f815c695740 1 --2- 172.21.15.120:0/2991123404 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602e726f760 0x5602e7277740 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.210+0000 7f815c695740 1 -- 172.21.15.120:0/2991123404 >> 172.21.15.120:0/2991123404 conn(0x5602e726bcf0 msgr2=0x5602e72719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:45.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.210+0000 7f815c695740 1 -- 172.21.15.120:0/2991123404 shutdown_connections 2024-09-19T09:07:45.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.210+0000 7f815c695740 1 -- 172.21.15.120:0/2991123404 wait complete. 2024-09-19T09:07:45.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.210+0000 7f815c695740 20 client.17220 trim_cache size 0 max 16384 2024-09-19T09:07:45.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.227+0000 7f815c695740 1 Processor -- start 2024-09-19T09:07:45.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.227+0000 7f815c695740 1 -- start start 2024-09-19T09:07:45.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.227+0000 7f815c695740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5602e726f760 0x5602e7291fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.227+0000 7f815c695740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5602e7270080 0x5602e7292500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.227+0000 7f815c695740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602e727a940 0x5602e72749a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.227+0000 7f815c695740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f81541af780 con 0x5602e727a940 2024-09-19T09:07:45.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.227+0000 7f815c695740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f81541b12c0 con 0x5602e726f760 2024-09-19T09:07:45.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.227+0000 7f815c695740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f81541afe50 con 0x5602e7270080 2024-09-19T09:07:45.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.228+0000 7f8153fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5602e726f760 0x5602e7291fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.228+0000 7f8158859640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602e727a940 0x5602e72749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.228+0000 7f81537fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5602e7270080 0x5602e7292500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.228+0000 7f8158859640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602e727a940 0x5602e72749a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56394/0 (socket says 172.21.15.120:56394) 2024-09-19T09:07:45.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.228+0000 7f8153fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5602e726f760 0x5602e7291fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:40416/0 (socket says 172.21.15.120:40416) 2024-09-19T09:07:45.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.228+0000 7f81537fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5602e7270080 0x5602e7292500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:48838/0 (socket says 172.21.15.120:48838) 2024-09-19T09:07:45.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.228+0000 7f8158859640 1 -- 172.21.15.120:0/724261779 learned_addr learned my addr 172.21.15.120:0/724261779 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:45.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.228+0000 7f8153fff640 1 -- 172.21.15.120:0/724261779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5602e7270080 msgr2=0x5602e7292500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:45.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.228+0000 7f8153fff640 1 --2- 172.21.15.120:0/724261779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5602e7270080 0x5602e7292500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.228+0000 7f8153fff640 1 -- 172.21.15.120:0/724261779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602e727a940 msgr2=0x5602e72749a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:45.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.228+0000 7f8153fff640 1 --2- 172.21.15.120:0/724261779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602e727a940 0x5602e72749a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.228+0000 7f8153fff640 1 -- 172.21.15.120:0/724261779 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602e726bb20 con 0x5602e726f760 2024-09-19T09:07:45.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.228+0000 7f8158859640 1 --2- 172.21.15.120:0/724261779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602e727a940 0x5602e72749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:07:45.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.228+0000 7f8153fff640 1 --2- 172.21.15.120:0/724261779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5602e726f760 0x5602e7291fc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f81440ba650 tx=0x7f814403b720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5495aff90c5536e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:45.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.228+0000 7f8149ffb640 1 -- 172.21.15.120:0/724261779 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81440bbee0 con 0x5602e726f760 2024-09-19T09:07:45.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.228+0000 7f8149ffb640 1 -- 172.21.15.120:0/724261779 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81440f8940 con 0x5602e726f760 2024-09-19T09:07:45.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.228+0000 7f8149ffb640 1 -- 172.21.15.120:0/724261779 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81440bbd00 con 0x5602e726f760 2024-09-19T09:07:45.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.229+0000 7f815c695740 1 -- 172.21.15.120:0/724261779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5602e726f760 msgr2=0x5602e7291fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:45.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.229+0000 7f815c695740 1 --2- 172.21.15.120:0/724261779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5602e726f760 0x5602e7291fc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f81440ba650 tx=0x7f814403b720 comp rx=0 tx=0).stop 2024-09-19T09:07:45.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.229+0000 7f815c695740 1 -- 172.21.15.120:0/724261779 shutdown_connections 2024-09-19T09:07:45.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.229+0000 7f815c695740 1 --2- 172.21.15.120:0/724261779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602e727a940 0x5602e72749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.229+0000 7f815c695740 1 --2- 172.21.15.120:0/724261779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5602e7270080 0x5602e7292500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.229+0000 7f815c695740 1 --2- 172.21.15.120:0/724261779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5602e726f760 0x5602e7291fc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.229+0000 7f815c695740 1 -- 172.21.15.120:0/724261779 >> 172.21.15.120:0/724261779 conn(0x5602e726bcf0 msgr2=0x5602e7276010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:45.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.229+0000 7f815c695740 1 -- 172.21.15.120:0/724261779 shutdown_connections 2024-09-19T09:07:45.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.229+0000 7f815c695740 1 -- 172.21.15.120:0/724261779 wait complete. 2024-09-19T09:07:45.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.230+0000 7f815c695740 1 -- start start 2024-09-19T09:07:45.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.230+0000 7f815c695740 1 Processor -- start 2024-09-19T09:07:45.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.230+0000 7f815c695740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5602e726f760 0x5602e7282cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.230+0000 7f815c695740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602e7270080 0x5602e727d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.230+0000 7f815c695740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5602e727a940 0x5602e727dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.230+0000 7f815c695740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f81541b0bf0 con 0x5602e7270080 2024-09-19T09:07:45.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.230+0000 7f815c695740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f81541b0520 con 0x5602e727a940 2024-09-19T09:07:45.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.230+0000 7f815c695740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f81541ad740 con 0x5602e726f760 2024-09-19T09:07:45.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.230+0000 7f815c695740 10 client.0 _ll_register_callbacks cb 0x7f81440f3d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:45.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.230+0000 7f8158859640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5602e727a940 0x5602e727dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.230+0000 7f81537fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602e7270080 0x5602e727d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.230+0000 7f8158859640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5602e727a940 0x5602e727dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:40432/0 (socket says 172.21.15.120:40432) 2024-09-19T09:07:45.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.230+0000 7f8158859640 1 -- 172.21.15.120:0/2403544518 learned_addr learned my addr 172.21.15.120:0/2403544518 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:45.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.230+0000 7f8153fff640 1 --2- 172.21.15.120:0/2403544518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5602e726f760 0x5602e7282cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.231+0000 7f81537fe640 1 -- 172.21.15.120:0/2403544518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5602e726f760 msgr2=0x5602e7282cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:45.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.231+0000 7f81537fe640 1 --2- 172.21.15.120:0/2403544518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5602e726f760 0x5602e7282cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.231+0000 7f81537fe640 1 -- 172.21.15.120:0/2403544518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5602e727a940 msgr2=0x5602e727dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:45.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.231+0000 7f81537fe640 1 --2- 172.21.15.120:0/2403544518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5602e727a940 0x5602e727dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.231+0000 7f81537fe640 1 -- 172.21.15.120:0/2403544518 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602e7286530 con 0x5602e7270080 2024-09-19T09:07:45.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.231+0000 7f8158859640 1 --2- 172.21.15.120:0/2403544518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5602e727a940 0x5602e727dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:07:45.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.231+0000 7f8153fff640 1 --2- 172.21.15.120:0/2403544518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5602e726f760 0x5602e7282cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:07:45.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.231+0000 7f81537fe640 1 --2- 172.21.15.120:0/2403544518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602e7270080 0x5602e727d660 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x5602e7278d80 tx=0x7f813c014900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d24e2538c8d1dc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:45.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.231+0000 7f81337fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:07:45.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.231+0000 7f81337fe640 1 -- 172.21.15.120:0/2403544518 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c005250 con 0x5602e7270080 2024-09-19T09:07:45.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.231+0000 7f81337fe640 1 -- 172.21.15.120:0/2403544518 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f813c0053f0 con 0x5602e7270080 2024-09-19T09:07:45.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.231+0000 7f815c695740 1 -- 172.21.15.120:0/2403544518 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81440f8940 con 0x5602e7270080 2024-09-19T09:07:45.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.232+0000 7f81337fe640 1 -- 172.21.15.120:0/2403544518 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f813c0055f0 con 0x5602e7270080 2024-09-19T09:07:45.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.232+0000 7f815c695740 10 client.17226 fetch_fsmap learned FSMap version 30 2024-09-19T09:07:45.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.232+0000 7f815c695740 1 -- 172.21.15.120:0/2403544518 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5602e7267260 con 0x5602e7270080 2024-09-19T09:07:45.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.232+0000 7f81337fe640 1 -- 172.21.15.120:0/2403544518 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81440f8940 con 0x5602e7270080 2024-09-19T09:07:45.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.232+0000 7f81337fe640 1 -- 172.21.15.120:0/2403544518 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f813c022020 con 0x5602e7270080 2024-09-19T09:07:45.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.232+0000 7f815c695740 10 client.17226 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:07:45.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.232+0000 7f815c695740 10 client.17226 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:07:45.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.232+0000 7f815c695740 20 client.17226 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:45.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.232+0000 7f815c695740 1 --2- 172.21.15.120:0/2403544518 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5602e728a250 0x5602e72f8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.232+0000 7f815c695740 4 client.17226 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}] 2024-09-19T09:07:45.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.232+0000 7f815c695740 1 -- 172.21.15.120:0/2403544518 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}) -- 0x5602e71dcd20 con 0x5602e728a250 2024-09-19T09:07:45.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.232+0000 7f8153fff640 1 --2- 172.21.15.120:0/2403544518 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5602e728a250 0x5602e72f8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.233+0000 7f81337fe640 10 client.17226 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:45.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.233+0000 7f8153fff640 1 --2- 172.21.15.120:0/2403544518 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5602e728a250 0x5602e72f8320 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bf56c5deb673491 server_cookie=9d58e7e7fa612e01 in_seq=0 out_seq=0 2024-09-19T09:07:45.264 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:07:45.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.341+0000 7f81337fe640 1 -- 172.21.15.120:0/2403544518 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5602e71dcd20 con 0x5602e728a250 2024-09-19T09:07:45.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.341+0000 7f81337fe640 10 client.17226 handle_command_reply: tid=0 2024-09-19T09:07:45.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.341+0000 7f815c695740 1 client.17226 shutdown 2024-09-19T09:07:45.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.341+0000 7f815c695740 1 -- 172.21.15.120:0/2403544518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602e7270080 msgr2=0x5602e727d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:45.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.341+0000 7f815c695740 1 --2- 172.21.15.120:0/2403544518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602e7270080 0x5602e727d660 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x5602e7278d80 tx=0x7f813c014900 comp rx=0 tx=0).stop 2024-09-19T09:07:45.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.341+0000 7f815c695740 1 -- 172.21.15.120:0/2403544518 shutdown_connections 2024-09-19T09:07:45.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.341+0000 7f815c695740 1 --2- 172.21.15.120:0/2403544518 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5602e728a250 0x5602e72f8320 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.341+0000 7f815c695740 1 --2- 172.21.15.120:0/2403544518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5602e727a940 0x5602e727dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.341+0000 7f815c695740 1 --2- 172.21.15.120:0/2403544518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5602e7270080 0x5602e727d660 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.341+0000 7f815c695740 1 --2- 172.21.15.120:0/2403544518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5602e726f760 0x5602e7282cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.341+0000 7f81337fe640 0 client.17226 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:45.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.341+0000 7f815c695740 1 -- 172.21.15.120:0/2403544518 >> 172.21.15.120:0/2403544518 conn(0x5602e726bcf0 msgr2=0x5602e726a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:45.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.342+0000 7f815c695740 1 -- 172.21.15.120:0/2403544518 shutdown_connections 2024-09-19T09:07:45.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.342+0000 7f815c695740 1 -- 172.21.15.120:0/2403544518 wait complete. 2024-09-19T09:07:45.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.342+0000 7f815c695740 20 client.17226 trim_cache size 0 max 16384 2024-09-19T09:07:45.344 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:07:45.344 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 30, 2024-09-19T09:07:45.344 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:07:45.344 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 22, 2024-09-19T09:07:45.344 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:07:45.344 INFO:teuthology.orchestra.run.smithi120.stdout: "75bdfb5": { 2024-09-19T09:07:45.344 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 22, 2024-09-19T09:07:45.344 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:07:45.344 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:07:45.344 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:07:45.344 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:07:45.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:07:45.345 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:07:45.345 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 149.8, 2024-09-19T09:07:45.345 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:07:45.345 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:07:45.345 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:07:45.345 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:07:45.345 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:07:45.345 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:07:45.345 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:07:45.345 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:07:45.345 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:07:45.345 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:07:45.345 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:07:45.345 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:07:45.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.342+0000 7f815a2bc640 1 -- 172.21.15.120:0/474604368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f81400a3240 msgr2=0x7f81400a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:45.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.342+0000 7f815a2bc640 1 --2- 172.21.15.120:0/474604368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f81400a3240 0x7f81400a5700 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f814c005fd0 tx=0x7f814c009500 comp rx=0 tx=0).stop 2024-09-19T09:07:45.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.342+0000 7f815a2bc640 1 -- 172.21.15.120:0/474604368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8154100a80 msgr2=0x7f8154197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:45.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.342+0000 7f815a2bc640 1 --2- 172.21.15.120:0/474604368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8154100a80 0x7f8154197100 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8144009230 tx=0x7f81440040c0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.342+0000 7f815a2bc640 1 -- 172.21.15.120:0/474604368 shutdown_connections 2024-09-19T09:07:45.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.342+0000 7f815a2bc640 1 --2- 172.21.15.120:0/474604368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f81400a3240 0x7f81400a5700 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.342+0000 7f815a2bc640 1 --2- 172.21.15.120:0/474604368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8154106a80 0x7f8154197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.342+0000 7f815a2bc640 1 --2- 172.21.15.120:0/474604368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8154101420 0x7f8154197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.342+0000 7f815a2bc640 1 --2- 172.21.15.120:0/474604368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8154100a80 0x7f8154197100 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.342+0000 7f815a2bc640 1 -- 172.21.15.120:0/474604368 >> 172.21.15.120:0/474604368 conn(0x7f81540fc7a0 msgr2=0x7f81540fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:45.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.342+0000 7f815a2bc640 1 -- 172.21.15.120:0/474604368 shutdown_connections 2024-09-19T09:07:45.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.342+0000 7f815a2bc640 1 -- 172.21.15.120:0/474604368 wait complete. 2024-09-19T09:07:45.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.777+0000 7f2714cf9640 1 -- 172.21.15.120:0/970497028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27101002f0 msgr2=0x7f27101006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:45.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.777+0000 7f2714cf9640 1 --2- 172.21.15.120:0/970497028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27101002f0 0x7f27101006d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f2700009930 tx=0x7f270002a420 comp rx=0 tx=0).stop 2024-09-19T09:07:45.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.777+0000 7f2714cf9640 1 -- 172.21.15.120:0/970497028 shutdown_connections 2024-09-19T09:07:45.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.777+0000 7f2714cf9640 1 --2- 172.21.15.120:0/970497028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2710106f50 0x7f271010eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.777+0000 7f2714cf9640 1 --2- 172.21.15.120:0/970497028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2710100ca0 0x7f2710106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.777+0000 7f2714cf9640 1 --2- 172.21.15.120:0/970497028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27101002f0 0x7f27101006d0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.777+0000 7f2714cf9640 1 -- 172.21.15.120:0/970497028 >> 172.21.15.120:0/970497028 conn(0x7f2710077210 msgr2=0x7f27100fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:45.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.777+0000 7f2714cf9640 1 -- 172.21.15.120:0/970497028 shutdown_connections 2024-09-19T09:07:45.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.777+0000 7f2714cf9640 1 -- 172.21.15.120:0/970497028 wait complete. 2024-09-19T09:07:45.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.778+0000 7f2714cf9640 1 Processor -- start 2024-09-19T09:07:45.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.778+0000 7f2714cf9640 1 -- start start 2024-09-19T09:07:45.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.778+0000 7f2714cf9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27101002f0 0x7f2710101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.778+0000 7f2714cf9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2710100ca0 0x7f27101017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.778+0000 7f2714cf9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2710106f50 0x7f2710101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.778+0000 7f2714cf9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2710111fa0 con 0x7f2710106f50 2024-09-19T09:07:45.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.778+0000 7f2714cf9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2710111e20 con 0x7f27101002f0 2024-09-19T09:07:45.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.778+0000 7f2714cf9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2710112120 con 0x7f2710100ca0 2024-09-19T09:07:45.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.778+0000 7f270ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2710106f50 0x7f2710101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.778+0000 7f270ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2710106f50 0x7f2710101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56418/0 (socket says 172.21.15.120:56418) 2024-09-19T09:07:45.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.778+0000 7f270ed9d640 1 -- 172.21.15.120:0/2715377580 learned_addr learned my addr 172.21.15.120:0/2715377580 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:45.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.778+0000 7f270e59c640 1 --2- 172.21.15.120:0/2715377580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27101002f0 0x7f2710101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.778+0000 7f270dd9b640 1 --2- 172.21.15.120:0/2715377580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2710100ca0 0x7f27101017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.778+0000 7f270ed9d640 1 -- 172.21.15.120:0/2715377580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2710100ca0 msgr2=0x7f27101017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:45.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.778+0000 7f270ed9d640 1 --2- 172.21.15.120:0/2715377580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2710100ca0 0x7f27101017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.778+0000 7f270ed9d640 1 -- 172.21.15.120:0/2715377580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27101002f0 msgr2=0x7f2710101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:45.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.778+0000 7f270ed9d640 1 --2- 172.21.15.120:0/2715377580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27101002f0 0x7f2710101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.778+0000 7f270ed9d640 1 -- 172.21.15.120:0/2715377580 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27101a69b0 con 0x7f2710106f50 2024-09-19T09:07:45.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.779+0000 7f270ed9d640 1 --2- 172.21.15.120:0/2715377580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2710106f50 0x7f2710101cf0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f270800eaa0 tx=0x7f270800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:45.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.779+0000 7f26ff7fe640 1 -- 172.21.15.120:0/2715377580 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270800cdb0 con 0x7f2710106f50 2024-09-19T09:07:45.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.779+0000 7f26ff7fe640 1 -- 172.21.15.120:0/2715377580 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2708005020 con 0x7f2710106f50 2024-09-19T09:07:45.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.779+0000 7f2714cf9640 1 -- 172.21.15.120:0/2715377580 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27101a6ca0 con 0x7f2710106f50 2024-09-19T09:07:45.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.779+0000 7f26ff7fe640 1 -- 172.21.15.120:0/2715377580 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27080116b0 con 0x7f2710106f50 2024-09-19T09:07:45.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.779+0000 7f2714cf9640 1 -- 172.21.15.120:0/2715377580 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27101a71e0 con 0x7f2710106f50 2024-09-19T09:07:45.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.781+0000 7f26ff7fe640 1 -- 172.21.15.120:0/2715377580 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f2708005560 con 0x7f2710106f50 2024-09-19T09:07:45.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.781+0000 7f27170d2740 1 Processor -- start 2024-09-19T09:07:45.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.781+0000 7f27170d2740 1 -- start start 2024-09-19T09:07:45.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.782+0000 7f27170d2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638b44ac4a0 0x5638b44ac880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.782+0000 7f27170d2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638b44acdc0 0x5638b44b7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.782+0000 7f27170d2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b44b7680 0x5638b44b9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.782+0000 7f27170d2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5638b44122e0 con 0x5638b44acdc0 2024-09-19T09:07:45.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.782+0000 7f27170d2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5638b43b9ce0 con 0x5638b44ac4a0 2024-09-19T09:07:45.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.782+0000 7f27170d2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5638b440e2a0 con 0x5638b44b7680 2024-09-19T09:07:45.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.782+0000 7f270e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638b44acdc0 0x5638b44b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.782+0000 7f270e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638b44acdc0 0x5638b44b7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56420/0 (socket says 172.21.15.120:56420) 2024-09-19T09:07:45.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.782+0000 7f270e59c640 1 -- 172.21.15.120:0/935755084 learned_addr learned my addr 172.21.15.120:0/935755084 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:45.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.782+0000 7f26ff7fe640 1 --2- 172.21.15.120:0/2715377580 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f26f80a3240 0x7f26f80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.782+0000 7f270dd9b640 1 --2- 172.21.15.120:0/935755084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638b44ac4a0 0x5638b44ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.782+0000 7f270ed9d640 1 --2- 172.21.15.120:0/2715377580 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f26f80a3240 0x7f26f80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.782+0000 7f270dd9b640 1 --2- 172.21.15.120:0/935755084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b44b7680 0x5638b44b9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.782+0000 7f270e59c640 1 -- 172.21.15.120:0/935755084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b44b7680 msgr2=0x5638b44b9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:45.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.782+0000 7f270e59c640 1 --2- 172.21.15.120:0/935755084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b44b7680 0x5638b44b9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.782+0000 7f270e59c640 1 -- 172.21.15.120:0/935755084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638b44ac4a0 msgr2=0x5638b44ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:45.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.782+0000 7f270e59c640 1 --2- 172.21.15.120:0/935755084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638b44ac4a0 0x5638b44ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.782+0000 7f270e59c640 1 -- 172.21.15.120:0/935755084 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638b439e440 con 0x5638b44acdc0 2024-09-19T09:07:45.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.782+0000 7f270dd9b640 1 --2- 172.21.15.120:0/935755084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b44b7680 0x5638b44b9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:07:45.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.782+0000 7f26ff7fe640 1 -- 172.21.15.120:0/2715377580 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f27080cfa40 con 0x7f2710106f50 2024-09-19T09:07:45.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.782+0000 7f270e59c640 1 --2- 172.21.15.120:0/935755084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638b44acdc0 0x5638b44b7140 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f270002a940 tx=0x7f2700005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d798ee41970b3c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:45.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.782+0000 7f26fcff9640 1 -- 172.21.15.120:0/935755084 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2700040070 con 0x5638b44acdc0 2024-09-19T09:07:45.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.783+0000 7f26fcff9640 1 -- 172.21.15.120:0/935755084 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27000319f0 con 0x5638b44acdc0 2024-09-19T09:07:45.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.783+0000 7f26fcff9640 1 -- 172.21.15.120:0/935755084 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2700035040 con 0x5638b44acdc0 2024-09-19T09:07:45.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.783+0000 7f270ed9d640 1 --2- 172.21.15.120:0/2715377580 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f26f80a3240 0x7f26f80a5700 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2708098570 tx=0x7f270806ef90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:45.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.783+0000 7f27170d2740 1 -- 172.21.15.120:0/935755084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638b44acdc0 msgr2=0x5638b44b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:45.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.783+0000 7f27170d2740 1 --2- 172.21.15.120:0/935755084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638b44acdc0 0x5638b44b7140 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f270002a940 tx=0x7f2700005c00 comp rx=0 tx=0).stop 2024-09-19T09:07:45.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.783+0000 7f27170d2740 1 -- 172.21.15.120:0/935755084 shutdown_connections 2024-09-19T09:07:45.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.783+0000 7f27170d2740 1 --2- 172.21.15.120:0/935755084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b44b7680 0x5638b44b9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.783+0000 7f27170d2740 1 --2- 172.21.15.120:0/935755084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638b44acdc0 0x5638b44b7140 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.783+0000 7f27170d2740 1 --2- 172.21.15.120:0/935755084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638b44ac4a0 0x5638b44ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.819 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:07:45.819 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:07:45.819 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-19T09:07:45.819 INFO:teuthology.orchestra.run.smithi120.stdout: "c41431e6-c965-492c-9325-e798620ad87b": { 2024-09-19T09:07:45.819 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-19T09:07:45.819 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "c41431e6-c965-492c-9325-e798620ad87b", 2024-09-19T09:07:45.819 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-19T09:07:45.819 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:07:45.820 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:07:45.820 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:07:45.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.783+0000 7f27170d2740 1 -- 172.21.15.120:0/935755084 >> 172.21.15.120:0/935755084 conn(0x5638b44a8a30 msgr2=0x5638b44a8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:45.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.783+0000 7f27170d2740 1 -- 172.21.15.120:0/935755084 shutdown_connections 2024-09-19T09:07:45.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.783+0000 7f27170d2740 1 -- 172.21.15.120:0/935755084 wait complete. 2024-09-19T09:07:45.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.784+0000 7f27170d2740 1 -- start start 2024-09-19T09:07:45.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.784+0000 7f27170d2740 1 Processor -- start 2024-09-19T09:07:45.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.784+0000 7f27170d2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638b44ac4a0 0x5638b44b6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.784+0000 7f27170d2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b44acdc0 0x5638b44b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.784+0000 7f27170d2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638b44b7680 0x5638b44b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.784+0000 7f27170d2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5638b44ba250 con 0x5638b44ac4a0 2024-09-19T09:07:45.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.784+0000 7f27170d2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5638b43cd050 con 0x5638b44b7680 2024-09-19T09:07:45.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.784+0000 7f27170d2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5638b44ba3d0 con 0x5638b44acdc0 2024-09-19T09:07:45.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.784+0000 7f27170d2740 10 client.0 _ll_register_callbacks cb 0x5638b440d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:45.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.784+0000 7f270dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638b44ac4a0 0x5638b44b6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.784+0000 7f270dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638b44ac4a0 0x5638b44b6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56432/0 (socket says 172.21.15.120:56432) 2024-09-19T09:07:45.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.784+0000 7f270dd9b640 1 -- 172.21.15.120:0/3118625418 learned_addr learned my addr 172.21.15.120:0/3118625418 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:45.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.784+0000 7f270e59c640 1 --2- 172.21.15.120:0/3118625418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b44acdc0 0x5638b44b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.784+0000 7f270dd9b640 1 --2- 172.21.15.120:0/3118625418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638b44b7680 0x5638b44b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.784+0000 7f270dd9b640 1 -- 172.21.15.120:0/3118625418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b44acdc0 msgr2=0x5638b44b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:45.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.784+0000 7f270dd9b640 1 --2- 172.21.15.120:0/3118625418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b44acdc0 0x5638b44b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.784+0000 7f270dd9b640 1 -- 172.21.15.120:0/3118625418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638b44b7680 msgr2=0x5638b44b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:45.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.784+0000 7f270dd9b640 1 --2- 172.21.15.120:0/3118625418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638b44b7680 0x5638b44b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.784+0000 7f270dd9b640 1 -- 172.21.15.120:0/3118625418 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638b41833c0 con 0x5638b44ac4a0 2024-09-19T09:07:45.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.784+0000 7f270e59c640 1 --2- 172.21.15.120:0/3118625418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b44acdc0 0x5638b44b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:07:45.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.785+0000 7f270dd9b640 1 --2- 172.21.15.120:0/3118625418 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638b44ac4a0 0x5638b44b6e40 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f2704012850 tx=0x7f2704004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759d5b19051be11d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:45.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.785+0000 7f26f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:07:45.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.785+0000 7f26f5ffb640 1 -- 172.21.15.120:0/3118625418 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270401b040 con 0x5638b44ac4a0 2024-09-19T09:07:45.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.785+0000 7f26f5ffb640 1 -- 172.21.15.120:0/3118625418 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f2704011cf0 con 0x5638b44ac4a0 2024-09-19T09:07:45.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.785+0000 7f27170d2740 1 -- 172.21.15.120:0/3118625418 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27000319f0 con 0x5638b44ac4a0 2024-09-19T09:07:45.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.785+0000 7f26f5ffb640 1 -- 172.21.15.120:0/3118625418 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2704027050 con 0x5638b44ac4a0 2024-09-19T09:07:45.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.785+0000 7f27170d2740 10 client.17238 fetch_fsmap learned FSMap version 30 2024-09-19T09:07:45.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.785+0000 7f27170d2740 1 -- 172.21.15.120:0/3118625418 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5638b44c5a80 con 0x5638b44ac4a0 2024-09-19T09:07:45.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.785+0000 7f26f5ffb640 1 -- 172.21.15.120:0/3118625418 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27000319f0 con 0x5638b44ac4a0 2024-09-19T09:07:45.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.785+0000 7f26f5ffb640 1 -- 172.21.15.120:0/3118625418 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2704004c20 con 0x5638b44ac4a0 2024-09-19T09:07:45.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.785+0000 7f27170d2740 10 client.17238 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:07:45.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.785+0000 7f27170d2740 10 client.17238 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:07:45.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.785+0000 7f27170d2740 20 client.17238 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:45.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.785+0000 7f27170d2740 1 --2- 172.21.15.120:0/3118625418 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5638b44c6e90 0x5638b44c9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.785+0000 7f27170d2740 4 client.17238 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:07:45.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.785+0000 7f27170d2740 1 -- 172.21.15.120:0/3118625418 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638b441f850 con 0x5638b44c6e90 2024-09-19T09:07:45.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.786+0000 7f270e59c640 1 --2- 172.21.15.120:0/3118625418 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5638b44c6e90 0x5638b44c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.786+0000 7f26f5ffb640 10 client.17238 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:45.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.786+0000 7f270e59c640 1 --2- 172.21.15.120:0/3118625418 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5638b44c6e90 0x5638b44c9280 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eb5d96c4ed4b633 server_cookie=ec6bc81f7e93bc40 in_seq=0 out_seq=0 2024-09-19T09:07:45.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.787+0000 7f26f5ffb640 1 -- 172.21.15.120:0/3118625418 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5638b441f850 con 0x5638b44c6e90 2024-09-19T09:07:45.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.787+0000 7f26f5ffb640 10 client.17238 handle_command_reply: tid=0 2024-09-19T09:07:45.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.787+0000 7f27170d2740 1 client.17238 shutdown 2024-09-19T09:07:45.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.787+0000 7f27170d2740 1 -- 172.21.15.120:0/3118625418 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638b44ac4a0 msgr2=0x5638b44b6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:45.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.787+0000 7f27170d2740 1 --2- 172.21.15.120:0/3118625418 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638b44ac4a0 0x5638b44b6e40 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f2704012850 tx=0x7f2704004730 comp rx=0 tx=0).stop 2024-09-19T09:07:45.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.787+0000 7f27170d2740 1 -- 172.21.15.120:0/3118625418 shutdown_connections 2024-09-19T09:07:45.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.787+0000 7f27170d2740 1 --2- 172.21.15.120:0/3118625418 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5638b44c6e90 0x5638b44c9280 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.787+0000 7f27170d2740 1 --2- 172.21.15.120:0/3118625418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638b44b7680 0x5638b44b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.787+0000 7f26f5ffb640 0 client.17238 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:45.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.787+0000 7f27170d2740 1 --2- 172.21.15.120:0/3118625418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b44acdc0 0x5638b44b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.787+0000 7f27170d2740 1 --2- 172.21.15.120:0/3118625418 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638b44ac4a0 0x5638b44b6e40 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.787+0000 7f27170d2740 1 -- 172.21.15.120:0/3118625418 >> 172.21.15.120:0/3118625418 conn(0x5638b44a8a30 msgr2=0x5638b44a7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:45.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.787+0000 7f27170d2740 1 -- 172.21.15.120:0/3118625418 shutdown_connections 2024-09-19T09:07:45.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.787+0000 7f27170d2740 1 -- 172.21.15.120:0/3118625418 wait complete. 2024-09-19T09:07:45.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.787+0000 7f27170d2740 20 client.17238 trim_cache size 0 max 16384 2024-09-19T09:07:45.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.804+0000 7f27170d2740 1 Processor -- start 2024-09-19T09:07:45.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.804+0000 7f27170d2740 1 -- start start 2024-09-19T09:07:45.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.804+0000 7f27170d2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638b44ac4a0 0x5638b44b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.804+0000 7f27170d2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638b44acdc0 0x5638b44ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.804+0000 7f27170d2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b44b7680 0x5638b44cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.804+0000 7f27170d2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f27101af920 con 0x5638b44ac4a0 2024-09-19T09:07:45.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.804+0000 7f27170d2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f27101b1460 con 0x5638b44acdc0 2024-09-19T09:07:45.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.804+0000 7f27170d2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f27101afff0 con 0x5638b44b7680 2024-09-19T09:07:45.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.805+0000 7f270dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638b44ac4a0 0x5638b44b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.805+0000 7f270dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638b44ac4a0 0x5638b44b29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56440/0 (socket says 172.21.15.120:56440) 2024-09-19T09:07:45.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.805+0000 7f270dd9b640 1 -- 172.21.15.120:0/563283955 learned_addr learned my addr 172.21.15.120:0/563283955 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:45.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.805+0000 7f270e59c640 1 --2- 172.21.15.120:0/563283955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638b44acdc0 0x5638b44ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.805+0000 7f270dd9b640 1 --2- 172.21.15.120:0/563283955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b44b7680 0x5638b44cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.805+0000 7f270dd9b640 1 -- 172.21.15.120:0/563283955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b44b7680 msgr2=0x5638b44cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:45.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.805+0000 7f270dd9b640 1 --2- 172.21.15.120:0/563283955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b44b7680 0x5638b44cf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.805+0000 7f270dd9b640 1 -- 172.21.15.120:0/563283955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638b44acdc0 msgr2=0x5638b44ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:45.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.805+0000 7f270dd9b640 1 --2- 172.21.15.120:0/563283955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638b44acdc0 0x5638b44ceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.805+0000 7f270dd9b640 1 -- 172.21.15.120:0/563283955 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638b44b30a0 con 0x5638b44ac4a0 2024-09-19T09:07:45.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.805+0000 7f270dd9b640 1 --2- 172.21.15.120:0/563283955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638b44ac4a0 0x5638b44b29c0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f27040106f0 tx=0x7f2704004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ea50cb6b16f485 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:45.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.805+0000 7f26f7fff640 1 -- 172.21.15.120:0/563283955 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2704011af0 con 0x5638b44ac4a0 2024-09-19T09:07:45.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.805+0000 7f26f7fff640 1 -- 172.21.15.120:0/563283955 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2704011c90 con 0x5638b44ac4a0 2024-09-19T09:07:45.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.805+0000 7f26f7fff640 1 -- 172.21.15.120:0/563283955 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27040110f0 con 0x5638b44ac4a0 2024-09-19T09:07:45.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.806+0000 7f27170d2740 1 -- 172.21.15.120:0/563283955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638b44ac4a0 msgr2=0x5638b44b29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:45.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.806+0000 7f27170d2740 1 --2- 172.21.15.120:0/563283955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638b44ac4a0 0x5638b44b29c0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f27040106f0 tx=0x7f2704004660 comp rx=0 tx=0).stop 2024-09-19T09:07:45.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.806+0000 7f27170d2740 1 -- 172.21.15.120:0/563283955 shutdown_connections 2024-09-19T09:07:45.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.806+0000 7f27170d2740 1 --2- 172.21.15.120:0/563283955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b44b7680 0x5638b44cf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.806+0000 7f27170d2740 1 --2- 172.21.15.120:0/563283955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638b44acdc0 0x5638b44ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.806+0000 7f27170d2740 1 --2- 172.21.15.120:0/563283955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638b44ac4a0 0x5638b44b29c0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.806+0000 7f27170d2740 1 -- 172.21.15.120:0/563283955 >> 172.21.15.120:0/563283955 conn(0x5638b44a8a30 msgr2=0x5638b44a7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:45.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.806+0000 7f27170d2740 1 -- 172.21.15.120:0/563283955 shutdown_connections 2024-09-19T09:07:45.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.806+0000 7f27170d2740 1 -- 172.21.15.120:0/563283955 wait complete. 2024-09-19T09:07:45.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.806+0000 7f27170d2740 1 -- start start 2024-09-19T09:07:45.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.807+0000 7f27170d2740 1 Processor -- start 2024-09-19T09:07:45.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.807+0000 7f27170d2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638b44ac4a0 0x5638b44b1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.807+0000 7f27170d2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b44acdc0 0x5638b44b1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.807+0000 7f27170d2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638b44b7680 0x5638b44b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.807+0000 7f27170d2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f27101b0d90 con 0x5638b44ac4a0 2024-09-19T09:07:45.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.807+0000 7f27170d2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f27101b06c0 con 0x5638b44b7680 2024-09-19T09:07:45.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.807+0000 7f27170d2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f27101ad8e0 con 0x5638b44acdc0 2024-09-19T09:07:45.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.807+0000 7f27170d2740 10 client.0 _ll_register_callbacks cb 0x7f270400be30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:45.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.807+0000 7f270dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638b44ac4a0 0x5638b44b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.807+0000 7f270dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638b44ac4a0 0x5638b44b1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56452/0 (socket says 172.21.15.120:56452) 2024-09-19T09:07:45.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.807+0000 7f270dd9b640 1 -- 172.21.15.120:0/1179961254 learned_addr learned my addr 172.21.15.120:0/1179961254 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:45.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.807+0000 7f270dd9b640 1 --2- 172.21.15.120:0/1179961254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638b44b7680 0x5638b44b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.807+0000 7f270e59c640 1 --2- 172.21.15.120:0/1179961254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b44acdc0 0x5638b44b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.807+0000 7f270dd9b640 1 -- 172.21.15.120:0/1179961254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b44acdc0 msgr2=0x5638b44b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:45.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.807+0000 7f270dd9b640 1 --2- 172.21.15.120:0/1179961254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b44acdc0 0x5638b44b1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.807+0000 7f270dd9b640 1 -- 172.21.15.120:0/1179961254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638b44b7680 msgr2=0x5638b44b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:45.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.807+0000 7f270dd9b640 1 --2- 172.21.15.120:0/1179961254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638b44b7680 0x5638b44b1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.807+0000 7f270dd9b640 1 -- 172.21.15.120:0/1179961254 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638b44bdc00 con 0x5638b44ac4a0 2024-09-19T09:07:45.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.807+0000 7f270dd9b640 1 --2- 172.21.15.120:0/1179961254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638b44ac4a0 0x5638b44b1150 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f27040106c0 tx=0x7f270400bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2402fcfaf05239cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:45.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.807+0000 7f26f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:07:45.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.808+0000 7f26f5ffb640 1 -- 172.21.15.120:0/1179961254 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2704010a70 con 0x5638b44ac4a0 2024-09-19T09:07:45.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.808+0000 7f26f5ffb640 1 -- 172.21.15.120:0/1179961254 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f2704010e70 con 0x5638b44ac4a0 2024-09-19T09:07:45.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.808+0000 7f27170d2740 1 -- 172.21.15.120:0/1179961254 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2704011c90 con 0x5638b44ac4a0 2024-09-19T09:07:45.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.808+0000 7f26f5ffb640 1 -- 172.21.15.120:0/1179961254 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2704011070 con 0x5638b44ac4a0 2024-09-19T09:07:45.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.808+0000 7f27170d2740 10 client.17244 fetch_fsmap learned FSMap version 30 2024-09-19T09:07:45.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.808+0000 7f27170d2740 1 -- 172.21.15.120:0/1179961254 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5638b4531d80 con 0x5638b44ac4a0 2024-09-19T09:07:45.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.808+0000 7f26f5ffb640 1 -- 172.21.15.120:0/1179961254 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2704011c90 con 0x5638b44ac4a0 2024-09-19T09:07:45.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.808+0000 7f26f5ffb640 1 -- 172.21.15.120:0/1179961254 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2704005470 con 0x5638b44ac4a0 2024-09-19T09:07:45.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.808+0000 7f27170d2740 10 client.17244 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:07:45.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.808+0000 7f27170d2740 10 client.17244 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:07:45.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.808+0000 7f27170d2740 20 client.17244 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:45.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.808+0000 7f27170d2740 1 --2- 172.21.15.120:0/1179961254 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5638b44c6e90 0x5638b45343a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:45.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.808+0000 7f27170d2740 4 client.17244 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:07:45.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.808+0000 7f27170d2740 1 -- 172.21.15.120:0/1179961254 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5638b4419d60 con 0x5638b44c6e90 2024-09-19T09:07:45.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.809+0000 7f270e59c640 1 --2- 172.21.15.120:0/1179961254 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5638b44c6e90 0x5638b45343a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:45.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.809+0000 7f26f5ffb640 10 client.17244 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:45.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.809+0000 7f270e59c640 1 --2- 172.21.15.120:0/1179961254 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5638b44c6e90 0x5638b45343a0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fb26202b1b8a281 server_cookie=fbd82c4beb2b866f in_seq=0 out_seq=0 2024-09-19T09:07:45.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.809+0000 7f26f5ffb640 1 -- 172.21.15.120:0/1179961254 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5638b4419d60 con 0x5638b44c6e90 2024-09-19T09:07:45.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.809+0000 7f26f5ffb640 10 client.17244 handle_command_reply: tid=0 2024-09-19T09:07:45.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.809+0000 7f27170d2740 1 client.17244 shutdown 2024-09-19T09:07:45.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.809+0000 7f27170d2740 1 -- 172.21.15.120:0/1179961254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638b44ac4a0 msgr2=0x5638b44b1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:45.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.809+0000 7f27170d2740 1 --2- 172.21.15.120:0/1179961254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638b44ac4a0 0x5638b44b1150 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f27040106c0 tx=0x7f270400bd80 comp rx=0 tx=0).stop 2024-09-19T09:07:45.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.809+0000 7f27170d2740 1 -- 172.21.15.120:0/1179961254 shutdown_connections 2024-09-19T09:07:45.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.809+0000 7f27170d2740 1 --2- 172.21.15.120:0/1179961254 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5638b44c6e90 0x5638b45343a0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.809+0000 7f27170d2740 1 --2- 172.21.15.120:0/1179961254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638b44b7680 0x5638b44b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.809+0000 7f27170d2740 1 --2- 172.21.15.120:0/1179961254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b44acdc0 0x5638b44b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.809+0000 7f26f5ffb640 0 client.17244 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:45.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.809+0000 7f27170d2740 1 --2- 172.21.15.120:0/1179961254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638b44ac4a0 0x5638b44b1150 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.809+0000 7f27170d2740 1 -- 172.21.15.120:0/1179961254 >> 172.21.15.120:0/1179961254 conn(0x5638b44a8a30 msgr2=0x5638b44a7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:45.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.809+0000 7f27170d2740 1 -- 172.21.15.120:0/1179961254 shutdown_connections 2024-09-19T09:07:45.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.809+0000 7f27170d2740 1 -- 172.21.15.120:0/1179961254 wait complete. 2024-09-19T09:07:45.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.809+0000 7f27170d2740 20 client.17244 trim_cache size 0 max 16384 2024-09-19T09:07:45.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.810+0000 7f2714cf9640 1 -- 172.21.15.120:0/2715377580 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f26f80a3240 msgr2=0x7f26f80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:45.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.810+0000 7f2714cf9640 1 --2- 172.21.15.120:0/2715377580 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f26f80a3240 0x7f26f80a5700 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2708098570 tx=0x7f270806ef90 comp rx=0 tx=0).stop 2024-09-19T09:07:45.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.810+0000 7f2714cf9640 1 -- 172.21.15.120:0/2715377580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2710106f50 msgr2=0x7f2710101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:45.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.810+0000 7f2714cf9640 1 --2- 172.21.15.120:0/2715377580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2710106f50 0x7f2710101cf0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f270800eaa0 tx=0x7f270800ef60 comp rx=0 tx=0).stop 2024-09-19T09:07:45.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.810+0000 7f2714cf9640 1 -- 172.21.15.120:0/2715377580 shutdown_connections 2024-09-19T09:07:45.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.810+0000 7f2714cf9640 1 --2- 172.21.15.120:0/2715377580 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f26f80a3240 0x7f26f80a5700 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.810+0000 7f2714cf9640 1 --2- 172.21.15.120:0/2715377580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2710106f50 0x7f2710101cf0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.810+0000 7f2714cf9640 1 --2- 172.21.15.120:0/2715377580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2710100ca0 0x7f27101017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.810+0000 7f2714cf9640 1 --2- 172.21.15.120:0/2715377580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27101002f0 0x7f2710101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:45.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.810+0000 7f2714cf9640 1 -- 172.21.15.120:0/2715377580 >> 172.21.15.120:0/2715377580 conn(0x7f2710077210 msgr2=0x7f27101052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:45.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.810+0000 7f2714cf9640 1 -- 172.21.15.120:0/2715377580 shutdown_connections 2024-09-19T09:07:45.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:07:45.810+0000 7f2714cf9640 1 -- 172.21.15.120:0/2715377580 wait complete. 2024-09-19T09:07:46.047 INFO:tasks.workunit.client.0.smithi120.stdout: 1 375454 7.89 MB/sec execute 101 sec latency 712.827 ms 2024-09-19T09:07:46.505 INFO:tasks.cephfs.filesystem:scrub status for tag:c41431e6-c965-492c-9325-e798620ad87b - {'path': '/', 'tag': 'c41431e6-c965-492c-9325-e798620ad87b', 'options': 'recursive,force'} 2024-09-19T09:07:47.047 INFO:tasks.workunit.client.0.smithi120.stdout: 1 375454 7.81 MB/sec execute 102 sec latency 1712.920 ms 2024-09-19T09:07:47.119 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:46 smithi120 ceph-mon[31150]: pgmap v489: 97 pgs: 97 active+clean; 28 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 149 op/s 2024-09-19T09:07:47.120 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '75bdfb5', quiesce took 6.8 seconds. Will release after: 23.0 2024-09-19T09:07:47.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:46 smithi196 ceph-mon[35150]: pgmap v489: 97 pgs: 97 active+clean; 28 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 149 op/s 2024-09-19T09:07:47.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:46 smithi153 ceph-mon[35071]: pgmap v489: 97 pgs: 97 active+clean; 28 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 149 op/s 2024-09-19T09:07:48.048 INFO:tasks.workunit.client.0.smithi120.stdout: 1 375454 7.74 MB/sec execute 103 sec latency 2712.955 ms 2024-09-19T09:07:49.048 INFO:tasks.workunit.client.0.smithi120.stdout: 1 375454 7.66 MB/sec execute 104 sec latency 3713.407 ms 2024-09-19T09:07:49.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:48 smithi196 ceph-mon[35150]: pgmap v490: 97 pgs: 97 active+clean; 32 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 213 op/s 2024-09-19T09:07:49.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:48 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:07:49.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:48 smithi153 ceph-mon[35071]: pgmap v490: 97 pgs: 97 active+clean; 32 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 213 op/s 2024-09-19T09:07:49.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:48 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:07:49.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:48 smithi120 ceph-mon[31150]: pgmap v490: 97 pgs: 97 active+clean; 32 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 213 op/s 2024-09-19T09:07:49.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:48 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:07:50.048 INFO:tasks.workunit.client.0.smithi120.stdout: 1 375454 7.59 MB/sec execute 105 sec latency 4713.512 ms 2024-09-19T09:07:51.048 INFO:tasks.workunit.client.0.smithi120.stdout: 1 375454 7.52 MB/sec execute 106 sec latency 5713.602 ms 2024-09-19T09:07:51.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:50 smithi196 ceph-mon[35150]: pgmap v491: 97 pgs: 97 active+clean; 32 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 205 op/s 2024-09-19T09:07:51.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:50 smithi153 ceph-mon[35071]: pgmap v491: 97 pgs: 97 active+clean; 32 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 205 op/s 2024-09-19T09:07:51.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:50 smithi120 ceph-mon[31150]: pgmap v491: 97 pgs: 97 active+clean; 32 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 205 op/s 2024-09-19T09:07:52.048 INFO:tasks.workunit.client.0.smithi120.stdout: 1 375454 7.45 MB/sec execute 107 sec latency 6713.699 ms 2024-09-19T09:07:53.048 INFO:tasks.workunit.client.0.smithi120.stdout: 1 375454 7.38 MB/sec execute 108 sec latency 7713.769 ms 2024-09-19T09:07:53.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:52 smithi196 ceph-mon[35150]: pgmap v492: 97 pgs: 97 active+clean; 32 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 191 op/s 2024-09-19T09:07:53.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:52 smithi153 ceph-mon[35071]: pgmap v492: 97 pgs: 97 active+clean; 32 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 191 op/s 2024-09-19T09:07:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.260+0000 7f9a87968640 1 -- 172.21.15.120:0/484860410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a80101b60 msgr2=0x7f9a80101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.260+0000 7f9a87968640 1 --2- 172.21.15.120:0/484860410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a80101b60 0x7f9a80101f60 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c009900 tx=0x7f9a7c02a410 comp rx=0 tx=0).stop 2024-09-19T09:07:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.260+0000 7f9a87968640 1 -- 172.21.15.120:0/484860410 shutdown_connections 2024-09-19T09:07:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.260+0000 7f9a87968640 1 --2- 172.21.15.120:0/484860410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a80103710 0x7f9a80109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.260+0000 7f9a87968640 1 --2- 172.21.15.120:0/484860410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a80102d50 0x7f9a801031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.260+0000 7f9a87968640 1 --2- 172.21.15.120:0/484860410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a80101b60 0x7f9a80101f60 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.260+0000 7f9a87968640 1 -- 172.21.15.120:0/484860410 >> 172.21.15.120:0/484860410 conn(0x7f9a800fd2e0 msgr2=0x7f9a800ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.260+0000 7f9a87968640 1 -- 172.21.15.120:0/484860410 shutdown_connections 2024-09-19T09:07:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.260+0000 7f9a87968640 1 -- 172.21.15.120:0/484860410 wait complete. 2024-09-19T09:07:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.261+0000 7f9a87968640 1 Processor -- start 2024-09-19T09:07:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.261+0000 7f9a87968640 1 -- start start 2024-09-19T09:07:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.261+0000 7f9a87968640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a80101b60 0x7f9a8019a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.261+0000 7f9a87968640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a80102d50 0x7f9a8019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.261+0000 7f9a87968640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a80103710 0x7f9a801a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.261+0000 7f9a87968640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8010d7e0 con 0x7f9a80102d50 2024-09-19T09:07:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.261+0000 7f9a87968640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8010d660 con 0x7f9a80103710 2024-09-19T09:07:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.261+0000 7f9a87968640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8010d960 con 0x7f9a80101b60 2024-09-19T09:07:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.261+0000 7f9a84f03640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a80102d50 0x7f9a8019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.261+0000 7f9a84f03640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a80102d50 0x7f9a8019a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46402/0 (socket says 172.21.15.120:46402) 2024-09-19T09:07:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.261+0000 7f9a84f03640 1 -- 172.21.15.120:0/1635550805 learned_addr learned my addr 172.21.15.120:0/1635550805 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.261+0000 7f9a85704640 1 --2- 172.21.15.120:0/1635550805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a80101b60 0x7f9a8019a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.261+0000 7f9a84f03640 1 -- 172.21.15.120:0/1635550805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a80101b60 msgr2=0x7f9a8019a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.261+0000 7f9a84f03640 1 --2- 172.21.15.120:0/1635550805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a80101b60 0x7f9a8019a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.261+0000 7f9a84f03640 1 -- 172.21.15.120:0/1635550805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a80103710 msgr2=0x7f9a801a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.261+0000 7f9a84f03640 1 --2- 172.21.15.120:0/1635550805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a80103710 0x7f9a801a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.261+0000 7f9a84f03640 1 -- 172.21.15.120:0/1635550805 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a801a1c60 con 0x7f9a80102d50 2024-09-19T09:07:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.261+0000 7f9a85704640 1 --2- 172.21.15.120:0/1635550805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a80101b60 0x7f9a8019a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:07:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.262+0000 7f9a84f03640 1 --2- 172.21.15.120:0/1635550805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a80102d50 0x7f9a8019a580 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f9a7000d770 tx=0x7f9a7000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.262+0000 7f9a767fc640 1 -- 172.21.15.120:0/1635550805 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a70017070 con 0x7f9a80102d50 2024-09-19T09:07:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.262+0000 7f9a767fc640 1 -- 172.21.15.120:0/1635550805 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a70005500 con 0x7f9a80102d50 2024-09-19T09:07:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.262+0000 7f9a87968640 1 -- 172.21.15.120:0/1635550805 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a801a1f50 con 0x7f9a80102d50 2024-09-19T09:07:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.262+0000 7f9a767fc640 1 -- 172.21.15.120:0/1635550805 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a70004e00 con 0x7f9a80102d50 2024-09-19T09:07:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.262+0000 7f9a87968640 1 -- 172.21.15.120:0/1635550805 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a801a23c0 con 0x7f9a80102d50 2024-09-19T09:07:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.264+0000 7f9a767fc640 1 -- 172.21.15.120:0/1635550805 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f9a70004fa0 con 0x7f9a80102d50 2024-09-19T09:07:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.265+0000 7f9a8a7da740 1 Processor -- start 2024-09-19T09:07:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.265+0000 7f9a8a7da740 1 -- start start 2024-09-19T09:07:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.265+0000 7f9a8a7da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30aae29a0 0x55d30aae2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.265+0000 7f9a8a7da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30aae32e0 0x55d30aae7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.265+0000 7f9a8a7da740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30aae8370 0x55d30aaea830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.265+0000 7f9a8a7da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d30aa462f0 con 0x55d30aae32e0 2024-09-19T09:07:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.265+0000 7f9a8a7da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d30a9ee270 con 0x55d30aae29a0 2024-09-19T09:07:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.265+0000 7f9a8a7da740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d30aa422b0 con 0x55d30aae8370 2024-09-19T09:07:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.265+0000 7f9a85704640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30aae32e0 0x55d30aae7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.265+0000 7f9a85704640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30aae32e0 0x55d30aae7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46406/0 (socket says 172.21.15.120:46406) 2024-09-19T09:07:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.265+0000 7f9a85704640 1 -- 172.21.15.120:0/4181103309 learned_addr learned my addr 172.21.15.120:0/4181103309 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.265+0000 7f9a767fc640 1 --2- 172.21.15.120:0/1635550805 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9a6c0a32f0 0x7f9a6c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.266+0000 7f9a85f05640 1 --2- 172.21.15.120:0/4181103309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30aae29a0 0x55d30aae2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.266+0000 7f9a84f03640 1 --2- 172.21.15.120:0/4181103309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30aae8370 0x55d30aaea830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.266+0000 7f9a85f05640 1 --2- 172.21.15.120:0/1635550805 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9a6c0a32f0 0x7f9a6c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.266+0000 7f9a85704640 1 -- 172.21.15.120:0/4181103309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30aae8370 msgr2=0x55d30aaea830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.266+0000 7f9a85704640 1 --2- 172.21.15.120:0/4181103309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30aae8370 0x55d30aaea830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.266+0000 7f9a85704640 1 -- 172.21.15.120:0/4181103309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30aae29a0 msgr2=0x55d30aae2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.266+0000 7f9a85704640 1 --2- 172.21.15.120:0/4181103309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30aae29a0 0x55d30aae2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.266+0000 7f9a85704640 1 -- 172.21.15.120:0/4181103309 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d30a9ed340 con 0x55d30aae32e0 2024-09-19T09:07:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.266+0000 7f9a85f05640 1 --2- 172.21.15.120:0/4181103309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30aae29a0 0x55d30aae2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:07:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.266+0000 7f9a767fc640 1 -- 172.21.15.120:0/1635550805 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f9a700cf480 con 0x7f9a80102d50 2024-09-19T09:07:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.266+0000 7f9a85704640 1 --2- 172.21.15.120:0/4181103309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30aae32e0 0x55d30aae7da0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c031440 tx=0x7f9a7c032de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0bbec25e036c9d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.266+0000 7f9a85f05640 1 --2- 172.21.15.120:0/1635550805 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9a6c0a32f0 0x7f9a6c0a57b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9a78009fd0 tx=0x7f9a78005e00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.266+0000 7f9a6b7fe640 1 -- 172.21.15.120:0/4181103309 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c03f070 con 0x55d30aae32e0 2024-09-19T09:07:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.266+0000 7f9a6b7fe640 1 -- 172.21.15.120:0/4181103309 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a7c0318b0 con 0x55d30aae32e0 2024-09-19T09:07:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.266+0000 7f9a6b7fe640 1 -- 172.21.15.120:0/4181103309 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c035070 con 0x55d30aae32e0 2024-09-19T09:07:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.267+0000 7f9a8a7da740 1 -- 172.21.15.120:0/4181103309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30aae32e0 msgr2=0x55d30aae7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.267+0000 7f9a8a7da740 1 --2- 172.21.15.120:0/4181103309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30aae32e0 0x55d30aae7da0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c031440 tx=0x7f9a7c032de0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.267+0000 7f9a8a7da740 1 -- 172.21.15.120:0/4181103309 shutdown_connections 2024-09-19T09:07:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.267+0000 7f9a8a7da740 1 --2- 172.21.15.120:0/4181103309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30aae8370 0x55d30aaea830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.267+0000 7f9a8a7da740 1 --2- 172.21.15.120:0/4181103309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30aae32e0 0x55d30aae7da0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.267+0000 7f9a8a7da740 1 --2- 172.21.15.120:0/4181103309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30aae29a0 0x55d30aae2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.267+0000 7f9a8a7da740 1 -- 172.21.15.120:0/4181103309 >> 172.21.15.120:0/4181103309 conn(0x55d30aade010 msgr2=0x55d30aade410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.267+0000 7f9a8a7da740 1 -- 172.21.15.120:0/4181103309 shutdown_connections 2024-09-19T09:07:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.267+0000 7f9a8a7da740 1 -- 172.21.15.120:0/4181103309 wait complete. 2024-09-19T09:07:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.267+0000 7f9a8a7da740 1 -- start start 2024-09-19T09:07:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.267+0000 7f9a8a7da740 1 Processor -- start 2024-09-19T09:07:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.268+0000 7f9a8a7da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30aae29a0 0x55d30aaecb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.268+0000 7f9a8a7da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30aae32e0 0x55d30aaed0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.268+0000 7f9a8a7da740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30aae8370 0x55d30aaf40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.268+0000 7f9a8a7da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d30aaeb2e0 con 0x55d30aae29a0 2024-09-19T09:07:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.268+0000 7f9a8a7da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d30a9cf290 con 0x55d30aae32e0 2024-09-19T09:07:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.268+0000 7f9a8a7da740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d30aaeb460 con 0x55d30aae8370 2024-09-19T09:07:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.268+0000 7f9a8a7da740 10 client.0 _ll_register_callbacks cb 0x55d30aa00140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.268+0000 7f9a85704640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30aae29a0 0x55d30aaecb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.268+0000 7f9a85704640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30aae29a0 0x55d30aaecb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46412/0 (socket says 172.21.15.120:46412) 2024-09-19T09:07:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.268+0000 7f9a85704640 1 -- 172.21.15.120:0/3947030747 learned_addr learned my addr 172.21.15.120:0/3947030747 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.268+0000 7f9a85f05640 1 --2- 172.21.15.120:0/3947030747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30aae8370 0x55d30aaf40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.268+0000 7f9a84f03640 1 --2- 172.21.15.120:0/3947030747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30aae32e0 0x55d30aaed0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.268+0000 7f9a85704640 1 -- 172.21.15.120:0/3947030747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30aae8370 msgr2=0x55d30aaf40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.268+0000 7f9a85704640 1 --2- 172.21.15.120:0/3947030747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30aae8370 0x55d30aaf40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.268+0000 7f9a85704640 1 -- 172.21.15.120:0/3947030747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30aae32e0 msgr2=0x55d30aaed0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.268+0000 7f9a85704640 1 --2- 172.21.15.120:0/3947030747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30aae32e0 0x55d30aaed0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.268+0000 7f9a85704640 1 -- 172.21.15.120:0/3947030747 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d30a759210 con 0x55d30aae29a0 2024-09-19T09:07:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.268+0000 7f9a84f03640 1 --2- 172.21.15.120:0/3947030747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30aae32e0 0x55d30aaed0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:07:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.268+0000 7f9a85704640 1 --2- 172.21.15.120:0/3947030747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30aae29a0 0x55d30aaecb70 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c006010 tx=0x7f9a7c003800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c614b37dd9db48a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.268+0000 7f9a68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:07:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.269+0000 7f9a68ff9640 1 -- 172.21.15.120:0/3947030747 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c04a020 con 0x55d30aae29a0 2024-09-19T09:07:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.269+0000 7f9a68ff9640 1 -- 172.21.15.120:0/3947030747 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f9a7c003b80 con 0x55d30aae29a0 2024-09-19T09:07:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.269+0000 7f9a8a7da740 1 -- 172.21.15.120:0/3947030747 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a7c0318b0 con 0x55d30aae29a0 2024-09-19T09:07:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.269+0000 7f9a68ff9640 1 -- 172.21.15.120:0/3947030747 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9a7c03f070 con 0x55d30aae29a0 2024-09-19T09:07:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.269+0000 7f9a8a7da740 10 client.17256 fetch_fsmap learned FSMap version 30 2024-09-19T09:07:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.269+0000 7f9a8a7da740 1 -- 172.21.15.120:0/3947030747 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d30a825620 con 0x55d30aae29a0 2024-09-19T09:07:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.269+0000 7f9a68ff9640 1 -- 172.21.15.120:0/3947030747 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a7c0318b0 con 0x55d30aae29a0 2024-09-19T09:07:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.269+0000 7f9a68ff9640 1 -- 172.21.15.120:0/3947030747 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9a7c043400 con 0x55d30aae29a0 2024-09-19T09:07:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.269+0000 7f9a8a7da740 10 client.17256 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:07:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.269+0000 7f9a8a7da740 10 client.17256 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:07:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.269+0000 7f9a8a7da740 20 client.17256 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.269+0000 7f9a8a7da740 1 --2- 172.21.15.120:0/3947030747 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d30aaf83f0 0x55d30aafa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.269+0000 7f9a8a7da740 4 client.17256 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:07:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.269+0000 7f9a8a7da740 1 -- 172.21.15.120:0/3947030747 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d30aa53a20 con 0x55d30aaf83f0 2024-09-19T09:07:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.270+0000 7f9a84f03640 1 --2- 172.21.15.120:0/3947030747 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d30aaf83f0 0x55d30aafa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.270+0000 7f9a84f03640 1 --2- 172.21.15.120:0/3947030747 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d30aaf83f0 0x55d30aafa7e0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99e5d5ed98eb0a34 server_cookie=7492577595102848 in_seq=0 out_seq=0 2024-09-19T09:07:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.270+0000 7f9a68ff9640 10 client.17256 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.271+0000 7f9a68ff9640 1 -- 172.21.15.120:0/3947030747 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d30aa53a20 con 0x55d30aaf83f0 2024-09-19T09:07:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.271+0000 7f9a68ff9640 10 client.17256 handle_command_reply: tid=0 2024-09-19T09:07:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.271+0000 7f9a8a7da740 1 client.17256 shutdown 2024-09-19T09:07:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.271+0000 7f9a8a7da740 1 -- 172.21.15.120:0/3947030747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30aae29a0 msgr2=0x55d30aaecb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.271+0000 7f9a8a7da740 1 --2- 172.21.15.120:0/3947030747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30aae29a0 0x55d30aaecb70 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c006010 tx=0x7f9a7c003800 comp rx=0 tx=0).stop 2024-09-19T09:07:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.271+0000 7f9a8a7da740 1 -- 172.21.15.120:0/3947030747 shutdown_connections 2024-09-19T09:07:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.271+0000 7f9a8a7da740 1 --2- 172.21.15.120:0/3947030747 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d30aaf83f0 0x55d30aafa7e0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.271+0000 7f9a8a7da740 1 --2- 172.21.15.120:0/3947030747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30aae8370 0x55d30aaf40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.271+0000 7f9a8a7da740 1 --2- 172.21.15.120:0/3947030747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30aae32e0 0x55d30aaed0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.271+0000 7f9a68ff9640 0 client.17256 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.271+0000 7f9a8a7da740 1 --2- 172.21.15.120:0/3947030747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30aae29a0 0x55d30aaecb70 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.271+0000 7f9a8a7da740 1 -- 172.21.15.120:0/3947030747 >> 172.21.15.120:0/3947030747 conn(0x55d30aade010 msgr2=0x55d30aadf820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.271+0000 7f9a8a7da740 1 -- 172.21.15.120:0/3947030747 shutdown_connections 2024-09-19T09:07:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.271+0000 7f9a8a7da740 1 -- 172.21.15.120:0/3947030747 wait complete. 2024-09-19T09:07:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.271+0000 7f9a8a7da740 20 client.17256 trim_cache size 0 max 16384 2024-09-19T09:07:53.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.285+0000 7f9a8a7da740 1 Processor -- start 2024-09-19T09:07:53.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.285+0000 7f9a8a7da740 1 -- start start 2024-09-19T09:07:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.286+0000 7f9a8a7da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30aae29a0 0x55d30aadee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.286+0000 7f9a8a7da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30aae32e0 0x55d30aaf40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.286+0000 7f9a8a7da740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30aae8370 0x55d30aadf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.286+0000 7f9a8a7da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9a801ab1a0 con 0x55d30aae29a0 2024-09-19T09:07:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.286+0000 7f9a8a7da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9a801acce0 con 0x55d30aae32e0 2024-09-19T09:07:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.286+0000 7f9a8a7da740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9a801ab870 con 0x55d30aae8370 2024-09-19T09:07:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.286+0000 7f9a85704640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30aae29a0 0x55d30aadee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.286+0000 7f9a85704640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30aae29a0 0x55d30aadee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46418/0 (socket says 172.21.15.120:46418) 2024-09-19T09:07:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.286+0000 7f9a85704640 1 -- 172.21.15.120:0/428725819 learned_addr learned my addr 172.21.15.120:0/428725819 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.286+0000 7f9a85704640 1 -- 172.21.15.120:0/428725819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30aae8370 msgr2=0x55d30aadf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.286+0000 7f9a85704640 1 --2- 172.21.15.120:0/428725819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30aae8370 0x55d30aadf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.286+0000 7f9a85704640 1 -- 172.21.15.120:0/428725819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30aae32e0 msgr2=0x55d30aaf40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.286+0000 7f9a84f03640 1 --2- 172.21.15.120:0/428725819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30aae32e0 0x55d30aaf40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.286+0000 7f9a85704640 1 --2- 172.21.15.120:0/428725819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30aae32e0 0x55d30aaf40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.286+0000 7f9a85704640 1 -- 172.21.15.120:0/428725819 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d30aae8180 con 0x55d30aae29a0 2024-09-19T09:07:53.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.286+0000 7f9a85704640 1 --2- 172.21.15.120:0/428725819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30aae29a0 0x55d30aadee90 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c02a930 tx=0x7f9a7c005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a19d34f82998260 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:53.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.286+0000 7f9a6affd640 1 -- 172.21.15.120:0/428725819 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c04a070 con 0x55d30aae29a0 2024-09-19T09:07:53.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.286+0000 7f9a6affd640 1 -- 172.21.15.120:0/428725819 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a7c043600 con 0x55d30aae29a0 2024-09-19T09:07:53.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.286+0000 7f9a6affd640 1 -- 172.21.15.120:0/428725819 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c035040 con 0x55d30aae29a0 2024-09-19T09:07:53.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.287+0000 7f9a8a7da740 1 -- 172.21.15.120:0/428725819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30aae29a0 msgr2=0x55d30aadee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:53.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.287+0000 7f9a8a7da740 1 --2- 172.21.15.120:0/428725819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30aae29a0 0x55d30aadee90 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c02a930 tx=0x7f9a7c005620 comp rx=0 tx=0).stop 2024-09-19T09:07:53.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.287+0000 7f9a8a7da740 1 -- 172.21.15.120:0/428725819 shutdown_connections 2024-09-19T09:07:53.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.287+0000 7f9a8a7da740 1 --2- 172.21.15.120:0/428725819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30aae8370 0x55d30aadf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.287+0000 7f9a8a7da740 1 --2- 172.21.15.120:0/428725819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30aae32e0 0x55d30aaf40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.287+0000 7f9a8a7da740 1 --2- 172.21.15.120:0/428725819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30aae29a0 0x55d30aadee90 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.287+0000 7f9a8a7da740 1 -- 172.21.15.120:0/428725819 >> 172.21.15.120:0/428725819 conn(0x55d30aade010 msgr2=0x55d30aaec920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:53.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.287+0000 7f9a8a7da740 1 -- 172.21.15.120:0/428725819 shutdown_connections 2024-09-19T09:07:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.287+0000 7f9a8a7da740 1 -- 172.21.15.120:0/428725819 wait complete. 2024-09-19T09:07:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.287+0000 7f9a8a7da740 1 -- start start 2024-09-19T09:07:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.287+0000 7f9a8a7da740 1 Processor -- start 2024-09-19T09:07:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.288+0000 7f9a8a7da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30aae29a0 0x55d30aaf1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.288+0000 7f9a8a7da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30aae32e0 0x55d30aaf1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.288+0000 7f9a8a7da740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30aae8370 0x55d30aaf2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.288+0000 7f9a8a7da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9a801ac610 con 0x55d30aae32e0 2024-09-19T09:07:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.288+0000 7f9a8a7da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9a801abf40 con 0x55d30aae29a0 2024-09-19T09:07:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.288+0000 7f9a8a7da740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9a801a9160 con 0x55d30aae8370 2024-09-19T09:07:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.288+0000 7f9a8a7da740 10 client.0 _ll_register_callbacks cb 0x7f9a7c032c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:07:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.288+0000 7f9a84f03640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30aae32e0 0x55d30aaf1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.288+0000 7f9a84f03640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30aae32e0 0x55d30aaf1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46428/0 (socket says 172.21.15.120:46428) 2024-09-19T09:07:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.288+0000 7f9a84f03640 1 -- 172.21.15.120:0/139999976 learned_addr learned my addr 172.21.15.120:0/139999976 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:07:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.288+0000 7f9a85704640 1 --2- 172.21.15.120:0/139999976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30aae29a0 0x55d30aaf1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.288+0000 7f9a84f03640 1 -- 172.21.15.120:0/139999976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30aae8370 msgr2=0x55d30aaf2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.288+0000 7f9a85f05640 1 --2- 172.21.15.120:0/139999976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30aae8370 0x55d30aaf2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.288+0000 7f9a84f03640 1 --2- 172.21.15.120:0/139999976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30aae8370 0x55d30aaf2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.288+0000 7f9a84f03640 1 -- 172.21.15.120:0/139999976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30aae29a0 msgr2=0x55d30aaf1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:07:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.288+0000 7f9a84f03640 1 --2- 172.21.15.120:0/139999976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30aae29a0 0x55d30aaf1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.288+0000 7f9a84f03640 1 -- 172.21.15.120:0/139999976 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d30aaf35e0 con 0x55d30aae32e0 2024-09-19T09:07:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.288+0000 7f9a85704640 1 --2- 172.21.15.120:0/139999976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30aae29a0 0x55d30aaf1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:07:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.288+0000 7f9a85f05640 1 --2- 172.21.15.120:0/139999976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30aae8370 0x55d30aaf2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:07:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.288+0000 7f9a84f03640 1 --2- 172.21.15.120:0/139999976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30aae32e0 0x55d30aaf1fc0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f9a70098430 tx=0x7f9a7006a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4288002ed36923c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:07:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.288+0000 7f9a68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:07:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.288+0000 7f9a68ff9640 1 -- 172.21.15.120:0/139999976 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a70020040 con 0x55d30aae32e0 2024-09-19T09:07:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.289+0000 7f9a68ff9640 1 -- 172.21.15.120:0/139999976 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f9a700cd050 con 0x55d30aae32e0 2024-09-19T09:07:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.289+0000 7f9a8a7da740 1 -- 172.21.15.120:0/139999976 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a7c043600 con 0x55d30aae32e0 2024-09-19T09:07:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.289+0000 7f9a68ff9640 1 -- 172.21.15.120:0/139999976 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9a7009ba60 con 0x55d30aae32e0 2024-09-19T09:07:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.289+0000 7f9a8a7da740 10 client.17262 fetch_fsmap learned FSMap version 30 2024-09-19T09:07:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.289+0000 7f9a8a7da740 1 -- 172.21.15.120:0/139999976 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d30ab633e0 con 0x55d30aae32e0 2024-09-19T09:07:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.289+0000 7f9a68ff9640 1 -- 172.21.15.120:0/139999976 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a7c043600 con 0x55d30aae32e0 2024-09-19T09:07:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.289+0000 7f9a68ff9640 1 -- 172.21.15.120:0/139999976 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9a700980d0 con 0x55d30aae32e0 2024-09-19T09:07:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.289+0000 7f9a8a7da740 10 client.17262 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:07:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.289+0000 7f9a8a7da740 10 client.17262 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:07:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.289+0000 7f9a8a7da740 20 client.17262 populate_metadata read hostname 'smithi120' 2024-09-19T09:07:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.289+0000 7f9a8a7da740 1 --2- 172.21.15.120:0/139999976 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d30ab63d40 0x55d30ab66160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:07:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.289+0000 7f9a8a7da740 4 client.17262 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:07:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.289+0000 7f9a8a7da740 1 -- 172.21.15.120:0/139999976 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d30aa4dd70 con 0x55d30ab63d40 2024-09-19T09:07:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.289+0000 7f9a85704640 1 --2- 172.21.15.120:0/139999976 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d30ab63d40 0x55d30ab66160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:07:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.290+0000 7f9a68ff9640 10 client.17262 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.290+0000 7f9a85704640 1 --2- 172.21.15.120:0/139999976 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d30ab63d40 0x55d30ab66160 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=450731a6a27d997 server_cookie=41adccf3ec1f22da in_seq=0 out_seq=0 2024-09-19T09:07:53.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:52 smithi120 ceph-mon[31150]: pgmap v492: 97 pgs: 97 active+clean; 32 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 191 op/s 2024-09-19T09:07:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.358+0000 7f9a68ff9640 1 -- 172.21.15.120:0/139999976 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d30aa4dd70 con 0x55d30ab63d40 2024-09-19T09:07:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.358+0000 7f9a68ff9640 10 client.17262 handle_command_reply: tid=0 2024-09-19T09:07:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.358+0000 7f9a8a7da740 1 client.17262 shutdown 2024-09-19T09:07:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.358+0000 7f9a8a7da740 1 -- 172.21.15.120:0/139999976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30aae32e0 msgr2=0x55d30aaf1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.358+0000 7f9a8a7da740 1 --2- 172.21.15.120:0/139999976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30aae32e0 0x55d30aaf1fc0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f9a70098430 tx=0x7f9a7006a7a0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.358+0000 7f9a8a7da740 1 -- 172.21.15.120:0/139999976 shutdown_connections 2024-09-19T09:07:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.358+0000 7f9a8a7da740 1 --2- 172.21.15.120:0/139999976 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d30ab63d40 0x55d30ab66160 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.358+0000 7f9a8a7da740 1 --2- 172.21.15.120:0/139999976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30aae8370 0x55d30aaf2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.358+0000 7f9a8a7da740 1 --2- 172.21.15.120:0/139999976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30aae32e0 0x55d30aaf1fc0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.358+0000 7f9a68ff9640 0 client.17262 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:07:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.358+0000 7f9a8a7da740 1 --2- 172.21.15.120:0/139999976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30aae29a0 0x55d30aaf1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.358+0000 7f9a8a7da740 1 -- 172.21.15.120:0/139999976 >> 172.21.15.120:0/139999976 conn(0x55d30aade010 msgr2=0x55d30aadc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.358+0000 7f9a8a7da740 1 -- 172.21.15.120:0/139999976 shutdown_connections 2024-09-19T09:07:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.358+0000 7f9a8a7da740 1 -- 172.21.15.120:0/139999976 wait complete. 2024-09-19T09:07:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.358+0000 7f9a8a7da740 20 client.17262 trim_cache size 0 max 16384 2024-09-19T09:07:53.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:07:53.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:07:53.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:07:53.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:07:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.359+0000 7f9a87968640 1 -- 172.21.15.120:0/1635550805 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9a6c0a32f0 msgr2=0x7f9a6c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.359+0000 7f9a87968640 1 --2- 172.21.15.120:0/1635550805 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9a6c0a32f0 0x7f9a6c0a57b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9a78009fd0 tx=0x7f9a78005e00 comp rx=0 tx=0).stop 2024-09-19T09:07:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.359+0000 7f9a87968640 1 -- 172.21.15.120:0/1635550805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a80102d50 msgr2=0x7f9a8019a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:07:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.359+0000 7f9a87968640 1 --2- 172.21.15.120:0/1635550805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a80102d50 0x7f9a8019a580 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f9a7000d770 tx=0x7f9a7000dc30 comp rx=0 tx=0).stop 2024-09-19T09:07:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.359+0000 7f9a87968640 1 -- 172.21.15.120:0/1635550805 shutdown_connections 2024-09-19T09:07:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.359+0000 7f9a87968640 1 --2- 172.21.15.120:0/1635550805 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9a6c0a32f0 0x7f9a6c0a57b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.359+0000 7f9a87968640 1 --2- 172.21.15.120:0/1635550805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a80103710 0x7f9a801a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.359+0000 7f9a87968640 1 --2- 172.21.15.120:0/1635550805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a80102d50 0x7f9a8019a580 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.359+0000 7f9a87968640 1 --2- 172.21.15.120:0/1635550805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a80101b60 0x7f9a8019a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:07:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.359+0000 7f9a87968640 1 -- 172.21.15.120:0/1635550805 >> 172.21.15.120:0/1635550805 conn(0x7f9a800fd2e0 msgr2=0x7f9a80074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:07:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.359+0000 7f9a87968640 1 -- 172.21.15.120:0/1635550805 shutdown_connections 2024-09-19T09:07:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:07:53.359+0000 7f9a87968640 1 -- 172.21.15.120:0/1635550805 wait complete. 2024-09-19T09:07:54.048 INFO:tasks.workunit.client.0.smithi120.stdout: 1 375454 7.31 MB/sec execute 109 sec latency 8713.862 ms 2024-09-19T09:07:55.049 INFO:tasks.workunit.client.0.smithi120.stdout: 1 375454 7.24 MB/sec execute 110 sec latency 9713.964 ms 2024-09-19T09:07:55.190 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:54 smithi196 ceph-mon[35150]: pgmap v493: 97 pgs: 97 active+clean; 32 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 898 KiB/s wr, 153 op/s 2024-09-19T09:07:55.191 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:54 smithi153 ceph-mon[35071]: pgmap v493: 97 pgs: 97 active+clean; 32 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 898 KiB/s wr, 153 op/s 2024-09-19T09:07:55.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:54 smithi120 ceph-mon[31150]: pgmap v493: 97 pgs: 97 active+clean; 32 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 898 KiB/s wr, 153 op/s 2024-09-19T09:07:56.049 INFO:tasks.workunit.client.0.smithi120.stdout: 1 375454 7.18 MB/sec execute 111 sec latency 10714.080 ms 2024-09-19T09:07:57.049 INFO:tasks.workunit.client.0.smithi120.stdout: 1 375454 7.11 MB/sec execute 112 sec latency 11714.183 ms 2024-09-19T09:07:57.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:56 smithi196 ceph-mon[35150]: pgmap v494: 97 pgs: 97 active+clean; 32 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 593 KiB/s wr, 119 op/s 2024-09-19T09:07:57.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:56 smithi153 ceph-mon[35071]: pgmap v494: 97 pgs: 97 active+clean; 32 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 593 KiB/s wr, 119 op/s 2024-09-19T09:07:57.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:56 smithi120 ceph-mon[31150]: pgmap v494: 97 pgs: 97 active+clean; 32 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 593 KiB/s wr, 119 op/s 2024-09-19T09:07:58.049 INFO:tasks.workunit.client.0.smithi120.stdout: 1 375454 7.05 MB/sec execute 113 sec latency 12714.275 ms 2024-09-19T09:07:59.049 INFO:tasks.workunit.client.0.smithi120.stdout: 1 375454 6.99 MB/sec execute 114 sec latency 13714.345 ms 2024-09-19T09:07:59.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:58 smithi196 ceph-mon[35150]: pgmap v495: 97 pgs: 97 active+clean; 28 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 512 KiB/s wr, 122 op/s 2024-09-19T09:07:59.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:58 smithi153 ceph-mon[35071]: pgmap v495: 97 pgs: 97 active+clean; 28 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 512 KiB/s wr, 122 op/s 2024-09-19T09:07:59.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:58 smithi120 ceph-mon[31150]: pgmap v495: 97 pgs: 97 active+clean; 28 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 512 KiB/s wr, 122 op/s 2024-09-19T09:08:00.049 INFO:tasks.workunit.client.0.smithi120.stdout: 1 375454 6.93 MB/sec execute 115 sec latency 14714.450 ms 2024-09-19T09:08:00.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:07:59 smithi120 ceph-mon[31150]: pgmap v496: 97 pgs: 97 active+clean; 28 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 81 KiB/s wr, 65 op/s 2024-09-19T09:08:00.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:07:59 smithi196 ceph-mon[35150]: pgmap v496: 97 pgs: 97 active+clean; 28 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 81 KiB/s wr, 65 op/s 2024-09-19T09:08:00.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:07:59 smithi153 ceph-mon[35071]: pgmap v496: 97 pgs: 97 active+clean; 28 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 81 KiB/s wr, 65 op/s 2024-09-19T09:08:01.049 INFO:tasks.workunit.client.0.smithi120.stdout: 1 375454 6.87 MB/sec execute 116 sec latency 15714.535 ms 2024-09-19T09:08:01.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:01 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:08:01.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:01 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:08:01.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:01 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:08:02.049 INFO:tasks.workunit.client.0.smithi120.stdout: 1 375454 6.81 MB/sec execute 117 sec latency 16714.586 ms 2024-09-19T09:08:02.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:02 smithi196 ceph-mon[35150]: pgmap v497: 97 pgs: 97 active+clean; 29 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 94 KiB/s wr, 59 op/s 2024-09-19T09:08:02.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:02 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:08:02.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:02 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:08:02.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:02 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:08:02.687 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:02 smithi153 ceph-mon[35071]: pgmap v497: 97 pgs: 97 active+clean; 29 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 94 KiB/s wr, 59 op/s 2024-09-19T09:08:02.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:02 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:08:02.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:02 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:08:02.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:02 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:08:02.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:02 smithi120 ceph-mon[31150]: pgmap v497: 97 pgs: 97 active+clean; 29 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 94 KiB/s wr, 59 op/s 2024-09-19T09:08:02.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:02 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:08:02.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:02 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:08:02.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:02 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:08:03.049 INFO:tasks.workunit.client.0.smithi120.stdout: 1 375454 6.75 MB/sec execute 118 sec latency 17714.624 ms 2024-09-19T09:08:04.049 INFO:tasks.workunit.client.0.smithi120.stdout: 1 375454 6.70 MB/sec execute 119 sec latency 18714.695 ms 2024-09-19T09:08:05.049 INFO:tasks.workunit.client.0.smithi120.stdout: 1 375454 6.64 MB/sec execute 120 sec latency 19714.745 ms 2024-09-19T09:08:05.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:04 smithi120 ceph-mon[31150]: pgmap v498: 97 pgs: 97 active+clean; 36 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 961 KiB/s wr, 72 op/s 2024-09-19T09:08:05.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:04 smithi196 ceph-mon[35150]: pgmap v498: 97 pgs: 97 active+clean; 36 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 961 KiB/s wr, 72 op/s 2024-09-19T09:08:05.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:04 smithi153 ceph-mon[35071]: pgmap v498: 97 pgs: 97 active+clean; 36 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 961 KiB/s wr, 72 op/s 2024-09-19T09:08:06.049 INFO:tasks.workunit.client.0.smithi120.stdout: 1 375454 6.58 MB/sec execute 121 sec latency 20714.839 ms 2024-09-19T09:08:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.456+0000 7fd773827640 1 -- 172.21.15.120:0/1564513833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd76c102d50 msgr2=0x7fd76c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.456+0000 7fd773827640 1 --2- 172.21.15.120:0/1564513833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd76c102d50 0x7fd76c1031d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fd75c009900 tx=0x7fd75c02a410 comp rx=0 tx=0).stop 2024-09-19T09:08:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.456+0000 7fd773827640 1 -- 172.21.15.120:0/1564513833 shutdown_connections 2024-09-19T09:08:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.456+0000 7fd773827640 1 --2- 172.21.15.120:0/1564513833 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd76c103710 0x7fd76c109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.456+0000 7fd773827640 1 --2- 172.21.15.120:0/1564513833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd76c102d50 0x7fd76c1031d0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.456+0000 7fd773827640 1 --2- 172.21.15.120:0/1564513833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd76c101b60 0x7fd76c101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.456+0000 7fd773827640 1 -- 172.21.15.120:0/1564513833 >> 172.21.15.120:0/1564513833 conn(0x7fd76c0fd2e0 msgr2=0x7fd76c0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.456+0000 7fd773827640 1 -- 172.21.15.120:0/1564513833 shutdown_connections 2024-09-19T09:08:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.456+0000 7fd773827640 1 -- 172.21.15.120:0/1564513833 wait complete. 2024-09-19T09:08:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.456+0000 7fd773827640 1 Processor -- start 2024-09-19T09:08:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.456+0000 7fd773827640 1 -- start start 2024-09-19T09:08:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.457+0000 7fd773827640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd76c101b60 0x7fd76c197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.457+0000 7fd773827640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd76c102d50 0x7fd76c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.457+0000 7fd773827640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd76c103710 0x7fd76c19f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.457+0000 7fd773827640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c10d830 con 0x7fd76c101b60 2024-09-19T09:08:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.457+0000 7fd773827640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c10d6b0 con 0x7fd76c102d50 2024-09-19T09:08:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.457+0000 7fd773827640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c10d9b0 con 0x7fd76c103710 2024-09-19T09:08:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.457+0000 7fd7715c3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd76c101b60 0x7fd76c197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.457+0000 7fd7715c3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd76c101b60 0x7fd76c197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37344/0 (socket says 172.21.15.120:37344) 2024-09-19T09:08:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.457+0000 7fd7715c3640 1 -- 172.21.15.120:0/2805683496 learned_addr learned my addr 172.21.15.120:0/2805683496 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.457+0000 7fd770dc2640 1 --2- 172.21.15.120:0/2805683496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd76c102d50 0x7fd76c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.457+0000 7fd771dc4640 1 --2- 172.21.15.120:0/2805683496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd76c103710 0x7fd76c19f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.457+0000 7fd7715c3640 1 -- 172.21.15.120:0/2805683496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd76c103710 msgr2=0x7fd76c19f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.457+0000 7fd7715c3640 1 --2- 172.21.15.120:0/2805683496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd76c103710 0x7fd76c19f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.457+0000 7fd7715c3640 1 -- 172.21.15.120:0/2805683496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd76c102d50 msgr2=0x7fd76c198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.457+0000 7fd7715c3640 1 --2- 172.21.15.120:0/2805683496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd76c102d50 0x7fd76c198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.457+0000 7fd7715c3640 1 -- 172.21.15.120:0/2805683496 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd76c19f9d0 con 0x7fd76c101b60 2024-09-19T09:08:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.457+0000 7fd7715c3640 1 --2- 172.21.15.120:0/2805683496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd76c101b60 0x7fd76c197dc0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fd76800e9c0 tx=0x7fd76800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.457+0000 7fd7627fc640 1 -- 172.21.15.120:0/2805683496 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76800cd80 con 0x7fd76c101b60 2024-09-19T09:08:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.457+0000 7fd7627fc640 1 -- 172.21.15.120:0/2805683496 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7680035f0 con 0x7fd76c101b60 2024-09-19T09:08:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.458+0000 7fd7627fc640 1 -- 172.21.15.120:0/2805683496 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd768010690 con 0x7fd76c101b60 2024-09-19T09:08:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.458+0000 7fd773827640 1 -- 172.21.15.120:0/2805683496 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd76c19fcc0 con 0x7fd76c101b60 2024-09-19T09:08:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.458+0000 7fd773827640 1 -- 172.21.15.120:0/2805683496 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd76c1a01c0 con 0x7fd76c101b60 2024-09-19T09:08:06.470 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.461+0000 7fd7765c0740 1 Processor -- start 2024-09-19T09:08:06.470 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.461+0000 7fd7627fc640 1 -- 172.21.15.120:0/2805683496 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd7680037a0 con 0x7fd76c101b60 2024-09-19T09:08:06.470 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.461+0000 7fd7765c0740 1 -- start start 2024-09-19T09:08:06.470 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.461+0000 7fd7765c0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642e4f3cdb0 0x5642e4f3d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:06.470 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.461+0000 7fd7765c0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642e4f3d6f0 0x5642e4f421b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:06.470 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.461+0000 7fd7765c0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4f42780 0x5642e4f44c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:06.470 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.461+0000 7fd7765c0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5642e4e9c2f0 con 0x5642e4f3d6f0 2024-09-19T09:08:06.470 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.461+0000 7fd7765c0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5642e4e44270 con 0x5642e4f3cdb0 2024-09-19T09:08:06.470 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.461+0000 7fd7765c0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5642e4e982b0 con 0x5642e4f42780 2024-09-19T09:08:06.470 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.461+0000 7fd7715c3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642e4f3d6f0 0x5642e4f421b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.461+0000 7fd7715c3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642e4f3d6f0 0x5642e4f421b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37360/0 (socket says 172.21.15.120:37360) 2024-09-19T09:08:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.461+0000 7fd7715c3640 1 -- 172.21.15.120:0/2448714136 learned_addr learned my addr 172.21.15.120:0/2448714136 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.461+0000 7fd771dc4640 1 --2- 172.21.15.120:0/2448714136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642e4f3cdb0 0x5642e4f3d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.461+0000 7fd770dc2640 1 --2- 172.21.15.120:0/2448714136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4f42780 0x5642e4f44c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.462+0000 7fd7715c3640 1 -- 172.21.15.120:0/2448714136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4f42780 msgr2=0x5642e4f44c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.462+0000 7fd7715c3640 1 --2- 172.21.15.120:0/2448714136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4f42780 0x5642e4f44c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.462+0000 7fd7715c3640 1 -- 172.21.15.120:0/2448714136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642e4f3cdb0 msgr2=0x5642e4f3d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.462+0000 7fd7627fc640 1 --2- 172.21.15.120:0/2805683496 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd7580a32f0 0x7fd7580a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.462+0000 7fd7715c3640 1 --2- 172.21.15.120:0/2448714136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642e4f3cdb0 0x5642e4f3d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.462+0000 7fd7715c3640 1 -- 172.21.15.120:0/2448714136 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642e4e43340 con 0x5642e4f3d6f0 2024-09-19T09:08:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.462+0000 7fd771dc4640 1 --2- 172.21.15.120:0/2805683496 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd7580a32f0 0x7fd7580a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.462+0000 7fd7627fc640 1 -- 172.21.15.120:0/2805683496 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fd7680cfcf0 con 0x7fd76c101b60 2024-09-19T09:08:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.462+0000 7fd7715c3640 1 --2- 172.21.15.120:0/2448714136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642e4f3d6f0 0x5642e4f421b0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fd76809f1a0 tx=0x7fd7680985e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8101b3e929c3945 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.462+0000 7fd7577fe640 1 -- 172.21.15.120:0/2448714136 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76809c0e0 con 0x5642e4f3d6f0 2024-09-19T09:08:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.462+0000 7fd7577fe640 1 -- 172.21.15.120:0/2448714136 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd76809f490 con 0x5642e4f3d6f0 2024-09-19T09:08:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.462+0000 7fd7577fe640 1 -- 172.21.15.120:0/2448714136 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76809f230 con 0x5642e4f3d6f0 2024-09-19T09:08:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.463+0000 7fd7765c0740 1 -- 172.21.15.120:0/2448714136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642e4f3d6f0 msgr2=0x5642e4f421b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.463+0000 7fd771dc4640 1 --2- 172.21.15.120:0/2805683496 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd7580a32f0 0x7fd7580a57b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd764005fd0 tx=0x7fd764004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.463+0000 7fd7765c0740 1 --2- 172.21.15.120:0/2448714136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642e4f3d6f0 0x5642e4f421b0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fd76809f1a0 tx=0x7fd7680985e0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.463+0000 7fd7765c0740 1 -- 172.21.15.120:0/2448714136 shutdown_connections 2024-09-19T09:08:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.463+0000 7fd7765c0740 1 --2- 172.21.15.120:0/2448714136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4f42780 0x5642e4f44c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.463+0000 7fd7765c0740 1 --2- 172.21.15.120:0/2448714136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642e4f3d6f0 0x5642e4f421b0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.463+0000 7fd7765c0740 1 --2- 172.21.15.120:0/2448714136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642e4f3cdb0 0x5642e4f3d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.463+0000 7fd7765c0740 1 -- 172.21.15.120:0/2448714136 >> 172.21.15.120:0/2448714136 conn(0x5642e4f34010 msgr2=0x5642e4f34410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.463+0000 7fd7765c0740 1 -- 172.21.15.120:0/2448714136 shutdown_connections 2024-09-19T09:08:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.463+0000 7fd7765c0740 1 -- 172.21.15.120:0/2448714136 wait complete. 2024-09-19T09:08:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.463+0000 7fd7765c0740 1 -- start start 2024-09-19T09:08:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.463+0000 7fd7765c0740 1 Processor -- start 2024-09-19T09:08:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.464+0000 7fd7765c0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642e4f3cdb0 0x5642e4f46ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.464+0000 7fd7765c0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642e4f3d6f0 0x5642e4f47400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.464+0000 7fd7765c0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4f42780 0x5642e4f4e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.464+0000 7fd7765c0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5642e4f45690 con 0x5642e4f3d6f0 2024-09-19T09:08:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.464+0000 7fd7765c0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5642e4e25290 con 0x5642e4f3cdb0 2024-09-19T09:08:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.464+0000 7fd7765c0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5642e4f45810 con 0x5642e4f42780 2024-09-19T09:08:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.464+0000 7fd7765c0740 10 client.0 _ll_register_callbacks cb 0x5642e4e56140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:08:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.464+0000 7fd770dc2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642e4f3d6f0 0x5642e4f47400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.464+0000 7fd770dc2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642e4f3d6f0 0x5642e4f47400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37376/0 (socket says 172.21.15.120:37376) 2024-09-19T09:08:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.464+0000 7fd770dc2640 1 -- 172.21.15.120:0/2447614225 learned_addr learned my addr 172.21.15.120:0/2447614225 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.464+0000 7fd771dc4640 1 --2- 172.21.15.120:0/2447614225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4f42780 0x5642e4f4e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.464+0000 7fd7715c3640 1 --2- 172.21.15.120:0/2447614225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642e4f3cdb0 0x5642e4f46ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.464+0000 7fd770dc2640 1 -- 172.21.15.120:0/2447614225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4f42780 msgr2=0x5642e4f4e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.464+0000 7fd770dc2640 1 --2- 172.21.15.120:0/2447614225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4f42780 0x5642e4f4e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.464+0000 7fd770dc2640 1 -- 172.21.15.120:0/2447614225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642e4f3cdb0 msgr2=0x5642e4f46ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.464+0000 7fd770dc2640 1 --2- 172.21.15.120:0/2447614225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642e4f3cdb0 0x5642e4f46ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.464+0000 7fd770dc2640 1 -- 172.21.15.120:0/2447614225 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642e4baf210 con 0x5642e4f3d6f0 2024-09-19T09:08:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.464+0000 7fd770dc2640 1 --2- 172.21.15.120:0/2447614225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642e4f3d6f0 0x5642e4f47400 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fd75c0314e0 tx=0x7fd75c0044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc329558c915718 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.464+0000 7fd754ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:08:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.465+0000 7fd754ff9640 1 -- 172.21.15.120:0/2447614225 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75c035040 con 0x5642e4f3d6f0 2024-09-19T09:08:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.465+0000 7fd754ff9640 1 -- 172.21.15.120:0/2447614225 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fd75c0054d0 con 0x5642e4f3d6f0 2024-09-19T09:08:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.465+0000 7fd7765c0740 1 -- 172.21.15.120:0/2447614225 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5642e4ea9a20 con 0x5642e4f3d6f0 2024-09-19T09:08:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.465+0000 7fd754ff9640 1 -- 172.21.15.120:0/2447614225 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd75c004770 con 0x5642e4f3d6f0 2024-09-19T09:08:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.465+0000 7fd7765c0740 10 client.17274 fetch_fsmap learned FSMap version 30 2024-09-19T09:08:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.465+0000 7fd7765c0740 1 -- 172.21.15.120:0/2447614225 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5642e4c7b620 con 0x5642e4f3d6f0 2024-09-19T09:08:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.465+0000 7fd754ff9640 1 -- 172.21.15.120:0/2447614225 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5642e4ea9a20 con 0x5642e4f3d6f0 2024-09-19T09:08:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.465+0000 7fd754ff9640 1 -- 172.21.15.120:0/2447614225 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd75c0035f0 con 0x5642e4f3d6f0 2024-09-19T09:08:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.465+0000 7fd7765c0740 10 client.17274 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:08:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.465+0000 7fd7765c0740 10 client.17274 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:08:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.465+0000 7fd7765c0740 20 client.17274 populate_metadata read hostname 'smithi120' 2024-09-19T09:08:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.465+0000 7fd7765c0740 1 --2- 172.21.15.120:0/2447614225 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5642e4f52ae0 0x5642e4f54f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.465+0000 7fd7765c0740 4 client.17274 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:08:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.465+0000 7fd7765c0740 1 -- 172.21.15.120:0/2447614225 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642e4ea3d70 con 0x5642e4f52ae0 2024-09-19T09:08:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.465+0000 7fd7715c3640 1 --2- 172.21.15.120:0/2447614225 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5642e4f52ae0 0x5642e4f54f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.466+0000 7fd7715c3640 1 --2- 172.21.15.120:0/2447614225 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5642e4f52ae0 0x5642e4f54f10 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c641fd6513e530e server_cookie=f726c6f649097668 in_seq=0 out_seq=0 2024-09-19T09:08:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.466+0000 7fd754ff9640 10 client.17274 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.467+0000 7fd754ff9640 1 -- 172.21.15.120:0/2447614225 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5642e4ea3d70 con 0x5642e4f52ae0 2024-09-19T09:08:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.467+0000 7fd754ff9640 10 client.17274 handle_command_reply: tid=0 2024-09-19T09:08:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.467+0000 7fd7765c0740 1 client.17274 shutdown 2024-09-19T09:08:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.467+0000 7fd7765c0740 1 -- 172.21.15.120:0/2447614225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642e4f3d6f0 msgr2=0x5642e4f47400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.467+0000 7fd7765c0740 1 --2- 172.21.15.120:0/2447614225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642e4f3d6f0 0x5642e4f47400 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fd75c0314e0 tx=0x7fd75c0044f0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.467+0000 7fd7765c0740 1 -- 172.21.15.120:0/2447614225 shutdown_connections 2024-09-19T09:08:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.467+0000 7fd7765c0740 1 --2- 172.21.15.120:0/2447614225 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5642e4f52ae0 0x5642e4f54f10 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.467+0000 7fd7765c0740 1 --2- 172.21.15.120:0/2447614225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4f42780 0x5642e4f4e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.467+0000 7fd7765c0740 1 --2- 172.21.15.120:0/2447614225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642e4f3d6f0 0x5642e4f47400 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.467+0000 7fd754ff9640 0 client.17274 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.467+0000 7fd7765c0740 1 --2- 172.21.15.120:0/2447614225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642e4f3cdb0 0x5642e4f46ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.467+0000 7fd7765c0740 1 -- 172.21.15.120:0/2447614225 >> 172.21.15.120:0/2447614225 conn(0x5642e4f34010 msgr2=0x5642e4f35820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.467+0000 7fd7765c0740 1 -- 172.21.15.120:0/2447614225 shutdown_connections 2024-09-19T09:08:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.467+0000 7fd7765c0740 1 -- 172.21.15.120:0/2447614225 wait complete. 2024-09-19T09:08:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.467+0000 7fd7765c0740 20 client.17274 trim_cache size 0 max 16384 2024-09-19T09:08:06.487 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.482+0000 7fd7765c0740 1 Processor -- start 2024-09-19T09:08:06.487 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.482+0000 7fd7765c0740 1 -- start start 2024-09-19T09:08:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.482+0000 7fd7765c0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4f3cdb0 0x5642e4f35400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.482+0000 7fd7765c0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642e4f3d6f0 0x5642e4f4e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.482+0000 7fd7765c0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642e4f42780 0x5642e4f35940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.482+0000 7fd7765c0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c1a6d10 con 0x5642e4f42780 2024-09-19T09:08:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.482+0000 7fd7765c0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c1a8850 con 0x5642e4f3d6f0 2024-09-19T09:08:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.482+0000 7fd7765c0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c1a73e0 con 0x5642e4f3cdb0 2024-09-19T09:08:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.482+0000 7fd771dc4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642e4f42780 0x5642e4f35940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.482+0000 7fd771dc4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642e4f42780 0x5642e4f35940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37382/0 (socket says 172.21.15.120:37382) 2024-09-19T09:08:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.482+0000 7fd771dc4640 1 -- 172.21.15.120:0/1243523837 learned_addr learned my addr 172.21.15.120:0/1243523837 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.482+0000 7fd7715c3640 1 --2- 172.21.15.120:0/1243523837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4f3cdb0 0x5642e4f35400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.482+0000 7fd771dc4640 1 -- 172.21.15.120:0/1243523837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4f3cdb0 msgr2=0x5642e4f35400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.482+0000 7fd771dc4640 1 --2- 172.21.15.120:0/1243523837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4f3cdb0 0x5642e4f35400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.482+0000 7fd771dc4640 1 -- 172.21.15.120:0/1243523837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642e4f3d6f0 msgr2=0x5642e4f4e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.482+0000 7fd770dc2640 1 --2- 172.21.15.120:0/1243523837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642e4f3d6f0 0x5642e4f4e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.482+0000 7fd771dc4640 1 --2- 172.21.15.120:0/1243523837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642e4f3d6f0 0x5642e4f4e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.482+0000 7fd771dc4640 1 -- 172.21.15.120:0/1243523837 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642e4f33e30 con 0x5642e4f42780 2024-09-19T09:08:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.482+0000 7fd771dc4640 1 --2- 172.21.15.120:0/1243523837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642e4f42780 0x5642e4f35940 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fd764012760 tx=0x7fd764016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14dc69aed203166 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.483+0000 7fd756ffd640 1 -- 172.21.15.120:0/1243523837 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764021410 con 0x5642e4f42780 2024-09-19T09:08:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.483+0000 7fd756ffd640 1 -- 172.21.15.120:0/1243523837 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd764021a90 con 0x5642e4f42780 2024-09-19T09:08:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.483+0000 7fd756ffd640 1 -- 172.21.15.120:0/1243523837 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764010c00 con 0x5642e4f42780 2024-09-19T09:08:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.483+0000 7fd7765c0740 1 -- 172.21.15.120:0/1243523837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642e4f42780 msgr2=0x5642e4f35940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.483+0000 7fd7765c0740 1 --2- 172.21.15.120:0/1243523837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642e4f42780 0x5642e4f35940 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fd764012760 tx=0x7fd764016000 comp rx=0 tx=0).stop 2024-09-19T09:08:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.483+0000 7fd7765c0740 1 -- 172.21.15.120:0/1243523837 shutdown_connections 2024-09-19T09:08:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.483+0000 7fd7765c0740 1 --2- 172.21.15.120:0/1243523837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642e4f42780 0x5642e4f35940 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.483+0000 7fd7765c0740 1 --2- 172.21.15.120:0/1243523837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642e4f3d6f0 0x5642e4f4e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.483+0000 7fd7765c0740 1 --2- 172.21.15.120:0/1243523837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4f3cdb0 0x5642e4f35400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.483+0000 7fd7765c0740 1 -- 172.21.15.120:0/1243523837 >> 172.21.15.120:0/1243523837 conn(0x5642e4f34010 msgr2=0x5642e4f331c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.483+0000 7fd7765c0740 1 -- 172.21.15.120:0/1243523837 shutdown_connections 2024-09-19T09:08:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.483+0000 7fd7765c0740 1 -- 172.21.15.120:0/1243523837 wait complete. 2024-09-19T09:08:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.484+0000 7fd7765c0740 1 -- start start 2024-09-19T09:08:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.484+0000 7fd7765c0740 1 Processor -- start 2024-09-19T09:08:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.484+0000 7fd7765c0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642e4f3cdb0 0x5642e4f4bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.484+0000 7fd7765c0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642e4f3d6f0 0x5642e4f4c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.484+0000 7fd7765c0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4f42780 0x5642e4f4d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.484+0000 7fd7765c0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c1a8180 con 0x5642e4f3cdb0 2024-09-19T09:08:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.484+0000 7fd7765c0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c1a7ab0 con 0x5642e4f3d6f0 2024-09-19T09:08:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.484+0000 7fd7765c0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c1a4cd0 con 0x5642e4f42780 2024-09-19T09:08:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.484+0000 7fd7765c0740 10 client.0 _ll_register_callbacks cb 0x7fd75c004180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:08:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.484+0000 7fd7715c3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642e4f3cdb0 0x5642e4f4bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.484+0000 7fd7715c3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642e4f3cdb0 0x5642e4f4bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37386/0 (socket says 172.21.15.120:37386) 2024-09-19T09:08:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.484+0000 7fd7715c3640 1 -- 172.21.15.120:0/3678942459 learned_addr learned my addr 172.21.15.120:0/3678942459 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.484+0000 7fd771dc4640 1 --2- 172.21.15.120:0/3678942459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4f42780 0x5642e4f4d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.484+0000 7fd770dc2640 1 --2- 172.21.15.120:0/3678942459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642e4f3d6f0 0x5642e4f4c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.485+0000 7fd7715c3640 1 -- 172.21.15.120:0/3678942459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4f42780 msgr2=0x5642e4f4d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.485+0000 7fd7715c3640 1 --2- 172.21.15.120:0/3678942459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4f42780 0x5642e4f4d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.485+0000 7fd7715c3640 1 -- 172.21.15.120:0/3678942459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642e4f3d6f0 msgr2=0x5642e4f4c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.485+0000 7fd7715c3640 1 --2- 172.21.15.120:0/3678942459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642e4f3d6f0 0x5642e4f4c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.485+0000 7fd7715c3640 1 -- 172.21.15.120:0/3678942459 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642e4f4dad0 con 0x5642e4f3cdb0 2024-09-19T09:08:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.485+0000 7fd771dc4640 1 --2- 172.21.15.120:0/3678942459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4f42780 0x5642e4f4d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:08:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.485+0000 7fd7715c3640 1 --2- 172.21.15.120:0/3678942459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642e4f3cdb0 0x5642e4f4bf70 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fd768004630 tx=0x7fd768005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e731f1e7e33d6459 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.485+0000 7fd754ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:08:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.485+0000 7fd754ff9640 1 -- 172.21.15.120:0/3678942459 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7680d2030 con 0x5642e4f3cdb0 2024-09-19T09:08:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.485+0000 7fd754ff9640 1 -- 172.21.15.120:0/3678942459 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fd7680abe80 con 0x5642e4f3cdb0 2024-09-19T09:08:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.485+0000 7fd7765c0740 1 -- 172.21.15.120:0/3678942459 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd764021a90 con 0x5642e4f3cdb0 2024-09-19T09:08:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.485+0000 7fd754ff9640 1 -- 172.21.15.120:0/3678942459 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd76809a450 con 0x5642e4f3cdb0 2024-09-19T09:08:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.485+0000 7fd7765c0740 10 client.17280 fetch_fsmap learned FSMap version 30 2024-09-19T09:08:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.485+0000 7fd7765c0740 1 -- 172.21.15.120:0/3678942459 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5642e4fbdca0 con 0x5642e4f3cdb0 2024-09-19T09:08:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.485+0000 7fd754ff9640 1 -- 172.21.15.120:0/3678942459 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd764021a90 con 0x5642e4f3cdb0 2024-09-19T09:08:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.485+0000 7fd754ff9640 1 -- 172.21.15.120:0/3678942459 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd7680cd870 con 0x5642e4f3cdb0 2024-09-19T09:08:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.485+0000 7fd7765c0740 10 client.17280 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:08:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.485+0000 7fd7765c0740 10 client.17280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:08:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.485+0000 7fd7765c0740 20 client.17280 populate_metadata read hostname 'smithi120' 2024-09-19T09:08:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.485+0000 7fd7765c0740 1 --2- 172.21.15.120:0/3678942459 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5642e4fbe1f0 0x5642e4fc0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:06.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.485+0000 7fd7765c0740 4 client.17280 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:08:06.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.485+0000 7fd7765c0740 1 -- 172.21.15.120:0/3678942459 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5642e4e20900 con 0x5642e4fbe1f0 2024-09-19T09:08:06.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.486+0000 7fd770dc2640 1 --2- 172.21.15.120:0/3678942459 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5642e4fbe1f0 0x5642e4fc0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:06.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.486+0000 7fd754ff9640 10 client.17280 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:06.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.486+0000 7fd770dc2640 1 --2- 172.21.15.120:0/3678942459 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5642e4fbe1f0 0x5642e4fc0610 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61099b39f736f6e6 server_cookie=c673b80394a05790 in_seq=0 out_seq=0 2024-09-19T09:08:06.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.514+0000 7fd754ff9640 1 -- 172.21.15.120:0/3678942459 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5642e4e20900 con 0x5642e4fbe1f0 2024-09-19T09:08:06.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.514+0000 7fd754ff9640 10 client.17280 handle_command_reply: tid=0 2024-09-19T09:08:06.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.514+0000 7fd7765c0740 1 client.17280 shutdown 2024-09-19T09:08:06.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.514+0000 7fd7765c0740 1 -- 172.21.15.120:0/3678942459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642e4f3cdb0 msgr2=0x5642e4f4bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:06.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.514+0000 7fd7765c0740 1 --2- 172.21.15.120:0/3678942459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642e4f3cdb0 0x5642e4f4bf70 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fd768004630 tx=0x7fd768005f10 comp rx=0 tx=0).stop 2024-09-19T09:08:06.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.514+0000 7fd7765c0740 1 -- 172.21.15.120:0/3678942459 shutdown_connections 2024-09-19T09:08:06.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.514+0000 7fd7765c0740 1 --2- 172.21.15.120:0/3678942459 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5642e4fbe1f0 0x5642e4fc0610 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.514+0000 7fd7765c0740 1 --2- 172.21.15.120:0/3678942459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4f42780 0x5642e4f4d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.514+0000 7fd754ff9640 0 client.17280 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:06.517 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.514+0000 7fd7765c0740 1 --2- 172.21.15.120:0/3678942459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642e4f3d6f0 0x5642e4f4c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.517 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.514+0000 7fd7765c0740 1 --2- 172.21.15.120:0/3678942459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642e4f3cdb0 0x5642e4f4bf70 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.517 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.514+0000 7fd7765c0740 1 -- 172.21.15.120:0/3678942459 >> 172.21.15.120:0/3678942459 conn(0x5642e4f34010 msgr2=0x5642e4f343f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:06.517 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.514+0000 7fd7765c0740 1 -- 172.21.15.120:0/3678942459 shutdown_connections 2024-09-19T09:08:06.517 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.514+0000 7fd7765c0740 1 -- 172.21.15.120:0/3678942459 wait complete. 2024-09-19T09:08:06.517 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.514+0000 7fd7765c0740 20 client.17280 trim_cache size 0 max 16384 2024-09-19T09:08:06.517 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:08:06.517 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:08:06.517 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:08:06.518 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:08:06.518 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.515+0000 7fd773827640 1 -- 172.21.15.120:0/2805683496 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd7580a32f0 msgr2=0x7fd7580a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:06.518 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.515+0000 7fd773827640 1 --2- 172.21.15.120:0/2805683496 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd7580a32f0 0x7fd7580a57b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd764005fd0 tx=0x7fd764004a40 comp rx=0 tx=0).stop 2024-09-19T09:08:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.515+0000 7fd773827640 1 -- 172.21.15.120:0/2805683496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd76c101b60 msgr2=0x7fd76c197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.515+0000 7fd773827640 1 --2- 172.21.15.120:0/2805683496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd76c101b60 0x7fd76c197dc0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fd76800e9c0 tx=0x7fd76800ee80 comp rx=0 tx=0).stop 2024-09-19T09:08:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.515+0000 7fd773827640 1 -- 172.21.15.120:0/2805683496 shutdown_connections 2024-09-19T09:08:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.515+0000 7fd773827640 1 --2- 172.21.15.120:0/2805683496 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd7580a32f0 0x7fd7580a57b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.515+0000 7fd773827640 1 --2- 172.21.15.120:0/2805683496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd76c103710 0x7fd76c19f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.515+0000 7fd773827640 1 --2- 172.21.15.120:0/2805683496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd76c102d50 0x7fd76c198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.515+0000 7fd773827640 1 --2- 172.21.15.120:0/2805683496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd76c101b60 0x7fd76c197dc0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.515+0000 7fd773827640 1 -- 172.21.15.120:0/2805683496 >> 172.21.15.120:0/2805683496 conn(0x7fd76c0fd2e0 msgr2=0x7fd76c106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.515+0000 7fd773827640 1 -- 172.21.15.120:0/2805683496 shutdown_connections 2024-09-19T09:08:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:06.515+0000 7fd773827640 1 -- 172.21.15.120:0/2805683496 wait complete. 2024-09-19T09:08:07.049 INFO:tasks.workunit.client.0.smithi120.stdout: 1 375454 6.53 MB/sec execute 122 sec latency 21714.918 ms 2024-09-19T09:08:07.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:06 smithi120 ceph-mon[31150]: pgmap v499: 97 pgs: 97 active+clean; 36 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 1.1 MiB/s wr, 75 op/s 2024-09-19T09:08:07.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:06 smithi196 ceph-mon[35150]: pgmap v499: 97 pgs: 97 active+clean; 36 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 1.1 MiB/s wr, 75 op/s 2024-09-19T09:08:07.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:06 smithi153 ceph-mon[35071]: pgmap v499: 97 pgs: 97 active+clean; 36 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 1.1 MiB/s wr, 75 op/s 2024-09-19T09:08:08.049 INFO:tasks.workunit.client.0.smithi120.stdout: 1 375454 6.48 MB/sec execute 123 sec latency 22715.016 ms 2024-09-19T09:08:09.050 INFO:tasks.workunit.client.0.smithi120.stdout: 1 375454 6.43 MB/sec execute 124 sec latency 23715.120 ms 2024-09-19T09:08:09.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:08 smithi120 ceph-mon[31150]: pgmap v500: 97 pgs: 97 active+clean; 33 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 1.2 MiB/s wr, 79 op/s 2024-09-19T09:08:09.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:08 smithi196 ceph-mon[35150]: pgmap v500: 97 pgs: 97 active+clean; 33 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 1.2 MiB/s wr, 79 op/s 2024-09-19T09:08:09.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:08 smithi153 ceph-mon[35071]: pgmap v500: 97 pgs: 97 active+clean; 33 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 1.2 MiB/s wr, 79 op/s 2024-09-19T09:08:10.050 INFO:tasks.workunit.client.0.smithi120.stdout: 1 375454 6.37 MB/sec execute 125 sec latency 24715.199 ms 2024-09-19T09:08:10.121 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:08:10.378 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:08:10.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.871+0000 7f33cad89640 1 -- 172.21.15.120:0/811399508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33c41027a0 msgr2=0x7f33c41067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:10.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.871+0000 7f33cad89640 1 --2- 172.21.15.120:0/811399508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33c41027a0 0x7f33c41067f0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009900 tx=0x7f33b002a410 comp rx=0 tx=0).stop 2024-09-19T09:08:10.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.871+0000 7f33cad89640 1 -- 172.21.15.120:0/811399508 shutdown_connections 2024-09-19T09:08:10.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.871+0000 7f33cad89640 1 --2- 172.21.15.120:0/811399508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33c4106f20 0x7f33c410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:10.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.871+0000 7f33cad89640 1 --2- 172.21.15.120:0/811399508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33c41027a0 0x7f33c41067f0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:10.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.871+0000 7f33cad89640 1 --2- 172.21.15.120:0/811399508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33c4101df0 0x7f33c41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:10.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.871+0000 7f33cad89640 1 -- 172.21.15.120:0/811399508 >> 172.21.15.120:0/811399508 conn(0x7f33c40fd8f0 msgr2=0x7f33c40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:10.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.871+0000 7f33cad89640 1 -- 172.21.15.120:0/811399508 shutdown_connections 2024-09-19T09:08:10.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.871+0000 7f33cad89640 1 -- 172.21.15.120:0/811399508 wait complete. 2024-09-19T09:08:10.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.872+0000 7f33cad89640 1 Processor -- start 2024-09-19T09:08:10.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.872+0000 7f33cad89640 1 -- start start 2024-09-19T09:08:10.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.872+0000 7f33cad89640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33c4101df0 0x7f33c419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:10.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.872+0000 7f33cad89640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33c41027a0 0x7f33c419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:10.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.872+0000 7f33cad89640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33c4106f20 0x7f33c419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:10.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.872+0000 7f33cad89640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f33c4111f80 con 0x7f33c4101df0 2024-09-19T09:08:10.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.872+0000 7f33cad89640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f33c4111e00 con 0x7f33c4106f20 2024-09-19T09:08:10.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.872+0000 7f33cad89640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f33c4112100 con 0x7f33c41027a0 2024-09-19T09:08:10.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.872+0000 7f33c3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33c41027a0 0x7f33c419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:10.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.872+0000 7f33c8b25640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33c4101df0 0x7f33c419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:10.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.872+0000 7f33c3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33c41027a0 0x7f33c419aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:40064/0 (socket says 172.21.15.120:40064) 2024-09-19T09:08:10.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.872+0000 7f33c8b25640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33c4101df0 0x7f33c419a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37400/0 (socket says 172.21.15.120:37400) 2024-09-19T09:08:10.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.872+0000 7f33c3fff640 1 -- 172.21.15.120:0/199193995 learned_addr learned my addr 172.21.15.120:0/199193995 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:10.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.872+0000 7f33c9326640 1 --2- 172.21.15.120:0/199193995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33c4106f20 0x7f33c419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:10.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.873+0000 7f33c8b25640 1 -- 172.21.15.120:0/199193995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33c41027a0 msgr2=0x7f33c419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:10.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.873+0000 7f33c8b25640 1 --2- 172.21.15.120:0/199193995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33c41027a0 0x7f33c419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:10.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.873+0000 7f33c8b25640 1 -- 172.21.15.120:0/199193995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33c4106f20 msgr2=0x7f33c419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:10.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.873+0000 7f33c8b25640 1 --2- 172.21.15.120:0/199193995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33c4106f20 0x7f33c419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:10.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.873+0000 7f33c8b25640 1 -- 172.21.15.120:0/199193995 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c419f610 con 0x7f33c4101df0 2024-09-19T09:08:10.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.873+0000 7f33c8b25640 1 --2- 172.21.15.120:0/199193995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33c4101df0 0x7f33c419a660 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f33b800d6d0 tx=0x7f33b800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:10.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.873+0000 7f33c1ffb640 1 -- 172.21.15.120:0/199193995 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b8017070 con 0x7f33c4101df0 2024-09-19T09:08:10.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.873+0000 7f33cad89640 1 -- 172.21.15.120:0/199193995 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c419f900 con 0x7f33c4101df0 2024-09-19T09:08:10.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.873+0000 7f33c1ffb640 1 -- 172.21.15.120:0/199193995 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33b8009820 con 0x7f33c4101df0 2024-09-19T09:08:10.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.873+0000 7f33c1ffb640 1 -- 172.21.15.120:0/199193995 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b8004d00 con 0x7f33c4101df0 2024-09-19T09:08:10.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.873+0000 7f33cad89640 1 -- 172.21.15.120:0/199193995 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c41a71e0 con 0x7f33c4101df0 2024-09-19T09:08:10.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.874+0000 7f33cad89640 1 -- 172.21.15.120:0/199193995 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c41021d0 con 0x7f33c4101df0 2024-09-19T09:08:10.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.875+0000 7f33c1ffb640 1 -- 172.21.15.120:0/199193995 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f33b80099d0 con 0x7f33c4101df0 2024-09-19T09:08:10.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.876+0000 7f33c1ffb640 1 --2- 172.21.15.120:0/199193995 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33b40a3240 0x7f33b40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:10.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.876+0000 7f33c1ffb640 1 -- 172.21.15.120:0/199193995 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f33b80cf430 con 0x7f33c4101df0 2024-09-19T09:08:10.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.876+0000 7f33c3fff640 1 --2- 172.21.15.120:0/199193995 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33b40a3240 0x7f33b40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:10.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.876+0000 7f33c3fff640 1 --2- 172.21.15.120:0/199193995 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33b40a3240 0x7f33b40a5700 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f33b0031550 tx=0x7f33b0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:10.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:10.880+0000 7f33c1ffb640 1 -- 172.21.15.120:0/199193995 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f33b809bb00 con 0x7f33c4101df0 2024-09-19T09:08:11.050 INFO:tasks.workunit.client.0.smithi120.stdout: 1 375454 6.32 MB/sec execute 126 sec latency 25715.254 ms 2024-09-19T09:08:11.053 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:10 smithi120 ceph-mon[31150]: pgmap v501: 97 pgs: 97 active+clean; 28 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.2 MiB/s wr, 68 op/s 2024-09-19T09:08:11.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:11.089+0000 7f33cad89640 1 -- 172.21.15.120:0/199193995 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33c41a73b0 con 0x7f33c4101df0 2024-09-19T09:08:11.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:11.090+0000 7f33c1ffb640 1 -- 172.21.15.120:0/199193995 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11985 (secure 0 0 0) 0x7f33b809fe00 con 0x7f33c4101df0 2024-09-19T09:08:11.091 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:08:11.091 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":30,"btime":"2024-09-19T09:07:14:812065+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:07:14.812046+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26351":{"gid":26351,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/364941779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":364941779},{"type":"v1","addr":"172.21.15.153:6835","nonce":364941779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:08:11.091 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 30 2024-09-19T09:08:11.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:11.093+0000 7f33cad89640 1 -- 172.21.15.120:0/199193995 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33b40a3240 msgr2=0x7f33b40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:11.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:11.093+0000 7f33cad89640 1 --2- 172.21.15.120:0/199193995 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33b40a3240 0x7f33b40a5700 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f33b0031550 tx=0x7f33b0005a20 comp rx=0 tx=0).stop 2024-09-19T09:08:11.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:11.093+0000 7f33cad89640 1 -- 172.21.15.120:0/199193995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33c4101df0 msgr2=0x7f33c419a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:11.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:11.093+0000 7f33cad89640 1 --2- 172.21.15.120:0/199193995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33c4101df0 0x7f33c419a660 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f33b800d6d0 tx=0x7f33b800db90 comp rx=0 tx=0).stop 2024-09-19T09:08:11.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:11.093+0000 7f33cad89640 1 -- 172.21.15.120:0/199193995 shutdown_connections 2024-09-19T09:08:11.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:11.093+0000 7f33cad89640 1 --2- 172.21.15.120:0/199193995 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f33b40a3240 0x7f33b40a5700 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:11.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:11.093+0000 7f33cad89640 1 --2- 172.21.15.120:0/199193995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33c4106f20 0x7f33c419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:11.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:11.093+0000 7f33cad89640 1 --2- 172.21.15.120:0/199193995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33c41027a0 0x7f33c419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:11.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:11.093+0000 7f33cad89640 1 --2- 172.21.15.120:0/199193995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33c4101df0 0x7f33c419a660 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:11.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:11.093+0000 7f33cad89640 1 -- 172.21.15.120:0/199193995 >> 172.21.15.120:0/199193995 conn(0x7f33c40fd8f0 msgr2=0x7f33c40feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:11.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:11.093+0000 7f33cad89640 1 -- 172.21.15.120:0/199193995 shutdown_connections 2024-09-19T09:08:11.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:11.093+0000 7f33cad89640 1 -- 172.21.15.120:0/199193995 wait complete. 2024-09-19T09:08:11.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:10 smithi196 ceph-mon[35150]: pgmap v501: 97 pgs: 97 active+clean; 28 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.2 MiB/s wr, 68 op/s 2024-09-19T09:08:11.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:10 smithi153 ceph-mon[35071]: pgmap v501: 97 pgs: 97 active+clean; 28 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.2 MiB/s wr, 68 op/s 2024-09-19T09:08:11.900 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db --set-id 75bdfb5 --release --await' 2024-09-19T09:08:11.900 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db --set-id 75bdfb5 --release --await 2024-09-19T09:08:12.043 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:11 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/199193995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:08:12.050 INFO:tasks.workunit.client.0.smithi120.stdout: 1 375454 6.27 MB/sec execute 127 sec latency 26715.317 ms 2024-09-19T09:08:12.153 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:08:12.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:11 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/199193995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:08:12.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:11 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/199193995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:08:12.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.798+0000 7f7e1dc21640 1 -- 172.21.15.120:0/2924850188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e181082a0 msgr2=0x7f7e1810ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:12.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.798+0000 7f7e1dc21640 1 --2- 172.21.15.120:0/2924850188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e181082a0 0x7f7e1810ec10 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009930 tx=0x7f7e1002a420 comp rx=0 tx=0).stop 2024-09-19T09:08:12.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.798+0000 7f7e1dc21640 1 -- 172.21.15.120:0/2924850188 shutdown_connections 2024-09-19T09:08:12.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.798+0000 7f7e1dc21640 1 --2- 172.21.15.120:0/2924850188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e181082a0 0x7f7e1810ec10 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.798+0000 7f7e1dc21640 1 --2- 172.21.15.120:0/2924850188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e18103720 0x7f7e18107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.798+0000 7f7e1dc21640 1 --2- 172.21.15.120:0/2924850188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e18102d70 0x7f7e18103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.798+0000 7f7e1dc21640 1 -- 172.21.15.120:0/2924850188 >> 172.21.15.120:0/2924850188 conn(0x7f7e180fc7d0 msgr2=0x7f7e180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:12.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.798+0000 7f7e1dc21640 1 -- 172.21.15.120:0/2924850188 shutdown_connections 2024-09-19T09:08:12.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.798+0000 7f7e1dc21640 1 -- 172.21.15.120:0/2924850188 wait complete. 2024-09-19T09:08:12.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.798+0000 7f7e1dc21640 1 Processor -- start 2024-09-19T09:08:12.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.799+0000 7f7e1dc21640 1 -- start start 2024-09-19T09:08:12.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.799+0000 7f7e1dc21640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e18102d70 0x7f7e18198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:12.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.799+0000 7f7e1dc21640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e18103720 0x7f7e18198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:12.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.799+0000 7f7e1dc21640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e181082a0 0x7f7e1819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:12.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.799+0000 7f7e1dc21640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7e18112110 con 0x7f7e18102d70 2024-09-19T09:08:12.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.799+0000 7f7e1dc21640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7e18111f90 con 0x7f7e181082a0 2024-09-19T09:08:12.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.799+0000 7f7e1dc21640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7e18112290 con 0x7f7e18103720 2024-09-19T09:08:12.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.799+0000 7f7e177fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e18102d70 0x7f7e18198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:12.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.799+0000 7f7e177fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e18102d70 0x7f7e18198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49664/0 (socket says 172.21.15.120:49664) 2024-09-19T09:08:12.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.799+0000 7f7e177fe640 1 -- 172.21.15.120:0/2977929010 learned_addr learned my addr 172.21.15.120:0/2977929010 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:12.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.799+0000 7f7e16ffd640 1 --2- 172.21.15.120:0/2977929010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e18103720 0x7f7e18198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:12.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.799+0000 7f7e17fff640 1 --2- 172.21.15.120:0/2977929010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e181082a0 0x7f7e1819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:12.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.799+0000 7f7e177fe640 1 -- 172.21.15.120:0/2977929010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e18103720 msgr2=0x7f7e18198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:12.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.799+0000 7f7e177fe640 1 --2- 172.21.15.120:0/2977929010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e18103720 0x7f7e18198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.799+0000 7f7e177fe640 1 -- 172.21.15.120:0/2977929010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e181082a0 msgr2=0x7f7e1819cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:12.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.799+0000 7f7e177fe640 1 --2- 172.21.15.120:0/2977929010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e181082a0 0x7f7e1819cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.799+0000 7f7e177fe640 1 -- 172.21.15.120:0/2977929010 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e1819d620 con 0x7f7e18102d70 2024-09-19T09:08:12.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.799+0000 7f7e177fe640 1 --2- 172.21.15.120:0/2977929010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e18102d70 0x7f7e18198670 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f7e0800db80 tx=0x7f7e0800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:12.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.800+0000 7f7e14ff9640 1 -- 172.21.15.120:0/2977929010 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e08017070 con 0x7f7e18102d70 2024-09-19T09:08:12.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.800+0000 7f7e14ff9640 1 -- 172.21.15.120:0/2977929010 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e08003b70 con 0x7f7e18102d70 2024-09-19T09:08:12.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.800+0000 7f7e1dc21640 1 -- 172.21.15.120:0/2977929010 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e1819d910 con 0x7f7e18102d70 2024-09-19T09:08:12.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.800+0000 7f7e14ff9640 1 -- 172.21.15.120:0/2977929010 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e080106c0 con 0x7f7e18102d70 2024-09-19T09:08:12.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.800+0000 7f7e1dc21640 1 -- 172.21.15.120:0/2977929010 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e181a51f0 con 0x7f7e18102d70 2024-09-19T09:08:12.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.802+0000 7f7e14ff9640 1 -- 172.21.15.120:0/2977929010 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f7e08004020 con 0x7f7e18102d70 2024-09-19T09:08:12.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.802+0000 7f7e1fffa740 1 Processor -- start 2024-09-19T09:08:12.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.802+0000 7f7e1fffa740 1 -- start start 2024-09-19T09:08:12.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.803+0000 7f7e1fffa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55953c23ef40 0x55953c23f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:12.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.803+0000 7f7e1fffa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55953c23f860 0x55953c24e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:12.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.803+0000 7f7e1fffa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55953c24e650 0x55953c250a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:12.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.803+0000 7f7e1fffa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55953c1a4620 con 0x55953c23ef40 2024-09-19T09:08:12.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.803+0000 7f7e1fffa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55953c14be50 con 0x55953c24e650 2024-09-19T09:08:12.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.803+0000 7f7e1fffa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55953c1a05e0 con 0x55953c23f860 2024-09-19T09:08:12.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.803+0000 7f7e14ff9640 1 --2- 172.21.15.120:0/2977929010 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7e000a3240 0x7f7e000a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:12.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.803+0000 7f7e17fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55953c23ef40 0x55953c23f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:12.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.803+0000 7f7e17fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55953c23ef40 0x55953c23f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49680/0 (socket says 172.21.15.120:49680) 2024-09-19T09:08:12.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.803+0000 7f7e17fff640 1 -- 172.21.15.120:0/3278887710 learned_addr learned my addr 172.21.15.120:0/3278887710 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:12.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.803+0000 7f7e17fff640 1 --2- 172.21.15.120:0/2977929010 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7e000a3240 0x7f7e000a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:12.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.803+0000 7f7e14ff9640 1 -- 172.21.15.120:0/2977929010 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f7e080cf770 con 0x7f7e18102d70 2024-09-19T09:08:12.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.803+0000 7f7e177fe640 1 --2- 172.21.15.120:0/3278887710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55953c23f860 0x55953c24e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:12.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.803+0000 7f7e16ffd640 1 --2- 172.21.15.120:0/3278887710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55953c24e650 0x55953c250a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:12.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.803+0000 7f7e17fff640 1 -- 172.21.15.120:0/3278887710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55953c23f860 msgr2=0x55953c24e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:12.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.803+0000 7f7e17fff640 1 --2- 172.21.15.120:0/3278887710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55953c23f860 0x55953c24e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.803+0000 7f7e17fff640 1 -- 172.21.15.120:0/3278887710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55953c24e650 msgr2=0x55953c250a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:12.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.803+0000 7f7e17fff640 1 --2- 172.21.15.120:0/3278887710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55953c24e650 0x55953c250a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.803+0000 7f7e17fff640 1 -- 172.21.15.120:0/3278887710 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55953c13cd40 con 0x55953c23ef40 2024-09-19T09:08:12.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.803+0000 7f7e16ffd640 1 --2- 172.21.15.120:0/3278887710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55953c24e650 0x55953c250a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:08:12.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.803+0000 7f7e17fff640 1 --2- 172.21.15.120:0/3278887710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55953c23ef40 0x55953c23f320 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f7e10003940 tx=0x7f7e10003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c420bf1d7346b0b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:12.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.803+0000 7f7e17fff640 1 --2- 172.21.15.120:0/2977929010 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7e000a3240 0x7f7e000a5700 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009230 tx=0x7f7e1003f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:12.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.803+0000 7f7dfdffb640 1 -- 172.21.15.120:0/3278887710 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10035040 con 0x55953c23ef40 2024-09-19T09:08:12.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.804+0000 7f7dfdffb640 1 -- 172.21.15.120:0/3278887710 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e1004d070 con 0x55953c23ef40 2024-09-19T09:08:12.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.804+0000 7f7dfdffb640 1 -- 172.21.15.120:0/3278887710 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10048670 con 0x55953c23ef40 2024-09-19T09:08:12.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.804+0000 7f7e1fffa740 1 -- 172.21.15.120:0/3278887710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55953c23ef40 msgr2=0x55953c23f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:12.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.804+0000 7f7e1fffa740 1 --2- 172.21.15.120:0/3278887710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55953c23ef40 0x55953c23f320 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f7e10003940 tx=0x7f7e10003970 comp rx=0 tx=0).stop 2024-09-19T09:08:12.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.804+0000 7f7e1fffa740 1 -- 172.21.15.120:0/3278887710 shutdown_connections 2024-09-19T09:08:12.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.804+0000 7f7e1fffa740 1 --2- 172.21.15.120:0/3278887710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55953c24e650 0x55953c250a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.804+0000 7f7e1fffa740 1 --2- 172.21.15.120:0/3278887710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55953c23f860 0x55953c24e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.804+0000 7f7e1fffa740 1 --2- 172.21.15.120:0/3278887710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55953c23ef40 0x55953c23f320 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.804+0000 7f7e1fffa740 1 -- 172.21.15.120:0/3278887710 >> 172.21.15.120:0/3278887710 conn(0x55953c23b5f0 msgr2=0x55953c23b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:12.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.804+0000 7f7e1fffa740 1 -- 172.21.15.120:0/3278887710 shutdown_connections 2024-09-19T09:08:12.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.804+0000 7f7e1fffa740 1 -- 172.21.15.120:0/3278887710 wait complete. 2024-09-19T09:08:12.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.805+0000 7f7e1fffa740 1 -- start start 2024-09-19T09:08:12.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.805+0000 7f7e1fffa740 1 Processor -- start 2024-09-19T09:08:12.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.805+0000 7f7e1fffa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55953c23ef40 0x55953c24b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:12.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.805+0000 7f7e1fffa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55953c23f860 0x55953c24bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:12.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.805+0000 7f7e1fffa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55953c24e650 0x55953c24c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:12.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.805+0000 7f7e1fffa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55953c251220 con 0x55953c23f860 2024-09-19T09:08:12.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.805+0000 7f7e1fffa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55953c12cf10 con 0x55953c23ef40 2024-09-19T09:08:12.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.805+0000 7f7e1fffa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55953c2513a0 con 0x55953c24e650 2024-09-19T09:08:12.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.805+0000 7f7e1fffa740 10 client.0 _ll_register_callbacks cb 0x55953c150db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:08:12.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.805+0000 7f7e16ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55953c23f860 0x55953c24bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:12.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.805+0000 7f7e16ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55953c23f860 0x55953c24bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49686/0 (socket says 172.21.15.120:49686) 2024-09-19T09:08:12.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.805+0000 7f7e16ffd640 1 -- 172.21.15.120:0/3244938202 learned_addr learned my addr 172.21.15.120:0/3244938202 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:12.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.805+0000 7f7e177fe640 1 --2- 172.21.15.120:0/3244938202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55953c23ef40 0x55953c24b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:12.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.805+0000 7f7e16ffd640 1 -- 172.21.15.120:0/3244938202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55953c24e650 msgr2=0x55953c24c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:12.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.805+0000 7f7e16ffd640 1 --2- 172.21.15.120:0/3244938202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55953c24e650 0x55953c24c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.805+0000 7f7e16ffd640 1 -- 172.21.15.120:0/3244938202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55953c23ef40 msgr2=0x55953c24b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:12.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.805+0000 7f7e16ffd640 1 --2- 172.21.15.120:0/3244938202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55953c23ef40 0x55953c24b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.805+0000 7f7e16ffd640 1 -- 172.21.15.120:0/3244938202 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55953beb3580 con 0x55953c23f860 2024-09-19T09:08:12.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.806+0000 7f7e16ffd640 1 --2- 172.21.15.120:0/3244938202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55953c23f860 0x55953c24bb70 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f7e18069bd0 tx=0x7f7e0c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b2fb60f85f65595 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:12.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.806+0000 7f7df6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:08:12.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.806+0000 7f7df6ffd640 1 -- 172.21.15.120:0/3244938202 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c012920 con 0x55953c23f860 2024-09-19T09:08:12.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.806+0000 7f7df6ffd640 1 -- 172.21.15.120:0/3244938202 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f7e0c012ac0 con 0x55953c23f860 2024-09-19T09:08:12.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.806+0000 7f7e1fffa740 1 -- 172.21.15.120:0/3244938202 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e1004d070 con 0x55953c23f860 2024-09-19T09:08:12.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.806+0000 7f7df6ffd640 1 -- 172.21.15.120:0/3244938202 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7e0c012cc0 con 0x55953c23f860 2024-09-19T09:08:12.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.806+0000 7f7e1fffa740 10 client.17298 fetch_fsmap learned FSMap version 30 2024-09-19T09:08:12.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.806+0000 7f7e1fffa740 1 -- 172.21.15.120:0/3244938202 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55953bf7d960 con 0x55953c23f860 2024-09-19T09:08:12.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.806+0000 7f7df6ffd640 1 -- 172.21.15.120:0/3244938202 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e1004d070 con 0x55953c23f860 2024-09-19T09:08:12.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.806+0000 7f7df6ffd640 1 -- 172.21.15.120:0/3244938202 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7e0c00f040 con 0x55953c23f860 2024-09-19T09:08:12.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.806+0000 7f7e1fffa740 10 client.17298 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:08:12.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.806+0000 7f7e1fffa740 10 client.17298 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:08:12.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.806+0000 7f7e1fffa740 20 client.17298 populate_metadata read hostname 'smithi120' 2024-09-19T09:08:12.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.806+0000 7f7e1fffa740 1 --2- 172.21.15.120:0/3244938202 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55953c25e3e0 0x55953c260820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:12.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.806+0000 7f7e1fffa740 4 client.17298 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:08:12.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.806+0000 7f7e1fffa740 1 -- 172.21.15.120:0/3244938202 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55953c1b1b90 con 0x55953c25e3e0 2024-09-19T09:08:12.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.807+0000 7f7e177fe640 1 --2- 172.21.15.120:0/3244938202 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55953c25e3e0 0x55953c260820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:12.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.807+0000 7f7e177fe640 1 --2- 172.21.15.120:0/3244938202 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55953c25e3e0 0x55953c260820 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77dd0a6dc23d10fe server_cookie=bc3c4a894395fed in_seq=0 out_seq=0 2024-09-19T09:08:12.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.807+0000 7f7df6ffd640 10 client.17298 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:12.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.808+0000 7f7df6ffd640 1 -- 172.21.15.120:0/3244938202 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55953c1b1b90 con 0x55953c25e3e0 2024-09-19T09:08:12.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.808+0000 7f7df6ffd640 10 client.17298 handle_command_reply: tid=0 2024-09-19T09:08:12.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.808+0000 7f7e1fffa740 1 client.17298 shutdown 2024-09-19T09:08:12.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.808+0000 7f7e1fffa740 1 -- 172.21.15.120:0/3244938202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55953c23f860 msgr2=0x55953c24bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:12.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.808+0000 7f7e1fffa740 1 --2- 172.21.15.120:0/3244938202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55953c23f860 0x55953c24bb70 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f7e18069bd0 tx=0x7f7e0c010800 comp rx=0 tx=0).stop 2024-09-19T09:08:12.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.808+0000 7f7e1fffa740 1 -- 172.21.15.120:0/3244938202 shutdown_connections 2024-09-19T09:08:12.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.808+0000 7f7e1fffa740 1 --2- 172.21.15.120:0/3244938202 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55953c25e3e0 0x55953c260820 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.808+0000 7f7e1fffa740 1 --2- 172.21.15.120:0/3244938202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55953c24e650 0x55953c24c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.808+0000 7f7e1fffa740 1 --2- 172.21.15.120:0/3244938202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55953c23f860 0x55953c24bb70 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.808+0000 7f7e1fffa740 1 --2- 172.21.15.120:0/3244938202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55953c23ef40 0x55953c24b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.808+0000 7f7e1fffa740 1 -- 172.21.15.120:0/3244938202 >> 172.21.15.120:0/3244938202 conn(0x55953c23b5f0 msgr2=0x55953c241350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:12.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.808+0000 7f7df6ffd640 0 client.17298 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:12.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.808+0000 7f7e1fffa740 1 -- 172.21.15.120:0/3244938202 shutdown_connections 2024-09-19T09:08:12.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.808+0000 7f7e1fffa740 1 -- 172.21.15.120:0/3244938202 wait complete. 2024-09-19T09:08:12.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.808+0000 7f7e1fffa740 20 client.17298 trim_cache size 0 max 16384 2024-09-19T09:08:12.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.825+0000 7f7e1fffa740 1 Processor -- start 2024-09-19T09:08:12.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.825+0000 7f7e1fffa740 1 -- start start 2024-09-19T09:08:12.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.825+0000 7f7e1fffa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55953c23b5f0 0x55953c24d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:12.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.825+0000 7f7e1fffa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55953c23ef40 0x55953c2660f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:12.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.825+0000 7f7e1fffa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55953c23f860 0x55953c266630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:12.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.825+0000 7f7e1fffa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7e181ab340 con 0x55953c23ef40 2024-09-19T09:08:12.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.825+0000 7f7e1fffa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7e181ace80 con 0x55953c23b5f0 2024-09-19T09:08:12.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.825+0000 7f7e1fffa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7e181aba10 con 0x55953c23f860 2024-09-19T09:08:12.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.826+0000 7f7e16ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55953c23ef40 0x55953c2660f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:12.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.826+0000 7f7e16ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55953c23ef40 0x55953c2660f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49696/0 (socket says 172.21.15.120:49696) 2024-09-19T09:08:12.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.826+0000 7f7e16ffd640 1 -- 172.21.15.120:0/625136744 learned_addr learned my addr 172.21.15.120:0/625136744 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:12.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.826+0000 7f7e177fe640 1 --2- 172.21.15.120:0/625136744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55953c23b5f0 0x55953c24d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:12.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.826+0000 7f7e17fff640 1 --2- 172.21.15.120:0/625136744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55953c23f860 0x55953c266630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:12.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.826+0000 7f7e16ffd640 1 -- 172.21.15.120:0/625136744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55953c23f860 msgr2=0x55953c266630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:12.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.826+0000 7f7e16ffd640 1 --2- 172.21.15.120:0/625136744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55953c23f860 0x55953c266630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.826+0000 7f7e16ffd640 1 -- 172.21.15.120:0/625136744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55953c23b5f0 msgr2=0x55953c24d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:12.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.826+0000 7f7e16ffd640 1 --2- 172.21.15.120:0/625136744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55953c23b5f0 0x55953c24d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.826+0000 7f7e16ffd640 1 -- 172.21.15.120:0/625136744 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55953c246420 con 0x55953c23ef40 2024-09-19T09:08:12.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.826+0000 7f7e16ffd640 1 --2- 172.21.15.120:0/625136744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55953c23ef40 0x55953c2660f0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c000c00 tx=0x7f7e0c013f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b69aacf7eff767 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:12.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.826+0000 7f7dfd7fa640 1 -- 172.21.15.120:0/625136744 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c012920 con 0x55953c23ef40 2024-09-19T09:08:12.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.826+0000 7f7dfd7fa640 1 -- 172.21.15.120:0/625136744 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e0c010180 con 0x55953c23ef40 2024-09-19T09:08:12.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.826+0000 7f7dfd7fa640 1 -- 172.21.15.120:0/625136744 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c027d50 con 0x55953c23ef40 2024-09-19T09:08:12.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.827+0000 7f7e1fffa740 1 -- 172.21.15.120:0/625136744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55953c23ef40 msgr2=0x55953c2660f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:12.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.827+0000 7f7e1fffa740 1 --2- 172.21.15.120:0/625136744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55953c23ef40 0x55953c2660f0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c000c00 tx=0x7f7e0c013f70 comp rx=0 tx=0).stop 2024-09-19T09:08:12.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.827+0000 7f7e1fffa740 1 -- 172.21.15.120:0/625136744 shutdown_connections 2024-09-19T09:08:12.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.827+0000 7f7e1fffa740 1 --2- 172.21.15.120:0/625136744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55953c23f860 0x55953c266630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.827+0000 7f7e1fffa740 1 --2- 172.21.15.120:0/625136744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55953c23ef40 0x55953c2660f0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.827+0000 7f7e1fffa740 1 --2- 172.21.15.120:0/625136744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55953c23b5f0 0x55953c24d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.827+0000 7f7e1fffa740 1 -- 172.21.15.120:0/625136744 >> 172.21.15.120:0/625136744 conn(0x55953c24c210 msgr2=0x55953c24ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:12.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.827+0000 7f7e1fffa740 1 -- 172.21.15.120:0/625136744 shutdown_connections 2024-09-19T09:08:12.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.827+0000 7f7e1fffa740 1 -- 172.21.15.120:0/625136744 wait complete. 2024-09-19T09:08:12.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.827+0000 7f7e1fffa740 1 -- start start 2024-09-19T09:08:12.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.828+0000 7f7e1fffa740 1 Processor -- start 2024-09-19T09:08:12.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.828+0000 7f7e1fffa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55953c23b5f0 0x55953c252e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:12.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.828+0000 7f7e1fffa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55953c23ef40 0x55953c2533c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:12.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.828+0000 7f7e1fffa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55953c23f860 0x55953c256480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:12.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.828+0000 7f7e1fffa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7e181ac7b0 con 0x55953c23ef40 2024-09-19T09:08:12.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.828+0000 7f7e1fffa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7e181ac0e0 con 0x55953c23b5f0 2024-09-19T09:08:12.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.828+0000 7f7e1fffa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7e181a9320 con 0x55953c23f860 2024-09-19T09:08:12.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.828+0000 7f7e1fffa740 10 client.0 _ll_register_callbacks cb 0x7f7e0c00eb70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:08:12.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.828+0000 7f7e16ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55953c23ef40 0x55953c2533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:12.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.828+0000 7f7e16ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55953c23ef40 0x55953c2533c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49708/0 (socket says 172.21.15.120:49708) 2024-09-19T09:08:12.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.828+0000 7f7e16ffd640 1 -- 172.21.15.120:0/2384713352 learned_addr learned my addr 172.21.15.120:0/2384713352 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:12.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.828+0000 7f7e177fe640 1 --2- 172.21.15.120:0/2384713352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55953c23b5f0 0x55953c252e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:12.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.828+0000 7f7e17fff640 1 --2- 172.21.15.120:0/2384713352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55953c23f860 0x55953c256480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:12.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.828+0000 7f7e16ffd640 1 -- 172.21.15.120:0/2384713352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55953c23f860 msgr2=0x55953c256480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:12.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.828+0000 7f7e16ffd640 1 --2- 172.21.15.120:0/2384713352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55953c23f860 0x55953c256480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.828+0000 7f7e16ffd640 1 -- 172.21.15.120:0/2384713352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55953c23b5f0 msgr2=0x55953c252e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:12.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.828+0000 7f7e16ffd640 1 --2- 172.21.15.120:0/2384713352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55953c23b5f0 0x55953c252e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.828+0000 7f7e16ffd640 1 -- 172.21.15.120:0/2384713352 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55953c256b20 con 0x55953c23ef40 2024-09-19T09:08:12.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.829+0000 7f7e16ffd640 1 --2- 172.21.15.120:0/2384713352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55953c23ef40 0x55953c2533c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c0278d0 tx=0x7f7e0c012950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e03d5e74391bc45 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:12.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.829+0000 7f7df6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:08:12.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.829+0000 7f7df6ffd640 1 -- 172.21.15.120:0/2384713352 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c005450 con 0x55953c23ef40 2024-09-19T09:08:12.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.829+0000 7f7df6ffd640 1 -- 172.21.15.120:0/2384713352 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f7e0c0035f0 con 0x55953c23ef40 2024-09-19T09:08:12.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.829+0000 7f7e1fffa740 1 -- 172.21.15.120:0/2384713352 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e0c010180 con 0x55953c23ef40 2024-09-19T09:08:12.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.829+0000 7f7df6ffd640 1 -- 172.21.15.120:0/2384713352 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7e0c041050 con 0x55953c23ef40 2024-09-19T09:08:12.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.829+0000 7f7e1fffa740 10 client.17304 fetch_fsmap learned FSMap version 30 2024-09-19T09:08:12.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.829+0000 7f7e1fffa740 1 -- 172.21.15.120:0/2384713352 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55953c2c9cd0 con 0x55953c23ef40 2024-09-19T09:08:12.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.829+0000 7f7df6ffd640 1 -- 172.21.15.120:0/2384713352 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e0c010180 con 0x55953c23ef40 2024-09-19T09:08:12.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.829+0000 7f7df6ffd640 1 -- 172.21.15.120:0/2384713352 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7e0c00e030 con 0x55953c23ef40 2024-09-19T09:08:12.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.829+0000 7f7e1fffa740 10 client.17304 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:08:12.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.829+0000 7f7e1fffa740 10 client.17304 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:08:12.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.829+0000 7f7e1fffa740 20 client.17304 populate_metadata read hostname 'smithi120' 2024-09-19T09:08:12.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.829+0000 7f7e1fffa740 1 --2- 172.21.15.120:0/2384713352 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55953c24e650 0x55953c2cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:12.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.829+0000 7f7e1fffa740 4 client.17304 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "75bdfb5", "release": true, "await": true}] 2024-09-19T09:08:12.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.829+0000 7f7e1fffa740 1 -- 172.21.15.120:0/2384713352 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "set_id": "75bdfb5", "release": true, "await": true}) -- 0x55953c1ac0a0 con 0x55953c24e650 2024-09-19T09:08:12.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.830+0000 7f7e177fe640 1 --2- 172.21.15.120:0/2384713352 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55953c24e650 0x55953c2cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:12.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.830+0000 7f7e177fe640 1 --2- 172.21.15.120:0/2384713352 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55953c24e650 0x55953c2cc3d0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5afcdee5b3507f0 server_cookie=bec5f48071593f85 in_seq=0 out_seq=0 2024-09-19T09:08:12.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.830+0000 7f7df6ffd640 10 client.17304 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:12.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.841+0000 7f7df6ffd640 1 -- 172.21.15.120:0/2384713352 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55953c1ac0a0 con 0x55953c24e650 2024-09-19T09:08:12.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.841+0000 7f7df6ffd640 10 client.17304 handle_command_reply: tid=0 2024-09-19T09:08:12.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.841+0000 7f7e1fffa740 1 client.17304 shutdown 2024-09-19T09:08:12.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.841+0000 7f7e1fffa740 1 -- 172.21.15.120:0/2384713352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55953c23ef40 msgr2=0x55953c2533c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:12.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.841+0000 7f7e1fffa740 1 --2- 172.21.15.120:0/2384713352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55953c23ef40 0x55953c2533c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c0278d0 tx=0x7f7e0c012950 comp rx=0 tx=0).stop 2024-09-19T09:08:12.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.841+0000 7f7e1fffa740 1 -- 172.21.15.120:0/2384713352 shutdown_connections 2024-09-19T09:08:12.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.841+0000 7f7e1fffa740 1 --2- 172.21.15.120:0/2384713352 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55953c24e650 0x55953c2cc3d0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.841+0000 7f7e1fffa740 1 --2- 172.21.15.120:0/2384713352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55953c23f860 0x55953c256480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.841+0000 7f7e1fffa740 1 --2- 172.21.15.120:0/2384713352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55953c23ef40 0x55953c2533c0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.841+0000 7f7df6ffd640 0 client.17304 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:12.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.841+0000 7f7e1fffa740 1 --2- 172.21.15.120:0/2384713352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55953c23b5f0 0x55953c252e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.841+0000 7f7e1fffa740 1 -- 172.21.15.120:0/2384713352 >> 172.21.15.120:0/2384713352 conn(0x55953c24c210 msgr2=0x55953c23a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:12.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.841+0000 7f7e1fffa740 1 -- 172.21.15.120:0/2384713352 shutdown_connections 2024-09-19T09:08:12.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.841+0000 7f7e1fffa740 1 -- 172.21.15.120:0/2384713352 wait complete. 2024-09-19T09:08:12.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.842+0000 7f7e1fffa740 20 client.17304 trim_cache size 0 max 16384 2024-09-19T09:08:12.845 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:08:12.845 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 30, 2024-09-19T09:08:12.845 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:08:12.845 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 24, 2024-09-19T09:08:12.845 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:08:12.845 INFO:teuthology.orchestra.run.smithi120.stdout: "75bdfb5": { 2024-09-19T09:08:12.845 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 24, 2024-09-19T09:08:12.845 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:08:12.845 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:08:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:08:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:08:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:08:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:08:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 149.8, 2024-09-19T09:08:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:08:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:08:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:08:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:08:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:08:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:08:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:08:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:08:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:08:12.847 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:08:12.847 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:08:12.847 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:08:12.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.842+0000 7f7e1dc21640 1 -- 172.21.15.120:0/2977929010 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7e000a3240 msgr2=0x7f7e000a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:12.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.842+0000 7f7e1dc21640 1 --2- 172.21.15.120:0/2977929010 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7e000a3240 0x7f7e000a5700 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009230 tx=0x7f7e1003f000 comp rx=0 tx=0).stop 2024-09-19T09:08:12.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.842+0000 7f7e1dc21640 1 -- 172.21.15.120:0/2977929010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e18102d70 msgr2=0x7f7e18198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:12.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.842+0000 7f7e1dc21640 1 --2- 172.21.15.120:0/2977929010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e18102d70 0x7f7e18198670 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f7e0800db80 tx=0x7f7e0800b630 comp rx=0 tx=0).stop 2024-09-19T09:08:12.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.842+0000 7f7e1dc21640 1 -- 172.21.15.120:0/2977929010 shutdown_connections 2024-09-19T09:08:12.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.842+0000 7f7e1dc21640 1 --2- 172.21.15.120:0/2977929010 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7e000a3240 0x7f7e000a5700 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.842+0000 7f7e1dc21640 1 --2- 172.21.15.120:0/2977929010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e181082a0 0x7f7e1819cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.842+0000 7f7e1dc21640 1 --2- 172.21.15.120:0/2977929010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e18103720 0x7f7e18198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.842+0000 7f7e1dc21640 1 --2- 172.21.15.120:0/2977929010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e18102d70 0x7f7e18198670 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:12.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.842+0000 7f7e1dc21640 1 -- 172.21.15.120:0/2977929010 >> 172.21.15.120:0/2977929010 conn(0x7f7e180fc7d0 msgr2=0x7f7e18106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:12.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.842+0000 7f7e1dc21640 1 -- 172.21.15.120:0/2977929010 shutdown_connections 2024-09-19T09:08:12.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:12.842+0000 7f7e1dc21640 1 -- 172.21.15.120:0/2977929010 wait complete. 2024-09-19T09:08:13.050 INFO:tasks.workunit.client.0.smithi120.stdout: 1 376121 6.27 MB/sec execute 128 sec latency 27510.068 ms 2024-09-19T09:08:13.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:12 smithi120 ceph-mon[31150]: pgmap v502: 97 pgs: 97 active+clean; 24 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 1.2 MiB/s wr, 60 op/s 2024-09-19T09:08:13.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:12 smithi196 ceph-mon[35150]: pgmap v502: 97 pgs: 97 active+clean; 24 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 1.2 MiB/s wr, 60 op/s 2024-09-19T09:08:13.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:12 smithi153 ceph-mon[35071]: pgmap v502: 97 pgs: 97 active+clean; 24 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 1.2 MiB/s wr, 60 op/s 2024-09-19T09:08:13.690 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '75bdfb5', total seconds elapsed: 33.3 2024-09-19T09:08:14.050 INFO:tasks.workunit.client.0.smithi120.stdout: 1 379035 6.33 MB/sec execute 129 sec latency 13.949 ms 2024-09-19T09:08:15.050 INFO:tasks.workunit.client.0.smithi120.stdout: 1 381999 6.38 MB/sec execute 130 sec latency 10.439 ms 2024-09-19T09:08:15.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:14 smithi120 ceph-mon[31150]: pgmap v503: 97 pgs: 97 active+clean; 24 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 1.2 MiB/s wr, 59 op/s 2024-09-19T09:08:15.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:14 smithi196 ceph-mon[35150]: pgmap v503: 97 pgs: 97 active+clean; 24 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 1.2 MiB/s wr, 59 op/s 2024-09-19T09:08:15.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:14 smithi153 ceph-mon[35071]: pgmap v503: 97 pgs: 97 active+clean; 24 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 1.2 MiB/s wr, 59 op/s 2024-09-19T09:08:16.050 INFO:tasks.workunit.client.0.smithi120.stdout: 1 384727 6.46 MB/sec execute 131 sec latency 11.239 ms 2024-09-19T09:08:16.507 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:08:16.769 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:08:17.050 INFO:tasks.workunit.client.0.smithi120.stdout: 1 387550 6.53 MB/sec execute 132 sec latency 15.989 ms 2024-09-19T09:08:17.066 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:16 smithi120 ceph-mon[31150]: pgmap v504: 97 pgs: 97 active+clean; 27 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 524 KiB/s wr, 78 op/s 2024-09-19T09:08:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:16 smithi196 ceph-mon[35150]: pgmap v504: 97 pgs: 97 active+clean; 27 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 524 KiB/s wr, 78 op/s 2024-09-19T09:08:17.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:16 smithi153 ceph-mon[35071]: pgmap v504: 97 pgs: 97 active+clean; 27 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 524 KiB/s wr, 78 op/s 2024-09-19T09:08:17.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.325+0000 7f5b778fd640 1 -- 172.21.15.120:0/1801003969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b70103780 msgr2=0x7f5b70107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:17.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.325+0000 7f5b778fd640 1 --2- 172.21.15.120:0/1801003969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b70103780 0x7f5b70107bc0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009930 tx=0x7f5b6002a420 comp rx=0 tx=0).stop 2024-09-19T09:08:17.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.325+0000 7f5b778fd640 1 -- 172.21.15.120:0/1801003969 shutdown_connections 2024-09-19T09:08:17.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.325+0000 7f5b778fd640 1 --2- 172.21.15.120:0/1801003969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b70108300 0x7f5b7010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.325+0000 7f5b778fd640 1 --2- 172.21.15.120:0/1801003969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b70103780 0x7f5b70107bc0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.325+0000 7f5b778fd640 1 --2- 172.21.15.120:0/1801003969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b70102dd0 0x7f5b701031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.325+0000 7f5b778fd640 1 -- 172.21.15.120:0/1801003969 >> 172.21.15.120:0/1801003969 conn(0x7f5b700fc7d0 msgr2=0x7f5b700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:17.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.325+0000 7f5b778fd640 1 -- 172.21.15.120:0/1801003969 shutdown_connections 2024-09-19T09:08:17.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.325+0000 7f5b778fd640 1 -- 172.21.15.120:0/1801003969 wait complete. 2024-09-19T09:08:17.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.325+0000 7f5b778fd640 1 Processor -- start 2024-09-19T09:08:17.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.325+0000 7f5b778fd640 1 -- start start 2024-09-19T09:08:17.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.326+0000 7f5b778fd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b70102dd0 0x7f5b70198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:17.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.326+0000 7f5b778fd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b70103780 0x7f5b70198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:17.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.326+0000 7f5b778fd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b70108300 0x7f5b7019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:17.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.326+0000 7f5b778fd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701120a0 con 0x7f5b70103780 2024-09-19T09:08:17.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.326+0000 7f5b778fd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5b70111f20 con 0x7f5b70102dd0 2024-09-19T09:08:17.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.326+0000 7f5b778fd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5b70112220 con 0x7f5b70108300 2024-09-19T09:08:17.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.326+0000 7f5b74e98640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b70103780 0x7f5b70198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:17.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.326+0000 7f5b75699640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b70102dd0 0x7f5b70198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:17.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.326+0000 7f5b74e98640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b70103780 0x7f5b70198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49724/0 (socket says 172.21.15.120:49724) 2024-09-19T09:08:17.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.326+0000 7f5b74e98640 1 -- 172.21.15.120:0/961878211 learned_addr learned my addr 172.21.15.120:0/961878211 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:17.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.326+0000 7f5b75e9a640 1 --2- 172.21.15.120:0/961878211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b70108300 0x7f5b7019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:17.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.326+0000 7f5b75699640 1 -- 172.21.15.120:0/961878211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b70108300 msgr2=0x7f5b7019cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:17.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.326+0000 7f5b75699640 1 --2- 172.21.15.120:0/961878211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b70108300 0x7f5b7019cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.326+0000 7f5b75699640 1 -- 172.21.15.120:0/961878211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b70103780 msgr2=0x7f5b70198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:17.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.326+0000 7f5b75699640 1 --2- 172.21.15.120:0/961878211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b70103780 0x7f5b70198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.326+0000 7f5b75699640 1 -- 172.21.15.120:0/961878211 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b7019d4d0 con 0x7f5b70102dd0 2024-09-19T09:08:17.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.326+0000 7f5b74e98640 1 --2- 172.21.15.120:0/961878211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b70103780 0x7f5b70198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:17.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.326+0000 7f5b75699640 1 --2- 172.21.15.120:0/961878211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b70102dd0 0x7f5b70198520 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00c970 tx=0x7f5b6c00ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:17.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.326+0000 7f5b667fc640 1 -- 172.21.15.120:0/961878211 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c007bb0 con 0x7f5b70102dd0 2024-09-19T09:08:17.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.326+0000 7f5b778fd640 1 -- 172.21.15.120:0/961878211 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b7019d7c0 con 0x7f5b70102dd0 2024-09-19T09:08:17.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.326+0000 7f5b667fc640 1 -- 172.21.15.120:0/961878211 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b6c007d50 con 0x7f5b70102dd0 2024-09-19T09:08:17.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.326+0000 7f5b667fc640 1 -- 172.21.15.120:0/961878211 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c018820 con 0x7f5b70102dd0 2024-09-19T09:08:17.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.327+0000 7f5b778fd640 1 -- 172.21.15.120:0/961878211 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b701a50a0 con 0x7f5b70102dd0 2024-09-19T09:08:17.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.329+0000 7f5b79cd6740 1 Processor -- start 2024-09-19T09:08:17.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.329+0000 7f5b79cd6740 1 -- start start 2024-09-19T09:08:17.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.329+0000 7f5b667fc640 1 -- 172.21.15.120:0/961878211 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5b6c0189c0 con 0x7f5b70102dd0 2024-09-19T09:08:17.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.329+0000 7f5b79cd6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb16cfc4a0 0x55cb16cfc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:17.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.329+0000 7f5b79cd6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb16cfcdc0 0x55cb16d07140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:17.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.329+0000 7f5b79cd6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb16d07680 0x55cb16d09a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:17.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.329+0000 7f5b79cd6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cb16c622e0 con 0x55cb16cfcdc0 2024-09-19T09:08:17.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.329+0000 7f5b79cd6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cb16c09ce0 con 0x55cb16cfc4a0 2024-09-19T09:08:17.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.329+0000 7f5b79cd6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cb16c5e2a0 con 0x55cb16d07680 2024-09-19T09:08:17.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.329+0000 7f5b75e9a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb16cfc4a0 0x55cb16cfc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:17.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.330+0000 7f5b75699640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb16cfcdc0 0x55cb16d07140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:17.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.330+0000 7f5b74e98640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb16d07680 0x55cb16d09a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:17.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.330+0000 7f5b75e9a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb16cfc4a0 0x55cb16cfc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:53686/0 (socket says 172.21.15.120:53686) 2024-09-19T09:08:17.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.330+0000 7f5b75e9a640 1 -- 172.21.15.120:0/367814121 learned_addr learned my addr 172.21.15.120:0/367814121 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:17.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.330+0000 7f5b75e9a640 1 -- 172.21.15.120:0/367814121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb16d07680 msgr2=0x55cb16d09a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:17.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.330+0000 7f5b75e9a640 1 --2- 172.21.15.120:0/367814121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb16d07680 0x55cb16d09a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.330+0000 7f5b75e9a640 1 -- 172.21.15.120:0/367814121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb16cfcdc0 msgr2=0x55cb16d07140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:17.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.330+0000 7f5b75e9a640 1 --2- 172.21.15.120:0/367814121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb16cfcdc0 0x55cb16d07140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.330+0000 7f5b75e9a640 1 -- 172.21.15.120:0/367814121 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb16bee440 con 0x55cb16cfc4a0 2024-09-19T09:08:17.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.330+0000 7f5b74e98640 1 --2- 172.21.15.120:0/367814121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb16d07680 0x55cb16d09a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:17.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.330+0000 7f5b667fc640 1 --2- 172.21.15.120:0/961878211 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5b5c0a3240 0x7f5b5c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:17.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.330+0000 7f5b75e9a640 1 --2- 172.21.15.120:0/367814121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb16cfc4a0 0x55cb16cfc880 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009f70 tx=0x7f5b68010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19d0af75c01c763e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:17.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.330+0000 7f5b75699640 1 --2- 172.21.15.120:0/961878211 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5b5c0a3240 0x7f5b5c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:17.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.330+0000 7f5b667fc640 1 -- 172.21.15.120:0/961878211 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f5b6c0cf8f0 con 0x7f5b70102dd0 2024-09-19T09:08:17.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.330+0000 7f5b5b7fe640 1 -- 172.21.15.120:0/367814121 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68016070 con 0x55cb16cfc4a0 2024-09-19T09:08:17.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.330+0000 7f5b5b7fe640 1 -- 172.21.15.120:0/367814121 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b68004d20 con 0x55cb16cfc4a0 2024-09-19T09:08:17.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.330+0000 7f5b5b7fe640 1 -- 172.21.15.120:0/367814121 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68012950 con 0x55cb16cfc4a0 2024-09-19T09:08:17.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.331+0000 7f5b79cd6740 1 -- 172.21.15.120:0/367814121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb16cfc4a0 msgr2=0x55cb16cfc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:17.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.331+0000 7f5b75699640 1 --2- 172.21.15.120:0/961878211 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5b5c0a3240 0x7f5b5c0a5700 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c0a4650 tx=0x7f5b6c0d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:17.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.331+0000 7f5b79cd6740 1 --2- 172.21.15.120:0/367814121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb16cfc4a0 0x55cb16cfc880 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009f70 tx=0x7f5b68010800 comp rx=0 tx=0).stop 2024-09-19T09:08:17.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.331+0000 7f5b79cd6740 1 -- 172.21.15.120:0/367814121 shutdown_connections 2024-09-19T09:08:17.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.331+0000 7f5b79cd6740 1 --2- 172.21.15.120:0/367814121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb16d07680 0x55cb16d09a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.331+0000 7f5b79cd6740 1 --2- 172.21.15.120:0/367814121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb16cfcdc0 0x55cb16d07140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.331+0000 7f5b79cd6740 1 --2- 172.21.15.120:0/367814121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb16cfc4a0 0x55cb16cfc880 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.331+0000 7f5b79cd6740 1 -- 172.21.15.120:0/367814121 >> 172.21.15.120:0/367814121 conn(0x55cb16cf8a30 msgr2=0x55cb16cf8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:17.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.331+0000 7f5b79cd6740 1 -- 172.21.15.120:0/367814121 shutdown_connections 2024-09-19T09:08:17.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.331+0000 7f5b79cd6740 1 -- 172.21.15.120:0/367814121 wait complete. 2024-09-19T09:08:17.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.332+0000 7f5b79cd6740 1 -- start start 2024-09-19T09:08:17.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.332+0000 7f5b79cd6740 1 Processor -- start 2024-09-19T09:08:17.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.332+0000 7f5b79cd6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb16cfc4a0 0x55cb16d07070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:17.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.332+0000 7f5b79cd6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb16cfcdc0 0x55cb16d04320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:17.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.332+0000 7f5b79cd6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb16d07680 0x55cb16d04910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:17.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.332+0000 7f5b79cd6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cb16d0a250 con 0x55cb16cfcdc0 2024-09-19T09:08:17.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.332+0000 7f5b79cd6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cb16c1d050 con 0x55cb16cfc4a0 2024-09-19T09:08:17.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.332+0000 7f5b79cd6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cb16d0a3d0 con 0x55cb16d07680 2024-09-19T09:08:17.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.332+0000 7f5b79cd6740 10 client.0 _ll_register_callbacks cb 0x55cb16c5d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:08:17.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.332+0000 7f5b74e98640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb16cfcdc0 0x55cb16d04320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:17.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.332+0000 7f5b74e98640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb16cfcdc0 0x55cb16d04320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49756/0 (socket says 172.21.15.120:49756) 2024-09-19T09:08:17.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.332+0000 7f5b74e98640 1 -- 172.21.15.120:0/1348751434 learned_addr learned my addr 172.21.15.120:0/1348751434 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:17.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.332+0000 7f5b75e9a640 1 --2- 172.21.15.120:0/1348751434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb16d07680 0x55cb16d04910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:17.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.332+0000 7f5b75e9a640 1 --2- 172.21.15.120:0/1348751434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb16cfc4a0 0x55cb16d07070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:17.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.332+0000 7f5b74e98640 1 -- 172.21.15.120:0/1348751434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb16d07680 msgr2=0x55cb16d04910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:17.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.332+0000 7f5b74e98640 1 --2- 172.21.15.120:0/1348751434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb16d07680 0x55cb16d04910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.332+0000 7f5b74e98640 1 -- 172.21.15.120:0/1348751434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb16cfc4a0 msgr2=0x55cb16d07070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:17.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.332+0000 7f5b74e98640 1 --2- 172.21.15.120:0/1348751434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb16cfc4a0 0x55cb16d07070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.332+0000 7f5b74e98640 1 -- 172.21.15.120:0/1348751434 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb169d33c0 con 0x55cb16cfcdc0 2024-09-19T09:08:17.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.333+0000 7f5b74e98640 1 --2- 172.21.15.120:0/1348751434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb16cfcdc0 0x55cb16d04320 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009230 tx=0x7f5b60005440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec7e7bab15f14e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:17.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.333+0000 7f5b58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:08:17.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.333+0000 7f5b58ff9640 1 -- 172.21.15.120:0/1348751434 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60035040 con 0x55cb16cfcdc0 2024-09-19T09:08:17.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.333+0000 7f5b58ff9640 1 -- 172.21.15.120:0/1348751434 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f5b60004b70 con 0x55cb16cfcdc0 2024-09-19T09:08:17.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.333+0000 7f5b79cd6740 1 -- 172.21.15.120:0/1348751434 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b68004d20 con 0x55cb16cfcdc0 2024-09-19T09:08:17.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.333+0000 7f5b58ff9640 1 -- 172.21.15.120:0/1348751434 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5b60004d70 con 0x55cb16cfcdc0 2024-09-19T09:08:17.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.333+0000 7f5b79cd6740 10 client.17316 fetch_fsmap learned FSMap version 30 2024-09-19T09:08:17.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.333+0000 7f5b79cd6740 1 -- 172.21.15.120:0/1348751434 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cb16d15cb0 con 0x55cb16cfcdc0 2024-09-19T09:08:17.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.333+0000 7f5b58ff9640 1 -- 172.21.15.120:0/1348751434 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b68004d20 con 0x55cb16cfcdc0 2024-09-19T09:08:17.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.334+0000 7f5b58ff9640 1 -- 172.21.15.120:0/1348751434 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5b60044020 con 0x55cb16cfcdc0 2024-09-19T09:08:17.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.334+0000 7f5b79cd6740 10 client.17316 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:08:17.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.334+0000 7f5b79cd6740 10 client.17316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:08:17.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.334+0000 7f5b79cd6740 20 client.17316 populate_metadata read hostname 'smithi120' 2024-09-19T09:08:17.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.334+0000 7f5b79cd6740 1 --2- 172.21.15.120:0/1348751434 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cb16d171c0 0x55cb16d195e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:17.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.334+0000 7f5b79cd6740 4 client.17316 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:08:17.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.334+0000 7f5b79cd6740 1 -- 172.21.15.120:0/1348751434 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb16c6f850 con 0x55cb16d171c0 2024-09-19T09:08:17.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.334+0000 7f5b75e9a640 1 --2- 172.21.15.120:0/1348751434 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cb16d171c0 0x55cb16d195e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:17.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.334+0000 7f5b58ff9640 10 client.17316 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:17.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.334+0000 7f5b75e9a640 1 --2- 172.21.15.120:0/1348751434 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cb16d171c0 0x55cb16d195e0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8e205d69a5644ee server_cookie=4750a5f6623e2adb in_seq=0 out_seq=0 2024-09-19T09:08:17.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.335+0000 7f5b58ff9640 1 -- 172.21.15.120:0/1348751434 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cb16c6f850 con 0x55cb16d171c0 2024-09-19T09:08:17.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.335+0000 7f5b58ff9640 10 client.17316 handle_command_reply: tid=0 2024-09-19T09:08:17.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.335+0000 7f5b79cd6740 1 client.17316 shutdown 2024-09-19T09:08:17.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.335+0000 7f5b79cd6740 1 -- 172.21.15.120:0/1348751434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb16cfcdc0 msgr2=0x55cb16d04320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:17.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.335+0000 7f5b79cd6740 1 --2- 172.21.15.120:0/1348751434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb16cfcdc0 0x55cb16d04320 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009230 tx=0x7f5b60005440 comp rx=0 tx=0).stop 2024-09-19T09:08:17.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.335+0000 7f5b79cd6740 1 -- 172.21.15.120:0/1348751434 shutdown_connections 2024-09-19T09:08:17.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.335+0000 7f5b79cd6740 1 --2- 172.21.15.120:0/1348751434 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cb16d171c0 0x55cb16d195e0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.335+0000 7f5b79cd6740 1 --2- 172.21.15.120:0/1348751434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb16d07680 0x55cb16d04910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.335+0000 7f5b58ff9640 0 client.17316 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:17.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.335+0000 7f5b79cd6740 1 --2- 172.21.15.120:0/1348751434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb16cfcdc0 0x55cb16d04320 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.335+0000 7f5b79cd6740 1 --2- 172.21.15.120:0/1348751434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb16cfc4a0 0x55cb16d07070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.335+0000 7f5b79cd6740 1 -- 172.21.15.120:0/1348751434 >> 172.21.15.120:0/1348751434 conn(0x55cb16cf8a30 msgr2=0x55cb16cfe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:17.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.335+0000 7f5b79cd6740 1 -- 172.21.15.120:0/1348751434 shutdown_connections 2024-09-19T09:08:17.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.335+0000 7f5b79cd6740 1 -- 172.21.15.120:0/1348751434 wait complete. 2024-09-19T09:08:17.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.335+0000 7f5b79cd6740 20 client.17316 trim_cache size 0 max 16384 2024-09-19T09:08:17.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.355+0000 7f5b79cd6740 1 Processor -- start 2024-09-19T09:08:17.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.355+0000 7f5b79cd6740 1 -- start start 2024-09-19T09:08:17.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.355+0000 7f5b79cd6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb16cfc4a0 0x55cb16d134b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:17.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.355+0000 7f5b79cd6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb16cfcdc0 0x55cb16d1eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:17.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.355+0000 7f5b79cd6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb16d07680 0x55cb16d1f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:17.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.355+0000 7f5b79cd6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701ad660 con 0x55cb16cfc4a0 2024-09-19T09:08:17.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.355+0000 7f5b79cd6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701af1a0 con 0x55cb16d07680 2024-09-19T09:08:17.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.355+0000 7f5b79cd6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701add30 con 0x55cb16cfcdc0 2024-09-19T09:08:17.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.355+0000 7f5b75e9a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb16cfc4a0 0x55cb16d134b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:17.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.355+0000 7f5b74e98640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb16cfcdc0 0x55cb16d1eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:17.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.355+0000 7f5b75e9a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb16cfc4a0 0x55cb16d134b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49770/0 (socket says 172.21.15.120:49770) 2024-09-19T09:08:17.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.355+0000 7f5b75e9a640 1 -- 172.21.15.120:0/1201186054 learned_addr learned my addr 172.21.15.120:0/1201186054 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:17.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.355+0000 7f5b75e9a640 1 --2- 172.21.15.120:0/1201186054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb16d07680 0x55cb16d1f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:17.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.355+0000 7f5b74e98640 1 -- 172.21.15.120:0/1201186054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb16d07680 msgr2=0x55cb16d1f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:17.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.355+0000 7f5b74e98640 1 --2- 172.21.15.120:0/1201186054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb16d07680 0x55cb16d1f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.355+0000 7f5b74e98640 1 -- 172.21.15.120:0/1201186054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb16cfc4a0 msgr2=0x55cb16d134b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:17.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.355+0000 7f5b74e98640 1 --2- 172.21.15.120:0/1201186054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb16cfc4a0 0x55cb16d134b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.355+0000 7f5b74e98640 1 -- 172.21.15.120:0/1201186054 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb16cf96c0 con 0x55cb16cfcdc0 2024-09-19T09:08:17.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.355+0000 7f5b75e9a640 1 --2- 172.21.15.120:0/1201186054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb16d07680 0x55cb16d1f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:08:17.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.356+0000 7f5b74e98640 1 --2- 172.21.15.120:0/1201186054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb16cfcdc0 0x55cb16d1eeb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x55cb16d058d0 tx=0x7f5b60009830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36b2cbb74dee84f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:17.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.356+0000 7f5b5affd640 1 -- 172.21.15.120:0/1201186054 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60054030 con 0x55cb16cfcdc0 2024-09-19T09:08:17.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.356+0000 7f5b5affd640 1 -- 172.21.15.120:0/1201186054 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b60005850 con 0x55cb16cfcdc0 2024-09-19T09:08:17.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.356+0000 7f5b5affd640 1 -- 172.21.15.120:0/1201186054 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60035030 con 0x55cb16cfcdc0 2024-09-19T09:08:17.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.357+0000 7f5b79cd6740 1 -- 172.21.15.120:0/1201186054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb16cfcdc0 msgr2=0x55cb16d1eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:17.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.357+0000 7f5b79cd6740 1 --2- 172.21.15.120:0/1201186054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb16cfcdc0 0x55cb16d1eeb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x55cb16d058d0 tx=0x7f5b60009830 comp rx=0 tx=0).stop 2024-09-19T09:08:17.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.357+0000 7f5b79cd6740 1 -- 172.21.15.120:0/1201186054 shutdown_connections 2024-09-19T09:08:17.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.357+0000 7f5b79cd6740 1 --2- 172.21.15.120:0/1201186054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb16d07680 0x55cb16d1f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.357+0000 7f5b79cd6740 1 --2- 172.21.15.120:0/1201186054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb16cfcdc0 0x55cb16d1eeb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.357+0000 7f5b79cd6740 1 --2- 172.21.15.120:0/1201186054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb16cfc4a0 0x55cb16d134b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.357+0000 7f5b79cd6740 1 -- 172.21.15.120:0/1201186054 >> 172.21.15.120:0/1201186054 conn(0x55cb16cf8a30 msgr2=0x55cb16cff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:17.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.357+0000 7f5b79cd6740 1 -- 172.21.15.120:0/1201186054 shutdown_connections 2024-09-19T09:08:17.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.357+0000 7f5b79cd6740 1 -- 172.21.15.120:0/1201186054 wait complete. 2024-09-19T09:08:17.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.357+0000 7f5b79cd6740 1 -- start start 2024-09-19T09:08:17.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.357+0000 7f5b79cd6740 1 Processor -- start 2024-09-19T09:08:17.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.358+0000 7f5b79cd6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb16cfc4a0 0x55cb16d0f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:17.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.358+0000 7f5b79cd6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb16cfcdc0 0x55cb16d0fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:17.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.358+0000 7f5b79cd6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb16d07680 0x55cb16d7b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:17.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.358+0000 7f5b79cd6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701aead0 con 0x55cb16cfc4a0 2024-09-19T09:08:17.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.358+0000 7f5b79cd6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701ae400 con 0x55cb16d07680 2024-09-19T09:08:17.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.358+0000 7f5b79cd6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701ab620 con 0x55cb16cfcdc0 2024-09-19T09:08:17.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.358+0000 7f5b79cd6740 10 client.0 _ll_register_callbacks cb 0x7f5b600043f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:08:17.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.358+0000 7f5b75e9a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb16cfc4a0 0x55cb16d0f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:17.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.358+0000 7f5b74e98640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb16cfcdc0 0x55cb16d0fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:17.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.358+0000 7f5b75e9a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb16cfc4a0 0x55cb16d0f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49774/0 (socket says 172.21.15.120:49774) 2024-09-19T09:08:17.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.358+0000 7f5b75e9a640 1 -- 172.21.15.120:0/2179203589 learned_addr learned my addr 172.21.15.120:0/2179203589 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:17.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.358+0000 7f5b75e9a640 1 --2- 172.21.15.120:0/2179203589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb16d07680 0x55cb16d7b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:17.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.358+0000 7f5b74e98640 1 -- 172.21.15.120:0/2179203589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb16d07680 msgr2=0x55cb16d7b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:17.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.358+0000 7f5b74e98640 1 --2- 172.21.15.120:0/2179203589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb16d07680 0x55cb16d7b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.358+0000 7f5b74e98640 1 -- 172.21.15.120:0/2179203589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb16cfc4a0 msgr2=0x55cb16d0f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:17.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.358+0000 7f5b74e98640 1 --2- 172.21.15.120:0/2179203589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb16cfc4a0 0x55cb16d0f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.358+0000 7f5b74e98640 1 -- 172.21.15.120:0/2179203589 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb16d7b580 con 0x55cb16cfcdc0 2024-09-19T09:08:17.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.358+0000 7f5b74e98640 1 --2- 172.21.15.120:0/2179203589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb16cfcdc0 0x55cb16d0fb60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5b600038c0 tx=0x7f5b60009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aabc1e8f9e07004d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:17.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.358+0000 7f5b58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:08:17.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.359+0000 7f5b58ff9640 1 -- 172.21.15.120:0/2179203589 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60054070 con 0x55cb16cfcdc0 2024-09-19T09:08:17.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.359+0000 7f5b58ff9640 1 -- 172.21.15.120:0/2179203589 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f5b600319c0 con 0x55cb16cfcdc0 2024-09-19T09:08:17.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.359+0000 7f5b79cd6740 1 -- 172.21.15.120:0/2179203589 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b60005850 con 0x55cb16cfcdc0 2024-09-19T09:08:17.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.359+0000 7f5b58ff9640 1 -- 172.21.15.120:0/2179203589 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5b60035040 con 0x55cb16cfcdc0 2024-09-19T09:08:17.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.359+0000 7f5b79cd6740 10 client.26509 fetch_fsmap learned FSMap version 30 2024-09-19T09:08:17.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.359+0000 7f5b79cd6740 1 -- 172.21.15.120:0/2179203589 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cb16d86a00 con 0x55cb16cfcdc0 2024-09-19T09:08:17.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.359+0000 7f5b58ff9640 1 -- 172.21.15.120:0/2179203589 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b60005850 con 0x55cb16cfcdc0 2024-09-19T09:08:17.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.359+0000 7f5b58ff9640 1 -- 172.21.15.120:0/2179203589 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5b60044040 con 0x55cb16cfcdc0 2024-09-19T09:08:17.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.359+0000 7f5b79cd6740 10 client.26509 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:08:17.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.359+0000 7f5b79cd6740 10 client.26509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:08:17.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.360+0000 7f5b79cd6740 20 client.26509 populate_metadata read hostname 'smithi120' 2024-09-19T09:08:17.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.360+0000 7f5b79cd6740 1 --2- 172.21.15.120:0/2179203589 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cb16d86fb0 0x55cb16d893a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:17.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.360+0000 7f5b79cd6740 4 client.26509 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:08:17.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.360+0000 7f5b79cd6740 1 -- 172.21.15.120:0/2179203589 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cb16c69d60 con 0x55cb16d86fb0 2024-09-19T09:08:17.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.360+0000 7f5b75e9a640 1 --2- 172.21.15.120:0/2179203589 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cb16d86fb0 0x55cb16d893a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:17.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.360+0000 7f5b75e9a640 1 --2- 172.21.15.120:0/2179203589 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cb16d86fb0 0x55cb16d893a0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f71aa59810aa0f server_cookie=fd0a214553f80448 in_seq=0 out_seq=0 2024-09-19T09:08:17.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.360+0000 7f5b58ff9640 10 client.26509 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:17.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.362+0000 7f5b58ff9640 1 -- 172.21.15.120:0/2179203589 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cb16c69d60 con 0x55cb16d86fb0 2024-09-19T09:08:17.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.362+0000 7f5b58ff9640 10 client.26509 handle_command_reply: tid=0 2024-09-19T09:08:17.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.362+0000 7f5b79cd6740 1 client.26509 shutdown 2024-09-19T09:08:17.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.362+0000 7f5b79cd6740 1 -- 172.21.15.120:0/2179203589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb16cfcdc0 msgr2=0x55cb16d0fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:17.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.362+0000 7f5b79cd6740 1 --2- 172.21.15.120:0/2179203589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb16cfcdc0 0x55cb16d0fb60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5b600038c0 tx=0x7f5b60009680 comp rx=0 tx=0).stop 2024-09-19T09:08:17.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.362+0000 7f5b79cd6740 1 -- 172.21.15.120:0/2179203589 shutdown_connections 2024-09-19T09:08:17.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.362+0000 7f5b79cd6740 1 --2- 172.21.15.120:0/2179203589 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cb16d86fb0 0x55cb16d893a0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.362+0000 7f5b79cd6740 1 --2- 172.21.15.120:0/2179203589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cb16d07680 0x55cb16d7b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.362+0000 7f5b79cd6740 1 --2- 172.21.15.120:0/2179203589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb16cfcdc0 0x55cb16d0fb60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.362+0000 7f5b58ff9640 0 client.26509 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:17.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.362+0000 7f5b79cd6740 1 --2- 172.21.15.120:0/2179203589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cb16cfc4a0 0x55cb16d0f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.362+0000 7f5b79cd6740 1 -- 172.21.15.120:0/2179203589 >> 172.21.15.120:0/2179203589 conn(0x55cb16cf8a30 msgr2=0x55cb16cf7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:17.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.362+0000 7f5b79cd6740 1 -- 172.21.15.120:0/2179203589 shutdown_connections 2024-09-19T09:08:17.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.362+0000 7f5b79cd6740 1 -- 172.21.15.120:0/2179203589 wait complete. 2024-09-19T09:08:17.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.362+0000 7f5b79cd6740 20 client.26509 trim_cache size 0 max 16384 2024-09-19T09:08:17.375 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:08:17.375 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:08:17.375 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:08:17.375 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:08:17.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.363+0000 7f5b778fd640 1 -- 172.21.15.120:0/961878211 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5b5c0a3240 msgr2=0x7f5b5c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:17.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.363+0000 7f5b778fd640 1 --2- 172.21.15.120:0/961878211 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5b5c0a3240 0x7f5b5c0a5700 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c0a4650 tx=0x7f5b6c0d5530 comp rx=0 tx=0).stop 2024-09-19T09:08:17.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.363+0000 7f5b778fd640 1 -- 172.21.15.120:0/961878211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b70102dd0 msgr2=0x7f5b70198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:17.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.363+0000 7f5b778fd640 1 --2- 172.21.15.120:0/961878211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b70102dd0 0x7f5b70198520 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00c970 tx=0x7f5b6c00ce30 comp rx=0 tx=0).stop 2024-09-19T09:08:17.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.363+0000 7f5b778fd640 1 -- 172.21.15.120:0/961878211 shutdown_connections 2024-09-19T09:08:17.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.363+0000 7f5b778fd640 1 --2- 172.21.15.120:0/961878211 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5b5c0a3240 0x7f5b5c0a5700 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.363+0000 7f5b778fd640 1 --2- 172.21.15.120:0/961878211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b70108300 0x7f5b7019cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.363+0000 7f5b778fd640 1 --2- 172.21.15.120:0/961878211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b70103780 0x7f5b70198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.363+0000 7f5b778fd640 1 --2- 172.21.15.120:0/961878211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b70102dd0 0x7f5b70198520 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:17.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.363+0000 7f5b778fd640 1 -- 172.21.15.120:0/961878211 >> 172.21.15.120:0/961878211 conn(0x7f5b700fc7d0 msgr2=0x7f5b701063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:17.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.363+0000 7f5b778fd640 1 -- 172.21.15.120:0/961878211 shutdown_connections 2024-09-19T09:08:17.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:17.363+0000 7f5b778fd640 1 -- 172.21.15.120:0/961878211 wait complete. 2024-09-19T09:08:18.050 INFO:tasks.workunit.client.0.smithi120.stdout: 1 390799 6.62 MB/sec execute 133 sec latency 10.376 ms 2024-09-19T09:08:18.083 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:08:18.083 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:08:18.347 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:08:18.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.853+0000 7fd83caec640 1 -- 172.21.15.120:0/3049879259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd838102ef0 msgr2=0x7fd83810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:18.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.853+0000 7fd83caec640 1 --2- 172.21.15.120:0/3049879259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd838102ef0 0x7fd83810da40 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fd830009900 tx=0x7fd83002a480 comp rx=0 tx=0).stop 2024-09-19T09:08:18.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.853+0000 7fd83caec640 1 -- 172.21.15.120:0/3049879259 shutdown_connections 2024-09-19T09:08:18.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.853+0000 7fd83caec640 1 --2- 172.21.15.120:0/3049879259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd838102ef0 0x7fd83810da40 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:18.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.853+0000 7fd83caec640 1 --2- 172.21.15.120:0/3049879259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd838102570 0x7fd8381029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:18.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.853+0000 7fd83caec640 1 --2- 172.21.15.120:0/3049879259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd838100e20 0x7fd838101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:18.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.853+0000 7fd83caec640 1 -- 172.21.15.120:0/3049879259 >> 172.21.15.120:0/3049879259 conn(0x7fd8380fc7d0 msgr2=0x7fd8380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:18.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.853+0000 7fd83caec640 1 -- 172.21.15.120:0/3049879259 shutdown_connections 2024-09-19T09:08:18.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.853+0000 7fd83caec640 1 -- 172.21.15.120:0/3049879259 wait complete. 2024-09-19T09:08:18.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.854+0000 7fd83caec640 1 Processor -- start 2024-09-19T09:08:18.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.854+0000 7fd83caec640 1 -- start start 2024-09-19T09:08:18.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.854+0000 7fd83caec640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd838100e20 0x7fd8381984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:18.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.854+0000 7fd83caec640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd838102570 0x7fd838198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:18.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.854+0000 7fd83caec640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd838102ef0 0x7fd83819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:18.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.854+0000 7fd83caec640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd838110fa0 con 0x7fd838102ef0 2024-09-19T09:08:18.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.854+0000 7fd83caec640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd838110e20 con 0x7fd838102570 2024-09-19T09:08:18.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.854+0000 7fd83caec640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd838111120 con 0x7fd838100e20 2024-09-19T09:08:18.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.854+0000 7fd836d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd838102ef0 0x7fd83819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:18.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.854+0000 7fd836d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd838102ef0 0x7fd83819cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49792/0 (socket says 172.21.15.120:49792) 2024-09-19T09:08:18.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.854+0000 7fd836d9d640 1 -- 172.21.15.120:0/4049785292 learned_addr learned my addr 172.21.15.120:0/4049785292 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:18.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.854+0000 7fd835d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd838102570 0x7fd838198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:18.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.854+0000 7fd83659c640 1 --2- 172.21.15.120:0/4049785292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd838100e20 0x7fd8381984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:18.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.855+0000 7fd836d9d640 1 -- 172.21.15.120:0/4049785292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd838100e20 msgr2=0x7fd8381984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:18.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.855+0000 7fd836d9d640 1 --2- 172.21.15.120:0/4049785292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd838100e20 0x7fd8381984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:18.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.855+0000 7fd836d9d640 1 -- 172.21.15.120:0/4049785292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd838102570 msgr2=0x7fd838198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:18.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.855+0000 7fd836d9d640 1 --2- 172.21.15.120:0/4049785292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd838102570 0x7fd838198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:18.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.855+0000 7fd836d9d640 1 -- 172.21.15.120:0/4049785292 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd83819d480 con 0x7fd838102ef0 2024-09-19T09:08:18.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.855+0000 7fd836d9d640 1 --2- 172.21.15.120:0/4049785292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd838102ef0 0x7fd83819cda0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fd830004770 tx=0x7fd830003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:18.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.855+0000 7fd82f7fe640 1 -- 172.21.15.120:0/4049785292 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83003f070 con 0x7fd838102ef0 2024-09-19T09:08:18.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.855+0000 7fd83caec640 1 -- 172.21.15.120:0/4049785292 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd83819d710 con 0x7fd838102ef0 2024-09-19T09:08:18.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.855+0000 7fd83caec640 1 -- 172.21.15.120:0/4049785292 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8381a5050 con 0x7fd838102ef0 2024-09-19T09:08:18.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.855+0000 7fd82f7fe640 1 -- 172.21.15.120:0/4049785292 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd830031c30 con 0x7fd838102ef0 2024-09-19T09:08:18.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.855+0000 7fd82f7fe640 1 -- 172.21.15.120:0/4049785292 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd830035030 con 0x7fd838102ef0 2024-09-19T09:08:18.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.856+0000 7fd83caec640 1 -- 172.21.15.120:0/4049785292 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd838101200 con 0x7fd838102ef0 2024-09-19T09:08:18.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.858+0000 7fd82f7fe640 1 -- 172.21.15.120:0/4049785292 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd830042070 con 0x7fd838102ef0 2024-09-19T09:08:18.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.858+0000 7fd82f7fe640 1 --2- 172.21.15.120:0/4049785292 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd8240a3240 0x7fd8240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:18.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.859+0000 7fd83659c640 1 --2- 172.21.15.120:0/4049785292 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd8240a3240 0x7fd8240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:18.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.859+0000 7fd82f7fe640 1 -- 172.21.15.120:0/4049785292 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fd8300f0870 con 0x7fd838102ef0 2024-09-19T09:08:18.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.859+0000 7fd83659c640 1 --2- 172.21.15.120:0/4049785292 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd8240a3240 0x7fd8240a5700 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd82800a400 tx=0x7fd82800d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:18.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:18.862+0000 7fd82f7fe640 1 -- 172.21.15.120:0/4049785292 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd8300bcf40 con 0x7fd838102ef0 2024-09-19T09:08:19.050 INFO:tasks.workunit.client.0.smithi120.stdout: 1 394015 6.71 MB/sec execute 134 sec latency 22.508 ms 2024-09-19T09:08:19.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:18 smithi120 ceph-mon[31150]: pgmap v505: 97 pgs: 97 active+clean; 27 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 466 KiB/s wr, 79 op/s 2024-09-19T09:08:19.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:19.082+0000 7fd83caec640 1 -- 172.21.15.120:0/4049785292 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd83810a0a0 con 0x7fd838102ef0 2024-09-19T09:08:19.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:19.083+0000 7fd82f7fe640 1 -- 172.21.15.120:0/4049785292 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11985 (secure 0 0 0) 0x7fd8300c1240 con 0x7fd838102ef0 2024-09-19T09:08:19.083 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:08:19.084 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":30,"btime":"2024-09-19T09:07:14:812065+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:07:14.812046+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26351":{"gid":26351,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/364941779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":364941779},{"type":"v1","addr":"172.21.15.153:6835","nonce":364941779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:08:19.084 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 30 2024-09-19T09:08:19.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:19.085+0000 7fd83caec640 1 -- 172.21.15.120:0/4049785292 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd8240a3240 msgr2=0x7fd8240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:19.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:19.085+0000 7fd83caec640 1 --2- 172.21.15.120:0/4049785292 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd8240a3240 0x7fd8240a5700 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd82800a400 tx=0x7fd82800d040 comp rx=0 tx=0).stop 2024-09-19T09:08:19.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:19.085+0000 7fd83caec640 1 -- 172.21.15.120:0/4049785292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd838102ef0 msgr2=0x7fd83819cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:19.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:19.085+0000 7fd83caec640 1 --2- 172.21.15.120:0/4049785292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd838102ef0 0x7fd83819cda0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fd830004770 tx=0x7fd830003c10 comp rx=0 tx=0).stop 2024-09-19T09:08:19.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:19.085+0000 7fd83caec640 1 -- 172.21.15.120:0/4049785292 shutdown_connections 2024-09-19T09:08:19.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:19.085+0000 7fd83caec640 1 --2- 172.21.15.120:0/4049785292 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd8240a3240 0x7fd8240a5700 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:19.085+0000 7fd83caec640 1 --2- 172.21.15.120:0/4049785292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd838102ef0 0x7fd83819cda0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:19.085+0000 7fd83caec640 1 --2- 172.21.15.120:0/4049785292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd838102570 0x7fd838198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:19.085+0000 7fd83caec640 1 --2- 172.21.15.120:0/4049785292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd838100e20 0x7fd8381984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:19.085+0000 7fd83caec640 1 -- 172.21.15.120:0/4049785292 >> 172.21.15.120:0/4049785292 conn(0x7fd8380fc7d0 msgr2=0x7fd83806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:19.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:19.085+0000 7fd83caec640 1 -- 172.21.15.120:0/4049785292 shutdown_connections 2024-09-19T09:08:19.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:19.085+0000 7fd83caec640 1 -- 172.21.15.120:0/4049785292 wait complete. 2024-09-19T09:08:19.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:18 smithi196 ceph-mon[35150]: pgmap v505: 97 pgs: 97 active+clean; 27 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 466 KiB/s wr, 79 op/s 2024-09-19T09:08:19.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:18 smithi153 ceph-mon[35071]: pgmap v505: 97 pgs: 97 active+clean; 27 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 466 KiB/s wr, 79 op/s 2024-09-19T09:08:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.604+0000 7f3be3531640 1 -- 172.21.15.120:0/2697515646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bdc1025c0 msgr2=0x7f3bdc1049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.604+0000 7f3be3531640 1 --2- 172.21.15.120:0/2697515646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bdc1025c0 0x7f3bdc1049b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009860 tx=0x7f3bd802a340 comp rx=0 tx=0).stop 2024-09-19T09:08:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.604+0000 7f3be3531640 1 -- 172.21.15.120:0/2697515646 shutdown_connections 2024-09-19T09:08:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.604+0000 7f3be3531640 1 --2- 172.21.15.120:0/2697515646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3bdc107820 0x7f3bdc109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.604+0000 7f3be3531640 1 --2- 172.21.15.120:0/2697515646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3bdc104ef0 0x7f3bdc1072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.604+0000 7f3be3531640 1 --2- 172.21.15.120:0/2697515646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bdc1025c0 0x7f3bdc1049b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.604+0000 7f3be3531640 1 -- 172.21.15.120:0/2697515646 >> 172.21.15.120:0/2697515646 conn(0x7f3bdc0fc1e0 msgr2=0x7f3bdc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.604+0000 7f3be3531640 1 -- 172.21.15.120:0/2697515646 shutdown_connections 2024-09-19T09:08:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.604+0000 7f3be3531640 1 -- 172.21.15.120:0/2697515646 wait complete. 2024-09-19T09:08:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.605+0000 7f3be3531640 1 Processor -- start 2024-09-19T09:08:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.605+0000 7f3be3531640 1 -- start start 2024-09-19T09:08:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.605+0000 7f3be3531640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3bdc1025c0 0x7f3bdc197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.605+0000 7f3be3531640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bdc104ef0 0x7f3bdc1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.605+0000 7f3be3531640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3bdc107820 0x7f3bdc19f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.605+0000 7f3be3531640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc10d800 con 0x7f3bdc1025c0 2024-09-19T09:08:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.605+0000 7f3be3531640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc10d680 con 0x7f3bdc107820 2024-09-19T09:08:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.605+0000 7f3be3531640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc10d980 con 0x7f3bdc104ef0 2024-09-19T09:08:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.605+0000 7f3be12cd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3bdc1025c0 0x7f3bdc197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.605+0000 7f3be12cd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3bdc1025c0 0x7f3bdc197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49804/0 (socket says 172.21.15.120:49804) 2024-09-19T09:08:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.605+0000 7f3be12cd640 1 -- 172.21.15.120:0/1144041719 learned_addr learned my addr 172.21.15.120:0/1144041719 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.605+0000 7f3be0acc640 1 --2- 172.21.15.120:0/1144041719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bdc104ef0 0x7f3bdc1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.605+0000 7f3be1ace640 1 --2- 172.21.15.120:0/1144041719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3bdc107820 0x7f3bdc19f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.606+0000 7f3be12cd640 1 -- 172.21.15.120:0/1144041719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bdc104ef0 msgr2=0x7f3bdc1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.606+0000 7f3be12cd640 1 --2- 172.21.15.120:0/1144041719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bdc104ef0 0x7f3bdc1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.606+0000 7f3be12cd640 1 -- 172.21.15.120:0/1144041719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3bdc107820 msgr2=0x7f3bdc19f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.606+0000 7f3be12cd640 1 --2- 172.21.15.120:0/1144041719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3bdc107820 0x7f3bdc19f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.606+0000 7f3be12cd640 1 -- 172.21.15.120:0/1144041719 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bdc1003b0 con 0x7f3bdc1025c0 2024-09-19T09:08:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.606+0000 7f3be12cd640 1 --2- 172.21.15.120:0/1144041719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3bdc1025c0 0x7f3bdc197e80 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8005940 tx=0x7f3bd8033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.606+0000 7f3bd27fc640 1 -- 172.21.15.120:0/1144041719 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd803f020 con 0x7f3bdc1025c0 2024-09-19T09:08:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.606+0000 7f3be3531640 1 -- 172.21.15.120:0/1144041719 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bdc100640 con 0x7f3bdc1025c0 2024-09-19T09:08:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.606+0000 7f3bd27fc640 1 -- 172.21.15.120:0/1144041719 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bd8030850 con 0x7f3bdc1025c0 2024-09-19T09:08:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.606+0000 7f3bd27fc640 1 -- 172.21.15.120:0/1144041719 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8037040 con 0x7f3bdc1025c0 2024-09-19T09:08:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.606+0000 7f3be3531640 1 -- 172.21.15.120:0/1144041719 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bdc100b80 con 0x7f3bdc1025c0 2024-09-19T09:08:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.608+0000 7f3bd27fc640 1 -- 172.21.15.120:0/1144041719 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f3bd8042020 con 0x7f3bdc1025c0 2024-09-19T09:08:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.609+0000 7f3be62ce740 1 Processor -- start 2024-09-19T09:08:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.609+0000 7f3be62ce740 1 -- start start 2024-09-19T09:08:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.609+0000 7f3be62ce740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e386999a0 0x561e38699da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.609+0000 7f3be62ce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e3869a2e0 0x561e3869eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.609+0000 7f3be62ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e3869f370 0x561e386a1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.609+0000 7f3be62ce740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561e385fd2f0 con 0x561e386999a0 2024-09-19T09:08:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.609+0000 7f3be62ce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561e385a5270 con 0x561e3869a2e0 2024-09-19T09:08:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.609+0000 7f3be62ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561e385f92b0 con 0x561e3869f370 2024-09-19T09:08:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.609+0000 7f3be1ace640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e386999a0 0x561e38699da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.609+0000 7f3be1ace640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e386999a0 0x561e38699da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49812/0 (socket says 172.21.15.120:49812) 2024-09-19T09:08:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.609+0000 7f3be1ace640 1 -- 172.21.15.120:0/2242562863 learned_addr learned my addr 172.21.15.120:0/2242562863 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.609+0000 7f3bd27fc640 1 --2- 172.21.15.120:0/1144041719 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3bc80a32f0 0x7f3bc80a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.609+0000 7f3be12cd640 1 --2- 172.21.15.120:0/2242562863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e3869a2e0 0x561e3869eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.609+0000 7f3be1ace640 1 --2- 172.21.15.120:0/1144041719 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3bc80a32f0 0x7f3bc80a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.609+0000 7f3be0acc640 1 --2- 172.21.15.120:0/2242562863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e3869f370 0x561e386a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.609+0000 7f3be1ace640 1 -- 172.21.15.120:0/2242562863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e3869f370 msgr2=0x561e386a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.610+0000 7f3be1ace640 1 --2- 172.21.15.120:0/2242562863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e3869f370 0x561e386a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.610+0000 7f3be1ace640 1 -- 172.21.15.120:0/2242562863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e3869a2e0 msgr2=0x561e3869eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.610+0000 7f3be1ace640 1 --2- 172.21.15.120:0/2242562863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e3869a2e0 0x561e3869eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.610+0000 7f3be1ace640 1 -- 172.21.15.120:0/2242562863 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e385a4340 con 0x561e386999a0 2024-09-19T09:08:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.610+0000 7f3bd27fc640 1 -- 172.21.15.120:0/1144041719 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f3bd80efc30 con 0x7f3bdc1025c0 2024-09-19T09:08:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.610+0000 7f3be1ace640 1 --2- 172.21.15.120:0/2242562863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e386999a0 0x561e38699da0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400e950 tx=0x7f3bd400ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c199225955641e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.610+0000 7f3bc77fe640 1 -- 172.21.15.120:0/2242562863 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4021040 con 0x561e386999a0 2024-09-19T09:08:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.610+0000 7f3bc77fe640 1 -- 172.21.15.120:0/2242562863 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bd4022020 con 0x561e386999a0 2024-09-19T09:08:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.610+0000 7f3be1ace640 1 --2- 172.21.15.120:0/1144041719 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3bc80a32f0 0x7f3bc80a57b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4023000 tx=0x7f3bd4020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.610+0000 7f3bc77fe640 1 -- 172.21.15.120:0/2242562863 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd401d9a0 con 0x561e386999a0 2024-09-19T09:08:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.611+0000 7f3be62ce740 1 -- 172.21.15.120:0/2242562863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e386999a0 msgr2=0x561e38699da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.611+0000 7f3be62ce740 1 --2- 172.21.15.120:0/2242562863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e386999a0 0x561e38699da0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400e950 tx=0x7f3bd400ee10 comp rx=0 tx=0).stop 2024-09-19T09:08:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.611+0000 7f3be62ce740 1 -- 172.21.15.120:0/2242562863 shutdown_connections 2024-09-19T09:08:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.611+0000 7f3be62ce740 1 --2- 172.21.15.120:0/2242562863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e3869f370 0x561e386a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.611+0000 7f3be62ce740 1 --2- 172.21.15.120:0/2242562863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e3869a2e0 0x561e3869eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.611+0000 7f3be62ce740 1 --2- 172.21.15.120:0/2242562863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e386999a0 0x561e38699da0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.611+0000 7f3be62ce740 1 -- 172.21.15.120:0/2242562863 >> 172.21.15.120:0/2242562863 conn(0x561e38695010 msgr2=0x561e38695410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.611+0000 7f3be62ce740 1 -- 172.21.15.120:0/2242562863 shutdown_connections 2024-09-19T09:08:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.611+0000 7f3be62ce740 1 -- 172.21.15.120:0/2242562863 wait complete. 2024-09-19T09:08:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.611+0000 7f3be62ce740 1 -- start start 2024-09-19T09:08:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.611+0000 7f3be62ce740 1 Processor -- start 2024-09-19T09:08:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.612+0000 7f3be62ce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e386999a0 0x561e386a3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.612+0000 7f3be62ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e3869a2e0 0x561e386a41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.612+0000 7f3be62ce740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e3869f370 0x561e386ab170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.612+0000 7f3be62ce740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561e386a2340 con 0x561e3869f370 2024-09-19T09:08:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.612+0000 7f3be62ce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561e38586290 con 0x561e386999a0 2024-09-19T09:08:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.612+0000 7f3be62ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561e386a24c0 con 0x561e3869a2e0 2024-09-19T09:08:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.612+0000 7f3be62ce740 10 client.0 _ll_register_callbacks cb 0x561e385b7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:08:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.612+0000 7f3be1ace640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e3869f370 0x561e386ab170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.612+0000 7f3be1ace640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e3869f370 0x561e386ab170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49816/0 (socket says 172.21.15.120:49816) 2024-09-19T09:08:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.612+0000 7f3be1ace640 1 -- 172.21.15.120:0/800829281 learned_addr learned my addr 172.21.15.120:0/800829281 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.612+0000 7f3be12cd640 1 --2- 172.21.15.120:0/800829281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e386999a0 0x561e386a3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.612+0000 7f3be0acc640 1 --2- 172.21.15.120:0/800829281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e3869a2e0 0x561e386a41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.612+0000 7f3be12cd640 1 -- 172.21.15.120:0/800829281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e3869a2e0 msgr2=0x561e386a41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.612+0000 7f3be12cd640 1 --2- 172.21.15.120:0/800829281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e3869a2e0 0x561e386a41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.612+0000 7f3be12cd640 1 -- 172.21.15.120:0/800829281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e3869f370 msgr2=0x561e386ab170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.612+0000 7f3be12cd640 1 --2- 172.21.15.120:0/800829281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e3869f370 0x561e386ab170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.612+0000 7f3be12cd640 1 -- 172.21.15.120:0/800829281 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e38310210 con 0x561e386999a0 2024-09-19T09:08:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.612+0000 7f3be12cd640 1 --2- 172.21.15.120:0/800829281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e386999a0 0x561e386a3c60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009830 tx=0x7f3bd80037d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f06c1361cd2c8914 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.612+0000 7f3bc4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:08:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.612+0000 7f3bc4ff9640 1 -- 172.21.15.120:0/800829281 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd80c4c30 con 0x561e386999a0 2024-09-19T09:08:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.612+0000 7f3bc4ff9640 1 -- 172.21.15.120:0/800829281 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f3bd80dc0b0 con 0x561e386999a0 2024-09-19T09:08:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.612+0000 7f3be62ce740 1 -- 172.21.15.120:0/800829281 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bd4022020 con 0x561e386999a0 2024-09-19T09:08:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.612+0000 7f3bc4ff9640 1 -- 172.21.15.120:0/800829281 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3bd80035f0 con 0x561e386999a0 2024-09-19T09:08:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.612+0000 7f3be62ce740 10 client.26612 fetch_fsmap learned FSMap version 30 2024-09-19T09:08:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.612+0000 7f3be62ce740 1 -- 172.21.15.120:0/800829281 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561e383dc620 con 0x561e386999a0 2024-09-19T09:08:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.613+0000 7f3bc4ff9640 1 -- 172.21.15.120:0/800829281 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bd4022020 con 0x561e386999a0 2024-09-19T09:08:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.613+0000 7f3bc4ff9640 1 -- 172.21.15.120:0/800829281 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3bd803a030 con 0x561e386999a0 2024-09-19T09:08:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.613+0000 7f3be62ce740 10 client.26612 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:08:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.613+0000 7f3be62ce740 10 client.26612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:08:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.613+0000 7f3be62ce740 20 client.26612 populate_metadata read hostname 'smithi120' 2024-09-19T09:08:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.613+0000 7f3be62ce740 1 --2- 172.21.15.120:0/800829281 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561e386af7b0 0x561e386b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.613+0000 7f3be62ce740 4 client.26612 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:08:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.613+0000 7f3be62ce740 1 -- 172.21.15.120:0/800829281 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e3860aa20 con 0x561e386af7b0 2024-09-19T09:08:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.613+0000 7f3be0acc640 1 --2- 172.21.15.120:0/800829281 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561e386af7b0 0x561e386b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.613+0000 7f3bc4ff9640 10 client.26612 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.613+0000 7f3be0acc640 1 --2- 172.21.15.120:0/800829281 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561e386af7b0 0x561e386b1bc0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38d2bcfd43a1175d server_cookie=f7c44fb3938bb4ab in_seq=0 out_seq=0 2024-09-19T09:08:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.614+0000 7f3bc4ff9640 1 -- 172.21.15.120:0/800829281 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e3860aa20 con 0x561e386af7b0 2024-09-19T09:08:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.614+0000 7f3bc4ff9640 10 client.26612 handle_command_reply: tid=0 2024-09-19T09:08:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.614+0000 7f3be62ce740 1 client.26612 shutdown 2024-09-19T09:08:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.614+0000 7f3be62ce740 1 -- 172.21.15.120:0/800829281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e386999a0 msgr2=0x561e386a3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.614+0000 7f3be62ce740 1 --2- 172.21.15.120:0/800829281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e386999a0 0x561e386a3c60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009830 tx=0x7f3bd80037d0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.614+0000 7f3be62ce740 1 -- 172.21.15.120:0/800829281 shutdown_connections 2024-09-19T09:08:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.614+0000 7f3be62ce740 1 --2- 172.21.15.120:0/800829281 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561e386af7b0 0x561e386b1bc0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.614+0000 7f3be62ce740 1 --2- 172.21.15.120:0/800829281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e3869f370 0x561e386ab170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.614+0000 7f3be62ce740 1 --2- 172.21.15.120:0/800829281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e3869a2e0 0x561e386a41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.614+0000 7f3be62ce740 1 --2- 172.21.15.120:0/800829281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e386999a0 0x561e386a3c60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.614+0000 7f3bc4ff9640 0 client.26612 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.614+0000 7f3be62ce740 1 -- 172.21.15.120:0/800829281 >> 172.21.15.120:0/800829281 conn(0x561e38695010 msgr2=0x561e38696820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.615+0000 7f3be62ce740 1 -- 172.21.15.120:0/800829281 shutdown_connections 2024-09-19T09:08:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.615+0000 7f3be62ce740 1 -- 172.21.15.120:0/800829281 wait complete. 2024-09-19T09:08:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.615+0000 7f3be62ce740 20 client.26612 trim_cache size 0 max 16384 2024-09-19T09:08:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.629+0000 7f3be62ce740 1 Processor -- start 2024-09-19T09:08:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.629+0000 7f3be62ce740 1 -- start start 2024-09-19T09:08:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.630+0000 7f3be62ce740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e386999a0 0x561e386b6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.630+0000 7f3be62ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e3869a2e0 0x561e386ab090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.630+0000 7f3be62ce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e3869f370 0x561e386b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.630+0000 7f3be62ce740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc1ab220 con 0x561e386999a0 2024-09-19T09:08:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.630+0000 7f3be62ce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc1acd60 con 0x561e3869f370 2024-09-19T09:08:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.630+0000 7f3be62ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc1ab8f0 con 0x561e3869a2e0 2024-09-19T09:08:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.630+0000 7f3be12cd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e386999a0 0x561e386b6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.630+0000 7f3be12cd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e386999a0 0x561e386b6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49822/0 (socket says 172.21.15.120:49822) 2024-09-19T09:08:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.630+0000 7f3be12cd640 1 -- 172.21.15.120:0/247379797 learned_addr learned my addr 172.21.15.120:0/247379797 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.630+0000 7f3be1ace640 1 --2- 172.21.15.120:0/247379797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e3869f370 0x561e386b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.630+0000 7f3be0acc640 1 --2- 172.21.15.120:0/247379797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e3869a2e0 0x561e386ab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.630+0000 7f3be12cd640 1 -- 172.21.15.120:0/247379797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e3869a2e0 msgr2=0x561e386ab090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.630+0000 7f3be12cd640 1 --2- 172.21.15.120:0/247379797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e3869a2e0 0x561e386ab090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.630+0000 7f3be12cd640 1 -- 172.21.15.120:0/247379797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e3869f370 msgr2=0x561e386b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.630+0000 7f3be12cd640 1 --2- 172.21.15.120:0/247379797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e3869f370 0x561e386b69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.630+0000 7f3be12cd640 1 -- 172.21.15.120:0/247379797 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e386b2210 con 0x561e386999a0 2024-09-19T09:08:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.630+0000 7f3be12cd640 1 --2- 172.21.15.120:0/247379797 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e386999a0 0x561e386b6490 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80bbf50 tx=0x7f3bd8005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5806d018ee76531e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.630+0000 7f3bc6ffd640 1 -- 172.21.15.120:0/247379797 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8030a00 con 0x561e386999a0 2024-09-19T09:08:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.630+0000 7f3bc6ffd640 1 -- 172.21.15.120:0/247379797 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bd80bc950 con 0x561e386999a0 2024-09-19T09:08:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.630+0000 7f3bc6ffd640 1 -- 172.21.15.120:0/247379797 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd80c03f0 con 0x561e386999a0 2024-09-19T09:08:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.631+0000 7f3be62ce740 1 -- 172.21.15.120:0/247379797 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e386999a0 msgr2=0x561e386b6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.631+0000 7f3be62ce740 1 --2- 172.21.15.120:0/247379797 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e386999a0 0x561e386b6490 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80bbf50 tx=0x7f3bd8005780 comp rx=0 tx=0).stop 2024-09-19T09:08:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.631+0000 7f3be62ce740 1 -- 172.21.15.120:0/247379797 shutdown_connections 2024-09-19T09:08:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.631+0000 7f3be62ce740 1 --2- 172.21.15.120:0/247379797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e3869f370 0x561e386b69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.631+0000 7f3be62ce740 1 --2- 172.21.15.120:0/247379797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e3869a2e0 0x561e386ab090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.631+0000 7f3be62ce740 1 --2- 172.21.15.120:0/247379797 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e386999a0 0x561e386b6490 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.631+0000 7f3be62ce740 1 -- 172.21.15.120:0/247379797 >> 172.21.15.120:0/247379797 conn(0x561e38695010 msgr2=0x561e386941c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.631+0000 7f3be62ce740 1 -- 172.21.15.120:0/247379797 shutdown_connections 2024-09-19T09:08:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.631+0000 7f3be62ce740 1 -- 172.21.15.120:0/247379797 wait complete. 2024-09-19T09:08:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.632+0000 7f3be62ce740 1 -- start start 2024-09-19T09:08:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.632+0000 7f3be62ce740 1 Processor -- start 2024-09-19T09:08:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.632+0000 7f3be62ce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e386999a0 0x561e386aa950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.632+0000 7f3be62ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e3869a2e0 0x561e386aae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.632+0000 7f3be62ce740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e3869f370 0x561e386a8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.632+0000 7f3be62ce740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc1ac690 con 0x561e3869f370 2024-09-19T09:08:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.632+0000 7f3be62ce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc1abfc0 con 0x561e386999a0 2024-09-19T09:08:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.632+0000 7f3be62ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc1a91e0 con 0x561e3869a2e0 2024-09-19T09:08:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.632+0000 7f3be62ce740 10 client.0 _ll_register_callbacks cb 0x7f3bd8005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:08:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.632+0000 7f3be1ace640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e3869f370 0x561e386a8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.632+0000 7f3be12cd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e386999a0 0x561e386aa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.632+0000 7f3be12cd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e386999a0 0x561e386aa950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:53776/0 (socket says 172.21.15.120:53776) 2024-09-19T09:08:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.632+0000 7f3be12cd640 1 -- 172.21.15.120:0/1418785961 learned_addr learned my addr 172.21.15.120:0/1418785961 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.632+0000 7f3be1ace640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e3869f370 0x561e386a8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49824/0 (socket says 172.21.15.120:49824) 2024-09-19T09:08:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.632+0000 7f3be0acc640 1 --2- 172.21.15.120:0/1418785961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e3869a2e0 0x561e386aae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.633+0000 7f3be1ace640 1 -- 172.21.15.120:0/1418785961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e3869a2e0 msgr2=0x561e386aae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.633+0000 7f3be1ace640 1 --2- 172.21.15.120:0/1418785961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e3869a2e0 0x561e386aae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.633+0000 7f3be1ace640 1 -- 172.21.15.120:0/1418785961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e386999a0 msgr2=0x561e386aa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.633+0000 7f3be1ace640 1 --2- 172.21.15.120:0/1418785961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e386999a0 0x561e386aa950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.633+0000 7f3be1ace640 1 -- 172.21.15.120:0/1418785961 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e386a9710 con 0x561e3869f370 2024-09-19T09:08:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.633+0000 7f3be0acc640 1 --2- 172.21.15.120:0/1418785961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e3869a2e0 0x561e386aae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.633+0000 7f3be12cd640 1 --2- 172.21.15.120:0/1418785961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e386999a0 0x561e386aa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.633+0000 7f3be1ace640 1 --2- 172.21.15.120:0/1418785961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e3869f370 0x561e386a8fd0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4024040 tx=0x7f3bd40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f72646f207885d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.633+0000 7f3bc4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:08:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.633+0000 7f3bc4ff9640 1 -- 172.21.15.120:0/1418785961 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4021040 con 0x561e3869f370 2024-09-19T09:08:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.633+0000 7f3bc4ff9640 1 -- 172.21.15.120:0/1418785961 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f3bd400e450 con 0x561e3869f370 2024-09-19T09:08:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.633+0000 7f3be62ce740 1 -- 172.21.15.120:0/1418785961 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bd80bc950 con 0x561e3869f370 2024-09-19T09:08:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.633+0000 7f3bc4ff9640 1 -- 172.21.15.120:0/1418785961 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3bd403a050 con 0x561e3869f370 2024-09-19T09:08:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.634+0000 7f3be62ce740 10 client.17346 fetch_fsmap learned FSMap version 30 2024-09-19T09:08:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.634+0000 7f3be62ce740 1 -- 172.21.15.120:0/1418785961 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561e38723720 con 0x561e3869f370 2024-09-19T09:08:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.634+0000 7f3bc4ff9640 1 -- 172.21.15.120:0/1418785961 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bd80bc950 con 0x561e3869f370 2024-09-19T09:08:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.634+0000 7f3bc4ff9640 1 -- 172.21.15.120:0/1418785961 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3bd400b040 con 0x561e3869f370 2024-09-19T09:08:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.634+0000 7f3be62ce740 10 client.17346 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:08:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.634+0000 7f3be62ce740 10 client.17346 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:08:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.634+0000 7f3be62ce740 20 client.17346 populate_metadata read hostname 'smithi120' 2024-09-19T09:08:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.634+0000 7f3be62ce740 1 --2- 172.21.15.120:0/1418785961 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561e38723f30 0x561e38726320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.634+0000 7f3be62ce740 4 client.17346 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:08:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.634+0000 7f3be62ce740 1 -- 172.21.15.120:0/1418785961 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e38604d70 con 0x561e38723f30 2024-09-19T09:08:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.634+0000 7f3be12cd640 1 --2- 172.21.15.120:0/1418785961 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561e38723f30 0x561e38726320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.635+0000 7f3bc4ff9640 10 client.17346 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.635+0000 7f3be12cd640 1 --2- 172.21.15.120:0/1418785961 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561e38723f30 0x561e38726320 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a71c7631dec4c83 server_cookie=436da4d8990dbbd8 in_seq=0 out_seq=0 2024-09-19T09:08:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.752+0000 7f3bc4ff9640 1 -- 172.21.15.120:0/1418785961 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561e38604d70 con 0x561e38723f30 2024-09-19T09:08:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.752+0000 7f3bc4ff9640 10 client.17346 handle_command_reply: tid=0 2024-09-19T09:08:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.753+0000 7f3be62ce740 1 client.17346 shutdown 2024-09-19T09:08:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.753+0000 7f3be62ce740 1 -- 172.21.15.120:0/1418785961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e3869f370 msgr2=0x561e386a8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.753+0000 7f3be62ce740 1 --2- 172.21.15.120:0/1418785961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e3869f370 0x561e386a8fd0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4024040 tx=0x7f3bd40095c0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.753+0000 7f3be62ce740 1 -- 172.21.15.120:0/1418785961 shutdown_connections 2024-09-19T09:08:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.753+0000 7f3be62ce740 1 --2- 172.21.15.120:0/1418785961 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561e38723f30 0x561e38726320 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.753+0000 7f3be62ce740 1 --2- 172.21.15.120:0/1418785961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e3869f370 0x561e386a8fd0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.753+0000 7f3be62ce740 1 --2- 172.21.15.120:0/1418785961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e3869a2e0 0x561e386aae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.753+0000 7f3be62ce740 1 --2- 172.21.15.120:0/1418785961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e386999a0 0x561e386aa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.753+0000 7f3be62ce740 1 -- 172.21.15.120:0/1418785961 >> 172.21.15.120:0/1418785961 conn(0x561e38695010 msgr2=0x561e386939a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.753+0000 7f3bc4ff9640 0 client.17346 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.753+0000 7f3be62ce740 1 -- 172.21.15.120:0/1418785961 shutdown_connections 2024-09-19T09:08:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.753+0000 7f3be62ce740 1 -- 172.21.15.120:0/1418785961 wait complete. 2024-09-19T09:08:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.753+0000 7f3be62ce740 20 client.17346 trim_cache size 0 max 16384 2024-09-19T09:08:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:08:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:08:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:08:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:08:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.754+0000 7f3be3531640 1 -- 172.21.15.120:0/1144041719 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3bc80a32f0 msgr2=0x7f3bc80a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.754+0000 7f3be3531640 1 --2- 172.21.15.120:0/1144041719 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3bc80a32f0 0x7f3bc80a57b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4023000 tx=0x7f3bd4020000 comp rx=0 tx=0).stop 2024-09-19T09:08:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.754+0000 7f3be3531640 1 -- 172.21.15.120:0/1144041719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3bdc1025c0 msgr2=0x7f3bdc197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.754+0000 7f3be3531640 1 --2- 172.21.15.120:0/1144041719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3bdc1025c0 0x7f3bdc197e80 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8005940 tx=0x7f3bd8033d90 comp rx=0 tx=0).stop 2024-09-19T09:08:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.754+0000 7f3be3531640 1 -- 172.21.15.120:0/1144041719 shutdown_connections 2024-09-19T09:08:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.754+0000 7f3be3531640 1 --2- 172.21.15.120:0/1144041719 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3bc80a32f0 0x7f3bc80a57b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.754+0000 7f3be3531640 1 --2- 172.21.15.120:0/1144041719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3bdc107820 0x7f3bdc19f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.754+0000 7f3be3531640 1 --2- 172.21.15.120:0/1144041719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bdc104ef0 0x7f3bdc1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.754+0000 7f3be3531640 1 --2- 172.21.15.120:0/1144041719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3bdc1025c0 0x7f3bdc197e80 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.754+0000 7f3be3531640 1 -- 172.21.15.120:0/1144041719 >> 172.21.15.120:0/1144041719 conn(0x7f3bdc0fc1e0 msgr2=0x7f3bdc102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.754+0000 7f3be3531640 1 -- 172.21.15.120:0/1144041719 shutdown_connections 2024-09-19T09:08:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:19.754+0000 7f3be3531640 1 -- 172.21.15.120:0/1144041719 wait complete. 2024-09-19T09:08:19.863 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:08:20.050 INFO:tasks.workunit.client.0.smithi120.stdout: 1 396699 6.75 MB/sec execute 135 sec latency 56.108 ms 2024-09-19T09:08:20.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:19 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/4049785292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:08:20.122 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:08:20.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:19 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/4049785292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:08:20.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:19 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/4049785292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:08:20.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.641+0000 7fd38f3e5640 1 -- 172.21.15.120:0/30512005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd388106f10 msgr2=0x7fd38810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:20.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.641+0000 7fd38f3e5640 1 --2- 172.21.15.120:0/30512005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd388106f10 0x7fd38810ea90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd380009be0 tx=0x7fd38002a6c0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.641+0000 7fd38f3e5640 1 -- 172.21.15.120:0/30512005 shutdown_connections 2024-09-19T09:08:20.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.641+0000 7fd38f3e5640 1 --2- 172.21.15.120:0/30512005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd388106f10 0x7fd38810ea90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.641+0000 7fd38f3e5640 1 --2- 172.21.15.120:0/30512005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd388102790 0x7fd3881067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.641+0000 7fd38f3e5640 1 --2- 172.21.15.120:0/30512005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd388101de0 0x7fd3881021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.641+0000 7fd38f3e5640 1 -- 172.21.15.120:0/30512005 >> 172.21.15.120:0/30512005 conn(0x7fd3880fd880 msgr2=0x7fd3880ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:20.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.641+0000 7fd38f3e5640 1 -- 172.21.15.120:0/30512005 shutdown_connections 2024-09-19T09:08:20.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.641+0000 7fd38f3e5640 1 -- 172.21.15.120:0/30512005 wait complete. 2024-09-19T09:08:20.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.642+0000 7fd38f3e5640 1 Processor -- start 2024-09-19T09:08:20.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.642+0000 7fd38f3e5640 1 -- start start 2024-09-19T09:08:20.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.642+0000 7fd38f3e5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd388101de0 0x7fd38810acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:20.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.642+0000 7fd38f3e5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd388102790 0x7fd38810b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:20.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.642+0000 7fd38f3e5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd388106f10 0x7fd38810b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:20.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.642+0000 7fd38f3e5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd3880751c0 con 0x7fd388101de0 2024-09-19T09:08:20.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.642+0000 7fd38f3e5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd388075040 con 0x7fd388106f10 2024-09-19T09:08:20.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.642+0000 7fd38f3e5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd388075340 con 0x7fd388102790 2024-09-19T09:08:20.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.642+0000 7fd38d181640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd388101de0 0x7fd38810acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:20.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.642+0000 7fd38d181640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd388101de0 0x7fd38810acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49846/0 (socket says 172.21.15.120:49846) 2024-09-19T09:08:20.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.642+0000 7fd38d181640 1 -- 172.21.15.120:0/1754174514 learned_addr learned my addr 172.21.15.120:0/1754174514 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:20.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.642+0000 7fd38d982640 1 --2- 172.21.15.120:0/1754174514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd388106f10 0x7fd38810b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:20.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.643+0000 7fd38c980640 1 --2- 172.21.15.120:0/1754174514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd388102790 0x7fd38810b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:20.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.643+0000 7fd38d181640 1 -- 172.21.15.120:0/1754174514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd388102790 msgr2=0x7fd38810b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:20.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.643+0000 7fd38d181640 1 --2- 172.21.15.120:0/1754174514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd388102790 0x7fd38810b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.643+0000 7fd38d181640 1 -- 172.21.15.120:0/1754174514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd388106f10 msgr2=0x7fd38810b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:20.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.643+0000 7fd38d181640 1 --2- 172.21.15.120:0/1754174514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd388106f10 0x7fd38810b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.643+0000 7fd38d181640 1 -- 172.21.15.120:0/1754174514 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3881aac10 con 0x7fd388101de0 2024-09-19T09:08:20.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.643+0000 7fd38d181640 1 --2- 172.21.15.120:0/1754174514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd388101de0 0x7fd38810acb0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fd38400d980 tx=0x7fd38400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:20.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.643+0000 7fd37e7fc640 1 -- 172.21.15.120:0/1754174514 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd384017070 con 0x7fd388101de0 2024-09-19T09:08:20.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.643+0000 7fd38f3e5640 1 -- 172.21.15.120:0/1754174514 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3881aaf00 con 0x7fd388101de0 2024-09-19T09:08:20.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.643+0000 7fd37e7fc640 1 -- 172.21.15.120:0/1754174514 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd384009c30 con 0x7fd388101de0 2024-09-19T09:08:20.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.643+0000 7fd37e7fc640 1 -- 172.21.15.120:0/1754174514 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38400f620 con 0x7fd388101de0 2024-09-19T09:08:20.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.643+0000 7fd38f3e5640 1 -- 172.21.15.120:0/1754174514 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3881ab400 con 0x7fd388101de0 2024-09-19T09:08:20.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.645+0000 7fd37e7fc640 1 -- 172.21.15.120:0/1754174514 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd384010460 con 0x7fd388101de0 2024-09-19T09:08:20.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.646+0000 7fd3917be740 1 Processor -- start 2024-09-19T09:08:20.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.646+0000 7fd3917be740 1 -- start start 2024-09-19T09:08:20.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.646+0000 7fd3917be740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de751f7b40 0x55de751f6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:20.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.646+0000 7fd3917be740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de751f65d0 0x55de751f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:20.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.646+0000 7fd3917be740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de75201370 0x55de75203760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:20.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.646+0000 7fd3917be740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55de7515c2f0 con 0x55de751f7b40 2024-09-19T09:08:20.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.646+0000 7fd3917be740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55de75103cf0 con 0x55de75201370 2024-09-19T09:08:20.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.646+0000 7fd3917be740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55de751582b0 con 0x55de751f65d0 2024-09-19T09:08:20.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.646+0000 7fd38d982640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de751f7b40 0x55de751f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:20.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.646+0000 7fd37e7fc640 1 --2- 172.21.15.120:0/1754174514 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd3740a3240 0x7fd3740a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:20.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.646+0000 7fd38d982640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de751f7b40 0x55de751f6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49850/0 (socket says 172.21.15.120:49850) 2024-09-19T09:08:20.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.646+0000 7fd38d982640 1 -- 172.21.15.120:0/3088492501 learned_addr learned my addr 172.21.15.120:0/3088492501 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:20.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.646+0000 7fd38c980640 1 --2- 172.21.15.120:0/3088492501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de75201370 0x55de75203760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:20.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.646+0000 7fd38d181640 1 --2- 172.21.15.120:0/3088492501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de751f65d0 0x55de751f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:20.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.647+0000 7fd38d982640 1 --2- 172.21.15.120:0/1754174514 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd3740a3240 0x7fd3740a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:20.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.647+0000 7fd37e7fc640 1 -- 172.21.15.120:0/1754174514 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fd3840cf540 con 0x7fd388101de0 2024-09-19T09:08:20.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.647+0000 7fd38d181640 1 -- 172.21.15.120:0/3088492501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de75201370 msgr2=0x55de75203760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:20.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.647+0000 7fd38d181640 1 --2- 172.21.15.120:0/3088492501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de75201370 0x55de75203760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.647+0000 7fd38d181640 1 -- 172.21.15.120:0/3088492501 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de751f7b40 msgr2=0x55de751f6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:20.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.647+0000 7fd38d181640 1 --2- 172.21.15.120:0/3088492501 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de751f7b40 0x55de751f6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.647+0000 7fd38d181640 1 -- 172.21.15.120:0/3088492501 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de750e83a0 con 0x55de751f65d0 2024-09-19T09:08:20.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.647+0000 7fd38d982640 1 --2- 172.21.15.120:0/3088492501 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de751f7b40 0x55de751f6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:20.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.647+0000 7fd38c980640 1 --2- 172.21.15.120:0/3088492501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de75201370 0x55de75203760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:20.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.647+0000 7fd38d181640 1 --2- 172.21.15.120:0/3088492501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de751f65d0 0x55de751f6a10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd38406ea60 tx=0x7fd38406ea90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47b8a3d9b738f989 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:20.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.647+0000 7fd38d982640 1 --2- 172.21.15.120:0/1754174514 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd3740a3240 0x7fd3740a5700 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd3800092f0 tx=0x7fd38002a6c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:20.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.647+0000 7fd3737fe640 1 -- 172.21.15.120:0/3088492501 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd384020070 con 0x55de751f65d0 2024-09-19T09:08:20.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.647+0000 7fd3737fe640 1 -- 172.21.15.120:0/3088492501 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd38409bc10 con 0x55de751f65d0 2024-09-19T09:08:20.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.647+0000 7fd3737fe640 1 -- 172.21.15.120:0/3088492501 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd384098280 con 0x55de751f65d0 2024-09-19T09:08:20.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.648+0000 7fd3917be740 1 -- 172.21.15.120:0/3088492501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de751f65d0 msgr2=0x55de751f6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:20.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.648+0000 7fd3917be740 1 --2- 172.21.15.120:0/3088492501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de751f65d0 0x55de751f6a10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd38406ea60 tx=0x7fd38406ea90 comp rx=0 tx=0).stop 2024-09-19T09:08:20.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.648+0000 7fd3917be740 1 -- 172.21.15.120:0/3088492501 shutdown_connections 2024-09-19T09:08:20.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.648+0000 7fd3917be740 1 --2- 172.21.15.120:0/3088492501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de75201370 0x55de75203760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.648+0000 7fd3917be740 1 --2- 172.21.15.120:0/3088492501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de751f65d0 0x55de751f6a10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.648+0000 7fd3917be740 1 --2- 172.21.15.120:0/3088492501 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de751f7b40 0x55de751f6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.648+0000 7fd3917be740 1 -- 172.21.15.120:0/3088492501 >> 172.21.15.120:0/3088492501 conn(0x55de751bf310 msgr2=0x55de751f2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:20.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.648+0000 7fd3917be740 1 -- 172.21.15.120:0/3088492501 shutdown_connections 2024-09-19T09:08:20.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.648+0000 7fd3917be740 1 -- 172.21.15.120:0/3088492501 wait complete. 2024-09-19T09:08:20.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.648+0000 7fd3917be740 1 -- start start 2024-09-19T09:08:20.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.649+0000 7fd3917be740 1 Processor -- start 2024-09-19T09:08:20.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.649+0000 7fd3917be740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de751f65d0 0x55de75200d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:20.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.649+0000 7fd3917be740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de751f7b40 0x55de751fe040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:20.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.649+0000 7fd3917be740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de75201370 0x55de751fe650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:20.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.649+0000 7fd3917be740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55de75204010 con 0x55de751f65d0 2024-09-19T09:08:20.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.649+0000 7fd3917be740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55de75117060 con 0x55de751f7b40 2024-09-19T09:08:20.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.649+0000 7fd3917be740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55de75204190 con 0x55de75201370 2024-09-19T09:08:20.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.649+0000 7fd3917be740 10 client.0 _ll_register_callbacks cb 0x55de74e6a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:08:20.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.649+0000 7fd38d181640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de751f65d0 0x55de75200d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:20.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.649+0000 7fd38d181640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de751f65d0 0x55de75200d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49866/0 (socket says 172.21.15.120:49866) 2024-09-19T09:08:20.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.649+0000 7fd38d181640 1 -- 172.21.15.120:0/2202913917 learned_addr learned my addr 172.21.15.120:0/2202913917 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:20.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.649+0000 7fd38c980640 1 --2- 172.21.15.120:0/2202913917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de751f7b40 0x55de751fe040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:20.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.649+0000 7fd38d982640 1 --2- 172.21.15.120:0/2202913917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de75201370 0x55de751fe650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:20.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.649+0000 7fd38d181640 1 -- 172.21.15.120:0/2202913917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de75201370 msgr2=0x55de751fe650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:20.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.649+0000 7fd38d181640 1 --2- 172.21.15.120:0/2202913917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de75201370 0x55de751fe650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.649+0000 7fd38d181640 1 -- 172.21.15.120:0/2202913917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de751f7b40 msgr2=0x55de751fe040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:20.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.649+0000 7fd38d181640 1 --2- 172.21.15.120:0/2202913917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de751f7b40 0x55de751fe040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.649+0000 7fd38d181640 1 -- 172.21.15.120:0/2202913917 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de74ecd3c0 con 0x55de751f65d0 2024-09-19T09:08:20.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.650+0000 7fd38d181640 1 --2- 172.21.15.120:0/2202913917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de751f65d0 0x55de75200d90 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fd3840a42a0 tx=0x7fd384097790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ff2f4e24f942df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:20.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.650+0000 7fd370ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:08:20.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.650+0000 7fd370ff9640 1 -- 172.21.15.120:0/2202913917 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd384020030 con 0x55de751f65d0 2024-09-19T09:08:20.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.650+0000 7fd370ff9640 1 -- 172.21.15.120:0/2202913917 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fd384098280 con 0x55de751f65d0 2024-09-19T09:08:20.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.650+0000 7fd3917be740 1 -- 172.21.15.120:0/2202913917 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd38409bc10 con 0x55de751f65d0 2024-09-19T09:08:20.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.650+0000 7fd370ff9640 1 -- 172.21.15.120:0/2202913917 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd38409ff10 con 0x55de751f65d0 2024-09-19T09:08:20.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.650+0000 7fd3917be740 10 client.17358 fetch_fsmap learned FSMap version 30 2024-09-19T09:08:20.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.650+0000 7fd3917be740 1 -- 172.21.15.120:0/2202913917 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55de7520fae0 con 0x55de751f65d0 2024-09-19T09:08:20.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.650+0000 7fd370ff9640 1 -- 172.21.15.120:0/2202913917 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd38409bc10 con 0x55de751f65d0 2024-09-19T09:08:20.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.650+0000 7fd370ff9640 1 -- 172.21.15.120:0/2202913917 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd3840662b0 con 0x55de751f65d0 2024-09-19T09:08:20.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.650+0000 7fd3917be740 10 client.17358 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:08:20.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.650+0000 7fd3917be740 10 client.17358 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:08:20.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.650+0000 7fd3917be740 20 client.17358 populate_metadata read hostname 'smithi120' 2024-09-19T09:08:20.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.651+0000 7fd3917be740 1 --2- 172.21.15.120:0/2202913917 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55de75211170 0x55de752135a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:20.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.651+0000 7fd3917be740 4 client.17358 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:08:20.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.651+0000 7fd3917be740 1 -- 172.21.15.120:0/2202913917 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de75169860 con 0x55de75211170 2024-09-19T09:08:20.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.651+0000 7fd38c980640 1 --2- 172.21.15.120:0/2202913917 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55de75211170 0x55de752135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:20.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.651+0000 7fd370ff9640 10 client.17358 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:08:20.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.651+0000 7fd38c980640 1 --2- 172.21.15.120:0/2202913917 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55de75211170 0x55de752135a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a25aef2af6aed728 server_cookie=523af926728c1393 in_seq=0 out_seq=0 2024-09-19T09:08:20.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.652+0000 7fd370ff9640 1 -- 172.21.15.120:0/2202913917 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55de75169860 con 0x55de75211170 2024-09-19T09:08:20.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.652+0000 7fd370ff9640 10 client.17358 handle_command_reply: tid=0 2024-09-19T09:08:20.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.652+0000 7fd3917be740 1 client.17358 shutdown 2024-09-19T09:08:20.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.652+0000 7fd3917be740 1 -- 172.21.15.120:0/2202913917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de751f65d0 msgr2=0x55de75200d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:20.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.652+0000 7fd3917be740 1 --2- 172.21.15.120:0/2202913917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de751f65d0 0x55de75200d90 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fd3840a42a0 tx=0x7fd384097790 comp rx=0 tx=0).stop 2024-09-19T09:08:20.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.652+0000 7fd3917be740 1 -- 172.21.15.120:0/2202913917 shutdown_connections 2024-09-19T09:08:20.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.652+0000 7fd3917be740 1 --2- 172.21.15.120:0/2202913917 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55de75211170 0x55de752135a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.653+0000 7fd3917be740 1 --2- 172.21.15.120:0/2202913917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de75201370 0x55de751fe650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.653+0000 7fd370ff9640 0 client.17358 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:08:20.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.653+0000 7fd3917be740 1 --2- 172.21.15.120:0/2202913917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de751f7b40 0x55de751fe040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.653+0000 7fd3917be740 1 --2- 172.21.15.120:0/2202913917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de751f65d0 0x55de75200d90 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.653+0000 7fd3917be740 1 -- 172.21.15.120:0/2202913917 >> 172.21.15.120:0/2202913917 conn(0x55de751bf310 msgr2=0x55de751f8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:20.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.653+0000 7fd3917be740 1 -- 172.21.15.120:0/2202913917 shutdown_connections 2024-09-19T09:08:20.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.653+0000 7fd3917be740 1 -- 172.21.15.120:0/2202913917 wait complete. 2024-09-19T09:08:20.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.653+0000 7fd3917be740 20 client.17358 trim_cache size 0 max 16384 2024-09-19T09:08:20.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.672+0000 7fd3917be740 1 Processor -- start 2024-09-19T09:08:20.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.673+0000 7fd3917be740 1 -- start start 2024-09-19T09:08:20.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.673+0000 7fd3917be740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de751f65d0 0x55de7520d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:20.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.673+0000 7fd3917be740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de751f7b40 0x55de751f2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:20.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.673+0000 7fd3917be740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de75201370 0x55de75218bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:20.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.673+0000 7fd3917be740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881b3ca0 con 0x55de751f65d0 2024-09-19T09:08:20.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.673+0000 7fd3917be740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881b57e0 con 0x55de75201370 2024-09-19T09:08:20.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.673+0000 7fd3917be740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881b4370 con 0x55de751f7b40 2024-09-19T09:08:20.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.673+0000 7fd38d181640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de751f65d0 0x55de7520d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:20.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.673+0000 7fd38d181640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de751f65d0 0x55de7520d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49876/0 (socket says 172.21.15.120:49876) 2024-09-19T09:08:20.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.673+0000 7fd38d181640 1 -- 172.21.15.120:0/1002132830 learned_addr learned my addr 172.21.15.120:0/1002132830 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:20.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.673+0000 7fd38c980640 1 --2- 172.21.15.120:0/1002132830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de751f7b40 0x55de751f2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:20.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.673+0000 7fd38d982640 1 --2- 172.21.15.120:0/1002132830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de75201370 0x55de75218bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:20.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.673+0000 7fd38d181640 1 -- 172.21.15.120:0/1002132830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de751f7b40 msgr2=0x55de751f2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:20.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.673+0000 7fd38d181640 1 --2- 172.21.15.120:0/1002132830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de751f7b40 0x55de751f2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.673+0000 7fd38d181640 1 -- 172.21.15.120:0/1002132830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de75201370 msgr2=0x55de75218bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:20.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.673+0000 7fd38d181640 1 --2- 172.21.15.120:0/1002132830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de75201370 0x55de75218bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.673+0000 7fd38d181640 1 -- 172.21.15.120:0/1002132830 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de751fe9c0 con 0x55de751f65d0 2024-09-19T09:08:20.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.673+0000 7fd38c980640 1 --2- 172.21.15.120:0/1002132830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de751f7b40 0x55de751f2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:20.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.674+0000 7fd38d181640 1 --2- 172.21.15.120:0/1002132830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de751f65d0 0x55de7520d430 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fd38409b910 tx=0x7fd38400b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eacbcd2428fb40f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:20.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.674+0000 7fd372ffd640 1 -- 172.21.15.120:0/1002132830 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd384020070 con 0x55de751f65d0 2024-09-19T09:08:20.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.674+0000 7fd372ffd640 1 -- 172.21.15.120:0/1002132830 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd384086410 con 0x55de751f65d0 2024-09-19T09:08:20.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.674+0000 7fd372ffd640 1 -- 172.21.15.120:0/1002132830 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38409a6e0 con 0x55de751f65d0 2024-09-19T09:08:20.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.675+0000 7fd3917be740 1 -- 172.21.15.120:0/1002132830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de751f65d0 msgr2=0x55de7520d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:20.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.675+0000 7fd3917be740 1 --2- 172.21.15.120:0/1002132830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de751f65d0 0x55de7520d430 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fd38409b910 tx=0x7fd38400b660 comp rx=0 tx=0).stop 2024-09-19T09:08:20.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.675+0000 7fd3917be740 1 -- 172.21.15.120:0/1002132830 shutdown_connections 2024-09-19T09:08:20.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.675+0000 7fd3917be740 1 --2- 172.21.15.120:0/1002132830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de75201370 0x55de75218bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.675+0000 7fd3917be740 1 --2- 172.21.15.120:0/1002132830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de751f7b40 0x55de751f2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.675+0000 7fd3917be740 1 --2- 172.21.15.120:0/1002132830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de751f65d0 0x55de7520d430 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.675+0000 7fd3917be740 1 -- 172.21.15.120:0/1002132830 >> 172.21.15.120:0/1002132830 conn(0x55de751bf310 msgr2=0x55de751f2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:20.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.675+0000 7fd3917be740 1 -- 172.21.15.120:0/1002132830 shutdown_connections 2024-09-19T09:08:20.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.675+0000 7fd3917be740 1 -- 172.21.15.120:0/1002132830 wait complete. 2024-09-19T09:08:20.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.675+0000 7fd3917be740 1 -- start start 2024-09-19T09:08:20.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.675+0000 7fd3917be740 1 Processor -- start 2024-09-19T09:08:20.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.676+0000 7fd3917be740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de751f65d0 0x55de75219110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:20.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.676+0000 7fd3917be740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de751f7b40 0x55de75216460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:20.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.676+0000 7fd3917be740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de75201370 0x55de752169a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:20.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.676+0000 7fd3917be740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881b5110 con 0x55de751f7b40 2024-09-19T09:08:20.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.676+0000 7fd3917be740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881b4a40 con 0x55de75201370 2024-09-19T09:08:20.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.676+0000 7fd3917be740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881b1c60 con 0x55de751f65d0 2024-09-19T09:08:20.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.676+0000 7fd3917be740 10 client.0 _ll_register_callbacks cb 0x7fd378005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:08:20.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.676+0000 7fd38c980640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de751f7b40 0x55de75216460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:20.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.676+0000 7fd38c980640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de751f7b40 0x55de75216460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49888/0 (socket says 172.21.15.120:49888) 2024-09-19T09:08:20.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.676+0000 7fd38c980640 1 -- 172.21.15.120:0/3781369321 learned_addr learned my addr 172.21.15.120:0/3781369321 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:20.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.676+0000 7fd38d982640 1 --2- 172.21.15.120:0/3781369321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de75201370 0x55de752169a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:20.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.676+0000 7fd38d181640 1 --2- 172.21.15.120:0/3781369321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de751f65d0 0x55de75219110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:20.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.676+0000 7fd38c980640 1 -- 172.21.15.120:0/3781369321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de751f65d0 msgr2=0x55de75219110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:20.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.676+0000 7fd38c980640 1 --2- 172.21.15.120:0/3781369321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de751f65d0 0x55de75219110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.676+0000 7fd38c980640 1 -- 172.21.15.120:0/3781369321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de75201370 msgr2=0x55de752169a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:20.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.676+0000 7fd38c980640 1 --2- 172.21.15.120:0/3781369321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de75201370 0x55de752169a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.676+0000 7fd38c980640 1 -- 172.21.15.120:0/3781369321 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de75216f60 con 0x55de751f7b40 2024-09-19T09:08:20.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.677+0000 7fd38c980640 1 --2- 172.21.15.120:0/3781369321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de751f7b40 0x55de75216460 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fd378000c00 tx=0x7fd378010a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4f649ecd743a92 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:20.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.677+0000 7fd370ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:08:20.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.677+0000 7fd370ff9640 1 -- 172.21.15.120:0/3781369321 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd378004020 con 0x55de751f7b40 2024-09-19T09:08:20.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.677+0000 7fd3917be740 1 -- 172.21.15.120:0/3781369321 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd384086410 con 0x55de751f7b40 2024-09-19T09:08:20.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.677+0000 7fd370ff9640 1 -- 172.21.15.120:0/3781369321 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fd3780041c0 con 0x55de751f7b40 2024-09-19T09:08:20.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.677+0000 7fd370ff9640 1 -- 172.21.15.120:0/3781369321 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd3780043c0 con 0x55de751f7b40 2024-09-19T09:08:20.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.677+0000 7fd3917be740 10 client.17364 fetch_fsmap learned FSMap version 30 2024-09-19T09:08:20.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.677+0000 7fd3917be740 1 -- 172.21.15.120:0/3781369321 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55de7527ba50 con 0x55de751f7b40 2024-09-19T09:08:20.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.678+0000 7fd370ff9640 1 -- 172.21.15.120:0/3781369321 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd384086410 con 0x55de751f7b40 2024-09-19T09:08:20.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.678+0000 7fd370ff9640 1 -- 172.21.15.120:0/3781369321 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd378027470 con 0x55de751f7b40 2024-09-19T09:08:20.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.678+0000 7fd3917be740 10 client.17364 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:08:20.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.678+0000 7fd3917be740 10 client.17364 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:08:20.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.678+0000 7fd3917be740 20 client.17364 populate_metadata read hostname 'smithi120' 2024-09-19T09:08:20.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.679+0000 7fd3917be740 1 --2- 172.21.15.120:0/3781369321 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55de75211170 0x55de7527e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:20.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.679+0000 7fd3917be740 4 client.17364 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:08:20.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.679+0000 7fd3917be740 1 -- 172.21.15.120:0/3781369321 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55de75163d70 con 0x55de75211170 2024-09-19T09:08:20.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.679+0000 7fd38d181640 1 --2- 172.21.15.120:0/3781369321 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55de75211170 0x55de7527e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:20.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.679+0000 7fd38d181640 1 --2- 172.21.15.120:0/3781369321 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55de75211170 0x55de7527e240 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4f76563b06567629 server_cookie=9d7a238985800f52 in_seq=0 out_seq=0 2024-09-19T09:08:20.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.679+0000 7fd370ff9640 10 client.17364 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:08:20.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.679+0000 7fd370ff9640 1 -- 172.21.15.120:0/3781369321 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55de75163d70 con 0x55de75211170 2024-09-19T09:08:20.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.679+0000 7fd370ff9640 10 client.17364 handle_command_reply: tid=0 2024-09-19T09:08:20.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.680+0000 7fd3917be740 1 client.17364 shutdown 2024-09-19T09:08:20.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.680+0000 7fd3917be740 1 -- 172.21.15.120:0/3781369321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de751f7b40 msgr2=0x55de75216460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:20.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.680+0000 7fd3917be740 1 --2- 172.21.15.120:0/3781369321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de751f7b40 0x55de75216460 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fd378000c00 tx=0x7fd378010a00 comp rx=0 tx=0).stop 2024-09-19T09:08:20.799 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:08:20.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.680+0000 7fd3917be740 1 -- 172.21.15.120:0/3781369321 shutdown_connections 2024-09-19T09:08:20.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.680+0000 7fd3917be740 1 --2- 172.21.15.120:0/3781369321 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55de75211170 0x55de7527e240 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.680+0000 7fd3917be740 1 --2- 172.21.15.120:0/3781369321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de75201370 0x55de752169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.680+0000 7fd3917be740 1 --2- 172.21.15.120:0/3781369321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de751f7b40 0x55de75216460 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.680+0000 7fd3917be740 1 --2- 172.21.15.120:0/3781369321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de751f65d0 0x55de75219110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.680+0000 7fd370ff9640 0 client.17364 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:08:20.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.680+0000 7fd3917be740 1 -- 172.21.15.120:0/3781369321 >> 172.21.15.120:0/3781369321 conn(0x55de751bf310 msgr2=0x55de751f14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:20.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.680+0000 7fd3917be740 1 -- 172.21.15.120:0/3781369321 shutdown_connections 2024-09-19T09:08:20.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.680+0000 7fd3917be740 1 -- 172.21.15.120:0/3781369321 wait complete. 2024-09-19T09:08:20.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.680+0000 7fd3917be740 20 client.17364 trim_cache size 0 max 16384 2024-09-19T09:08:20.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.681+0000 7fd38f3e5640 1 -- 172.21.15.120:0/1754174514 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd3740a3240 msgr2=0x7fd3740a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:20.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.681+0000 7fd38f3e5640 1 --2- 172.21.15.120:0/1754174514 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd3740a3240 0x7fd3740a5700 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd3800092f0 tx=0x7fd38002a6c0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.681+0000 7fd38f3e5640 1 -- 172.21.15.120:0/1754174514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd388101de0 msgr2=0x7fd38810acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:20.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.681+0000 7fd38f3e5640 1 --2- 172.21.15.120:0/1754174514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd388101de0 0x7fd38810acb0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fd38400d980 tx=0x7fd38400de40 comp rx=0 tx=0).stop 2024-09-19T09:08:20.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.681+0000 7fd38f3e5640 1 -- 172.21.15.120:0/1754174514 shutdown_connections 2024-09-19T09:08:20.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.681+0000 7fd38f3e5640 1 --2- 172.21.15.120:0/1754174514 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd3740a3240 0x7fd3740a5700 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.681+0000 7fd38f3e5640 1 --2- 172.21.15.120:0/1754174514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd388106f10 0x7fd38810b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.681+0000 7fd38f3e5640 1 --2- 172.21.15.120:0/1754174514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd388102790 0x7fd38810b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.681+0000 7fd38f3e5640 1 --2- 172.21.15.120:0/1754174514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd388101de0 0x7fd38810acb0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:20.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.681+0000 7fd38f3e5640 1 -- 172.21.15.120:0/1754174514 >> 172.21.15.120:0/1754174514 conn(0x7fd3880fd880 msgr2=0x7fd3880fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:20.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.681+0000 7fd38f3e5640 1 -- 172.21.15.120:0/1754174514 shutdown_connections 2024-09-19T09:08:20.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:20.681+0000 7fd38f3e5640 1 -- 172.21.15.120:0/1754174514 wait complete. 2024-09-19T09:08:20.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:20 smithi120 ceph-mon[31150]: pgmap v506: 97 pgs: 97 active+clean; 28 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 585 KiB/s wr, 97 op/s 2024-09-19T09:08:21.050 INFO:tasks.workunit.client.0.smithi120.stdout: 1 399276 6.79 MB/sec execute 136 sec latency 22.738 ms 2024-09-19T09:08:21.190 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:20 smithi153 ceph-mon[35071]: pgmap v506: 97 pgs: 97 active+clean; 28 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 585 KiB/s wr, 97 op/s 2024-09-19T09:08:21.191 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:20 smithi196 ceph-mon[35150]: pgmap v506: 97 pgs: 97 active+clean; 28 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 585 KiB/s wr, 97 op/s 2024-09-19T09:08:21.601 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:08:21.861 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:08:22.050 INFO:tasks.workunit.client.0.smithi120.stdout: 1 401771 6.85 MB/sec execute 137 sec latency 88.696 ms 2024-09-19T09:08:22.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.354+0000 7f1d2bd6c640 1 -- 172.21.15.120:0/2148899138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d2410e4e0 msgr2=0x7f1d241108d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:22.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.354+0000 7f1d2bd6c640 1 --2- 172.21.15.120:0/2148899138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d2410e4e0 0x7f1d241108d0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c009ca0 tx=0x7f1d1c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.354+0000 7f1d2bd6c640 1 -- 172.21.15.120:0/2148899138 shutdown_connections 2024-09-19T09:08:22.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.354+0000 7f1d2bd6c640 1 --2- 172.21.15.120:0/2148899138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d2410e4e0 0x7f1d241108d0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.354+0000 7f1d2bd6c640 1 --2- 172.21.15.120:0/2148899138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d241012e0 0x7f1d2410dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.354+0000 7f1d2bd6c640 1 --2- 172.21.15.120:0/2148899138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d24100930 0x7f1d24100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.354+0000 7f1d2bd6c640 1 -- 172.21.15.120:0/2148899138 >> 172.21.15.120:0/2148899138 conn(0x7f1d240fc7a0 msgr2=0x7f1d240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:22.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.354+0000 7f1d2bd6c640 1 -- 172.21.15.120:0/2148899138 shutdown_connections 2024-09-19T09:08:22.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.354+0000 7f1d2bd6c640 1 -- 172.21.15.120:0/2148899138 wait complete. 2024-09-19T09:08:22.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.354+0000 7f1d2bd6c640 1 Processor -- start 2024-09-19T09:08:22.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.354+0000 7f1d2bd6c640 1 -- start start 2024-09-19T09:08:22.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.355+0000 7f1d2bd6c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d24100930 0x7f1d2419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:22.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.355+0000 7f1d2bd6c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d241012e0 0x7f1d2419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:22.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.355+0000 7f1d2bd6c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d2410e4e0 0x7f1d2419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:22.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.355+0000 7f1d2bd6c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1d24114160 con 0x7f1d241012e0 2024-09-19T09:08:22.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.355+0000 7f1d2bd6c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1d24113fe0 con 0x7f1d2410e4e0 2024-09-19T09:08:22.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.355+0000 7f1d2bd6c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1d241142e0 con 0x7f1d24100930 2024-09-19T09:08:22.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.355+0000 7f1d29307640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d241012e0 0x7f1d2419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:22.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.355+0000 7f1d2a309640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d2410e4e0 0x7f1d2419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:22.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.355+0000 7f1d29307640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d241012e0 0x7f1d2419abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46894/0 (socket says 172.21.15.120:46894) 2024-09-19T09:08:22.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.355+0000 7f1d29307640 1 -- 172.21.15.120:0/2194531857 learned_addr learned my addr 172.21.15.120:0/2194531857 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:22.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.355+0000 7f1d29b08640 1 --2- 172.21.15.120:0/2194531857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d24100930 0x7f1d2419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:22.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.355+0000 7f1d2a309640 1 -- 172.21.15.120:0/2194531857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d24100930 msgr2=0x7f1d2419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:22.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.355+0000 7f1d2a309640 1 --2- 172.21.15.120:0/2194531857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d24100930 0x7f1d2419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.355+0000 7f1d2a309640 1 -- 172.21.15.120:0/2194531857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d241012e0 msgr2=0x7f1d2419abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:22.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.355+0000 7f1d2a309640 1 --2- 172.21.15.120:0/2194531857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d241012e0 0x7f1d2419abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.355+0000 7f1d2a309640 1 -- 172.21.15.120:0/2194531857 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d2419f650 con 0x7f1d2410e4e0 2024-09-19T09:08:22.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.355+0000 7f1d29307640 1 --2- 172.21.15.120:0/2194531857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d241012e0 0x7f1d2419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:22.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.355+0000 7f1d2a309640 1 --2- 172.21.15.120:0/2194531857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d2410e4e0 0x7f1d2419ef70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c02ade0 tx=0x7f1d1c0319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:22.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.355+0000 7f1d1affd640 1 -- 172.21.15.120:0/2194531857 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c003b70 con 0x7f1d2410e4e0 2024-09-19T09:08:22.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.356+0000 7f1d1affd640 1 -- 172.21.15.120:0/2194531857 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d1c003d10 con 0x7f1d2410e4e0 2024-09-19T09:08:22.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.356+0000 7f1d2bd6c640 1 -- 172.21.15.120:0/2194531857 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d2419f8e0 con 0x7f1d2410e4e0 2024-09-19T09:08:22.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.356+0000 7f1d1affd640 1 -- 172.21.15.120:0/2194531857 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c02fc10 con 0x7f1d2410e4e0 2024-09-19T09:08:22.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.356+0000 7f1d2bd6c640 1 -- 172.21.15.120:0/2194531857 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d241a7220 con 0x7f1d2410e4e0 2024-09-19T09:08:22.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.358+0000 7f1d1affd640 1 -- 172.21.15.120:0/2194531857 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f1d1c004050 con 0x7f1d2410e4e0 2024-09-19T09:08:22.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.358+0000 7f1d2e145740 1 Processor -- start 2024-09-19T09:08:22.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.358+0000 7f1d2e145740 1 -- start start 2024-09-19T09:08:22.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.359+0000 7f1d2e145740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d0dd4bb40 0x558d0dd4a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:22.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.359+0000 7f1d1affd640 1 --2- 172.21.15.120:0/2194531857 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1d100a3240 0x7f1d100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:22.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.359+0000 7f1d2e145740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d0dd4a5d0 0x558d0dd4aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:22.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.359+0000 7f1d2e145740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d0dd55370 0x558d0dd57760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:22.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.359+0000 7f1d2e145740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558d0dcb02f0 con 0x558d0dd4a5d0 2024-09-19T09:08:22.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.359+0000 7f1d2e145740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558d0dc57cf0 con 0x558d0dd4bb40 2024-09-19T09:08:22.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.359+0000 7f1d2e145740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558d0dcac2b0 con 0x558d0dd55370 2024-09-19T09:08:22.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.359+0000 7f1d29b08640 1 --2- 172.21.15.120:0/2194531857 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1d100a3240 0x7f1d100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:22.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.359+0000 7f1d2a309640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d0dd55370 0x558d0dd57760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:22.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.359+0000 7f1d29307640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d0dd4bb40 0x558d0dd4a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:22.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.359+0000 7f1d29307640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d0dd4bb40 0x558d0dd4a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:39448/0 (socket says 172.21.15.120:39448) 2024-09-19T09:08:22.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.359+0000 7f1d29307640 1 -- 172.21.15.120:0/3319129231 learned_addr learned my addr 172.21.15.120:0/3319129231 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:22.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.359+0000 7f1d29307640 1 --2- 172.21.15.120:0/3319129231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d0dd4a5d0 0x558d0dd4aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:22.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.359+0000 7f1d1affd640 1 -- 172.21.15.120:0/2194531857 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f1d1c0f0960 con 0x7f1d2410e4e0 2024-09-19T09:08:22.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.359+0000 7f1d29307640 1 -- 172.21.15.120:0/3319129231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d0dd55370 msgr2=0x558d0dd57760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:22.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.359+0000 7f1d29307640 1 --2- 172.21.15.120:0/3319129231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d0dd55370 0x558d0dd57760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.359+0000 7f1d29307640 1 -- 172.21.15.120:0/3319129231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d0dd4a5d0 msgr2=0x558d0dd4aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:22.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.359+0000 7f1d29307640 1 --2- 172.21.15.120:0/3319129231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d0dd4a5d0 0x558d0dd4aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.359+0000 7f1d29307640 1 -- 172.21.15.120:0/3319129231 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d0dc3c3a0 con 0x558d0dd4bb40 2024-09-19T09:08:22.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.359+0000 7f1d2a309640 1 --2- 172.21.15.120:0/3319129231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d0dd55370 0x558d0dd57760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:22.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.359+0000 7f1d29b08640 1 --2- 172.21.15.120:0/2194531857 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1d100a3240 0x7f1d100a5700 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1d14009bd0 tx=0x7f1d14005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:22.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.360+0000 7f1d29307640 1 --2- 172.21.15.120:0/3319129231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d0dd4bb40 0x558d0dd4a090 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1d20009620 tx=0x7f1d20009590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8ecaf25f20ed40b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:22.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.360+0000 7f1d0ffff640 1 -- 172.21.15.120:0/3319129231 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2000fc50 con 0x558d0dd4bb40 2024-09-19T09:08:22.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.360+0000 7f1d0ffff640 1 -- 172.21.15.120:0/3319129231 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d2000fdf0 con 0x558d0dd4bb40 2024-09-19T09:08:22.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.360+0000 7f1d0ffff640 1 -- 172.21.15.120:0/3319129231 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2000e6b0 con 0x558d0dd4bb40 2024-09-19T09:08:22.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.360+0000 7f1d2e145740 1 -- 172.21.15.120:0/3319129231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d0dd4bb40 msgr2=0x558d0dd4a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:22.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.360+0000 7f1d2e145740 1 --2- 172.21.15.120:0/3319129231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d0dd4bb40 0x558d0dd4a090 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1d20009620 tx=0x7f1d20009590 comp rx=0 tx=0).stop 2024-09-19T09:08:22.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.360+0000 7f1d2e145740 1 -- 172.21.15.120:0/3319129231 shutdown_connections 2024-09-19T09:08:22.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.360+0000 7f1d2e145740 1 --2- 172.21.15.120:0/3319129231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d0dd55370 0x558d0dd57760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.360+0000 7f1d2e145740 1 --2- 172.21.15.120:0/3319129231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d0dd4a5d0 0x558d0dd4aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.360+0000 7f1d2e145740 1 --2- 172.21.15.120:0/3319129231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d0dd4bb40 0x558d0dd4a090 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.360+0000 7f1d2e145740 1 -- 172.21.15.120:0/3319129231 >> 172.21.15.120:0/3319129231 conn(0x558d0dd13310 msgr2=0x558d0dd46c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:22.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.361+0000 7f1d2e145740 1 -- 172.21.15.120:0/3319129231 shutdown_connections 2024-09-19T09:08:22.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.361+0000 7f1d2e145740 1 -- 172.21.15.120:0/3319129231 wait complete. 2024-09-19T09:08:22.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.361+0000 7f1d2e145740 1 -- start start 2024-09-19T09:08:22.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.361+0000 7f1d2e145740 1 Processor -- start 2024-09-19T09:08:22.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.361+0000 7f1d2e145740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d0dd4a5d0 0x558d0dd54d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:22.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.361+0000 7f1d2e145740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d0dd4bb40 0x558d0dd52010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:22.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.361+0000 7f1d2e145740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d0dd55370 0x558d0dd52600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:22.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.361+0000 7f1d2e145740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558d0dd58070 con 0x558d0dd4bb40 2024-09-19T09:08:22.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.361+0000 7f1d2e145740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558d0dc6b060 con 0x558d0dd55370 2024-09-19T09:08:22.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.361+0000 7f1d2e145740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558d0dd581f0 con 0x558d0dd4a5d0 2024-09-19T09:08:22.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.362+0000 7f1d2e145740 10 client.0 _ll_register_callbacks cb 0x558d0d9be8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:08:22.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.362+0000 7f1d29307640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d0dd4bb40 0x558d0dd52010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:22.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.362+0000 7f1d29b08640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d0dd4a5d0 0x558d0dd54d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:22.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.362+0000 7f1d29307640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d0dd4bb40 0x558d0dd52010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46918/0 (socket says 172.21.15.120:46918) 2024-09-19T09:08:22.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.362+0000 7f1d29307640 1 -- 172.21.15.120:0/1422982871 learned_addr learned my addr 172.21.15.120:0/1422982871 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:22.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.362+0000 7f1d2a309640 1 --2- 172.21.15.120:0/1422982871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d0dd55370 0x558d0dd52600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:22.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.362+0000 7f1d29307640 1 -- 172.21.15.120:0/1422982871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d0dd4a5d0 msgr2=0x558d0dd54d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:22.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.362+0000 7f1d29307640 1 --2- 172.21.15.120:0/1422982871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d0dd4a5d0 0x558d0dd54d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.362+0000 7f1d29307640 1 -- 172.21.15.120:0/1422982871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d0dd55370 msgr2=0x558d0dd52600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:22.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.362+0000 7f1d29307640 1 --2- 172.21.15.120:0/1422982871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d0dd55370 0x558d0dd52600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.362+0000 7f1d29307640 1 -- 172.21.15.120:0/1422982871 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d0da213c0 con 0x558d0dd4bb40 2024-09-19T09:08:22.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.362+0000 7f1d29b08640 1 --2- 172.21.15.120:0/1422982871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d0dd4a5d0 0x558d0dd54d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:22.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.362+0000 7f1d29307640 1 --2- 172.21.15.120:0/1422982871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d0dd4bb40 0x558d0dd52010 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f1d20009560 tx=0x7f1d200054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1619248440f4b41f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:22.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.362+0000 7f1d0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:08:22.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.362+0000 7f1d0d7fa640 1 -- 172.21.15.120:0/1422982871 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20004220 con 0x558d0dd4bb40 2024-09-19T09:08:22.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.362+0000 7f1d0d7fa640 1 -- 172.21.15.120:0/1422982871 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f1d200043c0 con 0x558d0dd4bb40 2024-09-19T09:08:22.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.362+0000 7f1d2e145740 1 -- 172.21.15.120:0/1422982871 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d2000fdf0 con 0x558d0dd4bb40 2024-09-19T09:08:22.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.362+0000 7f1d0d7fa640 1 -- 172.21.15.120:0/1422982871 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1d2000fc20 con 0x558d0dd4bb40 2024-09-19T09:08:22.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.362+0000 7f1d2e145740 10 client.17376 fetch_fsmap learned FSMap version 30 2024-09-19T09:08:22.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.362+0000 7f1d2e145740 1 -- 172.21.15.120:0/1422982871 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558d0dd63b10 con 0x558d0dd4bb40 2024-09-19T09:08:22.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.363+0000 7f1d0d7fa640 1 -- 172.21.15.120:0/1422982871 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d2000fdf0 con 0x558d0dd4bb40 2024-09-19T09:08:22.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.363+0000 7f1d0d7fa640 1 -- 172.21.15.120:0/1422982871 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1d2001d020 con 0x558d0dd4bb40 2024-09-19T09:08:22.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.363+0000 7f1d2e145740 10 client.17376 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:08:22.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.363+0000 7f1d2e145740 10 client.17376 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:08:22.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.363+0000 7f1d2e145740 20 client.17376 populate_metadata read hostname 'smithi120' 2024-09-19T09:08:22.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.363+0000 7f1d2e145740 1 --2- 172.21.15.120:0/1422982871 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558d0dd64e60 0x558d0dd67250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:22.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.363+0000 7f1d2e145740 4 client.17376 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:08:22.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.363+0000 7f1d2e145740 1 -- 172.21.15.120:0/1422982871 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d0dcbd860 con 0x558d0dd64e60 2024-09-19T09:08:22.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.363+0000 7f1d29b08640 1 --2- 172.21.15.120:0/1422982871 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558d0dd64e60 0x558d0dd67250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:22.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.364+0000 7f1d29b08640 1 --2- 172.21.15.120:0/1422982871 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558d0dd64e60 0x558d0dd67250 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=457ceb8582a2b2f7 server_cookie=3358ae34b8c8324a in_seq=0 out_seq=0 2024-09-19T09:08:22.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.364+0000 7f1d0d7fa640 10 client.17376 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:08:22.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.364+0000 7f1d0d7fa640 1 -- 172.21.15.120:0/1422982871 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558d0dcbd860 con 0x558d0dd64e60 2024-09-19T09:08:22.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.364+0000 7f1d0d7fa640 10 client.17376 handle_command_reply: tid=0 2024-09-19T09:08:22.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.364+0000 7f1d2e145740 1 client.17376 shutdown 2024-09-19T09:08:22.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.365+0000 7f1d2e145740 1 -- 172.21.15.120:0/1422982871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d0dd4bb40 msgr2=0x558d0dd52010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:22.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.365+0000 7f1d2e145740 1 --2- 172.21.15.120:0/1422982871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d0dd4bb40 0x558d0dd52010 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f1d20009560 tx=0x7f1d200054a0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.365+0000 7f1d2e145740 1 -- 172.21.15.120:0/1422982871 shutdown_connections 2024-09-19T09:08:22.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.365+0000 7f1d2e145740 1 --2- 172.21.15.120:0/1422982871 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558d0dd64e60 0x558d0dd67250 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.365+0000 7f1d2e145740 1 --2- 172.21.15.120:0/1422982871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d0dd55370 0x558d0dd52600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.365+0000 7f1d2e145740 1 --2- 172.21.15.120:0/1422982871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d0dd4bb40 0x558d0dd52010 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.365+0000 7f1d2e145740 1 --2- 172.21.15.120:0/1422982871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d0dd4a5d0 0x558d0dd54d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.365+0000 7f1d0d7fa640 0 client.17376 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:08:22.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.365+0000 7f1d2e145740 1 -- 172.21.15.120:0/1422982871 >> 172.21.15.120:0/1422982871 conn(0x558d0dd13310 msgr2=0x558d0dd4c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:22.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.365+0000 7f1d2e145740 1 -- 172.21.15.120:0/1422982871 shutdown_connections 2024-09-19T09:08:22.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.365+0000 7f1d2e145740 1 -- 172.21.15.120:0/1422982871 wait complete. 2024-09-19T09:08:22.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.365+0000 7f1d2e145740 20 client.17376 trim_cache size 0 max 16384 2024-09-19T09:08:22.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.379+0000 7f1d2e145740 1 Processor -- start 2024-09-19T09:08:22.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.379+0000 7f1d2e145740 1 -- start start 2024-09-19T09:08:22.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.379+0000 7f1d2e145740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d0dd4a5d0 0x558d0dd53f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:22.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.379+0000 7f1d2e145740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d0dd4bb40 0x558d0dd4d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:22.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.379+0000 7f1d2e145740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d0dd55370 0x558d0dd4d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:22.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.379+0000 7f1d2e145740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1d241af780 con 0x558d0dd4bb40 2024-09-19T09:08:22.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.379+0000 7f1d2e145740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1d241b12c0 con 0x558d0dd55370 2024-09-19T09:08:22.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.379+0000 7f1d2e145740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1d241afe50 con 0x558d0dd4a5d0 2024-09-19T09:08:22.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.379+0000 7f1d29307640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d0dd4bb40 0x558d0dd4d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:22.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.379+0000 7f1d29307640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d0dd4bb40 0x558d0dd4d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46926/0 (socket says 172.21.15.120:46926) 2024-09-19T09:08:22.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.379+0000 7f1d29307640 1 -- 172.21.15.120:0/3120621105 learned_addr learned my addr 172.21.15.120:0/3120621105 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:22.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.379+0000 7f1d29b08640 1 --2- 172.21.15.120:0/3120621105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d0dd4a5d0 0x558d0dd53f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:22.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.379+0000 7f1d2a309640 1 --2- 172.21.15.120:0/3120621105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d0dd55370 0x558d0dd4d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:22.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.380+0000 7f1d29307640 1 -- 172.21.15.120:0/3120621105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d0dd4a5d0 msgr2=0x558d0dd53f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:22.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.380+0000 7f1d29307640 1 --2- 172.21.15.120:0/3120621105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d0dd4a5d0 0x558d0dd53f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.380+0000 7f1d29307640 1 -- 172.21.15.120:0/3120621105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d0dd55370 msgr2=0x558d0dd4d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:22.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.380+0000 7f1d29307640 1 --2- 172.21.15.120:0/3120621105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d0dd55370 0x558d0dd4d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.380+0000 7f1d29307640 1 -- 172.21.15.120:0/3120621105 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d0dd57ec0 con 0x558d0dd4bb40 2024-09-19T09:08:22.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.380+0000 7f1d29307640 1 --2- 172.21.15.120:0/3120621105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d0dd4bb40 0x558d0dd4d120 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f1d200115c0 tx=0x7f1d2001f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b278d5776d8fcb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:22.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.380+0000 7f1d0f7fe640 1 -- 172.21.15.120:0/3120621105 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2000fdc0 con 0x558d0dd4bb40 2024-09-19T09:08:22.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.380+0000 7f1d0f7fe640 1 -- 172.21.15.120:0/3120621105 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d20003d10 con 0x558d0dd4bb40 2024-09-19T09:08:22.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.380+0000 7f1d0f7fe640 1 -- 172.21.15.120:0/3120621105 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20011320 con 0x558d0dd4bb40 2024-09-19T09:08:22.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.381+0000 7f1d2e145740 1 -- 172.21.15.120:0/3120621105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d0dd4bb40 msgr2=0x558d0dd4d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:22.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.381+0000 7f1d2e145740 1 --2- 172.21.15.120:0/3120621105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d0dd4bb40 0x558d0dd4d120 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f1d200115c0 tx=0x7f1d2001f440 comp rx=0 tx=0).stop 2024-09-19T09:08:22.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.381+0000 7f1d2e145740 1 -- 172.21.15.120:0/3120621105 shutdown_connections 2024-09-19T09:08:22.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.381+0000 7f1d2e145740 1 --2- 172.21.15.120:0/3120621105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d0dd55370 0x558d0dd4d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.381+0000 7f1d2e145740 1 --2- 172.21.15.120:0/3120621105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d0dd4bb40 0x558d0dd4d120 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.381+0000 7f1d2e145740 1 --2- 172.21.15.120:0/3120621105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d0dd4a5d0 0x558d0dd53f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.381+0000 7f1d2e145740 1 -- 172.21.15.120:0/3120621105 >> 172.21.15.120:0/3120621105 conn(0x558d0dd13310 msgr2=0x558d0dd53870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:22.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.381+0000 7f1d2e145740 1 -- 172.21.15.120:0/3120621105 shutdown_connections 2024-09-19T09:08:22.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.381+0000 7f1d2e145740 1 -- 172.21.15.120:0/3120621105 wait complete. 2024-09-19T09:08:22.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.381+0000 7f1d2e145740 1 -- start start 2024-09-19T09:08:22.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.381+0000 7f1d2e145740 1 Processor -- start 2024-09-19T09:08:22.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.382+0000 7f1d2e145740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d0dd4a5d0 0x558d0dd5d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:22.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.382+0000 7f1d2e145740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d0dd4bb40 0x558d0dd5dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:22.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.382+0000 7f1d2e145740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d0dd55370 0x558d0dd4ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:22.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.382+0000 7f1d2e145740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1d241b0bf0 con 0x558d0dd4bb40 2024-09-19T09:08:22.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.382+0000 7f1d2e145740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1d241b0520 con 0x558d0dd4a5d0 2024-09-19T09:08:22.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.382+0000 7f1d2e145740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1d241ad740 con 0x558d0dd55370 2024-09-19T09:08:22.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.382+0000 7f1d2e145740 10 client.0 _ll_register_callbacks cb 0x7f1d20011710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:08:22.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.382+0000 7f1d29307640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d0dd4bb40 0x558d0dd5dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:22.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.382+0000 7f1d29b08640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d0dd4a5d0 0x558d0dd5d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:22.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.382+0000 7f1d29307640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d0dd4bb40 0x558d0dd5dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46940/0 (socket says 172.21.15.120:46940) 2024-09-19T09:08:22.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.382+0000 7f1d2a309640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d0dd55370 0x558d0dd4ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:22.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.382+0000 7f1d29307640 1 -- 172.21.15.120:0/3851057694 learned_addr learned my addr 172.21.15.120:0/3851057694 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:22.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.382+0000 7f1d29307640 1 -- 172.21.15.120:0/3851057694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d0dd55370 msgr2=0x558d0dd4ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:22.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.382+0000 7f1d29307640 1 --2- 172.21.15.120:0/3851057694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d0dd55370 0x558d0dd4ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.382+0000 7f1d29307640 1 -- 172.21.15.120:0/3851057694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d0dd4a5d0 msgr2=0x558d0dd5d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:22.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.382+0000 7f1d29307640 1 --2- 172.21.15.120:0/3851057694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d0dd4a5d0 0x558d0dd5d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.382+0000 7f1d29307640 1 -- 172.21.15.120:0/3851057694 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d0dd4f3f0 con 0x558d0dd4bb40 2024-09-19T09:08:22.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.382+0000 7f1d2a309640 1 --2- 172.21.15.120:0/3851057694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d0dd55370 0x558d0dd4ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:22.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.382+0000 7f1d29b08640 1 --2- 172.21.15.120:0/3851057694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d0dd4a5d0 0x558d0dd5d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:22.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.382+0000 7f1d29307640 1 --2- 172.21.15.120:0/3851057694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d0dd4bb40 0x558d0dd5dd80 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f1d20011960 tx=0x7f1d20003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f98ba9c29d868 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:22.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.382+0000 7f1d0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:08:22.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.383+0000 7f1d0d7fa640 1 -- 172.21.15.120:0/3851057694 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20040020 con 0x558d0dd4bb40 2024-09-19T09:08:22.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.383+0000 7f1d0d7fa640 1 -- 172.21.15.120:0/3851057694 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f1d200110a0 con 0x558d0dd4bb40 2024-09-19T09:08:22.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.383+0000 7f1d2e145740 1 -- 172.21.15.120:0/3851057694 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d20003d10 con 0x558d0dd4bb40 2024-09-19T09:08:22.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.383+0000 7f1d0d7fa640 1 -- 172.21.15.120:0/3851057694 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1d200056b0 con 0x558d0dd4bb40 2024-09-19T09:08:22.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.383+0000 7f1d2e145740 10 client.17382 fetch_fsmap learned FSMap version 30 2024-09-19T09:08:22.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.383+0000 7f1d2e145740 1 -- 172.21.15.120:0/3851057694 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558d0ddcf6a0 con 0x558d0dd4bb40 2024-09-19T09:08:22.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.383+0000 7f1d0d7fa640 1 -- 172.21.15.120:0/3851057694 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d20003d10 con 0x558d0dd4bb40 2024-09-19T09:08:22.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.383+0000 7f1d0d7fa640 1 -- 172.21.15.120:0/3851057694 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1d2001d070 con 0x558d0dd4bb40 2024-09-19T09:08:22.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.383+0000 7f1d2e145740 10 client.17382 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:08:22.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.383+0000 7f1d2e145740 10 client.17382 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:08:22.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.383+0000 7f1d2e145740 20 client.17382 populate_metadata read hostname 'smithi120' 2024-09-19T09:08:22.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.383+0000 7f1d2e145740 1 --2- 172.21.15.120:0/3851057694 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558d0ddcfd00 0x558d0ddd2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:22.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.383+0000 7f1d2e145740 4 client.17382 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:08:22.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.383+0000 7f1d2e145740 1 -- 172.21.15.120:0/3851057694 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558d0dcb7d70 con 0x558d0ddcfd00 2024-09-19T09:08:22.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.383+0000 7f1d29b08640 1 --2- 172.21.15.120:0/3851057694 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558d0ddcfd00 0x558d0ddd2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:22.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.384+0000 7f1d0d7fa640 10 client.17382 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:08:22.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.384+0000 7f1d29b08640 1 --2- 172.21.15.120:0/3851057694 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558d0ddcfd00 0x558d0ddd2120 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2dd7cefa2f4c7496 server_cookie=43421f86473ffa52 in_seq=0 out_seq=0 2024-09-19T09:08:22.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.384+0000 7f1d0d7fa640 1 -- 172.21.15.120:0/3851057694 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558d0dcb7d70 con 0x558d0ddcfd00 2024-09-19T09:08:22.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.384+0000 7f1d0d7fa640 10 client.17382 handle_command_reply: tid=0 2024-09-19T09:08:22.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.384+0000 7f1d2e145740 1 client.17382 shutdown 2024-09-19T09:08:22.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.384+0000 7f1d2e145740 1 -- 172.21.15.120:0/3851057694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d0dd4bb40 msgr2=0x558d0dd5dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:22.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.384+0000 7f1d2e145740 1 --2- 172.21.15.120:0/3851057694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d0dd4bb40 0x558d0dd5dd80 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f1d20011960 tx=0x7f1d20003f30 comp rx=0 tx=0).stop 2024-09-19T09:08:22.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.384+0000 7f1d2e145740 1 -- 172.21.15.120:0/3851057694 shutdown_connections 2024-09-19T09:08:22.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.384+0000 7f1d2e145740 1 --2- 172.21.15.120:0/3851057694 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x558d0ddcfd00 0x558d0ddd2120 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.384+0000 7f1d0d7fa640 0 client.17382 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:08:22.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.384+0000 7f1d2e145740 1 --2- 172.21.15.120:0/3851057694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d0dd55370 0x558d0dd4ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.384+0000 7f1d2e145740 1 --2- 172.21.15.120:0/3851057694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d0dd4bb40 0x558d0dd5dd80 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.384+0000 7f1d2e145740 1 --2- 172.21.15.120:0/3851057694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d0dd4a5d0 0x558d0dd5d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.384+0000 7f1d2e145740 1 -- 172.21.15.120:0/3851057694 >> 172.21.15.120:0/3851057694 conn(0x558d0dd13310 msgr2=0x558d0dd54090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:22.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.384+0000 7f1d2e145740 1 -- 172.21.15.120:0/3851057694 shutdown_connections 2024-09-19T09:08:22.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.384+0000 7f1d2e145740 1 -- 172.21.15.120:0/3851057694 wait complete. 2024-09-19T09:08:22.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.384+0000 7f1d2e145740 20 client.17382 trim_cache size 0 max 16384 2024-09-19T09:08:22.391 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:08:22.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.385+0000 7f1d2bd6c640 1 -- 172.21.15.120:0/2194531857 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1d100a3240 msgr2=0x7f1d100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:22.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.385+0000 7f1d2bd6c640 1 --2- 172.21.15.120:0/2194531857 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1d100a3240 0x7f1d100a5700 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1d14009bd0 tx=0x7f1d14005f90 comp rx=0 tx=0).stop 2024-09-19T09:08:22.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.385+0000 7f1d2bd6c640 1 -- 172.21.15.120:0/2194531857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d2410e4e0 msgr2=0x7f1d2419ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:22.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.385+0000 7f1d2bd6c640 1 --2- 172.21.15.120:0/2194531857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d2410e4e0 0x7f1d2419ef70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c02ade0 tx=0x7f1d1c0319f0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.385+0000 7f1d2bd6c640 1 -- 172.21.15.120:0/2194531857 shutdown_connections 2024-09-19T09:08:22.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.385+0000 7f1d2bd6c640 1 --2- 172.21.15.120:0/2194531857 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1d100a3240 0x7f1d100a5700 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.385+0000 7f1d2bd6c640 1 --2- 172.21.15.120:0/2194531857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d2410e4e0 0x7f1d2419ef70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.385+0000 7f1d2bd6c640 1 --2- 172.21.15.120:0/2194531857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d241012e0 0x7f1d2419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.385+0000 7f1d2bd6c640 1 --2- 172.21.15.120:0/2194531857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d24100930 0x7f1d2419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:22.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.385+0000 7f1d2bd6c640 1 -- 172.21.15.120:0/2194531857 >> 172.21.15.120:0/2194531857 conn(0x7f1d240fc7a0 msgr2=0x7f1d240feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:22.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.385+0000 7f1d2bd6c640 1 -- 172.21.15.120:0/2194531857 shutdown_connections 2024-09-19T09:08:22.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:22.385+0000 7f1d2bd6c640 1 -- 172.21.15.120:0/2194531857 wait complete. 2024-09-19T09:08:23.050 INFO:tasks.workunit.client.0.smithi120.stdout: 1 404958 6.93 MB/sec execute 138 sec latency 10.467 ms 2024-09-19T09:08:23.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:22 smithi120 ceph-mon[31150]: pgmap v507: 97 pgs: 97 active+clean; 27 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 886 KiB/s wr, 156 op/s 2024-09-19T09:08:23.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:22 smithi196 ceph-mon[35150]: pgmap v507: 97 pgs: 97 active+clean; 27 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 886 KiB/s wr, 156 op/s 2024-09-19T09:08:23.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:22 smithi153 ceph-mon[35071]: pgmap v507: 97 pgs: 97 active+clean; 27 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 886 KiB/s wr, 156 op/s 2024-09-19T09:08:23.223 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:08:23.482 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:08:24.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.992+0000 7f7165aed640 1 -- 172.21.15.120:0/551740815 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7160102570 msgr2=0x7f71601029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:24.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.992+0000 7f7165aed640 1 --2- 172.21.15.120:0/551740815 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7160102570 0x7f71601029b0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f7148009900 tx=0x7f714802a480 comp rx=0 tx=0).stop 2024-09-19T09:08:24.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.992+0000 7f7165aed640 1 -- 172.21.15.120:0/551740815 shutdown_connections 2024-09-19T09:08:24.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.992+0000 7f7165aed640 1 --2- 172.21.15.120:0/551740815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7160102ef0 0x7f716010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.992+0000 7f7165aed640 1 --2- 172.21.15.120:0/551740815 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7160102570 0x7f71601029b0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.992+0000 7f7165aed640 1 --2- 172.21.15.120:0/551740815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7160100e20 0x7f7160101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.992+0000 7f7165aed640 1 -- 172.21.15.120:0/551740815 >> 172.21.15.120:0/551740815 conn(0x7f71600fc7d0 msgr2=0x7f71600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:24.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.992+0000 7f7165aed640 1 -- 172.21.15.120:0/551740815 shutdown_connections 2024-09-19T09:08:24.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.992+0000 7f7165aed640 1 -- 172.21.15.120:0/551740815 wait complete. 2024-09-19T09:08:24.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.993+0000 7f7165aed640 1 Processor -- start 2024-09-19T09:08:24.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.993+0000 7f7165aed640 1 -- start start 2024-09-19T09:08:24.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.993+0000 7f7165aed640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7160100e20 0x7f71601984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:24.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.993+0000 7f7165aed640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7160102570 0x7f7160198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:24.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.993+0000 7f7165aed640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7160102ef0 0x7f716019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:24.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.993+0000 7f7165aed640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f71601120a0 con 0x7f7160100e20 2024-09-19T09:08:24.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.993+0000 7f7165aed640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7160111f20 con 0x7f7160102570 2024-09-19T09:08:24.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.993+0000 7f7165aed640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7160112220 con 0x7f7160102ef0 2024-09-19T09:08:24.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.993+0000 7f715f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7160100e20 0x7f71601984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:24.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.993+0000 7f715f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7160100e20 0x7f71601984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46962/0 (socket says 172.21.15.120:46962) 2024-09-19T09:08:24.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.993+0000 7f715f7fe640 1 -- 172.21.15.120:0/1409435853 learned_addr learned my addr 172.21.15.120:0/1409435853 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:24.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.993+0000 7f7157fff640 1 --2- 172.21.15.120:0/1409435853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7160102570 0x7f7160198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:24.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.993+0000 7f715ffff640 1 --2- 172.21.15.120:0/1409435853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7160102ef0 0x7f716019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:24.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.993+0000 7f715f7fe640 1 -- 172.21.15.120:0/1409435853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7160102ef0 msgr2=0x7f716019cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:24.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.993+0000 7f715f7fe640 1 --2- 172.21.15.120:0/1409435853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7160102ef0 0x7f716019cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.993+0000 7f715f7fe640 1 -- 172.21.15.120:0/1409435853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7160102570 msgr2=0x7f7160198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:24.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.993+0000 7f715f7fe640 1 --2- 172.21.15.120:0/1409435853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7160102570 0x7f7160198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.993+0000 7f715f7fe640 1 -- 172.21.15.120:0/1409435853 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f716019d4a0 con 0x7f7160100e20 2024-09-19T09:08:24.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.993+0000 7f715f7fe640 1 --2- 172.21.15.120:0/1409435853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7160100e20 0x7f71601984f0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f715000e9f0 tx=0x7f715000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:24.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.994+0000 7f715d7fa640 1 -- 172.21.15.120:0/1409435853 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715000cd80 con 0x7f7160100e20 2024-09-19T09:08:24.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.994+0000 7f715d7fa640 1 -- 172.21.15.120:0/1409435853 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7150005500 con 0x7f7160100e20 2024-09-19T09:08:24.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.994+0000 7f715d7fa640 1 -- 172.21.15.120:0/1409435853 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7150010690 con 0x7f7160100e20 2024-09-19T09:08:24.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.994+0000 7f7165aed640 1 -- 172.21.15.120:0/1409435853 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f716019d790 con 0x7f7160100e20 2024-09-19T09:08:24.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.994+0000 7f7165aed640 1 -- 172.21.15.120:0/1409435853 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71601a5070 con 0x7f7160100e20 2024-09-19T09:08:24.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.996+0000 7f715d7fa640 1 -- 172.21.15.120:0/1409435853 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f71500046c0 con 0x7f7160100e20 2024-09-19T09:08:24.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.996+0000 7f7167ec6740 1 Processor -- start 2024-09-19T09:08:24.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.996+0000 7f7167ec6740 1 -- start start 2024-09-19T09:08:24.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.997+0000 7f7167ec6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e18a681b40 0x55e18a680090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:24.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.997+0000 7f7167ec6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e18a6805d0 0x55e18a680a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:24.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.997+0000 7f7167ec6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e18a68b370 0x55e18a68d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:24.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.997+0000 7f7167ec6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e18a5e62f0 con 0x55e18a681b40 2024-09-19T09:08:24.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.997+0000 7f7167ec6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e18a58dcf0 con 0x55e18a6805d0 2024-09-19T09:08:24.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.997+0000 7f7167ec6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e18a5e22b0 con 0x55e18a68b370 2024-09-19T09:08:24.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.997+0000 7f715d7fa640 1 --2- 172.21.15.120:0/1409435853 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f714c0a3240 0x7f714c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:24.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.997+0000 7f715f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e18a6805d0 0x55e18a680a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:24.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.997+0000 7f7157fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e18a68b370 0x55e18a68d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:24.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.997+0000 7f715ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e18a681b40 0x55e18a680090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:24.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.997+0000 7f715f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e18a6805d0 0x55e18a680a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:39504/0 (socket says 172.21.15.120:39504) 2024-09-19T09:08:24.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.997+0000 7f715f7fe640 1 -- 172.21.15.120:0/4239543711 learned_addr learned my addr 172.21.15.120:0/4239543711 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:24.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.997+0000 7f7157fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e18a68b370 0x55e18a68d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:42232/0 (socket says 172.21.15.120:42232) 2024-09-19T09:08:24.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.997+0000 7f715ffff640 1 --2- 172.21.15.120:0/1409435853 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f714c0a3240 0x7f714c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:24.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.997+0000 7f715d7fa640 1 -- 172.21.15.120:0/1409435853 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f71500cfcf0 con 0x7f7160100e20 2024-09-19T09:08:24.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.997+0000 7f7157fff640 1 -- 172.21.15.120:0/4239543711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e18a6805d0 msgr2=0x55e18a680a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:24.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.997+0000 7f7157fff640 1 --2- 172.21.15.120:0/4239543711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e18a6805d0 0x55e18a680a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.997+0000 7f7157fff640 1 -- 172.21.15.120:0/4239543711 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e18a681b40 msgr2=0x55e18a680090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:24.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.997+0000 7f7157fff640 1 --2- 172.21.15.120:0/4239543711 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e18a681b40 0x55e18a680090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.997+0000 7f7157fff640 1 -- 172.21.15.120:0/4239543711 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e18a5723a0 con 0x55e18a68b370 2024-09-19T09:08:24.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.997+0000 7f715f7fe640 1 --2- 172.21.15.120:0/4239543711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e18a6805d0 0x55e18a680a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:24.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.997+0000 7f7157fff640 1 --2- 172.21.15.120:0/4239543711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e18a68b370 0x55e18a68d760 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7148005940 tx=0x7f7148005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96c221116b0be9be server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:24.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.998+0000 7f715ffff640 1 --2- 172.21.15.120:0/1409435853 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f714c0a3240 0x7f714c0a5700 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7158009fd0 tx=0x7f7158004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:24.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.998+0000 7f7155ffb640 1 -- 172.21.15.120:0/4239543711 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7148035030 con 0x55e18a68b370 2024-09-19T09:08:24.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.998+0000 7f7155ffb640 1 -- 172.21.15.120:0/4239543711 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7148040070 con 0x55e18a68b370 2024-09-19T09:08:24.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.998+0000 7f7155ffb640 1 -- 172.21.15.120:0/4239543711 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7148004d80 con 0x55e18a68b370 2024-09-19T09:08:24.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.998+0000 7f7167ec6740 1 -- 172.21.15.120:0/4239543711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e18a68b370 msgr2=0x55e18a68d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:24.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.998+0000 7f7167ec6740 1 --2- 172.21.15.120:0/4239543711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e18a68b370 0x55e18a68d760 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7148005940 tx=0x7f7148005e80 comp rx=0 tx=0).stop 2024-09-19T09:08:24.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.998+0000 7f7167ec6740 1 -- 172.21.15.120:0/4239543711 shutdown_connections 2024-09-19T09:08:24.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.998+0000 7f7167ec6740 1 --2- 172.21.15.120:0/4239543711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e18a68b370 0x55e18a68d760 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.998+0000 7f7167ec6740 1 --2- 172.21.15.120:0/4239543711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e18a6805d0 0x55e18a680a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.998+0000 7f7167ec6740 1 --2- 172.21.15.120:0/4239543711 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e18a681b40 0x55e18a680090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.998+0000 7f7167ec6740 1 -- 172.21.15.120:0/4239543711 >> 172.21.15.120:0/4239543711 conn(0x55e18a649310 msgr2=0x55e18a67cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:24.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.998+0000 7f7167ec6740 1 -- 172.21.15.120:0/4239543711 shutdown_connections 2024-09-19T09:08:24.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.998+0000 7f7167ec6740 1 -- 172.21.15.120:0/4239543711 wait complete. 2024-09-19T09:08:24.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.999+0000 7f7167ec6740 1 -- start start 2024-09-19T09:08:24.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.999+0000 7f7167ec6740 1 Processor -- start 2024-09-19T09:08:24.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.999+0000 7f7167ec6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e18a6805d0 0x55e18a68ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:24.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.999+0000 7f7167ec6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e18a681b40 0x55e18a687fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:24.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.999+0000 7f7167ec6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e18a68b370 0x55e18a6885d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:24.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.999+0000 7f7167ec6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e18a68e010 con 0x55e18a6805d0 2024-09-19T09:08:24.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.999+0000 7f7167ec6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e18a5a1060 con 0x55e18a68b370 2024-09-19T09:08:24.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.999+0000 7f7167ec6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e18a68e190 con 0x55e18a681b40 2024-09-19T09:08:24.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.999+0000 7f7167ec6740 10 client.0 _ll_register_callbacks cb 0x55e18a2f48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:08:24.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:23.999+0000 7f715f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e18a6805d0 0x55e18a68ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:24.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.000+0000 7f715f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e18a6805d0 0x55e18a68ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46988/0 (socket says 172.21.15.120:46988) 2024-09-19T09:08:24.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.000+0000 7f715f7fe640 1 -- 172.21.15.120:0/3576074079 learned_addr learned my addr 172.21.15.120:0/3576074079 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:24.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.000+0000 7f715ffff640 1 --2- 172.21.15.120:0/3576074079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e18a68b370 0x55e18a6885d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:24.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.000+0000 7f7157fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e18a681b40 0x55e18a687fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:24.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.000+0000 7f715f7fe640 1 -- 172.21.15.120:0/3576074079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e18a681b40 msgr2=0x55e18a687fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:24.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.000+0000 7f715f7fe640 1 --2- 172.21.15.120:0/3576074079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e18a681b40 0x55e18a687fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.000+0000 7f715f7fe640 1 -- 172.21.15.120:0/3576074079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e18a68b370 msgr2=0x55e18a6885d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:24.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.000+0000 7f715f7fe640 1 --2- 172.21.15.120:0/3576074079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e18a68b370 0x55e18a6885d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.000+0000 7f715f7fe640 1 -- 172.21.15.120:0/3576074079 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e18a3573c0 con 0x55e18a6805d0 2024-09-19T09:08:24.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.000+0000 7f715ffff640 1 --2- 172.21.15.120:0/3576074079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e18a68b370 0x55e18a6885d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:24.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.000+0000 7f715f7fe640 1 --2- 172.21.15.120:0/3576074079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e18a6805d0 0x55e18a68ad30 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f715009c2a0 tx=0x7f71500056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af30ba7a938b3dc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:24.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.000+0000 7f713effd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:08:24.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.000+0000 7f713effd640 1 -- 172.21.15.120:0/3576074079 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715009c310 con 0x55e18a6805d0 2024-09-19T09:08:24.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.000+0000 7f713effd640 1 -- 172.21.15.120:0/3576074079 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f71500b51a0 con 0x55e18a6805d0 2024-09-19T09:08:24.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.000+0000 7f7167ec6740 1 -- 172.21.15.120:0/3576074079 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7148040070 con 0x55e18a6805d0 2024-09-19T09:08:24.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.000+0000 7f713effd640 1 -- 172.21.15.120:0/3576074079 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f715009ca10 con 0x55e18a6805d0 2024-09-19T09:08:24.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.000+0000 7f7167ec6740 10 client.17394 fetch_fsmap learned FSMap version 30 2024-09-19T09:08:24.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.000+0000 7f7167ec6740 1 -- 172.21.15.120:0/3576074079 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e18a699a70 con 0x55e18a6805d0 2024-09-19T09:08:24.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.000+0000 7f713effd640 1 -- 172.21.15.120:0/3576074079 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7148040070 con 0x55e18a6805d0 2024-09-19T09:08:24.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.001+0000 7f713effd640 1 -- 172.21.15.120:0/3576074079 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f715009b760 con 0x55e18a6805d0 2024-09-19T09:08:24.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.001+0000 7f7167ec6740 10 client.17394 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:08:24.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.001+0000 7f7167ec6740 10 client.17394 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:08:24.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.001+0000 7f7167ec6740 20 client.17394 populate_metadata read hostname 'smithi120' 2024-09-19T09:08:24.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.001+0000 7f7167ec6740 1 --2- 172.21.15.120:0/3576074079 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55e18a69ae80 0x55e18a69d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:24.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.001+0000 7f7167ec6740 4 client.17394 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:08:24.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.001+0000 7f7167ec6740 1 -- 172.21.15.120:0/3576074079 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e18a5f3860 con 0x55e18a69ae80 2024-09-19T09:08:24.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.001+0000 7f7157fff640 1 --2- 172.21.15.120:0/3576074079 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55e18a69ae80 0x55e18a69d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:24.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.002+0000 7f713effd640 10 client.17394 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:08:24.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.002+0000 7f7157fff640 1 --2- 172.21.15.120:0/3576074079 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55e18a69ae80 0x55e18a69d270 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=33e7c12398d3b74d server_cookie=a0c7335368db0119 in_seq=0 out_seq=0 2024-09-19T09:08:24.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.002+0000 7f713effd640 1 -- 172.21.15.120:0/3576074079 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e18a5f3860 con 0x55e18a69ae80 2024-09-19T09:08:24.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.002+0000 7f713effd640 10 client.17394 handle_command_reply: tid=0 2024-09-19T09:08:24.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.002+0000 7f7167ec6740 1 client.17394 shutdown 2024-09-19T09:08:24.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.002+0000 7f7167ec6740 1 -- 172.21.15.120:0/3576074079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e18a6805d0 msgr2=0x55e18a68ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:24.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.002+0000 7f7167ec6740 1 --2- 172.21.15.120:0/3576074079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e18a6805d0 0x55e18a68ad30 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f715009c2a0 tx=0x7f71500056b0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.002+0000 7f7167ec6740 1 -- 172.21.15.120:0/3576074079 shutdown_connections 2024-09-19T09:08:24.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.002+0000 7f7167ec6740 1 --2- 172.21.15.120:0/3576074079 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55e18a69ae80 0x55e18a69d270 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.002+0000 7f7167ec6740 1 --2- 172.21.15.120:0/3576074079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e18a68b370 0x55e18a6885d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.002+0000 7f7167ec6740 1 --2- 172.21.15.120:0/3576074079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e18a681b40 0x55e18a687fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.002+0000 7f713effd640 0 client.17394 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:08:24.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.002+0000 7f7167ec6740 1 --2- 172.21.15.120:0/3576074079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e18a6805d0 0x55e18a68ad30 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.002+0000 7f7167ec6740 1 -- 172.21.15.120:0/3576074079 >> 172.21.15.120:0/3576074079 conn(0x55e18a649310 msgr2=0x55e18a682960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:24.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.003+0000 7f7167ec6740 1 -- 172.21.15.120:0/3576074079 shutdown_connections 2024-09-19T09:08:24.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.003+0000 7f7167ec6740 1 -- 172.21.15.120:0/3576074079 wait complete. 2024-09-19T09:08:24.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.003+0000 7f7167ec6740 20 client.17394 trim_cache size 0 max 16384 2024-09-19T09:08:24.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.016+0000 7f7167ec6740 1 Processor -- start 2024-09-19T09:08:24.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.016+0000 7f7167ec6740 1 -- start start 2024-09-19T09:08:24.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.016+0000 7f7167ec6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e18a6805d0 0x55e18a683120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:24.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.016+0000 7f7167ec6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e18a681b40 0x55e18a683660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:24.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.016+0000 7f7167ec6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e18a68b370 0x55e18a6a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:24.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.016+0000 7f7167ec6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f71601ad530 con 0x55e18a6805d0 2024-09-19T09:08:24.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.016+0000 7f7167ec6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f71601af070 con 0x55e18a68b370 2024-09-19T09:08:24.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.016+0000 7f7167ec6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f71601adc00 con 0x55e18a681b40 2024-09-19T09:08:24.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.017+0000 7f715f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e18a6805d0 0x55e18a683120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:24.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.017+0000 7f7157fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e18a681b40 0x55e18a683660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:24.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.017+0000 7f715ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e18a68b370 0x55e18a6a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:24.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.017+0000 7f715f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e18a6805d0 0x55e18a683120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47002/0 (socket says 172.21.15.120:47002) 2024-09-19T09:08:24.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.017+0000 7f715f7fe640 1 -- 172.21.15.120:0/1312668998 learned_addr learned my addr 172.21.15.120:0/1312668998 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:24.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.017+0000 7f715ffff640 1 -- 172.21.15.120:0/1312668998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e18a681b40 msgr2=0x55e18a683660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:24.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.017+0000 7f715ffff640 1 --2- 172.21.15.120:0/1312668998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e18a681b40 0x55e18a683660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.017+0000 7f715ffff640 1 -- 172.21.15.120:0/1312668998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e18a6805d0 msgr2=0x55e18a683120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:24.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.017+0000 7f715ffff640 1 --2- 172.21.15.120:0/1312668998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e18a6805d0 0x55e18a683120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.017+0000 7f715ffff640 1 -- 172.21.15.120:0/1312668998 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e18a68b0a0 con 0x55e18a68b370 2024-09-19T09:08:24.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.017+0000 7f715f7fe640 1 --2- 172.21.15.120:0/1312668998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e18a6805d0 0x55e18a683120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:24.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.017+0000 7f7157fff640 1 --2- 172.21.15.120:0/1312668998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e18a681b40 0x55e18a683660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:24.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.017+0000 7f715ffff640 1 --2- 172.21.15.120:0/1312668998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e18a68b370 0x55e18a6a2b20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7158009370 tx=0x7f7158009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=228e68ac5cbaec76 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:24.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.017+0000 7f71557fa640 1 -- 172.21.15.120:0/1312668998 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158010460 con 0x55e18a68b370 2024-09-19T09:08:24.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.017+0000 7f71557fa640 1 -- 172.21.15.120:0/1312668998 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7158028020 con 0x55e18a68b370 2024-09-19T09:08:24.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.017+0000 7f71557fa640 1 -- 172.21.15.120:0/1312668998 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158023840 con 0x55e18a68b370 2024-09-19T09:08:24.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.018+0000 7f7167ec6740 1 -- 172.21.15.120:0/1312668998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e18a68b370 msgr2=0x55e18a6a2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:24.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.018+0000 7f7167ec6740 1 --2- 172.21.15.120:0/1312668998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e18a68b370 0x55e18a6a2b20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7158009370 tx=0x7f7158009920 comp rx=0 tx=0).stop 2024-09-19T09:08:24.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.018+0000 7f7167ec6740 1 -- 172.21.15.120:0/1312668998 shutdown_connections 2024-09-19T09:08:24.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.018+0000 7f7167ec6740 1 --2- 172.21.15.120:0/1312668998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e18a68b370 0x55e18a6a2b20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.018+0000 7f7167ec6740 1 --2- 172.21.15.120:0/1312668998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e18a681b40 0x55e18a683660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.018+0000 7f7167ec6740 1 --2- 172.21.15.120:0/1312668998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e18a6805d0 0x55e18a683120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.018+0000 7f7167ec6740 1 -- 172.21.15.120:0/1312668998 >> 172.21.15.120:0/1312668998 conn(0x55e18a649310 msgr2=0x55e18a68a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:24.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.018+0000 7f7167ec6740 1 -- 172.21.15.120:0/1312668998 shutdown_connections 2024-09-19T09:08:24.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.018+0000 7f7167ec6740 1 -- 172.21.15.120:0/1312668998 wait complete. 2024-09-19T09:08:24.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.018+0000 7f7167ec6740 1 -- start start 2024-09-19T09:08:24.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.019+0000 7f7167ec6740 1 Processor -- start 2024-09-19T09:08:24.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.019+0000 7f7167ec6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e18a6805d0 0x55e18a6939d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:24.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.019+0000 7f7167ec6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e18a681b40 0x55e18a684c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:24.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.019+0000 7f7167ec6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e18a68b370 0x55e18a685160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:24.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.019+0000 7f7167ec6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f71601ae9a0 con 0x55e18a68b370 2024-09-19T09:08:24.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.019+0000 7f7167ec6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f71601ae2d0 con 0x55e18a681b40 2024-09-19T09:08:24.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.019+0000 7f7167ec6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f71601ab4f0 con 0x55e18a6805d0 2024-09-19T09:08:24.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.019+0000 7f7167ec6740 10 client.0 _ll_register_callbacks cb 0x7f715000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:08:24.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.019+0000 7f715ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e18a68b370 0x55e18a685160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:24.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.019+0000 7f7157fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e18a681b40 0x55e18a684c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:24.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.019+0000 7f715ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e18a68b370 0x55e18a685160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47004/0 (socket says 172.21.15.120:47004) 2024-09-19T09:08:24.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.019+0000 7f715ffff640 1 -- 172.21.15.120:0/3671234383 learned_addr learned my addr 172.21.15.120:0/3671234383 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:24.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.019+0000 7f715f7fe640 1 --2- 172.21.15.120:0/3671234383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e18a6805d0 0x55e18a6939d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:24.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.019+0000 7f7157fff640 1 -- 172.21.15.120:0/3671234383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e18a6805d0 msgr2=0x55e18a6939d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:24.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.019+0000 7f7157fff640 1 --2- 172.21.15.120:0/3671234383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e18a6805d0 0x55e18a6939d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.019+0000 7f7157fff640 1 -- 172.21.15.120:0/3671234383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e18a68b370 msgr2=0x55e18a685160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:24.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.019+0000 7f7157fff640 1 --2- 172.21.15.120:0/3671234383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e18a68b370 0x55e18a685160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.019+0000 7f7157fff640 1 -- 172.21.15.120:0/3671234383 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e18a685850 con 0x55e18a681b40 2024-09-19T09:08:24.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.019+0000 7f715ffff640 1 --2- 172.21.15.120:0/3671234383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e18a68b370 0x55e18a685160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:24.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.020+0000 7f7157fff640 1 --2- 172.21.15.120:0/3671234383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e18a681b40 0x55e18a684c20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f714802a9a0 tx=0x7f7148009660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e0512f2462af8a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:24.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.020+0000 7f713effd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:08:24.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.020+0000 7f713effd640 1 -- 172.21.15.120:0/3671234383 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7148035040 con 0x55e18a681b40 2024-09-19T09:08:24.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.020+0000 7f7167ec6740 1 -- 172.21.15.120:0/3671234383 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7158028020 con 0x55e18a681b40 2024-09-19T09:08:24.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.020+0000 7f713effd640 1 -- 172.21.15.120:0/3671234383 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f7148004300 con 0x55e18a681b40 2024-09-19T09:08:24.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.020+0000 7f713effd640 1 -- 172.21.15.120:0/3671234383 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f714805e050 con 0x55e18a681b40 2024-09-19T09:08:24.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.020+0000 7f7167ec6740 10 client.26672 fetch_fsmap learned FSMap version 30 2024-09-19T09:08:24.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.020+0000 7f7167ec6740 1 -- 172.21.15.120:0/3671234383 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e18a7059d0 con 0x55e18a681b40 2024-09-19T09:08:24.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.020+0000 7f713effd640 1 -- 172.21.15.120:0/3671234383 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7158028020 con 0x55e18a681b40 2024-09-19T09:08:24.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.020+0000 7f713effd640 1 -- 172.21.15.120:0/3671234383 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f71480035f0 con 0x55e18a681b40 2024-09-19T09:08:24.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.020+0000 7f7167ec6740 10 client.26672 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:08:24.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.020+0000 7f7167ec6740 10 client.26672 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:08:24.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.020+0000 7f7167ec6740 20 client.26672 populate_metadata read hostname 'smithi120' 2024-09-19T09:08:24.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.020+0000 7f7167ec6740 1 --2- 172.21.15.120:0/3671234383 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55e18a705d10 0x55e18a708100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:24.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.020+0000 7f7167ec6740 4 client.26672 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:08:24.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.020+0000 7f7167ec6740 1 -- 172.21.15.120:0/3671234383 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e18a5edd70 con 0x55e18a705d10 2024-09-19T09:08:24.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.021+0000 7f715f7fe640 1 --2- 172.21.15.120:0/3671234383 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55e18a705d10 0x55e18a708100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:24.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.021+0000 7f713effd640 10 client.26672 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:08:24.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.021+0000 7f715f7fe640 1 --2- 172.21.15.120:0/3671234383 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55e18a705d10 0x55e18a708100 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6b57974859bd8acf server_cookie=ec0615215696046c in_seq=0 out_seq=0 2024-09-19T09:08:24.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.021+0000 7f713effd640 1 -- 172.21.15.120:0/3671234383 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e18a5edd70 con 0x55e18a705d10 2024-09-19T09:08:24.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.021+0000 7f713effd640 10 client.26672 handle_command_reply: tid=0 2024-09-19T09:08:24.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.021+0000 7f7167ec6740 1 client.26672 shutdown 2024-09-19T09:08:24.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.021+0000 7f7167ec6740 1 -- 172.21.15.120:0/3671234383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e18a681b40 msgr2=0x55e18a684c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:24.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.021+0000 7f7167ec6740 1 --2- 172.21.15.120:0/3671234383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e18a681b40 0x55e18a684c20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f714802a9a0 tx=0x7f7148009660 comp rx=0 tx=0).stop 2024-09-19T09:08:24.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.021+0000 7f7167ec6740 1 -- 172.21.15.120:0/3671234383 shutdown_connections 2024-09-19T09:08:24.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.021+0000 7f7167ec6740 1 --2- 172.21.15.120:0/3671234383 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55e18a705d10 0x55e18a708100 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.021+0000 7f7167ec6740 1 --2- 172.21.15.120:0/3671234383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e18a68b370 0x55e18a685160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.021+0000 7f7167ec6740 1 --2- 172.21.15.120:0/3671234383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e18a681b40 0x55e18a684c20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.021+0000 7f7167ec6740 1 --2- 172.21.15.120:0/3671234383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e18a6805d0 0x55e18a6939d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.021+0000 7f713effd640 0 client.26672 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:08:24.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.021+0000 7f7167ec6740 1 -- 172.21.15.120:0/3671234383 >> 172.21.15.120:0/3671234383 conn(0x55e18a649310 msgr2=0x55e18a67b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:24.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.021+0000 7f7167ec6740 1 -- 172.21.15.120:0/3671234383 shutdown_connections 2024-09-19T09:08:24.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.021+0000 7f7167ec6740 1 -- 172.21.15.120:0/3671234383 wait complete. 2024-09-19T09:08:24.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.022+0000 7f7167ec6740 20 client.26672 trim_cache size 0 max 16384 2024-09-19T09:08:24.038 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:08:24.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.022+0000 7f7165aed640 1 -- 172.21.15.120:0/1409435853 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f714c0a3240 msgr2=0x7f714c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:24.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.022+0000 7f7165aed640 1 --2- 172.21.15.120:0/1409435853 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f714c0a3240 0x7f714c0a5700 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7158009fd0 tx=0x7f7158004ab0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.022+0000 7f7165aed640 1 -- 172.21.15.120:0/1409435853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7160100e20 msgr2=0x7f71601984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:24.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.022+0000 7f7165aed640 1 --2- 172.21.15.120:0/1409435853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7160100e20 0x7f71601984f0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f715000e9f0 tx=0x7f715000edf0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.022+0000 7f7165aed640 1 -- 172.21.15.120:0/1409435853 shutdown_connections 2024-09-19T09:08:24.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.022+0000 7f7165aed640 1 --2- 172.21.15.120:0/1409435853 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f714c0a3240 0x7f714c0a5700 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.022+0000 7f7165aed640 1 --2- 172.21.15.120:0/1409435853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7160102ef0 0x7f716019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.022+0000 7f7165aed640 1 --2- 172.21.15.120:0/1409435853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7160102570 0x7f7160198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.022+0000 7f7165aed640 1 --2- 172.21.15.120:0/1409435853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7160100e20 0x7f71601984f0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:24.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.022+0000 7f7165aed640 1 -- 172.21.15.120:0/1409435853 >> 172.21.15.120:0/1409435853 conn(0x7f71600fc7d0 msgr2=0x7f71600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:24.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.022+0000 7f7165aed640 1 -- 172.21.15.120:0/1409435853 shutdown_connections 2024-09-19T09:08:24.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:24.022+0000 7f7165aed640 1 -- 172.21.15.120:0/1409435853 wait complete. 2024-09-19T09:08:24.050 INFO:tasks.workunit.client.0.smithi120.stdout: 1 408200 7.02 MB/sec execute 139 sec latency 10.862 ms 2024-09-19T09:08:24.971 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:08:25.050 INFO:tasks.workunit.client.0.smithi120.stdout: 1 411123 7.06 MB/sec execute 140 sec latency 11.208 ms 2024-09-19T09:08:25.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:24 smithi120 ceph-mon[31150]: pgmap v508: 97 pgs: 97 active+clean; 31 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.3 MiB/s wr, 252 op/s 2024-09-19T09:08:25.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:24 smithi196 ceph-mon[35150]: pgmap v508: 97 pgs: 97 active+clean; 31 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.3 MiB/s wr, 252 op/s 2024-09-19T09:08:25.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:24 smithi153 ceph-mon[35071]: pgmap v508: 97 pgs: 97 active+clean; 31 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.3 MiB/s wr, 252 op/s 2024-09-19T09:08:25.228 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:08:25.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.749+0000 7fcbff069640 1 -- 172.21.15.120:0/436544413 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbf81036e0 msgr2=0x7fcbf8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:25.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.749+0000 7fcbff069640 1 --2- 172.21.15.120:0/436544413 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbf81036e0 0x7fcbf8107b90 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009930 tx=0x7fcbe002a420 comp rx=0 tx=0).stop 2024-09-19T09:08:25.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.749+0000 7fcbff069640 1 -- 172.21.15.120:0/436544413 shutdown_connections 2024-09-19T09:08:25.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.749+0000 7fcbff069640 1 --2- 172.21.15.120:0/436544413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbf81082a0 0x7fcbf810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.749+0000 7fcbff069640 1 --2- 172.21.15.120:0/436544413 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbf81036e0 0x7fcbf8107b90 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.749+0000 7fcbff069640 1 --2- 172.21.15.120:0/436544413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbf8102d30 0x7fcbf8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.749+0000 7fcbff069640 1 -- 172.21.15.120:0/436544413 >> 172.21.15.120:0/436544413 conn(0x7fcbf80fc7d0 msgr2=0x7fcbf80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:25.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.749+0000 7fcbff069640 1 -- 172.21.15.120:0/436544413 shutdown_connections 2024-09-19T09:08:25.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.749+0000 7fcbff069640 1 -- 172.21.15.120:0/436544413 wait complete. 2024-09-19T09:08:25.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.749+0000 7fcbff069640 1 Processor -- start 2024-09-19T09:08:25.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.750+0000 7fcbff069640 1 -- start start 2024-09-19T09:08:25.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.750+0000 7fcbff069640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbf8102d30 0x7fcbf81985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:25.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.750+0000 7fcbff069640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbf81036e0 0x7fcbf8198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:25.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.750+0000 7fcbff069640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbf81082a0 0x7fcbf819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:25.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.750+0000 7fcbff069640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf81120a0 con 0x7fcbf8102d30 2024-09-19T09:08:25.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.750+0000 7fcbff069640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf8111f20 con 0x7fcbf81036e0 2024-09-19T09:08:25.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.750+0000 7fcbff069640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf8112220 con 0x7fcbf81082a0 2024-09-19T09:08:25.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.750+0000 7fcbfce05640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbf8102d30 0x7fcbf81985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:25.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.750+0000 7fcbfce05640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbf8102d30 0x7fcbf81985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47028/0 (socket says 172.21.15.120:47028) 2024-09-19T09:08:25.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.750+0000 7fcbfce05640 1 -- 172.21.15.120:0/2322935918 learned_addr learned my addr 172.21.15.120:0/2322935918 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:25.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.750+0000 7fcbfce05640 1 -- 172.21.15.120:0/2322935918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbf81082a0 msgr2=0x7fcbf819ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:25.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.750+0000 7fcbfce05640 1 --2- 172.21.15.120:0/2322935918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbf81082a0 0x7fcbf819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.750+0000 7fcbfce05640 1 -- 172.21.15.120:0/2322935918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbf81036e0 msgr2=0x7fcbf8198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:25.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.750+0000 7fcbfce05640 1 --2- 172.21.15.120:0/2322935918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbf81036e0 0x7fcbf8198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.750+0000 7fcbfce05640 1 -- 172.21.15.120:0/2322935918 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf819d590 con 0x7fcbf8102d30 2024-09-19T09:08:25.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.750+0000 7fcbfce05640 1 --2- 172.21.15.120:0/2322935918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbf8102d30 0x7fcbf81985e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fcbe800eb90 tx=0x7fcbe800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:25.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.751+0000 7fcbf5ffb640 1 -- 172.21.15.120:0/2322935918 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8004d20 con 0x7fcbf8102d30 2024-09-19T09:08:25.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.751+0000 7fcbf5ffb640 1 -- 172.21.15.120:0/2322935918 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbe800ee90 con 0x7fcbf8102d30 2024-09-19T09:08:25.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.751+0000 7fcbf5ffb640 1 -- 172.21.15.120:0/2322935918 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8010830 con 0x7fcbf8102d30 2024-09-19T09:08:25.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.751+0000 7fcbff069640 1 -- 172.21.15.120:0/2322935918 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbf819d880 con 0x7fcbf8102d30 2024-09-19T09:08:25.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.751+0000 7fcbff069640 1 -- 172.21.15.120:0/2322935918 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbf81a5160 con 0x7fcbf8102d30 2024-09-19T09:08:25.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.753+0000 7fcbf5ffb640 1 -- 172.21.15.120:0/2322935918 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fcbe8005020 con 0x7fcbf8102d30 2024-09-19T09:08:25.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.754+0000 7fcc01442740 1 Processor -- start 2024-09-19T09:08:25.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.754+0000 7fcc01442740 1 -- start start 2024-09-19T09:08:25.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.754+0000 7fcbf5ffb640 1 --2- 172.21.15.120:0/2322935918 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcbe40a32c0 0x7fcbe40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:25.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.754+0000 7fcc01442740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f2a8b5b40 0x558f2a8b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:25.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.754+0000 7fcc01442740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f2a8b45d0 0x558f2a8b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:25.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.754+0000 7fcc01442740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f2a8bf370 0x558f2a8c1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:25.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.754+0000 7fcc01442740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558f2a81a2f0 con 0x558f2a8b45d0 2024-09-19T09:08:25.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.754+0000 7fcc01442740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558f2a7c1cf0 con 0x558f2a8b5b40 2024-09-19T09:08:25.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.754+0000 7fcc01442740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558f2a8162b0 con 0x558f2a8bf370 2024-09-19T09:08:25.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.754+0000 7fcbfce05640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f2a8b5b40 0x558f2a8b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:25.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.754+0000 7fcbfd606640 1 --2- 172.21.15.120:0/2322935918 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcbe40a32c0 0x7fcbe40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:25.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.754+0000 7fcbfce05640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f2a8b5b40 0x558f2a8b4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:39554/0 (socket says 172.21.15.120:39554) 2024-09-19T09:08:25.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.754+0000 7fcbfce05640 1 -- 172.21.15.120:0/1874863830 learned_addr learned my addr 172.21.15.120:0/1874863830 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:25.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.754+0000 7fcbf5ffb640 1 -- 172.21.15.120:0/2322935918 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fcbe80d0040 con 0x7fcbf8102d30 2024-09-19T09:08:25.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.754+0000 7fcbf7fff640 1 --2- 172.21.15.120:0/1874863830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f2a8b45d0 0x558f2a8b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:25.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.754+0000 7fcbfd606640 1 --2- 172.21.15.120:0/1874863830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f2a8bf370 0x558f2a8c1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:25.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.755+0000 7fcbfd606640 1 -- 172.21.15.120:0/1874863830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f2a8b5b40 msgr2=0x558f2a8b4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:25.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.755+0000 7fcbfd606640 1 --2- 172.21.15.120:0/1874863830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f2a8b5b40 0x558f2a8b4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.755+0000 7fcbfd606640 1 -- 172.21.15.120:0/1874863830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f2a8b45d0 msgr2=0x558f2a8b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:25.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.755+0000 7fcbfd606640 1 --2- 172.21.15.120:0/1874863830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f2a8b45d0 0x558f2a8b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.755+0000 7fcbfd606640 1 -- 172.21.15.120:0/1874863830 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f2a7a63a0 con 0x558f2a8bf370 2024-09-19T09:08:25.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.755+0000 7fcbf7fff640 1 --2- 172.21.15.120:0/1874863830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f2a8b45d0 0x558f2a8b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:25.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.755+0000 7fcbfce05640 1 --2- 172.21.15.120:0/1874863830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f2a8b5b40 0x558f2a8b4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:25.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.755+0000 7fcbfd606640 1 --2- 172.21.15.120:0/1874863830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f2a8bf370 0x558f2a8c1760 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0009fd0 tx=0x7fcbf0012960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3d8bd19a56e1fa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:25.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.755+0000 7fcbeeffd640 1 -- 172.21.15.120:0/1874863830 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0005250 con 0x558f2a8bf370 2024-09-19T09:08:25.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.755+0000 7fcbeeffd640 1 -- 172.21.15.120:0/1874863830 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbf0021070 con 0x558f2a8bf370 2024-09-19T09:08:25.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.755+0000 7fcbeeffd640 1 -- 172.21.15.120:0/1874863830 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf001c680 con 0x558f2a8bf370 2024-09-19T09:08:25.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.756+0000 7fcc01442740 1 -- 172.21.15.120:0/1874863830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f2a8bf370 msgr2=0x558f2a8c1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:25.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.756+0000 7fcc01442740 1 --2- 172.21.15.120:0/1874863830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f2a8bf370 0x558f2a8c1760 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0009fd0 tx=0x7fcbf0012960 comp rx=0 tx=0).stop 2024-09-19T09:08:25.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.756+0000 7fcbfd606640 1 --2- 172.21.15.120:0/2322935918 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcbe40a32c0 0x7fcbe40a5780 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0020000 tx=0x7fcbf001c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:25.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.756+0000 7fcc01442740 1 -- 172.21.15.120:0/1874863830 shutdown_connections 2024-09-19T09:08:25.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.756+0000 7fcc01442740 1 --2- 172.21.15.120:0/1874863830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f2a8bf370 0x558f2a8c1760 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.756+0000 7fcc01442740 1 --2- 172.21.15.120:0/1874863830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f2a8b45d0 0x558f2a8b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.756+0000 7fcc01442740 1 --2- 172.21.15.120:0/1874863830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f2a8b5b40 0x558f2a8b4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.756+0000 7fcc01442740 1 -- 172.21.15.120:0/1874863830 >> 172.21.15.120:0/1874863830 conn(0x558f2a87d310 msgr2=0x558f2a8b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:25.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.756+0000 7fcc01442740 1 -- 172.21.15.120:0/1874863830 shutdown_connections 2024-09-19T09:08:25.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.756+0000 7fcc01442740 1 -- 172.21.15.120:0/1874863830 wait complete. 2024-09-19T09:08:25.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.756+0000 7fcc01442740 1 -- start start 2024-09-19T09:08:25.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.756+0000 7fcc01442740 1 Processor -- start 2024-09-19T09:08:25.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.757+0000 7fcc01442740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f2a8b45d0 0x558f2a8bed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:25.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.757+0000 7fcc01442740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f2a8b5b40 0x558f2a8bc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:25.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.757+0000 7fcc01442740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f2a8bf370 0x558f2a8bc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:25.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.757+0000 7fcc01442740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558f2a8c6480 con 0x558f2a8b45d0 2024-09-19T09:08:25.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.757+0000 7fcc01442740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558f2a7d5060 con 0x558f2a8bf370 2024-09-19T09:08:25.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.757+0000 7fcc01442740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558f2a8c6600 con 0x558f2a8b5b40 2024-09-19T09:08:25.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.757+0000 7fcc01442740 10 client.0 _ll_register_callbacks cb 0x558f2a5288c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:08:25.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.757+0000 7fcbfce05640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f2a8b45d0 0x558f2a8bed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:25.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.757+0000 7fcbfce05640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f2a8b45d0 0x558f2a8bed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47054/0 (socket says 172.21.15.120:47054) 2024-09-19T09:08:25.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.757+0000 7fcbfd606640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f2a8bf370 0x558f2a8bc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:25.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.757+0000 7fcbfce05640 1 -- 172.21.15.120:0/359841808 learned_addr learned my addr 172.21.15.120:0/359841808 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:25.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.757+0000 7fcbf7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f2a8b5b40 0x558f2a8bc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:25.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.757+0000 7fcbfce05640 1 -- 172.21.15.120:0/359841808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f2a8b5b40 msgr2=0x558f2a8bc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:25.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.757+0000 7fcbfce05640 1 --2- 172.21.15.120:0/359841808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f2a8b5b40 0x558f2a8bc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.757+0000 7fcbfce05640 1 -- 172.21.15.120:0/359841808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f2a8bf370 msgr2=0x558f2a8bc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:25.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.757+0000 7fcbfce05640 1 --2- 172.21.15.120:0/359841808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f2a8bf370 0x558f2a8bc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.757+0000 7fcbfce05640 1 -- 172.21.15.120:0/359841808 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f2a58b3c0 con 0x558f2a8b45d0 2024-09-19T09:08:25.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.758+0000 7fcbfce05640 1 --2- 172.21.15.120:0/359841808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f2a8b45d0 0x558f2a8bed70 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8013040 tx=0x7fcbe800a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feab48fa43df1f46 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:25.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.758+0000 7fcbd7fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:08:25.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.758+0000 7fcbd7fff640 1 -- 172.21.15.120:0/359841808 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe80b5060 con 0x558f2a8b45d0 2024-09-19T09:08:25.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.758+0000 7fcc01442740 1 -- 172.21.15.120:0/359841808 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbf0021070 con 0x558f2a8b45d0 2024-09-19T09:08:25.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.758+0000 7fcbd7fff640 1 -- 172.21.15.120:0/359841808 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fcbe809a960 con 0x558f2a8b45d0 2024-09-19T09:08:25.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.758+0000 7fcbd7fff640 1 -- 172.21.15.120:0/359841808 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcbe80a3fd0 con 0x558f2a8b45d0 2024-09-19T09:08:25.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.758+0000 7fcc01442740 10 client.17412 fetch_fsmap learned FSMap version 30 2024-09-19T09:08:25.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.758+0000 7fcc01442740 1 -- 172.21.15.120:0/359841808 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558f2a8d1f20 con 0x558f2a8b45d0 2024-09-19T09:08:25.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.758+0000 7fcbd7fff640 1 -- 172.21.15.120:0/359841808 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbf0021070 con 0x558f2a8b45d0 2024-09-19T09:08:25.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.758+0000 7fcbd7fff640 1 -- 172.21.15.120:0/359841808 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fcbe80a3200 con 0x558f2a8b45d0 2024-09-19T09:08:25.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.758+0000 7fcc01442740 10 client.17412 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:08:25.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.758+0000 7fcc01442740 10 client.17412 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:08:25.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.758+0000 7fcc01442740 20 client.17412 populate_metadata read hostname 'smithi120' 2024-09-19T09:08:25.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.758+0000 7fcc01442740 1 --2- 172.21.15.120:0/359841808 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x558f2a8d3330 0x558f2a8d5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:25.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.758+0000 7fcc01442740 4 client.17412 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:08:25.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.758+0000 7fcc01442740 1 -- 172.21.15.120:0/359841808 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f2a827860 con 0x558f2a8d3330 2024-09-19T09:08:25.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.759+0000 7fcbf7fff640 1 --2- 172.21.15.120:0/359841808 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x558f2a8d3330 0x558f2a8d5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:25.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.759+0000 7fcbf7fff640 1 --2- 172.21.15.120:0/359841808 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x558f2a8d3330 0x558f2a8d5720 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6320a294daa2aa59 server_cookie=5ffa009eac96f402 in_seq=0 out_seq=0 2024-09-19T09:08:25.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.759+0000 7fcbd7fff640 10 client.17412 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:08:25.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.760+0000 7fcbd7fff640 1 -- 172.21.15.120:0/359841808 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558f2a827860 con 0x558f2a8d3330 2024-09-19T09:08:25.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.760+0000 7fcbd7fff640 10 client.17412 handle_command_reply: tid=0 2024-09-19T09:08:25.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.760+0000 7fcc01442740 1 client.17412 shutdown 2024-09-19T09:08:25.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.760+0000 7fcc01442740 1 -- 172.21.15.120:0/359841808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f2a8b45d0 msgr2=0x558f2a8bed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:25.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.760+0000 7fcc01442740 1 --2- 172.21.15.120:0/359841808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f2a8b45d0 0x558f2a8bed70 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8013040 tx=0x7fcbe800a440 comp rx=0 tx=0).stop 2024-09-19T09:08:25.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.760+0000 7fcc01442740 1 -- 172.21.15.120:0/359841808 shutdown_connections 2024-09-19T09:08:25.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.760+0000 7fcc01442740 1 --2- 172.21.15.120:0/359841808 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x558f2a8d3330 0x558f2a8d5720 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.760+0000 7fcc01442740 1 --2- 172.21.15.120:0/359841808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f2a8bf370 0x558f2a8bc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.760+0000 7fcc01442740 1 --2- 172.21.15.120:0/359841808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f2a8b5b40 0x558f2a8bc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.760+0000 7fcc01442740 1 --2- 172.21.15.120:0/359841808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f2a8b45d0 0x558f2a8bed70 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.760+0000 7fcbd7fff640 0 client.17412 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:08:25.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.760+0000 7fcc01442740 1 -- 172.21.15.120:0/359841808 >> 172.21.15.120:0/359841808 conn(0x558f2a87d310 msgr2=0x558f2a8b6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:25.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.760+0000 7fcc01442740 1 -- 172.21.15.120:0/359841808 shutdown_connections 2024-09-19T09:08:25.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.760+0000 7fcc01442740 1 -- 172.21.15.120:0/359841808 wait complete. 2024-09-19T09:08:25.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.760+0000 7fcc01442740 20 client.17412 trim_cache size 0 max 16384 2024-09-19T09:08:25.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.774+0000 7fcc01442740 1 Processor -- start 2024-09-19T09:08:25.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.775+0000 7fcc01442740 1 -- start start 2024-09-19T09:08:25.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.775+0000 7fcc01442740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f2a8b5b40 0x558f2a8cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:25.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.775+0000 7fcc01442740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f2a8bf370 0x558f2a8afc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:25.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.775+0000 7fcc01442740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f2a8b0f20 0x558f2a8dafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:25.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.775+0000 7fcc01442740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf81ab480 con 0x558f2a8b0f20 2024-09-19T09:08:25.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.775+0000 7fcc01442740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf81acfc0 con 0x558f2a8b5b40 2024-09-19T09:08:25.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.775+0000 7fcc01442740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf81abb50 con 0x558f2a8bf370 2024-09-19T09:08:25.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.775+0000 7fcbfd606640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f2a8b0f20 0x558f2a8dafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:25.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.775+0000 7fcbfce05640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f2a8b5b40 0x558f2a8cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:25.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.775+0000 7fcbfd606640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f2a8b0f20 0x558f2a8dafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47064/0 (socket says 172.21.15.120:47064) 2024-09-19T09:08:25.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.775+0000 7fcbfd606640 1 -- 172.21.15.120:0/1636109044 learned_addr learned my addr 172.21.15.120:0/1636109044 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:25.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.775+0000 7fcbf7fff640 1 --2- 172.21.15.120:0/1636109044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f2a8bf370 0x558f2a8afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:25.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.775+0000 7fcbfce05640 1 -- 172.21.15.120:0/1636109044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f2a8bf370 msgr2=0x558f2a8afc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:25.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.775+0000 7fcbfce05640 1 --2- 172.21.15.120:0/1636109044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f2a8bf370 0x558f2a8afc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.775+0000 7fcbfce05640 1 -- 172.21.15.120:0/1636109044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f2a8b0f20 msgr2=0x558f2a8dafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:25.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.775+0000 7fcbfce05640 1 --2- 172.21.15.120:0/1636109044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f2a8b0f20 0x558f2a8dafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.775+0000 7fcbfce05640 1 -- 172.21.15.120:0/1636109044 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f2a8b1960 con 0x558f2a8b5b40 2024-09-19T09:08:25.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.775+0000 7fcbfd606640 1 --2- 172.21.15.120:0/1636109044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f2a8b0f20 0x558f2a8dafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:25.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.775+0000 7fcbfce05640 1 --2- 172.21.15.120:0/1636109044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f2a8b5b40 0x558f2a8cf720 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcbe809c360 tx=0x7fcbe8009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad663559ecdb3912 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:25.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.776+0000 7fcbee7fc640 1 -- 172.21.15.120:0/1636109044 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe80b5050 con 0x558f2a8b5b40 2024-09-19T09:08:25.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.776+0000 7fcbee7fc640 1 -- 172.21.15.120:0/1636109044 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbe809cd60 con 0x558f2a8b5b40 2024-09-19T09:08:25.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.776+0000 7fcbee7fc640 1 -- 172.21.15.120:0/1636109044 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe80a0a10 con 0x558f2a8b5b40 2024-09-19T09:08:25.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.776+0000 7fcc01442740 1 -- 172.21.15.120:0/1636109044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f2a8b5b40 msgr2=0x558f2a8cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:25.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.776+0000 7fcc01442740 1 --2- 172.21.15.120:0/1636109044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f2a8b5b40 0x558f2a8cf720 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcbe809c360 tx=0x7fcbe8009220 comp rx=0 tx=0).stop 2024-09-19T09:08:25.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.776+0000 7fcc01442740 1 -- 172.21.15.120:0/1636109044 shutdown_connections 2024-09-19T09:08:25.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.776+0000 7fcc01442740 1 --2- 172.21.15.120:0/1636109044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f2a8b0f20 0x558f2a8dafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.776+0000 7fcc01442740 1 --2- 172.21.15.120:0/1636109044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f2a8bf370 0x558f2a8afc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.776+0000 7fcc01442740 1 --2- 172.21.15.120:0/1636109044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f2a8b5b40 0x558f2a8cf720 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.776+0000 7fcc01442740 1 -- 172.21.15.120:0/1636109044 >> 172.21.15.120:0/1636109044 conn(0x558f2a87d310 msgr2=0x558f2a8af500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:25.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.776+0000 7fcc01442740 1 -- 172.21.15.120:0/1636109044 shutdown_connections 2024-09-19T09:08:25.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.776+0000 7fcc01442740 1 -- 172.21.15.120:0/1636109044 wait complete. 2024-09-19T09:08:25.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.777+0000 7fcc01442740 1 -- start start 2024-09-19T09:08:25.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.777+0000 7fcc01442740 1 Processor -- start 2024-09-19T09:08:25.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.777+0000 7fcc01442740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f2a8b0f20 0x558f2a8cb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:25.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.777+0000 7fcc01442740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f2a8b5b40 0x558f2a8cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:25.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.777+0000 7fcc01442740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f2a8bf370 0x558f2a8c8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:25.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.777+0000 7fcc01442740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf81ac8f0 con 0x558f2a8b0f20 2024-09-19T09:08:25.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.777+0000 7fcc01442740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf81ac220 con 0x558f2a8b5b40 2024-09-19T09:08:25.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.777+0000 7fcc01442740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf81a9440 con 0x558f2a8bf370 2024-09-19T09:08:25.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.777+0000 7fcc01442740 10 client.0 _ll_register_callbacks cb 0x7fcbe8012f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:08:25.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.777+0000 7fcbfce05640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f2a8b0f20 0x558f2a8cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:25.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.777+0000 7fcbf7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f2a8b5b40 0x558f2a8cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:25.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.777+0000 7fcbfce05640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f2a8b0f20 0x558f2a8cb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47074/0 (socket says 172.21.15.120:47074) 2024-09-19T09:08:25.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.777+0000 7fcbfd606640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f2a8bf370 0x558f2a8c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:25.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.777+0000 7fcbf7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f2a8b5b40 0x558f2a8cbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:39588/0 (socket says 172.21.15.120:39588) 2024-09-19T09:08:25.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.777+0000 7fcbfce05640 1 -- 172.21.15.120:0/1605404139 learned_addr learned my addr 172.21.15.120:0/1605404139 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:25.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.778+0000 7fcbfce05640 1 -- 172.21.15.120:0/1605404139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f2a8bf370 msgr2=0x558f2a8c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:25.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.778+0000 7fcbfce05640 1 --2- 172.21.15.120:0/1605404139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f2a8bf370 0x558f2a8c8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.778+0000 7fcbfce05640 1 -- 172.21.15.120:0/1605404139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f2a8b5b40 msgr2=0x558f2a8cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:25.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.778+0000 7fcbfce05640 1 --2- 172.21.15.120:0/1605404139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f2a8b5b40 0x558f2a8cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.778+0000 7fcbfce05640 1 -- 172.21.15.120:0/1605404139 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f2a8c8da0 con 0x558f2a8b0f20 2024-09-19T09:08:25.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.778+0000 7fcbfce05640 1 --2- 172.21.15.120:0/1605404139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f2a8b0f20 0x558f2a8cb890 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8098fe0 tx=0x7fcbe809c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a16338386c7dcfb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:25.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.778+0000 7fcbd7fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:08:25.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.778+0000 7fcbd7fff640 1 -- 172.21.15.120:0/1605404139 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe80b5050 con 0x558f2a8b0f20 2024-09-19T09:08:25.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.778+0000 7fcbd7fff640 1 -- 172.21.15.120:0/1605404139 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fcbe8098040 con 0x558f2a8b0f20 2024-09-19T09:08:25.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.778+0000 7fcc01442740 1 -- 172.21.15.120:0/1605404139 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbe809cd60 con 0x558f2a8b0f20 2024-09-19T09:08:25.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.778+0000 7fcbd7fff640 1 -- 172.21.15.120:0/1605404139 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcbe80a52d0 con 0x558f2a8b0f20 2024-09-19T09:08:25.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.778+0000 7fcc01442740 10 client.17418 fetch_fsmap learned FSMap version 30 2024-09-19T09:08:25.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.778+0000 7fcc01442740 1 -- 172.21.15.120:0/1605404139 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558f2a8c8da0 con 0x558f2a8b0f20 2024-09-19T09:08:25.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.778+0000 7fcbd7fff640 1 -- 172.21.15.120:0/1605404139 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbe809cd60 con 0x558f2a8b0f20 2024-09-19T09:08:25.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.779+0000 7fcbd7fff640 1 -- 172.21.15.120:0/1605404139 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fcbe80a3200 con 0x558f2a8b0f20 2024-09-19T09:08:25.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.779+0000 7fcc01442740 10 client.17418 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:08:25.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.779+0000 7fcc01442740 10 client.17418 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:08:25.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.779+0000 7fcc01442740 20 client.17418 populate_metadata read hostname 'smithi120' 2024-09-19T09:08:25.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.779+0000 7fcc01442740 1 --2- 172.21.15.120:0/1605404139 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x558f2a93e7c0 0x558f2a940bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:25.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.779+0000 7fcc01442740 4 client.17418 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:08:25.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.779+0000 7fcc01442740 1 -- 172.21.15.120:0/1605404139 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f2a821d70 con 0x558f2a93e7c0 2024-09-19T09:08:25.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.779+0000 7fcbf7fff640 1 --2- 172.21.15.120:0/1605404139 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x558f2a93e7c0 0x558f2a940bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:25.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.779+0000 7fcbf7fff640 1 --2- 172.21.15.120:0/1605404139 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x558f2a93e7c0 0x558f2a940bb0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a5320e7a5de353e7 server_cookie=aef2edf461ffa6d2 in_seq=0 out_seq=0 2024-09-19T09:08:25.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.779+0000 7fcbd7fff640 10 client.17418 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:08:25.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.779+0000 7fcbd7fff640 1 -- 172.21.15.120:0/1605404139 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f2a821d70 con 0x558f2a93e7c0 2024-09-19T09:08:25.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.779+0000 7fcbd7fff640 10 client.17418 handle_command_reply: tid=0 2024-09-19T09:08:25.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.779+0000 7fcc01442740 1 client.17418 shutdown 2024-09-19T09:08:25.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.779+0000 7fcc01442740 1 -- 172.21.15.120:0/1605404139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f2a8b0f20 msgr2=0x558f2a8cb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:25.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.779+0000 7fcc01442740 1 --2- 172.21.15.120:0/1605404139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f2a8b0f20 0x558f2a8cb890 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8098fe0 tx=0x7fcbe809c760 comp rx=0 tx=0).stop 2024-09-19T09:08:25.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.780+0000 7fcc01442740 1 -- 172.21.15.120:0/1605404139 shutdown_connections 2024-09-19T09:08:25.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.780+0000 7fcc01442740 1 --2- 172.21.15.120:0/1605404139 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x558f2a93e7c0 0x558f2a940bb0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.780+0000 7fcc01442740 1 --2- 172.21.15.120:0/1605404139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f2a8bf370 0x558f2a8c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.780+0000 7fcbd7fff640 0 client.17418 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:08:25.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.780+0000 7fcc01442740 1 --2- 172.21.15.120:0/1605404139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f2a8b5b40 0x558f2a8cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.780+0000 7fcc01442740 1 --2- 172.21.15.120:0/1605404139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f2a8b0f20 0x558f2a8cb890 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.780+0000 7fcc01442740 1 -- 172.21.15.120:0/1605404139 >> 172.21.15.120:0/1605404139 conn(0x558f2a87d310 msgr2=0x558f2a8afd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:25.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.780+0000 7fcc01442740 1 -- 172.21.15.120:0/1605404139 shutdown_connections 2024-09-19T09:08:25.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.780+0000 7fcc01442740 1 -- 172.21.15.120:0/1605404139 wait complete. 2024-09-19T09:08:25.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.780+0000 7fcc01442740 20 client.17418 trim_cache size 0 max 16384 2024-09-19T09:08:25.814 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:08:25.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.780+0000 7fcbff069640 1 -- 172.21.15.120:0/2322935918 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcbe40a32c0 msgr2=0x7fcbe40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:25.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.780+0000 7fcbff069640 1 --2- 172.21.15.120:0/2322935918 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcbe40a32c0 0x7fcbe40a5780 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0020000 tx=0x7fcbf001c000 comp rx=0 tx=0).stop 2024-09-19T09:08:25.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.780+0000 7fcbff069640 1 -- 172.21.15.120:0/2322935918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbf8102d30 msgr2=0x7fcbf81985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:25.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.780+0000 7fcbff069640 1 --2- 172.21.15.120:0/2322935918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbf8102d30 0x7fcbf81985e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fcbe800eb90 tx=0x7fcbe800c6a0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.780+0000 7fcbff069640 1 -- 172.21.15.120:0/2322935918 shutdown_connections 2024-09-19T09:08:25.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.780+0000 7fcbff069640 1 --2- 172.21.15.120:0/2322935918 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcbe40a32c0 0x7fcbe40a5780 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.780+0000 7fcbff069640 1 --2- 172.21.15.120:0/2322935918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbf81082a0 0x7fcbf819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.780+0000 7fcbff069640 1 --2- 172.21.15.120:0/2322935918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbf81036e0 0x7fcbf8198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.780+0000 7fcbff069640 1 --2- 172.21.15.120:0/2322935918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcbf8102d30 0x7fcbf81985e0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:25.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.780+0000 7fcbff069640 1 -- 172.21.15.120:0/2322935918 >> 172.21.15.120:0/2322935918 conn(0x7fcbf80fc7d0 msgr2=0x7fcbf8105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:25.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.781+0000 7fcbff069640 1 -- 172.21.15.120:0/2322935918 shutdown_connections 2024-09-19T09:08:25.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:25.781+0000 7fcbff069640 1 -- 172.21.15.120:0/2322935918 wait complete. 2024-09-19T09:08:26.050 INFO:tasks.workunit.client.0.smithi120.stdout: 1 413840 7.10 MB/sec execute 141 sec latency 11.607 ms 2024-09-19T09:08:26.701 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:08:26.966 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:08:27.050 INFO:tasks.workunit.client.0.smithi120.stdout: 1 416247 7.15 MB/sec execute 142 sec latency 12.502 ms 2024-09-19T09:08:27.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:26 smithi120 ceph-mon[31150]: pgmap v509: 97 pgs: 97 active+clean; 34 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.7 MiB/s wr, 307 op/s 2024-09-19T09:08:27.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:26 smithi196 ceph-mon[35150]: pgmap v509: 97 pgs: 97 active+clean; 34 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.7 MiB/s wr, 307 op/s 2024-09-19T09:08:27.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:26 smithi153 ceph-mon[35071]: pgmap v509: 97 pgs: 97 active+clean; 34 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.7 MiB/s wr, 307 op/s 2024-09-19T09:08:27.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.549+0000 7fdfbbaad640 1 -- 172.21.15.120:0/3062021569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfb4102ef0 msgr2=0x7fdfb410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:27.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.549+0000 7fdfbbaad640 1 --2- 172.21.15.120:0/3062021569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfb4102ef0 0x7fdfb410da40 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fdfac009ca0 tx=0x7fdfac02a950 comp rx=0 tx=0).stop 2024-09-19T09:08:27.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.549+0000 7fdfbbaad640 1 -- 172.21.15.120:0/3062021569 shutdown_connections 2024-09-19T09:08:27.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.549+0000 7fdfbbaad640 1 --2- 172.21.15.120:0/3062021569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfb4102ef0 0x7fdfb410da40 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.549+0000 7fdfbbaad640 1 --2- 172.21.15.120:0/3062021569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfb4102570 0x7fdfb41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.549+0000 7fdfbbaad640 1 --2- 172.21.15.120:0/3062021569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdfb4100e20 0x7fdfb4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.549+0000 7fdfbbaad640 1 -- 172.21.15.120:0/3062021569 >> 172.21.15.120:0/3062021569 conn(0x7fdfb40fc7d0 msgr2=0x7fdfb40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:27.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.550+0000 7fdfbbaad640 1 -- 172.21.15.120:0/3062021569 shutdown_connections 2024-09-19T09:08:27.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.550+0000 7fdfbbaad640 1 -- 172.21.15.120:0/3062021569 wait complete. 2024-09-19T09:08:27.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.550+0000 7fdfbbaad640 1 Processor -- start 2024-09-19T09:08:27.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.550+0000 7fdfbbaad640 1 -- start start 2024-09-19T09:08:27.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.550+0000 7fdfbbaad640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfb4100e20 0x7fdfb41970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:27.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.550+0000 7fdfbbaad640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfb4102570 0x7fdfb41975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:27.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.550+0000 7fdfbbaad640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdfb4102ef0 0x7fdfb419c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:27.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.550+0000 7fdfbbaad640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb4110f80 con 0x7fdfb4102570 2024-09-19T09:08:27.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.550+0000 7fdfbbaad640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb4110e00 con 0x7fdfb4102ef0 2024-09-19T09:08:27.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.550+0000 7fdfbbaad640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb4111100 con 0x7fdfb4100e20 2024-09-19T09:08:27.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.551+0000 7fdfb9048640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfb4102570 0x7fdfb41975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:27.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.551+0000 7fdfb9849640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfb4100e20 0x7fdfb41970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:27.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.551+0000 7fdfb9048640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfb4102570 0x7fdfb41975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47102/0 (socket says 172.21.15.120:47102) 2024-09-19T09:08:27.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.551+0000 7fdfb9048640 1 -- 172.21.15.120:0/1409464267 learned_addr learned my addr 172.21.15.120:0/1409464267 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:27.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.551+0000 7fdfba04a640 1 --2- 172.21.15.120:0/1409464267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdfb4102ef0 0x7fdfb419c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:27.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.551+0000 7fdfb9048640 1 -- 172.21.15.120:0/1409464267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfb4100e20 msgr2=0x7fdfb41970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:27.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.551+0000 7fdfb9048640 1 --2- 172.21.15.120:0/1409464267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfb4100e20 0x7fdfb41970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.551+0000 7fdfb9048640 1 -- 172.21.15.120:0/1409464267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdfb4102ef0 msgr2=0x7fdfb419c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:27.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.551+0000 7fdfb9048640 1 --2- 172.21.15.120:0/1409464267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdfb4102ef0 0x7fdfb419c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.551+0000 7fdfb9048640 1 -- 172.21.15.120:0/1409464267 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb419c630 con 0x7fdfb4102570 2024-09-19T09:08:27.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.551+0000 7fdfb9849640 1 --2- 172.21.15.120:0/1409464267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfb4100e20 0x7fdfb41970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:27.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.551+0000 7fdfb9048640 1 --2- 172.21.15.120:0/1409464267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfb4102570 0x7fdfb41975e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fdfa400e9f0 tx=0x7fdfa400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:27.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.551+0000 7fdfaaffd640 1 -- 172.21.15.120:0/1409464267 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa400cd80 con 0x7fdfb4102570 2024-09-19T09:08:27.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.551+0000 7fdfbbaad640 1 -- 172.21.15.120:0/1409464267 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb419c920 con 0x7fdfb4102570 2024-09-19T09:08:27.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.551+0000 7fdfaaffd640 1 -- 172.21.15.120:0/1409464267 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfa4010910 con 0x7fdfb4102570 2024-09-19T09:08:27.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.551+0000 7fdfaaffd640 1 -- 172.21.15.120:0/1409464267 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4018990 con 0x7fdfb4102570 2024-09-19T09:08:27.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.551+0000 7fdfbbaad640 1 -- 172.21.15.120:0/1409464267 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb419ce60 con 0x7fdfb4102570 2024-09-19T09:08:27.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.553+0000 7fdfaaffd640 1 -- 172.21.15.120:0/1409464267 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fdfa4010ac0 con 0x7fdfb4102570 2024-09-19T09:08:27.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.554+0000 7fdfbde86740 1 Processor -- start 2024-09-19T09:08:27.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.554+0000 7fdfbde86740 1 -- start start 2024-09-19T09:08:27.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.555+0000 7fdfbde86740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8f5098b40 0x55e8f5097090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:27.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.555+0000 7fdfbde86740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8f50975d0 0x55e8f5097a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:27.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.555+0000 7fdfbde86740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f50a2370 0x55e8f50a4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:27.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.555+0000 7fdfaaffd640 1 --2- 172.21.15.120:0/1409464267 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdfa00a3240 0x7fdfa00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:27.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.555+0000 7fdfbde86740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e8f4ffd2f0 con 0x55e8f50975d0 2024-09-19T09:08:27.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.555+0000 7fdfbde86740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e8f4fa4cf0 con 0x55e8f5098b40 2024-09-19T09:08:27.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.555+0000 7fdfbde86740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e8f4ff92b0 con 0x55e8f50a2370 2024-09-19T09:08:27.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.555+0000 7fdfb9849640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8f50975d0 0x55e8f5097a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:27.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.555+0000 7fdfba04a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8f5098b40 0x55e8f5097090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:27.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.555+0000 7fdfba04a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8f5098b40 0x55e8f5097090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:39618/0 (socket says 172.21.15.120:39618) 2024-09-19T09:08:27.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.555+0000 7fdfba04a640 1 -- 172.21.15.120:0/900160583 learned_addr learned my addr 172.21.15.120:0/900160583 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:27.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.555+0000 7fdfb9849640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8f50975d0 0x55e8f5097a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47110/0 (socket says 172.21.15.120:47110) 2024-09-19T09:08:27.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.555+0000 7fdfaaffd640 1 -- 172.21.15.120:0/1409464267 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fdfa40cfbe0 con 0x7fdfb4102570 2024-09-19T09:08:27.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.555+0000 7fdfba04a640 1 --2- 172.21.15.120:0/1409464267 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdfa00a3240 0x7fdfa00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:27.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.555+0000 7fdfb9048640 1 --2- 172.21.15.120:0/900160583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f50a2370 0x55e8f50a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:27.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.555+0000 7fdfb9849640 1 -- 172.21.15.120:0/900160583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f50a2370 msgr2=0x55e8f50a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:27.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.555+0000 7fdfb9849640 1 --2- 172.21.15.120:0/900160583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f50a2370 0x55e8f50a4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.555+0000 7fdfb9849640 1 -- 172.21.15.120:0/900160583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8f5098b40 msgr2=0x55e8f5097090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:27.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.555+0000 7fdfb9849640 1 --2- 172.21.15.120:0/900160583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8f5098b40 0x55e8f5097090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.555+0000 7fdfb9849640 1 -- 172.21.15.120:0/900160583 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8f4f893a0 con 0x55e8f50975d0 2024-09-19T09:08:27.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.555+0000 7fdfb9849640 1 --2- 172.21.15.120:0/900160583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8f50975d0 0x55e8f5097a10 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009fd0 tx=0x7fdfb00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f58bfcc79ea32 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:27.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.555+0000 7fdfba04a640 1 --2- 172.21.15.120:0/1409464267 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdfa00a3240 0x7fdfa00a5700 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdfac005fe0 tx=0x7fdfac005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:27.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.556+0000 7fdf9ffff640 1 -- 172.21.15.120:0/900160583 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0004030 con 0x55e8f50975d0 2024-09-19T09:08:27.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.556+0000 7fdf9ffff640 1 -- 172.21.15.120:0/900160583 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfb0005500 con 0x55e8f50975d0 2024-09-19T09:08:27.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.556+0000 7fdf9ffff640 1 -- 172.21.15.120:0/900160583 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0004e10 con 0x55e8f50975d0 2024-09-19T09:08:27.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.556+0000 7fdfbde86740 1 -- 172.21.15.120:0/900160583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8f50975d0 msgr2=0x55e8f5097a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:27.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.556+0000 7fdfbde86740 1 --2- 172.21.15.120:0/900160583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8f50975d0 0x55e8f5097a10 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009fd0 tx=0x7fdfb00118d0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.556+0000 7fdfbde86740 1 -- 172.21.15.120:0/900160583 shutdown_connections 2024-09-19T09:08:27.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.556+0000 7fdfbde86740 1 --2- 172.21.15.120:0/900160583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f50a2370 0x55e8f50a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.556+0000 7fdfbde86740 1 --2- 172.21.15.120:0/900160583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8f50975d0 0x55e8f5097a10 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.556+0000 7fdfbde86740 1 --2- 172.21.15.120:0/900160583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8f5098b40 0x55e8f5097090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.556+0000 7fdfbde86740 1 -- 172.21.15.120:0/900160583 >> 172.21.15.120:0/900160583 conn(0x55e8f5060310 msgr2=0x55e8f5093c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:27.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.557+0000 7fdfbde86740 1 -- 172.21.15.120:0/900160583 shutdown_connections 2024-09-19T09:08:27.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.557+0000 7fdfbde86740 1 -- 172.21.15.120:0/900160583 wait complete. 2024-09-19T09:08:27.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.557+0000 7fdfbde86740 1 -- start start 2024-09-19T09:08:27.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.557+0000 7fdfbde86740 1 Processor -- start 2024-09-19T09:08:27.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.558+0000 7fdfbde86740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8f50975d0 0x55e8f50a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:27.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.558+0000 7fdfbde86740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f5098b40 0x55e8f509ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:27.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.558+0000 7fdfbde86740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8f50a2370 0x55e8f509f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:27.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.558+0000 7fdfbde86740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e8f50a5070 con 0x55e8f50975d0 2024-09-19T09:08:27.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.558+0000 7fdfbde86740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e8f4fb8060 con 0x55e8f50a2370 2024-09-19T09:08:27.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.558+0000 7fdfbde86740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e8f50a51f0 con 0x55e8f5098b40 2024-09-19T09:08:27.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.558+0000 7fdfbde86740 10 client.0 _ll_register_callbacks cb 0x55e8f4d0b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:08:27.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.558+0000 7fdfb9849640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8f50975d0 0x55e8f50a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:27.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.558+0000 7fdfb9849640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8f50975d0 0x55e8f50a1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47126/0 (socket says 172.21.15.120:47126) 2024-09-19T09:08:27.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.558+0000 7fdfba04a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8f50a2370 0x55e8f509f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:27.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.558+0000 7fdfb9048640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f5098b40 0x55e8f509ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:27.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.558+0000 7fdfb9849640 1 -- 172.21.15.120:0/1646066492 learned_addr learned my addr 172.21.15.120:0/1646066492 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:27.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.558+0000 7fdfb9048640 1 -- 172.21.15.120:0/1646066492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8f50a2370 msgr2=0x55e8f509f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:27.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.558+0000 7fdfb9048640 1 --2- 172.21.15.120:0/1646066492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8f50a2370 0x55e8f509f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.558+0000 7fdfb9048640 1 -- 172.21.15.120:0/1646066492 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8f50975d0 msgr2=0x55e8f50a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:27.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.558+0000 7fdfb9048640 1 --2- 172.21.15.120:0/1646066492 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8f50975d0 0x55e8f50a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.558+0000 7fdfb9048640 1 -- 172.21.15.120:0/1646066492 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8f4d6e3c0 con 0x55e8f5098b40 2024-09-19T09:08:27.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.558+0000 7fdfb9849640 1 --2- 172.21.15.120:0/1646066492 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8f50975d0 0x55e8f50a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:27.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.558+0000 7fdfb9048640 1 --2- 172.21.15.120:0/1646066492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f5098b40 0x55e8f509ef90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdfa400e9c0 tx=0x7fdfa409bbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=547c4e9e882b3293 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:27.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.558+0000 7fdf9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:08:27.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.559+0000 7fdf9d7fa640 1 -- 172.21.15.120:0/1646066492 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa409c270 con 0x55e8f5098b40 2024-09-19T09:08:27.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.559+0000 7fdf9d7fa640 1 -- 172.21.15.120:0/1646066492 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fdfa40bc0b0 con 0x55e8f5098b40 2024-09-19T09:08:27.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.559+0000 7fdfbde86740 1 -- 172.21.15.120:0/1646066492 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfb0005500 con 0x55e8f5098b40 2024-09-19T09:08:27.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.559+0000 7fdf9d7fa640 1 -- 172.21.15.120:0/1646066492 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdfa40bc2b0 con 0x55e8f5098b40 2024-09-19T09:08:27.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.559+0000 7fdfbde86740 10 client.26608 fetch_fsmap learned FSMap version 30 2024-09-19T09:08:27.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.559+0000 7fdfbde86740 1 -- 172.21.15.120:0/1646066492 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e8f50b0ae0 con 0x55e8f5098b40 2024-09-19T09:08:27.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.560+0000 7fdf9d7fa640 1 -- 172.21.15.120:0/1646066492 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfb0005500 con 0x55e8f5098b40 2024-09-19T09:08:27.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.560+0000 7fdf9d7fa640 1 -- 172.21.15.120:0/1646066492 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fdfa40cdb60 con 0x55e8f5098b40 2024-09-19T09:08:27.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.560+0000 7fdfbde86740 10 client.26608 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:08:27.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.560+0000 7fdfbde86740 10 client.26608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:08:27.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.560+0000 7fdfbde86740 20 client.26608 populate_metadata read hostname 'smithi120' 2024-09-19T09:08:27.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.560+0000 7fdfbde86740 1 --2- 172.21.15.120:0/1646066492 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e8f50b1ed0 0x55e8f50b42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:27.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.560+0000 7fdfbde86740 4 client.26608 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:08:27.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.560+0000 7fdfbde86740 1 -- 172.21.15.120:0/1646066492 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8f500a860 con 0x55e8f50b1ed0 2024-09-19T09:08:27.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.560+0000 7fdfb9849640 1 --2- 172.21.15.120:0/1646066492 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e8f50b1ed0 0x55e8f50b42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:27.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.560+0000 7fdf9d7fa640 10 client.26608 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:27.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.560+0000 7fdfb9849640 1 --2- 172.21.15.120:0/1646066492 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e8f50b1ed0 0x55e8f50b42e0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59c08f677b0ccd14 server_cookie=4dd808a78ce18d5d in_seq=0 out_seq=0 2024-09-19T09:08:27.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.562+0000 7fdf9d7fa640 1 -- 172.21.15.120:0/1646066492 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e8f500a860 con 0x55e8f50b1ed0 2024-09-19T09:08:27.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.562+0000 7fdf9d7fa640 10 client.26608 handle_command_reply: tid=0 2024-09-19T09:08:27.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.562+0000 7fdfbde86740 1 client.26608 shutdown 2024-09-19T09:08:27.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.562+0000 7fdfbde86740 1 -- 172.21.15.120:0/1646066492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f5098b40 msgr2=0x55e8f509ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:27.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.562+0000 7fdfbde86740 1 --2- 172.21.15.120:0/1646066492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f5098b40 0x55e8f509ef90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdfa400e9c0 tx=0x7fdfa409bbd0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.562+0000 7fdfbde86740 1 -- 172.21.15.120:0/1646066492 shutdown_connections 2024-09-19T09:08:27.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.562+0000 7fdfbde86740 1 --2- 172.21.15.120:0/1646066492 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e8f50b1ed0 0x55e8f50b42e0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.562+0000 7fdfbde86740 1 --2- 172.21.15.120:0/1646066492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8f50a2370 0x55e8f509f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.562+0000 7fdfbde86740 1 --2- 172.21.15.120:0/1646066492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f5098b40 0x55e8f509ef90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.562+0000 7fdfbde86740 1 --2- 172.21.15.120:0/1646066492 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8f50975d0 0x55e8f50a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.562+0000 7fdf9d7fa640 0 client.26608 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:27.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.562+0000 7fdfbde86740 1 -- 172.21.15.120:0/1646066492 >> 172.21.15.120:0/1646066492 conn(0x55e8f5060310 msgr2=0x55e8f5099960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:27.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.562+0000 7fdfbde86740 1 -- 172.21.15.120:0/1646066492 shutdown_connections 2024-09-19T09:08:27.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.562+0000 7fdfbde86740 1 -- 172.21.15.120:0/1646066492 wait complete. 2024-09-19T09:08:27.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.562+0000 7fdfbde86740 20 client.26608 trim_cache size 0 max 16384 2024-09-19T09:08:27.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.577+0000 7fdfbde86740 1 Processor -- start 2024-09-19T09:08:27.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.577+0000 7fdfbde86740 1 -- start start 2024-09-19T09:08:27.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.577+0000 7fdfbde86740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8f50975d0 0x55e8f5093e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:27.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.577+0000 7fdfbde86740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f5098b40 0x55e8f50ae310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:27.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.577+0000 7fdfbde86740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8f50a2370 0x55e8f50b9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:27.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.577+0000 7fdfbde86740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41b3b70 con 0x55e8f50a2370 2024-09-19T09:08:27.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.577+0000 7fdfbde86740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41b56b0 con 0x55e8f50975d0 2024-09-19T09:08:27.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.577+0000 7fdfbde86740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41b4240 con 0x55e8f5098b40 2024-09-19T09:08:27.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.577+0000 7fdfba04a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8f50a2370 0x55e8f50b9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:27.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.577+0000 7fdfb9849640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8f50975d0 0x55e8f5093e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:27.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.577+0000 7fdfb9849640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8f50975d0 0x55e8f5093e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:39644/0 (socket says 172.21.15.120:39644) 2024-09-19T09:08:27.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.577+0000 7fdfb9849640 1 -- 172.21.15.120:0/1616777796 learned_addr learned my addr 172.21.15.120:0/1616777796 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:27.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.577+0000 7fdfb9048640 1 --2- 172.21.15.120:0/1616777796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f5098b40 0x55e8f50ae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:27.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.578+0000 7fdfb9849640 1 -- 172.21.15.120:0/1616777796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f5098b40 msgr2=0x55e8f50ae310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:27.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.578+0000 7fdfb9849640 1 --2- 172.21.15.120:0/1616777796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f5098b40 0x55e8f50ae310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.578+0000 7fdfb9849640 1 -- 172.21.15.120:0/1616777796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8f50a2370 msgr2=0x55e8f50b9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:27.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.578+0000 7fdfb9849640 1 --2- 172.21.15.120:0/1616777796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8f50a2370 0x55e8f50b9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.578+0000 7fdfb9849640 1 -- 172.21.15.120:0/1616777796 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8f5094960 con 0x55e8f50975d0 2024-09-19T09:08:27.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.578+0000 7fdfb9849640 1 --2- 172.21.15.120:0/1616777796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8f50975d0 0x55e8f5093e10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0004910 tx=0x7fdfb0004630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92afcf355ddca7f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:27.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.578+0000 7fdf9f7fe640 1 -- 172.21.15.120:0/1616777796 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0032070 con 0x55e8f50975d0 2024-09-19T09:08:27.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.578+0000 7fdf9f7fe640 1 -- 172.21.15.120:0/1616777796 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfb0003ad0 con 0x55e8f50975d0 2024-09-19T09:08:27.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.578+0000 7fdf9f7fe640 1 -- 172.21.15.120:0/1616777796 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb000f7e0 con 0x55e8f50975d0 2024-09-19T09:08:27.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.579+0000 7fdfbde86740 1 -- 172.21.15.120:0/1616777796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8f50975d0 msgr2=0x55e8f5093e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:27.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.579+0000 7fdfbde86740 1 --2- 172.21.15.120:0/1616777796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8f50975d0 0x55e8f5093e10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0004910 tx=0x7fdfb0004630 comp rx=0 tx=0).stop 2024-09-19T09:08:27.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.579+0000 7fdfbde86740 1 -- 172.21.15.120:0/1616777796 shutdown_connections 2024-09-19T09:08:27.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.579+0000 7fdfbde86740 1 --2- 172.21.15.120:0/1616777796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8f50a2370 0x55e8f50b9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.579+0000 7fdfbde86740 1 --2- 172.21.15.120:0/1616777796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f5098b40 0x55e8f50ae310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.579+0000 7fdfbde86740 1 --2- 172.21.15.120:0/1616777796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8f50975d0 0x55e8f5093e10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.579+0000 7fdfbde86740 1 -- 172.21.15.120:0/1616777796 >> 172.21.15.120:0/1616777796 conn(0x55e8f5060310 msgr2=0x55e8f5093700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:27.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.579+0000 7fdfbde86740 1 -- 172.21.15.120:0/1616777796 shutdown_connections 2024-09-19T09:08:27.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.579+0000 7fdfbde86740 1 -- 172.21.15.120:0/1616777796 wait complete. 2024-09-19T09:08:27.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.579+0000 7fdfbde86740 1 -- start start 2024-09-19T09:08:27.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.579+0000 7fdfbde86740 1 Processor -- start 2024-09-19T09:08:27.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.580+0000 7fdfbde86740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8f50975d0 0x55e8f50aac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:27.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.580+0000 7fdfbde86740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f5098b40 0x55e8f509bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:27.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.580+0000 7fdfbde86740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8f50a2370 0x55e8f509c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:27.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.580+0000 7fdfbde86740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41b4fe0 con 0x55e8f50975d0 2024-09-19T09:08:27.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.580+0000 7fdfbde86740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41b4910 con 0x55e8f50a2370 2024-09-19T09:08:27.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.580+0000 7fdfbde86740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41b1b30 con 0x55e8f5098b40 2024-09-19T09:08:27.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.580+0000 7fdfb9849640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8f50975d0 0x55e8f50aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:27.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.580+0000 7fdfbde86740 10 client.0 _ll_register_callbacks cb 0x7fdfa40985c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:08:27.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.580+0000 7fdfb9849640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8f50975d0 0x55e8f50aac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47150/0 (socket says 172.21.15.120:47150) 2024-09-19T09:08:27.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.580+0000 7fdfb9849640 1 -- 172.21.15.120:0/2571697016 learned_addr learned my addr 172.21.15.120:0/2571697016 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:27.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.580+0000 7fdfba04a640 1 --2- 172.21.15.120:0/2571697016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8f50a2370 0x55e8f509c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:27.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.580+0000 7fdfb9048640 1 --2- 172.21.15.120:0/2571697016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f5098b40 0x55e8f509bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:27.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.580+0000 7fdfb9849640 1 -- 172.21.15.120:0/2571697016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f5098b40 msgr2=0x55e8f509bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:27.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.580+0000 7fdfb9849640 1 --2- 172.21.15.120:0/2571697016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f5098b40 0x55e8f509bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.580+0000 7fdfb9849640 1 -- 172.21.15.120:0/2571697016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8f50a2370 msgr2=0x55e8f509c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:27.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.580+0000 7fdfb9849640 1 --2- 172.21.15.120:0/2571697016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8f50a2370 0x55e8f509c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.580+0000 7fdfb9849640 1 -- 172.21.15.120:0/2571697016 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8f509ca40 con 0x55e8f50975d0 2024-09-19T09:08:27.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.580+0000 7fdfb9849640 1 --2- 172.21.15.120:0/2571697016 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8f50975d0 0x55e8f50aac60 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0013040 tx=0x7fdfb00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744bed2c7852974 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:27.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.580+0000 7fdf9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:08:27.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.581+0000 7fdf9d7fa640 1 -- 172.21.15.120:0/2571697016 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0032070 con 0x55e8f50975d0 2024-09-19T09:08:27.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.581+0000 7fdf9d7fa640 1 -- 172.21.15.120:0/2571697016 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fdfb00037f0 con 0x55e8f50975d0 2024-09-19T09:08:27.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.581+0000 7fdfbde86740 1 -- 172.21.15.120:0/2571697016 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfb0003ad0 con 0x55e8f50975d0 2024-09-19T09:08:27.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.581+0000 7fdf9d7fa640 1 -- 172.21.15.120:0/2571697016 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdfb0003e90 con 0x55e8f50975d0 2024-09-19T09:08:27.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.581+0000 7fdfbde86740 10 client.17436 fetch_fsmap learned FSMap version 30 2024-09-19T09:08:27.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.581+0000 7fdfbde86740 1 -- 172.21.15.120:0/2571697016 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e8f511cb80 con 0x55e8f50975d0 2024-09-19T09:08:27.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.581+0000 7fdf9d7fa640 1 -- 172.21.15.120:0/2571697016 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfb0003ad0 con 0x55e8f50975d0 2024-09-19T09:08:27.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.581+0000 7fdf9d7fa640 1 -- 172.21.15.120:0/2571697016 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fdfb00196b0 con 0x55e8f50975d0 2024-09-19T09:08:27.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.581+0000 7fdfbde86740 10 client.17436 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:08:27.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.581+0000 7fdfbde86740 10 client.17436 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:08:27.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.582+0000 7fdfbde86740 20 client.17436 populate_metadata read hostname 'smithi120' 2024-09-19T09:08:27.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.582+0000 7fdfbde86740 1 --2- 172.21.15.120:0/2571697016 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e8f511d050 0x55e8f511f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:27.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.582+0000 7fdfbde86740 4 client.17436 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:08:27.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.582+0000 7fdfbde86740 1 -- 172.21.15.120:0/2571697016 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e8f5004d70 con 0x55e8f511d050 2024-09-19T09:08:27.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.582+0000 7fdfb9048640 1 --2- 172.21.15.120:0/2571697016 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e8f511d050 0x55e8f511f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:27.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.582+0000 7fdfb9048640 1 --2- 172.21.15.120:0/2571697016 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e8f511d050 0x55e8f511f470 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c472d37b8bb43b server_cookie=9ded7422a913e8a8 in_seq=0 out_seq=0 2024-09-19T09:08:27.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.582+0000 7fdf9d7fa640 10 client.17436 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:27.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.582+0000 7fdf9d7fa640 1 -- 172.21.15.120:0/2571697016 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e8f5004d70 con 0x55e8f511d050 2024-09-19T09:08:27.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.582+0000 7fdf9d7fa640 10 client.17436 handle_command_reply: tid=0 2024-09-19T09:08:27.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.582+0000 7fdfbde86740 1 client.17436 shutdown 2024-09-19T09:08:27.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.583+0000 7fdfbde86740 1 -- 172.21.15.120:0/2571697016 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8f50975d0 msgr2=0x55e8f50aac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:27.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.583+0000 7fdfbde86740 1 --2- 172.21.15.120:0/2571697016 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8f50975d0 0x55e8f50aac60 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0013040 tx=0x7fdfb00118d0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.583+0000 7fdfbde86740 1 -- 172.21.15.120:0/2571697016 shutdown_connections 2024-09-19T09:08:27.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.583+0000 7fdfbde86740 1 --2- 172.21.15.120:0/2571697016 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e8f511d050 0x55e8f511f470 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.583+0000 7fdfbde86740 1 --2- 172.21.15.120:0/2571697016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8f50a2370 0x55e8f509c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.583+0000 7fdfbde86740 1 --2- 172.21.15.120:0/2571697016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8f5098b40 0x55e8f509bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.583+0000 7fdfbde86740 1 --2- 172.21.15.120:0/2571697016 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8f50975d0 0x55e8f50aac60 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.583+0000 7fdf9d7fa640 0 client.17436 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:27.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.583+0000 7fdfbde86740 1 -- 172.21.15.120:0/2571697016 >> 172.21.15.120:0/2571697016 conn(0x55e8f5060310 msgr2=0x55e8f50924e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:27.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.583+0000 7fdfbde86740 1 -- 172.21.15.120:0/2571697016 shutdown_connections 2024-09-19T09:08:27.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.583+0000 7fdfbde86740 1 -- 172.21.15.120:0/2571697016 wait complete. 2024-09-19T09:08:27.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.583+0000 7fdfbde86740 20 client.17436 trim_cache size 0 max 16384 2024-09-19T09:08:27.600 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:08:27.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.583+0000 7fdfbbaad640 1 -- 172.21.15.120:0/1409464267 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdfa00a3240 msgr2=0x7fdfa00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:27.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.583+0000 7fdfbbaad640 1 --2- 172.21.15.120:0/1409464267 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdfa00a3240 0x7fdfa00a5700 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdfac005fe0 tx=0x7fdfac005c50 comp rx=0 tx=0).stop 2024-09-19T09:08:27.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.583+0000 7fdfbbaad640 1 -- 172.21.15.120:0/1409464267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfb4102570 msgr2=0x7fdfb41975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:27.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.583+0000 7fdfbbaad640 1 --2- 172.21.15.120:0/1409464267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfb4102570 0x7fdfb41975e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fdfa400e9f0 tx=0x7fdfa400eeb0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.584+0000 7fdfbbaad640 1 -- 172.21.15.120:0/1409464267 shutdown_connections 2024-09-19T09:08:27.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.584+0000 7fdfbbaad640 1 --2- 172.21.15.120:0/1409464267 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdfa00a3240 0x7fdfa00a5700 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.584+0000 7fdfbbaad640 1 --2- 172.21.15.120:0/1409464267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdfb4102ef0 0x7fdfb419c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.584+0000 7fdfbbaad640 1 --2- 172.21.15.120:0/1409464267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfb4102570 0x7fdfb41975e0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.584+0000 7fdfbbaad640 1 --2- 172.21.15.120:0/1409464267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfb4100e20 0x7fdfb41970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:27.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.584+0000 7fdfbbaad640 1 -- 172.21.15.120:0/1409464267 >> 172.21.15.120:0/1409464267 conn(0x7fdfb40fc7d0 msgr2=0x7fdfb4106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:27.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.584+0000 7fdfbbaad640 1 -- 172.21.15.120:0/1409464267 shutdown_connections 2024-09-19T09:08:27.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:27.584+0000 7fdfbbaad640 1 -- 172.21.15.120:0/1409464267 wait complete. 2024-09-19T09:08:28.050 INFO:tasks.workunit.client.0.smithi120.stdout: 1 419203 7.22 MB/sec execute 143 sec latency 39.630 ms 2024-09-19T09:08:29.051 INFO:tasks.workunit.client.0.smithi120.stdout: 1 422553 7.31 MB/sec execute 144 sec latency 11.001 ms 2024-09-19T09:08:29.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:28 smithi120 ceph-mon[31150]: pgmap v510: 97 pgs: 97 active+clean; 35 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.6 MiB/s wr, 286 op/s 2024-09-19T09:08:29.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:28 smithi196 ceph-mon[35150]: pgmap v510: 97 pgs: 97 active+clean; 35 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.6 MiB/s wr, 286 op/s 2024-09-19T09:08:29.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:28 smithi153 ceph-mon[35071]: pgmap v510: 97 pgs: 97 active+clean; 35 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.6 MiB/s wr, 286 op/s 2024-09-19T09:08:29.525 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:08:29.525 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:08:29.807 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:08:30.051 INFO:tasks.workunit.client.0.smithi120.stdout: 1 425897 7.38 MB/sec execute 145 sec latency 17.998 ms 2024-09-19T09:08:30.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.316+0000 7f47c0f4a640 1 -- 172.21.15.120:0/200872876 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47bc106f10 msgr2=0x7f47bc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:30.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.316+0000 7f47c0f4a640 1 --2- 172.21.15.120:0/200872876 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47bc106f10 0x7f47bc10ea90 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f47b4009ca0 tx=0x7f47b402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.316+0000 7f47c0f4a640 1 -- 172.21.15.120:0/200872876 shutdown_connections 2024-09-19T09:08:30.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.316+0000 7f47c0f4a640 1 --2- 172.21.15.120:0/200872876 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47bc106f10 0x7f47bc10ea90 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.316+0000 7f47c0f4a640 1 --2- 172.21.15.120:0/200872876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47bc102790 0x7f47bc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.316+0000 7f47c0f4a640 1 --2- 172.21.15.120:0/200872876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47bc101de0 0x7f47bc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.316+0000 7f47c0f4a640 1 -- 172.21.15.120:0/200872876 >> 172.21.15.120:0/200872876 conn(0x7f47bc0fd8c0 msgr2=0x7f47bc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:30.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.316+0000 7f47c0f4a640 1 -- 172.21.15.120:0/200872876 shutdown_connections 2024-09-19T09:08:30.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.316+0000 7f47c0f4a640 1 -- 172.21.15.120:0/200872876 wait complete. 2024-09-19T09:08:30.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.317+0000 7f47c0f4a640 1 Processor -- start 2024-09-19T09:08:30.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.317+0000 7f47c0f4a640 1 -- start start 2024-09-19T09:08:30.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.317+0000 7f47c0f4a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47bc101de0 0x7f47bc1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:30.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.317+0000 7f47c0f4a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47bc102790 0x7f47bc1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:30.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.317+0000 7f47c0f4a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47bc106f10 0x7f47bc19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:30.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.317+0000 7f47c0f4a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc111f70 con 0x7f47bc101de0 2024-09-19T09:08:30.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.317+0000 7f47c0f4a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc111df0 con 0x7f47bc106f10 2024-09-19T09:08:30.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.317+0000 7f47c0f4a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc1120f0 con 0x7f47bc102790 2024-09-19T09:08:30.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.317+0000 7f47ba59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47bc101de0 0x7f47bc1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:30.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.317+0000 7f47ba59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47bc101de0 0x7f47bc1983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47178/0 (socket says 172.21.15.120:47178) 2024-09-19T09:08:30.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.317+0000 7f47ba59c640 1 -- 172.21.15.120:0/326466384 learned_addr learned my addr 172.21.15.120:0/326466384 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:30.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.317+0000 7f47bad9d640 1 --2- 172.21.15.120:0/326466384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47bc106f10 0x7f47bc19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:30.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.317+0000 7f47ba59c640 1 -- 172.21.15.120:0/326466384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47bc102790 msgr2=0x7f47bc1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:30.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.317+0000 7f47b3fff640 1 --2- 172.21.15.120:0/326466384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47bc102790 0x7f47bc1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:30.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.317+0000 7f47ba59c640 1 --2- 172.21.15.120:0/326466384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47bc102790 0x7f47bc1988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.317+0000 7f47ba59c640 1 -- 172.21.15.120:0/326466384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47bc106f10 msgr2=0x7f47bc19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:30.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.317+0000 7f47ba59c640 1 --2- 172.21.15.120:0/326466384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47bc106f10 0x7f47bc19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.317+0000 7f47ba59c640 1 -- 172.21.15.120:0/326466384 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47bc19d360 con 0x7f47bc101de0 2024-09-19T09:08:30.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.317+0000 7f47ba59c640 1 --2- 172.21.15.120:0/326466384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47bc101de0 0x7f47bc1983b0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00e9f0 tx=0x7f47ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:30.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.318+0000 7f47b37fe640 1 -- 172.21.15.120:0/326466384 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac00cd80 con 0x7f47bc101de0 2024-09-19T09:08:30.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.318+0000 7f47b37fe640 1 -- 172.21.15.120:0/326466384 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47ac010910 con 0x7f47bc101de0 2024-09-19T09:08:30.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.318+0000 7f47c0f4a640 1 -- 172.21.15.120:0/326466384 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47bc19d650 con 0x7f47bc101de0 2024-09-19T09:08:30.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.318+0000 7f47b37fe640 1 -- 172.21.15.120:0/326466384 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac018990 con 0x7f47bc101de0 2024-09-19T09:08:30.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.318+0000 7f47c0f4a640 1 -- 172.21.15.120:0/326466384 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47bc1a4f30 con 0x7f47bc101de0 2024-09-19T09:08:30.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.321+0000 7f47b37fe640 1 -- 172.21.15.120:0/326466384 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f47ac010ac0 con 0x7f47bc101de0 2024-09-19T09:08:30.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.321+0000 7f47c3323740 1 Processor -- start 2024-09-19T09:08:30.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.321+0000 7f47c3323740 1 -- start start 2024-09-19T09:08:30.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.321+0000 7f47c3323740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e0144b60 0x55b1e0144f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:30.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.321+0000 7f47c3323740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e0145480 0x55b1e014f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:30.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.321+0000 7f47c3323740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e014fd40 0x55b1e0152130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:30.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.321+0000 7f47c3323740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b1e00aa280 con 0x55b1e0145480 2024-09-19T09:08:30.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.321+0000 7f47c3323740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b1e0051b10 con 0x55b1e014fd40 2024-09-19T09:08:30.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.321+0000 7f47c3323740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b1e00a6240 con 0x55b1e0144b60 2024-09-19T09:08:30.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.321+0000 7f47ba59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e0145480 0x55b1e014f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:30.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.322+0000 7f47bad9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e0144b60 0x55b1e0144f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:30.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.322+0000 7f47b3fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e014fd40 0x55b1e0152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:30.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.322+0000 7f47ba59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e0145480 0x55b1e014f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47184/0 (socket says 172.21.15.120:47184) 2024-09-19T09:08:30.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.322+0000 7f47ba59c640 1 -- 172.21.15.120:0/2598402267 learned_addr learned my addr 172.21.15.120:0/2598402267 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:30.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.322+0000 7f47b37fe640 1 --2- 172.21.15.120:0/326466384 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f47a80a32c0 0x7f47a80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:30.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.322+0000 7f47bad9d640 1 -- 172.21.15.120:0/2598402267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e014fd40 msgr2=0x55b1e0152130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:30.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.322+0000 7f47bad9d640 1 --2- 172.21.15.120:0/2598402267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e014fd40 0x55b1e0152130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.322+0000 7f47bad9d640 1 -- 172.21.15.120:0/2598402267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e0145480 msgr2=0x55b1e014f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:30.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.322+0000 7f47bad9d640 1 --2- 172.21.15.120:0/2598402267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e0145480 0x55b1e014f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.322+0000 7f47bad9d640 1 -- 172.21.15.120:0/2598402267 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1e005e1f0 con 0x55b1e0144b60 2024-09-19T09:08:30.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.322+0000 7f47b3fff640 1 --2- 172.21.15.120:0/2598402267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e014fd40 0x55b1e0152130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:30.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.322+0000 7f47ba59c640 1 --2- 172.21.15.120:0/2598402267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e0145480 0x55b1e014f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:30.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.322+0000 7f47bad9d640 1 --2- 172.21.15.120:0/326466384 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f47a80a32c0 0x7f47a80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:30.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.322+0000 7f47b37fe640 1 -- 172.21.15.120:0/326466384 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f47ac0cfc30 con 0x7f47bc101de0 2024-09-19T09:08:30.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.322+0000 7f47bad9d640 1 --2- 172.21.15.120:0/2598402267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e0144b60 0x55b1e0144f40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f47b4006010 tx=0x7f47b4009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a73a5112ae5a9f92 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:30.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.322+0000 7f47b0ff9640 1 -- 172.21.15.120:0/2598402267 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b4042040 con 0x55b1e0144b60 2024-09-19T09:08:30.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.322+0000 7f47bad9d640 1 --2- 172.21.15.120:0/326466384 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f47a80a32c0 0x7f47a80a5780 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f47b4005e30 tx=0x7f47b404e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:30.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.322+0000 7f47b0ff9640 1 -- 172.21.15.120:0/2598402267 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47b4036040 con 0x55b1e0144b60 2024-09-19T09:08:30.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.322+0000 7f47b0ff9640 1 -- 172.21.15.120:0/2598402267 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b402f8f0 con 0x55b1e0144b60 2024-09-19T09:08:30.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.323+0000 7f47c3323740 1 -- 172.21.15.120:0/2598402267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e0144b60 msgr2=0x55b1e0144f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:30.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.323+0000 7f47c3323740 1 --2- 172.21.15.120:0/2598402267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e0144b60 0x55b1e0144f40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f47b4006010 tx=0x7f47b4009800 comp rx=0 tx=0).stop 2024-09-19T09:08:30.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.323+0000 7f47c3323740 1 -- 172.21.15.120:0/2598402267 shutdown_connections 2024-09-19T09:08:30.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.323+0000 7f47c3323740 1 --2- 172.21.15.120:0/2598402267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e014fd40 0x55b1e0152130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.323+0000 7f47c3323740 1 --2- 172.21.15.120:0/2598402267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e0145480 0x55b1e014f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.323+0000 7f47c3323740 1 --2- 172.21.15.120:0/2598402267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e0144b60 0x55b1e0144f40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.323+0000 7f47c3323740 1 -- 172.21.15.120:0/2598402267 >> 172.21.15.120:0/2598402267 conn(0x55b1e01410f0 msgr2=0x55b1e01414d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:30.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.323+0000 7f47c3323740 1 -- 172.21.15.120:0/2598402267 shutdown_connections 2024-09-19T09:08:30.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.323+0000 7f47c3323740 1 -- 172.21.15.120:0/2598402267 wait complete. 2024-09-19T09:08:30.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.324+0000 7f47c3323740 1 -- start start 2024-09-19T09:08:30.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.324+0000 7f47c3323740 1 Processor -- start 2024-09-19T09:08:30.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.324+0000 7f47c3323740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e0144b60 0x55b1e014cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:30.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.324+0000 7f47c3323740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e0145480 0x55b1e014d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:30.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.324+0000 7f47c3323740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e014fd40 0x55b1e014d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:30.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.324+0000 7f47c3323740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b1e0152910 con 0x55b1e0145480 2024-09-19T09:08:30.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.324+0000 7f47c3323740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b1e0064f10 con 0x55b1e0144b60 2024-09-19T09:08:30.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.324+0000 7f47c3323740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b1e0152a90 con 0x55b1e014fd40 2024-09-19T09:08:30.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.324+0000 7f47c3323740 10 client.0 _ll_register_callbacks cb 0x55b1e002da40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:08:30.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.324+0000 7f47b3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e0145480 0x55b1e014d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:30.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.324+0000 7f47ba59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e0144b60 0x55b1e014cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:30.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.324+0000 7f47b3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e0145480 0x55b1e014d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47188/0 (socket says 172.21.15.120:47188) 2024-09-19T09:08:30.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.324+0000 7f47b3fff640 1 -- 172.21.15.120:0/35146566 learned_addr learned my addr 172.21.15.120:0/35146566 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:30.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.324+0000 7f47ba59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e0144b60 0x55b1e014cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:39692/0 (socket says 172.21.15.120:39692) 2024-09-19T09:08:30.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.324+0000 7f47bad9d640 1 --2- 172.21.15.120:0/35146566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e014fd40 0x55b1e014d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:30.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.325+0000 7f47ba59c640 1 -- 172.21.15.120:0/35146566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e014fd40 msgr2=0x55b1e014d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:30.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.325+0000 7f47ba59c640 1 --2- 172.21.15.120:0/35146566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e014fd40 0x55b1e014d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.325+0000 7f47ba59c640 1 -- 172.21.15.120:0/35146566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e0145480 msgr2=0x55b1e014d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:30.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.325+0000 7f47ba59c640 1 --2- 172.21.15.120:0/35146566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e0145480 0x55b1e014d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.325+0000 7f47ba59c640 1 -- 172.21.15.120:0/35146566 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1dfe1b6a0 con 0x55b1e0144b60 2024-09-19T09:08:30.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.325+0000 7f47b3fff640 1 --2- 172.21.15.120:0/35146566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e0145480 0x55b1e014d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:30.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.325+0000 7f47ba59c640 1 --2- 172.21.15.120:0/35146566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e0144b60 0x55b1e014cca0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f47ac09bf50 tx=0x7f47ac0cd670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5238dda4dddcdba2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:30.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.325+0000 7f47a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:08:30.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.325+0000 7f47a1ffb640 1 -- 172.21.15.120:0/35146566 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac09c950 con 0x55b1e0144b60 2024-09-19T09:08:30.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.325+0000 7f47c3323740 1 -- 172.21.15.120:0/35146566 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47b4036040 con 0x55b1e0144b60 2024-09-19T09:08:30.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.325+0000 7f47a1ffb640 1 -- 172.21.15.120:0/35146566 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f47ac0bc0b0 con 0x55b1e0144b60 2024-09-19T09:08:30.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.325+0000 7f47a1ffb640 1 -- 172.21.15.120:0/35146566 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f47ac09c250 con 0x55b1e0144b60 2024-09-19T09:08:30.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.325+0000 7f47c3323740 10 client.26717 fetch_fsmap learned FSMap version 30 2024-09-19T09:08:30.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.325+0000 7f47c3323740 1 -- 172.21.15.120:0/35146566 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b1e015e5e0 con 0x55b1e0144b60 2024-09-19T09:08:30.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.325+0000 7f47a1ffb640 1 -- 172.21.15.120:0/35146566 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47b4036040 con 0x55b1e0144b60 2024-09-19T09:08:30.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.325+0000 7f47a1ffb640 1 -- 172.21.15.120:0/35146566 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f47ac0d6020 con 0x55b1e0144b60 2024-09-19T09:08:30.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.326+0000 7f47c3323740 10 client.26717 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:08:30.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.326+0000 7f47c3323740 10 client.26717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:08:30.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.326+0000 7f47c3323740 20 client.26717 populate_metadata read hostname 'smithi120' 2024-09-19T09:08:30.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.326+0000 7f47c3323740 1 --2- 172.21.15.120:0/35146566 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b1e015faa0 0x55b1e0161f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:30.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.326+0000 7f47c3323740 4 client.26717 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:08:30.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.326+0000 7f47c3323740 1 -- 172.21.15.120:0/35146566 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1e00b77f0 con 0x55b1e015faa0 2024-09-19T09:08:30.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.326+0000 7f47b3fff640 1 --2- 172.21.15.120:0/35146566 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b1e015faa0 0x55b1e0161f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:30.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.326+0000 7f47b3fff640 1 --2- 172.21.15.120:0/35146566 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b1e015faa0 0x55b1e0161f40 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7d5a5b997933197 server_cookie=572b887c796e8980 in_seq=0 out_seq=0 2024-09-19T09:08:30.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.326+0000 7f47a1ffb640 10 client.26717 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:30.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.328+0000 7f47a1ffb640 1 -- 172.21.15.120:0/35146566 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b1e00b77f0 con 0x55b1e015faa0 2024-09-19T09:08:30.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.328+0000 7f47a1ffb640 10 client.26717 handle_command_reply: tid=0 2024-09-19T09:08:30.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.328+0000 7f47c3323740 1 client.26717 shutdown 2024-09-19T09:08:30.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.328+0000 7f47c3323740 1 -- 172.21.15.120:0/35146566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e0144b60 msgr2=0x55b1e014cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:30.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.328+0000 7f47c3323740 1 --2- 172.21.15.120:0/35146566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e0144b60 0x55b1e014cca0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f47ac09bf50 tx=0x7f47ac0cd670 comp rx=0 tx=0).stop 2024-09-19T09:08:30.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.328+0000 7f47c3323740 1 -- 172.21.15.120:0/35146566 shutdown_connections 2024-09-19T09:08:30.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.328+0000 7f47c3323740 1 --2- 172.21.15.120:0/35146566 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b1e015faa0 0x55b1e0161f40 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.328+0000 7f47c3323740 1 --2- 172.21.15.120:0/35146566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e014fd40 0x55b1e014d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.328+0000 7f47c3323740 1 --2- 172.21.15.120:0/35146566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e0145480 0x55b1e014d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.328+0000 7f47a1ffb640 0 client.26717 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:30.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.328+0000 7f47c3323740 1 --2- 172.21.15.120:0/35146566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e0144b60 0x55b1e014cca0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.328+0000 7f47c3323740 1 -- 172.21.15.120:0/35146566 >> 172.21.15.120:0/35146566 conn(0x55b1e01410f0 msgr2=0x55b1e0146e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:30.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.328+0000 7f47c3323740 1 -- 172.21.15.120:0/35146566 shutdown_connections 2024-09-19T09:08:30.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.328+0000 7f47c3323740 1 -- 172.21.15.120:0/35146566 wait complete. 2024-09-19T09:08:30.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.328+0000 7f47c3323740 20 client.26717 trim_cache size 0 max 16384 2024-09-19T09:08:30.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.346+0000 7f47c3323740 1 Processor -- start 2024-09-19T09:08:30.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.346+0000 7f47c3323740 1 -- start start 2024-09-19T09:08:30.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.347+0000 7f47c3323740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e0144b60 0x55b1e0147610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:30.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.347+0000 7f47c3323740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e0145480 0x55b1e014eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:30.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.347+0000 7f47c3323740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e014fd40 0x55b1e0167810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:30.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.347+0000 7f47c3323740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc1ad500 con 0x55b1e0145480 2024-09-19T09:08:30.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.347+0000 7f47c3323740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc1af040 con 0x55b1e014fd40 2024-09-19T09:08:30.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.347+0000 7f47c3323740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc1adbd0 con 0x55b1e0144b60 2024-09-19T09:08:30.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.347+0000 7f47b3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e0145480 0x55b1e014eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:30.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.347+0000 7f47b3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e0145480 0x55b1e014eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47202/0 (socket says 172.21.15.120:47202) 2024-09-19T09:08:30.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.347+0000 7f47b3fff640 1 -- 172.21.15.120:0/1702626122 learned_addr learned my addr 172.21.15.120:0/1702626122 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:30.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.347+0000 7f47bad9d640 1 --2- 172.21.15.120:0/1702626122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e014fd40 0x55b1e0167810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:30.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.347+0000 7f47ba59c640 1 --2- 172.21.15.120:0/1702626122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e0144b60 0x55b1e0147610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:30.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.347+0000 7f47b3fff640 1 -- 172.21.15.120:0/1702626122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e0144b60 msgr2=0x55b1e0147610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:30.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.347+0000 7f47b3fff640 1 --2- 172.21.15.120:0/1702626122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e0144b60 0x55b1e0147610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.347+0000 7f47b3fff640 1 -- 172.21.15.120:0/1702626122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e014fd40 msgr2=0x55b1e0167810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:30.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.347+0000 7f47b3fff640 1 --2- 172.21.15.120:0/1702626122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e014fd40 0x55b1e0167810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.347+0000 7f47b3fff640 1 -- 172.21.15.120:0/1702626122 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1e01421a0 con 0x55b1e0145480 2024-09-19T09:08:30.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.347+0000 7f47b3fff640 1 --2- 172.21.15.120:0/1702626122 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e0145480 0x55b1e014eb00 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x55b1e014e2e0 tx=0x7f47a4014ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24625a26d8cd31f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:30.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.347+0000 7f47a3fff640 1 -- 172.21.15.120:0/1702626122 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a4024030 con 0x55b1e0145480 2024-09-19T09:08:30.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.347+0000 7f47a3fff640 1 -- 172.21.15.120:0/1702626122 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47a4020540 con 0x55b1e0145480 2024-09-19T09:08:30.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.347+0000 7f47a3fff640 1 -- 172.21.15.120:0/1702626122 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a40038e0 con 0x55b1e0145480 2024-09-19T09:08:30.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.348+0000 7f47c3323740 1 -- 172.21.15.120:0/1702626122 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e0145480 msgr2=0x55b1e014eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:30.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.348+0000 7f47c3323740 1 --2- 172.21.15.120:0/1702626122 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e0145480 0x55b1e014eb00 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x55b1e014e2e0 tx=0x7f47a4014ad0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.348+0000 7f47c3323740 1 -- 172.21.15.120:0/1702626122 shutdown_connections 2024-09-19T09:08:30.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.348+0000 7f47c3323740 1 --2- 172.21.15.120:0/1702626122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e014fd40 0x55b1e0167810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.348+0000 7f47c3323740 1 --2- 172.21.15.120:0/1702626122 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e0145480 0x55b1e014eb00 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.348+0000 7f47c3323740 1 --2- 172.21.15.120:0/1702626122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e0144b60 0x55b1e0147610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.348+0000 7f47c3323740 1 -- 172.21.15.120:0/1702626122 >> 172.21.15.120:0/1702626122 conn(0x55b1e01410f0 msgr2=0x55b1e014e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:30.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.348+0000 7f47c3323740 1 -- 172.21.15.120:0/1702626122 shutdown_connections 2024-09-19T09:08:30.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.348+0000 7f47c3323740 1 -- 172.21.15.120:0/1702626122 wait complete. 2024-09-19T09:08:30.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.349+0000 7f47c3323740 1 -- start start 2024-09-19T09:08:30.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.349+0000 7f47c3323740 1 Processor -- start 2024-09-19T09:08:30.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.349+0000 7f47c3323740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e0144b60 0x55b1e0149cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:30.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.349+0000 7f47c3323740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e0145480 0x55b1e014a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:30.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.349+0000 7f47c3323740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e014fd40 0x55b1e014a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:30.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.349+0000 7f47c3323740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc1ae970 con 0x55b1e014fd40 2024-09-19T09:08:30.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.349+0000 7f47c3323740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc1ae2a0 con 0x55b1e0145480 2024-09-19T09:08:30.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.349+0000 7f47c3323740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc1ab4c0 con 0x55b1e0144b60 2024-09-19T09:08:30.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.349+0000 7f47c3323740 10 client.0 _ll_register_callbacks cb 0x7f47ac00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:08:30.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.349+0000 7f47bad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e014fd40 0x55b1e014a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:30.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.349+0000 7f47b3fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e0145480 0x55b1e014a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:30.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.349+0000 7f47bad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e014fd40 0x55b1e014a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47204/0 (socket says 172.21.15.120:47204) 2024-09-19T09:08:30.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.349+0000 7f47b3fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e0145480 0x55b1e014a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:39720/0 (socket says 172.21.15.120:39720) 2024-09-19T09:08:30.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.349+0000 7f47ba59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e0144b60 0x55b1e0149cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:30.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.349+0000 7f47bad9d640 1 -- 172.21.15.120:0/386185019 learned_addr learned my addr 172.21.15.120:0/386185019 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:30.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.349+0000 7f47bad9d640 1 -- 172.21.15.120:0/386185019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e0144b60 msgr2=0x55b1e0149cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:30.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.349+0000 7f47bad9d640 1 --2- 172.21.15.120:0/386185019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e0144b60 0x55b1e0149cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.349+0000 7f47bad9d640 1 -- 172.21.15.120:0/386185019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e0145480 msgr2=0x55b1e014a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:30.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.349+0000 7f47bad9d640 1 --2- 172.21.15.120:0/386185019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e0145480 0x55b1e014a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.349+0000 7f47bad9d640 1 -- 172.21.15.120:0/386185019 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1e01cdc40 con 0x55b1e014fd40 2024-09-19T09:08:30.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.350+0000 7f47bad9d640 1 --2- 172.21.15.120:0/386185019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e014fd40 0x55b1e014a770 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f47b4049000 tx=0x7f47b4009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b174c900110024 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:30.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.350+0000 7f47a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:08:30.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.350+0000 7f47a1ffb640 1 -- 172.21.15.120:0/386185019 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b4042040 con 0x55b1e014fd40 2024-09-19T09:08:30.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.350+0000 7f47a1ffb640 1 -- 172.21.15.120:0/386185019 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f47b402fd70 con 0x55b1e014fd40 2024-09-19T09:08:30.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.350+0000 7f47c3323740 1 -- 172.21.15.120:0/386185019 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47a4020540 con 0x55b1e014fd40 2024-09-19T09:08:30.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.350+0000 7f47a1ffb640 1 -- 172.21.15.120:0/386185019 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f47b4003ec0 con 0x55b1e014fd40 2024-09-19T09:08:30.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.350+0000 7f47c3323740 10 client.17454 fetch_fsmap learned FSMap version 30 2024-09-19T09:08:30.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.350+0000 7f47c3323740 1 -- 172.21.15.120:0/386185019 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b1e01cef60 con 0x55b1e014fd40 2024-09-19T09:08:30.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.350+0000 7f47a1ffb640 1 -- 172.21.15.120:0/386185019 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47a4020540 con 0x55b1e014fd40 2024-09-19T09:08:30.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.350+0000 7f47a1ffb640 1 -- 172.21.15.120:0/386185019 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f47b405a070 con 0x55b1e014fd40 2024-09-19T09:08:30.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.350+0000 7f47c3323740 10 client.17454 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:08:30.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.350+0000 7f47c3323740 10 client.17454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:08:30.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.351+0000 7f47c3323740 20 client.17454 populate_metadata read hostname 'smithi120' 2024-09-19T09:08:30.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.351+0000 7f47c3323740 1 --2- 172.21.15.120:0/386185019 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b1e01cf230 0x55b1e01d1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:30.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.351+0000 7f47c3323740 4 client.17454 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:08:30.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.351+0000 7f47c3323740 1 -- 172.21.15.120:0/386185019 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b1e00b1d00 con 0x55b1e01cf230 2024-09-19T09:08:30.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.351+0000 7f47ba59c640 1 --2- 172.21.15.120:0/386185019 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b1e01cf230 0x55b1e01d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:30.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.351+0000 7f47a1ffb640 10 client.17454 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:30.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.351+0000 7f47ba59c640 1 --2- 172.21.15.120:0/386185019 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b1e01cf230 0x55b1e01d1620 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecf316eae302a61e server_cookie=cb5539dce34231f1 in_seq=0 out_seq=0 2024-09-19T09:08:30.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.366+0000 7f47a1ffb640 1 -- 172.21.15.120:0/386185019 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b1e00b1d00 con 0x55b1e01cf230 2024-09-19T09:08:30.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.366+0000 7f47a1ffb640 10 client.17454 handle_command_reply: tid=0 2024-09-19T09:08:30.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.366+0000 7f47c3323740 1 client.17454 shutdown 2024-09-19T09:08:30.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.366+0000 7f47c3323740 1 -- 172.21.15.120:0/386185019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e014fd40 msgr2=0x55b1e014a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:30.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.366+0000 7f47c3323740 1 --2- 172.21.15.120:0/386185019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e014fd40 0x55b1e014a770 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f47b4049000 tx=0x7f47b4009360 comp rx=0 tx=0).stop 2024-09-19T09:08:30.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.366+0000 7f47c3323740 1 -- 172.21.15.120:0/386185019 shutdown_connections 2024-09-19T09:08:30.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.366+0000 7f47c3323740 1 --2- 172.21.15.120:0/386185019 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b1e01cf230 0x55b1e01d1620 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.366+0000 7f47c3323740 1 --2- 172.21.15.120:0/386185019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1e014fd40 0x55b1e014a770 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.366+0000 7f47c3323740 1 --2- 172.21.15.120:0/386185019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1e0145480 0x55b1e014a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.366+0000 7f47c3323740 1 --2- 172.21.15.120:0/386185019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1e0144b60 0x55b1e0149cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.366+0000 7f47a1ffb640 0 client.17454 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:30.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.366+0000 7f47c3323740 1 -- 172.21.15.120:0/386185019 >> 172.21.15.120:0/386185019 conn(0x55b1e01410f0 msgr2=0x55b1e0140270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:30.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.367+0000 7f47c3323740 1 -- 172.21.15.120:0/386185019 shutdown_connections 2024-09-19T09:08:30.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.367+0000 7f47c3323740 1 -- 172.21.15.120:0/386185019 wait complete. 2024-09-19T09:08:30.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.367+0000 7f47c3323740 20 client.17454 trim_cache size 0 max 16384 2024-09-19T09:08:30.368 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:08:30.368 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:08:30.369 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "d5bb44fa-e214-4948-abc0-987db37435ed", 2024-09-19T09:08:30.369 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:08:30.369 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:08:30.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.367+0000 7f47c0f4a640 1 -- 172.21.15.120:0/326466384 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f47a80a32c0 msgr2=0x7f47a80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:30.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.367+0000 7f47c0f4a640 1 --2- 172.21.15.120:0/326466384 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f47a80a32c0 0x7f47a80a5780 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f47b4005e30 tx=0x7f47b404e000 comp rx=0 tx=0).stop 2024-09-19T09:08:30.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.367+0000 7f47c0f4a640 1 -- 172.21.15.120:0/326466384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47bc101de0 msgr2=0x7f47bc1983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:30.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.367+0000 7f47c0f4a640 1 --2- 172.21.15.120:0/326466384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47bc101de0 0x7f47bc1983b0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00e9f0 tx=0x7f47ac00eeb0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.367+0000 7f47c0f4a640 1 -- 172.21.15.120:0/326466384 shutdown_connections 2024-09-19T09:08:30.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.367+0000 7f47c0f4a640 1 --2- 172.21.15.120:0/326466384 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f47a80a32c0 0x7f47a80a5780 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.367+0000 7f47c0f4a640 1 --2- 172.21.15.120:0/326466384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47bc106f10 0x7f47bc19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.367+0000 7f47c0f4a640 1 --2- 172.21.15.120:0/326466384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47bc102790 0x7f47bc1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.367+0000 7f47c0f4a640 1 --2- 172.21.15.120:0/326466384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47bc101de0 0x7f47bc1983b0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:30.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.367+0000 7f47c0f4a640 1 -- 172.21.15.120:0/326466384 >> 172.21.15.120:0/326466384 conn(0x7f47bc0fd8c0 msgr2=0x7f47bc0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:30.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.367+0000 7f47c0f4a640 1 -- 172.21.15.120:0/326466384 shutdown_connections 2024-09-19T09:08:30.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:30.367+0000 7f47c0f4a640 1 -- 172.21.15.120:0/326466384 wait complete. 2024-09-19T09:08:31.051 INFO:tasks.workunit.client.0.smithi120.stdout: 1 428977 7.43 MB/sec execute 146 sec latency 10.324 ms 2024-09-19T09:08:31.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:30 smithi120 ceph-mon[31150]: pgmap v511: 97 pgs: 97 active+clean; 36 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.6 MiB/s wr, 289 op/s 2024-09-19T09:08:31.153 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:08:31.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:30 smithi196 ceph-mon[35150]: pgmap v511: 97 pgs: 97 active+clean; 36 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.6 MiB/s wr, 289 op/s 2024-09-19T09:08:31.269 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:30 smithi153 ceph-mon[35071]: pgmap v511: 97 pgs: 97 active+clean; 36 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.6 MiB/s wr, 289 op/s 2024-09-19T09:08:31.407 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:08:31.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.908+0000 7fa8b22a8640 1 -- 172.21.15.120:0/1994373372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8ac102780 msgr2=0x7fa8ac1067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:31.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.908+0000 7fa8b22a8640 1 --2- 172.21.15.120:0/1994373372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8ac102780 0x7fa8ac1067d0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0009900 tx=0x7fa8a002a410 comp rx=0 tx=0).stop 2024-09-19T09:08:31.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.908+0000 7fa8b22a8640 1 -- 172.21.15.120:0/1994373372 shutdown_connections 2024-09-19T09:08:31.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.908+0000 7fa8b22a8640 1 --2- 172.21.15.120:0/1994373372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8ac106f00 0x7fa8ac10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.908+0000 7fa8b22a8640 1 --2- 172.21.15.120:0/1994373372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8ac102780 0x7fa8ac1067d0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.908+0000 7fa8b22a8640 1 --2- 172.21.15.120:0/1994373372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8ac101dd0 0x7fa8ac1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.908+0000 7fa8b22a8640 1 -- 172.21.15.120:0/1994373372 >> 172.21.15.120:0/1994373372 conn(0x7fa8ac0fd8b0 msgr2=0x7fa8ac0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:31.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.908+0000 7fa8b22a8640 1 -- 172.21.15.120:0/1994373372 shutdown_connections 2024-09-19T09:08:31.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.908+0000 7fa8b22a8640 1 -- 172.21.15.120:0/1994373372 wait complete. 2024-09-19T09:08:31.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.908+0000 7fa8b22a8640 1 Processor -- start 2024-09-19T09:08:31.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.908+0000 7fa8b22a8640 1 -- start start 2024-09-19T09:08:31.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.909+0000 7fa8b22a8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8ac101dd0 0x7fa8ac1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:31.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.909+0000 7fa8b22a8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8ac102780 0x7fa8ac198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:31.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.909+0000 7fa8b22a8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8ac106f00 0x7fa8ac19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:31.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.909+0000 7fa8b22a8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac111f60 con 0x7fa8ac102780 2024-09-19T09:08:31.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.909+0000 7fa8b22a8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac111de0 con 0x7fa8ac101dd0 2024-09-19T09:08:31.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.909+0000 7fa8b22a8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac1120e0 con 0x7fa8ac106f00 2024-09-19T09:08:31.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.909+0000 7fa8ab7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8ac102780 0x7fa8ac198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:31.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.909+0000 7fa8ab7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8ac102780 0x7fa8ac198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46848/0 (socket says 172.21.15.120:46848) 2024-09-19T09:08:31.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.909+0000 7fa8ab7fe640 1 -- 172.21.15.120:0/2180976484 learned_addr learned my addr 172.21.15.120:0/2180976484 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:31.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.909+0000 7fa8b0845640 1 --2- 172.21.15.120:0/2180976484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8ac106f00 0x7fa8ac19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:31.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.909+0000 7fa8abfff640 1 --2- 172.21.15.120:0/2180976484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8ac101dd0 0x7fa8ac1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:31.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.909+0000 7fa8ab7fe640 1 -- 172.21.15.120:0/2180976484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8ac106f00 msgr2=0x7fa8ac19cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:31.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.909+0000 7fa8ab7fe640 1 --2- 172.21.15.120:0/2180976484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8ac106f00 0x7fa8ac19cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.909+0000 7fa8ab7fe640 1 -- 172.21.15.120:0/2180976484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8ac101dd0 msgr2=0x7fa8ac1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:31.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.909+0000 7fa8ab7fe640 1 --2- 172.21.15.120:0/2180976484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8ac101dd0 0x7fa8ac1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.909+0000 7fa8ab7fe640 1 -- 172.21.15.120:0/2180976484 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8ac19d470 con 0x7fa8ac102780 2024-09-19T09:08:31.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.910+0000 7fa8abfff640 1 --2- 172.21.15.120:0/2180976484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8ac101dd0 0x7fa8ac1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:31.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.910+0000 7fa8ab7fe640 1 --2- 172.21.15.120:0/2180976484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8ac102780 0x7fa8ac198a00 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0006010 tx=0x7fa8a0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:31.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.910+0000 7fa8a97fa640 1 -- 172.21.15.120:0/2180976484 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a003f030 con 0x7fa8ac102780 2024-09-19T09:08:31.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.910+0000 7fa8a97fa640 1 -- 172.21.15.120:0/2180976484 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8a0004440 con 0x7fa8ac102780 2024-09-19T09:08:31.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.910+0000 7fa8a97fa640 1 -- 172.21.15.120:0/2180976484 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0035070 con 0x7fa8ac102780 2024-09-19T09:08:31.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.910+0000 7fa8b22a8640 1 -- 172.21.15.120:0/2180976484 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8ac19d700 con 0x7fa8ac102780 2024-09-19T09:08:31.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.910+0000 7fa8b22a8640 1 -- 172.21.15.120:0/2180976484 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8ac19db90 con 0x7fa8ac102780 2024-09-19T09:08:31.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.912+0000 7fa8a97fa640 1 -- 172.21.15.120:0/2180976484 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa8a0042030 con 0x7fa8ac102780 2024-09-19T09:08:31.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.913+0000 7fa8b4681740 1 Processor -- start 2024-09-19T09:08:31.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.913+0000 7fa8b4681740 1 -- start start 2024-09-19T09:08:31.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.913+0000 7fa8b4681740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a07e7c94a0 0x55a07e7c9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:31.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.913+0000 7fa8b4681740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a07e7c9dc0 0x55a07e7d4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:31.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.913+0000 7fa8a97fa640 1 --2- 172.21.15.120:0/2180976484 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa8940a3240 0x7fa8940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:31.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.913+0000 7fa8b4681740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a07e7d4680 0x55a07e7d6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:31.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.913+0000 7fa8b4681740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a07e72f2e0 con 0x55a07e7d4680 2024-09-19T09:08:31.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.913+0000 7fa8b4681740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a07e6d6ce0 con 0x55a07e7c9dc0 2024-09-19T09:08:31.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.913+0000 7fa8b4681740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a07e72b2a0 con 0x55a07e7c94a0 2024-09-19T09:08:31.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.913+0000 7fa8b0845640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a07e7c94a0 0x55a07e7c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:31.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.913+0000 7fa8abfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a07e7c9dc0 0x55a07e7d4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:31.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.913+0000 7fa8abfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a07e7c9dc0 0x55a07e7d4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:58018/0 (socket says 172.21.15.120:58018) 2024-09-19T09:08:31.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.913+0000 7fa8b0845640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a07e7c94a0 0x55a07e7c9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36812/0 (socket says 172.21.15.120:36812) 2024-09-19T09:08:31.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.913+0000 7fa8abfff640 1 -- 172.21.15.120:0/2661064548 learned_addr learned my addr 172.21.15.120:0/2661064548 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:31.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.913+0000 7fa8b0845640 1 --2- 172.21.15.120:0/2661064548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a07e7d4680 0x55a07e7d6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:31.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.913+0000 7fa8a97fa640 1 -- 172.21.15.120:0/2180976484 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fa8a00efc50 con 0x7fa8ac102780 2024-09-19T09:08:31.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.913+0000 7fa8ab7fe640 1 --2- 172.21.15.120:0/2180976484 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa8940a3240 0x7fa8940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:31.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.914+0000 7fa8b0845640 1 -- 172.21.15.120:0/2661064548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a07e7c9dc0 msgr2=0x55a07e7d4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:31.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.914+0000 7fa8b0845640 1 --2- 172.21.15.120:0/2661064548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a07e7c9dc0 0x55a07e7d4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.914+0000 7fa8b0845640 1 -- 172.21.15.120:0/2661064548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a07e7d4680 msgr2=0x55a07e7d6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:31.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.914+0000 7fa8b0845640 1 --2- 172.21.15.120:0/2661064548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a07e7d4680 0x55a07e7d6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.914+0000 7fa8b0845640 1 -- 172.21.15.120:0/2661064548 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a07e6bb440 con 0x55a07e7c94a0 2024-09-19T09:08:31.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.914+0000 7fa8b0845640 1 --2- 172.21.15.120:0/2661064548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a07e7c94a0 0x55a07e7c9880 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4005fd0 tx=0x7fa8a4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24ef009f7a6875ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:31.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.914+0000 7fa8ab7fe640 1 --2- 172.21.15.120:0/2180976484 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa8940a3240 0x7fa8940a5700 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0005ab0 tx=0x7fa8a00045f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:31.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.914+0000 7fa8927fc640 1 -- 172.21.15.120:0/2661064548 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4011050 con 0x55a07e7c94a0 2024-09-19T09:08:31.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.914+0000 7fa8927fc640 1 -- 172.21.15.120:0/2661064548 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8a40111f0 con 0x55a07e7c94a0 2024-09-19T09:08:31.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.914+0000 7fa8927fc640 1 -- 172.21.15.120:0/2661064548 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a400fab0 con 0x55a07e7c94a0 2024-09-19T09:08:31.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.915+0000 7fa8b4681740 1 -- 172.21.15.120:0/2661064548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a07e7c94a0 msgr2=0x55a07e7c9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:31.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.915+0000 7fa8b4681740 1 --2- 172.21.15.120:0/2661064548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a07e7c94a0 0x55a07e7c9880 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4005fd0 tx=0x7fa8a4004a40 comp rx=0 tx=0).stop 2024-09-19T09:08:31.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.915+0000 7fa8b4681740 1 -- 172.21.15.120:0/2661064548 shutdown_connections 2024-09-19T09:08:31.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.915+0000 7fa8b4681740 1 --2- 172.21.15.120:0/2661064548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a07e7d4680 0x55a07e7d6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.915+0000 7fa8b4681740 1 --2- 172.21.15.120:0/2661064548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a07e7c9dc0 0x55a07e7d4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.915+0000 7fa8b4681740 1 --2- 172.21.15.120:0/2661064548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a07e7c94a0 0x55a07e7c9880 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.915+0000 7fa8b4681740 1 -- 172.21.15.120:0/2661064548 >> 172.21.15.120:0/2661064548 conn(0x55a07e7c5a30 msgr2=0x55a07e7c5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:31.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.915+0000 7fa8b4681740 1 -- 172.21.15.120:0/2661064548 shutdown_connections 2024-09-19T09:08:31.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.915+0000 7fa8b4681740 1 -- 172.21.15.120:0/2661064548 wait complete. 2024-09-19T09:08:31.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.915+0000 7fa8b4681740 1 -- start start 2024-09-19T09:08:31.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.915+0000 7fa8b4681740 1 Processor -- start 2024-09-19T09:08:31.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.916+0000 7fa8b4681740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a07e7c94a0 0x55a07e7d4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:31.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.916+0000 7fa8b4681740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a07e7c9dc0 0x55a07e7d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:31.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.916+0000 7fa8b4681740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a07e7d4680 0x55a07e7d18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:31.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.916+0000 7fa8b4681740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a07e7db660 con 0x55a07e7d4680 2024-09-19T09:08:31.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.916+0000 7fa8b4681740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a07e6ea050 con 0x55a07e7c9dc0 2024-09-19T09:08:31.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.916+0000 7fa8b4681740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a07e7db7e0 con 0x55a07e7c94a0 2024-09-19T09:08:31.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.916+0000 7fa8b4681740 10 client.0 _ll_register_callbacks cb 0x55a07e72a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:08:31.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.916+0000 7fa8b0845640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a07e7c94a0 0x55a07e7d4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:31.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.916+0000 7fa8b0845640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a07e7c94a0 0x55a07e7d4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36822/0 (socket says 172.21.15.120:36822) 2024-09-19T09:08:31.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.916+0000 7fa8b0845640 1 -- 172.21.15.120:0/1850404881 learned_addr learned my addr 172.21.15.120:0/1850404881 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:31.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.916+0000 7fa8abfff640 1 --2- 172.21.15.120:0/1850404881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a07e7c9dc0 0x55a07e7d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:31.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.916+0000 7fa8b0845640 1 --2- 172.21.15.120:0/1850404881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a07e7d4680 0x55a07e7d18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:31.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.916+0000 7fa8b0845640 1 -- 172.21.15.120:0/1850404881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a07e7c9dc0 msgr2=0x55a07e7d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:31.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.916+0000 7fa8b0845640 1 --2- 172.21.15.120:0/1850404881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a07e7c9dc0 0x55a07e7d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.916+0000 7fa8b0845640 1 -- 172.21.15.120:0/1850404881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a07e7d4680 msgr2=0x55a07e7d18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:31.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.916+0000 7fa8b0845640 1 --2- 172.21.15.120:0/1850404881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a07e7d4680 0x55a07e7d18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.916+0000 7fa8b0845640 1 -- 172.21.15.120:0/1850404881 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a07e4a03c0 con 0x55a07e7c94a0 2024-09-19T09:08:31.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.916+0000 7fa8b0845640 1 --2- 172.21.15.120:0/1850404881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a07e7c94a0 0x55a07e7d4060 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4005f40 tx=0x7fa8a4003ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebee84af40dd775 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:31.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.916+0000 7fa88b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:08:31.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.917+0000 7fa88b7fe640 1 -- 172.21.15.120:0/1850404881 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4028040 con 0x55a07e7c94a0 2024-09-19T09:08:31.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.917+0000 7fa8b4681740 1 -- 172.21.15.120:0/1850404881 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8a40111f0 con 0x55a07e7c94a0 2024-09-19T09:08:31.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.917+0000 7fa88b7fe640 1 -- 172.21.15.120:0/1850404881 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fa8a400fa30 con 0x55a07e7c94a0 2024-09-19T09:08:31.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.917+0000 7fa88b7fe640 1 -- 172.21.15.120:0/1850404881 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa8a4035050 con 0x55a07e7c94a0 2024-09-19T09:08:31.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.917+0000 7fa8b4681740 10 client.26638 fetch_fsmap learned FSMap version 30 2024-09-19T09:08:31.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.917+0000 7fa8b4681740 1 -- 172.21.15.120:0/1850404881 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a07e7e70a0 con 0x55a07e7c94a0 2024-09-19T09:08:31.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.917+0000 7fa88b7fe640 1 -- 172.21.15.120:0/1850404881 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8a40111f0 con 0x55a07e7c94a0 2024-09-19T09:08:31.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.917+0000 7fa88b7fe640 1 -- 172.21.15.120:0/1850404881 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa8a401f440 con 0x55a07e7c94a0 2024-09-19T09:08:31.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.917+0000 7fa8b4681740 10 client.26638 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:08:31.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.917+0000 7fa8b4681740 10 client.26638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:08:31.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.917+0000 7fa8b4681740 20 client.26638 populate_metadata read hostname 'smithi120' 2024-09-19T09:08:31.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.917+0000 7fa8b4681740 1 --2- 172.21.15.120:0/1850404881 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a07e7e83f0 0x55a07e7ea7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:31.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.917+0000 7fa8b4681740 4 client.26638 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:08:31.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.917+0000 7fa8b4681740 1 -- 172.21.15.120:0/1850404881 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a07e73c850 con 0x55a07e7e83f0 2024-09-19T09:08:31.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.917+0000 7fa8abfff640 1 --2- 172.21.15.120:0/1850404881 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a07e7e83f0 0x55a07e7ea7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:31.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.918+0000 7fa88b7fe640 10 client.26638 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:31.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.918+0000 7fa8abfff640 1 --2- 172.21.15.120:0/1850404881 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a07e7e83f0 0x55a07e7ea7e0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9aec8ca05459325a server_cookie=58539a65d02e34af in_seq=0 out_seq=0 2024-09-19T09:08:31.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.919+0000 7fa88b7fe640 1 -- 172.21.15.120:0/1850404881 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a07e73c850 con 0x55a07e7e83f0 2024-09-19T09:08:31.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.919+0000 7fa88b7fe640 10 client.26638 handle_command_reply: tid=0 2024-09-19T09:08:31.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.920+0000 7fa8b4681740 1 client.26638 shutdown 2024-09-19T09:08:31.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.920+0000 7fa8b4681740 1 -- 172.21.15.120:0/1850404881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a07e7c94a0 msgr2=0x55a07e7d4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:31.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.920+0000 7fa8b4681740 1 --2- 172.21.15.120:0/1850404881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a07e7c94a0 0x55a07e7d4060 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4005f40 tx=0x7fa8a4003ee0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.920+0000 7fa8b4681740 1 -- 172.21.15.120:0/1850404881 shutdown_connections 2024-09-19T09:08:31.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.920+0000 7fa8b4681740 1 --2- 172.21.15.120:0/1850404881 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a07e7e83f0 0x55a07e7ea7e0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.920+0000 7fa8b4681740 1 --2- 172.21.15.120:0/1850404881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a07e7d4680 0x55a07e7d18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.920+0000 7fa8b4681740 1 --2- 172.21.15.120:0/1850404881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a07e7c9dc0 0x55a07e7d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.920+0000 7fa8b4681740 1 --2- 172.21.15.120:0/1850404881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a07e7c94a0 0x55a07e7d4060 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.920+0000 7fa8b4681740 1 -- 172.21.15.120:0/1850404881 >> 172.21.15.120:0/1850404881 conn(0x55a07e7c5a30 msgr2=0x55a07e7cb710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:31.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.920+0000 7fa88b7fe640 0 client.26638 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:31.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.920+0000 7fa8b4681740 1 -- 172.21.15.120:0/1850404881 shutdown_connections 2024-09-19T09:08:31.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.920+0000 7fa8b4681740 1 -- 172.21.15.120:0/1850404881 wait complete. 2024-09-19T09:08:31.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.920+0000 7fa8b4681740 20 client.26638 trim_cache size 0 max 16384 2024-09-19T09:08:31.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.937+0000 7fa8b4681740 1 Processor -- start 2024-09-19T09:08:31.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.937+0000 7fa8b4681740 1 -- start start 2024-09-19T09:08:31.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.938+0000 7fa8b4681740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a07e7c94a0 0x55a07e7e48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:31.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.938+0000 7fa8b4681740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a07e7c9dc0 0x55a07e7eff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:31.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.938+0000 7fa8b4681740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a07e7d4680 0x55a07e7f0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:31.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.938+0000 7fa8b4681740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac1ab3f0 con 0x55a07e7c9dc0 2024-09-19T09:08:31.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.938+0000 7fa8b4681740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac1acf30 con 0x55a07e7d4680 2024-09-19T09:08:31.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.938+0000 7fa8b4681740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac1abac0 con 0x55a07e7c94a0 2024-09-19T09:08:31.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.938+0000 7fa8abfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a07e7c9dc0 0x55a07e7eff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:31.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.938+0000 7fa8abfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a07e7c9dc0 0x55a07e7eff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46886/0 (socket says 172.21.15.120:46886) 2024-09-19T09:08:31.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.938+0000 7fa8abfff640 1 -- 172.21.15.120:0/3839822038 learned_addr learned my addr 172.21.15.120:0/3839822038 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:31.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.938+0000 7fa8b0845640 1 --2- 172.21.15.120:0/3839822038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a07e7c94a0 0x55a07e7e48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:31.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.938+0000 7fa8b0845640 1 --2- 172.21.15.120:0/3839822038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a07e7d4680 0x55a07e7f0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:31.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.938+0000 7fa8b0845640 1 -- 172.21.15.120:0/3839822038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a07e7d4680 msgr2=0x55a07e7f0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:31.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.938+0000 7fa8b0845640 1 --2- 172.21.15.120:0/3839822038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a07e7d4680 0x55a07e7f0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.938+0000 7fa8b0845640 1 -- 172.21.15.120:0/3839822038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a07e7c9dc0 msgr2=0x55a07e7eff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:31.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.938+0000 7fa8b0845640 1 --2- 172.21.15.120:0/3839822038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a07e7c9dc0 0x55a07e7eff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.938+0000 7fa8b0845640 1 -- 172.21.15.120:0/3839822038 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a07e7d1c50 con 0x55a07e7c94a0 2024-09-19T09:08:31.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.938+0000 7fa8abfff640 1 --2- 172.21.15.120:0/3839822038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a07e7c9dc0 0x55a07e7eff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:31.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.938+0000 7fa8b0845640 1 --2- 172.21.15.120:0/3839822038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a07e7c94a0 0x55a07e7e48c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4009a70 tx=0x7fa8a400ffd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4b703386391aba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:31.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.939+0000 7fa891ffb640 1 -- 172.21.15.120:0/3839822038 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4009d20 con 0x55a07e7c94a0 2024-09-19T09:08:31.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.939+0000 7fa891ffb640 1 -- 172.21.15.120:0/3839822038 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8a4003bf0 con 0x55a07e7c94a0 2024-09-19T09:08:31.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.939+0000 7fa891ffb640 1 -- 172.21.15.120:0/3839822038 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a400f790 con 0x55a07e7c94a0 2024-09-19T09:08:31.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.939+0000 7fa8b4681740 1 -- 172.21.15.120:0/3839822038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a07e7c94a0 msgr2=0x55a07e7e48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:31.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.939+0000 7fa8b4681740 1 --2- 172.21.15.120:0/3839822038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a07e7c94a0 0x55a07e7e48c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4009a70 tx=0x7fa8a400ffd0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.940+0000 7fa8b4681740 1 -- 172.21.15.120:0/3839822038 shutdown_connections 2024-09-19T09:08:31.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.940+0000 7fa8b4681740 1 --2- 172.21.15.120:0/3839822038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a07e7d4680 0x55a07e7f0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.940+0000 7fa8b4681740 1 --2- 172.21.15.120:0/3839822038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a07e7c9dc0 0x55a07e7eff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.940+0000 7fa8b4681740 1 --2- 172.21.15.120:0/3839822038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a07e7c94a0 0x55a07e7e48c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.940+0000 7fa8b4681740 1 -- 172.21.15.120:0/3839822038 >> 172.21.15.120:0/3839822038 conn(0x55a07e7c5a30 msgr2=0x55a07e7cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:31.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.940+0000 7fa8b4681740 1 -- 172.21.15.120:0/3839822038 shutdown_connections 2024-09-19T09:08:31.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.940+0000 7fa8b4681740 1 -- 172.21.15.120:0/3839822038 wait complete. 2024-09-19T09:08:31.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.940+0000 7fa8b4681740 1 -- start start 2024-09-19T09:08:31.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.940+0000 7fa8b4681740 1 Processor -- start 2024-09-19T09:08:31.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.941+0000 7fa8b4681740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a07e7c94a0 0x55a07e7dad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:31.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.941+0000 7fa8b4681740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a07e7c9dc0 0x55a07e7e0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:31.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.941+0000 7fa8b4681740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a07e7d4680 0x55a07e7e0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:31.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.941+0000 7fa8b4681740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac1ac860 con 0x55a07e7c9dc0 2024-09-19T09:08:31.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.941+0000 7fa8b4681740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac1ac190 con 0x55a07e7c94a0 2024-09-19T09:08:31.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.941+0000 7fa8b4681740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac1a93b0 con 0x55a07e7d4680 2024-09-19T09:08:31.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.941+0000 7fa8b4681740 10 client.0 _ll_register_callbacks cb 0x7fa8a40114e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:08:31.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.941+0000 7fa8abfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a07e7c9dc0 0x55a07e7e0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:31.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.941+0000 7fa8abfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a07e7c9dc0 0x55a07e7e0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46892/0 (socket says 172.21.15.120:46892) 2024-09-19T09:08:31.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.941+0000 7fa8abfff640 1 -- 172.21.15.120:0/1467522636 learned_addr learned my addr 172.21.15.120:0/1467522636 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:31.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.941+0000 7fa8b0845640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a07e7c94a0 0x55a07e7dad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:31.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.941+0000 7fa8b0845640 1 --2- 172.21.15.120:0/1467522636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a07e7d4680 0x55a07e7e0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:31.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.941+0000 7fa8b0845640 1 -- 172.21.15.120:0/1467522636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a07e7d4680 msgr2=0x55a07e7e0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:31.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.941+0000 7fa8b0845640 1 --2- 172.21.15.120:0/1467522636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a07e7d4680 0x55a07e7e0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.941+0000 7fa8b0845640 1 -- 172.21.15.120:0/1467522636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a07e7c9dc0 msgr2=0x55a07e7e0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:31.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.941+0000 7fa8b0845640 1 --2- 172.21.15.120:0/1467522636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a07e7c9dc0 0x55a07e7e0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.941+0000 7fa8b0845640 1 -- 172.21.15.120:0/1467522636 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a07e7d80f0 con 0x55a07e7c94a0 2024-09-19T09:08:31.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.941+0000 7fa8abfff640 1 --2- 172.21.15.120:0/1467522636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a07e7c9dc0 0x55a07e7e0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:31.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.941+0000 7fa8b0845640 1 --2- 172.21.15.120:0/1467522636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a07e7c94a0 0x55a07e7dad00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa8a401f980 tx=0x7fa8a4003da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdbf13011ff812c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:31.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.941+0000 7fa88b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:08:31.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.941+0000 7fa88b7fe640 1 -- 172.21.15.120:0/1467522636 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4009d20 con 0x55a07e7c94a0 2024-09-19T09:08:31.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.941+0000 7fa8b4681740 1 -- 172.21.15.120:0/1467522636 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8a4003bf0 con 0x55a07e7c94a0 2024-09-19T09:08:31.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.941+0000 7fa88b7fe640 1 -- 172.21.15.120:0/1467522636 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fa8a401f5b0 con 0x55a07e7c94a0 2024-09-19T09:08:31.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.942+0000 7fa88b7fe640 1 -- 172.21.15.120:0/1467522636 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa8a401fbb0 con 0x55a07e7c94a0 2024-09-19T09:08:31.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.942+0000 7fa8b4681740 10 client.26741 fetch_fsmap learned FSMap version 30 2024-09-19T09:08:31.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.942+0000 7fa8b4681740 1 -- 172.21.15.120:0/1467522636 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a07e8577e0 con 0x55a07e7c94a0 2024-09-19T09:08:31.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.942+0000 7fa88b7fe640 1 -- 172.21.15.120:0/1467522636 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8a4003bf0 con 0x55a07e7c94a0 2024-09-19T09:08:31.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.942+0000 7fa88b7fe640 1 -- 172.21.15.120:0/1467522636 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa8a400f800 con 0x55a07e7c94a0 2024-09-19T09:08:31.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.942+0000 7fa8b4681740 10 client.26741 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:08:31.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.942+0000 7fa8b4681740 10 client.26741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:08:31.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.942+0000 7fa8b4681740 20 client.26741 populate_metadata read hostname 'smithi120' 2024-09-19T09:08:31.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.942+0000 7fa8b4681740 1 --2- 172.21.15.120:0/1467522636 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a07e857e00 0x55a07e85a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:31.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.942+0000 7fa8b4681740 4 client.26741 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:08:31.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.942+0000 7fa8b4681740 1 -- 172.21.15.120:0/1467522636 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a07e736d60 con 0x55a07e857e00 2024-09-19T09:08:31.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.942+0000 7fa8abfff640 1 --2- 172.21.15.120:0/1467522636 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a07e857e00 0x55a07e85a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:31.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.943+0000 7fa88b7fe640 10 client.26741 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:31.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.943+0000 7fa8abfff640 1 --2- 172.21.15.120:0/1467522636 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a07e857e00 0x55a07e85a1f0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aa42fae77ddd9fc server_cookie=7acb79761dfa85f5 in_seq=0 out_seq=0 2024-09-19T09:08:31.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.943+0000 7fa88b7fe640 1 -- 172.21.15.120:0/1467522636 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a07e736d60 con 0x55a07e857e00 2024-09-19T09:08:31.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.943+0000 7fa88b7fe640 10 client.26741 handle_command_reply: tid=0 2024-09-19T09:08:31.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.943+0000 7fa8b4681740 1 client.26741 shutdown 2024-09-19T09:08:31.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.943+0000 7fa8b4681740 1 -- 172.21.15.120:0/1467522636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a07e7c94a0 msgr2=0x55a07e7dad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:31.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.943+0000 7fa8b4681740 1 --2- 172.21.15.120:0/1467522636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a07e7c94a0 0x55a07e7dad00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa8a401f980 tx=0x7fa8a4003da0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.943+0000 7fa8b4681740 1 -- 172.21.15.120:0/1467522636 shutdown_connections 2024-09-19T09:08:31.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.943+0000 7fa8b4681740 1 --2- 172.21.15.120:0/1467522636 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a07e857e00 0x55a07e85a1f0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.943+0000 7fa8b4681740 1 --2- 172.21.15.120:0/1467522636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a07e7d4680 0x55a07e7e0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.943+0000 7fa8b4681740 1 --2- 172.21.15.120:0/1467522636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a07e7c9dc0 0x55a07e7e0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.943+0000 7fa8b4681740 1 --2- 172.21.15.120:0/1467522636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a07e7c94a0 0x55a07e7dad00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.943+0000 7fa88b7fe640 0 client.26741 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:31.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.943+0000 7fa8b4681740 1 -- 172.21.15.120:0/1467522636 >> 172.21.15.120:0/1467522636 conn(0x55a07e7c5a30 msgr2=0x55a07e7d2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:31.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.943+0000 7fa8b4681740 1 -- 172.21.15.120:0/1467522636 shutdown_connections 2024-09-19T09:08:31.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.943+0000 7fa8b4681740 1 -- 172.21.15.120:0/1467522636 wait complete. 2024-09-19T09:08:31.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.944+0000 7fa8b4681740 20 client.26741 trim_cache size 0 max 16384 2024-09-19T09:08:31.959 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:08:31.959 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:08:31.960 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-19T09:08:31.960 INFO:teuthology.orchestra.run.smithi120.stdout: "d5bb44fa-e214-4948-abc0-987db37435ed": { 2024-09-19T09:08:31.960 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-19T09:08:31.960 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "d5bb44fa-e214-4948-abc0-987db37435ed", 2024-09-19T09:08:31.960 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-19T09:08:31.960 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:08:31.960 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:08:31.960 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:08:31.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.944+0000 7fa8b22a8640 1 -- 172.21.15.120:0/2180976484 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa8940a3240 msgr2=0x7fa8940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:31.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.944+0000 7fa8b22a8640 1 --2- 172.21.15.120:0/2180976484 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa8940a3240 0x7fa8940a5700 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0005ab0 tx=0x7fa8a00045f0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.944+0000 7fa8b22a8640 1 -- 172.21.15.120:0/2180976484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8ac102780 msgr2=0x7fa8ac198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:31.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.944+0000 7fa8b22a8640 1 --2- 172.21.15.120:0/2180976484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8ac102780 0x7fa8ac198a00 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0006010 tx=0x7fa8a0004040 comp rx=0 tx=0).stop 2024-09-19T09:08:31.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.944+0000 7fa8b22a8640 1 -- 172.21.15.120:0/2180976484 shutdown_connections 2024-09-19T09:08:31.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.944+0000 7fa8b22a8640 1 --2- 172.21.15.120:0/2180976484 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa8940a3240 0x7fa8940a5700 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.944+0000 7fa8b22a8640 1 --2- 172.21.15.120:0/2180976484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8ac106f00 0x7fa8ac19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.944+0000 7fa8b22a8640 1 --2- 172.21.15.120:0/2180976484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8ac102780 0x7fa8ac198a00 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.944+0000 7fa8b22a8640 1 --2- 172.21.15.120:0/2180976484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8ac101dd0 0x7fa8ac1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:31.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.944+0000 7fa8b22a8640 1 -- 172.21.15.120:0/2180976484 >> 172.21.15.120:0/2180976484 conn(0x7fa8ac0fd8b0 msgr2=0x7fa8ac0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:31.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.944+0000 7fa8b22a8640 1 -- 172.21.15.120:0/2180976484 shutdown_connections 2024-09-19T09:08:31.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:08:31.944+0000 7fa8b22a8640 1 -- 172.21.15.120:0/2180976484 wait complete. 2024-09-19T09:08:32.051 INFO:tasks.workunit.client.0.smithi120.stdout: 1 431677 7.47 MB/sec execute 147 sec latency 22.423 ms 2024-09-19T09:08:32.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:31 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:08:32.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:31 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:08:32.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:31 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:08:32.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:31 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:08:32.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:31 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:08:32.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:31 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:08:32.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:31 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:08:32.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:31 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:08:32.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:31 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:08:32.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:31 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:08:32.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:31 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:08:32.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:31 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:08:32.830 INFO:tasks.cephfs.filesystem:scrub status for tag:d5bb44fa-e214-4948-abc0-987db37435ed - {'path': '/', 'tag': 'd5bb44fa-e214-4948-abc0-987db37435ed', 'options': 'recursive,force'} 2024-09-19T09:08:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.852+0000 7f9363ae6640 1 -- 172.21.15.120:0/1489838692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f935c101c30 msgr2=0x7f935c1020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.852+0000 7f9363ae6640 1 --2- 172.21.15.120:0/1489838692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f935c101c30 0x7f935c1020b0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f934c009900 tx=0x7f934c02a410 comp rx=0 tx=0).stop 2024-09-19T09:08:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.852+0000 7f9363ae6640 1 -- 172.21.15.120:0/1489838692 shutdown_connections 2024-09-19T09:08:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.852+0000 7f9363ae6640 1 --2- 172.21.15.120:0/1489838692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f935c1025f0 0x7f935c108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.852+0000 7f9363ae6640 1 --2- 172.21.15.120:0/1489838692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f935c101c30 0x7f935c1020b0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.852+0000 7f9363ae6640 1 --2- 172.21.15.120:0/1489838692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f935c100a40 0x7f935c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.852+0000 7f9363ae6640 1 -- 172.21.15.120:0/1489838692 >> 172.21.15.120:0/1489838692 conn(0x7f935c0fc1e0 msgr2=0x7f935c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.852+0000 7f9363ae6640 1 -- 172.21.15.120:0/1489838692 shutdown_connections 2024-09-19T09:08:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.852+0000 7f9363ae6640 1 -- 172.21.15.120:0/1489838692 wait complete. 2024-09-19T09:08:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.853+0000 7f9363ae6640 1 Processor -- start 2024-09-19T09:08:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.853+0000 7f9363ae6640 1 -- start start 2024-09-19T09:08:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.853+0000 7f9363ae6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f935c100a40 0x7f935c06a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.853+0000 7f9363ae6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f935c101c30 0x7f935c19cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.853+0000 7f9363ae6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f935c1025f0 0x7f935c19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.853+0000 7f9363ae6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f935c10c6a0 con 0x7f935c100a40 2024-09-19T09:08:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.853+0000 7f9363ae6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f935c10c520 con 0x7f935c1025f0 2024-09-19T09:08:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.853+0000 7f9363ae6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f935c10c820 con 0x7f935c101c30 2024-09-19T09:08:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.853+0000 7f9361882640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f935c100a40 0x7f935c06a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.853+0000 7f9361882640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f935c100a40 0x7f935c06a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46906/0 (socket says 172.21.15.120:46906) 2024-09-19T09:08:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.853+0000 7f9361882640 1 -- 172.21.15.120:0/3863166923 learned_addr learned my addr 172.21.15.120:0/3863166923 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.853+0000 7f9362083640 1 --2- 172.21.15.120:0/3863166923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f935c1025f0 0x7f935c19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.853+0000 7f9361081640 1 --2- 172.21.15.120:0/3863166923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f935c101c30 0x7f935c19cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.854+0000 7f9361882640 1 -- 172.21.15.120:0/3863166923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f935c101c30 msgr2=0x7f935c19cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.854+0000 7f9361882640 1 --2- 172.21.15.120:0/3863166923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f935c101c30 0x7f935c19cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.854+0000 7f9361882640 1 -- 172.21.15.120:0/3863166923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f935c1025f0 msgr2=0x7f935c19f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.854+0000 7f9361882640 1 --2- 172.21.15.120:0/3863166923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f935c1025f0 0x7f935c19f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.854+0000 7f9361882640 1 -- 172.21.15.120:0/3863166923 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935c19fa40 con 0x7f935c100a40 2024-09-19T09:08:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.854+0000 7f9361882640 1 --2- 172.21.15.120:0/3863166923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f935c100a40 0x7f935c06a410 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f935800e9c0 tx=0x7f935800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.854+0000 7f9352ffd640 1 -- 172.21.15.120:0/3863166923 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935800cde0 con 0x7f935c100a40 2024-09-19T09:08:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.854+0000 7f9363ae6640 1 -- 172.21.15.120:0/3863166923 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f935c19fcd0 con 0x7f935c100a40 2024-09-19T09:08:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.854+0000 7f9352ffd640 1 -- 172.21.15.120:0/3863166923 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9358010910 con 0x7f935c100a40 2024-09-19T09:08:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.854+0000 7f9352ffd640 1 -- 172.21.15.120:0/3863166923 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9358018a10 con 0x7f935c100a40 2024-09-19T09:08:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.854+0000 7f9363ae6640 1 -- 172.21.15.120:0/3863166923 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f935c1a0190 con 0x7f935c100a40 2024-09-19T09:08:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.857+0000 7f9352ffd640 1 -- 172.21.15.120:0/3863166923 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f935801f020 con 0x7f935c100a40 2024-09-19T09:08:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.857+0000 7f936687d740 1 Processor -- start 2024-09-19T09:08:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.857+0000 7f936687d740 1 -- start start 2024-09-19T09:08:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.858+0000 7f936687d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56333ed78db0 0x56333ed791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.858+0000 7f936687d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56333ed796f0 0x56333ed7e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.858+0000 7f936687d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56333ed7e780 0x56333ed80c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.858+0000 7f936687d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56333ecd82f0 con 0x56333ed796f0 2024-09-19T09:08:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.858+0000 7f936687d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56333ec80270 con 0x56333ed78db0 2024-09-19T09:08:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.858+0000 7f936687d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56333ecd42b0 con 0x56333ed7e780 2024-09-19T09:08:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.858+0000 7f9361882640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56333ed796f0 0x56333ed7e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.858+0000 7f9362083640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56333ed78db0 0x56333ed791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.858+0000 7f9361081640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56333ed7e780 0x56333ed80c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.858+0000 7f9361882640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56333ed796f0 0x56333ed7e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46908/0 (socket says 172.21.15.120:46908) 2024-09-19T09:08:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.858+0000 7f9362083640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56333ed78db0 0x56333ed791b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:58078/0 (socket says 172.21.15.120:58078) 2024-09-19T09:08:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.858+0000 7f9361882640 1 -- 172.21.15.120:0/3202590558 learned_addr learned my addr 172.21.15.120:0/3202590558 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.858+0000 7f9352ffd640 1 --2- 172.21.15.120:0/3863166923 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93480a32f0 0x7f93480a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.858+0000 7f9361081640 1 -- 172.21.15.120:0/3202590558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56333ed78db0 msgr2=0x56333ed791b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.858+0000 7f9361081640 1 --2- 172.21.15.120:0/3202590558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56333ed78db0 0x56333ed791b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.858+0000 7f9361081640 1 -- 172.21.15.120:0/3202590558 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56333ed796f0 msgr2=0x56333ed7e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.858+0000 7f9361081640 1 --2- 172.21.15.120:0/3202590558 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56333ed796f0 0x56333ed7e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.858+0000 7f9361081640 1 -- 172.21.15.120:0/3202590558 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56333ec7f340 con 0x56333ed7e780 2024-09-19T09:08:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.858+0000 7f9362083640 1 --2- 172.21.15.120:0/3863166923 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93480a32f0 0x7f93480a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.858+0000 7f9352ffd640 1 -- 172.21.15.120:0/3863166923 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f93580cfb80 con 0x7f935c100a40 2024-09-19T09:08:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.858+0000 7f9361081640 1 --2- 172.21.15.120:0/3202590558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56333ed7e780 0x56333ed80c40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f934c0314e0 tx=0x7f934c005e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bfeb2cbed1e7156 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.858+0000 7f9347fff640 1 -- 172.21.15.120:0/3202590558 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c03f070 con 0x56333ed7e780 2024-09-19T09:08:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.859+0000 7f9347fff640 1 -- 172.21.15.120:0/3202590558 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f934c004030 con 0x56333ed7e780 2024-09-19T09:08:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.859+0000 7f9347fff640 1 -- 172.21.15.120:0/3202590558 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c035070 con 0x56333ed7e780 2024-09-19T09:08:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.859+0000 7f936687d740 1 -- 172.21.15.120:0/3202590558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56333ed7e780 msgr2=0x56333ed80c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.859+0000 7f9362083640 1 --2- 172.21.15.120:0/3863166923 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93480a32f0 0x7f93480a57b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9354009fd0 tx=0x7f9354004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.859+0000 7f936687d740 1 --2- 172.21.15.120:0/3202590558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56333ed7e780 0x56333ed80c40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f934c0314e0 tx=0x7f934c005e60 comp rx=0 tx=0).stop 2024-09-19T09:08:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.859+0000 7f936687d740 1 -- 172.21.15.120:0/3202590558 shutdown_connections 2024-09-19T09:08:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.859+0000 7f936687d740 1 --2- 172.21.15.120:0/3202590558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56333ed7e780 0x56333ed80c40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.859+0000 7f936687d740 1 --2- 172.21.15.120:0/3202590558 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56333ed796f0 0x56333ed7e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.859+0000 7f936687d740 1 --2- 172.21.15.120:0/3202590558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56333ed78db0 0x56333ed791b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.859+0000 7f936687d740 1 -- 172.21.15.120:0/3202590558 >> 172.21.15.120:0/3202590558 conn(0x56333ed70010 msgr2=0x56333ed70410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.859+0000 7f936687d740 1 -- 172.21.15.120:0/3202590558 shutdown_connections 2024-09-19T09:08:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.859+0000 7f936687d740 1 -- 172.21.15.120:0/3202590558 wait complete. 2024-09-19T09:08:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.860+0000 7f936687d740 1 -- start start 2024-09-19T09:08:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.860+0000 7f936687d740 1 Processor -- start 2024-09-19T09:08:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.860+0000 7f936687d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56333ed78db0 0x56333ed82f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.860+0000 7f936687d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56333ed796f0 0x56333ed834b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.860+0000 7f936687d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56333ed7e780 0x56333ed8a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.860+0000 7f936687d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56333ed816f0 con 0x56333ed78db0 2024-09-19T09:08:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.860+0000 7f936687d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56333ec61290 con 0x56333ed7e780 2024-09-19T09:08:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.860+0000 7f936687d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56333ed81870 con 0x56333ed796f0 2024-09-19T09:08:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.860+0000 7f936687d740 10 client.0 _ll_register_callbacks cb 0x56333ec92140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:08:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.860+0000 7f9361882640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56333ed78db0 0x56333ed82f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.860+0000 7f9361882640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56333ed78db0 0x56333ed82f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46922/0 (socket says 172.21.15.120:46922) 2024-09-19T09:08:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.860+0000 7f9361882640 1 -- 172.21.15.120:0/2483455188 learned_addr learned my addr 172.21.15.120:0/2483455188 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.860+0000 7f9361081640 1 --2- 172.21.15.120:0/2483455188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56333ed796f0 0x56333ed834b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.861+0000 7f9362083640 1 --2- 172.21.15.120:0/2483455188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56333ed7e780 0x56333ed8a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.861+0000 7f9361882640 1 -- 172.21.15.120:0/2483455188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56333ed796f0 msgr2=0x56333ed834b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.861+0000 7f9361882640 1 --2- 172.21.15.120:0/2483455188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56333ed796f0 0x56333ed834b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.861+0000 7f9361882640 1 -- 172.21.15.120:0/2483455188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56333ed7e780 msgr2=0x56333ed8a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.861+0000 7f9361882640 1 --2- 172.21.15.120:0/2483455188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56333ed7e780 0x56333ed8a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.861+0000 7f9361882640 1 -- 172.21.15.120:0/2483455188 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56333e9eb210 con 0x56333ed78db0 2024-09-19T09:08:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.861+0000 7f9361882640 1 --2- 172.21.15.120:0/2483455188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56333ed78db0 0x56333ed82f70 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f93580a48e0 tx=0x7f9358010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13adaf97910f9e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.861+0000 7f93457fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:08:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.861+0000 7f93457fa640 1 -- 172.21.15.120:0/2483455188 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935809bb00 con 0x56333ed78db0 2024-09-19T09:08:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.861+0000 7f93457fa640 1 -- 172.21.15.120:0/2483455188 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f9358010ac0 con 0x56333ed78db0 2024-09-19T09:08:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.861+0000 7f936687d740 1 -- 172.21.15.120:0/2483455188 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f934c004030 con 0x56333ed78db0 2024-09-19T09:08:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.861+0000 7f93457fa640 1 -- 172.21.15.120:0/2483455188 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f93580b5050 con 0x56333ed78db0 2024-09-19T09:08:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.861+0000 7f936687d740 10 client.17484 fetch_fsmap learned FSMap version 30 2024-09-19T09:08:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.861+0000 7f936687d740 1 -- 172.21.15.120:0/2483455188 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56333eab7620 con 0x56333ed78db0 2024-09-19T09:08:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.861+0000 7f93457fa640 1 -- 172.21.15.120:0/2483455188 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f934c004030 con 0x56333ed78db0 2024-09-19T09:08:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.862+0000 7f93457fa640 1 -- 172.21.15.120:0/2483455188 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f93580bc0d0 con 0x56333ed78db0 2024-09-19T09:08:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.862+0000 7f936687d740 10 client.17484 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:08:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.862+0000 7f936687d740 10 client.17484 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:08:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.862+0000 7f936687d740 20 client.17484 populate_metadata read hostname 'smithi120' 2024-09-19T09:08:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.862+0000 7f936687d740 1 --2- 172.21.15.120:0/2483455188 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56333ed8ec10 0x56333ed91000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.862+0000 7f936687d740 4 client.17484 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:08:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.862+0000 7f936687d740 1 -- 172.21.15.120:0/2483455188 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56333ece5a20 con 0x56333ed8ec10 2024-09-19T09:08:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.862+0000 7f9361081640 1 --2- 172.21.15.120:0/2483455188 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56333ed8ec10 0x56333ed91000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.862+0000 7f93457fa640 10 client.17484 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.862+0000 7f9361081640 1 --2- 172.21.15.120:0/2483455188 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56333ed8ec10 0x56333ed91000 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=982a0ba6001fb79c server_cookie=9e5f1360ee68ab08 in_seq=0 out_seq=0 2024-09-19T09:08:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.863+0000 7f93457fa640 1 -- 172.21.15.120:0/2483455188 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56333ece5a20 con 0x56333ed8ec10 2024-09-19T09:08:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.863+0000 7f93457fa640 10 client.17484 handle_command_reply: tid=0 2024-09-19T09:08:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.863+0000 7f936687d740 1 client.17484 shutdown 2024-09-19T09:08:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.864+0000 7f936687d740 1 -- 172.21.15.120:0/2483455188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56333ed78db0 msgr2=0x56333ed82f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.864+0000 7f936687d740 1 --2- 172.21.15.120:0/2483455188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56333ed78db0 0x56333ed82f70 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f93580a48e0 tx=0x7f9358010f10 comp rx=0 tx=0).stop 2024-09-19T09:08:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.864+0000 7f936687d740 1 -- 172.21.15.120:0/2483455188 shutdown_connections 2024-09-19T09:08:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.864+0000 7f936687d740 1 --2- 172.21.15.120:0/2483455188 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56333ed8ec10 0x56333ed91000 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.864+0000 7f936687d740 1 --2- 172.21.15.120:0/2483455188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56333ed7e780 0x56333ed8a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.864+0000 7f936687d740 1 --2- 172.21.15.120:0/2483455188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56333ed796f0 0x56333ed834b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.864+0000 7f93457fa640 0 client.17484 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.864+0000 7f936687d740 1 --2- 172.21.15.120:0/2483455188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56333ed78db0 0x56333ed82f70 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.864+0000 7f936687d740 1 -- 172.21.15.120:0/2483455188 >> 172.21.15.120:0/2483455188 conn(0x56333ed70010 msgr2=0x56333ed71820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.864+0000 7f936687d740 1 -- 172.21.15.120:0/2483455188 shutdown_connections 2024-09-19T09:08:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.864+0000 7f936687d740 1 -- 172.21.15.120:0/2483455188 wait complete. 2024-09-19T09:08:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.864+0000 7f936687d740 20 client.17484 trim_cache size 0 max 16384 2024-09-19T09:08:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.878+0000 7f936687d740 1 Processor -- start 2024-09-19T09:08:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.878+0000 7f936687d740 1 -- start start 2024-09-19T09:08:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.879+0000 7f936687d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56333ed78db0 0x56333ed70ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.879+0000 7f936687d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56333ed796f0 0x56333ed8a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.879+0000 7f936687d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56333ed7e780 0x56333ed71400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.879+0000 7f936687d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f935c1a6d30 con 0x56333ed796f0 2024-09-19T09:08:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.879+0000 7f936687d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f935c1a8870 con 0x56333ed78db0 2024-09-19T09:08:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.879+0000 7f936687d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f935c1a7400 con 0x56333ed7e780 2024-09-19T09:08:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.879+0000 7f9361081640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56333ed796f0 0x56333ed8a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.879+0000 7f9361081640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56333ed796f0 0x56333ed8a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46934/0 (socket says 172.21.15.120:46934) 2024-09-19T09:08:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.879+0000 7f9361081640 1 -- 172.21.15.120:0/2617462277 learned_addr learned my addr 172.21.15.120:0/2617462277 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.879+0000 7f9362083640 1 --2- 172.21.15.120:0/2617462277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56333ed7e780 0x56333ed71400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.879+0000 7f9361882640 1 --2- 172.21.15.120:0/2617462277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56333ed78db0 0x56333ed70ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.879+0000 7f9361081640 1 -- 172.21.15.120:0/2617462277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56333ed7e780 msgr2=0x56333ed71400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.879+0000 7f9361081640 1 --2- 172.21.15.120:0/2617462277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56333ed7e780 0x56333ed71400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.879+0000 7f9361081640 1 -- 172.21.15.120:0/2617462277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56333ed78db0 msgr2=0x56333ed70ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.879+0000 7f9361081640 1 --2- 172.21.15.120:0/2617462277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56333ed78db0 0x56333ed70ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.879+0000 7f9361081640 1 -- 172.21.15.120:0/2617462277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56333ed91370 con 0x56333ed796f0 2024-09-19T09:08:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.879+0000 7f9361081640 1 --2- 172.21.15.120:0/2617462277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56333ed796f0 0x56333ed8a480 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f934c005940 tx=0x7f934c02aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a011cc02608caff5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.879+0000 7f93477fe640 1 -- 172.21.15.120:0/2617462277 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c035040 con 0x56333ed796f0 2024-09-19T09:08:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.879+0000 7f93477fe640 1 -- 172.21.15.120:0/2617462277 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f934c0318b0 con 0x56333ed796f0 2024-09-19T09:08:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.879+0000 7f93477fe640 1 -- 172.21.15.120:0/2617462277 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c03f040 con 0x56333ed796f0 2024-09-19T09:08:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.880+0000 7f936687d740 1 -- 172.21.15.120:0/2617462277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56333ed796f0 msgr2=0x56333ed8a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.880+0000 7f936687d740 1 --2- 172.21.15.120:0/2617462277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56333ed796f0 0x56333ed8a480 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f934c005940 tx=0x7f934c02aa90 comp rx=0 tx=0).stop 2024-09-19T09:08:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.880+0000 7f936687d740 1 -- 172.21.15.120:0/2617462277 shutdown_connections 2024-09-19T09:08:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.880+0000 7f936687d740 1 --2- 172.21.15.120:0/2617462277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56333ed7e780 0x56333ed71400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.880+0000 7f936687d740 1 --2- 172.21.15.120:0/2617462277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56333ed796f0 0x56333ed8a480 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.880+0000 7f936687d740 1 --2- 172.21.15.120:0/2617462277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56333ed78db0 0x56333ed70ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.880+0000 7f936687d740 1 -- 172.21.15.120:0/2617462277 >> 172.21.15.120:0/2617462277 conn(0x56333ed70010 msgr2=0x56333ed82d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.880+0000 7f936687d740 1 -- 172.21.15.120:0/2617462277 shutdown_connections 2024-09-19T09:08:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.880+0000 7f936687d740 1 -- 172.21.15.120:0/2617462277 wait complete. 2024-09-19T09:08:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.880+0000 7f936687d740 1 -- start start 2024-09-19T09:08:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.881+0000 7f936687d740 1 Processor -- start 2024-09-19T09:08:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.881+0000 7f936687d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56333ed78db0 0x56333ed876d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.881+0000 7f936687d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56333ed796f0 0x56333ed87c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.881+0000 7f936687d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56333ed7e780 0x56333ed88c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.881+0000 7f936687d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f935c1a81a0 con 0x56333ed78db0 2024-09-19T09:08:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.881+0000 7f936687d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f935c1a7ad0 con 0x56333ed796f0 2024-09-19T09:08:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.881+0000 7f936687d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f935c1a4cf0 con 0x56333ed7e780 2024-09-19T09:08:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.881+0000 7f936687d740 10 client.0 _ll_register_callbacks cb 0x7f9358018420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:08:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.881+0000 7f9361882640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56333ed78db0 0x56333ed876d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.881+0000 7f9361882640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56333ed78db0 0x56333ed876d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46952/0 (socket says 172.21.15.120:46952) 2024-09-19T09:08:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.881+0000 7f9361882640 1 -- 172.21.15.120:0/1266751413 learned_addr learned my addr 172.21.15.120:0/1266751413 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.881+0000 7f9361081640 1 --2- 172.21.15.120:0/1266751413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56333ed796f0 0x56333ed87c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.881+0000 7f9362083640 1 --2- 172.21.15.120:0/1266751413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56333ed7e780 0x56333ed88c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.881+0000 7f9361882640 1 -- 172.21.15.120:0/1266751413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56333ed7e780 msgr2=0x56333ed88c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.881+0000 7f9361882640 1 --2- 172.21.15.120:0/1266751413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56333ed7e780 0x56333ed88c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.881+0000 7f9361882640 1 -- 172.21.15.120:0/1266751413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56333ed796f0 msgr2=0x56333ed87c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.881+0000 7f9361882640 1 --2- 172.21.15.120:0/1266751413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56333ed796f0 0x56333ed87c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.881+0000 7f9361882640 1 -- 172.21.15.120:0/1266751413 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56333ed89230 con 0x56333ed78db0 2024-09-19T09:08:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.881+0000 7f9361882640 1 --2- 172.21.15.120:0/1266751413 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56333ed78db0 0x56333ed876d0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f93580058c0 tx=0x7f935809be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405fe78a2f35d485 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.881+0000 7f93457fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:08:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.882+0000 7f93457fa640 1 -- 172.21.15.120:0/1266751413 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935809bb00 con 0x56333ed78db0 2024-09-19T09:08:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.882+0000 7f93457fa640 1 -- 172.21.15.120:0/1266751413 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f93580a4980 con 0x56333ed78db0 2024-09-19T09:08:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.882+0000 7f936687d740 1 -- 172.21.15.120:0/1266751413 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f934c0318b0 con 0x56333ed78db0 2024-09-19T09:08:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.882+0000 7f93457fa640 1 -- 172.21.15.120:0/1266751413 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f935809c8a0 con 0x56333ed78db0 2024-09-19T09:08:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.882+0000 7f936687d740 10 client.17490 fetch_fsmap learned FSMap version 30 2024-09-19T09:08:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.882+0000 7f936687d740 1 -- 172.21.15.120:0/1266751413 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56333ed8a5b0 con 0x56333ed78db0 2024-09-19T09:08:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.882+0000 7f93457fa640 1 -- 172.21.15.120:0/1266751413 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f934c0318b0 con 0x56333ed78db0 2024-09-19T09:08:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.882+0000 7f93457fa640 1 -- 172.21.15.120:0/1266751413 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f93580cdb40 con 0x56333ed78db0 2024-09-19T09:08:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.882+0000 7f936687d740 10 client.17490 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:08:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.882+0000 7f936687d740 10 client.17490 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:08:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.882+0000 7f936687d740 20 client.17490 populate_metadata read hostname 'smithi120' 2024-09-19T09:08:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.882+0000 7f936687d740 1 --2- 172.21.15.120:0/1266751413 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56333edf9cf0 0x56333edfc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.882+0000 7f936687d740 4 client.17490 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:08:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.882+0000 7f936687d740 1 -- 172.21.15.120:0/1266751413 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56333ecdfd70 con 0x56333edf9cf0 2024-09-19T09:08:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.882+0000 7f9361081640 1 --2- 172.21.15.120:0/1266751413 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56333edf9cf0 0x56333edfc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.883+0000 7f9361081640 1 --2- 172.21.15.120:0/1266751413 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56333edf9cf0 0x56333edfc110 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1daa8f371470 server_cookie=2063f127014b4ecc in_seq=0 out_seq=0 2024-09-19T09:08:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:32.883+0000 7f93457fa640 10 client.17490 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:33.025+0000 7f93457fa640 1 -- 172.21.15.120:0/1266751413 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56333ecdfd70 con 0x56333edf9cf0 2024-09-19T09:08:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:33.025+0000 7f93457fa640 10 client.17490 handle_command_reply: tid=0 2024-09-19T09:08:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:33.025+0000 7f936687d740 1 client.17490 shutdown 2024-09-19T09:08:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:33.025+0000 7f936687d740 1 -- 172.21.15.120:0/1266751413 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56333ed78db0 msgr2=0x56333ed876d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:33.025+0000 7f936687d740 1 --2- 172.21.15.120:0/1266751413 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56333ed78db0 0x56333ed876d0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f93580058c0 tx=0x7f935809be10 comp rx=0 tx=0).stop 2024-09-19T09:08:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:33.025+0000 7f936687d740 1 -- 172.21.15.120:0/1266751413 shutdown_connections 2024-09-19T09:08:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:33.025+0000 7f936687d740 1 --2- 172.21.15.120:0/1266751413 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56333edf9cf0 0x56333edfc110 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:33.025+0000 7f936687d740 1 --2- 172.21.15.120:0/1266751413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56333ed7e780 0x56333ed88c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:33.025+0000 7f936687d740 1 --2- 172.21.15.120:0/1266751413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56333ed796f0 0x56333ed87c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:33.025+0000 7f936687d740 1 --2- 172.21.15.120:0/1266751413 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56333ed78db0 0x56333ed876d0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:33.025+0000 7f936687d740 1 -- 172.21.15.120:0/1266751413 >> 172.21.15.120:0/1266751413 conn(0x56333ed70010 msgr2=0x56333ed6e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:33.025+0000 7f93457fa640 0 client.17490 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:33.025+0000 7f936687d740 1 -- 172.21.15.120:0/1266751413 shutdown_connections 2024-09-19T09:08:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:33.025+0000 7f936687d740 1 -- 172.21.15.120:0/1266751413 wait complete. 2024-09-19T09:08:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:33.025+0000 7f936687d740 20 client.17490 trim_cache size 0 max 16384 2024-09-19T09:08:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:08:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:08:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:08:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:08:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:33.026+0000 7f9363ae6640 1 -- 172.21.15.120:0/3863166923 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93480a32f0 msgr2=0x7f93480a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:33.026+0000 7f9363ae6640 1 --2- 172.21.15.120:0/3863166923 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93480a32f0 0x7f93480a57b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9354009fd0 tx=0x7f9354004a40 comp rx=0 tx=0).stop 2024-09-19T09:08:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:33.026+0000 7f9363ae6640 1 -- 172.21.15.120:0/3863166923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f935c100a40 msgr2=0x7f935c06a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:33.027+0000 7f9363ae6640 1 --2- 172.21.15.120:0/3863166923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f935c100a40 0x7f935c06a410 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f935800e9c0 tx=0x7f935800ee80 comp rx=0 tx=0).stop 2024-09-19T09:08:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:33.027+0000 7f9363ae6640 1 -- 172.21.15.120:0/3863166923 shutdown_connections 2024-09-19T09:08:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:33.027+0000 7f9363ae6640 1 --2- 172.21.15.120:0/3863166923 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93480a32f0 0x7f93480a57b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:33.027+0000 7f9363ae6640 1 --2- 172.21.15.120:0/3863166923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f935c1025f0 0x7f935c19f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:33.027+0000 7f9363ae6640 1 --2- 172.21.15.120:0/3863166923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f935c101c30 0x7f935c19cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:33.027+0000 7f9363ae6640 1 --2- 172.21.15.120:0/3863166923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f935c100a40 0x7f935c06a410 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:33.027+0000 7f9363ae6640 1 -- 172.21.15.120:0/3863166923 >> 172.21.15.120:0/3863166923 conn(0x7f935c0fc1e0 msgr2=0x7f935c104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:33.028+0000 7f9363ae6640 1 -- 172.21.15.120:0/3863166923 shutdown_connections 2024-09-19T09:08:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:33.028+0000 7f9363ae6640 1 -- 172.21.15.120:0/3863166923 wait complete. 2024-09-19T09:08:33.051 INFO:tasks.workunit.client.0.smithi120.stdout: 1 433834 7.51 MB/sec execute 148 sec latency 37.232 ms 2024-09-19T09:08:33.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:32 smithi120 ceph-mon[31150]: pgmap v512: 97 pgs: 97 active+clean; 42 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.9 MiB/s wr, 348 op/s 2024-09-19T09:08:33.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:32 smithi196 ceph-mon[35150]: pgmap v512: 97 pgs: 97 active+clean; 42 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.9 MiB/s wr, 348 op/s 2024-09-19T09:08:33.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:32 smithi153 ceph-mon[35071]: pgmap v512: 97 pgs: 97 active+clean; 42 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.9 MiB/s wr, 348 op/s 2024-09-19T09:08:34.051 INFO:tasks.workunit.client.0.smithi120.stdout: 1 436873 7.57 MB/sec execute 149 sec latency 10.706 ms 2024-09-19T09:08:34.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:33 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:08:34.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:33 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:08:34.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:33 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:08:35.051 INFO:tasks.workunit.client.0.smithi120.stdout: 1 440136 7.65 MB/sec execute 150 sec latency 11.097 ms 2024-09-19T09:08:35.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:34 smithi196 ceph-mon[35150]: pgmap v513: 97 pgs: 97 active+clean; 41 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.0 MiB/s wr, 353 op/s 2024-09-19T09:08:35.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:34 smithi153 ceph-mon[35071]: pgmap v513: 97 pgs: 97 active+clean; 41 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.0 MiB/s wr, 353 op/s 2024-09-19T09:08:35.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:34 smithi120 ceph-mon[31150]: pgmap v513: 97 pgs: 97 active+clean; 41 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.0 MiB/s wr, 353 op/s 2024-09-19T09:08:36.051 INFO:tasks.workunit.client.0.smithi120.stdout: 1 443365 7.72 MB/sec execute 151 sec latency 12.192 ms 2024-09-19T09:08:37.051 INFO:tasks.workunit.client.0.smithi120.stdout: 1 445905 7.73 MB/sec execute 152 sec latency 18.840 ms 2024-09-19T09:08:37.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:36 smithi196 ceph-mon[35150]: pgmap v514: 97 pgs: 97 active+clean; 38 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.7 MiB/s wr, 280 op/s 2024-09-19T09:08:37.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:36 smithi153 ceph-mon[35071]: pgmap v514: 97 pgs: 97 active+clean; 38 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.7 MiB/s wr, 280 op/s 2024-09-19T09:08:37.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:36 smithi120 ceph-mon[31150]: pgmap v514: 97 pgs: 97 active+clean; 38 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.7 MiB/s wr, 280 op/s 2024-09-19T09:08:38.051 INFO:tasks.workunit.client.0.smithi120.stdout: 1 448638 7.78 MB/sec execute 153 sec latency 15.740 ms 2024-09-19T09:08:39.051 INFO:tasks.workunit.client.0.smithi120.stdout: 1 451369 7.83 MB/sec execute 154 sec latency 10.644 ms 2024-09-19T09:08:39.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:38 smithi196 ceph-mon[35150]: pgmap v515: 97 pgs: 97 active+clean; 35 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.4 MiB/s wr, 280 op/s 2024-09-19T09:08:39.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:38 smithi153 ceph-mon[35071]: pgmap v515: 97 pgs: 97 active+clean; 35 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.4 MiB/s wr, 280 op/s 2024-09-19T09:08:39.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:38 smithi120 ceph-mon[31150]: pgmap v515: 97 pgs: 97 active+clean; 35 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.4 MiB/s wr, 280 op/s 2024-09-19T09:08:40.051 INFO:tasks.workunit.client.0.smithi120.stdout: 1 454541 7.90 MB/sec execute 155 sec latency 6.560 ms 2024-09-19T09:08:41.051 INFO:tasks.workunit.client.0.smithi120.stdout: 1 457838 7.97 MB/sec execute 156 sec latency 15.500 ms 2024-09-19T09:08:41.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:40 smithi196 ceph-mon[35150]: pgmap v516: 97 pgs: 97 active+clean; 38 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.6 MiB/s wr, 339 op/s 2024-09-19T09:08:41.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:40 smithi153 ceph-mon[35071]: pgmap v516: 97 pgs: 97 active+clean; 38 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.6 MiB/s wr, 339 op/s 2024-09-19T09:08:41.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:40 smithi120 ceph-mon[31150]: pgmap v516: 97 pgs: 97 active+clean; 38 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.6 MiB/s wr, 339 op/s 2024-09-19T09:08:42.051 INFO:tasks.workunit.client.0.smithi120.stdout: 1 459595 8.07 MB/sec execute 157 sec latency 57.179 ms 2024-09-19T09:08:43.051 INFO:tasks.workunit.client.0.smithi120.stdout: 1 462694 8.13 MB/sec execute 158 sec latency 8.836 ms 2024-09-19T09:08:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:42 smithi196 ceph-mon[35150]: pgmap v517: 97 pgs: 97 active+clean; 44 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.1 MiB/s wr, 418 op/s 2024-09-19T09:08:43.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:42 smithi153 ceph-mon[35071]: pgmap v517: 97 pgs: 97 active+clean; 44 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.1 MiB/s wr, 418 op/s 2024-09-19T09:08:43.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:42 smithi120 ceph-mon[31150]: pgmap v517: 97 pgs: 97 active+clean; 44 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.1 MiB/s wr, 418 op/s 2024-09-19T09:08:44.051 INFO:tasks.workunit.client.0.smithi120.stdout: 1 465886 8.19 MB/sec execute 159 sec latency 23.352 ms 2024-09-19T09:08:45.051 INFO:tasks.workunit.client.0.smithi120.stdout: 1 469262 8.26 MB/sec execute 160 sec latency 11.117 ms 2024-09-19T09:08:45.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:44 smithi196 ceph-mon[35150]: pgmap v518: 97 pgs: 97 active+clean; 38 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.7 MiB/s wr, 345 op/s 2024-09-19T09:08:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:44 smithi153 ceph-mon[35071]: pgmap v518: 97 pgs: 97 active+clean; 38 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.7 MiB/s wr, 345 op/s 2024-09-19T09:08:45.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:44 smithi120 ceph-mon[31150]: pgmap v518: 97 pgs: 97 active+clean; 38 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.7 MiB/s wr, 345 op/s 2024-09-19T09:08:46.051 INFO:tasks.workunit.client.0.smithi120.stdout: 1 472533 8.32 MB/sec execute 161 sec latency 10.224 ms 2024-09-19T09:08:46.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.118+0000 7f22b759d640 1 -- 172.21.15.120:0/3983032855 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22b0101a30 msgr2=0x7f22b0101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:46.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.118+0000 7f22b759d640 1 --2- 172.21.15.120:0/3983032855 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22b0101a30 0x7f22b0101eb0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f22a0009900 tx=0x7f22a002a410 comp rx=0 tx=0).stop 2024-09-19T09:08:46.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.118+0000 7f22b759d640 1 -- 172.21.15.120:0/3983032855 shutdown_connections 2024-09-19T09:08:46.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.118+0000 7f22b759d640 1 --2- 172.21.15.120:0/3983032855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22b01023f0 0x7f22b0108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.118+0000 7f22b759d640 1 --2- 172.21.15.120:0/3983032855 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22b0101a30 0x7f22b0101eb0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.118+0000 7f22b759d640 1 --2- 172.21.15.120:0/3983032855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22b0100840 0x7f22b0100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.118+0000 7f22b759d640 1 -- 172.21.15.120:0/3983032855 >> 172.21.15.120:0/3983032855 conn(0x7f22b00fbfa0 msgr2=0x7f22b00fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:46.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.118+0000 7f22b759d640 1 -- 172.21.15.120:0/3983032855 shutdown_connections 2024-09-19T09:08:46.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.118+0000 7f22b759d640 1 -- 172.21.15.120:0/3983032855 wait complete. 2024-09-19T09:08:46.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.119+0000 7f22b759d640 1 Processor -- start 2024-09-19T09:08:46.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.119+0000 7f22b759d640 1 -- start start 2024-09-19T09:08:46.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.119+0000 7f22b759d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22b0100840 0x7f22b0197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:46.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.119+0000 7f22b759d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22b0101a30 0x7f22b0198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:46.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.119+0000 7f22b759d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22b01023f0 0x7f22b019f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:46.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.119+0000 7f22b759d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f22b010c4c0 con 0x7f22b0100840 2024-09-19T09:08:46.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.119+0000 7f22b759d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f22b010c340 con 0x7f22b01023f0 2024-09-19T09:08:46.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.119+0000 7f22b759d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f22b010c640 con 0x7f22b0101a30 2024-09-19T09:08:46.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.119+0000 7f22b659b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22b0100840 0x7f22b0197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:46.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.119+0000 7f22b659b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22b0100840 0x7f22b0197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36322/0 (socket says 172.21.15.120:36322) 2024-09-19T09:08:46.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.119+0000 7f22b659b640 1 -- 172.21.15.120:0/819475969 learned_addr learned my addr 172.21.15.120:0/819475969 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:46.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.119+0000 7f22b6d9c640 1 --2- 172.21.15.120:0/819475969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22b01023f0 0x7f22b019f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:46.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.119+0000 7f22b5d9a640 1 --2- 172.21.15.120:0/819475969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22b0101a30 0x7f22b0198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:46.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.119+0000 7f22b659b640 1 -- 172.21.15.120:0/819475969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22b0101a30 msgr2=0x7f22b0198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:46.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.119+0000 7f22b659b640 1 --2- 172.21.15.120:0/819475969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22b0101a30 0x7f22b0198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.119+0000 7f22b659b640 1 -- 172.21.15.120:0/819475969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22b01023f0 msgr2=0x7f22b019f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:46.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.119+0000 7f22b659b640 1 --2- 172.21.15.120:0/819475969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22b01023f0 0x7f22b019f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.119+0000 7f22b659b640 1 -- 172.21.15.120:0/819475969 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b019f900 con 0x7f22b0100840 2024-09-19T09:08:46.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.119+0000 7f22b659b640 1 --2- 172.21.15.120:0/819475969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22b0100840 0x7f22b0197cf0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00d950 tx=0x7f22ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:46.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.120+0000 7f22a77fe640 1 -- 172.21.15.120:0/819475969 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac017070 con 0x7f22b0100840 2024-09-19T09:08:46.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.120+0000 7f22a77fe640 1 -- 172.21.15.120:0/819475969 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22ac009c30 con 0x7f22b0100840 2024-09-19T09:08:46.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.120+0000 7f22b759d640 1 -- 172.21.15.120:0/819475969 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22b019fbf0 con 0x7f22b0100840 2024-09-19T09:08:46.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.120+0000 7f22a77fe640 1 -- 172.21.15.120:0/819475969 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac00f600 con 0x7f22b0100840 2024-09-19T09:08:46.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.120+0000 7f22b759d640 1 -- 172.21.15.120:0/819475969 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22b01a00f0 con 0x7f22b0100840 2024-09-19T09:08:46.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.122+0000 7f22a77fe640 1 -- 172.21.15.120:0/819475969 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f22ac01a070 con 0x7f22b0100840 2024-09-19T09:08:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.122+0000 7f22bb66e740 1 Processor -- start 2024-09-19T09:08:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.122+0000 7f22bb66e740 1 -- start start 2024-09-19T09:08:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.123+0000 7f22bb66e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9a53ef80 0x558d9a53f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.123+0000 7f22bb66e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9a53f8c0 0x558d9a544380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.123+0000 7f22bb66e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9a544950 0x558d9a546e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.123+0000 7f22bb66e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558d9a49e2f0 con 0x558d9a53ef80 2024-09-19T09:08:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.123+0000 7f22bb66e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558d9a446270 con 0x558d9a53f8c0 2024-09-19T09:08:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.123+0000 7f22bb66e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558d9a49a2b0 con 0x558d9a544950 2024-09-19T09:08:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.123+0000 7f22b6d9c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9a53ef80 0x558d9a53f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.123+0000 7f22b6d9c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9a53ef80 0x558d9a53f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36326/0 (socket says 172.21.15.120:36326) 2024-09-19T09:08:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.123+0000 7f22b6d9c640 1 -- 172.21.15.120:0/3946862155 learned_addr learned my addr 172.21.15.120:0/3946862155 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.123+0000 7f22b659b640 1 --2- 172.21.15.120:0/3946862155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9a53f8c0 0x558d9a544380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.123+0000 7f22b5d9a640 1 --2- 172.21.15.120:0/3946862155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9a544950 0x558d9a546e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.123+0000 7f22a77fe640 1 --2- 172.21.15.120:0/819475969 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f229c0a3240 0x7f229c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.123+0000 7f22b6d9c640 1 -- 172.21.15.120:0/3946862155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9a544950 msgr2=0x558d9a546e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.123+0000 7f22b6d9c640 1 --2- 172.21.15.120:0/3946862155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9a544950 0x558d9a546e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.123+0000 7f22b6d9c640 1 -- 172.21.15.120:0/3946862155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9a53f8c0 msgr2=0x558d9a544380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.123+0000 7f22b6d9c640 1 --2- 172.21.15.120:0/3946862155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9a53f8c0 0x558d9a544380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.123+0000 7f22b6d9c640 1 -- 172.21.15.120:0/3946862155 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d9a445340 con 0x558d9a53ef80 2024-09-19T09:08:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.123+0000 7f22b5d9a640 1 --2- 172.21.15.120:0/3946862155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9a544950 0x558d9a546e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:08:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.123+0000 7f22b659b640 1 --2- 172.21.15.120:0/3946862155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9a53f8c0 0x558d9a544380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:08:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.123+0000 7f22a77fe640 1 -- 172.21.15.120:0/819475969 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f22ac0cf3d0 con 0x7f22b0100840 2024-09-19T09:08:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.123+0000 7f22b6d9c640 1 --2- 172.21.15.120:0/3946862155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9a53ef80 0x558d9a53f380 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f22a8010370 tx=0x7f22a8010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d68febbdab48839 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.123+0000 7f22b6d9c640 1 --2- 172.21.15.120:0/819475969 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f229c0a3240 0x7f229c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.124+0000 7f22a4ff9640 1 -- 172.21.15.120:0/3946862155 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a8022070 con 0x558d9a53ef80 2024-09-19T09:08:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.124+0000 7f22a4ff9640 1 -- 172.21.15.120:0/3946862155 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22a80113e0 con 0x558d9a53ef80 2024-09-19T09:08:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.124+0000 7f22a4ff9640 1 -- 172.21.15.120:0/3946862155 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a8016070 con 0x558d9a53ef80 2024-09-19T09:08:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.125+0000 7f22bb66e740 1 -- 172.21.15.120:0/3946862155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9a53ef80 msgr2=0x558d9a53f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.125+0000 7f22bb66e740 1 --2- 172.21.15.120:0/3946862155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9a53ef80 0x558d9a53f380 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f22a8010370 tx=0x7f22a8010830 comp rx=0 tx=0).stop 2024-09-19T09:08:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.125+0000 7f22b6d9c640 1 --2- 172.21.15.120:0/819475969 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f229c0a3240 0x7f229c0a5700 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f22a801f000 tx=0x7f22a8022000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.125+0000 7f22bb66e740 1 -- 172.21.15.120:0/3946862155 shutdown_connections 2024-09-19T09:08:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.125+0000 7f22bb66e740 1 --2- 172.21.15.120:0/3946862155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9a544950 0x558d9a546e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.125+0000 7f22bb66e740 1 --2- 172.21.15.120:0/3946862155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9a53f8c0 0x558d9a544380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.125+0000 7f22bb66e740 1 --2- 172.21.15.120:0/3946862155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9a53ef80 0x558d9a53f380 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.125+0000 7f22bb66e740 1 -- 172.21.15.120:0/3946862155 >> 172.21.15.120:0/3946862155 conn(0x558d9a5361e0 msgr2=0x558d9a5365e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.125+0000 7f22bb66e740 1 -- 172.21.15.120:0/3946862155 shutdown_connections 2024-09-19T09:08:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.125+0000 7f22bb66e740 1 -- 172.21.15.120:0/3946862155 wait complete. 2024-09-19T09:08:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.125+0000 7f22bb66e740 1 -- start start 2024-09-19T09:08:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.125+0000 7f22bb66e740 1 Processor -- start 2024-09-19T09:08:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.126+0000 7f22bb66e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9a53ef80 0x558d9a5491b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.126+0000 7f22bb66e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9a53f8c0 0x558d9a5496f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.126+0000 7f22bb66e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9a544950 0x558d9a550730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.126+0000 7f22bb66e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558d9a547920 con 0x558d9a53f8c0 2024-09-19T09:08:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.126+0000 7f22bb66e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558d9a427290 con 0x558d9a53ef80 2024-09-19T09:08:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.126+0000 7f22bb66e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558d9a547aa0 con 0x558d9a544950 2024-09-19T09:08:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.126+0000 7f22bb66e740 10 client.0 _ll_register_callbacks cb 0x558d9a458140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:08:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.126+0000 7f22b5d9a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9a53f8c0 0x558d9a5496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.126+0000 7f22b659b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9a53ef80 0x558d9a5491b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.126+0000 7f22b5d9a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9a53f8c0 0x558d9a5496f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36334/0 (socket says 172.21.15.120:36334) 2024-09-19T09:08:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.126+0000 7f22b5d9a640 1 -- 172.21.15.120:0/570665616 learned_addr learned my addr 172.21.15.120:0/570665616 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.126+0000 7f22b6d9c640 1 --2- 172.21.15.120:0/570665616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9a544950 0x558d9a550730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.126+0000 7f22b5d9a640 1 -- 172.21.15.120:0/570665616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9a544950 msgr2=0x558d9a550730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.126+0000 7f22b5d9a640 1 --2- 172.21.15.120:0/570665616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9a544950 0x558d9a550730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.126+0000 7f22b5d9a640 1 -- 172.21.15.120:0/570665616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9a53ef80 msgr2=0x558d9a5491b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.126+0000 7f22b5d9a640 1 --2- 172.21.15.120:0/570665616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9a53ef80 0x558d9a5491b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.126+0000 7f22b5d9a640 1 -- 172.21.15.120:0/570665616 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d9a1b1210 con 0x558d9a53f8c0 2024-09-19T09:08:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.126+0000 7f22b5d9a640 1 --2- 172.21.15.120:0/570665616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9a53f8c0 0x558d9a5496f0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f22a002a930 tx=0x7f22a0009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc9a49f9f894193 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.126+0000 7f2299ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:08:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.126+0000 7f2299ffb640 1 -- 172.21.15.120:0/570665616 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0035040 con 0x558d9a53f8c0 2024-09-19T09:08:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.126+0000 7f2299ffb640 1 -- 172.21.15.120:0/570665616 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f22a00319b0 con 0x558d9a53f8c0 2024-09-19T09:08:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.126+0000 7f22bb66e740 1 -- 172.21.15.120:0/570665616 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22a80113e0 con 0x558d9a53f8c0 2024-09-19T09:08:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.126+0000 7f2299ffb640 1 -- 172.21.15.120:0/570665616 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f22a0004040 con 0x558d9a53f8c0 2024-09-19T09:08:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.126+0000 7f22bb66e740 10 client.17502 fetch_fsmap learned FSMap version 30 2024-09-19T09:08:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.127+0000 7f22bb66e740 1 -- 172.21.15.120:0/570665616 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558d9a27d620 con 0x558d9a53f8c0 2024-09-19T09:08:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.127+0000 7f2299ffb640 1 -- 172.21.15.120:0/570665616 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22a0005510 con 0x558d9a53f8c0 2024-09-19T09:08:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.127+0000 7f2299ffb640 1 -- 172.21.15.120:0/570665616 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f22a0004ba0 con 0x558d9a53f8c0 2024-09-19T09:08:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.127+0000 7f22bb66e740 10 client.17502 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:08:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.127+0000 7f22bb66e740 10 client.17502 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:08:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.127+0000 7f22bb66e740 20 client.17502 populate_metadata read hostname 'smithi120' 2024-09-19T09:08:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.127+0000 7f22bb66e740 1 --2- 172.21.15.120:0/570665616 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558d9a554e70 0x558d9a557260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.127+0000 7f22bb66e740 4 client.17502 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:08:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.127+0000 7f22bb66e740 1 -- 172.21.15.120:0/570665616 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f22a80113e0 con 0x558d9a554e70 2024-09-19T09:08:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.127+0000 7f22b659b640 1 --2- 172.21.15.120:0/570665616 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558d9a554e70 0x558d9a557260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.127+0000 7f22b659b640 1 --2- 172.21.15.120:0/570665616 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558d9a554e70 0x558d9a557260 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e53d9ddd4b9b7629 server_cookie=be0dbb19f6b933a7 in_seq=0 out_seq=0 2024-09-19T09:08:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.127+0000 7f2299ffb640 10 client.17502 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.128+0000 7f2299ffb640 1 -- 172.21.15.120:0/570665616 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f22a80113e0 con 0x558d9a554e70 2024-09-19T09:08:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.128+0000 7f2299ffb640 10 client.17502 handle_command_reply: tid=0 2024-09-19T09:08:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.128+0000 7f22bb66e740 1 client.17502 shutdown 2024-09-19T09:08:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.129+0000 7f22bb66e740 1 -- 172.21.15.120:0/570665616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9a53f8c0 msgr2=0x558d9a5496f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.129+0000 7f22bb66e740 1 --2- 172.21.15.120:0/570665616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9a53f8c0 0x558d9a5496f0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f22a002a930 tx=0x7f22a0009670 comp rx=0 tx=0).stop 2024-09-19T09:08:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.129+0000 7f22bb66e740 1 -- 172.21.15.120:0/570665616 shutdown_connections 2024-09-19T09:08:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.129+0000 7f22bb66e740 1 --2- 172.21.15.120:0/570665616 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558d9a554e70 0x558d9a557260 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.129+0000 7f22bb66e740 1 --2- 172.21.15.120:0/570665616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9a544950 0x558d9a550730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.129+0000 7f22bb66e740 1 --2- 172.21.15.120:0/570665616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9a53f8c0 0x558d9a5496f0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.129+0000 7f2299ffb640 0 client.17502 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.129+0000 7f22bb66e740 1 --2- 172.21.15.120:0/570665616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9a53ef80 0x558d9a5491b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.129+0000 7f22bb66e740 1 -- 172.21.15.120:0/570665616 >> 172.21.15.120:0/570665616 conn(0x558d9a5361e0 msgr2=0x558d9a5379f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.129+0000 7f22bb66e740 1 -- 172.21.15.120:0/570665616 shutdown_connections 2024-09-19T09:08:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.129+0000 7f22bb66e740 1 -- 172.21.15.120:0/570665616 wait complete. 2024-09-19T09:08:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.129+0000 7f22bb66e740 20 client.17502 trim_cache size 0 max 16384 2024-09-19T09:08:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.143+0000 7f22bb66e740 1 Processor -- start 2024-09-19T09:08:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.143+0000 7f22bb66e740 1 -- start start 2024-09-19T09:08:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.143+0000 7f22bb66e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9a53ef80 0x558d9a537ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.143+0000 7f22bb66e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9a53f8c0 0x558d9a5506f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.143+0000 7f22bb66e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9a544950 0x558d9a55bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.143+0000 7f22bb66e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f22b01a6cf0 con 0x558d9a544950 2024-09-19T09:08:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.143+0000 7f22bb66e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f22b01a8830 con 0x558d9a53ef80 2024-09-19T09:08:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.143+0000 7f22bb66e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f22b01a73c0 con 0x558d9a53f8c0 2024-09-19T09:08:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.143+0000 7f22b6d9c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9a544950 0x558d9a55bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.144+0000 7f22b6d9c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9a544950 0x558d9a55bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36348/0 (socket says 172.21.15.120:36348) 2024-09-19T09:08:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.144+0000 7f22b659b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9a53ef80 0x558d9a537ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.144+0000 7f22b6d9c640 1 -- 172.21.15.120:0/283674999 learned_addr learned my addr 172.21.15.120:0/283674999 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.144+0000 7f22b5d9a640 1 --2- 172.21.15.120:0/283674999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9a53f8c0 0x558d9a5506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.144+0000 7f22b6d9c640 1 -- 172.21.15.120:0/283674999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9a53f8c0 msgr2=0x558d9a5506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.144+0000 7f22b6d9c640 1 --2- 172.21.15.120:0/283674999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9a53f8c0 0x558d9a5506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.144+0000 7f22b6d9c640 1 -- 172.21.15.120:0/283674999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9a53ef80 msgr2=0x558d9a537ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.144+0000 7f22b6d9c640 1 --2- 172.21.15.120:0/283674999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9a53ef80 0x558d9a537ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.144+0000 7f22b6d9c640 1 -- 172.21.15.120:0/283674999 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d9a536000 con 0x558d9a544950 2024-09-19T09:08:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.144+0000 7f22b6d9c640 1 --2- 172.21.15.120:0/283674999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9a544950 0x558d9a55bb40 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f22a8010d30 tx=0x7f22a8027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303fade9886776c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.144+0000 7f229bfff640 1 -- 172.21.15.120:0/283674999 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a8016040 con 0x558d9a544950 2024-09-19T09:08:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.144+0000 7f229bfff640 1 -- 172.21.15.120:0/283674999 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22a8005bf0 con 0x558d9a544950 2024-09-19T09:08:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.144+0000 7f229bfff640 1 -- 172.21.15.120:0/283674999 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a8004500 con 0x558d9a544950 2024-09-19T09:08:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.145+0000 7f22bb66e740 1 -- 172.21.15.120:0/283674999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9a544950 msgr2=0x558d9a55bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.145+0000 7f22bb66e740 1 --2- 172.21.15.120:0/283674999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9a544950 0x558d9a55bb40 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f22a8010d30 tx=0x7f22a8027000 comp rx=0 tx=0).stop 2024-09-19T09:08:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.145+0000 7f22bb66e740 1 -- 172.21.15.120:0/283674999 shutdown_connections 2024-09-19T09:08:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.145+0000 7f22bb66e740 1 --2- 172.21.15.120:0/283674999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9a544950 0x558d9a55bb40 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.145+0000 7f22bb66e740 1 --2- 172.21.15.120:0/283674999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9a53f8c0 0x558d9a5506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.145+0000 7f22bb66e740 1 --2- 172.21.15.120:0/283674999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9a53ef80 0x558d9a537ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.145+0000 7f22bb66e740 1 -- 172.21.15.120:0/283674999 >> 172.21.15.120:0/283674999 conn(0x558d9a5361e0 msgr2=0x558d9a5365c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.145+0000 7f22bb66e740 1 -- 172.21.15.120:0/283674999 shutdown_connections 2024-09-19T09:08:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.145+0000 7f22bb66e740 1 -- 172.21.15.120:0/283674999 wait complete. 2024-09-19T09:08:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.145+0000 7f22bb66e740 1 -- start start 2024-09-19T09:08:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.145+0000 7f22bb66e740 1 Processor -- start 2024-09-19T09:08:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.145+0000 7f22bb66e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9a53ef80 0x558d9a54e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.145+0000 7f22bb66e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9a53f8c0 0x558d9a54eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.145+0000 7f22bb66e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9a544950 0x558d9a54fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:46.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.145+0000 7f22bb66e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f22b01a8160 con 0x558d9a53ef80 2024-09-19T09:08:46.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.146+0000 7f22bb66e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f22b01a7a90 con 0x558d9a53f8c0 2024-09-19T09:08:46.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.146+0000 7f22bb66e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f22b01a4cb0 con 0x558d9a544950 2024-09-19T09:08:46.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.146+0000 7f22bb66e740 10 client.0 _ll_register_callbacks cb 0x7f22a0009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:08:46.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.146+0000 7f22b659b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9a53ef80 0x558d9a54e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:46.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.146+0000 7f22b659b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9a53ef80 0x558d9a54e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36358/0 (socket says 172.21.15.120:36358) 2024-09-19T09:08:46.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.146+0000 7f22b659b640 1 -- 172.21.15.120:0/3235957393 learned_addr learned my addr 172.21.15.120:0/3235957393 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:46.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.146+0000 7f22b5d9a640 1 --2- 172.21.15.120:0/3235957393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9a53f8c0 0x558d9a54eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:46.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.146+0000 7f22b6d9c640 1 --2- 172.21.15.120:0/3235957393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9a544950 0x558d9a54fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:46.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.146+0000 7f22b659b640 1 -- 172.21.15.120:0/3235957393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9a544950 msgr2=0x558d9a54fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:46.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.146+0000 7f22b659b640 1 --2- 172.21.15.120:0/3235957393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9a544950 0x558d9a54fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.146+0000 7f22b659b640 1 -- 172.21.15.120:0/3235957393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9a53f8c0 msgr2=0x558d9a54eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:46.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.146+0000 7f22b659b640 1 --2- 172.21.15.120:0/3235957393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9a53f8c0 0x558d9a54eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.146+0000 7f22b659b640 1 -- 172.21.15.120:0/3235957393 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d9a5500e0 con 0x558d9a53ef80 2024-09-19T09:08:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.146+0000 7f22b659b640 1 --2- 172.21.15.120:0/3235957393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9a53ef80 0x558d9a54e580 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f22ac0983a0 tx=0x7f22ac009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd5557a5e68c737 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.146+0000 7f2299ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:08:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.146+0000 7f2299ffb640 1 -- 172.21.15.120:0/3235957393 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac021040 con 0x558d9a53ef80 2024-09-19T09:08:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.146+0000 7f2299ffb640 1 -- 172.21.15.120:0/3235957393 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f22ac098040 con 0x558d9a53ef80 2024-09-19T09:08:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.146+0000 7f22bb66e740 1 -- 172.21.15.120:0/3235957393 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22a8005bf0 con 0x558d9a53ef80 2024-09-19T09:08:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.146+0000 7f2299ffb640 1 -- 172.21.15.120:0/3235957393 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f22ac0a4620 con 0x558d9a53ef80 2024-09-19T09:08:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.146+0000 7f22bb66e740 10 client.17508 fetch_fsmap learned FSMap version 30 2024-09-19T09:08:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.146+0000 7f22bb66e740 1 -- 172.21.15.120:0/3235957393 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558d9a5c0240 con 0x558d9a53ef80 2024-09-19T09:08:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.147+0000 7f2299ffb640 1 -- 172.21.15.120:0/3235957393 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22a8005bf0 con 0x558d9a53ef80 2024-09-19T09:08:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.147+0000 7f2299ffb640 1 -- 172.21.15.120:0/3235957393 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f22ac09cce0 con 0x558d9a53ef80 2024-09-19T09:08:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.147+0000 7f22bb66e740 10 client.17508 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:08:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.147+0000 7f22bb66e740 10 client.17508 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:08:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.147+0000 7f22bb66e740 20 client.17508 populate_metadata read hostname 'smithi120' 2024-09-19T09:08:46.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.147+0000 7f22bb66e740 1 --2- 172.21.15.120:0/3235957393 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558d9a5c0b90 0x558d9a5c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:46.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.147+0000 7f22bb66e740 4 client.17508 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:08:46.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.147+0000 7f22bb66e740 1 -- 172.21.15.120:0/3235957393 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558d9a4aba20 con 0x558d9a5c0b90 2024-09-19T09:08:46.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.147+0000 7f22b5d9a640 1 --2- 172.21.15.120:0/3235957393 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558d9a5c0b90 0x558d9a5c2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:46.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.147+0000 7f2299ffb640 10 client.17508 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:46.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.147+0000 7f22b5d9a640 1 --2- 172.21.15.120:0/3235957393 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558d9a5c0b90 0x558d9a5c2f80 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf44448fd8d4e6b8 server_cookie=e568c9046f2b2641 in_seq=0 out_seq=0 2024-09-19T09:08:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.323+0000 7f2299ffb640 1 -- 172.21.15.120:0/3235957393 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558d9a4aba20 con 0x558d9a5c0b90 2024-09-19T09:08:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.323+0000 7f2299ffb640 10 client.17508 handle_command_reply: tid=0 2024-09-19T09:08:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.323+0000 7f22bb66e740 1 client.17508 shutdown 2024-09-19T09:08:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.324+0000 7f22bb66e740 1 -- 172.21.15.120:0/3235957393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9a53ef80 msgr2=0x558d9a54e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.324+0000 7f22bb66e740 1 --2- 172.21.15.120:0/3235957393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9a53ef80 0x558d9a54e580 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f22ac0983a0 tx=0x7f22ac009de0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.324+0000 7f22bb66e740 1 -- 172.21.15.120:0/3235957393 shutdown_connections 2024-09-19T09:08:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.324+0000 7f22bb66e740 1 --2- 172.21.15.120:0/3235957393 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558d9a5c0b90 0x558d9a5c2f80 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.324+0000 7f22bb66e740 1 --2- 172.21.15.120:0/3235957393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9a544950 0x558d9a54fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.324+0000 7f2299ffb640 0 client.17508 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.324+0000 7f22bb66e740 1 --2- 172.21.15.120:0/3235957393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9a53f8c0 0x558d9a54eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.324+0000 7f22bb66e740 1 --2- 172.21.15.120:0/3235957393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9a53ef80 0x558d9a54e580 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.324+0000 7f22bb66e740 1 -- 172.21.15.120:0/3235957393 >> 172.21.15.120:0/3235957393 conn(0x558d9a5361e0 msgr2=0x558d9a536de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.324+0000 7f22bb66e740 1 -- 172.21.15.120:0/3235957393 shutdown_connections 2024-09-19T09:08:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.324+0000 7f22bb66e740 1 -- 172.21.15.120:0/3235957393 wait complete. 2024-09-19T09:08:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.324+0000 7f22bb66e740 20 client.17508 trim_cache size 0 max 16384 2024-09-19T09:08:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:08:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:08:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:08:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:08:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.324+0000 7f22b759d640 1 -- 172.21.15.120:0/819475969 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f229c0a3240 msgr2=0x7f229c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.324+0000 7f22b759d640 1 --2- 172.21.15.120:0/819475969 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f229c0a3240 0x7f229c0a5700 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f22a801f000 tx=0x7f22a8022000 comp rx=0 tx=0).stop 2024-09-19T09:08:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.324+0000 7f22b759d640 1 -- 172.21.15.120:0/819475969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22b0100840 msgr2=0x7f22b0197cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.324+0000 7f22b759d640 1 --2- 172.21.15.120:0/819475969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22b0100840 0x7f22b0197cf0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00d950 tx=0x7f22ac00de10 comp rx=0 tx=0).stop 2024-09-19T09:08:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.325+0000 7f22b759d640 1 -- 172.21.15.120:0/819475969 shutdown_connections 2024-09-19T09:08:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.325+0000 7f22b759d640 1 --2- 172.21.15.120:0/819475969 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f229c0a3240 0x7f229c0a5700 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.325+0000 7f22b759d640 1 --2- 172.21.15.120:0/819475969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22b01023f0 0x7f22b019f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.325+0000 7f22b759d640 1 --2- 172.21.15.120:0/819475969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22b0101a30 0x7f22b0198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.325+0000 7f22b759d640 1 --2- 172.21.15.120:0/819475969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22b0100840 0x7f22b0197cf0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.325+0000 7f22b759d640 1 -- 172.21.15.120:0/819475969 >> 172.21.15.120:0/819475969 conn(0x7f22b00fbfa0 msgr2=0x7f22b0069950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.325+0000 7f22b759d640 1 -- 172.21.15.120:0/819475969 shutdown_connections 2024-09-19T09:08:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:46.325+0000 7f22b759d640 1 -- 172.21.15.120:0/819475969 wait complete. 2024-09-19T09:08:47.051 INFO:tasks.workunit.client.0.smithi120.stdout: 1 474956 8.32 MB/sec execute 162 sec latency 98.931 ms 2024-09-19T09:08:47.126 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:46 smithi120 ceph-mon[31150]: pgmap v519: 97 pgs: 97 active+clean; 36 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.4 MiB/s wr, 281 op/s 2024-09-19T09:08:47.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:46 smithi196 ceph-mon[35150]: pgmap v519: 97 pgs: 97 active+clean; 36 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.4 MiB/s wr, 281 op/s 2024-09-19T09:08:47.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:46 smithi153 ceph-mon[35071]: pgmap v519: 97 pgs: 97 active+clean; 36 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.4 MiB/s wr, 281 op/s 2024-09-19T09:08:48.051 INFO:tasks.workunit.client.0.smithi120.stdout: 1 477932 8.38 MB/sec execute 163 sec latency 12.777 ms 2024-09-19T09:08:49.051 INFO:tasks.workunit.client.0.smithi120.stdout: 1 480555 8.41 MB/sec execute 164 sec latency 68.895 ms 2024-09-19T09:08:49.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:48 smithi196 ceph-mon[35150]: pgmap v520: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.8 MiB/s wr, 352 op/s 2024-09-19T09:08:49.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:48 smithi153 ceph-mon[35071]: pgmap v520: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.8 MiB/s wr, 352 op/s 2024-09-19T09:08:49.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:48 smithi120 ceph-mon[31150]: pgmap v520: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.8 MiB/s wr, 352 op/s 2024-09-19T09:08:50.051 INFO:tasks.workunit.client.0.smithi120.stdout: 1 483810 8.47 MB/sec execute 165 sec latency 19.743 ms 2024-09-19T09:08:51.051 INFO:tasks.workunit.client.0.smithi120.stdout: 1 487187 8.54 MB/sec execute 166 sec latency 14.934 ms 2024-09-19T09:08:51.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:50 smithi196 ceph-mon[35150]: pgmap v521: 97 pgs: 97 active+clean; 27 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 412 op/s 2024-09-19T09:08:51.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:50 smithi153 ceph-mon[35071]: pgmap v521: 97 pgs: 97 active+clean; 27 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 412 op/s 2024-09-19T09:08:51.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:50 smithi120 ceph-mon[31150]: pgmap v521: 97 pgs: 97 active+clean; 27 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 412 op/s 2024-09-19T09:08:52.051 INFO:tasks.workunit.client.0.smithi120.stdout: 1 490547 8.60 MB/sec execute 167 sec latency 11.498 ms 2024-09-19T09:08:53.051 INFO:tasks.workunit.client.0.smithi120.stdout: 1 493835 8.65 MB/sec execute 168 sec latency 12.389 ms 2024-09-19T09:08:53.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:52 smithi196 ceph-mon[35150]: pgmap v522: 97 pgs: 97 active+clean; 27 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 447 op/s 2024-09-19T09:08:53.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:52 smithi153 ceph-mon[35071]: pgmap v522: 97 pgs: 97 active+clean; 27 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 447 op/s 2024-09-19T09:08:53.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:52 smithi120 ceph-mon[31150]: pgmap v522: 97 pgs: 97 active+clean; 27 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 447 op/s 2024-09-19T09:08:54.051 INFO:tasks.workunit.client.0.smithi120.stdout: 1 496281 8.66 MB/sec execute 169 sec latency 71.293 ms 2024-09-19T09:08:55.052 INFO:tasks.workunit.client.0.smithi120.stdout: 1 498930 8.71 MB/sec execute 170 sec latency 17.954 ms 2024-09-19T09:08:55.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:54 smithi196 ceph-mon[35150]: pgmap v523: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 408 op/s 2024-09-19T09:08:55.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:54 smithi153 ceph-mon[35071]: pgmap v523: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 408 op/s 2024-09-19T09:08:55.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:54 smithi120 ceph-mon[31150]: pgmap v523: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 408 op/s 2024-09-19T09:08:56.052 INFO:tasks.workunit.client.0.smithi120.stdout: 1 502077 8.75 MB/sec execute 171 sec latency 10.209 ms 2024-09-19T09:08:57.052 INFO:tasks.workunit.client.0.smithi120.stdout: 1 505089 8.80 MB/sec execute 172 sec latency 12.569 ms 2024-09-19T09:08:57.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:56 smithi196 ceph-mon[35150]: pgmap v524: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 3.2 MiB/s wr, 419 op/s 2024-09-19T09:08:57.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:56 smithi153 ceph-mon[35071]: pgmap v524: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 3.2 MiB/s wr, 419 op/s 2024-09-19T09:08:57.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:56 smithi120 ceph-mon[31150]: pgmap v524: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 3.2 MiB/s wr, 419 op/s 2024-09-19T09:08:58.053 INFO:tasks.workunit.client.0.smithi120.stdout: 1 508434 8.86 MB/sec execute 173 sec latency 11.296 ms 2024-09-19T09:08:59.052 INFO:tasks.workunit.client.0.smithi120.stdout: 1 511561 8.91 MB/sec execute 174 sec latency 30.526 ms 2024-09-19T09:08:59.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:08:58 smithi196 ceph-mon[35150]: pgmap v525: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 3.7 MiB/s wr, 463 op/s 2024-09-19T09:08:59.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:08:58 smithi153 ceph-mon[35071]: pgmap v525: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 3.7 MiB/s wr, 463 op/s 2024-09-19T09:08:59.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:08:58 smithi120 ceph-mon[31150]: pgmap v525: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 3.7 MiB/s wr, 463 op/s 2024-09-19T09:08:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.414+0000 7f04e5131640 1 -- 172.21.15.120:0/1216004997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04e0103710 msgr2=0x7f04e0109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.414+0000 7f04e5131640 1 --2- 172.21.15.120:0/1216004997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04e0103710 0x7f04e0109fa0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009ca0 tx=0x7f04d802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.414+0000 7f04e5131640 1 -- 172.21.15.120:0/1216004997 shutdown_connections 2024-09-19T09:08:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.414+0000 7f04e5131640 1 --2- 172.21.15.120:0/1216004997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04e0103710 0x7f04e0109fa0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.414+0000 7f04e5131640 1 --2- 172.21.15.120:0/1216004997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04e0102d50 0x7f04e01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.414+0000 7f04e5131640 1 --2- 172.21.15.120:0/1216004997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04e0101b60 0x7f04e0101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.414+0000 7f04e5131640 1 -- 172.21.15.120:0/1216004997 >> 172.21.15.120:0/1216004997 conn(0x7f04e00fd2e0 msgr2=0x7f04e00ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.414+0000 7f04e5131640 1 -- 172.21.15.120:0/1216004997 shutdown_connections 2024-09-19T09:08:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.414+0000 7f04e5131640 1 -- 172.21.15.120:0/1216004997 wait complete. 2024-09-19T09:08:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.415+0000 7f04e5131640 1 Processor -- start 2024-09-19T09:08:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.415+0000 7f04e5131640 1 -- start start 2024-09-19T09:08:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.415+0000 7f04e5131640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04e0101b60 0x7f04e0197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.415+0000 7f04e5131640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04e0102d50 0x7f04e0198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.415+0000 7f04e5131640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04e0103710 0x7f04e019f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.415+0000 7f04e5131640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f04e010d7e0 con 0x7f04e0102d50 2024-09-19T09:08:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.415+0000 7f04e5131640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f04e010d660 con 0x7f04e0101b60 2024-09-19T09:08:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.415+0000 7f04e5131640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f04e010d960 con 0x7f04e0103710 2024-09-19T09:08:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.415+0000 7f04de59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04e0102d50 0x7f04e0198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.415+0000 7f04de59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04e0102d50 0x7f04e0198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52064/0 (socket says 172.21.15.120:52064) 2024-09-19T09:08:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.415+0000 7f04de59c640 1 -- 172.21.15.120:0/3851190005 learned_addr learned my addr 172.21.15.120:0/3851190005 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.415+0000 7f04ded9d640 1 --2- 172.21.15.120:0/3851190005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04e0101b60 0x7f04e0197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.415+0000 7f04df59e640 1 --2- 172.21.15.120:0/3851190005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04e0103710 0x7f04e019f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.415+0000 7f04de59c640 1 -- 172.21.15.120:0/3851190005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04e0103710 msgr2=0x7f04e019f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.416+0000 7f04de59c640 1 --2- 172.21.15.120:0/3851190005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04e0103710 0x7f04e019f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.416+0000 7f04de59c640 1 -- 172.21.15.120:0/3851190005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04e0101b60 msgr2=0x7f04e0197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.416+0000 7f04de59c640 1 --2- 172.21.15.120:0/3851190005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04e0101b60 0x7f04e0197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.416+0000 7f04de59c640 1 -- 172.21.15.120:0/3851190005 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e019f9d0 con 0x7f04e0102d50 2024-09-19T09:08:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.416+0000 7f04ded9d640 1 --2- 172.21.15.120:0/3851190005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04e0101b60 0x7f04e0197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.416+0000 7f04de59c640 1 --2- 172.21.15.120:0/3851190005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04e0102d50 0x7f04e0198300 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f04d400e9f0 tx=0x7f04d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.416+0000 7f04c7fff640 1 -- 172.21.15.120:0/3851190005 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d400cd80 con 0x7f04e0102d50 2024-09-19T09:08:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.416+0000 7f04e5131640 1 -- 172.21.15.120:0/3851190005 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e019fcc0 con 0x7f04e0102d50 2024-09-19T09:08:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.416+0000 7f04c7fff640 1 -- 172.21.15.120:0/3851190005 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d4010910 con 0x7f04e0102d50 2024-09-19T09:08:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.416+0000 7f04c7fff640 1 -- 172.21.15.120:0/3851190005 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d40188f0 con 0x7f04e0102d50 2024-09-19T09:08:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.416+0000 7f04e5131640 1 -- 172.21.15.120:0/3851190005 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e01a01c0 con 0x7f04e0102d50 2024-09-19T09:08:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.418+0000 7f04c7fff640 1 -- 172.21.15.120:0/3851190005 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f04d4010ac0 con 0x7f04e0102d50 2024-09-19T09:08:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.418+0000 7f04e7ecb740 1 Processor -- start 2024-09-19T09:08:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.418+0000 7f04e7ecb740 1 -- start start 2024-09-19T09:08:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.419+0000 7f04e7ecb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5e36129a0 0x55d5e3612da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.419+0000 7f04e7ecb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e36132e0 0x55d5e3617da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.419+0000 7f04e7ecb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e3618370 0x55d5e361a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.419+0000 7f04e7ecb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d5e35762f0 con 0x55d5e3618370 2024-09-19T09:08:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.419+0000 7f04e7ecb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d5e351e270 con 0x55d5e36132e0 2024-09-19T09:08:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.419+0000 7f04e7ecb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d5e35722b0 con 0x55d5e36129a0 2024-09-19T09:08:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.419+0000 7f04de59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e3618370 0x55d5e361a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.419+0000 7f04de59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e3618370 0x55d5e361a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52078/0 (socket says 172.21.15.120:52078) 2024-09-19T09:08:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.419+0000 7f04de59c640 1 -- 172.21.15.120:0/273702824 learned_addr learned my addr 172.21.15.120:0/273702824 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.419+0000 7f04de59c640 1 -- 172.21.15.120:0/273702824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5e36129a0 msgr2=0x55d5e3612da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.419+0000 7f04de59c640 1 --2- 172.21.15.120:0/273702824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5e36129a0 0x55d5e3612da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.419+0000 7f04c7fff640 1 --2- 172.21.15.120:0/3851190005 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f04c80a32f0 0x7f04c80a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.419+0000 7f04de59c640 1 -- 172.21.15.120:0/273702824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e36132e0 msgr2=0x55d5e3617da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.419+0000 7f04de59c640 1 --2- 172.21.15.120:0/273702824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e36132e0 0x55d5e3617da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.419+0000 7f04de59c640 1 -- 172.21.15.120:0/273702824 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5e351d340 con 0x55d5e3618370 2024-09-19T09:08:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.419+0000 7f04df59e640 1 --2- 172.21.15.120:0/3851190005 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f04c80a32f0 0x7f04c80a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.419+0000 7f04de59c640 1 --2- 172.21.15.120:0/273702824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e3618370 0x55d5e361a830 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f04d4098db0 tx=0x7f04d4098680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c253b49303f0bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.419+0000 7f04c7fff640 1 -- 172.21.15.120:0/3851190005 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f04d40cfda0 con 0x7f04e0102d50 2024-09-19T09:08:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.419+0000 7f04c57fa640 1 -- 172.21.15.120:0/273702824 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d409c240 con 0x55d5e3618370 2024-09-19T09:08:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.419+0000 7f04c57fa640 1 -- 172.21.15.120:0/273702824 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d4096030 con 0x55d5e3618370 2024-09-19T09:08:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.419+0000 7f04c57fa640 1 -- 172.21.15.120:0/273702824 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d409a5b0 con 0x55d5e3618370 2024-09-19T09:08:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.420+0000 7f04df59e640 1 --2- 172.21.15.120:0/3851190005 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f04c80a32f0 0x7f04c80a57b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f04d8006010 tx=0x7f04d8005df0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.420+0000 7f04e7ecb740 1 -- 172.21.15.120:0/273702824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e3618370 msgr2=0x55d5e361a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.420+0000 7f04e7ecb740 1 --2- 172.21.15.120:0/273702824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e3618370 0x55d5e361a830 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f04d4098db0 tx=0x7f04d4098680 comp rx=0 tx=0).stop 2024-09-19T09:08:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.420+0000 7f04e7ecb740 1 -- 172.21.15.120:0/273702824 shutdown_connections 2024-09-19T09:08:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.420+0000 7f04e7ecb740 1 --2- 172.21.15.120:0/273702824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e3618370 0x55d5e361a830 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.420+0000 7f04e7ecb740 1 --2- 172.21.15.120:0/273702824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e36132e0 0x55d5e3617da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.420+0000 7f04e7ecb740 1 --2- 172.21.15.120:0/273702824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5e36129a0 0x55d5e3612da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.420+0000 7f04e7ecb740 1 -- 172.21.15.120:0/273702824 >> 172.21.15.120:0/273702824 conn(0x55d5e360e010 msgr2=0x55d5e360e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.420+0000 7f04e7ecb740 1 -- 172.21.15.120:0/273702824 shutdown_connections 2024-09-19T09:08:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.420+0000 7f04e7ecb740 1 -- 172.21.15.120:0/273702824 wait complete. 2024-09-19T09:08:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.420+0000 7f04e7ecb740 1 -- start start 2024-09-19T09:08:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.421+0000 7f04e7ecb740 1 Processor -- start 2024-09-19T09:08:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.421+0000 7f04e7ecb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e36129a0 0x55d5e361cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.421+0000 7f04e7ecb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e36132e0 0x55d5e361d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.421+0000 7f04e7ecb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5e3618370 0x55d5e36240b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.421+0000 7f04e7ecb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d5e361b280 con 0x55d5e36132e0 2024-09-19T09:08:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.421+0000 7f04e7ecb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d5e34ff290 con 0x55d5e36129a0 2024-09-19T09:08:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.421+0000 7f04e7ecb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d5e361b400 con 0x55d5e3618370 2024-09-19T09:08:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.421+0000 7f04e7ecb740 10 client.0 _ll_register_callbacks cb 0x55d5e3530140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:08:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.421+0000 7f04de59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e36132e0 0x55d5e361d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.421+0000 7f04de59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e36132e0 0x55d5e361d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52084/0 (socket says 172.21.15.120:52084) 2024-09-19T09:08:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.421+0000 7f04de59c640 1 -- 172.21.15.120:0/3632127766 learned_addr learned my addr 172.21.15.120:0/3632127766 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.421+0000 7f04ded9d640 1 --2- 172.21.15.120:0/3632127766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e36129a0 0x55d5e361cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.421+0000 7f04df59e640 1 --2- 172.21.15.120:0/3632127766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5e3618370 0x55d5e36240b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.421+0000 7f04de59c640 1 -- 172.21.15.120:0/3632127766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5e3618370 msgr2=0x55d5e36240b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.421+0000 7f04de59c640 1 --2- 172.21.15.120:0/3632127766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5e3618370 0x55d5e36240b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.421+0000 7f04de59c640 1 -- 172.21.15.120:0/3632127766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e36129a0 msgr2=0x55d5e361cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.421+0000 7f04de59c640 1 --2- 172.21.15.120:0/3632127766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e36129a0 0x55d5e361cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.421+0000 7f04de59c640 1 -- 172.21.15.120:0/3632127766 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5e3289210 con 0x55d5e36132e0 2024-09-19T09:08:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.421+0000 7f04de59c640 1 --2- 172.21.15.120:0/3632127766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e36132e0 0x55d5e361d070 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f04d40a4b00 tx=0x7f04d40d0090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9156684aba5aa5a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.421+0000 7f04c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:08:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.421+0000 7f04c2ff5640 1 -- 172.21.15.120:0/3632127766 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d409bdd0 con 0x55d5e36132e0 2024-09-19T09:08:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.421+0000 7f04c2ff5640 1 -- 172.21.15.120:0/3632127766 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f04d409c6d0 con 0x55d5e36132e0 2024-09-19T09:08:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.422+0000 7f04e7ecb740 1 -- 172.21.15.120:0/3632127766 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04d4096030 con 0x55d5e36132e0 2024-09-19T09:08:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.422+0000 7f04c2ff5640 1 -- 172.21.15.120:0/3632127766 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f04d409bfb0 con 0x55d5e36132e0 2024-09-19T09:08:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.422+0000 7f04e7ecb740 10 client.17520 fetch_fsmap learned FSMap version 30 2024-09-19T09:08:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.422+0000 7f04e7ecb740 1 -- 172.21.15.120:0/3632127766 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d5e3355620 con 0x55d5e36132e0 2024-09-19T09:08:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.422+0000 7f04c2ff5640 1 -- 172.21.15.120:0/3632127766 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d4096030 con 0x55d5e36132e0 2024-09-19T09:08:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.422+0000 7f04c2ff5640 1 -- 172.21.15.120:0/3632127766 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f04d40a4ba0 con 0x55d5e36132e0 2024-09-19T09:08:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.422+0000 7f04e7ecb740 10 client.17520 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:08:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.422+0000 7f04e7ecb740 10 client.17520 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:08:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.422+0000 7f04e7ecb740 20 client.17520 populate_metadata read hostname 'smithi120' 2024-09-19T09:08:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.422+0000 7f04e7ecb740 1 --2- 172.21.15.120:0/3632127766 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d5e36283b0 0x55d5e362a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.422+0000 7f04e7ecb740 4 client.17520 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:08:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.422+0000 7f04e7ecb740 1 -- 172.21.15.120:0/3632127766 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5e3583a20 con 0x55d5e36283b0 2024-09-19T09:08:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.422+0000 7f04ded9d640 1 --2- 172.21.15.120:0/3632127766 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d5e36283b0 0x55d5e362a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.423+0000 7f04c2ff5640 10 client.17520 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.423+0000 7f04ded9d640 1 --2- 172.21.15.120:0/3632127766 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d5e36283b0 0x55d5e362a7a0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12107862265d4682 server_cookie=62e9dbba432dff in_seq=0 out_seq=0 2024-09-19T09:08:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.423+0000 7f04c2ff5640 1 -- 172.21.15.120:0/3632127766 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d5e3583a20 con 0x55d5e36283b0 2024-09-19T09:08:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.423+0000 7f04c2ff5640 10 client.17520 handle_command_reply: tid=0 2024-09-19T09:08:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.423+0000 7f04e7ecb740 1 client.17520 shutdown 2024-09-19T09:08:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.424+0000 7f04e7ecb740 1 -- 172.21.15.120:0/3632127766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e36132e0 msgr2=0x55d5e361d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.424+0000 7f04e7ecb740 1 --2- 172.21.15.120:0/3632127766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e36132e0 0x55d5e361d070 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f04d40a4b00 tx=0x7f04d40d0090 comp rx=0 tx=0).stop 2024-09-19T09:08:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.424+0000 7f04e7ecb740 1 -- 172.21.15.120:0/3632127766 shutdown_connections 2024-09-19T09:08:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.424+0000 7f04e7ecb740 1 --2- 172.21.15.120:0/3632127766 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d5e36283b0 0x55d5e362a7a0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.424+0000 7f04e7ecb740 1 --2- 172.21.15.120:0/3632127766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5e3618370 0x55d5e36240b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.424+0000 7f04e7ecb740 1 --2- 172.21.15.120:0/3632127766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e36132e0 0x55d5e361d070 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.424+0000 7f04e7ecb740 1 --2- 172.21.15.120:0/3632127766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e36129a0 0x55d5e361cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.424+0000 7f04c2ff5640 0 client.17520 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.424+0000 7f04e7ecb740 1 -- 172.21.15.120:0/3632127766 >> 172.21.15.120:0/3632127766 conn(0x55d5e360e010 msgr2=0x55d5e360f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.424+0000 7f04e7ecb740 1 -- 172.21.15.120:0/3632127766 shutdown_connections 2024-09-19T09:08:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.424+0000 7f04e7ecb740 1 -- 172.21.15.120:0/3632127766 wait complete. 2024-09-19T09:08:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.424+0000 7f04e7ecb740 20 client.17520 trim_cache size 0 max 16384 2024-09-19T09:08:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.438+0000 7f04e7ecb740 1 Processor -- start 2024-09-19T09:08:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.438+0000 7f04e7ecb740 1 -- start start 2024-09-19T09:08:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.438+0000 7f04e7ecb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e36129a0 0x55d5e360eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.438+0000 7f04e7ecb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5e36132e0 0x55d5e3624070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.438+0000 7f04e7ecb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e3618370 0x55d5e360f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.438+0000 7f04e7ecb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01a90f0 con 0x55d5e36129a0 2024-09-19T09:08:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.438+0000 7f04e7ecb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01aac30 con 0x55d5e3618370 2024-09-19T09:08:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.438+0000 7f04e7ecb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01a97c0 con 0x55d5e36132e0 2024-09-19T09:08:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.438+0000 7f04ded9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e36129a0 0x55d5e360eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.438+0000 7f04ded9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e36129a0 0x55d5e360eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52100/0 (socket says 172.21.15.120:52100) 2024-09-19T09:08:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.438+0000 7f04ded9d640 1 -- 172.21.15.120:0/3848872875 learned_addr learned my addr 172.21.15.120:0/3848872875 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.438+0000 7f04de59c640 1 --2- 172.21.15.120:0/3848872875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5e36132e0 0x55d5e3624070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.438+0000 7f04df59e640 1 --2- 172.21.15.120:0/3848872875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e3618370 0x55d5e360f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.438+0000 7f04ded9d640 1 -- 172.21.15.120:0/3848872875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5e36132e0 msgr2=0x55d5e3624070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.438+0000 7f04ded9d640 1 --2- 172.21.15.120:0/3848872875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5e36132e0 0x55d5e3624070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.438+0000 7f04ded9d640 1 -- 172.21.15.120:0/3848872875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e3618370 msgr2=0x55d5e360f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.438+0000 7f04ded9d640 1 --2- 172.21.15.120:0/3848872875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e3618370 0x55d5e360f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.438+0000 7f04ded9d640 1 -- 172.21.15.120:0/3848872875 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5e360eca0 con 0x55d5e36129a0 2024-09-19T09:08:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.439+0000 7f04ded9d640 1 --2- 172.21.15.120:0/3848872875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e36129a0 0x55d5e360eec0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f04e0199290 tx=0x7f04d0013920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa57fe7964e896e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.439+0000 7f04c4ff9640 1 -- 172.21.15.120:0/3848872875 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0004020 con 0x55d5e36129a0 2024-09-19T09:08:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.439+0000 7f04c4ff9640 1 -- 172.21.15.120:0/3848872875 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d00041c0 con 0x55d5e36129a0 2024-09-19T09:08:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.439+0000 7f04c4ff9640 1 -- 172.21.15.120:0/3848872875 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0003900 con 0x55d5e36129a0 2024-09-19T09:08:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.439+0000 7f04e7ecb740 1 -- 172.21.15.120:0/3848872875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e36129a0 msgr2=0x55d5e360eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.439+0000 7f04e7ecb740 1 --2- 172.21.15.120:0/3848872875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e36129a0 0x55d5e360eec0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f04e0199290 tx=0x7f04d0013920 comp rx=0 tx=0).stop 2024-09-19T09:08:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.439+0000 7f04e7ecb740 1 -- 172.21.15.120:0/3848872875 shutdown_connections 2024-09-19T09:08:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.439+0000 7f04e7ecb740 1 --2- 172.21.15.120:0/3848872875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e3618370 0x55d5e360f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.439+0000 7f04e7ecb740 1 --2- 172.21.15.120:0/3848872875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5e36132e0 0x55d5e3624070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.439+0000 7f04e7ecb740 1 --2- 172.21.15.120:0/3848872875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e36129a0 0x55d5e360eec0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.439+0000 7f04e7ecb740 1 -- 172.21.15.120:0/3848872875 >> 172.21.15.120:0/3848872875 conn(0x55d5e360e010 msgr2=0x55d5e361cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.439+0000 7f04e7ecb740 1 -- 172.21.15.120:0/3848872875 shutdown_connections 2024-09-19T09:08:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.439+0000 7f04e7ecb740 1 -- 172.21.15.120:0/3848872875 wait complete. 2024-09-19T09:08:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.440+0000 7f04e7ecb740 1 -- start start 2024-09-19T09:08:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.440+0000 7f04e7ecb740 1 Processor -- start 2024-09-19T09:08:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.440+0000 7f04e7ecb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5e36129a0 0x55d5e36219a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.440+0000 7f04e7ecb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e36132e0 0x55d5e3621ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.440+0000 7f04e7ecb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e3618370 0x55d5e3622e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.440+0000 7f04e7ecb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01aa560 con 0x55d5e36132e0 2024-09-19T09:08:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.440+0000 7f04e7ecb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01a9e90 con 0x55d5e3618370 2024-09-19T09:08:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.440+0000 7f04e7ecb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01a70b0 con 0x55d5e36129a0 2024-09-19T09:08:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.440+0000 7f04e7ecb740 10 client.0 _ll_register_callbacks cb 0x7f04d40d3d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:08:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.440+0000 7f04de59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e36132e0 0x55d5e3621ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.440+0000 7f04ded9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5e36129a0 0x55d5e36219a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.440+0000 7f04de59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e36132e0 0x55d5e3621ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52102/0 (socket says 172.21.15.120:52102) 2024-09-19T09:08:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.440+0000 7f04de59c640 1 -- 172.21.15.120:0/3821828471 learned_addr learned my addr 172.21.15.120:0/3821828471 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:08:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.440+0000 7f04df59e640 1 --2- 172.21.15.120:0/3821828471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e3618370 0x55d5e3622e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.440+0000 7f04de59c640 1 -- 172.21.15.120:0/3821828471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5e36129a0 msgr2=0x55d5e36219a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.440+0000 7f04de59c640 1 --2- 172.21.15.120:0/3821828471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5e36129a0 0x55d5e36219a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.440+0000 7f04de59c640 1 -- 172.21.15.120:0/3821828471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e3618370 msgr2=0x55d5e3622e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:08:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.440+0000 7f04de59c640 1 --2- 172.21.15.120:0/3821828471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e3618370 0x55d5e3622e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.440+0000 7f04de59c640 1 -- 172.21.15.120:0/3821828471 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5e3623460 con 0x55d5e36132e0 2024-09-19T09:08:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.440+0000 7f04ded9d640 1 --2- 172.21.15.120:0/3821828471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5e36129a0 0x55d5e36219a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:08:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.440+0000 7f04de59c640 1 --2- 172.21.15.120:0/3821828471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e36132e0 0x55d5e3621ee0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f04d40a5180 tx=0x7f04d409c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf87cc2d5abf1f4e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:08:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.440+0000 7f04c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:08:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.440+0000 7f04c2ff5640 1 -- 172.21.15.120:0/3821828471 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d409bdd0 con 0x55d5e36132e0 2024-09-19T09:08:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.441+0000 7f04c2ff5640 1 -- 172.21.15.120:0/3821828471 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f04d4010640 con 0x55d5e36132e0 2024-09-19T09:08:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.441+0000 7f04e7ecb740 1 -- 172.21.15.120:0/3821828471 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04d00041c0 con 0x55d5e36132e0 2024-09-19T09:08:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.441+0000 7f04c2ff5640 1 -- 172.21.15.120:0/3821828471 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f04d409c240 con 0x55d5e36132e0 2024-09-19T09:08:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.441+0000 7f04e7ecb740 10 client.17526 fetch_fsmap learned FSMap version 30 2024-09-19T09:08:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.441+0000 7f04e7ecb740 1 -- 172.21.15.120:0/3821828471 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d5e369b990 con 0x55d5e36132e0 2024-09-19T09:08:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.441+0000 7f04c2ff5640 1 -- 172.21.15.120:0/3821828471 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d00041c0 con 0x55d5e36132e0 2024-09-19T09:08:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.441+0000 7f04c2ff5640 1 -- 172.21.15.120:0/3821828471 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f04d40b51c0 con 0x55d5e36132e0 2024-09-19T09:08:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.441+0000 7f04e7ecb740 10 client.17526 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:08:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.441+0000 7f04e7ecb740 10 client.17526 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:08:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.441+0000 7f04e7ecb740 20 client.17526 populate_metadata read hostname 'smithi120' 2024-09-19T09:08:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.441+0000 7f04e7ecb740 1 --2- 172.21.15.120:0/3821828471 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d5e369c1b0 0x55d5e369e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:08:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.441+0000 7f04e7ecb740 4 client.17526 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:08:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.441+0000 7f04e7ecb740 1 -- 172.21.15.120:0/3821828471 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d5e357dd70 con 0x55d5e369c1b0 2024-09-19T09:08:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.441+0000 7f04ded9d640 1 --2- 172.21.15.120:0/3821828471 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d5e369c1b0 0x55d5e369e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:08:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.442+0000 7f04c2ff5640 10 client.17526 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.442+0000 7f04ded9d640 1 --2- 172.21.15.120:0/3821828471 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d5e369c1b0 0x55d5e369e5d0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9db46b20e8fb47d9 server_cookie=3586c8cf71b8f4af in_seq=0 out_seq=0 2024-09-19T09:08:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.643+0000 7f04c2ff5640 1 -- 172.21.15.120:0/3821828471 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d5e357dd70 con 0x55d5e369c1b0 2024-09-19T09:08:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.643+0000 7f04c2ff5640 10 client.17526 handle_command_reply: tid=0 2024-09-19T09:08:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.643+0000 7f04e7ecb740 1 client.17526 shutdown 2024-09-19T09:08:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.643+0000 7f04e7ecb740 1 -- 172.21.15.120:0/3821828471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e36132e0 msgr2=0x55d5e3621ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.643+0000 7f04e7ecb740 1 --2- 172.21.15.120:0/3821828471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e36132e0 0x55d5e3621ee0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f04d40a5180 tx=0x7f04d409c4e0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.644+0000 7f04e7ecb740 1 -- 172.21.15.120:0/3821828471 shutdown_connections 2024-09-19T09:08:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.644+0000 7f04e7ecb740 1 --2- 172.21.15.120:0/3821828471 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d5e369c1b0 0x55d5e369e5d0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.644+0000 7f04c2ff5640 0 client.17526 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:08:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.644+0000 7f04e7ecb740 1 --2- 172.21.15.120:0/3821828471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e3618370 0x55d5e3622e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.644+0000 7f04e7ecb740 1 --2- 172.21.15.120:0/3821828471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e36132e0 0x55d5e3621ee0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.644+0000 7f04e7ecb740 1 --2- 172.21.15.120:0/3821828471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5e36129a0 0x55d5e36219a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.644+0000 7f04e7ecb740 1 -- 172.21.15.120:0/3821828471 >> 172.21.15.120:0/3821828471 conn(0x55d5e360e010 msgr2=0x55d5e360c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.644+0000 7f04e7ecb740 1 -- 172.21.15.120:0/3821828471 shutdown_connections 2024-09-19T09:08:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.644+0000 7f04e7ecb740 1 -- 172.21.15.120:0/3821828471 wait complete. 2024-09-19T09:08:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.644+0000 7f04e7ecb740 20 client.17526 trim_cache size 0 max 16384 2024-09-19T09:08:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:08:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:08:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:08:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:08:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.645+0000 7f04e5131640 1 -- 172.21.15.120:0/3851190005 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f04c80a32f0 msgr2=0x7f04c80a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.645+0000 7f04e5131640 1 --2- 172.21.15.120:0/3851190005 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f04c80a32f0 0x7f04c80a57b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f04d8006010 tx=0x7f04d8005df0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.645+0000 7f04e5131640 1 -- 172.21.15.120:0/3851190005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04e0102d50 msgr2=0x7f04e0198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:08:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.645+0000 7f04e5131640 1 --2- 172.21.15.120:0/3851190005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04e0102d50 0x7f04e0198300 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f04d400e9f0 tx=0x7f04d400eeb0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.645+0000 7f04e5131640 1 -- 172.21.15.120:0/3851190005 shutdown_connections 2024-09-19T09:08:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.645+0000 7f04e5131640 1 --2- 172.21.15.120:0/3851190005 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f04c80a32f0 0x7f04c80a57b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.645+0000 7f04e5131640 1 --2- 172.21.15.120:0/3851190005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04e0103710 0x7f04e019f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.645+0000 7f04e5131640 1 --2- 172.21.15.120:0/3851190005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04e0102d50 0x7f04e0198300 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.645+0000 7f04e5131640 1 --2- 172.21.15.120:0/3851190005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04e0101b60 0x7f04e0197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:08:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.645+0000 7f04e5131640 1 -- 172.21.15.120:0/3851190005 >> 172.21.15.120:0/3851190005 conn(0x7f04e00fd2e0 msgr2=0x7f04e0106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:08:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.645+0000 7f04e5131640 1 -- 172.21.15.120:0/3851190005 shutdown_connections 2024-09-19T09:08:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:08:59.645+0000 7f04e5131640 1 -- 172.21.15.120:0/3851190005 wait complete. 2024-09-19T09:09:00.052 INFO:tasks.workunit.client.0.smithi120.stdout: 1 513423 8.89 MB/sec execute 175 sec latency 80.368 ms 2024-09-19T09:09:01.052 INFO:tasks.workunit.client.0.smithi120.stdout: 1 516238 8.93 MB/sec execute 176 sec latency 11.389 ms 2024-09-19T09:09:01.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:00 smithi196 ceph-mon[35150]: pgmap v526: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 3.8 MiB/s wr, 388 op/s 2024-09-19T09:09:01.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:00 smithi153 ceph-mon[35071]: pgmap v526: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 3.8 MiB/s wr, 388 op/s 2024-09-19T09:09:01.328 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:00 smithi120 ceph-mon[31150]: pgmap v526: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 3.8 MiB/s wr, 388 op/s 2024-09-19T09:09:02.052 INFO:tasks.workunit.client.0.smithi120.stdout: 1 519368 8.98 MB/sec execute 177 sec latency 11.555 ms 2024-09-19T09:09:02.832 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:09:03.052 INFO:tasks.workunit.client.0.smithi120.stdout: 1 522585 9.04 MB/sec execute 178 sec latency 15.243 ms 2024-09-19T09:09:03.096 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:09:03.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:02 smithi196 ceph-mon[35150]: pgmap v527: 97 pgs: 97 active+clean; 24 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 4.3 MiB/s wr, 395 op/s 2024-09-19T09:09:03.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:02 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:09:03.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:02 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:09:03.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:02 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:09:03.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:02 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:09:03.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:02 smithi153 ceph-mon[35071]: pgmap v527: 97 pgs: 97 active+clean; 24 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 4.3 MiB/s wr, 395 op/s 2024-09-19T09:09:03.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:02 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:09:03.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:02 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:09:03.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:02 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:09:03.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:02 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:09:03.218 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:02 smithi120 ceph-mon[31150]: pgmap v527: 97 pgs: 97 active+clean; 24 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 4.3 MiB/s wr, 395 op/s 2024-09-19T09:09:03.218 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:02 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:09:03.218 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:02 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:09:03.218 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:02 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:09:03.218 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:02 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:09:03.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.621+0000 7fadc0c0d640 1 -- 172.21.15.120:0/630262884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadbc10c2f0 msgr2=0x7fadbc10e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:03.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.621+0000 7fadc0c0d640 1 --2- 172.21.15.120:0/630262884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadbc10c2f0 0x7fadbc10e790 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fadb4009ca0 tx=0x7fadb402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.621+0000 7fadc0c0d640 1 -- 172.21.15.120:0/630262884 shutdown_connections 2024-09-19T09:09:03.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.621+0000 7fadc0c0d640 1 --2- 172.21.15.120:0/630262884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadbc10c2f0 0x7fadbc10e790 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.621+0000 7fadc0c0d640 1 --2- 172.21.15.120:0/630262884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadbc1037a0 0x7fadbc10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.621+0000 7fadc0c0d640 1 --2- 172.21.15.120:0/630262884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadbc102df0 0x7fadbc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.621+0000 7fadc0c0d640 1 -- 172.21.15.120:0/630262884 >> 172.21.15.120:0/630262884 conn(0x7fadbc0fc7d0 msgr2=0x7fadbc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:03.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.621+0000 7fadc0c0d640 1 -- 172.21.15.120:0/630262884 shutdown_connections 2024-09-19T09:09:03.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.621+0000 7fadc0c0d640 1 -- 172.21.15.120:0/630262884 wait complete. 2024-09-19T09:09:03.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.622+0000 7fadc0c0d640 1 Processor -- start 2024-09-19T09:09:03.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.622+0000 7fadc0c0d640 1 -- start start 2024-09-19T09:09:03.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.622+0000 7fadc0c0d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadbc102df0 0x7fadbc100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:03.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.622+0000 7fadc0c0d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadbc1037a0 0x7fadbc100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:03.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.622+0000 7fadc0c0d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadbc10c2f0 0x7fadbc1014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:03.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.622+0000 7fadc0c0d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc112140 con 0x7fadbc102df0 2024-09-19T09:09:03.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.622+0000 7fadc0c0d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc111fc0 con 0x7fadbc10c2f0 2024-09-19T09:09:03.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.622+0000 7fadc0c0d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc1122c0 con 0x7fadbc1037a0 2024-09-19T09:09:03.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.622+0000 7fadba59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadbc102df0 0x7fadbc100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:03.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.622+0000 7fadba59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadbc102df0 0x7fadbc100a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58374/0 (socket says 172.21.15.120:58374) 2024-09-19T09:09:03.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.622+0000 7fadb3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadbc1037a0 0x7fadbc100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:03.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.622+0000 7fadba59c640 1 -- 172.21.15.120:0/41530690 learned_addr learned my addr 172.21.15.120:0/41530690 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:03.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.622+0000 7fadbad9d640 1 --2- 172.21.15.120:0/41530690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadbc10c2f0 0x7fadbc1014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:03.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.622+0000 7fadba59c640 1 -- 172.21.15.120:0/41530690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadbc1037a0 msgr2=0x7fadbc100f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:03.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.622+0000 7fadba59c640 1 --2- 172.21.15.120:0/41530690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadbc1037a0 0x7fadbc100f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.622+0000 7fadba59c640 1 -- 172.21.15.120:0/41530690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadbc10c2f0 msgr2=0x7fadbc1014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:03.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.622+0000 7fadba59c640 1 --2- 172.21.15.120:0/41530690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadbc10c2f0 0x7fadbc1014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.622+0000 7fadba59c640 1 -- 172.21.15.120:0/41530690 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadbc1a6a70 con 0x7fadbc102df0 2024-09-19T09:09:03.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.623+0000 7fadba59c640 1 --2- 172.21.15.120:0/41530690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadbc102df0 0x7fadbc100a50 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fadac00e9f0 tx=0x7fadac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:03.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.623+0000 7fadb37fe640 1 -- 172.21.15.120:0/41530690 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac00cd80 con 0x7fadbc102df0 2024-09-19T09:09:03.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.623+0000 7fadc0c0d640 1 -- 172.21.15.120:0/41530690 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadbc1a6d60 con 0x7fadbc102df0 2024-09-19T09:09:03.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.623+0000 7fadc0c0d640 1 -- 172.21.15.120:0/41530690 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadbc1a7260 con 0x7fadbc102df0 2024-09-19T09:09:03.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.623+0000 7fadb37fe640 1 -- 172.21.15.120:0/41530690 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadac010910 con 0x7fadbc102df0 2024-09-19T09:09:03.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.623+0000 7fadb37fe640 1 -- 172.21.15.120:0/41530690 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac018980 con 0x7fadbc102df0 2024-09-19T09:09:03.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.625+0000 7fadb37fe640 1 -- 172.21.15.120:0/41530690 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fadac010ac0 con 0x7fadbc102df0 2024-09-19T09:09:03.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.626+0000 7fadc2fe6740 1 Processor -- start 2024-09-19T09:09:03.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.626+0000 7fadc2fe6740 1 -- start start 2024-09-19T09:09:03.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.627+0000 7fadb37fe640 1 --2- 172.21.15.120:0/41530690 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fada80a3240 0x7fada80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:03.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.627+0000 7fadc2fe6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d7b22014a0 0x55d7b2201880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:03.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.627+0000 7fadc2fe6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7b2201dc0 0x55d7b220c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:03.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.627+0000 7fadbad9d640 1 --2- 172.21.15.120:0/41530690 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fada80a3240 0x7fada80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:03.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.627+0000 7fadc2fe6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7b220c680 0x55d7b220ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:03.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.627+0000 7fadc2fe6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d7b21672e0 con 0x55d7b22014a0 2024-09-19T09:09:03.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.627+0000 7fadc2fe6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d7b210ece0 con 0x55d7b2201dc0 2024-09-19T09:09:03.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.627+0000 7fadc2fe6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d7b21632a0 con 0x55d7b220c680 2024-09-19T09:09:03.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.627+0000 7fadba59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d7b22014a0 0x55d7b2201880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:03.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.627+0000 7fadba59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d7b22014a0 0x55d7b2201880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58390/0 (socket says 172.21.15.120:58390) 2024-09-19T09:09:03.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.627+0000 7fadba59c640 1 -- 172.21.15.120:0/3849987001 learned_addr learned my addr 172.21.15.120:0/3849987001 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:03.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.627+0000 7fadb37fe640 1 -- 172.21.15.120:0/41530690 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fadac0cfb90 con 0x7fadbc102df0 2024-09-19T09:09:03.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.627+0000 7fadbad9d640 1 --2- 172.21.15.120:0/3849987001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7b220c680 0x55d7b220ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:03.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.627+0000 7fadb3fff640 1 --2- 172.21.15.120:0/3849987001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7b2201dc0 0x55d7b220c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:03.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.627+0000 7fadba59c640 1 -- 172.21.15.120:0/3849987001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7b220c680 msgr2=0x55d7b220ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:03.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.627+0000 7fadba59c640 1 --2- 172.21.15.120:0/3849987001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7b220c680 0x55d7b220ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.627+0000 7fadba59c640 1 -- 172.21.15.120:0/3849987001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7b2201dc0 msgr2=0x55d7b220c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:03.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.627+0000 7fadba59c640 1 --2- 172.21.15.120:0/3849987001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7b2201dc0 0x55d7b220c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.627+0000 7fadba59c640 1 -- 172.21.15.120:0/3849987001 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7b20f3440 con 0x55d7b22014a0 2024-09-19T09:09:03.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.627+0000 7fadbad9d640 1 --2- 172.21.15.120:0/41530690 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fada80a3240 0x7fada80a5700 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fadb4006010 tx=0x7fadb4031c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:03.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.627+0000 7fadba59c640 1 --2- 172.21.15.120:0/3849987001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d7b22014a0 0x55d7b2201880 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fadac09c1f0 tx=0x7fadac098630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5dd3e9d4902805c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:03.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.628+0000 7fadb0ff9640 1 -- 172.21.15.120:0/3849987001 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac0a0560 con 0x55d7b22014a0 2024-09-19T09:09:03.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.628+0000 7fadb0ff9640 1 -- 172.21.15.120:0/3849987001 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadac0a4e20 con 0x55d7b22014a0 2024-09-19T09:09:03.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.628+0000 7fadb0ff9640 1 -- 172.21.15.120:0/3849987001 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac0c15a0 con 0x55d7b22014a0 2024-09-19T09:09:03.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.628+0000 7fadc2fe6740 1 -- 172.21.15.120:0/3849987001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d7b22014a0 msgr2=0x55d7b2201880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:03.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.628+0000 7fadc2fe6740 1 --2- 172.21.15.120:0/3849987001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d7b22014a0 0x55d7b2201880 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fadac09c1f0 tx=0x7fadac098630 comp rx=0 tx=0).stop 2024-09-19T09:09:03.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.629+0000 7fadc2fe6740 1 -- 172.21.15.120:0/3849987001 shutdown_connections 2024-09-19T09:09:03.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.629+0000 7fadc2fe6740 1 --2- 172.21.15.120:0/3849987001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7b220c680 0x55d7b220ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.629+0000 7fadc2fe6740 1 --2- 172.21.15.120:0/3849987001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7b2201dc0 0x55d7b220c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.629+0000 7fadc2fe6740 1 --2- 172.21.15.120:0/3849987001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d7b22014a0 0x55d7b2201880 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.629+0000 7fadc2fe6740 1 -- 172.21.15.120:0/3849987001 >> 172.21.15.120:0/3849987001 conn(0x55d7b21fda30 msgr2=0x55d7b21fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:03.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.629+0000 7fadc2fe6740 1 -- 172.21.15.120:0/3849987001 shutdown_connections 2024-09-19T09:09:03.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.629+0000 7fadc2fe6740 1 -- 172.21.15.120:0/3849987001 wait complete. 2024-09-19T09:09:03.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.629+0000 7fadc2fe6740 1 -- start start 2024-09-19T09:09:03.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.629+0000 7fadc2fe6740 1 Processor -- start 2024-09-19T09:09:03.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.629+0000 7fadc2fe6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7b22014a0 0x55d7b220bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:03.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.629+0000 7fadc2fe6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7b2201dc0 0x55d7b2209200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:03.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.629+0000 7fadc2fe6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d7b220c680 0x55d7b2209740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:03.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.629+0000 7fadc2fe6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d7b220f250 con 0x55d7b220c680 2024-09-19T09:09:03.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.629+0000 7fadc2fe6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d7b2122050 con 0x55d7b22014a0 2024-09-19T09:09:03.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.629+0000 7fadc2fe6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d7b220f3d0 con 0x55d7b2201dc0 2024-09-19T09:09:03.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.629+0000 7fadc2fe6740 10 client.0 _ll_register_callbacks cb 0x55d7b2162160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:03.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.629+0000 7fadbad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d7b220c680 0x55d7b2209740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:03.906 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:09:03.906 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:09:03.907 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:09:03.907 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:09:03.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.629+0000 7fadbad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d7b220c680 0x55d7b2209740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58404/0 (socket says 172.21.15.120:58404) 2024-09-19T09:09:03.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.629+0000 7fadbad9d640 1 -- 172.21.15.120:0/2581974424 learned_addr learned my addr 172.21.15.120:0/2581974424 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:03.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.629+0000 7fadba59c640 1 --2- 172.21.15.120:0/2581974424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7b22014a0 0x55d7b220bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:03.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.629+0000 7fadb3fff640 1 --2- 172.21.15.120:0/2581974424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7b2201dc0 0x55d7b2209200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:03.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.630+0000 7fadbad9d640 1 -- 172.21.15.120:0/2581974424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7b2201dc0 msgr2=0x55d7b2209200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:03.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.630+0000 7fadbad9d640 1 --2- 172.21.15.120:0/2581974424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7b2201dc0 0x55d7b2209200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.630+0000 7fadbad9d640 1 -- 172.21.15.120:0/2581974424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7b22014a0 msgr2=0x55d7b220bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:03.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.630+0000 7fadbad9d640 1 --2- 172.21.15.120:0/2581974424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7b22014a0 0x55d7b220bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.630+0000 7fadbad9d640 1 -- 172.21.15.120:0/2581974424 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7b1ed83c0 con 0x55d7b220c680 2024-09-19T09:09:03.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.630+0000 7fadbad9d640 1 --2- 172.21.15.120:0/2581974424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d7b220c680 0x55d7b2209740 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fadb4035000 tx=0x7fadb402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9a969c49c250d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:03.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.630+0000 7fada1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:09:03.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.630+0000 7fada1ffb640 1 -- 172.21.15.120:0/2581974424 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb4045040 con 0x55d7b220c680 2024-09-19T09:09:03.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.630+0000 7fada1ffb640 1 -- 172.21.15.120:0/2581974424 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fadb4004240 con 0x55d7b220c680 2024-09-19T09:09:03.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.630+0000 7fadc2fe6740 1 -- 172.21.15.120:0/2581974424 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadac0a4e20 con 0x55d7b220c680 2024-09-19T09:09:03.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.630+0000 7fada1ffb640 1 -- 172.21.15.120:0/2581974424 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fadb4003e50 con 0x55d7b220c680 2024-09-19T09:09:03.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.630+0000 7fadc2fe6740 10 client.17538 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:03.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.630+0000 7fadc2fe6740 1 -- 172.21.15.120:0/2581974424 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d7b221ac30 con 0x55d7b220c680 2024-09-19T09:09:03.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.630+0000 7fada1ffb640 1 -- 172.21.15.120:0/2581974424 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadac0a4e20 con 0x55d7b220c680 2024-09-19T09:09:03.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.631+0000 7fada1ffb640 1 -- 172.21.15.120:0/2581974424 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fadb40305c0 con 0x55d7b220c680 2024-09-19T09:09:03.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.631+0000 7fadc2fe6740 10 client.17538 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:03.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.631+0000 7fadc2fe6740 10 client.17538 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:09:03.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.631+0000 7fadc2fe6740 20 client.17538 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:03.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.631+0000 7fadc2fe6740 1 --2- 172.21.15.120:0/2581974424 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d7b221c1b0 0x55d7b221e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:03.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.631+0000 7fadc2fe6740 4 client.17538 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:09:03.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.631+0000 7fadc2fe6740 1 -- 172.21.15.120:0/2581974424 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7b2174850 con 0x55d7b221c1b0 2024-09-19T09:09:03.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.631+0000 7fadba59c640 1 --2- 172.21.15.120:0/2581974424 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d7b221c1b0 0x55d7b221e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:03.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.631+0000 7fadba59c640 1 --2- 172.21.15.120:0/2581974424 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d7b221c1b0 0x55d7b221e610 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dd74da40cf96861 server_cookie=93a9b44f8655307d in_seq=0 out_seq=0 2024-09-19T09:09:03.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.631+0000 7fada1ffb640 10 client.17538 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:03.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.632+0000 7fada1ffb640 1 -- 172.21.15.120:0/2581974424 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d7b2174850 con 0x55d7b221c1b0 2024-09-19T09:09:03.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.632+0000 7fada1ffb640 10 client.17538 handle_command_reply: tid=0 2024-09-19T09:09:03.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.632+0000 7fadc2fe6740 1 client.17538 shutdown 2024-09-19T09:09:03.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.632+0000 7fadc2fe6740 1 -- 172.21.15.120:0/2581974424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d7b220c680 msgr2=0x55d7b2209740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:03.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.632+0000 7fadc2fe6740 1 --2- 172.21.15.120:0/2581974424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d7b220c680 0x55d7b2209740 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fadb4035000 tx=0x7fadb402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.632+0000 7fadc2fe6740 1 -- 172.21.15.120:0/2581974424 shutdown_connections 2024-09-19T09:09:03.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.632+0000 7fadc2fe6740 1 --2- 172.21.15.120:0/2581974424 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d7b221c1b0 0x55d7b221e610 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.632+0000 7fadc2fe6740 1 --2- 172.21.15.120:0/2581974424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d7b220c680 0x55d7b2209740 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.632+0000 7fada1ffb640 0 client.17538 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:03.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.632+0000 7fadc2fe6740 1 --2- 172.21.15.120:0/2581974424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7b2201dc0 0x55d7b2209200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.632+0000 7fadc2fe6740 1 --2- 172.21.15.120:0/2581974424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7b22014a0 0x55d7b220bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.632+0000 7fadc2fe6740 1 -- 172.21.15.120:0/2581974424 >> 172.21.15.120:0/2581974424 conn(0x55d7b21fda30 msgr2=0x55d7b2203760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:03.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.632+0000 7fadc2fe6740 1 -- 172.21.15.120:0/2581974424 shutdown_connections 2024-09-19T09:09:03.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.632+0000 7fadc2fe6740 1 -- 172.21.15.120:0/2581974424 wait complete. 2024-09-19T09:09:03.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.633+0000 7fadc2fe6740 20 client.17538 trim_cache size 0 max 16384 2024-09-19T09:09:03.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.650+0000 7fadc2fe6740 1 Processor -- start 2024-09-19T09:09:03.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.650+0000 7fadc2fe6740 1 -- start start 2024-09-19T09:09:03.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.650+0000 7fadc2fe6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7b22014a0 0x55d7b2218510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:03.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.650+0000 7fadc2fe6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7b2201dc0 0x55d7b2223f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:03.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.650+0000 7fadc2fe6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d7b220c680 0x55d7b2224460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:03.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.650+0000 7fadc2fe6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c00e450 con 0x55d7b220c680 2024-09-19T09:09:03.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.650+0000 7fadc2fe6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c00ff90 con 0x55d7b22014a0 2024-09-19T09:09:03.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.650+0000 7fadc2fe6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c00eb20 con 0x55d7b2201dc0 2024-09-19T09:09:03.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.650+0000 7fadbad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d7b220c680 0x55d7b2224460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:03.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.650+0000 7fadba59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7b22014a0 0x55d7b2218510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:03.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.650+0000 7fadb3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7b2201dc0 0x55d7b2223f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:03.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.650+0000 7fadbad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d7b220c680 0x55d7b2224460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58412/0 (socket says 172.21.15.120:58412) 2024-09-19T09:09:03.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.650+0000 7fadbad9d640 1 -- 172.21.15.120:0/502933775 learned_addr learned my addr 172.21.15.120:0/502933775 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:03.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.651+0000 7fadba59c640 1 -- 172.21.15.120:0/502933775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7b2201dc0 msgr2=0x55d7b2223f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:03.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.651+0000 7fadba59c640 1 --2- 172.21.15.120:0/502933775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7b2201dc0 0x55d7b2223f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.651+0000 7fadba59c640 1 -- 172.21.15.120:0/502933775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d7b220c680 msgr2=0x55d7b2224460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:03.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.651+0000 7fadba59c640 1 --2- 172.21.15.120:0/502933775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d7b220c680 0x55d7b2224460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.651+0000 7fadba59c640 1 -- 172.21.15.120:0/502933775 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7b220edb0 con 0x55d7b22014a0 2024-09-19T09:09:03.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.651+0000 7fadb3fff640 1 --2- 172.21.15.120:0/502933775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7b2201dc0 0x55d7b2223f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:03.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.651+0000 7fadba59c640 1 --2- 172.21.15.120:0/502933775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7b22014a0 0x55d7b2218510 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fadac00e9c0 tx=0x7fadac010db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19ccb6a2b6a2903c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:03.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.651+0000 7fada3fff640 1 -- 172.21.15.120:0/502933775 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac09a560 con 0x55d7b22014a0 2024-09-19T09:09:03.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.651+0000 7fada3fff640 1 -- 172.21.15.120:0/502933775 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadac0b0070 con 0x55d7b22014a0 2024-09-19T09:09:03.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.651+0000 7fada3fff640 1 -- 172.21.15.120:0/502933775 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac0c13a0 con 0x55d7b22014a0 2024-09-19T09:09:03.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.652+0000 7fadc2fe6740 1 -- 172.21.15.120:0/502933775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7b22014a0 msgr2=0x55d7b2218510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:03.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.652+0000 7fadc2fe6740 1 --2- 172.21.15.120:0/502933775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7b22014a0 0x55d7b2218510 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fadac00e9c0 tx=0x7fadac010db0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.652+0000 7fadc2fe6740 1 -- 172.21.15.120:0/502933775 shutdown_connections 2024-09-19T09:09:03.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.652+0000 7fadc2fe6740 1 --2- 172.21.15.120:0/502933775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d7b220c680 0x55d7b2224460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.652+0000 7fadc2fe6740 1 --2- 172.21.15.120:0/502933775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7b2201dc0 0x55d7b2223f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.652+0000 7fadc2fe6740 1 --2- 172.21.15.120:0/502933775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7b22014a0 0x55d7b2218510 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.652+0000 7fadc2fe6740 1 -- 172.21.15.120:0/502933775 >> 172.21.15.120:0/502933775 conn(0x55d7b21fda30 msgr2=0x55d7b22082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:03.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.652+0000 7fadc2fe6740 1 -- 172.21.15.120:0/502933775 shutdown_connections 2024-09-19T09:09:03.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.652+0000 7fadc2fe6740 1 -- 172.21.15.120:0/502933775 wait complete. 2024-09-19T09:09:03.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.652+0000 7fadc2fe6740 1 -- start start 2024-09-19T09:09:03.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.652+0000 7fadc2fe6740 1 Processor -- start 2024-09-19T09:09:03.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.653+0000 7fadc2fe6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7b22014a0 0x55d7b2214ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:03.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.653+0000 7fadc2fe6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7b2201dc0 0x55d7b2211ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:03.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.653+0000 7fadc2fe6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d7b220c680 0x55d7b2212530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:03.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.653+0000 7fadc2fe6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c00f8c0 con 0x55d7b220c680 2024-09-19T09:09:03.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.653+0000 7fadc2fe6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c00f1f0 con 0x55d7b2201dc0 2024-09-19T09:09:03.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.653+0000 7fadc2fe6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c00c410 con 0x55d7b22014a0 2024-09-19T09:09:03.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.653+0000 7fadc2fe6740 10 client.0 _ll_register_callbacks cb 0x7fadac005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:03.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.653+0000 7fadbad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d7b220c680 0x55d7b2212530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:03.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.653+0000 7fadb3fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7b2201dc0 0x55d7b2211ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:03.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.653+0000 7fadb3fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7b2201dc0 0x55d7b2211ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59594/0 (socket says 172.21.15.120:59594) 2024-09-19T09:09:03.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.653+0000 7fadb3fff640 1 -- 172.21.15.120:0/1267935991 learned_addr learned my addr 172.21.15.120:0/1267935991 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:03.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.653+0000 7fadbad9d640 1 --2- 172.21.15.120:0/1267935991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d7b220c680 0x55d7b2212530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58416/0 (socket says 172.21.15.120:58416) 2024-09-19T09:09:03.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.653+0000 7fadb3fff640 1 -- 172.21.15.120:0/1267935991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7b22014a0 msgr2=0x55d7b2214ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:09:03.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.653+0000 7fadb3fff640 1 --2- 172.21.15.120:0/1267935991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7b22014a0 0x55d7b2214ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.653+0000 7fadb3fff640 1 -- 172.21.15.120:0/1267935991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d7b220c680 msgr2=0x55d7b2212530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:03.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.653+0000 7fadb3fff640 1 --2- 172.21.15.120:0/1267935991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d7b220c680 0x55d7b2212530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.653+0000 7fadb3fff640 1 -- 172.21.15.120:0/1267935991 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7b2212c20 con 0x55d7b2201dc0 2024-09-19T09:09:03.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.653+0000 7fadbad9d640 1 --2- 172.21.15.120:0/1267935991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d7b220c680 0x55d7b2212530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:03.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.653+0000 7fadb3fff640 1 --2- 172.21.15.120:0/1267935991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7b2201dc0 0x55d7b2211ff0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fadbc06a2b0 tx=0x7fada4012c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7342f48c94805496 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:03.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.653+0000 7fada1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:09:03.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.654+0000 7fada1ffb640 1 -- 172.21.15.120:0/1267935991 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4023060 con 0x55d7b2201dc0 2024-09-19T09:09:03.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.654+0000 7fada1ffb640 1 -- 172.21.15.120:0/1267935991 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fada4004230 con 0x55d7b2201dc0 2024-09-19T09:09:03.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.654+0000 7fadc2fe6740 1 -- 172.21.15.120:0/1267935991 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadac0b0070 con 0x55d7b2201dc0 2024-09-19T09:09:03.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.654+0000 7fada1ffb640 1 -- 172.21.15.120:0/1267935991 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fada4004430 con 0x55d7b2201dc0 2024-09-19T09:09:03.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.654+0000 7fadc2fe6740 10 client.26810 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:03.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.654+0000 7fadc2fe6740 1 -- 172.21.15.120:0/1267935991 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d7b2286e60 con 0x55d7b2201dc0 2024-09-19T09:09:03.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.654+0000 7fada1ffb640 1 -- 172.21.15.120:0/1267935991 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadac0b0070 con 0x55d7b2201dc0 2024-09-19T09:09:03.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.654+0000 7fada1ffb640 1 -- 172.21.15.120:0/1267935991 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fada4028070 con 0x55d7b2201dc0 2024-09-19T09:09:03.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.654+0000 7fadc2fe6740 10 client.26810 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:03.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.654+0000 7fadc2fe6740 10 client.26810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:09:03.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.654+0000 7fadc2fe6740 20 client.26810 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:03.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.654+0000 7fadc2fe6740 1 --2- 172.21.15.120:0/1267935991 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d7b2287640 0x55d7b2289a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:03.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.654+0000 7fadc2fe6740 4 client.26810 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:09:03.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.654+0000 7fadc2fe6740 1 -- 172.21.15.120:0/1267935991 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d7b216ed60 con 0x55d7b2287640 2024-09-19T09:09:03.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.654+0000 7fadba59c640 1 --2- 172.21.15.120:0/1267935991 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d7b2287640 0x55d7b2289a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:03.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.655+0000 7fada1ffb640 10 client.26810 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:03.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.655+0000 7fadba59c640 1 --2- 172.21.15.120:0/1267935991 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d7b2287640 0x55d7b2289a60 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b9bb9132ab9deb4 server_cookie=77c78de2bd4befba in_seq=0 out_seq=0 2024-09-19T09:09:03.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.655+0000 7fada1ffb640 1 -- 172.21.15.120:0/1267935991 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d7b216ed60 con 0x55d7b2287640 2024-09-19T09:09:03.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.655+0000 7fada1ffb640 10 client.26810 handle_command_reply: tid=0 2024-09-19T09:09:03.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.655+0000 7fadc2fe6740 1 client.26810 shutdown 2024-09-19T09:09:03.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.656+0000 7fadc2fe6740 1 -- 172.21.15.120:0/1267935991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7b2201dc0 msgr2=0x55d7b2211ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:03.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.656+0000 7fadc2fe6740 1 --2- 172.21.15.120:0/1267935991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7b2201dc0 0x55d7b2211ff0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fadbc06a2b0 tx=0x7fada4012c80 comp rx=0 tx=0).stop 2024-09-19T09:09:03.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.656+0000 7fadc2fe6740 1 -- 172.21.15.120:0/1267935991 shutdown_connections 2024-09-19T09:09:03.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.656+0000 7fadc2fe6740 1 --2- 172.21.15.120:0/1267935991 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d7b2287640 0x55d7b2289a60 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.656+0000 7fadc2fe6740 1 --2- 172.21.15.120:0/1267935991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d7b220c680 0x55d7b2212530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.656+0000 7fada1ffb640 0 client.26810 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:03.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.656+0000 7fadc2fe6740 1 --2- 172.21.15.120:0/1267935991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d7b2201dc0 0x55d7b2211ff0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.656+0000 7fadc2fe6740 1 --2- 172.21.15.120:0/1267935991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7b22014a0 0x55d7b2214ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.656+0000 7fadc2fe6740 1 -- 172.21.15.120:0/1267935991 >> 172.21.15.120:0/1267935991 conn(0x55d7b21fda30 msgr2=0x55d7b220aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:03.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.656+0000 7fadc2fe6740 1 -- 172.21.15.120:0/1267935991 shutdown_connections 2024-09-19T09:09:03.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.656+0000 7fadc2fe6740 1 -- 172.21.15.120:0/1267935991 wait complete. 2024-09-19T09:09:03.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.656+0000 7fadc2fe6740 20 client.26810 trim_cache size 0 max 16384 2024-09-19T09:09:03.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.656+0000 7fadc0c0d640 1 -- 172.21.15.120:0/41530690 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fada80a3240 msgr2=0x7fada80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:03.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.656+0000 7fadc0c0d640 1 --2- 172.21.15.120:0/41530690 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fada80a3240 0x7fada80a5700 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fadb4006010 tx=0x7fadb4031c90 comp rx=0 tx=0).stop 2024-09-19T09:09:03.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.656+0000 7fadc0c0d640 1 -- 172.21.15.120:0/41530690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadbc102df0 msgr2=0x7fadbc100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:03.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.656+0000 7fadc0c0d640 1 --2- 172.21.15.120:0/41530690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadbc102df0 0x7fadbc100a50 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fadac00e9f0 tx=0x7fadac00eeb0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.657+0000 7fadc0c0d640 1 -- 172.21.15.120:0/41530690 shutdown_connections 2024-09-19T09:09:03.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.657+0000 7fadc0c0d640 1 --2- 172.21.15.120:0/41530690 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fada80a3240 0x7fada80a5700 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.657+0000 7fadc0c0d640 1 --2- 172.21.15.120:0/41530690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadbc10c2f0 0x7fadbc1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.657+0000 7fadc0c0d640 1 --2- 172.21.15.120:0/41530690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadbc1037a0 0x7fadbc100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.657+0000 7fadc0c0d640 1 --2- 172.21.15.120:0/41530690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadbc102df0 0x7fadbc100a50 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:03.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.657+0000 7fadc0c0d640 1 -- 172.21.15.120:0/41530690 >> 172.21.15.120:0/41530690 conn(0x7fadbc0fc7d0 msgr2=0x7fadbc0fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:03.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.657+0000 7fadc0c0d640 1 -- 172.21.15.120:0/41530690 shutdown_connections 2024-09-19T09:09:03.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:03.657+0000 7fadc0c0d640 1 -- 172.21.15.120:0/41530690 wait complete. 2024-09-19T09:09:04.052 INFO:tasks.workunit.client.0.smithi120.stdout: 1 525590 9.07 MB/sec execute 179 sec latency 50.276 ms 2024-09-19T09:09:05.017 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:09:05.017 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:09:05.052 INFO:tasks.workunit.client.0.smithi120.stdout: 1 528024 9.08 MB/sec execute 180 sec latency 81.535 ms 2024-09-19T09:09:05.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:04 smithi196 ceph-mon[35150]: pgmap v528: 97 pgs: 97 active+clean; 24 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 453 op/s 2024-09-19T09:09:05.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:04 smithi153 ceph-mon[35071]: pgmap v528: 97 pgs: 97 active+clean; 24 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 453 op/s 2024-09-19T09:09:05.277 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:09:05.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:04 smithi120 ceph-mon[31150]: pgmap v528: 97 pgs: 97 active+clean; 24 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 453 op/s 2024-09-19T09:09:05.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.801+0000 7fdab19c1640 1 -- 172.21.15.120:0/2505030413 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaac102570 msgr2=0x7fdaac1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:05.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.801+0000 7fdab19c1640 1 --2- 172.21.15.120:0/2505030413 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaac102570 0x7fdaac1029b0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fda94009900 tx=0x7fda9402a480 comp rx=0 tx=0).stop 2024-09-19T09:09:05.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.801+0000 7fdab19c1640 1 -- 172.21.15.120:0/2505030413 shutdown_connections 2024-09-19T09:09:05.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.801+0000 7fdab19c1640 1 --2- 172.21.15.120:0/2505030413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaac102ef0 0x7fdaac10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:05.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.801+0000 7fdab19c1640 1 --2- 172.21.15.120:0/2505030413 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaac102570 0x7fdaac1029b0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:05.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.801+0000 7fdab19c1640 1 --2- 172.21.15.120:0/2505030413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdaac100e20 0x7fdaac101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:05.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.801+0000 7fdab19c1640 1 -- 172.21.15.120:0/2505030413 >> 172.21.15.120:0/2505030413 conn(0x7fdaac0fc7d0 msgr2=0x7fdaac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:05.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.801+0000 7fdab19c1640 1 -- 172.21.15.120:0/2505030413 shutdown_connections 2024-09-19T09:09:05.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.801+0000 7fdab19c1640 1 -- 172.21.15.120:0/2505030413 wait complete. 2024-09-19T09:09:05.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.802+0000 7fdab19c1640 1 Processor -- start 2024-09-19T09:09:05.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.802+0000 7fdab19c1640 1 -- start start 2024-09-19T09:09:05.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.803+0000 7fdab19c1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaac100e20 0x7fdaac1962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:05.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.803+0000 7fdab19c1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaac102570 0x7fdaac1967e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:05.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.803+0000 7fdab19c1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdaac102ef0 0x7fdaac19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:05.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.803+0000 7fdab19c1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac10fe80 con 0x7fdaac100e20 2024-09-19T09:09:05.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.803+0000 7fdab19c1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac10fd00 con 0x7fdaac102ef0 2024-09-19T09:09:05.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.803+0000 7fdab19c1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac110000 con 0x7fdaac102570 2024-09-19T09:09:05.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.803+0000 7fdaaaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaac100e20 0x7fdaac1962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:05.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.803+0000 7fdaaaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaac100e20 0x7fdaac1962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58436/0 (socket says 172.21.15.120:58436) 2024-09-19T09:09:05.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.803+0000 7fdaaaffd640 1 -- 172.21.15.120:0/3794966394 learned_addr learned my addr 172.21.15.120:0/3794966394 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:05.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.803+0000 7fdaab7fe640 1 --2- 172.21.15.120:0/3794966394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdaac102ef0 0x7fdaac19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:05.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.803+0000 7fdaaa7fc640 1 --2- 172.21.15.120:0/3794966394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaac102570 0x7fdaac1967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:05.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.803+0000 7fdaaaffd640 1 -- 172.21.15.120:0/3794966394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaac102570 msgr2=0x7fdaac1967e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:05.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.803+0000 7fdaaaffd640 1 --2- 172.21.15.120:0/3794966394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaac102570 0x7fdaac1967e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:05.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.803+0000 7fdaaaffd640 1 -- 172.21.15.120:0/3794966394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdaac102ef0 msgr2=0x7fdaac19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:05.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.803+0000 7fdaaaffd640 1 --2- 172.21.15.120:0/3794966394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdaac102ef0 0x7fdaac19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:05.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.803+0000 7fdaaaffd640 1 -- 172.21.15.120:0/3794966394 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaac19b250 con 0x7fdaac100e20 2024-09-19T09:09:05.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.803+0000 7fdaaaffd640 1 --2- 172.21.15.120:0/3794966394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaac100e20 0x7fdaac1962a0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400ebc0 tx=0x7fdaa400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:05.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.803+0000 7fdab09bf640 1 -- 172.21.15.120:0/3794966394 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4010630 con 0x7fdaac100e20 2024-09-19T09:09:05.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.803+0000 7fdab19c1640 1 -- 172.21.15.120:0/3794966394 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaac19b540 con 0x7fdaac100e20 2024-09-19T09:09:05.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.803+0000 7fdab09bf640 1 -- 172.21.15.120:0/3794966394 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaa4010cb0 con 0x7fdaac100e20 2024-09-19T09:09:05.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.803+0000 7fdab09bf640 1 -- 172.21.15.120:0/3794966394 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4018bd0 con 0x7fdaac100e20 2024-09-19T09:09:05.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.803+0000 7fdab19c1640 1 -- 172.21.15.120:0/3794966394 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaac1a2e20 con 0x7fdaac100e20 2024-09-19T09:09:05.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.804+0000 7fdab19c1640 1 -- 172.21.15.120:0/3794966394 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaac101200 con 0x7fdaac100e20 2024-09-19T09:09:05.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.806+0000 7fdab09bf640 1 -- 172.21.15.120:0/3794966394 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fdaa40107d0 con 0x7fdaac100e20 2024-09-19T09:09:05.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.807+0000 7fdab09bf640 1 --2- 172.21.15.120:0/3794966394 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fda980a3240 0x7fda980a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:05.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.807+0000 7fdaaa7fc640 1 --2- 172.21.15.120:0/3794966394 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fda980a3240 0x7fda980a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:05.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.807+0000 7fdab09bf640 1 -- 172.21.15.120:0/3794966394 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fdaa40d01b0 con 0x7fdaac100e20 2024-09-19T09:09:05.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.807+0000 7fdaaa7fc640 1 --2- 172.21.15.120:0/3794966394 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fda980a3240 0x7fda980a5700 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdaac0ffd90 tx=0x7fda94005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:05.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:05.810+0000 7fdab09bf640 1 -- 172.21.15.120:0/3794966394 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdaa409ced0 con 0x7fdaac100e20 2024-09-19T09:09:06.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:06.041+0000 7fdab19c1640 1 -- 172.21.15.120:0/3794966394 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdaac1a2ff0 con 0x7fdaac100e20 2024-09-19T09:09:06.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:06.042+0000 7fdab09bf640 1 -- 172.21.15.120:0/3794966394 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11985 (secure 0 0 0) 0x7fdaa4010ac0 con 0x7fdaac100e20 2024-09-19T09:09:06.042 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:09:06.042 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":30,"btime":"2024-09-19T09:07:14:812065+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:07:14.812046+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26351":{"gid":26351,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/364941779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":364941779},{"type":"v1","addr":"172.21.15.153:6835","nonce":364941779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:09:06.043 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 30 2024-09-19T09:09:06.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:06.044+0000 7fdab19c1640 1 -- 172.21.15.120:0/3794966394 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fda980a3240 msgr2=0x7fda980a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:06.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:06.044+0000 7fdab19c1640 1 --2- 172.21.15.120:0/3794966394 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fda980a3240 0x7fda980a5700 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdaac0ffd90 tx=0x7fda94005a20 comp rx=0 tx=0).stop 2024-09-19T09:09:06.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:06.044+0000 7fdab19c1640 1 -- 172.21.15.120:0/3794966394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaac100e20 msgr2=0x7fdaac1962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:06.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:06.044+0000 7fdab19c1640 1 --2- 172.21.15.120:0/3794966394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaac100e20 0x7fdaac1962a0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400ebc0 tx=0x7fdaa400c6a0 comp rx=0 tx=0).stop 2024-09-19T09:09:06.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:06.044+0000 7fdab19c1640 1 -- 172.21.15.120:0/3794966394 shutdown_connections 2024-09-19T09:09:06.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:06.044+0000 7fdab19c1640 1 --2- 172.21.15.120:0/3794966394 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fda980a3240 0x7fda980a5700 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:06.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:06.044+0000 7fdab19c1640 1 --2- 172.21.15.120:0/3794966394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdaac102ef0 0x7fdaac19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:06.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:06.044+0000 7fdab19c1640 1 --2- 172.21.15.120:0/3794966394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaac102570 0x7fdaac1967e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:06.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:06.044+0000 7fdab19c1640 1 --2- 172.21.15.120:0/3794966394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaac100e20 0x7fdaac1962a0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:06.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:06.044+0000 7fdab19c1640 1 -- 172.21.15.120:0/3794966394 >> 172.21.15.120:0/3794966394 conn(0x7fdaac0fc7d0 msgr2=0x7fdaac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:06.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:06.044+0000 7fdab19c1640 1 -- 172.21.15.120:0/3794966394 shutdown_connections 2024-09-19T09:09:06.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:06.044+0000 7fdab19c1640 1 -- 172.21.15.120:0/3794966394 wait complete. 2024-09-19T09:09:06.052 INFO:tasks.workunit.client.0.smithi120.stdout: 1 530732 9.13 MB/sec execute 181 sec latency 43.994 ms 2024-09-19T09:09:07.037 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:09:07.052 INFO:tasks.workunit.client.0.smithi120.stdout: 1 533861 9.16 MB/sec execute 182 sec latency 12.147 ms 2024-09-19T09:09:07.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:06 smithi196 ceph-mon[35150]: pgmap v529: 97 pgs: 97 active+clean; 25 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 3.8 MiB/s wr, 408 op/s 2024-09-19T09:09:07.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:06 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3794966394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:09:07.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:06 smithi153 ceph-mon[35071]: pgmap v529: 97 pgs: 97 active+clean; 25 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 3.8 MiB/s wr, 408 op/s 2024-09-19T09:09:07.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:06 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3794966394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:09:07.310 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:09:07.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:06 smithi120 ceph-mon[31150]: pgmap v529: 97 pgs: 97 active+clean; 25 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 3.8 MiB/s wr, 408 op/s 2024-09-19T09:09:07.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:06 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3794966394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:09:07.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.830+0000 7f8ada6fe640 1 -- 172.21.15.120:0/1286181965 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ad4102d40 msgr2=0x7f8ad4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:07.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.830+0000 7f8ada6fe640 1 --2- 172.21.15.120:0/1286181965 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ad4102d40 0x7f8ad4103120 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f8ac400dc90 tx=0x7f8ac402e400 comp rx=0 tx=0).stop 2024-09-19T09:09:07.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.830+0000 7f8ada6fe640 1 -- 172.21.15.120:0/1286181965 shutdown_connections 2024-09-19T09:09:07.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.830+0000 7f8ada6fe640 1 --2- 172.21.15.120:0/1286181965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ad410c170 0x7f8ad410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.830+0000 7f8ada6fe640 1 --2- 172.21.15.120:0/1286181965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ad4103660 0x7f8ad410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.830+0000 7f8ada6fe640 1 --2- 172.21.15.120:0/1286181965 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ad4102d40 0x7f8ad4103120 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.830+0000 7f8ada6fe640 1 -- 172.21.15.120:0/1286181965 >> 172.21.15.120:0/1286181965 conn(0x7f8ad40fc7a0 msgr2=0x7f8ad40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.830+0000 7f8ada6fe640 1 -- 172.21.15.120:0/1286181965 shutdown_connections 2024-09-19T09:09:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.830+0000 7f8ada6fe640 1 -- 172.21.15.120:0/1286181965 wait complete. 2024-09-19T09:09:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.831+0000 7f8ada6fe640 1 Processor -- start 2024-09-19T09:09:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.831+0000 7f8ada6fe640 1 -- start start 2024-09-19T09:09:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.832+0000 7f8ada6fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ad4102d40 0x7f8ad41a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.832+0000 7f8ada6fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ad4103660 0x7f8ad41a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.832+0000 7f8ada6fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ad410c170 0x7f8ad419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.832+0000 7f8ada6fe640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad410fd70 con 0x7f8ad410c170 2024-09-19T09:09:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.832+0000 7f8ada6fe640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad410fbf0 con 0x7f8ad4102d40 2024-09-19T09:09:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.832+0000 7f8ada6fe640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad410fef0 con 0x7f8ad4103660 2024-09-19T09:09:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.832+0000 7f8ad3fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ad4102d40 0x7f8ad41a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.832+0000 7f8ad3fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ad4102d40 0x7f8ad41a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59628/0 (socket says 172.21.15.120:59628) 2024-09-19T09:09:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.832+0000 7f8ad3fff640 1 -- 172.21.15.120:0/667645726 learned_addr learned my addr 172.21.15.120:0/667645726 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.832+0000 7f8ad8c9b640 1 --2- 172.21.15.120:0/667645726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ad410c170 0x7f8ad419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.832+0000 7f8ad37fe640 1 --2- 172.21.15.120:0/667645726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ad4103660 0x7f8ad41a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.833+0000 7f8ad8c9b640 1 -- 172.21.15.120:0/667645726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ad4103660 msgr2=0x7f8ad41a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.833+0000 7f8ad8c9b640 1 --2- 172.21.15.120:0/667645726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ad4103660 0x7f8ad41a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.833+0000 7f8ad8c9b640 1 -- 172.21.15.120:0/667645726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ad4102d40 msgr2=0x7f8ad41a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.833+0000 7f8ad8c9b640 1 --2- 172.21.15.120:0/667645726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ad4102d40 0x7f8ad41a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.833+0000 7f8ad8c9b640 1 -- 172.21.15.120:0/667645726 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad419efc0 con 0x7f8ad410c170 2024-09-19T09:09:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.833+0000 7f8ad8c9b640 1 --2- 172.21.15.120:0/667645726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ad410c170 0x7f8ad419e890 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009fd0 tx=0x7f8acc00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.833+0000 7f8ad17fa640 1 -- 172.21.15.120:0/667645726 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc00ce20 con 0x7f8ad410c170 2024-09-19T09:09:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.833+0000 7f8ad17fa640 1 -- 172.21.15.120:0/667645726 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8acc005020 con 0x7f8ad410c170 2024-09-19T09:09:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.833+0000 7f8ad17fa640 1 -- 172.21.15.120:0/667645726 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc010720 con 0x7f8ad410c170 2024-09-19T09:09:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.833+0000 7f8ada6fe640 1 -- 172.21.15.120:0/667645726 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad419f2b0 con 0x7f8ad410c170 2024-09-19T09:09:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.833+0000 7f8ada6fe640 1 -- 172.21.15.120:0/667645726 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad406bc10 con 0x7f8ad410c170 2024-09-19T09:09:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.836+0000 7f8adcad7740 1 Processor -- start 2024-09-19T09:09:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.837+0000 7f8ad17fa640 1 -- 172.21.15.120:0/667645726 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f8acc005540 con 0x7f8ad410c170 2024-09-19T09:09:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.837+0000 7f8adcad7740 1 -- start start 2024-09-19T09:09:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.837+0000 7f8adcad7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af6417db40 0x55af6417c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.837+0000 7f8adcad7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af6417c5d0 0x55af6417ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.837+0000 7f8adcad7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af64187370 0x55af64189760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.837+0000 7f8adcad7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55af640e22f0 con 0x55af6417c5d0 2024-09-19T09:09:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.837+0000 7f8adcad7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55af64089cf0 con 0x55af6417db40 2024-09-19T09:09:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.837+0000 7f8adcad7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55af640de2b0 con 0x55af64187370 2024-09-19T09:09:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.837+0000 7f8ad3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af6417c5d0 0x55af6417ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.837+0000 7f8ad37fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af64187370 0x55af64189760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.837+0000 7f8ad37fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af64187370 0x55af64189760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:44820/0 (socket says 172.21.15.120:44820) 2024-09-19T09:09:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.837+0000 7f8ad37fe640 1 -- 172.21.15.120:0/3557067719 learned_addr learned my addr 172.21.15.120:0/3557067719 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.837+0000 7f8ad3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af6417c5d0 0x55af6417ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58454/0 (socket says 172.21.15.120:58454) 2024-09-19T09:09:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.838+0000 7f8ad17fa640 1 --2- 172.21.15.120:0/667645726 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8abc0a32c0 0x7f8abc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.838+0000 7f8ad37fe640 1 --2- 172.21.15.120:0/3557067719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af6417db40 0x55af6417c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.838+0000 7f8ad17fa640 1 -- 172.21.15.120:0/667645726 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f8acc0cfe90 con 0x7f8ad410c170 2024-09-19T09:09:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.838+0000 7f8ad37fe640 1 -- 172.21.15.120:0/3557067719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af6417db40 msgr2=0x55af6417c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.838+0000 7f8ad37fe640 1 --2- 172.21.15.120:0/3557067719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af6417db40 0x55af6417c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.838+0000 7f8ad37fe640 1 -- 172.21.15.120:0/3557067719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af6417c5d0 msgr2=0x55af6417ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.838+0000 7f8ad37fe640 1 --2- 172.21.15.120:0/3557067719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af6417c5d0 0x55af6417ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.838+0000 7f8ad37fe640 1 -- 172.21.15.120:0/3557067719 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af6406e3a0 con 0x55af64187370 2024-09-19T09:09:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.838+0000 7f8ad8c9b640 1 --2- 172.21.15.120:0/667645726 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8abc0a32c0 0x7f8abc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.839+0000 7f8ad37fe640 1 --2- 172.21.15.120:0/3557067719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af64187370 0x55af64189760 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8009f00 tx=0x7f8ac8009570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb4f290146e0be52 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.839+0000 7f8aba7fc640 1 -- 172.21.15.120:0/3557067719 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8017470 con 0x55af64187370 2024-09-19T09:09:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.839+0000 7f8aba7fc640 1 -- 172.21.15.120:0/3557067719 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ac801c070 con 0x55af64187370 2024-09-19T09:09:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.839+0000 7f8aba7fc640 1 -- 172.21.15.120:0/3557067719 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8015880 con 0x55af64187370 2024-09-19T09:09:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.840+0000 7f8adcad7740 1 -- 172.21.15.120:0/3557067719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af64187370 msgr2=0x55af64189760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.840+0000 7f8ad8c9b640 1 --2- 172.21.15.120:0/667645726 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8abc0a32c0 0x7f8abc0a5780 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8acc0058f0 tx=0x7f8acc005940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.840+0000 7f8adcad7740 1 --2- 172.21.15.120:0/3557067719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af64187370 0x55af64189760 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8009f00 tx=0x7f8ac8009570 comp rx=0 tx=0).stop 2024-09-19T09:09:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.840+0000 7f8adcad7740 1 -- 172.21.15.120:0/3557067719 shutdown_connections 2024-09-19T09:09:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.840+0000 7f8adcad7740 1 --2- 172.21.15.120:0/3557067719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af64187370 0x55af64189760 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.840+0000 7f8adcad7740 1 --2- 172.21.15.120:0/3557067719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af6417c5d0 0x55af6417ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.840+0000 7f8adcad7740 1 --2- 172.21.15.120:0/3557067719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af6417db40 0x55af6417c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.840+0000 7f8adcad7740 1 -- 172.21.15.120:0/3557067719 >> 172.21.15.120:0/3557067719 conn(0x55af64145310 msgr2=0x55af64178c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.840+0000 7f8adcad7740 1 -- 172.21.15.120:0/3557067719 shutdown_connections 2024-09-19T09:09:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.840+0000 7f8adcad7740 1 -- 172.21.15.120:0/3557067719 wait complete. 2024-09-19T09:09:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.840+0000 7f8adcad7740 1 -- start start 2024-09-19T09:09:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.841+0000 7f8adcad7740 1 Processor -- start 2024-09-19T09:09:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.841+0000 7f8adcad7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af6417c5d0 0x55af64186dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.841+0000 7f8adcad7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af6417db40 0x55af64184080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:07.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.841+0000 7f8adcad7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af64187370 0x55af64184810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:07.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.841+0000 7f8adcad7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55af6418a070 con 0x55af64187370 2024-09-19T09:09:07.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.841+0000 7f8adcad7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55af6409d060 con 0x55af6417c5d0 2024-09-19T09:09:07.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.841+0000 7f8adcad7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55af6418a1f0 con 0x55af6417db40 2024-09-19T09:09:07.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.841+0000 7f8adcad7740 10 client.0 _ll_register_callbacks cb 0x55af63df08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:07.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.841+0000 7f8ad3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af6417db40 0x55af64184080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:07.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.841+0000 7f8ad3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af6417db40 0x55af64184080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:44834/0 (socket says 172.21.15.120:44834) 2024-09-19T09:09:07.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.841+0000 7f8ad3fff640 1 -- 172.21.15.120:0/1230268412 learned_addr learned my addr 172.21.15.120:0/1230268412 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:07.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.841+0000 7f8ad37fe640 1 --2- 172.21.15.120:0/1230268412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af64187370 0x55af64184810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:07.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.841+0000 7f8ad37fe640 1 --2- 172.21.15.120:0/1230268412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af6417c5d0 0x55af64186dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:07.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.842+0000 7f8ad37fe640 1 -- 172.21.15.120:0/1230268412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af6417db40 msgr2=0x55af64184080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:07.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.842+0000 7f8ad37fe640 1 --2- 172.21.15.120:0/1230268412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af6417db40 0x55af64184080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.842+0000 7f8ad37fe640 1 -- 172.21.15.120:0/1230268412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af6417c5d0 msgr2=0x55af64186dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:07.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.842+0000 7f8ad37fe640 1 --2- 172.21.15.120:0/1230268412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af6417c5d0 0x55af64186dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.842+0000 7f8ad37fe640 1 -- 172.21.15.120:0/1230268412 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af63e533c0 con 0x55af64187370 2024-09-19T09:09:07.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.842+0000 7f8ad37fe640 1 --2- 172.21.15.120:0/1230268412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af64187370 0x55af64184810 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8001eb0 tx=0x7f8ac8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0d497af271f07a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:07.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.842+0000 7f8ab37fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:09:07.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.842+0000 7f8ab37fe640 1 -- 172.21.15.120:0/1230268412 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac80059c0 con 0x55af64187370 2024-09-19T09:09:07.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.842+0000 7f8adcad7740 1 -- 172.21.15.120:0/1230268412 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ac801c070 con 0x55af64187370 2024-09-19T09:09:07.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.842+0000 7f8ab37fe640 1 -- 172.21.15.120:0/1230268412 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f8ac8015600 con 0x55af64187370 2024-09-19T09:09:07.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.843+0000 7f8ab37fe640 1 -- 172.21.15.120:0/1230268412 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8ac8015800 con 0x55af64187370 2024-09-19T09:09:07.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.843+0000 7f8adcad7740 10 client.17562 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:07.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.843+0000 7f8adcad7740 1 -- 172.21.15.120:0/1230268412 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55af64195b90 con 0x55af64187370 2024-09-19T09:09:07.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.843+0000 7f8ab37fe640 1 -- 172.21.15.120:0/1230268412 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ac801c070 con 0x55af64187370 2024-09-19T09:09:07.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.843+0000 7f8ab37fe640 1 -- 172.21.15.120:0/1230268412 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8ac8021930 con 0x55af64187370 2024-09-19T09:09:07.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.843+0000 7f8adcad7740 10 client.17562 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:07.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.843+0000 7f8adcad7740 10 client.17562 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:09:07.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.843+0000 7f8adcad7740 20 client.17562 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:07.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.843+0000 7f8adcad7740 1 --2- 172.21.15.120:0/1230268412 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55af64196ee0 0x55af641992d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:07.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.843+0000 7f8adcad7740 4 client.17562 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:09:07.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.843+0000 7f8adcad7740 1 -- 172.21.15.120:0/1230268412 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af640ef860 con 0x55af64196ee0 2024-09-19T09:09:07.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.844+0000 7f8ad3fff640 1 --2- 172.21.15.120:0/1230268412 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55af64196ee0 0x55af641992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:07.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.844+0000 7f8ad3fff640 1 --2- 172.21.15.120:0/1230268412 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55af64196ee0 0x55af641992d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e5a669c08a3bd0d8 server_cookie=d33727524714403 in_seq=0 out_seq=0 2024-09-19T09:09:07.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.844+0000 7f8ab37fe640 10 client.17562 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:09:07.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.845+0000 7f8ab37fe640 1 -- 172.21.15.120:0/1230268412 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af640ef860 con 0x55af64196ee0 2024-09-19T09:09:07.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.845+0000 7f8ab37fe640 10 client.17562 handle_command_reply: tid=0 2024-09-19T09:09:07.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.845+0000 7f8adcad7740 1 client.17562 shutdown 2024-09-19T09:09:07.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.846+0000 7f8adcad7740 1 -- 172.21.15.120:0/1230268412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af64187370 msgr2=0x55af64184810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:07.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.846+0000 7f8adcad7740 1 --2- 172.21.15.120:0/1230268412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af64187370 0x55af64184810 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8001eb0 tx=0x7f8ac8008040 comp rx=0 tx=0).stop 2024-09-19T09:09:07.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.846+0000 7f8adcad7740 1 -- 172.21.15.120:0/1230268412 shutdown_connections 2024-09-19T09:09:07.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.846+0000 7f8adcad7740 1 --2- 172.21.15.120:0/1230268412 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55af64196ee0 0x55af641992d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.846+0000 7f8adcad7740 1 --2- 172.21.15.120:0/1230268412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af64187370 0x55af64184810 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.846+0000 7f8adcad7740 1 --2- 172.21.15.120:0/1230268412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af6417db40 0x55af64184080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.846+0000 7f8adcad7740 1 --2- 172.21.15.120:0/1230268412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af6417c5d0 0x55af64186dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.846+0000 7f8adcad7740 1 -- 172.21.15.120:0/1230268412 >> 172.21.15.120:0/1230268412 conn(0x55af64145310 msgr2=0x55af6417e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:07.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.846+0000 7f8ab37fe640 0 client.17562 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:09:07.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.846+0000 7f8adcad7740 1 -- 172.21.15.120:0/1230268412 shutdown_connections 2024-09-19T09:09:07.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.846+0000 7f8adcad7740 1 -- 172.21.15.120:0/1230268412 wait complete. 2024-09-19T09:09:07.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.846+0000 7f8adcad7740 20 client.17562 trim_cache size 0 max 16384 2024-09-19T09:09:07.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.866+0000 7f8adcad7740 1 Processor -- start 2024-09-19T09:09:07.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.866+0000 7f8adcad7740 1 -- start start 2024-09-19T09:09:07.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.866+0000 7f8adcad7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af6417c5d0 0x55af64183060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:07.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.866+0000 7f8adcad7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af6417db40 0x55af6419e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:07.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.866+0000 7f8adcad7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af64187370 0x55af6419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:07.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.866+0000 7f8adcad7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad4074250 con 0x55af64187370 2024-09-19T09:09:07.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.866+0000 7f8adcad7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad41bdf20 con 0x55af6417c5d0 2024-09-19T09:09:07.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.866+0000 7f8adcad7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad41bcab0 con 0x55af6417db40 2024-09-19T09:09:07.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.867+0000 7f8ad37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af64187370 0x55af6419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:07.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.867+0000 7f8ad37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af64187370 0x55af6419ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58472/0 (socket says 172.21.15.120:58472) 2024-09-19T09:09:07.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.867+0000 7f8ad37fe640 1 -- 172.21.15.120:0/123706388 learned_addr learned my addr 172.21.15.120:0/123706388 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:07.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.867+0000 7f8ad37fe640 1 --2- 172.21.15.120:0/123706388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af6417c5d0 0x55af64183060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:07.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.867+0000 7f8ad3fff640 1 --2- 172.21.15.120:0/123706388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af6417db40 0x55af6419e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:07.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.867+0000 7f8ad37fe640 1 -- 172.21.15.120:0/123706388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af6417db40 msgr2=0x55af6419e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:07.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.867+0000 7f8ad37fe640 1 --2- 172.21.15.120:0/123706388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af6417db40 0x55af6419e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.867+0000 7f8ad37fe640 1 -- 172.21.15.120:0/123706388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af6417c5d0 msgr2=0x55af64183060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:07.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.867+0000 7f8ad37fe640 1 --2- 172.21.15.120:0/123706388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af6417c5d0 0x55af64183060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.867+0000 7f8ad37fe640 1 -- 172.21.15.120:0/123706388 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af64179d60 con 0x55af64187370 2024-09-19T09:09:07.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.867+0000 7f8ad37fe640 1 --2- 172.21.15.120:0/123706388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af64187370 0x55af6419ee40 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8019040 tx=0x7f8ac8010040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d24ea9410b7950a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:07.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.870+0000 7f8ab9ffb640 1 -- 172.21.15.120:0/123706388 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8017d90 con 0x55af64187370 2024-09-19T09:09:07.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.870+0000 7f8ab9ffb640 1 -- 172.21.15.120:0/123706388 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ac801c030 con 0x55af64187370 2024-09-19T09:09:07.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.870+0000 7f8ab9ffb640 1 -- 172.21.15.120:0/123706388 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8022a60 con 0x55af64187370 2024-09-19T09:09:07.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.870+0000 7f8adcad7740 1 -- 172.21.15.120:0/123706388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af64187370 msgr2=0x55af6419ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:07.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.870+0000 7f8adcad7740 1 --2- 172.21.15.120:0/123706388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af64187370 0x55af6419ee40 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8019040 tx=0x7f8ac8010040 comp rx=0 tx=0).stop 2024-09-19T09:09:07.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.870+0000 7f8adcad7740 1 -- 172.21.15.120:0/123706388 shutdown_connections 2024-09-19T09:09:07.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.870+0000 7f8adcad7740 1 --2- 172.21.15.120:0/123706388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af64187370 0x55af6419ee40 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.870+0000 7f8adcad7740 1 --2- 172.21.15.120:0/123706388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af6417db40 0x55af6419e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.870+0000 7f8adcad7740 1 --2- 172.21.15.120:0/123706388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af6417c5d0 0x55af64183060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.870+0000 7f8adcad7740 1 -- 172.21.15.120:0/123706388 >> 172.21.15.120:0/123706388 conn(0x55af64145310 msgr2=0x55af64182950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:07.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.870+0000 7f8adcad7740 1 -- 172.21.15.120:0/123706388 shutdown_connections 2024-09-19T09:09:07.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.870+0000 7f8adcad7740 1 -- 172.21.15.120:0/123706388 wait complete. 2024-09-19T09:09:07.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.871+0000 7f8adcad7740 1 -- start start 2024-09-19T09:09:07.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.871+0000 7f8adcad7740 1 Processor -- start 2024-09-19T09:09:07.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.871+0000 7f8adcad7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af6417c5d0 0x55af6418f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:07.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.871+0000 7f8adcad7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af6417db40 0x55af6418f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:07.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.871+0000 7f8adcad7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af64187370 0x55af641806b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:07.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.871+0000 7f8adcad7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad41bd850 con 0x55af64187370 2024-09-19T09:09:07.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.871+0000 7f8adcad7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad41bd180 con 0x55af6417db40 2024-09-19T09:09:07.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.871+0000 7f8adcad7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad4072210 con 0x55af6417c5d0 2024-09-19T09:09:07.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.871+0000 7f8adcad7740 10 client.0 _ll_register_callbacks cb 0x7f8ac400db90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:07.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.872+0000 7f8ad37fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af6417c5d0 0x55af6418f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:07.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.872+0000 7f8ad37fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af6417c5d0 0x55af6418f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:44842/0 (socket says 172.21.15.120:44842) 2024-09-19T09:09:07.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.872+0000 7f8ad37fe640 1 -- 172.21.15.120:0/2924606219 learned_addr learned my addr 172.21.15.120:0/2924606219 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:07.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.872+0000 7f8ad3fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af6417db40 0x55af6418f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:07.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.872+0000 7f8ad37fe640 1 --2- 172.21.15.120:0/2924606219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af64187370 0x55af641806b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:07.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.872+0000 7f8ad37fe640 1 -- 172.21.15.120:0/2924606219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af6417c5d0 msgr2=0x55af6418f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:07.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.872+0000 7f8ad37fe640 1 --2- 172.21.15.120:0/2924606219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af6417c5d0 0x55af6418f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.872+0000 7f8ad37fe640 1 -- 172.21.15.120:0/2924606219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af6417db40 msgr2=0x55af6418f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:07.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.872+0000 7f8ad37fe640 1 --2- 172.21.15.120:0/2924606219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af6417db40 0x55af6418f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.872+0000 7f8ad37fe640 1 -- 172.21.15.120:0/2924606219 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af64180c70 con 0x55af64187370 2024-09-19T09:09:07.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.872+0000 7f8ad37fe640 1 --2- 172.21.15.120:0/2924606219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af64187370 0x55af641806b0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8021ef0 tx=0x7f8ac80047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8779cd107dd04f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:07.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.872+0000 7f8ab37fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:09:07.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.872+0000 7f8ab37fe640 1 -- 172.21.15.120:0/2924606219 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8017780 con 0x55af64187370 2024-09-19T09:09:07.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.872+0000 7f8ab37fe640 1 -- 172.21.15.120:0/2924606219 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f8ac8017b50 con 0x55af64187370 2024-09-19T09:09:07.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.872+0000 7f8adcad7740 1 -- 172.21.15.120:0/2924606219 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ac801c030 con 0x55af64187370 2024-09-19T09:09:07.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.872+0000 7f8ab37fe640 1 -- 172.21.15.120:0/2924606219 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8ac8022410 con 0x55af64187370 2024-09-19T09:09:07.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.872+0000 7f8adcad7740 10 client.17568 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:07.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.873+0000 7f8adcad7740 1 -- 172.21.15.120:0/2924606219 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55af64201190 con 0x55af64187370 2024-09-19T09:09:07.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.873+0000 7f8ab37fe640 1 -- 172.21.15.120:0/2924606219 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ac801c030 con 0x55af64187370 2024-09-19T09:09:07.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.873+0000 7f8ab37fe640 1 -- 172.21.15.120:0/2924606219 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8ac8022c50 con 0x55af64187370 2024-09-19T09:09:07.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.873+0000 7f8adcad7740 10 client.17568 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:07.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.873+0000 7f8adcad7740 10 client.17568 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:09:07.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.873+0000 7f8adcad7740 20 client.17568 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:07.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.873+0000 7f8adcad7740 1 --2- 172.21.15.120:0/2924606219 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55af64201750 0x55af64203b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:07.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.873+0000 7f8adcad7740 4 client.17568 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:09:07.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.873+0000 7f8adcad7740 1 -- 172.21.15.120:0/2924606219 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55af640e9d70 con 0x55af64201750 2024-09-19T09:09:07.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.873+0000 7f8ad3fff640 1 --2- 172.21.15.120:0/2924606219 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55af64201750 0x55af64203b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:07.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.873+0000 7f8ab37fe640 10 client.17568 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:09:07.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.873+0000 7f8ad3fff640 1 --2- 172.21.15.120:0/2924606219 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55af64201750 0x55af64203b40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b501f2215179e90b server_cookie=ad7afecd95f38f0a in_seq=0 out_seq=0 2024-09-19T09:09:07.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.874+0000 7f8ab37fe640 1 -- 172.21.15.120:0/2924606219 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55af640e9d70 con 0x55af64201750 2024-09-19T09:09:07.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.874+0000 7f8ab37fe640 10 client.17568 handle_command_reply: tid=0 2024-09-19T09:09:07.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.874+0000 7f8adcad7740 1 client.17568 shutdown 2024-09-19T09:09:07.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.874+0000 7f8adcad7740 1 -- 172.21.15.120:0/2924606219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af64187370 msgr2=0x55af641806b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:07.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.874+0000 7f8adcad7740 1 --2- 172.21.15.120:0/2924606219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af64187370 0x55af641806b0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8021ef0 tx=0x7f8ac80047d0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.874+0000 7f8adcad7740 1 -- 172.21.15.120:0/2924606219 shutdown_connections 2024-09-19T09:09:07.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.874+0000 7f8adcad7740 1 --2- 172.21.15.120:0/2924606219 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55af64201750 0x55af64203b40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.874+0000 7f8adcad7740 1 --2- 172.21.15.120:0/2924606219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55af64187370 0x55af641806b0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.874+0000 7f8ab37fe640 0 client.17568 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:09:07.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.874+0000 7f8adcad7740 1 --2- 172.21.15.120:0/2924606219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55af6417db40 0x55af6418f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.874+0000 7f8adcad7740 1 --2- 172.21.15.120:0/2924606219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af6417c5d0 0x55af6418f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.874+0000 7f8adcad7740 1 -- 172.21.15.120:0/2924606219 >> 172.21.15.120:0/2924606219 conn(0x55af64145310 msgr2=0x55af641858e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:07.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.874+0000 7f8adcad7740 1 -- 172.21.15.120:0/2924606219 shutdown_connections 2024-09-19T09:09:07.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.874+0000 7f8adcad7740 1 -- 172.21.15.120:0/2924606219 wait complete. 2024-09-19T09:09:07.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.874+0000 7f8adcad7740 20 client.17568 trim_cache size 0 max 16384 2024-09-19T09:09:07.883 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:09:07.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.875+0000 7f8ada6fe640 1 -- 172.21.15.120:0/667645726 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8abc0a32c0 msgr2=0x7f8abc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:07.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.875+0000 7f8ada6fe640 1 --2- 172.21.15.120:0/667645726 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8abc0a32c0 0x7f8abc0a5780 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8acc0058f0 tx=0x7f8acc005940 comp rx=0 tx=0).stop 2024-09-19T09:09:07.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.875+0000 7f8ada6fe640 1 -- 172.21.15.120:0/667645726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ad410c170 msgr2=0x7f8ad419e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:07.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.875+0000 7f8ada6fe640 1 --2- 172.21.15.120:0/667645726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ad410c170 0x7f8ad419e890 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009fd0 tx=0x7f8acc00ef10 comp rx=0 tx=0).stop 2024-09-19T09:09:07.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.875+0000 7f8ada6fe640 1 -- 172.21.15.120:0/667645726 shutdown_connections 2024-09-19T09:09:07.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.875+0000 7f8ada6fe640 1 --2- 172.21.15.120:0/667645726 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8abc0a32c0 0x7f8abc0a5780 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.875+0000 7f8ada6fe640 1 --2- 172.21.15.120:0/667645726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ad410c170 0x7f8ad419e890 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.875+0000 7f8ada6fe640 1 --2- 172.21.15.120:0/667645726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ad4103660 0x7f8ad41a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.875+0000 7f8ada6fe640 1 --2- 172.21.15.120:0/667645726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ad4102d40 0x7f8ad41a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:07.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.875+0000 7f8ada6fe640 1 -- 172.21.15.120:0/667645726 >> 172.21.15.120:0/667645726 conn(0x7f8ad40fc7a0 msgr2=0x7f8ad40fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:07.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.875+0000 7f8ada6fe640 1 -- 172.21.15.120:0/667645726 shutdown_connections 2024-09-19T09:09:07.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:07.875+0000 7f8ada6fe640 1 -- 172.21.15.120:0/667645726 wait complete. 2024-09-19T09:09:08.052 INFO:tasks.workunit.client.0.smithi120.stdout: 1 536873 9.21 MB/sec execute 183 sec latency 93.434 ms 2024-09-19T09:09:08.599 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:09:08.851 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:09:09.052 INFO:tasks.workunit.client.0.smithi120.stdout: 1 540210 9.26 MB/sec execute 184 sec latency 10.559 ms 2024-09-19T09:09:09.138 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:08 smithi120 ceph-mon[31150]: pgmap v530: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 459 op/s 2024-09-19T09:09:09.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:08 smithi196 ceph-mon[35150]: pgmap v530: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 459 op/s 2024-09-19T09:09:09.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:08 smithi153 ceph-mon[35071]: pgmap v530: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 459 op/s 2024-09-19T09:09:09.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.638+0000 7f2cce2c8640 1 -- 172.21.15.120:0/149089931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cc8101de0 msgr2=0x7f2cc81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:09.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.638+0000 7f2cce2c8640 1 --2- 172.21.15.120:0/149089931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cc8101de0 0x7f2cc81021c0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8009900 tx=0x7f2cb802a410 comp rx=0 tx=0).stop 2024-09-19T09:09:09.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.638+0000 7f2cce2c8640 1 -- 172.21.15.120:0/149089931 shutdown_connections 2024-09-19T09:09:09.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.638+0000 7f2cce2c8640 1 --2- 172.21.15.120:0/149089931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2cc8106f10 0x7f2cc810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.638+0000 7f2cce2c8640 1 --2- 172.21.15.120:0/149089931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2cc8102790 0x7f2cc81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.638+0000 7f2cce2c8640 1 --2- 172.21.15.120:0/149089931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cc8101de0 0x7f2cc81021c0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.638+0000 7f2cce2c8640 1 -- 172.21.15.120:0/149089931 >> 172.21.15.120:0/149089931 conn(0x7f2cc80fd8c0 msgr2=0x7f2cc80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:09.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.638+0000 7f2cce2c8640 1 -- 172.21.15.120:0/149089931 shutdown_connections 2024-09-19T09:09:09.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.638+0000 7f2cce2c8640 1 -- 172.21.15.120:0/149089931 wait complete. 2024-09-19T09:09:09.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.639+0000 7f2cce2c8640 1 Processor -- start 2024-09-19T09:09:09.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.639+0000 7f2cce2c8640 1 -- start start 2024-09-19T09:09:09.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.639+0000 7f2cce2c8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cc8101de0 0x7f2cc8198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:09.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.639+0000 7f2cce2c8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2cc8102790 0x7f2cc8198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:09.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.639+0000 7f2cce2c8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2cc8106f10 0x7f2cc819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:09.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.639+0000 7f2cce2c8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc8112010 con 0x7f2cc8101de0 2024-09-19T09:09:09.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.639+0000 7f2cce2c8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc8111e90 con 0x7f2cc8102790 2024-09-19T09:09:09.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.639+0000 7f2cce2c8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc8112190 con 0x7f2cc8106f10 2024-09-19T09:09:09.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.639+0000 7f2cc7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cc8101de0 0x7f2cc8198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:09.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.639+0000 7f2cc7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cc8101de0 0x7f2cc8198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58510/0 (socket says 172.21.15.120:58510) 2024-09-19T09:09:09.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.639+0000 7f2cc7fff640 1 -- 172.21.15.120:0/1751612948 learned_addr learned my addr 172.21.15.120:0/1751612948 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:09.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.639+0000 7f2cc77fe640 1 --2- 172.21.15.120:0/1751612948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2cc8102790 0x7f2cc8198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:09.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.640+0000 7f2ccc865640 1 --2- 172.21.15.120:0/1751612948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2cc8106f10 0x7f2cc819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:09.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.640+0000 7f2cc7fff640 1 -- 172.21.15.120:0/1751612948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2cc8106f10 msgr2=0x7f2cc819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:09.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.640+0000 7f2cc7fff640 1 --2- 172.21.15.120:0/1751612948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2cc8106f10 0x7f2cc819ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.640+0000 7f2cc7fff640 1 -- 172.21.15.120:0/1751612948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2cc8102790 msgr2=0x7f2cc8198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:09.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.640+0000 7f2cc7fff640 1 --2- 172.21.15.120:0/1751612948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2cc8102790 0x7f2cc8198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.640+0000 7f2cc7fff640 1 -- 172.21.15.120:0/1751612948 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc819d500 con 0x7f2cc8101de0 2024-09-19T09:09:09.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.640+0000 7f2cc7fff640 1 --2- 172.21.15.120:0/1751612948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cc8101de0 0x7f2cc8198550 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8005ab0 tx=0x7f2cb80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:09.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.640+0000 7f2cc57fa640 1 -- 172.21.15.120:0/1751612948 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb803f030 con 0x7f2cc8101de0 2024-09-19T09:09:09.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.640+0000 7f2cc57fa640 1 -- 172.21.15.120:0/1751612948 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cb8003ce0 con 0x7f2cc8101de0 2024-09-19T09:09:09.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.640+0000 7f2cc57fa640 1 -- 172.21.15.120:0/1751612948 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8035070 con 0x7f2cc8101de0 2024-09-19T09:09:09.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.640+0000 7f2cce2c8640 1 -- 172.21.15.120:0/1751612948 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc819d790 con 0x7f2cc8101de0 2024-09-19T09:09:09.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.640+0000 7f2cce2c8640 1 -- 172.21.15.120:0/1751612948 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc81a50d0 con 0x7f2cc8101de0 2024-09-19T09:09:09.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.642+0000 7f2cc57fa640 1 -- 172.21.15.120:0/1751612948 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f2cb8042030 con 0x7f2cc8101de0 2024-09-19T09:09:09.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.643+0000 7f2cd06a1740 1 Processor -- start 2024-09-19T09:09:09.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.643+0000 7f2cd06a1740 1 -- start start 2024-09-19T09:09:09.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.643+0000 7f2cd06a1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e3efde5d0 0x560e3efde9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:09.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.643+0000 7f2cd06a1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3efd9280 0x560e3efd96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:09.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.643+0000 7f2cd06a1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e3efd9c00 0x560e3efe5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:09.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.643+0000 7f2cd06a1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560e3ef352f0 con 0x560e3efde5d0 2024-09-19T09:09:09.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.643+0000 7f2cd06a1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560e3eedccf0 con 0x560e3efd9280 2024-09-19T09:09:09.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.643+0000 7f2cd06a1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560e3ef312b0 con 0x560e3efd9c00 2024-09-19T09:09:09.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.643+0000 7f2ccc865640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e3efde5d0 0x560e3efde9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:09.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.643+0000 7f2ccc865640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e3efde5d0 0x560e3efde9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58516/0 (socket says 172.21.15.120:58516) 2024-09-19T09:09:09.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.643+0000 7f2ccc865640 1 -- 172.21.15.120:0/1022794395 learned_addr learned my addr 172.21.15.120:0/1022794395 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:09.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.643+0000 7f2cc57fa640 1 --2- 172.21.15.120:0/1751612948 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2cb00a32c0 0x7f2cb00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:09.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.643+0000 7f2cc77fe640 1 --2- 172.21.15.120:0/1022794395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e3efd9c00 0x560e3efe5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:09.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.643+0000 7f2ccc865640 1 -- 172.21.15.120:0/1022794395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e3efd9c00 msgr2=0x560e3efe5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:09.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.643+0000 7f2ccc865640 1 --2- 172.21.15.120:0/1022794395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e3efd9c00 0x560e3efe5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.643+0000 7f2ccc865640 1 -- 172.21.15.120:0/1022794395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3efd9280 msgr2=0x560e3efd96c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:09:09.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.643+0000 7f2ccc865640 1 --2- 172.21.15.120:0/1022794395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3efd9280 0x560e3efd96c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.643+0000 7f2ccc865640 1 -- 172.21.15.120:0/1022794395 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e3eec13a0 con 0x560e3efde5d0 2024-09-19T09:09:09.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.644+0000 7f2ccc865640 1 --2- 172.21.15.120:0/1751612948 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2cb00a32c0 0x7f2cb00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:09.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.644+0000 7f2cc57fa640 1 -- 172.21.15.120:0/1751612948 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f2cb80f0210 con 0x7f2cc8101de0 2024-09-19T09:09:09.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.644+0000 7f2ccc865640 1 --2- 172.21.15.120:0/1022794395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e3efde5d0 0x560e3efde9b0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0007f40 tx=0x7f2cc000c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83642f243f4dfde server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:09.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.644+0000 7f2cae7fc640 1 -- 172.21.15.120:0/1022794395 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0017030 con 0x560e3efde5d0 2024-09-19T09:09:09.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.644+0000 7f2cae7fc640 1 -- 172.21.15.120:0/1022794395 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc0013d90 con 0x560e3efde5d0 2024-09-19T09:09:09.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.644+0000 7f2cae7fc640 1 -- 172.21.15.120:0/1022794395 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc001d9d0 con 0x560e3efde5d0 2024-09-19T09:09:09.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.645+0000 7f2cd06a1740 1 -- 172.21.15.120:0/1022794395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e3efde5d0 msgr2=0x560e3efde9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:09.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.645+0000 7f2cd06a1740 1 --2- 172.21.15.120:0/1022794395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e3efde5d0 0x560e3efde9b0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0007f40 tx=0x7f2cc000c620 comp rx=0 tx=0).stop 2024-09-19T09:09:09.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.645+0000 7f2ccc865640 1 --2- 172.21.15.120:0/1751612948 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2cb00a32c0 0x7f2cb00a5780 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2cc000cfd0 tx=0x7f2cc001c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:09.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.645+0000 7f2cd06a1740 1 -- 172.21.15.120:0/1022794395 shutdown_connections 2024-09-19T09:09:09.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.645+0000 7f2cd06a1740 1 --2- 172.21.15.120:0/1022794395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e3efd9c00 0x560e3efe5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.645+0000 7f2cd06a1740 1 --2- 172.21.15.120:0/1022794395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3efd9280 0x560e3efd96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.645+0000 7f2cd06a1740 1 --2- 172.21.15.120:0/1022794395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e3efde5d0 0x560e3efde9b0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.645+0000 7f2cd06a1740 1 -- 172.21.15.120:0/1022794395 >> 172.21.15.120:0/1022794395 conn(0x560e3ef98310 msgr2=0x560e3efcb6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:09.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.645+0000 7f2cd06a1740 1 -- 172.21.15.120:0/1022794395 shutdown_connections 2024-09-19T09:09:09.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.645+0000 7f2cd06a1740 1 -- 172.21.15.120:0/1022794395 wait complete. 2024-09-19T09:09:09.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.645+0000 7f2cd06a1740 1 -- start start 2024-09-19T09:09:09.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.645+0000 7f2cd06a1740 1 Processor -- start 2024-09-19T09:09:09.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.646+0000 7f2cd06a1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e3efd9280 0x560e3efe51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:09.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.646+0000 7f2cd06a1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e3efd9c00 0x560e3efe2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:09.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.646+0000 7f2cd06a1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3efde5d0 0x560e3efe29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:09.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.646+0000 7f2cd06a1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560e3efe58d0 con 0x560e3efd9c00 2024-09-19T09:09:09.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.646+0000 7f2cd06a1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560e3eef0060 con 0x560e3efde5d0 2024-09-19T09:09:09.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.646+0000 7f2cd06a1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560e3efe5a50 con 0x560e3efd9280 2024-09-19T09:09:09.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.646+0000 7f2cd06a1740 10 client.0 _ll_register_callbacks cb 0x560e3ec438c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:09.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.646+0000 7f2cc77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e3efd9c00 0x560e3efe2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:09.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.646+0000 7f2ccc865640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3efde5d0 0x560e3efe29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:09.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.646+0000 7f2cc77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e3efd9c00 0x560e3efe2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58518/0 (socket says 172.21.15.120:58518) 2024-09-19T09:09:09.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.646+0000 7f2cc77fe640 1 -- 172.21.15.120:0/1851213454 learned_addr learned my addr 172.21.15.120:0/1851213454 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:09.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.646+0000 7f2cc7fff640 1 --2- 172.21.15.120:0/1851213454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e3efd9280 0x560e3efe51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:09.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.646+0000 7f2ccc865640 1 -- 172.21.15.120:0/1851213454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e3efd9280 msgr2=0x560e3efe51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:09.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.646+0000 7f2ccc865640 1 --2- 172.21.15.120:0/1851213454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e3efd9280 0x560e3efe51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.646+0000 7f2ccc865640 1 -- 172.21.15.120:0/1851213454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e3efd9c00 msgr2=0x560e3efe2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:09.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.646+0000 7f2ccc865640 1 --2- 172.21.15.120:0/1851213454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e3efd9c00 0x560e3efe2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.646+0000 7f2ccc865640 1 -- 172.21.15.120:0/1851213454 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e3eca63c0 con 0x560e3efde5d0 2024-09-19T09:09:09.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.646+0000 7f2ccc865640 1 --2- 172.21.15.120:0/1851213454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3efde5d0 0x560e3efe29f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0004b90 tx=0x7f2cc001cdc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=728f87188fa05456 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:09.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.646+0000 7f2ca77fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:09:09.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.647+0000 7f2ca77fe640 1 -- 172.21.15.120:0/1851213454 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0029020 con 0x560e3efde5d0 2024-09-19T09:09:09.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.647+0000 7f2cd06a1740 1 -- 172.21.15.120:0/1851213454 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cc0013d90 con 0x560e3efde5d0 2024-09-19T09:09:09.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.647+0000 7f2ca77fe640 1 -- 172.21.15.120:0/1851213454 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f2cc0025620 con 0x560e3efde5d0 2024-09-19T09:09:09.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.647+0000 7f2ca77fe640 1 -- 172.21.15.120:0/1851213454 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2cc0017040 con 0x560e3efde5d0 2024-09-19T09:09:09.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.647+0000 7f2cd06a1740 10 client.26837 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:09.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.647+0000 7f2cd06a1740 1 -- 172.21.15.120:0/1851213454 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560e3eff12a0 con 0x560e3efde5d0 2024-09-19T09:09:09.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.647+0000 7f2ca77fe640 1 -- 172.21.15.120:0/1851213454 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc0013d90 con 0x560e3efde5d0 2024-09-19T09:09:09.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.647+0000 7f2ca77fe640 1 -- 172.21.15.120:0/1851213454 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2cc0024380 con 0x560e3efde5d0 2024-09-19T09:09:09.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.647+0000 7f2cd06a1740 10 client.26837 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:09.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.647+0000 7f2cd06a1740 10 client.26837 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:09:09.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.647+0000 7f2cd06a1740 20 client.26837 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:09.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.647+0000 7f2cd06a1740 1 --2- 172.21.15.120:0/1851213454 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x560e3eff26a0 0x560e3eff4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:09.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.647+0000 7f2cd06a1740 4 client.26837 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:09:09.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.647+0000 7f2cd06a1740 1 -- 172.21.15.120:0/1851213454 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e3ef42860 con 0x560e3eff26a0 2024-09-19T09:09:09.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.648+0000 7f2cc7fff640 1 --2- 172.21.15.120:0/1851213454 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x560e3eff26a0 0x560e3eff4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:09.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.648+0000 7f2cc7fff640 1 --2- 172.21.15.120:0/1851213454 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x560e3eff26a0 0x560e3eff4a90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9cea40ad3ad1e727 server_cookie=ed463543039073e3 in_seq=0 out_seq=0 2024-09-19T09:09:09.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.648+0000 7f2ca77fe640 10 client.26837 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:09:09.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.648+0000 7f2ca77fe640 1 -- 172.21.15.120:0/1851213454 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e3ef42860 con 0x560e3eff26a0 2024-09-19T09:09:09.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.649+0000 7f2ca77fe640 10 client.26837 handle_command_reply: tid=0 2024-09-19T09:09:09.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.649+0000 7f2cd06a1740 1 client.26837 shutdown 2024-09-19T09:09:09.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.649+0000 7f2cd06a1740 1 -- 172.21.15.120:0/1851213454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3efde5d0 msgr2=0x560e3efe29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:09.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.649+0000 7f2cd06a1740 1 --2- 172.21.15.120:0/1851213454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3efde5d0 0x560e3efe29f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0004b90 tx=0x7f2cc001cdc0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.649+0000 7f2cd06a1740 1 -- 172.21.15.120:0/1851213454 shutdown_connections 2024-09-19T09:09:09.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.649+0000 7f2cd06a1740 1 --2- 172.21.15.120:0/1851213454 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x560e3eff26a0 0x560e3eff4a90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.649+0000 7f2cd06a1740 1 --2- 172.21.15.120:0/1851213454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3efde5d0 0x560e3efe29f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.649+0000 7f2cd06a1740 1 --2- 172.21.15.120:0/1851213454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e3efd9c00 0x560e3efe2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.649+0000 7f2cd06a1740 1 --2- 172.21.15.120:0/1851213454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e3efd9280 0x560e3efe51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.649+0000 7f2cd06a1740 1 -- 172.21.15.120:0/1851213454 >> 172.21.15.120:0/1851213454 conn(0x560e3ef98310 msgr2=0x560e3efd8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:09.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.649+0000 7f2ca77fe640 0 client.26837 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:09:09.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.649+0000 7f2cd06a1740 1 -- 172.21.15.120:0/1851213454 shutdown_connections 2024-09-19T09:09:09.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.649+0000 7f2cd06a1740 1 -- 172.21.15.120:0/1851213454 wait complete. 2024-09-19T09:09:09.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.649+0000 7f2cd06a1740 20 client.26837 trim_cache size 0 max 16384 2024-09-19T09:09:09.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.662+0000 7f2cd06a1740 1 Processor -- start 2024-09-19T09:09:09.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.662+0000 7f2cd06a1740 1 -- start start 2024-09-19T09:09:09.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.662+0000 7f2cd06a1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e3efd9280 0x560e3efcb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:09.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.662+0000 7f2cd06a1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e3efd9c00 0x560e3efe4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:09.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.662+0000 7f2cd06a1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3efde5d0 0x560e3effa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:09.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.662+0000 7f2cd06a1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc81ab4b0 con 0x560e3efd9280 2024-09-19T09:09:09.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.662+0000 7f2cd06a1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc81acff0 con 0x560e3efde5d0 2024-09-19T09:09:09.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.662+0000 7f2cd06a1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc81abb80 con 0x560e3efd9c00 2024-09-19T09:09:09.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.662+0000 7f2cc7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e3efd9280 0x560e3efcb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:09.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.662+0000 7f2cc7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e3efd9280 0x560e3efcb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58532/0 (socket says 172.21.15.120:58532) 2024-09-19T09:09:09.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.662+0000 7f2cc7fff640 1 -- 172.21.15.120:0/2302560037 learned_addr learned my addr 172.21.15.120:0/2302560037 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:09.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.662+0000 7f2cc77fe640 1 --2- 172.21.15.120:0/2302560037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e3efd9c00 0x560e3efe4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:09.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.662+0000 7f2ccc865640 1 --2- 172.21.15.120:0/2302560037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3efde5d0 0x560e3effa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:09.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.662+0000 7f2cc7fff640 1 -- 172.21.15.120:0/2302560037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e3efd9c00 msgr2=0x560e3efe4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:09.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.662+0000 7f2cc7fff640 1 --2- 172.21.15.120:0/2302560037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e3efd9c00 0x560e3efe4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.662+0000 7f2cc7fff640 1 -- 172.21.15.120:0/2302560037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3efde5d0 msgr2=0x560e3effa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:09.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.662+0000 7f2cc7fff640 1 --2- 172.21.15.120:0/2302560037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3efde5d0 0x560e3effa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.662+0000 7f2cc7fff640 1 -- 172.21.15.120:0/2302560037 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e3efe5550 con 0x560e3efd9280 2024-09-19T09:09:09.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.662+0000 7f2cc7fff640 1 --2- 172.21.15.120:0/2302560037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e3efd9280 0x560e3efcb510 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8000c00 tx=0x7f2cb803b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949ae9993a5376d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:09.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.663+0000 7f2cadffb640 1 -- 172.21.15.120:0/2302560037 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8049040 con 0x560e3efd9280 2024-09-19T09:09:09.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.663+0000 7f2cadffb640 1 -- 172.21.15.120:0/2302560037 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cb80bc830 con 0x560e3efd9280 2024-09-19T09:09:09.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.663+0000 7f2cadffb640 1 -- 172.21.15.120:0/2302560037 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb803bd00 con 0x560e3efd9280 2024-09-19T09:09:09.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.663+0000 7f2cd06a1740 1 -- 172.21.15.120:0/2302560037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e3efd9280 msgr2=0x560e3efcb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:09.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.663+0000 7f2cd06a1740 1 --2- 172.21.15.120:0/2302560037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e3efd9280 0x560e3efcb510 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8000c00 tx=0x7f2cb803b790 comp rx=0 tx=0).stop 2024-09-19T09:09:09.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.663+0000 7f2cd06a1740 1 -- 172.21.15.120:0/2302560037 shutdown_connections 2024-09-19T09:09:09.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.663+0000 7f2cd06a1740 1 --2- 172.21.15.120:0/2302560037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3efde5d0 0x560e3effa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.663+0000 7f2cd06a1740 1 --2- 172.21.15.120:0/2302560037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e3efd9c00 0x560e3efe4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.663+0000 7f2cd06a1740 1 --2- 172.21.15.120:0/2302560037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e3efd9280 0x560e3efcb510 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.663+0000 7f2cd06a1740 1 -- 172.21.15.120:0/2302560037 >> 172.21.15.120:0/2302560037 conn(0x560e3ef98310 msgr2=0x560e3efe3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:09.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.663+0000 7f2cd06a1740 1 -- 172.21.15.120:0/2302560037 shutdown_connections 2024-09-19T09:09:09.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.663+0000 7f2cd06a1740 1 -- 172.21.15.120:0/2302560037 wait complete. 2024-09-19T09:09:09.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.664+0000 7f2cd06a1740 1 -- start start 2024-09-19T09:09:09.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.664+0000 7f2cd06a1740 1 Processor -- start 2024-09-19T09:09:09.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.664+0000 7f2cd06a1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e3efd9280 0x560e3effaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:09.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.664+0000 7f2cd06a1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e3efd9c00 0x560e3eff7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:09.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.664+0000 7f2cd06a1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3efde5d0 0x560e3eff83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:09.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.664+0000 7f2cd06a1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc81ac920 con 0x560e3efd9c00 2024-09-19T09:09:09.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.664+0000 7f2cd06a1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc81ac250 con 0x560e3efde5d0 2024-09-19T09:09:09.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.664+0000 7f2cd06a1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc81a9470 con 0x560e3efd9280 2024-09-19T09:09:09.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.664+0000 7f2cd06a1740 10 client.0 _ll_register_callbacks cb 0x7f2cc0007d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:09.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.664+0000 7f2cc77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e3efd9c00 0x560e3eff7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:09.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.664+0000 7f2cc7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e3efd9280 0x560e3effaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:09.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.664+0000 7f2cc7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e3efd9280 0x560e3effaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:44904/0 (socket says 172.21.15.120:44904) 2024-09-19T09:09:09.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.664+0000 7f2cc7fff640 1 -- 172.21.15.120:0/3351104797 learned_addr learned my addr 172.21.15.120:0/3351104797 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:09.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.664+0000 7f2ccc865640 1 --2- 172.21.15.120:0/3351104797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3efde5d0 0x560e3eff83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:09.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.665+0000 7f2cc77fe640 1 -- 172.21.15.120:0/3351104797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e3efd9280 msgr2=0x560e3effaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:09.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.665+0000 7f2cc77fe640 1 --2- 172.21.15.120:0/3351104797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e3efd9280 0x560e3effaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.665+0000 7f2cc77fe640 1 -- 172.21.15.120:0/3351104797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3efde5d0 msgr2=0x560e3eff83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:09.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.665+0000 7f2cc77fe640 1 --2- 172.21.15.120:0/3351104797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3efde5d0 0x560e3eff83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.665+0000 7f2cc77fe640 1 -- 172.21.15.120:0/3351104797 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e3eff4e00 con 0x560e3efd9c00 2024-09-19T09:09:09.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.665+0000 7f2cc7fff640 1 --2- 172.21.15.120:0/3351104797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e3efd9280 0x560e3effaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:09.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.665+0000 7f2cc77fe640 1 --2- 172.21.15.120:0/3351104797 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e3efd9c00 0x560e3eff7dc0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc006ca0 tx=0x7f2cbc017b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcaebed7762f461e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:09.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.665+0000 7f2ca77fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:09:09.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.665+0000 7f2ca77fe640 1 -- 172.21.15.120:0/3351104797 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc015d00 con 0x560e3efd9c00 2024-09-19T09:09:09.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.665+0000 7f2ca77fe640 1 -- 172.21.15.120:0/3351104797 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f2cbc01f3e0 con 0x560e3efd9c00 2024-09-19T09:09:09.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.665+0000 7f2cd06a1740 1 -- 172.21.15.120:0/3351104797 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cb80bc830 con 0x560e3efd9c00 2024-09-19T09:09:09.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.665+0000 7f2ca77fe640 1 -- 172.21.15.120:0/3351104797 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2cbc015ea0 con 0x560e3efd9c00 2024-09-19T09:09:09.765 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:09:09.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.665+0000 7f2cd06a1740 10 client.17586 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:09.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.665+0000 7f2cd06a1740 1 -- 172.21.15.120:0/3351104797 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560e3f05d1c0 con 0x560e3efd9c00 2024-09-19T09:09:09.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.665+0000 7f2ca77fe640 1 -- 172.21.15.120:0/3351104797 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cb80bc830 con 0x560e3efd9c00 2024-09-19T09:09:09.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.665+0000 7f2ca77fe640 1 -- 172.21.15.120:0/3351104797 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2cbc01e070 con 0x560e3efd9c00 2024-09-19T09:09:09.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.665+0000 7f2cd06a1740 10 client.17586 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:09.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.665+0000 7f2cd06a1740 10 client.17586 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:09:09.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.665+0000 7f2cd06a1740 20 client.17586 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:09.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.666+0000 7f2cd06a1740 1 --2- 172.21.15.120:0/3351104797 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x560e3f05d940 0x560e3f05fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:09.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.666+0000 7f2cd06a1740 4 client.17586 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:09:09.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.666+0000 7f2cd06a1740 1 -- 172.21.15.120:0/3351104797 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560e3ef3cd70 con 0x560e3f05d940 2024-09-19T09:09:09.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.666+0000 7f2cc7fff640 1 --2- 172.21.15.120:0/3351104797 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x560e3f05d940 0x560e3f05fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:09.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.666+0000 7f2ca77fe640 10 client.17586 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:09:09.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.666+0000 7f2cc7fff640 1 --2- 172.21.15.120:0/3351104797 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x560e3f05d940 0x560e3f05fd60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ca2e0cff01f99e96 server_cookie=2d5f44325d703ff0 in_seq=0 out_seq=0 2024-09-19T09:09:09.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.666+0000 7f2ca77fe640 1 -- 172.21.15.120:0/3351104797 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560e3ef3cd70 con 0x560e3f05d940 2024-09-19T09:09:09.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.666+0000 7f2ca77fe640 10 client.17586 handle_command_reply: tid=0 2024-09-19T09:09:09.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.666+0000 7f2cd06a1740 1 client.17586 shutdown 2024-09-19T09:09:09.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.666+0000 7f2cd06a1740 1 -- 172.21.15.120:0/3351104797 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e3efd9c00 msgr2=0x560e3eff7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:09.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.666+0000 7f2cd06a1740 1 --2- 172.21.15.120:0/3351104797 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e3efd9c00 0x560e3eff7dc0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc006ca0 tx=0x7f2cbc017b50 comp rx=0 tx=0).stop 2024-09-19T09:09:09.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.666+0000 7f2cd06a1740 1 -- 172.21.15.120:0/3351104797 shutdown_connections 2024-09-19T09:09:09.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.666+0000 7f2cd06a1740 1 --2- 172.21.15.120:0/3351104797 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x560e3f05d940 0x560e3f05fd60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.666+0000 7f2cd06a1740 1 --2- 172.21.15.120:0/3351104797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e3efde5d0 0x560e3eff83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.666+0000 7f2cd06a1740 1 --2- 172.21.15.120:0/3351104797 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e3efd9c00 0x560e3eff7dc0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.666+0000 7f2ca77fe640 0 client.17586 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:09:09.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.666+0000 7f2cd06a1740 1 --2- 172.21.15.120:0/3351104797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e3efd9280 0x560e3effaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.667+0000 7f2cd06a1740 1 -- 172.21.15.120:0/3351104797 >> 172.21.15.120:0/3351104797 conn(0x560e3ef98310 msgr2=0x560e3efd8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:09.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.667+0000 7f2cd06a1740 1 -- 172.21.15.120:0/3351104797 shutdown_connections 2024-09-19T09:09:09.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.667+0000 7f2cd06a1740 1 -- 172.21.15.120:0/3351104797 wait complete. 2024-09-19T09:09:09.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.667+0000 7f2cd06a1740 20 client.17586 trim_cache size 0 max 16384 2024-09-19T09:09:09.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.667+0000 7f2cce2c8640 1 -- 172.21.15.120:0/1751612948 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2cb00a32c0 msgr2=0x7f2cb00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:09.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.667+0000 7f2cce2c8640 1 --2- 172.21.15.120:0/1751612948 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2cb00a32c0 0x7f2cb00a5780 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2cc000cfd0 tx=0x7f2cc001c000 comp rx=0 tx=0).stop 2024-09-19T09:09:09.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.667+0000 7f2cce2c8640 1 -- 172.21.15.120:0/1751612948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cc8101de0 msgr2=0x7f2cc8198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:09.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.667+0000 7f2cce2c8640 1 --2- 172.21.15.120:0/1751612948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cc8101de0 0x7f2cc8198550 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8005ab0 tx=0x7f2cb80042f0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.667+0000 7f2cce2c8640 1 -- 172.21.15.120:0/1751612948 shutdown_connections 2024-09-19T09:09:09.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.667+0000 7f2cce2c8640 1 --2- 172.21.15.120:0/1751612948 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2cb00a32c0 0x7f2cb00a5780 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.667+0000 7f2cce2c8640 1 --2- 172.21.15.120:0/1751612948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2cc8106f10 0x7f2cc819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.667+0000 7f2cce2c8640 1 --2- 172.21.15.120:0/1751612948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2cc8102790 0x7f2cc8198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.667+0000 7f2cce2c8640 1 --2- 172.21.15.120:0/1751612948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cc8101de0 0x7f2cc8198550 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:09.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.667+0000 7f2cce2c8640 1 -- 172.21.15.120:0/1751612948 >> 172.21.15.120:0/1751612948 conn(0x7f2cc80fd8c0 msgr2=0x7f2cc80fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:09.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.667+0000 7f2cce2c8640 1 -- 172.21.15.120:0/1751612948 shutdown_connections 2024-09-19T09:09:09.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:09.667+0000 7f2cce2c8640 1 -- 172.21.15.120:0/1751612948 wait complete. 2024-09-19T09:09:10.052 INFO:tasks.workunit.client.0.smithi120.stdout: 1 542357 9.25 MB/sec execute 185 sec latency 101.022 ms 2024-09-19T09:09:10.519 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:09:10.770 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:09:11.052 INFO:tasks.workunit.client.0.smithi120.stdout: 1 544955 9.29 MB/sec execute 186 sec latency 68.153 ms 2024-09-19T09:09:11.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:10 smithi196 ceph-mon[35150]: pgmap v531: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 4.1 MiB/s wr, 431 op/s 2024-09-19T09:09:11.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:10 smithi153 ceph-mon[35071]: pgmap v531: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 4.1 MiB/s wr, 431 op/s 2024-09-19T09:09:11.205 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:10 smithi120 ceph-mon[31150]: pgmap v531: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 4.1 MiB/s wr, 431 op/s 2024-09-19T09:09:11.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.288+0000 7f12f5aa0640 1 -- 172.21.15.120:0/2255568590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12f0101de0 msgr2=0x7f12f01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:11.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.288+0000 7f12f5aa0640 1 --2- 172.21.15.120:0/2255568590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12f0101de0 0x7f12f01021c0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f12e0009900 tx=0x7f12e002a410 comp rx=0 tx=0).stop 2024-09-19T09:09:11.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.288+0000 7f12f5aa0640 1 -- 172.21.15.120:0/2255568590 shutdown_connections 2024-09-19T09:09:11.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.288+0000 7f12f5aa0640 1 --2- 172.21.15.120:0/2255568590 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12f0106f10 0x7f12f010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.288+0000 7f12f5aa0640 1 --2- 172.21.15.120:0/2255568590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12f0102790 0x7f12f01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.288+0000 7f12f5aa0640 1 --2- 172.21.15.120:0/2255568590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12f0101de0 0x7f12f01021c0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.288+0000 7f12f5aa0640 1 -- 172.21.15.120:0/2255568590 >> 172.21.15.120:0/2255568590 conn(0x7f12f00fd8e0 msgr2=0x7f12f00ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:11.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.288+0000 7f12f5aa0640 1 -- 172.21.15.120:0/2255568590 shutdown_connections 2024-09-19T09:09:11.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.288+0000 7f12f5aa0640 1 -- 172.21.15.120:0/2255568590 wait complete. 2024-09-19T09:09:11.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.289+0000 7f12f5aa0640 1 Processor -- start 2024-09-19T09:09:11.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.289+0000 7f12f5aa0640 1 -- start start 2024-09-19T09:09:11.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.289+0000 7f12f5aa0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12f0101de0 0x7f12f0198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:11.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.289+0000 7f12f5aa0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12f0102790 0x7f12f0198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:11.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.289+0000 7f12f5aa0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12f0106f10 0x7f12f019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:11.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.289+0000 7f12f5aa0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f12f0111fc0 con 0x7f12f0101de0 2024-09-19T09:09:11.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.289+0000 7f12f5aa0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f12f0111e40 con 0x7f12f0102790 2024-09-19T09:09:11.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.289+0000 7f12f5aa0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f12f0112140 con 0x7f12f0106f10 2024-09-19T09:09:11.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.289+0000 7f12ef7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12f0101de0 0x7f12f0198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:11.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.289+0000 7f12ef7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12f0101de0 0x7f12f0198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47796/0 (socket says 172.21.15.120:47796) 2024-09-19T09:09:11.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.289+0000 7f12ef7fe640 1 -- 172.21.15.120:0/4060267847 learned_addr learned my addr 172.21.15.120:0/4060267847 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:11.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.289+0000 7f12eeffd640 1 --2- 172.21.15.120:0/4060267847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12f0102790 0x7f12f0198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:11.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.289+0000 7f12effff640 1 --2- 172.21.15.120:0/4060267847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12f0106f10 0x7f12f019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:11.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.289+0000 7f12ef7fe640 1 -- 172.21.15.120:0/4060267847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12f0106f10 msgr2=0x7f12f019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:11.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.289+0000 7f12ef7fe640 1 --2- 172.21.15.120:0/4060267847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12f0106f10 0x7f12f019ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.289+0000 7f12ef7fe640 1 -- 172.21.15.120:0/4060267847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12f0102790 msgr2=0x7f12f0198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:11.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.289+0000 7f12ef7fe640 1 --2- 172.21.15.120:0/4060267847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12f0102790 0x7f12f0198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.290+0000 7f12ef7fe640 1 -- 172.21.15.120:0/4060267847 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12f019d3d0 con 0x7f12f0101de0 2024-09-19T09:09:11.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.290+0000 7f12ef7fe640 1 --2- 172.21.15.120:0/4060267847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12f0101de0 0x7f12f0198420 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f12e0004770 tx=0x7f12e0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:11.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.290+0000 7f12ecff9640 1 -- 172.21.15.120:0/4060267847 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e003f070 con 0x7f12f0101de0 2024-09-19T09:09:11.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.290+0000 7f12f5aa0640 1 -- 172.21.15.120:0/4060267847 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12f019d660 con 0x7f12f0101de0 2024-09-19T09:09:11.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.290+0000 7f12ecff9640 1 -- 172.21.15.120:0/4060267847 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12e0031ca0 con 0x7f12f0101de0 2024-09-19T09:09:11.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.290+0000 7f12ecff9640 1 -- 172.21.15.120:0/4060267847 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e0035030 con 0x7f12f0101de0 2024-09-19T09:09:11.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.290+0000 7f12f5aa0640 1 -- 172.21.15.120:0/4060267847 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12f01a4fa0 con 0x7f12f0101de0 2024-09-19T09:09:11.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.292+0000 7f12ecff9640 1 -- 172.21.15.120:0/4060267847 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f12e0042070 con 0x7f12f0101de0 2024-09-19T09:09:11.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.292+0000 7f12f7e79740 1 Processor -- start 2024-09-19T09:09:11.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.293+0000 7f12f7e79740 1 -- start start 2024-09-19T09:09:11.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.293+0000 7f12f7e79740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c991697b40 0x55c991696090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:11.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.293+0000 7f12f7e79740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9916965d0 0x55c991696a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:11.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.293+0000 7f12f7e79740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9916a1370 0x55c9916a3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:11.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.293+0000 7f12f7e79740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c9915fc2f0 con 0x55c9916965d0 2024-09-19T09:09:11.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.293+0000 7f12f7e79740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c9915a3cf0 con 0x55c991697b40 2024-09-19T09:09:11.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.293+0000 7f12f7e79740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c9915f82b0 con 0x55c9916a1370 2024-09-19T09:09:11.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.293+0000 7f12ef7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9916965d0 0x55c991696a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:11.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.293+0000 7f12effff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c991697b40 0x55c991696090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:11.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.293+0000 7f12effff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c991697b40 0x55c991696090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38322/0 (socket says 172.21.15.120:38322) 2024-09-19T09:09:11.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.293+0000 7f12ecff9640 1 --2- 172.21.15.120:0/4060267847 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f12d80a3240 0x7f12d80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:11.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.293+0000 7f12effff640 1 -- 172.21.15.120:0/1070469516 learned_addr learned my addr 172.21.15.120:0/1070469516 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:11.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.293+0000 7f12eeffd640 1 --2- 172.21.15.120:0/1070469516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9916a1370 0x55c9916a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:11.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.293+0000 7f12effff640 1 --2- 172.21.15.120:0/4060267847 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f12d80a3240 0x7f12d80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:11.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.293+0000 7f12effff640 1 -- 172.21.15.120:0/1070469516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9916a1370 msgr2=0x55c9916a3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:11.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.293+0000 7f12effff640 1 --2- 172.21.15.120:0/1070469516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9916a1370 0x55c9916a3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.293+0000 7f12effff640 1 -- 172.21.15.120:0/1070469516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9916965d0 msgr2=0x55c991696a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:11.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.293+0000 7f12effff640 1 --2- 172.21.15.120:0/1070469516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9916965d0 0x55c991696a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.293+0000 7f12effff640 1 -- 172.21.15.120:0/1070469516 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9915883a0 con 0x55c991697b40 2024-09-19T09:09:11.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.293+0000 7f12eeffd640 1 --2- 172.21.15.120:0/1070469516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9916a1370 0x55c9916a3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:11.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.293+0000 7f12ef7fe640 1 --2- 172.21.15.120:0/1070469516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9916965d0 0x55c991696a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:11.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.293+0000 7f12ecff9640 1 -- 172.21.15.120:0/4060267847 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f12e00eff90 con 0x7f12f0101de0 2024-09-19T09:09:11.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.294+0000 7f12effff640 1 --2- 172.21.15.120:0/1070469516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c991697b40 0x55c991696090 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f12e8005f40 tx=0x7f12e8012f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb6c117bfa324f48 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:11.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.294+0000 7f12effff640 1 --2- 172.21.15.120:0/4060267847 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f12d80a3240 0x7f12d80a5700 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f12e801b000 tx=0x7f12e801d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:11.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.294+0000 7f12d5ffb640 1 -- 172.21.15.120:0/1070469516 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e8022070 con 0x55c991697b40 2024-09-19T09:09:11.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.294+0000 7f12d5ffb640 1 -- 172.21.15.120:0/1070469516 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12e8011c40 con 0x55c991697b40 2024-09-19T09:09:11.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.294+0000 7f12d5ffb640 1 -- 172.21.15.120:0/1070469516 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e800f3c0 con 0x55c991697b40 2024-09-19T09:09:11.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.294+0000 7f12f7e79740 1 -- 172.21.15.120:0/1070469516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c991697b40 msgr2=0x55c991696090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:11.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.294+0000 7f12f7e79740 1 --2- 172.21.15.120:0/1070469516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c991697b40 0x55c991696090 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f12e8005f40 tx=0x7f12e8012f00 comp rx=0 tx=0).stop 2024-09-19T09:09:11.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.295+0000 7f12f7e79740 1 -- 172.21.15.120:0/1070469516 shutdown_connections 2024-09-19T09:09:11.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.295+0000 7f12f7e79740 1 --2- 172.21.15.120:0/1070469516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9916a1370 0x55c9916a3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.295+0000 7f12f7e79740 1 --2- 172.21.15.120:0/1070469516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9916965d0 0x55c991696a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.295+0000 7f12f7e79740 1 --2- 172.21.15.120:0/1070469516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c991697b40 0x55c991696090 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.295+0000 7f12f7e79740 1 -- 172.21.15.120:0/1070469516 >> 172.21.15.120:0/1070469516 conn(0x55c99165f310 msgr2=0x55c991692c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:11.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.295+0000 7f12f7e79740 1 -- 172.21.15.120:0/1070469516 shutdown_connections 2024-09-19T09:09:11.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.295+0000 7f12f7e79740 1 -- 172.21.15.120:0/1070469516 wait complete. 2024-09-19T09:09:11.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.295+0000 7f12f7e79740 1 -- start start 2024-09-19T09:09:11.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.295+0000 7f12f7e79740 1 Processor -- start 2024-09-19T09:09:11.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.295+0000 7f12f7e79740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9916965d0 0x55c9916a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:11.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.296+0000 7f12f7e79740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c991697b40 0x55c99169e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:11.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.296+0000 7f12f7e79740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9916a1370 0x55c99169e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:11.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.296+0000 7f12f7e79740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c9916a4010 con 0x55c991697b40 2024-09-19T09:09:11.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.296+0000 7f12f7e79740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c9915b7060 con 0x55c9916a1370 2024-09-19T09:09:11.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.296+0000 7f12f7e79740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c9916a4190 con 0x55c9916965d0 2024-09-19T09:09:11.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.296+0000 7f12f7e79740 10 client.0 _ll_register_callbacks cb 0x55c99130a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:11.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.296+0000 7f12ef7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9916965d0 0x55c9916a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:11.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.296+0000 7f12eeffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c991697b40 0x55c99169e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:11.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.296+0000 7f12effff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9916a1370 0x55c99169e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:11.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.296+0000 7f12ef7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9916965d0 0x55c9916a0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:59458/0 (socket says 172.21.15.120:59458) 2024-09-19T09:09:11.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.296+0000 7f12eeffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c991697b40 0x55c99169e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47814/0 (socket says 172.21.15.120:47814) 2024-09-19T09:09:11.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.296+0000 7f12ef7fe640 1 -- 172.21.15.120:0/2106870567 learned_addr learned my addr 172.21.15.120:0/2106870567 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:11.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.296+0000 7f12effff640 1 -- 172.21.15.120:0/2106870567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9916965d0 msgr2=0x55c9916a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:11.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.296+0000 7f12effff640 1 --2- 172.21.15.120:0/2106870567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9916965d0 0x55c9916a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.296+0000 7f12effff640 1 -- 172.21.15.120:0/2106870567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c991697b40 msgr2=0x55c99169e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:11.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.296+0000 7f12effff640 1 --2- 172.21.15.120:0/2106870567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c991697b40 0x55c99169e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.296+0000 7f12effff640 1 -- 172.21.15.120:0/2106870567 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c99136d3c0 con 0x55c9916a1370 2024-09-19T09:09:11.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.296+0000 7f12eeffd640 1 --2- 172.21.15.120:0/2106870567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c991697b40 0x55c99169e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:11.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.296+0000 7f12effff640 1 --2- 172.21.15.120:0/2106870567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9916a1370 0x55c99169e610 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f12e8012460 tx=0x7f12e800e910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c55e5937d4f08b7b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:11.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.296+0000 7f12ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:09:11.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.296+0000 7f12ceffd640 1 -- 172.21.15.120:0/2106870567 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e8022070 con 0x55c9916a1370 2024-09-19T09:09:11.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.296+0000 7f12ceffd640 1 -- 172.21.15.120:0/2106870567 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f12e801d400 con 0x55c9916a1370 2024-09-19T09:09:11.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.297+0000 7f12f7e79740 1 -- 172.21.15.120:0/2106870567 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12e8011c40 con 0x55c9916a1370 2024-09-19T09:09:11.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.297+0000 7f12ceffd640 1 -- 172.21.15.120:0/2106870567 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f12e801d600 con 0x55c9916a1370 2024-09-19T09:09:11.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.297+0000 7f12f7e79740 10 client.26855 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:11.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.297+0000 7f12f7e79740 1 -- 172.21.15.120:0/2106870567 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c9916afab0 con 0x55c9916a1370 2024-09-19T09:09:11.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.297+0000 7f12ceffd640 1 -- 172.21.15.120:0/2106870567 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12e8011c40 con 0x55c9916a1370 2024-09-19T09:09:11.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.298+0000 7f12ceffd640 1 -- 172.21.15.120:0/2106870567 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f12e8030c80 con 0x55c9916a1370 2024-09-19T09:09:11.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.298+0000 7f12f7e79740 10 client.26855 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:11.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.298+0000 7f12f7e79740 10 client.26855 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:09:11.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.298+0000 7f12f7e79740 20 client.26855 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:11.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.298+0000 7f12f7e79740 1 --2- 172.21.15.120:0/2106870567 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55c9916b0f10 0x55c9916b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:11.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.298+0000 7f12f7e79740 4 client.26855 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:09:11.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.298+0000 7f12f7e79740 1 -- 172.21.15.120:0/2106870567 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c991609860 con 0x55c9916b0f10 2024-09-19T09:09:11.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.298+0000 7f12ef7fe640 1 --2- 172.21.15.120:0/2106870567 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55c9916b0f10 0x55c9916b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:11.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.299+0000 7f12ef7fe640 1 --2- 172.21.15.120:0/2106870567 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55c9916b0f10 0x55c9916b3300 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c6a05f16fb079de1 server_cookie=589661558f4844ae in_seq=0 out_seq=0 2024-09-19T09:09:11.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.299+0000 7f12ceffd640 10 client.26855 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:09:11.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.299+0000 7f12ceffd640 1 -- 172.21.15.120:0/2106870567 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c991609860 con 0x55c9916b0f10 2024-09-19T09:09:11.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.299+0000 7f12ceffd640 10 client.26855 handle_command_reply: tid=0 2024-09-19T09:09:11.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.299+0000 7f12f7e79740 1 client.26855 shutdown 2024-09-19T09:09:11.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.300+0000 7f12f7e79740 1 -- 172.21.15.120:0/2106870567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9916a1370 msgr2=0x55c99169e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:11.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.300+0000 7f12f7e79740 1 --2- 172.21.15.120:0/2106870567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9916a1370 0x55c99169e610 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f12e8012460 tx=0x7f12e800e910 comp rx=0 tx=0).stop 2024-09-19T09:09:11.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.300+0000 7f12f7e79740 1 -- 172.21.15.120:0/2106870567 shutdown_connections 2024-09-19T09:09:11.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.300+0000 7f12f7e79740 1 --2- 172.21.15.120:0/2106870567 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55c9916b0f10 0x55c9916b3300 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.300+0000 7f12f7e79740 1 --2- 172.21.15.120:0/2106870567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9916a1370 0x55c99169e610 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.300+0000 7f12f7e79740 1 --2- 172.21.15.120:0/2106870567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c991697b40 0x55c99169e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.300+0000 7f12f7e79740 1 --2- 172.21.15.120:0/2106870567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9916965d0 0x55c9916a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.300+0000 7f12ceffd640 0 client.26855 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:09:11.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.300+0000 7f12f7e79740 1 -- 172.21.15.120:0/2106870567 >> 172.21.15.120:0/2106870567 conn(0x55c99165f310 msgr2=0x55c991698960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:11.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.300+0000 7f12f7e79740 1 -- 172.21.15.120:0/2106870567 shutdown_connections 2024-09-19T09:09:11.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.300+0000 7f12f7e79740 1 -- 172.21.15.120:0/2106870567 wait complete. 2024-09-19T09:09:11.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.300+0000 7f12f7e79740 20 client.26855 trim_cache size 0 max 16384 2024-09-19T09:09:11.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.313+0000 7f12f7e79740 1 Processor -- start 2024-09-19T09:09:11.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.313+0000 7f12f7e79740 1 -- start start 2024-09-19T09:09:11.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.313+0000 7f12f7e79740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9916965d0 0x55c99169d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:11.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.313+0000 7f12f7e79740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c991697b40 0x55c9916ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:11.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.313+0000 7f12f7e79740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9916a1370 0x55c9916b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:11.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.313+0000 7f12f7e79740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f12f01ad5a0 con 0x55c9916965d0 2024-09-19T09:09:11.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.313+0000 7f12f7e79740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f12f01af0e0 con 0x55c9916a1370 2024-09-19T09:09:11.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.313+0000 7f12f7e79740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f12f01adc70 con 0x55c991697b40 2024-09-19T09:09:11.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.313+0000 7f12ef7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9916965d0 0x55c99169d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:11.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.313+0000 7f12ef7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9916965d0 0x55c99169d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47818/0 (socket says 172.21.15.120:47818) 2024-09-19T09:09:11.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.313+0000 7f12ef7fe640 1 -- 172.21.15.120:0/48947242 learned_addr learned my addr 172.21.15.120:0/48947242 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:11.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.313+0000 7f12effff640 1 --2- 172.21.15.120:0/48947242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9916a1370 0x55c9916b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:11.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.313+0000 7f12eeffd640 1 --2- 172.21.15.120:0/48947242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c991697b40 0x55c9916ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:11.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.313+0000 7f12ef7fe640 1 -- 172.21.15.120:0/48947242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c991697b40 msgr2=0x55c9916ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:11.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.313+0000 7f12ef7fe640 1 --2- 172.21.15.120:0/48947242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c991697b40 0x55c9916ad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.313+0000 7f12ef7fe640 1 -- 172.21.15.120:0/48947242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9916a1370 msgr2=0x55c9916b8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:11.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.313+0000 7f12ef7fe640 1 --2- 172.21.15.120:0/48947242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9916a1370 0x55c9916b8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.313+0000 7f12ef7fe640 1 -- 172.21.15.120:0/48947242 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9916b3670 con 0x55c9916965d0 2024-09-19T09:09:11.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.313+0000 7f12eeffd640 1 --2- 172.21.15.120:0/48947242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c991697b40 0x55c9916ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:09:11.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.313+0000 7f12ef7fe640 1 --2- 172.21.15.120:0/48947242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9916965d0 0x55c99169d8e0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f12e00bc360 tx=0x7f12e00f2420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746dddaee0346142 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:11.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.314+0000 7f12d57fa640 1 -- 172.21.15.120:0/48947242 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e0049040 con 0x55c9916965d0 2024-09-19T09:09:11.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.314+0000 7f12d57fa640 1 -- 172.21.15.120:0/48947242 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12e00bc660 con 0x55c9916965d0 2024-09-19T09:09:11.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.314+0000 7f12d57fa640 1 -- 172.21.15.120:0/48947242 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e00edba0 con 0x55c9916965d0 2024-09-19T09:09:11.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.314+0000 7f12f7e79740 1 -- 172.21.15.120:0/48947242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9916965d0 msgr2=0x55c99169d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:11.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.314+0000 7f12f7e79740 1 --2- 172.21.15.120:0/48947242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9916965d0 0x55c99169d8e0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f12e00bc360 tx=0x7f12e00f2420 comp rx=0 tx=0).stop 2024-09-19T09:09:11.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.314+0000 7f12f7e79740 1 -- 172.21.15.120:0/48947242 shutdown_connections 2024-09-19T09:09:11.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.314+0000 7f12f7e79740 1 --2- 172.21.15.120:0/48947242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9916a1370 0x55c9916b8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.314+0000 7f12f7e79740 1 --2- 172.21.15.120:0/48947242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c991697b40 0x55c9916ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.314+0000 7f12f7e79740 1 --2- 172.21.15.120:0/48947242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9916965d0 0x55c99169d8e0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.314+0000 7f12f7e79740 1 -- 172.21.15.120:0/48947242 >> 172.21.15.120:0/48947242 conn(0x55c99165f310 msgr2=0x55c99169c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:11.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.314+0000 7f12f7e79740 1 -- 172.21.15.120:0/48947242 shutdown_connections 2024-09-19T09:09:11.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.314+0000 7f12f7e79740 1 -- 172.21.15.120:0/48947242 wait complete. 2024-09-19T09:09:11.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.315+0000 7f12f7e79740 1 -- start start 2024-09-19T09:09:11.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.315+0000 7f12f7e79740 1 Processor -- start 2024-09-19T09:09:11.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.315+0000 7f12f7e79740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9916965d0 0x55c9916a99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:11.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.315+0000 7f12f7e79740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c991697b40 0x55c9916a4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:11.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.315+0000 7f12f7e79740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9916a1370 0x55c9916a5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:11.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.315+0000 7f12f7e79740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f12f01aea10 con 0x55c9916a1370 2024-09-19T09:09:11.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.315+0000 7f12f7e79740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f12f01ae340 con 0x55c991697b40 2024-09-19T09:09:11.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.315+0000 7f12f7e79740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f12f01ab560 con 0x55c9916965d0 2024-09-19T09:09:11.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.315+0000 7f12f7e79740 10 client.0 _ll_register_callbacks cb 0x7f12e8020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:11.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.315+0000 7f12effff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9916a1370 0x55c9916a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:11.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.315+0000 7f12effff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9916a1370 0x55c9916a5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47828/0 (socket says 172.21.15.120:47828) 2024-09-19T09:09:11.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.315+0000 7f12effff640 1 -- 172.21.15.120:0/2112036751 learned_addr learned my addr 172.21.15.120:0/2112036751 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:11.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.316+0000 7f12effff640 1 -- 172.21.15.120:0/2112036751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9916965d0 msgr2=0x55c9916a99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:11.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.316+0000 7f12effff640 1 --2- 172.21.15.120:0/2112036751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9916965d0 0x55c9916a99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.316+0000 7f12effff640 1 -- 172.21.15.120:0/2112036751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c991697b40 msgr2=0x55c9916a4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:11.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.316+0000 7f12effff640 1 --2- 172.21.15.120:0/2112036751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c991697b40 0x55c9916a4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.316+0000 7f12effff640 1 -- 172.21.15.120:0/2112036751 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9916a58b0 con 0x55c9916a1370 2024-09-19T09:09:11.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.316+0000 7f12effff640 1 --2- 172.21.15.120:0/2112036751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9916a1370 0x55c9916a5240 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f12e800a010 tx=0x7f12e8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50dab8adcbbbc5d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:11.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.316+0000 7f12ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:09:11.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.316+0000 7f12ceffd640 1 -- 172.21.15.120:0/2112036751 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e8022040 con 0x55c9916a1370 2024-09-19T09:09:11.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.316+0000 7f12ceffd640 1 -- 172.21.15.120:0/2112036751 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f12e8026ad0 con 0x55c9916a1370 2024-09-19T09:09:11.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.316+0000 7f12f7e79740 1 -- 172.21.15.120:0/2112036751 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12e00bc660 con 0x55c9916a1370 2024-09-19T09:09:11.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.316+0000 7f12ceffd640 1 -- 172.21.15.120:0/2112036751 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f12e804b050 con 0x55c9916a1370 2024-09-19T09:09:11.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.316+0000 7f12f7e79740 10 client.17604 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:11.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.316+0000 7f12f7e79740 1 -- 172.21.15.120:0/2112036751 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c99171b8d0 con 0x55c9916a1370 2024-09-19T09:09:11.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.316+0000 7f12ceffd640 1 -- 172.21.15.120:0/2112036751 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12e00bc660 con 0x55c9916a1370 2024-09-19T09:09:11.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.317+0000 7f12ceffd640 1 -- 172.21.15.120:0/2112036751 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f12e8030470 con 0x55c9916a1370 2024-09-19T09:09:11.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.317+0000 7f12f7e79740 10 client.17604 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:11.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.317+0000 7f12f7e79740 10 client.17604 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:09:11.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.317+0000 7f12f7e79740 20 client.17604 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:11.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.317+0000 7f12f7e79740 1 --2- 172.21.15.120:0/2112036751 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55c99171bf90 0x55c99171e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:11.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.317+0000 7f12f7e79740 4 client.17604 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:09:11.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.317+0000 7f12f7e79740 1 -- 172.21.15.120:0/2112036751 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c991603d70 con 0x55c99171bf90 2024-09-19T09:09:11.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.317+0000 7f12ef7fe640 1 --2- 172.21.15.120:0/2112036751 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55c99171bf90 0x55c99171e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:11.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.317+0000 7f12ceffd640 10 client.17604 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:09:11.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.317+0000 7f12ef7fe640 1 --2- 172.21.15.120:0/2112036751 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55c99171bf90 0x55c99171e430 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fc458d5c83750724 server_cookie=895bb4c8a20733e5 in_seq=0 out_seq=0 2024-09-19T09:09:11.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.317+0000 7f12ceffd640 1 -- 172.21.15.120:0/2112036751 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c991603d70 con 0x55c99171bf90 2024-09-19T09:09:11.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.317+0000 7f12ceffd640 10 client.17604 handle_command_reply: tid=0 2024-09-19T09:09:11.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.318+0000 7f12f7e79740 1 client.17604 shutdown 2024-09-19T09:09:11.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.318+0000 7f12f7e79740 1 -- 172.21.15.120:0/2112036751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9916a1370 msgr2=0x55c9916a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:11.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.318+0000 7f12f7e79740 1 --2- 172.21.15.120:0/2112036751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9916a1370 0x55c9916a5240 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f12e800a010 tx=0x7f12e8009ca0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.318+0000 7f12f7e79740 1 -- 172.21.15.120:0/2112036751 shutdown_connections 2024-09-19T09:09:11.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.318+0000 7f12f7e79740 1 --2- 172.21.15.120:0/2112036751 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55c99171bf90 0x55c99171e430 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.318+0000 7f12f7e79740 1 --2- 172.21.15.120:0/2112036751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9916a1370 0x55c9916a5240 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.318+0000 7f12f7e79740 1 --2- 172.21.15.120:0/2112036751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c991697b40 0x55c9916a4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.318+0000 7f12f7e79740 1 --2- 172.21.15.120:0/2112036751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9916965d0 0x55c9916a99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.318+0000 7f12f7e79740 1 -- 172.21.15.120:0/2112036751 >> 172.21.15.120:0/2112036751 conn(0x55c99165f310 msgr2=0x55c9916a00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:11.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.318+0000 7f12ceffd640 0 client.17604 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:09:11.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.318+0000 7f12f7e79740 1 -- 172.21.15.120:0/2112036751 shutdown_connections 2024-09-19T09:09:11.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.318+0000 7f12f7e79740 1 -- 172.21.15.120:0/2112036751 wait complete. 2024-09-19T09:09:11.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.318+0000 7f12f7e79740 20 client.17604 trim_cache size 0 max 16384 2024-09-19T09:09:11.326 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:09:11.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.318+0000 7f12f5aa0640 1 -- 172.21.15.120:0/4060267847 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f12d80a3240 msgr2=0x7f12d80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:11.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.318+0000 7f12f5aa0640 1 --2- 172.21.15.120:0/4060267847 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f12d80a3240 0x7f12d80a5700 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f12e801b000 tx=0x7f12e801d000 comp rx=0 tx=0).stop 2024-09-19T09:09:11.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.318+0000 7f12f5aa0640 1 -- 172.21.15.120:0/4060267847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12f0101de0 msgr2=0x7f12f0198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:11.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.318+0000 7f12f5aa0640 1 --2- 172.21.15.120:0/4060267847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12f0101de0 0x7f12f0198420 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f12e0004770 tx=0x7f12e0003c10 comp rx=0 tx=0).stop 2024-09-19T09:09:11.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.319+0000 7f12f5aa0640 1 -- 172.21.15.120:0/4060267847 shutdown_connections 2024-09-19T09:09:11.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.319+0000 7f12f5aa0640 1 --2- 172.21.15.120:0/4060267847 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f12d80a3240 0x7f12d80a5700 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.319+0000 7f12f5aa0640 1 --2- 172.21.15.120:0/4060267847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12f0106f10 0x7f12f019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.319+0000 7f12f5aa0640 1 --2- 172.21.15.120:0/4060267847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12f0102790 0x7f12f0198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.319+0000 7f12f5aa0640 1 --2- 172.21.15.120:0/4060267847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12f0101de0 0x7f12f0198420 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:11.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.319+0000 7f12f5aa0640 1 -- 172.21.15.120:0/4060267847 >> 172.21.15.120:0/4060267847 conn(0x7f12f00fd8e0 msgr2=0x7f12f00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:11.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.319+0000 7f12f5aa0640 1 -- 172.21.15.120:0/4060267847 shutdown_connections 2024-09-19T09:09:11.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:11.319+0000 7f12f5aa0640 1 -- 172.21.15.120:0/4060267847 wait complete. 2024-09-19T09:09:12.052 INFO:tasks.workunit.client.0.smithi120.stdout: 1 547993 9.32 MB/sec execute 187 sec latency 17.853 ms 2024-09-19T09:09:12.057 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:09:12.313 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:09:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.738+0000 7f8f69027640 1 -- 172.21.15.120:0/4088310103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f64100a40 msgr2=0x7f8f64100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.738+0000 7f8f69027640 1 --2- 172.21.15.120:0/4088310103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f64100a40 0x7f8f64100e40 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009ca0 tx=0x7f8f5c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.738+0000 7f8f69027640 1 -- 172.21.15.120:0/4088310103 shutdown_connections 2024-09-19T09:09:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.738+0000 7f8f69027640 1 --2- 172.21.15.120:0/4088310103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f641025f0 0x7f8f64108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.738+0000 7f8f69027640 1 --2- 172.21.15.120:0/4088310103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f64101c30 0x7f8f641020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.738+0000 7f8f69027640 1 --2- 172.21.15.120:0/4088310103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f64100a40 0x7f8f64100e40 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.738+0000 7f8f69027640 1 -- 172.21.15.120:0/4088310103 >> 172.21.15.120:0/4088310103 conn(0x7f8f640fc1e0 msgr2=0x7f8f640fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.738+0000 7f8f69027640 1 -- 172.21.15.120:0/4088310103 shutdown_connections 2024-09-19T09:09:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.739+0000 7f8f69027640 1 -- 172.21.15.120:0/4088310103 wait complete. 2024-09-19T09:09:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.739+0000 7f8f69027640 1 Processor -- start 2024-09-19T09:09:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.739+0000 7f8f69027640 1 -- start start 2024-09-19T09:09:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.740+0000 7f8f69027640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f64100a40 0x7f8f64197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.740+0000 7f8f69027640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f64101c30 0x7f8f641983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.740+0000 7f8f69027640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f641025f0 0x7f8f6419f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.740+0000 7f8f69027640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6410c6f0 con 0x7f8f64100a40 2024-09-19T09:09:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.740+0000 7f8f69027640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6410c570 con 0x7f8f641025f0 2024-09-19T09:09:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.740+0000 7f8f69027640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6410c870 con 0x7f8f64101c30 2024-09-19T09:09:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.740+0000 7f8f62d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f64100a40 0x7f8f64197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.740+0000 7f8f6359e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f641025f0 0x7f8f6419f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.740+0000 7f8f62d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f64100a40 0x7f8f64197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47856/0 (socket says 172.21.15.120:47856) 2024-09-19T09:09:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.740+0000 7f8f62d9d640 1 -- 172.21.15.120:0/2741144019 learned_addr learned my addr 172.21.15.120:0/2741144019 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.740+0000 7f8f6259c640 1 --2- 172.21.15.120:0/2741144019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f64101c30 0x7f8f641983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.740+0000 7f8f6359e640 1 -- 172.21.15.120:0/2741144019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f64101c30 msgr2=0x7f8f641983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.740+0000 7f8f6359e640 1 --2- 172.21.15.120:0/2741144019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f64101c30 0x7f8f641983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.740+0000 7f8f6359e640 1 -- 172.21.15.120:0/2741144019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f64100a40 msgr2=0x7f8f64197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.740+0000 7f8f6359e640 1 --2- 172.21.15.120:0/2741144019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f64100a40 0x7f8f64197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.740+0000 7f8f6359e640 1 -- 172.21.15.120:0/2741144019 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6419fa90 con 0x7f8f641025f0 2024-09-19T09:09:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.740+0000 7f8f6359e640 1 --2- 172.21.15.120:0/2741144019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f641025f0 0x7f8f6419f420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400e970 tx=0x7f8f5400ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.741+0000 7f8f4bfff640 1 -- 172.21.15.120:0/2741144019 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5400ccc0 con 0x7f8f641025f0 2024-09-19T09:09:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.741+0000 7f8f69027640 1 -- 172.21.15.120:0/2741144019 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f6419fd80 con 0x7f8f641025f0 2024-09-19T09:09:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.741+0000 7f8f4bfff640 1 -- 172.21.15.120:0/2741144019 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f54005020 con 0x7f8f641025f0 2024-09-19T09:09:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.741+0000 7f8f4bfff640 1 -- 172.21.15.120:0/2741144019 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54004a00 con 0x7f8f641025f0 2024-09-19T09:09:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.741+0000 7f8f69027640 1 -- 172.21.15.120:0/2741144019 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f641a0270 con 0x7f8f641025f0 2024-09-19T09:09:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.743+0000 7f8f4bfff640 1 -- 172.21.15.120:0/2741144019 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f8f540051d0 con 0x7f8f641025f0 2024-09-19T09:09:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.743+0000 7f8f6bdc7740 1 Processor -- start 2024-09-19T09:09:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.743+0000 7f8f6bdc7740 1 -- start start 2024-09-19T09:09:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.743+0000 7f8f6bdc7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574b63819a0 0x5574b6381da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.743+0000 7f8f6bdc7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574b63822e0 0x5574b6386da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.743+0000 7f8f6bdc7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574b6387370 0x5574b6389830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.743+0000 7f8f6bdc7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5574b62e52f0 con 0x5574b6387370 2024-09-19T09:09:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.743+0000 7f8f6bdc7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5574b628d270 con 0x5574b63822e0 2024-09-19T09:09:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.743+0000 7f8f6bdc7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5574b62e12b0 con 0x5574b63819a0 2024-09-19T09:09:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.743+0000 7f8f62d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574b63822e0 0x5574b6386da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.743+0000 7f8f62d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574b63822e0 0x5574b6386da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38362/0 (socket says 172.21.15.120:38362) 2024-09-19T09:09:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.743+0000 7f8f62d9d640 1 -- 172.21.15.120:0/1401627245 learned_addr learned my addr 172.21.15.120:0/1401627245 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.743+0000 7f8f6259c640 1 --2- 172.21.15.120:0/1401627245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574b6387370 0x5574b6389830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.744+0000 7f8f6259c640 1 --2- 172.21.15.120:0/1401627245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574b63819a0 0x5574b6381da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.744+0000 7f8f4bfff640 1 --2- 172.21.15.120:0/2741144019 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8f4c0a32f0 0x7f8f4c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.744+0000 7f8f6359e640 1 --2- 172.21.15.120:0/2741144019 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8f4c0a32f0 0x7f8f4c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.744+0000 7f8f6259c640 1 -- 172.21.15.120:0/1401627245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574b63819a0 msgr2=0x5574b6381da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.744+0000 7f8f6259c640 1 --2- 172.21.15.120:0/1401627245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574b63819a0 0x5574b6381da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.744+0000 7f8f6259c640 1 -- 172.21.15.120:0/1401627245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574b63822e0 msgr2=0x5574b6386da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.744+0000 7f8f6259c640 1 --2- 172.21.15.120:0/1401627245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574b63822e0 0x5574b6386da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.744+0000 7f8f6259c640 1 -- 172.21.15.120:0/1401627245 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574b628c340 con 0x5574b6387370 2024-09-19T09:09:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.744+0000 7f8f62d9d640 1 --2- 172.21.15.120:0/1401627245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574b63822e0 0x5574b6386da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.744+0000 7f8f4bfff640 1 -- 172.21.15.120:0/2741144019 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f8f540cf880 con 0x7f8f641025f0 2024-09-19T09:09:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.744+0000 7f8f6259c640 1 --2- 172.21.15.120:0/1401627245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574b6387370 0x5574b6389830 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009730 tx=0x7f8f580096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8d8244a092bded server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.744+0000 7f8f497fa640 1 -- 172.21.15.120:0/1401627245 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f58004050 con 0x5574b6387370 2024-09-19T09:09:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.744+0000 7f8f6359e640 1 --2- 172.21.15.120:0/2741144019 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8f4c0a32f0 0x7f8f4c0a57b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8f54098300 tx=0x7f8f54004590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.744+0000 7f8f497fa640 1 -- 172.21.15.120:0/1401627245 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f58005500 con 0x5574b6387370 2024-09-19T09:09:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.744+0000 7f8f497fa640 1 -- 172.21.15.120:0/1401627245 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f58004d90 con 0x5574b6387370 2024-09-19T09:09:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.745+0000 7f8f6bdc7740 1 -- 172.21.15.120:0/1401627245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574b6387370 msgr2=0x5574b6389830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.745+0000 7f8f6bdc7740 1 --2- 172.21.15.120:0/1401627245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574b6387370 0x5574b6389830 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009730 tx=0x7f8f580096c0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.745+0000 7f8f6bdc7740 1 -- 172.21.15.120:0/1401627245 shutdown_connections 2024-09-19T09:09:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.745+0000 7f8f6bdc7740 1 --2- 172.21.15.120:0/1401627245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574b6387370 0x5574b6389830 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.745+0000 7f8f6bdc7740 1 --2- 172.21.15.120:0/1401627245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574b63822e0 0x5574b6386da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.745+0000 7f8f6bdc7740 1 --2- 172.21.15.120:0/1401627245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574b63819a0 0x5574b6381da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.745+0000 7f8f6bdc7740 1 -- 172.21.15.120:0/1401627245 >> 172.21.15.120:0/1401627245 conn(0x5574b637d010 msgr2=0x5574b637d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.745+0000 7f8f6bdc7740 1 -- 172.21.15.120:0/1401627245 shutdown_connections 2024-09-19T09:09:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.745+0000 7f8f6bdc7740 1 -- 172.21.15.120:0/1401627245 wait complete. 2024-09-19T09:09:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.745+0000 7f8f6bdc7740 1 -- start start 2024-09-19T09:09:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.745+0000 7f8f6bdc7740 1 Processor -- start 2024-09-19T09:09:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.745+0000 7f8f6bdc7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574b63819a0 0x5574b638bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.745+0000 7f8f6bdc7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574b63822e0 0x5574b638c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.746+0000 7f8f6bdc7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574b6387370 0x5574b63931a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.746+0000 7f8f6bdc7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5574b638a340 con 0x5574b6387370 2024-09-19T09:09:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.746+0000 7f8f6bdc7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5574b626e290 con 0x5574b63819a0 2024-09-19T09:09:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.746+0000 7f8f6bdc7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5574b638a4c0 con 0x5574b63822e0 2024-09-19T09:09:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.746+0000 7f8f6bdc7740 10 client.0 _ll_register_callbacks cb 0x5574b629f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.746+0000 7f8f6259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574b63819a0 0x5574b638bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.746+0000 7f8f62d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574b63822e0 0x5574b638c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.746+0000 7f8f6259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574b6387370 0x5574b63931a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.746+0000 7f8f62d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574b63822e0 0x5574b638c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:59500/0 (socket says 172.21.15.120:59500) 2024-09-19T09:09:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.746+0000 7f8f62d9d640 1 -- 172.21.15.120:0/3172020752 learned_addr learned my addr 172.21.15.120:0/3172020752 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.746+0000 7f8f6259c640 1 -- 172.21.15.120:0/3172020752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574b63822e0 msgr2=0x5574b638c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.746+0000 7f8f6259c640 1 --2- 172.21.15.120:0/3172020752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574b63822e0 0x5574b638c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.746+0000 7f8f6259c640 1 -- 172.21.15.120:0/3172020752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574b63819a0 msgr2=0x5574b638bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.746+0000 7f8f6259c640 1 --2- 172.21.15.120:0/3172020752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574b63819a0 0x5574b638bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.746+0000 7f8f6259c640 1 -- 172.21.15.120:0/3172020752 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574b5ff8210 con 0x5574b6387370 2024-09-19T09:09:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.746+0000 7f8f6259c640 1 --2- 172.21.15.120:0/3172020752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574b6387370 0x5574b63931a0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x5574b638b150 tx=0x7f8f58005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c088fc3bc4382f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.746+0000 7f8f46ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:09:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.746+0000 7f8f46ff5640 1 -- 172.21.15.120:0/3172020752 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f58023030 con 0x5574b6387370 2024-09-19T09:09:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.746+0000 7f8f46ff5640 1 -- 172.21.15.120:0/3172020752 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f8f58005240 con 0x5574b6387370 2024-09-19T09:09:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.746+0000 7f8f6bdc7740 1 -- 172.21.15.120:0/3172020752 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f58005500 con 0x5574b6387370 2024-09-19T09:09:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.746+0000 7f8f46ff5640 1 -- 172.21.15.120:0/3172020752 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8f58035050 con 0x5574b6387370 2024-09-19T09:09:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.746+0000 7f8f6bdc7740 10 client.17616 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.746+0000 7f8f6bdc7740 1 -- 172.21.15.120:0/3172020752 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5574b60c4620 con 0x5574b6387370 2024-09-19T09:09:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.747+0000 7f8f46ff5640 1 -- 172.21.15.120:0/3172020752 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f58005500 con 0x5574b6387370 2024-09-19T09:09:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.747+0000 7f8f46ff5640 1 -- 172.21.15.120:0/3172020752 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8f5801f820 con 0x5574b6387370 2024-09-19T09:09:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.747+0000 7f8f6bdc7740 10 client.17616 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.747+0000 7f8f6bdc7740 10 client.17616 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:09:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.747+0000 7f8f6bdc7740 20 client.17616 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.747+0000 7f8f6bdc7740 1 --2- 172.21.15.120:0/3172020752 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5574b63975c0 0x5574b63999b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.747+0000 7f8f6bdc7740 4 client.17616 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:09:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.747+0000 7f8f6bdc7740 1 -- 172.21.15.120:0/3172020752 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574b62f2a20 con 0x5574b63975c0 2024-09-19T09:09:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.747+0000 7f8f62d9d640 1 --2- 172.21.15.120:0/3172020752 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5574b63975c0 0x5574b63999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.748+0000 7f8f46ff5640 10 client.17616 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.748+0000 7f8f62d9d640 1 --2- 172.21.15.120:0/3172020752 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5574b63975c0 0x5574b63999b0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70bc882e1030a32e server_cookie=18832689b2a21cb0 in_seq=0 out_seq=0 2024-09-19T09:09:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.748+0000 7f8f46ff5640 1 -- 172.21.15.120:0/3172020752 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5574b62f2a20 con 0x5574b63975c0 2024-09-19T09:09:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.748+0000 7f8f46ff5640 10 client.17616 handle_command_reply: tid=0 2024-09-19T09:09:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.748+0000 7f8f6bdc7740 1 client.17616 shutdown 2024-09-19T09:09:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.748+0000 7f8f6bdc7740 1 -- 172.21.15.120:0/3172020752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574b6387370 msgr2=0x5574b63931a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.748+0000 7f8f6bdc7740 1 --2- 172.21.15.120:0/3172020752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574b6387370 0x5574b63931a0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x5574b638b150 tx=0x7f8f58005f90 comp rx=0 tx=0).stop 2024-09-19T09:09:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.748+0000 7f8f6bdc7740 1 -- 172.21.15.120:0/3172020752 shutdown_connections 2024-09-19T09:09:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.748+0000 7f8f6bdc7740 1 --2- 172.21.15.120:0/3172020752 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5574b63975c0 0x5574b63999b0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.748+0000 7f8f6bdc7740 1 --2- 172.21.15.120:0/3172020752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574b6387370 0x5574b63931a0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.748+0000 7f8f46ff5640 0 client.17616 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.748+0000 7f8f6bdc7740 1 --2- 172.21.15.120:0/3172020752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574b63822e0 0x5574b638c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.748+0000 7f8f6bdc7740 1 --2- 172.21.15.120:0/3172020752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574b63819a0 0x5574b638bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.748+0000 7f8f6bdc7740 1 -- 172.21.15.120:0/3172020752 >> 172.21.15.120:0/3172020752 conn(0x5574b637d010 msgr2=0x5574b637e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.748+0000 7f8f6bdc7740 1 -- 172.21.15.120:0/3172020752 shutdown_connections 2024-09-19T09:09:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.748+0000 7f8f6bdc7740 1 -- 172.21.15.120:0/3172020752 wait complete. 2024-09-19T09:09:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.748+0000 7f8f6bdc7740 20 client.17616 trim_cache size 0 max 16384 2024-09-19T09:09:12.758 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:12 smithi120 ceph-mon[31150]: pgmap v532: 97 pgs: 97 active+clean; 30 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 470 op/s 2024-09-19T09:09:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.768+0000 7f8f6bdc7740 1 Processor -- start 2024-09-19T09:09:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.768+0000 7f8f6bdc7740 1 -- start start 2024-09-19T09:09:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574b63819a0 0x5574b6381d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574b63822e0 0x5574b6393160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574b6387370 0x5574b637b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8f641a90b0 con 0x5574b6387370 2024-09-19T09:09:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8f641aabf0 con 0x5574b63822e0 2024-09-19T09:09:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8f641a9780 con 0x5574b63819a0 2024-09-19T09:09:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574b6387370 0x5574b637b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f62d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574b63822e0 0x5574b6393160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574b6387370 0x5574b637b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47878/0 (socket says 172.21.15.120:47878) 2024-09-19T09:09:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f62d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574b63822e0 0x5574b6393160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38384/0 (socket says 172.21.15.120:38384) 2024-09-19T09:09:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6259c640 1 -- 172.21.15.120:0/2222762256 learned_addr learned my addr 172.21.15.120:0/2222762256 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6259c640 1 -- 172.21.15.120:0/2222762256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574b63819a0 msgr2=0x5574b6381d80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:09:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6259c640 1 --2- 172.21.15.120:0/2222762256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574b63819a0 0x5574b6381d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6259c640 1 -- 172.21.15.120:0/2222762256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574b63822e0 msgr2=0x5574b6393160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6259c640 1 --2- 172.21.15.120:0/2222762256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574b63822e0 0x5574b6393160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6259c640 1 -- 172.21.15.120:0/2222762256 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574b637ce30 con 0x5574b6387370 2024-09-19T09:09:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f62d9d640 1 --2- 172.21.15.120:0/2222762256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574b63822e0 0x5574b6393160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6259c640 1 --2- 172.21.15.120:0/2222762256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574b6387370 0x5574b637b9a0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f8f58000c00 tx=0x7f8f58009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06f909c17426343 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f48ff9640 1 -- 172.21.15.120:0/2222762256 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f58023040 con 0x5574b6387370 2024-09-19T09:09:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f48ff9640 1 -- 172.21.15.120:0/2222762256 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f58005a70 con 0x5574b6387370 2024-09-19T09:09:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f48ff9640 1 -- 172.21.15.120:0/2222762256 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f58026070 con 0x5574b6387370 2024-09-19T09:09:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 1 -- 172.21.15.120:0/2222762256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574b6387370 msgr2=0x5574b637b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 1 --2- 172.21.15.120:0/2222762256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574b6387370 0x5574b637b9a0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f8f58000c00 tx=0x7f8f58009f90 comp rx=0 tx=0).stop 2024-09-19T09:09:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 1 -- 172.21.15.120:0/2222762256 shutdown_connections 2024-09-19T09:09:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 1 --2- 172.21.15.120:0/2222762256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574b6387370 0x5574b637b9a0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 1 --2- 172.21.15.120:0/2222762256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574b63822e0 0x5574b6393160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 1 --2- 172.21.15.120:0/2222762256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574b63819a0 0x5574b6381d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 1 -- 172.21.15.120:0/2222762256 >> 172.21.15.120:0/2222762256 conn(0x5574b637d010 msgr2=0x5574b638bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 1 -- 172.21.15.120:0/2222762256 shutdown_connections 2024-09-19T09:09:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 1 -- 172.21.15.120:0/2222762256 wait complete. 2024-09-19T09:09:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 1 -- start start 2024-09-19T09:09:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 1 Processor -- start 2024-09-19T09:09:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574b63819a0 0x5574b638dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574b63822e0 0x5574b638e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574b6387370 0x5574b64045d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8f641aa520 con 0x5574b6387370 2024-09-19T09:09:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8f641a9e50 con 0x5574b63819a0 2024-09-19T09:09:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8f641a7070 con 0x5574b63822e0 2024-09-19T09:09:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 10 client.0 _ll_register_callbacks cb 0x7f8f58005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574b6387370 0x5574b64045d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574b6387370 0x5574b64045d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47884/0 (socket says 172.21.15.120:47884) 2024-09-19T09:09:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6259c640 1 -- 172.21.15.120:0/732341247 learned_addr learned my addr 172.21.15.120:0/732341247 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f62d9d640 1 --2- 172.21.15.120:0/732341247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574b63822e0 0x5574b638e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6259c640 1 -- 172.21.15.120:0/732341247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574b63822e0 msgr2=0x5574b638e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6259c640 1 --2- 172.21.15.120:0/732341247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574b63822e0 0x5574b638e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6259c640 1 -- 172.21.15.120:0/732341247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574b63819a0 msgr2=0x5574b638dd50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:09:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6259c640 1 --2- 172.21.15.120:0/732341247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574b63819a0 0x5574b638dd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6259c640 1 -- 172.21.15.120:0/732341247 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574b6393390 con 0x5574b6387370 2024-09-19T09:09:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6259c640 1 --2- 172.21.15.120:0/732341247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574b6387370 0x5574b64045d0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f8f580059f0 tx=0x7f8f58009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b3176aacc2f85d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f46ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:09:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f46ff5640 1 -- 172.21.15.120:0/732341247 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f58023070 con 0x5574b6387370 2024-09-19T09:09:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f46ff5640 1 -- 172.21.15.120:0/732341247 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f8f580099f0 con 0x5574b6387370 2024-09-19T09:09:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 1 -- 172.21.15.120:0/732341247 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f58005a70 con 0x5574b6387370 2024-09-19T09:09:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f46ff5640 1 -- 172.21.15.120:0/732341247 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8f58026040 con 0x5574b6387370 2024-09-19T09:09:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 10 client.17622 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 1 -- 172.21.15.120:0/732341247 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5574b6405b10 con 0x5574b6387370 2024-09-19T09:09:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f46ff5640 1 -- 172.21.15.120:0/732341247 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f58005a70 con 0x5574b6387370 2024-09-19T09:09:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f46ff5640 1 -- 172.21.15.120:0/732341247 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8f5804b030 con 0x5574b6387370 2024-09-19T09:09:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 10 client.17622 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 10 client.17622 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:09:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 20 client.17622 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 1 --2- 172.21.15.120:0/732341247 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5574b64063c0 0x5574b64087e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 4 client.17622 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:09:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.769+0000 7f8f6bdc7740 1 -- 172.21.15.120:0/732341247 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5574b62ecd70 con 0x5574b64063c0 2024-09-19T09:09:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.778+0000 7f8f62d9d640 1 --2- 172.21.15.120:0/732341247 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5574b64063c0 0x5574b64087e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.778+0000 7f8f62d9d640 1 --2- 172.21.15.120:0/732341247 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5574b64063c0 0x5574b64087e0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48b4bbb9804f814c server_cookie=f497c2e1dc9bc85b in_seq=0 out_seq=0 2024-09-19T09:09:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.778+0000 7f8f46ff5640 10 client.17622 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:12.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:12 smithi196 ceph-mon[35150]: pgmap v532: 97 pgs: 97 active+clean; 30 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 470 op/s 2024-09-19T09:09:12.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:12 smithi153 ceph-mon[35071]: pgmap v532: 97 pgs: 97 active+clean; 30 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 470 op/s 2024-09-19T09:09:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.958+0000 7f8f46ff5640 1 -- 172.21.15.120:0/732341247 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5574b62ecd70 con 0x5574b64063c0 2024-09-19T09:09:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.958+0000 7f8f46ff5640 10 client.17622 handle_command_reply: tid=0 2024-09-19T09:09:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.958+0000 7f8f6bdc7740 1 client.17622 shutdown 2024-09-19T09:09:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.958+0000 7f8f6bdc7740 1 -- 172.21.15.120:0/732341247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574b6387370 msgr2=0x5574b64045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.958+0000 7f8f6bdc7740 1 --2- 172.21.15.120:0/732341247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574b6387370 0x5574b64045d0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f8f580059f0 tx=0x7f8f58009880 comp rx=0 tx=0).stop 2024-09-19T09:09:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.958+0000 7f8f6bdc7740 1 -- 172.21.15.120:0/732341247 shutdown_connections 2024-09-19T09:09:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.958+0000 7f8f6bdc7740 1 --2- 172.21.15.120:0/732341247 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5574b64063c0 0x5574b64087e0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.958+0000 7f8f6bdc7740 1 --2- 172.21.15.120:0/732341247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574b6387370 0x5574b64045d0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.958+0000 7f8f6bdc7740 1 --2- 172.21.15.120:0/732341247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574b63822e0 0x5574b638e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.958+0000 7f8f6bdc7740 1 --2- 172.21.15.120:0/732341247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574b63819a0 0x5574b638dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.958+0000 7f8f46ff5640 0 client.17622 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.958+0000 7f8f6bdc7740 1 -- 172.21.15.120:0/732341247 >> 172.21.15.120:0/732341247 conn(0x5574b637d010 msgr2=0x5574b637b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.958+0000 7f8f6bdc7740 1 -- 172.21.15.120:0/732341247 shutdown_connections 2024-09-19T09:09:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.958+0000 7f8f6bdc7740 1 -- 172.21.15.120:0/732341247 wait complete. 2024-09-19T09:09:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.958+0000 7f8f6bdc7740 20 client.17622 trim_cache size 0 max 16384 2024-09-19T09:09:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:09:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:09:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:09:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:09:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.959+0000 7f8f69027640 1 -- 172.21.15.120:0/2741144019 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8f4c0a32f0 msgr2=0x7f8f4c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.959+0000 7f8f69027640 1 --2- 172.21.15.120:0/2741144019 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8f4c0a32f0 0x7f8f4c0a57b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8f54098300 tx=0x7f8f54004590 comp rx=0 tx=0).stop 2024-09-19T09:09:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.959+0000 7f8f69027640 1 -- 172.21.15.120:0/2741144019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f641025f0 msgr2=0x7f8f6419f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.959+0000 7f8f69027640 1 --2- 172.21.15.120:0/2741144019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f641025f0 0x7f8f6419f420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400e970 tx=0x7f8f5400ee30 comp rx=0 tx=0).stop 2024-09-19T09:09:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.959+0000 7f8f69027640 1 -- 172.21.15.120:0/2741144019 shutdown_connections 2024-09-19T09:09:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.959+0000 7f8f69027640 1 --2- 172.21.15.120:0/2741144019 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8f4c0a32f0 0x7f8f4c0a57b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.959+0000 7f8f69027640 1 --2- 172.21.15.120:0/2741144019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f641025f0 0x7f8f6419f420 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.959+0000 7f8f69027640 1 --2- 172.21.15.120:0/2741144019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f64101c30 0x7f8f641983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.959+0000 7f8f69027640 1 --2- 172.21.15.120:0/2741144019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f64100a40 0x7f8f64197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.959+0000 7f8f69027640 1 -- 172.21.15.120:0/2741144019 >> 172.21.15.120:0/2741144019 conn(0x7f8f640fc1e0 msgr2=0x7f8f640fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.959+0000 7f8f69027640 1 -- 172.21.15.120:0/2741144019 shutdown_connections 2024-09-19T09:09:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:12.959+0000 7f8f69027640 1 -- 172.21.15.120:0/2741144019 wait complete. 2024-09-19T09:09:13.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.995+0000 7f0c2abbe640 1 -- 172.21.15.120:0/2211968455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c24105760 msgr2=0x7f0c24105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:13.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.995+0000 7f0c2abbe640 1 --2- 172.21.15.120:0/2211968455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c24105760 0x7f0c24105b40 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f0c18009900 tx=0x7f0c1802a480 comp rx=0 tx=0).stop 2024-09-19T09:09:13.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.995+0000 7f0c2abbe640 1 -- 172.21.15.120:0/2211968455 shutdown_connections 2024-09-19T09:09:13.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.995+0000 7f0c2abbe640 1 --2- 172.21.15.120:0/2211968455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c24112b10 0x7f0c24114f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.995+0000 7f0c2abbe640 1 --2- 172.21.15.120:0/2211968455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c24106080 0x7f0c241125d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.995+0000 7f0c2abbe640 1 --2- 172.21.15.120:0/2211968455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c24105760 0x7f0c24105b40 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.995+0000 7f0c2abbe640 1 -- 172.21.15.120:0/2211968455 >> 172.21.15.120:0/2211968455 conn(0x7f0c24077210 msgr2=0x7f0c24077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:13.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.995+0000 7f0c2abbe640 1 -- 172.21.15.120:0/2211968455 shutdown_connections 2024-09-19T09:09:13.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.995+0000 7f0c2abbe640 1 -- 172.21.15.120:0/2211968455 wait complete. 2024-09-19T09:09:13.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.996+0000 7f0c2abbe640 1 Processor -- start 2024-09-19T09:09:13.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.996+0000 7f0c2abbe640 1 -- start start 2024-09-19T09:09:13.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.996+0000 7f0c2abbe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c24105760 0x7f0c241a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:13.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.996+0000 7f0c2abbe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c24106080 0x7f0c241a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:13.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.996+0000 7f0c2abbe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c24112b10 0x7f0c2419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:13.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.996+0000 7f0c2abbe640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0c24118630 con 0x7f0c24112b10 2024-09-19T09:09:13.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.996+0000 7f0c2abbe640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0c241184b0 con 0x7f0c24105760 2024-09-19T09:09:13.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.996+0000 7f0c2abbe640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0c241187b0 con 0x7f0c24106080 2024-09-19T09:09:13.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.996+0000 7f0c2915b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c24112b10 0x7f0c2419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:13.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.996+0000 7f0c2895a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c24105760 0x7f0c241a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:13.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.996+0000 7f0c2915b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c24112b10 0x7f0c2419e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47896/0 (socket says 172.21.15.120:47896) 2024-09-19T09:09:13.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.996+0000 7f0c2915b640 1 -- 172.21.15.120:0/1222931028 learned_addr learned my addr 172.21.15.120:0/1222931028 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:13.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.996+0000 7f0c23fff640 1 --2- 172.21.15.120:0/1222931028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c24106080 0x7f0c241a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:13.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.996+0000 7f0c2915b640 1 -- 172.21.15.120:0/1222931028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c24106080 msgr2=0x7f0c241a4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:13.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.996+0000 7f0c2915b640 1 --2- 172.21.15.120:0/1222931028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c24106080 0x7f0c241a4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.996+0000 7f0c2915b640 1 -- 172.21.15.120:0/1222931028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c24105760 msgr2=0x7f0c241a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:13.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.996+0000 7f0c2915b640 1 --2- 172.21.15.120:0/1222931028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c24105760 0x7f0c241a47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.996+0000 7f0c2915b640 1 -- 172.21.15.120:0/1222931028 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2419f090 con 0x7f0c24112b10 2024-09-19T09:09:13.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.997+0000 7f0c2915b640 1 --2- 172.21.15.120:0/1222931028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c24112b10 0x7f0c2419e960 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00eb40 tx=0x7f0c1c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:13.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.997+0000 7f0c21ffb640 1 -- 172.21.15.120:0/1222931028 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c004b20 con 0x7f0c24112b10 2024-09-19T09:09:13.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.997+0000 7f0c2abbe640 1 -- 172.21.15.120:0/1222931028 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c2419f380 con 0x7f0c24112b10 2024-09-19T09:09:13.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.997+0000 7f0c21ffb640 1 -- 172.21.15.120:0/1222931028 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c1c004cc0 con 0x7f0c24112b10 2024-09-19T09:09:13.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.997+0000 7f0c21ffb640 1 -- 172.21.15.120:0/1222931028 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c010850 con 0x7f0c24112b10 2024-09-19T09:09:13.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.997+0000 7f0c2abbe640 1 -- 172.21.15.120:0/1222931028 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c241ab670 con 0x7f0c24112b10 2024-09-19T09:09:13.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:12.999+0000 7f0c21ffb640 1 -- 172.21.15.120:0/1222931028 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f0c1c005020 con 0x7f0c24112b10 2024-09-19T09:09:13.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.000+0000 7f0c21ffb640 1 --2- 172.21.15.120:0/1222931028 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0c140a3240 0x7f0c140a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:13.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.000+0000 7f0c21ffb640 1 -- 172.21.15.120:0/1222931028 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f0c1c0d0170 con 0x7f0c24112b10 2024-09-19T09:09:13.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.000+0000 7f0c2895a640 1 --2- 172.21.15.120:0/1222931028 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0c140a3240 0x7f0c140a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:13.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.001+0000 7f0c2895a640 1 --2- 172.21.15.120:0/1222931028 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0c140a3240 0x7f0c140a5700 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0c2419faa0 tx=0x7f0c180058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:13.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.001+0000 7f0c2cf97740 1 Processor -- start 2024-09-19T09:09:13.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.001+0000 7f0c2cf97740 1 -- start start 2024-09-19T09:09:13.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.001+0000 7f0c2cf97740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607cab03b40 0x5607cab02090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:13.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.001+0000 7f0c2cf97740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5607cab025d0 0x5607cab02a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:13.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.001+0000 7f0c2cf97740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607cab0d370 0x5607cab0f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:13.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.001+0000 7f0c2cf97740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5607caa682f0 con 0x5607cab03b40 2024-09-19T09:09:13.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.001+0000 7f0c2cf97740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5607caa0fcf0 con 0x5607cab025d0 2024-09-19T09:09:13.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.001+0000 7f0c2cf97740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5607caa642b0 con 0x5607cab0d370 2024-09-19T09:09:13.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.002+0000 7f0c23fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5607cab025d0 0x5607cab02a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:13.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.002+0000 7f0c2895a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607cab03b40 0x5607cab02090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:13.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.002+0000 7f0c2895a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607cab03b40 0x5607cab02090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47910/0 (socket says 172.21.15.120:47910) 2024-09-19T09:09:13.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.002+0000 7f0c23fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5607cab025d0 0x5607cab02a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38410/0 (socket says 172.21.15.120:38410) 2024-09-19T09:09:13.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.002+0000 7f0c2895a640 1 -- 172.21.15.120:0/1982689991 learned_addr learned my addr 172.21.15.120:0/1982689991 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:13.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.002+0000 7f0c2915b640 1 --2- 172.21.15.120:0/1982689991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607cab0d370 0x5607cab0f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:13.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.002+0000 7f0c2915b640 1 -- 172.21.15.120:0/1982689991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5607cab025d0 msgr2=0x5607cab02a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:13.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.002+0000 7f0c2915b640 1 --2- 172.21.15.120:0/1982689991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5607cab025d0 0x5607cab02a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.002+0000 7f0c2915b640 1 -- 172.21.15.120:0/1982689991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607cab03b40 msgr2=0x5607cab02090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:13.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.002+0000 7f0c2915b640 1 --2- 172.21.15.120:0/1982689991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607cab03b40 0x5607cab02090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.002+0000 7f0c2915b640 1 -- 172.21.15.120:0/1982689991 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607ca9f43a0 con 0x5607cab0d370 2024-09-19T09:09:13.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.002+0000 7f0c2895a640 1 --2- 172.21.15.120:0/1982689991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607cab03b40 0x5607cab02090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:13.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.002+0000 7f0c23fff640 1 --2- 172.21.15.120:0/1982689991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5607cab025d0 0x5607cab02a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:13.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.002+0000 7f0c2915b640 1 --2- 172.21.15.120:0/1982689991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607cab0d370 0x5607cab0f760 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c09c490 tx=0x7f0c1c09c690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3db15ccc97c27cd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:13.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.002+0000 7f0c0effd640 1 -- 172.21.15.120:0/1982689991 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c09ce90 con 0x5607cab0d370 2024-09-19T09:09:13.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.003+0000 7f0c0effd640 1 -- 172.21.15.120:0/1982689991 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c1c0b5070 con 0x5607cab0d370 2024-09-19T09:09:13.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.003+0000 7f0c0effd640 1 -- 172.21.15.120:0/1982689991 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c098e00 con 0x5607cab0d370 2024-09-19T09:09:13.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.003+0000 7f0c2cf97740 1 -- 172.21.15.120:0/1982689991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607cab0d370 msgr2=0x5607cab0f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:13.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.003+0000 7f0c2cf97740 1 --2- 172.21.15.120:0/1982689991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607cab0d370 0x5607cab0f760 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c09c490 tx=0x7f0c1c09c690 comp rx=0 tx=0).stop 2024-09-19T09:09:13.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.003+0000 7f0c2cf97740 1 -- 172.21.15.120:0/1982689991 shutdown_connections 2024-09-19T09:09:13.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.003+0000 7f0c2cf97740 1 --2- 172.21.15.120:0/1982689991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607cab0d370 0x5607cab0f760 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.003+0000 7f0c2cf97740 1 --2- 172.21.15.120:0/1982689991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5607cab025d0 0x5607cab02a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.003+0000 7f0c2cf97740 1 --2- 172.21.15.120:0/1982689991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607cab03b40 0x5607cab02090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.003+0000 7f0c2cf97740 1 -- 172.21.15.120:0/1982689991 >> 172.21.15.120:0/1982689991 conn(0x5607caacb310 msgr2=0x5607caafec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:13.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.003+0000 7f0c2cf97740 1 -- 172.21.15.120:0/1982689991 shutdown_connections 2024-09-19T09:09:13.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.004+0000 7f0c2cf97740 1 -- 172.21.15.120:0/1982689991 wait complete. 2024-09-19T09:09:13.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.004+0000 7f0c2cf97740 1 -- start start 2024-09-19T09:09:13.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.004+0000 7f0c2cf97740 1 Processor -- start 2024-09-19T09:09:13.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.005+0000 7f0c2cf97740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5607cab025d0 0x5607cab0ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:13.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.005+0000 7f0c2cf97740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607cab03b40 0x5607cab0a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:13.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.005+0000 7f0c2cf97740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607cab0d370 0x5607cab0a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:13.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.005+0000 7f0c2cf97740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5607cab10070 con 0x5607cab0d370 2024-09-19T09:09:13.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.005+0000 7f0c2cf97740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5607caa23060 con 0x5607cab025d0 2024-09-19T09:09:13.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.005+0000 7f0c2cf97740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5607cab101f0 con 0x5607cab03b40 2024-09-19T09:09:13.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.005+0000 7f0c2895a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5607cab025d0 0x5607cab0ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:13.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.005+0000 7f0c2cf97740 10 client.0 _ll_register_callbacks cb 0x5607ca7768c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:13.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.005+0000 7f0c23fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607cab03b40 0x5607cab0a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:13.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.005+0000 7f0c2895a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5607cab025d0 0x5607cab0ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38412/0 (socket says 172.21.15.120:38412) 2024-09-19T09:09:13.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.005+0000 7f0c2895a640 1 -- 172.21.15.120:0/1946413279 learned_addr learned my addr 172.21.15.120:0/1946413279 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:13.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.005+0000 7f0c2915b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607cab0d370 0x5607cab0a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:13.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.005+0000 7f0c2895a640 1 -- 172.21.15.120:0/1946413279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607cab03b40 msgr2=0x5607cab0a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:13.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.005+0000 7f0c2895a640 1 --2- 172.21.15.120:0/1946413279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607cab03b40 0x5607cab0a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.005+0000 7f0c2895a640 1 -- 172.21.15.120:0/1946413279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607cab0d370 msgr2=0x5607cab0a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:13.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.005+0000 7f0c2895a640 1 --2- 172.21.15.120:0/1946413279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607cab0d370 0x5607cab0a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.006+0000 7f0c2895a640 1 -- 172.21.15.120:0/1946413279 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607ca7d93c0 con 0x5607cab025d0 2024-09-19T09:09:13.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.006+0000 7f0c2915b640 1 --2- 172.21.15.120:0/1946413279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607cab0d370 0x5607cab0a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:13.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.006+0000 7f0c2895a640 1 --2- 172.21.15.120:0/1946413279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5607cab025d0 0x5607cab0ce00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0c18031ab0 tx=0x7f0c18009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f480b3c309348c4d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:13.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.006+0000 7f0c03fff640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:09:13.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.006+0000 7f0c03fff640 1 -- 172.21.15.120:0/1946413279 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c18035040 con 0x5607cab025d0 2024-09-19T09:09:13.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.006+0000 7f0c2cf97740 1 -- 172.21.15.120:0/1946413279 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c1c0b5070 con 0x5607cab025d0 2024-09-19T09:09:13.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.006+0000 7f0c03fff640 1 -- 172.21.15.120:0/1946413279 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f0c180038c0 con 0x5607cab025d0 2024-09-19T09:09:13.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.006+0000 7f0c03fff640 1 -- 172.21.15.120:0/1946413279 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0c18003ac0 con 0x5607cab025d0 2024-09-19T09:09:13.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.006+0000 7f0c2cf97740 10 client.26882 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:13.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.006+0000 7f0c2cf97740 1 -- 172.21.15.120:0/1946413279 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5607cab1bbf0 con 0x5607cab025d0 2024-09-19T09:09:13.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.006+0000 7f0c03fff640 1 -- 172.21.15.120:0/1946413279 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c1c0b5070 con 0x5607cab025d0 2024-09-19T09:09:13.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.006+0000 7f0c03fff640 1 -- 172.21.15.120:0/1946413279 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0c18049420 con 0x5607cab025d0 2024-09-19T09:09:13.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.007+0000 7f0c2cf97740 10 client.26882 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:13.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.007+0000 7f0c2cf97740 10 client.26882 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:09:13.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.007+0000 7f0c2cf97740 20 client.26882 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:13.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.007+0000 7f0c2cf97740 1 --2- 172.21.15.120:0/1946413279 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5607cab1d000 0x5607cab1f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:13.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.007+0000 7f0c2cf97740 4 client.26882 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:09:13.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.007+0000 7f0c2cf97740 1 -- 172.21.15.120:0/1946413279 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607caa75860 con 0x5607cab1d000 2024-09-19T09:09:13.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.007+0000 7f0c23fff640 1 --2- 172.21.15.120:0/1946413279 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5607cab1d000 0x5607cab1f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:13.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.007+0000 7f0c03fff640 10 client.26882 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:09:13.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.007+0000 7f0c23fff640 1 --2- 172.21.15.120:0/1946413279 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5607cab1d000 0x5607cab1f440 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bae20a10b08068cd server_cookie=14a36bae9757c2c3 in_seq=0 out_seq=0 2024-09-19T09:09:13.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.008+0000 7f0c03fff640 1 -- 172.21.15.120:0/1946413279 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5607caa75860 con 0x5607cab1d000 2024-09-19T09:09:13.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.008+0000 7f0c03fff640 10 client.26882 handle_command_reply: tid=0 2024-09-19T09:09:13.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.008+0000 7f0c2cf97740 1 client.26882 shutdown 2024-09-19T09:09:13.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.008+0000 7f0c2cf97740 1 -- 172.21.15.120:0/1946413279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5607cab025d0 msgr2=0x5607cab0ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:13.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.008+0000 7f0c2cf97740 1 --2- 172.21.15.120:0/1946413279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5607cab025d0 0x5607cab0ce00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0c18031ab0 tx=0x7f0c18009260 comp rx=0 tx=0).stop 2024-09-19T09:09:13.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.008+0000 7f0c2cf97740 1 -- 172.21.15.120:0/1946413279 shutdown_connections 2024-09-19T09:09:13.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.008+0000 7f0c2cf97740 1 --2- 172.21.15.120:0/1946413279 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5607cab1d000 0x5607cab1f440 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.008+0000 7f0c2cf97740 1 --2- 172.21.15.120:0/1946413279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607cab0d370 0x5607cab0a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.008+0000 7f0c2cf97740 1 --2- 172.21.15.120:0/1946413279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607cab03b40 0x5607cab0a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.008+0000 7f0c2cf97740 1 --2- 172.21.15.120:0/1946413279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5607cab025d0 0x5607cab0ce00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.008+0000 7f0c03fff640 0 client.26882 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:09:13.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.008+0000 7f0c2cf97740 1 -- 172.21.15.120:0/1946413279 >> 172.21.15.120:0/1946413279 conn(0x5607caacb310 msgr2=0x5607cab04930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:13.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.008+0000 7f0c2cf97740 1 -- 172.21.15.120:0/1946413279 shutdown_connections 2024-09-19T09:09:13.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.008+0000 7f0c2cf97740 1 -- 172.21.15.120:0/1946413279 wait complete. 2024-09-19T09:09:13.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.008+0000 7f0c2cf97740 20 client.26882 trim_cache size 0 max 16384 2024-09-19T09:09:13.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.022+0000 7f0c2cf97740 1 Processor -- start 2024-09-19T09:09:13.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.022+0000 7f0c2cf97740 1 -- start start 2024-09-19T09:09:13.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.023+0000 7f0c2cf97740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5607cab025d0 0x5607cab09970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:13.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.023+0000 7f0c2cf97740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607cab03b40 0x5607cab05120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:13.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.023+0000 7f0c2cf97740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607cab0d370 0x5607cab05660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:13.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.023+0000 7f0c2cf97740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0c241b3cd0 con 0x5607cab0d370 2024-09-19T09:09:13.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.023+0000 7f0c2cf97740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0c241b5810 con 0x5607cab025d0 2024-09-19T09:09:13.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.023+0000 7f0c2cf97740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0c241b43a0 con 0x5607cab03b40 2024-09-19T09:09:13.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.023+0000 7f0c2915b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607cab0d370 0x5607cab05660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:13.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.023+0000 7f0c2915b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607cab0d370 0x5607cab05660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47922/0 (socket says 172.21.15.120:47922) 2024-09-19T09:09:13.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.023+0000 7f0c2915b640 1 -- 172.21.15.120:0/2105858790 learned_addr learned my addr 172.21.15.120:0/2105858790 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:13.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.023+0000 7f0c2895a640 1 --2- 172.21.15.120:0/2105858790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5607cab025d0 0x5607cab09970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:13.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.023+0000 7f0c23fff640 1 --2- 172.21.15.120:0/2105858790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607cab03b40 0x5607cab05120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:13.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.023+0000 7f0c2915b640 1 -- 172.21.15.120:0/2105858790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607cab03b40 msgr2=0x5607cab05120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:13.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.023+0000 7f0c2915b640 1 --2- 172.21.15.120:0/2105858790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607cab03b40 0x5607cab05120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.023+0000 7f0c2915b640 1 -- 172.21.15.120:0/2105858790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5607cab025d0 msgr2=0x5607cab09970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:13.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.023+0000 7f0c2915b640 1 --2- 172.21.15.120:0/2105858790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5607cab025d0 0x5607cab09970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.023+0000 7f0c2915b640 1 -- 172.21.15.120:0/2105858790 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607cab0ca20 con 0x5607cab0d370 2024-09-19T09:09:13.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.023+0000 7f0c2895a640 1 --2- 172.21.15.120:0/2105858790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5607cab025d0 0x5607cab09970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:13.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.023+0000 7f0c2915b640 1 --2- 172.21.15.120:0/2105858790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607cab0d370 0x5607cab05660 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c09c490 tx=0x7f0c1c0870d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc633ddbbd60ad4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:13.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.023+0000 7f0c0e7fc640 1 -- 172.21.15.120:0/2105858790 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c09ce90 con 0x5607cab0d370 2024-09-19T09:09:13.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.023+0000 7f0c0e7fc640 1 -- 172.21.15.120:0/2105858790 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c1c0b9030 con 0x5607cab0d370 2024-09-19T09:09:13.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.024+0000 7f0c0e7fc640 1 -- 172.21.15.120:0/2105858790 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0a51d0 con 0x5607cab0d370 2024-09-19T09:09:13.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.024+0000 7f0c2cf97740 1 -- 172.21.15.120:0/2105858790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607cab0d370 msgr2=0x5607cab05660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:13.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.024+0000 7f0c2cf97740 1 --2- 172.21.15.120:0/2105858790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607cab0d370 0x5607cab05660 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c09c490 tx=0x7f0c1c0870d0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.024+0000 7f0c2cf97740 1 -- 172.21.15.120:0/2105858790 shutdown_connections 2024-09-19T09:09:13.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.024+0000 7f0c2cf97740 1 --2- 172.21.15.120:0/2105858790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607cab0d370 0x5607cab05660 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.024+0000 7f0c2cf97740 1 --2- 172.21.15.120:0/2105858790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607cab03b40 0x5607cab05120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.024+0000 7f0c2cf97740 1 --2- 172.21.15.120:0/2105858790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5607cab025d0 0x5607cab09970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.024+0000 7f0c2cf97740 1 -- 172.21.15.120:0/2105858790 >> 172.21.15.120:0/2105858790 conn(0x5607caacb310 msgr2=0x5607cab091a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:13.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.024+0000 7f0c2cf97740 1 -- 172.21.15.120:0/2105858790 shutdown_connections 2024-09-19T09:09:13.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.024+0000 7f0c2cf97740 1 -- 172.21.15.120:0/2105858790 wait complete. 2024-09-19T09:09:13.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.025+0000 7f0c2cf97740 1 -- start start 2024-09-19T09:09:13.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.025+0000 7f0c2cf97740 1 Processor -- start 2024-09-19T09:09:13.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.025+0000 7f0c2cf97740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607cab025d0 0x5607cab157b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:13.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.025+0000 7f0c2cf97740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5607cab03b40 0x5607cab15cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:13.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.025+0000 7f0c2cf97740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607cab0d370 0x5607cab10c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:13.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.025+0000 7f0c2cf97740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0c241b5140 con 0x5607cab025d0 2024-09-19T09:09:13.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.025+0000 7f0c2cf97740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0c241b4a70 con 0x5607cab03b40 2024-09-19T09:09:13.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.025+0000 7f0c2cf97740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0c241b1c90 con 0x5607cab0d370 2024-09-19T09:09:13.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.025+0000 7f0c2cf97740 10 client.0 _ll_register_callbacks cb 0x7f0c18009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:13.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.025+0000 7f0c2895a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607cab025d0 0x5607cab157b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:13.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.025+0000 7f0c2895a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607cab025d0 0x5607cab157b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47924/0 (socket says 172.21.15.120:47924) 2024-09-19T09:09:13.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.025+0000 7f0c2895a640 1 -- 172.21.15.120:0/2107885958 learned_addr learned my addr 172.21.15.120:0/2107885958 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:13.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.025+0000 7f0c2915b640 1 --2- 172.21.15.120:0/2107885958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607cab0d370 0x5607cab10c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:13.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.026+0000 7f0c23fff640 1 --2- 172.21.15.120:0/2107885958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5607cab03b40 0x5607cab15cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:13.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.026+0000 7f0c2895a640 1 -- 172.21.15.120:0/2107885958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607cab0d370 msgr2=0x5607cab10c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:13.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.026+0000 7f0c2895a640 1 --2- 172.21.15.120:0/2107885958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607cab0d370 0x5607cab10c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.026+0000 7f0c2895a640 1 -- 172.21.15.120:0/2107885958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5607cab03b40 msgr2=0x5607cab15cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:13.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.026+0000 7f0c2895a640 1 --2- 172.21.15.120:0/2107885958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5607cab03b40 0x5607cab15cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.026+0000 7f0c2895a640 1 -- 172.21.15.120:0/2107885958 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607cab11460 con 0x5607cab025d0 2024-09-19T09:09:13.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.026+0000 7f0c2895a640 1 --2- 172.21.15.120:0/2107885958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607cab025d0 0x5607cab157b0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f0c18049810 tx=0x7f0c18031c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7195c5b879ae1aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:13.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.026+0000 7f0c03fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:09:13.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.026+0000 7f0c03fff640 1 -- 172.21.15.120:0/2107885958 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c18035040 con 0x5607cab025d0 2024-09-19T09:09:13.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.026+0000 7f0c03fff640 1 -- 172.21.15.120:0/2107885958 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f0c1805c460 con 0x5607cab025d0 2024-09-19T09:09:13.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.026+0000 7f0c2cf97740 1 -- 172.21.15.120:0/2107885958 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c1c0b9030 con 0x5607cab025d0 2024-09-19T09:09:13.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.026+0000 7f0c03fff640 1 -- 172.21.15.120:0/2107885958 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0c18049c90 con 0x5607cab025d0 2024-09-19T09:09:13.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.026+0000 7f0c2cf97740 10 client.17640 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:13.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.026+0000 7f0c2cf97740 1 -- 172.21.15.120:0/2107885958 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5607cab878c0 con 0x5607cab025d0 2024-09-19T09:09:13.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.027+0000 7f0c03fff640 1 -- 172.21.15.120:0/2107885958 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c1c0b9030 con 0x5607cab025d0 2024-09-19T09:09:13.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.027+0000 7f0c03fff640 1 -- 172.21.15.120:0/2107885958 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0c1803c440 con 0x5607cab025d0 2024-09-19T09:09:13.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.027+0000 7f0c2cf97740 10 client.17640 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:13.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.027+0000 7f0c2cf97740 10 client.17640 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:09:13.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.027+0000 7f0c2cf97740 20 client.17640 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:13.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.027+0000 7f0c2cf97740 1 --2- 172.21.15.120:0/2107885958 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5607cab87f50 0x5607cab8a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:13.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.027+0000 7f0c2cf97740 4 client.17640 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:09:13.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.027+0000 7f0c2cf97740 1 -- 172.21.15.120:0/2107885958 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5607caa6fd70 con 0x5607cab87f50 2024-09-19T09:09:13.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.027+0000 7f0c23fff640 1 --2- 172.21.15.120:0/2107885958 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5607cab87f50 0x5607cab8a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:13.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.027+0000 7f0c23fff640 1 --2- 172.21.15.120:0/2107885958 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5607cab87f50 0x5607cab8a370 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2546f995183c0d9b server_cookie=f491099569d812b7 in_seq=0 out_seq=0 2024-09-19T09:09:13.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.027+0000 7f0c03fff640 10 client.17640 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:09:13.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.028+0000 7f0c03fff640 1 -- 172.21.15.120:0/2107885958 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5607caa6fd70 con 0x5607cab87f50 2024-09-19T09:09:13.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.028+0000 7f0c03fff640 10 client.17640 handle_command_reply: tid=0 2024-09-19T09:09:13.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.028+0000 7f0c2cf97740 1 client.17640 shutdown 2024-09-19T09:09:13.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.028+0000 7f0c2cf97740 1 -- 172.21.15.120:0/2107885958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607cab025d0 msgr2=0x5607cab157b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:13.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.028+0000 7f0c2cf97740 1 --2- 172.21.15.120:0/2107885958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607cab025d0 0x5607cab157b0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f0c18049810 tx=0x7f0c18031c60 comp rx=0 tx=0).stop 2024-09-19T09:09:13.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.028+0000 7f0c2cf97740 1 -- 172.21.15.120:0/2107885958 shutdown_connections 2024-09-19T09:09:13.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.028+0000 7f0c2cf97740 1 --2- 172.21.15.120:0/2107885958 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5607cab87f50 0x5607cab8a370 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.028+0000 7f0c2cf97740 1 --2- 172.21.15.120:0/2107885958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607cab0d370 0x5607cab10c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.028+0000 7f0c2cf97740 1 --2- 172.21.15.120:0/2107885958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5607cab03b40 0x5607cab15cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.028+0000 7f0c2cf97740 1 --2- 172.21.15.120:0/2107885958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5607cab025d0 0x5607cab157b0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.028+0000 7f0c03fff640 0 client.17640 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:09:13.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.028+0000 7f0c2cf97740 1 -- 172.21.15.120:0/2107885958 >> 172.21.15.120:0/2107885958 conn(0x5607caacb310 msgr2=0x5607caafd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:13.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.028+0000 7f0c2cf97740 1 -- 172.21.15.120:0/2107885958 shutdown_connections 2024-09-19T09:09:13.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.028+0000 7f0c2cf97740 1 -- 172.21.15.120:0/2107885958 wait complete. 2024-09-19T09:09:13.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.028+0000 7f0c2cf97740 20 client.17640 trim_cache size 0 max 16384 2024-09-19T09:09:13.033 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:09:13.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.029+0000 7f0c2abbe640 1 -- 172.21.15.120:0/1222931028 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0c140a3240 msgr2=0x7f0c140a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:13.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.029+0000 7f0c2abbe640 1 --2- 172.21.15.120:0/1222931028 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0c140a3240 0x7f0c140a5700 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0c2419faa0 tx=0x7f0c180058d0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.029+0000 7f0c2abbe640 1 -- 172.21.15.120:0/1222931028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c24112b10 msgr2=0x7f0c2419e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:13.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.029+0000 7f0c2abbe640 1 --2- 172.21.15.120:0/1222931028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c24112b10 0x7f0c2419e960 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00eb40 tx=0x7f0c1c00c2d0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.029+0000 7f0c2abbe640 1 -- 172.21.15.120:0/1222931028 shutdown_connections 2024-09-19T09:09:13.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.029+0000 7f0c2abbe640 1 --2- 172.21.15.120:0/1222931028 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0c140a3240 0x7f0c140a5700 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.029+0000 7f0c2abbe640 1 --2- 172.21.15.120:0/1222931028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c24112b10 0x7f0c2419e960 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.029+0000 7f0c2abbe640 1 --2- 172.21.15.120:0/1222931028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c24106080 0x7f0c241a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.029+0000 7f0c2abbe640 1 --2- 172.21.15.120:0/1222931028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c24105760 0x7f0c241a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:13.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.029+0000 7f0c2abbe640 1 -- 172.21.15.120:0/1222931028 >> 172.21.15.120:0/1222931028 conn(0x7f0c24077210 msgr2=0x7f0c24111450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:13.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.029+0000 7f0c2abbe640 1 -- 172.21.15.120:0/1222931028 shutdown_connections 2024-09-19T09:09:13.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:13.029+0000 7f0c2abbe640 1 -- 172.21.15.120:0/1222931028 wait complete. 2024-09-19T09:09:13.052 INFO:tasks.workunit.client.0.smithi120.stdout: 1 550899 9.37 MB/sec execute 188 sec latency 68.918 ms 2024-09-19T09:09:13.855 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:09:14.052 INFO:tasks.workunit.client.0.smithi120.stdout: 1 554126 9.41 MB/sec execute 189 sec latency 14.784 ms 2024-09-19T09:09:14.160 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:09:14.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.721+0000 7fe7f69f3640 1 -- 172.21.15.120:0/4097839123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7f0102570 msgr2=0x7fe7f01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:14.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.721+0000 7fe7f69f3640 1 --2- 172.21.15.120:0/4097839123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7f0102570 0x7fe7f01029b0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009900 tx=0x7fe7e402a480 comp rx=0 tx=0).stop 2024-09-19T09:09:14.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.721+0000 7fe7f69f3640 1 -- 172.21.15.120:0/4097839123 shutdown_connections 2024-09-19T09:09:14.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.721+0000 7fe7f69f3640 1 --2- 172.21.15.120:0/4097839123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7f0102ef0 0x7fe7f010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.721+0000 7fe7f69f3640 1 --2- 172.21.15.120:0/4097839123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7f0102570 0x7fe7f01029b0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.721+0000 7fe7f69f3640 1 --2- 172.21.15.120:0/4097839123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7f0100e20 0x7fe7f0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.721+0000 7fe7f69f3640 1 -- 172.21.15.120:0/4097839123 >> 172.21.15.120:0/4097839123 conn(0x7fe7f00fc7d0 msgr2=0x7fe7f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:14.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.721+0000 7fe7f69f3640 1 -- 172.21.15.120:0/4097839123 shutdown_connections 2024-09-19T09:09:14.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.721+0000 7fe7f69f3640 1 -- 172.21.15.120:0/4097839123 wait complete. 2024-09-19T09:09:14.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.722+0000 7fe7f69f3640 1 Processor -- start 2024-09-19T09:09:14.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.722+0000 7fe7f69f3640 1 -- start start 2024-09-19T09:09:14.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.723+0000 7fe7f69f3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7f0100e20 0x7fe7f0199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:14.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.723+0000 7fe7f69f3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7f0102570 0x7fe7f01998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:14.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.723+0000 7fe7f69f3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7f0102ef0 0x7fe7f01a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:14.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.723+0000 7fe7f69f3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f0110f50 con 0x7fe7f0102570 2024-09-19T09:09:14.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.723+0000 7fe7f69f3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f0110dd0 con 0x7fe7f0102ef0 2024-09-19T09:09:14.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.723+0000 7fe7f69f3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f01110d0 con 0x7fe7f0100e20 2024-09-19T09:09:14.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.723+0000 7fe7ef7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7f0102570 0x7fe7f01998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:14.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.723+0000 7fe7ef7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7f0102570 0x7fe7f01998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47934/0 (socket says 172.21.15.120:47934) 2024-09-19T09:09:14.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.723+0000 7fe7ef7fe640 1 -- 172.21.15.120:0/691306342 learned_addr learned my addr 172.21.15.120:0/691306342 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:14.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.723+0000 7fe7effff640 1 --2- 172.21.15.120:0/691306342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7f0100e20 0x7fe7f0199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:14.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.723+0000 7fe7f4f90640 1 --2- 172.21.15.120:0/691306342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7f0102ef0 0x7fe7f01a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:14.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.723+0000 7fe7effff640 1 -- 172.21.15.120:0/691306342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7f0102ef0 msgr2=0x7fe7f01a43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:14.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.723+0000 7fe7effff640 1 --2- 172.21.15.120:0/691306342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7f0102ef0 0x7fe7f01a43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.723+0000 7fe7effff640 1 -- 172.21.15.120:0/691306342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7f0102570 msgr2=0x7fe7f01998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:14.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.723+0000 7fe7effff640 1 --2- 172.21.15.120:0/691306342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7f0102570 0x7fe7f01998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.723+0000 7fe7effff640 1 -- 172.21.15.120:0/691306342 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f01a4920 con 0x7fe7f0100e20 2024-09-19T09:09:14.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.723+0000 7fe7ef7fe640 1 --2- 172.21.15.120:0/691306342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7f0102570 0x7fe7f01998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:14.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.723+0000 7fe7f4f90640 1 --2- 172.21.15.120:0/691306342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7f0102ef0 0x7fe7f01a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:14.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.724+0000 7fe7effff640 1 --2- 172.21.15.120:0/691306342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7f0100e20 0x7fe7f0199360 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000e9f0 tx=0x7fe7e000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:14.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.724+0000 7fe7ed7fa640 1 -- 172.21.15.120:0/691306342 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e000ccc0 con 0x7fe7f0100e20 2024-09-19T09:09:14.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.724+0000 7fe7f69f3640 1 -- 172.21.15.120:0/691306342 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f0197150 con 0x7fe7f0100e20 2024-09-19T09:09:14.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.724+0000 7fe7f69f3640 1 -- 172.21.15.120:0/691306342 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f01976e0 con 0x7fe7f0100e20 2024-09-19T09:09:14.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.724+0000 7fe7ed7fa640 1 -- 172.21.15.120:0/691306342 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7e0010910 con 0x7fe7f0100e20 2024-09-19T09:09:14.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.724+0000 7fe7ed7fa640 1 -- 172.21.15.120:0/691306342 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0018880 con 0x7fe7f0100e20 2024-09-19T09:09:14.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.726+0000 7fe7f8dcc740 1 Processor -- start 2024-09-19T09:09:14.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.726+0000 7fe7f8dcc740 1 -- start start 2024-09-19T09:09:14.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.726+0000 7fe7ed7fa640 1 -- 172.21.15.120:0/691306342 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe7e0010430 con 0x7fe7f0100e20 2024-09-19T09:09:14.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.727+0000 7fe7f8dcc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56076c7deb40 0x56076c7dd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:14.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.727+0000 7fe7f8dcc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56076c7dd5d0 0x56076c7dda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:14.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.727+0000 7fe7f8dcc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56076c7e8370 0x56076c7ea760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:14.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.727+0000 7fe7f8dcc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56076c7432f0 con 0x56076c7dd5d0 2024-09-19T09:09:14.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.727+0000 7fe7f8dcc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56076c6eacf0 con 0x56076c7e8370 2024-09-19T09:09:14.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.727+0000 7fe7f8dcc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56076c73f2b0 con 0x56076c7deb40 2024-09-19T09:09:14.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.727+0000 7fe7effff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56076c7dd5d0 0x56076c7dda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:14.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.727+0000 7fe7f4f90640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56076c7deb40 0x56076c7dd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:14.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.727+0000 7fe7f4f90640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56076c7deb40 0x56076c7dd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:59596/0 (socket says 172.21.15.120:59596) 2024-09-19T09:09:14.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.727+0000 7fe7f4f90640 1 -- 172.21.15.120:0/2182222143 learned_addr learned my addr 172.21.15.120:0/2182222143 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:14.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.727+0000 7fe7ef7fe640 1 --2- 172.21.15.120:0/2182222143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56076c7e8370 0x56076c7ea760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:14.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.727+0000 7fe7f4f90640 1 -- 172.21.15.120:0/2182222143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56076c7e8370 msgr2=0x56076c7ea760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:14.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.727+0000 7fe7f4f90640 1 --2- 172.21.15.120:0/2182222143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56076c7e8370 0x56076c7ea760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.727+0000 7fe7f4f90640 1 -- 172.21.15.120:0/2182222143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56076c7dd5d0 msgr2=0x56076c7dda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:14.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.727+0000 7fe7f4f90640 1 --2- 172.21.15.120:0/2182222143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56076c7dd5d0 0x56076c7dda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.727+0000 7fe7f4f90640 1 -- 172.21.15.120:0/2182222143 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56076c6cf3a0 con 0x56076c7deb40 2024-09-19T09:09:14.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.727+0000 7fe7ed7fa640 1 --2- 172.21.15.120:0/691306342 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe7d80a3240 0x7fe7d80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:14.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.727+0000 7fe7f4f90640 1 --2- 172.21.15.120:0/2182222143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56076c7deb40 0x56076c7dd090 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8010a30 tx=0x7fe7e8009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e80b5068dd90518 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:14.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.727+0000 7fe7effff640 1 --2- 172.21.15.120:0/691306342 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe7d80a3240 0x7fe7d80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:14.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.727+0000 7fe7ed7fa640 1 -- 172.21.15.120:0/691306342 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fe7e00cfb70 con 0x7fe7f0100e20 2024-09-19T09:09:14.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.728+0000 7fe7d67fc640 1 -- 172.21.15.120:0/2182222143 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8003a30 con 0x56076c7deb40 2024-09-19T09:09:14.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.728+0000 7fe7d67fc640 1 -- 172.21.15.120:0/2182222143 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7e800be20 con 0x56076c7deb40 2024-09-19T09:09:14.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.728+0000 7fe7d67fc640 1 -- 172.21.15.120:0/2182222143 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8004700 con 0x56076c7deb40 2024-09-19T09:09:14.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.728+0000 7fe7f8dcc740 1 -- 172.21.15.120:0/2182222143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56076c7deb40 msgr2=0x56076c7dd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:14.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.728+0000 7fe7effff640 1 --2- 172.21.15.120:0/691306342 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe7d80a3240 0x7fe7d80a5700 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00a4890 tx=0x7fe7e00d35b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:14.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.728+0000 7fe7f8dcc740 1 --2- 172.21.15.120:0/2182222143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56076c7deb40 0x56076c7dd090 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8010a30 tx=0x7fe7e8009500 comp rx=0 tx=0).stop 2024-09-19T09:09:14.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.728+0000 7fe7f8dcc740 1 -- 172.21.15.120:0/2182222143 shutdown_connections 2024-09-19T09:09:14.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.728+0000 7fe7f8dcc740 1 --2- 172.21.15.120:0/2182222143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56076c7e8370 0x56076c7ea760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.728+0000 7fe7f8dcc740 1 --2- 172.21.15.120:0/2182222143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56076c7dd5d0 0x56076c7dda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.728+0000 7fe7f8dcc740 1 --2- 172.21.15.120:0/2182222143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56076c7deb40 0x56076c7dd090 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.728+0000 7fe7f8dcc740 1 -- 172.21.15.120:0/2182222143 >> 172.21.15.120:0/2182222143 conn(0x56076c7a6310 msgr2=0x56076c7d9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:14.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.729+0000 7fe7f8dcc740 1 -- 172.21.15.120:0/2182222143 shutdown_connections 2024-09-19T09:09:14.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.729+0000 7fe7f8dcc740 1 -- 172.21.15.120:0/2182222143 wait complete. 2024-09-19T09:09:14.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.729+0000 7fe7f8dcc740 1 -- start start 2024-09-19T09:09:14.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.729+0000 7fe7f8dcc740 1 Processor -- start 2024-09-19T09:09:14.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.729+0000 7fe7f8dcc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56076c7dd5d0 0x56076c7e7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:14.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.729+0000 7fe7f8dcc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56076c7deb40 0x56076c7e4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:14.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.729+0000 7fe7f8dcc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56076c7e8370 0x56076c7e55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:14.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.729+0000 7fe7f8dcc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56076c7eb070 con 0x56076c7deb40 2024-09-19T09:09:14.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.729+0000 7fe7f8dcc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56076c6fe060 con 0x56076c7e8370 2024-09-19T09:09:14.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.729+0000 7fe7f8dcc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56076c7eb1f0 con 0x56076c7dd5d0 2024-09-19T09:09:14.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.730+0000 7fe7f8dcc740 10 client.0 _ll_register_callbacks cb 0x56076c4518c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:14.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.730+0000 7fe7ef7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56076c7deb40 0x56076c7e4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:14.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.730+0000 7fe7f4f90640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56076c7dd5d0 0x56076c7e7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:14.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.730+0000 7fe7ef7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56076c7deb40 0x56076c7e4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47962/0 (socket says 172.21.15.120:47962) 2024-09-19T09:09:14.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.730+0000 7fe7ef7fe640 1 -- 172.21.15.120:0/2568035922 learned_addr learned my addr 172.21.15.120:0/2568035922 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:14.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.730+0000 7fe7f4f90640 1 --2- 172.21.15.120:0/2568035922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56076c7e8370 0x56076c7e55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:14.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.730+0000 7fe7ef7fe640 1 -- 172.21.15.120:0/2568035922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56076c7dd5d0 msgr2=0x56076c7e7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:14.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.730+0000 7fe7ef7fe640 1 --2- 172.21.15.120:0/2568035922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56076c7dd5d0 0x56076c7e7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.730+0000 7fe7ef7fe640 1 -- 172.21.15.120:0/2568035922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56076c7e8370 msgr2=0x56076c7e55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:14.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.730+0000 7fe7ef7fe640 1 --2- 172.21.15.120:0/2568035922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56076c7e8370 0x56076c7e55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.730+0000 7fe7ef7fe640 1 -- 172.21.15.120:0/2568035922 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56076c4b43c0 con 0x56076c7deb40 2024-09-19T09:09:14.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.730+0000 7fe7ef7fe640 1 --2- 172.21.15.120:0/2568035922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56076c7deb40 0x56076c7e4fe0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4000c00 tx=0x7fe7e40035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1408e22256495bad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:14.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.730+0000 7fe7cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:09:14.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.730+0000 7fe7cf7fe640 1 -- 172.21.15.120:0/2568035922 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4035040 con 0x56076c7deb40 2024-09-19T09:09:14.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.730+0000 7fe7cf7fe640 1 -- 172.21.15.120:0/2568035922 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fe7e4003bf0 con 0x56076c7deb40 2024-09-19T09:09:14.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.730+0000 7fe7f8dcc740 1 -- 172.21.15.120:0/2568035922 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7e800be20 con 0x56076c7deb40 2024-09-19T09:09:14.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.731+0000 7fe7cf7fe640 1 -- 172.21.15.120:0/2568035922 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe7e4003df0 con 0x56076c7deb40 2024-09-19T09:09:14.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.731+0000 7fe7f8dcc740 10 client.17652 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:14.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.731+0000 7fe7f8dcc740 1 -- 172.21.15.120:0/2568035922 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56076c7f6b60 con 0x56076c7deb40 2024-09-19T09:09:14.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.731+0000 7fe7cf7fe640 1 -- 172.21.15.120:0/2568035922 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7e800be20 con 0x56076c7deb40 2024-09-19T09:09:14.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.731+0000 7fe7cf7fe640 1 -- 172.21.15.120:0/2568035922 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe7e4046700 con 0x56076c7deb40 2024-09-19T09:09:14.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.731+0000 7fe7f8dcc740 10 client.17652 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:14.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.731+0000 7fe7f8dcc740 10 client.17652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:09:14.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.731+0000 7fe7f8dcc740 20 client.17652 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:14.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.731+0000 7fe7f8dcc740 1 --2- 172.21.15.120:0/2568035922 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56076c7f80c0 0x56076c7fa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:14.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.731+0000 7fe7f8dcc740 4 client.17652 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:09:14.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.731+0000 7fe7f8dcc740 1 -- 172.21.15.120:0/2568035922 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56076c750860 con 0x56076c7f80c0 2024-09-19T09:09:14.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.731+0000 7fe7f4f90640 1 --2- 172.21.15.120:0/2568035922 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56076c7f80c0 0x56076c7fa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:14.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.732+0000 7fe7f4f90640 1 --2- 172.21.15.120:0/2568035922 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56076c7f80c0 0x56076c7fa4b0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=376bdb3db65ed612 server_cookie=e7e50da6be35e5a1 in_seq=0 out_seq=0 2024-09-19T09:09:14.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.732+0000 7fe7cf7fe640 10 client.17652 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:14.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.733+0000 7fe7cf7fe640 1 -- 172.21.15.120:0/2568035922 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56076c750860 con 0x56076c7f80c0 2024-09-19T09:09:14.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.733+0000 7fe7cf7fe640 10 client.17652 handle_command_reply: tid=0 2024-09-19T09:09:14.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.734+0000 7fe7f8dcc740 1 client.17652 shutdown 2024-09-19T09:09:14.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.734+0000 7fe7f8dcc740 1 -- 172.21.15.120:0/2568035922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56076c7deb40 msgr2=0x56076c7e4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:14.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.734+0000 7fe7f8dcc740 1 --2- 172.21.15.120:0/2568035922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56076c7deb40 0x56076c7e4fe0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4000c00 tx=0x7fe7e40035f0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.734+0000 7fe7f8dcc740 1 -- 172.21.15.120:0/2568035922 shutdown_connections 2024-09-19T09:09:14.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.734+0000 7fe7f8dcc740 1 --2- 172.21.15.120:0/2568035922 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56076c7f80c0 0x56076c7fa4b0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.734+0000 7fe7f8dcc740 1 --2- 172.21.15.120:0/2568035922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56076c7e8370 0x56076c7e55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.734+0000 7fe7f8dcc740 1 --2- 172.21.15.120:0/2568035922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56076c7deb40 0x56076c7e4fe0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.734+0000 7fe7f8dcc740 1 --2- 172.21.15.120:0/2568035922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56076c7dd5d0 0x56076c7e7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.734+0000 7fe7f8dcc740 1 -- 172.21.15.120:0/2568035922 >> 172.21.15.120:0/2568035922 conn(0x56076c7a6310 msgr2=0x56076c7df930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:14.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.734+0000 7fe7cf7fe640 0 client.17652 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:14.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.734+0000 7fe7f8dcc740 1 -- 172.21.15.120:0/2568035922 shutdown_connections 2024-09-19T09:09:14.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.734+0000 7fe7f8dcc740 1 -- 172.21.15.120:0/2568035922 wait complete. 2024-09-19T09:09:14.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.734+0000 7fe7f8dcc740 20 client.17652 trim_cache size 0 max 16384 2024-09-19T09:09:14.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.747+0000 7fe7f8dcc740 1 Processor -- start 2024-09-19T09:09:14.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.747+0000 7fe7f8dcc740 1 -- start start 2024-09-19T09:09:14.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.747+0000 7fe7f8dcc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56076c7dd5d0 0x56076c7f4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:14.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.747+0000 7fe7f8dcc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56076c7deb40 0x56076c7e3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:14.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.747+0000 7fe7f8dcc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56076c7e8370 0x56076c7ffd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:14.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.747+0000 7fe7f8dcc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc00e4a0 con 0x56076c7deb40 2024-09-19T09:09:14.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.747+0000 7fe7f8dcc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc00ffe0 con 0x56076c7e8370 2024-09-19T09:09:14.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.747+0000 7fe7f8dcc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc00eb70 con 0x56076c7dd5d0 2024-09-19T09:09:14.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.748+0000 7fe7ef7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56076c7deb40 0x56076c7e3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:14.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.748+0000 7fe7f4f90640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56076c7dd5d0 0x56076c7f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:14.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.748+0000 7fe7ef7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56076c7deb40 0x56076c7e3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47970/0 (socket says 172.21.15.120:47970) 2024-09-19T09:09:14.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.748+0000 7fe7f4f90640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56076c7dd5d0 0x56076c7f4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:59608/0 (socket says 172.21.15.120:59608) 2024-09-19T09:09:14.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.748+0000 7fe7ef7fe640 1 -- 172.21.15.120:0/2494396890 learned_addr learned my addr 172.21.15.120:0/2494396890 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:14.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.748+0000 7fe7f4f90640 1 --2- 172.21.15.120:0/2494396890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56076c7e8370 0x56076c7ffd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:14.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.748+0000 7fe7ef7fe640 1 -- 172.21.15.120:0/2494396890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56076c7dd5d0 msgr2=0x56076c7f4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:14.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.748+0000 7fe7ef7fe640 1 --2- 172.21.15.120:0/2494396890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56076c7dd5d0 0x56076c7f4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.748+0000 7fe7ef7fe640 1 -- 172.21.15.120:0/2494396890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56076c7e8370 msgr2=0x56076c7ffd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:14.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.748+0000 7fe7ef7fe640 1 --2- 172.21.15.120:0/2494396890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56076c7e8370 0x56076c7ffd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.748+0000 7fe7ef7fe640 1 -- 172.21.15.120:0/2494396890 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56076c7e0120 con 0x56076c7deb40 2024-09-19T09:09:14.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.748+0000 7fe7ef7fe640 1 --2- 172.21.15.120:0/2494396890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56076c7deb40 0x56076c7e3fc0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4046b50 tx=0x7fe7e402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb77d73b23f0bd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:14.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.748+0000 7fe7d5ffb640 1 -- 172.21.15.120:0/2494396890 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4035070 con 0x56076c7deb40 2024-09-19T09:09:14.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.748+0000 7fe7d5ffb640 1 -- 172.21.15.120:0/2494396890 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7e4055020 con 0x56076c7deb40 2024-09-19T09:09:14.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.748+0000 7fe7d5ffb640 1 -- 172.21.15.120:0/2494396890 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4005390 con 0x56076c7deb40 2024-09-19T09:09:14.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.749+0000 7fe7f8dcc740 1 -- 172.21.15.120:0/2494396890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56076c7deb40 msgr2=0x56076c7e3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:14.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.749+0000 7fe7f8dcc740 1 --2- 172.21.15.120:0/2494396890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56076c7deb40 0x56076c7e3fc0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4046b50 tx=0x7fe7e402a480 comp rx=0 tx=0).stop 2024-09-19T09:09:14.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.749+0000 7fe7f8dcc740 1 -- 172.21.15.120:0/2494396890 shutdown_connections 2024-09-19T09:09:14.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.749+0000 7fe7f8dcc740 1 --2- 172.21.15.120:0/2494396890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56076c7e8370 0x56076c7ffd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.749+0000 7fe7f8dcc740 1 --2- 172.21.15.120:0/2494396890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56076c7deb40 0x56076c7e3fc0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.749+0000 7fe7f8dcc740 1 --2- 172.21.15.120:0/2494396890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56076c7dd5d0 0x56076c7f4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.749+0000 7fe7f8dcc740 1 -- 172.21.15.120:0/2494396890 >> 172.21.15.120:0/2494396890 conn(0x56076c7a6310 msgr2=0x56076c7e38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:14.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.749+0000 7fe7f8dcc740 1 -- 172.21.15.120:0/2494396890 shutdown_connections 2024-09-19T09:09:14.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.749+0000 7fe7f8dcc740 1 -- 172.21.15.120:0/2494396890 wait complete. 2024-09-19T09:09:14.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.749+0000 7fe7f8dcc740 1 -- start start 2024-09-19T09:09:14.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.750+0000 7fe7f8dcc740 1 Processor -- start 2024-09-19T09:09:14.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.750+0000 7fe7f8dcc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56076c7dd5d0 0x56076c7f0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:14.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.750+0000 7fe7f8dcc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56076c7deb40 0x56076c7ebf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:14.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.750+0000 7fe7f8dcc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56076c7e8370 0x56076c7ec4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:14.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.750+0000 7fe7f8dcc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc00f910 con 0x56076c7deb40 2024-09-19T09:09:14.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.750+0000 7fe7f8dcc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc00f240 con 0x56076c7e8370 2024-09-19T09:09:14.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.750+0000 7fe7f8dcc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc00c460 con 0x56076c7dd5d0 2024-09-19T09:09:14.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.750+0000 7fe7f8dcc740 10 client.0 _ll_register_callbacks cb 0x7fe7e4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:14.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.750+0000 7fe7ef7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56076c7deb40 0x56076c7ebf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:14.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.750+0000 7fe7f4f90640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56076c7dd5d0 0x56076c7f0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:14.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.750+0000 7fe7ef7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56076c7deb40 0x56076c7ebf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47978/0 (socket says 172.21.15.120:47978) 2024-09-19T09:09:14.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.750+0000 7fe7ef7fe640 1 -- 172.21.15.120:0/2650782825 learned_addr learned my addr 172.21.15.120:0/2650782825 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:14.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.750+0000 7fe7f4f90640 1 --2- 172.21.15.120:0/2650782825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56076c7e8370 0x56076c7ec4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:14.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.750+0000 7fe7ef7fe640 1 -- 172.21.15.120:0/2650782825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56076c7dd5d0 msgr2=0x56076c7f0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:14.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.750+0000 7fe7ef7fe640 1 --2- 172.21.15.120:0/2650782825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56076c7dd5d0 0x56076c7f0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.750+0000 7fe7ef7fe640 1 -- 172.21.15.120:0/2650782825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56076c7e8370 msgr2=0x56076c7ec4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:14.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.750+0000 7fe7ef7fe640 1 --2- 172.21.15.120:0/2650782825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56076c7e8370 0x56076c7ec4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.750+0000 7fe7ef7fe640 1 -- 172.21.15.120:0/2650782825 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56076c7ecc80 con 0x56076c7deb40 2024-09-19T09:09:14.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.751+0000 7fe7ef7fe640 1 --2- 172.21.15.120:0/2650782825 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56076c7deb40 0x56076c7ebf60 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4000c00 tx=0x7fe7e4009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554f5042bf00da79 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:14.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.751+0000 7fe7cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:09:14.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.751+0000 7fe7cf7fe640 1 -- 172.21.15.120:0/2650782825 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4035070 con 0x56076c7deb40 2024-09-19T09:09:14.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.751+0000 7fe7cf7fe640 1 -- 172.21.15.120:0/2650782825 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fe7e4005320 con 0x56076c7deb40 2024-09-19T09:09:14.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.751+0000 7fe7f8dcc740 1 -- 172.21.15.120:0/2650782825 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7e4055020 con 0x56076c7deb40 2024-09-19T09:09:14.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.751+0000 7fe7cf7fe640 1 -- 172.21.15.120:0/2650782825 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe7e40039f0 con 0x56076c7deb40 2024-09-19T09:09:14.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.752+0000 7fe7f8dcc740 10 client.17658 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:14.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.752+0000 7fe7f8dcc740 1 -- 172.21.15.120:0/2650782825 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56076c862d60 con 0x56076c7deb40 2024-09-19T09:09:14.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.752+0000 7fe7cf7fe640 1 -- 172.21.15.120:0/2650782825 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7e4055020 con 0x56076c7deb40 2024-09-19T09:09:14.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.752+0000 7fe7cf7fe640 1 -- 172.21.15.120:0/2650782825 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe7e4059020 con 0x56076c7deb40 2024-09-19T09:09:14.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.752+0000 7fe7f8dcc740 10 client.17658 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:14.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.752+0000 7fe7f8dcc740 10 client.17658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:09:14.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.752+0000 7fe7f8dcc740 20 client.17658 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:14.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.752+0000 7fe7f8dcc740 1 --2- 172.21.15.120:0/2650782825 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56076c8631e0 0x56076c8655d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:14.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.752+0000 7fe7f8dcc740 4 client.17658 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:09:14.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.752+0000 7fe7f8dcc740 1 -- 172.21.15.120:0/2650782825 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56076c74ad70 con 0x56076c8631e0 2024-09-19T09:09:14.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.752+0000 7fe7f4f90640 1 --2- 172.21.15.120:0/2650782825 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56076c8631e0 0x56076c8655d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:14.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.753+0000 7fe7f4f90640 1 --2- 172.21.15.120:0/2650782825 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56076c8631e0 0x56076c8655d0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9f803032b3e7576 server_cookie=2b84b3ab7c4d6203 in_seq=0 out_seq=0 2024-09-19T09:09:14.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.753+0000 7fe7cf7fe640 10 client.17658 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:14.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.753+0000 7fe7cf7fe640 1 -- 172.21.15.120:0/2650782825 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56076c74ad70 con 0x56076c8631e0 2024-09-19T09:09:14.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.753+0000 7fe7cf7fe640 10 client.17658 handle_command_reply: tid=0 2024-09-19T09:09:14.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.753+0000 7fe7f8dcc740 1 client.17658 shutdown 2024-09-19T09:09:14.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.753+0000 7fe7f8dcc740 1 -- 172.21.15.120:0/2650782825 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56076c7deb40 msgr2=0x56076c7ebf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:14.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.753+0000 7fe7f8dcc740 1 --2- 172.21.15.120:0/2650782825 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56076c7deb40 0x56076c7ebf60 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4000c00 tx=0x7fe7e4009560 comp rx=0 tx=0).stop 2024-09-19T09:09:14.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.753+0000 7fe7f8dcc740 1 -- 172.21.15.120:0/2650782825 shutdown_connections 2024-09-19T09:09:14.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.753+0000 7fe7f8dcc740 1 --2- 172.21.15.120:0/2650782825 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56076c8631e0 0x56076c8655d0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.753+0000 7fe7f8dcc740 1 --2- 172.21.15.120:0/2650782825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56076c7e8370 0x56076c7ec4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.753+0000 7fe7f8dcc740 1 --2- 172.21.15.120:0/2650782825 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56076c7deb40 0x56076c7ebf60 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.753+0000 7fe7f8dcc740 1 --2- 172.21.15.120:0/2650782825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56076c7dd5d0 0x56076c7f0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.753+0000 7fe7cf7fe640 0 client.17658 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:14.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.753+0000 7fe7f8dcc740 1 -- 172.21.15.120:0/2650782825 >> 172.21.15.120:0/2650782825 conn(0x56076c7a6310 msgr2=0x56076c7e6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:14.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.753+0000 7fe7f8dcc740 1 -- 172.21.15.120:0/2650782825 shutdown_connections 2024-09-19T09:09:14.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.753+0000 7fe7f8dcc740 1 -- 172.21.15.120:0/2650782825 wait complete. 2024-09-19T09:09:14.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.754+0000 7fe7f8dcc740 20 client.17658 trim_cache size 0 max 16384 2024-09-19T09:09:14.760 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:09:14.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.754+0000 7fe7f69f3640 1 -- 172.21.15.120:0/691306342 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe7d80a3240 msgr2=0x7fe7d80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:14.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.754+0000 7fe7f69f3640 1 --2- 172.21.15.120:0/691306342 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe7d80a3240 0x7fe7d80a5700 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00a4890 tx=0x7fe7e00d35b0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.754+0000 7fe7f69f3640 1 -- 172.21.15.120:0/691306342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7f0100e20 msgr2=0x7fe7f0199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:14.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.754+0000 7fe7f69f3640 1 --2- 172.21.15.120:0/691306342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7f0100e20 0x7fe7f0199360 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000e9f0 tx=0x7fe7e000eeb0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.754+0000 7fe7f69f3640 1 -- 172.21.15.120:0/691306342 shutdown_connections 2024-09-19T09:09:14.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.754+0000 7fe7f69f3640 1 --2- 172.21.15.120:0/691306342 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe7d80a3240 0x7fe7d80a5700 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.754+0000 7fe7f69f3640 1 --2- 172.21.15.120:0/691306342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7f0102ef0 0x7fe7f01a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.754+0000 7fe7f69f3640 1 --2- 172.21.15.120:0/691306342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7f0102570 0x7fe7f01998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.754+0000 7fe7f69f3640 1 --2- 172.21.15.120:0/691306342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7f0100e20 0x7fe7f0199360 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:14.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.754+0000 7fe7f69f3640 1 -- 172.21.15.120:0/691306342 >> 172.21.15.120:0/691306342 conn(0x7fe7f00fc7d0 msgr2=0x7fe7f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:14.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.754+0000 7fe7f69f3640 1 -- 172.21.15.120:0/691306342 shutdown_connections 2024-09-19T09:09:14.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:14.754+0000 7fe7f69f3640 1 -- 172.21.15.120:0/691306342 wait complete. 2024-09-19T09:09:14.979 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:14 smithi120 ceph-mon[31150]: pgmap v533: 97 pgs: 97 active+clean; 27 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 4.2 MiB/s wr, 433 op/s 2024-09-19T09:09:15.054 INFO:tasks.workunit.client.0.smithi120.stdout: 1 555855 9.40 MB/sec execute 190 sec latency 101.764 ms 2024-09-19T09:09:15.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:14 smithi196 ceph-mon[35150]: pgmap v533: 97 pgs: 97 active+clean; 27 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 4.2 MiB/s wr, 433 op/s 2024-09-19T09:09:15.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:14 smithi153 ceph-mon[35071]: pgmap v533: 97 pgs: 97 active+clean; 27 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 4.2 MiB/s wr, 433 op/s 2024-09-19T09:09:16.052 INFO:tasks.workunit.client.0.smithi120.stdout: 1 558695 9.43 MB/sec execute 191 sec latency 18.146 ms 2024-09-19T09:09:16.474 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:09:16.474 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:09:16.739 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:09:17.044 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:16 smithi120 ceph-mon[31150]: pgmap v534: 97 pgs: 97 active+clean; 30 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 3.9 MiB/s wr, 327 op/s 2024-09-19T09:09:17.052 INFO:tasks.workunit.client.0.smithi120.stdout: 1 561729 9.47 MB/sec execute 192 sec latency 13.899 ms 2024-09-19T09:09:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:16 smithi196 ceph-mon[35150]: pgmap v534: 97 pgs: 97 active+clean; 30 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 3.9 MiB/s wr, 327 op/s 2024-09-19T09:09:17.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:16 smithi153 ceph-mon[35071]: pgmap v534: 97 pgs: 97 active+clean; 30 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 3.9 MiB/s wr, 327 op/s 2024-09-19T09:09:17.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.267+0000 7f7bd4b88640 1 -- 172.21.15.120:0/271586588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bd0100e20 msgr2=0x7f7bd0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:17.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.267+0000 7f7bd4b88640 1 --2- 172.21.15.120:0/271586588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bd0100e20 0x7f7bd0101200 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0009900 tx=0x7f7bc002a480 comp rx=0 tx=0).stop 2024-09-19T09:09:17.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.267+0000 7f7bd4b88640 1 -- 172.21.15.120:0/271586588 shutdown_connections 2024-09-19T09:09:17.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.267+0000 7f7bd4b88640 1 --2- 172.21.15.120:0/271586588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7bd0102ef0 0x7f7bd010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.267+0000 7f7bd4b88640 1 --2- 172.21.15.120:0/271586588 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bd0102570 0x7f7bd01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.267+0000 7f7bd4b88640 1 --2- 172.21.15.120:0/271586588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bd0100e20 0x7f7bd0101200 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.267+0000 7f7bd4b88640 1 -- 172.21.15.120:0/271586588 >> 172.21.15.120:0/271586588 conn(0x7f7bd00fc7d0 msgr2=0x7f7bd00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:17.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.267+0000 7f7bd4b88640 1 -- 172.21.15.120:0/271586588 shutdown_connections 2024-09-19T09:09:17.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.267+0000 7f7bd4b88640 1 -- 172.21.15.120:0/271586588 wait complete. 2024-09-19T09:09:17.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.268+0000 7f7bd4b88640 1 Processor -- start 2024-09-19T09:09:17.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.268+0000 7f7bd4b88640 1 -- start start 2024-09-19T09:09:17.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.268+0000 7f7bd4b88640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bd0100e20 0x7f7bd01987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:17.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.268+0000 7f7bd4b88640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bd0102570 0x7f7bd0198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:17.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.268+0000 7f7bd4b88640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7bd0102ef0 0x7f7bd019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:17.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.268+0000 7f7bd4b88640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd0110f50 con 0x7f7bd0102570 2024-09-19T09:09:17.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.268+0000 7f7bd4b88640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd0110dd0 con 0x7f7bd0102ef0 2024-09-19T09:09:17.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.268+0000 7f7bd4b88640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01110d0 con 0x7f7bd0100e20 2024-09-19T09:09:17.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.268+0000 7f7bc7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bd0102570 0x7f7bd0198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:17.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.268+0000 7f7bc7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bd0102570 0x7f7bd0198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47992/0 (socket says 172.21.15.120:47992) 2024-09-19T09:09:17.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.268+0000 7f7bc7fff640 1 -- 172.21.15.120:0/3283807113 learned_addr learned my addr 172.21.15.120:0/3283807113 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:17.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.268+0000 7f7bce59c640 1 --2- 172.21.15.120:0/3283807113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bd0100e20 0x7f7bd01987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:17.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.269+0000 7f7bced9d640 1 --2- 172.21.15.120:0/3283807113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7bd0102ef0 0x7f7bd019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:17.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.269+0000 7f7bc7fff640 1 -- 172.21.15.120:0/3283807113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bd0100e20 msgr2=0x7f7bd01987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:17.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.269+0000 7f7bc7fff640 1 --2- 172.21.15.120:0/3283807113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bd0100e20 0x7f7bd01987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.269+0000 7f7bc7fff640 1 -- 172.21.15.120:0/3283807113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7bd0102ef0 msgr2=0x7f7bd019ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:17.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.269+0000 7f7bc7fff640 1 --2- 172.21.15.120:0/3283807113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7bd0102ef0 0x7f7bd019ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.269+0000 7f7bc7fff640 1 -- 172.21.15.120:0/3283807113 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd019d530 con 0x7f7bd0102570 2024-09-19T09:09:17.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.269+0000 7f7bc7fff640 1 --2- 172.21.15.120:0/3283807113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bd0102570 0x7f7bd0198d20 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800e9f0 tx=0x7f7bb800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:17.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.269+0000 7f7bc77fe640 1 -- 172.21.15.120:0/3283807113 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb800cd80 con 0x7f7bd0102570 2024-09-19T09:09:17.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.269+0000 7f7bd4b88640 1 -- 172.21.15.120:0/3283807113 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bd019d820 con 0x7f7bd0102570 2024-09-19T09:09:17.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.269+0000 7f7bc77fe640 1 -- 172.21.15.120:0/3283807113 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bb8010910 con 0x7f7bd0102570 2024-09-19T09:09:17.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.269+0000 7f7bd4b88640 1 -- 172.21.15.120:0/3283807113 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bd01a5070 con 0x7f7bd0102570 2024-09-19T09:09:17.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.269+0000 7f7bc77fe640 1 -- 172.21.15.120:0/3283807113 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb80188f0 con 0x7f7bd0102570 2024-09-19T09:09:17.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.272+0000 7f7bc77fe640 1 -- 172.21.15.120:0/3283807113 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f7bb8010ac0 con 0x7f7bd0102570 2024-09-19T09:09:17.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.272+0000 7f7bd6f61740 1 Processor -- start 2024-09-19T09:09:17.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.272+0000 7f7bd6f61740 1 -- start start 2024-09-19T09:09:17.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.273+0000 7f7bc77fe640 1 --2- 172.21.15.120:0/3283807113 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7bbc0a3240 0x7f7bbc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:17.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.273+0000 7f7bd6f61740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fa337a5a60 0x55fa337a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:17.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.273+0000 7f7bd6f61740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa337a6380 0x55fa337b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:17.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.273+0000 7f7bd6f61740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa337b5150 0x55fa337b7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:17.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.273+0000 7f7bd6f61740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fa3370b280 con 0x55fa337a5a60 2024-09-19T09:09:17.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.273+0000 7f7bd6f61740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fa336b2b10 con 0x55fa337b5150 2024-09-19T09:09:17.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.273+0000 7f7bd6f61740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fa33707240 con 0x55fa337a6380 2024-09-19T09:09:17.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.273+0000 7f7bc77fe640 1 -- 172.21.15.120:0/3283807113 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f7bb80cfca0 con 0x7f7bd0102570 2024-09-19T09:09:17.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.273+0000 7f7bce59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fa337a5a60 0x55fa337a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:17.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.273+0000 7f7bce59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fa337a5a60 0x55fa337a5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48004/0 (socket says 172.21.15.120:48004) 2024-09-19T09:09:17.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.273+0000 7f7bce59c640 1 -- 172.21.15.120:0/3674551892 learned_addr learned my addr 172.21.15.120:0/3674551892 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:17.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.273+0000 7f7bced9d640 1 --2- 172.21.15.120:0/3283807113 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7bbc0a3240 0x7f7bbc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:17.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.273+0000 7f7bc7fff640 1 --2- 172.21.15.120:0/3674551892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa337a6380 0x55fa337b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:17.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.273+0000 7f7bced9d640 1 --2- 172.21.15.120:0/3674551892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa337b5150 0x55fa337b7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:17.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.273+0000 7f7bce59c640 1 -- 172.21.15.120:0/3674551892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa337a6380 msgr2=0x55fa337b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:17.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.273+0000 7f7bce59c640 1 --2- 172.21.15.120:0/3674551892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa337a6380 0x55fa337b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.273+0000 7f7bce59c640 1 -- 172.21.15.120:0/3674551892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa337b5150 msgr2=0x55fa337b7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:17.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.273+0000 7f7bce59c640 1 --2- 172.21.15.120:0/3674551892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa337b5150 0x55fa337b7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.273+0000 7f7bce59c640 1 -- 172.21.15.120:0/3674551892 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa336bf1f0 con 0x55fa337a5a60 2024-09-19T09:09:17.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.273+0000 7f7bced9d640 1 --2- 172.21.15.120:0/3283807113 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7bbc0a3240 0x7f7bbc0a5700 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8005fd0 tx=0x7f7bc8009b80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:17.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.273+0000 7f7bce59c640 1 --2- 172.21.15.120:0/3674551892 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fa337a5a60 0x55fa337a5e40 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f7bc002a9a0 tx=0x7f7bc0005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4700add1df28116 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:17.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.274+0000 7f7bc4ff9640 1 -- 172.21.15.120:0/3674551892 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0035030 con 0x55fa337a5a60 2024-09-19T09:09:17.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.274+0000 7f7bc4ff9640 1 -- 172.21.15.120:0/3674551892 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bc0040070 con 0x55fa337a5a60 2024-09-19T09:09:17.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.274+0000 7f7bc4ff9640 1 -- 172.21.15.120:0/3674551892 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0031830 con 0x55fa337a5a60 2024-09-19T09:09:17.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.275+0000 7f7bd6f61740 1 -- 172.21.15.120:0/3674551892 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fa337a5a60 msgr2=0x55fa337a5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:17.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.275+0000 7f7bd6f61740 1 --2- 172.21.15.120:0/3674551892 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fa337a5a60 0x55fa337a5e40 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f7bc002a9a0 tx=0x7f7bc0005dd0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.275+0000 7f7bd6f61740 1 -- 172.21.15.120:0/3674551892 shutdown_connections 2024-09-19T09:09:17.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.275+0000 7f7bd6f61740 1 --2- 172.21.15.120:0/3674551892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa337b5150 0x55fa337b7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.275+0000 7f7bd6f61740 1 --2- 172.21.15.120:0/3674551892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa337a6380 0x55fa337b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.275+0000 7f7bd6f61740 1 --2- 172.21.15.120:0/3674551892 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fa337a5a60 0x55fa337a5e40 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.275+0000 7f7bd6f61740 1 -- 172.21.15.120:0/3674551892 >> 172.21.15.120:0/3674551892 conn(0x55fa337a20f0 msgr2=0x55fa337a24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:17.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.275+0000 7f7bd6f61740 1 -- 172.21.15.120:0/3674551892 shutdown_connections 2024-09-19T09:09:17.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.275+0000 7f7bd6f61740 1 -- 172.21.15.120:0/3674551892 wait complete. 2024-09-19T09:09:17.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.275+0000 7f7bd6f61740 1 -- start start 2024-09-19T09:09:17.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.275+0000 7f7bd6f61740 1 Processor -- start 2024-09-19T09:09:17.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.276+0000 7f7bd6f61740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa337a5a60 0x55fa337b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:17.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.276+0000 7f7bd6f61740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa337a6380 0x55fa337b2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:17.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.276+0000 7f7bd6f61740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fa337b5150 0x55fa337b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:17.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.276+0000 7f7bd6f61740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fa337b7d20 con 0x55fa337b5150 2024-09-19T09:09:17.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.276+0000 7f7bd6f61740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fa336c5f10 con 0x55fa337a5a60 2024-09-19T09:09:17.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.276+0000 7f7bd6f61740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fa337b7ea0 con 0x55fa337a6380 2024-09-19T09:09:17.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.276+0000 7f7bced9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fa337b5150 0x55fa337b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:17.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.276+0000 7f7bd6f61740 10 client.0 _ll_register_callbacks cb 0x55fa3368ea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:17.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.276+0000 7f7bced9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fa337b5150 0x55fa337b2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48018/0 (socket says 172.21.15.120:48018) 2024-09-19T09:09:17.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.276+0000 7f7bced9d640 1 -- 172.21.15.120:0/546765773 learned_addr learned my addr 172.21.15.120:0/546765773 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:17.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.276+0000 7f7bce59c640 1 --2- 172.21.15.120:0/546765773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa337a5a60 0x55fa337b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:17.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.276+0000 7f7bc7fff640 1 --2- 172.21.15.120:0/546765773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa337a6380 0x55fa337b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:17.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.276+0000 7f7bced9d640 1 -- 172.21.15.120:0/546765773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa337a6380 msgr2=0x55fa337b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:17.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.276+0000 7f7bced9d640 1 --2- 172.21.15.120:0/546765773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa337a6380 0x55fa337b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.276+0000 7f7bced9d640 1 -- 172.21.15.120:0/546765773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa337a5a60 msgr2=0x55fa337b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:17.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.276+0000 7f7bced9d640 1 --2- 172.21.15.120:0/546765773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa337a5a60 0x55fa337b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.276+0000 7f7bced9d640 1 -- 172.21.15.120:0/546765773 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa3347c6a0 con 0x55fa337b5150 2024-09-19T09:09:17.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.276+0000 7f7bced9d640 1 --2- 172.21.15.120:0/546765773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fa337b5150 0x55fa337b2a70 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800fd20 tx=0x7f7bc8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c0bcbb504a8dc84 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:17.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.276+0000 7f7bb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:09:17.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.277+0000 7f7bb5ffb640 1 -- 172.21.15.120:0/546765773 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8004b20 con 0x55fa337b5150 2024-09-19T09:09:17.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.277+0000 7f7bb5ffb640 1 -- 172.21.15.120:0/546765773 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f7bc8004cc0 con 0x55fa337b5150 2024-09-19T09:09:17.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.277+0000 7f7bd6f61740 1 -- 172.21.15.120:0/546765773 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bc0040070 con 0x55fa337b5150 2024-09-19T09:09:17.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.277+0000 7f7bb5ffb640 1 -- 172.21.15.120:0/546765773 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7bc8005020 con 0x55fa337b5150 2024-09-19T09:09:17.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.277+0000 7f7bd6f61740 10 client.17670 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:17.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.277+0000 7f7bd6f61740 1 -- 172.21.15.120:0/546765773 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fa337c3980 con 0x55fa337b5150 2024-09-19T09:09:17.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.277+0000 7f7bb5ffb640 1 -- 172.21.15.120:0/546765773 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bc0040070 con 0x55fa337b5150 2024-09-19T09:09:17.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.277+0000 7f7bb5ffb640 1 -- 172.21.15.120:0/546765773 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7bc801f070 con 0x55fa337b5150 2024-09-19T09:09:17.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.277+0000 7f7bd6f61740 10 client.17670 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:17.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.277+0000 7f7bd6f61740 10 client.17670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:09:17.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.277+0000 7f7bd6f61740 20 client.17670 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:17.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.277+0000 7f7bd6f61740 1 --2- 172.21.15.120:0/546765773 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fa337c5220 0x55fa337c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:17.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.277+0000 7f7bd6f61740 4 client.17670 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:09:17.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.277+0000 7f7bd6f61740 1 -- 172.21.15.120:0/546765773 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa337187f0 con 0x55fa337c5220 2024-09-19T09:09:17.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.277+0000 7f7bce59c640 1 --2- 172.21.15.120:0/546765773 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fa337c5220 0x55fa337c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:17.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.278+0000 7f7bb5ffb640 10 client.17670 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:17.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.278+0000 7f7bce59c640 1 --2- 172.21.15.120:0/546765773 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fa337c5220 0x55fa337c76c0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bb3495c805e4ac6 server_cookie=d65ac7666a4cd26 in_seq=0 out_seq=0 2024-09-19T09:09:17.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.279+0000 7f7bb5ffb640 1 -- 172.21.15.120:0/546765773 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fa337187f0 con 0x55fa337c5220 2024-09-19T09:09:17.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.279+0000 7f7bb5ffb640 10 client.17670 handle_command_reply: tid=0 2024-09-19T09:09:17.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.279+0000 7f7bd6f61740 1 client.17670 shutdown 2024-09-19T09:09:17.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.279+0000 7f7bd6f61740 1 -- 172.21.15.120:0/546765773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fa337b5150 msgr2=0x55fa337b2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:17.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.279+0000 7f7bd6f61740 1 --2- 172.21.15.120:0/546765773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fa337b5150 0x55fa337b2a70 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800fd20 tx=0x7f7bc8004ab0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.279+0000 7f7bd6f61740 1 -- 172.21.15.120:0/546765773 shutdown_connections 2024-09-19T09:09:17.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.279+0000 7f7bd6f61740 1 --2- 172.21.15.120:0/546765773 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fa337c5220 0x55fa337c76c0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.279+0000 7f7bd6f61740 1 --2- 172.21.15.120:0/546765773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fa337b5150 0x55fa337b2a70 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.279+0000 7f7bd6f61740 1 --2- 172.21.15.120:0/546765773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa337a6380 0x55fa337b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.279+0000 7f7bd6f61740 1 --2- 172.21.15.120:0/546765773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa337a5a60 0x55fa337b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.279+0000 7f7bd6f61740 1 -- 172.21.15.120:0/546765773 >> 172.21.15.120:0/546765773 conn(0x55fa337a20f0 msgr2=0x55fa337a7e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:17.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.279+0000 7f7bb5ffb640 0 client.17670 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:17.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.279+0000 7f7bd6f61740 1 -- 172.21.15.120:0/546765773 shutdown_connections 2024-09-19T09:09:17.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.280+0000 7f7bd6f61740 1 -- 172.21.15.120:0/546765773 wait complete. 2024-09-19T09:09:17.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.280+0000 7f7bd6f61740 20 client.17670 trim_cache size 0 max 16384 2024-09-19T09:09:17.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.299+0000 7f7bd6f61740 1 Processor -- start 2024-09-19T09:09:17.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.299+0000 7f7bd6f61740 1 -- start start 2024-09-19T09:09:17.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.300+0000 7f7bd6f61740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa337a5a60 0x55fa337a7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:17.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.300+0000 7f7bd6f61740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fa337a6380 0x55fa337ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:17.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.300+0000 7f7bc7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fa337a6380 0x55fa337ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:17.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.300+0000 7f7bc7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fa337a6380 0x55fa337ccfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48022/0 (socket says 172.21.15.120:48022) 2024-09-19T09:09:17.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.300+0000 7f7bce59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa337a5a60 0x55fa337a7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:17.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.300+0000 7f7bce59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa337a5a60 0x55fa337a7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38540/0 (socket says 172.21.15.120:38540) 2024-09-19T09:09:17.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.300+0000 7f7bd6f61740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa337b5150 0x55fa337cd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:17.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.300+0000 7f7bd6f61740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01ab340 con 0x55fa337a6380 2024-09-19T09:09:17.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.300+0000 7f7bc7fff640 1 -- 172.21.15.120:0/139730009 learned_addr learned my addr 172.21.15.120:0/139730009 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:17.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.300+0000 7f7bd6f61740 1 -- 172.21.15.120:0/139730009 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01ace80 con 0x55fa337a5a60 2024-09-19T09:09:17.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.300+0000 7f7bd6f61740 1 -- 172.21.15.120:0/139730009 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01aba10 con 0x55fa337b5150 2024-09-19T09:09:17.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.300+0000 7f7bced9d640 1 --2- 172.21.15.120:0/139730009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa337b5150 0x55fa337cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:17.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.301+0000 7f7bce59c640 1 -- 172.21.15.120:0/139730009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa337b5150 msgr2=0x55fa337cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:17.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.301+0000 7f7bce59c640 1 --2- 172.21.15.120:0/139730009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa337b5150 0x55fa337cd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.301+0000 7f7bce59c640 1 -- 172.21.15.120:0/139730009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fa337a6380 msgr2=0x55fa337ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:17.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.301+0000 7f7bce59c640 1 --2- 172.21.15.120:0/139730009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fa337a6380 0x55fa337ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.301+0000 7f7bce59c640 1 -- 172.21.15.120:0/139730009 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa337a61b0 con 0x55fa337a5a60 2024-09-19T09:09:17.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.301+0000 7f7bc7fff640 1 --2- 172.21.15.120:0/139730009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fa337a6380 0x55fa337ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:17.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.301+0000 7f7bced9d640 1 --2- 172.21.15.120:0/139730009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa337b5150 0x55fa337cd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:17.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.301+0000 7f7bce59c640 1 --2- 172.21.15.120:0/139730009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa337a5a60 0x55fa337a7ca0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0005c40 tx=0x7f7bc002a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db05a9abb57980df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:17.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.301+0000 7f7bb7fff640 1 -- 172.21.15.120:0/139730009 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0035040 con 0x55fa337a5a60 2024-09-19T09:09:17.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.301+0000 7f7bb7fff640 1 -- 172.21.15.120:0/139730009 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bc0040070 con 0x55fa337a5a60 2024-09-19T09:09:17.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.301+0000 7f7bb7fff640 1 -- 172.21.15.120:0/139730009 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc004f480 con 0x55fa337a5a60 2024-09-19T09:09:17.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.302+0000 7f7bd6f61740 1 -- 172.21.15.120:0/139730009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa337a5a60 msgr2=0x55fa337a7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:17.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.302+0000 7f7bd6f61740 1 --2- 172.21.15.120:0/139730009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa337a5a60 0x55fa337a7ca0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0005c40 tx=0x7f7bc002a480 comp rx=0 tx=0).stop 2024-09-19T09:09:17.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.302+0000 7f7bd6f61740 1 -- 172.21.15.120:0/139730009 shutdown_connections 2024-09-19T09:09:17.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.302+0000 7f7bd6f61740 1 --2- 172.21.15.120:0/139730009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa337b5150 0x55fa337cd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.302+0000 7f7bd6f61740 1 --2- 172.21.15.120:0/139730009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fa337a6380 0x55fa337ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.302+0000 7f7bd6f61740 1 --2- 172.21.15.120:0/139730009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa337a5a60 0x55fa337a7ca0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.302+0000 7f7bd6f61740 1 -- 172.21.15.120:0/139730009 >> 172.21.15.120:0/139730009 conn(0x55fa337a20f0 msgr2=0x55fa337a7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:17.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.302+0000 7f7bd6f61740 1 -- 172.21.15.120:0/139730009 shutdown_connections 2024-09-19T09:09:17.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.302+0000 7f7bd6f61740 1 -- 172.21.15.120:0/139730009 wait complete. 2024-09-19T09:09:17.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.302+0000 7f7bd6f61740 1 -- start start 2024-09-19T09:09:17.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.302+0000 7f7bd6f61740 1 Processor -- start 2024-09-19T09:09:17.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.303+0000 7f7bd6f61740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa337a5a60 0x55fa337bd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:17.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.303+0000 7f7bd6f61740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fa337a6380 0x55fa337b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:17.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.303+0000 7f7bd6f61740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa337b5150 0x55fa337b91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:17.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.303+0000 7f7bd6f61740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01ac7b0 con 0x55fa337a6380 2024-09-19T09:09:17.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.303+0000 7f7bd6f61740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01ac0e0 con 0x55fa337b5150 2024-09-19T09:09:17.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.303+0000 7f7bd6f61740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01a9320 con 0x55fa337a5a60 2024-09-19T09:09:17.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.303+0000 7f7bd6f61740 10 client.0 _ll_register_callbacks cb 0x7f7bc8013dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:17.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.303+0000 7f7bc7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fa337a6380 0x55fa337b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:17.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.303+0000 7f7bc7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fa337a6380 0x55fa337b8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48026/0 (socket says 172.21.15.120:48026) 2024-09-19T09:09:17.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.303+0000 7f7bc7fff640 1 -- 172.21.15.120:0/4063651929 learned_addr learned my addr 172.21.15.120:0/4063651929 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:17.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.303+0000 7f7bced9d640 1 --2- 172.21.15.120:0/4063651929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa337b5150 0x55fa337b91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:17.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.303+0000 7f7bce59c640 1 --2- 172.21.15.120:0/4063651929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa337a5a60 0x55fa337bd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:17.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.303+0000 7f7bced9d640 1 -- 172.21.15.120:0/4063651929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa337a5a60 msgr2=0x55fa337bd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:17.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.303+0000 7f7bced9d640 1 --2- 172.21.15.120:0/4063651929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa337a5a60 0x55fa337bd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.303+0000 7f7bced9d640 1 -- 172.21.15.120:0/4063651929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fa337a6380 msgr2=0x55fa337b8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:17.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.303+0000 7f7bced9d640 1 --2- 172.21.15.120:0/4063651929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fa337a6380 0x55fa337b8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.303+0000 7f7bced9d640 1 -- 172.21.15.120:0/4063651929 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa337b99d0 con 0x55fa337b5150 2024-09-19T09:09:17.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.303+0000 7f7bce59c640 1 --2- 172.21.15.120:0/4063651929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa337a5a60 0x55fa337bd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:17.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.303+0000 7f7bc7fff640 1 --2- 172.21.15.120:0/4063651929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fa337a6380 0x55fa337b8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:17.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.304+0000 7f7bced9d640 1 --2- 172.21.15.120:0/4063651929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa337b5150 0x55fa337b91a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7bc802a700 tx=0x7f7bc802bdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7006be80d570cf72 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:17.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.304+0000 7f7bb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:09:17.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.304+0000 7f7bb5ffb640 1 -- 172.21.15.120:0/4063651929 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8011ca0 con 0x55fa337b5150 2024-09-19T09:09:17.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.304+0000 7f7bb5ffb640 1 -- 172.21.15.120:0/4063651929 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f7bc8004d30 con 0x55fa337b5150 2024-09-19T09:09:17.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.304+0000 7f7bd6f61740 1 -- 172.21.15.120:0/4063651929 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bc0040070 con 0x55fa337b5150 2024-09-19T09:09:17.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.304+0000 7f7bb5ffb640 1 -- 172.21.15.120:0/4063651929 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7bc8013360 con 0x55fa337b5150 2024-09-19T09:09:17.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.304+0000 7f7bd6f61740 10 client.26924 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:17.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.304+0000 7f7bd6f61740 1 -- 172.21.15.120:0/4063651929 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fa33830070 con 0x55fa337b5150 2024-09-19T09:09:17.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.304+0000 7f7bb5ffb640 1 -- 172.21.15.120:0/4063651929 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bc0040070 con 0x55fa337b5150 2024-09-19T09:09:17.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.305+0000 7f7bb5ffb640 1 -- 172.21.15.120:0/4063651929 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7bc801f070 con 0x55fa337b5150 2024-09-19T09:09:17.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.305+0000 7f7bd6f61740 10 client.26924 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:17.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.305+0000 7f7bd6f61740 10 client.26924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:09:17.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.305+0000 7f7bd6f61740 20 client.26924 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:17.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.305+0000 7f7bd6f61740 1 --2- 172.21.15.120:0/4063651929 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fa33830780 0x55fa33832b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:17.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.305+0000 7f7bd6f61740 4 client.26924 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:09:17.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.305+0000 7f7bd6f61740 1 -- 172.21.15.120:0/4063651929 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fa33712d00 con 0x55fa33830780 2024-09-19T09:09:17.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.305+0000 7f7bce59c640 1 --2- 172.21.15.120:0/4063651929 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fa33830780 0x55fa33832b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:17.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.305+0000 7f7bce59c640 1 --2- 172.21.15.120:0/4063651929 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fa33830780 0x55fa33832b70 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d89a4dd612727087 server_cookie=1e3f9fab4d194ac1 in_seq=0 out_seq=0 2024-09-19T09:09:17.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.305+0000 7f7bb5ffb640 10 client.26924 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:17.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.306+0000 7f7bb5ffb640 1 -- 172.21.15.120:0/4063651929 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fa33712d00 con 0x55fa33830780 2024-09-19T09:09:17.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.306+0000 7f7bb5ffb640 10 client.26924 handle_command_reply: tid=0 2024-09-19T09:09:17.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.306+0000 7f7bd6f61740 1 client.26924 shutdown 2024-09-19T09:09:17.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.307+0000 7f7bd6f61740 1 -- 172.21.15.120:0/4063651929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa337b5150 msgr2=0x55fa337b91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:17.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.307+0000 7f7bd6f61740 1 --2- 172.21.15.120:0/4063651929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa337b5150 0x55fa337b91a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7bc802a700 tx=0x7f7bc802bdf0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.307+0000 7f7bd6f61740 1 -- 172.21.15.120:0/4063651929 shutdown_connections 2024-09-19T09:09:17.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.307+0000 7f7bd6f61740 1 --2- 172.21.15.120:0/4063651929 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fa33830780 0x55fa33832b70 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.307+0000 7f7bd6f61740 1 --2- 172.21.15.120:0/4063651929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa337b5150 0x55fa337b91a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.307+0000 7f7bd6f61740 1 --2- 172.21.15.120:0/4063651929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fa337a6380 0x55fa337b8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.307+0000 7f7bb5ffb640 0 client.26924 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:17.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.307+0000 7f7bd6f61740 1 --2- 172.21.15.120:0/4063651929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa337a5a60 0x55fa337bd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.307+0000 7f7bd6f61740 1 -- 172.21.15.120:0/4063651929 >> 172.21.15.120:0/4063651929 conn(0x55fa337a20f0 msgr2=0x55fa337b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:17.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.307+0000 7f7bd6f61740 1 -- 172.21.15.120:0/4063651929 shutdown_connections 2024-09-19T09:09:17.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.307+0000 7f7bd6f61740 1 -- 172.21.15.120:0/4063651929 wait complete. 2024-09-19T09:09:17.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.307+0000 7f7bd6f61740 20 client.26924 trim_cache size 0 max 16384 2024-09-19T09:09:17.317 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:09:17.317 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:09:17.317 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "33ecd61a-606e-43f3-8ba2-afeca56720da", 2024-09-19T09:09:17.317 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:09:17.317 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:09:17.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.308+0000 7f7bd4b88640 1 -- 172.21.15.120:0/3283807113 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7bbc0a3240 msgr2=0x7f7bbc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:17.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.308+0000 7f7bd4b88640 1 --2- 172.21.15.120:0/3283807113 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7bbc0a3240 0x7f7bbc0a5700 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8005fd0 tx=0x7f7bc8009b80 comp rx=0 tx=0).stop 2024-09-19T09:09:17.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.308+0000 7f7bd4b88640 1 -- 172.21.15.120:0/3283807113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bd0102570 msgr2=0x7f7bd0198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:17.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.308+0000 7f7bd4b88640 1 --2- 172.21.15.120:0/3283807113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bd0102570 0x7f7bd0198d20 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800e9f0 tx=0x7f7bb800eeb0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.308+0000 7f7bd4b88640 1 -- 172.21.15.120:0/3283807113 shutdown_connections 2024-09-19T09:09:17.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.308+0000 7f7bd4b88640 1 --2- 172.21.15.120:0/3283807113 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7bbc0a3240 0x7f7bbc0a5700 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.308+0000 7f7bd4b88640 1 --2- 172.21.15.120:0/3283807113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7bd0102ef0 0x7f7bd019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.308+0000 7f7bd4b88640 1 --2- 172.21.15.120:0/3283807113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bd0102570 0x7f7bd0198d20 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.308+0000 7f7bd4b88640 1 --2- 172.21.15.120:0/3283807113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bd0100e20 0x7f7bd01987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:17.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.308+0000 7f7bd4b88640 1 -- 172.21.15.120:0/3283807113 >> 172.21.15.120:0/3283807113 conn(0x7f7bd00fc7d0 msgr2=0x7f7bd0103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:17.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.308+0000 7f7bd4b88640 1 -- 172.21.15.120:0/3283807113 shutdown_connections 2024-09-19T09:09:17.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:17.308+0000 7f7bd4b88640 1 -- 172.21.15.120:0/3283807113 wait complete. 2024-09-19T09:09:18.053 INFO:tasks.workunit.client.0.smithi120.stdout: 1 564936 9.52 MB/sec execute 193 sec latency 24.833 ms 2024-09-19T09:09:18.113 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:09:18.383 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:09:18.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.903+0000 7fa924a34640 1 -- 172.21.15.120:0/3046271791 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9201009d0 msgr2=0x7fa920100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:18.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.903+0000 7fa924a34640 1 --2- 172.21.15.120:0/3046271791 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9201009d0 0x7fa920100db0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fa910009900 tx=0x7fa91002a410 comp rx=0 tx=0).stop 2024-09-19T09:09:18.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.903+0000 7fa924a34640 1 -- 172.21.15.120:0/3046271791 shutdown_connections 2024-09-19T09:09:18.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.903+0000 7fa924a34640 1 --2- 172.21.15.120:0/3046271791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa92010e4f0 0x7fa920110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.903+0000 7fa924a34640 1 --2- 172.21.15.120:0/3046271791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa920101380 0x7fa92010dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.903+0000 7fa924a34640 1 --2- 172.21.15.120:0/3046271791 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9201009d0 0x7fa920100db0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.903+0000 7fa924a34640 1 -- 172.21.15.120:0/3046271791 >> 172.21.15.120:0/3046271791 conn(0x7fa9200fc7d0 msgr2=0x7fa9200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:18.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.903+0000 7fa924a34640 1 -- 172.21.15.120:0/3046271791 shutdown_connections 2024-09-19T09:09:18.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.903+0000 7fa924a34640 1 -- 172.21.15.120:0/3046271791 wait complete. 2024-09-19T09:09:18.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.904+0000 7fa924a34640 1 Processor -- start 2024-09-19T09:09:18.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.904+0000 7fa924a34640 1 -- start start 2024-09-19T09:09:18.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.904+0000 7fa924a34640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9201009d0 0x7fa9201a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:18.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.904+0000 7fa924a34640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa920101380 0x7fa9201a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:18.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.904+0000 7fa924a34640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa92010e4f0 0x7fa92019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:18.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.904+0000 7fa924a34640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa920114270 con 0x7fa9201009d0 2024-09-19T09:09:18.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.904+0000 7fa924a34640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa9201140f0 con 0x7fa920101380 2024-09-19T09:09:18.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.904+0000 7fa924a34640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9201143f0 con 0x7fa92010e4f0 2024-09-19T09:09:18.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.905+0000 7fa91e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9201009d0 0x7fa9201a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:18.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.905+0000 7fa91e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9201009d0 0x7fa9201a0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48046/0 (socket says 172.21.15.120:48046) 2024-09-19T09:09:18.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.905+0000 7fa91e59c640 1 -- 172.21.15.120:0/2573804389 learned_addr learned my addr 172.21.15.120:0/2573804389 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:18.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.905+0000 7fa91dd9b640 1 --2- 172.21.15.120:0/2573804389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa920101380 0x7fa9201a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:18.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.905+0000 7fa91ed9d640 1 --2- 172.21.15.120:0/2573804389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa92010e4f0 0x7fa92019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:18.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.905+0000 7fa91e59c640 1 -- 172.21.15.120:0/2573804389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa92010e4f0 msgr2=0x7fa92019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:18.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.905+0000 7fa91e59c640 1 --2- 172.21.15.120:0/2573804389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa92010e4f0 0x7fa92019a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.905+0000 7fa91e59c640 1 -- 172.21.15.120:0/2573804389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa920101380 msgr2=0x7fa9201a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:18.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.905+0000 7fa91e59c640 1 --2- 172.21.15.120:0/2573804389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa920101380 0x7fa9201a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.905+0000 7fa91e59c640 1 -- 172.21.15.120:0/2573804389 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92019aeb0 con 0x7fa9201009d0 2024-09-19T09:09:18.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.905+0000 7fa91dd9b640 1 --2- 172.21.15.120:0/2573804389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa920101380 0x7fa9201a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:18.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.905+0000 7fa91ed9d640 1 --2- 172.21.15.120:0/2573804389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa92010e4f0 0x7fa92019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:18.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.905+0000 7fa91e59c640 1 --2- 172.21.15.120:0/2573804389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9201009d0 0x7fa9201a0520 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fa910006010 tx=0x7fa910004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:18.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.905+0000 7fa90f7fe640 1 -- 172.21.15.120:0/2573804389 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91003f030 con 0x7fa9201009d0 2024-09-19T09:09:18.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.905+0000 7fa90f7fe640 1 -- 172.21.15.120:0/2573804389 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa910004440 con 0x7fa9201009d0 2024-09-19T09:09:18.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.905+0000 7fa924a34640 1 -- 172.21.15.120:0/2573804389 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa92019b140 con 0x7fa9201009d0 2024-09-19T09:09:18.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.905+0000 7fa90f7fe640 1 -- 172.21.15.120:0/2573804389 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910035070 con 0x7fa9201009d0 2024-09-19T09:09:18.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.905+0000 7fa924a34640 1 -- 172.21.15.120:0/2573804389 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9201a7340 con 0x7fa9201009d0 2024-09-19T09:09:18.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.908+0000 7fa90f7fe640 1 -- 172.21.15.120:0/2573804389 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa910042030 con 0x7fa9201009d0 2024-09-19T09:09:18.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.908+0000 7fa926e0d740 1 Processor -- start 2024-09-19T09:09:18.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.908+0000 7fa926e0d740 1 -- start start 2024-09-19T09:09:18.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.908+0000 7fa90f7fe640 1 --2- 172.21.15.120:0/2573804389 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9080a3240 0x7fa9080a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:18.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.909+0000 7fa926e0d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582eacbb4a0 0x5582eacbb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:18.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.909+0000 7fa926e0d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582eacbbdc0 0x5582eacc6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:18.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.909+0000 7fa926e0d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582eacc6680 0x5582eacc8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:18.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.909+0000 7fa926e0d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5582eac212e0 con 0x5582eacc6680 2024-09-19T09:09:18.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.909+0000 7fa926e0d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5582eabc8ce0 con 0x5582eacbb4a0 2024-09-19T09:09:18.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.909+0000 7fa926e0d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5582eac1d2a0 con 0x5582eacbbdc0 2024-09-19T09:09:18.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.909+0000 7fa91ed9d640 1 --2- 172.21.15.120:0/2573804389 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9080a3240 0x7fa9080a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:18.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.909+0000 7fa91ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582eacc6680 0x5582eacc8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:18.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.909+0000 7fa91dd9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582eacbbdc0 0x5582eacc6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:18.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.909+0000 7fa90f7fe640 1 -- 172.21.15.120:0/2573804389 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fa9100efc60 con 0x7fa9201009d0 2024-09-19T09:09:18.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.909+0000 7fa91ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582eacc6680 0x5582eacc8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48060/0 (socket says 172.21.15.120:48060) 2024-09-19T09:09:18.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.909+0000 7fa91ed9d640 1 -- 172.21.15.120:0/3955162403 learned_addr learned my addr 172.21.15.120:0/3955162403 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:18.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.909+0000 7fa91e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582eacbb4a0 0x5582eacbb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:18.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.909+0000 7fa91e59c640 1 -- 172.21.15.120:0/3955162403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582eacbbdc0 msgr2=0x5582eacc6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:18.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.909+0000 7fa91e59c640 1 --2- 172.21.15.120:0/3955162403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582eacbbdc0 0x5582eacc6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.909+0000 7fa91e59c640 1 -- 172.21.15.120:0/3955162403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582eacc6680 msgr2=0x5582eacc8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:18.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.909+0000 7fa91e59c640 1 --2- 172.21.15.120:0/3955162403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582eacc6680 0x5582eacc8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.909+0000 7fa91e59c640 1 -- 172.21.15.120:0/3955162403 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582eabad440 con 0x5582eacbb4a0 2024-09-19T09:09:18.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.909+0000 7fa91ed9d640 1 --2- 172.21.15.120:0/2573804389 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9080a3240 0x7fa9080a5700 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa91800f410 tx=0x7fa918011710 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:18.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.909+0000 7fa91e59c640 1 --2- 172.21.15.120:0/3955162403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582eacbb4a0 0x5582eacbb880 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa9100ed760 tx=0x7fa9100bbc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36dc8312b2d15811 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:18.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.909+0000 7fa90cff9640 1 -- 172.21.15.120:0/3955162403 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910049040 con 0x5582eacbb4a0 2024-09-19T09:09:18.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.910+0000 7fa90cff9640 1 -- 172.21.15.120:0/3955162403 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9100bc330 con 0x5582eacbb4a0 2024-09-19T09:09:18.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.910+0000 7fa90cff9640 1 -- 172.21.15.120:0/3955162403 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9100b89a0 con 0x5582eacbb4a0 2024-09-19T09:09:18.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.910+0000 7fa926e0d740 1 -- 172.21.15.120:0/3955162403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582eacbb4a0 msgr2=0x5582eacbb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:18.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.910+0000 7fa926e0d740 1 --2- 172.21.15.120:0/3955162403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582eacbb4a0 0x5582eacbb880 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa9100ed760 tx=0x7fa9100bbc90 comp rx=0 tx=0).stop 2024-09-19T09:09:18.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.910+0000 7fa926e0d740 1 -- 172.21.15.120:0/3955162403 shutdown_connections 2024-09-19T09:09:18.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.910+0000 7fa926e0d740 1 --2- 172.21.15.120:0/3955162403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582eacc6680 0x5582eacc8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.910+0000 7fa926e0d740 1 --2- 172.21.15.120:0/3955162403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582eacbbdc0 0x5582eacc6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.910+0000 7fa926e0d740 1 --2- 172.21.15.120:0/3955162403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582eacbb4a0 0x5582eacbb880 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.910+0000 7fa926e0d740 1 -- 172.21.15.120:0/3955162403 >> 172.21.15.120:0/3955162403 conn(0x5582eacb7a30 msgr2=0x5582eacb7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:18.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.910+0000 7fa926e0d740 1 -- 172.21.15.120:0/3955162403 shutdown_connections 2024-09-19T09:09:18.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.910+0000 7fa926e0d740 1 -- 172.21.15.120:0/3955162403 wait complete. 2024-09-19T09:09:18.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.911+0000 7fa926e0d740 1 -- start start 2024-09-19T09:09:18.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.911+0000 7fa926e0d740 1 Processor -- start 2024-09-19T09:09:18.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.911+0000 7fa926e0d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582eacbb4a0 0x5582eacc5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:18.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.911+0000 7fa926e0d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582eacbbdc0 0x5582eacc3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:18.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.911+0000 7fa926e0d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582eacc6680 0x5582eacc3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:18.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.911+0000 7fa926e0d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5582eacc9010 con 0x5582eacbb4a0 2024-09-19T09:09:18.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.911+0000 7fa91e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582eacbb4a0 0x5582eacc5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:18.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.911+0000 7fa91e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582eacbb4a0 0x5582eacc5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48076/0 (socket says 172.21.15.120:48076) 2024-09-19T09:09:18.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.911+0000 7fa91e59c640 1 -- 172.21.15.120:0/3260775736 learned_addr learned my addr 172.21.15.120:0/3260775736 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:18.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.911+0000 7fa926e0d740 1 -- 172.21.15.120:0/3260775736 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5582eabdc050 con 0x5582eacbbdc0 2024-09-19T09:09:18.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.911+0000 7fa91dd9b640 1 --2- 172.21.15.120:0/3260775736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582eacbbdc0 0x5582eacc3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:18.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.911+0000 7fa926e0d740 1 -- 172.21.15.120:0/3260775736 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5582eacc9190 con 0x5582eacc6680 2024-09-19T09:09:18.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.911+0000 7fa91ed9d640 1 --2- 172.21.15.120:0/3260775736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582eacc6680 0x5582eacc3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:18.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.911+0000 7fa926e0d740 10 client.0 _ll_register_callbacks cb 0x5582eac1c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:18.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.912+0000 7fa91e59c640 1 -- 172.21.15.120:0/3260775736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582eacc6680 msgr2=0x5582eacc3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:18.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.912+0000 7fa91e59c640 1 --2- 172.21.15.120:0/3260775736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582eacc6680 0x5582eacc3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.912+0000 7fa91e59c640 1 -- 172.21.15.120:0/3260775736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582eacbbdc0 msgr2=0x5582eacc3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:18.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.912+0000 7fa91e59c640 1 --2- 172.21.15.120:0/3260775736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582eacbbdc0 0x5582eacc3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.912+0000 7fa91e59c640 1 -- 172.21.15.120:0/3260775736 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582ea9923c0 con 0x5582eacbb4a0 2024-09-19T09:09:18.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.912+0000 7fa91ed9d640 1 --2- 172.21.15.120:0/3260775736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582eacc6680 0x5582eacc3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:09:18.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.912+0000 7fa91e59c640 1 --2- 172.21.15.120:0/3260775736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582eacbb4a0 0x5582eacc5f70 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fa91003d040 tx=0x7fa9100ed8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8814b00bee6798 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:18.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.912+0000 7fa905ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:09:18.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.912+0000 7fa905ffb640 1 -- 172.21.15.120:0/3260775736 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910049070 con 0x5582eacbb4a0 2024-09-19T09:09:18.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.912+0000 7fa905ffb640 1 -- 172.21.15.120:0/3260775736 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fa9100d5190 con 0x5582eacbb4a0 2024-09-19T09:09:18.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.912+0000 7fa926e0d740 1 -- 172.21.15.120:0/3260775736 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9100bc330 con 0x5582eacbb4a0 2024-09-19T09:09:18.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.912+0000 7fa905ffb640 1 -- 172.21.15.120:0/3260775736 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa9100c4ef0 con 0x5582eacbb4a0 2024-09-19T09:09:18.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.912+0000 7fa926e0d740 10 client.17688 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:18.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.912+0000 7fa926e0d740 1 -- 172.21.15.120:0/3260775736 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5582eacc9010 con 0x5582eacbb4a0 2024-09-19T09:09:18.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.912+0000 7fa905ffb640 1 -- 172.21.15.120:0/3260775736 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9100db120 con 0x5582eacbb4a0 2024-09-19T09:09:18.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.912+0000 7fa905ffb640 1 -- 172.21.15.120:0/3260775736 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa9100c4a60 con 0x5582eacbb4a0 2024-09-19T09:09:18.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.912+0000 7fa926e0d740 10 client.17688 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:18.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.913+0000 7fa926e0d740 10 client.17688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:09:18.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.913+0000 7fa926e0d740 20 client.17688 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:18.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.913+0000 7fa926e0d740 1 --2- 172.21.15.120:0/3260775736 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5582eacd5a30 0x5582eacd7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:18.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.913+0000 7fa926e0d740 4 client.17688 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:09:18.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.913+0000 7fa926e0d740 1 -- 172.21.15.120:0/3260775736 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa9100bc330 con 0x5582eacd5a30 2024-09-19T09:09:18.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.913+0000 7fa91dd9b640 1 --2- 172.21.15.120:0/3260775736 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5582eacd5a30 0x5582eacd7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:18.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.913+0000 7fa905ffb640 10 client.17688 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:18.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.913+0000 7fa91dd9b640 1 --2- 172.21.15.120:0/3260775736 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5582eacd5a30 0x5582eacd7e20 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57501018356290be server_cookie=9e05085740bb9d6d in_seq=0 out_seq=0 2024-09-19T09:09:18.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.914+0000 7fa905ffb640 1 -- 172.21.15.120:0/3260775736 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fa9100bc330 con 0x5582eacd5a30 2024-09-19T09:09:18.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.914+0000 7fa905ffb640 10 client.17688 handle_command_reply: tid=0 2024-09-19T09:09:18.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.914+0000 7fa926e0d740 1 client.17688 shutdown 2024-09-19T09:09:18.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.914+0000 7fa926e0d740 1 -- 172.21.15.120:0/3260775736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582eacbb4a0 msgr2=0x5582eacc5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:18.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.914+0000 7fa926e0d740 1 --2- 172.21.15.120:0/3260775736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582eacbb4a0 0x5582eacc5f70 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fa91003d040 tx=0x7fa9100ed8d0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.914+0000 7fa926e0d740 1 -- 172.21.15.120:0/3260775736 shutdown_connections 2024-09-19T09:09:18.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.914+0000 7fa926e0d740 1 --2- 172.21.15.120:0/3260775736 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5582eacd5a30 0x5582eacd7e20 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.914+0000 7fa926e0d740 1 --2- 172.21.15.120:0/3260775736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582eacc6680 0x5582eacc3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.914+0000 7fa905ffb640 0 client.17688 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:18.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.914+0000 7fa926e0d740 1 --2- 172.21.15.120:0/3260775736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582eacbbdc0 0x5582eacc3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.914+0000 7fa926e0d740 1 --2- 172.21.15.120:0/3260775736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582eacbb4a0 0x5582eacc5f70 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.914+0000 7fa926e0d740 1 -- 172.21.15.120:0/3260775736 >> 172.21.15.120:0/3260775736 conn(0x5582eacb7a30 msgr2=0x5582eacbd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:18.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.914+0000 7fa926e0d740 1 -- 172.21.15.120:0/3260775736 shutdown_connections 2024-09-19T09:09:18.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.914+0000 7fa926e0d740 1 -- 172.21.15.120:0/3260775736 wait complete. 2024-09-19T09:09:18.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.914+0000 7fa926e0d740 20 client.17688 trim_cache size 0 max 16384 2024-09-19T09:09:18.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.932+0000 7fa926e0d740 1 Processor -- start 2024-09-19T09:09:18.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.932+0000 7fa926e0d740 1 -- start start 2024-09-19T09:09:18.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.932+0000 7fa926e0d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582eacbb4a0 0x5582eacd21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:18.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.932+0000 7fa926e0d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582eacbbdc0 0x5582eacbe030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:18.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.932+0000 7fa926e0d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582eacc6680 0x5582eacdd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:18.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.932+0000 7fa926e0d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9201af7c0 con 0x5582eacbbdc0 2024-09-19T09:09:18.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.932+0000 7fa926e0d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa9201b1300 con 0x5582eacbb4a0 2024-09-19T09:09:18.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.932+0000 7fa926e0d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9201afe90 con 0x5582eacc6680 2024-09-19T09:09:18.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.933+0000 7fa91dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582eacbbdc0 0x5582eacbe030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:18.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.933+0000 7fa91ed9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582eacc6680 0x5582eacdd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:18.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.933+0000 7fa91e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582eacbb4a0 0x5582eacd21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:18.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.933+0000 7fa91ed9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582eacc6680 0x5582eacdd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:59722/0 (socket says 172.21.15.120:59722) 2024-09-19T09:09:18.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.933+0000 7fa91e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582eacbb4a0 0x5582eacd21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38612/0 (socket says 172.21.15.120:38612) 2024-09-19T09:09:18.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.933+0000 7fa91dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582eacbbdc0 0x5582eacbe030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48078/0 (socket says 172.21.15.120:48078) 2024-09-19T09:09:18.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.933+0000 7fa91ed9d640 1 -- 172.21.15.120:0/584001264 learned_addr learned my addr 172.21.15.120:0/584001264 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:18.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.933+0000 7fa91ed9d640 1 -- 172.21.15.120:0/584001264 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582eacbb4a0 msgr2=0x5582eacd21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:18.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.933+0000 7fa91ed9d640 1 --2- 172.21.15.120:0/584001264 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582eacbb4a0 0x5582eacd21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.933+0000 7fa91ed9d640 1 -- 172.21.15.120:0/584001264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582eacbbdc0 msgr2=0x5582eacbe030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:18.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.933+0000 7fa91ed9d640 1 --2- 172.21.15.120:0/584001264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582eacbbdc0 0x5582eacbe030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.933+0000 7fa91ed9d640 1 -- 172.21.15.120:0/584001264 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582eacc62e0 con 0x5582eacc6680 2024-09-19T09:09:18.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.933+0000 7fa91dd9b640 1 --2- 172.21.15.120:0/584001264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582eacbbdc0 0x5582eacbe030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:18.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.933+0000 7fa91ed9d640 1 --2- 172.21.15.120:0/584001264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582eacc6680 0x5582eacdd610 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa91800fca0 tx=0x7fa918021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48a2305ffda780b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:18.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.933+0000 7fa907fff640 1 -- 172.21.15.120:0/584001264 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa918010b30 con 0x5582eacc6680 2024-09-19T09:09:18.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.933+0000 7fa907fff640 1 -- 172.21.15.120:0/584001264 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa918010cd0 con 0x5582eacc6680 2024-09-19T09:09:18.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.933+0000 7fa907fff640 1 -- 172.21.15.120:0/584001264 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa918005830 con 0x5582eacc6680 2024-09-19T09:09:18.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.934+0000 7fa926e0d740 1 -- 172.21.15.120:0/584001264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582eacc6680 msgr2=0x5582eacdd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:18.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.934+0000 7fa926e0d740 1 --2- 172.21.15.120:0/584001264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582eacc6680 0x5582eacdd610 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa91800fca0 tx=0x7fa918021000 comp rx=0 tx=0).stop 2024-09-19T09:09:18.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.934+0000 7fa926e0d740 1 -- 172.21.15.120:0/584001264 shutdown_connections 2024-09-19T09:09:18.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.934+0000 7fa926e0d740 1 --2- 172.21.15.120:0/584001264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582eacc6680 0x5582eacdd610 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.934+0000 7fa926e0d740 1 --2- 172.21.15.120:0/584001264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582eacbbdc0 0x5582eacbe030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.934+0000 7fa926e0d740 1 --2- 172.21.15.120:0/584001264 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582eacbb4a0 0x5582eacd21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.934+0000 7fa926e0d740 1 -- 172.21.15.120:0/584001264 >> 172.21.15.120:0/584001264 conn(0x5582eacb7a30 msgr2=0x5582eacbd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:18.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.934+0000 7fa926e0d740 1 -- 172.21.15.120:0/584001264 shutdown_connections 2024-09-19T09:09:18.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.934+0000 7fa926e0d740 1 -- 172.21.15.120:0/584001264 wait complete. 2024-09-19T09:09:18.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.934+0000 7fa926e0d740 1 -- start start 2024-09-19T09:09:18.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.935+0000 7fa926e0d740 1 Processor -- start 2024-09-19T09:09:18.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.935+0000 7fa926e0d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582eacbb4a0 0x5582eacce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:18.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.935+0000 7fa926e0d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582eacbbdc0 0x5582eacc9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:18.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.935+0000 7fa926e0d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582eacc6680 0x5582eacca140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:18.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.935+0000 7fa926e0d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9201b0c30 con 0x5582eacbbdc0 2024-09-19T09:09:18.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.935+0000 7fa926e0d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa9201b0560 con 0x5582eacbb4a0 2024-09-19T09:09:18.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.935+0000 7fa926e0d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9201ad780 con 0x5582eacc6680 2024-09-19T09:09:18.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.935+0000 7fa926e0d740 10 client.0 _ll_register_callbacks cb 0x7fa918011b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:18.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.935+0000 7fa91e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582eacbb4a0 0x5582eacce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:18.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.935+0000 7fa91dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582eacbbdc0 0x5582eacc9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:18.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.935+0000 7fa91ed9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582eacc6680 0x5582eacca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:18.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.935+0000 7fa91dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582eacbbdc0 0x5582eacc9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48090/0 (socket says 172.21.15.120:48090) 2024-09-19T09:09:18.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.935+0000 7fa91dd9b640 1 -- 172.21.15.120:0/1435704807 learned_addr learned my addr 172.21.15.120:0/1435704807 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:18.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.935+0000 7fa91dd9b640 1 -- 172.21.15.120:0/1435704807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582eacc6680 msgr2=0x5582eacca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:18.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.935+0000 7fa91dd9b640 1 --2- 172.21.15.120:0/1435704807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582eacc6680 0x5582eacca140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.935+0000 7fa91dd9b640 1 -- 172.21.15.120:0/1435704807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582eacbb4a0 msgr2=0x5582eacce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:18.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.935+0000 7fa91dd9b640 1 --2- 172.21.15.120:0/1435704807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582eacbb4a0 0x5582eacce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.935+0000 7fa91dd9b640 1 -- 172.21.15.120:0/1435704807 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582eacca7b0 con 0x5582eacbbdc0 2024-09-19T09:09:18.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.935+0000 7fa91ed9d640 1 --2- 172.21.15.120:0/1435704807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582eacc6680 0x5582eacca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:18.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.936+0000 7fa91dd9b640 1 --2- 172.21.15.120:0/1435704807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582eacbbdc0 0x5582eacc9c00 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x5582eacb8890 tx=0x7fa914012b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c18f5a324eb734d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:18.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.936+0000 7fa905ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:09:18.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.936+0000 7fa905ffb640 1 -- 172.21.15.120:0/1435704807 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91402c060 con 0x5582eacbbdc0 2024-09-19T09:09:18.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.936+0000 7fa905ffb640 1 -- 172.21.15.120:0/1435704807 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fa914004440 con 0x5582eacbbdc0 2024-09-19T09:09:18.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.936+0000 7fa926e0d740 1 -- 172.21.15.120:0/1435704807 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa918010cd0 con 0x5582eacbbdc0 2024-09-19T09:09:18.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.936+0000 7fa905ffb640 1 -- 172.21.15.120:0/1435704807 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa914012e30 con 0x5582eacbbdc0 2024-09-19T09:09:18.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.936+0000 7fa926e0d740 10 client.17694 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:18.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.936+0000 7fa926e0d740 1 -- 172.21.15.120:0/1435704807 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5582ead404f0 con 0x5582eacbbdc0 2024-09-19T09:09:18.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.936+0000 7fa905ffb640 1 -- 172.21.15.120:0/1435704807 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa918010cd0 con 0x5582eacbbdc0 2024-09-19T09:09:18.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.936+0000 7fa905ffb640 1 -- 172.21.15.120:0/1435704807 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa914031070 con 0x5582eacbbdc0 2024-09-19T09:09:18.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.936+0000 7fa926e0d740 10 client.17694 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:18.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.936+0000 7fa926e0d740 10 client.17694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:09:18.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.936+0000 7fa926e0d740 20 client.17694 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:18.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.936+0000 7fa926e0d740 1 --2- 172.21.15.120:0/1435704807 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5582eacd5a30 0x5582ead42c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:18.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.936+0000 7fa926e0d740 4 client.17694 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:09:18.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.936+0000 7fa926e0d740 1 -- 172.21.15.120:0/1435704807 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5582eacc3af0 con 0x5582eacd5a30 2024-09-19T09:09:18.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.937+0000 7fa91e59c640 1 --2- 172.21.15.120:0/1435704807 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5582eacd5a30 0x5582ead42c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:18.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.937+0000 7fa91e59c640 1 --2- 172.21.15.120:0/1435704807 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5582eacd5a30 0x5582ead42c90 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9f131f66c4d04cc server_cookie=e454bff5debdd9cf in_seq=0 out_seq=0 2024-09-19T09:09:18.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.937+0000 7fa905ffb640 10 client.17694 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:18.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.937+0000 7fa905ffb640 1 -- 172.21.15.120:0/1435704807 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5582eacc3af0 con 0x5582eacd5a30 2024-09-19T09:09:18.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.937+0000 7fa905ffb640 10 client.17694 handle_command_reply: tid=0 2024-09-19T09:09:18.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.937+0000 7fa926e0d740 1 client.17694 shutdown 2024-09-19T09:09:18.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.937+0000 7fa926e0d740 1 -- 172.21.15.120:0/1435704807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582eacbbdc0 msgr2=0x5582eacc9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:18.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.937+0000 7fa926e0d740 1 --2- 172.21.15.120:0/1435704807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582eacbbdc0 0x5582eacc9c00 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x5582eacb8890 tx=0x7fa914012b80 comp rx=0 tx=0).stop 2024-09-19T09:09:18.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.938+0000 7fa926e0d740 1 -- 172.21.15.120:0/1435704807 shutdown_connections 2024-09-19T09:09:18.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.938+0000 7fa926e0d740 1 --2- 172.21.15.120:0/1435704807 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5582eacd5a30 0x5582ead42c90 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.938+0000 7fa926e0d740 1 --2- 172.21.15.120:0/1435704807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5582eacc6680 0x5582eacca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.938+0000 7fa926e0d740 1 --2- 172.21.15.120:0/1435704807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582eacbbdc0 0x5582eacc9c00 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.938+0000 7fa926e0d740 1 --2- 172.21.15.120:0/1435704807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582eacbb4a0 0x5582eacce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.938+0000 7fa926e0d740 1 -- 172.21.15.120:0/1435704807 >> 172.21.15.120:0/1435704807 conn(0x5582eacb7a30 msgr2=0x5582eacc19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:18.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.938+0000 7fa905ffb640 0 client.17694 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:18.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.938+0000 7fa926e0d740 1 -- 172.21.15.120:0/1435704807 shutdown_connections 2024-09-19T09:09:18.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.938+0000 7fa926e0d740 1 -- 172.21.15.120:0/1435704807 wait complete. 2024-09-19T09:09:18.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.938+0000 7fa926e0d740 20 client.17694 trim_cache size 0 max 16384 2024-09-19T09:09:18.944 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:09:18.944 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:09:18.945 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:09:18.945 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:09:18.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.938+0000 7fa924a34640 1 -- 172.21.15.120:0/2573804389 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9080a3240 msgr2=0x7fa9080a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:18.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.938+0000 7fa924a34640 1 --2- 172.21.15.120:0/2573804389 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9080a3240 0x7fa9080a5700 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa91800f410 tx=0x7fa918011710 comp rx=0 tx=0).stop 2024-09-19T09:09:18.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.938+0000 7fa924a34640 1 -- 172.21.15.120:0/2573804389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9201009d0 msgr2=0x7fa9201a0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:18.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.938+0000 7fa924a34640 1 --2- 172.21.15.120:0/2573804389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9201009d0 0x7fa9201a0520 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fa910006010 tx=0x7fa910004040 comp rx=0 tx=0).stop 2024-09-19T09:09:18.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.938+0000 7fa924a34640 1 -- 172.21.15.120:0/2573804389 shutdown_connections 2024-09-19T09:09:18.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.938+0000 7fa924a34640 1 --2- 172.21.15.120:0/2573804389 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9080a3240 0x7fa9080a5700 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.938+0000 7fa924a34640 1 --2- 172.21.15.120:0/2573804389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa92010e4f0 0x7fa92019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.938+0000 7fa924a34640 1 --2- 172.21.15.120:0/2573804389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa920101380 0x7fa9201a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.938+0000 7fa924a34640 1 --2- 172.21.15.120:0/2573804389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9201009d0 0x7fa9201a0520 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:18.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.939+0000 7fa924a34640 1 -- 172.21.15.120:0/2573804389 >> 172.21.15.120:0/2573804389 conn(0x7fa9200fc7d0 msgr2=0x7fa9200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:18.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.939+0000 7fa924a34640 1 -- 172.21.15.120:0/2573804389 shutdown_connections 2024-09-19T09:09:18.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:18.939+0000 7fa924a34640 1 -- 172.21.15.120:0/2573804389 wait complete. 2024-09-19T09:09:19.053 INFO:tasks.workunit.client.0.smithi120.stdout: 1 567555 9.53 MB/sec execute 194 sec latency 39.593 ms 2024-09-19T09:09:19.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:18 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:09:19.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:18 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:09:19.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:18 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:09:19.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:18 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:09:19.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:18 smithi120 ceph-mon[31150]: pgmap v535: 97 pgs: 97 active+clean; 37 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.1 MiB/s wr, 449 op/s 2024-09-19T09:09:19.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:18 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:09:19.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:18 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:09:19.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:18 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:09:19.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:18 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:09:19.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:18 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:09:19.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:18 smithi196 ceph-mon[35150]: pgmap v535: 97 pgs: 97 active+clean; 37 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.1 MiB/s wr, 449 op/s 2024-09-19T09:09:19.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:18 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:09:19.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:18 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:09:19.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:18 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:09:19.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:18 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:09:19.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:18 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:09:19.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:18 smithi153 ceph-mon[35071]: pgmap v535: 97 pgs: 97 active+clean; 37 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.1 MiB/s wr, 449 op/s 2024-09-19T09:09:19.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:18 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:09:19.782 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:09:19.782 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:09:20.053 INFO:tasks.workunit.client.0.smithi120.stdout: 1 569387 9.53 MB/sec execute 195 sec latency 60.412 ms 2024-09-19T09:09:20.082 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:09:20.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.566+0000 7fe894a9e640 1 -- 172.21.15.120:0/2295462391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe890101df0 msgr2=0x7fe8901021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:20.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.566+0000 7fe894a9e640 1 --2- 172.21.15.120:0/2295462391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe890101df0 0x7fe8901021d0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fe880009900 tx=0x7fe88002a410 comp rx=0 tx=0).stop 2024-09-19T09:09:20.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.566+0000 7fe894a9e640 1 -- 172.21.15.120:0/2295462391 shutdown_connections 2024-09-19T09:09:20.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.566+0000 7fe894a9e640 1 --2- 172.21.15.120:0/2295462391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe890106f20 0x7fe89010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:20.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.566+0000 7fe894a9e640 1 --2- 172.21.15.120:0/2295462391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8901027a0 0x7fe8901067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:20.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.566+0000 7fe894a9e640 1 --2- 172.21.15.120:0/2295462391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe890101df0 0x7fe8901021d0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:20.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.566+0000 7fe894a9e640 1 -- 172.21.15.120:0/2295462391 >> 172.21.15.120:0/2295462391 conn(0x7fe8900fd8f0 msgr2=0x7fe8900ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:20.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.566+0000 7fe894a9e640 1 -- 172.21.15.120:0/2295462391 shutdown_connections 2024-09-19T09:09:20.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.566+0000 7fe894a9e640 1 -- 172.21.15.120:0/2295462391 wait complete. 2024-09-19T09:09:20.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.567+0000 7fe894a9e640 1 Processor -- start 2024-09-19T09:09:20.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.567+0000 7fe894a9e640 1 -- start start 2024-09-19T09:09:20.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.567+0000 7fe894a9e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe890101df0 0x7fe8901984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:20.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.567+0000 7fe894a9e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8901027a0 0x7fe890198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:20.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.567+0000 7fe894a9e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe890106f20 0x7fe89019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:20.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.567+0000 7fe894a9e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe890111f80 con 0x7fe890101df0 2024-09-19T09:09:20.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.567+0000 7fe894a9e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe890111e00 con 0x7fe890106f20 2024-09-19T09:09:20.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.567+0000 7fe894a9e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe890112100 con 0x7fe8901027a0 2024-09-19T09:09:20.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.568+0000 7fe88e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe890101df0 0x7fe8901984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:20.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.568+0000 7fe88ed9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe890106f20 0x7fe89019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:20.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.568+0000 7fe88e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe890101df0 0x7fe8901984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48108/0 (socket says 172.21.15.120:48108) 2024-09-19T09:09:20.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.568+0000 7fe885d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8901027a0 0x7fe890198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:20.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.568+0000 7fe88e59c640 1 -- 172.21.15.120:0/486357372 learned_addr learned my addr 172.21.15.120:0/486357372 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:20.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.568+0000 7fe88e59c640 1 -- 172.21.15.120:0/486357372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8901027a0 msgr2=0x7fe890198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:20.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.568+0000 7fe88e59c640 1 --2- 172.21.15.120:0/486357372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8901027a0 0x7fe890198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:20.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.568+0000 7fe88e59c640 1 -- 172.21.15.120:0/486357372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe890106f20 msgr2=0x7fe89019cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:20.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.568+0000 7fe88e59c640 1 --2- 172.21.15.120:0/486357372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe890106f20 0x7fe89019cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:20.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.568+0000 7fe88e59c640 1 -- 172.21.15.120:0/486357372 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89019d470 con 0x7fe890101df0 2024-09-19T09:09:20.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.568+0000 7fe88ed9d640 1 --2- 172.21.15.120:0/486357372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe890106f20 0x7fe89019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:20.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.568+0000 7fe88e59c640 1 --2- 172.21.15.120:0/486357372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe890101df0 0x7fe8901984c0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fe880006010 tx=0x7fe880004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:20.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.568+0000 7fe887fff640 1 -- 172.21.15.120:0/486357372 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88003f030 con 0x7fe890101df0 2024-09-19T09:09:20.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.568+0000 7fe894a9e640 1 -- 172.21.15.120:0/486357372 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe89019d700 con 0x7fe890101df0 2024-09-19T09:09:20.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.568+0000 7fe894a9e640 1 -- 172.21.15.120:0/486357372 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8901a5040 con 0x7fe890101df0 2024-09-19T09:09:20.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.568+0000 7fe887fff640 1 -- 172.21.15.120:0/486357372 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe880004440 con 0x7fe890101df0 2024-09-19T09:09:20.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.568+0000 7fe887fff640 1 -- 172.21.15.120:0/486357372 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe880049060 con 0x7fe890101df0 2024-09-19T09:09:20.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.569+0000 7fe894a9e640 1 -- 172.21.15.120:0/486357372 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe870005290 con 0x7fe890101df0 2024-09-19T09:09:20.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.570+0000 7fe887fff640 1 -- 172.21.15.120:0/486357372 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe880042030 con 0x7fe890101df0 2024-09-19T09:09:20.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.571+0000 7fe887fff640 1 --2- 172.21.15.120:0/486357372 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe87c0a3240 0x7fe87c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:20.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.571+0000 7fe885d9b640 1 --2- 172.21.15.120:0/486357372 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe87c0a3240 0x7fe87c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:20.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.571+0000 7fe887fff640 1 -- 172.21.15.120:0/486357372 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fe8800ef8d0 con 0x7fe890101df0 2024-09-19T09:09:20.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.572+0000 7fe885d9b640 1 --2- 172.21.15.120:0/486357372 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe87c0a3240 0x7fe87c0a5700 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe8900ff910 tx=0x7fe878005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:20.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.576+0000 7fe887fff640 1 -- 172.21.15.120:0/486357372 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe8800bbfa0 con 0x7fe890101df0 2024-09-19T09:09:20.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.801+0000 7fe894a9e640 1 -- 172.21.15.120:0/486357372 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe870005580 con 0x7fe890101df0 2024-09-19T09:09:20.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.801+0000 7fe887fff640 1 -- 172.21.15.120:0/486357372 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11985 (secure 0 0 0) 0x7fe8800c02a0 con 0x7fe890101df0 2024-09-19T09:09:20.803 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:09:20.803 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":30,"btime":"2024-09-19T09:07:14:812065+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:07:14.812046+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26351":{"gid":26351,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/364941779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":364941779},{"type":"v1","addr":"172.21.15.153:6835","nonce":364941779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:09:20.803 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 30 2024-09-19T09:09:20.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.803+0000 7fe894a9e640 1 -- 172.21.15.120:0/486357372 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe87c0a3240 msgr2=0x7fe87c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:20.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.803+0000 7fe894a9e640 1 --2- 172.21.15.120:0/486357372 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe87c0a3240 0x7fe87c0a5700 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe8900ff910 tx=0x7fe878005f90 comp rx=0 tx=0).stop 2024-09-19T09:09:20.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.803+0000 7fe894a9e640 1 -- 172.21.15.120:0/486357372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe890101df0 msgr2=0x7fe8901984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:20.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.803+0000 7fe894a9e640 1 --2- 172.21.15.120:0/486357372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe890101df0 0x7fe8901984c0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fe880006010 tx=0x7fe880004040 comp rx=0 tx=0).stop 2024-09-19T09:09:20.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.803+0000 7fe894a9e640 1 -- 172.21.15.120:0/486357372 shutdown_connections 2024-09-19T09:09:20.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.803+0000 7fe894a9e640 1 --2- 172.21.15.120:0/486357372 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe87c0a3240 0x7fe87c0a5700 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:20.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.803+0000 7fe894a9e640 1 --2- 172.21.15.120:0/486357372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe890106f20 0x7fe89019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:20.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.803+0000 7fe894a9e640 1 --2- 172.21.15.120:0/486357372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8901027a0 0x7fe890198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:20.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.803+0000 7fe894a9e640 1 --2- 172.21.15.120:0/486357372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe890101df0 0x7fe8901984c0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:20.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.803+0000 7fe894a9e640 1 -- 172.21.15.120:0/486357372 >> 172.21.15.120:0/486357372 conn(0x7fe8900fd8f0 msgr2=0x7fe8900feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:20.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.803+0000 7fe894a9e640 1 -- 172.21.15.120:0/486357372 shutdown_connections 2024-09-19T09:09:20.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:20.803+0000 7fe894a9e640 1 -- 172.21.15.120:0/486357372 wait complete. 2024-09-19T09:09:21.053 INFO:tasks.workunit.client.0.smithi120.stdout: 1 572327 9.57 MB/sec execute 196 sec latency 76.337 ms 2024-09-19T09:09:21.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:20 smithi120 ceph-mon[31150]: pgmap v536: 97 pgs: 97 active+clean; 44 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.5 MiB/s wr, 443 op/s 2024-09-19T09:09:21.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:20 smithi196 ceph-mon[35150]: pgmap v536: 97 pgs: 97 active+clean; 44 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.5 MiB/s wr, 443 op/s 2024-09-19T09:09:21.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:20 smithi153 ceph-mon[35071]: pgmap v536: 97 pgs: 97 active+clean; 44 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.5 MiB/s wr, 443 op/s 2024-09-19T09:09:21.831 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:09:22.053 INFO:tasks.workunit.client.0.smithi120.stdout: 1 575595 9.61 MB/sec execute 197 sec latency 14.920 ms 2024-09-19T09:09:22.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:21 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/486357372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:09:22.098 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:09:22.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:21 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/486357372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:09:22.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:21 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/486357372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:09:22.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.594+0000 7f8428f57640 1 -- 172.21.15.120:0/3214400788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8424102570 msgr2=0x7f84241029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:22.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.594+0000 7f8428f57640 1 --2- 172.21.15.120:0/3214400788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8424102570 0x7f84241029b0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f840c009900 tx=0x7f840c02a480 comp rx=0 tx=0).stop 2024-09-19T09:09:22.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.594+0000 7f8428f57640 1 -- 172.21.15.120:0/3214400788 shutdown_connections 2024-09-19T09:09:22.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.594+0000 7f8428f57640 1 --2- 172.21.15.120:0/3214400788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8424102ef0 0x7f842410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.594+0000 7f8428f57640 1 --2- 172.21.15.120:0/3214400788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8424102570 0x7f84241029b0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.594+0000 7f8428f57640 1 --2- 172.21.15.120:0/3214400788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8424100e20 0x7f8424101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.594+0000 7f8428f57640 1 -- 172.21.15.120:0/3214400788 >> 172.21.15.120:0/3214400788 conn(0x7f84240fc7d0 msgr2=0x7f84240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:22.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.594+0000 7f8428f57640 1 -- 172.21.15.120:0/3214400788 shutdown_connections 2024-09-19T09:09:22.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.594+0000 7f8428f57640 1 -- 172.21.15.120:0/3214400788 wait complete. 2024-09-19T09:09:22.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.595+0000 7f8428f57640 1 Processor -- start 2024-09-19T09:09:22.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.595+0000 7f8428f57640 1 -- start start 2024-09-19T09:09:22.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.595+0000 7f8428f57640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8424100e20 0x7f8424198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:22.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.595+0000 7f8428f57640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8424102570 0x7f8424198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:22.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.595+0000 7f8428f57640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8424102ef0 0x7f842419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:22.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.595+0000 7f8428f57640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8424110fa0 con 0x7f8424102570 2024-09-19T09:09:22.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.595+0000 7f8428f57640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8424110e20 con 0x7f8424100e20 2024-09-19T09:09:22.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.595+0000 7f8428f57640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8424111120 con 0x7f8424102ef0 2024-09-19T09:09:22.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.595+0000 7f842259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8424100e20 0x7f8424198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:22.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.595+0000 7f841bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8424102570 0x7f8424198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:22.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.595+0000 7f842259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8424100e20 0x7f8424198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59590/0 (socket says 172.21.15.120:59590) 2024-09-19T09:09:22.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.595+0000 7f842259c640 1 -- 172.21.15.120:0/3072163160 learned_addr learned my addr 172.21.15.120:0/3072163160 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:22.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.595+0000 7f8422d9d640 1 --2- 172.21.15.120:0/3072163160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8424102ef0 0x7f842419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:22.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.596+0000 7f841bfff640 1 -- 172.21.15.120:0/3072163160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8424102ef0 msgr2=0x7f842419ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:22.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.596+0000 7f841bfff640 1 --2- 172.21.15.120:0/3072163160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8424102ef0 0x7f842419ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.596+0000 7f841bfff640 1 -- 172.21.15.120:0/3072163160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8424100e20 msgr2=0x7f8424198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:22.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.596+0000 7f841bfff640 1 --2- 172.21.15.120:0/3072163160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8424100e20 0x7f8424198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.596+0000 7f841bfff640 1 -- 172.21.15.120:0/3072163160 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842419d510 con 0x7f8424102570 2024-09-19T09:09:22.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.596+0000 7f8422d9d640 1 --2- 172.21.15.120:0/3072163160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8424102ef0 0x7f842419ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:22.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.596+0000 7f842259c640 1 --2- 172.21.15.120:0/3072163160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8424100e20 0x7f8424198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:22.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.596+0000 7f841bfff640 1 --2- 172.21.15.120:0/3072163160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8424102570 0x7f8424198aa0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f840c02a9a0 tx=0x7f840c003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:22.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.596+0000 7f841b7fe640 1 -- 172.21.15.120:0/3072163160 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c03f070 con 0x7f8424102570 2024-09-19T09:09:22.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.596+0000 7f8428f57640 1 -- 172.21.15.120:0/3072163160 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f842419d7a0 con 0x7f8424102570 2024-09-19T09:09:22.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.596+0000 7f841b7fe640 1 -- 172.21.15.120:0/3072163160 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f840c031c20 con 0x7f8424102570 2024-09-19T09:09:22.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.596+0000 7f841b7fe640 1 -- 172.21.15.120:0/3072163160 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c035030 con 0x7f8424102570 2024-09-19T09:09:22.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.596+0000 7f8428f57640 1 -- 172.21.15.120:0/3072163160 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84241a50e0 con 0x7f8424102570 2024-09-19T09:09:22.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.598+0000 7f841b7fe640 1 -- 172.21.15.120:0/3072163160 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f840c042070 con 0x7f8424102570 2024-09-19T09:09:22.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.599+0000 7f842b330740 1 Processor -- start 2024-09-19T09:09:22.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.599+0000 7f842b330740 1 -- start start 2024-09-19T09:09:22.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.599+0000 7f842b330740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cb32d0b40 0x559cb32cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:22.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.599+0000 7f842b330740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cb32cf5d0 0x559cb32cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:22.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.599+0000 7f842b330740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cb32de780 0x559cb32e0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:22.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.599+0000 7f842b330740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559cb32352f0 con 0x559cb32de780 2024-09-19T09:09:22.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.599+0000 7f842b330740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559cb31dccf0 con 0x559cb32d0b40 2024-09-19T09:09:22.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.599+0000 7f842b330740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559cb32312b0 con 0x559cb32cf5d0 2024-09-19T09:09:22.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.599+0000 7f841b7fe640 1 --2- 172.21.15.120:0/3072163160 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f84100a3240 0x7f84100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:22.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.599+0000 7f842259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cb32cf5d0 0x559cb32cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:22.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.599+0000 7f8422d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cb32d0b40 0x559cb32cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:22.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.599+0000 7f842259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cb32cf5d0 0x559cb32cfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:45566/0 (socket says 172.21.15.120:45566) 2024-09-19T09:09:22.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.599+0000 7f842259c640 1 -- 172.21.15.120:0/1005014100 learned_addr learned my addr 172.21.15.120:0/1005014100 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:22.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.599+0000 7f841bfff640 1 --2- 172.21.15.120:0/1005014100 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cb32de780 0x559cb32e0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:22.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.599+0000 7f8422d9d640 1 --2- 172.21.15.120:0/3072163160 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f84100a3240 0x7f84100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:22.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.599+0000 7f841b7fe640 1 -- 172.21.15.120:0/3072163160 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f840c0efec0 con 0x7f8424102570 2024-09-19T09:09:22.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.600+0000 7f842259c640 1 -- 172.21.15.120:0/1005014100 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cb32d0b40 msgr2=0x559cb32cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:22.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.600+0000 7f842259c640 1 --2- 172.21.15.120:0/1005014100 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cb32d0b40 0x559cb32cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.600+0000 7f842259c640 1 -- 172.21.15.120:0/1005014100 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cb32de780 msgr2=0x559cb32e0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:22.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.600+0000 7f842259c640 1 --2- 172.21.15.120:0/1005014100 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cb32de780 0x559cb32e0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.600+0000 7f842259c640 1 -- 172.21.15.120:0/1005014100 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cb31c13a0 con 0x559cb32cf5d0 2024-09-19T09:09:22.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.600+0000 7f8422d9d640 1 --2- 172.21.15.120:0/1005014100 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cb32d0b40 0x559cb32cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:22.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.600+0000 7f841bfff640 1 --2- 172.21.15.120:0/1005014100 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cb32de780 0x559cb32e0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:09:22.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.600+0000 7f842259c640 1 --2- 172.21.15.120:0/1005014100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cb32cf5d0 0x559cb32cfa10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8414011650 tx=0x7f8414009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5b8bf6c715f98a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:22.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.600+0000 7f8422d9d640 1 --2- 172.21.15.120:0/3072163160 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f84100a3240 0x7f84100a5700 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f841c0099e0 tx=0x7f841c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:22.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.600+0000 7f8418ff9640 1 -- 172.21.15.120:0/1005014100 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841401e070 con 0x559cb32cf5d0 2024-09-19T09:09:22.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.600+0000 7f8418ff9640 1 -- 172.21.15.120:0/1005014100 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f841400fba0 con 0x559cb32cf5d0 2024-09-19T09:09:22.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.600+0000 7f8418ff9640 1 -- 172.21.15.120:0/1005014100 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8414019c70 con 0x559cb32cf5d0 2024-09-19T09:09:22.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.601+0000 7f842b330740 1 -- 172.21.15.120:0/1005014100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cb32cf5d0 msgr2=0x559cb32cfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:22.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.601+0000 7f842b330740 1 --2- 172.21.15.120:0/1005014100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cb32cf5d0 0x559cb32cfa10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8414011650 tx=0x7f8414009f90 comp rx=0 tx=0).stop 2024-09-19T09:09:22.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.601+0000 7f842b330740 1 -- 172.21.15.120:0/1005014100 shutdown_connections 2024-09-19T09:09:22.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.601+0000 7f842b330740 1 --2- 172.21.15.120:0/1005014100 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cb32de780 0x559cb32e0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.601+0000 7f842b330740 1 --2- 172.21.15.120:0/1005014100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cb32cf5d0 0x559cb32cfa10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.601+0000 7f842b330740 1 --2- 172.21.15.120:0/1005014100 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cb32d0b40 0x559cb32cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.601+0000 7f842b330740 1 -- 172.21.15.120:0/1005014100 >> 172.21.15.120:0/1005014100 conn(0x559cb3298310 msgr2=0x559cb32cbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:22.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.601+0000 7f842b330740 1 -- 172.21.15.120:0/1005014100 shutdown_connections 2024-09-19T09:09:22.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.601+0000 7f842b330740 1 -- 172.21.15.120:0/1005014100 wait complete. 2024-09-19T09:09:22.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.601+0000 7f842b330740 1 -- start start 2024-09-19T09:09:22.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.601+0000 7f842b330740 1 Processor -- start 2024-09-19T09:09:22.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.601+0000 7f842b330740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cb32cf5d0 0x559cb32de170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:22.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.602+0000 7f842b330740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cb32d0b40 0x559cb32db420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:22.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.602+0000 7f842b330740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cb32de780 0x559cb32dba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:22.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.602+0000 7f842b330740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559cb32e1480 con 0x559cb32cf5d0 2024-09-19T09:09:22.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.602+0000 7f842b330740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559cb31f0060 con 0x559cb32d0b40 2024-09-19T09:09:22.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.602+0000 7f842b330740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559cb32e1600 con 0x559cb32de780 2024-09-19T09:09:22.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.602+0000 7f842b330740 10 client.0 _ll_register_callbacks cb 0x559cb2f438c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:22.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.602+0000 7f842259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cb32cf5d0 0x559cb32de170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:22.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.602+0000 7f842259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cb32cf5d0 0x559cb32de170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35866/0 (socket says 172.21.15.120:35866) 2024-09-19T09:09:22.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.602+0000 7f842259c640 1 -- 172.21.15.120:0/563879467 learned_addr learned my addr 172.21.15.120:0/563879467 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:22.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.602+0000 7f8422d9d640 1 --2- 172.21.15.120:0/563879467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cb32de780 0x559cb32dba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:22.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.602+0000 7f841bfff640 1 --2- 172.21.15.120:0/563879467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cb32d0b40 0x559cb32db420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:22.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.602+0000 7f842259c640 1 -- 172.21.15.120:0/563879467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cb32de780 msgr2=0x559cb32dba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:22.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.602+0000 7f842259c640 1 --2- 172.21.15.120:0/563879467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cb32de780 0x559cb32dba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.602+0000 7f842259c640 1 -- 172.21.15.120:0/563879467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cb32d0b40 msgr2=0x559cb32db420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:22.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.602+0000 7f842259c640 1 --2- 172.21.15.120:0/563879467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cb32d0b40 0x559cb32db420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.602+0000 7f842259c640 1 -- 172.21.15.120:0/563879467 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cb2fa63c0 con 0x559cb32cf5d0 2024-09-19T09:09:22.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.602+0000 7f8422d9d640 1 --2- 172.21.15.120:0/563879467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cb32de780 0x559cb32dba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:22.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.602+0000 7f842259c640 1 --2- 172.21.15.120:0/563879467 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cb32cf5d0 0x559cb32de170 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f8414019850 tx=0x7f84140056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2555c3a3ae15c87 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:22.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.602+0000 7f8409ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:09:22.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.602+0000 7f8409ffb640 1 -- 172.21.15.120:0/563879467 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841401e040 con 0x559cb32cf5d0 2024-09-19T09:09:22.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.602+0000 7f842b330740 1 -- 172.21.15.120:0/563879467 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f841400fba0 con 0x559cb32cf5d0 2024-09-19T09:09:22.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.602+0000 7f8409ffb640 1 -- 172.21.15.120:0/563879467 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f8414009cc0 con 0x559cb32cf5d0 2024-09-19T09:09:22.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.602+0000 7f8409ffb640 1 -- 172.21.15.120:0/563879467 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f841400fd50 con 0x559cb32cf5d0 2024-09-19T09:09:22.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.603+0000 7f842b330740 10 client.17712 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:22.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.603+0000 7f842b330740 1 -- 172.21.15.120:0/563879467 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559cb32ece90 con 0x559cb32cf5d0 2024-09-19T09:09:22.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.603+0000 7f8409ffb640 1 -- 172.21.15.120:0/563879467 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f841400fba0 con 0x559cb32cf5d0 2024-09-19T09:09:22.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.603+0000 7f8409ffb640 1 -- 172.21.15.120:0/563879467 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8414021070 con 0x559cb32cf5d0 2024-09-19T09:09:22.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.603+0000 7f842b330740 10 client.17712 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:22.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.603+0000 7f842b330740 10 client.17712 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:09:22.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.603+0000 7f842b330740 20 client.17712 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:22.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.603+0000 7f842b330740 1 --2- 172.21.15.120:0/563879467 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x559cb32ee610 0x559cb32f0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:22.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.603+0000 7f842b330740 4 client.17712 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:09:22.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.603+0000 7f842b330740 1 -- 172.21.15.120:0/563879467 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cb3242860 con 0x559cb32ee610 2024-09-19T09:09:22.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.603+0000 7f841bfff640 1 --2- 172.21.15.120:0/563879467 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x559cb32ee610 0x559cb32f0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:22.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.604+0000 7f8409ffb640 10 client.17712 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:09:22.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.604+0000 7f841bfff640 1 --2- 172.21.15.120:0/563879467 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x559cb32ee610 0x559cb32f0a60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cba5395cea208d43 server_cookie=ab4e70ca2866b54c in_seq=0 out_seq=0 2024-09-19T09:09:22.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.604+0000 7f8409ffb640 1 -- 172.21.15.120:0/563879467 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f840c0a1e80 con 0x559cb32ee610 2024-09-19T09:09:22.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.604+0000 7f8409ffb640 10 client.17712 handle_command_reply: tid=0 2024-09-19T09:09:22.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.604+0000 7f842b330740 1 client.17712 shutdown 2024-09-19T09:09:22.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.605+0000 7f842b330740 1 -- 172.21.15.120:0/563879467 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cb32cf5d0 msgr2=0x559cb32de170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:22.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.605+0000 7f842b330740 1 --2- 172.21.15.120:0/563879467 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cb32cf5d0 0x559cb32de170 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f8414019850 tx=0x7f84140056d0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.605+0000 7f842b330740 1 -- 172.21.15.120:0/563879467 shutdown_connections 2024-09-19T09:09:22.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.605+0000 7f842b330740 1 --2- 172.21.15.120:0/563879467 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x559cb32ee610 0x559cb32f0a60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.605+0000 7f842b330740 1 --2- 172.21.15.120:0/563879467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cb32de780 0x559cb32dba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.605+0000 7f842b330740 1 --2- 172.21.15.120:0/563879467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cb32d0b40 0x559cb32db420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.605+0000 7f842b330740 1 --2- 172.21.15.120:0/563879467 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cb32cf5d0 0x559cb32de170 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.605+0000 7f842b330740 1 -- 172.21.15.120:0/563879467 >> 172.21.15.120:0/563879467 conn(0x559cb3298310 msgr2=0x559cb32d1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:22.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.605+0000 7f8409ffb640 0 client.17712 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:09:22.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.605+0000 7f842b330740 1 -- 172.21.15.120:0/563879467 shutdown_connections 2024-09-19T09:09:22.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.605+0000 7f842b330740 1 -- 172.21.15.120:0/563879467 wait complete. 2024-09-19T09:09:22.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.605+0000 7f842b330740 20 client.17712 trim_cache size 0 max 16384 2024-09-19T09:09:22.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.618+0000 7f842b330740 1 Processor -- start 2024-09-19T09:09:22.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.618+0000 7f842b330740 1 -- start start 2024-09-19T09:09:22.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.618+0000 7f842b330740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cb32cf5d0 0x559cb32cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:22.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.618+0000 7f842b330740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cb32d0b40 0x559cb32db3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:22.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.618+0000 7f842b330740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cb32de780 0x559cb32ea720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:22.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.618+0000 7f842b330740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f84241ab3c0 con 0x559cb32cf5d0 2024-09-19T09:09:22.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.618+0000 7f842b330740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f84241acf00 con 0x559cb32de780 2024-09-19T09:09:22.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.618+0000 7f842b330740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84241aba90 con 0x559cb32d0b40 2024-09-19T09:09:22.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.618+0000 7f842259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cb32cf5d0 0x559cb32cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:22.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.619+0000 7f842259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cb32cf5d0 0x559cb32cbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35876/0 (socket says 172.21.15.120:35876) 2024-09-19T09:09:22.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.619+0000 7f842259c640 1 -- 172.21.15.120:0/1114177692 learned_addr learned my addr 172.21.15.120:0/1114177692 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:22.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.619+0000 7f841bfff640 1 --2- 172.21.15.120:0/1114177692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cb32d0b40 0x559cb32db3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:22.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.619+0000 7f8422d9d640 1 --2- 172.21.15.120:0/1114177692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cb32de780 0x559cb32ea720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:22.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.619+0000 7f842259c640 1 -- 172.21.15.120:0/1114177692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cb32d0b40 msgr2=0x559cb32db3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:22.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.619+0000 7f842259c640 1 --2- 172.21.15.120:0/1114177692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cb32d0b40 0x559cb32db3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.619+0000 7f842259c640 1 -- 172.21.15.120:0/1114177692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cb32de780 msgr2=0x559cb32ea720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:22.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.619+0000 7f842259c640 1 --2- 172.21.15.120:0/1114177692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cb32de780 0x559cb32ea720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.619+0000 7f842259c640 1 -- 172.21.15.120:0/1114177692 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cb32d9970 con 0x559cb32cf5d0 2024-09-19T09:09:22.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.619+0000 7f8422d9d640 1 --2- 172.21.15.120:0/1114177692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cb32de780 0x559cb32ea720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:09:22.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.619+0000 7f842259c640 1 --2- 172.21.15.120:0/1114177692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cb32cf5d0 0x559cb32cbe10 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f8414000bc0 tx=0x7f8414003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace8ad1b8bd9db00 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:22.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.619+0000 7f840bfff640 1 -- 172.21.15.120:0/1114177692 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841401e040 con 0x559cb32cf5d0 2024-09-19T09:09:22.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.619+0000 7f840bfff640 1 -- 172.21.15.120:0/1114177692 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8414027070 con 0x559cb32cf5d0 2024-09-19T09:09:22.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.619+0000 7f840bfff640 1 -- 172.21.15.120:0/1114177692 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84140198b0 con 0x559cb32cf5d0 2024-09-19T09:09:22.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.620+0000 7f842b330740 1 -- 172.21.15.120:0/1114177692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cb32cf5d0 msgr2=0x559cb32cbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:22.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.620+0000 7f842b330740 1 --2- 172.21.15.120:0/1114177692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cb32cf5d0 0x559cb32cbe10 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f8414000bc0 tx=0x7f8414003c00 comp rx=0 tx=0).stop 2024-09-19T09:09:22.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.620+0000 7f842b330740 1 -- 172.21.15.120:0/1114177692 shutdown_connections 2024-09-19T09:09:22.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.620+0000 7f842b330740 1 --2- 172.21.15.120:0/1114177692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cb32de780 0x559cb32ea720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.620+0000 7f842b330740 1 --2- 172.21.15.120:0/1114177692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cb32d0b40 0x559cb32db3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.620+0000 7f842b330740 1 --2- 172.21.15.120:0/1114177692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cb32cf5d0 0x559cb32cbe10 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.620+0000 7f842b330740 1 -- 172.21.15.120:0/1114177692 >> 172.21.15.120:0/1114177692 conn(0x559cb3298310 msgr2=0x559cb32cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:22.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.620+0000 7f842b330740 1 -- 172.21.15.120:0/1114177692 shutdown_connections 2024-09-19T09:09:22.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.620+0000 7f842b330740 1 -- 172.21.15.120:0/1114177692 wait complete. 2024-09-19T09:09:22.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.620+0000 7f842b330740 1 -- start start 2024-09-19T09:09:22.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.620+0000 7f842b330740 1 Processor -- start 2024-09-19T09:09:22.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.621+0000 7f842b330740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cb32cf5d0 0x559cb32e1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:22.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.621+0000 7f842b330740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cb32d0b40 0x559cb32e1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:22.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.621+0000 7f842b330740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cb32de780 0x559cb32e22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:22.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.621+0000 7f842b330740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f84241ac830 con 0x559cb32cf5d0 2024-09-19T09:09:22.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.621+0000 7f842b330740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f84241ac160 con 0x559cb32d0b40 2024-09-19T09:09:22.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.621+0000 7f842b330740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84241a9380 con 0x559cb32de780 2024-09-19T09:09:22.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.621+0000 7f842b330740 10 client.0 _ll_register_callbacks cb 0x7f841401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:22.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.621+0000 7f842259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cb32cf5d0 0x559cb32e1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:22.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.621+0000 7f842259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cb32cf5d0 0x559cb32e1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35882/0 (socket says 172.21.15.120:35882) 2024-09-19T09:09:22.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.621+0000 7f842259c640 1 -- 172.21.15.120:0/934086357 learned_addr learned my addr 172.21.15.120:0/934086357 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:22.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.621+0000 7f841bfff640 1 --2- 172.21.15.120:0/934086357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cb32d0b40 0x559cb32e1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:22.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.621+0000 7f8422d9d640 1 --2- 172.21.15.120:0/934086357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cb32de780 0x559cb32e22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:22.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.621+0000 7f842259c640 1 -- 172.21.15.120:0/934086357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cb32de780 msgr2=0x559cb32e22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:22.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.621+0000 7f842259c640 1 --2- 172.21.15.120:0/934086357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cb32de780 0x559cb32e22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.621+0000 7f842259c640 1 -- 172.21.15.120:0/934086357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cb32d0b40 msgr2=0x559cb32e1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:22.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.621+0000 7f842259c640 1 --2- 172.21.15.120:0/934086357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cb32d0b40 0x559cb32e1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.621+0000 7f842259c640 1 -- 172.21.15.120:0/934086357 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cb3357440 con 0x559cb32cf5d0 2024-09-19T09:09:22.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.621+0000 7f842259c640 1 --2- 172.21.15.120:0/934086357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cb32cf5d0 0x559cb32e1850 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f841400fb40 tx=0x7f8414005360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4811ad31e0e6fec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:22.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.621+0000 7f8409ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:09:22.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.621+0000 7f8409ffb640 1 -- 172.21.15.120:0/934086357 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841403b070 con 0x559cb32cf5d0 2024-09-19T09:09:22.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.621+0000 7f8409ffb640 1 -- 172.21.15.120:0/934086357 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f8414019a30 con 0x559cb32cf5d0 2024-09-19T09:09:22.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.621+0000 7f842b330740 1 -- 172.21.15.120:0/934086357 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8414027070 con 0x559cb32cf5d0 2024-09-19T09:09:22.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.622+0000 7f8409ffb640 1 -- 172.21.15.120:0/934086357 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f841401e040 con 0x559cb32cf5d0 2024-09-19T09:09:22.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.622+0000 7f842b330740 10 client.17718 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:22.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.622+0000 7f842b330740 1 -- 172.21.15.120:0/934086357 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559cb3358740 con 0x559cb32cf5d0 2024-09-19T09:09:22.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.622+0000 7f8409ffb640 1 -- 172.21.15.120:0/934086357 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8414027070 con 0x559cb32cf5d0 2024-09-19T09:09:22.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.622+0000 7f8409ffb640 1 -- 172.21.15.120:0/934086357 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f841403e020 con 0x559cb32cf5d0 2024-09-19T09:09:22.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.622+0000 7f842b330740 10 client.17718 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:22.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.622+0000 7f842b330740 10 client.17718 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:09:22.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.622+0000 7f842b330740 20 client.17718 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:22.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.622+0000 7f842b330740 1 --2- 172.21.15.120:0/934086357 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x559cb3358df0 0x559cb335b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:22.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.622+0000 7f842b330740 4 client.17718 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:09:22.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.622+0000 7f842b330740 1 -- 172.21.15.120:0/934086357 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559cb323cd70 con 0x559cb3358df0 2024-09-19T09:09:22.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.622+0000 7f841bfff640 1 --2- 172.21.15.120:0/934086357 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x559cb3358df0 0x559cb335b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:22.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.623+0000 7f8409ffb640 10 client.17718 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:09:22.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.623+0000 7f841bfff640 1 --2- 172.21.15.120:0/934086357 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x559cb3358df0 0x559cb335b210 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=82b1754f9655e915 server_cookie=4dfaacafd0d55176 in_seq=0 out_seq=0 2024-09-19T09:09:22.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.623+0000 7f8409ffb640 1 -- 172.21.15.120:0/934086357 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559cb323cd70 con 0x559cb3358df0 2024-09-19T09:09:22.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.623+0000 7f8409ffb640 10 client.17718 handle_command_reply: tid=0 2024-09-19T09:09:22.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.623+0000 7f842b330740 1 client.17718 shutdown 2024-09-19T09:09:22.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.623+0000 7f842b330740 1 -- 172.21.15.120:0/934086357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cb32cf5d0 msgr2=0x559cb32e1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:22.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.623+0000 7f842b330740 1 --2- 172.21.15.120:0/934086357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cb32cf5d0 0x559cb32e1850 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f841400fb40 tx=0x7f8414005360 comp rx=0 tx=0).stop 2024-09-19T09:09:22.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.623+0000 7f842b330740 1 -- 172.21.15.120:0/934086357 shutdown_connections 2024-09-19T09:09:22.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.623+0000 7f842b330740 1 --2- 172.21.15.120:0/934086357 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x559cb3358df0 0x559cb335b210 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.623+0000 7f842b330740 1 --2- 172.21.15.120:0/934086357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559cb32de780 0x559cb32e22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.623+0000 7f842b330740 1 --2- 172.21.15.120:0/934086357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cb32d0b40 0x559cb32e1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.623+0000 7f842b330740 1 --2- 172.21.15.120:0/934086357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cb32cf5d0 0x559cb32e1850 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.623+0000 7f842b330740 1 -- 172.21.15.120:0/934086357 >> 172.21.15.120:0/934086357 conn(0x559cb3298310 msgr2=0x559cb32cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:22.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.623+0000 7f8409ffb640 0 client.17718 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:09:22.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.623+0000 7f842b330740 1 -- 172.21.15.120:0/934086357 shutdown_connections 2024-09-19T09:09:22.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.623+0000 7f842b330740 1 -- 172.21.15.120:0/934086357 wait complete. 2024-09-19T09:09:22.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.623+0000 7f842b330740 20 client.17718 trim_cache size 0 max 16384 2024-09-19T09:09:22.644 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:09:22.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.624+0000 7f8428f57640 1 -- 172.21.15.120:0/3072163160 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f84100a3240 msgr2=0x7f84100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:22.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.624+0000 7f8428f57640 1 --2- 172.21.15.120:0/3072163160 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f84100a3240 0x7f84100a5700 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f841c0099e0 tx=0x7f841c009500 comp rx=0 tx=0).stop 2024-09-19T09:09:22.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.624+0000 7f8428f57640 1 -- 172.21.15.120:0/3072163160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8424102570 msgr2=0x7f8424198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:22.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.624+0000 7f8428f57640 1 --2- 172.21.15.120:0/3072163160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8424102570 0x7f8424198aa0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f840c02a9a0 tx=0x7f840c003b20 comp rx=0 tx=0).stop 2024-09-19T09:09:22.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.624+0000 7f8428f57640 1 -- 172.21.15.120:0/3072163160 shutdown_connections 2024-09-19T09:09:22.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.624+0000 7f8428f57640 1 --2- 172.21.15.120:0/3072163160 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f84100a3240 0x7f84100a5700 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.624+0000 7f8428f57640 1 --2- 172.21.15.120:0/3072163160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8424102ef0 0x7f842419ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.624+0000 7f8428f57640 1 --2- 172.21.15.120:0/3072163160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8424102570 0x7f8424198aa0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.624+0000 7f8428f57640 1 --2- 172.21.15.120:0/3072163160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8424100e20 0x7f8424198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:22.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.624+0000 7f8428f57640 1 -- 172.21.15.120:0/3072163160 >> 172.21.15.120:0/3072163160 conn(0x7f84240fc7d0 msgr2=0x7f842406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:22.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.624+0000 7f8428f57640 1 -- 172.21.15.120:0/3072163160 shutdown_connections 2024-09-19T09:09:22.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:22.624+0000 7f8428f57640 1 -- 172.21.15.120:0/3072163160 wait complete. 2024-09-19T09:09:23.053 INFO:tasks.workunit.client.0.smithi120.stdout: 1 578213 9.63 MB/sec execute 198 sec latency 56.383 ms 2024-09-19T09:09:23.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:22 smithi120 ceph-mon[31150]: pgmap v537: 97 pgs: 97 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.8 MiB/s wr, 498 op/s 2024-09-19T09:09:23.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:22 smithi196 ceph-mon[35150]: pgmap v537: 97 pgs: 97 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.8 MiB/s wr, 498 op/s 2024-09-19T09:09:23.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:22 smithi153 ceph-mon[35071]: pgmap v537: 97 pgs: 97 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.8 MiB/s wr, 498 op/s 2024-09-19T09:09:23.443 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:09:23.711 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:09:24.053 INFO:tasks.workunit.client.0.smithi120.stdout: 1 580536 9.65 MB/sec execute 199 sec latency 16.771 ms 2024-09-19T09:09:24.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.222+0000 7ff8f3e28640 1 -- 172.21.15.120:0/3210566192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8ec100aa0 msgr2=0x7ff8ec10ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:24.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.222+0000 7ff8f3e28640 1 --2- 172.21.15.120:0/3210566192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8ec100aa0 0x7ff8ec10ea20 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009ca0 tx=0x7ff8e402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.222+0000 7ff8f3e28640 1 -- 172.21.15.120:0/3210566192 shutdown_connections 2024-09-19T09:09:24.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.222+0000 7ff8f3e28640 1 --2- 172.21.15.120:0/3210566192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8ec100aa0 0x7ff8ec10ea20 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.222+0000 7ff8f3e28640 1 --2- 172.21.15.120:0/3210566192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8ec100100 0x7ff8ec100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.222+0000 7ff8f3e28640 1 --2- 172.21.15.120:0/3210566192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8ec106b80 0x7ff8ec106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.222+0000 7ff8f3e28640 1 -- 172.21.15.120:0/3210566192 >> 172.21.15.120:0/3210566192 conn(0x7ff8ec077230 msgr2=0x7ff8ec0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:24.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.222+0000 7ff8f3e28640 1 -- 172.21.15.120:0/3210566192 shutdown_connections 2024-09-19T09:09:24.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.222+0000 7ff8f3e28640 1 -- 172.21.15.120:0/3210566192 wait complete. 2024-09-19T09:09:24.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.222+0000 7ff8f3e28640 1 Processor -- start 2024-09-19T09:09:24.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.222+0000 7ff8f3e28640 1 -- start start 2024-09-19T09:09:24.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.223+0000 7ff8f3e28640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8ec100100 0x7ff8ec198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:24.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.223+0000 7ff8f3e28640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8ec100aa0 0x7ff8ec198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:24.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.223+0000 7ff8f3e28640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8ec106b80 0x7ff8ec19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:24.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.223+0000 7ff8f3e28640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec112050 con 0x7ff8ec106b80 2024-09-19T09:09:24.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.223+0000 7ff8f3e28640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec111ed0 con 0x7ff8ec100100 2024-09-19T09:09:24.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.223+0000 7ff8f3e28640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec1121d0 con 0x7ff8ec100aa0 2024-09-19T09:09:24.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.223+0000 7ff8f23c5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8ec106b80 0x7ff8ec19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:24.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.223+0000 7ff8f1bc4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8ec100100 0x7ff8ec198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:24.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.223+0000 7ff8f23c5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8ec106b80 0x7ff8ec19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35896/0 (socket says 172.21.15.120:35896) 2024-09-19T09:09:24.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.223+0000 7ff8f1bc4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8ec100100 0x7ff8ec198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59644/0 (socket says 172.21.15.120:59644) 2024-09-19T09:09:24.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.223+0000 7ff8f23c5640 1 -- 172.21.15.120:0/1337844453 learned_addr learned my addr 172.21.15.120:0/1337844453 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:24.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.223+0000 7ff8f13c3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8ec100aa0 0x7ff8ec198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:24.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.223+0000 7ff8f23c5640 1 -- 172.21.15.120:0/1337844453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8ec100aa0 msgr2=0x7ff8ec198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:24.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.223+0000 7ff8f23c5640 1 --2- 172.21.15.120:0/1337844453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8ec100aa0 0x7ff8ec198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.223+0000 7ff8f23c5640 1 -- 172.21.15.120:0/1337844453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8ec100100 msgr2=0x7ff8ec198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:24.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.223+0000 7ff8f23c5640 1 --2- 172.21.15.120:0/1337844453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8ec100100 0x7ff8ec198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.223+0000 7ff8f23c5640 1 -- 172.21.15.120:0/1337844453 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ec19d400 con 0x7ff8ec106b80 2024-09-19T09:09:24.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.223+0000 7ff8f23c5640 1 --2- 172.21.15.120:0/1337844453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8ec106b80 0x7ff8ec19cd20 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7ff8e40319f0 tx=0x7ff8e4031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:24.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.224+0000 7ff8e2ffd640 1 -- 172.21.15.120:0/1337844453 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4003c30 con 0x7ff8ec106b80 2024-09-19T09:09:24.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.224+0000 7ff8e2ffd640 1 -- 172.21.15.120:0/1337844453 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8e4003dd0 con 0x7ff8ec106b80 2024-09-19T09:09:24.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.224+0000 7ff8f3e28640 1 -- 172.21.15.120:0/1337844453 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8ec19d690 con 0x7ff8ec106b80 2024-09-19T09:09:24.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.224+0000 7ff8e2ffd640 1 -- 172.21.15.120:0/1337844453 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e402fdd0 con 0x7ff8ec106b80 2024-09-19T09:09:24.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.224+0000 7ff8f3e28640 1 -- 172.21.15.120:0/1337844453 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8ec1a4fd0 con 0x7ff8ec106b80 2024-09-19T09:09:24.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.227+0000 7ff8e2ffd640 1 -- 172.21.15.120:0/1337844453 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7ff8e4004050 con 0x7ff8ec106b80 2024-09-19T09:09:24.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.227+0000 7ff8f6201740 1 Processor -- start 2024-09-19T09:09:24.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.227+0000 7ff8f6201740 1 -- start start 2024-09-19T09:09:24.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.227+0000 7ff8f6201740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a9f4cb40 0x5647a9f4b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:24.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.227+0000 7ff8f6201740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647a9f4b5d0 0x5647a9f4ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:24.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.227+0000 7ff8f6201740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a9f56370 0x5647a9f58760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:24.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.227+0000 7ff8f6201740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5647a9eb12f0 con 0x5647a9f56370 2024-09-19T09:09:24.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.227+0000 7ff8f6201740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5647a9e58cf0 con 0x5647a9f4cb40 2024-09-19T09:09:24.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.227+0000 7ff8f6201740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5647a9ead2b0 con 0x5647a9f4b5d0 2024-09-19T09:09:24.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.227+0000 7ff8f13c3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a9f4cb40 0x5647a9f4b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:24.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.227+0000 7ff8f1bc4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647a9f4b5d0 0x5647a9f4ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:24.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.227+0000 7ff8f1bc4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647a9f4b5d0 0x5647a9f4ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:45610/0 (socket says 172.21.15.120:45610) 2024-09-19T09:09:24.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.227+0000 7ff8f13c3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a9f4cb40 0x5647a9f4b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59654/0 (socket says 172.21.15.120:59654) 2024-09-19T09:09:24.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.227+0000 7ff8f1bc4640 1 -- 172.21.15.120:0/1341548262 learned_addr learned my addr 172.21.15.120:0/1341548262 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:24.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.227+0000 7ff8f13c3640 1 --2- 172.21.15.120:0/1341548262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a9f56370 0x5647a9f58760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:24.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.228+0000 7ff8e2ffd640 1 --2- 172.21.15.120:0/1337844453 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff8d80a32c0 0x7ff8d80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:24.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.228+0000 7ff8f1bc4640 1 -- 172.21.15.120:0/1341548262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a9f4cb40 msgr2=0x5647a9f4b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:24.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.228+0000 7ff8f1bc4640 1 --2- 172.21.15.120:0/1341548262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a9f4cb40 0x5647a9f4b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.228+0000 7ff8f1bc4640 1 -- 172.21.15.120:0/1341548262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a9f56370 msgr2=0x5647a9f58760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:24.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.228+0000 7ff8f1bc4640 1 --2- 172.21.15.120:0/1341548262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a9f56370 0x5647a9f58760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.228+0000 7ff8f1bc4640 1 -- 172.21.15.120:0/1341548262 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647a9e3d3a0 con 0x5647a9f4b5d0 2024-09-19T09:09:24.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.228+0000 7ff8f23c5640 1 --2- 172.21.15.120:0/1337844453 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff8d80a32c0 0x7ff8d80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:24.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.228+0000 7ff8e2ffd640 1 -- 172.21.15.120:0/1337844453 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7ff8e40f0ad0 con 0x7ff8ec106b80 2024-09-19T09:09:24.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.228+0000 7ff8f1bc4640 1 --2- 172.21.15.120:0/1341548262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647a9f4b5d0 0x5647a9f4ba10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff8e800bc50 tx=0x7ff8e80107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17c6b81f8fa9b1a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:24.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.228+0000 7ff8d7fff640 1 -- 172.21.15.120:0/1341548262 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e801c480 con 0x5647a9f4b5d0 2024-09-19T09:09:24.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.228+0000 7ff8f23c5640 1 --2- 172.21.15.120:0/1337844453 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff8d80a32c0 0x7ff8d80a5780 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff8e40c5830 tx=0x7ff8e408bec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:24.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.228+0000 7ff8d7fff640 1 -- 172.21.15.120:0/1341548262 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8e801cb00 con 0x5647a9f4b5d0 2024-09-19T09:09:24.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.228+0000 7ff8d7fff640 1 -- 172.21.15.120:0/1341548262 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8011c00 con 0x5647a9f4b5d0 2024-09-19T09:09:24.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.229+0000 7ff8f6201740 1 -- 172.21.15.120:0/1341548262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647a9f4b5d0 msgr2=0x5647a9f4ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:24.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.229+0000 7ff8f6201740 1 --2- 172.21.15.120:0/1341548262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647a9f4b5d0 0x5647a9f4ba10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff8e800bc50 tx=0x7ff8e80107a0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.229+0000 7ff8f6201740 1 -- 172.21.15.120:0/1341548262 shutdown_connections 2024-09-19T09:09:24.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.229+0000 7ff8f6201740 1 --2- 172.21.15.120:0/1341548262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a9f56370 0x5647a9f58760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.229+0000 7ff8f6201740 1 --2- 172.21.15.120:0/1341548262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647a9f4b5d0 0x5647a9f4ba10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.229+0000 7ff8f6201740 1 --2- 172.21.15.120:0/1341548262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a9f4cb40 0x5647a9f4b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.229+0000 7ff8f6201740 1 -- 172.21.15.120:0/1341548262 >> 172.21.15.120:0/1341548262 conn(0x5647a9f14310 msgr2=0x5647a9f47c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:24.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.229+0000 7ff8f6201740 1 -- 172.21.15.120:0/1341548262 shutdown_connections 2024-09-19T09:09:24.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.229+0000 7ff8f6201740 1 -- 172.21.15.120:0/1341548262 wait complete. 2024-09-19T09:09:24.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.229+0000 7ff8f6201740 1 -- start start 2024-09-19T09:09:24.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.229+0000 7ff8f6201740 1 Processor -- start 2024-09-19T09:09:24.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.230+0000 7ff8f6201740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a9f4b5d0 0x5647a9f55da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:24.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.230+0000 7ff8f6201740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647a9f4cb40 0x5647a9f53050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:24.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.230+0000 7ff8f6201740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a9f56370 0x5647a9f537e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:24.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.230+0000 7ff8f6201740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5647a9f59070 con 0x5647a9f56370 2024-09-19T09:09:24.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.230+0000 7ff8f6201740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5647a9e6c060 con 0x5647a9f4b5d0 2024-09-19T09:09:24.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.230+0000 7ff8f6201740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5647a9f591f0 con 0x5647a9f4cb40 2024-09-19T09:09:24.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.230+0000 7ff8f6201740 10 client.0 _ll_register_callbacks cb 0x5647a9bbf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:24.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.230+0000 7ff8f13c3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a9f4b5d0 0x5647a9f55da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:24.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.230+0000 7ff8f1bc4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647a9f4cb40 0x5647a9f53050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:24.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.230+0000 7ff8f1bc4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647a9f4cb40 0x5647a9f53050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:45624/0 (socket says 172.21.15.120:45624) 2024-09-19T09:09:24.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.230+0000 7ff8f13c3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a9f56370 0x5647a9f537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:24.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.230+0000 7ff8f1bc4640 1 -- 172.21.15.120:0/2132647820 learned_addr learned my addr 172.21.15.120:0/2132647820 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:24.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.230+0000 7ff8f1bc4640 1 -- 172.21.15.120:0/2132647820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a9f4b5d0 msgr2=0x5647a9f55da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:24.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.230+0000 7ff8f1bc4640 1 --2- 172.21.15.120:0/2132647820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a9f4b5d0 0x5647a9f55da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.230+0000 7ff8f1bc4640 1 -- 172.21.15.120:0/2132647820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a9f56370 msgr2=0x5647a9f537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:24.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.230+0000 7ff8f1bc4640 1 --2- 172.21.15.120:0/2132647820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a9f56370 0x5647a9f537e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.230+0000 7ff8f1bc4640 1 -- 172.21.15.120:0/2132647820 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647a9c223c0 con 0x5647a9f4cb40 2024-09-19T09:09:24.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.231+0000 7ff8f1bc4640 1 --2- 172.21.15.120:0/2132647820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647a9f4cb40 0x5647a9f53050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8000c00 tx=0x7ff8e8011760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1ffd7343f178afc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:24.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.231+0000 7ff8d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:09:24.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.231+0000 7ff8d57fa640 1 -- 172.21.15.120:0/2132647820 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8004020 con 0x5647a9f4cb40 2024-09-19T09:09:24.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.231+0000 7ff8f6201740 1 -- 172.21.15.120:0/2132647820 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8e801cb00 con 0x5647a9f4cb40 2024-09-19T09:09:24.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.231+0000 7ff8d57fa640 1 -- 172.21.15.120:0/2132647820 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7ff8e80041c0 con 0x5647a9f4cb40 2024-09-19T09:09:24.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.231+0000 7ff8d57fa640 1 -- 172.21.15.120:0/2132647820 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff8e80043c0 con 0x5647a9f4cb40 2024-09-19T09:09:24.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.231+0000 7ff8f6201740 10 client.26881 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:24.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.231+0000 7ff8f6201740 1 -- 172.21.15.120:0/2132647820 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5647a9f64b50 con 0x5647a9f4cb40 2024-09-19T09:09:24.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.231+0000 7ff8d57fa640 1 -- 172.21.15.120:0/2132647820 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8e801cb00 con 0x5647a9f4cb40 2024-09-19T09:09:24.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.231+0000 7ff8d57fa640 1 -- 172.21.15.120:0/2132647820 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff8e8011bd0 con 0x5647a9f4cb40 2024-09-19T09:09:24.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.231+0000 7ff8f6201740 10 client.26881 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:24.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.231+0000 7ff8f6201740 10 client.26881 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:09:24.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.231+0000 7ff8f6201740 20 client.26881 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:24.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.231+0000 7ff8f6201740 1 --2- 172.21.15.120:0/2132647820 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5647a9f65ea0 0x5647a9f68290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:24.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.231+0000 7ff8f6201740 4 client.26881 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:09:24.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.231+0000 7ff8f6201740 1 -- 172.21.15.120:0/2132647820 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647a9ebe860 con 0x5647a9f65ea0 2024-09-19T09:09:24.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.231+0000 7ff8f13c3640 1 --2- 172.21.15.120:0/2132647820 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5647a9f65ea0 0x5647a9f68290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:24.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.232+0000 7ff8d57fa640 10 client.26881 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:09:24.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.232+0000 7ff8f13c3640 1 --2- 172.21.15.120:0/2132647820 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5647a9f65ea0 0x5647a9f68290 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7a9f73900a3bdda6 server_cookie=6aac7843976703ad in_seq=0 out_seq=0 2024-09-19T09:09:24.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.232+0000 7ff8d57fa640 1 -- 172.21.15.120:0/2132647820 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5647a9ebe860 con 0x5647a9f65ea0 2024-09-19T09:09:24.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.232+0000 7ff8d57fa640 10 client.26881 handle_command_reply: tid=0 2024-09-19T09:09:24.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.232+0000 7ff8f6201740 1 client.26881 shutdown 2024-09-19T09:09:24.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.233+0000 7ff8f6201740 1 -- 172.21.15.120:0/2132647820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647a9f4cb40 msgr2=0x5647a9f53050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:24.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.233+0000 7ff8f6201740 1 --2- 172.21.15.120:0/2132647820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647a9f4cb40 0x5647a9f53050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8000c00 tx=0x7ff8e8011760 comp rx=0 tx=0).stop 2024-09-19T09:09:24.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.233+0000 7ff8f6201740 1 -- 172.21.15.120:0/2132647820 shutdown_connections 2024-09-19T09:09:24.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.233+0000 7ff8f6201740 1 --2- 172.21.15.120:0/2132647820 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5647a9f65ea0 0x5647a9f68290 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.233+0000 7ff8f6201740 1 --2- 172.21.15.120:0/2132647820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a9f56370 0x5647a9f537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.233+0000 7ff8f6201740 1 --2- 172.21.15.120:0/2132647820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647a9f4cb40 0x5647a9f53050 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.233+0000 7ff8f6201740 1 --2- 172.21.15.120:0/2132647820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a9f4b5d0 0x5647a9f55da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.233+0000 7ff8f6201740 1 -- 172.21.15.120:0/2132647820 >> 172.21.15.120:0/2132647820 conn(0x5647a9f14310 msgr2=0x5647a9f4d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:24.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.233+0000 7ff8d57fa640 0 client.26881 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:09:24.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.233+0000 7ff8f6201740 1 -- 172.21.15.120:0/2132647820 shutdown_connections 2024-09-19T09:09:24.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.233+0000 7ff8f6201740 1 -- 172.21.15.120:0/2132647820 wait complete. 2024-09-19T09:09:24.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.233+0000 7ff8f6201740 20 client.26881 trim_cache size 0 max 16384 2024-09-19T09:09:24.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.246+0000 7ff8f6201740 1 Processor -- start 2024-09-19T09:09:24.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.246+0000 7ff8f6201740 1 -- start start 2024-09-19T09:09:24.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.246+0000 7ff8f6201740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647a9f4b5d0 0x5647a9f54fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:24.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.246+0000 7ff8f6201740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a9f4cb40 0x5647a9f4e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:24.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.246+0000 7ff8f6201740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a9f56370 0x5647a9f4e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:24.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.246+0000 7ff8f6201740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec1ad5f0 con 0x5647a9f56370 2024-09-19T09:09:24.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.246+0000 7ff8f6201740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec1af130 con 0x5647a9f4cb40 2024-09-19T09:09:24.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.246+0000 7ff8f6201740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec1adcc0 con 0x5647a9f4b5d0 2024-09-19T09:09:24.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.246+0000 7ff8f13c3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a9f56370 0x5647a9f4e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:24.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.246+0000 7ff8f13c3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a9f56370 0x5647a9f4e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35922/0 (socket says 172.21.15.120:35922) 2024-09-19T09:09:24.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.246+0000 7ff8f13c3640 1 -- 172.21.15.120:0/3539964838 learned_addr learned my addr 172.21.15.120:0/3539964838 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:24.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.246+0000 7ff8f1bc4640 1 --2- 172.21.15.120:0/3539964838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a9f4cb40 0x5647a9f4e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:24.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.246+0000 7ff8f13c3640 1 --2- 172.21.15.120:0/3539964838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647a9f4b5d0 0x5647a9f54fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:24.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.247+0000 7ff8f13c3640 1 -- 172.21.15.120:0/3539964838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647a9f4b5d0 msgr2=0x5647a9f54fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:24.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.247+0000 7ff8f13c3640 1 --2- 172.21.15.120:0/3539964838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647a9f4b5d0 0x5647a9f54fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.247+0000 7ff8f13c3640 1 -- 172.21.15.120:0/3539964838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a9f4cb40 msgr2=0x5647a9f4e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:24.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.247+0000 7ff8f13c3640 1 --2- 172.21.15.120:0/3539964838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a9f4cb40 0x5647a9f4e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.247+0000 7ff8f13c3640 1 -- 172.21.15.120:0/3539964838 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647a9f53ff0 con 0x5647a9f56370 2024-09-19T09:09:24.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.247+0000 7ff8f13c3640 1 --2- 172.21.15.120:0/3539964838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a9f56370 0x5647a9f4e660 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc000c00 tx=0x7ff8dc00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4ebe73fa3bdc00 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:24.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.247+0000 7ff8d77fe640 1 -- 172.21.15.120:0/3539964838 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc00dd20 con 0x5647a9f56370 2024-09-19T09:09:24.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.247+0000 7ff8d77fe640 1 -- 172.21.15.120:0/3539964838 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8dc0035f0 con 0x5647a9f56370 2024-09-19T09:09:24.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.247+0000 7ff8d77fe640 1 -- 172.21.15.120:0/3539964838 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc0236a0 con 0x5647a9f56370 2024-09-19T09:09:24.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.248+0000 7ff8f6201740 1 -- 172.21.15.120:0/3539964838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a9f56370 msgr2=0x5647a9f4e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:24.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.248+0000 7ff8f6201740 1 --2- 172.21.15.120:0/3539964838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a9f56370 0x5647a9f4e660 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc000c00 tx=0x7ff8dc00b410 comp rx=0 tx=0).stop 2024-09-19T09:09:24.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.248+0000 7ff8f6201740 1 -- 172.21.15.120:0/3539964838 shutdown_connections 2024-09-19T09:09:24.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.248+0000 7ff8f6201740 1 --2- 172.21.15.120:0/3539964838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a9f56370 0x5647a9f4e660 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.248+0000 7ff8f6201740 1 --2- 172.21.15.120:0/3539964838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a9f4cb40 0x5647a9f4e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.248+0000 7ff8f6201740 1 --2- 172.21.15.120:0/3539964838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647a9f4b5d0 0x5647a9f54fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.248+0000 7ff8f6201740 1 -- 172.21.15.120:0/3539964838 >> 172.21.15.120:0/3539964838 conn(0x5647a9f14310 msgr2=0x5647a9f548b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:24.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.248+0000 7ff8f6201740 1 -- 172.21.15.120:0/3539964838 shutdown_connections 2024-09-19T09:09:24.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.248+0000 7ff8f6201740 1 -- 172.21.15.120:0/3539964838 wait complete. 2024-09-19T09:09:24.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.248+0000 7ff8f6201740 1 -- start start 2024-09-19T09:09:24.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.248+0000 7ff8f6201740 1 Processor -- start 2024-09-19T09:09:24.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.249+0000 7ff8f6201740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647a9f4b5d0 0x5647a9f50070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:24.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.249+0000 7ff8f6201740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a9f4cb40 0x5647a9f505b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:24.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.249+0000 7ff8f6201740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a9f56370 0x5647a9f50af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:24.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.249+0000 7ff8f6201740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec1aea60 con 0x5647a9f4cb40 2024-09-19T09:09:24.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.249+0000 7ff8f6201740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec1ae390 con 0x5647a9f56370 2024-09-19T09:09:24.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.249+0000 7ff8f6201740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec1ab5b0 con 0x5647a9f4b5d0 2024-09-19T09:09:24.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.249+0000 7ff8f6201740 10 client.0 _ll_register_callbacks cb 0x7ff8dc00bee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:24.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.249+0000 7ff8f1bc4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a9f4cb40 0x5647a9f505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:24.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.249+0000 7ff8f1bc4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a9f4cb40 0x5647a9f505b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35932/0 (socket says 172.21.15.120:35932) 2024-09-19T09:09:24.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.249+0000 7ff8f1bc4640 1 -- 172.21.15.120:0/4195447137 learned_addr learned my addr 172.21.15.120:0/4195447137 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:24.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.249+0000 7ff8f13c3640 1 --2- 172.21.15.120:0/4195447137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647a9f4b5d0 0x5647a9f50070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:24.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.249+0000 7ff8f13c3640 1 --2- 172.21.15.120:0/4195447137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a9f56370 0x5647a9f50af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:24.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.249+0000 7ff8f1bc4640 1 -- 172.21.15.120:0/4195447137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647a9f4b5d0 msgr2=0x5647a9f50070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:24.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.249+0000 7ff8f1bc4640 1 --2- 172.21.15.120:0/4195447137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647a9f4b5d0 0x5647a9f50070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.249+0000 7ff8f1bc4640 1 -- 172.21.15.120:0/4195447137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a9f56370 msgr2=0x5647a9f50af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:24.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.249+0000 7ff8f1bc4640 1 --2- 172.21.15.120:0/4195447137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a9f56370 0x5647a9f50af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.249+0000 7ff8f1bc4640 1 -- 172.21.15.120:0/4195447137 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647a9fcf890 con 0x5647a9f4cb40 2024-09-19T09:09:24.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.249+0000 7ff8f13c3640 1 --2- 172.21.15.120:0/4195447137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647a9f4b5d0 0x5647a9f50070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:24.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.249+0000 7ff8f1bc4640 1 --2- 172.21.15.120:0/4195447137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a9f4cb40 0x5647a9f505b0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7ff8e800b7b0 tx=0x7ff8e8011650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d492a6467d7d8e6a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:24.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.249+0000 7ff8d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:09:24.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.250+0000 7ff8d57fa640 1 -- 172.21.15.120:0/4195447137 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e803e060 con 0x5647a9f4cb40 2024-09-19T09:09:24.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.250+0000 7ff8d57fa640 1 -- 172.21.15.120:0/4195447137 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7ff8e801c480 con 0x5647a9f4cb40 2024-09-19T09:09:24.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.250+0000 7ff8f6201740 1 -- 172.21.15.120:0/4195447137 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8dc0035f0 con 0x5647a9f4cb40 2024-09-19T09:09:24.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.250+0000 7ff8d57fa640 1 -- 172.21.15.120:0/4195447137 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff8e8035040 con 0x5647a9f4cb40 2024-09-19T09:09:24.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.250+0000 7ff8f6201740 10 client.17736 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:24.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.250+0000 7ff8f6201740 1 -- 172.21.15.120:0/4195447137 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5647a9fd0b70 con 0x5647a9f4cb40 2024-09-19T09:09:24.270 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:09:24.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.250+0000 7ff8d57fa640 1 -- 172.21.15.120:0/4195447137 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8dc0035f0 con 0x5647a9f4cb40 2024-09-19T09:09:24.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.250+0000 7ff8d57fa640 1 -- 172.21.15.120:0/4195447137 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff8e8043070 con 0x5647a9f4cb40 2024-09-19T09:09:24.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.250+0000 7ff8f6201740 10 client.17736 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:24.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.250+0000 7ff8f6201740 10 client.17736 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:09:24.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.250+0000 7ff8f6201740 20 client.17736 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:24.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.250+0000 7ff8f6201740 1 --2- 172.21.15.120:0/4195447137 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5647a9fd12c0 0x5647a9fd36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:24.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.250+0000 7ff8f6201740 4 client.17736 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:09:24.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.250+0000 7ff8f6201740 1 -- 172.21.15.120:0/4195447137 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5647a9eb8d70 con 0x5647a9fd12c0 2024-09-19T09:09:24.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.250+0000 7ff8f13c3640 1 --2- 172.21.15.120:0/4195447137 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5647a9fd12c0 0x5647a9fd36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:24.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.251+0000 7ff8f13c3640 1 --2- 172.21.15.120:0/4195447137 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5647a9fd12c0 0x5647a9fd36e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6bbbf8eb54b82d54 server_cookie=cbca2c5fdd295547 in_seq=0 out_seq=0 2024-09-19T09:09:24.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.251+0000 7ff8d57fa640 10 client.17736 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:09:24.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.251+0000 7ff8d57fa640 1 -- 172.21.15.120:0/4195447137 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5647a9eb8d70 con 0x5647a9fd12c0 2024-09-19T09:09:24.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.251+0000 7ff8d57fa640 10 client.17736 handle_command_reply: tid=0 2024-09-19T09:09:24.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.251+0000 7ff8f6201740 1 client.17736 shutdown 2024-09-19T09:09:24.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.251+0000 7ff8f6201740 1 -- 172.21.15.120:0/4195447137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a9f4cb40 msgr2=0x5647a9f505b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:24.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.251+0000 7ff8f6201740 1 --2- 172.21.15.120:0/4195447137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a9f4cb40 0x5647a9f505b0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7ff8e800b7b0 tx=0x7ff8e8011650 comp rx=0 tx=0).stop 2024-09-19T09:09:24.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.251+0000 7ff8f6201740 1 -- 172.21.15.120:0/4195447137 shutdown_connections 2024-09-19T09:09:24.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.251+0000 7ff8f6201740 1 --2- 172.21.15.120:0/4195447137 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5647a9fd12c0 0x5647a9fd36e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.251+0000 7ff8f6201740 1 --2- 172.21.15.120:0/4195447137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a9f56370 0x5647a9f50af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.251+0000 7ff8f6201740 1 --2- 172.21.15.120:0/4195447137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a9f4cb40 0x5647a9f505b0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.251+0000 7ff8d57fa640 0 client.17736 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:09:24.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.251+0000 7ff8f6201740 1 --2- 172.21.15.120:0/4195447137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647a9f4b5d0 0x5647a9f50070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.251+0000 7ff8f6201740 1 -- 172.21.15.120:0/4195447137 >> 172.21.15.120:0/4195447137 conn(0x5647a9f14310 msgr2=0x5647a9f550d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:24.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.251+0000 7ff8f6201740 1 -- 172.21.15.120:0/4195447137 shutdown_connections 2024-09-19T09:09:24.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.251+0000 7ff8f6201740 1 -- 172.21.15.120:0/4195447137 wait complete. 2024-09-19T09:09:24.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.251+0000 7ff8f6201740 20 client.17736 trim_cache size 0 max 16384 2024-09-19T09:09:24.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.252+0000 7ff8f3e28640 1 -- 172.21.15.120:0/1337844453 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff8d80a32c0 msgr2=0x7ff8d80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:24.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.252+0000 7ff8f3e28640 1 --2- 172.21.15.120:0/1337844453 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff8d80a32c0 0x7ff8d80a5780 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff8e40c5830 tx=0x7ff8e408bec0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.252+0000 7ff8f3e28640 1 -- 172.21.15.120:0/1337844453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8ec106b80 msgr2=0x7ff8ec19cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:24.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.252+0000 7ff8f3e28640 1 --2- 172.21.15.120:0/1337844453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8ec106b80 0x7ff8ec19cd20 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7ff8e40319f0 tx=0x7ff8e4031a20 comp rx=0 tx=0).stop 2024-09-19T09:09:24.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.252+0000 7ff8f3e28640 1 -- 172.21.15.120:0/1337844453 shutdown_connections 2024-09-19T09:09:24.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.252+0000 7ff8f3e28640 1 --2- 172.21.15.120:0/1337844453 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff8d80a32c0 0x7ff8d80a5780 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.252+0000 7ff8f3e28640 1 --2- 172.21.15.120:0/1337844453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8ec106b80 0x7ff8ec19cd20 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.252+0000 7ff8f3e28640 1 --2- 172.21.15.120:0/1337844453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8ec100aa0 0x7ff8ec198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.252+0000 7ff8f3e28640 1 --2- 172.21.15.120:0/1337844453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8ec100100 0x7ff8ec198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:24.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.252+0000 7ff8f3e28640 1 -- 172.21.15.120:0/1337844453 >> 172.21.15.120:0/1337844453 conn(0x7ff8ec077230 msgr2=0x7ff8ec0fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:24.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.252+0000 7ff8f3e28640 1 -- 172.21.15.120:0/1337844453 shutdown_connections 2024-09-19T09:09:24.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:24.252+0000 7ff8f3e28640 1 -- 172.21.15.120:0/1337844453 wait complete. 2024-09-19T09:09:25.053 INFO:tasks.workunit.client.0.smithi120.stdout: 1 582970 9.67 MB/sec execute 200 sec latency 101.842 ms 2024-09-19T09:09:25.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:24 smithi120 ceph-mon[31150]: pgmap v538: 97 pgs: 97 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.7 MiB/s wr, 472 op/s 2024-09-19T09:09:25.114 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:09:25.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:24 smithi196 ceph-mon[35150]: pgmap v538: 97 pgs: 97 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.7 MiB/s wr, 472 op/s 2024-09-19T09:09:25.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:24 smithi153 ceph-mon[35071]: pgmap v538: 97 pgs: 97 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.7 MiB/s wr, 472 op/s 2024-09-19T09:09:25.368 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:09:25.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.857+0000 7f056ba78640 1 -- 172.21.15.120:0/2928228832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0564108290 msgr2=0x7f056410ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:25.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.857+0000 7f056ba78640 1 --2- 172.21.15.120:0/2928228832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0564108290 0x7f056410ec30 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f055c009cd0 tx=0x7f055c02a8e0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.857+0000 7f056ba78640 1 -- 172.21.15.120:0/2928228832 shutdown_connections 2024-09-19T09:09:25.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.857+0000 7f056ba78640 1 --2- 172.21.15.120:0/2928228832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0564108290 0x7f056410ec30 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.857+0000 7f056ba78640 1 --2- 172.21.15.120:0/2928228832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0564103740 0x7f0564107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.857+0000 7f056ba78640 1 --2- 172.21.15.120:0/2928228832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0564102d90 0x7f0564103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.857+0000 7f056ba78640 1 -- 172.21.15.120:0/2928228832 >> 172.21.15.120:0/2928228832 conn(0x7f05640fc7d0 msgr2=0x7f05640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:25.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.857+0000 7f056ba78640 1 -- 172.21.15.120:0/2928228832 shutdown_connections 2024-09-19T09:09:25.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.857+0000 7f056ba78640 1 -- 172.21.15.120:0/2928228832 wait complete. 2024-09-19T09:09:25.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.858+0000 7f056ba78640 1 Processor -- start 2024-09-19T09:09:25.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.858+0000 7f056ba78640 1 -- start start 2024-09-19T09:09:25.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.858+0000 7f056ba78640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0564102d90 0x7f05641985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:25.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.858+0000 7f056ba78640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0564103740 0x7f0564198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:25.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.858+0000 7f056ba78640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0564108290 0x7f056419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:25.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.858+0000 7f056ba78640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0564112090 con 0x7f0564103740 2024-09-19T09:09:25.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.858+0000 7f056ba78640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0564111f10 con 0x7f0564102d90 2024-09-19T09:09:25.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.858+0000 7f056ba78640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0564112210 con 0x7f0564108290 2024-09-19T09:09:25.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.858+0000 7f0569013640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0564103740 0x7f0564198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:25.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.858+0000 7f0569013640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0564103740 0x7f0564198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35942/0 (socket says 172.21.15.120:35942) 2024-09-19T09:09:25.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.858+0000 7f0569013640 1 -- 172.21.15.120:0/3859736486 learned_addr learned my addr 172.21.15.120:0/3859736486 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:25.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.858+0000 7f0569814640 1 --2- 172.21.15.120:0/3859736486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0564102d90 0x7f05641985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:25.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.858+0000 7f056a015640 1 --2- 172.21.15.120:0/3859736486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0564108290 0x7f056419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:25.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.858+0000 7f0569013640 1 -- 172.21.15.120:0/3859736486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0564108290 msgr2=0x7f056419ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:25.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.858+0000 7f0569013640 1 --2- 172.21.15.120:0/3859736486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0564108290 0x7f056419ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.858+0000 7f0569013640 1 -- 172.21.15.120:0/3859736486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0564102d90 msgr2=0x7f05641985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:25.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.858+0000 7f0569013640 1 --2- 172.21.15.120:0/3859736486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0564102d90 0x7f05641985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.858+0000 7f0569013640 1 -- 172.21.15.120:0/3859736486 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056419d590 con 0x7f0564103740 2024-09-19T09:09:25.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.859+0000 7f0569013640 1 --2- 172.21.15.120:0/3859736486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0564103740 0x7f0564198b20 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f055400e9f0 tx=0x7f055400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:25.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.859+0000 7f055affd640 1 -- 172.21.15.120:0/3859736486 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055400ccc0 con 0x7f0564103740 2024-09-19T09:09:25.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.859+0000 7f056ba78640 1 -- 172.21.15.120:0/3859736486 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f056419d880 con 0x7f0564103740 2024-09-19T09:09:25.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.859+0000 7f055affd640 1 -- 172.21.15.120:0/3859736486 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0554005500 con 0x7f0564103740 2024-09-19T09:09:25.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.859+0000 7f055affd640 1 -- 172.21.15.120:0/3859736486 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0554010690 con 0x7f0564103740 2024-09-19T09:09:25.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.859+0000 7f056ba78640 1 -- 172.21.15.120:0/3859736486 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05641a5160 con 0x7f0564103740 2024-09-19T09:09:25.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.861+0000 7f055affd640 1 -- 172.21.15.120:0/3859736486 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f05540046c0 con 0x7f0564103740 2024-09-19T09:09:25.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.862+0000 7f056de51740 1 Processor -- start 2024-09-19T09:09:25.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.862+0000 7f055affd640 1 --2- 172.21.15.120:0/3859736486 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f05500a3240 0x7f05500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:25.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.862+0000 7f056de51740 1 -- start start 2024-09-19T09:09:25.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.862+0000 7f056a015640 1 --2- 172.21.15.120:0/3859736486 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f05500a3240 0x7f05500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:25.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.862+0000 7f055affd640 1 -- 172.21.15.120:0/3859736486 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f05540cfc00 con 0x7f0564103740 2024-09-19T09:09:25.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.862+0000 7f056de51740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56347ec9db40 0x56347ec9c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:25.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.862+0000 7f056de51740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56347ec9c5d0 0x56347ec9ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:25.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.862+0000 7f056de51740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56347eca7370 0x56347eca9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:25.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.862+0000 7f056de51740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56347ec022f0 con 0x56347ec9c5d0 2024-09-19T09:09:25.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.862+0000 7f056de51740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56347eba9cf0 con 0x56347ec9db40 2024-09-19T09:09:25.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.862+0000 7f056de51740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56347ebfe2b0 con 0x56347eca7370 2024-09-19T09:09:25.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.862+0000 7f0569013640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56347ec9c5d0 0x56347ec9ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:25.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.862+0000 7f0569814640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56347ec9db40 0x56347ec9c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:25.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.862+0000 7f0569814640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56347ec9db40 0x56347ec9c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59696/0 (socket says 172.21.15.120:59696) 2024-09-19T09:09:25.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.862+0000 7f0569013640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56347ec9c5d0 0x56347ec9ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35952/0 (socket says 172.21.15.120:35952) 2024-09-19T09:09:25.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.862+0000 7f0569814640 1 -- 172.21.15.120:0/776645591 learned_addr learned my addr 172.21.15.120:0/776645591 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:25.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.863+0000 7f056a015640 1 --2- 172.21.15.120:0/776645591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56347eca7370 0x56347eca9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:25.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.863+0000 7f056a015640 1 --2- 172.21.15.120:0/3859736486 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f05500a3240 0x7f05500a5700 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f055c004ab0 tx=0x7f055c031430 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:25.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.863+0000 7f0569013640 1 -- 172.21.15.120:0/776645591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56347eca7370 msgr2=0x56347eca9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:25.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.863+0000 7f0569013640 1 --2- 172.21.15.120:0/776645591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56347eca7370 0x56347eca9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.863+0000 7f0569013640 1 -- 172.21.15.120:0/776645591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56347ec9db40 msgr2=0x56347ec9c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:25.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.863+0000 7f0569013640 1 --2- 172.21.15.120:0/776645591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56347ec9db40 0x56347ec9c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.863+0000 7f0569013640 1 -- 172.21.15.120:0/776645591 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56347eb8e3a0 con 0x56347ec9c5d0 2024-09-19T09:09:25.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.863+0000 7f0569814640 1 --2- 172.21.15.120:0/776645591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56347ec9db40 0x56347ec9c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:25.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.863+0000 7f0569013640 1 --2- 172.21.15.120:0/776645591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56347ec9c5d0 0x56347ec9ca10 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f055400e9c0 tx=0x7f055409bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb591d8f41f8c16 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:25.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.863+0000 7f054ffff640 1 -- 172.21.15.120:0/776645591 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05540b5070 con 0x56347ec9c5d0 2024-09-19T09:09:25.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.863+0000 7f054ffff640 1 -- 172.21.15.120:0/776645591 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05540b5210 con 0x56347ec9c5d0 2024-09-19T09:09:25.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.863+0000 7f054ffff640 1 -- 172.21.15.120:0/776645591 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05540bae60 con 0x56347ec9c5d0 2024-09-19T09:09:25.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.864+0000 7f056de51740 1 -- 172.21.15.120:0/776645591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56347ec9c5d0 msgr2=0x56347ec9ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:25.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.864+0000 7f056de51740 1 --2- 172.21.15.120:0/776645591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56347ec9c5d0 0x56347ec9ca10 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f055400e9c0 tx=0x7f055409bd60 comp rx=0 tx=0).stop 2024-09-19T09:09:25.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.864+0000 7f056de51740 1 -- 172.21.15.120:0/776645591 shutdown_connections 2024-09-19T09:09:25.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.864+0000 7f056de51740 1 --2- 172.21.15.120:0/776645591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56347eca7370 0x56347eca9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.864+0000 7f056de51740 1 --2- 172.21.15.120:0/776645591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56347ec9c5d0 0x56347ec9ca10 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.864+0000 7f056de51740 1 --2- 172.21.15.120:0/776645591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56347ec9db40 0x56347ec9c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.864+0000 7f056de51740 1 -- 172.21.15.120:0/776645591 >> 172.21.15.120:0/776645591 conn(0x56347ec65310 msgr2=0x56347ec98c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:25.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.864+0000 7f056de51740 1 -- 172.21.15.120:0/776645591 shutdown_connections 2024-09-19T09:09:25.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.864+0000 7f056de51740 1 -- 172.21.15.120:0/776645591 wait complete. 2024-09-19T09:09:25.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.864+0000 7f056de51740 1 -- start start 2024-09-19T09:09:25.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.864+0000 7f056de51740 1 Processor -- start 2024-09-19T09:09:25.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.865+0000 7f056de51740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56347ec9c5d0 0x56347eca6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:25.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.865+0000 7f056de51740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56347ec9db40 0x56347eca3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:25.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.865+0000 7f056de51740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56347eca7370 0x56347eca4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:25.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.865+0000 7f056de51740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56347ecaa070 con 0x56347eca7370 2024-09-19T09:09:25.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.865+0000 7f056de51740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56347ebbd060 con 0x56347ec9c5d0 2024-09-19T09:09:25.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.865+0000 7f056de51740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56347ecaa1f0 con 0x56347ec9db40 2024-09-19T09:09:25.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.865+0000 7f056de51740 10 client.0 _ll_register_callbacks cb 0x56347e9108c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:25.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.865+0000 7f056a015640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56347eca7370 0x56347eca4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:25.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.865+0000 7f056a015640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56347eca7370 0x56347eca4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35958/0 (socket says 172.21.15.120:35958) 2024-09-19T09:09:25.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.865+0000 7f056a015640 1 -- 172.21.15.120:0/2271175523 learned_addr learned my addr 172.21.15.120:0/2271175523 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:25.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.865+0000 7f0569013640 1 --2- 172.21.15.120:0/2271175523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56347ec9db40 0x56347eca3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:25.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.865+0000 7f0569814640 1 --2- 172.21.15.120:0/2271175523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56347ec9c5d0 0x56347eca6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:25.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.865+0000 7f056a015640 1 -- 172.21.15.120:0/2271175523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56347ec9db40 msgr2=0x56347eca3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:25.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.865+0000 7f056a015640 1 --2- 172.21.15.120:0/2271175523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56347ec9db40 0x56347eca3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.865+0000 7f056a015640 1 -- 172.21.15.120:0/2271175523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56347ec9c5d0 msgr2=0x56347eca6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:25.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.865+0000 7f056a015640 1 --2- 172.21.15.120:0/2271175523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56347ec9c5d0 0x56347eca6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.865+0000 7f056a015640 1 -- 172.21.15.120:0/2271175523 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56347e9733c0 con 0x56347eca7370 2024-09-19T09:09:25.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.866+0000 7f056a015640 1 --2- 172.21.15.120:0/2271175523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56347eca7370 0x56347eca4560 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f055c009cd0 tx=0x7f055c02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e85fb91c29bedd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:25.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.866+0000 7f054d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:09:25.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.866+0000 7f054d7fa640 1 -- 172.21.15.120:0/2271175523 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c041040 con 0x56347eca7370 2024-09-19T09:09:25.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.866+0000 7f054d7fa640 1 -- 172.21.15.120:0/2271175523 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f055c004240 con 0x56347eca7370 2024-09-19T09:09:25.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.866+0000 7f056de51740 1 -- 172.21.15.120:0/2271175523 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05540b5210 con 0x56347eca7370 2024-09-19T09:09:25.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.866+0000 7f054d7fa640 1 -- 172.21.15.120:0/2271175523 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f055c004440 con 0x56347eca7370 2024-09-19T09:09:25.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.866+0000 7f056de51740 10 client.17748 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:25.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.866+0000 7f056de51740 1 -- 172.21.15.120:0/2271175523 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56347ecb5ac0 con 0x56347eca7370 2024-09-19T09:09:25.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.866+0000 7f054d7fa640 1 -- 172.21.15.120:0/2271175523 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05540b5210 con 0x56347eca7370 2024-09-19T09:09:25.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.866+0000 7f054d7fa640 1 -- 172.21.15.120:0/2271175523 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f055c04e020 con 0x56347eca7370 2024-09-19T09:09:25.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.866+0000 7f056de51740 10 client.17748 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:25.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.866+0000 7f056de51740 10 client.17748 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:09:25.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.866+0000 7f056de51740 20 client.17748 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:25.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.866+0000 7f056de51740 1 --2- 172.21.15.120:0/2271175523 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56347ecb6f80 0x56347ecb9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:25.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.866+0000 7f056de51740 4 client.17748 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:09:25.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.866+0000 7f056de51740 1 -- 172.21.15.120:0/2271175523 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56347ec0f860 con 0x56347ecb6f80 2024-09-19T09:09:25.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.866+0000 7f0569814640 1 --2- 172.21.15.120:0/2271175523 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56347ecb6f80 0x56347ecb9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:25.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.867+0000 7f054d7fa640 10 client.17748 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:09:25.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.867+0000 7f0569814640 1 --2- 172.21.15.120:0/2271175523 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56347ecb6f80 0x56347ecb9370 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fd1239797ab222b server_cookie=31d3331cfa3553ca in_seq=0 out_seq=0 2024-09-19T09:09:25.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.867+0000 7f054d7fa640 1 -- 172.21.15.120:0/2271175523 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56347ec0f860 con 0x56347ecb6f80 2024-09-19T09:09:25.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.867+0000 7f054d7fa640 10 client.17748 handle_command_reply: tid=0 2024-09-19T09:09:25.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.868+0000 7f056de51740 1 client.17748 shutdown 2024-09-19T09:09:25.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.868+0000 7f056de51740 1 -- 172.21.15.120:0/2271175523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56347eca7370 msgr2=0x56347eca4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:25.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.868+0000 7f056de51740 1 --2- 172.21.15.120:0/2271175523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56347eca7370 0x56347eca4560 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f055c009cd0 tx=0x7f055c02a8e0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.868+0000 7f056de51740 1 -- 172.21.15.120:0/2271175523 shutdown_connections 2024-09-19T09:09:25.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.868+0000 7f056de51740 1 --2- 172.21.15.120:0/2271175523 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56347ecb6f80 0x56347ecb9370 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.868+0000 7f056de51740 1 --2- 172.21.15.120:0/2271175523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56347eca7370 0x56347eca4560 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.868+0000 7f056de51740 1 --2- 172.21.15.120:0/2271175523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56347ec9db40 0x56347eca3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.868+0000 7f056de51740 1 --2- 172.21.15.120:0/2271175523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56347ec9c5d0 0x56347eca6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.868+0000 7f054d7fa640 0 client.17748 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:09:25.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.868+0000 7f056de51740 1 -- 172.21.15.120:0/2271175523 >> 172.21.15.120:0/2271175523 conn(0x56347ec65310 msgr2=0x56347ec9e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:25.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.868+0000 7f056de51740 1 -- 172.21.15.120:0/2271175523 shutdown_connections 2024-09-19T09:09:25.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.868+0000 7f056de51740 1 -- 172.21.15.120:0/2271175523 wait complete. 2024-09-19T09:09:25.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.868+0000 7f056de51740 20 client.17748 trim_cache size 0 max 16384 2024-09-19T09:09:25.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.881+0000 7f056de51740 1 Processor -- start 2024-09-19T09:09:25.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.881+0000 7f056de51740 1 -- start start 2024-09-19T09:09:25.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.881+0000 7f056de51740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56347ec9c5d0 0x56347eca5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:25.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.881+0000 7f056de51740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56347ec9db40 0x56347ec9f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:25.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.881+0000 7f056de51740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56347eca7370 0x56347ec9f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:25.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.881+0000 7f056de51740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f05641ad5c0 con 0x56347eca7370 2024-09-19T09:09:25.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.881+0000 7f056de51740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f05641af100 con 0x56347ec9db40 2024-09-19T09:09:25.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.881+0000 7f056de51740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f05641adc90 con 0x56347ec9c5d0 2024-09-19T09:09:25.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.881+0000 7f056a015640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56347eca7370 0x56347ec9f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:25.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.881+0000 7f056a015640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56347eca7370 0x56347ec9f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35968/0 (socket says 172.21.15.120:35968) 2024-09-19T09:09:25.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.881+0000 7f056a015640 1 -- 172.21.15.120:0/3153680420 learned_addr learned my addr 172.21.15.120:0/3153680420 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:25.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.881+0000 7f0569013640 1 --2- 172.21.15.120:0/3153680420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56347ec9db40 0x56347ec9f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:25.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.881+0000 7f0569814640 1 --2- 172.21.15.120:0/3153680420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56347ec9c5d0 0x56347eca5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:25.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.882+0000 7f056a015640 1 -- 172.21.15.120:0/3153680420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56347ec9c5d0 msgr2=0x56347eca5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:25.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.882+0000 7f056a015640 1 --2- 172.21.15.120:0/3153680420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56347ec9c5d0 0x56347eca5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.882+0000 7f056a015640 1 -- 172.21.15.120:0/3153680420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56347ec9db40 msgr2=0x56347ec9f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:25.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.882+0000 7f056a015640 1 --2- 172.21.15.120:0/3153680420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56347ec9db40 0x56347ec9f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.882+0000 7f056a015640 1 -- 172.21.15.120:0/3153680420 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56347eca2510 con 0x56347eca7370 2024-09-19T09:09:25.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.882+0000 7f056a015640 1 --2- 172.21.15.120:0/3153680420 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56347eca7370 0x56347ec9f6c0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f055c003f50 tx=0x7f055c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2291b5918e089751 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:25.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.882+0000 7f054f7fe640 1 -- 172.21.15.120:0/3153680420 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c03f020 con 0x56347eca7370 2024-09-19T09:09:25.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.882+0000 7f054f7fe640 1 -- 172.21.15.120:0/3153680420 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f055c043050 con 0x56347eca7370 2024-09-19T09:09:25.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.882+0000 7f054f7fe640 1 -- 172.21.15.120:0/3153680420 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c041040 con 0x56347eca7370 2024-09-19T09:09:25.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.883+0000 7f056de51740 1 -- 172.21.15.120:0/3153680420 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56347eca7370 msgr2=0x56347ec9f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:25.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.883+0000 7f056de51740 1 --2- 172.21.15.120:0/3153680420 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56347eca7370 0x56347ec9f6c0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f055c003f50 tx=0x7f055c004a40 comp rx=0 tx=0).stop 2024-09-19T09:09:25.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.883+0000 7f056de51740 1 -- 172.21.15.120:0/3153680420 shutdown_connections 2024-09-19T09:09:25.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.883+0000 7f056de51740 1 --2- 172.21.15.120:0/3153680420 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56347eca7370 0x56347ec9f6c0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.883+0000 7f056de51740 1 --2- 172.21.15.120:0/3153680420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56347ec9db40 0x56347ec9f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.883+0000 7f056de51740 1 --2- 172.21.15.120:0/3153680420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56347ec9c5d0 0x56347eca5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.883+0000 7f056de51740 1 -- 172.21.15.120:0/3153680420 >> 172.21.15.120:0/3153680420 conn(0x56347ec65310 msgr2=0x56347eca5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:25.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.883+0000 7f056de51740 1 -- 172.21.15.120:0/3153680420 shutdown_connections 2024-09-19T09:09:25.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.883+0000 7f056de51740 1 -- 172.21.15.120:0/3153680420 wait complete. 2024-09-19T09:09:25.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.883+0000 7f056de51740 1 -- start start 2024-09-19T09:09:25.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.883+0000 7f056de51740 1 Processor -- start 2024-09-19T09:09:25.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.884+0000 7f056de51740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56347ec9c5d0 0x56347eca11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:25.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.884+0000 7f056de51740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56347ec9db40 0x56347eca1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:25.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.884+0000 7f056de51740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56347eca7370 0x56347eca1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:25.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.884+0000 7f056de51740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f05641aea30 con 0x56347eca7370 2024-09-19T09:09:25.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.884+0000 7f056de51740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f05641ae360 con 0x56347ec9db40 2024-09-19T09:09:25.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.884+0000 7f056de51740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f05641ab580 con 0x56347ec9c5d0 2024-09-19T09:09:25.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.884+0000 7f056de51740 10 client.0 _ll_register_callbacks cb 0x7f055c031bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:25.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.884+0000 7f0569814640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56347ec9c5d0 0x56347eca11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:25.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.884+0000 7f0569814640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56347ec9c5d0 0x56347eca11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:45704/0 (socket says 172.21.15.120:45704) 2024-09-19T09:09:25.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.884+0000 7f0569814640 1 -- 172.21.15.120:0/4140241499 learned_addr learned my addr 172.21.15.120:0/4140241499 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:25.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.884+0000 7f056a015640 1 --2- 172.21.15.120:0/4140241499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56347eca7370 0x56347eca1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:25.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.884+0000 7f0569013640 1 --2- 172.21.15.120:0/4140241499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56347ec9db40 0x56347eca1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:25.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.884+0000 7f0569814640 1 -- 172.21.15.120:0/4140241499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56347ec9db40 msgr2=0x56347eca1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:25.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.884+0000 7f0569814640 1 --2- 172.21.15.120:0/4140241499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56347ec9db40 0x56347eca1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.884+0000 7f0569814640 1 -- 172.21.15.120:0/4140241499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56347eca7370 msgr2=0x56347eca1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:25.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.884+0000 7f0569814640 1 --2- 172.21.15.120:0/4140241499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56347eca7370 0x56347eca1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.884+0000 7f0569814640 1 -- 172.21.15.120:0/4140241499 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56347ed25220 con 0x56347ec9c5d0 2024-09-19T09:09:25.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.885+0000 7f0569814640 1 --2- 172.21.15.120:0/4140241499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56347ec9c5d0 0x56347eca11d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0560009a90 tx=0x7f0560004350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e703cabfb7e5110 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:25.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.885+0000 7f054d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:09:25.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.885+0000 7f054d7fa640 1 -- 172.21.15.120:0/4140241499 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056002d070 con 0x56347ec9c5d0 2024-09-19T09:09:25.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.885+0000 7f054d7fa640 1 -- 172.21.15.120:0/4140241499 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f0560029950 con 0x56347ec9c5d0 2024-09-19T09:09:25.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.885+0000 7f056de51740 1 -- 172.21.15.120:0/4140241499 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f055c043050 con 0x56347ec9c5d0 2024-09-19T09:09:25.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.885+0000 7f054d7fa640 1 -- 172.21.15.120:0/4140241499 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0560029b50 con 0x56347ec9c5d0 2024-09-19T09:09:25.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.885+0000 7f056de51740 10 client.26905 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:25.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.885+0000 7f056de51740 1 -- 172.21.15.120:0/4140241499 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56347ed20e70 con 0x56347ec9c5d0 2024-09-19T09:09:25.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.885+0000 7f054d7fa640 1 -- 172.21.15.120:0/4140241499 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f055c043050 con 0x56347ec9c5d0 2024-09-19T09:09:25.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.885+0000 7f054d7fa640 1 -- 172.21.15.120:0/4140241499 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0560030020 con 0x56347ec9c5d0 2024-09-19T09:09:25.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.885+0000 7f056de51740 10 client.26905 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:25.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.885+0000 7f056de51740 10 client.26905 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:09:25.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.885+0000 7f056de51740 20 client.26905 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:25.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.885+0000 7f056de51740 1 --2- 172.21.15.120:0/4140241499 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56347ed21590 0x56347ed239b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:25.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.885+0000 7f056de51740 4 client.26905 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:09:25.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.885+0000 7f056de51740 1 -- 172.21.15.120:0/4140241499 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56347ec09d70 con 0x56347ed21590 2024-09-19T09:09:25.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.885+0000 7f0569013640 1 --2- 172.21.15.120:0/4140241499 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56347ed21590 0x56347ed239b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:25.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.886+0000 7f0569013640 1 --2- 172.21.15.120:0/4140241499 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56347ed21590 0x56347ed239b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=29369304570238ba server_cookie=927661fb6a5e66fa in_seq=0 out_seq=0 2024-09-19T09:09:25.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.886+0000 7f054d7fa640 10 client.26905 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:09:25.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.886+0000 7f054d7fa640 1 -- 172.21.15.120:0/4140241499 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56347ec09d70 con 0x56347ed21590 2024-09-19T09:09:25.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.886+0000 7f054d7fa640 10 client.26905 handle_command_reply: tid=0 2024-09-19T09:09:25.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.886+0000 7f056de51740 1 client.26905 shutdown 2024-09-19T09:09:25.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.886+0000 7f056de51740 1 -- 172.21.15.120:0/4140241499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56347ec9c5d0 msgr2=0x56347eca11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:25.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.886+0000 7f056de51740 1 --2- 172.21.15.120:0/4140241499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56347ec9c5d0 0x56347eca11d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0560009a90 tx=0x7f0560004350 comp rx=0 tx=0).stop 2024-09-19T09:09:25.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.886+0000 7f056de51740 1 -- 172.21.15.120:0/4140241499 shutdown_connections 2024-09-19T09:09:25.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.886+0000 7f056de51740 1 --2- 172.21.15.120:0/4140241499 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56347ed21590 0x56347ed239b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.886+0000 7f056de51740 1 --2- 172.21.15.120:0/4140241499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56347eca7370 0x56347eca1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.886+0000 7f056de51740 1 --2- 172.21.15.120:0/4140241499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56347ec9db40 0x56347eca1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.886+0000 7f056de51740 1 --2- 172.21.15.120:0/4140241499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56347ec9c5d0 0x56347eca11d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.886+0000 7f054d7fa640 0 client.26905 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:09:25.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.886+0000 7f056de51740 1 -- 172.21.15.120:0/4140241499 >> 172.21.15.120:0/4140241499 conn(0x56347ec65310 msgr2=0x56347eca6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:25.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.886+0000 7f056de51740 1 -- 172.21.15.120:0/4140241499 shutdown_connections 2024-09-19T09:09:25.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.887+0000 7f056de51740 1 -- 172.21.15.120:0/4140241499 wait complete. 2024-09-19T09:09:25.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.887+0000 7f056de51740 20 client.26905 trim_cache size 0 max 16384 2024-09-19T09:09:25.897 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:09:25.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.887+0000 7f056ba78640 1 -- 172.21.15.120:0/3859736486 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f05500a3240 msgr2=0x7f05500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:25.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.887+0000 7f056ba78640 1 --2- 172.21.15.120:0/3859736486 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f05500a3240 0x7f05500a5700 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f055c004ab0 tx=0x7f055c031430 comp rx=0 tx=0).stop 2024-09-19T09:09:25.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.887+0000 7f056ba78640 1 -- 172.21.15.120:0/3859736486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0564103740 msgr2=0x7f0564198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:25.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.887+0000 7f056ba78640 1 --2- 172.21.15.120:0/3859736486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0564103740 0x7f0564198b20 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f055400e9f0 tx=0x7f055400eeb0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.888+0000 7f056ba78640 1 -- 172.21.15.120:0/3859736486 shutdown_connections 2024-09-19T09:09:25.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.888+0000 7f056ba78640 1 --2- 172.21.15.120:0/3859736486 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f05500a3240 0x7f05500a5700 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.888+0000 7f056ba78640 1 --2- 172.21.15.120:0/3859736486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0564108290 0x7f056419ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.888+0000 7f056ba78640 1 --2- 172.21.15.120:0/3859736486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0564103740 0x7f0564198b20 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.888+0000 7f056ba78640 1 --2- 172.21.15.120:0/3859736486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0564102d90 0x7f05641985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:25.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.888+0000 7f056ba78640 1 -- 172.21.15.120:0/3859736486 >> 172.21.15.120:0/3859736486 conn(0x7f05640fc7d0 msgr2=0x7f0564106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:25.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.888+0000 7f056ba78640 1 -- 172.21.15.120:0/3859736486 shutdown_connections 2024-09-19T09:09:25.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:25.888+0000 7f056ba78640 1 -- 172.21.15.120:0/3859736486 wait complete. 2024-09-19T09:09:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.050+0000 7f3136623640 1 -- 172.21.15.120:0/3668137196 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3130100a50 msgr2=0x7f3130100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.050+0000 7f3136623640 1 --2- 172.21.15.120:0/3668137196 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3130100a50 0x7f3130100e50 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f3120009900 tx=0x7f312002a410 comp rx=0 tx=0).stop 2024-09-19T09:09:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.050+0000 7f3136623640 1 -- 172.21.15.120:0/3668137196 shutdown_connections 2024-09-19T09:09:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.050+0000 7f3136623640 1 --2- 172.21.15.120:0/3668137196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3130102600 0x7f3130108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.050+0000 7f3136623640 1 --2- 172.21.15.120:0/3668137196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3130101c40 0x7f31301020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.050+0000 7f3136623640 1 --2- 172.21.15.120:0/3668137196 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3130100a50 0x7f3130100e50 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.050+0000 7f3136623640 1 -- 172.21.15.120:0/3668137196 >> 172.21.15.120:0/3668137196 conn(0x7f31300fc1b0 msgr2=0x7f31300fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.050+0000 7f3136623640 1 -- 172.21.15.120:0/3668137196 shutdown_connections 2024-09-19T09:09:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.050+0000 7f3136623640 1 -- 172.21.15.120:0/3668137196 wait complete. 2024-09-19T09:09:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.051+0000 7f3136623640 1 Processor -- start 2024-09-19T09:09:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.051+0000 7f3136623640 1 -- start start 2024-09-19T09:09:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.051+0000 7f3136623640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3130100a50 0x7f3130197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.051+0000 7f3136623640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3130101c40 0x7f3130198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.051+0000 7f3136623640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3130102600 0x7f313019f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.051+0000 7f3136623640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f313010c6b0 con 0x7f3130102600 2024-09-19T09:09:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.051+0000 7f3136623640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f313010c530 con 0x7f3130101c40 2024-09-19T09:09:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.051+0000 7f3136623640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f313010c830 con 0x7f3130100a50 2024-09-19T09:09:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.051+0000 7f312ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3130100a50 0x7f3130197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.051+0000 7f3134bc0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3130102600 0x7f313019f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.051+0000 7f312ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3130100a50 0x7f3130197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:45734/0 (socket says 172.21.15.120:45734) 2024-09-19T09:09:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.051+0000 7f3134bc0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3130102600 0x7f313019f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35984/0 (socket says 172.21.15.120:35984) 2024-09-19T09:09:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.051+0000 7f312f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3130101c40 0x7f3130198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.051+0000 7f312ffff640 1 -- 172.21.15.120:0/1609497322 learned_addr learned my addr 172.21.15.120:0/1609497322 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.052+0000 7f312f7fe640 1 -- 172.21.15.120:0/1609497322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3130100a50 msgr2=0x7f3130197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.052+0000 7f312f7fe640 1 --2- 172.21.15.120:0/1609497322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3130100a50 0x7f3130197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.052+0000 7f312f7fe640 1 -- 172.21.15.120:0/1609497322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3130102600 msgr2=0x7f313019f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.052+0000 7f312f7fe640 1 --2- 172.21.15.120:0/1609497322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3130102600 0x7f313019f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.052+0000 7f312f7fe640 1 -- 172.21.15.120:0/1609497322 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313019f9e0 con 0x7f3130101c40 2024-09-19T09:09:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.052+0000 7f3134bc0640 1 --2- 172.21.15.120:0/1609497322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3130102600 0x7f313019f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.052+0000 7f312ffff640 1 --2- 172.21.15.120:0/1609497322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3130100a50 0x7f3130197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:26.057 INFO:tasks.workunit.client.0.smithi120.stdout: 1 586252 9.71 MB/sec execute 201 sec latency 32.159 ms 2024-09-19T09:09:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.052+0000 7f312f7fe640 1 --2- 172.21.15.120:0/1609497322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3130101c40 0x7f3130198300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f312400e9f0 tx=0x7f312400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.052+0000 7f312d7fa640 1 -- 172.21.15.120:0/1609497322 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312400ccc0 con 0x7f3130101c40 2024-09-19T09:09:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.052+0000 7f3136623640 1 -- 172.21.15.120:0/1609497322 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313019fcd0 con 0x7f3130101c40 2024-09-19T09:09:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.052+0000 7f312d7fa640 1 -- 172.21.15.120:0/1609497322 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3124010910 con 0x7f3130101c40 2024-09-19T09:09:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.052+0000 7f312d7fa640 1 -- 172.21.15.120:0/1609497322 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124018960 con 0x7f3130101c40 2024-09-19T09:09:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.052+0000 7f3136623640 1 -- 172.21.15.120:0/1609497322 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31301a0210 con 0x7f3130101c40 2024-09-19T09:09:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.054+0000 7f312d7fa640 1 -- 172.21.15.120:0/1609497322 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f3124010ac0 con 0x7f3130101c40 2024-09-19T09:09:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.054+0000 7f31393bc740 1 Processor -- start 2024-09-19T09:09:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.055+0000 7f31393bc740 1 -- start start 2024-09-19T09:09:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.055+0000 7f31393bc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811617b9a0 0x55811617bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.055+0000 7f31393bc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55811617c2e0 0x558116180da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.055+0000 7f31393bc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558116181370 0x558116183830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.055+0000 7f31393bc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5581160df2f0 con 0x55811617b9a0 2024-09-19T09:09:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.055+0000 7f31393bc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558116087270 con 0x55811617c2e0 2024-09-19T09:09:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.055+0000 7f31393bc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5581160db2b0 con 0x558116181370 2024-09-19T09:09:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.055+0000 7f3134bc0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811617b9a0 0x55811617bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.055+0000 7f3134bc0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811617b9a0 0x55811617bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35990/0 (socket says 172.21.15.120:35990) 2024-09-19T09:09:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.055+0000 7f3134bc0640 1 -- 172.21.15.120:0/500461717 learned_addr learned my addr 172.21.15.120:0/500461717 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.055+0000 7f312ffff640 1 --2- 172.21.15.120:0/500461717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55811617c2e0 0x558116180da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.055+0000 7f3134bc0640 1 -- 172.21.15.120:0/500461717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558116181370 msgr2=0x558116183830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.055+0000 7f3134bc0640 1 --2- 172.21.15.120:0/500461717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558116181370 0x558116183830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.055+0000 7f3134bc0640 1 -- 172.21.15.120:0/500461717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55811617c2e0 msgr2=0x558116180da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.055+0000 7f3134bc0640 1 --2- 172.21.15.120:0/500461717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55811617c2e0 0x558116180da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.055+0000 7f3134bc0640 1 -- 172.21.15.120:0/500461717 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558116086340 con 0x55811617b9a0 2024-09-19T09:09:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.055+0000 7f312d7fa640 1 --2- 172.21.15.120:0/1609497322 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f31180a32f0 0x7f31180a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.055+0000 7f312ffff640 1 --2- 172.21.15.120:0/1609497322 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f31180a32f0 0x7f31180a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.055+0000 7f3134bc0640 1 --2- 172.21.15.120:0/500461717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811617b9a0 0x55811617bda0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f31280117e0 tx=0x7f3128011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ca72e4f1531a65 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.055+0000 7f312d7fa640 1 -- 172.21.15.120:0/1609497322 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f31240cfcc0 con 0x7f3130101c40 2024-09-19T09:09:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.056+0000 7f31167fc640 1 -- 172.21.15.120:0/500461717 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128011310 con 0x55811617b9a0 2024-09-19T09:09:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.056+0000 7f31167fc640 1 -- 172.21.15.120:0/500461717 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3128003e50 con 0x55811617b9a0 2024-09-19T09:09:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.056+0000 7f31167fc640 1 -- 172.21.15.120:0/500461717 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128004590 con 0x55811617b9a0 2024-09-19T09:09:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.056+0000 7f31393bc740 1 -- 172.21.15.120:0/500461717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811617b9a0 msgr2=0x55811617bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.056+0000 7f31393bc740 1 --2- 172.21.15.120:0/500461717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811617b9a0 0x55811617bda0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f31280117e0 tx=0x7f3128011ca0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.056+0000 7f312ffff640 1 --2- 172.21.15.120:0/1609497322 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f31180a32f0 0x7f31180a57b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3120000c00 tx=0x7f31200046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.056+0000 7f31393bc740 1 -- 172.21.15.120:0/500461717 shutdown_connections 2024-09-19T09:09:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.056+0000 7f31393bc740 1 --2- 172.21.15.120:0/500461717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558116181370 0x558116183830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.056+0000 7f31393bc740 1 --2- 172.21.15.120:0/500461717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55811617c2e0 0x558116180da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.056+0000 7f31393bc740 1 --2- 172.21.15.120:0/500461717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811617b9a0 0x55811617bda0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.056+0000 7f31393bc740 1 -- 172.21.15.120:0/500461717 >> 172.21.15.120:0/500461717 conn(0x558116177010 msgr2=0x558116177410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.056+0000 7f31393bc740 1 -- 172.21.15.120:0/500461717 shutdown_connections 2024-09-19T09:09:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.056+0000 7f31393bc740 1 -- 172.21.15.120:0/500461717 wait complete. 2024-09-19T09:09:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.057+0000 7f31393bc740 1 -- start start 2024-09-19T09:09:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.057+0000 7f31393bc740 1 Processor -- start 2024-09-19T09:09:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.057+0000 7f31393bc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811617b9a0 0x558116185c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.057+0000 7f31393bc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55811617c2e0 0x558116186190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.057+0000 7f31393bc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558116181370 0x55811618d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.057+0000 7f31393bc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558116184340 con 0x55811617b9a0 2024-09-19T09:09:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.057+0000 7f31393bc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558116068290 con 0x55811617c2e0 2024-09-19T09:09:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.057+0000 7f31393bc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5581161844c0 con 0x558116181370 2024-09-19T09:09:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.057+0000 7f31393bc740 10 client.0 _ll_register_callbacks cb 0x558116099140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.057+0000 7f312ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811617b9a0 0x558116185c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.057+0000 7f312ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811617b9a0 0x558116185c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35998/0 (socket says 172.21.15.120:35998) 2024-09-19T09:09:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.057+0000 7f312ffff640 1 -- 172.21.15.120:0/266374032 learned_addr learned my addr 172.21.15.120:0/266374032 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.057+0000 7f312f7fe640 1 --2- 172.21.15.120:0/266374032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55811617c2e0 0x558116186190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.057+0000 7f3134bc0640 1 --2- 172.21.15.120:0/266374032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558116181370 0x55811618d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.057+0000 7f312ffff640 1 -- 172.21.15.120:0/266374032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558116181370 msgr2=0x55811618d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.057+0000 7f312ffff640 1 --2- 172.21.15.120:0/266374032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558116181370 0x55811618d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.057+0000 7f312ffff640 1 -- 172.21.15.120:0/266374032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55811617c2e0 msgr2=0x558116186190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.057+0000 7f312ffff640 1 --2- 172.21.15.120:0/266374032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55811617c2e0 0x558116186190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.058+0000 7f312ffff640 1 -- 172.21.15.120:0/266374032 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558115df2210 con 0x55811617b9a0 2024-09-19T09:09:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.058+0000 7f312ffff640 1 --2- 172.21.15.120:0/266374032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811617b9a0 0x558116185c50 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f3120009230 tx=0x7f3120009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2d2fdb1890fecf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.058+0000 7f310f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:09:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.058+0000 7f310f7fe640 1 -- 172.21.15.120:0/266374032 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312004d070 con 0x55811617b9a0 2024-09-19T09:09:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.058+0000 7f310f7fe640 1 -- 172.21.15.120:0/266374032 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f3120005220 con 0x55811617b9a0 2024-09-19T09:09:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.058+0000 7f31393bc740 1 -- 172.21.15.120:0/266374032 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3128003e50 con 0x55811617b9a0 2024-09-19T09:09:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.058+0000 7f310f7fe640 1 -- 172.21.15.120:0/266374032 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3120043070 con 0x55811617b9a0 2024-09-19T09:09:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.058+0000 7f31393bc740 10 client.17766 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.058+0000 7f31393bc740 1 -- 172.21.15.120:0/266374032 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558115ebe620 con 0x55811617b9a0 2024-09-19T09:09:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.058+0000 7f310f7fe640 1 -- 172.21.15.120:0/266374032 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3128003e50 con 0x55811617b9a0 2024-09-19T09:09:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.058+0000 7f310f7fe640 1 -- 172.21.15.120:0/266374032 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3120004960 con 0x55811617b9a0 2024-09-19T09:09:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.058+0000 7f31393bc740 10 client.17766 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.058+0000 7f31393bc740 10 client.17766 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:09:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.058+0000 7f31393bc740 20 client.17766 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.058+0000 7f31393bc740 1 --2- 172.21.15.120:0/266374032 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5581161918c0 0x558116193cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.058+0000 7f31393bc740 4 client.17766 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:09:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.058+0000 7f31393bc740 1 -- 172.21.15.120:0/266374032 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581160eca20 con 0x5581161918c0 2024-09-19T09:09:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.059+0000 7f312f7fe640 1 --2- 172.21.15.120:0/266374032 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5581161918c0 0x558116193cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.059+0000 7f310f7fe640 10 client.17766 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.059+0000 7f312f7fe640 1 --2- 172.21.15.120:0/266374032 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5581161918c0 0x558116193cb0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f00e9c3cde53ce3 server_cookie=c5c79539cd10ea7b in_seq=0 out_seq=0 2024-09-19T09:09:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.061+0000 7f310f7fe640 1 -- 172.21.15.120:0/266374032 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5581160eca20 con 0x5581161918c0 2024-09-19T09:09:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.061+0000 7f310f7fe640 10 client.17766 handle_command_reply: tid=0 2024-09-19T09:09:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.061+0000 7f31393bc740 1 client.17766 shutdown 2024-09-19T09:09:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.061+0000 7f31393bc740 1 -- 172.21.15.120:0/266374032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811617b9a0 msgr2=0x558116185c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.061+0000 7f31393bc740 1 --2- 172.21.15.120:0/266374032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811617b9a0 0x558116185c50 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f3120009230 tx=0x7f3120009260 comp rx=0 tx=0).stop 2024-09-19T09:09:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.061+0000 7f31393bc740 1 -- 172.21.15.120:0/266374032 shutdown_connections 2024-09-19T09:09:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.061+0000 7f31393bc740 1 --2- 172.21.15.120:0/266374032 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5581161918c0 0x558116193cb0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.061+0000 7f31393bc740 1 --2- 172.21.15.120:0/266374032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558116181370 0x55811618d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.061+0000 7f31393bc740 1 --2- 172.21.15.120:0/266374032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55811617c2e0 0x558116186190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.061+0000 7f31393bc740 1 --2- 172.21.15.120:0/266374032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811617b9a0 0x558116185c50 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.061+0000 7f31393bc740 1 -- 172.21.15.120:0/266374032 >> 172.21.15.120:0/266374032 conn(0x558116177010 msgr2=0x558116178820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.061+0000 7f310f7fe640 0 client.17766 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.061+0000 7f31393bc740 1 -- 172.21.15.120:0/266374032 shutdown_connections 2024-09-19T09:09:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.061+0000 7f31393bc740 1 -- 172.21.15.120:0/266374032 wait complete. 2024-09-19T09:09:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.062+0000 7f31393bc740 20 client.17766 trim_cache size 0 max 16384 2024-09-19T09:09:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.076+0000 7f31393bc740 1 Processor -- start 2024-09-19T09:09:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.076+0000 7f31393bc740 1 -- start start 2024-09-19T09:09:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.076+0000 7f31393bc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55811617b9a0 0x558116186220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.076+0000 7f31393bc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811617c2e0 0x55811618d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.076+0000 7f31393bc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558116181370 0x5581161760b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.076+0000 7f31393bc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f31301a9030 con 0x55811617c2e0 2024-09-19T09:09:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.076+0000 7f31393bc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f31301aab70 con 0x558116181370 2024-09-19T09:09:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.076+0000 7f31393bc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f31301a9700 con 0x55811617b9a0 2024-09-19T09:09:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.076+0000 7f312f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811617c2e0 0x55811618d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.076+0000 7f312f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811617c2e0 0x55811618d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36008/0 (socket says 172.21.15.120:36008) 2024-09-19T09:09:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.076+0000 7f312f7fe640 1 -- 172.21.15.120:0/157939383 learned_addr learned my addr 172.21.15.120:0/157939383 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.076+0000 7f312ffff640 1 --2- 172.21.15.120:0/157939383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55811617b9a0 0x558116186220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.076+0000 7f3134bc0640 1 --2- 172.21.15.120:0/157939383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558116181370 0x5581161760b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.076+0000 7f312f7fe640 1 -- 172.21.15.120:0/157939383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55811617b9a0 msgr2=0x558116186220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.076+0000 7f312f7fe640 1 --2- 172.21.15.120:0/157939383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55811617b9a0 0x558116186220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.076+0000 7f312f7fe640 1 -- 172.21.15.120:0/157939383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558116181370 msgr2=0x5581161760b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.076+0000 7f312f7fe640 1 --2- 172.21.15.120:0/157939383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558116181370 0x5581161760b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.077+0000 7f312f7fe640 1 -- 172.21.15.120:0/157939383 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558116185790 con 0x55811617c2e0 2024-09-19T09:09:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.077+0000 7f312f7fe640 1 --2- 172.21.15.120:0/157939383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811617c2e0 0x55811618d190 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f312409c2e0 tx=0x7f3124009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc76274af05c9f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.077+0000 7f3115ffb640 1 -- 172.21.15.120:0/157939383 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31240b5070 con 0x55811617c2e0 2024-09-19T09:09:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.077+0000 7f3115ffb640 1 -- 172.21.15.120:0/157939383 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31240a4f10 con 0x55811617c2e0 2024-09-19T09:09:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.077+0000 7f3115ffb640 1 -- 172.21.15.120:0/157939383 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31240a0170 con 0x55811617c2e0 2024-09-19T09:09:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.077+0000 7f31393bc740 1 -- 172.21.15.120:0/157939383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811617c2e0 msgr2=0x55811618d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.077+0000 7f31393bc740 1 --2- 172.21.15.120:0/157939383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811617c2e0 0x55811618d190 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f312409c2e0 tx=0x7f3124009700 comp rx=0 tx=0).stop 2024-09-19T09:09:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.078+0000 7f31393bc740 1 -- 172.21.15.120:0/157939383 shutdown_connections 2024-09-19T09:09:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.078+0000 7f31393bc740 1 --2- 172.21.15.120:0/157939383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558116181370 0x5581161760b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.078+0000 7f31393bc740 1 --2- 172.21.15.120:0/157939383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811617c2e0 0x55811618d190 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.078+0000 7f31393bc740 1 --2- 172.21.15.120:0/157939383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55811617b9a0 0x558116186220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.078+0000 7f31393bc740 1 -- 172.21.15.120:0/157939383 >> 172.21.15.120:0/157939383 conn(0x558116177010 msgr2=0x5581161759a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.078+0000 7f31393bc740 1 -- 172.21.15.120:0/157939383 shutdown_connections 2024-09-19T09:09:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.078+0000 7f31393bc740 1 -- 172.21.15.120:0/157939383 wait complete. 2024-09-19T09:09:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.078+0000 7f31393bc740 1 -- start start 2024-09-19T09:09:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.078+0000 7f31393bc740 1 Processor -- start 2024-09-19T09:09:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.078+0000 7f31393bc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55811617b9a0 0x55811618a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.078+0000 7f31393bc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811617c2e0 0x55811618a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.078+0000 7f31393bc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558116181370 0x55811618b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.078+0000 7f31393bc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f31301aa4a0 con 0x55811617c2e0 2024-09-19T09:09:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.078+0000 7f31393bc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f31301a9dd0 con 0x55811617b9a0 2024-09-19T09:09:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.078+0000 7f31393bc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f31301a6ff0 con 0x558116181370 2024-09-19T09:09:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.078+0000 7f31393bc740 10 client.0 _ll_register_callbacks cb 0x7f3120009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.078+0000 7f312f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811617c2e0 0x55811618a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.078+0000 7f312ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55811617b9a0 0x55811618a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.078+0000 7f312f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811617c2e0 0x55811618a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36012/0 (socket says 172.21.15.120:36012) 2024-09-19T09:09:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.078+0000 7f312f7fe640 1 -- 172.21.15.120:0/3438063654 learned_addr learned my addr 172.21.15.120:0/3438063654 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.078+0000 7f3134bc0640 1 --2- 172.21.15.120:0/3438063654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558116181370 0x55811618b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.079+0000 7f312f7fe640 1 -- 172.21.15.120:0/3438063654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558116181370 msgr2=0x55811618b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.079+0000 7f312f7fe640 1 --2- 172.21.15.120:0/3438063654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558116181370 0x55811618b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.079+0000 7f312f7fe640 1 -- 172.21.15.120:0/3438063654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55811617b9a0 msgr2=0x55811618a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.079+0000 7f312f7fe640 1 --2- 172.21.15.120:0/3438063654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55811617b9a0 0x55811618a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.079+0000 7f312f7fe640 1 -- 172.21.15.120:0/3438063654 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55811618bf00 con 0x55811617c2e0 2024-09-19T09:09:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.079+0000 7f312ffff640 1 --2- 172.21.15.120:0/3438063654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55811617b9a0 0x55811618a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.079+0000 7f312f7fe640 1 --2- 172.21.15.120:0/3438063654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811617c2e0 0x55811618a8e0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f3124010f50 tx=0x7f3124097f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e801a60b6875a1f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.079+0000 7f310f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:09:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.079+0000 7f310f7fe640 1 -- 172.21.15.120:0/3438063654 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31240b5040 con 0x55811617c2e0 2024-09-19T09:09:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.079+0000 7f310f7fe640 1 -- 172.21.15.120:0/3438063654 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f31240bc0d0 con 0x55811617c2e0 2024-09-19T09:09:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.079+0000 7f31393bc740 1 -- 172.21.15.120:0/3438063654 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31240a4f10 con 0x55811617c2e0 2024-09-19T09:09:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.079+0000 7f310f7fe640 1 -- 172.21.15.120:0/3438063654 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f31240bc2d0 con 0x55811617c2e0 2024-09-19T09:09:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.079+0000 7f31393bc740 10 client.17772 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.079+0000 7f31393bc740 1 -- 172.21.15.120:0/3438063654 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55811618d280 con 0x55811617c2e0 2024-09-19T09:09:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.079+0000 7f310f7fe640 1 -- 172.21.15.120:0/3438063654 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31240a4f10 con 0x55811617c2e0 2024-09-19T09:09:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.079+0000 7f310f7fe640 1 -- 172.21.15.120:0/3438063654 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f312409b8c0 con 0x55811617c2e0 2024-09-19T09:09:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.079+0000 7f31393bc740 10 client.17772 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.079+0000 7f31393bc740 10 client.17772 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:09:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.079+0000 7f31393bc740 20 client.17772 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.079+0000 7f31393bc740 1 --2- 172.21.15.120:0/3438063654 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5581161fc9f0 0x5581161fee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.079+0000 7f31393bc740 4 client.17772 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:09:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.079+0000 7f31393bc740 1 -- 172.21.15.120:0/3438063654 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558116181130 con 0x5581161fc9f0 2024-09-19T09:09:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.080+0000 7f312ffff640 1 --2- 172.21.15.120:0/3438063654 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5581161fc9f0 0x5581161fee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.080+0000 7f310f7fe640 10 client.17772 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.080+0000 7f312ffff640 1 --2- 172.21.15.120:0/3438063654 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5581161fc9f0 0x5581161fee10 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f1230b22a2e8128 server_cookie=f3927e4ddefe3887 in_seq=0 out_seq=0 2024-09-19T09:09:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.253+0000 7f310f7fe640 1 -- 172.21.15.120:0/3438063654 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558116181130 con 0x5581161fc9f0 2024-09-19T09:09:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.253+0000 7f310f7fe640 10 client.17772 handle_command_reply: tid=0 2024-09-19T09:09:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.253+0000 7f31393bc740 1 client.17772 shutdown 2024-09-19T09:09:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.254+0000 7f31393bc740 1 -- 172.21.15.120:0/3438063654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811617c2e0 msgr2=0x55811618a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.254+0000 7f31393bc740 1 --2- 172.21.15.120:0/3438063654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811617c2e0 0x55811618a8e0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f3124010f50 tx=0x7f3124097f60 comp rx=0 tx=0).stop 2024-09-19T09:09:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.254+0000 7f31393bc740 1 -- 172.21.15.120:0/3438063654 shutdown_connections 2024-09-19T09:09:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.254+0000 7f31393bc740 1 --2- 172.21.15.120:0/3438063654 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5581161fc9f0 0x5581161fee10 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.254+0000 7f31393bc740 1 --2- 172.21.15.120:0/3438063654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558116181370 0x55811618b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.254+0000 7f310f7fe640 0 client.17772 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.254+0000 7f31393bc740 1 --2- 172.21.15.120:0/3438063654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55811617c2e0 0x55811618a8e0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.254+0000 7f31393bc740 1 --2- 172.21.15.120:0/3438063654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55811617b9a0 0x55811618a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.254+0000 7f31393bc740 1 -- 172.21.15.120:0/3438063654 >> 172.21.15.120:0/3438063654 conn(0x558116177010 msgr2=0x5581161761c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.254+0000 7f31393bc740 1 -- 172.21.15.120:0/3438063654 shutdown_connections 2024-09-19T09:09:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.254+0000 7f31393bc740 1 -- 172.21.15.120:0/3438063654 wait complete. 2024-09-19T09:09:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.254+0000 7f31393bc740 20 client.17772 trim_cache size 0 max 16384 2024-09-19T09:09:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:09:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:09:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:09:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:09:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.255+0000 7f3136623640 1 -- 172.21.15.120:0/1609497322 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f31180a32f0 msgr2=0x7f31180a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.255+0000 7f3136623640 1 --2- 172.21.15.120:0/1609497322 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f31180a32f0 0x7f31180a57b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3120000c00 tx=0x7f31200046c0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.255+0000 7f3136623640 1 -- 172.21.15.120:0/1609497322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3130101c40 msgr2=0x7f3130198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.255+0000 7f3136623640 1 --2- 172.21.15.120:0/1609497322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3130101c40 0x7f3130198300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f312400e9f0 tx=0x7f312400eeb0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.255+0000 7f3136623640 1 -- 172.21.15.120:0/1609497322 shutdown_connections 2024-09-19T09:09:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.255+0000 7f3136623640 1 --2- 172.21.15.120:0/1609497322 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f31180a32f0 0x7f31180a57b0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.255+0000 7f3136623640 1 --2- 172.21.15.120:0/1609497322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3130102600 0x7f313019f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.255+0000 7f3136623640 1 --2- 172.21.15.120:0/1609497322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3130101c40 0x7f3130198300 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.255+0000 7f3136623640 1 --2- 172.21.15.120:0/1609497322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3130100a50 0x7f3130197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.255+0000 7f3136623640 1 -- 172.21.15.120:0/1609497322 >> 172.21.15.120:0/1609497322 conn(0x7f31300fc1b0 msgr2=0x7f313006a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.255+0000 7f3136623640 1 -- 172.21.15.120:0/1609497322 shutdown_connections 2024-09-19T09:09:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:26.255+0000 7f3136623640 1 -- 172.21.15.120:0/1609497322 wait complete. 2024-09-19T09:09:26.760 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:09:27.030 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:09:27.053 INFO:tasks.workunit.client.0.smithi120.stdout: 1 588526 9.70 MB/sec execute 202 sec latency 124.002 ms 2024-09-19T09:09:27.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:26 smithi120 ceph-mon[31150]: pgmap v539: 97 pgs: 97 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.2 MiB/s wr, 398 op/s 2024-09-19T09:09:27.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:26 smithi196 ceph-mon[35150]: pgmap v539: 97 pgs: 97 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.2 MiB/s wr, 398 op/s 2024-09-19T09:09:27.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:26 smithi153 ceph-mon[35071]: pgmap v539: 97 pgs: 97 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.2 MiB/s wr, 398 op/s 2024-09-19T09:09:28.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.003+0000 7fc898fd7640 1 -- 172.21.15.120:0/2647765223 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc894102570 msgr2=0x7fc8941029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:28.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.003+0000 7fc898fd7640 1 --2- 172.21.15.120:0/2647765223 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc894102570 0x7fc8941029b0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fc888009900 tx=0x7fc88802a480 comp rx=0 tx=0).stop 2024-09-19T09:09:28.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.003+0000 7fc898fd7640 1 -- 172.21.15.120:0/2647765223 shutdown_connections 2024-09-19T09:09:28.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.003+0000 7fc898fd7640 1 --2- 172.21.15.120:0/2647765223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc894102ef0 0x7fc89410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.003+0000 7fc898fd7640 1 --2- 172.21.15.120:0/2647765223 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc894102570 0x7fc8941029b0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.003+0000 7fc898fd7640 1 --2- 172.21.15.120:0/2647765223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc894100e20 0x7fc894101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.003+0000 7fc898fd7640 1 -- 172.21.15.120:0/2647765223 >> 172.21.15.120:0/2647765223 conn(0x7fc8940fc7d0 msgr2=0x7fc8940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:28.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.003+0000 7fc898fd7640 1 -- 172.21.15.120:0/2647765223 shutdown_connections 2024-09-19T09:09:28.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.003+0000 7fc898fd7640 1 -- 172.21.15.120:0/2647765223 wait complete. 2024-09-19T09:09:28.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.004+0000 7fc898fd7640 1 Processor -- start 2024-09-19T09:09:28.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.004+0000 7fc898fd7640 1 -- start start 2024-09-19T09:09:28.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.005+0000 7fc898fd7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc894100e20 0x7fc894197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:28.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.005+0000 7fc898fd7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc894102570 0x7fc894197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:28.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.005+0000 7fc898fd7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc894102ef0 0x7fc89419bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:28.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.005+0000 7fc898fd7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc894110fd0 con 0x7fc894102ef0 2024-09-19T09:09:28.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.005+0000 7fc898fd7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc894110e50 con 0x7fc894102570 2024-09-19T09:09:28.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.005+0000 7fc892d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc894102ef0 0x7fc89419bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:28.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.005+0000 7fc898fd7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc894111150 con 0x7fc894100e20 2024-09-19T09:09:28.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.005+0000 7fc892d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc894102ef0 0x7fc89419bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36034/0 (socket says 172.21.15.120:36034) 2024-09-19T09:09:28.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.005+0000 7fc892d9d640 1 -- 172.21.15.120:0/809335970 learned_addr learned my addr 172.21.15.120:0/809335970 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:28.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.005+0000 7fc891d9b640 1 --2- 172.21.15.120:0/809335970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc894102570 0x7fc894197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:28.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.005+0000 7fc89259c640 1 --2- 172.21.15.120:0/809335970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc894100e20 0x7fc894197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:28.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.005+0000 7fc892d9d640 1 -- 172.21.15.120:0/809335970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc894100e20 msgr2=0x7fc894197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:28.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.005+0000 7fc892d9d640 1 --2- 172.21.15.120:0/809335970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc894100e20 0x7fc894197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.005+0000 7fc892d9d640 1 -- 172.21.15.120:0/809335970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc894102570 msgr2=0x7fc894197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:28.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.005+0000 7fc892d9d640 1 --2- 172.21.15.120:0/809335970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc894102570 0x7fc894197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.005+0000 7fc892d9d640 1 -- 172.21.15.120:0/809335970 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc89419c3e0 con 0x7fc894102ef0 2024-09-19T09:09:28.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.005+0000 7fc892d9d640 1 --2- 172.21.15.120:0/809335970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc894102ef0 0x7fc89419bd00 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00eb10 tx=0x7fc88c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:28.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.005+0000 7fc8837fe640 1 -- 172.21.15.120:0/809335970 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c004b20 con 0x7fc894102ef0 2024-09-19T09:09:28.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.005+0000 7fc898fd7640 1 -- 172.21.15.120:0/809335970 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc89419c6d0 con 0x7fc894102ef0 2024-09-19T09:09:28.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.005+0000 7fc8837fe640 1 -- 172.21.15.120:0/809335970 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc88c004cc0 con 0x7fc894102ef0 2024-09-19T09:09:28.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.005+0000 7fc8837fe640 1 -- 172.21.15.120:0/809335970 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c010760 con 0x7fc894102ef0 2024-09-19T09:09:28.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.006+0000 7fc898fd7640 1 -- 172.21.15.120:0/809335970 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc894069f70 con 0x7fc894102ef0 2024-09-19T09:09:28.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.008+0000 7fc8837fe640 1 -- 172.21.15.120:0/809335970 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fc88c005020 con 0x7fc894102ef0 2024-09-19T09:09:28.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.008+0000 7fc89b3b0740 1 Processor -- start 2024-09-19T09:09:28.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.008+0000 7fc89b3b0740 1 -- start start 2024-09-19T09:09:28.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.008+0000 7fc8837fe640 1 --2- 172.21.15.120:0/809335970 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc87c0a3240 0x7fc87c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:28.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.008+0000 7fc89b3b0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc65531b40 0x55dc65530090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:28.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.008+0000 7fc89b3b0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc655305d0 0x55dc65530a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:28.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.009+0000 7fc89b3b0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc6553b370 0x55dc6553d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:28.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.009+0000 7fc89b3b0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dc654962f0 con 0x55dc655305d0 2024-09-19T09:09:28.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.009+0000 7fc89b3b0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55dc6543dcf0 con 0x55dc65531b40 2024-09-19T09:09:28.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.009+0000 7fc89b3b0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dc654922b0 con 0x55dc6553b370 2024-09-19T09:09:28.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.009+0000 7fc891d9b640 1 --2- 172.21.15.120:0/809335970 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc87c0a3240 0x7fc87c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:28.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.009+0000 7fc8837fe640 1 -- 172.21.15.120:0/809335970 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fc88c0cfe00 con 0x7fc894102ef0 2024-09-19T09:09:28.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.009+0000 7fc89259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc65531b40 0x55dc65530090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:28.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.009+0000 7fc89259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc65531b40 0x55dc65530090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59796/0 (socket says 172.21.15.120:59796) 2024-09-19T09:09:28.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.009+0000 7fc89259c640 1 -- 172.21.15.120:0/299409119 learned_addr learned my addr 172.21.15.120:0/299409119 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:28.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.009+0000 7fc891d9b640 1 --2- 172.21.15.120:0/299409119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc655305d0 0x55dc65530a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:28.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.009+0000 7fc892d9d640 1 --2- 172.21.15.120:0/299409119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc6553b370 0x55dc6553d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:28.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.009+0000 7fc89259c640 1 -- 172.21.15.120:0/299409119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc6553b370 msgr2=0x55dc6553d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:28.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.009+0000 7fc89259c640 1 --2- 172.21.15.120:0/299409119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc6553b370 0x55dc6553d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.009+0000 7fc89259c640 1 -- 172.21.15.120:0/299409119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc655305d0 msgr2=0x55dc65530a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:28.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.009+0000 7fc89259c640 1 --2- 172.21.15.120:0/299409119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc655305d0 0x55dc65530a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.009+0000 7fc89259c640 1 -- 172.21.15.120:0/299409119 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc654223a0 con 0x55dc65531b40 2024-09-19T09:09:28.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.009+0000 7fc892d9d640 1 --2- 172.21.15.120:0/299409119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc6553b370 0x55dc6553d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:09:28.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.009+0000 7fc891d9b640 1 --2- 172.21.15.120:0/809335970 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc87c0a3240 0x7fc87c0a5700 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc888000c00 tx=0x7fc888009630 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:28.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.009+0000 7fc89259c640 1 --2- 172.21.15.120:0/299409119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc65531b40 0x55dc65530090 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc88400bd10 tx=0x7fc8840107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73735906e5e80fb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:28.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.009+0000 7fc880ff9640 1 -- 172.21.15.120:0/299409119 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884004030 con 0x55dc65531b40 2024-09-19T09:09:28.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.010+0000 7fc880ff9640 1 -- 172.21.15.120:0/299409119 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8840041d0 con 0x55dc65531b40 2024-09-19T09:09:28.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.010+0000 7fc880ff9640 1 -- 172.21.15.120:0/299409119 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884011800 con 0x55dc65531b40 2024-09-19T09:09:28.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.010+0000 7fc89b3b0740 1 -- 172.21.15.120:0/299409119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc65531b40 msgr2=0x55dc65530090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:28.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.010+0000 7fc89b3b0740 1 --2- 172.21.15.120:0/299409119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc65531b40 0x55dc65530090 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc88400bd10 tx=0x7fc8840107a0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.010+0000 7fc89b3b0740 1 -- 172.21.15.120:0/299409119 shutdown_connections 2024-09-19T09:09:28.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.010+0000 7fc89b3b0740 1 --2- 172.21.15.120:0/299409119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc6553b370 0x55dc6553d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.010+0000 7fc89b3b0740 1 --2- 172.21.15.120:0/299409119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc655305d0 0x55dc65530a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.010+0000 7fc89b3b0740 1 --2- 172.21.15.120:0/299409119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc65531b40 0x55dc65530090 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.010+0000 7fc89b3b0740 1 -- 172.21.15.120:0/299409119 >> 172.21.15.120:0/299409119 conn(0x55dc654f9310 msgr2=0x55dc6552cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:28.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.010+0000 7fc89b3b0740 1 -- 172.21.15.120:0/299409119 shutdown_connections 2024-09-19T09:09:28.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.010+0000 7fc89b3b0740 1 -- 172.21.15.120:0/299409119 wait complete. 2024-09-19T09:09:28.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.011+0000 7fc89b3b0740 1 -- start start 2024-09-19T09:09:28.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.011+0000 7fc89b3b0740 1 Processor -- start 2024-09-19T09:09:28.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.011+0000 7fc89b3b0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc655305d0 0x55dc6553ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:28.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.011+0000 7fc89b3b0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc65531b40 0x55dc65538000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:28.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.011+0000 7fc89b3b0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc6553b370 0x55dc655385f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:28.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.011+0000 7fc89b3b0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dc6553e070 con 0x55dc655305d0 2024-09-19T09:09:28.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.011+0000 7fc89b3b0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55dc65451060 con 0x55dc6553b370 2024-09-19T09:09:28.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.011+0000 7fc89b3b0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dc6553e1f0 con 0x55dc65531b40 2024-09-19T09:09:28.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.011+0000 7fc89b3b0740 10 client.0 _ll_register_callbacks cb 0x55dc651a48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:28.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.011+0000 7fc89259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc655305d0 0x55dc6553ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:28.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.011+0000 7fc89259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc655305d0 0x55dc6553ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36056/0 (socket says 172.21.15.120:36056) 2024-09-19T09:09:28.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.011+0000 7fc89259c640 1 -- 172.21.15.120:0/2163067630 learned_addr learned my addr 172.21.15.120:0/2163067630 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:28.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.011+0000 7fc892d9d640 1 --2- 172.21.15.120:0/2163067630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc6553b370 0x55dc655385f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:28.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.011+0000 7fc891d9b640 1 --2- 172.21.15.120:0/2163067630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc65531b40 0x55dc65538000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:28.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.012+0000 7fc89259c640 1 -- 172.21.15.120:0/2163067630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc65531b40 msgr2=0x55dc65538000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:28.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.012+0000 7fc89259c640 1 --2- 172.21.15.120:0/2163067630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc65531b40 0x55dc65538000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.012+0000 7fc89259c640 1 -- 172.21.15.120:0/2163067630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc6553b370 msgr2=0x55dc655385f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:28.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.012+0000 7fc89259c640 1 --2- 172.21.15.120:0/2163067630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc6553b370 0x55dc655385f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.012+0000 7fc89259c640 1 -- 172.21.15.120:0/2163067630 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc652073c0 con 0x55dc655305d0 2024-09-19T09:09:28.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.012+0000 7fc89259c640 1 --2- 172.21.15.120:0/2163067630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc655305d0 0x55dc6553ad00 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fc88400b620 tx=0x7fc8840058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a570109e2ae85e8f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:28.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.012+0000 7fc879ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:09:28.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.012+0000 7fc879ffb640 1 -- 172.21.15.120:0/2163067630 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884029060 con 0x55dc655305d0 2024-09-19T09:09:28.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.012+0000 7fc879ffb640 1 -- 172.21.15.120:0/2163067630 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fc884004380 con 0x55dc655305d0 2024-09-19T09:09:28.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.012+0000 7fc89b3b0740 1 -- 172.21.15.120:0/2163067630 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8840041d0 con 0x55dc655305d0 2024-09-19T09:09:28.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.012+0000 7fc879ffb640 1 -- 172.21.15.120:0/2163067630 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc8840035f0 con 0x55dc655305d0 2024-09-19T09:09:28.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.012+0000 7fc89b3b0740 10 client.17784 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:28.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.012+0000 7fc89b3b0740 1 -- 172.21.15.120:0/2163067630 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dc65549ae0 con 0x55dc655305d0 2024-09-19T09:09:28.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.012+0000 7fc879ffb640 1 -- 172.21.15.120:0/2163067630 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8840041d0 con 0x55dc655305d0 2024-09-19T09:09:28.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.012+0000 7fc879ffb640 1 -- 172.21.15.120:0/2163067630 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc88402e070 con 0x55dc655305d0 2024-09-19T09:09:28.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.013+0000 7fc89b3b0740 10 client.17784 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:28.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.013+0000 7fc89b3b0740 10 client.17784 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:09:28.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.013+0000 7fc89b3b0740 20 client.17784 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:28.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.013+0000 7fc89b3b0740 1 --2- 172.21.15.120:0/2163067630 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55dc6554aee0 0x55dc6554d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:28.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.013+0000 7fc89b3b0740 4 client.17784 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:09:28.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.013+0000 7fc89b3b0740 1 -- 172.21.15.120:0/2163067630 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc654a3860 con 0x55dc6554aee0 2024-09-19T09:09:28.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.013+0000 7fc891d9b640 1 --2- 172.21.15.120:0/2163067630 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55dc6554aee0 0x55dc6554d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:28.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.013+0000 7fc879ffb640 10 client.17784 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:09:28.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.013+0000 7fc891d9b640 1 --2- 172.21.15.120:0/2163067630 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55dc6554aee0 0x55dc6554d2d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7d2a5e085c5f5097 server_cookie=f7fe012e2a0af375 in_seq=0 out_seq=0 2024-09-19T09:09:28.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.014+0000 7fc879ffb640 1 -- 172.21.15.120:0/2163067630 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dc654a3860 con 0x55dc6554aee0 2024-09-19T09:09:28.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.014+0000 7fc879ffb640 10 client.17784 handle_command_reply: tid=0 2024-09-19T09:09:28.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.014+0000 7fc89b3b0740 1 client.17784 shutdown 2024-09-19T09:09:28.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.014+0000 7fc89b3b0740 1 -- 172.21.15.120:0/2163067630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc655305d0 msgr2=0x55dc6553ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:28.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.014+0000 7fc89b3b0740 1 --2- 172.21.15.120:0/2163067630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc655305d0 0x55dc6553ad00 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fc88400b620 tx=0x7fc8840058e0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.014+0000 7fc89b3b0740 1 -- 172.21.15.120:0/2163067630 shutdown_connections 2024-09-19T09:09:28.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.014+0000 7fc89b3b0740 1 --2- 172.21.15.120:0/2163067630 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55dc6554aee0 0x55dc6554d2d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.014+0000 7fc89b3b0740 1 --2- 172.21.15.120:0/2163067630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc6553b370 0x55dc655385f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.014+0000 7fc89b3b0740 1 --2- 172.21.15.120:0/2163067630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc65531b40 0x55dc65538000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.014+0000 7fc879ffb640 0 client.17784 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:09:28.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.014+0000 7fc89b3b0740 1 --2- 172.21.15.120:0/2163067630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc655305d0 0x55dc6553ad00 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.014+0000 7fc89b3b0740 1 -- 172.21.15.120:0/2163067630 >> 172.21.15.120:0/2163067630 conn(0x55dc654f9310 msgr2=0x55dc65532960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:28.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.014+0000 7fc89b3b0740 1 -- 172.21.15.120:0/2163067630 shutdown_connections 2024-09-19T09:09:28.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.014+0000 7fc89b3b0740 1 -- 172.21.15.120:0/2163067630 wait complete. 2024-09-19T09:09:28.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.014+0000 7fc89b3b0740 20 client.17784 trim_cache size 0 max 16384 2024-09-19T09:09:28.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.027+0000 7fc89b3b0740 1 Processor -- start 2024-09-19T09:09:28.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.027+0000 7fc89b3b0740 1 -- start start 2024-09-19T09:09:28.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.027+0000 7fc89b3b0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc655305d0 0x55dc65547310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:28.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.027+0000 7fc89b3b0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc65531b40 0x55dc65539860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:28.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.028+0000 7fc89b3b0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc6553b370 0x55dc65543860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:28.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.028+0000 7fc89b3b0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc8941ad570 con 0x55dc65531b40 2024-09-19T09:09:28.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.028+0000 7fc89b3b0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc8941af0b0 con 0x55dc6553b370 2024-09-19T09:09:28.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.028+0000 7fc89b3b0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc8941adc40 con 0x55dc655305d0 2024-09-19T09:09:28.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.028+0000 7fc891d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc65531b40 0x55dc65539860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:28.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.028+0000 7fc891d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc65531b40 0x55dc65539860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36070/0 (socket says 172.21.15.120:36070) 2024-09-19T09:09:28.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.028+0000 7fc891d9b640 1 -- 172.21.15.120:0/4235882507 learned_addr learned my addr 172.21.15.120:0/4235882507 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:28.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.028+0000 7fc892d9d640 1 --2- 172.21.15.120:0/4235882507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc6553b370 0x55dc65543860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:28.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.028+0000 7fc89259c640 1 --2- 172.21.15.120:0/4235882507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc655305d0 0x55dc65547310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:28.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.028+0000 7fc891d9b640 1 -- 172.21.15.120:0/4235882507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc655305d0 msgr2=0x55dc65547310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:28.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.028+0000 7fc891d9b640 1 --2- 172.21.15.120:0/4235882507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc655305d0 0x55dc65547310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.028+0000 7fc891d9b640 1 -- 172.21.15.120:0/4235882507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc6553b370 msgr2=0x55dc65543860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:28.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.028+0000 7fc891d9b640 1 --2- 172.21.15.120:0/4235882507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc6553b370 0x55dc65543860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.028+0000 7fc891d9b640 1 -- 172.21.15.120:0/4235882507 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc6553e1a0 con 0x55dc65531b40 2024-09-19T09:09:28.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.028+0000 7fc89259c640 1 --2- 172.21.15.120:0/4235882507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc655305d0 0x55dc65547310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:09:28.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.028+0000 7fc891d9b640 1 --2- 172.21.15.120:0/4235882507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc65531b40 0x55dc65539860 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fc888009850 tx=0x7fc88802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d8cf15f0a481b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:28.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.028+0000 7fc87bfff640 1 -- 172.21.15.120:0/4235882507 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc888003cc0 con 0x55dc65531b40 2024-09-19T09:09:28.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.028+0000 7fc87bfff640 1 -- 172.21.15.120:0/4235882507 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc88805a070 con 0x55dc65531b40 2024-09-19T09:09:28.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.028+0000 7fc87bfff640 1 -- 172.21.15.120:0/4235882507 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8880318e0 con 0x55dc65531b40 2024-09-19T09:09:28.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.029+0000 7fc89b3b0740 1 -- 172.21.15.120:0/4235882507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc65531b40 msgr2=0x55dc65539860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:28.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.029+0000 7fc89b3b0740 1 --2- 172.21.15.120:0/4235882507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc65531b40 0x55dc65539860 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fc888009850 tx=0x7fc88802a480 comp rx=0 tx=0).stop 2024-09-19T09:09:28.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.029+0000 7fc89b3b0740 1 -- 172.21.15.120:0/4235882507 shutdown_connections 2024-09-19T09:09:28.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.029+0000 7fc89b3b0740 1 --2- 172.21.15.120:0/4235882507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc6553b370 0x55dc65543860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.029+0000 7fc89b3b0740 1 --2- 172.21.15.120:0/4235882507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc65531b40 0x55dc65539860 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.029+0000 7fc89b3b0740 1 --2- 172.21.15.120:0/4235882507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc655305d0 0x55dc65547310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.029+0000 7fc89b3b0740 1 -- 172.21.15.120:0/4235882507 >> 172.21.15.120:0/4235882507 conn(0x55dc654f9310 msgr2=0x55dc6552bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:28.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.029+0000 7fc89b3b0740 1 -- 172.21.15.120:0/4235882507 shutdown_connections 2024-09-19T09:09:28.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.029+0000 7fc89b3b0740 1 -- 172.21.15.120:0/4235882507 wait complete. 2024-09-19T09:09:28.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.030+0000 7fc89b3b0740 1 -- start start 2024-09-19T09:09:28.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.030+0000 7fc89b3b0740 1 Processor -- start 2024-09-19T09:09:28.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.030+0000 7fc89b3b0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc655305d0 0x55dc655ad800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:28.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.030+0000 7fc89b3b0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc65531b40 0x55dc655add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:28.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.030+0000 7fc89b3b0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc6553b370 0x55dc655ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:28.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.030+0000 7fc89b3b0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc8941ae9e0 con 0x55dc6553b370 2024-09-19T09:09:28.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.030+0000 7fc89b3b0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc8941ae310 con 0x55dc655305d0 2024-09-19T09:09:28.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.030+0000 7fc89b3b0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc8941ab530 con 0x55dc65531b40 2024-09-19T09:09:28.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.030+0000 7fc89b3b0740 10 client.0 _ll_register_callbacks cb 0x7fc888009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:28.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.030+0000 7fc89259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc655305d0 0x55dc655ad800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:28.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.030+0000 7fc892d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc6553b370 0x55dc655ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:28.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.030+0000 7fc89259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc655305d0 0x55dc655ad800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59824/0 (socket says 172.21.15.120:59824) 2024-09-19T09:09:28.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.030+0000 7fc89259c640 1 -- 172.21.15.120:0/2513375237 learned_addr learned my addr 172.21.15.120:0/2513375237 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:28.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.030+0000 7fc892d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc6553b370 0x55dc655ae280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36086/0 (socket says 172.21.15.120:36086) 2024-09-19T09:09:28.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.030+0000 7fc891d9b640 1 --2- 172.21.15.120:0/2513375237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc65531b40 0x55dc655add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:28.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.031+0000 7fc892d9d640 1 -- 172.21.15.120:0/2513375237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc65531b40 msgr2=0x55dc655add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:28.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.031+0000 7fc892d9d640 1 --2- 172.21.15.120:0/2513375237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc65531b40 0x55dc655add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.031+0000 7fc892d9d640 1 -- 172.21.15.120:0/2513375237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc655305d0 msgr2=0x55dc655ad800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:28.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.031+0000 7fc892d9d640 1 --2- 172.21.15.120:0/2513375237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc655305d0 0x55dc655ad800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.031+0000 7fc892d9d640 1 -- 172.21.15.120:0/2513375237 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc655b8b20 con 0x55dc6553b370 2024-09-19T09:09:28.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.031+0000 7fc89259c640 1 --2- 172.21.15.120:0/2513375237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc655305d0 0x55dc655ad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:28.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.031+0000 7fc892d9d640 1 --2- 172.21.15.120:0/2513375237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc6553b370 0x55dc655ae280 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0cd900 tx=0x7fc88c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a8aa76d3fc909a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:28.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.031+0000 7fc879ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:09:28.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.031+0000 7fc879ffb640 1 -- 172.21.15.120:0/2513375237 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c09cb20 con 0x55dc6553b370 2024-09-19T09:09:28.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.031+0000 7fc879ffb640 1 -- 172.21.15.120:0/2513375237 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fc88c0b5190 con 0x55dc6553b370 2024-09-19T09:09:28.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.031+0000 7fc89b3b0740 1 -- 172.21.15.120:0/2513375237 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc88805a070 con 0x55dc6553b370 2024-09-19T09:09:28.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.031+0000 7fc879ffb640 1 -- 172.21.15.120:0/2513375237 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc88c0120f0 con 0x55dc6553b370 2024-09-19T09:09:28.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.031+0000 7fc89b3b0740 10 client.17790 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:28.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.031+0000 7fc89b3b0740 1 -- 172.21.15.120:0/2513375237 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dc655b9e40 con 0x55dc6553b370 2024-09-19T09:09:28.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.031+0000 7fc879ffb640 1 -- 172.21.15.120:0/2513375237 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc88805a070 con 0x55dc6553b370 2024-09-19T09:09:28.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.032+0000 7fc879ffb640 1 -- 172.21.15.120:0/2513375237 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc88c0a4c00 con 0x55dc6553b370 2024-09-19T09:09:28.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.032+0000 7fc89b3b0740 10 client.17790 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:28.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.032+0000 7fc89b3b0740 10 client.17790 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:09:28.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.032+0000 7fc89b3b0740 20 client.17790 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:28.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.032+0000 7fc89b3b0740 1 --2- 172.21.15.120:0/2513375237 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55dc655ba5b0 0x55dc655bc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:28.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.032+0000 7fc89b3b0740 4 client.17790 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:09:28.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.032+0000 7fc89b3b0740 1 -- 172.21.15.120:0/2513375237 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc6549dd70 con 0x55dc655ba5b0 2024-09-19T09:09:28.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.032+0000 7fc89259c640 1 --2- 172.21.15.120:0/2513375237 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55dc655ba5b0 0x55dc655bc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:28.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.032+0000 7fc879ffb640 10 client.17790 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:09:28.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.032+0000 7fc89259c640 1 --2- 172.21.15.120:0/2513375237 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55dc655ba5b0 0x55dc655bc9a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d2ff854abf15519a server_cookie=b6ac51fe7bc7b7a2 in_seq=0 out_seq=0 2024-09-19T09:09:28.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.033+0000 7fc879ffb640 1 -- 172.21.15.120:0/2513375237 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc6549dd70 con 0x55dc655ba5b0 2024-09-19T09:09:28.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.033+0000 7fc879ffb640 10 client.17790 handle_command_reply: tid=0 2024-09-19T09:09:28.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.033+0000 7fc89b3b0740 1 client.17790 shutdown 2024-09-19T09:09:28.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.033+0000 7fc89b3b0740 1 -- 172.21.15.120:0/2513375237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc6553b370 msgr2=0x55dc655ae280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:28.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.033+0000 7fc89b3b0740 1 --2- 172.21.15.120:0/2513375237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc6553b370 0x55dc655ae280 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0cd900 tx=0x7fc88c00cf40 comp rx=0 tx=0).stop 2024-09-19T09:09:28.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.033+0000 7fc89b3b0740 1 -- 172.21.15.120:0/2513375237 shutdown_connections 2024-09-19T09:09:28.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.033+0000 7fc89b3b0740 1 --2- 172.21.15.120:0/2513375237 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55dc655ba5b0 0x55dc655bc9a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.033+0000 7fc89b3b0740 1 --2- 172.21.15.120:0/2513375237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc6553b370 0x55dc655ae280 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.033+0000 7fc89b3b0740 1 --2- 172.21.15.120:0/2513375237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc65531b40 0x55dc655add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.033+0000 7fc89b3b0740 1 --2- 172.21.15.120:0/2513375237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc655305d0 0x55dc655ad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.033+0000 7fc89b3b0740 1 -- 172.21.15.120:0/2513375237 >> 172.21.15.120:0/2513375237 conn(0x55dc654f9310 msgr2=0x55dc65546300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:28.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.033+0000 7fc879ffb640 0 client.17790 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:09:28.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.033+0000 7fc89b3b0740 1 -- 172.21.15.120:0/2513375237 shutdown_connections 2024-09-19T09:09:28.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.033+0000 7fc89b3b0740 1 -- 172.21.15.120:0/2513375237 wait complete. 2024-09-19T09:09:28.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.033+0000 7fc89b3b0740 20 client.17790 trim_cache size 0 max 16384 2024-09-19T09:09:28.040 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:09:28.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.034+0000 7fc898fd7640 1 -- 172.21.15.120:0/809335970 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc87c0a3240 msgr2=0x7fc87c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:28.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.034+0000 7fc898fd7640 1 --2- 172.21.15.120:0/809335970 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc87c0a3240 0x7fc87c0a5700 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc888000c00 tx=0x7fc888009630 comp rx=0 tx=0).stop 2024-09-19T09:09:28.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.034+0000 7fc898fd7640 1 -- 172.21.15.120:0/809335970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc894102ef0 msgr2=0x7fc89419bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:28.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.034+0000 7fc898fd7640 1 --2- 172.21.15.120:0/809335970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc894102ef0 0x7fc89419bd00 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00eb10 tx=0x7fc88c00c2d0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.034+0000 7fc898fd7640 1 -- 172.21.15.120:0/809335970 shutdown_connections 2024-09-19T09:09:28.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.034+0000 7fc898fd7640 1 --2- 172.21.15.120:0/809335970 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc87c0a3240 0x7fc87c0a5700 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.034+0000 7fc898fd7640 1 --2- 172.21.15.120:0/809335970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc894102ef0 0x7fc89419bd00 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.034+0000 7fc898fd7640 1 --2- 172.21.15.120:0/809335970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc894102570 0x7fc894197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.034+0000 7fc898fd7640 1 --2- 172.21.15.120:0/809335970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc894100e20 0x7fc894197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:28.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.034+0000 7fc898fd7640 1 -- 172.21.15.120:0/809335970 >> 172.21.15.120:0/809335970 conn(0x7fc8940fc7d0 msgr2=0x7fc8940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:28.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.034+0000 7fc898fd7640 1 -- 172.21.15.120:0/809335970 shutdown_connections 2024-09-19T09:09:28.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:28.034+0000 7fc898fd7640 1 -- 172.21.15.120:0/809335970 wait complete. 2024-09-19T09:09:28.053 INFO:tasks.workunit.client.0.smithi120.stdout: 1 591055 9.74 MB/sec execute 203 sec latency 20.388 ms 2024-09-19T09:09:29.053 INFO:tasks.workunit.client.0.smithi120.stdout: 1 593897 9.76 MB/sec execute 204 sec latency 14.061 ms 2024-09-19T09:09:29.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:28 smithi120 ceph-mon[31150]: pgmap v540: 97 pgs: 97 active+clean; 42 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.3 MiB/s wr, 453 op/s 2024-09-19T09:09:29.167 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:09:29.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:28 smithi196 ceph-mon[35150]: pgmap v540: 97 pgs: 97 active+clean; 42 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.3 MiB/s wr, 453 op/s 2024-09-19T09:09:29.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:28 smithi153 ceph-mon[35071]: pgmap v540: 97 pgs: 97 active+clean; 42 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.3 MiB/s wr, 453 op/s 2024-09-19T09:09:29.421 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:09:29.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.963+0000 7fd3e0633640 1 -- 172.21.15.120:0/3962584708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd3d8104990 msgr2=0x7fd3d81089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:29.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.963+0000 7fd3e0633640 1 --2- 172.21.15.120:0/3962584708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd3d8104990 0x7fd3d81089e0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8009900 tx=0x7fd3c802a410 comp rx=0 tx=0).stop 2024-09-19T09:09:29.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.963+0000 7fd3e0633640 1 -- 172.21.15.120:0/3962584708 shutdown_connections 2024-09-19T09:09:29.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.963+0000 7fd3e0633640 1 --2- 172.21.15.120:0/3962584708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3d8109110 0x7fd3d8110c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:29.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.963+0000 7fd3e0633640 1 --2- 172.21.15.120:0/3962584708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd3d8104990 0x7fd3d81089e0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:29.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.963+0000 7fd3e0633640 1 --2- 172.21.15.120:0/3962584708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3d8103fe0 0x7fd3d81043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:29.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.963+0000 7fd3e0633640 1 -- 172.21.15.120:0/3962584708 >> 172.21.15.120:0/3962584708 conn(0x7fd3d80ffa40 msgr2=0x7fd3d8101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:29.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.963+0000 7fd3e0633640 1 -- 172.21.15.120:0/3962584708 shutdown_connections 2024-09-19T09:09:29.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.963+0000 7fd3e0633640 1 -- 172.21.15.120:0/3962584708 wait complete. 2024-09-19T09:09:29.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.964+0000 7fd3e0633640 1 Processor -- start 2024-09-19T09:09:29.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.964+0000 7fd3e0633640 1 -- start start 2024-09-19T09:09:29.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.965+0000 7fd3e0633640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd3d8103fe0 0x7fd3d819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:29.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.965+0000 7fd3e0633640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3d8104990 0x7fd3d819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:29.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.965+0000 7fd3e0633640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3d8109110 0x7fd3d819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:29.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.965+0000 7fd3ddbce640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3d8104990 0x7fd3d819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:29.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.965+0000 7fd3e0633640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d8114110 con 0x7fd3d8103fe0 2024-09-19T09:09:29.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.965+0000 7fd3e0633640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d8113f90 con 0x7fd3d8109110 2024-09-19T09:09:29.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.965+0000 7fd3e0633640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d8114290 con 0x7fd3d8104990 2024-09-19T09:09:29.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.965+0000 7fd3ddbce640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3d8104990 0x7fd3d819ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:45824/0 (socket says 172.21.15.120:45824) 2024-09-19T09:09:29.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.965+0000 7fd3ddbce640 1 -- 172.21.15.120:0/4227555815 learned_addr learned my addr 172.21.15.120:0/4227555815 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:29.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.965+0000 7fd3de3cf640 1 --2- 172.21.15.120:0/4227555815 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd3d8103fe0 0x7fd3d819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:29.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.965+0000 7fd3debd0640 1 --2- 172.21.15.120:0/4227555815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3d8109110 0x7fd3d819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:29.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.965+0000 7fd3de3cf640 1 -- 172.21.15.120:0/4227555815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3d8104990 msgr2=0x7fd3d819ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:29.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.965+0000 7fd3de3cf640 1 --2- 172.21.15.120:0/4227555815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3d8104990 0x7fd3d819ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:29.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.965+0000 7fd3de3cf640 1 -- 172.21.15.120:0/4227555815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3d8109110 msgr2=0x7fd3d819f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:29.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.965+0000 7fd3de3cf640 1 --2- 172.21.15.120:0/4227555815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3d8109110 0x7fd3d819f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:29.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.965+0000 7fd3de3cf640 1 -- 172.21.15.120:0/4227555815 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d819f700 con 0x7fd3d8103fe0 2024-09-19T09:09:29.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.966+0000 7fd3ddbce640 1 --2- 172.21.15.120:0/4227555815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3d8104990 0x7fd3d819ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:29.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.966+0000 7fd3de3cf640 1 --2- 172.21.15.120:0/4227555815 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd3d8103fe0 0x7fd3d819a750 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fd3d400e9f0 tx=0x7fd3d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:29.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.966+0000 7fd3cf7fe640 1 -- 172.21.15.120:0/4227555815 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d400ccc0 con 0x7fd3d8103fe0 2024-09-19T09:09:29.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.966+0000 7fd3cf7fe640 1 -- 172.21.15.120:0/4227555815 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3d4010910 con 0x7fd3d8103fe0 2024-09-19T09:09:29.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.966+0000 7fd3e0633640 1 -- 172.21.15.120:0/4227555815 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d819f9f0 con 0x7fd3d8103fe0 2024-09-19T09:09:29.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.967+0000 7fd3cf7fe640 1 -- 172.21.15.120:0/4227555815 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4018960 con 0x7fd3d8103fe0 2024-09-19T09:09:29.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.967+0000 7fd3e0633640 1 -- 172.21.15.120:0/4227555815 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d81a72d0 con 0x7fd3d8103fe0 2024-09-19T09:09:29.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.970+0000 7fd3cf7fe640 1 -- 172.21.15.120:0/4227555815 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd3d4010430 con 0x7fd3d8103fe0 2024-09-19T09:09:29.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.971+0000 7fd3cf7fe640 1 --2- 172.21.15.120:0/4227555815 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd3c40a3410 0x7fd3c40a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:29.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.971+0000 7fd3cf7fe640 1 -- 172.21.15.120:0/4227555815 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fd3d40cfc30 con 0x7fd3d8103fe0 2024-09-19T09:09:29.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.972+0000 7fd3ddbce640 1 --2- 172.21.15.120:0/4227555815 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd3c40a3410 0x7fd3c40a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:29.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.972+0000 7fd3e2a0c740 1 Processor -- start 2024-09-19T09:09:29.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.972+0000 7fd3e2a0c740 1 -- start start 2024-09-19T09:09:29.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.973+0000 7fd3e2a0c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3fa4bbb40 0x55b3fa4ba090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:29.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.973+0000 7fd3e2a0c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3fa4ba5d0 0x55b3fa4baa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:29.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.973+0000 7fd3e2a0c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3fa4c5370 0x55b3fa4c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:29.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.973+0000 7fd3e2a0c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b3fa4202f0 con 0x55b3fa4ba5d0 2024-09-19T09:09:29.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.973+0000 7fd3e2a0c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b3fa3c7cf0 con 0x55b3fa4c5370 2024-09-19T09:09:29.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.973+0000 7fd3e2a0c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b3fa41c2b0 con 0x55b3fa4bbb40 2024-09-19T09:09:29.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.973+0000 7fd3debd0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3fa4c5370 0x55b3fa4c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:29.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.973+0000 7fd3debd0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3fa4c5370 0x55b3fa4c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59858/0 (socket says 172.21.15.120:59858) 2024-09-19T09:09:29.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.973+0000 7fd3debd0640 1 -- 172.21.15.120:0/1853297872 learned_addr learned my addr 172.21.15.120:0/1853297872 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:29.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.973+0000 7fd3debd0640 1 -- 172.21.15.120:0/1853297872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3fa4bbb40 msgr2=0x55b3fa4ba090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:09:29.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.973+0000 7fd3debd0640 1 --2- 172.21.15.120:0/1853297872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3fa4bbb40 0x55b3fa4ba090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:29.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.973+0000 7fd3debd0640 1 -- 172.21.15.120:0/1853297872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3fa4ba5d0 msgr2=0x55b3fa4baa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:29.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.973+0000 7fd3debd0640 1 --2- 172.21.15.120:0/1853297872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3fa4ba5d0 0x55b3fa4baa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:29.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.973+0000 7fd3debd0640 1 -- 172.21.15.120:0/1853297872 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3fa3ac3a0 con 0x55b3fa4c5370 2024-09-19T09:09:29.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.974+0000 7fd3ddbce640 1 --2- 172.21.15.120:0/4227555815 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd3c40a3410 0x7fd3c40a58d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8031550 tx=0x7fd3c8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:29.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.974+0000 7fd3debd0640 1 --2- 172.21.15.120:0/1853297872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3fa4c5370 0x55b3fa4c7760 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0009f50 tx=0x7fd3d00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f61ab8eed05f90aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:29.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.974+0000 7fd3ccff9640 1 -- 172.21.15.120:0/1853297872 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0004b20 con 0x55b3fa4c5370 2024-09-19T09:09:29.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.974+0000 7fd3ccff9640 1 -- 172.21.15.120:0/1853297872 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3d0004cc0 con 0x55b3fa4c5370 2024-09-19T09:09:29.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.974+0000 7fd3ccff9640 1 -- 172.21.15.120:0/1853297872 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0005020 con 0x55b3fa4c5370 2024-09-19T09:09:29.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.975+0000 7fd3e2a0c740 1 -- 172.21.15.120:0/1853297872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3fa4c5370 msgr2=0x55b3fa4c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:29.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.975+0000 7fd3e2a0c740 1 --2- 172.21.15.120:0/1853297872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3fa4c5370 0x55b3fa4c7760 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0009f50 tx=0x7fd3d00118d0 comp rx=0 tx=0).stop 2024-09-19T09:09:29.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.975+0000 7fd3e2a0c740 1 -- 172.21.15.120:0/1853297872 shutdown_connections 2024-09-19T09:09:29.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.975+0000 7fd3e2a0c740 1 --2- 172.21.15.120:0/1853297872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3fa4c5370 0x55b3fa4c7760 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:29.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.975+0000 7fd3e2a0c740 1 --2- 172.21.15.120:0/1853297872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3fa4ba5d0 0x55b3fa4baa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:29.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.975+0000 7fd3e2a0c740 1 --2- 172.21.15.120:0/1853297872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3fa4bbb40 0x55b3fa4ba090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:29.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.975+0000 7fd3e2a0c740 1 -- 172.21.15.120:0/1853297872 >> 172.21.15.120:0/1853297872 conn(0x55b3fa483310 msgr2=0x55b3fa4b6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:29.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.975+0000 7fd3e2a0c740 1 -- 172.21.15.120:0/1853297872 shutdown_connections 2024-09-19T09:09:29.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.976+0000 7fd3e2a0c740 1 -- 172.21.15.120:0/1853297872 wait complete. 2024-09-19T09:09:29.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.976+0000 7fd3e2a0c740 1 -- start start 2024-09-19T09:09:29.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.976+0000 7fd3e2a0c740 1 Processor -- start 2024-09-19T09:09:29.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.977+0000 7fd3e2a0c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3fa4ba5d0 0x55b3fa4c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:29.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.977+0000 7fd3e2a0c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3fa4bbb40 0x55b3fa4c2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:29.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.977+0000 7fd3e2a0c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3fa4c5370 0x55b3fa4c27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:29.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.977+0000 7fd3e2a0c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b3fa4d0880 con 0x55b3fa4ba5d0 2024-09-19T09:09:29.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.977+0000 7fd3e2a0c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b3fa3db060 con 0x55b3fa4c5370 2024-09-19T09:09:29.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.977+0000 7fd3e2a0c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b3fa4d0a00 con 0x55b3fa4bbb40 2024-09-19T09:09:29.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.977+0000 7fd3e2a0c740 10 client.0 _ll_register_callbacks cb 0x55b3fa12e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:29.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.977+0000 7fd3debd0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3fa4c5370 0x55b3fa4c27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:29.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.977+0000 7fd3ddbce640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3fa4bbb40 0x55b3fa4c2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:29.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.977+0000 7fd3debd0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3fa4c5370 0x55b3fa4c27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59864/0 (socket says 172.21.15.120:59864) 2024-09-19T09:09:29.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.977+0000 7fd3ddbce640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3fa4bbb40 0x55b3fa4c2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:45828/0 (socket says 172.21.15.120:45828) 2024-09-19T09:09:29.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.977+0000 7fd3debd0640 1 -- 172.21.15.120:0/2201760441 learned_addr learned my addr 172.21.15.120:0/2201760441 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:29.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.977+0000 7fd3de3cf640 1 --2- 172.21.15.120:0/2201760441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3fa4ba5d0 0x55b3fa4c4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:29.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.977+0000 7fd3debd0640 1 -- 172.21.15.120:0/2201760441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3fa4bbb40 msgr2=0x55b3fa4c2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:29.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.977+0000 7fd3debd0640 1 --2- 172.21.15.120:0/2201760441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3fa4bbb40 0x55b3fa4c2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:29.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.977+0000 7fd3debd0640 1 -- 172.21.15.120:0/2201760441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3fa4ba5d0 msgr2=0x55b3fa4c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:29.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.977+0000 7fd3debd0640 1 --2- 172.21.15.120:0/2201760441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3fa4ba5d0 0x55b3fa4c4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:29.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.977+0000 7fd3debd0640 1 -- 172.21.15.120:0/2201760441 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3fa1913c0 con 0x55b3fa4c5370 2024-09-19T09:09:29.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.977+0000 7fd3de3cf640 1 --2- 172.21.15.120:0/2201760441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3fa4ba5d0 0x55b3fa4c4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:09:29.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.977+0000 7fd3debd0640 1 --2- 172.21.15.120:0/2201760441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3fa4c5370 0x55b3fa4c27e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0005f40 tx=0x7fd3d0009be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cffeba59f90151db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:29.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.977+0000 7fd3c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:09:29.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.978+0000 7fd3c1ffb640 1 -- 172.21.15.120:0/2201760441 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0029020 con 0x55b3fa4c5370 2024-09-19T09:09:29.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.978+0000 7fd3e2a0c740 1 -- 172.21.15.120:0/2201760441 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3d0004cc0 con 0x55b3fa4c5370 2024-09-19T09:09:29.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.978+0000 7fd3c1ffb640 1 -- 172.21.15.120:0/2201760441 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fd3d0003d80 con 0x55b3fa4c5370 2024-09-19T09:09:29.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.978+0000 7fd3c1ffb640 1 -- 172.21.15.120:0/2201760441 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd3d0003f80 con 0x55b3fa4c5370 2024-09-19T09:09:29.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.978+0000 7fd3e2a0c740 10 client.27050 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:29.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.978+0000 7fd3e2a0c740 1 -- 172.21.15.120:0/2201760441 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b3fa4dc360 con 0x55b3fa4c5370 2024-09-19T09:09:29.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.978+0000 7fd3c1ffb640 1 -- 172.21.15.120:0/2201760441 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3d0004cc0 con 0x55b3fa4c5370 2024-09-19T09:09:29.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.978+0000 7fd3c1ffb640 1 -- 172.21.15.120:0/2201760441 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd3d0025470 con 0x55b3fa4c5370 2024-09-19T09:09:29.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.978+0000 7fd3e2a0c740 10 client.27050 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:29.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.978+0000 7fd3e2a0c740 10 client.27050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:09:29.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.978+0000 7fd3e2a0c740 20 client.27050 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:29.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.978+0000 7fd3e2a0c740 1 --2- 172.21.15.120:0/2201760441 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b3fa4ddc00 0x55b3fa4e0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:29.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.978+0000 7fd3e2a0c740 4 client.27050 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:09:29.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.978+0000 7fd3e2a0c740 1 -- 172.21.15.120:0/2201760441 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3fa42d860 con 0x55b3fa4ddc00 2024-09-19T09:09:29.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.979+0000 7fd3de3cf640 1 --2- 172.21.15.120:0/2201760441 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b3fa4ddc00 0x55b3fa4e0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:29.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.979+0000 7fd3c1ffb640 10 client.27050 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:29.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.979+0000 7fd3de3cf640 1 --2- 172.21.15.120:0/2201760441 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b3fa4ddc00 0x55b3fa4e0010 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a39552d4b28f63e9 server_cookie=f9c911da9845cb03 in_seq=0 out_seq=0 2024-09-19T09:09:29.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.980+0000 7fd3c1ffb640 1 -- 172.21.15.120:0/2201760441 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b3fa42d860 con 0x55b3fa4ddc00 2024-09-19T09:09:29.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.980+0000 7fd3c1ffb640 10 client.27050 handle_command_reply: tid=0 2024-09-19T09:09:29.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.980+0000 7fd3e2a0c740 1 client.27050 shutdown 2024-09-19T09:09:29.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.980+0000 7fd3e2a0c740 1 -- 172.21.15.120:0/2201760441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3fa4c5370 msgr2=0x55b3fa4c27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:29.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.980+0000 7fd3e2a0c740 1 --2- 172.21.15.120:0/2201760441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3fa4c5370 0x55b3fa4c27e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0005f40 tx=0x7fd3d0009be0 comp rx=0 tx=0).stop 2024-09-19T09:09:29.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.980+0000 7fd3e2a0c740 1 -- 172.21.15.120:0/2201760441 shutdown_connections 2024-09-19T09:09:29.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.980+0000 7fd3e2a0c740 1 --2- 172.21.15.120:0/2201760441 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b3fa4ddc00 0x55b3fa4e0010 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:29.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.980+0000 7fd3e2a0c740 1 --2- 172.21.15.120:0/2201760441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3fa4c5370 0x55b3fa4c27e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:29.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.980+0000 7fd3e2a0c740 1 --2- 172.21.15.120:0/2201760441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3fa4bbb40 0x55b3fa4c2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:29.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.980+0000 7fd3e2a0c740 1 --2- 172.21.15.120:0/2201760441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3fa4ba5d0 0x55b3fa4c4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:29.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.980+0000 7fd3e2a0c740 1 -- 172.21.15.120:0/2201760441 >> 172.21.15.120:0/2201760441 conn(0x55b3fa483310 msgr2=0x55b3fa4bc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:29.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.980+0000 7fd3c1ffb640 0 client.27050 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:29.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.980+0000 7fd3e2a0c740 1 -- 172.21.15.120:0/2201760441 shutdown_connections 2024-09-19T09:09:29.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.981+0000 7fd3e2a0c740 1 -- 172.21.15.120:0/2201760441 wait complete. 2024-09-19T09:09:29.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.981+0000 7fd3e2a0c740 20 client.27050 trim_cache size 0 max 16384 2024-09-19T09:09:29.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.995+0000 7fd3e2a0c740 1 Processor -- start 2024-09-19T09:09:29.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.995+0000 7fd3e2a0c740 1 -- start start 2024-09-19T09:09:29.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.995+0000 7fd3e2a0c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3fa4ba5d0 0x55b3fa4b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:29.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.995+0000 7fd3e2a0c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3fa4bbb40 0x55b3fa4b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:29.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.995+0000 7fd3e2a0c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3fa4c5370 0x55b3fa4c1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:29.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.995+0000 7fd3e2a0c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d81ab3f0 con 0x55b3fa4c5370 2024-09-19T09:09:29.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.995+0000 7fd3e2a0c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d81acf30 con 0x55b3fa4ba5d0 2024-09-19T09:09:29.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.995+0000 7fd3e2a0c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d81abac0 con 0x55b3fa4bbb40 2024-09-19T09:09:29.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.996+0000 7fd3ddbce640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3fa4bbb40 0x55b3fa4b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:29.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.996+0000 7fd3ddbce640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3fa4bbb40 0x55b3fa4b1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:45832/0 (socket says 172.21.15.120:45832) 2024-09-19T09:09:29.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.996+0000 7fd3ddbce640 1 -- 172.21.15.120:0/2472328101 learned_addr learned my addr 172.21.15.120:0/2472328101 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:29.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.996+0000 7fd3debd0640 1 --2- 172.21.15.120:0/2472328101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3fa4c5370 0x55b3fa4c1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:29.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.996+0000 7fd3de3cf640 1 --2- 172.21.15.120:0/2472328101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3fa4ba5d0 0x55b3fa4b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:29.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.996+0000 7fd3ddbce640 1 -- 172.21.15.120:0/2472328101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3fa4ba5d0 msgr2=0x55b3fa4b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:29.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.996+0000 7fd3ddbce640 1 --2- 172.21.15.120:0/2472328101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3fa4ba5d0 0x55b3fa4b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:29.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.996+0000 7fd3ddbce640 1 -- 172.21.15.120:0/2472328101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3fa4c5370 msgr2=0x55b3fa4c1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:29.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.996+0000 7fd3ddbce640 1 --2- 172.21.15.120:0/2472328101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3fa4c5370 0x55b3fa4c1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:29.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.996+0000 7fd3ddbce640 1 -- 172.21.15.120:0/2472328101 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3fa4c1020 con 0x55b3fa4bbb40 2024-09-19T09:09:29.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.996+0000 7fd3debd0640 1 --2- 172.21.15.120:0/2472328101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3fa4c5370 0x55b3fa4c1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:29.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.997+0000 7fd3ddbce640 1 --2- 172.21.15.120:0/2472328101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3fa4bbb40 0x55b3fa4b1ad0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8000c00 tx=0x7fd3c802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=985bf3cb1857c181 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:29.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.997+0000 7fd3c3fff640 1 -- 172.21.15.120:0/2472328101 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8035040 con 0x55b3fa4bbb40 2024-09-19T09:09:29.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.997+0000 7fd3c3fff640 1 -- 172.21.15.120:0/2472328101 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3c80043c0 con 0x55b3fa4bbb40 2024-09-19T09:09:29.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.997+0000 7fd3c3fff640 1 -- 172.21.15.120:0/2472328101 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c80035f0 con 0x55b3fa4bbb40 2024-09-19T09:09:30.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.998+0000 7fd3e2a0c740 1 -- 172.21.15.120:0/2472328101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3fa4bbb40 msgr2=0x55b3fa4b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:30.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.998+0000 7fd3e2a0c740 1 --2- 172.21.15.120:0/2472328101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3fa4bbb40 0x55b3fa4b1ad0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8000c00 tx=0x7fd3c802a410 comp rx=0 tx=0).stop 2024-09-19T09:09:30.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.998+0000 7fd3e2a0c740 1 -- 172.21.15.120:0/2472328101 shutdown_connections 2024-09-19T09:09:30.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.998+0000 7fd3e2a0c740 1 --2- 172.21.15.120:0/2472328101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3fa4c5370 0x55b3fa4c1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:30.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.998+0000 7fd3e2a0c740 1 --2- 172.21.15.120:0/2472328101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3fa4bbb40 0x55b3fa4b1ad0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:30.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.998+0000 7fd3e2a0c740 1 --2- 172.21.15.120:0/2472328101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3fa4ba5d0 0x55b3fa4b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:30.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.998+0000 7fd3e2a0c740 1 -- 172.21.15.120:0/2472328101 >> 172.21.15.120:0/2472328101 conn(0x55b3fa483310 msgr2=0x55b3fa4b6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:30.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.999+0000 7fd3e2a0c740 1 -- 172.21.15.120:0/2472328101 shutdown_connections 2024-09-19T09:09:30.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.999+0000 7fd3e2a0c740 1 -- 172.21.15.120:0/2472328101 wait complete. 2024-09-19T09:09:30.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.999+0000 7fd3e2a0c740 1 -- start start 2024-09-19T09:09:30.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.999+0000 7fd3e2a0c740 1 Processor -- start 2024-09-19T09:09:30.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:29.999+0000 7fd3e2a0c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3fa4ba5d0 0x55b3fa4d5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:30.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.000+0000 7fd3e2a0c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3fa4bbb40 0x55b3fa4d61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:30.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.000+0000 7fd3e2a0c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3fa4c5370 0x55b3fa4be380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:30.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.000+0000 7fd3e2a0c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d81ac860 con 0x55b3fa4bbb40 2024-09-19T09:09:30.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.000+0000 7fd3e2a0c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d81ac190 con 0x55b3fa4ba5d0 2024-09-19T09:09:30.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.000+0000 7fd3e2a0c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d81a93b0 con 0x55b3fa4c5370 2024-09-19T09:09:30.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.000+0000 7fd3ddbce640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3fa4bbb40 0x55b3fa4d61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:30.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.000+0000 7fd3ddbce640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3fa4bbb40 0x55b3fa4d61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36144/0 (socket says 172.21.15.120:36144) 2024-09-19T09:09:30.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.000+0000 7fd3ddbce640 1 -- 172.21.15.120:0/4281781367 learned_addr learned my addr 172.21.15.120:0/4281781367 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:30.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.000+0000 7fd3de3cf640 1 --2- 172.21.15.120:0/4281781367 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3fa4ba5d0 0x55b3fa4d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:30.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.000+0000 7fd3e2a0c740 10 client.0 _ll_register_callbacks cb 0x7fd3d0009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:30.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.000+0000 7fd3debd0640 1 --2- 172.21.15.120:0/4281781367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3fa4c5370 0x55b3fa4be380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:30.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.000+0000 7fd3ddbce640 1 -- 172.21.15.120:0/4281781367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3fa4c5370 msgr2=0x55b3fa4be380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:30.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.000+0000 7fd3ddbce640 1 --2- 172.21.15.120:0/4281781367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3fa4c5370 0x55b3fa4be380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:30.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.000+0000 7fd3ddbce640 1 -- 172.21.15.120:0/4281781367 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3fa4ba5d0 msgr2=0x55b3fa4d5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:30.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.000+0000 7fd3ddbce640 1 --2- 172.21.15.120:0/4281781367 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3fa4ba5d0 0x55b3fa4d5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:30.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.000+0000 7fd3ddbce640 1 -- 172.21.15.120:0/4281781367 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3fa4d6890 con 0x55b3fa4bbb40 2024-09-19T09:09:30.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.000+0000 7fd3de3cf640 1 --2- 172.21.15.120:0/4281781367 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3fa4ba5d0 0x55b3fa4d5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:30.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.001+0000 7fd3ddbce640 1 --2- 172.21.15.120:0/4281781367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3fa4bbb40 0x55b3fa4d61d0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8005ab0 tx=0x7fd3c802ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a7985d516cfb10 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:30.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.001+0000 7fd3c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:09:30.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.001+0000 7fd3c1ffb640 1 -- 172.21.15.120:0/4281781367 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8035070 con 0x55b3fa4bbb40 2024-09-19T09:09:30.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.001+0000 7fd3c1ffb640 1 -- 172.21.15.120:0/4281781367 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fd3c8004b20 con 0x55b3fa4bbb40 2024-09-19T09:09:30.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.001+0000 7fd3e2a0c740 1 -- 172.21.15.120:0/4281781367 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3c80043c0 con 0x55b3fa4bbb40 2024-09-19T09:09:30.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.001+0000 7fd3c1ffb640 1 -- 172.21.15.120:0/4281781367 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd3c8004d20 con 0x55b3fa4bbb40 2024-09-19T09:09:30.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.002+0000 7fd3e2a0c740 10 client.17805 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:30.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.002+0000 7fd3e2a0c740 1 -- 172.21.15.120:0/4281781367 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b3fa558e80 con 0x55b3fa4bbb40 2024-09-19T09:09:30.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.002+0000 7fd3c1ffb640 1 -- 172.21.15.120:0/4281781367 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3c8005520 con 0x55b3fa4bbb40 2024-09-19T09:09:30.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.002+0000 7fd3c1ffb640 1 -- 172.21.15.120:0/4281781367 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd3c8003cf0 con 0x55b3fa4bbb40 2024-09-19T09:09:30.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.002+0000 7fd3e2a0c740 10 client.17805 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:30.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.002+0000 7fd3e2a0c740 10 client.17805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:09:30.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.002+0000 7fd3e2a0c740 20 client.17805 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:30.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.002+0000 7fd3e2a0c740 1 --2- 172.21.15.120:0/4281781367 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b3fa559530 0x55b3fa55b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:30.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.002+0000 7fd3e2a0c740 4 client.17805 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:09:30.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.002+0000 7fd3e2a0c740 1 -- 172.21.15.120:0/4281781367 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd3c80043c0 con 0x55b3fa559530 2024-09-19T09:09:30.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.003+0000 7fd3de3cf640 1 --2- 172.21.15.120:0/4281781367 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b3fa559530 0x55b3fa55b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:30.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.003+0000 7fd3de3cf640 1 --2- 172.21.15.120:0/4281781367 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b3fa559530 0x55b3fa55b950 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=381982dbc5572470 server_cookie=174fe30e1e48259c in_seq=0 out_seq=0 2024-09-19T09:09:30.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.003+0000 7fd3c1ffb640 10 client.17805 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:30.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.003+0000 7fd3c1ffb640 1 -- 172.21.15.120:0/4281781367 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd3c80043c0 con 0x55b3fa559530 2024-09-19T09:09:30.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.003+0000 7fd3c1ffb640 10 client.17805 handle_command_reply: tid=0 2024-09-19T09:09:30.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.004+0000 7fd3e2a0c740 1 client.17805 shutdown 2024-09-19T09:09:30.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.004+0000 7fd3e2a0c740 1 -- 172.21.15.120:0/4281781367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3fa4bbb40 msgr2=0x55b3fa4d61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:30.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.004+0000 7fd3e2a0c740 1 --2- 172.21.15.120:0/4281781367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3fa4bbb40 0x55b3fa4d61d0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8005ab0 tx=0x7fd3c802ab90 comp rx=0 tx=0).stop 2024-09-19T09:09:30.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.004+0000 7fd3e2a0c740 1 -- 172.21.15.120:0/4281781367 shutdown_connections 2024-09-19T09:09:30.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.004+0000 7fd3e2a0c740 1 --2- 172.21.15.120:0/4281781367 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b3fa559530 0x55b3fa55b950 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:30.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.004+0000 7fd3e2a0c740 1 --2- 172.21.15.120:0/4281781367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3fa4c5370 0x55b3fa4be380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:30.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.004+0000 7fd3e2a0c740 1 --2- 172.21.15.120:0/4281781367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3fa4bbb40 0x55b3fa4d61d0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:30.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.004+0000 7fd3e2a0c740 1 --2- 172.21.15.120:0/4281781367 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b3fa4ba5d0 0x55b3fa4d5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:30.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.004+0000 7fd3e2a0c740 1 -- 172.21.15.120:0/4281781367 >> 172.21.15.120:0/4281781367 conn(0x55b3fa483310 msgr2=0x55b3fa4c38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:30.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.004+0000 7fd3c1ffb640 0 client.17805 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:30.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.004+0000 7fd3e2a0c740 1 -- 172.21.15.120:0/4281781367 shutdown_connections 2024-09-19T09:09:30.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.004+0000 7fd3e2a0c740 1 -- 172.21.15.120:0/4281781367 wait complete. 2024-09-19T09:09:30.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.005+0000 7fd3e2a0c740 20 client.17805 trim_cache size 0 max 16384 2024-09-19T09:09:30.009 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:09:30.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.005+0000 7fd3e0633640 1 -- 172.21.15.120:0/4227555815 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd3c40a3410 msgr2=0x7fd3c40a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:30.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.005+0000 7fd3e0633640 1 --2- 172.21.15.120:0/4227555815 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd3c40a3410 0x7fd3c40a58d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8031550 tx=0x7fd3c8005a20 comp rx=0 tx=0).stop 2024-09-19T09:09:30.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.005+0000 7fd3e0633640 1 -- 172.21.15.120:0/4227555815 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd3d8103fe0 msgr2=0x7fd3d819a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:30.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.005+0000 7fd3e0633640 1 --2- 172.21.15.120:0/4227555815 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd3d8103fe0 0x7fd3d819a750 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fd3d400e9f0 tx=0x7fd3d400eeb0 comp rx=0 tx=0).stop 2024-09-19T09:09:30.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.005+0000 7fd3e0633640 1 -- 172.21.15.120:0/4227555815 shutdown_connections 2024-09-19T09:09:30.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.005+0000 7fd3e0633640 1 --2- 172.21.15.120:0/4227555815 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd3c40a3410 0x7fd3c40a58d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:30.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.005+0000 7fd3e0633640 1 --2- 172.21.15.120:0/4227555815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3d8109110 0x7fd3d819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:30.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.005+0000 7fd3e0633640 1 --2- 172.21.15.120:0/4227555815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3d8104990 0x7fd3d819ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:30.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.005+0000 7fd3e0633640 1 --2- 172.21.15.120:0/4227555815 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd3d8103fe0 0x7fd3d819a750 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:30.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.005+0000 7fd3e0633640 1 -- 172.21.15.120:0/4227555815 >> 172.21.15.120:0/4227555815 conn(0x7fd3d80ffa40 msgr2=0x7fd3d8100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:30.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.006+0000 7fd3e0633640 1 -- 172.21.15.120:0/4227555815 shutdown_connections 2024-09-19T09:09:30.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:30.006+0000 7fd3e0633640 1 -- 172.21.15.120:0/4227555815 wait complete. 2024-09-19T09:09:30.053 INFO:tasks.workunit.client.0.smithi120.stdout: 1 596827 9.80 MB/sec execute 205 sec latency 51.435 ms 2024-09-19T09:09:31.053 INFO:tasks.workunit.client.0.smithi120.stdout: 1 599692 9.82 MB/sec execute 206 sec latency 10.675 ms 2024-09-19T09:09:31.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:30 smithi120 ceph-mon[31150]: pgmap v541: 97 pgs: 97 active+clean; 42 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.5 MiB/s wr, 402 op/s 2024-09-19T09:09:31.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:30 smithi196 ceph-mon[35150]: pgmap v541: 97 pgs: 97 active+clean; 42 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.5 MiB/s wr, 402 op/s 2024-09-19T09:09:31.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:30 smithi153 ceph-mon[35071]: pgmap v541: 97 pgs: 97 active+clean; 42 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.5 MiB/s wr, 402 op/s 2024-09-19T09:09:32.053 INFO:tasks.workunit.client.0.smithi120.stdout: 1 602234 9.83 MB/sec execute 207 sec latency 10.572 ms 2024-09-19T09:09:32.230 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:09:32.231 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:09:32.485 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:09:32.991 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:32 smithi120 ceph-mon[31150]: pgmap v542: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 467 op/s 2024-09-19T09:09:33.053 INFO:tasks.workunit.client.0.smithi120.stdout: 1 604783 9.85 MB/sec execute 208 sec latency 18.669 ms 2024-09-19T09:09:33.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.076+0000 7fe9cfb33640 1 -- 172.21.15.120:0/1595530533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe9c8102ef0 msgr2=0x7fe9c810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:33.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.076+0000 7fe9cfb33640 1 --2- 172.21.15.120:0/1595530533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe9c8102ef0 0x7fe9c810da40 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0009900 tx=0x7fe9c002a480 comp rx=0 tx=0).stop 2024-09-19T09:09:33.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.076+0000 7fe9cfb33640 1 -- 172.21.15.120:0/1595530533 shutdown_connections 2024-09-19T09:09:33.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.076+0000 7fe9cfb33640 1 --2- 172.21.15.120:0/1595530533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe9c8102ef0 0x7fe9c810da40 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.076+0000 7fe9cfb33640 1 --2- 172.21.15.120:0/1595530533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9c8102570 0x7fe9c81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.076+0000 7fe9cfb33640 1 --2- 172.21.15.120:0/1595530533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9c8100e20 0x7fe9c8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.076+0000 7fe9cfb33640 1 -- 172.21.15.120:0/1595530533 >> 172.21.15.120:0/1595530533 conn(0x7fe9c80fc7d0 msgr2=0x7fe9c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:33.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.076+0000 7fe9cfb33640 1 -- 172.21.15.120:0/1595530533 shutdown_connections 2024-09-19T09:09:33.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.076+0000 7fe9cfb33640 1 -- 172.21.15.120:0/1595530533 wait complete. 2024-09-19T09:09:33.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.076+0000 7fe9cfb33640 1 Processor -- start 2024-09-19T09:09:33.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.076+0000 7fe9cfb33640 1 -- start start 2024-09-19T09:09:33.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.077+0000 7fe9cfb33640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9c8100e20 0x7fe9c819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:33.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.077+0000 7fe9cfb33640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9c8102570 0x7fe9c819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:33.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.077+0000 7fe9cfb33640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe9c8102ef0 0x7fe9c819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:33.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.077+0000 7fe9cfb33640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c8110f80 con 0x7fe9c8102ef0 2024-09-19T09:09:33.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.077+0000 7fe9cfb33640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c8110e00 con 0x7fe9c8102570 2024-09-19T09:09:33.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.077+0000 7fe9cfb33640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c8111100 con 0x7fe9c8100e20 2024-09-19T09:09:33.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.077+0000 7fe9ce0d0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe9c8102ef0 0x7fe9c819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:33.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.077+0000 7fe9cd0ce640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9c8102570 0x7fe9c819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:33.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.077+0000 7fe9cd8cf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9c8100e20 0x7fe9c819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:33.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.077+0000 7fe9ce0d0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe9c8102ef0 0x7fe9c819ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60312/0 (socket says 172.21.15.120:60312) 2024-09-19T09:09:33.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.077+0000 7fe9cd0ce640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9c8102570 0x7fe9c819abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37392/0 (socket says 172.21.15.120:37392) 2024-09-19T09:09:33.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.077+0000 7fe9ce0d0640 1 -- 172.21.15.120:0/4087573795 learned_addr learned my addr 172.21.15.120:0/4087573795 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:33.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.077+0000 7fe9cd8cf640 1 -- 172.21.15.120:0/4087573795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9c8102570 msgr2=0x7fe9c819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:33.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.077+0000 7fe9cd8cf640 1 --2- 172.21.15.120:0/4087573795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9c8102570 0x7fe9c819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.077+0000 7fe9cd8cf640 1 -- 172.21.15.120:0/4087573795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe9c8102ef0 msgr2=0x7fe9c819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:33.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.077+0000 7fe9cd8cf640 1 --2- 172.21.15.120:0/4087573795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe9c8102ef0 0x7fe9c819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.077+0000 7fe9cd8cf640 1 -- 172.21.15.120:0/4087573795 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c819f650 con 0x7fe9c8100e20 2024-09-19T09:09:33.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.077+0000 7fe9cd8cf640 1 --2- 172.21.15.120:0/4087573795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9c8100e20 0x7fe9c819a6a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009fd0 tx=0x7fe9b800ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:33.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.077+0000 7fe9c6ffd640 1 -- 172.21.15.120:0/4087573795 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8004b20 con 0x7fe9c8100e20 2024-09-19T09:09:33.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.078+0000 7fe9cfb33640 1 -- 172.21.15.120:0/4087573795 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9c819f940 con 0x7fe9c8100e20 2024-09-19T09:09:33.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.078+0000 7fe9c6ffd640 1 -- 172.21.15.120:0/4087573795 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9b8004cc0 con 0x7fe9c8100e20 2024-09-19T09:09:33.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.078+0000 7fe9c6ffd640 1 -- 172.21.15.120:0/4087573795 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b80107b0 con 0x7fe9c8100e20 2024-09-19T09:09:33.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.078+0000 7fe9cfb33640 1 -- 172.21.15.120:0/4087573795 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9c81a7220 con 0x7fe9c8100e20 2024-09-19T09:09:33.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.080+0000 7fe9c6ffd640 1 -- 172.21.15.120:0/4087573795 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe9b8005020 con 0x7fe9c8100e20 2024-09-19T09:09:33.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.080+0000 7fe9d1f0c740 1 Processor -- start 2024-09-19T09:09:33.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.080+0000 7fe9d1f0c740 1 -- start start 2024-09-19T09:09:33.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.080+0000 7fe9d1f0c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646a64f5b60 0x5646a64f5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:33.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.080+0000 7fe9d1f0c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646a64f6480 0x5646a6500800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:33.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.080+0000 7fe9d1f0c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646a6500d40 0x5646a6503130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:33.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.080+0000 7fe9d1f0c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5646a645b280 con 0x5646a64f5b60 2024-09-19T09:09:33.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.080+0000 7fe9d1f0c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5646a6402b10 con 0x5646a64f6480 2024-09-19T09:09:33.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.080+0000 7fe9d1f0c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5646a6457240 con 0x5646a6500d40 2024-09-19T09:09:33.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.080+0000 7fe9ce0d0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646a64f5b60 0x5646a64f5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:33.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.080+0000 7fe9ce0d0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646a64f5b60 0x5646a64f5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60324/0 (socket says 172.21.15.120:60324) 2024-09-19T09:09:33.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.081+0000 7fe9ce0d0640 1 -- 172.21.15.120:0/1956411474 learned_addr learned my addr 172.21.15.120:0/1956411474 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:33.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.080+0000 7fe9cd8cf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646a64f6480 0x5646a6500800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:33.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.081+0000 7fe9cd0ce640 1 --2- 172.21.15.120:0/1956411474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646a6500d40 0x5646a6503130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:33.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.081+0000 7fe9ce0d0640 1 -- 172.21.15.120:0/1956411474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646a6500d40 msgr2=0x5646a6503130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:33.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.081+0000 7fe9ce0d0640 1 --2- 172.21.15.120:0/1956411474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646a6500d40 0x5646a6503130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.081+0000 7fe9ce0d0640 1 -- 172.21.15.120:0/1956411474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646a64f6480 msgr2=0x5646a6500800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:33.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.081+0000 7fe9ce0d0640 1 --2- 172.21.15.120:0/1956411474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646a64f6480 0x5646a6500800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.081+0000 7fe9ce0d0640 1 -- 172.21.15.120:0/1956411474 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646a640f1f0 con 0x5646a64f5b60 2024-09-19T09:09:33.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.081+0000 7fe9c6ffd640 1 --2- 172.21.15.120:0/4087573795 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9b00a3240 0x7fe9b00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:33.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.081+0000 7fe9ce0d0640 1 --2- 172.21.15.120:0/1956411474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646a64f5b60 0x5646a64f5f40 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0006010 tx=0x7fe9c0005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ca15bf818c93fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:33.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.081+0000 7fe9cd8cf640 1 --2- 172.21.15.120:0/4087573795 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9b00a3240 0x7fe9b00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:33.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.081+0000 7fe9c6ffd640 1 -- 172.21.15.120:0/4087573795 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fe9b80d0090 con 0x7fe9c8100e20 2024-09-19T09:09:33.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.081+0000 7fe9affff640 1 -- 172.21.15.120:0/1956411474 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0035030 con 0x5646a64f5b60 2024-09-19T09:09:33.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.081+0000 7fe9affff640 1 -- 172.21.15.120:0/1956411474 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c0040070 con 0x5646a64f5b60 2024-09-19T09:09:33.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.081+0000 7fe9affff640 1 -- 172.21.15.120:0/1956411474 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c00318b0 con 0x5646a64f5b60 2024-09-19T09:09:33.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.082+0000 7fe9d1f0c740 1 -- 172.21.15.120:0/1956411474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646a64f5b60 msgr2=0x5646a64f5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:33.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.082+0000 7fe9d1f0c740 1 --2- 172.21.15.120:0/1956411474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646a64f5b60 0x5646a64f5f40 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0006010 tx=0x7fe9c0005dd0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.082+0000 7fe9cd8cf640 1 --2- 172.21.15.120:0/4087573795 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9b00a3240 0x7fe9b00a5700 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800c760 tx=0x7fe9b80d3b10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:33.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.082+0000 7fe9d1f0c740 1 -- 172.21.15.120:0/1956411474 shutdown_connections 2024-09-19T09:09:33.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.082+0000 7fe9d1f0c740 1 --2- 172.21.15.120:0/1956411474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646a6500d40 0x5646a6503130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.082+0000 7fe9d1f0c740 1 --2- 172.21.15.120:0/1956411474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646a64f6480 0x5646a6500800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.082+0000 7fe9d1f0c740 1 --2- 172.21.15.120:0/1956411474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646a64f5b60 0x5646a64f5f40 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.082+0000 7fe9d1f0c740 1 -- 172.21.15.120:0/1956411474 >> 172.21.15.120:0/1956411474 conn(0x5646a64f20f0 msgr2=0x5646a64f24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:33.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.082+0000 7fe9d1f0c740 1 -- 172.21.15.120:0/1956411474 shutdown_connections 2024-09-19T09:09:33.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.082+0000 7fe9d1f0c740 1 -- 172.21.15.120:0/1956411474 wait complete. 2024-09-19T09:09:33.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.082+0000 7fe9d1f0c740 1 -- start start 2024-09-19T09:09:33.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.082+0000 7fe9d1f0c740 1 Processor -- start 2024-09-19T09:09:33.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.083+0000 7fe9d1f0c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646a64f5b60 0x5646a64fdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:33.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.083+0000 7fe9d1f0c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646a64f6480 0x5646a64fe150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:33.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.083+0000 7fe9d1f0c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646a6500d40 0x5646a64fe690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:33.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.083+0000 7fe9d1f0c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5646a6503910 con 0x5646a64f6480 2024-09-19T09:09:33.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.083+0000 7fe9d1f0c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5646a6415f10 con 0x5646a64f5b60 2024-09-19T09:09:33.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.083+0000 7fe9d1f0c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5646a6503a90 con 0x5646a6500d40 2024-09-19T09:09:33.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.083+0000 7fe9d1f0c740 10 client.0 _ll_register_callbacks cb 0x5646a63dea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:33.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.083+0000 7fe9cd0ce640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646a64f6480 0x5646a64fe150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:33.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.083+0000 7fe9ce0d0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646a64f5b60 0x5646a64fdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:33.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.083+0000 7fe9ce0d0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646a64f5b60 0x5646a64fdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37412/0 (socket says 172.21.15.120:37412) 2024-09-19T09:09:33.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.083+0000 7fe9ce0d0640 1 -- 172.21.15.120:0/1106124313 learned_addr learned my addr 172.21.15.120:0/1106124313 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:33.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.083+0000 7fe9ce0d0640 1 --2- 172.21.15.120:0/1106124313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646a6500d40 0x5646a64fe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:33.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.083+0000 7fe9ce0d0640 1 -- 172.21.15.120:0/1106124313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646a6500d40 msgr2=0x5646a64fe690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:33.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.083+0000 7fe9ce0d0640 1 --2- 172.21.15.120:0/1106124313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646a6500d40 0x5646a64fe690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.083+0000 7fe9ce0d0640 1 -- 172.21.15.120:0/1106124313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646a64f6480 msgr2=0x5646a64fe150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:33.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.083+0000 7fe9ce0d0640 1 --2- 172.21.15.120:0/1106124313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646a64f6480 0x5646a64fe150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.083+0000 7fe9ce0d0640 1 -- 172.21.15.120:0/1106124313 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646a61cc6a0 con 0x5646a64f5b60 2024-09-19T09:09:33.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.084+0000 7fe9ce0d0640 1 --2- 172.21.15.120:0/1106124313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646a64f5b60 0x5646a64fdc10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0003e00 tx=0x7fe9c003e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d9b496ea5777785 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:33.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.084+0000 7fe9ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:09:33.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.084+0000 7fe9ad7fa640 1 -- 172.21.15.120:0/1106124313 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0035070 con 0x5646a64f5b60 2024-09-19T09:09:33.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.084+0000 7fe9d1f0c740 1 -- 172.21.15.120:0/1106124313 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9c0040070 con 0x5646a64f5b60 2024-09-19T09:09:33.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.084+0000 7fe9ad7fa640 1 -- 172.21.15.120:0/1106124313 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fe9c0004410 con 0x5646a64f5b60 2024-09-19T09:09:33.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.084+0000 7fe9ad7fa640 1 -- 172.21.15.120:0/1106124313 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe9c005b050 con 0x5646a64f5b60 2024-09-19T09:09:33.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.084+0000 7fe9d1f0c740 10 client.27068 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:33.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.084+0000 7fe9d1f0c740 1 -- 172.21.15.120:0/1106124313 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5646a650f510 con 0x5646a64f5b60 2024-09-19T09:09:33.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.084+0000 7fe9ad7fa640 1 -- 172.21.15.120:0/1106124313 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c0040070 con 0x5646a64f5b60 2024-09-19T09:09:33.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.084+0000 7fe9ad7fa640 1 -- 172.21.15.120:0/1106124313 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe9c0053020 con 0x5646a64f5b60 2024-09-19T09:09:33.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.084+0000 7fe9d1f0c740 10 client.27068 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:33.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.084+0000 7fe9d1f0c740 10 client.27068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:09:33.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.084+0000 7fe9d1f0c740 20 client.27068 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:33.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.084+0000 7fe9d1f0c740 1 --2- 172.21.15.120:0/1106124313 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5646a6510860 0x5646a6512c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:33.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.084+0000 7fe9d1f0c740 4 client.27068 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:09:33.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.084+0000 7fe9d1f0c740 1 -- 172.21.15.120:0/1106124313 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646a64687f0 con 0x5646a6510860 2024-09-19T09:09:33.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.085+0000 7fe9cd0ce640 1 --2- 172.21.15.120:0/1106124313 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5646a6510860 0x5646a6512c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:33.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.085+0000 7fe9cd0ce640 1 --2- 172.21.15.120:0/1106124313 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5646a6510860 0x5646a6512c50 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9b813b6414f967b server_cookie=401346dc62e7c9 in_seq=0 out_seq=0 2024-09-19T09:09:33.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.085+0000 7fe9ad7fa640 10 client.27068 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:33.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.085+0000 7fe9ad7fa640 1 -- 172.21.15.120:0/1106124313 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5646a64687f0 con 0x5646a6510860 2024-09-19T09:09:33.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.085+0000 7fe9ad7fa640 10 client.27068 handle_command_reply: tid=0 2024-09-19T09:09:33.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.086+0000 7fe9d1f0c740 1 client.27068 shutdown 2024-09-19T09:09:33.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.086+0000 7fe9d1f0c740 1 -- 172.21.15.120:0/1106124313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646a64f5b60 msgr2=0x5646a64fdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:33.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.086+0000 7fe9d1f0c740 1 --2- 172.21.15.120:0/1106124313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646a64f5b60 0x5646a64fdc10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0003e00 tx=0x7fe9c003e040 comp rx=0 tx=0).stop 2024-09-19T09:09:33.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.086+0000 7fe9d1f0c740 1 -- 172.21.15.120:0/1106124313 shutdown_connections 2024-09-19T09:09:33.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.086+0000 7fe9d1f0c740 1 --2- 172.21.15.120:0/1106124313 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5646a6510860 0x5646a6512c50 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.086+0000 7fe9d1f0c740 1 --2- 172.21.15.120:0/1106124313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646a6500d40 0x5646a64fe690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.086+0000 7fe9d1f0c740 1 --2- 172.21.15.120:0/1106124313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646a64f6480 0x5646a64fe150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.086+0000 7fe9ad7fa640 0 client.27068 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:33.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.086+0000 7fe9d1f0c740 1 --2- 172.21.15.120:0/1106124313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646a64f5b60 0x5646a64fdc10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.086+0000 7fe9d1f0c740 1 -- 172.21.15.120:0/1106124313 >> 172.21.15.120:0/1106124313 conn(0x5646a64f20f0 msgr2=0x5646a64f1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:33.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.086+0000 7fe9d1f0c740 1 -- 172.21.15.120:0/1106124313 shutdown_connections 2024-09-19T09:09:33.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.086+0000 7fe9d1f0c740 1 -- 172.21.15.120:0/1106124313 wait complete. 2024-09-19T09:09:33.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.086+0000 7fe9d1f0c740 20 client.27068 trim_cache size 0 max 16384 2024-09-19T09:09:33.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.103+0000 7fe9d1f0c740 1 Processor -- start 2024-09-19T09:09:33.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.103+0000 7fe9d1f0c740 1 -- start start 2024-09-19T09:09:33.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.103+0000 7fe9d1f0c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646a64f5b60 0x5646a64fae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:33.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.103+0000 7fe9d1f0c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646a64f6480 0x5646a64fb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:33.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.103+0000 7fe9d1f0c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646a6500d40 0x5646a650bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:33.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.104+0000 7fe9d1f0c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c81af6b0 con 0x5646a6500d40 2024-09-19T09:09:33.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.104+0000 7fe9d1f0c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c81b11f0 con 0x5646a64f6480 2024-09-19T09:09:33.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.104+0000 7fe9d1f0c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c81afd80 con 0x5646a64f5b60 2024-09-19T09:09:33.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.104+0000 7fe9ce0d0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646a6500d40 0x5646a650bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:33.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.104+0000 7fe9ce0d0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646a6500d40 0x5646a650bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60342/0 (socket says 172.21.15.120:60342) 2024-09-19T09:09:33.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.104+0000 7fe9ce0d0640 1 -- 172.21.15.120:0/3485124432 learned_addr learned my addr 172.21.15.120:0/3485124432 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:33.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.104+0000 7fe9ce0d0640 1 --2- 172.21.15.120:0/3485124432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646a64f5b60 0x5646a64fae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:33.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.104+0000 7fe9cd0ce640 1 --2- 172.21.15.120:0/3485124432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646a64f6480 0x5646a64fb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:33.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.104+0000 7fe9ce0d0640 1 -- 172.21.15.120:0/3485124432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646a64f5b60 msgr2=0x5646a64fae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:33.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.104+0000 7fe9ce0d0640 1 --2- 172.21.15.120:0/3485124432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646a64f5b60 0x5646a64fae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.104+0000 7fe9ce0d0640 1 -- 172.21.15.120:0/3485124432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646a64f6480 msgr2=0x5646a64fb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:33.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.104+0000 7fe9ce0d0640 1 --2- 172.21.15.120:0/3485124432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646a64f6480 0x5646a64fb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.104+0000 7fe9ce0d0640 1 -- 172.21.15.120:0/3485124432 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646a64f1b90 con 0x5646a6500d40 2024-09-19T09:09:33.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.104+0000 7fe9ce0d0640 1 --2- 172.21.15.120:0/3485124432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646a6500d40 0x5646a650bce0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00096c0 tx=0x7fe9c0005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0604d59e36e7e30 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:33.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.104+0000 7fe9af7fe640 1 -- 172.21.15.120:0/3485124432 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0035020 con 0x5646a6500d40 2024-09-19T09:09:33.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.104+0000 7fe9af7fe640 1 -- 172.21.15.120:0/3485124432 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c0040030 con 0x5646a6500d40 2024-09-19T09:09:33.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.104+0000 7fe9af7fe640 1 -- 172.21.15.120:0/3485124432 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0031ab0 con 0x5646a6500d40 2024-09-19T09:09:33.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.105+0000 7fe9d1f0c740 1 -- 172.21.15.120:0/3485124432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646a6500d40 msgr2=0x5646a650bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:33.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.105+0000 7fe9d1f0c740 1 --2- 172.21.15.120:0/3485124432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646a6500d40 0x5646a650bce0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00096c0 tx=0x7fe9c0005a20 comp rx=0 tx=0).stop 2024-09-19T09:09:33.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.105+0000 7fe9d1f0c740 1 -- 172.21.15.120:0/3485124432 shutdown_connections 2024-09-19T09:09:33.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.105+0000 7fe9d1f0c740 1 --2- 172.21.15.120:0/3485124432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646a6500d40 0x5646a650bce0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.105+0000 7fe9d1f0c740 1 --2- 172.21.15.120:0/3485124432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646a64f6480 0x5646a64fb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.105+0000 7fe9d1f0c740 1 --2- 172.21.15.120:0/3485124432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646a64f5b60 0x5646a64fae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.105+0000 7fe9d1f0c740 1 -- 172.21.15.120:0/3485124432 >> 172.21.15.120:0/3485124432 conn(0x5646a64f20f0 msgr2=0x5646a64ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:33.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.105+0000 7fe9d1f0c740 1 -- 172.21.15.120:0/3485124432 shutdown_connections 2024-09-19T09:09:33.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.105+0000 7fe9d1f0c740 1 -- 172.21.15.120:0/3485124432 wait complete. 2024-09-19T09:09:33.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.105+0000 7fe9d1f0c740 1 -- start start 2024-09-19T09:09:33.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.105+0000 7fe9d1f0c740 1 Processor -- start 2024-09-19T09:09:33.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.106+0000 7fe9d1f0c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646a64f5b60 0x5646a6513d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:33.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.106+0000 7fe9d1f0c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646a64f6480 0x5646a65142b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:33.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.106+0000 7fe9d1f0c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646a6500d40 0x5646a657a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:33.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.106+0000 7fe9d1f0c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c81b0b20 con 0x5646a6500d40 2024-09-19T09:09:33.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.106+0000 7fe9d1f0c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c81b0450 con 0x5646a64f6480 2024-09-19T09:09:33.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.106+0000 7fe9d1f0c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c81ad670 con 0x5646a64f5b60 2024-09-19T09:09:33.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.106+0000 7fe9d1f0c740 10 client.0 _ll_register_callbacks cb 0x7fe9b80d39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:33.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.106+0000 7fe9ce0d0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646a6500d40 0x5646a657a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:33.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.106+0000 7fe9ce0d0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646a6500d40 0x5646a657a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60356/0 (socket says 172.21.15.120:60356) 2024-09-19T09:09:33.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.106+0000 7fe9ce0d0640 1 -- 172.21.15.120:0/41840771 learned_addr learned my addr 172.21.15.120:0/41840771 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:33.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.106+0000 7fe9cd0ce640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646a64f6480 0x5646a65142b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:33.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.106+0000 7fe9ce0d0640 1 --2- 172.21.15.120:0/41840771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646a64f5b60 0x5646a6513d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:33.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.106+0000 7fe9cd0ce640 1 -- 172.21.15.120:0/41840771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646a64f5b60 msgr2=0x5646a6513d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:33.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.106+0000 7fe9cd0ce640 1 --2- 172.21.15.120:0/41840771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646a64f5b60 0x5646a6513d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.106+0000 7fe9cd0ce640 1 -- 172.21.15.120:0/41840771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646a6500d40 msgr2=0x5646a657a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:33.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.106+0000 7fe9cd0ce640 1 --2- 172.21.15.120:0/41840771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646a6500d40 0x5646a657a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.106+0000 7fe9cd0ce640 1 -- 172.21.15.120:0/41840771 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646a6514b90 con 0x5646a64f6480 2024-09-19T09:09:33.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.106+0000 7fe9ce0d0640 1 --2- 172.21.15.120:0/41840771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646a6500d40 0x5646a657a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:33.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.106+0000 7fe9cd0ce640 1 --2- 172.21.15.120:0/41840771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646a64f6480 0x5646a65142b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x5646a6503b50 tx=0x7fe9bc0041e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d8acd927153c094 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:33.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.106+0000 7fe9ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:09:33.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.107+0000 7fe9ad7fa640 1 -- 172.21.15.120:0/41840771 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc02b060 con 0x5646a64f6480 2024-09-19T09:09:33.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.107+0000 7fe9d1f0c740 1 -- 172.21.15.120:0/41840771 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9c0040030 con 0x5646a64f6480 2024-09-19T09:09:33.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.107+0000 7fe9ad7fa640 1 -- 172.21.15.120:0/41840771 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fe9bc020410 con 0x5646a64f6480 2024-09-19T09:09:33.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.107+0000 7fe9ad7fa640 1 -- 172.21.15.120:0/41840771 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe9bc003e50 con 0x5646a64f6480 2024-09-19T09:09:33.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.107+0000 7fe9d1f0c740 10 client.27074 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:33.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.107+0000 7fe9d1f0c740 1 -- 172.21.15.120:0/41840771 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5646a657b890 con 0x5646a64f6480 2024-09-19T09:09:33.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.107+0000 7fe9ad7fa640 1 -- 172.21.15.120:0/41840771 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c0040030 con 0x5646a64f6480 2024-09-19T09:09:33.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.107+0000 7fe9ad7fa640 1 -- 172.21.15.120:0/41840771 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe9bc030070 con 0x5646a64f6480 2024-09-19T09:09:33.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.107+0000 7fe9d1f0c740 10 client.27074 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:33.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.107+0000 7fe9d1f0c740 10 client.27074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:09:33.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.107+0000 7fe9d1f0c740 20 client.27074 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:33.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.107+0000 7fe9d1f0c740 1 --2- 172.21.15.120:0/41840771 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5646a6510860 0x5646a657df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:33.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.107+0000 7fe9d1f0c740 4 client.27074 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:09:33.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.107+0000 7fe9d1f0c740 1 -- 172.21.15.120:0/41840771 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5646a6462d00 con 0x5646a6510860 2024-09-19T09:09:33.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.107+0000 7fe9ce0d0640 1 --2- 172.21.15.120:0/41840771 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5646a6510860 0x5646a657df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:33.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.108+0000 7fe9ad7fa640 10 client.27074 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:33.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.108+0000 7fe9ce0d0640 1 --2- 172.21.15.120:0/41840771 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5646a6510860 0x5646a657df70 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d83df9ba38cee07 server_cookie=6b357592d89febb5 in_seq=0 out_seq=0 2024-09-19T09:09:33.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.109+0000 7fe9ad7fa640 1 -- 172.21.15.120:0/41840771 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5646a6462d00 con 0x5646a6510860 2024-09-19T09:09:33.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.109+0000 7fe9ad7fa640 10 client.27074 handle_command_reply: tid=0 2024-09-19T09:09:33.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.109+0000 7fe9d1f0c740 1 client.27074 shutdown 2024-09-19T09:09:33.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.109+0000 7fe9d1f0c740 1 -- 172.21.15.120:0/41840771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646a64f6480 msgr2=0x5646a65142b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:33.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.109+0000 7fe9d1f0c740 1 --2- 172.21.15.120:0/41840771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646a64f6480 0x5646a65142b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x5646a6503b50 tx=0x7fe9bc0041e0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.109+0000 7fe9d1f0c740 1 -- 172.21.15.120:0/41840771 shutdown_connections 2024-09-19T09:09:33.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.109+0000 7fe9d1f0c740 1 --2- 172.21.15.120:0/41840771 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5646a6510860 0x5646a657df70 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.109+0000 7fe9d1f0c740 1 --2- 172.21.15.120:0/41840771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646a6500d40 0x5646a657a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.109+0000 7fe9d1f0c740 1 --2- 172.21.15.120:0/41840771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646a64f6480 0x5646a65142b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.109+0000 7fe9d1f0c740 1 --2- 172.21.15.120:0/41840771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646a64f5b60 0x5646a6513d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.109+0000 7fe9ad7fa640 0 client.27074 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:33.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.109+0000 7fe9d1f0c740 1 -- 172.21.15.120:0/41840771 >> 172.21.15.120:0/41840771 conn(0x5646a64f20f0 msgr2=0x5646a650bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:33.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.109+0000 7fe9d1f0c740 1 -- 172.21.15.120:0/41840771 shutdown_connections 2024-09-19T09:09:33.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.109+0000 7fe9d1f0c740 1 -- 172.21.15.120:0/41840771 wait complete. 2024-09-19T09:09:33.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.109+0000 7fe9d1f0c740 20 client.27074 trim_cache size 0 max 16384 2024-09-19T09:09:33.115 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:09:33.116 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:09:33.116 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "56eee442-ddd1-4611-9b9f-08bd0332de2f", 2024-09-19T09:09:33.116 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:09:33.116 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:09:33.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.110+0000 7fe9cfb33640 1 -- 172.21.15.120:0/4087573795 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9b00a3240 msgr2=0x7fe9b00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:33.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.110+0000 7fe9cfb33640 1 --2- 172.21.15.120:0/4087573795 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9b00a3240 0x7fe9b00a5700 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800c760 tx=0x7fe9b80d3b10 comp rx=0 tx=0).stop 2024-09-19T09:09:33.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.110+0000 7fe9cfb33640 1 -- 172.21.15.120:0/4087573795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9c8100e20 msgr2=0x7fe9c819a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:33.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.110+0000 7fe9cfb33640 1 --2- 172.21.15.120:0/4087573795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9c8100e20 0x7fe9c819a6a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009fd0 tx=0x7fe9b800ef90 comp rx=0 tx=0).stop 2024-09-19T09:09:33.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.110+0000 7fe9cfb33640 1 -- 172.21.15.120:0/4087573795 shutdown_connections 2024-09-19T09:09:33.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.110+0000 7fe9cfb33640 1 --2- 172.21.15.120:0/4087573795 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9b00a3240 0x7fe9b00a5700 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.110+0000 7fe9cfb33640 1 --2- 172.21.15.120:0/4087573795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe9c8102ef0 0x7fe9c819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.110+0000 7fe9cfb33640 1 --2- 172.21.15.120:0/4087573795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9c8102570 0x7fe9c819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.110+0000 7fe9cfb33640 1 --2- 172.21.15.120:0/4087573795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9c8100e20 0x7fe9c819a6a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:33.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.110+0000 7fe9cfb33640 1 -- 172.21.15.120:0/4087573795 >> 172.21.15.120:0/4087573795 conn(0x7fe9c80fc7d0 msgr2=0x7fe9c806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:33.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.110+0000 7fe9cfb33640 1 -- 172.21.15.120:0/4087573795 shutdown_connections 2024-09-19T09:09:33.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:33.110+0000 7fe9cfb33640 1 -- 172.21.15.120:0/4087573795 wait complete. 2024-09-19T09:09:33.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:32 smithi196 ceph-mon[35150]: pgmap v542: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 467 op/s 2024-09-19T09:09:33.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:32 smithi153 ceph-mon[35071]: pgmap v542: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 467 op/s 2024-09-19T09:09:34.054 INFO:tasks.workunit.client.0.smithi120.stdout: 1 607938 9.89 MB/sec execute 209 sec latency 8.937 ms 2024-09-19T09:09:34.075 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:09:34.339 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:09:34.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.888+0000 7f0b33dd2640 1 -- 172.21.15.120:0/3580212744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b2c1008d0 msgr2=0x7f0b2c100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:34.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.888+0000 7f0b33dd2640 1 --2- 172.21.15.120:0/3580212744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b2c1008d0 0x7f0b2c100cb0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009860 tx=0x7f0b2802a340 comp rx=0 tx=0).stop 2024-09-19T09:09:34.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.888+0000 7f0b33dd2640 1 -- 172.21.15.120:0/3580212744 shutdown_connections 2024-09-19T09:09:34.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.888+0000 7f0b33dd2640 1 --2- 172.21.15.120:0/3580212744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b2c10e480 0x7f0b2c110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.888+0000 7f0b33dd2640 1 --2- 172.21.15.120:0/3580212744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b2c101280 0x7f0b2c10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.888+0000 7f0b33dd2640 1 --2- 172.21.15.120:0/3580212744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b2c1008d0 0x7f0b2c100cb0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.888+0000 7f0b33dd2640 1 -- 172.21.15.120:0/3580212744 >> 172.21.15.120:0/3580212744 conn(0x7f0b2c0fc7a0 msgr2=0x7f0b2c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:34.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.889+0000 7f0b33dd2640 1 -- 172.21.15.120:0/3580212744 shutdown_connections 2024-09-19T09:09:34.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.889+0000 7f0b33dd2640 1 -- 172.21.15.120:0/3580212744 wait complete. 2024-09-19T09:09:34.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.889+0000 7f0b33dd2640 1 Processor -- start 2024-09-19T09:09:34.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.889+0000 7f0b33dd2640 1 -- start start 2024-09-19T09:09:34.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.889+0000 7f0b33dd2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b2c1008d0 0x7f0b2c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:34.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.889+0000 7f0b33dd2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b2c101280 0x7f0b2c19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:34.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.889+0000 7f0b33dd2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b2c10e480 0x7f0b2c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:34.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.889+0000 7f0b33dd2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c1141b0 con 0x7f0b2c101280 2024-09-19T09:09:34.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.889+0000 7f0b33dd2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c114030 con 0x7f0b2c10e480 2024-09-19T09:09:34.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.889+0000 7f0b33dd2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c114330 con 0x7f0b2c1008d0 2024-09-19T09:09:34.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.890+0000 7f0b3136d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b2c101280 0x7f0b2c19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:34.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.890+0000 7f0b3136d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b2c101280 0x7f0b2c19abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60380/0 (socket says 172.21.15.120:60380) 2024-09-19T09:09:34.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.890+0000 7f0b3136d640 1 -- 172.21.15.120:0/3056410582 learned_addr learned my addr 172.21.15.120:0/3056410582 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:34.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.890+0000 7f0b3236f640 1 --2- 172.21.15.120:0/3056410582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b2c10e480 0x7f0b2c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:34.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.890+0000 7f0b31b6e640 1 --2- 172.21.15.120:0/3056410582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b2c1008d0 0x7f0b2c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:34.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.890+0000 7f0b3136d640 1 -- 172.21.15.120:0/3056410582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b2c1008d0 msgr2=0x7f0b2c19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:34.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.890+0000 7f0b3136d640 1 --2- 172.21.15.120:0/3056410582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b2c1008d0 0x7f0b2c19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.890+0000 7f0b3136d640 1 -- 172.21.15.120:0/3056410582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b2c10e480 msgr2=0x7f0b2c19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:34.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.890+0000 7f0b3136d640 1 --2- 172.21.15.120:0/3056410582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b2c10e480 0x7f0b2c19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.890+0000 7f0b3136d640 1 -- 172.21.15.120:0/3056410582 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2c19f640 con 0x7f0b2c101280 2024-09-19T09:09:34.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.890+0000 7f0b31b6e640 1 --2- 172.21.15.120:0/3056410582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b2c1008d0 0x7f0b2c19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:09:34.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.890+0000 7f0b3136d640 1 --2- 172.21.15.120:0/3056410582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b2c101280 0x7f0b2c19abd0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00e9c0 tx=0x7f0b1c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:34.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.890+0000 7f0b22ffd640 1 -- 172.21.15.120:0/3056410582 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c00ccc0 con 0x7f0b2c101280 2024-09-19T09:09:34.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.890+0000 7f0b33dd2640 1 -- 172.21.15.120:0/3056410582 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b2c19f930 con 0x7f0b2c101280 2024-09-19T09:09:34.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.890+0000 7f0b33dd2640 1 -- 172.21.15.120:0/3056410582 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b2c1a7210 con 0x7f0b2c101280 2024-09-19T09:09:34.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.890+0000 7f0b22ffd640 1 -- 172.21.15.120:0/3056410582 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b1c005500 con 0x7f0b2c101280 2024-09-19T09:09:34.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.890+0000 7f0b22ffd640 1 -- 172.21.15.120:0/3056410582 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c010430 con 0x7f0b2c101280 2024-09-19T09:09:34.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.893+0000 7f0b22ffd640 1 -- 172.21.15.120:0/3056410582 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f0b1c0046c0 con 0x7f0b2c101280 2024-09-19T09:09:34.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.893+0000 7f0b361ab740 1 Processor -- start 2024-09-19T09:09:34.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.893+0000 7f0b361ab740 1 -- start start 2024-09-19T09:09:34.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.894+0000 7f0b361ab740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55649da734a0 0x55649da73880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:34.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.894+0000 7f0b361ab740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55649da73dc0 0x55649da7e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:34.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.894+0000 7f0b361ab740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55649da7e680 0x55649da80a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:34.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.894+0000 7f0b361ab740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55649d9d92e0 con 0x55649da73dc0 2024-09-19T09:09:34.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.894+0000 7f0b361ab740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55649d980ce0 con 0x55649da7e680 2024-09-19T09:09:34.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.894+0000 7f0b361ab740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55649d9d52a0 con 0x55649da734a0 2024-09-19T09:09:34.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.894+0000 7f0b31b6e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55649da73dc0 0x55649da7e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:34.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.894+0000 7f0b22ffd640 1 --2- 172.21.15.120:0/3056410582 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0b180a3240 0x7f0b180a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:34.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.894+0000 7f0b3236f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55649da734a0 0x55649da73880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:34.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.894+0000 7f0b3136d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55649da7e680 0x55649da80a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:34.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.894+0000 7f0b31b6e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55649da73dc0 0x55649da7e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60394/0 (socket says 172.21.15.120:60394) 2024-09-19T09:09:34.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.894+0000 7f0b3236f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55649da734a0 0x55649da73880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:44868/0 (socket says 172.21.15.120:44868) 2024-09-19T09:09:34.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.894+0000 7f0b31b6e640 1 -- 172.21.15.120:0/552970962 learned_addr learned my addr 172.21.15.120:0/552970962 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:34.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.894+0000 7f0b31b6e640 1 -- 172.21.15.120:0/552970962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55649da734a0 msgr2=0x55649da73880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:34.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.894+0000 7f0b31b6e640 1 --2- 172.21.15.120:0/552970962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55649da734a0 0x55649da73880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.894+0000 7f0b3236f640 1 --2- 172.21.15.120:0/3056410582 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0b180a3240 0x7f0b180a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:34.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.894+0000 7f0b31b6e640 1 -- 172.21.15.120:0/552970962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55649da7e680 msgr2=0x55649da80a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:34.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.894+0000 7f0b31b6e640 1 --2- 172.21.15.120:0/552970962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55649da7e680 0x55649da80a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.894+0000 7f0b31b6e640 1 -- 172.21.15.120:0/552970962 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55649d965440 con 0x55649da73dc0 2024-09-19T09:09:34.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.894+0000 7f0b22ffd640 1 -- 172.21.15.120:0/3056410582 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f0b1c0cfaa0 con 0x7f0b2c101280 2024-09-19T09:09:34.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.894+0000 7f0b31b6e640 1 --2- 172.21.15.120:0/552970962 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55649da73dc0 0x55649da7e140 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009860 tx=0x7f0b280303c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=facc2e0ffa9b5404 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:34.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.894+0000 7f0b3236f640 1 --2- 172.21.15.120:0/3056410582 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0b180a3240 0x7f0b180a5700 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c06a0f0 tx=0x7f0b24004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:34.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.894+0000 7f0b17fff640 1 -- 172.21.15.120:0/552970962 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28037040 con 0x55649da73dc0 2024-09-19T09:09:34.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.894+0000 7f0b17fff640 1 -- 172.21.15.120:0/552970962 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b28030550 con 0x55649da73dc0 2024-09-19T09:09:34.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.895+0000 7f0b17fff640 1 -- 172.21.15.120:0/552970962 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28005540 con 0x55649da73dc0 2024-09-19T09:09:34.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.895+0000 7f0b361ab740 1 -- 172.21.15.120:0/552970962 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55649da73dc0 msgr2=0x55649da7e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:34.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.895+0000 7f0b361ab740 1 --2- 172.21.15.120:0/552970962 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55649da73dc0 0x55649da7e140 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009860 tx=0x7f0b280303c0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.895+0000 7f0b361ab740 1 -- 172.21.15.120:0/552970962 shutdown_connections 2024-09-19T09:09:34.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.895+0000 7f0b361ab740 1 --2- 172.21.15.120:0/552970962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55649da7e680 0x55649da80a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.895+0000 7f0b361ab740 1 --2- 172.21.15.120:0/552970962 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55649da73dc0 0x55649da7e140 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.895+0000 7f0b361ab740 1 --2- 172.21.15.120:0/552970962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55649da734a0 0x55649da73880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.895+0000 7f0b361ab740 1 -- 172.21.15.120:0/552970962 >> 172.21.15.120:0/552970962 conn(0x55649da6fa30 msgr2=0x55649da6fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:34.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.896+0000 7f0b361ab740 1 -- 172.21.15.120:0/552970962 shutdown_connections 2024-09-19T09:09:34.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.896+0000 7f0b361ab740 1 -- 172.21.15.120:0/552970962 wait complete. 2024-09-19T09:09:34.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.896+0000 7f0b361ab740 1 -- start start 2024-09-19T09:09:34.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.896+0000 7f0b361ab740 1 Processor -- start 2024-09-19T09:09:34.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.897+0000 7f0b361ab740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55649da734a0 0x55649da7dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:34.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.897+0000 7f0b361ab740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55649da73dc0 0x55649da7b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:34.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.897+0000 7f0b3136d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55649da73dc0 0x55649da7b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:34.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.897+0000 7f0b361ab740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55649da7e680 0x55649da7b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:34.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.897+0000 7f0b31b6e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55649da734a0 0x55649da7dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:34.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.897+0000 7f0b3136d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55649da73dc0 0x55649da7b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60398/0 (socket says 172.21.15.120:60398) 2024-09-19T09:09:34.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.897+0000 7f0b361ab740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55649da81250 con 0x55649da73dc0 2024-09-19T09:09:34.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.897+0000 7f0b3136d640 1 -- 172.21.15.120:0/2900078427 learned_addr learned my addr 172.21.15.120:0/2900078427 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:34.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.897+0000 7f0b361ab740 1 -- 172.21.15.120:0/2900078427 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55649d994050 con 0x55649da734a0 2024-09-19T09:09:34.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.897+0000 7f0b361ab740 1 -- 172.21.15.120:0/2900078427 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55649da813d0 con 0x55649da7e680 2024-09-19T09:09:34.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.897+0000 7f0b361ab740 10 client.0 _ll_register_callbacks cb 0x55649d9d4160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:34.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.897+0000 7f0b3236f640 1 --2- 172.21.15.120:0/2900078427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55649da7e680 0x55649da7b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:34.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.897+0000 7f0b3136d640 1 -- 172.21.15.120:0/2900078427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55649da7e680 msgr2=0x55649da7b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:34.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.897+0000 7f0b3136d640 1 --2- 172.21.15.120:0/2900078427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55649da7e680 0x55649da7b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.897+0000 7f0b3136d640 1 -- 172.21.15.120:0/2900078427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55649da734a0 msgr2=0x55649da7dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:34.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.897+0000 7f0b3136d640 1 --2- 172.21.15.120:0/2900078427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55649da734a0 0x55649da7dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.897+0000 7f0b3136d640 1 -- 172.21.15.120:0/2900078427 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55649d74a3c0 con 0x55649da73dc0 2024-09-19T09:09:34.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.897+0000 7f0b3136d640 1 --2- 172.21.15.120:0/2900078427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55649da73dc0 0x55649da7b190 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c09c050 tx=0x7f0b1c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ab1d6a4986c6cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:34.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.897+0000 7f0b157fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:09:34.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.897+0000 7f0b157fa640 1 -- 172.21.15.120:0/2900078427 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c09c7c0 con 0x55649da73dc0 2024-09-19T09:09:34.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.897+0000 7f0b361ab740 1 -- 172.21.15.120:0/2900078427 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b28030550 con 0x55649da73dc0 2024-09-19T09:09:34.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.897+0000 7f0b157fa640 1 -- 172.21.15.120:0/2900078427 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f0b1c098730 con 0x55649da73dc0 2024-09-19T09:09:34.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.898+0000 7f0b157fa640 1 -- 172.21.15.120:0/2900078427 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0b1c0a4d30 con 0x55649da73dc0 2024-09-19T09:09:34.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.898+0000 7f0b361ab740 10 client.17835 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:34.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.898+0000 7f0b361ab740 1 -- 172.21.15.120:0/2900078427 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55649da8cc80 con 0x55649da73dc0 2024-09-19T09:09:34.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.898+0000 7f0b157fa640 1 -- 172.21.15.120:0/2900078427 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b28030550 con 0x55649da73dc0 2024-09-19T09:09:34.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.898+0000 7f0b157fa640 1 -- 172.21.15.120:0/2900078427 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0b1c09b510 con 0x55649da73dc0 2024-09-19T09:09:34.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.898+0000 7f0b361ab740 10 client.17835 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:34.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.898+0000 7f0b361ab740 10 client.17835 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:09:34.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.898+0000 7f0b361ab740 20 client.17835 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:34.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.898+0000 7f0b361ab740 1 --2- 172.21.15.120:0/2900078427 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55649da8e0e0 0x55649da90580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:34.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.898+0000 7f0b361ab740 4 client.17835 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:09:34.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.898+0000 7f0b361ab740 1 -- 172.21.15.120:0/2900078427 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55649d9e6850 con 0x55649da8e0e0 2024-09-19T09:09:34.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.898+0000 7f0b31b6e640 1 --2- 172.21.15.120:0/2900078427 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55649da8e0e0 0x55649da90580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:34.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.899+0000 7f0b31b6e640 1 --2- 172.21.15.120:0/2900078427 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55649da8e0e0 0x55649da90580 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b42d68195167e9e1 server_cookie=18ef1f0c9202ba57 in_seq=0 out_seq=0 2024-09-19T09:09:34.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.899+0000 7f0b157fa640 10 client.17835 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:34.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.900+0000 7f0b157fa640 1 -- 172.21.15.120:0/2900078427 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55649d9e6850 con 0x55649da8e0e0 2024-09-19T09:09:34.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.900+0000 7f0b157fa640 10 client.17835 handle_command_reply: tid=0 2024-09-19T09:09:34.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.900+0000 7f0b361ab740 1 client.17835 shutdown 2024-09-19T09:09:34.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.900+0000 7f0b361ab740 1 -- 172.21.15.120:0/2900078427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55649da73dc0 msgr2=0x55649da7b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:34.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.900+0000 7f0b361ab740 1 --2- 172.21.15.120:0/2900078427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55649da73dc0 0x55649da7b190 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c09c050 tx=0x7f0b1c009f90 comp rx=0 tx=0).stop 2024-09-19T09:09:34.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.900+0000 7f0b361ab740 1 -- 172.21.15.120:0/2900078427 shutdown_connections 2024-09-19T09:09:34.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.900+0000 7f0b361ab740 1 --2- 172.21.15.120:0/2900078427 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55649da8e0e0 0x55649da90580 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.900+0000 7f0b361ab740 1 --2- 172.21.15.120:0/2900078427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55649da7e680 0x55649da7b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.900+0000 7f0b361ab740 1 --2- 172.21.15.120:0/2900078427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55649da73dc0 0x55649da7b190 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.900+0000 7f0b361ab740 1 --2- 172.21.15.120:0/2900078427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55649da734a0 0x55649da7dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.900+0000 7f0b157fa640 0 client.17835 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:34.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.900+0000 7f0b361ab740 1 -- 172.21.15.120:0/2900078427 >> 172.21.15.120:0/2900078427 conn(0x55649da6fa30 msgr2=0x55649da75710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:34.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.900+0000 7f0b361ab740 1 -- 172.21.15.120:0/2900078427 shutdown_connections 2024-09-19T09:09:34.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.900+0000 7f0b361ab740 1 -- 172.21.15.120:0/2900078427 wait complete. 2024-09-19T09:09:34.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.900+0000 7f0b361ab740 20 client.17835 trim_cache size 0 max 16384 2024-09-19T09:09:34.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.919+0000 7f0b361ab740 1 Processor -- start 2024-09-19T09:09:34.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.919+0000 7f0b361ab740 1 -- start start 2024-09-19T09:09:34.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.919+0000 7f0b361ab740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55649da734a0 0x55649da76900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:34.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.919+0000 7f0b361ab740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55649da73dc0 0x55649da79a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:34.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.919+0000 7f0b361ab740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55649da7e680 0x55649da6ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:34.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.919+0000 7f0b361ab740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1000e450 con 0x55649da73dc0 2024-09-19T09:09:34.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.919+0000 7f0b361ab740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1000ff90 con 0x55649da734a0 2024-09-19T09:09:34.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.919+0000 7f0b361ab740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1000eb20 con 0x55649da7e680 2024-09-19T09:09:34.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.919+0000 7f0b3136d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55649da73dc0 0x55649da79a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:34.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.919+0000 7f0b3136d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55649da73dc0 0x55649da79a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60404/0 (socket says 172.21.15.120:60404) 2024-09-19T09:09:34.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.919+0000 7f0b3136d640 1 -- 172.21.15.120:0/3678616955 learned_addr learned my addr 172.21.15.120:0/3678616955 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:34.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.919+0000 7f0b31b6e640 1 --2- 172.21.15.120:0/3678616955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55649da734a0 0x55649da76900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:34.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.919+0000 7f0b3236f640 1 --2- 172.21.15.120:0/3678616955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55649da7e680 0x55649da6ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:34.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.920+0000 7f0b31b6e640 1 -- 172.21.15.120:0/3678616955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55649da7e680 msgr2=0x55649da6ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:34.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.920+0000 7f0b31b6e640 1 --2- 172.21.15.120:0/3678616955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55649da7e680 0x55649da6ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.920+0000 7f0b31b6e640 1 -- 172.21.15.120:0/3678616955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55649da73dc0 msgr2=0x55649da79a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:34.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.920+0000 7f0b31b6e640 1 --2- 172.21.15.120:0/3678616955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55649da73dc0 0x55649da79a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.920+0000 7f0b31b6e640 1 -- 172.21.15.120:0/3678616955 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55649da7a140 con 0x55649da734a0 2024-09-19T09:09:34.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.920+0000 7f0b3136d640 1 --2- 172.21.15.120:0/3678616955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55649da73dc0 0x55649da79a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:34.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.920+0000 7f0b31b6e640 1 --2- 172.21.15.120:0/3678616955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55649da734a0 0x55649da76900 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0b2802a930 tx=0x7f0b28009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40470ae86f1f2797 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:34.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.920+0000 7f0b177fe640 1 -- 172.21.15.120:0/3678616955 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2804d070 con 0x55649da734a0 2024-09-19T09:09:34.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.920+0000 7f0b177fe640 1 -- 172.21.15.120:0/3678616955 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b28030550 con 0x55649da734a0 2024-09-19T09:09:34.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.920+0000 7f0b177fe640 1 -- 172.21.15.120:0/3678616955 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28037040 con 0x55649da734a0 2024-09-19T09:09:34.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.921+0000 7f0b361ab740 1 -- 172.21.15.120:0/3678616955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55649da734a0 msgr2=0x55649da76900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:34.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.921+0000 7f0b361ab740 1 --2- 172.21.15.120:0/3678616955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55649da734a0 0x55649da76900 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0b2802a930 tx=0x7f0b28009490 comp rx=0 tx=0).stop 2024-09-19T09:09:34.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.921+0000 7f0b361ab740 1 -- 172.21.15.120:0/3678616955 shutdown_connections 2024-09-19T09:09:34.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.921+0000 7f0b361ab740 1 --2- 172.21.15.120:0/3678616955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55649da7e680 0x55649da6ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.921+0000 7f0b361ab740 1 --2- 172.21.15.120:0/3678616955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55649da73dc0 0x55649da79a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.921+0000 7f0b361ab740 1 --2- 172.21.15.120:0/3678616955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55649da734a0 0x55649da76900 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.921+0000 7f0b361ab740 1 -- 172.21.15.120:0/3678616955 >> 172.21.15.120:0/3678616955 conn(0x55649da6fa30 msgr2=0x55649da6e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:34.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.921+0000 7f0b361ab740 1 -- 172.21.15.120:0/3678616955 shutdown_connections 2024-09-19T09:09:34.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.921+0000 7f0b361ab740 1 -- 172.21.15.120:0/3678616955 wait complete. 2024-09-19T09:09:34.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.921+0000 7f0b361ab740 1 -- start start 2024-09-19T09:09:34.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.921+0000 7f0b361ab740 1 Processor -- start 2024-09-19T09:09:34.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.922+0000 7f0b361ab740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55649da734a0 0x55649da82540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:34.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.922+0000 7f0b361ab740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55649da73dc0 0x55649da82a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:34.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.922+0000 7f0b361ab740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55649da7e680 0x55649da82fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:34.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.922+0000 7f0b361ab740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1000f8c0 con 0x55649da734a0 2024-09-19T09:09:34.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.922+0000 7f0b361ab740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1000f1f0 con 0x55649da73dc0 2024-09-19T09:09:34.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.922+0000 7f0b361ab740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1000c410 con 0x55649da7e680 2024-09-19T09:09:34.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.922+0000 7f0b361ab740 10 client.0 _ll_register_callbacks cb 0x7f0b1c0049b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:34.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.922+0000 7f0b31b6e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55649da734a0 0x55649da82540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:34.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.922+0000 7f0b3136d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55649da73dc0 0x55649da82a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:34.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.922+0000 7f0b31b6e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55649da734a0 0x55649da82540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60412/0 (socket says 172.21.15.120:60412) 2024-09-19T09:09:34.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.922+0000 7f0b31b6e640 1 -- 172.21.15.120:0/484579251 learned_addr learned my addr 172.21.15.120:0/484579251 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:34.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.922+0000 7f0b3136d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55649da73dc0 0x55649da82a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37484/0 (socket says 172.21.15.120:37484) 2024-09-19T09:09:34.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.922+0000 7f0b3236f640 1 --2- 172.21.15.120:0/484579251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55649da7e680 0x55649da82fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:34.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.922+0000 7f0b3136d640 1 -- 172.21.15.120:0/484579251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55649da7e680 msgr2=0x55649da82fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:34.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.922+0000 7f0b3136d640 1 --2- 172.21.15.120:0/484579251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55649da7e680 0x55649da82fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.922+0000 7f0b3136d640 1 -- 172.21.15.120:0/484579251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55649da734a0 msgr2=0x55649da82540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:34.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.922+0000 7f0b3136d640 1 --2- 172.21.15.120:0/484579251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55649da734a0 0x55649da82540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.922+0000 7f0b3136d640 1 -- 172.21.15.120:0/484579251 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55649da7e250 con 0x55649da73dc0 2024-09-19T09:09:34.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.922+0000 7f0b31b6e640 1 --2- 172.21.15.120:0/484579251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55649da734a0 0x55649da82540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:34.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.922+0000 7f0b3136d640 1 --2- 172.21.15.120:0/484579251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55649da73dc0 0x55649da82a80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00c6a0 tx=0x7f0b1c0a0370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=925b7ba71eba94f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:34.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.922+0000 7f0b157fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:09:34.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.923+0000 7f0b157fa640 1 -- 172.21.15.120:0/484579251 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c0a3580 con 0x55649da73dc0 2024-09-19T09:09:34.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.923+0000 7f0b157fa640 1 -- 172.21.15.120:0/484579251 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f0b1c005020 con 0x55649da73dc0 2024-09-19T09:09:34.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.923+0000 7f0b361ab740 1 -- 172.21.15.120:0/484579251 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b28030550 con 0x55649da73dc0 2024-09-19T09:09:34.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.923+0000 7f0b157fa640 1 -- 172.21.15.120:0/484579251 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0b1c0a4b00 con 0x55649da73dc0 2024-09-19T09:09:34.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.923+0000 7f0b361ab740 10 client.27092 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:34.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.923+0000 7f0b361ab740 1 -- 172.21.15.120:0/484579251 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55649daf9060 con 0x55649da73dc0 2024-09-19T09:09:34.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.923+0000 7f0b157fa640 1 -- 172.21.15.120:0/484579251 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b28030550 con 0x55649da73dc0 2024-09-19T09:09:34.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.923+0000 7f0b157fa640 1 -- 172.21.15.120:0/484579251 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0b1c09af80 con 0x55649da73dc0 2024-09-19T09:09:34.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.923+0000 7f0b361ab740 10 client.27092 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:34.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.923+0000 7f0b361ab740 10 client.27092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:09:34.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.923+0000 7f0b361ab740 20 client.27092 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:34.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.923+0000 7f0b361ab740 1 --2- 172.21.15.120:0/484579251 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55649daf9700 0x55649dafbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:34.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.923+0000 7f0b361ab740 4 client.27092 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:09:34.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.923+0000 7f0b361ab740 1 -- 172.21.15.120:0/484579251 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55649d9e0d60 con 0x55649daf9700 2024-09-19T09:09:34.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.923+0000 7f0b31b6e640 1 --2- 172.21.15.120:0/484579251 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55649daf9700 0x55649dafbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:34.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.924+0000 7f0b157fa640 10 client.27092 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:34.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.924+0000 7f0b31b6e640 1 --2- 172.21.15.120:0/484579251 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55649daf9700 0x55649dafbb20 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e5ac66239c3275b server_cookie=f0b02cd1b9dff214 in_seq=0 out_seq=0 2024-09-19T09:09:34.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.925+0000 7f0b157fa640 1 -- 172.21.15.120:0/484579251 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55649d9e0d60 con 0x55649daf9700 2024-09-19T09:09:34.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.925+0000 7f0b157fa640 10 client.27092 handle_command_reply: tid=0 2024-09-19T09:09:34.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.925+0000 7f0b361ab740 1 client.27092 shutdown 2024-09-19T09:09:34.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.925+0000 7f0b361ab740 1 -- 172.21.15.120:0/484579251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55649da73dc0 msgr2=0x55649da82a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:34.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.925+0000 7f0b361ab740 1 --2- 172.21.15.120:0/484579251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55649da73dc0 0x55649da82a80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00c6a0 tx=0x7f0b1c0a0370 comp rx=0 tx=0).stop 2024-09-19T09:09:34.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.925+0000 7f0b361ab740 1 -- 172.21.15.120:0/484579251 shutdown_connections 2024-09-19T09:09:34.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.925+0000 7f0b361ab740 1 --2- 172.21.15.120:0/484579251 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55649daf9700 0x55649dafbb20 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.925+0000 7f0b361ab740 1 --2- 172.21.15.120:0/484579251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55649da7e680 0x55649da82fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.925+0000 7f0b361ab740 1 --2- 172.21.15.120:0/484579251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55649da73dc0 0x55649da82a80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.925+0000 7f0b361ab740 1 --2- 172.21.15.120:0/484579251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55649da734a0 0x55649da82540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.925+0000 7f0b157fa640 0 client.27092 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:34.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.925+0000 7f0b361ab740 1 -- 172.21.15.120:0/484579251 >> 172.21.15.120:0/484579251 conn(0x55649da6fa30 msgr2=0x55649da7d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:34.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.925+0000 7f0b361ab740 1 -- 172.21.15.120:0/484579251 shutdown_connections 2024-09-19T09:09:34.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.925+0000 7f0b361ab740 1 -- 172.21.15.120:0/484579251 wait complete. 2024-09-19T09:09:34.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.925+0000 7f0b361ab740 20 client.27092 trim_cache size 0 max 16384 2024-09-19T09:09:34.934 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:09:34.934 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:09:34.934 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-19T09:09:34.934 INFO:teuthology.orchestra.run.smithi120.stdout: "56eee442-ddd1-4611-9b9f-08bd0332de2f": { 2024-09-19T09:09:34.934 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-19T09:09:34.934 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "56eee442-ddd1-4611-9b9f-08bd0332de2f", 2024-09-19T09:09:34.935 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-19T09:09:34.935 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:09:34.935 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:09:34.935 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:09:34.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.926+0000 7f0b33dd2640 1 -- 172.21.15.120:0/3056410582 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0b180a3240 msgr2=0x7f0b180a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:34.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.926+0000 7f0b33dd2640 1 --2- 172.21.15.120:0/3056410582 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0b180a3240 0x7f0b180a5700 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c06a0f0 tx=0x7f0b24004ab0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.926+0000 7f0b33dd2640 1 -- 172.21.15.120:0/3056410582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b2c101280 msgr2=0x7f0b2c19abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:34.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.926+0000 7f0b33dd2640 1 --2- 172.21.15.120:0/3056410582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b2c101280 0x7f0b2c19abd0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00e9c0 tx=0x7f0b1c00ee80 comp rx=0 tx=0).stop 2024-09-19T09:09:34.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.926+0000 7f0b33dd2640 1 -- 172.21.15.120:0/3056410582 shutdown_connections 2024-09-19T09:09:34.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.926+0000 7f0b33dd2640 1 --2- 172.21.15.120:0/3056410582 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0b180a3240 0x7f0b180a5700 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.926+0000 7f0b33dd2640 1 --2- 172.21.15.120:0/3056410582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b2c10e480 0x7f0b2c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.926+0000 7f0b33dd2640 1 --2- 172.21.15.120:0/3056410582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b2c101280 0x7f0b2c19abd0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.926+0000 7f0b33dd2640 1 --2- 172.21.15.120:0/3056410582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b2c1008d0 0x7f0b2c19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:34.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.926+0000 7f0b33dd2640 1 -- 172.21.15.120:0/3056410582 >> 172.21.15.120:0/3056410582 conn(0x7f0b2c0fc7a0 msgr2=0x7f0b2c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:34.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.926+0000 7f0b33dd2640 1 -- 172.21.15.120:0/3056410582 shutdown_connections 2024-09-19T09:09:34.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:09:34.926+0000 7f0b33dd2640 1 -- 172.21.15.120:0/3056410582 wait complete. 2024-09-19T09:09:35.053 INFO:tasks.workunit.client.0.smithi120.stdout: 1 610708 9.91 MB/sec execute 210 sec latency 113.936 ms 2024-09-19T09:09:35.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:34 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:09:35.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:34 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:09:35.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:34 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:09:35.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:34 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:09:35.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:34 smithi120 ceph-mon[31150]: pgmap v543: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.2 MiB/s wr, 426 op/s 2024-09-19T09:09:35.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:34 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:09:35.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:34 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:09:35.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:34 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:09:35.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:34 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:09:35.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:34 smithi196 ceph-mon[35150]: pgmap v543: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.2 MiB/s wr, 426 op/s 2024-09-19T09:09:35.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:34 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:09:35.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:34 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:09:35.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:34 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:09:35.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:34 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:09:35.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:34 smithi153 ceph-mon[35071]: pgmap v543: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.2 MiB/s wr, 426 op/s 2024-09-19T09:09:35.783 INFO:tasks.cephfs.filesystem:scrub status for tag:56eee442-ddd1-4611-9b9f-08bd0332de2f - {'path': '/', 'tag': '56eee442-ddd1-4611-9b9f-08bd0332de2f', 'options': 'recursive,force'} 2024-09-19T09:09:36.054 INFO:tasks.workunit.client.0.smithi120.stdout: 1 613335 9.92 MB/sec execute 211 sec latency 14.264 ms 2024-09-19T09:09:37.054 INFO:tasks.workunit.client.0.smithi120.stdout: 1 615925 9.96 MB/sec execute 212 sec latency 20.830 ms 2024-09-19T09:09:37.122 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:36 smithi120 ceph-mon[31150]: pgmap v544: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.8 MiB/s wr, 444 op/s 2024-09-19T09:09:37.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:36 smithi196 ceph-mon[35150]: pgmap v544: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.8 MiB/s wr, 444 op/s 2024-09-19T09:09:37.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:36 smithi153 ceph-mon[35071]: pgmap v544: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.8 MiB/s wr, 444 op/s 2024-09-19T09:09:38.054 INFO:tasks.workunit.client.0.smithi120.stdout: 1 618723 9.98 MB/sec execute 213 sec latency 26.629 ms 2024-09-19T09:09:39.054 INFO:tasks.workunit.client.0.smithi120.stdout: 1 621894 10.02 MB/sec execute 214 sec latency 18.101 ms 2024-09-19T09:09:39.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:38 smithi196 ceph-mon[35150]: pgmap v545: 97 pgs: 97 active+clean; 47 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.2 MiB/s wr, 459 op/s 2024-09-19T09:09:39.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:38 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:09:39.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:38 smithi153 ceph-mon[35071]: pgmap v545: 97 pgs: 97 active+clean; 47 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.2 MiB/s wr, 459 op/s 2024-09-19T09:09:39.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:38 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:09:39.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:38 smithi120 ceph-mon[31150]: pgmap v545: 97 pgs: 97 active+clean; 47 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.2 MiB/s wr, 459 op/s 2024-09-19T09:09:39.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:38 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:09:39.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.347+0000 7f5ad3053640 1 -- 172.21.15.120:0/341945224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5acc1025f0 msgr2=0x7f5acc108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:39.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.347+0000 7f5ad3053640 1 --2- 172.21.15.120:0/341945224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5acc1025f0 0x7f5acc108e80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009ca0 tx=0x7f5ac402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.347+0000 7f5ad3053640 1 -- 172.21.15.120:0/341945224 shutdown_connections 2024-09-19T09:09:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.347+0000 7f5ad3053640 1 --2- 172.21.15.120:0/341945224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5acc1025f0 0x7f5acc108e80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.347+0000 7f5ad3053640 1 --2- 172.21.15.120:0/341945224 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5acc101c30 0x7f5acc1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.347+0000 7f5ad3053640 1 --2- 172.21.15.120:0/341945224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5acc100a40 0x7f5acc100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.347+0000 7f5ad3053640 1 -- 172.21.15.120:0/341945224 >> 172.21.15.120:0/341945224 conn(0x7f5acc0fc1e0 msgr2=0x7f5acc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.347+0000 7f5ad3053640 1 -- 172.21.15.120:0/341945224 shutdown_connections 2024-09-19T09:09:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.347+0000 7f5ad3053640 1 -- 172.21.15.120:0/341945224 wait complete. 2024-09-19T09:09:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.347+0000 7f5ad3053640 1 Processor -- start 2024-09-19T09:09:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.347+0000 7f5ad3053640 1 -- start start 2024-09-19T09:09:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.348+0000 7f5ad3053640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5acc100a40 0x7f5acc197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.348+0000 7f5ad3053640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5acc101c30 0x7f5acc1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.348+0000 7f5ad3053640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5acc1025f0 0x7f5acc19f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.348+0000 7f5ad3053640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc10d810 con 0x7f5acc100a40 2024-09-19T09:09:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.348+0000 7f5ad3053640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc10d690 con 0x7f5acc101c30 2024-09-19T09:09:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.348+0000 7f5ad3053640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc10d990 con 0x7f5acc1025f0 2024-09-19T09:09:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.348+0000 7f5ad0def640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5acc100a40 0x7f5acc197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.348+0000 7f5ad0def640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5acc100a40 0x7f5acc197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60430/0 (socket says 172.21.15.120:60430) 2024-09-19T09:09:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.348+0000 7f5ad0def640 1 -- 172.21.15.120:0/19527104 learned_addr learned my addr 172.21.15.120:0/19527104 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.348+0000 7f5ad15f0640 1 --2- 172.21.15.120:0/19527104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5acc1025f0 0x7f5acc19f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.348+0000 7f5acbfff640 1 --2- 172.21.15.120:0/19527104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5acc101c30 0x7f5acc1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.348+0000 7f5ad0def640 1 -- 172.21.15.120:0/19527104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5acc1025f0 msgr2=0x7f5acc19f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.348+0000 7f5ad0def640 1 --2- 172.21.15.120:0/19527104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5acc1025f0 0x7f5acc19f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.348+0000 7f5ad0def640 1 -- 172.21.15.120:0/19527104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5acc101c30 msgr2=0x7f5acc1982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.348+0000 7f5ad0def640 1 --2- 172.21.15.120:0/19527104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5acc101c30 0x7f5acc1982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.348+0000 7f5ad0def640 1 -- 172.21.15.120:0/19527104 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5acc19f9c0 con 0x7f5acc100a40 2024-09-19T09:09:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.348+0000 7f5acbfff640 1 --2- 172.21.15.120:0/19527104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5acc101c30 0x7f5acc1982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:09:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.348+0000 7f5ad0def640 1 --2- 172.21.15.120:0/19527104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5acc100a40 0x7f5acc197db0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000e9c0 tx=0x7f5ac000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:39.352 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.348+0000 7f5ac9ffb640 1 -- 172.21.15.120:0/19527104 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac000cd80 con 0x7f5acc100a40 2024-09-19T09:09:39.352 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.348+0000 7f5ad3053640 1 -- 172.21.15.120:0/19527104 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5acc19fcb0 con 0x7f5acc100a40 2024-09-19T09:09:39.352 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.348+0000 7f5ac9ffb640 1 -- 172.21.15.120:0/19527104 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ac00035f0 con 0x7f5acc100a40 2024-09-19T09:09:39.352 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.348+0000 7f5ac9ffb640 1 -- 172.21.15.120:0/19527104 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0010690 con 0x7f5acc100a40 2024-09-19T09:09:39.352 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.348+0000 7f5ad3053640 1 -- 172.21.15.120:0/19527104 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5acc1a01b0 con 0x7f5acc100a40 2024-09-19T09:09:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.351+0000 7f5ac9ffb640 1 -- 172.21.15.120:0/19527104 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5ac00037a0 con 0x7f5acc100a40 2024-09-19T09:09:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.351+0000 7f5ad5df0740 1 Processor -- start 2024-09-19T09:09:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.351+0000 7f5ad5df0740 1 -- start start 2024-09-19T09:09:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.351+0000 7f5ad5df0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d4f2949a0 0x564d4f294da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.351+0000 7f5ad5df0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d4f2952e0 0x564d4f299da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.351+0000 7f5ad5df0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d4f29a370 0x564d4f29c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.351+0000 7f5ad5df0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564d4f1f82f0 con 0x564d4f2952e0 2024-09-19T09:09:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.351+0000 7f5ad5df0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564d4f1a0270 con 0x564d4f29a370 2024-09-19T09:09:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.351+0000 7f5ad5df0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564d4f1f42b0 con 0x564d4f2949a0 2024-09-19T09:09:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.351+0000 7f5ad0def640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d4f2952e0 0x564d4f299da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.351+0000 7f5ad15f0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d4f2949a0 0x564d4f294da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.351+0000 7f5ad15f0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d4f2949a0 0x564d4f294da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:44934/0 (socket says 172.21.15.120:44934) 2024-09-19T09:09:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.351+0000 7f5ad15f0640 1 -- 172.21.15.120:0/1482713186 learned_addr learned my addr 172.21.15.120:0/1482713186 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.351+0000 7f5acbfff640 1 --2- 172.21.15.120:0/1482713186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d4f29a370 0x564d4f29c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.351+0000 7f5ac9ffb640 1 --2- 172.21.15.120:0/19527104 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5abc0a32f0 0x7f5abc0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.352+0000 7f5ad0def640 1 -- 172.21.15.120:0/1482713186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d4f2949a0 msgr2=0x564d4f294da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.352+0000 7f5ad0def640 1 --2- 172.21.15.120:0/1482713186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d4f2949a0 0x564d4f294da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.352+0000 7f5ad0def640 1 -- 172.21.15.120:0/1482713186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d4f29a370 msgr2=0x564d4f29c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.352+0000 7f5ad0def640 1 --2- 172.21.15.120:0/1482713186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d4f29a370 0x564d4f29c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.352+0000 7f5ad0def640 1 -- 172.21.15.120:0/1482713186 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d4f19f340 con 0x564d4f2952e0 2024-09-19T09:09:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.352+0000 7f5ad15f0640 1 --2- 172.21.15.120:0/19527104 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5abc0a32f0 0x7f5abc0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.352+0000 7f5ac9ffb640 1 -- 172.21.15.120:0/19527104 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f5ac00cfda0 con 0x7f5acc100a40 2024-09-19T09:09:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.352+0000 7f5ad0def640 1 --2- 172.21.15.120:0/1482713186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d4f2952e0 0x564d4f299da0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00cd840 tx=0x7f5ac0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2df3b5befea0143 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.352+0000 7f5ab6ffd640 1 -- 172.21.15.120:0/1482713186 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac009bf00 con 0x564d4f2952e0 2024-09-19T09:09:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.352+0000 7f5ad15f0640 1 --2- 172.21.15.120:0/19527104 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5abc0a32f0 0x7f5abc0a57b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4005e30 tx=0x7f5ac40095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.352+0000 7f5ab6ffd640 1 -- 172.21.15.120:0/1482713186 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ac009cac0 con 0x564d4f2952e0 2024-09-19T09:09:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.352+0000 7f5ab6ffd640 1 -- 172.21.15.120:0/1482713186 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0065280 con 0x564d4f2952e0 2024-09-19T09:09:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.353+0000 7f5ad5df0740 1 -- 172.21.15.120:0/1482713186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d4f2952e0 msgr2=0x564d4f299da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.353+0000 7f5ad5df0740 1 --2- 172.21.15.120:0/1482713186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d4f2952e0 0x564d4f299da0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00cd840 tx=0x7f5ac0003f10 comp rx=0 tx=0).stop 2024-09-19T09:09:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.353+0000 7f5ad5df0740 1 -- 172.21.15.120:0/1482713186 shutdown_connections 2024-09-19T09:09:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.353+0000 7f5ad5df0740 1 --2- 172.21.15.120:0/1482713186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d4f29a370 0x564d4f29c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.353+0000 7f5ad5df0740 1 --2- 172.21.15.120:0/1482713186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d4f2952e0 0x564d4f299da0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.353+0000 7f5ad5df0740 1 --2- 172.21.15.120:0/1482713186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d4f2949a0 0x564d4f294da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.353+0000 7f5ad5df0740 1 -- 172.21.15.120:0/1482713186 >> 172.21.15.120:0/1482713186 conn(0x564d4f290010 msgr2=0x564d4f290410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.353+0000 7f5ad5df0740 1 -- 172.21.15.120:0/1482713186 shutdown_connections 2024-09-19T09:09:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.353+0000 7f5ad5df0740 1 -- 172.21.15.120:0/1482713186 wait complete. 2024-09-19T09:09:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.354+0000 7f5ad5df0740 1 -- start start 2024-09-19T09:09:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.354+0000 7f5ad5df0740 1 Processor -- start 2024-09-19T09:09:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.354+0000 7f5ad5df0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d4f2949a0 0x564d4f29eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.354+0000 7f5ad5df0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d4f2952e0 0x564d4f29f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.354+0000 7f5ad5df0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d4f29a370 0x564d4f2a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.354+0000 7f5ad5df0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564d4f29d2e0 con 0x564d4f2952e0 2024-09-19T09:09:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.354+0000 7f5ad5df0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564d4f181290 con 0x564d4f2949a0 2024-09-19T09:09:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.354+0000 7f5ad5df0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564d4f29d460 con 0x564d4f29a370 2024-09-19T09:09:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.354+0000 7f5ad5df0740 10 client.0 _ll_register_callbacks cb 0x564d4f1b2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.354+0000 7f5acbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d4f2952e0 0x564d4f29f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.354+0000 7f5acbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d4f2952e0 0x564d4f29f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60440/0 (socket says 172.21.15.120:60440) 2024-09-19T09:09:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.354+0000 7f5acbfff640 1 -- 172.21.15.120:0/3829419475 learned_addr learned my addr 172.21.15.120:0/3829419475 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.354+0000 7f5ad0def640 1 --2- 172.21.15.120:0/3829419475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d4f2949a0 0x564d4f29eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.354+0000 7f5ad15f0640 1 --2- 172.21.15.120:0/3829419475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d4f29a370 0x564d4f2a60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.355+0000 7f5acbfff640 1 -- 172.21.15.120:0/3829419475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d4f29a370 msgr2=0x564d4f2a60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.355+0000 7f5acbfff640 1 --2- 172.21.15.120:0/3829419475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d4f29a370 0x564d4f2a60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.355+0000 7f5acbfff640 1 -- 172.21.15.120:0/3829419475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d4f2949a0 msgr2=0x564d4f29eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.355+0000 7f5acbfff640 1 --2- 172.21.15.120:0/3829419475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d4f2949a0 0x564d4f29eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.355+0000 7f5acbfff640 1 -- 172.21.15.120:0/3829419475 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d4ef0b210 con 0x564d4f2952e0 2024-09-19T09:09:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.355+0000 7f5acbfff640 1 --2- 172.21.15.120:0/3829419475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d4f2952e0 0x564d4f29f070 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8012770 tx=0x7f5ab8012c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1504f3eb9c405bc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.355+0000 7f5aabfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:09:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.355+0000 7f5aabfff640 1 -- 172.21.15.120:0/3829419475 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8011bb0 con 0x564d4f2952e0 2024-09-19T09:09:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.355+0000 7f5aabfff640 1 -- 172.21.15.120:0/3829419475 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f5ab8011d50 con 0x564d4f2952e0 2024-09-19T09:09:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.355+0000 7f5ad5df0740 1 -- 172.21.15.120:0/3829419475 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ac009cac0 con 0x564d4f2952e0 2024-09-19T09:09:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.355+0000 7f5aabfff640 1 -- 172.21.15.120:0/3829419475 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5ab800f460 con 0x564d4f2952e0 2024-09-19T09:09:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.355+0000 7f5ad5df0740 10 client.17853 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.355+0000 7f5ad5df0740 1 -- 172.21.15.120:0/3829419475 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564d4efd7620 con 0x564d4f2952e0 2024-09-19T09:09:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.355+0000 7f5aabfff640 1 -- 172.21.15.120:0/3829419475 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ac009cac0 con 0x564d4f2952e0 2024-09-19T09:09:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.355+0000 7f5aabfff640 1 -- 172.21.15.120:0/3829419475 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5ab801c780 con 0x564d4f2952e0 2024-09-19T09:09:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.355+0000 7f5ad5df0740 10 client.17853 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.355+0000 7f5ad5df0740 10 client.17853 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:09:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.355+0000 7f5ad5df0740 20 client.17853 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.355+0000 7f5ad5df0740 1 --2- 172.21.15.120:0/3829419475 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564d4f2aa820 0x564d4f2acc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.355+0000 7f5ad5df0740 4 client.17853 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:09:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.355+0000 7f5ad5df0740 1 -- 172.21.15.120:0/3829419475 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d4f205a20 con 0x564d4f2aa820 2024-09-19T09:09:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.356+0000 7f5ad0def640 1 --2- 172.21.15.120:0/3829419475 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564d4f2aa820 0x564d4f2acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.356+0000 7f5ad0def640 1 --2- 172.21.15.120:0/3829419475 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564d4f2aa820 0x564d4f2acc90 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2608c2ad7a4f788 server_cookie=704f8c612bd8416 in_seq=0 out_seq=0 2024-09-19T09:09:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.356+0000 7f5aabfff640 10 client.17853 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.357+0000 7f5aabfff640 1 -- 172.21.15.120:0/3829419475 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564d4f205a20 con 0x564d4f2aa820 2024-09-19T09:09:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.357+0000 7f5aabfff640 10 client.17853 handle_command_reply: tid=0 2024-09-19T09:09:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.357+0000 7f5ad5df0740 1 client.17853 shutdown 2024-09-19T09:09:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.358+0000 7f5ad5df0740 1 -- 172.21.15.120:0/3829419475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d4f2952e0 msgr2=0x564d4f29f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.358+0000 7f5ad5df0740 1 --2- 172.21.15.120:0/3829419475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d4f2952e0 0x564d4f29f070 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8012770 tx=0x7f5ab8012c30 comp rx=0 tx=0).stop 2024-09-19T09:09:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.358+0000 7f5ad5df0740 1 -- 172.21.15.120:0/3829419475 shutdown_connections 2024-09-19T09:09:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.358+0000 7f5ad5df0740 1 --2- 172.21.15.120:0/3829419475 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564d4f2aa820 0x564d4f2acc90 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.358+0000 7f5ad5df0740 1 --2- 172.21.15.120:0/3829419475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d4f29a370 0x564d4f2a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.358+0000 7f5ad5df0740 1 --2- 172.21.15.120:0/3829419475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d4f2952e0 0x564d4f29f070 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.358+0000 7f5ad5df0740 1 --2- 172.21.15.120:0/3829419475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d4f2949a0 0x564d4f29eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.358+0000 7f5ad5df0740 1 -- 172.21.15.120:0/3829419475 >> 172.21.15.120:0/3829419475 conn(0x564d4f290010 msgr2=0x564d4f291820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.358+0000 7f5aabfff640 0 client.17853 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.358+0000 7f5ad5df0740 1 -- 172.21.15.120:0/3829419475 shutdown_connections 2024-09-19T09:09:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.358+0000 7f5ad5df0740 1 -- 172.21.15.120:0/3829419475 wait complete. 2024-09-19T09:09:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.358+0000 7f5ad5df0740 20 client.17853 trim_cache size 0 max 16384 2024-09-19T09:09:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.372+0000 7f5ad5df0740 1 Processor -- start 2024-09-19T09:09:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.372+0000 7f5ad5df0740 1 -- start start 2024-09-19T09:09:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.372+0000 7f5ad5df0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d4f2949a0 0x564d4f28f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.372+0000 7f5ad5df0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d4f2952e0 0x564d4f2a6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.372+0000 7f5ad5df0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d4f29a370 0x564d4f291400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.372+0000 7f5ad5df0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc1a8f90 con 0x564d4f2949a0 2024-09-19T09:09:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.372+0000 7f5ad5df0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc1aaad0 con 0x564d4f2952e0 2024-09-19T09:09:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.372+0000 7f5ad5df0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc1a9660 con 0x564d4f29a370 2024-09-19T09:09:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.372+0000 7f5ad0def640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d4f2949a0 0x564d4f28f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.372+0000 7f5ad0def640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d4f2949a0 0x564d4f28f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60456/0 (socket says 172.21.15.120:60456) 2024-09-19T09:09:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.372+0000 7f5ad0def640 1 -- 172.21.15.120:0/295547084 learned_addr learned my addr 172.21.15.120:0/295547084 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.372+0000 7f5acbfff640 1 --2- 172.21.15.120:0/295547084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d4f2952e0 0x564d4f2a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.372+0000 7f5ad15f0640 1 --2- 172.21.15.120:0/295547084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d4f29a370 0x564d4f291400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.373+0000 7f5ad0def640 1 -- 172.21.15.120:0/295547084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d4f29a370 msgr2=0x564d4f291400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.373+0000 7f5ad0def640 1 --2- 172.21.15.120:0/295547084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d4f29a370 0x564d4f291400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.373+0000 7f5ad0def640 1 -- 172.21.15.120:0/295547084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d4f2952e0 msgr2=0x564d4f2a6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.373+0000 7f5ad0def640 1 --2- 172.21.15.120:0/295547084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d4f2952e0 0x564d4f2a6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.373+0000 7f5ad0def640 1 -- 172.21.15.120:0/295547084 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d4f29c760 con 0x564d4f2949a0 2024-09-19T09:09:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.373+0000 7f5ad0def640 1 --2- 172.21.15.120:0/295547084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d4f2949a0 0x564d4f28f0b0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f5ac009c350 tx=0x7f5ac00d3820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38ca4335eb19d58 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.373+0000 7f5ab67fc640 1 -- 172.21.15.120:0/295547084 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac009a500 con 0x564d4f2949a0 2024-09-19T09:09:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.373+0000 7f5ab67fc640 1 -- 172.21.15.120:0/295547084 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ac009cac0 con 0x564d4f2949a0 2024-09-19T09:09:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.373+0000 7f5ab67fc640 1 -- 172.21.15.120:0/295547084 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac009bd20 con 0x564d4f2949a0 2024-09-19T09:09:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.374+0000 7f5ad5df0740 1 -- 172.21.15.120:0/295547084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d4f2949a0 msgr2=0x564d4f28f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.374+0000 7f5ad5df0740 1 --2- 172.21.15.120:0/295547084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d4f2949a0 0x564d4f28f0b0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f5ac009c350 tx=0x7f5ac00d3820 comp rx=0 tx=0).stop 2024-09-19T09:09:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.374+0000 7f5ad5df0740 1 -- 172.21.15.120:0/295547084 shutdown_connections 2024-09-19T09:09:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.374+0000 7f5ad5df0740 1 --2- 172.21.15.120:0/295547084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d4f29a370 0x564d4f291400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.374+0000 7f5ad5df0740 1 --2- 172.21.15.120:0/295547084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d4f2952e0 0x564d4f2a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.374+0000 7f5ad5df0740 1 --2- 172.21.15.120:0/295547084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d4f2949a0 0x564d4f28f0b0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.374+0000 7f5ad5df0740 1 -- 172.21.15.120:0/295547084 >> 172.21.15.120:0/295547084 conn(0x564d4f290010 msgr2=0x564d4f28e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.374+0000 7f5ad5df0740 1 -- 172.21.15.120:0/295547084 shutdown_connections 2024-09-19T09:09:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.374+0000 7f5ad5df0740 1 -- 172.21.15.120:0/295547084 wait complete. 2024-09-19T09:09:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.374+0000 7f5ad5df0740 1 -- start start 2024-09-19T09:09:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.374+0000 7f5ad5df0740 1 Processor -- start 2024-09-19T09:09:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.375+0000 7f5ad5df0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d4f2949a0 0x564d4f2a32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.375+0000 7f5ad5df0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d4f2952e0 0x564d4f2a3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.375+0000 7f5ad5df0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d4f29a370 0x564d4f2a4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.375+0000 7f5ad5df0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc1aa400 con 0x564d4f2949a0 2024-09-19T09:09:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.375+0000 7f5ad5df0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc1a9d30 con 0x564d4f29a370 2024-09-19T09:09:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.375+0000 7f5ad5df0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc1a6f50 con 0x564d4f2952e0 2024-09-19T09:09:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.375+0000 7f5ad5df0740 10 client.0 _ll_register_callbacks cb 0x7f5ab8009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.375+0000 7f5ad0def640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d4f2949a0 0x564d4f2a32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.375+0000 7f5ad0def640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d4f2949a0 0x564d4f2a32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60470/0 (socket says 172.21.15.120:60470) 2024-09-19T09:09:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.375+0000 7f5ad0def640 1 -- 172.21.15.120:0/3976272011 learned_addr learned my addr 172.21.15.120:0/3976272011 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.375+0000 7f5ad15f0640 1 --2- 172.21.15.120:0/3976272011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d4f29a370 0x564d4f2a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.375+0000 7f5acbfff640 1 --2- 172.21.15.120:0/3976272011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d4f2952e0 0x564d4f2a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.375+0000 7f5ad0def640 1 -- 172.21.15.120:0/3976272011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d4f2952e0 msgr2=0x564d4f2a3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.375+0000 7f5ad0def640 1 --2- 172.21.15.120:0/3976272011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d4f2952e0 0x564d4f2a3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.375+0000 7f5ad0def640 1 -- 172.21.15.120:0/3976272011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d4f29a370 msgr2=0x564d4f2a4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.375+0000 7f5ad0def640 1 --2- 172.21.15.120:0/3976272011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d4f29a370 0x564d4f2a4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.375+0000 7f5ad0def640 1 -- 172.21.15.120:0/3976272011 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d4f2a4e30 con 0x564d4f2949a0 2024-09-19T09:09:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.375+0000 7f5acbfff640 1 --2- 172.21.15.120:0/3976272011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d4f2952e0 0x564d4f2a3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:09:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.375+0000 7f5ad0def640 1 --2- 172.21.15.120:0/3976272011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d4f2949a0 0x564d4f2a32d0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00a4b00 tx=0x7f5ac009bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1462a553df1e1cd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.375+0000 7f5aabfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:09:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.375+0000 7f5aabfff640 1 -- 172.21.15.120:0/3976272011 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac00a5030 con 0x564d4f2949a0 2024-09-19T09:09:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.375+0000 7f5aabfff640 1 -- 172.21.15.120:0/3976272011 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f5ac00cd9b0 con 0x564d4f2949a0 2024-09-19T09:09:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.375+0000 7f5ad5df0740 1 -- 172.21.15.120:0/3976272011 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ac009cac0 con 0x564d4f2949a0 2024-09-19T09:09:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.376+0000 7f5aabfff640 1 -- 172.21.15.120:0/3976272011 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5ac0003b10 con 0x564d4f2949a0 2024-09-19T09:09:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.376+0000 7f5ad5df0740 10 client.17859 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.376+0000 7f5ad5df0740 1 -- 172.21.15.120:0/3976272011 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564d4f2a6140 con 0x564d4f2949a0 2024-09-19T09:09:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.376+0000 7f5aabfff640 1 -- 172.21.15.120:0/3976272011 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ac009cac0 con 0x564d4f2949a0 2024-09-19T09:09:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.376+0000 7f5aabfff640 1 -- 172.21.15.120:0/3976272011 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5ac00d3050 con 0x564d4f2949a0 2024-09-19T09:09:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.376+0000 7f5ad5df0740 10 client.17859 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.376+0000 7f5ad5df0740 10 client.17859 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:09:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.376+0000 7f5ad5df0740 20 client.17859 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.376+0000 7f5ad5df0740 1 --2- 172.21.15.120:0/3976272011 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564d4f315630 0x564d4f317a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.376+0000 7f5ad5df0740 4 client.17859 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:09:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.376+0000 7f5ad5df0740 1 -- 172.21.15.120:0/3976272011 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564d4f1ffd70 con 0x564d4f315630 2024-09-19T09:09:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.376+0000 7f5acbfff640 1 --2- 172.21.15.120:0/3976272011 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564d4f315630 0x564d4f317a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.377+0000 7f5aabfff640 10 client.17859 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.377+0000 7f5acbfff640 1 --2- 172.21.15.120:0/3976272011 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564d4f315630 0x564d4f317a20 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=485f6dce40c77861 server_cookie=958f8b68afd76c4f in_seq=0 out_seq=0 2024-09-19T09:09:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.525+0000 7f5aabfff640 1 -- 172.21.15.120:0/3976272011 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564d4f1ffd70 con 0x564d4f315630 2024-09-19T09:09:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.525+0000 7f5aabfff640 10 client.17859 handle_command_reply: tid=0 2024-09-19T09:09:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.525+0000 7f5ad5df0740 1 client.17859 shutdown 2024-09-19T09:09:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.525+0000 7f5ad5df0740 1 -- 172.21.15.120:0/3976272011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d4f2949a0 msgr2=0x564d4f2a32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.525+0000 7f5ad5df0740 1 --2- 172.21.15.120:0/3976272011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d4f2949a0 0x564d4f2a32d0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00a4b00 tx=0x7f5ac009bd20 comp rx=0 tx=0).stop 2024-09-19T09:09:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.525+0000 7f5ad5df0740 1 -- 172.21.15.120:0/3976272011 shutdown_connections 2024-09-19T09:09:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.525+0000 7f5ad5df0740 1 --2- 172.21.15.120:0/3976272011 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564d4f315630 0x564d4f317a20 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.525+0000 7f5ad5df0740 1 --2- 172.21.15.120:0/3976272011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d4f29a370 0x564d4f2a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.525+0000 7f5aabfff640 0 client.17859 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.525+0000 7f5ad5df0740 1 --2- 172.21.15.120:0/3976272011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d4f2952e0 0x564d4f2a3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.525+0000 7f5ad5df0740 1 --2- 172.21.15.120:0/3976272011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d4f2949a0 0x564d4f2a32d0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.525+0000 7f5ad5df0740 1 -- 172.21.15.120:0/3976272011 >> 172.21.15.120:0/3976272011 conn(0x564d4f290010 msgr2=0x564d4f28f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.525+0000 7f5ad5df0740 1 -- 172.21.15.120:0/3976272011 shutdown_connections 2024-09-19T09:09:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.525+0000 7f5ad5df0740 1 -- 172.21.15.120:0/3976272011 wait complete. 2024-09-19T09:09:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.526+0000 7f5ad5df0740 20 client.17859 trim_cache size 0 max 16384 2024-09-19T09:09:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:09:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:09:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:09:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:09:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.526+0000 7f5ad3053640 1 -- 172.21.15.120:0/19527104 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5abc0a32f0 msgr2=0x7f5abc0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.526+0000 7f5ad3053640 1 --2- 172.21.15.120:0/19527104 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5abc0a32f0 0x7f5abc0a57b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4005e30 tx=0x7f5ac40095c0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.526+0000 7f5ad3053640 1 -- 172.21.15.120:0/19527104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5acc100a40 msgr2=0x7f5acc197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.526+0000 7f5ad3053640 1 --2- 172.21.15.120:0/19527104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5acc100a40 0x7f5acc197db0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000e9c0 tx=0x7f5ac000ee80 comp rx=0 tx=0).stop 2024-09-19T09:09:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.526+0000 7f5ad3053640 1 -- 172.21.15.120:0/19527104 shutdown_connections 2024-09-19T09:09:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.526+0000 7f5ad3053640 1 --2- 172.21.15.120:0/19527104 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5abc0a32f0 0x7f5abc0a57b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.526+0000 7f5ad3053640 1 --2- 172.21.15.120:0/19527104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5acc1025f0 0x7f5acc19f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.526+0000 7f5ad3053640 1 --2- 172.21.15.120:0/19527104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5acc101c30 0x7f5acc1982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.526+0000 7f5ad3053640 1 --2- 172.21.15.120:0/19527104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5acc100a40 0x7f5acc197db0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.526+0000 7f5ad3053640 1 -- 172.21.15.120:0/19527104 >> 172.21.15.120:0/19527104 conn(0x7f5acc0fc1e0 msgr2=0x7f5acc104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.526+0000 7f5ad3053640 1 -- 172.21.15.120:0/19527104 shutdown_connections 2024-09-19T09:09:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:39.526+0000 7f5ad3053640 1 -- 172.21.15.120:0/19527104 wait complete. 2024-09-19T09:09:40.054 INFO:tasks.workunit.client.0.smithi120.stdout: 1 624116 10.01 MB/sec execute 215 sec latency 93.280 ms 2024-09-19T09:09:41.054 INFO:tasks.workunit.client.0.smithi120.stdout: 1 626260 10.02 MB/sec execute 216 sec latency 39.953 ms 2024-09-19T09:09:41.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:40 smithi196 ceph-mon[35150]: pgmap v546: 97 pgs: 97 active+clean; 49 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.4 MiB/s wr, 472 op/s 2024-09-19T09:09:41.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:40 smithi153 ceph-mon[35071]: pgmap v546: 97 pgs: 97 active+clean; 49 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.4 MiB/s wr, 472 op/s 2024-09-19T09:09:41.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:40 smithi120 ceph-mon[31150]: pgmap v546: 97 pgs: 97 active+clean; 49 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.4 MiB/s wr, 472 op/s 2024-09-19T09:09:42.054 INFO:tasks.workunit.client.0.smithi120.stdout: 1 629193 10.06 MB/sec execute 217 sec latency 7.577 ms 2024-09-19T09:09:43.054 INFO:tasks.workunit.client.0.smithi120.stdout: 1 632242 10.09 MB/sec execute 218 sec latency 15.984 ms 2024-09-19T09:09:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:42 smithi196 ceph-mon[35150]: pgmap v547: 97 pgs: 97 active+clean; 44 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 452 op/s 2024-09-19T09:09:43.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:42 smithi153 ceph-mon[35071]: pgmap v547: 97 pgs: 97 active+clean; 44 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 452 op/s 2024-09-19T09:09:43.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:42 smithi120 ceph-mon[31150]: pgmap v547: 97 pgs: 97 active+clean; 44 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 452 op/s 2024-09-19T09:09:44.054 INFO:tasks.workunit.client.0.smithi120.stdout: 1 634774 10.09 MB/sec execute 219 sec latency 14.996 ms 2024-09-19T09:09:45.054 INFO:tasks.workunit.client.0.smithi120.stdout: 1 637239 10.12 MB/sec execute 220 sec latency 11.142 ms 2024-09-19T09:09:45.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:44 smithi196 ceph-mon[35150]: pgmap v548: 97 pgs: 97 active+clean; 40 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.1 MiB/s wr, 426 op/s 2024-09-19T09:09:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:44 smithi153 ceph-mon[35071]: pgmap v548: 97 pgs: 97 active+clean; 40 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.1 MiB/s wr, 426 op/s 2024-09-19T09:09:45.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:44 smithi120 ceph-mon[31150]: pgmap v548: 97 pgs: 97 active+clean; 40 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.1 MiB/s wr, 426 op/s 2024-09-19T09:09:46.054 INFO:tasks.workunit.client.0.smithi120.stdout: 1 639688 10.13 MB/sec execute 221 sec latency 81.263 ms 2024-09-19T09:09:47.054 INFO:tasks.workunit.client.0.smithi120.stdout: 1 642895 10.17 MB/sec execute 222 sec latency 10.875 ms 2024-09-19T09:09:47.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:46 smithi196 ceph-mon[35150]: pgmap v549: 97 pgs: 97 active+clean; 44 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 459 op/s 2024-09-19T09:09:47.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:46 smithi153 ceph-mon[35071]: pgmap v549: 97 pgs: 97 active+clean; 44 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 459 op/s 2024-09-19T09:09:47.190 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:46 smithi120 ceph-mon[31150]: pgmap v549: 97 pgs: 97 active+clean; 44 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 459 op/s 2024-09-19T09:09:48.054 INFO:tasks.workunit.client.0.smithi120.stdout: 1 645693 10.18 MB/sec execute 223 sec latency 10.616 ms 2024-09-19T09:09:49.054 INFO:tasks.workunit.client.0.smithi120.stdout: 1 648176 10.19 MB/sec execute 224 sec latency 12.424 ms 2024-09-19T09:09:49.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:48 smithi196 ceph-mon[35150]: pgmap v550: 97 pgs: 97 active+clean; 44 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 423 op/s 2024-09-19T09:09:49.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:48 smithi153 ceph-mon[35071]: pgmap v550: 97 pgs: 97 active+clean; 44 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 423 op/s 2024-09-19T09:09:49.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:48 smithi120 ceph-mon[31150]: pgmap v550: 97 pgs: 97 active+clean; 44 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 423 op/s 2024-09-19T09:09:50.054 INFO:tasks.workunit.client.0.smithi120.stdout: 1 650775 10.21 MB/sec execute 225 sec latency 20.999 ms 2024-09-19T09:09:51.054 INFO:tasks.workunit.client.0.smithi120.stdout: 1 653681 10.25 MB/sec execute 226 sec latency 55.595 ms 2024-09-19T09:09:51.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:50 smithi196 ceph-mon[35150]: pgmap v551: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.1 MiB/s wr, 453 op/s 2024-09-19T09:09:51.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:50 smithi153 ceph-mon[35071]: pgmap v551: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.1 MiB/s wr, 453 op/s 2024-09-19T09:09:51.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:50 smithi120 ceph-mon[31150]: pgmap v551: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.1 MiB/s wr, 453 op/s 2024-09-19T09:09:52.054 INFO:tasks.workunit.client.0.smithi120.stdout: 1 656432 10.26 MB/sec execute 227 sec latency 10.891 ms 2024-09-19T09:09:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.613+0000 7f6245a82640 1 -- 172.21.15.120:0/1284293555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62401025f0 msgr2=0x7f6240108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.613+0000 7f6245a82640 1 --2- 172.21.15.120:0/1284293555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62401025f0 0x7f6240108e80 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f6238009ca0 tx=0x7f623802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.613+0000 7f6245a82640 1 -- 172.21.15.120:0/1284293555 shutdown_connections 2024-09-19T09:09:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.613+0000 7f6245a82640 1 --2- 172.21.15.120:0/1284293555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62401025f0 0x7f6240108e80 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.613+0000 7f6245a82640 1 --2- 172.21.15.120:0/1284293555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6240101c30 0x7f62401020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.613+0000 7f6245a82640 1 --2- 172.21.15.120:0/1284293555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6240100a40 0x7f6240100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.613+0000 7f6245a82640 1 -- 172.21.15.120:0/1284293555 >> 172.21.15.120:0/1284293555 conn(0x7f62400fc1e0 msgr2=0x7f62400fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.613+0000 7f6245a82640 1 -- 172.21.15.120:0/1284293555 shutdown_connections 2024-09-19T09:09:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.613+0000 7f6245a82640 1 -- 172.21.15.120:0/1284293555 wait complete. 2024-09-19T09:09:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.614+0000 7f6245a82640 1 Processor -- start 2024-09-19T09:09:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.614+0000 7f6245a82640 1 -- start start 2024-09-19T09:09:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.614+0000 7f6245a82640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6240100a40 0x7f624006a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.614+0000 7f6245a82640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6240101c30 0x7f624019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.614+0000 7f6245a82640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62401025f0 0x7f624019f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.614+0000 7f6245a82640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f624010c6c0 con 0x7f62401025f0 2024-09-19T09:09:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.614+0000 7f6245a82640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f624010c540 con 0x7f6240100a40 2024-09-19T09:09:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.614+0000 7f6245a82640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f624010c840 con 0x7f6240101c30 2024-09-19T09:09:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.614+0000 7f623ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62401025f0 0x7f624019f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.614+0000 7f623ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62401025f0 0x7f624019f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56788/0 (socket says 172.21.15.120:56788) 2024-09-19T09:09:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.614+0000 7f623ffff640 1 -- 172.21.15.120:0/4000174496 learned_addr learned my addr 172.21.15.120:0/4000174496 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.614+0000 7f623f7fe640 1 --2- 172.21.15.120:0/4000174496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6240100a40 0x7f624006a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.614+0000 7f623effd640 1 --2- 172.21.15.120:0/4000174496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6240101c30 0x7f624019cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.614+0000 7f623ffff640 1 -- 172.21.15.120:0/4000174496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6240101c30 msgr2=0x7f624019cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.614+0000 7f623ffff640 1 --2- 172.21.15.120:0/4000174496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6240101c30 0x7f624019cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.614+0000 7f623ffff640 1 -- 172.21.15.120:0/4000174496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6240100a40 msgr2=0x7f624006a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.614+0000 7f623ffff640 1 --2- 172.21.15.120:0/4000174496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6240100a40 0x7f624006a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.614+0000 7f623ffff640 1 -- 172.21.15.120:0/4000174496 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f624019fa10 con 0x7f62401025f0 2024-09-19T09:09:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.615+0000 7f623ffff640 1 --2- 172.21.15.120:0/4000174496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62401025f0 0x7f624019f100 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f6238031ae0 tx=0x7f6238031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.615+0000 7f623cff9640 1 -- 172.21.15.120:0/4000174496 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6238003c30 con 0x7f62401025f0 2024-09-19T09:09:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.615+0000 7f6245a82640 1 -- 172.21.15.120:0/4000174496 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f624019fc40 con 0x7f62401025f0 2024-09-19T09:09:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.615+0000 7f623cff9640 1 -- 172.21.15.120:0/4000174496 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6238003dd0 con 0x7f62401025f0 2024-09-19T09:09:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.615+0000 7f623cff9640 1 -- 172.21.15.120:0/4000174496 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623802fcb0 con 0x7f62401025f0 2024-09-19T09:09:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.615+0000 7f6245a82640 1 -- 172.21.15.120:0/4000174496 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62401a0190 con 0x7f62401025f0 2024-09-19T09:09:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.618+0000 7f623cff9640 1 -- 172.21.15.120:0/4000174496 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f6238004050 con 0x7f62401025f0 2024-09-19T09:09:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.618+0000 7f6248822740 1 Processor -- start 2024-09-19T09:09:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.618+0000 7f6248822740 1 -- start start 2024-09-19T09:09:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.618+0000 7f6248822740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56196946a9a0 0x56196946ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.618+0000 7f6248822740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56196946b2e0 0x56196946fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.618+0000 7f6248822740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561969470370 0x561969472830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.618+0000 7f6248822740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5619693ce2f0 con 0x56196946b2e0 2024-09-19T09:09:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.618+0000 7f6248822740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561969376270 con 0x56196946a9a0 2024-09-19T09:09:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.618+0000 7f6248822740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5619693ca2b0 con 0x561969470370 2024-09-19T09:09:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.618+0000 7f623f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56196946b2e0 0x56196946fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.618+0000 7f623f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56196946b2e0 0x56196946fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56792/0 (socket says 172.21.15.120:56792) 2024-09-19T09:09:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.618+0000 7f623f7fe640 1 -- 172.21.15.120:0/3141149648 learned_addr learned my addr 172.21.15.120:0/3141149648 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.618+0000 7f623effd640 1 --2- 172.21.15.120:0/3141149648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56196946a9a0 0x56196946ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.618+0000 7f623effd640 1 --2- 172.21.15.120:0/3141149648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561969470370 0x561969472830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.618+0000 7f623f7fe640 1 -- 172.21.15.120:0/3141149648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561969470370 msgr2=0x561969472830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.618+0000 7f623f7fe640 1 --2- 172.21.15.120:0/3141149648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561969470370 0x561969472830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.618+0000 7f623f7fe640 1 -- 172.21.15.120:0/3141149648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56196946a9a0 msgr2=0x56196946ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.618+0000 7f623f7fe640 1 --2- 172.21.15.120:0/3141149648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56196946a9a0 0x56196946ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.618+0000 7f623f7fe640 1 -- 172.21.15.120:0/3141149648 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561969375340 con 0x56196946b2e0 2024-09-19T09:09:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.618+0000 7f623cff9640 1 --2- 172.21.15.120:0/4000174496 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f62280a32f0 0x7f62280a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.619+0000 7f623f7fe640 1 --2- 172.21.15.120:0/3141149648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56196946b2e0 0x56196946fda0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f623000bcb0 tx=0x7f62300107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9140ac4714dc820f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.619+0000 7f623effd640 1 --2- 172.21.15.120:0/4000174496 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f62280a32f0 0x7f62280a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.619+0000 7f623cff9640 1 -- 172.21.15.120:0/4000174496 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f62380f0b30 con 0x7f62401025f0 2024-09-19T09:09:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.619+0000 7f6225ffb640 1 -- 172.21.15.120:0/3141149648 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6230005020 con 0x56196946b2e0 2024-09-19T09:09:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.619+0000 7f6225ffb640 1 -- 172.21.15.120:0/3141149648 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62300047a0 con 0x56196946b2e0 2024-09-19T09:09:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.619+0000 7f6225ffb640 1 -- 172.21.15.120:0/3141149648 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62300117c0 con 0x56196946b2e0 2024-09-19T09:09:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.619+0000 7f6248822740 1 -- 172.21.15.120:0/3141149648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56196946b2e0 msgr2=0x56196946fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.619+0000 7f6248822740 1 --2- 172.21.15.120:0/3141149648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56196946b2e0 0x56196946fda0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f623000bcb0 tx=0x7f62300107a0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.619+0000 7f623effd640 1 --2- 172.21.15.120:0/4000174496 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f62280a32f0 0x7f62280a57b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6234009730 tx=0x7f62340096c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.620+0000 7f6248822740 1 -- 172.21.15.120:0/3141149648 shutdown_connections 2024-09-19T09:09:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.620+0000 7f6248822740 1 --2- 172.21.15.120:0/3141149648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561969470370 0x561969472830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.620+0000 7f6248822740 1 --2- 172.21.15.120:0/3141149648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56196946b2e0 0x56196946fda0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.620+0000 7f6248822740 1 --2- 172.21.15.120:0/3141149648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56196946a9a0 0x56196946ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.620+0000 7f6248822740 1 -- 172.21.15.120:0/3141149648 >> 172.21.15.120:0/3141149648 conn(0x561969466010 msgr2=0x561969466410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.620+0000 7f6248822740 1 -- 172.21.15.120:0/3141149648 shutdown_connections 2024-09-19T09:09:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.620+0000 7f6248822740 1 -- 172.21.15.120:0/3141149648 wait complete. 2024-09-19T09:09:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.620+0000 7f6248822740 1 -- start start 2024-09-19T09:09:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.620+0000 7f6248822740 1 Processor -- start 2024-09-19T09:09:52.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.620+0000 7f6248822740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56196946a9a0 0x561969474b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:52.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.620+0000 7f6248822740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56196946b2e0 0x5619694750d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:52.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.620+0000 7f6248822740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561969470370 0x56196947c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:52.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.620+0000 7f6248822740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561969473340 con 0x56196946b2e0 2024-09-19T09:09:52.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.620+0000 7f6248822740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561969357290 con 0x561969470370 2024-09-19T09:09:52.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.620+0000 7f6248822740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5619694734c0 con 0x56196946a9a0 2024-09-19T09:09:52.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.620+0000 7f6248822740 10 client.0 _ll_register_callbacks cb 0x561969388140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.620+0000 7f623effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56196946b2e0 0x5619694750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.621+0000 7f623f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56196946a9a0 0x561969474b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.621+0000 7f623effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56196946b2e0 0x5619694750d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56798/0 (socket says 172.21.15.120:56798) 2024-09-19T09:09:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.621+0000 7f623effd640 1 -- 172.21.15.120:0/1182443065 learned_addr learned my addr 172.21.15.120:0/1182443065 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.621+0000 7f623ffff640 1 --2- 172.21.15.120:0/1182443065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561969470370 0x56196947c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.621+0000 7f623effd640 1 -- 172.21.15.120:0/1182443065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56196946a9a0 msgr2=0x561969474b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.621+0000 7f623effd640 1 --2- 172.21.15.120:0/1182443065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56196946a9a0 0x561969474b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.621+0000 7f623effd640 1 -- 172.21.15.120:0/1182443065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561969470370 msgr2=0x56196947c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.621+0000 7f623effd640 1 --2- 172.21.15.120:0/1182443065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561969470370 0x56196947c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.621+0000 7f623effd640 1 -- 172.21.15.120:0/1182443065 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619690e1210 con 0x56196946b2e0 2024-09-19T09:09:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.621+0000 7f623effd640 1 --2- 172.21.15.120:0/1182443065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56196946b2e0 0x5619694750d0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f62340038a0 tx=0x7f6234005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202ccbad392d0b4f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.621+0000 7f621effd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:09:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.621+0000 7f621effd640 1 -- 172.21.15.120:0/1182443065 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623400fb40 con 0x56196946b2e0 2024-09-19T09:09:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.621+0000 7f621effd640 1 -- 172.21.15.120:0/1182443065 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f623400fce0 con 0x56196946b2e0 2024-09-19T09:09:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.621+0000 7f6248822740 1 -- 172.21.15.120:0/1182443065 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62300047a0 con 0x56196946b2e0 2024-09-19T09:09:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.621+0000 7f621effd640 1 -- 172.21.15.120:0/1182443065 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6234021420 con 0x56196946b2e0 2024-09-19T09:09:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.621+0000 7f6248822740 10 client.17871 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.621+0000 7f6248822740 1 -- 172.21.15.120:0/1182443065 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5619691ad620 con 0x56196946b2e0 2024-09-19T09:09:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.621+0000 7f621effd640 1 -- 172.21.15.120:0/1182443065 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62300047a0 con 0x56196946b2e0 2024-09-19T09:09:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.622+0000 7f621effd640 1 -- 172.21.15.120:0/1182443065 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f623401f640 con 0x56196946b2e0 2024-09-19T09:09:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.622+0000 7f6248822740 10 client.17871 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.622+0000 7f6248822740 10 client.17871 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:09:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.622+0000 7f6248822740 20 client.17871 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.622+0000 7f6248822740 1 --2- 172.21.15.120:0/1182443065 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561969480880 0x561969482c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.622+0000 7f6248822740 4 client.17871 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:09:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.622+0000 7f6248822740 1 -- 172.21.15.120:0/1182443065 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619693dba20 con 0x561969480880 2024-09-19T09:09:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.622+0000 7f623f7fe640 1 --2- 172.21.15.120:0/1182443065 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561969480880 0x561969482c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.622+0000 7f621effd640 10 client.17871 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.622+0000 7f623f7fe640 1 --2- 172.21.15.120:0/1182443065 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561969480880 0x561969482c90 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ccaf82eafad305e server_cookie=e1023602ac4a9d29 in_seq=0 out_seq=0 2024-09-19T09:09:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.624+0000 7f621effd640 1 -- 172.21.15.120:0/1182443065 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5619693dba20 con 0x561969480880 2024-09-19T09:09:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.624+0000 7f621effd640 10 client.17871 handle_command_reply: tid=0 2024-09-19T09:09:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.624+0000 7f6248822740 1 client.17871 shutdown 2024-09-19T09:09:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.624+0000 7f6248822740 1 -- 172.21.15.120:0/1182443065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56196946b2e0 msgr2=0x5619694750d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.624+0000 7f6248822740 1 --2- 172.21.15.120:0/1182443065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56196946b2e0 0x5619694750d0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f62340038a0 tx=0x7f6234005f90 comp rx=0 tx=0).stop 2024-09-19T09:09:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.624+0000 7f6248822740 1 -- 172.21.15.120:0/1182443065 shutdown_connections 2024-09-19T09:09:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.624+0000 7f6248822740 1 --2- 172.21.15.120:0/1182443065 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561969480880 0x561969482c90 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.624+0000 7f6248822740 1 --2- 172.21.15.120:0/1182443065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561969470370 0x56196947c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.624+0000 7f6248822740 1 --2- 172.21.15.120:0/1182443065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56196946b2e0 0x5619694750d0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.624+0000 7f621effd640 0 client.17871 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.624+0000 7f6248822740 1 --2- 172.21.15.120:0/1182443065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56196946a9a0 0x561969474b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.624+0000 7f6248822740 1 -- 172.21.15.120:0/1182443065 >> 172.21.15.120:0/1182443065 conn(0x561969466010 msgr2=0x561969467820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.624+0000 7f6248822740 1 -- 172.21.15.120:0/1182443065 shutdown_connections 2024-09-19T09:09:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.624+0000 7f6248822740 1 -- 172.21.15.120:0/1182443065 wait complete. 2024-09-19T09:09:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.624+0000 7f6248822740 20 client.17871 trim_cache size 0 max 16384 2024-09-19T09:09:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.638+0000 7f6248822740 1 Processor -- start 2024-09-19T09:09:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.638+0000 7f6248822740 1 -- start start 2024-09-19T09:09:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.638+0000 7f6248822740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56196946a9a0 0x561969482c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.638+0000 7f6248822740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56196946b2e0 0x56196947c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.638+0000 7f6248822740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561969470370 0x5619694649a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.638+0000 7f6248822740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f62401a8fe0 con 0x561969470370 2024-09-19T09:09:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.638+0000 7f6248822740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f62401aab20 con 0x56196946a9a0 2024-09-19T09:09:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.638+0000 7f6248822740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f62401a96b0 con 0x56196946b2e0 2024-09-19T09:09:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.638+0000 7f623ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561969470370 0x5619694649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.638+0000 7f623ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561969470370 0x5619694649a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56808/0 (socket says 172.21.15.120:56808) 2024-09-19T09:09:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.638+0000 7f623ffff640 1 -- 172.21.15.120:0/3281640863 learned_addr learned my addr 172.21.15.120:0/3281640863 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.638+0000 7f623effd640 1 --2- 172.21.15.120:0/3281640863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56196946b2e0 0x56196947c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.638+0000 7f623f7fe640 1 --2- 172.21.15.120:0/3281640863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56196946a9a0 0x561969482c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.639+0000 7f623ffff640 1 -- 172.21.15.120:0/3281640863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56196946b2e0 msgr2=0x56196947c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.639+0000 7f623ffff640 1 --2- 172.21.15.120:0/3281640863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56196946b2e0 0x56196947c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.639+0000 7f623ffff640 1 -- 172.21.15.120:0/3281640863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56196946a9a0 msgr2=0x561969482c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.639+0000 7f623ffff640 1 --2- 172.21.15.120:0/3281640863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56196946a9a0 0x561969482c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.639+0000 7f623ffff640 1 -- 172.21.15.120:0/3281640863 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56196947f1d0 con 0x561969470370 2024-09-19T09:09:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.639+0000 7f623f7fe640 1 --2- 172.21.15.120:0/3281640863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56196946a9a0 0x561969482c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:09:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.639+0000 7f623ffff640 1 --2- 172.21.15.120:0/3281640863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561969470370 0x5619694649a0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f62380096a0 tx=0x7f6238033040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f551d9654beb959d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.639+0000 7f62257fa640 1 -- 172.21.15.120:0/3281640863 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62380c1500 con 0x561969470370 2024-09-19T09:09:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.639+0000 7f62257fa640 1 -- 172.21.15.120:0/3281640863 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62380d6070 con 0x561969470370 2024-09-19T09:09:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.639+0000 7f62257fa640 1 -- 172.21.15.120:0/3281640863 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62380dd0c0 con 0x561969470370 2024-09-19T09:09:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.639+0000 7f6248822740 1 -- 172.21.15.120:0/3281640863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561969470370 msgr2=0x5619694649a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.639+0000 7f6248822740 1 --2- 172.21.15.120:0/3281640863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561969470370 0x5619694649a0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f62380096a0 tx=0x7f6238033040 comp rx=0 tx=0).stop 2024-09-19T09:09:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.639+0000 7f6248822740 1 -- 172.21.15.120:0/3281640863 shutdown_connections 2024-09-19T09:09:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.640+0000 7f6248822740 1 --2- 172.21.15.120:0/3281640863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561969470370 0x5619694649a0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.640+0000 7f6248822740 1 --2- 172.21.15.120:0/3281640863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56196946b2e0 0x56196947c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.640+0000 7f6248822740 1 --2- 172.21.15.120:0/3281640863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56196946a9a0 0x561969482c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.640+0000 7f6248822740 1 -- 172.21.15.120:0/3281640863 >> 172.21.15.120:0/3281640863 conn(0x561969466010 msgr2=0x56196946c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.640+0000 7f6248822740 1 -- 172.21.15.120:0/3281640863 shutdown_connections 2024-09-19T09:09:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.640+0000 7f6248822740 1 -- 172.21.15.120:0/3281640863 wait complete. 2024-09-19T09:09:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.640+0000 7f6248822740 1 -- start start 2024-09-19T09:09:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.640+0000 7f6248822740 1 Processor -- start 2024-09-19T09:09:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.640+0000 7f6248822740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56196946a9a0 0x5619694792d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.640+0000 7f6248822740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56196946b2e0 0x561969479810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.640+0000 7f6248822740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561969470370 0x56196947a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.640+0000 7f6248822740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f62401aa450 con 0x561969470370 2024-09-19T09:09:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.640+0000 7f6248822740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f62401a9d80 con 0x56196946b2e0 2024-09-19T09:09:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.640+0000 7f6248822740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f62401a6fa0 con 0x56196946a9a0 2024-09-19T09:09:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.640+0000 7f6248822740 10 client.0 _ll_register_callbacks cb 0x7f6234009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:09:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.640+0000 7f623ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561969470370 0x56196947a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.640+0000 7f623effd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56196946b2e0 0x561969479810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.640+0000 7f623ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561969470370 0x56196947a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56810/0 (socket says 172.21.15.120:56810) 2024-09-19T09:09:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.640+0000 7f623ffff640 1 -- 172.21.15.120:0/700647982 learned_addr learned my addr 172.21.15.120:0/700647982 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:09:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.641+0000 7f623ffff640 1 -- 172.21.15.120:0/700647982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56196946a9a0 msgr2=0x5619694792d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.641+0000 7f623ffff640 1 --2- 172.21.15.120:0/700647982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56196946a9a0 0x5619694792d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.641+0000 7f623ffff640 1 -- 172.21.15.120:0/700647982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56196946b2e0 msgr2=0x561969479810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:09:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.641+0000 7f623ffff640 1 --2- 172.21.15.120:0/700647982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56196946b2e0 0x561969479810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.641+0000 7f623ffff640 1 -- 172.21.15.120:0/700647982 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56196947ae30 con 0x561969470370 2024-09-19T09:09:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.641+0000 7f623ffff640 1 --2- 172.21.15.120:0/700647982 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561969470370 0x56196947a820 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f62380bd0e0 tx=0x7f62380ee5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d500447eb5481b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:09:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.641+0000 7f621effd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:09:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.641+0000 7f621effd640 1 -- 172.21.15.120:0/700647982 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62380c5dc0 con 0x561969470370 2024-09-19T09:09:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.641+0000 7f621effd640 1 -- 172.21.15.120:0/700647982 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f62380bb450 con 0x561969470370 2024-09-19T09:09:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.641+0000 7f6248822740 1 -- 172.21.15.120:0/700647982 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62380d6070 con 0x561969470370 2024-09-19T09:09:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.641+0000 7f621effd640 1 -- 172.21.15.120:0/700647982 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f62380c1500 con 0x561969470370 2024-09-19T09:09:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.641+0000 7f6248822740 10 client.17877 fetch_fsmap learned FSMap version 30 2024-09-19T09:09:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.641+0000 7f6248822740 1 -- 172.21.15.120:0/700647982 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56196947c140 con 0x561969470370 2024-09-19T09:09:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.641+0000 7f621effd640 1 -- 172.21.15.120:0/700647982 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62380d6070 con 0x561969470370 2024-09-19T09:09:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.641+0000 7f621effd640 1 -- 172.21.15.120:0/700647982 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f62380a4030 con 0x561969470370 2024-09-19T09:09:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.641+0000 7f6248822740 10 client.17877 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:09:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.641+0000 7f6248822740 10 client.17877 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:09:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.641+0000 7f6248822740 20 client.17877 populate_metadata read hostname 'smithi120' 2024-09-19T09:09:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.641+0000 7f6248822740 1 --2- 172.21.15.120:0/700647982 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5619694eb9b0 0x5619694eddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:09:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.641+0000 7f6248822740 4 client.17877 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:09:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.641+0000 7f6248822740 1 -- 172.21.15.120:0/700647982 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5619693d5d70 con 0x5619694eb9b0 2024-09-19T09:09:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.642+0000 7f623f7fe640 1 --2- 172.21.15.120:0/700647982 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5619694eb9b0 0x5619694eddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:09:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.642+0000 7f621effd640 10 client.17877 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.642+0000 7f623f7fe640 1 --2- 172.21.15.120:0/700647982 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5619694eb9b0 0x5619694eddd0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7f1af24987fbc0b server_cookie=80e9b64cdbda2763 in_seq=0 out_seq=0 2024-09-19T09:09:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.825+0000 7f621effd640 1 -- 172.21.15.120:0/700647982 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5619693d5d70 con 0x5619694eb9b0 2024-09-19T09:09:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.825+0000 7f621effd640 10 client.17877 handle_command_reply: tid=0 2024-09-19T09:09:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.825+0000 7f6248822740 1 client.17877 shutdown 2024-09-19T09:09:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.825+0000 7f6248822740 1 -- 172.21.15.120:0/700647982 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561969470370 msgr2=0x56196947a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.825+0000 7f6248822740 1 --2- 172.21.15.120:0/700647982 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561969470370 0x56196947a820 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f62380bd0e0 tx=0x7f62380ee5d0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.825+0000 7f6248822740 1 -- 172.21.15.120:0/700647982 shutdown_connections 2024-09-19T09:09:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.825+0000 7f6248822740 1 --2- 172.21.15.120:0/700647982 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5619694eb9b0 0x5619694eddd0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.825+0000 7f621effd640 0 client.17877 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:09:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.825+0000 7f6248822740 1 --2- 172.21.15.120:0/700647982 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561969470370 0x56196947a820 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.825+0000 7f6248822740 1 --2- 172.21.15.120:0/700647982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56196946b2e0 0x561969479810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.825+0000 7f6248822740 1 --2- 172.21.15.120:0/700647982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56196946a9a0 0x5619694792d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.825+0000 7f6248822740 1 -- 172.21.15.120:0/700647982 >> 172.21.15.120:0/700647982 conn(0x561969466010 msgr2=0x56196946c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.825+0000 7f6248822740 1 -- 172.21.15.120:0/700647982 shutdown_connections 2024-09-19T09:09:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.825+0000 7f6248822740 1 -- 172.21.15.120:0/700647982 wait complete. 2024-09-19T09:09:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.825+0000 7f6248822740 20 client.17877 trim_cache size 0 max 16384 2024-09-19T09:09:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:09:52.828 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:09:52.828 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:09:52.828 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:09:52.828 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.826+0000 7f6245a82640 1 -- 172.21.15.120:0/4000174496 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f62280a32f0 msgr2=0x7f62280a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.826+0000 7f6245a82640 1 --2- 172.21.15.120:0/4000174496 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f62280a32f0 0x7f62280a57b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6234009730 tx=0x7f62340096c0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.826+0000 7f6245a82640 1 -- 172.21.15.120:0/4000174496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62401025f0 msgr2=0x7f624019f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:09:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.826+0000 7f6245a82640 1 --2- 172.21.15.120:0/4000174496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62401025f0 0x7f624019f100 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f6238031ae0 tx=0x7f6238031ee0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.826+0000 7f6245a82640 1 -- 172.21.15.120:0/4000174496 shutdown_connections 2024-09-19T09:09:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.826+0000 7f6245a82640 1 --2- 172.21.15.120:0/4000174496 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f62280a32f0 0x7f62280a57b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.826+0000 7f6245a82640 1 --2- 172.21.15.120:0/4000174496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62401025f0 0x7f624019f100 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.826+0000 7f6245a82640 1 --2- 172.21.15.120:0/4000174496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6240101c30 0x7f624019cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.826+0000 7f6245a82640 1 --2- 172.21.15.120:0/4000174496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6240100a40 0x7f624006a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:09:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.826+0000 7f6245a82640 1 -- 172.21.15.120:0/4000174496 >> 172.21.15.120:0/4000174496 conn(0x7f62400fc1e0 msgr2=0x7f6240104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:09:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.826+0000 7f6245a82640 1 -- 172.21.15.120:0/4000174496 shutdown_connections 2024-09-19T09:09:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:09:52.826+0000 7f6245a82640 1 -- 172.21.15.120:0/4000174496 wait complete. 2024-09-19T09:09:53.055 INFO:tasks.workunit.client.0.smithi120.stdout: 1 657792 10.24 MB/sec execute 228 sec latency 106.302 ms 2024-09-19T09:09:53.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:52 smithi196 ceph-mon[35150]: pgmap v552: 97 pgs: 97 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.1 MiB/s wr, 408 op/s 2024-09-19T09:09:53.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:52 smithi153 ceph-mon[35071]: pgmap v552: 97 pgs: 97 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.1 MiB/s wr, 408 op/s 2024-09-19T09:09:53.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:52 smithi120 ceph-mon[31150]: pgmap v552: 97 pgs: 97 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.1 MiB/s wr, 408 op/s 2024-09-19T09:09:54.054 INFO:tasks.workunit.client.0.smithi120.stdout: 1 660930 10.28 MB/sec execute 229 sec latency 9.139 ms 2024-09-19T09:09:55.054 INFO:tasks.workunit.client.0.smithi120.stdout: 1 664075 10.31 MB/sec execute 230 sec latency 11.405 ms 2024-09-19T09:09:55.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:54 smithi196 ceph-mon[35150]: pgmap v553: 97 pgs: 97 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.4 MiB/s wr, 454 op/s 2024-09-19T09:09:55.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:54 smithi153 ceph-mon[35071]: pgmap v553: 97 pgs: 97 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.4 MiB/s wr, 454 op/s 2024-09-19T09:09:55.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:54 smithi120 ceph-mon[31150]: pgmap v553: 97 pgs: 97 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.4 MiB/s wr, 454 op/s 2024-09-19T09:09:56.054 INFO:tasks.workunit.client.0.smithi120.stdout: 1 665899 10.30 MB/sec execute 231 sec latency 84.338 ms 2024-09-19T09:09:57.055 INFO:tasks.workunit.client.0.smithi120.stdout: 1 668513 10.32 MB/sec execute 232 sec latency 11.041 ms 2024-09-19T09:09:57.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:56 smithi196 ceph-mon[35150]: pgmap v554: 97 pgs: 97 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 381 op/s 2024-09-19T09:09:57.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:56 smithi153 ceph-mon[35071]: pgmap v554: 97 pgs: 97 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 381 op/s 2024-09-19T09:09:57.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:56 smithi120 ceph-mon[31150]: pgmap v554: 97 pgs: 97 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 381 op/s 2024-09-19T09:09:58.055 INFO:tasks.workunit.client.0.smithi120.stdout: 1 671574 10.35 MB/sec execute 233 sec latency 51.879 ms 2024-09-19T09:09:59.055 INFO:tasks.workunit.client.0.smithi120.stdout: 1 674774 10.38 MB/sec execute 234 sec latency 12.133 ms 2024-09-19T09:09:59.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:09:58 smithi120 ceph-mon[31150]: pgmap v555: 97 pgs: 97 active+clean; 41 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.8 MiB/s wr, 415 op/s 2024-09-19T09:09:59.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:09:58 smithi196 ceph-mon[35150]: pgmap v555: 97 pgs: 97 active+clean; 41 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.8 MiB/s wr, 415 op/s 2024-09-19T09:09:59.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:09:58 smithi153 ceph-mon[35071]: pgmap v555: 97 pgs: 97 active+clean; 41 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.8 MiB/s wr, 415 op/s 2024-09-19T09:10:00.055 INFO:tasks.workunit.client.0.smithi120.stdout: 1 677373 10.38 MB/sec execute 235 sec latency 11.017 ms 2024-09-19T09:10:01.055 INFO:tasks.workunit.client.0.smithi120.stdout: 1 679382 10.39 MB/sec execute 236 sec latency 69.533 ms 2024-09-19T09:10:01.122 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:00 smithi153 ceph-mon[35071]: pgmap v556: 97 pgs: 97 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.2 MiB/s wr, 461 op/s 2024-09-19T09:10:01.122 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:00 smithi153 ceph-mon[35071]: overall HEALTH_OK 2024-09-19T09:10:01.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:00 smithi196 ceph-mon[35150]: pgmap v556: 97 pgs: 97 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.2 MiB/s wr, 461 op/s 2024-09-19T09:10:01.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:00 smithi196 ceph-mon[35150]: overall HEALTH_OK 2024-09-19T09:10:01.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:00 smithi120 ceph-mon[31150]: pgmap v556: 97 pgs: 97 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.2 MiB/s wr, 461 op/s 2024-09-19T09:10:01.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:00 smithi120 ceph-mon[31150]: overall HEALTH_OK 2024-09-19T09:10:02.055 INFO:tasks.workunit.client.0.smithi120.stdout: 1 682326 10.42 MB/sec execute 237 sec latency 19.467 ms 2024-09-19T09:10:03.055 INFO:tasks.workunit.client.0.smithi120.stdout: 1 685508 10.45 MB/sec execute 238 sec latency 15.278 ms 2024-09-19T09:10:03.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:02 smithi196 ceph-mon[35150]: pgmap v557: 97 pgs: 97 active+clean; 44 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 429 op/s 2024-09-19T09:10:03.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:02 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:10:03.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:02 smithi153 ceph-mon[35071]: pgmap v557: 97 pgs: 97 active+clean; 44 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 429 op/s 2024-09-19T09:10:03.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:02 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:10:03.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:02 smithi120 ceph-mon[31150]: pgmap v557: 97 pgs: 97 active+clean; 44 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 429 op/s 2024-09-19T09:10:03.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:02 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:10:04.055 INFO:tasks.workunit.client.0.smithi120.stdout: 1 688293 10.46 MB/sec execute 239 sec latency 17.921 ms 2024-09-19T09:10:04.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:03 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:03 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:03 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:03 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:03 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:03 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:03 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:03 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:03 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:03 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:03 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:03 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:03 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:10:04.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:03 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:10:04.188 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:03 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:10:04.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:03 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:03 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:03 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:03 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:03 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:03 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:03 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:03 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:03 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:03 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:03 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:03 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:03 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:10:04.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:03 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:10:04.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:03 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:10:04.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:03 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:03 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:03 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:03 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:03 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:03 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:03 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:03 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:03 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:03 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:03 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:03 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:10:04.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:03 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:10:04.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:03 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:10:04.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:03 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:10:04.377 INFO:tasks.quiescer.fs.[cephfs]:Total duration (51.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T09:10:04.377 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.75 2024-09-19T09:10:04.378 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:10:04.635 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:10:05.055 INFO:tasks.workunit.client.0.smithi120.stdout: 1 690643 10.47 MB/sec execute 240 sec latency 16.711 ms 2024-09-19T09:10:05.067 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:04 smithi120 ceph-mon[31150]: Adjusting osd_memory_target on smithi196 to 176.0M 2024-09-19T09:10:05.067 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:04 smithi120 ceph-mon[31150]: Adjusting osd_memory_target on smithi120 to 176.0M 2024-09-19T09:10:05.067 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:04 smithi120 ceph-mon[31150]: Adjusting osd_memory_target on smithi153 to 176.0M 2024-09-19T09:10:05.067 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:04 smithi120 ceph-mon[31150]: Unable to set osd_memory_target on smithi196 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-19T09:10:05.067 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:04 smithi120 ceph-mon[31150]: Unable to set osd_memory_target on smithi120 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-19T09:10:05.067 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:04 smithi120 ceph-mon[31150]: Unable to set osd_memory_target on smithi153 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-19T09:10:05.068 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:04 smithi120 ceph-mon[31150]: pgmap v558: 97 pgs: 97 active+clean; 49 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 450 op/s 2024-09-19T09:10:05.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.149+0000 7f29f93a0640 1 -- 172.21.15.120:0/2681740562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29f4075110 msgr2=0x7f29f410dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:05.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.149+0000 7f29f93a0640 1 --2- 172.21.15.120:0/2681740562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29f4075110 0x7f29f410dce0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009900 tx=0x7f29dc02a480 comp rx=0 tx=0).stop 2024-09-19T09:10:05.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.149+0000 7f29f93a0640 1 -- 172.21.15.120:0/2681740562 shutdown_connections 2024-09-19T09:10:05.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.149+0000 7f29f93a0640 1 --2- 172.21.15.120:0/2681740562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29f410e300 0x7f29f4110790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.149+0000 7f29f93a0640 1 --2- 172.21.15.120:0/2681740562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29f4075110 0x7f29f410dce0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.149+0000 7f29f93a0640 1 --2- 172.21.15.120:0/2681740562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29f40747f0 0x7f29f4074bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.149+0000 7f29f93a0640 1 -- 172.21.15.120:0/2681740562 >> 172.21.15.120:0/2681740562 conn(0x7f29f40fd700 msgr2=0x7f29f40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:05.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.149+0000 7f29f93a0640 1 -- 172.21.15.120:0/2681740562 shutdown_connections 2024-09-19T09:10:05.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.149+0000 7f29f93a0640 1 -- 172.21.15.120:0/2681740562 wait complete. 2024-09-19T09:10:05.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.150+0000 7f29f93a0640 1 Processor -- start 2024-09-19T09:10:05.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.150+0000 7f29f93a0640 1 -- start start 2024-09-19T09:10:05.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.150+0000 7f29f93a0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29f40747f0 0x7f29f419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:05.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.150+0000 7f29f93a0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29f4075110 0x7f29f419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:05.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.150+0000 7f29f93a0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29f410e300 0x7f29f419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:05.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.150+0000 7f29f93a0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f29f41140d0 con 0x7f29f40747f0 2024-09-19T09:10:05.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.150+0000 7f29f93a0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f29f4113f50 con 0x7f29f4075110 2024-09-19T09:10:05.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.150+0000 7f29f93a0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f29f4114250 con 0x7f29f410e300 2024-09-19T09:10:05.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.150+0000 7f29f3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29f40747f0 0x7f29f419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:05.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.150+0000 7f29f3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29f40747f0 0x7f29f419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48336/0 (socket says 172.21.15.120:48336) 2024-09-19T09:10:05.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.150+0000 7f29f3fff640 1 -- 172.21.15.120:0/2026209775 learned_addr learned my addr 172.21.15.120:0/2026209775 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:05.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.150+0000 7f29ebfff640 1 --2- 172.21.15.120:0/2026209775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29f4075110 0x7f29f419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:05.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.150+0000 7f29f8b9f640 1 --2- 172.21.15.120:0/2026209775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29f410e300 0x7f29f419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:05.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.150+0000 7f29f3fff640 1 -- 172.21.15.120:0/2026209775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29f410e300 msgr2=0x7f29f419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:05.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.150+0000 7f29f3fff640 1 --2- 172.21.15.120:0/2026209775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29f410e300 0x7f29f419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.150+0000 7f29f3fff640 1 -- 172.21.15.120:0/2026209775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29f4075110 msgr2=0x7f29f419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:05.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.150+0000 7f29f3fff640 1 --2- 172.21.15.120:0/2026209775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29f4075110 0x7f29f419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.150+0000 7f29f3fff640 1 -- 172.21.15.120:0/2026209775 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f419f5e0 con 0x7f29f40747f0 2024-09-19T09:10:05.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.150+0000 7f29f3fff640 1 --2- 172.21.15.120:0/2026209775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29f40747f0 0x7f29f419a630 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f29e400d770 tx=0x7f29e400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:05.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.151+0000 7f29f1ffb640 1 -- 172.21.15.120:0/2026209775 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e4017070 con 0x7f29f40747f0 2024-09-19T09:10:05.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.151+0000 7f29f93a0640 1 -- 172.21.15.120:0/2026209775 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29f419f8d0 con 0x7f29f40747f0 2024-09-19T09:10:05.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.151+0000 7f29f1ffb640 1 -- 172.21.15.120:0/2026209775 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29e4005500 con 0x7f29f40747f0 2024-09-19T09:10:05.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.151+0000 7f29f1ffb640 1 -- 172.21.15.120:0/2026209775 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e4004e00 con 0x7f29f40747f0 2024-09-19T09:10:05.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.151+0000 7f29f93a0640 1 -- 172.21.15.120:0/2026209775 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29f41a71b0 con 0x7f29f40747f0 2024-09-19T09:10:05.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.152+0000 7f29f93a0640 1 -- 172.21.15.120:0/2026209775 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29f4075cd0 con 0x7f29f40747f0 2024-09-19T09:10:05.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.153+0000 7f29f1ffb640 1 -- 172.21.15.120:0/2026209775 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f29e4004fa0 con 0x7f29f40747f0 2024-09-19T09:10:05.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.154+0000 7f29f1ffb640 1 --2- 172.21.15.120:0/2026209775 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f29e00a3240 0x7f29e00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:05.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.154+0000 7f29ebfff640 1 --2- 172.21.15.120:0/2026209775 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f29e00a3240 0x7f29e00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:05.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.154+0000 7f29f1ffb640 1 -- 172.21.15.120:0/2026209775 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f29e40cf710 con 0x7f29f40747f0 2024-09-19T09:10:05.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.154+0000 7f29ebfff640 1 --2- 172.21.15.120:0/2026209775 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f29e00a3240 0x7f29e00a5700 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f29dc004770 tx=0x7f29dc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:05.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.158+0000 7f29f1ffb640 1 -- 172.21.15.120:0/2026209775 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f29e409bde0 con 0x7f29f40747f0 2024-09-19T09:10:05.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:04 smithi196 ceph-mon[35150]: Adjusting osd_memory_target on smithi196 to 176.0M 2024-09-19T09:10:05.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:04 smithi196 ceph-mon[35150]: Adjusting osd_memory_target on smithi120 to 176.0M 2024-09-19T09:10:05.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:04 smithi196 ceph-mon[35150]: Adjusting osd_memory_target on smithi153 to 176.0M 2024-09-19T09:10:05.188 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:04 smithi196 ceph-mon[35150]: Unable to set osd_memory_target on smithi196 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-19T09:10:05.188 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:04 smithi196 ceph-mon[35150]: Unable to set osd_memory_target on smithi120 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-19T09:10:05.188 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:04 smithi196 ceph-mon[35150]: Unable to set osd_memory_target on smithi153 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-19T09:10:05.188 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:04 smithi196 ceph-mon[35150]: pgmap v558: 97 pgs: 97 active+clean; 49 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 450 op/s 2024-09-19T09:10:05.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:04 smithi153 ceph-mon[35071]: Adjusting osd_memory_target on smithi196 to 176.0M 2024-09-19T09:10:05.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:04 smithi153 ceph-mon[35071]: Adjusting osd_memory_target on smithi120 to 176.0M 2024-09-19T09:10:05.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:04 smithi153 ceph-mon[35071]: Adjusting osd_memory_target on smithi153 to 176.0M 2024-09-19T09:10:05.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:04 smithi153 ceph-mon[35071]: Unable to set osd_memory_target on smithi196 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-19T09:10:05.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:04 smithi153 ceph-mon[35071]: Unable to set osd_memory_target on smithi120 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-19T09:10:05.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:04 smithi153 ceph-mon[35071]: Unable to set osd_memory_target on smithi153 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-19T09:10:05.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:04 smithi153 ceph-mon[35071]: pgmap v558: 97 pgs: 97 active+clean; 49 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 450 op/s 2024-09-19T09:10:05.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.393+0000 7f29f93a0640 1 -- 172.21.15.120:0/2026209775 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29f4074bd0 con 0x7f29f40747f0 2024-09-19T09:10:05.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.394+0000 7f29f1ffb640 1 -- 172.21.15.120:0/2026209775 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11985 (secure 0 0 0) 0x7f29e40a00e0 con 0x7f29f40747f0 2024-09-19T09:10:05.396 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:10:05.396 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":30,"btime":"2024-09-19T09:07:14:812065+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:07:14.812046+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26351":{"gid":26351,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/364941779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":364941779},{"type":"v1","addr":"172.21.15.153:6835","nonce":364941779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:10:05.396 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 30 2024-09-19T09:10:05.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.396+0000 7f29f93a0640 1 -- 172.21.15.120:0/2026209775 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f29e00a3240 msgr2=0x7f29e00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:05.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.396+0000 7f29f93a0640 1 --2- 172.21.15.120:0/2026209775 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f29e00a3240 0x7f29e00a5700 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f29dc004770 tx=0x7f29dc0058d0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.396+0000 7f29f93a0640 1 -- 172.21.15.120:0/2026209775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29f40747f0 msgr2=0x7f29f419a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:05.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.396+0000 7f29f93a0640 1 --2- 172.21.15.120:0/2026209775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29f40747f0 0x7f29f419a630 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f29e400d770 tx=0x7f29e400dc30 comp rx=0 tx=0).stop 2024-09-19T09:10:05.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.396+0000 7f29f93a0640 1 -- 172.21.15.120:0/2026209775 shutdown_connections 2024-09-19T09:10:05.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.396+0000 7f29f93a0640 1 --2- 172.21.15.120:0/2026209775 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f29e00a3240 0x7f29e00a5700 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.396+0000 7f29f93a0640 1 --2- 172.21.15.120:0/2026209775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29f410e300 0x7f29f419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.396+0000 7f29f93a0640 1 --2- 172.21.15.120:0/2026209775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29f4075110 0x7f29f419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.396+0000 7f29f93a0640 1 --2- 172.21.15.120:0/2026209775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29f40747f0 0x7f29f419a630 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.396+0000 7f29f93a0640 1 -- 172.21.15.120:0/2026209775 >> 172.21.15.120:0/2026209775 conn(0x7f29f40fd700 msgr2=0x7f29f40fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:05.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.396+0000 7f29f93a0640 1 -- 172.21.15.120:0/2026209775 shutdown_connections 2024-09-19T09:10:05.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:05.396+0000 7f29f93a0640 1 -- 172.21.15.120:0/2026209775 wait complete. 2024-09-19T09:10:05.784 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:10:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.918+0000 7f4fa36f1640 1 -- 172.21.15.120:0/2194663722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f9c100d60 msgr2=0x7f4f9c1011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.918+0000 7f4fa36f1640 1 --2- 172.21.15.120:0/2194663722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f9c100d60 0x7f4f9c1011c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009900 tx=0x7f4f8c02a410 comp rx=0 tx=0).stop 2024-09-19T09:10:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.918+0000 7f4fa36f1640 1 -- 172.21.15.120:0/2194663722 shutdown_connections 2024-09-19T09:10:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.918+0000 7f4fa36f1640 1 --2- 172.21.15.120:0/2194663722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f9c101700 0x7f4f9c101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.918+0000 7f4fa36f1640 1 --2- 172.21.15.120:0/2194663722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f9c100d60 0x7f4f9c1011c0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.918+0000 7f4fa36f1640 1 --2- 172.21.15.120:0/2194663722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f9c100420 0x7f4f9c100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.918+0000 7f4fa36f1640 1 -- 172.21.15.120:0/2194663722 >> 172.21.15.120:0/2194663722 conn(0x7f4f9c0fc1b0 msgr2=0x7f4f9c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.918+0000 7f4fa36f1640 1 -- 172.21.15.120:0/2194663722 shutdown_connections 2024-09-19T09:10:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.918+0000 7f4fa36f1640 1 -- 172.21.15.120:0/2194663722 wait complete. 2024-09-19T09:10:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.919+0000 7f4fa36f1640 1 Processor -- start 2024-09-19T09:10:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.919+0000 7f4fa36f1640 1 -- start start 2024-09-19T09:10:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.919+0000 7f4fa36f1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f9c100420 0x7f4f9c199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.919+0000 7f4fa36f1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f9c100d60 0x7f4f9c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.919+0000 7f4fa36f1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f9c101700 0x7f4f9c1a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.919+0000 7f4fa36f1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c10f940 con 0x7f4f9c101700 2024-09-19T09:10:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.919+0000 7f4fa36f1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c10f7c0 con 0x7f4f9c100420 2024-09-19T09:10:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.919+0000 7f4fa36f1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c10fac0 con 0x7f4f9c100d60 2024-09-19T09:10:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.919+0000 7f4fa148d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f9c100420 0x7f4f9c199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.919+0000 7f4fa1c8e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f9c101700 0x7f4f9c1a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.919+0000 7f4fa148d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f9c100420 0x7f4f9c199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:46724/0 (socket says 172.21.15.120:46724) 2024-09-19T09:10:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.919+0000 7f4fa148d640 1 -- 172.21.15.120:0/3231415007 learned_addr learned my addr 172.21.15.120:0/3231415007 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.919+0000 7f4fa0c8c640 1 --2- 172.21.15.120:0/3231415007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f9c100d60 0x7f4f9c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.919+0000 7f4fa148d640 1 -- 172.21.15.120:0/3231415007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f9c100d60 msgr2=0x7f4f9c19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.919+0000 7f4fa148d640 1 --2- 172.21.15.120:0/3231415007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f9c100d60 0x7f4f9c19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.919+0000 7f4fa148d640 1 -- 172.21.15.120:0/3231415007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f9c101700 msgr2=0x7f4f9c1a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.919+0000 7f4fa148d640 1 --2- 172.21.15.120:0/3231415007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f9c101700 0x7f4f9c1a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.919+0000 7f4fa148d640 1 -- 172.21.15.120:0/3231415007 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f9c1a1b90 con 0x7f4f9c100420 2024-09-19T09:10:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.920+0000 7f4fa148d640 1 --2- 172.21.15.120:0/3231415007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f9c100420 0x7f4f9c199f70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800ea60 tx=0x7f4f9800ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.920+0000 7f4f927fc640 1 -- 172.21.15.120:0/3231415007 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9800cd20 con 0x7f4f9c100420 2024-09-19T09:10:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.920+0000 7f4f927fc640 1 -- 172.21.15.120:0/3231415007 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f980035f0 con 0x7f4f9c100420 2024-09-19T09:10:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.920+0000 7f4fa36f1640 1 -- 172.21.15.120:0/3231415007 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f9c1a1e80 con 0x7f4f9c100420 2024-09-19T09:10:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.920+0000 7f4f927fc640 1 -- 172.21.15.120:0/3231415007 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98010690 con 0x7f4f9c100420 2024-09-19T09:10:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.920+0000 7f4fa36f1640 1 -- 172.21.15.120:0/3231415007 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f9c1a22e0 con 0x7f4f9c100420 2024-09-19T09:10:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.922+0000 7f4f927fc640 1 -- 172.21.15.120:0/3231415007 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f4f980037a0 con 0x7f4f9c100420 2024-09-19T09:10:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.923+0000 7f4fa648c740 1 Processor -- start 2024-09-19T09:10:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.923+0000 7f4fa648c740 1 -- start start 2024-09-19T09:10:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.923+0000 7f4fa648c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a7247ad9a0 0x55a7247adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.923+0000 7f4fa648c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a7247ae2e0 0x55a7247b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.923+0000 7f4fa648c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7247b3370 0x55a7247b5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.923+0000 7f4fa648c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a7247112f0 con 0x55a7247ad9a0 2024-09-19T09:10:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.923+0000 7f4fa648c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a7246b9270 con 0x55a7247ae2e0 2024-09-19T09:10:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.923+0000 7f4fa648c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a72470d2b0 con 0x55a7247b3370 2024-09-19T09:10:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.923+0000 7f4f927fc640 1 --2- 172.21.15.120:0/3231415007 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4f880a32f0 0x7f4f880a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.923+0000 7f4fa1c8e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a7247ad9a0 0x55a7247adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.923+0000 7f4fa148d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a7247ae2e0 0x55a7247b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.923+0000 7f4fa1c8e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a7247ad9a0 0x55a7247adda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48348/0 (socket says 172.21.15.120:48348) 2024-09-19T09:10:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.923+0000 7f4fa0c8c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7247b3370 0x55a7247b5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.923+0000 7f4fa1c8e640 1 -- 172.21.15.120:0/1396184257 learned_addr learned my addr 172.21.15.120:0/1396184257 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.923+0000 7f4fa1c8e640 1 --2- 172.21.15.120:0/3231415007 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4f880a32f0 0x7f4f880a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.923+0000 7f4f927fc640 1 -- 172.21.15.120:0/3231415007 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f4f980cfc50 con 0x7f4f9c100420 2024-09-19T09:10:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.923+0000 7f4fa0c8c640 1 -- 172.21.15.120:0/1396184257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a7247ae2e0 msgr2=0x55a7247b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.923+0000 7f4fa0c8c640 1 --2- 172.21.15.120:0/1396184257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a7247ae2e0 0x55a7247b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.923+0000 7f4fa0c8c640 1 -- 172.21.15.120:0/1396184257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a7247ad9a0 msgr2=0x55a7247adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.923+0000 7f4fa0c8c640 1 --2- 172.21.15.120:0/1396184257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a7247ad9a0 0x55a7247adda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.923+0000 7f4fa0c8c640 1 -- 172.21.15.120:0/1396184257 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7246b8340 con 0x55a7247b3370 2024-09-19T09:10:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.924+0000 7f4fa1c8e640 1 --2- 172.21.15.120:0/3231415007 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4f880a32f0 0x7f4f880a57b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4f94005c40 tx=0x7f4f94009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.924+0000 7f4fa0c8c640 1 --2- 172.21.15.120:0/1396184257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7247b3370 0x55a7247b5830 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c031440 tx=0x7f4f8c005e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de3e6e5d308933fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.924+0000 7f4f877fe640 1 -- 172.21.15.120:0/1396184257 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c040070 con 0x55a7247b3370 2024-09-19T09:10:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.924+0000 7f4f877fe640 1 -- 172.21.15.120:0/1396184257 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f8c004030 con 0x55a7247b3370 2024-09-19T09:10:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.924+0000 7f4f877fe640 1 -- 172.21.15.120:0/1396184257 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c035040 con 0x55a7247b3370 2024-09-19T09:10:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.925+0000 7f4fa648c740 1 -- 172.21.15.120:0/1396184257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7247b3370 msgr2=0x55a7247b5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.925+0000 7f4fa648c740 1 --2- 172.21.15.120:0/1396184257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7247b3370 0x55a7247b5830 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c031440 tx=0x7f4f8c005e00 comp rx=0 tx=0).stop 2024-09-19T09:10:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.925+0000 7f4fa648c740 1 -- 172.21.15.120:0/1396184257 shutdown_connections 2024-09-19T09:10:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.925+0000 7f4fa648c740 1 --2- 172.21.15.120:0/1396184257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7247b3370 0x55a7247b5830 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.925+0000 7f4fa648c740 1 --2- 172.21.15.120:0/1396184257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a7247ae2e0 0x55a7247b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.925+0000 7f4fa648c740 1 --2- 172.21.15.120:0/1396184257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a7247ad9a0 0x55a7247adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.925+0000 7f4fa648c740 1 -- 172.21.15.120:0/1396184257 >> 172.21.15.120:0/1396184257 conn(0x55a7247a9010 msgr2=0x55a7247a9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.925+0000 7f4fa648c740 1 -- 172.21.15.120:0/1396184257 shutdown_connections 2024-09-19T09:10:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.925+0000 7f4fa648c740 1 -- 172.21.15.120:0/1396184257 wait complete. 2024-09-19T09:10:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.925+0000 7f4fa648c740 1 -- start start 2024-09-19T09:10:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.925+0000 7f4fa648c740 1 Processor -- start 2024-09-19T09:10:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.926+0000 7f4fa648c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7247ad9a0 0x55a7247b7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.926+0000 7f4fa648c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a7247ae2e0 0x55a7247b80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.926+0000 7f4fa648c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a7247b3370 0x55a7247bf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.926+0000 7f4fa648c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a7247b6340 con 0x55a7247ae2e0 2024-09-19T09:10:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.926+0000 7f4fa648c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a72469a290 con 0x55a7247b3370 2024-09-19T09:10:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.926+0000 7f4fa648c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a7247b64c0 con 0x55a7247ad9a0 2024-09-19T09:10:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.926+0000 7f4fa648c740 10 client.0 _ll_register_callbacks cb 0x55a7246cb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.926+0000 7f4fa0c8c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a7247ae2e0 0x55a7247b80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.926+0000 7f4fa148d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7247ad9a0 0x55a7247b7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.926+0000 7f4fa1c8e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a7247b3370 0x55a7247bf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.926+0000 7f4fa148d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7247ad9a0 0x55a7247b7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:49012/0 (socket says 172.21.15.120:49012) 2024-09-19T09:10:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.926+0000 7f4fa148d640 1 -- 172.21.15.120:0/3384630661 learned_addr learned my addr 172.21.15.120:0/3384630661 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.926+0000 7f4fa1c8e640 1 -- 172.21.15.120:0/3384630661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7247ad9a0 msgr2=0x55a7247b7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.926+0000 7f4fa1c8e640 1 --2- 172.21.15.120:0/3384630661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7247ad9a0 0x55a7247b7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.926+0000 7f4fa1c8e640 1 -- 172.21.15.120:0/3384630661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a7247ae2e0 msgr2=0x55a7247b80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.926+0000 7f4fa1c8e640 1 --2- 172.21.15.120:0/3384630661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a7247ae2e0 0x55a7247b80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.926+0000 7f4fa1c8e640 1 -- 172.21.15.120:0/3384630661 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a724424210 con 0x55a7247b3370 2024-09-19T09:10:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.926+0000 7f4fa148d640 1 --2- 172.21.15.120:0/3384630661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7247ad9a0 0x55a7247b7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:10:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.926+0000 7f4fa0c8c640 1 --2- 172.21.15.120:0/3384630661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a7247ae2e0 0x55a7247b80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:10:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.926+0000 7f4fa1c8e640 1 --2- 172.21.15.120:0/3384630661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a7247b3370 0x55a7247bf0f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4f94005c10 tx=0x7f4f94004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad7363d92f75a6d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.926+0000 7f4f84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:10:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.927+0000 7f4f84ff9640 1 -- 172.21.15.120:0/3384630661 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f94004780 con 0x55a7247b3370 2024-09-19T09:10:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.927+0000 7f4f84ff9640 1 -- 172.21.15.120:0/3384630661 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f4f94004b20 con 0x55a7247b3370 2024-09-19T09:10:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.927+0000 7f4fa648c740 1 -- 172.21.15.120:0/3384630661 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f8c004030 con 0x55a7247b3370 2024-09-19T09:10:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.927+0000 7f4f84ff9640 1 -- 172.21.15.120:0/3384630661 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4f94004d20 con 0x55a7247b3370 2024-09-19T09:10:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.927+0000 7f4fa648c740 10 client.27146 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.927+0000 7f4fa648c740 1 -- 172.21.15.120:0/3384630661 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a7244f0620 con 0x55a7247b3370 2024-09-19T09:10:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.927+0000 7f4f84ff9640 1 -- 172.21.15.120:0/3384630661 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f8c004030 con 0x55a7247b3370 2024-09-19T09:10:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.927+0000 7f4f84ff9640 1 -- 172.21.15.120:0/3384630661 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4f94027030 con 0x55a7247b3370 2024-09-19T09:10:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.927+0000 7f4fa648c740 10 client.27146 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.927+0000 7f4fa648c740 10 client.27146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:10:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.927+0000 7f4fa648c740 20 client.27146 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.927+0000 7f4fa648c740 1 --2- 172.21.15.120:0/3384630661 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a7247c3850 0x55a7247c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.927+0000 7f4fa648c740 4 client.27146 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:10:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.927+0000 7f4fa648c740 1 -- 172.21.15.120:0/3384630661 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a72471ea20 con 0x55a7247c3850 2024-09-19T09:10:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.928+0000 7f4fa148d640 1 --2- 172.21.15.120:0/3384630661 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a7247c3850 0x55a7247c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.928+0000 7f4f84ff9640 10 client.27146 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.928+0000 7f4fa148d640 1 --2- 172.21.15.120:0/3384630661 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a7247c3850 0x55a7247c5c40 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c34df2c157c4479c server_cookie=1318ec3b46191e3 in_seq=0 out_seq=0 2024-09-19T09:10:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.929+0000 7f4f84ff9640 1 -- 172.21.15.120:0/3384630661 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a72471ea20 con 0x55a7247c3850 2024-09-19T09:10:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.929+0000 7f4f84ff9640 10 client.27146 handle_command_reply: tid=0 2024-09-19T09:10:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.929+0000 7f4fa648c740 1 client.27146 shutdown 2024-09-19T09:10:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.929+0000 7f4fa648c740 1 -- 172.21.15.120:0/3384630661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a7247b3370 msgr2=0x55a7247bf0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.929+0000 7f4fa648c740 1 --2- 172.21.15.120:0/3384630661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a7247b3370 0x55a7247bf0f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4f94005c10 tx=0x7f4f94004a40 comp rx=0 tx=0).stop 2024-09-19T09:10:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.929+0000 7f4fa648c740 1 -- 172.21.15.120:0/3384630661 shutdown_connections 2024-09-19T09:10:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.929+0000 7f4fa648c740 1 --2- 172.21.15.120:0/3384630661 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a7247c3850 0x55a7247c5c40 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.929+0000 7f4fa648c740 1 --2- 172.21.15.120:0/3384630661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a7247b3370 0x55a7247bf0f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.929+0000 7f4fa648c740 1 --2- 172.21.15.120:0/3384630661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a7247ae2e0 0x55a7247b80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.929+0000 7f4f84ff9640 0 client.27146 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.929+0000 7f4fa648c740 1 --2- 172.21.15.120:0/3384630661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7247ad9a0 0x55a7247b7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.929+0000 7f4fa648c740 1 -- 172.21.15.120:0/3384630661 >> 172.21.15.120:0/3384630661 conn(0x55a7247a9010 msgr2=0x55a7247aa820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.929+0000 7f4fa648c740 1 -- 172.21.15.120:0/3384630661 shutdown_connections 2024-09-19T09:10:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.929+0000 7f4fa648c740 1 -- 172.21.15.120:0/3384630661 wait complete. 2024-09-19T09:10:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.929+0000 7f4fa648c740 20 client.27146 trim_cache size 0 max 16384 2024-09-19T09:10:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.958+0000 7f4fa648c740 1 Processor -- start 2024-09-19T09:10:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.958+0000 7f4fa648c740 1 -- start start 2024-09-19T09:10:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.958+0000 7f4fa648c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7247ad9a0 0x55a7247a80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.958+0000 7f4fa648c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a7247ae2e0 0x55a7247bf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.958+0000 7f4fa648c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a7247b3370 0x55a7247ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.958+0000 7f4fa648c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c1ab180 con 0x55a7247ae2e0 2024-09-19T09:10:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.958+0000 7f4fa648c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c1accc0 con 0x55a7247b3370 2024-09-19T09:10:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.958+0000 7f4fa648c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c1ab850 con 0x55a7247ad9a0 2024-09-19T09:10:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.958+0000 7f4fa0c8c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a7247ae2e0 0x55a7247bf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.958+0000 7f4fa1c8e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a7247b3370 0x55a7247ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.958+0000 7f4fa0c8c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a7247ae2e0 0x55a7247bf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48366/0 (socket says 172.21.15.120:48366) 2024-09-19T09:10:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.958+0000 7f4fa1c8e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a7247b3370 0x55a7247ca540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:46754/0 (socket says 172.21.15.120:46754) 2024-09-19T09:10:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.958+0000 7f4fa0c8c640 1 -- 172.21.15.120:0/3826844749 learned_addr learned my addr 172.21.15.120:0/3826844749 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.958+0000 7f4fa148d640 1 --2- 172.21.15.120:0/3826844749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7247ad9a0 0x55a7247a80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.958+0000 7f4fa0c8c640 1 -- 172.21.15.120:0/3826844749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7247ad9a0 msgr2=0x55a7247a80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.958+0000 7f4fa0c8c640 1 --2- 172.21.15.120:0/3826844749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7247ad9a0 0x55a7247a80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.958+0000 7f4fa0c8c640 1 -- 172.21.15.120:0/3826844749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a7247b3370 msgr2=0x55a7247ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.958+0000 7f4fa0c8c640 1 --2- 172.21.15.120:0/3826844749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a7247b3370 0x55a7247ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.958+0000 7f4fa0c8c640 1 -- 172.21.15.120:0/3826844749 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7247c5fb0 con 0x55a7247ae2e0 2024-09-19T09:10:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.958+0000 7f4fa1c8e640 1 --2- 172.21.15.120:0/3826844749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a7247b3370 0x55a7247ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:10:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.959+0000 7f4fa0c8c640 1 --2- 172.21.15.120:0/3826844749 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a7247ae2e0 0x55a7247bf0b0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c005ab0 tx=0x7f4f8c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ce28662e46a415 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.959+0000 7f4f86ffd640 1 -- 172.21.15.120:0/3826844749 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c040070 con 0x55a7247ae2e0 2024-09-19T09:10:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.959+0000 7f4f86ffd640 1 -- 172.21.15.120:0/3826844749 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f8c032e70 con 0x55a7247ae2e0 2024-09-19T09:10:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.959+0000 7f4f86ffd640 1 -- 172.21.15.120:0/3826844749 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c035040 con 0x55a7247ae2e0 2024-09-19T09:10:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.960+0000 7f4fa648c740 1 -- 172.21.15.120:0/3826844749 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a7247ae2e0 msgr2=0x55a7247bf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.960+0000 7f4fa648c740 1 --2- 172.21.15.120:0/3826844749 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a7247ae2e0 0x55a7247bf0b0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c005ab0 tx=0x7f4f8c000bc0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.960+0000 7f4fa648c740 1 -- 172.21.15.120:0/3826844749 shutdown_connections 2024-09-19T09:10:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.960+0000 7f4fa648c740 1 --2- 172.21.15.120:0/3826844749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a7247b3370 0x55a7247ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.960+0000 7f4fa648c740 1 --2- 172.21.15.120:0/3826844749 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a7247ae2e0 0x55a7247bf0b0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.960+0000 7f4fa648c740 1 --2- 172.21.15.120:0/3826844749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7247ad9a0 0x55a7247a80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.960+0000 7f4fa648c740 1 -- 172.21.15.120:0/3826844749 >> 172.21.15.120:0/3826844749 conn(0x55a7247a9010 msgr2=0x55a7247a79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.960+0000 7f4fa648c740 1 -- 172.21.15.120:0/3826844749 shutdown_connections 2024-09-19T09:10:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.960+0000 7f4fa648c740 1 -- 172.21.15.120:0/3826844749 wait complete. 2024-09-19T09:10:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.960+0000 7f4fa648c740 1 -- start start 2024-09-19T09:10:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.960+0000 7f4fa648c740 1 Processor -- start 2024-09-19T09:10:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.961+0000 7f4fa648c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a7247ad9a0 0x55a7247bc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.961+0000 7f4fa648c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a7247ae2e0 0x55a7247bc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.961+0000 7f4fa648c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7247b3370 0x55a72482c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.961+0000 7f4fa648c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c1ac5f0 con 0x55a7247ad9a0 2024-09-19T09:10:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.961+0000 7f4fa648c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c1abf20 con 0x55a7247ae2e0 2024-09-19T09:10:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.961+0000 7f4fa648c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c1a9140 con 0x55a7247b3370 2024-09-19T09:10:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.961+0000 7f4fa648c740 10 client.0 _ll_register_callbacks cb 0x7f4f94011a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.961+0000 7f4fa148d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a7247ad9a0 0x55a7247bc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.961+0000 7f4fa1c8e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7247b3370 0x55a72482c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.961+0000 7f4fa0c8c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a7247ae2e0 0x55a7247bc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.961+0000 7f4fa148d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a7247ad9a0 0x55a7247bc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48374/0 (socket says 172.21.15.120:48374) 2024-09-19T09:10:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.961+0000 7f4fa1c8e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7247b3370 0x55a72482c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:49034/0 (socket says 172.21.15.120:49034) 2024-09-19T09:10:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.961+0000 7f4fa148d640 1 -- 172.21.15.120:0/578271829 learned_addr learned my addr 172.21.15.120:0/578271829 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.961+0000 7f4fa148d640 1 -- 172.21.15.120:0/578271829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7247b3370 msgr2=0x55a72482c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.961+0000 7f4fa148d640 1 --2- 172.21.15.120:0/578271829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7247b3370 0x55a72482c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.961+0000 7f4fa148d640 1 -- 172.21.15.120:0/578271829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a7247ae2e0 msgr2=0x55a7247bc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.961+0000 7f4fa148d640 1 --2- 172.21.15.120:0/578271829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a7247ae2e0 0x55a7247bc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.961+0000 7f4fa148d640 1 -- 172.21.15.120:0/578271829 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a72482caa0 con 0x55a7247ad9a0 2024-09-19T09:10:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.961+0000 7f4fa0c8c640 1 --2- 172.21.15.120:0/578271829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a7247ae2e0 0x55a7247bc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:10:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.961+0000 7f4fa148d640 1 --2- 172.21.15.120:0/578271829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a7247ad9a0 0x55a7247bc0b0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800c6a0 tx=0x7f4f9806af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b909c93f7ed275d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.961+0000 7f4f84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.961+0000 7f4f84ff9640 1 -- 172.21.15.120:0/578271829 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9807ae60 con 0x55a7247ad9a0 2024-09-19T09:10:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.961+0000 7f4f84ff9640 1 -- 172.21.15.120:0/578271829 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f4f98003b30 con 0x55a7247ad9a0 2024-09-19T09:10:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.961+0000 7f4fa648c740 1 -- 172.21.15.120:0/578271829 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a724718d70 con 0x55a7247ad9a0 2024-09-19T09:10:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.962+0000 7f4f84ff9640 1 -- 172.21.15.120:0/578271829 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4f9809c230 con 0x55a7247ad9a0 2024-09-19T09:10:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.962+0000 7f4fa648c740 10 client.17901 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.962+0000 7f4fa648c740 1 -- 172.21.15.120:0/578271829 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a72482de80 con 0x55a7247ad9a0 2024-09-19T09:10:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.962+0000 7f4f84ff9640 1 -- 172.21.15.120:0/578271829 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a724718d70 con 0x55a7247ad9a0 2024-09-19T09:10:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.962+0000 7f4f84ff9640 1 -- 172.21.15.120:0/578271829 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4f980851e0 con 0x55a7247ad9a0 2024-09-19T09:10:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.962+0000 7f4fa648c740 10 client.17901 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.962+0000 7f4fa648c740 10 client.17901 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:10:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.962+0000 7f4fa648c740 20 client.17901 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.962+0000 7f4fa648c740 1 --2- 172.21.15.120:0/578271829 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a72482e760 0x55a724830b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.962+0000 7f4fa648c740 4 client.17901 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:10:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.962+0000 7f4fa648c740 1 -- 172.21.15.120:0/578271829 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a724695900 con 0x55a72482e760 2024-09-19T09:10:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.962+0000 7f4fa0c8c640 1 --2- 172.21.15.120:0/578271829 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a72482e760 0x55a724830b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.963+0000 7f4f84ff9640 10 client.17901 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:05.963+0000 7f4fa0c8c640 1 --2- 172.21.15.120:0/578271829 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a72482e760 0x55a724830b80 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f11c82e8493e9a2a server_cookie=ddf095a6c4232b6b in_seq=0 out_seq=0 2024-09-19T09:10:06.056 INFO:tasks.workunit.client.0.smithi120.stdout: 1 692963 10.48 MB/sec execute 241 sec latency 86.559 ms 2024-09-19T09:10:06.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:05 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2026209775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:10:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:06.132+0000 7f4f84ff9640 1 -- 172.21.15.120:0/578271829 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a724695900 con 0x55a72482e760 2024-09-19T09:10:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:06.132+0000 7f4f84ff9640 10 client.17901 handle_command_reply: tid=0 2024-09-19T09:10:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:06.132+0000 7f4fa648c740 1 client.17901 shutdown 2024-09-19T09:10:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:06.132+0000 7f4fa648c740 1 -- 172.21.15.120:0/578271829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a7247ad9a0 msgr2=0x55a7247bc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:06.132+0000 7f4fa648c740 1 --2- 172.21.15.120:0/578271829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a7247ad9a0 0x55a7247bc0b0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800c6a0 tx=0x7f4f9806af70 comp rx=0 tx=0).stop 2024-09-19T09:10:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:06.132+0000 7f4fa648c740 1 -- 172.21.15.120:0/578271829 shutdown_connections 2024-09-19T09:10:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:06.132+0000 7f4fa648c740 1 --2- 172.21.15.120:0/578271829 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a72482e760 0x55a724830b80 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:06.132+0000 7f4f84ff9640 0 client.17901 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:06.132+0000 7f4fa648c740 1 --2- 172.21.15.120:0/578271829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7247b3370 0x55a72482c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:06.132+0000 7f4fa648c740 1 --2- 172.21.15.120:0/578271829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a7247ae2e0 0x55a7247bc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:06.132+0000 7f4fa648c740 1 --2- 172.21.15.120:0/578271829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a7247ad9a0 0x55a7247bc0b0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:06.132+0000 7f4fa648c740 1 -- 172.21.15.120:0/578271829 >> 172.21.15.120:0/578271829 conn(0x55a7247a9010 msgr2=0x55a7247a79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:06.133+0000 7f4fa648c740 1 -- 172.21.15.120:0/578271829 shutdown_connections 2024-09-19T09:10:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:06.133+0000 7f4fa648c740 1 -- 172.21.15.120:0/578271829 wait complete. 2024-09-19T09:10:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:06.133+0000 7f4fa648c740 20 client.17901 trim_cache size 0 max 16384 2024-09-19T09:10:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:10:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:10:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:10:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:10:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:06.135+0000 7f4fa36f1640 1 -- 172.21.15.120:0/3231415007 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4f880a32f0 msgr2=0x7f4f880a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:06.135+0000 7f4fa36f1640 1 --2- 172.21.15.120:0/3231415007 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4f880a32f0 0x7f4f880a57b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4f94005c40 tx=0x7f4f94009500 comp rx=0 tx=0).stop 2024-09-19T09:10:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:06.135+0000 7f4fa36f1640 1 -- 172.21.15.120:0/3231415007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f9c100420 msgr2=0x7f4f9c199f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:06.135+0000 7f4fa36f1640 1 --2- 172.21.15.120:0/3231415007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f9c100420 0x7f4f9c199f70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800ea60 tx=0x7f4f9800ee60 comp rx=0 tx=0).stop 2024-09-19T09:10:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:06.135+0000 7f4fa36f1640 1 -- 172.21.15.120:0/3231415007 shutdown_connections 2024-09-19T09:10:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:06.135+0000 7f4fa36f1640 1 --2- 172.21.15.120:0/3231415007 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4f880a32f0 0x7f4f880a57b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:06.135+0000 7f4fa36f1640 1 --2- 172.21.15.120:0/3231415007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f9c101700 0x7f4f9c1a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:06.135+0000 7f4fa36f1640 1 --2- 172.21.15.120:0/3231415007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f9c100d60 0x7f4f9c19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:06.135+0000 7f4fa36f1640 1 --2- 172.21.15.120:0/3231415007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f9c100420 0x7f4f9c199f70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:06.135+0000 7f4fa36f1640 1 -- 172.21.15.120:0/3231415007 >> 172.21.15.120:0/3231415007 conn(0x7f4f9c0fc1b0 msgr2=0x7f4f9c0fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:06.139 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0' 2024-09-19T09:10:06.139 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0 2024-09-19T09:10:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:06.135+0000 7f4fa36f1640 1 -- 172.21.15.120:0/3231415007 shutdown_connections 2024-09-19T09:10:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:06.135+0000 7f4fa36f1640 1 -- 172.21.15.120:0/3231415007 wait complete. 2024-09-19T09:10:06.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:05 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2026209775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:10:06.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:05 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2026209775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:10:06.217 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:10:06.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.713+0000 7f0238421640 1 -- 172.21.15.120:0/41478034 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0230100e20 msgr2=0x7f0230101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:06.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.713+0000 7f0238421640 1 --2- 172.21.15.120:0/41478034 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0230100e20 0x7f0230101200 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f022c009900 tx=0x7f022c02a480 comp rx=0 tx=0).stop 2024-09-19T09:10:06.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.713+0000 7f0238421640 1 -- 172.21.15.120:0/41478034 shutdown_connections 2024-09-19T09:10:06.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.713+0000 7f0238421640 1 --2- 172.21.15.120:0/41478034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0230102ef0 0x7f023010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.713+0000 7f0238421640 1 --2- 172.21.15.120:0/41478034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0230102570 0x7f02301029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.713+0000 7f0238421640 1 --2- 172.21.15.120:0/41478034 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0230100e20 0x7f0230101200 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.713+0000 7f0238421640 1 -- 172.21.15.120:0/41478034 >> 172.21.15.120:0/41478034 conn(0x7f02300fc7d0 msgr2=0x7f02300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:06.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.713+0000 7f0238421640 1 -- 172.21.15.120:0/41478034 shutdown_connections 2024-09-19T09:10:06.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.713+0000 7f0238421640 1 -- 172.21.15.120:0/41478034 wait complete. 2024-09-19T09:10:06.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.713+0000 7f0238421640 1 Processor -- start 2024-09-19T09:10:06.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.713+0000 7f0238421640 1 -- start start 2024-09-19T09:10:06.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.714+0000 7f0238421640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0230100e20 0x7f02301993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:06.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.714+0000 7f0238421640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0230102570 0x7f0230199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:06.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.714+0000 7f0238421640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0230102ef0 0x7f02301a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:06.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.714+0000 7f0238421640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0230110f80 con 0x7f0230100e20 2024-09-19T09:10:06.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.714+0000 7f0238421640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0230110e00 con 0x7f0230102ef0 2024-09-19T09:10:06.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.714+0000 7f0238421640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0230111100 con 0x7f0230102570 2024-09-19T09:10:06.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.714+0000 7f02361bd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0230100e20 0x7f02301993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:06.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.714+0000 7f02361bd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0230100e20 0x7f02301993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48396/0 (socket says 172.21.15.120:48396) 2024-09-19T09:10:06.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.714+0000 7f02361bd640 1 -- 172.21.15.120:0/1001929838 learned_addr learned my addr 172.21.15.120:0/1001929838 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:06.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.714+0000 7f02359bc640 1 --2- 172.21.15.120:0/1001929838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0230102570 0x7f0230199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:06.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.714+0000 7f02369be640 1 --2- 172.21.15.120:0/1001929838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0230102ef0 0x7f02301a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:06.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.714+0000 7f02361bd640 1 -- 172.21.15.120:0/1001929838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0230102570 msgr2=0x7f0230199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:06.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.714+0000 7f02361bd640 1 --2- 172.21.15.120:0/1001929838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0230102570 0x7f0230199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.714+0000 7f02361bd640 1 -- 172.21.15.120:0/1001929838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0230102ef0 msgr2=0x7f02301a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:06.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.714+0000 7f02361bd640 1 --2- 172.21.15.120:0/1001929838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0230102ef0 0x7f02301a4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.714+0000 7f02361bd640 1 -- 172.21.15.120:0/1001929838 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02301a4990 con 0x7f0230100e20 2024-09-19T09:10:06.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.714+0000 7f02359bc640 1 --2- 172.21.15.120:0/1001929838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0230102570 0x7f0230199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:10:06.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.714+0000 7f02361bd640 1 --2- 172.21.15.120:0/1001929838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0230100e20 0x7f02301993d0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f022c006010 tx=0x7f022c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:06.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.714+0000 7f02277fe640 1 -- 172.21.15.120:0/1001929838 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c03f070 con 0x7f0230100e20 2024-09-19T09:10:06.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.715+0000 7f0238421640 1 -- 172.21.15.120:0/1001929838 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02301a4b60 con 0x7f0230100e20 2024-09-19T09:10:06.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.715+0000 7f02277fe640 1 -- 172.21.15.120:0/1001929838 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f022c031c30 con 0x7f0230100e20 2024-09-19T09:10:06.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.715+0000 7f02277fe640 1 -- 172.21.15.120:0/1001929838 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c035030 con 0x7f0230100e20 2024-09-19T09:10:06.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.715+0000 7f0238421640 1 -- 172.21.15.120:0/1001929838 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02301a4fc0 con 0x7f0230100e20 2024-09-19T09:10:06.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.717+0000 7f02277fe640 1 -- 172.21.15.120:0/1001929838 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f022c042070 con 0x7f0230100e20 2024-09-19T09:10:06.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.717+0000 7f023a7fa740 1 Processor -- start 2024-09-19T09:10:06.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.717+0000 7f023a7fa740 1 -- start start 2024-09-19T09:10:06.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.718+0000 7f023a7fa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a65102c380 0x55a65102c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:06.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.718+0000 7f023a7fa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a65102cca0 0x55a65103b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:06.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.718+0000 7f023a7fa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a65103ba90 0x55a65103de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:06.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.718+0000 7f023a7fa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a650f922e0 con 0x55a65102cca0 2024-09-19T09:10:06.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.718+0000 7f023a7fa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a650f39ce0 con 0x55a65102c380 2024-09-19T09:10:06.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.718+0000 7f023a7fa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a650f8e2a0 con 0x55a65103ba90 2024-09-19T09:10:06.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.718+0000 7f02277fe640 1 --2- 172.21.15.120:0/1001929838 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f021c0a3240 0x7f021c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:06.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.718+0000 7f02361bd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a65102cca0 0x55a65103b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:06.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.718+0000 7f02361bd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a65102cca0 0x55a65103b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48404/0 (socket says 172.21.15.120:48404) 2024-09-19T09:10:06.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.718+0000 7f02361bd640 1 -- 172.21.15.120:0/1573047471 learned_addr learned my addr 172.21.15.120:0/1573047471 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:06.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.718+0000 7f02369be640 1 --2- 172.21.15.120:0/1573047471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a65102c380 0x55a65102c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:06.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.718+0000 7f02359bc640 1 --2- 172.21.15.120:0/1573047471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a65103ba90 0x55a65103de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:06.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.718+0000 7f02369be640 1 --2- 172.21.15.120:0/1001929838 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f021c0a3240 0x7f021c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:06.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.718+0000 7f02359bc640 1 -- 172.21.15.120:0/1573047471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a65102c380 msgr2=0x55a65102c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:06.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.718+0000 7f02359bc640 1 --2- 172.21.15.120:0/1573047471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a65102c380 0x55a65102c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.718+0000 7f02359bc640 1 -- 172.21.15.120:0/1573047471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a65102cca0 msgr2=0x55a65103b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:06.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.718+0000 7f02359bc640 1 --2- 172.21.15.120:0/1573047471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a65102cca0 0x55a65103b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.718+0000 7f02359bc640 1 -- 172.21.15.120:0/1573047471 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a650f1e440 con 0x55a65103ba90 2024-09-19T09:10:06.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.718+0000 7f02361bd640 1 --2- 172.21.15.120:0/1573047471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a65102cca0 0x55a65103b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:10:06.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.718+0000 7f02369be640 1 --2- 172.21.15.120:0/1573047471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a65102c380 0x55a65102c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:10:06.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.718+0000 7f02277fe640 1 -- 172.21.15.120:0/1001929838 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f022c0eff70 con 0x7f0230100e20 2024-09-19T09:10:06.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.718+0000 7f02359bc640 1 --2- 172.21.15.120:0/1573047471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a65103ba90 0x55a65103de80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0220009fa0 tx=0x7f02200118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f5a9d368ea60fb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:06.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.718+0000 7f02369be640 1 --2- 172.21.15.120:0/1001929838 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f021c0a3240 0x7f021c0a5700 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0228009fd0 tx=0x7f0228009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:06.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.719+0000 7f0224dea640 1 -- 172.21.15.120:0/1573047471 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220004310 con 0x55a65103ba90 2024-09-19T09:10:06.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.719+0000 7f0224dea640 1 -- 172.21.15.120:0/1573047471 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02200044b0 con 0x55a65103ba90 2024-09-19T09:10:06.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.719+0000 7f0224dea640 1 -- 172.21.15.120:0/1573047471 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02200035f0 con 0x55a65103ba90 2024-09-19T09:10:06.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.720+0000 7f023a7fa740 1 -- 172.21.15.120:0/1573047471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a65103ba90 msgr2=0x55a65103de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:06.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.720+0000 7f023a7fa740 1 --2- 172.21.15.120:0/1573047471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a65103ba90 0x55a65103de80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0220009fa0 tx=0x7f02200118d0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.720+0000 7f023a7fa740 1 -- 172.21.15.120:0/1573047471 shutdown_connections 2024-09-19T09:10:06.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.720+0000 7f023a7fa740 1 --2- 172.21.15.120:0/1573047471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a65103ba90 0x55a65103de80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.720+0000 7f023a7fa740 1 --2- 172.21.15.120:0/1573047471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a65102cca0 0x55a65103b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.720+0000 7f023a7fa740 1 --2- 172.21.15.120:0/1573047471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a65102c380 0x55a65102c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.720+0000 7f023a7fa740 1 -- 172.21.15.120:0/1573047471 >> 172.21.15.120:0/1573047471 conn(0x55a651028a30 msgr2=0x55a651028e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:06.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.720+0000 7f023a7fa740 1 -- 172.21.15.120:0/1573047471 shutdown_connections 2024-09-19T09:10:06.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.720+0000 7f023a7fa740 1 -- 172.21.15.120:0/1573047471 wait complete. 2024-09-19T09:10:06.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.720+0000 7f023a7fa740 1 -- start start 2024-09-19T09:10:06.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.720+0000 7f023a7fa740 1 Processor -- start 2024-09-19T09:10:06.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.720+0000 7f023a7fa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a65102c380 0x55a65103b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:06.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.720+0000 7f023a7fa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a65102cca0 0x55a6510386c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:06.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.720+0000 7f023a7fa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a65103ba90 0x55a651038c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:06.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.720+0000 7f023a7fa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a65103e660 con 0x55a65102cca0 2024-09-19T09:10:06.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.720+0000 7f023a7fa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a650f4d050 con 0x55a65102c380 2024-09-19T09:10:06.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.720+0000 7f023a7fa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a65103e7e0 con 0x55a65103ba90 2024-09-19T09:10:06.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.720+0000 7f023a7fa740 10 client.0 _ll_register_callbacks cb 0x55a650f8d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:06.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.721+0000 7f02359bc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a65102cca0 0x55a6510386c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:06.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.721+0000 7f02361bd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a65102c380 0x55a65103b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:06.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.721+0000 7f02369be640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a65103ba90 0x55a651038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:06.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.721+0000 7f02359bc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a65102cca0 0x55a6510386c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48414/0 (socket says 172.21.15.120:48414) 2024-09-19T09:10:06.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.721+0000 7f02361bd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a65102c380 0x55a65103b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:46790/0 (socket says 172.21.15.120:46790) 2024-09-19T09:10:06.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.721+0000 7f02359bc640 1 -- 172.21.15.120:0/3000962081 learned_addr learned my addr 172.21.15.120:0/3000962081 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:06.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.721+0000 7f02369be640 1 -- 172.21.15.120:0/3000962081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a65102c380 msgr2=0x55a65103b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:06.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.721+0000 7f02369be640 1 --2- 172.21.15.120:0/3000962081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a65102c380 0x55a65103b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.721+0000 7f02369be640 1 -- 172.21.15.120:0/3000962081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a65102cca0 msgr2=0x55a6510386c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:06.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.721+0000 7f02369be640 1 --2- 172.21.15.120:0/3000962081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a65102cca0 0x55a6510386c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.721+0000 7f02369be640 1 -- 172.21.15.120:0/3000962081 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a650d033c0 con 0x55a65103ba90 2024-09-19T09:10:06.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.721+0000 7f02359bc640 1 --2- 172.21.15.120:0/3000962081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a65102cca0 0x55a6510386c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:10:06.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.721+0000 7f02361bd640 1 --2- 172.21.15.120:0/3000962081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a65102c380 0x55a65103b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:10:06.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.721+0000 7f02369be640 1 --2- 172.21.15.120:0/3000962081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a65103ba90 0x55a651038c40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f02280119e0 tx=0x7f0228004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b62483b755f7c191 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:06.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.721+0000 7f0219ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:10:06.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.721+0000 7f0219ffb640 1 -- 172.21.15.120:0/3000962081 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022800fe40 con 0x55a65103ba90 2024-09-19T09:10:06.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.721+0000 7f023a7fa740 1 -- 172.21.15.120:0/3000962081 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02200044b0 con 0x55a65103ba90 2024-09-19T09:10:06.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.721+0000 7f0219ffb640 1 -- 172.21.15.120:0/3000962081 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f022801fce0 con 0x55a65103ba90 2024-09-19T09:10:06.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.721+0000 7f0219ffb640 1 -- 172.21.15.120:0/3000962081 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0228004b20 con 0x55a65103ba90 2024-09-19T09:10:06.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.721+0000 7f023a7fa740 10 client.27058 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:06.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.721+0000 7f023a7fa740 1 -- 172.21.15.120:0/3000962081 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a65104a0e0 con 0x55a65103ba90 2024-09-19T09:10:06.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.722+0000 7f0219ffb640 1 -- 172.21.15.120:0/3000962081 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02200044b0 con 0x55a65103ba90 2024-09-19T09:10:06.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.722+0000 7f0219ffb640 1 -- 172.21.15.120:0/3000962081 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f022801d070 con 0x55a65103ba90 2024-09-19T09:10:06.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.722+0000 7f023a7fa740 10 client.27058 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:06.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.722+0000 7f023a7fa740 10 client.27058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:10:06.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.722+0000 7f023a7fa740 20 client.27058 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:06.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.722+0000 7f023a7fa740 1 --2- 172.21.15.120:0/3000962081 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a65104b5b0 0x55a65104da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:06.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.722+0000 7f023a7fa740 4 client.27058 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:10:06.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.722+0000 7f023a7fa740 1 -- 172.21.15.120:0/3000962081 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a650f9f850 con 0x55a65104b5b0 2024-09-19T09:10:06.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.722+0000 7f02361bd640 1 --2- 172.21.15.120:0/3000962081 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a65104b5b0 0x55a65104da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:06.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.722+0000 7f0219ffb640 10 client.27058 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:06.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.722+0000 7f02361bd640 1 --2- 172.21.15.120:0/3000962081 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a65104b5b0 0x55a65104da30 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f45cbb5416b24b2 server_cookie=eb2558b09531b7db in_seq=0 out_seq=0 2024-09-19T09:10:06.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.723+0000 7f0219ffb640 1 -- 172.21.15.120:0/3000962081 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a650f9f850 con 0x55a65104b5b0 2024-09-19T09:10:06.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.723+0000 7f0219ffb640 10 client.27058 handle_command_reply: tid=0 2024-09-19T09:10:06.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.723+0000 7f023a7fa740 1 client.27058 shutdown 2024-09-19T09:10:06.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.723+0000 7f023a7fa740 1 -- 172.21.15.120:0/3000962081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a65103ba90 msgr2=0x55a651038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:06.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.723+0000 7f023a7fa740 1 --2- 172.21.15.120:0/3000962081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a65103ba90 0x55a651038c40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f02280119e0 tx=0x7f0228004a40 comp rx=0 tx=0).stop 2024-09-19T09:10:06.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.723+0000 7f023a7fa740 1 -- 172.21.15.120:0/3000962081 shutdown_connections 2024-09-19T09:10:06.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.723+0000 7f023a7fa740 1 --2- 172.21.15.120:0/3000962081 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a65104b5b0 0x55a65104da30 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.723+0000 7f023a7fa740 1 --2- 172.21.15.120:0/3000962081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a65103ba90 0x55a651038c40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.723+0000 7f023a7fa740 1 --2- 172.21.15.120:0/3000962081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a65102cca0 0x55a6510386c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.723+0000 7f0219ffb640 0 client.27058 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:06.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.723+0000 7f023a7fa740 1 --2- 172.21.15.120:0/3000962081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a65102c380 0x55a65103b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.723+0000 7f023a7fa740 1 -- 172.21.15.120:0/3000962081 >> 172.21.15.120:0/3000962081 conn(0x55a651028a30 msgr2=0x55a65102e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:06.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.723+0000 7f023a7fa740 1 -- 172.21.15.120:0/3000962081 shutdown_connections 2024-09-19T09:10:06.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.723+0000 7f023a7fa740 1 -- 172.21.15.120:0/3000962081 wait complete. 2024-09-19T09:10:06.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.724+0000 7f023a7fa740 20 client.27058 trim_cache size 0 max 16384 2024-09-19T09:10:06.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.741+0000 7f023a7fa740 1 Processor -- start 2024-09-19T09:10:06.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.741+0000 7f023a7fa740 1 -- start start 2024-09-19T09:10:06.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.741+0000 7f023a7fa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a65102c380 0x55a651047900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:06.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.741+0000 7f023a7fa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a65102cca0 0x55a651053300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:06.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.741+0000 7f023a7fa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a65103ba90 0x55a651053840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:06.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.741+0000 7f023a7fa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f02301ab2b0 con 0x55a65103ba90 2024-09-19T09:10:06.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.741+0000 7f023a7fa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f02301acdf0 con 0x55a65102c380 2024-09-19T09:10:06.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.741+0000 7f023a7fa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f02301ab980 con 0x55a65102cca0 2024-09-19T09:10:06.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.741+0000 7f02369be640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a65103ba90 0x55a651053840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:06.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.741+0000 7f02369be640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a65103ba90 0x55a651053840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48424/0 (socket says 172.21.15.120:48424) 2024-09-19T09:10:06.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.741+0000 7f02361bd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a65102c380 0x55a651047900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:06.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.741+0000 7f02369be640 1 -- 172.21.15.120:0/2333586976 learned_addr learned my addr 172.21.15.120:0/2333586976 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:06.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.741+0000 7f02359bc640 1 --2- 172.21.15.120:0/2333586976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a65102cca0 0x55a651053300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:06.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.741+0000 7f02369be640 1 -- 172.21.15.120:0/2333586976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a65102cca0 msgr2=0x55a651053300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:06.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.741+0000 7f02369be640 1 --2- 172.21.15.120:0/2333586976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a65102cca0 0x55a651053300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.741+0000 7f02369be640 1 -- 172.21.15.120:0/2333586976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a65102c380 msgr2=0x55a651047900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:06.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.741+0000 7f02369be640 1 --2- 172.21.15.120:0/2333586976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a65102c380 0x55a651047900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.741+0000 7f02369be640 1 -- 172.21.15.120:0/2333586976 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a65104e0b0 con 0x55a65103ba90 2024-09-19T09:10:06.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.742+0000 7f02369be640 1 --2- 172.21.15.120:0/2333586976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a65103ba90 0x55a651053840 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f0228034390 tx=0x7f0228004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d03c3f9105f4f900 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:06.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.742+0000 7f021bfff640 1 -- 172.21.15.120:0/2333586976 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0228039070 con 0x55a65103ba90 2024-09-19T09:10:06.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.742+0000 7f021bfff640 1 -- 172.21.15.120:0/2333586976 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0228003e50 con 0x55a65103ba90 2024-09-19T09:10:06.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.742+0000 7f021bfff640 1 -- 172.21.15.120:0/2333586976 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0228034b80 con 0x55a65103ba90 2024-09-19T09:10:06.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.743+0000 7f023a7fa740 1 -- 172.21.15.120:0/2333586976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a65103ba90 msgr2=0x55a651053840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:06.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.743+0000 7f023a7fa740 1 --2- 172.21.15.120:0/2333586976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a65103ba90 0x55a651053840 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f0228034390 tx=0x7f0228004ab0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.743+0000 7f023a7fa740 1 -- 172.21.15.120:0/2333586976 shutdown_connections 2024-09-19T09:10:06.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.743+0000 7f023a7fa740 1 --2- 172.21.15.120:0/2333586976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a65103ba90 0x55a651053840 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.743+0000 7f023a7fa740 1 --2- 172.21.15.120:0/2333586976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a65102cca0 0x55a651053300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.743+0000 7f023a7fa740 1 --2- 172.21.15.120:0/2333586976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a65102c380 0x55a651047900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.743+0000 7f023a7fa740 1 -- 172.21.15.120:0/2333586976 >> 172.21.15.120:0/2333586976 conn(0x55a651028a30 msgr2=0x55a651033d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:06.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.743+0000 7f023a7fa740 1 -- 172.21.15.120:0/2333586976 shutdown_connections 2024-09-19T09:10:06.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.743+0000 7f023a7fa740 1 -- 172.21.15.120:0/2333586976 wait complete. 2024-09-19T09:10:06.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.743+0000 7f023a7fa740 1 -- start start 2024-09-19T09:10:06.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.743+0000 7f023a7fa740 1 Processor -- start 2024-09-19T09:10:06.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.743+0000 7f023a7fa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a65102c380 0x55a65103efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:06.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.743+0000 7f023a7fa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a65102cca0 0x55a65103f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:06.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.743+0000 7f023a7fa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a65103ba90 0x55a6510b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:06.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.743+0000 7f023a7fa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f02301ac720 con 0x55a65102cca0 2024-09-19T09:10:06.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.743+0000 7f023a7fa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f02301ac050 con 0x55a65103ba90 2024-09-19T09:10:06.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.743+0000 7f023a7fa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f02301a9270 con 0x55a65102c380 2024-09-19T09:10:06.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.744+0000 7f023a7fa740 10 client.0 _ll_register_callbacks cb 0x7f0228009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:06.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.744+0000 7f02359bc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a65102cca0 0x55a65103f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:06.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.744+0000 7f02359bc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a65102cca0 0x55a65103f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48426/0 (socket says 172.21.15.120:48426) 2024-09-19T09:10:06.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.744+0000 7f02359bc640 1 -- 172.21.15.120:0/1248495496 learned_addr learned my addr 172.21.15.120:0/1248495496 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:06.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.744+0000 7f02369be640 1 --2- 172.21.15.120:0/1248495496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a65103ba90 0x55a6510b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:06.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.744+0000 7f02361bd640 1 --2- 172.21.15.120:0/1248495496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a65102c380 0x55a65103efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:06.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.744+0000 7f02359bc640 1 -- 172.21.15.120:0/1248495496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a65102c380 msgr2=0x55a65103efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:06.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.744+0000 7f02359bc640 1 --2- 172.21.15.120:0/1248495496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a65102c380 0x55a65103efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.744+0000 7f02359bc640 1 -- 172.21.15.120:0/1248495496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a65103ba90 msgr2=0x55a6510b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:06.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.744+0000 7f02359bc640 1 --2- 172.21.15.120:0/1248495496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a65103ba90 0x55a6510b42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.744+0000 7f02359bc640 1 -- 172.21.15.120:0/1248495496 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6510b4900 con 0x55a65102cca0 2024-09-19T09:10:06.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.744+0000 7f02359bc640 1 --2- 172.21.15.120:0/1248495496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a65102cca0 0x55a65103f510 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f0220004c80 tx=0x7f0220004820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36540db0374e018c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:06.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.744+0000 7f0219ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:06.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.744+0000 7f0219ffb640 1 -- 172.21.15.120:0/1248495496 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022001d040 con 0x55a65102cca0 2024-09-19T09:10:06.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.744+0000 7f0219ffb640 1 -- 172.21.15.120:0/1248495496 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f02200037f0 con 0x55a65102cca0 2024-09-19T09:10:06.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.744+0000 7f023a7fa740 1 -- 172.21.15.120:0/1248495496 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0228003e50 con 0x55a65102cca0 2024-09-19T09:10:06.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.744+0000 7f0219ffb640 1 -- 172.21.15.120:0/1248495496 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0220004990 con 0x55a65102cca0 2024-09-19T09:10:06.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.744+0000 7f023a7fa740 10 client.17919 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:06.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.744+0000 7f023a7fa740 1 -- 172.21.15.120:0/1248495496 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a6510b5bf0 con 0x55a65102cca0 2024-09-19T09:10:06.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.745+0000 7f0219ffb640 1 -- 172.21.15.120:0/1248495496 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0228003e50 con 0x55a65102cca0 2024-09-19T09:10:06.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.745+0000 7f0219ffb640 1 -- 172.21.15.120:0/1248495496 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f022000f4e0 con 0x55a65102cca0 2024-09-19T09:10:06.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.745+0000 7f023a7fa740 10 client.17919 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:06.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.745+0000 7f023a7fa740 10 client.17919 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:10:06.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.745+0000 7f023a7fa740 20 client.17919 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:06.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.745+0000 7f023a7fa740 1 --2- 172.21.15.120:0/1248495496 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a6510b6200 0x55a6510b8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:06.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.745+0000 7f023a7fa740 4 client.17919 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:10:06.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.745+0000 7f023a7fa740 1 -- 172.21.15.120:0/1248495496 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a650f99d60 con 0x55a6510b6200 2024-09-19T09:10:06.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.745+0000 7f02361bd640 1 --2- 172.21.15.120:0/1248495496 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a6510b6200 0x55a6510b8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:06.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.745+0000 7f02361bd640 1 --2- 172.21.15.120:0/1248495496 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a6510b6200 0x55a6510b8620 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6ab098693540110 server_cookie=e955c39155e17c64 in_seq=0 out_seq=0 2024-09-19T09:10:06.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.745+0000 7f0219ffb640 10 client.17919 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:06.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.746+0000 7f0219ffb640 1 -- 172.21.15.120:0/1248495496 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a650f99d60 con 0x55a6510b6200 2024-09-19T09:10:06.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.746+0000 7f0219ffb640 10 client.17919 handle_command_reply: tid=0 2024-09-19T09:10:06.766 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:10:06.766 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:10:06.767 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:10:06.767 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:10:06.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.746+0000 7f023a7fa740 1 client.17919 shutdown 2024-09-19T09:10:06.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.746+0000 7f023a7fa740 1 -- 172.21.15.120:0/1248495496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a65102cca0 msgr2=0x55a65103f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:06.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.746+0000 7f023a7fa740 1 --2- 172.21.15.120:0/1248495496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a65102cca0 0x55a65103f510 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f0220004c80 tx=0x7f0220004820 comp rx=0 tx=0).stop 2024-09-19T09:10:06.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.746+0000 7f023a7fa740 1 -- 172.21.15.120:0/1248495496 shutdown_connections 2024-09-19T09:10:06.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.746+0000 7f023a7fa740 1 --2- 172.21.15.120:0/1248495496 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a6510b6200 0x55a6510b8620 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.746+0000 7f023a7fa740 1 --2- 172.21.15.120:0/1248495496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a65103ba90 0x55a6510b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.746+0000 7f023a7fa740 1 --2- 172.21.15.120:0/1248495496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a65102cca0 0x55a65103f510 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.746+0000 7f023a7fa740 1 --2- 172.21.15.120:0/1248495496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a65102c380 0x55a65103efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.746+0000 7f0219ffb640 0 client.17919 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:06.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.746+0000 7f023a7fa740 1 -- 172.21.15.120:0/1248495496 >> 172.21.15.120:0/1248495496 conn(0x55a651028a30 msgr2=0x55a651036f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:06.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.746+0000 7f023a7fa740 1 -- 172.21.15.120:0/1248495496 shutdown_connections 2024-09-19T09:10:06.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.746+0000 7f023a7fa740 1 -- 172.21.15.120:0/1248495496 wait complete. 2024-09-19T09:10:06.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.746+0000 7f023a7fa740 20 client.17919 trim_cache size 0 max 16384 2024-09-19T09:10:06.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.746+0000 7f0238421640 1 -- 172.21.15.120:0/1001929838 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f021c0a3240 msgr2=0x7f021c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:06.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.746+0000 7f0238421640 1 --2- 172.21.15.120:0/1001929838 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f021c0a3240 0x7f021c0a5700 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0228009fd0 tx=0x7f0228009500 comp rx=0 tx=0).stop 2024-09-19T09:10:06.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.746+0000 7f0238421640 1 -- 172.21.15.120:0/1001929838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0230100e20 msgr2=0x7f02301993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:06.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.746+0000 7f0238421640 1 --2- 172.21.15.120:0/1001929838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0230100e20 0x7f02301993d0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f022c006010 tx=0x7f022c003c10 comp rx=0 tx=0).stop 2024-09-19T09:10:06.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.746+0000 7f0238421640 1 -- 172.21.15.120:0/1001929838 shutdown_connections 2024-09-19T09:10:06.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.746+0000 7f0238421640 1 --2- 172.21.15.120:0/1001929838 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f021c0a3240 0x7f021c0a5700 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.746+0000 7f0238421640 1 --2- 172.21.15.120:0/1001929838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0230102ef0 0x7f02301a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.746+0000 7f0238421640 1 --2- 172.21.15.120:0/1001929838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0230102570 0x7f0230199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.746+0000 7f0238421640 1 --2- 172.21.15.120:0/1001929838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0230100e20 0x7f02301993d0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:06.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.746+0000 7f0238421640 1 -- 172.21.15.120:0/1001929838 >> 172.21.15.120:0/1001929838 conn(0x7f02300fc7d0 msgr2=0x7f02300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:06.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.747+0000 7f0238421640 1 -- 172.21.15.120:0/1001929838 shutdown_connections 2024-09-19T09:10:06.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:06.747+0000 7f0238421640 1 -- 172.21.15.120:0/1001929838 wait complete. 2024-09-19T09:10:07.056 INFO:tasks.workunit.client.0.smithi120.stdout: 1 695842 10.50 MB/sec execute 242 sec latency 84.305 ms 2024-09-19T09:10:07.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:06 smithi120 ceph-mon[31150]: pgmap v559: 97 pgs: 97 active+clean; 49 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.4 MiB/s wr, 353 op/s 2024-09-19T09:10:07.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:06 smithi196 ceph-mon[35150]: pgmap v559: 97 pgs: 97 active+clean; 49 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.4 MiB/s wr, 353 op/s 2024-09-19T09:10:07.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:06 smithi153 ceph-mon[35071]: pgmap v559: 97 pgs: 97 active+clean; 49 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.4 MiB/s wr, 353 op/s 2024-09-19T09:10:07.494 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:10:07.524 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:10:07.524 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:10:08.056 INFO:tasks.workunit.client.0.smithi120.stdout: 1 697866 10.50 MB/sec execute 243 sec latency 41.833 ms 2024-09-19T09:10:08.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.122+0000 7fe15915a640 1 -- 172.21.15.120:0/3781403551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe154102790 msgr2=0x7fe1541067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:08.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.122+0000 7fe15915a640 1 --2- 172.21.15.120:0/3781403551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe154102790 0x7fe1541067e0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009900 tx=0x7fe13c02a410 comp rx=0 tx=0).stop 2024-09-19T09:10:08.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.122+0000 7fe15915a640 1 -- 172.21.15.120:0/3781403551 shutdown_connections 2024-09-19T09:10:08.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.122+0000 7fe15915a640 1 --2- 172.21.15.120:0/3781403551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe154106f10 0x7fe15410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.122+0000 7fe15915a640 1 --2- 172.21.15.120:0/3781403551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe154102790 0x7fe1541067e0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.122+0000 7fe15915a640 1 --2- 172.21.15.120:0/3781403551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe154101de0 0x7fe1541021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.122+0000 7fe15915a640 1 -- 172.21.15.120:0/3781403551 >> 172.21.15.120:0/3781403551 conn(0x7fe1540fd8c0 msgr2=0x7fe1540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.122+0000 7fe15915a640 1 -- 172.21.15.120:0/3781403551 shutdown_connections 2024-09-19T09:10:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.122+0000 7fe15915a640 1 -- 172.21.15.120:0/3781403551 wait complete. 2024-09-19T09:10:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.123+0000 7fe15915a640 1 Processor -- start 2024-09-19T09:10:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.123+0000 7fe15915a640 1 -- start start 2024-09-19T09:10:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.123+0000 7fe15915a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe154101de0 0x7fe1540744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.123+0000 7fe15915a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe154102790 0x7fe154074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.123+0000 7fe15915a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe154106f10 0x7fe154074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.123+0000 7fe15915a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe154111f70 con 0x7fe154106f10 2024-09-19T09:10:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.123+0000 7fe15915a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe154111df0 con 0x7fe154101de0 2024-09-19T09:10:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.123+0000 7fe15915a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe1541120f0 con 0x7fe154102790 2024-09-19T09:10:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.123+0000 7fe15359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe154106f10 0x7fe154074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.123+0000 7fe15359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe154106f10 0x7fe154074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48440/0 (socket says 172.21.15.120:48440) 2024-09-19T09:10:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.123+0000 7fe15359e640 1 -- 172.21.15.120:0/1031600626 learned_addr learned my addr 172.21.15.120:0/1031600626 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.123+0000 7fe152d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe154101de0 0x7fe1540744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.123+0000 7fe14bfff640 1 --2- 172.21.15.120:0/1031600626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe154102790 0x7fe154074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.124+0000 7fe15359e640 1 -- 172.21.15.120:0/1031600626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe154102790 msgr2=0x7fe154074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.124+0000 7fe15359e640 1 --2- 172.21.15.120:0/1031600626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe154102790 0x7fe154074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.124+0000 7fe15359e640 1 -- 172.21.15.120:0/1031600626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe154101de0 msgr2=0x7fe1540744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.124+0000 7fe15359e640 1 --2- 172.21.15.120:0/1031600626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe154101de0 0x7fe1540744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.124+0000 7fe15359e640 1 -- 172.21.15.120:0/1031600626 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1541a6970 con 0x7fe154106f10 2024-09-19T09:10:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.124+0000 7fe15359e640 1 --2- 172.21.15.120:0/1031600626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe154106f10 0x7fe154074f70 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fe14c00eb10 tx=0x7fe14c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.124+0000 7fe150d99640 1 -- 172.21.15.120:0/1031600626 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c004b20 con 0x7fe154106f10 2024-09-19T09:10:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.124+0000 7fe150d99640 1 -- 172.21.15.120:0/1031600626 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe14c004cc0 con 0x7fe154106f10 2024-09-19T09:10:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.124+0000 7fe15915a640 1 -- 172.21.15.120:0/1031600626 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1541a6c60 con 0x7fe154106f10 2024-09-19T09:10:08.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.124+0000 7fe150d99640 1 -- 172.21.15.120:0/1031600626 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c010760 con 0x7fe154106f10 2024-09-19T09:10:08.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.124+0000 7fe15915a640 1 -- 172.21.15.120:0/1031600626 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1541a7120 con 0x7fe154106f10 2024-09-19T09:10:08.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.126+0000 7fe150d99640 1 -- 172.21.15.120:0/1031600626 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe14c005020 con 0x7fe154106f10 2024-09-19T09:10:08.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.127+0000 7fe15b533740 1 Processor -- start 2024-09-19T09:10:08.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.127+0000 7fe15b533740 1 -- start start 2024-09-19T09:10:08.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.127+0000 7fe150d99640 1 --2- 172.21.15.120:0/1031600626 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe1400a3240 0x7fe1400a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:08.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.127+0000 7fe14bfff640 1 --2- 172.21.15.120:0/1031600626 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe1400a3240 0x7fe1400a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:08.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.127+0000 7fe150d99640 1 -- 172.21.15.120:0/1031600626 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fe14c0cfe00 con 0x7fe154106f10 2024-09-19T09:10:08.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.127+0000 7fe15b533740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa7259f850 0x55aa7259fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:08.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.127+0000 7fe15b533740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa725a0170 0x55aa725aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:08.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.127+0000 7fe15b533740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa725aaa30 0x55aa725ace20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:08.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.127+0000 7fe15b533740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55aa72504160 con 0x55aa725a0170 2024-09-19T09:10:08.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.127+0000 7fe15b533740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55aa724ab720 con 0x55aa725aaa30 2024-09-19T09:10:08.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.127+0000 7fe15b533740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55aa72500120 con 0x55aa7259f850 2024-09-19T09:10:08.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.127+0000 7fe152d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa7259f850 0x55aa7259fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:08.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.127+0000 7fe15359e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa725aaa30 0x55aa725ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:08.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.127+0000 7fe15359e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa725aaa30 0x55aa725ace20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:46836/0 (socket says 172.21.15.120:46836) 2024-09-19T09:10:08.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.127+0000 7fe15359e640 1 -- 172.21.15.120:0/722752906 learned_addr learned my addr 172.21.15.120:0/722752906 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:08.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.128+0000 7fe14bfff640 1 --2- 172.21.15.120:0/722752906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa725a0170 0x55aa725aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:08.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.128+0000 7fe15359e640 1 -- 172.21.15.120:0/722752906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa7259f850 msgr2=0x55aa7259fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:08.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.128+0000 7fe15359e640 1 --2- 172.21.15.120:0/722752906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa7259f850 0x55aa7259fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.128+0000 7fe15359e640 1 -- 172.21.15.120:0/722752906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa725a0170 msgr2=0x55aa725aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:08.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.128+0000 7fe15359e640 1 --2- 172.21.15.120:0/722752906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa725a0170 0x55aa725aa4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.128+0000 7fe15359e640 1 -- 172.21.15.120:0/722752906 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa72275ba0 con 0x55aa725aaa30 2024-09-19T09:10:08.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.128+0000 7fe14bfff640 1 --2- 172.21.15.120:0/1031600626 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe1400a3240 0x7fe1400a5700 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009230 tx=0x7fe13c005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:08.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.128+0000 7fe15359e640 1 --2- 172.21.15.120:0/722752906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa725aaa30 0x55aa725ace20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0a4b60 tx=0x7fe14c0058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd3a59374e88a3f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:08.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.128+0000 7fe1497fa640 1 -- 172.21.15.120:0/722752906 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c0b5070 con 0x55aa725aaa30 2024-09-19T09:10:08.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.128+0000 7fe1497fa640 1 -- 172.21.15.120:0/722752906 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe14c09cb20 con 0x55aa725aaa30 2024-09-19T09:10:08.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.128+0000 7fe1497fa640 1 -- 172.21.15.120:0/722752906 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c0b5210 con 0x55aa725aaa30 2024-09-19T09:10:08.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.129+0000 7fe15b533740 1 -- 172.21.15.120:0/722752906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa725aaa30 msgr2=0x55aa725ace20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:08.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.129+0000 7fe15b533740 1 --2- 172.21.15.120:0/722752906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa725aaa30 0x55aa725ace20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0a4b60 tx=0x7fe14c0058d0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.129+0000 7fe15b533740 1 -- 172.21.15.120:0/722752906 shutdown_connections 2024-09-19T09:10:08.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.129+0000 7fe15b533740 1 --2- 172.21.15.120:0/722752906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa725aaa30 0x55aa725ace20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.129+0000 7fe15b533740 1 --2- 172.21.15.120:0/722752906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa725a0170 0x55aa725aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.129+0000 7fe15b533740 1 --2- 172.21.15.120:0/722752906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa7259f850 0x55aa7259fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.129+0000 7fe15b533740 1 -- 172.21.15.120:0/722752906 >> 172.21.15.120:0/722752906 conn(0x55aa7259bde0 msgr2=0x55aa7259c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:08.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.129+0000 7fe15b533740 1 -- 172.21.15.120:0/722752906 shutdown_connections 2024-09-19T09:10:08.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.129+0000 7fe15b533740 1 -- 172.21.15.120:0/722752906 wait complete. 2024-09-19T09:10:08.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.129+0000 7fe15b533740 1 -- start start 2024-09-19T09:10:08.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.129+0000 7fe15b533740 1 Processor -- start 2024-09-19T09:10:08.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.130+0000 7fe15b533740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa7259f850 0x55aa725a78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:08.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.130+0000 7fe15b533740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa725a0170 0x55aa725a7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:08.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.130+0000 7fe15b533740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa725aaa30 0x55aa725a8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:08.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.130+0000 7fe15b533740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55aa725ad600 con 0x55aa7259f850 2024-09-19T09:10:08.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.130+0000 7fe15b533740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55aa7248c900 con 0x55aa725a0170 2024-09-19T09:10:08.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.130+0000 7fe15b533740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55aa725ad780 con 0x55aa725aaa30 2024-09-19T09:10:08.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.130+0000 7fe15b533740 10 client.0 _ll_register_callbacks cb 0x55aa7248a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:08.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.130+0000 7fe152d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa7259f850 0x55aa725a78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:08.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.130+0000 7fe152d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa7259f850 0x55aa725a78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48448/0 (socket says 172.21.15.120:48448) 2024-09-19T09:10:08.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.130+0000 7fe152d9d640 1 -- 172.21.15.120:0/1575318526 learned_addr learned my addr 172.21.15.120:0/1575318526 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:08.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.130+0000 7fe14bfff640 1 --2- 172.21.15.120:0/1575318526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa725a0170 0x55aa725a7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:08.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.130+0000 7fe15359e640 1 --2- 172.21.15.120:0/1575318526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa725aaa30 0x55aa725a8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:08.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.130+0000 7fe15359e640 1 -- 172.21.15.120:0/1575318526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa725a0170 msgr2=0x55aa725a7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:08.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.130+0000 7fe15359e640 1 --2- 172.21.15.120:0/1575318526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa725a0170 0x55aa725a7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.130+0000 7fe15359e640 1 -- 172.21.15.120:0/1575318526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa7259f850 msgr2=0x55aa725a78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.130+0000 7fe15359e640 1 --2- 172.21.15.120:0/1575318526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa7259f850 0x55aa725a78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.130+0000 7fe15359e640 1 -- 172.21.15.120:0/1575318526 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa724ccc50 con 0x55aa725aaa30 2024-09-19T09:10:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.130+0000 7fe152d9d640 1 --2- 172.21.15.120:0/1575318526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa7259f850 0x55aa725a78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:10:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.130+0000 7fe15359e640 1 --2- 172.21.15.120:0/1575318526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa725aaa30 0x55aa725a8330 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55aa7259d160 tx=0x7fe14c0cd890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66809de3f1fb014b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.130+0000 7fe13a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:10:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.130+0000 7fe13a7fc640 1 -- 172.21.15.120:0/1575318526 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c0b5040 con 0x55aa725aaa30 2024-09-19T09:10:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.130+0000 7fe15b533740 1 -- 172.21.15.120:0/1575318526 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe14c09cb20 con 0x55aa725aaa30 2024-09-19T09:10:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.130+0000 7fe13a7fc640 1 -- 172.21.15.120:0/1575318526 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fe14c0a4e00 con 0x55aa725aaa30 2024-09-19T09:10:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.130+0000 7fe13a7fc640 1 -- 172.21.15.120:0/1575318526 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe14c080030 con 0x55aa725aaa30 2024-09-19T09:10:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.130+0000 7fe15b533740 10 client.27076 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.131+0000 7fe15b533740 1 -- 172.21.15.120:0/1575318526 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55aa725ad600 con 0x55aa725aaa30 2024-09-19T09:10:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.131+0000 7fe13a7fc640 1 -- 172.21.15.120:0/1575318526 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe14c09cb20 con 0x55aa725aaa30 2024-09-19T09:10:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.131+0000 7fe13a7fc640 1 -- 172.21.15.120:0/1575318526 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe14c005380 con 0x55aa725aaa30 2024-09-19T09:10:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.131+0000 7fe15b533740 10 client.27076 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:08.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.131+0000 7fe15b533740 10 client.27076 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:10:08.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.131+0000 7fe15b533740 20 client.27076 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:08.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.131+0000 7fe15b533740 1 --2- 172.21.15.120:0/1575318526 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55aa725ba3f0 0x55aa725bc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:08.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.131+0000 7fe15b533740 4 client.27076 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:10:08.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.131+0000 7fe15b533740 1 -- 172.21.15.120:0/1575318526 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa72511890 con 0x55aa725ba3f0 2024-09-19T09:10:08.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.131+0000 7fe152d9d640 1 --2- 172.21.15.120:0/1575318526 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55aa725ba3f0 0x55aa725bc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:08.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.131+0000 7fe152d9d640 1 --2- 172.21.15.120:0/1575318526 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55aa725ba3f0 0x55aa725bc7e0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1da96b5c451763c server_cookie=9a008ef44d52d9b0 in_seq=0 out_seq=0 2024-09-19T09:10:08.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.131+0000 7fe13a7fc640 10 client.27076 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:08.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.132+0000 7fe13a7fc640 1 -- 172.21.15.120:0/1575318526 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aa72511890 con 0x55aa725ba3f0 2024-09-19T09:10:08.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.132+0000 7fe13a7fc640 10 client.27076 handle_command_reply: tid=0 2024-09-19T09:10:08.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.133+0000 7fe15b533740 1 client.27076 shutdown 2024-09-19T09:10:08.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.133+0000 7fe15b533740 1 -- 172.21.15.120:0/1575318526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa725aaa30 msgr2=0x55aa725a8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:08.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.133+0000 7fe15b533740 1 --2- 172.21.15.120:0/1575318526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa725aaa30 0x55aa725a8330 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55aa7259d160 tx=0x7fe14c0cd890 comp rx=0 tx=0).stop 2024-09-19T09:10:08.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.133+0000 7fe15b533740 1 -- 172.21.15.120:0/1575318526 shutdown_connections 2024-09-19T09:10:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.133+0000 7fe15b533740 1 --2- 172.21.15.120:0/1575318526 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55aa725ba3f0 0x55aa725bc7e0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.133+0000 7fe15b533740 1 --2- 172.21.15.120:0/1575318526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa725aaa30 0x55aa725a8330 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.133+0000 7fe15b533740 1 --2- 172.21.15.120:0/1575318526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa725a0170 0x55aa725a7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.133+0000 7fe13a7fc640 0 client.27076 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.133+0000 7fe15b533740 1 --2- 172.21.15.120:0/1575318526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa7259f850 0x55aa725a78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.133+0000 7fe15b533740 1 -- 172.21.15.120:0/1575318526 >> 172.21.15.120:0/1575318526 conn(0x55aa7259bde0 msgr2=0x55aa725a1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.133+0000 7fe15b533740 1 -- 172.21.15.120:0/1575318526 shutdown_connections 2024-09-19T09:10:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.133+0000 7fe15b533740 1 -- 172.21.15.120:0/1575318526 wait complete. 2024-09-19T09:10:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.133+0000 7fe15b533740 20 client.27076 trim_cache size 0 max 16384 2024-09-19T09:10:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.150+0000 7fe15b533740 1 Processor -- start 2024-09-19T09:10:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.150+0000 7fe15b533740 1 -- start start 2024-09-19T09:10:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.150+0000 7fe15b533740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa7259f850 0x55aa725a8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.150+0000 7fe15b533740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa725a0170 0x55aa725a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.150+0000 7fe15b533740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa725aaa30 0x55aa725c1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.150+0000 7fe15b533740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe1541af830 con 0x55aa725aaa30 2024-09-19T09:10:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.150+0000 7fe15b533740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe1541b1370 con 0x55aa725a0170 2024-09-19T09:10:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.150+0000 7fe15b533740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe1541aff00 con 0x55aa7259f850 2024-09-19T09:10:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.151+0000 7fe15359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa725aaa30 0x55aa725c1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.151+0000 7fe152d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa7259f850 0x55aa725a8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.151+0000 7fe15359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa725aaa30 0x55aa725c1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48462/0 (socket says 172.21.15.120:48462) 2024-09-19T09:10:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.151+0000 7fe14bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa725a0170 0x55aa725a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.151+0000 7fe152d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa7259f850 0x55aa725a8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:49136/0 (socket says 172.21.15.120:49136) 2024-09-19T09:10:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.151+0000 7fe15359e640 1 -- 172.21.15.120:0/2381124669 learned_addr learned my addr 172.21.15.120:0/2381124669 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.151+0000 7fe15359e640 1 -- 172.21.15.120:0/2381124669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa7259f850 msgr2=0x55aa725a8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.151+0000 7fe15359e640 1 --2- 172.21.15.120:0/2381124669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa7259f850 0x55aa725a8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.151+0000 7fe15359e640 1 -- 172.21.15.120:0/2381124669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa725a0170 msgr2=0x55aa725a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.151+0000 7fe15359e640 1 --2- 172.21.15.120:0/2381124669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa725a0170 0x55aa725a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.151+0000 7fe15359e640 1 -- 172.21.15.120:0/2381124669 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa7259bc10 con 0x55aa725aaa30 2024-09-19T09:10:08.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.151+0000 7fe152d9d640 1 --2- 172.21.15.120:0/2381124669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa7259f850 0x55aa725a8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:10:08.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.151+0000 7fe15359e640 1 --2- 172.21.15.120:0/2381124669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa725aaa30 0x55aa725c1ea0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fe14c09c5e0 tx=0x7fe14c0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daca44d23fb30d43 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:08.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.151+0000 7fe148ff9640 1 -- 172.21.15.120:0/2381124669 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c0b5040 con 0x55aa725aaa30 2024-09-19T09:10:08.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.151+0000 7fe148ff9640 1 -- 172.21.15.120:0/2381124669 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe14c0b53f0 con 0x55aa725aaa30 2024-09-19T09:10:08.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.151+0000 7fe148ff9640 1 -- 172.21.15.120:0/2381124669 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c0a5090 con 0x55aa725aaa30 2024-09-19T09:10:08.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.152+0000 7fe15b533740 1 -- 172.21.15.120:0/2381124669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa725aaa30 msgr2=0x55aa725c1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:08.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.152+0000 7fe15b533740 1 --2- 172.21.15.120:0/2381124669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa725aaa30 0x55aa725c1ea0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fe14c09c5e0 tx=0x7fe14c0058d0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.152+0000 7fe15b533740 1 -- 172.21.15.120:0/2381124669 shutdown_connections 2024-09-19T09:10:08.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.152+0000 7fe15b533740 1 --2- 172.21.15.120:0/2381124669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa725aaa30 0x55aa725c1ea0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.152+0000 7fe15b533740 1 --2- 172.21.15.120:0/2381124669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa725a0170 0x55aa725a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.152+0000 7fe15b533740 1 --2- 172.21.15.120:0/2381124669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa7259f850 0x55aa725a8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.152+0000 7fe15b533740 1 -- 172.21.15.120:0/2381124669 >> 172.21.15.120:0/2381124669 conn(0x55aa7259bde0 msgr2=0x55aa725a99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:08.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.152+0000 7fe15b533740 1 -- 172.21.15.120:0/2381124669 shutdown_connections 2024-09-19T09:10:08.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.152+0000 7fe15b533740 1 -- 172.21.15.120:0/2381124669 wait complete. 2024-09-19T09:10:08.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.153+0000 7fe15b533740 1 -- start start 2024-09-19T09:10:08.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.153+0000 7fe15b533740 1 Processor -- start 2024-09-19T09:10:08.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.153+0000 7fe15b533740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa7259f850 0x55aa725b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:08.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.153+0000 7fe15b533740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa725a0170 0x55aa725a4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:08.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.153+0000 7fe15b533740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa725aaa30 0x55aa725a4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:08.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.153+0000 7fe15b533740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe1541b0ca0 con 0x55aa725a0170 2024-09-19T09:10:08.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.153+0000 7fe15b533740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe1541b05d0 con 0x55aa725aaa30 2024-09-19T09:10:08.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.153+0000 7fe15b533740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe1541ad7f0 con 0x55aa7259f850 2024-09-19T09:10:08.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.153+0000 7fe15b533740 10 client.0 _ll_register_callbacks cb 0x7fe14c004e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:08.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.153+0000 7fe14bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa725a0170 0x55aa725a4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:08.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.153+0000 7fe15359e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa725aaa30 0x55aa725a4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:08.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.153+0000 7fe14bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa725a0170 0x55aa725a4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48476/0 (socket says 172.21.15.120:48476) 2024-09-19T09:10:08.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.153+0000 7fe15359e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa725aaa30 0x55aa725a4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:46868/0 (socket says 172.21.15.120:46868) 2024-09-19T09:10:08.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.153+0000 7fe14bfff640 1 -- 172.21.15.120:0/2421958554 learned_addr learned my addr 172.21.15.120:0/2421958554 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:08.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.153+0000 7fe152d9d640 1 --2- 172.21.15.120:0/2421958554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa7259f850 0x55aa725b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:08.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.153+0000 7fe15359e640 1 -- 172.21.15.120:0/2421958554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa7259f850 msgr2=0x55aa725b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:08.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.153+0000 7fe15359e640 1 --2- 172.21.15.120:0/2421958554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa7259f850 0x55aa725b30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.153+0000 7fe15359e640 1 -- 172.21.15.120:0/2421958554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa725a0170 msgr2=0x55aa725a4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:08.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.153+0000 7fe15359e640 1 --2- 172.21.15.120:0/2421958554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa725a0170 0x55aa725a4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.153+0000 7fe15359e640 1 -- 172.21.15.120:0/2421958554 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa725a52c0 con 0x55aa725aaa30 2024-09-19T09:10:08.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.153+0000 7fe14bfff640 1 --2- 172.21.15.120:0/2421958554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa725a0170 0x55aa725a4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:10:08.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.154+0000 7fe15359e640 1 --2- 172.21.15.120:0/2421958554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa725aaa30 0x55aa725a4a90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe14c098df0 tx=0x7fe14c009a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b9a41daca4b2cce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:08.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.154+0000 7fe13a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:10:08.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.154+0000 7fe13a7fc640 1 -- 172.21.15.120:0/2421958554 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c0b5040 con 0x55aa725aaa30 2024-09-19T09:10:08.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.154+0000 7fe15b533740 1 -- 172.21.15.120:0/2421958554 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe14c0b53f0 con 0x55aa725aaa30 2024-09-19T09:10:08.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.154+0000 7fe13a7fc640 1 -- 172.21.15.120:0/2421958554 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fe14c081070 con 0x55aa725aaa30 2024-09-19T09:10:08.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.154+0000 7fe13a7fc640 1 -- 172.21.15.120:0/2421958554 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe14c0a07d0 con 0x55aa725aaa30 2024-09-19T09:10:08.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.154+0000 7fe15b533740 10 client.27188 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:08.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.154+0000 7fe15b533740 1 -- 172.21.15.120:0/2421958554 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55aa725a52c0 con 0x55aa725aaa30 2024-09-19T09:10:08.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.154+0000 7fe13a7fc640 1 -- 172.21.15.120:0/2421958554 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe14c0b53f0 con 0x55aa725aaa30 2024-09-19T09:10:08.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.154+0000 7fe13a7fc640 1 -- 172.21.15.120:0/2421958554 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe14c0a4c00 con 0x55aa725aaa30 2024-09-19T09:10:08.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.154+0000 7fe15b533740 10 client.27188 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:08.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.154+0000 7fe15b533740 10 client.27188 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:10:08.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.154+0000 7fe15b533740 20 client.27188 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:08.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.154+0000 7fe15b533740 1 --2- 172.21.15.120:0/2421958554 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55aa726250e0 0x55aa726274d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:08.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.154+0000 7fe15b533740 4 client.27188 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}] 2024-09-19T09:10:08.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.154+0000 7fe15b533740 1 -- 172.21.15.120:0/2421958554 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}) -- 0x55aa7250bbe0 con 0x55aa726250e0 2024-09-19T09:10:08.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.155+0000 7fe152d9d640 1 --2- 172.21.15.120:0/2421958554 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55aa726250e0 0x55aa726274d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:08.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.155+0000 7fe13a7fc640 10 client.27188 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:08.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.155+0000 7fe152d9d640 1 --2- 172.21.15.120:0/2421958554 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55aa726250e0 0x55aa726274d0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5405c91d3fd45e8 server_cookie=d22d382e64d57105 in_seq=0 out_seq=0 2024-09-19T09:10:08.198 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:10:08.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.240+0000 7fe13a7fc640 1 -- 172.21.15.120:0/2421958554 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55aa7250bbe0 con 0x55aa726250e0 2024-09-19T09:10:08.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.240+0000 7fe13a7fc640 10 client.27188 handle_command_reply: tid=0 2024-09-19T09:10:08.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.240+0000 7fe15b533740 1 client.27188 shutdown 2024-09-19T09:10:08.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.240+0000 7fe15b533740 1 -- 172.21.15.120:0/2421958554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa725aaa30 msgr2=0x55aa725a4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:08.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.240+0000 7fe15b533740 1 --2- 172.21.15.120:0/2421958554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa725aaa30 0x55aa725a4a90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe14c098df0 tx=0x7fe14c009a70 comp rx=0 tx=0).stop 2024-09-19T09:10:08.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.240+0000 7fe15b533740 1 -- 172.21.15.120:0/2421958554 shutdown_connections 2024-09-19T09:10:08.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.240+0000 7fe15b533740 1 --2- 172.21.15.120:0/2421958554 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55aa726250e0 0x55aa726274d0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.240+0000 7fe15b533740 1 --2- 172.21.15.120:0/2421958554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa725aaa30 0x55aa725a4a90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.240+0000 7fe15b533740 1 --2- 172.21.15.120:0/2421958554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa725a0170 0x55aa725a4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.240+0000 7fe15b533740 1 --2- 172.21.15.120:0/2421958554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aa7259f850 0x55aa725b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.240+0000 7fe13a7fc640 0 client.27188 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:08.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.240+0000 7fe15b533740 1 -- 172.21.15.120:0/2421958554 >> 172.21.15.120:0/2421958554 conn(0x55aa7259bde0 msgr2=0x55aa725a6870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:08.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.240+0000 7fe15b533740 1 -- 172.21.15.120:0/2421958554 shutdown_connections 2024-09-19T09:10:08.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.240+0000 7fe15b533740 1 -- 172.21.15.120:0/2421958554 wait complete. 2024-09-19T09:10:08.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.240+0000 7fe15b533740 20 client.27188 trim_cache size 0 max 16384 2024-09-19T09:10:08.243 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:10:08.243 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 30, 2024-09-19T09:10:08.243 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:10:08.243 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 26, 2024-09-19T09:10:08.243 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:10:08.243 INFO:teuthology.orchestra.run.smithi120.stdout: "aeb2f3f6": { 2024-09-19T09:10:08.244 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 26, 2024-09-19T09:10:08.244 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:10:08.244 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:10:08.244 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:10:08.244 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:10:08.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:10:08.244 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:10:08.244 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 145.8, 2024-09-19T09:10:08.244 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:10:08.244 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:10:08.244 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:10:08.244 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:10:08.244 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:10:08.244 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:10:08.245 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:10:08.245 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:10:08.245 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:10:08.245 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:10:08.245 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:10:08.245 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:10:08.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.241+0000 7fe15915a640 1 -- 172.21.15.120:0/1031600626 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe1400a3240 msgr2=0x7fe1400a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:08.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.241+0000 7fe15915a640 1 --2- 172.21.15.120:0/1031600626 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe1400a3240 0x7fe1400a5700 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009230 tx=0x7fe13c005d50 comp rx=0 tx=0).stop 2024-09-19T09:10:08.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.241+0000 7fe15915a640 1 -- 172.21.15.120:0/1031600626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe154106f10 msgr2=0x7fe154074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:08.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.241+0000 7fe15915a640 1 --2- 172.21.15.120:0/1031600626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe154106f10 0x7fe154074f70 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fe14c00eb10 tx=0x7fe14c00c2d0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.241+0000 7fe15915a640 1 -- 172.21.15.120:0/1031600626 shutdown_connections 2024-09-19T09:10:08.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.241+0000 7fe15915a640 1 --2- 172.21.15.120:0/1031600626 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe1400a3240 0x7fe1400a5700 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.241+0000 7fe15915a640 1 --2- 172.21.15.120:0/1031600626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe154106f10 0x7fe154074f70 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.241+0000 7fe15915a640 1 --2- 172.21.15.120:0/1031600626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe154102790 0x7fe154074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.241+0000 7fe15915a640 1 --2- 172.21.15.120:0/1031600626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe154101de0 0x7fe1540744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.241+0000 7fe15915a640 1 -- 172.21.15.120:0/1031600626 >> 172.21.15.120:0/1031600626 conn(0x7fe1540fd8c0 msgr2=0x7fe1540fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:08.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.241+0000 7fe15915a640 1 -- 172.21.15.120:0/1031600626 shutdown_connections 2024-09-19T09:10:08.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.241+0000 7fe15915a640 1 -- 172.21.15.120:0/1031600626 wait complete. 2024-09-19T09:10:08.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.742+0000 7ff8e45ba640 1 -- 172.21.15.120:0/2298966024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8dc102570 msgr2=0x7ff8dc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:08.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.742+0000 7ff8e45ba640 1 --2- 172.21.15.120:0/2298966024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8dc102570 0x7ff8dc1029b0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009900 tx=0x7ff8d002a480 comp rx=0 tx=0).stop 2024-09-19T09:10:08.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.742+0000 7ff8e45ba640 1 -- 172.21.15.120:0/2298966024 shutdown_connections 2024-09-19T09:10:08.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.742+0000 7ff8e45ba640 1 --2- 172.21.15.120:0/2298966024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8dc102ef0 0x7ff8dc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.742+0000 7ff8e45ba640 1 --2- 172.21.15.120:0/2298966024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8dc102570 0x7ff8dc1029b0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.742+0000 7ff8e45ba640 1 --2- 172.21.15.120:0/2298966024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8dc100e20 0x7ff8dc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.742+0000 7ff8e45ba640 1 -- 172.21.15.120:0/2298966024 >> 172.21.15.120:0/2298966024 conn(0x7ff8dc0fc7d0 msgr2=0x7ff8dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:08.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.743+0000 7ff8e45ba640 1 -- 172.21.15.120:0/2298966024 shutdown_connections 2024-09-19T09:10:08.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.743+0000 7ff8e45ba640 1 -- 172.21.15.120:0/2298966024 wait complete. 2024-09-19T09:10:08.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.743+0000 7ff8e45ba640 1 Processor -- start 2024-09-19T09:10:08.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.743+0000 7ff8e45ba640 1 -- start start 2024-09-19T09:10:08.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.743+0000 7ff8e45ba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8dc100e20 0x7ff8dc198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:08.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.743+0000 7ff8e45ba640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8dc102570 0x7ff8dc198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:08.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.743+0000 7ff8e45ba640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8dc102ef0 0x7ff8dc19ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:08.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.743+0000 7ff8e45ba640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff8dc110fc0 con 0x7ff8dc102570 2024-09-19T09:10:08.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.743+0000 7ff8e45ba640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff8dc110e40 con 0x7ff8dc102ef0 2024-09-19T09:10:08.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.743+0000 7ff8e45ba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff8dc111140 con 0x7ff8dc100e20 2024-09-19T09:10:08.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.743+0000 7ff8e1b55640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8dc102570 0x7ff8dc198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:08.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.744+0000 7ff8e1b55640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8dc102570 0x7ff8dc198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48500/0 (socket says 172.21.15.120:48500) 2024-09-19T09:10:08.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.744+0000 7ff8e1b55640 1 -- 172.21.15.120:0/3653900435 learned_addr learned my addr 172.21.15.120:0/3653900435 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:08.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.744+0000 7ff8e2356640 1 --2- 172.21.15.120:0/3653900435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8dc100e20 0x7ff8dc198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:08.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.744+0000 7ff8e1b55640 1 -- 172.21.15.120:0/3653900435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8dc100e20 msgr2=0x7ff8dc198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:08.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.744+0000 7ff8e2b57640 1 --2- 172.21.15.120:0/3653900435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8dc102ef0 0x7ff8dc19ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:08.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.744+0000 7ff8e1b55640 1 --2- 172.21.15.120:0/3653900435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8dc100e20 0x7ff8dc198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.744+0000 7ff8e1b55640 1 -- 172.21.15.120:0/3653900435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8dc102ef0 msgr2=0x7ff8dc19ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:08.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.744+0000 7ff8e1b55640 1 --2- 172.21.15.120:0/3653900435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8dc102ef0 0x7ff8dc19ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.744+0000 7ff8e1b55640 1 -- 172.21.15.120:0/3653900435 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8dc19d530 con 0x7ff8dc102570 2024-09-19T09:10:08.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.744+0000 7ff8e1b55640 1 --2- 172.21.15.120:0/3653900435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8dc102570 0x7ff8dc198ac0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0006010 tx=0x7ff8d0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:08.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.744+0000 7ff8db7fe640 1 -- 172.21.15.120:0/3653900435 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d003f070 con 0x7ff8dc102570 2024-09-19T09:10:08.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.744+0000 7ff8db7fe640 1 -- 172.21.15.120:0/3653900435 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8d0031c30 con 0x7ff8dc102570 2024-09-19T09:10:08.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.744+0000 7ff8e45ba640 1 -- 172.21.15.120:0/3653900435 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8dc19d7c0 con 0x7ff8dc102570 2024-09-19T09:10:08.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.744+0000 7ff8db7fe640 1 -- 172.21.15.120:0/3653900435 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0035030 con 0x7ff8dc102570 2024-09-19T09:10:08.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.744+0000 7ff8e45ba640 1 -- 172.21.15.120:0/3653900435 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8dc1a5100 con 0x7ff8dc102570 2024-09-19T09:10:08.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.745+0000 7ff8e45ba640 1 -- 172.21.15.120:0/3653900435 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8dc101200 con 0x7ff8dc102570 2024-09-19T09:10:08.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.746+0000 7ff8db7fe640 1 -- 172.21.15.120:0/3653900435 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7ff8d0042070 con 0x7ff8dc102570 2024-09-19T09:10:08.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.747+0000 7ff8db7fe640 1 --2- 172.21.15.120:0/3653900435 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff8c40a32c0 0x7ff8c40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:08.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.747+0000 7ff8e2356640 1 --2- 172.21.15.120:0/3653900435 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff8c40a32c0 0x7ff8c40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:08.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.747+0000 7ff8db7fe640 1 -- 172.21.15.120:0/3653900435 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7ff8d00f0140 con 0x7ff8dc102570 2024-09-19T09:10:08.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.747+0000 7ff8e2356640 1 --2- 172.21.15.120:0/3653900435 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff8c40a32c0 0x7ff8c40a5780 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc00a400 tx=0x7ff8cc00d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:08.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.751+0000 7ff8db7fe640 1 -- 172.21.15.120:0/3653900435 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff8d0031de0 con 0x7ff8dc102570 2024-09-19T09:10:08.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.957+0000 7ff8e45ba640 1 -- 172.21.15.120:0/3653900435 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8dc10a0a0 con 0x7ff8dc102570 2024-09-19T09:10:08.959 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:08 smithi120 ceph-mon[31150]: pgmap v560: 97 pgs: 97 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.9 MiB/s wr, 396 op/s 2024-09-19T09:10:08.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.958+0000 7ff8db7fe640 1 -- 172.21.15.120:0/3653900435 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11985 (secure 0 0 0) 0x7ff8d00bce60 con 0x7ff8dc102570 2024-09-19T09:10:08.960 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:10:08.960 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":30,"btime":"2024-09-19T09:07:14:812065+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:07:14.812046+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26351":{"gid":26351,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/364941779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":364941779},{"type":"v1","addr":"172.21.15.153:6835","nonce":364941779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:10:08.960 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 30 2024-09-19T09:10:08.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.960+0000 7ff8e45ba640 1 -- 172.21.15.120:0/3653900435 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff8c40a32c0 msgr2=0x7ff8c40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:08.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.960+0000 7ff8e45ba640 1 --2- 172.21.15.120:0/3653900435 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff8c40a32c0 0x7ff8c40a5780 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc00a400 tx=0x7ff8cc00d040 comp rx=0 tx=0).stop 2024-09-19T09:10:08.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.960+0000 7ff8e45ba640 1 -- 172.21.15.120:0/3653900435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8dc102570 msgr2=0x7ff8dc198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:08.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.960+0000 7ff8e45ba640 1 --2- 172.21.15.120:0/3653900435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8dc102570 0x7ff8dc198ac0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0006010 tx=0x7ff8d0003c10 comp rx=0 tx=0).stop 2024-09-19T09:10:08.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.960+0000 7ff8e45ba640 1 -- 172.21.15.120:0/3653900435 shutdown_connections 2024-09-19T09:10:08.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.960+0000 7ff8e45ba640 1 --2- 172.21.15.120:0/3653900435 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff8c40a32c0 0x7ff8c40a5780 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.960+0000 7ff8e45ba640 1 --2- 172.21.15.120:0/3653900435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8dc102ef0 0x7ff8dc19ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.960+0000 7ff8e45ba640 1 --2- 172.21.15.120:0/3653900435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8dc102570 0x7ff8dc198ac0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.960+0000 7ff8e45ba640 1 --2- 172.21.15.120:0/3653900435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8dc100e20 0x7ff8dc198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:08.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.960+0000 7ff8e45ba640 1 -- 172.21.15.120:0/3653900435 >> 172.21.15.120:0/3653900435 conn(0x7ff8dc0fc7d0 msgr2=0x7ff8dc06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:08.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.960+0000 7ff8e45ba640 1 -- 172.21.15.120:0/3653900435 shutdown_connections 2024-09-19T09:10:08.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:08.960+0000 7ff8e45ba640 1 -- 172.21.15.120:0/3653900435 wait complete. 2024-09-19T09:10:09.056 INFO:tasks.workunit.client.0.smithi120.stdout: 1 698442 10.46 MB/sec execute 244 sec latency 864.737 ms 2024-09-19T09:10:09.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:08 smithi196 ceph-mon[35150]: pgmap v560: 97 pgs: 97 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.9 MiB/s wr, 396 op/s 2024-09-19T09:10:09.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:08 smithi153 ceph-mon[35071]: pgmap v560: 97 pgs: 97 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.9 MiB/s wr, 396 op/s 2024-09-19T09:10:09.901 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'aeb2f3f6', quiesce took 5.5 seconds. Will release after: 20.25 2024-09-19T09:10:10.056 INFO:tasks.workunit.client.0.smithi120.stdout: 1 698442 10.42 MB/sec execute 245 sec latency 1864.804 ms 2024-09-19T09:10:10.169 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:09 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3653900435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:10:10.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:09 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3653900435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:10:10.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:09 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3653900435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:10:10.203 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:10:10.451 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:10:11.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.980+0000 7fe5da131640 1 -- 172.21.15.120:0/329731699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5d4100a80 msgr2=0x7fe5d4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:11.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.980+0000 7fe5da131640 1 --2- 172.21.15.120:0/329731699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5d4100a80 0x7fe5d4100ee0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009930 tx=0x7fe5bc02a440 comp rx=0 tx=0).stop 2024-09-19T09:10:11.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.980+0000 7fe5da131640 1 -- 172.21.15.120:0/329731699 shutdown_connections 2024-09-19T09:10:11.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.980+0000 7fe5da131640 1 --2- 172.21.15.120:0/329731699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5d4101420 0x7fe5d410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.980+0000 7fe5da131640 1 --2- 172.21.15.120:0/329731699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5d4100a80 0x7fe5d4100ee0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.980+0000 7fe5da131640 1 --2- 172.21.15.120:0/329731699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5d4106a80 0x7fe5d4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.980+0000 7fe5da131640 1 -- 172.21.15.120:0/329731699 >> 172.21.15.120:0/329731699 conn(0x7fe5d40fc7a0 msgr2=0x7fe5d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:11.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.980+0000 7fe5da131640 1 -- 172.21.15.120:0/329731699 shutdown_connections 2024-09-19T09:10:11.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.980+0000 7fe5da131640 1 -- 172.21.15.120:0/329731699 wait complete. 2024-09-19T09:10:11.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.980+0000 7fe5da131640 1 Processor -- start 2024-09-19T09:10:11.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.980+0000 7fe5da131640 1 -- start start 2024-09-19T09:10:11.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.981+0000 7fe5da131640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5d4100a80 0x7fe5d4198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:11.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.981+0000 7fe5da131640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5d4101420 0x7fe5d4198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:11.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.981+0000 7fe5da131640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5d4106a80 0x7fe5d419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:11.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.981+0000 7fe5da131640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d4110f30 con 0x7fe5d4101420 2024-09-19T09:10:11.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.981+0000 7fe5da131640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d4110db0 con 0x7fe5d4106a80 2024-09-19T09:10:11.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.981+0000 7fe5da131640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d41110b0 con 0x7fe5d4100a80 2024-09-19T09:10:11.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.981+0000 7fe5caffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5d4101420 0x7fe5d4198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:11.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.981+0000 7fe5caffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5d4101420 0x7fe5d4198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48528/0 (socket says 172.21.15.120:48528) 2024-09-19T09:10:11.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.981+0000 7fe5caffd640 1 -- 172.21.15.120:0/2372448653 learned_addr learned my addr 172.21.15.120:0/2372448653 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:11.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.981+0000 7fe5caffd640 1 -- 172.21.15.120:0/2372448653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5d4100a80 msgr2=0x7fe5d4198550 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T09:10:11.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.981+0000 7fe5caffd640 1 --2- 172.21.15.120:0/2372448653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5d4100a80 0x7fe5d4198550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.981+0000 7fe5caffd640 1 -- 172.21.15.120:0/2372448653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5d4106a80 msgr2=0x7fe5d419ce20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T09:10:11.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.981+0000 7fe5caffd640 1 --2- 172.21.15.120:0/2372448653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5d4106a80 0x7fe5d419ce20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.981+0000 7fe5caffd640 1 -- 172.21.15.120:0/2372448653 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d419d500 con 0x7fe5d4101420 2024-09-19T09:10:11.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.981+0000 7fe5caffd640 1 --2- 172.21.15.120:0/2372448653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5d4101420 0x7fe5d4198a90 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc02a960 tx=0x7fe5bc003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:11.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.981+0000 7fe5d17fa640 1 -- 172.21.15.120:0/2372448653 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc03f070 con 0x7fe5d4101420 2024-09-19T09:10:11.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.982+0000 7fe5da131640 1 -- 172.21.15.120:0/2372448653 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5d419d790 con 0x7fe5d4101420 2024-09-19T09:10:11.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.982+0000 7fe5d17fa640 1 -- 172.21.15.120:0/2372448653 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5bc031c90 con 0x7fe5d4101420 2024-09-19T09:10:11.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.982+0000 7fe5d17fa640 1 -- 172.21.15.120:0/2372448653 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc035030 con 0x7fe5d4101420 2024-09-19T09:10:11.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.982+0000 7fe5da131640 1 -- 172.21.15.120:0/2372448653 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5d41a50d0 con 0x7fe5d4101420 2024-09-19T09:10:11.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.984+0000 7fe5d17fa640 1 -- 172.21.15.120:0/2372448653 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe5bc042070 con 0x7fe5d4101420 2024-09-19T09:10:11.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.984+0000 7fe5dc50a740 1 Processor -- start 2024-09-19T09:10:11.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.984+0000 7fe5dc50a740 1 -- start start 2024-09-19T09:10:11.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.984+0000 7fe5dc50a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c3525b40 0x5573c3524090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:11.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.984+0000 7fe5d17fa640 1 --2- 172.21.15.120:0/2372448653 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5c00a3240 0x7fe5c00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:11.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.984+0000 7fe5dc50a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c35245d0 0x5573c3524a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:11.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.984+0000 7fe5dc50a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573c352f370 0x5573c3531760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:11.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.984+0000 7fe5dc50a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5573c348a2f0 con 0x5573c3525b40 2024-09-19T09:10:11.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.984+0000 7fe5dc50a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5573c3431cf0 con 0x5573c352f370 2024-09-19T09:10:11.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.984+0000 7fe5dc50a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5573c34862b0 con 0x5573c35245d0 2024-09-19T09:10:11.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.985+0000 7fe5d3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c3525b40 0x5573c3524090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:11.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.985+0000 7fe5d37fe640 1 --2- 172.21.15.120:0/2372448653 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5c00a3240 0x7fe5c00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:11.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.985+0000 7fe5d3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c3525b40 0x5573c3524090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48534/0 (socket says 172.21.15.120:48534) 2024-09-19T09:10:11.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.985+0000 7fe5d17fa640 1 -- 172.21.15.120:0/2372448653 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fe5bc0f0190 con 0x7fe5d4101420 2024-09-19T09:10:11.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.985+0000 7fe5d3fff640 1 -- 172.21.15.120:0/4088990855 learned_addr learned my addr 172.21.15.120:0/4088990855 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:11.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.985+0000 7fe5d3fff640 1 --2- 172.21.15.120:0/4088990855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573c352f370 0x5573c3531760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:11.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.985+0000 7fe5caffd640 1 --2- 172.21.15.120:0/4088990855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c35245d0 0x5573c3524a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:11.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.985+0000 7fe5d3fff640 1 -- 172.21.15.120:0/4088990855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c35245d0 msgr2=0x5573c3524a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:11.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.985+0000 7fe5d3fff640 1 --2- 172.21.15.120:0/4088990855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c35245d0 0x5573c3524a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.985+0000 7fe5d3fff640 1 -- 172.21.15.120:0/4088990855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573c352f370 msgr2=0x5573c3531760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:11.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.985+0000 7fe5d3fff640 1 --2- 172.21.15.120:0/4088990855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573c352f370 0x5573c3531760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.985+0000 7fe5d3fff640 1 -- 172.21.15.120:0/4088990855 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573c34163a0 con 0x5573c3525b40 2024-09-19T09:10:11.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.985+0000 7fe5d37fe640 1 --2- 172.21.15.120:0/2372448653 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5c00a3240 0x7fe5c00a5700 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4009670 tx=0x7fe5c4006bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:11.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.985+0000 7fe5d3fff640 1 --2- 172.21.15.120:0/4088990855 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c3525b40 0x5573c3524090 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x5573c3525070 tx=0x7fe5cc009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517428ab7c64de39 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:11.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.985+0000 7fe5c9ffb640 1 -- 172.21.15.120:0/4088990855 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc004d20 con 0x5573c3525b40 2024-09-19T09:10:11.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.985+0000 7fe5c9ffb640 1 -- 172.21.15.120:0/4088990855 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5cc005500 con 0x5573c3525b40 2024-09-19T09:10:11.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.985+0000 7fe5c9ffb640 1 -- 172.21.15.120:0/4088990855 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc0138a0 con 0x5573c3525b40 2024-09-19T09:10:11.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.986+0000 7fe5dc50a740 1 -- 172.21.15.120:0/4088990855 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c3525b40 msgr2=0x5573c3524090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:11.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.986+0000 7fe5dc50a740 1 --2- 172.21.15.120:0/4088990855 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c3525b40 0x5573c3524090 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x5573c3525070 tx=0x7fe5cc009650 comp rx=0 tx=0).stop 2024-09-19T09:10:11.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.986+0000 7fe5dc50a740 1 -- 172.21.15.120:0/4088990855 shutdown_connections 2024-09-19T09:10:11.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.986+0000 7fe5dc50a740 1 --2- 172.21.15.120:0/4088990855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573c352f370 0x5573c3531760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.986+0000 7fe5dc50a740 1 --2- 172.21.15.120:0/4088990855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c35245d0 0x5573c3524a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.986+0000 7fe5dc50a740 1 --2- 172.21.15.120:0/4088990855 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c3525b40 0x5573c3524090 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.986+0000 7fe5dc50a740 1 -- 172.21.15.120:0/4088990855 >> 172.21.15.120:0/4088990855 conn(0x5573c34ed310 msgr2=0x5573c3520c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:11.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.986+0000 7fe5dc50a740 1 -- 172.21.15.120:0/4088990855 shutdown_connections 2024-09-19T09:10:11.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.986+0000 7fe5dc50a740 1 -- 172.21.15.120:0/4088990855 wait complete. 2024-09-19T09:10:11.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.987+0000 7fe5dc50a740 1 -- start start 2024-09-19T09:10:11.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.987+0000 7fe5dc50a740 1 Processor -- start 2024-09-19T09:10:11.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.987+0000 7fe5dc50a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c35245d0 0x5573c352ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:11.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.987+0000 7fe5dc50a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c3525b40 0x5573c352c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:11.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.987+0000 7fe5dc50a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573c352f370 0x5573c352c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:11.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.987+0000 7fe5dc50a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5573c3532070 con 0x5573c35245d0 2024-09-19T09:10:11.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.987+0000 7fe5dc50a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5573c3445060 con 0x5573c352f370 2024-09-19T09:10:11.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.987+0000 7fe5dc50a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5573c35321f0 con 0x5573c3525b40 2024-09-19T09:10:11.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.987+0000 7fe5dc50a740 10 client.0 _ll_register_callbacks cb 0x5573c31988c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:11.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.987+0000 7fe5d37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c35245d0 0x5573c352ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:11.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.987+0000 7fe5d37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c35245d0 0x5573c352ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48550/0 (socket says 172.21.15.120:48550) 2024-09-19T09:10:11.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.987+0000 7fe5d37fe640 1 -- 172.21.15.120:0/2529743104 learned_addr learned my addr 172.21.15.120:0/2529743104 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:11.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.988+0000 7fe5d3fff640 1 --2- 172.21.15.120:0/2529743104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573c352f370 0x5573c352c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:11.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.988+0000 7fe5d37fe640 1 -- 172.21.15.120:0/2529743104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c3525b40 msgr2=0x5573c352c0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:10:11.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.988+0000 7fe5d37fe640 1 --2- 172.21.15.120:0/2529743104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c3525b40 0x5573c352c0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.988+0000 7fe5d37fe640 1 -- 172.21.15.120:0/2529743104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573c352f370 msgr2=0x5573c352c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:11.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.988+0000 7fe5d37fe640 1 --2- 172.21.15.120:0/2529743104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573c352f370 0x5573c352c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.988+0000 7fe5d37fe640 1 -- 172.21.15.120:0/2529743104 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573c31fb3c0 con 0x5573c35245d0 2024-09-19T09:10:11.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.988+0000 7fe5d37fe640 1 --2- 172.21.15.120:0/2529743104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c35245d0 0x5573c352ee10 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4013fc0 tx=0x7fe5c40046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb62d3035c8f2aa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:11.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.988+0000 7fe5b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:11.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.988+0000 7fe5b2ffd640 1 -- 172.21.15.120:0/2529743104 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c401d5f0 con 0x5573c35245d0 2024-09-19T09:10:11.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.988+0000 7fe5b2ffd640 1 -- 172.21.15.120:0/2529743104 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fe5c401d790 con 0x5573c35245d0 2024-09-19T09:10:11.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.988+0000 7fe5dc50a740 1 -- 172.21.15.120:0/2529743104 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5cc005500 con 0x5573c35245d0 2024-09-19T09:10:11.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.988+0000 7fe5b2ffd640 1 -- 172.21.15.120:0/2529743104 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe5c401d990 con 0x5573c35245d0 2024-09-19T09:10:11.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.988+0000 7fe5dc50a740 10 client.17955 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:11.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.988+0000 7fe5dc50a740 1 -- 172.21.15.120:0/2529743104 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5573c353dc20 con 0x5573c35245d0 2024-09-19T09:10:11.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.988+0000 7fe5b2ffd640 1 -- 172.21.15.120:0/2529743104 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5cc005500 con 0x5573c35245d0 2024-09-19T09:10:11.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.988+0000 7fe5b2ffd640 1 -- 172.21.15.120:0/2529743104 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe5c4012070 con 0x5573c35245d0 2024-09-19T09:10:11.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.988+0000 7fe5dc50a740 10 client.17955 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:11.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.989+0000 7fe5dc50a740 10 client.17955 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:10:11.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.989+0000 7fe5dc50a740 20 client.17955 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:11.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.989+0000 7fe5dc50a740 1 --2- 172.21.15.120:0/2529743104 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5573c353f130 0x5573c3541560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:11.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.989+0000 7fe5dc50a740 4 client.17955 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:10:11.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.989+0000 7fe5dc50a740 1 -- 172.21.15.120:0/2529743104 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573c3497860 con 0x5573c353f130 2024-09-19T09:10:11.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.989+0000 7fe5caffd640 1 --2- 172.21.15.120:0/2529743104 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5573c353f130 0x5573c3541560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:11.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.989+0000 7fe5b2ffd640 10 client.17955 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:10:11.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.989+0000 7fe5caffd640 1 --2- 172.21.15.120:0/2529743104 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5573c353f130 0x5573c3541560 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a838a81d1e6a2920 server_cookie=8a5c65e046d96c37 in_seq=0 out_seq=0 2024-09-19T09:10:11.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.990+0000 7fe5b2ffd640 1 -- 172.21.15.120:0/2529743104 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5573c3497860 con 0x5573c353f130 2024-09-19T09:10:11.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.990+0000 7fe5b2ffd640 10 client.17955 handle_command_reply: tid=0 2024-09-19T09:10:11.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.990+0000 7fe5dc50a740 1 client.17955 shutdown 2024-09-19T09:10:11.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.990+0000 7fe5dc50a740 1 -- 172.21.15.120:0/2529743104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c35245d0 msgr2=0x5573c352ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:11.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.990+0000 7fe5dc50a740 1 --2- 172.21.15.120:0/2529743104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c35245d0 0x5573c352ee10 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4013fc0 tx=0x7fe5c40046e0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.990+0000 7fe5dc50a740 1 -- 172.21.15.120:0/2529743104 shutdown_connections 2024-09-19T09:10:11.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.990+0000 7fe5dc50a740 1 --2- 172.21.15.120:0/2529743104 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5573c353f130 0x5573c3541560 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.990+0000 7fe5dc50a740 1 --2- 172.21.15.120:0/2529743104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573c352f370 0x5573c352c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.990+0000 7fe5dc50a740 1 --2- 172.21.15.120:0/2529743104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c3525b40 0x5573c352c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.990+0000 7fe5b2ffd640 0 client.17955 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:10:11.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.990+0000 7fe5dc50a740 1 --2- 172.21.15.120:0/2529743104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c35245d0 0x5573c352ee10 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.990+0000 7fe5dc50a740 1 -- 172.21.15.120:0/2529743104 >> 172.21.15.120:0/2529743104 conn(0x5573c34ed310 msgr2=0x5573c3526960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:11.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.990+0000 7fe5dc50a740 1 -- 172.21.15.120:0/2529743104 shutdown_connections 2024-09-19T09:10:11.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.990+0000 7fe5dc50a740 1 -- 172.21.15.120:0/2529743104 wait complete. 2024-09-19T09:10:11.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:10.990+0000 7fe5dc50a740 20 client.17955 trim_cache size 0 max 16384 2024-09-19T09:10:11.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.003+0000 7fe5dc50a740 1 Processor -- start 2024-09-19T09:10:11.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.003+0000 7fe5dc50a740 1 -- start start 2024-09-19T09:10:11.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.003+0000 7fe5dc50a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c35245d0 0x5573c3525300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:11.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.003+0000 7fe5dc50a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573c3525b40 0x5573c3546e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:11.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.003+0000 7fe5dc50a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c352f370 0x5573c3547370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:11.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.003+0000 7fe5dc50a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d41ad650 con 0x5573c35245d0 2024-09-19T09:10:11.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.003+0000 7fe5dc50a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d41af190 con 0x5573c3525b40 2024-09-19T09:10:11.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.003+0000 7fe5dc50a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d41add20 con 0x5573c352f370 2024-09-19T09:10:11.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.003+0000 7fe5d37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c35245d0 0x5573c3525300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:11.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.003+0000 7fe5d37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c35245d0 0x5573c3525300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48558/0 (socket says 172.21.15.120:48558) 2024-09-19T09:10:11.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.003+0000 7fe5d37fe640 1 -- 172.21.15.120:0/3987715883 learned_addr learned my addr 172.21.15.120:0/3987715883 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:11.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.004+0000 7fe5caffd640 1 --2- 172.21.15.120:0/3987715883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573c3525b40 0x5573c3546e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:11.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.004+0000 7fe5d3fff640 1 --2- 172.21.15.120:0/3987715883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c352f370 0x5573c3547370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:11.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.004+0000 7fe5d37fe640 1 -- 172.21.15.120:0/3987715883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c352f370 msgr2=0x5573c3547370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:11.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.004+0000 7fe5d37fe640 1 --2- 172.21.15.120:0/3987715883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c352f370 0x5573c3547370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.004+0000 7fe5d37fe640 1 -- 172.21.15.120:0/3987715883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573c3525b40 msgr2=0x5573c3546e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:11.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.004+0000 7fe5d37fe640 1 --2- 172.21.15.120:0/3987715883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573c3525b40 0x5573c3546e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.004+0000 7fe5d37fe640 1 -- 172.21.15.120:0/3987715883 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573c352a610 con 0x5573c35245d0 2024-09-19T09:10:11.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.004+0000 7fe5caffd640 1 --2- 172.21.15.120:0/3987715883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573c3525b40 0x5573c3546e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:10:11.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.004+0000 7fe5d37fe640 1 --2- 172.21.15.120:0/3987715883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c35245d0 0x5573c3525300 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4000c00 tx=0x7fe5c4006aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a59e1129c30185 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:11.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.004+0000 7fe5c97fa640 1 -- 172.21.15.120:0/3987715883 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4037070 con 0x5573c35245d0 2024-09-19T09:10:11.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.004+0000 7fe5c97fa640 1 -- 172.21.15.120:0/3987715883 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5c4022070 con 0x5573c35245d0 2024-09-19T09:10:11.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.004+0000 7fe5c97fa640 1 -- 172.21.15.120:0/3987715883 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4026400 con 0x5573c35245d0 2024-09-19T09:10:11.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.005+0000 7fe5dc50a740 1 -- 172.21.15.120:0/3987715883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c35245d0 msgr2=0x5573c3525300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:11.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.005+0000 7fe5dc50a740 1 --2- 172.21.15.120:0/3987715883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c35245d0 0x5573c3525300 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4000c00 tx=0x7fe5c4006aa0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.005+0000 7fe5dc50a740 1 -- 172.21.15.120:0/3987715883 shutdown_connections 2024-09-19T09:10:11.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.005+0000 7fe5dc50a740 1 --2- 172.21.15.120:0/3987715883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c352f370 0x5573c3547370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.005+0000 7fe5dc50a740 1 --2- 172.21.15.120:0/3987715883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573c3525b40 0x5573c3546e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.005+0000 7fe5dc50a740 1 --2- 172.21.15.120:0/3987715883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c35245d0 0x5573c3525300 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.005+0000 7fe5dc50a740 1 -- 172.21.15.120:0/3987715883 >> 172.21.15.120:0/3987715883 conn(0x5573c34ed310 msgr2=0x5573c351ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:11.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.005+0000 7fe5dc50a740 1 -- 172.21.15.120:0/3987715883 shutdown_connections 2024-09-19T09:10:11.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.005+0000 7fe5dc50a740 1 -- 172.21.15.120:0/3987715883 wait complete. 2024-09-19T09:10:11.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.005+0000 7fe5dc50a740 1 -- start start 2024-09-19T09:10:11.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.005+0000 7fe5dc50a740 1 Processor -- start 2024-09-19T09:10:11.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.006+0000 7fe5dc50a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c35245d0 0x5573c3532a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:11.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.006+0000 7fe5dc50a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c3525b40 0x5573c3532f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:11.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.006+0000 7fe5dc50a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573c352f370 0x5573c35a7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:11.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.006+0000 7fe5dc50a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d41aeac0 con 0x5573c35245d0 2024-09-19T09:10:11.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.006+0000 7fe5dc50a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d41ae3f0 con 0x5573c352f370 2024-09-19T09:10:11.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.006+0000 7fe5dc50a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d41ab610 con 0x5573c3525b40 2024-09-19T09:10:11.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.006+0000 7fe5dc50a740 10 client.0 _ll_register_callbacks cb 0x7fe5c4013430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:11.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.006+0000 7fe5d37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c35245d0 0x5573c3532a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:11.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.006+0000 7fe5d37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c35245d0 0x5573c3532a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48566/0 (socket says 172.21.15.120:48566) 2024-09-19T09:10:11.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.006+0000 7fe5d37fe640 1 -- 172.21.15.120:0/563470927 learned_addr learned my addr 172.21.15.120:0/563470927 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:11.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.006+0000 7fe5caffd640 1 --2- 172.21.15.120:0/563470927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c3525b40 0x5573c3532f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:11.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.006+0000 7fe5d37fe640 1 -- 172.21.15.120:0/563470927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c3525b40 msgr2=0x5573c3532f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:11.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.006+0000 7fe5d37fe640 1 --2- 172.21.15.120:0/563470927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c3525b40 0x5573c3532f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.006+0000 7fe5d37fe640 1 -- 172.21.15.120:0/563470927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573c352f370 msgr2=0x5573c35a7df0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:10:11.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.006+0000 7fe5d37fe640 1 --2- 172.21.15.120:0/563470927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573c352f370 0x5573c35a7df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.006+0000 7fe5d37fe640 1 -- 172.21.15.120:0/563470927 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573c35a8360 con 0x5573c35245d0 2024-09-19T09:10:11.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.006+0000 7fe5d37fe640 1 --2- 172.21.15.120:0/563470927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c35245d0 0x5573c3532a00 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4027ac0 tx=0x7fe5c4026f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4086c3cf9d2c66c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:11.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.006+0000 7fe5b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:11.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.006+0000 7fe5b2ffd640 1 -- 172.21.15.120:0/563470927 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4042060 con 0x5573c35245d0 2024-09-19T09:10:11.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.006+0000 7fe5b2ffd640 1 -- 172.21.15.120:0/563470927 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fe5c401d3f0 con 0x5573c35245d0 2024-09-19T09:10:11.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.006+0000 7fe5dc50a740 1 -- 172.21.15.120:0/563470927 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5c4022070 con 0x5573c35245d0 2024-09-19T09:10:11.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.007+0000 7fe5b2ffd640 1 -- 172.21.15.120:0/563470927 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe5c4037070 con 0x5573c35245d0 2024-09-19T09:10:11.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.007+0000 7fe5dc50a740 10 client.17961 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:11.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.007+0000 7fe5dc50a740 1 -- 172.21.15.120:0/563470927 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5573c35a95d0 con 0x5573c35245d0 2024-09-19T09:10:11.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.007+0000 7fe5b2ffd640 1 -- 172.21.15.120:0/563470927 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5c4022070 con 0x5573c35245d0 2024-09-19T09:10:11.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.007+0000 7fe5b2ffd640 1 -- 172.21.15.120:0/563470927 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe5c4047070 con 0x5573c35245d0 2024-09-19T09:10:11.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.007+0000 7fe5dc50a740 10 client.17961 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:11.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.007+0000 7fe5dc50a740 10 client.17961 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:10:11.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.007+0000 7fe5dc50a740 20 client.17961 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:11.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.007+0000 7fe5dc50a740 1 --2- 172.21.15.120:0/563470927 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5573c35a9bd0 0x5573c35abff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:11.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.007+0000 7fe5dc50a740 4 client.17961 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:10:11.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.007+0000 7fe5dc50a740 1 -- 172.21.15.120:0/563470927 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5573c3491d70 con 0x5573c35a9bd0 2024-09-19T09:10:11.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.007+0000 7fe5caffd640 1 --2- 172.21.15.120:0/563470927 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5573c35a9bd0 0x5573c35abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:11.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.008+0000 7fe5b2ffd640 10 client.17961 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:10:11.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.008+0000 7fe5caffd640 1 --2- 172.21.15.120:0/563470927 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5573c35a9bd0 0x5573c35abff0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b7af9236f0953281 server_cookie=eaf38f2c0caea9db in_seq=0 out_seq=0 2024-09-19T09:10:11.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.008+0000 7fe5b2ffd640 1 -- 172.21.15.120:0/563470927 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5573c3491d70 con 0x5573c35a9bd0 2024-09-19T09:10:11.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.008+0000 7fe5b2ffd640 10 client.17961 handle_command_reply: tid=0 2024-09-19T09:10:11.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.008+0000 7fe5dc50a740 1 client.17961 shutdown 2024-09-19T09:10:11.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.008+0000 7fe5dc50a740 1 -- 172.21.15.120:0/563470927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c35245d0 msgr2=0x5573c3532a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:11.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.008+0000 7fe5dc50a740 1 --2- 172.21.15.120:0/563470927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c35245d0 0x5573c3532a00 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4027ac0 tx=0x7fe5c4026f80 comp rx=0 tx=0).stop 2024-09-19T09:10:11.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.008+0000 7fe5dc50a740 1 -- 172.21.15.120:0/563470927 shutdown_connections 2024-09-19T09:10:11.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.008+0000 7fe5dc50a740 1 --2- 172.21.15.120:0/563470927 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5573c35a9bd0 0x5573c35abff0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.008+0000 7fe5dc50a740 1 --2- 172.21.15.120:0/563470927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573c352f370 0x5573c35a7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.008+0000 7fe5b2ffd640 0 client.17961 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:10:11.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.008+0000 7fe5dc50a740 1 --2- 172.21.15.120:0/563470927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573c3525b40 0x5573c3532f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.008+0000 7fe5dc50a740 1 --2- 172.21.15.120:0/563470927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573c35245d0 0x5573c3532a00 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.008+0000 7fe5dc50a740 1 -- 172.21.15.120:0/563470927 >> 172.21.15.120:0/563470927 conn(0x5573c34ed310 msgr2=0x5573c35250f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:11.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.008+0000 7fe5dc50a740 1 -- 172.21.15.120:0/563470927 shutdown_connections 2024-09-19T09:10:11.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.008+0000 7fe5dc50a740 1 -- 172.21.15.120:0/563470927 wait complete. 2024-09-19T09:10:11.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.008+0000 7fe5dc50a740 20 client.17961 trim_cache size 0 max 16384 2024-09-19T09:10:11.020 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:10:11.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.009+0000 7fe5da131640 1 -- 172.21.15.120:0/2372448653 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5c00a3240 msgr2=0x7fe5c00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:11.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.009+0000 7fe5da131640 1 --2- 172.21.15.120:0/2372448653 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5c00a3240 0x7fe5c00a5700 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4009670 tx=0x7fe5c4006bd0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.009+0000 7fe5da131640 1 -- 172.21.15.120:0/2372448653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5d4101420 msgr2=0x7fe5d4198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:11.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.009+0000 7fe5da131640 1 --2- 172.21.15.120:0/2372448653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5d4101420 0x7fe5d4198a90 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc02a960 tx=0x7fe5bc003b90 comp rx=0 tx=0).stop 2024-09-19T09:10:11.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.009+0000 7fe5da131640 1 -- 172.21.15.120:0/2372448653 shutdown_connections 2024-09-19T09:10:11.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.009+0000 7fe5da131640 1 --2- 172.21.15.120:0/2372448653 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5c00a3240 0x7fe5c00a5700 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.009+0000 7fe5da131640 1 --2- 172.21.15.120:0/2372448653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5d4106a80 0x7fe5d419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.009+0000 7fe5da131640 1 --2- 172.21.15.120:0/2372448653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5d4101420 0x7fe5d4198a90 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.009+0000 7fe5da131640 1 --2- 172.21.15.120:0/2372448653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5d4100a80 0x7fe5d4198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:11.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.009+0000 7fe5da131640 1 -- 172.21.15.120:0/2372448653 >> 172.21.15.120:0/2372448653 conn(0x7fe5d40fc7a0 msgr2=0x7fe5d406aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:11.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.009+0000 7fe5da131640 1 -- 172.21.15.120:0/2372448653 shutdown_connections 2024-09-19T09:10:11.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:11.009+0000 7fe5da131640 1 -- 172.21.15.120:0/2372448653 wait complete. 2024-09-19T09:10:11.056 INFO:tasks.workunit.client.0.smithi120.stdout: 1 698442 10.38 MB/sec execute 246 sec latency 2864.857 ms 2024-09-19T09:10:11.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:10 smithi196 ceph-mon[35150]: pgmap v561: 97 pgs: 97 active+clean; 39 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.1 MiB/s wr, 416 op/s 2024-09-19T09:10:11.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:10 smithi153 ceph-mon[35071]: pgmap v561: 97 pgs: 97 active+clean; 39 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.1 MiB/s wr, 416 op/s 2024-09-19T09:10:11.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:10 smithi120 ceph-mon[31150]: pgmap v561: 97 pgs: 97 active+clean; 39 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.1 MiB/s wr, 416 op/s 2024-09-19T09:10:11.943 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:10:12.056 INFO:tasks.workunit.client.0.smithi120.stdout: 1 698442 10.33 MB/sec execute 247 sec latency 3864.925 ms 2024-09-19T09:10:12.187 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:10:12.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.828+0000 7f203d52a640 1 -- 172.21.15.120:0/2529251843 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f203810c170 msgr2=0x7f203810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:12.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.828+0000 7f203d52a640 1 --2- 172.21.15.120:0/2529251843 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f203810c170 0x7f203810e610 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f2030009900 tx=0x7f203002a480 comp rx=0 tx=0).stop 2024-09-19T09:10:12.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.828+0000 7f203d52a640 1 -- 172.21.15.120:0/2529251843 shutdown_connections 2024-09-19T09:10:12.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.828+0000 7f203d52a640 1 --2- 172.21.15.120:0/2529251843 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f203810c170 0x7f203810e610 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.828+0000 7f203d52a640 1 --2- 172.21.15.120:0/2529251843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2038103660 0x7f203810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.828+0000 7f203d52a640 1 --2- 172.21.15.120:0/2529251843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2038102d40 0x7f2038103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.828+0000 7f203d52a640 1 -- 172.21.15.120:0/2529251843 >> 172.21.15.120:0/2529251843 conn(0x7f20380fc7a0 msgr2=0x7f20380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:12.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.828+0000 7f203d52a640 1 -- 172.21.15.120:0/2529251843 shutdown_connections 2024-09-19T09:10:12.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.829+0000 7f203d52a640 1 -- 172.21.15.120:0/2529251843 wait complete. 2024-09-19T09:10:12.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.829+0000 7f203d52a640 1 Processor -- start 2024-09-19T09:10:12.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.829+0000 7f203d52a640 1 -- start start 2024-09-19T09:10:12.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.829+0000 7f203d52a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2038102d40 0x7f2038198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:12.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.829+0000 7f203d52a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2038103660 0x7f2038198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:12.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.829+0000 7f203d52a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f203810c170 0x7f203819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:12.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.829+0000 7f203d52a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2038111f70 con 0x7f2038102d40 2024-09-19T09:10:12.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.829+0000 7f203d52a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2038111df0 con 0x7f2038103660 2024-09-19T09:10:12.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.830+0000 7f203d52a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f20381120f0 con 0x7f203810c170 2024-09-19T09:10:12.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.830+0000 7f2036ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2038102d40 0x7f2038198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:12.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.830+0000 7f2036ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2038102d40 0x7f2038198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55048/0 (socket says 172.21.15.120:55048) 2024-09-19T09:10:12.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.830+0000 7f2036ffd640 1 -- 172.21.15.120:0/264653726 learned_addr learned my addr 172.21.15.120:0/264653726 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:12.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.830+0000 7f20377fe640 1 --2- 172.21.15.120:0/264653726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f203810c170 0x7f203819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:12.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.830+0000 7f20367fc640 1 --2- 172.21.15.120:0/264653726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2038103660 0x7f2038198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:12.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.830+0000 7f2036ffd640 1 -- 172.21.15.120:0/264653726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f203810c170 msgr2=0x7f203819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:12.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.830+0000 7f2036ffd640 1 --2- 172.21.15.120:0/264653726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f203810c170 0x7f203819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.830+0000 7f2036ffd640 1 -- 172.21.15.120:0/264653726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2038103660 msgr2=0x7f2038198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:12.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.830+0000 7f2036ffd640 1 --2- 172.21.15.120:0/264653726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2038103660 0x7f2038198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.830+0000 7f2036ffd640 1 -- 172.21.15.120:0/264653726 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f203819d3c0 con 0x7f2038102d40 2024-09-19T09:10:12.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.830+0000 7f2036ffd640 1 --2- 172.21.15.120:0/264653726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2038102d40 0x7f2038198410 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f202800ea80 tx=0x7f202800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:12.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.830+0000 7f201ffff640 1 -- 172.21.15.120:0/264653726 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202800ce40 con 0x7f2038102d40 2024-09-19T09:10:12.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.830+0000 7f203d52a640 1 -- 172.21.15.120:0/264653726 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f203819d6b0 con 0x7f2038102d40 2024-09-19T09:10:12.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.830+0000 7f201ffff640 1 -- 172.21.15.120:0/264653726 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2028005020 con 0x7f2038102d40 2024-09-19T09:10:12.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.830+0000 7f201ffff640 1 -- 172.21.15.120:0/264653726 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2028011760 con 0x7f2038102d40 2024-09-19T09:10:12.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.830+0000 7f203d52a640 1 -- 172.21.15.120:0/264653726 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20381a4f90 con 0x7f2038102d40 2024-09-19T09:10:12.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.832+0000 7f201ffff640 1 -- 172.21.15.120:0/264653726 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f20280051d0 con 0x7f2038102d40 2024-09-19T09:10:12.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.833+0000 7f203f903740 1 Processor -- start 2024-09-19T09:10:12.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.833+0000 7f203f903740 1 -- start start 2024-09-19T09:10:12.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.833+0000 7f203f903740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56400f89eb40 0x56400f89d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:12.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.833+0000 7f203f903740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56400f89d5d0 0x56400f89da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:12.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.833+0000 7f203f903740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56400f8a8370 0x56400f8aa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:12.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.833+0000 7f203f903740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56400f8032f0 con 0x56400f89d5d0 2024-09-19T09:10:12.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.833+0000 7f203f903740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56400f7aacf0 con 0x56400f89eb40 2024-09-19T09:10:12.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.833+0000 7f203f903740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56400f7ff2b0 con 0x56400f8a8370 2024-09-19T09:10:12.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.833+0000 7f201ffff640 1 --2- 172.21.15.120:0/264653726 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f20200a3240 0x7f20200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:12.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.833+0000 7f2036ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56400f89d5d0 0x56400f89da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:12.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.833+0000 7f2036ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56400f89d5d0 0x56400f89da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55056/0 (socket says 172.21.15.120:55056) 2024-09-19T09:10:12.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.833+0000 7f2036ffd640 1 -- 172.21.15.120:0/48461799 learned_addr learned my addr 172.21.15.120:0/48461799 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:12.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.833+0000 7f20377fe640 1 --2- 172.21.15.120:0/264653726 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f20200a3240 0x7f20200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:12.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.833+0000 7f20367fc640 1 --2- 172.21.15.120:0/48461799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56400f8a8370 0x56400f8aa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:12.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.833+0000 7f20377fe640 1 --2- 172.21.15.120:0/48461799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56400f89eb40 0x56400f89d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:12.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.834+0000 7f201ffff640 1 -- 172.21.15.120:0/264653726 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f20280cffb0 con 0x7f2038102d40 2024-09-19T09:10:12.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.834+0000 7f2036ffd640 1 -- 172.21.15.120:0/48461799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56400f8a8370 msgr2=0x56400f8aa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:12.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.834+0000 7f2036ffd640 1 --2- 172.21.15.120:0/48461799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56400f8a8370 0x56400f8aa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.834+0000 7f2036ffd640 1 -- 172.21.15.120:0/48461799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56400f89eb40 msgr2=0x56400f89d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:12.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.834+0000 7f2036ffd640 1 --2- 172.21.15.120:0/48461799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56400f89eb40 0x56400f89d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.834+0000 7f2036ffd640 1 -- 172.21.15.120:0/48461799 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56400f78f3a0 con 0x56400f89d5d0 2024-09-19T09:10:12.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.834+0000 7f20367fc640 1 --2- 172.21.15.120:0/48461799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56400f8a8370 0x56400f8aa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:10:12.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.834+0000 7f2036ffd640 1 --2- 172.21.15.120:0/48461799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56400f89d5d0 0x56400f89da10 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f20280a4db0 tx=0x7f202809bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eeb684726697f02 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:12.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.834+0000 7f20377fe640 1 --2- 172.21.15.120:0/264653726 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f20200a3240 0x7f20200a5700 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2030031550 tx=0x7f20300058b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:12.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.834+0000 7f201d7fa640 1 -- 172.21.15.120:0/48461799 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202801f070 con 0x56400f89d5d0 2024-09-19T09:10:12.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.834+0000 7f201d7fa640 1 -- 172.21.15.120:0/48461799 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20280a0a20 con 0x56400f89d5d0 2024-09-19T09:10:12.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.834+0000 7f201d7fa640 1 -- 172.21.15.120:0/48461799 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20280d20d0 con 0x56400f89d5d0 2024-09-19T09:10:12.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.835+0000 7f203f903740 1 -- 172.21.15.120:0/48461799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56400f89d5d0 msgr2=0x56400f89da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:12.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.835+0000 7f203f903740 1 --2- 172.21.15.120:0/48461799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56400f89d5d0 0x56400f89da10 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f20280a4db0 tx=0x7f202809bfd0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.835+0000 7f203f903740 1 -- 172.21.15.120:0/48461799 shutdown_connections 2024-09-19T09:10:12.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.835+0000 7f203f903740 1 --2- 172.21.15.120:0/48461799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56400f8a8370 0x56400f8aa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.835+0000 7f203f903740 1 --2- 172.21.15.120:0/48461799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56400f89d5d0 0x56400f89da10 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.835+0000 7f203f903740 1 --2- 172.21.15.120:0/48461799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56400f89eb40 0x56400f89d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.835+0000 7f203f903740 1 -- 172.21.15.120:0/48461799 >> 172.21.15.120:0/48461799 conn(0x56400f866310 msgr2=0x56400f899c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:12.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.835+0000 7f203f903740 1 -- 172.21.15.120:0/48461799 shutdown_connections 2024-09-19T09:10:12.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.835+0000 7f203f903740 1 -- 172.21.15.120:0/48461799 wait complete. 2024-09-19T09:10:12.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.835+0000 7f203f903740 1 -- start start 2024-09-19T09:10:12.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.835+0000 7f203f903740 1 Processor -- start 2024-09-19T09:10:12.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.836+0000 7f203f903740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56400f89d5d0 0x56400f8a7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:12.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.836+0000 7f203f903740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56400f89eb40 0x56400f8a5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:12.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.836+0000 7f203f903740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56400f8a8370 0x56400f8a5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:12.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.836+0000 7f203f903740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56400f8af3c0 con 0x56400f89eb40 2024-09-19T09:10:12.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.836+0000 7f203f903740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56400f7be060 con 0x56400f89d5d0 2024-09-19T09:10:12.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.836+0000 7f203f903740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56400f8af540 con 0x56400f8a8370 2024-09-19T09:10:12.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.836+0000 7f203f903740 10 client.0 _ll_register_callbacks cb 0x56400f5118c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:12.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.836+0000 7f20367fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56400f89eb40 0x56400f8a5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:12.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.836+0000 7f20367fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56400f89eb40 0x56400f8a5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55070/0 (socket says 172.21.15.120:55070) 2024-09-19T09:10:12.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.836+0000 7f20367fc640 1 -- 172.21.15.120:0/3584303601 learned_addr learned my addr 172.21.15.120:0/3584303601 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:12.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.836+0000 7f2036ffd640 1 --2- 172.21.15.120:0/3584303601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56400f89d5d0 0x56400f8a7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:12.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.836+0000 7f20377fe640 1 --2- 172.21.15.120:0/3584303601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56400f8a8370 0x56400f8a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:12.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.836+0000 7f20367fc640 1 -- 172.21.15.120:0/3584303601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56400f8a8370 msgr2=0x56400f8a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:12.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.836+0000 7f20367fc640 1 --2- 172.21.15.120:0/3584303601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56400f8a8370 0x56400f8a5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.836+0000 7f20367fc640 1 -- 172.21.15.120:0/3584303601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56400f89d5d0 msgr2=0x56400f8a7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:12.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.836+0000 7f20367fc640 1 --2- 172.21.15.120:0/3584303601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56400f89d5d0 0x56400f8a7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.836+0000 7f20367fc640 1 -- 172.21.15.120:0/3584303601 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56400f5743c0 con 0x56400f89eb40 2024-09-19T09:10:12.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.836+0000 7f20367fc640 1 --2- 172.21.15.120:0/3584303601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56400f89eb40 0x56400f8a5010 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f202c0106d0 tx=0x7f202c010b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787634dc4f39e10c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:12.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.836+0000 7f201aff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:12.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.836+0000 7f201aff5640 1 -- 172.21.15.120:0/3584303601 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c017070 con 0x56400f89eb40 2024-09-19T09:10:12.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.836+0000 7f201aff5640 1 -- 172.21.15.120:0/3584303601 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f202c00eb30 con 0x56400f89eb40 2024-09-19T09:10:12.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.836+0000 7f203f903740 1 -- 172.21.15.120:0/3584303601 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20280a0a20 con 0x56400f89eb40 2024-09-19T09:10:12.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.836+0000 7f201aff5640 1 -- 172.21.15.120:0/3584303601 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f202c00ed30 con 0x56400f89eb40 2024-09-19T09:10:12.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.837+0000 7f203f903740 10 client.17973 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:12.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.837+0000 7f203f903740 1 -- 172.21.15.120:0/3584303601 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56400f8baec0 con 0x56400f89eb40 2024-09-19T09:10:12.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.837+0000 7f201aff5640 1 -- 172.21.15.120:0/3584303601 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20280a0a20 con 0x56400f89eb40 2024-09-19T09:10:12.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.837+0000 7f201aff5640 1 -- 172.21.15.120:0/3584303601 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f202c011440 con 0x56400f89eb40 2024-09-19T09:10:12.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.837+0000 7f203f903740 10 client.17973 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:12.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.837+0000 7f203f903740 10 client.17973 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:10:12.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.837+0000 7f203f903740 20 client.17973 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:12.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.837+0000 7f203f903740 1 --2- 172.21.15.120:0/3584303601 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56400f8bc370 0x56400f8be790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:12.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.837+0000 7f203f903740 4 client.17973 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:10:12.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.837+0000 7f203f903740 1 -- 172.21.15.120:0/3584303601 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56400f810860 con 0x56400f8bc370 2024-09-19T09:10:12.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.837+0000 7f2036ffd640 1 --2- 172.21.15.120:0/3584303601 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56400f8bc370 0x56400f8be790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:12.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.838+0000 7f2036ffd640 1 --2- 172.21.15.120:0/3584303601 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56400f8bc370 0x56400f8be790 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4c311305a2eca4d7 server_cookie=ca628d1feeb78b56 in_seq=0 out_seq=0 2024-09-19T09:10:12.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.838+0000 7f201aff5640 10 client.17973 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:10:12.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.838+0000 7f201aff5640 1 -- 172.21.15.120:0/3584303601 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56400f810860 con 0x56400f8bc370 2024-09-19T09:10:12.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.838+0000 7f201aff5640 10 client.17973 handle_command_reply: tid=0 2024-09-19T09:10:12.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.838+0000 7f203f903740 1 client.17973 shutdown 2024-09-19T09:10:12.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.838+0000 7f203f903740 1 -- 172.21.15.120:0/3584303601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56400f89eb40 msgr2=0x56400f8a5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:12.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.839+0000 7f203f903740 1 --2- 172.21.15.120:0/3584303601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56400f89eb40 0x56400f8a5010 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f202c0106d0 tx=0x7f202c010b90 comp rx=0 tx=0).stop 2024-09-19T09:10:12.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.839+0000 7f203f903740 1 -- 172.21.15.120:0/3584303601 shutdown_connections 2024-09-19T09:10:12.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.839+0000 7f203f903740 1 --2- 172.21.15.120:0/3584303601 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56400f8bc370 0x56400f8be790 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.839+0000 7f203f903740 1 --2- 172.21.15.120:0/3584303601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56400f8a8370 0x56400f8a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.839+0000 7f203f903740 1 --2- 172.21.15.120:0/3584303601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56400f89eb40 0x56400f8a5010 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.839+0000 7f201aff5640 0 client.17973 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:10:12.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.839+0000 7f203f903740 1 --2- 172.21.15.120:0/3584303601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56400f89d5d0 0x56400f8a7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.839+0000 7f203f903740 1 -- 172.21.15.120:0/3584303601 >> 172.21.15.120:0/3584303601 conn(0x56400f866310 msgr2=0x56400f89f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:12.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.839+0000 7f203f903740 1 -- 172.21.15.120:0/3584303601 shutdown_connections 2024-09-19T09:10:12.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.839+0000 7f203f903740 1 -- 172.21.15.120:0/3584303601 wait complete. 2024-09-19T09:10:12.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.839+0000 7f203f903740 20 client.17973 trim_cache size 0 max 16384 2024-09-19T09:10:12.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.852+0000 7f203f903740 1 Processor -- start 2024-09-19T09:10:12.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.852+0000 7f203f903740 1 -- start start 2024-09-19T09:10:12.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.852+0000 7f203f903740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56400f89d5d0 0x56400f8b8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:12.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.852+0000 7f203f903740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56400f89eb40 0x56400f8a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:12.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.852+0000 7f203f903740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56400f8a8370 0x56400f8c4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:12.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.852+0000 7f203f903740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f20381ab460 con 0x56400f89eb40 2024-09-19T09:10:12.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.852+0000 7f203f903740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f20381acfa0 con 0x56400f89d5d0 2024-09-19T09:10:12.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.852+0000 7f203f903740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f20381abb30 con 0x56400f8a8370 2024-09-19T09:10:12.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.852+0000 7f20367fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56400f89eb40 0x56400f8a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:12.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.852+0000 7f20367fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56400f89eb40 0x56400f8a3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55080/0 (socket says 172.21.15.120:55080) 2024-09-19T09:10:12.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.852+0000 7f20367fc640 1 -- 172.21.15.120:0/2172409186 learned_addr learned my addr 172.21.15.120:0/2172409186 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:12.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.852+0000 7f2036ffd640 1 --2- 172.21.15.120:0/2172409186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56400f89d5d0 0x56400f8b8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:12.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.852+0000 7f20367fc640 1 -- 172.21.15.120:0/2172409186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56400f8a8370 msgr2=0x56400f8c4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:12.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.852+0000 7f20367fc640 1 --2- 172.21.15.120:0/2172409186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56400f8a8370 0x56400f8c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.852+0000 7f20367fc640 1 -- 172.21.15.120:0/2172409186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56400f89d5d0 msgr2=0x56400f8b8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:12.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.852+0000 7f20367fc640 1 --2- 172.21.15.120:0/2172409186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56400f89d5d0 0x56400f8b8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.852+0000 7f20367fc640 1 -- 172.21.15.120:0/2172409186 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56400f8a3560 con 0x56400f89eb40 2024-09-19T09:10:12.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.852+0000 7f20367fc640 1 --2- 172.21.15.120:0/2172409186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56400f89eb40 0x56400f8a3ff0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f202c005d90 tx=0x7f202c00c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98338f6461a7b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:12.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.852+0000 7f201cff9640 1 -- 172.21.15.120:0/2172409186 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c017040 con 0x56400f89eb40 2024-09-19T09:10:12.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.852+0000 7f201cff9640 1 -- 172.21.15.120:0/2172409186 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f202c00e720 con 0x56400f89eb40 2024-09-19T09:10:12.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.853+0000 7f201cff9640 1 -- 172.21.15.120:0/2172409186 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c01ba20 con 0x56400f89eb40 2024-09-19T09:10:12.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.853+0000 7f203f903740 1 -- 172.21.15.120:0/2172409186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56400f89eb40 msgr2=0x56400f8a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:12.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.853+0000 7f203f903740 1 --2- 172.21.15.120:0/2172409186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56400f89eb40 0x56400f8a3ff0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f202c005d90 tx=0x7f202c00c660 comp rx=0 tx=0).stop 2024-09-19T09:10:12.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.853+0000 7f203f903740 1 -- 172.21.15.120:0/2172409186 shutdown_connections 2024-09-19T09:10:12.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.853+0000 7f203f903740 1 --2- 172.21.15.120:0/2172409186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56400f8a8370 0x56400f8c4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.853+0000 7f203f903740 1 --2- 172.21.15.120:0/2172409186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56400f89eb40 0x56400f8a3ff0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.853+0000 7f203f903740 1 --2- 172.21.15.120:0/2172409186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56400f89d5d0 0x56400f8b8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.853+0000 7f203f903740 1 -- 172.21.15.120:0/2172409186 >> 172.21.15.120:0/2172409186 conn(0x56400f866310 msgr2=0x56400f8a38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:12.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.853+0000 7f203f903740 1 -- 172.21.15.120:0/2172409186 shutdown_connections 2024-09-19T09:10:12.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.853+0000 7f203f903740 1 -- 172.21.15.120:0/2172409186 wait complete. 2024-09-19T09:10:12.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.853+0000 7f203f903740 1 -- start start 2024-09-19T09:10:12.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.853+0000 7f203f903740 1 Processor -- start 2024-09-19T09:10:12.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.854+0000 7f203f903740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56400f89d5d0 0x56400f8b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:12.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.854+0000 7f203f903740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56400f89eb40 0x56400f8b0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:12.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.854+0000 7f203f903740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56400f8a8370 0x56400f8b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:12.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.854+0000 7f203f903740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f20381ac8d0 con 0x56400f8a8370 2024-09-19T09:10:12.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.854+0000 7f203f903740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f20381ac200 con 0x56400f89eb40 2024-09-19T09:10:12.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.854+0000 7f203f903740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f20381a9440 con 0x56400f89d5d0 2024-09-19T09:10:12.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.854+0000 7f203f903740 10 client.0 _ll_register_callbacks cb 0x7f203806b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:12.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.854+0000 7f20377fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56400f8a8370 0x56400f8b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:12.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.854+0000 7f20377fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56400f8a8370 0x56400f8b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55086/0 (socket says 172.21.15.120:55086) 2024-09-19T09:10:12.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.854+0000 7f20377fe640 1 -- 172.21.15.120:0/2944042565 learned_addr learned my addr 172.21.15.120:0/2944042565 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:12.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.854+0000 7f2036ffd640 1 --2- 172.21.15.120:0/2944042565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56400f89d5d0 0x56400f8b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:12.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.854+0000 7f20367fc640 1 --2- 172.21.15.120:0/2944042565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56400f89eb40 0x56400f8b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:12.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.854+0000 7f20377fe640 1 -- 172.21.15.120:0/2944042565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56400f89d5d0 msgr2=0x56400f8b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:12.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.854+0000 7f20377fe640 1 --2- 172.21.15.120:0/2944042565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56400f89d5d0 0x56400f8b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.854+0000 7f20377fe640 1 -- 172.21.15.120:0/2944042565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56400f89eb40 msgr2=0x56400f8b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:12.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.854+0000 7f20377fe640 1 --2- 172.21.15.120:0/2944042565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56400f89eb40 0x56400f8b0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.854+0000 7f20377fe640 1 -- 172.21.15.120:0/2944042565 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56400f8b1180 con 0x56400f8a8370 2024-09-19T09:10:12.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.854+0000 7f2036ffd640 1 --2- 172.21.15.120:0/2944042565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56400f89d5d0 0x56400f8b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:10:12.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.854+0000 7f20367fc640 1 --2- 172.21.15.120:0/2944042565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56400f89eb40 0x56400f8b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:10:12.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.854+0000 7f20377fe640 1 --2- 172.21.15.120:0/2944042565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56400f8a8370 0x56400f8b0950 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f2030009900 tx=0x7f2030009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95be6c513a7b6934 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:12.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.854+0000 7f201aff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:12.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.854+0000 7f201aff5640 1 -- 172.21.15.120:0/2944042565 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203004e030 con 0x56400f8a8370 2024-09-19T09:10:12.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.855+0000 7f201aff5640 1 -- 172.21.15.120:0/2944042565 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f2030005250 con 0x56400f8a8370 2024-09-19T09:10:12.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.855+0000 7f203f903740 1 -- 172.21.15.120:0/2944042565 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f202c00e720 con 0x56400f8a8370 2024-09-19T09:10:12.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.855+0000 7f201aff5640 1 -- 172.21.15.120:0/2944042565 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2030035040 con 0x56400f8a8370 2024-09-19T09:10:12.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.855+0000 7f203f903740 10 client.17979 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:12.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.855+0000 7f203f903740 1 -- 172.21.15.120:0/2944042565 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56400f927140 con 0x56400f8a8370 2024-09-19T09:10:12.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.855+0000 7f201aff5640 1 -- 172.21.15.120:0/2944042565 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f202c00e720 con 0x56400f8a8370 2024-09-19T09:10:12.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.855+0000 7f201aff5640 1 -- 172.21.15.120:0/2944042565 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2030048040 con 0x56400f8a8370 2024-09-19T09:10:12.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.855+0000 7f203f903740 10 client.17979 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:12.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.855+0000 7f203f903740 10 client.17979 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:10:12.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.855+0000 7f203f903740 20 client.17979 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:12.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.855+0000 7f203f903740 1 --2- 172.21.15.120:0/2944042565 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56400f927890 0x56400f929cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:12.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.855+0000 7f203f903740 4 client.17979 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:10:12.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.855+0000 7f203f903740 1 -- 172.21.15.120:0/2944042565 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56400f80ad70 con 0x56400f927890 2024-09-19T09:10:12.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.855+0000 7f2036ffd640 1 --2- 172.21.15.120:0/2944042565 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56400f927890 0x56400f929cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:12.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.856+0000 7f2036ffd640 1 --2- 172.21.15.120:0/2944042565 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56400f927890 0x56400f929cb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3250ceac651eba46 server_cookie=22bdd46a2e924b93 in_seq=0 out_seq=0 2024-09-19T09:10:12.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.856+0000 7f201aff5640 10 client.17979 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:10:12.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.856+0000 7f201aff5640 1 -- 172.21.15.120:0/2944042565 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56400f80ad70 con 0x56400f927890 2024-09-19T09:10:12.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.856+0000 7f201aff5640 10 client.17979 handle_command_reply: tid=0 2024-09-19T09:10:12.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.856+0000 7f203f903740 1 client.17979 shutdown 2024-09-19T09:10:12.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.856+0000 7f203f903740 1 -- 172.21.15.120:0/2944042565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56400f8a8370 msgr2=0x56400f8b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:12.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.856+0000 7f203f903740 1 --2- 172.21.15.120:0/2944042565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56400f8a8370 0x56400f8b0950 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f2030009900 tx=0x7f2030009680 comp rx=0 tx=0).stop 2024-09-19T09:10:12.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.856+0000 7f203f903740 1 -- 172.21.15.120:0/2944042565 shutdown_connections 2024-09-19T09:10:12.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.856+0000 7f203f903740 1 --2- 172.21.15.120:0/2944042565 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56400f927890 0x56400f929cb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.856+0000 7f203f903740 1 --2- 172.21.15.120:0/2944042565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56400f8a8370 0x56400f8b0950 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.856+0000 7f203f903740 1 --2- 172.21.15.120:0/2944042565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56400f89eb40 0x56400f8b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.856+0000 7f201aff5640 0 client.17979 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:10:12.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.856+0000 7f203f903740 1 --2- 172.21.15.120:0/2944042565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56400f89d5d0 0x56400f8b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.856+0000 7f203f903740 1 -- 172.21.15.120:0/2944042565 >> 172.21.15.120:0/2944042565 conn(0x56400f866310 msgr2=0x56400f89f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:12.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.856+0000 7f203f903740 1 -- 172.21.15.120:0/2944042565 shutdown_connections 2024-09-19T09:10:12.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.856+0000 7f203f903740 1 -- 172.21.15.120:0/2944042565 wait complete. 2024-09-19T09:10:12.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.856+0000 7f203f903740 20 client.17979 trim_cache size 0 max 16384 2024-09-19T09:10:12.966 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:12 smithi120 ceph-mon[31150]: pgmap v562: 97 pgs: 97 active+clean; 44 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 413 op/s 2024-09-19T09:10:12.966 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:10:12.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.857+0000 7f203d52a640 1 -- 172.21.15.120:0/264653726 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f20200a3240 msgr2=0x7f20200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:12.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.857+0000 7f203d52a640 1 --2- 172.21.15.120:0/264653726 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f20200a3240 0x7f20200a5700 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2030031550 tx=0x7f20300058b0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.857+0000 7f203d52a640 1 -- 172.21.15.120:0/264653726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2038102d40 msgr2=0x7f2038198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:12.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.857+0000 7f203d52a640 1 --2- 172.21.15.120:0/264653726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2038102d40 0x7f2038198410 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f202800ea80 tx=0x7f202800ef40 comp rx=0 tx=0).stop 2024-09-19T09:10:12.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.857+0000 7f203d52a640 1 -- 172.21.15.120:0/264653726 shutdown_connections 2024-09-19T09:10:12.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.857+0000 7f203d52a640 1 --2- 172.21.15.120:0/264653726 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f20200a3240 0x7f20200a5700 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.857+0000 7f203d52a640 1 --2- 172.21.15.120:0/264653726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f203810c170 0x7f203819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.857+0000 7f203d52a640 1 --2- 172.21.15.120:0/264653726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2038103660 0x7f2038198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.857+0000 7f203d52a640 1 --2- 172.21.15.120:0/264653726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2038102d40 0x7f2038198410 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:12.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.857+0000 7f203d52a640 1 -- 172.21.15.120:0/264653726 >> 172.21.15.120:0/264653726 conn(0x7f20380fc7a0 msgr2=0x7f20380fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:12.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.857+0000 7f203d52a640 1 -- 172.21.15.120:0/264653726 shutdown_connections 2024-09-19T09:10:12.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:12.857+0000 7f203d52a640 1 -- 172.21.15.120:0/264653726 wait complete. 2024-09-19T09:10:13.056 INFO:tasks.workunit.client.0.smithi120.stdout: 1 698442 10.29 MB/sec execute 248 sec latency 4864.951 ms 2024-09-19T09:10:13.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:12 smithi196 ceph-mon[35150]: pgmap v562: 97 pgs: 97 active+clean; 44 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 413 op/s 2024-09-19T09:10:13.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:12 smithi153 ceph-mon[35071]: pgmap v562: 97 pgs: 97 active+clean; 44 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 413 op/s 2024-09-19T09:10:14.057 INFO:tasks.workunit.client.0.smithi120.stdout: 1 698442 10.25 MB/sec execute 249 sec latency 5865.008 ms 2024-09-19T09:10:14.065 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:10:14.315 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:10:14.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.805+0000 7f598d99f640 1 -- 172.21.15.120:0/173837848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5988102570 msgr2=0x7f59881029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:14.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.805+0000 7f598d99f640 1 --2- 172.21.15.120:0/173837848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5988102570 0x7f59881029b0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f5970009900 tx=0x7f597002a480 comp rx=0 tx=0).stop 2024-09-19T09:10:14.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.806+0000 7f598d99f640 1 -- 172.21.15.120:0/173837848 shutdown_connections 2024-09-19T09:10:14.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.806+0000 7f598d99f640 1 --2- 172.21.15.120:0/173837848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5988102ef0 0x7f598810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.806+0000 7f598d99f640 1 --2- 172.21.15.120:0/173837848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5988102570 0x7f59881029b0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.806+0000 7f598d99f640 1 --2- 172.21.15.120:0/173837848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5988100e20 0x7f5988101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.806+0000 7f598d99f640 1 -- 172.21.15.120:0/173837848 >> 172.21.15.120:0/173837848 conn(0x7f59880fc7d0 msgr2=0x7f59880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:14.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.806+0000 7f598d99f640 1 -- 172.21.15.120:0/173837848 shutdown_connections 2024-09-19T09:10:14.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.806+0000 7f598d99f640 1 -- 172.21.15.120:0/173837848 wait complete. 2024-09-19T09:10:14.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.806+0000 7f598d99f640 1 Processor -- start 2024-09-19T09:10:14.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.806+0000 7f598d99f640 1 -- start start 2024-09-19T09:10:14.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.807+0000 7f598d99f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5988100e20 0x7f5988198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:14.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.807+0000 7f598d99f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5988102570 0x7f5988198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:14.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.807+0000 7f598d99f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5988102ef0 0x7f598819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:14.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.807+0000 7f598d99f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5988110f50 con 0x7f5988102ef0 2024-09-19T09:10:14.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.807+0000 7f598d99f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5988110dd0 con 0x7f5988102570 2024-09-19T09:10:14.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.807+0000 7f598d99f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f59881110d0 con 0x7f5988100e20 2024-09-19T09:10:14.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.807+0000 7f5986ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5988100e20 0x7f5988198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:14.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.807+0000 7f59877fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5988102ef0 0x7f598819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:14.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.807+0000 7f5986ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5988100e20 0x7f5988198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:58160/0 (socket says 172.21.15.120:58160) 2024-09-19T09:10:14.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.807+0000 7f5986ffd640 1 -- 172.21.15.120:0/3418102470 learned_addr learned my addr 172.21.15.120:0/3418102470 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:14.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.807+0000 7f59877fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5988102ef0 0x7f598819cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55112/0 (socket says 172.21.15.120:55112) 2024-09-19T09:10:14.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.807+0000 7f59867fc640 1 --2- 172.21.15.120:0/3418102470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5988102570 0x7f5988198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:14.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.807+0000 7f59877fe640 1 -- 172.21.15.120:0/3418102470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5988100e20 msgr2=0x7f5988198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:14.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.807+0000 7f59877fe640 1 --2- 172.21.15.120:0/3418102470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5988100e20 0x7f5988198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.807+0000 7f59877fe640 1 -- 172.21.15.120:0/3418102470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5988102570 msgr2=0x7f5988198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:14.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.807+0000 7f59877fe640 1 --2- 172.21.15.120:0/3418102470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5988102570 0x7f5988198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.807+0000 7f59877fe640 1 -- 172.21.15.120:0/3418102470 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f598819d4b0 con 0x7f5988102ef0 2024-09-19T09:10:14.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.807+0000 7f59867fc640 1 --2- 172.21.15.120:0/3418102470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5988102570 0x7f5988198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:10:14.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.807+0000 7f5986ffd640 1 --2- 172.21.15.120:0/3418102470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5988100e20 0x7f5988198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:10:14.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.807+0000 7f59877fe640 1 --2- 172.21.15.120:0/3418102470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5988102ef0 0x7f598819cdd0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f598000d6b0 tx=0x7f598000db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:14.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.808+0000 7f598c99d640 1 -- 172.21.15.120:0/3418102470 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5980017070 con 0x7f5988102ef0 2024-09-19T09:10:14.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.808+0000 7f598d99f640 1 -- 172.21.15.120:0/3418102470 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f598819d7a0 con 0x7f5988102ef0 2024-09-19T09:10:14.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.808+0000 7f598c99d640 1 -- 172.21.15.120:0/3418102470 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5980009d00 con 0x7f5988102ef0 2024-09-19T09:10:14.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.808+0000 7f598c99d640 1 -- 172.21.15.120:0/3418102470 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5980003d60 con 0x7f5988102ef0 2024-09-19T09:10:14.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.808+0000 7f598d99f640 1 -- 172.21.15.120:0/3418102470 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59881a5080 con 0x7f5988102ef0 2024-09-19T09:10:14.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.810+0000 7f598c99d640 1 -- 172.21.15.120:0/3418102470 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5980009820 con 0x7f5988102ef0 2024-09-19T09:10:14.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.810+0000 7f598fd78740 1 Processor -- start 2024-09-19T09:10:14.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.810+0000 7f598fd78740 1 -- start start 2024-09-19T09:10:14.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.811+0000 7f598fd78740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f0a211b40 0x562f0a210090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:14.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.811+0000 7f598fd78740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f0a2105d0 0x562f0a210a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:14.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.811+0000 7f598fd78740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f0a21b370 0x562f0a21d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:14.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.811+0000 7f598fd78740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562f0a1762f0 con 0x562f0a2105d0 2024-09-19T09:10:14.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.811+0000 7f598fd78740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562f0a11dcf0 con 0x562f0a21b370 2024-09-19T09:10:14.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.811+0000 7f598fd78740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562f0a1722b0 con 0x562f0a211b40 2024-09-19T09:10:14.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.811+0000 7f5986ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f0a2105d0 0x562f0a210a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:14.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.811+0000 7f5986ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f0a2105d0 0x562f0a210a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55128/0 (socket says 172.21.15.120:55128) 2024-09-19T09:10:14.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.811+0000 7f5986ffd640 1 -- 172.21.15.120:0/1111876679 learned_addr learned my addr 172.21.15.120:0/1111876679 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:14.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.811+0000 7f59867fc640 1 --2- 172.21.15.120:0/1111876679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f0a21b370 0x562f0a21d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:14.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.811+0000 7f59867fc640 1 --2- 172.21.15.120:0/1111876679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f0a211b40 0x562f0a210090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:14.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.811+0000 7f5986ffd640 1 -- 172.21.15.120:0/1111876679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f0a211b40 msgr2=0x562f0a210090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:14.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.811+0000 7f598c99d640 1 --2- 172.21.15.120:0/3418102470 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f59740a3240 0x7f59740a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:14.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.811+0000 7f5986ffd640 1 --2- 172.21.15.120:0/1111876679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f0a211b40 0x562f0a210090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.811+0000 7f5986ffd640 1 -- 172.21.15.120:0/1111876679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f0a21b370 msgr2=0x562f0a21d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:14.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.811+0000 7f5986ffd640 1 --2- 172.21.15.120:0/1111876679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f0a21b370 0x562f0a21d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.811+0000 7f5986ffd640 1 -- 172.21.15.120:0/1111876679 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f0a1023a0 con 0x562f0a2105d0 2024-09-19T09:10:14.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.811+0000 7f59867fc640 1 --2- 172.21.15.120:0/1111876679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f0a211b40 0x562f0a210090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:10:14.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.811+0000 7f59877fe640 1 --2- 172.21.15.120:0/3418102470 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f59740a3240 0x7f59740a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:14.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.811+0000 7f5986ffd640 1 --2- 172.21.15.120:0/1111876679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f0a2105d0 0x562f0a210a10 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f597800f790 tx=0x7f597800fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b938052d34febb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:14.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.811+0000 7f598c99d640 1 -- 172.21.15.120:0/3418102470 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f59800cf260 con 0x7f5988102ef0 2024-09-19T09:10:14.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.811+0000 7f597dffb640 1 -- 172.21.15.120:0/1111876679 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597800fea0 con 0x562f0a2105d0 2024-09-19T09:10:14.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.812+0000 7f597dffb640 1 -- 172.21.15.120:0/1111876679 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f597800dd30 con 0x562f0a2105d0 2024-09-19T09:10:14.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.812+0000 7f597dffb640 1 -- 172.21.15.120:0/1111876679 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59780186b0 con 0x562f0a2105d0 2024-09-19T09:10:14.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.812+0000 7f598fd78740 1 -- 172.21.15.120:0/1111876679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f0a2105d0 msgr2=0x562f0a210a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:14.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.812+0000 7f59877fe640 1 --2- 172.21.15.120:0/3418102470 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f59740a3240 0x7f59740a5700 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5980096320 tx=0x7f598006e700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:14.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.812+0000 7f598fd78740 1 --2- 172.21.15.120:0/1111876679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f0a2105d0 0x562f0a210a10 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f597800f790 tx=0x7f597800fc50 comp rx=0 tx=0).stop 2024-09-19T09:10:14.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.812+0000 7f598fd78740 1 -- 172.21.15.120:0/1111876679 shutdown_connections 2024-09-19T09:10:14.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.812+0000 7f598fd78740 1 --2- 172.21.15.120:0/1111876679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f0a21b370 0x562f0a21d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.812+0000 7f598fd78740 1 --2- 172.21.15.120:0/1111876679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f0a2105d0 0x562f0a210a10 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.812+0000 7f598fd78740 1 --2- 172.21.15.120:0/1111876679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f0a211b40 0x562f0a210090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.812+0000 7f598fd78740 1 -- 172.21.15.120:0/1111876679 >> 172.21.15.120:0/1111876679 conn(0x562f0a1d9310 msgr2=0x562f0a20cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:14.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.812+0000 7f598fd78740 1 -- 172.21.15.120:0/1111876679 shutdown_connections 2024-09-19T09:10:14.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.812+0000 7f598fd78740 1 -- 172.21.15.120:0/1111876679 wait complete. 2024-09-19T09:10:14.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.813+0000 7f598fd78740 1 -- start start 2024-09-19T09:10:14.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.813+0000 7f598fd78740 1 Processor -- start 2024-09-19T09:10:14.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.813+0000 7f598fd78740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f0a2105d0 0x562f0a21ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:14.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.813+0000 7f598fd78740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f0a211b40 0x562f0a217fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:14.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.813+0000 7f598fd78740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f0a21b370 0x562f0a218570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:14.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.813+0000 7f598fd78740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562f0a21e070 con 0x562f0a2105d0 2024-09-19T09:10:14.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.813+0000 7f598fd78740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562f0a131060 con 0x562f0a21b370 2024-09-19T09:10:14.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.813+0000 7f598fd78740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562f0a21e1f0 con 0x562f0a211b40 2024-09-19T09:10:14.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.813+0000 7f598fd78740 10 client.0 _ll_register_callbacks cb 0x562f09e848c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:14.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.813+0000 7f59867fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f0a2105d0 0x562f0a21ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:14.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.813+0000 7f59867fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f0a2105d0 0x562f0a21ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55130/0 (socket says 172.21.15.120:55130) 2024-09-19T09:10:14.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.813+0000 7f59867fc640 1 -- 172.21.15.120:0/3695573082 learned_addr learned my addr 172.21.15.120:0/3695573082 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:14.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.813+0000 7f5986ffd640 1 --2- 172.21.15.120:0/3695573082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f0a211b40 0x562f0a217fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:14.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.813+0000 7f59867fc640 1 -- 172.21.15.120:0/3695573082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f0a211b40 msgr2=0x562f0a217fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:14.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.814+0000 7f59867fc640 1 --2- 172.21.15.120:0/3695573082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f0a211b40 0x562f0a217fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.814+0000 7f59867fc640 1 -- 172.21.15.120:0/3695573082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f0a21b370 msgr2=0x562f0a218570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:14.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.814+0000 7f59867fc640 1 --2- 172.21.15.120:0/3695573082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f0a21b370 0x562f0a218570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.814+0000 7f59867fc640 1 -- 172.21.15.120:0/3695573082 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f09ee73c0 con 0x562f0a2105d0 2024-09-19T09:10:14.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.814+0000 7f5986ffd640 1 --2- 172.21.15.120:0/3695573082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f0a211b40 0x562f0a217fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:10:14.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.814+0000 7f59867fc640 1 --2- 172.21.15.120:0/3695573082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f0a2105d0 0x562f0a21ad20 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f597002a9a0 tx=0x7f5970003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82af0021d3a97fc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:14.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.814+0000 7f5966ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:14.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.814+0000 7f5966ffd640 1 -- 172.21.15.120:0/3695573082 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5970035040 con 0x562f0a2105d0 2024-09-19T09:10:14.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.814+0000 7f5966ffd640 1 -- 172.21.15.120:0/3695573082 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f5970031540 con 0x562f0a2105d0 2024-09-19T09:10:14.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.814+0000 7f598fd78740 1 -- 172.21.15.120:0/3695573082 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f597800dd30 con 0x562f0a2105d0 2024-09-19T09:10:14.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.814+0000 7f5966ffd640 1 -- 172.21.15.120:0/3695573082 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5970003e60 con 0x562f0a2105d0 2024-09-19T09:10:14.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.814+0000 7f598fd78740 10 client.17991 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:14.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.814+0000 7f598fd78740 1 -- 172.21.15.120:0/3695573082 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562f0a229a80 con 0x562f0a2105d0 2024-09-19T09:10:14.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.814+0000 7f5966ffd640 1 -- 172.21.15.120:0/3695573082 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f597800dd30 con 0x562f0a2105d0 2024-09-19T09:10:14.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.814+0000 7f5966ffd640 1 -- 172.21.15.120:0/3695573082 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f597003a040 con 0x562f0a2105d0 2024-09-19T09:10:14.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.814+0000 7f598fd78740 10 client.17991 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:14.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.814+0000 7f598fd78740 10 client.17991 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:10:14.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.814+0000 7f598fd78740 20 client.17991 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:14.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.814+0000 7f598fd78740 1 --2- 172.21.15.120:0/3695573082 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x562f0a22ae90 0x562f0a22d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:14.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.814+0000 7f598fd78740 4 client.17991 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:10:14.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.815+0000 7f598fd78740 1 -- 172.21.15.120:0/3695573082 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f0a183860 con 0x562f0a22ae90 2024-09-19T09:10:14.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.815+0000 7f5986ffd640 1 --2- 172.21.15.120:0/3695573082 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x562f0a22ae90 0x562f0a22d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:14.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.815+0000 7f5966ffd640 10 client.17991 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:10:14.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.815+0000 7f5986ffd640 1 --2- 172.21.15.120:0/3695573082 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x562f0a22ae90 0x562f0a22d280 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=418e50ee468e78a3 server_cookie=8c2459aa1c66446f in_seq=0 out_seq=0 2024-09-19T09:10:14.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.816+0000 7f5966ffd640 1 -- 172.21.15.120:0/3695573082 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562f0a183860 con 0x562f0a22ae90 2024-09-19T09:10:14.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.816+0000 7f5966ffd640 10 client.17991 handle_command_reply: tid=0 2024-09-19T09:10:14.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.816+0000 7f598fd78740 1 client.17991 shutdown 2024-09-19T09:10:14.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.816+0000 7f598fd78740 1 -- 172.21.15.120:0/3695573082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f0a2105d0 msgr2=0x562f0a21ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:14.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.816+0000 7f598fd78740 1 --2- 172.21.15.120:0/3695573082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f0a2105d0 0x562f0a21ad20 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f597002a9a0 tx=0x7f5970003820 comp rx=0 tx=0).stop 2024-09-19T09:10:14.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.816+0000 7f598fd78740 1 -- 172.21.15.120:0/3695573082 shutdown_connections 2024-09-19T09:10:14.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.816+0000 7f598fd78740 1 --2- 172.21.15.120:0/3695573082 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x562f0a22ae90 0x562f0a22d280 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.816+0000 7f598fd78740 1 --2- 172.21.15.120:0/3695573082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f0a21b370 0x562f0a218570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.816+0000 7f598fd78740 1 --2- 172.21.15.120:0/3695573082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f0a211b40 0x562f0a217fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.816+0000 7f598fd78740 1 --2- 172.21.15.120:0/3695573082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f0a2105d0 0x562f0a21ad20 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.816+0000 7f598fd78740 1 -- 172.21.15.120:0/3695573082 >> 172.21.15.120:0/3695573082 conn(0x562f0a1d9310 msgr2=0x562f0a212960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:14.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.816+0000 7f5966ffd640 0 client.17991 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:10:14.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.816+0000 7f598fd78740 1 -- 172.21.15.120:0/3695573082 shutdown_connections 2024-09-19T09:10:14.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.816+0000 7f598fd78740 1 -- 172.21.15.120:0/3695573082 wait complete. 2024-09-19T09:10:14.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.816+0000 7f598fd78740 20 client.17991 trim_cache size 0 max 16384 2024-09-19T09:10:14.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.829+0000 7f598fd78740 1 Processor -- start 2024-09-19T09:10:14.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.829+0000 7f598fd78740 1 -- start start 2024-09-19T09:10:14.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.829+0000 7f598fd78740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f0a2105d0 0x562f0a227310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:14.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.829+0000 7f598fd78740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f0a211b40 0x562f0a21c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:14.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.829+0000 7f598fd78740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f0a21b370 0x562f0a21cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:14.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.829+0000 7f598fd78740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f59881ad4c0 con 0x562f0a211b40 2024-09-19T09:10:14.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.829+0000 7f598fd78740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f59881af000 con 0x562f0a21b370 2024-09-19T09:10:14.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.829+0000 7f598fd78740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f59881adb90 con 0x562f0a2105d0 2024-09-19T09:10:14.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.830+0000 7f5986ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f0a211b40 0x562f0a21c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:14.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.830+0000 7f5986ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f0a211b40 0x562f0a21c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55144/0 (socket says 172.21.15.120:55144) 2024-09-19T09:10:14.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.830+0000 7f5986ffd640 1 -- 172.21.15.120:0/200090916 learned_addr learned my addr 172.21.15.120:0/200090916 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:14.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.830+0000 7f59867fc640 1 --2- 172.21.15.120:0/200090916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f0a2105d0 0x562f0a227310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:14.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.830+0000 7f5986ffd640 1 -- 172.21.15.120:0/200090916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f0a2105d0 msgr2=0x562f0a227310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:14.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.830+0000 7f5986ffd640 1 --2- 172.21.15.120:0/200090916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f0a2105d0 0x562f0a227310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.830+0000 7f5986ffd640 1 -- 172.21.15.120:0/200090916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f0a21b370 msgr2=0x562f0a21cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:14.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.830+0000 7f5986ffd640 1 --2- 172.21.15.120:0/200090916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f0a21b370 0x562f0a21cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.830+0000 7f5986ffd640 1 -- 172.21.15.120:0/200090916 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f0a216520 con 0x562f0a211b40 2024-09-19T09:10:14.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.830+0000 7f59867fc640 1 --2- 172.21.15.120:0/200090916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f0a2105d0 0x562f0a227310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:10:14.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.830+0000 7f5986ffd640 1 --2- 172.21.15.120:0/200090916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f0a211b40 0x562f0a21c760 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f5978007f20 tx=0x7f597800f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc163fe68124de5f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:14.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.830+0000 7f597d7fa640 1 -- 172.21.15.120:0/200090916 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597800fea0 con 0x562f0a211b40 2024-09-19T09:10:14.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.830+0000 7f597d7fa640 1 -- 172.21.15.120:0/200090916 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f597800dbb0 con 0x562f0a211b40 2024-09-19T09:10:14.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.830+0000 7f597d7fa640 1 -- 172.21.15.120:0/200090916 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5978003bc0 con 0x562f0a211b40 2024-09-19T09:10:14.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.831+0000 7f598fd78740 1 -- 172.21.15.120:0/200090916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f0a211b40 msgr2=0x562f0a21c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:14.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.831+0000 7f598fd78740 1 --2- 172.21.15.120:0/200090916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f0a211b40 0x562f0a21c760 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f5978007f20 tx=0x7f597800f420 comp rx=0 tx=0).stop 2024-09-19T09:10:14.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.831+0000 7f598fd78740 1 -- 172.21.15.120:0/200090916 shutdown_connections 2024-09-19T09:10:14.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.831+0000 7f598fd78740 1 --2- 172.21.15.120:0/200090916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f0a21b370 0x562f0a21cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.831+0000 7f598fd78740 1 --2- 172.21.15.120:0/200090916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f0a211b40 0x562f0a21c760 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.831+0000 7f598fd78740 1 --2- 172.21.15.120:0/200090916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f0a2105d0 0x562f0a227310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.831+0000 7f598fd78740 1 -- 172.21.15.120:0/200090916 >> 172.21.15.120:0/200090916 conn(0x562f0a1d9310 msgr2=0x562f0a217510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:14.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.831+0000 7f598fd78740 1 -- 172.21.15.120:0/200090916 shutdown_connections 2024-09-19T09:10:14.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.831+0000 7f598fd78740 1 -- 172.21.15.120:0/200090916 wait complete. 2024-09-19T09:10:14.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.831+0000 7f598fd78740 1 -- start start 2024-09-19T09:10:14.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.831+0000 7f598fd78740 1 Processor -- start 2024-09-19T09:10:14.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.832+0000 7f598fd78740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f0a2105d0 0x562f0a223490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:14.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.832+0000 7f598fd78740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f0a211b40 0x562f0a2239d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:14.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.832+0000 7f598fd78740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f0a21b370 0x562f0a21e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:14.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.832+0000 7f598fd78740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f59881ae930 con 0x562f0a21b370 2024-09-19T09:10:14.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.832+0000 7f598fd78740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f59881ae260 con 0x562f0a211b40 2024-09-19T09:10:14.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.832+0000 7f598fd78740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f59881ab480 con 0x562f0a2105d0 2024-09-19T09:10:14.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.832+0000 7f598fd78740 10 client.0 _ll_register_callbacks cb 0x7f5970009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:14.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.832+0000 7f59867fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f0a21b370 0x562f0a21e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:14.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.832+0000 7f59867fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f0a21b370 0x562f0a21e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55146/0 (socket says 172.21.15.120:55146) 2024-09-19T09:10:14.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.832+0000 7f59867fc640 1 -- 172.21.15.120:0/1017917601 learned_addr learned my addr 172.21.15.120:0/1017917601 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:14.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.832+0000 7f5986ffd640 1 --2- 172.21.15.120:0/1017917601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f0a211b40 0x562f0a2239d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:14.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.832+0000 7f59867fc640 1 --2- 172.21.15.120:0/1017917601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f0a2105d0 0x562f0a223490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:14.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.832+0000 7f59867fc640 1 -- 172.21.15.120:0/1017917601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f0a2105d0 msgr2=0x562f0a223490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:14.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.832+0000 7f59867fc640 1 --2- 172.21.15.120:0/1017917601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f0a2105d0 0x562f0a223490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.832+0000 7f59867fc640 1 -- 172.21.15.120:0/1017917601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f0a211b40 msgr2=0x562f0a2239d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:14.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.832+0000 7f59867fc640 1 --2- 172.21.15.120:0/1017917601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f0a211b40 0x562f0a2239d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.832+0000 7f59867fc640 1 -- 172.21.15.120:0/1017917601 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f0a224010 con 0x562f0a21b370 2024-09-19T09:10:14.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.832+0000 7f59867fc640 1 --2- 172.21.15.120:0/1017917601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f0a21b370 0x562f0a21e440 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f5970005c40 tx=0x7f5970003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4a1de2e6b91f6b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:14.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.832+0000 7f5966ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:14.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.832+0000 7f5966ffd640 1 -- 172.21.15.120:0/1017917601 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5970035070 con 0x562f0a21b370 2024-09-19T09:10:14.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.832+0000 7f598fd78740 1 -- 172.21.15.120:0/1017917601 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f597800dbb0 con 0x562f0a21b370 2024-09-19T09:10:14.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.832+0000 7f5966ffd640 1 -- 172.21.15.120:0/1017917601 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f597003d650 con 0x562f0a21b370 2024-09-19T09:10:14.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.833+0000 7f5966ffd640 1 -- 172.21.15.120:0/1017917601 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f597003ddb0 con 0x562f0a21b370 2024-09-19T09:10:14.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.833+0000 7f598fd78740 10 client.17997 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:14.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.833+0000 7f598fd78740 1 -- 172.21.15.120:0/1017917601 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562f0a296000 con 0x562f0a21b370 2024-09-19T09:10:14.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.833+0000 7f5966ffd640 1 -- 172.21.15.120:0/1017917601 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f597800dbb0 con 0x562f0a21b370 2024-09-19T09:10:14.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.833+0000 7f5966ffd640 1 -- 172.21.15.120:0/1017917601 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5970004a70 con 0x562f0a21b370 2024-09-19T09:10:14.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.833+0000 7f598fd78740 10 client.17997 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:14.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.833+0000 7f598fd78740 10 client.17997 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:10:14.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.833+0000 7f598fd78740 20 client.17997 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:14.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.833+0000 7f598fd78740 1 --2- 172.21.15.120:0/1017917601 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x562f0a296770 0x562f0a298b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:14.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.833+0000 7f598fd78740 4 client.17997 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:10:14.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.833+0000 7f598fd78740 1 -- 172.21.15.120:0/1017917601 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562f0a17dd70 con 0x562f0a296770 2024-09-19T09:10:14.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.833+0000 7f5986ffd640 1 --2- 172.21.15.120:0/1017917601 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x562f0a296770 0x562f0a298b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:14.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.834+0000 7f5986ffd640 1 --2- 172.21.15.120:0/1017917601 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x562f0a296770 0x562f0a298b60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4664f71683a13ecf server_cookie=bfb728aa24c497a in_seq=0 out_seq=0 2024-09-19T09:10:14.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.834+0000 7f5966ffd640 10 client.17997 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:10:14.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.834+0000 7f5966ffd640 1 -- 172.21.15.120:0/1017917601 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562f0a17dd70 con 0x562f0a296770 2024-09-19T09:10:14.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.834+0000 7f5966ffd640 10 client.17997 handle_command_reply: tid=0 2024-09-19T09:10:14.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.834+0000 7f598fd78740 1 client.17997 shutdown 2024-09-19T09:10:14.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.834+0000 7f598fd78740 1 -- 172.21.15.120:0/1017917601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f0a21b370 msgr2=0x562f0a21e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:14.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.834+0000 7f598fd78740 1 --2- 172.21.15.120:0/1017917601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f0a21b370 0x562f0a21e440 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f5970005c40 tx=0x7f5970003820 comp rx=0 tx=0).stop 2024-09-19T09:10:14.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.834+0000 7f598fd78740 1 -- 172.21.15.120:0/1017917601 shutdown_connections 2024-09-19T09:10:14.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.834+0000 7f598fd78740 1 --2- 172.21.15.120:0/1017917601 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x562f0a296770 0x562f0a298b60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.834+0000 7f598fd78740 1 --2- 172.21.15.120:0/1017917601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f0a21b370 0x562f0a21e440 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.834+0000 7f598fd78740 1 --2- 172.21.15.120:0/1017917601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f0a211b40 0x562f0a2239d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.834+0000 7f5966ffd640 0 client.17997 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:10:14.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.834+0000 7f598fd78740 1 --2- 172.21.15.120:0/1017917601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f0a2105d0 0x562f0a223490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.834+0000 7f598fd78740 1 -- 172.21.15.120:0/1017917601 >> 172.21.15.120:0/1017917601 conn(0x562f0a1d9310 msgr2=0x562f0a20b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:14.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.834+0000 7f598fd78740 1 -- 172.21.15.120:0/1017917601 shutdown_connections 2024-09-19T09:10:14.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.834+0000 7f598fd78740 1 -- 172.21.15.120:0/1017917601 wait complete. 2024-09-19T09:10:14.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.834+0000 7f598fd78740 20 client.17997 trim_cache size 0 max 16384 2024-09-19T09:10:14.840 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:10:14.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.835+0000 7f598d99f640 1 -- 172.21.15.120:0/3418102470 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f59740a3240 msgr2=0x7f59740a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:14.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.835+0000 7f598d99f640 1 --2- 172.21.15.120:0/3418102470 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f59740a3240 0x7f59740a5700 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5980096320 tx=0x7f598006e700 comp rx=0 tx=0).stop 2024-09-19T09:10:14.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.835+0000 7f598d99f640 1 -- 172.21.15.120:0/3418102470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5988102ef0 msgr2=0x7f598819cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:14.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.835+0000 7f598d99f640 1 --2- 172.21.15.120:0/3418102470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5988102ef0 0x7f598819cdd0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f598000d6b0 tx=0x7f598000db70 comp rx=0 tx=0).stop 2024-09-19T09:10:14.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.835+0000 7f598d99f640 1 -- 172.21.15.120:0/3418102470 shutdown_connections 2024-09-19T09:10:14.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.835+0000 7f598d99f640 1 --2- 172.21.15.120:0/3418102470 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f59740a3240 0x7f59740a5700 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.835+0000 7f598d99f640 1 --2- 172.21.15.120:0/3418102470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5988102ef0 0x7f598819cdd0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.835+0000 7f598d99f640 1 --2- 172.21.15.120:0/3418102470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5988102570 0x7f5988198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.835+0000 7f598d99f640 1 --2- 172.21.15.120:0/3418102470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5988100e20 0x7f5988198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:14.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.835+0000 7f598d99f640 1 -- 172.21.15.120:0/3418102470 >> 172.21.15.120:0/3418102470 conn(0x7f59880fc7d0 msgr2=0x7f598806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:14.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.835+0000 7f598d99f640 1 -- 172.21.15.120:0/3418102470 shutdown_connections 2024-09-19T09:10:14.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:14.835+0000 7f598d99f640 1 -- 172.21.15.120:0/3418102470 wait complete. 2024-09-19T09:10:15.057 INFO:tasks.workunit.client.0.smithi120.stdout: 1 698442 10.21 MB/sec execute 250 sec latency 6865.088 ms 2024-09-19T09:10:15.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:14 smithi120 ceph-mon[31150]: pgmap v563: 97 pgs: 97 active+clean; 49 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 401 op/s 2024-09-19T09:10:15.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:14 smithi196 ceph-mon[35150]: pgmap v563: 97 pgs: 97 active+clean; 49 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 401 op/s 2024-09-19T09:10:15.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:14 smithi153 ceph-mon[35071]: pgmap v563: 97 pgs: 97 active+clean; 49 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 401 op/s 2024-09-19T09:10:15.912 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:10:16.057 INFO:tasks.workunit.client.0.smithi120.stdout: 1 698442 10.17 MB/sec execute 251 sec latency 7865.158 ms 2024-09-19T09:10:16.177 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:10:16.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.667+0000 7f170cbe5640 1 -- 172.21.15.120:0/3783502357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1708102790 msgr2=0x7f17081067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:16.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.667+0000 7f170cbe5640 1 --2- 172.21.15.120:0/3783502357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1708102790 0x7f17081067e0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f16f0009900 tx=0x7f16f002a410 comp rx=0 tx=0).stop 2024-09-19T09:10:16.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.667+0000 7f170cbe5640 1 -- 172.21.15.120:0/3783502357 shutdown_connections 2024-09-19T09:10:16.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.667+0000 7f170cbe5640 1 --2- 172.21.15.120:0/3783502357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1708106f10 0x7f170810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.667+0000 7f170cbe5640 1 --2- 172.21.15.120:0/3783502357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1708102790 0x7f17081067e0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.667+0000 7f170cbe5640 1 --2- 172.21.15.120:0/3783502357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1708101de0 0x7f17081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.667+0000 7f170cbe5640 1 -- 172.21.15.120:0/3783502357 >> 172.21.15.120:0/3783502357 conn(0x7f17080fd8c0 msgr2=0x7f17080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:16.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.667+0000 7f170cbe5640 1 -- 172.21.15.120:0/3783502357 shutdown_connections 2024-09-19T09:10:16.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.667+0000 7f170cbe5640 1 -- 172.21.15.120:0/3783502357 wait complete. 2024-09-19T09:10:16.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.668+0000 7f170cbe5640 1 Processor -- start 2024-09-19T09:10:16.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.668+0000 7f170cbe5640 1 -- start start 2024-09-19T09:10:16.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.668+0000 7f170cbe5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1708101de0 0x7f17081984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:16.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.668+0000 7f170cbe5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1708102790 0x7f17081989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:16.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.668+0000 7f170cbe5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1708106f10 0x7f170819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:16.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.668+0000 7f170cbe5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1708111f70 con 0x7f1708101de0 2024-09-19T09:10:16.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.668+0000 7f170cbe5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1708111df0 con 0x7f1708102790 2024-09-19T09:10:16.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.668+0000 7f170cbe5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17081120f0 con 0x7f1708106f10 2024-09-19T09:10:16.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.669+0000 7f170659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1708101de0 0x7f17081984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:16.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.669+0000 7f170659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1708101de0 0x7f17081984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55168/0 (socket says 172.21.15.120:55168) 2024-09-19T09:10:16.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.669+0000 7f1705d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1708102790 0x7f17081989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:16.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.669+0000 7f170659c640 1 -- 172.21.15.120:0/375796948 learned_addr learned my addr 172.21.15.120:0/375796948 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:16.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.669+0000 7f1706d9d640 1 --2- 172.21.15.120:0/375796948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1708106f10 0x7f170819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:16.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.669+0000 7f170659c640 1 -- 172.21.15.120:0/375796948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1708106f10 msgr2=0x7f170819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:16.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.669+0000 7f170659c640 1 --2- 172.21.15.120:0/375796948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1708106f10 0x7f170819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.669+0000 7f170659c640 1 -- 172.21.15.120:0/375796948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1708102790 msgr2=0x7f17081989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:16.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.669+0000 7f170659c640 1 --2- 172.21.15.120:0/375796948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1708102790 0x7f17081989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.669+0000 7f170659c640 1 -- 172.21.15.120:0/375796948 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f170819d460 con 0x7f1708101de0 2024-09-19T09:10:16.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.669+0000 7f170659c640 1 --2- 172.21.15.120:0/375796948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1708101de0 0x7f17081984b0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f170000eb90 tx=0x7f170000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:16.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.669+0000 7f16ff7fe640 1 -- 172.21.15.120:0/375796948 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1700004d20 con 0x7f1708101de0 2024-09-19T09:10:16.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.669+0000 7f170cbe5640 1 -- 172.21.15.120:0/375796948 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f170819d750 con 0x7f1708101de0 2024-09-19T09:10:16.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.669+0000 7f16ff7fe640 1 -- 172.21.15.120:0/375796948 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f170000ee90 con 0x7f1708101de0 2024-09-19T09:10:16.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.669+0000 7f16ff7fe640 1 -- 172.21.15.120:0/375796948 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1700010830 con 0x7f1708101de0 2024-09-19T09:10:16.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.669+0000 7f170cbe5640 1 -- 172.21.15.120:0/375796948 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17081a5030 con 0x7f1708101de0 2024-09-19T09:10:16.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.671+0000 7f16ff7fe640 1 -- 172.21.15.120:0/375796948 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f1700005020 con 0x7f1708101de0 2024-09-19T09:10:16.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.672+0000 7f170efbe740 1 Processor -- start 2024-09-19T09:10:16.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.672+0000 7f170efbe740 1 -- start start 2024-09-19T09:10:16.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.672+0000 7f16ff7fe640 1 --2- 172.21.15.120:0/375796948 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16f40a3240 0x7f16f40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:16.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.672+0000 7f170efbe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ad8448b40 0x558ad8447090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:16.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.672+0000 7f170efbe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ad84475d0 0x558ad8447a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:16.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.672+0000 7f170efbe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ad8452370 0x558ad8454760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:16.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.672+0000 7f170efbe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558ad83ad2f0 con 0x558ad84475d0 2024-09-19T09:10:16.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.672+0000 7f170efbe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558ad8354cf0 con 0x558ad8452370 2024-09-19T09:10:16.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.672+0000 7f170efbe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558ad83a92b0 con 0x558ad8448b40 2024-09-19T09:10:16.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.672+0000 7f16ff7fe640 1 -- 172.21.15.120:0/375796948 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f17000cff50 con 0x7f1708101de0 2024-09-19T09:10:16.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.672+0000 7f1706d9d640 1 --2- 172.21.15.120:0/375796948 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16f40a3240 0x7f16f40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:16.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.673+0000 7f1705d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ad84475d0 0x558ad8447a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:16.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.673+0000 7f1705d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ad84475d0 0x558ad8447a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55170/0 (socket says 172.21.15.120:55170) 2024-09-19T09:10:16.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.673+0000 7f170659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ad8448b40 0x558ad8447090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:16.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.673+0000 7f1705d9b640 1 -- 172.21.15.120:0/1179610238 learned_addr learned my addr 172.21.15.120:0/1179610238 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:16.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.673+0000 7f1706d9d640 1 --2- 172.21.15.120:0/1179610238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ad8452370 0x558ad8454760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:16.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.673+0000 7f1706d9d640 1 --2- 172.21.15.120:0/375796948 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16f40a3240 0x7f16f40a5700 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f16f800b730 tx=0x7f16f800bb30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:16.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.673+0000 7f1705d9b640 1 -- 172.21.15.120:0/1179610238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ad8448b40 msgr2=0x558ad8447090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:16.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.673+0000 7f1705d9b640 1 --2- 172.21.15.120:0/1179610238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ad8448b40 0x558ad8447090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.673+0000 7f1705d9b640 1 -- 172.21.15.120:0/1179610238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ad8452370 msgr2=0x558ad8454760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:16.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.673+0000 7f1705d9b640 1 --2- 172.21.15.120:0/1179610238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ad8452370 0x558ad8454760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.673+0000 7f1705d9b640 1 -- 172.21.15.120:0/1179610238 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ad83393a0 con 0x558ad84475d0 2024-09-19T09:10:16.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.673+0000 7f1706d9d640 1 --2- 172.21.15.120:0/1179610238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ad8452370 0x558ad8454760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:10:16.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.673+0000 7f1705d9b640 1 --2- 172.21.15.120:0/1179610238 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ad84475d0 0x558ad8447a10 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f16f0009900 tx=0x7f16f0005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abdea77e9e14e38d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:16.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.673+0000 7f16fcff9640 1 -- 172.21.15.120:0/1179610238 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f0040070 con 0x558ad84475d0 2024-09-19T09:10:16.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.673+0000 7f16fcff9640 1 -- 172.21.15.120:0/1179610238 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16f00318f0 con 0x558ad84475d0 2024-09-19T09:10:16.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.673+0000 7f16fcff9640 1 -- 172.21.15.120:0/1179610238 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f0035040 con 0x558ad84475d0 2024-09-19T09:10:16.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.674+0000 7f170efbe740 1 -- 172.21.15.120:0/1179610238 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ad84475d0 msgr2=0x558ad8447a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:16.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.674+0000 7f170efbe740 1 --2- 172.21.15.120:0/1179610238 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ad84475d0 0x558ad8447a10 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f16f0009900 tx=0x7f16f0005e00 comp rx=0 tx=0).stop 2024-09-19T09:10:16.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.674+0000 7f170efbe740 1 -- 172.21.15.120:0/1179610238 shutdown_connections 2024-09-19T09:10:16.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.674+0000 7f170efbe740 1 --2- 172.21.15.120:0/1179610238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ad8452370 0x558ad8454760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.674+0000 7f170efbe740 1 --2- 172.21.15.120:0/1179610238 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ad84475d0 0x558ad8447a10 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.674+0000 7f170efbe740 1 --2- 172.21.15.120:0/1179610238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ad8448b40 0x558ad8447090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.674+0000 7f170efbe740 1 -- 172.21.15.120:0/1179610238 >> 172.21.15.120:0/1179610238 conn(0x558ad8410310 msgr2=0x558ad8443c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:16.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.674+0000 7f170efbe740 1 -- 172.21.15.120:0/1179610238 shutdown_connections 2024-09-19T09:10:16.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.674+0000 7f170efbe740 1 -- 172.21.15.120:0/1179610238 wait complete. 2024-09-19T09:10:16.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.674+0000 7f170efbe740 1 -- start start 2024-09-19T09:10:16.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.675+0000 7f170efbe740 1 Processor -- start 2024-09-19T09:10:16.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.675+0000 7f170efbe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ad84475d0 0x558ad8451d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:16.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.675+0000 7f170efbe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ad8448b40 0x558ad844efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:16.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.675+0000 7f170efbe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ad8452370 0x558ad844f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:16.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.675+0000 7f170efbe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558ad8459420 con 0x558ad84475d0 2024-09-19T09:10:16.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.675+0000 7f170efbe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558ad8368060 con 0x558ad8448b40 2024-09-19T09:10:16.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.675+0000 7f170efbe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558ad84595a0 con 0x558ad8452370 2024-09-19T09:10:16.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.675+0000 7f170efbe740 10 client.0 _ll_register_callbacks cb 0x558ad80bb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:16.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.675+0000 7f170659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ad84475d0 0x558ad8451d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:16.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.675+0000 7f170659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ad84475d0 0x558ad8451d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55176/0 (socket says 172.21.15.120:55176) 2024-09-19T09:10:16.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.675+0000 7f170659c640 1 -- 172.21.15.120:0/318046478 learned_addr learned my addr 172.21.15.120:0/318046478 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:16.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.675+0000 7f1705d9b640 1 --2- 172.21.15.120:0/318046478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ad8448b40 0x558ad844efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:16.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.675+0000 7f1706d9d640 1 --2- 172.21.15.120:0/318046478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ad8452370 0x558ad844f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:16.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.675+0000 7f170659c640 1 -- 172.21.15.120:0/318046478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ad8452370 msgr2=0x558ad844f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:16.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.675+0000 7f170659c640 1 --2- 172.21.15.120:0/318046478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ad8452370 0x558ad844f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.675+0000 7f170659c640 1 -- 172.21.15.120:0/318046478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ad8448b40 msgr2=0x558ad844efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:16.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.675+0000 7f170659c640 1 --2- 172.21.15.120:0/318046478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ad8448b40 0x558ad844efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.675+0000 7f170659c640 1 -- 172.21.15.120:0/318046478 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ad811e3c0 con 0x558ad84475d0 2024-09-19T09:10:16.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.675+0000 7f1706d9d640 1 --2- 172.21.15.120:0/318046478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ad8452370 0x558ad844f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:10:16.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.675+0000 7f170659c640 1 --2- 172.21.15.120:0/318046478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ad84475d0 0x558ad8451d20 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f170000cba0 tx=0x7f1700098940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290acae6e2f3892c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:16.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.675+0000 7f16edffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:16.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.675+0000 7f16edffb640 1 -- 172.21.15.120:0/318046478 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170009c570 con 0x558ad84475d0 2024-09-19T09:10:16.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.676+0000 7f16edffb640 1 -- 172.21.15.120:0/318046478 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f17000cdb60 con 0x558ad84475d0 2024-09-19T09:10:16.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.676+0000 7f170efbe740 1 -- 172.21.15.120:0/318046478 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16f00318f0 con 0x558ad84475d0 2024-09-19T09:10:16.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.676+0000 7f16edffb640 1 -- 172.21.15.120:0/318046478 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f17000a0920 con 0x558ad84475d0 2024-09-19T09:10:16.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.676+0000 7f170efbe740 10 client.18009 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:16.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.676+0000 7f170efbe740 1 -- 172.21.15.120:0/318046478 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558ad8464ed0 con 0x558ad84475d0 2024-09-19T09:10:16.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.676+0000 7f16edffb640 1 -- 172.21.15.120:0/318046478 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16f00318f0 con 0x558ad84475d0 2024-09-19T09:10:16.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.676+0000 7f16edffb640 1 -- 172.21.15.120:0/318046478 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f17000a4d50 con 0x558ad84475d0 2024-09-19T09:10:16.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.676+0000 7f170efbe740 10 client.18009 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:16.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.676+0000 7f170efbe740 10 client.18009 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:10:16.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.676+0000 7f170efbe740 20 client.18009 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:16.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.676+0000 7f170efbe740 1 --2- 172.21.15.120:0/318046478 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x558ad8466330 0x558ad8468780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:16.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.676+0000 7f170efbe740 4 client.18009 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:10:16.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.676+0000 7f170efbe740 1 -- 172.21.15.120:0/318046478 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ad83ba860 con 0x558ad8466330 2024-09-19T09:10:16.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.676+0000 7f1705d9b640 1 --2- 172.21.15.120:0/318046478 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x558ad8466330 0x558ad8468780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:16.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.677+0000 7f16edffb640 10 client.18009 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:10:16.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.677+0000 7f1705d9b640 1 --2- 172.21.15.120:0/318046478 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x558ad8466330 0x558ad8468780 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3948e7caddb09fbd server_cookie=78fc10a87564ce8d in_seq=0 out_seq=0 2024-09-19T09:10:16.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.677+0000 7f16edffb640 1 -- 172.21.15.120:0/318046478 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558ad83ba860 con 0x558ad8466330 2024-09-19T09:10:16.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.677+0000 7f16edffb640 10 client.18009 handle_command_reply: tid=0 2024-09-19T09:10:16.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.677+0000 7f170efbe740 1 client.18009 shutdown 2024-09-19T09:10:16.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.677+0000 7f170efbe740 1 -- 172.21.15.120:0/318046478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ad84475d0 msgr2=0x558ad8451d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:16.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.677+0000 7f170efbe740 1 --2- 172.21.15.120:0/318046478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ad84475d0 0x558ad8451d20 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f170000cba0 tx=0x7f1700098940 comp rx=0 tx=0).stop 2024-09-19T09:10:16.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.677+0000 7f170efbe740 1 -- 172.21.15.120:0/318046478 shutdown_connections 2024-09-19T09:10:16.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.677+0000 7f170efbe740 1 --2- 172.21.15.120:0/318046478 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x558ad8466330 0x558ad8468780 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.678+0000 7f170efbe740 1 --2- 172.21.15.120:0/318046478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ad8452370 0x558ad844f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.678+0000 7f170efbe740 1 --2- 172.21.15.120:0/318046478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ad8448b40 0x558ad844efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.678+0000 7f16edffb640 0 client.18009 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:10:16.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.678+0000 7f170efbe740 1 --2- 172.21.15.120:0/318046478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ad84475d0 0x558ad8451d20 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.678+0000 7f170efbe740 1 -- 172.21.15.120:0/318046478 >> 172.21.15.120:0/318046478 conn(0x558ad8410310 msgr2=0x558ad8449960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:16.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.678+0000 7f170efbe740 1 -- 172.21.15.120:0/318046478 shutdown_connections 2024-09-19T09:10:16.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.678+0000 7f170efbe740 1 -- 172.21.15.120:0/318046478 wait complete. 2024-09-19T09:10:16.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.678+0000 7f170efbe740 20 client.18009 trim_cache size 0 max 16384 2024-09-19T09:10:16.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.690+0000 7f170efbe740 1 Processor -- start 2024-09-19T09:10:16.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.690+0000 7f170efbe740 1 -- start start 2024-09-19T09:10:16.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.690+0000 7f170efbe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ad84475d0 0x558ad84626c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:16.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.690+0000 7f170efbe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ad8448b40 0x558ad8453760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:16.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.691+0000 7f170efbe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ad8452370 0x558ad8453ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:16.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.691+0000 7f170efbe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f17081ab2f0 con 0x558ad84475d0 2024-09-19T09:10:16.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.691+0000 7f170efbe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f17081ace30 con 0x558ad8452370 2024-09-19T09:10:16.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.691+0000 7f170efbe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17081ab9c0 con 0x558ad8448b40 2024-09-19T09:10:16.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.691+0000 7f170659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ad84475d0 0x558ad84626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:16.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.691+0000 7f170659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ad84475d0 0x558ad84626c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55184/0 (socket says 172.21.15.120:55184) 2024-09-19T09:10:16.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.691+0000 7f170659c640 1 -- 172.21.15.120:0/1988854034 learned_addr learned my addr 172.21.15.120:0/1988854034 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:16.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.691+0000 7f1706d9d640 1 --2- 172.21.15.120:0/1988854034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ad8452370 0x558ad8453ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:16.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.691+0000 7f1705d9b640 1 --2- 172.21.15.120:0/1988854034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ad8448b40 0x558ad8453760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:16.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.691+0000 7f170659c640 1 -- 172.21.15.120:0/1988854034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ad8448b40 msgr2=0x558ad8453760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:16.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.691+0000 7f170659c640 1 --2- 172.21.15.120:0/1988854034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ad8448b40 0x558ad8453760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.691+0000 7f170659c640 1 -- 172.21.15.120:0/1988854034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ad8452370 msgr2=0x558ad8453ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:16.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.691+0000 7f170659c640 1 --2- 172.21.15.120:0/1988854034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ad8452370 0x558ad8453ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.691+0000 7f170659c640 1 -- 172.21.15.120:0/1988854034 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ad844d520 con 0x558ad84475d0 2024-09-19T09:10:16.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.691+0000 7f1705d9b640 1 --2- 172.21.15.120:0/1988854034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ad8448b40 0x558ad8453760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:10:16.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.691+0000 7f1706d9d640 1 --2- 172.21.15.120:0/1988854034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ad8452370 0x558ad8453ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:10:16.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.691+0000 7f170659c640 1 --2- 172.21.15.120:0/1988854034 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ad84475d0 0x558ad84626c0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f170009c270 tx=0x7f170000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273c2b0621692d47 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:16.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.691+0000 7f16effff640 1 -- 172.21.15.120:0/1988854034 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170009bed0 con 0x558ad84475d0 2024-09-19T09:10:16.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.691+0000 7f16effff640 1 -- 172.21.15.120:0/1988854034 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f170009c570 con 0x558ad84475d0 2024-09-19T09:10:16.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.691+0000 7f16effff640 1 -- 172.21.15.120:0/1988854034 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170009c340 con 0x558ad84475d0 2024-09-19T09:10:16.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.692+0000 7f170efbe740 1 -- 172.21.15.120:0/1988854034 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ad84475d0 msgr2=0x558ad84626c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:16.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.692+0000 7f170efbe740 1 --2- 172.21.15.120:0/1988854034 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ad84475d0 0x558ad84626c0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f170009c270 tx=0x7f170000ccd0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.692+0000 7f170efbe740 1 -- 172.21.15.120:0/1988854034 shutdown_connections 2024-09-19T09:10:16.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.692+0000 7f170efbe740 1 --2- 172.21.15.120:0/1988854034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ad8452370 0x558ad8453ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.692+0000 7f170efbe740 1 --2- 172.21.15.120:0/1988854034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ad8448b40 0x558ad8453760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.692+0000 7f170efbe740 1 --2- 172.21.15.120:0/1988854034 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ad84475d0 0x558ad84626c0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.692+0000 7f170efbe740 1 -- 172.21.15.120:0/1988854034 >> 172.21.15.120:0/1988854034 conn(0x558ad8410310 msgr2=0x558ad8443700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:16.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.692+0000 7f170efbe740 1 -- 172.21.15.120:0/1988854034 shutdown_connections 2024-09-19T09:10:16.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.692+0000 7f170efbe740 1 -- 172.21.15.120:0/1988854034 wait complete. 2024-09-19T09:10:16.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.692+0000 7f170efbe740 1 -- start start 2024-09-19T09:10:16.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.692+0000 7f170efbe740 1 Processor -- start 2024-09-19T09:10:16.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.693+0000 7f170efbe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ad84475d0 0x558ad844cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:16.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.693+0000 7f170efbe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ad8448b40 0x558ad845e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:16.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.693+0000 7f170efbe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ad8452370 0x558ad845efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:16.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.693+0000 7f170efbe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f17081ac760 con 0x558ad8452370 2024-09-19T09:10:16.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.693+0000 7f170efbe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f17081ac090 con 0x558ad8448b40 2024-09-19T09:10:16.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.693+0000 7f170efbe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17081a92b0 con 0x558ad84475d0 2024-09-19T09:10:16.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.693+0000 7f170efbe740 10 client.0 _ll_register_callbacks cb 0x7f17000099d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:16.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.693+0000 7f1706d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ad8452370 0x558ad845efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:16.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.693+0000 7f1706d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ad8452370 0x558ad845efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55194/0 (socket says 172.21.15.120:55194) 2024-09-19T09:10:16.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.693+0000 7f1706d9d640 1 -- 172.21.15.120:0/3968126789 learned_addr learned my addr 172.21.15.120:0/3968126789 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:16.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.693+0000 7f1705d9b640 1 --2- 172.21.15.120:0/3968126789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ad8448b40 0x558ad845e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:16.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.693+0000 7f170659c640 1 --2- 172.21.15.120:0/3968126789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ad84475d0 0x558ad844cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:16.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.693+0000 7f1706d9d640 1 -- 172.21.15.120:0/3968126789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ad84475d0 msgr2=0x558ad844cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:16.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.693+0000 7f1706d9d640 1 --2- 172.21.15.120:0/3968126789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ad84475d0 0x558ad844cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.693+0000 7f1706d9d640 1 -- 172.21.15.120:0/3968126789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ad8448b40 msgr2=0x558ad845e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:16.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.693+0000 7f1706d9d640 1 --2- 172.21.15.120:0/3968126789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ad8448b40 0x558ad845e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.693+0000 7f1706d9d640 1 -- 172.21.15.120:0/3968126789 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ad846dcc0 con 0x558ad8452370 2024-09-19T09:10:16.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.693+0000 7f170659c640 1 --2- 172.21.15.120:0/3968126789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ad84475d0 0x558ad844cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:10:16.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.693+0000 7f1705d9b640 1 --2- 172.21.15.120:0/3968126789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ad8448b40 0x558ad845e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:10:16.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.693+0000 7f1706d9d640 1 --2- 172.21.15.120:0/3968126789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ad8452370 0x558ad845efc0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f16f800bce0 tx=0x7f16f8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35c254bffbc77bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:16.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.693+0000 7f16edffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:16.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.693+0000 7f16edffb640 1 -- 172.21.15.120:0/3968126789 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f801d420 con 0x558ad8452370 2024-09-19T09:10:16.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.693+0000 7f170efbe740 1 -- 172.21.15.120:0/3968126789 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f170009c570 con 0x558ad8452370 2024-09-19T09:10:16.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.693+0000 7f16edffb640 1 -- 172.21.15.120:0/3968126789 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f16f801d5c0 con 0x558ad8452370 2024-09-19T09:10:16.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.694+0000 7f16edffb640 1 -- 172.21.15.120:0/3968126789 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f16f8030050 con 0x558ad8452370 2024-09-19T09:10:16.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.694+0000 7f170efbe740 10 client.18015 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:16.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.694+0000 7f170efbe740 1 -- 172.21.15.120:0/3968126789 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558ad84d12b0 con 0x558ad8452370 2024-09-19T09:10:16.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.694+0000 7f16edffb640 1 -- 172.21.15.120:0/3968126789 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f170009c570 con 0x558ad8452370 2024-09-19T09:10:16.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.694+0000 7f16edffb640 1 -- 172.21.15.120:0/3968126789 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f16f8027070 con 0x558ad8452370 2024-09-19T09:10:16.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.694+0000 7f170efbe740 10 client.18015 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:16.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.694+0000 7f170efbe740 10 client.18015 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:10:16.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.694+0000 7f170efbe740 20 client.18015 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:16.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.694+0000 7f170efbe740 1 --2- 172.21.15.120:0/3968126789 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x558ad84d1a00 0x558ad84d3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:16.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.694+0000 7f170efbe740 4 client.18015 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:10:16.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.694+0000 7f170efbe740 1 -- 172.21.15.120:0/3968126789 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ad83b4d70 con 0x558ad84d1a00 2024-09-19T09:10:16.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.694+0000 7f170659c640 1 --2- 172.21.15.120:0/3968126789 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x558ad84d1a00 0x558ad84d3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:16.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.695+0000 7f16edffb640 10 client.18015 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:10:16.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.695+0000 7f170659c640 1 --2- 172.21.15.120:0/3968126789 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x558ad84d1a00 0x558ad84d3e20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5c8e0eaf729572c4 server_cookie=ce84ad2d16fb7a55 in_seq=0 out_seq=0 2024-09-19T09:10:16.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.695+0000 7f16edffb640 1 -- 172.21.15.120:0/3968126789 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ad83b4d70 con 0x558ad84d1a00 2024-09-19T09:10:16.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.695+0000 7f16edffb640 10 client.18015 handle_command_reply: tid=0 2024-09-19T09:10:16.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.695+0000 7f170efbe740 1 client.18015 shutdown 2024-09-19T09:10:16.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.695+0000 7f170efbe740 1 -- 172.21.15.120:0/3968126789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ad8452370 msgr2=0x558ad845efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:16.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.695+0000 7f170efbe740 1 --2- 172.21.15.120:0/3968126789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ad8452370 0x558ad845efc0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f16f800bce0 tx=0x7f16f8017000 comp rx=0 tx=0).stop 2024-09-19T09:10:16.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.695+0000 7f170efbe740 1 -- 172.21.15.120:0/3968126789 shutdown_connections 2024-09-19T09:10:16.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.695+0000 7f170efbe740 1 --2- 172.21.15.120:0/3968126789 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x558ad84d1a00 0x558ad84d3e20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.695+0000 7f170efbe740 1 --2- 172.21.15.120:0/3968126789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ad8452370 0x558ad845efc0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.695+0000 7f170efbe740 1 --2- 172.21.15.120:0/3968126789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ad8448b40 0x558ad845e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.695+0000 7f170efbe740 1 --2- 172.21.15.120:0/3968126789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ad84475d0 0x558ad844cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.695+0000 7f170efbe740 1 -- 172.21.15.120:0/3968126789 >> 172.21.15.120:0/3968126789 conn(0x558ad8410310 msgr2=0x558ad8442310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:16.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.695+0000 7f16edffb640 0 client.18015 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:10:16.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.695+0000 7f170efbe740 1 -- 172.21.15.120:0/3968126789 shutdown_connections 2024-09-19T09:10:16.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.695+0000 7f170efbe740 1 -- 172.21.15.120:0/3968126789 wait complete. 2024-09-19T09:10:16.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.695+0000 7f170efbe740 20 client.18015 trim_cache size 0 max 16384 2024-09-19T09:10:16.727 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:10:16.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.696+0000 7f170cbe5640 1 -- 172.21.15.120:0/375796948 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16f40a3240 msgr2=0x7f16f40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:16.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.696+0000 7f170cbe5640 1 --2- 172.21.15.120:0/375796948 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16f40a3240 0x7f16f40a5700 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f16f800b730 tx=0x7f16f800bb30 comp rx=0 tx=0).stop 2024-09-19T09:10:16.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.696+0000 7f170cbe5640 1 -- 172.21.15.120:0/375796948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1708101de0 msgr2=0x7f17081984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:16.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.696+0000 7f170cbe5640 1 --2- 172.21.15.120:0/375796948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1708101de0 0x7f17081984b0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f170000eb90 tx=0x7f170000c6a0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.696+0000 7f170cbe5640 1 -- 172.21.15.120:0/375796948 shutdown_connections 2024-09-19T09:10:16.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.696+0000 7f170cbe5640 1 --2- 172.21.15.120:0/375796948 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16f40a3240 0x7f16f40a5700 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.696+0000 7f170cbe5640 1 --2- 172.21.15.120:0/375796948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1708106f10 0x7f170819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.696+0000 7f170cbe5640 1 --2- 172.21.15.120:0/375796948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1708102790 0x7f17081989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.696+0000 7f170cbe5640 1 --2- 172.21.15.120:0/375796948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1708101de0 0x7f17081984b0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:16.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.696+0000 7f170cbe5640 1 -- 172.21.15.120:0/375796948 >> 172.21.15.120:0/375796948 conn(0x7f17080fd8c0 msgr2=0x7f17080fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:16.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.696+0000 7f170cbe5640 1 -- 172.21.15.120:0/375796948 shutdown_connections 2024-09-19T09:10:16.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:16.696+0000 7f170cbe5640 1 -- 172.21.15.120:0/375796948 wait complete. 2024-09-19T09:10:17.057 INFO:tasks.workunit.client.0.smithi120.stdout: 1 698442 10.13 MB/sec execute 252 sec latency 8865.234 ms 2024-09-19T09:10:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:16 smithi196 ceph-mon[35150]: pgmap v564: 97 pgs: 97 active+clean; 48 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 302 op/s 2024-09-19T09:10:17.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:16 smithi153 ceph-mon[35071]: pgmap v564: 97 pgs: 97 active+clean; 48 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 302 op/s 2024-09-19T09:10:17.320 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:16 smithi120 ceph-mon[31150]: pgmap v564: 97 pgs: 97 active+clean; 48 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 302 op/s 2024-09-19T09:10:17.590 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:10:17.835 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:10:18.057 INFO:tasks.workunit.client.0.smithi120.stdout: 1 698442 10.09 MB/sec execute 253 sec latency 9865.271 ms 2024-09-19T09:10:18.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.302+0000 7ff71382b640 1 -- 172.21.15.120:0/3029411415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff70c106a80 msgr2=0x7ff70c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:18.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.302+0000 7ff71382b640 1 --2- 172.21.15.120:0/3029411415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff70c106a80 0x7ff70c106e60 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7ff708009930 tx=0x7ff70802a420 comp rx=0 tx=0).stop 2024-09-19T09:10:18.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.302+0000 7ff71382b640 1 -- 172.21.15.120:0/3029411415 shutdown_connections 2024-09-19T09:10:18.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.302+0000 7ff71382b640 1 --2- 172.21.15.120:0/3029411415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff70c101420 0x7ff70c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.302+0000 7ff71382b640 1 --2- 172.21.15.120:0/3029411415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff70c100a80 0x7ff70c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.302+0000 7ff71382b640 1 --2- 172.21.15.120:0/3029411415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff70c106a80 0x7ff70c106e60 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.302+0000 7ff71382b640 1 -- 172.21.15.120:0/3029411415 >> 172.21.15.120:0/3029411415 conn(0x7ff70c0fc7a0 msgr2=0x7ff70c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:18.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.302+0000 7ff71382b640 1 -- 172.21.15.120:0/3029411415 shutdown_connections 2024-09-19T09:10:18.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.302+0000 7ff71382b640 1 -- 172.21.15.120:0/3029411415 wait complete. 2024-09-19T09:10:18.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.303+0000 7ff71382b640 1 Processor -- start 2024-09-19T09:10:18.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.303+0000 7ff71382b640 1 -- start start 2024-09-19T09:10:18.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.303+0000 7ff71382b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff70c100a80 0x7ff70c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:18.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.303+0000 7ff71382b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff70c101420 0x7ff70c199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:18.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.303+0000 7ff71382b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff70c106a80 0x7ff70c1a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:18.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.303+0000 7ff71382b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c110e80 con 0x7ff70c106a80 2024-09-19T09:10:18.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.303+0000 7ff71382b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c110d00 con 0x7ff70c101420 2024-09-19T09:10:18.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.303+0000 7ff71382b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c111000 con 0x7ff70c100a80 2024-09-19T09:10:18.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.303+0000 7ff711dc8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff70c106a80 0x7ff70c1a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:18.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.303+0000 7ff711dc8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff70c106a80 0x7ff70c1a4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55210/0 (socket says 172.21.15.120:55210) 2024-09-19T09:10:18.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.303+0000 7ff711dc8640 1 -- 172.21.15.120:0/2679080218 learned_addr learned my addr 172.21.15.120:0/2679080218 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:18.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.303+0000 7ff710dc6640 1 --2- 172.21.15.120:0/2679080218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff70c101420 0x7ff70c199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:18.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.303+0000 7ff7115c7640 1 --2- 172.21.15.120:0/2679080218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff70c100a80 0x7ff70c1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:18.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.303+0000 7ff711dc8640 1 -- 172.21.15.120:0/2679080218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff70c100a80 msgr2=0x7ff70c1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:18.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.303+0000 7ff711dc8640 1 --2- 172.21.15.120:0/2679080218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff70c100a80 0x7ff70c1992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.303+0000 7ff711dc8640 1 -- 172.21.15.120:0/2679080218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff70c101420 msgr2=0x7ff70c199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:18.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.303+0000 7ff711dc8640 1 --2- 172.21.15.120:0/2679080218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff70c101420 0x7ff70c199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.303+0000 7ff711dc8640 1 -- 172.21.15.120:0/2679080218 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff70c1a48b0 con 0x7ff70c106a80 2024-09-19T09:10:18.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.304+0000 7ff711dc8640 1 --2- 172.21.15.120:0/2679080218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff70c106a80 0x7ff70c1a4370 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7ff70400cad0 tx=0x7ff70400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:18.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.304+0000 7ff7027fc640 1 -- 172.21.15.120:0/2679080218 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70400ae20 con 0x7ff70c106a80 2024-09-19T09:10:18.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.304+0000 7ff71382b640 1 -- 172.21.15.120:0/2679080218 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff70c1a4a80 con 0x7ff70c106a80 2024-09-19T09:10:18.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.304+0000 7ff7027fc640 1 -- 172.21.15.120:0/2679080218 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff704005020 con 0x7ff70c106a80 2024-09-19T09:10:18.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.304+0000 7ff7027fc640 1 -- 172.21.15.120:0/2679080218 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70400f6f0 con 0x7ff70c106a80 2024-09-19T09:10:18.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.304+0000 7ff71382b640 1 -- 172.21.15.120:0/2679080218 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff70c1a4f40 con 0x7ff70c106a80 2024-09-19T09:10:18.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.307+0000 7ff7027fc640 1 -- 172.21.15.120:0/2679080218 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7ff7040051d0 con 0x7ff70c106a80 2024-09-19T09:10:18.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.307+0000 7ff715c04740 1 Processor -- start 2024-09-19T09:10:18.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.307+0000 7ff715c04740 1 -- start start 2024-09-19T09:10:18.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.307+0000 7ff715c04740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f8af34b40 0x560f8af33090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:18.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.307+0000 7ff715c04740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f8af335d0 0x560f8af33a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:18.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.307+0000 7ff715c04740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f8af42780 0x560f8af44b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:18.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.307+0000 7ff715c04740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560f8ae992f0 con 0x560f8af42780 2024-09-19T09:10:18.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.307+0000 7ff715c04740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560f8ae40cf0 con 0x560f8af335d0 2024-09-19T09:10:18.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.307+0000 7ff715c04740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560f8ae952b0 con 0x560f8af34b40 2024-09-19T09:10:18.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.307+0000 7ff710dc6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f8af42780 0x560f8af44b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:18.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.307+0000 7ff710dc6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f8af42780 0x560f8af44b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55220/0 (socket says 172.21.15.120:55220) 2024-09-19T09:10:18.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.307+0000 7ff710dc6640 1 -- 172.21.15.120:0/2588375095 learned_addr learned my addr 172.21.15.120:0/2588375095 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:18.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.308+0000 7ff710dc6640 1 --2- 172.21.15.120:0/2588375095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f8af34b40 0x560f8af33090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:18.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.308+0000 7ff7115c7640 1 --2- 172.21.15.120:0/2588375095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f8af335d0 0x560f8af33a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:18.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.308+0000 7ff7027fc640 1 --2- 172.21.15.120:0/2679080218 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff6f80a3240 0x7ff6f80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:18.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.308+0000 7ff711dc8640 1 --2- 172.21.15.120:0/2679080218 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff6f80a3240 0x7ff6f80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:18.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.308+0000 7ff710dc6640 1 -- 172.21.15.120:0/2588375095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f8af34b40 msgr2=0x560f8af33090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:18.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.308+0000 7ff710dc6640 1 --2- 172.21.15.120:0/2588375095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f8af34b40 0x560f8af33090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.308+0000 7ff710dc6640 1 -- 172.21.15.120:0/2588375095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f8af335d0 msgr2=0x560f8af33a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:18.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.308+0000 7ff710dc6640 1 --2- 172.21.15.120:0/2588375095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f8af335d0 0x560f8af33a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.308+0000 7ff710dc6640 1 -- 172.21.15.120:0/2588375095 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f8ae253a0 con 0x560f8af42780 2024-09-19T09:10:18.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.308+0000 7ff7027fc640 1 -- 172.21.15.120:0/2679080218 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7ff7040cf940 con 0x7ff70c106a80 2024-09-19T09:10:18.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.308+0000 7ff710dc6640 1 --2- 172.21.15.120:0/2588375095 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f8af42780 0x560f8af44b70 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc0127c0 tx=0x7ff6fc00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e220ba401cc998e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:18.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.308+0000 7ff6f77fe640 1 -- 172.21.15.120:0/2588375095 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc012e00 con 0x560f8af42780 2024-09-19T09:10:18.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.308+0000 7ff711dc8640 1 --2- 172.21.15.120:0/2679080218 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff6f80a3240 0x7ff6f80a5700 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff70409bc60 tx=0x7ff704066e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:18.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.308+0000 7ff6f77fe640 1 -- 172.21.15.120:0/2588375095 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6fc010c50 con 0x560f8af42780 2024-09-19T09:10:18.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.308+0000 7ff6f77fe640 1 -- 172.21.15.120:0/2588375095 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc01ebc0 con 0x560f8af42780 2024-09-19T09:10:18.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.309+0000 7ff715c04740 1 -- 172.21.15.120:0/2588375095 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f8af42780 msgr2=0x560f8af44b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:18.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.309+0000 7ff715c04740 1 --2- 172.21.15.120:0/2588375095 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f8af42780 0x560f8af44b70 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc0127c0 tx=0x7ff6fc00b7c0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.309+0000 7ff715c04740 1 -- 172.21.15.120:0/2588375095 shutdown_connections 2024-09-19T09:10:18.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.309+0000 7ff715c04740 1 --2- 172.21.15.120:0/2588375095 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f8af42780 0x560f8af44b70 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.309+0000 7ff715c04740 1 --2- 172.21.15.120:0/2588375095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f8af335d0 0x560f8af33a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.309+0000 7ff715c04740 1 --2- 172.21.15.120:0/2588375095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f8af34b40 0x560f8af33090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.309+0000 7ff715c04740 1 -- 172.21.15.120:0/2588375095 >> 172.21.15.120:0/2588375095 conn(0x560f8aefc310 msgr2=0x560f8af2fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:18.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.309+0000 7ff715c04740 1 -- 172.21.15.120:0/2588375095 shutdown_connections 2024-09-19T09:10:18.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.309+0000 7ff715c04740 1 -- 172.21.15.120:0/2588375095 wait complete. 2024-09-19T09:10:18.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.309+0000 7ff715c04740 1 -- start start 2024-09-19T09:10:18.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.309+0000 7ff715c04740 1 Processor -- start 2024-09-19T09:10:18.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.310+0000 7ff715c04740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f8af335d0 0x560f8af42130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:18.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.310+0000 7ff715c04740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f8af34b40 0x560f8af3f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:18.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.310+0000 7ff715c04740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f8af42780 0x560f8af3f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:18.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.310+0000 7ff715c04740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560f8af45480 con 0x560f8af34b40 2024-09-19T09:10:18.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.310+0000 7ff715c04740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560f8ae54060 con 0x560f8af335d0 2024-09-19T09:10:18.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.310+0000 7ff715c04740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560f8af45600 con 0x560f8af42780 2024-09-19T09:10:18.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.310+0000 7ff715c04740 10 client.0 _ll_register_callbacks cb 0x560f8aba78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:18.381 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:10:18.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.310+0000 7ff7115c7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f8af34b40 0x560f8af3f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:18.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.310+0000 7ff7115c7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f8af34b40 0x560f8af3f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55230/0 (socket says 172.21.15.120:55230) 2024-09-19T09:10:18.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.310+0000 7ff7115c7640 1 -- 172.21.15.120:0/2330970680 learned_addr learned my addr 172.21.15.120:0/2330970680 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:18.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.310+0000 7ff710dc6640 1 --2- 172.21.15.120:0/2330970680 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f8af335d0 0x560f8af42130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:18.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.310+0000 7ff710dc6640 1 --2- 172.21.15.120:0/2330970680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f8af42780 0x560f8af3f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:18.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.310+0000 7ff7115c7640 1 -- 172.21.15.120:0/2330970680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f8af42780 msgr2=0x560f8af3f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:18.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.310+0000 7ff7115c7640 1 --2- 172.21.15.120:0/2330970680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f8af42780 0x560f8af3f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.310+0000 7ff7115c7640 1 -- 172.21.15.120:0/2330970680 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f8af335d0 msgr2=0x560f8af42130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:18.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.310+0000 7ff7115c7640 1 --2- 172.21.15.120:0/2330970680 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f8af335d0 0x560f8af42130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.310+0000 7ff7115c7640 1 -- 172.21.15.120:0/2330970680 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f8ac0a3c0 con 0x560f8af34b40 2024-09-19T09:10:18.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.310+0000 7ff7115c7640 1 --2- 172.21.15.120:0/2330970680 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f8af34b40 0x560f8af3f3e0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7ff708009930 tx=0x7ff7080319b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de2358c44cc7b8c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:18.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.310+0000 7ff6f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:18.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.311+0000 7ff6f4ff9640 1 -- 172.21.15.120:0/2330970680 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff708035040 con 0x560f8af34b40 2024-09-19T09:10:18.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.311+0000 7ff6f4ff9640 1 -- 172.21.15.120:0/2330970680 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7ff708004240 con 0x560f8af34b40 2024-09-19T09:10:18.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.311+0000 7ff715c04740 1 -- 172.21.15.120:0/2330970680 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6fc010c50 con 0x560f8af34b40 2024-09-19T09:10:18.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.311+0000 7ff6f4ff9640 1 -- 172.21.15.120:0/2330970680 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff708004440 con 0x560f8af34b40 2024-09-19T09:10:18.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.311+0000 7ff715c04740 10 client.18027 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:18.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.311+0000 7ff715c04740 1 -- 172.21.15.120:0/2330970680 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560f8af50f00 con 0x560f8af34b40 2024-09-19T09:10:18.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.311+0000 7ff6f4ff9640 1 -- 172.21.15.120:0/2330970680 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6fc010c50 con 0x560f8af34b40 2024-09-19T09:10:18.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.311+0000 7ff6f4ff9640 1 -- 172.21.15.120:0/2330970680 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff708004c60 con 0x560f8af34b40 2024-09-19T09:10:18.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.311+0000 7ff715c04740 10 client.18027 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:18.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.311+0000 7ff715c04740 10 client.18027 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:10:18.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.311+0000 7ff715c04740 20 client.18027 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:18.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.311+0000 7ff715c04740 1 --2- 172.21.15.120:0/2330970680 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560f8af52410 0x560f8af54850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:18.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.311+0000 7ff715c04740 4 client.18027 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:10:18.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.311+0000 7ff715c04740 1 -- 172.21.15.120:0/2330970680 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f8aea6860 con 0x560f8af52410 2024-09-19T09:10:18.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.311+0000 7ff710dc6640 1 --2- 172.21.15.120:0/2330970680 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560f8af52410 0x560f8af54850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:18.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.312+0000 7ff6f4ff9640 10 client.18027 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:18.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.312+0000 7ff710dc6640 1 --2- 172.21.15.120:0/2330970680 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560f8af52410 0x560f8af54850 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeeeff875ddaf44c server_cookie=e5b7403facdcce9f in_seq=0 out_seq=0 2024-09-19T09:10:18.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.312+0000 7ff6f4ff9640 1 -- 172.21.15.120:0/2330970680 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f8aea6860 con 0x560f8af52410 2024-09-19T09:10:18.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.312+0000 7ff6f4ff9640 10 client.18027 handle_command_reply: tid=0 2024-09-19T09:10:18.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.312+0000 7ff715c04740 1 client.18027 shutdown 2024-09-19T09:10:18.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.313+0000 7ff715c04740 1 -- 172.21.15.120:0/2330970680 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f8af34b40 msgr2=0x560f8af3f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:18.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.313+0000 7ff715c04740 1 --2- 172.21.15.120:0/2330970680 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f8af34b40 0x560f8af3f3e0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7ff708009930 tx=0x7ff7080319b0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.313+0000 7ff715c04740 1 -- 172.21.15.120:0/2330970680 shutdown_connections 2024-09-19T09:10:18.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.313+0000 7ff715c04740 1 --2- 172.21.15.120:0/2330970680 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560f8af52410 0x560f8af54850 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.313+0000 7ff715c04740 1 --2- 172.21.15.120:0/2330970680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f8af42780 0x560f8af3f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.313+0000 7ff715c04740 1 --2- 172.21.15.120:0/2330970680 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f8af34b40 0x560f8af3f3e0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.313+0000 7ff6f4ff9640 0 client.18027 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:18.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.313+0000 7ff715c04740 1 --2- 172.21.15.120:0/2330970680 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f8af335d0 0x560f8af42130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.313+0000 7ff715c04740 1 -- 172.21.15.120:0/2330970680 >> 172.21.15.120:0/2330970680 conn(0x560f8aefc310 msgr2=0x560f8af35960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:18.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.313+0000 7ff715c04740 1 -- 172.21.15.120:0/2330970680 shutdown_connections 2024-09-19T09:10:18.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.313+0000 7ff715c04740 1 -- 172.21.15.120:0/2330970680 wait complete. 2024-09-19T09:10:18.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.313+0000 7ff715c04740 20 client.18027 trim_cache size 0 max 16384 2024-09-19T09:10:18.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.326+0000 7ff715c04740 1 Processor -- start 2024-09-19T09:10:18.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.326+0000 7ff715c04740 1 -- start start 2024-09-19T09:10:18.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.326+0000 7ff715c04740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f8af335d0 0x560f8af2fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:18.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.326+0000 7ff715c04740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f8af34b40 0x560f8af4e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:18.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.326+0000 7ff715c04740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f8af42780 0x560f8af5a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:18.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.326+0000 7ff715c04740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c1ab3d0 con 0x560f8af335d0 2024-09-19T09:10:18.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.326+0000 7ff715c04740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c1acf10 con 0x560f8af42780 2024-09-19T09:10:18.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.326+0000 7ff715c04740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c1abaa0 con 0x560f8af34b40 2024-09-19T09:10:18.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.326+0000 7ff710dc6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f8af335d0 0x560f8af2fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:18.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.326+0000 7ff710dc6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f8af335d0 0x560f8af2fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55246/0 (socket says 172.21.15.120:55246) 2024-09-19T09:10:18.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.326+0000 7ff710dc6640 1 -- 172.21.15.120:0/857572777 learned_addr learned my addr 172.21.15.120:0/857572777 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:18.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.326+0000 7ff710dc6640 1 -- 172.21.15.120:0/857572777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f8af34b40 msgr2=0x560f8af4e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:18.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.326+0000 7ff7115c7640 1 --2- 172.21.15.120:0/857572777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f8af34b40 0x560f8af4e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:18.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.326+0000 7ff710dc6640 1 --2- 172.21.15.120:0/857572777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f8af34b40 0x560f8af4e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.326+0000 7ff710dc6640 1 -- 172.21.15.120:0/857572777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f8af42780 msgr2=0x560f8af5a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:18.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.326+0000 7ff710dc6640 1 --2- 172.21.15.120:0/857572777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f8af42780 0x560f8af5a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.326+0000 7ff710dc6640 1 -- 172.21.15.120:0/857572777 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f8af2f4f0 con 0x560f8af335d0 2024-09-19T09:10:18.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.326+0000 7ff7115c7640 1 --2- 172.21.15.120:0/857572777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f8af34b40 0x560f8af4e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:10:18.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.326+0000 7ff710dc6640 1 --2- 172.21.15.120:0/857572777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f8af335d0 0x560f8af2fe10 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc01e8b0 tx=0x7ff6fc014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e71616951a3e1f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:18.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.327+0000 7ff6f6ffd640 1 -- 172.21.15.120:0/857572777 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc024040 con 0x560f8af335d0 2024-09-19T09:10:18.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.327+0000 7ff6f6ffd640 1 -- 172.21.15.120:0/857572777 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6fc005510 con 0x560f8af335d0 2024-09-19T09:10:18.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.327+0000 7ff6f6ffd640 1 -- 172.21.15.120:0/857572777 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc01ed80 con 0x560f8af335d0 2024-09-19T09:10:18.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.327+0000 7ff715c04740 1 -- 172.21.15.120:0/857572777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f8af335d0 msgr2=0x560f8af2fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:18.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.327+0000 7ff715c04740 1 --2- 172.21.15.120:0/857572777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f8af335d0 0x560f8af2fe10 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc01e8b0 tx=0x7ff6fc014040 comp rx=0 tx=0).stop 2024-09-19T09:10:18.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.327+0000 7ff715c04740 1 -- 172.21.15.120:0/857572777 shutdown_connections 2024-09-19T09:10:18.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.327+0000 7ff715c04740 1 --2- 172.21.15.120:0/857572777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f8af42780 0x560f8af5a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.327+0000 7ff715c04740 1 --2- 172.21.15.120:0/857572777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f8af34b40 0x560f8af4e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.327+0000 7ff715c04740 1 --2- 172.21.15.120:0/857572777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f8af335d0 0x560f8af2fe10 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.327+0000 7ff715c04740 1 -- 172.21.15.120:0/857572777 >> 172.21.15.120:0/857572777 conn(0x560f8aefc310 msgr2=0x560f8af2f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:18.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.327+0000 7ff715c04740 1 -- 172.21.15.120:0/857572777 shutdown_connections 2024-09-19T09:10:18.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.327+0000 7ff715c04740 1 -- 172.21.15.120:0/857572777 wait complete. 2024-09-19T09:10:18.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.328+0000 7ff715c04740 1 -- start start 2024-09-19T09:10:18.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.328+0000 7ff715c04740 1 Processor -- start 2024-09-19T09:10:18.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.328+0000 7ff715c04740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f8af335d0 0x560f8af46df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:18.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.328+0000 7ff715c04740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f8af34b40 0x560f8af47330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:18.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.328+0000 7ff715c04740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f8af42780 0x560f8afbc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:18.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.328+0000 7ff715c04740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c1ac840 con 0x560f8af335d0 2024-09-19T09:10:18.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.328+0000 7ff715c04740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c1ac170 con 0x560f8af34b40 2024-09-19T09:10:18.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.328+0000 7ff715c04740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c1a9390 con 0x560f8af42780 2024-09-19T09:10:18.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.328+0000 7ff715c04740 10 client.0 _ll_register_callbacks cb 0x7ff7080314d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:18.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.328+0000 7ff710dc6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f8af335d0 0x560f8af46df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:18.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.328+0000 7ff710dc6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f8af335d0 0x560f8af46df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55252/0 (socket says 172.21.15.120:55252) 2024-09-19T09:10:18.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.328+0000 7ff710dc6640 1 -- 172.21.15.120:0/1964348731 learned_addr learned my addr 172.21.15.120:0/1964348731 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:18.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.328+0000 7ff7115c7640 1 --2- 172.21.15.120:0/1964348731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f8af34b40 0x560f8af47330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:18.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.328+0000 7ff710dc6640 1 --2- 172.21.15.120:0/1964348731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f8af42780 0x560f8afbc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:18.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.329+0000 7ff710dc6640 1 -- 172.21.15.120:0/1964348731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f8af42780 msgr2=0x560f8afbc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:18.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.329+0000 7ff710dc6640 1 --2- 172.21.15.120:0/1964348731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f8af42780 0x560f8afbc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.329+0000 7ff710dc6640 1 -- 172.21.15.120:0/1964348731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f8af34b40 msgr2=0x560f8af47330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:18.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.329+0000 7ff710dc6640 1 --2- 172.21.15.120:0/1964348731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f8af34b40 0x560f8af47330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.329+0000 7ff710dc6640 1 -- 172.21.15.120:0/1964348731 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f8afbc6b0 con 0x560f8af335d0 2024-09-19T09:10:18.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.329+0000 7ff710dc6640 1 --2- 172.21.15.120:0/1964348731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f8af335d0 0x560f8af46df0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc00b3d0 tx=0x7ff6fc012760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ae910a7ab2179a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:18.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.329+0000 7ff6f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:18.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.329+0000 7ff6f4ff9640 1 -- 172.21.15.120:0/1964348731 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc03a020 con 0x560f8af335d0 2024-09-19T09:10:18.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.329+0000 7ff6f4ff9640 1 -- 172.21.15.120:0/1964348731 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7ff6fc0050d0 con 0x560f8af335d0 2024-09-19T09:10:18.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.329+0000 7ff715c04740 1 -- 172.21.15.120:0/1964348731 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560f8aea0d70 con 0x560f8af335d0 2024-09-19T09:10:18.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.329+0000 7ff6f4ff9640 1 -- 172.21.15.120:0/1964348731 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff6fc024040 con 0x560f8af335d0 2024-09-19T09:10:18.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.329+0000 7ff715c04740 10 client.18033 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:18.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.329+0000 7ff715c04740 1 -- 172.21.15.120:0/1964348731 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560f8afbd8e0 con 0x560f8af335d0 2024-09-19T09:10:18.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.329+0000 7ff6f4ff9640 1 -- 172.21.15.120:0/1964348731 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560f8aea0d70 con 0x560f8af335d0 2024-09-19T09:10:18.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.329+0000 7ff6f4ff9640 1 -- 172.21.15.120:0/1964348731 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff6fc010c40 con 0x560f8af335d0 2024-09-19T09:10:18.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.329+0000 7ff715c04740 10 client.18033 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:18.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.329+0000 7ff715c04740 10 client.18033 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:10:18.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.329+0000 7ff715c04740 20 client.18033 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:18.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.329+0000 7ff715c04740 1 --2- 172.21.15.120:0/1964348731 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560f8afbdd10 0x560f8afc0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:18.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.329+0000 7ff715c04740 4 client.18033 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:10:18.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.329+0000 7ff715c04740 1 -- 172.21.15.120:0/1964348731 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f8ae1d170 con 0x560f8afbdd10 2024-09-19T09:10:18.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.330+0000 7ff7115c7640 1 --2- 172.21.15.120:0/1964348731 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560f8afbdd10 0x560f8afc0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:18.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.330+0000 7ff6f4ff9640 10 client.18033 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:18.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.330+0000 7ff7115c7640 1 --2- 172.21.15.120:0/1964348731 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560f8afbdd10 0x560f8afc0100 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccc41fea8d099bd0 server_cookie=d669e8dcacc9d34f in_seq=0 out_seq=0 2024-09-19T09:10:18.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.330+0000 7ff6f4ff9640 1 -- 172.21.15.120:0/1964348731 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f8ae1d170 con 0x560f8afbdd10 2024-09-19T09:10:18.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.330+0000 7ff6f4ff9640 10 client.18033 handle_command_reply: tid=0 2024-09-19T09:10:18.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.330+0000 7ff715c04740 1 client.18033 shutdown 2024-09-19T09:10:18.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.330+0000 7ff715c04740 1 -- 172.21.15.120:0/1964348731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f8af335d0 msgr2=0x560f8af46df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:18.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.330+0000 7ff715c04740 1 --2- 172.21.15.120:0/1964348731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f8af335d0 0x560f8af46df0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc00b3d0 tx=0x7ff6fc012760 comp rx=0 tx=0).stop 2024-09-19T09:10:18.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.330+0000 7ff715c04740 1 -- 172.21.15.120:0/1964348731 shutdown_connections 2024-09-19T09:10:18.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.330+0000 7ff715c04740 1 --2- 172.21.15.120:0/1964348731 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560f8afbdd10 0x560f8afc0100 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.330+0000 7ff715c04740 1 --2- 172.21.15.120:0/1964348731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f8af42780 0x560f8afbc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.330+0000 7ff6f4ff9640 0 client.18033 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:18.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.330+0000 7ff715c04740 1 --2- 172.21.15.120:0/1964348731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f8af34b40 0x560f8af47330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.330+0000 7ff715c04740 1 --2- 172.21.15.120:0/1964348731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f8af335d0 0x560f8af46df0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.330+0000 7ff715c04740 1 -- 172.21.15.120:0/1964348731 >> 172.21.15.120:0/1964348731 conn(0x560f8aefc310 msgr2=0x560f8af40c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:18.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.331+0000 7ff715c04740 1 -- 172.21.15.120:0/1964348731 shutdown_connections 2024-09-19T09:10:18.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.331+0000 7ff715c04740 1 -- 172.21.15.120:0/1964348731 wait complete. 2024-09-19T09:10:18.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.331+0000 7ff715c04740 20 client.18033 trim_cache size 0 max 16384 2024-09-19T09:10:18.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.331+0000 7ff71382b640 1 -- 172.21.15.120:0/2679080218 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff6f80a3240 msgr2=0x7ff6f80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:18.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.331+0000 7ff71382b640 1 --2- 172.21.15.120:0/2679080218 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff6f80a3240 0x7ff6f80a5700 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff70409bc60 tx=0x7ff704066e40 comp rx=0 tx=0).stop 2024-09-19T09:10:18.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.331+0000 7ff71382b640 1 -- 172.21.15.120:0/2679080218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff70c106a80 msgr2=0x7ff70c1a4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:18.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.331+0000 7ff71382b640 1 --2- 172.21.15.120:0/2679080218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff70c106a80 0x7ff70c1a4370 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7ff70400cad0 tx=0x7ff70400cf90 comp rx=0 tx=0).stop 2024-09-19T09:10:18.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.331+0000 7ff71382b640 1 -- 172.21.15.120:0/2679080218 shutdown_connections 2024-09-19T09:10:18.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.331+0000 7ff71382b640 1 --2- 172.21.15.120:0/2679080218 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff6f80a3240 0x7ff6f80a5700 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.331+0000 7ff71382b640 1 --2- 172.21.15.120:0/2679080218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff70c106a80 0x7ff70c1a4370 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.331+0000 7ff71382b640 1 --2- 172.21.15.120:0/2679080218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff70c101420 0x7ff70c199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.331+0000 7ff71382b640 1 --2- 172.21.15.120:0/2679080218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff70c100a80 0x7ff70c1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:18.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.331+0000 7ff71382b640 1 -- 172.21.15.120:0/2679080218 >> 172.21.15.120:0/2679080218 conn(0x7ff70c0fc7a0 msgr2=0x7ff70c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:18.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.331+0000 7ff71382b640 1 -- 172.21.15.120:0/2679080218 shutdown_connections 2024-09-19T09:10:18.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:18.331+0000 7ff71382b640 1 -- 172.21.15.120:0/2679080218 wait complete. 2024-09-19T09:10:19.057 INFO:tasks.workunit.client.0.smithi120.stdout: 1 698442 10.05 MB/sec execute 254 sec latency 10865.369 ms 2024-09-19T09:10:19.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:18 smithi120 ceph-mon[31150]: pgmap v565: 97 pgs: 97 active+clean; 43 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 314 op/s 2024-09-19T09:10:19.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:18 smithi196 ceph-mon[35150]: pgmap v565: 97 pgs: 97 active+clean; 43 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 314 op/s 2024-09-19T09:10:19.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:18 smithi153 ceph-mon[35071]: pgmap v565: 97 pgs: 97 active+clean; 43 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 314 op/s 2024-09-19T09:10:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.240+0000 7f23f9a82640 1 -- 172.21.15.120:0/3705648198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23f4100a40 msgr2=0x7f23f4100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.240+0000 7f23f9a82640 1 --2- 172.21.15.120:0/3705648198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23f4100a40 0x7f23f4100e40 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009900 tx=0x7f23e402a410 comp rx=0 tx=0).stop 2024-09-19T09:10:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.240+0000 7f23f9a82640 1 -- 172.21.15.120:0/3705648198 shutdown_connections 2024-09-19T09:10:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.240+0000 7f23f9a82640 1 --2- 172.21.15.120:0/3705648198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23f41025f0 0x7f23f4108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.240+0000 7f23f9a82640 1 --2- 172.21.15.120:0/3705648198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23f4101c30 0x7f23f41020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.240+0000 7f23f9a82640 1 --2- 172.21.15.120:0/3705648198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23f4100a40 0x7f23f4100e40 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.240+0000 7f23f9a82640 1 -- 172.21.15.120:0/3705648198 >> 172.21.15.120:0/3705648198 conn(0x7f23f40fc1e0 msgr2=0x7f23f40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.240+0000 7f23f9a82640 1 -- 172.21.15.120:0/3705648198 shutdown_connections 2024-09-19T09:10:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.240+0000 7f23f9a82640 1 -- 172.21.15.120:0/3705648198 wait complete. 2024-09-19T09:10:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.241+0000 7f23f9a82640 1 Processor -- start 2024-09-19T09:10:19.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.241+0000 7f23f9a82640 1 -- start start 2024-09-19T09:10:19.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.241+0000 7f23f9a82640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23f4100a40 0x7f23f4197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:19.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.241+0000 7f23f9a82640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23f4101c30 0x7f23f4198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:19.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.241+0000 7f23f9a82640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23f41025f0 0x7f23f419f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:19.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.241+0000 7f23f9a82640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f23f410d810 con 0x7f23f4101c30 2024-09-19T09:10:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.241+0000 7f23f9a82640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f23f410d690 con 0x7f23f4100a40 2024-09-19T09:10:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.241+0000 7f23f9a82640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f23f410d990 con 0x7f23f41025f0 2024-09-19T09:10:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.241+0000 7f23f2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23f4101c30 0x7f23f4198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.241+0000 7f23f2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23f4101c30 0x7f23f4198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55268/0 (socket says 172.21.15.120:55268) 2024-09-19T09:10:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.241+0000 7f23f2ffd640 1 -- 172.21.15.120:0/879187204 learned_addr learned my addr 172.21.15.120:0/879187204 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.242+0000 7f23f3fff640 1 --2- 172.21.15.120:0/879187204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23f41025f0 0x7f23f419f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.241+0000 7f23f37fe640 1 --2- 172.21.15.120:0/879187204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23f4100a40 0x7f23f4197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.242+0000 7f23f2ffd640 1 -- 172.21.15.120:0/879187204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23f41025f0 msgr2=0x7f23f419f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.242+0000 7f23f2ffd640 1 --2- 172.21.15.120:0/879187204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23f41025f0 0x7f23f419f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.242+0000 7f23f2ffd640 1 -- 172.21.15.120:0/879187204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23f4100a40 msgr2=0x7f23f4197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.242+0000 7f23f2ffd640 1 --2- 172.21.15.120:0/879187204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23f4100a40 0x7f23f4197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.242+0000 7f23f2ffd640 1 -- 172.21.15.120:0/879187204 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f419fa10 con 0x7f23f4101c30 2024-09-19T09:10:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.242+0000 7f23f3fff640 1 --2- 172.21.15.120:0/879187204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23f41025f0 0x7f23f419f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:10:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.242+0000 7f23f2ffd640 1 --2- 172.21.15.120:0/879187204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23f4101c30 0x7f23f4198330 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f23e800e9f0 tx=0x7f23e800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.242+0000 7f23f0ff9640 1 -- 172.21.15.120:0/879187204 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e800cd80 con 0x7f23f4101c30 2024-09-19T09:10:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.242+0000 7f23f0ff9640 1 -- 172.21.15.120:0/879187204 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23e8005500 con 0x7f23f4101c30 2024-09-19T09:10:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.242+0000 7f23f9a82640 1 -- 172.21.15.120:0/879187204 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f419fd00 con 0x7f23f4101c30 2024-09-19T09:10:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.242+0000 7f23f0ff9640 1 -- 172.21.15.120:0/879187204 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e8010690 con 0x7f23f4101c30 2024-09-19T09:10:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.242+0000 7f23f9a82640 1 -- 172.21.15.120:0/879187204 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f41a01a0 con 0x7f23f4101c30 2024-09-19T09:10:19.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.245+0000 7f23f0ff9640 1 -- 172.21.15.120:0/879187204 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f23e80046c0 con 0x7f23f4101c30 2024-09-19T09:10:19.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.245+0000 7f23f0ff9640 1 --2- 172.21.15.120:0/879187204 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f23dc0a32f0 0x7f23dc0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:19.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.246+0000 7f23f37fe640 1 --2- 172.21.15.120:0/879187204 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f23dc0a32f0 0x7f23dc0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:19.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.246+0000 7f23f0ff9640 1 -- 172.21.15.120:0/879187204 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f23e80cfd70 con 0x7f23f4101c30 2024-09-19T09:10:19.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.246+0000 7f23fc820740 1 Processor -- start 2024-09-19T09:10:19.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.246+0000 7f23f37fe640 1 --2- 172.21.15.120:0/879187204 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f23dc0a32f0 0x7f23dc0a57b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f23e4005ab0 tx=0x7f23e4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.246+0000 7f23fc820740 1 -- start start 2024-09-19T09:10:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.246+0000 7f23fc820740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc393c69a0 0x55dc393c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.246+0000 7f23fc820740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc393c72e0 0x55dc393cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.246+0000 7f23fc820740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc393cc370 0x55dc393ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.246+0000 7f23fc820740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dc3932a2f0 con 0x55dc393c72e0 2024-09-19T09:10:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.246+0000 7f23fc820740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55dc392d2270 con 0x55dc393cc370 2024-09-19T09:10:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.246+0000 7f23fc820740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dc393262b0 con 0x55dc393c69a0 2024-09-19T09:10:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.246+0000 7f23f2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc393c72e0 0x55dc393cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.247+0000 7f23f2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc393c72e0 0x55dc393cbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55274/0 (socket says 172.21.15.120:55274) 2024-09-19T09:10:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.247+0000 7f23f2ffd640 1 -- 172.21.15.120:0/3227366401 learned_addr learned my addr 172.21.15.120:0/3227366401 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.247+0000 7f23f37fe640 1 --2- 172.21.15.120:0/3227366401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc393c69a0 0x55dc393c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.247+0000 7f23f3fff640 1 --2- 172.21.15.120:0/3227366401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc393cc370 0x55dc393ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:19.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.247+0000 7f23f2ffd640 1 -- 172.21.15.120:0/3227366401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc393c69a0 msgr2=0x55dc393c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:19.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.247+0000 7f23f2ffd640 1 --2- 172.21.15.120:0/3227366401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc393c69a0 0x55dc393c6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.247+0000 7f23f2ffd640 1 -- 172.21.15.120:0/3227366401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc393cc370 msgr2=0x55dc393ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:19.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.247+0000 7f23f2ffd640 1 --2- 172.21.15.120:0/3227366401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc393cc370 0x55dc393ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.247+0000 7f23f2ffd640 1 -- 172.21.15.120:0/3227366401 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc392d1340 con 0x55dc393c72e0 2024-09-19T09:10:19.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.247+0000 7f23f3fff640 1 --2- 172.21.15.120:0/3227366401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc393cc370 0x55dc393ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:10:19.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.247+0000 7f23f2ffd640 1 --2- 172.21.15.120:0/3227366401 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc393c72e0 0x55dc393cbda0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f23e809c2e0 tx=0x7f23e809bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe1a0e8ec9189c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:19.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.247+0000 7f23d9ffb640 1 -- 172.21.15.120:0/3227366401 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e80b5070 con 0x55dc393c72e0 2024-09-19T09:10:19.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.247+0000 7f23d9ffb640 1 -- 172.21.15.120:0/3227366401 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23e80a0700 con 0x55dc393c72e0 2024-09-19T09:10:19.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.247+0000 7f23d9ffb640 1 -- 172.21.15.120:0/3227366401 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e80a4b30 con 0x55dc393c72e0 2024-09-19T09:10:19.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.248+0000 7f23fc820740 1 -- 172.21.15.120:0/3227366401 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc393c72e0 msgr2=0x55dc393cbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:19.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.248+0000 7f23fc820740 1 --2- 172.21.15.120:0/3227366401 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc393c72e0 0x55dc393cbda0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f23e809c2e0 tx=0x7f23e809bf50 comp rx=0 tx=0).stop 2024-09-19T09:10:19.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.248+0000 7f23fc820740 1 -- 172.21.15.120:0/3227366401 shutdown_connections 2024-09-19T09:10:19.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.248+0000 7f23fc820740 1 --2- 172.21.15.120:0/3227366401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc393cc370 0x55dc393ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.248+0000 7f23fc820740 1 --2- 172.21.15.120:0/3227366401 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc393c72e0 0x55dc393cbda0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.248+0000 7f23fc820740 1 --2- 172.21.15.120:0/3227366401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc393c69a0 0x55dc393c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.248+0000 7f23fc820740 1 -- 172.21.15.120:0/3227366401 >> 172.21.15.120:0/3227366401 conn(0x55dc393c2010 msgr2=0x55dc393c2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:19.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.248+0000 7f23fc820740 1 -- 172.21.15.120:0/3227366401 shutdown_connections 2024-09-19T09:10:19.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.248+0000 7f23fc820740 1 -- 172.21.15.120:0/3227366401 wait complete. 2024-09-19T09:10:19.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.249+0000 7f23fc820740 1 -- start start 2024-09-19T09:10:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.249+0000 7f23fc820740 1 Processor -- start 2024-09-19T09:10:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.249+0000 7f23fc820740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc393c69a0 0x55dc393d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.249+0000 7f23fc820740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc393c72e0 0x55dc393d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.249+0000 7f23fc820740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc393cc370 0x55dc393d81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.249+0000 7f23fc820740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dc393cf340 con 0x55dc393c72e0 2024-09-19T09:10:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.249+0000 7f23fc820740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55dc392b3290 con 0x55dc393c69a0 2024-09-19T09:10:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.249+0000 7f23fc820740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dc393cf4c0 con 0x55dc393cc370 2024-09-19T09:10:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.249+0000 7f23fc820740 10 client.0 _ll_register_callbacks cb 0x55dc392e4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.249+0000 7f23f2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc393c72e0 0x55dc393d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.249+0000 7f23f2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc393c72e0 0x55dc393d1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55278/0 (socket says 172.21.15.120:55278) 2024-09-19T09:10:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.249+0000 7f23f2ffd640 1 -- 172.21.15.120:0/4155273441 learned_addr learned my addr 172.21.15.120:0/4155273441 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.249+0000 7f23f37fe640 1 --2- 172.21.15.120:0/4155273441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc393c69a0 0x55dc393d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.249+0000 7f23f3fff640 1 --2- 172.21.15.120:0/4155273441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc393cc370 0x55dc393d81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.250+0000 7f23f2ffd640 1 -- 172.21.15.120:0/4155273441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc393cc370 msgr2=0x55dc393d81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.250+0000 7f23f2ffd640 1 --2- 172.21.15.120:0/4155273441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc393cc370 0x55dc393d81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.250+0000 7f23f2ffd640 1 -- 172.21.15.120:0/4155273441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc393c69a0 msgr2=0x55dc393d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.250+0000 7f23f2ffd640 1 --2- 172.21.15.120:0/4155273441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc393c69a0 0x55dc393d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.250+0000 7f23f2ffd640 1 -- 172.21.15.120:0/4155273441 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc3903d210 con 0x55dc393c72e0 2024-09-19T09:10:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.250+0000 7f23f2ffd640 1 --2- 172.21.15.120:0/4155273441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc393c72e0 0x55dc393d1160 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f23e8004a80 tx=0x7f23e809c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed377523f323438 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.250+0000 7f23d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.250+0000 7f23d2ffd640 1 -- 172.21.15.120:0/4155273441 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e80b9020 con 0x55dc393c72e0 2024-09-19T09:10:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.250+0000 7f23d2ffd640 1 -- 172.21.15.120:0/4155273441 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f23e80a4b30 con 0x55dc393c72e0 2024-09-19T09:10:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.250+0000 7f23fc820740 1 -- 172.21.15.120:0/4155273441 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23e80a0700 con 0x55dc393c72e0 2024-09-19T09:10:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.250+0000 7f23d2ffd640 1 -- 172.21.15.120:0/4155273441 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f23e80b5040 con 0x55dc393c72e0 2024-09-19T09:10:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.250+0000 7f23fc820740 10 client.18045 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.250+0000 7f23fc820740 1 -- 172.21.15.120:0/4155273441 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dc39109620 con 0x55dc393c72e0 2024-09-19T09:10:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.250+0000 7f23d2ffd640 1 -- 172.21.15.120:0/4155273441 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23e80a0700 con 0x55dc393c72e0 2024-09-19T09:10:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.251+0000 7f23d2ffd640 1 -- 172.21.15.120:0/4155273441 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f23e80a0200 con 0x55dc393c72e0 2024-09-19T09:10:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.251+0000 7f23fc820740 10 client.18045 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.251+0000 7f23fc820740 10 client.18045 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:10:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.251+0000 7f23fc820740 20 client.18045 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.251+0000 7f23fc820740 1 --2- 172.21.15.120:0/4155273441 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dc393dc4a0 0x55dc393de890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.251+0000 7f23fc820740 4 client.18045 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:10:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.251+0000 7f23fc820740 1 -- 172.21.15.120:0/4155273441 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc39337a20 con 0x55dc393dc4a0 2024-09-19T09:10:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.251+0000 7f23f37fe640 1 --2- 172.21.15.120:0/4155273441 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dc393dc4a0 0x55dc393de890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.251+0000 7f23d2ffd640 10 client.18045 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.251+0000 7f23f37fe640 1 --2- 172.21.15.120:0/4155273441 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dc393dc4a0 0x55dc393de890 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dfc926269ec1efa server_cookie=3cf5b31359ab4b9 in_seq=0 out_seq=0 2024-09-19T09:10:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.252+0000 7f23d2ffd640 1 -- 172.21.15.120:0/4155273441 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dc39337a20 con 0x55dc393dc4a0 2024-09-19T09:10:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.252+0000 7f23d2ffd640 10 client.18045 handle_command_reply: tid=0 2024-09-19T09:10:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.252+0000 7f23fc820740 1 client.18045 shutdown 2024-09-19T09:10:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.252+0000 7f23fc820740 1 -- 172.21.15.120:0/4155273441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc393c72e0 msgr2=0x55dc393d1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.252+0000 7f23fc820740 1 --2- 172.21.15.120:0/4155273441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc393c72e0 0x55dc393d1160 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f23e8004a80 tx=0x7f23e809c480 comp rx=0 tx=0).stop 2024-09-19T09:10:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.253+0000 7f23fc820740 1 -- 172.21.15.120:0/4155273441 shutdown_connections 2024-09-19T09:10:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.253+0000 7f23fc820740 1 --2- 172.21.15.120:0/4155273441 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dc393dc4a0 0x55dc393de890 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.253+0000 7f23d2ffd640 0 client.18045 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.253+0000 7f23fc820740 1 --2- 172.21.15.120:0/4155273441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc393cc370 0x55dc393d81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.253+0000 7f23fc820740 1 --2- 172.21.15.120:0/4155273441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc393c72e0 0x55dc393d1160 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.253+0000 7f23fc820740 1 --2- 172.21.15.120:0/4155273441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc393c69a0 0x55dc393d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.253+0000 7f23fc820740 1 -- 172.21.15.120:0/4155273441 >> 172.21.15.120:0/4155273441 conn(0x55dc393c2010 msgr2=0x55dc393c3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.253+0000 7f23fc820740 1 -- 172.21.15.120:0/4155273441 shutdown_connections 2024-09-19T09:10:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.253+0000 7f23fc820740 1 -- 172.21.15.120:0/4155273441 wait complete. 2024-09-19T09:10:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.253+0000 7f23fc820740 20 client.18045 trim_cache size 0 max 16384 2024-09-19T09:10:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.273+0000 7f23fc820740 1 Processor -- start 2024-09-19T09:10:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.273+0000 7f23fc820740 1 -- start start 2024-09-19T09:10:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.273+0000 7f23fc820740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc393c69a0 0x55dc393c2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.273+0000 7f23fc820740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc393c72e0 0x55dc393c3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.273+0000 7f23fc820740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc393cc370 0x55dc393c3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.273+0000 7f23fc820740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f23f41a9080 con 0x55dc393cc370 2024-09-19T09:10:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.273+0000 7f23fc820740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f23f41aabc0 con 0x55dc393c72e0 2024-09-19T09:10:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.273+0000 7f23fc820740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f23f41a9750 con 0x55dc393c69a0 2024-09-19T09:10:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.273+0000 7f23f3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc393cc370 0x55dc393c3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.273+0000 7f23f3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc393cc370 0x55dc393c3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55288/0 (socket says 172.21.15.120:55288) 2024-09-19T09:10:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.273+0000 7f23f3fff640 1 -- 172.21.15.120:0/1849390499 learned_addr learned my addr 172.21.15.120:0/1849390499 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.274+0000 7f23f37fe640 1 --2- 172.21.15.120:0/1849390499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc393c69a0 0x55dc393c2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.274+0000 7f23f2ffd640 1 --2- 172.21.15.120:0/1849390499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc393c72e0 0x55dc393c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.274+0000 7f23f3fff640 1 -- 172.21.15.120:0/1849390499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc393c69a0 msgr2=0x55dc393c2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.274+0000 7f23f3fff640 1 --2- 172.21.15.120:0/1849390499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc393c69a0 0x55dc393c2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.274+0000 7f23f3fff640 1 -- 172.21.15.120:0/1849390499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc393c72e0 msgr2=0x55dc393c3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.274+0000 7f23f3fff640 1 --2- 172.21.15.120:0/1849390499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc393c72e0 0x55dc393c3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.274+0000 7f23f3fff640 1 -- 172.21.15.120:0/1849390499 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc393ceb70 con 0x55dc393cc370 2024-09-19T09:10:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.274+0000 7f23f3fff640 1 --2- 172.21.15.120:0/1849390499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc393cc370 0x55dc393c3940 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f23ec012900 tx=0x7f23ec009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=917c33c09c64d7eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.274+0000 7f23d97fa640 1 -- 172.21.15.120:0/1849390499 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec022070 con 0x55dc393cc370 2024-09-19T09:10:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.274+0000 7f23d97fa640 1 -- 172.21.15.120:0/1849390499 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23ec003e50 con 0x55dc393cc370 2024-09-19T09:10:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.274+0000 7f23d97fa640 1 -- 172.21.15.120:0/1849390499 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec004410 con 0x55dc393cc370 2024-09-19T09:10:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.275+0000 7f23fc820740 1 -- 172.21.15.120:0/1849390499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc393cc370 msgr2=0x55dc393c3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.275+0000 7f23fc820740 1 --2- 172.21.15.120:0/1849390499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc393cc370 0x55dc393c3940 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f23ec012900 tx=0x7f23ec009500 comp rx=0 tx=0).stop 2024-09-19T09:10:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.275+0000 7f23fc820740 1 -- 172.21.15.120:0/1849390499 shutdown_connections 2024-09-19T09:10:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.275+0000 7f23fc820740 1 --2- 172.21.15.120:0/1849390499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc393cc370 0x55dc393c3940 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.275+0000 7f23fc820740 1 --2- 172.21.15.120:0/1849390499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc393c72e0 0x55dc393c3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.275+0000 7f23fc820740 1 --2- 172.21.15.120:0/1849390499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc393c69a0 0x55dc393c2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.275+0000 7f23fc820740 1 -- 172.21.15.120:0/1849390499 >> 172.21.15.120:0/1849390499 conn(0x55dc393c2010 msgr2=0x55dc393d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.275+0000 7f23fc820740 1 -- 172.21.15.120:0/1849390499 shutdown_connections 2024-09-19T09:10:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.275+0000 7f23fc820740 1 -- 172.21.15.120:0/1849390499 wait complete. 2024-09-19T09:10:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.275+0000 7f23fc820740 1 -- start start 2024-09-19T09:10:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.276+0000 7f23fc820740 1 Processor -- start 2024-09-19T09:10:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.276+0000 7f23fc820740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc393c69a0 0x55dc39442440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.276+0000 7f23fc820740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc393c72e0 0x55dc39442980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.276+0000 7f23fc820740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc393cc370 0x55dc394499c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.276+0000 7f23fc820740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f23f41aa4f0 con 0x55dc393c69a0 2024-09-19T09:10:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.276+0000 7f23fc820740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f23f41a9e20 con 0x55dc393c72e0 2024-09-19T09:10:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.276+0000 7f23fc820740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f23f41a7040 con 0x55dc393cc370 2024-09-19T09:10:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.276+0000 7f23fc820740 10 client.0 _ll_register_callbacks cb 0x7f23e80b3070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.276+0000 7f23f37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc393c69a0 0x55dc39442440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.276+0000 7f23f37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc393c69a0 0x55dc39442440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55302/0 (socket says 172.21.15.120:55302) 2024-09-19T09:10:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.276+0000 7f23f37fe640 1 -- 172.21.15.120:0/1829526532 learned_addr learned my addr 172.21.15.120:0/1829526532 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.276+0000 7f23f2ffd640 1 --2- 172.21.15.120:0/1829526532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc393c72e0 0x55dc39442980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.276+0000 7f23f3fff640 1 --2- 172.21.15.120:0/1829526532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc393cc370 0x55dc394499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.276+0000 7f23f37fe640 1 -- 172.21.15.120:0/1829526532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc393cc370 msgr2=0x55dc394499c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.276+0000 7f23f37fe640 1 --2- 172.21.15.120:0/1829526532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc393cc370 0x55dc394499c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.276+0000 7f23f37fe640 1 -- 172.21.15.120:0/1829526532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc393c72e0 msgr2=0x55dc39442980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.276+0000 7f23f37fe640 1 --2- 172.21.15.120:0/1829526532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc393c72e0 0x55dc39442980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.276+0000 7f23f37fe640 1 -- 172.21.15.120:0/1829526532 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc39449fd0 con 0x55dc393c69a0 2024-09-19T09:10:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.276+0000 7f23f37fe640 1 --2- 172.21.15.120:0/1829526532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc393c69a0 0x55dc39442440 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f23e4031ba0 tx=0x7f23e4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70388657dd1798cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.276+0000 7f23d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.276+0000 7f23d2ffd640 1 -- 172.21.15.120:0/1829526532 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4035040 con 0x55dc393c69a0 2024-09-19T09:10:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.277+0000 7f23d2ffd640 1 -- 172.21.15.120:0/1829526532 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f23e4005250 con 0x55dc393c69a0 2024-09-19T09:10:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.277+0000 7f23fc820740 1 -- 172.21.15.120:0/1829526532 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23ec003e50 con 0x55dc393c69a0 2024-09-19T09:10:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.277+0000 7f23d2ffd640 1 -- 172.21.15.120:0/1829526532 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f23e40044c0 con 0x55dc393c69a0 2024-09-19T09:10:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.277+0000 7f23fc820740 10 client.18051 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.277+0000 7f23fc820740 1 -- 172.21.15.120:0/1829526532 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dc3944b350 con 0x55dc393c69a0 2024-09-19T09:10:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.277+0000 7f23d2ffd640 1 -- 172.21.15.120:0/1829526532 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23ec003e50 con 0x55dc393c69a0 2024-09-19T09:10:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.277+0000 7f23d2ffd640 1 -- 172.21.15.120:0/1829526532 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f23e4052070 con 0x55dc393c69a0 2024-09-19T09:10:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.277+0000 7f23fc820740 10 client.18051 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.277+0000 7f23fc820740 10 client.18051 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:10:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.277+0000 7f23fc820740 20 client.18051 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.277+0000 7f23fc820740 1 --2- 172.21.15.120:0/1829526532 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dc3944bd30 0x55dc3944e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.277+0000 7f23fc820740 4 client.18051 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:10:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.277+0000 7f23fc820740 1 -- 172.21.15.120:0/1829526532 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dc39331d70 con 0x55dc3944bd30 2024-09-19T09:10:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.277+0000 7f23f2ffd640 1 --2- 172.21.15.120:0/1829526532 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dc3944bd30 0x55dc3944e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.278+0000 7f23f2ffd640 1 --2- 172.21.15.120:0/1829526532 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dc3944bd30 0x55dc3944e150 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c18d8ccfe1fc3981 server_cookie=8f4d0aef58e01644 in_seq=0 out_seq=0 2024-09-19T09:10:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.278+0000 7f23d2ffd640 10 client.18051 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:19.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.357+0000 7f23d2ffd640 1 -- 172.21.15.120:0/1829526532 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dc39331d70 con 0x55dc3944bd30 2024-09-19T09:10:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.357+0000 7f23d2ffd640 10 client.18051 handle_command_reply: tid=0 2024-09-19T09:10:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.357+0000 7f23fc820740 1 client.18051 shutdown 2024-09-19T09:10:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.357+0000 7f23fc820740 1 -- 172.21.15.120:0/1829526532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc393c69a0 msgr2=0x55dc39442440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.357+0000 7f23fc820740 1 --2- 172.21.15.120:0/1829526532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc393c69a0 0x55dc39442440 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f23e4031ba0 tx=0x7f23e4009260 comp rx=0 tx=0).stop 2024-09-19T09:10:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.357+0000 7f23fc820740 1 -- 172.21.15.120:0/1829526532 shutdown_connections 2024-09-19T09:10:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.357+0000 7f23fc820740 1 --2- 172.21.15.120:0/1829526532 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dc3944bd30 0x55dc3944e150 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.357+0000 7f23fc820740 1 --2- 172.21.15.120:0/1829526532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc393cc370 0x55dc394499c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.357+0000 7f23fc820740 1 --2- 172.21.15.120:0/1829526532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc393c72e0 0x55dc39442980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.357+0000 7f23d2ffd640 0 client.18051 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.357+0000 7f23fc820740 1 --2- 172.21.15.120:0/1829526532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc393c69a0 0x55dc39442440 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.357+0000 7f23fc820740 1 -- 172.21.15.120:0/1829526532 >> 172.21.15.120:0/1829526532 conn(0x55dc393c2010 msgr2=0x55dc393c09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.357+0000 7f23fc820740 1 -- 172.21.15.120:0/1829526532 shutdown_connections 2024-09-19T09:10:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.357+0000 7f23fc820740 1 -- 172.21.15.120:0/1829526532 wait complete. 2024-09-19T09:10:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.357+0000 7f23fc820740 20 client.18051 trim_cache size 0 max 16384 2024-09-19T09:10:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:10:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:10:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:10:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:10:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.358+0000 7f23f9a82640 1 -- 172.21.15.120:0/879187204 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f23dc0a32f0 msgr2=0x7f23dc0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.358+0000 7f23f9a82640 1 --2- 172.21.15.120:0/879187204 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f23dc0a32f0 0x7f23dc0a57b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f23e4005ab0 tx=0x7f23e4000bc0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.358+0000 7f23f9a82640 1 -- 172.21.15.120:0/879187204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23f4101c30 msgr2=0x7f23f4198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.358+0000 7f23f9a82640 1 --2- 172.21.15.120:0/879187204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23f4101c30 0x7f23f4198330 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f23e800e9f0 tx=0x7f23e800edf0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.358+0000 7f23f9a82640 1 -- 172.21.15.120:0/879187204 shutdown_connections 2024-09-19T09:10:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.358+0000 7f23f9a82640 1 --2- 172.21.15.120:0/879187204 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f23dc0a32f0 0x7f23dc0a57b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.358+0000 7f23f9a82640 1 --2- 172.21.15.120:0/879187204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23f41025f0 0x7f23f419f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.358+0000 7f23f9a82640 1 --2- 172.21.15.120:0/879187204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23f4101c30 0x7f23f4198330 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.358+0000 7f23f9a82640 1 --2- 172.21.15.120:0/879187204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23f4100a40 0x7f23f4197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.358+0000 7f23f9a82640 1 -- 172.21.15.120:0/879187204 >> 172.21.15.120:0/879187204 conn(0x7f23f40fc1e0 msgr2=0x7f23f4104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.358+0000 7f23f9a82640 1 -- 172.21.15.120:0/879187204 shutdown_connections 2024-09-19T09:10:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:19.358+0000 7f23f9a82640 1 -- 172.21.15.120:0/879187204 wait complete. 2024-09-19T09:10:20.057 INFO:tasks.workunit.client.0.smithi120.stdout: 1 698442 10.01 MB/sec execute 255 sec latency 11865.448 ms 2024-09-19T09:10:20.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:19 smithi196 ceph-mon[35150]: pgmap v566: 97 pgs: 97 active+clean; 37 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 264 op/s 2024-09-19T09:10:20.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:19 smithi153 ceph-mon[35071]: pgmap v566: 97 pgs: 97 active+clean; 37 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 264 op/s 2024-09-19T09:10:20.268 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:10:20.269 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:10:20.296 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:19 smithi120 ceph-mon[31150]: pgmap v566: 97 pgs: 97 active+clean; 37 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 264 op/s 2024-09-19T09:10:20.525 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:10:21.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.032+0000 7fc033171640 1 -- 172.21.15.120:0/2773429391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc02c100a80 msgr2=0x7fc02c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:21.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.032+0000 7fc033171640 1 --2- 172.21.15.120:0/2773429391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc02c100a80 0x7fc02c100ee0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fc020009930 tx=0x7fc02002a420 comp rx=0 tx=0).stop 2024-09-19T09:10:21.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.032+0000 7fc033171640 1 -- 172.21.15.120:0/2773429391 shutdown_connections 2024-09-19T09:10:21.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.032+0000 7fc033171640 1 --2- 172.21.15.120:0/2773429391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc02c101420 0x7fc02c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.032+0000 7fc033171640 1 --2- 172.21.15.120:0/2773429391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc02c100a80 0x7fc02c100ee0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.032+0000 7fc033171640 1 --2- 172.21.15.120:0/2773429391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc02c106a80 0x7fc02c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.032+0000 7fc033171640 1 -- 172.21.15.120:0/2773429391 >> 172.21.15.120:0/2773429391 conn(0x7fc02c0fc7a0 msgr2=0x7fc02c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:21.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.032+0000 7fc033171640 1 -- 172.21.15.120:0/2773429391 shutdown_connections 2024-09-19T09:10:21.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.032+0000 7fc033171640 1 -- 172.21.15.120:0/2773429391 wait complete. 2024-09-19T09:10:21.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.033+0000 7fc033171640 1 Processor -- start 2024-09-19T09:10:21.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.033+0000 7fc033171640 1 -- start start 2024-09-19T09:10:21.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.033+0000 7fc033171640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc02c100a80 0x7fc02c197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:21.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.033+0000 7fc033171640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc02c101420 0x7fc02c1975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:21.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.033+0000 7fc033171640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc02c106a80 0x7fc02c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:21.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.033+0000 7fc033171640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c110e80 con 0x7fc02c100a80 2024-09-19T09:10:21.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.033+0000 7fc033171640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c110d00 con 0x7fc02c106a80 2024-09-19T09:10:21.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.033+0000 7fc033171640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c111000 con 0x7fc02c101420 2024-09-19T09:10:21.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.033+0000 7fc030f0d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc02c100a80 0x7fc02c197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:21.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.033+0000 7fc030f0d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc02c100a80 0x7fc02c197080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55326/0 (socket says 172.21.15.120:55326) 2024-09-19T09:10:21.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.033+0000 7fc030f0d640 1 -- 172.21.15.120:0/2754131742 learned_addr learned my addr 172.21.15.120:0/2754131742 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:21.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.034+0000 7fc03170e640 1 --2- 172.21.15.120:0/2754131742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc02c106a80 0x7fc02c197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:21.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.034+0000 7fc030f0d640 1 -- 172.21.15.120:0/2754131742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc02c101420 msgr2=0x7fc02c1975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:21.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.034+0000 7fc030f0d640 1 --2- 172.21.15.120:0/2754131742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc02c101420 0x7fc02c1975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.034+0000 7fc030f0d640 1 -- 172.21.15.120:0/2754131742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc02c106a80 msgr2=0x7fc02c197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:21.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.034+0000 7fc030f0d640 1 --2- 172.21.15.120:0/2754131742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc02c106a80 0x7fc02c197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.034+0000 7fc030f0d640 1 -- 172.21.15.120:0/2754131742 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02c1a68f0 con 0x7fc02c100a80 2024-09-19T09:10:21.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.034+0000 7fc030f0d640 1 --2- 172.21.15.120:0/2754131742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc02c100a80 0x7fc02c197080 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fc02400eab0 tx=0x7fc02400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:21.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.034+0000 7fc029ffb640 1 -- 172.21.15.120:0/2754131742 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02400ce40 con 0x7fc02c100a80 2024-09-19T09:10:21.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.034+0000 7fc029ffb640 1 -- 172.21.15.120:0/2754131742 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc024011950 con 0x7fc02c100a80 2024-09-19T09:10:21.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.034+0000 7fc033171640 1 -- 172.21.15.120:0/2754131742 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc02c1a6be0 con 0x7fc02c100a80 2024-09-19T09:10:21.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.034+0000 7fc029ffb640 1 -- 172.21.15.120:0/2754131742 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024010ae0 con 0x7fc02c100a80 2024-09-19T09:10:21.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.034+0000 7fc033171640 1 -- 172.21.15.120:0/2754131742 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc02c1a7170 con 0x7fc02c100a80 2024-09-19T09:10:21.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.036+0000 7fc029ffb640 1 -- 172.21.15.120:0/2754131742 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fc024011b00 con 0x7fc02c100a80 2024-09-19T09:10:21.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.037+0000 7fc03554a740 1 Processor -- start 2024-09-19T09:10:21.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.037+0000 7fc03554a740 1 -- start start 2024-09-19T09:10:21.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.037+0000 7fc03554a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55851d3fab60 0x55851d3faf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:21.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.037+0000 7fc03554a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55851d3fb480 0x55851d405800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:21.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.037+0000 7fc03554a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55851d405d40 0x55851d408130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:21.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.037+0000 7fc03554a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55851d360280 con 0x55851d3fab60 2024-09-19T09:10:21.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.037+0000 7fc03554a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55851d307b10 con 0x55851d405d40 2024-09-19T09:10:21.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.037+0000 7fc03554a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55851d35c240 con 0x55851d3fb480 2024-09-19T09:10:21.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.037+0000 7fc029ffb640 1 --2- 172.21.15.120:0/2754131742 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc0140a32c0 0x7fc0140a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:21.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.037+0000 7fc03170e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55851d3fab60 0x55851d3faf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:21.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.037+0000 7fc03170e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55851d3fab60 0x55851d3faf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55340/0 (socket says 172.21.15.120:55340) 2024-09-19T09:10:21.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.037+0000 7fc03170e640 1 -- 172.21.15.120:0/3127471866 learned_addr learned my addr 172.21.15.120:0/3127471866 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:21.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.037+0000 7fc030f0d640 1 --2- 172.21.15.120:0/3127471866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55851d3fb480 0x55851d405800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:21.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.037+0000 7fc03170e640 1 --2- 172.21.15.120:0/2754131742 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc0140a32c0 0x7fc0140a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:21.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.037+0000 7fc029ffb640 1 -- 172.21.15.120:0/2754131742 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fc0240cff80 con 0x7fc02c100a80 2024-09-19T09:10:21.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.037+0000 7fc02bfff640 1 --2- 172.21.15.120:0/3127471866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55851d405d40 0x55851d408130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:21.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.037+0000 7fc03170e640 1 -- 172.21.15.120:0/3127471866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55851d3fb480 msgr2=0x55851d405800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:21.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.037+0000 7fc03170e640 1 --2- 172.21.15.120:0/3127471866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55851d3fb480 0x55851d405800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.037+0000 7fc03170e640 1 -- 172.21.15.120:0/3127471866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55851d405d40 msgr2=0x55851d408130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:21.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.037+0000 7fc03170e640 1 --2- 172.21.15.120:0/3127471866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55851d405d40 0x55851d408130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.037+0000 7fc03170e640 1 -- 172.21.15.120:0/3127471866 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55851d3141f0 con 0x55851d3fab60 2024-09-19T09:10:21.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.038+0000 7fc03170e640 1 --2- 172.21.15.120:0/3127471866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55851d3fab60 0x55851d3faf40 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fc01c012bc0 tx=0x7fc01c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76925f712d286d2e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:21.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.038+0000 7fc03170e640 1 --2- 172.21.15.120:0/2754131742 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc0140a32c0 0x7fc0140a5780 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc01c005fd0 tx=0x7fc01c01b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:21.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.038+0000 7fc012ffd640 1 -- 172.21.15.120:0/3127471866 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c021070 con 0x55851d3fab60 2024-09-19T09:10:21.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.038+0000 7fc012ffd640 1 -- 172.21.15.120:0/3127471866 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc01c00ad20 con 0x55851d3fab60 2024-09-19T09:10:21.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.038+0000 7fc012ffd640 1 -- 172.21.15.120:0/3127471866 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c004a00 con 0x55851d3fab60 2024-09-19T09:10:21.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.038+0000 7fc03554a740 1 -- 172.21.15.120:0/3127471866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55851d3fab60 msgr2=0x55851d3faf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:21.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.038+0000 7fc03554a740 1 --2- 172.21.15.120:0/3127471866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55851d3fab60 0x55851d3faf40 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fc01c012bc0 tx=0x7fc01c0107a0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.038+0000 7fc03554a740 1 -- 172.21.15.120:0/3127471866 shutdown_connections 2024-09-19T09:10:21.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.038+0000 7fc03554a740 1 --2- 172.21.15.120:0/3127471866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55851d405d40 0x55851d408130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.039+0000 7fc03554a740 1 --2- 172.21.15.120:0/3127471866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55851d3fb480 0x55851d405800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.039+0000 7fc03554a740 1 --2- 172.21.15.120:0/3127471866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55851d3fab60 0x55851d3faf40 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.039+0000 7fc03554a740 1 -- 172.21.15.120:0/3127471866 >> 172.21.15.120:0/3127471866 conn(0x55851d3f70f0 msgr2=0x55851d3f74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:21.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.039+0000 7fc03554a740 1 -- 172.21.15.120:0/3127471866 shutdown_connections 2024-09-19T09:10:21.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.039+0000 7fc03554a740 1 -- 172.21.15.120:0/3127471866 wait complete. 2024-09-19T09:10:21.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.039+0000 7fc03554a740 1 -- start start 2024-09-19T09:10:21.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.039+0000 7fc03554a740 1 Processor -- start 2024-09-19T09:10:21.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.039+0000 7fc03554a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55851d3fab60 0x55851d402c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:21.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.039+0000 7fc03554a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55851d3fb480 0x55851d403180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:21.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.039+0000 7fc03554a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55851d405d40 0x55851d4036c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:21.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.039+0000 7fc03554a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55851d408910 con 0x55851d3fab60 2024-09-19T09:10:21.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.039+0000 7fc03554a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55851d31af10 con 0x55851d405d40 2024-09-19T09:10:21.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.039+0000 7fc03554a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55851d408a90 con 0x55851d3fb480 2024-09-19T09:10:21.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.039+0000 7fc03554a740 10 client.0 _ll_register_callbacks cb 0x55851d2e3a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:21.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.039+0000 7fc030f0d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55851d3fab60 0x55851d402c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:21.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.039+0000 7fc030f0d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55851d3fab60 0x55851d402c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55342/0 (socket says 172.21.15.120:55342) 2024-09-19T09:10:21.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.039+0000 7fc030f0d640 1 -- 172.21.15.120:0/2977179586 learned_addr learned my addr 172.21.15.120:0/2977179586 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:21.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.040+0000 7fc02bfff640 1 --2- 172.21.15.120:0/2977179586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55851d3fb480 0x55851d403180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:21.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.040+0000 7fc03170e640 1 --2- 172.21.15.120:0/2977179586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55851d405d40 0x55851d4036c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:21.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.040+0000 7fc030f0d640 1 -- 172.21.15.120:0/2977179586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55851d3fb480 msgr2=0x55851d403180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:21.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.040+0000 7fc030f0d640 1 --2- 172.21.15.120:0/2977179586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55851d3fb480 0x55851d403180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.040+0000 7fc030f0d640 1 -- 172.21.15.120:0/2977179586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55851d405d40 msgr2=0x55851d4036c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:21.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.040+0000 7fc030f0d640 1 --2- 172.21.15.120:0/2977179586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55851d405d40 0x55851d4036c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.040+0000 7fc030f0d640 1 -- 172.21.15.120:0/2977179586 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55851d0d16a0 con 0x55851d3fab60 2024-09-19T09:10:21.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.040+0000 7fc030f0d640 1 --2- 172.21.15.120:0/2977179586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55851d3fab60 0x55851d402c40 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fc0240cda80 tx=0x7fc024015ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49db54fc95e518d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:21.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.040+0000 7fc00bfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:21.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.040+0000 7fc00bfff640 1 -- 172.21.15.120:0/2977179586 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02401c0c0 con 0x55851d3fab60 2024-09-19T09:10:21.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.040+0000 7fc00bfff640 1 -- 172.21.15.120:0/2977179586 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fc02409c800 con 0x55851d3fab60 2024-09-19T09:10:21.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.040+0000 7fc03554a740 1 -- 172.21.15.120:0/2977179586 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc01c00ad20 con 0x55851d3fab60 2024-09-19T09:10:21.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.040+0000 7fc00bfff640 1 -- 172.21.15.120:0/2977179586 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc02409cca0 con 0x55851d3fab60 2024-09-19T09:10:21.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.040+0000 7fc03554a740 10 client.18063 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:21.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.040+0000 7fc03554a740 1 -- 172.21.15.120:0/2977179586 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55851d414540 con 0x55851d3fab60 2024-09-19T09:10:21.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.040+0000 7fc00bfff640 1 -- 172.21.15.120:0/2977179586 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc01c00ad20 con 0x55851d3fab60 2024-09-19T09:10:21.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.041+0000 7fc00bfff640 1 -- 172.21.15.120:0/2977179586 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc02400f040 con 0x55851d3fab60 2024-09-19T09:10:21.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.041+0000 7fc03554a740 10 client.18063 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:21.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.041+0000 7fc03554a740 10 client.18063 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:10:21.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.041+0000 7fc03554a740 20 client.18063 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:21.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.041+0000 7fc03554a740 1 --2- 172.21.15.120:0/2977179586 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55851d415a00 0x55851d417e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:21.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.041+0000 7fc03554a740 4 client.18063 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:10:21.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.041+0000 7fc03554a740 1 -- 172.21.15.120:0/2977179586 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55851d36d7f0 con 0x55851d415a00 2024-09-19T09:10:21.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.041+0000 7fc02bfff640 1 --2- 172.21.15.120:0/2977179586 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55851d415a00 0x55851d417e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:21.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.041+0000 7fc02bfff640 1 --2- 172.21.15.120:0/2977179586 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55851d415a00 0x55851d417e50 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b195f9bb142827c server_cookie=1ba5f5d96ac7bb62 in_seq=0 out_seq=0 2024-09-19T09:10:21.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.041+0000 7fc00bfff640 10 client.18063 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:21.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.042+0000 7fc00bfff640 1 -- 172.21.15.120:0/2977179586 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55851d36d7f0 con 0x55851d415a00 2024-09-19T09:10:21.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.042+0000 7fc00bfff640 10 client.18063 handle_command_reply: tid=0 2024-09-19T09:10:21.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.042+0000 7fc03554a740 1 client.18063 shutdown 2024-09-19T09:10:21.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.042+0000 7fc03554a740 1 -- 172.21.15.120:0/2977179586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55851d3fab60 msgr2=0x55851d402c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:21.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.042+0000 7fc03554a740 1 --2- 172.21.15.120:0/2977179586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55851d3fab60 0x55851d402c40 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fc0240cda80 tx=0x7fc024015ec0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.042+0000 7fc03554a740 1 -- 172.21.15.120:0/2977179586 shutdown_connections 2024-09-19T09:10:21.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.042+0000 7fc03554a740 1 --2- 172.21.15.120:0/2977179586 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55851d415a00 0x55851d417e50 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.042+0000 7fc03554a740 1 --2- 172.21.15.120:0/2977179586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55851d405d40 0x55851d4036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.042+0000 7fc03554a740 1 --2- 172.21.15.120:0/2977179586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55851d3fb480 0x55851d403180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.042+0000 7fc00bfff640 0 client.18063 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:21.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.042+0000 7fc03554a740 1 --2- 172.21.15.120:0/2977179586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55851d3fab60 0x55851d402c40 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.042+0000 7fc03554a740 1 -- 172.21.15.120:0/2977179586 >> 172.21.15.120:0/2977179586 conn(0x55851d3f70f0 msgr2=0x55851d3f6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:21.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.042+0000 7fc03554a740 1 -- 172.21.15.120:0/2977179586 shutdown_connections 2024-09-19T09:10:21.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.042+0000 7fc03554a740 1 -- 172.21.15.120:0/2977179586 wait complete. 2024-09-19T09:10:21.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.042+0000 7fc03554a740 20 client.18063 trim_cache size 0 max 16384 2024-09-19T09:10:21.068 INFO:tasks.workunit.client.0.smithi120.stdout: 1 698442 9.97 MB/sec execute 256 sec latency 12865.476 ms 2024-09-19T09:10:21.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.060+0000 7fc03554a740 1 Processor -- start 2024-09-19T09:10:21.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.060+0000 7fc03554a740 1 -- start start 2024-09-19T09:10:21.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.060+0000 7fc03554a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55851d3fb480 0x55851d3fd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:21.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.060+0000 7fc03554a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55851d405d40 0x55851d41d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:21.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.060+0000 7fc03554a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55851d401be0 0x55851d41dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:21.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.060+0000 7fc03554a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c1af810 con 0x55851d3fb480 2024-09-19T09:10:21.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.060+0000 7fc03554a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c1b1350 con 0x55851d405d40 2024-09-19T09:10:21.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.060+0000 7fc03554a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c1afee0 con 0x55851d401be0 2024-09-19T09:10:21.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.060+0000 7fc030f0d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55851d3fb480 0x55851d3fd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:21.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.060+0000 7fc030f0d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55851d3fb480 0x55851d3fd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55348/0 (socket says 172.21.15.120:55348) 2024-09-19T09:10:21.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.060+0000 7fc030f0d640 1 -- 172.21.15.120:0/2100079678 learned_addr learned my addr 172.21.15.120:0/2100079678 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:21.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.060+0000 7fc03170e640 1 --2- 172.21.15.120:0/2100079678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55851d401be0 0x55851d41dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:21.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.060+0000 7fc02bfff640 1 --2- 172.21.15.120:0/2100079678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55851d405d40 0x55851d41d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:21.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.061+0000 7fc030f0d640 1 -- 172.21.15.120:0/2100079678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55851d401be0 msgr2=0x55851d41dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:21.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.061+0000 7fc030f0d640 1 --2- 172.21.15.120:0/2100079678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55851d401be0 0x55851d41dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.061+0000 7fc030f0d640 1 -- 172.21.15.120:0/2100079678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55851d405d40 msgr2=0x55851d41d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:21.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.061+0000 7fc030f0d640 1 --2- 172.21.15.120:0/2100079678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55851d405d40 0x55851d41d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.061+0000 7fc030f0d640 1 -- 172.21.15.120:0/2100079678 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55851d418480 con 0x55851d3fb480 2024-09-19T09:10:21.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.061+0000 7fc030f0d640 1 --2- 172.21.15.120:0/2100079678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55851d3fb480 0x55851d3fd5c0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fc0240a52e0 tx=0x7fc0240094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6674c4315e51ca8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:21.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.061+0000 7fc0127fc640 1 -- 172.21.15.120:0/2100079678 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02409c5a0 con 0x55851d3fb480 2024-09-19T09:10:21.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.061+0000 7fc0127fc640 1 -- 172.21.15.120:0/2100079678 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0240cdb00 con 0x55851d3fb480 2024-09-19T09:10:21.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.061+0000 7fc0127fc640 1 -- 172.21.15.120:0/2100079678 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02409b9f0 con 0x55851d3fb480 2024-09-19T09:10:21.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.062+0000 7fc03554a740 1 -- 172.21.15.120:0/2100079678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55851d3fb480 msgr2=0x55851d3fd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:21.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.062+0000 7fc03554a740 1 --2- 172.21.15.120:0/2100079678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55851d3fb480 0x55851d3fd5c0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fc0240a52e0 tx=0x7fc0240094a0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.062+0000 7fc03554a740 1 -- 172.21.15.120:0/2100079678 shutdown_connections 2024-09-19T09:10:21.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.062+0000 7fc03554a740 1 --2- 172.21.15.120:0/2100079678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55851d401be0 0x55851d41dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.062+0000 7fc03554a740 1 --2- 172.21.15.120:0/2100079678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55851d405d40 0x55851d41d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.062+0000 7fc03554a740 1 --2- 172.21.15.120:0/2100079678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55851d3fb480 0x55851d3fd5c0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.062+0000 7fc03554a740 1 -- 172.21.15.120:0/2100079678 >> 172.21.15.120:0/2100079678 conn(0x55851d3f70f0 msgr2=0x55851d404bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:21.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.062+0000 7fc03554a740 1 -- 172.21.15.120:0/2100079678 shutdown_connections 2024-09-19T09:10:21.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.062+0000 7fc03554a740 1 -- 172.21.15.120:0/2100079678 wait complete. 2024-09-19T09:10:21.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.062+0000 7fc03554a740 1 -- start start 2024-09-19T09:10:21.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.062+0000 7fc03554a740 1 Processor -- start 2024-09-19T09:10:21.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.062+0000 7fc03554a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55851d3fb480 0x55851d40acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:21.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.063+0000 7fc03554a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55851d401be0 0x55851d40b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:21.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.063+0000 7fc03554a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55851d405d40 0x55851d40b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:21.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.063+0000 7fc03554a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c1b0c80 con 0x55851d3fb480 2024-09-19T09:10:21.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.063+0000 7fc03554a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c1b05b0 con 0x55851d401be0 2024-09-19T09:10:21.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.063+0000 7fc03554a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c1ad7d0 con 0x55851d405d40 2024-09-19T09:10:21.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.063+0000 7fc03554a740 10 client.0 _ll_register_callbacks cb 0x7fc0240103d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:21.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.063+0000 7fc030f0d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55851d3fb480 0x55851d40acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:21.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.063+0000 7fc030f0d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55851d3fb480 0x55851d40acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55358/0 (socket says 172.21.15.120:55358) 2024-09-19T09:10:21.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.063+0000 7fc030f0d640 1 -- 172.21.15.120:0/3046792014 learned_addr learned my addr 172.21.15.120:0/3046792014 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:21.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.063+0000 7fc03170e640 1 --2- 172.21.15.120:0/3046792014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55851d405d40 0x55851d40b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:21.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.063+0000 7fc02bfff640 1 --2- 172.21.15.120:0/3046792014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55851d401be0 0x55851d40b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:21.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.063+0000 7fc030f0d640 1 -- 172.21.15.120:0/3046792014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55851d405d40 msgr2=0x55851d40b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:21.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.063+0000 7fc030f0d640 1 --2- 172.21.15.120:0/3046792014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55851d405d40 0x55851d40b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.063+0000 7fc030f0d640 1 -- 172.21.15.120:0/3046792014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55851d401be0 msgr2=0x55851d40b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:21.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.063+0000 7fc030f0d640 1 --2- 172.21.15.120:0/3046792014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55851d401be0 0x55851d40b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.063+0000 7fc030f0d640 1 -- 172.21.15.120:0/3046792014 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55851d480730 con 0x55851d3fb480 2024-09-19T09:10:21.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.063+0000 7fc03170e640 1 --2- 172.21.15.120:0/3046792014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55851d405d40 0x55851d40b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:10:21.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.063+0000 7fc02bfff640 1 --2- 172.21.15.120:0/3046792014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55851d401be0 0x55851d40b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:10:21.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.063+0000 7fc030f0d640 1 --2- 172.21.15.120:0/3046792014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55851d3fb480 0x55851d40acb0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fc024098f70 tx=0x7fc0240a4fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d426ac111d5571b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:21.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.063+0000 7fc00bfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:21.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.063+0000 7fc00bfff640 1 -- 172.21.15.120:0/3046792014 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0240b9020 con 0x55851d3fb480 2024-09-19T09:10:21.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.063+0000 7fc00bfff640 1 -- 172.21.15.120:0/3046792014 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fc02409c800 con 0x55851d3fb480 2024-09-19T09:10:21.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.063+0000 7fc03554a740 1 -- 172.21.15.120:0/3046792014 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0240cdb00 con 0x55851d3fb480 2024-09-19T09:10:21.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.063+0000 7fc00bfff640 1 -- 172.21.15.120:0/3046792014 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc0240a0950 con 0x55851d3fb480 2024-09-19T09:10:21.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.063+0000 7fc03554a740 10 client.18069 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:21.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.063+0000 7fc03554a740 1 -- 172.21.15.120:0/3046792014 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55851d480730 con 0x55851d3fb480 2024-09-19T09:10:21.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.064+0000 7fc00bfff640 1 -- 172.21.15.120:0/3046792014 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0240cdb00 con 0x55851d3fb480 2024-09-19T09:10:21.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.064+0000 7fc00bfff640 1 -- 172.21.15.120:0/3046792014 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc0240b0ac0 con 0x55851d3fb480 2024-09-19T09:10:21.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.064+0000 7fc03554a740 10 client.18069 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:21.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.064+0000 7fc03554a740 10 client.18069 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:10:21.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.064+0000 7fc03554a740 20 client.18069 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:21.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.064+0000 7fc03554a740 1 --2- 172.21.15.120:0/3046792014 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55851d481ca0 0x55851d484090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:21.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.064+0000 7fc03554a740 4 client.18069 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:10:21.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.064+0000 7fc03554a740 1 -- 172.21.15.120:0/3046792014 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55851d367d00 con 0x55851d481ca0 2024-09-19T09:10:21.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.064+0000 7fc02bfff640 1 --2- 172.21.15.120:0/3046792014 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55851d481ca0 0x55851d484090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:21.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.065+0000 7fc02bfff640 1 --2- 172.21.15.120:0/3046792014 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55851d481ca0 0x55851d484090 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95a5d0301b7922eb server_cookie=49602c896c9add0 in_seq=0 out_seq=0 2024-09-19T09:10:21.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.065+0000 7fc00bfff640 10 client.18069 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:21.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.066+0000 7fc00bfff640 1 -- 172.21.15.120:0/3046792014 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55851d367d00 con 0x55851d481ca0 2024-09-19T09:10:21.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.066+0000 7fc00bfff640 10 client.18069 handle_command_reply: tid=0 2024-09-19T09:10:21.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.066+0000 7fc03554a740 1 client.18069 shutdown 2024-09-19T09:10:21.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.066+0000 7fc03554a740 1 -- 172.21.15.120:0/3046792014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55851d3fb480 msgr2=0x55851d40acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:21.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.066+0000 7fc03554a740 1 --2- 172.21.15.120:0/3046792014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55851d3fb480 0x55851d40acb0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fc024098f70 tx=0x7fc0240a4fb0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.066+0000 7fc03554a740 1 -- 172.21.15.120:0/3046792014 shutdown_connections 2024-09-19T09:10:21.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.066+0000 7fc03554a740 1 --2- 172.21.15.120:0/3046792014 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55851d481ca0 0x55851d484090 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.066+0000 7fc00bfff640 0 client.18069 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:21.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.066+0000 7fc03554a740 1 --2- 172.21.15.120:0/3046792014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55851d405d40 0x55851d40b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.066+0000 7fc03554a740 1 --2- 172.21.15.120:0/3046792014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55851d401be0 0x55851d40b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.066+0000 7fc03554a740 1 --2- 172.21.15.120:0/3046792014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55851d3fb480 0x55851d40acb0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.066+0000 7fc03554a740 1 -- 172.21.15.120:0/3046792014 >> 172.21.15.120:0/3046792014 conn(0x55851d3f70f0 msgr2=0x55851d3f5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:21.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.066+0000 7fc03554a740 1 -- 172.21.15.120:0/3046792014 shutdown_connections 2024-09-19T09:10:21.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.066+0000 7fc03554a740 1 -- 172.21.15.120:0/3046792014 wait complete. 2024-09-19T09:10:21.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.066+0000 7fc03554a740 20 client.18069 trim_cache size 0 max 16384 2024-09-19T09:10:21.082 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:10:21.083 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:10:21.083 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "83be93a9-db7d-4d5d-ab41-d0395294c05a", 2024-09-19T09:10:21.083 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:10:21.083 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:10:21.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.067+0000 7fc033171640 1 -- 172.21.15.120:0/2754131742 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc0140a32c0 msgr2=0x7fc0140a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:21.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.067+0000 7fc033171640 1 --2- 172.21.15.120:0/2754131742 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc0140a32c0 0x7fc0140a5780 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc01c005fd0 tx=0x7fc01c01b000 comp rx=0 tx=0).stop 2024-09-19T09:10:21.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.067+0000 7fc033171640 1 -- 172.21.15.120:0/2754131742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc02c100a80 msgr2=0x7fc02c197080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:21.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.067+0000 7fc033171640 1 --2- 172.21.15.120:0/2754131742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc02c100a80 0x7fc02c197080 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fc02400eab0 tx=0x7fc02400ef70 comp rx=0 tx=0).stop 2024-09-19T09:10:21.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.067+0000 7fc033171640 1 -- 172.21.15.120:0/2754131742 shutdown_connections 2024-09-19T09:10:21.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.067+0000 7fc033171640 1 --2- 172.21.15.120:0/2754131742 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc0140a32c0 0x7fc0140a5780 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.067+0000 7fc033171640 1 --2- 172.21.15.120:0/2754131742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc02c106a80 0x7fc02c197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.067+0000 7fc033171640 1 --2- 172.21.15.120:0/2754131742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc02c101420 0x7fc02c1975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.067+0000 7fc033171640 1 --2- 172.21.15.120:0/2754131742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc02c100a80 0x7fc02c197080 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:21.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.067+0000 7fc033171640 1 -- 172.21.15.120:0/2754131742 >> 172.21.15.120:0/2754131742 conn(0x7fc02c0fc7a0 msgr2=0x7fc02c0fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:21.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.067+0000 7fc033171640 1 -- 172.21.15.120:0/2754131742 shutdown_connections 2024-09-19T09:10:21.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:21.067+0000 7fc033171640 1 -- 172.21.15.120:0/2754131742 wait complete. 2024-09-19T09:10:21.957 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:10:22.057 INFO:tasks.workunit.client.0.smithi120.stdout: 1 698442 9.93 MB/sec execute 257 sec latency 13865.529 ms 2024-09-19T09:10:22.273 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:10:22.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.791+0000 7ff332d40640 1 -- 172.21.15.120:0/4034736126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff32c1082a0 msgr2=0x7ff32c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:22.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.791+0000 7ff332d40640 1 --2- 172.21.15.120:0/4034736126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff32c1082a0 0x7ff32c10ec40 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7ff324009930 tx=0x7ff32402a420 comp rx=0 tx=0).stop 2024-09-19T09:10:22.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.791+0000 7ff332d40640 1 -- 172.21.15.120:0/4034736126 shutdown_connections 2024-09-19T09:10:22.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.791+0000 7ff332d40640 1 --2- 172.21.15.120:0/4034736126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff32c1082a0 0x7ff32c10ec40 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.791+0000 7ff332d40640 1 --2- 172.21.15.120:0/4034736126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff32c1036e0 0x7ff32c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.791+0000 7ff332d40640 1 --2- 172.21.15.120:0/4034736126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff32c102d30 0x7ff32c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.791+0000 7ff332d40640 1 -- 172.21.15.120:0/4034736126 >> 172.21.15.120:0/4034736126 conn(0x7ff32c0fc7d0 msgr2=0x7ff32c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:22.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.792+0000 7ff332d40640 1 -- 172.21.15.120:0/4034736126 shutdown_connections 2024-09-19T09:10:22.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.792+0000 7ff332d40640 1 -- 172.21.15.120:0/4034736126 wait complete. 2024-09-19T09:10:22.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.792+0000 7ff332d40640 1 Processor -- start 2024-09-19T09:10:22.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.792+0000 7ff332d40640 1 -- start start 2024-09-19T09:10:22.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.792+0000 7ff332d40640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff32c102d30 0x7ff32c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:22.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.792+0000 7ff332d40640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff32c1036e0 0x7ff32c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:22.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.792+0000 7ff332d40640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff32c1082a0 0x7ff32c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:22.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.792+0000 7ff332d40640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff32c1120a0 con 0x7ff32c1082a0 2024-09-19T09:10:22.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.792+0000 7ff332d40640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff32c111f20 con 0x7ff32c102d30 2024-09-19T09:10:22.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.792+0000 7ff332d40640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff32c112220 con 0x7ff32c1036e0 2024-09-19T09:10:22.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.792+0000 7ff3312dd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff32c1082a0 0x7ff32c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:22.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.792+0000 7ff3312dd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff32c1082a0 0x7ff32c101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57498/0 (socket says 172.21.15.120:57498) 2024-09-19T09:10:22.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.792+0000 7ff3312dd640 1 -- 172.21.15.120:0/1732433575 learned_addr learned my addr 172.21.15.120:0/1732433575 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:22.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.793+0000 7ff330adc640 1 --2- 172.21.15.120:0/1732433575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff32c102d30 0x7ff32c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:22.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.793+0000 7ff32bfff640 1 --2- 172.21.15.120:0/1732433575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff32c1036e0 0x7ff32c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:22.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.793+0000 7ff3312dd640 1 -- 172.21.15.120:0/1732433575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff32c1036e0 msgr2=0x7ff32c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:22.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.793+0000 7ff3312dd640 1 --2- 172.21.15.120:0/1732433575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff32c1036e0 0x7ff32c100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.793+0000 7ff3312dd640 1 -- 172.21.15.120:0/1732433575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff32c102d30 msgr2=0x7ff32c100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:22.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.793+0000 7ff3312dd640 1 --2- 172.21.15.120:0/1732433575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff32c102d30 0x7ff32c100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.793+0000 7ff3312dd640 1 -- 172.21.15.120:0/1732433575 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff32c1a6a30 con 0x7ff32c1082a0 2024-09-19T09:10:22.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.793+0000 7ff3312dd640 1 --2- 172.21.15.120:0/1732433575 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff32c1082a0 0x7ff32c101410 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7ff324005ab0 tx=0x7ff3240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:22.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.793+0000 7ff329ffb640 1 -- 172.21.15.120:0/1732433575 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32403f030 con 0x7ff32c1082a0 2024-09-19T09:10:22.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.793+0000 7ff329ffb640 1 -- 172.21.15.120:0/1732433575 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff324003ce0 con 0x7ff32c1082a0 2024-09-19T09:10:22.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.793+0000 7ff329ffb640 1 -- 172.21.15.120:0/1732433575 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff324035070 con 0x7ff32c1082a0 2024-09-19T09:10:22.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.793+0000 7ff332d40640 1 -- 172.21.15.120:0/1732433575 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff32c1a6cc0 con 0x7ff32c1082a0 2024-09-19T09:10:22.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.793+0000 7ff332d40640 1 -- 172.21.15.120:0/1732433575 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff32c1a71a0 con 0x7ff32c1082a0 2024-09-19T09:10:22.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.795+0000 7ff329ffb640 1 -- 172.21.15.120:0/1732433575 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7ff324042030 con 0x7ff32c1082a0 2024-09-19T09:10:22.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.796+0000 7ff335119740 1 Processor -- start 2024-09-19T09:10:22.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.796+0000 7ff335119740 1 -- start start 2024-09-19T09:10:22.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.796+0000 7ff335119740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574222844a0 0x557422284880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:22.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.796+0000 7ff335119740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557422284dc0 0x55742228f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:22.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.796+0000 7ff335119740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742228f680 0x557422291a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:22.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.796+0000 7ff335119740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5574221ea2e0 con 0x55742228f680 2024-09-19T09:10:22.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.796+0000 7ff335119740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557422191ce0 con 0x5574222844a0 2024-09-19T09:10:22.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.796+0000 7ff335119740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5574221e62a0 con 0x557422284dc0 2024-09-19T09:10:22.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.796+0000 7ff329ffb640 1 --2- 172.21.15.120:0/1732433575 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff3180a3240 0x7ff3180a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:22.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.796+0000 7ff32bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574222844a0 0x557422284880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:22.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.796+0000 7ff32bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574222844a0 0x557422284880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:60350/0 (socket says 172.21.15.120:60350) 2024-09-19T09:10:22.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.796+0000 7ff3312dd640 1 --2- 172.21.15.120:0/1732433575 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff3180a3240 0x7ff3180a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:22.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.796+0000 7ff32bfff640 1 -- 172.21.15.120:0/841965587 learned_addr learned my addr 172.21.15.120:0/841965587 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:22.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.796+0000 7ff32bfff640 1 --2- 172.21.15.120:0/841965587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742228f680 0x557422291a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:22.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.796+0000 7ff329ffb640 1 -- 172.21.15.120:0/1732433575 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7ff3240efd00 con 0x7ff32c1082a0 2024-09-19T09:10:22.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.796+0000 7ff330adc640 1 --2- 172.21.15.120:0/841965587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557422284dc0 0x55742228f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:22.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.796+0000 7ff32bfff640 1 -- 172.21.15.120:0/841965587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557422284dc0 msgr2=0x55742228f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:22.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.796+0000 7ff32bfff640 1 --2- 172.21.15.120:0/841965587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557422284dc0 0x55742228f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.797+0000 7ff32bfff640 1 -- 172.21.15.120:0/841965587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742228f680 msgr2=0x557422291a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:22.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.797+0000 7ff32bfff640 1 --2- 172.21.15.120:0/841965587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742228f680 0x557422291a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.797+0000 7ff32bfff640 1 -- 172.21.15.120:0/841965587 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557422176440 con 0x5574222844a0 2024-09-19T09:10:22.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.797+0000 7ff330adc640 1 --2- 172.21.15.120:0/841965587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557422284dc0 0x55742228f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:10:22.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.797+0000 7ff3312dd640 1 --2- 172.21.15.120:0/1732433575 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff3180a3240 0x7ff3180a5700 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff324009900 tx=0x7ff324086e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:22.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.797+0000 7ff32bfff640 1 --2- 172.21.15.120:0/841965587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574222844a0 0x557422284880 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x557422281db0 tx=0x7ff31400a5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=146a15092f6e47d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:22.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.797+0000 7ff322ffd640 1 -- 172.21.15.120:0/841965587 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff314004020 con 0x5574222844a0 2024-09-19T09:10:22.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.797+0000 7ff322ffd640 1 -- 172.21.15.120:0/841965587 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff314005500 con 0x5574222844a0 2024-09-19T09:10:22.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.797+0000 7ff322ffd640 1 -- 172.21.15.120:0/841965587 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff314004ce0 con 0x5574222844a0 2024-09-19T09:10:22.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.798+0000 7ff335119740 1 -- 172.21.15.120:0/841965587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574222844a0 msgr2=0x557422284880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:22.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.798+0000 7ff335119740 1 --2- 172.21.15.120:0/841965587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574222844a0 0x557422284880 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x557422281db0 tx=0x7ff31400a5e0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.798+0000 7ff335119740 1 -- 172.21.15.120:0/841965587 shutdown_connections 2024-09-19T09:10:22.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.798+0000 7ff335119740 1 --2- 172.21.15.120:0/841965587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742228f680 0x557422291a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.798+0000 7ff335119740 1 --2- 172.21.15.120:0/841965587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557422284dc0 0x55742228f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.798+0000 7ff335119740 1 --2- 172.21.15.120:0/841965587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574222844a0 0x557422284880 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.798+0000 7ff335119740 1 -- 172.21.15.120:0/841965587 >> 172.21.15.120:0/841965587 conn(0x557422280a30 msgr2=0x557422280e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:22.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.798+0000 7ff335119740 1 -- 172.21.15.120:0/841965587 shutdown_connections 2024-09-19T09:10:22.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.798+0000 7ff335119740 1 -- 172.21.15.120:0/841965587 wait complete. 2024-09-19T09:10:22.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.798+0000 7ff335119740 1 -- start start 2024-09-19T09:10:22.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.798+0000 7ff335119740 1 Processor -- start 2024-09-19T09:10:22.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.799+0000 7ff335119740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574222844a0 0x55742228f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:22.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.799+0000 7ff335119740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557422284dc0 0x55742228c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:22.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.799+0000 7ff335119740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742228f680 0x55742228c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:22.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.799+0000 7ff335119740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557422292250 con 0x55742228f680 2024-09-19T09:10:22.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.799+0000 7ff335119740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5574221a5050 con 0x5574222844a0 2024-09-19T09:10:22.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.799+0000 7ff335119740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5574222923d0 con 0x557422284dc0 2024-09-19T09:10:22.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.799+0000 7ff335119740 10 client.0 _ll_register_callbacks cb 0x5574221e5160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:22.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.799+0000 7ff32bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742228f680 0x55742228c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:22.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.799+0000 7ff330adc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557422284dc0 0x55742228c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:22.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.799+0000 7ff32bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742228f680 0x55742228c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57518/0 (socket says 172.21.15.120:57518) 2024-09-19T09:10:22.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.799+0000 7ff32bfff640 1 -- 172.21.15.120:0/2284473282 learned_addr learned my addr 172.21.15.120:0/2284473282 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:22.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.799+0000 7ff330adc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557422284dc0 0x55742228c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:50722/0 (socket says 172.21.15.120:50722) 2024-09-19T09:10:22.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.799+0000 7ff32bfff640 1 --2- 172.21.15.120:0/2284473282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574222844a0 0x55742228f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:22.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.799+0000 7ff32bfff640 1 -- 172.21.15.120:0/2284473282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557422284dc0 msgr2=0x55742228c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:22.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.799+0000 7ff32bfff640 1 --2- 172.21.15.120:0/2284473282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557422284dc0 0x55742228c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.799+0000 7ff32bfff640 1 -- 172.21.15.120:0/2284473282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574222844a0 msgr2=0x55742228f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:22.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.799+0000 7ff32bfff640 1 --2- 172.21.15.120:0/2284473282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574222844a0 0x55742228f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.799+0000 7ff32bfff640 1 -- 172.21.15.120:0/2284473282 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557421f5b3c0 con 0x55742228f680 2024-09-19T09:10:22.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.799+0000 7ff32bfff640 1 --2- 172.21.15.120:0/2284473282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742228f680 0x55742228c840 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7ff3140058c0 tx=0x7ff3140049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48166313cb2ceead server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:22.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.799+0000 7ff30bfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:22.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.799+0000 7ff30bfff640 1 -- 172.21.15.120:0/2284473282 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff314005210 con 0x55742228f680 2024-09-19T09:10:22.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.799+0000 7ff30bfff640 1 -- 172.21.15.120:0/2284473282 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7ff31400f640 con 0x55742228f680 2024-09-19T09:10:22.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.800+0000 7ff335119740 1 -- 172.21.15.120:0/2284473282 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff314005500 con 0x55742228f680 2024-09-19T09:10:22.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.800+0000 7ff30bfff640 1 -- 172.21.15.120:0/2284473282 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff314033050 con 0x55742228f680 2024-09-19T09:10:22.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.800+0000 7ff335119740 10 client.18081 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:22.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.800+0000 7ff335119740 1 -- 172.21.15.120:0/2284473282 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55742229dca0 con 0x55742228f680 2024-09-19T09:10:22.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.800+0000 7ff30bfff640 1 -- 172.21.15.120:0/2284473282 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff314005500 con 0x55742228f680 2024-09-19T09:10:22.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.800+0000 7ff30bfff640 1 -- 172.21.15.120:0/2284473282 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff31401e740 con 0x55742228f680 2024-09-19T09:10:22.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.800+0000 7ff335119740 10 client.18081 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:22.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.800+0000 7ff335119740 10 client.18081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:10:22.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.800+0000 7ff335119740 20 client.18081 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:22.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.800+0000 7ff335119740 1 --2- 172.21.15.120:0/2284473282 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55742229f0a0 0x5574222a1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:22.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.800+0000 7ff335119740 4 client.18081 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:10:22.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.800+0000 7ff335119740 1 -- 172.21.15.120:0/2284473282 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574221f7850 con 0x55742229f0a0 2024-09-19T09:10:22.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.800+0000 7ff330adc640 1 --2- 172.21.15.120:0/2284473282 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55742229f0a0 0x5574222a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:22.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.801+0000 7ff30bfff640 10 client.18081 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:22.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.801+0000 7ff330adc640 1 --2- 172.21.15.120:0/2284473282 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55742229f0a0 0x5574222a1490 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cde34f1cd56b7e server_cookie=ce2d7f1a6134b123 in_seq=0 out_seq=0 2024-09-19T09:10:22.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.801+0000 7ff30bfff640 1 -- 172.21.15.120:0/2284473282 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5574221f7850 con 0x55742229f0a0 2024-09-19T09:10:22.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.801+0000 7ff30bfff640 10 client.18081 handle_command_reply: tid=0 2024-09-19T09:10:22.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.801+0000 7ff335119740 1 client.18081 shutdown 2024-09-19T09:10:22.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.801+0000 7ff335119740 1 -- 172.21.15.120:0/2284473282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742228f680 msgr2=0x55742228c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:22.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.801+0000 7ff335119740 1 --2- 172.21.15.120:0/2284473282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742228f680 0x55742228c840 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7ff3140058c0 tx=0x7ff3140049c0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.801+0000 7ff335119740 1 -- 172.21.15.120:0/2284473282 shutdown_connections 2024-09-19T09:10:22.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.802+0000 7ff335119740 1 --2- 172.21.15.120:0/2284473282 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55742229f0a0 0x5574222a1490 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.802+0000 7ff335119740 1 --2- 172.21.15.120:0/2284473282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742228f680 0x55742228c840 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.802+0000 7ff335119740 1 --2- 172.21.15.120:0/2284473282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557422284dc0 0x55742228c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.802+0000 7ff30bfff640 0 client.18081 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:22.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.802+0000 7ff335119740 1 --2- 172.21.15.120:0/2284473282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574222844a0 0x55742228f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.802+0000 7ff335119740 1 -- 172.21.15.120:0/2284473282 >> 172.21.15.120:0/2284473282 conn(0x557422280a30 msgr2=0x557422286710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:22.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.802+0000 7ff335119740 1 -- 172.21.15.120:0/2284473282 shutdown_connections 2024-09-19T09:10:22.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.802+0000 7ff335119740 1 -- 172.21.15.120:0/2284473282 wait complete. 2024-09-19T09:10:22.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.802+0000 7ff335119740 20 client.18081 trim_cache size 0 max 16384 2024-09-19T09:10:22.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.819+0000 7ff335119740 1 Processor -- start 2024-09-19T09:10:22.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.819+0000 7ff335119740 1 -- start start 2024-09-19T09:10:22.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.819+0000 7ff335119740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574222844a0 0x55742229b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:22.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.819+0000 7ff335119740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557422284dc0 0x5574222a6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:22.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.819+0000 7ff335119740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742228f680 0x5574222a7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:22.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.819+0000 7ff335119740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff32c1af810 con 0x55742228f680 2024-09-19T09:10:22.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.819+0000 7ff335119740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff32c1b1350 con 0x557422284dc0 2024-09-19T09:10:22.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.819+0000 7ff335119740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff32c1afee0 con 0x5574222844a0 2024-09-19T09:10:22.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.820+0000 7ff32bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742228f680 0x5574222a7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:22.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.820+0000 7ff32bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742228f680 0x5574222a7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57534/0 (socket says 172.21.15.120:57534) 2024-09-19T09:10:22.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.820+0000 7ff32bfff640 1 -- 172.21.15.120:0/3823260402 learned_addr learned my addr 172.21.15.120:0/3823260402 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:22.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.820+0000 7ff32bfff640 1 --2- 172.21.15.120:0/3823260402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574222844a0 0x55742229b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:22.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.820+0000 7ff330adc640 1 --2- 172.21.15.120:0/3823260402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557422284dc0 0x5574222a6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:22.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.820+0000 7ff32bfff640 1 -- 172.21.15.120:0/3823260402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574222844a0 msgr2=0x55742229b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:22.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.820+0000 7ff32bfff640 1 --2- 172.21.15.120:0/3823260402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574222844a0 0x55742229b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.820+0000 7ff32bfff640 1 -- 172.21.15.120:0/3823260402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557422284dc0 msgr2=0x5574222a6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:22.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.820+0000 7ff32bfff640 1 --2- 172.21.15.120:0/3823260402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557422284dc0 0x5574222a6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.820+0000 7ff32bfff640 1 -- 172.21.15.120:0/3823260402 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55742228f380 con 0x55742228f680 2024-09-19T09:10:22.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.820+0000 7ff32bfff640 1 --2- 172.21.15.120:0/3823260402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742228f680 0x5574222a7160 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7ff31400b010 tx=0x7ff314016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b16d60cbc2292228 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:22.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.821+0000 7ff3227fc640 1 -- 172.21.15.120:0/3823260402 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31400a5d0 con 0x55742228f680 2024-09-19T09:10:22.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.821+0000 7ff3227fc640 1 -- 172.21.15.120:0/3823260402 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff31400a8d0 con 0x55742228f680 2024-09-19T09:10:22.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.821+0000 7ff3227fc640 1 -- 172.21.15.120:0/3823260402 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31401e6d0 con 0x55742228f680 2024-09-19T09:10:22.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.822+0000 7ff335119740 1 -- 172.21.15.120:0/3823260402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742228f680 msgr2=0x5574222a7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:22.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.822+0000 7ff335119740 1 --2- 172.21.15.120:0/3823260402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742228f680 0x5574222a7160 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7ff31400b010 tx=0x7ff314016040 comp rx=0 tx=0).stop 2024-09-19T09:10:22.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.822+0000 7ff335119740 1 -- 172.21.15.120:0/3823260402 shutdown_connections 2024-09-19T09:10:22.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.822+0000 7ff335119740 1 --2- 172.21.15.120:0/3823260402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742228f680 0x5574222a7160 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.822+0000 7ff335119740 1 --2- 172.21.15.120:0/3823260402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557422284dc0 0x5574222a6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.822+0000 7ff335119740 1 --2- 172.21.15.120:0/3823260402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574222844a0 0x55742229b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.822+0000 7ff335119740 1 -- 172.21.15.120:0/3823260402 >> 172.21.15.120:0/3823260402 conn(0x557422280a30 msgr2=0x557422287900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:22.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.822+0000 7ff335119740 1 -- 172.21.15.120:0/3823260402 shutdown_connections 2024-09-19T09:10:22.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.822+0000 7ff335119740 1 -- 172.21.15.120:0/3823260402 wait complete. 2024-09-19T09:10:22.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.822+0000 7ff335119740 1 -- start start 2024-09-19T09:10:22.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.822+0000 7ff335119740 1 Processor -- start 2024-09-19T09:10:22.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.823+0000 7ff335119740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574222844a0 0x557422301ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:22.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.823+0000 7ff335119740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557422284dc0 0x557422302420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:22.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.823+0000 7ff335119740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55742228f680 0x557422292be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:22.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.823+0000 7ff335119740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff32c1b0c80 con 0x5574222844a0 2024-09-19T09:10:22.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.823+0000 7ff335119740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff32c1b05b0 con 0x557422284dc0 2024-09-19T09:10:22.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.823+0000 7ff335119740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff32c1ad7d0 con 0x55742228f680 2024-09-19T09:10:22.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.823+0000 7ff335119740 10 client.0 _ll_register_callbacks cb 0x7ff31400acd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:22.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.823+0000 7ff32bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574222844a0 0x557422301ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:22.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.823+0000 7ff32bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574222844a0 0x557422301ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57546/0 (socket says 172.21.15.120:57546) 2024-09-19T09:10:22.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.823+0000 7ff32bfff640 1 -- 172.21.15.120:0/3386718548 learned_addr learned my addr 172.21.15.120:0/3386718548 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:22.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.823+0000 7ff330adc640 1 --2- 172.21.15.120:0/3386718548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557422284dc0 0x557422302420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:22.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.823+0000 7ff32bfff640 1 -- 172.21.15.120:0/3386718548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55742228f680 msgr2=0x557422292be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:22.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.823+0000 7ff32bfff640 1 --2- 172.21.15.120:0/3386718548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55742228f680 0x557422292be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.823+0000 7ff32bfff640 1 -- 172.21.15.120:0/3386718548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557422284dc0 msgr2=0x557422302420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:22.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.823+0000 7ff32bfff640 1 --2- 172.21.15.120:0/3386718548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557422284dc0 0x557422302420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.823+0000 7ff32bfff640 1 -- 172.21.15.120:0/3386718548 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557422293410 con 0x5574222844a0 2024-09-19T09:10:22.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.823+0000 7ff32bfff640 1 --2- 172.21.15.120:0/3386718548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574222844a0 0x557422301ee0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7ff314005b10 tx=0x7ff314005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9e300aa2530868 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:22.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.823+0000 7ff30bfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:22.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.823+0000 7ff30bfff640 1 -- 172.21.15.120:0/3386718548 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3140053e0 con 0x5574222844a0 2024-09-19T09:10:22.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.823+0000 7ff30bfff640 1 -- 172.21.15.120:0/3386718548 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7ff314011030 con 0x5574222844a0 2024-09-19T09:10:22.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.823+0000 7ff335119740 1 -- 172.21.15.120:0/3386718548 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff31400a8d0 con 0x5574222844a0 2024-09-19T09:10:22.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.824+0000 7ff30bfff640 1 -- 172.21.15.120:0/3386718548 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff314047050 con 0x5574222844a0 2024-09-19T09:10:22.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.824+0000 7ff335119740 10 client.18087 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:22.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.824+0000 7ff335119740 1 -- 172.21.15.120:0/3386718548 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557422309700 con 0x5574222844a0 2024-09-19T09:10:22.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.824+0000 7ff30bfff640 1 -- 172.21.15.120:0/3386718548 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff31400a8d0 con 0x5574222844a0 2024-09-19T09:10:22.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.824+0000 7ff30bfff640 1 -- 172.21.15.120:0/3386718548 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff31400fa20 con 0x5574222844a0 2024-09-19T09:10:22.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.824+0000 7ff335119740 10 client.18087 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:22.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.824+0000 7ff335119740 10 client.18087 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:10:22.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.824+0000 7ff335119740 20 client.18087 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:22.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.824+0000 7ff335119740 1 --2- 172.21.15.120:0/3386718548 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557422309d00 0x55742230c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:22.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.824+0000 7ff335119740 4 client.18087 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:10:22.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.824+0000 7ff335119740 1 -- 172.21.15.120:0/3386718548 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5574221f1d60 con 0x557422309d00 2024-09-19T09:10:22.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.824+0000 7ff330adc640 1 --2- 172.21.15.120:0/3386718548 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557422309d00 0x55742230c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:22.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.825+0000 7ff30bfff640 10 client.18087 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:22.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.825+0000 7ff330adc640 1 --2- 172.21.15.120:0/3386718548 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557422309d00 0x55742230c0f0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4baf200c7e06fd58 server_cookie=59d9d306eff4f7c9 in_seq=0 out_seq=0 2024-09-19T09:10:22.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.825+0000 7ff30bfff640 1 -- 172.21.15.120:0/3386718548 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5574221f1d60 con 0x557422309d00 2024-09-19T09:10:22.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.825+0000 7ff30bfff640 10 client.18087 handle_command_reply: tid=0 2024-09-19T09:10:22.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.825+0000 7ff335119740 1 client.18087 shutdown 2024-09-19T09:10:22.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.825+0000 7ff335119740 1 -- 172.21.15.120:0/3386718548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574222844a0 msgr2=0x557422301ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:22.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.825+0000 7ff335119740 1 --2- 172.21.15.120:0/3386718548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574222844a0 0x557422301ee0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7ff314005b10 tx=0x7ff314005280 comp rx=0 tx=0).stop 2024-09-19T09:10:22.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.825+0000 7ff335119740 1 -- 172.21.15.120:0/3386718548 shutdown_connections 2024-09-19T09:10:22.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.825+0000 7ff335119740 1 --2- 172.21.15.120:0/3386718548 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557422309d00 0x55742230c0f0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.825+0000 7ff335119740 1 --2- 172.21.15.120:0/3386718548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55742228f680 0x557422292be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.825+0000 7ff335119740 1 --2- 172.21.15.120:0/3386718548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557422284dc0 0x557422302420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.825+0000 7ff30bfff640 0 client.18087 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:22.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.825+0000 7ff335119740 1 --2- 172.21.15.120:0/3386718548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574222844a0 0x557422301ee0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.825+0000 7ff335119740 1 -- 172.21.15.120:0/3386718548 >> 172.21.15.120:0/3386718548 conn(0x557422280a30 msgr2=0x55742228db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:22.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.825+0000 7ff335119740 1 -- 172.21.15.120:0/3386718548 shutdown_connections 2024-09-19T09:10:22.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.825+0000 7ff335119740 1 -- 172.21.15.120:0/3386718548 wait complete. 2024-09-19T09:10:22.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.825+0000 7ff335119740 20 client.18087 trim_cache size 0 max 16384 2024-09-19T09:10:22.841 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:10:22.841 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:10:22.841 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-19T09:10:22.841 INFO:teuthology.orchestra.run.smithi120.stdout: "83be93a9-db7d-4d5d-ab41-d0395294c05a": { 2024-09-19T09:10:22.841 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-19T09:10:22.841 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "83be93a9-db7d-4d5d-ab41-d0395294c05a", 2024-09-19T09:10:22.841 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-19T09:10:22.841 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:10:22.841 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:10:22.841 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:10:22.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.826+0000 7ff332d40640 1 -- 172.21.15.120:0/1732433575 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff3180a3240 msgr2=0x7ff3180a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:22.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.826+0000 7ff332d40640 1 --2- 172.21.15.120:0/1732433575 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff3180a3240 0x7ff3180a5700 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff324009900 tx=0x7ff324086e10 comp rx=0 tx=0).stop 2024-09-19T09:10:22.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.826+0000 7ff332d40640 1 -- 172.21.15.120:0/1732433575 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff32c1082a0 msgr2=0x7ff32c101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:22.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.826+0000 7ff332d40640 1 --2- 172.21.15.120:0/1732433575 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff32c1082a0 0x7ff32c101410 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7ff324005ab0 tx=0x7ff3240042f0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.826+0000 7ff332d40640 1 -- 172.21.15.120:0/1732433575 shutdown_connections 2024-09-19T09:10:22.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.826+0000 7ff332d40640 1 --2- 172.21.15.120:0/1732433575 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff3180a3240 0x7ff3180a5700 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.826+0000 7ff332d40640 1 --2- 172.21.15.120:0/1732433575 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff32c1082a0 0x7ff32c101410 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.826+0000 7ff332d40640 1 --2- 172.21.15.120:0/1732433575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff32c1036e0 0x7ff32c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.826+0000 7ff332d40640 1 --2- 172.21.15.120:0/1732433575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff32c102d30 0x7ff32c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:22.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.826+0000 7ff332d40640 1 -- 172.21.15.120:0/1732433575 >> 172.21.15.120:0/1732433575 conn(0x7ff32c0fc7d0 msgr2=0x7ff32c105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:22.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.826+0000 7ff332d40640 1 -- 172.21.15.120:0/1732433575 shutdown_connections 2024-09-19T09:10:22.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:22.826+0000 7ff332d40640 1 -- 172.21.15.120:0/1732433575 wait complete. 2024-09-19T09:10:23.057 INFO:tasks.workunit.client.0.smithi120.stdout: 1 698442 9.89 MB/sec execute 258 sec latency 14865.634 ms 2024-09-19T09:10:23.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:22 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:10:23.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:22 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:10:23.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:22 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:10:23.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:22 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:10:23.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:22 smithi120 ceph-mon[31150]: pgmap v567: 97 pgs: 97 active+clean; 37 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 2.6 MiB/s wr, 172 op/s 2024-09-19T09:10:23.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:22 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:10:23.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:22 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:10:23.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:22 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:10:23.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:22 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:10:23.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:22 smithi196 ceph-mon[35150]: pgmap v567: 97 pgs: 97 active+clean; 37 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 2.6 MiB/s wr, 172 op/s 2024-09-19T09:10:23.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:22 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:10:23.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:22 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:10:23.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:22 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:10:23.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:22 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:10:23.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:22 smithi153 ceph-mon[35071]: pgmap v567: 97 pgs: 97 active+clean; 37 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 2.6 MiB/s wr, 172 op/s 2024-09-19T09:10:23.675 INFO:tasks.cephfs.filesystem:scrub status for tag:83be93a9-db7d-4d5d-ab41-d0395294c05a - {'path': '/', 'tag': '83be93a9-db7d-4d5d-ab41-d0395294c05a', 'options': 'recursive,force'} 2024-09-19T09:10:24.057 INFO:tasks.workunit.client.0.smithi120.stdout: 1 698442 9.86 MB/sec execute 259 sec latency 15865.757 ms 2024-09-19T09:10:24.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:23 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:10:24.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:23 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:10:24.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:23 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:10:25.058 INFO:tasks.workunit.client.0.smithi120.stdout: 1 698442 9.82 MB/sec execute 260 sec latency 16865.862 ms 2024-09-19T09:10:25.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:24 smithi120 ceph-mon[31150]: pgmap v568: 97 pgs: 97 active+clean; 36 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.5 MiB/s wr, 145 op/s 2024-09-19T09:10:25.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:24 smithi196 ceph-mon[35150]: pgmap v568: 97 pgs: 97 active+clean; 36 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.5 MiB/s wr, 145 op/s 2024-09-19T09:10:25.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:24 smithi153 ceph-mon[35071]: pgmap v568: 97 pgs: 97 active+clean; 36 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.5 MiB/s wr, 145 op/s 2024-09-19T09:10:26.057 INFO:tasks.workunit.client.0.smithi120.stdout: 1 698442 9.78 MB/sec execute 261 sec latency 17865.908 ms 2024-09-19T09:10:27.058 INFO:tasks.workunit.client.0.smithi120.stdout: 1 698442 9.74 MB/sec execute 262 sec latency 18866.004 ms 2024-09-19T09:10:27.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:26 smithi120 ceph-mon[31150]: pgmap v569: 97 pgs: 97 active+clean; 36 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 2.1 MiB/s wr, 139 op/s 2024-09-19T09:10:27.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:26 smithi196 ceph-mon[35150]: pgmap v569: 97 pgs: 97 active+clean; 36 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 2.1 MiB/s wr, 139 op/s 2024-09-19T09:10:27.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:26 smithi153 ceph-mon[35071]: pgmap v569: 97 pgs: 97 active+clean; 36 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 2.1 MiB/s wr, 139 op/s 2024-09-19T09:10:28.058 INFO:tasks.workunit.client.0.smithi120.stdout: 1 698442 9.71 MB/sec execute 263 sec latency 19866.065 ms 2024-09-19T09:10:29.061 INFO:tasks.workunit.client.0.smithi120.stdout: 1 698442 9.67 MB/sec execute 264 sec latency 20866.160 ms 2024-09-19T09:10:29.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:28 smithi120 ceph-mon[31150]: pgmap v570: 97 pgs: 97 active+clean; 33 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 2.2 MiB/s wr, 151 op/s 2024-09-19T09:10:29.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:28 smithi196 ceph-mon[35150]: pgmap v570: 97 pgs: 97 active+clean; 33 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 2.2 MiB/s wr, 151 op/s 2024-09-19T09:10:29.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:28 smithi153 ceph-mon[35071]: pgmap v570: 97 pgs: 97 active+clean; 33 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 2.2 MiB/s wr, 151 op/s 2024-09-19T09:10:30.058 INFO:tasks.workunit.client.0.smithi120.stdout: 1 698442 9.63 MB/sec execute 265 sec latency 21866.272 ms 2024-09-19T09:10:30.152 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:10:30.408 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:10:30.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.910+0000 7f87fde06640 1 -- 172.21.15.120:0/2002378029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87f81036c0 msgr2=0x7f87f810bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:30.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.910+0000 7f87fde06640 1 --2- 172.21.15.120:0/2002378029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87f81036c0 0x7f87f810bbb0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f87ec009900 tx=0x7f87ec02a480 comp rx=0 tx=0).stop 2024-09-19T09:10:30.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.910+0000 7f87fde06640 1 -- 172.21.15.120:0/2002378029 shutdown_connections 2024-09-19T09:10:30.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.910+0000 7f87fde06640 1 --2- 172.21.15.120:0/2002378029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87f810c180 0x7f87f810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:30.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.910+0000 7f87fde06640 1 --2- 172.21.15.120:0/2002378029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87f81036c0 0x7f87f810bbb0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:30.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.910+0000 7f87fde06640 1 --2- 172.21.15.120:0/2002378029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87f8102da0 0x7f87f8103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:30.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.910+0000 7f87fde06640 1 -- 172.21.15.120:0/2002378029 >> 172.21.15.120:0/2002378029 conn(0x7f87f80fc7a0 msgr2=0x7f87f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:30.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.910+0000 7f87fde06640 1 -- 172.21.15.120:0/2002378029 shutdown_connections 2024-09-19T09:10:30.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.910+0000 7f87fde06640 1 -- 172.21.15.120:0/2002378029 wait complete. 2024-09-19T09:10:30.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.910+0000 7f87fde06640 1 Processor -- start 2024-09-19T09:10:30.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.911+0000 7f87fde06640 1 -- start start 2024-09-19T09:10:30.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.911+0000 7f87fde06640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87f8102da0 0x7f87f8198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:30.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.911+0000 7f87fde06640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87f81036c0 0x7f87f8198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:30.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.911+0000 7f87fde06640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87f810c180 0x7f87f819ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:30.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.911+0000 7f87fde06640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f87f8112020 con 0x7f87f8102da0 2024-09-19T09:10:30.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.911+0000 7f87fde06640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f87f8111ea0 con 0x7f87f810c180 2024-09-19T09:10:30.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.911+0000 7f87fde06640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f87f81121a0 con 0x7f87f81036c0 2024-09-19T09:10:30.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.911+0000 7f87f77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87f8102da0 0x7f87f8198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:30.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.911+0000 7f87f77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87f8102da0 0x7f87f8198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57560/0 (socket says 172.21.15.120:57560) 2024-09-19T09:10:30.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.911+0000 7f87f77fe640 1 -- 172.21.15.120:0/912080888 learned_addr learned my addr 172.21.15.120:0/912080888 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:30.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.911+0000 7f87f7fff640 1 --2- 172.21.15.120:0/912080888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87f810c180 0x7f87f819ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:30.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.911+0000 7f87f6ffd640 1 --2- 172.21.15.120:0/912080888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87f81036c0 0x7f87f8198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:30.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.911+0000 7f87f77fe640 1 -- 172.21.15.120:0/912080888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87f81036c0 msgr2=0x7f87f8198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:30.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.911+0000 7f87f77fe640 1 --2- 172.21.15.120:0/912080888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87f81036c0 0x7f87f8198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:30.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.911+0000 7f87f77fe640 1 -- 172.21.15.120:0/912080888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87f810c180 msgr2=0x7f87f819ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:30.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.911+0000 7f87f77fe640 1 --2- 172.21.15.120:0/912080888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87f810c180 0x7f87f819ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:30.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.911+0000 7f87f77fe640 1 -- 172.21.15.120:0/912080888 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f819d5b0 con 0x7f87f8102da0 2024-09-19T09:10:30.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.911+0000 7f87f77fe640 1 --2- 172.21.15.120:0/912080888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87f8102da0 0x7f87f8198600 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f87e800b4a0 tx=0x7f87e800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:30.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.912+0000 7f87f4ff9640 1 -- 172.21.15.120:0/912080888 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e8014070 con 0x7f87f8102da0 2024-09-19T09:10:30.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.912+0000 7f87f4ff9640 1 -- 172.21.15.120:0/912080888 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87e8010af0 con 0x7f87f8102da0 2024-09-19T09:10:30.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.912+0000 7f87f4ff9640 1 -- 172.21.15.120:0/912080888 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e8017d10 con 0x7f87f8102da0 2024-09-19T09:10:30.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.912+0000 7f87fde06640 1 -- 172.21.15.120:0/912080888 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f819d8a0 con 0x7f87f8102da0 2024-09-19T09:10:30.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.912+0000 7f87fde06640 1 -- 172.21.15.120:0/912080888 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f81a5180 con 0x7f87f8102da0 2024-09-19T09:10:30.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.913+0000 7f87fde06640 1 -- 172.21.15.120:0/912080888 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87f8103200 con 0x7f87f8102da0 2024-09-19T09:10:30.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.914+0000 7f87f4ff9640 1 -- 172.21.15.120:0/912080888 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f87e8017070 con 0x7f87f8102da0 2024-09-19T09:10:30.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.914+0000 7f87f4ff9640 1 --2- 172.21.15.120:0/912080888 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f87e00a3240 0x7f87e00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:30.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.914+0000 7f87f6ffd640 1 --2- 172.21.15.120:0/912080888 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f87e00a3240 0x7f87e00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:30.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.915+0000 7f87f4ff9640 1 -- 172.21.15.120:0/912080888 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f87e80cec30 con 0x7f87f8102da0 2024-09-19T09:10:30.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.915+0000 7f87f6ffd640 1 --2- 172.21.15.120:0/912080888 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f87e00a3240 0x7f87e00a5700 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f87ec000c00 tx=0x7f87ec0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:30.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:30.919+0000 7f87f4ff9640 1 -- 172.21.15.120:0/912080888 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f87e809b300 con 0x7f87f8102da0 2024-09-19T09:10:31.058 INFO:tasks.workunit.client.0.smithi120.stdout: 1 698442 9.60 MB/sec execute 266 sec latency 22866.340 ms 2024-09-19T09:10:31.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:30 smithi120 ceph-mon[31150]: pgmap v571: 97 pgs: 97 active+clean; 34 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.1 MiB/s wr, 141 op/s 2024-09-19T09:10:31.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:31.129+0000 7f87fde06640 1 -- 172.21.15.120:0/912080888 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87f81071c0 con 0x7f87f8102da0 2024-09-19T09:10:31.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:31.129+0000 7f87f4ff9640 1 -- 172.21.15.120:0/912080888 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11985 (secure 0 0 0) 0x7f87e809f600 con 0x7f87f8102da0 2024-09-19T09:10:31.130 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:10:31.130 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":30,"btime":"2024-09-19T09:07:14:812065+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:07:14.812046+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26351":{"gid":26351,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/364941779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":364941779},{"type":"v1","addr":"172.21.15.153:6835","nonce":364941779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:10:31.131 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 30 2024-09-19T09:10:31.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:31.131+0000 7f87fde06640 1 -- 172.21.15.120:0/912080888 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f87e00a3240 msgr2=0x7f87e00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:31.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:31.131+0000 7f87fde06640 1 --2- 172.21.15.120:0/912080888 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f87e00a3240 0x7f87e00a5700 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f87ec000c00 tx=0x7f87ec0058d0 comp rx=0 tx=0).stop 2024-09-19T09:10:31.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:31.131+0000 7f87fde06640 1 -- 172.21.15.120:0/912080888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87f8102da0 msgr2=0x7f87f8198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:31.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:31.131+0000 7f87fde06640 1 --2- 172.21.15.120:0/912080888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87f8102da0 0x7f87f8198600 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f87e800b4a0 tx=0x7f87e800b960 comp rx=0 tx=0).stop 2024-09-19T09:10:31.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:31.132+0000 7f87fde06640 1 -- 172.21.15.120:0/912080888 shutdown_connections 2024-09-19T09:10:31.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:31.132+0000 7f87fde06640 1 --2- 172.21.15.120:0/912080888 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f87e00a3240 0x7f87e00a5700 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:31.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:31.132+0000 7f87fde06640 1 --2- 172.21.15.120:0/912080888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87f810c180 0x7f87f819ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:31.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:31.132+0000 7f87fde06640 1 --2- 172.21.15.120:0/912080888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87f81036c0 0x7f87f8198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:31.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:31.132+0000 7f87fde06640 1 --2- 172.21.15.120:0/912080888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87f8102da0 0x7f87f8198600 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:31.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:31.132+0000 7f87fde06640 1 -- 172.21.15.120:0/912080888 >> 172.21.15.120:0/912080888 conn(0x7f87f80fc7a0 msgr2=0x7f87f80fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:31.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:31.132+0000 7f87fde06640 1 -- 172.21.15.120:0/912080888 shutdown_connections 2024-09-19T09:10:31.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:31.132+0000 7f87fde06640 1 -- 172.21.15.120:0/912080888 wait complete. 2024-09-19T09:10:31.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:30 smithi196 ceph-mon[35150]: pgmap v571: 97 pgs: 97 active+clean; 34 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.1 MiB/s wr, 141 op/s 2024-09-19T09:10:31.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:30 smithi153 ceph-mon[35071]: pgmap v571: 97 pgs: 97 active+clean; 34 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.1 MiB/s wr, 141 op/s 2024-09-19T09:10:32.017 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db --set-id aeb2f3f6 --release --await' 2024-09-19T09:10:32.018 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db --set-id aeb2f3f6 --release --await 2024-09-19T09:10:32.058 INFO:tasks.workunit.client.0.smithi120.stdout: 1 698442 9.56 MB/sec execute 267 sec latency 23866.372 ms 2024-09-19T09:10:32.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:31 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/912080888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:10:32.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:31 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/912080888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:10:32.264 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:10:32.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:31 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/912080888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:10:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.446+0000 7f72fa4f1640 1 -- 172.21.15.120:0/3588307210 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72f4101c40 msgr2=0x7f72f41020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.446+0000 7f72fa4f1640 1 --2- 172.21.15.120:0/3588307210 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72f4101c40 0x7f72f41020c0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f72e8009900 tx=0x7f72e802a410 comp rx=0 tx=0).stop 2024-09-19T09:10:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.446+0000 7f72fa4f1640 1 -- 172.21.15.120:0/3588307210 shutdown_connections 2024-09-19T09:10:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.446+0000 7f72fa4f1640 1 --2- 172.21.15.120:0/3588307210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72f4102600 0x7f72f4108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.446+0000 7f72fa4f1640 1 --2- 172.21.15.120:0/3588307210 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72f4101c40 0x7f72f41020c0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.446+0000 7f72fa4f1640 1 --2- 172.21.15.120:0/3588307210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72f4100a50 0x7f72f4100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.446+0000 7f72fa4f1640 1 -- 172.21.15.120:0/3588307210 >> 172.21.15.120:0/3588307210 conn(0x7f72f40fc1b0 msgr2=0x7f72f40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.446+0000 7f72fa4f1640 1 -- 172.21.15.120:0/3588307210 shutdown_connections 2024-09-19T09:10:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.446+0000 7f72fa4f1640 1 -- 172.21.15.120:0/3588307210 wait complete. 2024-09-19T09:10:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.446+0000 7f72fa4f1640 1 Processor -- start 2024-09-19T09:10:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.447+0000 7f72fa4f1640 1 -- start start 2024-09-19T09:10:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.447+0000 7f72fa4f1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72f4100a50 0x7f72f4196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.447+0000 7f72fa4f1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72f4101c40 0x7f72f41973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.447+0000 7f72fa4f1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72f4102600 0x7f72f419e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.447+0000 7f72fa4f1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f72f410c7d0 con 0x7f72f4100a50 2024-09-19T09:10:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.447+0000 7f72fa4f1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f72f410c650 con 0x7f72f4102600 2024-09-19T09:10:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.447+0000 7f72fa4f1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f72f410c950 con 0x7f72f4101c40 2024-09-19T09:10:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.447+0000 7f72f3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72f4100a50 0x7f72f4196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.447+0000 7f72f3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72f4100a50 0x7f72f4196eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40118/0 (socket says 172.21.15.120:40118) 2024-09-19T09:10:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.447+0000 7f72f3fff640 1 -- 172.21.15.120:0/28301310 learned_addr learned my addr 172.21.15.120:0/28301310 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.447+0000 7f72f8a8e640 1 --2- 172.21.15.120:0/28301310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72f4102600 0x7f72f419e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.447+0000 7f72f3fff640 1 -- 172.21.15.120:0/28301310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72f4101c40 msgr2=0x7f72f41973f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.447+0000 7f72f3fff640 1 --2- 172.21.15.120:0/28301310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72f4101c40 0x7f72f41973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.447+0000 7f72f3fff640 1 -- 172.21.15.120:0/28301310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72f4102600 msgr2=0x7f72f419e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.447+0000 7f72f3fff640 1 --2- 172.21.15.120:0/28301310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72f4102600 0x7f72f419e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.447+0000 7f72f3fff640 1 -- 172.21.15.120:0/28301310 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72f419ead0 con 0x7f72f4100a50 2024-09-19T09:10:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.447+0000 7f72f8a8e640 1 --2- 172.21.15.120:0/28301310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72f4102600 0x7f72f419e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:10:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.447+0000 7f72f3fff640 1 --2- 172.21.15.120:0/28301310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72f4100a50 0x7f72f4196eb0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f72e400b470 tx=0x7f72e400b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.448+0000 7f72f17fa640 1 -- 172.21.15.120:0/28301310 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e4015020 con 0x7f72f4100a50 2024-09-19T09:10:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.448+0000 7f72f17fa640 1 -- 172.21.15.120:0/28301310 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72e4011d70 con 0x7f72f4100a50 2024-09-19T09:10:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.448+0000 7f72fa4f1640 1 -- 172.21.15.120:0/28301310 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72f419edc0 con 0x7f72f4100a50 2024-09-19T09:10:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.448+0000 7f72f17fa640 1 -- 172.21.15.120:0/28301310 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e400f340 con 0x7f72f4100a50 2024-09-19T09:10:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.448+0000 7f72fa4f1640 1 -- 172.21.15.120:0/28301310 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72f419f2b0 con 0x7f72f4100a50 2024-09-19T09:10:32.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.450+0000 7f72f17fa640 1 -- 172.21.15.120:0/28301310 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f72e4018020 con 0x7f72f4100a50 2024-09-19T09:10:32.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.450+0000 7f72fd290740 1 Processor -- start 2024-09-19T09:10:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.450+0000 7f72fd290740 1 -- start start 2024-09-19T09:10:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.450+0000 7f72fd290740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d65cacbb0 0x563d65cacfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.450+0000 7f72fd290740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d65cad4f0 0x563d65cb1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.450+0000 7f72fd290740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d65cb2580 0x563d65cb4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.450+0000 7f72fd290740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563d65c0e2f0 con 0x563d65cad4f0 2024-09-19T09:10:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.450+0000 7f72fd290740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563d65bb6270 con 0x563d65cacbb0 2024-09-19T09:10:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.450+0000 7f72fd290740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563d65c0a2b0 con 0x563d65cb2580 2024-09-19T09:10:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.450+0000 7f72f3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d65cad4f0 0x563d65cb1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.450+0000 7f72f3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d65cad4f0 0x563d65cb1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40122/0 (socket says 172.21.15.120:40122) 2024-09-19T09:10:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.450+0000 7f72f3fff640 1 -- 172.21.15.120:0/3208091577 learned_addr learned my addr 172.21.15.120:0/3208091577 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.450+0000 7f72f17fa640 1 --2- 172.21.15.120:0/28301310 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f72dc0a32f0 0x7f72dc0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.451+0000 7f72f37fe640 1 --2- 172.21.15.120:0/3208091577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d65cb2580 0x563d65cb4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.451+0000 7f72f8a8e640 1 --2- 172.21.15.120:0/3208091577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d65cacbb0 0x563d65cacfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.451+0000 7f72f8a8e640 1 --2- 172.21.15.120:0/28301310 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f72dc0a32f0 0x7f72dc0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.451+0000 7f72f17fa640 1 -- 172.21.15.120:0/28301310 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f72e40cf4a0 con 0x7f72f4100a50 2024-09-19T09:10:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.451+0000 7f72f3fff640 1 -- 172.21.15.120:0/3208091577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d65cb2580 msgr2=0x563d65cb4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.451+0000 7f72f3fff640 1 --2- 172.21.15.120:0/3208091577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d65cb2580 0x563d65cb4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.451+0000 7f72f3fff640 1 -- 172.21.15.120:0/3208091577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d65cacbb0 msgr2=0x563d65cacfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.451+0000 7f72f3fff640 1 --2- 172.21.15.120:0/3208091577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d65cacbb0 0x563d65cacfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.451+0000 7f72f3fff640 1 -- 172.21.15.120:0/3208091577 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d65bb5340 con 0x563d65cad4f0 2024-09-19T09:10:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.451+0000 7f72f3fff640 1 --2- 172.21.15.120:0/3208091577 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d65cad4f0 0x563d65cb1fb0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f72e40ccfa0 tx=0x7f72e40107f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f04328fdd2111e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.451+0000 7f72f8a8e640 1 --2- 172.21.15.120:0/28301310 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f72dc0a32f0 0x7f72dc0a57b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00e870 tx=0x7f72ec008220 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.451+0000 7f72da7fc640 1 -- 172.21.15.120:0/3208091577 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e409b420 con 0x563d65cad4f0 2024-09-19T09:10:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.451+0000 7f72da7fc640 1 -- 172.21.15.120:0/3208091577 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72e40d2e50 con 0x563d65cad4f0 2024-09-19T09:10:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.451+0000 7f72da7fc640 1 -- 172.21.15.120:0/3208091577 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e40cd0b0 con 0x563d65cad4f0 2024-09-19T09:10:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.452+0000 7f72fd290740 1 -- 172.21.15.120:0/3208091577 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d65cad4f0 msgr2=0x563d65cb1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.452+0000 7f72fd290740 1 --2- 172.21.15.120:0/3208091577 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d65cad4f0 0x563d65cb1fb0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f72e40ccfa0 tx=0x7f72e40107f0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.452+0000 7f72fd290740 1 -- 172.21.15.120:0/3208091577 shutdown_connections 2024-09-19T09:10:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.452+0000 7f72fd290740 1 --2- 172.21.15.120:0/3208091577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d65cb2580 0x563d65cb4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.452+0000 7f72fd290740 1 --2- 172.21.15.120:0/3208091577 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d65cad4f0 0x563d65cb1fb0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.452+0000 7f72fd290740 1 --2- 172.21.15.120:0/3208091577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d65cacbb0 0x563d65cacfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.452+0000 7f72fd290740 1 -- 172.21.15.120:0/3208091577 >> 172.21.15.120:0/3208091577 conn(0x563d65ca6010 msgr2=0x563d65ca6410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.452+0000 7f72fd290740 1 -- 172.21.15.120:0/3208091577 shutdown_connections 2024-09-19T09:10:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.452+0000 7f72fd290740 1 -- 172.21.15.120:0/3208091577 wait complete. 2024-09-19T09:10:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.452+0000 7f72fd290740 1 -- start start 2024-09-19T09:10:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.452+0000 7f72fd290740 1 Processor -- start 2024-09-19T09:10:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.453+0000 7f72fd290740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d65cacbb0 0x563d65cb6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.453+0000 7f72fd290740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d65cad4f0 0x563d65cb7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.453+0000 7f72fd290740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d65cb2580 0x563d65cbe290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.453+0000 7f72fd290740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563d65cb5490 con 0x563d65cb2580 2024-09-19T09:10:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.453+0000 7f72fd290740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563d65b97290 con 0x563d65cad4f0 2024-09-19T09:10:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.453+0000 7f72fd290740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563d65cb5610 con 0x563d65cacbb0 2024-09-19T09:10:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.453+0000 7f72fd290740 10 client.0 _ll_register_callbacks cb 0x563d65bc8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.453+0000 7f72f8a8e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d65cb2580 0x563d65cbe290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.453+0000 7f72f8a8e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d65cb2580 0x563d65cbe290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40128/0 (socket says 172.21.15.120:40128) 2024-09-19T09:10:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.453+0000 7f72f8a8e640 1 -- 172.21.15.120:0/818234209 learned_addr learned my addr 172.21.15.120:0/818234209 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.453+0000 7f72f37fe640 1 --2- 172.21.15.120:0/818234209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d65cad4f0 0x563d65cb7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.453+0000 7f72f3fff640 1 --2- 172.21.15.120:0/818234209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d65cacbb0 0x563d65cb6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.453+0000 7f72f8a8e640 1 -- 172.21.15.120:0/818234209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d65cacbb0 msgr2=0x563d65cb6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.453+0000 7f72f8a8e640 1 --2- 172.21.15.120:0/818234209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d65cacbb0 0x563d65cb6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.453+0000 7f72f8a8e640 1 -- 172.21.15.120:0/818234209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d65cad4f0 msgr2=0x563d65cb7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.453+0000 7f72f8a8e640 1 --2- 172.21.15.120:0/818234209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d65cad4f0 0x563d65cb7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.453+0000 7f72f8a8e640 1 -- 172.21.15.120:0/818234209 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d65921210 con 0x563d65cb2580 2024-09-19T09:10:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.453+0000 7f72f8a8e640 1 --2- 172.21.15.120:0/818234209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d65cb2580 0x563d65cbe290 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00bcd0 tx=0x7f72ec004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3942ed36ccbc74ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.453+0000 7f72d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.453+0000 7f72d37fe640 1 -- 172.21.15.120:0/818234209 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec01b650 con 0x563d65cb2580 2024-09-19T09:10:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.453+0000 7f72d37fe640 1 -- 172.21.15.120:0/818234209 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f72ec01b7f0 con 0x563d65cb2580 2024-09-19T09:10:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.453+0000 7f72fd290740 1 -- 172.21.15.120:0/818234209 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72e40d2e50 con 0x563d65cb2580 2024-09-19T09:10:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.454+0000 7f72d37fe640 1 -- 172.21.15.120:0/818234209 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f72ec01b9f0 con 0x563d65cb2580 2024-09-19T09:10:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.454+0000 7f72fd290740 10 client.18105 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.454+0000 7f72fd290740 1 -- 172.21.15.120:0/818234209 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563d659ed620 con 0x563d65cb2580 2024-09-19T09:10:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.454+0000 7f72d37fe640 1 -- 172.21.15.120:0/818234209 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72e40d2e50 con 0x563d65cb2580 2024-09-19T09:10:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.454+0000 7f72d37fe640 1 -- 172.21.15.120:0/818234209 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f72ec018040 con 0x563d65cb2580 2024-09-19T09:10:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.454+0000 7f72fd290740 10 client.18105 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.454+0000 7f72fd290740 10 client.18105 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:10:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.454+0000 7f72fd290740 20 client.18105 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.454+0000 7f72fd290740 1 --2- 172.21.15.120:0/818234209 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x563d65cc2a50 0x563d65cc4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.454+0000 7f72fd290740 4 client.18105 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:10:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.454+0000 7f72fd290740 1 -- 172.21.15.120:0/818234209 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d65c1ba20 con 0x563d65cc2a50 2024-09-19T09:10:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.454+0000 7f72f3fff640 1 --2- 172.21.15.120:0/818234209 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x563d65cc2a50 0x563d65cc4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.455+0000 7f72d37fe640 10 client.18105 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.455+0000 7f72f3fff640 1 --2- 172.21.15.120:0/818234209 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x563d65cc2a50 0x563d65cc4e80 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68fca86523361021 server_cookie=8f1037de27491e76 in_seq=0 out_seq=0 2024-09-19T09:10:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.455+0000 7f72d37fe640 1 -- 172.21.15.120:0/818234209 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d65c1ba20 con 0x563d65cc2a50 2024-09-19T09:10:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.455+0000 7f72d37fe640 10 client.18105 handle_command_reply: tid=0 2024-09-19T09:10:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.456+0000 7f72fd290740 1 client.18105 shutdown 2024-09-19T09:10:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.456+0000 7f72fd290740 1 -- 172.21.15.120:0/818234209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d65cb2580 msgr2=0x563d65cbe290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.456+0000 7f72fd290740 1 --2- 172.21.15.120:0/818234209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d65cb2580 0x563d65cbe290 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00bcd0 tx=0x7f72ec004b50 comp rx=0 tx=0).stop 2024-09-19T09:10:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.456+0000 7f72fd290740 1 -- 172.21.15.120:0/818234209 shutdown_connections 2024-09-19T09:10:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.456+0000 7f72fd290740 1 --2- 172.21.15.120:0/818234209 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x563d65cc2a50 0x563d65cc4e80 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.456+0000 7f72fd290740 1 --2- 172.21.15.120:0/818234209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d65cb2580 0x563d65cbe290 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.456+0000 7f72fd290740 1 --2- 172.21.15.120:0/818234209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d65cad4f0 0x563d65cb7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.456+0000 7f72d37fe640 0 client.18105 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.456+0000 7f72fd290740 1 --2- 172.21.15.120:0/818234209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d65cacbb0 0x563d65cb6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.456+0000 7f72fd290740 1 -- 172.21.15.120:0/818234209 >> 172.21.15.120:0/818234209 conn(0x563d65ca6010 msgr2=0x563d65ca7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.456+0000 7f72fd290740 1 -- 172.21.15.120:0/818234209 shutdown_connections 2024-09-19T09:10:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.456+0000 7f72fd290740 1 -- 172.21.15.120:0/818234209 wait complete. 2024-09-19T09:10:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.456+0000 7f72fd290740 20 client.18105 trim_cache size 0 max 16384 2024-09-19T09:10:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.470+0000 7f72fd290740 1 Processor -- start 2024-09-19T09:10:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.470+0000 7f72fd290740 1 -- start start 2024-09-19T09:10:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.470+0000 7f72fd290740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d65cacbb0 0x563d65cacf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.470+0000 7f72fd290740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d65cad4f0 0x563d65cbe250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.470+0000 7f72fd290740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d65cb2580 0x563d65ca63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.470+0000 7f72fd290740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f72f41a6fe0 con 0x563d65cb2580 2024-09-19T09:10:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.470+0000 7f72fd290740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f72f41a8b20 con 0x563d65cad4f0 2024-09-19T09:10:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.470+0000 7f72fd290740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f72f41a76b0 con 0x563d65cacbb0 2024-09-19T09:10:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.470+0000 7f72f8a8e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d65cb2580 0x563d65ca63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.470+0000 7f72f8a8e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d65cb2580 0x563d65ca63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40130/0 (socket says 172.21.15.120:40130) 2024-09-19T09:10:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.470+0000 7f72f8a8e640 1 -- 172.21.15.120:0/3936779701 learned_addr learned my addr 172.21.15.120:0/3936779701 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.470+0000 7f72f8a8e640 1 -- 172.21.15.120:0/3936779701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d65cacbb0 msgr2=0x563d65cacf90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:10:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.470+0000 7f72f8a8e640 1 --2- 172.21.15.120:0/3936779701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d65cacbb0 0x563d65cacf90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.470+0000 7f72f8a8e640 1 -- 172.21.15.120:0/3936779701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d65cad4f0 msgr2=0x563d65cbe250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:10:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.470+0000 7f72f8a8e640 1 --2- 172.21.15.120:0/3936779701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d65cad4f0 0x563d65cbe250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.470+0000 7f72f8a8e640 1 -- 172.21.15.120:0/3936779701 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d65ca5e30 con 0x563d65cb2580 2024-09-19T09:10:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.471+0000 7f72f8a8e640 1 --2- 172.21.15.120:0/3936779701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d65cb2580 0x563d65ca63f0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f72ec000f80 tx=0x7f72ec00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7624b8a3711acb31 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.471+0000 7f72d9ffb640 1 -- 172.21.15.120:0/3936779701 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec036020 con 0x563d65cb2580 2024-09-19T09:10:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.471+0000 7f72d9ffb640 1 -- 172.21.15.120:0/3936779701 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72ec00dc40 con 0x563d65cb2580 2024-09-19T09:10:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.471+0000 7f72d9ffb640 1 -- 172.21.15.120:0/3936779701 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec030460 con 0x563d65cb2580 2024-09-19T09:10:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.471+0000 7f72fd290740 1 -- 172.21.15.120:0/3936779701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d65cb2580 msgr2=0x563d65ca63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.471+0000 7f72fd290740 1 --2- 172.21.15.120:0/3936779701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d65cb2580 0x563d65ca63f0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f72ec000f80 tx=0x7f72ec00bfd0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.471+0000 7f72fd290740 1 -- 172.21.15.120:0/3936779701 shutdown_connections 2024-09-19T09:10:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.471+0000 7f72fd290740 1 --2- 172.21.15.120:0/3936779701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d65cb2580 0x563d65ca63f0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.471+0000 7f72fd290740 1 --2- 172.21.15.120:0/3936779701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d65cad4f0 0x563d65cbe250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.471+0000 7f72fd290740 1 --2- 172.21.15.120:0/3936779701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d65cacbb0 0x563d65cacf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.471+0000 7f72fd290740 1 -- 172.21.15.120:0/3936779701 >> 172.21.15.120:0/3936779701 conn(0x563d65ca6010 msgr2=0x563d65cb6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.471+0000 7f72fd290740 1 -- 172.21.15.120:0/3936779701 shutdown_connections 2024-09-19T09:10:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.471+0000 7f72fd290740 1 -- 172.21.15.120:0/3936779701 wait complete. 2024-09-19T09:10:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.472+0000 7f72fd290740 1 -- start start 2024-09-19T09:10:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.472+0000 7f72fd290740 1 Processor -- start 2024-09-19T09:10:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.472+0000 7f72fd290740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d65cacbb0 0x563d65cbb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.472+0000 7f72fd290740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d65cad4f0 0x563d65cbb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.472+0000 7f72fd290740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d65cb2580 0x563d65d2b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.472+0000 7f72fd290740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f72f41a8450 con 0x563d65cb2580 2024-09-19T09:10:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.472+0000 7f72fd290740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f72f41a7d80 con 0x563d65cacbb0 2024-09-19T09:10:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.472+0000 7f72fd290740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f72f41a4fa0 con 0x563d65cad4f0 2024-09-19T09:10:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.472+0000 7f72fd290740 10 client.0 _ll_register_callbacks cb 0x7f72ec00e4b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.472+0000 7f72f8a8e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d65cb2580 0x563d65d2b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.472+0000 7f72f8a8e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d65cb2580 0x563d65d2b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40146/0 (socket says 172.21.15.120:40146) 2024-09-19T09:10:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.472+0000 7f72f8a8e640 1 -- 172.21.15.120:0/205587893 learned_addr learned my addr 172.21.15.120:0/205587893 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.472+0000 7f72f8a8e640 1 -- 172.21.15.120:0/205587893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d65cad4f0 msgr2=0x563d65cbb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.472+0000 7f72f8a8e640 1 --2- 172.21.15.120:0/205587893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d65cad4f0 0x563d65cbb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.472+0000 7f72f8a8e640 1 -- 172.21.15.120:0/205587893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d65cacbb0 msgr2=0x563d65cbb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.472+0000 7f72f8a8e640 1 --2- 172.21.15.120:0/205587893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d65cacbb0 0x563d65cbb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.472+0000 7f72f8a8e640 1 -- 172.21.15.120:0/205587893 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d65d2bd60 con 0x563d65cb2580 2024-09-19T09:10:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.472+0000 7f72f8a8e640 1 --2- 172.21.15.120:0/205587893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d65cb2580 0x563d65d2b750 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f72ec004b90 tx=0x7f72ec025510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7aff9b505a98d1c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.472+0000 7f72d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.473+0000 7f72d37fe640 1 -- 172.21.15.120:0/205587893 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec040060 con 0x563d65cb2580 2024-09-19T09:10:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.473+0000 7f72fd290740 1 -- 172.21.15.120:0/205587893 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72ec00dc40 con 0x563d65cb2580 2024-09-19T09:10:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.473+0000 7f72d37fe640 1 -- 172.21.15.120:0/205587893 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f72ec01b450 con 0x563d65cb2580 2024-09-19T09:10:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.473+0000 7f72d37fe640 1 -- 172.21.15.120:0/205587893 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f72ec036070 con 0x563d65cb2580 2024-09-19T09:10:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.473+0000 7f72fd290740 10 client.18111 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.473+0000 7f72fd290740 1 -- 172.21.15.120:0/205587893 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563d65d2d070 con 0x563d65cb2580 2024-09-19T09:10:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.473+0000 7f72d37fe640 1 -- 172.21.15.120:0/205587893 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72ec00dc40 con 0x563d65cb2580 2024-09-19T09:10:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.473+0000 7f72d37fe640 1 -- 172.21.15.120:0/205587893 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f72ec045070 con 0x563d65cb2580 2024-09-19T09:10:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.473+0000 7f72fd290740 10 client.18111 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.473+0000 7f72fd290740 10 client.18111 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:10:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.473+0000 7f72fd290740 20 client.18111 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.473+0000 7f72fd290740 1 --2- 172.21.15.120:0/205587893 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x563d65d2d970 0x563d65d2fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.473+0000 7f72fd290740 4 client.18111 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:10:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.473+0000 7f72fd290740 1 -- 172.21.15.120:0/205587893 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563d65c15d70 con 0x563d65d2d970 2024-09-19T09:10:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.473+0000 7f72f3fff640 1 --2- 172.21.15.120:0/205587893 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x563d65d2d970 0x563d65d2fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.474+0000 7f72d37fe640 10 client.18111 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.474+0000 7f72f3fff640 1 --2- 172.21.15.120:0/205587893 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x563d65d2d970 0x563d65d2fd90 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80a0ef0388d630ff server_cookie=629042004f7ac158 in_seq=0 out_seq=0 2024-09-19T09:10:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.558+0000 7f72d37fe640 1 -- 172.21.15.120:0/205587893 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563d65c15d70 con 0x563d65d2d970 2024-09-19T09:10:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.558+0000 7f72d37fe640 10 client.18111 handle_command_reply: tid=0 2024-09-19T09:10:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.558+0000 7f72fd290740 1 client.18111 shutdown 2024-09-19T09:10:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.559+0000 7f72fd290740 1 -- 172.21.15.120:0/205587893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d65cb2580 msgr2=0x563d65d2b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.559+0000 7f72fd290740 1 --2- 172.21.15.120:0/205587893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d65cb2580 0x563d65d2b750 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f72ec004b90 tx=0x7f72ec025510 comp rx=0 tx=0).stop 2024-09-19T09:10:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.559+0000 7f72fd290740 1 -- 172.21.15.120:0/205587893 shutdown_connections 2024-09-19T09:10:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.559+0000 7f72fd290740 1 --2- 172.21.15.120:0/205587893 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x563d65d2d970 0x563d65d2fd90 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.559+0000 7f72fd290740 1 --2- 172.21.15.120:0/205587893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d65cb2580 0x563d65d2b750 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.559+0000 7f72fd290740 1 --2- 172.21.15.120:0/205587893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d65cad4f0 0x563d65cbb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.559+0000 7f72fd290740 1 --2- 172.21.15.120:0/205587893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d65cacbb0 0x563d65cbb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.559+0000 7f72d37fe640 0 client.18111 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.559+0000 7f72fd290740 1 -- 172.21.15.120:0/205587893 >> 172.21.15.120:0/205587893 conn(0x563d65ca6010 msgr2=0x563d65ca49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.559+0000 7f72fd290740 1 -- 172.21.15.120:0/205587893 shutdown_connections 2024-09-19T09:10:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.559+0000 7f72fd290740 1 -- 172.21.15.120:0/205587893 wait complete. 2024-09-19T09:10:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.559+0000 7f72fd290740 20 client.18111 trim_cache size 0 max 16384 2024-09-19T09:10:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:10:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:10:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:10:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:10:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.559+0000 7f72fa4f1640 1 -- 172.21.15.120:0/28301310 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f72dc0a32f0 msgr2=0x7f72dc0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.559+0000 7f72fa4f1640 1 --2- 172.21.15.120:0/28301310 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f72dc0a32f0 0x7f72dc0a57b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00e870 tx=0x7f72ec008220 comp rx=0 tx=0).stop 2024-09-19T09:10:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.559+0000 7f72fa4f1640 1 -- 172.21.15.120:0/28301310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72f4100a50 msgr2=0x7f72f4196eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.559+0000 7f72fa4f1640 1 --2- 172.21.15.120:0/28301310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72f4100a50 0x7f72f4196eb0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f72e400b470 tx=0x7f72e400b930 comp rx=0 tx=0).stop 2024-09-19T09:10:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.560+0000 7f72fa4f1640 1 -- 172.21.15.120:0/28301310 shutdown_connections 2024-09-19T09:10:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.560+0000 7f72fa4f1640 1 --2- 172.21.15.120:0/28301310 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f72dc0a32f0 0x7f72dc0a57b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.560+0000 7f72fa4f1640 1 --2- 172.21.15.120:0/28301310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72f4102600 0x7f72f419e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.560+0000 7f72fa4f1640 1 --2- 172.21.15.120:0/28301310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72f4101c40 0x7f72f41973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.560+0000 7f72fa4f1640 1 --2- 172.21.15.120:0/28301310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72f4100a50 0x7f72f4196eb0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.560+0000 7f72fa4f1640 1 -- 172.21.15.120:0/28301310 >> 172.21.15.120:0/28301310 conn(0x7f72f40fc1b0 msgr2=0x7f72f41050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.560+0000 7f72fa4f1640 1 -- 172.21.15.120:0/28301310 shutdown_connections 2024-09-19T09:10:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:32.560+0000 7f72fa4f1640 1 -- 172.21.15.120:0/28301310 wait complete. 2024-09-19T09:10:32.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.863+0000 7fe24f2e5640 1 -- 172.21.15.120:0/4225221416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe248108290 msgr2=0x7fe24810ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:32.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.863+0000 7fe24f2e5640 1 --2- 172.21.15.120:0/4225221416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe248108290 0x7fe24810ec30 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fe240009930 tx=0x7fe24002a420 comp rx=0 tx=0).stop 2024-09-19T09:10:32.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.863+0000 7fe24f2e5640 1 -- 172.21.15.120:0/4225221416 shutdown_connections 2024-09-19T09:10:32.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.863+0000 7fe24f2e5640 1 --2- 172.21.15.120:0/4225221416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe248108290 0x7fe24810ec30 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.863+0000 7fe24f2e5640 1 --2- 172.21.15.120:0/4225221416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe248103740 0x7fe248107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.863+0000 7fe24f2e5640 1 --2- 172.21.15.120:0/4225221416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe248102d90 0x7fe248103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.863+0000 7fe24f2e5640 1 -- 172.21.15.120:0/4225221416 >> 172.21.15.120:0/4225221416 conn(0x7fe2480fc7d0 msgr2=0x7fe2480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:32.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.863+0000 7fe24f2e5640 1 -- 172.21.15.120:0/4225221416 shutdown_connections 2024-09-19T09:10:32.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.863+0000 7fe24f2e5640 1 -- 172.21.15.120:0/4225221416 wait complete. 2024-09-19T09:10:32.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.863+0000 7fe24f2e5640 1 Processor -- start 2024-09-19T09:10:32.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.863+0000 7fe24f2e5640 1 -- start start 2024-09-19T09:10:32.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.864+0000 7fe24f2e5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe248102d90 0x7fe248198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.864+0000 7fe24f2e5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe248103740 0x7fe248198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.864+0000 7fe24f2e5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe248108290 0x7fe24819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.864+0000 7fe24f2e5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe248112090 con 0x7fe248108290 2024-09-19T09:10:32.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.864+0000 7fe24f2e5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe248111f10 con 0x7fe248103740 2024-09-19T09:10:32.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.864+0000 7fe24f2e5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe248112210 con 0x7fe248102d90 2024-09-19T09:10:32.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.864+0000 7fe24d882640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe248108290 0x7fe24819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:32.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.864+0000 7fe24d882640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe248108290 0x7fe24819ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40168/0 (socket says 172.21.15.120:40168) 2024-09-19T09:10:32.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.864+0000 7fe24d882640 1 -- 172.21.15.120:0/2999206166 learned_addr learned my addr 172.21.15.120:0/2999206166 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:32.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.864+0000 7fe24c880640 1 --2- 172.21.15.120:0/2999206166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe248103740 0x7fe248198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:32.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.864+0000 7fe24d882640 1 -- 172.21.15.120:0/2999206166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe248102d90 msgr2=0x7fe248198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:32.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.864+0000 7fe24d882640 1 --2- 172.21.15.120:0/2999206166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe248102d90 0x7fe248198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.864+0000 7fe24d882640 1 -- 172.21.15.120:0/2999206166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe248103740 msgr2=0x7fe248198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:32.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.864+0000 7fe24d882640 1 --2- 172.21.15.120:0/2999206166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe248103740 0x7fe248198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.864+0000 7fe24d882640 1 -- 172.21.15.120:0/2999206166 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24819d4e0 con 0x7fe248108290 2024-09-19T09:10:32.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.864+0000 7fe24d882640 1 --2- 172.21.15.120:0/2999206166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe248108290 0x7fe24819ce00 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fe240005ab0 tx=0x7fe2400042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:32.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.864+0000 7fe2467fc640 1 -- 172.21.15.120:0/2999206166 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24003f030 con 0x7fe248108290 2024-09-19T09:10:32.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.864+0000 7fe24f2e5640 1 -- 172.21.15.120:0/2999206166 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe24819d770 con 0x7fe248108290 2024-09-19T09:10:32.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.864+0000 7fe2467fc640 1 -- 172.21.15.120:0/2999206166 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe240003ce0 con 0x7fe248108290 2024-09-19T09:10:32.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.864+0000 7fe2467fc640 1 -- 172.21.15.120:0/2999206166 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240035070 con 0x7fe248108290 2024-09-19T09:10:32.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.864+0000 7fe24f2e5640 1 -- 172.21.15.120:0/2999206166 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2481a50b0 con 0x7fe248108290 2024-09-19T09:10:32.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.867+0000 7fe2467fc640 1 -- 172.21.15.120:0/2999206166 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe240042030 con 0x7fe248108290 2024-09-19T09:10:32.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.867+0000 7fe2516be740 1 Processor -- start 2024-09-19T09:10:32.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.867+0000 7fe2516be740 1 -- start start 2024-09-19T09:10:32.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.867+0000 7fe2516be740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e04838060 0x563e04838440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.867+0000 7fe2516be740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e04838980 0x563e04842d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.867+0000 7fe2516be740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e04843240 0x563e04845630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.867+0000 7fe2516be740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563e0479d620 con 0x563e04843240 2024-09-19T09:10:32.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.867+0000 7fe2516be740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563e04744e50 con 0x563e04838980 2024-09-19T09:10:32.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.867+0000 7fe2516be740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563e047995e0 con 0x563e04838060 2024-09-19T09:10:32.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.867+0000 7fe2467fc640 1 --2- 172.21.15.120:0/2999206166 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe2300a3240 0x7fe2300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.867+0000 7fe24c880640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e04843240 0x563e04845630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:32.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.867+0000 7fe24c880640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e04843240 0x563e04845630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40170/0 (socket says 172.21.15.120:40170) 2024-09-19T09:10:32.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.867+0000 7fe24c880640 1 -- 172.21.15.120:0/287402462 learned_addr learned my addr 172.21.15.120:0/287402462 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:32.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.868+0000 7fe24d882640 1 --2- 172.21.15.120:0/2999206166 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe2300a3240 0x7fe2300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:32.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.868+0000 7fe24d081640 1 --2- 172.21.15.120:0/287402462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e04838980 0x563e04842d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:32.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.868+0000 7fe24c880640 1 --2- 172.21.15.120:0/287402462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e04838060 0x563e04838440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:32.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.868+0000 7fe2467fc640 1 -- 172.21.15.120:0/2999206166 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fe2400efdb0 con 0x7fe248108290 2024-09-19T09:10:32.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.868+0000 7fe24c880640 1 -- 172.21.15.120:0/287402462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e04838060 msgr2=0x563e04838440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:32.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.868+0000 7fe24c880640 1 --2- 172.21.15.120:0/287402462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e04838060 0x563e04838440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.868+0000 7fe24c880640 1 -- 172.21.15.120:0/287402462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e04838980 msgr2=0x563e04842d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:32.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.868+0000 7fe24c880640 1 --2- 172.21.15.120:0/287402462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e04838980 0x563e04842d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.868+0000 7fe24c880640 1 -- 172.21.15.120:0/287402462 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e04735d40 con 0x563e04843240 2024-09-19T09:10:32.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.868+0000 7fe24c880640 1 --2- 172.21.15.120:0/287402462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e04843240 0x563e04845630 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fe23c012490 tx=0x7fe23c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de760b3eb5e3f1e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:32.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.868+0000 7fe24d882640 1 --2- 172.21.15.120:0/2999206166 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe2300a3240 0x7fe2300a5700 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe2400bc0d0 tx=0x7fe240049dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:32.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.868+0000 7fe22f7fe640 1 -- 172.21.15.120:0/287402462 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c004030 con 0x563e04843240 2024-09-19T09:10:32.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.868+0000 7fe22f7fe640 1 -- 172.21.15.120:0/287402462 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe23c0041d0 con 0x563e04843240 2024-09-19T09:10:32.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.868+0000 7fe22f7fe640 1 -- 172.21.15.120:0/287402462 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c0106a0 con 0x563e04843240 2024-09-19T09:10:32.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.869+0000 7fe2516be740 1 -- 172.21.15.120:0/287402462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e04843240 msgr2=0x563e04845630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:32.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.869+0000 7fe2516be740 1 --2- 172.21.15.120:0/287402462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e04843240 0x563e04845630 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fe23c012490 tx=0x7fe23c00b790 comp rx=0 tx=0).stop 2024-09-19T09:10:32.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.869+0000 7fe2516be740 1 -- 172.21.15.120:0/287402462 shutdown_connections 2024-09-19T09:10:32.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.869+0000 7fe2516be740 1 --2- 172.21.15.120:0/287402462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e04843240 0x563e04845630 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.869+0000 7fe2516be740 1 --2- 172.21.15.120:0/287402462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e04838980 0x563e04842d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.869+0000 7fe2516be740 1 --2- 172.21.15.120:0/287402462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e04838060 0x563e04838440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.869+0000 7fe2516be740 1 -- 172.21.15.120:0/287402462 >> 172.21.15.120:0/287402462 conn(0x563e048345f0 msgr2=0x563e048349d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:32.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.869+0000 7fe2516be740 1 -- 172.21.15.120:0/287402462 shutdown_connections 2024-09-19T09:10:32.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.869+0000 7fe2516be740 1 -- 172.21.15.120:0/287402462 wait complete. 2024-09-19T09:10:32.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.869+0000 7fe2516be740 1 -- start start 2024-09-19T09:10:32.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.869+0000 7fe2516be740 1 Processor -- start 2024-09-19T09:10:32.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.870+0000 7fe2516be740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e04838060 0x563e04840140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.870+0000 7fe2516be740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e04838980 0x563e04840680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.870+0000 7fe2516be740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e04843240 0x563e04840bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.870+0000 7fe2516be740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563e04845e10 con 0x563e04838980 2024-09-19T09:10:32.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.870+0000 7fe2516be740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563e04725f10 con 0x563e04843240 2024-09-19T09:10:32.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.870+0000 7fe2516be740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563e04845f90 con 0x563e04838060 2024-09-19T09:10:32.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.870+0000 7fe2516be740 10 client.0 _ll_register_callbacks cb 0x563e04749db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:32.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.870+0000 7fe24d081640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e04838980 0x563e04840680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:32.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.870+0000 7fe24d081640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e04838980 0x563e04840680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40184/0 (socket says 172.21.15.120:40184) 2024-09-19T09:10:32.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.870+0000 7fe24d081640 1 -- 172.21.15.120:0/1213491392 learned_addr learned my addr 172.21.15.120:0/1213491392 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:32.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.870+0000 7fe24c880640 1 --2- 172.21.15.120:0/1213491392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e04843240 0x563e04840bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:32.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.870+0000 7fe24d081640 1 -- 172.21.15.120:0/1213491392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e04838060 msgr2=0x563e04840140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:32.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.870+0000 7fe24d081640 1 --2- 172.21.15.120:0/1213491392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e04838060 0x563e04840140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.870+0000 7fe24d081640 1 -- 172.21.15.120:0/1213491392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e04843240 msgr2=0x563e04840bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:32.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.870+0000 7fe24d081640 1 --2- 172.21.15.120:0/1213491392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e04843240 0x563e04840bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.870+0000 7fe24d081640 1 -- 172.21.15.120:0/1213491392 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e044ac580 con 0x563e04838980 2024-09-19T09:10:32.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.870+0000 7fe24d081640 1 --2- 172.21.15.120:0/1213491392 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e04838980 0x563e04840680 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fe238010760 tx=0x7fe238010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=658427b7f88b25c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:32.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.870+0000 7fe22cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:32.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.870+0000 7fe22cff9640 1 -- 172.21.15.120:0/1213491392 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23800ece0 con 0x563e04838980 2024-09-19T09:10:32.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.871+0000 7fe22cff9640 1 -- 172.21.15.120:0/1213491392 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fe238012450 con 0x563e04838980 2024-09-19T09:10:32.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.871+0000 7fe2516be740 1 -- 172.21.15.120:0/1213491392 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe23c0041d0 con 0x563e04838980 2024-09-19T09:10:32.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.871+0000 7fe22cff9640 1 -- 172.21.15.120:0/1213491392 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe23800ee80 con 0x563e04838980 2024-09-19T09:10:32.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.871+0000 7fe2516be740 10 client.18123 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:32.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.871+0000 7fe2516be740 1 -- 172.21.15.120:0/1213491392 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563e04576960 con 0x563e04838980 2024-09-19T09:10:32.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.871+0000 7fe22cff9640 1 -- 172.21.15.120:0/1213491392 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe23c0041d0 con 0x563e04838980 2024-09-19T09:10:32.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.871+0000 7fe22cff9640 1 -- 172.21.15.120:0/1213491392 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe23800f040 con 0x563e04838980 2024-09-19T09:10:32.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.871+0000 7fe2516be740 10 client.18123 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:32.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.871+0000 7fe2516be740 10 client.18123 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:10:32.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.871+0000 7fe2516be740 20 client.18123 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:32.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.871+0000 7fe2516be740 1 --2- 172.21.15.120:0/1213491392 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x563e04852e70 0x563e048552b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.871+0000 7fe2516be740 4 client.18123 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:10:32.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.871+0000 7fe2516be740 1 -- 172.21.15.120:0/1213491392 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e047aab90 con 0x563e04852e70 2024-09-19T09:10:32.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.871+0000 7fe24c880640 1 --2- 172.21.15.120:0/1213491392 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x563e04852e70 0x563e048552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:32.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.872+0000 7fe22cff9640 10 client.18123 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:32.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.872+0000 7fe24c880640 1 --2- 172.21.15.120:0/1213491392 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x563e04852e70 0x563e048552b0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=225df19781c902d8 server_cookie=a2e4c2f325e56785 in_seq=0 out_seq=0 2024-09-19T09:10:32.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.873+0000 7fe22cff9640 1 -- 172.21.15.120:0/1213491392 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e047aab90 con 0x563e04852e70 2024-09-19T09:10:32.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.873+0000 7fe22cff9640 10 client.18123 handle_command_reply: tid=0 2024-09-19T09:10:32.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.873+0000 7fe2516be740 1 client.18123 shutdown 2024-09-19T09:10:32.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.873+0000 7fe2516be740 1 -- 172.21.15.120:0/1213491392 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e04838980 msgr2=0x563e04840680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:32.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.873+0000 7fe2516be740 1 --2- 172.21.15.120:0/1213491392 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e04838980 0x563e04840680 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fe238010760 tx=0x7fe238010c20 comp rx=0 tx=0).stop 2024-09-19T09:10:32.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.873+0000 7fe2516be740 1 -- 172.21.15.120:0/1213491392 shutdown_connections 2024-09-19T09:10:32.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.873+0000 7fe2516be740 1 --2- 172.21.15.120:0/1213491392 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x563e04852e70 0x563e048552b0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.873+0000 7fe2516be740 1 --2- 172.21.15.120:0/1213491392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e04843240 0x563e04840bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.873+0000 7fe2516be740 1 --2- 172.21.15.120:0/1213491392 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e04838980 0x563e04840680 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.873+0000 7fe22cff9640 0 client.18123 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:32.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.873+0000 7fe2516be740 1 --2- 172.21.15.120:0/1213491392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e04838060 0x563e04840140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.873+0000 7fe2516be740 1 -- 172.21.15.120:0/1213491392 >> 172.21.15.120:0/1213491392 conn(0x563e048345f0 msgr2=0x563e0483a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:32.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.873+0000 7fe2516be740 1 -- 172.21.15.120:0/1213491392 shutdown_connections 2024-09-19T09:10:32.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.873+0000 7fe2516be740 1 -- 172.21.15.120:0/1213491392 wait complete. 2024-09-19T09:10:32.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.873+0000 7fe2516be740 20 client.18123 trim_cache size 0 max 16384 2024-09-19T09:10:32.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.890+0000 7fe2516be740 1 Processor -- start 2024-09-19T09:10:32.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.890+0000 7fe2516be740 1 -- start start 2024-09-19T09:10:32.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.890+0000 7fe2516be740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e04838060 0x563e0483eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.890+0000 7fe2516be740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e04838980 0x563e0485ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.890+0000 7fe2516be740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e04843240 0x563e0485b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.890+0000 7fe2516be740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe2481ad5c0 con 0x563e04838060 2024-09-19T09:10:32.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.890+0000 7fe2516be740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe2481af100 con 0x563e04843240 2024-09-19T09:10:32.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.890+0000 7fe2516be740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe2481adc90 con 0x563e04838980 2024-09-19T09:10:32.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.890+0000 7fe24c880640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e04838060 0x563e0483eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:32.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.890+0000 7fe24c880640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e04838060 0x563e0483eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40190/0 (socket says 172.21.15.120:40190) 2024-09-19T09:10:32.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.890+0000 7fe24c880640 1 -- 172.21.15.120:0/4174326211 learned_addr learned my addr 172.21.15.120:0/4174326211 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:32.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.890+0000 7fe24c880640 1 -- 172.21.15.120:0/4174326211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e04838980 msgr2=0x563e0485ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:32.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.890+0000 7fe24c880640 1 --2- 172.21.15.120:0/4174326211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e04838980 0x563e0485ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.890+0000 7fe24c880640 1 -- 172.21.15.120:0/4174326211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e04843240 msgr2=0x563e0485b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:32.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.890+0000 7fe24c880640 1 --2- 172.21.15.120:0/4174326211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e04843240 0x563e0485b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.890+0000 7fe24c880640 1 -- 172.21.15.120:0/4174326211 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e04832bb0 con 0x563e04838060 2024-09-19T09:10:32.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.890+0000 7fe24c880640 1 --2- 172.21.15.120:0/4174326211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e04838060 0x563e0483eff0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fe23c003da0 tx=0x7fe23c003dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83a001aeed5a98f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:32.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.891+0000 7fe22effd640 1 -- 172.21.15.120:0/4174326211 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c0035f0 con 0x563e04838060 2024-09-19T09:10:32.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.891+0000 7fe22effd640 1 -- 172.21.15.120:0/4174326211 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe23c005040 con 0x563e04838060 2024-09-19T09:10:32.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.891+0000 7fe22effd640 1 -- 172.21.15.120:0/4174326211 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c02b670 con 0x563e04838060 2024-09-19T09:10:32.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.891+0000 7fe2516be740 1 -- 172.21.15.120:0/4174326211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e04838060 msgr2=0x563e0483eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:32.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.891+0000 7fe2516be740 1 --2- 172.21.15.120:0/4174326211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e04838060 0x563e0483eff0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fe23c003da0 tx=0x7fe23c003dd0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.891+0000 7fe2516be740 1 -- 172.21.15.120:0/4174326211 shutdown_connections 2024-09-19T09:10:32.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.891+0000 7fe2516be740 1 --2- 172.21.15.120:0/4174326211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e04843240 0x563e0485b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.891+0000 7fe2516be740 1 --2- 172.21.15.120:0/4174326211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e04838980 0x563e0485ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.891+0000 7fe2516be740 1 --2- 172.21.15.120:0/4174326211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e04838060 0x563e0483eff0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.891+0000 7fe2516be740 1 -- 172.21.15.120:0/4174326211 >> 172.21.15.120:0/4174326211 conn(0x563e048345f0 msgr2=0x563e0483e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:32.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.891+0000 7fe2516be740 1 -- 172.21.15.120:0/4174326211 shutdown_connections 2024-09-19T09:10:32.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.891+0000 7fe2516be740 1 -- 172.21.15.120:0/4174326211 wait complete. 2024-09-19T09:10:32.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.892+0000 7fe2516be740 1 -- start start 2024-09-19T09:10:32.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.892+0000 7fe2516be740 1 Processor -- start 2024-09-19T09:10:32.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.892+0000 7fe2516be740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e04838060 0x563e0484b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.892+0000 7fe2516be740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e04838980 0x563e04847a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.892+0000 7fe2516be740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e04843240 0x563e04847f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.892+0000 7fe2516be740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe2481aea30 con 0x563e04838980 2024-09-19T09:10:32.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.892+0000 7fe2516be740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe2481ae360 con 0x563e04843240 2024-09-19T09:10:32.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.892+0000 7fe2516be740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe2481ab580 con 0x563e04838060 2024-09-19T09:10:32.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.892+0000 7fe2516be740 10 client.0 _ll_register_callbacks cb 0x7fe238010370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:32.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.892+0000 7fe24d081640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e04838980 0x563e04847a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:32.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.892+0000 7fe24d081640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e04838980 0x563e04847a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40198/0 (socket says 172.21.15.120:40198) 2024-09-19T09:10:32.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.892+0000 7fe24d081640 1 -- 172.21.15.120:0/2739646739 learned_addr learned my addr 172.21.15.120:0/2739646739 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:32.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.892+0000 7fe24d081640 1 -- 172.21.15.120:0/2739646739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e04838060 msgr2=0x563e0484b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:32.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.892+0000 7fe24d081640 1 --2- 172.21.15.120:0/2739646739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e04838060 0x563e0484b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.892+0000 7fe24d081640 1 -- 172.21.15.120:0/2739646739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e04843240 msgr2=0x563e04847f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:32.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.892+0000 7fe24d081640 1 --2- 172.21.15.120:0/2739646739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e04843240 0x563e04847f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.892+0000 7fe24d081640 1 -- 172.21.15.120:0/2739646739 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e04848500 con 0x563e04838980 2024-09-19T09:10:32.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.893+0000 7fe24d081640 1 --2- 172.21.15.120:0/2739646739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e04838980 0x563e04847a00 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fe238002230 tx=0x7fe23800e790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22fa189e1721a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:32.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.893+0000 7fe22cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:32.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.893+0000 7fe22cff9640 1 -- 172.21.15.120:0/2739646739 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23801c3f0 con 0x563e04838980 2024-09-19T09:10:32.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.893+0000 7fe22cff9640 1 -- 172.21.15.120:0/2739646739 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fe23801ca00 con 0x563e04838980 2024-09-19T09:10:32.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.893+0000 7fe2516be740 1 -- 172.21.15.120:0/2739646739 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe23c005040 con 0x563e04838980 2024-09-19T09:10:32.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.893+0000 7fe22cff9640 1 -- 172.21.15.120:0/2739646739 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe23801c590 con 0x563e04838980 2024-09-19T09:10:32.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.893+0000 7fe2516be740 10 client.18129 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:32.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.893+0000 7fe2516be740 1 -- 172.21.15.120:0/2739646739 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563e048be930 con 0x563e04838980 2024-09-19T09:10:32.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.893+0000 7fe22cff9640 1 -- 172.21.15.120:0/2739646739 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe23c005040 con 0x563e04838980 2024-09-19T09:10:32.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.893+0000 7fe22cff9640 1 -- 172.21.15.120:0/2739646739 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe238027950 con 0x563e04838980 2024-09-19T09:10:32.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.893+0000 7fe2516be740 10 client.18129 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:32.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.893+0000 7fe2516be740 10 client.18129 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:10:32.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.893+0000 7fe2516be740 20 client.18129 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:32.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.893+0000 7fe2516be740 1 --2- 172.21.15.120:0/2739646739 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x563e048bf000 0x563e048c1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:32.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.893+0000 7fe2516be740 4 client.18129 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "aeb2f3f6", "release": true, "await": true}] 2024-09-19T09:10:32.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.893+0000 7fe2516be740 1 -- 172.21.15.120:0/2739646739 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "set_id": "aeb2f3f6", "release": true, "await": true}) -- 0x563e047a50a0 con 0x563e048bf000 2024-09-19T09:10:32.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.894+0000 7fe24c880640 1 --2- 172.21.15.120:0/2739646739 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x563e048bf000 0x563e048c1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:32.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.894+0000 7fe22cff9640 10 client.18129 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:32.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.894+0000 7fe24c880640 1 --2- 172.21.15.120:0/2739646739 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x563e048bf000 0x563e048c1420 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=640b23c21e864af1 server_cookie=c08d889915d31d3a in_seq=0 out_seq=0 2024-09-19T09:10:32.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.915+0000 7fe22cff9640 1 -- 172.21.15.120:0/2739646739 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563e047a50a0 con 0x563e048bf000 2024-09-19T09:10:32.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.915+0000 7fe22cff9640 10 client.18129 handle_command_reply: tid=0 2024-09-19T09:10:32.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.915+0000 7fe2516be740 1 client.18129 shutdown 2024-09-19T09:10:32.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.915+0000 7fe2516be740 1 -- 172.21.15.120:0/2739646739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e04838980 msgr2=0x563e04847a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:32.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.915+0000 7fe2516be740 1 --2- 172.21.15.120:0/2739646739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e04838980 0x563e04847a00 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fe238002230 tx=0x7fe23800e790 comp rx=0 tx=0).stop 2024-09-19T09:10:32.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.915+0000 7fe2516be740 1 -- 172.21.15.120:0/2739646739 shutdown_connections 2024-09-19T09:10:32.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.915+0000 7fe2516be740 1 --2- 172.21.15.120:0/2739646739 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x563e048bf000 0x563e048c1420 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.915+0000 7fe2516be740 1 --2- 172.21.15.120:0/2739646739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e04843240 0x563e04847f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.915+0000 7fe2516be740 1 --2- 172.21.15.120:0/2739646739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e04838980 0x563e04847a00 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.915+0000 7fe2516be740 1 --2- 172.21.15.120:0/2739646739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e04838060 0x563e0484b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.915+0000 7fe2516be740 1 -- 172.21.15.120:0/2739646739 >> 172.21.15.120:0/2739646739 conn(0x563e048345f0 msgr2=0x563e04833750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:32.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.915+0000 7fe22cff9640 0 client.18129 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:32.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.915+0000 7fe2516be740 1 -- 172.21.15.120:0/2739646739 shutdown_connections 2024-09-19T09:10:32.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.915+0000 7fe2516be740 1 -- 172.21.15.120:0/2739646739 wait complete. 2024-09-19T09:10:32.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.915+0000 7fe2516be740 20 client.18129 trim_cache size 0 max 16384 2024-09-19T09:10:32.917 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:10:32.917 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 30, 2024-09-19T09:10:32.917 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:10:32.917 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 28, 2024-09-19T09:10:32.917 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:10:32.917 INFO:teuthology.orchestra.run.smithi120.stdout: "aeb2f3f6": { 2024-09-19T09:10:32.917 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 28, 2024-09-19T09:10:32.917 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:10:32.917 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:10:32.917 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:10:32.918 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:10:32.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:10:32.918 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:10:32.918 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 145.8, 2024-09-19T09:10:32.918 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:10:32.918 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:10:32.918 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:10:32.918 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:10:32.918 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:10:32.918 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:10:32.918 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:10:32.918 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:10:32.918 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:10:32.918 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:10:32.919 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:10:32.919 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:10:32.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.915+0000 7fe24f2e5640 1 -- 172.21.15.120:0/2999206166 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe2300a3240 msgr2=0x7fe2300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:32.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.915+0000 7fe24f2e5640 1 --2- 172.21.15.120:0/2999206166 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe2300a3240 0x7fe2300a5700 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe2400bc0d0 tx=0x7fe240049dd0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.915+0000 7fe24f2e5640 1 -- 172.21.15.120:0/2999206166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe248108290 msgr2=0x7fe24819ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:32.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.915+0000 7fe24f2e5640 1 --2- 172.21.15.120:0/2999206166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe248108290 0x7fe24819ce00 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fe240005ab0 tx=0x7fe2400042f0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.915+0000 7fe24f2e5640 1 -- 172.21.15.120:0/2999206166 shutdown_connections 2024-09-19T09:10:32.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.915+0000 7fe24f2e5640 1 --2- 172.21.15.120:0/2999206166 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe2300a3240 0x7fe2300a5700 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.915+0000 7fe24f2e5640 1 --2- 172.21.15.120:0/2999206166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe248108290 0x7fe24819ce00 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.915+0000 7fe24f2e5640 1 --2- 172.21.15.120:0/2999206166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe248103740 0x7fe248198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.915+0000 7fe24f2e5640 1 --2- 172.21.15.120:0/2999206166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe248102d90 0x7fe248198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:32.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.915+0000 7fe24f2e5640 1 -- 172.21.15.120:0/2999206166 >> 172.21.15.120:0/2999206166 conn(0x7fe2480fc7d0 msgr2=0x7fe248105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:32.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.916+0000 7fe24f2e5640 1 -- 172.21.15.120:0/2999206166 shutdown_connections 2024-09-19T09:10:32.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:32.916+0000 7fe24f2e5640 1 -- 172.21.15.120:0/2999206166 wait complete. 2024-09-19T09:10:33.058 INFO:tasks.workunit.client.0.smithi120.stdout: 1 698817 9.54 MB/sec execute 268 sec latency 24706.540 ms 2024-09-19T09:10:33.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:32 smithi120 ceph-mon[31150]: pgmap v572: 97 pgs: 97 active+clean; 36 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 2.3 MiB/s wr, 126 op/s 2024-09-19T09:10:33.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:32 smithi196 ceph-mon[35150]: pgmap v572: 97 pgs: 97 active+clean; 36 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 2.3 MiB/s wr, 126 op/s 2024-09-19T09:10:33.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:32 smithi153 ceph-mon[35071]: pgmap v572: 97 pgs: 97 active+clean; 36 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 2.3 MiB/s wr, 126 op/s 2024-09-19T09:10:33.902 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'aeb2f3f6', total seconds elapsed: 29.5 2024-09-19T09:10:33.902 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.75 2024-09-19T09:10:33.902 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:10:34.058 INFO:tasks.workunit.client.0.smithi120.stdout: 1 701035 9.55 MB/sec execute 269 sec latency 13.014 ms 2024-09-19T09:10:34.201 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:10:34.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.708+0000 7fc6b580c640 1 -- 172.21.15.120:0/2879223435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6b0100e20 msgr2=0x7fc6b0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:34.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.708+0000 7fc6b580c640 1 --2- 172.21.15.120:0/2879223435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6b0100e20 0x7fc6b0101200 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009900 tx=0x7fc6a002a480 comp rx=0 tx=0).stop 2024-09-19T09:10:34.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.708+0000 7fc6b580c640 1 -- 172.21.15.120:0/2879223435 shutdown_connections 2024-09-19T09:10:34.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.708+0000 7fc6b580c640 1 --2- 172.21.15.120:0/2879223435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6b0102ef0 0x7fc6b010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:34.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.708+0000 7fc6b580c640 1 --2- 172.21.15.120:0/2879223435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6b0102570 0x7fc6b01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:34.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.708+0000 7fc6b580c640 1 --2- 172.21.15.120:0/2879223435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6b0100e20 0x7fc6b0101200 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:34.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.708+0000 7fc6b580c640 1 -- 172.21.15.120:0/2879223435 >> 172.21.15.120:0/2879223435 conn(0x7fc6b00fc7d0 msgr2=0x7fc6b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:34.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.708+0000 7fc6b580c640 1 -- 172.21.15.120:0/2879223435 shutdown_connections 2024-09-19T09:10:34.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.708+0000 7fc6b580c640 1 -- 172.21.15.120:0/2879223435 wait complete. 2024-09-19T09:10:34.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.709+0000 7fc6b580c640 1 Processor -- start 2024-09-19T09:10:34.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.709+0000 7fc6b580c640 1 -- start start 2024-09-19T09:10:34.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.709+0000 7fc6b580c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6b0100e20 0x7fc6b01985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:34.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.709+0000 7fc6b580c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6b0102570 0x7fc6b0198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:34.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.709+0000 7fc6b580c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6b0102ef0 0x7fc6b019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:34.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.709+0000 7fc6b580c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b0110f80 con 0x7fc6b0102570 2024-09-19T09:10:34.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.709+0000 7fc6b580c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b0110e00 con 0x7fc6b0100e20 2024-09-19T09:10:34.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.709+0000 7fc6b580c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b0111100 con 0x7fc6b0102ef0 2024-09-19T09:10:34.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.709+0000 7fc6a67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6b0102570 0x7fc6b0198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:34.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.709+0000 7fc6a67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6b0102570 0x7fc6b0198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40210/0 (socket says 172.21.15.120:40210) 2024-09-19T09:10:34.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.709+0000 7fc6a67fc640 1 -- 172.21.15.120:0/4263674279 learned_addr learned my addr 172.21.15.120:0/4263674279 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:34.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.709+0000 7fc6aeffd640 1 --2- 172.21.15.120:0/4263674279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6b0100e20 0x7fc6b01985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:34.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.709+0000 7fc6af7fe640 1 --2- 172.21.15.120:0/4263674279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6b0102ef0 0x7fc6b019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:34.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.709+0000 7fc6a67fc640 1 -- 172.21.15.120:0/4263674279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6b0102ef0 msgr2=0x7fc6b019ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:34.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.709+0000 7fc6a67fc640 1 --2- 172.21.15.120:0/4263674279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6b0102ef0 0x7fc6b019ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:34.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.709+0000 7fc6a67fc640 1 -- 172.21.15.120:0/4263674279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6b0100e20 msgr2=0x7fc6b01985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:34.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.709+0000 7fc6a67fc640 1 --2- 172.21.15.120:0/4263674279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6b0100e20 0x7fc6b01985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:34.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.709+0000 7fc6a67fc640 1 -- 172.21.15.120:0/4263674279 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b019d550 con 0x7fc6b0102570 2024-09-19T09:10:34.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.710+0000 7fc6a67fc640 1 --2- 172.21.15.120:0/4263674279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6b0102570 0x7fc6b0198ae0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fc69800e9f0 tx=0x7fc69800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:34.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.710+0000 7fc6b480a640 1 -- 172.21.15.120:0/4263674279 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69800cd80 con 0x7fc6b0102570 2024-09-19T09:10:34.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.710+0000 7fc6b580c640 1 -- 172.21.15.120:0/4263674279 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6b019d840 con 0x7fc6b0102570 2024-09-19T09:10:34.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.710+0000 7fc6b480a640 1 -- 172.21.15.120:0/4263674279 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc698010910 con 0x7fc6b0102570 2024-09-19T09:10:34.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.710+0000 7fc6b480a640 1 -- 172.21.15.120:0/4263674279 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6980188f0 con 0x7fc6b0102570 2024-09-19T09:10:34.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.710+0000 7fc6b580c640 1 -- 172.21.15.120:0/4263674279 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6b01a5120 con 0x7fc6b0102570 2024-09-19T09:10:34.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.711+0000 7fc6b580c640 1 -- 172.21.15.120:0/4263674279 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6b0101200 con 0x7fc6b0102570 2024-09-19T09:10:34.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.712+0000 7fc6b480a640 1 -- 172.21.15.120:0/4263674279 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fc698010ac0 con 0x7fc6b0102570 2024-09-19T09:10:34.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.713+0000 7fc6b480a640 1 --2- 172.21.15.120:0/4263674279 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc69c0a3240 0x7fc69c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:34.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.713+0000 7fc6aeffd640 1 --2- 172.21.15.120:0/4263674279 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc69c0a3240 0x7fc69c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:34.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.713+0000 7fc6b480a640 1 -- 172.21.15.120:0/4263674279 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fc6980cfca0 con 0x7fc6b0102570 2024-09-19T09:10:34.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.713+0000 7fc6aeffd640 1 --2- 172.21.15.120:0/4263674279 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc69c0a3240 0x7fc69c0a5700 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009230 tx=0x7fc6a00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:34.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.717+0000 7fc6b480a640 1 -- 172.21.15.120:0/4263674279 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc69809c370 con 0x7fc6b0102570 2024-09-19T09:10:34.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.930+0000 7fc6b580c640 1 -- 172.21.15.120:0/4263674279 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6b01a5320 con 0x7fc6b0102570 2024-09-19T09:10:34.931 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:34 smithi120 ceph-mon[31150]: pgmap v573: 97 pgs: 97 active+clean; 37 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 2.1 MiB/s wr, 118 op/s 2024-09-19T09:10:34.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.931+0000 7fc6b480a640 1 -- 172.21.15.120:0/4263674279 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11985 (secure 0 0 0) 0x7fc6980a0670 con 0x7fc6b0102570 2024-09-19T09:10:34.932 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:10:34.932 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":30,"btime":"2024-09-19T09:07:14:812065+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:07:14.812046+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26351":{"gid":26351,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/364941779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":364941779},{"type":"v1","addr":"172.21.15.153:6835","nonce":364941779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:10:34.932 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 30 2024-09-19T09:10:34.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.933+0000 7fc6b580c640 1 -- 172.21.15.120:0/4263674279 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc69c0a3240 msgr2=0x7fc69c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:34.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.933+0000 7fc6b580c640 1 --2- 172.21.15.120:0/4263674279 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc69c0a3240 0x7fc69c0a5700 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009230 tx=0x7fc6a00058d0 comp rx=0 tx=0).stop 2024-09-19T09:10:34.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.933+0000 7fc6b580c640 1 -- 172.21.15.120:0/4263674279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6b0102570 msgr2=0x7fc6b0198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:34.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.933+0000 7fc6b580c640 1 --2- 172.21.15.120:0/4263674279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6b0102570 0x7fc6b0198ae0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fc69800e9f0 tx=0x7fc69800eeb0 comp rx=0 tx=0).stop 2024-09-19T09:10:34.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.933+0000 7fc6b580c640 1 -- 172.21.15.120:0/4263674279 shutdown_connections 2024-09-19T09:10:34.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.933+0000 7fc6b580c640 1 --2- 172.21.15.120:0/4263674279 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc69c0a3240 0x7fc69c0a5700 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:34.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.933+0000 7fc6b580c640 1 --2- 172.21.15.120:0/4263674279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6b0102ef0 0x7fc6b019ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:34.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.933+0000 7fc6b580c640 1 --2- 172.21.15.120:0/4263674279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6b0102570 0x7fc6b0198ae0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:34.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.933+0000 7fc6b580c640 1 --2- 172.21.15.120:0/4263674279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6b0100e20 0x7fc6b01985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:34.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.933+0000 7fc6b580c640 1 -- 172.21.15.120:0/4263674279 >> 172.21.15.120:0/4263674279 conn(0x7fc6b00fc7d0 msgr2=0x7fc6b0069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:34.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.933+0000 7fc6b580c640 1 -- 172.21.15.120:0/4263674279 shutdown_connections 2024-09-19T09:10:34.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:34.933+0000 7fc6b580c640 1 -- 172.21.15.120:0/4263674279 wait complete. 2024-09-19T09:10:35.058 INFO:tasks.workunit.client.0.smithi120.stdout: 1 703750 9.57 MB/sec execute 270 sec latency 18.957 ms 2024-09-19T09:10:35.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:34 smithi196 ceph-mon[35150]: pgmap v573: 97 pgs: 97 active+clean; 37 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 2.1 MiB/s wr, 118 op/s 2024-09-19T09:10:35.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:34 smithi153 ceph-mon[35071]: pgmap v573: 97 pgs: 97 active+clean; 37 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 2.1 MiB/s wr, 118 op/s 2024-09-19T09:10:35.771 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0' 2024-09-19T09:10:35.771 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0 2024-09-19T09:10:36.030 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:10:36.058 INFO:tasks.workunit.client.0.smithi120.stdout: 1 706909 9.60 MB/sec execute 271 sec latency 12.638 ms 2024-09-19T09:10:36.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:35 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/4263674279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:10:36.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:35 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/4263674279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:10:36.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:35 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/4263674279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:10:36.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.584+0000 7f4451b8e640 1 -- 172.21.15.120:0/339169178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f444c100b20 msgr2=0x7f444c100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:36.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.584+0000 7f4451b8e640 1 --2- 172.21.15.120:0/339169178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f444c100b20 0x7f444c100f00 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f443c009900 tx=0x7f443c02a410 comp rx=0 tx=0).stop 2024-09-19T09:10:36.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.584+0000 7f4451b8e640 1 -- 172.21.15.120:0/339169178 shutdown_connections 2024-09-19T09:10:36.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.584+0000 7f4451b8e640 1 --2- 172.21.15.120:0/339169178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f444c10e500 0x7f444c110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.584+0000 7f4451b8e640 1 --2- 172.21.15.120:0/339169178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f444c101440 0x7f444c10df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.584+0000 7f4451b8e640 1 --2- 172.21.15.120:0/339169178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f444c100b20 0x7f444c100f00 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.584+0000 7f4451b8e640 1 -- 172.21.15.120:0/339169178 >> 172.21.15.120:0/339169178 conn(0x7f444c0fc7d0 msgr2=0x7f444c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:36.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.584+0000 7f4451b8e640 1 -- 172.21.15.120:0/339169178 shutdown_connections 2024-09-19T09:10:36.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.584+0000 7f4451b8e640 1 -- 172.21.15.120:0/339169178 wait complete. 2024-09-19T09:10:36.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.585+0000 7f4451b8e640 1 Processor -- start 2024-09-19T09:10:36.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.585+0000 7f4451b8e640 1 -- start start 2024-09-19T09:10:36.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.586+0000 7f4451b8e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f444c100b20 0x7f444c19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:36.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.586+0000 7f4451b8e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f444c101440 0x7f444c19ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:36.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.586+0000 7f4451b8e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f444c10e500 0x7f444c19e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:36.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.586+0000 7f4451b8e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f444c114280 con 0x7f444c100b20 2024-09-19T09:10:36.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.586+0000 7f4451b8e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f444c114100 con 0x7f444c101440 2024-09-19T09:10:36.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.586+0000 7f4451b8e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f444c114400 con 0x7f444c10e500 2024-09-19T09:10:36.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.586+0000 7f444b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f444c100b20 0x7f444c19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:36.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.586+0000 7f444b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f444c100b20 0x7f444c19a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40232/0 (socket says 172.21.15.120:40232) 2024-09-19T09:10:36.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.586+0000 7f444b7fe640 1 -- 172.21.15.120:0/426158225 learned_addr learned my addr 172.21.15.120:0/426158225 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:36.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.586+0000 7f444affd640 1 --2- 172.21.15.120:0/426158225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f444c101440 0x7f444c19ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:36.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.586+0000 7f444bfff640 1 --2- 172.21.15.120:0/426158225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f444c10e500 0x7f444c19e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:36.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.586+0000 7f444b7fe640 1 -- 172.21.15.120:0/426158225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f444c10e500 msgr2=0x7f444c19e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:36.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.586+0000 7f444b7fe640 1 --2- 172.21.15.120:0/426158225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f444c10e500 0x7f444c19e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.586+0000 7f444b7fe640 1 -- 172.21.15.120:0/426158225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f444c101440 msgr2=0x7f444c19ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:36.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.586+0000 7f444b7fe640 1 --2- 172.21.15.120:0/426158225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f444c101440 0x7f444c19ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.586+0000 7f444b7fe640 1 -- 172.21.15.120:0/426158225 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f444c19e730 con 0x7f444c100b20 2024-09-19T09:10:36.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.586+0000 7f444affd640 1 --2- 172.21.15.120:0/426158225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f444c101440 0x7f444c19ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:10:36.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.586+0000 7f444b7fe640 1 --2- 172.21.15.120:0/426158225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f444c100b20 0x7f444c19a7e0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f443c006010 tx=0x7f443c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:36.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.587+0000 7f4448ff9640 1 -- 172.21.15.120:0/426158225 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c03f030 con 0x7f444c100b20 2024-09-19T09:10:36.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.587+0000 7f4448ff9640 1 -- 172.21.15.120:0/426158225 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f443c004440 con 0x7f444c100b20 2024-09-19T09:10:36.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.587+0000 7f4451b8e640 1 -- 172.21.15.120:0/426158225 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f444c19e9c0 con 0x7f444c100b20 2024-09-19T09:10:36.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.587+0000 7f4448ff9640 1 -- 172.21.15.120:0/426158225 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c035070 con 0x7f444c100b20 2024-09-19T09:10:36.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.587+0000 7f4451b8e640 1 -- 172.21.15.120:0/426158225 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f444c1a7390 con 0x7f444c100b20 2024-09-19T09:10:36.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.589+0000 7f4448ff9640 1 -- 172.21.15.120:0/426158225 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f443c042030 con 0x7f444c100b20 2024-09-19T09:10:36.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.590+0000 7f4453f67740 1 Processor -- start 2024-09-19T09:10:36.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.590+0000 7f4453f67740 1 -- start start 2024-09-19T09:10:36.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.590+0000 7f4448ff9640 1 --2- 172.21.15.120:0/426158225 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f44340a3240 0x7f44340a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:36.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.590+0000 7f4453f67740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e212235850 0x55e212235c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:36.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.590+0000 7f4453f67740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e212236170 0x55e2122404f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:36.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.590+0000 7f4453f67740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e212251a40 0x55e212253e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:36.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.590+0000 7f4453f67740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e21219a160 con 0x55e212235850 2024-09-19T09:10:36.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.590+0000 7f4453f67740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e212141720 con 0x55e212236170 2024-09-19T09:10:36.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.590+0000 7f4453f67740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e212196120 con 0x55e212251a40 2024-09-19T09:10:36.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.590+0000 7f4448ff9640 1 -- 172.21.15.120:0/426158225 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f443c0efc70 con 0x7f444c100b20 2024-09-19T09:10:36.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.590+0000 7f444affd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e212236170 0x55e2122404f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:36.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.590+0000 7f444bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e212251a40 0x55e212253e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:36.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.590+0000 7f444b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e212235850 0x55e212235c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:36.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.590+0000 7f444bfff640 1 --2- 172.21.15.120:0/426158225 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f44340a3240 0x7f44340a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:36.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.590+0000 7f444b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e212235850 0x55e212235c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40240/0 (socket says 172.21.15.120:40240) 2024-09-19T09:10:36.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.590+0000 7f444affd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e212236170 0x55e2122404f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:34334/0 (socket says 172.21.15.120:34334) 2024-09-19T09:10:36.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.590+0000 7f444b7fe640 1 -- 172.21.15.120:0/1924763778 learned_addr learned my addr 172.21.15.120:0/1924763778 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:36.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.591+0000 7f444b7fe640 1 -- 172.21.15.120:0/1924763778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e212251a40 msgr2=0x55e212253e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:36.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.591+0000 7f444b7fe640 1 --2- 172.21.15.120:0/1924763778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e212251a40 0x55e212253e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.591+0000 7f444b7fe640 1 -- 172.21.15.120:0/1924763778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e212236170 msgr2=0x55e2122404f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:36.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.591+0000 7f444b7fe640 1 --2- 172.21.15.120:0/1924763778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e212236170 0x55e2122404f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.591+0000 7f444b7fe640 1 -- 172.21.15.120:0/1924763778 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e211f0bba0 con 0x55e212235850 2024-09-19T09:10:36.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.591+0000 7f444bfff640 1 --2- 172.21.15.120:0/426158225 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f44340a3240 0x7f44340a5700 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55e212232ed0 tx=0x7f4444009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:36.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.591+0000 7f444b7fe640 1 --2- 172.21.15.120:0/1924763778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e212235850 0x55e212235c30 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f443c0c49d0 tx=0x7f443c0bc1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74c481b84f6ba40 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:36.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.591+0000 7f4431ffb640 1 -- 172.21.15.120:0/1924763778 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c049040 con 0x55e212235850 2024-09-19T09:10:36.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.591+0000 7f4431ffb640 1 -- 172.21.15.120:0/1924763778 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f443c034170 con 0x55e212235850 2024-09-19T09:10:36.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.591+0000 7f4431ffb640 1 -- 172.21.15.120:0/1924763778 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c0c0640 con 0x55e212235850 2024-09-19T09:10:36.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.592+0000 7f4453f67740 1 -- 172.21.15.120:0/1924763778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e212235850 msgr2=0x55e212235c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:36.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.592+0000 7f4453f67740 1 --2- 172.21.15.120:0/1924763778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e212235850 0x55e212235c30 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f443c0c49d0 tx=0x7f443c0bc1d0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.592+0000 7f4453f67740 1 -- 172.21.15.120:0/1924763778 shutdown_connections 2024-09-19T09:10:36.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.592+0000 7f4453f67740 1 --2- 172.21.15.120:0/1924763778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e212251a40 0x55e212253e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.592+0000 7f4453f67740 1 --2- 172.21.15.120:0/1924763778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e212236170 0x55e2122404f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.592+0000 7f4453f67740 1 --2- 172.21.15.120:0/1924763778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e212235850 0x55e212235c30 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.592+0000 7f4453f67740 1 -- 172.21.15.120:0/1924763778 >> 172.21.15.120:0/1924763778 conn(0x55e212231de0 msgr2=0x55e2122321c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:36.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.592+0000 7f4453f67740 1 -- 172.21.15.120:0/1924763778 shutdown_connections 2024-09-19T09:10:36.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.592+0000 7f4453f67740 1 -- 172.21.15.120:0/1924763778 wait complete. 2024-09-19T09:10:36.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.593+0000 7f4453f67740 1 -- start start 2024-09-19T09:10:36.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.593+0000 7f4453f67740 1 Processor -- start 2024-09-19T09:10:36.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.593+0000 7f4453f67740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e212235850 0x55e21223da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:36.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.593+0000 7f4453f67740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e212236170 0x55e21223df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:36.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.593+0000 7f4453f67740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e212251a40 0x55e21223e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:36.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.593+0000 7f4453f67740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e2122543d0 con 0x55e212235850 2024-09-19T09:10:36.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.593+0000 7f4453f67740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e212122900 con 0x55e212251a40 2024-09-19T09:10:36.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.593+0000 7f4453f67740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e212254550 con 0x55e212236170 2024-09-19T09:10:36.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.593+0000 7f4453f67740 10 client.0 _ll_register_callbacks cb 0x55e212120780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:36.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.593+0000 7f444b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e212235850 0x55e21223da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:36.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.593+0000 7f444b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e212235850 0x55e21223da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40250/0 (socket says 172.21.15.120:40250) 2024-09-19T09:10:36.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.593+0000 7f444b7fe640 1 -- 172.21.15.120:0/1311114573 learned_addr learned my addr 172.21.15.120:0/1311114573 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:36.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.594+0000 7f444affd640 1 --2- 172.21.15.120:0/1311114573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e212236170 0x55e21223df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:36.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.594+0000 7f444bfff640 1 --2- 172.21.15.120:0/1311114573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e212251a40 0x55e21223e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:36.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.594+0000 7f444b7fe640 1 -- 172.21.15.120:0/1311114573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e212236170 msgr2=0x55e21223df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:36.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.594+0000 7f444b7fe640 1 --2- 172.21.15.120:0/1311114573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e212236170 0x55e21223df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.594+0000 7f444b7fe640 1 -- 172.21.15.120:0/1311114573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e212251a40 msgr2=0x55e21223e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:36.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.594+0000 7f444b7fe640 1 --2- 172.21.15.120:0/1311114573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e212251a40 0x55e21223e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.594+0000 7f444b7fe640 1 -- 172.21.15.120:0/1311114573 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e212162c50 con 0x55e212235850 2024-09-19T09:10:36.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.594+0000 7f444b7fe640 1 --2- 172.21.15.120:0/1311114573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e212235850 0x55e21223da10 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f443c04bff0 tx=0x7f443c0bc340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf061cc358b6d1e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:36.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.594+0000 7f442affd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:36.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.594+0000 7f442affd640 1 -- 172.21.15.120:0/1311114573 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c0d4020 con 0x55e212235850 2024-09-19T09:10:36.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.594+0000 7f442affd640 1 -- 172.21.15.120:0/1311114573 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f443c0d5220 con 0x55e212235850 2024-09-19T09:10:36.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.594+0000 7f4453f67740 1 -- 172.21.15.120:0/1311114573 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f443c034170 con 0x55e212235850 2024-09-19T09:10:36.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.594+0000 7f442affd640 1 -- 172.21.15.120:0/1311114573 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f443c049070 con 0x55e212235850 2024-09-19T09:10:36.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.594+0000 7f4453f67740 10 client.18147 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:36.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.594+0000 7f4453f67740 1 -- 172.21.15.120:0/1311114573 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e2122543d0 con 0x55e212235850 2024-09-19T09:10:36.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.595+0000 7f442affd640 1 -- 172.21.15.120:0/1311114573 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f443c034170 con 0x55e212235850 2024-09-19T09:10:36.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.595+0000 7f442affd640 1 -- 172.21.15.120:0/1311114573 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f443c0b89b0 con 0x55e212235850 2024-09-19T09:10:36.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.595+0000 7f4453f67740 10 client.18147 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:36.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.595+0000 7f4453f67740 10 client.18147 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:10:36.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.595+0000 7f4453f67740 20 client.18147 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:36.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.595+0000 7f4453f67740 1 --2- 172.21.15.120:0/1311114573 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e212261260 0x55e212263650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:36.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.595+0000 7f4453f67740 4 client.18147 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:10:36.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.595+0000 7f4453f67740 1 -- 172.21.15.120:0/1311114573 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2121a7890 con 0x55e212261260 2024-09-19T09:10:36.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.595+0000 7f444affd640 1 --2- 172.21.15.120:0/1311114573 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e212261260 0x55e212263650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:36.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.595+0000 7f444affd640 1 --2- 172.21.15.120:0/1311114573 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e212261260 0x55e212263650 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a5ac2aeafdc40af server_cookie=a336ac3427ef17b in_seq=0 out_seq=0 2024-09-19T09:10:36.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.595+0000 7f442affd640 10 client.18147 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:36.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.597+0000 7f442affd640 1 -- 172.21.15.120:0/1311114573 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e2121a7890 con 0x55e212261260 2024-09-19T09:10:36.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.597+0000 7f442affd640 10 client.18147 handle_command_reply: tid=0 2024-09-19T09:10:36.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.597+0000 7f4453f67740 1 client.18147 shutdown 2024-09-19T09:10:36.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.598+0000 7f4453f67740 1 -- 172.21.15.120:0/1311114573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e212235850 msgr2=0x55e21223da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:36.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.598+0000 7f4453f67740 1 --2- 172.21.15.120:0/1311114573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e212235850 0x55e21223da10 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f443c04bff0 tx=0x7f443c0bc340 comp rx=0 tx=0).stop 2024-09-19T09:10:36.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.598+0000 7f4453f67740 1 -- 172.21.15.120:0/1311114573 shutdown_connections 2024-09-19T09:10:36.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.598+0000 7f4453f67740 1 --2- 172.21.15.120:0/1311114573 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e212261260 0x55e212263650 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.598+0000 7f4453f67740 1 --2- 172.21.15.120:0/1311114573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e212251a40 0x55e21223e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.598+0000 7f4453f67740 1 --2- 172.21.15.120:0/1311114573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e212236170 0x55e21223df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.598+0000 7f4453f67740 1 --2- 172.21.15.120:0/1311114573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e212235850 0x55e21223da10 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.598+0000 7f442affd640 0 client.18147 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:36.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.598+0000 7f4453f67740 1 -- 172.21.15.120:0/1311114573 >> 172.21.15.120:0/1311114573 conn(0x55e212231de0 msgr2=0x55e212237ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:36.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.598+0000 7f4453f67740 1 -- 172.21.15.120:0/1311114573 shutdown_connections 2024-09-19T09:10:36.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.598+0000 7f4453f67740 1 -- 172.21.15.120:0/1311114573 wait complete. 2024-09-19T09:10:36.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.598+0000 7f4453f67740 20 client.18147 trim_cache size 0 max 16384 2024-09-19T09:10:36.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.618+0000 7f4453f67740 1 Processor -- start 2024-09-19T09:10:36.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.618+0000 7f4453f67740 1 -- start start 2024-09-19T09:10:36.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.618+0000 7f4453f67740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e212235850 0x55e2122382b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:36.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.618+0000 7f4453f67740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e212236170 0x55e212268d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:36.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.618+0000 7f4453f67740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e212251a40 0x55e212269280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:36.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.618+0000 7f4453f67740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f444c06faf0 con 0x55e212251a40 2024-09-19T09:10:36.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.618+0000 7f4453f67740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f444c071630 con 0x55e212235850 2024-09-19T09:10:36.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.618+0000 7f4453f67740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f444c0701c0 con 0x55e212236170 2024-09-19T09:10:36.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.618+0000 7f444bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e212251a40 0x55e212269280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:36.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.618+0000 7f444b7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e212235850 0x55e2122382b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:36.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.618+0000 7f444bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e212251a40 0x55e212269280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40254/0 (socket says 172.21.15.120:40254) 2024-09-19T09:10:36.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.618+0000 7f444bfff640 1 -- 172.21.15.120:0/3935556873 learned_addr learned my addr 172.21.15.120:0/3935556873 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:36.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.618+0000 7f444affd640 1 --2- 172.21.15.120:0/3935556873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e212236170 0x55e212268d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:36.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.618+0000 7f444bfff640 1 -- 172.21.15.120:0/3935556873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e212236170 msgr2=0x55e212268d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:36.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.618+0000 7f444bfff640 1 --2- 172.21.15.120:0/3935556873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e212236170 0x55e212268d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.618+0000 7f444bfff640 1 -- 172.21.15.120:0/3935556873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e212235850 msgr2=0x55e2122382b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:36.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.618+0000 7f444bfff640 1 --2- 172.21.15.120:0/3935556873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e212235850 0x55e2122382b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.618+0000 7f444bfff640 1 -- 172.21.15.120:0/3935556873 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e212235fa0 con 0x55e212251a40 2024-09-19T09:10:36.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.619+0000 7f444bfff640 1 --2- 172.21.15.120:0/3935556873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e212251a40 0x55e212269280 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f4444010850 tx=0x7f4444015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af875922ffcb3a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:36.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.619+0000 7f44317fa640 1 -- 172.21.15.120:0/3935556873 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4444016030 con 0x55e212251a40 2024-09-19T09:10:36.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.619+0000 7f44317fa640 1 -- 172.21.15.120:0/3935556873 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4444020980 con 0x55e212251a40 2024-09-19T09:10:36.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.619+0000 7f44317fa640 1 -- 172.21.15.120:0/3935556873 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4444019040 con 0x55e212251a40 2024-09-19T09:10:36.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.620+0000 7f4453f67740 1 -- 172.21.15.120:0/3935556873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e212251a40 msgr2=0x55e212269280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:36.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.620+0000 7f4453f67740 1 --2- 172.21.15.120:0/3935556873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e212251a40 0x55e212269280 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f4444010850 tx=0x7f4444015000 comp rx=0 tx=0).stop 2024-09-19T09:10:36.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.620+0000 7f4453f67740 1 -- 172.21.15.120:0/3935556873 shutdown_connections 2024-09-19T09:10:36.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.620+0000 7f4453f67740 1 --2- 172.21.15.120:0/3935556873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e212251a40 0x55e212269280 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.620+0000 7f4453f67740 1 --2- 172.21.15.120:0/3935556873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e212236170 0x55e212268d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.620+0000 7f4453f67740 1 --2- 172.21.15.120:0/3935556873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e212235850 0x55e2122382b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.620+0000 7f4453f67740 1 -- 172.21.15.120:0/3935556873 >> 172.21.15.120:0/3935556873 conn(0x55e212231de0 msgr2=0x55e21223e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:36.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.620+0000 7f4453f67740 1 -- 172.21.15.120:0/3935556873 shutdown_connections 2024-09-19T09:10:36.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.620+0000 7f4453f67740 1 -- 172.21.15.120:0/3935556873 wait complete. 2024-09-19T09:10:36.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.620+0000 7f4453f67740 1 -- start start 2024-09-19T09:10:36.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.620+0000 7f4453f67740 1 Processor -- start 2024-09-19T09:10:36.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.621+0000 7f4453f67740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e212235850 0x55e212255780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:36.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.621+0000 7f4453f67740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e212236170 0x55e212255cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:36.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.621+0000 7f4453f67740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e212251a40 0x55e212256200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:36.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.621+0000 7f4453f67740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f444c070f60 con 0x55e212235850 2024-09-19T09:10:36.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.621+0000 7f4453f67740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f444c070890 con 0x55e212236170 2024-09-19T09:10:36.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.621+0000 7f4453f67740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f444c06dab0 con 0x55e212251a40 2024-09-19T09:10:36.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.621+0000 7f4453f67740 10 client.0 _ll_register_callbacks cb 0x7f4440017bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:36.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.621+0000 7f444b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e212235850 0x55e212255780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:36.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.621+0000 7f444bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e212251a40 0x55e212256200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:36.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.621+0000 7f444b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e212235850 0x55e212255780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40270/0 (socket says 172.21.15.120:40270) 2024-09-19T09:10:36.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.621+0000 7f444bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e212251a40 0x55e212256200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:38132/0 (socket says 172.21.15.120:38132) 2024-09-19T09:10:36.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.621+0000 7f444affd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e212236170 0x55e212255cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:36.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.621+0000 7f444b7fe640 1 -- 172.21.15.120:0/3807974907 learned_addr learned my addr 172.21.15.120:0/3807974907 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:36.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.621+0000 7f444bfff640 1 -- 172.21.15.120:0/3807974907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e212236170 msgr2=0x55e212255cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:36.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.621+0000 7f444bfff640 1 --2- 172.21.15.120:0/3807974907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e212236170 0x55e212255cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.621+0000 7f444bfff640 1 -- 172.21.15.120:0/3807974907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e212235850 msgr2=0x55e212255780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:36.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.621+0000 7f444bfff640 1 --2- 172.21.15.120:0/3807974907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e212235850 0x55e212255780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.621+0000 7f444bfff640 1 -- 172.21.15.120:0/3807974907 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2122cb000 con 0x55e212251a40 2024-09-19T09:10:36.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.621+0000 7f444b7fe640 1 --2- 172.21.15.120:0/3807974907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e212235850 0x55e212255780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:10:36.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.622+0000 7f444bfff640 1 --2- 172.21.15.120:0/3807974907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e212251a40 0x55e212256200 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4444010710 tx=0x7f4444020d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9699f860529d936b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:36.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.622+0000 7f442affd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:10:36.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.622+0000 7f442affd640 1 -- 172.21.15.120:0/3807974907 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4444019040 con 0x55e212251a40 2024-09-19T09:10:36.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.622+0000 7f442affd640 1 -- 172.21.15.120:0/3807974907 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f444400fe40 con 0x55e212251a40 2024-09-19T09:10:36.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.622+0000 7f4453f67740 1 -- 172.21.15.120:0/3807974907 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4444020980 con 0x55e212251a40 2024-09-19T09:10:36.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.622+0000 7f442affd640 1 -- 172.21.15.120:0/3807974907 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4444004a40 con 0x55e212251a40 2024-09-19T09:10:36.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.622+0000 7f4453f67740 10 client.27220 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:36.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.622+0000 7f4453f67740 1 -- 172.21.15.120:0/3807974907 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e2122cc260 con 0x55e212251a40 2024-09-19T09:10:36.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.622+0000 7f442affd640 1 -- 172.21.15.120:0/3807974907 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4444020980 con 0x55e212251a40 2024-09-19T09:10:36.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.622+0000 7f442affd640 1 -- 172.21.15.120:0/3807974907 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f444402b480 con 0x55e212251a40 2024-09-19T09:10:36.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.622+0000 7f4453f67740 10 client.27220 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:36.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.622+0000 7f4453f67740 10 client.27220 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:10:36.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.622+0000 7f4453f67740 20 client.27220 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:36.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.622+0000 7f4453f67740 1 --2- 172.21.15.120:0/3807974907 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e2122cc940 0x55e2122ced60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:36.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.622+0000 7f4453f67740 4 client.27220 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}] 2024-09-19T09:10:36.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.622+0000 7f4453f67740 1 -- 172.21.15.120:0/3807974907 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}) -- 0x55e2121a1be0 con 0x55e2122cc940 2024-09-19T09:10:36.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.623+0000 7f444b7fe640 1 --2- 172.21.15.120:0/3807974907 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e2122cc940 0x55e2122ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:36.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.623+0000 7f442affd640 10 client.27220 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:36.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.623+0000 7f444b7fe640 1 --2- 172.21.15.120:0/3807974907 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e2122cc940 0x55e2122ced60 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d015382501fb3b1 server_cookie=9248bec902aa62b2 in_seq=0 out_seq=0 2024-09-19T09:10:36.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.697+0000 7f442affd640 1 -- 172.21.15.120:0/3807974907 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e2121a1be0 con 0x55e2122cc940 2024-09-19T09:10:36.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.697+0000 7f442affd640 10 client.27220 handle_command_reply: tid=0 2024-09-19T09:10:36.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.697+0000 7f4453f67740 1 client.27220 shutdown 2024-09-19T09:10:36.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.697+0000 7f4453f67740 1 -- 172.21.15.120:0/3807974907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e212251a40 msgr2=0x55e212256200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:36.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.697+0000 7f4453f67740 1 --2- 172.21.15.120:0/3807974907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e212251a40 0x55e212256200 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4444010710 tx=0x7f4444020d60 comp rx=0 tx=0).stop 2024-09-19T09:10:36.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.697+0000 7f4453f67740 1 -- 172.21.15.120:0/3807974907 shutdown_connections 2024-09-19T09:10:36.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.697+0000 7f4453f67740 1 --2- 172.21.15.120:0/3807974907 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e2122cc940 0x55e2122ced60 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.697+0000 7f4453f67740 1 --2- 172.21.15.120:0/3807974907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e212251a40 0x55e212256200 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.697+0000 7f442affd640 0 client.27220 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:36.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.697+0000 7f4453f67740 1 --2- 172.21.15.120:0/3807974907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e212236170 0x55e212255cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.697+0000 7f4453f67740 1 --2- 172.21.15.120:0/3807974907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e212235850 0x55e212255780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.697+0000 7f4453f67740 1 -- 172.21.15.120:0/3807974907 >> 172.21.15.120:0/3807974907 conn(0x55e212231de0 msgr2=0x55e2122306a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:36.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.697+0000 7f4453f67740 1 -- 172.21.15.120:0/3807974907 shutdown_connections 2024-09-19T09:10:36.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.697+0000 7f4453f67740 1 -- 172.21.15.120:0/3807974907 wait complete. 2024-09-19T09:10:36.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.697+0000 7f4453f67740 20 client.27220 trim_cache size 0 max 16384 2024-09-19T09:10:36.699 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:10:36.699 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 30, 2024-09-19T09:10:36.700 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:10:36.700 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 30, 2024-09-19T09:10:36.700 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:10:36.700 INFO:teuthology.orchestra.run.smithi120.stdout: "aeca8af5": { 2024-09-19T09:10:36.700 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 30, 2024-09-19T09:10:36.700 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:10:36.700 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:10:36.700 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:10:36.700 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:10:36.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:10:36.700 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:10:36.700 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 145.8, 2024-09-19T09:10:36.700 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:10:36.700 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:10:36.701 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:10:36.701 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:10:36.701 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:10:36.701 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:10:36.701 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:10:36.701 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:10:36.701 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:10:36.701 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:10:36.701 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:10:36.701 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:10:36.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.698+0000 7f4451b8e640 1 -- 172.21.15.120:0/426158225 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f44340a3240 msgr2=0x7f44340a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:36.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.698+0000 7f4451b8e640 1 --2- 172.21.15.120:0/426158225 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f44340a3240 0x7f44340a5700 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55e212232ed0 tx=0x7f4444009500 comp rx=0 tx=0).stop 2024-09-19T09:10:36.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.698+0000 7f4451b8e640 1 -- 172.21.15.120:0/426158225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f444c100b20 msgr2=0x7f444c19a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:36.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.698+0000 7f4451b8e640 1 --2- 172.21.15.120:0/426158225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f444c100b20 0x7f444c19a7e0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f443c006010 tx=0x7f443c004040 comp rx=0 tx=0).stop 2024-09-19T09:10:36.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.698+0000 7f4451b8e640 1 -- 172.21.15.120:0/426158225 shutdown_connections 2024-09-19T09:10:36.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.698+0000 7f4451b8e640 1 --2- 172.21.15.120:0/426158225 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f44340a3240 0x7f44340a5700 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.698+0000 7f4451b8e640 1 --2- 172.21.15.120:0/426158225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f444c10e500 0x7f444c19e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.698+0000 7f4451b8e640 1 --2- 172.21.15.120:0/426158225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f444c101440 0x7f444c19ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.698+0000 7f4451b8e640 1 --2- 172.21.15.120:0/426158225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f444c100b20 0x7f444c19a7e0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:36.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.698+0000 7f4451b8e640 1 -- 172.21.15.120:0/426158225 >> 172.21.15.120:0/426158225 conn(0x7f444c0fc7d0 msgr2=0x7f444c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:36.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.698+0000 7f4451b8e640 1 -- 172.21.15.120:0/426158225 shutdown_connections 2024-09-19T09:10:36.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:36.698+0000 7f4451b8e640 1 -- 172.21.15.120:0/426158225 wait complete. 2024-09-19T09:10:37.058 INFO:tasks.workunit.client.0.smithi120.stdout: 1 707664 9.57 MB/sec execute 272 sec latency 419.139 ms 2024-09-19T09:10:37.162 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:36 smithi120 ceph-mon[31150]: pgmap v574: 97 pgs: 97 active+clean; 37 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.8 MiB/s wr, 93 op/s 2024-09-19T09:10:37.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:36 smithi196 ceph-mon[35150]: pgmap v574: 97 pgs: 97 active+clean; 37 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.8 MiB/s wr, 93 op/s 2024-09-19T09:10:37.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:36 smithi153 ceph-mon[35071]: pgmap v574: 97 pgs: 97 active+clean; 37 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.8 MiB/s wr, 93 op/s 2024-09-19T09:10:37.598 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'aeca8af5', quiesce took 3.7 seconds. Will release after: 22.05 2024-09-19T09:10:38.058 INFO:tasks.workunit.client.0.smithi120.stdout: 1 707664 9.54 MB/sec execute 273 sec latency 1419.180 ms 2024-09-19T09:10:39.058 INFO:tasks.workunit.client.0.smithi120.stdout: 1 707664 9.50 MB/sec execute 274 sec latency 2419.264 ms 2024-09-19T09:10:39.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:38 smithi196 ceph-mon[35150]: pgmap v575: 97 pgs: 97 active+clean; 36 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 159 op/s 2024-09-19T09:10:39.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:38 smithi153 ceph-mon[35071]: pgmap v575: 97 pgs: 97 active+clean; 36 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 159 op/s 2024-09-19T09:10:39.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:38 smithi120 ceph-mon[31150]: pgmap v575: 97 pgs: 97 active+clean; 36 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 159 op/s 2024-09-19T09:10:40.058 INFO:tasks.workunit.client.0.smithi120.stdout: 1 707664 9.47 MB/sec execute 275 sec latency 3419.359 ms 2024-09-19T09:10:41.059 INFO:tasks.workunit.client.0.smithi120.stdout: 1 707664 9.43 MB/sec execute 276 sec latency 4419.410 ms 2024-09-19T09:10:41.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:40 smithi196 ceph-mon[35150]: pgmap v576: 97 pgs: 97 active+clean; 34 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 184 op/s 2024-09-19T09:10:41.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:40 smithi153 ceph-mon[35071]: pgmap v576: 97 pgs: 97 active+clean; 34 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 184 op/s 2024-09-19T09:10:41.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:40 smithi120 ceph-mon[31150]: pgmap v576: 97 pgs: 97 active+clean; 34 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 184 op/s 2024-09-19T09:10:42.059 INFO:tasks.workunit.client.0.smithi120.stdout: 1 707664 9.40 MB/sec execute 277 sec latency 5419.530 ms 2024-09-19T09:10:43.059 INFO:tasks.workunit.client.0.smithi120.stdout: 1 707664 9.37 MB/sec execute 278 sec latency 6419.614 ms 2024-09-19T09:10:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:42 smithi196 ceph-mon[35150]: pgmap v577: 97 pgs: 97 active+clean; 32 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 185 op/s 2024-09-19T09:10:43.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:42 smithi153 ceph-mon[35071]: pgmap v577: 97 pgs: 97 active+clean; 32 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 185 op/s 2024-09-19T09:10:43.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:42 smithi120 ceph-mon[31150]: pgmap v577: 97 pgs: 97 active+clean; 32 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 185 op/s 2024-09-19T09:10:44.059 INFO:tasks.workunit.client.0.smithi120.stdout: 1 707664 9.33 MB/sec execute 279 sec latency 7419.710 ms 2024-09-19T09:10:45.059 INFO:tasks.workunit.client.0.smithi120.stdout: 1 707664 9.30 MB/sec execute 280 sec latency 8419.827 ms 2024-09-19T09:10:45.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:44 smithi196 ceph-mon[35150]: pgmap v578: 97 pgs: 97 active+clean; 33 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 177 op/s 2024-09-19T09:10:45.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:44 smithi153 ceph-mon[35071]: pgmap v578: 97 pgs: 97 active+clean; 33 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 177 op/s 2024-09-19T09:10:45.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:44 smithi120 ceph-mon[31150]: pgmap v578: 97 pgs: 97 active+clean; 33 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 177 op/s 2024-09-19T09:10:45.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.653+0000 7f38da23c640 1 -- 172.21.15.120:0/2544246765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38d4101c60 msgr2=0x7f38d41020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:45.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.653+0000 7f38da23c640 1 --2- 172.21.15.120:0/2544246765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38d4101c60 0x7f38d41020e0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f38c8009900 tx=0x7f38c802a410 comp rx=0 tx=0).stop 2024-09-19T09:10:45.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.653+0000 7f38da23c640 1 -- 172.21.15.120:0/2544246765 shutdown_connections 2024-09-19T09:10:45.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.653+0000 7f38da23c640 1 --2- 172.21.15.120:0/2544246765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38d4102620 0x7f38d4108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.653+0000 7f38da23c640 1 --2- 172.21.15.120:0/2544246765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38d4101c60 0x7f38d41020e0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.653+0000 7f38da23c640 1 --2- 172.21.15.120:0/2544246765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38d4100a70 0x7f38d4100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.653+0000 7f38da23c640 1 -- 172.21.15.120:0/2544246765 >> 172.21.15.120:0/2544246765 conn(0x7f38d40fc1f0 msgr2=0x7f38d40fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.653+0000 7f38da23c640 1 -- 172.21.15.120:0/2544246765 shutdown_connections 2024-09-19T09:10:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.653+0000 7f38da23c640 1 -- 172.21.15.120:0/2544246765 wait complete. 2024-09-19T09:10:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.654+0000 7f38da23c640 1 Processor -- start 2024-09-19T09:10:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.654+0000 7f38da23c640 1 -- start start 2024-09-19T09:10:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.654+0000 7f38da23c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38d4100a70 0x7f38d4197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.654+0000 7f38da23c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38d4101c60 0x7f38d41982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.654+0000 7f38da23c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38d4102620 0x7f38d419f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.654+0000 7f38da23c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f38d410c720 con 0x7f38d4100a70 2024-09-19T09:10:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.654+0000 7f38da23c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f38d410c5a0 con 0x7f38d4101c60 2024-09-19T09:10:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.654+0000 7f38da23c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f38d410c8a0 con 0x7f38d4102620 2024-09-19T09:10:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.655+0000 7f38d37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38d4100a70 0x7f38d4197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.655+0000 7f38d37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38d4100a70 0x7f38d4197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43312/0 (socket says 172.21.15.120:43312) 2024-09-19T09:10:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.655+0000 7f38d37fe640 1 -- 172.21.15.120:0/215361136 learned_addr learned my addr 172.21.15.120:0/215361136 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.655+0000 7f38d2ffd640 1 --2- 172.21.15.120:0/215361136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38d4101c60 0x7f38d41982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.655+0000 7f38d37fe640 1 -- 172.21.15.120:0/215361136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38d4102620 msgr2=0x7f38d419f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.655+0000 7f38d37fe640 1 --2- 172.21.15.120:0/215361136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38d4102620 0x7f38d419f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.655+0000 7f38d37fe640 1 -- 172.21.15.120:0/215361136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38d4101c60 msgr2=0x7f38d41982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.655+0000 7f38d37fe640 1 --2- 172.21.15.120:0/215361136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38d4101c60 0x7f38d41982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.655+0000 7f38d37fe640 1 -- 172.21.15.120:0/215361136 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d419f9c0 con 0x7f38d4100a70 2024-09-19T09:10:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.655+0000 7f38d2ffd640 1 --2- 172.21.15.120:0/215361136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38d4101c60 0x7f38d41982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:10:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.655+0000 7f38d37fe640 1 --2- 172.21.15.120:0/215361136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38d4100a70 0x7f38d4197db0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f38c400c970 tx=0x7f38c400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.655+0000 7f38d0ff9640 1 -- 172.21.15.120:0/215361136 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c4004220 con 0x7f38d4100a70 2024-09-19T09:10:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.655+0000 7f38d0ff9640 1 -- 172.21.15.120:0/215361136 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38c40043c0 con 0x7f38d4100a70 2024-09-19T09:10:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.655+0000 7f38da23c640 1 -- 172.21.15.120:0/215361136 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38d419fcb0 con 0x7f38d4100a70 2024-09-19T09:10:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.655+0000 7f38d0ff9640 1 -- 172.21.15.120:0/215361136 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c400f450 con 0x7f38d4100a70 2024-09-19T09:10:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.655+0000 7f38da23c640 1 -- 172.21.15.120:0/215361136 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38d41a01b0 con 0x7f38d4100a70 2024-09-19T09:10:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.657+0000 7f38d0ff9640 1 -- 172.21.15.120:0/215361136 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f38c40079b0 con 0x7f38d4100a70 2024-09-19T09:10:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.658+0000 7f38dcfd4740 1 Processor -- start 2024-09-19T09:10:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.658+0000 7f38dcfd4740 1 -- start start 2024-09-19T09:10:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.658+0000 7f38dcfd4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ecccddb0 0x5637eccce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.658+0000 7f38dcfd4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637eccce6f0 0x5637eccd31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.658+0000 7f38dcfd4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637eccd3780 0x5637eccd5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.658+0000 7f38d0ff9640 1 --2- 172.21.15.120:0/215361136 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f38bc0a32f0 0x7f38bc0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.658+0000 7f38dcfd4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5637ecc2d2f0 con 0x5637ecccddb0 2024-09-19T09:10:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.658+0000 7f38dcfd4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5637ecbd5270 con 0x5637eccd3780 2024-09-19T09:10:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.658+0000 7f38dcfd4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5637ecc292b0 con 0x5637eccce6f0 2024-09-19T09:10:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.658+0000 7f38d3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ecccddb0 0x5637eccce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.658+0000 7f38d2ffd640 1 --2- 172.21.15.120:0/215361136 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f38bc0a32f0 0x7f38bc0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.658+0000 7f38d3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ecccddb0 0x5637eccce1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43326/0 (socket says 172.21.15.120:43326) 2024-09-19T09:10:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.658+0000 7f38d3fff640 1 -- 172.21.15.120:0/2700250117 learned_addr learned my addr 172.21.15.120:0/2700250117 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.658+0000 7f38d37fe640 1 --2- 172.21.15.120:0/2700250117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637eccce6f0 0x5637eccd31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.658+0000 7f38d0ff9640 1 -- 172.21.15.120:0/215361136 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f38c40cf7e0 con 0x7f38d4100a70 2024-09-19T09:10:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.659+0000 7f38d3fff640 1 --2- 172.21.15.120:0/2700250117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637eccd3780 0x5637eccd5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.659+0000 7f38d3fff640 1 -- 172.21.15.120:0/2700250117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637eccce6f0 msgr2=0x5637eccd31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.659+0000 7f38d3fff640 1 --2- 172.21.15.120:0/2700250117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637eccce6f0 0x5637eccd31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.659+0000 7f38d3fff640 1 -- 172.21.15.120:0/2700250117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637eccd3780 msgr2=0x5637eccd5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.659+0000 7f38d3fff640 1 --2- 172.21.15.120:0/2700250117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637eccd3780 0x5637eccd5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.659+0000 7f38d3fff640 1 -- 172.21.15.120:0/2700250117 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637ecbd4340 con 0x5637ecccddb0 2024-09-19T09:10:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.659+0000 7f38d2ffd640 1 --2- 172.21.15.120:0/215361136 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f38bc0a32f0 0x7f38bc0a57b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f38c802a930 tx=0x7f38c8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.659+0000 7f38d3fff640 1 --2- 172.21.15.120:0/2700250117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ecccddb0 0x5637eccce1b0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f38cc0136d0 tx=0x7f38cc007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c092ce0849e0fc55 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.659+0000 7f38b9ffb640 1 -- 172.21.15.120:0/2700250117 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc004d20 con 0x5637ecccddb0 2024-09-19T09:10:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.659+0000 7f38b9ffb640 1 -- 172.21.15.120:0/2700250117 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38cc00ecd0 con 0x5637ecccddb0 2024-09-19T09:10:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.659+0000 7f38b9ffb640 1 -- 172.21.15.120:0/2700250117 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc005740 con 0x5637ecccddb0 2024-09-19T09:10:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.660+0000 7f38dcfd4740 1 -- 172.21.15.120:0/2700250117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ecccddb0 msgr2=0x5637eccce1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.660+0000 7f38dcfd4740 1 --2- 172.21.15.120:0/2700250117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ecccddb0 0x5637eccce1b0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f38cc0136d0 tx=0x7f38cc007720 comp rx=0 tx=0).stop 2024-09-19T09:10:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.660+0000 7f38dcfd4740 1 -- 172.21.15.120:0/2700250117 shutdown_connections 2024-09-19T09:10:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.660+0000 7f38dcfd4740 1 --2- 172.21.15.120:0/2700250117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637eccd3780 0x5637eccd5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.660+0000 7f38dcfd4740 1 --2- 172.21.15.120:0/2700250117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637eccce6f0 0x5637eccd31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.660+0000 7f38dcfd4740 1 --2- 172.21.15.120:0/2700250117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ecccddb0 0x5637eccce1b0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.660+0000 7f38dcfd4740 1 -- 172.21.15.120:0/2700250117 >> 172.21.15.120:0/2700250117 conn(0x5637eccc5010 msgr2=0x5637eccc5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.660+0000 7f38dcfd4740 1 -- 172.21.15.120:0/2700250117 shutdown_connections 2024-09-19T09:10:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.660+0000 7f38dcfd4740 1 -- 172.21.15.120:0/2700250117 wait complete. 2024-09-19T09:10:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.660+0000 7f38dcfd4740 1 -- start start 2024-09-19T09:10:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.661+0000 7f38dcfd4740 1 Processor -- start 2024-09-19T09:10:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.661+0000 7f38dcfd4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637ecccddb0 0x5637eccd7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.661+0000 7f38dcfd4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637eccce6f0 0x5637eccd8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.661+0000 7f38dcfd4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637eccd3780 0x5637eccdf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.661+0000 7f38dcfd4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5637eccd6750 con 0x5637eccce6f0 2024-09-19T09:10:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.661+0000 7f38dcfd4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5637ecbb6290 con 0x5637eccd3780 2024-09-19T09:10:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.661+0000 7f38dcfd4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5637eccd68d0 con 0x5637ecccddb0 2024-09-19T09:10:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.661+0000 7f38dcfd4740 10 client.0 _ll_register_callbacks cb 0x5637ecbe7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.661+0000 7f38d2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637eccce6f0 0x5637eccd8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.661+0000 7f38d2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637eccce6f0 0x5637eccd8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43340/0 (socket says 172.21.15.120:43340) 2024-09-19T09:10:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.661+0000 7f38d2ffd640 1 -- 172.21.15.120:0/4207722115 learned_addr learned my addr 172.21.15.120:0/4207722115 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.661+0000 7f38d37fe640 1 --2- 172.21.15.120:0/4207722115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637ecccddb0 0x5637eccd7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.661+0000 7f38d3fff640 1 --2- 172.21.15.120:0/4207722115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637eccd3780 0x5637eccdf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.661+0000 7f38d2ffd640 1 -- 172.21.15.120:0/4207722115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637ecccddb0 msgr2=0x5637eccd7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.661+0000 7f38d2ffd640 1 --2- 172.21.15.120:0/4207722115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637ecccddb0 0x5637eccd7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.661+0000 7f38d2ffd640 1 -- 172.21.15.120:0/4207722115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637eccd3780 msgr2=0x5637eccdf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.661+0000 7f38d2ffd640 1 --2- 172.21.15.120:0/4207722115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637eccd3780 0x5637eccdf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.661+0000 7f38d2ffd640 1 -- 172.21.15.120:0/4207722115 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637ec940210 con 0x5637eccce6f0 2024-09-19T09:10:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.661+0000 7f38d3fff640 1 --2- 172.21.15.120:0/4207722115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637eccd3780 0x5637eccdf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:10:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.662+0000 7f38d2ffd640 1 --2- 172.21.15.120:0/4207722115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637eccce6f0 0x5637eccd8510 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f38c80052c0 tx=0x7f38c8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b5903b574a7fb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.662+0000 7f38b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.662+0000 7f38b2ffd640 1 -- 172.21.15.120:0/4207722115 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8041070 con 0x5637eccce6f0 2024-09-19T09:10:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.662+0000 7f38b2ffd640 1 -- 172.21.15.120:0/4207722115 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f38c80048c0 con 0x5637eccce6f0 2024-09-19T09:10:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.662+0000 7f38dcfd4740 1 -- 172.21.15.120:0/4207722115 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38cc00ecd0 con 0x5637eccce6f0 2024-09-19T09:10:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.662+0000 7f38b2ffd640 1 -- 172.21.15.120:0/4207722115 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f38c8035050 con 0x5637eccce6f0 2024-09-19T09:10:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.662+0000 7f38dcfd4740 10 client.18165 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.662+0000 7f38dcfd4740 1 -- 172.21.15.120:0/4207722115 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5637eca0c620 con 0x5637eccce6f0 2024-09-19T09:10:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.662+0000 7f38b2ffd640 1 -- 172.21.15.120:0/4207722115 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38cc00ecd0 con 0x5637eccce6f0 2024-09-19T09:10:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.662+0000 7f38b2ffd640 1 -- 172.21.15.120:0/4207722115 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f38c80035f0 con 0x5637eccce6f0 2024-09-19T09:10:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.662+0000 7f38dcfd4740 10 client.18165 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.662+0000 7f38dcfd4740 10 client.18165 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:10:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.662+0000 7f38dcfd4740 20 client.18165 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.662+0000 7f38dcfd4740 1 --2- 172.21.15.120:0/4207722115 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5637ecce3cb0 0x5637ecce6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.662+0000 7f38dcfd4740 4 client.18165 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:10:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.662+0000 7f38dcfd4740 1 -- 172.21.15.120:0/4207722115 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637ecc3aa20 con 0x5637ecce3cb0 2024-09-19T09:10:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.663+0000 7f38d37fe640 1 --2- 172.21.15.120:0/4207722115 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5637ecce3cb0 0x5637ecce6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.663+0000 7f38b2ffd640 10 client.18165 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.663+0000 7f38d37fe640 1 --2- 172.21.15.120:0/4207722115 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5637ecce3cb0 0x5637ecce6100 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adac84097b730ee server_cookie=229319fc44d21d9b in_seq=0 out_seq=0 2024-09-19T09:10:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.664+0000 7f38b2ffd640 1 -- 172.21.15.120:0/4207722115 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5637ecc3aa20 con 0x5637ecce3cb0 2024-09-19T09:10:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.664+0000 7f38b2ffd640 10 client.18165 handle_command_reply: tid=0 2024-09-19T09:10:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.664+0000 7f38dcfd4740 1 client.18165 shutdown 2024-09-19T09:10:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.664+0000 7f38dcfd4740 1 -- 172.21.15.120:0/4207722115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637eccce6f0 msgr2=0x5637eccd8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.664+0000 7f38dcfd4740 1 --2- 172.21.15.120:0/4207722115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637eccce6f0 0x5637eccd8510 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f38c80052c0 tx=0x7f38c8009260 comp rx=0 tx=0).stop 2024-09-19T09:10:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.664+0000 7f38dcfd4740 1 -- 172.21.15.120:0/4207722115 shutdown_connections 2024-09-19T09:10:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.664+0000 7f38dcfd4740 1 --2- 172.21.15.120:0/4207722115 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5637ecce3cb0 0x5637ecce6100 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.664+0000 7f38dcfd4740 1 --2- 172.21.15.120:0/4207722115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637eccd3780 0x5637eccdf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.664+0000 7f38dcfd4740 1 --2- 172.21.15.120:0/4207722115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637eccce6f0 0x5637eccd8510 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.664+0000 7f38dcfd4740 1 --2- 172.21.15.120:0/4207722115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637ecccddb0 0x5637eccd7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.664+0000 7f38b2ffd640 0 client.18165 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.664+0000 7f38dcfd4740 1 -- 172.21.15.120:0/4207722115 >> 172.21.15.120:0/4207722115 conn(0x5637eccc5010 msgr2=0x5637eccc6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.664+0000 7f38dcfd4740 1 -- 172.21.15.120:0/4207722115 shutdown_connections 2024-09-19T09:10:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.664+0000 7f38dcfd4740 1 -- 172.21.15.120:0/4207722115 wait complete. 2024-09-19T09:10:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.664+0000 7f38dcfd4740 20 client.18165 trim_cache size 0 max 16384 2024-09-19T09:10:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.679+0000 7f38dcfd4740 1 Processor -- start 2024-09-19T09:10:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.679+0000 7f38dcfd4740 1 -- start start 2024-09-19T09:10:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.679+0000 7f38dcfd4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637ecccddb0 0x5637eccc6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.679+0000 7f38dcfd4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637eccce6f0 0x5637eccdf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.679+0000 7f38dcfd4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637eccd3780 0x5637eccc6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.679+0000 7f38dcfd4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f38d41a6e60 con 0x5637eccd3780 2024-09-19T09:10:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.679+0000 7f38dcfd4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f38d41a89a0 con 0x5637eccce6f0 2024-09-19T09:10:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.679+0000 7f38dcfd4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f38d41a7530 con 0x5637ecccddb0 2024-09-19T09:10:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.679+0000 7f38d3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637eccd3780 0x5637eccc6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.679+0000 7f38d3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637eccd3780 0x5637eccc6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43354/0 (socket says 172.21.15.120:43354) 2024-09-19T09:10:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.679+0000 7f38d3fff640 1 -- 172.21.15.120:0/166777498 learned_addr learned my addr 172.21.15.120:0/166777498 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.679+0000 7f38d3fff640 1 -- 172.21.15.120:0/166777498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637ecccddb0 msgr2=0x5637eccc6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.679+0000 7f38d37fe640 1 --2- 172.21.15.120:0/166777498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637ecccddb0 0x5637eccc6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.679+0000 7f38d3fff640 1 --2- 172.21.15.120:0/166777498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637ecccddb0 0x5637eccc6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.679+0000 7f38d3fff640 1 -- 172.21.15.120:0/166777498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637eccce6f0 msgr2=0x5637eccdf510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:10:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.679+0000 7f38d3fff640 1 --2- 172.21.15.120:0/166777498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637eccce6f0 0x5637eccdf510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.679+0000 7f38d3fff640 1 -- 172.21.15.120:0/166777498 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637ecccebd0 con 0x5637eccd3780 2024-09-19T09:10:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.679+0000 7f38d37fe640 1 --2- 172.21.15.120:0/166777498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637ecccddb0 0x5637eccc6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:10:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.680+0000 7f38d3fff640 1 --2- 172.21.15.120:0/166777498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637eccd3780 0x5637eccc6940 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f38cc006010 tx=0x7f38cc0051b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b36c07a026d2ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.680+0000 7f38b97fa640 1 -- 172.21.15.120:0/166777498 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc02c070 con 0x5637eccd3780 2024-09-19T09:10:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.680+0000 7f38b97fa640 1 -- 172.21.15.120:0/166777498 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38cc007cf0 con 0x5637eccd3780 2024-09-19T09:10:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.680+0000 7f38b97fa640 1 -- 172.21.15.120:0/166777498 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc00ea50 con 0x5637eccd3780 2024-09-19T09:10:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.680+0000 7f38dcfd4740 1 -- 172.21.15.120:0/166777498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637eccd3780 msgr2=0x5637eccc6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.680+0000 7f38dcfd4740 1 --2- 172.21.15.120:0/166777498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637eccd3780 0x5637eccc6940 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f38cc006010 tx=0x7f38cc0051b0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.680+0000 7f38dcfd4740 1 -- 172.21.15.120:0/166777498 shutdown_connections 2024-09-19T09:10:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.680+0000 7f38dcfd4740 1 --2- 172.21.15.120:0/166777498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637eccd3780 0x5637eccc6940 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.680+0000 7f38dcfd4740 1 --2- 172.21.15.120:0/166777498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637eccce6f0 0x5637eccdf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.680+0000 7f38dcfd4740 1 --2- 172.21.15.120:0/166777498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637ecccddb0 0x5637eccc6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.680+0000 7f38dcfd4740 1 -- 172.21.15.120:0/166777498 >> 172.21.15.120:0/166777498 conn(0x5637eccc5010 msgr2=0x5637eccc41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.680+0000 7f38dcfd4740 1 -- 172.21.15.120:0/166777498 shutdown_connections 2024-09-19T09:10:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.680+0000 7f38dcfd4740 1 -- 172.21.15.120:0/166777498 wait complete. 2024-09-19T09:10:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.681+0000 7f38dcfd4740 1 -- start start 2024-09-19T09:10:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.681+0000 7f38dcfd4740 1 Processor -- start 2024-09-19T09:10:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.681+0000 7f38dcfd4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637ecccddb0 0x5637eccdd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.681+0000 7f38dcfd4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637eccce6f0 0x5637eccdd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.681+0000 7f38dcfd4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637eccd3780 0x5637eccde680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.681+0000 7f38dcfd4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f38d41a82d0 con 0x5637eccd3780 2024-09-19T09:10:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.681+0000 7f38dcfd4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f38d41a7c00 con 0x5637eccce6f0 2024-09-19T09:10:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.681+0000 7f38dcfd4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f38d41a4e20 con 0x5637ecccddb0 2024-09-19T09:10:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.681+0000 7f38dcfd4740 10 client.0 _ll_register_callbacks cb 0x7f38c8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.681+0000 7f38d3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637eccd3780 0x5637eccde680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.681+0000 7f38d3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637eccd3780 0x5637eccde680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43366/0 (socket says 172.21.15.120:43366) 2024-09-19T09:10:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.681+0000 7f38d3fff640 1 -- 172.21.15.120:0/2312986870 learned_addr learned my addr 172.21.15.120:0/2312986870 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.681+0000 7f38d37fe640 1 --2- 172.21.15.120:0/2312986870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637ecccddb0 0x5637eccdd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.681+0000 7f38d3fff640 1 -- 172.21.15.120:0/2312986870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637ecccddb0 msgr2=0x5637eccdd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.681+0000 7f38d3fff640 1 --2- 172.21.15.120:0/2312986870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637ecccddb0 0x5637eccdd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.681+0000 7f38d3fff640 1 -- 172.21.15.120:0/2312986870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637eccce6f0 msgr2=0x5637eccdd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.681+0000 7f38d3fff640 1 --2- 172.21.15.120:0/2312986870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637eccce6f0 0x5637eccdd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.681+0000 7f38d3fff640 1 -- 172.21.15.120:0/2312986870 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637eccdec90 con 0x5637eccd3780 2024-09-19T09:10:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.682+0000 7f38d3fff640 1 --2- 172.21.15.120:0/2312986870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637eccd3780 0x5637eccde680 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f38cc02a040 tx=0x7f38cc007c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806686ba14966f04 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.682+0000 7f38b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.682+0000 7f38b2ffd640 1 -- 172.21.15.120:0/2312986870 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc02c040 con 0x5637eccd3780 2024-09-19T09:10:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.682+0000 7f38b2ffd640 1 -- 172.21.15.120:0/2312986870 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f38cc0052d0 con 0x5637eccd3780 2024-09-19T09:10:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.682+0000 7f38dcfd4740 1 -- 172.21.15.120:0/2312986870 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38cc007cf0 con 0x5637eccd3780 2024-09-19T09:10:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.682+0000 7f38b2ffd640 1 -- 172.21.15.120:0/2312986870 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f38cc007ea0 con 0x5637eccd3780 2024-09-19T09:10:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.682+0000 7f38dcfd4740 10 client.18171 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.682+0000 7f38dcfd4740 1 -- 172.21.15.120:0/2312986870 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5637ecd4ed90 con 0x5637eccd3780 2024-09-19T09:10:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.682+0000 7f38b2ffd640 1 -- 172.21.15.120:0/2312986870 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38cc007cf0 con 0x5637eccd3780 2024-09-19T09:10:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.682+0000 7f38b2ffd640 1 -- 172.21.15.120:0/2312986870 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f38cc039020 con 0x5637eccd3780 2024-09-19T09:10:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.682+0000 7f38dcfd4740 10 client.18171 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.682+0000 7f38dcfd4740 10 client.18171 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:10:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.682+0000 7f38dcfd4740 20 client.18171 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.682+0000 7f38dcfd4740 1 --2- 172.21.15.120:0/2312986870 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5637ecd4f4f0 0x5637ecd51910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.682+0000 7f38dcfd4740 4 client.18171 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:10:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.682+0000 7f38dcfd4740 1 -- 172.21.15.120:0/2312986870 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5637ecc34d70 con 0x5637ecd4f4f0 2024-09-19T09:10:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.683+0000 7f38d37fe640 1 --2- 172.21.15.120:0/2312986870 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5637ecd4f4f0 0x5637ecd51910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.683+0000 7f38d37fe640 1 --2- 172.21.15.120:0/2312986870 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5637ecd4f4f0 0x5637ecd51910 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa14bc39c674ad server_cookie=f221f3eadda58d6f in_seq=0 out_seq=0 2024-09-19T09:10:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.683+0000 7f38b2ffd640 10 client.18171 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:45.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.759+0000 7f38b2ffd640 1 -- 172.21.15.120:0/2312986870 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5637ecc34d70 con 0x5637ecd4f4f0 2024-09-19T09:10:45.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.759+0000 7f38b2ffd640 10 client.18171 handle_command_reply: tid=0 2024-09-19T09:10:45.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.759+0000 7f38dcfd4740 1 client.18171 shutdown 2024-09-19T09:10:45.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.759+0000 7f38dcfd4740 1 -- 172.21.15.120:0/2312986870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637eccd3780 msgr2=0x5637eccde680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:45.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.759+0000 7f38dcfd4740 1 --2- 172.21.15.120:0/2312986870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637eccd3780 0x5637eccde680 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f38cc02a040 tx=0x7f38cc007c20 comp rx=0 tx=0).stop 2024-09-19T09:10:45.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.759+0000 7f38dcfd4740 1 -- 172.21.15.120:0/2312986870 shutdown_connections 2024-09-19T09:10:45.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.759+0000 7f38dcfd4740 1 --2- 172.21.15.120:0/2312986870 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5637ecd4f4f0 0x5637ecd51910 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.759+0000 7f38dcfd4740 1 --2- 172.21.15.120:0/2312986870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637eccd3780 0x5637eccde680 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.759+0000 7f38dcfd4740 1 --2- 172.21.15.120:0/2312986870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637eccce6f0 0x5637eccdd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.759+0000 7f38b2ffd640 0 client.18171 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:45.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.759+0000 7f38dcfd4740 1 --2- 172.21.15.120:0/2312986870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637ecccddb0 0x5637eccdd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.759+0000 7f38dcfd4740 1 -- 172.21.15.120:0/2312986870 >> 172.21.15.120:0/2312986870 conn(0x5637eccc5010 msgr2=0x5637eccc53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:45.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.759+0000 7f38dcfd4740 1 -- 172.21.15.120:0/2312986870 shutdown_connections 2024-09-19T09:10:45.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.759+0000 7f38dcfd4740 1 -- 172.21.15.120:0/2312986870 wait complete. 2024-09-19T09:10:45.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.759+0000 7f38dcfd4740 20 client.18171 trim_cache size 0 max 16384 2024-09-19T09:10:45.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:10:45.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:10:45.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:10:45.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:10:45.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.760+0000 7f38da23c640 1 -- 172.21.15.120:0/215361136 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f38bc0a32f0 msgr2=0x7f38bc0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.760+0000 7f38da23c640 1 --2- 172.21.15.120:0/215361136 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f38bc0a32f0 0x7f38bc0a57b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f38c802a930 tx=0x7f38c8000bc0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.760+0000 7f38da23c640 1 -- 172.21.15.120:0/215361136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38d4100a70 msgr2=0x7f38d4197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.760+0000 7f38da23c640 1 --2- 172.21.15.120:0/215361136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38d4100a70 0x7f38d4197db0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f38c400c970 tx=0x7f38c400ce30 comp rx=0 tx=0).stop 2024-09-19T09:10:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.760+0000 7f38da23c640 1 -- 172.21.15.120:0/215361136 shutdown_connections 2024-09-19T09:10:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.760+0000 7f38da23c640 1 --2- 172.21.15.120:0/215361136 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f38bc0a32f0 0x7f38bc0a57b0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.760+0000 7f38da23c640 1 --2- 172.21.15.120:0/215361136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38d4102620 0x7f38d419f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.760+0000 7f38da23c640 1 --2- 172.21.15.120:0/215361136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38d4101c60 0x7f38d41982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.760+0000 7f38da23c640 1 --2- 172.21.15.120:0/215361136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38d4100a70 0x7f38d4197db0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.760+0000 7f38da23c640 1 -- 172.21.15.120:0/215361136 >> 172.21.15.120:0/215361136 conn(0x7f38d40fc1f0 msgr2=0x7f38d40fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.760+0000 7f38da23c640 1 -- 172.21.15.120:0/215361136 shutdown_connections 2024-09-19T09:10:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:45.760+0000 7f38da23c640 1 -- 172.21.15.120:0/215361136 wait complete. 2024-09-19T09:10:46.060 INFO:tasks.workunit.client.0.smithi120.stdout: 1 707664 9.27 MB/sec execute 281 sec latency 9419.968 ms 2024-09-19T09:10:47.059 INFO:tasks.workunit.client.0.smithi120.stdout: 1 707664 9.23 MB/sec execute 282 sec latency 10420.058 ms 2024-09-19T09:10:47.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:46 smithi196 ceph-mon[35150]: pgmap v579: 97 pgs: 97 active+clean; 33 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 154 op/s 2024-09-19T09:10:47.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:46 smithi153 ceph-mon[35071]: pgmap v579: 97 pgs: 97 active+clean; 33 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 154 op/s 2024-09-19T09:10:47.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:46 smithi120 ceph-mon[31150]: pgmap v579: 97 pgs: 97 active+clean; 33 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 154 op/s 2024-09-19T09:10:48.059 INFO:tasks.workunit.client.0.smithi120.stdout: 1 707664 9.20 MB/sec execute 283 sec latency 11420.106 ms 2024-09-19T09:10:49.059 INFO:tasks.workunit.client.0.smithi120.stdout: 1 707664 9.17 MB/sec execute 284 sec latency 12420.207 ms 2024-09-19T09:10:49.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:48 smithi196 ceph-mon[35150]: pgmap v580: 97 pgs: 97 active+clean; 26 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 160 op/s 2024-09-19T09:10:49.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:48 smithi153 ceph-mon[35071]: pgmap v580: 97 pgs: 97 active+clean; 26 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 160 op/s 2024-09-19T09:10:49.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:48 smithi120 ceph-mon[31150]: pgmap v580: 97 pgs: 97 active+clean; 26 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 160 op/s 2024-09-19T09:10:50.059 INFO:tasks.workunit.client.0.smithi120.stdout: 1 707664 9.14 MB/sec execute 285 sec latency 13420.318 ms 2024-09-19T09:10:51.060 INFO:tasks.workunit.client.0.smithi120.stdout: 1 707664 9.10 MB/sec execute 286 sec latency 14420.384 ms 2024-09-19T09:10:51.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:50 smithi153 ceph-mon[35071]: pgmap v581: 97 pgs: 97 active+clean; 26 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-09-19T09:10:51.189 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:50 smithi196 ceph-mon[35150]: pgmap v581: 97 pgs: 97 active+clean; 26 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-09-19T09:10:51.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:50 smithi120 ceph-mon[31150]: pgmap v581: 97 pgs: 97 active+clean; 26 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-09-19T09:10:52.060 INFO:tasks.workunit.client.0.smithi120.stdout: 1 707664 9.07 MB/sec execute 287 sec latency 15420.490 ms 2024-09-19T09:10:53.060 INFO:tasks.workunit.client.0.smithi120.stdout: 1 707664 9.04 MB/sec execute 288 sec latency 16420.591 ms 2024-09-19T09:10:53.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:52 smithi196 ceph-mon[35150]: pgmap v582: 97 pgs: 97 active+clean; 26 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 480 KiB/s wr, 75 op/s 2024-09-19T09:10:53.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:52 smithi153 ceph-mon[35071]: pgmap v582: 97 pgs: 97 active+clean; 26 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 480 KiB/s wr, 75 op/s 2024-09-19T09:10:53.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:52 smithi120 ceph-mon[31150]: pgmap v582: 97 pgs: 97 active+clean; 26 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 480 KiB/s wr, 75 op/s 2024-09-19T09:10:53.677 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:10:53.938 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:10:54.060 INFO:tasks.workunit.client.0.smithi120.stdout: 1 707664 9.01 MB/sec execute 289 sec latency 17420.619 ms 2024-09-19T09:10:54.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.429+0000 7f7eddb92640 1 -- 172.21.15.120:0/1925745346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ed8100a80 msgr2=0x7f7ed8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:54.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.429+0000 7f7eddb92640 1 --2- 172.21.15.120:0/1925745346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ed8100a80 0x7f7ed8100ee0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc009930 tx=0x7f7ecc02a420 comp rx=0 tx=0).stop 2024-09-19T09:10:54.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.429+0000 7f7eddb92640 1 -- 172.21.15.120:0/1925745346 shutdown_connections 2024-09-19T09:10:54.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.429+0000 7f7eddb92640 1 --2- 172.21.15.120:0/1925745346 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ed8101420 0x7f7ed810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.429+0000 7f7eddb92640 1 --2- 172.21.15.120:0/1925745346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ed8100a80 0x7f7ed8100ee0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.429+0000 7f7eddb92640 1 --2- 172.21.15.120:0/1925745346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ed8106a80 0x7f7ed8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.429+0000 7f7eddb92640 1 -- 172.21.15.120:0/1925745346 >> 172.21.15.120:0/1925745346 conn(0x7f7ed80fc7a0 msgr2=0x7f7ed80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:54.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.430+0000 7f7eddb92640 1 -- 172.21.15.120:0/1925745346 shutdown_connections 2024-09-19T09:10:54.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.430+0000 7f7eddb92640 1 -- 172.21.15.120:0/1925745346 wait complete. 2024-09-19T09:10:54.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.430+0000 7f7eddb92640 1 Processor -- start 2024-09-19T09:10:54.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.430+0000 7f7eddb92640 1 -- start start 2024-09-19T09:10:54.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.430+0000 7f7eddb92640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ed8100a80 0x7f7ed81985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:54.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.430+0000 7f7eddb92640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ed8101420 0x7f7ed8198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:54.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.430+0000 7f7eddb92640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ed8106a80 0x7f7ed819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:54.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.430+0000 7f7eddb92640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed8112020 con 0x7f7ed8100a80 2024-09-19T09:10:54.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.430+0000 7f7eddb92640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed8111ea0 con 0x7f7ed8106a80 2024-09-19T09:10:54.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.430+0000 7f7eddb92640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed81121a0 con 0x7f7ed8101420 2024-09-19T09:10:54.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.431+0000 7f7ed77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ed8100a80 0x7f7ed81985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:54.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.431+0000 7f7ed77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ed8100a80 0x7f7ed81985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60596/0 (socket says 172.21.15.120:60596) 2024-09-19T09:10:54.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.431+0000 7f7ed77fe640 1 -- 172.21.15.120:0/2830935788 learned_addr learned my addr 172.21.15.120:0/2830935788 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:54.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.431+0000 7f7ed6ffd640 1 --2- 172.21.15.120:0/2830935788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ed8101420 0x7f7ed8198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:54.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.431+0000 7f7ed7fff640 1 --2- 172.21.15.120:0/2830935788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ed8106a80 0x7f7ed819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:54.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.431+0000 7f7ed77fe640 1 -- 172.21.15.120:0/2830935788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ed8101420 msgr2=0x7f7ed8198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:54.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.431+0000 7f7ed77fe640 1 --2- 172.21.15.120:0/2830935788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ed8101420 0x7f7ed8198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.431+0000 7f7ed77fe640 1 -- 172.21.15.120:0/2830935788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ed8106a80 msgr2=0x7f7ed819ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:54.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.431+0000 7f7ed77fe640 1 --2- 172.21.15.120:0/2830935788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ed8106a80 0x7f7ed819ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.431+0000 7f7ed77fe640 1 -- 172.21.15.120:0/2830935788 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed819d570 con 0x7f7ed8100a80 2024-09-19T09:10:54.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.431+0000 7f7ed6ffd640 1 --2- 172.21.15.120:0/2830935788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ed8101420 0x7f7ed8198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:10:54.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.431+0000 7f7ed7fff640 1 --2- 172.21.15.120:0/2830935788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ed8106a80 0x7f7ed819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:10:54.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.431+0000 7f7ed77fe640 1 --2- 172.21.15.120:0/2830935788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ed8100a80 0x7f7ed81985c0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800b4a0 tx=0x7f7ec800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:54.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.431+0000 7f7ed4ff9640 1 -- 172.21.15.120:0/2830935788 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8014070 con 0x7f7ed8100a80 2024-09-19T09:10:54.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.431+0000 7f7ed4ff9640 1 -- 172.21.15.120:0/2830935788 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ec8010af0 con 0x7f7ed8100a80 2024-09-19T09:10:54.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.431+0000 7f7eddb92640 1 -- 172.21.15.120:0/2830935788 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed819d860 con 0x7f7ed8100a80 2024-09-19T09:10:54.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.431+0000 7f7ed4ff9640 1 -- 172.21.15.120:0/2830935788 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8017d10 con 0x7f7ed8100a80 2024-09-19T09:10:54.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.431+0000 7f7eddb92640 1 -- 172.21.15.120:0/2830935788 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed81a5140 con 0x7f7ed8100a80 2024-09-19T09:10:54.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.434+0000 7f7ed4ff9640 1 -- 172.21.15.120:0/2830935788 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f7ec8017070 con 0x7f7ed8100a80 2024-09-19T09:10:54.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.434+0000 7f7edff6b740 1 Processor -- start 2024-09-19T09:10:54.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.434+0000 7f7edff6b740 1 -- start start 2024-09-19T09:10:54.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.434+0000 7f7edff6b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beebdd94a0 0x55beebdd9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:54.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.434+0000 7f7edff6b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beebdd9dc0 0x55beebde4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:54.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.434+0000 7f7edff6b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55beebde4680 0x55beebde6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:54.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.434+0000 7f7edff6b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55beebd3f2e0 con 0x55beebdd94a0 2024-09-19T09:10:54.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.434+0000 7f7edff6b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55beebce6ce0 con 0x55beebdd9dc0 2024-09-19T09:10:54.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.434+0000 7f7edff6b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55beebd3b2a0 con 0x55beebde4680 2024-09-19T09:10:54.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.435+0000 7f7ed4ff9640 1 --2- 172.21.15.120:0/2830935788 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7ec00a3240 0x7f7ec00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:54.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.435+0000 7f7ed7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beebdd94a0 0x55beebdd9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:54.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.435+0000 7f7ed7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beebdd94a0 0x55beebdd9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60612/0 (socket says 172.21.15.120:60612) 2024-09-19T09:10:54.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.435+0000 7f7ed7fff640 1 -- 172.21.15.120:0/139542082 learned_addr learned my addr 172.21.15.120:0/139542082 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:54.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.435+0000 7f7ed7fff640 1 --2- 172.21.15.120:0/2830935788 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7ec00a3240 0x7f7ec00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:54.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.435+0000 7f7ed4ff9640 1 -- 172.21.15.120:0/2830935788 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f7ec80cee00 con 0x7f7ed8100a80 2024-09-19T09:10:54.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.435+0000 7f7ed6ffd640 1 --2- 172.21.15.120:0/139542082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55beebde4680 0x55beebde6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:54.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.435+0000 7f7ed77fe640 1 --2- 172.21.15.120:0/139542082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beebdd9dc0 0x55beebde4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:54.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.435+0000 7f7ed7fff640 1 -- 172.21.15.120:0/139542082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55beebde4680 msgr2=0x55beebde6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:54.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.435+0000 7f7ed7fff640 1 --2- 172.21.15.120:0/139542082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55beebde4680 0x55beebde6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.435+0000 7f7ed7fff640 1 -- 172.21.15.120:0/139542082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beebdd9dc0 msgr2=0x55beebde4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:54.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.435+0000 7f7ed7fff640 1 --2- 172.21.15.120:0/139542082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beebdd9dc0 0x55beebde4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.435+0000 7f7ed7fff640 1 -- 172.21.15.120:0/139542082 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beebccb440 con 0x55beebdd94a0 2024-09-19T09:10:54.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.435+0000 7f7ed7fff640 1 --2- 172.21.15.120:0/139542082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beebdd94a0 0x55beebdd9880 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000afa0 tx=0x7f7ed0016700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1785c5b241bf33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:54.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.435+0000 7f7ed7fff640 1 --2- 172.21.15.120:0/2830935788 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7ec00a3240 0x7f7ec00a5700 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000afd0 tx=0x7f7ed001a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:54.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.435+0000 7f7ebdffb640 1 -- 172.21.15.120:0/139542082 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0026070 con 0x55beebdd94a0 2024-09-19T09:10:54.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.435+0000 7f7ebdffb640 1 -- 172.21.15.120:0/139542082 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ed0009d00 con 0x55beebdd94a0 2024-09-19T09:10:54.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.435+0000 7f7ebdffb640 1 -- 172.21.15.120:0/139542082 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0013040 con 0x55beebdd94a0 2024-09-19T09:10:54.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.436+0000 7f7edff6b740 1 -- 172.21.15.120:0/139542082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beebdd94a0 msgr2=0x55beebdd9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:54.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.436+0000 7f7edff6b740 1 --2- 172.21.15.120:0/139542082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beebdd94a0 0x55beebdd9880 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000afa0 tx=0x7f7ed0016700 comp rx=0 tx=0).stop 2024-09-19T09:10:54.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.436+0000 7f7edff6b740 1 -- 172.21.15.120:0/139542082 shutdown_connections 2024-09-19T09:10:54.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.436+0000 7f7edff6b740 1 --2- 172.21.15.120:0/139542082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55beebde4680 0x55beebde6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.436+0000 7f7edff6b740 1 --2- 172.21.15.120:0/139542082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beebdd9dc0 0x55beebde4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.436+0000 7f7edff6b740 1 --2- 172.21.15.120:0/139542082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beebdd94a0 0x55beebdd9880 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.436+0000 7f7edff6b740 1 -- 172.21.15.120:0/139542082 >> 172.21.15.120:0/139542082 conn(0x55beebdd5a30 msgr2=0x55beebdd5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:54.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.436+0000 7f7edff6b740 1 -- 172.21.15.120:0/139542082 shutdown_connections 2024-09-19T09:10:54.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.436+0000 7f7edff6b740 1 -- 172.21.15.120:0/139542082 wait complete. 2024-09-19T09:10:54.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.436+0000 7f7edff6b740 1 -- start start 2024-09-19T09:10:54.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.437+0000 7f7edff6b740 1 Processor -- start 2024-09-19T09:10:54.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.437+0000 7f7edff6b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beebdd94a0 0x55beebde4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:54.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.437+0000 7f7edff6b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55beebdd9dc0 0x55beebde12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:54.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.437+0000 7f7edff6b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beebde4680 0x55beebde1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:54.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.437+0000 7f7edff6b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55beebde7250 con 0x55beebdd94a0 2024-09-19T09:10:54.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.437+0000 7f7edff6b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55beebcfa050 con 0x55beebde4680 2024-09-19T09:10:54.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.437+0000 7f7edff6b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55beebde73d0 con 0x55beebdd9dc0 2024-09-19T09:10:54.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.437+0000 7f7edff6b740 10 client.0 _ll_register_callbacks cb 0x55beebd3a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:54.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.437+0000 7f7ed77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beebdd94a0 0x55beebde4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:54.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.437+0000 7f7ed77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beebdd94a0 0x55beebde4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60620/0 (socket says 172.21.15.120:60620) 2024-09-19T09:10:54.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.437+0000 7f7ed77fe640 1 -- 172.21.15.120:0/2782588652 learned_addr learned my addr 172.21.15.120:0/2782588652 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:54.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.437+0000 7f7ed7fff640 1 --2- 172.21.15.120:0/2782588652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beebde4680 0x55beebde1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:54.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.437+0000 7f7ed6ffd640 1 --2- 172.21.15.120:0/2782588652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55beebdd9dc0 0x55beebde12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:54.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.437+0000 7f7ed77fe640 1 -- 172.21.15.120:0/2782588652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55beebdd9dc0 msgr2=0x55beebde12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:54.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.437+0000 7f7ed77fe640 1 --2- 172.21.15.120:0/2782588652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55beebdd9dc0 0x55beebde12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.437+0000 7f7ed77fe640 1 -- 172.21.15.120:0/2782588652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beebde4680 msgr2=0x55beebde1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:54.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.437+0000 7f7ed77fe640 1 --2- 172.21.15.120:0/2782588652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beebde4680 0x55beebde1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.437+0000 7f7ed77fe640 1 -- 172.21.15.120:0/2782588652 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beebab03c0 con 0x55beebdd94a0 2024-09-19T09:10:54.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.437+0000 7f7ed7fff640 1 --2- 172.21.15.120:0/2782588652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beebde4680 0x55beebde1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:10:54.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.437+0000 7f7ed77fe640 1 --2- 172.21.15.120:0/2782588652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beebdd94a0 0x55beebde4030 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f7ec80cc8c0 tx=0x7f7ec8009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23051e10bdddc97d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:54.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.437+0000 7f7eb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:54.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.438+0000 7f7eb6ffd640 1 -- 172.21.15.120:0/2782588652 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec80b3020 con 0x55beebdd94a0 2024-09-19T09:10:54.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.438+0000 7f7edff6b740 1 -- 172.21.15.120:0/2782588652 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ed0009d00 con 0x55beebdd94a0 2024-09-19T09:10:54.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.438+0000 7f7eb6ffd640 1 -- 172.21.15.120:0/2782588652 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f7ec8010ca0 con 0x55beebdd94a0 2024-09-19T09:10:54.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.438+0000 7f7eb6ffd640 1 -- 172.21.15.120:0/2782588652 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7ec801e040 con 0x55beebdd94a0 2024-09-19T09:10:54.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.438+0000 7f7edff6b740 10 client.18183 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:54.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.438+0000 7f7edff6b740 1 -- 172.21.15.120:0/2782588652 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55beebdf2cb0 con 0x55beebdd94a0 2024-09-19T09:10:54.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.438+0000 7f7eb6ffd640 1 -- 172.21.15.120:0/2782588652 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ed0009d00 con 0x55beebdd94a0 2024-09-19T09:10:54.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.438+0000 7f7eb6ffd640 1 -- 172.21.15.120:0/2782588652 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7ec8097a50 con 0x55beebdd94a0 2024-09-19T09:10:54.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.438+0000 7f7edff6b740 10 client.18183 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:54.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.438+0000 7f7edff6b740 10 client.18183 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:10:54.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.438+0000 7f7edff6b740 20 client.18183 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:54.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.438+0000 7f7edff6b740 1 --2- 172.21.15.120:0/2782588652 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55beebdf40c0 0x55beebdf64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:54.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.438+0000 7f7edff6b740 4 client.18183 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:10:54.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.438+0000 7f7edff6b740 1 -- 172.21.15.120:0/2782588652 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55beebd4c850 con 0x55beebdf40c0 2024-09-19T09:10:54.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.438+0000 7f7ed6ffd640 1 --2- 172.21.15.120:0/2782588652 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55beebdf40c0 0x55beebdf64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:54.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.439+0000 7f7eb6ffd640 10 client.18183 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:54.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.439+0000 7f7ed6ffd640 1 --2- 172.21.15.120:0/2782588652 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55beebdf40c0 0x55beebdf64b0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a9d0e9a0fdf6377 server_cookie=540bd4e2595ccdc8 in_seq=0 out_seq=0 2024-09-19T09:10:54.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.440+0000 7f7eb6ffd640 1 -- 172.21.15.120:0/2782588652 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55beebd4c850 con 0x55beebdf40c0 2024-09-19T09:10:54.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.440+0000 7f7eb6ffd640 10 client.18183 handle_command_reply: tid=0 2024-09-19T09:10:54.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.440+0000 7f7edff6b740 1 client.18183 shutdown 2024-09-19T09:10:54.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.440+0000 7f7edff6b740 1 -- 172.21.15.120:0/2782588652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beebdd94a0 msgr2=0x55beebde4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:54.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.440+0000 7f7edff6b740 1 --2- 172.21.15.120:0/2782588652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beebdd94a0 0x55beebde4030 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f7ec80cc8c0 tx=0x7f7ec8009390 comp rx=0 tx=0).stop 2024-09-19T09:10:54.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.440+0000 7f7edff6b740 1 -- 172.21.15.120:0/2782588652 shutdown_connections 2024-09-19T09:10:54.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.440+0000 7f7edff6b740 1 --2- 172.21.15.120:0/2782588652 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55beebdf40c0 0x55beebdf64b0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.440+0000 7f7edff6b740 1 --2- 172.21.15.120:0/2782588652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beebde4680 0x55beebde1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.440+0000 7f7edff6b740 1 --2- 172.21.15.120:0/2782588652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55beebdd9dc0 0x55beebde12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.440+0000 7f7edff6b740 1 --2- 172.21.15.120:0/2782588652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beebdd94a0 0x55beebde4030 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.440+0000 7f7edff6b740 1 -- 172.21.15.120:0/2782588652 >> 172.21.15.120:0/2782588652 conn(0x55beebdd5a30 msgr2=0x55beebdd4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:54.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.440+0000 7f7eb6ffd640 0 client.18183 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:54.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.440+0000 7f7edff6b740 1 -- 172.21.15.120:0/2782588652 shutdown_connections 2024-09-19T09:10:54.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.440+0000 7f7edff6b740 1 -- 172.21.15.120:0/2782588652 wait complete. 2024-09-19T09:10:54.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.440+0000 7f7edff6b740 20 client.18183 trim_cache size 0 max 16384 2024-09-19T09:10:54.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.457+0000 7f7edff6b740 1 Processor -- start 2024-09-19T09:10:54.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.457+0000 7f7edff6b740 1 -- start start 2024-09-19T09:10:54.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.457+0000 7f7edff6b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beebdd94a0 0x55beebdd4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:54.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.457+0000 7f7edff6b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beebdd9dc0 0x55beebdfbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:54.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.457+0000 7f7edff6b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55beebde4680 0x55beebdfc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:54.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.457+0000 7f7edff6b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed81ab460 con 0x55beebdd9dc0 2024-09-19T09:10:54.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.457+0000 7f7edff6b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed81acfa0 con 0x55beebdd94a0 2024-09-19T09:10:54.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.457+0000 7f7edff6b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed81abb30 con 0x55beebde4680 2024-09-19T09:10:54.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.458+0000 7f7ed6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beebdd9dc0 0x55beebdfbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:54.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.458+0000 7f7ed6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beebdd9dc0 0x55beebdfbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60630/0 (socket says 172.21.15.120:60630) 2024-09-19T09:10:54.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.458+0000 7f7ed6ffd640 1 -- 172.21.15.120:0/900853394 learned_addr learned my addr 172.21.15.120:0/900853394 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:54.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.458+0000 7f7ed77fe640 1 --2- 172.21.15.120:0/900853394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beebdd94a0 0x55beebdd4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:54.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.458+0000 7f7ed7fff640 1 --2- 172.21.15.120:0/900853394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55beebde4680 0x55beebdfc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:54.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.458+0000 7f7ed6ffd640 1 -- 172.21.15.120:0/900853394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55beebde4680 msgr2=0x55beebdfc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:54.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.458+0000 7f7ed6ffd640 1 --2- 172.21.15.120:0/900853394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55beebde4680 0x55beebdfc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.458+0000 7f7ed6ffd640 1 -- 172.21.15.120:0/900853394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beebdd94a0 msgr2=0x55beebdd4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:54.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.458+0000 7f7ed6ffd640 1 --2- 172.21.15.120:0/900853394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beebdd94a0 0x55beebdd4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.458+0000 7f7ed6ffd640 1 -- 172.21.15.120:0/900853394 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beebde0290 con 0x55beebdd9dc0 2024-09-19T09:10:54.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.458+0000 7f7ed6ffd640 1 --2- 172.21.15.120:0/900853394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beebdd9dc0 0x55beebdfbd30 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc000c00 tx=0x7f7ecc0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc95c1b9456f0cd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:54.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.458+0000 7f7ebd7fa640 1 -- 172.21.15.120:0/900853394 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc035040 con 0x55beebdd9dc0 2024-09-19T09:10:54.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.458+0000 7f7ebd7fa640 1 -- 172.21.15.120:0/900853394 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ecc005520 con 0x55beebdd9dc0 2024-09-19T09:10:54.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.458+0000 7f7ebd7fa640 1 -- 172.21.15.120:0/900853394 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc03bc10 con 0x55beebdd9dc0 2024-09-19T09:10:54.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.459+0000 7f7edff6b740 1 -- 172.21.15.120:0/900853394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beebdd9dc0 msgr2=0x55beebdfbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:54.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.459+0000 7f7edff6b740 1 --2- 172.21.15.120:0/900853394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beebdd9dc0 0x55beebdfbd30 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc000c00 tx=0x7f7ecc0058b0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.459+0000 7f7edff6b740 1 -- 172.21.15.120:0/900853394 shutdown_connections 2024-09-19T09:10:54.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.459+0000 7f7edff6b740 1 --2- 172.21.15.120:0/900853394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55beebde4680 0x55beebdfc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.459+0000 7f7edff6b740 1 --2- 172.21.15.120:0/900853394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beebdd9dc0 0x55beebdfbd30 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.459+0000 7f7edff6b740 1 --2- 172.21.15.120:0/900853394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beebdd94a0 0x55beebdd4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.459+0000 7f7edff6b740 1 -- 172.21.15.120:0/900853394 >> 172.21.15.120:0/900853394 conn(0x55beebdd5a30 msgr2=0x55beebdd4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:54.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.459+0000 7f7edff6b740 1 -- 172.21.15.120:0/900853394 shutdown_connections 2024-09-19T09:10:54.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.459+0000 7f7edff6b740 1 -- 172.21.15.120:0/900853394 wait complete. 2024-09-19T09:10:54.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.459+0000 7f7edff6b740 1 -- start start 2024-09-19T09:10:54.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.460+0000 7f7edff6b740 1 Processor -- start 2024-09-19T09:10:54.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.460+0000 7f7edff6b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beebdd94a0 0x55beebdecc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:54.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.460+0000 7f7edff6b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55beebdd9dc0 0x55beebdea060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:54.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.460+0000 7f7edff6b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beebde4680 0x55beebdea5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:54.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.460+0000 7f7edff6b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed81ac8d0 con 0x55beebde4680 2024-09-19T09:10:54.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.460+0000 7f7edff6b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed81ac200 con 0x55beebdd94a0 2024-09-19T09:10:54.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.460+0000 7f7edff6b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed81a9420 con 0x55beebdd9dc0 2024-09-19T09:10:54.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.460+0000 7f7edff6b740 10 client.0 _ll_register_callbacks cb 0x7f7ec8097680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:54.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.460+0000 7f7ed7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beebde4680 0x55beebdea5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:54.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.460+0000 7f7ed7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beebde4680 0x55beebdea5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60632/0 (socket says 172.21.15.120:60632) 2024-09-19T09:10:54.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.460+0000 7f7ed7fff640 1 -- 172.21.15.120:0/1481289947 learned_addr learned my addr 172.21.15.120:0/1481289947 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:54.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.460+0000 7f7ed77fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beebdd94a0 0x55beebdecc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:54.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.460+0000 7f7ed6ffd640 1 --2- 172.21.15.120:0/1481289947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55beebdd9dc0 0x55beebdea060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:54.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.460+0000 7f7ed7fff640 1 -- 172.21.15.120:0/1481289947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55beebdd9dc0 msgr2=0x55beebdea060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:54.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.460+0000 7f7ed7fff640 1 --2- 172.21.15.120:0/1481289947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55beebdd9dc0 0x55beebdea060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.460+0000 7f7ed7fff640 1 -- 172.21.15.120:0/1481289947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beebdd94a0 msgr2=0x55beebdecc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:54.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.460+0000 7f7ed7fff640 1 --2- 172.21.15.120:0/1481289947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beebdd94a0 0x55beebdecc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.460+0000 7f7ed7fff640 1 -- 172.21.15.120:0/1481289947 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beebdeadd0 con 0x55beebde4680 2024-09-19T09:10:54.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.460+0000 7f7ed77fe640 1 --2- 172.21.15.120:0/1481289947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beebdd94a0 0x55beebdecc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:10:54.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.460+0000 7f7ed7fff640 1 --2- 172.21.15.120:0/1481289947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beebde4680 0x55beebdea5a0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0009440 tx=0x7f7ed000a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6566439ea467595 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:54.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.460+0000 7f7eb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:54.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.461+0000 7f7eb6ffd640 1 -- 172.21.15.120:0/1481289947 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0030070 con 0x55beebde4680 2024-09-19T09:10:54.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.461+0000 7f7eb6ffd640 1 -- 172.21.15.120:0/1481289947 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f7ed0019c50 con 0x55beebde4680 2024-09-19T09:10:54.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.461+0000 7f7edff6b740 1 -- 172.21.15.120:0/1481289947 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ecc005520 con 0x55beebde4680 2024-09-19T09:10:54.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.461+0000 7f7eb6ffd640 1 -- 172.21.15.120:0/1481289947 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7ed003c050 con 0x55beebde4680 2024-09-19T09:10:54.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.461+0000 7f7edff6b740 10 client.18189 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:54.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.461+0000 7f7edff6b740 1 -- 172.21.15.120:0/1481289947 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55beebe5ec20 con 0x55beebde4680 2024-09-19T09:10:54.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.461+0000 7f7eb6ffd640 1 -- 172.21.15.120:0/1481289947 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ecc005520 con 0x55beebde4680 2024-09-19T09:10:54.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.461+0000 7f7eb6ffd640 1 -- 172.21.15.120:0/1481289947 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7ed002c610 con 0x55beebde4680 2024-09-19T09:10:54.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.461+0000 7f7edff6b740 10 client.18189 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:54.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.461+0000 7f7edff6b740 10 client.18189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:10:54.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.461+0000 7f7edff6b740 20 client.18189 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:54.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.461+0000 7f7edff6b740 1 --2- 172.21.15.120:0/1481289947 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55beebe5f0c0 0x55beebe614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:54.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.461+0000 7f7edff6b740 4 client.18189 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:10:54.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.461+0000 7f7edff6b740 1 -- 172.21.15.120:0/1481289947 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55beebd46d60 con 0x55beebe5f0c0 2024-09-19T09:10:54.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.462+0000 7f7ed77fe640 1 --2- 172.21.15.120:0/1481289947 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55beebe5f0c0 0x55beebe614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:54.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.462+0000 7f7ed77fe640 1 --2- 172.21.15.120:0/1481289947 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55beebe5f0c0 0x55beebe614b0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e428deccdddae33 server_cookie=4fe9f5c25264ea4b in_seq=0 out_seq=0 2024-09-19T09:10:54.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.462+0000 7f7eb6ffd640 10 client.18189 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:54.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.462+0000 7f7eb6ffd640 1 -- 172.21.15.120:0/1481289947 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55beebd46d60 con 0x55beebe5f0c0 2024-09-19T09:10:54.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.462+0000 7f7eb6ffd640 10 client.18189 handle_command_reply: tid=0 2024-09-19T09:10:54.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.462+0000 7f7edff6b740 1 client.18189 shutdown 2024-09-19T09:10:54.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.463+0000 7f7edff6b740 1 -- 172.21.15.120:0/1481289947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beebde4680 msgr2=0x55beebdea5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:54.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.463+0000 7f7edff6b740 1 --2- 172.21.15.120:0/1481289947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beebde4680 0x55beebdea5a0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0009440 tx=0x7f7ed000a720 comp rx=0 tx=0).stop 2024-09-19T09:10:54.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.463+0000 7f7edff6b740 1 -- 172.21.15.120:0/1481289947 shutdown_connections 2024-09-19T09:10:54.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.463+0000 7f7edff6b740 1 --2- 172.21.15.120:0/1481289947 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55beebe5f0c0 0x55beebe614b0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.463+0000 7f7edff6b740 1 --2- 172.21.15.120:0/1481289947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beebde4680 0x55beebdea5a0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.463+0000 7f7edff6b740 1 --2- 172.21.15.120:0/1481289947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55beebdd9dc0 0x55beebdea060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.463+0000 7f7edff6b740 1 --2- 172.21.15.120:0/1481289947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beebdd94a0 0x55beebdecc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.463+0000 7f7edff6b740 1 -- 172.21.15.120:0/1481289947 >> 172.21.15.120:0/1481289947 conn(0x55beebdd5a30 msgr2=0x55beebdd4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:54.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.463+0000 7f7eb6ffd640 0 client.18189 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:54.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.463+0000 7f7edff6b740 1 -- 172.21.15.120:0/1481289947 shutdown_connections 2024-09-19T09:10:54.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.463+0000 7f7edff6b740 1 -- 172.21.15.120:0/1481289947 wait complete. 2024-09-19T09:10:54.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.463+0000 7f7edff6b740 20 client.18189 trim_cache size 0 max 16384 2024-09-19T09:10:54.473 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:10:54.473 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:10:54.473 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:10:54.473 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:10:54.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.464+0000 7f7eddb92640 1 -- 172.21.15.120:0/2830935788 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7ec00a3240 msgr2=0x7f7ec00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:54.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.464+0000 7f7eddb92640 1 --2- 172.21.15.120:0/2830935788 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7ec00a3240 0x7f7ec00a5700 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000afd0 tx=0x7f7ed001a000 comp rx=0 tx=0).stop 2024-09-19T09:10:54.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.464+0000 7f7eddb92640 1 -- 172.21.15.120:0/2830935788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ed8100a80 msgr2=0x7f7ed81985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:54.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.464+0000 7f7eddb92640 1 --2- 172.21.15.120:0/2830935788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ed8100a80 0x7f7ed81985c0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800b4a0 tx=0x7f7ec800b960 comp rx=0 tx=0).stop 2024-09-19T09:10:54.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.464+0000 7f7eddb92640 1 -- 172.21.15.120:0/2830935788 shutdown_connections 2024-09-19T09:10:54.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.464+0000 7f7eddb92640 1 --2- 172.21.15.120:0/2830935788 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7ec00a3240 0x7f7ec00a5700 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.464+0000 7f7eddb92640 1 --2- 172.21.15.120:0/2830935788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ed8106a80 0x7f7ed819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.464+0000 7f7eddb92640 1 --2- 172.21.15.120:0/2830935788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ed8101420 0x7f7ed8198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.464+0000 7f7eddb92640 1 --2- 172.21.15.120:0/2830935788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ed8100a80 0x7f7ed81985c0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:54.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.464+0000 7f7eddb92640 1 -- 172.21.15.120:0/2830935788 >> 172.21.15.120:0/2830935788 conn(0x7f7ed80fc7a0 msgr2=0x7f7ed80fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:54.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.464+0000 7f7eddb92640 1 -- 172.21.15.120:0/2830935788 shutdown_connections 2024-09-19T09:10:54.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:54.464+0000 7f7eddb92640 1 -- 172.21.15.120:0/2830935788 wait complete. 2024-09-19T09:10:55.060 INFO:tasks.workunit.client.0.smithi120.stdout: 1 707664 8.98 MB/sec execute 290 sec latency 18420.694 ms 2024-09-19T09:10:55.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:54 smithi196 ceph-mon[35150]: pgmap v583: 97 pgs: 97 active+clean; 26 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 169 KiB/s wr, 65 op/s 2024-09-19T09:10:55.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:54 smithi153 ceph-mon[35071]: pgmap v583: 97 pgs: 97 active+clean; 26 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 169 KiB/s wr, 65 op/s 2024-09-19T09:10:55.247 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:54 smithi120 ceph-mon[31150]: pgmap v583: 97 pgs: 97 active+clean; 26 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 169 KiB/s wr, 65 op/s 2024-09-19T09:10:55.275 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:10:55.276 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:10:55.521 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:10:56.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.022+0000 7fa1b5ae9640 1 -- 172.21.15.120:0/763550379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1b0101420 msgr2=0x7fa1b010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:56.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.022+0000 7fa1b5ae9640 1 --2- 172.21.15.120:0/763550379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1b0101420 0x7fa1b010d950 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009930 tx=0x7fa1a802a420 comp rx=0 tx=0).stop 2024-09-19T09:10:56.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.022+0000 7fa1b5ae9640 1 -- 172.21.15.120:0/763550379 shutdown_connections 2024-09-19T09:10:56.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.022+0000 7fa1b5ae9640 1 --2- 172.21.15.120:0/763550379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1b0101420 0x7fa1b010d950 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:56.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.022+0000 7fa1b5ae9640 1 --2- 172.21.15.120:0/763550379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1b0100a80 0x7fa1b0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:56.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.022+0000 7fa1b5ae9640 1 --2- 172.21.15.120:0/763550379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1b0106a80 0x7fa1b0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:56.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.022+0000 7fa1b5ae9640 1 -- 172.21.15.120:0/763550379 >> 172.21.15.120:0/763550379 conn(0x7fa1b00fc7a0 msgr2=0x7fa1b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:56.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.022+0000 7fa1b5ae9640 1 -- 172.21.15.120:0/763550379 shutdown_connections 2024-09-19T09:10:56.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.022+0000 7fa1b5ae9640 1 -- 172.21.15.120:0/763550379 wait complete. 2024-09-19T09:10:56.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.023+0000 7fa1b5ae9640 1 Processor -- start 2024-09-19T09:10:56.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.023+0000 7fa1b5ae9640 1 -- start start 2024-09-19T09:10:56.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.023+0000 7fa1b5ae9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1b0100a80 0x7fa1b0199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:56.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.023+0000 7fa1b5ae9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1b0101420 0x7fa1b01996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:56.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.023+0000 7fa1b5ae9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1b0106a80 0x7fa1b01a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:56.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.023+0000 7fa1b5ae9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b0110e80 con 0x7fa1b0101420 2024-09-19T09:10:56.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.023+0000 7fa1b5ae9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b0110d00 con 0x7fa1b0100a80 2024-09-19T09:10:56.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.023+0000 7fa1b5ae9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b0111000 con 0x7fa1b0106a80 2024-09-19T09:10:56.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.023+0000 7fa1aeffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1b0101420 0x7fa1b01996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:56.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.023+0000 7fa1aeffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1b0101420 0x7fa1b01996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60644/0 (socket says 172.21.15.120:60644) 2024-09-19T09:10:56.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.023+0000 7fa1aeffd640 1 -- 172.21.15.120:0/3490142166 learned_addr learned my addr 172.21.15.120:0/3490142166 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:56.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.023+0000 7fa1af7fe640 1 --2- 172.21.15.120:0/3490142166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1b0100a80 0x7fa1b0199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:56.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.023+0000 7fa1affff640 1 --2- 172.21.15.120:0/3490142166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1b0106a80 0x7fa1b01a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:56.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.023+0000 7fa1aeffd640 1 -- 172.21.15.120:0/3490142166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1b0106a80 msgr2=0x7fa1b01a41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:56.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.023+0000 7fa1aeffd640 1 --2- 172.21.15.120:0/3490142166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1b0106a80 0x7fa1b01a41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:56.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.023+0000 7fa1aeffd640 1 -- 172.21.15.120:0/3490142166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1b0100a80 msgr2=0x7fa1b0199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:56.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.023+0000 7fa1aeffd640 1 --2- 172.21.15.120:0/3490142166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1b0100a80 0x7fa1b0199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:56.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.023+0000 7fa1aeffd640 1 -- 172.21.15.120:0/3490142166 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b01a4730 con 0x7fa1b0101420 2024-09-19T09:10:56.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.023+0000 7fa1affff640 1 --2- 172.21.15.120:0/3490142166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1b0106a80 0x7fa1b01a41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:10:56.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.023+0000 7fa1af7fe640 1 --2- 172.21.15.120:0/3490142166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1b0100a80 0x7fa1b0199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:10:56.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.024+0000 7fa1aeffd640 1 --2- 172.21.15.120:0/3490142166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1b0101420 0x7fa1b01996b0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fa1a400cce0 tx=0x7fa1a400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:56.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.024+0000 7fa1acff9640 1 -- 172.21.15.120:0/3490142166 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a400f650 con 0x7fa1b0101420 2024-09-19T09:10:56.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.024+0000 7fa1b5ae9640 1 -- 172.21.15.120:0/3490142166 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b01a4900 con 0x7fa1b0101420 2024-09-19T09:10:56.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.024+0000 7fa1acff9640 1 -- 172.21.15.120:0/3490142166 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a400fcd0 con 0x7fa1b0101420 2024-09-19T09:10:56.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.024+0000 7fa1acff9640 1 -- 172.21.15.120:0/3490142166 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4018c70 con 0x7fa1b0101420 2024-09-19T09:10:56.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.024+0000 7fa1b5ae9640 1 -- 172.21.15.120:0/3490142166 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b01a4e10 con 0x7fa1b0101420 2024-09-19T09:10:56.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.025+0000 7fa1b5ae9640 1 -- 172.21.15.120:0/3490142166 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1b0100ee0 con 0x7fa1b0101420 2024-09-19T09:10:56.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.026+0000 7fa1acff9640 1 -- 172.21.15.120:0/3490142166 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa1a400f7f0 con 0x7fa1b0101420 2024-09-19T09:10:56.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.026+0000 7fa1acff9640 1 --2- 172.21.15.120:0/3490142166 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa1980a3240 0x7fa1980a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:56.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.027+0000 7fa1acff9640 1 -- 172.21.15.120:0/3490142166 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fa1a40d0170 con 0x7fa1b0101420 2024-09-19T09:10:56.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.027+0000 7fa1af7fe640 1 --2- 172.21.15.120:0/3490142166 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa1980a3240 0x7fa1980a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:56.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.027+0000 7fa1af7fe640 1 --2- 172.21.15.120:0/3490142166 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa1980a3240 0x7fa1980a5700 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa1a000baa0 tx=0x7fa1a0005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:56.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.030+0000 7fa1acff9640 1 -- 172.21.15.120:0/3490142166 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa1a4014030 con 0x7fa1b0101420 2024-09-19T09:10:56.060 INFO:tasks.workunit.client.0.smithi120.stdout: 1 707664 8.95 MB/sec execute 291 sec latency 19420.759 ms 2024-09-19T09:10:56.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.232+0000 7fa1b5ae9640 1 -- 172.21.15.120:0/3490142166 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1b01a50a0 con 0x7fa1b0101420 2024-09-19T09:10:56.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.233+0000 7fa1acff9640 1 -- 172.21.15.120:0/3490142166 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11985 (secure 0 0 0) 0x7fa1a409cf20 con 0x7fa1b0101420 2024-09-19T09:10:56.234 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:10:56.234 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":30,"btime":"2024-09-19T09:07:14:812065+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:07:14.812046+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26351":{"gid":26351,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/364941779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":364941779},{"type":"v1","addr":"172.21.15.153:6835","nonce":364941779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:10:56.234 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 30 2024-09-19T09:10:56.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.236+0000 7fa1b5ae9640 1 -- 172.21.15.120:0/3490142166 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa1980a3240 msgr2=0x7fa1980a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:56.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.236+0000 7fa1b5ae9640 1 --2- 172.21.15.120:0/3490142166 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa1980a3240 0x7fa1980a5700 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa1a000baa0 tx=0x7fa1a0005d20 comp rx=0 tx=0).stop 2024-09-19T09:10:56.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.236+0000 7fa1b5ae9640 1 -- 172.21.15.120:0/3490142166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1b0101420 msgr2=0x7fa1b01996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:56.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.236+0000 7fa1b5ae9640 1 --2- 172.21.15.120:0/3490142166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1b0101420 0x7fa1b01996b0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fa1a400cce0 tx=0x7fa1a400a620 comp rx=0 tx=0).stop 2024-09-19T09:10:56.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.236+0000 7fa1b5ae9640 1 -- 172.21.15.120:0/3490142166 shutdown_connections 2024-09-19T09:10:56.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.236+0000 7fa1b5ae9640 1 --2- 172.21.15.120:0/3490142166 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa1980a3240 0x7fa1980a5700 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:56.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.236+0000 7fa1b5ae9640 1 --2- 172.21.15.120:0/3490142166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1b0106a80 0x7fa1b01a41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:56.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.236+0000 7fa1b5ae9640 1 --2- 172.21.15.120:0/3490142166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1b0101420 0x7fa1b01996b0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:56.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.236+0000 7fa1b5ae9640 1 --2- 172.21.15.120:0/3490142166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1b0100a80 0x7fa1b0199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:56.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.236+0000 7fa1b5ae9640 1 -- 172.21.15.120:0/3490142166 >> 172.21.15.120:0/3490142166 conn(0x7fa1b00fc7a0 msgr2=0x7fa1b00fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:56.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.236+0000 7fa1b5ae9640 1 -- 172.21.15.120:0/3490142166 shutdown_connections 2024-09-19T09:10:56.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:56.236+0000 7fa1b5ae9640 1 -- 172.21.15.120:0/3490142166 wait complete. 2024-09-19T09:10:57.037 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:10:57.060 INFO:tasks.workunit.client.0.smithi120.stdout: 1 707664 8.92 MB/sec execute 292 sec latency 20420.785 ms 2024-09-19T09:10:57.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:56 smithi196 ceph-mon[35150]: pgmap v584: 97 pgs: 97 active+clean; 26 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 39 KiB/s wr, 58 op/s 2024-09-19T09:10:57.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:56 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3490142166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:10:57.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:56 smithi153 ceph-mon[35071]: pgmap v584: 97 pgs: 97 active+clean; 26 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 39 KiB/s wr, 58 op/s 2024-09-19T09:10:57.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:56 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3490142166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:10:57.282 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:10:57.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:56 smithi120 ceph-mon[31150]: pgmap v584: 97 pgs: 97 active+clean; 26 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 39 KiB/s wr, 58 op/s 2024-09-19T09:10:57.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:56 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3490142166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:10:57.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.749+0000 7f7c39091640 1 -- 172.21.15.120:0/3213108868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c34106a80 msgr2=0x7f7c34106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:57.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.749+0000 7f7c39091640 1 --2- 172.21.15.120:0/3213108868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c34106a80 0x7f7c34106e60 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c009930 tx=0x7f7c1c02a420 comp rx=0 tx=0).stop 2024-09-19T09:10:57.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.749+0000 7f7c39091640 1 -- 172.21.15.120:0/3213108868 shutdown_connections 2024-09-19T09:10:57.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.749+0000 7f7c39091640 1 --2- 172.21.15.120:0/3213108868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c34101420 0x7f7c3410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.749+0000 7f7c39091640 1 --2- 172.21.15.120:0/3213108868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c34100a80 0x7f7c34100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.749+0000 7f7c39091640 1 --2- 172.21.15.120:0/3213108868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c34106a80 0x7f7c34106e60 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.749+0000 7f7c39091640 1 -- 172.21.15.120:0/3213108868 >> 172.21.15.120:0/3213108868 conn(0x7f7c340fc7a0 msgr2=0x7f7c340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:57.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.750+0000 7f7c39091640 1 -- 172.21.15.120:0/3213108868 shutdown_connections 2024-09-19T09:10:57.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.750+0000 7f7c39091640 1 -- 172.21.15.120:0/3213108868 wait complete. 2024-09-19T09:10:57.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.750+0000 7f7c39091640 1 Processor -- start 2024-09-19T09:10:57.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.751+0000 7f7c39091640 1 -- start start 2024-09-19T09:10:57.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.751+0000 7f7c39091640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c34100a80 0x7f7c341985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:57.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.751+0000 7f7c39091640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c34101420 0x7f7c34198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:57.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.751+0000 7f7c39091640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c34106a80 0x7f7c3419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:57.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.751+0000 7f7c39091640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7c34112020 con 0x7f7c34100a80 2024-09-19T09:10:57.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.751+0000 7f7c39091640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7c34111ea0 con 0x7f7c34106a80 2024-09-19T09:10:57.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.751+0000 7f7c39091640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341121a0 con 0x7f7c34101420 2024-09-19T09:10:57.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.751+0000 7f7c32d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c34100a80 0x7f7c341985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:57.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.751+0000 7f7c32d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c34100a80 0x7f7c341985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60654/0 (socket says 172.21.15.120:60654) 2024-09-19T09:10:57.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.751+0000 7f7c32d9d640 1 -- 172.21.15.120:0/2099881131 learned_addr learned my addr 172.21.15.120:0/2099881131 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:57.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.751+0000 7f7c3359e640 1 --2- 172.21.15.120:0/2099881131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c34106a80 0x7f7c3419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:57.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.751+0000 7f7c3259c640 1 --2- 172.21.15.120:0/2099881131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c34101420 0x7f7c34198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:57.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.751+0000 7f7c32d9d640 1 -- 172.21.15.120:0/2099881131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c34101420 msgr2=0x7f7c34198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:57.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.751+0000 7f7c32d9d640 1 --2- 172.21.15.120:0/2099881131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c34101420 0x7f7c34198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.751+0000 7f7c32d9d640 1 -- 172.21.15.120:0/2099881131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c34106a80 msgr2=0x7f7c3419ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:57.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.751+0000 7f7c32d9d640 1 --2- 172.21.15.120:0/2099881131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c34106a80 0x7f7c3419ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.751+0000 7f7c32d9d640 1 -- 172.21.15.120:0/2099881131 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3419d590 con 0x7f7c34100a80 2024-09-19T09:10:57.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.751+0000 7f7c3359e640 1 --2- 172.21.15.120:0/2099881131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c34106a80 0x7f7c3419ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:10:57.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.751+0000 7f7c3259c640 1 --2- 172.21.15.120:0/2099881131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c34101420 0x7f7c34198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:10:57.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.751+0000 7f7c32d9d640 1 --2- 172.21.15.120:0/2099881131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c34100a80 0x7f7c341985e0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c009a50 tx=0x7f7c1c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:57.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.752+0000 7f7c2bfff640 1 -- 172.21.15.120:0/2099881131 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c03f030 con 0x7f7c34100a80 2024-09-19T09:10:57.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.752+0000 7f7c39091640 1 -- 172.21.15.120:0/2099881131 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c3419d820 con 0x7f7c34100a80 2024-09-19T09:10:57.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.752+0000 7f7c2bfff640 1 -- 172.21.15.120:0/2099881131 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c1c003ce0 con 0x7f7c34100a80 2024-09-19T09:10:57.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.752+0000 7f7c2bfff640 1 -- 172.21.15.120:0/2099881131 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c035070 con 0x7f7c34100a80 2024-09-19T09:10:57.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.752+0000 7f7c39091640 1 -- 172.21.15.120:0/2099881131 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c341a5160 con 0x7f7c34100a80 2024-09-19T09:10:57.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.755+0000 7f7c3b46a740 1 Processor -- start 2024-09-19T09:10:57.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.755+0000 7f7c2bfff640 1 -- 172.21.15.120:0/2099881131 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f7c1c042030 con 0x7f7c34100a80 2024-09-19T09:10:57.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.755+0000 7f7c3b46a740 1 -- start start 2024-09-19T09:10:57.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.755+0000 7f7c3b46a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d9a1c1b40 0x556d9a1c0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:57.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.755+0000 7f7c3b46a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d9a1c05d0 0x556d9a1c0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:57.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.755+0000 7f7c3b46a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d9a1cf780 0x556d9a1d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:57.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.755+0000 7f7c3b46a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556d9a1262f0 con 0x556d9a1c05d0 2024-09-19T09:10:57.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.755+0000 7f7c3b46a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556d9a0cdcf0 con 0x556d9a1cf780 2024-09-19T09:10:57.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.755+0000 7f7c3b46a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556d9a1222b0 con 0x556d9a1c1b40 2024-09-19T09:10:57.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.755+0000 7f7c32d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d9a1c05d0 0x556d9a1c0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:57.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.755+0000 7f7c32d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d9a1c05d0 0x556d9a1c0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60658/0 (socket says 172.21.15.120:60658) 2024-09-19T09:10:57.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.755+0000 7f7c32d9d640 1 -- 172.21.15.120:0/747308660 learned_addr learned my addr 172.21.15.120:0/747308660 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:57.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.756+0000 7f7c3259c640 1 --2- 172.21.15.120:0/747308660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d9a1cf780 0x556d9a1d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:57.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.756+0000 7f7c3359e640 1 --2- 172.21.15.120:0/747308660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d9a1c1b40 0x556d9a1c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:57.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.756+0000 7f7c32d9d640 1 -- 172.21.15.120:0/747308660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d9a1c1b40 msgr2=0x556d9a1c0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:57.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.756+0000 7f7c32d9d640 1 --2- 172.21.15.120:0/747308660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d9a1c1b40 0x556d9a1c0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.756+0000 7f7c32d9d640 1 -- 172.21.15.120:0/747308660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d9a1cf780 msgr2=0x556d9a1d1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:57.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.756+0000 7f7c32d9d640 1 --2- 172.21.15.120:0/747308660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d9a1cf780 0x556d9a1d1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.756+0000 7f7c32d9d640 1 -- 172.21.15.120:0/747308660 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d9a0b23a0 con 0x556d9a1c05d0 2024-09-19T09:10:57.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.756+0000 7f7c3259c640 1 --2- 172.21.15.120:0/747308660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d9a1cf780 0x556d9a1d1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:10:57.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.756+0000 7f7c3359e640 1 --2- 172.21.15.120:0/747308660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d9a1c1b40 0x556d9a1c0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:10:57.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.756+0000 7f7c2bfff640 1 --2- 172.21.15.120:0/2099881131 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7c200a3240 0x7f7c200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:57.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.756+0000 7f7c32d9d640 1 --2- 172.21.15.120:0/747308660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d9a1c05d0 0x556d9a1c0a10 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0f8290 tx=0x7f7c1c0f3f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6ef7e774f07678 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:57.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.756+0000 7f7c3359e640 1 --2- 172.21.15.120:0/2099881131 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7c200a3240 0x7f7c200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:57.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.756+0000 7f7c2bfff640 1 -- 172.21.15.120:0/2099881131 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f7c1c0eff80 con 0x7f7c34100a80 2024-09-19T09:10:57.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.756+0000 7f7c297fa640 1 -- 172.21.15.120:0/747308660 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c0b8840 con 0x556d9a1c05d0 2024-09-19T09:10:57.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.756+0000 7f7c297fa640 1 -- 172.21.15.120:0/747308660 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c1c0fd070 con 0x556d9a1c05d0 2024-09-19T09:10:57.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.756+0000 7f7c297fa640 1 -- 172.21.15.120:0/747308660 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c0bfe30 con 0x556d9a1c05d0 2024-09-19T09:10:57.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.757+0000 7f7c3b46a740 1 -- 172.21.15.120:0/747308660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d9a1c05d0 msgr2=0x556d9a1c0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:57.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.757+0000 7f7c3359e640 1 --2- 172.21.15.120:0/2099881131 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7c200a3240 0x7f7c200a5700 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c007ee0 tx=0x7f7c2c007430 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:57.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.757+0000 7f7c3b46a740 1 --2- 172.21.15.120:0/747308660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d9a1c05d0 0x556d9a1c0a10 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0f8290 tx=0x7f7c1c0f3f90 comp rx=0 tx=0).stop 2024-09-19T09:10:57.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.757+0000 7f7c3b46a740 1 -- 172.21.15.120:0/747308660 shutdown_connections 2024-09-19T09:10:57.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.757+0000 7f7c3b46a740 1 --2- 172.21.15.120:0/747308660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d9a1cf780 0x556d9a1d1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.757+0000 7f7c3b46a740 1 --2- 172.21.15.120:0/747308660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d9a1c05d0 0x556d9a1c0a10 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.757+0000 7f7c3b46a740 1 --2- 172.21.15.120:0/747308660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d9a1c1b40 0x556d9a1c0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.757+0000 7f7c3b46a740 1 -- 172.21.15.120:0/747308660 >> 172.21.15.120:0/747308660 conn(0x556d9a189310 msgr2=0x556d9a1bcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:57.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.757+0000 7f7c3b46a740 1 -- 172.21.15.120:0/747308660 shutdown_connections 2024-09-19T09:10:57.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.757+0000 7f7c3b46a740 1 -- 172.21.15.120:0/747308660 wait complete. 2024-09-19T09:10:57.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.757+0000 7f7c3b46a740 1 -- start start 2024-09-19T09:10:57.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.758+0000 7f7c3b46a740 1 Processor -- start 2024-09-19T09:10:57.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.758+0000 7f7c3b46a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d9a1c05d0 0x556d9a1cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:57.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.758+0000 7f7c3b46a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d9a1c1b40 0x556d9a1cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:57.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.758+0000 7f7c3b46a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d9a1cf780 0x556d9a1cc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:57.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.758+0000 7f7c3b46a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556d9a1d2240 con 0x556d9a1c05d0 2024-09-19T09:10:57.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.758+0000 7f7c3b46a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556d9a0e1060 con 0x556d9a1cf780 2024-09-19T09:10:57.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.758+0000 7f7c3b46a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556d9a1d23c0 con 0x556d9a1c1b40 2024-09-19T09:10:57.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.758+0000 7f7c3b46a740 10 client.0 _ll_register_callbacks cb 0x556d99e348c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:57.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.758+0000 7f7c32d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d9a1c05d0 0x556d9a1cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:57.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.758+0000 7f7c32d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d9a1c05d0 0x556d9a1cf130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60674/0 (socket says 172.21.15.120:60674) 2024-09-19T09:10:57.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.758+0000 7f7c32d9d640 1 -- 172.21.15.120:0/38634611 learned_addr learned my addr 172.21.15.120:0/38634611 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:57.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.758+0000 7f7c3359e640 1 --2- 172.21.15.120:0/38634611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d9a1cf780 0x556d9a1cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:57.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.758+0000 7f7c3259c640 1 --2- 172.21.15.120:0/38634611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d9a1c1b40 0x556d9a1cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:57.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.758+0000 7f7c32d9d640 1 -- 172.21.15.120:0/38634611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d9a1c1b40 msgr2=0x556d9a1cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:57.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.758+0000 7f7c32d9d640 1 --2- 172.21.15.120:0/38634611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d9a1c1b40 0x556d9a1cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.758+0000 7f7c32d9d640 1 -- 172.21.15.120:0/38634611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d9a1cf780 msgr2=0x556d9a1cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:57.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.758+0000 7f7c32d9d640 1 --2- 172.21.15.120:0/38634611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d9a1cf780 0x556d9a1cc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.758+0000 7f7c32d9d640 1 -- 172.21.15.120:0/38634611 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d99e973c0 con 0x556d9a1c05d0 2024-09-19T09:10:57.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.758+0000 7f7c3359e640 1 --2- 172.21.15.120:0/38634611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d9a1cf780 0x556d9a1cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:10:57.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.758+0000 7f7c3259c640 1 --2- 172.21.15.120:0/38634611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d9a1c1b40 0x556d9a1cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:10:57.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.758+0000 7f7c32d9d640 1 --2- 172.21.15.120:0/38634611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d9a1c05d0 0x556d9a1cf130 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0f8810 tx=0x7f7c1c03b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3073fee8dcf99f5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:57.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.758+0000 7f7c1a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:57.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.759+0000 7f7c1a7fc640 1 -- 172.21.15.120:0/38634611 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c049050 con 0x556d9a1c05d0 2024-09-19T09:10:57.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.759+0000 7f7c1a7fc640 1 -- 172.21.15.120:0/38634611 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f7c1c0edb90 con 0x556d9a1c05d0 2024-09-19T09:10:57.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.759+0000 7f7c3b46a740 1 -- 172.21.15.120:0/38634611 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c1c0fd070 con 0x556d9a1c05d0 2024-09-19T09:10:57.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.759+0000 7f7c1a7fc640 1 -- 172.21.15.120:0/38634611 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7c1c0c0090 con 0x556d9a1c05d0 2024-09-19T09:10:57.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.759+0000 7f7c3b46a740 10 client.18207 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:57.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.759+0000 7f7c3b46a740 1 -- 172.21.15.120:0/38634611 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556d9a1d2240 con 0x556d9a1c05d0 2024-09-19T09:10:57.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.759+0000 7f7c1a7fc640 1 -- 172.21.15.120:0/38634611 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c1c0fd070 con 0x556d9a1c05d0 2024-09-19T09:10:57.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.759+0000 7f7c1a7fc640 1 -- 172.21.15.120:0/38634611 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7c1c03b460 con 0x556d9a1c05d0 2024-09-19T09:10:57.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.759+0000 7f7c3b46a740 10 client.18207 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:57.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.759+0000 7f7c3b46a740 10 client.18207 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:10:57.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.759+0000 7f7c3b46a740 20 client.18207 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:57.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.759+0000 7f7c3b46a740 1 --2- 172.21.15.120:0/38634611 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x556d9a1decc0 0x556d9a1e10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:57.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.759+0000 7f7c3b46a740 4 client.18207 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:10:57.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.759+0000 7f7c3b46a740 1 -- 172.21.15.120:0/38634611 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d9a133860 con 0x556d9a1decc0 2024-09-19T09:10:57.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.759+0000 7f7c3259c640 1 --2- 172.21.15.120:0/38634611 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x556d9a1decc0 0x556d9a1e10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:57.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.760+0000 7f7c1a7fc640 10 client.18207 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:10:57.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.760+0000 7f7c3259c640 1 --2- 172.21.15.120:0/38634611 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x556d9a1decc0 0x556d9a1e10b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bfcf6a5315604894 server_cookie=2a93f3d7048c301f in_seq=0 out_seq=0 2024-09-19T09:10:57.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.760+0000 7f7c1a7fc640 1 -- 172.21.15.120:0/38634611 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556d9a133860 con 0x556d9a1decc0 2024-09-19T09:10:57.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.760+0000 7f7c1a7fc640 10 client.18207 handle_command_reply: tid=0 2024-09-19T09:10:57.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.761+0000 7f7c3b46a740 1 client.18207 shutdown 2024-09-19T09:10:57.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.761+0000 7f7c3b46a740 1 -- 172.21.15.120:0/38634611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d9a1c05d0 msgr2=0x556d9a1cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:57.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.761+0000 7f7c3b46a740 1 --2- 172.21.15.120:0/38634611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d9a1c05d0 0x556d9a1cf130 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0f8810 tx=0x7f7c1c03b800 comp rx=0 tx=0).stop 2024-09-19T09:10:57.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.761+0000 7f7c3b46a740 1 -- 172.21.15.120:0/38634611 shutdown_connections 2024-09-19T09:10:57.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.761+0000 7f7c3b46a740 1 --2- 172.21.15.120:0/38634611 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x556d9a1decc0 0x556d9a1e10b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.761+0000 7f7c3b46a740 1 --2- 172.21.15.120:0/38634611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d9a1cf780 0x556d9a1cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.761+0000 7f7c1a7fc640 0 client.18207 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:10:57.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.761+0000 7f7c3b46a740 1 --2- 172.21.15.120:0/38634611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d9a1c1b40 0x556d9a1cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.761+0000 7f7c3b46a740 1 --2- 172.21.15.120:0/38634611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d9a1c05d0 0x556d9a1cf130 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.761+0000 7f7c3b46a740 1 -- 172.21.15.120:0/38634611 >> 172.21.15.120:0/38634611 conn(0x556d9a189310 msgr2=0x556d9a1c2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:57.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.761+0000 7f7c3b46a740 1 -- 172.21.15.120:0/38634611 shutdown_connections 2024-09-19T09:10:57.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.761+0000 7f7c3b46a740 1 -- 172.21.15.120:0/38634611 wait complete. 2024-09-19T09:10:57.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.761+0000 7f7c3b46a740 20 client.18207 trim_cache size 0 max 16384 2024-09-19T09:10:57.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.774+0000 7f7c3b46a740 1 Processor -- start 2024-09-19T09:10:57.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.774+0000 7f7c3b46a740 1 -- start start 2024-09-19T09:10:57.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.774+0000 7f7c3b46a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d9a1c05d0 0x556d9a1db4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:57.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.774+0000 7f7c3b46a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d9a1c1b40 0x556d9a1d0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:57.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.774+0000 7f7c3b46a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d9a1cf780 0x556d9a1d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:57.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.774+0000 7f7c3b46a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341ab500 con 0x556d9a1c05d0 2024-09-19T09:10:57.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.774+0000 7f7c3b46a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341ad040 con 0x556d9a1c1b40 2024-09-19T09:10:57.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.774+0000 7f7c3b46a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341abbd0 con 0x556d9a1cf780 2024-09-19T09:10:57.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.774+0000 7f7c32d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d9a1c05d0 0x556d9a1db4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:57.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.774+0000 7f7c32d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d9a1c05d0 0x556d9a1db4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60678/0 (socket says 172.21.15.120:60678) 2024-09-19T09:10:57.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.774+0000 7f7c32d9d640 1 -- 172.21.15.120:0/2488724911 learned_addr learned my addr 172.21.15.120:0/2488724911 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:57.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.774+0000 7f7c32d9d640 1 -- 172.21.15.120:0/2488724911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d9a1cf780 msgr2=0x556d9a1d10b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:10:57.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.774+0000 7f7c32d9d640 1 --2- 172.21.15.120:0/2488724911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d9a1cf780 0x556d9a1d10b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.774+0000 7f7c32d9d640 1 -- 172.21.15.120:0/2488724911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d9a1c1b40 msgr2=0x556d9a1d0b70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:10:57.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.774+0000 7f7c32d9d640 1 --2- 172.21.15.120:0/2488724911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d9a1c1b40 0x556d9a1d0b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.774+0000 7f7c32d9d640 1 -- 172.21.15.120:0/2488724911 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d9a1d15f0 con 0x556d9a1c05d0 2024-09-19T09:10:57.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.775+0000 7f7c32d9d640 1 --2- 172.21.15.120:0/2488724911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d9a1c05d0 0x556d9a1db4a0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c031550 tx=0x7f7c1c0f3f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dca62d0ed134cb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.775+0000 7f7c28ff9640 1 -- 172.21.15.120:0/2488724911 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c049070 con 0x556d9a1c05d0 2024-09-19T09:10:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.775+0000 7f7c28ff9640 1 -- 172.21.15.120:0/2488724911 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c1c0fd030 con 0x556d9a1c05d0 2024-09-19T09:10:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.775+0000 7f7c28ff9640 1 -- 172.21.15.120:0/2488724911 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c0c06f0 con 0x556d9a1c05d0 2024-09-19T09:10:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.775+0000 7f7c3b46a740 1 -- 172.21.15.120:0/2488724911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d9a1c05d0 msgr2=0x556d9a1db4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.775+0000 7f7c3b46a740 1 --2- 172.21.15.120:0/2488724911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d9a1c05d0 0x556d9a1db4a0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c031550 tx=0x7f7c1c0f3f90 comp rx=0 tx=0).stop 2024-09-19T09:10:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.776+0000 7f7c3b46a740 1 -- 172.21.15.120:0/2488724911 shutdown_connections 2024-09-19T09:10:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.776+0000 7f7c3b46a740 1 --2- 172.21.15.120:0/2488724911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d9a1cf780 0x556d9a1d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.776+0000 7f7c3b46a740 1 --2- 172.21.15.120:0/2488724911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d9a1c1b40 0x556d9a1d0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.776+0000 7f7c3b46a740 1 --2- 172.21.15.120:0/2488724911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d9a1c05d0 0x556d9a1db4a0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.776+0000 7f7c3b46a740 1 -- 172.21.15.120:0/2488724911 >> 172.21.15.120:0/2488724911 conn(0x556d9a189310 msgr2=0x556d9a1c7720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.776+0000 7f7c3b46a740 1 -- 172.21.15.120:0/2488724911 shutdown_connections 2024-09-19T09:10:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.776+0000 7f7c3b46a740 1 -- 172.21.15.120:0/2488724911 wait complete. 2024-09-19T09:10:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.776+0000 7f7c3b46a740 1 -- start start 2024-09-19T09:10:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.776+0000 7f7c3b46a740 1 Processor -- start 2024-09-19T09:10:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.776+0000 7f7c3b46a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d9a1c05d0 0x556d9a1ca130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.776+0000 7f7c3b46a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d9a1c1b40 0x556d9a1d7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.776+0000 7f7c3b46a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d9a1cf780 0x556d9a1d7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.776+0000 7f7c3b46a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341ac970 con 0x556d9a1c1b40 2024-09-19T09:10:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.776+0000 7f7c3b46a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341ac2a0 con 0x556d9a1cf780 2024-09-19T09:10:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.776+0000 7f7c3b46a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341a94c0 con 0x556d9a1c05d0 2024-09-19T09:10:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.776+0000 7f7c3b46a740 10 client.0 _ll_register_callbacks cb 0x7f7c1c0f80e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.776+0000 7f7c3259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d9a1c1b40 0x556d9a1d7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.776+0000 7f7c3259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d9a1c1b40 0x556d9a1d7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60694/0 (socket says 172.21.15.120:60694) 2024-09-19T09:10:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.776+0000 7f7c3259c640 1 -- 172.21.15.120:0/3596835776 learned_addr learned my addr 172.21.15.120:0/3596835776 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.777+0000 7f7c3259c640 1 -- 172.21.15.120:0/3596835776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d9a1c05d0 msgr2=0x556d9a1ca130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.777+0000 7f7c3259c640 1 --2- 172.21.15.120:0/3596835776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d9a1c05d0 0x556d9a1ca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.777+0000 7f7c3259c640 1 -- 172.21.15.120:0/3596835776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d9a1cf780 msgr2=0x556d9a1d7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.777+0000 7f7c3259c640 1 --2- 172.21.15.120:0/3596835776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d9a1cf780 0x556d9a1d7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.777+0000 7f7c3259c640 1 -- 172.21.15.120:0/3596835776 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d9a1e67a0 con 0x556d9a1c1b40 2024-09-19T09:10:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.777+0000 7f7c3259c640 1 --2- 172.21.15.120:0/3596835776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d9a1c1b40 0x556d9a1d7610 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f7c24000c00 tx=0x7f7c240137d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a63c29c8daaf3a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.777+0000 7f7c1a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.777+0000 7f7c1a7fc640 1 -- 172.21.15.120:0/3596835776 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c240186a0 con 0x556d9a1c1b40 2024-09-19T09:10:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.777+0000 7f7c3b46a740 1 -- 172.21.15.120:0/3596835776 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c1c0fd030 con 0x556d9a1c1b40 2024-09-19T09:10:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.777+0000 7f7c1a7fc640 1 -- 172.21.15.120:0/3596835776 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f7c24018840 con 0x556d9a1c1b40 2024-09-19T09:10:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.777+0000 7f7c1a7fc640 1 -- 172.21.15.120:0/3596835776 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7c24018a40 con 0x556d9a1c1b40 2024-09-19T09:10:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.777+0000 7f7c3b46a740 10 client.18213 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.777+0000 7f7c3b46a740 1 -- 172.21.15.120:0/3596835776 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556d9a249da0 con 0x556d9a1c1b40 2024-09-19T09:10:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.777+0000 7f7c1a7fc640 1 -- 172.21.15.120:0/3596835776 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c1c0fd030 con 0x556d9a1c1b40 2024-09-19T09:10:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.777+0000 7f7c1a7fc640 1 -- 172.21.15.120:0/3596835776 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7c240203d0 con 0x556d9a1c1b40 2024-09-19T09:10:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.777+0000 7f7c3b46a740 10 client.18213 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.777+0000 7f7c3b46a740 10 client.18213 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:10:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.777+0000 7f7c3b46a740 20 client.18213 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:57.852 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:10:57.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.777+0000 7f7c3b46a740 1 --2- 172.21.15.120:0/3596835776 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x556d9a24a550 0x556d9a24c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:57.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.778+0000 7f7c3b46a740 4 client.18213 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:10:57.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.778+0000 7f7c3b46a740 1 -- 172.21.15.120:0/3596835776 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556d9a12dd70 con 0x556d9a24a550 2024-09-19T09:10:57.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.778+0000 7f7c32d9d640 1 --2- 172.21.15.120:0/3596835776 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x556d9a24a550 0x556d9a24c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:57.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.779+0000 7f7c1a7fc640 10 client.18213 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:10:57.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.779+0000 7f7c32d9d640 1 --2- 172.21.15.120:0/3596835776 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x556d9a24a550 0x556d9a24c970 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ef30aa960b5fd8d8 server_cookie=45785ee6ad5caffc in_seq=0 out_seq=0 2024-09-19T09:10:57.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.779+0000 7f7c1a7fc640 1 -- 172.21.15.120:0/3596835776 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556d9a12dd70 con 0x556d9a24a550 2024-09-19T09:10:57.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.779+0000 7f7c1a7fc640 10 client.18213 handle_command_reply: tid=0 2024-09-19T09:10:57.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.779+0000 7f7c3b46a740 1 client.18213 shutdown 2024-09-19T09:10:57.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.779+0000 7f7c3b46a740 1 -- 172.21.15.120:0/3596835776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d9a1c1b40 msgr2=0x556d9a1d7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:57.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.779+0000 7f7c3b46a740 1 --2- 172.21.15.120:0/3596835776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d9a1c1b40 0x556d9a1d7610 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f7c24000c00 tx=0x7f7c240137d0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.779+0000 7f7c3b46a740 1 -- 172.21.15.120:0/3596835776 shutdown_connections 2024-09-19T09:10:57.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.779+0000 7f7c3b46a740 1 --2- 172.21.15.120:0/3596835776 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x556d9a24a550 0x556d9a24c970 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.779+0000 7f7c3b46a740 1 --2- 172.21.15.120:0/3596835776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d9a1cf780 0x556d9a1d7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.779+0000 7f7c3b46a740 1 --2- 172.21.15.120:0/3596835776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d9a1c1b40 0x556d9a1d7610 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.779+0000 7f7c1a7fc640 0 client.18213 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:10:57.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.779+0000 7f7c3b46a740 1 --2- 172.21.15.120:0/3596835776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d9a1c05d0 0x556d9a1ca130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.779+0000 7f7c3b46a740 1 -- 172.21.15.120:0/3596835776 >> 172.21.15.120:0/3596835776 conn(0x556d9a189310 msgr2=0x556d9a1bc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:57.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.779+0000 7f7c3b46a740 1 -- 172.21.15.120:0/3596835776 shutdown_connections 2024-09-19T09:10:57.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.779+0000 7f7c3b46a740 1 -- 172.21.15.120:0/3596835776 wait complete. 2024-09-19T09:10:57.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.779+0000 7f7c3b46a740 20 client.18213 trim_cache size 0 max 16384 2024-09-19T09:10:57.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.780+0000 7f7c39091640 1 -- 172.21.15.120:0/2099881131 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7c200a3240 msgr2=0x7f7c200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:57.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.780+0000 7f7c39091640 1 --2- 172.21.15.120:0/2099881131 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7c200a3240 0x7f7c200a5700 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c007ee0 tx=0x7f7c2c007430 comp rx=0 tx=0).stop 2024-09-19T09:10:57.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.780+0000 7f7c39091640 1 -- 172.21.15.120:0/2099881131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c34100a80 msgr2=0x7f7c341985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:57.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.780+0000 7f7c39091640 1 --2- 172.21.15.120:0/2099881131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c34100a80 0x7f7c341985e0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c009a50 tx=0x7f7c1c004270 comp rx=0 tx=0).stop 2024-09-19T09:10:57.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.780+0000 7f7c39091640 1 -- 172.21.15.120:0/2099881131 shutdown_connections 2024-09-19T09:10:57.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.780+0000 7f7c39091640 1 --2- 172.21.15.120:0/2099881131 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7c200a3240 0x7f7c200a5700 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.780+0000 7f7c39091640 1 --2- 172.21.15.120:0/2099881131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c34106a80 0x7f7c3419ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.780+0000 7f7c39091640 1 --2- 172.21.15.120:0/2099881131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c34101420 0x7f7c34198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.780+0000 7f7c39091640 1 --2- 172.21.15.120:0/2099881131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c34100a80 0x7f7c341985e0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:57.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.780+0000 7f7c39091640 1 -- 172.21.15.120:0/2099881131 >> 172.21.15.120:0/2099881131 conn(0x7f7c340fc7a0 msgr2=0x7f7c340fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:57.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.780+0000 7f7c39091640 1 -- 172.21.15.120:0/2099881131 shutdown_connections 2024-09-19T09:10:57.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:57.780+0000 7f7c39091640 1 -- 172.21.15.120:0/2099881131 wait complete. 2024-09-19T09:10:58.060 INFO:tasks.workunit.client.0.smithi120.stdout: 1 707664 8.89 MB/sec execute 293 sec latency 21420.860 ms 2024-09-19T09:10:58.722 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:10:58.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.851+0000 7f163b55f640 1 -- 172.21.15.120:0/3405323748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16341025f0 msgr2=0x7f1634108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:58.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.851+0000 7f163b55f640 1 --2- 172.21.15.120:0/3405323748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16341025f0 0x7f1634108e80 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f162c009900 tx=0x7f162c02a410 comp rx=0 tx=0).stop 2024-09-19T09:10:58.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.851+0000 7f163b55f640 1 -- 172.21.15.120:0/3405323748 shutdown_connections 2024-09-19T09:10:58.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.851+0000 7f163b55f640 1 --2- 172.21.15.120:0/3405323748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16341025f0 0x7f1634108e80 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.851+0000 7f163b55f640 1 --2- 172.21.15.120:0/3405323748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1634101c30 0x7f16341020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.851+0000 7f163b55f640 1 --2- 172.21.15.120:0/3405323748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1634100a40 0x7f1634100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.851+0000 7f163b55f640 1 -- 172.21.15.120:0/3405323748 >> 172.21.15.120:0/3405323748 conn(0x7f16340fc1e0 msgr2=0x7f16340fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:58.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.851+0000 7f163b55f640 1 -- 172.21.15.120:0/3405323748 shutdown_connections 2024-09-19T09:10:58.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.851+0000 7f163b55f640 1 -- 172.21.15.120:0/3405323748 wait complete. 2024-09-19T09:10:58.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.851+0000 7f163b55f640 1 Processor -- start 2024-09-19T09:10:58.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.851+0000 7f163b55f640 1 -- start start 2024-09-19T09:10:58.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.852+0000 7f163b55f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1634100a40 0x7f163406a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:58.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.852+0000 7f163b55f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1634101c30 0x7f163419cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:58.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.852+0000 7f163b55f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16341025f0 0x7f163419f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:58.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.852+0000 7f163b55f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f163410c710 con 0x7f1634100a40 2024-09-19T09:10:58.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.852+0000 7f163b55f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f163410c590 con 0x7f1634101c30 2024-09-19T09:10:58.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.852+0000 7f163b55f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f163410c890 con 0x7f16341025f0 2024-09-19T09:10:58.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.852+0000 7f16392fb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1634100a40 0x7f163406a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:58.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.852+0000 7f16392fb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1634100a40 0x7f163406a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60716/0 (socket says 172.21.15.120:60716) 2024-09-19T09:10:58.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.852+0000 7f16392fb640 1 -- 172.21.15.120:0/1114806379 learned_addr learned my addr 172.21.15.120:0/1114806379 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:58.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.852+0000 7f1639afc640 1 --2- 172.21.15.120:0/1114806379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16341025f0 0x7f163419f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:58.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.852+0000 7f16392fb640 1 -- 172.21.15.120:0/1114806379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16341025f0 msgr2=0x7f163419f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:58.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.852+0000 7f16392fb640 1 --2- 172.21.15.120:0/1114806379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16341025f0 0x7f163419f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.852+0000 7f16392fb640 1 -- 172.21.15.120:0/1114806379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1634101c30 msgr2=0x7f163419cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:58.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.852+0000 7f16392fb640 1 --2- 172.21.15.120:0/1114806379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1634101c30 0x7f163419cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.852+0000 7f16392fb640 1 -- 172.21.15.120:0/1114806379 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f163419f9f0 con 0x7f1634100a40 2024-09-19T09:10:58.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.852+0000 7f16392fb640 1 --2- 172.21.15.120:0/1114806379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1634100a40 0x7f163406a410 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f163000ea80 tx=0x7f163000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:58.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.852+0000 7f162a7fc640 1 -- 172.21.15.120:0/1114806379 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163000ce40 con 0x7f1634100a40 2024-09-19T09:10:58.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.852+0000 7f163b55f640 1 -- 172.21.15.120:0/1114806379 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f163419fc80 con 0x7f1634100a40 2024-09-19T09:10:58.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.852+0000 7f162a7fc640 1 -- 172.21.15.120:0/1114806379 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1630011950 con 0x7f1634100a40 2024-09-19T09:10:58.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.852+0000 7f162a7fc640 1 -- 172.21.15.120:0/1114806379 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163001ab80 con 0x7f1634100a40 2024-09-19T09:10:58.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.852+0000 7f163b55f640 1 -- 172.21.15.120:0/1114806379 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16341a01d0 con 0x7f1634100a40 2024-09-19T09:10:58.862 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.855+0000 7f162a7fc640 1 -- 172.21.15.120:0/1114806379 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f16300233f0 con 0x7f1634100a40 2024-09-19T09:10:58.862 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.855+0000 7f163e3d5740 1 Processor -- start 2024-09-19T09:10:58.862 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.855+0000 7f163e3d5740 1 -- start start 2024-09-19T09:10:58.862 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.855+0000 7f163e3d5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cb2b29a0 0x5652cb2b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:58.862 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.855+0000 7f163e3d5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652cb2b32e0 0x5652cb2b7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.855+0000 7f163e3d5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5652cb2b8370 0x5652cb2ba830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.855+0000 7f163e3d5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5652cb2162f0 con 0x5652cb2b32e0 2024-09-19T09:10:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.855+0000 7f163e3d5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5652cb1be270 con 0x5652cb2b8370 2024-09-19T09:10:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.855+0000 7f163e3d5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5652cb2122b0 con 0x5652cb2b29a0 2024-09-19T09:10:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.855+0000 7f16392fb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652cb2b32e0 0x5652cb2b7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.855+0000 7f16392fb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652cb2b32e0 0x5652cb2b7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60722/0 (socket says 172.21.15.120:60722) 2024-09-19T09:10:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.855+0000 7f16392fb640 1 -- 172.21.15.120:0/3970178607 learned_addr learned my addr 172.21.15.120:0/3970178607 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.855+0000 7f1638afa640 1 --2- 172.21.15.120:0/3970178607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5652cb2b8370 0x5652cb2ba830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.856+0000 7f1639afc640 1 --2- 172.21.15.120:0/3970178607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cb2b29a0 0x5652cb2b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.856+0000 7f162a7fc640 1 --2- 172.21.15.120:0/1114806379 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16200a32f0 0x7f16200a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.856+0000 7f16392fb640 1 -- 172.21.15.120:0/3970178607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cb2b29a0 msgr2=0x5652cb2b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.856+0000 7f16392fb640 1 --2- 172.21.15.120:0/3970178607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cb2b29a0 0x5652cb2b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.856+0000 7f16392fb640 1 -- 172.21.15.120:0/3970178607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5652cb2b8370 msgr2=0x5652cb2ba830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.856+0000 7f16392fb640 1 --2- 172.21.15.120:0/3970178607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5652cb2b8370 0x5652cb2ba830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.856+0000 7f16392fb640 1 -- 172.21.15.120:0/3970178607 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652cb1bd340 con 0x5652cb2b32e0 2024-09-19T09:10:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.856+0000 7f1639afc640 1 --2- 172.21.15.120:0/1114806379 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16200a32f0 0x7f16200a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.856+0000 7f162a7fc640 1 -- 172.21.15.120:0/1114806379 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f16300cfff0 con 0x7f1634100a40 2024-09-19T09:10:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.856+0000 7f16392fb640 1 --2- 172.21.15.120:0/3970178607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652cb2b32e0 0x5652cb2b7da0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f16300c9ab0 tx=0x7f163006b350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb9cf5f90d40529c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.856+0000 7f161f7fe640 1 -- 172.21.15.120:0/3970178607 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163009c150 con 0x5652cb2b32e0 2024-09-19T09:10:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.856+0000 7f161f7fe640 1 -- 172.21.15.120:0/3970178607 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1630098040 con 0x5652cb2b32e0 2024-09-19T09:10:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.856+0000 7f161f7fe640 1 -- 172.21.15.120:0/3970178607 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16300d3780 con 0x5652cb2b32e0 2024-09-19T09:10:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.857+0000 7f163e3d5740 1 -- 172.21.15.120:0/3970178607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652cb2b32e0 msgr2=0x5652cb2b7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.857+0000 7f163e3d5740 1 --2- 172.21.15.120:0/3970178607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652cb2b32e0 0x5652cb2b7da0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f16300c9ab0 tx=0x7f163006b350 comp rx=0 tx=0).stop 2024-09-19T09:10:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.857+0000 7f1639afc640 1 --2- 172.21.15.120:0/1114806379 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16200a32f0 0x7f16200a57b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f162c006010 tx=0x7f162c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.857+0000 7f163e3d5740 1 -- 172.21.15.120:0/3970178607 shutdown_connections 2024-09-19T09:10:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.857+0000 7f163e3d5740 1 --2- 172.21.15.120:0/3970178607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5652cb2b8370 0x5652cb2ba830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.857+0000 7f163e3d5740 1 --2- 172.21.15.120:0/3970178607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652cb2b32e0 0x5652cb2b7da0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.857+0000 7f163e3d5740 1 --2- 172.21.15.120:0/3970178607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cb2b29a0 0x5652cb2b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.857+0000 7f163e3d5740 1 -- 172.21.15.120:0/3970178607 >> 172.21.15.120:0/3970178607 conn(0x5652cb2ae010 msgr2=0x5652cb2ae410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.857+0000 7f163e3d5740 1 -- 172.21.15.120:0/3970178607 shutdown_connections 2024-09-19T09:10:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.857+0000 7f163e3d5740 1 -- 172.21.15.120:0/3970178607 wait complete. 2024-09-19T09:10:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.857+0000 7f163e3d5740 1 -- start start 2024-09-19T09:10:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.858+0000 7f163e3d5740 1 Processor -- start 2024-09-19T09:10:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.858+0000 7f163e3d5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5652cb2b29a0 0x5652cb2bcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.858+0000 7f163e3d5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cb2b32e0 0x5652cb2bd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.858+0000 7f163e3d5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652cb2b8370 0x5652cb2c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.858+0000 7f163e3d5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5652cb2bb2e0 con 0x5652cb2b8370 2024-09-19T09:10:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.858+0000 7f163e3d5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5652cb19f290 con 0x5652cb2b29a0 2024-09-19T09:10:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.858+0000 7f163e3d5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5652cb2bb460 con 0x5652cb2b32e0 2024-09-19T09:10:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.858+0000 7f163e3d5740 10 client.0 _ll_register_callbacks cb 0x5652cb1d0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.858+0000 7f1639afc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652cb2b8370 0x5652cb2c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.858+0000 7f1639afc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652cb2b8370 0x5652cb2c40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60736/0 (socket says 172.21.15.120:60736) 2024-09-19T09:10:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.858+0000 7f1639afc640 1 -- 172.21.15.120:0/617789949 learned_addr learned my addr 172.21.15.120:0/617789949 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.858+0000 7f16392fb640 1 --2- 172.21.15.120:0/617789949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5652cb2b29a0 0x5652cb2bcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.858+0000 7f1638afa640 1 --2- 172.21.15.120:0/617789949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cb2b32e0 0x5652cb2bd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.858+0000 7f1639afc640 1 -- 172.21.15.120:0/617789949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cb2b32e0 msgr2=0x5652cb2bd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.858+0000 7f1639afc640 1 --2- 172.21.15.120:0/617789949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cb2b32e0 0x5652cb2bd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.858+0000 7f1639afc640 1 -- 172.21.15.120:0/617789949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5652cb2b29a0 msgr2=0x5652cb2bcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.858+0000 7f1639afc640 1 --2- 172.21.15.120:0/617789949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5652cb2b29a0 0x5652cb2bcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.858+0000 7f1639afc640 1 -- 172.21.15.120:0/617789949 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652caf29210 con 0x5652cb2b8370 2024-09-19T09:10:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.859+0000 7f1639afc640 1 --2- 172.21.15.120:0/617789949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652cb2b8370 0x5652cb2c40c0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f162c009900 tx=0x7f162c0315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af9e5118c757f10 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.859+0000 7f161cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.859+0000 7f161cff9640 1 -- 172.21.15.120:0/617789949 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c04d020 con 0x5652cb2b8370 2024-09-19T09:10:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.859+0000 7f161cff9640 1 -- 172.21.15.120:0/617789949 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f162c004ad0 con 0x5652cb2b8370 2024-09-19T09:10:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.859+0000 7f163e3d5740 1 -- 172.21.15.120:0/617789949 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1630098040 con 0x5652cb2b8370 2024-09-19T09:10:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.859+0000 7f161cff9640 1 -- 172.21.15.120:0/617789949 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f162c035040 con 0x5652cb2b8370 2024-09-19T09:10:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.859+0000 7f163e3d5740 10 client.18225 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.859+0000 7f163e3d5740 1 -- 172.21.15.120:0/617789949 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5652caff5620 con 0x5652cb2b8370 2024-09-19T09:10:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.859+0000 7f161cff9640 1 -- 172.21.15.120:0/617789949 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1630098040 con 0x5652cb2b8370 2024-09-19T09:10:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.859+0000 7f161cff9640 1 -- 172.21.15.120:0/617789949 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f162c003710 con 0x5652cb2b8370 2024-09-19T09:10:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.859+0000 7f163e3d5740 10 client.18225 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.859+0000 7f163e3d5740 10 client.18225 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:10:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.859+0000 7f163e3d5740 20 client.18225 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.859+0000 7f163e3d5740 1 --2- 172.21.15.120:0/617789949 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5652cb2c8820 0x5652cb2caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.859+0000 7f163e3d5740 4 client.18225 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:10:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.859+0000 7f163e3d5740 1 -- 172.21.15.120:0/617789949 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652cb223a20 con 0x5652cb2c8820 2024-09-19T09:10:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.860+0000 7f16392fb640 1 --2- 172.21.15.120:0/617789949 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5652cb2c8820 0x5652cb2caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.860+0000 7f161cff9640 10 client.18225 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.860+0000 7f16392fb640 1 --2- 172.21.15.120:0/617789949 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5652cb2c8820 0x5652cb2caca0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c27e452e55e9e149 server_cookie=a4afe4ad9ce2ee4 in_seq=0 out_seq=0 2024-09-19T09:10:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.861+0000 7f161cff9640 1 -- 172.21.15.120:0/617789949 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5652cb223a20 con 0x5652cb2c8820 2024-09-19T09:10:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.861+0000 7f161cff9640 10 client.18225 handle_command_reply: tid=0 2024-09-19T09:10:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.861+0000 7f163e3d5740 1 client.18225 shutdown 2024-09-19T09:10:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.861+0000 7f163e3d5740 1 -- 172.21.15.120:0/617789949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652cb2b8370 msgr2=0x5652cb2c40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.861+0000 7f163e3d5740 1 --2- 172.21.15.120:0/617789949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652cb2b8370 0x5652cb2c40c0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f162c009900 tx=0x7f162c0315b0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.861+0000 7f163e3d5740 1 -- 172.21.15.120:0/617789949 shutdown_connections 2024-09-19T09:10:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.861+0000 7f163e3d5740 1 --2- 172.21.15.120:0/617789949 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5652cb2c8820 0x5652cb2caca0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.861+0000 7f163e3d5740 1 --2- 172.21.15.120:0/617789949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652cb2b8370 0x5652cb2c40c0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.861+0000 7f163e3d5740 1 --2- 172.21.15.120:0/617789949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cb2b32e0 0x5652cb2bd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.861+0000 7f161cff9640 0 client.18225 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.861+0000 7f163e3d5740 1 --2- 172.21.15.120:0/617789949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5652cb2b29a0 0x5652cb2bcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.861+0000 7f163e3d5740 1 -- 172.21.15.120:0/617789949 >> 172.21.15.120:0/617789949 conn(0x5652cb2ae010 msgr2=0x5652cb2af820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.861+0000 7f163e3d5740 1 -- 172.21.15.120:0/617789949 shutdown_connections 2024-09-19T09:10:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.861+0000 7f163e3d5740 1 -- 172.21.15.120:0/617789949 wait complete. 2024-09-19T09:10:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.861+0000 7f163e3d5740 20 client.18225 trim_cache size 0 max 16384 2024-09-19T09:10:58.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.875+0000 7f163e3d5740 1 Processor -- start 2024-09-19T09:10:58.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.875+0000 7f163e3d5740 1 -- start start 2024-09-19T09:10:58.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.875+0000 7f163e3d5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652cb2b29a0 0x5652cb2bcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:58.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.875+0000 7f163e3d5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5652cb2b32e0 0x5652cb2c4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:58.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.875+0000 7f163e3d5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cb2b8370 0x5652cb2bd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:58.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.875+0000 7f163e3d5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f16341a8fc0 con 0x5652cb2b29a0 2024-09-19T09:10:58.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.875+0000 7f163e3d5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f16341aab00 con 0x5652cb2b32e0 2024-09-19T09:10:58.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.875+0000 7f163e3d5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f16341a9690 con 0x5652cb2b8370 2024-09-19T09:10:58.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.876+0000 7f16392fb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652cb2b29a0 0x5652cb2bcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:58.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.876+0000 7f16392fb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652cb2b29a0 0x5652cb2bcb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60738/0 (socket says 172.21.15.120:60738) 2024-09-19T09:10:58.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.876+0000 7f16392fb640 1 -- 172.21.15.120:0/4046983692 learned_addr learned my addr 172.21.15.120:0/4046983692 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:58.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.876+0000 7f16392fb640 1 -- 172.21.15.120:0/4046983692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cb2b8370 msgr2=0x5652cb2bd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:58.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.876+0000 7f16392fb640 1 --2- 172.21.15.120:0/4046983692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cb2b8370 0x5652cb2bd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.876+0000 7f16392fb640 1 -- 172.21.15.120:0/4046983692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5652cb2b32e0 msgr2=0x5652cb2c4080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:10:58.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.876+0000 7f16392fb640 1 --2- 172.21.15.120:0/4046983692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5652cb2b32e0 0x5652cb2c4080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.876+0000 7f16392fb640 1 -- 172.21.15.120:0/4046983692 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652cb2aeca0 con 0x5652cb2b29a0 2024-09-19T09:10:58.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.876+0000 7f16392fb640 1 --2- 172.21.15.120:0/4046983692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652cb2b29a0 0x5652cb2bcb40 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f163000ea50 tx=0x7f163001a5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88638f805f730bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:58.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.876+0000 7f161effd640 1 -- 172.21.15.120:0/4046983692 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1630064060 con 0x5652cb2b29a0 2024-09-19T09:10:58.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.876+0000 7f161effd640 1 -- 172.21.15.120:0/4046983692 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1630098040 con 0x5652cb2b29a0 2024-09-19T09:10:58.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.876+0000 7f161effd640 1 -- 172.21.15.120:0/4046983692 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1630064230 con 0x5652cb2b29a0 2024-09-19T09:10:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.877+0000 7f163e3d5740 1 -- 172.21.15.120:0/4046983692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652cb2b29a0 msgr2=0x5652cb2bcb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.877+0000 7f163e3d5740 1 --2- 172.21.15.120:0/4046983692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652cb2b29a0 0x5652cb2bcb40 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f163000ea50 tx=0x7f163001a5a0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.877+0000 7f163e3d5740 1 -- 172.21.15.120:0/4046983692 shutdown_connections 2024-09-19T09:10:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.877+0000 7f163e3d5740 1 --2- 172.21.15.120:0/4046983692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cb2b8370 0x5652cb2bd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.877+0000 7f163e3d5740 1 --2- 172.21.15.120:0/4046983692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5652cb2b32e0 0x5652cb2c4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.877+0000 7f163e3d5740 1 --2- 172.21.15.120:0/4046983692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652cb2b29a0 0x5652cb2bcb40 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.877+0000 7f163e3d5740 1 -- 172.21.15.120:0/4046983692 >> 172.21.15.120:0/4046983692 conn(0x5652cb2ae010 msgr2=0x5652cb2b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.877+0000 7f163e3d5740 1 -- 172.21.15.120:0/4046983692 shutdown_connections 2024-09-19T09:10:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.877+0000 7f163e3d5740 1 -- 172.21.15.120:0/4046983692 wait complete. 2024-09-19T09:10:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.877+0000 7f163e3d5740 1 -- start start 2024-09-19T09:10:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.877+0000 7f163e3d5740 1 Processor -- start 2024-09-19T09:10:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.878+0000 7f163e3d5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652cb2b29a0 0x5652cb2c1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.878+0000 7f163e3d5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5652cb2b32e0 0x5652cb2c1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.878+0000 7f163e3d5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cb2b8370 0x5652cb2c2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.878+0000 7f163e3d5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f16341aa430 con 0x5652cb2b29a0 2024-09-19T09:10:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.878+0000 7f163e3d5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f16341a9d60 con 0x5652cb2b32e0 2024-09-19T09:10:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.878+0000 7f163e3d5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f16341a6f80 con 0x5652cb2b8370 2024-09-19T09:10:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.878+0000 7f163e3d5740 10 client.0 _ll_register_callbacks cb 0x7f1624005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.878+0000 7f16392fb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652cb2b29a0 0x5652cb2c1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.878+0000 7f16392fb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652cb2b29a0 0x5652cb2c1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60740/0 (socket says 172.21.15.120:60740) 2024-09-19T09:10:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.878+0000 7f16392fb640 1 -- 172.21.15.120:0/661150449 learned_addr learned my addr 172.21.15.120:0/661150449 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.878+0000 7f1638afa640 1 --2- 172.21.15.120:0/661150449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5652cb2b32e0 0x5652cb2c1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.878+0000 7f1639afc640 1 --2- 172.21.15.120:0/661150449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cb2b8370 0x5652cb2c2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.878+0000 7f16392fb640 1 -- 172.21.15.120:0/661150449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cb2b8370 msgr2=0x5652cb2c2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.878+0000 7f16392fb640 1 --2- 172.21.15.120:0/661150449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cb2b8370 0x5652cb2c2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.878+0000 7f16392fb640 1 -- 172.21.15.120:0/661150449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5652cb2b32e0 msgr2=0x5652cb2c1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.878+0000 7f16392fb640 1 --2- 172.21.15.120:0/661150449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5652cb2b32e0 0x5652cb2c1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.878+0000 7f16392fb640 1 -- 172.21.15.120:0/661150449 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652cb2c30f0 con 0x5652cb2b29a0 2024-09-19T09:10:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.878+0000 7f16392fb640 1 --2- 172.21.15.120:0/661150449 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652cb2b29a0 0x5652cb2c1600 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f1630019000 tx=0x7f16300981f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ccc2b8e2d5a9d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.878+0000 7f161cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.878+0000 7f161cff9640 1 -- 172.21.15.120:0/661150449 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16300a0c20 con 0x5652cb2b29a0 2024-09-19T09:10:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.878+0000 7f161cff9640 1 -- 172.21.15.120:0/661150449 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f163009a6b0 con 0x5652cb2b29a0 2024-09-19T09:10:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.878+0000 7f163e3d5740 1 -- 172.21.15.120:0/661150449 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1630098040 con 0x5652cb2b29a0 2024-09-19T09:10:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.879+0000 7f161cff9640 1 -- 172.21.15.120:0/661150449 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f163009bf70 con 0x5652cb2b29a0 2024-09-19T09:10:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.879+0000 7f163e3d5740 10 client.18231 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.879+0000 7f163e3d5740 1 -- 172.21.15.120:0/661150449 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5652cb2c4400 con 0x5652cb2b29a0 2024-09-19T09:10:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.879+0000 7f161cff9640 1 -- 172.21.15.120:0/661150449 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1630098040 con 0x5652cb2b29a0 2024-09-19T09:10:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.879+0000 7f161cff9640 1 -- 172.21.15.120:0/661150449 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f163006d070 con 0x5652cb2b29a0 2024-09-19T09:10:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.879+0000 7f163e3d5740 10 client.18231 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.879+0000 7f163e3d5740 10 client.18231 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:10:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.879+0000 7f163e3d5740 20 client.18231 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.879+0000 7f163e3d5740 1 --2- 172.21.15.120:0/661150449 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5652cb3337f0 0x5652cb335be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.879+0000 7f163e3d5740 4 client.18231 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:10:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.879+0000 7f163e3d5740 1 -- 172.21.15.120:0/661150449 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5652cb21dd70 con 0x5652cb3337f0 2024-09-19T09:10:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.879+0000 7f1638afa640 1 --2- 172.21.15.120:0/661150449 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5652cb3337f0 0x5652cb335be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.880+0000 7f161cff9640 10 client.18231 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.880+0000 7f1638afa640 1 --2- 172.21.15.120:0/661150449 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5652cb3337f0 0x5652cb335be0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2513365f0c60fdca server_cookie=95cc71616b200fe2 in_seq=0 out_seq=0 2024-09-19T09:10:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.896+0000 7f161cff9640 1 -- 172.21.15.120:0/661150449 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5652cb21dd70 con 0x5652cb3337f0 2024-09-19T09:10:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.896+0000 7f161cff9640 10 client.18231 handle_command_reply: tid=0 2024-09-19T09:10:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.896+0000 7f163e3d5740 1 client.18231 shutdown 2024-09-19T09:10:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.896+0000 7f163e3d5740 1 -- 172.21.15.120:0/661150449 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652cb2b29a0 msgr2=0x5652cb2c1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.896+0000 7f163e3d5740 1 --2- 172.21.15.120:0/661150449 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652cb2b29a0 0x5652cb2c1600 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f1630019000 tx=0x7f16300981f0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.896+0000 7f163e3d5740 1 -- 172.21.15.120:0/661150449 shutdown_connections 2024-09-19T09:10:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.896+0000 7f163e3d5740 1 --2- 172.21.15.120:0/661150449 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5652cb3337f0 0x5652cb335be0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.896+0000 7f163e3d5740 1 --2- 172.21.15.120:0/661150449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5652cb2b8370 0x5652cb2c2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.896+0000 7f163e3d5740 1 --2- 172.21.15.120:0/661150449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5652cb2b32e0 0x5652cb2c1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.896+0000 7f161cff9640 0 client.18231 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:10:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.896+0000 7f163e3d5740 1 --2- 172.21.15.120:0/661150449 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5652cb2b29a0 0x5652cb2c1600 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.896+0000 7f163e3d5740 1 -- 172.21.15.120:0/661150449 >> 172.21.15.120:0/661150449 conn(0x5652cb2ae010 msgr2=0x5652cb2bcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.896+0000 7f163e3d5740 1 -- 172.21.15.120:0/661150449 shutdown_connections 2024-09-19T09:10:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.896+0000 7f163e3d5740 1 -- 172.21.15.120:0/661150449 wait complete. 2024-09-19T09:10:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.896+0000 7f163e3d5740 20 client.18231 trim_cache size 0 max 16384 2024-09-19T09:10:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:10:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:10:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:10:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:10:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.897+0000 7f163b55f640 1 -- 172.21.15.120:0/1114806379 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16200a32f0 msgr2=0x7f16200a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.897+0000 7f163b55f640 1 --2- 172.21.15.120:0/1114806379 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16200a32f0 0x7f16200a57b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f162c006010 tx=0x7f162c009260 comp rx=0 tx=0).stop 2024-09-19T09:10:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.897+0000 7f163b55f640 1 -- 172.21.15.120:0/1114806379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1634100a40 msgr2=0x7f163406a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.897+0000 7f163b55f640 1 --2- 172.21.15.120:0/1114806379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1634100a40 0x7f163406a410 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f163000ea80 tx=0x7f163000ef40 comp rx=0 tx=0).stop 2024-09-19T09:10:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.897+0000 7f163b55f640 1 -- 172.21.15.120:0/1114806379 shutdown_connections 2024-09-19T09:10:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.897+0000 7f163b55f640 1 --2- 172.21.15.120:0/1114806379 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16200a32f0 0x7f16200a57b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.897+0000 7f163b55f640 1 --2- 172.21.15.120:0/1114806379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16341025f0 0x7f163419f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.897+0000 7f163b55f640 1 --2- 172.21.15.120:0/1114806379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1634101c30 0x7f163419cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.897+0000 7f163b55f640 1 --2- 172.21.15.120:0/1114806379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1634100a40 0x7f163406a410 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.897+0000 7f163b55f640 1 -- 172.21.15.120:0/1114806379 >> 172.21.15.120:0/1114806379 conn(0x7f16340fc1e0 msgr2=0x7f1634104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.897+0000 7f163b55f640 1 -- 172.21.15.120:0/1114806379 shutdown_connections 2024-09-19T09:10:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:10:58.897+0000 7f163b55f640 1 -- 172.21.15.120:0/1114806379 wait complete. 2024-09-19T09:10:59.021 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:10:59.060 INFO:tasks.workunit.client.0.smithi120.stdout: 1 707664 8.86 MB/sec execute 294 sec latency 22420.959 ms 2024-09-19T09:10:59.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:10:58 smithi120 ceph-mon[31150]: pgmap v585: 97 pgs: 97 active+clean; 26 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 58 op/s 2024-09-19T09:10:59.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:10:58 smithi196 ceph-mon[35150]: pgmap v585: 97 pgs: 97 active+clean; 26 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 58 op/s 2024-09-19T09:10:59.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:10:58 smithi153 ceph-mon[35071]: pgmap v585: 97 pgs: 97 active+clean; 26 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 58 op/s 2024-09-19T09:10:59.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.511+0000 7f568eeec640 1 -- 172.21.15.120:0/2398399150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5688106f20 msgr2=0x7f568810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:59.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.511+0000 7f568eeec640 1 --2- 172.21.15.120:0/2398399150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5688106f20 0x7f568810eaa0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f5680009ca0 tx=0x7f568002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.511+0000 7f568eeec640 1 -- 172.21.15.120:0/2398399150 shutdown_connections 2024-09-19T09:10:59.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.511+0000 7f568eeec640 1 --2- 172.21.15.120:0/2398399150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5688106f20 0x7f568810eaa0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.511+0000 7f568eeec640 1 --2- 172.21.15.120:0/2398399150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56881027a0 0x7f56881067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.511+0000 7f568eeec640 1 --2- 172.21.15.120:0/2398399150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5688101df0 0x7f56881021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.511+0000 7f568eeec640 1 -- 172.21.15.120:0/2398399150 >> 172.21.15.120:0/2398399150 conn(0x7f56880fd8f0 msgr2=0x7f56880ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:59.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.511+0000 7f568eeec640 1 -- 172.21.15.120:0/2398399150 shutdown_connections 2024-09-19T09:10:59.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.511+0000 7f568eeec640 1 -- 172.21.15.120:0/2398399150 wait complete. 2024-09-19T09:10:59.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.512+0000 7f568eeec640 1 Processor -- start 2024-09-19T09:10:59.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.512+0000 7f568eeec640 1 -- start start 2024-09-19T09:10:59.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.512+0000 7f568eeec640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5688101df0 0x7f5688198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:59.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.512+0000 7f568eeec640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f56881027a0 0x7f5688198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:59.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.512+0000 7f568eeec640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5688106f20 0x7f568819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:59.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.512+0000 7f568eeec640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5688111fa0 con 0x7f56881027a0 2024-09-19T09:10:59.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.512+0000 7f568eeec640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5688111e20 con 0x7f5688101df0 2024-09-19T09:10:59.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.512+0000 7f568eeec640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5688112120 con 0x7f5688106f20 2024-09-19T09:10:59.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.512+0000 7f567ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f56881027a0 0x7f5688198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:59.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.512+0000 7f567ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f56881027a0 0x7f5688198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60760/0 (socket says 172.21.15.120:60760) 2024-09-19T09:10:59.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.512+0000 7f567ffff640 1 -- 172.21.15.120:0/1953285238 learned_addr learned my addr 172.21.15.120:0/1953285238 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:59.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.512+0000 7f568cc88640 1 --2- 172.21.15.120:0/1953285238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5688101df0 0x7f5688198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:59.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.512+0000 7f568d489640 1 --2- 172.21.15.120:0/1953285238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5688106f20 0x7f568819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:59.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.512+0000 7f567ffff640 1 -- 172.21.15.120:0/1953285238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5688106f20 msgr2=0x7f568819cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:59.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.512+0000 7f567ffff640 1 --2- 172.21.15.120:0/1953285238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5688106f20 0x7f568819cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.512+0000 7f567ffff640 1 -- 172.21.15.120:0/1953285238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5688101df0 msgr2=0x7f5688198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:59.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.512+0000 7f567ffff640 1 --2- 172.21.15.120:0/1953285238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5688101df0 0x7f5688198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.512+0000 7f567ffff640 1 -- 172.21.15.120:0/1953285238 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568819d4b0 con 0x7f56881027a0 2024-09-19T09:10:59.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.513+0000 7f567ffff640 1 --2- 172.21.15.120:0/1953285238 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f56881027a0 0x7f5688198a40 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f567800d950 tx=0x7f567800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:59.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.513+0000 7f567dffb640 1 -- 172.21.15.120:0/1953285238 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5678017070 con 0x7f56881027a0 2024-09-19T09:10:59.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.513+0000 7f567dffb640 1 -- 172.21.15.120:0/1953285238 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f567800de80 con 0x7f56881027a0 2024-09-19T09:10:59.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.513+0000 7f568eeec640 1 -- 172.21.15.120:0/1953285238 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f568819d7a0 con 0x7f56881027a0 2024-09-19T09:10:59.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.513+0000 7f567dffb640 1 -- 172.21.15.120:0/1953285238 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567800f620 con 0x7f56881027a0 2024-09-19T09:10:59.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.513+0000 7f568eeec640 1 -- 172.21.15.120:0/1953285238 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56881a5080 con 0x7f56881027a0 2024-09-19T09:10:59.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.515+0000 7f567dffb640 1 -- 172.21.15.120:0/1953285238 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5678009c30 con 0x7f56881027a0 2024-09-19T09:10:59.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.515+0000 7f56912c5740 1 Processor -- start 2024-09-19T09:10:59.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.516+0000 7f56912c5740 1 -- start start 2024-09-19T09:10:59.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.516+0000 7f56912c5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd49e6cb40 0x55fd49e6b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:59.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.516+0000 7f567dffb640 1 --2- 172.21.15.120:0/1953285238 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f56700a3240 0x7f56700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:59.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.516+0000 7f56912c5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd49e6b5d0 0x55fd49e6ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:59.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.516+0000 7f56912c5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd49e7a780 0x55fd49e7cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:59.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.516+0000 7f56912c5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fd49dd12f0 con 0x55fd49e7a780 2024-09-19T09:10:59.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.516+0000 7f56912c5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fd49d78cf0 con 0x55fd49e6cb40 2024-09-19T09:10:59.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.516+0000 7f56912c5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fd49dcd2b0 con 0x55fd49e6b5d0 2024-09-19T09:10:59.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.516+0000 7f568cc88640 1 --2- 172.21.15.120:0/1953285238 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f56700a3240 0x7f56700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:59.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.516+0000 7f568d489640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd49e7a780 0x55fd49e7cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:59.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.516+0000 7f568d489640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd49e7a780 0x55fd49e7cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60770/0 (socket says 172.21.15.120:60770) 2024-09-19T09:10:59.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.516+0000 7f568d489640 1 -- 172.21.15.120:0/4239530467 learned_addr learned my addr 172.21.15.120:0/4239530467 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:59.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.516+0000 7f567dffb640 1 -- 172.21.15.120:0/1953285238 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f56780cf510 con 0x7f56881027a0 2024-09-19T09:10:59.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.516+0000 7f568d489640 1 --2- 172.21.15.120:0/4239530467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd49e6cb40 0x55fd49e6b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:59.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.516+0000 7f567ffff640 1 --2- 172.21.15.120:0/4239530467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd49e6b5d0 0x55fd49e6ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:59.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.516+0000 7f568d489640 1 -- 172.21.15.120:0/4239530467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd49e6b5d0 msgr2=0x55fd49e6ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:59.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.516+0000 7f568d489640 1 --2- 172.21.15.120:0/4239530467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd49e6b5d0 0x55fd49e6ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.516+0000 7f568d489640 1 -- 172.21.15.120:0/4239530467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd49e6cb40 msgr2=0x55fd49e6b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:59.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.516+0000 7f568d489640 1 --2- 172.21.15.120:0/4239530467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd49e6cb40 0x55fd49e6b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.516+0000 7f568d489640 1 -- 172.21.15.120:0/4239530467 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd49d5d3a0 con 0x55fd49e7a780 2024-09-19T09:10:59.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.516+0000 7f568cc88640 1 --2- 172.21.15.120:0/1953285238 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f56700a3240 0x7f56700a5700 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5684009990 tx=0x7f5684009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:59.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.516+0000 7f568d489640 1 --2- 172.21.15.120:0/4239530467 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd49e7a780 0x55fd49e7cb70 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f5680004af0 tx=0x7f568002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3515ee9791ab7b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:59.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.517+0000 7f566effd640 1 -- 172.21.15.120:0/4239530467 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56800314d0 con 0x55fd49e7a780 2024-09-19T09:10:59.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.517+0000 7f566effd640 1 -- 172.21.15.120:0/4239530467 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5680036050 con 0x55fd49e7a780 2024-09-19T09:10:59.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.517+0000 7f566effd640 1 -- 172.21.15.120:0/4239530467 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56800305c0 con 0x55fd49e7a780 2024-09-19T09:10:59.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.517+0000 7f56912c5740 1 -- 172.21.15.120:0/4239530467 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd49e7a780 msgr2=0x55fd49e7cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:59.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.517+0000 7f56912c5740 1 --2- 172.21.15.120:0/4239530467 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd49e7a780 0x55fd49e7cb70 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f5680004af0 tx=0x7f568002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.517+0000 7f56912c5740 1 -- 172.21.15.120:0/4239530467 shutdown_connections 2024-09-19T09:10:59.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.517+0000 7f56912c5740 1 --2- 172.21.15.120:0/4239530467 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd49e7a780 0x55fd49e7cb70 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.517+0000 7f56912c5740 1 --2- 172.21.15.120:0/4239530467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd49e6b5d0 0x55fd49e6ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.517+0000 7f56912c5740 1 --2- 172.21.15.120:0/4239530467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd49e6cb40 0x55fd49e6b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.517+0000 7f56912c5740 1 -- 172.21.15.120:0/4239530467 >> 172.21.15.120:0/4239530467 conn(0x55fd49e34310 msgr2=0x55fd49e67c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:59.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.517+0000 7f56912c5740 1 -- 172.21.15.120:0/4239530467 shutdown_connections 2024-09-19T09:10:59.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.517+0000 7f56912c5740 1 -- 172.21.15.120:0/4239530467 wait complete. 2024-09-19T09:10:59.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.518+0000 7f56912c5740 1 -- start start 2024-09-19T09:10:59.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.518+0000 7f56912c5740 1 Processor -- start 2024-09-19T09:10:59.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.518+0000 7f56912c5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd49e6b5d0 0x55fd49e7a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:59.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.518+0000 7f56912c5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd49e6cb40 0x55fd49e77440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:59.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.518+0000 7f56912c5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd49e7a780 0x55fd49e77a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:59.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.518+0000 7f56912c5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fd49e7d2a0 con 0x55fd49e7a780 2024-09-19T09:10:59.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.518+0000 7f56912c5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fd49d8c060 con 0x55fd49e6cb40 2024-09-19T09:10:59.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.518+0000 7f56912c5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fd49e7d420 con 0x55fd49e6b5d0 2024-09-19T09:10:59.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.518+0000 7f56912c5740 10 client.0 _ll_register_callbacks cb 0x55fd49adf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:59.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.518+0000 7f568d489640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd49e7a780 0x55fd49e77a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:59.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.518+0000 7f568d489640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd49e7a780 0x55fd49e77a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60782/0 (socket says 172.21.15.120:60782) 2024-09-19T09:10:59.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.518+0000 7f568d489640 1 -- 172.21.15.120:0/823580449 learned_addr learned my addr 172.21.15.120:0/823580449 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:59.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.519+0000 7f567ffff640 1 --2- 172.21.15.120:0/823580449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd49e6cb40 0x55fd49e77440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:59.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.519+0000 7f568cc88640 1 --2- 172.21.15.120:0/823580449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd49e6b5d0 0x55fd49e7a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:59.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.519+0000 7f568d489640 1 -- 172.21.15.120:0/823580449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd49e6b5d0 msgr2=0x55fd49e7a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:59.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.519+0000 7f568d489640 1 --2- 172.21.15.120:0/823580449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd49e6b5d0 0x55fd49e7a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.519+0000 7f568d489640 1 -- 172.21.15.120:0/823580449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd49e6cb40 msgr2=0x55fd49e77440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:59.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.519+0000 7f568d489640 1 --2- 172.21.15.120:0/823580449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd49e6cb40 0x55fd49e77440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.519+0000 7f568d489640 1 -- 172.21.15.120:0/823580449 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd49b423c0 con 0x55fd49e7a780 2024-09-19T09:10:59.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.519+0000 7f568cc88640 1 --2- 172.21.15.120:0/823580449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd49e6b5d0 0x55fd49e7a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:10:59.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.519+0000 7f568d489640 1 --2- 172.21.15.120:0/823580449 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd49e7a780 0x55fd49e77a50 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f5680005e30 tx=0x7f56800095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54a852a89b204ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:59.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.519+0000 7f5667fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:59.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.519+0000 7f5667fff640 1 -- 172.21.15.120:0/823580449 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568004e070 con 0x55fd49e7a780 2024-09-19T09:10:59.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.519+0000 7f5667fff640 1 -- 172.21.15.120:0/823580449 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f56800314d0 con 0x55fd49e7a780 2024-09-19T09:10:59.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.519+0000 7f56912c5740 1 -- 172.21.15.120:0/823580449 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5680036050 con 0x55fd49e7a780 2024-09-19T09:10:59.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.519+0000 7f5667fff640 1 -- 172.21.15.120:0/823580449 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f568005a050 con 0x55fd49e7a780 2024-09-19T09:10:59.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.519+0000 7f56912c5740 10 client.18243 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:59.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.519+0000 7f56912c5740 1 -- 172.21.15.120:0/823580449 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fd49e7d2a0 con 0x55fd49e7a780 2024-09-19T09:10:59.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.519+0000 7f5667fff640 1 -- 172.21.15.120:0/823580449 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5680036050 con 0x55fd49e7a780 2024-09-19T09:10:59.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.519+0000 7f5667fff640 1 -- 172.21.15.120:0/823580449 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f568003b400 con 0x55fd49e7a780 2024-09-19T09:10:59.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.519+0000 7f56912c5740 10 client.18243 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:59.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.519+0000 7f56912c5740 10 client.18243 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:10:59.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.519+0000 7f56912c5740 20 client.18243 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:59.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.519+0000 7f56912c5740 1 --2- 172.21.15.120:0/823580449 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55fd49e89ef0 0x55fd49e8c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:59.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.520+0000 7f56912c5740 4 client.18243 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:10:59.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.520+0000 7f56912c5740 1 -- 172.21.15.120:0/823580449 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd49dde860 con 0x55fd49e89ef0 2024-09-19T09:10:59.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.520+0000 7f568cc88640 1 --2- 172.21.15.120:0/823580449 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55fd49e89ef0 0x55fd49e8c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:59.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.520+0000 7f568cc88640 1 --2- 172.21.15.120:0/823580449 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55fd49e89ef0 0x55fd49e8c2e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1299fb812c360b05 server_cookie=7e3bb2a509ae1e12 in_seq=0 out_seq=0 2024-09-19T09:10:59.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.520+0000 7f5667fff640 10 client.18243 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:10:59.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.521+0000 7f5667fff640 1 -- 172.21.15.120:0/823580449 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fd49dde860 con 0x55fd49e89ef0 2024-09-19T09:10:59.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.521+0000 7f5667fff640 10 client.18243 handle_command_reply: tid=0 2024-09-19T09:10:59.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.521+0000 7f56912c5740 1 client.18243 shutdown 2024-09-19T09:10:59.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.521+0000 7f56912c5740 1 -- 172.21.15.120:0/823580449 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd49e7a780 msgr2=0x55fd49e77a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:59.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.521+0000 7f56912c5740 1 --2- 172.21.15.120:0/823580449 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd49e7a780 0x55fd49e77a50 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f5680005e30 tx=0x7f56800095c0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.521+0000 7f56912c5740 1 -- 172.21.15.120:0/823580449 shutdown_connections 2024-09-19T09:10:59.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.521+0000 7f56912c5740 1 --2- 172.21.15.120:0/823580449 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55fd49e89ef0 0x55fd49e8c2e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.521+0000 7f56912c5740 1 --2- 172.21.15.120:0/823580449 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd49e7a780 0x55fd49e77a50 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.521+0000 7f56912c5740 1 --2- 172.21.15.120:0/823580449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd49e6cb40 0x55fd49e77440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.521+0000 7f56912c5740 1 --2- 172.21.15.120:0/823580449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd49e6b5d0 0x55fd49e7a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.521+0000 7f5667fff640 0 client.18243 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:10:59.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.521+0000 7f56912c5740 1 -- 172.21.15.120:0/823580449 >> 172.21.15.120:0/823580449 conn(0x55fd49e34310 msgr2=0x55fd49e6d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:59.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.521+0000 7f56912c5740 1 -- 172.21.15.120:0/823580449 shutdown_connections 2024-09-19T09:10:59.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.521+0000 7f56912c5740 1 -- 172.21.15.120:0/823580449 wait complete. 2024-09-19T09:10:59.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.521+0000 7f56912c5740 20 client.18243 trim_cache size 0 max 16384 2024-09-19T09:10:59.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.534+0000 7f56912c5740 1 Processor -- start 2024-09-19T09:10:59.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.534+0000 7f56912c5740 1 -- start start 2024-09-19T09:10:59.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.534+0000 7f56912c5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd49e6b5d0 0x55fd49e86540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:59.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.534+0000 7f56912c5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd49e6cb40 0x55fd49e91b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:59.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.534+0000 7f56912c5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd49e7a780 0x55fd49e92050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:59.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.535+0000 7f56912c5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f56881ab2f0 con 0x55fd49e6b5d0 2024-09-19T09:10:59.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.535+0000 7f56912c5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f56881ace30 con 0x55fd49e6cb40 2024-09-19T09:10:59.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.535+0000 7f56912c5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56881ab9c0 con 0x55fd49e7a780 2024-09-19T09:10:59.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.535+0000 7f568cc88640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd49e6b5d0 0x55fd49e86540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:59.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.535+0000 7f568cc88640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd49e6b5d0 0x55fd49e86540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60790/0 (socket says 172.21.15.120:60790) 2024-09-19T09:10:59.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.535+0000 7f568cc88640 1 -- 172.21.15.120:0/2070436035 learned_addr learned my addr 172.21.15.120:0/2070436035 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:59.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.535+0000 7f568d489640 1 --2- 172.21.15.120:0/2070436035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd49e7a780 0x55fd49e92050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:59.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.535+0000 7f567ffff640 1 --2- 172.21.15.120:0/2070436035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd49e6cb40 0x55fd49e91b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:59.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.535+0000 7f568cc88640 1 -- 172.21.15.120:0/2070436035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd49e7a780 msgr2=0x55fd49e92050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:59.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.535+0000 7f568cc88640 1 --2- 172.21.15.120:0/2070436035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd49e7a780 0x55fd49e92050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.535+0000 7f568cc88640 1 -- 172.21.15.120:0/2070436035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd49e6cb40 msgr2=0x55fd49e91b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:59.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.535+0000 7f568cc88640 1 --2- 172.21.15.120:0/2070436035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd49e6cb40 0x55fd49e91b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.535+0000 7f568cc88640 1 -- 172.21.15.120:0/2070436035 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd49e7a500 con 0x55fd49e6b5d0 2024-09-19T09:10:59.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.535+0000 7f568cc88640 1 --2- 172.21.15.120:0/2070436035 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd49e6b5d0 0x55fd49e86540 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x55fd49e8ca80 tx=0x7f5684011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3764f980af122ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:59.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.535+0000 7f566e7fc640 1 -- 172.21.15.120:0/2070436035 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684015700 con 0x55fd49e6b5d0 2024-09-19T09:10:59.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.535+0000 7f566e7fc640 1 -- 172.21.15.120:0/2070436035 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f568402e070 con 0x55fd49e6b5d0 2024-09-19T09:10:59.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.535+0000 7f566e7fc640 1 -- 172.21.15.120:0/2070436035 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684021b90 con 0x55fd49e6b5d0 2024-09-19T09:10:59.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.536+0000 7f56912c5740 1 -- 172.21.15.120:0/2070436035 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd49e6b5d0 msgr2=0x55fd49e86540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:59.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.536+0000 7f56912c5740 1 --2- 172.21.15.120:0/2070436035 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd49e6b5d0 0x55fd49e86540 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x55fd49e8ca80 tx=0x7f5684011000 comp rx=0 tx=0).stop 2024-09-19T09:10:59.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.536+0000 7f56912c5740 1 -- 172.21.15.120:0/2070436035 shutdown_connections 2024-09-19T09:10:59.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.536+0000 7f56912c5740 1 --2- 172.21.15.120:0/2070436035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd49e7a780 0x55fd49e92050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.536+0000 7f56912c5740 1 --2- 172.21.15.120:0/2070436035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd49e6cb40 0x55fd49e91b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.536+0000 7f56912c5740 1 --2- 172.21.15.120:0/2070436035 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd49e6b5d0 0x55fd49e86540 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.536+0000 7f56912c5740 1 -- 172.21.15.120:0/2070436035 >> 172.21.15.120:0/2070436035 conn(0x55fd49e34310 msgr2=0x55fd49e794c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:59.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.536+0000 7f56912c5740 1 -- 172.21.15.120:0/2070436035 shutdown_connections 2024-09-19T09:10:59.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.536+0000 7f56912c5740 1 -- 172.21.15.120:0/2070436035 wait complete. 2024-09-19T09:10:59.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.536+0000 7f56912c5740 1 -- start start 2024-09-19T09:10:59.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.537+0000 7f56912c5740 1 Processor -- start 2024-09-19T09:10:59.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.537+0000 7f56912c5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd49e6b5d0 0x55fd49e82cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:59.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.537+0000 7f56912c5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd49e6cb40 0x55fd49e74070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:59.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.537+0000 7f56912c5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd49e7a780 0x55fd49e745b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:59.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.537+0000 7f56912c5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f56881ac760 con 0x55fd49e6cb40 2024-09-19T09:10:59.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.537+0000 7f56912c5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f56881ac090 con 0x55fd49e7a780 2024-09-19T09:10:59.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.537+0000 7f56912c5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56881a92d0 con 0x55fd49e6b5d0 2024-09-19T09:10:59.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.537+0000 7f56912c5740 10 client.0 _ll_register_callbacks cb 0x7f5680031bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:10:59.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.537+0000 7f567ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd49e6cb40 0x55fd49e74070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:59.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.537+0000 7f567ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd49e6cb40 0x55fd49e74070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60794/0 (socket says 172.21.15.120:60794) 2024-09-19T09:10:59.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.537+0000 7f567ffff640 1 -- 172.21.15.120:0/1716207621 learned_addr learned my addr 172.21.15.120:0/1716207621 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:10:59.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.537+0000 7f568cc88640 1 --2- 172.21.15.120:0/1716207621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd49e6b5d0 0x55fd49e82cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:59.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.537+0000 7f567ffff640 1 -- 172.21.15.120:0/1716207621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd49e6b5d0 msgr2=0x55fd49e82cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:10:59.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.537+0000 7f567ffff640 1 --2- 172.21.15.120:0/1716207621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd49e6b5d0 0x55fd49e82cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.537+0000 7f567ffff640 1 -- 172.21.15.120:0/1716207621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd49e7a780 msgr2=0x55fd49e745b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:10:59.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.537+0000 7f567ffff640 1 --2- 172.21.15.120:0/1716207621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd49e7a780 0x55fd49e745b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.537+0000 7f567ffff640 1 -- 172.21.15.120:0/1716207621 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd49e74ca0 con 0x55fd49e6cb40 2024-09-19T09:10:59.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.537+0000 7f568cc88640 1 --2- 172.21.15.120:0/1716207621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd49e6b5d0 0x55fd49e82cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:10:59.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.538+0000 7f567ffff640 1 --2- 172.21.15.120:0/1716207621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd49e6cb40 0x55fd49e74070 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f5678005fd0 tx=0x7f567809c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3522b6fe9faddade server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:10:59.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.538+0000 7f5667fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:10:59.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.538+0000 7f5667fff640 1 -- 172.21.15.120:0/1716207621 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5678020040 con 0x55fd49e6cb40 2024-09-19T09:10:59.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.538+0000 7f5667fff640 1 -- 172.21.15.120:0/1716207621 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f56780981a0 con 0x55fd49e6cb40 2024-09-19T09:10:59.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.538+0000 7f56912c5740 1 -- 172.21.15.120:0/1716207621 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f568402e070 con 0x55fd49e6cb40 2024-09-19T09:10:59.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.538+0000 7f5667fff640 1 -- 172.21.15.120:0/1716207621 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f567809ce60 con 0x55fd49e6cb40 2024-09-19T09:10:59.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.538+0000 7f56912c5740 10 client.18249 fetch_fsmap learned FSMap version 30 2024-09-19T09:10:59.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.538+0000 7f56912c5740 1 -- 172.21.15.120:0/1716207621 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fd49ef49a0 con 0x55fd49e6cb40 2024-09-19T09:10:59.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.538+0000 7f5667fff640 1 -- 172.21.15.120:0/1716207621 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f568402e070 con 0x55fd49e6cb40 2024-09-19T09:10:59.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.538+0000 7f5667fff640 1 -- 172.21.15.120:0/1716207621 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f56780cf0f0 con 0x55fd49e6cb40 2024-09-19T09:10:59.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.538+0000 7f56912c5740 10 client.18249 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:10:59.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.538+0000 7f56912c5740 10 client.18249 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:10:59.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.538+0000 7f56912c5740 20 client.18249 populate_metadata read hostname 'smithi120' 2024-09-19T09:10:59.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.538+0000 7f56912c5740 1 --2- 172.21.15.120:0/1716207621 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55fd49ef51a0 0x55fd49ef75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:10:59.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.538+0000 7f56912c5740 4 client.18249 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:10:59.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.538+0000 7f56912c5740 1 -- 172.21.15.120:0/1716207621 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd49dd8d70 con 0x55fd49ef51a0 2024-09-19T09:10:59.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.540+0000 7f568cc88640 1 --2- 172.21.15.120:0/1716207621 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55fd49ef51a0 0x55fd49ef75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:10:59.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.542+0000 7f5667fff640 10 client.18249 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:10:59.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.542+0000 7f568cc88640 1 --2- 172.21.15.120:0/1716207621 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55fd49ef51a0 0x55fd49ef75c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9b32ee37eca43dd server_cookie=321d3a34f13ca906 in_seq=0 out_seq=0 2024-09-19T09:10:59.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.543+0000 7f5667fff640 1 -- 172.21.15.120:0/1716207621 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd49dd8d70 con 0x55fd49ef51a0 2024-09-19T09:10:59.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.543+0000 7f5667fff640 10 client.18249 handle_command_reply: tid=0 2024-09-19T09:10:59.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.543+0000 7f56912c5740 1 client.18249 shutdown 2024-09-19T09:10:59.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.544+0000 7f56912c5740 1 -- 172.21.15.120:0/1716207621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd49e6cb40 msgr2=0x55fd49e74070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:59.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.544+0000 7f56912c5740 1 --2- 172.21.15.120:0/1716207621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd49e6cb40 0x55fd49e74070 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f5678005fd0 tx=0x7f567809c230 comp rx=0 tx=0).stop 2024-09-19T09:10:59.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.544+0000 7f56912c5740 1 -- 172.21.15.120:0/1716207621 shutdown_connections 2024-09-19T09:10:59.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.544+0000 7f56912c5740 1 --2- 172.21.15.120:0/1716207621 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55fd49ef51a0 0x55fd49ef75c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.544+0000 7f56912c5740 1 --2- 172.21.15.120:0/1716207621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd49e7a780 0x55fd49e745b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.544+0000 7f56912c5740 1 --2- 172.21.15.120:0/1716207621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd49e6cb40 0x55fd49e74070 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.544+0000 7f56912c5740 1 --2- 172.21.15.120:0/1716207621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd49e6b5d0 0x55fd49e82cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.544+0000 7f5667fff640 0 client.18249 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:10:59.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.544+0000 7f56912c5740 1 -- 172.21.15.120:0/1716207621 >> 172.21.15.120:0/1716207621 conn(0x55fd49e34310 msgr2=0x55fd49e6d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:59.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.544+0000 7f56912c5740 1 -- 172.21.15.120:0/1716207621 shutdown_connections 2024-09-19T09:10:59.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.544+0000 7f56912c5740 1 -- 172.21.15.120:0/1716207621 wait complete. 2024-09-19T09:10:59.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.544+0000 7f56912c5740 20 client.18249 trim_cache size 0 max 16384 2024-09-19T09:10:59.552 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:10:59.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.544+0000 7f568eeec640 1 -- 172.21.15.120:0/1953285238 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f56700a3240 msgr2=0x7f56700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:59.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.544+0000 7f568eeec640 1 --2- 172.21.15.120:0/1953285238 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f56700a3240 0x7f56700a5700 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5684009990 tx=0x7f5684009110 comp rx=0 tx=0).stop 2024-09-19T09:10:59.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.544+0000 7f568eeec640 1 -- 172.21.15.120:0/1953285238 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f56881027a0 msgr2=0x7f5688198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:10:59.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.544+0000 7f568eeec640 1 --2- 172.21.15.120:0/1953285238 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f56881027a0 0x7f5688198a40 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f567800d950 tx=0x7f567800dd50 comp rx=0 tx=0).stop 2024-09-19T09:10:59.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.544+0000 7f568eeec640 1 -- 172.21.15.120:0/1953285238 shutdown_connections 2024-09-19T09:10:59.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.544+0000 7f568eeec640 1 --2- 172.21.15.120:0/1953285238 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f56700a3240 0x7f56700a5700 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.544+0000 7f568eeec640 1 --2- 172.21.15.120:0/1953285238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5688106f20 0x7f568819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.544+0000 7f568eeec640 1 --2- 172.21.15.120:0/1953285238 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f56881027a0 0x7f5688198a40 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.544+0000 7f568eeec640 1 --2- 172.21.15.120:0/1953285238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5688101df0 0x7f5688198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:10:59.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.544+0000 7f568eeec640 1 -- 172.21.15.120:0/1953285238 >> 172.21.15.120:0/1953285238 conn(0x7f56880fd8f0 msgr2=0x7f56880fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:10:59.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.545+0000 7f568eeec640 1 -- 172.21.15.120:0/1953285238 shutdown_connections 2024-09-19T09:10:59.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:10:59.545+0000 7f568eeec640 1 -- 172.21.15.120:0/1953285238 wait complete. 2024-09-19T09:10:59.649 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:11:00.060 INFO:tasks.workunit.client.0.smithi120.stdout: 1 707664 8.83 MB/sec execute 295 sec latency 23421.031 ms 2024-09-19T09:11:00.390 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:11:00.531 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:11:01.060 INFO:tasks.workunit.client.0.smithi120.stdout: 1 707664 8.80 MB/sec execute 296 sec latency 24421.079 ms 2024-09-19T09:11:01.067 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:00 smithi120 ceph-mon[31150]: pgmap v586: 97 pgs: 97 active+clean; 26 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 KiB/s wr, 39 op/s 2024-09-19T09:11:01.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.146+0000 7f8217d00640 1 -- 172.21.15.120:0/92134031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8210102790 msgr2=0x7f82101067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:01.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.146+0000 7f8217d00640 1 --2- 172.21.15.120:0/92134031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8210102790 0x7f82101067e0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f8200009900 tx=0x7f820002a410 comp rx=0 tx=0).stop 2024-09-19T09:11:01.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.146+0000 7f8217d00640 1 -- 172.21.15.120:0/92134031 shutdown_connections 2024-09-19T09:11:01.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.146+0000 7f8217d00640 1 --2- 172.21.15.120:0/92134031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8210106f10 0x7f821010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:01.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.146+0000 7f8217d00640 1 --2- 172.21.15.120:0/92134031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8210102790 0x7f82101067e0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:01.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.146+0000 7f8217d00640 1 --2- 172.21.15.120:0/92134031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8210101de0 0x7f82101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:01.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.146+0000 7f8217d00640 1 -- 172.21.15.120:0/92134031 >> 172.21.15.120:0/92134031 conn(0x7f82100fd8c0 msgr2=0x7f82100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:01.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.146+0000 7f8217d00640 1 -- 172.21.15.120:0/92134031 shutdown_connections 2024-09-19T09:11:01.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.146+0000 7f8217d00640 1 -- 172.21.15.120:0/92134031 wait complete. 2024-09-19T09:11:01.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.146+0000 7f8217d00640 1 Processor -- start 2024-09-19T09:11:01.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.147+0000 7f8217d00640 1 -- start start 2024-09-19T09:11:01.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.147+0000 7f8217d00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8210101de0 0x7f821019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:01.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.147+0000 7f8217d00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8210102790 0x7f821019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:01.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.147+0000 7f8217d00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8210106f10 0x7f821019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:01.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.147+0000 7f8217d00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8210111f70 con 0x7f8210101de0 2024-09-19T09:11:01.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.147+0000 7f8217d00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8210111df0 con 0x7f8210106f10 2024-09-19T09:11:01.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.147+0000 7f8217d00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f82101120f0 con 0x7f8210102790 2024-09-19T09:11:01.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.147+0000 7f8215a9c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8210101de0 0x7f821019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:01.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.147+0000 7f8215a9c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8210101de0 0x7f821019a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60812/0 (socket says 172.21.15.120:60812) 2024-09-19T09:11:01.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.147+0000 7f8215a9c640 1 -- 172.21.15.120:0/3436849309 learned_addr learned my addr 172.21.15.120:0/3436849309 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:01.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.147+0000 7f821529b640 1 --2- 172.21.15.120:0/3436849309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8210102790 0x7f821019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:01.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.147+0000 7f821629d640 1 --2- 172.21.15.120:0/3436849309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8210106f10 0x7f821019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:01.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.147+0000 7f8215a9c640 1 -- 172.21.15.120:0/3436849309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8210102790 msgr2=0x7f821019ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:01.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.147+0000 7f8215a9c640 1 --2- 172.21.15.120:0/3436849309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8210102790 0x7f821019ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:01.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.147+0000 7f8215a9c640 1 -- 172.21.15.120:0/3436849309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8210106f10 msgr2=0x7f821019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:01.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.147+0000 7f8215a9c640 1 --2- 172.21.15.120:0/3436849309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8210106f10 0x7f821019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:01.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.147+0000 7f8215a9c640 1 -- 172.21.15.120:0/3436849309 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f821019f5c0 con 0x7f8210101de0 2024-09-19T09:11:01.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.147+0000 7f8215a9c640 1 --2- 172.21.15.120:0/3436849309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8210101de0 0x7f821019a610 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f820c00e9f0 tx=0x7f820c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:01.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.147+0000 7f8206ffd640 1 -- 172.21.15.120:0/3436849309 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c00cd80 con 0x7f8210101de0 2024-09-19T09:11:01.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.147+0000 7f8206ffd640 1 -- 172.21.15.120:0/3436849309 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f820c010910 con 0x7f8210101de0 2024-09-19T09:11:01.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.147+0000 7f8217d00640 1 -- 172.21.15.120:0/3436849309 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f821019f8b0 con 0x7f8210101de0 2024-09-19T09:11:01.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.147+0000 7f8206ffd640 1 -- 172.21.15.120:0/3436849309 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c018990 con 0x7f8210101de0 2024-09-19T09:11:01.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.148+0000 7f8217d00640 1 -- 172.21.15.120:0/3436849309 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82101a7190 con 0x7f8210101de0 2024-09-19T09:11:01.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.149+0000 7f8217d00640 1 -- 172.21.15.120:0/3436849309 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82101021c0 con 0x7f8210101de0 2024-09-19T09:11:01.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.150+0000 7f8206ffd640 1 -- 172.21.15.120:0/3436849309 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f820c010ac0 con 0x7f8210101de0 2024-09-19T09:11:01.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.150+0000 7f8206ffd640 1 --2- 172.21.15.120:0/3436849309 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f81fc0a3240 0x7f81fc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:01.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.150+0000 7f821529b640 1 --2- 172.21.15.120:0/3436849309 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f81fc0a3240 0x7f81fc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:01.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.151+0000 7f8206ffd640 1 -- 172.21.15.120:0/3436849309 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f820c0d01c0 con 0x7f8210101de0 2024-09-19T09:11:01.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.151+0000 7f821529b640 1 --2- 172.21.15.120:0/3436849309 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f81fc0a3240 0x7f81fc0a5700 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f82100ff740 tx=0x7f8200005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:01.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.154+0000 7f8206ffd640 1 -- 172.21.15.120:0/3436849309 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f820c09cee0 con 0x7f8210101de0 2024-09-19T09:11:01.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:00 smithi153 ceph-mon[35071]: pgmap v586: 97 pgs: 97 active+clean; 26 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 KiB/s wr, 39 op/s 2024-09-19T09:11:01.188 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:00 smithi196 ceph-mon[35150]: pgmap v586: 97 pgs: 97 active+clean; 26 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 KiB/s wr, 39 op/s 2024-09-19T09:11:01.306 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:11:01.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.366+0000 7f8217d00640 1 -- 172.21.15.120:0/3436849309 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82101a7430 con 0x7f8210101de0 2024-09-19T09:11:01.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.367+0000 7f8206ffd640 1 -- 172.21.15.120:0/3436849309 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11985 (secure 0 0 0) 0x7f820c09c7e0 con 0x7f8210101de0 2024-09-19T09:11:01.368 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:11:01.369 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":30,"btime":"2024-09-19T09:07:14:812065+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:07:14.812046+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26351":{"gid":26351,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/364941779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":364941779},{"type":"v1","addr":"172.21.15.153:6835","nonce":364941779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:11:01.369 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 30 2024-09-19T09:11:01.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.369+0000 7f8217d00640 1 -- 172.21.15.120:0/3436849309 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f81fc0a3240 msgr2=0x7f81fc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:01.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.369+0000 7f8217d00640 1 --2- 172.21.15.120:0/3436849309 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f81fc0a3240 0x7f81fc0a5700 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f82100ff740 tx=0x7f8200005a20 comp rx=0 tx=0).stop 2024-09-19T09:11:01.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.369+0000 7f8217d00640 1 -- 172.21.15.120:0/3436849309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8210101de0 msgr2=0x7f821019a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:01.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.369+0000 7f8217d00640 1 --2- 172.21.15.120:0/3436849309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8210101de0 0x7f821019a610 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f820c00e9f0 tx=0x7f820c00eeb0 comp rx=0 tx=0).stop 2024-09-19T09:11:01.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.369+0000 7f8217d00640 1 -- 172.21.15.120:0/3436849309 shutdown_connections 2024-09-19T09:11:01.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.369+0000 7f8217d00640 1 --2- 172.21.15.120:0/3436849309 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f81fc0a3240 0x7f81fc0a5700 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:01.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.369+0000 7f8217d00640 1 --2- 172.21.15.120:0/3436849309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8210106f10 0x7f821019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:01.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.369+0000 7f8217d00640 1 --2- 172.21.15.120:0/3436849309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8210102790 0x7f821019ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:01.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.369+0000 7f8217d00640 1 --2- 172.21.15.120:0/3436849309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8210101de0 0x7f821019a610 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:01.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.369+0000 7f8217d00640 1 -- 172.21.15.120:0/3436849309 >> 172.21.15.120:0/3436849309 conn(0x7f82100fd8c0 msgr2=0x7f82100feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:01.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.369+0000 7f8217d00640 1 -- 172.21.15.120:0/3436849309 shutdown_connections 2024-09-19T09:11:01.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.369+0000 7f8217d00640 1 -- 172.21.15.120:0/3436849309 wait complete. 2024-09-19T09:11:02.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.995+0000 7f5dde4ad640 1 -- 172.21.15.120:0/1922835362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5dd8106a80 msgr2=0x7f5dd8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:02.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.995+0000 7f5dde4ad640 1 --2- 172.21.15.120:0/1922835362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5dd8106a80 0x7f5dd8106e60 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8009930 tx=0x7f5dc802a420 comp rx=0 tx=0).stop 2024-09-19T09:11:02.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.995+0000 7f5dde4ad640 1 -- 172.21.15.120:0/1922835362 shutdown_connections 2024-09-19T09:11:02.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.995+0000 7f5dde4ad640 1 --2- 172.21.15.120:0/1922835362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5dd8101420 0x7f5dd810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.995+0000 7f5dde4ad640 1 --2- 172.21.15.120:0/1922835362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dd8100a80 0x7f5dd8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.995+0000 7f5dde4ad640 1 --2- 172.21.15.120:0/1922835362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5dd8106a80 0x7f5dd8106e60 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.995+0000 7f5dde4ad640 1 -- 172.21.15.120:0/1922835362 >> 172.21.15.120:0/1922835362 conn(0x7f5dd80fc7a0 msgr2=0x7f5dd80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:02.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.995+0000 7f5dde4ad640 1 -- 172.21.15.120:0/1922835362 shutdown_connections 2024-09-19T09:11:02.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.995+0000 7f5dde4ad640 1 -- 172.21.15.120:0/1922835362 wait complete. 2024-09-19T09:11:02.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.995+0000 7f5dde4ad640 1 Processor -- start 2024-09-19T09:11:02.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.995+0000 7f5dde4ad640 1 -- start start 2024-09-19T09:11:02.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.996+0000 7f5dde4ad640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dd8100a80 0x7f5dd8199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:02.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.996+0000 7f5dde4ad640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5dd8101420 0x7f5dd8199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:02.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.996+0000 7f5dde4ad640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5dd8106a80 0x7f5dd81a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:02.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.996+0000 7f5dde4ad640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd8110e40 con 0x7f5dd8101420 2024-09-19T09:11:02.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.996+0000 7f5dde4ad640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd8110cc0 con 0x7f5dd8106a80 2024-09-19T09:11:02.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.996+0000 7f5dde4ad640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd8110fc0 con 0x7f5dd8100a80 2024-09-19T09:11:02.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.996+0000 7f5dcffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5dd8101420 0x7f5dd8199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:02.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.996+0000 7f5dcffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5dd8101420 0x7f5dd8199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52108/0 (socket says 172.21.15.120:52108) 2024-09-19T09:11:02.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.996+0000 7f5dcffff640 1 -- 172.21.15.120:0/3599096971 learned_addr learned my addr 172.21.15.120:0/3599096971 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:02.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.996+0000 7f5ddca4a640 1 --2- 172.21.15.120:0/3599096971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5dd8106a80 0x7f5dd81a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:02.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.996+0000 7f5dd7fff640 1 --2- 172.21.15.120:0/3599096971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dd8100a80 0x7f5dd8199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:02.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.996+0000 7f5dcffff640 1 -- 172.21.15.120:0/3599096971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dd8100a80 msgr2=0x7f5dd8199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:02.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.996+0000 7f5dcffff640 1 --2- 172.21.15.120:0/3599096971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dd8100a80 0x7f5dd8199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.996+0000 7f5dcffff640 1 -- 172.21.15.120:0/3599096971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5dd8106a80 msgr2=0x7f5dd81a42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:02.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.996+0000 7f5dcffff640 1 --2- 172.21.15.120:0/3599096971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5dd8106a80 0x7f5dd81a42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.996+0000 7f5dcffff640 1 -- 172.21.15.120:0/3599096971 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd81a4810 con 0x7f5dd8101420 2024-09-19T09:11:02.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.996+0000 7f5dcffff640 1 --2- 172.21.15.120:0/3599096971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5dd8101420 0x7f5dd8199790 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000c920 tx=0x7f5dc000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:02.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.996+0000 7f5dd5ffb640 1 -- 172.21.15.120:0/3599096971 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0007c70 con 0x7f5dd8101420 2024-09-19T09:11:02.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.996+0000 7f5dde4ad640 1 -- 172.21.15.120:0/3599096971 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dd81a49e0 con 0x7f5dd8101420 2024-09-19T09:11:02.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.996+0000 7f5dd5ffb640 1 -- 172.21.15.120:0/3599096971 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dc0007e10 con 0x7f5dd8101420 2024-09-19T09:11:02.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.996+0000 7f5dd5ffb640 1 -- 172.21.15.120:0/3599096971 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0018890 con 0x7f5dd8101420 2024-09-19T09:11:02.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.996+0000 7f5dde4ad640 1 -- 172.21.15.120:0/3599096971 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dd81a4ef0 con 0x7f5dd8101420 2024-09-19T09:11:02.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.999+0000 7f5dd5ffb640 1 -- 172.21.15.120:0/3599096971 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5dc0018a70 con 0x7f5dd8101420 2024-09-19T09:11:02.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.999+0000 7f5de0886740 1 Processor -- start 2024-09-19T09:11:02.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.999+0000 7f5de0886740 1 -- start start 2024-09-19T09:11:02.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.999+0000 7f5de0886740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5556c0b40 0x55a5556bf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:02.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.999+0000 7f5de0886740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5556bf5d0 0x55a5556bfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:02.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.999+0000 7f5de0886740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5556ca370 0x55a5556cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:02.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.999+0000 7f5de0886740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a5556252f0 con 0x55a5556ca370 2024-09-19T09:11:02.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.999+0000 7f5de0886740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a5555cccf0 con 0x55a5556bf5d0 2024-09-19T09:11:02.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.999+0000 7f5de0886740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a5556212b0 con 0x55a5556c0b40 2024-09-19T09:11:02.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.999+0000 7f5dcffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5556ca370 0x55a5556cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:02.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.999+0000 7f5dd7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5556bf5d0 0x55a5556bfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:02.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.999+0000 7f5dcffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5556ca370 0x55a5556cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52118/0 (socket says 172.21.15.120:52118) 2024-09-19T09:11:02.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:01.999+0000 7f5dcffff640 1 -- 172.21.15.120:0/2918678594 learned_addr learned my addr 172.21.15.120:0/2918678594 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:02.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.000+0000 7f5ddca4a640 1 --2- 172.21.15.120:0/2918678594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5556c0b40 0x55a5556bf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:02.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.000+0000 7f5dd7fff640 1 -- 172.21.15.120:0/2918678594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5556c0b40 msgr2=0x55a5556bf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:02.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.000+0000 7f5dd7fff640 1 --2- 172.21.15.120:0/2918678594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5556c0b40 0x55a5556bf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.000+0000 7f5dd5ffb640 1 --2- 172.21.15.120:0/3599096971 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5dc40a3240 0x7f5dc40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:02.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.000+0000 7f5dd7fff640 1 -- 172.21.15.120:0/2918678594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5556ca370 msgr2=0x55a5556cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:02.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.000+0000 7f5dd7fff640 1 --2- 172.21.15.120:0/2918678594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5556ca370 0x55a5556cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.000+0000 7f5dd7fff640 1 -- 172.21.15.120:0/2918678594 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5555b13a0 con 0x55a5556bf5d0 2024-09-19T09:11:02.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.000+0000 7f5dcffff640 1 --2- 172.21.15.120:0/2918678594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5556ca370 0x55a5556cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:02.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.000+0000 7f5ddca4a640 1 --2- 172.21.15.120:0/3599096971 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5dc40a3240 0x7f5dc40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:02.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.000+0000 7f5dd5ffb640 1 -- 172.21.15.120:0/3599096971 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f5dc00cfa60 con 0x7f5dd8101420 2024-09-19T09:11:02.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.000+0000 7f5dd7fff640 1 --2- 172.21.15.120:0/2918678594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5556bf5d0 0x55a5556bfa10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8009900 tx=0x7f5dc8032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe11113aacc402cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:02.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.000+0000 7f5dce7fc640 1 -- 172.21.15.120:0/2918678594 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc803f070 con 0x55a5556bf5d0 2024-09-19T09:11:02.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.000+0000 7f5dce7fc640 1 -- 172.21.15.120:0/2918678594 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dc80317b0 con 0x55a5556bf5d0 2024-09-19T09:11:02.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.000+0000 7f5dce7fc640 1 -- 172.21.15.120:0/2918678594 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8035070 con 0x55a5556bf5d0 2024-09-19T09:11:02.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.001+0000 7f5de0886740 1 -- 172.21.15.120:0/2918678594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5556bf5d0 msgr2=0x55a5556bfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:02.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.001+0000 7f5ddca4a640 1 --2- 172.21.15.120:0/3599096971 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5dc40a3240 0x7f5dc40a5700 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0005fd0 tx=0x7f5dd0009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:02.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.001+0000 7f5de0886740 1 --2- 172.21.15.120:0/2918678594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5556bf5d0 0x55a5556bfa10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8009900 tx=0x7f5dc8032ce0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.001+0000 7f5de0886740 1 -- 172.21.15.120:0/2918678594 shutdown_connections 2024-09-19T09:11:02.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.001+0000 7f5de0886740 1 --2- 172.21.15.120:0/2918678594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5556ca370 0x55a5556cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.001+0000 7f5de0886740 1 --2- 172.21.15.120:0/2918678594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5556bf5d0 0x55a5556bfa10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.001+0000 7f5de0886740 1 --2- 172.21.15.120:0/2918678594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5556c0b40 0x55a5556bf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.001+0000 7f5de0886740 1 -- 172.21.15.120:0/2918678594 >> 172.21.15.120:0/2918678594 conn(0x55a555688310 msgr2=0x55a5556bbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:02.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.001+0000 7f5de0886740 1 -- 172.21.15.120:0/2918678594 shutdown_connections 2024-09-19T09:11:02.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.001+0000 7f5de0886740 1 -- 172.21.15.120:0/2918678594 wait complete. 2024-09-19T09:11:02.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.001+0000 7f5de0886740 1 -- start start 2024-09-19T09:11:02.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.002+0000 7f5de0886740 1 Processor -- start 2024-09-19T09:11:02.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.002+0000 7f5de0886740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5556bf5d0 0x55a5556c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:02.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.002+0000 7f5de0886740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5556c0b40 0x55a5556c7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:02.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.002+0000 7f5de0886740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5556ca370 0x55a5556c7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:02.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.002+0000 7f5de0886740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a5556cd010 con 0x55a5556bf5d0 2024-09-19T09:11:02.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.002+0000 7f5de0886740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a5555e0060 con 0x55a5556c0b40 2024-09-19T09:11:02.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.002+0000 7f5de0886740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a5556cd190 con 0x55a5556ca370 2024-09-19T09:11:02.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.002+0000 7f5de0886740 10 client.0 _ll_register_callbacks cb 0x55a5553338c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:02.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.002+0000 7f5dd7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5556bf5d0 0x55a5556c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:02.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.002+0000 7f5dd7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5556bf5d0 0x55a5556c9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52128/0 (socket says 172.21.15.120:52128) 2024-09-19T09:11:02.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.002+0000 7f5dd7fff640 1 -- 172.21.15.120:0/1481857348 learned_addr learned my addr 172.21.15.120:0/1481857348 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:02.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.002+0000 7f5ddca4a640 1 --2- 172.21.15.120:0/1481857348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5556ca370 0x55a5556c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:02.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.002+0000 7f5dcffff640 1 --2- 172.21.15.120:0/1481857348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5556c0b40 0x55a5556c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:02.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.002+0000 7f5dd7fff640 1 -- 172.21.15.120:0/1481857348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5556ca370 msgr2=0x55a5556c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:02.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.002+0000 7f5dd7fff640 1 --2- 172.21.15.120:0/1481857348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5556ca370 0x55a5556c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.002+0000 7f5dd7fff640 1 -- 172.21.15.120:0/1481857348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5556c0b40 msgr2=0x55a5556c7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:02.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.002+0000 7f5dd7fff640 1 --2- 172.21.15.120:0/1481857348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5556c0b40 0x55a5556c7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.002+0000 7f5dd7fff640 1 -- 172.21.15.120:0/1481857348 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5553963c0 con 0x55a5556bf5d0 2024-09-19T09:11:02.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.002+0000 7f5ddca4a640 1 --2- 172.21.15.120:0/1481857348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5556ca370 0x55a5556c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:11:02.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.002+0000 7f5dcffff640 1 --2- 172.21.15.120:0/1481857348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5556c0b40 0x55a5556c7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:11:02.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.003+0000 7f5dd7fff640 1 --2- 172.21.15.120:0/1481857348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5556bf5d0 0x55a5556c9d60 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f5dc80096c0 tx=0x7f5dc80315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32abe3229b25c04 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:02.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.003+0000 7f5db77fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:11:02.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.003+0000 7f5db77fe640 1 -- 172.21.15.120:0/1481857348 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8035070 con 0x55a5556bf5d0 2024-09-19T09:11:02.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.003+0000 7f5db77fe640 1 -- 172.21.15.120:0/1481857348 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f5dc80035f0 con 0x55a5556bf5d0 2024-09-19T09:11:02.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.003+0000 7f5de0886740 1 -- 172.21.15.120:0/1481857348 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dc80317b0 con 0x55a5556bf5d0 2024-09-19T09:11:02.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.003+0000 7f5db77fe640 1 -- 172.21.15.120:0/1481857348 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5dc803f040 con 0x55a5556bf5d0 2024-09-19T09:11:02.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.003+0000 7f5de0886740 10 client.18267 fetch_fsmap learned FSMap version 30 2024-09-19T09:11:02.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.003+0000 7f5de0886740 1 -- 172.21.15.120:0/1481857348 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a5556d8ab0 con 0x55a5556bf5d0 2024-09-19T09:11:02.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.003+0000 7f5db77fe640 1 -- 172.21.15.120:0/1481857348 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dc80317b0 con 0x55a5556bf5d0 2024-09-19T09:11:02.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.003+0000 7f5db77fe640 1 -- 172.21.15.120:0/1481857348 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5dc8004bf0 con 0x55a5556bf5d0 2024-09-19T09:11:02.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.003+0000 7f5de0886740 10 client.18267 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:11:02.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.003+0000 7f5de0886740 10 client.18267 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:11:02.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.003+0000 7f5de0886740 20 client.18267 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:02.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.003+0000 7f5de0886740 1 --2- 172.21.15.120:0/1481857348 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55a5556d9e00 0x55a5556dc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:02.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.003+0000 7f5de0886740 4 client.18267 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:11:02.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.003+0000 7f5de0886740 1 -- 172.21.15.120:0/1481857348 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a555632860 con 0x55a5556d9e00 2024-09-19T09:11:02.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.004+0000 7f5dcffff640 1 --2- 172.21.15.120:0/1481857348 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55a5556d9e00 0x55a5556dc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:02.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.004+0000 7f5dcffff640 1 --2- 172.21.15.120:0/1481857348 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55a5556d9e00 0x55a5556dc1f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2af019e546a85a89 server_cookie=e384435ff2b95543 in_seq=0 out_seq=0 2024-09-19T09:11:02.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.004+0000 7f5db77fe640 10 client.18267 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:11:02.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.005+0000 7f5db77fe640 1 -- 172.21.15.120:0/1481857348 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a555632860 con 0x55a5556d9e00 2024-09-19T09:11:02.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.005+0000 7f5db77fe640 10 client.18267 handle_command_reply: tid=0 2024-09-19T09:11:02.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.005+0000 7f5de0886740 1 client.18267 shutdown 2024-09-19T09:11:02.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.005+0000 7f5de0886740 1 -- 172.21.15.120:0/1481857348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5556bf5d0 msgr2=0x55a5556c9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:02.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.005+0000 7f5de0886740 1 --2- 172.21.15.120:0/1481857348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5556bf5d0 0x55a5556c9d60 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f5dc80096c0 tx=0x7f5dc80315b0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.005+0000 7f5de0886740 1 -- 172.21.15.120:0/1481857348 shutdown_connections 2024-09-19T09:11:02.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.005+0000 7f5de0886740 1 --2- 172.21.15.120:0/1481857348 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55a5556d9e00 0x55a5556dc1f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.005+0000 7f5de0886740 1 --2- 172.21.15.120:0/1481857348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5556ca370 0x55a5556c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.005+0000 7f5de0886740 1 --2- 172.21.15.120:0/1481857348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5556c0b40 0x55a5556c7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.005+0000 7f5db77fe640 0 client.18267 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:11:02.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.005+0000 7f5de0886740 1 --2- 172.21.15.120:0/1481857348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5556bf5d0 0x55a5556c9d60 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.005+0000 7f5de0886740 1 -- 172.21.15.120:0/1481857348 >> 172.21.15.120:0/1481857348 conn(0x55a555688310 msgr2=0x55a5556c1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:02.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.005+0000 7f5de0886740 1 -- 172.21.15.120:0/1481857348 shutdown_connections 2024-09-19T09:11:02.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.005+0000 7f5de0886740 1 -- 172.21.15.120:0/1481857348 wait complete. 2024-09-19T09:11:02.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.005+0000 7f5de0886740 20 client.18267 trim_cache size 0 max 16384 2024-09-19T09:11:02.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.018+0000 7f5de0886740 1 Processor -- start 2024-09-19T09:11:02.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.018+0000 7f5de0886740 1 -- start start 2024-09-19T09:11:02.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.018+0000 7f5de0886740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5556bf5d0 0x55a5556c0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:02.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.018+0000 7f5de0886740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5556c0b40 0x55a5556d62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:02.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.018+0000 7f5de0886740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5556ca370 0x55a5556cb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:02.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.018+0000 7f5de0886740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd81ad500 con 0x55a5556bf5d0 2024-09-19T09:11:02.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.018+0000 7f5de0886740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd81af040 con 0x55a5556ca370 2024-09-19T09:11:02.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.018+0000 7f5de0886740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd81adbd0 con 0x55a5556c0b40 2024-09-19T09:11:02.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.018+0000 7f5dd7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5556bf5d0 0x55a5556c0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:02.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.018+0000 7f5dd7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5556bf5d0 0x55a5556c0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52134/0 (socket says 172.21.15.120:52134) 2024-09-19T09:11:02.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.018+0000 7f5dd7fff640 1 -- 172.21.15.120:0/3421757984 learned_addr learned my addr 172.21.15.120:0/3421757984 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:02.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.018+0000 7f5dcffff640 1 --2- 172.21.15.120:0/3421757984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5556c0b40 0x55a5556d62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:02.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.018+0000 7f5ddca4a640 1 --2- 172.21.15.120:0/3421757984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5556ca370 0x55a5556cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:02.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.018+0000 7f5dd7fff640 1 -- 172.21.15.120:0/3421757984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5556c0b40 msgr2=0x55a5556d62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:02.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.018+0000 7f5dd7fff640 1 --2- 172.21.15.120:0/3421757984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5556c0b40 0x55a5556d62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.018+0000 7f5dd7fff640 1 -- 172.21.15.120:0/3421757984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5556ca370 msgr2=0x55a5556cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:02.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.018+0000 7f5dd7fff640 1 --2- 172.21.15.120:0/3421757984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5556ca370 0x55a5556cb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.018+0000 7f5dd7fff640 1 -- 172.21.15.120:0/3421757984 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5556ca090 con 0x55a5556bf5d0 2024-09-19T09:11:02.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.018+0000 7f5dcffff640 1 --2- 172.21.15.120:0/3421757984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5556c0b40 0x55a5556d62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:11:02.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.018+0000 7f5ddca4a640 1 --2- 172.21.15.120:0/3421757984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5556ca370 0x55a5556cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:11:02.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.019+0000 7f5dd7fff640 1 --2- 172.21.15.120:0/3421757984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5556bf5d0 0x55a5556c0410 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8006010 tx=0x7f5dc8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d203cbd0aa4c48 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:02.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.019+0000 7f5dcdffb640 1 -- 172.21.15.120:0/3421757984 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8035040 con 0x55a5556bf5d0 2024-09-19T09:11:02.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.019+0000 7f5dcdffb640 1 -- 172.21.15.120:0/3421757984 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dc804c070 con 0x55a5556bf5d0 2024-09-19T09:11:02.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.019+0000 7f5dcdffb640 1 -- 172.21.15.120:0/3421757984 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8031dd0 con 0x55a5556bf5d0 2024-09-19T09:11:02.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.019+0000 7f5de0886740 1 -- 172.21.15.120:0/3421757984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5556bf5d0 msgr2=0x55a5556c0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:02.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.019+0000 7f5de0886740 1 --2- 172.21.15.120:0/3421757984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5556bf5d0 0x55a5556c0410 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8006010 tx=0x7f5dc8000bc0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.019+0000 7f5de0886740 1 -- 172.21.15.120:0/3421757984 shutdown_connections 2024-09-19T09:11:02.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.019+0000 7f5de0886740 1 --2- 172.21.15.120:0/3421757984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5556ca370 0x55a5556cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.019+0000 7f5de0886740 1 --2- 172.21.15.120:0/3421757984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5556c0b40 0x55a5556d62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.019+0000 7f5de0886740 1 --2- 172.21.15.120:0/3421757984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5556bf5d0 0x55a5556c0410 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.020+0000 7f5de0886740 1 -- 172.21.15.120:0/3421757984 >> 172.21.15.120:0/3421757984 conn(0x55a555688310 msgr2=0x55a5556c22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:02.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.020+0000 7f5de0886740 1 -- 172.21.15.120:0/3421757984 shutdown_connections 2024-09-19T09:11:02.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.020+0000 7f5de0886740 1 -- 172.21.15.120:0/3421757984 wait complete. 2024-09-19T09:11:02.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.020+0000 7f5de0886740 1 -- start start 2024-09-19T09:11:02.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.020+0000 7f5de0886740 1 Processor -- start 2024-09-19T09:11:02.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.020+0000 7f5de0886740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5556bf5d0 0x55a5556d2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:02.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.020+0000 7f5de0886740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5556c0b40 0x55a5556cb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:02.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.020+0000 7f5de0886740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5556ca370 0x55a5556cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:02.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.020+0000 7f5de0886740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd81ae970 con 0x55a5556c0b40 2024-09-19T09:11:02.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.020+0000 7f5de0886740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd81ae2a0 con 0x55a5556ca370 2024-09-19T09:11:02.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.020+0000 7f5de0886740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd81ab4c0 con 0x55a5556bf5d0 2024-09-19T09:11:02.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.020+0000 7f5dcffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5556c0b40 0x55a5556cb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:02.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.020+0000 7f5dcffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5556c0b40 0x55a5556cb8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52140/0 (socket says 172.21.15.120:52140) 2024-09-19T09:11:02.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.020+0000 7f5de0886740 10 client.0 _ll_register_callbacks cb 0x7f5dc00d33b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:02.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.020+0000 7f5dcffff640 1 -- 172.21.15.120:0/3521159182 learned_addr learned my addr 172.21.15.120:0/3521159182 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:02.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.021+0000 7f5ddca4a640 1 --2- 172.21.15.120:0/3521159182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5556ca370 0x55a5556cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:02.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.021+0000 7f5dd7fff640 1 --2- 172.21.15.120:0/3521159182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5556bf5d0 0x55a5556d2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:02.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.021+0000 7f5dcffff640 1 -- 172.21.15.120:0/3521159182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5556bf5d0 msgr2=0x55a5556d2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:02.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.021+0000 7f5dcffff640 1 --2- 172.21.15.120:0/3521159182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5556bf5d0 0x55a5556d2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.021+0000 7f5dcffff640 1 -- 172.21.15.120:0/3521159182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5556ca370 msgr2=0x55a5556cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:02.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.021+0000 7f5dcffff640 1 --2- 172.21.15.120:0/3521159182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5556ca370 0x55a5556cbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.021+0000 7f5dcffff640 1 -- 172.21.15.120:0/3521159182 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5556cc610 con 0x55a5556c0b40 2024-09-19T09:11:02.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.021+0000 7f5ddca4a640 1 --2- 172.21.15.120:0/3521159182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5556ca370 0x55a5556cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:11:02.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.021+0000 7f5dcffff640 1 --2- 172.21.15.120:0/3521159182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5556c0b40 0x55a5556cb8f0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0098a50 tx=0x7f5dc00d3460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6ce9fce739712d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:02.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.021+0000 7f5db77fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:11:02.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.021+0000 7f5db77fe640 1 -- 172.21.15.120:0/3521159182 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc00b4020 con 0x55a5556c0b40 2024-09-19T09:11:02.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.021+0000 7f5db77fe640 1 -- 172.21.15.120:0/3521159182 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f5dc00cd670 con 0x55a5556c0b40 2024-09-19T09:11:02.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.021+0000 7f5de0886740 1 -- 172.21.15.120:0/3521159182 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dc804c070 con 0x55a5556c0b40 2024-09-19T09:11:02.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.021+0000 7f5db77fe640 1 -- 172.21.15.120:0/3521159182 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5dc009c780 con 0x55a5556c0b40 2024-09-19T09:11:02.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.021+0000 7f5de0886740 10 client.18273 fetch_fsmap learned FSMap version 30 2024-09-19T09:11:02.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.021+0000 7f5de0886740 1 -- 172.21.15.120:0/3521159182 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a555748bd0 con 0x55a5556c0b40 2024-09-19T09:11:02.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.021+0000 7f5db77fe640 1 -- 172.21.15.120:0/3521159182 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dc804c070 con 0x55a5556c0b40 2024-09-19T09:11:02.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.022+0000 7f5db77fe640 1 -- 172.21.15.120:0/3521159182 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5dc00986f0 con 0x55a5556c0b40 2024-09-19T09:11:02.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.022+0000 7f5de0886740 10 client.18273 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:11:02.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.022+0000 7f5de0886740 10 client.18273 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:11:02.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.022+0000 7f5de0886740 20 client.18273 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:02.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.022+0000 7f5de0886740 1 --2- 172.21.15.120:0/3521159182 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55a555749280 0x55a55574b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:02.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.022+0000 7f5de0886740 4 client.18273 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:11:02.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.022+0000 7f5de0886740 1 -- 172.21.15.120:0/3521159182 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a55562cd70 con 0x55a555749280 2024-09-19T09:11:02.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.022+0000 7f5dd7fff640 1 --2- 172.21.15.120:0/3521159182 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55a555749280 0x55a55574b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:02.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.022+0000 7f5db77fe640 10 client.18273 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:11:02.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.022+0000 7f5dd7fff640 1 --2- 172.21.15.120:0/3521159182 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55a555749280 0x55a55574b6a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a2478b3484ecce53 server_cookie=6afbeaa655e6bd9d in_seq=0 out_seq=0 2024-09-19T09:11:02.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.022+0000 7f5db77fe640 1 -- 172.21.15.120:0/3521159182 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a55562cd70 con 0x55a555749280 2024-09-19T09:11:02.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.022+0000 7f5db77fe640 10 client.18273 handle_command_reply: tid=0 2024-09-19T09:11:02.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.022+0000 7f5de0886740 1 client.18273 shutdown 2024-09-19T09:11:02.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.023+0000 7f5de0886740 1 -- 172.21.15.120:0/3521159182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5556c0b40 msgr2=0x55a5556cb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:02.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.023+0000 7f5de0886740 1 --2- 172.21.15.120:0/3521159182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5556c0b40 0x55a5556cb8f0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0098a50 tx=0x7f5dc00d3460 comp rx=0 tx=0).stop 2024-09-19T09:11:02.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.023+0000 7f5de0886740 1 -- 172.21.15.120:0/3521159182 shutdown_connections 2024-09-19T09:11:02.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.023+0000 7f5de0886740 1 --2- 172.21.15.120:0/3521159182 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55a555749280 0x55a55574b6a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.023+0000 7f5de0886740 1 --2- 172.21.15.120:0/3521159182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a5556ca370 0x55a5556cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.023+0000 7f5de0886740 1 --2- 172.21.15.120:0/3521159182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5556c0b40 0x55a5556cb8f0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.023+0000 7f5de0886740 1 --2- 172.21.15.120:0/3521159182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a5556bf5d0 0x55a5556d2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.023+0000 7f5de0886740 1 -- 172.21.15.120:0/3521159182 >> 172.21.15.120:0/3521159182 conn(0x55a555688310 msgr2=0x55a5556c10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:02.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.023+0000 7f5db77fe640 0 client.18273 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:11:02.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.023+0000 7f5de0886740 1 -- 172.21.15.120:0/3521159182 shutdown_connections 2024-09-19T09:11:02.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.023+0000 7f5de0886740 1 -- 172.21.15.120:0/3521159182 wait complete. 2024-09-19T09:11:02.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.023+0000 7f5de0886740 20 client.18273 trim_cache size 0 max 16384 2024-09-19T09:11:02.037 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:11:02.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.023+0000 7f5dde4ad640 1 -- 172.21.15.120:0/3599096971 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5dc40a3240 msgr2=0x7f5dc40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:02.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.023+0000 7f5dde4ad640 1 --2- 172.21.15.120:0/3599096971 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5dc40a3240 0x7f5dc40a5700 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0005fd0 tx=0x7f5dd0009600 comp rx=0 tx=0).stop 2024-09-19T09:11:02.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.023+0000 7f5dde4ad640 1 -- 172.21.15.120:0/3599096971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5dd8101420 msgr2=0x7f5dd8199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:02.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.023+0000 7f5dde4ad640 1 --2- 172.21.15.120:0/3599096971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5dd8101420 0x7f5dd8199790 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000c920 tx=0x7f5dc000cde0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.023+0000 7f5dde4ad640 1 -- 172.21.15.120:0/3599096971 shutdown_connections 2024-09-19T09:11:02.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.023+0000 7f5dde4ad640 1 --2- 172.21.15.120:0/3599096971 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5dc40a3240 0x7f5dc40a5700 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.023+0000 7f5dde4ad640 1 --2- 172.21.15.120:0/3599096971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5dd8106a80 0x7f5dd81a42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.023+0000 7f5dde4ad640 1 --2- 172.21.15.120:0/3599096971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5dd8101420 0x7f5dd8199790 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.023+0000 7f5dde4ad640 1 --2- 172.21.15.120:0/3599096971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dd8100a80 0x7f5dd8199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:02.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.023+0000 7f5dde4ad640 1 -- 172.21.15.120:0/3599096971 >> 172.21.15.120:0/3599096971 conn(0x7f5dd80fc7a0 msgr2=0x7f5dd80fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:02.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.024+0000 7f5dde4ad640 1 -- 172.21.15.120:0/3599096971 shutdown_connections 2024-09-19T09:11:02.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:02.024+0000 7f5dde4ad640 1 -- 172.21.15.120:0/3599096971 wait complete. 2024-09-19T09:11:02.060 INFO:tasks.workunit.client.0.smithi120.stdout: 1 707664 8.77 MB/sec execute 297 sec latency 25421.113 ms 2024-09-19T09:11:02.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:01 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3436849309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:11:02.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:01 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3436849309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:11:02.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:01 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3436849309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:11:02.928 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db --set-id aeca8af5 --release --await' 2024-09-19T09:11:02.929 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db --set-id aeca8af5 --release --await 2024-09-19T09:11:03.060 INFO:tasks.workunit.client.0.smithi120.stdout: 1 707664 8.74 MB/sec execute 298 sec latency 26421.157 ms 2024-09-19T09:11:03.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:02 smithi196 ceph-mon[35150]: pgmap v587: 97 pgs: 97 active+clean; 26 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.4 KiB/s wr, 17 op/s 2024-09-19T09:11:03.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:02 smithi153 ceph-mon[35071]: pgmap v587: 97 pgs: 97 active+clean; 26 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.4 KiB/s wr, 17 op/s 2024-09-19T09:11:03.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:02 smithi120 ceph-mon[31150]: pgmap v587: 97 pgs: 97 active+clean; 26 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.4 KiB/s wr, 17 op/s 2024-09-19T09:11:03.494 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:11:03.616 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:11:04.014 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:03 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:11:04.014 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:03 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:11:04.014 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:03 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:11:04.015 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:03 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:11:04.060 INFO:tasks.workunit.client.0.smithi120.stdout: 1 707664 8.71 MB/sec execute 299 sec latency 27421.210 ms 2024-09-19T09:11:04.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.100+0000 7fc42e88d640 1 -- 172.21.15.120:0/3560732942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc428100a80 msgr2=0x7fc428100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:04.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.100+0000 7fc42e88d640 1 --2- 172.21.15.120:0/3560732942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc428100a80 0x7fc428100ee0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fc410009930 tx=0x7fc41002a420 comp rx=0 tx=0).stop 2024-09-19T09:11:04.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.100+0000 7fc42e88d640 1 -- 172.21.15.120:0/3560732942 shutdown_connections 2024-09-19T09:11:04.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.100+0000 7fc42e88d640 1 --2- 172.21.15.120:0/3560732942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc428101420 0x7fc42810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.100+0000 7fc42e88d640 1 --2- 172.21.15.120:0/3560732942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc428100a80 0x7fc428100ee0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.100+0000 7fc42e88d640 1 --2- 172.21.15.120:0/3560732942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc428106a80 0x7fc428106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.100+0000 7fc42e88d640 1 -- 172.21.15.120:0/3560732942 >> 172.21.15.120:0/3560732942 conn(0x7fc4280fc7a0 msgr2=0x7fc4280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:04.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.100+0000 7fc42e88d640 1 -- 172.21.15.120:0/3560732942 shutdown_connections 2024-09-19T09:11:04.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.100+0000 7fc42e88d640 1 -- 172.21.15.120:0/3560732942 wait complete. 2024-09-19T09:11:04.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.101+0000 7fc42e88d640 1 Processor -- start 2024-09-19T09:11:04.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.101+0000 7fc42e88d640 1 -- start start 2024-09-19T09:11:04.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.101+0000 7fc42e88d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc428100a80 0x7fc4281992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:04.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.101+0000 7fc42e88d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc428101420 0x7fc428199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:04.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.101+0000 7fc42e88d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc428106a80 0x7fc4281a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:04.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.101+0000 7fc42e88d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc428110e40 con 0x7fc428100a80 2024-09-19T09:11:04.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.101+0000 7fc42e88d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc428110cc0 con 0x7fc428101420 2024-09-19T09:11:04.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.101+0000 7fc42e88d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc428110fc0 con 0x7fc428106a80 2024-09-19T09:11:04.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.101+0000 7fc427fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc428100a80 0x7fc4281992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:04.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.101+0000 7fc427fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc428100a80 0x7fc4281992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52146/0 (socket says 172.21.15.120:52146) 2024-09-19T09:11:04.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.101+0000 7fc427fff640 1 -- 172.21.15.120:0/3866959648 learned_addr learned my addr 172.21.15.120:0/3866959648 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:04.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.101+0000 7fc41ffff640 1 --2- 172.21.15.120:0/3866959648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc428101420 0x7fc428199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:04.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.101+0000 7fc42ce2a640 1 --2- 172.21.15.120:0/3866959648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc428106a80 0x7fc4281a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:04.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.102+0000 7fc427fff640 1 -- 172.21.15.120:0/3866959648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc428106a80 msgr2=0x7fc4281a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:04.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.102+0000 7fc427fff640 1 --2- 172.21.15.120:0/3866959648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc428106a80 0x7fc4281a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.102+0000 7fc427fff640 1 -- 172.21.15.120:0/3866959648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc428101420 msgr2=0x7fc428199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:04.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.102+0000 7fc427fff640 1 --2- 172.21.15.120:0/3866959648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc428101420 0x7fc428199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.102+0000 7fc427fff640 1 -- 172.21.15.120:0/3866959648 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4281a4890 con 0x7fc428100a80 2024-09-19T09:11:04.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.102+0000 7fc42ce2a640 1 --2- 172.21.15.120:0/3866959648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc428106a80 0x7fc4281a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:11:04.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.102+0000 7fc427fff640 1 --2- 172.21.15.120:0/3866959648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc428100a80 0x7fc4281992d0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fc41800d950 tx=0x7fc41800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:04.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.102+0000 7fc425ffb640 1 -- 172.21.15.120:0/3866959648 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc418017030 con 0x7fc428100a80 2024-09-19T09:11:04.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.102+0000 7fc425ffb640 1 -- 172.21.15.120:0/3866959648 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc418004020 con 0x7fc428100a80 2024-09-19T09:11:04.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.102+0000 7fc42e88d640 1 -- 172.21.15.120:0/3866959648 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4281a4a60 con 0x7fc428100a80 2024-09-19T09:11:04.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.102+0000 7fc425ffb640 1 -- 172.21.15.120:0/3866959648 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4180047a0 con 0x7fc428100a80 2024-09-19T09:11:04.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.102+0000 7fc42e88d640 1 -- 172.21.15.120:0/3866959648 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4281a4f70 con 0x7fc428100a80 2024-09-19T09:11:04.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.105+0000 7fc425ffb640 1 -- 172.21.15.120:0/3866959648 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fc4180041d0 con 0x7fc428100a80 2024-09-19T09:11:04.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.105+0000 7fc430c66740 1 Processor -- start 2024-09-19T09:11:04.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.105+0000 7fc430c66740 1 -- start start 2024-09-19T09:11:04.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.105+0000 7fc430c66740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647551a1f40 0x5647551a2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:04.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.105+0000 7fc430c66740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647551a2860 0x5647551b1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:04.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.105+0000 7fc430c66740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647551b1650 0x5647551b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:04.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.105+0000 7fc430c66740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564755107620 con 0x5647551b1650 2024-09-19T09:11:04.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.105+0000 7fc430c66740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5647550aee50 con 0x5647551a1f40 2024-09-19T09:11:04.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.105+0000 7fc430c66740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5647551035e0 con 0x5647551a2860 2024-09-19T09:11:04.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.105+0000 7fc41ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647551b1650 0x5647551b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:04.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.105+0000 7fc41ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647551b1650 0x5647551b3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52160/0 (socket says 172.21.15.120:52160) 2024-09-19T09:11:04.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.105+0000 7fc41ffff640 1 -- 172.21.15.120:0/3764662221 learned_addr learned my addr 172.21.15.120:0/3764662221 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:04.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.106+0000 7fc42ce2a640 1 --2- 172.21.15.120:0/3764662221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647551a1f40 0x5647551a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:04.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.106+0000 7fc427fff640 1 --2- 172.21.15.120:0/3764662221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647551a2860 0x5647551b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:04.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.106+0000 7fc425ffb640 1 --2- 172.21.15.120:0/3866959648 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc4140a3240 0x7fc4140a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:04.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.106+0000 7fc41ffff640 1 -- 172.21.15.120:0/3764662221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647551a2860 msgr2=0x5647551b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:04.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.106+0000 7fc41ffff640 1 --2- 172.21.15.120:0/3764662221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647551a2860 0x5647551b1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.106+0000 7fc41ffff640 1 -- 172.21.15.120:0/3764662221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647551a1f40 msgr2=0x5647551a2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:04.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.106+0000 7fc41ffff640 1 --2- 172.21.15.120:0/3764662221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647551a1f40 0x5647551a2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.106+0000 7fc41ffff640 1 -- 172.21.15.120:0/3764662221 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56475509fd40 con 0x5647551b1650 2024-09-19T09:11:04.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.106+0000 7fc42ce2a640 1 --2- 172.21.15.120:0/3866959648 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc4140a3240 0x7fc4140a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:04.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.106+0000 7fc425ffb640 1 -- 172.21.15.120:0/3866959648 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fc4180cf260 con 0x7fc428100a80 2024-09-19T09:11:04.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.106+0000 7fc41ffff640 1 --2- 172.21.15.120:0/3764662221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647551b1650 0x5647551b3a40 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fc410009930 tx=0x7fc410031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c472f02d9a2e0ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:04.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.106+0000 7fc41e7fc640 1 -- 172.21.15.120:0/3764662221 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc410040070 con 0x5647551b1650 2024-09-19T09:11:04.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.106+0000 7fc41e7fc640 1 -- 172.21.15.120:0/3764662221 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc410005020 con 0x5647551b1650 2024-09-19T09:11:04.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.106+0000 7fc41e7fc640 1 -- 172.21.15.120:0/3764662221 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc410035040 con 0x5647551b1650 2024-09-19T09:11:04.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.107+0000 7fc430c66740 1 -- 172.21.15.120:0/3764662221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647551b1650 msgr2=0x5647551b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:04.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.107+0000 7fc430c66740 1 --2- 172.21.15.120:0/3764662221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647551b1650 0x5647551b3a40 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fc410009930 tx=0x7fc410031630 comp rx=0 tx=0).stop 2024-09-19T09:11:04.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.107+0000 7fc42ce2a640 1 --2- 172.21.15.120:0/3866959648 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc4140a3240 0x7fc4140a5700 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc42000af00 tx=0x7fc420004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:04.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.107+0000 7fc430c66740 1 -- 172.21.15.120:0/3764662221 shutdown_connections 2024-09-19T09:11:04.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.107+0000 7fc430c66740 1 --2- 172.21.15.120:0/3764662221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647551b1650 0x5647551b3a40 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.107+0000 7fc430c66740 1 --2- 172.21.15.120:0/3764662221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647551a2860 0x5647551b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.107+0000 7fc430c66740 1 --2- 172.21.15.120:0/3764662221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647551a1f40 0x5647551a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.107+0000 7fc430c66740 1 -- 172.21.15.120:0/3764662221 >> 172.21.15.120:0/3764662221 conn(0x56475519e5f0 msgr2=0x56475519e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:04.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.107+0000 7fc430c66740 1 -- 172.21.15.120:0/3764662221 shutdown_connections 2024-09-19T09:11:04.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.107+0000 7fc430c66740 1 -- 172.21.15.120:0/3764662221 wait complete. 2024-09-19T09:11:04.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.107+0000 7fc430c66740 1 -- start start 2024-09-19T09:11:04.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.107+0000 7fc430c66740 1 Processor -- start 2024-09-19T09:11:04.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.108+0000 7fc430c66740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647551a1f40 0x5647551ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:04.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.108+0000 7fc430c66740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647551a2860 0x5647551ae930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:04.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.108+0000 7fc430c66740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647551b1650 0x5647551aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:04.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.108+0000 7fc430c66740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5647551b4220 con 0x5647551a2860 2024-09-19T09:11:04.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.108+0000 7fc430c66740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56475508ff10 con 0x5647551b1650 2024-09-19T09:11:04.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.108+0000 7fc430c66740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5647551b43a0 con 0x5647551a1f40 2024-09-19T09:11:04.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.108+0000 7fc430c66740 10 client.0 _ll_register_callbacks cb 0x5647550b3db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:04.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.108+0000 7fc41ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647551a2860 0x5647551ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:04.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.108+0000 7fc41ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647551a2860 0x5647551ae930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52162/0 (socket says 172.21.15.120:52162) 2024-09-19T09:11:04.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.108+0000 7fc41ffff640 1 -- 172.21.15.120:0/3753491047 learned_addr learned my addr 172.21.15.120:0/3753491047 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:04.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.108+0000 7fc42ce2a640 1 --2- 172.21.15.120:0/3753491047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647551b1650 0x5647551aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:04.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.108+0000 7fc427fff640 1 --2- 172.21.15.120:0/3753491047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647551a1f40 0x5647551ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:04.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.108+0000 7fc41ffff640 1 -- 172.21.15.120:0/3753491047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647551a1f40 msgr2=0x5647551ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:04.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.108+0000 7fc41ffff640 1 --2- 172.21.15.120:0/3753491047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647551a1f40 0x5647551ae3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.108+0000 7fc41ffff640 1 -- 172.21.15.120:0/3753491047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647551b1650 msgr2=0x5647551aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:04.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.108+0000 7fc41ffff640 1 --2- 172.21.15.120:0/3753491047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647551b1650 0x5647551aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.108+0000 7fc41ffff640 1 -- 172.21.15.120:0/3753491047 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564754e16580 con 0x5647551a2860 2024-09-19T09:11:04.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.108+0000 7fc427fff640 1 --2- 172.21.15.120:0/3753491047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647551a1f40 0x5647551ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:11:04.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.108+0000 7fc41ffff640 1 --2- 172.21.15.120:0/3753491047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647551a2860 0x5647551ae930 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fc410005940 tx=0x7fc410009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d56a95d13ea914 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:04.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.108+0000 7fc4077fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:11:04.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.108+0000 7fc4077fe640 1 -- 172.21.15.120:0/3753491047 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc410040070 con 0x5647551a2860 2024-09-19T09:11:04.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.108+0000 7fc4077fe640 1 -- 172.21.15.120:0/3753491047 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fc410005560 con 0x5647551a2860 2024-09-19T09:11:04.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.108+0000 7fc430c66740 1 -- 172.21.15.120:0/3753491047 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc410005020 con 0x5647551a2860 2024-09-19T09:11:04.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.109+0000 7fc4077fe640 1 -- 172.21.15.120:0/3753491047 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc410035070 con 0x5647551a2860 2024-09-19T09:11:04.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.109+0000 7fc430c66740 10 client.18285 fetch_fsmap learned FSMap version 30 2024-09-19T09:11:04.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.109+0000 7fc430c66740 1 -- 172.21.15.120:0/3753491047 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564754ee0960 con 0x5647551a2860 2024-09-19T09:11:04.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.109+0000 7fc4077fe640 1 -- 172.21.15.120:0/3753491047 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc410005020 con 0x5647551a2860 2024-09-19T09:11:04.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.109+0000 7fc4077fe640 1 -- 172.21.15.120:0/3753491047 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc41003cac0 con 0x5647551a2860 2024-09-19T09:11:04.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.109+0000 7fc430c66740 10 client.18285 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:11:04.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.109+0000 7fc430c66740 10 client.18285 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:11:04.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.109+0000 7fc430c66740 20 client.18285 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:04.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.109+0000 7fc430c66740 1 --2- 172.21.15.120:0/3753491047 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5647551c0f80 0x5647551c3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:04.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.109+0000 7fc430c66740 4 client.18285 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:11:04.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.109+0000 7fc430c66740 1 -- 172.21.15.120:0/3753491047 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564755114b90 con 0x5647551c0f80 2024-09-19T09:11:04.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.109+0000 7fc427fff640 1 --2- 172.21.15.120:0/3753491047 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5647551c0f80 0x5647551c3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:04.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.110+0000 7fc4077fe640 10 client.18285 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:04.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.110+0000 7fc427fff640 1 --2- 172.21.15.120:0/3753491047 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5647551c0f80 0x5647551c3370 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec191cb018f41f30 server_cookie=36df443b1c74a5a3 in_seq=0 out_seq=0 2024-09-19T09:11:04.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.110+0000 7fc4077fe640 1 -- 172.21.15.120:0/3753491047 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564755114b90 con 0x5647551c0f80 2024-09-19T09:11:04.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.110+0000 7fc4077fe640 10 client.18285 handle_command_reply: tid=0 2024-09-19T09:11:04.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.110+0000 7fc430c66740 1 client.18285 shutdown 2024-09-19T09:11:04.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.110+0000 7fc430c66740 1 -- 172.21.15.120:0/3753491047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647551a2860 msgr2=0x5647551ae930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:04.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.110+0000 7fc430c66740 1 --2- 172.21.15.120:0/3753491047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647551a2860 0x5647551ae930 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fc410005940 tx=0x7fc410009660 comp rx=0 tx=0).stop 2024-09-19T09:11:04.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.110+0000 7fc430c66740 1 -- 172.21.15.120:0/3753491047 shutdown_connections 2024-09-19T09:11:04.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.110+0000 7fc430c66740 1 --2- 172.21.15.120:0/3753491047 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5647551c0f80 0x5647551c3370 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.110+0000 7fc430c66740 1 --2- 172.21.15.120:0/3753491047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647551b1650 0x5647551aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.110+0000 7fc430c66740 1 --2- 172.21.15.120:0/3753491047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647551a2860 0x5647551ae930 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.110+0000 7fc4077fe640 0 client.18285 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:04.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.110+0000 7fc430c66740 1 --2- 172.21.15.120:0/3753491047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647551a1f40 0x5647551ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.110+0000 7fc430c66740 1 -- 172.21.15.120:0/3753491047 >> 172.21.15.120:0/3753491047 conn(0x56475519e5f0 msgr2=0x5647551a4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:04.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.111+0000 7fc430c66740 1 -- 172.21.15.120:0/3753491047 shutdown_connections 2024-09-19T09:11:04.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.111+0000 7fc430c66740 1 -- 172.21.15.120:0/3753491047 wait complete. 2024-09-19T09:11:04.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.111+0000 7fc430c66740 20 client.18285 trim_cache size 0 max 16384 2024-09-19T09:11:04.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.127+0000 7fc430c66740 1 Processor -- start 2024-09-19T09:11:04.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.127+0000 7fc430c66740 1 -- start start 2024-09-19T09:11:04.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.128+0000 7fc430c66740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647551a1f40 0x5647551b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:04.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.128+0000 7fc430c66740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647551a2860 0x5647551b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:04.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.128+0000 7fc430c66740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647551b1650 0x5647551ab650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:04.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.128+0000 7fc430c66740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc4281ab450 con 0x5647551a1f40 2024-09-19T09:11:04.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.128+0000 7fc430c66740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc4281acf90 con 0x5647551b1650 2024-09-19T09:11:04.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.128+0000 7fc430c66740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc4281abb20 con 0x5647551a2860 2024-09-19T09:11:04.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.128+0000 7fc427fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647551a1f40 0x5647551b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:04.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.128+0000 7fc427fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647551a1f40 0x5647551b2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52176/0 (socket says 172.21.15.120:52176) 2024-09-19T09:11:04.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.128+0000 7fc427fff640 1 -- 172.21.15.120:0/1937430378 learned_addr learned my addr 172.21.15.120:0/1937430378 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:04.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.128+0000 7fc41ffff640 1 --2- 172.21.15.120:0/1937430378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647551a2860 0x5647551b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:04.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.128+0000 7fc42ce2a640 1 --2- 172.21.15.120:0/1937430378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647551b1650 0x5647551ab650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:04.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.128+0000 7fc427fff640 1 -- 172.21.15.120:0/1937430378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647551a2860 msgr2=0x5647551b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:04.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.128+0000 7fc427fff640 1 --2- 172.21.15.120:0/1937430378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647551a2860 0x5647551b2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.128+0000 7fc427fff640 1 -- 172.21.15.120:0/1937430378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647551b1650 msgr2=0x5647551ab650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:04.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.128+0000 7fc427fff640 1 --2- 172.21.15.120:0/1937430378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647551b1650 0x5647551ab650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.128+0000 7fc427fff640 1 -- 172.21.15.120:0/1937430378 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647551b42a0 con 0x5647551a1f40 2024-09-19T09:11:04.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.128+0000 7fc427fff640 1 --2- 172.21.15.120:0/1937430378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647551a1f40 0x5647551b2a40 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fc418000c00 tx=0x7fc418005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10041fccbd7728b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:04.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.128+0000 7fc41dffb640 1 -- 172.21.15.120:0/1937430378 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41809b1e0 con 0x5647551a1f40 2024-09-19T09:11:04.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.128+0000 7fc41dffb640 1 -- 172.21.15.120:0/1937430378 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc41809bf80 con 0x5647551a1f40 2024-09-19T09:11:04.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.128+0000 7fc41dffb640 1 -- 172.21.15.120:0/1937430378 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4180a4060 con 0x5647551a1f40 2024-09-19T09:11:04.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.129+0000 7fc430c66740 1 -- 172.21.15.120:0/1937430378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647551a1f40 msgr2=0x5647551b2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:04.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.129+0000 7fc430c66740 1 --2- 172.21.15.120:0/1937430378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647551a1f40 0x5647551b2a40 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fc418000c00 tx=0x7fc418005dc0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.129+0000 7fc430c66740 1 -- 172.21.15.120:0/1937430378 shutdown_connections 2024-09-19T09:11:04.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.129+0000 7fc430c66740 1 --2- 172.21.15.120:0/1937430378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647551b1650 0x5647551ab650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.129+0000 7fc430c66740 1 --2- 172.21.15.120:0/1937430378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647551a2860 0x5647551b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.129+0000 7fc430c66740 1 --2- 172.21.15.120:0/1937430378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647551a1f40 0x5647551b2a40 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.129+0000 7fc430c66740 1 -- 172.21.15.120:0/1937430378 >> 172.21.15.120:0/1937430378 conn(0x56475519e5f0 msgr2=0x56475519cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:04.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.129+0000 7fc430c66740 1 -- 172.21.15.120:0/1937430378 shutdown_connections 2024-09-19T09:11:04.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.129+0000 7fc430c66740 1 -- 172.21.15.120:0/1937430378 wait complete. 2024-09-19T09:11:04.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.130+0000 7fc430c66740 1 -- start start 2024-09-19T09:11:04.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.130+0000 7fc430c66740 1 Processor -- start 2024-09-19T09:11:04.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.130+0000 7fc430c66740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647551a1f40 0x5647551c2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:04.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.130+0000 7fc430c66740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647551a2860 0x5647551c2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:04.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.130+0000 7fc430c66740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647551b1650 0x56475522a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:04.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.130+0000 7fc430c66740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc4281ac8c0 con 0x5647551a2860 2024-09-19T09:11:04.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.130+0000 7fc430c66740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc4281ac1f0 con 0x5647551a1f40 2024-09-19T09:11:04.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.130+0000 7fc430c66740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc4281a9410 con 0x5647551b1650 2024-09-19T09:11:04.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.130+0000 7fc430c66740 10 client.0 _ll_register_callbacks cb 0x7fc41002aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:04.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.130+0000 7fc41ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647551a2860 0x5647551c2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:04.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.130+0000 7fc41ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647551a2860 0x5647551c2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52186/0 (socket says 172.21.15.120:52186) 2024-09-19T09:11:04.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.130+0000 7fc41ffff640 1 -- 172.21.15.120:0/3575678781 learned_addr learned my addr 172.21.15.120:0/3575678781 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:04.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.130+0000 7fc42ce2a640 1 --2- 172.21.15.120:0/3575678781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647551b1650 0x56475522a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:04.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.130+0000 7fc427fff640 1 --2- 172.21.15.120:0/3575678781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647551a1f40 0x5647551c2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:04.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.131+0000 7fc41ffff640 1 -- 172.21.15.120:0/3575678781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647551b1650 msgr2=0x56475522a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:04.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.131+0000 7fc41ffff640 1 --2- 172.21.15.120:0/3575678781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647551b1650 0x56475522a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.131+0000 7fc41ffff640 1 -- 172.21.15.120:0/3575678781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647551a1f40 msgr2=0x5647551c2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:04.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.131+0000 7fc41ffff640 1 --2- 172.21.15.120:0/3575678781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647551a1f40 0x5647551c2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.131+0000 7fc41ffff640 1 -- 172.21.15.120:0/3575678781 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56475522adf0 con 0x5647551a2860 2024-09-19T09:11:04.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.131+0000 7fc41ffff640 1 --2- 172.21.15.120:0/3575678781 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647551a2860 0x5647551c2db0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fc41002a940 tx=0x7fc410005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1c3704f050cb1d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:04.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.131+0000 7fc4077fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:11:04.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.131+0000 7fc4077fe640 1 -- 172.21.15.120:0/3575678781 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc410040040 con 0x5647551a2860 2024-09-19T09:11:04.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.131+0000 7fc4077fe640 1 -- 172.21.15.120:0/3575678781 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fc4100500b0 con 0x5647551a2860 2024-09-19T09:11:04.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.131+0000 7fc430c66740 1 -- 172.21.15.120:0/3575678781 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc41809bf80 con 0x5647551a2860 2024-09-19T09:11:04.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.131+0000 7fc4077fe640 1 -- 172.21.15.120:0/3575678781 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc41003cc40 con 0x5647551a2860 2024-09-19T09:11:04.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.131+0000 7fc430c66740 10 client.18291 fetch_fsmap learned FSMap version 30 2024-09-19T09:11:04.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.131+0000 7fc430c66740 1 -- 172.21.15.120:0/3575678781 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56475522c100 con 0x5647551a2860 2024-09-19T09:11:04.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.131+0000 7fc4077fe640 1 -- 172.21.15.120:0/3575678781 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc41809bf80 con 0x5647551a2860 2024-09-19T09:11:04.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.131+0000 7fc4077fe640 1 -- 172.21.15.120:0/3575678781 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc410004830 con 0x5647551a2860 2024-09-19T09:11:04.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.132+0000 7fc430c66740 10 client.18291 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:11:04.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.132+0000 7fc430c66740 10 client.18291 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:11:04.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.132+0000 7fc430c66740 20 client.18291 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:04.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.132+0000 7fc430c66740 1 --2- 172.21.15.120:0/3575678781 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56475522c790 0x56475522ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:04.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.132+0000 7fc430c66740 4 client.18291 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "aeca8af5", "release": true, "await": true}] 2024-09-19T09:11:04.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.132+0000 7fc430c66740 1 -- 172.21.15.120:0/3575678781 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "set_id": "aeca8af5", "release": true, "await": true}) -- 0x56475510f0a0 con 0x56475522c790 2024-09-19T09:11:04.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.132+0000 7fc427fff640 1 --2- 172.21.15.120:0/3575678781 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56475522c790 0x56475522ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:04.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.132+0000 7fc427fff640 1 --2- 172.21.15.120:0/3575678781 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56475522c790 0x56475522ebb0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f49d2ca7234c2a52 server_cookie=7573575279083611 in_seq=0 out_seq=0 2024-09-19T09:11:04.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.132+0000 7fc4077fe640 10 client.18291 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:04.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.142+0000 7fc4077fe640 1 -- 172.21.15.120:0/3575678781 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56475510f0a0 con 0x56475522c790 2024-09-19T09:11:04.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.142+0000 7fc4077fe640 10 client.18291 handle_command_reply: tid=0 2024-09-19T09:11:04.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.142+0000 7fc430c66740 1 client.18291 shutdown 2024-09-19T09:11:04.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.142+0000 7fc430c66740 1 -- 172.21.15.120:0/3575678781 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647551a2860 msgr2=0x5647551c2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:04.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.142+0000 7fc430c66740 1 --2- 172.21.15.120:0/3575678781 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647551a2860 0x5647551c2db0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fc41002a940 tx=0x7fc410005fd0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.142+0000 7fc430c66740 1 -- 172.21.15.120:0/3575678781 shutdown_connections 2024-09-19T09:11:04.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.142+0000 7fc430c66740 1 --2- 172.21.15.120:0/3575678781 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56475522c790 0x56475522ebb0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.142+0000 7fc430c66740 1 --2- 172.21.15.120:0/3575678781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647551b1650 0x56475522a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.142+0000 7fc430c66740 1 --2- 172.21.15.120:0/3575678781 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647551a2860 0x5647551c2db0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.142+0000 7fc430c66740 1 --2- 172.21.15.120:0/3575678781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647551a1f40 0x5647551c2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.142+0000 7fc430c66740 1 -- 172.21.15.120:0/3575678781 >> 172.21.15.120:0/3575678781 conn(0x56475519e5f0 msgr2=0x5647551acb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:04.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.142+0000 7fc4077fe640 0 client.18291 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:04.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.143+0000 7fc430c66740 1 -- 172.21.15.120:0/3575678781 shutdown_connections 2024-09-19T09:11:04.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.143+0000 7fc430c66740 1 -- 172.21.15.120:0/3575678781 wait complete. 2024-09-19T09:11:04.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.143+0000 7fc430c66740 20 client.18291 trim_cache size 0 max 16384 2024-09-19T09:11:04.145 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:11:04.145 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 30, 2024-09-19T09:11:04.145 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:11:04.145 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 32, 2024-09-19T09:11:04.145 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:11:04.145 INFO:teuthology.orchestra.run.smithi120.stdout: "aeca8af5": { 2024-09-19T09:11:04.145 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 32, 2024-09-19T09:11:04.146 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:11:04.146 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:11:04.146 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:11:04.146 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:11:04.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:11:04.146 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:11:04.146 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 145.8, 2024-09-19T09:11:04.146 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:11:04.146 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:11:04.147 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:11:04.147 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:11:04.147 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:11:04.147 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:11:04.147 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:11:04.147 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:11:04.147 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:11:04.147 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:11:04.147 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:11:04.147 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:11:04.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.143+0000 7fc42e88d640 1 -- 172.21.15.120:0/3866959648 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc4140a3240 msgr2=0x7fc4140a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:04.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.143+0000 7fc42e88d640 1 --2- 172.21.15.120:0/3866959648 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc4140a3240 0x7fc4140a5700 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc42000af00 tx=0x7fc420004a40 comp rx=0 tx=0).stop 2024-09-19T09:11:04.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.143+0000 7fc42e88d640 1 -- 172.21.15.120:0/3866959648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc428100a80 msgr2=0x7fc4281992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:04.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.143+0000 7fc42e88d640 1 --2- 172.21.15.120:0/3866959648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc428100a80 0x7fc4281992d0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fc41800d950 tx=0x7fc41800de10 comp rx=0 tx=0).stop 2024-09-19T09:11:04.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.143+0000 7fc42e88d640 1 -- 172.21.15.120:0/3866959648 shutdown_connections 2024-09-19T09:11:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.143+0000 7fc42e88d640 1 --2- 172.21.15.120:0/3866959648 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc4140a3240 0x7fc4140a5700 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.143+0000 7fc42e88d640 1 --2- 172.21.15.120:0/3866959648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc428106a80 0x7fc4281a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.143+0000 7fc42e88d640 1 --2- 172.21.15.120:0/3866959648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc428101420 0x7fc428199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.143+0000 7fc42e88d640 1 --2- 172.21.15.120:0/3866959648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc428100a80 0x7fc4281992d0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.143+0000 7fc42e88d640 1 -- 172.21.15.120:0/3866959648 >> 172.21.15.120:0/3866959648 conn(0x7fc4280fc7a0 msgr2=0x7fc4280fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.144+0000 7fc42e88d640 1 -- 172.21.15.120:0/3866959648 shutdown_connections 2024-09-19T09:11:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:04.144+0000 7fc42e88d640 1 -- 172.21.15.120:0/3866959648 wait complete. 2024-09-19T09:11:04.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:03 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:11:04.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:03 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:11:04.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:03 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:11:04.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:03 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:11:04.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:03 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:11:04.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:03 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:11:04.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:03 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:11:04.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:03 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:11:04.894 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'aeca8af5', total seconds elapsed: 31.0 2024-09-19T09:11:05.003 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:11:05.060 INFO:tasks.workunit.client.0.smithi120.stdout: 1 710639 8.74 MB/sec execute 300 sec latency 27502.544 ms 2024-09-19T09:11:05.126 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:04 smithi120 ceph-mon[31150]: pgmap v588: 97 pgs: 97 active+clean; 31 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 547 KiB/s wr, 23 op/s 2024-09-19T09:11:05.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:04 smithi196 ceph-mon[35150]: pgmap v588: 97 pgs: 97 active+clean; 31 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 547 KiB/s wr, 23 op/s 2024-09-19T09:11:05.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:04 smithi153 ceph-mon[35071]: pgmap v588: 97 pgs: 97 active+clean; 31 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 547 KiB/s wr, 23 op/s 2024-09-19T09:11:05.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.508+0000 7fccbde83640 1 -- 172.21.15.120:0/3254192579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccb8100e20 msgr2=0x7fccb8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:05.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.508+0000 7fccbde83640 1 --2- 172.21.15.120:0/3254192579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccb8100e20 0x7fccb8101200 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009900 tx=0x7fcca802a480 comp rx=0 tx=0).stop 2024-09-19T09:11:05.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.508+0000 7fccbde83640 1 -- 172.21.15.120:0/3254192579 shutdown_connections 2024-09-19T09:11:05.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.508+0000 7fccbde83640 1 --2- 172.21.15.120:0/3254192579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccb8102ef0 0x7fccb810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.508+0000 7fccbde83640 1 --2- 172.21.15.120:0/3254192579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccb8102570 0x7fccb81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.508+0000 7fccbde83640 1 --2- 172.21.15.120:0/3254192579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccb8100e20 0x7fccb8101200 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.508+0000 7fccbde83640 1 -- 172.21.15.120:0/3254192579 >> 172.21.15.120:0/3254192579 conn(0x7fccb80fc7d0 msgr2=0x7fccb80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:05.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.508+0000 7fccbde83640 1 -- 172.21.15.120:0/3254192579 shutdown_connections 2024-09-19T09:11:05.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.508+0000 7fccbde83640 1 -- 172.21.15.120:0/3254192579 wait complete. 2024-09-19T09:11:05.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.509+0000 7fccbde83640 1 Processor -- start 2024-09-19T09:11:05.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.509+0000 7fccbde83640 1 -- start start 2024-09-19T09:11:05.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.509+0000 7fccbde83640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccb8100e20 0x7fccb8198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:05.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.509+0000 7fccbde83640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccb8102570 0x7fccb8198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:05.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.509+0000 7fccbde83640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccb8102ef0 0x7fccb819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:05.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.509+0000 7fccbde83640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fccb8110f80 con 0x7fccb8102570 2024-09-19T09:11:05.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.509+0000 7fccbde83640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fccb8110e00 con 0x7fccb8100e20 2024-09-19T09:11:05.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.509+0000 7fccbde83640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fccb8111100 con 0x7fccb8102ef0 2024-09-19T09:11:05.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.509+0000 7fccb6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccb8102570 0x7fccb8198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:05.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.509+0000 7fccb6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccb8102570 0x7fccb8198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52204/0 (socket says 172.21.15.120:52204) 2024-09-19T09:11:05.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.509+0000 7fccb6ffd640 1 -- 172.21.15.120:0/2855087427 learned_addr learned my addr 172.21.15.120:0/2855087427 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:05.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.509+0000 7fccb77fe640 1 --2- 172.21.15.120:0/2855087427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccb8100e20 0x7fccb8198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:05.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.509+0000 7fccb7fff640 1 --2- 172.21.15.120:0/2855087427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccb8102ef0 0x7fccb819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:05.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.510+0000 7fccb6ffd640 1 -- 172.21.15.120:0/2855087427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccb8102ef0 msgr2=0x7fccb819ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:05.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.510+0000 7fccb6ffd640 1 --2- 172.21.15.120:0/2855087427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccb8102ef0 0x7fccb819ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.510+0000 7fccb6ffd640 1 -- 172.21.15.120:0/2855087427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccb8100e20 msgr2=0x7fccb8198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:05.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.510+0000 7fccb6ffd640 1 --2- 172.21.15.120:0/2855087427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccb8100e20 0x7fccb8198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.510+0000 7fccb6ffd640 1 -- 172.21.15.120:0/2855087427 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb819d4e0 con 0x7fccb8102570 2024-09-19T09:11:05.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.510+0000 7fccb6ffd640 1 --2- 172.21.15.120:0/2855087427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccb8102570 0x7fccb8198a70 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fccac00e9f0 tx=0x7fccac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:05.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.510+0000 7fccb4ff9640 1 -- 172.21.15.120:0/2855087427 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac00cd80 con 0x7fccb8102570 2024-09-19T09:11:05.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.510+0000 7fccbde83640 1 -- 172.21.15.120:0/2855087427 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb819d7d0 con 0x7fccb8102570 2024-09-19T09:11:05.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.510+0000 7fccb4ff9640 1 -- 172.21.15.120:0/2855087427 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccac005500 con 0x7fccb8102570 2024-09-19T09:11:05.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.510+0000 7fccb4ff9640 1 -- 172.21.15.120:0/2855087427 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac010690 con 0x7fccb8102570 2024-09-19T09:11:05.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.510+0000 7fccbde83640 1 -- 172.21.15.120:0/2855087427 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb81a50b0 con 0x7fccb8102570 2024-09-19T09:11:05.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.512+0000 7fccb4ff9640 1 -- 172.21.15.120:0/2855087427 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fccac0046c0 con 0x7fccb8102570 2024-09-19T09:11:05.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.513+0000 7fccb4ff9640 1 --2- 172.21.15.120:0/2855087427 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcca00a3240 0x7fcca00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:05.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.513+0000 7fccc025c740 1 Processor -- start 2024-09-19T09:11:05.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.513+0000 7fccb4ff9640 1 -- 172.21.15.120:0/2855087427 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fccac0cfcf0 con 0x7fccb8102570 2024-09-19T09:11:05.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.513+0000 7fccb77fe640 1 --2- 172.21.15.120:0/2855087427 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcca00a3240 0x7fcca00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:05.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.513+0000 7fccc025c740 1 -- start start 2024-09-19T09:11:05.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.513+0000 7fccc025c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfc588b40 0x55abfc587090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:05.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.513+0000 7fccc025c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abfc5875d0 0x55abfc587a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:05.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.514+0000 7fccc025c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abfc592370 0x55abfc594760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:05.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.514+0000 7fccc025c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55abfc4ed2f0 con 0x55abfc592370 2024-09-19T09:11:05.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.514+0000 7fccc025c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55abfc494cf0 con 0x55abfc588b40 2024-09-19T09:11:05.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.514+0000 7fccc025c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55abfc4e92b0 con 0x55abfc5875d0 2024-09-19T09:11:05.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.514+0000 7fccb77fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfc588b40 0x55abfc587090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:05.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.514+0000 7fccb7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abfc592370 0x55abfc594760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:05.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.514+0000 7fccb6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abfc5875d0 0x55abfc587a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:05.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.514+0000 7fccb77fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfc588b40 0x55abfc587090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:33746/0 (socket says 172.21.15.120:33746) 2024-09-19T09:11:05.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.514+0000 7fccb77fe640 1 -- 172.21.15.120:0/3389569783 learned_addr learned my addr 172.21.15.120:0/3389569783 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:05.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.514+0000 7fccb77fe640 1 --2- 172.21.15.120:0/2855087427 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcca00a3240 0x7fcca00a5700 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009230 tx=0x7fcca8005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:05.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.514+0000 7fccb6ffd640 1 -- 172.21.15.120:0/3389569783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfc588b40 msgr2=0x55abfc587090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:05.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.514+0000 7fccb6ffd640 1 --2- 172.21.15.120:0/3389569783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfc588b40 0x55abfc587090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.514+0000 7fccb6ffd640 1 -- 172.21.15.120:0/3389569783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abfc592370 msgr2=0x55abfc594760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:05.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.514+0000 7fccb6ffd640 1 --2- 172.21.15.120:0/3389569783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abfc592370 0x55abfc594760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.514+0000 7fccb6ffd640 1 -- 172.21.15.120:0/3389569783 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abfc4793a0 con 0x55abfc5875d0 2024-09-19T09:11:05.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.514+0000 7fccb6ffd640 1 --2- 172.21.15.120:0/3389569783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abfc5875d0 0x55abfc587a10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fccac00c6a0 tx=0x7fccac09bc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27568af2a81eed3f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:05.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.514+0000 7fcc9dffb640 1 -- 172.21.15.120:0/3389569783 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac0b5070 con 0x55abfc5875d0 2024-09-19T09:11:05.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.514+0000 7fcc9dffb640 1 -- 172.21.15.120:0/3389569783 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccac0a06c0 con 0x55abfc5875d0 2024-09-19T09:11:05.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.514+0000 7fcc9dffb640 1 -- 172.21.15.120:0/3389569783 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac0a4d50 con 0x55abfc5875d0 2024-09-19T09:11:05.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.515+0000 7fccc025c740 1 -- 172.21.15.120:0/3389569783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abfc5875d0 msgr2=0x55abfc587a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:05.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.515+0000 7fccc025c740 1 --2- 172.21.15.120:0/3389569783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abfc5875d0 0x55abfc587a10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fccac00c6a0 tx=0x7fccac09bc70 comp rx=0 tx=0).stop 2024-09-19T09:11:05.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.515+0000 7fccc025c740 1 -- 172.21.15.120:0/3389569783 shutdown_connections 2024-09-19T09:11:05.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.515+0000 7fccc025c740 1 --2- 172.21.15.120:0/3389569783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abfc592370 0x55abfc594760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.515+0000 7fccc025c740 1 --2- 172.21.15.120:0/3389569783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abfc5875d0 0x55abfc587a10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.515+0000 7fccc025c740 1 --2- 172.21.15.120:0/3389569783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfc588b40 0x55abfc587090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.515+0000 7fccc025c740 1 -- 172.21.15.120:0/3389569783 >> 172.21.15.120:0/3389569783 conn(0x55abfc550310 msgr2=0x55abfc583c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:05.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.515+0000 7fccc025c740 1 -- 172.21.15.120:0/3389569783 shutdown_connections 2024-09-19T09:11:05.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.515+0000 7fccc025c740 1 -- 172.21.15.120:0/3389569783 wait complete. 2024-09-19T09:11:05.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.516+0000 7fccc025c740 1 -- start start 2024-09-19T09:11:05.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.516+0000 7fccc025c740 1 Processor -- start 2024-09-19T09:11:05.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.516+0000 7fccc025c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfc5875d0 0x55abfc591df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:05.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.516+0000 7fccc025c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abfc588b40 0x55abfc58f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:05.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.516+0000 7fccc025c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abfc592370 0x55abfc58f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:05.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.516+0000 7fccc025c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55abfc595070 con 0x55abfc592370 2024-09-19T09:11:05.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.516+0000 7fccc025c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55abfc4a8060 con 0x55abfc5875d0 2024-09-19T09:11:05.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.516+0000 7fccc025c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55abfc5951f0 con 0x55abfc588b40 2024-09-19T09:11:05.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.516+0000 7fccc025c740 10 client.0 _ll_register_callbacks cb 0x55abfc1fb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:05.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.516+0000 7fccb77fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfc5875d0 0x55abfc591df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:05.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.516+0000 7fccb6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abfc588b40 0x55abfc58f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:05.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.516+0000 7fccb7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abfc592370 0x55abfc58f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:05.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.516+0000 7fccb77fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfc5875d0 0x55abfc591df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:33748/0 (socket says 172.21.15.120:33748) 2024-09-19T09:11:05.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.516+0000 7fccb77fe640 1 -- 172.21.15.120:0/1973537460 learned_addr learned my addr 172.21.15.120:0/1973537460 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:05.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.517+0000 7fccb6ffd640 1 -- 172.21.15.120:0/1973537460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfc5875d0 msgr2=0x55abfc591df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:05.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.517+0000 7fccb6ffd640 1 --2- 172.21.15.120:0/1973537460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfc5875d0 0x55abfc591df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.517+0000 7fccb6ffd640 1 -- 172.21.15.120:0/1973537460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abfc592370 msgr2=0x55abfc58f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:05.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.517+0000 7fccb6ffd640 1 --2- 172.21.15.120:0/1973537460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abfc592370 0x55abfc58f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.517+0000 7fccb6ffd640 1 -- 172.21.15.120:0/1973537460 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abfc25e3c0 con 0x55abfc588b40 2024-09-19T09:11:05.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.517+0000 7fccb7fff640 1 --2- 172.21.15.120:0/1973537460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abfc592370 0x55abfc58f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:05.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.517+0000 7fccb6ffd640 1 --2- 172.21.15.120:0/1973537460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abfc588b40 0x55abfc58f0a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55abfc584ac0 tx=0x7fccac09c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da41f2e5d65f58da server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:05.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.517+0000 7fcc96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:11:05.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.517+0000 7fcc96ffd640 1 -- 172.21.15.120:0/1973537460 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac0ba060 con 0x55abfc588b40 2024-09-19T09:11:05.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.517+0000 7fcc96ffd640 1 -- 172.21.15.120:0/1973537460 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fccac0b51e0 con 0x55abfc588b40 2024-09-19T09:11:05.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.517+0000 7fccc025c740 1 -- 172.21.15.120:0/1973537460 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccac0a06c0 con 0x55abfc588b40 2024-09-19T09:11:05.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.517+0000 7fcc96ffd640 1 -- 172.21.15.120:0/1973537460 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fccac0b5040 con 0x55abfc588b40 2024-09-19T09:11:05.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.517+0000 7fccc025c740 10 client.27313 fetch_fsmap learned FSMap version 30 2024-09-19T09:11:05.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.517+0000 7fccc025c740 1 -- 172.21.15.120:0/1973537460 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55abfc595070 con 0x55abfc588b40 2024-09-19T09:11:05.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.517+0000 7fcc96ffd640 1 -- 172.21.15.120:0/1973537460 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccac0a06c0 con 0x55abfc588b40 2024-09-19T09:11:05.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.517+0000 7fcc96ffd640 1 -- 172.21.15.120:0/1973537460 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fccac098060 con 0x55abfc588b40 2024-09-19T09:11:05.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.518+0000 7fccc025c740 10 client.27313 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:11:05.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.518+0000 7fccc025c740 10 client.27313 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:11:05.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.518+0000 7fccc025c740 20 client.27313 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:05.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.518+0000 7fccc025c740 1 --2- 172.21.15.120:0/1973537460 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55abfc5a1d20 0x55abfc5a4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:05.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.518+0000 7fccc025c740 4 client.27313 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:11:05.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.518+0000 7fccc025c740 1 -- 172.21.15.120:0/1973537460 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abfc4fa860 con 0x55abfc5a1d20 2024-09-19T09:11:05.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.518+0000 7fccb77fe640 1 --2- 172.21.15.120:0/1973537460 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55abfc5a1d20 0x55abfc5a4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:05.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.518+0000 7fccb77fe640 1 --2- 172.21.15.120:0/1973537460 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55abfc5a1d20 0x55abfc5a4110 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e6e109cae51374a4 server_cookie=13c764443ff25f3e in_seq=0 out_seq=0 2024-09-19T09:11:05.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.518+0000 7fcc96ffd640 10 client.27313 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:11:05.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.519+0000 7fcc96ffd640 1 -- 172.21.15.120:0/1973537460 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55abfc4fa860 con 0x55abfc5a1d20 2024-09-19T09:11:05.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.519+0000 7fcc96ffd640 10 client.27313 handle_command_reply: tid=0 2024-09-19T09:11:05.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.519+0000 7fccc025c740 1 client.27313 shutdown 2024-09-19T09:11:05.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.519+0000 7fccc025c740 1 -- 172.21.15.120:0/1973537460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abfc588b40 msgr2=0x55abfc58f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:05.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.519+0000 7fccc025c740 1 --2- 172.21.15.120:0/1973537460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abfc588b40 0x55abfc58f0a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55abfc584ac0 tx=0x7fccac09c490 comp rx=0 tx=0).stop 2024-09-19T09:11:05.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.519+0000 7fccc025c740 1 -- 172.21.15.120:0/1973537460 shutdown_connections 2024-09-19T09:11:05.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.519+0000 7fccc025c740 1 --2- 172.21.15.120:0/1973537460 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55abfc5a1d20 0x55abfc5a4110 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.519+0000 7fccc025c740 1 --2- 172.21.15.120:0/1973537460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abfc592370 0x55abfc58f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.519+0000 7fcc96ffd640 0 client.27313 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:11:05.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.519+0000 7fccc025c740 1 --2- 172.21.15.120:0/1973537460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abfc588b40 0x55abfc58f0a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.519+0000 7fccc025c740 1 --2- 172.21.15.120:0/1973537460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfc5875d0 0x55abfc591df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.519+0000 7fccc025c740 1 -- 172.21.15.120:0/1973537460 >> 172.21.15.120:0/1973537460 conn(0x55abfc550310 msgr2=0x55abfc589960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:05.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.519+0000 7fccc025c740 1 -- 172.21.15.120:0/1973537460 shutdown_connections 2024-09-19T09:11:05.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.519+0000 7fccc025c740 1 -- 172.21.15.120:0/1973537460 wait complete. 2024-09-19T09:11:05.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.519+0000 7fccc025c740 20 client.27313 trim_cache size 0 max 16384 2024-09-19T09:11:05.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.532+0000 7fccc025c740 1 Processor -- start 2024-09-19T09:11:05.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.532+0000 7fccc025c740 1 -- start start 2024-09-19T09:11:05.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.532+0000 7fccc025c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfc5875d0 0x55abfc58e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:05.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.532+0000 7fccc025c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abfc588b40 0x55abfc59e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:05.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.532+0000 7fccc025c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abfc592370 0x55abfc582c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:05.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.532+0000 7fccc025c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fccb81ad5e0 con 0x55abfc588b40 2024-09-19T09:11:05.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.532+0000 7fccc025c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fccb81af120 con 0x55abfc5875d0 2024-09-19T09:11:05.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.532+0000 7fccc025c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fccb81adcb0 con 0x55abfc592370 2024-09-19T09:11:05.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.533+0000 7fccb6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abfc588b40 0x55abfc59e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:05.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.533+0000 7fccb6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abfc588b40 0x55abfc59e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52236/0 (socket says 172.21.15.120:52236) 2024-09-19T09:11:05.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.533+0000 7fccb6ffd640 1 -- 172.21.15.120:0/129818680 learned_addr learned my addr 172.21.15.120:0/129818680 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:05.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.533+0000 7fccb77fe640 1 --2- 172.21.15.120:0/129818680 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfc5875d0 0x55abfc58e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:05.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.533+0000 7fccb7fff640 1 --2- 172.21.15.120:0/129818680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abfc592370 0x55abfc582c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:05.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.533+0000 7fccb6ffd640 1 -- 172.21.15.120:0/129818680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abfc592370 msgr2=0x55abfc582c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:05.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.533+0000 7fccb6ffd640 1 --2- 172.21.15.120:0/129818680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abfc592370 0x55abfc582c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.533+0000 7fccb6ffd640 1 -- 172.21.15.120:0/129818680 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfc5875d0 msgr2=0x55abfc58e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:05.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.533+0000 7fccb6ffd640 1 --2- 172.21.15.120:0/129818680 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfc5875d0 0x55abfc58e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.533+0000 7fccb6ffd640 1 -- 172.21.15.120:0/129818680 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abfc58a120 con 0x55abfc588b40 2024-09-19T09:11:05.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.533+0000 7fccb6ffd640 1 --2- 172.21.15.120:0/129818680 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abfc588b40 0x55abfc59e310 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fccac09c010 tx=0x7fccac09a820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ae02de07c5b824 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:05.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.533+0000 7fcc9d7fa640 1 -- 172.21.15.120:0/129818680 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac0ba030 con 0x55abfc588b40 2024-09-19T09:11:05.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.533+0000 7fcc9d7fa640 1 -- 172.21.15.120:0/129818680 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccac09ca10 con 0x55abfc588b40 2024-09-19T09:11:05.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.533+0000 7fcc9d7fa640 1 -- 172.21.15.120:0/129818680 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac0b0040 con 0x55abfc588b40 2024-09-19T09:11:05.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.534+0000 7fccc025c740 1 -- 172.21.15.120:0/129818680 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abfc588b40 msgr2=0x55abfc59e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:05.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.534+0000 7fccc025c740 1 --2- 172.21.15.120:0/129818680 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abfc588b40 0x55abfc59e310 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fccac09c010 tx=0x7fccac09a820 comp rx=0 tx=0).stop 2024-09-19T09:11:05.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.534+0000 7fccc025c740 1 -- 172.21.15.120:0/129818680 shutdown_connections 2024-09-19T09:11:05.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.534+0000 7fccc025c740 1 --2- 172.21.15.120:0/129818680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abfc592370 0x55abfc582c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.534+0000 7fccc025c740 1 --2- 172.21.15.120:0/129818680 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abfc588b40 0x55abfc59e310 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.534+0000 7fccc025c740 1 --2- 172.21.15.120:0/129818680 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfc5875d0 0x55abfc58e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.534+0000 7fccc025c740 1 -- 172.21.15.120:0/129818680 >> 172.21.15.120:0/129818680 conn(0x55abfc550310 msgr2=0x55abfc582500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:05.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.534+0000 7fccc025c740 1 -- 172.21.15.120:0/129818680 shutdown_connections 2024-09-19T09:11:05.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.534+0000 7fccc025c740 1 -- 172.21.15.120:0/129818680 wait complete. 2024-09-19T09:11:05.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.534+0000 7fccc025c740 1 -- start start 2024-09-19T09:11:05.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.535+0000 7fccc025c740 1 Processor -- start 2024-09-19T09:11:05.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.535+0000 7fccc025c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abfc5875d0 0x55abfc59a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:05.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.535+0000 7fccc025c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfc588b40 0x55abfc58bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:05.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.535+0000 7fccc025c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abfc592370 0x55abfc58c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:05.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.535+0000 7fccc025c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fccb81aea50 con 0x55abfc592370 2024-09-19T09:11:05.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.535+0000 7fccc025c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fccb81ae380 con 0x55abfc588b40 2024-09-19T09:11:05.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.535+0000 7fccc025c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fccb81ab5a0 con 0x55abfc5875d0 2024-09-19T09:11:05.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.535+0000 7fccc025c740 10 client.0 _ll_register_callbacks cb 0x7fccac0b3070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:05.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.535+0000 7fccb6ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfc588b40 0x55abfc58bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:05.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.535+0000 7fccb7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abfc592370 0x55abfc58c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:05.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.535+0000 7fccb6ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfc588b40 0x55abfc58bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:33772/0 (socket says 172.21.15.120:33772) 2024-09-19T09:11:05.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.535+0000 7fccb6ffd640 1 -- 172.21.15.120:0/3806905673 learned_addr learned my addr 172.21.15.120:0/3806905673 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:05.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.535+0000 7fccb77fe640 1 --2- 172.21.15.120:0/3806905673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abfc5875d0 0x55abfc59a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:05.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.535+0000 7fccb7fff640 1 -- 172.21.15.120:0/3806905673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abfc5875d0 msgr2=0x55abfc59a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:05.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.535+0000 7fccb7fff640 1 --2- 172.21.15.120:0/3806905673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abfc5875d0 0x55abfc59a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.535+0000 7fccb7fff640 1 -- 172.21.15.120:0/3806905673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfc588b40 msgr2=0x55abfc58bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:05.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.535+0000 7fccb7fff640 1 --2- 172.21.15.120:0/3806905673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfc588b40 0x55abfc58bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.535+0000 7fccb7fff640 1 -- 172.21.15.120:0/3806905673 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abfc58c5e0 con 0x55abfc592370 2024-09-19T09:11:05.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.535+0000 7fccb77fe640 1 --2- 172.21.15.120:0/3806905673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abfc5875d0 0x55abfc59a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:05.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.535+0000 7fccb6ffd640 1 --2- 172.21.15.120:0/3806905673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfc588b40 0x55abfc58bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:05.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.536+0000 7fccb7fff640 1 --2- 172.21.15.120:0/3806905673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abfc592370 0x55abfc58c0a0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x55abfc588070 tx=0x7fccb0013c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607925b6e152f574 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:05.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.536+0000 7fcc96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:11:05.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.536+0000 7fcc96ffd640 1 -- 172.21.15.120:0/3806905673 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb0004780 con 0x55abfc592370 2024-09-19T09:11:05.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.536+0000 7fcc96ffd640 1 -- 172.21.15.120:0/3806905673 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fccb0004b20 con 0x55abfc592370 2024-09-19T09:11:05.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.536+0000 7fccc025c740 1 -- 172.21.15.120:0/3806905673 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccac09ca10 con 0x55abfc592370 2024-09-19T09:11:05.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.536+0000 7fcc96ffd640 1 -- 172.21.15.120:0/3806905673 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fccb0004d20 con 0x55abfc592370 2024-09-19T09:11:05.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.536+0000 7fccc025c740 10 client.18309 fetch_fsmap learned FSMap version 30 2024-09-19T09:11:05.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.536+0000 7fccc025c740 1 -- 172.21.15.120:0/3806905673 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55abfc60c4f0 con 0x55abfc592370 2024-09-19T09:11:05.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.536+0000 7fcc96ffd640 1 -- 172.21.15.120:0/3806905673 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccac09ca10 con 0x55abfc592370 2024-09-19T09:11:05.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.536+0000 7fcc96ffd640 1 -- 172.21.15.120:0/3806905673 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fccb0025020 con 0x55abfc592370 2024-09-19T09:11:05.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.536+0000 7fccc025c740 10 client.18309 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:11:05.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.536+0000 7fccc025c740 10 client.18309 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:11:05.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.536+0000 7fccc025c740 20 client.18309 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:05.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.536+0000 7fccc025c740 1 --2- 172.21.15.120:0/3806905673 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55abfc60cca0 0x55abfc60f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:05.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.536+0000 7fccc025c740 4 client.18309 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:11:05.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.536+0000 7fccc025c740 1 -- 172.21.15.120:0/3806905673 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55abfc4f4d70 con 0x55abfc60cca0 2024-09-19T09:11:05.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.537+0000 7fccb77fe640 1 --2- 172.21.15.120:0/3806905673 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55abfc60cca0 0x55abfc60f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:05.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.537+0000 7fccb77fe640 1 --2- 172.21.15.120:0/3806905673 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55abfc60cca0 0x55abfc60f0c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f672301b2ba3c961 server_cookie=d7a74770efece75c in_seq=0 out_seq=0 2024-09-19T09:11:05.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.537+0000 7fcc96ffd640 10 client.18309 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:11:05.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.537+0000 7fcc96ffd640 1 -- 172.21.15.120:0/3806905673 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55abfc4f4d70 con 0x55abfc60cca0 2024-09-19T09:11:05.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.537+0000 7fcc96ffd640 10 client.18309 handle_command_reply: tid=0 2024-09-19T09:11:05.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.537+0000 7fccc025c740 1 client.18309 shutdown 2024-09-19T09:11:05.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.537+0000 7fccc025c740 1 -- 172.21.15.120:0/3806905673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abfc592370 msgr2=0x55abfc58c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:05.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.537+0000 7fccc025c740 1 --2- 172.21.15.120:0/3806905673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abfc592370 0x55abfc58c0a0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x55abfc588070 tx=0x7fccb0013c00 comp rx=0 tx=0).stop 2024-09-19T09:11:05.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.537+0000 7fccc025c740 1 -- 172.21.15.120:0/3806905673 shutdown_connections 2024-09-19T09:11:05.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.537+0000 7fccc025c740 1 --2- 172.21.15.120:0/3806905673 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55abfc60cca0 0x55abfc60f0c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.537+0000 7fccc025c740 1 --2- 172.21.15.120:0/3806905673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abfc592370 0x55abfc58c0a0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.537+0000 7fccc025c740 1 --2- 172.21.15.120:0/3806905673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abfc588b40 0x55abfc58bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.537+0000 7fcc96ffd640 0 client.18309 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:11:05.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.537+0000 7fccc025c740 1 --2- 172.21.15.120:0/3806905673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abfc5875d0 0x55abfc59a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.537+0000 7fccc025c740 1 -- 172.21.15.120:0/3806905673 >> 172.21.15.120:0/3806905673 conn(0x55abfc550310 msgr2=0x55abfc582d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:05.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.538+0000 7fccc025c740 1 -- 172.21.15.120:0/3806905673 shutdown_connections 2024-09-19T09:11:05.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.538+0000 7fccc025c740 1 -- 172.21.15.120:0/3806905673 wait complete. 2024-09-19T09:11:05.616 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:11:05.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.538+0000 7fccc025c740 20 client.18309 trim_cache size 0 max 16384 2024-09-19T09:11:05.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.538+0000 7fccbde83640 1 -- 172.21.15.120:0/2855087427 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcca00a3240 msgr2=0x7fcca00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:05.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.538+0000 7fccbde83640 1 --2- 172.21.15.120:0/2855087427 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcca00a3240 0x7fcca00a5700 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009230 tx=0x7fcca8005e50 comp rx=0 tx=0).stop 2024-09-19T09:11:05.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.538+0000 7fccbde83640 1 -- 172.21.15.120:0/2855087427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccb8102570 msgr2=0x7fccb8198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:05.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.538+0000 7fccbde83640 1 --2- 172.21.15.120:0/2855087427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccb8102570 0x7fccb8198a70 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fccac00e9f0 tx=0x7fccac00edf0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.538+0000 7fccbde83640 1 -- 172.21.15.120:0/2855087427 shutdown_connections 2024-09-19T09:11:05.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.538+0000 7fccbde83640 1 --2- 172.21.15.120:0/2855087427 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcca00a3240 0x7fcca00a5700 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.538+0000 7fccbde83640 1 --2- 172.21.15.120:0/2855087427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccb8102ef0 0x7fccb819ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.538+0000 7fccbde83640 1 --2- 172.21.15.120:0/2855087427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccb8102570 0x7fccb8198a70 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.538+0000 7fccbde83640 1 --2- 172.21.15.120:0/2855087427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccb8100e20 0x7fccb8198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:05.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.538+0000 7fccbde83640 1 -- 172.21.15.120:0/2855087427 >> 172.21.15.120:0/2855087427 conn(0x7fccb80fc7d0 msgr2=0x7fccb806a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:05.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.538+0000 7fccbde83640 1 -- 172.21.15.120:0/2855087427 shutdown_connections 2024-09-19T09:11:05.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:05.538+0000 7fccbde83640 1 -- 172.21.15.120:0/2855087427 wait complete. 2024-09-19T09:11:06.062 INFO:tasks.workunit.client.0.smithi120.stdout: 1 713844 8.77 MB/sec execute 301 sec latency 21.004 ms 2024-09-19T09:11:06.476 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:11:06.750 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:11:07.060 INFO:tasks.workunit.client.0.smithi120.stdout: 1 716659 8.78 MB/sec execute 302 sec latency 14.681 ms 2024-09-19T09:11:07.136 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:06 smithi120 ceph-mon[31150]: pgmap v589: 97 pgs: 97 active+clean; 31 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 643 KiB/s wr, 25 op/s 2024-09-19T09:11:07.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:06 smithi196 ceph-mon[35150]: pgmap v589: 97 pgs: 97 active+clean; 31 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 643 KiB/s wr, 25 op/s 2024-09-19T09:11:07.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:06 smithi153 ceph-mon[35071]: pgmap v589: 97 pgs: 97 active+clean; 31 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 643 KiB/s wr, 25 op/s 2024-09-19T09:11:07.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.402+0000 7f2df4725640 1 -- 172.21.15.120:0/1314893783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dec100e20 msgr2=0x7f2dec101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:07.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.402+0000 7f2df4725640 1 --2- 172.21.15.120:0/1314893783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dec100e20 0x7f2dec101200 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f2de8009900 tx=0x7f2de802a480 comp rx=0 tx=0).stop 2024-09-19T09:11:07.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.403+0000 7f2df4725640 1 -- 172.21.15.120:0/1314893783 shutdown_connections 2024-09-19T09:11:07.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.403+0000 7f2df4725640 1 --2- 172.21.15.120:0/1314893783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2dec102ef0 0x7f2dec10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.403+0000 7f2df4725640 1 --2- 172.21.15.120:0/1314893783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dec102570 0x7f2dec1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.403+0000 7f2df4725640 1 --2- 172.21.15.120:0/1314893783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dec100e20 0x7f2dec101200 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.403+0000 7f2df4725640 1 -- 172.21.15.120:0/1314893783 >> 172.21.15.120:0/1314893783 conn(0x7f2dec0fc7d0 msgr2=0x7f2dec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:07.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.403+0000 7f2df4725640 1 -- 172.21.15.120:0/1314893783 shutdown_connections 2024-09-19T09:11:07.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.403+0000 7f2df4725640 1 -- 172.21.15.120:0/1314893783 wait complete. 2024-09-19T09:11:07.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.403+0000 7f2df4725640 1 Processor -- start 2024-09-19T09:11:07.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.403+0000 7f2df4725640 1 -- start start 2024-09-19T09:11:07.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.403+0000 7f2df4725640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dec100e20 0x7f2dec198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:07.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.403+0000 7f2df4725640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2dec102570 0x7f2dec198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:07.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.403+0000 7f2df4725640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dec102ef0 0x7f2dec19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:07.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.403+0000 7f2df4725640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec110f80 con 0x7f2dec102ef0 2024-09-19T09:11:07.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.403+0000 7f2df4725640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec110e00 con 0x7f2dec100e20 2024-09-19T09:11:07.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.403+0000 7f2df4725640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec111100 con 0x7f2dec102570 2024-09-19T09:11:07.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.404+0000 7f2df2cc2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dec102ef0 0x7f2dec19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:07.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.404+0000 7f2df24c1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dec100e20 0x7f2dec198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:07.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.404+0000 7f2df2cc2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dec102ef0 0x7f2dec19cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52258/0 (socket says 172.21.15.120:52258) 2024-09-19T09:11:07.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.404+0000 7f2df24c1640 1 --2- 172.21.15.120:0/2099136488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dec100e20 0x7f2dec198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:33798/0 (socket says 172.21.15.120:33798) 2024-09-19T09:11:07.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.404+0000 7f2df2cc2640 1 -- 172.21.15.120:0/2099136488 learned_addr learned my addr 172.21.15.120:0/2099136488 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:07.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.404+0000 7f2df1cc0640 1 --2- 172.21.15.120:0/2099136488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2dec102570 0x7f2dec198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:07.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.404+0000 7f2df2cc2640 1 -- 172.21.15.120:0/2099136488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2dec102570 msgr2=0x7f2dec198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:07.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.404+0000 7f2df2cc2640 1 --2- 172.21.15.120:0/2099136488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2dec102570 0x7f2dec198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.404+0000 7f2df2cc2640 1 -- 172.21.15.120:0/2099136488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dec100e20 msgr2=0x7f2dec198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:07.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.404+0000 7f2df2cc2640 1 --2- 172.21.15.120:0/2099136488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dec100e20 0x7f2dec198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.404+0000 7f2df2cc2640 1 -- 172.21.15.120:0/2099136488 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dec19d4b0 con 0x7f2dec102ef0 2024-09-19T09:11:07.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.404+0000 7f2df24c1640 1 --2- 172.21.15.120:0/2099136488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dec100e20 0x7f2dec198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:07.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.404+0000 7f2df2cc2640 1 --2- 172.21.15.120:0/2099136488 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dec102ef0 0x7f2dec19cdd0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f2de400eb10 tx=0x7f2de400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:07.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.404+0000 7f2de37fe640 1 -- 172.21.15.120:0/2099136488 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de4004b20 con 0x7f2dec102ef0 2024-09-19T09:11:07.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.404+0000 7f2de37fe640 1 -- 172.21.15.120:0/2099136488 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de4004cc0 con 0x7f2dec102ef0 2024-09-19T09:11:07.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.404+0000 7f2df4725640 1 -- 172.21.15.120:0/2099136488 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dec19d7a0 con 0x7f2dec102ef0 2024-09-19T09:11:07.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.404+0000 7f2de37fe640 1 -- 172.21.15.120:0/2099136488 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de4010760 con 0x7f2dec102ef0 2024-09-19T09:11:07.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.404+0000 7f2df4725640 1 -- 172.21.15.120:0/2099136488 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dec1a5080 con 0x7f2dec102ef0 2024-09-19T09:11:07.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.407+0000 7f2de37fe640 1 -- 172.21.15.120:0/2099136488 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f2de4005020 con 0x7f2dec102ef0 2024-09-19T09:11:07.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.407+0000 7f2df6afe740 1 Processor -- start 2024-09-19T09:11:07.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.407+0000 7f2df6afe740 1 -- start start 2024-09-19T09:11:07.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.407+0000 7f2df6afe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d883bfdb40 0x55d883bfc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:07.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.407+0000 7f2df6afe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d883bfc5d0 0x55d883bfca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:07.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.407+0000 7f2df6afe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d883c07370 0x55d883c09760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:07.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.407+0000 7f2df6afe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d883b622f0 con 0x55d883bfc5d0 2024-09-19T09:11:07.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.407+0000 7f2df6afe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d883b09cf0 con 0x55d883bfdb40 2024-09-19T09:11:07.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.407+0000 7f2df6afe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d883b5e2b0 con 0x55d883c07370 2024-09-19T09:11:07.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.407+0000 7f2df24c1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d883bfc5d0 0x55d883bfca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:07.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.407+0000 7f2df1cc0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d883bfdb40 0x55d883bfc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:07.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.407+0000 7f2df24c1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d883bfc5d0 0x55d883bfca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52262/0 (socket says 172.21.15.120:52262) 2024-09-19T09:11:07.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.408+0000 7f2df1cc0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d883bfdb40 0x55d883bfc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:33804/0 (socket says 172.21.15.120:33804) 2024-09-19T09:11:07.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.408+0000 7f2df24c1640 1 -- 172.21.15.120:0/281783160 learned_addr learned my addr 172.21.15.120:0/281783160 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:07.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.408+0000 7f2df1cc0640 1 --2- 172.21.15.120:0/281783160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d883c07370 0x55d883c09760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:07.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.408+0000 7f2de37fe640 1 --2- 172.21.15.120:0/2099136488 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2dd80a32c0 0x7f2dd80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:07.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.408+0000 7f2df1cc0640 1 -- 172.21.15.120:0/281783160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d883c07370 msgr2=0x55d883c09760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:07.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.408+0000 7f2df1cc0640 1 --2- 172.21.15.120:0/281783160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d883c07370 0x55d883c09760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.408+0000 7f2df1cc0640 1 -- 172.21.15.120:0/281783160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d883bfc5d0 msgr2=0x55d883bfca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:07.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.408+0000 7f2df1cc0640 1 --2- 172.21.15.120:0/281783160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d883bfc5d0 0x55d883bfca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.408+0000 7f2df1cc0640 1 -- 172.21.15.120:0/281783160 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d883aee3a0 con 0x55d883bfdb40 2024-09-19T09:11:07.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.408+0000 7f2df2cc2640 1 --2- 172.21.15.120:0/2099136488 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2dd80a32c0 0x7f2dd80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:07.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.408+0000 7f2de37fe640 1 -- 172.21.15.120:0/2099136488 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f2de40cff30 con 0x7f2dec102ef0 2024-09-19T09:11:07.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.408+0000 7f2df1cc0640 1 --2- 172.21.15.120:0/281783160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d883bfdb40 0x55d883bfc090 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc009730 tx=0x7f2ddc0096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a8d238185f836e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:07.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.408+0000 7f2de0ff9640 1 -- 172.21.15.120:0/281783160 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc004250 con 0x55d883bfdb40 2024-09-19T09:11:07.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.408+0000 7f2df2cc2640 1 --2- 172.21.15.120:0/2099136488 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2dd80a32c0 0x7f2dd80a5780 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2de40049d0 tx=0x7f2de401bb30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:07.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.408+0000 7f2de0ff9640 1 -- 172.21.15.120:0/281783160 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ddc0043f0 con 0x55d883bfdb40 2024-09-19T09:11:07.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.408+0000 7f2de0ff9640 1 -- 172.21.15.120:0/281783160 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc01b7c0 con 0x55d883bfdb40 2024-09-19T09:11:07.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.409+0000 7f2df6afe740 1 -- 172.21.15.120:0/281783160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d883bfdb40 msgr2=0x55d883bfc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:07.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.409+0000 7f2df6afe740 1 --2- 172.21.15.120:0/281783160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d883bfdb40 0x55d883bfc090 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc009730 tx=0x7f2ddc0096c0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.409+0000 7f2df6afe740 1 -- 172.21.15.120:0/281783160 shutdown_connections 2024-09-19T09:11:07.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.409+0000 7f2df6afe740 1 --2- 172.21.15.120:0/281783160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d883c07370 0x55d883c09760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.409+0000 7f2df6afe740 1 --2- 172.21.15.120:0/281783160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d883bfc5d0 0x55d883bfca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.409+0000 7f2df6afe740 1 --2- 172.21.15.120:0/281783160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d883bfdb40 0x55d883bfc090 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.409+0000 7f2df6afe740 1 -- 172.21.15.120:0/281783160 >> 172.21.15.120:0/281783160 conn(0x55d883bc5310 msgr2=0x55d883bf8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:07.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.409+0000 7f2df6afe740 1 -- 172.21.15.120:0/281783160 shutdown_connections 2024-09-19T09:11:07.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.409+0000 7f2df6afe740 1 -- 172.21.15.120:0/281783160 wait complete. 2024-09-19T09:11:07.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.409+0000 7f2df6afe740 1 -- start start 2024-09-19T09:11:07.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.410+0000 7f2df6afe740 1 Processor -- start 2024-09-19T09:11:07.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.410+0000 7f2df6afe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d883bfc5d0 0x55d883c06d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:07.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.410+0000 7f2df6afe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d883bfdb40 0x55d883c04010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:07.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.410+0000 7f2df6afe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d883c07370 0x55d883c04600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:07.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.410+0000 7f2df6afe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d883c0a070 con 0x55d883c07370 2024-09-19T09:11:07.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.410+0000 7f2df6afe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d883b1d060 con 0x55d883bfdb40 2024-09-19T09:11:07.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.410+0000 7f2df6afe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d883c0a1f0 con 0x55d883bfc5d0 2024-09-19T09:11:07.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.410+0000 7f2df6afe740 10 client.0 _ll_register_callbacks cb 0x55d8838708c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:07.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.410+0000 7f2df1cc0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d883c07370 0x55d883c04600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:07.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.410+0000 7f2df24c1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d883bfdb40 0x55d883c04010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:07.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.410+0000 7f2df1cc0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d883c07370 0x55d883c04600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52270/0 (socket says 172.21.15.120:52270) 2024-09-19T09:11:07.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.410+0000 7f2df1cc0640 1 -- 172.21.15.120:0/1814368943 learned_addr learned my addr 172.21.15.120:0/1814368943 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:07.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.410+0000 7f2df1cc0640 1 --2- 172.21.15.120:0/1814368943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d883bfc5d0 0x55d883c06d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:07.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.410+0000 7f2df24c1640 1 -- 172.21.15.120:0/1814368943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d883bfc5d0 msgr2=0x55d883c06d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:07.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.410+0000 7f2df24c1640 1 --2- 172.21.15.120:0/1814368943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d883bfc5d0 0x55d883c06d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.410+0000 7f2df24c1640 1 -- 172.21.15.120:0/1814368943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d883c07370 msgr2=0x55d883c04600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:07.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.410+0000 7f2df24c1640 1 --2- 172.21.15.120:0/1814368943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d883c07370 0x55d883c04600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.410+0000 7f2df24c1640 1 -- 172.21.15.120:0/1814368943 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8838d33c0 con 0x55d883bfdb40 2024-09-19T09:11:07.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.410+0000 7f2df1cc0640 1 --2- 172.21.15.120:0/1814368943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d883c07370 0x55d883c04600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:07.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.410+0000 7f2df24c1640 1 --2- 172.21.15.120:0/1814368943 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d883bfdb40 0x55d883c04010 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55d883bf9ac0 tx=0x7f2de80035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0df3fcf5e3369b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:07.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.410+0000 7f2dd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:11:07.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.411+0000 7f2dd5ffb640 1 -- 172.21.15.120:0/1814368943 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8035040 con 0x55d883bfdb40 2024-09-19T09:11:07.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.411+0000 7f2dd5ffb640 1 -- 172.21.15.120:0/1814368943 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f2de8003cb0 con 0x55d883bfdb40 2024-09-19T09:11:07.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.411+0000 7f2df6afe740 1 -- 172.21.15.120:0/1814368943 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ddc0043f0 con 0x55d883bfdb40 2024-09-19T09:11:07.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.411+0000 7f2dd5ffb640 1 -- 172.21.15.120:0/1814368943 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2de8005020 con 0x55d883bfdb40 2024-09-19T09:11:07.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.411+0000 7f2df6afe740 10 client.27452 fetch_fsmap learned FSMap version 30 2024-09-19T09:11:07.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.411+0000 7f2df6afe740 1 -- 172.21.15.120:0/1814368943 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d883c15b10 con 0x55d883bfdb40 2024-09-19T09:11:07.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.411+0000 7f2dd5ffb640 1 -- 172.21.15.120:0/1814368943 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ddc0043f0 con 0x55d883bfdb40 2024-09-19T09:11:07.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.411+0000 7f2dd5ffb640 1 -- 172.21.15.120:0/1814368943 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2de8031540 con 0x55d883bfdb40 2024-09-19T09:11:07.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.411+0000 7f2df6afe740 10 client.27452 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:11:07.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.411+0000 7f2df6afe740 10 client.27452 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:11:07.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.411+0000 7f2df6afe740 20 client.27452 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:07.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.411+0000 7f2df6afe740 1 --2- 172.21.15.120:0/1814368943 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d883c17020 0x55d883c19410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:07.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.411+0000 7f2df6afe740 4 client.27452 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:11:07.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.411+0000 7f2df6afe740 1 -- 172.21.15.120:0/1814368943 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d883b6f860 con 0x55d883c17020 2024-09-19T09:11:07.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.411+0000 7f2df1cc0640 1 --2- 172.21.15.120:0/1814368943 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d883c17020 0x55d883c19410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:07.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.412+0000 7f2dd5ffb640 10 client.27452 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:07.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.412+0000 7f2df1cc0640 1 --2- 172.21.15.120:0/1814368943 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d883c17020 0x55d883c19410 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b53c6c254ed13a6f server_cookie=73f42e0af826af5d in_seq=0 out_seq=0 2024-09-19T09:11:07.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.412+0000 7f2dd5ffb640 1 -- 172.21.15.120:0/1814368943 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d883b6f860 con 0x55d883c17020 2024-09-19T09:11:07.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.412+0000 7f2dd5ffb640 10 client.27452 handle_command_reply: tid=0 2024-09-19T09:11:07.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.412+0000 7f2df6afe740 1 client.27452 shutdown 2024-09-19T09:11:07.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.412+0000 7f2df6afe740 1 -- 172.21.15.120:0/1814368943 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d883bfdb40 msgr2=0x55d883c04010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:07.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.412+0000 7f2df6afe740 1 --2- 172.21.15.120:0/1814368943 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d883bfdb40 0x55d883c04010 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55d883bf9ac0 tx=0x7f2de80035f0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.413+0000 7f2df6afe740 1 -- 172.21.15.120:0/1814368943 shutdown_connections 2024-09-19T09:11:07.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.413+0000 7f2df6afe740 1 --2- 172.21.15.120:0/1814368943 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d883c17020 0x55d883c19410 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.413+0000 7f2df6afe740 1 --2- 172.21.15.120:0/1814368943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d883c07370 0x55d883c04600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.413+0000 7f2dd5ffb640 0 client.27452 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:07.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.413+0000 7f2df6afe740 1 --2- 172.21.15.120:0/1814368943 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d883bfdb40 0x55d883c04010 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.413+0000 7f2df6afe740 1 --2- 172.21.15.120:0/1814368943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d883bfc5d0 0x55d883c06d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.413+0000 7f2df6afe740 1 -- 172.21.15.120:0/1814368943 >> 172.21.15.120:0/1814368943 conn(0x55d883bc5310 msgr2=0x55d883bfe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:07.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.413+0000 7f2df6afe740 1 -- 172.21.15.120:0/1814368943 shutdown_connections 2024-09-19T09:11:07.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.413+0000 7f2df6afe740 1 -- 172.21.15.120:0/1814368943 wait complete. 2024-09-19T09:11:07.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.413+0000 7f2df6afe740 20 client.27452 trim_cache size 0 max 16384 2024-09-19T09:11:07.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.426+0000 7f2df6afe740 1 Processor -- start 2024-09-19T09:11:07.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.426+0000 7f2df6afe740 1 -- start start 2024-09-19T09:11:07.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.426+0000 7f2df6afe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d883bfc5d0 0x55d883c13310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:07.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.426+0000 7f2df6afe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d883bfdb40 0x55d883c02ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:07.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.426+0000 7f2df6afe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d883c07370 0x55d883c1ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:07.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.426+0000 7f2df6afe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1ad640 con 0x55d883bfdb40 2024-09-19T09:11:07.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.426+0000 7f2df6afe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1af180 con 0x55d883c07370 2024-09-19T09:11:07.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.426+0000 7f2df6afe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1add10 con 0x55d883bfc5d0 2024-09-19T09:11:07.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.426+0000 7f2df24c1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d883bfdb40 0x55d883c02ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:07.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.426+0000 7f2df1cc0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d883bfc5d0 0x55d883c13310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:07.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.426+0000 7f2df24c1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d883bfdb40 0x55d883c02ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52272/0 (socket says 172.21.15.120:52272) 2024-09-19T09:11:07.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.426+0000 7f2df24c1640 1 -- 172.21.15.120:0/526060986 learned_addr learned my addr 172.21.15.120:0/526060986 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:07.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.426+0000 7f2df1cc0640 1 --2- 172.21.15.120:0/526060986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d883c07370 0x55d883c1ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:07.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.426+0000 7f2df24c1640 1 -- 172.21.15.120:0/526060986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d883bfc5d0 msgr2=0x55d883c13310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:07.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.426+0000 7f2df24c1640 1 --2- 172.21.15.120:0/526060986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d883bfc5d0 0x55d883c13310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.426+0000 7f2df24c1640 1 -- 172.21.15.120:0/526060986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d883c07370 msgr2=0x55d883c1ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:07.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.426+0000 7f2df24c1640 1 --2- 172.21.15.120:0/526060986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d883c07370 0x55d883c1ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.426+0000 7f2df24c1640 1 -- 172.21.15.120:0/526060986 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d883c04970 con 0x55d883bfdb40 2024-09-19T09:11:07.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.427+0000 7f2df24c1640 1 --2- 172.21.15.120:0/526060986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d883bfdb40 0x55d883c02ff0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f2de8031940 tx=0x7f2de8032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1cdd36b18b19117 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:07.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.427+0000 7f2dd7fff640 1 -- 172.21.15.120:0/526060986 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8053020 con 0x55d883bfdb40 2024-09-19T09:11:07.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.427+0000 7f2dd7fff640 1 -- 172.21.15.120:0/526060986 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de8031740 con 0x55d883bfdb40 2024-09-19T09:11:07.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.427+0000 7f2dd7fff640 1 -- 172.21.15.120:0/526060986 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8035070 con 0x55d883bfdb40 2024-09-19T09:11:07.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.428+0000 7f2df6afe740 1 -- 172.21.15.120:0/526060986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d883bfdb40 msgr2=0x55d883c02ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:07.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.428+0000 7f2df6afe740 1 --2- 172.21.15.120:0/526060986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d883bfdb40 0x55d883c02ff0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f2de8031940 tx=0x7f2de8032fc0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.428+0000 7f2df6afe740 1 -- 172.21.15.120:0/526060986 shutdown_connections 2024-09-19T09:11:07.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.428+0000 7f2df6afe740 1 --2- 172.21.15.120:0/526060986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d883c07370 0x55d883c1ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.428+0000 7f2df6afe740 1 --2- 172.21.15.120:0/526060986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d883bfdb40 0x55d883c02ff0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.428+0000 7f2df6afe740 1 --2- 172.21.15.120:0/526060986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d883bfc5d0 0x55d883c13310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.428+0000 7f2df6afe740 1 -- 172.21.15.120:0/526060986 >> 172.21.15.120:0/526060986 conn(0x55d883bc5310 msgr2=0x55d883c028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:07.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.428+0000 7f2df6afe740 1 -- 172.21.15.120:0/526060986 shutdown_connections 2024-09-19T09:11:07.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.428+0000 7f2df6afe740 1 -- 172.21.15.120:0/526060986 wait complete. 2024-09-19T09:11:07.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.428+0000 7f2df6afe740 1 -- start start 2024-09-19T09:11:07.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.428+0000 7f2df6afe740 1 Processor -- start 2024-09-19T09:11:07.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.428+0000 7f2df6afe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d883bfc5d0 0x55d883c0f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:07.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.428+0000 7f2df6afe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d883bfdb40 0x55d883c0acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:07.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.428+0000 7f2df6afe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d883c07370 0x55d883c0b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:07.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.428+0000 7f2df6afe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1aeab0 con 0x55d883c07370 2024-09-19T09:11:07.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.428+0000 7f2df6afe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1ae3e0 con 0x55d883bfdb40 2024-09-19T09:11:07.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.428+0000 7f2df6afe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1ab600 con 0x55d883bfc5d0 2024-09-19T09:11:07.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.429+0000 7f2df6afe740 10 client.0 _ll_register_callbacks cb 0x7f2de8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:07.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.429+0000 7f2df1cc0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d883bfc5d0 0x55d883c0f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:07.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.429+0000 7f2df24c1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d883bfdb40 0x55d883c0acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:07.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.429+0000 7f2df24c1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d883bfdb40 0x55d883c0acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:33826/0 (socket says 172.21.15.120:33826) 2024-09-19T09:11:07.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.429+0000 7f2df24c1640 1 -- 172.21.15.120:0/2191118305 learned_addr learned my addr 172.21.15.120:0/2191118305 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:07.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.429+0000 7f2df1cc0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d883bfc5d0 0x55d883c0f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:32800/0 (socket says 172.21.15.120:32800) 2024-09-19T09:11:07.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.429+0000 7f2df1cc0640 1 --2- 172.21.15.120:0/2191118305 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d883c07370 0x55d883c0b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:07.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.429+0000 7f2df1cc0640 1 -- 172.21.15.120:0/2191118305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d883bfdb40 msgr2=0x55d883c0acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:07.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.429+0000 7f2df1cc0640 1 --2- 172.21.15.120:0/2191118305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d883bfdb40 0x55d883c0acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.429+0000 7f2df1cc0640 1 -- 172.21.15.120:0/2191118305 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d883c07370 msgr2=0x55d883c0b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:07.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.429+0000 7f2df1cc0640 1 --2- 172.21.15.120:0/2191118305 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d883c07370 0x55d883c0b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.429+0000 7f2df1cc0640 1 -- 172.21.15.120:0/2191118305 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d883c0b7c0 con 0x55d883bfc5d0 2024-09-19T09:11:07.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.429+0000 7f2df24c1640 1 --2- 172.21.15.120:0/2191118305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d883bfdb40 0x55d883c0acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:07.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.429+0000 7f2df1cc0640 1 --2- 172.21.15.120:0/2191118305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d883bfc5d0 0x55d883c0f990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x55d883c0a340 tx=0x7f2de8005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4341e366286a3912 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:07.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.429+0000 7f2dd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:11:07.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.429+0000 7f2dd5ffb640 1 -- 172.21.15.120:0/2191118305 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc005020 con 0x55d883bfc5d0 2024-09-19T09:11:07.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.429+0000 7f2df6afe740 1 -- 172.21.15.120:0/2191118305 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2de8031740 con 0x55d883bfc5d0 2024-09-19T09:11:07.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.429+0000 7f2dd5ffb640 1 -- 172.21.15.120:0/2191118305 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f2ddc036040 con 0x55d883bfc5d0 2024-09-19T09:11:07.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.429+0000 7f2dd5ffb640 1 -- 172.21.15.120:0/2191118305 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2ddc003850 con 0x55d883bfc5d0 2024-09-19T09:11:07.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.430+0000 7f2df6afe740 10 client.27334 fetch_fsmap learned FSMap version 30 2024-09-19T09:11:07.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.430+0000 7f2df6afe740 1 -- 172.21.15.120:0/2191118305 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d883c81ac0 con 0x55d883bfc5d0 2024-09-19T09:11:07.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.430+0000 7f2dd5ffb640 1 -- 172.21.15.120:0/2191118305 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de8031740 con 0x55d883bfc5d0 2024-09-19T09:11:07.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.430+0000 7f2dd5ffb640 1 -- 172.21.15.120:0/2191118305 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2ddc01bc10 con 0x55d883bfc5d0 2024-09-19T09:11:07.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.430+0000 7f2df6afe740 10 client.27334 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:11:07.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.430+0000 7f2df6afe740 10 client.27334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:11:07.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.430+0000 7f2df6afe740 20 client.27334 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:07.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.430+0000 7f2df6afe740 1 --2- 172.21.15.120:0/2191118305 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d883c82050 0x55d883c84440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:07.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.430+0000 7f2df6afe740 4 client.27334 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:11:07.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.430+0000 7f2df6afe740 1 -- 172.21.15.120:0/2191118305 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d883b69d70 con 0x55d883c82050 2024-09-19T09:11:07.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.430+0000 7f2df24c1640 1 --2- 172.21.15.120:0/2191118305 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d883c82050 0x55d883c84440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:07.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.430+0000 7f2df24c1640 1 --2- 172.21.15.120:0/2191118305 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d883c82050 0x55d883c84440 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=458f0419ca7189d1 server_cookie=ac3021a567242827 in_seq=0 out_seq=0 2024-09-19T09:11:07.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.430+0000 7f2dd5ffb640 10 client.27334 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:07.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.431+0000 7f2dd5ffb640 1 -- 172.21.15.120:0/2191118305 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d883b69d70 con 0x55d883c82050 2024-09-19T09:11:07.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.431+0000 7f2dd5ffb640 10 client.27334 handle_command_reply: tid=0 2024-09-19T09:11:07.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.431+0000 7f2df6afe740 1 client.27334 shutdown 2024-09-19T09:11:07.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.431+0000 7f2df6afe740 1 -- 172.21.15.120:0/2191118305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d883bfc5d0 msgr2=0x55d883c0f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:07.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.431+0000 7f2df6afe740 1 --2- 172.21.15.120:0/2191118305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d883bfc5d0 0x55d883c0f990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x55d883c0a340 tx=0x7f2de8005d50 comp rx=0 tx=0).stop 2024-09-19T09:11:07.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.431+0000 7f2df6afe740 1 -- 172.21.15.120:0/2191118305 shutdown_connections 2024-09-19T09:11:07.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.431+0000 7f2df6afe740 1 --2- 172.21.15.120:0/2191118305 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d883c82050 0x55d883c84440 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.431+0000 7f2df6afe740 1 --2- 172.21.15.120:0/2191118305 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d883c07370 0x55d883c0b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.431+0000 7f2df6afe740 1 --2- 172.21.15.120:0/2191118305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d883bfdb40 0x55d883c0acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.431+0000 7f2dd5ffb640 0 client.27334 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:07.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.431+0000 7f2df6afe740 1 --2- 172.21.15.120:0/2191118305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d883bfc5d0 0x55d883c0f990 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.431+0000 7f2df6afe740 1 -- 172.21.15.120:0/2191118305 >> 172.21.15.120:0/2191118305 conn(0x55d883bc5310 msgr2=0x55d883bf74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:07.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.431+0000 7f2df6afe740 1 -- 172.21.15.120:0/2191118305 shutdown_connections 2024-09-19T09:11:07.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.431+0000 7f2df6afe740 1 -- 172.21.15.120:0/2191118305 wait complete. 2024-09-19T09:11:07.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.432+0000 7f2df6afe740 20 client.27334 trim_cache size 0 max 16384 2024-09-19T09:11:07.450 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:11:07.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.432+0000 7f2df4725640 1 -- 172.21.15.120:0/2099136488 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2dd80a32c0 msgr2=0x7f2dd80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:07.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.432+0000 7f2df4725640 1 --2- 172.21.15.120:0/2099136488 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2dd80a32c0 0x7f2dd80a5780 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2de40049d0 tx=0x7f2de401bb30 comp rx=0 tx=0).stop 2024-09-19T09:11:07.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.432+0000 7f2df4725640 1 -- 172.21.15.120:0/2099136488 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dec102ef0 msgr2=0x7f2dec19cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:07.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.432+0000 7f2df4725640 1 --2- 172.21.15.120:0/2099136488 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dec102ef0 0x7f2dec19cdd0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f2de400eb10 tx=0x7f2de400c2d0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.432+0000 7f2df4725640 1 -- 172.21.15.120:0/2099136488 shutdown_connections 2024-09-19T09:11:07.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.432+0000 7f2df4725640 1 --2- 172.21.15.120:0/2099136488 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2dd80a32c0 0x7f2dd80a5780 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.432+0000 7f2df4725640 1 --2- 172.21.15.120:0/2099136488 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dec102ef0 0x7f2dec19cdd0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.432+0000 7f2df4725640 1 --2- 172.21.15.120:0/2099136488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2dec102570 0x7f2dec198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.432+0000 7f2df4725640 1 --2- 172.21.15.120:0/2099136488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dec100e20 0x7f2dec198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:07.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.432+0000 7f2df4725640 1 -- 172.21.15.120:0/2099136488 >> 172.21.15.120:0/2099136488 conn(0x7f2dec0fc7d0 msgr2=0x7f2dec06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:07.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.432+0000 7f2df4725640 1 -- 172.21.15.120:0/2099136488 shutdown_connections 2024-09-19T09:11:07.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:07.432+0000 7f2df4725640 1 -- 172.21.15.120:0/2099136488 wait complete. 2024-09-19T09:11:08.060 INFO:tasks.workunit.client.0.smithi120.stdout: 1 719017 8.80 MB/sec execute 303 sec latency 23.793 ms 2024-09-19T09:11:09.060 INFO:tasks.workunit.client.0.smithi120.stdout: 1 721780 8.82 MB/sec execute 304 sec latency 13.678 ms 2024-09-19T09:11:09.159 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:11:09.160 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:11:09.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:08 smithi196 ceph-mon[35150]: pgmap v590: 97 pgs: 97 active+clean; 30 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 844 KiB/s wr, 39 op/s 2024-09-19T09:11:09.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:08 smithi153 ceph-mon[35071]: pgmap v590: 97 pgs: 97 active+clean; 30 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 844 KiB/s wr, 39 op/s 2024-09-19T09:11:09.188 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:08 smithi120 ceph-mon[31150]: pgmap v590: 97 pgs: 97 active+clean; 30 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 844 KiB/s wr, 39 op/s 2024-09-19T09:11:09.423 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:11:09.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.959+0000 7ffa557c3640 1 -- 172.21.15.120:0/1237822580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa50100a80 msgr2=0x7ffa50100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:09.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.959+0000 7ffa557c3640 1 --2- 172.21.15.120:0/1237822580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa50100a80 0x7ffa50100ee0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009930 tx=0x7ffa4402a420 comp rx=0 tx=0).stop 2024-09-19T09:11:09.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.959+0000 7ffa557c3640 1 -- 172.21.15.120:0/1237822580 shutdown_connections 2024-09-19T09:11:09.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.959+0000 7ffa557c3640 1 --2- 172.21.15.120:0/1237822580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa50101420 0x7ffa5010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:09.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.959+0000 7ffa557c3640 1 --2- 172.21.15.120:0/1237822580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa50100a80 0x7ffa50100ee0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:09.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.959+0000 7ffa557c3640 1 --2- 172.21.15.120:0/1237822580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa50106a80 0x7ffa50106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:09.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.959+0000 7ffa557c3640 1 -- 172.21.15.120:0/1237822580 >> 172.21.15.120:0/1237822580 conn(0x7ffa500fc7a0 msgr2=0x7ffa500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:09.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.959+0000 7ffa557c3640 1 -- 172.21.15.120:0/1237822580 shutdown_connections 2024-09-19T09:11:09.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.959+0000 7ffa557c3640 1 -- 172.21.15.120:0/1237822580 wait complete. 2024-09-19T09:11:09.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.959+0000 7ffa557c3640 1 Processor -- start 2024-09-19T09:11:09.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.960+0000 7ffa557c3640 1 -- start start 2024-09-19T09:11:09.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.960+0000 7ffa557c3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa50100a80 0x7ffa50199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:09.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.960+0000 7ffa557c3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa50101420 0x7ffa50199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:09.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.960+0000 7ffa557c3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa50106a80 0x7ffa501a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:09.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.960+0000 7ffa557c3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa50110e80 con 0x7ffa50106a80 2024-09-19T09:11:09.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.960+0000 7ffa557c3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffa50110d00 con 0x7ffa50100a80 2024-09-19T09:11:09.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.960+0000 7ffa557c3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffa50111000 con 0x7ffa50101420 2024-09-19T09:11:09.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.960+0000 7ffa4f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa50106a80 0x7ffa501a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:09.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.960+0000 7ffa4effd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa50100a80 0x7ffa50199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:09.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.960+0000 7ffa4effd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa50100a80 0x7ffa50199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:33834/0 (socket says 172.21.15.120:33834) 2024-09-19T09:11:09.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.960+0000 7ffa4effd640 1 -- 172.21.15.120:0/3449170221 learned_addr learned my addr 172.21.15.120:0/3449170221 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:09.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.960+0000 7ffa4e7fc640 1 --2- 172.21.15.120:0/3449170221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa50101420 0x7ffa50199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:09.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.960+0000 7ffa4f7fe640 1 -- 172.21.15.120:0/3449170221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa50101420 msgr2=0x7ffa50199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:09.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.960+0000 7ffa4f7fe640 1 --2- 172.21.15.120:0/3449170221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa50101420 0x7ffa50199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:09.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.960+0000 7ffa4f7fe640 1 -- 172.21.15.120:0/3449170221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa50100a80 msgr2=0x7ffa50199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:09.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.960+0000 7ffa4f7fe640 1 --2- 172.21.15.120:0/3449170221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa50100a80 0x7ffa50199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:09.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.960+0000 7ffa4f7fe640 1 -- 172.21.15.120:0/3449170221 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa501a48d0 con 0x7ffa50106a80 2024-09-19T09:11:09.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.960+0000 7ffa4e7fc640 1 --2- 172.21.15.120:0/3449170221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa50101420 0x7ffa50199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:11:09.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.960+0000 7ffa4f7fe640 1 --2- 172.21.15.120:0/3449170221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa50106a80 0x7ffa501a4390 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800da10 tx=0x7ffa4800ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:09.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.961+0000 7ffa37fff640 1 -- 172.21.15.120:0/3449170221 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4800bd70 con 0x7ffa50106a80 2024-09-19T09:11:09.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.961+0000 7ffa37fff640 1 -- 172.21.15.120:0/3449170221 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa48003e50 con 0x7ffa50106a80 2024-09-19T09:11:09.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.961+0000 7ffa557c3640 1 -- 172.21.15.120:0/3449170221 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa501a4aa0 con 0x7ffa50106a80 2024-09-19T09:11:09.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.961+0000 7ffa37fff640 1 -- 172.21.15.120:0/3449170221 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa480106a0 con 0x7ffa50106a80 2024-09-19T09:11:09.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.961+0000 7ffa557c3640 1 -- 172.21.15.120:0/3449170221 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa501a4fb0 con 0x7ffa50106a80 2024-09-19T09:11:09.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.963+0000 7ffa37fff640 1 -- 172.21.15.120:0/3449170221 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7ffa48003970 con 0x7ffa50106a80 2024-09-19T09:11:09.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.963+0000 7ffa57b9c740 1 Processor -- start 2024-09-19T09:11:09.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.963+0000 7ffa57b9c740 1 -- start start 2024-09-19T09:11:09.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.963+0000 7ffa37fff640 1 --2- 172.21.15.120:0/3449170221 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ffa380a32c0 0x7ffa380a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:09.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.963+0000 7ffa57b9c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aca435ba40 0x55aca435be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:09.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.964+0000 7ffa57b9c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca435c360 0x55aca436ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:09.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.964+0000 7ffa57b9c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca436b150 0x55aca436d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:09.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.964+0000 7ffa57b9c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55aca42c1280 con 0x55aca435c360 2024-09-19T09:11:09.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.964+0000 7ffa57b9c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55aca4268b10 con 0x55aca436b150 2024-09-19T09:11:09.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.964+0000 7ffa57b9c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55aca42bd240 con 0x55aca435ba40 2024-09-19T09:11:09.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.964+0000 7ffa4e7fc640 1 --2- 172.21.15.120:0/3449170221 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ffa380a32c0 0x7ffa380a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:09.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.964+0000 7ffa4effd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aca435ba40 0x55aca435be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:09.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.964+0000 7ffa4f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca436b150 0x55aca436d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:09.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.964+0000 7ffa4effd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aca435ba40 0x55aca435be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:32826/0 (socket says 172.21.15.120:32826) 2024-09-19T09:11:09.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.964+0000 7ffa4effd640 1 -- 172.21.15.120:0/270725316 learned_addr learned my addr 172.21.15.120:0/270725316 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:09.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.964+0000 7ffa4e7fc640 1 --2- 172.21.15.120:0/270725316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca435c360 0x55aca436ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:09.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.964+0000 7ffa37fff640 1 -- 172.21.15.120:0/3449170221 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7ffa480cfd20 con 0x7ffa50106a80 2024-09-19T09:11:09.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.964+0000 7ffa4f7fe640 1 -- 172.21.15.120:0/270725316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aca435ba40 msgr2=0x55aca435be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:09.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.964+0000 7ffa4f7fe640 1 --2- 172.21.15.120:0/270725316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aca435ba40 0x55aca435be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:09.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.964+0000 7ffa4f7fe640 1 -- 172.21.15.120:0/270725316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca435c360 msgr2=0x55aca436ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:09.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.964+0000 7ffa4f7fe640 1 --2- 172.21.15.120:0/270725316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca435c360 0x55aca436ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:09.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.964+0000 7ffa4f7fe640 1 -- 172.21.15.120:0/270725316 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aca42751f0 con 0x55aca436b150 2024-09-19T09:11:09.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.964+0000 7ffa4effd640 1 --2- 172.21.15.120:0/270725316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aca435ba40 0x55aca435be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:09.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.964+0000 7ffa4f7fe640 1 --2- 172.21.15.120:0/270725316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca436b150 0x55aca436d540 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ffa480cd820 tx=0x7ffa4809bca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee8df5351986c727 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:09.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.964+0000 7ffa4e7fc640 1 --2- 172.21.15.120:0/3449170221 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ffa380a32c0 0x7ffa380a5780 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ffa44032fd0 tx=0x7ffa44005dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:09.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.964+0000 7ffa357fa640 1 -- 172.21.15.120:0/270725316 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa48068040 con 0x55aca436b150 2024-09-19T09:11:09.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.964+0000 7ffa357fa640 1 -- 172.21.15.120:0/270725316 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa4809c340 con 0x55aca436b150 2024-09-19T09:11:09.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.965+0000 7ffa357fa640 1 -- 172.21.15.120:0/270725316 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa48004000 con 0x55aca436b150 2024-09-19T09:11:09.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.965+0000 7ffa57b9c740 1 -- 172.21.15.120:0/270725316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca436b150 msgr2=0x55aca436d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:09.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.965+0000 7ffa57b9c740 1 --2- 172.21.15.120:0/270725316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca436b150 0x55aca436d540 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ffa480cd820 tx=0x7ffa4809bca0 comp rx=0 tx=0).stop 2024-09-19T09:11:09.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.965+0000 7ffa57b9c740 1 -- 172.21.15.120:0/270725316 shutdown_connections 2024-09-19T09:11:09.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.965+0000 7ffa57b9c740 1 --2- 172.21.15.120:0/270725316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca436b150 0x55aca436d540 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:09.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.965+0000 7ffa57b9c740 1 --2- 172.21.15.120:0/270725316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca435c360 0x55aca436ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:09.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.965+0000 7ffa57b9c740 1 --2- 172.21.15.120:0/270725316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aca435ba40 0x55aca435be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:09.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.965+0000 7ffa57b9c740 1 -- 172.21.15.120:0/270725316 >> 172.21.15.120:0/270725316 conn(0x55aca43580f0 msgr2=0x55aca43584d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:09.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.965+0000 7ffa57b9c740 1 -- 172.21.15.120:0/270725316 shutdown_connections 2024-09-19T09:11:09.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.965+0000 7ffa57b9c740 1 -- 172.21.15.120:0/270725316 wait complete. 2024-09-19T09:11:09.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.966+0000 7ffa57b9c740 1 -- start start 2024-09-19T09:11:09.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.966+0000 7ffa57b9c740 1 Processor -- start 2024-09-19T09:11:09.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.966+0000 7ffa57b9c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aca435ba40 0x55aca43680c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:09.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.966+0000 7ffa57b9c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca435c360 0x55aca4368600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:09.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.966+0000 7ffa57b9c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca436b150 0x55aca4368b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:09.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.966+0000 7ffa57b9c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55aca436dd20 con 0x55aca435c360 2024-09-19T09:11:09.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.966+0000 7ffa57b9c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55aca427bf10 con 0x55aca436b150 2024-09-19T09:11:09.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.966+0000 7ffa57b9c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55aca436dea0 con 0x55aca435ba40 2024-09-19T09:11:09.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.966+0000 7ffa57b9c740 10 client.0 _ll_register_callbacks cb 0x55aca4244a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:09.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.966+0000 7ffa4e7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca435c360 0x55aca4368600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:09.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.966+0000 7ffa4effd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aca435ba40 0x55aca43680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:09.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.966+0000 7ffa4effd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aca435ba40 0x55aca43680c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:32830/0 (socket says 172.21.15.120:32830) 2024-09-19T09:11:09.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.966+0000 7ffa4effd640 1 -- 172.21.15.120:0/1651819221 learned_addr learned my addr 172.21.15.120:0/1651819221 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:09.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.966+0000 7ffa4f7fe640 1 --2- 172.21.15.120:0/1651819221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca436b150 0x55aca4368b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:09.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.966+0000 7ffa4e7fc640 1 -- 172.21.15.120:0/1651819221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aca435ba40 msgr2=0x55aca43680c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:09.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.966+0000 7ffa4e7fc640 1 --2- 172.21.15.120:0/1651819221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aca435ba40 0x55aca43680c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:09.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.966+0000 7ffa4e7fc640 1 -- 172.21.15.120:0/1651819221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca436b150 msgr2=0x55aca4368b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:09.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.966+0000 7ffa4e7fc640 1 --2- 172.21.15.120:0/1651819221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca436b150 0x55aca4368b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:09.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.966+0000 7ffa4e7fc640 1 -- 172.21.15.120:0/1651819221 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aca40326a0 con 0x55aca435c360 2024-09-19T09:11:09.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.966+0000 7ffa4effd640 1 --2- 172.21.15.120:0/1651819221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aca435ba40 0x55aca43680c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:09.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.966+0000 7ffa4f7fe640 1 --2- 172.21.15.120:0/1651819221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca436b150 0x55aca4368b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:09.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.967+0000 7ffa4e7fc640 1 --2- 172.21.15.120:0/1651819221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca435c360 0x55aca4368600 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7ffa5006b2e0 tx=0x7ffa44005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b9826c7f5f9f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:09.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.967+0000 7ffa32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:11:09.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.967+0000 7ffa32ff5640 1 -- 172.21.15.120:0/1651819221 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4404e020 con 0x55aca435c360 2024-09-19T09:11:09.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.967+0000 7ffa57b9c740 1 -- 172.21.15.120:0/1651819221 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa4809c340 con 0x55aca435c360 2024-09-19T09:11:09.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.967+0000 7ffa32ff5640 1 -- 172.21.15.120:0/1651819221 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7ffa44003ac0 con 0x55aca435c360 2024-09-19T09:11:09.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.967+0000 7ffa32ff5640 1 -- 172.21.15.120:0/1651819221 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffa44035040 con 0x55aca435c360 2024-09-19T09:11:09.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.967+0000 7ffa57b9c740 10 client.18339 fetch_fsmap learned FSMap version 30 2024-09-19T09:11:09.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.967+0000 7ffa57b9c740 1 -- 172.21.15.120:0/1651819221 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55aca43799f0 con 0x55aca435c360 2024-09-19T09:11:09.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.967+0000 7ffa32ff5640 1 -- 172.21.15.120:0/1651819221 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa4809c340 con 0x55aca435c360 2024-09-19T09:11:09.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.967+0000 7ffa32ff5640 1 -- 172.21.15.120:0/1651819221 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ffa4403a030 con 0x55aca435c360 2024-09-19T09:11:09.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.967+0000 7ffa57b9c740 10 client.18339 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:11:09.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.967+0000 7ffa57b9c740 10 client.18339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:11:09.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.968+0000 7ffa57b9c740 20 client.18339 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:09.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.968+0000 7ffa57b9c740 1 --2- 172.21.15.120:0/1651819221 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55aca437ae50 0x55aca437d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:09.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.968+0000 7ffa57b9c740 4 client.18339 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:11:09.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.968+0000 7ffa57b9c740 1 -- 172.21.15.120:0/1651819221 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aca42ce7f0 con 0x55aca437ae50 2024-09-19T09:11:09.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.968+0000 7ffa4effd640 1 --2- 172.21.15.120:0/1651819221 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55aca437ae50 0x55aca437d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:09.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.968+0000 7ffa32ff5640 10 client.18339 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:09.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.968+0000 7ffa4effd640 1 --2- 172.21.15.120:0/1651819221 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55aca437ae50 0x55aca437d270 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfbf9f489be98a0b server_cookie=6db0193ed1d441f8 in_seq=0 out_seq=0 2024-09-19T09:11:09.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.969+0000 7ffa32ff5640 1 -- 172.21.15.120:0/1651819221 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aca42ce7f0 con 0x55aca437ae50 2024-09-19T09:11:09.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.969+0000 7ffa32ff5640 10 client.18339 handle_command_reply: tid=0 2024-09-19T09:11:09.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.969+0000 7ffa57b9c740 1 client.18339 shutdown 2024-09-19T09:11:09.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.969+0000 7ffa57b9c740 1 -- 172.21.15.120:0/1651819221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca435c360 msgr2=0x55aca4368600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:09.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.969+0000 7ffa57b9c740 1 --2- 172.21.15.120:0/1651819221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca435c360 0x55aca4368600 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7ffa5006b2e0 tx=0x7ffa44005fd0 comp rx=0 tx=0).stop 2024-09-19T09:11:09.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.969+0000 7ffa57b9c740 1 -- 172.21.15.120:0/1651819221 shutdown_connections 2024-09-19T09:11:09.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.969+0000 7ffa57b9c740 1 --2- 172.21.15.120:0/1651819221 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55aca437ae50 0x55aca437d270 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:09.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.969+0000 7ffa32ff5640 0 client.18339 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:09.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.969+0000 7ffa57b9c740 1 --2- 172.21.15.120:0/1651819221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca436b150 0x55aca4368b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:09.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.969+0000 7ffa57b9c740 1 --2- 172.21.15.120:0/1651819221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca435c360 0x55aca4368600 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:09.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.969+0000 7ffa57b9c740 1 --2- 172.21.15.120:0/1651819221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aca435ba40 0x55aca43680c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:09.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.969+0000 7ffa57b9c740 1 -- 172.21.15.120:0/1651819221 >> 172.21.15.120:0/1651819221 conn(0x55aca43580f0 msgr2=0x55aca435de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:09.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.969+0000 7ffa57b9c740 1 -- 172.21.15.120:0/1651819221 shutdown_connections 2024-09-19T09:11:09.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.969+0000 7ffa57b9c740 1 -- 172.21.15.120:0/1651819221 wait complete. 2024-09-19T09:11:09.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.969+0000 7ffa57b9c740 20 client.18339 trim_cache size 0 max 16384 2024-09-19T09:11:09.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.989+0000 7ffa57b9c740 1 Processor -- start 2024-09-19T09:11:09.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.989+0000 7ffa57b9c740 1 -- start start 2024-09-19T09:11:09.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.989+0000 7ffa57b9c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aca435ba40 0x55aca4369f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:09.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.989+0000 7ffa57b9c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca435c360 0x55aca4382b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:09.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.989+0000 7ffa57b9c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca436b150 0x55aca4383080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:09.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.989+0000 7ffa57b9c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501ab4f0 con 0x55aca435c360 2024-09-19T09:11:09.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.989+0000 7ffa57b9c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501ad030 con 0x55aca436b150 2024-09-19T09:11:09.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.989+0000 7ffa57b9c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501abbc0 con 0x55aca435ba40 2024-09-19T09:11:09.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.989+0000 7ffa4e7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca435c360 0x55aca4382b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:09.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.989+0000 7ffa4effd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aca435ba40 0x55aca4369f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:09.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.989+0000 7ffa4e7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca435c360 0x55aca4382b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52316/0 (socket says 172.21.15.120:52316) 2024-09-19T09:11:09.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.989+0000 7ffa4e7fc640 1 -- 172.21.15.120:0/3059815952 learned_addr learned my addr 172.21.15.120:0/3059815952 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:09.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.989+0000 7ffa4f7fe640 1 --2- 172.21.15.120:0/3059815952 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca436b150 0x55aca4383080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:09.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.989+0000 7ffa4e7fc640 1 -- 172.21.15.120:0/3059815952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aca435ba40 msgr2=0x55aca4369f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:09.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.989+0000 7ffa4e7fc640 1 --2- 172.21.15.120:0/3059815952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aca435ba40 0x55aca4369f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:09.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.989+0000 7ffa4e7fc640 1 -- 172.21.15.120:0/3059815952 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca436b150 msgr2=0x55aca4383080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:09.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.989+0000 7ffa4e7fc640 1 --2- 172.21.15.120:0/3059815952 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca436b150 0x55aca4383080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:09.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.989+0000 7ffa4e7fc640 1 -- 172.21.15.120:0/3059815952 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aca437d5e0 con 0x55aca435c360 2024-09-19T09:11:09.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.989+0000 7ffa4e7fc640 1 --2- 172.21.15.120:0/3059815952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca435c360 0x55aca4382b40 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7ffa44057040 tx=0x7ffa44032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29a1edebe5c71fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:09.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.990+0000 7ffa34ff9640 1 -- 172.21.15.120:0/3059815952 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4404e040 con 0x55aca435c360 2024-09-19T09:11:09.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.990+0000 7ffa34ff9640 1 -- 172.21.15.120:0/3059815952 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa440048c0 con 0x55aca435c360 2024-09-19T09:11:09.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.990+0000 7ffa34ff9640 1 -- 172.21.15.120:0/3059815952 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44055a60 con 0x55aca435c360 2024-09-19T09:11:09.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.990+0000 7ffa57b9c740 1 -- 172.21.15.120:0/3059815952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca435c360 msgr2=0x55aca4382b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:09.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.990+0000 7ffa57b9c740 1 --2- 172.21.15.120:0/3059815952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca435c360 0x55aca4382b40 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7ffa44057040 tx=0x7ffa44032d50 comp rx=0 tx=0).stop 2024-09-19T09:11:09.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.990+0000 7ffa57b9c740 1 -- 172.21.15.120:0/3059815952 shutdown_connections 2024-09-19T09:11:09.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.991+0000 7ffa57b9c740 1 --2- 172.21.15.120:0/3059815952 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca436b150 0x55aca4383080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:09.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.991+0000 7ffa57b9c740 1 --2- 172.21.15.120:0/3059815952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca435c360 0x55aca4382b40 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:09.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.991+0000 7ffa57b9c740 1 --2- 172.21.15.120:0/3059815952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aca435ba40 0x55aca4369f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:09.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.991+0000 7ffa57b9c740 1 -- 172.21.15.120:0/3059815952 >> 172.21.15.120:0/3059815952 conn(0x55aca43580f0 msgr2=0x55aca4369810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:09.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.991+0000 7ffa57b9c740 1 -- 172.21.15.120:0/3059815952 shutdown_connections 2024-09-19T09:11:09.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.991+0000 7ffa57b9c740 1 -- 172.21.15.120:0/3059815952 wait complete. 2024-09-19T09:11:09.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.991+0000 7ffa57b9c740 1 -- start start 2024-09-19T09:11:09.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.991+0000 7ffa57b9c740 1 Processor -- start 2024-09-19T09:11:09.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.991+0000 7ffa57b9c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aca435ba40 0x55aca43738c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:09.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.991+0000 7ffa57b9c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca435c360 0x55aca436ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:09.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.991+0000 7ffa57b9c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca436b150 0x55aca436f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:09.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.991+0000 7ffa57b9c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501ac960 con 0x55aca435c360 2024-09-19T09:11:10.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.991+0000 7ffa57b9c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501ac290 con 0x55aca436b150 2024-09-19T09:11:10.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.991+0000 7ffa57b9c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501a94b0 con 0x55aca435ba40 2024-09-19T09:11:10.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.991+0000 7ffa57b9c740 10 client.0 _ll_register_callbacks cb 0x7ffa44009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:10.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.991+0000 7ffa4e7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca435c360 0x55aca436ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:10.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.991+0000 7ffa4e7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca435c360 0x55aca436ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52318/0 (socket says 172.21.15.120:52318) 2024-09-19T09:11:10.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.991+0000 7ffa4e7fc640 1 -- 172.21.15.120:0/4154054782 learned_addr learned my addr 172.21.15.120:0/4154054782 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:10.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.991+0000 7ffa4f7fe640 1 --2- 172.21.15.120:0/4154054782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca436b150 0x55aca436f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:10.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.991+0000 7ffa4effd640 1 --2- 172.21.15.120:0/4154054782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aca435ba40 0x55aca43738c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:10.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.992+0000 7ffa4e7fc640 1 -- 172.21.15.120:0/4154054782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aca435ba40 msgr2=0x55aca43738c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:10.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.992+0000 7ffa4e7fc640 1 --2- 172.21.15.120:0/4154054782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aca435ba40 0x55aca43738c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:10.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.992+0000 7ffa4e7fc640 1 -- 172.21.15.120:0/4154054782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca436b150 msgr2=0x55aca436f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:10.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.992+0000 7ffa4e7fc640 1 --2- 172.21.15.120:0/4154054782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca436b150 0x55aca436f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:10.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.992+0000 7ffa4e7fc640 1 -- 172.21.15.120:0/4154054782 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aca436f930 con 0x55aca435c360 2024-09-19T09:11:10.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.992+0000 7ffa4e7fc640 1 --2- 172.21.15.120:0/4154054782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca435c360 0x55aca436ebf0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7ffa4402a460 tx=0x7ffa44000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2b48125e64ad72 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:10.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.992+0000 7ffa32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:11:10.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.992+0000 7ffa32ff5640 1 -- 172.21.15.120:0/4154054782 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4404e070 con 0x55aca435c360 2024-09-19T09:11:10.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.992+0000 7ffa32ff5640 1 -- 172.21.15.120:0/4154054782 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7ffa440559f0 con 0x55aca435c360 2024-09-19T09:11:10.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.992+0000 7ffa57b9c740 1 -- 172.21.15.120:0/4154054782 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa440048c0 con 0x55aca435c360 2024-09-19T09:11:10.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.992+0000 7ffa32ff5640 1 -- 172.21.15.120:0/4154054782 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffa44004350 con 0x55aca435c360 2024-09-19T09:11:10.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.992+0000 7ffa57b9c740 10 client.18345 fetch_fsmap learned FSMap version 30 2024-09-19T09:11:10.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.992+0000 7ffa57b9c740 1 -- 172.21.15.120:0/4154054782 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55aca43e5b80 con 0x55aca435c360 2024-09-19T09:11:10.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.992+0000 7ffa32ff5640 1 -- 172.21.15.120:0/4154054782 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa440048c0 con 0x55aca435c360 2024-09-19T09:11:10.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.993+0000 7ffa32ff5640 1 -- 172.21.15.120:0/4154054782 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ffa44066030 con 0x55aca435c360 2024-09-19T09:11:10.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.993+0000 7ffa57b9c740 10 client.18345 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:11:10.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.993+0000 7ffa57b9c740 10 client.18345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:11:10.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.993+0000 7ffa57b9c740 20 client.18345 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:10.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.993+0000 7ffa57b9c740 1 --2- 172.21.15.120:0/4154054782 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55aca437ae50 0x55aca43e82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:10.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.993+0000 7ffa57b9c740 4 client.18345 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:11:10.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.993+0000 7ffa57b9c740 1 -- 172.21.15.120:0/4154054782 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55aca42c8d00 con 0x55aca437ae50 2024-09-19T09:11:10.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.993+0000 7ffa4effd640 1 --2- 172.21.15.120:0/4154054782 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55aca437ae50 0x55aca43e82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:10.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.993+0000 7ffa4effd640 1 --2- 172.21.15.120:0/4154054782 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55aca437ae50 0x55aca43e82d0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7631a196900a7a7c server_cookie=fbf16352599dc1ea in_seq=0 out_seq=0 2024-09-19T09:11:10.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.993+0000 7ffa32ff5640 10 client.18345 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:10.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.994+0000 7ffa32ff5640 1 -- 172.21.15.120:0/4154054782 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55aca42c8d00 con 0x55aca437ae50 2024-09-19T09:11:10.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.994+0000 7ffa32ff5640 10 client.18345 handle_command_reply: tid=0 2024-09-19T09:11:10.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.994+0000 7ffa57b9c740 1 client.18345 shutdown 2024-09-19T09:11:10.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.994+0000 7ffa57b9c740 1 -- 172.21.15.120:0/4154054782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca435c360 msgr2=0x55aca436ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:10.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.994+0000 7ffa57b9c740 1 --2- 172.21.15.120:0/4154054782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca435c360 0x55aca436ebf0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7ffa4402a460 tx=0x7ffa44000bc0 comp rx=0 tx=0).stop 2024-09-19T09:11:10.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.994+0000 7ffa57b9c740 1 -- 172.21.15.120:0/4154054782 shutdown_connections 2024-09-19T09:11:10.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.994+0000 7ffa57b9c740 1 --2- 172.21.15.120:0/4154054782 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55aca437ae50 0x55aca43e82d0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:10.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.994+0000 7ffa57b9c740 1 --2- 172.21.15.120:0/4154054782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca436b150 0x55aca436f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:10.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.994+0000 7ffa57b9c740 1 --2- 172.21.15.120:0/4154054782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca435c360 0x55aca436ebf0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:10.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.994+0000 7ffa32ff5640 0 client.18345 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:10.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.994+0000 7ffa57b9c740 1 --2- 172.21.15.120:0/4154054782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aca435ba40 0x55aca43738c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:10.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.994+0000 7ffa57b9c740 1 -- 172.21.15.120:0/4154054782 >> 172.21.15.120:0/4154054782 conn(0x55aca43580f0 msgr2=0x55aca435dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:10.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.994+0000 7ffa57b9c740 1 -- 172.21.15.120:0/4154054782 shutdown_connections 2024-09-19T09:11:10.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.994+0000 7ffa57b9c740 1 -- 172.21.15.120:0/4154054782 wait complete. 2024-09-19T09:11:10.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.995+0000 7ffa57b9c740 20 client.18345 trim_cache size 0 max 16384 2024-09-19T09:11:10.003 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:11:10.004 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:11:10.004 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "7abe1d33-e71e-488f-9f20-c377b4dd48a8", 2024-09-19T09:11:10.004 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:11:10.004 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:11:10.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.995+0000 7ffa557c3640 1 -- 172.21.15.120:0/3449170221 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ffa380a32c0 msgr2=0x7ffa380a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:10.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.995+0000 7ffa557c3640 1 --2- 172.21.15.120:0/3449170221 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ffa380a32c0 0x7ffa380a5780 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ffa44032fd0 tx=0x7ffa44005dd0 comp rx=0 tx=0).stop 2024-09-19T09:11:10.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.995+0000 7ffa557c3640 1 -- 172.21.15.120:0/3449170221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa50106a80 msgr2=0x7ffa501a4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:10.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.995+0000 7ffa557c3640 1 --2- 172.21.15.120:0/3449170221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa50106a80 0x7ffa501a4390 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800da10 tx=0x7ffa4800ded0 comp rx=0 tx=0).stop 2024-09-19T09:11:10.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.995+0000 7ffa557c3640 1 -- 172.21.15.120:0/3449170221 shutdown_connections 2024-09-19T09:11:10.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.995+0000 7ffa557c3640 1 --2- 172.21.15.120:0/3449170221 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ffa380a32c0 0x7ffa380a5780 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:10.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.995+0000 7ffa557c3640 1 --2- 172.21.15.120:0/3449170221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa50106a80 0x7ffa501a4390 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:10.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.995+0000 7ffa557c3640 1 --2- 172.21.15.120:0/3449170221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa50101420 0x7ffa50199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:10.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.995+0000 7ffa557c3640 1 --2- 172.21.15.120:0/3449170221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa50100a80 0x7ffa50199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:10.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.995+0000 7ffa557c3640 1 -- 172.21.15.120:0/3449170221 >> 172.21.15.120:0/3449170221 conn(0x7ffa500fc7a0 msgr2=0x7ffa500fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:10.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.995+0000 7ffa557c3640 1 -- 172.21.15.120:0/3449170221 shutdown_connections 2024-09-19T09:11:10.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:09.995+0000 7ffa557c3640 1 -- 172.21.15.120:0/3449170221 wait complete. 2024-09-19T09:11:10.061 INFO:tasks.workunit.client.0.smithi120.stdout: 1 724843 8.85 MB/sec execute 305 sec latency 32.746 ms 2024-09-19T09:11:10.867 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:11:11.060 INFO:tasks.workunit.client.0.smithi120.stdout: 1 728163 8.88 MB/sec execute 306 sec latency 13.939 ms 2024-09-19T09:11:11.178 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:11:11.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:10 smithi196 ceph-mon[35150]: pgmap v591: 97 pgs: 97 active+clean; 30 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 108 op/s 2024-09-19T09:11:11.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:10 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:11:11.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:10 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:11:11.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:10 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:11:11.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:10 smithi153 ceph-mon[35071]: pgmap v591: 97 pgs: 97 active+clean; 30 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 108 op/s 2024-09-19T09:11:11.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:10 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:11:11.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:10 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:11:11.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:10 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:11:11.299 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:10 smithi120 ceph-mon[31150]: pgmap v591: 97 pgs: 97 active+clean; 30 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 108 op/s 2024-09-19T09:11:11.300 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:10 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:11:11.300 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:10 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:11:11.300 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:10 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:11:11.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.714+0000 7fb02144a640 1 -- 172.21.15.120:0/3176242692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb01c102780 msgr2=0x7fb01c1067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:11.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.714+0000 7fb02144a640 1 --2- 172.21.15.120:0/3176242692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb01c102780 0x7fb01c1067d0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fb004009900 tx=0x7fb00402a410 comp rx=0 tx=0).stop 2024-09-19T09:11:11.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.714+0000 7fb02144a640 1 -- 172.21.15.120:0/3176242692 shutdown_connections 2024-09-19T09:11:11.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.714+0000 7fb02144a640 1 --2- 172.21.15.120:0/3176242692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb01c106f00 0x7fb01c10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.714+0000 7fb02144a640 1 --2- 172.21.15.120:0/3176242692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb01c102780 0x7fb01c1067d0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.714+0000 7fb02144a640 1 --2- 172.21.15.120:0/3176242692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb01c101dd0 0x7fb01c1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.714+0000 7fb02144a640 1 -- 172.21.15.120:0/3176242692 >> 172.21.15.120:0/3176242692 conn(0x7fb01c0fd890 msgr2=0x7fb01c0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:11.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.715+0000 7fb02144a640 1 -- 172.21.15.120:0/3176242692 shutdown_connections 2024-09-19T09:11:11.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.715+0000 7fb02144a640 1 -- 172.21.15.120:0/3176242692 wait complete. 2024-09-19T09:11:11.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.715+0000 7fb02144a640 1 Processor -- start 2024-09-19T09:11:11.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.715+0000 7fb02144a640 1 -- start start 2024-09-19T09:11:11.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.715+0000 7fb02144a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb01c101dd0 0x7fb01c1962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:11.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.715+0000 7fb02144a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb01c102780 0x7fb01c196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:11.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.715+0000 7fb02144a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb01c106f00 0x7fb01c19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:11.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.715+0000 7fb02144a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c10fd50 con 0x7fb01c101dd0 2024-09-19T09:11:11.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.715+0000 7fb02144a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c10fbd0 con 0x7fb01c106f00 2024-09-19T09:11:11.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.715+0000 7fb02144a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c10fed0 con 0x7fb01c102780 2024-09-19T09:11:11.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.716+0000 7fb01affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb01c101dd0 0x7fb01c1962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:11.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.716+0000 7fb01affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb01c101dd0 0x7fb01c1962d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46826/0 (socket says 172.21.15.120:46826) 2024-09-19T09:11:11.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.716+0000 7fb013fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb01c102780 0x7fb01c196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:11.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.716+0000 7fb01affd640 1 -- 172.21.15.120:0/2891846894 learned_addr learned my addr 172.21.15.120:0/2891846894 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:11.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.716+0000 7fb01b7fe640 1 --2- 172.21.15.120:0/2891846894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb01c106f00 0x7fb01c19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:11.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.716+0000 7fb01affd640 1 -- 172.21.15.120:0/2891846894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb01c102780 msgr2=0x7fb01c196810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:11.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.716+0000 7fb01affd640 1 --2- 172.21.15.120:0/2891846894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb01c102780 0x7fb01c196810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.716+0000 7fb01affd640 1 -- 172.21.15.120:0/2891846894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb01c106f00 msgr2=0x7fb01c19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:11.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.716+0000 7fb01affd640 1 --2- 172.21.15.120:0/2891846894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb01c106f00 0x7fb01c19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.716+0000 7fb01affd640 1 -- 172.21.15.120:0/2891846894 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01c19b280 con 0x7fb01c101dd0 2024-09-19T09:11:11.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.716+0000 7fb013fff640 1 --2- 172.21.15.120:0/2891846894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb01c102780 0x7fb01c196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:11.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.716+0000 7fb01affd640 1 --2- 172.21.15.120:0/2891846894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb01c101dd0 0x7fb01c1962d0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00e9f0 tx=0x7fb00c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:11.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.716+0000 7fb018ff9640 1 -- 172.21.15.120:0/2891846894 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c00cd80 con 0x7fb01c101dd0 2024-09-19T09:11:11.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.716+0000 7fb018ff9640 1 -- 172.21.15.120:0/2891846894 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb00c010910 con 0x7fb01c101dd0 2024-09-19T09:11:11.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.716+0000 7fb018ff9640 1 -- 172.21.15.120:0/2891846894 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c018990 con 0x7fb01c101dd0 2024-09-19T09:11:11.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.716+0000 7fb02144a640 1 -- 172.21.15.120:0/2891846894 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb01c19b570 con 0x7fb01c101dd0 2024-09-19T09:11:11.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.717+0000 7fb02144a640 1 -- 172.21.15.120:0/2891846894 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb01c1a2e50 con 0x7fb01c101dd0 2024-09-19T09:11:11.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.719+0000 7fb018ff9640 1 -- 172.21.15.120:0/2891846894 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fb00c010ac0 con 0x7fb01c101dd0 2024-09-19T09:11:11.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.719+0000 7fb023823740 1 Processor -- start 2024-09-19T09:11:11.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.719+0000 7fb023823740 1 -- start start 2024-09-19T09:11:11.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.720+0000 7fb023823740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56178e0b04a0 0x56178e0b0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:11.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.720+0000 7fb023823740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56178e0b0dc0 0x56178e0bb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:11.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.720+0000 7fb023823740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56178e0bb680 0x56178e0bda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:11.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.720+0000 7fb023823740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56178e0162e0 con 0x56178e0b04a0 2024-09-19T09:11:11.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.720+0000 7fb023823740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56178dfbdce0 con 0x56178e0bb680 2024-09-19T09:11:11.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.720+0000 7fb023823740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56178e0122a0 con 0x56178e0b0dc0 2024-09-19T09:11:11.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.720+0000 7fb01b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56178e0b04a0 0x56178e0b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:11.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.720+0000 7fb01b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56178e0b04a0 0x56178e0b0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46836/0 (socket says 172.21.15.120:46836) 2024-09-19T09:11:11.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.720+0000 7fb01affd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56178e0b0dc0 0x56178e0bb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:11.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.720+0000 7fb01b7fe640 1 -- 172.21.15.120:0/2145990955 learned_addr learned my addr 172.21.15.120:0/2145990955 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:11.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.720+0000 7fb013fff640 1 --2- 172.21.15.120:0/2145990955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56178e0bb680 0x56178e0bda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:11.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.720+0000 7fb018ff9640 1 --2- 172.21.15.120:0/2891846894 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb0080a32c0 0x7fb0080a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:11.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.720+0000 7fb01b7fe640 1 --2- 172.21.15.120:0/2891846894 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb0080a32c0 0x7fb0080a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:11.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.720+0000 7fb01b7fe640 1 -- 172.21.15.120:0/2145990955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56178e0b0dc0 msgr2=0x56178e0bb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:11.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.720+0000 7fb01b7fe640 1 --2- 172.21.15.120:0/2145990955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56178e0b0dc0 0x56178e0bb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.720+0000 7fb01b7fe640 1 -- 172.21.15.120:0/2145990955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56178e0bb680 msgr2=0x56178e0bda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:11.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.720+0000 7fb01b7fe640 1 --2- 172.21.15.120:0/2145990955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56178e0bb680 0x56178e0bda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.720+0000 7fb018ff9640 1 -- 172.21.15.120:0/2891846894 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fb00c0cfb80 con 0x7fb01c101dd0 2024-09-19T09:11:11.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.720+0000 7fb01b7fe640 1 -- 172.21.15.120:0/2145990955 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56178dfa2440 con 0x56178e0b04a0 2024-09-19T09:11:11.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.720+0000 7fb01affd640 1 --2- 172.21.15.120:0/2145990955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56178e0b0dc0 0x56178e0bb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:11:11.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.721+0000 7fb01b7fe640 1 --2- 172.21.15.120:0/2145990955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56178e0b04a0 0x56178e0b0880 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fb01400ebc0 tx=0x7fb01401c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ab78c784625857 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:11.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.721+0000 7fb01b7fe640 1 --2- 172.21.15.120:0/2891846894 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb0080a32c0 0x7fb0080a5780 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb01401cbb0 tx=0x7fb014023000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:11.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.721+0000 7fb0117fa640 1 -- 172.21.15.120:0/2145990955 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014023070 con 0x56178e0b04a0 2024-09-19T09:11:11.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.721+0000 7fb0117fa640 1 -- 172.21.15.120:0/2145990955 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb01401fdb0 con 0x56178e0b04a0 2024-09-19T09:11:11.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.721+0000 7fb0117fa640 1 -- 172.21.15.120:0/2145990955 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01401e3c0 con 0x56178e0b04a0 2024-09-19T09:11:11.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.722+0000 7fb023823740 1 -- 172.21.15.120:0/2145990955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56178e0b04a0 msgr2=0x56178e0b0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:11.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.722+0000 7fb023823740 1 --2- 172.21.15.120:0/2145990955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56178e0b04a0 0x56178e0b0880 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fb01400ebc0 tx=0x7fb01401c6b0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.722+0000 7fb023823740 1 -- 172.21.15.120:0/2145990955 shutdown_connections 2024-09-19T09:11:11.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.722+0000 7fb023823740 1 --2- 172.21.15.120:0/2145990955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56178e0bb680 0x56178e0bda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.722+0000 7fb023823740 1 --2- 172.21.15.120:0/2145990955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56178e0b0dc0 0x56178e0bb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.722+0000 7fb023823740 1 --2- 172.21.15.120:0/2145990955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56178e0b04a0 0x56178e0b0880 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.722+0000 7fb023823740 1 -- 172.21.15.120:0/2145990955 >> 172.21.15.120:0/2145990955 conn(0x56178e0aca30 msgr2=0x56178e0ace10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:11.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.722+0000 7fb023823740 1 -- 172.21.15.120:0/2145990955 shutdown_connections 2024-09-19T09:11:11.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.722+0000 7fb023823740 1 -- 172.21.15.120:0/2145990955 wait complete. 2024-09-19T09:11:11.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.722+0000 7fb023823740 1 -- start start 2024-09-19T09:11:11.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.722+0000 7fb023823740 1 Processor -- start 2024-09-19T09:11:11.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.722+0000 7fb023823740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56178e0b04a0 0x56178e0baf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:11.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.722+0000 7fb023823740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56178e0b0dc0 0x56178e0b8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:11.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.722+0000 7fb023823740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56178e0bb680 0x56178e0b8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:11.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.722+0000 7fb023823740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56178e0be1f0 con 0x56178e0bb680 2024-09-19T09:11:11.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.723+0000 7fb023823740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56178dfd1050 con 0x56178e0b0dc0 2024-09-19T09:11:11.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.723+0000 7fb023823740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56178e0be370 con 0x56178e0b04a0 2024-09-19T09:11:11.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.723+0000 7fb023823740 10 client.0 _ll_register_callbacks cb 0x56178e011160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:11.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.723+0000 7fb01b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56178e0bb680 0x56178e0b8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:11.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.723+0000 7fb013fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56178e0b0dc0 0x56178e0b8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:11.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.723+0000 7fb01b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56178e0bb680 0x56178e0b8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46848/0 (socket says 172.21.15.120:46848) 2024-09-19T09:11:11.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.723+0000 7fb01b7fe640 1 -- 172.21.15.120:0/3426236878 learned_addr learned my addr 172.21.15.120:0/3426236878 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:11.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.723+0000 7fb01affd640 1 --2- 172.21.15.120:0/3426236878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56178e0b04a0 0x56178e0baf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:11.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.723+0000 7fb01b7fe640 1 -- 172.21.15.120:0/3426236878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56178e0b04a0 msgr2=0x56178e0baf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:11.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.723+0000 7fb01b7fe640 1 --2- 172.21.15.120:0/3426236878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56178e0b04a0 0x56178e0baf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.723+0000 7fb01b7fe640 1 -- 172.21.15.120:0/3426236878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56178e0b0dc0 msgr2=0x56178e0b8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:11.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.723+0000 7fb01b7fe640 1 --2- 172.21.15.120:0/3426236878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56178e0b0dc0 0x56178e0b8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.723+0000 7fb01b7fe640 1 -- 172.21.15.120:0/3426236878 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56178dd873c0 con 0x56178e0bb680 2024-09-19T09:11:11.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.723+0000 7fb01b7fe640 1 --2- 172.21.15.120:0/3426236878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56178e0bb680 0x56178e0b8790 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fb01401fcc0 tx=0x7fb01400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7377168650f51510 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:11.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.723+0000 7fb0027fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:11:11.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.723+0000 7fb0027fc640 1 -- 172.21.15.120:0/3426236878 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014034070 con 0x56178e0bb680 2024-09-19T09:11:11.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.723+0000 7fb0027fc640 1 -- 172.21.15.120:0/3426236878 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fb014027610 con 0x56178e0bb680 2024-09-19T09:11:11.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.723+0000 7fb023823740 1 -- 172.21.15.120:0/3426236878 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb01401fdb0 con 0x56178e0bb680 2024-09-19T09:11:11.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.723+0000 7fb0027fc640 1 -- 172.21.15.120:0/3426236878 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb014027810 con 0x56178e0bb680 2024-09-19T09:11:11.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.724+0000 7fb023823740 10 client.18357 fetch_fsmap learned FSMap version 30 2024-09-19T09:11:11.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.724+0000 7fb023823740 1 -- 172.21.15.120:0/3426236878 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56178e0c9b70 con 0x56178e0bb680 2024-09-19T09:11:11.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.724+0000 7fb0027fc640 1 -- 172.21.15.120:0/3426236878 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb01401fdb0 con 0x56178e0bb680 2024-09-19T09:11:11.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.724+0000 7fb0027fc640 1 -- 172.21.15.120:0/3426236878 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb014030440 con 0x56178e0bb680 2024-09-19T09:11:11.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.724+0000 7fb023823740 10 client.18357 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:11:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.724+0000 7fb023823740 10 client.18357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:11:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.724+0000 7fb023823740 20 client.18357 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.724+0000 7fb023823740 1 --2- 172.21.15.120:0/3426236878 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56178e0cb030 0x56178e0cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.724+0000 7fb023823740 4 client.18357 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:11:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.724+0000 7fb023823740 1 -- 172.21.15.120:0/3426236878 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56178e023850 con 0x56178e0cb030 2024-09-19T09:11:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.724+0000 7fb01affd640 1 --2- 172.21.15.120:0/3426236878 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56178e0cb030 0x56178e0cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.725+0000 7fb01affd640 1 --2- 172.21.15.120:0/3426236878 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56178e0cb030 0x56178e0cd420 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab308803d1a3d84c server_cookie=a2e76e8b84b3bf in_seq=0 out_seq=0 2024-09-19T09:11:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.725+0000 7fb0027fc640 10 client.18357 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.726+0000 7fb0027fc640 1 -- 172.21.15.120:0/3426236878 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56178e023850 con 0x56178e0cb030 2024-09-19T09:11:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.726+0000 7fb0027fc640 10 client.18357 handle_command_reply: tid=0 2024-09-19T09:11:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.726+0000 7fb023823740 1 client.18357 shutdown 2024-09-19T09:11:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.726+0000 7fb023823740 1 -- 172.21.15.120:0/3426236878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56178e0bb680 msgr2=0x56178e0b8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.726+0000 7fb023823740 1 --2- 172.21.15.120:0/3426236878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56178e0bb680 0x56178e0b8790 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fb01401fcc0 tx=0x7fb01400ecb0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.726+0000 7fb023823740 1 -- 172.21.15.120:0/3426236878 shutdown_connections 2024-09-19T09:11:11.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.726+0000 7fb023823740 1 --2- 172.21.15.120:0/3426236878 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56178e0cb030 0x56178e0cd420 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.726+0000 7fb023823740 1 --2- 172.21.15.120:0/3426236878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56178e0bb680 0x56178e0b8790 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.726+0000 7fb023823740 1 --2- 172.21.15.120:0/3426236878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56178e0b0dc0 0x56178e0b8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.726+0000 7fb0027fc640 0 client.18357 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:11.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.726+0000 7fb023823740 1 --2- 172.21.15.120:0/3426236878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56178e0b04a0 0x56178e0baf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.726+0000 7fb023823740 1 -- 172.21.15.120:0/3426236878 >> 172.21.15.120:0/3426236878 conn(0x56178e0aca30 msgr2=0x56178e0abb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:11.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.726+0000 7fb023823740 1 -- 172.21.15.120:0/3426236878 shutdown_connections 2024-09-19T09:11:11.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.726+0000 7fb023823740 1 -- 172.21.15.120:0/3426236878 wait complete. 2024-09-19T09:11:11.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.726+0000 7fb023823740 20 client.18357 trim_cache size 0 max 16384 2024-09-19T09:11:11.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.743+0000 7fb023823740 1 Processor -- start 2024-09-19T09:11:11.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.743+0000 7fb023823740 1 -- start start 2024-09-19T09:11:11.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.744+0000 7fb023823740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56178e0b04a0 0x56178e0c7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:11.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.744+0000 7fb023823740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56178e0b0dc0 0x56178e0d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:11.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.744+0000 7fb023823740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56178e0bb680 0x56178e0d2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:11.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.744+0000 7fb023823740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c1ab400 con 0x56178e0bb680 2024-09-19T09:11:11.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.744+0000 7fb023823740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c1acf40 con 0x56178e0b0dc0 2024-09-19T09:11:11.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.744+0000 7fb023823740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c1abad0 con 0x56178e0b04a0 2024-09-19T09:11:11.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.744+0000 7fb01b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56178e0bb680 0x56178e0d2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:11.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.744+0000 7fb01b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56178e0bb680 0x56178e0d2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46864/0 (socket says 172.21.15.120:46864) 2024-09-19T09:11:11.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.744+0000 7fb01b7fe640 1 -- 172.21.15.120:0/1682800229 learned_addr learned my addr 172.21.15.120:0/1682800229 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:11.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.744+0000 7fb013fff640 1 --2- 172.21.15.120:0/1682800229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56178e0b0dc0 0x56178e0d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:11.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.744+0000 7fb01affd640 1 --2- 172.21.15.120:0/1682800229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56178e0b04a0 0x56178e0c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:11.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.744+0000 7fb01b7fe640 1 -- 172.21.15.120:0/1682800229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56178e0b04a0 msgr2=0x56178e0c7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:11.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.744+0000 7fb01b7fe640 1 --2- 172.21.15.120:0/1682800229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56178e0b04a0 0x56178e0c7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.744+0000 7fb01b7fe640 1 -- 172.21.15.120:0/1682800229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56178e0b0dc0 msgr2=0x56178e0d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:11.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.744+0000 7fb01b7fe640 1 --2- 172.21.15.120:0/1682800229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56178e0b0dc0 0x56178e0d29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.744+0000 7fb01b7fe640 1 -- 172.21.15.120:0/1682800229 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56178e0adae0 con 0x56178e0bb680 2024-09-19T09:11:11.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.745+0000 7fb01b7fe640 1 --2- 172.21.15.120:0/1682800229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56178e0bb680 0x56178e0d2f30 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fb014030860 tx=0x7fb01401c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce2cc1ca805aceb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:11.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.745+0000 7fb010ff9640 1 -- 172.21.15.120:0/1682800229 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014034070 con 0x56178e0bb680 2024-09-19T09:11:11.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.745+0000 7fb010ff9640 1 -- 172.21.15.120:0/1682800229 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb014048070 con 0x56178e0bb680 2024-09-19T09:11:11.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.745+0000 7fb010ff9640 1 -- 172.21.15.120:0/1682800229 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014030cd0 con 0x56178e0bb680 2024-09-19T09:11:11.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.745+0000 7fb023823740 1 -- 172.21.15.120:0/1682800229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56178e0bb680 msgr2=0x56178e0d2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:11.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.745+0000 7fb023823740 1 --2- 172.21.15.120:0/1682800229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56178e0bb680 0x56178e0d2f30 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fb014030860 tx=0x7fb01401c6b0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.746+0000 7fb023823740 1 -- 172.21.15.120:0/1682800229 shutdown_connections 2024-09-19T09:11:11.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.746+0000 7fb023823740 1 --2- 172.21.15.120:0/1682800229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56178e0bb680 0x56178e0d2f30 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.746+0000 7fb023823740 1 --2- 172.21.15.120:0/1682800229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56178e0b0dc0 0x56178e0d29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.746+0000 7fb023823740 1 --2- 172.21.15.120:0/1682800229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56178e0b04a0 0x56178e0c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.746+0000 7fb023823740 1 -- 172.21.15.120:0/1682800229 >> 172.21.15.120:0/1682800229 conn(0x56178e0aca30 msgr2=0x56178e0b2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:11.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.746+0000 7fb023823740 1 -- 172.21.15.120:0/1682800229 shutdown_connections 2024-09-19T09:11:11.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.746+0000 7fb023823740 1 -- 172.21.15.120:0/1682800229 wait complete. 2024-09-19T09:11:11.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.746+0000 7fb023823740 1 -- start start 2024-09-19T09:11:11.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.746+0000 7fb023823740 1 Processor -- start 2024-09-19T09:11:11.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.746+0000 7fb023823740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56178e0b04a0 0x56178e0bf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:11.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.746+0000 7fb023823740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56178e0b0dc0 0x56178e0bfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:11.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.746+0000 7fb023823740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56178e0bb680 0x56178e0c00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:11.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.746+0000 7fb023823740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c1ac870 con 0x56178e0b0dc0 2024-09-19T09:11:11.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.746+0000 7fb023823740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c1ac1a0 con 0x56178e0bb680 2024-09-19T09:11:11.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.746+0000 7fb023823740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c1a93c0 con 0x56178e0b04a0 2024-09-19T09:11:11.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.746+0000 7fb023823740 10 client.0 _ll_register_callbacks cb 0x7fb0140093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:11.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.746+0000 7fb013fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56178e0b0dc0 0x56178e0bfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:11.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.746+0000 7fb013fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56178e0b0dc0 0x56178e0bfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46876/0 (socket says 172.21.15.120:46876) 2024-09-19T09:11:11.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.746+0000 7fb013fff640 1 -- 172.21.15.120:0/3557707535 learned_addr learned my addr 172.21.15.120:0/3557707535 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:11.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.746+0000 7fb01affd640 1 --2- 172.21.15.120:0/3557707535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56178e0b04a0 0x56178e0bf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:11.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.746+0000 7fb01b7fe640 1 --2- 172.21.15.120:0/3557707535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56178e0bb680 0x56178e0c00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:11.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.747+0000 7fb013fff640 1 -- 172.21.15.120:0/3557707535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56178e0b04a0 msgr2=0x56178e0bf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:11.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.747+0000 7fb013fff640 1 --2- 172.21.15.120:0/3557707535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56178e0b04a0 0x56178e0bf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.747+0000 7fb013fff640 1 -- 172.21.15.120:0/3557707535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56178e0bb680 msgr2=0x56178e0c00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:11.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.747+0000 7fb013fff640 1 --2- 172.21.15.120:0/3557707535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56178e0bb680 0x56178e0c00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.747+0000 7fb013fff640 1 -- 172.21.15.120:0/3557707535 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56178e134cb0 con 0x56178e0b0dc0 2024-09-19T09:11:11.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.747+0000 7fb01b7fe640 1 --2- 172.21.15.120:0/3557707535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56178e0bb680 0x56178e0c00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:11:11.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.747+0000 7fb013fff640 1 --2- 172.21.15.120:0/3557707535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56178e0b0dc0 0x56178e0bfb70 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fb004009900 tx=0x7fb004031a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad4bc8d9dfe9c01 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:11.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.747+0000 7fb0027fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:11:11.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.747+0000 7fb0027fc640 1 -- 172.21.15.120:0/3557707535 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00404c020 con 0x56178e0b0dc0 2024-09-19T09:11:11.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.747+0000 7fb0027fc640 1 -- 172.21.15.120:0/3557707535 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fb004005020 con 0x56178e0b0dc0 2024-09-19T09:11:11.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.747+0000 7fb023823740 1 -- 172.21.15.120:0/3557707535 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb014048070 con 0x56178e0b0dc0 2024-09-19T09:11:11.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.747+0000 7fb0027fc640 1 -- 172.21.15.120:0/3557707535 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb004035040 con 0x56178e0b0dc0 2024-09-19T09:11:11.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.747+0000 7fb023823740 10 client.18363 fetch_fsmap learned FSMap version 30 2024-09-19T09:11:11.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.747+0000 7fb023823740 1 -- 172.21.15.120:0/3557707535 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56178e135ff0 con 0x56178e0b0dc0 2024-09-19T09:11:11.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.747+0000 7fb0027fc640 1 -- 172.21.15.120:0/3557707535 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb014048070 con 0x56178e0b0dc0 2024-09-19T09:11:11.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.748+0000 7fb0027fc640 1 -- 172.21.15.120:0/3557707535 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb004047070 con 0x56178e0b0dc0 2024-09-19T09:11:11.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.748+0000 7fb023823740 10 client.18363 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:11:11.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.748+0000 7fb023823740 10 client.18363 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:11:11.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.748+0000 7fb023823740 20 client.18363 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:11.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.748+0000 7fb023823740 1 --2- 172.21.15.120:0/3557707535 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56178e1367f0 0x56178e138c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:11.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.748+0000 7fb023823740 4 client.18363 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:11:11.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.748+0000 7fb023823740 1 -- 172.21.15.120:0/3557707535 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56178e0bdef0 con 0x56178e1367f0 2024-09-19T09:11:11.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.748+0000 7fb01affd640 1 --2- 172.21.15.120:0/3557707535 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56178e1367f0 0x56178e138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:11.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.748+0000 7fb01affd640 1 --2- 172.21.15.120:0/3557707535 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56178e1367f0 0x56178e138c10 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=176104f2e79420e0 server_cookie=ce68d866510e6558 in_seq=0 out_seq=0 2024-09-19T09:11:11.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.748+0000 7fb0027fc640 10 client.18363 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:11.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.748+0000 7fb0027fc640 1 -- 172.21.15.120:0/3557707535 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56178e0bdef0 con 0x56178e1367f0 2024-09-19T09:11:11.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.748+0000 7fb0027fc640 10 client.18363 handle_command_reply: tid=0 2024-09-19T09:11:11.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.748+0000 7fb023823740 1 client.18363 shutdown 2024-09-19T09:11:11.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.748+0000 7fb023823740 1 -- 172.21.15.120:0/3557707535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56178e0b0dc0 msgr2=0x56178e0bfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:11.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.748+0000 7fb023823740 1 --2- 172.21.15.120:0/3557707535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56178e0b0dc0 0x56178e0bfb70 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fb004009900 tx=0x7fb004031a80 comp rx=0 tx=0).stop 2024-09-19T09:11:11.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.749+0000 7fb023823740 1 -- 172.21.15.120:0/3557707535 shutdown_connections 2024-09-19T09:11:11.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.749+0000 7fb023823740 1 --2- 172.21.15.120:0/3557707535 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56178e1367f0 0x56178e138c10 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.749+0000 7fb023823740 1 --2- 172.21.15.120:0/3557707535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56178e0bb680 0x56178e0c00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.749+0000 7fb023823740 1 --2- 172.21.15.120:0/3557707535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56178e0b0dc0 0x56178e0bfb70 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.749+0000 7fb023823740 1 --2- 172.21.15.120:0/3557707535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56178e0b04a0 0x56178e0bf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.749+0000 7fb0027fc640 0 client.18363 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:11.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.749+0000 7fb023823740 1 -- 172.21.15.120:0/3557707535 >> 172.21.15.120:0/3557707535 conn(0x56178e0aca30 msgr2=0x56178e0b9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:11.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.749+0000 7fb023823740 1 -- 172.21.15.120:0/3557707535 shutdown_connections 2024-09-19T09:11:11.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.749+0000 7fb023823740 1 -- 172.21.15.120:0/3557707535 wait complete. 2024-09-19T09:11:11.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.749+0000 7fb023823740 20 client.18363 trim_cache size 0 max 16384 2024-09-19T09:11:11.756 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:11:11.756 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:11:11.756 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-19T09:11:11.756 INFO:teuthology.orchestra.run.smithi120.stdout: "7abe1d33-e71e-488f-9f20-c377b4dd48a8": { 2024-09-19T09:11:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-19T09:11:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "7abe1d33-e71e-488f-9f20-c377b4dd48a8", 2024-09-19T09:11:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-19T09:11:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:11:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:11:11.757 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:11:11.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.749+0000 7fb02144a640 1 -- 172.21.15.120:0/2891846894 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb0080a32c0 msgr2=0x7fb0080a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:11.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.749+0000 7fb02144a640 1 --2- 172.21.15.120:0/2891846894 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb0080a32c0 0x7fb0080a5780 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb01401cbb0 tx=0x7fb014023000 comp rx=0 tx=0).stop 2024-09-19T09:11:11.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.749+0000 7fb02144a640 1 -- 172.21.15.120:0/2891846894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb01c101dd0 msgr2=0x7fb01c1962d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:11.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.749+0000 7fb02144a640 1 --2- 172.21.15.120:0/2891846894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb01c101dd0 0x7fb01c1962d0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00e9f0 tx=0x7fb00c00eeb0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.749+0000 7fb02144a640 1 -- 172.21.15.120:0/2891846894 shutdown_connections 2024-09-19T09:11:11.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.749+0000 7fb02144a640 1 --2- 172.21.15.120:0/2891846894 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb0080a32c0 0x7fb0080a5780 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.749+0000 7fb02144a640 1 --2- 172.21.15.120:0/2891846894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb01c106f00 0x7fb01c19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.749+0000 7fb02144a640 1 --2- 172.21.15.120:0/2891846894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb01c102780 0x7fb01c196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.749+0000 7fb02144a640 1 --2- 172.21.15.120:0/2891846894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb01c101dd0 0x7fb01c1962d0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.749+0000 7fb02144a640 1 -- 172.21.15.120:0/2891846894 >> 172.21.15.120:0/2891846894 conn(0x7fb01c0fd890 msgr2=0x7fb01c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:11.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.750+0000 7fb02144a640 1 -- 172.21.15.120:0/2891846894 shutdown_connections 2024-09-19T09:11:11.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:11.750+0000 7fb02144a640 1 -- 172.21.15.120:0/2891846894 wait complete. 2024-09-19T09:11:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.985+0000 7f91dd76f640 1 -- 172.21.15.120:0/2336042425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91d81025f0 msgr2=0x7f91d8108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.985+0000 7f91dd76f640 1 --2- 172.21.15.120:0/2336042425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91d81025f0 0x7f91d8108e80 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009ca0 tx=0x7f91d002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.985+0000 7f91dd76f640 1 -- 172.21.15.120:0/2336042425 shutdown_connections 2024-09-19T09:11:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.985+0000 7f91dd76f640 1 --2- 172.21.15.120:0/2336042425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91d81025f0 0x7f91d8108e80 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.985+0000 7f91dd76f640 1 --2- 172.21.15.120:0/2336042425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91d8101c30 0x7f91d81020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.985+0000 7f91dd76f640 1 --2- 172.21.15.120:0/2336042425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91d8100a40 0x7f91d8100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.985+0000 7f91dd76f640 1 -- 172.21.15.120:0/2336042425 >> 172.21.15.120:0/2336042425 conn(0x7f91d80fc1e0 msgr2=0x7f91d80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.985+0000 7f91dd76f640 1 -- 172.21.15.120:0/2336042425 shutdown_connections 2024-09-19T09:11:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.985+0000 7f91dd76f640 1 -- 172.21.15.120:0/2336042425 wait complete. 2024-09-19T09:11:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.986+0000 7f91dd76f640 1 Processor -- start 2024-09-19T09:11:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.986+0000 7f91dd76f640 1 -- start start 2024-09-19T09:11:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.986+0000 7f91dd76f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91d8100a40 0x7f91d8196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.986+0000 7f91dd76f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91d8101c30 0x7f91d8197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.986+0000 7f91dd76f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91d81025f0 0x7f91d819e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.986+0000 7f91dd76f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f91d810c770 con 0x7f91d8100a40 2024-09-19T09:11:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.986+0000 7f91dd76f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f91d810c5f0 con 0x7f91d8101c30 2024-09-19T09:11:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.986+0000 7f91dd76f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f91d810c8f0 con 0x7f91d81025f0 2024-09-19T09:11:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.986+0000 7f91d6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91d8100a40 0x7f91d8196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.986+0000 7f91d6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91d8100a40 0x7f91d8196cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46898/0 (socket says 172.21.15.120:46898) 2024-09-19T09:11:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.986+0000 7f91d6ffd640 1 -- 172.21.15.120:0/1866833124 learned_addr learned my addr 172.21.15.120:0/1866833124 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.986+0000 7f91d77fe640 1 --2- 172.21.15.120:0/1866833124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91d81025f0 0x7f91d819e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.986+0000 7f91d67fc640 1 --2- 172.21.15.120:0/1866833124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91d8101c30 0x7f91d8197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.987+0000 7f91d6ffd640 1 -- 172.21.15.120:0/1866833124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91d81025f0 msgr2=0x7f91d819e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.987+0000 7f91d6ffd640 1 --2- 172.21.15.120:0/1866833124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91d81025f0 0x7f91d819e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.987+0000 7f91d6ffd640 1 -- 172.21.15.120:0/1866833124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91d8101c30 msgr2=0x7f91d8197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.987+0000 7f91d6ffd640 1 --2- 172.21.15.120:0/1866833124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91d8101c30 0x7f91d8197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.987+0000 7f91d6ffd640 1 -- 172.21.15.120:0/1866833124 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d819e900 con 0x7f91d8100a40 2024-09-19T09:11:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.987+0000 7f91d67fc640 1 --2- 172.21.15.120:0/1866833124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91d8101c30 0x7f91d8197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:11:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.987+0000 7f91d77fe640 1 --2- 172.21.15.120:0/1866833124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91d81025f0 0x7f91d819e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:11:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.987+0000 7f91d6ffd640 1 --2- 172.21.15.120:0/1866833124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91d8100a40 0x7f91d8196cf0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f91c800e9c0 tx=0x7f91c800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.987+0000 7f91bffff640 1 -- 172.21.15.120:0/1866833124 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c800cd80 con 0x7f91d8100a40 2024-09-19T09:11:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.987+0000 7f91bffff640 1 -- 172.21.15.120:0/1866833124 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91c80035f0 con 0x7f91d8100a40 2024-09-19T09:11:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.987+0000 7f91dd76f640 1 -- 172.21.15.120:0/1866833124 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91d819ebf0 con 0x7f91d8100a40 2024-09-19T09:11:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.987+0000 7f91bffff640 1 -- 172.21.15.120:0/1866833124 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c8010690 con 0x7f91d8100a40 2024-09-19T09:11:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.987+0000 7f91dd76f640 1 -- 172.21.15.120:0/1866833124 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91d819f190 con 0x7f91d8100a40 2024-09-19T09:11:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.989+0000 7f91bffff640 1 -- 172.21.15.120:0/1866833124 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f91c80037a0 con 0x7f91d8100a40 2024-09-19T09:11:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.989+0000 7f91e05e1740 1 Processor -- start 2024-09-19T09:11:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.989+0000 7f91e05e1740 1 -- start start 2024-09-19T09:11:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.990+0000 7f91e05e1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a43e7519a0 0x55a43e751da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.990+0000 7f91e05e1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a43e7522e0 0x55a43e756da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.990+0000 7f91e05e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a43e757370 0x55a43e759830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.990+0000 7f91e05e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a43e6b52f0 con 0x55a43e757370 2024-09-19T09:11:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.990+0000 7f91e05e1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a43e65d270 con 0x55a43e7522e0 2024-09-19T09:11:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.990+0000 7f91e05e1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a43e6b12b0 con 0x55a43e7519a0 2024-09-19T09:11:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.990+0000 7f91d67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a43e757370 0x55a43e759830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.990+0000 7f91d77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a43e7519a0 0x55a43e751da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.990+0000 7f91d67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a43e757370 0x55a43e759830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46908/0 (socket says 172.21.15.120:46908) 2024-09-19T09:11:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.990+0000 7f91d67fc640 1 -- 172.21.15.120:0/4154771391 learned_addr learned my addr 172.21.15.120:0/4154771391 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.990+0000 7f91d6ffd640 1 --2- 172.21.15.120:0/4154771391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a43e7522e0 0x55a43e756da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.990+0000 7f91bffff640 1 --2- 172.21.15.120:0/1866833124 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f91c00a3370 0x7f91c00a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.990+0000 7f91d67fc640 1 -- 172.21.15.120:0/4154771391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a43e7519a0 msgr2=0x55a43e751da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.990+0000 7f91d67fc640 1 --2- 172.21.15.120:0/4154771391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a43e7519a0 0x55a43e751da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.990+0000 7f91d67fc640 1 -- 172.21.15.120:0/4154771391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a43e7522e0 msgr2=0x55a43e756da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.990+0000 7f91d67fc640 1 --2- 172.21.15.120:0/4154771391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a43e7522e0 0x55a43e756da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.990+0000 7f91d67fc640 1 -- 172.21.15.120:0/4154771391 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a43e65c340 con 0x55a43e757370 2024-09-19T09:11:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.990+0000 7f91d77fe640 1 --2- 172.21.15.120:0/1866833124 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f91c00a3370 0x7f91c00a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.990+0000 7f91bffff640 1 -- 172.21.15.120:0/1866833124 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f91c80cfea0 con 0x7f91d8100a40 2024-09-19T09:11:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.990+0000 7f91d67fc640 1 --2- 172.21.15.120:0/4154771391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a43e757370 0x55a43e759830 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f91cc007850 tx=0x7f91cc007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e902c41fc56ba7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.990+0000 7f91bd7fa640 1 -- 172.21.15.120:0/4154771391 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91cc00d950 con 0x55a43e757370 2024-09-19T09:11:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.990+0000 7f91bd7fa640 1 -- 172.21.15.120:0/4154771391 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91cc018070 con 0x55a43e757370 2024-09-19T09:11:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.990+0000 7f91bd7fa640 1 -- 172.21.15.120:0/4154771391 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91cc0123e0 con 0x55a43e757370 2024-09-19T09:11:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.991+0000 7f91e05e1740 1 -- 172.21.15.120:0/4154771391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a43e757370 msgr2=0x55a43e759830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.991+0000 7f91d77fe640 1 --2- 172.21.15.120:0/1866833124 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f91c00a3370 0x7f91c00a5830 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009430 tx=0x7f91d00095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.991+0000 7f91e05e1740 1 --2- 172.21.15.120:0/4154771391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a43e757370 0x55a43e759830 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f91cc007850 tx=0x7f91cc007d10 comp rx=0 tx=0).stop 2024-09-19T09:11:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.991+0000 7f91e05e1740 1 -- 172.21.15.120:0/4154771391 shutdown_connections 2024-09-19T09:11:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.991+0000 7f91e05e1740 1 --2- 172.21.15.120:0/4154771391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a43e757370 0x55a43e759830 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.991+0000 7f91e05e1740 1 --2- 172.21.15.120:0/4154771391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a43e7522e0 0x55a43e756da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.991+0000 7f91e05e1740 1 --2- 172.21.15.120:0/4154771391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a43e7519a0 0x55a43e751da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.991+0000 7f91e05e1740 1 -- 172.21.15.120:0/4154771391 >> 172.21.15.120:0/4154771391 conn(0x55a43e74d010 msgr2=0x55a43e74d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.991+0000 7f91e05e1740 1 -- 172.21.15.120:0/4154771391 shutdown_connections 2024-09-19T09:11:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.991+0000 7f91e05e1740 1 -- 172.21.15.120:0/4154771391 wait complete. 2024-09-19T09:11:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.991+0000 7f91e05e1740 1 -- start start 2024-09-19T09:11:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.992+0000 7f91e05e1740 1 Processor -- start 2024-09-19T09:11:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.992+0000 7f91e05e1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a43e7519a0 0x55a43e75dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.992+0000 7f91e05e1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a43e7522e0 0x55a43e75e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.992+0000 7f91e05e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a43e757370 0x55a43e765320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.992+0000 7f91e05e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a43e75a340 con 0x55a43e757370 2024-09-19T09:11:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.992+0000 7f91e05e1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a43e63e290 con 0x55a43e7519a0 2024-09-19T09:11:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.992+0000 7f91e05e1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a43e75a4c0 con 0x55a43e7522e0 2024-09-19T09:11:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.992+0000 7f91e05e1740 10 client.0 _ll_register_callbacks cb 0x55a43e66f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.992+0000 7f91d77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a43e757370 0x55a43e765320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.992+0000 7f91d6ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a43e7519a0 0x55a43e75dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.992+0000 7f91d77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a43e757370 0x55a43e765320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46910/0 (socket says 172.21.15.120:46910) 2024-09-19T09:11:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.992+0000 7f91d77fe640 1 -- 172.21.15.120:0/2120013378 learned_addr learned my addr 172.21.15.120:0/2120013378 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.992+0000 7f91d67fc640 1 --2- 172.21.15.120:0/2120013378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a43e7522e0 0x55a43e75e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.992+0000 7f91d77fe640 1 -- 172.21.15.120:0/2120013378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a43e7522e0 msgr2=0x55a43e75e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.992+0000 7f91d77fe640 1 --2- 172.21.15.120:0/2120013378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a43e7522e0 0x55a43e75e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.992+0000 7f91d77fe640 1 -- 172.21.15.120:0/2120013378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a43e7519a0 msgr2=0x55a43e75dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.992+0000 7f91d77fe640 1 --2- 172.21.15.120:0/2120013378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a43e7519a0 0x55a43e75dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.992+0000 7f91d77fe640 1 -- 172.21.15.120:0/2120013378 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a43e3c8210 con 0x55a43e757370 2024-09-19T09:11:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.992+0000 7f91d77fe640 1 --2- 172.21.15.120:0/2120013378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a43e757370 0x55a43e765320 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009ca0 tx=0x7f91d002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b119ea41f4603fee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.992+0000 7f91baff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:11:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.992+0000 7f91baff5640 1 -- 172.21.15.120:0/2120013378 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0003b70 con 0x55a43e757370 2024-09-19T09:11:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.993+0000 7f91baff5640 1 -- 172.21.15.120:0/2120013378 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f91d0003d10 con 0x55a43e757370 2024-09-19T09:11:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.993+0000 7f91e05e1740 1 -- 172.21.15.120:0/2120013378 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91cc018070 con 0x55a43e757370 2024-09-19T09:11:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.993+0000 7f91baff5640 1 -- 172.21.15.120:0/2120013378 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f91d00307d0 con 0x55a43e757370 2024-09-19T09:11:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.993+0000 7f91e05e1740 10 client.18375 fetch_fsmap learned FSMap version 30 2024-09-19T09:11:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.993+0000 7f91e05e1740 1 -- 172.21.15.120:0/2120013378 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a43e494620 con 0x55a43e757370 2024-09-19T09:11:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.993+0000 7f91baff5640 1 -- 172.21.15.120:0/2120013378 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91cc018070 con 0x55a43e757370 2024-09-19T09:11:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.993+0000 7f91baff5640 1 -- 172.21.15.120:0/2120013378 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f91d0045070 con 0x55a43e757370 2024-09-19T09:11:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.993+0000 7f91e05e1740 10 client.18375 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:11:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.993+0000 7f91e05e1740 10 client.18375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:11:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.993+0000 7f91e05e1740 20 client.18375 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.993+0000 7f91e05e1740 1 --2- 172.21.15.120:0/2120013378 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a43e769ab0 0x55a43e76bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.993+0000 7f91e05e1740 4 client.18375 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:11:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.993+0000 7f91e05e1740 1 -- 172.21.15.120:0/2120013378 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a43e6c2a20 con 0x55a43e769ab0 2024-09-19T09:11:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.993+0000 7f91d6ffd640 1 --2- 172.21.15.120:0/2120013378 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a43e769ab0 0x55a43e76bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.994+0000 7f91baff5640 10 client.18375 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.994+0000 7f91d6ffd640 1 --2- 172.21.15.120:0/2120013378 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a43e769ab0 0x55a43e76bf50 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7d9ecbe8926a3ee server_cookie=4917000ee5af02e8 in_seq=0 out_seq=0 2024-09-19T09:11:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.994+0000 7f91baff5640 1 -- 172.21.15.120:0/2120013378 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a43e6c2a20 con 0x55a43e769ab0 2024-09-19T09:11:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.994+0000 7f91baff5640 10 client.18375 handle_command_reply: tid=0 2024-09-19T09:11:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.995+0000 7f91e05e1740 1 client.18375 shutdown 2024-09-19T09:11:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.995+0000 7f91e05e1740 1 -- 172.21.15.120:0/2120013378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a43e757370 msgr2=0x55a43e765320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.995+0000 7f91e05e1740 1 --2- 172.21.15.120:0/2120013378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a43e757370 0x55a43e765320 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009ca0 tx=0x7f91d002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:11:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.995+0000 7f91e05e1740 1 -- 172.21.15.120:0/2120013378 shutdown_connections 2024-09-19T09:11:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.995+0000 7f91e05e1740 1 --2- 172.21.15.120:0/2120013378 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a43e769ab0 0x55a43e76bf50 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.995+0000 7f91e05e1740 1 --2- 172.21.15.120:0/2120013378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a43e757370 0x55a43e765320 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.995+0000 7f91baff5640 0 client.18375 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.995+0000 7f91e05e1740 1 --2- 172.21.15.120:0/2120013378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a43e7522e0 0x55a43e75e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.995+0000 7f91e05e1740 1 --2- 172.21.15.120:0/2120013378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a43e7519a0 0x55a43e75dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.995+0000 7f91e05e1740 1 -- 172.21.15.120:0/2120013378 >> 172.21.15.120:0/2120013378 conn(0x55a43e74d010 msgr2=0x55a43e74e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.995+0000 7f91e05e1740 1 -- 172.21.15.120:0/2120013378 shutdown_connections 2024-09-19T09:11:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.995+0000 7f91e05e1740 1 -- 172.21.15.120:0/2120013378 wait complete. 2024-09-19T09:11:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:11.995+0000 7f91e05e1740 20 client.18375 trim_cache size 0 max 16384 2024-09-19T09:11:12.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.009+0000 7f91e05e1740 1 Processor -- start 2024-09-19T09:11:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.009+0000 7f91e05e1740 1 -- start start 2024-09-19T09:11:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.009+0000 7f91e05e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a43e7519a0 0x55a43e74dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.009+0000 7f91e05e1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a43e7522e0 0x55a43e7652e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.009+0000 7f91e05e1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a43e757370 0x55a43e74e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.009+0000 7f91e05e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f91d81a6e90 con 0x55a43e7519a0 2024-09-19T09:11:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.009+0000 7f91e05e1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f91d81a89d0 con 0x55a43e7522e0 2024-09-19T09:11:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.009+0000 7f91e05e1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f91d81a7560 con 0x55a43e757370 2024-09-19T09:11:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.009+0000 7f91d6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a43e7519a0 0x55a43e74dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.009+0000 7f91d6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a43e7519a0 0x55a43e74dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46914/0 (socket says 172.21.15.120:46914) 2024-09-19T09:11:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.009+0000 7f91d6ffd640 1 -- 172.21.15.120:0/2068018726 learned_addr learned my addr 172.21.15.120:0/2068018726 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.009+0000 7f91d67fc640 1 --2- 172.21.15.120:0/2068018726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a43e7522e0 0x55a43e7652e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.010+0000 7f91d77fe640 1 --2- 172.21.15.120:0/2068018726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a43e757370 0x55a43e74e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.010+0000 7f91d6ffd640 1 -- 172.21.15.120:0/2068018726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a43e757370 msgr2=0x55a43e74e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.010+0000 7f91d6ffd640 1 --2- 172.21.15.120:0/2068018726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a43e757370 0x55a43e74e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.010+0000 7f91d6ffd640 1 -- 172.21.15.120:0/2068018726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a43e7522e0 msgr2=0x55a43e7652e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.010+0000 7f91d6ffd640 1 --2- 172.21.15.120:0/2068018726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a43e7522e0 0x55a43e7652e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.010+0000 7f91d6ffd640 1 -- 172.21.15.120:0/2068018726 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a43e7527c0 con 0x55a43e7519a0 2024-09-19T09:11:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.010+0000 7f91d6ffd640 1 --2- 172.21.15.120:0/2068018726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a43e7519a0 0x55a43e74dec0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f91c800c6a0 tx=0x7f91c80987c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2e41d83b1248ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.010+0000 7f91bcff9640 1 -- 172.21.15.120:0/2068018726 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c80b4030 con 0x55a43e7519a0 2024-09-19T09:11:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.010+0000 7f91bcff9640 1 -- 172.21.15.120:0/2068018726 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91c809cb80 con 0x55a43e7519a0 2024-09-19T09:11:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.010+0000 7f91bcff9640 1 -- 172.21.15.120:0/2068018726 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c80cda70 con 0x55a43e7519a0 2024-09-19T09:11:12.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.011+0000 7f91e05e1740 1 -- 172.21.15.120:0/2068018726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a43e7519a0 msgr2=0x55a43e74dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:12.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.011+0000 7f91e05e1740 1 --2- 172.21.15.120:0/2068018726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a43e7519a0 0x55a43e74dec0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f91c800c6a0 tx=0x7f91c80987c0 comp rx=0 tx=0).stop 2024-09-19T09:11:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.011+0000 7f91e05e1740 1 -- 172.21.15.120:0/2068018726 shutdown_connections 2024-09-19T09:11:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.011+0000 7f91e05e1740 1 --2- 172.21.15.120:0/2068018726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a43e757370 0x55a43e74e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.011+0000 7f91e05e1740 1 --2- 172.21.15.120:0/2068018726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a43e7522e0 0x55a43e7652e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.011+0000 7f91e05e1740 1 --2- 172.21.15.120:0/2068018726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a43e7519a0 0x55a43e74dec0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.011+0000 7f91e05e1740 1 -- 172.21.15.120:0/2068018726 >> 172.21.15.120:0/2068018726 conn(0x55a43e74d010 msgr2=0x55a43e75dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.011+0000 7f91e05e1740 1 -- 172.21.15.120:0/2068018726 shutdown_connections 2024-09-19T09:11:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.011+0000 7f91e05e1740 1 -- 172.21.15.120:0/2068018726 wait complete. 2024-09-19T09:11:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.011+0000 7f91e05e1740 1 -- start start 2024-09-19T09:11:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.012+0000 7f91e05e1740 1 Processor -- start 2024-09-19T09:11:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.012+0000 7f91e05e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a43e7519a0 0x55a43e762af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.012+0000 7f91e05e1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a43e7522e0 0x55a43e763030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.012+0000 7f91e05e1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a43e757370 0x55a43e764040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.012+0000 7f91e05e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f91d81a8300 con 0x55a43e7519a0 2024-09-19T09:11:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.012+0000 7f91e05e1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f91d81a7c30 con 0x55a43e757370 2024-09-19T09:11:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.012+0000 7f91e05e1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f91d81a4e50 con 0x55a43e7522e0 2024-09-19T09:11:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.012+0000 7f91e05e1740 10 client.0 _ll_register_callbacks cb 0x7f91d0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.012+0000 7f91d6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a43e7519a0 0x55a43e762af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.012+0000 7f91d6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a43e7519a0 0x55a43e762af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46922/0 (socket says 172.21.15.120:46922) 2024-09-19T09:11:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.012+0000 7f91d6ffd640 1 -- 172.21.15.120:0/2547168211 learned_addr learned my addr 172.21.15.120:0/2547168211 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.012+0000 7f91d77fe640 1 --2- 172.21.15.120:0/2547168211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a43e757370 0x55a43e764040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.012+0000 7f91d6ffd640 1 -- 172.21.15.120:0/2547168211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a43e7522e0 msgr2=0x55a43e763030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.012+0000 7f91d67fc640 1 --2- 172.21.15.120:0/2547168211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a43e7522e0 0x55a43e763030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.012+0000 7f91d6ffd640 1 --2- 172.21.15.120:0/2547168211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a43e7522e0 0x55a43e763030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.012+0000 7f91d6ffd640 1 -- 172.21.15.120:0/2547168211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a43e757370 msgr2=0x55a43e764040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.012+0000 7f91d6ffd640 1 --2- 172.21.15.120:0/2547168211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a43e757370 0x55a43e764040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.012+0000 7f91d6ffd640 1 -- 172.21.15.120:0/2547168211 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a43e764650 con 0x55a43e7519a0 2024-09-19T09:11:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.012+0000 7f91baff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:11:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.012+0000 7f91d6ffd640 1 --2- 172.21.15.120:0/2547168211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a43e7519a0 0x55a43e762af0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f91c809c180 tx=0x7f91c809a990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b93e7adf04f26caa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.013+0000 7f91baff5640 1 -- 172.21.15.120:0/2547168211 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c80b4070 con 0x55a43e7519a0 2024-09-19T09:11:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.013+0000 7f91baff5640 1 -- 172.21.15.120:0/2547168211 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f91c8098a60 con 0x55a43e7519a0 2024-09-19T09:11:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.013+0000 7f91e05e1740 1 -- 172.21.15.120:0/2547168211 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91c809cb80 con 0x55a43e7519a0 2024-09-19T09:11:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.013+0000 7f91baff5640 1 -- 172.21.15.120:0/2547168211 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f91c80a0830 con 0x55a43e7519a0 2024-09-19T09:11:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.013+0000 7f91e05e1740 10 client.18381 fetch_fsmap learned FSMap version 30 2024-09-19T09:11:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.013+0000 7f91e05e1740 1 -- 172.21.15.120:0/2547168211 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a43e7d4930 con 0x55a43e7519a0 2024-09-19T09:11:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.013+0000 7f91baff5640 1 -- 172.21.15.120:0/2547168211 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91c809cb80 con 0x55a43e7519a0 2024-09-19T09:11:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.013+0000 7f91baff5640 1 -- 172.21.15.120:0/2547168211 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f91c80d3040 con 0x55a43e7519a0 2024-09-19T09:11:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.013+0000 7f91e05e1740 10 client.18381 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T09:11:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.013+0000 7f91e05e1740 10 client.18381 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:11:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.013+0000 7f91e05e1740 20 client.18381 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.013+0000 7f91e05e1740 1 --2- 172.21.15.120:0/2547168211 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a43e7d51e0 0x55a43e7d7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.013+0000 7f91e05e1740 4 client.18381 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:11:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.013+0000 7f91e05e1740 1 -- 172.21.15.120:0/2547168211 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a43e6bcd70 con 0x55a43e7d51e0 2024-09-19T09:11:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.013+0000 7f91d67fc640 1 --2- 172.21.15.120:0/2547168211 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a43e7d51e0 0x55a43e7d7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.014+0000 7f91d67fc640 1 --2- 172.21.15.120:0/2547168211 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a43e7d51e0 0x55a43e7d7600 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=237ffb6240a2780a server_cookie=cbeb9c0482f4c876 in_seq=0 out_seq=0 2024-09-19T09:11:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.014+0000 7f91baff5640 10 client.18381 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:12.063 INFO:tasks.workunit.client.0.smithi120.stdout: 1 730665 8.89 MB/sec execute 307 sec latency 54.452 ms 2024-09-19T09:11:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.122+0000 7f91baff5640 1 -- 172.21.15.120:0/2547168211 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a43e6bcd70 con 0x55a43e7d51e0 2024-09-19T09:11:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.122+0000 7f91baff5640 10 client.18381 handle_command_reply: tid=0 2024-09-19T09:11:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.122+0000 7f91e05e1740 1 client.18381 shutdown 2024-09-19T09:11:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.122+0000 7f91e05e1740 1 -- 172.21.15.120:0/2547168211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a43e7519a0 msgr2=0x55a43e762af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.122+0000 7f91e05e1740 1 --2- 172.21.15.120:0/2547168211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a43e7519a0 0x55a43e762af0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f91c809c180 tx=0x7f91c809a990 comp rx=0 tx=0).stop 2024-09-19T09:11:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.123+0000 7f91e05e1740 1 -- 172.21.15.120:0/2547168211 shutdown_connections 2024-09-19T09:11:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.123+0000 7f91e05e1740 1 --2- 172.21.15.120:0/2547168211 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a43e7d51e0 0x55a43e7d7600 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.123+0000 7f91e05e1740 1 --2- 172.21.15.120:0/2547168211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a43e757370 0x55a43e764040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.123+0000 7f91baff5640 0 client.18381 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.123+0000 7f91e05e1740 1 --2- 172.21.15.120:0/2547168211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a43e7522e0 0x55a43e763030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.123+0000 7f91e05e1740 1 --2- 172.21.15.120:0/2547168211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a43e7519a0 0x55a43e762af0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.123+0000 7f91e05e1740 1 -- 172.21.15.120:0/2547168211 >> 172.21.15.120:0/2547168211 conn(0x55a43e74d010 msgr2=0x55a43e74b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.123+0000 7f91e05e1740 1 -- 172.21.15.120:0/2547168211 shutdown_connections 2024-09-19T09:11:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.123+0000 7f91e05e1740 1 -- 172.21.15.120:0/2547168211 wait complete. 2024-09-19T09:11:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.124+0000 7f91e05e1740 20 client.18381 trim_cache size 0 max 16384 2024-09-19T09:11:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:11:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:11:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:11:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:11:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.125+0000 7f91dd76f640 1 -- 172.21.15.120:0/1866833124 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f91c00a3370 msgr2=0x7f91c00a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.125+0000 7f91dd76f640 1 --2- 172.21.15.120:0/1866833124 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f91c00a3370 0x7f91c00a5830 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009430 tx=0x7f91d00095c0 comp rx=0 tx=0).stop 2024-09-19T09:11:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.125+0000 7f91dd76f640 1 -- 172.21.15.120:0/1866833124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91d8100a40 msgr2=0x7f91d8196cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.125+0000 7f91dd76f640 1 --2- 172.21.15.120:0/1866833124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91d8100a40 0x7f91d8196cf0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f91c800e9c0 tx=0x7f91c800ee80 comp rx=0 tx=0).stop 2024-09-19T09:11:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.125+0000 7f91dd76f640 1 -- 172.21.15.120:0/1866833124 shutdown_connections 2024-09-19T09:11:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.125+0000 7f91dd76f640 1 --2- 172.21.15.120:0/1866833124 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f91c00a3370 0x7f91c00a5830 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.125+0000 7f91dd76f640 1 --2- 172.21.15.120:0/1866833124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91d81025f0 0x7f91d819e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.125+0000 7f91dd76f640 1 --2- 172.21.15.120:0/1866833124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91d8101c30 0x7f91d8197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.125+0000 7f91dd76f640 1 --2- 172.21.15.120:0/1866833124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91d8100a40 0x7f91d8196cf0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.125+0000 7f91dd76f640 1 -- 172.21.15.120:0/1866833124 >> 172.21.15.120:0/1866833124 conn(0x7f91d80fc1e0 msgr2=0x7f91d8104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.125+0000 7f91dd76f640 1 -- 172.21.15.120:0/1866833124 shutdown_connections 2024-09-19T09:11:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:12.125+0000 7f91dd76f640 1 -- 172.21.15.120:0/1866833124 wait complete. 2024-09-19T09:11:12.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:11 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:11:12.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:11 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:11:12.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:11 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:11:12.460 INFO:tasks.cephfs.filesystem:scrub status for tag:7abe1d33-e71e-488f-9f20-c377b4dd48a8 - {'path': '/', 'tag': '7abe1d33-e71e-488f-9f20-c377b4dd48a8', 'options': 'recursive,force'} 2024-09-19T09:11:12.688 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:11:12 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-e[63911]: ignoring --setuser ceph since I am not root 2024-09-19T09:11:12.688 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:11:12 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-e[63911]: ignoring --setgroup ceph since I am not root 2024-09-19T09:11:12.688 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:11:12 smithi153 ceph-mds[63915]: -- 172.21.15.153:0/2878609025 <== mon.2 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a2a0ba1c20 con 0x55a2a0b7f000 2024-09-19T09:11:12.688 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:11:12 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-e[63911]: starting mds.e at 2024-09-19T09:11:13.062 INFO:tasks.workunit.client.0.smithi120.stdout: 1 732846 8.90 MB/sec execute 308 sec latency 30.934 ms 2024-09-19T09:11:13.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:12 smithi196 ceph-mon[35150]: pgmap v592: 97 pgs: 97 active+clean; 31 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-09-19T09:11:13.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:12 smithi196 ceph-mon[35150]: daemon mds.e restarted 2024-09-19T09:11:13.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:12 smithi196 ceph-mon[35150]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T09:11:13.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:12 smithi153 ceph-mon[35071]: pgmap v592: 97 pgs: 97 active+clean; 31 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-09-19T09:11:13.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:12 smithi153 ceph-mon[35071]: daemon mds.e restarted 2024-09-19T09:11:13.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:12 smithi153 ceph-mon[35071]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T09:11:13.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:12 smithi120 ceph-mon[31150]: pgmap v592: 97 pgs: 97 active+clean; 31 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-09-19T09:11:13.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:12 smithi120 ceph-mon[31150]: daemon mds.e restarted 2024-09-19T09:11:13.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:12 smithi120 ceph-mon[31150]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T09:11:14.062 INFO:tasks.workunit.client.0.smithi120.stdout: 1 735521 8.92 MB/sec execute 309 sec latency 71.189 ms 2024-09-19T09:11:14.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:13 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.153:6834/3214776406,v1:172.21.15.153:6835/3214776406] up:boot 2024-09-19T09:11:14.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:13 smithi196 ceph-mon[35150]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-19T09:11:14.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:13 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:11:14.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:13 smithi196 ceph-mon[35150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:11:14.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:13 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:11:14.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:13 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.153:6834/3214776406,v1:172.21.15.153:6835/3214776406] up:boot 2024-09-19T09:11:14.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:13 smithi153 ceph-mon[35071]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-19T09:11:14.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:13 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:11:14.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:13 smithi153 ceph-mon[35071]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:11:14.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:13 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:11:14.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:13 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.153:6834/3214776406,v1:172.21.15.153:6835/3214776406] up:boot 2024-09-19T09:11:14.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:13 smithi120 ceph-mon[31150]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-19T09:11:14.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:13 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:11:14.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:13 smithi120 ceph-mon[31150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:11:14.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:13 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:11:15.062 INFO:tasks.workunit.client.0.smithi120.stdout: 1 738782 8.95 MB/sec execute 310 sec latency 17.881 ms 2024-09-19T09:11:15.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:14 smithi196 ceph-mon[35150]: pgmap v594: 97 pgs: 97 active+clean; 27 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.6 MiB/s wr, 256 op/s 2024-09-19T09:11:15.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:14 smithi153 ceph-mon[35071]: pgmap v594: 97 pgs: 97 active+clean; 27 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.6 MiB/s wr, 256 op/s 2024-09-19T09:11:15.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:14 smithi120 ceph-mon[31150]: pgmap v594: 97 pgs: 97 active+clean; 27 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.6 MiB/s wr, 256 op/s 2024-09-19T09:11:16.063 INFO:tasks.workunit.client.0.smithi120.stdout: 1 741501 8.96 MB/sec execute 311 sec latency 10.869 ms 2024-09-19T09:11:16.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:15 smithi196 ceph-mon[35150]: pgmap v595: 97 pgs: 97 active+clean; 30 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.8 MiB/s wr, 330 op/s 2024-09-19T09:11:16.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:15 smithi153 ceph-mon[35071]: pgmap v595: 97 pgs: 97 active+clean; 30 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.8 MiB/s wr, 330 op/s 2024-09-19T09:11:16.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:15 smithi120 ceph-mon[31150]: pgmap v595: 97 pgs: 97 active+clean; 30 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.8 MiB/s wr, 330 op/s 2024-09-19T09:11:17.063 INFO:tasks.workunit.client.0.smithi120.stdout: 1 743834 8.97 MB/sec execute 312 sec latency 13.728 ms 2024-09-19T09:11:18.063 INFO:tasks.workunit.client.0.smithi120.stdout: 1 746589 8.99 MB/sec execute 313 sec latency 15.837 ms 2024-09-19T09:11:19.063 INFO:tasks.workunit.client.0.smithi120.stdout: 1 749630 9.02 MB/sec execute 314 sec latency 10.581 ms 2024-09-19T09:11:19.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:18 smithi196 ceph-mon[35150]: pgmap v596: 97 pgs: 97 active+clean; 31 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.7 MiB/s wr, 364 op/s 2024-09-19T09:11:19.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:18 smithi153 ceph-mon[35071]: pgmap v596: 97 pgs: 97 active+clean; 31 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.7 MiB/s wr, 364 op/s 2024-09-19T09:11:19.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:18 smithi120 ceph-mon[31150]: pgmap v596: 97 pgs: 97 active+clean; 31 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.7 MiB/s wr, 364 op/s 2024-09-19T09:11:20.063 INFO:tasks.workunit.client.0.smithi120.stdout: 1 752756 9.05 MB/sec execute 315 sec latency 16.202 ms 2024-09-19T09:11:21.063 INFO:tasks.workunit.client.0.smithi120.stdout: 1 755330 9.05 MB/sec execute 316 sec latency 13.984 ms 2024-09-19T09:11:21.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:20 smithi196 ceph-mon[35150]: pgmap v597: 97 pgs: 97 active+clean; 33 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.6 MiB/s wr, 327 op/s 2024-09-19T09:11:21.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:20 smithi153 ceph-mon[35071]: pgmap v597: 97 pgs: 97 active+clean; 33 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.6 MiB/s wr, 327 op/s 2024-09-19T09:11:21.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:20 smithi120 ceph-mon[31150]: pgmap v597: 97 pgs: 97 active+clean; 33 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.6 MiB/s wr, 327 op/s 2024-09-19T09:11:22.063 INFO:tasks.workunit.client.0.smithi120.stdout: 1 757936 9.07 MB/sec execute 317 sec latency 12.106 ms 2024-09-19T09:11:23.063 INFO:tasks.workunit.client.0.smithi120.stdout: 1 760740 9.09 MB/sec execute 318 sec latency 14.343 ms 2024-09-19T09:11:23.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:22 smithi196 ceph-mon[35150]: pgmap v598: 97 pgs: 97 active+clean; 35 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.7 MiB/s wr, 344 op/s 2024-09-19T09:11:23.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:22 smithi153 ceph-mon[35071]: pgmap v598: 97 pgs: 97 active+clean; 35 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.7 MiB/s wr, 344 op/s 2024-09-19T09:11:23.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:22 smithi120 ceph-mon[31150]: pgmap v598: 97 pgs: 97 active+clean; 35 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.7 MiB/s wr, 344 op/s 2024-09-19T09:11:24.063 INFO:tasks.workunit.client.0.smithi120.stdout: 1 763880 9.12 MB/sec execute 319 sec latency 10.137 ms 2024-09-19T09:11:25.063 INFO:tasks.workunit.client.0.smithi120.stdout: 1 767225 9.15 MB/sec execute 320 sec latency 11.210 ms 2024-09-19T09:11:25.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:24 smithi196 ceph-mon[35150]: pgmap v599: 97 pgs: 97 active+clean; 41 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.1 MiB/s wr, 398 op/s 2024-09-19T09:11:25.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:24 smithi153 ceph-mon[35071]: pgmap v599: 97 pgs: 97 active+clean; 41 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.1 MiB/s wr, 398 op/s 2024-09-19T09:11:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.225+0000 7feedc542640 1 -- 172.21.15.120:0/2432163291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feed4101c30 msgr2=0x7feed41020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.225+0000 7feedc542640 1 --2- 172.21.15.120:0/2432163291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feed4101c30 0x7feed41020b0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7feec4009900 tx=0x7feec402a410 comp rx=0 tx=0).stop 2024-09-19T09:11:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.225+0000 7feedc542640 1 -- 172.21.15.120:0/2432163291 shutdown_connections 2024-09-19T09:11:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.225+0000 7feedc542640 1 --2- 172.21.15.120:0/2432163291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feed41025f0 0x7feed4108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.225+0000 7feedc542640 1 --2- 172.21.15.120:0/2432163291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feed4101c30 0x7feed41020b0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.225+0000 7feedc542640 1 --2- 172.21.15.120:0/2432163291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feed4100a40 0x7feed4100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.225+0000 7feedc542640 1 -- 172.21.15.120:0/2432163291 >> 172.21.15.120:0/2432163291 conn(0x7feed40fc1e0 msgr2=0x7feed40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.225+0000 7feedc542640 1 -- 172.21.15.120:0/2432163291 shutdown_connections 2024-09-19T09:11:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.225+0000 7feedc542640 1 -- 172.21.15.120:0/2432163291 wait complete. 2024-09-19T09:11:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.226+0000 7feedc542640 1 Processor -- start 2024-09-19T09:11:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.226+0000 7feedc542640 1 -- start start 2024-09-19T09:11:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.226+0000 7feedc542640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feed4100a40 0x7feed41049d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.226+0000 7feedc542640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feed4101c30 0x7feed4103010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.226+0000 7feedc542640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feed41025f0 0x7feed4103550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.226+0000 7feedc542640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feed410c6a0 con 0x7feed4100a40 2024-09-19T09:11:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.226+0000 7feedc542640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feed410c520 con 0x7feed41025f0 2024-09-19T09:11:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.226+0000 7feedc542640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feed410c820 con 0x7feed4101c30 2024-09-19T09:11:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.226+0000 7feeda2de640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feed4100a40 0x7feed41049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.226+0000 7feeda2de640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feed4100a40 0x7feed41049d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44010/0 (socket says 172.21.15.120:44010) 2024-09-19T09:11:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.226+0000 7feeda2de640 1 -- 172.21.15.120:0/1322370437 learned_addr learned my addr 172.21.15.120:0/1322370437 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.226+0000 7feedaadf640 1 --2- 172.21.15.120:0/1322370437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feed41025f0 0x7feed4103550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.226+0000 7feed9add640 1 --2- 172.21.15.120:0/1322370437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feed4101c30 0x7feed4103010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.226+0000 7feeda2de640 1 -- 172.21.15.120:0/1322370437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feed4101c30 msgr2=0x7feed4103010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.226+0000 7feeda2de640 1 --2- 172.21.15.120:0/1322370437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feed4101c30 0x7feed4103010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.226+0000 7feeda2de640 1 -- 172.21.15.120:0/1322370437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feed41025f0 msgr2=0x7feed4103550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.226+0000 7feeda2de640 1 --2- 172.21.15.120:0/1322370437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feed41025f0 0x7feed4103550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.226+0000 7feeda2de640 1 -- 172.21.15.120:0/1322370437 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed4103de0 con 0x7feed4100a40 2024-09-19T09:11:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.227+0000 7feeda2de640 1 --2- 172.21.15.120:0/1322370437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feed4100a40 0x7feed41049d0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7feed000d950 tx=0x7feed000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.227+0000 7feecb7fe640 1 -- 172.21.15.120:0/1322370437 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed0017020 con 0x7feed4100a40 2024-09-19T09:11:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.227+0000 7feecb7fe640 1 -- 172.21.15.120:0/1322370437 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feed0009c30 con 0x7feed4100a40 2024-09-19T09:11:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.227+0000 7feedc542640 1 -- 172.21.15.120:0/1322370437 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed41a1f20 con 0x7feed4100a40 2024-09-19T09:11:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.227+0000 7feecb7fe640 1 -- 172.21.15.120:0/1322370437 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed000f740 con 0x7feed4100a40 2024-09-19T09:11:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.227+0000 7feedc542640 1 -- 172.21.15.120:0/1322370437 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed41a2320 con 0x7feed4100a40 2024-09-19T09:11:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.230+0000 7feedf2db740 1 Processor -- start 2024-09-19T09:11:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.230+0000 7feedf2db740 1 -- start start 2024-09-19T09:11:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.230+0000 7feecb7fe640 1 -- 172.21.15.120:0/1322370437 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7feed001a020 con 0x7feed4100a40 2024-09-19T09:11:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.230+0000 7feedf2db740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9979c59a0 0x55e9979c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.230+0000 7feedf2db740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9979c62e0 0x55e9979cada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.230+0000 7feedf2db740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9979cb370 0x55e9979cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.230+0000 7feedf2db740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e9979292f0 con 0x55e9979c62e0 2024-09-19T09:11:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.230+0000 7feedf2db740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e9978d1270 con 0x55e9979c59a0 2024-09-19T09:11:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.230+0000 7feedf2db740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e9979252b0 con 0x55e9979cb370 2024-09-19T09:11:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.230+0000 7feeda2de640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9979c62e0 0x55e9979cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.230+0000 7feedaadf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9979c59a0 0x55e9979c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.230+0000 7feedaadf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9979c59a0 0x55e9979c5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:47230/0 (socket says 172.21.15.120:47230) 2024-09-19T09:11:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.230+0000 7feedaadf640 1 -- 172.21.15.120:0/2295305967 learned_addr learned my addr 172.21.15.120:0/2295305967 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.231+0000 7feed9add640 1 --2- 172.21.15.120:0/2295305967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9979cb370 0x55e9979cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.231+0000 7feedaadf640 1 -- 172.21.15.120:0/2295305967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9979cb370 msgr2=0x55e9979cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.231+0000 7feedaadf640 1 --2- 172.21.15.120:0/2295305967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9979cb370 0x55e9979cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.231+0000 7feedaadf640 1 -- 172.21.15.120:0/2295305967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9979c62e0 msgr2=0x55e9979cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.231+0000 7feedaadf640 1 --2- 172.21.15.120:0/2295305967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9979c62e0 0x55e9979cada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.231+0000 7feedaadf640 1 -- 172.21.15.120:0/2295305967 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9978d0340 con 0x55e9979c59a0 2024-09-19T09:11:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.231+0000 7feecb7fe640 1 --2- 172.21.15.120:0/1322370437 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7feec00a32f0 0x7feec00a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.231+0000 7feedaadf640 1 --2- 172.21.15.120:0/2295305967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9979c59a0 0x55e9979c5da0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7feecc00ae10 tx=0x7feecc00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6e4bd506b02ba28 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.231+0000 7feeda2de640 1 --2- 172.21.15.120:0/1322370437 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7feec00a32f0 0x7feec00a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.231+0000 7feec8bfe640 1 -- 172.21.15.120:0/2295305967 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc003b70 con 0x55e9979c59a0 2024-09-19T09:11:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.231+0000 7feecb7fe640 1 -- 172.21.15.120:0/1322370437 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7feed00cf2e0 con 0x7feed4100a40 2024-09-19T09:11:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.231+0000 7feec8bfe640 1 -- 172.21.15.120:0/2295305967 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feecc003d10 con 0x55e9979c59a0 2024-09-19T09:11:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.231+0000 7feec8bfe640 1 -- 172.21.15.120:0/2295305967 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc010b40 con 0x55e9979c59a0 2024-09-19T09:11:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.232+0000 7feedf2db740 1 -- 172.21.15.120:0/2295305967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9979c59a0 msgr2=0x55e9979c5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.232+0000 7feeda2de640 1 --2- 172.21.15.120:0/1322370437 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7feec00a32f0 0x7feec00a57b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7feed00982d0 tx=0x7feed00d2d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.232+0000 7feedf2db740 1 --2- 172.21.15.120:0/2295305967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9979c59a0 0x55e9979c5da0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7feecc00ae10 tx=0x7feecc00f790 comp rx=0 tx=0).stop 2024-09-19T09:11:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.232+0000 7feedf2db740 1 -- 172.21.15.120:0/2295305967 shutdown_connections 2024-09-19T09:11:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.232+0000 7feedf2db740 1 --2- 172.21.15.120:0/2295305967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9979cb370 0x55e9979cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.232+0000 7feedf2db740 1 --2- 172.21.15.120:0/2295305967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9979c62e0 0x55e9979cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.232+0000 7feedf2db740 1 --2- 172.21.15.120:0/2295305967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9979c59a0 0x55e9979c5da0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.232+0000 7feedf2db740 1 -- 172.21.15.120:0/2295305967 >> 172.21.15.120:0/2295305967 conn(0x55e9979c1010 msgr2=0x55e9979c1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.232+0000 7feedf2db740 1 -- 172.21.15.120:0/2295305967 shutdown_connections 2024-09-19T09:11:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.232+0000 7feedf2db740 1 -- 172.21.15.120:0/2295305967 wait complete. 2024-09-19T09:11:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.232+0000 7feedf2db740 1 -- start start 2024-09-19T09:11:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.232+0000 7feedf2db740 1 Processor -- start 2024-09-19T09:11:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.233+0000 7feedf2db740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9979c59a0 0x55e9979cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.233+0000 7feedf2db740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9979c62e0 0x55e9979d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.233+0000 7feedf2db740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9979cb370 0x55e9979d70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.233+0000 7feedf2db740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e9979ce340 con 0x55e9979cb370 2024-09-19T09:11:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.233+0000 7feedf2db740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e9978b2290 con 0x55e9979c59a0 2024-09-19T09:11:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.233+0000 7feedf2db740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e9979ce4c0 con 0x55e9979c62e0 2024-09-19T09:11:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.233+0000 7feedf2db740 10 client.0 _ll_register_callbacks cb 0x55e9978e3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.233+0000 7feed9add640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9979c62e0 0x55e9979d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.233+0000 7feedaadf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9979c59a0 0x55e9979cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:25.242 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.233+0000 7feed9add640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9979c62e0 0x55e9979d00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36384/0 (socket says 172.21.15.120:36384) 2024-09-19T09:11:25.242 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.233+0000 7feed9add640 1 -- 172.21.15.120:0/3096628581 learned_addr learned my addr 172.21.15.120:0/3096628581 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:25.242 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.233+0000 7feedaadf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9979c59a0 0x55e9979cfb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:47236/0 (socket says 172.21.15.120:47236) 2024-09-19T09:11:25.242 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.233+0000 7feedaadf640 1 --2- 172.21.15.120:0/3096628581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9979cb370 0x55e9979d70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:25.242 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.233+0000 7feed9add640 1 -- 172.21.15.120:0/3096628581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9979c59a0 msgr2=0x55e9979cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:25.242 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.233+0000 7feed9add640 1 --2- 172.21.15.120:0/3096628581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9979c59a0 0x55e9979cfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.242 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.233+0000 7feed9add640 1 -- 172.21.15.120:0/3096628581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9979cb370 msgr2=0x55e9979d70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:25.242 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.233+0000 7feed9add640 1 --2- 172.21.15.120:0/3096628581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9979cb370 0x55e9979d70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.242 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.233+0000 7feed9add640 1 -- 172.21.15.120:0/3096628581 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e99763c210 con 0x55e9979c62e0 2024-09-19T09:11:25.242 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.233+0000 7feedaadf640 1 --2- 172.21.15.120:0/3096628581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9979c59a0 0x55e9979cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:25.242 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.233+0000 7feed9add640 1 --2- 172.21.15.120:0/3096628581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9979c62e0 0x55e9979d00a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feec402a930 tx=0x7feec40318e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69780da76bb25d59 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:25.242 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.233+0000 7feebdffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:11:25.242 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.233+0000 7feebdffb640 1 -- 172.21.15.120:0/3096628581 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4035040 con 0x55e9979c62e0 2024-09-19T09:11:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.234+0000 7feedf2db740 1 -- 172.21.15.120:0/3096628581 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feecc003d10 con 0x55e9979c62e0 2024-09-19T09:11:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.234+0000 7feebdffb640 1 -- 172.21.15.120:0/3096628581 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7feec4004240 con 0x55e9979c62e0 2024-09-19T09:11:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.234+0000 7feebdffb640 1 -- 172.21.15.120:0/3096628581 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feec4032e60 con 0x55e9979c62e0 2024-09-19T09:11:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.234+0000 7feedf2db740 10 client.27394 fetch_fsmap learned FSMap version 32 2024-09-19T09:11:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.234+0000 7feedf2db740 1 -- 172.21.15.120:0/3096628581 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e997708620 con 0x55e9979c62e0 2024-09-19T09:11:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.234+0000 7feebdffb640 1 -- 172.21.15.120:0/3096628581 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feecc003d10 con 0x55e9979c62e0 2024-09-19T09:11:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.234+0000 7feebdffb640 1 -- 172.21.15.120:0/3096628581 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7feec4004b60 con 0x55e9979c62e0 2024-09-19T09:11:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.234+0000 7feedf2db740 10 client.27394 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:11:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.234+0000 7feedf2db740 10 client.27394 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:11:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.234+0000 7feedf2db740 20 client.27394 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.234+0000 7feedf2db740 1 --2- 172.21.15.120:0/3096628581 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e9979db820 0x55e9979ddc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.234+0000 7feedf2db740 4 client.27394 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:11:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.234+0000 7feedf2db740 1 -- 172.21.15.120:0/3096628581 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e997936a20 con 0x55e9979db820 2024-09-19T09:11:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.234+0000 7feedaadf640 1 --2- 172.21.15.120:0/3096628581 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e9979db820 0x55e9979ddc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.235+0000 7feedaadf640 1 --2- 172.21.15.120:0/3096628581 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e9979db820 0x55e9979ddc10 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a328a276f4e48ff3 server_cookie=4aa5e6c1f90b4d3f in_seq=0 out_seq=0 2024-09-19T09:11:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.235+0000 7feebdffb640 10 client.27394 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.235+0000 7feebdffb640 1 -- 172.21.15.120:0/3096628581 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e997936a20 con 0x55e9979db820 2024-09-19T09:11:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.235+0000 7feebdffb640 10 client.27394 handle_command_reply: tid=0 2024-09-19T09:11:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.235+0000 7feedf2db740 1 client.27394 shutdown 2024-09-19T09:11:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.236+0000 7feedf2db740 1 -- 172.21.15.120:0/3096628581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9979c62e0 msgr2=0x55e9979d00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.236+0000 7feedf2db740 1 --2- 172.21.15.120:0/3096628581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9979c62e0 0x55e9979d00a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feec402a930 tx=0x7feec40318e0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.236+0000 7feedf2db740 1 -- 172.21.15.120:0/3096628581 shutdown_connections 2024-09-19T09:11:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.236+0000 7feedf2db740 1 --2- 172.21.15.120:0/3096628581 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e9979db820 0x55e9979ddc10 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.236+0000 7feedf2db740 1 --2- 172.21.15.120:0/3096628581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9979cb370 0x55e9979d70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.236+0000 7feebdffb640 0 client.27394 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.236+0000 7feedf2db740 1 --2- 172.21.15.120:0/3096628581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9979c62e0 0x55e9979d00a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.236+0000 7feedf2db740 1 --2- 172.21.15.120:0/3096628581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9979c59a0 0x55e9979cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.236+0000 7feedf2db740 1 -- 172.21.15.120:0/3096628581 >> 172.21.15.120:0/3096628581 conn(0x55e9979c1010 msgr2=0x55e9979c2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.236+0000 7feedf2db740 1 -- 172.21.15.120:0/3096628581 shutdown_connections 2024-09-19T09:11:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.236+0000 7feedf2db740 1 -- 172.21.15.120:0/3096628581 wait complete. 2024-09-19T09:11:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.236+0000 7feedf2db740 20 client.27394 trim_cache size 0 max 16384 2024-09-19T09:11:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.252+0000 7feedf2db740 1 Processor -- start 2024-09-19T09:11:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.252+0000 7feedf2db740 1 -- start start 2024-09-19T09:11:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.252+0000 7feedf2db740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9979c59a0 0x55e9979c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.252+0000 7feedf2db740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9979c62e0 0x55e9979d70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.252+0000 7feedf2db740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9979cb370 0x55e9979e24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.252+0000 7feedf2db740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feed41ab250 con 0x55e9979c59a0 2024-09-19T09:11:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.252+0000 7feedf2db740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feed41acd90 con 0x55e9979cb370 2024-09-19T09:11:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.252+0000 7feedf2db740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feed41ab920 con 0x55e9979c62e0 2024-09-19T09:11:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.252+0000 7feedaadf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9979c59a0 0x55e9979c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.253+0000 7feedaadf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9979c59a0 0x55e9979c00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44032/0 (socket says 172.21.15.120:44032) 2024-09-19T09:11:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.253+0000 7feedaadf640 1 -- 172.21.15.120:0/2954113495 learned_addr learned my addr 172.21.15.120:0/2954113495 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.253+0000 7feedaadf640 1 --2- 172.21.15.120:0/2954113495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9979cb370 0x55e9979e24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.253+0000 7feed9add640 1 --2- 172.21.15.120:0/2954113495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9979c62e0 0x55e9979d70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.253+0000 7feedaadf640 1 -- 172.21.15.120:0/2954113495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9979c62e0 msgr2=0x55e9979d70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.253+0000 7feedaadf640 1 --2- 172.21.15.120:0/2954113495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9979c62e0 0x55e9979d70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.253+0000 7feedaadf640 1 -- 172.21.15.120:0/2954113495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9979cb370 msgr2=0x55e9979e24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.253+0000 7feedaadf640 1 --2- 172.21.15.120:0/2954113495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9979cb370 0x55e9979e24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.253+0000 7feedaadf640 1 -- 172.21.15.120:0/2954113495 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9979ddf80 con 0x55e9979c59a0 2024-09-19T09:11:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.253+0000 7feedaadf640 1 --2- 172.21.15.120:0/2954113495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9979c59a0 0x55e9979c00b0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7feecc004960 tx=0x7feecc00a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50d63b80fe9b846 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.253+0000 7feebffff640 1 -- 172.21.15.120:0/2954113495 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc010b80 con 0x55e9979c59a0 2024-09-19T09:11:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.253+0000 7feebffff640 1 -- 172.21.15.120:0/2954113495 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feecc0106f0 con 0x55e9979c59a0 2024-09-19T09:11:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.253+0000 7feebffff640 1 -- 172.21.15.120:0/2954113495 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc005740 con 0x55e9979c59a0 2024-09-19T09:11:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.254+0000 7feedf2db740 1 -- 172.21.15.120:0/2954113495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9979c59a0 msgr2=0x55e9979c00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.254+0000 7feedf2db740 1 --2- 172.21.15.120:0/2954113495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9979c59a0 0x55e9979c00b0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7feecc004960 tx=0x7feecc00a4d0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.254+0000 7feedf2db740 1 -- 172.21.15.120:0/2954113495 shutdown_connections 2024-09-19T09:11:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.254+0000 7feedf2db740 1 --2- 172.21.15.120:0/2954113495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9979cb370 0x55e9979e24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.254+0000 7feedf2db740 1 --2- 172.21.15.120:0/2954113495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9979c62e0 0x55e9979d70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.254+0000 7feedf2db740 1 --2- 172.21.15.120:0/2954113495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9979c59a0 0x55e9979c00b0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.254+0000 7feedf2db740 1 -- 172.21.15.120:0/2954113495 >> 172.21.15.120:0/2954113495 conn(0x55e9979c1010 msgr2=0x55e9979bf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.254+0000 7feedf2db740 1 -- 172.21.15.120:0/2954113495 shutdown_connections 2024-09-19T09:11:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.254+0000 7feedf2db740 1 -- 172.21.15.120:0/2954113495 wait complete. 2024-09-19T09:11:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.254+0000 7feedf2db740 1 -- start start 2024-09-19T09:11:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.254+0000 7feedf2db740 1 Processor -- start 2024-09-19T09:11:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.254+0000 7feedf2db740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9979c59a0 0x55e9979d4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.254+0000 7feedf2db740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9979c62e0 0x55e9979d4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.255+0000 7feedf2db740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9979cb370 0x55e9979d5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.255+0000 7feedf2db740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feed41ac6c0 con 0x55e9979c62e0 2024-09-19T09:11:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.255+0000 7feedf2db740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feed41abff0 con 0x55e9979c59a0 2024-09-19T09:11:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.255+0000 7feedf2db740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feed41a9210 con 0x55e9979cb370 2024-09-19T09:11:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.255+0000 7feedf2db740 10 client.0 _ll_register_callbacks cb 0x7feec4005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.255+0000 7feed9add640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9979c62e0 0x55e9979d4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.255+0000 7feedaadf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9979c59a0 0x55e9979d4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.255+0000 7feed9add640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9979c62e0 0x55e9979d4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44034/0 (socket says 172.21.15.120:44034) 2024-09-19T09:11:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.255+0000 7feed9add640 1 -- 172.21.15.120:0/4130372798 learned_addr learned my addr 172.21.15.120:0/4130372798 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.255+0000 7feedaadf640 1 --2- 172.21.15.120:0/4130372798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9979cb370 0x55e9979d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.255+0000 7feed9add640 1 -- 172.21.15.120:0/4130372798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9979cb370 msgr2=0x55e9979d5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.255+0000 7feed9add640 1 --2- 172.21.15.120:0/4130372798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9979cb370 0x55e9979d5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.255+0000 7feed9add640 1 -- 172.21.15.120:0/4130372798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9979c59a0 msgr2=0x55e9979d4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.255+0000 7feed9add640 1 --2- 172.21.15.120:0/4130372798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9979c59a0 0x55e9979d4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.255+0000 7feed9add640 1 -- 172.21.15.120:0/4130372798 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9979d61d0 con 0x55e9979c62e0 2024-09-19T09:11:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.255+0000 7feedaadf640 1 --2- 172.21.15.120:0/4130372798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9979cb370 0x55e9979d5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:11:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.255+0000 7feed9add640 1 --2- 172.21.15.120:0/4130372798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9979c62e0 0x55e9979d4c70 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7feec40035f0 tx=0x7feec4004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28159ae2a4aaa2d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.255+0000 7feebdffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:11:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.255+0000 7feebdffb640 1 -- 172.21.15.120:0/4130372798 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4035070 con 0x55e9979c62e0 2024-09-19T09:11:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.255+0000 7feebdffb640 1 -- 172.21.15.120:0/4130372798 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7feec4005410 con 0x55e9979c62e0 2024-09-19T09:11:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.255+0000 7feedf2db740 1 -- 172.21.15.120:0/4130372798 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feecc0106f0 con 0x55e9979c62e0 2024-09-19T09:11:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.255+0000 7feebdffb640 1 -- 172.21.15.120:0/4130372798 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feec4060050 con 0x55e9979c62e0 2024-09-19T09:11:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.255+0000 7feedf2db740 10 client.18405 fetch_fsmap learned FSMap version 32 2024-09-19T09:11:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.255+0000 7feedf2db740 1 -- 172.21.15.120:0/4130372798 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e997a464d0 con 0x55e9979c62e0 2024-09-19T09:11:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.256+0000 7feebdffb640 1 -- 172.21.15.120:0/4130372798 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feecc0106f0 con 0x55e9979c62e0 2024-09-19T09:11:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.256+0000 7feebdffb640 1 -- 172.21.15.120:0/4130372798 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7feec40041f0 con 0x55e9979c62e0 2024-09-19T09:11:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.256+0000 7feedf2db740 10 client.18405 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:11:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.256+0000 7feedf2db740 10 client.18405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:11:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.256+0000 7feedf2db740 20 client.18405 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.256+0000 7feedf2db740 1 --2- 172.21.15.120:0/4130372798 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e997a46e10 0x55e997a49200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.256+0000 7feedf2db740 4 client.18405 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:11:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.256+0000 7feedf2db740 1 -- 172.21.15.120:0/4130372798 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e997930d70 con 0x55e997a46e10 2024-09-19T09:11:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.256+0000 7feedaadf640 1 --2- 172.21.15.120:0/4130372798 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e997a46e10 0x55e997a49200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.256+0000 7feedaadf640 1 --2- 172.21.15.120:0/4130372798 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e997a46e10 0x55e997a49200 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bfd7cfe4e3cbb6e server_cookie=6c2fd731c0f44006 in_seq=0 out_seq=0 2024-09-19T09:11:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.256+0000 7feebdffb640 10 client.18405 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:25.327 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:24 smithi120 ceph-mon[31150]: pgmap v599: 97 pgs: 97 active+clean; 41 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.1 MiB/s wr, 398 op/s 2024-09-19T09:11:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.417+0000 7feebdffb640 1 -- 172.21.15.120:0/4130372798 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e997930d70 con 0x55e997a46e10 2024-09-19T09:11:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.417+0000 7feebdffb640 10 client.18405 handle_command_reply: tid=0 2024-09-19T09:11:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.417+0000 7feedf2db740 1 client.18405 shutdown 2024-09-19T09:11:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.417+0000 7feedf2db740 1 -- 172.21.15.120:0/4130372798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9979c62e0 msgr2=0x55e9979d4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.417+0000 7feedf2db740 1 --2- 172.21.15.120:0/4130372798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9979c62e0 0x55e9979d4c70 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7feec40035f0 tx=0x7feec4004e40 comp rx=0 tx=0).stop 2024-09-19T09:11:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.417+0000 7feedf2db740 1 -- 172.21.15.120:0/4130372798 shutdown_connections 2024-09-19T09:11:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.418+0000 7feedf2db740 1 --2- 172.21.15.120:0/4130372798 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e997a46e10 0x55e997a49200 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.418+0000 7feedf2db740 1 --2- 172.21.15.120:0/4130372798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9979cb370 0x55e9979d5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.418+0000 7feedf2db740 1 --2- 172.21.15.120:0/4130372798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9979c62e0 0x55e9979d4c70 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.418+0000 7feebdffb640 0 client.18405 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.418+0000 7feedf2db740 1 --2- 172.21.15.120:0/4130372798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9979c59a0 0x55e9979d4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.418+0000 7feedf2db740 1 -- 172.21.15.120:0/4130372798 >> 172.21.15.120:0/4130372798 conn(0x55e9979c1010 msgr2=0x55e9979c01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.418+0000 7feedf2db740 1 -- 172.21.15.120:0/4130372798 shutdown_connections 2024-09-19T09:11:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.418+0000 7feedf2db740 1 -- 172.21.15.120:0/4130372798 wait complete. 2024-09-19T09:11:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.418+0000 7feedf2db740 20 client.18405 trim_cache size 0 max 16384 2024-09-19T09:11:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:11:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:11:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:11:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:11:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.418+0000 7feedc542640 1 -- 172.21.15.120:0/1322370437 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7feec00a32f0 msgr2=0x7feec00a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.418+0000 7feedc542640 1 --2- 172.21.15.120:0/1322370437 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7feec00a32f0 0x7feec00a57b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7feed00982d0 tx=0x7feed00d2d60 comp rx=0 tx=0).stop 2024-09-19T09:11:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.418+0000 7feedc542640 1 -- 172.21.15.120:0/1322370437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feed4100a40 msgr2=0x7feed41049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.418+0000 7feedc542640 1 --2- 172.21.15.120:0/1322370437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feed4100a40 0x7feed41049d0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7feed000d950 tx=0x7feed000de10 comp rx=0 tx=0).stop 2024-09-19T09:11:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.418+0000 7feedc542640 1 -- 172.21.15.120:0/1322370437 shutdown_connections 2024-09-19T09:11:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.418+0000 7feedc542640 1 --2- 172.21.15.120:0/1322370437 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7feec00a32f0 0x7feec00a57b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.418+0000 7feedc542640 1 --2- 172.21.15.120:0/1322370437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feed41025f0 0x7feed4103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.418+0000 7feedc542640 1 --2- 172.21.15.120:0/1322370437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feed4101c30 0x7feed4103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.418+0000 7feedc542640 1 --2- 172.21.15.120:0/1322370437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feed4100a40 0x7feed41049d0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.418+0000 7feedc542640 1 -- 172.21.15.120:0/1322370437 >> 172.21.15.120:0/1322370437 conn(0x7feed40fc1e0 msgr2=0x7feed40fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.418+0000 7feedc542640 1 -- 172.21.15.120:0/1322370437 shutdown_connections 2024-09-19T09:11:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:25.418+0000 7feedc542640 1 -- 172.21.15.120:0/1322370437 wait complete. 2024-09-19T09:11:26.063 INFO:tasks.workunit.client.0.smithi120.stdout: 1 769857 9.16 MB/sec execute 321 sec latency 33.145 ms 2024-09-19T09:11:27.063 INFO:tasks.workunit.client.0.smithi120.stdout: 1 772676 9.17 MB/sec execute 322 sec latency 10.806 ms 2024-09-19T09:11:27.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:26 smithi196 ceph-mon[35150]: pgmap v600: 97 pgs: 97 active+clean; 42 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.6 MiB/s wr, 270 op/s 2024-09-19T09:11:27.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:26 smithi153 ceph-mon[35071]: pgmap v600: 97 pgs: 97 active+clean; 42 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.6 MiB/s wr, 270 op/s 2024-09-19T09:11:27.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:26 smithi120 ceph-mon[31150]: pgmap v600: 97 pgs: 97 active+clean; 42 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.6 MiB/s wr, 270 op/s 2024-09-19T09:11:28.063 INFO:tasks.workunit.client.0.smithi120.stdout: 1 775116 9.19 MB/sec execute 323 sec latency 19.137 ms 2024-09-19T09:11:29.063 INFO:tasks.workunit.client.0.smithi120.stdout: 1 778025 9.21 MB/sec execute 324 sec latency 48.639 ms 2024-09-19T09:11:29.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:28 smithi196 ceph-mon[35150]: pgmap v601: 97 pgs: 97 active+clean; 42 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.4 MiB/s wr, 255 op/s 2024-09-19T09:11:29.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:28 smithi153 ceph-mon[35071]: pgmap v601: 97 pgs: 97 active+clean; 42 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.4 MiB/s wr, 255 op/s 2024-09-19T09:11:29.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:28 smithi120 ceph-mon[31150]: pgmap v601: 97 pgs: 97 active+clean; 42 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.4 MiB/s wr, 255 op/s 2024-09-19T09:11:30.063 INFO:tasks.workunit.client.0.smithi120.stdout: 1 781253 9.24 MB/sec execute 325 sec latency 16.481 ms 2024-09-19T09:11:31.063 INFO:tasks.workunit.client.0.smithi120.stdout: 1 784070 9.25 MB/sec execute 326 sec latency 11.436 ms 2024-09-19T09:11:31.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:30 smithi196 ceph-mon[35150]: pgmap v602: 97 pgs: 97 active+clean; 44 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.9 MiB/s wr, 357 op/s 2024-09-19T09:11:31.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:30 smithi153 ceph-mon[35071]: pgmap v602: 97 pgs: 97 active+clean; 44 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.9 MiB/s wr, 357 op/s 2024-09-19T09:11:31.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:30 smithi120 ceph-mon[31150]: pgmap v602: 97 pgs: 97 active+clean; 44 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.9 MiB/s wr, 357 op/s 2024-09-19T09:11:32.063 INFO:tasks.workunit.client.0.smithi120.stdout: 1 787009 9.27 MB/sec execute 327 sec latency 14.249 ms 2024-09-19T09:11:33.063 INFO:tasks.workunit.client.0.smithi120.stdout: 1 789447 9.28 MB/sec execute 328 sec latency 14.208 ms 2024-09-19T09:11:33.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:32 smithi196 ceph-mon[35150]: pgmap v603: 97 pgs: 97 active+clean; 41 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 340 op/s 2024-09-19T09:11:33.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:32 smithi153 ceph-mon[35071]: pgmap v603: 97 pgs: 97 active+clean; 41 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 340 op/s 2024-09-19T09:11:33.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:32 smithi120 ceph-mon[31150]: pgmap v603: 97 pgs: 97 active+clean; 41 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 340 op/s 2024-09-19T09:11:34.063 INFO:tasks.workunit.client.0.smithi120.stdout: 1 792392 9.31 MB/sec execute 329 sec latency 13.353 ms 2024-09-19T09:11:35.063 INFO:tasks.workunit.client.0.smithi120.stdout: 1 795620 9.34 MB/sec execute 330 sec latency 11.407 ms 2024-09-19T09:11:35.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:34 smithi196 ceph-mon[35150]: pgmap v604: 97 pgs: 97 active+clean; 44 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.8 MiB/s wr, 363 op/s 2024-09-19T09:11:35.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:34 smithi153 ceph-mon[35071]: pgmap v604: 97 pgs: 97 active+clean; 44 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.8 MiB/s wr, 363 op/s 2024-09-19T09:11:35.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:34 smithi120 ceph-mon[31150]: pgmap v604: 97 pgs: 97 active+clean; 44 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.8 MiB/s wr, 363 op/s 2024-09-19T09:11:36.063 INFO:tasks.workunit.client.0.smithi120.stdout: 1 798307 9.35 MB/sec execute 331 sec latency 32.575 ms 2024-09-19T09:11:37.063 INFO:tasks.workunit.client.0.smithi120.stdout: 1 801046 9.35 MB/sec execute 332 sec latency 12.125 ms 2024-09-19T09:11:37.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:36 smithi196 ceph-mon[35150]: pgmap v605: 97 pgs: 97 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.6 MiB/s wr, 341 op/s 2024-09-19T09:11:37.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:36 smithi153 ceph-mon[35071]: pgmap v605: 97 pgs: 97 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.6 MiB/s wr, 341 op/s 2024-09-19T09:11:37.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:36 smithi120 ceph-mon[31150]: pgmap v605: 97 pgs: 97 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.6 MiB/s wr, 341 op/s 2024-09-19T09:11:38.063 INFO:tasks.workunit.client.0.smithi120.stdout: 1 803598 9.37 MB/sec execute 333 sec latency 10.401 ms 2024-09-19T09:11:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.507+0000 7f2906d42640 1 -- 172.21.15.120:0/2689526917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2900107880 msgr2=0x7f2900109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.507+0000 7f2906d42640 1 --2- 172.21.15.120:0/2689526917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2900107880 0x7f2900109c90 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009ca0 tx=0x7f28f802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.507+0000 7f2906d42640 1 -- 172.21.15.120:0/2689526917 shutdown_connections 2024-09-19T09:11:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.507+0000 7f2906d42640 1 --2- 172.21.15.120:0/2689526917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2900107880 0x7f2900109c90 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.507+0000 7f2906d42640 1 --2- 172.21.15.120:0/2689526917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2900104f50 0x7f2900107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.507+0000 7f2906d42640 1 --2- 172.21.15.120:0/2689526917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2900102620 0x7f2900104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.507+0000 7f2906d42640 1 -- 172.21.15.120:0/2689526917 >> 172.21.15.120:0/2689526917 conn(0x7f29000fc1e0 msgr2=0x7f29000fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.507+0000 7f2906d42640 1 -- 172.21.15.120:0/2689526917 shutdown_connections 2024-09-19T09:11:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.507+0000 7f2906d42640 1 -- 172.21.15.120:0/2689526917 wait complete. 2024-09-19T09:11:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.508+0000 7f2906d42640 1 Processor -- start 2024-09-19T09:11:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.508+0000 7f2906d42640 1 -- start start 2024-09-19T09:11:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.508+0000 7f2906d42640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2900102620 0x7f2900101f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.508+0000 7f2906d42640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2900104f50 0x7f29001005d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.508+0000 7f2906d42640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2900107880 0x7f2900100b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.508+0000 7f2906d42640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f290010d800 con 0x7f2900104f50 2024-09-19T09:11:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.508+0000 7f2906d42640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f290010d680 con 0x7f2900102620 2024-09-19T09:11:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.508+0000 7f2906d42640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f290010d980 con 0x7f2900107880 2024-09-19T09:11:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.508+0000 7f28fffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2900104f50 0x7f29001005d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.508+0000 7f28fffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2900104f50 0x7f29001005d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40624/0 (socket says 172.21.15.120:40624) 2024-09-19T09:11:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.508+0000 7f28fffff640 1 -- 172.21.15.120:0/2873160368 learned_addr learned my addr 172.21.15.120:0/2873160368 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.508+0000 7f2904ade640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2900102620 0x7f2900101f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.508+0000 7f29052df640 1 --2- 172.21.15.120:0/2873160368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2900107880 0x7f2900100b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.508+0000 7f28fffff640 1 -- 172.21.15.120:0/2873160368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2900107880 msgr2=0x7f2900100b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.508+0000 7f28fffff640 1 --2- 172.21.15.120:0/2873160368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2900107880 0x7f2900100b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.508+0000 7f28fffff640 1 -- 172.21.15.120:0/2873160368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2900102620 msgr2=0x7f2900101f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.508+0000 7f28fffff640 1 --2- 172.21.15.120:0/2873160368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2900102620 0x7f2900101f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.508+0000 7f28fffff640 1 -- 172.21.15.120:0/2873160368 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29001013a0 con 0x7f2900104f50 2024-09-19T09:11:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.508+0000 7f2904ade640 1 --2- 172.21.15.120:0/2873160368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2900102620 0x7f2900101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.508+0000 7f28fffff640 1 --2- 172.21.15.120:0/2873160368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2900104f50 0x7f29001005d0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f28ec00e9c0 tx=0x7f28ec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.508+0000 7f28fdffb640 1 -- 172.21.15.120:0/2873160368 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ec00cd80 con 0x7f2900104f50 2024-09-19T09:11:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.508+0000 7f28fdffb640 1 -- 172.21.15.120:0/2873160368 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28ec005500 con 0x7f2900104f50 2024-09-19T09:11:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.508+0000 7f2906d42640 1 -- 172.21.15.120:0/2873160368 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29001a1f20 con 0x7f2900104f50 2024-09-19T09:11:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.508+0000 7f28fdffb640 1 -- 172.21.15.120:0/2873160368 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ec010690 con 0x7f2900104f50 2024-09-19T09:11:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.508+0000 7f2906d42640 1 -- 172.21.15.120:0/2873160368 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29001a2390 con 0x7f2900104f50 2024-09-19T09:11:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.511+0000 7f28fdffb640 1 -- 172.21.15.120:0/2873160368 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f28ec0046c0 con 0x7f2900104f50 2024-09-19T09:11:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.511+0000 7f2909ada740 1 Processor -- start 2024-09-19T09:11:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.511+0000 7f2909ada740 1 -- start start 2024-09-19T09:11:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.511+0000 7f2909ada740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56086f2f39a0 0x56086f2f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.511+0000 7f2909ada740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56086f2f42e0 0x56086f2f8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.511+0000 7f2909ada740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56086f2f9370 0x56086f2fb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.511+0000 7f2909ada740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56086f2572f0 con 0x56086f2f39a0 2024-09-19T09:11:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.511+0000 7f2909ada740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56086f1ff270 con 0x56086f2f9370 2024-09-19T09:11:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.511+0000 7f2909ada740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56086f2532b0 con 0x56086f2f42e0 2024-09-19T09:11:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.511+0000 7f29052df640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56086f2f39a0 0x56086f2f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.511+0000 7f28fdffb640 1 --2- 172.21.15.120:0/2873160368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f28f00a3370 0x7f28f00a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.511+0000 7f29052df640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56086f2f39a0 0x56086f2f3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40632/0 (socket says 172.21.15.120:40632) 2024-09-19T09:11:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.511+0000 7f2904ade640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56086f2f42e0 0x56086f2f8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.511+0000 7f29052df640 1 -- 172.21.15.120:0/931219742 learned_addr learned my addr 172.21.15.120:0/931219742 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.511+0000 7f28fffff640 1 --2- 172.21.15.120:0/931219742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56086f2f9370 0x56086f2fb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.511+0000 7f29052df640 1 --2- 172.21.15.120:0/2873160368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f28f00a3370 0x7f28f00a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.511+0000 7f2904ade640 1 -- 172.21.15.120:0/931219742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56086f2f9370 msgr2=0x56086f2fb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.511+0000 7f2904ade640 1 --2- 172.21.15.120:0/931219742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56086f2f9370 0x56086f2fb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.511+0000 7f2904ade640 1 -- 172.21.15.120:0/931219742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56086f2f39a0 msgr2=0x56086f2f3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.511+0000 7f2904ade640 1 --2- 172.21.15.120:0/931219742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56086f2f39a0 0x56086f2f3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.511+0000 7f28fdffb640 1 -- 172.21.15.120:0/2873160368 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f28ec0cfc00 con 0x7f2900104f50 2024-09-19T09:11:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.511+0000 7f2904ade640 1 -- 172.21.15.120:0/931219742 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56086f1fe340 con 0x56086f2f42e0 2024-09-19T09:11:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.513+0000 7f2904ade640 1 --2- 172.21.15.120:0/931219742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56086f2f42e0 0x56086f2f8da0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009fd0 tx=0x7f28f40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=698b834ce6860fc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.513+0000 7f29052df640 1 --2- 172.21.15.120:0/2873160368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f28f00a3370 0x7f28f00a5830 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f290006a980 tx=0x7f28f802a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.513+0000 7f28eaffd640 1 -- 172.21.15.120:0/931219742 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f4004310 con 0x56086f2f42e0 2024-09-19T09:11:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.513+0000 7f28eaffd640 1 -- 172.21.15.120:0/931219742 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28f40044b0 con 0x56086f2f42e0 2024-09-19T09:11:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.513+0000 7f28eaffd640 1 -- 172.21.15.120:0/931219742 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f40035f0 con 0x56086f2f42e0 2024-09-19T09:11:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.513+0000 7f2909ada740 1 -- 172.21.15.120:0/931219742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56086f2f42e0 msgr2=0x56086f2f8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.513+0000 7f2909ada740 1 --2- 172.21.15.120:0/931219742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56086f2f42e0 0x56086f2f8da0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009fd0 tx=0x7f28f40118d0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.513+0000 7f2909ada740 1 -- 172.21.15.120:0/931219742 shutdown_connections 2024-09-19T09:11:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.513+0000 7f2909ada740 1 --2- 172.21.15.120:0/931219742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56086f2f9370 0x56086f2fb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.513+0000 7f2909ada740 1 --2- 172.21.15.120:0/931219742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56086f2f42e0 0x56086f2f8da0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.513+0000 7f2909ada740 1 --2- 172.21.15.120:0/931219742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56086f2f39a0 0x56086f2f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.513+0000 7f2909ada740 1 -- 172.21.15.120:0/931219742 >> 172.21.15.120:0/931219742 conn(0x56086f2ef010 msgr2=0x56086f2ef410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.513+0000 7f2909ada740 1 -- 172.21.15.120:0/931219742 shutdown_connections 2024-09-19T09:11:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.513+0000 7f2909ada740 1 -- 172.21.15.120:0/931219742 wait complete. 2024-09-19T09:11:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.513+0000 7f2909ada740 1 -- start start 2024-09-19T09:11:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.513+0000 7f2909ada740 1 Processor -- start 2024-09-19T09:11:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.513+0000 7f2909ada740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56086f2f39a0 0x56086f2fdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.513+0000 7f2909ada740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56086f2f42e0 0x56086f2fe0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.513+0000 7f2909ada740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56086f2f9370 0x56086f3050e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.513+0000 7f2909ada740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56086f2fc340 con 0x56086f2f9370 2024-09-19T09:11:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.513+0000 7f2909ada740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56086f1e0290 con 0x56086f2f39a0 2024-09-19T09:11:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.513+0000 7f2909ada740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56086f2fc4c0 con 0x56086f2f42e0 2024-09-19T09:11:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.513+0000 7f2909ada740 10 client.0 _ll_register_callbacks cb 0x56086f211140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.515+0000 7f2904ade640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56086f2f39a0 0x56086f2fdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.515+0000 7f28fffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56086f2f42e0 0x56086f2fe0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.515+0000 7f29052df640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56086f2f9370 0x56086f3050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.515+0000 7f28fffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56086f2f42e0 0x56086f2fe0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36678/0 (socket says 172.21.15.120:36678) 2024-09-19T09:11:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.515+0000 7f28fffff640 1 -- 172.21.15.120:0/832875381 learned_addr learned my addr 172.21.15.120:0/832875381 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.515+0000 7f2904ade640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56086f2f39a0 0x56086f2fdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:45458/0 (socket says 172.21.15.120:45458) 2024-09-19T09:11:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.515+0000 7f28fffff640 1 -- 172.21.15.120:0/832875381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56086f2f39a0 msgr2=0x56086f2fdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.515+0000 7f28fffff640 1 --2- 172.21.15.120:0/832875381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56086f2f39a0 0x56086f2fdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.515+0000 7f28fffff640 1 -- 172.21.15.120:0/832875381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56086f2f9370 msgr2=0x56086f3050e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.515+0000 7f28fffff640 1 --2- 172.21.15.120:0/832875381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56086f2f9370 0x56086f3050e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.515+0000 7f28fffff640 1 -- 172.21.15.120:0/832875381 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56086ef6a210 con 0x56086f2f42e0 2024-09-19T09:11:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.515+0000 7f28fffff640 1 --2- 172.21.15.120:0/832875381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56086f2f42e0 0x56086f2fe0a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f28ec098bf0 tx=0x7f28ec009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b76a2f945b136db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.515+0000 7f28dffff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:11:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.515+0000 7f28dffff640 1 -- 172.21.15.120:0/832875381 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ec0bae60 con 0x56086f2f42e0 2024-09-19T09:11:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.515+0000 7f28dffff640 1 -- 172.21.15.120:0/832875381 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f28ec0a4a00 con 0x56086f2f42e0 2024-09-19T09:11:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.515+0000 7f2909ada740 1 -- 172.21.15.120:0/832875381 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28f40044b0 con 0x56086f2f42e0 2024-09-19T09:11:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.515+0000 7f28dffff640 1 -- 172.21.15.120:0/832875381 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f28ec0bc270 con 0x56086f2f42e0 2024-09-19T09:11:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.515+0000 7f2909ada740 10 client.27409 fetch_fsmap learned FSMap version 32 2024-09-19T09:11:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.515+0000 7f2909ada740 1 -- 172.21.15.120:0/832875381 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56086f036620 con 0x56086f2f42e0 2024-09-19T09:11:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.515+0000 7f28dffff640 1 -- 172.21.15.120:0/832875381 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28f40044b0 con 0x56086f2f42e0 2024-09-19T09:11:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.516+0000 7f28dffff640 1 -- 172.21.15.120:0/832875381 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f28ec0cdbc0 con 0x56086f2f42e0 2024-09-19T09:11:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.516+0000 7f2909ada740 10 client.27409 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:11:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.516+0000 7f2909ada740 10 client.27409 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:11:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.516+0000 7f2909ada740 20 client.27409 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.516+0000 7f2909ada740 1 --2- 172.21.15.120:0/832875381 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56086f309820 0x56086f30bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.516+0000 7f2909ada740 4 client.27409 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:11:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.516+0000 7f2909ada740 1 -- 172.21.15.120:0/832875381 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56086f264a20 con 0x56086f309820 2024-09-19T09:11:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.516+0000 7f2904ade640 1 --2- 172.21.15.120:0/832875381 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56086f309820 0x56086f30bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.516+0000 7f28dffff640 10 client.27409 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.516+0000 7f2904ade640 1 --2- 172.21.15.120:0/832875381 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56086f309820 0x56086f30bc10 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fc974c546fd33d9 server_cookie=810f7af4ba66f81a in_seq=0 out_seq=0 2024-09-19T09:11:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.517+0000 7f28dffff640 1 -- 172.21.15.120:0/832875381 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56086f264a20 con 0x56086f309820 2024-09-19T09:11:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.517+0000 7f28dffff640 10 client.27409 handle_command_reply: tid=0 2024-09-19T09:11:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.517+0000 7f2909ada740 1 client.27409 shutdown 2024-09-19T09:11:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.518+0000 7f2909ada740 1 -- 172.21.15.120:0/832875381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56086f2f42e0 msgr2=0x56086f2fe0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.518+0000 7f2909ada740 1 --2- 172.21.15.120:0/832875381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56086f2f42e0 0x56086f2fe0a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f28ec098bf0 tx=0x7f28ec009700 comp rx=0 tx=0).stop 2024-09-19T09:11:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.518+0000 7f2909ada740 1 -- 172.21.15.120:0/832875381 shutdown_connections 2024-09-19T09:11:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.518+0000 7f2909ada740 1 --2- 172.21.15.120:0/832875381 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56086f309820 0x56086f30bc10 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.518+0000 7f2909ada740 1 --2- 172.21.15.120:0/832875381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56086f2f9370 0x56086f3050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.518+0000 7f28dffff640 0 client.27409 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.518+0000 7f2909ada740 1 --2- 172.21.15.120:0/832875381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56086f2f42e0 0x56086f2fe0a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.518+0000 7f2909ada740 1 --2- 172.21.15.120:0/832875381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56086f2f39a0 0x56086f2fdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.518+0000 7f2909ada740 1 -- 172.21.15.120:0/832875381 >> 172.21.15.120:0/832875381 conn(0x56086f2ef010 msgr2=0x56086f2f0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.518+0000 7f2909ada740 1 -- 172.21.15.120:0/832875381 shutdown_connections 2024-09-19T09:11:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.518+0000 7f2909ada740 1 -- 172.21.15.120:0/832875381 wait complete. 2024-09-19T09:11:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.518+0000 7f2909ada740 20 client.27409 trim_cache size 0 max 16384 2024-09-19T09:11:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.531+0000 7f2909ada740 1 Processor -- start 2024-09-19T09:11:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.531+0000 7f2909ada740 1 -- start start 2024-09-19T09:11:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.531+0000 7f2909ada740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56086f2f39a0 0x56086f2efec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.531+0000 7f2909ada740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56086f2f42e0 0x56086f3050a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.531+0000 7f2909ada740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56086f2f9370 0x56086f2f0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.531+0000 7f2909ada740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f29001ab1d0 con 0x56086f2f9370 2024-09-19T09:11:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.531+0000 7f2909ada740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f29001acd10 con 0x56086f2f42e0 2024-09-19T09:11:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.531+0000 7f2909ada740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f29001ab8a0 con 0x56086f2f39a0 2024-09-19T09:11:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.531+0000 7f29052df640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56086f2f9370 0x56086f2f0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.531+0000 7f29052df640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56086f2f9370 0x56086f2f0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40656/0 (socket says 172.21.15.120:40656) 2024-09-19T09:11:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.531+0000 7f29052df640 1 -- 172.21.15.120:0/2212641633 learned_addr learned my addr 172.21.15.120:0/2212641633 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.531+0000 7f28fffff640 1 --2- 172.21.15.120:0/2212641633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56086f2f42e0 0x56086f3050a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.531+0000 7f2904ade640 1 --2- 172.21.15.120:0/2212641633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56086f2f39a0 0x56086f2efec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.531+0000 7f29052df640 1 -- 172.21.15.120:0/2212641633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56086f2f39a0 msgr2=0x56086f2efec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.531+0000 7f29052df640 1 --2- 172.21.15.120:0/2212641633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56086f2f39a0 0x56086f2efec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.531+0000 7f29052df640 1 -- 172.21.15.120:0/2212641633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56086f2f42e0 msgr2=0x56086f3050a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.531+0000 7f29052df640 1 --2- 172.21.15.120:0/2212641633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56086f2f42e0 0x56086f3050a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.531+0000 7f29052df640 1 -- 172.21.15.120:0/2212641633 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56086f2ea2d0 con 0x56086f2f9370 2024-09-19T09:11:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.533+0000 7f29052df640 1 --2- 172.21.15.120:0/2212641633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56086f2f9370 0x56086f2f0400 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009ca0 tx=0x7f28f8005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a0048e5699bac5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.533+0000 7f28ea7fc640 1 -- 172.21.15.120:0/2212641633 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f8003cf0 con 0x56086f2f9370 2024-09-19T09:11:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.533+0000 7f28ea7fc640 1 -- 172.21.15.120:0/2212641633 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28f8036040 con 0x56086f2f9370 2024-09-19T09:11:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.533+0000 7f28ea7fc640 1 -- 172.21.15.120:0/2212641633 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f802fb50 con 0x56086f2f9370 2024-09-19T09:11:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.533+0000 7f2909ada740 1 -- 172.21.15.120:0/2212641633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56086f2f9370 msgr2=0x56086f2f0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.533+0000 7f2909ada740 1 --2- 172.21.15.120:0/2212641633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56086f2f9370 0x56086f2f0400 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009ca0 tx=0x7f28f8005df0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.533+0000 7f2909ada740 1 -- 172.21.15.120:0/2212641633 shutdown_connections 2024-09-19T09:11:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.533+0000 7f2909ada740 1 --2- 172.21.15.120:0/2212641633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56086f2f9370 0x56086f2f0400 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.533+0000 7f2909ada740 1 --2- 172.21.15.120:0/2212641633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56086f2f42e0 0x56086f3050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.533+0000 7f2909ada740 1 --2- 172.21.15.120:0/2212641633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56086f2f39a0 0x56086f2efec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.533+0000 7f2909ada740 1 -- 172.21.15.120:0/2212641633 >> 172.21.15.120:0/2212641633 conn(0x56086f2ef010 msgr2=0x56086f2f5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.534+0000 7f2909ada740 1 -- 172.21.15.120:0/2212641633 shutdown_connections 2024-09-19T09:11:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.534+0000 7f2909ada740 1 -- 172.21.15.120:0/2212641633 wait complete. 2024-09-19T09:11:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.534+0000 7f2909ada740 1 -- start start 2024-09-19T09:11:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.534+0000 7f2909ada740 1 Processor -- start 2024-09-19T09:11:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.534+0000 7f2909ada740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56086f2f39a0 0x56086f3024e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.534+0000 7f2909ada740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56086f2f42e0 0x56086f302a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.534+0000 7f2909ada740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56086f2f9370 0x56086f303a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.534+0000 7f2909ada740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f29001ac640 con 0x56086f2f42e0 2024-09-19T09:11:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.534+0000 7f2909ada740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f29001abf70 con 0x56086f2f9370 2024-09-19T09:11:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.534+0000 7f2909ada740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f29001a9190 con 0x56086f2f39a0 2024-09-19T09:11:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.534+0000 7f2909ada740 10 client.0 _ll_register_callbacks cb 0x7f28ec009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.534+0000 7f28fffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56086f2f42e0 0x56086f302a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.534+0000 7f2904ade640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56086f2f39a0 0x56086f3024e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.534+0000 7f28fffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56086f2f42e0 0x56086f302a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40660/0 (socket says 172.21.15.120:40660) 2024-09-19T09:11:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.534+0000 7f2904ade640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56086f2f39a0 0x56086f3024e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36700/0 (socket says 172.21.15.120:36700) 2024-09-19T09:11:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.534+0000 7f28fffff640 1 -- 172.21.15.120:0/3246793861 learned_addr learned my addr 172.21.15.120:0/3246793861 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.534+0000 7f29052df640 1 --2- 172.21.15.120:0/3246793861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56086f2f9370 0x56086f303a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.535+0000 7f28fffff640 1 -- 172.21.15.120:0/3246793861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56086f2f39a0 msgr2=0x56086f3024e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.535+0000 7f28fffff640 1 --2- 172.21.15.120:0/3246793861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56086f2f39a0 0x56086f3024e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.535+0000 7f28fffff640 1 -- 172.21.15.120:0/3246793861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56086f2f9370 msgr2=0x56086f303a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.535+0000 7f28fffff640 1 --2- 172.21.15.120:0/3246793861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56086f2f9370 0x56086f303a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.535+0000 7f28fffff640 1 -- 172.21.15.120:0/3246793861 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56086f304040 con 0x56086f2f42e0 2024-09-19T09:11:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.535+0000 7f28fffff640 1 --2- 172.21.15.120:0/3246793861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56086f2f42e0 0x56086f302a20 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0a4960 tx=0x7f28ec00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ccd62c94d27b9a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.535+0000 7f28dffff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:11:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.535+0000 7f28dffff640 1 -- 172.21.15.120:0/3246793861 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ec0bae60 con 0x56086f2f42e0 2024-09-19T09:11:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.535+0000 7f28dffff640 1 -- 172.21.15.120:0/3246793861 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f28ec09b670 con 0x56086f2f42e0 2024-09-19T09:11:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.535+0000 7f2909ada740 1 -- 172.21.15.120:0/3246793861 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28f8036040 con 0x56086f2f42e0 2024-09-19T09:11:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.535+0000 7f28dffff640 1 -- 172.21.15.120:0/3246793861 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f28ec0bc0b0 con 0x56086f2f42e0 2024-09-19T09:11:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.535+0000 7f2909ada740 10 client.18423 fetch_fsmap learned FSMap version 32 2024-09-19T09:11:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.535+0000 7f2909ada740 1 -- 172.21.15.120:0/3246793861 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56086f3053c0 con 0x56086f2f42e0 2024-09-19T09:11:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.535+0000 7f28dffff640 1 -- 172.21.15.120:0/3246793861 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28f8036040 con 0x56086f2f42e0 2024-09-19T09:11:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.535+0000 7f28dffff640 1 -- 172.21.15.120:0/3246793861 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f28ec09b0e0 con 0x56086f2f42e0 2024-09-19T09:11:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.535+0000 7f2909ada740 10 client.18423 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:11:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.535+0000 7f2909ada740 10 client.18423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:11:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.535+0000 7f2909ada740 20 client.18423 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.535+0000 7f2909ada740 1 --2- 172.21.15.120:0/3246793861 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56086f309820 0x56086f376c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.535+0000 7f2909ada740 4 client.18423 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:11:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.535+0000 7f2909ada740 1 -- 172.21.15.120:0/3246793861 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56086f25ed70 con 0x56086f309820 2024-09-19T09:11:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.536+0000 7f2904ade640 1 --2- 172.21.15.120:0/3246793861 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56086f309820 0x56086f376c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.536+0000 7f28dffff640 10 client.18423 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.536+0000 7f2904ade640 1 --2- 172.21.15.120:0/3246793861 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56086f309820 0x56086f376c50 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffb3a86fd923d8cc server_cookie=958aaad72f840dbb in_seq=0 out_seq=0 2024-09-19T09:11:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.691+0000 7f28dffff640 1 -- 172.21.15.120:0/3246793861 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56086f25ed70 con 0x56086f309820 2024-09-19T09:11:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.691+0000 7f28dffff640 10 client.18423 handle_command_reply: tid=0 2024-09-19T09:11:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.691+0000 7f2909ada740 1 client.18423 shutdown 2024-09-19T09:11:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.692+0000 7f2909ada740 1 -- 172.21.15.120:0/3246793861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56086f2f42e0 msgr2=0x56086f302a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.692+0000 7f2909ada740 1 --2- 172.21.15.120:0/3246793861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56086f2f42e0 0x56086f302a20 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0a4960 tx=0x7f28ec00cf20 comp rx=0 tx=0).stop 2024-09-19T09:11:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.692+0000 7f2909ada740 1 -- 172.21.15.120:0/3246793861 shutdown_connections 2024-09-19T09:11:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.692+0000 7f2909ada740 1 --2- 172.21.15.120:0/3246793861 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56086f309820 0x56086f376c50 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.692+0000 7f2909ada740 1 --2- 172.21.15.120:0/3246793861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56086f2f9370 0x56086f303a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.692+0000 7f2909ada740 1 --2- 172.21.15.120:0/3246793861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56086f2f42e0 0x56086f302a20 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.692+0000 7f2909ada740 1 --2- 172.21.15.120:0/3246793861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56086f2f39a0 0x56086f3024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.692+0000 7f2909ada740 1 -- 172.21.15.120:0/3246793861 >> 172.21.15.120:0/3246793861 conn(0x56086f2ef010 msgr2=0x56086f2ed9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.692+0000 7f28dffff640 0 client.18423 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.692+0000 7f2909ada740 1 -- 172.21.15.120:0/3246793861 shutdown_connections 2024-09-19T09:11:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.692+0000 7f2909ada740 1 -- 172.21.15.120:0/3246793861 wait complete. 2024-09-19T09:11:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.692+0000 7f2909ada740 20 client.18423 trim_cache size 0 max 16384 2024-09-19T09:11:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:11:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:11:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:11:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:11:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.693+0000 7f2906d42640 1 -- 172.21.15.120:0/2873160368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f28f00a3370 msgr2=0x7f28f00a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.693+0000 7f2906d42640 1 --2- 172.21.15.120:0/2873160368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f28f00a3370 0x7f28f00a5830 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f290006a980 tx=0x7f28f802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.693+0000 7f2906d42640 1 -- 172.21.15.120:0/2873160368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2900104f50 msgr2=0x7f29001005d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.693+0000 7f2906d42640 1 --2- 172.21.15.120:0/2873160368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2900104f50 0x7f29001005d0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f28ec00e9c0 tx=0x7f28ec00ee80 comp rx=0 tx=0).stop 2024-09-19T09:11:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.693+0000 7f2906d42640 1 -- 172.21.15.120:0/2873160368 shutdown_connections 2024-09-19T09:11:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.693+0000 7f2906d42640 1 --2- 172.21.15.120:0/2873160368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f28f00a3370 0x7f28f00a5830 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.693+0000 7f2906d42640 1 --2- 172.21.15.120:0/2873160368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2900107880 0x7f2900100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.693+0000 7f2906d42640 1 --2- 172.21.15.120:0/2873160368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2900104f50 0x7f29001005d0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.693+0000 7f2906d42640 1 --2- 172.21.15.120:0/2873160368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2900102620 0x7f2900101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:38.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.693+0000 7f2906d42640 1 -- 172.21.15.120:0/2873160368 >> 172.21.15.120:0/2873160368 conn(0x7f29000fc1e0 msgr2=0x7f2900102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:38.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.693+0000 7f2906d42640 1 -- 172.21.15.120:0/2873160368 shutdown_connections 2024-09-19T09:11:38.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:38.693+0000 7f2906d42640 1 -- 172.21.15.120:0/2873160368 wait complete. 2024-09-19T09:11:39.064 INFO:tasks.workunit.client.0.smithi120.stdout: 1 806336 9.40 MB/sec execute 334 sec latency 92.747 ms 2024-09-19T09:11:39.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:38 smithi196 ceph-mon[35150]: pgmap v606: 97 pgs: 97 active+clean; 50 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.9 MiB/s wr, 354 op/s 2024-09-19T09:11:39.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:38 smithi153 ceph-mon[35071]: pgmap v606: 97 pgs: 97 active+clean; 50 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.9 MiB/s wr, 354 op/s 2024-09-19T09:11:39.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:38 smithi120 ceph-mon[31150]: pgmap v606: 97 pgs: 97 active+clean; 50 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.9 MiB/s wr, 354 op/s 2024-09-19T09:11:40.064 INFO:tasks.workunit.client.0.smithi120.stdout: 1 809571 9.42 MB/sec execute 335 sec latency 17.138 ms 2024-09-19T09:11:41.064 INFO:tasks.workunit.client.0.smithi120.stdout: 1 811911 9.42 MB/sec execute 336 sec latency 54.028 ms 2024-09-19T09:11:41.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:40 smithi196 ceph-mon[35150]: pgmap v607: 97 pgs: 97 active+clean; 50 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.0 MiB/s wr, 335 op/s 2024-09-19T09:11:41.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:40 smithi153 ceph-mon[35071]: pgmap v607: 97 pgs: 97 active+clean; 50 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.0 MiB/s wr, 335 op/s 2024-09-19T09:11:41.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:40 smithi120 ceph-mon[31150]: pgmap v607: 97 pgs: 97 active+clean; 50 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.0 MiB/s wr, 335 op/s 2024-09-19T09:11:42.064 INFO:tasks.workunit.client.0.smithi120.stdout: 1 814233 9.43 MB/sec execute 337 sec latency 17.327 ms 2024-09-19T09:11:42.461 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:11:42.716 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:11:43.064 INFO:tasks.workunit.client.0.smithi120.stdout: 1 817189 9.45 MB/sec execute 338 sec latency 11.450 ms 2024-09-19T09:11:43.096 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:42 smithi120 ceph-mon[31150]: pgmap v608: 97 pgs: 97 active+clean; 51 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.2 MiB/s wr, 327 op/s 2024-09-19T09:11:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:42 smithi196 ceph-mon[35150]: pgmap v608: 97 pgs: 97 active+clean; 51 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.2 MiB/s wr, 327 op/s 2024-09-19T09:11:43.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:42 smithi153 ceph-mon[35071]: pgmap v608: 97 pgs: 97 active+clean; 51 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.2 MiB/s wr, 327 op/s 2024-09-19T09:11:43.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.330+0000 7f8b78f04640 1 -- 172.21.15.120:0/1673743217 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b74106f20 msgr2=0x7f8b7410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:43.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.330+0000 7f8b78f04640 1 --2- 172.21.15.120:0/1673743217 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b74106f20 0x7f8b7410eaa0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c009ca0 tx=0x7f8b6c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.330+0000 7f8b78f04640 1 -- 172.21.15.120:0/1673743217 shutdown_connections 2024-09-19T09:11:43.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.330+0000 7f8b78f04640 1 --2- 172.21.15.120:0/1673743217 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b74106f20 0x7f8b7410eaa0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.330+0000 7f8b78f04640 1 --2- 172.21.15.120:0/1673743217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b741027a0 0x7f8b741067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.330+0000 7f8b78f04640 1 --2- 172.21.15.120:0/1673743217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b74101df0 0x7f8b741021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.330+0000 7f8b78f04640 1 -- 172.21.15.120:0/1673743217 >> 172.21.15.120:0/1673743217 conn(0x7f8b740fd8f0 msgr2=0x7f8b740ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:43.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.330+0000 7f8b78f04640 1 -- 172.21.15.120:0/1673743217 shutdown_connections 2024-09-19T09:11:43.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.330+0000 7f8b78f04640 1 -- 172.21.15.120:0/1673743217 wait complete. 2024-09-19T09:11:43.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.331+0000 7f8b78f04640 1 Processor -- start 2024-09-19T09:11:43.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.331+0000 7f8b78f04640 1 -- start start 2024-09-19T09:11:43.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.331+0000 7f8b78f04640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b74101df0 0x7f8b74198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:43.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.331+0000 7f8b78f04640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b741027a0 0x7f8b741989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:43.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.331+0000 7f8b78f04640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b74106f20 0x7f8b7419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:43.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.331+0000 7f8b78f04640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8b74111f80 con 0x7f8b741027a0 2024-09-19T09:11:43.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.331+0000 7f8b78f04640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8b74111e00 con 0x7f8b74101df0 2024-09-19T09:11:43.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.331+0000 7f8b78f04640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8b74112100 con 0x7f8b74106f20 2024-09-19T09:11:43.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.331+0000 7f8b71d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b741027a0 0x7f8b741989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:43.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.331+0000 7f8b71d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b741027a0 0x7f8b741989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56386/0 (socket says 172.21.15.120:56386) 2024-09-19T09:11:43.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.331+0000 7f8b71d9b640 1 -- 172.21.15.120:0/3095335410 learned_addr learned my addr 172.21.15.120:0/3095335410 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:43.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.331+0000 7f8b7259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b74101df0 0x7f8b74198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:43.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.331+0000 7f8b72d9d640 1 --2- 172.21.15.120:0/3095335410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b74106f20 0x7f8b7419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:43.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.332+0000 7f8b71d9b640 1 -- 172.21.15.120:0/3095335410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b74106f20 msgr2=0x7f8b7419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:43.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.332+0000 7f8b71d9b640 1 --2- 172.21.15.120:0/3095335410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b74106f20 0x7f8b7419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.332+0000 7f8b71d9b640 1 -- 172.21.15.120:0/3095335410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b74101df0 msgr2=0x7f8b74198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:43.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.332+0000 7f8b71d9b640 1 --2- 172.21.15.120:0/3095335410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b74101df0 0x7f8b74198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.332+0000 7f8b71d9b640 1 -- 172.21.15.120:0/3095335410 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7419d430 con 0x7f8b741027a0 2024-09-19T09:11:43.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.332+0000 7f8b71d9b640 1 --2- 172.21.15.120:0/3095335410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b741027a0 0x7f8b741989c0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800d950 tx=0x7f8b6800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:43.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.332+0000 7f8b637fe640 1 -- 172.21.15.120:0/3095335410 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b68017030 con 0x7f8b741027a0 2024-09-19T09:11:43.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.332+0000 7f8b637fe640 1 -- 172.21.15.120:0/3095335410 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b68004020 con 0x7f8b741027a0 2024-09-19T09:11:43.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.332+0000 7f8b78f04640 1 -- 172.21.15.120:0/3095335410 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b7419d720 con 0x7f8b741027a0 2024-09-19T09:11:43.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.332+0000 7f8b637fe640 1 -- 172.21.15.120:0/3095335410 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b680047a0 con 0x7f8b741027a0 2024-09-19T09:11:43.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.332+0000 7f8b78f04640 1 -- 172.21.15.120:0/3095335410 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b741a5000 con 0x7f8b741027a0 2024-09-19T09:11:43.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.334+0000 7f8b637fe640 1 -- 172.21.15.120:0/3095335410 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f8b680041d0 con 0x7f8b741027a0 2024-09-19T09:11:43.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.335+0000 7f8b7b2dd740 1 Processor -- start 2024-09-19T09:11:43.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.335+0000 7f8b7b2dd740 1 -- start start 2024-09-19T09:11:43.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.335+0000 7f8b637fe640 1 --2- 172.21.15.120:0/3095335410 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8b5c0a32c0 0x7f8b5c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:43.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.335+0000 7f8b7b2dd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558647cbe4a0 0x558647cbe880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:43.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.335+0000 7f8b7b2dd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558647cbedc0 0x558647cc9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:43.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.335+0000 7f8b7b2dd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558647cc9680 0x558647ccba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:43.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.335+0000 7f8b7b2dd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558647c242e0 con 0x558647cc9680 2024-09-19T09:11:43.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.335+0000 7f8b7b2dd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558647bcbce0 con 0x558647cbe4a0 2024-09-19T09:11:43.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.335+0000 7f8b7b2dd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558647c202a0 con 0x558647cbedc0 2024-09-19T09:11:43.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.335+0000 7f8b7259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558647cbe4a0 0x558647cbe880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:43.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.335+0000 7f8b71d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558647cbedc0 0x558647cc9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:43.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.335+0000 7f8b72d9d640 1 --2- 172.21.15.120:0/3095335410 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8b5c0a32c0 0x7f8b5c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:43.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.335+0000 7f8b7259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558647cbe4a0 0x558647cbe880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:49254/0 (socket says 172.21.15.120:49254) 2024-09-19T09:11:43.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.335+0000 7f8b7259c640 1 -- 172.21.15.120:0/2720197591 learned_addr learned my addr 172.21.15.120:0/2720197591 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:43.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.335+0000 7f8b71d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558647cbedc0 0x558647cc9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:33866/0 (socket says 172.21.15.120:33866) 2024-09-19T09:11:43.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.335+0000 7f8b72d9d640 1 --2- 172.21.15.120:0/2720197591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558647cc9680 0x558647ccba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:43.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.336+0000 7f8b637fe640 1 -- 172.21.15.120:0/3095335410 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f8b680cf160 con 0x7f8b741027a0 2024-09-19T09:11:43.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.336+0000 7f8b7259c640 1 -- 172.21.15.120:0/2720197591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558647cbedc0 msgr2=0x558647cc9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:43.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.336+0000 7f8b7259c640 1 --2- 172.21.15.120:0/2720197591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558647cbedc0 0x558647cc9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.336+0000 7f8b7259c640 1 -- 172.21.15.120:0/2720197591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558647cc9680 msgr2=0x558647ccba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:43.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.336+0000 7f8b7259c640 1 --2- 172.21.15.120:0/2720197591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558647cc9680 0x558647ccba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.336+0000 7f8b7259c640 1 -- 172.21.15.120:0/2720197591 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558647bb0440 con 0x558647cbe4a0 2024-09-19T09:11:43.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.336+0000 7f8b72d9d640 1 --2- 172.21.15.120:0/3095335410 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8b5c0a32c0 0x7f8b5c0a5780 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c004af0 tx=0x7f8b6c02f4b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:43.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.336+0000 7f8b7259c640 1 --2- 172.21.15.120:0/2720197591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558647cbe4a0 0x558647cbe880 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8b64009cf0 tx=0x7f8b64011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a45a6ddfc879bd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:43.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.336+0000 7f8b60ff9640 1 -- 172.21.15.120:0/2720197591 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b64004230 con 0x558647cbe4a0 2024-09-19T09:11:43.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.336+0000 7f8b60ff9640 1 -- 172.21.15.120:0/2720197591 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b640043d0 con 0x558647cbe4a0 2024-09-19T09:11:43.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.336+0000 7f8b60ff9640 1 -- 172.21.15.120:0/2720197591 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b64019840 con 0x558647cbe4a0 2024-09-19T09:11:43.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.337+0000 7f8b7b2dd740 1 -- 172.21.15.120:0/2720197591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558647cbe4a0 msgr2=0x558647cbe880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:43.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.337+0000 7f8b7b2dd740 1 --2- 172.21.15.120:0/2720197591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558647cbe4a0 0x558647cbe880 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8b64009cf0 tx=0x7f8b64011810 comp rx=0 tx=0).stop 2024-09-19T09:11:43.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.337+0000 7f8b7b2dd740 1 -- 172.21.15.120:0/2720197591 shutdown_connections 2024-09-19T09:11:43.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.337+0000 7f8b7b2dd740 1 --2- 172.21.15.120:0/2720197591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558647cc9680 0x558647ccba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.337+0000 7f8b7b2dd740 1 --2- 172.21.15.120:0/2720197591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558647cbedc0 0x558647cc9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.337+0000 7f8b7b2dd740 1 --2- 172.21.15.120:0/2720197591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558647cbe4a0 0x558647cbe880 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.337+0000 7f8b7b2dd740 1 -- 172.21.15.120:0/2720197591 >> 172.21.15.120:0/2720197591 conn(0x558647cbaa30 msgr2=0x558647cbae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:43.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.337+0000 7f8b7b2dd740 1 -- 172.21.15.120:0/2720197591 shutdown_connections 2024-09-19T09:11:43.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.337+0000 7f8b7b2dd740 1 -- 172.21.15.120:0/2720197591 wait complete. 2024-09-19T09:11:43.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.337+0000 7f8b7b2dd740 1 -- start start 2024-09-19T09:11:43.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.338+0000 7f8b7b2dd740 1 Processor -- start 2024-09-19T09:11:43.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.338+0000 7f8b7b2dd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558647cbe4a0 0x558647cc8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:43.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.338+0000 7f8b7b2dd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558647cbedc0 0x558647cc6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:43.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.338+0000 7f8b7b2dd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558647cc9680 0x558647cc66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:43.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.338+0000 7f8b7b2dd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558647ccc250 con 0x558647cc9680 2024-09-19T09:11:43.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.338+0000 7f8b7b2dd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558647bdf050 con 0x558647cbe4a0 2024-09-19T09:11:43.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.338+0000 7f8b7b2dd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558647ccc3d0 con 0x558647cbedc0 2024-09-19T09:11:43.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.338+0000 7f8b7b2dd740 10 client.0 _ll_register_callbacks cb 0x558647c1f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:43.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.338+0000 7f8b72d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558647cc9680 0x558647cc66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:43.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.338+0000 7f8b7259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558647cbe4a0 0x558647cc8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:43.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.338+0000 7f8b7259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558647cbe4a0 0x558647cc8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:49260/0 (socket says 172.21.15.120:49260) 2024-09-19T09:11:43.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.338+0000 7f8b7259c640 1 -- 172.21.15.120:0/1295434522 learned_addr learned my addr 172.21.15.120:0/1295434522 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:43.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.338+0000 7f8b71d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558647cbedc0 0x558647cc6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:43.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.338+0000 7f8b72d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558647cc9680 0x558647cc66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56410/0 (socket says 172.21.15.120:56410) 2024-09-19T09:11:43.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.338+0000 7f8b7259c640 1 -- 172.21.15.120:0/1295434522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558647cbedc0 msgr2=0x558647cc6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:43.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.338+0000 7f8b7259c640 1 --2- 172.21.15.120:0/1295434522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558647cbedc0 0x558647cc6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.338+0000 7f8b7259c640 1 -- 172.21.15.120:0/1295434522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558647cc9680 msgr2=0x558647cc66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:43.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.338+0000 7f8b7259c640 1 --2- 172.21.15.120:0/1295434522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558647cc9680 0x558647cc66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.338+0000 7f8b7259c640 1 -- 172.21.15.120:0/1295434522 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586479953c0 con 0x558647cbe4a0 2024-09-19T09:11:43.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.338+0000 7f8b7259c640 1 --2- 172.21.15.120:0/1295434522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558647cbe4a0 0x558647cc8ee0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8b64000c00 tx=0x7f8b64005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48cd8d843ad6c56e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:43.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.338+0000 7f8b59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:11:43.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.339+0000 7f8b59ffb640 1 -- 172.21.15.120:0/1295434522 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b64027070 con 0x558647cbe4a0 2024-09-19T09:11:43.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.339+0000 7f8b7b2dd740 1 -- 172.21.15.120:0/1295434522 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b640043d0 con 0x558647cbe4a0 2024-09-19T09:11:43.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.339+0000 7f8b59ffb640 1 -- 172.21.15.120:0/1295434522 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f8b640037f0 con 0x558647cbe4a0 2024-09-19T09:11:43.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.339+0000 7f8b59ffb640 1 -- 172.21.15.120:0/1295434522 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8b64033050 con 0x558647cbe4a0 2024-09-19T09:11:43.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.339+0000 7f8b7b2dd740 10 client.27566 fetch_fsmap learned FSMap version 32 2024-09-19T09:11:43.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.339+0000 7f8b7b2dd740 1 -- 172.21.15.120:0/1295434522 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558647cd7b40 con 0x558647cbe4a0 2024-09-19T09:11:43.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.339+0000 7f8b59ffb640 1 -- 172.21.15.120:0/1295434522 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b640043d0 con 0x558647cbe4a0 2024-09-19T09:11:43.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.339+0000 7f8b59ffb640 1 -- 172.21.15.120:0/1295434522 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f8b64004ba0 con 0x558647cbe4a0 2024-09-19T09:11:43.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.339+0000 7f8b7b2dd740 10 client.27566 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:11:43.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.339+0000 7f8b7b2dd740 10 client.27566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:11:43.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.339+0000 7f8b7b2dd740 20 client.27566 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:43.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.339+0000 7f8b7b2dd740 1 --2- 172.21.15.120:0/1295434522 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558647cd8f50 0x558647cdb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:43.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.339+0000 7f8b7b2dd740 4 client.27566 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:11:43.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.339+0000 7f8b7b2dd740 1 -- 172.21.15.120:0/1295434522 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558647c31850 con 0x558647cd8f50 2024-09-19T09:11:43.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.339+0000 7f8b71d9b640 1 --2- 172.21.15.120:0/1295434522 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558647cd8f50 0x558647cdb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:43.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.340+0000 7f8b59ffb640 10 client.27566 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:43.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.340+0000 7f8b71d9b640 1 --2- 172.21.15.120:0/1295434522 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558647cd8f50 0x558647cdb340 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=920c80a69d77d8d server_cookie=20e18df76cd9136c in_seq=0 out_seq=0 2024-09-19T09:11:43.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.342+0000 7f8b59ffb640 1 -- 172.21.15.120:0/1295434522 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558647c31850 con 0x558647cd8f50 2024-09-19T09:11:43.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.342+0000 7f8b59ffb640 10 client.27566 handle_command_reply: tid=0 2024-09-19T09:11:43.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.342+0000 7f8b7b2dd740 1 client.27566 shutdown 2024-09-19T09:11:43.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.342+0000 7f8b7b2dd740 1 -- 172.21.15.120:0/1295434522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558647cbe4a0 msgr2=0x558647cc8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:43.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.342+0000 7f8b7b2dd740 1 --2- 172.21.15.120:0/1295434522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558647cbe4a0 0x558647cc8ee0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8b64000c00 tx=0x7f8b64005270 comp rx=0 tx=0).stop 2024-09-19T09:11:43.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.342+0000 7f8b7b2dd740 1 -- 172.21.15.120:0/1295434522 shutdown_connections 2024-09-19T09:11:43.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.342+0000 7f8b7b2dd740 1 --2- 172.21.15.120:0/1295434522 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558647cd8f50 0x558647cdb340 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.342+0000 7f8b7b2dd740 1 --2- 172.21.15.120:0/1295434522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558647cc9680 0x558647cc66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.342+0000 7f8b59ffb640 0 client.27566 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:43.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.342+0000 7f8b7b2dd740 1 --2- 172.21.15.120:0/1295434522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558647cbedc0 0x558647cc6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.342+0000 7f8b7b2dd740 1 --2- 172.21.15.120:0/1295434522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558647cbe4a0 0x558647cc8ee0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.342+0000 7f8b7b2dd740 1 -- 172.21.15.120:0/1295434522 >> 172.21.15.120:0/1295434522 conn(0x558647cbaa30 msgr2=0x558647cc0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:43.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.342+0000 7f8b7b2dd740 1 -- 172.21.15.120:0/1295434522 shutdown_connections 2024-09-19T09:11:43.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.342+0000 7f8b7b2dd740 1 -- 172.21.15.120:0/1295434522 wait complete. 2024-09-19T09:11:43.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.342+0000 7f8b7b2dd740 20 client.27566 trim_cache size 0 max 16384 2024-09-19T09:11:43.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.359+0000 7f8b7b2dd740 1 Processor -- start 2024-09-19T09:11:43.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.359+0000 7f8b7b2dd740 1 -- start start 2024-09-19T09:11:43.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.359+0000 7f8b7b2dd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558647cbe4a0 0x558647ce0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:43.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.360+0000 7f8b7b2dd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558647cbedc0 0x558647ce1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:43.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.360+0000 7f8b7b2dd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558647cc9680 0x558647cd1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:43.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.360+0000 7f8b7b2dd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741ab400 con 0x558647cbe4a0 2024-09-19T09:11:43.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.360+0000 7f8b7b2dd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741acf40 con 0x558647cc9680 2024-09-19T09:11:43.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.360+0000 7f8b7b2dd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741abad0 con 0x558647cbedc0 2024-09-19T09:11:43.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.360+0000 7f8b7259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558647cbe4a0 0x558647ce0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:43.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.360+0000 7f8b71d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558647cbedc0 0x558647ce1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:43.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.360+0000 7f8b71d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558647cbedc0 0x558647ce1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:33888/0 (socket says 172.21.15.120:33888) 2024-09-19T09:11:43.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.360+0000 7f8b7259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558647cbe4a0 0x558647ce0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56426/0 (socket says 172.21.15.120:56426) 2024-09-19T09:11:43.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.360+0000 7f8b71d9b640 1 -- 172.21.15.120:0/2387063150 learned_addr learned my addr 172.21.15.120:0/2387063150 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:43.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.360+0000 7f8b72d9d640 1 --2- 172.21.15.120:0/2387063150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558647cc9680 0x558647cd1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:43.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.360+0000 7f8b7259c640 1 -- 172.21.15.120:0/2387063150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558647cbedc0 msgr2=0x558647ce1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:43.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.360+0000 7f8b7259c640 1 --2- 172.21.15.120:0/2387063150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558647cbedc0 0x558647ce1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.360+0000 7f8b7259c640 1 -- 172.21.15.120:0/2387063150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558647cc9680 msgr2=0x558647cd1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:43.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.360+0000 7f8b7259c640 1 --2- 172.21.15.120:0/2387063150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558647cc9680 0x558647cd1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.360+0000 7f8b7259c640 1 -- 172.21.15.120:0/2387063150 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558647cbebf0 con 0x558647cbe4a0 2024-09-19T09:11:43.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.360+0000 7f8b71d9b640 1 --2- 172.21.15.120:0/2387063150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558647cbedc0 0x558647ce1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:43.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.360+0000 7f8b7259c640 1 --2- 172.21.15.120:0/2387063150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558647cbe4a0 0x558647ce0ad0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f8b64009cf0 tx=0x7f8b64009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843681741c021ab1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:43.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.361+0000 7f8b5bfff640 1 -- 172.21.15.120:0/2387063150 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b64030070 con 0x558647cbe4a0 2024-09-19T09:11:43.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.361+0000 7f8b5bfff640 1 -- 172.21.15.120:0/2387063150 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b640052e0 con 0x558647cbe4a0 2024-09-19T09:11:43.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.361+0000 7f8b5bfff640 1 -- 172.21.15.120:0/2387063150 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b64027030 con 0x558647cbe4a0 2024-09-19T09:11:43.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.361+0000 7f8b7b2dd740 1 -- 172.21.15.120:0/2387063150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558647cbe4a0 msgr2=0x558647ce0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:43.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.361+0000 7f8b7b2dd740 1 --2- 172.21.15.120:0/2387063150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558647cbe4a0 0x558647ce0ad0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f8b64009cf0 tx=0x7f8b64009b70 comp rx=0 tx=0).stop 2024-09-19T09:11:43.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.361+0000 7f8b7b2dd740 1 -- 172.21.15.120:0/2387063150 shutdown_connections 2024-09-19T09:11:43.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.361+0000 7f8b7b2dd740 1 --2- 172.21.15.120:0/2387063150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558647cc9680 0x558647cd1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.361+0000 7f8b7b2dd740 1 --2- 172.21.15.120:0/2387063150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558647cbedc0 0x558647ce1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.361+0000 7f8b7b2dd740 1 --2- 172.21.15.120:0/2387063150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558647cbe4a0 0x558647ce0ad0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.361+0000 7f8b7b2dd740 1 -- 172.21.15.120:0/2387063150 >> 172.21.15.120:0/2387063150 conn(0x558647cbaa30 msgr2=0x558647cc5150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:43.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.362+0000 7f8b7b2dd740 1 -- 172.21.15.120:0/2387063150 shutdown_connections 2024-09-19T09:11:43.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.362+0000 7f8b7b2dd740 1 -- 172.21.15.120:0/2387063150 wait complete. 2024-09-19T09:11:43.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.362+0000 7f8b7b2dd740 1 -- start start 2024-09-19T09:11:43.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.362+0000 7f8b7b2dd740 1 Processor -- start 2024-09-19T09:11:43.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.362+0000 7f8b7b2dd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558647cbe4a0 0x558647cd1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:43.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.362+0000 7f8b7b2dd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558647cbedc0 0x558647cced70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:43.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.362+0000 7f8b7b2dd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558647cc9680 0x558647ccf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:43.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.363+0000 7f8b7b2dd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741ac870 con 0x558647cc9680 2024-09-19T09:11:43.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.363+0000 7f8b7b2dd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741ac1a0 con 0x558647cbe4a0 2024-09-19T09:11:43.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.363+0000 7f8b7b2dd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741a93c0 con 0x558647cbedc0 2024-09-19T09:11:43.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.363+0000 7f8b7b2dd740 10 client.0 _ll_register_callbacks cb 0x7f8b6c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:43.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.363+0000 7f8b72d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558647cc9680 0x558647ccf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:43.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.363+0000 7f8b7259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558647cbe4a0 0x558647cd1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:43.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.363+0000 7f8b72d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558647cc9680 0x558647ccf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56440/0 (socket says 172.21.15.120:56440) 2024-09-19T09:11:43.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.363+0000 7f8b72d9d640 1 -- 172.21.15.120:0/2623282494 learned_addr learned my addr 172.21.15.120:0/2623282494 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:43.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.363+0000 7f8b71d9b640 1 --2- 172.21.15.120:0/2623282494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558647cbedc0 0x558647cced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:43.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.363+0000 7f8b7259c640 1 -- 172.21.15.120:0/2623282494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558647cbedc0 msgr2=0x558647cced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:43.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.363+0000 7f8b7259c640 1 --2- 172.21.15.120:0/2623282494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558647cbedc0 0x558647cced70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.363+0000 7f8b7259c640 1 -- 172.21.15.120:0/2623282494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558647cc9680 msgr2=0x558647ccf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:43.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.363+0000 7f8b7259c640 1 --2- 172.21.15.120:0/2623282494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558647cc9680 0x558647ccf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.363+0000 7f8b7259c640 1 -- 172.21.15.120:0/2623282494 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558647ccf7f0 con 0x558647cbe4a0 2024-09-19T09:11:43.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.363+0000 7f8b72d9d640 1 --2- 172.21.15.120:0/2623282494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558647cc9680 0x558647ccf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:43.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.363+0000 7f8b7259c640 1 --2- 172.21.15.120:0/2623282494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558647cbe4a0 0x558647cd1a20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8b64004f40 tx=0x7f8b64011d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec94e9309662fe54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:43.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.363+0000 7f8b59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:11:43.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.363+0000 7f8b59ffb640 1 -- 172.21.15.120:0/2623282494 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b64030070 con 0x558647cbe4a0 2024-09-19T09:11:43.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.363+0000 7f8b59ffb640 1 -- 172.21.15.120:0/2623282494 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f8b64019d10 con 0x558647cbe4a0 2024-09-19T09:11:43.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.363+0000 7f8b7b2dd740 1 -- 172.21.15.120:0/2623282494 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b640052e0 con 0x558647cbe4a0 2024-09-19T09:11:43.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.364+0000 7f8b59ffb640 1 -- 172.21.15.120:0/2623282494 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8b64047050 con 0x558647cbe4a0 2024-09-19T09:11:43.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.364+0000 7f8b7b2dd740 10 client.27572 fetch_fsmap learned FSMap version 32 2024-09-19T09:11:43.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.364+0000 7f8b7b2dd740 1 -- 172.21.15.120:0/2623282494 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558647d43770 con 0x558647cbe4a0 2024-09-19T09:11:43.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.364+0000 7f8b59ffb640 1 -- 172.21.15.120:0/2623282494 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b640052e0 con 0x558647cbe4a0 2024-09-19T09:11:43.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.364+0000 7f8b59ffb640 1 -- 172.21.15.120:0/2623282494 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f8b640046d0 con 0x558647cbe4a0 2024-09-19T09:11:43.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.364+0000 7f8b7b2dd740 10 client.27572 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:11:43.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.364+0000 7f8b7b2dd740 10 client.27572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:11:43.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.364+0000 7f8b7b2dd740 20 client.27572 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:43.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.364+0000 7f8b7b2dd740 1 --2- 172.21.15.120:0/2623282494 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558647d43d80 0x558647d46170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:43.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.364+0000 7f8b7b2dd740 4 client.27572 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:11:43.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.364+0000 7f8b7b2dd740 1 -- 172.21.15.120:0/2623282494 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558647c2bd60 con 0x558647d43d80 2024-09-19T09:11:43.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.364+0000 7f8b71d9b640 1 --2- 172.21.15.120:0/2623282494 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558647d43d80 0x558647d46170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:43.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.365+0000 7f8b71d9b640 1 --2- 172.21.15.120:0/2623282494 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558647d43d80 0x558647d46170 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82f2bdde91560fae server_cookie=3fb450902f59ffea in_seq=0 out_seq=0 2024-09-19T09:11:43.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.365+0000 7f8b59ffb640 10 client.27572 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:43.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.366+0000 7f8b59ffb640 1 -- 172.21.15.120:0/2623282494 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558647c2bd60 con 0x558647d43d80 2024-09-19T09:11:43.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.366+0000 7f8b59ffb640 10 client.27572 handle_command_reply: tid=0 2024-09-19T09:11:43.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.366+0000 7f8b7b2dd740 1 client.27572 shutdown 2024-09-19T09:11:43.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.366+0000 7f8b7b2dd740 1 -- 172.21.15.120:0/2623282494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558647cbe4a0 msgr2=0x558647cd1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:43.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.366+0000 7f8b7b2dd740 1 --2- 172.21.15.120:0/2623282494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558647cbe4a0 0x558647cd1a20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8b64004f40 tx=0x7f8b64011d80 comp rx=0 tx=0).stop 2024-09-19T09:11:43.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.366+0000 7f8b7b2dd740 1 -- 172.21.15.120:0/2623282494 shutdown_connections 2024-09-19T09:11:43.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.366+0000 7f8b7b2dd740 1 --2- 172.21.15.120:0/2623282494 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558647d43d80 0x558647d46170 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.366+0000 7f8b7b2dd740 1 --2- 172.21.15.120:0/2623282494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558647cc9680 0x558647ccf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.366+0000 7f8b7b2dd740 1 --2- 172.21.15.120:0/2623282494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558647cbedc0 0x558647cced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.366+0000 7f8b7b2dd740 1 --2- 172.21.15.120:0/2623282494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558647cbe4a0 0x558647cd1a20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.366+0000 7f8b7b2dd740 1 -- 172.21.15.120:0/2623282494 >> 172.21.15.120:0/2623282494 conn(0x558647cbaa30 msgr2=0x558647cb9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:43.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.366+0000 7f8b59ffb640 0 client.27572 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:43.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.366+0000 7f8b7b2dd740 1 -- 172.21.15.120:0/2623282494 shutdown_connections 2024-09-19T09:11:43.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.366+0000 7f8b7b2dd740 1 -- 172.21.15.120:0/2623282494 wait complete. 2024-09-19T09:11:43.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.366+0000 7f8b7b2dd740 20 client.27572 trim_cache size 0 max 16384 2024-09-19T09:11:43.372 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:11:43.372 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:11:43.372 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:11:43.372 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:11:43.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.366+0000 7f8b78f04640 1 -- 172.21.15.120:0/3095335410 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8b5c0a32c0 msgr2=0x7f8b5c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:43.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.366+0000 7f8b78f04640 1 --2- 172.21.15.120:0/3095335410 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8b5c0a32c0 0x7f8b5c0a5780 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c004af0 tx=0x7f8b6c02f4b0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.366+0000 7f8b78f04640 1 -- 172.21.15.120:0/3095335410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b741027a0 msgr2=0x7f8b741989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:43.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.366+0000 7f8b78f04640 1 --2- 172.21.15.120:0/3095335410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b741027a0 0x7f8b741989c0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800d950 tx=0x7f8b6800de10 comp rx=0 tx=0).stop 2024-09-19T09:11:43.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.367+0000 7f8b78f04640 1 -- 172.21.15.120:0/3095335410 shutdown_connections 2024-09-19T09:11:43.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.367+0000 7f8b78f04640 1 --2- 172.21.15.120:0/3095335410 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8b5c0a32c0 0x7f8b5c0a5780 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.367+0000 7f8b78f04640 1 --2- 172.21.15.120:0/3095335410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b74106f20 0x7f8b7419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.367+0000 7f8b78f04640 1 --2- 172.21.15.120:0/3095335410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b741027a0 0x7f8b741989c0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.367+0000 7f8b78f04640 1 --2- 172.21.15.120:0/3095335410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b74101df0 0x7f8b74198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:43.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.367+0000 7f8b78f04640 1 -- 172.21.15.120:0/3095335410 >> 172.21.15.120:0/3095335410 conn(0x7f8b740fd8f0 msgr2=0x7f8b740fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:43.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.367+0000 7f8b78f04640 1 -- 172.21.15.120:0/3095335410 shutdown_connections 2024-09-19T09:11:43.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:43.367+0000 7f8b78f04640 1 -- 172.21.15.120:0/3095335410 wait complete. 2024-09-19T09:11:44.064 INFO:tasks.workunit.client.0.smithi120.stdout: 1 820352 9.48 MB/sec execute 339 sec latency 22.827 ms 2024-09-19T09:11:44.298 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:11:44.298 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:11:44.554 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:11:44.981 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:44 smithi120 ceph-mon[31150]: pgmap v609: 97 pgs: 97 active+clean; 46 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.0 MiB/s wr, 432 op/s 2024-09-19T09:11:45.064 INFO:tasks.workunit.client.0.smithi120.stdout: 1 823064 9.49 MB/sec execute 340 sec latency 13.292 ms 2024-09-19T09:11:45.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.061+0000 7fc200f1c640 1 -- 172.21.15.120:0/3413486169 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1fc100e20 msgr2=0x7fc1fc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:45.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.061+0000 7fc200f1c640 1 --2- 172.21.15.120:0/3413486169 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1fc100e20 0x7fc1fc101200 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec009900 tx=0x7fc1ec02a480 comp rx=0 tx=0).stop 2024-09-19T09:11:45.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.061+0000 7fc200f1c640 1 -- 172.21.15.120:0/3413486169 shutdown_connections 2024-09-19T09:11:45.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.061+0000 7fc200f1c640 1 --2- 172.21.15.120:0/3413486169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1fc102ef0 0x7fc1fc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:45.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.061+0000 7fc200f1c640 1 --2- 172.21.15.120:0/3413486169 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1fc102570 0x7fc1fc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:45.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.061+0000 7fc200f1c640 1 --2- 172.21.15.120:0/3413486169 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1fc100e20 0x7fc1fc101200 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:45.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.061+0000 7fc200f1c640 1 -- 172.21.15.120:0/3413486169 >> 172.21.15.120:0/3413486169 conn(0x7fc1fc0fc7d0 msgr2=0x7fc1fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:45.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.061+0000 7fc200f1c640 1 -- 172.21.15.120:0/3413486169 shutdown_connections 2024-09-19T09:11:45.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.061+0000 7fc200f1c640 1 -- 172.21.15.120:0/3413486169 wait complete. 2024-09-19T09:11:45.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.062+0000 7fc200f1c640 1 Processor -- start 2024-09-19T09:11:45.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.062+0000 7fc200f1c640 1 -- start start 2024-09-19T09:11:45.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.062+0000 7fc200f1c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1fc100e20 0x7fc1fc199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:45.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.062+0000 7fc200f1c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1fc102570 0x7fc1fc199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:45.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.062+0000 7fc200f1c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1fc102ef0 0x7fc1fc1a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:45.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.062+0000 7fc200f1c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc1fc110f80 con 0x7fc1fc100e20 2024-09-19T09:11:45.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.062+0000 7fc200f1c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc1fc110e00 con 0x7fc1fc102ef0 2024-09-19T09:11:45.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.062+0000 7fc200f1c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc1fc111100 con 0x7fc1fc102570 2024-09-19T09:11:45.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.062+0000 7fc1fa59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1fc100e20 0x7fc1fc199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:45.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.062+0000 7fc1fad9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1fc102ef0 0x7fc1fc1a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:45.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.062+0000 7fc1fad9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1fc102ef0 0x7fc1fc1a43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:49294/0 (socket says 172.21.15.120:49294) 2024-09-19T09:11:45.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.062+0000 7fc1fa59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1fc100e20 0x7fc1fc199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56456/0 (socket says 172.21.15.120:56456) 2024-09-19T09:11:45.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.062+0000 7fc1fad9d640 1 -- 172.21.15.120:0/3201236841 learned_addr learned my addr 172.21.15.120:0/3201236841 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:45.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.062+0000 7fc1f3fff640 1 --2- 172.21.15.120:0/3201236841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1fc102570 0x7fc1fc199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:45.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.063+0000 7fc1fad9d640 1 -- 172.21.15.120:0/3201236841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1fc102570 msgr2=0x7fc1fc199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:45.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.063+0000 7fc1fad9d640 1 --2- 172.21.15.120:0/3201236841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1fc102570 0x7fc1fc199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:45.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.063+0000 7fc1fad9d640 1 -- 172.21.15.120:0/3201236841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1fc100e20 msgr2=0x7fc1fc199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:45.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.063+0000 7fc1fad9d640 1 --2- 172.21.15.120:0/3201236841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1fc100e20 0x7fc1fc199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:45.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.063+0000 7fc1fad9d640 1 -- 172.21.15.120:0/3201236841 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1fc1a48f0 con 0x7fc1fc102ef0 2024-09-19T09:11:45.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.063+0000 7fc1f3fff640 1 --2- 172.21.15.120:0/3201236841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1fc102570 0x7fc1fc199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:11:45.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.063+0000 7fc1fad9d640 1 --2- 172.21.15.120:0/3201236841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1fc102ef0 0x7fc1fc1a43b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400eaa0 tx=0x7fc1f400ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:45.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.063+0000 7fc1f37fe640 1 -- 172.21.15.120:0/3201236841 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f400cdb0 con 0x7fc1fc102ef0 2024-09-19T09:11:45.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.063+0000 7fc200f1c640 1 -- 172.21.15.120:0/3201236841 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1fc1a4ac0 con 0x7fc1fc102ef0 2024-09-19T09:11:45.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.063+0000 7fc1f37fe640 1 -- 172.21.15.120:0/3201236841 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1f4005020 con 0x7fc1fc102ef0 2024-09-19T09:11:45.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.063+0000 7fc1f37fe640 1 -- 172.21.15.120:0/3201236841 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f40116d0 con 0x7fc1fc102ef0 2024-09-19T09:11:45.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.063+0000 7fc200f1c640 1 -- 172.21.15.120:0/3201236841 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1fc1a5000 con 0x7fc1fc102ef0 2024-09-19T09:11:45.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.064+0000 7fc200f1c640 1 -- 172.21.15.120:0/3201236841 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1fc101200 con 0x7fc1fc102ef0 2024-09-19T09:11:45.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.066+0000 7fc1f37fe640 1 -- 172.21.15.120:0/3201236841 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fc1f40054d0 con 0x7fc1fc102ef0 2024-09-19T09:11:45.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.066+0000 7fc1f37fe640 1 --2- 172.21.15.120:0/3201236841 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc1e80a3240 0x7fc1e80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:45.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.066+0000 7fc1fa59c640 1 --2- 172.21.15.120:0/3201236841 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc1e80a3240 0x7fc1e80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:45.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.066+0000 7fc1f37fe640 1 -- 172.21.15.120:0/3201236841 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fc1f40cfe60 con 0x7fc1fc102ef0 2024-09-19T09:11:45.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.067+0000 7fc1fa59c640 1 --2- 172.21.15.120:0/3201236841 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc1e80a3240 0x7fc1e80a5700 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec009230 tx=0x7fc1ec0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:45.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.070+0000 7fc1f37fe640 1 -- 172.21.15.120:0/3201236841 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc1f4016030 con 0x7fc1fc102ef0 2024-09-19T09:11:45.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:44 smithi196 ceph-mon[35150]: pgmap v609: 97 pgs: 97 active+clean; 46 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.0 MiB/s wr, 432 op/s 2024-09-19T09:11:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:44 smithi153 ceph-mon[35071]: pgmap v609: 97 pgs: 97 active+clean; 46 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.0 MiB/s wr, 432 op/s 2024-09-19T09:11:45.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.279+0000 7fc200f1c640 1 -- 172.21.15.120:0/3201236841 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1fc109090 con 0x7fc1fc102ef0 2024-09-19T09:11:45.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.280+0000 7fc1f37fe640 1 -- 172.21.15.120:0/3201236841 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11988 (secure 0 0 0) 0x7fc1f409cb40 con 0x7fc1fc102ef0 2024-09-19T09:11:45.281 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:11:45.281 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":32,"btime":"2024-09-19T09:11:12:904684+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:11:12.904670+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27518":{"gid":27518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3214776406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3214776406},{"type":"v1","addr":"172.21.15.153:6835","nonce":3214776406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:11:45.281 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 32 2024-09-19T09:11:45.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.282+0000 7fc200f1c640 1 -- 172.21.15.120:0/3201236841 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc1e80a3240 msgr2=0x7fc1e80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:45.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.282+0000 7fc200f1c640 1 --2- 172.21.15.120:0/3201236841 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc1e80a3240 0x7fc1e80a5700 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec009230 tx=0x7fc1ec0058d0 comp rx=0 tx=0).stop 2024-09-19T09:11:45.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.282+0000 7fc200f1c640 1 -- 172.21.15.120:0/3201236841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1fc102ef0 msgr2=0x7fc1fc1a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:45.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.282+0000 7fc200f1c640 1 --2- 172.21.15.120:0/3201236841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1fc102ef0 0x7fc1fc1a43b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400eaa0 tx=0x7fc1f400ef60 comp rx=0 tx=0).stop 2024-09-19T09:11:45.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.282+0000 7fc200f1c640 1 -- 172.21.15.120:0/3201236841 shutdown_connections 2024-09-19T09:11:45.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.282+0000 7fc200f1c640 1 --2- 172.21.15.120:0/3201236841 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc1e80a3240 0x7fc1e80a5700 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:45.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.282+0000 7fc200f1c640 1 --2- 172.21.15.120:0/3201236841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1fc102ef0 0x7fc1fc1a43b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:45.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.282+0000 7fc200f1c640 1 --2- 172.21.15.120:0/3201236841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1fc102570 0x7fc1fc199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:45.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.282+0000 7fc200f1c640 1 --2- 172.21.15.120:0/3201236841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1fc100e20 0x7fc1fc199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:45.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.282+0000 7fc200f1c640 1 -- 172.21.15.120:0/3201236841 >> 172.21.15.120:0/3201236841 conn(0x7fc1fc0fc7d0 msgr2=0x7fc1fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:45.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.282+0000 7fc200f1c640 1 -- 172.21.15.120:0/3201236841 shutdown_connections 2024-09-19T09:11:45.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:45.282+0000 7fc200f1c640 1 -- 172.21.15.120:0/3201236841 wait complete. 2024-09-19T09:11:46.032 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:11:46.064 INFO:tasks.workunit.client.0.smithi120.stdout: 1 825137 9.50 MB/sec execute 341 sec latency 57.196 ms 2024-09-19T09:11:46.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:45 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3201236841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:11:46.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:45 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3201236841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:11:46.294 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:11:46.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:45 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3201236841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:11:46.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.849+0000 7fed4391e640 1 -- 172.21.15.120:0/1630695317 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed3c1027c0 msgr2=0x7fed3c106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:46.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.849+0000 7fed4391e640 1 --2- 172.21.15.120:0/1630695317 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed3c1027c0 0x7fed3c106810 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fed38009900 tx=0x7fed3802a410 comp rx=0 tx=0).stop 2024-09-19T09:11:46.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.849+0000 7fed4391e640 1 -- 172.21.15.120:0/1630695317 shutdown_connections 2024-09-19T09:11:46.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.849+0000 7fed4391e640 1 --2- 172.21.15.120:0/1630695317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed3c106f40 0x7fed3c10eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.849+0000 7fed4391e640 1 --2- 172.21.15.120:0/1630695317 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed3c1027c0 0x7fed3c106810 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.849+0000 7fed4391e640 1 --2- 172.21.15.120:0/1630695317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed3c101e10 0x7fed3c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.849+0000 7fed4391e640 1 -- 172.21.15.120:0/1630695317 >> 172.21.15.120:0/1630695317 conn(0x7fed3c0fd8d0 msgr2=0x7fed3c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:46.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.849+0000 7fed4391e640 1 -- 172.21.15.120:0/1630695317 shutdown_connections 2024-09-19T09:11:46.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.849+0000 7fed4391e640 1 -- 172.21.15.120:0/1630695317 wait complete. 2024-09-19T09:11:46.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.850+0000 7fed4391e640 1 Processor -- start 2024-09-19T09:11:46.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.850+0000 7fed4391e640 1 -- start start 2024-09-19T09:11:46.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.850+0000 7fed4391e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed3c101e10 0x7fed3c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:46.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.850+0000 7fed4391e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed3c1027c0 0x7fed3c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:46.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.850+0000 7fed4391e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed3c106f40 0x7fed3c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:46.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.850+0000 7fed4391e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c111fa0 con 0x7fed3c1027c0 2024-09-19T09:11:46.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.850+0000 7fed4391e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c111e20 con 0x7fed3c106f40 2024-09-19T09:11:46.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.850+0000 7fed4391e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c112120 con 0x7fed3c101e10 2024-09-19T09:11:46.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.850+0000 7fed40eb9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed3c1027c0 0x7fed3c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:46.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.850+0000 7fed41ebb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed3c106f40 0x7fed3c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:46.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.850+0000 7fed416ba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed3c101e10 0x7fed3c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:46.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.850+0000 7fed40eb9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed3c1027c0 0x7fed3c1989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56472/0 (socket says 172.21.15.120:56472) 2024-09-19T09:11:46.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.850+0000 7fed40eb9640 1 -- 172.21.15.120:0/197693707 learned_addr learned my addr 172.21.15.120:0/197693707 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:46.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.851+0000 7fed40eb9640 1 -- 172.21.15.120:0/197693707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed3c101e10 msgr2=0x7fed3c198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:46.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.851+0000 7fed40eb9640 1 --2- 172.21.15.120:0/197693707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed3c101e10 0x7fed3c198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.851+0000 7fed40eb9640 1 -- 172.21.15.120:0/197693707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed3c106f40 msgr2=0x7fed3c19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:46.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.851+0000 7fed40eb9640 1 --2- 172.21.15.120:0/197693707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed3c106f40 0x7fed3c19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.851+0000 7fed40eb9640 1 -- 172.21.15.120:0/197693707 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed3c19d430 con 0x7fed3c1027c0 2024-09-19T09:11:46.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.851+0000 7fed41ebb640 1 --2- 172.21.15.120:0/197693707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed3c106f40 0x7fed3c19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:46.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.851+0000 7fed40eb9640 1 --2- 172.21.15.120:0/197693707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed3c1027c0 0x7fed3c1989c0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fed38006010 tx=0x7fed38004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:46.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.851+0000 7fed2e7fc640 1 -- 172.21.15.120:0/197693707 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3803f030 con 0x7fed3c1027c0 2024-09-19T09:11:46.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.851+0000 7fed2e7fc640 1 -- 172.21.15.120:0/197693707 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed38004440 con 0x7fed3c1027c0 2024-09-19T09:11:46.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.851+0000 7fed4391e640 1 -- 172.21.15.120:0/197693707 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed3c19d6c0 con 0x7fed3c1027c0 2024-09-19T09:11:46.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.851+0000 7fed2e7fc640 1 -- 172.21.15.120:0/197693707 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38035070 con 0x7fed3c1027c0 2024-09-19T09:11:46.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.851+0000 7fed4391e640 1 -- 172.21.15.120:0/197693707 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed3c1a5000 con 0x7fed3c1027c0 2024-09-19T09:11:46.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.853+0000 7fed2e7fc640 1 -- 172.21.15.120:0/197693707 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fed38042030 con 0x7fed3c1027c0 2024-09-19T09:11:46.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.854+0000 7fed45cf7740 1 Processor -- start 2024-09-19T09:11:46.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.854+0000 7fed45cf7740 1 -- start start 2024-09-19T09:11:46.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.854+0000 7fed45cf7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0b1a70b40 0x55f0b1a6f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:46.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.854+0000 7fed45cf7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0b1a6f5d0 0x55f0b1a6fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:46.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.854+0000 7fed45cf7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0b1a7e780 0x55f0b1a80b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:46.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.854+0000 7fed45cf7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f0b19d52f0 con 0x55f0b1a7e780 2024-09-19T09:11:46.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.854+0000 7fed45cf7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f0b197ccf0 con 0x55f0b1a6f5d0 2024-09-19T09:11:46.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.854+0000 7fed45cf7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f0b19d12b0 con 0x55f0b1a70b40 2024-09-19T09:11:46.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.854+0000 7fed2e7fc640 1 --2- 172.21.15.120:0/197693707 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fed280a32c0 0x7fed280a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:46.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.854+0000 7fed416ba640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0b1a6f5d0 0x55f0b1a6fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:46.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.854+0000 7fed40eb9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0b1a7e780 0x55f0b1a80b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:46.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.854+0000 7fed416ba640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0b1a6f5d0 0x55f0b1a6fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:49308/0 (socket says 172.21.15.120:49308) 2024-09-19T09:11:46.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.854+0000 7fed416ba640 1 -- 172.21.15.120:0/1046307594 learned_addr learned my addr 172.21.15.120:0/1046307594 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:46.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.854+0000 7fed41ebb640 1 --2- 172.21.15.120:0/1046307594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0b1a70b40 0x55f0b1a6f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:46.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.854+0000 7fed41ebb640 1 --2- 172.21.15.120:0/197693707 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fed280a32c0 0x7fed280a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:46.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.854+0000 7fed416ba640 1 -- 172.21.15.120:0/1046307594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0b1a70b40 msgr2=0x55f0b1a6f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:46.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.854+0000 7fed416ba640 1 --2- 172.21.15.120:0/1046307594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0b1a70b40 0x55f0b1a6f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.854+0000 7fed416ba640 1 -- 172.21.15.120:0/1046307594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0b1a7e780 msgr2=0x55f0b1a80b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:46.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.854+0000 7fed416ba640 1 --2- 172.21.15.120:0/1046307594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0b1a7e780 0x55f0b1a80b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.854+0000 7fed416ba640 1 -- 172.21.15.120:0/1046307594 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0b19613a0 con 0x55f0b1a6f5d0 2024-09-19T09:11:46.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.854+0000 7fed2e7fc640 1 -- 172.21.15.120:0/197693707 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fed380ef870 con 0x7fed3c1027c0 2024-09-19T09:11:46.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.855+0000 7fed416ba640 1 --2- 172.21.15.120:0/1046307594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0b1a6f5d0 0x55f0b1a6fa10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fed30009d20 tx=0x7fed30011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf6bf431132cbc5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:46.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.855+0000 7fed41ebb640 1 --2- 172.21.15.120:0/197693707 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fed280a32c0 0x7fed280a5780 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fed34005c40 tx=0x7fed34009f00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:46.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.855+0000 7fed277fe640 1 -- 172.21.15.120:0/1046307594 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3001e070 con 0x55f0b1a6f5d0 2024-09-19T09:11:46.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.855+0000 7fed277fe640 1 -- 172.21.15.120:0/1046307594 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed30019b10 con 0x55f0b1a6f5d0 2024-09-19T09:11:46.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.855+0000 7fed277fe640 1 -- 172.21.15.120:0/1046307594 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3000eaf0 con 0x55f0b1a6f5d0 2024-09-19T09:11:46.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.856+0000 7fed45cf7740 1 -- 172.21.15.120:0/1046307594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0b1a6f5d0 msgr2=0x55f0b1a6fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:46.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.856+0000 7fed45cf7740 1 --2- 172.21.15.120:0/1046307594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0b1a6f5d0 0x55f0b1a6fa10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fed30009d20 tx=0x7fed30011810 comp rx=0 tx=0).stop 2024-09-19T09:11:46.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.856+0000 7fed45cf7740 1 -- 172.21.15.120:0/1046307594 shutdown_connections 2024-09-19T09:11:46.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.856+0000 7fed45cf7740 1 --2- 172.21.15.120:0/1046307594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0b1a7e780 0x55f0b1a80b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.856+0000 7fed45cf7740 1 --2- 172.21.15.120:0/1046307594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0b1a6f5d0 0x55f0b1a6fa10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.856+0000 7fed45cf7740 1 --2- 172.21.15.120:0/1046307594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0b1a70b40 0x55f0b1a6f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.856+0000 7fed45cf7740 1 -- 172.21.15.120:0/1046307594 >> 172.21.15.120:0/1046307594 conn(0x55f0b1a38310 msgr2=0x55f0b1a6bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:46.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.856+0000 7fed45cf7740 1 -- 172.21.15.120:0/1046307594 shutdown_connections 2024-09-19T09:11:46.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.856+0000 7fed45cf7740 1 -- 172.21.15.120:0/1046307594 wait complete. 2024-09-19T09:11:46.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.856+0000 7fed45cf7740 1 -- start start 2024-09-19T09:11:46.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.856+0000 7fed45cf7740 1 Processor -- start 2024-09-19T09:11:46.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.856+0000 7fed45cf7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0b1a6f5d0 0x55f0b1a7e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:46.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.856+0000 7fed45cf7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0b1a70b40 0x55f0b1a7b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:46.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.856+0000 7fed45cf7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0b1a7e780 0x55f0b1a7b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:46.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.856+0000 7fed45cf7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f0b1a81480 con 0x55f0b1a6f5d0 2024-09-19T09:11:46.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.856+0000 7fed45cf7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f0b1990060 con 0x55f0b1a7e780 2024-09-19T09:11:46.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.856+0000 7fed45cf7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f0b1a81600 con 0x55f0b1a70b40 2024-09-19T09:11:46.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.856+0000 7fed45cf7740 10 client.0 _ll_register_callbacks cb 0x55f0b16e38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:46.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.857+0000 7fed416ba640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0b1a6f5d0 0x55f0b1a7e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:46.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.857+0000 7fed41ebb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0b1a7e780 0x55f0b1a7b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:46.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.857+0000 7fed416ba640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0b1a6f5d0 0x55f0b1a7e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56490/0 (socket says 172.21.15.120:56490) 2024-09-19T09:11:46.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.857+0000 7fed416ba640 1 -- 172.21.15.120:0/23796687 learned_addr learned my addr 172.21.15.120:0/23796687 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:46.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.857+0000 7fed40eb9640 1 --2- 172.21.15.120:0/23796687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0b1a70b40 0x55f0b1a7b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:46.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.857+0000 7fed416ba640 1 -- 172.21.15.120:0/23796687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0b1a70b40 msgr2=0x55f0b1a7b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:46.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.857+0000 7fed416ba640 1 --2- 172.21.15.120:0/23796687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0b1a70b40 0x55f0b1a7b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.857+0000 7fed416ba640 1 -- 172.21.15.120:0/23796687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0b1a7e780 msgr2=0x55f0b1a7b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:46.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.857+0000 7fed416ba640 1 --2- 172.21.15.120:0/23796687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0b1a7e780 0x55f0b1a7b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.857+0000 7fed416ba640 1 -- 172.21.15.120:0/23796687 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0b17463c0 con 0x55f0b1a6f5d0 2024-09-19T09:11:46.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.857+0000 7fed40eb9640 1 --2- 172.21.15.120:0/23796687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0b1a70b40 0x55f0b1a7b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:11:46.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.857+0000 7fed416ba640 1 --2- 172.21.15.120:0/23796687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0b1a6f5d0 0x55f0b1a7e150 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fed3000e670 tx=0x7fed30009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146c147d35ba7b61 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:46.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.857+0000 7fed24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:11:46.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.857+0000 7fed24ff9640 1 -- 172.21.15.120:0/23796687 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30027030 con 0x55f0b1a6f5d0 2024-09-19T09:11:46.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.857+0000 7fed24ff9640 1 -- 172.21.15.120:0/23796687 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fed300196c0 con 0x55f0b1a6f5d0 2024-09-19T09:11:46.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.857+0000 7fed45cf7740 1 -- 172.21.15.120:0/23796687 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed30019b10 con 0x55f0b1a6f5d0 2024-09-19T09:11:46.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.857+0000 7fed24ff9640 1 -- 172.21.15.120:0/23796687 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fed3001e070 con 0x55f0b1a6f5d0 2024-09-19T09:11:46.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.857+0000 7fed45cf7740 10 client.18459 fetch_fsmap learned FSMap version 32 2024-09-19T09:11:46.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.857+0000 7fed45cf7740 1 -- 172.21.15.120:0/23796687 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f0b1a8cef0 con 0x55f0b1a6f5d0 2024-09-19T09:11:46.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.858+0000 7fed24ff9640 1 -- 172.21.15.120:0/23796687 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed30019b10 con 0x55f0b1a6f5d0 2024-09-19T09:11:46.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.858+0000 7fed24ff9640 1 -- 172.21.15.120:0/23796687 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fed3000e870 con 0x55f0b1a6f5d0 2024-09-19T09:11:46.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.858+0000 7fed45cf7740 10 client.18459 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:11:46.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.858+0000 7fed45cf7740 10 client.18459 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:11:46.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.858+0000 7fed45cf7740 20 client.18459 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:46.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.858+0000 7fed45cf7740 1 --2- 172.21.15.120:0/23796687 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55f0b1a8e2f0 0x55f0b1a906e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:46.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.858+0000 7fed45cf7740 4 client.18459 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:11:46.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.858+0000 7fed45cf7740 1 -- 172.21.15.120:0/23796687 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0b19e2860 con 0x55f0b1a8e2f0 2024-09-19T09:11:46.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.858+0000 7fed40eb9640 1 --2- 172.21.15.120:0/23796687 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55f0b1a8e2f0 0x55f0b1a906e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:46.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.858+0000 7fed24ff9640 10 client.18459 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:11:46.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.858+0000 7fed40eb9640 1 --2- 172.21.15.120:0/23796687 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55f0b1a8e2f0 0x55f0b1a906e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=75f05a5f95babf53 server_cookie=7f7e2026e4eb0b4b in_seq=0 out_seq=0 2024-09-19T09:11:46.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.859+0000 7fed24ff9640 1 -- 172.21.15.120:0/23796687 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f0b19e2860 con 0x55f0b1a8e2f0 2024-09-19T09:11:46.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.859+0000 7fed24ff9640 10 client.18459 handle_command_reply: tid=0 2024-09-19T09:11:46.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.859+0000 7fed45cf7740 1 client.18459 shutdown 2024-09-19T09:11:46.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.859+0000 7fed45cf7740 1 -- 172.21.15.120:0/23796687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0b1a6f5d0 msgr2=0x55f0b1a7e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:46.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.859+0000 7fed45cf7740 1 --2- 172.21.15.120:0/23796687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0b1a6f5d0 0x55f0b1a7e150 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fed3000e670 tx=0x7fed30009910 comp rx=0 tx=0).stop 2024-09-19T09:11:46.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.859+0000 7fed45cf7740 1 -- 172.21.15.120:0/23796687 shutdown_connections 2024-09-19T09:11:46.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.859+0000 7fed45cf7740 1 --2- 172.21.15.120:0/23796687 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55f0b1a8e2f0 0x55f0b1a906e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.859+0000 7fed45cf7740 1 --2- 172.21.15.120:0/23796687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0b1a7e780 0x55f0b1a7b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.859+0000 7fed45cf7740 1 --2- 172.21.15.120:0/23796687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0b1a70b40 0x55f0b1a7b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.859+0000 7fed45cf7740 1 --2- 172.21.15.120:0/23796687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0b1a6f5d0 0x55f0b1a7e150 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.859+0000 7fed45cf7740 1 -- 172.21.15.120:0/23796687 >> 172.21.15.120:0/23796687 conn(0x55f0b1a38310 msgr2=0x55f0b1a71960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:46.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.859+0000 7fed24ff9640 0 client.18459 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:11:46.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.859+0000 7fed45cf7740 1 -- 172.21.15.120:0/23796687 shutdown_connections 2024-09-19T09:11:46.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.859+0000 7fed45cf7740 1 -- 172.21.15.120:0/23796687 wait complete. 2024-09-19T09:11:46.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.860+0000 7fed45cf7740 20 client.18459 trim_cache size 0 max 16384 2024-09-19T09:11:46.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.873+0000 7fed45cf7740 1 Processor -- start 2024-09-19T09:11:46.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.873+0000 7fed45cf7740 1 -- start start 2024-09-19T09:11:46.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.873+0000 7fed45cf7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0b1a6f5d0 0x55f0b1a8a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:46.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.873+0000 7fed45cf7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0b1a70b40 0x55f0b1a7a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:46.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.873+0000 7fed45cf7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0b1a7e780 0x55f0b1a95e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:46.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.873+0000 7fed45cf7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1ab390 con 0x55f0b1a7e780 2024-09-19T09:11:46.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.873+0000 7fed45cf7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1aced0 con 0x55f0b1a70b40 2024-09-19T09:11:46.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.873+0000 7fed45cf7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1aba60 con 0x55f0b1a6f5d0 2024-09-19T09:11:46.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.873+0000 7fed41ebb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0b1a7e780 0x55f0b1a95e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:46.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.874+0000 7fed41ebb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0b1a7e780 0x55f0b1a95e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56504/0 (socket says 172.21.15.120:56504) 2024-09-19T09:11:46.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.874+0000 7fed40eb9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0b1a70b40 0x55f0b1a7a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:46.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.874+0000 7fed41ebb640 1 -- 172.21.15.120:0/3799617666 learned_addr learned my addr 172.21.15.120:0/3799617666 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:46.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.874+0000 7fed416ba640 1 --2- 172.21.15.120:0/3799617666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0b1a6f5d0 0x55f0b1a8a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:46.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.874+0000 7fed41ebb640 1 -- 172.21.15.120:0/3799617666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0b1a6f5d0 msgr2=0x55f0b1a8a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:46.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.874+0000 7fed41ebb640 1 --2- 172.21.15.120:0/3799617666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0b1a6f5d0 0x55f0b1a8a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.874+0000 7fed41ebb640 1 -- 172.21.15.120:0/3799617666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0b1a70b40 msgr2=0x55f0b1a7a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:46.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.874+0000 7fed41ebb640 1 --2- 172.21.15.120:0/3799617666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0b1a70b40 0x55f0b1a7a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.874+0000 7fed41ebb640 1 -- 172.21.15.120:0/3799617666 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0b1a70740 con 0x55f0b1a7e780 2024-09-19T09:11:46.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.874+0000 7fed40eb9640 1 --2- 172.21.15.120:0/3799617666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0b1a70b40 0x55f0b1a7a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:46.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.874+0000 7fed41ebb640 1 --2- 172.21.15.120:0/3799617666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0b1a7e780 0x55f0b1a95e50 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fed34011dc0 tx=0x7fed3401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9193902a176f8b02 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:46.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.874+0000 7fed26ffd640 1 -- 172.21.15.120:0/3799617666 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3400ed80 con 0x55f0b1a7e780 2024-09-19T09:11:46.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.874+0000 7fed26ffd640 1 -- 172.21.15.120:0/3799617666 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed34005020 con 0x55f0b1a7e780 2024-09-19T09:11:46.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.874+0000 7fed26ffd640 1 -- 172.21.15.120:0/3799617666 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3402b6e0 con 0x55f0b1a7e780 2024-09-19T09:11:46.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.875+0000 7fed45cf7740 1 -- 172.21.15.120:0/3799617666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0b1a7e780 msgr2=0x55f0b1a95e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:46.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.875+0000 7fed45cf7740 1 --2- 172.21.15.120:0/3799617666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0b1a7e780 0x55f0b1a95e50 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fed34011dc0 tx=0x7fed3401d000 comp rx=0 tx=0).stop 2024-09-19T09:11:46.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.875+0000 7fed45cf7740 1 -- 172.21.15.120:0/3799617666 shutdown_connections 2024-09-19T09:11:46.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.875+0000 7fed45cf7740 1 --2- 172.21.15.120:0/3799617666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0b1a7e780 0x55f0b1a95e50 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.875+0000 7fed45cf7740 1 --2- 172.21.15.120:0/3799617666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0b1a70b40 0x55f0b1a7a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.875+0000 7fed45cf7740 1 --2- 172.21.15.120:0/3799617666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0b1a6f5d0 0x55f0b1a8a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.875+0000 7fed45cf7740 1 -- 172.21.15.120:0/3799617666 >> 172.21.15.120:0/3799617666 conn(0x55f0b1a38310 msgr2=0x55f0b1a79cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:46.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.875+0000 7fed45cf7740 1 -- 172.21.15.120:0/3799617666 shutdown_connections 2024-09-19T09:11:46.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.875+0000 7fed45cf7740 1 -- 172.21.15.120:0/3799617666 wait complete. 2024-09-19T09:11:46.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.875+0000 7fed45cf7740 1 -- start start 2024-09-19T09:11:46.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.875+0000 7fed45cf7740 1 Processor -- start 2024-09-19T09:11:46.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.876+0000 7fed45cf7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0b1a6f5d0 0x55f0b1a86df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:46.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.876+0000 7fed45cf7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0b1a70b40 0x55f0b1a77fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:46.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.876+0000 7fed45cf7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0b1a7e780 0x55f0b1a784f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:46.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.876+0000 7fed45cf7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1ac800 con 0x55f0b1a70b40 2024-09-19T09:11:46.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.876+0000 7fed45cf7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1ac130 con 0x55f0b1a7e780 2024-09-19T09:11:46.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.876+0000 7fed45cf7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1a9350 con 0x55f0b1a6f5d0 2024-09-19T09:11:46.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.876+0000 7fed45cf7740 10 client.0 _ll_register_callbacks cb 0x7fed3001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:46.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.876+0000 7fed40eb9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0b1a70b40 0x55f0b1a77fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:46.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.876+0000 7fed416ba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0b1a6f5d0 0x55f0b1a86df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:46.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.876+0000 7fed41ebb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0b1a7e780 0x55f0b1a784f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:46.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.876+0000 7fed40eb9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0b1a70b40 0x55f0b1a77fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56512/0 (socket says 172.21.15.120:56512) 2024-09-19T09:11:46.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.876+0000 7fed416ba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0b1a6f5d0 0x55f0b1a86df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:33974/0 (socket says 172.21.15.120:33974) 2024-09-19T09:11:46.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.876+0000 7fed40eb9640 1 -- 172.21.15.120:0/3343519191 learned_addr learned my addr 172.21.15.120:0/3343519191 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:46.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.876+0000 7fed40eb9640 1 -- 172.21.15.120:0/3343519191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0b1a6f5d0 msgr2=0x55f0b1a86df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:46.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.876+0000 7fed40eb9640 1 --2- 172.21.15.120:0/3343519191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0b1a6f5d0 0x55f0b1a86df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.876+0000 7fed40eb9640 1 -- 172.21.15.120:0/3343519191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0b1a7e780 msgr2=0x55f0b1a784f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:46.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.876+0000 7fed40eb9640 1 --2- 172.21.15.120:0/3343519191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0b1a7e780 0x55f0b1a784f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.876+0000 7fed40eb9640 1 -- 172.21.15.120:0/3343519191 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0b1a78b30 con 0x55f0b1a70b40 2024-09-19T09:11:46.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.876+0000 7fed40eb9640 1 --2- 172.21.15.120:0/3343519191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0b1a70b40 0x55f0b1a77fb0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fed380b88d0 tx=0x7fed380c6040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b006aead4d540867 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:46.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.876+0000 7fed24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:11:46.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.877+0000 7fed24ff9640 1 -- 172.21.15.120:0/3343519191 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38049040 con 0x55f0b1a70b40 2024-09-19T09:11:46.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.877+0000 7fed24ff9640 1 -- 172.21.15.120:0/3343519191 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fed38003cd0 con 0x55f0b1a70b40 2024-09-19T09:11:46.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.877+0000 7fed45cf7740 1 -- 172.21.15.120:0/3343519191 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed34005020 con 0x55f0b1a70b40 2024-09-19T09:11:46.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.877+0000 7fed24ff9640 1 -- 172.21.15.120:0/3343519191 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fed380c0200 con 0x55f0b1a70b40 2024-09-19T09:11:46.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.877+0000 7fed45cf7740 10 client.18465 fetch_fsmap learned FSMap version 32 2024-09-19T09:11:46.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.877+0000 7fed45cf7740 1 -- 172.21.15.120:0/3343519191 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f0b1af8cd0 con 0x55f0b1a70b40 2024-09-19T09:11:46.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.877+0000 7fed24ff9640 1 -- 172.21.15.120:0/3343519191 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed34005020 con 0x55f0b1a70b40 2024-09-19T09:11:46.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.877+0000 7fed24ff9640 1 -- 172.21.15.120:0/3343519191 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fed380c4630 con 0x55f0b1a70b40 2024-09-19T09:11:46.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.877+0000 7fed45cf7740 10 client.18465 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:11:46.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.877+0000 7fed45cf7740 10 client.18465 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:11:46.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.877+0000 7fed45cf7740 20 client.18465 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:46.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.877+0000 7fed45cf7740 1 --2- 172.21.15.120:0/3343519191 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55f0b1af93a0 0x55f0b1afb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:46.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.877+0000 7fed45cf7740 4 client.18465 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:11:46.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.877+0000 7fed45cf7740 1 -- 172.21.15.120:0/3343519191 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f0b19dcd70 con 0x55f0b1af93a0 2024-09-19T09:11:46.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.877+0000 7fed416ba640 1 --2- 172.21.15.120:0/3343519191 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55f0b1af93a0 0x55f0b1afb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:46.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.878+0000 7fed24ff9640 10 client.18465 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:11:46.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.878+0000 7fed416ba640 1 --2- 172.21.15.120:0/3343519191 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55f0b1af93a0 0x55f0b1afb790 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=734a6b59687392e0 server_cookie=bd852fc36ff3a89a in_seq=0 out_seq=0 2024-09-19T09:11:46.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.878+0000 7fed24ff9640 1 -- 172.21.15.120:0/3343519191 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f0b19dcd70 con 0x55f0b1af93a0 2024-09-19T09:11:46.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.878+0000 7fed24ff9640 10 client.18465 handle_command_reply: tid=0 2024-09-19T09:11:46.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.878+0000 7fed45cf7740 1 client.18465 shutdown 2024-09-19T09:11:46.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.878+0000 7fed45cf7740 1 -- 172.21.15.120:0/3343519191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0b1a70b40 msgr2=0x55f0b1a77fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:46.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.878+0000 7fed45cf7740 1 --2- 172.21.15.120:0/3343519191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0b1a70b40 0x55f0b1a77fb0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fed380b88d0 tx=0x7fed380c6040 comp rx=0 tx=0).stop 2024-09-19T09:11:46.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.878+0000 7fed45cf7740 1 -- 172.21.15.120:0/3343519191 shutdown_connections 2024-09-19T09:11:46.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.878+0000 7fed45cf7740 1 --2- 172.21.15.120:0/3343519191 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55f0b1af93a0 0x55f0b1afb790 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.878+0000 7fed45cf7740 1 --2- 172.21.15.120:0/3343519191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0b1a7e780 0x55f0b1a784f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.878+0000 7fed45cf7740 1 --2- 172.21.15.120:0/3343519191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0b1a70b40 0x55f0b1a77fb0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.878+0000 7fed45cf7740 1 --2- 172.21.15.120:0/3343519191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0b1a6f5d0 0x55f0b1a86df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.878+0000 7fed45cf7740 1 -- 172.21.15.120:0/3343519191 >> 172.21.15.120:0/3343519191 conn(0x55f0b1a38310 msgr2=0x55f0b1a7a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:46.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.878+0000 7fed24ff9640 0 client.18465 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:11:46.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.878+0000 7fed45cf7740 1 -- 172.21.15.120:0/3343519191 shutdown_connections 2024-09-19T09:11:46.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.878+0000 7fed45cf7740 1 -- 172.21.15.120:0/3343519191 wait complete. 2024-09-19T09:11:46.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.878+0000 7fed45cf7740 20 client.18465 trim_cache size 0 max 16384 2024-09-19T09:11:46.893 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:11:46.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.879+0000 7fed4391e640 1 -- 172.21.15.120:0/197693707 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fed280a32c0 msgr2=0x7fed280a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:46.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.879+0000 7fed4391e640 1 --2- 172.21.15.120:0/197693707 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fed280a32c0 0x7fed280a5780 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fed34005c40 tx=0x7fed34009f00 comp rx=0 tx=0).stop 2024-09-19T09:11:46.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.879+0000 7fed4391e640 1 -- 172.21.15.120:0/197693707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed3c1027c0 msgr2=0x7fed3c1989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:46.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.879+0000 7fed4391e640 1 --2- 172.21.15.120:0/197693707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed3c1027c0 0x7fed3c1989c0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fed38006010 tx=0x7fed38004040 comp rx=0 tx=0).stop 2024-09-19T09:11:46.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.879+0000 7fed4391e640 1 -- 172.21.15.120:0/197693707 shutdown_connections 2024-09-19T09:11:46.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.879+0000 7fed4391e640 1 --2- 172.21.15.120:0/197693707 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fed280a32c0 0x7fed280a5780 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.879+0000 7fed4391e640 1 --2- 172.21.15.120:0/197693707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed3c106f40 0x7fed3c19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.879+0000 7fed4391e640 1 --2- 172.21.15.120:0/197693707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed3c1027c0 0x7fed3c1989c0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.879+0000 7fed4391e640 1 --2- 172.21.15.120:0/197693707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed3c101e10 0x7fed3c198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:46.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.879+0000 7fed4391e640 1 -- 172.21.15.120:0/197693707 >> 172.21.15.120:0/197693707 conn(0x7fed3c0fd8d0 msgr2=0x7fed3c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:46.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.879+0000 7fed4391e640 1 -- 172.21.15.120:0/197693707 shutdown_connections 2024-09-19T09:11:46.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:46.879+0000 7fed4391e640 1 -- 172.21.15.120:0/197693707 wait complete. 2024-09-19T09:11:47.064 INFO:tasks.workunit.client.0.smithi120.stdout: 1 827693 9.51 MB/sec execute 342 sec latency 31.566 ms 2024-09-19T09:11:47.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:46 smithi120 ceph-mon[31150]: pgmap v610: 97 pgs: 97 active+clean; 46 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.0 MiB/s wr, 402 op/s 2024-09-19T09:11:47.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:46 smithi196 ceph-mon[35150]: pgmap v610: 97 pgs: 97 active+clean; 46 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.0 MiB/s wr, 402 op/s 2024-09-19T09:11:47.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:46 smithi153 ceph-mon[35071]: pgmap v610: 97 pgs: 97 active+clean; 46 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.0 MiB/s wr, 402 op/s 2024-09-19T09:11:47.695 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:11:47.992 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:11:48.064 INFO:tasks.workunit.client.0.smithi120.stdout: 1 830868 9.54 MB/sec execute 343 sec latency 13.108 ms 2024-09-19T09:11:48.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.673+0000 7fa42cb00640 1 -- 172.21.15.120:0/608741881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa428102dd0 msgr2=0x7fa4281031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:48.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.673+0000 7fa42cb00640 1 --2- 172.21.15.120:0/608741881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa428102dd0 0x7fa4281031b0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fa420009930 tx=0x7fa42002a420 comp rx=0 tx=0).stop 2024-09-19T09:11:48.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.673+0000 7fa42cb00640 1 -- 172.21.15.120:0/608741881 shutdown_connections 2024-09-19T09:11:48.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.673+0000 7fa42cb00640 1 --2- 172.21.15.120:0/608741881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa428108300 0x7fa42810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.673+0000 7fa42cb00640 1 --2- 172.21.15.120:0/608741881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa428103780 0x7fa428107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.673+0000 7fa42cb00640 1 --2- 172.21.15.120:0/608741881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa428102dd0 0x7fa4281031b0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.673+0000 7fa42cb00640 1 -- 172.21.15.120:0/608741881 >> 172.21.15.120:0/608741881 conn(0x7fa4280fc7d0 msgr2=0x7fa4280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:48.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.674+0000 7fa42cb00640 1 -- 172.21.15.120:0/608741881 shutdown_connections 2024-09-19T09:11:48.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.674+0000 7fa42cb00640 1 -- 172.21.15.120:0/608741881 wait complete. 2024-09-19T09:11:48.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.674+0000 7fa42cb00640 1 Processor -- start 2024-09-19T09:11:48.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.674+0000 7fa42cb00640 1 -- start start 2024-09-19T09:11:48.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.675+0000 7fa42cb00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa428102dd0 0x7fa428100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:48.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.675+0000 7fa42cb00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa428103780 0x7fa428100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:48.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.675+0000 7fa42cb00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa428108300 0x7fa4281014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:48.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.675+0000 7fa42cb00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa428112090 con 0x7fa428108300 2024-09-19T09:11:48.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.675+0000 7fa42cb00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa428111f10 con 0x7fa428103780 2024-09-19T09:11:48.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.675+0000 7fa42cb00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa428112210 con 0x7fa428102dd0 2024-09-19T09:11:48.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.675+0000 7fa42659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa428102dd0 0x7fa428100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:48.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.675+0000 7fa42659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa428102dd0 0x7fa428100a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:33988/0 (socket says 172.21.15.120:33988) 2024-09-19T09:11:48.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.675+0000 7fa42659c640 1 -- 172.21.15.120:0/1231842321 learned_addr learned my addr 172.21.15.120:0/1231842321 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:48.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.675+0000 7fa426d9d640 1 --2- 172.21.15.120:0/1231842321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa428108300 0x7fa4281014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:48.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.675+0000 7fa425d9b640 1 --2- 172.21.15.120:0/1231842321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa428103780 0x7fa428100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:48.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.675+0000 7fa42659c640 1 -- 172.21.15.120:0/1231842321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa428103780 msgr2=0x7fa428100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:48.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.675+0000 7fa42659c640 1 --2- 172.21.15.120:0/1231842321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa428103780 0x7fa428100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.675+0000 7fa42659c640 1 -- 172.21.15.120:0/1231842321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa428108300 msgr2=0x7fa4281014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:48.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.675+0000 7fa42659c640 1 --2- 172.21.15.120:0/1231842321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa428108300 0x7fa4281014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.675+0000 7fa42659c640 1 -- 172.21.15.120:0/1231842321 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4281a6b10 con 0x7fa428102dd0 2024-09-19T09:11:48.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.675+0000 7fa42659c640 1 --2- 172.21.15.120:0/1231842321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa428102dd0 0x7fa428100a30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa420009230 tx=0x7fa420004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:48.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.675+0000 7fa4177fe640 1 -- 172.21.15.120:0/1231842321 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42003f030 con 0x7fa428102dd0 2024-09-19T09:11:48.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.676+0000 7fa42cb00640 1 -- 172.21.15.120:0/1231842321 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4281a6da0 con 0x7fa428102dd0 2024-09-19T09:11:48.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.676+0000 7fa4177fe640 1 -- 172.21.15.120:0/1231842321 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa420003ad0 con 0x7fa428102dd0 2024-09-19T09:11:48.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.676+0000 7fa4177fe640 1 -- 172.21.15.120:0/1231842321 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420035070 con 0x7fa428102dd0 2024-09-19T09:11:48.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.676+0000 7fa42cb00640 1 -- 172.21.15.120:0/1231842321 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4281a7220 con 0x7fa428102dd0 2024-09-19T09:11:48.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.678+0000 7fa4177fe640 1 -- 172.21.15.120:0/1231842321 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa420042030 con 0x7fa428102dd0 2024-09-19T09:11:48.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.678+0000 7fa42eed9740 1 Processor -- start 2024-09-19T09:11:48.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.678+0000 7fa42eed9740 1 -- start start 2024-09-19T09:11:48.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.678+0000 7fa42eed9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642ddea8b40 0x5642ddea7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:48.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.679+0000 7fa42eed9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642ddea75d0 0x5642ddea7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:48.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.679+0000 7fa42eed9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642ddeb2370 0x5642ddeb4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:48.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.679+0000 7fa42eed9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5642dde0d2f0 con 0x5642ddeb2370 2024-09-19T09:11:48.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.679+0000 7fa42eed9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5642dddb4cf0 con 0x5642ddea75d0 2024-09-19T09:11:48.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.679+0000 7fa42eed9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5642dde092b0 con 0x5642ddea8b40 2024-09-19T09:11:48.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.679+0000 7fa426d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642ddea8b40 0x5642ddea7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:48.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.679+0000 7fa425d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642ddeb2370 0x5642ddeb4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:48.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.679+0000 7fa426d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642ddea8b40 0x5642ddea7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:34000/0 (socket says 172.21.15.120:34000) 2024-09-19T09:11:48.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.679+0000 7fa426d9d640 1 -- 172.21.15.120:0/2511301925 learned_addr learned my addr 172.21.15.120:0/2511301925 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:48.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.679+0000 7fa42659c640 1 --2- 172.21.15.120:0/2511301925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642ddea75d0 0x5642ddea7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:48.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.679+0000 7fa4177fe640 1 --2- 172.21.15.120:0/1231842321 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa4100a3240 0x7fa4100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:48.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.679+0000 7fa426d9d640 1 -- 172.21.15.120:0/2511301925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642ddea75d0 msgr2=0x5642ddea7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:48.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.679+0000 7fa426d9d640 1 --2- 172.21.15.120:0/2511301925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642ddea75d0 0x5642ddea7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.679+0000 7fa426d9d640 1 -- 172.21.15.120:0/2511301925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642ddeb2370 msgr2=0x5642ddeb4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:48.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.679+0000 7fa426d9d640 1 --2- 172.21.15.120:0/2511301925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642ddeb2370 0x5642ddeb4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.679+0000 7fa426d9d640 1 -- 172.21.15.120:0/2511301925 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642ddd993a0 con 0x5642ddea8b40 2024-09-19T09:11:48.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.679+0000 7fa425d9b640 1 --2- 172.21.15.120:0/2511301925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642ddeb2370 0x5642ddeb4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:48.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.679+0000 7fa426d9d640 1 --2- 172.21.15.120:0/1231842321 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa4100a3240 0x7fa4100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:48.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.679+0000 7fa4177fe640 1 -- 172.21.15.120:0/1231842321 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa4200efea0 con 0x7fa428102dd0 2024-09-19T09:11:48.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.679+0000 7fa426d9d640 1 --2- 172.21.15.120:0/2511301925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642ddea8b40 0x5642ddea7090 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa41800b780 tx=0x7fa41800bc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee3433db11a3d4c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:48.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.679+0000 7fa414ff9640 1 -- 172.21.15.120:0/2511301925 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa418024070 con 0x5642ddea8b40 2024-09-19T09:11:48.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.680+0000 7fa414ff9640 1 -- 172.21.15.120:0/2511301925 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa41801e960 con 0x5642ddea8b40 2024-09-19T09:11:48.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.680+0000 7fa414ff9640 1 -- 172.21.15.120:0/2511301925 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa418011860 con 0x5642ddea8b40 2024-09-19T09:11:48.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.680+0000 7fa426d9d640 1 --2- 172.21.15.120:0/1231842321 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa4100a3240 0x7fa4100a5700 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa418021000 tx=0x7fa41801e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:48.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.680+0000 7fa42eed9740 1 -- 172.21.15.120:0/2511301925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642ddea8b40 msgr2=0x5642ddea7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:48.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.680+0000 7fa42eed9740 1 --2- 172.21.15.120:0/2511301925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642ddea8b40 0x5642ddea7090 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa41800b780 tx=0x7fa41800bc40 comp rx=0 tx=0).stop 2024-09-19T09:11:48.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.680+0000 7fa42eed9740 1 -- 172.21.15.120:0/2511301925 shutdown_connections 2024-09-19T09:11:48.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.680+0000 7fa42eed9740 1 --2- 172.21.15.120:0/2511301925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642ddeb2370 0x5642ddeb4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.680+0000 7fa42eed9740 1 --2- 172.21.15.120:0/2511301925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642ddea75d0 0x5642ddea7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.680+0000 7fa42eed9740 1 --2- 172.21.15.120:0/2511301925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642ddea8b40 0x5642ddea7090 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.680+0000 7fa42eed9740 1 -- 172.21.15.120:0/2511301925 >> 172.21.15.120:0/2511301925 conn(0x5642dde70310 msgr2=0x5642ddea3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:48.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.680+0000 7fa42eed9740 1 -- 172.21.15.120:0/2511301925 shutdown_connections 2024-09-19T09:11:48.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.680+0000 7fa42eed9740 1 -- 172.21.15.120:0/2511301925 wait complete. 2024-09-19T09:11:48.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.681+0000 7fa42eed9740 1 -- start start 2024-09-19T09:11:48.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.681+0000 7fa42eed9740 1 Processor -- start 2024-09-19T09:11:48.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.681+0000 7fa42eed9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642ddea75d0 0x5642ddeb1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:48.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.681+0000 7fa42eed9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642ddea8b40 0x5642ddeaf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:48.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.681+0000 7fa42eed9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642ddeb2370 0x5642ddeaf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:48.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.681+0000 7fa42eed9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5642ddeb5070 con 0x5642ddea75d0 2024-09-19T09:11:48.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.681+0000 7fa42eed9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5642dddc8060 con 0x5642ddeb2370 2024-09-19T09:11:48.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.681+0000 7fa42eed9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5642ddeb51f0 con 0x5642ddea8b40 2024-09-19T09:11:48.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.681+0000 7fa42eed9740 10 client.0 _ll_register_callbacks cb 0x5642ddb1b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:48.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.681+0000 7fa42659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642ddea75d0 0x5642ddeb1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:48.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.681+0000 7fa42659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642ddea75d0 0x5642ddeb1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56548/0 (socket says 172.21.15.120:56548) 2024-09-19T09:11:48.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.681+0000 7fa42659c640 1 -- 172.21.15.120:0/3414837579 learned_addr learned my addr 172.21.15.120:0/3414837579 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:48.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.681+0000 7fa425d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642ddea8b40 0x5642ddeaf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:48.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.681+0000 7fa426d9d640 1 --2- 172.21.15.120:0/3414837579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642ddeb2370 0x5642ddeaf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:48.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.682+0000 7fa42659c640 1 -- 172.21.15.120:0/3414837579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642ddea8b40 msgr2=0x5642ddeaf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:48.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.682+0000 7fa42659c640 1 --2- 172.21.15.120:0/3414837579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642ddea8b40 0x5642ddeaf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.682+0000 7fa42659c640 1 -- 172.21.15.120:0/3414837579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642ddeb2370 msgr2=0x5642ddeaf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:48.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.682+0000 7fa42659c640 1 --2- 172.21.15.120:0/3414837579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642ddeb2370 0x5642ddeaf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.682+0000 7fa42659c640 1 -- 172.21.15.120:0/3414837579 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642ddb7e3c0 con 0x5642ddea75d0 2024-09-19T09:11:48.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.682+0000 7fa42659c640 1 --2- 172.21.15.120:0/3414837579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642ddea75d0 0x5642ddeb1d50 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fa4200ed9a0 tx=0x7fa4200b8210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3123d7fd8c1afff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:48.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.682+0000 7fa40dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:11:48.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.682+0000 7fa40dffb640 1 -- 172.21.15.120:0/3414837579 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4200ede60 con 0x5642ddea75d0 2024-09-19T09:11:48.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.682+0000 7fa40dffb640 1 -- 172.21.15.120:0/3414837579 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa42003bd60 con 0x5642ddea75d0 2024-09-19T09:11:48.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.682+0000 7fa42eed9740 1 -- 172.21.15.120:0/3414837579 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa41801e960 con 0x5642ddea75d0 2024-09-19T09:11:48.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.682+0000 7fa40dffb640 1 -- 172.21.15.120:0/3414837579 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa420049070 con 0x5642ddea75d0 2024-09-19T09:11:48.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.682+0000 7fa42eed9740 10 client.18477 fetch_fsmap learned FSMap version 32 2024-09-19T09:11:48.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.682+0000 7fa42eed9740 1 -- 172.21.15.120:0/3414837579 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5642ddec0af0 con 0x5642ddea75d0 2024-09-19T09:11:48.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.682+0000 7fa40dffb640 1 -- 172.21.15.120:0/3414837579 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa41801e960 con 0x5642ddea75d0 2024-09-19T09:11:48.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.682+0000 7fa40dffb640 1 -- 172.21.15.120:0/3414837579 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa4200b8be0 con 0x5642ddea75d0 2024-09-19T09:11:48.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.682+0000 7fa42eed9740 10 client.18477 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:11:48.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.682+0000 7fa42eed9740 10 client.18477 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:11:48.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.683+0000 7fa42eed9740 20 client.18477 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:48.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.683+0000 7fa42eed9740 1 --2- 172.21.15.120:0/3414837579 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5642ddec1ee0 0x5642ddec42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:48.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.683+0000 7fa42eed9740 4 client.18477 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:11:48.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.683+0000 7fa42eed9740 1 -- 172.21.15.120:0/3414837579 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642dde1a860 con 0x5642ddec1ee0 2024-09-19T09:11:48.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.683+0000 7fa425d9b640 1 --2- 172.21.15.120:0/3414837579 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5642ddec1ee0 0x5642ddec42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:48.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.683+0000 7fa425d9b640 1 --2- 172.21.15.120:0/3414837579 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5642ddec1ee0 0x5642ddec42f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4bef49ece600f9ff server_cookie=a05a74df23f69722 in_seq=0 out_seq=0 2024-09-19T09:11:48.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.683+0000 7fa40dffb640 10 client.18477 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:11:48.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.684+0000 7fa40dffb640 1 -- 172.21.15.120:0/3414837579 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5642dde1a860 con 0x5642ddec1ee0 2024-09-19T09:11:48.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.684+0000 7fa40dffb640 10 client.18477 handle_command_reply: tid=0 2024-09-19T09:11:48.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.684+0000 7fa42eed9740 1 client.18477 shutdown 2024-09-19T09:11:48.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.684+0000 7fa42eed9740 1 -- 172.21.15.120:0/3414837579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642ddea75d0 msgr2=0x5642ddeb1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:48.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.684+0000 7fa42eed9740 1 --2- 172.21.15.120:0/3414837579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642ddea75d0 0x5642ddeb1d50 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fa4200ed9a0 tx=0x7fa4200b8210 comp rx=0 tx=0).stop 2024-09-19T09:11:48.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.684+0000 7fa42eed9740 1 -- 172.21.15.120:0/3414837579 shutdown_connections 2024-09-19T09:11:48.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.684+0000 7fa42eed9740 1 --2- 172.21.15.120:0/3414837579 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5642ddec1ee0 0x5642ddec42f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.684+0000 7fa42eed9740 1 --2- 172.21.15.120:0/3414837579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642ddeb2370 0x5642ddeaf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.684+0000 7fa42eed9740 1 --2- 172.21.15.120:0/3414837579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642ddea8b40 0x5642ddeaf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.684+0000 7fa40dffb640 0 client.18477 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:11:48.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.684+0000 7fa42eed9740 1 --2- 172.21.15.120:0/3414837579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642ddea75d0 0x5642ddeb1d50 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.684+0000 7fa42eed9740 1 -- 172.21.15.120:0/3414837579 >> 172.21.15.120:0/3414837579 conn(0x5642dde70310 msgr2=0x5642ddea9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:48.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.684+0000 7fa42eed9740 1 -- 172.21.15.120:0/3414837579 shutdown_connections 2024-09-19T09:11:48.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.684+0000 7fa42eed9740 1 -- 172.21.15.120:0/3414837579 wait complete. 2024-09-19T09:11:48.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.684+0000 7fa42eed9740 20 client.18477 trim_cache size 0 max 16384 2024-09-19T09:11:48.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.698+0000 7fa42eed9740 1 Processor -- start 2024-09-19T09:11:48.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.699+0000 7fa42eed9740 1 -- start start 2024-09-19T09:11:48.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.699+0000 7fa42eed9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642ddea75d0 0x5642ddeae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:48.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.699+0000 7fa42eed9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642ddea8b40 0x5642ddeaa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:48.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.699+0000 7fa42eed9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642ddeb2370 0x5642ddeaa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:48.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.699+0000 7fa42eed9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa4281af790 con 0x5642ddea8b40 2024-09-19T09:11:48.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.699+0000 7fa42eed9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa4281b12d0 con 0x5642ddea75d0 2024-09-19T09:11:48.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.699+0000 7fa42eed9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa4281afe60 con 0x5642ddeb2370 2024-09-19T09:11:48.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.699+0000 7fa425d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642ddea8b40 0x5642ddeaa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:48.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.699+0000 7fa425d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642ddea8b40 0x5642ddeaa120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56562/0 (socket says 172.21.15.120:56562) 2024-09-19T09:11:48.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.699+0000 7fa42659c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642ddea75d0 0x5642ddeae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:48.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.699+0000 7fa425d9b640 1 -- 172.21.15.120:0/2269328368 learned_addr learned my addr 172.21.15.120:0/2269328368 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:48.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.699+0000 7fa426d9d640 1 --2- 172.21.15.120:0/2269328368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642ddeb2370 0x5642ddeaa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:48.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.699+0000 7fa425d9b640 1 -- 172.21.15.120:0/2269328368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642ddeb2370 msgr2=0x5642ddeaa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:48.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.699+0000 7fa425d9b640 1 --2- 172.21.15.120:0/2269328368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642ddeb2370 0x5642ddeaa660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.699+0000 7fa425d9b640 1 -- 172.21.15.120:0/2269328368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642ddea75d0 msgr2=0x5642ddeae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:48.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.699+0000 7fa425d9b640 1 --2- 172.21.15.120:0/2269328368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642ddea75d0 0x5642ddeae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.699+0000 7fa425d9b640 1 -- 172.21.15.120:0/2269328368 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642ddeadfb0 con 0x5642ddea8b40 2024-09-19T09:11:48.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.699+0000 7fa425d9b640 1 --2- 172.21.15.120:0/2269328368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642ddea8b40 0x5642ddeaa120 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fa41c011690 tx=0x7fa41c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0fca60fd204a14d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:48.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.699+0000 7fa40ffff640 1 -- 172.21.15.120:0/2269328368 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c004b20 con 0x5642ddea8b40 2024-09-19T09:11:48.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.700+0000 7fa40ffff640 1 -- 172.21.15.120:0/2269328368 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa41c004cc0 con 0x5642ddea8b40 2024-09-19T09:11:48.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.700+0000 7fa40ffff640 1 -- 172.21.15.120:0/2269328368 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c005550 con 0x5642ddea8b40 2024-09-19T09:11:48.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.700+0000 7fa42eed9740 1 -- 172.21.15.120:0/2269328368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642ddea8b40 msgr2=0x5642ddeaa120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:48.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.700+0000 7fa42eed9740 1 --2- 172.21.15.120:0/2269328368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642ddea8b40 0x5642ddeaa120 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fa41c011690 tx=0x7fa41c009500 comp rx=0 tx=0).stop 2024-09-19T09:11:48.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.700+0000 7fa42eed9740 1 -- 172.21.15.120:0/2269328368 shutdown_connections 2024-09-19T09:11:48.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.700+0000 7fa42eed9740 1 --2- 172.21.15.120:0/2269328368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642ddeb2370 0x5642ddeaa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.700+0000 7fa42eed9740 1 --2- 172.21.15.120:0/2269328368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642ddea8b40 0x5642ddeaa120 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.700+0000 7fa42eed9740 1 --2- 172.21.15.120:0/2269328368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642ddea75d0 0x5642ddeae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.700+0000 7fa42eed9740 1 -- 172.21.15.120:0/2269328368 >> 172.21.15.120:0/2269328368 conn(0x5642dde70310 msgr2=0x5642ddea2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:48.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.700+0000 7fa42eed9740 1 -- 172.21.15.120:0/2269328368 shutdown_connections 2024-09-19T09:11:48.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.700+0000 7fa42eed9740 1 -- 172.21.15.120:0/2269328368 wait complete. 2024-09-19T09:11:48.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.701+0000 7fa42eed9740 1 -- start start 2024-09-19T09:11:48.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.701+0000 7fa42eed9740 1 Processor -- start 2024-09-19T09:11:48.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.701+0000 7fa42eed9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642ddea75d0 0x5642ddeb64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:48.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.701+0000 7fa42eed9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642ddea8b40 0x5642ddeb6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:48.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.701+0000 7fa42eed9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642ddeb2370 0x5642ddeb6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:48.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.701+0000 7fa42eed9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa4281b0c00 con 0x5642ddea8b40 2024-09-19T09:11:48.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.701+0000 7fa42eed9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa4281b0530 con 0x5642ddeb2370 2024-09-19T09:11:48.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.701+0000 7fa42eed9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa4281ad750 con 0x5642ddea75d0 2024-09-19T09:11:48.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.701+0000 7fa42eed9740 10 client.0 _ll_register_callbacks cb 0x7fa420009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:48.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.701+0000 7fa425d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642ddea8b40 0x5642ddeb6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:48.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.701+0000 7fa425d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642ddea8b40 0x5642ddeb6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56572/0 (socket says 172.21.15.120:56572) 2024-09-19T09:11:48.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.701+0000 7fa42659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642ddea75d0 0x5642ddeb64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:48.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.701+0000 7fa425d9b640 1 -- 172.21.15.120:0/156238351 learned_addr learned my addr 172.21.15.120:0/156238351 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:48.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.701+0000 7fa426d9d640 1 --2- 172.21.15.120:0/156238351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642ddeb2370 0x5642ddeb6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:48.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.701+0000 7fa425d9b640 1 -- 172.21.15.120:0/156238351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642ddea75d0 msgr2=0x5642ddeb64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:48.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.701+0000 7fa425d9b640 1 --2- 172.21.15.120:0/156238351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642ddea75d0 0x5642ddeb64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.701+0000 7fa425d9b640 1 -- 172.21.15.120:0/156238351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642ddeb2370 msgr2=0x5642ddeb6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:48.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.701+0000 7fa425d9b640 1 --2- 172.21.15.120:0/156238351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642ddeb2370 0x5642ddeb6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.701+0000 7fa425d9b640 1 -- 172.21.15.120:0/156238351 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642ddf2be40 con 0x5642ddea8b40 2024-09-19T09:11:48.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.702+0000 7fa425d9b640 1 --2- 172.21.15.120:0/156238351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642ddea8b40 0x5642ddeb6a00 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fa41c005230 tx=0x7fa41c003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4833d8b9671974c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:48.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.702+0000 7fa40dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:11:48.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.702+0000 7fa40dffb640 1 -- 172.21.15.120:0/156238351 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c0054d0 con 0x5642ddea8b40 2024-09-19T09:11:48.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.702+0000 7fa40dffb640 1 -- 172.21.15.120:0/156238351 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa41c005670 con 0x5642ddea8b40 2024-09-19T09:11:48.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.702+0000 7fa42eed9740 1 -- 172.21.15.120:0/156238351 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa41c004cc0 con 0x5642ddea8b40 2024-09-19T09:11:48.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.702+0000 7fa40dffb640 1 -- 172.21.15.120:0/156238351 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa41c038050 con 0x5642ddea8b40 2024-09-19T09:11:48.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.702+0000 7fa42eed9740 10 client.18483 fetch_fsmap learned FSMap version 32 2024-09-19T09:11:48.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.702+0000 7fa42eed9740 1 -- 172.21.15.120:0/156238351 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5642ddf2d120 con 0x5642ddea8b40 2024-09-19T09:11:48.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.702+0000 7fa40dffb640 1 -- 172.21.15.120:0/156238351 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa41c004cc0 con 0x5642ddea8b40 2024-09-19T09:11:48.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.702+0000 7fa40dffb640 1 -- 172.21.15.120:0/156238351 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa41c026540 con 0x5642ddea8b40 2024-09-19T09:11:48.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.702+0000 7fa42eed9740 10 client.18483 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:11:48.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.703+0000 7fa42eed9740 10 client.18483 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:11:48.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.703+0000 7fa42eed9740 20 client.18483 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:48.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.703+0000 7fa42eed9740 1 --2- 172.21.15.120:0/156238351 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5642ddf2d710 0x5642ddf2fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:48.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.703+0000 7fa42eed9740 4 client.18483 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:11:48.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.703+0000 7fa42eed9740 1 -- 172.21.15.120:0/156238351 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5642dde14d70 con 0x5642ddf2d710 2024-09-19T09:11:48.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.703+0000 7fa42659c640 1 --2- 172.21.15.120:0/156238351 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5642ddf2d710 0x5642ddf2fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:48.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.703+0000 7fa42659c640 1 --2- 172.21.15.120:0/156238351 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5642ddf2d710 0x5642ddf2fb30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=79b1c8b1a718de87 server_cookie=daa489ebe62712fb in_seq=0 out_seq=0 2024-09-19T09:11:48.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.703+0000 7fa40dffb640 10 client.18483 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:11:48.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.703+0000 7fa40dffb640 1 -- 172.21.15.120:0/156238351 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5642dde14d70 con 0x5642ddf2d710 2024-09-19T09:11:48.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.703+0000 7fa40dffb640 10 client.18483 handle_command_reply: tid=0 2024-09-19T09:11:48.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.703+0000 7fa42eed9740 1 client.18483 shutdown 2024-09-19T09:11:48.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.703+0000 7fa42eed9740 1 -- 172.21.15.120:0/156238351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642ddea8b40 msgr2=0x5642ddeb6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:48.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.703+0000 7fa42eed9740 1 --2- 172.21.15.120:0/156238351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642ddea8b40 0x5642ddeb6a00 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fa41c005230 tx=0x7fa41c003a00 comp rx=0 tx=0).stop 2024-09-19T09:11:48.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.703+0000 7fa42eed9740 1 -- 172.21.15.120:0/156238351 shutdown_connections 2024-09-19T09:11:48.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.703+0000 7fa42eed9740 1 --2- 172.21.15.120:0/156238351 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5642ddf2d710 0x5642ddf2fb30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.704+0000 7fa40dffb640 0 client.18483 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:11:48.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.704+0000 7fa42eed9740 1 --2- 172.21.15.120:0/156238351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642ddeb2370 0x5642ddeb6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.704+0000 7fa42eed9740 1 --2- 172.21.15.120:0/156238351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642ddea8b40 0x5642ddeb6a00 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.704+0000 7fa42eed9740 1 --2- 172.21.15.120:0/156238351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642ddea75d0 0x5642ddeb64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.704+0000 7fa42eed9740 1 -- 172.21.15.120:0/156238351 >> 172.21.15.120:0/156238351 conn(0x5642dde70310 msgr2=0x5642ddea2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:48.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.704+0000 7fa42eed9740 1 -- 172.21.15.120:0/156238351 shutdown_connections 2024-09-19T09:11:48.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.704+0000 7fa42eed9740 1 -- 172.21.15.120:0/156238351 wait complete. 2024-09-19T09:11:48.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.704+0000 7fa42eed9740 20 client.18483 trim_cache size 0 max 16384 2024-09-19T09:11:48.710 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:11:48.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.704+0000 7fa42cb00640 1 -- 172.21.15.120:0/1231842321 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa4100a3240 msgr2=0x7fa4100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:48.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.704+0000 7fa42cb00640 1 --2- 172.21.15.120:0/1231842321 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa4100a3240 0x7fa4100a5700 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa418021000 tx=0x7fa41801e000 comp rx=0 tx=0).stop 2024-09-19T09:11:48.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.704+0000 7fa42cb00640 1 -- 172.21.15.120:0/1231842321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa428102dd0 msgr2=0x7fa428100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:48.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.704+0000 7fa42cb00640 1 --2- 172.21.15.120:0/1231842321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa428102dd0 0x7fa428100a30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa420009230 tx=0x7fa420004270 comp rx=0 tx=0).stop 2024-09-19T09:11:48.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.704+0000 7fa42cb00640 1 -- 172.21.15.120:0/1231842321 shutdown_connections 2024-09-19T09:11:48.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.704+0000 7fa42cb00640 1 --2- 172.21.15.120:0/1231842321 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa4100a3240 0x7fa4100a5700 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.704+0000 7fa42cb00640 1 --2- 172.21.15.120:0/1231842321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa428108300 0x7fa4281014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.704+0000 7fa42cb00640 1 --2- 172.21.15.120:0/1231842321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa428103780 0x7fa428100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.704+0000 7fa42cb00640 1 --2- 172.21.15.120:0/1231842321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa428102dd0 0x7fa428100a30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:48.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.704+0000 7fa42cb00640 1 -- 172.21.15.120:0/1231842321 >> 172.21.15.120:0/1231842321 conn(0x7fa4280fc7d0 msgr2=0x7fa428105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:48.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.704+0000 7fa42cb00640 1 -- 172.21.15.120:0/1231842321 shutdown_connections 2024-09-19T09:11:48.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:48.704+0000 7fa42cb00640 1 -- 172.21.15.120:0/1231842321 wait complete. 2024-09-19T09:11:49.064 INFO:tasks.workunit.client.0.smithi120.stdout: 1 833334 9.54 MB/sec execute 344 sec latency 37.594 ms 2024-09-19T09:11:49.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:48 smithi120 ceph-mon[31150]: pgmap v611: 97 pgs: 97 active+clean; 47 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.0 MiB/s wr, 361 op/s 2024-09-19T09:11:49.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:48 smithi196 ceph-mon[35150]: pgmap v611: 97 pgs: 97 active+clean; 47 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.0 MiB/s wr, 361 op/s 2024-09-19T09:11:49.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:48 smithi153 ceph-mon[35071]: pgmap v611: 97 pgs: 97 active+clean; 47 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.0 MiB/s wr, 361 op/s 2024-09-19T09:11:49.594 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:11:49.855 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:11:50.064 INFO:tasks.workunit.client.0.smithi120.stdout: 1 835849 9.55 MB/sec execute 345 sec latency 11.631 ms 2024-09-19T09:11:50.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.388+0000 7f0e0d970640 1 -- 172.21.15.120:0/136407596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e08101dd0 msgr2=0x7f0e081021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:50.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.388+0000 7f0e0d970640 1 --2- 172.21.15.120:0/136407596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e08101dd0 0x7f0e081021b0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009900 tx=0x7f0df802a410 comp rx=0 tx=0).stop 2024-09-19T09:11:50.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.388+0000 7f0e0d970640 1 -- 172.21.15.120:0/136407596 shutdown_connections 2024-09-19T09:11:50.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.388+0000 7f0e0d970640 1 --2- 172.21.15.120:0/136407596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e08106f00 0x7f0e0810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.388+0000 7f0e0d970640 1 --2- 172.21.15.120:0/136407596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e08102780 0x7f0e081067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.388+0000 7f0e0d970640 1 --2- 172.21.15.120:0/136407596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e08101dd0 0x7f0e081021b0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.388+0000 7f0e0d970640 1 -- 172.21.15.120:0/136407596 >> 172.21.15.120:0/136407596 conn(0x7f0e080fd8b0 msgr2=0x7f0e080ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:50.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.388+0000 7f0e0d970640 1 -- 172.21.15.120:0/136407596 shutdown_connections 2024-09-19T09:11:50.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.388+0000 7f0e0d970640 1 -- 172.21.15.120:0/136407596 wait complete. 2024-09-19T09:11:50.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.389+0000 7f0e0d970640 1 Processor -- start 2024-09-19T09:11:50.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.389+0000 7f0e0d970640 1 -- start start 2024-09-19T09:11:50.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.390+0000 7f0e0d970640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e08101dd0 0x7f0e08198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:50.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.390+0000 7f0e0d970640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e08102780 0x7f0e081989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:50.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.390+0000 7f0e0d970640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e08106f00 0x7f0e0819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:50.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.390+0000 7f0e0d970640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0e08111f60 con 0x7f0e08102780 2024-09-19T09:11:50.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.390+0000 7f0e0d970640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0e08111de0 con 0x7f0e08106f00 2024-09-19T09:11:50.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.390+0000 7f0e0d970640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e081120e0 con 0x7f0e08101dd0 2024-09-19T09:11:50.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.390+0000 7f0dfffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e08102780 0x7f0e081989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:50.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.390+0000 7f0e06ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e08101dd0 0x7f0e08198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:50.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.390+0000 7f0e06ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e08101dd0 0x7f0e08198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:34038/0 (socket says 172.21.15.120:34038) 2024-09-19T09:11:50.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.390+0000 7f0e06ffd640 1 -- 172.21.15.120:0/2033571274 learned_addr learned my addr 172.21.15.120:0/2033571274 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:50.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.390+0000 7f0e077fe640 1 --2- 172.21.15.120:0/2033571274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e08106f00 0x7f0e0819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:50.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.390+0000 7f0e06ffd640 1 -- 172.21.15.120:0/2033571274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e08106f00 msgr2=0x7f0e0819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:50.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.390+0000 7f0e06ffd640 1 --2- 172.21.15.120:0/2033571274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e08106f00 0x7f0e0819cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.390+0000 7f0e06ffd640 1 -- 172.21.15.120:0/2033571274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e08102780 msgr2=0x7f0e081989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:50.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.390+0000 7f0e06ffd640 1 --2- 172.21.15.120:0/2033571274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e08102780 0x7f0e081989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.390+0000 7f0e06ffd640 1 -- 172.21.15.120:0/2033571274 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e0819d430 con 0x7f0e08101dd0 2024-09-19T09:11:50.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.390+0000 7f0dfffff640 1 --2- 172.21.15.120:0/2033571274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e08102780 0x7f0e081989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:11:50.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.390+0000 7f0e06ffd640 1 --2- 172.21.15.120:0/2033571274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e08101dd0 0x7f0e08198480 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0df8006010 tx=0x7f0df80042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:50.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.391+0000 7f0e0c96e640 1 -- 172.21.15.120:0/2033571274 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df803f030 con 0x7f0e08101dd0 2024-09-19T09:11:50.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.391+0000 7f0e0c96e640 1 -- 172.21.15.120:0/2033571274 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0df8003ce0 con 0x7f0e08101dd0 2024-09-19T09:11:50.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.391+0000 7f0e0d970640 1 -- 172.21.15.120:0/2033571274 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e0819d6c0 con 0x7f0e08101dd0 2024-09-19T09:11:50.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.391+0000 7f0e0c96e640 1 -- 172.21.15.120:0/2033571274 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df8035070 con 0x7f0e08101dd0 2024-09-19T09:11:50.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.391+0000 7f0e0d970640 1 -- 172.21.15.120:0/2033571274 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e081a5000 con 0x7f0e08101dd0 2024-09-19T09:11:50.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.393+0000 7f0e0c96e640 1 -- 172.21.15.120:0/2033571274 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f0df8042030 con 0x7f0e08101dd0 2024-09-19T09:11:50.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.393+0000 7f0e0fd49740 1 Processor -- start 2024-09-19T09:11:50.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.393+0000 7f0e0fd49740 1 -- start start 2024-09-19T09:11:50.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.394+0000 7f0e0fd49740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aad55ab40 0x557aad559090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:50.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.394+0000 7f0e0fd49740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aad5595d0 0x557aad559a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:50.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.394+0000 7f0e0fd49740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aad568780 0x557aad56ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:50.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.394+0000 7f0e0fd49740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557aad4bf2f0 con 0x557aad5595d0 2024-09-19T09:11:50.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.394+0000 7f0e0fd49740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557aad466cf0 con 0x557aad568780 2024-09-19T09:11:50.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.394+0000 7f0e0fd49740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557aad4bb2b0 con 0x557aad55ab40 2024-09-19T09:11:50.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.394+0000 7f0e077fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aad55ab40 0x557aad559090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:50.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.394+0000 7f0e06ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aad5595d0 0x557aad559a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:50.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.394+0000 7f0e077fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aad55ab40 0x557aad559090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:34048/0 (socket says 172.21.15.120:34048) 2024-09-19T09:11:50.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.394+0000 7f0e077fe640 1 -- 172.21.15.120:0/941991520 learned_addr learned my addr 172.21.15.120:0/941991520 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:50.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.394+0000 7f0dfffff640 1 --2- 172.21.15.120:0/941991520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aad568780 0x557aad56ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:50.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.394+0000 7f0e077fe640 1 -- 172.21.15.120:0/941991520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aad568780 msgr2=0x557aad56ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:50.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.394+0000 7f0e077fe640 1 --2- 172.21.15.120:0/941991520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aad568780 0x557aad56ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.394+0000 7f0e077fe640 1 -- 172.21.15.120:0/941991520 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aad5595d0 msgr2=0x557aad559a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:50.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.394+0000 7f0e077fe640 1 --2- 172.21.15.120:0/941991520 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aad5595d0 0x557aad559a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.394+0000 7f0e077fe640 1 -- 172.21.15.120:0/941991520 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557aad44b3a0 con 0x557aad55ab40 2024-09-19T09:11:50.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.394+0000 7f0e0c96e640 1 --2- 172.21.15.120:0/2033571274 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0df40a3240 0x7f0df40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:50.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.394+0000 7f0dfffff640 1 --2- 172.21.15.120:0/941991520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aad568780 0x557aad56ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:50.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.394+0000 7f0e0c96e640 1 -- 172.21.15.120:0/2033571274 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f0df80efef0 con 0x7f0e08101dd0 2024-09-19T09:11:50.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.394+0000 7f0dfffff640 1 --2- 172.21.15.120:0/2033571274 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0df40a3240 0x7f0df40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:50.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.395+0000 7f0e077fe640 1 --2- 172.21.15.120:0/941991520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aad55ab40 0x557aad559090 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009f90 tx=0x7f0e00010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc063b23aa13a487 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:50.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.395+0000 7f0dfdffb640 1 -- 172.21.15.120:0/941991520 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00011700 con 0x557aad55ab40 2024-09-19T09:11:50.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.395+0000 7f0dfdffb640 1 -- 172.21.15.120:0/941991520 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e00011d80 con 0x557aad55ab40 2024-09-19T09:11:50.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.395+0000 7f0dfdffb640 1 -- 172.21.15.120:0/941991520 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00019e40 con 0x557aad55ab40 2024-09-19T09:11:50.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.396+0000 7f0e0fd49740 1 -- 172.21.15.120:0/941991520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aad55ab40 msgr2=0x557aad559090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:50.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.396+0000 7f0dfffff640 1 --2- 172.21.15.120:0/2033571274 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0df40a3240 0x7f0df40a5700 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x557aad55a070 tx=0x7f0df0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:50.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.396+0000 7f0e0fd49740 1 --2- 172.21.15.120:0/941991520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aad55ab40 0x557aad559090 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009f90 tx=0x7f0e00010860 comp rx=0 tx=0).stop 2024-09-19T09:11:50.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.396+0000 7f0e0fd49740 1 -- 172.21.15.120:0/941991520 shutdown_connections 2024-09-19T09:11:50.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.396+0000 7f0e0fd49740 1 --2- 172.21.15.120:0/941991520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aad568780 0x557aad56ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.396+0000 7f0e0fd49740 1 --2- 172.21.15.120:0/941991520 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aad5595d0 0x557aad559a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.396+0000 7f0e0fd49740 1 --2- 172.21.15.120:0/941991520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aad55ab40 0x557aad559090 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.396+0000 7f0e0fd49740 1 -- 172.21.15.120:0/941991520 >> 172.21.15.120:0/941991520 conn(0x557aad522310 msgr2=0x557aad555c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:50.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.396+0000 7f0e0fd49740 1 -- 172.21.15.120:0/941991520 shutdown_connections 2024-09-19T09:11:50.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.396+0000 7f0e0fd49740 1 -- 172.21.15.120:0/941991520 wait complete. 2024-09-19T09:11:50.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.396+0000 7f0e0fd49740 1 -- start start 2024-09-19T09:11:50.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.396+0000 7f0e0fd49740 1 Processor -- start 2024-09-19T09:11:50.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.397+0000 7f0e0fd49740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aad5595d0 0x557aad568130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:50.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.397+0000 7f0e0fd49740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aad55ab40 0x557aad5653e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:50.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.397+0000 7f0e0fd49740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aad568780 0x557aad565980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:50.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.397+0000 7f0e0fd49740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557aad56b480 con 0x557aad5595d0 2024-09-19T09:11:50.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.397+0000 7f0e0fd49740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557aad47a060 con 0x557aad55ab40 2024-09-19T09:11:50.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.397+0000 7f0e0fd49740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557aad56b600 con 0x557aad568780 2024-09-19T09:11:50.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.397+0000 7f0e0fd49740 10 client.0 _ll_register_callbacks cb 0x557aad1cd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:50.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.397+0000 7f0dfffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aad55ab40 0x557aad5653e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:50.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.397+0000 7f0e077fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aad568780 0x557aad565980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:50.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.397+0000 7f0dfffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aad55ab40 0x557aad5653e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:49450/0 (socket says 172.21.15.120:49450) 2024-09-19T09:11:50.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.397+0000 7f0e077fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aad568780 0x557aad565980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:34060/0 (socket says 172.21.15.120:34060) 2024-09-19T09:11:50.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.397+0000 7f0dfffff640 1 -- 172.21.15.120:0/946948284 learned_addr learned my addr 172.21.15.120:0/946948284 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:50.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.397+0000 7f0e06ffd640 1 --2- 172.21.15.120:0/946948284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aad5595d0 0x557aad568130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:50.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.397+0000 7f0dfffff640 1 -- 172.21.15.120:0/946948284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aad568780 msgr2=0x557aad565980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:50.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.397+0000 7f0dfffff640 1 --2- 172.21.15.120:0/946948284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aad568780 0x557aad565980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.397+0000 7f0dfffff640 1 -- 172.21.15.120:0/946948284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aad5595d0 msgr2=0x557aad568130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:50.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.397+0000 7f0dfffff640 1 --2- 172.21.15.120:0/946948284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aad5595d0 0x557aad568130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.397+0000 7f0dfffff640 1 -- 172.21.15.120:0/946948284 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557aad2303c0 con 0x557aad55ab40 2024-09-19T09:11:50.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.397+0000 7f0e077fe640 1 --2- 172.21.15.120:0/946948284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aad568780 0x557aad565980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:50.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.397+0000 7f0e06ffd640 1 --2- 172.21.15.120:0/946948284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aad5595d0 0x557aad568130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:50.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.397+0000 7f0dfffff640 1 --2- 172.21.15.120:0/946948284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aad55ab40 0x557aad5653e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0df001f9d0 tx=0x7f0df0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a5f08d040fb9a79 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:50.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.397+0000 7f0de6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:11:50.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.397+0000 7f0de6ffd640 1 -- 172.21.15.120:0/946948284 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df0009840 con 0x557aad55ab40 2024-09-19T09:11:50.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.397+0000 7f0e0fd49740 1 -- 172.21.15.120:0/946948284 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e00011d80 con 0x557aad55ab40 2024-09-19T09:11:50.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.397+0000 7f0de6ffd640 1 -- 172.21.15.120:0/946948284 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f0df0005020 con 0x557aad55ab40 2024-09-19T09:11:50.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.398+0000 7f0de6ffd640 1 -- 172.21.15.120:0/946948284 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0df0005220 con 0x557aad55ab40 2024-09-19T09:11:50.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.398+0000 7f0e0fd49740 10 client.27626 fetch_fsmap learned FSMap version 32 2024-09-19T09:11:50.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.398+0000 7f0e0fd49740 1 -- 172.21.15.120:0/946948284 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557aad576f80 con 0x557aad55ab40 2024-09-19T09:11:50.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.398+0000 7f0de6ffd640 1 -- 172.21.15.120:0/946948284 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e00011d80 con 0x557aad55ab40 2024-09-19T09:11:50.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.398+0000 7f0de6ffd640 1 -- 172.21.15.120:0/946948284 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f0df001d880 con 0x557aad55ab40 2024-09-19T09:11:50.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.398+0000 7f0e0fd49740 10 client.27626 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:11:50.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.398+0000 7f0e0fd49740 10 client.27626 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:11:50.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.398+0000 7f0e0fd49740 20 client.27626 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:50.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.398+0000 7f0e0fd49740 1 --2- 172.21.15.120:0/946948284 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557aad578390 0x557aad57a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:50.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.398+0000 7f0e0fd49740 4 client.27626 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:11:50.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.398+0000 7f0e0fd49740 1 -- 172.21.15.120:0/946948284 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557aad4cc860 con 0x557aad578390 2024-09-19T09:11:50.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.398+0000 7f0e06ffd640 1 --2- 172.21.15.120:0/946948284 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557aad578390 0x557aad57a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:50.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.399+0000 7f0e06ffd640 1 --2- 172.21.15.120:0/946948284 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557aad578390 0x557aad57a7f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=571dfacf86040e70 server_cookie=60d8e9de7e622167 in_seq=0 out_seq=0 2024-09-19T09:11:50.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.399+0000 7f0de6ffd640 10 client.27626 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:11:50.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.399+0000 7f0de6ffd640 1 -- 172.21.15.120:0/946948284 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557aad4cc860 con 0x557aad578390 2024-09-19T09:11:50.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.399+0000 7f0de6ffd640 10 client.27626 handle_command_reply: tid=0 2024-09-19T09:11:50.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.399+0000 7f0e0fd49740 1 client.27626 shutdown 2024-09-19T09:11:50.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.399+0000 7f0e0fd49740 1 -- 172.21.15.120:0/946948284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aad55ab40 msgr2=0x557aad5653e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:50.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.399+0000 7f0e0fd49740 1 --2- 172.21.15.120:0/946948284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aad55ab40 0x557aad5653e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0df001f9d0 tx=0x7f0df0004730 comp rx=0 tx=0).stop 2024-09-19T09:11:50.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.399+0000 7f0e0fd49740 1 -- 172.21.15.120:0/946948284 shutdown_connections 2024-09-19T09:11:50.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.399+0000 7f0e0fd49740 1 --2- 172.21.15.120:0/946948284 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557aad578390 0x557aad57a7f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.399+0000 7f0e0fd49740 1 --2- 172.21.15.120:0/946948284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aad568780 0x557aad565980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.399+0000 7f0e0fd49740 1 --2- 172.21.15.120:0/946948284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aad55ab40 0x557aad5653e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.399+0000 7f0e0fd49740 1 --2- 172.21.15.120:0/946948284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aad5595d0 0x557aad568130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.399+0000 7f0e0fd49740 1 -- 172.21.15.120:0/946948284 >> 172.21.15.120:0/946948284 conn(0x557aad522310 msgr2=0x557aad55b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:50.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.400+0000 7f0de6ffd640 0 client.27626 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:11:50.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.400+0000 7f0e0fd49740 1 -- 172.21.15.120:0/946948284 shutdown_connections 2024-09-19T09:11:50.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.400+0000 7f0e0fd49740 1 -- 172.21.15.120:0/946948284 wait complete. 2024-09-19T09:11:50.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.400+0000 7f0e0fd49740 20 client.27626 trim_cache size 0 max 16384 2024-09-19T09:11:50.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.415+0000 7f0e0fd49740 1 Processor -- start 2024-09-19T09:11:50.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.415+0000 7f0e0fd49740 1 -- start start 2024-09-19T09:11:50.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.415+0000 7f0e0fd49740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aad5595d0 0x557aad574720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:50.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.415+0000 7f0e0fd49740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aad55ab40 0x557aad563cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:50.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.415+0000 7f0e0fd49740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aad568780 0x557aad5641f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:50.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.415+0000 7f0e0fd49740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0e081ab430 con 0x557aad568780 2024-09-19T09:11:50.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.415+0000 7f0e0fd49740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0e081acf70 con 0x557aad5595d0 2024-09-19T09:11:50.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.415+0000 7f0e0fd49740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e081abb00 con 0x557aad55ab40 2024-09-19T09:11:50.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.415+0000 7f0e06ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aad5595d0 0x557aad574720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:50.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.415+0000 7f0e077fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aad568780 0x557aad5641f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:50.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.415+0000 7f0dfffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aad55ab40 0x557aad563cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:50.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.415+0000 7f0e06ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aad5595d0 0x557aad574720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:49454/0 (socket says 172.21.15.120:49454) 2024-09-19T09:11:50.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.415+0000 7f0e06ffd640 1 -- 172.21.15.120:0/930732145 learned_addr learned my addr 172.21.15.120:0/930732145 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:50.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.415+0000 7f0e06ffd640 1 -- 172.21.15.120:0/930732145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aad55ab40 msgr2=0x557aad563cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:50.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.415+0000 7f0e06ffd640 1 --2- 172.21.15.120:0/930732145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aad55ab40 0x557aad563cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.415+0000 7f0e06ffd640 1 -- 172.21.15.120:0/930732145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aad568780 msgr2=0x557aad5641f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:50.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.415+0000 7f0e06ffd640 1 --2- 172.21.15.120:0/930732145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aad568780 0x557aad5641f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.415+0000 7f0e06ffd640 1 -- 172.21.15.120:0/930732145 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557aad556730 con 0x557aad5595d0 2024-09-19T09:11:50.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.415+0000 7f0dfffff640 1 --2- 172.21.15.120:0/930732145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aad55ab40 0x557aad563cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:50.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.416+0000 7f0e06ffd640 1 --2- 172.21.15.120:0/930732145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aad5595d0 0x557aad574720 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0df80c4d10 tx=0x7f0df80f38f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34fe93d7973ebc73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:50.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.416+0000 7f0dfd7fa640 1 -- 172.21.15.120:0/930732145 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df8049040 con 0x557aad5595d0 2024-09-19T09:11:50.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.416+0000 7f0dfd7fa640 1 -- 172.21.15.120:0/930732145 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0df80bccd0 con 0x557aad5595d0 2024-09-19T09:11:50.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.416+0000 7f0dfd7fa640 1 -- 172.21.15.120:0/930732145 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df80bc5d0 con 0x557aad5595d0 2024-09-19T09:11:50.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.417+0000 7f0e0fd49740 1 -- 172.21.15.120:0/930732145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aad5595d0 msgr2=0x557aad574720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:50.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.417+0000 7f0e0fd49740 1 --2- 172.21.15.120:0/930732145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aad5595d0 0x557aad574720 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0df80c4d10 tx=0x7f0df80f38f0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.417+0000 7f0e0fd49740 1 -- 172.21.15.120:0/930732145 shutdown_connections 2024-09-19T09:11:50.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.417+0000 7f0e0fd49740 1 --2- 172.21.15.120:0/930732145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aad568780 0x557aad5641f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.417+0000 7f0e0fd49740 1 --2- 172.21.15.120:0/930732145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aad55ab40 0x557aad563cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.417+0000 7f0e0fd49740 1 --2- 172.21.15.120:0/930732145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aad5595d0 0x557aad574720 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.417+0000 7f0e0fd49740 1 -- 172.21.15.120:0/930732145 >> 172.21.15.120:0/930732145 conn(0x557aad522310 msgr2=0x557aad560720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:50.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.417+0000 7f0e0fd49740 1 -- 172.21.15.120:0/930732145 shutdown_connections 2024-09-19T09:11:50.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.417+0000 7f0e0fd49740 1 -- 172.21.15.120:0/930732145 wait complete. 2024-09-19T09:11:50.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.417+0000 7f0e0fd49740 1 -- start start 2024-09-19T09:11:50.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.417+0000 7f0e0fd49740 1 Processor -- start 2024-09-19T09:11:50.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.418+0000 7f0e0fd49740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aad5595d0 0x557aad5db380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:50.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.418+0000 7f0e0fd49740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aad55ab40 0x557aad5db8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:50.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.418+0000 7f0e0fd49740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aad568780 0x557aad56be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:50.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.418+0000 7f0e0fd49740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0e081ac8a0 con 0x557aad55ab40 2024-09-19T09:11:50.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.418+0000 7f0e0fd49740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0e081ac1d0 con 0x557aad568780 2024-09-19T09:11:50.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.418+0000 7f0e0fd49740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e081a93f0 con 0x557aad5595d0 2024-09-19T09:11:50.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.418+0000 7f0e0fd49740 10 client.0 _ll_register_callbacks cb 0x7f0df0009760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:50.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.418+0000 7f0e06ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aad5595d0 0x557aad5db380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:50.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.418+0000 7f0e06ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aad5595d0 0x557aad5db380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:34078/0 (socket says 172.21.15.120:34078) 2024-09-19T09:11:50.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.418+0000 7f0e06ffd640 1 -- 172.21.15.120:0/3841394570 learned_addr learned my addr 172.21.15.120:0/3841394570 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:50.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.418+0000 7f0e077fe640 1 --2- 172.21.15.120:0/3841394570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aad568780 0x557aad56be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:50.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.418+0000 7f0dfffff640 1 --2- 172.21.15.120:0/3841394570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aad55ab40 0x557aad5db8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:50.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.418+0000 7f0e06ffd640 1 -- 172.21.15.120:0/3841394570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aad568780 msgr2=0x557aad56be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:50.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.418+0000 7f0e06ffd640 1 --2- 172.21.15.120:0/3841394570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aad568780 0x557aad56be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.418+0000 7f0e06ffd640 1 -- 172.21.15.120:0/3841394570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aad55ab40 msgr2=0x557aad5db8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:50.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.418+0000 7f0e06ffd640 1 --2- 172.21.15.120:0/3841394570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aad55ab40 0x557aad5db8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.418+0000 7f0e06ffd640 1 -- 172.21.15.120:0/3841394570 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557aad56c640 con 0x557aad5595d0 2024-09-19T09:11:50.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.418+0000 7f0dfffff640 1 --2- 172.21.15.120:0/3841394570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aad55ab40 0x557aad5db8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:11:50.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.418+0000 7f0e06ffd640 1 --2- 172.21.15.120:0/3841394570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aad5595d0 0x557aad5db380 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0df8000c00 tx=0x7f0df80f0170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a76dac3704d6a5d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:50.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.418+0000 7f0de6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:11:50.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.419+0000 7f0de6ffd640 1 -- 172.21.15.120:0/3841394570 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df8049070 con 0x557aad5595d0 2024-09-19T09:11:50.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.419+0000 7f0de6ffd640 1 -- 172.21.15.120:0/3841394570 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f0df80b8c40 con 0x557aad5595d0 2024-09-19T09:11:50.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.419+0000 7f0e0fd49740 1 -- 172.21.15.120:0/3841394570 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0df80bccd0 con 0x557aad5595d0 2024-09-19T09:11:50.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.419+0000 7f0de6ffd640 1 -- 172.21.15.120:0/3841394570 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0df80b8870 con 0x557aad5595d0 2024-09-19T09:11:50.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.419+0000 7f0e0fd49740 10 client.27493 fetch_fsmap learned FSMap version 32 2024-09-19T09:11:50.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.419+0000 7f0e0fd49740 1 -- 172.21.15.120:0/3841394570 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557aad5e2b40 con 0x557aad5595d0 2024-09-19T09:11:50.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.419+0000 7f0de6ffd640 1 -- 172.21.15.120:0/3841394570 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0df80bccd0 con 0x557aad5595d0 2024-09-19T09:11:50.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.419+0000 7f0de6ffd640 1 -- 172.21.15.120:0/3841394570 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f0df80bc830 con 0x557aad5595d0 2024-09-19T09:11:50.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.419+0000 7f0e0fd49740 10 client.27493 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:11:50.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.419+0000 7f0e0fd49740 10 client.27493 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:11:50.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.419+0000 7f0e0fd49740 20 client.27493 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:50.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.419+0000 7f0e0fd49740 1 --2- 172.21.15.120:0/3841394570 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557aad5e3250 0x557aad5e5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:50.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.419+0000 7f0e0fd49740 4 client.27493 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:11:50.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.419+0000 7f0e0fd49740 1 -- 172.21.15.120:0/3841394570 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557aad4c6d70 con 0x557aad5e3250 2024-09-19T09:11:50.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.419+0000 7f0dfffff640 1 --2- 172.21.15.120:0/3841394570 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557aad5e3250 0x557aad5e5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:50.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.420+0000 7f0de6ffd640 10 client.27493 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:11:50.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.420+0000 7f0dfffff640 1 --2- 172.21.15.120:0/3841394570 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557aad5e3250 0x557aad5e5670 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5e6d10ba1b17acde server_cookie=ee061400ea021194 in_seq=0 out_seq=0 2024-09-19T09:11:50.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.420+0000 7f0de6ffd640 1 -- 172.21.15.120:0/3841394570 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557aad4c6d70 con 0x557aad5e3250 2024-09-19T09:11:50.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.420+0000 7f0de6ffd640 10 client.27493 handle_command_reply: tid=0 2024-09-19T09:11:50.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.420+0000 7f0e0fd49740 1 client.27493 shutdown 2024-09-19T09:11:50.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.420+0000 7f0e0fd49740 1 -- 172.21.15.120:0/3841394570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aad5595d0 msgr2=0x557aad5db380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:50.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.420+0000 7f0e0fd49740 1 --2- 172.21.15.120:0/3841394570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aad5595d0 0x557aad5db380 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0df8000c00 tx=0x7f0df80f0170 comp rx=0 tx=0).stop 2024-09-19T09:11:50.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.420+0000 7f0e0fd49740 1 -- 172.21.15.120:0/3841394570 shutdown_connections 2024-09-19T09:11:50.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.420+0000 7f0e0fd49740 1 --2- 172.21.15.120:0/3841394570 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557aad5e3250 0x557aad5e5670 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.420+0000 7f0e0fd49740 1 --2- 172.21.15.120:0/3841394570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aad568780 0x557aad56be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.420+0000 7f0de6ffd640 0 client.27493 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:11:50.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.420+0000 7f0e0fd49740 1 --2- 172.21.15.120:0/3841394570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aad55ab40 0x557aad5db8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.420+0000 7f0e0fd49740 1 --2- 172.21.15.120:0/3841394570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aad5595d0 0x557aad5db380 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.420+0000 7f0e0fd49740 1 -- 172.21.15.120:0/3841394570 >> 172.21.15.120:0/3841394570 conn(0x557aad522310 msgr2=0x557aad563c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:50.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.420+0000 7f0e0fd49740 1 -- 172.21.15.120:0/3841394570 shutdown_connections 2024-09-19T09:11:50.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.420+0000 7f0e0fd49740 1 -- 172.21.15.120:0/3841394570 wait complete. 2024-09-19T09:11:50.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.420+0000 7f0e0fd49740 20 client.27493 trim_cache size 0 max 16384 2024-09-19T09:11:50.434 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:11:50.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.421+0000 7f0e0d970640 1 -- 172.21.15.120:0/2033571274 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0df40a3240 msgr2=0x7f0df40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:50.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.421+0000 7f0e0d970640 1 --2- 172.21.15.120:0/2033571274 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0df40a3240 0x7f0df40a5700 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x557aad55a070 tx=0x7f0df0005f90 comp rx=0 tx=0).stop 2024-09-19T09:11:50.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.421+0000 7f0e0d970640 1 -- 172.21.15.120:0/2033571274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e08101dd0 msgr2=0x7f0e08198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:50.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.421+0000 7f0e0d970640 1 --2- 172.21.15.120:0/2033571274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e08101dd0 0x7f0e08198480 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0df8006010 tx=0x7f0df80042f0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.421+0000 7f0e0d970640 1 -- 172.21.15.120:0/2033571274 shutdown_connections 2024-09-19T09:11:50.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.421+0000 7f0e0d970640 1 --2- 172.21.15.120:0/2033571274 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0df40a3240 0x7f0df40a5700 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.421+0000 7f0e0d970640 1 --2- 172.21.15.120:0/2033571274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e08106f00 0x7f0e0819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.421+0000 7f0e0d970640 1 --2- 172.21.15.120:0/2033571274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e08102780 0x7f0e081989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.421+0000 7f0e0d970640 1 --2- 172.21.15.120:0/2033571274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e08101dd0 0x7f0e08198480 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:50.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.421+0000 7f0e0d970640 1 -- 172.21.15.120:0/2033571274 >> 172.21.15.120:0/2033571274 conn(0x7f0e080fd8b0 msgr2=0x7f0e080fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:50.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.421+0000 7f0e0d970640 1 -- 172.21.15.120:0/2033571274 shutdown_connections 2024-09-19T09:11:50.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:50.421+0000 7f0e0d970640 1 -- 172.21.15.120:0/2033571274 wait complete. 2024-09-19T09:11:51.064 INFO:tasks.workunit.client.0.smithi120.stdout: 1 838413 9.57 MB/sec execute 346 sec latency 58.180 ms 2024-09-19T09:11:51.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:50 smithi196 ceph-mon[35150]: pgmap v612: 97 pgs: 97 active+clean; 48 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.5 MiB/s wr, 398 op/s 2024-09-19T09:11:51.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:50 smithi153 ceph-mon[35071]: pgmap v612: 97 pgs: 97 active+clean; 48 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.5 MiB/s wr, 398 op/s 2024-09-19T09:11:51.310 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:50 smithi120 ceph-mon[31150]: pgmap v612: 97 pgs: 97 active+clean; 48 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.5 MiB/s wr, 398 op/s 2024-09-19T09:11:51.348 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:11:51.627 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:11:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.788+0000 7fe979de8640 1 -- 172.21.15.120:0/2359222952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe974101b60 msgr2=0x7fe974101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.788+0000 7fe979de8640 1 --2- 172.21.15.120:0/2359222952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe974101b60 0x7fe974101f60 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fe964009ca0 tx=0x7fe96402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.788+0000 7fe979de8640 1 -- 172.21.15.120:0/2359222952 shutdown_connections 2024-09-19T09:11:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.788+0000 7fe979de8640 1 --2- 172.21.15.120:0/2359222952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe974103710 0x7fe974109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.788+0000 7fe979de8640 1 --2- 172.21.15.120:0/2359222952 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe974102d50 0x7fe9741031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.788+0000 7fe979de8640 1 --2- 172.21.15.120:0/2359222952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe974101b60 0x7fe974101f60 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.788+0000 7fe979de8640 1 -- 172.21.15.120:0/2359222952 >> 172.21.15.120:0/2359222952 conn(0x7fe9740fd2c0 msgr2=0x7fe9740ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.788+0000 7fe979de8640 1 -- 172.21.15.120:0/2359222952 shutdown_connections 2024-09-19T09:11:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.788+0000 7fe979de8640 1 -- 172.21.15.120:0/2359222952 wait complete. 2024-09-19T09:11:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.789+0000 7fe979de8640 1 Processor -- start 2024-09-19T09:11:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.789+0000 7fe979de8640 1 -- start start 2024-09-19T09:11:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.789+0000 7fe979de8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe974101b60 0x7fe974197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.790+0000 7fe979de8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe974102d50 0x7fe974198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.790+0000 7fe979de8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe974103710 0x7fe97419f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.790+0000 7fe979de8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe97410d830 con 0x7fe974102d50 2024-09-19T09:11:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.790+0000 7fe979de8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe97410d6b0 con 0x7fe974103710 2024-09-19T09:11:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.790+0000 7fe979de8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe97410d9b0 con 0x7fe974101b60 2024-09-19T09:11:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.790+0000 7fe9737fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe974101b60 0x7fe974197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.790+0000 7fe972ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe974102d50 0x7fe974198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.790+0000 7fe973fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe974103710 0x7fe97419f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.790+0000 7fe972ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe974102d50 0x7fe974198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46140/0 (socket says 172.21.15.120:46140) 2024-09-19T09:11:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.790+0000 7fe9737fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe974101b60 0x7fe974197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:46580/0 (socket says 172.21.15.120:46580) 2024-09-19T09:11:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.790+0000 7fe972ffd640 1 -- 172.21.15.120:0/1097124143 learned_addr learned my addr 172.21.15.120:0/1097124143 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.790+0000 7fe972ffd640 1 -- 172.21.15.120:0/1097124143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe974101b60 msgr2=0x7fe974197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.790+0000 7fe972ffd640 1 --2- 172.21.15.120:0/1097124143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe974101b60 0x7fe974197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.790+0000 7fe972ffd640 1 -- 172.21.15.120:0/1097124143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe974103710 msgr2=0x7fe97419f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.790+0000 7fe972ffd640 1 --2- 172.21.15.120:0/1097124143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe974103710 0x7fe97419f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.790+0000 7fe972ffd640 1 -- 172.21.15.120:0/1097124143 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97419fa70 con 0x7fe974102d50 2024-09-19T09:11:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.790+0000 7fe973fff640 1 --2- 172.21.15.120:0/1097124143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe974103710 0x7fe97419f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.790+0000 7fe972ffd640 1 --2- 172.21.15.120:0/1097124143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe974102d50 0x7fe974198390 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00e9f0 tx=0x7fe95c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.790+0000 7fe970ff9640 1 -- 172.21.15.120:0/1097124143 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c00cd80 con 0x7fe974102d50 2024-09-19T09:11:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.791+0000 7fe979de8640 1 -- 172.21.15.120:0/1097124143 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe97419fd60 con 0x7fe974102d50 2024-09-19T09:11:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.791+0000 7fe970ff9640 1 -- 172.21.15.120:0/1097124143 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe95c010910 con 0x7fe974102d50 2024-09-19T09:11:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.791+0000 7fe970ff9640 1 -- 172.21.15.120:0/1097124143 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c0188f0 con 0x7fe974102d50 2024-09-19T09:11:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.791+0000 7fe979de8640 1 -- 172.21.15.120:0/1097124143 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9741a0220 con 0x7fe974102d50 2024-09-19T09:11:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.793+0000 7fe970ff9640 1 -- 172.21.15.120:0/1097124143 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe95c018ad0 con 0x7fe974102d50 2024-09-19T09:11:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.793+0000 7fe97cc61740 1 Processor -- start 2024-09-19T09:11:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.793+0000 7fe97cc61740 1 -- start start 2024-09-19T09:11:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.793+0000 7fe97cc61740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643bfd499a0 0x5643bfd49da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.793+0000 7fe97cc61740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643bfd4a2e0 0x5643bfd4eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.793+0000 7fe97cc61740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643bfd4f370 0x5643bfd51830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.793+0000 7fe97cc61740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5643bfcad2f0 con 0x5643bfd4a2e0 2024-09-19T09:11:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.793+0000 7fe97cc61740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5643bfc55270 con 0x5643bfd4f370 2024-09-19T09:11:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.793+0000 7fe97cc61740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5643bfca92b0 con 0x5643bfd499a0 2024-09-19T09:11:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.793+0000 7fe9737fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643bfd4a2e0 0x5643bfd4eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.793+0000 7fe9737fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643bfd4a2e0 0x5643bfd4eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46146/0 (socket says 172.21.15.120:46146) 2024-09-19T09:11:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.793+0000 7fe9737fe640 1 -- 172.21.15.120:0/2286220222 learned_addr learned my addr 172.21.15.120:0/2286220222 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.793+0000 7fe972ffd640 1 --2- 172.21.15.120:0/2286220222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643bfd4f370 0x5643bfd51830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.793+0000 7fe973fff640 1 --2- 172.21.15.120:0/2286220222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643bfd499a0 0x5643bfd49da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.794+0000 7fe970ff9640 1 --2- 172.21.15.120:0/1097124143 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9600a32f0 0x7fe9600a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.794+0000 7fe9737fe640 1 -- 172.21.15.120:0/2286220222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643bfd499a0 msgr2=0x5643bfd49da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.794+0000 7fe9737fe640 1 --2- 172.21.15.120:0/2286220222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643bfd499a0 0x5643bfd49da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.794+0000 7fe9737fe640 1 -- 172.21.15.120:0/2286220222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643bfd4f370 msgr2=0x5643bfd51830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.794+0000 7fe9737fe640 1 --2- 172.21.15.120:0/2286220222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643bfd4f370 0x5643bfd51830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.794+0000 7fe9737fe640 1 -- 172.21.15.120:0/2286220222 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643bfc54340 con 0x5643bfd4a2e0 2024-09-19T09:11:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.794+0000 7fe973fff640 1 --2- 172.21.15.120:0/2286220222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643bfd499a0 0x5643bfd49da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:11:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.794+0000 7fe973fff640 1 --2- 172.21.15.120:0/1097124143 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9600a32f0 0x7fe9600a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.794+0000 7fe970ff9640 1 -- 172.21.15.120:0/1097124143 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fe95c0cfcb0 con 0x7fe974102d50 2024-09-19T09:11:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.794+0000 7fe9737fe640 1 --2- 172.21.15.120:0/2286220222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643bfd4a2e0 0x5643bfd4eda0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fe964009ca0 tx=0x7fe964009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6714dca9f8f4791f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.794+0000 7fe969ffb640 1 -- 172.21.15.120:0/2286220222 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964003b70 con 0x5643bfd4a2e0 2024-09-19T09:11:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.794+0000 7fe969ffb640 1 -- 172.21.15.120:0/2286220222 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe964036030 con 0x5643bfd4a2e0 2024-09-19T09:11:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.794+0000 7fe969ffb640 1 -- 172.21.15.120:0/2286220222 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964004720 con 0x5643bfd4a2e0 2024-09-19T09:11:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.794+0000 7fe973fff640 1 --2- 172.21.15.120:0/1097124143 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9600a32f0 0x7fe9600a57b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe96c0125f0 tx=0x7fe96c0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.795+0000 7fe97cc61740 1 -- 172.21.15.120:0/2286220222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643bfd4a2e0 msgr2=0x5643bfd4eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.795+0000 7fe97cc61740 1 --2- 172.21.15.120:0/2286220222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643bfd4a2e0 0x5643bfd4eda0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fe964009ca0 tx=0x7fe964009800 comp rx=0 tx=0).stop 2024-09-19T09:11:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.795+0000 7fe97cc61740 1 -- 172.21.15.120:0/2286220222 shutdown_connections 2024-09-19T09:11:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.795+0000 7fe97cc61740 1 --2- 172.21.15.120:0/2286220222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643bfd4f370 0x5643bfd51830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.795+0000 7fe97cc61740 1 --2- 172.21.15.120:0/2286220222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643bfd4a2e0 0x5643bfd4eda0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.795+0000 7fe97cc61740 1 --2- 172.21.15.120:0/2286220222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643bfd499a0 0x5643bfd49da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.795+0000 7fe97cc61740 1 -- 172.21.15.120:0/2286220222 >> 172.21.15.120:0/2286220222 conn(0x5643bfd45010 msgr2=0x5643bfd45410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.795+0000 7fe97cc61740 1 -- 172.21.15.120:0/2286220222 shutdown_connections 2024-09-19T09:11:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.795+0000 7fe97cc61740 1 -- 172.21.15.120:0/2286220222 wait complete. 2024-09-19T09:11:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.795+0000 7fe97cc61740 1 -- start start 2024-09-19T09:11:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.795+0000 7fe97cc61740 1 Processor -- start 2024-09-19T09:11:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.796+0000 7fe97cc61740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643bfd499a0 0x5643bfd57d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.796+0000 7fe97cc61740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643bfd4a2e0 0x5643bfd582d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.796+0000 7fe97cc61740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643bfd4f370 0x5643bfd5f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.796+0000 7fe97cc61740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5643bfd52160 con 0x5643bfd4a2e0 2024-09-19T09:11:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.796+0000 7fe97cc61740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5643bfc36290 con 0x5643bfd499a0 2024-09-19T09:11:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.796+0000 7fe97cc61740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5643bfd522e0 con 0x5643bfd4f370 2024-09-19T09:11:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.796+0000 7fe97cc61740 10 client.0 _ll_register_callbacks cb 0x5643bfc67140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.796+0000 7fe972ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643bfd4a2e0 0x5643bfd582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.796+0000 7fe9737fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643bfd499a0 0x5643bfd57d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.796+0000 7fe972ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643bfd4a2e0 0x5643bfd582d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46152/0 (socket says 172.21.15.120:46152) 2024-09-19T09:11:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.796+0000 7fe972ffd640 1 -- 172.21.15.120:0/1215766721 learned_addr learned my addr 172.21.15.120:0/1215766721 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.796+0000 7fe973fff640 1 --2- 172.21.15.120:0/1215766721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643bfd4f370 0x5643bfd5f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.796+0000 7fe9737fe640 1 -- 172.21.15.120:0/1215766721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643bfd4f370 msgr2=0x5643bfd5f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.796+0000 7fe9737fe640 1 --2- 172.21.15.120:0/1215766721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643bfd4f370 0x5643bfd5f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.796+0000 7fe9737fe640 1 -- 172.21.15.120:0/1215766721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643bfd4a2e0 msgr2=0x5643bfd582d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.796+0000 7fe9737fe640 1 --2- 172.21.15.120:0/1215766721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643bfd4a2e0 0x5643bfd582d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.796+0000 7fe9737fe640 1 -- 172.21.15.120:0/1215766721 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643bf9c0210 con 0x5643bfd499a0 2024-09-19T09:11:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.796+0000 7fe972ffd640 1 --2- 172.21.15.120:0/1215766721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643bfd4a2e0 0x5643bfd582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.796+0000 7fe9737fe640 1 --2- 172.21.15.120:0/1215766721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643bfd499a0 0x5643bfd57d90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe96402ade0 tx=0x7fe964009970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cd46133a32224ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.796+0000 7fe952ffd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:11:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.796+0000 7fe952ffd640 1 -- 172.21.15.120:0/1215766721 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96404a040 con 0x5643bfd499a0 2024-09-19T09:11:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.797+0000 7fe952ffd640 1 -- 172.21.15.120:0/1215766721 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fe964003970 con 0x5643bfd499a0 2024-09-19T09:11:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.797+0000 7fe97cc61740 1 -- 172.21.15.120:0/1215766721 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe964036030 con 0x5643bfd499a0 2024-09-19T09:11:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.797+0000 7fe952ffd640 1 -- 172.21.15.120:0/1215766721 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe964003b70 con 0x5643bfd499a0 2024-09-19T09:11:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.797+0000 7fe97cc61740 10 client.27644 fetch_fsmap learned FSMap version 32 2024-09-19T09:11:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.797+0000 7fe97cc61740 1 -- 172.21.15.120:0/1215766721 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5643bfa8c620 con 0x5643bfd499a0 2024-09-19T09:11:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.797+0000 7fe952ffd640 1 -- 172.21.15.120:0/1215766721 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe964036030 con 0x5643bfd499a0 2024-09-19T09:11:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.797+0000 7fe952ffd640 1 -- 172.21.15.120:0/1215766721 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe964059020 con 0x5643bfd499a0 2024-09-19T09:11:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.797+0000 7fe97cc61740 10 client.27644 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:11:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.797+0000 7fe97cc61740 10 client.27644 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:11:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.797+0000 7fe97cc61740 20 client.27644 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.797+0000 7fe97cc61740 1 --2- 172.21.15.120:0/1215766721 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5643bfd63720 0x5643bfd65b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.797+0000 7fe97cc61740 4 client.27644 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:11:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.797+0000 7fe97cc61740 1 -- 172.21.15.120:0/1215766721 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643bfcbaa20 con 0x5643bfd63720 2024-09-19T09:11:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.798+0000 7fe972ffd640 1 --2- 172.21.15.120:0/1215766721 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5643bfd63720 0x5643bfd65b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.798+0000 7fe972ffd640 1 --2- 172.21.15.120:0/1215766721 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5643bfd63720 0x5643bfd65b10 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9875402a11bc7211 server_cookie=7e1f3390ec69e487 in_seq=0 out_seq=0 2024-09-19T09:11:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.798+0000 7fe952ffd640 10 client.27644 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.798+0000 7fe952ffd640 1 -- 172.21.15.120:0/1215766721 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5643bfcbaa20 con 0x5643bfd63720 2024-09-19T09:11:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.799+0000 7fe952ffd640 10 client.27644 handle_command_reply: tid=0 2024-09-19T09:11:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.799+0000 7fe97cc61740 1 client.27644 shutdown 2024-09-19T09:11:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.799+0000 7fe97cc61740 1 -- 172.21.15.120:0/1215766721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643bfd499a0 msgr2=0x5643bfd57d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.799+0000 7fe97cc61740 1 --2- 172.21.15.120:0/1215766721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643bfd499a0 0x5643bfd57d90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe96402ade0 tx=0x7fe964009970 comp rx=0 tx=0).stop 2024-09-19T09:11:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.799+0000 7fe97cc61740 1 -- 172.21.15.120:0/1215766721 shutdown_connections 2024-09-19T09:11:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.799+0000 7fe97cc61740 1 --2- 172.21.15.120:0/1215766721 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5643bfd63720 0x5643bfd65b10 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.799+0000 7fe97cc61740 1 --2- 172.21.15.120:0/1215766721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643bfd4f370 0x5643bfd5f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.799+0000 7fe97cc61740 1 --2- 172.21.15.120:0/1215766721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643bfd4a2e0 0x5643bfd582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.799+0000 7fe952ffd640 0 client.27644 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.799+0000 7fe97cc61740 1 --2- 172.21.15.120:0/1215766721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643bfd499a0 0x5643bfd57d90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.799+0000 7fe97cc61740 1 -- 172.21.15.120:0/1215766721 >> 172.21.15.120:0/1215766721 conn(0x5643bfd45010 msgr2=0x5643bfd46820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.799+0000 7fe97cc61740 1 -- 172.21.15.120:0/1215766721 shutdown_connections 2024-09-19T09:11:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.799+0000 7fe97cc61740 1 -- 172.21.15.120:0/1215766721 wait complete. 2024-09-19T09:11:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.799+0000 7fe97cc61740 20 client.27644 trim_cache size 0 max 16384 2024-09-19T09:11:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.813+0000 7fe97cc61740 1 Processor -- start 2024-09-19T09:11:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.813+0000 7fe97cc61740 1 -- start start 2024-09-19T09:11:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.813+0000 7fe97cc61740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643bfd499a0 0x5643bfd49d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.813+0000 7fe97cc61740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643bfd4a2e0 0x5643bfd5f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.813+0000 7fe97cc61740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643bfd4f370 0x5643bfd440b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.813+0000 7fe97cc61740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe9741a6d60 con 0x5643bfd4f370 2024-09-19T09:11:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.813+0000 7fe97cc61740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe9741a88a0 con 0x5643bfd499a0 2024-09-19T09:11:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.813+0000 7fe97cc61740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe9741a7430 con 0x5643bfd4a2e0 2024-09-19T09:11:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.814+0000 7fe973fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643bfd4f370 0x5643bfd440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.814+0000 7fe9737fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643bfd499a0 0x5643bfd49d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.814+0000 7fe973fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643bfd4f370 0x5643bfd440b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46160/0 (socket says 172.21.15.120:46160) 2024-09-19T09:11:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.814+0000 7fe973fff640 1 -- 172.21.15.120:0/3582836170 learned_addr learned my addr 172.21.15.120:0/3582836170 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.814+0000 7fe972ffd640 1 --2- 172.21.15.120:0/3582836170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643bfd4a2e0 0x5643bfd5f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.814+0000 7fe972ffd640 1 -- 172.21.15.120:0/3582836170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643bfd499a0 msgr2=0x5643bfd49d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.814+0000 7fe972ffd640 1 --2- 172.21.15.120:0/3582836170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643bfd499a0 0x5643bfd49d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.814+0000 7fe972ffd640 1 -- 172.21.15.120:0/3582836170 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643bfd4f370 msgr2=0x5643bfd440b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.814+0000 7fe972ffd640 1 --2- 172.21.15.120:0/3582836170 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643bfd4f370 0x5643bfd440b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.814+0000 7fe972ffd640 1 -- 172.21.15.120:0/3582836170 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643bfd4e720 con 0x5643bfd4a2e0 2024-09-19T09:11:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.814+0000 7fe9737fe640 1 --2- 172.21.15.120:0/3582836170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643bfd499a0 0x5643bfd49d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.814+0000 7fe972ffd640 1 --2- 172.21.15.120:0/3582836170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643bfd4a2e0 0x5643bfd5f2d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00e9c0 tx=0x7fe95c0bc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0b3b2a4485f379e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.814+0000 7fe9697fa640 1 -- 172.21.15.120:0/3582836170 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c09bce0 con 0x5643bfd4a2e0 2024-09-19T09:11:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.814+0000 7fe9697fa640 1 -- 172.21.15.120:0/3582836170 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe95c09c380 con 0x5643bfd4a2e0 2024-09-19T09:11:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.814+0000 7fe9697fa640 1 -- 172.21.15.120:0/3582836170 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c09b7d0 con 0x5643bfd4a2e0 2024-09-19T09:11:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.815+0000 7fe97cc61740 1 -- 172.21.15.120:0/3582836170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643bfd4a2e0 msgr2=0x5643bfd5f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.815+0000 7fe97cc61740 1 --2- 172.21.15.120:0/3582836170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643bfd4a2e0 0x5643bfd5f2d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00e9c0 tx=0x7fe95c0bc040 comp rx=0 tx=0).stop 2024-09-19T09:11:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.815+0000 7fe97cc61740 1 -- 172.21.15.120:0/3582836170 shutdown_connections 2024-09-19T09:11:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.815+0000 7fe97cc61740 1 --2- 172.21.15.120:0/3582836170 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643bfd4f370 0x5643bfd440b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.815+0000 7fe97cc61740 1 --2- 172.21.15.120:0/3582836170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643bfd4a2e0 0x5643bfd5f2d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.815+0000 7fe97cc61740 1 --2- 172.21.15.120:0/3582836170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643bfd499a0 0x5643bfd49d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.815+0000 7fe97cc61740 1 -- 172.21.15.120:0/3582836170 >> 172.21.15.120:0/3582836170 conn(0x5643bfd45010 msgr2=0x5643bfd439a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.815+0000 7fe97cc61740 1 -- 172.21.15.120:0/3582836170 shutdown_connections 2024-09-19T09:11:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.815+0000 7fe97cc61740 1 -- 172.21.15.120:0/3582836170 wait complete. 2024-09-19T09:11:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.815+0000 7fe97cc61740 1 -- start start 2024-09-19T09:11:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.816+0000 7fe97cc61740 1 Processor -- start 2024-09-19T09:11:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.816+0000 7fe97cc61740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643bfd499a0 0x5643bfd5ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.816+0000 7fe97cc61740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643bfd4a2e0 0x5643bfd5cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.816+0000 7fe97cc61740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643bfd4f370 0x5643bfd5dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.816+0000 7fe97cc61740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe9741a81d0 con 0x5643bfd4f370 2024-09-19T09:11:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.816+0000 7fe97cc61740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe9741a7b00 con 0x5643bfd4a2e0 2024-09-19T09:11:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.816+0000 7fe97cc61740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe9741a4d20 con 0x5643bfd499a0 2024-09-19T09:11:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.816+0000 7fe97cc61740 10 client.0 _ll_register_callbacks cb 0x7fe95c0d3600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.816+0000 7fe973fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643bfd4f370 0x5643bfd5dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.816+0000 7fe972ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643bfd4a2e0 0x5643bfd5cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.816+0000 7fe973fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643bfd4f370 0x5643bfd5dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46164/0 (socket says 172.21.15.120:46164) 2024-09-19T09:11:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.816+0000 7fe973fff640 1 -- 172.21.15.120:0/2873446672 learned_addr learned my addr 172.21.15.120:0/2873446672 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.816+0000 7fe9737fe640 1 --2- 172.21.15.120:0/2873446672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643bfd499a0 0x5643bfd5ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.816+0000 7fe973fff640 1 -- 172.21.15.120:0/2873446672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643bfd499a0 msgr2=0x5643bfd5ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.816+0000 7fe973fff640 1 --2- 172.21.15.120:0/2873446672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643bfd499a0 0x5643bfd5ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.816+0000 7fe973fff640 1 -- 172.21.15.120:0/2873446672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643bfd4a2e0 msgr2=0x5643bfd5cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.816+0000 7fe973fff640 1 --2- 172.21.15.120:0/2873446672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643bfd4a2e0 0x5643bfd5cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.816+0000 7fe973fff640 1 -- 172.21.15.120:0/2873446672 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643bfd5e5f0 con 0x5643bfd4f370 2024-09-19T09:11:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.816+0000 7fe972ffd640 1 --2- 172.21.15.120:0/2873446672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643bfd4a2e0 0x5643bfd5cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.816+0000 7fe973fff640 1 --2- 172.21.15.120:0/2873446672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643bfd4f370 0x5643bfd5dfe0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fe96c011be0 tx=0x7fe96c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3dd92195fd20d8f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.816+0000 7fe952ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:11:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.817+0000 7fe952ffd640 1 -- 172.21.15.120:0/2873446672 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c02c060 con 0x5643bfd4f370 2024-09-19T09:11:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.817+0000 7fe952ffd640 1 -- 172.21.15.120:0/2873446672 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fe96c01cba0 con 0x5643bfd4f370 2024-09-19T09:11:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.817+0000 7fe97cc61740 1 -- 172.21.15.120:0/2873446672 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe95c09c380 con 0x5643bfd4f370 2024-09-19T09:11:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.817+0000 7fe952ffd640 1 -- 172.21.15.120:0/2873446672 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe96c011e50 con 0x5643bfd4f370 2024-09-19T09:11:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.817+0000 7fe97cc61740 10 client.18519 fetch_fsmap learned FSMap version 32 2024-09-19T09:11:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.817+0000 7fe97cc61740 1 -- 172.21.15.120:0/2873446672 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5643bfdce540 con 0x5643bfd4f370 2024-09-19T09:11:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.817+0000 7fe952ffd640 1 -- 172.21.15.120:0/2873446672 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe95c09c380 con 0x5643bfd4f370 2024-09-19T09:11:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.817+0000 7fe952ffd640 1 -- 172.21.15.120:0/2873446672 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe96c031070 con 0x5643bfd4f370 2024-09-19T09:11:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.817+0000 7fe97cc61740 10 client.18519 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:11:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.817+0000 7fe97cc61740 10 client.18519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:11:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.817+0000 7fe97cc61740 20 client.18519 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.817+0000 7fe97cc61740 1 --2- 172.21.15.120:0/2873446672 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5643bfdcef20 0x5643bfdd1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.817+0000 7fe97cc61740 4 client.18519 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:11:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.817+0000 7fe97cc61740 1 -- 172.21.15.120:0/2873446672 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5643bfcb4d70 con 0x5643bfdcef20 2024-09-19T09:11:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.817+0000 7fe9737fe640 1 --2- 172.21.15.120:0/2873446672 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5643bfdcef20 0x5643bfdd1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.818+0000 7fe9737fe640 1 --2- 172.21.15.120:0/2873446672 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5643bfdcef20 0x5643bfdd1340 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0051a4efdf2c994 server_cookie=ecd6b39ad3fc00e1 in_seq=0 out_seq=0 2024-09-19T09:11:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.818+0000 7fe952ffd640 10 client.18519 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.969+0000 7fe952ffd640 1 -- 172.21.15.120:0/2873446672 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5643bfcb4d70 con 0x5643bfdcef20 2024-09-19T09:11:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.969+0000 7fe952ffd640 10 client.18519 handle_command_reply: tid=0 2024-09-19T09:11:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.969+0000 7fe97cc61740 1 client.18519 shutdown 2024-09-19T09:11:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.969+0000 7fe97cc61740 1 -- 172.21.15.120:0/2873446672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643bfd4f370 msgr2=0x5643bfd5dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.969+0000 7fe97cc61740 1 --2- 172.21.15.120:0/2873446672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643bfd4f370 0x5643bfd5dfe0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fe96c011be0 tx=0x7fe96c004730 comp rx=0 tx=0).stop 2024-09-19T09:11:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.969+0000 7fe97cc61740 1 -- 172.21.15.120:0/2873446672 shutdown_connections 2024-09-19T09:11:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.969+0000 7fe97cc61740 1 --2- 172.21.15.120:0/2873446672 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5643bfdcef20 0x5643bfdd1340 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.969+0000 7fe97cc61740 1 --2- 172.21.15.120:0/2873446672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643bfd4f370 0x5643bfd5dfe0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.969+0000 7fe952ffd640 0 client.18519 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.969+0000 7fe97cc61740 1 --2- 172.21.15.120:0/2873446672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643bfd4a2e0 0x5643bfd5cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.969+0000 7fe97cc61740 1 --2- 172.21.15.120:0/2873446672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643bfd499a0 0x5643bfd5ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.969+0000 7fe97cc61740 1 -- 172.21.15.120:0/2873446672 >> 172.21.15.120:0/2873446672 conn(0x5643bfd45010 msgr2=0x5643bfd441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.969+0000 7fe97cc61740 1 -- 172.21.15.120:0/2873446672 shutdown_connections 2024-09-19T09:11:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.969+0000 7fe97cc61740 1 -- 172.21.15.120:0/2873446672 wait complete. 2024-09-19T09:11:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.969+0000 7fe97cc61740 20 client.18519 trim_cache size 0 max 16384 2024-09-19T09:11:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:11:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:11:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:11:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:11:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.969+0000 7fe979de8640 1 -- 172.21.15.120:0/1097124143 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9600a32f0 msgr2=0x7fe9600a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.969+0000 7fe979de8640 1 --2- 172.21.15.120:0/1097124143 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9600a32f0 0x7fe9600a57b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe96c0125f0 tx=0x7fe96c0046c0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.969+0000 7fe979de8640 1 -- 172.21.15.120:0/1097124143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe974102d50 msgr2=0x7fe974198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.969+0000 7fe979de8640 1 --2- 172.21.15.120:0/1097124143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe974102d50 0x7fe974198390 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00e9f0 tx=0x7fe95c00eeb0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.969+0000 7fe979de8640 1 -- 172.21.15.120:0/1097124143 shutdown_connections 2024-09-19T09:11:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.969+0000 7fe979de8640 1 --2- 172.21.15.120:0/1097124143 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9600a32f0 0x7fe9600a57b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.969+0000 7fe979de8640 1 --2- 172.21.15.120:0/1097124143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe974103710 0x7fe97419f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.969+0000 7fe979de8640 1 --2- 172.21.15.120:0/1097124143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe974102d50 0x7fe974198390 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.969+0000 7fe979de8640 1 --2- 172.21.15.120:0/1097124143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe974101b60 0x7fe974197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.969+0000 7fe979de8640 1 -- 172.21.15.120:0/1097124143 >> 172.21.15.120:0/1097124143 conn(0x7fe9740fd2c0 msgr2=0x7fe9741060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.969+0000 7fe979de8640 1 -- 172.21.15.120:0/1097124143 shutdown_connections 2024-09-19T09:11:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:11:51.969+0000 7fe979de8640 1 -- 172.21.15.120:0/1097124143 wait complete. 2024-09-19T09:11:52.064 INFO:tasks.workunit.client.0.smithi120.stdout: 1 840897 9.58 MB/sec execute 347 sec latency 92.447 ms 2024-09-19T09:11:52.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.169+0000 7f11814d1640 1 -- 172.21.15.120:0/2676665496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f117c100aa0 msgr2=0x7f117c10ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:52.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.169+0000 7f11814d1640 1 --2- 172.21.15.120:0/2676665496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f117c100aa0 0x7f117c10ea20 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f1174009ca0 tx=0x7f117402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.169+0000 7f11814d1640 1 -- 172.21.15.120:0/2676665496 shutdown_connections 2024-09-19T09:11:52.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.169+0000 7f11814d1640 1 --2- 172.21.15.120:0/2676665496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f117c100aa0 0x7f117c10ea20 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.169+0000 7f11814d1640 1 --2- 172.21.15.120:0/2676665496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f117c100100 0x7f117c100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.169+0000 7f11814d1640 1 --2- 172.21.15.120:0/2676665496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f117c106b80 0x7f117c106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.169+0000 7f11814d1640 1 -- 172.21.15.120:0/2676665496 >> 172.21.15.120:0/2676665496 conn(0x7f117c077230 msgr2=0x7f117c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:52.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.169+0000 7f11814d1640 1 -- 172.21.15.120:0/2676665496 shutdown_connections 2024-09-19T09:11:52.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.169+0000 7f11814d1640 1 -- 172.21.15.120:0/2676665496 wait complete. 2024-09-19T09:11:52.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.169+0000 7f11814d1640 1 Processor -- start 2024-09-19T09:11:52.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.170+0000 7f11814d1640 1 -- start start 2024-09-19T09:11:52.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.170+0000 7f11814d1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f117c100100 0x7f117c198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:52.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.170+0000 7f11814d1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f117c100aa0 0x7f117c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:52.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.170+0000 7f11814d1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f117c106b80 0x7f117c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:52.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.170+0000 7f11814d1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f117c112050 con 0x7f117c106b80 2024-09-19T09:11:52.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.170+0000 7f11814d1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f117c111ed0 con 0x7f117c100100 2024-09-19T09:11:52.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.170+0000 7f11814d1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f117c1121d0 con 0x7f117c100aa0 2024-09-19T09:11:52.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.170+0000 7f117b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f117c106b80 0x7f117c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:52.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.170+0000 7f117affd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f117c100100 0x7f117c198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:52.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.170+0000 7f117b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f117c106b80 0x7f117c19ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46190/0 (socket says 172.21.15.120:46190) 2024-09-19T09:11:52.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.170+0000 7f117b7fe640 1 -- 172.21.15.120:0/4155238935 learned_addr learned my addr 172.21.15.120:0/4155238935 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:52.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.170+0000 7f117a7fc640 1 --2- 172.21.15.120:0/4155238935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f117c100aa0 0x7f117c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:52.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.170+0000 7f117affd640 1 -- 172.21.15.120:0/4155238935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f117c100aa0 msgr2=0x7f117c198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:52.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.170+0000 7f117affd640 1 --2- 172.21.15.120:0/4155238935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f117c100aa0 0x7f117c198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.170+0000 7f117affd640 1 -- 172.21.15.120:0/4155238935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f117c106b80 msgr2=0x7f117c19ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:52.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.170+0000 7f117affd640 1 --2- 172.21.15.120:0/4155238935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f117c106b80 0x7f117c19ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.170+0000 7f117affd640 1 -- 172.21.15.120:0/4155238935 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f117c19d4e0 con 0x7f117c100100 2024-09-19T09:11:52.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.171+0000 7f117affd640 1 --2- 172.21.15.120:0/4155238935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f117c100100 0x7f117c198530 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f116c00e9f0 tx=0x7f116c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:52.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.171+0000 7f1163fff640 1 -- 172.21.15.120:0/4155238935 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c00ccc0 con 0x7f117c100100 2024-09-19T09:11:52.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.171+0000 7f11814d1640 1 -- 172.21.15.120:0/4155238935 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f117c19d7d0 con 0x7f117c100100 2024-09-19T09:11:52.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.171+0000 7f1163fff640 1 -- 172.21.15.120:0/4155238935 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f116c005020 con 0x7f117c100100 2024-09-19T09:11:52.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.171+0000 7f1163fff640 1 -- 172.21.15.120:0/4155238935 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c0049e0 con 0x7f117c100100 2024-09-19T09:11:52.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.171+0000 7f11814d1640 1 -- 172.21.15.120:0/4155238935 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f117c1a50b0 con 0x7f117c100100 2024-09-19T09:11:52.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.173+0000 7f1163fff640 1 -- 172.21.15.120:0/4155238935 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f116c005560 con 0x7f117c100100 2024-09-19T09:11:52.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.174+0000 7f11838aa740 1 Processor -- start 2024-09-19T09:11:52.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.174+0000 7f11838aa740 1 -- start start 2024-09-19T09:11:52.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.174+0000 7f11838aa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594ff47eb40 0x5594ff47d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:52.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.174+0000 7f11838aa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594ff47d5d0 0x5594ff47da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:52.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.174+0000 7f11838aa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594ff488370 0x5594ff48a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:52.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.174+0000 7f11838aa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5594ff3e32f0 con 0x5594ff47eb40 2024-09-19T09:11:52.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.174+0000 7f11838aa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5594ff38acf0 con 0x5594ff488370 2024-09-19T09:11:52.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.174+0000 7f11838aa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5594ff3df2b0 con 0x5594ff47d5d0 2024-09-19T09:11:52.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.174+0000 7f1163fff640 1 --2- 172.21.15.120:0/4155238935 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f11640a76d0 0x7f11640a9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:52.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.174+0000 7f117b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594ff47eb40 0x5594ff47d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:52.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.174+0000 7f117b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594ff47eb40 0x5594ff47d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46200/0 (socket says 172.21.15.120:46200) 2024-09-19T09:11:52.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.174+0000 7f117b7fe640 1 -- 172.21.15.120:0/2331620715 learned_addr learned my addr 172.21.15.120:0/2331620715 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:52.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.174+0000 7f117a7fc640 1 --2- 172.21.15.120:0/2331620715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594ff488370 0x5594ff48a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:52.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.174+0000 7f1163fff640 1 -- 172.21.15.120:0/4155238935 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f116c0cf7d0 con 0x7f117c100100 2024-09-19T09:11:52.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.174+0000 7f117affd640 1 --2- 172.21.15.120:0/2331620715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594ff47d5d0 0x5594ff47da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:52.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.174+0000 7f117b7fe640 1 --2- 172.21.15.120:0/4155238935 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f11640a76d0 0x7f11640a9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:52.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.175+0000 7f117a7fc640 1 -- 172.21.15.120:0/2331620715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594ff47d5d0 msgr2=0x5594ff47da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:52.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.175+0000 7f117a7fc640 1 --2- 172.21.15.120:0/2331620715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594ff47d5d0 0x5594ff47da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.175+0000 7f117a7fc640 1 -- 172.21.15.120:0/2331620715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594ff47eb40 msgr2=0x5594ff47d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:52.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.175+0000 7f117a7fc640 1 --2- 172.21.15.120:0/2331620715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594ff47eb40 0x5594ff47d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.175+0000 7f117a7fc640 1 -- 172.21.15.120:0/2331620715 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594ff36f3a0 con 0x5594ff488370 2024-09-19T09:11:52.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.175+0000 7f117b7fe640 1 --2- 172.21.15.120:0/2331620715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594ff47eb40 0x5594ff47d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:11:52.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.175+0000 7f117a7fc640 1 --2- 172.21.15.120:0/2331620715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594ff488370 0x5594ff48a760 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1170009bf0 tx=0x7f1170011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d57d1918ba0477b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:52.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.175+0000 7f117b7fe640 1 --2- 172.21.15.120:0/4155238935 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f11640a76d0 0x7f11640a9b90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1174009430 tx=0x7f117402a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:52.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.175+0000 7f11617fa640 1 -- 172.21.15.120:0/2331620715 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1170004050 con 0x5594ff488370 2024-09-19T09:11:52.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.175+0000 7f11617fa640 1 -- 172.21.15.120:0/2331620715 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1170005500 con 0x5594ff488370 2024-09-19T09:11:52.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.175+0000 7f11617fa640 1 -- 172.21.15.120:0/2331620715 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1170004de0 con 0x5594ff488370 2024-09-19T09:11:52.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.176+0000 7f11838aa740 1 -- 172.21.15.120:0/2331620715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594ff488370 msgr2=0x5594ff48a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:52.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.176+0000 7f11838aa740 1 --2- 172.21.15.120:0/2331620715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594ff488370 0x5594ff48a760 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1170009bf0 tx=0x7f1170011810 comp rx=0 tx=0).stop 2024-09-19T09:11:52.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.176+0000 7f11838aa740 1 -- 172.21.15.120:0/2331620715 shutdown_connections 2024-09-19T09:11:52.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.176+0000 7f11838aa740 1 --2- 172.21.15.120:0/2331620715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594ff488370 0x5594ff48a760 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.176+0000 7f11838aa740 1 --2- 172.21.15.120:0/2331620715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594ff47d5d0 0x5594ff47da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.176+0000 7f11838aa740 1 --2- 172.21.15.120:0/2331620715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594ff47eb40 0x5594ff47d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.176+0000 7f11838aa740 1 -- 172.21.15.120:0/2331620715 >> 172.21.15.120:0/2331620715 conn(0x5594ff446310 msgr2=0x5594ff479c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:52.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.176+0000 7f11838aa740 1 -- 172.21.15.120:0/2331620715 shutdown_connections 2024-09-19T09:11:52.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.176+0000 7f11838aa740 1 -- 172.21.15.120:0/2331620715 wait complete. 2024-09-19T09:11:52.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.176+0000 7f11838aa740 1 -- start start 2024-09-19T09:11:52.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.177+0000 7f11838aa740 1 Processor -- start 2024-09-19T09:11:52.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.177+0000 7f11838aa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594ff47d5d0 0x5594ff487cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:52.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.177+0000 7f11838aa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594ff47eb40 0x5594ff484fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:52.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.177+0000 7f11838aa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594ff488370 0x5594ff485520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:52.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.177+0000 7f11838aa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5594ff48b070 con 0x5594ff47d5d0 2024-09-19T09:11:52.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.177+0000 7f11838aa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5594ff39e060 con 0x5594ff47eb40 2024-09-19T09:11:52.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.177+0000 7f11838aa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5594ff48b1f0 con 0x5594ff488370 2024-09-19T09:11:52.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.177+0000 7f11838aa740 10 client.0 _ll_register_callbacks cb 0x5594ff0f18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:52.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.177+0000 7f117a7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594ff47eb40 0x5594ff484fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:52.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.177+0000 7f117affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594ff47d5d0 0x5594ff487cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:52.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.177+0000 7f117a7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594ff47eb40 0x5594ff484fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59680/0 (socket says 172.21.15.120:59680) 2024-09-19T09:11:52.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.177+0000 7f117a7fc640 1 -- 172.21.15.120:0/3360372586 learned_addr learned my addr 172.21.15.120:0/3360372586 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:52.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.177+0000 7f117b7fe640 1 --2- 172.21.15.120:0/3360372586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594ff488370 0x5594ff485520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:52.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.177+0000 7f117a7fc640 1 -- 172.21.15.120:0/3360372586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594ff488370 msgr2=0x5594ff485520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:52.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.177+0000 7f117a7fc640 1 --2- 172.21.15.120:0/3360372586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594ff488370 0x5594ff485520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.177+0000 7f117a7fc640 1 -- 172.21.15.120:0/3360372586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594ff47d5d0 msgr2=0x5594ff487cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:52.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.177+0000 7f117a7fc640 1 --2- 172.21.15.120:0/3360372586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594ff47d5d0 0x5594ff487cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.177+0000 7f117a7fc640 1 -- 172.21.15.120:0/3360372586 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594ff1543c0 con 0x5594ff47eb40 2024-09-19T09:11:52.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.177+0000 7f117affd640 1 --2- 172.21.15.120:0/3360372586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594ff47d5d0 0x5594ff487cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:52.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.178+0000 7f117a7fc640 1 --2- 172.21.15.120:0/3360372586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594ff47eb40 0x5594ff484fa0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f117001b040 tx=0x7f1170004530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e91f7af87f2107b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:52.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.178+0000 7f115eff5640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:11:52.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.178+0000 7f115eff5640 1 -- 172.21.15.120:0/3360372586 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1170029070 con 0x5594ff47eb40 2024-09-19T09:11:52.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.178+0000 7f11838aa740 1 -- 172.21.15.120:0/3360372586 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1170005500 con 0x5594ff47eb40 2024-09-19T09:11:52.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.178+0000 7f115eff5640 1 -- 172.21.15.120:0/3360372586 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f1170004d60 con 0x5594ff47eb40 2024-09-19T09:11:52.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.178+0000 7f115eff5640 1 -- 172.21.15.120:0/3360372586 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1170035050 con 0x5594ff47eb40 2024-09-19T09:11:52.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.178+0000 7f11838aa740 10 client.27662 fetch_fsmap learned FSMap version 32 2024-09-19T09:11:52.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.178+0000 7f11838aa740 1 -- 172.21.15.120:0/3360372586 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5594ff496aa0 con 0x5594ff47eb40 2024-09-19T09:11:52.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.178+0000 7f115eff5640 1 -- 172.21.15.120:0/3360372586 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1170005500 con 0x5594ff47eb40 2024-09-19T09:11:52.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.178+0000 7f115eff5640 1 -- 172.21.15.120:0/3360372586 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f1170023430 con 0x5594ff47eb40 2024-09-19T09:11:52.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.178+0000 7f11838aa740 10 client.27662 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:11:52.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.178+0000 7f11838aa740 10 client.27662 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:11:52.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.178+0000 7f11838aa740 20 client.27662 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:52.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.178+0000 7f11838aa740 1 --2- 172.21.15.120:0/3360372586 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5594ff497e50 0x5594ff49a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:52.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.178+0000 7f11838aa740 4 client.27662 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:11:52.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.178+0000 7f11838aa740 1 -- 172.21.15.120:0/3360372586 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594ff3f0860 con 0x5594ff497e50 2024-09-19T09:11:52.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.179+0000 7f117affd640 1 --2- 172.21.15.120:0/3360372586 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5594ff497e50 0x5594ff49a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:52.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.179+0000 7f117affd640 1 --2- 172.21.15.120:0/3360372586 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5594ff497e50 0x5594ff49a240 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b0a05b139cf81080 server_cookie=a91bc4f917d515a1 in_seq=0 out_seq=0 2024-09-19T09:11:52.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.179+0000 7f115eff5640 10 client.27662 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:11:52.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.180+0000 7f115eff5640 1 -- 172.21.15.120:0/3360372586 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5594ff3f0860 con 0x5594ff497e50 2024-09-19T09:11:52.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.180+0000 7f115eff5640 10 client.27662 handle_command_reply: tid=0 2024-09-19T09:11:52.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.180+0000 7f11838aa740 1 client.27662 shutdown 2024-09-19T09:11:52.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.180+0000 7f11838aa740 1 -- 172.21.15.120:0/3360372586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594ff47eb40 msgr2=0x5594ff484fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:52.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.180+0000 7f11838aa740 1 --2- 172.21.15.120:0/3360372586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594ff47eb40 0x5594ff484fa0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f117001b040 tx=0x7f1170004530 comp rx=0 tx=0).stop 2024-09-19T09:11:52.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.180+0000 7f11838aa740 1 -- 172.21.15.120:0/3360372586 shutdown_connections 2024-09-19T09:11:52.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.180+0000 7f11838aa740 1 --2- 172.21.15.120:0/3360372586 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5594ff497e50 0x5594ff49a240 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.180+0000 7f11838aa740 1 --2- 172.21.15.120:0/3360372586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594ff488370 0x5594ff485520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.180+0000 7f11838aa740 1 --2- 172.21.15.120:0/3360372586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594ff47eb40 0x5594ff484fa0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.180+0000 7f11838aa740 1 --2- 172.21.15.120:0/3360372586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594ff47d5d0 0x5594ff487cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.180+0000 7f115eff5640 0 client.27662 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:11:52.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.180+0000 7f11838aa740 1 -- 172.21.15.120:0/3360372586 >> 172.21.15.120:0/3360372586 conn(0x5594ff446310 msgr2=0x5594ff47f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:52.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.180+0000 7f11838aa740 1 -- 172.21.15.120:0/3360372586 shutdown_connections 2024-09-19T09:11:52.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.180+0000 7f11838aa740 1 -- 172.21.15.120:0/3360372586 wait complete. 2024-09-19T09:11:52.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.180+0000 7f11838aa740 20 client.27662 trim_cache size 0 max 16384 2024-09-19T09:11:52.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.193+0000 7f11838aa740 1 Processor -- start 2024-09-19T09:11:52.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.193+0000 7f11838aa740 1 -- start start 2024-09-19T09:11:52.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.193+0000 7f11838aa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594ff47d5d0 0x5594ff494310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:52.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.193+0000 7f11838aa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594ff47eb40 0x5594ff480120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:52.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.193+0000 7f11838aa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594ff488370 0x5594ff480660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:52.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.193+0000 7f11838aa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f117c1ab450 con 0x5594ff488370 2024-09-19T09:11:52.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.193+0000 7f11838aa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f117c1acf90 con 0x5594ff47eb40 2024-09-19T09:11:52.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.193+0000 7f11838aa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f117c1abb20 con 0x5594ff47d5d0 2024-09-19T09:11:52.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.194+0000 7f117b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594ff488370 0x5594ff480660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:52.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.194+0000 7f117b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594ff488370 0x5594ff480660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46214/0 (socket says 172.21.15.120:46214) 2024-09-19T09:11:52.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.194+0000 7f117b7fe640 1 -- 172.21.15.120:0/2209065666 learned_addr learned my addr 172.21.15.120:0/2209065666 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:52.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.194+0000 7f117affd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594ff47d5d0 0x5594ff494310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:52.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.194+0000 7f117a7fc640 1 --2- 172.21.15.120:0/2209065666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594ff47eb40 0x5594ff480120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:52.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.194+0000 7f117affd640 1 -- 172.21.15.120:0/2209065666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594ff47eb40 msgr2=0x5594ff480120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:52.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.194+0000 7f117affd640 1 --2- 172.21.15.120:0/2209065666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594ff47eb40 0x5594ff480120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.194+0000 7f117affd640 1 -- 172.21.15.120:0/2209065666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594ff488370 msgr2=0x5594ff480660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:52.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.194+0000 7f117affd640 1 --2- 172.21.15.120:0/2209065666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594ff488370 0x5594ff480660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.194+0000 7f117affd640 1 -- 172.21.15.120:0/2209065666 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594ff49a820 con 0x5594ff47d5d0 2024-09-19T09:11:52.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.194+0000 7f117a7fc640 1 --2- 172.21.15.120:0/2209065666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594ff47eb40 0x5594ff480120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:52.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.195+0000 7f117affd640 1 --2- 172.21.15.120:0/2209065666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594ff47d5d0 0x5594ff494310 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f116c098210 tx=0x7f116c0054d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a612b6ad94e64e31 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:52.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.195+0000 7f1160ff9640 1 -- 172.21.15.120:0/2209065666 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c01c040 con 0x5594ff47d5d0 2024-09-19T09:11:52.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.195+0000 7f1160ff9640 1 -- 172.21.15.120:0/2209065666 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f116c09bdb0 con 0x5594ff47d5d0 2024-09-19T09:11:52.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.195+0000 7f1160ff9640 1 -- 172.21.15.120:0/2209065666 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c0a4590 con 0x5594ff47d5d0 2024-09-19T09:11:52.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.196+0000 7f11838aa740 1 -- 172.21.15.120:0/2209065666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594ff47d5d0 msgr2=0x5594ff494310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:52.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.196+0000 7f11838aa740 1 --2- 172.21.15.120:0/2209065666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594ff47d5d0 0x5594ff494310 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f116c098210 tx=0x7f116c0054d0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.196+0000 7f11838aa740 1 -- 172.21.15.120:0/2209065666 shutdown_connections 2024-09-19T09:11:52.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.196+0000 7f11838aa740 1 --2- 172.21.15.120:0/2209065666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594ff488370 0x5594ff480660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.196+0000 7f11838aa740 1 --2- 172.21.15.120:0/2209065666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594ff47eb40 0x5594ff480120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.196+0000 7f11838aa740 1 --2- 172.21.15.120:0/2209065666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594ff47d5d0 0x5594ff494310 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.196+0000 7f11838aa740 1 -- 172.21.15.120:0/2209065666 >> 172.21.15.120:0/2209065666 conn(0x5594ff446310 msgr2=0x5594ff487020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:52.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.196+0000 7f11838aa740 1 -- 172.21.15.120:0/2209065666 shutdown_connections 2024-09-19T09:11:52.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.196+0000 7f11838aa740 1 -- 172.21.15.120:0/2209065666 wait complete. 2024-09-19T09:11:52.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.196+0000 7f11838aa740 1 -- start start 2024-09-19T09:11:52.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.196+0000 7f11838aa740 1 Processor -- start 2024-09-19T09:11:52.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.197+0000 7f11838aa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594ff47d5d0 0x5594ff490ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:52.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.197+0000 7f11838aa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594ff47eb40 0x5594ff481bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:52.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.197+0000 7f11838aa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594ff488370 0x5594ff482110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:52.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.197+0000 7f11838aa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f117c1ac8c0 con 0x5594ff47eb40 2024-09-19T09:11:52.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.197+0000 7f11838aa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f117c1ac1f0 con 0x5594ff488370 2024-09-19T09:11:52.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.197+0000 7f11838aa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f117c1a9410 con 0x5594ff47d5d0 2024-09-19T09:11:52.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.197+0000 7f11838aa740 10 client.0 _ll_register_callbacks cb 0x7f116c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:52.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.197+0000 7f117a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594ff47eb40 0x5594ff481bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:52.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.197+0000 7f117a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594ff47eb40 0x5594ff481bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46216/0 (socket says 172.21.15.120:46216) 2024-09-19T09:11:52.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.197+0000 7f117a7fc640 1 -- 172.21.15.120:0/1614612288 learned_addr learned my addr 172.21.15.120:0/1614612288 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:52.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.197+0000 7f117affd640 1 --2- 172.21.15.120:0/1614612288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594ff47d5d0 0x5594ff490ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:52.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.197+0000 7f117b7fe640 1 --2- 172.21.15.120:0/1614612288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594ff488370 0x5594ff482110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:52.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.197+0000 7f117a7fc640 1 -- 172.21.15.120:0/1614612288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594ff47d5d0 msgr2=0x5594ff490ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:52.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.197+0000 7f117a7fc640 1 --2- 172.21.15.120:0/1614612288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594ff47d5d0 0x5594ff490ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.197+0000 7f117a7fc640 1 -- 172.21.15.120:0/1614612288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594ff488370 msgr2=0x5594ff482110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:52.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.197+0000 7f117a7fc640 1 --2- 172.21.15.120:0/1614612288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594ff488370 0x5594ff482110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.197+0000 7f117a7fc640 1 -- 172.21.15.120:0/1614612288 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594ff482800 con 0x5594ff47eb40 2024-09-19T09:11:52.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.197+0000 7f117a7fc640 1 --2- 172.21.15.120:0/1614612288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594ff47eb40 0x5594ff481bd0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f1170009250 tx=0x7f1170011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e1400620e155eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:52.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.197+0000 7f115eff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:11:52.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.198+0000 7f115eff5640 1 -- 172.21.15.120:0/1614612288 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117003e060 con 0x5594ff47eb40 2024-09-19T09:11:52.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.198+0000 7f115eff5640 1 -- 172.21.15.120:0/1614612288 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f11700035f0 con 0x5594ff47eb40 2024-09-19T09:11:52.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.198+0000 7f11838aa740 1 -- 172.21.15.120:0/1614612288 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f116c09bdb0 con 0x5594ff47eb40 2024-09-19T09:11:52.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.198+0000 7f115eff5640 1 -- 172.21.15.120:0/1614612288 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1170029050 con 0x5594ff47eb40 2024-09-19T09:11:52.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.198+0000 7f11838aa740 10 client.18537 fetch_fsmap learned FSMap version 32 2024-09-19T09:11:52.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.198+0000 7f11838aa740 1 -- 172.21.15.120:0/1614612288 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5594ff502940 con 0x5594ff47eb40 2024-09-19T09:11:52.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.198+0000 7f115eff5640 1 -- 172.21.15.120:0/1614612288 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f116c09bdb0 con 0x5594ff47eb40 2024-09-19T09:11:52.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.198+0000 7f115eff5640 1 -- 172.21.15.120:0/1614612288 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f117002e440 con 0x5594ff47eb40 2024-09-19T09:11:52.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.198+0000 7f11838aa740 10 client.18537 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:11:52.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.198+0000 7f11838aa740 10 client.18537 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:11:52.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.198+0000 7f11838aa740 20 client.18537 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:52.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.198+0000 7f11838aa740 1 --2- 172.21.15.120:0/1614612288 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5594ff503030 0x5594ff505450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:52.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.198+0000 7f11838aa740 4 client.18537 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:11:52.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.198+0000 7f11838aa740 1 -- 172.21.15.120:0/1614612288 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5594ff3ead70 con 0x5594ff503030 2024-09-19T09:11:52.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.198+0000 7f117affd640 1 --2- 172.21.15.120:0/1614612288 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5594ff503030 0x5594ff505450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:52.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.199+0000 7f115eff5640 10 client.18537 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:11:52.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.199+0000 7f117affd640 1 --2- 172.21.15.120:0/1614612288 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5594ff503030 0x5594ff505450 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2d82becd8f6f01e2 server_cookie=718674401e5c1731 in_seq=0 out_seq=0 2024-09-19T09:11:52.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.199+0000 7f115eff5640 1 -- 172.21.15.120:0/1614612288 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5594ff3ead70 con 0x5594ff503030 2024-09-19T09:11:52.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.199+0000 7f115eff5640 10 client.18537 handle_command_reply: tid=0 2024-09-19T09:11:52.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.199+0000 7f11838aa740 1 client.18537 shutdown 2024-09-19T09:11:52.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.199+0000 7f11838aa740 1 -- 172.21.15.120:0/1614612288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594ff47eb40 msgr2=0x5594ff481bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:52.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.199+0000 7f11838aa740 1 --2- 172.21.15.120:0/1614612288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594ff47eb40 0x5594ff481bd0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f1170009250 tx=0x7f1170011dc0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.199+0000 7f11838aa740 1 -- 172.21.15.120:0/1614612288 shutdown_connections 2024-09-19T09:11:52.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.199+0000 7f11838aa740 1 --2- 172.21.15.120:0/1614612288 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5594ff503030 0x5594ff505450 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.199+0000 7f11838aa740 1 --2- 172.21.15.120:0/1614612288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594ff488370 0x5594ff482110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.199+0000 7f11838aa740 1 --2- 172.21.15.120:0/1614612288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594ff47eb40 0x5594ff481bd0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.199+0000 7f11838aa740 1 --2- 172.21.15.120:0/1614612288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5594ff47d5d0 0x5594ff490ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.199+0000 7f11838aa740 1 -- 172.21.15.120:0/1614612288 >> 172.21.15.120:0/1614612288 conn(0x5594ff446310 msgr2=0x5594ff4784e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:52.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.199+0000 7f115eff5640 0 client.18537 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:11:52.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.199+0000 7f11838aa740 1 -- 172.21.15.120:0/1614612288 shutdown_connections 2024-09-19T09:11:52.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.199+0000 7f11838aa740 1 -- 172.21.15.120:0/1614612288 wait complete. 2024-09-19T09:11:52.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.199+0000 7f11838aa740 20 client.18537 trim_cache size 0 max 16384 2024-09-19T09:11:52.205 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:11:52.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.200+0000 7f11814d1640 1 -- 172.21.15.120:0/4155238935 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f11640a76d0 msgr2=0x7f11640a9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:52.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.200+0000 7f11814d1640 1 --2- 172.21.15.120:0/4155238935 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f11640a76d0 0x7f11640a9b90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1174009430 tx=0x7f117402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.200+0000 7f11814d1640 1 -- 172.21.15.120:0/4155238935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f117c100100 msgr2=0x7f117c198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:52.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.200+0000 7f11814d1640 1 --2- 172.21.15.120:0/4155238935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f117c100100 0x7f117c198530 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f116c00e9f0 tx=0x7f116c00eeb0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.200+0000 7f11814d1640 1 -- 172.21.15.120:0/4155238935 shutdown_connections 2024-09-19T09:11:52.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.200+0000 7f11814d1640 1 --2- 172.21.15.120:0/4155238935 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f11640a76d0 0x7f11640a9b90 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.200+0000 7f11814d1640 1 --2- 172.21.15.120:0/4155238935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f117c106b80 0x7f117c19ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.200+0000 7f11814d1640 1 --2- 172.21.15.120:0/4155238935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f117c100aa0 0x7f117c198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.200+0000 7f11814d1640 1 --2- 172.21.15.120:0/4155238935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f117c100100 0x7f117c198530 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:52.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.200+0000 7f11814d1640 1 -- 172.21.15.120:0/4155238935 >> 172.21.15.120:0/4155238935 conn(0x7f117c077230 msgr2=0x7f117c104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:52.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.200+0000 7f11814d1640 1 -- 172.21.15.120:0/4155238935 shutdown_connections 2024-09-19T09:11:52.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:52.200+0000 7f11814d1640 1 -- 172.21.15.120:0/4155238935 wait complete. 2024-09-19T09:11:53.064 INFO:tasks.workunit.client.0.smithi120.stdout: 1 843442 9.58 MB/sec execute 348 sec latency 12.210 ms 2024-09-19T09:11:53.087 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:11:53.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:52 smithi196 ceph-mon[35150]: pgmap v613: 97 pgs: 97 active+clean; 54 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.3 MiB/s wr, 452 op/s 2024-09-19T09:11:53.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:52 smithi153 ceph-mon[35071]: pgmap v613: 97 pgs: 97 active+clean; 54 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.3 MiB/s wr, 452 op/s 2024-09-19T09:11:53.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:52 smithi120 ceph-mon[31150]: pgmap v613: 97 pgs: 97 active+clean; 54 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.3 MiB/s wr, 452 op/s 2024-09-19T09:11:53.353 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:11:54.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.022+0000 7febab91e640 1 -- 172.21.15.120:0/1547651892 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feba4100e20 msgr2=0x7feba4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:54.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.022+0000 7febab91e640 1 --2- 172.21.15.120:0/1547651892 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feba4100e20 0x7feba4101200 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7feba0009900 tx=0x7feba002a480 comp rx=0 tx=0).stop 2024-09-19T09:11:54.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.022+0000 7febab91e640 1 -- 172.21.15.120:0/1547651892 shutdown_connections 2024-09-19T09:11:54.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.022+0000 7febab91e640 1 --2- 172.21.15.120:0/1547651892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feba4102ef0 0x7feba410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.022+0000 7febab91e640 1 --2- 172.21.15.120:0/1547651892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feba4102570 0x7feba41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.022+0000 7febab91e640 1 --2- 172.21.15.120:0/1547651892 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feba4100e20 0x7feba4101200 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.022+0000 7febab91e640 1 -- 172.21.15.120:0/1547651892 >> 172.21.15.120:0/1547651892 conn(0x7feba40fc7d0 msgr2=0x7feba40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:54.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.022+0000 7febab91e640 1 -- 172.21.15.120:0/1547651892 shutdown_connections 2024-09-19T09:11:54.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.022+0000 7febab91e640 1 -- 172.21.15.120:0/1547651892 wait complete. 2024-09-19T09:11:54.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.022+0000 7febab91e640 1 Processor -- start 2024-09-19T09:11:54.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.023+0000 7febab91e640 1 -- start start 2024-09-19T09:11:54.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.023+0000 7febab91e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feba4100e20 0x7feba4199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:54.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.023+0000 7febab91e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feba4102570 0x7feba4199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:54.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.023+0000 7febab91e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feba4102ef0 0x7feba41a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:54.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.023+0000 7febab91e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feba4110f80 con 0x7feba4102ef0 2024-09-19T09:11:54.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.023+0000 7febab91e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feba4110e00 con 0x7feba4100e20 2024-09-19T09:11:54.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.023+0000 7febab91e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feba4111100 con 0x7feba4102570 2024-09-19T09:11:54.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.023+0000 7feba9ebb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feba4102ef0 0x7feba41a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:54.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.023+0000 7feba96ba640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feba4100e20 0x7feba4199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:54.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.023+0000 7feba9ebb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feba4102ef0 0x7feba41a43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46236/0 (socket says 172.21.15.120:46236) 2024-09-19T09:11:54.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.023+0000 7feba9ebb640 1 -- 172.21.15.120:0/2552830269 learned_addr learned my addr 172.21.15.120:0/2552830269 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:54.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.023+0000 7feba8eb9640 1 --2- 172.21.15.120:0/2552830269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feba4102570 0x7feba4199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:54.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.023+0000 7feba9ebb640 1 -- 172.21.15.120:0/2552830269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feba4102570 msgr2=0x7feba4199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:54.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.023+0000 7feba9ebb640 1 --2- 172.21.15.120:0/2552830269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feba4102570 0x7feba4199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.023+0000 7feba9ebb640 1 -- 172.21.15.120:0/2552830269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feba4100e20 msgr2=0x7feba4199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:54.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.023+0000 7feba9ebb640 1 --2- 172.21.15.120:0/2552830269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feba4100e20 0x7feba4199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.023+0000 7feba9ebb640 1 -- 172.21.15.120:0/2552830269 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba41a48e0 con 0x7feba4102ef0 2024-09-19T09:11:54.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.023+0000 7feba96ba640 1 --2- 172.21.15.120:0/2552830269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feba4100e20 0x7feba4199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:54.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.023+0000 7feba9ebb640 1 --2- 172.21.15.120:0/2552830269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feba4102ef0 0x7feba41a43a0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7feb9c00eb10 tx=0x7feb9c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:54.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.024+0000 7feb9a7fc640 1 -- 172.21.15.120:0/2552830269 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c004b20 con 0x7feba4102ef0 2024-09-19T09:11:54.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.024+0000 7feb9a7fc640 1 -- 172.21.15.120:0/2552830269 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb9c004cc0 con 0x7feba4102ef0 2024-09-19T09:11:54.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.024+0000 7febab91e640 1 -- 172.21.15.120:0/2552830269 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba41a4ab0 con 0x7feba4102ef0 2024-09-19T09:11:54.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.024+0000 7feb9a7fc640 1 -- 172.21.15.120:0/2552830269 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c010760 con 0x7feba4102ef0 2024-09-19T09:11:54.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.024+0000 7febab91e640 1 -- 172.21.15.120:0/2552830269 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba41a4fc0 con 0x7feba4102ef0 2024-09-19T09:11:54.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.026+0000 7feb9a7fc640 1 -- 172.21.15.120:0/2552830269 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7feb9c005020 con 0x7feba4102ef0 2024-09-19T09:11:54.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.027+0000 7febadcf7740 1 Processor -- start 2024-09-19T09:11:54.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.027+0000 7febadcf7740 1 -- start start 2024-09-19T09:11:54.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.027+0000 7feb9a7fc640 1 --2- 172.21.15.120:0/2552830269 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7feb900a3340 0x7feb900a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:54.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.027+0000 7febadcf7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557872fab40 0x5557872f9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:54.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.027+0000 7febadcf7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557872f95d0 0x5557872f9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:54.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.027+0000 7febadcf7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555787304370 0x555787306760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:54.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.027+0000 7febadcf7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55578725f2f0 con 0x555787304370 2024-09-19T09:11:54.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.027+0000 7febadcf7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555787206cf0 con 0x5557872f95d0 2024-09-19T09:11:54.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.027+0000 7febadcf7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55578725b2b0 con 0x5557872fab40 2024-09-19T09:11:54.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.027+0000 7feb9a7fc640 1 -- 172.21.15.120:0/2552830269 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7feb9c0cfde0 con 0x7feba4102ef0 2024-09-19T09:11:54.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.027+0000 7feba96ba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557872fab40 0x5557872f9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:54.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.027+0000 7feba96ba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557872fab40 0x5557872f9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:46718/0 (socket says 172.21.15.120:46718) 2024-09-19T09:11:54.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.028+0000 7feba8eb9640 1 --2- 172.21.15.120:0/2552830269 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7feb900a3340 0x7feb900a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:54.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.028+0000 7feba96ba640 1 -- 172.21.15.120:0/1762254145 learned_addr learned my addr 172.21.15.120:0/1762254145 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:54.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.028+0000 7feba8eb9640 1 --2- 172.21.15.120:0/1762254145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557872f95d0 0x5557872f9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:54.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.028+0000 7feba9ebb640 1 --2- 172.21.15.120:0/1762254145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555787304370 0x555787306760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:54.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.028+0000 7feba96ba640 1 -- 172.21.15.120:0/1762254145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557872f95d0 msgr2=0x5557872f9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:54.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.028+0000 7feba96ba640 1 --2- 172.21.15.120:0/1762254145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557872f95d0 0x5557872f9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.028+0000 7feba96ba640 1 -- 172.21.15.120:0/1762254145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555787304370 msgr2=0x555787306760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:54.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.028+0000 7feba96ba640 1 --2- 172.21.15.120:0/1762254145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555787304370 0x555787306760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.028+0000 7feba96ba640 1 -- 172.21.15.120:0/1762254145 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557871eb3a0 con 0x5557872fab40 2024-09-19T09:11:54.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.028+0000 7feba8eb9640 1 --2- 172.21.15.120:0/2552830269 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7feb900a3340 0x7feb900a5800 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7feba406aac0 tx=0x7feb94009990 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:54.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.028+0000 7feba96ba640 1 --2- 172.21.15.120:0/1762254145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557872fab40 0x5557872f9090 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7feba002a9a0 tx=0x7feba0005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8a39829b62a96ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:54.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.028+0000 7feb8f7fe640 1 -- 172.21.15.120:0/1762254145 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba0035030 con 0x5557872fab40 2024-09-19T09:11:54.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.029+0000 7feb8f7fe640 1 -- 172.21.15.120:0/1762254145 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feba0040070 con 0x5557872fab40 2024-09-19T09:11:54.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.029+0000 7feb8f7fe640 1 -- 172.21.15.120:0/1762254145 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba0031b40 con 0x5557872fab40 2024-09-19T09:11:54.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.029+0000 7febadcf7740 1 -- 172.21.15.120:0/1762254145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557872fab40 msgr2=0x5557872f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:54.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.029+0000 7febadcf7740 1 --2- 172.21.15.120:0/1762254145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557872fab40 0x5557872f9090 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7feba002a9a0 tx=0x7feba0005dd0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.029+0000 7febadcf7740 1 -- 172.21.15.120:0/1762254145 shutdown_connections 2024-09-19T09:11:54.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.029+0000 7febadcf7740 1 --2- 172.21.15.120:0/1762254145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555787304370 0x555787306760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.029+0000 7febadcf7740 1 --2- 172.21.15.120:0/1762254145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557872f95d0 0x5557872f9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.029+0000 7febadcf7740 1 --2- 172.21.15.120:0/1762254145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557872fab40 0x5557872f9090 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.029+0000 7febadcf7740 1 -- 172.21.15.120:0/1762254145 >> 172.21.15.120:0/1762254145 conn(0x5557872c2310 msgr2=0x5557872f5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:54.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.029+0000 7febadcf7740 1 -- 172.21.15.120:0/1762254145 shutdown_connections 2024-09-19T09:11:54.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.029+0000 7febadcf7740 1 -- 172.21.15.120:0/1762254145 wait complete. 2024-09-19T09:11:54.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.030+0000 7febadcf7740 1 -- start start 2024-09-19T09:11:54.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.030+0000 7febadcf7740 1 Processor -- start 2024-09-19T09:11:54.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.030+0000 7febadcf7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557872f95d0 0x555787303d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:54.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.030+0000 7febadcf7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557872fab40 0x555787301020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:54.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.030+0000 7febadcf7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555787304370 0x555787301610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:54.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.030+0000 7febadcf7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555787307010 con 0x5557872f95d0 2024-09-19T09:11:54.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.030+0000 7febadcf7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55578721a060 con 0x5557872fab40 2024-09-19T09:11:54.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.030+0000 7febadcf7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555787307190 con 0x555787304370 2024-09-19T09:11:54.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.030+0000 7febadcf7740 10 client.0 _ll_register_callbacks cb 0x555786f6d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:54.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.030+0000 7feba96ba640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557872f95d0 0x555787303d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:54.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.030+0000 7feba96ba640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557872f95d0 0x555787303d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46244/0 (socket says 172.21.15.120:46244) 2024-09-19T09:11:54.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.030+0000 7feba9ebb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555787304370 0x555787301610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:54.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.030+0000 7feba96ba640 1 -- 172.21.15.120:0/3148650376 learned_addr learned my addr 172.21.15.120:0/3148650376 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:54.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.030+0000 7feba8eb9640 1 --2- 172.21.15.120:0/3148650376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557872fab40 0x555787301020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:54.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.030+0000 7feba96ba640 1 -- 172.21.15.120:0/3148650376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555787304370 msgr2=0x555787301610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:54.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.030+0000 7feba96ba640 1 --2- 172.21.15.120:0/3148650376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555787304370 0x555787301610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.031+0000 7feba96ba640 1 -- 172.21.15.120:0/3148650376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557872fab40 msgr2=0x555787301020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:54.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.031+0000 7feba96ba640 1 --2- 172.21.15.120:0/3148650376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557872fab40 0x555787301020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.031+0000 7feba96ba640 1 -- 172.21.15.120:0/3148650376 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555786fd03c0 con 0x5557872f95d0 2024-09-19T09:11:54.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.031+0000 7feba96ba640 1 --2- 172.21.15.120:0/3148650376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557872f95d0 0x555787303d70 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7feba0031720 tx=0x7feba00315d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b780f36607fb445 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:54.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.031+0000 7feb8cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:11:54.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.031+0000 7feb8cff9640 1 -- 172.21.15.120:0/3148650376 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba004d030 con 0x5557872f95d0 2024-09-19T09:11:54.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.031+0000 7feb8cff9640 1 -- 172.21.15.120:0/3148650376 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7feba0031a90 con 0x5557872f95d0 2024-09-19T09:11:54.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.031+0000 7febadcf7740 1 -- 172.21.15.120:0/3148650376 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feba0040070 con 0x5557872f95d0 2024-09-19T09:11:54.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.031+0000 7feb8cff9640 1 -- 172.21.15.120:0/3148650376 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feba0035070 con 0x5557872f95d0 2024-09-19T09:11:54.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.031+0000 7febadcf7740 10 client.18549 fetch_fsmap learned FSMap version 32 2024-09-19T09:11:54.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.031+0000 7febadcf7740 1 -- 172.21.15.120:0/3148650376 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555787312ab0 con 0x5557872f95d0 2024-09-19T09:11:54.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.031+0000 7feb8cff9640 1 -- 172.21.15.120:0/3148650376 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feba0040070 con 0x5557872f95d0 2024-09-19T09:11:54.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.032+0000 7feb8cff9640 1 -- 172.21.15.120:0/3148650376 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7feba0004a80 con 0x5557872f95d0 2024-09-19T09:11:54.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.032+0000 7febadcf7740 10 client.18549 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:11:54.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.032+0000 7febadcf7740 10 client.18549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:11:54.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.032+0000 7febadcf7740 20 client.18549 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:54.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.032+0000 7febadcf7740 1 --2- 172.21.15.120:0/3148650376 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555787313e00 0x5557873161f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:54.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.032+0000 7febadcf7740 4 client.18549 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:11:54.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.032+0000 7febadcf7740 1 -- 172.21.15.120:0/3148650376 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55578726c860 con 0x555787313e00 2024-09-19T09:11:54.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.032+0000 7feba8eb9640 1 --2- 172.21.15.120:0/3148650376 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555787313e00 0x5557873161f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:54.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.032+0000 7feb8cff9640 10 client.18549 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:54.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.032+0000 7feba8eb9640 1 --2- 172.21.15.120:0/3148650376 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555787313e00 0x5557873161f0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9d92074b462c12e server_cookie=cd5e7a86b605a186 in_seq=0 out_seq=0 2024-09-19T09:11:54.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.033+0000 7feb8cff9640 1 -- 172.21.15.120:0/3148650376 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55578726c860 con 0x555787313e00 2024-09-19T09:11:54.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.033+0000 7feb8cff9640 10 client.18549 handle_command_reply: tid=0 2024-09-19T09:11:54.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.033+0000 7febadcf7740 1 client.18549 shutdown 2024-09-19T09:11:54.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.033+0000 7febadcf7740 1 -- 172.21.15.120:0/3148650376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557872f95d0 msgr2=0x555787303d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:54.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.033+0000 7febadcf7740 1 --2- 172.21.15.120:0/3148650376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557872f95d0 0x555787303d70 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7feba0031720 tx=0x7feba00315d0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.033+0000 7febadcf7740 1 -- 172.21.15.120:0/3148650376 shutdown_connections 2024-09-19T09:11:54.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.033+0000 7febadcf7740 1 --2- 172.21.15.120:0/3148650376 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555787313e00 0x5557873161f0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.033+0000 7febadcf7740 1 --2- 172.21.15.120:0/3148650376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555787304370 0x555787301610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.033+0000 7febadcf7740 1 --2- 172.21.15.120:0/3148650376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557872fab40 0x555787301020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.033+0000 7febadcf7740 1 --2- 172.21.15.120:0/3148650376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557872f95d0 0x555787303d70 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.033+0000 7feb8cff9640 0 client.18549 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:54.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.033+0000 7febadcf7740 1 -- 172.21.15.120:0/3148650376 >> 172.21.15.120:0/3148650376 conn(0x5557872c2310 msgr2=0x5557872fb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:54.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.033+0000 7febadcf7740 1 -- 172.21.15.120:0/3148650376 shutdown_connections 2024-09-19T09:11:54.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.033+0000 7febadcf7740 1 -- 172.21.15.120:0/3148650376 wait complete. 2024-09-19T09:11:54.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.034+0000 7febadcf7740 20 client.18549 trim_cache size 0 max 16384 2024-09-19T09:11:54.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.050+0000 7febadcf7740 1 Processor -- start 2024-09-19T09:11:54.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.050+0000 7febadcf7740 1 -- start start 2024-09-19T09:11:54.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.050+0000 7febadcf7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557872f95d0 0x5557872f4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:54.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.050+0000 7febadcf7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557872fab40 0x5557872f5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:54.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.050+0000 7febadcf7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555787304370 0x5557872fc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:54.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.050+0000 7febadcf7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feba41ad590 con 0x5557872fab40 2024-09-19T09:11:54.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.050+0000 7febadcf7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feba41af0d0 con 0x555787304370 2024-09-19T09:11:54.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.050+0000 7febadcf7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feba41adc60 con 0x5557872f95d0 2024-09-19T09:11:54.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.050+0000 7feba8eb9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557872fab40 0x5557872f5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:54.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.050+0000 7feba8eb9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557872fab40 0x5557872f5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46254/0 (socket says 172.21.15.120:46254) 2024-09-19T09:11:54.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.050+0000 7feba8eb9640 1 -- 172.21.15.120:0/873594899 learned_addr learned my addr 172.21.15.120:0/873594899 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:54.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.050+0000 7feba96ba640 1 --2- 172.21.15.120:0/873594899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557872f95d0 0x5557872f4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:54.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.050+0000 7feba9ebb640 1 --2- 172.21.15.120:0/873594899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555787304370 0x5557872fc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:54.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.051+0000 7feba8eb9640 1 -- 172.21.15.120:0/873594899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557872f95d0 msgr2=0x5557872f4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:54.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.051+0000 7feba8eb9640 1 --2- 172.21.15.120:0/873594899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557872f95d0 0x5557872f4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.051+0000 7feba8eb9640 1 -- 172.21.15.120:0/873594899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555787304370 msgr2=0x5557872fc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:54.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.051+0000 7feba8eb9640 1 --2- 172.21.15.120:0/873594899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555787304370 0x5557872fc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.051+0000 7feba8eb9640 1 -- 172.21.15.120:0/873594899 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557873040a0 con 0x5557872fab40 2024-09-19T09:11:54.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.051+0000 7feba96ba640 1 --2- 172.21.15.120:0/873594899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557872f95d0 0x5557872f4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:54.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.051+0000 7feba8eb9640 1 --2- 172.21.15.120:0/873594899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557872fab40 0x5557872f5130 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7feb9401d000 tx=0x7feb9401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74d5944110ca49 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:54.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.051+0000 7feb8effd640 1 -- 172.21.15.120:0/873594899 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb94005590 con 0x5557872fab40 2024-09-19T09:11:54.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.051+0000 7feb8effd640 1 -- 172.21.15.120:0/873594899 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb94023070 con 0x5557872fab40 2024-09-19T09:11:54.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.051+0000 7feb8effd640 1 -- 172.21.15.120:0/873594899 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9402c6a0 con 0x5557872fab40 2024-09-19T09:11:54.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.052+0000 7febadcf7740 1 -- 172.21.15.120:0/873594899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557872fab40 msgr2=0x5557872f5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:54.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.052+0000 7febadcf7740 1 --2- 172.21.15.120:0/873594899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557872fab40 0x5557872f5130 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7feb9401d000 tx=0x7feb9401b000 comp rx=0 tx=0).stop 2024-09-19T09:11:54.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.052+0000 7febadcf7740 1 -- 172.21.15.120:0/873594899 shutdown_connections 2024-09-19T09:11:54.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.052+0000 7febadcf7740 1 --2- 172.21.15.120:0/873594899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555787304370 0x5557872fc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.052+0000 7febadcf7740 1 --2- 172.21.15.120:0/873594899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557872fab40 0x5557872f5130 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.052+0000 7febadcf7740 1 --2- 172.21.15.120:0/873594899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557872f95d0 0x5557872f4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.052+0000 7febadcf7740 1 -- 172.21.15.120:0/873594899 >> 172.21.15.120:0/873594899 conn(0x5557872c2310 msgr2=0x5557872f44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:54.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.052+0000 7febadcf7740 1 -- 172.21.15.120:0/873594899 shutdown_connections 2024-09-19T09:11:54.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.052+0000 7febadcf7740 1 -- 172.21.15.120:0/873594899 wait complete. 2024-09-19T09:11:54.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.052+0000 7febadcf7740 1 -- start start 2024-09-19T09:11:54.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.052+0000 7febadcf7740 1 Processor -- start 2024-09-19T09:11:54.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.053+0000 7febadcf7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557872f95d0 0x555787308330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:54.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.053+0000 7febadcf7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557872fab40 0x555787308870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:54.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.053+0000 7febadcf7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555787304370 0x555787308db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:54.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.053+0000 7febadcf7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feba41aea00 con 0x555787304370 2024-09-19T09:11:54.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.053+0000 7febadcf7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feba41ae330 con 0x5557872f95d0 2024-09-19T09:11:54.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.053+0000 7febadcf7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feba41ab550 con 0x5557872fab40 2024-09-19T09:11:54.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.053+0000 7feba9ebb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555787304370 0x555787308db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:54.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.053+0000 7feba96ba640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557872f95d0 0x555787308330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:54.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.053+0000 7feba8eb9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557872fab40 0x555787308870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:54.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.053+0000 7feba9ebb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555787304370 0x555787308db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46262/0 (socket says 172.21.15.120:46262) 2024-09-19T09:11:54.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.053+0000 7feba9ebb640 1 -- 172.21.15.120:0/3946205775 learned_addr learned my addr 172.21.15.120:0/3946205775 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:54.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.053+0000 7feba96ba640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557872f95d0 0x555787308330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59766/0 (socket says 172.21.15.120:59766) 2024-09-19T09:11:54.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.053+0000 7febadcf7740 10 client.0 _ll_register_callbacks cb 0x7feb94009820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:54.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.053+0000 7feba9ebb640 1 -- 172.21.15.120:0/3946205775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557872fab40 msgr2=0x555787308870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:54.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.053+0000 7feba9ebb640 1 --2- 172.21.15.120:0/3946205775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557872fab40 0x555787308870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.053+0000 7feba9ebb640 1 -- 172.21.15.120:0/3946205775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557872f95d0 msgr2=0x555787308330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:54.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.053+0000 7feba9ebb640 1 --2- 172.21.15.120:0/3946205775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557872f95d0 0x555787308330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.053+0000 7feba9ebb640 1 -- 172.21.15.120:0/3946205775 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55578737da10 con 0x555787304370 2024-09-19T09:11:54.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.053+0000 7feba8eb9640 1 --2- 172.21.15.120:0/3946205775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557872fab40 0x555787308870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:54.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.054+0000 7feba9ebb640 1 --2- 172.21.15.120:0/3946205775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555787304370 0x555787308db0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7feb9c0049d0 tx=0x7feb9c0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b2335b4bcc18d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:54.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.054+0000 7feb8cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:11:54.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.054+0000 7feb8cff9640 1 -- 172.21.15.120:0/3946205775 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c0b9070 con 0x555787304370 2024-09-19T09:11:54.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.054+0000 7feb8cff9640 1 -- 172.21.15.120:0/3946205775 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7feb9c0a4da0 con 0x555787304370 2024-09-19T09:11:54.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.054+0000 7febadcf7740 1 -- 172.21.15.120:0/3946205775 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb94023070 con 0x555787304370 2024-09-19T09:11:54.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.054+0000 7feb8cff9640 1 -- 172.21.15.120:0/3946205775 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feb9c09c3c0 con 0x555787304370 2024-09-19T09:11:54.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.054+0000 7febadcf7740 10 client.18555 fetch_fsmap learned FSMap version 32 2024-09-19T09:11:54.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.054+0000 7febadcf7740 1 -- 172.21.15.120:0/3946205775 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55578737ecb0 con 0x555787304370 2024-09-19T09:11:54.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.054+0000 7feb8cff9640 1 -- 172.21.15.120:0/3946205775 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb94023070 con 0x555787304370 2024-09-19T09:11:54.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.054+0000 7feb8cff9640 1 -- 172.21.15.120:0/3946205775 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7feb9c09b8a0 con 0x555787304370 2024-09-19T09:11:54.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.054+0000 7febadcf7740 10 client.18555 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:11:54.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.054+0000 7febadcf7740 10 client.18555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:11:54.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.054+0000 7febadcf7740 20 client.18555 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:54.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.055+0000 7febadcf7740 1 --2- 172.21.15.120:0/3946205775 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55578737f3d0 0x5557873817f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:54.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.055+0000 7febadcf7740 4 client.18555 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:11:54.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.055+0000 7febadcf7740 1 -- 172.21.15.120:0/3946205775 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555787266d70 con 0x55578737f3d0 2024-09-19T09:11:54.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.055+0000 7feba96ba640 1 --2- 172.21.15.120:0/3946205775 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55578737f3d0 0x5557873817f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:54.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.055+0000 7feba96ba640 1 --2- 172.21.15.120:0/3946205775 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55578737f3d0 0x5557873817f0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3fce0108dc48fcb server_cookie=1451a88f28dc3cd4 in_seq=0 out_seq=0 2024-09-19T09:11:54.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.055+0000 7feb8cff9640 10 client.18555 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:54.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.055+0000 7feb8cff9640 1 -- 172.21.15.120:0/3946205775 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555787266d70 con 0x55578737f3d0 2024-09-19T09:11:54.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.055+0000 7feb8cff9640 10 client.18555 handle_command_reply: tid=0 2024-09-19T09:11:54.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.055+0000 7febadcf7740 1 client.18555 shutdown 2024-09-19T09:11:54.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.056+0000 7febadcf7740 1 -- 172.21.15.120:0/3946205775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555787304370 msgr2=0x555787308db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:54.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.056+0000 7febadcf7740 1 --2- 172.21.15.120:0/3946205775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555787304370 0x555787308db0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7feb9c0049d0 tx=0x7feb9c0058d0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.056+0000 7febadcf7740 1 -- 172.21.15.120:0/3946205775 shutdown_connections 2024-09-19T09:11:54.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.056+0000 7febadcf7740 1 --2- 172.21.15.120:0/3946205775 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55578737f3d0 0x5557873817f0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.056+0000 7febadcf7740 1 --2- 172.21.15.120:0/3946205775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555787304370 0x555787308db0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.056+0000 7febadcf7740 1 --2- 172.21.15.120:0/3946205775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557872fab40 0x555787308870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.056+0000 7febadcf7740 1 --2- 172.21.15.120:0/3946205775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557872f95d0 0x555787308330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.056+0000 7febadcf7740 1 -- 172.21.15.120:0/3946205775 >> 172.21.15.120:0/3946205775 conn(0x5557872c2310 msgr2=0x5557872f4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:54.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.056+0000 7feb8cff9640 0 client.18555 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:54.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.056+0000 7febadcf7740 1 -- 172.21.15.120:0/3946205775 shutdown_connections 2024-09-19T09:11:54.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.056+0000 7febadcf7740 1 -- 172.21.15.120:0/3946205775 wait complete. 2024-09-19T09:11:54.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.056+0000 7febadcf7740 20 client.18555 trim_cache size 0 max 16384 2024-09-19T09:11:54.071 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:11:54.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.057+0000 7febab91e640 1 -- 172.21.15.120:0/2552830269 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7feb900a3340 msgr2=0x7feb900a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:54.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.057+0000 7febab91e640 1 --2- 172.21.15.120:0/2552830269 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7feb900a3340 0x7feb900a5800 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7feba406aac0 tx=0x7feb94009990 comp rx=0 tx=0).stop 2024-09-19T09:11:54.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.057+0000 7febab91e640 1 -- 172.21.15.120:0/2552830269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feba4102ef0 msgr2=0x7feba41a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:54.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.057+0000 7febab91e640 1 --2- 172.21.15.120:0/2552830269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feba4102ef0 0x7feba41a43a0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7feb9c00eb10 tx=0x7feb9c00c2d0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.057+0000 7febab91e640 1 -- 172.21.15.120:0/2552830269 shutdown_connections 2024-09-19T09:11:54.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.057+0000 7febab91e640 1 --2- 172.21.15.120:0/2552830269 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7feb900a3340 0x7feb900a5800 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.057+0000 7febab91e640 1 --2- 172.21.15.120:0/2552830269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feba4102ef0 0x7feba41a43a0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.057+0000 7febab91e640 1 --2- 172.21.15.120:0/2552830269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feba4102570 0x7feba4199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.057+0000 7febab91e640 1 --2- 172.21.15.120:0/2552830269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feba4100e20 0x7feba4199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:54.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.057+0000 7febab91e640 1 -- 172.21.15.120:0/2552830269 >> 172.21.15.120:0/2552830269 conn(0x7feba40fc7d0 msgr2=0x7feba40fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:54.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.057+0000 7febab91e640 1 -- 172.21.15.120:0/2552830269 shutdown_connections 2024-09-19T09:11:54.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:54.057+0000 7febab91e640 1 -- 172.21.15.120:0/2552830269 wait complete. 2024-09-19T09:11:54.073 INFO:tasks.workunit.client.0.smithi120.stdout: 1 845760 9.60 MB/sec execute 349 sec latency 85.030 ms 2024-09-19T09:11:55.064 INFO:tasks.workunit.client.0.smithi120.stdout: 1 848932 9.62 MB/sec execute 350 sec latency 13.082 ms 2024-09-19T09:11:55.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:54 smithi196 ceph-mon[35150]: pgmap v614: 97 pgs: 97 active+clean; 50 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.0 MiB/s wr, 401 op/s 2024-09-19T09:11:55.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:54 smithi153 ceph-mon[35071]: pgmap v614: 97 pgs: 97 active+clean; 50 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.0 MiB/s wr, 401 op/s 2024-09-19T09:11:55.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:54 smithi120 ceph-mon[31150]: pgmap v614: 97 pgs: 97 active+clean; 50 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.0 MiB/s wr, 401 op/s 2024-09-19T09:11:55.966 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:11:55.966 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:11:56.064 INFO:tasks.workunit.client.0.smithi120.stdout: 1 851544 9.63 MB/sec execute 351 sec latency 85.527 ms 2024-09-19T09:11:56.236 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:11:56.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.736+0000 7f2d6ffa6640 1 -- 172.21.15.120:0/2943750871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d68106a80 msgr2=0x7f2d68106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:56.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.736+0000 7f2d6ffa6640 1 --2- 172.21.15.120:0/2943750871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d68106a80 0x7f2d68106e60 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009930 tx=0x7f2d6402a420 comp rx=0 tx=0).stop 2024-09-19T09:11:56.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.737+0000 7f2d6ffa6640 1 -- 172.21.15.120:0/2943750871 shutdown_connections 2024-09-19T09:11:56.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.737+0000 7f2d6ffa6640 1 --2- 172.21.15.120:0/2943750871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d68101420 0x7f2d6810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.737+0000 7f2d6ffa6640 1 --2- 172.21.15.120:0/2943750871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d68100a80 0x7f2d68100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.737+0000 7f2d6ffa6640 1 --2- 172.21.15.120:0/2943750871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d68106a80 0x7f2d68106e60 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.737+0000 7f2d6ffa6640 1 -- 172.21.15.120:0/2943750871 >> 172.21.15.120:0/2943750871 conn(0x7f2d680fc7a0 msgr2=0x7f2d680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:56.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.737+0000 7f2d6ffa6640 1 -- 172.21.15.120:0/2943750871 shutdown_connections 2024-09-19T09:11:56.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.737+0000 7f2d6ffa6640 1 -- 172.21.15.120:0/2943750871 wait complete. 2024-09-19T09:11:56.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.737+0000 7f2d6ffa6640 1 Processor -- start 2024-09-19T09:11:56.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.737+0000 7f2d6ffa6640 1 -- start start 2024-09-19T09:11:56.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.737+0000 7f2d6ffa6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d68100a80 0x7f2d680fce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:56.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.737+0000 7f2d6ffa6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d68101420 0x7f2d680fd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:56.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.738+0000 7f2d6ffa6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d68106a80 0x7f2d680fd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:56.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.738+0000 7f2d6ffa6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2d68110e40 con 0x7f2d68101420 2024-09-19T09:11:56.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.738+0000 7f2d6ffa6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2d68110cc0 con 0x7f2d68100a80 2024-09-19T09:11:56.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.738+0000 7f2d6ffa6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2d68110fc0 con 0x7f2d68106a80 2024-09-19T09:11:56.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.738+0000 7f2d6d541640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d68101420 0x7f2d680fd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:56.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.738+0000 7f2d6dd42640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d68100a80 0x7f2d680fce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:56.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.738+0000 7f2d6d541640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d68101420 0x7f2d680fd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46284/0 (socket says 172.21.15.120:46284) 2024-09-19T09:11:56.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.738+0000 7f2d6d541640 1 -- 172.21.15.120:0/21012835 learned_addr learned my addr 172.21.15.120:0/21012835 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:56.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.738+0000 7f2d6e543640 1 --2- 172.21.15.120:0/21012835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d68106a80 0x7f2d680fd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:56.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.738+0000 7f2d6d541640 1 -- 172.21.15.120:0/21012835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d68106a80 msgr2=0x7f2d680fd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:56.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.738+0000 7f2d6d541640 1 --2- 172.21.15.120:0/21012835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d68106a80 0x7f2d680fd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.738+0000 7f2d6d541640 1 -- 172.21.15.120:0/21012835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d68100a80 msgr2=0x7f2d680fce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:56.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.738+0000 7f2d6d541640 1 --2- 172.21.15.120:0/21012835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d68100a80 0x7f2d680fce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.738+0000 7f2d6d541640 1 -- 172.21.15.120:0/21012835 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d681a6960 con 0x7f2d68101420 2024-09-19T09:11:56.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.738+0000 7f2d6d541640 1 --2- 172.21.15.120:0/21012835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d68101420 0x7f2d680fd390 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800d950 tx=0x7f2d5800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:56.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.738+0000 7f2d5effd640 1 -- 172.21.15.120:0/21012835 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d58017070 con 0x7f2d68101420 2024-09-19T09:11:56.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.738+0000 7f2d6ffa6640 1 -- 172.21.15.120:0/21012835 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d681a6c50 con 0x7f2d68101420 2024-09-19T09:11:56.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.738+0000 7f2d5effd640 1 -- 172.21.15.120:0/21012835 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d58009c30 con 0x7f2d68101420 2024-09-19T09:11:56.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.738+0000 7f2d5effd640 1 -- 172.21.15.120:0/21012835 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5800f6b0 con 0x7f2d68101420 2024-09-19T09:11:56.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.738+0000 7f2d6ffa6640 1 -- 172.21.15.120:0/21012835 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d681a7190 con 0x7f2d68101420 2024-09-19T09:11:56.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.741+0000 7f2d5effd640 1 -- 172.21.15.120:0/21012835 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f2d580109d0 con 0x7f2d68101420 2024-09-19T09:11:56.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.741+0000 7f2d7237f740 1 Processor -- start 2024-09-19T09:11:56.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.741+0000 7f2d7237f740 1 -- start start 2024-09-19T09:11:56.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.741+0000 7f2d7237f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b77d2d7b60 0x55b77d2d7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:56.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.741+0000 7f2d7237f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b77d2d8480 0x55b77d2e2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:56.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.741+0000 7f2d7237f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b77d2e2d40 0x55b77d2e5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:56.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.741+0000 7f2d7237f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b77d23d280 con 0x55b77d2e2d40 2024-09-19T09:11:56.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.741+0000 7f2d7237f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b77d1e4b10 con 0x55b77d2d8480 2024-09-19T09:11:56.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.741+0000 7f2d7237f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b77d239240 con 0x55b77d2d7b60 2024-09-19T09:11:56.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.741+0000 7f2d6dd42640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b77d2d8480 0x55b77d2e2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:56.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.741+0000 7f2d6d541640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b77d2e2d40 0x55b77d2e5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:56.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.741+0000 7f2d6dd42640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b77d2d8480 0x55b77d2e2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59794/0 (socket says 172.21.15.120:59794) 2024-09-19T09:11:56.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.741+0000 7f2d6dd42640 1 -- 172.21.15.120:0/178900985 learned_addr learned my addr 172.21.15.120:0/178900985 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:56.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.742+0000 7f2d6e543640 1 --2- 172.21.15.120:0/178900985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b77d2d7b60 0x55b77d2d7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:56.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.742+0000 7f2d5effd640 1 --2- 172.21.15.120:0/21012835 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2d540a3240 0x7f2d540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:56.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.742+0000 7f2d6dd42640 1 -- 172.21.15.120:0/178900985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b77d2d7b60 msgr2=0x55b77d2d7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:56.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.742+0000 7f2d6dd42640 1 --2- 172.21.15.120:0/178900985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b77d2d7b60 0x55b77d2d7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.742+0000 7f2d6dd42640 1 -- 172.21.15.120:0/178900985 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b77d2e2d40 msgr2=0x55b77d2e5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:56.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.742+0000 7f2d6dd42640 1 --2- 172.21.15.120:0/178900985 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b77d2e2d40 0x55b77d2e5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.742+0000 7f2d6dd42640 1 -- 172.21.15.120:0/178900985 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b77d1f11f0 con 0x55b77d2d8480 2024-09-19T09:11:56.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.742+0000 7f2d6e543640 1 --2- 172.21.15.120:0/21012835 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2d540a3240 0x7f2d540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:56.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.742+0000 7f2d5effd640 1 -- 172.21.15.120:0/21012835 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f2d580cf4b0 con 0x7f2d68101420 2024-09-19T09:11:56.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.742+0000 7f2d6dd42640 1 --2- 172.21.15.120:0/178900985 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b77d2d8480 0x55b77d2e2800 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2d6402a940 tx=0x7f2d64005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=200524b3e3451a87 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:56.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.742+0000 7f2d6e543640 1 --2- 172.21.15.120:0/21012835 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2d540a3240 0x7f2d540a5700 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2d60005c40 tx=0x7f2d60004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:56.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.742+0000 7f2d53fff640 1 -- 172.21.15.120:0/178900985 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d64040070 con 0x55b77d2d8480 2024-09-19T09:11:56.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.742+0000 7f2d53fff640 1 -- 172.21.15.120:0/178900985 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d640318f0 con 0x55b77d2d8480 2024-09-19T09:11:56.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.742+0000 7f2d53fff640 1 -- 172.21.15.120:0/178900985 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d64035040 con 0x55b77d2d8480 2024-09-19T09:11:56.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.743+0000 7f2d7237f740 1 -- 172.21.15.120:0/178900985 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b77d2d8480 msgr2=0x55b77d2e2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:56.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.743+0000 7f2d7237f740 1 --2- 172.21.15.120:0/178900985 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b77d2d8480 0x55b77d2e2800 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2d6402a940 tx=0x7f2d64005c00 comp rx=0 tx=0).stop 2024-09-19T09:11:56.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.743+0000 7f2d7237f740 1 -- 172.21.15.120:0/178900985 shutdown_connections 2024-09-19T09:11:56.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.743+0000 7f2d7237f740 1 --2- 172.21.15.120:0/178900985 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b77d2e2d40 0x55b77d2e5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.743+0000 7f2d7237f740 1 --2- 172.21.15.120:0/178900985 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b77d2d8480 0x55b77d2e2800 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.743+0000 7f2d7237f740 1 --2- 172.21.15.120:0/178900985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b77d2d7b60 0x55b77d2d7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.743+0000 7f2d7237f740 1 -- 172.21.15.120:0/178900985 >> 172.21.15.120:0/178900985 conn(0x55b77d2d40f0 msgr2=0x55b77d2d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:56.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.743+0000 7f2d7237f740 1 -- 172.21.15.120:0/178900985 shutdown_connections 2024-09-19T09:11:56.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.743+0000 7f2d7237f740 1 -- 172.21.15.120:0/178900985 wait complete. 2024-09-19T09:11:56.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.743+0000 7f2d7237f740 1 -- start start 2024-09-19T09:11:56.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.743+0000 7f2d7237f740 1 Processor -- start 2024-09-19T09:11:56.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.744+0000 7f2d7237f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b77d2d7b60 0x55b77d2e27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:56.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.744+0000 7f2d7237f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b77d2d8480 0x55b77d2dfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:56.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.744+0000 7f2d7237f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b77d2e2d40 0x55b77d2e0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:56.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.744+0000 7f2d7237f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b77d2e5910 con 0x55b77d2e2d40 2024-09-19T09:11:56.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.744+0000 7f2d7237f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b77d1f7f10 con 0x55b77d2d7b60 2024-09-19T09:11:56.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.744+0000 7f2d7237f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b77d2e5a90 con 0x55b77d2d8480 2024-09-19T09:11:56.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.744+0000 7f2d7237f740 10 client.0 _ll_register_callbacks cb 0x55b77d1c0a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:56.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.744+0000 7f2d6dd42640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b77d2d7b60 0x55b77d2e27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:56.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.744+0000 7f2d6e543640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b77d2e2d40 0x55b77d2e0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:56.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.744+0000 7f2d6dd42640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b77d2d7b60 0x55b77d2e27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59804/0 (socket says 172.21.15.120:59804) 2024-09-19T09:11:56.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.744+0000 7f2d6d541640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b77d2d8480 0x55b77d2dfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:56.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.744+0000 7f2d6e543640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b77d2e2d40 0x55b77d2e0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46306/0 (socket says 172.21.15.120:46306) 2024-09-19T09:11:56.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.744+0000 7f2d6dd42640 1 -- 172.21.15.120:0/4033622568 learned_addr learned my addr 172.21.15.120:0/4033622568 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:56.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.744+0000 7f2d6dd42640 1 -- 172.21.15.120:0/4033622568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b77d2d8480 msgr2=0x55b77d2dfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:56.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.744+0000 7f2d6dd42640 1 --2- 172.21.15.120:0/4033622568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b77d2d8480 0x55b77d2dfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.744+0000 7f2d6dd42640 1 -- 172.21.15.120:0/4033622568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b77d2e2d40 msgr2=0x55b77d2e0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:56.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.744+0000 7f2d6dd42640 1 --2- 172.21.15.120:0/4033622568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b77d2e2d40 0x55b77d2e0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.744+0000 7f2d6dd42640 1 -- 172.21.15.120:0/4033622568 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b77cfae6a0 con 0x55b77d2d7b60 2024-09-19T09:11:56.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.744+0000 7f2d6e543640 1 --2- 172.21.15.120:0/4033622568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b77d2e2d40 0x55b77d2e0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:56.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.744+0000 7f2d6d541640 1 --2- 172.21.15.120:0/4033622568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b77d2d8480 0x55b77d2dfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:56.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.745+0000 7f2d6dd42640 1 --2- 172.21.15.120:0/4033622568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b77d2d7b60 0x55b77d2e27b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2d64006010 tx=0x7f2d64005250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=419803f1bff70230 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:56.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.745+0000 7f2d517fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:11:56.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.745+0000 7f2d517fa640 1 -- 172.21.15.120:0/4033622568 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d64040030 con 0x55b77d2d7b60 2024-09-19T09:11:56.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.745+0000 7f2d7237f740 1 -- 172.21.15.120:0/4033622568 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d640318f0 con 0x55b77d2d7b60 2024-09-19T09:11:56.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.745+0000 7f2d517fa640 1 -- 172.21.15.120:0/4033622568 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f2d64004260 con 0x55b77d2d7b60 2024-09-19T09:11:56.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.745+0000 7f2d517fa640 1 -- 172.21.15.120:0/4033622568 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2d64035040 con 0x55b77d2d7b60 2024-09-19T09:11:56.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.745+0000 7f2d7237f740 10 client.27698 fetch_fsmap learned FSMap version 32 2024-09-19T09:11:56.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.745+0000 7f2d7237f740 1 -- 172.21.15.120:0/4033622568 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b77d2f1460 con 0x55b77d2d7b60 2024-09-19T09:11:56.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.745+0000 7f2d517fa640 1 -- 172.21.15.120:0/4033622568 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d640318f0 con 0x55b77d2d7b60 2024-09-19T09:11:56.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.746+0000 7f2d517fa640 1 -- 172.21.15.120:0/4033622568 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f2d640036f0 con 0x55b77d2d7b60 2024-09-19T09:11:56.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.746+0000 7f2d7237f740 10 client.27698 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:11:56.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.746+0000 7f2d7237f740 10 client.27698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:11:56.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.746+0000 7f2d7237f740 20 client.27698 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:56.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.746+0000 7f2d7237f740 1 --2- 172.21.15.120:0/4033622568 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b77d2f27b0 0x55b77d2f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:56.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.746+0000 7f2d7237f740 4 client.27698 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:11:56.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.746+0000 7f2d7237f740 1 -- 172.21.15.120:0/4033622568 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b77d24a7f0 con 0x55b77d2f27b0 2024-09-19T09:11:56.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.746+0000 7f2d6d541640 1 --2- 172.21.15.120:0/4033622568 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b77d2f27b0 0x55b77d2f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:56.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.746+0000 7f2d517fa640 10 client.27698 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:56.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.746+0000 7f2d6d541640 1 --2- 172.21.15.120:0/4033622568 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b77d2f27b0 0x55b77d2f4ba0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b3b748c0bd51b1e server_cookie=3467323cf48a75af in_seq=0 out_seq=0 2024-09-19T09:11:56.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.749+0000 7f2d517fa640 1 -- 172.21.15.120:0/4033622568 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b77d24a7f0 con 0x55b77d2f27b0 2024-09-19T09:11:56.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.749+0000 7f2d517fa640 10 client.27698 handle_command_reply: tid=0 2024-09-19T09:11:56.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.749+0000 7f2d7237f740 1 client.27698 shutdown 2024-09-19T09:11:56.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.749+0000 7f2d7237f740 1 -- 172.21.15.120:0/4033622568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b77d2d7b60 msgr2=0x55b77d2e27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:56.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.749+0000 7f2d7237f740 1 --2- 172.21.15.120:0/4033622568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b77d2d7b60 0x55b77d2e27b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2d64006010 tx=0x7f2d64005250 comp rx=0 tx=0).stop 2024-09-19T09:11:56.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.749+0000 7f2d7237f740 1 -- 172.21.15.120:0/4033622568 shutdown_connections 2024-09-19T09:11:56.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.749+0000 7f2d7237f740 1 --2- 172.21.15.120:0/4033622568 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b77d2f27b0 0x55b77d2f4ba0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.749+0000 7f2d7237f740 1 --2- 172.21.15.120:0/4033622568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b77d2e2d40 0x55b77d2e0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.749+0000 7f2d7237f740 1 --2- 172.21.15.120:0/4033622568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b77d2d8480 0x55b77d2dfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.749+0000 7f2d7237f740 1 --2- 172.21.15.120:0/4033622568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b77d2d7b60 0x55b77d2e27b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.749+0000 7f2d7237f740 1 -- 172.21.15.120:0/4033622568 >> 172.21.15.120:0/4033622568 conn(0x55b77d2d40f0 msgr2=0x55b77d2d9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:56.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.749+0000 7f2d517fa640 0 client.27698 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:56.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.749+0000 7f2d7237f740 1 -- 172.21.15.120:0/4033622568 shutdown_connections 2024-09-19T09:11:56.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.749+0000 7f2d7237f740 1 -- 172.21.15.120:0/4033622568 wait complete. 2024-09-19T09:11:56.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.749+0000 7f2d7237f740 20 client.27698 trim_cache size 0 max 16384 2024-09-19T09:11:56.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.767+0000 7f2d7237f740 1 Processor -- start 2024-09-19T09:11:56.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.767+0000 7f2d7237f740 1 -- start start 2024-09-19T09:11:56.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.767+0000 7f2d7237f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b77d2d7b60 0x55b77d2eebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:56.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.767+0000 7f2d7237f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b77d2d8480 0x55b77d2deae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:56.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.767+0000 7f2d7237f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b77d2e2d40 0x55b77d2fa200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:56.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.767+0000 7f2d7237f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2d681af7b0 con 0x55b77d2d7b60 2024-09-19T09:11:56.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.768+0000 7f2d7237f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2d681b12f0 con 0x55b77d2d8480 2024-09-19T09:11:56.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.768+0000 7f2d7237f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2d681afe80 con 0x55b77d2e2d40 2024-09-19T09:11:56.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.768+0000 7f2d6d541640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b77d2d8480 0x55b77d2deae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:56.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.768+0000 7f2d6e543640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b77d2e2d40 0x55b77d2fa200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:56.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.768+0000 7f2d6d541640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b77d2d8480 0x55b77d2deae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59816/0 (socket says 172.21.15.120:59816) 2024-09-19T09:11:56.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.768+0000 7f2d6e543640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b77d2e2d40 0x55b77d2fa200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:46794/0 (socket says 172.21.15.120:46794) 2024-09-19T09:11:56.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.768+0000 7f2d6d541640 1 -- 172.21.15.120:0/3855031182 learned_addr learned my addr 172.21.15.120:0/3855031182 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:56.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.768+0000 7f2d6dd42640 1 --2- 172.21.15.120:0/3855031182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b77d2d7b60 0x55b77d2eebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:56.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.768+0000 7f2d6e543640 1 -- 172.21.15.120:0/3855031182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b77d2d8480 msgr2=0x55b77d2deae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:56.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.768+0000 7f2d6e543640 1 --2- 172.21.15.120:0/3855031182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b77d2d8480 0x55b77d2deae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.768+0000 7f2d6e543640 1 -- 172.21.15.120:0/3855031182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b77d2d7b60 msgr2=0x55b77d2eebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:56.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.768+0000 7f2d6e543640 1 --2- 172.21.15.120:0/3855031182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b77d2d7b60 0x55b77d2eebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.768+0000 7f2d6e543640 1 -- 172.21.15.120:0/3855031182 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b77d2d3f20 con 0x55b77d2e2d40 2024-09-19T09:11:56.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.768+0000 7f2d6e543640 1 --2- 172.21.15.120:0/3855031182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b77d2e2d40 0x55b77d2fa200 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009fd0 tx=0x7f2d60009be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a38655ba82b5ec56 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:56.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.768+0000 7f2d537fe640 1 -- 172.21.15.120:0/3855031182 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6001a040 con 0x55b77d2e2d40 2024-09-19T09:11:56.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.768+0000 7f2d537fe640 1 -- 172.21.15.120:0/3855031182 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d6000fd40 con 0x55b77d2e2d40 2024-09-19T09:11:56.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.768+0000 7f2d537fe640 1 -- 172.21.15.120:0/3855031182 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60005470 con 0x55b77d2e2d40 2024-09-19T09:11:56.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.769+0000 7f2d7237f740 1 -- 172.21.15.120:0/3855031182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b77d2e2d40 msgr2=0x55b77d2fa200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:56.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.769+0000 7f2d7237f740 1 --2- 172.21.15.120:0/3855031182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b77d2e2d40 0x55b77d2fa200 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009fd0 tx=0x7f2d60009be0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.769+0000 7f2d7237f740 1 -- 172.21.15.120:0/3855031182 shutdown_connections 2024-09-19T09:11:56.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.769+0000 7f2d7237f740 1 --2- 172.21.15.120:0/3855031182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b77d2e2d40 0x55b77d2fa200 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.769+0000 7f2d7237f740 1 --2- 172.21.15.120:0/3855031182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b77d2d8480 0x55b77d2deae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.769+0000 7f2d7237f740 1 --2- 172.21.15.120:0/3855031182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b77d2d7b60 0x55b77d2eebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.769+0000 7f2d7237f740 1 -- 172.21.15.120:0/3855031182 >> 172.21.15.120:0/3855031182 conn(0x55b77d2d40f0 msgr2=0x55b77d2de3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:56.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.770+0000 7f2d7237f740 1 -- 172.21.15.120:0/3855031182 shutdown_connections 2024-09-19T09:11:56.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.770+0000 7f2d7237f740 1 -- 172.21.15.120:0/3855031182 wait complete. 2024-09-19T09:11:56.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.770+0000 7f2d7237f740 1 -- start start 2024-09-19T09:11:56.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.770+0000 7f2d7237f740 1 Processor -- start 2024-09-19T09:11:56.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.771+0000 7f2d7237f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b77d2d7b60 0x55b77d2eb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:56.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.771+0000 7f2d7237f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b77d2d8480 0x55b77d2dc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:56.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.771+0000 7f2d7237f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b77d2e2d40 0x55b77d2dcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:56.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.771+0000 7f2d7237f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2d681b0c20 con 0x55b77d2e2d40 2024-09-19T09:11:56.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.771+0000 7f2d7237f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2d681b0550 con 0x55b77d2d7b60 2024-09-19T09:11:56.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.771+0000 7f2d7237f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2d681ad770 con 0x55b77d2d8480 2024-09-19T09:11:56.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.771+0000 7f2d7237f740 10 client.0 _ll_register_callbacks cb 0x7f2d580d2e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:56.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.771+0000 7f2d6dd42640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b77d2d7b60 0x55b77d2eb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:56.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.771+0000 7f2d6dd42640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b77d2d7b60 0x55b77d2eb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59818/0 (socket says 172.21.15.120:59818) 2024-09-19T09:11:56.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.771+0000 7f2d6dd42640 1 -- 172.21.15.120:0/75916198 learned_addr learned my addr 172.21.15.120:0/75916198 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:56.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.772+0000 7f2d6e543640 1 --2- 172.21.15.120:0/75916198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b77d2e2d40 0x55b77d2dcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:56.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.772+0000 7f2d6dd42640 1 -- 172.21.15.120:0/75916198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b77d2d8480 msgr2=0x55b77d2dc870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:11:56.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.772+0000 7f2d6dd42640 1 --2- 172.21.15.120:0/75916198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b77d2d8480 0x55b77d2dc870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.772+0000 7f2d6dd42640 1 -- 172.21.15.120:0/75916198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b77d2e2d40 msgr2=0x55b77d2dcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:56.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.772+0000 7f2d6dd42640 1 --2- 172.21.15.120:0/75916198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b77d2e2d40 0x55b77d2dcdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.772+0000 7f2d6dd42640 1 -- 172.21.15.120:0/75916198 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b77d2d8a90 con 0x55b77d2d7b60 2024-09-19T09:11:56.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.772+0000 7f2d6e543640 1 --2- 172.21.15.120:0/75916198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b77d2e2d40 0x55b77d2dcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:11:56.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.772+0000 7f2d6dd42640 1 --2- 172.21.15.120:0/75916198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b77d2d7b60 0x55b77d2eb530 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2d64003ae0 tx=0x7f2d64009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca92a3c19832bab6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:56.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.772+0000 7f2d517fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:11:56.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.772+0000 7f2d517fa640 1 -- 172.21.15.120:0/75916198 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d64040070 con 0x55b77d2d7b60 2024-09-19T09:11:56.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.772+0000 7f2d517fa640 1 -- 172.21.15.120:0/75916198 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f2d640035f0 con 0x55b77d2d7b60 2024-09-19T09:11:56.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.773+0000 7f2d7237f740 1 -- 172.21.15.120:0/75916198 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d6000fd40 con 0x55b77d2d7b60 2024-09-19T09:11:56.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.773+0000 7f2d517fa640 1 -- 172.21.15.120:0/75916198 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2d64070050 con 0x55b77d2d7b60 2024-09-19T09:11:56.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.773+0000 7f2d7237f740 10 client.27704 fetch_fsmap learned FSMap version 32 2024-09-19T09:11:56.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.773+0000 7f2d7237f740 1 -- 172.21.15.120:0/75916198 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b77d35d1d0 con 0x55b77d2d7b60 2024-09-19T09:11:56.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.773+0000 7f2d517fa640 1 -- 172.21.15.120:0/75916198 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d6404d070 con 0x55b77d2d7b60 2024-09-19T09:11:56.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.773+0000 7f2d517fa640 1 -- 172.21.15.120:0/75916198 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f2d64004770 con 0x55b77d2d7b60 2024-09-19T09:11:56.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.773+0000 7f2d7237f740 10 client.27704 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:11:56.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.773+0000 7f2d7237f740 10 client.27704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:11:56.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.773+0000 7f2d7237f740 20 client.27704 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:56.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.773+0000 7f2d7237f740 1 --2- 172.21.15.120:0/75916198 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b77d35d780 0x55b77d35fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:56.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.773+0000 7f2d7237f740 4 client.27704 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:11:56.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.773+0000 7f2d7237f740 1 -- 172.21.15.120:0/75916198 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f2d6000fd40 con 0x55b77d35d780 2024-09-19T09:11:56.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.774+0000 7f2d6d541640 1 --2- 172.21.15.120:0/75916198 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b77d35d780 0x55b77d35fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:56.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.774+0000 7f2d517fa640 10 client.27704 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:56.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.774+0000 7f2d6d541640 1 --2- 172.21.15.120:0/75916198 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b77d35d780 0x55b77d35fb70 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52379c2db9000a3b server_cookie=9325d68565b8b592 in_seq=0 out_seq=0 2024-09-19T09:11:56.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.776+0000 7f2d517fa640 1 -- 172.21.15.120:0/75916198 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f2d6000fd40 con 0x55b77d35d780 2024-09-19T09:11:56.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.776+0000 7f2d517fa640 10 client.27704 handle_command_reply: tid=0 2024-09-19T09:11:56.812 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:11:56.812 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:11:56.812 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "1655c039-d9fa-43ef-89a9-4807041a4d75", 2024-09-19T09:11:56.812 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:11:56.812 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:11:56.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.776+0000 7f2d7237f740 1 client.27704 shutdown 2024-09-19T09:11:56.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.776+0000 7f2d7237f740 1 -- 172.21.15.120:0/75916198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b77d2d7b60 msgr2=0x55b77d2eb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.776+0000 7f2d7237f740 1 --2- 172.21.15.120:0/75916198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b77d2d7b60 0x55b77d2eb530 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2d64003ae0 tx=0x7f2d64009520 comp rx=0 tx=0).stop 2024-09-19T09:11:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.776+0000 7f2d7237f740 1 -- 172.21.15.120:0/75916198 shutdown_connections 2024-09-19T09:11:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.776+0000 7f2d7237f740 1 --2- 172.21.15.120:0/75916198 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b77d35d780 0x55b77d35fb70 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.776+0000 7f2d7237f740 1 --2- 172.21.15.120:0/75916198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b77d2e2d40 0x55b77d2dcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.776+0000 7f2d517fa640 0 client.27704 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.777+0000 7f2d7237f740 1 --2- 172.21.15.120:0/75916198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b77d2d8480 0x55b77d2dc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.777+0000 7f2d7237f740 1 --2- 172.21.15.120:0/75916198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b77d2d7b60 0x55b77d2eb530 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.777+0000 7f2d7237f740 1 -- 172.21.15.120:0/75916198 >> 172.21.15.120:0/75916198 conn(0x55b77d2d40f0 msgr2=0x55b77d2debf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.777+0000 7f2d7237f740 1 -- 172.21.15.120:0/75916198 shutdown_connections 2024-09-19T09:11:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.777+0000 7f2d7237f740 1 -- 172.21.15.120:0/75916198 wait complete. 2024-09-19T09:11:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.777+0000 7f2d7237f740 20 client.27704 trim_cache size 0 max 16384 2024-09-19T09:11:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.777+0000 7f2d6ffa6640 1 -- 172.21.15.120:0/21012835 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2d540a3240 msgr2=0x7f2d540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.777+0000 7f2d6ffa6640 1 --2- 172.21.15.120:0/21012835 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2d540a3240 0x7f2d540a5700 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2d60005c40 tx=0x7f2d60004ab0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.777+0000 7f2d6ffa6640 1 -- 172.21.15.120:0/21012835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d68101420 msgr2=0x7f2d680fd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.777+0000 7f2d6ffa6640 1 --2- 172.21.15.120:0/21012835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d68101420 0x7f2d680fd390 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800d950 tx=0x7f2d5800de10 comp rx=0 tx=0).stop 2024-09-19T09:11:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.777+0000 7f2d6ffa6640 1 -- 172.21.15.120:0/21012835 shutdown_connections 2024-09-19T09:11:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.777+0000 7f2d6ffa6640 1 --2- 172.21.15.120:0/21012835 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2d540a3240 0x7f2d540a5700 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.777+0000 7f2d6ffa6640 1 --2- 172.21.15.120:0/21012835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d68106a80 0x7f2d680fd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.777+0000 7f2d6ffa6640 1 --2- 172.21.15.120:0/21012835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d68101420 0x7f2d680fd390 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.777+0000 7f2d6ffa6640 1 --2- 172.21.15.120:0/21012835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d68100a80 0x7f2d680fce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.777+0000 7f2d6ffa6640 1 -- 172.21.15.120:0/21012835 >> 172.21.15.120:0/21012835 conn(0x7f2d680fc7a0 msgr2=0x7f2d6806aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.778+0000 7f2d6ffa6640 1 -- 172.21.15.120:0/21012835 shutdown_connections 2024-09-19T09:11:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:56.778+0000 7f2d6ffa6640 1 -- 172.21.15.120:0/21012835 wait complete. 2024-09-19T09:11:57.064 INFO:tasks.workunit.client.0.smithi120.stdout: 1 853682 9.64 MB/sec execute 352 sec latency 44.217 ms 2024-09-19T09:11:57.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:56 smithi120 ceph-mon[31150]: pgmap v615: 97 pgs: 97 active+clean; 45 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 346 op/s 2024-09-19T09:11:57.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:56 smithi196 ceph-mon[35150]: pgmap v615: 97 pgs: 97 active+clean; 45 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 346 op/s 2024-09-19T09:11:57.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:56 smithi153 ceph-mon[35071]: pgmap v615: 97 pgs: 97 active+clean; 45 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 346 op/s 2024-09-19T09:11:57.579 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:11:57.840 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:11:58.064 INFO:tasks.workunit.client.0.smithi120.stdout: 1 856405 9.65 MB/sec execute 353 sec latency 11.249 ms 2024-09-19T09:11:58.133 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:57 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:11:58.134 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:57 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:11:58.134 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:57 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:11:58.134 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:57 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:11:58.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:57 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:11:58.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:57 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:11:58.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:57 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:11:58.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:57 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:11:58.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:57 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:11:58.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:57 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:11:58.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:57 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:11:58.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:57 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:11:58.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.375+0000 7f48999b9640 1 -- 172.21.15.120:0/2854487278 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4894106f20 msgr2=0x7f489410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:58.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.375+0000 7f48999b9640 1 --2- 172.21.15.120:0/2854487278 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4894106f20 0x7f489410eaa0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f488c009ca0 tx=0x7f488c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.375+0000 7f48999b9640 1 -- 172.21.15.120:0/2854487278 shutdown_connections 2024-09-19T09:11:58.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.375+0000 7f48999b9640 1 --2- 172.21.15.120:0/2854487278 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4894106f20 0x7f489410eaa0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.375+0000 7f48999b9640 1 --2- 172.21.15.120:0/2854487278 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48941027a0 0x7f48941067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.375+0000 7f48999b9640 1 --2- 172.21.15.120:0/2854487278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4894101df0 0x7f48941021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.375+0000 7f48999b9640 1 -- 172.21.15.120:0/2854487278 >> 172.21.15.120:0/2854487278 conn(0x7f48940fd8f0 msgr2=0x7f48940ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:58.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.375+0000 7f48999b9640 1 -- 172.21.15.120:0/2854487278 shutdown_connections 2024-09-19T09:11:58.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.375+0000 7f48999b9640 1 -- 172.21.15.120:0/2854487278 wait complete. 2024-09-19T09:11:58.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.376+0000 7f48999b9640 1 Processor -- start 2024-09-19T09:11:58.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.376+0000 7f48999b9640 1 -- start start 2024-09-19T09:11:58.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.376+0000 7f48999b9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4894101df0 0x7f4894198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:58.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.376+0000 7f48999b9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48941027a0 0x7f4894198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:58.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.376+0000 7f48999b9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4894106f20 0x7f489419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:58.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.376+0000 7f48999b9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4894111f80 con 0x7f4894106f20 2024-09-19T09:11:58.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.376+0000 7f48999b9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4894111e00 con 0x7f48941027a0 2024-09-19T09:11:58.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.376+0000 7f48999b9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4894112100 con 0x7f4894101df0 2024-09-19T09:11:58.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.376+0000 7f48937fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4894106f20 0x7f489419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:58.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.376+0000 7f48937fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4894106f20 0x7f489419ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46368/0 (socket says 172.21.15.120:46368) 2024-09-19T09:11:58.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.376+0000 7f48937fe640 1 -- 172.21.15.120:0/130424309 learned_addr learned my addr 172.21.15.120:0/130424309 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:58.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.376+0000 7f488bfff640 1 --2- 172.21.15.120:0/130424309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48941027a0 0x7f4894198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:58.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.376+0000 7f4892ffd640 1 --2- 172.21.15.120:0/130424309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4894101df0 0x7f4894198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:58.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.376+0000 7f488bfff640 1 -- 172.21.15.120:0/130424309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4894101df0 msgr2=0x7f4894198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:58.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.376+0000 7f488bfff640 1 --2- 172.21.15.120:0/130424309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4894101df0 0x7f4894198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.376+0000 7f488bfff640 1 -- 172.21.15.120:0/130424309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4894106f20 msgr2=0x7f489419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:58.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.376+0000 7f488bfff640 1 --2- 172.21.15.120:0/130424309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4894106f20 0x7f489419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.377+0000 7f488bfff640 1 -- 172.21.15.120:0/130424309 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489419d3b0 con 0x7f48941027a0 2024-09-19T09:11:58.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.377+0000 7f48937fe640 1 --2- 172.21.15.120:0/130424309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4894106f20 0x7f489419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:58.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.377+0000 7f488bfff640 1 --2- 172.21.15.120:0/130424309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48941027a0 0x7f4894198940 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f487c00e9f0 tx=0x7f487c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:58.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.377+0000 7f48989b7640 1 -- 172.21.15.120:0/130424309 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487c00ccc0 con 0x7f48941027a0 2024-09-19T09:11:58.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.377+0000 7f48989b7640 1 -- 172.21.15.120:0/130424309 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f487c010910 con 0x7f48941027a0 2024-09-19T09:11:58.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.377+0000 7f48989b7640 1 -- 172.21.15.120:0/130424309 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487c018960 con 0x7f48941027a0 2024-09-19T09:11:58.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.377+0000 7f48999b9640 1 -- 172.21.15.120:0/130424309 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f489419d6a0 con 0x7f48941027a0 2024-09-19T09:11:58.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.377+0000 7f48999b9640 1 -- 172.21.15.120:0/130424309 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4894074420 con 0x7f48941027a0 2024-09-19T09:11:58.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.380+0000 7f48989b7640 1 -- 172.21.15.120:0/130424309 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f487c010ac0 con 0x7f48941027a0 2024-09-19T09:11:58.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.380+0000 7f489bd92740 1 Processor -- start 2024-09-19T09:11:58.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.380+0000 7f489bd92740 1 -- start start 2024-09-19T09:11:58.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.380+0000 7f489bd92740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f5669a4a0 0x562f5669a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:58.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.381+0000 7f489bd92740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5669adc0 0x562f566a5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:58.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.381+0000 7f489bd92740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f566a5680 0x562f566a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:58.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.381+0000 7f489bd92740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562f566002e0 con 0x562f5669adc0 2024-09-19T09:11:58.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.381+0000 7f489bd92740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562f565a7ce0 con 0x562f5669a4a0 2024-09-19T09:11:58.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.381+0000 7f489bd92740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562f565fc2a0 con 0x562f566a5680 2024-09-19T09:11:58.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.381+0000 7f48937fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f5669a4a0 0x562f5669a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:58.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.381+0000 7f4892ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5669adc0 0x562f566a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:58.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.381+0000 7f48937fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f5669a4a0 0x562f5669a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59842/0 (socket says 172.21.15.120:59842) 2024-09-19T09:11:58.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.381+0000 7f48937fe640 1 -- 172.21.15.120:0/3913521620 learned_addr learned my addr 172.21.15.120:0/3913521620 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:58.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.381+0000 7f488bfff640 1 --2- 172.21.15.120:0/3913521620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f566a5680 0x562f566a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:58.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.381+0000 7f48937fe640 1 -- 172.21.15.120:0/3913521620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f566a5680 msgr2=0x562f566a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:58.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.381+0000 7f48937fe640 1 --2- 172.21.15.120:0/3913521620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f566a5680 0x562f566a7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.381+0000 7f48937fe640 1 -- 172.21.15.120:0/3913521620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5669adc0 msgr2=0x562f566a5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:58.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.381+0000 7f48937fe640 1 --2- 172.21.15.120:0/3913521620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5669adc0 0x562f566a5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.381+0000 7f48937fe640 1 -- 172.21.15.120:0/3913521620 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f5658c440 con 0x562f5669a4a0 2024-09-19T09:11:58.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.381+0000 7f488bfff640 1 --2- 172.21.15.120:0/3913521620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f566a5680 0x562f566a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:11:58.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.381+0000 7f48989b7640 1 --2- 172.21.15.120:0/130424309 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f48800a32c0 0x7f48800a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:58.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.381+0000 7f48937fe640 1 --2- 172.21.15.120:0/3913521620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f5669a4a0 0x562f5669a880 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f488c031f80 tx=0x7f488c005f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=803e49707197d976 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:58.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.381+0000 7f4892ffd640 1 --2- 172.21.15.120:0/130424309 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f48800a32c0 0x7f48800a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:58.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.381+0000 7f4889ffb640 1 -- 172.21.15.120:0/3913521620 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c003b70 con 0x562f5669a4a0 2024-09-19T09:11:58.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.381+0000 7f4889ffb640 1 -- 172.21.15.120:0/3913521620 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f488c036040 con 0x562f5669a4a0 2024-09-19T09:11:58.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.381+0000 7f4889ffb640 1 -- 172.21.15.120:0/3913521620 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c004040 con 0x562f5669a4a0 2024-09-19T09:11:58.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.381+0000 7f48989b7640 1 -- 172.21.15.120:0/130424309 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f487c0cfbc0 con 0x7f48941027a0 2024-09-19T09:11:58.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.382+0000 7f489bd92740 1 -- 172.21.15.120:0/3913521620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f5669a4a0 msgr2=0x562f5669a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:58.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.382+0000 7f4892ffd640 1 --2- 172.21.15.120:0/130424309 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f48800a32c0 0x7f48800a5780 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x562f56697890 tx=0x7f4884005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:58.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.382+0000 7f489bd92740 1 --2- 172.21.15.120:0/3913521620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f5669a4a0 0x562f5669a880 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f488c031f80 tx=0x7f488c005f70 comp rx=0 tx=0).stop 2024-09-19T09:11:58.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.382+0000 7f489bd92740 1 -- 172.21.15.120:0/3913521620 shutdown_connections 2024-09-19T09:11:58.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.382+0000 7f489bd92740 1 --2- 172.21.15.120:0/3913521620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f566a5680 0x562f566a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.382+0000 7f489bd92740 1 --2- 172.21.15.120:0/3913521620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5669adc0 0x562f566a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.382+0000 7f489bd92740 1 --2- 172.21.15.120:0/3913521620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f5669a4a0 0x562f5669a880 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.382+0000 7f489bd92740 1 -- 172.21.15.120:0/3913521620 >> 172.21.15.120:0/3913521620 conn(0x562f56696a30 msgr2=0x562f56696e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:58.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.382+0000 7f489bd92740 1 -- 172.21.15.120:0/3913521620 shutdown_connections 2024-09-19T09:11:58.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.382+0000 7f489bd92740 1 -- 172.21.15.120:0/3913521620 wait complete. 2024-09-19T09:11:58.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.383+0000 7f489bd92740 1 -- start start 2024-09-19T09:11:58.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.383+0000 7f489bd92740 1 Processor -- start 2024-09-19T09:11:58.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.383+0000 7f489bd92740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f5669a4a0 0x562f566a4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:58.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.383+0000 7f489bd92740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f5669adc0 0x562f566a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:58.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.383+0000 7f489bd92740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f566a5680 0x562f566a2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:58.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.383+0000 7f489bd92740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562f566a80d0 con 0x562f566a5680 2024-09-19T09:11:58.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.383+0000 7f489bd92740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562f565bb050 con 0x562f5669a4a0 2024-09-19T09:11:58.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.383+0000 7f489bd92740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562f566a8250 con 0x562f5669adc0 2024-09-19T09:11:58.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.383+0000 7f489bd92740 10 client.0 _ll_register_callbacks cb 0x562f565fb160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:58.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.383+0000 7f4892ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f5669a4a0 0x562f566a4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:58.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.383+0000 7f48937fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f566a5680 0x562f566a2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:58.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.384+0000 7f48937fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f566a5680 0x562f566a2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46392/0 (socket says 172.21.15.120:46392) 2024-09-19T09:11:58.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.384+0000 7f4892ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f5669a4a0 0x562f566a4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59848/0 (socket says 172.21.15.120:59848) 2024-09-19T09:11:58.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.384+0000 7f48937fe640 1 -- 172.21.15.120:0/3818353060 learned_addr learned my addr 172.21.15.120:0/3818353060 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:58.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.384+0000 7f488bfff640 1 --2- 172.21.15.120:0/3818353060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f5669adc0 0x562f566a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:58.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.384+0000 7f48937fe640 1 -- 172.21.15.120:0/3818353060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f5669adc0 msgr2=0x562f566a21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:58.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.384+0000 7f48937fe640 1 --2- 172.21.15.120:0/3818353060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f5669adc0 0x562f566a21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.384+0000 7f48937fe640 1 -- 172.21.15.120:0/3818353060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f5669a4a0 msgr2=0x562f566a4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:58.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.384+0000 7f48937fe640 1 --2- 172.21.15.120:0/3818353060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f5669a4a0 0x562f566a4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.384+0000 7f48937fe640 1 -- 172.21.15.120:0/3818353060 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f563713c0 con 0x562f566a5680 2024-09-19T09:11:58.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.384+0000 7f4892ffd640 1 --2- 172.21.15.120:0/3818353060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f5669a4a0 0x562f566a4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:58.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.384+0000 7f48937fe640 1 --2- 172.21.15.120:0/3818353060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f566a5680 0x562f566a2710 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f488c009dc0 tx=0x7f488c003ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ff3b640f2f2145 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:58.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.384+0000 7f4872ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:11:58.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.384+0000 7f4872ffd640 1 -- 172.21.15.120:0/3818353060 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c031650 con 0x562f566a5680 2024-09-19T09:11:58.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.384+0000 7f4872ffd640 1 -- 172.21.15.120:0/3818353060 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f488c0317f0 con 0x562f566a5680 2024-09-19T09:11:58.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.384+0000 7f489bd92740 1 -- 172.21.15.120:0/3818353060 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f488c036040 con 0x562f566a5680 2024-09-19T09:11:58.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.384+0000 7f4872ffd640 1 -- 172.21.15.120:0/3818353060 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f488c03f220 con 0x562f566a5680 2024-09-19T09:11:58.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.384+0000 7f489bd92740 10 client.18585 fetch_fsmap learned FSMap version 32 2024-09-19T09:11:58.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.384+0000 7f489bd92740 1 -- 172.21.15.120:0/3818353060 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562f566a80d0 con 0x562f566a5680 2024-09-19T09:11:58.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.385+0000 7f4872ffd640 1 -- 172.21.15.120:0/3818353060 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f488c036040 con 0x562f566a5680 2024-09-19T09:11:58.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.385+0000 7f4872ffd640 1 -- 172.21.15.120:0/3818353060 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f488c047070 con 0x562f566a5680 2024-09-19T09:11:58.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.385+0000 7f489bd92740 10 client.18585 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:11:58.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.385+0000 7f489bd92740 10 client.18585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:11:58.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.385+0000 7f489bd92740 20 client.18585 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:58.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.385+0000 7f489bd92740 1 --2- 172.21.15.120:0/3818353060 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562f566bd3b0 0x562f566bf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:58.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.385+0000 7f489bd92740 4 client.18585 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:11:58.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.385+0000 7f489bd92740 1 -- 172.21.15.120:0/3818353060 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f5660d850 con 0x562f566bd3b0 2024-09-19T09:11:58.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.385+0000 7f4892ffd640 1 --2- 172.21.15.120:0/3818353060 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562f566bd3b0 0x562f566bf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:58.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.385+0000 7f4872ffd640 10 client.18585 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:58.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.385+0000 7f4892ffd640 1 --2- 172.21.15.120:0/3818353060 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562f566bd3b0 0x562f566bf7a0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5634ec120043da0 server_cookie=52556dd303e6130b in_seq=0 out_seq=0 2024-09-19T09:11:58.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.386+0000 7f4872ffd640 1 -- 172.21.15.120:0/3818353060 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562f5660d850 con 0x562f566bd3b0 2024-09-19T09:11:58.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.386+0000 7f4872ffd640 10 client.18585 handle_command_reply: tid=0 2024-09-19T09:11:58.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.386+0000 7f489bd92740 1 client.18585 shutdown 2024-09-19T09:11:58.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.386+0000 7f489bd92740 1 -- 172.21.15.120:0/3818353060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f566a5680 msgr2=0x562f566a2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:58.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.386+0000 7f489bd92740 1 --2- 172.21.15.120:0/3818353060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f566a5680 0x562f566a2710 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f488c009dc0 tx=0x7f488c003ca0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.387+0000 7f489bd92740 1 -- 172.21.15.120:0/3818353060 shutdown_connections 2024-09-19T09:11:58.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.387+0000 7f489bd92740 1 --2- 172.21.15.120:0/3818353060 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562f566bd3b0 0x562f566bf7a0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.387+0000 7f4872ffd640 0 client.18585 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:58.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.387+0000 7f489bd92740 1 --2- 172.21.15.120:0/3818353060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f566a5680 0x562f566a2710 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.387+0000 7f489bd92740 1 --2- 172.21.15.120:0/3818353060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f5669adc0 0x562f566a21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.387+0000 7f489bd92740 1 --2- 172.21.15.120:0/3818353060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f5669a4a0 0x562f566a4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.387+0000 7f489bd92740 1 -- 172.21.15.120:0/3818353060 >> 172.21.15.120:0/3818353060 conn(0x562f56696a30 msgr2=0x562f5669c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:58.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.387+0000 7f489bd92740 1 -- 172.21.15.120:0/3818353060 shutdown_connections 2024-09-19T09:11:58.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.387+0000 7f489bd92740 1 -- 172.21.15.120:0/3818353060 wait complete. 2024-09-19T09:11:58.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.387+0000 7f489bd92740 20 client.18585 trim_cache size 0 max 16384 2024-09-19T09:11:58.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.405+0000 7f489bd92740 1 Processor -- start 2024-09-19T09:11:58.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.405+0000 7f489bd92740 1 -- start start 2024-09-19T09:11:58.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.405+0000 7f489bd92740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f5669a4a0 0x562f566a4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:58.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.405+0000 7f489bd92740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f5669adc0 0x562f566c5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:58.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.405+0000 7f489bd92740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f566a5680 0x562f566c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:58.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.405+0000 7f489bd92740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f48941ab430 con 0x562f566a5680 2024-09-19T09:11:58.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.405+0000 7f489bd92740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f48941acf70 con 0x562f5669a4a0 2024-09-19T09:11:58.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.405+0000 7f489bd92740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f48941abb00 con 0x562f5669adc0 2024-09-19T09:11:58.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.406+0000 7f48937fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f566a5680 0x562f566c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:58.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.406+0000 7f4892ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f5669a4a0 0x562f566a4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:58.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.406+0000 7f488bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f5669adc0 0x562f566c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:58.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.406+0000 7f48937fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f566a5680 0x562f566c5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46408/0 (socket says 172.21.15.120:46408) 2024-09-19T09:11:58.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.406+0000 7f48937fe640 1 -- 172.21.15.120:0/189895911 learned_addr learned my addr 172.21.15.120:0/189895911 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:58.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.406+0000 7f48937fe640 1 -- 172.21.15.120:0/189895911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f5669adc0 msgr2=0x562f566c5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:58.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.406+0000 7f48937fe640 1 --2- 172.21.15.120:0/189895911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f5669adc0 0x562f566c5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.406+0000 7f48937fe640 1 -- 172.21.15.120:0/189895911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f5669a4a0 msgr2=0x562f566a4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:58.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.406+0000 7f48937fe640 1 --2- 172.21.15.120:0/189895911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f5669a4a0 0x562f566a4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.406+0000 7f48937fe640 1 -- 172.21.15.120:0/189895911 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f56697ae0 con 0x562f566a5680 2024-09-19T09:11:58.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.406+0000 7f4892ffd640 1 --2- 172.21.15.120:0/189895911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f5669a4a0 0x562f566a4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:11:58.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.406+0000 7f48937fe640 1 --2- 172.21.15.120:0/189895911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f566a5680 0x562f566c5540 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f488c000f80 tx=0x7f488c005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cefba9971c8aaf73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:58.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.407+0000 7f48897fa640 1 -- 172.21.15.120:0/189895911 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c03f3c0 con 0x562f566a5680 2024-09-19T09:11:58.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.407+0000 7f48897fa640 1 -- 172.21.15.120:0/189895911 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f488c036070 con 0x562f566a5680 2024-09-19T09:11:58.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.407+0000 7f48897fa640 1 -- 172.21.15.120:0/189895911 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c0316b0 con 0x562f566a5680 2024-09-19T09:11:58.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.408+0000 7f489bd92740 1 -- 172.21.15.120:0/189895911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f566a5680 msgr2=0x562f566c5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:58.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.408+0000 7f489bd92740 1 --2- 172.21.15.120:0/189895911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f566a5680 0x562f566c5540 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f488c000f80 tx=0x7f488c005f70 comp rx=0 tx=0).stop 2024-09-19T09:11:58.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.408+0000 7f489bd92740 1 -- 172.21.15.120:0/189895911 shutdown_connections 2024-09-19T09:11:58.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.408+0000 7f489bd92740 1 --2- 172.21.15.120:0/189895911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f566a5680 0x562f566c5540 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.408+0000 7f489bd92740 1 --2- 172.21.15.120:0/189895911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f5669adc0 0x562f566c5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.408+0000 7f489bd92740 1 --2- 172.21.15.120:0/189895911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f5669a4a0 0x562f566a4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.408+0000 7f489bd92740 1 -- 172.21.15.120:0/189895911 >> 172.21.15.120:0/189895911 conn(0x562f56696a30 msgr2=0x562f56695b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:58.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.408+0000 7f489bd92740 1 -- 172.21.15.120:0/189895911 shutdown_connections 2024-09-19T09:11:58.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.408+0000 7f489bd92740 1 -- 172.21.15.120:0/189895911 wait complete. 2024-09-19T09:11:58.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.409+0000 7f489bd92740 1 -- start start 2024-09-19T09:11:58.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.409+0000 7f489bd92740 1 Processor -- start 2024-09-19T09:11:58.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.409+0000 7f489bd92740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5669a4a0 0x562f566b64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:58.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.409+0000 7f489bd92740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f5669adc0 0x562f566b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:58.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.409+0000 7f489bd92740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f566a5680 0x562f566b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:58.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.409+0000 7f489bd92740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f48941ac8a0 con 0x562f5669a4a0 2024-09-19T09:11:58.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.409+0000 7f489bd92740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f48941ac1d0 con 0x562f566a5680 2024-09-19T09:11:58.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.409+0000 7f489bd92740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f48941a93f0 con 0x562f5669adc0 2024-09-19T09:11:58.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.409+0000 7f489bd92740 10 client.0 _ll_register_callbacks cb 0x7f4884009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:11:58.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.409+0000 7f4892ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5669a4a0 0x562f566b64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:58.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.409+0000 7f488bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f5669adc0 0x562f566b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:58.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.409+0000 7f488bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f5669adc0 0x562f566b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:46848/0 (socket says 172.21.15.120:46848) 2024-09-19T09:11:58.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.409+0000 7f488bfff640 1 -- 172.21.15.120:0/3228663738 learned_addr learned my addr 172.21.15.120:0/3228663738 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:58.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.409+0000 7f48937fe640 1 --2- 172.21.15.120:0/3228663738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f566a5680 0x562f566b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:58.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.410+0000 7f488bfff640 1 -- 172.21.15.120:0/3228663738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f566a5680 msgr2=0x562f566b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:58.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.410+0000 7f488bfff640 1 --2- 172.21.15.120:0/3228663738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f566a5680 0x562f566b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.410+0000 7f488bfff640 1 -- 172.21.15.120:0/3228663738 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5669a4a0 msgr2=0x562f566b64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:11:58.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.410+0000 7f488bfff640 1 --2- 172.21.15.120:0/3228663738 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5669a4a0 0x562f566b64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.410+0000 7f488bfff640 1 -- 172.21.15.120:0/3228663738 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f566976c0 con 0x562f5669adc0 2024-09-19T09:11:58.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.410+0000 7f488bfff640 1 --2- 172.21.15.120:0/3228663738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f5669adc0 0x562f566b38b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f487c00cf10 tx=0x7f487c09c130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d92ad59d31b4c747 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:58.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.410+0000 7f4872ffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:11:58.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.410+0000 7f4872ffd640 1 -- 172.21.15.120:0/3228663738 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487c0d4040 con 0x562f5669adc0 2024-09-19T09:11:58.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.410+0000 7f4872ffd640 1 -- 172.21.15.120:0/3228663738 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f487c0cd7d0 con 0x562f5669adc0 2024-09-19T09:11:58.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.410+0000 7f489bd92740 1 -- 172.21.15.120:0/3228663738 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f488c036070 con 0x562f5669adc0 2024-09-19T09:11:58.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.410+0000 7f4872ffd640 1 -- 172.21.15.120:0/3228663738 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f487c0b51c0 con 0x562f5669adc0 2024-09-19T09:11:58.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.410+0000 7f489bd92740 10 client.27580 fetch_fsmap learned FSMap version 32 2024-09-19T09:11:58.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.410+0000 7f489bd92740 1 -- 172.21.15.120:0/3228663738 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562f56727e40 con 0x562f5669adc0 2024-09-19T09:11:58.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.411+0000 7f4872ffd640 1 -- 172.21.15.120:0/3228663738 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f488c036070 con 0x562f5669adc0 2024-09-19T09:11:58.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.411+0000 7f4872ffd640 1 -- 172.21.15.120:0/3228663738 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f487c0a4a50 con 0x562f5669adc0 2024-09-19T09:11:58.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.411+0000 7f489bd92740 10 client.27580 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:11:58.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.411+0000 7f489bd92740 10 client.27580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:11:58.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.411+0000 7f489bd92740 20 client.27580 populate_metadata read hostname 'smithi120' 2024-09-19T09:11:58.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.411+0000 7f489bd92740 1 --2- 172.21.15.120:0/3228663738 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562f56728590 0x562f5672a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:58.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.411+0000 7f489bd92740 4 client.27580 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:11:58.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.411+0000 7f489bd92740 1 -- 172.21.15.120:0/3228663738 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562f56607d60 con 0x562f56728590 2024-09-19T09:11:58.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.411+0000 7f4892ffd640 1 --2- 172.21.15.120:0/3228663738 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562f56728590 0x562f5672a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:58.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.412+0000 7f4872ffd640 10 client.27580 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:58.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.412+0000 7f4892ffd640 1 --2- 172.21.15.120:0/3228663738 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562f56728590 0x562f5672a9b0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e316f354e32a4eff server_cookie=11ae6d3dde52a1b6 in_seq=0 out_seq=0 2024-09-19T09:11:58.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.412+0000 7f4872ffd640 1 -- 172.21.15.120:0/3228663738 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562f56607d60 con 0x562f56728590 2024-09-19T09:11:58.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.412+0000 7f4872ffd640 10 client.27580 handle_command_reply: tid=0 2024-09-19T09:11:58.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.412+0000 7f489bd92740 1 client.27580 shutdown 2024-09-19T09:11:58.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.413+0000 7f489bd92740 1 -- 172.21.15.120:0/3228663738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f5669adc0 msgr2=0x562f566b38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:58.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.413+0000 7f489bd92740 1 --2- 172.21.15.120:0/3228663738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f5669adc0 0x562f566b38b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f487c00cf10 tx=0x7f487c09c130 comp rx=0 tx=0).stop 2024-09-19T09:11:58.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.413+0000 7f489bd92740 1 -- 172.21.15.120:0/3228663738 shutdown_connections 2024-09-19T09:11:58.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.413+0000 7f489bd92740 1 --2- 172.21.15.120:0/3228663738 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562f56728590 0x562f5672a9b0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.413+0000 7f489bd92740 1 --2- 172.21.15.120:0/3228663738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f566a5680 0x562f566b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.413+0000 7f4872ffd640 0 client.27580 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:11:58.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.413+0000 7f489bd92740 1 --2- 172.21.15.120:0/3228663738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f5669adc0 0x562f566b38b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.413+0000 7f489bd92740 1 --2- 172.21.15.120:0/3228663738 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f5669a4a0 0x562f566b64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.413+0000 7f489bd92740 1 -- 172.21.15.120:0/3228663738 >> 172.21.15.120:0/3228663738 conn(0x562f56696a30 msgr2=0x562f566a1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:58.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.413+0000 7f489bd92740 1 -- 172.21.15.120:0/3228663738 shutdown_connections 2024-09-19T09:11:58.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.413+0000 7f489bd92740 1 -- 172.21.15.120:0/3228663738 wait complete. 2024-09-19T09:11:58.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.413+0000 7f489bd92740 20 client.27580 trim_cache size 0 max 16384 2024-09-19T09:11:58.417 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:11:58.417 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:11:58.418 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:11:58.418 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:11:58.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.413+0000 7f48999b9640 1 -- 172.21.15.120:0/130424309 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f48800a32c0 msgr2=0x7f48800a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:58.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.413+0000 7f48999b9640 1 --2- 172.21.15.120:0/130424309 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f48800a32c0 0x7f48800a5780 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x562f56697890 tx=0x7f4884005f90 comp rx=0 tx=0).stop 2024-09-19T09:11:58.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.413+0000 7f48999b9640 1 -- 172.21.15.120:0/130424309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48941027a0 msgr2=0x7f4894198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:58.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.413+0000 7f48999b9640 1 --2- 172.21.15.120:0/130424309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48941027a0 0x7f4894198940 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f487c00e9f0 tx=0x7f487c00eeb0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.413+0000 7f48999b9640 1 -- 172.21.15.120:0/130424309 shutdown_connections 2024-09-19T09:11:58.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.413+0000 7f48999b9640 1 --2- 172.21.15.120:0/130424309 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f48800a32c0 0x7f48800a5780 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.414+0000 7f48999b9640 1 --2- 172.21.15.120:0/130424309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4894106f20 0x7f489419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.414+0000 7f48999b9640 1 --2- 172.21.15.120:0/130424309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48941027a0 0x7f4894198940 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.414+0000 7f48999b9640 1 --2- 172.21.15.120:0/130424309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4894101df0 0x7f4894198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:58.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.414+0000 7f48999b9640 1 -- 172.21.15.120:0/130424309 >> 172.21.15.120:0/130424309 conn(0x7f48940fd8f0 msgr2=0x7f48940feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:58.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.414+0000 7f48999b9640 1 -- 172.21.15.120:0/130424309 shutdown_connections 2024-09-19T09:11:58.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:58.414+0000 7f48999b9640 1 -- 172.21.15.120:0/130424309 wait complete. 2024-09-19T09:11:59.064 INFO:tasks.workunit.client.0.smithi120.stdout: 1 859597 9.68 MB/sec execute 354 sec latency 8.942 ms 2024-09-19T09:11:59.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:58 smithi120 ceph-mon[31150]: pgmap v616: 97 pgs: 97 active+clean; 45 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 414 op/s 2024-09-19T09:11:59.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:11:58 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:11:59.142 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:11:59.142 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:11:59.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:58 smithi196 ceph-mon[35150]: pgmap v616: 97 pgs: 97 active+clean; 45 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 414 op/s 2024-09-19T09:11:59.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:11:58 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:11:59.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:58 smithi153 ceph-mon[35071]: pgmap v616: 97 pgs: 97 active+clean; 45 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 414 op/s 2024-09-19T09:11:59.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:11:58 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:11:59.420 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:11:59.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.931+0000 7f29b5c9c640 1 -- 172.21.15.120:0/3517929959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29b0102570 msgr2=0x7f29b01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:59.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.931+0000 7f29b5c9c640 1 --2- 172.21.15.120:0/3517929959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29b0102570 0x7f29b01029b0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f2998009900 tx=0x7f299802a480 comp rx=0 tx=0).stop 2024-09-19T09:11:59.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.931+0000 7f29b5c9c640 1 -- 172.21.15.120:0/3517929959 shutdown_connections 2024-09-19T09:11:59.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.931+0000 7f29b5c9c640 1 --2- 172.21.15.120:0/3517929959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29b0102ef0 0x7f29b010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:59.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.931+0000 7f29b5c9c640 1 --2- 172.21.15.120:0/3517929959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29b0102570 0x7f29b01029b0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:59.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.931+0000 7f29b5c9c640 1 --2- 172.21.15.120:0/3517929959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29b0100e20 0x7f29b0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:59.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.931+0000 7f29b5c9c640 1 -- 172.21.15.120:0/3517929959 >> 172.21.15.120:0/3517929959 conn(0x7f29b00fc7d0 msgr2=0x7f29b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:11:59.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.931+0000 7f29b5c9c640 1 -- 172.21.15.120:0/3517929959 shutdown_connections 2024-09-19T09:11:59.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.931+0000 7f29b5c9c640 1 -- 172.21.15.120:0/3517929959 wait complete. 2024-09-19T09:11:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.932+0000 7f29b5c9c640 1 Processor -- start 2024-09-19T09:11:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.932+0000 7f29b5c9c640 1 -- start start 2024-09-19T09:11:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.932+0000 7f29b5c9c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29b0100e20 0x7f29b0197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.932+0000 7f29b5c9c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29b0102570 0x7f29b01979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.932+0000 7f29b5c9c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29b0102ef0 0x7f29b019bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.932+0000 7f29b5c9c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f29b0110f80 con 0x7f29b0100e20 2024-09-19T09:11:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.932+0000 7f29b5c9c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f29b0110e00 con 0x7f29b0102ef0 2024-09-19T09:11:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.932+0000 7f29b5c9c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f29b0111100 con 0x7f29b0102570 2024-09-19T09:11:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.932+0000 7f29af7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29b0100e20 0x7f29b0197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.932+0000 7f29af7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29b0100e20 0x7f29b0197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46428/0 (socket says 172.21.15.120:46428) 2024-09-19T09:11:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.932+0000 7f29af7fe640 1 -- 172.21.15.120:0/1804998189 learned_addr learned my addr 172.21.15.120:0/1804998189 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:11:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.932+0000 7f29affff640 1 --2- 172.21.15.120:0/1804998189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29b0102ef0 0x7f29b019bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.932+0000 7f29aeffd640 1 --2- 172.21.15.120:0/1804998189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29b0102570 0x7f29b01979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.933+0000 7f29af7fe640 1 -- 172.21.15.120:0/1804998189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29b0102570 msgr2=0x7f29b01979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.933+0000 7f29af7fe640 1 --2- 172.21.15.120:0/1804998189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29b0102570 0x7f29b01979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.933+0000 7f29af7fe640 1 -- 172.21.15.120:0/1804998189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29b0102ef0 msgr2=0x7f29b019bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:11:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.933+0000 7f29af7fe640 1 --2- 172.21.15.120:0/1804998189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29b0102ef0 0x7f29b019bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:11:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.933+0000 7f29af7fe640 1 -- 172.21.15.120:0/1804998189 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b019c410 con 0x7f29b0100e20 2024-09-19T09:11:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.933+0000 7f29af7fe640 1 --2- 172.21.15.120:0/1804998189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29b0100e20 0x7f29b0197460 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f29a000e9f0 tx=0x7f29a000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.933+0000 7f29acff9640 1 -- 172.21.15.120:0/1804998189 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a000cd80 con 0x7f29b0100e20 2024-09-19T09:11:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.933+0000 7f29b5c9c640 1 -- 172.21.15.120:0/1804998189 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29b019c700 con 0x7f29b0100e20 2024-09-19T09:11:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.933+0000 7f29acff9640 1 -- 172.21.15.120:0/1804998189 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29a0005500 con 0x7f29b0100e20 2024-09-19T09:11:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.933+0000 7f29acff9640 1 -- 172.21.15.120:0/1804998189 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a0010690 con 0x7f29b0100e20 2024-09-19T09:11:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.933+0000 7f29b5c9c640 1 -- 172.21.15.120:0/1804998189 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29b0069f70 con 0x7f29b0100e20 2024-09-19T09:11:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.934+0000 7f29b5c9c640 1 -- 172.21.15.120:0/1804998189 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29b01021b0 con 0x7f29b0100e20 2024-09-19T09:11:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.935+0000 7f29acff9640 1 -- 172.21.15.120:0/1804998189 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f29a00046c0 con 0x7f29b0100e20 2024-09-19T09:11:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.937+0000 7f29acff9640 1 --2- 172.21.15.120:0/1804998189 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f299c0a54d0 0x7f299c0a7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:11:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.937+0000 7f29aeffd640 1 --2- 172.21.15.120:0/1804998189 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f299c0a54d0 0x7f299c0a7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:11:59.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.937+0000 7f29acff9640 1 -- 172.21.15.120:0/1804998189 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f29a00cfc90 con 0x7f29b0100e20 2024-09-19T09:11:59.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.937+0000 7f29aeffd640 1 --2- 172.21.15.120:0/1804998189 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f299c0a54d0 0x7f299c0a7990 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2998009230 tx=0x7f29980058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:11:59.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:11:59.941+0000 7f29acff9640 1 -- 172.21.15.120:0/1804998189 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f29a009c9b0 con 0x7f29b0100e20 2024-09-19T09:12:00.064 INFO:tasks.workunit.client.0.smithi120.stdout: 1 862863 9.70 MB/sec execute 355 sec latency 11.887 ms 2024-09-19T09:12:00.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:00.163+0000 7f29b5c9c640 1 -- 172.21.15.120:0/1804998189 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29b006a9f0 con 0x7f29b0100e20 2024-09-19T09:12:00.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:00.164+0000 7f29acff9640 1 -- 172.21.15.120:0/1804998189 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11988 (secure 0 0 0) 0x7f29a0096e20 con 0x7f29b0100e20 2024-09-19T09:12:00.165 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:12:00.165 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":32,"btime":"2024-09-19T09:11:12:904684+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:11:12.904670+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27518":{"gid":27518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3214776406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3214776406},{"type":"v1","addr":"172.21.15.153:6835","nonce":3214776406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:12:00.165 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 32 2024-09-19T09:12:00.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:00.167+0000 7f29b5c9c640 1 -- 172.21.15.120:0/1804998189 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f299c0a54d0 msgr2=0x7f299c0a7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:00.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:00.167+0000 7f29b5c9c640 1 --2- 172.21.15.120:0/1804998189 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f299c0a54d0 0x7f299c0a7990 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2998009230 tx=0x7f29980058d0 comp rx=0 tx=0).stop 2024-09-19T09:12:00.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:00.167+0000 7f29b5c9c640 1 -- 172.21.15.120:0/1804998189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29b0100e20 msgr2=0x7f29b0197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:00.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:00.167+0000 7f29b5c9c640 1 --2- 172.21.15.120:0/1804998189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29b0100e20 0x7f29b0197460 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f29a000e9f0 tx=0x7f29a000edf0 comp rx=0 tx=0).stop 2024-09-19T09:12:00.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:00.167+0000 7f29b5c9c640 1 -- 172.21.15.120:0/1804998189 shutdown_connections 2024-09-19T09:12:00.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:00.167+0000 7f29b5c9c640 1 --2- 172.21.15.120:0/1804998189 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f299c0a54d0 0x7f299c0a7990 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:00.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:00.167+0000 7f29b5c9c640 1 --2- 172.21.15.120:0/1804998189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29b0102ef0 0x7f29b019bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:00.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:00.167+0000 7f29b5c9c640 1 --2- 172.21.15.120:0/1804998189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29b0102570 0x7f29b01979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:00.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:00.167+0000 7f29b5c9c640 1 --2- 172.21.15.120:0/1804998189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29b0100e20 0x7f29b0197460 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:00.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:00.167+0000 7f29b5c9c640 1 -- 172.21.15.120:0/1804998189 >> 172.21.15.120:0/1804998189 conn(0x7f29b00fc7d0 msgr2=0x7f29b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:00.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:00.167+0000 7f29b5c9c640 1 -- 172.21.15.120:0/1804998189 shutdown_connections 2024-09-19T09:12:00.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:00.167+0000 7f29b5c9c640 1 -- 172.21.15.120:0/1804998189 wait complete. 2024-09-19T09:12:00.961 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:00 smithi120 ceph-mon[31150]: pgmap v617: 97 pgs: 97 active+clean; 45 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.2 MiB/s wr, 429 op/s 2024-09-19T09:12:00.961 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:00 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1804998189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:12:00.987 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:12:01.064 INFO:tasks.workunit.client.0.smithi120.stdout: 1 864835 9.69 MB/sec execute 356 sec latency 114.663 ms 2024-09-19T09:12:01.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:00 smithi196 ceph-mon[35150]: pgmap v617: 97 pgs: 97 active+clean; 45 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.2 MiB/s wr, 429 op/s 2024-09-19T09:12:01.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:00 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1804998189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:12:01.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:00 smithi153 ceph-mon[35071]: pgmap v617: 97 pgs: 97 active+clean; 45 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.2 MiB/s wr, 429 op/s 2024-09-19T09:12:01.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:00 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1804998189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:12:01.255 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:12:01.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.784+0000 7ff3d7165640 1 -- 172.21.15.120:0/2585349304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3d0100e20 msgr2=0x7ff3d0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:01.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.784+0000 7ff3d7165640 1 --2- 172.21.15.120:0/2585349304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3d0100e20 0x7ff3d0101200 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc009900 tx=0x7ff3cc02a480 comp rx=0 tx=0).stop 2024-09-19T09:12:01.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.784+0000 7ff3d7165640 1 -- 172.21.15.120:0/2585349304 shutdown_connections 2024-09-19T09:12:01.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.784+0000 7ff3d7165640 1 --2- 172.21.15.120:0/2585349304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3d0102ef0 0x7ff3d010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.784+0000 7ff3d7165640 1 --2- 172.21.15.120:0/2585349304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3d0102570 0x7ff3d01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.784+0000 7ff3d7165640 1 --2- 172.21.15.120:0/2585349304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3d0100e20 0x7ff3d0101200 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.784+0000 7ff3d7165640 1 -- 172.21.15.120:0/2585349304 >> 172.21.15.120:0/2585349304 conn(0x7ff3d00fc7d0 msgr2=0x7ff3d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:01.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.784+0000 7ff3d7165640 1 -- 172.21.15.120:0/2585349304 shutdown_connections 2024-09-19T09:12:01.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.784+0000 7ff3d7165640 1 -- 172.21.15.120:0/2585349304 wait complete. 2024-09-19T09:12:01.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.785+0000 7ff3d7165640 1 Processor -- start 2024-09-19T09:12:01.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.785+0000 7ff3d7165640 1 -- start start 2024-09-19T09:12:01.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.785+0000 7ff3d7165640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3d0100e20 0x7ff3d019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:01.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.785+0000 7ff3d7165640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3d0102570 0x7ff3d019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:01.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.785+0000 7ff3d7165640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3d0102ef0 0x7ff3d019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:01.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.785+0000 7ff3d7165640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d010fe80 con 0x7ff3d0102ef0 2024-09-19T09:12:01.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.785+0000 7ff3d7165640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d010fd00 con 0x7ff3d0100e20 2024-09-19T09:12:01.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.785+0000 7ff3d7165640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d0110000 con 0x7ff3d0102570 2024-09-19T09:12:01.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.785+0000 7ff3d5702640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3d0102ef0 0x7ff3d019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:01.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.785+0000 7ff3c7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3d0102570 0x7ff3d019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:01.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.785+0000 7ff3d4f01640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3d0100e20 0x7ff3d019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:01.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.785+0000 7ff3d4f01640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3d0100e20 0x7ff3d019a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:40684/0 (socket says 172.21.15.120:40684) 2024-09-19T09:12:01.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.785+0000 7ff3c7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3d0102570 0x7ff3d019ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:35186/0 (socket says 172.21.15.120:35186) 2024-09-19T09:12:01.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.785+0000 7ff3d5702640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3d0102ef0 0x7ff3d019efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46682/0 (socket says 172.21.15.120:46682) 2024-09-19T09:12:01.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.785+0000 7ff3d4f01640 1 -- 172.21.15.120:0/600375630 learned_addr learned my addr 172.21.15.120:0/600375630 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:01.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.785+0000 7ff3d5702640 1 -- 172.21.15.120:0/600375630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3d0102570 msgr2=0x7ff3d019ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:01.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.785+0000 7ff3d5702640 1 --2- 172.21.15.120:0/600375630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3d0102570 0x7ff3d019ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.785+0000 7ff3d5702640 1 -- 172.21.15.120:0/600375630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3d0100e20 msgr2=0x7ff3d019a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:01.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.785+0000 7ff3d5702640 1 --2- 172.21.15.120:0/600375630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3d0100e20 0x7ff3d019a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.786+0000 7ff3d5702640 1 -- 172.21.15.120:0/600375630 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d019f6b0 con 0x7ff3d0102ef0 2024-09-19T09:12:01.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.786+0000 7ff3d4f01640 1 --2- 172.21.15.120:0/600375630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3d0100e20 0x7ff3d019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:01.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.786+0000 7ff3d5702640 1 --2- 172.21.15.120:0/600375630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3d0102ef0 0x7ff3d019efd0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800da10 tx=0x7ff3c800ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:01.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.786+0000 7ff3c5ffb640 1 -- 172.21.15.120:0/600375630 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c800bd70 con 0x7ff3d0102ef0 2024-09-19T09:12:01.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.786+0000 7ff3d7165640 1 -- 172.21.15.120:0/600375630 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d019f9a0 con 0x7ff3d0102ef0 2024-09-19T09:12:01.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.786+0000 7ff3c5ffb640 1 -- 172.21.15.120:0/600375630 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3c8003e50 con 0x7ff3d0102ef0 2024-09-19T09:12:01.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.786+0000 7ff3c5ffb640 1 -- 172.21.15.120:0/600375630 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c80106a0 con 0x7ff3d0102ef0 2024-09-19T09:12:01.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.786+0000 7ff3d7165640 1 -- 172.21.15.120:0/600375630 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d01a7280 con 0x7ff3d0102ef0 2024-09-19T09:12:01.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.788+0000 7ff3c5ffb640 1 -- 172.21.15.120:0/600375630 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7ff3c8003970 con 0x7ff3d0102ef0 2024-09-19T09:12:01.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.789+0000 7ff3d953e740 1 Processor -- start 2024-09-19T09:12:01.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.789+0000 7ff3d953e740 1 -- start start 2024-09-19T09:12:01.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.789+0000 7ff3d953e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ae2112b40 0x563ae2111090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:01.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.789+0000 7ff3d953e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ae21115d0 0x563ae2111a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:01.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.789+0000 7ff3c5ffb640 1 --2- 172.21.15.120:0/600375630 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff3b80a3230 0x7ff3b80a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:01.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.789+0000 7ff3d953e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563ae211c370 0x563ae211e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:01.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.789+0000 7ff3d953e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563ae20772f0 con 0x563ae211c370 2024-09-19T09:12:01.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.789+0000 7ff3d953e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563ae201ecf0 con 0x563ae2112b40 2024-09-19T09:12:01.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.789+0000 7ff3d953e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563ae20732b0 con 0x563ae21115d0 2024-09-19T09:12:01.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.789+0000 7ff3c7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ae2112b40 0x563ae2111090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:01.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.789+0000 7ff3c7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ae2112b40 0x563ae2111090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:40698/0 (socket says 172.21.15.120:40698) 2024-09-19T09:12:01.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.789+0000 7ff3c7fff640 1 -- 172.21.15.120:0/2987953542 learned_addr learned my addr 172.21.15.120:0/2987953542 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:01.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.789+0000 7ff3c7fff640 1 --2- 172.21.15.120:0/600375630 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff3b80a3230 0x7ff3b80a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:01.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.789+0000 7ff3d5702640 1 --2- 172.21.15.120:0/2987953542 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563ae211c370 0x563ae211e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:01.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.789+0000 7ff3d4f01640 1 --2- 172.21.15.120:0/2987953542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ae21115d0 0x563ae2111a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:01.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.789+0000 7ff3c7fff640 1 -- 172.21.15.120:0/2987953542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ae21115d0 msgr2=0x563ae2111a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:01.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.789+0000 7ff3c7fff640 1 --2- 172.21.15.120:0/2987953542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ae21115d0 0x563ae2111a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.789+0000 7ff3c7fff640 1 -- 172.21.15.120:0/2987953542 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563ae211c370 msgr2=0x563ae211e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:01.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.789+0000 7ff3c7fff640 1 --2- 172.21.15.120:0/2987953542 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563ae211c370 0x563ae211e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.789+0000 7ff3c7fff640 1 -- 172.21.15.120:0/2987953542 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ae20033a0 con 0x563ae2112b40 2024-09-19T09:12:01.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.789+0000 7ff3c5ffb640 1 -- 172.21.15.120:0/600375630 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ff3c80cfcf0 con 0x7ff3d0102ef0 2024-09-19T09:12:01.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.790+0000 7ff3c7fff640 1 --2- 172.21.15.120:0/2987953542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ae2112b40 0x563ae2111090 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00058c0 tx=0x7ff3c0011f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c608e033c9546af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:01.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.790+0000 7ff3c7fff640 1 --2- 172.21.15.120:0/600375630 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff3b80a3230 0x7ff3b80a56f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0011fd0 tx=0x7ff3c0018000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:01.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.790+0000 7ff3b6ffd640 1 -- 172.21.15.120:0/2987953542 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0011050 con 0x563ae2112b40 2024-09-19T09:12:01.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.790+0000 7ff3b6ffd640 1 -- 172.21.15.120:0/2987953542 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3c0009a20 con 0x563ae2112b40 2024-09-19T09:12:01.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.790+0000 7ff3b6ffd640 1 -- 172.21.15.120:0/2987953542 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c00035f0 con 0x563ae2112b40 2024-09-19T09:12:01.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.791+0000 7ff3d953e740 1 -- 172.21.15.120:0/2987953542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ae2112b40 msgr2=0x563ae2111090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:01.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.791+0000 7ff3d953e740 1 --2- 172.21.15.120:0/2987953542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ae2112b40 0x563ae2111090 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00058c0 tx=0x7ff3c0011f60 comp rx=0 tx=0).stop 2024-09-19T09:12:01.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.791+0000 7ff3d953e740 1 -- 172.21.15.120:0/2987953542 shutdown_connections 2024-09-19T09:12:01.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.791+0000 7ff3d953e740 1 --2- 172.21.15.120:0/2987953542 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563ae211c370 0x563ae211e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.791+0000 7ff3d953e740 1 --2- 172.21.15.120:0/2987953542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ae21115d0 0x563ae2111a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.791+0000 7ff3d953e740 1 --2- 172.21.15.120:0/2987953542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ae2112b40 0x563ae2111090 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.791+0000 7ff3d953e740 1 -- 172.21.15.120:0/2987953542 >> 172.21.15.120:0/2987953542 conn(0x563ae20da310 msgr2=0x563ae210dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:01.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.791+0000 7ff3d953e740 1 -- 172.21.15.120:0/2987953542 shutdown_connections 2024-09-19T09:12:01.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.791+0000 7ff3d953e740 1 -- 172.21.15.120:0/2987953542 wait complete. 2024-09-19T09:12:01.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.791+0000 7ff3d953e740 1 -- start start 2024-09-19T09:12:01.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.791+0000 7ff3d953e740 1 Processor -- start 2024-09-19T09:12:01.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.792+0000 7ff3d953e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ae21115d0 0x563ae211bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:01.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.792+0000 7ff3d953e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563ae2112b40 0x563ae2119080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:01.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.792+0000 7ff3d953e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ae211c370 0x563ae2119810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:01.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.792+0000 7ff3d953e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563ae211f010 con 0x563ae2112b40 2024-09-19T09:12:01.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.792+0000 7ff3d953e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563ae2032060 con 0x563ae21115d0 2024-09-19T09:12:01.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.792+0000 7ff3d953e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563ae211f190 con 0x563ae211c370 2024-09-19T09:12:01.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.792+0000 7ff3d953e740 10 client.0 _ll_register_callbacks cb 0x563ae1d858c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:01.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.792+0000 7ff3c7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563ae2112b40 0x563ae2119080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:01.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.792+0000 7ff3d4f01640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ae21115d0 0x563ae211bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:01.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.792+0000 7ff3c7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563ae2112b40 0x563ae2119080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46702/0 (socket says 172.21.15.120:46702) 2024-09-19T09:12:01.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.792+0000 7ff3c7fff640 1 -- 172.21.15.120:0/2194108659 learned_addr learned my addr 172.21.15.120:0/2194108659 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:01.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.792+0000 7ff3d5702640 1 --2- 172.21.15.120:0/2194108659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ae211c370 0x563ae2119810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:01.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.792+0000 7ff3c7fff640 1 -- 172.21.15.120:0/2194108659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ae211c370 msgr2=0x563ae2119810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:01.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.792+0000 7ff3c7fff640 1 --2- 172.21.15.120:0/2194108659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ae211c370 0x563ae2119810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.792+0000 7ff3c7fff640 1 -- 172.21.15.120:0/2194108659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ae21115d0 msgr2=0x563ae211bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:01.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.792+0000 7ff3c7fff640 1 --2- 172.21.15.120:0/2194108659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ae21115d0 0x563ae211bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.792+0000 7ff3c7fff640 1 -- 172.21.15.120:0/2194108659 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ae1de83c0 con 0x563ae2112b40 2024-09-19T09:12:01.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.793+0000 7ff3c7fff640 1 --2- 172.21.15.120:0/2194108659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563ae2112b40 0x563ae2119080 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0011430 tx=0x7ff3c000e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22dda8bf819946d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:01.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.793+0000 7ff3affff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:01.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.793+0000 7ff3affff640 1 -- 172.21.15.120:0/2194108659 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c000ed70 con 0x563ae2112b40 2024-09-19T09:12:01.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.793+0000 7ff3affff640 1 -- 172.21.15.120:0/2194108659 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ff3c0005110 con 0x563ae2112b40 2024-09-19T09:12:01.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.793+0000 7ff3d953e740 1 -- 172.21.15.120:0/2194108659 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3c0009a20 con 0x563ae2112b40 2024-09-19T09:12:01.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.793+0000 7ff3affff640 1 -- 172.21.15.120:0/2194108659 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff3c0005310 con 0x563ae2112b40 2024-09-19T09:12:01.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.793+0000 7ff3d953e740 10 client.18609 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:01.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.793+0000 7ff3d953e740 1 -- 172.21.15.120:0/2194108659 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563ae212ab30 con 0x563ae2112b40 2024-09-19T09:12:01.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.793+0000 7ff3affff640 1 -- 172.21.15.120:0/2194108659 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3c0009a20 con 0x563ae2112b40 2024-09-19T09:12:01.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.793+0000 7ff3affff640 1 -- 172.21.15.120:0/2194108659 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff3c0030420 con 0x563ae2112b40 2024-09-19T09:12:01.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.793+0000 7ff3d953e740 10 client.18609 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:01.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.793+0000 7ff3d953e740 10 client.18609 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:12:01.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.793+0000 7ff3d953e740 20 client.18609 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:01.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.793+0000 7ff3d953e740 1 --2- 172.21.15.120:0/2194108659 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x563ae212bff0 0x563ae212e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:01.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.793+0000 7ff3d953e740 4 client.18609 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:12:01.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.793+0000 7ff3d953e740 1 -- 172.21.15.120:0/2194108659 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ae2084860 con 0x563ae212bff0 2024-09-19T09:12:01.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.794+0000 7ff3d4f01640 1 --2- 172.21.15.120:0/2194108659 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x563ae212bff0 0x563ae212e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:01.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.794+0000 7ff3d4f01640 1 --2- 172.21.15.120:0/2194108659 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x563ae212bff0 0x563ae212e3e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a3d252fffb47b33b server_cookie=68cfe245ace2ac58 in_seq=0 out_seq=0 2024-09-19T09:12:01.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.794+0000 7ff3affff640 10 client.18609 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:12:01.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.795+0000 7ff3affff640 1 -- 172.21.15.120:0/2194108659 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563ae2084860 con 0x563ae212bff0 2024-09-19T09:12:01.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.795+0000 7ff3affff640 10 client.18609 handle_command_reply: tid=0 2024-09-19T09:12:01.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.795+0000 7ff3d953e740 1 client.18609 shutdown 2024-09-19T09:12:01.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.795+0000 7ff3d953e740 1 -- 172.21.15.120:0/2194108659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563ae2112b40 msgr2=0x563ae2119080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:01.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.795+0000 7ff3d953e740 1 --2- 172.21.15.120:0/2194108659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563ae2112b40 0x563ae2119080 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0011430 tx=0x7ff3c000e990 comp rx=0 tx=0).stop 2024-09-19T09:12:01.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.795+0000 7ff3d953e740 1 -- 172.21.15.120:0/2194108659 shutdown_connections 2024-09-19T09:12:01.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.795+0000 7ff3d953e740 1 --2- 172.21.15.120:0/2194108659 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x563ae212bff0 0x563ae212e3e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.795+0000 7ff3d953e740 1 --2- 172.21.15.120:0/2194108659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ae211c370 0x563ae2119810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.795+0000 7ff3d953e740 1 --2- 172.21.15.120:0/2194108659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563ae2112b40 0x563ae2119080 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.795+0000 7ff3affff640 0 client.18609 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:12:01.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.795+0000 7ff3d953e740 1 --2- 172.21.15.120:0/2194108659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ae21115d0 0x563ae211bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.795+0000 7ff3d953e740 1 -- 172.21.15.120:0/2194108659 >> 172.21.15.120:0/2194108659 conn(0x563ae20da310 msgr2=0x563ae2113960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:01.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.795+0000 7ff3d953e740 1 -- 172.21.15.120:0/2194108659 shutdown_connections 2024-09-19T09:12:01.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.795+0000 7ff3d953e740 1 -- 172.21.15.120:0/2194108659 wait complete. 2024-09-19T09:12:01.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.795+0000 7ff3d953e740 20 client.18609 trim_cache size 0 max 16384 2024-09-19T09:12:01.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.810+0000 7ff3d953e740 1 Processor -- start 2024-09-19T09:12:01.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.810+0000 7ff3d953e740 1 -- start start 2024-09-19T09:12:01.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.810+0000 7ff3d953e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ae21115d0 0x563ae2118060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:01.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.810+0000 7ff3d953e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563ae2112b40 0x563ae2133b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:01.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.810+0000 7ff3d953e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ae211c370 0x563ae21340b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:01.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.810+0000 7ff3d953e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d01af7a0 con 0x563ae2112b40 2024-09-19T09:12:01.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.810+0000 7ff3d953e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d01b12e0 con 0x563ae211c370 2024-09-19T09:12:01.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.810+0000 7ff3d953e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d01afe70 con 0x563ae21115d0 2024-09-19T09:12:01.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.810+0000 7ff3c7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563ae2112b40 0x563ae2133b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:01.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.810+0000 7ff3c7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563ae2112b40 0x563ae2133b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46708/0 (socket says 172.21.15.120:46708) 2024-09-19T09:12:01.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.810+0000 7ff3c7fff640 1 -- 172.21.15.120:0/2661263812 learned_addr learned my addr 172.21.15.120:0/2661263812 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:01.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.810+0000 7ff3d5702640 1 --2- 172.21.15.120:0/2661263812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ae211c370 0x563ae21340b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:01.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.810+0000 7ff3d4f01640 1 --2- 172.21.15.120:0/2661263812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ae21115d0 0x563ae2118060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:01.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.810+0000 7ff3c7fff640 1 -- 172.21.15.120:0/2661263812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ae21115d0 msgr2=0x563ae2118060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:01.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.810+0000 7ff3c7fff640 1 --2- 172.21.15.120:0/2661263812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ae21115d0 0x563ae2118060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.810+0000 7ff3c7fff640 1 -- 172.21.15.120:0/2661263812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ae211c370 msgr2=0x563ae21340b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:01.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.810+0000 7ff3c7fff640 1 --2- 172.21.15.120:0/2661263812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ae211c370 0x563ae21340b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.810+0000 7ff3c7fff640 1 -- 172.21.15.120:0/2661263812 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ae2119b80 con 0x563ae2112b40 2024-09-19T09:12:01.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.810+0000 7ff3d4f01640 1 --2- 172.21.15.120:0/2661263812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ae21115d0 0x563ae2118060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:01.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.811+0000 7ff3c7fff640 1 --2- 172.21.15.120:0/2661263812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563ae2112b40 0x563ae2133b70 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0006010 tx=0x7ff3c0011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f38da531e5b821 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:01.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.811+0000 7ff3b67fc640 1 -- 172.21.15.120:0/2661263812 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c00054b0 con 0x563ae2112b40 2024-09-19T09:12:01.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.811+0000 7ff3b67fc640 1 -- 172.21.15.120:0/2661263812 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3c0009940 con 0x563ae2112b40 2024-09-19T09:12:01.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.811+0000 7ff3b67fc640 1 -- 172.21.15.120:0/2661263812 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0030a10 con 0x563ae2112b40 2024-09-19T09:12:01.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.811+0000 7ff3d953e740 1 -- 172.21.15.120:0/2661263812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563ae2112b40 msgr2=0x563ae2133b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:01.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.811+0000 7ff3d953e740 1 --2- 172.21.15.120:0/2661263812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563ae2112b40 0x563ae2133b70 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0006010 tx=0x7ff3c0011f60 comp rx=0 tx=0).stop 2024-09-19T09:12:01.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.812+0000 7ff3d953e740 1 -- 172.21.15.120:0/2661263812 shutdown_connections 2024-09-19T09:12:01.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.812+0000 7ff3d953e740 1 --2- 172.21.15.120:0/2661263812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ae211c370 0x563ae21340b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.812+0000 7ff3d953e740 1 --2- 172.21.15.120:0/2661263812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563ae2112b40 0x563ae2133b70 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.812+0000 7ff3d953e740 1 --2- 172.21.15.120:0/2661263812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ae21115d0 0x563ae2118060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.812+0000 7ff3d953e740 1 -- 172.21.15.120:0/2661263812 >> 172.21.15.120:0/2661263812 conn(0x563ae20da310 msgr2=0x563ae2117950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:01.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.812+0000 7ff3d953e740 1 -- 172.21.15.120:0/2661263812 shutdown_connections 2024-09-19T09:12:01.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.812+0000 7ff3d953e740 1 -- 172.21.15.120:0/2661263812 wait complete. 2024-09-19T09:12:01.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.812+0000 7ff3d953e740 1 -- start start 2024-09-19T09:12:01.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.812+0000 7ff3d953e740 1 Processor -- start 2024-09-19T09:12:01.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.812+0000 7ff3d953e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563ae21115d0 0x563ae211fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:01.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.812+0000 7ff3d953e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ae2112b40 0x563ae21200a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:01.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.812+0000 7ff3d953e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ae211c370 0x563ae218eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:01.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.812+0000 7ff3d953e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d01b0c10 con 0x563ae21115d0 2024-09-19T09:12:01.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.812+0000 7ff3d953e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d01b0540 con 0x563ae211c370 2024-09-19T09:12:01.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.812+0000 7ff3d953e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d01ad760 con 0x563ae2112b40 2024-09-19T09:12:01.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.812+0000 7ff3d953e740 10 client.0 _ll_register_callbacks cb 0x7ff3c80baf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:01.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.813+0000 7ff3d4f01640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563ae21115d0 0x563ae211fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:01.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.813+0000 7ff3d4f01640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563ae21115d0 0x563ae211fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46722/0 (socket says 172.21.15.120:46722) 2024-09-19T09:12:01.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.813+0000 7ff3d4f01640 1 -- 172.21.15.120:0/3293741330 learned_addr learned my addr 172.21.15.120:0/3293741330 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:01.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.813+0000 7ff3c7fff640 1 --2- 172.21.15.120:0/3293741330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ae2112b40 0x563ae21200a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:01.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.813+0000 7ff3d5702640 1 --2- 172.21.15.120:0/3293741330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ae211c370 0x563ae218eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:01.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.813+0000 7ff3d4f01640 1 -- 172.21.15.120:0/3293741330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ae2112b40 msgr2=0x563ae21200a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:01.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.813+0000 7ff3d4f01640 1 --2- 172.21.15.120:0/3293741330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ae2112b40 0x563ae21200a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.813+0000 7ff3d4f01640 1 -- 172.21.15.120:0/3293741330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ae211c370 msgr2=0x563ae218eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:01.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.813+0000 7ff3d4f01640 1 --2- 172.21.15.120:0/3293741330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ae211c370 0x563ae218eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.813+0000 7ff3d4f01640 1 -- 172.21.15.120:0/3293741330 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ae218f1a0 con 0x563ae21115d0 2024-09-19T09:12:01.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.813+0000 7ff3d4f01640 1 --2- 172.21.15.120:0/3293741330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563ae21115d0 0x563ae211fb60 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc006010 tx=0x7ff3cc009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3970f97bea535010 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:01.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.813+0000 7ff3affff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:01.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.813+0000 7ff3affff640 1 -- 172.21.15.120:0/3293741330 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc035040 con 0x563ae21115d0 2024-09-19T09:12:01.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.813+0000 7ff3affff640 1 -- 172.21.15.120:0/3293741330 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ff3cc004760 con 0x563ae21115d0 2024-09-19T09:12:01.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.813+0000 7ff3d953e740 1 -- 172.21.15.120:0/3293741330 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3c0009940 con 0x563ae21115d0 2024-09-19T09:12:01.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.813+0000 7ff3affff640 1 -- 172.21.15.120:0/3293741330 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff3cc005610 con 0x563ae21115d0 2024-09-19T09:12:01.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.813+0000 7ff3d953e740 10 client.18615 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:01.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.813+0000 7ff3d953e740 1 -- 172.21.15.120:0/3293741330 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563ae21965b0 con 0x563ae21115d0 2024-09-19T09:12:01.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.814+0000 7ff3affff640 1 -- 172.21.15.120:0/3293741330 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3c0009940 con 0x563ae21115d0 2024-09-19T09:12:01.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.814+0000 7ff3affff640 1 -- 172.21.15.120:0/3293741330 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff3cc031540 con 0x563ae21115d0 2024-09-19T09:12:01.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.814+0000 7ff3d953e740 10 client.18615 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:01.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.814+0000 7ff3d953e740 10 client.18615 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:12:01.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.814+0000 7ff3d953e740 20 client.18615 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:01.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.814+0000 7ff3d953e740 1 --2- 172.21.15.120:0/3293741330 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x563ae2196ce0 0x563ae2199100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:01.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.814+0000 7ff3d953e740 4 client.18615 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:12:01.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.814+0000 7ff3d953e740 1 -- 172.21.15.120:0/3293741330 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563ae207ed70 con 0x563ae2196ce0 2024-09-19T09:12:01.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.814+0000 7ff3c7fff640 1 --2- 172.21.15.120:0/3293741330 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x563ae2196ce0 0x563ae2199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:01.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.815+0000 7ff3affff640 10 client.18615 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:12:01.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.815+0000 7ff3c7fff640 1 --2- 172.21.15.120:0/3293741330 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x563ae2196ce0 0x563ae2199100 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=15d1ab98996e92c server_cookie=fbf186976bb442dc in_seq=0 out_seq=0 2024-09-19T09:12:01.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.815+0000 7ff3affff640 1 -- 172.21.15.120:0/3293741330 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563ae207ed70 con 0x563ae2196ce0 2024-09-19T09:12:01.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.815+0000 7ff3affff640 10 client.18615 handle_command_reply: tid=0 2024-09-19T09:12:01.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.815+0000 7ff3d953e740 1 client.18615 shutdown 2024-09-19T09:12:01.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.815+0000 7ff3d953e740 1 -- 172.21.15.120:0/3293741330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563ae21115d0 msgr2=0x563ae211fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:01.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.815+0000 7ff3d953e740 1 --2- 172.21.15.120:0/3293741330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563ae21115d0 0x563ae211fb60 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc006010 tx=0x7ff3cc009680 comp rx=0 tx=0).stop 2024-09-19T09:12:01.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.815+0000 7ff3d953e740 1 -- 172.21.15.120:0/3293741330 shutdown_connections 2024-09-19T09:12:01.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.815+0000 7ff3d953e740 1 --2- 172.21.15.120:0/3293741330 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x563ae2196ce0 0x563ae2199100 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.815+0000 7ff3d953e740 1 --2- 172.21.15.120:0/3293741330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563ae211c370 0x563ae218eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.815+0000 7ff3d953e740 1 --2- 172.21.15.120:0/3293741330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ae2112b40 0x563ae21200a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.815+0000 7ff3affff640 0 client.18615 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:12:01.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.815+0000 7ff3d953e740 1 --2- 172.21.15.120:0/3293741330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563ae21115d0 0x563ae211fb60 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.815+0000 7ff3d953e740 1 -- 172.21.15.120:0/3293741330 >> 172.21.15.120:0/3293741330 conn(0x563ae20da310 msgr2=0x563ae21272a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:01.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.815+0000 7ff3d953e740 1 -- 172.21.15.120:0/3293741330 shutdown_connections 2024-09-19T09:12:01.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.815+0000 7ff3d953e740 1 -- 172.21.15.120:0/3293741330 wait complete. 2024-09-19T09:12:01.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.815+0000 7ff3d953e740 20 client.18615 trim_cache size 0 max 16384 2024-09-19T09:12:01.821 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:12:01.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.816+0000 7ff3d7165640 1 -- 172.21.15.120:0/600375630 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff3b80a3230 msgr2=0x7ff3b80a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:01.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.816+0000 7ff3d7165640 1 --2- 172.21.15.120:0/600375630 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff3b80a3230 0x7ff3b80a56f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0011fd0 tx=0x7ff3c0018000 comp rx=0 tx=0).stop 2024-09-19T09:12:01.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.816+0000 7ff3d7165640 1 -- 172.21.15.120:0/600375630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3d0102ef0 msgr2=0x7ff3d019efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:01.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.816+0000 7ff3d7165640 1 --2- 172.21.15.120:0/600375630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3d0102ef0 0x7ff3d019efd0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800da10 tx=0x7ff3c800ded0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.816+0000 7ff3d7165640 1 -- 172.21.15.120:0/600375630 shutdown_connections 2024-09-19T09:12:01.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.816+0000 7ff3d7165640 1 --2- 172.21.15.120:0/600375630 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff3b80a3230 0x7ff3b80a56f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.816+0000 7ff3d7165640 1 --2- 172.21.15.120:0/600375630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3d0102ef0 0x7ff3d019efd0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.816+0000 7ff3d7165640 1 --2- 172.21.15.120:0/600375630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3d0102570 0x7ff3d019ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.816+0000 7ff3d7165640 1 --2- 172.21.15.120:0/600375630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3d0100e20 0x7ff3d019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:01.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.816+0000 7ff3d7165640 1 -- 172.21.15.120:0/600375630 >> 172.21.15.120:0/600375630 conn(0x7ff3d00fc7d0 msgr2=0x7ff3d00fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:01.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.816+0000 7ff3d7165640 1 -- 172.21.15.120:0/600375630 shutdown_connections 2024-09-19T09:12:01.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:01.816+0000 7ff3d7165640 1 -- 172.21.15.120:0/600375630 wait complete. 2024-09-19T09:12:02.064 INFO:tasks.workunit.client.0.smithi120.stdout: 1 867237 9.70 MB/sec execute 357 sec latency 20.219 ms 2024-09-19T09:12:02.561 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:12:02.834 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:12:03.064 INFO:tasks.workunit.client.0.smithi120.stdout: 1 870312 9.73 MB/sec execute 358 sec latency 10.532 ms 2024-09-19T09:12:03.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:02 smithi196 ceph-mon[35150]: pgmap v618: 97 pgs: 97 active+clean; 51 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.3 MiB/s wr, 469 op/s 2024-09-19T09:12:03.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:02 smithi153 ceph-mon[35071]: pgmap v618: 97 pgs: 97 active+clean; 51 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.3 MiB/s wr, 469 op/s 2024-09-19T09:12:03.222 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:02 smithi120 ceph-mon[31150]: pgmap v618: 97 pgs: 97 active+clean; 51 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.3 MiB/s wr, 469 op/s 2024-09-19T09:12:03.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.447+0000 7f577797c640 1 -- 172.21.15.120:0/1826089959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5770100e20 msgr2=0x7f5770101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:03.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.447+0000 7f577797c640 1 --2- 172.21.15.120:0/1826089959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5770100e20 0x7f5770101200 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f576c009900 tx=0x7f576c02a480 comp rx=0 tx=0).stop 2024-09-19T09:12:03.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.447+0000 7f577797c640 1 -- 172.21.15.120:0/1826089959 shutdown_connections 2024-09-19T09:12:03.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.447+0000 7f577797c640 1 --2- 172.21.15.120:0/1826089959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5770102ef0 0x7f577010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.447+0000 7f577797c640 1 --2- 172.21.15.120:0/1826089959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5770102570 0x7f57701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.447+0000 7f577797c640 1 --2- 172.21.15.120:0/1826089959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5770100e20 0x7f5770101200 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.447+0000 7f577797c640 1 -- 172.21.15.120:0/1826089959 >> 172.21.15.120:0/1826089959 conn(0x7f57700fc7d0 msgr2=0x7f57700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:03.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.447+0000 7f577797c640 1 -- 172.21.15.120:0/1826089959 shutdown_connections 2024-09-19T09:12:03.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.447+0000 7f577797c640 1 -- 172.21.15.120:0/1826089959 wait complete. 2024-09-19T09:12:03.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.447+0000 7f577797c640 1 Processor -- start 2024-09-19T09:12:03.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.447+0000 7f577797c640 1 -- start start 2024-09-19T09:12:03.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.448+0000 7f577797c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5770100e20 0x7f5770199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:03.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.448+0000 7f577797c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5770102570 0x7f5770199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:03.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.448+0000 7f577797c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5770102ef0 0x7f57701a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:03.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.448+0000 7f577797c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5770110f80 con 0x7f5770102570 2024-09-19T09:12:03.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.448+0000 7f577797c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5770110e00 con 0x7f5770100e20 2024-09-19T09:12:03.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.448+0000 7f577797c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5770111100 con 0x7f5770102ef0 2024-09-19T09:12:03.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.448+0000 7f5774f17640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5770102570 0x7f5770199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:03.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.448+0000 7f5775f19640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5770102ef0 0x7f57701a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:03.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.448+0000 7f5774f17640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5770102570 0x7f5770199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46732/0 (socket says 172.21.15.120:46732) 2024-09-19T09:12:03.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.448+0000 7f5774f17640 1 -- 172.21.15.120:0/2072256003 learned_addr learned my addr 172.21.15.120:0/2072256003 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:03.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.448+0000 7f5775718640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5770100e20 0x7f5770199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:03.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.448+0000 7f5775f19640 1 -- 172.21.15.120:0/2072256003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5770100e20 msgr2=0x7f5770199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:03.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.448+0000 7f5775f19640 1 --2- 172.21.15.120:0/2072256003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5770100e20 0x7f5770199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.448+0000 7f5775f19640 1 -- 172.21.15.120:0/2072256003 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5770102570 msgr2=0x7f5770199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:03.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.448+0000 7f5775f19640 1 --2- 172.21.15.120:0/2072256003 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5770102570 0x7f5770199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.448+0000 7f5775f19640 1 -- 172.21.15.120:0/2072256003 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57701a48f0 con 0x7f5770102ef0 2024-09-19T09:12:03.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.448+0000 7f5774f17640 1 --2- 172.21.15.120:0/2072256003 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5770102570 0x7f5770199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:03.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.448+0000 7f5775f19640 1 --2- 172.21.15.120:0/2072256003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5770102ef0 0x7f57701a43b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5768009fd0 tx=0x7f576800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:03.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.448+0000 7f57667fc640 1 -- 172.21.15.120:0/2072256003 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576800ce20 con 0x7f5770102ef0 2024-09-19T09:12:03.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.448+0000 7f57667fc640 1 -- 172.21.15.120:0/2072256003 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5768005020 con 0x7f5770102ef0 2024-09-19T09:12:03.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.448+0000 7f577797c640 1 -- 172.21.15.120:0/2072256003 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57701a4ac0 con 0x7f5770102ef0 2024-09-19T09:12:03.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.448+0000 7f57667fc640 1 -- 172.21.15.120:0/2072256003 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5768010720 con 0x7f5770102ef0 2024-09-19T09:12:03.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.448+0000 7f577797c640 1 -- 172.21.15.120:0/2072256003 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57701a4fd0 con 0x7f5770102ef0 2024-09-19T09:12:03.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.451+0000 7f57667fc640 1 -- 172.21.15.120:0/2072256003 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5768005540 con 0x7f5770102ef0 2024-09-19T09:12:03.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.451+0000 7f5779d55740 1 Processor -- start 2024-09-19T09:12:03.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.452+0000 7f5779d55740 1 -- start start 2024-09-19T09:12:03.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.452+0000 7f5779d55740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf89cb0b40 0x55bf89caf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:03.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.452+0000 7f5779d55740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89caf5d0 0x55bf89cafa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:03.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.452+0000 7f5779d55740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf89cba370 0x55bf89cbc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:03.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.452+0000 7f5779d55740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bf89c152f0 con 0x55bf89cb0b40 2024-09-19T09:12:03.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.452+0000 7f5779d55740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bf89bbccf0 con 0x55bf89caf5d0 2024-09-19T09:12:03.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.452+0000 7f5779d55740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bf89c112b0 con 0x55bf89cba370 2024-09-19T09:12:03.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.452+0000 7f5774f17640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf89cb0b40 0x55bf89caf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:03.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.452+0000 7f5775718640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89caf5d0 0x55bf89cafa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:03.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.452+0000 7f5774f17640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf89cb0b40 0x55bf89caf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46740/0 (socket says 172.21.15.120:46740) 2024-09-19T09:12:03.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.452+0000 7f5775718640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89caf5d0 0x55bf89cafa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:40758/0 (socket says 172.21.15.120:40758) 2024-09-19T09:12:03.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.452+0000 7f5774f17640 1 -- 172.21.15.120:0/105032391 learned_addr learned my addr 172.21.15.120:0/105032391 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:03.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.452+0000 7f5775718640 1 -- 172.21.15.120:0/105032391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf89cba370 msgr2=0x55bf89cbc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:03.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.452+0000 7f5774f17640 1 --2- 172.21.15.120:0/105032391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf89cba370 0x55bf89cbc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:03.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.452+0000 7f5775718640 1 --2- 172.21.15.120:0/105032391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf89cba370 0x55bf89cbc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.452+0000 7f5775718640 1 -- 172.21.15.120:0/105032391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf89cb0b40 msgr2=0x55bf89caf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:03.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.452+0000 7f5775718640 1 --2- 172.21.15.120:0/105032391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf89cb0b40 0x55bf89caf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.452+0000 7f5775718640 1 -- 172.21.15.120:0/105032391 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf89ba13a0 con 0x55bf89caf5d0 2024-09-19T09:12:03.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.452+0000 7f57667fc640 1 --2- 172.21.15.120:0/2072256003 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f575c0a3240 0x7f575c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:03.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.452+0000 7f5774f17640 1 --2- 172.21.15.120:0/105032391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf89cba370 0x55bf89cbc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:12:03.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.452+0000 7f57667fc640 1 -- 172.21.15.120:0/2072256003 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f57680cfea0 con 0x7f5770102ef0 2024-09-19T09:12:03.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.452+0000 7f5775718640 1 --2- 172.21.15.120:0/105032391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89caf5d0 0x55bf89cafa10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f576c005940 tx=0x7f576c005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fd492755a66d58b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:03.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.452+0000 7f5774f17640 1 --2- 172.21.15.120:0/2072256003 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f575c0a3240 0x7f575c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:03.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.453+0000 7f575b7fe640 1 -- 172.21.15.120:0/105032391 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c035030 con 0x55bf89caf5d0 2024-09-19T09:12:03.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.453+0000 7f575b7fe640 1 -- 172.21.15.120:0/105032391 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f576c040070 con 0x55bf89caf5d0 2024-09-19T09:12:03.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.453+0000 7f575b7fe640 1 -- 172.21.15.120:0/105032391 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c004ce0 con 0x55bf89caf5d0 2024-09-19T09:12:03.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.453+0000 7f5774f17640 1 --2- 172.21.15.120:0/2072256003 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f575c0a3240 0x7f575c0a5700 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x55bf89cb0070 tx=0x7f5760009cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:03.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.453+0000 7f5779d55740 1 -- 172.21.15.120:0/105032391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89caf5d0 msgr2=0x55bf89cafa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:03.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.453+0000 7f5779d55740 1 --2- 172.21.15.120:0/105032391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89caf5d0 0x55bf89cafa10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f576c005940 tx=0x7f576c005dd0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.454+0000 7f5779d55740 1 -- 172.21.15.120:0/105032391 shutdown_connections 2024-09-19T09:12:03.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.454+0000 7f5779d55740 1 --2- 172.21.15.120:0/105032391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf89cba370 0x55bf89cbc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.454+0000 7f5779d55740 1 --2- 172.21.15.120:0/105032391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89caf5d0 0x55bf89cafa10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.454+0000 7f5779d55740 1 --2- 172.21.15.120:0/105032391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf89cb0b40 0x55bf89caf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.454+0000 7f5779d55740 1 -- 172.21.15.120:0/105032391 >> 172.21.15.120:0/105032391 conn(0x55bf89c78310 msgr2=0x55bf89cabc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:03.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.454+0000 7f5779d55740 1 -- 172.21.15.120:0/105032391 shutdown_connections 2024-09-19T09:12:03.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.454+0000 7f5779d55740 1 -- 172.21.15.120:0/105032391 wait complete. 2024-09-19T09:12:03.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.454+0000 7f5779d55740 1 -- start start 2024-09-19T09:12:03.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.454+0000 7f5779d55740 1 Processor -- start 2024-09-19T09:12:03.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.454+0000 7f5779d55740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf89caf5d0 0x55bf89cb9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:03.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.454+0000 7f5779d55740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf89cb0b40 0x55bf89cb6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:03.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.454+0000 7f5779d55740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89cba370 0x55bf89cb7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:03.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.454+0000 7f5779d55740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bf89cbd010 con 0x55bf89caf5d0 2024-09-19T09:12:03.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.454+0000 7f5779d55740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bf89bd0060 con 0x55bf89cba370 2024-09-19T09:12:03.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.454+0000 7f5779d55740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bf89cbd190 con 0x55bf89cb0b40 2024-09-19T09:12:03.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.455+0000 7f5779d55740 10 client.0 _ll_register_callbacks cb 0x55bf899238c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:03.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.455+0000 7f5775f19640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89cba370 0x55bf89cb7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:03.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.455+0000 7f5774f17640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf89cb0b40 0x55bf89cb6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:03.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.455+0000 7f5775718640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf89caf5d0 0x55bf89cb9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:03.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.455+0000 7f5775f19640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89cba370 0x55bf89cb7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:40760/0 (socket says 172.21.15.120:40760) 2024-09-19T09:12:03.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.455+0000 7f5775f19640 1 -- 172.21.15.120:0/2668070527 learned_addr learned my addr 172.21.15.120:0/2668070527 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:03.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.455+0000 7f5774f17640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf89cb0b40 0x55bf89cb6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:35264/0 (socket says 172.21.15.120:35264) 2024-09-19T09:12:03.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.455+0000 7f5775f19640 1 -- 172.21.15.120:0/2668070527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf89cb0b40 msgr2=0x55bf89cb6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:03.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.455+0000 7f5775f19640 1 --2- 172.21.15.120:0/2668070527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf89cb0b40 0x55bf89cb6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.455+0000 7f5775f19640 1 -- 172.21.15.120:0/2668070527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf89caf5d0 msgr2=0x55bf89cb9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:03.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.455+0000 7f5775f19640 1 --2- 172.21.15.120:0/2668070527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf89caf5d0 0x55bf89cb9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.455+0000 7f5775f19640 1 -- 172.21.15.120:0/2668070527 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf899863c0 con 0x55bf89cba370 2024-09-19T09:12:03.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.455+0000 7f5775718640 1 --2- 172.21.15.120:0/2668070527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf89caf5d0 0x55bf89cb9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:03.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.455+0000 7f5775f19640 1 --2- 172.21.15.120:0/2668070527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89cba370 0x55bf89cb7560 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f576809c450 tx=0x7f576809cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a44ec8824f63314e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:03.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.455+0000 7f5758ff9640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:12:03.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.455+0000 7f5758ff9640 1 -- 172.21.15.120:0/2668070527 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57680cde60 con 0x55bf89cba370 2024-09-19T09:12:03.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.455+0000 7f5758ff9640 1 -- 172.21.15.120:0/2668070527 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f57680051d0 con 0x55bf89cba370 2024-09-19T09:12:03.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.455+0000 7f5779d55740 1 -- 172.21.15.120:0/2668070527 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f576c040070 con 0x55bf89cba370 2024-09-19T09:12:03.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.456+0000 7f5758ff9640 1 -- 172.21.15.120:0/2668070527 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f57680a5130 con 0x55bf89cba370 2024-09-19T09:12:03.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.456+0000 7f5779d55740 10 client.27758 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:03.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.456+0000 7f5779d55740 1 -- 172.21.15.120:0/2668070527 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bf89cc89d0 con 0x55bf89cba370 2024-09-19T09:12:03.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.456+0000 7f5758ff9640 1 -- 172.21.15.120:0/2668070527 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f576c040070 con 0x55bf89cba370 2024-09-19T09:12:03.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.456+0000 7f5758ff9640 1 -- 172.21.15.120:0/2668070527 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f576809b380 con 0x55bf89cba370 2024-09-19T09:12:03.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.456+0000 7f5779d55740 10 client.27758 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:03.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.456+0000 7f5779d55740 10 client.27758 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:12:03.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.456+0000 7f5779d55740 20 client.27758 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:03.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.456+0000 7f5779d55740 1 --2- 172.21.15.120:0/2668070527 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55bf89cc9de0 0x55bf89ccc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:03.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.456+0000 7f5779d55740 4 client.27758 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:12:03.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.456+0000 7f5779d55740 1 -- 172.21.15.120:0/2668070527 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf89c22860 con 0x55bf89cc9de0 2024-09-19T09:12:03.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.456+0000 7f5775718640 1 --2- 172.21.15.120:0/2668070527 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55bf89cc9de0 0x55bf89ccc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:03.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.457+0000 7f5775718640 1 --2- 172.21.15.120:0/2668070527 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55bf89cc9de0 0x55bf89ccc280 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=433a336ab723a7d7 server_cookie=42a2ff39abb97e12 in_seq=0 out_seq=0 2024-09-19T09:12:03.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.457+0000 7f5758ff9640 10 client.27758 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:12:03.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.457+0000 7f5758ff9640 1 -- 172.21.15.120:0/2668070527 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf89c22860 con 0x55bf89cc9de0 2024-09-19T09:12:03.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.457+0000 7f5758ff9640 10 client.27758 handle_command_reply: tid=0 2024-09-19T09:12:03.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.457+0000 7f5779d55740 1 client.27758 shutdown 2024-09-19T09:12:03.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.458+0000 7f5779d55740 1 -- 172.21.15.120:0/2668070527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89cba370 msgr2=0x55bf89cb7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:03.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.458+0000 7f5779d55740 1 --2- 172.21.15.120:0/2668070527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89cba370 0x55bf89cb7560 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f576809c450 tx=0x7f576809cc80 comp rx=0 tx=0).stop 2024-09-19T09:12:03.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.458+0000 7f5779d55740 1 -- 172.21.15.120:0/2668070527 shutdown_connections 2024-09-19T09:12:03.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.458+0000 7f5779d55740 1 --2- 172.21.15.120:0/2668070527 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55bf89cc9de0 0x55bf89ccc280 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.458+0000 7f5779d55740 1 --2- 172.21.15.120:0/2668070527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89cba370 0x55bf89cb7560 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.458+0000 7f5779d55740 1 --2- 172.21.15.120:0/2668070527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf89cb0b40 0x55bf89cb6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.458+0000 7f5779d55740 1 --2- 172.21.15.120:0/2668070527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf89caf5d0 0x55bf89cb9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.458+0000 7f5779d55740 1 -- 172.21.15.120:0/2668070527 >> 172.21.15.120:0/2668070527 conn(0x55bf89c78310 msgr2=0x55bf89cb1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:03.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.458+0000 7f5758ff9640 0 client.27758 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:12:03.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.458+0000 7f5779d55740 1 -- 172.21.15.120:0/2668070527 shutdown_connections 2024-09-19T09:12:03.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.458+0000 7f5779d55740 1 -- 172.21.15.120:0/2668070527 wait complete. 2024-09-19T09:12:03.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.458+0000 7f5779d55740 20 client.27758 trim_cache size 0 max 16384 2024-09-19T09:12:03.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.472+0000 7f5779d55740 1 Processor -- start 2024-09-19T09:12:03.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.472+0000 7f5779d55740 1 -- start start 2024-09-19T09:12:03.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.473+0000 7f5779d55740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf89caf5d0 0x55bf89cc62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:03.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.473+0000 7f5779d55740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf89cb0b40 0x55bf89cb58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:03.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.473+0000 7f5779d55740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89cba370 0x55bf89cb5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:03.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.473+0000 7f5779d55740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f57701ab380 con 0x55bf89cb0b40 2024-09-19T09:12:03.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.473+0000 7f5779d55740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f57701acec0 con 0x55bf89cba370 2024-09-19T09:12:03.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.473+0000 7f5779d55740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f57701aba50 con 0x55bf89caf5d0 2024-09-19T09:12:03.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.473+0000 7f5774f17640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf89cb0b40 0x55bf89cb58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:03.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.473+0000 7f5775718640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf89caf5d0 0x55bf89cc62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:03.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.473+0000 7f5775f19640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89cba370 0x55bf89cb5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:03.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.473+0000 7f5774f17640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf89cb0b40 0x55bf89cb58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46756/0 (socket says 172.21.15.120:46756) 2024-09-19T09:12:03.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.473+0000 7f5775718640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf89caf5d0 0x55bf89cc62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:35270/0 (socket says 172.21.15.120:35270) 2024-09-19T09:12:03.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.473+0000 7f5774f17640 1 -- 172.21.15.120:0/2369151099 learned_addr learned my addr 172.21.15.120:0/2369151099 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:03.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.473+0000 7f5775f19640 1 -- 172.21.15.120:0/2369151099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf89caf5d0 msgr2=0x55bf89cc62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:03.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.473+0000 7f5775f19640 1 --2- 172.21.15.120:0/2369151099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf89caf5d0 0x55bf89cc62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.473+0000 7f5775f19640 1 -- 172.21.15.120:0/2369151099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf89cb0b40 msgr2=0x55bf89cb58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:03.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.473+0000 7f5775f19640 1 --2- 172.21.15.120:0/2369151099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf89cb0b40 0x55bf89cb58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.473+0000 7f5775f19640 1 -- 172.21.15.120:0/2369151099 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf89cba080 con 0x55bf89cba370 2024-09-19T09:12:03.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.473+0000 7f5775f19640 1 --2- 172.21.15.120:0/2369151099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89cba370 0x55bf89cb5e10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5768098e90 tx=0x7f576809a9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=736ccd128a657d7d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:03.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.473+0000 7f575affd640 1 -- 172.21.15.120:0/2369151099 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57680cde60 con 0x55bf89cba370 2024-09-19T09:12:03.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.474+0000 7f575affd640 1 -- 172.21.15.120:0/2369151099 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f576807f030 con 0x55bf89cba370 2024-09-19T09:12:03.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.474+0000 7f575affd640 1 -- 172.21.15.120:0/2369151099 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576809b380 con 0x55bf89cba370 2024-09-19T09:12:03.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.474+0000 7f5779d55740 1 -- 172.21.15.120:0/2369151099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89cba370 msgr2=0x55bf89cb5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:03.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.474+0000 7f5779d55740 1 --2- 172.21.15.120:0/2369151099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89cba370 0x55bf89cb5e10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5768098e90 tx=0x7f576809a9d0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.474+0000 7f5779d55740 1 -- 172.21.15.120:0/2369151099 shutdown_connections 2024-09-19T09:12:03.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.474+0000 7f5779d55740 1 --2- 172.21.15.120:0/2369151099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89cba370 0x55bf89cb5e10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.474+0000 7f5779d55740 1 --2- 172.21.15.120:0/2369151099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf89cb0b40 0x55bf89cb58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.474+0000 7f5779d55740 1 --2- 172.21.15.120:0/2369151099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf89caf5d0 0x55bf89cc62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.474+0000 7f5779d55740 1 -- 172.21.15.120:0/2369151099 >> 172.21.15.120:0/2369151099 conn(0x55bf89c78310 msgr2=0x55bf89cb22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:03.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.474+0000 7f5779d55740 1 -- 172.21.15.120:0/2369151099 shutdown_connections 2024-09-19T09:12:03.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.475+0000 7f5779d55740 1 -- 172.21.15.120:0/2369151099 wait complete. 2024-09-19T09:12:03.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.475+0000 7f5779d55740 1 -- start start 2024-09-19T09:12:03.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.475+0000 7f5779d55740 1 Processor -- start 2024-09-19T09:12:03.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.475+0000 7f5779d55740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf89caf5d0 0x55bf89cb4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:03.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.475+0000 7f5779d55740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf89cb0b40 0x55bf89cc2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:03.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.475+0000 7f5779d55740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89cba370 0x55bf89cc2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:03.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.475+0000 7f5779d55740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f57701ac7f0 con 0x55bf89caf5d0 2024-09-19T09:12:03.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.475+0000 7f5779d55740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f57701ac120 con 0x55bf89cba370 2024-09-19T09:12:03.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.475+0000 7f5779d55740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f57701a9340 con 0x55bf89cb0b40 2024-09-19T09:12:03.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.476+0000 7f5779d55740 10 client.0 _ll_register_callbacks cb 0x7f576809c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:03.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.476+0000 7f5775718640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf89caf5d0 0x55bf89cb4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:03.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.476+0000 7f5775f19640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89cba370 0x55bf89cc2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:03.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.476+0000 7f5774f17640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf89cb0b40 0x55bf89cc2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:03.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.476+0000 7f5775718640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf89caf5d0 0x55bf89cb4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46764/0 (socket says 172.21.15.120:46764) 2024-09-19T09:12:03.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.476+0000 7f5775718640 1 -- 172.21.15.120:0/2479339093 learned_addr learned my addr 172.21.15.120:0/2479339093 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:03.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.476+0000 7f5775718640 1 -- 172.21.15.120:0/2479339093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf89cb0b40 msgr2=0x55bf89cc2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:03.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.476+0000 7f5775718640 1 --2- 172.21.15.120:0/2479339093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf89cb0b40 0x55bf89cc2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.476+0000 7f5775718640 1 -- 172.21.15.120:0/2479339093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89cba370 msgr2=0x55bf89cc2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:03.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.476+0000 7f5775718640 1 --2- 172.21.15.120:0/2479339093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89cba370 0x55bf89cc2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.476+0000 7f5775718640 1 -- 172.21.15.120:0/2479339093 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf89cb32f0 con 0x55bf89caf5d0 2024-09-19T09:12:03.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.476+0000 7f5775f19640 1 --2- 172.21.15.120:0/2479339093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89cba370 0x55bf89cc2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:03.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.476+0000 7f5775718640 1 --2- 172.21.15.120:0/2479339093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf89caf5d0 0x55bf89cb4ce0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f576c031470 tx=0x7f576c004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b854b4bd20fc018c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:03.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.476+0000 7f5758ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:03.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.476+0000 7f5758ff9640 1 -- 172.21.15.120:0/2479339093 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c049040 con 0x55bf89caf5d0 2024-09-19T09:12:03.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.476+0000 7f5758ff9640 1 -- 172.21.15.120:0/2479339093 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f576c003a00 con 0x55bf89caf5d0 2024-09-19T09:12:03.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.476+0000 7f5779d55740 1 -- 172.21.15.120:0/2479339093 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f576807f030 con 0x55bf89caf5d0 2024-09-19T09:12:03.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.477+0000 7f5758ff9640 1 -- 172.21.15.120:0/2479339093 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f576c035040 con 0x55bf89caf5d0 2024-09-19T09:12:03.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.477+0000 7f5779d55740 10 client.18633 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:03.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.477+0000 7f5779d55740 1 -- 172.21.15.120:0/2479339093 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bf89d33ea0 con 0x55bf89caf5d0 2024-09-19T09:12:03.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.477+0000 7f5758ff9640 1 -- 172.21.15.120:0/2479339093 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f576807f030 con 0x55bf89caf5d0 2024-09-19T09:12:03.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.477+0000 7f5758ff9640 1 -- 172.21.15.120:0/2479339093 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f576c03c440 con 0x55bf89caf5d0 2024-09-19T09:12:03.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.477+0000 7f5779d55740 10 client.18633 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:03.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.477+0000 7f5779d55740 10 client.18633 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:12:03.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.477+0000 7f5779d55740 20 client.18633 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:03.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.477+0000 7f5779d55740 1 --2- 172.21.15.120:0/2479339093 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55bf89d34180 0x55bf89d36570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:03.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.477+0000 7f5779d55740 4 client.18633 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:12:03.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.477+0000 7f5779d55740 1 -- 172.21.15.120:0/2479339093 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf89c1cd70 con 0x55bf89d34180 2024-09-19T09:12:03.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.477+0000 7f5774f17640 1 --2- 172.21.15.120:0/2479339093 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55bf89d34180 0x55bf89d36570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:03.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.477+0000 7f5774f17640 1 --2- 172.21.15.120:0/2479339093 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55bf89d34180 0x55bf89d36570 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a3740cdd222383bc server_cookie=3588b1dfec89c4f6 in_seq=0 out_seq=0 2024-09-19T09:12:03.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.477+0000 7f5758ff9640 10 client.18633 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:12:03.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.478+0000 7f5758ff9640 1 -- 172.21.15.120:0/2479339093 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf89c1cd70 con 0x55bf89d34180 2024-09-19T09:12:03.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.478+0000 7f5758ff9640 10 client.18633 handle_command_reply: tid=0 2024-09-19T09:12:03.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.478+0000 7f5779d55740 1 client.18633 shutdown 2024-09-19T09:12:03.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.478+0000 7f5779d55740 1 -- 172.21.15.120:0/2479339093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf89caf5d0 msgr2=0x55bf89cb4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:03.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.478+0000 7f5779d55740 1 --2- 172.21.15.120:0/2479339093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf89caf5d0 0x55bf89cb4ce0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f576c031470 tx=0x7f576c004310 comp rx=0 tx=0).stop 2024-09-19T09:12:03.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.478+0000 7f5779d55740 1 -- 172.21.15.120:0/2479339093 shutdown_connections 2024-09-19T09:12:03.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.478+0000 7f5779d55740 1 --2- 172.21.15.120:0/2479339093 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55bf89d34180 0x55bf89d36570 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.478+0000 7f5779d55740 1 --2- 172.21.15.120:0/2479339093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf89cba370 0x55bf89cc2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.478+0000 7f5758ff9640 0 client.18633 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:12:03.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.478+0000 7f5779d55740 1 --2- 172.21.15.120:0/2479339093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf89cb0b40 0x55bf89cc2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.478+0000 7f5779d55740 1 --2- 172.21.15.120:0/2479339093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf89caf5d0 0x55bf89cb4ce0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.478+0000 7f5779d55740 1 -- 172.21.15.120:0/2479339093 >> 172.21.15.120:0/2479339093 conn(0x55bf89c78310 msgr2=0x55bf89cab6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:03.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.478+0000 7f5779d55740 1 -- 172.21.15.120:0/2479339093 shutdown_connections 2024-09-19T09:12:03.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.478+0000 7f5779d55740 1 -- 172.21.15.120:0/2479339093 wait complete. 2024-09-19T09:12:03.508 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:12:03.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.478+0000 7f5779d55740 20 client.18633 trim_cache size 0 max 16384 2024-09-19T09:12:03.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.479+0000 7f577797c640 1 -- 172.21.15.120:0/2072256003 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f575c0a3240 msgr2=0x7f575c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:03.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.479+0000 7f577797c640 1 --2- 172.21.15.120:0/2072256003 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f575c0a3240 0x7f575c0a5700 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x55bf89cb0070 tx=0x7f5760009cc0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.479+0000 7f577797c640 1 -- 172.21.15.120:0/2072256003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5770102ef0 msgr2=0x7f57701a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:03.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.479+0000 7f577797c640 1 --2- 172.21.15.120:0/2072256003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5770102ef0 0x7f57701a43b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5768009fd0 tx=0x7f576800ef10 comp rx=0 tx=0).stop 2024-09-19T09:12:03.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.479+0000 7f577797c640 1 -- 172.21.15.120:0/2072256003 shutdown_connections 2024-09-19T09:12:03.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.479+0000 7f577797c640 1 --2- 172.21.15.120:0/2072256003 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f575c0a3240 0x7f575c0a5700 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.479+0000 7f577797c640 1 --2- 172.21.15.120:0/2072256003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5770102ef0 0x7f57701a43b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.479+0000 7f577797c640 1 --2- 172.21.15.120:0/2072256003 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5770102570 0x7f5770199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.479+0000 7f577797c640 1 --2- 172.21.15.120:0/2072256003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5770100e20 0x7f5770199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:03.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.479+0000 7f577797c640 1 -- 172.21.15.120:0/2072256003 >> 172.21.15.120:0/2072256003 conn(0x7f57700fc7d0 msgr2=0x7f57700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:03.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.479+0000 7f577797c640 1 -- 172.21.15.120:0/2072256003 shutdown_connections 2024-09-19T09:12:03.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:03.479+0000 7f577797c640 1 -- 172.21.15.120:0/2072256003 wait complete. 2024-09-19T09:12:04.065 INFO:tasks.workunit.client.0.smithi120.stdout: 1 873471 9.75 MB/sec execute 359 sec latency 12.555 ms 2024-09-19T09:12:04.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:03 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:12:04.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:03 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:12:04.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:03 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:12:04.986 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:04 smithi120 ceph-mon[31150]: pgmap v619: 97 pgs: 97 active+clean; 52 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 429 op/s 2024-09-19T09:12:05.020 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:12:05.065 INFO:tasks.workunit.client.0.smithi120.stdout: 1 875943 9.75 MB/sec execute 360 sec latency 29.807 ms 2024-09-19T09:12:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.069+0000 7f276bb8d640 1 -- 172.21.15.120:0/3605891262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2764102d50 msgr2=0x7f27641031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.069+0000 7f276bb8d640 1 --2- 172.21.15.120:0/3605891262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2764102d50 0x7f27641031d0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f2754009900 tx=0x7f275402a410 comp rx=0 tx=0).stop 2024-09-19T09:12:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.069+0000 7f276bb8d640 1 -- 172.21.15.120:0/3605891262 shutdown_connections 2024-09-19T09:12:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.069+0000 7f276bb8d640 1 --2- 172.21.15.120:0/3605891262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2764103710 0x7f2764109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.069+0000 7f276bb8d640 1 --2- 172.21.15.120:0/3605891262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2764102d50 0x7f27641031d0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.069+0000 7f276bb8d640 1 --2- 172.21.15.120:0/3605891262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2764101b60 0x7f2764101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.069+0000 7f276bb8d640 1 -- 172.21.15.120:0/3605891262 >> 172.21.15.120:0/3605891262 conn(0x7f27640fd2e0 msgr2=0x7f27640ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.069+0000 7f276bb8d640 1 -- 172.21.15.120:0/3605891262 shutdown_connections 2024-09-19T09:12:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.069+0000 7f276bb8d640 1 -- 172.21.15.120:0/3605891262 wait complete. 2024-09-19T09:12:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.070+0000 7f276bb8d640 1 Processor -- start 2024-09-19T09:12:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.070+0000 7f276bb8d640 1 -- start start 2024-09-19T09:12:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.070+0000 7f276bb8d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2764101b60 0x7f2764075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.070+0000 7f276bb8d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2764102d50 0x7f27640745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.070+0000 7f276bb8d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2764103710 0x7f2764074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.070+0000 7f276bb8d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f276410d7c0 con 0x7f2764102d50 2024-09-19T09:12:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.070+0000 7f276bb8d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f276410d640 con 0x7f2764101b60 2024-09-19T09:12:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.070+0000 7f276bb8d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f276410d940 con 0x7f2764103710 2024-09-19T09:12:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.070+0000 7f2769128640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2764102d50 0x7f27640745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.071+0000 7f2769128640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2764102d50 0x7f27640745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46778/0 (socket says 172.21.15.120:46778) 2024-09-19T09:12:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.071+0000 7f2769128640 1 -- 172.21.15.120:0/1099740185 learned_addr learned my addr 172.21.15.120:0/1099740185 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.071+0000 7f2769929640 1 --2- 172.21.15.120:0/1099740185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2764101b60 0x7f2764075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.071+0000 7f276a12a640 1 --2- 172.21.15.120:0/1099740185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2764103710 0x7f2764074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.071+0000 7f2769128640 1 -- 172.21.15.120:0/1099740185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2764103710 msgr2=0x7f2764074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.071+0000 7f2769128640 1 --2- 172.21.15.120:0/1099740185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2764103710 0x7f2764074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.071+0000 7f2769128640 1 -- 172.21.15.120:0/1099740185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2764101b60 msgr2=0x7f2764075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.071+0000 7f2769128640 1 --2- 172.21.15.120:0/1099740185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2764101b60 0x7f2764075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.071+0000 7f2769128640 1 -- 172.21.15.120:0/1099740185 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27640753b0 con 0x7f2764102d50 2024-09-19T09:12:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.071+0000 7f2769929640 1 --2- 172.21.15.120:0/1099740185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2764101b60 0x7f2764075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.071+0000 7f2769128640 1 --2- 172.21.15.120:0/1099740185 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2764102d50 0x7f27640745e0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f2754009230 tx=0x7f2754031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.071+0000 7f275affd640 1 -- 172.21.15.120:0/1099740185 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275403f030 con 0x7f2764102d50 2024-09-19T09:12:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.071+0000 7f275affd640 1 -- 172.21.15.120:0/1099740185 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2754003ad0 con 0x7f2764102d50 2024-09-19T09:12:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.071+0000 7f276bb8d640 1 -- 172.21.15.120:0/1099740185 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27641a1ea0 con 0x7f2764102d50 2024-09-19T09:12:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.071+0000 7f275affd640 1 -- 172.21.15.120:0/1099740185 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2754035070 con 0x7f2764102d50 2024-09-19T09:12:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.071+0000 7f276bb8d640 1 -- 172.21.15.120:0/1099740185 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27641a2300 con 0x7f2764102d50 2024-09-19T09:12:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.074+0000 7f275affd640 1 -- 172.21.15.120:0/1099740185 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f2754042030 con 0x7f2764102d50 2024-09-19T09:12:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.074+0000 7f276ea04740 1 Processor -- start 2024-09-19T09:12:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.075+0000 7f276ea04740 1 -- start start 2024-09-19T09:12:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.075+0000 7f276ea04740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643cf3a2db0 0x5643cf3a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.075+0000 7f276ea04740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643cf3a36f0 0x5643cf3a81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.075+0000 7f276ea04740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cf3a87d0 0x5643cf3aac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.075+0000 7f276ea04740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5643cf30b2f0 con 0x5643cf3a36f0 2024-09-19T09:12:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.075+0000 7f276ea04740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5643cf2b3270 con 0x5643cf3a2db0 2024-09-19T09:12:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.075+0000 7f276ea04740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5643cf3072b0 con 0x5643cf3a87d0 2024-09-19T09:12:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.075+0000 7f2769929640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643cf3a36f0 0x5643cf3a81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.075+0000 7f2769929640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643cf3a36f0 0x5643cf3a81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46794/0 (socket says 172.21.15.120:46794) 2024-09-19T09:12:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.075+0000 7f2769929640 1 -- 172.21.15.120:0/3200276725 learned_addr learned my addr 172.21.15.120:0/3200276725 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.075+0000 7f2769128640 1 --2- 172.21.15.120:0/3200276725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cf3a87d0 0x5643cf3aac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.075+0000 7f276a12a640 1 --2- 172.21.15.120:0/3200276725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643cf3a2db0 0x5643cf3a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.076+0000 7f2769929640 1 -- 172.21.15.120:0/3200276725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cf3a87d0 msgr2=0x5643cf3aac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.076+0000 7f2769929640 1 --2- 172.21.15.120:0/3200276725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cf3a87d0 0x5643cf3aac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.076+0000 7f2769929640 1 -- 172.21.15.120:0/3200276725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643cf3a2db0 msgr2=0x5643cf3a31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.076+0000 7f2769929640 1 --2- 172.21.15.120:0/3200276725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643cf3a2db0 0x5643cf3a31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.076+0000 7f2769929640 1 -- 172.21.15.120:0/3200276725 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643cf2b2340 con 0x5643cf3a36f0 2024-09-19T09:12:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.076+0000 7f276a12a640 1 --2- 172.21.15.120:0/3200276725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643cf3a2db0 0x5643cf3a31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:12:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.076+0000 7f2769128640 1 --2- 172.21.15.120:0/3200276725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cf3a87d0 0x5643cf3aac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:12:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.076+0000 7f275affd640 1 --2- 172.21.15.120:0/1099740185 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f27500a32f0 0x7f27500a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.076+0000 7f2769929640 1 --2- 172.21.15.120:0/3200276725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643cf3a36f0 0x5643cf3a81b0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f2760032460 tx=0x7f2760032920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50cdd111ad7bbd0a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.076+0000 7f276a12a640 1 --2- 172.21.15.120:0/1099740185 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f27500a32f0 0x7f27500a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.076+0000 7f275affd640 1 -- 172.21.15.120:0/1099740185 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f27540efdb0 con 0x7f2764102d50 2024-09-19T09:12:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.077+0000 7f274ffff640 1 -- 172.21.15.120:0/3200276725 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2760033770 con 0x5643cf3a36f0 2024-09-19T09:12:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.077+0000 7f274ffff640 1 -- 172.21.15.120:0/3200276725 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2760033df0 con 0x5643cf3a36f0 2024-09-19T09:12:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.077+0000 7f274ffff640 1 -- 172.21.15.120:0/3200276725 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276003c380 con 0x5643cf3a36f0 2024-09-19T09:12:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.077+0000 7f276ea04740 1 -- 172.21.15.120:0/3200276725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643cf3a36f0 msgr2=0x5643cf3a81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.077+0000 7f276a12a640 1 --2- 172.21.15.120:0/1099740185 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f27500a32f0 0x7f27500a57b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f275c005fd0 tx=0x7f275c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.077+0000 7f276ea04740 1 --2- 172.21.15.120:0/3200276725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643cf3a36f0 0x5643cf3a81b0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f2760032460 tx=0x7f2760032920 comp rx=0 tx=0).stop 2024-09-19T09:12:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.078+0000 7f276ea04740 1 -- 172.21.15.120:0/3200276725 shutdown_connections 2024-09-19T09:12:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.078+0000 7f276ea04740 1 --2- 172.21.15.120:0/3200276725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cf3a87d0 0x5643cf3aac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.078+0000 7f276ea04740 1 --2- 172.21.15.120:0/3200276725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643cf3a36f0 0x5643cf3a81b0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.078+0000 7f276ea04740 1 --2- 172.21.15.120:0/3200276725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643cf3a2db0 0x5643cf3a31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.078+0000 7f276ea04740 1 -- 172.21.15.120:0/3200276725 >> 172.21.15.120:0/3200276725 conn(0x5643cf3ab820 msgr2=0x5643cf3abc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.078+0000 7f276ea04740 1 -- 172.21.15.120:0/3200276725 shutdown_connections 2024-09-19T09:12:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.078+0000 7f276ea04740 1 -- 172.21.15.120:0/3200276725 wait complete. 2024-09-19T09:12:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.078+0000 7f276ea04740 1 -- start start 2024-09-19T09:12:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.078+0000 7f276ea04740 1 Processor -- start 2024-09-19T09:12:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.079+0000 7f276ea04740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643cf3a2db0 0x5643cf3b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.079+0000 7f276ea04740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643cf3a36f0 0x5643cf3b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.079+0000 7f276ea04740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cf3a87d0 0x5643cf3b8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.079+0000 7f2769929640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643cf3a2db0 0x5643cf3b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.079+0000 7f2769128640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643cf3a36f0 0x5643cf3b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.079+0000 7f2769929640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643cf3a2db0 0x5643cf3b1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46808/0 (socket says 172.21.15.120:46808) 2024-09-19T09:12:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.079+0000 7f2769128640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643cf3a36f0 0x5643cf3b1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:40822/0 (socket says 172.21.15.120:40822) 2024-09-19T09:12:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.079+0000 7f276ea04740 1 -- 172.21.15.120:0/3522033788 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5643cf3b0070 con 0x5643cf3a2db0 2024-09-19T09:12:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.079+0000 7f2769929640 1 -- 172.21.15.120:0/3522033788 learned_addr learned my addr 172.21.15.120:0/3522033788 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.079+0000 7f276ea04740 1 -- 172.21.15.120:0/3522033788 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5643cf294290 con 0x5643cf3a36f0 2024-09-19T09:12:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.079+0000 7f276ea04740 1 -- 172.21.15.120:0/3522033788 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5643cf3b01f0 con 0x5643cf3a87d0 2024-09-19T09:12:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.079+0000 7f276ea04740 10 client.0 _ll_register_callbacks cb 0x5643cf2c5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.079+0000 7f276a12a640 1 --2- 172.21.15.120:0/3522033788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cf3a87d0 0x5643cf3b8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.079+0000 7f2769128640 1 -- 172.21.15.120:0/3522033788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cf3a87d0 msgr2=0x5643cf3b8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.079+0000 7f2769128640 1 --2- 172.21.15.120:0/3522033788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cf3a87d0 0x5643cf3b8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.079+0000 7f2769128640 1 -- 172.21.15.120:0/3522033788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643cf3a2db0 msgr2=0x5643cf3b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.079+0000 7f2769128640 1 --2- 172.21.15.120:0/3522033788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643cf3a2db0 0x5643cf3b1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.079+0000 7f2769128640 1 -- 172.21.15.120:0/3522033788 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643cf01e210 con 0x5643cf3a36f0 2024-09-19T09:12:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.079+0000 7f2769929640 1 --2- 172.21.15.120:0/3522033788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643cf3a2db0 0x5643cf3b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.080+0000 7f2769128640 1 --2- 172.21.15.120:0/3522033788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643cf3a36f0 0x5643cf3b1e40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f27540314e0 tx=0x7f27540bbf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c42210f020aa729 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.080+0000 7f274d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:12:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.080+0000 7f274d7fa640 1 -- 172.21.15.120:0/3522033788 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2754049070 con 0x5643cf3a36f0 2024-09-19T09:12:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.080+0000 7f274d7fa640 1 -- 172.21.15.120:0/3522033788 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f27540b8a40 con 0x5643cf3a36f0 2024-09-19T09:12:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.080+0000 7f276ea04740 1 -- 172.21.15.120:0/3522033788 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2760033df0 con 0x5643cf3a36f0 2024-09-19T09:12:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.080+0000 7f274d7fa640 1 -- 172.21.15.120:0/3522033788 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f27540bcad0 con 0x5643cf3a36f0 2024-09-19T09:12:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.080+0000 7f276ea04740 10 client.27776 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.080+0000 7f276ea04740 1 -- 172.21.15.120:0/3522033788 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5643cf0ea620 con 0x5643cf3a36f0 2024-09-19T09:12:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.080+0000 7f274d7fa640 1 -- 172.21.15.120:0/3522033788 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2760033df0 con 0x5643cf3a36f0 2024-09-19T09:12:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.081+0000 7f274d7fa640 1 -- 172.21.15.120:0/3522033788 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f27540bc630 con 0x5643cf3a36f0 2024-09-19T09:12:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.081+0000 7f276ea04740 10 client.27776 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.081+0000 7f276ea04740 10 client.27776 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:12:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.081+0000 7f276ea04740 20 client.27776 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.081+0000 7f276ea04740 1 --2- 172.21.15.120:0/3522033788 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5643cf3bd550 0x5643cf3bf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.081+0000 7f276ea04740 4 client.27776 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:12:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.081+0000 7f276ea04740 1 -- 172.21.15.120:0/3522033788 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643cf318a20 con 0x5643cf3bd550 2024-09-19T09:12:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.081+0000 7f2769929640 1 --2- 172.21.15.120:0/3522033788 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5643cf3bd550 0x5643cf3bf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.081+0000 7f274d7fa640 10 client.27776 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.081+0000 7f2769929640 1 --2- 172.21.15.120:0/3522033788 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5643cf3bd550 0x5643cf3bf9a0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2db7fd405df33ef server_cookie=c34a492376e9a792 in_seq=0 out_seq=0 2024-09-19T09:12:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.082+0000 7f274d7fa640 1 -- 172.21.15.120:0/3522033788 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5643cf318a20 con 0x5643cf3bd550 2024-09-19T09:12:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.082+0000 7f274d7fa640 10 client.27776 handle_command_reply: tid=0 2024-09-19T09:12:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.082+0000 7f276ea04740 1 client.27776 shutdown 2024-09-19T09:12:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.082+0000 7f276ea04740 1 -- 172.21.15.120:0/3522033788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643cf3a36f0 msgr2=0x5643cf3b1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.082+0000 7f276ea04740 1 --2- 172.21.15.120:0/3522033788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643cf3a36f0 0x5643cf3b1e40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f27540314e0 tx=0x7f27540bbf10 comp rx=0 tx=0).stop 2024-09-19T09:12:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.083+0000 7f276ea04740 1 -- 172.21.15.120:0/3522033788 shutdown_connections 2024-09-19T09:12:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.083+0000 7f276ea04740 1 --2- 172.21.15.120:0/3522033788 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5643cf3bd550 0x5643cf3bf9a0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.083+0000 7f274d7fa640 0 client.27776 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.083+0000 7f276ea04740 1 --2- 172.21.15.120:0/3522033788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cf3a87d0 0x5643cf3b8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.083+0000 7f276ea04740 1 --2- 172.21.15.120:0/3522033788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643cf3a36f0 0x5643cf3b1e40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.083+0000 7f276ea04740 1 --2- 172.21.15.120:0/3522033788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643cf3a2db0 0x5643cf3b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.083+0000 7f276ea04740 1 -- 172.21.15.120:0/3522033788 >> 172.21.15.120:0/3522033788 conn(0x5643cf3ab820 msgr2=0x5643cf3ad030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.083+0000 7f276ea04740 1 -- 172.21.15.120:0/3522033788 shutdown_connections 2024-09-19T09:12:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.083+0000 7f276ea04740 1 -- 172.21.15.120:0/3522033788 wait complete. 2024-09-19T09:12:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.083+0000 7f276ea04740 20 client.27776 trim_cache size 0 max 16384 2024-09-19T09:12:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.100+0000 7f276ea04740 1 Processor -- start 2024-09-19T09:12:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.100+0000 7f276ea04740 1 -- start start 2024-09-19T09:12:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.100+0000 7f276ea04740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643cf3a2db0 0x5643cf3a20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.100+0000 7f276ea04740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cf3a36f0 0x5643cf3a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.100+0000 7f276ea04740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643cf3a87d0 0x5643cf3b8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.100+0000 7f276ea04740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f27641a6e00 con 0x5643cf3a2db0 2024-09-19T09:12:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.100+0000 7f276ea04740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f27641a8940 con 0x5643cf3a87d0 2024-09-19T09:12:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.100+0000 7f276ea04740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f27641a74d0 con 0x5643cf3a36f0 2024-09-19T09:12:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.101+0000 7f2769929640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643cf3a2db0 0x5643cf3a20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.101+0000 7f2769929640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643cf3a2db0 0x5643cf3a20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46822/0 (socket says 172.21.15.120:46822) 2024-09-19T09:12:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.101+0000 7f2769929640 1 -- 172.21.15.120:0/2306071457 learned_addr learned my addr 172.21.15.120:0/2306071457 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.101+0000 7f276a12a640 1 --2- 172.21.15.120:0/2306071457 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643cf3a87d0 0x5643cf3b8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.101+0000 7f2769128640 1 --2- 172.21.15.120:0/2306071457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cf3a36f0 0x5643cf3a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.101+0000 7f2769929640 1 -- 172.21.15.120:0/2306071457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cf3a36f0 msgr2=0x5643cf3a25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.101+0000 7f2769929640 1 --2- 172.21.15.120:0/2306071457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cf3a36f0 0x5643cf3a25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.101+0000 7f2769929640 1 -- 172.21.15.120:0/2306071457 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643cf3a87d0 msgr2=0x5643cf3b8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.101+0000 7f2769929640 1 --2- 172.21.15.120:0/2306071457 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643cf3a87d0 0x5643cf3b8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.101+0000 7f2769929640 1 -- 172.21.15.120:0/2306071457 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643cf3b1440 con 0x5643cf3a2db0 2024-09-19T09:12:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.101+0000 7f2769929640 1 --2- 172.21.15.120:0/2306071457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643cf3a2db0 0x5643cf3a20b0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f276000b620 tx=0x7f2760035040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12eaf4deb338f8df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.101+0000 7f274f7fe640 1 -- 172.21.15.120:0/2306071457 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27600052b0 con 0x5643cf3a2db0 2024-09-19T09:12:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.101+0000 7f274f7fe640 1 -- 172.21.15.120:0/2306071457 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f276003eca0 con 0x5643cf3a2db0 2024-09-19T09:12:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.101+0000 7f274f7fe640 1 -- 172.21.15.120:0/2306071457 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2760003cf0 con 0x5643cf3a2db0 2024-09-19T09:12:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.102+0000 7f276ea04740 1 -- 172.21.15.120:0/2306071457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643cf3a2db0 msgr2=0x5643cf3a20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.102+0000 7f276ea04740 1 --2- 172.21.15.120:0/2306071457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643cf3a2db0 0x5643cf3a20b0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f276000b620 tx=0x7f2760035040 comp rx=0 tx=0).stop 2024-09-19T09:12:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.102+0000 7f276ea04740 1 -- 172.21.15.120:0/2306071457 shutdown_connections 2024-09-19T09:12:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.102+0000 7f276ea04740 1 --2- 172.21.15.120:0/2306071457 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643cf3a87d0 0x5643cf3b8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.102+0000 7f276ea04740 1 --2- 172.21.15.120:0/2306071457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cf3a36f0 0x5643cf3a25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.102+0000 7f276ea04740 1 --2- 172.21.15.120:0/2306071457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643cf3a2db0 0x5643cf3a20b0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.102+0000 7f276ea04740 1 -- 172.21.15.120:0/2306071457 >> 172.21.15.120:0/2306071457 conn(0x5643cf3ab820 msgr2=0x5643cf3a19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.103+0000 7f276ea04740 1 -- 172.21.15.120:0/2306071457 shutdown_connections 2024-09-19T09:12:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.103+0000 7f276ea04740 1 -- 172.21.15.120:0/2306071457 wait complete. 2024-09-19T09:12:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.103+0000 7f276ea04740 1 -- start start 2024-09-19T09:12:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.103+0000 7f276ea04740 1 Processor -- start 2024-09-19T09:12:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.103+0000 7f276ea04740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cf3a2db0 0x5643cf3b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.103+0000 7f276ea04740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643cf3a36f0 0x5643cf3b57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.103+0000 7f276ea04740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643cf3a87d0 0x5643cf3b5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.104+0000 7f276ea04740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f27641a8270 con 0x5643cf3a36f0 2024-09-19T09:12:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.104+0000 7f276ea04740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f27641a7ba0 con 0x5643cf3a87d0 2024-09-19T09:12:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.104+0000 7f276ea04740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f27641a4dc0 con 0x5643cf3a2db0 2024-09-19T09:12:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.104+0000 7f276ea04740 10 client.0 _ll_register_callbacks cb 0x7f2754009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.104+0000 7f2769128640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643cf3a36f0 0x5643cf3b57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.104+0000 7f2769929640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cf3a2db0 0x5643cf3b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.104+0000 7f276a12a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643cf3a87d0 0x5643cf3b5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.104+0000 7f2769929640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cf3a2db0 0x5643cf3b8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:35344/0 (socket says 172.21.15.120:35344) 2024-09-19T09:12:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.104+0000 7f2769929640 1 -- 172.21.15.120:0/922209844 learned_addr learned my addr 172.21.15.120:0/922209844 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.104+0000 7f2769128640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643cf3a36f0 0x5643cf3b57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46828/0 (socket says 172.21.15.120:46828) 2024-09-19T09:12:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.104+0000 7f2769929640 1 -- 172.21.15.120:0/922209844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643cf3a87d0 msgr2=0x5643cf3b5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.104+0000 7f2769929640 1 --2- 172.21.15.120:0/922209844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643cf3a87d0 0x5643cf3b5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.104+0000 7f2769929640 1 -- 172.21.15.120:0/922209844 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643cf3a36f0 msgr2=0x5643cf3b57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.104+0000 7f2769929640 1 --2- 172.21.15.120:0/922209844 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643cf3a36f0 0x5643cf3b57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.104+0000 7f2769929640 1 -- 172.21.15.120:0/922209844 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643cf427130 con 0x5643cf3a2db0 2024-09-19T09:12:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.104+0000 7f2769128640 1 --2- 172.21.15.120:0/922209844 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643cf3a36f0 0x5643cf3b57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.104+0000 7f2769929640 1 --2- 172.21.15.120:0/922209844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cf3a2db0 0x5643cf3b8ce0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f276000b5b0 tx=0x7f276000bc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a0d8299d1bd9f6d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.104+0000 7f274d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:12:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.104+0000 7f274d7fa640 1 -- 172.21.15.120:0/922209844 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2760003a70 con 0x5643cf3a2db0 2024-09-19T09:12:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.104+0000 7f276ea04740 1 -- 172.21.15.120:0/922209844 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f276003eca0 con 0x5643cf3a2db0 2024-09-19T09:12:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.104+0000 7f274d7fa640 1 -- 172.21.15.120:0/922209844 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f2760003c10 con 0x5643cf3a2db0 2024-09-19T09:12:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.105+0000 7f274d7fa640 1 -- 172.21.15.120:0/922209844 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2760003e50 con 0x5643cf3a2db0 2024-09-19T09:12:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.105+0000 7f276ea04740 10 client.27631 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.105+0000 7f276ea04740 1 -- 172.21.15.120:0/922209844 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5643cf4283c0 con 0x5643cf3a2db0 2024-09-19T09:12:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.105+0000 7f274d7fa640 1 -- 172.21.15.120:0/922209844 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f276003eca0 con 0x5643cf3a2db0 2024-09-19T09:12:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.105+0000 7f274d7fa640 1 -- 172.21.15.120:0/922209844 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f27600337e0 con 0x5643cf3a2db0 2024-09-19T09:12:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.105+0000 7f276ea04740 10 client.27631 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.105+0000 7f276ea04740 10 client.27631 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:12:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.105+0000 7f276ea04740 20 client.27631 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.105+0000 7f276ea04740 1 --2- 172.21.15.120:0/922209844 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5643cf428bc0 0x5643cf42afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.105+0000 7f276ea04740 4 client.27631 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:12:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.105+0000 7f276ea04740 1 -- 172.21.15.120:0/922209844 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5643cf312d70 con 0x5643cf428bc0 2024-09-19T09:12:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.105+0000 7f2769128640 1 --2- 172.21.15.120:0/922209844 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5643cf428bc0 0x5643cf42afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.106+0000 7f274d7fa640 10 client.27631 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.106+0000 7f2769128640 1 --2- 172.21.15.120:0/922209844 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5643cf428bc0 0x5643cf42afe0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8ea8546560012b0 server_cookie=30d0eb5884d30674 in_seq=0 out_seq=0 2024-09-19T09:12:05.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:04 smithi196 ceph-mon[35150]: pgmap v619: 97 pgs: 97 active+clean; 52 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 429 op/s 2024-09-19T09:12:05.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:04 smithi153 ceph-mon[35071]: pgmap v619: 97 pgs: 97 active+clean; 52 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 429 op/s 2024-09-19T09:12:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.282+0000 7f274d7fa640 1 -- 172.21.15.120:0/922209844 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5643cf312d70 con 0x5643cf428bc0 2024-09-19T09:12:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.282+0000 7f274d7fa640 10 client.27631 handle_command_reply: tid=0 2024-09-19T09:12:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.283+0000 7f276ea04740 1 client.27631 shutdown 2024-09-19T09:12:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.285+0000 7f276ea04740 1 -- 172.21.15.120:0/922209844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cf3a2db0 msgr2=0x5643cf3b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.285+0000 7f276ea04740 1 --2- 172.21.15.120:0/922209844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cf3a2db0 0x5643cf3b8ce0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f276000b5b0 tx=0x7f276000bc90 comp rx=0 tx=0).stop 2024-09-19T09:12:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.285+0000 7f276ea04740 1 -- 172.21.15.120:0/922209844 shutdown_connections 2024-09-19T09:12:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.285+0000 7f276ea04740 1 --2- 172.21.15.120:0/922209844 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5643cf428bc0 0x5643cf42afe0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.285+0000 7f276ea04740 1 --2- 172.21.15.120:0/922209844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643cf3a87d0 0x5643cf3b5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.285+0000 7f276ea04740 1 --2- 172.21.15.120:0/922209844 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643cf3a36f0 0x5643cf3b57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.285+0000 7f276ea04740 1 --2- 172.21.15.120:0/922209844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643cf3a2db0 0x5643cf3b8ce0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.285+0000 7f276ea04740 1 -- 172.21.15.120:0/922209844 >> 172.21.15.120:0/922209844 conn(0x5643cf3ab820 msgr2=0x5643cf3a19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.285+0000 7f274d7fa640 0 client.27631 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.285+0000 7f276ea04740 1 -- 172.21.15.120:0/922209844 shutdown_connections 2024-09-19T09:12:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.285+0000 7f276ea04740 1 -- 172.21.15.120:0/922209844 wait complete. 2024-09-19T09:12:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.286+0000 7f276ea04740 20 client.27631 trim_cache size 0 max 16384 2024-09-19T09:12:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:12:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:12:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:12:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:12:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.289+0000 7f276bb8d640 1 -- 172.21.15.120:0/1099740185 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f27500a32f0 msgr2=0x7f27500a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.289+0000 7f276bb8d640 1 --2- 172.21.15.120:0/1099740185 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f27500a32f0 0x7f27500a57b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f275c005fd0 tx=0x7f275c009500 comp rx=0 tx=0).stop 2024-09-19T09:12:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.289+0000 7f276bb8d640 1 -- 172.21.15.120:0/1099740185 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2764102d50 msgr2=0x7f27640745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.289+0000 7f276bb8d640 1 --2- 172.21.15.120:0/1099740185 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2764102d50 0x7f27640745e0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f2754009230 tx=0x7f2754031b60 comp rx=0 tx=0).stop 2024-09-19T09:12:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.290+0000 7f276bb8d640 1 -- 172.21.15.120:0/1099740185 shutdown_connections 2024-09-19T09:12:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.290+0000 7f276bb8d640 1 --2- 172.21.15.120:0/1099740185 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f27500a32f0 0x7f27500a57b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.290+0000 7f276bb8d640 1 --2- 172.21.15.120:0/1099740185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2764103710 0x7f2764074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.290+0000 7f276bb8d640 1 --2- 172.21.15.120:0/1099740185 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2764102d50 0x7f27640745e0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.290+0000 7f276bb8d640 1 --2- 172.21.15.120:0/1099740185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2764101b60 0x7f2764075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.290+0000 7f276bb8d640 1 -- 172.21.15.120:0/1099740185 >> 172.21.15.120:0/1099740185 conn(0x7f27640fd2e0 msgr2=0x7f2764106120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.290+0000 7f276bb8d640 1 -- 172.21.15.120:0/1099740185 shutdown_connections 2024-09-19T09:12:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:05.290+0000 7f276bb8d640 1 -- 172.21.15.120:0/1099740185 wait complete. 2024-09-19T09:12:05.388 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:12:05.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.970+0000 7f9f4b3b7640 1 -- 172.21.15.120:0/1506663230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f44103740 msgr2=0x7f9f44107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:05.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.970+0000 7f9f4b3b7640 1 --2- 172.21.15.120:0/1506663230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f44103740 0x7f9f44107b80 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f9f34009930 tx=0x7f9f3402a420 comp rx=0 tx=0).stop 2024-09-19T09:12:05.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.970+0000 7f9f4b3b7640 1 -- 172.21.15.120:0/1506663230 shutdown_connections 2024-09-19T09:12:05.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.970+0000 7f9f4b3b7640 1 --2- 172.21.15.120:0/1506663230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f44108290 0x7f9f4410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.970+0000 7f9f4b3b7640 1 --2- 172.21.15.120:0/1506663230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f44103740 0x7f9f44107b80 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.970+0000 7f9f4b3b7640 1 --2- 172.21.15.120:0/1506663230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f44102d90 0x7f9f44103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.970+0000 7f9f4b3b7640 1 -- 172.21.15.120:0/1506663230 >> 172.21.15.120:0/1506663230 conn(0x7f9f440fc7d0 msgr2=0x7f9f440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:05.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.970+0000 7f9f4b3b7640 1 -- 172.21.15.120:0/1506663230 shutdown_connections 2024-09-19T09:12:05.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.970+0000 7f9f4b3b7640 1 -- 172.21.15.120:0/1506663230 wait complete. 2024-09-19T09:12:05.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.971+0000 7f9f4b3b7640 1 Processor -- start 2024-09-19T09:12:05.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.971+0000 7f9f4b3b7640 1 -- start start 2024-09-19T09:12:05.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.971+0000 7f9f4b3b7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f44102d90 0x7f9f44198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.971+0000 7f9f4b3b7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f44103740 0x7f9f44198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.971+0000 7f9f4b3b7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f44108290 0x7f9f4419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.971+0000 7f9f4b3b7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9f44112090 con 0x7f9f44102d90 2024-09-19T09:12:05.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.971+0000 7f9f4b3b7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9f44111f10 con 0x7f9f44108290 2024-09-19T09:12:05.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.971+0000 7f9f4b3b7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9f44112210 con 0x7f9f44103740 2024-09-19T09:12:05.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.971+0000 7f9f49153640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f44102d90 0x7f9f44198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.971+0000 7f9f49153640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f44102d90 0x7f9f44198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46852/0 (socket says 172.21.15.120:46852) 2024-09-19T09:12:05.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.971+0000 7f9f49153640 1 -- 172.21.15.120:0/3265657481 learned_addr learned my addr 172.21.15.120:0/3265657481 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:05.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.971+0000 7f9f48952640 1 --2- 172.21.15.120:0/3265657481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f44103740 0x7f9f44198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.971+0000 7f9f49954640 1 --2- 172.21.15.120:0/3265657481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f44108290 0x7f9f4419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.971+0000 7f9f49153640 1 -- 172.21.15.120:0/3265657481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f44103740 msgr2=0x7f9f44198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:05.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.971+0000 7f9f49153640 1 --2- 172.21.15.120:0/3265657481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f44103740 0x7f9f44198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.971+0000 7f9f49153640 1 -- 172.21.15.120:0/3265657481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f44108290 msgr2=0x7f9f4419ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:05.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.971+0000 7f9f49153640 1 --2- 172.21.15.120:0/3265657481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f44108290 0x7f9f4419ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.971+0000 7f9f49153640 1 -- 172.21.15.120:0/3265657481 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4419d540 con 0x7f9f44102d90 2024-09-19T09:12:05.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.972+0000 7f9f49153640 1 --2- 172.21.15.120:0/3265657481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f44102d90 0x7f9f44198590 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000e9f0 tx=0x7f9f4000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:05.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.972+0000 7f9f3a7fc640 1 -- 172.21.15.120:0/3265657481 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4000cd80 con 0x7f9f44102d90 2024-09-19T09:12:05.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.972+0000 7f9f4b3b7640 1 -- 172.21.15.120:0/3265657481 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f4419d830 con 0x7f9f44102d90 2024-09-19T09:12:05.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.972+0000 7f9f3a7fc640 1 -- 172.21.15.120:0/3265657481 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f40010910 con 0x7f9f44102d90 2024-09-19T09:12:05.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.972+0000 7f9f3a7fc640 1 -- 172.21.15.120:0/3265657481 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f400188f0 con 0x7f9f44102d90 2024-09-19T09:12:05.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.972+0000 7f9f4b3b7640 1 -- 172.21.15.120:0/3265657481 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f441a5110 con 0x7f9f44102d90 2024-09-19T09:12:05.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.974+0000 7f9f3a7fc640 1 -- 172.21.15.120:0/3265657481 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f9f40010ac0 con 0x7f9f44102d90 2024-09-19T09:12:05.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.974+0000 7f9f4d790740 1 Processor -- start 2024-09-19T09:12:05.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.974+0000 7f9f4d790740 1 -- start start 2024-09-19T09:12:05.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.975+0000 7f9f4d790740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770798db40 0x55770798c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.975+0000 7f9f4d790740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770798c5d0 0x55770798ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.975+0000 7f9f4d790740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557707997370 0x557707999760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.975+0000 7f9f4d790740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5577078f22f0 con 0x55770798db40 2024-09-19T09:12:05.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.975+0000 7f9f4d790740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557707899cf0 con 0x557707997370 2024-09-19T09:12:05.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.975+0000 7f9f4d790740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5577078ee2b0 con 0x55770798c5d0 2024-09-19T09:12:05.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.975+0000 7f9f49954640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770798db40 0x55770798c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.975+0000 7f9f49954640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770798db40 0x55770798c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46860/0 (socket says 172.21.15.120:46860) 2024-09-19T09:12:05.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.975+0000 7f9f49954640 1 -- 172.21.15.120:0/651467351 learned_addr learned my addr 172.21.15.120:0/651467351 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:05.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.975+0000 7f9f48952640 1 --2- 172.21.15.120:0/651467351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557707997370 0x557707999760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.975+0000 7f9f49153640 1 --2- 172.21.15.120:0/651467351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770798c5d0 0x55770798ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.975+0000 7f9f3a7fc640 1 --2- 172.21.15.120:0/3265657481 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9f300a3240 0x7f9f300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.975+0000 7f9f49954640 1 -- 172.21.15.120:0/651467351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770798c5d0 msgr2=0x55770798ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:05.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.975+0000 7f9f49954640 1 --2- 172.21.15.120:0/651467351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770798c5d0 0x55770798ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.975+0000 7f9f49954640 1 -- 172.21.15.120:0/651467351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557707997370 msgr2=0x557707999760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:05.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.975+0000 7f9f49954640 1 --2- 172.21.15.120:0/651467351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557707997370 0x557707999760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.975+0000 7f9f49954640 1 -- 172.21.15.120:0/651467351 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55770787e3a0 con 0x55770798db40 2024-09-19T09:12:05.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.975+0000 7f9f49954640 1 --2- 172.21.15.120:0/3265657481 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9f300a3240 0x7f9f300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.975+0000 7f9f3a7fc640 1 -- 172.21.15.120:0/3265657481 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f9f400cfca0 con 0x7f9f44102d90 2024-09-19T09:12:05.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.975+0000 7f9f49954640 1 --2- 172.21.15.120:0/651467351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770798db40 0x55770798c090 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0072e0 tx=0x7f9f3c00f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a14572732ab1c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:05.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.976+0000 7f9f2f7fe640 1 -- 172.21.15.120:0/651467351 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c019760 con 0x55770798db40 2024-09-19T09:12:05.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.976+0000 7f9f2f7fe640 1 -- 172.21.15.120:0/651467351 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f3c019de0 con 0x55770798db40 2024-09-19T09:12:05.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.976+0000 7f9f2f7fe640 1 -- 172.21.15.120:0/651467351 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c010d70 con 0x55770798db40 2024-09-19T09:12:05.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.976+0000 7f9f49954640 1 --2- 172.21.15.120:0/3265657481 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9f300a3240 0x7f9f300a5700 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c020000 tx=0x7f9f3c01d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:05.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.976+0000 7f9f4d790740 1 -- 172.21.15.120:0/651467351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770798db40 msgr2=0x55770798c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:05.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.976+0000 7f9f4d790740 1 --2- 172.21.15.120:0/651467351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770798db40 0x55770798c090 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0072e0 tx=0x7f9f3c00f850 comp rx=0 tx=0).stop 2024-09-19T09:12:05.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.976+0000 7f9f4d790740 1 -- 172.21.15.120:0/651467351 shutdown_connections 2024-09-19T09:12:05.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.976+0000 7f9f4d790740 1 --2- 172.21.15.120:0/651467351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557707997370 0x557707999760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.976+0000 7f9f4d790740 1 --2- 172.21.15.120:0/651467351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770798c5d0 0x55770798ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.976+0000 7f9f4d790740 1 --2- 172.21.15.120:0/651467351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770798db40 0x55770798c090 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.976+0000 7f9f4d790740 1 -- 172.21.15.120:0/651467351 >> 172.21.15.120:0/651467351 conn(0x557707955310 msgr2=0x557707988c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:05.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.977+0000 7f9f4d790740 1 -- 172.21.15.120:0/651467351 shutdown_connections 2024-09-19T09:12:05.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.977+0000 7f9f4d790740 1 -- 172.21.15.120:0/651467351 wait complete. 2024-09-19T09:12:05.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.977+0000 7f9f4d790740 1 -- start start 2024-09-19T09:12:05.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.977+0000 7f9f4d790740 1 Processor -- start 2024-09-19T09:12:05.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.977+0000 7f9f4d790740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770798c5d0 0x557707996d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.977+0000 7f9f4d790740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770798db40 0x557707994020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.977+0000 7f9f4d790740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557707997370 0x557707994610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.977+0000 7f9f4d790740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55770799a070 con 0x55770798c5d0 2024-09-19T09:12:05.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.977+0000 7f9f4d790740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5577078ad060 con 0x55770798db40 2024-09-19T09:12:05.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.977+0000 7f9f4d790740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55770799a1f0 con 0x557707997370 2024-09-19T09:12:05.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.977+0000 7f9f4d790740 10 client.0 _ll_register_callbacks cb 0x5577076008c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:05.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.978+0000 7f9f49153640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770798c5d0 0x557707996d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.978+0000 7f9f49153640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770798c5d0 0x557707996d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46868/0 (socket says 172.21.15.120:46868) 2024-09-19T09:12:05.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.978+0000 7f9f49153640 1 -- 172.21.15.120:0/2353312612 learned_addr learned my addr 172.21.15.120:0/2353312612 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:05.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.978+0000 7f9f48952640 1 --2- 172.21.15.120:0/2353312612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770798db40 0x557707994020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.978+0000 7f9f49954640 1 --2- 172.21.15.120:0/2353312612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557707997370 0x557707994610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.978+0000 7f9f49153640 1 -- 172.21.15.120:0/2353312612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557707997370 msgr2=0x557707994610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:05.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.978+0000 7f9f49153640 1 --2- 172.21.15.120:0/2353312612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557707997370 0x557707994610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.978+0000 7f9f49153640 1 -- 172.21.15.120:0/2353312612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770798db40 msgr2=0x557707994020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:05.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.978+0000 7f9f49153640 1 --2- 172.21.15.120:0/2353312612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770798db40 0x557707994020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.978+0000 7f9f49153640 1 -- 172.21.15.120:0/2353312612 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577076633c0 con 0x55770798c5d0 2024-09-19T09:12:05.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.978+0000 7f9f49153640 1 --2- 172.21.15.120:0/2353312612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770798c5d0 0x557707996d70 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f9f400cd7a0 tx=0x7f9f4009beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8552de9df20cde9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:05.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.978+0000 7f9f2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:05.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.978+0000 7f9f2cff9640 1 -- 172.21.15.120:0/2353312612 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f400a0670 con 0x55770798c5d0 2024-09-19T09:12:05.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.978+0000 7f9f2cff9640 1 -- 172.21.15.120:0/2353312612 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f9f400bc0d0 con 0x55770798c5d0 2024-09-19T09:12:05.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.978+0000 7f9f4d790740 1 -- 172.21.15.120:0/2353312612 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f3c019de0 con 0x55770798c5d0 2024-09-19T09:12:05.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.978+0000 7f9f2cff9640 1 -- 172.21.15.120:0/2353312612 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9f400a4f30 con 0x55770798c5d0 2024-09-19T09:12:05.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.978+0000 7f9f4d790740 10 client.18663 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:05.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.978+0000 7f9f4d790740 1 -- 172.21.15.120:0/2353312612 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5577079a5bc0 con 0x55770798c5d0 2024-09-19T09:12:05.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.979+0000 7f9f2cff9640 1 -- 172.21.15.120:0/2353312612 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f3c019de0 con 0x55770798c5d0 2024-09-19T09:12:05.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.979+0000 7f9f2cff9640 1 -- 172.21.15.120:0/2353312612 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f9f4009b930 con 0x55770798c5d0 2024-09-19T09:12:05.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.979+0000 7f9f4d790740 10 client.18663 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:05.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.979+0000 7f9f4d790740 10 client.18663 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:12:05.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.979+0000 7f9f4d790740 20 client.18663 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:05.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.979+0000 7f9f4d790740 1 --2- 172.21.15.120:0/2353312612 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5577079a7080 0x5577079a9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.979+0000 7f9f4d790740 4 client.18663 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:12:05.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.979+0000 7f9f4d790740 1 -- 172.21.15.120:0/2353312612 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577078ff860 con 0x5577079a7080 2024-09-19T09:12:05.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.979+0000 7f9f48952640 1 --2- 172.21.15.120:0/2353312612 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5577079a7080 0x5577079a9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.979+0000 7f9f2cff9640 10 client.18663 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:12:05.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.979+0000 7f9f48952640 1 --2- 172.21.15.120:0/2353312612 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5577079a7080 0x5577079a9470 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be93f2014b9d899c server_cookie=8574fefef2508726 in_seq=0 out_seq=0 2024-09-19T09:12:05.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.980+0000 7f9f2cff9640 1 -- 172.21.15.120:0/2353312612 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5577078ff860 con 0x5577079a7080 2024-09-19T09:12:05.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.980+0000 7f9f2cff9640 10 client.18663 handle_command_reply: tid=0 2024-09-19T09:12:05.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.980+0000 7f9f4d790740 1 client.18663 shutdown 2024-09-19T09:12:05.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.980+0000 7f9f4d790740 1 -- 172.21.15.120:0/2353312612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770798c5d0 msgr2=0x557707996d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:05.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.980+0000 7f9f4d790740 1 --2- 172.21.15.120:0/2353312612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770798c5d0 0x557707996d70 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f9f400cd7a0 tx=0x7f9f4009beb0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.980+0000 7f9f4d790740 1 -- 172.21.15.120:0/2353312612 shutdown_connections 2024-09-19T09:12:05.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.980+0000 7f9f4d790740 1 --2- 172.21.15.120:0/2353312612 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5577079a7080 0x5577079a9470 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.980+0000 7f9f4d790740 1 --2- 172.21.15.120:0/2353312612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557707997370 0x557707994610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.980+0000 7f9f4d790740 1 --2- 172.21.15.120:0/2353312612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770798db40 0x557707994020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.980+0000 7f9f4d790740 1 --2- 172.21.15.120:0/2353312612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770798c5d0 0x557707996d70 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.980+0000 7f9f4d790740 1 -- 172.21.15.120:0/2353312612 >> 172.21.15.120:0/2353312612 conn(0x557707955310 msgr2=0x55770798e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:05.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.980+0000 7f9f2cff9640 0 client.18663 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:12:05.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.980+0000 7f9f4d790740 1 -- 172.21.15.120:0/2353312612 shutdown_connections 2024-09-19T09:12:05.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.981+0000 7f9f4d790740 1 -- 172.21.15.120:0/2353312612 wait complete. 2024-09-19T09:12:05.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.981+0000 7f9f4d790740 20 client.18663 trim_cache size 0 max 16384 2024-09-19T09:12:05.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.994+0000 7f9f4d790740 1 Processor -- start 2024-09-19T09:12:05.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.994+0000 7f9f4d790740 1 -- start start 2024-09-19T09:12:05.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.994+0000 7f9f4d790740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770798c5d0 0x5577079a3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.994+0000 7f9f4d790740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770798db40 0x557707995f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.994+0000 7f9f4d790740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557707997370 0x5577079964d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:05.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.994+0000 7f9f4d790740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9f441ad590 con 0x557707997370 2024-09-19T09:12:05.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.994+0000 7f9f4d790740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9f441af0d0 con 0x55770798c5d0 2024-09-19T09:12:05.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.994+0000 7f9f4d790740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9f441adc60 con 0x55770798db40 2024-09-19T09:12:05.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.994+0000 7f9f49954640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557707997370 0x5577079964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.994+0000 7f9f49954640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557707997370 0x5577079964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46880/0 (socket says 172.21.15.120:46880) 2024-09-19T09:12:05.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.994+0000 7f9f49954640 1 -- 172.21.15.120:0/4090278989 learned_addr learned my addr 172.21.15.120:0/4090278989 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:05.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.994+0000 7f9f49153640 1 --2- 172.21.15.120:0/4090278989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770798c5d0 0x5577079a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.995+0000 7f9f48952640 1 --2- 172.21.15.120:0/4090278989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770798db40 0x557707995f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:05.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.995+0000 7f9f49954640 1 -- 172.21.15.120:0/4090278989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770798db40 msgr2=0x557707995f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:05.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.995+0000 7f9f49954640 1 --2- 172.21.15.120:0/4090278989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770798db40 0x557707995f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.995+0000 7f9f49954640 1 -- 172.21.15.120:0/4090278989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770798c5d0 msgr2=0x5577079a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:05.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.995+0000 7f9f49954640 1 --2- 172.21.15.120:0/4090278989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770798c5d0 0x5577079a3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:05.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.995+0000 7f9f49954640 1 -- 172.21.15.120:0/4090278989 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577079a9ad0 con 0x557707997370 2024-09-19T09:12:05.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.995+0000 7f9f48952640 1 --2- 172.21.15.120:0/4090278989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770798db40 0x557707995f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:12:05.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.995+0000 7f9f49954640 1 --2- 172.21.15.120:0/4090278989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557707997370 0x5577079964d0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c010a40 tx=0x7f9f3c005f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=782f0a814ffb1175 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:05.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.995+0000 7f9f2effd640 1 -- 172.21.15.120:0/4090278989 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c004980 con 0x557707997370 2024-09-19T09:12:05.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.995+0000 7f9f2effd640 1 -- 172.21.15.120:0/4090278989 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f3c012250 con 0x557707997370 2024-09-19T09:12:05.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.995+0000 7f9f2effd640 1 -- 172.21.15.120:0/4090278989 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c019940 con 0x557707997370 2024-09-19T09:12:06.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.996+0000 7f9f4d790740 1 -- 172.21.15.120:0/4090278989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557707997370 msgr2=0x5577079964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:06.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.996+0000 7f9f4d790740 1 --2- 172.21.15.120:0/4090278989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557707997370 0x5577079964d0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c010a40 tx=0x7f9f3c005f20 comp rx=0 tx=0).stop 2024-09-19T09:12:06.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.996+0000 7f9f4d790740 1 -- 172.21.15.120:0/4090278989 shutdown_connections 2024-09-19T09:12:06.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.996+0000 7f9f4d790740 1 --2- 172.21.15.120:0/4090278989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557707997370 0x5577079964d0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:06.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.996+0000 7f9f4d790740 1 --2- 172.21.15.120:0/4090278989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770798db40 0x557707995f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:06.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.996+0000 7f9f4d790740 1 --2- 172.21.15.120:0/4090278989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770798c5d0 0x5577079a3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:06.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.996+0000 7f9f4d790740 1 -- 172.21.15.120:0/4090278989 >> 172.21.15.120:0/4090278989 conn(0x557707955310 msgr2=0x557707995880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:06.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.996+0000 7f9f4d790740 1 -- 172.21.15.120:0/4090278989 shutdown_connections 2024-09-19T09:12:06.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.996+0000 7f9f4d790740 1 -- 172.21.15.120:0/4090278989 wait complete. 2024-09-19T09:12:06.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.996+0000 7f9f4d790740 1 -- start start 2024-09-19T09:12:06.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.996+0000 7f9f4d790740 1 Processor -- start 2024-09-19T09:12:06.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.997+0000 7f9f4d790740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770798c5d0 0x55770799f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:06.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.997+0000 7f9f4d790740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770798db40 0x55770799fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:06.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.997+0000 7f9f4d790740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557707997370 0x55770799c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:06.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.997+0000 7f9f4d790740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9f441aea00 con 0x55770798db40 2024-09-19T09:12:06.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.997+0000 7f9f4d790740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9f441ae330 con 0x55770798c5d0 2024-09-19T09:12:06.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.997+0000 7f9f4d790740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9f441ab550 con 0x557707997370 2024-09-19T09:12:06.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.997+0000 7f9f4d790740 10 client.0 _ll_register_callbacks cb 0x7f9f4009c230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:06.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.997+0000 7f9f48952640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770798db40 0x55770799fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:06.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.997+0000 7f9f49954640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557707997370 0x55770799c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:06.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.997+0000 7f9f49153640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770798c5d0 0x55770799f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:06.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.997+0000 7f9f49954640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557707997370 0x55770799c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:35418/0 (socket says 172.21.15.120:35418) 2024-09-19T09:12:06.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.997+0000 7f9f48952640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770798db40 0x55770799fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46884/0 (socket says 172.21.15.120:46884) 2024-09-19T09:12:06.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.997+0000 7f9f49954640 1 -- 172.21.15.120:0/560997075 learned_addr learned my addr 172.21.15.120:0/560997075 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:06.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.997+0000 7f9f49954640 1 -- 172.21.15.120:0/560997075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770798c5d0 msgr2=0x55770799f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:06.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.997+0000 7f9f49954640 1 --2- 172.21.15.120:0/560997075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770798c5d0 0x55770799f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:06.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.997+0000 7f9f49954640 1 -- 172.21.15.120:0/560997075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770798db40 msgr2=0x55770799fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:06.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.997+0000 7f9f49954640 1 --2- 172.21.15.120:0/560997075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770798db40 0x55770799fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:06.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.997+0000 7f9f49954640 1 -- 172.21.15.120:0/560997075 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55770799caf0 con 0x557707997370 2024-09-19T09:12:06.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.997+0000 7f9f48952640 1 --2- 172.21.15.120:0/560997075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770798db40 0x55770799fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:06.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.998+0000 7f9f49954640 1 --2- 172.21.15.120:0/560997075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557707997370 0x55770799c450 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c007600 tx=0x7f9f3c010e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77b5bb909984ec4e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:06.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.998+0000 7f9f2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:12:06.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.998+0000 7f9f2cff9640 1 -- 172.21.15.120:0/560997075 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c03d060 con 0x557707997370 2024-09-19T09:12:06.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.998+0000 7f9f2cff9640 1 -- 172.21.15.120:0/560997075 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f9f3c010640 con 0x557707997370 2024-09-19T09:12:06.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.998+0000 7f9f4d790740 1 -- 172.21.15.120:0/560997075 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f3c012250 con 0x557707997370 2024-09-19T09:12:06.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.998+0000 7f9f2cff9640 1 -- 172.21.15.120:0/560997075 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9f3c004980 con 0x557707997370 2024-09-19T09:12:06.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.998+0000 7f9f4d790740 10 client.27646 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:06.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.998+0000 7f9f4d790740 1 -- 172.21.15.120:0/560997075 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557707a12660 con 0x557707997370 2024-09-19T09:12:06.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.998+0000 7f9f2cff9640 1 -- 172.21.15.120:0/560997075 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f3c012250 con 0x557707997370 2024-09-19T09:12:06.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.998+0000 7f9f2cff9640 1 -- 172.21.15.120:0/560997075 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f9f3c042070 con 0x557707997370 2024-09-19T09:12:06.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.998+0000 7f9f4d790740 10 client.27646 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:06.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.998+0000 7f9f4d790740 10 client.27646 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:12:06.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.998+0000 7f9f4d790740 20 client.27646 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:06.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.999+0000 7f9f4d790740 1 --2- 172.21.15.120:0/560997075 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557707a12ad0 0x557707a14ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:06.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.999+0000 7f9f4d790740 4 client.27646 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:12:06.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.999+0000 7f9f4d790740 1 -- 172.21.15.120:0/560997075 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5577078f9d70 con 0x557707a12ad0 2024-09-19T09:12:06.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.999+0000 7f9f49153640 1 --2- 172.21.15.120:0/560997075 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557707a12ad0 0x557707a14ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:06.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.999+0000 7f9f2cff9640 10 client.27646 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:12:06.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.999+0000 7f9f49153640 1 --2- 172.21.15.120:0/560997075 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557707a12ad0 0x557707a14ec0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ebdb27e452c7399e server_cookie=1b2297dc8bb618b9 in_seq=0 out_seq=0 2024-09-19T09:12:06.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.999+0000 7f9f2cff9640 1 -- 172.21.15.120:0/560997075 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5577078f9d70 con 0x557707a12ad0 2024-09-19T09:12:06.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.999+0000 7f9f2cff9640 10 client.27646 handle_command_reply: tid=0 2024-09-19T09:12:06.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.999+0000 7f9f4d790740 1 client.27646 shutdown 2024-09-19T09:12:06.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.999+0000 7f9f4d790740 1 -- 172.21.15.120:0/560997075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557707997370 msgr2=0x55770799c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:06.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.999+0000 7f9f4d790740 1 --2- 172.21.15.120:0/560997075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557707997370 0x55770799c450 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c007600 tx=0x7f9f3c010e20 comp rx=0 tx=0).stop 2024-09-19T09:12:06.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.999+0000 7f9f4d790740 1 -- 172.21.15.120:0/560997075 shutdown_connections 2024-09-19T09:12:06.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.999+0000 7f9f4d790740 1 --2- 172.21.15.120:0/560997075 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557707a12ad0 0x557707a14ec0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:06.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.999+0000 7f9f4d790740 1 --2- 172.21.15.120:0/560997075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557707997370 0x55770799c450 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:06.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.999+0000 7f9f4d790740 1 --2- 172.21.15.120:0/560997075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770798db40 0x55770799fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:06.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.999+0000 7f9f4d790740 1 --2- 172.21.15.120:0/560997075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770798c5d0 0x55770799f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:06.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.999+0000 7f9f4d790740 1 -- 172.21.15.120:0/560997075 >> 172.21.15.120:0/560997075 conn(0x557707955310 msgr2=0x5577079960a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:06.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:05.999+0000 7f9f2cff9640 0 client.27646 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:12:06.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:06.000+0000 7f9f4d790740 1 -- 172.21.15.120:0/560997075 shutdown_connections 2024-09-19T09:12:06.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:06.000+0000 7f9f4d790740 1 -- 172.21.15.120:0/560997075 wait complete. 2024-09-19T09:12:06.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:06.000+0000 7f9f4d790740 20 client.27646 trim_cache size 0 max 16384 2024-09-19T09:12:06.007 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:12:06.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:06.000+0000 7f9f4b3b7640 1 -- 172.21.15.120:0/3265657481 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9f300a3240 msgr2=0x7f9f300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:06.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:06.000+0000 7f9f4b3b7640 1 --2- 172.21.15.120:0/3265657481 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9f300a3240 0x7f9f300a5700 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c020000 tx=0x7f9f3c01d000 comp rx=0 tx=0).stop 2024-09-19T09:12:06.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:06.000+0000 7f9f4b3b7640 1 -- 172.21.15.120:0/3265657481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f44102d90 msgr2=0x7f9f44198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:06.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:06.000+0000 7f9f4b3b7640 1 --2- 172.21.15.120:0/3265657481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f44102d90 0x7f9f44198590 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000e9f0 tx=0x7f9f4000eeb0 comp rx=0 tx=0).stop 2024-09-19T09:12:06.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:06.000+0000 7f9f4b3b7640 1 -- 172.21.15.120:0/3265657481 shutdown_connections 2024-09-19T09:12:06.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:06.000+0000 7f9f4b3b7640 1 --2- 172.21.15.120:0/3265657481 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9f300a3240 0x7f9f300a5700 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:06.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:06.000+0000 7f9f4b3b7640 1 --2- 172.21.15.120:0/3265657481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f44108290 0x7f9f4419ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:06.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:06.000+0000 7f9f4b3b7640 1 --2- 172.21.15.120:0/3265657481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f44103740 0x7f9f44198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:06.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:06.000+0000 7f9f4b3b7640 1 --2- 172.21.15.120:0/3265657481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f44102d90 0x7f9f44198590 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:06.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:06.000+0000 7f9f4b3b7640 1 -- 172.21.15.120:0/3265657481 >> 172.21.15.120:0/3265657481 conn(0x7f9f440fc7d0 msgr2=0x7f9f44106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:06.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:06.000+0000 7f9f4b3b7640 1 -- 172.21.15.120:0/3265657481 shutdown_connections 2024-09-19T09:12:06.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:06.000+0000 7f9f4b3b7640 1 -- 172.21.15.120:0/3265657481 wait complete. 2024-09-19T09:12:06.065 INFO:tasks.workunit.client.0.smithi120.stdout: 1 877463 9.75 MB/sec execute 361 sec latency 108.604 ms 2024-09-19T09:12:06.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:05 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:12:06.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:05 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:12:06.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:05 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:12:06.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:05 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:12:06.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:05 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:12:06.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:05 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:12:06.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:05 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:12:06.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:05 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:12:06.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:05 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:12:06.970 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:12:07.065 INFO:tasks.workunit.client.0.smithi120.stdout: 1 880593 9.77 MB/sec execute 362 sec latency 17.493 ms 2024-09-19T09:12:07.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:06 smithi196 ceph-mon[35150]: pgmap v620: 97 pgs: 97 active+clean; 52 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 355 op/s 2024-09-19T09:12:07.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:06 smithi153 ceph-mon[35071]: pgmap v620: 97 pgs: 97 active+clean; 52 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 355 op/s 2024-09-19T09:12:07.227 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:12:07.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:06 smithi120 ceph-mon[31150]: pgmap v620: 97 pgs: 97 active+clean; 52 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 355 op/s 2024-09-19T09:12:07.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.723+0000 7fd44c592640 1 -- 172.21.15.120:0/3691310440 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4441027a0 msgr2=0x7fd4441067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:07.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.723+0000 7fd44c592640 1 --2- 172.21.15.120:0/3691310440 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4441027a0 0x7fd4441067f0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fd434009900 tx=0x7fd43402a410 comp rx=0 tx=0).stop 2024-09-19T09:12:07.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.723+0000 7fd44c592640 1 -- 172.21.15.120:0/3691310440 shutdown_connections 2024-09-19T09:12:07.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.723+0000 7fd44c592640 1 --2- 172.21.15.120:0/3691310440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd444106f20 0x7fd44410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.723+0000 7fd44c592640 1 --2- 172.21.15.120:0/3691310440 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4441027a0 0x7fd4441067f0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.723+0000 7fd44c592640 1 --2- 172.21.15.120:0/3691310440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd444101df0 0x7fd4441021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.723+0000 7fd44c592640 1 -- 172.21.15.120:0/3691310440 >> 172.21.15.120:0/3691310440 conn(0x7fd4440fd8f0 msgr2=0x7fd4440ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:07.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.723+0000 7fd44c592640 1 -- 172.21.15.120:0/3691310440 shutdown_connections 2024-09-19T09:12:07.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.723+0000 7fd44c592640 1 -- 172.21.15.120:0/3691310440 wait complete. 2024-09-19T09:12:07.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.724+0000 7fd44c592640 1 Processor -- start 2024-09-19T09:12:07.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.724+0000 7fd44c592640 1 -- start start 2024-09-19T09:12:07.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.724+0000 7fd44c592640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd444101df0 0x7fd4441984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:07.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.724+0000 7fd44c592640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4441027a0 0x7fd444198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:07.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.724+0000 7fd44c592640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd444106f20 0x7fd44419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:07.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.724+0000 7fd44c592640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd444111f80 con 0x7fd444101df0 2024-09-19T09:12:07.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.724+0000 7fd44c592640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd444111e00 con 0x7fd4441027a0 2024-09-19T09:12:07.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.724+0000 7fd44c592640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd444112100 con 0x7fd444106f20 2024-09-19T09:12:07.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.724+0000 7fd44a32e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd444101df0 0x7fd4441984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:07.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.724+0000 7fd44a32e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd444101df0 0x7fd4441984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46900/0 (socket says 172.21.15.120:46900) 2024-09-19T09:12:07.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.724+0000 7fd44a32e640 1 -- 172.21.15.120:0/2300543191 learned_addr learned my addr 172.21.15.120:0/2300543191 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:07.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.724+0000 7fd449b2d640 1 --2- 172.21.15.120:0/2300543191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4441027a0 0x7fd444198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:07.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.724+0000 7fd44a32e640 1 -- 172.21.15.120:0/2300543191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd444106f20 msgr2=0x7fd44419cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:07.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.724+0000 7fd44ab2f640 1 --2- 172.21.15.120:0/2300543191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd444106f20 0x7fd44419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:07.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.725+0000 7fd44a32e640 1 --2- 172.21.15.120:0/2300543191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd444106f20 0x7fd44419cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.725+0000 7fd44a32e640 1 -- 172.21.15.120:0/2300543191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4441027a0 msgr2=0x7fd444198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:07.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.725+0000 7fd44a32e640 1 --2- 172.21.15.120:0/2300543191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4441027a0 0x7fd444198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.725+0000 7fd44a32e640 1 -- 172.21.15.120:0/2300543191 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd44419d490 con 0x7fd444101df0 2024-09-19T09:12:07.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.725+0000 7fd44a32e640 1 --2- 172.21.15.120:0/2300543191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd444101df0 0x7fd4441984e0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fd44000e9f0 tx=0x7fd44000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:07.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.725+0000 7fd43b7fe640 1 -- 172.21.15.120:0/2300543191 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44000cd80 con 0x7fd444101df0 2024-09-19T09:12:07.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.725+0000 7fd43b7fe640 1 -- 172.21.15.120:0/2300543191 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd440010910 con 0x7fd444101df0 2024-09-19T09:12:07.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.725+0000 7fd44c592640 1 -- 172.21.15.120:0/2300543191 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd44419d780 con 0x7fd444101df0 2024-09-19T09:12:07.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.725+0000 7fd43b7fe640 1 -- 172.21.15.120:0/2300543191 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440018990 con 0x7fd444101df0 2024-09-19T09:12:07.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.725+0000 7fd44c592640 1 -- 172.21.15.120:0/2300543191 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4441a5060 con 0x7fd444101df0 2024-09-19T09:12:07.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.728+0000 7fd44e96b740 1 Processor -- start 2024-09-19T09:12:07.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.728+0000 7fd43b7fe640 1 -- 172.21.15.120:0/2300543191 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd440010ac0 con 0x7fd444101df0 2024-09-19T09:12:07.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.728+0000 7fd44e96b740 1 -- start start 2024-09-19T09:12:07.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.728+0000 7fd44e96b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ffaf67b40 0x557ffaf66090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:07.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.728+0000 7fd44e96b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ffaf665d0 0x557ffaf66a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:07.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.728+0000 7fd44e96b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ffaf71370 0x557ffaf73760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:07.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.728+0000 7fd44e96b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557ffaecc2f0 con 0x557ffaf71370 2024-09-19T09:12:07.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.728+0000 7fd44e96b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557ffae73cf0 con 0x557ffaf67b40 2024-09-19T09:12:07.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.728+0000 7fd44e96b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557ffaec82b0 con 0x557ffaf665d0 2024-09-19T09:12:07.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.729+0000 7fd449b2d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ffaf71370 0x557ffaf73760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:07.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.729+0000 7fd44ab2f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ffaf67b40 0x557ffaf66090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:07.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.729+0000 7fd44ab2f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ffaf67b40 0x557ffaf66090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:40900/0 (socket says 172.21.15.120:40900) 2024-09-19T09:12:07.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.729+0000 7fd44ab2f640 1 -- 172.21.15.120:0/4075256590 learned_addr learned my addr 172.21.15.120:0/4075256590 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:07.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.729+0000 7fd449b2d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ffaf71370 0x557ffaf73760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46916/0 (socket says 172.21.15.120:46916) 2024-09-19T09:12:07.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.729+0000 7fd44a32e640 1 --2- 172.21.15.120:0/4075256590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ffaf665d0 0x557ffaf66a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:07.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.729+0000 7fd44ab2f640 1 -- 172.21.15.120:0/4075256590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ffaf665d0 msgr2=0x557ffaf66a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:07.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.729+0000 7fd44ab2f640 1 --2- 172.21.15.120:0/4075256590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ffaf665d0 0x557ffaf66a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.729+0000 7fd44ab2f640 1 -- 172.21.15.120:0/4075256590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ffaf71370 msgr2=0x557ffaf73760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:07.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.729+0000 7fd44ab2f640 1 --2- 172.21.15.120:0/4075256590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ffaf71370 0x557ffaf73760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.729+0000 7fd44ab2f640 1 -- 172.21.15.120:0/4075256590 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ffae583a0 con 0x557ffaf67b40 2024-09-19T09:12:07.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.729+0000 7fd43b7fe640 1 --2- 172.21.15.120:0/2300543191 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd4300a32c0 0x7fd4300a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:07.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.729+0000 7fd44ab2f640 1 --2- 172.21.15.120:0/4075256590 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ffaf67b40 0x557ffaf66090 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd43c008c80 tx=0x7fd43c00d710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb7a6b47564c6d42 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:07.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.729+0000 7fd44a32e640 1 --2- 172.21.15.120:0/2300543191 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd4300a32c0 0x7fd4300a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:07.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.729+0000 7fd43b7fe640 1 -- 172.21.15.120:0/2300543191 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fd4400cfc30 con 0x7fd444101df0 2024-09-19T09:12:07.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.729+0000 7fd438ff9640 1 -- 172.21.15.120:0/4075256590 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c018680 con 0x557ffaf67b40 2024-09-19T09:12:07.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.729+0000 7fd438ff9640 1 -- 172.21.15.120:0/4075256590 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd43c018d00 con 0x557ffaf67b40 2024-09-19T09:12:07.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.729+0000 7fd438ff9640 1 -- 172.21.15.120:0/4075256590 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c00fdd0 con 0x557ffaf67b40 2024-09-19T09:12:07.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.730+0000 7fd44e96b740 1 -- 172.21.15.120:0/4075256590 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ffaf67b40 msgr2=0x557ffaf66090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:07.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.730+0000 7fd44a32e640 1 --2- 172.21.15.120:0/2300543191 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd4300a32c0 0x7fd4300a5780 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd4400a4990 tx=0x7fd440000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:07.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.730+0000 7fd44e96b740 1 --2- 172.21.15.120:0/4075256590 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ffaf67b40 0x557ffaf66090 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd43c008c80 tx=0x7fd43c00d710 comp rx=0 tx=0).stop 2024-09-19T09:12:07.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.730+0000 7fd44e96b740 1 -- 172.21.15.120:0/4075256590 shutdown_connections 2024-09-19T09:12:07.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.730+0000 7fd44e96b740 1 --2- 172.21.15.120:0/4075256590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ffaf71370 0x557ffaf73760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.730+0000 7fd44e96b740 1 --2- 172.21.15.120:0/4075256590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ffaf665d0 0x557ffaf66a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.730+0000 7fd44e96b740 1 --2- 172.21.15.120:0/4075256590 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ffaf67b40 0x557ffaf66090 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.730+0000 7fd44e96b740 1 -- 172.21.15.120:0/4075256590 >> 172.21.15.120:0/4075256590 conn(0x557ffaf2f310 msgr2=0x557ffaf62c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:07.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.730+0000 7fd44e96b740 1 -- 172.21.15.120:0/4075256590 shutdown_connections 2024-09-19T09:12:07.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.730+0000 7fd44e96b740 1 -- 172.21.15.120:0/4075256590 wait complete. 2024-09-19T09:12:07.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.731+0000 7fd44e96b740 1 -- start start 2024-09-19T09:12:07.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.731+0000 7fd44e96b740 1 Processor -- start 2024-09-19T09:12:07.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.731+0000 7fd44e96b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ffaf665d0 0x557ffaf70d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:07.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.731+0000 7fd44e96b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ffaf67b40 0x557ffaf6e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:07.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.731+0000 7fd44e96b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ffaf71370 0x557ffaf6e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:07.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.731+0000 7fd44e96b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557ffaf74070 con 0x557ffaf67b40 2024-09-19T09:12:07.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.731+0000 7fd44e96b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557ffae87060 con 0x557ffaf71370 2024-09-19T09:12:07.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.731+0000 7fd44e96b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557ffaf741f0 con 0x557ffaf665d0 2024-09-19T09:12:07.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.731+0000 7fd44e96b740 10 client.0 _ll_register_callbacks cb 0x557ffabda8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:07.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.731+0000 7fd449b2d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ffaf67b40 0x557ffaf6e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:07.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.731+0000 7fd449b2d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ffaf67b40 0x557ffaf6e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46918/0 (socket says 172.21.15.120:46918) 2024-09-19T09:12:07.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.731+0000 7fd449b2d640 1 -- 172.21.15.120:0/148108110 learned_addr learned my addr 172.21.15.120:0/148108110 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:07.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.731+0000 7fd44ab2f640 1 --2- 172.21.15.120:0/148108110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ffaf71370 0x557ffaf6e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:07.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.731+0000 7fd44ab2f640 1 --2- 172.21.15.120:0/148108110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ffaf665d0 0x557ffaf70d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:07.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.731+0000 7fd449b2d640 1 -- 172.21.15.120:0/148108110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ffaf665d0 msgr2=0x557ffaf70d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:07.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.731+0000 7fd449b2d640 1 --2- 172.21.15.120:0/148108110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ffaf665d0 0x557ffaf70d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.731+0000 7fd449b2d640 1 -- 172.21.15.120:0/148108110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ffaf71370 msgr2=0x557ffaf6e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:07.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.732+0000 7fd449b2d640 1 --2- 172.21.15.120:0/148108110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ffaf71370 0x557ffaf6e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.732+0000 7fd449b2d640 1 -- 172.21.15.120:0/148108110 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ffac3d3c0 con 0x557ffaf67b40 2024-09-19T09:12:07.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.732+0000 7fd44ab2f640 1 --2- 172.21.15.120:0/148108110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ffaf71370 0x557ffaf6e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:12:07.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.732+0000 7fd449b2d640 1 --2- 172.21.15.120:0/148108110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ffaf67b40 0x557ffaf6e040 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fd434009900 tx=0x7fd434031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2478dac2630bbc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:07.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.732+0000 7fd42dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:07.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.732+0000 7fd42dffb640 1 -- 172.21.15.120:0/148108110 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd434035040 con 0x557ffaf67b40 2024-09-19T09:12:07.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.732+0000 7fd42dffb640 1 -- 172.21.15.120:0/148108110 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fd434004240 con 0x557ffaf67b40 2024-09-19T09:12:07.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.732+0000 7fd44e96b740 1 -- 172.21.15.120:0/148108110 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd43c018d00 con 0x557ffaf67b40 2024-09-19T09:12:07.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.732+0000 7fd42dffb640 1 -- 172.21.15.120:0/148108110 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd434004440 con 0x557ffaf67b40 2024-09-19T09:12:07.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.732+0000 7fd44e96b740 10 client.18681 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:07.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.732+0000 7fd44e96b740 1 -- 172.21.15.120:0/148108110 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557ffaf7fb90 con 0x557ffaf67b40 2024-09-19T09:12:07.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.732+0000 7fd42dffb640 1 -- 172.21.15.120:0/148108110 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd43c018d00 con 0x557ffaf67b40 2024-09-19T09:12:07.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.732+0000 7fd42dffb640 1 -- 172.21.15.120:0/148108110 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd4340035f0 con 0x557ffaf67b40 2024-09-19T09:12:07.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.732+0000 7fd44e96b740 10 client.18681 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:07.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.732+0000 7fd44e96b740 10 client.18681 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:12:07.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.732+0000 7fd44e96b740 20 client.18681 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:07.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.732+0000 7fd44e96b740 1 --2- 172.21.15.120:0/148108110 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x557ffaf81050 0x557ffaf83470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:07.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.732+0000 7fd44e96b740 4 client.18681 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:12:07.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.732+0000 7fd44e96b740 1 -- 172.21.15.120:0/148108110 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ffaed9860 con 0x557ffaf81050 2024-09-19T09:12:07.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.733+0000 7fd44ab2f640 1 --2- 172.21.15.120:0/148108110 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x557ffaf81050 0x557ffaf83470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:07.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.733+0000 7fd42dffb640 10 client.18681 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:12:07.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.733+0000 7fd44ab2f640 1 --2- 172.21.15.120:0/148108110 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x557ffaf81050 0x557ffaf83470 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bcec5d5690194f21 server_cookie=aa98993d66898075 in_seq=0 out_seq=0 2024-09-19T09:12:07.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.734+0000 7fd42dffb640 1 -- 172.21.15.120:0/148108110 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557ffaed9860 con 0x557ffaf81050 2024-09-19T09:12:07.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.734+0000 7fd42dffb640 10 client.18681 handle_command_reply: tid=0 2024-09-19T09:12:07.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.734+0000 7fd44e96b740 1 client.18681 shutdown 2024-09-19T09:12:07.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.734+0000 7fd44e96b740 1 -- 172.21.15.120:0/148108110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ffaf67b40 msgr2=0x557ffaf6e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:07.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.734+0000 7fd44e96b740 1 --2- 172.21.15.120:0/148108110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ffaf67b40 0x557ffaf6e040 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fd434009900 tx=0x7fd434031920 comp rx=0 tx=0).stop 2024-09-19T09:12:07.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.734+0000 7fd44e96b740 1 -- 172.21.15.120:0/148108110 shutdown_connections 2024-09-19T09:12:07.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.734+0000 7fd44e96b740 1 --2- 172.21.15.120:0/148108110 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x557ffaf81050 0x557ffaf83470 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.734+0000 7fd42dffb640 0 client.18681 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:12:07.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.734+0000 7fd44e96b740 1 --2- 172.21.15.120:0/148108110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ffaf71370 0x557ffaf6e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.734+0000 7fd44e96b740 1 --2- 172.21.15.120:0/148108110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ffaf67b40 0x557ffaf6e040 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.734+0000 7fd44e96b740 1 --2- 172.21.15.120:0/148108110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ffaf665d0 0x557ffaf70d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.734+0000 7fd44e96b740 1 -- 172.21.15.120:0/148108110 >> 172.21.15.120:0/148108110 conn(0x557ffaf2f310 msgr2=0x557ffaf68960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:07.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.734+0000 7fd44e96b740 1 -- 172.21.15.120:0/148108110 shutdown_connections 2024-09-19T09:12:07.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.734+0000 7fd44e96b740 1 -- 172.21.15.120:0/148108110 wait complete. 2024-09-19T09:12:07.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.734+0000 7fd44e96b740 20 client.18681 trim_cache size 0 max 16384 2024-09-19T09:12:07.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.747+0000 7fd44e96b740 1 Processor -- start 2024-09-19T09:12:07.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.747+0000 7fd44e96b740 1 -- start start 2024-09-19T09:12:07.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.748+0000 7fd44e96b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ffaf665d0 0x557ffaf7d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:07.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.748+0000 7fd44e96b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ffaf67b40 0x557ffaf6d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:07.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.748+0000 7fd44e96b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ffaf71370 0x557ffaf88d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:07.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.748+0000 7fd44e96b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd4441ad5f0 con 0x557ffaf67b40 2024-09-19T09:12:07.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.748+0000 7fd44e96b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd4441af130 con 0x557ffaf665d0 2024-09-19T09:12:07.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.748+0000 7fd44e96b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd4441adcc0 con 0x557ffaf71370 2024-09-19T09:12:07.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.748+0000 7fd449b2d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ffaf67b40 0x557ffaf6d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:07.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.748+0000 7fd449b2d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ffaf67b40 0x557ffaf6d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46932/0 (socket says 172.21.15.120:46932) 2024-09-19T09:12:07.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.748+0000 7fd449b2d640 1 -- 172.21.15.120:0/903322045 learned_addr learned my addr 172.21.15.120:0/903322045 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:07.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.748+0000 7fd44ab2f640 1 --2- 172.21.15.120:0/903322045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ffaf665d0 0x557ffaf7d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:07.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.748+0000 7fd44ab2f640 1 --2- 172.21.15.120:0/903322045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ffaf71370 0x557ffaf88d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:07.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.748+0000 7fd449b2d640 1 -- 172.21.15.120:0/903322045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ffaf71370 msgr2=0x557ffaf88d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:07.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.748+0000 7fd449b2d640 1 --2- 172.21.15.120:0/903322045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ffaf71370 0x557ffaf88d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.748+0000 7fd449b2d640 1 -- 172.21.15.120:0/903322045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ffaf665d0 msgr2=0x557ffaf7d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:07.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.748+0000 7fd449b2d640 1 --2- 172.21.15.120:0/903322045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ffaf665d0 0x557ffaf7d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.748+0000 7fd449b2d640 1 -- 172.21.15.120:0/903322045 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ffaf63dd0 con 0x557ffaf67b40 2024-09-19T09:12:07.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.748+0000 7fd449b2d640 1 --2- 172.21.15.120:0/903322045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ffaf67b40 0x557ffaf6d020 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fd43402a930 tx=0x7fd434001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb7c48a54b57790 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:07.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.749+0000 7fd42ffff640 1 -- 172.21.15.120:0/903322045 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd434035030 con 0x557ffaf67b40 2024-09-19T09:12:07.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.749+0000 7fd42ffff640 1 -- 172.21.15.120:0/903322045 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd434005520 con 0x557ffaf67b40 2024-09-19T09:12:07.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.749+0000 7fd42ffff640 1 -- 172.21.15.120:0/903322045 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4340046c0 con 0x557ffaf67b40 2024-09-19T09:12:07.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.749+0000 7fd44e96b740 1 -- 172.21.15.120:0/903322045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ffaf67b40 msgr2=0x557ffaf6d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:07.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.749+0000 7fd44e96b740 1 --2- 172.21.15.120:0/903322045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ffaf67b40 0x557ffaf6d020 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fd43402a930 tx=0x7fd434001e70 comp rx=0 tx=0).stop 2024-09-19T09:12:07.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.749+0000 7fd44e96b740 1 -- 172.21.15.120:0/903322045 shutdown_connections 2024-09-19T09:12:07.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.749+0000 7fd44e96b740 1 --2- 172.21.15.120:0/903322045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ffaf71370 0x557ffaf88d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.749+0000 7fd44e96b740 1 --2- 172.21.15.120:0/903322045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ffaf67b40 0x557ffaf6d020 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.749+0000 7fd44e96b740 1 --2- 172.21.15.120:0/903322045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ffaf665d0 0x557ffaf7d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.749+0000 7fd44e96b740 1 -- 172.21.15.120:0/903322045 >> 172.21.15.120:0/903322045 conn(0x557ffaf2f310 msgr2=0x557ffaf6c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:07.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.749+0000 7fd44e96b740 1 -- 172.21.15.120:0/903322045 shutdown_connections 2024-09-19T09:12:07.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.750+0000 7fd44e96b740 1 -- 172.21.15.120:0/903322045 wait complete. 2024-09-19T09:12:07.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.750+0000 7fd44e96b740 1 -- start start 2024-09-19T09:12:07.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.750+0000 7fd44e96b740 1 Processor -- start 2024-09-19T09:12:07.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.750+0000 7fd44e96b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ffaf665d0 0x557ffaf79970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:07.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.750+0000 7fd44e96b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ffaf67b40 0x557ffaf74ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:07.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.750+0000 7fd44e96b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ffaf71370 0x557ffaf751e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:07.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.750+0000 7fd44e96b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd4441aea60 con 0x557ffaf665d0 2024-09-19T09:12:07.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.750+0000 7fd44e96b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd4441ae390 con 0x557ffaf67b40 2024-09-19T09:12:07.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.750+0000 7fd44e96b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd4441ab5b0 con 0x557ffaf71370 2024-09-19T09:12:07.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.750+0000 7fd44e96b740 10 client.0 _ll_register_callbacks cb 0x7fd434009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:07.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.750+0000 7fd44ab2f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ffaf665d0 0x557ffaf79970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:07.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.750+0000 7fd44ab2f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ffaf665d0 0x557ffaf79970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46934/0 (socket says 172.21.15.120:46934) 2024-09-19T09:12:07.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.750+0000 7fd44ab2f640 1 -- 172.21.15.120:0/2223144053 learned_addr learned my addr 172.21.15.120:0/2223144053 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:07.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.750+0000 7fd449b2d640 1 --2- 172.21.15.120:0/2223144053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ffaf67b40 0x557ffaf74ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:07.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.750+0000 7fd44ab2f640 1 --2- 172.21.15.120:0/2223144053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ffaf71370 0x557ffaf751e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:07.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.751+0000 7fd449b2d640 1 -- 172.21.15.120:0/2223144053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ffaf71370 msgr2=0x557ffaf751e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:07.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.751+0000 7fd449b2d640 1 --2- 172.21.15.120:0/2223144053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ffaf71370 0x557ffaf751e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.751+0000 7fd449b2d640 1 -- 172.21.15.120:0/2223144053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ffaf665d0 msgr2=0x557ffaf79970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:07.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.751+0000 7fd449b2d640 1 --2- 172.21.15.120:0/2223144053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ffaf665d0 0x557ffaf79970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.751+0000 7fd449b2d640 1 -- 172.21.15.120:0/2223144053 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ffaf69120 con 0x557ffaf67b40 2024-09-19T09:12:07.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.751+0000 7fd44ab2f640 1 --2- 172.21.15.120:0/2223144053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ffaf665d0 0x557ffaf79970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:07.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.751+0000 7fd449b2d640 1 --2- 172.21.15.120:0/2223144053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ffaf67b40 0x557ffaf74ca0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd434003bb0 tx=0x7fd434005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75a8638dada031fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:07.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.751+0000 7fd42dffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:12:07.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.751+0000 7fd42dffb640 1 -- 172.21.15.120:0/2223144053 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd434066060 con 0x557ffaf67b40 2024-09-19T09:12:07.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.751+0000 7fd42dffb640 1 -- 172.21.15.120:0/2223144053 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fd434046480 con 0x557ffaf67b40 2024-09-19T09:12:07.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.751+0000 7fd44e96b740 1 -- 172.21.15.120:0/2223144053 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd434005520 con 0x557ffaf67b40 2024-09-19T09:12:07.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.751+0000 7fd42dffb640 1 -- 172.21.15.120:0/2223144053 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd434035070 con 0x557ffaf67b40 2024-09-19T09:12:07.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.751+0000 7fd44e96b740 10 client.27818 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:07.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.751+0000 7fd44e96b740 1 -- 172.21.15.120:0/2223144053 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557ffafefce0 con 0x557ffaf67b40 2024-09-19T09:12:07.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.751+0000 7fd42dffb640 1 -- 172.21.15.120:0/2223144053 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd434005520 con 0x557ffaf67b40 2024-09-19T09:12:07.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.752+0000 7fd42dffb640 1 -- 172.21.15.120:0/2223144053 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd43406b070 con 0x557ffaf67b40 2024-09-19T09:12:07.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.752+0000 7fd44e96b740 10 client.27818 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:07.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.752+0000 7fd44e96b740 10 client.27818 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:12:07.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.752+0000 7fd44e96b740 20 client.27818 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:07.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.752+0000 7fd44e96b740 1 --2- 172.21.15.120:0/2223144053 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x557ffaf81050 0x557ffaff2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:07.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.752+0000 7fd44e96b740 4 client.27818 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:12:07.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.752+0000 7fd44e96b740 1 -- 172.21.15.120:0/2223144053 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557ffaed3d70 con 0x557ffaf81050 2024-09-19T09:12:07.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.752+0000 7fd44ab2f640 1 --2- 172.21.15.120:0/2223144053 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x557ffaf81050 0x557ffaff2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:07.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.752+0000 7fd42dffb640 10 client.27818 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:12:07.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.752+0000 7fd44ab2f640 1 --2- 172.21.15.120:0/2223144053 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x557ffaf81050 0x557ffaff2360 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=917352350e69430f server_cookie=8d787a280b582438 in_seq=0 out_seq=0 2024-09-19T09:12:07.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.752+0000 7fd42dffb640 1 -- 172.21.15.120:0/2223144053 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557ffaed3d70 con 0x557ffaf81050 2024-09-19T09:12:07.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.752+0000 7fd42dffb640 10 client.27818 handle_command_reply: tid=0 2024-09-19T09:12:07.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.753+0000 7fd44e96b740 1 client.27818 shutdown 2024-09-19T09:12:07.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.753+0000 7fd44e96b740 1 -- 172.21.15.120:0/2223144053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ffaf67b40 msgr2=0x557ffaf74ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:07.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.753+0000 7fd44e96b740 1 --2- 172.21.15.120:0/2223144053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ffaf67b40 0x557ffaf74ca0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd434003bb0 tx=0x7fd434005fd0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.753+0000 7fd44e96b740 1 -- 172.21.15.120:0/2223144053 shutdown_connections 2024-09-19T09:12:07.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.753+0000 7fd44e96b740 1 --2- 172.21.15.120:0/2223144053 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x557ffaf81050 0x557ffaff2360 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.753+0000 7fd44e96b740 1 --2- 172.21.15.120:0/2223144053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ffaf71370 0x557ffaf751e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.753+0000 7fd44e96b740 1 --2- 172.21.15.120:0/2223144053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ffaf67b40 0x557ffaf74ca0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.753+0000 7fd42dffb640 0 client.27818 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:12:07.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.753+0000 7fd44e96b740 1 --2- 172.21.15.120:0/2223144053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ffaf665d0 0x557ffaf79970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.753+0000 7fd44e96b740 1 -- 172.21.15.120:0/2223144053 >> 172.21.15.120:0/2223144053 conn(0x557ffaf2f310 msgr2=0x557ffaf6f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:07.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.753+0000 7fd44e96b740 1 -- 172.21.15.120:0/2223144053 shutdown_connections 2024-09-19T09:12:07.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.753+0000 7fd44e96b740 1 -- 172.21.15.120:0/2223144053 wait complete. 2024-09-19T09:12:07.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.753+0000 7fd44e96b740 20 client.27818 trim_cache size 0 max 16384 2024-09-19T09:12:07.786 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:12:07.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.753+0000 7fd44c592640 1 -- 172.21.15.120:0/2300543191 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd4300a32c0 msgr2=0x7fd4300a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:07.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.753+0000 7fd44c592640 1 --2- 172.21.15.120:0/2300543191 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd4300a32c0 0x7fd4300a5780 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd4400a4990 tx=0x7fd440000bc0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.753+0000 7fd44c592640 1 -- 172.21.15.120:0/2300543191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd444101df0 msgr2=0x7fd4441984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:07.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.753+0000 7fd44c592640 1 --2- 172.21.15.120:0/2300543191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd444101df0 0x7fd4441984e0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fd44000e9f0 tx=0x7fd44000eeb0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.753+0000 7fd44c592640 1 -- 172.21.15.120:0/2300543191 shutdown_connections 2024-09-19T09:12:07.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.753+0000 7fd44c592640 1 --2- 172.21.15.120:0/2300543191 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd4300a32c0 0x7fd4300a5780 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.753+0000 7fd44c592640 1 --2- 172.21.15.120:0/2300543191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd444106f20 0x7fd44419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.753+0000 7fd44c592640 1 --2- 172.21.15.120:0/2300543191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4441027a0 0x7fd444198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.753+0000 7fd44c592640 1 --2- 172.21.15.120:0/2300543191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd444101df0 0x7fd4441984e0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:07.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.753+0000 7fd44c592640 1 -- 172.21.15.120:0/2300543191 >> 172.21.15.120:0/2300543191 conn(0x7fd4440fd8f0 msgr2=0x7fd4440fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:07.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.754+0000 7fd44c592640 1 -- 172.21.15.120:0/2300543191 shutdown_connections 2024-09-19T09:12:07.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:07.754+0000 7fd44c592640 1 -- 172.21.15.120:0/2300543191 wait complete. 2024-09-19T09:12:08.065 INFO:tasks.workunit.client.0.smithi120.stdout: 1 883258 9.78 MB/sec execute 363 sec latency 37.862 ms 2024-09-19T09:12:09.025 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:12:09.065 INFO:tasks.workunit.client.0.smithi120.stdout: 1 885666 9.79 MB/sec execute 364 sec latency 60.006 ms 2024-09-19T09:12:09.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:08 smithi196 ceph-mon[35150]: pgmap v621: 97 pgs: 97 active+clean; 44 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 367 op/s 2024-09-19T09:12:09.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:08 smithi153 ceph-mon[35071]: pgmap v621: 97 pgs: 97 active+clean; 44 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 367 op/s 2024-09-19T09:12:09.288 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:12:09.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:08 smithi120 ceph-mon[31150]: pgmap v621: 97 pgs: 97 active+clean; 44 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 367 op/s 2024-09-19T09:12:09.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.794+0000 7fea732bc640 1 -- 172.21.15.120:0/3679984110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea6c1027a0 msgr2=0x7fea6c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:09.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.794+0000 7fea732bc640 1 --2- 172.21.15.120:0/3679984110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea6c1027a0 0x7fea6c1067f0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009900 tx=0x7fea5c02a410 comp rx=0 tx=0).stop 2024-09-19T09:12:09.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.794+0000 7fea732bc640 1 -- 172.21.15.120:0/3679984110 shutdown_connections 2024-09-19T09:12:09.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.794+0000 7fea732bc640 1 --2- 172.21.15.120:0/3679984110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea6c106f20 0x7fea6c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.794+0000 7fea732bc640 1 --2- 172.21.15.120:0/3679984110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea6c1027a0 0x7fea6c1067f0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.794+0000 7fea732bc640 1 --2- 172.21.15.120:0/3679984110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea6c101df0 0x7fea6c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.794+0000 7fea732bc640 1 -- 172.21.15.120:0/3679984110 >> 172.21.15.120:0/3679984110 conn(0x7fea6c0fd8f0 msgr2=0x7fea6c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:09.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.794+0000 7fea732bc640 1 -- 172.21.15.120:0/3679984110 shutdown_connections 2024-09-19T09:12:09.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.794+0000 7fea732bc640 1 -- 172.21.15.120:0/3679984110 wait complete. 2024-09-19T09:12:09.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.795+0000 7fea732bc640 1 Processor -- start 2024-09-19T09:12:09.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.795+0000 7fea732bc640 1 -- start start 2024-09-19T09:12:09.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.795+0000 7fea732bc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea6c101df0 0x7fea6c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:09.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.795+0000 7fea732bc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea6c1027a0 0x7fea6c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:09.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.795+0000 7fea732bc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea6c106f20 0x7fea6c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:09.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.795+0000 7fea732bc640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fea6c111f40 con 0x7fea6c101df0 2024-09-19T09:12:09.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.795+0000 7fea732bc640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fea6c111dc0 con 0x7fea6c106f20 2024-09-19T09:12:09.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.795+0000 7fea732bc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fea6c1120c0 con 0x7fea6c1027a0 2024-09-19T09:12:09.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.795+0000 7fea71058640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea6c101df0 0x7fea6c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:09.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.795+0000 7fea71058640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea6c101df0 0x7fea6c198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46954/0 (socket says 172.21.15.120:46954) 2024-09-19T09:12:09.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.795+0000 7fea71058640 1 -- 172.21.15.120:0/1664456093 learned_addr learned my addr 172.21.15.120:0/1664456093 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:09.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.795+0000 7fea71859640 1 --2- 172.21.15.120:0/1664456093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea6c106f20 0x7fea6c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:09.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.795+0000 7fea70857640 1 --2- 172.21.15.120:0/1664456093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea6c1027a0 0x7fea6c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:09.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.796+0000 7fea71058640 1 -- 172.21.15.120:0/1664456093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea6c1027a0 msgr2=0x7fea6c198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:09.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.796+0000 7fea71058640 1 --2- 172.21.15.120:0/1664456093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea6c1027a0 0x7fea6c198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.796+0000 7fea71058640 1 -- 172.21.15.120:0/1664456093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea6c106f20 msgr2=0x7fea6c19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:09.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.796+0000 7fea71058640 1 --2- 172.21.15.120:0/1664456093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea6c106f20 0x7fea6c19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.796+0000 7fea71058640 1 -- 172.21.15.120:0/1664456093 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6c19d3f0 con 0x7fea6c101df0 2024-09-19T09:12:09.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.796+0000 7fea71058640 1 --2- 172.21.15.120:0/1664456093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea6c101df0 0x7fea6c198440 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fea6800cce0 tx=0x7fea6800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:09.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.796+0000 7fea627fc640 1 -- 172.21.15.120:0/1664456093 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6800f650 con 0x7fea6c101df0 2024-09-19T09:12:09.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.796+0000 7fea732bc640 1 -- 172.21.15.120:0/1664456093 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea6c19d6e0 con 0x7fea6c101df0 2024-09-19T09:12:09.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.796+0000 7fea627fc640 1 -- 172.21.15.120:0/1664456093 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea6800fcd0 con 0x7fea6c101df0 2024-09-19T09:12:09.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.796+0000 7fea627fc640 1 -- 172.21.15.120:0/1664456093 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea68018c70 con 0x7fea6c101df0 2024-09-19T09:12:09.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.796+0000 7fea732bc640 1 -- 172.21.15.120:0/1664456093 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea6c1a4fc0 con 0x7fea6c101df0 2024-09-19T09:12:09.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.799+0000 7fea627fc640 1 -- 172.21.15.120:0/1664456093 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fea6800f7f0 con 0x7fea6c101df0 2024-09-19T09:12:09.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.799+0000 7fea75695740 1 Processor -- start 2024-09-19T09:12:09.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.799+0000 7fea75695740 1 -- start start 2024-09-19T09:12:09.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.799+0000 7fea75695740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea80841b40 0x55ea80840090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:09.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.800+0000 7fea75695740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea808405d0 0x55ea80840a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:09.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.800+0000 7fea75695740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea8084b370 0x55ea8084d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:09.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.800+0000 7fea75695740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ea807a62f0 con 0x55ea80841b40 2024-09-19T09:12:09.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.800+0000 7fea75695740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ea8074dcf0 con 0x55ea808405d0 2024-09-19T09:12:09.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.800+0000 7fea75695740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ea807a22b0 con 0x55ea8084b370 2024-09-19T09:12:09.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.800+0000 7fea71859640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea80841b40 0x55ea80840090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:09.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.800+0000 7fea71859640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea80841b40 0x55ea80840090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46964/0 (socket says 172.21.15.120:46964) 2024-09-19T09:12:09.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.800+0000 7fea71859640 1 -- 172.21.15.120:0/2492182002 learned_addr learned my addr 172.21.15.120:0/2492182002 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:09.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.800+0000 7fea70857640 1 --2- 172.21.15.120:0/2492182002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea8084b370 0x55ea8084d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:09.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.800+0000 7fea71058640 1 --2- 172.21.15.120:0/2492182002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea808405d0 0x55ea80840a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:09.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.800+0000 7fea627fc640 1 --2- 172.21.15.120:0/1664456093 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fea580a3240 0x7fea580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:09.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.800+0000 7fea71859640 1 -- 172.21.15.120:0/2492182002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea8084b370 msgr2=0x55ea8084d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:09.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.800+0000 7fea71859640 1 --2- 172.21.15.120:0/2492182002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea8084b370 0x55ea8084d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.800+0000 7fea71859640 1 -- 172.21.15.120:0/2492182002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea808405d0 msgr2=0x55ea80840a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:09.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.800+0000 7fea71859640 1 --2- 172.21.15.120:0/2492182002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea808405d0 0x55ea80840a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.800+0000 7fea71859640 1 -- 172.21.15.120:0/2492182002 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea807323a0 con 0x55ea80841b40 2024-09-19T09:12:09.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.800+0000 7fea627fc640 1 -- 172.21.15.120:0/1664456093 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fea680cfd60 con 0x7fea6c101df0 2024-09-19T09:12:09.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.800+0000 7fea71859640 1 --2- 172.21.15.120:0/1664456093 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fea580a3240 0x7fea580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:09.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.800+0000 7fea71859640 1 --2- 172.21.15.120:0/2492182002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea80841b40 0x55ea80840090 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fea64009da0 tx=0x7fea64011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a13c66ee85e9a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:09.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.801+0000 7fea577fe640 1 -- 172.21.15.120:0/2492182002 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea64017050 con 0x55ea80841b40 2024-09-19T09:12:09.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.801+0000 7fea577fe640 1 -- 172.21.15.120:0/2492182002 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea64026920 con 0x55ea80841b40 2024-09-19T09:12:09.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.801+0000 7fea577fe640 1 -- 172.21.15.120:0/2492182002 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6400f820 con 0x55ea80841b40 2024-09-19T09:12:09.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.801+0000 7fea71859640 1 --2- 172.21.15.120:0/1664456093 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fea580a3240 0x7fea580a5700 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fea64023000 tx=0x7fea64024000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:09.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.801+0000 7fea75695740 1 -- 172.21.15.120:0/2492182002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea80841b40 msgr2=0x55ea80840090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:09.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.801+0000 7fea75695740 1 --2- 172.21.15.120:0/2492182002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea80841b40 0x55ea80840090 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fea64009da0 tx=0x7fea64011810 comp rx=0 tx=0).stop 2024-09-19T09:12:09.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.801+0000 7fea75695740 1 -- 172.21.15.120:0/2492182002 shutdown_connections 2024-09-19T09:12:09.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.801+0000 7fea75695740 1 --2- 172.21.15.120:0/2492182002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea8084b370 0x55ea8084d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.801+0000 7fea75695740 1 --2- 172.21.15.120:0/2492182002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea808405d0 0x55ea80840a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.801+0000 7fea75695740 1 --2- 172.21.15.120:0/2492182002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea80841b40 0x55ea80840090 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.801+0000 7fea75695740 1 -- 172.21.15.120:0/2492182002 >> 172.21.15.120:0/2492182002 conn(0x55ea80809310 msgr2=0x55ea8083cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:09.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.801+0000 7fea75695740 1 -- 172.21.15.120:0/2492182002 shutdown_connections 2024-09-19T09:12:09.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.801+0000 7fea75695740 1 -- 172.21.15.120:0/2492182002 wait complete. 2024-09-19T09:12:09.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.802+0000 7fea75695740 1 -- start start 2024-09-19T09:12:09.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.802+0000 7fea75695740 1 Processor -- start 2024-09-19T09:12:09.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.802+0000 7fea75695740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea808405d0 0x55ea8084ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:09.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.802+0000 7fea75695740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea80841b40 0x55ea80848000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:09.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.802+0000 7fea75695740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea8084b370 0x55ea808485f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:09.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.802+0000 7fea75695740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ea8084e070 con 0x55ea808405d0 2024-09-19T09:12:09.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.802+0000 7fea75695740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ea80761060 con 0x55ea8084b370 2024-09-19T09:12:09.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.802+0000 7fea75695740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ea8084e1f0 con 0x55ea80841b40 2024-09-19T09:12:09.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.802+0000 7fea75695740 10 client.0 _ll_register_callbacks cb 0x55ea804b48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:09.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.802+0000 7fea71058640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea808405d0 0x55ea8084ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:09.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.802+0000 7fea71058640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea808405d0 0x55ea8084ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46970/0 (socket says 172.21.15.120:46970) 2024-09-19T09:12:09.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.802+0000 7fea71058640 1 -- 172.21.15.120:0/2793093884 learned_addr learned my addr 172.21.15.120:0/2793093884 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:09.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.802+0000 7fea70857640 1 --2- 172.21.15.120:0/2793093884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea80841b40 0x55ea80848000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:09.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.802+0000 7fea71859640 1 --2- 172.21.15.120:0/2793093884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea8084b370 0x55ea808485f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:09.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.803+0000 7fea71058640 1 -- 172.21.15.120:0/2793093884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea80841b40 msgr2=0x55ea80848000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:09.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.803+0000 7fea71058640 1 --2- 172.21.15.120:0/2793093884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea80841b40 0x55ea80848000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.803+0000 7fea71058640 1 -- 172.21.15.120:0/2793093884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea8084b370 msgr2=0x55ea808485f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:09.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.803+0000 7fea71058640 1 --2- 172.21.15.120:0/2793093884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea8084b370 0x55ea808485f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.803+0000 7fea71058640 1 -- 172.21.15.120:0/2793093884 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea805173c0 con 0x55ea808405d0 2024-09-19T09:12:09.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.803+0000 7fea70857640 1 --2- 172.21.15.120:0/2793093884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea80841b40 0x55ea80848000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:09.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.803+0000 7fea71058640 1 --2- 172.21.15.120:0/2793093884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea808405d0 0x55ea8084ad00 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fea68098d50 tx=0x7fea68007850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d3d4450adb5405 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:09.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.803+0000 7fea54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:09.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.803+0000 7fea54ff9640 1 -- 172.21.15.120:0/2793093884 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6809bce0 con 0x55ea808405d0 2024-09-19T09:12:09.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.803+0000 7fea54ff9640 1 -- 172.21.15.120:0/2793093884 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fea680a4b60 con 0x55ea808405d0 2024-09-19T09:12:09.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.803+0000 7fea75695740 1 -- 172.21.15.120:0/2793093884 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea64026920 con 0x55ea808405d0 2024-09-19T09:12:09.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.803+0000 7fea54ff9640 1 -- 172.21.15.120:0/2793093884 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fea6809c5e0 con 0x55ea808405d0 2024-09-19T09:12:09.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.803+0000 7fea75695740 10 client.18699 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:09.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.803+0000 7fea75695740 1 -- 172.21.15.120:0/2793093884 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ea80859b40 con 0x55ea808405d0 2024-09-19T09:12:09.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.803+0000 7fea54ff9640 1 -- 172.21.15.120:0/2793093884 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea64026920 con 0x55ea808405d0 2024-09-19T09:12:09.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.803+0000 7fea54ff9640 1 -- 172.21.15.120:0/2793093884 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fea6809b7d0 con 0x55ea808405d0 2024-09-19T09:12:09.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.804+0000 7fea75695740 10 client.18699 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:09.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.804+0000 7fea75695740 10 client.18699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:12:09.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.804+0000 7fea75695740 20 client.18699 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:09.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.804+0000 7fea75695740 1 --2- 172.21.15.120:0/2793093884 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ea8085b000 0x55ea8085d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:09.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.804+0000 7fea75695740 4 client.18699 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:12:09.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.804+0000 7fea75695740 1 -- 172.21.15.120:0/2793093884 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea807b3860 con 0x55ea8085b000 2024-09-19T09:12:09.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.804+0000 7fea70857640 1 --2- 172.21.15.120:0/2793093884 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ea8085b000 0x55ea8085d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:09.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.804+0000 7fea70857640 1 --2- 172.21.15.120:0/2793093884 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ea8085b000 0x55ea8085d3f0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2aba8c6c4fc88a58 server_cookie=b114ddf75da092e0 in_seq=0 out_seq=0 2024-09-19T09:12:09.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.804+0000 7fea54ff9640 10 client.18699 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:09.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.805+0000 7fea54ff9640 1 -- 172.21.15.120:0/2793093884 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ea807b3860 con 0x55ea8085b000 2024-09-19T09:12:09.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.805+0000 7fea54ff9640 10 client.18699 handle_command_reply: tid=0 2024-09-19T09:12:09.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.805+0000 7fea75695740 1 client.18699 shutdown 2024-09-19T09:12:09.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.805+0000 7fea75695740 1 -- 172.21.15.120:0/2793093884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea808405d0 msgr2=0x55ea8084ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:09.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.805+0000 7fea75695740 1 --2- 172.21.15.120:0/2793093884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea808405d0 0x55ea8084ad00 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fea68098d50 tx=0x7fea68007850 comp rx=0 tx=0).stop 2024-09-19T09:12:09.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.805+0000 7fea75695740 1 -- 172.21.15.120:0/2793093884 shutdown_connections 2024-09-19T09:12:09.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.805+0000 7fea75695740 1 --2- 172.21.15.120:0/2793093884 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ea8085b000 0x55ea8085d3f0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.805+0000 7fea75695740 1 --2- 172.21.15.120:0/2793093884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea8084b370 0x55ea808485f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.805+0000 7fea75695740 1 --2- 172.21.15.120:0/2793093884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea80841b40 0x55ea80848000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.805+0000 7fea54ff9640 0 client.18699 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:09.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.805+0000 7fea75695740 1 --2- 172.21.15.120:0/2793093884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea808405d0 0x55ea8084ad00 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.805+0000 7fea75695740 1 -- 172.21.15.120:0/2793093884 >> 172.21.15.120:0/2793093884 conn(0x55ea80809310 msgr2=0x55ea80842960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:09.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.805+0000 7fea75695740 1 -- 172.21.15.120:0/2793093884 shutdown_connections 2024-09-19T09:12:09.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.805+0000 7fea75695740 1 -- 172.21.15.120:0/2793093884 wait complete. 2024-09-19T09:12:09.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.805+0000 7fea75695740 20 client.18699 trim_cache size 0 max 16384 2024-09-19T09:12:09.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.819+0000 7fea75695740 1 Processor -- start 2024-09-19T09:12:09.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.819+0000 7fea75695740 1 -- start start 2024-09-19T09:12:09.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.820+0000 7fea75695740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea808405d0 0x55ea8083fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:09.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.820+0000 7fea75695740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea80841b40 0x55ea80843120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:09.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.820+0000 7fea75695740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea8084b370 0x55ea80843660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:09.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.820+0000 7fea75695740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fea6c1ad4d0 con 0x55ea8084b370 2024-09-19T09:12:09.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.820+0000 7fea75695740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fea6c1af010 con 0x55ea80841b40 2024-09-19T09:12:09.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.820+0000 7fea75695740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fea6c1adba0 con 0x55ea808405d0 2024-09-19T09:12:09.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.820+0000 7fea71859640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea8084b370 0x55ea80843660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:09.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.820+0000 7fea71058640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea808405d0 0x55ea8083fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:09.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.820+0000 7fea71859640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea8084b370 0x55ea80843660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46980/0 (socket says 172.21.15.120:46980) 2024-09-19T09:12:09.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.820+0000 7fea71859640 1 -- 172.21.15.120:0/1732411428 learned_addr learned my addr 172.21.15.120:0/1732411428 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:09.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.820+0000 7fea70857640 1 --2- 172.21.15.120:0/1732411428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea80841b40 0x55ea80843120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:09.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.820+0000 7fea71058640 1 -- 172.21.15.120:0/1732411428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea80841b40 msgr2=0x55ea80843120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:09.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.820+0000 7fea71058640 1 --2- 172.21.15.120:0/1732411428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea80841b40 0x55ea80843120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.820+0000 7fea71058640 1 -- 172.21.15.120:0/1732411428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea8084b370 msgr2=0x55ea80843660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:09.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.820+0000 7fea71058640 1 --2- 172.21.15.120:0/1732411428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea8084b370 0x55ea80843660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.820+0000 7fea71058640 1 -- 172.21.15.120:0/1732411428 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea80846fb0 con 0x55ea808405d0 2024-09-19T09:12:09.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.820+0000 7fea71859640 1 --2- 172.21.15.120:0/1732411428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea8084b370 0x55ea80843660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:09.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.820+0000 7fea71058640 1 --2- 172.21.15.120:0/1732411428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea808405d0 0x55ea8083fe00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fea6809c080 tx=0x7fea6809a890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1f5c1f8f34fe4d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:09.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.821+0000 7fea56ffd640 1 -- 172.21.15.120:0/1732411428 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6809ca80 con 0x55ea808405d0 2024-09-19T09:12:09.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.821+0000 7fea56ffd640 1 -- 172.21.15.120:0/1732411428 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea6800fae0 con 0x55ea808405d0 2024-09-19T09:12:09.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.821+0000 7fea56ffd640 1 -- 172.21.15.120:0/1732411428 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea680cd8e0 con 0x55ea808405d0 2024-09-19T09:12:09.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.821+0000 7fea75695740 1 -- 172.21.15.120:0/1732411428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea808405d0 msgr2=0x55ea8083fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:09.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.821+0000 7fea75695740 1 --2- 172.21.15.120:0/1732411428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea808405d0 0x55ea8083fe00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fea6809c080 tx=0x7fea6809a890 comp rx=0 tx=0).stop 2024-09-19T09:12:09.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.821+0000 7fea75695740 1 -- 172.21.15.120:0/1732411428 shutdown_connections 2024-09-19T09:12:09.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.821+0000 7fea75695740 1 --2- 172.21.15.120:0/1732411428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea8084b370 0x55ea80843660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.821+0000 7fea75695740 1 --2- 172.21.15.120:0/1732411428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea80841b40 0x55ea80843120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.821+0000 7fea75695740 1 --2- 172.21.15.120:0/1732411428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea808405d0 0x55ea8083fe00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.821+0000 7fea75695740 1 -- 172.21.15.120:0/1732411428 >> 172.21.15.120:0/1732411428 conn(0x55ea80809310 msgr2=0x55ea8083bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:09.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.821+0000 7fea75695740 1 -- 172.21.15.120:0/1732411428 shutdown_connections 2024-09-19T09:12:09.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.821+0000 7fea75695740 1 -- 172.21.15.120:0/1732411428 wait complete. 2024-09-19T09:12:09.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.822+0000 7fea75695740 1 -- start start 2024-09-19T09:12:09.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.822+0000 7fea75695740 1 Processor -- start 2024-09-19T09:12:09.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.822+0000 7fea75695740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea808405d0 0x55ea8084f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:09.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.822+0000 7fea75695740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea80841b40 0x55ea8084f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:09.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.822+0000 7fea75695740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea8084b370 0x55ea8084fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:09.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.822+0000 7fea75695740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fea6c1ae940 con 0x55ea808405d0 2024-09-19T09:12:09.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.822+0000 7fea75695740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fea6c1ae270 con 0x55ea80841b40 2024-09-19T09:12:09.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.822+0000 7fea75695740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fea6c1ab490 con 0x55ea8084b370 2024-09-19T09:12:09.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.822+0000 7fea75695740 10 client.0 _ll_register_callbacks cb 0x7fea68007f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:09.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.822+0000 7fea71058640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea808405d0 0x55ea8084f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:09.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.822+0000 7fea71058640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea808405d0 0x55ea8084f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46988/0 (socket says 172.21.15.120:46988) 2024-09-19T09:12:09.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.822+0000 7fea71058640 1 -- 172.21.15.120:0/4067801702 learned_addr learned my addr 172.21.15.120:0/4067801702 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:09.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.822+0000 7fea70857640 1 --2- 172.21.15.120:0/4067801702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea80841b40 0x55ea8084f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:09.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.822+0000 7fea71859640 1 --2- 172.21.15.120:0/4067801702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea8084b370 0x55ea8084fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:09.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.823+0000 7fea71058640 1 -- 172.21.15.120:0/4067801702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea8084b370 msgr2=0x55ea8084fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:09.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.823+0000 7fea71058640 1 --2- 172.21.15.120:0/4067801702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea8084b370 0x55ea8084fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.823+0000 7fea71058640 1 -- 172.21.15.120:0/4067801702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea80841b40 msgr2=0x55ea8084f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:09.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.823+0000 7fea71058640 1 --2- 172.21.15.120:0/4067801702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea80841b40 0x55ea8084f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.823+0000 7fea71058640 1 -- 172.21.15.120:0/4067801702 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea80848960 con 0x55ea808405d0 2024-09-19T09:12:09.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.823+0000 7fea71058640 1 --2- 172.21.15.120:0/4067801702 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea808405d0 0x55ea8084f380 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fea68007980 tx=0x7fea68000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2067386bafb5f61c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:09.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.823+0000 7fea54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:09.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.823+0000 7fea54ff9640 1 -- 172.21.15.120:0/4067801702 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea680a0730 con 0x55ea808405d0 2024-09-19T09:12:09.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.823+0000 7fea54ff9640 1 -- 172.21.15.120:0/4067801702 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fea680980d0 con 0x55ea808405d0 2024-09-19T09:12:09.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.823+0000 7fea75695740 1 -- 172.21.15.120:0/4067801702 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea6800fae0 con 0x55ea808405d0 2024-09-19T09:12:09.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.823+0000 7fea54ff9640 1 -- 172.21.15.120:0/4067801702 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fea680a4ff0 con 0x55ea808405d0 2024-09-19T09:12:09.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.823+0000 7fea75695740 10 client.18705 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:09.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.823+0000 7fea75695740 1 -- 172.21.15.120:0/4067801702 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ea808c5ef0 con 0x55ea808405d0 2024-09-19T09:12:09.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.823+0000 7fea54ff9640 1 -- 172.21.15.120:0/4067801702 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea6800fae0 con 0x55ea808405d0 2024-09-19T09:12:09.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.823+0000 7fea54ff9640 1 -- 172.21.15.120:0/4067801702 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fea680a0150 con 0x55ea808405d0 2024-09-19T09:12:09.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.823+0000 7fea75695740 10 client.18705 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:09.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.823+0000 7fea75695740 10 client.18705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:12:09.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.823+0000 7fea75695740 20 client.18705 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:09.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.823+0000 7fea75695740 1 --2- 172.21.15.120:0/4067801702 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ea808c6370 0x55ea808c8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:09.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.823+0000 7fea75695740 4 client.18705 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:12:09.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.823+0000 7fea75695740 1 -- 172.21.15.120:0/4067801702 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ea807add70 con 0x55ea808c6370 2024-09-19T09:12:09.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.824+0000 7fea70857640 1 --2- 172.21.15.120:0/4067801702 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ea808c6370 0x55ea808c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:09.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.824+0000 7fea54ff9640 10 client.18705 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:09.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.824+0000 7fea70857640 1 --2- 172.21.15.120:0/4067801702 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ea808c6370 0x55ea808c8760 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60e50d77381b7508 server_cookie=646b659f7c0daddb in_seq=0 out_seq=0 2024-09-19T09:12:09.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.825+0000 7fea54ff9640 1 -- 172.21.15.120:0/4067801702 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ea807add70 con 0x55ea808c6370 2024-09-19T09:12:09.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.825+0000 7fea54ff9640 10 client.18705 handle_command_reply: tid=0 2024-09-19T09:12:09.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.825+0000 7fea75695740 1 client.18705 shutdown 2024-09-19T09:12:09.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.825+0000 7fea75695740 1 -- 172.21.15.120:0/4067801702 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea808405d0 msgr2=0x55ea8084f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:09.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.825+0000 7fea75695740 1 --2- 172.21.15.120:0/4067801702 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea808405d0 0x55ea8084f380 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fea68007980 tx=0x7fea68000f40 comp rx=0 tx=0).stop 2024-09-19T09:12:09.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.825+0000 7fea75695740 1 -- 172.21.15.120:0/4067801702 shutdown_connections 2024-09-19T09:12:09.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.825+0000 7fea75695740 1 --2- 172.21.15.120:0/4067801702 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ea808c6370 0x55ea808c8760 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.825+0000 7fea75695740 1 --2- 172.21.15.120:0/4067801702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea8084b370 0x55ea8084fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.825+0000 7fea75695740 1 --2- 172.21.15.120:0/4067801702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea80841b40 0x55ea8084f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.825+0000 7fea75695740 1 --2- 172.21.15.120:0/4067801702 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea808405d0 0x55ea8084f380 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.825+0000 7fea75695740 1 -- 172.21.15.120:0/4067801702 >> 172.21.15.120:0/4067801702 conn(0x55ea80809310 msgr2=0x55ea80849860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:09.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.825+0000 7fea54ff9640 0 client.18705 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:09.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.825+0000 7fea75695740 1 -- 172.21.15.120:0/4067801702 shutdown_connections 2024-09-19T09:12:09.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.825+0000 7fea75695740 1 -- 172.21.15.120:0/4067801702 wait complete. 2024-09-19T09:12:09.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.825+0000 7fea75695740 20 client.18705 trim_cache size 0 max 16384 2024-09-19T09:12:09.836 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:12:09.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.826+0000 7fea732bc640 1 -- 172.21.15.120:0/1664456093 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fea580a3240 msgr2=0x7fea580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:09.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.826+0000 7fea732bc640 1 --2- 172.21.15.120:0/1664456093 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fea580a3240 0x7fea580a5700 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fea64023000 tx=0x7fea64024000 comp rx=0 tx=0).stop 2024-09-19T09:12:09.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.826+0000 7fea732bc640 1 -- 172.21.15.120:0/1664456093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea6c101df0 msgr2=0x7fea6c198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:09.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.826+0000 7fea732bc640 1 --2- 172.21.15.120:0/1664456093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea6c101df0 0x7fea6c198440 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fea6800cce0 tx=0x7fea6800a620 comp rx=0 tx=0).stop 2024-09-19T09:12:09.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.826+0000 7fea732bc640 1 -- 172.21.15.120:0/1664456093 shutdown_connections 2024-09-19T09:12:09.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.826+0000 7fea732bc640 1 --2- 172.21.15.120:0/1664456093 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fea580a3240 0x7fea580a5700 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.826+0000 7fea732bc640 1 --2- 172.21.15.120:0/1664456093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea6c106f20 0x7fea6c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.826+0000 7fea732bc640 1 --2- 172.21.15.120:0/1664456093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea6c1027a0 0x7fea6c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.826+0000 7fea732bc640 1 --2- 172.21.15.120:0/1664456093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea6c101df0 0x7fea6c198440 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:09.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.826+0000 7fea732bc640 1 -- 172.21.15.120:0/1664456093 >> 172.21.15.120:0/1664456093 conn(0x7fea6c0fd8f0 msgr2=0x7fea6c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:09.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.826+0000 7fea732bc640 1 -- 172.21.15.120:0/1664456093 shutdown_connections 2024-09-19T09:12:09.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:09.826+0000 7fea732bc640 1 -- 172.21.15.120:0/1664456093 wait complete. 2024-09-19T09:12:10.065 INFO:tasks.workunit.client.0.smithi120.stdout: 1 888299 9.80 MB/sec execute 365 sec latency 46.300 ms 2024-09-19T09:12:11.065 INFO:tasks.workunit.client.0.smithi120.stdout: 1 891207 9.82 MB/sec execute 366 sec latency 62.066 ms 2024-09-19T09:12:11.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:10 smithi196 ceph-mon[35150]: pgmap v622: 97 pgs: 97 active+clean; 50 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.2 MiB/s wr, 450 op/s 2024-09-19T09:12:11.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:10 smithi153 ceph-mon[35071]: pgmap v622: 97 pgs: 97 active+clean; 50 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.2 MiB/s wr, 450 op/s 2024-09-19T09:12:11.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:10 smithi120 ceph-mon[31150]: pgmap v622: 97 pgs: 97 active+clean; 50 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.2 MiB/s wr, 450 op/s 2024-09-19T09:12:11.981 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:12:11.981 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:12:12.065 INFO:tasks.workunit.client.0.smithi120.stdout: 1 893717 9.82 MB/sec execute 367 sec latency 81.153 ms 2024-09-19T09:12:12.238 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:12:12.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.751+0000 7f73e907f640 1 -- 172.21.15.120:0/2070849820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e41036e0 msgr2=0x7f73e4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:12.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.751+0000 7f73e907f640 1 --2- 172.21.15.120:0/2070849820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e41036e0 0x7f73e4107b90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009900 tx=0x7f73d802a410 comp rx=0 tx=0).stop 2024-09-19T09:12:12.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.751+0000 7f73e907f640 1 -- 172.21.15.120:0/2070849820 shutdown_connections 2024-09-19T09:12:12.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.751+0000 7f73e907f640 1 --2- 172.21.15.120:0/2070849820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73e41082a0 0x7f73e410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.751+0000 7f73e907f640 1 --2- 172.21.15.120:0/2070849820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e41036e0 0x7f73e4107b90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.751+0000 7f73e907f640 1 --2- 172.21.15.120:0/2070849820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73e4102d30 0x7f73e4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.751+0000 7f73e907f640 1 -- 172.21.15.120:0/2070849820 >> 172.21.15.120:0/2070849820 conn(0x7f73e40fc7d0 msgr2=0x7f73e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:12.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.751+0000 7f73e907f640 1 -- 172.21.15.120:0/2070849820 shutdown_connections 2024-09-19T09:12:12.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.751+0000 7f73e907f640 1 -- 172.21.15.120:0/2070849820 wait complete. 2024-09-19T09:12:12.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.752+0000 7f73e907f640 1 Processor -- start 2024-09-19T09:12:12.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.752+0000 7f73e907f640 1 -- start start 2024-09-19T09:12:12.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.752+0000 7f73e907f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73e4102d30 0x7f73e41985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:12.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.752+0000 7f73e907f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73e41036e0 0x7f73e4198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:12.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.752+0000 7f73e907f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e41082a0 0x7f73e419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:12.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.752+0000 7f73e907f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f73e41120a0 con 0x7f73e41036e0 2024-09-19T09:12:12.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.752+0000 7f73e907f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f73e4111f20 con 0x7f73e4102d30 2024-09-19T09:12:12.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.752+0000 7f73e907f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f73e4112220 con 0x7f73e41082a0 2024-09-19T09:12:12.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.752+0000 7f73e259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73e41036e0 0x7f73e4198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:12.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.753+0000 7f73e259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73e41036e0 0x7f73e4198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56018/0 (socket says 172.21.15.120:56018) 2024-09-19T09:12:12.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.753+0000 7f73e259c640 1 -- 172.21.15.120:0/4085745150 learned_addr learned my addr 172.21.15.120:0/4085745150 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:12.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.753+0000 7f73e2d9d640 1 --2- 172.21.15.120:0/4085745150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73e4102d30 0x7f73e41985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:12.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.753+0000 7f73e359e640 1 --2- 172.21.15.120:0/4085745150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e41082a0 0x7f73e419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:12.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.753+0000 7f73e259c640 1 -- 172.21.15.120:0/4085745150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e41082a0 msgr2=0x7f73e419ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:12.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.753+0000 7f73e259c640 1 --2- 172.21.15.120:0/4085745150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e41082a0 0x7f73e419ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.753+0000 7f73e259c640 1 -- 172.21.15.120:0/4085745150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73e4102d30 msgr2=0x7f73e41985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:12.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.753+0000 7f73e259c640 1 --2- 172.21.15.120:0/4085745150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73e4102d30 0x7f73e41985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.753+0000 7f73e259c640 1 -- 172.21.15.120:0/4085745150 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e419d590 con 0x7f73e41036e0 2024-09-19T09:12:12.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.753+0000 7f73e2d9d640 1 --2- 172.21.15.120:0/4085745150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73e4102d30 0x7f73e41985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:12.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.753+0000 7f73e259c640 1 --2- 172.21.15.120:0/4085745150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73e41036e0 0x7f73e4198b20 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f73d8005940 tx=0x7f73d8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:12.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.753+0000 7f73cbfff640 1 -- 172.21.15.120:0/4085745150 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d803f070 con 0x7f73e41036e0 2024-09-19T09:12:12.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.753+0000 7f73e907f640 1 -- 172.21.15.120:0/4085745150 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e419d820 con 0x7f73e41036e0 2024-09-19T09:12:12.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.753+0000 7f73cbfff640 1 -- 172.21.15.120:0/4085745150 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d8031c30 con 0x7f73e41036e0 2024-09-19T09:12:12.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.753+0000 7f73cbfff640 1 -- 172.21.15.120:0/4085745150 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8035030 con 0x7f73e41036e0 2024-09-19T09:12:12.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.753+0000 7f73e907f640 1 -- 172.21.15.120:0/4085745150 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e41a5160 con 0x7f73e41036e0 2024-09-19T09:12:12.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.756+0000 7f73cbfff640 1 -- 172.21.15.120:0/4085745150 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f73d8042070 con 0x7f73e41036e0 2024-09-19T09:12:12.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.756+0000 7f73eb458740 1 Processor -- start 2024-09-19T09:12:12.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.756+0000 7f73eb458740 1 -- start start 2024-09-19T09:12:12.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.756+0000 7f73eb458740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d40b00bb60 0x55d40b00bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:12.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.756+0000 7f73eb458740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40b00c480 0x55d40b016800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:12.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.756+0000 7f73eb458740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d40b016d40 0x55d40b019130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:12.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.757+0000 7f73eb458740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d40af71280 con 0x55d40b00bb60 2024-09-19T09:12:12.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.757+0000 7f73eb458740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d40af18b10 con 0x55d40b00c480 2024-09-19T09:12:12.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.757+0000 7f73eb458740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d40af6d240 con 0x55d40b016d40 2024-09-19T09:12:12.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.757+0000 7f73e2d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40b00c480 0x55d40b016800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:12.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.757+0000 7f73e2d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40b00c480 0x55d40b016800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:51908/0 (socket says 172.21.15.120:51908) 2024-09-19T09:12:12.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.757+0000 7f73e2d9d640 1 -- 172.21.15.120:0/870340690 learned_addr learned my addr 172.21.15.120:0/870340690 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:12.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.757+0000 7f73e359e640 1 --2- 172.21.15.120:0/870340690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d40b00bb60 0x55d40b00bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:12.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.757+0000 7f73e259c640 1 --2- 172.21.15.120:0/870340690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d40b016d40 0x55d40b019130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:12.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.757+0000 7f73e2d9d640 1 -- 172.21.15.120:0/870340690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d40b016d40 msgr2=0x55d40b019130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:12.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.757+0000 7f73e2d9d640 1 --2- 172.21.15.120:0/870340690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d40b016d40 0x55d40b019130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.757+0000 7f73e2d9d640 1 -- 172.21.15.120:0/870340690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d40b00bb60 msgr2=0x55d40b00bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:12.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.757+0000 7f73cbfff640 1 --2- 172.21.15.120:0/4085745150 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f73cc0a3240 0x7f73cc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:12.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.757+0000 7f73e2d9d640 1 --2- 172.21.15.120:0/870340690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d40b00bb60 0x55d40b00bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.757+0000 7f73e2d9d640 1 -- 172.21.15.120:0/870340690 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d40af251f0 con 0x55d40b00c480 2024-09-19T09:12:12.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.757+0000 7f73e259c640 1 --2- 172.21.15.120:0/870340690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d40b016d40 0x55d40b019130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:12:12.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.757+0000 7f73e359e640 1 --2- 172.21.15.120:0/870340690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d40b00bb60 0x55d40b00bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:12.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.757+0000 7f73cbfff640 1 -- 172.21.15.120:0/4085745150 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f73d80effb0 con 0x7f73e41036e0 2024-09-19T09:12:12.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.757+0000 7f73e2d9d640 1 --2- 172.21.15.120:0/870340690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40b00c480 0x55d40b016800 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f73d40115c0 tx=0x7f73d4011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6dcc14992bbe8a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:12.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.757+0000 7f73e259c640 1 --2- 172.21.15.120:0/4085745150 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f73cc0a3240 0x7f73cc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:12.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.758+0000 7f73c97fa640 1 -- 172.21.15.120:0/870340690 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d400f640 con 0x55d40b00c480 2024-09-19T09:12:12.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.758+0000 7f73c97fa640 1 -- 172.21.15.120:0/870340690 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d400fcc0 con 0x55d40b00c480 2024-09-19T09:12:12.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.758+0000 7f73c97fa640 1 -- 172.21.15.120:0/870340690 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d4019e60 con 0x55d40b00c480 2024-09-19T09:12:12.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.758+0000 7f73eb458740 1 -- 172.21.15.120:0/870340690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40b00c480 msgr2=0x55d40b016800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:12.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.758+0000 7f73e259c640 1 --2- 172.21.15.120:0/4085745150 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f73cc0a3240 0x7f73cc0a5700 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f73d80b8b80 tx=0x7f73d8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:12.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.758+0000 7f73eb458740 1 --2- 172.21.15.120:0/870340690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40b00c480 0x55d40b016800 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f73d40115c0 tx=0x7f73d4011a80 comp rx=0 tx=0).stop 2024-09-19T09:12:12.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.758+0000 7f73eb458740 1 -- 172.21.15.120:0/870340690 shutdown_connections 2024-09-19T09:12:12.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.758+0000 7f73eb458740 1 --2- 172.21.15.120:0/870340690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d40b016d40 0x55d40b019130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.758+0000 7f73eb458740 1 --2- 172.21.15.120:0/870340690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40b00c480 0x55d40b016800 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.759+0000 7f73eb458740 1 --2- 172.21.15.120:0/870340690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d40b00bb60 0x55d40b00bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.759+0000 7f73eb458740 1 -- 172.21.15.120:0/870340690 >> 172.21.15.120:0/870340690 conn(0x55d40b0080f0 msgr2=0x55d40b0084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:12.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.759+0000 7f73eb458740 1 -- 172.21.15.120:0/870340690 shutdown_connections 2024-09-19T09:12:12.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.759+0000 7f73eb458740 1 -- 172.21.15.120:0/870340690 wait complete. 2024-09-19T09:12:12.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.759+0000 7f73eb458740 1 -- start start 2024-09-19T09:12:12.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.759+0000 7f73eb458740 1 Processor -- start 2024-09-19T09:12:12.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.759+0000 7f73eb458740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40b00bb60 0x55d40b013c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:12.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.759+0000 7f73eb458740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d40b00c480 0x55d40b014140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:12.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.759+0000 7f73eb458740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d40b016d40 0x55d40b014680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:12.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.759+0000 7f73eb458740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d40b019910 con 0x55d40b00c480 2024-09-19T09:12:12.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.759+0000 7f73eb458740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d40af2bf10 con 0x55d40b00bb60 2024-09-19T09:12:12.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.759+0000 7f73eb458740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d40b019a90 con 0x55d40b016d40 2024-09-19T09:12:12.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.759+0000 7f73eb458740 10 client.0 _ll_register_callbacks cb 0x55d40aef4a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:12.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.759+0000 7f73e359e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40b00bb60 0x55d40b013c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:12.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.759+0000 7f73e2d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d40b00c480 0x55d40b014140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:12.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.759+0000 7f73e359e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40b00bb60 0x55d40b013c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:51910/0 (socket says 172.21.15.120:51910) 2024-09-19T09:12:12.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.759+0000 7f73e359e640 1 -- 172.21.15.120:0/3717921720 learned_addr learned my addr 172.21.15.120:0/3717921720 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:12.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.760+0000 7f73e359e640 1 --2- 172.21.15.120:0/3717921720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d40b016d40 0x55d40b014680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:12.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.760+0000 7f73e2d9d640 1 -- 172.21.15.120:0/3717921720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d40b016d40 msgr2=0x55d40b014680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:12.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.760+0000 7f73e2d9d640 1 --2- 172.21.15.120:0/3717921720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d40b016d40 0x55d40b014680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.760+0000 7f73e2d9d640 1 -- 172.21.15.120:0/3717921720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40b00bb60 msgr2=0x55d40b013c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:12.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.760+0000 7f73e2d9d640 1 --2- 172.21.15.120:0/3717921720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40b00bb60 0x55d40b013c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.760+0000 7f73e2d9d640 1 -- 172.21.15.120:0/3717921720 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d40ace26a0 con 0x55d40b00c480 2024-09-19T09:12:12.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.760+0000 7f73e359e640 1 --2- 172.21.15.120:0/3717921720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40b00bb60 0x55d40b013c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:12.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.760+0000 7f73e2d9d640 1 --2- 172.21.15.120:0/3717921720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d40b00c480 0x55d40b014140 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f73d40115c0 tx=0x7f73d4019e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de8d7f82bd4b2cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:12.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.760+0000 7f73c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:12.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.760+0000 7f73c6ff5640 1 -- 172.21.15.120:0/3717921720 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d4026020 con 0x55d40b00c480 2024-09-19T09:12:12.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.760+0000 7f73c6ff5640 1 -- 172.21.15.120:0/3717921720 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f73d4004250 con 0x55d40b00c480 2024-09-19T09:12:12.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.760+0000 7f73eb458740 1 -- 172.21.15.120:0/3717921720 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73d400fcc0 con 0x55d40b00c480 2024-09-19T09:12:12.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.760+0000 7f73c6ff5640 1 -- 172.21.15.120:0/3717921720 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f73d4004450 con 0x55d40b00c480 2024-09-19T09:12:12.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.761+0000 7f73eb458740 10 client.18717 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:12.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.761+0000 7f73eb458740 1 -- 172.21.15.120:0/3717921720 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d40b0254f0 con 0x55d40b00c480 2024-09-19T09:12:12.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.761+0000 7f73c6ff5640 1 -- 172.21.15.120:0/3717921720 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d400fcc0 con 0x55d40b00c480 2024-09-19T09:12:12.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.761+0000 7f73c6ff5640 1 -- 172.21.15.120:0/3717921720 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f73d4003890 con 0x55d40b00c480 2024-09-19T09:12:12.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.761+0000 7f73eb458740 10 client.18717 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:12.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.761+0000 7f73eb458740 10 client.18717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:12:12.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.761+0000 7f73eb458740 20 client.18717 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:12.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.761+0000 7f73eb458740 1 --2- 172.21.15.120:0/3717921720 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d40b0268a0 0x55d40b028c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:12.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.761+0000 7f73eb458740 4 client.18717 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:12:12.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.761+0000 7f73eb458740 1 -- 172.21.15.120:0/3717921720 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d40af7e7f0 con 0x55d40b0268a0 2024-09-19T09:12:12.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.761+0000 7f73e359e640 1 --2- 172.21.15.120:0/3717921720 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d40b0268a0 0x55d40b028c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:12.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.762+0000 7f73e359e640 1 --2- 172.21.15.120:0/3717921720 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d40b0268a0 0x55d40b028c90 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed30bb51fff4e123 server_cookie=3a00a3b097f36150 in_seq=0 out_seq=0 2024-09-19T09:12:12.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.762+0000 7f73c6ff5640 10 client.18717 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:12.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.762+0000 7f73c6ff5640 1 -- 172.21.15.120:0/3717921720 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d40af7e7f0 con 0x55d40b0268a0 2024-09-19T09:12:12.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.762+0000 7f73c6ff5640 10 client.18717 handle_command_reply: tid=0 2024-09-19T09:12:12.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.762+0000 7f73eb458740 1 client.18717 shutdown 2024-09-19T09:12:12.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.763+0000 7f73eb458740 1 -- 172.21.15.120:0/3717921720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d40b00c480 msgr2=0x55d40b014140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:12.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.763+0000 7f73eb458740 1 --2- 172.21.15.120:0/3717921720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d40b00c480 0x55d40b014140 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f73d40115c0 tx=0x7f73d4019e10 comp rx=0 tx=0).stop 2024-09-19T09:12:12.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.763+0000 7f73eb458740 1 -- 172.21.15.120:0/3717921720 shutdown_connections 2024-09-19T09:12:12.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.763+0000 7f73eb458740 1 --2- 172.21.15.120:0/3717921720 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d40b0268a0 0x55d40b028c90 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.763+0000 7f73eb458740 1 --2- 172.21.15.120:0/3717921720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d40b016d40 0x55d40b014680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.763+0000 7f73eb458740 1 --2- 172.21.15.120:0/3717921720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d40b00c480 0x55d40b014140 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.763+0000 7f73eb458740 1 --2- 172.21.15.120:0/3717921720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40b00bb60 0x55d40b013c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.763+0000 7f73eb458740 1 -- 172.21.15.120:0/3717921720 >> 172.21.15.120:0/3717921720 conn(0x55d40b0080f0 msgr2=0x55d40b00de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:12.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.763+0000 7f73c6ff5640 0 client.18717 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:12.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.763+0000 7f73eb458740 1 -- 172.21.15.120:0/3717921720 shutdown_connections 2024-09-19T09:12:12.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.763+0000 7f73eb458740 1 -- 172.21.15.120:0/3717921720 wait complete. 2024-09-19T09:12:12.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.763+0000 7f73eb458740 20 client.18717 trim_cache size 0 max 16384 2024-09-19T09:12:12.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.780+0000 7f73eb458740 1 Processor -- start 2024-09-19T09:12:12.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.780+0000 7f73eb458740 1 -- start start 2024-09-19T09:12:12.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.781+0000 7f73eb458740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d40b00bb60 0x55d40b00e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:12.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.781+0000 7f73eb458740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d40b00c480 0x55d40b015a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:12.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.781+0000 7f73eb458740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40b016d40 0x55d40b02e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:12.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.781+0000 7f73eb458740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f73e41ad630 con 0x55d40b00c480 2024-09-19T09:12:12.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.781+0000 7f73eb458740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f73e41af170 con 0x55d40b016d40 2024-09-19T09:12:12.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.781+0000 7f73eb458740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f73e41add00 con 0x55d40b00bb60 2024-09-19T09:12:12.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.781+0000 7f73e2d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d40b00c480 0x55d40b015a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:12.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.781+0000 7f73e2d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d40b00c480 0x55d40b015a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56034/0 (socket says 172.21.15.120:56034) 2024-09-19T09:12:12.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.781+0000 7f73e2d9d640 1 -- 172.21.15.120:0/2344342483 learned_addr learned my addr 172.21.15.120:0/2344342483 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:12.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.781+0000 7f73e359e640 1 --2- 172.21.15.120:0/2344342483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d40b00bb60 0x55d40b00e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:12.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.781+0000 7f73e359e640 1 --2- 172.21.15.120:0/2344342483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40b016d40 0x55d40b02e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:12.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.781+0000 7f73e2d9d640 1 -- 172.21.15.120:0/2344342483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d40b00bb60 msgr2=0x55d40b00e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:12.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.781+0000 7f73e2d9d640 1 --2- 172.21.15.120:0/2344342483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d40b00bb60 0x55d40b00e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.781+0000 7f73e2d9d640 1 -- 172.21.15.120:0/2344342483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40b016d40 msgr2=0x55d40b02e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:12.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.781+0000 7f73e2d9d640 1 --2- 172.21.15.120:0/2344342483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40b016d40 0x55d40b02e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.781+0000 7f73e2d9d640 1 -- 172.21.15.120:0/2344342483 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d40b00b8e0 con 0x55d40b00c480 2024-09-19T09:12:12.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.781+0000 7f73e2d9d640 1 --2- 172.21.15.120:0/2344342483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d40b00c480 0x55d40b015a60 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f73d40058c0 tx=0x7f73d4009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee01cc51b87a8e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:12.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.782+0000 7f73c8ff9640 1 -- 172.21.15.120:0/2344342483 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d4026030 con 0x55d40b00c480 2024-09-19T09:12:12.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.782+0000 7f73c8ff9640 1 -- 172.21.15.120:0/2344342483 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d4019720 con 0x55d40b00c480 2024-09-19T09:12:12.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.782+0000 7f73c8ff9640 1 -- 172.21.15.120:0/2344342483 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d4005040 con 0x55d40b00c480 2024-09-19T09:12:12.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.782+0000 7f73eb458740 1 -- 172.21.15.120:0/2344342483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d40b00c480 msgr2=0x55d40b015a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:12.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.782+0000 7f73eb458740 1 --2- 172.21.15.120:0/2344342483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d40b00c480 0x55d40b015a60 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f73d40058c0 tx=0x7f73d4009f60 comp rx=0 tx=0).stop 2024-09-19T09:12:12.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.782+0000 7f73eb458740 1 -- 172.21.15.120:0/2344342483 shutdown_connections 2024-09-19T09:12:12.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.782+0000 7f73eb458740 1 --2- 172.21.15.120:0/2344342483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40b016d40 0x55d40b02e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.782+0000 7f73eb458740 1 --2- 172.21.15.120:0/2344342483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d40b00c480 0x55d40b015a60 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.782+0000 7f73eb458740 1 --2- 172.21.15.120:0/2344342483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d40b00bb60 0x55d40b00e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.782+0000 7f73eb458740 1 -- 172.21.15.120:0/2344342483 >> 172.21.15.120:0/2344342483 conn(0x55d40b0080f0 msgr2=0x55d40b015350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:12.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.783+0000 7f73eb458740 1 -- 172.21.15.120:0/2344342483 shutdown_connections 2024-09-19T09:12:12.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.783+0000 7f73eb458740 1 -- 172.21.15.120:0/2344342483 wait complete. 2024-09-19T09:12:12.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.783+0000 7f73eb458740 1 -- start start 2024-09-19T09:12:12.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.783+0000 7f73eb458740 1 Processor -- start 2024-09-19T09:12:12.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.783+0000 7f73eb458740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d40b00bb60 0x55d40b08c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:12.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.783+0000 7f73eb458740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40b00c480 0x55d40b01ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:12.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.783+0000 7f73eb458740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d40b016d40 0x55d40b01f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:12.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.783+0000 7f73eb458740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f73e41aeaa0 con 0x55d40b00bb60 2024-09-19T09:12:12.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.783+0000 7f73eb458740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f73e41ae3d0 con 0x55d40b00c480 2024-09-19T09:12:12.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.783+0000 7f73eb458740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f73e41ab5f0 con 0x55d40b016d40 2024-09-19T09:12:12.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.783+0000 7f73eb458740 10 client.0 _ll_register_callbacks cb 0x7f73dc009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:12.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.783+0000 7f73e359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d40b00bb60 0x55d40b08c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:12.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.783+0000 7f73e359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d40b00bb60 0x55d40b08c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56050/0 (socket says 172.21.15.120:56050) 2024-09-19T09:12:12.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.783+0000 7f73e359e640 1 -- 172.21.15.120:0/1913538210 learned_addr learned my addr 172.21.15.120:0/1913538210 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:12.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.784+0000 7f73e2d9d640 1 --2- 172.21.15.120:0/1913538210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40b00c480 0x55d40b01ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:12.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.784+0000 7f73e359e640 1 --2- 172.21.15.120:0/1913538210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d40b016d40 0x55d40b01f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:12.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.784+0000 7f73e359e640 1 -- 172.21.15.120:0/1913538210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d40b016d40 msgr2=0x55d40b01f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:12.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.784+0000 7f73e359e640 1 --2- 172.21.15.120:0/1913538210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d40b016d40 0x55d40b01f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.784+0000 7f73e359e640 1 -- 172.21.15.120:0/1913538210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40b00c480 msgr2=0x55d40b01ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:12.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.784+0000 7f73e359e640 1 --2- 172.21.15.120:0/1913538210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40b00c480 0x55d40b01ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.784+0000 7f73e359e640 1 -- 172.21.15.120:0/1913538210 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d40b01f7e0 con 0x55d40b00bb60 2024-09-19T09:12:12.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.784+0000 7f73e2d9d640 1 --2- 172.21.15.120:0/1913538210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40b00c480 0x55d40b01ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:12:12.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.784+0000 7f73e359e640 1 --2- 172.21.15.120:0/1913538210 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d40b00bb60 0x55d40b08c970 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f73dc013b80 tx=0x7f73d4019a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4667c25b72a6d49 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:12.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.784+0000 7f73c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:12.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.784+0000 7f73c6ff5640 1 -- 172.21.15.120:0/1913538210 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc026070 con 0x55d40b00bb60 2024-09-19T09:12:12.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.784+0000 7f73c6ff5640 1 -- 172.21.15.120:0/1913538210 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f73dc003970 con 0x55d40b00bb60 2024-09-19T09:12:12.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.784+0000 7f73eb458740 1 -- 172.21.15.120:0/1913538210 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73d4019720 con 0x55d40b00bb60 2024-09-19T09:12:12.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.784+0000 7f73c6ff5640 1 -- 172.21.15.120:0/1913538210 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f73dc003b70 con 0x55d40b00bb60 2024-09-19T09:12:12.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.784+0000 7f73eb458740 10 client.18723 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:12.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.784+0000 7f73eb458740 1 -- 172.21.15.120:0/1913538210 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d40b095760 con 0x55d40b00bb60 2024-09-19T09:12:12.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.785+0000 7f73c6ff5640 1 -- 172.21.15.120:0/1913538210 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d4019720 con 0x55d40b00bb60 2024-09-19T09:12:12.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.785+0000 7f73c6ff5640 1 -- 172.21.15.120:0/1913538210 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f73dc037050 con 0x55d40b00bb60 2024-09-19T09:12:12.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.785+0000 7f73eb458740 10 client.18723 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:12.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.785+0000 7f73eb458740 10 client.18723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:12:12.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.785+0000 7f73eb458740 20 client.18723 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:12.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.785+0000 7f73eb458740 1 --2- 172.21.15.120:0/1913538210 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d40b095f00 0x55d40b098320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:12.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.785+0000 7f73eb458740 4 client.18723 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:12:12.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.785+0000 7f73eb458740 1 -- 172.21.15.120:0/1913538210 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d40af78d00 con 0x55d40b095f00 2024-09-19T09:12:12.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.785+0000 7f73e2d9d640 1 --2- 172.21.15.120:0/1913538210 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d40b095f00 0x55d40b098320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:12.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.786+0000 7f73e2d9d640 1 --2- 172.21.15.120:0/1913538210 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d40b095f00 0x55d40b098320 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac06c12faed1e3e0 server_cookie=833760fbc4a50d56 in_seq=0 out_seq=0 2024-09-19T09:12:12.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.786+0000 7f73c6ff5640 10 client.18723 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:12.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.788+0000 7f73c6ff5640 1 -- 172.21.15.120:0/1913538210 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d40af78d00 con 0x55d40b095f00 2024-09-19T09:12:12.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.788+0000 7f73c6ff5640 10 client.18723 handle_command_reply: tid=0 2024-09-19T09:12:12.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.788+0000 7f73eb458740 1 client.18723 shutdown 2024-09-19T09:12:12.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.788+0000 7f73eb458740 1 -- 172.21.15.120:0/1913538210 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d40b00bb60 msgr2=0x55d40b08c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:12.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.788+0000 7f73eb458740 1 --2- 172.21.15.120:0/1913538210 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d40b00bb60 0x55d40b08c970 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f73dc013b80 tx=0x7f73d4019a40 comp rx=0 tx=0).stop 2024-09-19T09:12:12.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.788+0000 7f73eb458740 1 -- 172.21.15.120:0/1913538210 shutdown_connections 2024-09-19T09:12:12.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.788+0000 7f73eb458740 1 --2- 172.21.15.120:0/1913538210 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d40b095f00 0x55d40b098320 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.788+0000 7f73eb458740 1 --2- 172.21.15.120:0/1913538210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d40b016d40 0x55d40b01f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.788+0000 7f73eb458740 1 --2- 172.21.15.120:0/1913538210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40b00c480 0x55d40b01ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.788+0000 7f73eb458740 1 --2- 172.21.15.120:0/1913538210 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d40b00bb60 0x55d40b08c970 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.788+0000 7f73eb458740 1 -- 172.21.15.120:0/1913538210 >> 172.21.15.120:0/1913538210 conn(0x55d40b0080f0 msgr2=0x55d40b012bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:12.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.788+0000 7f73c6ff5640 0 client.18723 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:12.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.788+0000 7f73eb458740 1 -- 172.21.15.120:0/1913538210 shutdown_connections 2024-09-19T09:12:12.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.788+0000 7f73eb458740 1 -- 172.21.15.120:0/1913538210 wait complete. 2024-09-19T09:12:12.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.788+0000 7f73eb458740 20 client.18723 trim_cache size 0 max 16384 2024-09-19T09:12:12.794 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:12:12.794 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:12:12.794 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "aa34314a-0f94-43a7-859f-82199630dea8", 2024-09-19T09:12:12.794 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:12:12.794 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:12:12.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.789+0000 7f73e907f640 1 -- 172.21.15.120:0/4085745150 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f73cc0a3240 msgr2=0x7f73cc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:12.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.789+0000 7f73e907f640 1 --2- 172.21.15.120:0/4085745150 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f73cc0a3240 0x7f73cc0a5700 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f73d80b8b80 tx=0x7f73d8000bc0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.789+0000 7f73e907f640 1 -- 172.21.15.120:0/4085745150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73e41036e0 msgr2=0x7f73e4198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:12.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.789+0000 7f73e907f640 1 --2- 172.21.15.120:0/4085745150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73e41036e0 0x7f73e4198b20 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f73d8005940 tx=0x7f73d8003c10 comp rx=0 tx=0).stop 2024-09-19T09:12:12.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.789+0000 7f73e907f640 1 -- 172.21.15.120:0/4085745150 shutdown_connections 2024-09-19T09:12:12.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.789+0000 7f73e907f640 1 --2- 172.21.15.120:0/4085745150 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f73cc0a3240 0x7f73cc0a5700 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.789+0000 7f73e907f640 1 --2- 172.21.15.120:0/4085745150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e41082a0 0x7f73e419ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.789+0000 7f73e907f640 1 --2- 172.21.15.120:0/4085745150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73e41036e0 0x7f73e4198b20 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.789+0000 7f73e907f640 1 --2- 172.21.15.120:0/4085745150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73e4102d30 0x7f73e41985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:12.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.789+0000 7f73e907f640 1 -- 172.21.15.120:0/4085745150 >> 172.21.15.120:0/4085745150 conn(0x7f73e40fc7d0 msgr2=0x7f73e4105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:12.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.789+0000 7f73e907f640 1 -- 172.21.15.120:0/4085745150 shutdown_connections 2024-09-19T09:12:12.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:12.789+0000 7f73e907f640 1 -- 172.21.15.120:0/4085745150 wait complete. 2024-09-19T09:12:13.065 INFO:tasks.workunit.client.0.smithi120.stdout: 1 896352 9.84 MB/sec execute 368 sec latency 32.943 ms 2024-09-19T09:12:13.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:12 smithi120 ceph-mon[31150]: pgmap v623: 97 pgs: 97 active+clean; 46 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 447 op/s 2024-09-19T09:12:13.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:12 smithi196 ceph-mon[35150]: pgmap v623: 97 pgs: 97 active+clean; 46 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 447 op/s 2024-09-19T09:12:13.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:12 smithi153 ceph-mon[35071]: pgmap v623: 97 pgs: 97 active+clean; 46 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 447 op/s 2024-09-19T09:12:13.552 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:12:13.799 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:12:14.065 INFO:tasks.workunit.client.0.smithi120.stdout: 1 899189 9.85 MB/sec execute 369 sec latency 20.717 ms 2024-09-19T09:12:14.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:13 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:12:14.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:13 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:12:14.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:13 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:12:14.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:13 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:12:14.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:13 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:12:14.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:13 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:12:14.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:13 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:12:14.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:13 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:12:14.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:13 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:12:14.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:13 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:12:14.219 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:13 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:12:14.219 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:13 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:12:14.219 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:13 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:12:14.219 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:13 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:12:14.219 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:13 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:12:14.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.305+0000 7fe975c15640 1 -- 172.21.15.120:0/3805096070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe970102d90 msgr2=0x7fe970103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:14.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.305+0000 7fe975c15640 1 --2- 172.21.15.120:0/3805096070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe970102d90 0x7fe970103170 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fe960009930 tx=0x7fe96002a420 comp rx=0 tx=0).stop 2024-09-19T09:12:14.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.305+0000 7fe975c15640 1 -- 172.21.15.120:0/3805096070 shutdown_connections 2024-09-19T09:12:14.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.305+0000 7fe975c15640 1 --2- 172.21.15.120:0/3805096070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe970108290 0x7fe97010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.305+0000 7fe975c15640 1 --2- 172.21.15.120:0/3805096070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe970103740 0x7fe970107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.305+0000 7fe975c15640 1 --2- 172.21.15.120:0/3805096070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe970102d90 0x7fe970103170 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.305+0000 7fe975c15640 1 -- 172.21.15.120:0/3805096070 >> 172.21.15.120:0/3805096070 conn(0x7fe9700fc7d0 msgr2=0x7fe9700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:14.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.305+0000 7fe975c15640 1 -- 172.21.15.120:0/3805096070 shutdown_connections 2024-09-19T09:12:14.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.305+0000 7fe975c15640 1 -- 172.21.15.120:0/3805096070 wait complete. 2024-09-19T09:12:14.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.306+0000 7fe975c15640 1 Processor -- start 2024-09-19T09:12:14.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.306+0000 7fe975c15640 1 -- start start 2024-09-19T09:12:14.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.306+0000 7fe975c15640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe970102d90 0x7fe9701009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:14.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.306+0000 7fe975c15640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe970103740 0x7fe970100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:14.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.306+0000 7fe975c15640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe970108290 0x7fe970101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:14.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.306+0000 7fe975c15640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe970112090 con 0x7fe970108290 2024-09-19T09:12:14.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.307+0000 7fe975c15640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe970111f10 con 0x7fe970103740 2024-09-19T09:12:14.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.307+0000 7fe975c15640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe970112210 con 0x7fe970102d90 2024-09-19T09:12:14.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.307+0000 7fe96ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe970108290 0x7fe970101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:14.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.307+0000 7fe967fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe970103740 0x7fe970100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:14.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.307+0000 7fe96ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe970108290 0x7fe970101470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56068/0 (socket says 172.21.15.120:56068) 2024-09-19T09:12:14.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.307+0000 7fe967fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe970103740 0x7fe970100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:51944/0 (socket says 172.21.15.120:51944) 2024-09-19T09:12:14.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.307+0000 7fe96ffff640 1 -- 172.21.15.120:0/2232661819 learned_addr learned my addr 172.21.15.120:0/2232661819 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:14.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.307+0000 7fe96f7fe640 1 --2- 172.21.15.120:0/2232661819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe970102d90 0x7fe9701009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:14.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.307+0000 7fe96ffff640 1 -- 172.21.15.120:0/2232661819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe970102d90 msgr2=0x7fe9701009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:14.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.307+0000 7fe96ffff640 1 --2- 172.21.15.120:0/2232661819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe970102d90 0x7fe9701009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.307+0000 7fe96ffff640 1 -- 172.21.15.120:0/2232661819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe970103740 msgr2=0x7fe970100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:14.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.307+0000 7fe96ffff640 1 --2- 172.21.15.120:0/2232661819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe970103740 0x7fe970100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.307+0000 7fe96ffff640 1 -- 172.21.15.120:0/2232661819 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9701a6b10 con 0x7fe970108290 2024-09-19T09:12:14.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.307+0000 7fe96ffff640 1 --2- 172.21.15.120:0/2232661819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe970108290 0x7fe970101470 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fe96800eb10 tx=0x7fe96800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:14.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.307+0000 7fe96d7fa640 1 -- 172.21.15.120:0/2232661819 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe968004b20 con 0x7fe970108290 2024-09-19T09:12:14.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.308+0000 7fe96d7fa640 1 -- 172.21.15.120:0/2232661819 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe968004cc0 con 0x7fe970108290 2024-09-19T09:12:14.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.308+0000 7fe975c15640 1 -- 172.21.15.120:0/2232661819 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9701a6e00 con 0x7fe970108290 2024-09-19T09:12:14.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.308+0000 7fe96d7fa640 1 -- 172.21.15.120:0/2232661819 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe968010760 con 0x7fe970108290 2024-09-19T09:12:14.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.308+0000 7fe975c15640 1 -- 172.21.15.120:0/2232661819 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9701a72c0 con 0x7fe970108290 2024-09-19T09:12:14.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.310+0000 7fe96d7fa640 1 -- 172.21.15.120:0/2232661819 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe968005020 con 0x7fe970108290 2024-09-19T09:12:14.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.310+0000 7fe977fee740 1 Processor -- start 2024-09-19T09:12:14.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.310+0000 7fe977fee740 1 -- start start 2024-09-19T09:12:14.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.310+0000 7fe977fee740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c680a34a0 0x562c680a3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:14.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.310+0000 7fe977fee740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c680a3dc0 0x562c680ae140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:14.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.310+0000 7fe977fee740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c680ae680 0x562c680b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:14.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.310+0000 7fe977fee740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562c680092e0 con 0x562c680a3dc0 2024-09-19T09:12:14.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.310+0000 7fe977fee740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562c67fb0ce0 con 0x562c680ae680 2024-09-19T09:12:14.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.310+0000 7fe977fee740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562c680052a0 con 0x562c680a34a0 2024-09-19T09:12:14.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.310+0000 7fe96f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c680a3dc0 0x562c680ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:14.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.310+0000 7fe96f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c680a3dc0 0x562c680ae140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56084/0 (socket says 172.21.15.120:56084) 2024-09-19T09:12:14.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.310+0000 7fe967fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c680a34a0 0x562c680a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:14.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.310+0000 7fe96f7fe640 1 -- 172.21.15.120:0/4152714031 learned_addr learned my addr 172.21.15.120:0/4152714031 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:14.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.311+0000 7fe967fff640 1 --2- 172.21.15.120:0/4152714031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c680ae680 0x562c680b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:14.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.311+0000 7fe96d7fa640 1 --2- 172.21.15.120:0/2232661819 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe95c0a3240 0x7fe95c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:14.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.311+0000 7fe96f7fe640 1 -- 172.21.15.120:0/4152714031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c680a34a0 msgr2=0x562c680a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:14.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.311+0000 7fe96f7fe640 1 --2- 172.21.15.120:0/4152714031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c680a34a0 0x562c680a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.311+0000 7fe96f7fe640 1 -- 172.21.15.120:0/4152714031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c680ae680 msgr2=0x562c680b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:14.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.311+0000 7fe96f7fe640 1 --2- 172.21.15.120:0/4152714031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c680ae680 0x562c680b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.311+0000 7fe96f7fe640 1 -- 172.21.15.120:0/4152714031 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c67f95440 con 0x562c680a3dc0 2024-09-19T09:12:14.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.311+0000 7fe96ffff640 1 --2- 172.21.15.120:0/2232661819 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe95c0a3240 0x7fe95c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:14.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.311+0000 7fe96f7fe640 1 --2- 172.21.15.120:0/4152714031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c680a3dc0 0x562c680ae140 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fe96002a940 tx=0x7fe960005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5ec14dc4523f22 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:14.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.311+0000 7fe96d7fa640 1 -- 172.21.15.120:0/2232661819 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fe9680cfdf0 con 0x7fe970108290 2024-09-19T09:12:14.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.311+0000 7fe965ffb640 1 -- 172.21.15.120:0/4152714031 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe960040070 con 0x562c680a3dc0 2024-09-19T09:12:14.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.311+0000 7fe965ffb640 1 -- 172.21.15.120:0/4152714031 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe960031870 con 0x562c680a3dc0 2024-09-19T09:12:14.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.311+0000 7fe965ffb640 1 -- 172.21.15.120:0/4152714031 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe960035040 con 0x562c680a3dc0 2024-09-19T09:12:14.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.312+0000 7fe977fee740 1 -- 172.21.15.120:0/4152714031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c680a3dc0 msgr2=0x562c680ae140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:14.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.312+0000 7fe96ffff640 1 --2- 172.21.15.120:0/2232661819 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe95c0a3240 0x7fe95c0a5700 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe968098de0 tx=0x7fe96806b150 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:14.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.312+0000 7fe977fee740 1 --2- 172.21.15.120:0/4152714031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c680a3dc0 0x562c680ae140 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fe96002a940 tx=0x7fe960005bd0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.312+0000 7fe977fee740 1 -- 172.21.15.120:0/4152714031 shutdown_connections 2024-09-19T09:12:14.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.312+0000 7fe977fee740 1 --2- 172.21.15.120:0/4152714031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c680ae680 0x562c680b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.312+0000 7fe977fee740 1 --2- 172.21.15.120:0/4152714031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c680a3dc0 0x562c680ae140 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.312+0000 7fe977fee740 1 --2- 172.21.15.120:0/4152714031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c680a34a0 0x562c680a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.312+0000 7fe977fee740 1 -- 172.21.15.120:0/4152714031 >> 172.21.15.120:0/4152714031 conn(0x562c6809fa30 msgr2=0x562c6809fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:14.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.312+0000 7fe977fee740 1 -- 172.21.15.120:0/4152714031 shutdown_connections 2024-09-19T09:12:14.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.312+0000 7fe977fee740 1 -- 172.21.15.120:0/4152714031 wait complete. 2024-09-19T09:12:14.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.312+0000 7fe977fee740 1 -- start start 2024-09-19T09:12:14.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.313+0000 7fe977fee740 1 Processor -- start 2024-09-19T09:12:14.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.313+0000 7fe977fee740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c680a34a0 0x562c680ade80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:14.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.313+0000 7fe977fee740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c680a3dc0 0x562c680ab130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:14.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.313+0000 7fe977fee740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c680ae680 0x562c680ab670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:14.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.313+0000 7fe977fee740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562c680b1250 con 0x562c680ae680 2024-09-19T09:12:14.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.313+0000 7fe977fee740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562c67fc4050 con 0x562c680a3dc0 2024-09-19T09:12:14.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.313+0000 7fe977fee740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562c680b13d0 con 0x562c680a34a0 2024-09-19T09:12:14.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.313+0000 7fe977fee740 10 client.0 _ll_register_callbacks cb 0x562c68004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:14.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.313+0000 7fe967fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c680ae680 0x562c680ab670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:14.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.313+0000 7fe967fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c680ae680 0x562c680ab670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56098/0 (socket says 172.21.15.120:56098) 2024-09-19T09:12:14.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.313+0000 7fe967fff640 1 -- 172.21.15.120:0/2385369519 learned_addr learned my addr 172.21.15.120:0/2385369519 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:14.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.313+0000 7fe96f7fe640 1 --2- 172.21.15.120:0/2385369519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c680a3dc0 0x562c680ab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:14.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.313+0000 7fe967fff640 1 --2- 172.21.15.120:0/2385369519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c680a34a0 0x562c680ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:14.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.313+0000 7fe967fff640 1 -- 172.21.15.120:0/2385369519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c680a34a0 msgr2=0x562c680ade80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:14.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.313+0000 7fe967fff640 1 --2- 172.21.15.120:0/2385369519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c680a34a0 0x562c680ade80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.313+0000 7fe967fff640 1 -- 172.21.15.120:0/2385369519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c680a3dc0 msgr2=0x562c680ab130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:14.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.313+0000 7fe967fff640 1 --2- 172.21.15.120:0/2385369519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c680a3dc0 0x562c680ab130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.313+0000 7fe967fff640 1 -- 172.21.15.120:0/2385369519 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c67d7a3c0 con 0x562c680ae680 2024-09-19T09:12:14.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.313+0000 7fe967fff640 1 --2- 172.21.15.120:0/2385369519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c680ae680 0x562c680ab670 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fe958011c70 tx=0x7fe958009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aceb555ce8b80272 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:14.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.313+0000 7fe94effd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:14.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.313+0000 7fe94effd640 1 -- 172.21.15.120:0/2385369519 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958011e20 con 0x562c680ae680 2024-09-19T09:12:14.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.314+0000 7fe94effd640 1 -- 172.21.15.120:0/2385369519 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fe958022430 con 0x562c680ae680 2024-09-19T09:12:14.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.314+0000 7fe977fee740 1 -- 172.21.15.120:0/2385369519 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe960031870 con 0x562c680ae680 2024-09-19T09:12:14.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.314+0000 7fe94effd640 1 -- 172.21.15.120:0/2385369519 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe95800fe60 con 0x562c680ae680 2024-09-19T09:12:14.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.314+0000 7fe977fee740 10 client.18735 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:14.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.314+0000 7fe977fee740 1 -- 172.21.15.120:0/2385369519 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562c680bcb40 con 0x562c680ae680 2024-09-19T09:12:14.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.314+0000 7fe94effd640 1 -- 172.21.15.120:0/2385369519 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe960031870 con 0x562c680ae680 2024-09-19T09:12:14.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.314+0000 7fe94effd640 1 -- 172.21.15.120:0/2385369519 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe958021520 con 0x562c680ae680 2024-09-19T09:12:14.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.314+0000 7fe977fee740 10 client.18735 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:14.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.314+0000 7fe977fee740 10 client.18735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:12:14.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.314+0000 7fe977fee740 20 client.18735 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:14.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.314+0000 7fe977fee740 1 --2- 172.21.15.120:0/2385369519 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562c680be000 0x562c680c0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:14.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.314+0000 7fe977fee740 4 client.18735 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:12:14.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.314+0000 7fe977fee740 1 -- 172.21.15.120:0/2385369519 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c68016850 con 0x562c680be000 2024-09-19T09:12:14.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.314+0000 7fe96f7fe640 1 --2- 172.21.15.120:0/2385369519 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562c680be000 0x562c680c0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:14.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.315+0000 7fe94effd640 10 client.18735 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:14.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.315+0000 7fe96f7fe640 1 --2- 172.21.15.120:0/2385369519 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562c680be000 0x562c680c0430 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df044c19aa32c2c4 server_cookie=c88b5f3e4a09a01e in_seq=0 out_seq=0 2024-09-19T09:12:14.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.316+0000 7fe94effd640 1 -- 172.21.15.120:0/2385369519 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562c68016850 con 0x562c680be000 2024-09-19T09:12:14.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.316+0000 7fe94effd640 10 client.18735 handle_command_reply: tid=0 2024-09-19T09:12:14.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.316+0000 7fe977fee740 1 client.18735 shutdown 2024-09-19T09:12:14.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.316+0000 7fe977fee740 1 -- 172.21.15.120:0/2385369519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c680ae680 msgr2=0x562c680ab670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:14.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.316+0000 7fe977fee740 1 --2- 172.21.15.120:0/2385369519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c680ae680 0x562c680ab670 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fe958011c70 tx=0x7fe958009ac0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.316+0000 7fe977fee740 1 -- 172.21.15.120:0/2385369519 shutdown_connections 2024-09-19T09:12:14.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.316+0000 7fe977fee740 1 --2- 172.21.15.120:0/2385369519 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562c680be000 0x562c680c0430 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.316+0000 7fe977fee740 1 --2- 172.21.15.120:0/2385369519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c680ae680 0x562c680ab670 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.316+0000 7fe977fee740 1 --2- 172.21.15.120:0/2385369519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c680a3dc0 0x562c680ab130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.316+0000 7fe977fee740 1 --2- 172.21.15.120:0/2385369519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c680a34a0 0x562c680ade80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.316+0000 7fe977fee740 1 -- 172.21.15.120:0/2385369519 >> 172.21.15.120:0/2385369519 conn(0x562c6809fa30 msgr2=0x562c680a5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:14.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.316+0000 7fe94effd640 0 client.18735 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:14.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.316+0000 7fe977fee740 1 -- 172.21.15.120:0/2385369519 shutdown_connections 2024-09-19T09:12:14.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.316+0000 7fe977fee740 1 -- 172.21.15.120:0/2385369519 wait complete. 2024-09-19T09:12:14.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.316+0000 7fe977fee740 20 client.18735 trim_cache size 0 max 16384 2024-09-19T09:12:14.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.334+0000 7fe977fee740 1 Processor -- start 2024-09-19T09:12:14.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.334+0000 7fe977fee740 1 -- start start 2024-09-19T09:12:14.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.334+0000 7fe977fee740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c6809fa30 0x562c680ad0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:14.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.334+0000 7fe977fee740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c680a34a0 0x562c680c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:14.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.334+0000 7fe977fee740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c680a3dc0 0x562c680c6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:14.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.334+0000 7fe977fee740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe9701af880 con 0x562c6809fa30 2024-09-19T09:12:14.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.334+0000 7fe977fee740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe9701b13c0 con 0x562c680a34a0 2024-09-19T09:12:14.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.334+0000 7fe977fee740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe9701aff50 con 0x562c680a3dc0 2024-09-19T09:12:14.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.334+0000 7fe967fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c6809fa30 0x562c680ad0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:14.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.334+0000 7fe96f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c680a34a0 0x562c680c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:14.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.334+0000 7fe967fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c6809fa30 0x562c680ad0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56106/0 (socket says 172.21.15.120:56106) 2024-09-19T09:12:14.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.334+0000 7fe967fff640 1 -- 172.21.15.120:0/3487785326 learned_addr learned my addr 172.21.15.120:0/3487785326 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:14.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.334+0000 7fe967fff640 1 --2- 172.21.15.120:0/3487785326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c680a3dc0 0x562c680c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:14.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.334+0000 7fe967fff640 1 -- 172.21.15.120:0/3487785326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c680a3dc0 msgr2=0x562c680c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:14.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.334+0000 7fe967fff640 1 --2- 172.21.15.120:0/3487785326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c680a3dc0 0x562c680c6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.334+0000 7fe967fff640 1 -- 172.21.15.120:0/3487785326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c680a34a0 msgr2=0x562c680c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:14.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.334+0000 7fe967fff640 1 --2- 172.21.15.120:0/3487785326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c680a34a0 0x562c680c5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.335+0000 7fe967fff640 1 -- 172.21.15.120:0/3487785326 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c680ae1f0 con 0x562c6809fa30 2024-09-19T09:12:14.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.335+0000 7fe96f7fe640 1 --2- 172.21.15.120:0/3487785326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c680a34a0 0x562c680c5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:14.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.335+0000 7fe967fff640 1 --2- 172.21.15.120:0/3487785326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c6809fa30 0x562c680ad0a0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fe958022000 tx=0x7fe958005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20b0c4f1e97d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:14.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.335+0000 7fe9657fa640 1 -- 172.21.15.120:0/3487785326 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958031040 con 0x562c6809fa30 2024-09-19T09:12:14.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.335+0000 7fe9657fa640 1 -- 172.21.15.120:0/3487785326 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe958009e90 con 0x562c6809fa30 2024-09-19T09:12:14.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.335+0000 7fe9657fa640 1 -- 172.21.15.120:0/3487785326 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958004680 con 0x562c6809fa30 2024-09-19T09:12:14.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.336+0000 7fe977fee740 1 -- 172.21.15.120:0/3487785326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c6809fa30 msgr2=0x562c680ad0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:14.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.336+0000 7fe977fee740 1 --2- 172.21.15.120:0/3487785326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c6809fa30 0x562c680ad0a0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fe958022000 tx=0x7fe958005f90 comp rx=0 tx=0).stop 2024-09-19T09:12:14.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.336+0000 7fe977fee740 1 -- 172.21.15.120:0/3487785326 shutdown_connections 2024-09-19T09:12:14.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.336+0000 7fe977fee740 1 --2- 172.21.15.120:0/3487785326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c680a3dc0 0x562c680c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.336+0000 7fe977fee740 1 --2- 172.21.15.120:0/3487785326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c680a34a0 0x562c680c5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.336+0000 7fe977fee740 1 --2- 172.21.15.120:0/3487785326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c6809fa30 0x562c680ad0a0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.336+0000 7fe977fee740 1 -- 172.21.15.120:0/3487785326 >> 172.21.15.120:0/3487785326 conn(0x562c680ab7d0 msgr2=0x562c680ac990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:14.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.336+0000 7fe977fee740 1 -- 172.21.15.120:0/3487785326 shutdown_connections 2024-09-19T09:12:14.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.336+0000 7fe977fee740 1 -- 172.21.15.120:0/3487785326 wait complete. 2024-09-19T09:12:14.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.337+0000 7fe977fee740 1 -- start start 2024-09-19T09:12:14.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.337+0000 7fe977fee740 1 Processor -- start 2024-09-19T09:12:14.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.337+0000 7fe977fee740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c6809fa30 0x562c680b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:14.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.337+0000 7fe977fee740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c680a34a0 0x562c680b4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:14.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.337+0000 7fe977fee740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c680a3dc0 0x562c680b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:14.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.337+0000 7fe977fee740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe9701b0cf0 con 0x562c680a3dc0 2024-09-19T09:12:14.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.337+0000 7fe977fee740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe9701b0620 con 0x562c680a34a0 2024-09-19T09:12:14.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.337+0000 7fe977fee740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe9701ad840 con 0x562c6809fa30 2024-09-19T09:12:14.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.337+0000 7fe977fee740 10 client.0 _ll_register_callbacks cb 0x7fe9580115b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:14.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.338+0000 7fe96f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c680a34a0 0x562c680b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:14.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.338+0000 7fe96f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c680a34a0 0x562c680b4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:51990/0 (socket says 172.21.15.120:51990) 2024-09-19T09:12:14.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.338+0000 7fe96f7fe640 1 -- 172.21.15.120:0/1476782534 learned_addr learned my addr 172.21.15.120:0/1476782534 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:14.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.338+0000 7fe967fff640 1 --2- 172.21.15.120:0/1476782534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c6809fa30 0x562c680b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:14.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.338+0000 7fe967fff640 1 --2- 172.21.15.120:0/1476782534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c680a3dc0 0x562c680b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:14.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.338+0000 7fe96f7fe640 1 -- 172.21.15.120:0/1476782534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c6809fa30 msgr2=0x562c680b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:14.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.338+0000 7fe96f7fe640 1 --2- 172.21.15.120:0/1476782534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c6809fa30 0x562c680b6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.338+0000 7fe96f7fe640 1 -- 172.21.15.120:0/1476782534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c680a3dc0 msgr2=0x562c680b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:14.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.338+0000 7fe96f7fe640 1 --2- 172.21.15.120:0/1476782534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c680a3dc0 0x562c680b4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.338+0000 7fe96f7fe640 1 -- 172.21.15.120:0/1476782534 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c68127ce0 con 0x562c680a34a0 2024-09-19T09:12:14.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.338+0000 7fe967fff640 1 --2- 172.21.15.120:0/1476782534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c6809fa30 0x562c680b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:12:14.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.338+0000 7fe96f7fe640 1 --2- 172.21.15.120:0/1476782534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c680a34a0 0x562c680b4290 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe96002a940 tx=0x7fe960005380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b0534c299817726 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:14.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.338+0000 7fe94effd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:12:14.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.338+0000 7fe94effd640 1 -- 172.21.15.120:0/1476782534 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe960054060 con 0x562c680a34a0 2024-09-19T09:12:14.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.338+0000 7fe94effd640 1 -- 172.21.15.120:0/1476782534 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fe960004240 con 0x562c680a34a0 2024-09-19T09:12:14.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.338+0000 7fe977fee740 1 -- 172.21.15.120:0/1476782534 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe958009e90 con 0x562c680a34a0 2024-09-19T09:12:14.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.339+0000 7fe94effd640 1 -- 172.21.15.120:0/1476782534 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe960040070 con 0x562c680a34a0 2024-09-19T09:12:14.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.339+0000 7fe977fee740 10 client.27872 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:14.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.339+0000 7fe977fee740 1 -- 172.21.15.120:0/1476782534 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562c68128e70 con 0x562c680a34a0 2024-09-19T09:12:14.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.339+0000 7fe94effd640 1 -- 172.21.15.120:0/1476782534 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe960031830 con 0x562c680a34a0 2024-09-19T09:12:14.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.339+0000 7fe94effd640 1 -- 172.21.15.120:0/1476782534 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe960059070 con 0x562c680a34a0 2024-09-19T09:12:14.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.339+0000 7fe977fee740 10 client.27872 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:14.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.339+0000 7fe977fee740 10 client.27872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:12:14.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.339+0000 7fe977fee740 20 client.27872 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:14.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.339+0000 7fe977fee740 1 --2- 172.21.15.120:0/1476782534 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562c680ae680 0x562c6812b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:14.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.339+0000 7fe977fee740 4 client.27872 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:12:14.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.339+0000 7fe977fee740 1 -- 172.21.15.120:0/1476782534 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fe958009e90 con 0x562c680ae680 2024-09-19T09:12:14.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.339+0000 7fe967fff640 1 --2- 172.21.15.120:0/1476782534 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562c680ae680 0x562c6812b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:14.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.340+0000 7fe94effd640 10 client.27872 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:14.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.340+0000 7fe967fff640 1 --2- 172.21.15.120:0/1476782534 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562c680ae680 0x562c6812b210 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccdcb5d828df050 server_cookie=3292114e5d14e35d in_seq=0 out_seq=0 2024-09-19T09:12:14.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.341+0000 7fe94effd640 1 -- 172.21.15.120:0/1476782534 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fe958009e90 con 0x562c680ae680 2024-09-19T09:12:14.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.341+0000 7fe94effd640 10 client.27872 handle_command_reply: tid=0 2024-09-19T09:12:14.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.342+0000 7fe977fee740 1 client.27872 shutdown 2024-09-19T09:12:14.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.342+0000 7fe977fee740 1 -- 172.21.15.120:0/1476782534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c680a34a0 msgr2=0x562c680b4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:14.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.342+0000 7fe977fee740 1 --2- 172.21.15.120:0/1476782534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c680a34a0 0x562c680b4290 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe96002a940 tx=0x7fe960005380 comp rx=0 tx=0).stop 2024-09-19T09:12:14.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.342+0000 7fe977fee740 1 -- 172.21.15.120:0/1476782534 shutdown_connections 2024-09-19T09:12:14.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.342+0000 7fe977fee740 1 --2- 172.21.15.120:0/1476782534 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562c680ae680 0x562c6812b210 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.342+0000 7fe977fee740 1 --2- 172.21.15.120:0/1476782534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c680a3dc0 0x562c680b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.342+0000 7fe977fee740 1 --2- 172.21.15.120:0/1476782534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c680a34a0 0x562c680b4290 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.342+0000 7fe977fee740 1 --2- 172.21.15.120:0/1476782534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c6809fa30 0x562c680b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.342+0000 7fe94effd640 0 client.27872 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:14.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.342+0000 7fe977fee740 1 -- 172.21.15.120:0/1476782534 >> 172.21.15.120:0/1476782534 conn(0x562c680ab7d0 msgr2=0x562c680ad1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:14.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.342+0000 7fe977fee740 1 -- 172.21.15.120:0/1476782534 shutdown_connections 2024-09-19T09:12:14.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.342+0000 7fe977fee740 1 -- 172.21.15.120:0/1476782534 wait complete. 2024-09-19T09:12:14.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.342+0000 7fe977fee740 20 client.27872 trim_cache size 0 max 16384 2024-09-19T09:12:14.349 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:12:14.349 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:12:14.349 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:12:14.349 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:12:14.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.342+0000 7fe975c15640 1 -- 172.21.15.120:0/2232661819 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe95c0a3240 msgr2=0x7fe95c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:14.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.343+0000 7fe975c15640 1 --2- 172.21.15.120:0/2232661819 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe95c0a3240 0x7fe95c0a5700 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe968098de0 tx=0x7fe96806b150 comp rx=0 tx=0).stop 2024-09-19T09:12:14.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.343+0000 7fe975c15640 1 -- 172.21.15.120:0/2232661819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe970108290 msgr2=0x7fe970101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:14.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.343+0000 7fe975c15640 1 --2- 172.21.15.120:0/2232661819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe970108290 0x7fe970101470 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fe96800eb10 tx=0x7fe96800c2d0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.343+0000 7fe975c15640 1 -- 172.21.15.120:0/2232661819 shutdown_connections 2024-09-19T09:12:14.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.343+0000 7fe975c15640 1 --2- 172.21.15.120:0/2232661819 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe95c0a3240 0x7fe95c0a5700 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.343+0000 7fe975c15640 1 --2- 172.21.15.120:0/2232661819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe970108290 0x7fe970101470 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.343+0000 7fe975c15640 1 --2- 172.21.15.120:0/2232661819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe970103740 0x7fe970100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.343+0000 7fe975c15640 1 --2- 172.21.15.120:0/2232661819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe970102d90 0x7fe9701009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:14.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.343+0000 7fe975c15640 1 -- 172.21.15.120:0/2232661819 >> 172.21.15.120:0/2232661819 conn(0x7fe9700fc7d0 msgr2=0x7fe970105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:14.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.343+0000 7fe975c15640 1 -- 172.21.15.120:0/2232661819 shutdown_connections 2024-09-19T09:12:14.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:14.343+0000 7fe975c15640 1 -- 172.21.15.120:0/2232661819 wait complete. 2024-09-19T09:12:15.043 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:14 smithi120 ceph-mon[31150]: pgmap v624: 97 pgs: 97 active+clean; 47 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 412 op/s 2024-09-19T09:12:15.065 INFO:tasks.workunit.client.0.smithi120.stdout: 1 902296 9.88 MB/sec execute 370 sec latency 15.660 ms 2024-09-19T09:12:15.070 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:12:15.070 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:12:15.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:14 smithi196 ceph-mon[35150]: pgmap v624: 97 pgs: 97 active+clean; 47 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 412 op/s 2024-09-19T09:12:15.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:14 smithi153 ceph-mon[35071]: pgmap v624: 97 pgs: 97 active+clean; 47 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 412 op/s 2024-09-19T09:12:15.321 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:12:15.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.865+0000 7f086ed2a640 1 -- 172.21.15.120:0/2389664151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0868100e20 msgr2=0x7f0868101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:15.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.865+0000 7f086ed2a640 1 --2- 172.21.15.120:0/2389664151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0868100e20 0x7f0868101200 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f0860009900 tx=0x7f086002a480 comp rx=0 tx=0).stop 2024-09-19T09:12:15.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.865+0000 7f086ed2a640 1 -- 172.21.15.120:0/2389664151 shutdown_connections 2024-09-19T09:12:15.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.865+0000 7f086ed2a640 1 --2- 172.21.15.120:0/2389664151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0868102ef0 0x7f086810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:15.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.865+0000 7f086ed2a640 1 --2- 172.21.15.120:0/2389664151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0868102570 0x7f08681029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:15.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.865+0000 7f086ed2a640 1 --2- 172.21.15.120:0/2389664151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0868100e20 0x7f0868101200 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:15.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.865+0000 7f086ed2a640 1 -- 172.21.15.120:0/2389664151 >> 172.21.15.120:0/2389664151 conn(0x7f08680fc7d0 msgr2=0x7f08680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:15.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.865+0000 7f086ed2a640 1 -- 172.21.15.120:0/2389664151 shutdown_connections 2024-09-19T09:12:15.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.865+0000 7f086ed2a640 1 -- 172.21.15.120:0/2389664151 wait complete. 2024-09-19T09:12:15.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.865+0000 7f086ed2a640 1 Processor -- start 2024-09-19T09:12:15.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.865+0000 7f086ed2a640 1 -- start start 2024-09-19T09:12:15.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.865+0000 7f086ed2a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0868100e20 0x7f0868197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:15.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.866+0000 7f086ed2a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0868102570 0x7f0868197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:15.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.866+0000 7f086ed2a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0868102ef0 0x7f086819bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:15.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.866+0000 7f086ed2a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0868110f80 con 0x7f0868102570 2024-09-19T09:12:15.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.866+0000 7f086ed2a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0868110e00 con 0x7f0868100e20 2024-09-19T09:12:15.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.866+0000 7f086ed2a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0868111100 con 0x7f0868102ef0 2024-09-19T09:12:15.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.866+0000 7f0867fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0868102570 0x7f0868197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:15.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.866+0000 7f086cac6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0868100e20 0x7f0868197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:15.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.866+0000 7f0867fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0868102570 0x7f0868197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56114/0 (socket says 172.21.15.120:56114) 2024-09-19T09:12:15.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.866+0000 7f0867fff640 1 -- 172.21.15.120:0/3097996409 learned_addr learned my addr 172.21.15.120:0/3097996409 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:15.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.866+0000 7f086d2c7640 1 --2- 172.21.15.120:0/3097996409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0868102ef0 0x7f086819bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:15.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.866+0000 7f0867fff640 1 -- 172.21.15.120:0/3097996409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0868102ef0 msgr2=0x7f086819bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:15.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.866+0000 7f0867fff640 1 --2- 172.21.15.120:0/3097996409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0868102ef0 0x7f086819bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:15.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.866+0000 7f0867fff640 1 -- 172.21.15.120:0/3097996409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0868100e20 msgr2=0x7f0868197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:15.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.866+0000 7f0867fff640 1 --2- 172.21.15.120:0/3097996409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0868100e20 0x7f0868197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:15.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.866+0000 7f0867fff640 1 -- 172.21.15.120:0/3097996409 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f086819c3c0 con 0x7f0868102570 2024-09-19T09:12:15.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.866+0000 7f086cac6640 1 --2- 172.21.15.120:0/3097996409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0868100e20 0x7f0868197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:15.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.866+0000 7f0867fff640 1 --2- 172.21.15.120:0/3097996409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0868102570 0x7f0868197950 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f085800ea80 tx=0x7f085800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:15.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.866+0000 7f0865ffb640 1 -- 172.21.15.120:0/3097996409 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085800ce40 con 0x7f0868102570 2024-09-19T09:12:15.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.866+0000 7f0865ffb640 1 -- 172.21.15.120:0/3097996409 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0858011570 con 0x7f0868102570 2024-09-19T09:12:15.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.866+0000 7f0865ffb640 1 -- 172.21.15.120:0/3097996409 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08580053a0 con 0x7f0868102570 2024-09-19T09:12:15.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.866+0000 7f086ed2a640 1 -- 172.21.15.120:0/3097996409 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f086819c6b0 con 0x7f0868102570 2024-09-19T09:12:15.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.867+0000 7f086ed2a640 1 -- 172.21.15.120:0/3097996409 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0868069f70 con 0x7f0868102570 2024-09-19T09:12:15.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.868+0000 7f086ed2a640 1 -- 172.21.15.120:0/3097996409 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0868101200 con 0x7f0868102570 2024-09-19T09:12:15.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.869+0000 7f0865ffb640 1 -- 172.21.15.120:0/3097996409 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f0858011720 con 0x7f0868102570 2024-09-19T09:12:15.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.869+0000 7f0865ffb640 1 --2- 172.21.15.120:0/3097996409 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f08540a32c0 0x7f08540a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:15.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.869+0000 7f086cac6640 1 --2- 172.21.15.120:0/3097996409 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f08540a32c0 0x7f08540a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:15.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.869+0000 7f0865ffb640 1 -- 172.21.15.120:0/3097996409 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f08580d1950 con 0x7f0868102570 2024-09-19T09:12:15.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.870+0000 7f086cac6640 1 --2- 172.21.15.120:0/3097996409 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f08540a32c0 0x7f08540a5780 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0860009230 tx=0x7f08600058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:15.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:15.874+0000 7f0865ffb640 1 -- 172.21.15.120:0/3097996409 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f085809dfe0 con 0x7f0868102570 2024-09-19T09:12:16.065 INFO:tasks.workunit.client.0.smithi120.stdout: 1 905502 9.90 MB/sec execute 371 sec latency 53.687 ms 2024-09-19T09:12:16.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:16.086+0000 7f086ed2a640 1 -- 172.21.15.120:0/3097996409 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f086806a990 con 0x7f0868102570 2024-09-19T09:12:16.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:16.087+0000 7f0865ffb640 1 -- 172.21.15.120:0/3097996409 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11988 (secure 0 0 0) 0x7f08580a22e0 con 0x7f0868102570 2024-09-19T09:12:16.089 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:12:16.089 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":32,"btime":"2024-09-19T09:11:12:904684+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:11:12.904670+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27518":{"gid":27518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3214776406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3214776406},{"type":"v1","addr":"172.21.15.153:6835","nonce":3214776406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:12:16.089 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 32 2024-09-19T09:12:16.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:16.089+0000 7f086ed2a640 1 -- 172.21.15.120:0/3097996409 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f08540a32c0 msgr2=0x7f08540a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:16.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:16.089+0000 7f086ed2a640 1 --2- 172.21.15.120:0/3097996409 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f08540a32c0 0x7f08540a5780 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0860009230 tx=0x7f08600058d0 comp rx=0 tx=0).stop 2024-09-19T09:12:16.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:16.089+0000 7f086ed2a640 1 -- 172.21.15.120:0/3097996409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0868102570 msgr2=0x7f0868197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:16.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:16.089+0000 7f086ed2a640 1 --2- 172.21.15.120:0/3097996409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0868102570 0x7f0868197950 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f085800ea80 tx=0x7f085800ef40 comp rx=0 tx=0).stop 2024-09-19T09:12:16.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:16.089+0000 7f086ed2a640 1 -- 172.21.15.120:0/3097996409 shutdown_connections 2024-09-19T09:12:16.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:16.090+0000 7f086ed2a640 1 --2- 172.21.15.120:0/3097996409 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f08540a32c0 0x7f08540a5780 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:16.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:16.090+0000 7f086ed2a640 1 --2- 172.21.15.120:0/3097996409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0868102ef0 0x7f086819bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:16.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:16.090+0000 7f086ed2a640 1 --2- 172.21.15.120:0/3097996409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0868102570 0x7f0868197950 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:16.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:16.090+0000 7f086ed2a640 1 --2- 172.21.15.120:0/3097996409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0868100e20 0x7f0868197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:16.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:16.090+0000 7f086ed2a640 1 -- 172.21.15.120:0/3097996409 >> 172.21.15.120:0/3097996409 conn(0x7f08680fc7d0 msgr2=0x7f08680fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:16.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:16.090+0000 7f086ed2a640 1 -- 172.21.15.120:0/3097996409 shutdown_connections 2024-09-19T09:12:16.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:16.090+0000 7f086ed2a640 1 -- 172.21.15.120:0/3097996409 wait complete. 2024-09-19T09:12:16.862 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:12:17.065 INFO:tasks.workunit.client.0.smithi120.stdout: 1 907490 9.89 MB/sec execute 372 sec latency 84.132 ms 2024-09-19T09:12:17.140 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:12:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:16 smithi196 ceph-mon[35150]: pgmap v625: 97 pgs: 97 active+clean; 49 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.5 MiB/s wr, 398 op/s 2024-09-19T09:12:17.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:16 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3097996409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:12:17.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:16 smithi153 ceph-mon[35071]: pgmap v625: 97 pgs: 97 active+clean; 49 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.5 MiB/s wr, 398 op/s 2024-09-19T09:12:17.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:16 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3097996409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:12:17.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:16 smithi120 ceph-mon[31150]: pgmap v625: 97 pgs: 97 active+clean; 49 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.5 MiB/s wr, 398 op/s 2024-09-19T09:12:17.263 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:16 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3097996409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:12:17.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.657+0000 7f3dcd98f640 1 -- 172.21.15.120:0/890008862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3dc8101e20 msgr2=0x7f3dc81056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:17.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.657+0000 7f3dcd98f640 1 --2- 172.21.15.120:0/890008862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3dc8101e20 0x7f3dc81056b0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc009900 tx=0x7f3dbc02a410 comp rx=0 tx=0).stop 2024-09-19T09:12:17.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.657+0000 7f3dcd98f640 1 -- 172.21.15.120:0/890008862 shutdown_connections 2024-09-19T09:12:17.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.657+0000 7f3dcd98f640 1 --2- 172.21.15.120:0/890008862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3dc8105de0 0x7f3dc810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.657+0000 7f3dcd98f640 1 --2- 172.21.15.120:0/890008862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3dc8101e20 0x7f3dc81056b0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.657+0000 7f3dcd98f640 1 --2- 172.21.15.120:0/890008862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3dc8101470 0x7f3dc8101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.657+0000 7f3dcd98f640 1 -- 172.21.15.120:0/890008862 >> 172.21.15.120:0/890008862 conn(0x7f3dc8077230 msgr2=0x7f3dc80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:17.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.657+0000 7f3dcd98f640 1 -- 172.21.15.120:0/890008862 shutdown_connections 2024-09-19T09:12:17.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.657+0000 7f3dcd98f640 1 -- 172.21.15.120:0/890008862 wait complete. 2024-09-19T09:12:17.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.658+0000 7f3dcd98f640 1 Processor -- start 2024-09-19T09:12:17.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.658+0000 7f3dcd98f640 1 -- start start 2024-09-19T09:12:17.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.658+0000 7f3dcd98f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3dc8101470 0x7f3dc8198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:17.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.658+0000 7f3dcd98f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3dc8101e20 0x7f3dc81989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:17.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.658+0000 7f3dcd98f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3dc8105de0 0x7f3dc819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:17.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.658+0000 7f3dcd98f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc8110e00 con 0x7f3dc8101e20 2024-09-19T09:12:17.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.658+0000 7f3dcd98f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc8110c80 con 0x7f3dc8101470 2024-09-19T09:12:17.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.658+0000 7f3dcd98f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc8110f80 con 0x7f3dc8105de0 2024-09-19T09:12:17.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.659+0000 7f3dc67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3dc8101e20 0x7f3dc81989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:17.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.659+0000 7f3dc67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3dc8101e20 0x7f3dc81989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56136/0 (socket says 172.21.15.120:56136) 2024-09-19T09:12:17.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.659+0000 7f3dc67fc640 1 -- 172.21.15.120:0/4116468462 learned_addr learned my addr 172.21.15.120:0/4116468462 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:17.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.659+0000 7f3dc6ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3dc8101470 0x7f3dc8198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:17.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.659+0000 7f3dc77fe640 1 --2- 172.21.15.120:0/4116468462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3dc8105de0 0x7f3dc819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:17.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.659+0000 7f3dc67fc640 1 -- 172.21.15.120:0/4116468462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3dc8105de0 msgr2=0x7f3dc819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:17.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.659+0000 7f3dc67fc640 1 --2- 172.21.15.120:0/4116468462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3dc8105de0 0x7f3dc819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.659+0000 7f3dc67fc640 1 -- 172.21.15.120:0/4116468462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3dc8101470 msgr2=0x7f3dc8198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:17.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.659+0000 7f3dc67fc640 1 --2- 172.21.15.120:0/4116468462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3dc8101470 0x7f3dc8198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.659+0000 7f3dc67fc640 1 -- 172.21.15.120:0/4116468462 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc819d410 con 0x7f3dc8101e20 2024-09-19T09:12:17.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.659+0000 7f3dc77fe640 1 --2- 172.21.15.120:0/4116468462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3dc8105de0 0x7f3dc819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:12:17.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.659+0000 7f3dc67fc640 1 --2- 172.21.15.120:0/4116468462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3dc8101e20 0x7f3dc81989a0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc006010 tx=0x7f3dbc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:17.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.659+0000 7f3dcc98d640 1 -- 172.21.15.120:0/4116468462 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc03f030 con 0x7f3dc8101e20 2024-09-19T09:12:17.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.659+0000 7f3dcc98d640 1 -- 172.21.15.120:0/4116468462 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dbc003ce0 con 0x7f3dc8101e20 2024-09-19T09:12:17.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.659+0000 7f3dcd98f640 1 -- 172.21.15.120:0/4116468462 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc819d6a0 con 0x7f3dc8101e20 2024-09-19T09:12:17.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.659+0000 7f3dcc98d640 1 -- 172.21.15.120:0/4116468462 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc035070 con 0x7f3dc8101e20 2024-09-19T09:12:17.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.659+0000 7f3dcd98f640 1 -- 172.21.15.120:0/4116468462 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc81a4fe0 con 0x7f3dc8101e20 2024-09-19T09:12:17.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.661+0000 7f3dcc98d640 1 -- 172.21.15.120:0/4116468462 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f3dbc042030 con 0x7f3dc8101e20 2024-09-19T09:12:17.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.662+0000 7f3dcfd68740 1 Processor -- start 2024-09-19T09:12:17.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.662+0000 7f3dcfd68740 1 -- start start 2024-09-19T09:12:17.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.662+0000 7f3dcfd68740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55589be42840 0x55589be42c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:17.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.662+0000 7f3dcfd68740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55589be41120 0x55589be41560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:17.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.662+0000 7f3dcc98d640 1 --2- 172.21.15.120:0/4116468462 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3db00a3240 0x7f3db00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:17.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.662+0000 7f3dcfd68740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55589be41b80 0x55589be56f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:17.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.662+0000 7f3dcfd68740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55589bda72f0 con 0x55589be41120 2024-09-19T09:12:17.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.662+0000 7f3dcfd68740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55589bd4ecf0 con 0x55589be41b80 2024-09-19T09:12:17.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.662+0000 7f3dcfd68740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55589bda32b0 con 0x55589be42840 2024-09-19T09:12:17.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.663+0000 7f3dc6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55589be41120 0x55589be41560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:17.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.663+0000 7f3dc67fc640 1 --2- 172.21.15.120:0/4116468462 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3db00a3240 0x7f3db00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:17.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.663+0000 7f3dc6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55589be41120 0x55589be41560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56152/0 (socket says 172.21.15.120:56152) 2024-09-19T09:12:17.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.663+0000 7f3dc6ffd640 1 -- 172.21.15.120:0/506098618 learned_addr learned my addr 172.21.15.120:0/506098618 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:17.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.663+0000 7f3dc77fe640 1 --2- 172.21.15.120:0/506098618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55589be41b80 0x55589be56f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:17.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.663+0000 7f3dc77fe640 1 --2- 172.21.15.120:0/506098618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55589be42840 0x55589be42c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:17.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.663+0000 7f3dc6ffd640 1 -- 172.21.15.120:0/506098618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55589be42840 msgr2=0x55589be42c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:17.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.663+0000 7f3dc6ffd640 1 --2- 172.21.15.120:0/506098618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55589be42840 0x55589be42c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.663+0000 7f3dc6ffd640 1 -- 172.21.15.120:0/506098618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55589be41b80 msgr2=0x55589be56f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:17.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.663+0000 7f3dc6ffd640 1 --2- 172.21.15.120:0/506098618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55589be41b80 0x55589be56f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.663+0000 7f3dc6ffd640 1 -- 172.21.15.120:0/506098618 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55589bd333a0 con 0x55589be41120 2024-09-19T09:12:17.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.663+0000 7f3dcc98d640 1 -- 172.21.15.120:0/4116468462 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f3dbc0eff40 con 0x7f3dc8101e20 2024-09-19T09:12:17.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.663+0000 7f3dc77fe640 1 --2- 172.21.15.120:0/506098618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55589be42840 0x55589be42c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:12:17.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.663+0000 7f3dc6ffd640 1 --2- 172.21.15.120:0/506098618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55589be41120 0x55589be41560 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f3db800bcd0 tx=0x7f3db80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b33b4f4b28cbfe9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:17.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.663+0000 7f3dc67fc640 1 --2- 172.21.15.120:0/4116468462 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3db00a3240 0x7f3db00a5700 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc0b8a30 tx=0x7f3dbc0d5050 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:17.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.663+0000 7f3dadffb640 1 -- 172.21.15.120:0/506098618 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db8005220 con 0x55589be41120 2024-09-19T09:12:17.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.663+0000 7f3dadffb640 1 -- 172.21.15.120:0/506098618 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3db8004c80 con 0x55589be41120 2024-09-19T09:12:17.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.663+0000 7f3dadffb640 1 -- 172.21.15.120:0/506098618 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db8011860 con 0x55589be41120 2024-09-19T09:12:17.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.664+0000 7f3dcfd68740 1 -- 172.21.15.120:0/506098618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55589be41120 msgr2=0x55589be41560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:17.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.664+0000 7f3dcfd68740 1 --2- 172.21.15.120:0/506098618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55589be41120 0x55589be41560 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f3db800bcd0 tx=0x7f3db80107a0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.665+0000 7f3dcfd68740 1 -- 172.21.15.120:0/506098618 shutdown_connections 2024-09-19T09:12:17.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.665+0000 7f3dcfd68740 1 --2- 172.21.15.120:0/506098618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55589be41b80 0x55589be56f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.665+0000 7f3dcfd68740 1 --2- 172.21.15.120:0/506098618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55589be41120 0x55589be41560 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.665+0000 7f3dcfd68740 1 --2- 172.21.15.120:0/506098618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55589be42840 0x55589be42c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.665+0000 7f3dcfd68740 1 -- 172.21.15.120:0/506098618 >> 172.21.15.120:0/506098618 conn(0x55589be0a310 msgr2=0x55589be3dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:17.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.665+0000 7f3dcfd68740 1 -- 172.21.15.120:0/506098618 shutdown_connections 2024-09-19T09:12:17.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.665+0000 7f3dcfd68740 1 -- 172.21.15.120:0/506098618 wait complete. 2024-09-19T09:12:17.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.665+0000 7f3dcfd68740 1 -- start start 2024-09-19T09:12:17.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.666+0000 7f3dcfd68740 1 Processor -- start 2024-09-19T09:12:17.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.666+0000 7f3dcfd68740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55589be41120 0x55589be56ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:17.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.666+0000 7f3dcfd68740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55589be41b80 0x55589be54170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:17.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.666+0000 7f3dcfd68740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55589be42840 0x55589be54780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:17.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.666+0000 7f3dcfd68740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55589be57930 con 0x55589be41b80 2024-09-19T09:12:17.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.666+0000 7f3dcfd68740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55589bd62060 con 0x55589be42840 2024-09-19T09:12:17.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.666+0000 7f3dcfd68740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55589be57ab0 con 0x55589be41120 2024-09-19T09:12:17.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.666+0000 7f3dcfd68740 10 client.0 _ll_register_callbacks cb 0x55589bab58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:17.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.666+0000 7f3dc77fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55589be42840 0x55589be54780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:17.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.666+0000 7f3dc77fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55589be42840 0x55589be54780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52030/0 (socket says 172.21.15.120:52030) 2024-09-19T09:12:17.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.666+0000 7f3dc6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55589be41b80 0x55589be54170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:17.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.666+0000 7f3dc77fe640 1 -- 172.21.15.120:0/3207227867 learned_addr learned my addr 172.21.15.120:0/3207227867 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:17.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.666+0000 7f3dc77fe640 1 --2- 172.21.15.120:0/3207227867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55589be41120 0x55589be56ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:17.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.667+0000 7f3dc6ffd640 1 -- 172.21.15.120:0/3207227867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55589be41120 msgr2=0x55589be56ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:17.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.667+0000 7f3dc6ffd640 1 --2- 172.21.15.120:0/3207227867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55589be41120 0x55589be56ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.667+0000 7f3dc6ffd640 1 -- 172.21.15.120:0/3207227867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55589be42840 msgr2=0x55589be54780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:17.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.667+0000 7f3dc6ffd640 1 --2- 172.21.15.120:0/3207227867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55589be42840 0x55589be54780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.667+0000 7f3dc6ffd640 1 -- 172.21.15.120:0/3207227867 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55589bb183c0 con 0x55589be41b80 2024-09-19T09:12:17.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.667+0000 7f3dc77fe640 1 --2- 172.21.15.120:0/3207227867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55589be42840 0x55589be54780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:17.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.667+0000 7f3dc6ffd640 1 --2- 172.21.15.120:0/3207227867 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55589be41b80 0x55589be54170 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f3db8010ca0 tx=0x7f3db80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20a82a9d7ea8d8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:17.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.667+0000 7f3da6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:17.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.667+0000 7f3da6ffd640 1 -- 172.21.15.120:0/3207227867 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db802a060 con 0x55589be41b80 2024-09-19T09:12:17.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.667+0000 7f3da6ffd640 1 -- 172.21.15.120:0/3207227867 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f3db8004a60 con 0x55589be41b80 2024-09-19T09:12:17.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.667+0000 7f3dcfd68740 1 -- 172.21.15.120:0/3207227867 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3db8004c80 con 0x55589be41b80 2024-09-19T09:12:17.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.667+0000 7f3da6ffd640 1 -- 172.21.15.120:0/3207227867 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3db800be80 con 0x55589be41b80 2024-09-19T09:12:17.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.667+0000 7f3dcfd68740 10 client.18759 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:17.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.667+0000 7f3dcfd68740 1 -- 172.21.15.120:0/3207227867 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55589be63400 con 0x55589be41b80 2024-09-19T09:12:17.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.667+0000 7f3da6ffd640 1 -- 172.21.15.120:0/3207227867 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3db8004c80 con 0x55589be41b80 2024-09-19T09:12:17.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.667+0000 7f3da6ffd640 1 -- 172.21.15.120:0/3207227867 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f3db802f070 con 0x55589be41b80 2024-09-19T09:12:17.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.667+0000 7f3dcfd68740 10 client.18759 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:17.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.667+0000 7f3dcfd68740 10 client.18759 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:12:17.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.668+0000 7f3dcfd68740 20 client.18759 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:17.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.668+0000 7f3dcfd68740 1 --2- 172.21.15.120:0/3207227867 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55589be64750 0x55589be66b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:17.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.668+0000 7f3dcfd68740 4 client.18759 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:12:17.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.668+0000 7f3dcfd68740 1 -- 172.21.15.120:0/3207227867 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55589bdb4860 con 0x55589be64750 2024-09-19T09:12:17.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.668+0000 7f3dc77fe640 1 --2- 172.21.15.120:0/3207227867 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55589be64750 0x55589be66b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:17.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.668+0000 7f3da6ffd640 10 client.18759 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:12:17.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.668+0000 7f3dc77fe640 1 --2- 172.21.15.120:0/3207227867 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55589be64750 0x55589be66b40 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3edd8355063d7b00 server_cookie=aec6438cf9e36e39 in_seq=0 out_seq=0 2024-09-19T09:12:17.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.669+0000 7f3da6ffd640 1 -- 172.21.15.120:0/3207227867 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55589bdb4860 con 0x55589be64750 2024-09-19T09:12:17.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.669+0000 7f3da6ffd640 10 client.18759 handle_command_reply: tid=0 2024-09-19T09:12:17.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.669+0000 7f3dcfd68740 1 client.18759 shutdown 2024-09-19T09:12:17.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.669+0000 7f3dcfd68740 1 -- 172.21.15.120:0/3207227867 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55589be41b80 msgr2=0x55589be54170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:17.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.669+0000 7f3dcfd68740 1 --2- 172.21.15.120:0/3207227867 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55589be41b80 0x55589be54170 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f3db8010ca0 tx=0x7f3db80048f0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.669+0000 7f3dcfd68740 1 -- 172.21.15.120:0/3207227867 shutdown_connections 2024-09-19T09:12:17.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.669+0000 7f3dcfd68740 1 --2- 172.21.15.120:0/3207227867 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55589be64750 0x55589be66b40 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.669+0000 7f3dcfd68740 1 --2- 172.21.15.120:0/3207227867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55589be42840 0x55589be54780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.669+0000 7f3dcfd68740 1 --2- 172.21.15.120:0/3207227867 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55589be41b80 0x55589be54170 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.669+0000 7f3dcfd68740 1 --2- 172.21.15.120:0/3207227867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55589be41120 0x55589be56ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.669+0000 7f3dcfd68740 1 -- 172.21.15.120:0/3207227867 >> 172.21.15.120:0/3207227867 conn(0x55589be0a310 msgr2=0x55589be439a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:17.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.669+0000 7f3da6ffd640 0 client.18759 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:12:17.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.669+0000 7f3dcfd68740 1 -- 172.21.15.120:0/3207227867 shutdown_connections 2024-09-19T09:12:17.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.669+0000 7f3dcfd68740 1 -- 172.21.15.120:0/3207227867 wait complete. 2024-09-19T09:12:17.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.669+0000 7f3dcfd68740 20 client.18759 trim_cache size 0 max 16384 2024-09-19T09:12:17.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.683+0000 7f3dcfd68740 1 Processor -- start 2024-09-19T09:12:17.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.684+0000 7f3dcfd68740 1 -- start start 2024-09-19T09:12:17.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.684+0000 7f3dcfd68740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55589be41120 0x55589be60bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:17.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.684+0000 7f3dcfd68740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55589be41b80 0x55589be3cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:17.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.684+0000 7f3dcfd68740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55589be42840 0x55589be43180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:17.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.684+0000 7f3dcfd68740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81ab410 con 0x55589be41b80 2024-09-19T09:12:17.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.684+0000 7f3dcfd68740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81acf50 con 0x55589be41120 2024-09-19T09:12:17.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.684+0000 7f3dcfd68740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81abae0 con 0x55589be42840 2024-09-19T09:12:17.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.684+0000 7f3dc77fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55589be41120 0x55589be60bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:17.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.684+0000 7f3dc6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55589be41b80 0x55589be3cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:17.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.684+0000 7f3dc77fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55589be41120 0x55589be60bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52046/0 (socket says 172.21.15.120:52046) 2024-09-19T09:12:17.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.684+0000 7f3dc77fe640 1 -- 172.21.15.120:0/1616156684 learned_addr learned my addr 172.21.15.120:0/1616156684 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:17.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.684+0000 7f3dc77fe640 1 --2- 172.21.15.120:0/1616156684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55589be42840 0x55589be43180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:17.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.684+0000 7f3dc77fe640 1 -- 172.21.15.120:0/1616156684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55589be42840 msgr2=0x55589be43180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:17.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.684+0000 7f3dc77fe640 1 --2- 172.21.15.120:0/1616156684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55589be42840 0x55589be43180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.684+0000 7f3dc77fe640 1 -- 172.21.15.120:0/1616156684 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55589be41b80 msgr2=0x55589be3cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:17.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.684+0000 7f3dc77fe640 1 --2- 172.21.15.120:0/1616156684 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55589be41b80 0x55589be3cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.684+0000 7f3dc77fe640 1 -- 172.21.15.120:0/1616156684 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55589be3e960 con 0x55589be41120 2024-09-19T09:12:17.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.684+0000 7f3dc6ffd640 1 --2- 172.21.15.120:0/1616156684 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55589be41b80 0x55589be3cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:17.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.685+0000 7f3dc77fe640 1 --2- 172.21.15.120:0/1616156684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55589be41120 0x55589be60bd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000a550 tx=0x7f3dc0005c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83e6051b551c9da4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:17.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.685+0000 7f3dad7fa640 1 -- 172.21.15.120:0/1616156684 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc00228c0 con 0x55589be41120 2024-09-19T09:12:17.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.685+0000 7f3dad7fa640 1 -- 172.21.15.120:0/1616156684 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dc0022a60 con 0x55589be41120 2024-09-19T09:12:17.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.685+0000 7f3dad7fa640 1 -- 172.21.15.120:0/1616156684 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0021420 con 0x55589be41120 2024-09-19T09:12:17.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.686+0000 7f3dcfd68740 1 -- 172.21.15.120:0/1616156684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55589be41120 msgr2=0x55589be60bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:17.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.686+0000 7f3dcfd68740 1 --2- 172.21.15.120:0/1616156684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55589be41120 0x55589be60bd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000a550 tx=0x7f3dc0005c30 comp rx=0 tx=0).stop 2024-09-19T09:12:17.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.686+0000 7f3dcfd68740 1 -- 172.21.15.120:0/1616156684 shutdown_connections 2024-09-19T09:12:17.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.686+0000 7f3dcfd68740 1 --2- 172.21.15.120:0/1616156684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55589be42840 0x55589be43180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.686+0000 7f3dcfd68740 1 --2- 172.21.15.120:0/1616156684 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55589be41b80 0x55589be3cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.686+0000 7f3dcfd68740 1 --2- 172.21.15.120:0/1616156684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55589be41120 0x55589be60bd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.686+0000 7f3dcfd68740 1 -- 172.21.15.120:0/1616156684 >> 172.21.15.120:0/1616156684 conn(0x55589be0a310 msgr2=0x55589be3c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:17.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.686+0000 7f3dcfd68740 1 -- 172.21.15.120:0/1616156684 shutdown_connections 2024-09-19T09:12:17.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.686+0000 7f3dcfd68740 1 -- 172.21.15.120:0/1616156684 wait complete. 2024-09-19T09:12:17.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.686+0000 7f3dcfd68740 1 -- start start 2024-09-19T09:12:17.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.686+0000 7f3dcfd68740 1 Processor -- start 2024-09-19T09:12:17.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.687+0000 7f3dcfd68740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55589be41120 0x55589be6b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:17.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.687+0000 7f3dcfd68740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55589be41b80 0x55589be6b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:17.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.687+0000 7f3dcfd68740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55589be42840 0x55589be6bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:17.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.687+0000 7f3dcfd68740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81ac880 con 0x55589be41b80 2024-09-19T09:12:17.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.687+0000 7f3dcfd68740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81ac1b0 con 0x55589be41120 2024-09-19T09:12:17.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.687+0000 7f3dcfd68740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81a93d0 con 0x55589be42840 2024-09-19T09:12:17.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.687+0000 7f3dcfd68740 10 client.0 _ll_register_callbacks cb 0x7f3dc000aab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:17.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.687+0000 7f3dc6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55589be41b80 0x55589be6b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:17.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.687+0000 7f3dc77fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55589be41120 0x55589be6b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:17.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.687+0000 7f3dc6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55589be41b80 0x55589be6b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56172/0 (socket says 172.21.15.120:56172) 2024-09-19T09:12:17.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.687+0000 7f3dc77fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55589be41120 0x55589be6b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52052/0 (socket says 172.21.15.120:52052) 2024-09-19T09:12:17.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.687+0000 7f3dc6ffd640 1 -- 172.21.15.120:0/2953699009 learned_addr learned my addr 172.21.15.120:0/2953699009 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:17.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.687+0000 7f3dc77fe640 1 --2- 172.21.15.120:0/2953699009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55589be42840 0x55589be6bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:17.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.687+0000 7f3dc6ffd640 1 -- 172.21.15.120:0/2953699009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55589be42840 msgr2=0x55589be6bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:17.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.687+0000 7f3dc6ffd640 1 --2- 172.21.15.120:0/2953699009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55589be42840 0x55589be6bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.687+0000 7f3dc6ffd640 1 -- 172.21.15.120:0/2953699009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55589be41120 msgr2=0x55589be6b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:17.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.687+0000 7f3dc6ffd640 1 --2- 172.21.15.120:0/2953699009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55589be41120 0x55589be6b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.687+0000 7f3dc6ffd640 1 -- 172.21.15.120:0/2953699009 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55589be5bdf0 con 0x55589be41b80 2024-09-19T09:12:17.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.687+0000 7f3dc77fe640 1 --2- 172.21.15.120:0/2953699009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55589be41120 0x55589be6b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:17.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.687+0000 7f3dc6ffd640 1 --2- 172.21.15.120:0/2953699009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55589be41b80 0x55589be6b680 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f3db8011c20 tx=0x7f3db8005210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cbfd58f19bc9989 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:17.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.687+0000 7f3da6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:17.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.687+0000 7f3da6ffd640 1 -- 172.21.15.120:0/2953699009 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db8035070 con 0x55589be41b80 2024-09-19T09:12:17.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.687+0000 7f3da6ffd640 1 -- 172.21.15.120:0/2953699009 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f3db8004960 con 0x55589be41b80 2024-09-19T09:12:17.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.687+0000 7f3dcfd68740 1 -- 172.21.15.120:0/2953699009 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dc0022a60 con 0x55589be41b80 2024-09-19T09:12:17.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.688+0000 7f3da6ffd640 1 -- 172.21.15.120:0/2953699009 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3db8041050 con 0x55589be41b80 2024-09-19T09:12:17.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.688+0000 7f3dcfd68740 10 client.18765 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:17.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.688+0000 7f3dcfd68740 1 -- 172.21.15.120:0/2953699009 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55589be5d0d0 con 0x55589be41b80 2024-09-19T09:12:17.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.688+0000 7f3da6ffd640 1 -- 172.21.15.120:0/2953699009 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dc0022a60 con 0x55589be41b80 2024-09-19T09:12:17.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.688+0000 7f3da6ffd640 1 -- 172.21.15.120:0/2953699009 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f3db802f070 con 0x55589be41b80 2024-09-19T09:12:17.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.688+0000 7f3dcfd68740 10 client.18765 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:17.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.688+0000 7f3dcfd68740 10 client.18765 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:12:17.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.688+0000 7f3dcfd68740 20 client.18765 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:17.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.688+0000 7f3dcfd68740 1 --2- 172.21.15.120:0/2953699009 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55589becf120 0x55589bed1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:17.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.688+0000 7f3dcfd68740 4 client.18765 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:12:17.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.688+0000 7f3dcfd68740 1 -- 172.21.15.120:0/2953699009 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55589bdaed70 con 0x55589becf120 2024-09-19T09:12:17.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.689+0000 7f3dc77fe640 1 --2- 172.21.15.120:0/2953699009 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55589becf120 0x55589bed1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:17.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.689+0000 7f3dc77fe640 1 --2- 172.21.15.120:0/2953699009 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55589becf120 0x55589bed1510 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b4b09348966b72fd server_cookie=4c2b300b9e131a8d in_seq=0 out_seq=0 2024-09-19T09:12:17.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.689+0000 7f3da6ffd640 10 client.18765 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:12:17.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.689+0000 7f3da6ffd640 1 -- 172.21.15.120:0/2953699009 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55589bdaed70 con 0x55589becf120 2024-09-19T09:12:17.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.689+0000 7f3da6ffd640 10 client.18765 handle_command_reply: tid=0 2024-09-19T09:12:17.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.689+0000 7f3dcfd68740 1 client.18765 shutdown 2024-09-19T09:12:17.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.689+0000 7f3dcfd68740 1 -- 172.21.15.120:0/2953699009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55589be41b80 msgr2=0x55589be6b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:17.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.689+0000 7f3dcfd68740 1 --2- 172.21.15.120:0/2953699009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55589be41b80 0x55589be6b680 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f3db8011c20 tx=0x7f3db8005210 comp rx=0 tx=0).stop 2024-09-19T09:12:17.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.689+0000 7f3dcfd68740 1 -- 172.21.15.120:0/2953699009 shutdown_connections 2024-09-19T09:12:17.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.689+0000 7f3dcfd68740 1 --2- 172.21.15.120:0/2953699009 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55589becf120 0x55589bed1510 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.689+0000 7f3dcfd68740 1 --2- 172.21.15.120:0/2953699009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55589be42840 0x55589be6bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.689+0000 7f3dcfd68740 1 --2- 172.21.15.120:0/2953699009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55589be41b80 0x55589be6b680 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.689+0000 7f3dcfd68740 1 --2- 172.21.15.120:0/2953699009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55589be41120 0x55589be6b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.689+0000 7f3dcfd68740 1 -- 172.21.15.120:0/2953699009 >> 172.21.15.120:0/2953699009 conn(0x55589be0a310 msgr2=0x55589be53130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:17.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.689+0000 7f3da6ffd640 0 client.18765 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:12:17.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.690+0000 7f3dcfd68740 1 -- 172.21.15.120:0/2953699009 shutdown_connections 2024-09-19T09:12:17.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.690+0000 7f3dcfd68740 1 -- 172.21.15.120:0/2953699009 wait complete. 2024-09-19T09:12:17.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.690+0000 7f3dcfd68740 20 client.18765 trim_cache size 0 max 16384 2024-09-19T09:12:17.697 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:12:17.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.690+0000 7f3dcd98f640 1 -- 172.21.15.120:0/4116468462 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3db00a3240 msgr2=0x7f3db00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:17.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.690+0000 7f3dcd98f640 1 --2- 172.21.15.120:0/4116468462 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3db00a3240 0x7f3db00a5700 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc0b8a30 tx=0x7f3dbc0d5050 comp rx=0 tx=0).stop 2024-09-19T09:12:17.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.690+0000 7f3dcd98f640 1 -- 172.21.15.120:0/4116468462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3dc8101e20 msgr2=0x7f3dc81989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:17.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.690+0000 7f3dcd98f640 1 --2- 172.21.15.120:0/4116468462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3dc8101e20 0x7f3dc81989a0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc006010 tx=0x7f3dbc0042f0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.691+0000 7f3dcd98f640 1 -- 172.21.15.120:0/4116468462 shutdown_connections 2024-09-19T09:12:17.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.691+0000 7f3dcd98f640 1 --2- 172.21.15.120:0/4116468462 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3db00a3240 0x7f3db00a5700 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.691+0000 7f3dcd98f640 1 --2- 172.21.15.120:0/4116468462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3dc8105de0 0x7f3dc819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.691+0000 7f3dcd98f640 1 --2- 172.21.15.120:0/4116468462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3dc8101e20 0x7f3dc81989a0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.691+0000 7f3dcd98f640 1 --2- 172.21.15.120:0/4116468462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3dc8101470 0x7f3dc8198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:17.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.691+0000 7f3dcd98f640 1 -- 172.21.15.120:0/4116468462 >> 172.21.15.120:0/4116468462 conn(0x7f3dc8077230 msgr2=0x7f3dc81045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:17.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.691+0000 7f3dcd98f640 1 -- 172.21.15.120:0/4116468462 shutdown_connections 2024-09-19T09:12:17.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:17.691+0000 7f3dcd98f640 1 -- 172.21.15.120:0/4116468462 wait complete. 2024-09-19T09:12:18.065 INFO:tasks.workunit.client.0.smithi120.stdout: 1 909641 9.90 MB/sec execute 373 sec latency 34.519 ms 2024-09-19T09:12:18.289 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:17 smithi120 ceph-mon[31150]: pgmap v626: 97 pgs: 97 active+clean; 50 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 412 op/s 2024-09-19T09:12:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.396+0000 7f7413dc4640 1 -- 172.21.15.120:0/977491773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f740c102600 msgr2=0x7f740c108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.396+0000 7f7413dc4640 1 --2- 172.21.15.120:0/977491773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f740c102600 0x7f740c108e90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7404009be0 tx=0x7f740402a6c0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.396+0000 7f7413dc4640 1 -- 172.21.15.120:0/977491773 shutdown_connections 2024-09-19T09:12:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.396+0000 7f7413dc4640 1 --2- 172.21.15.120:0/977491773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f740c102600 0x7f740c108e90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.396+0000 7f7413dc4640 1 --2- 172.21.15.120:0/977491773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f740c101c40 0x7f740c1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.396+0000 7f7413dc4640 1 --2- 172.21.15.120:0/977491773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c100a50 0x7f740c100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.396+0000 7f7413dc4640 1 -- 172.21.15.120:0/977491773 >> 172.21.15.120:0/977491773 conn(0x7f740c0fc1b0 msgr2=0x7f740c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.397+0000 7f7413dc4640 1 -- 172.21.15.120:0/977491773 shutdown_connections 2024-09-19T09:12:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.397+0000 7f7413dc4640 1 -- 172.21.15.120:0/977491773 wait complete. 2024-09-19T09:12:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.397+0000 7f7413dc4640 1 Processor -- start 2024-09-19T09:12:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.397+0000 7f7413dc4640 1 -- start start 2024-09-19T09:12:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.397+0000 7f7413dc4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f740c100a50 0x7f740c197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.397+0000 7f7413dc4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f740c101c40 0x7f740c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.397+0000 7f7413dc4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c102600 0x7f740c19f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.397+0000 7f7413dc4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f740c10c6b0 con 0x7f740c101c40 2024-09-19T09:12:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.398+0000 7f7413dc4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f740c10c530 con 0x7f740c100a50 2024-09-19T09:12:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.398+0000 7f7413dc4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f740c10c830 con 0x7f740c102600 2024-09-19T09:12:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.398+0000 7f7411b60640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f740c100a50 0x7f740c197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.398+0000 7f741135f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f740c101c40 0x7f740c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.398+0000 7f741135f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f740c101c40 0x7f740c198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56196/0 (socket says 172.21.15.120:56196) 2024-09-19T09:12:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.398+0000 7f7411b60640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f740c100a50 0x7f740c197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52064/0 (socket says 172.21.15.120:52064) 2024-09-19T09:12:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.398+0000 7f741135f640 1 -- 172.21.15.120:0/1699565869 learned_addr learned my addr 172.21.15.120:0/1699565869 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.398+0000 7f7412361640 1 --2- 172.21.15.120:0/1699565869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c102600 0x7f740c19f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.398+0000 7f7412361640 1 -- 172.21.15.120:0/1699565869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f740c100a50 msgr2=0x7f740c197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.398+0000 7f7412361640 1 --2- 172.21.15.120:0/1699565869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f740c100a50 0x7f740c197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.398+0000 7f7412361640 1 -- 172.21.15.120:0/1699565869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f740c101c40 msgr2=0x7f740c198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.398+0000 7f7412361640 1 --2- 172.21.15.120:0/1699565869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f740c101c40 0x7f740c198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.398+0000 7f7412361640 1 -- 172.21.15.120:0/1699565869 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740c19f9d0 con 0x7f740c102600 2024-09-19T09:12:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.398+0000 7f7411b60640 1 --2- 172.21.15.120:0/1699565869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f740c100a50 0x7f740c197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.398+0000 7f7412361640 1 --2- 172.21.15.120:0/1699565869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c102600 0x7f740c19f330 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7404034040 tx=0x7f74040305b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.399+0000 7f7402ffd640 1 -- 172.21.15.120:0/1699565869 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7404041020 con 0x7f740c102600 2024-09-19T09:12:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.399+0000 7f7413dc4640 1 -- 172.21.15.120:0/1699565869 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f740c19fc60 con 0x7f740c102600 2024-09-19T09:12:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.399+0000 7f7413dc4640 1 -- 172.21.15.120:0/1699565869 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f740c1a0170 con 0x7f740c102600 2024-09-19T09:12:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.399+0000 7f7402ffd640 1 -- 172.21.15.120:0/1699565869 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7404030720 con 0x7f740c102600 2024-09-19T09:12:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.399+0000 7f7402ffd640 1 -- 172.21.15.120:0/1699565869 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7404036040 con 0x7f740c102600 2024-09-19T09:12:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.401+0000 7f7416c3a740 1 Processor -- start 2024-09-19T09:12:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.401+0000 7f7402ffd640 1 -- 172.21.15.120:0/1699565869 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f7404044020 con 0x7f740c102600 2024-09-19T09:12:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.402+0000 7f7416c3a740 1 -- start start 2024-09-19T09:12:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.402+0000 7f7416c3a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559df42be9a0 0x559df42beda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.402+0000 7f7416c3a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559df42bf2e0 0x559df42c3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.402+0000 7f7416c3a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559df42c4370 0x559df42c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.402+0000 7f7416c3a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559df42222f0 con 0x559df42c4370 2024-09-19T09:12:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.402+0000 7f7416c3a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559df41ca270 con 0x559df42be9a0 2024-09-19T09:12:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.402+0000 7f7416c3a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559df421e2b0 con 0x559df42bf2e0 2024-09-19T09:12:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.402+0000 7f7411b60640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559df42bf2e0 0x559df42c3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.402+0000 7f741135f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559df42be9a0 0x559df42beda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.402+0000 7f7411b60640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559df42bf2e0 0x559df42c3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:52018/0 (socket says 172.21.15.120:52018) 2024-09-19T09:12:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.402+0000 7f7411b60640 1 -- 172.21.15.120:0/558090542 learned_addr learned my addr 172.21.15.120:0/558090542 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.402+0000 7f741135f640 1 --2- 172.21.15.120:0/558090542 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559df42c4370 0x559df42c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.402+0000 7f7411b60640 1 -- 172.21.15.120:0/558090542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559df42be9a0 msgr2=0x559df42beda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.402+0000 7f7411b60640 1 --2- 172.21.15.120:0/558090542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559df42be9a0 0x559df42beda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.402+0000 7f7411b60640 1 -- 172.21.15.120:0/558090542 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559df42c4370 msgr2=0x559df42c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.402+0000 7f7411b60640 1 --2- 172.21.15.120:0/558090542 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559df42c4370 0x559df42c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.402+0000 7f7411b60640 1 -- 172.21.15.120:0/558090542 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559df41c9340 con 0x559df42bf2e0 2024-09-19T09:12:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.403+0000 7f7402ffd640 1 --2- 172.21.15.120:0/1699565869 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f73f80a32f0 0x7f73f80a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.403+0000 7f7411b60640 1 --2- 172.21.15.120:0/558090542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559df42bf2e0 0x559df42c3da0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f74080104d0 tx=0x7f7408010990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e0be4864563500a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.403+0000 7f741135f640 1 --2- 172.21.15.120:0/1699565869 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f73f80a32f0 0x7f73f80a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.403+0000 7f73f7fff640 1 -- 172.21.15.120:0/558090542 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7408004230 con 0x559df42bf2e0 2024-09-19T09:12:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.403+0000 7f7402ffd640 1 -- 172.21.15.120:0/1699565869 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f74040f06c0 con 0x7f740c102600 2024-09-19T09:12:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.403+0000 7f73f7fff640 1 -- 172.21.15.120:0/558090542 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74080043d0 con 0x559df42bf2e0 2024-09-19T09:12:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.403+0000 7f73f7fff640 1 -- 172.21.15.120:0/558090542 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7408003ac0 con 0x559df42bf2e0 2024-09-19T09:12:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.404+0000 7f7416c3a740 1 -- 172.21.15.120:0/558090542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559df42bf2e0 msgr2=0x559df42c3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.404+0000 7f741135f640 1 --2- 172.21.15.120:0/1699565869 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f73f80a32f0 0x7f73f80a57b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0096c0 tx=0x7f73fc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.404+0000 7f7416c3a740 1 --2- 172.21.15.120:0/558090542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559df42bf2e0 0x559df42c3da0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f74080104d0 tx=0x7f7408010990 comp rx=0 tx=0).stop 2024-09-19T09:12:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.404+0000 7f7416c3a740 1 -- 172.21.15.120:0/558090542 shutdown_connections 2024-09-19T09:12:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.404+0000 7f7416c3a740 1 --2- 172.21.15.120:0/558090542 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559df42c4370 0x559df42c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.404+0000 7f7416c3a740 1 --2- 172.21.15.120:0/558090542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559df42bf2e0 0x559df42c3da0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.404+0000 7f7416c3a740 1 --2- 172.21.15.120:0/558090542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559df42be9a0 0x559df42beda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.404+0000 7f7416c3a740 1 -- 172.21.15.120:0/558090542 >> 172.21.15.120:0/558090542 conn(0x559df42ba010 msgr2=0x559df42ba410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.404+0000 7f7416c3a740 1 -- 172.21.15.120:0/558090542 shutdown_connections 2024-09-19T09:12:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.404+0000 7f7416c3a740 1 -- 172.21.15.120:0/558090542 wait complete. 2024-09-19T09:12:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.404+0000 7f7416c3a740 1 -- start start 2024-09-19T09:12:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.404+0000 7f7416c3a740 1 Processor -- start 2024-09-19T09:12:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.405+0000 7f7416c3a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559df42be9a0 0x559df42ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.405+0000 7f7416c3a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559df42bf2e0 0x559df42cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.405+0000 7f7416c3a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559df42c4370 0x559df42d44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.405+0000 7f7416c3a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559df42c7340 con 0x559df42c4370 2024-09-19T09:12:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.405+0000 7f7416c3a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559df41ab290 con 0x559df42bf2e0 2024-09-19T09:12:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.405+0000 7f7416c3a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559df42c74c0 con 0x559df42be9a0 2024-09-19T09:12:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.405+0000 7f7416c3a740 10 client.0 _ll_register_callbacks cb 0x559df41dc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.405+0000 7f7411b60640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559df42be9a0 0x559df42ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.405+0000 7f741135f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559df42bf2e0 0x559df42cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.405+0000 7f7411b60640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559df42be9a0 0x559df42ccf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:52030/0 (socket says 172.21.15.120:52030) 2024-09-19T09:12:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.405+0000 7f7411b60640 1 -- 172.21.15.120:0/2723607973 learned_addr learned my addr 172.21.15.120:0/2723607973 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.405+0000 7f7412361640 1 --2- 172.21.15.120:0/2723607973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559df42c4370 0x559df42d44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.405+0000 7f7411b60640 1 -- 172.21.15.120:0/2723607973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559df42bf2e0 msgr2=0x559df42cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.405+0000 7f7411b60640 1 --2- 172.21.15.120:0/2723607973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559df42bf2e0 0x559df42cd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.405+0000 7f7411b60640 1 -- 172.21.15.120:0/2723607973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559df42c4370 msgr2=0x559df42d44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.405+0000 7f7411b60640 1 --2- 172.21.15.120:0/2723607973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559df42c4370 0x559df42d44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.405+0000 7f7411b60640 1 -- 172.21.15.120:0/2723607973 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559df3f35210 con 0x559df42be9a0 2024-09-19T09:12:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.405+0000 7f7411b60640 1 --2- 172.21.15.120:0/2723607973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559df42be9a0 0x559df42ccf70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7408006010 tx=0x7f740800be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f192660f3e8f38e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.405+0000 7f73f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:12:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.406+0000 7f73f57fa640 1 -- 172.21.15.120:0/2723607973 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7408005600 con 0x559df42be9a0 2024-09-19T09:12:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.406+0000 7f73f57fa640 1 -- 172.21.15.120:0/2723607973 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f7408011e00 con 0x559df42be9a0 2024-09-19T09:12:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.406+0000 7f7416c3a740 1 -- 172.21.15.120:0/2723607973 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74080043d0 con 0x559df42be9a0 2024-09-19T09:12:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.406+0000 7f73f57fa640 1 -- 172.21.15.120:0/2723607973 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7408011620 con 0x559df42be9a0 2024-09-19T09:12:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.406+0000 7f7416c3a740 10 client.27736 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.406+0000 7f7416c3a740 1 -- 172.21.15.120:0/2723607973 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559df4001620 con 0x559df42be9a0 2024-09-19T09:12:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.406+0000 7f73f57fa640 1 -- 172.21.15.120:0/2723607973 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74080043d0 con 0x559df42be9a0 2024-09-19T09:12:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.406+0000 7f73f57fa640 1 -- 172.21.15.120:0/2723607973 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f7408026420 con 0x559df42be9a0 2024-09-19T09:12:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.406+0000 7f7416c3a740 10 client.27736 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.406+0000 7f7416c3a740 10 client.27736 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:12:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.406+0000 7f7416c3a740 20 client.27736 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.406+0000 7f7416c3a740 1 --2- 172.21.15.120:0/2723607973 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559df42d8970 0x559df42dadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.406+0000 7f7416c3a740 4 client.27736 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:12:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.406+0000 7f7416c3a740 1 -- 172.21.15.120:0/2723607973 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559df422fa20 con 0x559df42d8970 2024-09-19T09:12:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.406+0000 7f741135f640 1 --2- 172.21.15.120:0/2723607973 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559df42d8970 0x559df42dadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.407+0000 7f73f57fa640 10 client.27736 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.407+0000 7f741135f640 1 --2- 172.21.15.120:0/2723607973 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559df42d8970 0x559df42dadf0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=110315be7f0517ab server_cookie=d77cb4779a68be91 in_seq=0 out_seq=0 2024-09-19T09:12:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.408+0000 7f73f57fa640 1 -- 172.21.15.120:0/2723607973 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559df422fa20 con 0x559df42d8970 2024-09-19T09:12:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.408+0000 7f73f57fa640 10 client.27736 handle_command_reply: tid=0 2024-09-19T09:12:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.408+0000 7f7416c3a740 1 client.27736 shutdown 2024-09-19T09:12:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.408+0000 7f7416c3a740 1 -- 172.21.15.120:0/2723607973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559df42be9a0 msgr2=0x559df42ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.408+0000 7f7416c3a740 1 --2- 172.21.15.120:0/2723607973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559df42be9a0 0x559df42ccf70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7408006010 tx=0x7f740800be30 comp rx=0 tx=0).stop 2024-09-19T09:12:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.408+0000 7f7416c3a740 1 -- 172.21.15.120:0/2723607973 shutdown_connections 2024-09-19T09:12:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.408+0000 7f7416c3a740 1 --2- 172.21.15.120:0/2723607973 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559df42d8970 0x559df42dadf0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.408+0000 7f7416c3a740 1 --2- 172.21.15.120:0/2723607973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559df42c4370 0x559df42d44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.408+0000 7f73f57fa640 0 client.27736 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.408+0000 7f7416c3a740 1 --2- 172.21.15.120:0/2723607973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559df42bf2e0 0x559df42cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.408+0000 7f7416c3a740 1 --2- 172.21.15.120:0/2723607973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559df42be9a0 0x559df42ccf70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.408+0000 7f7416c3a740 1 -- 172.21.15.120:0/2723607973 >> 172.21.15.120:0/2723607973 conn(0x559df42ba010 msgr2=0x559df42bb820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.408+0000 7f7416c3a740 1 -- 172.21.15.120:0/2723607973 shutdown_connections 2024-09-19T09:12:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.408+0000 7f7416c3a740 1 -- 172.21.15.120:0/2723607973 wait complete. 2024-09-19T09:12:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.408+0000 7f7416c3a740 20 client.27736 trim_cache size 0 max 16384 2024-09-19T09:12:18.422 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:12:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.425+0000 7f7416c3a740 1 Processor -- start 2024-09-19T09:12:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.425+0000 7f7416c3a740 1 -- start start 2024-09-19T09:12:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.425+0000 7f7416c3a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559df42be9a0 0x559df42b90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.425+0000 7f7416c3a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559df42bf2e0 0x559df42d44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.425+0000 7f7416c3a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559df42c4370 0x559df42bb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.425+0000 7f7416c3a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f740c1a6e00 con 0x559df42be9a0 2024-09-19T09:12:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.425+0000 7f7416c3a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f740c1a8940 con 0x559df42bf2e0 2024-09-19T09:12:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.425+0000 7f7416c3a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f740c1a74d0 con 0x559df42c4370 2024-09-19T09:12:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.425+0000 7f7411b60640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559df42be9a0 0x559df42b90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.425+0000 7f7412361640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559df42c4370 0x559df42bb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.425+0000 7f7411b60640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559df42be9a0 0x559df42b90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56220/0 (socket says 172.21.15.120:56220) 2024-09-19T09:12:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.425+0000 7f7411b60640 1 -- 172.21.15.120:0/1417270103 learned_addr learned my addr 172.21.15.120:0/1417270103 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.425+0000 7f7412361640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559df42c4370 0x559df42bb400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:52034/0 (socket says 172.21.15.120:52034) 2024-09-19T09:12:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.425+0000 7f741135f640 1 --2- 172.21.15.120:0/1417270103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559df42bf2e0 0x559df42d44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.426+0000 7f7411b60640 1 -- 172.21.15.120:0/1417270103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559df42c4370 msgr2=0x559df42bb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.426+0000 7f7411b60640 1 --2- 172.21.15.120:0/1417270103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559df42c4370 0x559df42bb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.426+0000 7f7411b60640 1 -- 172.21.15.120:0/1417270103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559df42bf2e0 msgr2=0x559df42d44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.426+0000 7f7411b60640 1 --2- 172.21.15.120:0/1417270103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559df42bf2e0 0x559df42d44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.426+0000 7f7411b60640 1 -- 172.21.15.120:0/1417270103 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559df42b9e30 con 0x559df42be9a0 2024-09-19T09:12:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.426+0000 7f7412361640 1 --2- 172.21.15.120:0/1417270103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559df42c4370 0x559df42bb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.426+0000 7f7411b60640 1 --2- 172.21.15.120:0/1417270103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559df42be9a0 0x559df42b90b0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f7408000c00 tx=0x7f74080035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab09879f515e1aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.426+0000 7f73f77fe640 1 -- 172.21.15.120:0/1417270103 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7408005600 con 0x559df42be9a0 2024-09-19T09:12:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.426+0000 7f73f77fe640 1 -- 172.21.15.120:0/1417270103 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7408011ca0 con 0x559df42be9a0 2024-09-19T09:12:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.426+0000 7f73f77fe640 1 -- 172.21.15.120:0/1417270103 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7408004850 con 0x559df42be9a0 2024-09-19T09:12:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.427+0000 7f7416c3a740 1 -- 172.21.15.120:0/1417270103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559df42be9a0 msgr2=0x559df42b90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.427+0000 7f7416c3a740 1 --2- 172.21.15.120:0/1417270103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559df42be9a0 0x559df42b90b0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f7408000c00 tx=0x7f74080035f0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.427+0000 7f7416c3a740 1 -- 172.21.15.120:0/1417270103 shutdown_connections 2024-09-19T09:12:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.427+0000 7f7416c3a740 1 --2- 172.21.15.120:0/1417270103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559df42c4370 0x559df42bb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.427+0000 7f7416c3a740 1 --2- 172.21.15.120:0/1417270103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559df42bf2e0 0x559df42d44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.427+0000 7f7416c3a740 1 --2- 172.21.15.120:0/1417270103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559df42be9a0 0x559df42b90b0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.427+0000 7f7416c3a740 1 -- 172.21.15.120:0/1417270103 >> 172.21.15.120:0/1417270103 conn(0x559df42ba010 msgr2=0x559df42b89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.427+0000 7f7416c3a740 1 -- 172.21.15.120:0/1417270103 shutdown_connections 2024-09-19T09:12:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.427+0000 7f7416c3a740 1 -- 172.21.15.120:0/1417270103 wait complete. 2024-09-19T09:12:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.427+0000 7f7416c3a740 1 -- start start 2024-09-19T09:12:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.427+0000 7f7416c3a740 1 Processor -- start 2024-09-19T09:12:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.428+0000 7f7416c3a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559df42be9a0 0x559df42d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.428+0000 7f7416c3a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559df42bf2e0 0x559df42d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.428+0000 7f7416c3a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559df42c4370 0x559df42d2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.428+0000 7f7416c3a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f740c1a8270 con 0x559df42c4370 2024-09-19T09:12:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.428+0000 7f7416c3a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f740c1a7ba0 con 0x559df42bf2e0 2024-09-19T09:12:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.428+0000 7f7416c3a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f740c1a4dc0 con 0x559df42be9a0 2024-09-19T09:12:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.428+0000 7f7416c3a740 10 client.0 _ll_register_callbacks cb 0x7f73fc009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.428+0000 7f7412361640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559df42c4370 0x559df42d2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.428+0000 7f7411b60640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559df42be9a0 0x559df42d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.428+0000 7f741135f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559df42bf2e0 0x559df42d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.428+0000 7f7411b60640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559df42be9a0 0x559df42d19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:52040/0 (socket says 172.21.15.120:52040) 2024-09-19T09:12:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.428+0000 7f7412361640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559df42c4370 0x559df42d2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56232/0 (socket says 172.21.15.120:56232) 2024-09-19T09:12:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.428+0000 7f7411b60640 1 -- 172.21.15.120:0/507296673 learned_addr learned my addr 172.21.15.120:0/507296673 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.428+0000 7f7411b60640 1 -- 172.21.15.120:0/507296673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559df42bf2e0 msgr2=0x559df42d1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.428+0000 7f7411b60640 1 --2- 172.21.15.120:0/507296673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559df42bf2e0 0x559df42d1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.428+0000 7f7411b60640 1 -- 172.21.15.120:0/507296673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559df42c4370 msgr2=0x559df42d2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.428+0000 7f7411b60640 1 --2- 172.21.15.120:0/507296673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559df42c4370 0x559df42d2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.428+0000 7f7411b60640 1 -- 172.21.15.120:0/507296673 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559df42d3540 con 0x559df42be9a0 2024-09-19T09:12:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.428+0000 7f741135f640 1 --2- 172.21.15.120:0/507296673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559df42bf2e0 0x559df42d1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.428+0000 7f7412361640 1 --2- 172.21.15.120:0/507296673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559df42c4370 0x559df42d2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.428+0000 7f7411b60640 1 --2- 172.21.15.120:0/507296673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559df42be9a0 0x559df42d19e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f74080109d0 tx=0x7f7408026730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f2ec54bf27b4d9b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.428+0000 7f73f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:12:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.428+0000 7f73f57fa640 1 -- 172.21.15.120:0/507296673 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7408005600 con 0x559df42be9a0 2024-09-19T09:12:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.429+0000 7f73f57fa640 1 -- 172.21.15.120:0/507296673 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f74080119d0 con 0x559df42be9a0 2024-09-19T09:12:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.429+0000 7f7416c3a740 1 -- 172.21.15.120:0/507296673 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7408011ca0 con 0x559df42be9a0 2024-09-19T09:12:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.429+0000 7f73f57fa640 1 -- 172.21.15.120:0/507296673 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7408049050 con 0x559df42be9a0 2024-09-19T09:12:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.429+0000 7f7416c3a740 10 client.27742 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.429+0000 7f7416c3a740 1 -- 172.21.15.120:0/507296673 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559df42d4850 con 0x559df42be9a0 2024-09-19T09:12:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.429+0000 7f73f57fa640 1 -- 172.21.15.120:0/507296673 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7408011ca0 con 0x559df42be9a0 2024-09-19T09:12:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.429+0000 7f73f57fa640 1 -- 172.21.15.120:0/507296673 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f7408039420 con 0x559df42be9a0 2024-09-19T09:12:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.429+0000 7f7416c3a740 10 client.27742 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.429+0000 7f7416c3a740 10 client.27742 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:12:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.429+0000 7f7416c3a740 20 client.27742 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.429+0000 7f7416c3a740 1 --2- 172.21.15.120:0/507296673 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559df4343e10 0x559df4346230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.429+0000 7f7416c3a740 4 client.27742 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:12:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.429+0000 7f7416c3a740 1 -- 172.21.15.120:0/507296673 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559df4229d70 con 0x559df4343e10 2024-09-19T09:12:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.429+0000 7f741135f640 1 --2- 172.21.15.120:0/507296673 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559df4343e10 0x559df4346230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.430+0000 7f741135f640 1 --2- 172.21.15.120:0/507296673 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559df4343e10 0x559df4346230 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ea826b7e86dc779 server_cookie=89cae229a662f561 in_seq=0 out_seq=0 2024-09-19T09:12:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.430+0000 7f73f57fa640 10 client.27742 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:18.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:17 smithi153 ceph-mon[35071]: pgmap v626: 97 pgs: 97 active+clean; 50 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 412 op/s 2024-09-19T09:12:18.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:17 smithi196 ceph-mon[35150]: pgmap v626: 97 pgs: 97 active+clean; 50 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 412 op/s 2024-09-19T09:12:18.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.596+0000 7f73f57fa640 1 -- 172.21.15.120:0/507296673 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559df4229d70 con 0x559df4343e10 2024-09-19T09:12:18.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.596+0000 7f73f57fa640 10 client.27742 handle_command_reply: tid=0 2024-09-19T09:12:18.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.596+0000 7f7416c3a740 1 client.27742 shutdown 2024-09-19T09:12:18.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.596+0000 7f7416c3a740 1 -- 172.21.15.120:0/507296673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559df42be9a0 msgr2=0x559df42d19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:18.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.596+0000 7f7416c3a740 1 --2- 172.21.15.120:0/507296673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559df42be9a0 0x559df42d19e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f74080109d0 tx=0x7f7408026730 comp rx=0 tx=0).stop 2024-09-19T09:12:18.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.596+0000 7f7416c3a740 1 -- 172.21.15.120:0/507296673 shutdown_connections 2024-09-19T09:12:18.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.596+0000 7f7416c3a740 1 --2- 172.21.15.120:0/507296673 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559df4343e10 0x559df4346230 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.596+0000 7f73f57fa640 0 client.27742 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:18.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.596+0000 7f7416c3a740 1 --2- 172.21.15.120:0/507296673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559df42c4370 0x559df42d2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.596+0000 7f7416c3a740 1 --2- 172.21.15.120:0/507296673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559df42bf2e0 0x559df42d1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.596+0000 7f7416c3a740 1 --2- 172.21.15.120:0/507296673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559df42be9a0 0x559df42d19e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.596+0000 7f7416c3a740 1 -- 172.21.15.120:0/507296673 >> 172.21.15.120:0/507296673 conn(0x559df42ba010 msgr2=0x559df42b91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:18.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.597+0000 7f7416c3a740 1 -- 172.21.15.120:0/507296673 shutdown_connections 2024-09-19T09:12:18.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.597+0000 7f7416c3a740 1 -- 172.21.15.120:0/507296673 wait complete. 2024-09-19T09:12:18.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.597+0000 7f7416c3a740 20 client.27742 trim_cache size 0 max 16384 2024-09-19T09:12:18.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:12:18.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:12:18.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:12:18.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:12:18.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.597+0000 7f7413dc4640 1 -- 172.21.15.120:0/1699565869 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f73f80a32f0 msgr2=0x7f73f80a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:18.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.597+0000 7f7413dc4640 1 --2- 172.21.15.120:0/1699565869 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f73f80a32f0 0x7f73f80a57b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0096c0 tx=0x7f73fc009260 comp rx=0 tx=0).stop 2024-09-19T09:12:18.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.597+0000 7f7413dc4640 1 -- 172.21.15.120:0/1699565869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c102600 msgr2=0x7f740c19f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:18.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.597+0000 7f7413dc4640 1 --2- 172.21.15.120:0/1699565869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c102600 0x7f740c19f330 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7404034040 tx=0x7f74040305b0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.598+0000 7f7413dc4640 1 -- 172.21.15.120:0/1699565869 shutdown_connections 2024-09-19T09:12:18.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.598+0000 7f7413dc4640 1 --2- 172.21.15.120:0/1699565869 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f73f80a32f0 0x7f73f80a57b0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.598+0000 7f7413dc4640 1 --2- 172.21.15.120:0/1699565869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c102600 0x7f740c19f330 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.598+0000 7f7413dc4640 1 --2- 172.21.15.120:0/1699565869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f740c101c40 0x7f740c198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.598+0000 7f7413dc4640 1 --2- 172.21.15.120:0/1699565869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f740c100a50 0x7f740c197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:18.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.598+0000 7f7413dc4640 1 -- 172.21.15.120:0/1699565869 >> 172.21.15.120:0/1699565869 conn(0x7f740c0fc1b0 msgr2=0x7f740c0fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:18.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.598+0000 7f7413dc4640 1 -- 172.21.15.120:0/1699565869 shutdown_connections 2024-09-19T09:12:18.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:18.598+0000 7f7413dc4640 1 -- 172.21.15.120:0/1699565869 wait complete. 2024-09-19T09:12:18.742 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:12:19.065 INFO:tasks.workunit.client.0.smithi120.stdout: 1 912595 9.92 MB/sec execute 374 sec latency 123.477 ms 2024-09-19T09:12:19.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.286+0000 7f2415daf640 1 -- 172.21.15.120:0/3464545417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2410100e20 msgr2=0x7f2410101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:19.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.286+0000 7f2415daf640 1 --2- 172.21.15.120:0/3464545417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2410100e20 0x7f2410101200 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f2400009ca0 tx=0x7f240002a950 comp rx=0 tx=0).stop 2024-09-19T09:12:19.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.286+0000 7f2415daf640 1 -- 172.21.15.120:0/3464545417 shutdown_connections 2024-09-19T09:12:19.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.286+0000 7f2415daf640 1 --2- 172.21.15.120:0/3464545417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2410102ef0 0x7f241010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.286+0000 7f2415daf640 1 --2- 172.21.15.120:0/3464545417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2410102570 0x7f24101029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.286+0000 7f2415daf640 1 --2- 172.21.15.120:0/3464545417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2410100e20 0x7f2410101200 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.286+0000 7f2415daf640 1 -- 172.21.15.120:0/3464545417 >> 172.21.15.120:0/3464545417 conn(0x7f24100fc7d0 msgr2=0x7f24100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:19.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.287+0000 7f2415daf640 1 -- 172.21.15.120:0/3464545417 shutdown_connections 2024-09-19T09:12:19.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.288+0000 7f2415daf640 1 -- 172.21.15.120:0/3464545417 wait complete. 2024-09-19T09:12:19.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.288+0000 7f2415daf640 1 Processor -- start 2024-09-19T09:12:19.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.289+0000 7f2415daf640 1 -- start start 2024-09-19T09:12:19.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.289+0000 7f2415daf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2410100e20 0x7f2410198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:19.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.289+0000 7f2415daf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2410102570 0x7f2410198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:19.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.289+0000 7f2415daf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2410102ef0 0x7f241019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:19.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.289+0000 7f2415daf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2410110f80 con 0x7f2410102570 2024-09-19T09:12:19.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.289+0000 7f240f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2410100e20 0x7f2410198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:19.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.289+0000 7f2415daf640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2410110e00 con 0x7f2410100e20 2024-09-19T09:12:19.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.289+0000 7f240effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2410102570 0x7f2410198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:19.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.289+0000 7f240f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2410100e20 0x7f2410198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52122/0 (socket says 172.21.15.120:52122) 2024-09-19T09:12:19.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.289+0000 7f2415daf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2410111100 con 0x7f2410102ef0 2024-09-19T09:12:19.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.289+0000 7f240f7fe640 1 -- 172.21.15.120:0/1680134282 learned_addr learned my addr 172.21.15.120:0/1680134282 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:19.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.289+0000 7f240ffff640 1 --2- 172.21.15.120:0/1680134282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2410102ef0 0x7f241019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:19.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.289+0000 7f240f7fe640 1 -- 172.21.15.120:0/1680134282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2410102ef0 msgr2=0x7f241019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:19.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.289+0000 7f240f7fe640 1 --2- 172.21.15.120:0/1680134282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2410102ef0 0x7f241019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.289+0000 7f240f7fe640 1 -- 172.21.15.120:0/1680134282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2410102570 msgr2=0x7f2410198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:19.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.289+0000 7f240f7fe640 1 --2- 172.21.15.120:0/1680134282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2410102570 0x7f2410198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.289+0000 7f240f7fe640 1 -- 172.21.15.120:0/1680134282 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241019d4b0 con 0x7f2410100e20 2024-09-19T09:12:19.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.290+0000 7f240effd640 1 --2- 172.21.15.120:0/1680134282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2410102570 0x7f2410198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:19.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.290+0000 7f240f7fe640 1 --2- 172.21.15.120:0/1680134282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2410100e20 0x7f2410198500 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2400005cc0 tx=0x7f240002f420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:19.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.290+0000 7f240cff9640 1 -- 172.21.15.120:0/1680134282 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2400034040 con 0x7f2410100e20 2024-09-19T09:12:19.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.290+0000 7f2415daf640 1 -- 172.21.15.120:0/1680134282 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f241019d740 con 0x7f2410100e20 2024-09-19T09:12:19.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.290+0000 7f240cff9640 1 -- 172.21.15.120:0/1680134282 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2400030920 con 0x7f2410100e20 2024-09-19T09:12:19.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.290+0000 7f240cff9640 1 -- 172.21.15.120:0/1680134282 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24000046e0 con 0x7f2410100e20 2024-09-19T09:12:19.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.290+0000 7f2415daf640 1 -- 172.21.15.120:0/1680134282 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24101a5080 con 0x7f2410100e20 2024-09-19T09:12:19.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.292+0000 7f240cff9640 1 -- 172.21.15.120:0/1680134282 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f2400037020 con 0x7f2410100e20 2024-09-19T09:12:19.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.293+0000 7f2418188740 1 Processor -- start 2024-09-19T09:12:19.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.293+0000 7f2418188740 1 -- start start 2024-09-19T09:12:19.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.293+0000 7f2418188740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbbc9e6b40 0x55cbbc9e5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:19.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.293+0000 7f2418188740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbbc9e55d0 0x55cbbc9e5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:19.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.293+0000 7f2418188740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbbc9f0370 0x55cbbc9f2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:19.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.293+0000 7f2418188740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cbbc94b2f0 con 0x55cbbc9f0370 2024-09-19T09:12:19.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.293+0000 7f2418188740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cbbc8f2cf0 con 0x55cbbc9e55d0 2024-09-19T09:12:19.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.293+0000 7f2418188740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cbbc9472b0 con 0x55cbbc9e6b40 2024-09-19T09:12:19.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.293+0000 7f240effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbbc9f0370 0x55cbbc9f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:19.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.293+0000 7f240effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbbc9f0370 0x55cbbc9f2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56274/0 (socket says 172.21.15.120:56274) 2024-09-19T09:12:19.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.293+0000 7f240ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbbc9e6b40 0x55cbbc9e5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:19.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.293+0000 7f240effd640 1 -- 172.21.15.120:0/2915641593 learned_addr learned my addr 172.21.15.120:0/2915641593 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:19.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.293+0000 7f240f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbbc9e55d0 0x55cbbc9e5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:19.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.294+0000 7f240cff9640 1 --2- 172.21.15.120:0/1680134282 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f23f80a3240 0x7f23f80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:19.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.294+0000 7f240effd640 1 -- 172.21.15.120:0/2915641593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbbc9e6b40 msgr2=0x55cbbc9e5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:19.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.294+0000 7f240effd640 1 --2- 172.21.15.120:0/2915641593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbbc9e6b40 0x55cbbc9e5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.294+0000 7f240effd640 1 -- 172.21.15.120:0/2915641593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbbc9e55d0 msgr2=0x55cbbc9e5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:19.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.294+0000 7f240effd640 1 --2- 172.21.15.120:0/2915641593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbbc9e55d0 0x55cbbc9e5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.294+0000 7f240effd640 1 -- 172.21.15.120:0/2915641593 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbbc8d73a0 con 0x55cbbc9f0370 2024-09-19T09:12:19.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.294+0000 7f240ffff640 1 --2- 172.21.15.120:0/1680134282 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f23f80a3240 0x7f23f80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:19.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.294+0000 7f240cff9640 1 -- 172.21.15.120:0/1680134282 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f24000f0810 con 0x7f2410100e20 2024-09-19T09:12:19.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.294+0000 7f240effd640 1 --2- 172.21.15.120:0/2915641593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbbc9f0370 0x55cbbc9f2760 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f24040114f0 tx=0x7f24040119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41db1ec42a439cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:19.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.294+0000 7f23f5ffb640 1 -- 172.21.15.120:0/2915641593 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240401e070 con 0x55cbbc9f0370 2024-09-19T09:12:19.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.294+0000 7f23f5ffb640 1 -- 172.21.15.120:0/2915641593 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f240400faa0 con 0x55cbbc9f0370 2024-09-19T09:12:19.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.294+0000 7f23f5ffb640 1 -- 172.21.15.120:0/2915641593 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2404019c40 con 0x55cbbc9f0370 2024-09-19T09:12:19.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.295+0000 7f2418188740 1 -- 172.21.15.120:0/2915641593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbbc9f0370 msgr2=0x55cbbc9f2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:19.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.295+0000 7f240ffff640 1 --2- 172.21.15.120:0/1680134282 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f23f80a3240 0x7f23f80a5700 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2408009d10 tx=0x7f24080046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:19.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.295+0000 7f2418188740 1 --2- 172.21.15.120:0/2915641593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbbc9f0370 0x55cbbc9f2760 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f24040114f0 tx=0x7f24040119b0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.296+0000 7f2418188740 1 -- 172.21.15.120:0/2915641593 shutdown_connections 2024-09-19T09:12:19.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.296+0000 7f2418188740 1 --2- 172.21.15.120:0/2915641593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbbc9f0370 0x55cbbc9f2760 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.296+0000 7f2418188740 1 --2- 172.21.15.120:0/2915641593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbbc9e55d0 0x55cbbc9e5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.296+0000 7f2418188740 1 --2- 172.21.15.120:0/2915641593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbbc9e6b40 0x55cbbc9e5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.296+0000 7f2418188740 1 -- 172.21.15.120:0/2915641593 >> 172.21.15.120:0/2915641593 conn(0x55cbbc9ae310 msgr2=0x55cbbc9e1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:19.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.296+0000 7f2418188740 1 -- 172.21.15.120:0/2915641593 shutdown_connections 2024-09-19T09:12:19.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.296+0000 7f2418188740 1 -- 172.21.15.120:0/2915641593 wait complete. 2024-09-19T09:12:19.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.296+0000 7f2418188740 1 -- start start 2024-09-19T09:12:19.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.297+0000 7f2418188740 1 Processor -- start 2024-09-19T09:12:19.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.297+0000 7f2418188740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbbc9e55d0 0x55cbbc9efd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:19.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.297+0000 7f2418188740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbbc9e6b40 0x55cbbc9ecfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:19.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.297+0000 7f2418188740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbbc9f0370 0x55cbbc9ed5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:19.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.297+0000 7f2418188740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cbbc9f3070 con 0x55cbbc9e6b40 2024-09-19T09:12:19.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.297+0000 7f2418188740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cbbc906060 con 0x55cbbc9e55d0 2024-09-19T09:12:19.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.297+0000 7f2418188740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cbbc9f31f0 con 0x55cbbc9f0370 2024-09-19T09:12:19.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.297+0000 7f2418188740 10 client.0 _ll_register_callbacks cb 0x55cbbc6598c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:19.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.297+0000 7f240effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbbc9e6b40 0x55cbbc9ecfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:19.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.297+0000 7f240f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbbc9e55d0 0x55cbbc9efd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:19.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.297+0000 7f240effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbbc9e6b40 0x55cbbc9ecfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56288/0 (socket says 172.21.15.120:56288) 2024-09-19T09:12:19.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.297+0000 7f240effd640 1 -- 172.21.15.120:0/561746605 learned_addr learned my addr 172.21.15.120:0/561746605 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:19.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.297+0000 7f240ffff640 1 --2- 172.21.15.120:0/561746605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbbc9f0370 0x55cbbc9ed5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:19.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.297+0000 7f240effd640 1 -- 172.21.15.120:0/561746605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbbc9f0370 msgr2=0x55cbbc9ed5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:19.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.297+0000 7f240effd640 1 --2- 172.21.15.120:0/561746605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbbc9f0370 0x55cbbc9ed5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.297+0000 7f240effd640 1 -- 172.21.15.120:0/561746605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbbc9e55d0 msgr2=0x55cbbc9efd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:19.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.297+0000 7f240effd640 1 --2- 172.21.15.120:0/561746605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbbc9e55d0 0x55cbbc9efd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.297+0000 7f240effd640 1 -- 172.21.15.120:0/561746605 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbbc6bc3c0 con 0x55cbbc9e6b40 2024-09-19T09:12:19.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.297+0000 7f240f7fe640 1 --2- 172.21.15.120:0/561746605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbbc9e55d0 0x55cbbc9efd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:19.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.298+0000 7f240effd640 1 --2- 172.21.15.120:0/561746605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbbc9e6b40 0x55cbbc9ecfe0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f24040198a0 tx=0x7f2404019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ccf51026a6ac98 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:19.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.298+0000 7f23eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:19.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.298+0000 7f23eeffd640 1 -- 172.21.15.120:0/561746605 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2404029060 con 0x55cbbc9e6b40 2024-09-19T09:12:19.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.298+0000 7f2418188740 1 -- 172.21.15.120:0/561746605 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f240400faa0 con 0x55cbbc9e6b40 2024-09-19T09:12:19.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.298+0000 7f23eeffd640 1 -- 172.21.15.120:0/561746605 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f240400f640 con 0x55cbbc9e6b40 2024-09-19T09:12:19.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.298+0000 7f23eeffd640 1 -- 172.21.15.120:0/561746605 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f240401e040 con 0x55cbbc9e6b40 2024-09-19T09:12:19.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.298+0000 7f2418188740 10 client.18795 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:19.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.298+0000 7f2418188740 1 -- 172.21.15.120:0/561746605 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cbbc9feb00 con 0x55cbbc9e6b40 2024-09-19T09:12:19.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.298+0000 7f23eeffd640 1 -- 172.21.15.120:0/561746605 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f240400faa0 con 0x55cbbc9e6b40 2024-09-19T09:12:19.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.298+0000 7f23eeffd640 1 -- 172.21.15.120:0/561746605 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f240402e070 con 0x55cbbc9e6b40 2024-09-19T09:12:19.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.298+0000 7f2418188740 10 client.18795 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:19.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.298+0000 7f2418188740 10 client.18795 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:12:19.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.298+0000 7f2418188740 20 client.18795 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:19.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.298+0000 7f2418188740 1 --2- 172.21.15.120:0/561746605 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55cbbc9ffe50 0x55cbbca02240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:19.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.298+0000 7f2418188740 4 client.18795 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:12:19.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.298+0000 7f2418188740 1 -- 172.21.15.120:0/561746605 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbbc958860 con 0x55cbbc9ffe50 2024-09-19T09:12:19.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.299+0000 7f240f7fe640 1 --2- 172.21.15.120:0/561746605 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55cbbc9ffe50 0x55cbbca02240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:19.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.299+0000 7f23eeffd640 10 client.18795 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:12:19.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.299+0000 7f240f7fe640 1 --2- 172.21.15.120:0/561746605 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55cbbc9ffe50 0x55cbbca02240 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5f2c6090b83f8523 server_cookie=e8ac4ad36c1b9c70 in_seq=0 out_seq=0 2024-09-19T09:12:19.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.300+0000 7f23eeffd640 1 -- 172.21.15.120:0/561746605 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cbbc958860 con 0x55cbbc9ffe50 2024-09-19T09:12:19.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.300+0000 7f23eeffd640 10 client.18795 handle_command_reply: tid=0 2024-09-19T09:12:19.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.300+0000 7f2418188740 1 client.18795 shutdown 2024-09-19T09:12:19.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.300+0000 7f2418188740 1 -- 172.21.15.120:0/561746605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbbc9e6b40 msgr2=0x55cbbc9ecfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:19.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.300+0000 7f2418188740 1 --2- 172.21.15.120:0/561746605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbbc9e6b40 0x55cbbc9ecfe0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f24040198a0 tx=0x7f2404019640 comp rx=0 tx=0).stop 2024-09-19T09:12:19.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.300+0000 7f2418188740 1 -- 172.21.15.120:0/561746605 shutdown_connections 2024-09-19T09:12:19.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.300+0000 7f2418188740 1 --2- 172.21.15.120:0/561746605 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55cbbc9ffe50 0x55cbbca02240 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.300+0000 7f2418188740 1 --2- 172.21.15.120:0/561746605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbbc9f0370 0x55cbbc9ed5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.300+0000 7f2418188740 1 --2- 172.21.15.120:0/561746605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbbc9e6b40 0x55cbbc9ecfe0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.300+0000 7f23eeffd640 0 client.18795 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:12:19.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.300+0000 7f2418188740 1 --2- 172.21.15.120:0/561746605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbbc9e55d0 0x55cbbc9efd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.300+0000 7f2418188740 1 -- 172.21.15.120:0/561746605 >> 172.21.15.120:0/561746605 conn(0x55cbbc9ae310 msgr2=0x55cbbc9e7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:19.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.300+0000 7f2418188740 1 -- 172.21.15.120:0/561746605 shutdown_connections 2024-09-19T09:12:19.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.300+0000 7f2418188740 1 -- 172.21.15.120:0/561746605 wait complete. 2024-09-19T09:12:19.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.300+0000 7f2418188740 20 client.18795 trim_cache size 0 max 16384 2024-09-19T09:12:19.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.315+0000 7f2418188740 1 Processor -- start 2024-09-19T09:12:19.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.315+0000 7f2418188740 1 -- start start 2024-09-19T09:12:19.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.315+0000 7f2418188740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbbc9e55d0 0x55cbbc9fc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:19.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.315+0000 7f2418188740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbbc9f0370 0x55cbbc9e0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:19.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.315+0000 7f2418188740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbbc9ebd10 0x55cbbca079d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:19.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.315+0000 7f2418188740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f24101af7b0 con 0x55cbbc9e55d0 2024-09-19T09:12:19.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.315+0000 7f2418188740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f24101b12f0 con 0x55cbbc9f0370 2024-09-19T09:12:19.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.315+0000 7f2418188740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f24101afe80 con 0x55cbbc9ebd10 2024-09-19T09:12:19.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.316+0000 7f240f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbbc9e55d0 0x55cbbc9fc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:19.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.316+0000 7f240effd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbbc9f0370 0x55cbbc9e0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:19.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.316+0000 7f240ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbbc9ebd10 0x55cbbca079d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:19.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.316+0000 7f240effd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbbc9f0370 0x55cbbc9e0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52150/0 (socket says 172.21.15.120:52150) 2024-09-19T09:12:19.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.316+0000 7f240effd640 1 -- 172.21.15.120:0/2894649698 learned_addr learned my addr 172.21.15.120:0/2894649698 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:19.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.316+0000 7f240f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbbc9e55d0 0x55cbbc9fc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56292/0 (socket says 172.21.15.120:56292) 2024-09-19T09:12:19.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.316+0000 7f240effd640 1 -- 172.21.15.120:0/2894649698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbbc9ebd10 msgr2=0x55cbbca079d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:19.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.316+0000 7f240effd640 1 --2- 172.21.15.120:0/2894649698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbbc9ebd10 0x55cbbca079d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.316+0000 7f240effd640 1 -- 172.21.15.120:0/2894649698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbbc9e55d0 msgr2=0x55cbbc9fc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:19.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.316+0000 7f240effd640 1 --2- 172.21.15.120:0/2894649698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbbc9e55d0 0x55cbbc9fc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.316+0000 7f240effd640 1 -- 172.21.15.120:0/2894649698 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbbc9ed940 con 0x55cbbc9f0370 2024-09-19T09:12:19.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.316+0000 7f240effd640 1 --2- 172.21.15.120:0/2894649698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbbc9f0370 0x55cbbc9e0bf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2404019f10 tx=0x7f2404005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d29c943e81223e17 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:19.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.316+0000 7f23f57fa640 1 -- 172.21.15.120:0/2894649698 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2404034020 con 0x55cbbc9f0370 2024-09-19T09:12:19.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.316+0000 7f23f57fa640 1 -- 172.21.15.120:0/2894649698 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2404005160 con 0x55cbbc9f0370 2024-09-19T09:12:19.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.316+0000 7f23f57fa640 1 -- 172.21.15.120:0/2894649698 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2404029070 con 0x55cbbc9f0370 2024-09-19T09:12:19.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.317+0000 7f2418188740 1 -- 172.21.15.120:0/2894649698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbbc9f0370 msgr2=0x55cbbc9e0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:19.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.317+0000 7f2418188740 1 --2- 172.21.15.120:0/2894649698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbbc9f0370 0x55cbbc9e0bf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2404019f10 tx=0x7f2404005f90 comp rx=0 tx=0).stop 2024-09-19T09:12:19.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.317+0000 7f2418188740 1 -- 172.21.15.120:0/2894649698 shutdown_connections 2024-09-19T09:12:19.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.317+0000 7f2418188740 1 --2- 172.21.15.120:0/2894649698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbbc9ebd10 0x55cbbca079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.317+0000 7f2418188740 1 --2- 172.21.15.120:0/2894649698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbbc9f0370 0x55cbbc9e0bf0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.317+0000 7f2418188740 1 --2- 172.21.15.120:0/2894649698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbbc9e55d0 0x55cbbc9fc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.317+0000 7f2418188740 1 -- 172.21.15.120:0/2894649698 >> 172.21.15.120:0/2894649698 conn(0x55cbbc9ae310 msgr2=0x55cbbc9e04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:19.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.317+0000 7f2418188740 1 -- 172.21.15.120:0/2894649698 shutdown_connections 2024-09-19T09:12:19.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.317+0000 7f2418188740 1 -- 172.21.15.120:0/2894649698 wait complete. 2024-09-19T09:12:19.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.317+0000 7f2418188740 1 -- start start 2024-09-19T09:12:19.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.318+0000 7f2418188740 1 Processor -- start 2024-09-19T09:12:19.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.318+0000 7f2418188740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbbc9e55d0 0x55cbbca63c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:19.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.318+0000 7f2418188740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbbc9ebd10 0x55cbbca64170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:19.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.318+0000 7f2418188740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbbc9f0370 0x55cbbc9f4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:19.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.318+0000 7f2418188740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f24101b0c20 con 0x55cbbc9ebd10 2024-09-19T09:12:19.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.318+0000 7f2418188740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f24101b0550 con 0x55cbbc9f0370 2024-09-19T09:12:19.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.318+0000 7f2418188740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f24101ad770 con 0x55cbbc9e55d0 2024-09-19T09:12:19.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.318+0000 7f2418188740 10 client.0 _ll_register_callbacks cb 0x7f240401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:19.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.318+0000 7f240effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbbc9ebd10 0x55cbbca64170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:19.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.318+0000 7f240effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbbc9ebd10 0x55cbbca64170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56306/0 (socket says 172.21.15.120:56306) 2024-09-19T09:12:19.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.318+0000 7f240effd640 1 -- 172.21.15.120:0/1184890348 learned_addr learned my addr 172.21.15.120:0/1184890348 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:19.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.318+0000 7f240ffff640 1 --2- 172.21.15.120:0/1184890348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbbc9f0370 0x55cbbc9f4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:19.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.318+0000 7f240f7fe640 1 --2- 172.21.15.120:0/1184890348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbbc9e55d0 0x55cbbca63c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:19.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.318+0000 7f240effd640 1 -- 172.21.15.120:0/1184890348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbbc9e55d0 msgr2=0x55cbbca63c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:19.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.318+0000 7f240effd640 1 --2- 172.21.15.120:0/1184890348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbbc9e55d0 0x55cbbca63c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.318+0000 7f240effd640 1 -- 172.21.15.120:0/1184890348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbbc9f0370 msgr2=0x55cbbc9f4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:19.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.318+0000 7f240effd640 1 --2- 172.21.15.120:0/1184890348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbbc9f0370 0x55cbbc9f4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.318+0000 7f240effd640 1 -- 172.21.15.120:0/1184890348 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbbc9f5170 con 0x55cbbc9ebd10 2024-09-19T09:12:19.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.319+0000 7f240effd640 1 --2- 172.21.15.120:0/1184890348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbbc9ebd10 0x55cbbca64170 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f2404019fb0 tx=0x7f2404019b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa47d7787c09ae30 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:19.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.319+0000 7f23eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:19.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.319+0000 7f23eeffd640 1 -- 172.21.15.120:0/1184890348 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2404029030 con 0x55cbbc9ebd10 2024-09-19T09:12:19.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.319+0000 7f23eeffd640 1 -- 172.21.15.120:0/1184890348 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f240400f7d0 con 0x55cbbc9ebd10 2024-09-19T09:12:19.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.319+0000 7f2418188740 1 -- 172.21.15.120:0/1184890348 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2404005160 con 0x55cbbc9ebd10 2024-09-19T09:12:19.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.319+0000 7f23eeffd640 1 -- 172.21.15.120:0/1184890348 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f240400f9d0 con 0x55cbbc9ebd10 2024-09-19T09:12:19.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.319+0000 7f2418188740 10 client.18801 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:19.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.319+0000 7f2418188740 1 -- 172.21.15.120:0/1184890348 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cbbca6b490 con 0x55cbbc9ebd10 2024-09-19T09:12:19.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.319+0000 7f23eeffd640 1 -- 172.21.15.120:0/1184890348 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2404005160 con 0x55cbbc9ebd10 2024-09-19T09:12:19.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.319+0000 7f23eeffd640 1 -- 172.21.15.120:0/1184890348 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f240402e030 con 0x55cbbc9ebd10 2024-09-19T09:12:19.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.319+0000 7f2418188740 10 client.18801 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:19.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.319+0000 7f2418188740 10 client.18801 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:12:19.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.319+0000 7f2418188740 20 client.18801 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:19.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.319+0000 7f2418188740 1 --2- 172.21.15.120:0/1184890348 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55cbbca6bb90 0x55cbbca6dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:19.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.319+0000 7f2418188740 4 client.18801 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:12:19.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.319+0000 7f2418188740 1 -- 172.21.15.120:0/1184890348 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cbbc952d70 con 0x55cbbca6bb90 2024-09-19T09:12:19.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.320+0000 7f240f7fe640 1 --2- 172.21.15.120:0/1184890348 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55cbbca6bb90 0x55cbbca6dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:19.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.320+0000 7f240f7fe640 1 --2- 172.21.15.120:0/1184890348 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55cbbca6bb90 0x55cbbca6dfb0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1c87e5bec9d98521 server_cookie=4d435697cac7e424 in_seq=0 out_seq=0 2024-09-19T09:12:19.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.320+0000 7f23eeffd640 10 client.18801 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:12:19.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.320+0000 7f23eeffd640 1 -- 172.21.15.120:0/1184890348 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cbbc952d70 con 0x55cbbca6bb90 2024-09-19T09:12:19.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.320+0000 7f23eeffd640 10 client.18801 handle_command_reply: tid=0 2024-09-19T09:12:19.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.320+0000 7f2418188740 1 client.18801 shutdown 2024-09-19T09:12:19.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.320+0000 7f2418188740 1 -- 172.21.15.120:0/1184890348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbbc9ebd10 msgr2=0x55cbbca64170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:19.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.320+0000 7f2418188740 1 --2- 172.21.15.120:0/1184890348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbbc9ebd10 0x55cbbca64170 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f2404019fb0 tx=0x7f2404019b00 comp rx=0 tx=0).stop 2024-09-19T09:12:19.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.320+0000 7f2418188740 1 -- 172.21.15.120:0/1184890348 shutdown_connections 2024-09-19T09:12:19.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.320+0000 7f2418188740 1 --2- 172.21.15.120:0/1184890348 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55cbbca6bb90 0x55cbbca6dfb0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.320+0000 7f2418188740 1 --2- 172.21.15.120:0/1184890348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbbc9f0370 0x55cbbc9f4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.320+0000 7f2418188740 1 --2- 172.21.15.120:0/1184890348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cbbc9ebd10 0x55cbbca64170 secure :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f2404019fb0 tx=0x7f2404019b00 comp rx=0 tx=0).stop 2024-09-19T09:12:19.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.320+0000 7f2418188740 1 --2- 172.21.15.120:0/1184890348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbbc9e55d0 0x55cbbca63c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.320+0000 7f2418188740 1 -- 172.21.15.120:0/1184890348 >> 172.21.15.120:0/1184890348 conn(0x55cbbc9ae310 msgr2=0x55cbbc9e0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:19.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.320+0000 7f23eeffd640 0 client.18801 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:12:19.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.321+0000 7f2418188740 1 -- 172.21.15.120:0/1184890348 shutdown_connections 2024-09-19T09:12:19.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.321+0000 7f2418188740 1 -- 172.21.15.120:0/1184890348 wait complete. 2024-09-19T09:12:19.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.321+0000 7f2418188740 20 client.18801 trim_cache size 0 max 16384 2024-09-19T09:12:19.330 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:12:19.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.322+0000 7f2415daf640 1 -- 172.21.15.120:0/1680134282 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f23f80a3240 msgr2=0x7f23f80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:19.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.322+0000 7f2415daf640 1 --2- 172.21.15.120:0/1680134282 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f23f80a3240 0x7f23f80a5700 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2408009d10 tx=0x7f24080046c0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.322+0000 7f2415daf640 1 -- 172.21.15.120:0/1680134282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2410100e20 msgr2=0x7f2410198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:19.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.322+0000 7f2415daf640 1 --2- 172.21.15.120:0/1680134282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2410100e20 0x7f2410198500 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2400005cc0 tx=0x7f240002f420 comp rx=0 tx=0).stop 2024-09-19T09:12:19.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.322+0000 7f2415daf640 1 -- 172.21.15.120:0/1680134282 shutdown_connections 2024-09-19T09:12:19.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.322+0000 7f2415daf640 1 --2- 172.21.15.120:0/1680134282 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f23f80a3240 0x7f23f80a5700 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.322+0000 7f2415daf640 1 --2- 172.21.15.120:0/1680134282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2410102ef0 0x7f241019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.322+0000 7f2415daf640 1 --2- 172.21.15.120:0/1680134282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2410102570 0x7f2410198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.322+0000 7f2415daf640 1 --2- 172.21.15.120:0/1680134282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2410100e20 0x7f2410198500 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:19.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.322+0000 7f2415daf640 1 -- 172.21.15.120:0/1680134282 >> 172.21.15.120:0/1680134282 conn(0x7f24100fc7d0 msgr2=0x7f241006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:19.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.322+0000 7f2415daf640 1 -- 172.21.15.120:0/1680134282 shutdown_connections 2024-09-19T09:12:19.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:19.322+0000 7f2415daf640 1 -- 172.21.15.120:0/1680134282 wait complete. 2024-09-19T09:12:20.066 INFO:tasks.workunit.client.0.smithi120.stdout: 1 914978 9.92 MB/sec execute 375 sec latency 61.185 ms 2024-09-19T09:12:20.071 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:12:20.343 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:12:21.066 INFO:tasks.workunit.client.0.smithi120.stdout: 1 916927 9.95 MB/sec execute 376 sec latency 38.600 ms 2024-09-19T09:12:21.136 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:20 smithi120 ceph-mon[31150]: pgmap v627: 97 pgs: 97 active+clean; 50 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.5 MiB/s wr, 445 op/s 2024-09-19T09:12:21.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:20 smithi196 ceph-mon[35150]: pgmap v627: 97 pgs: 97 active+clean; 50 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.5 MiB/s wr, 445 op/s 2024-09-19T09:12:21.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:20 smithi153 ceph-mon[35071]: pgmap v627: 97 pgs: 97 active+clean; 50 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.5 MiB/s wr, 445 op/s 2024-09-19T09:12:21.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.225+0000 7fa0c0f4a640 1 -- 172.21.15.120:0/424579175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0bc101df0 msgr2=0x7fa0bc1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:21.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.225+0000 7fa0c0f4a640 1 --2- 172.21.15.120:0/424579175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0bc101df0 0x7fa0bc1021d0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac009900 tx=0x7fa0ac02a410 comp rx=0 tx=0).stop 2024-09-19T09:12:21.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.225+0000 7fa0c0f4a640 1 -- 172.21.15.120:0/424579175 shutdown_connections 2024-09-19T09:12:21.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.225+0000 7fa0c0f4a640 1 --2- 172.21.15.120:0/424579175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0bc106f20 0x7fa0bc10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.225+0000 7fa0c0f4a640 1 --2- 172.21.15.120:0/424579175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0bc1027a0 0x7fa0bc1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.225+0000 7fa0c0f4a640 1 --2- 172.21.15.120:0/424579175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0bc101df0 0x7fa0bc1021d0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.225+0000 7fa0c0f4a640 1 -- 172.21.15.120:0/424579175 >> 172.21.15.120:0/424579175 conn(0x7fa0bc0fd8f0 msgr2=0x7fa0bc0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:21.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.225+0000 7fa0c0f4a640 1 -- 172.21.15.120:0/424579175 shutdown_connections 2024-09-19T09:12:21.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.225+0000 7fa0c0f4a640 1 -- 172.21.15.120:0/424579175 wait complete. 2024-09-19T09:12:21.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.226+0000 7fa0c0f4a640 1 Processor -- start 2024-09-19T09:12:21.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.227+0000 7fa0c0f4a640 1 -- start start 2024-09-19T09:12:21.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.227+0000 7fa0c0f4a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0bc101df0 0x7fa0bc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:21.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.227+0000 7fa0c0f4a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0bc1027a0 0x7fa0bc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:21.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.227+0000 7fa0c0f4a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0bc106f20 0x7fa0bc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:21.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.227+0000 7fa0c0f4a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc111f80 con 0x7fa0bc106f20 2024-09-19T09:12:21.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.227+0000 7fa0c0f4a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc111e00 con 0x7fa0bc1027a0 2024-09-19T09:12:21.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.227+0000 7fa0c0f4a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc112100 con 0x7fa0bc101df0 2024-09-19T09:12:21.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.227+0000 7fa0bad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0bc106f20 0x7fa0bc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:21.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.227+0000 7fa0bad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0bc106f20 0x7fa0bc074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47972/0 (socket says 172.21.15.120:47972) 2024-09-19T09:12:21.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.227+0000 7fa0bad9d640 1 -- 172.21.15.120:0/3318183904 learned_addr learned my addr 172.21.15.120:0/3318183904 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:21.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.227+0000 7fa0b9d9b640 1 --2- 172.21.15.120:0/3318183904 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0bc1027a0 0x7fa0bc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:21.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.228+0000 7fa0bad9d640 1 -- 172.21.15.120:0/3318183904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0bc101df0 msgr2=0x7fa0bc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:21.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.228+0000 7fa0bad9d640 1 --2- 172.21.15.120:0/3318183904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0bc101df0 0x7fa0bc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.228+0000 7fa0bad9d640 1 -- 172.21.15.120:0/3318183904 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0bc1027a0 msgr2=0x7fa0bc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:21.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.228+0000 7fa0bad9d640 1 --2- 172.21.15.120:0/3318183904 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0bc1027a0 0x7fa0bc074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.228+0000 7fa0bad9d640 1 -- 172.21.15.120:0/3318183904 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0bc1aad70 con 0x7fa0bc106f20 2024-09-19T09:12:21.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.228+0000 7fa0bad9d640 1 --2- 172.21.15.120:0/3318183904 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0bc106f20 0x7fa0bc074f70 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fa0b400eb10 tx=0x7fa0b400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:21.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.230+0000 7fa0b37fe640 1 -- 172.21.15.120:0/3318183904 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4004b20 con 0x7fa0bc106f20 2024-09-19T09:12:21.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.230+0000 7fa0b37fe640 1 -- 172.21.15.120:0/3318183904 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0b4004cc0 con 0x7fa0bc106f20 2024-09-19T09:12:21.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.230+0000 7fa0b37fe640 1 -- 172.21.15.120:0/3318183904 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4010760 con 0x7fa0bc106f20 2024-09-19T09:12:21.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.230+0000 7fa0c0f4a640 1 -- 172.21.15.120:0/3318183904 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0bc1aafa0 con 0x7fa0bc106f20 2024-09-19T09:12:21.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.230+0000 7fa0c0f4a640 1 -- 172.21.15.120:0/3318183904 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0bc1ab430 con 0x7fa0bc106f20 2024-09-19T09:12:21.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.235+0000 7fa0c3323740 1 Processor -- start 2024-09-19T09:12:21.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.236+0000 7fa0b37fe640 1 -- 172.21.15.120:0/3318183904 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa0b4005000 con 0x7fa0bc106f20 2024-09-19T09:12:21.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.237+0000 7fa0c3323740 1 -- start start 2024-09-19T09:12:21.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.237+0000 7fa0c3323740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56038971eb40 0x56038971d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:21.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.237+0000 7fa0c3323740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56038971d5d0 0x56038971da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:21.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.237+0000 7fa0c3323740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560389728370 0x56038972a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:21.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.237+0000 7fa0c3323740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5603896832f0 con 0x56038971eb40 2024-09-19T09:12:21.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.237+0000 7fa0c3323740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56038962acf0 con 0x56038971d5d0 2024-09-19T09:12:21.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.237+0000 7fa0c3323740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56038967f2b0 con 0x560389728370 2024-09-19T09:12:21.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.238+0000 7fa0b37fe640 1 --2- 172.21.15.120:0/3318183904 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa0a80a3340 0x7fa0a80a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:21.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.239+0000 7fa0b9d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560389728370 0x56038972a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:21.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.239+0000 7fa0b37fe640 1 -- 172.21.15.120:0/3318183904 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa0b40d0040 con 0x7fa0bc106f20 2024-09-19T09:12:21.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.242+0000 7fa0b9d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560389728370 0x56038972a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:47926/0 (socket says 172.21.15.120:47926) 2024-09-19T09:12:21.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.242+0000 7fa0b9d9b640 1 -- 172.21.15.120:0/791584128 learned_addr learned my addr 172.21.15.120:0/791584128 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:21.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.243+0000 7fa0b9d9b640 1 --2- 172.21.15.120:0/791584128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56038971eb40 0x56038971d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:21.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.244+0000 7fa0b9d9b640 1 -- 172.21.15.120:0/791584128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56038971d5d0 msgr2=0x56038971da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:21.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.244+0000 7fa0ba59c640 1 --2- 172.21.15.120:0/791584128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56038971d5d0 0x56038971da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:21.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.244+0000 7fa0b9d9b640 1 --2- 172.21.15.120:0/791584128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56038971d5d0 0x56038971da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.244+0000 7fa0b9d9b640 1 -- 172.21.15.120:0/791584128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56038971eb40 msgr2=0x56038971d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:21.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.244+0000 7fa0b9d9b640 1 --2- 172.21.15.120:0/791584128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56038971eb40 0x56038971d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.244+0000 7fa0b9d9b640 1 -- 172.21.15.120:0/791584128 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56038960f3a0 con 0x560389728370 2024-09-19T09:12:21.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.245+0000 7fa0b9d9b640 1 --2- 172.21.15.120:0/791584128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560389728370 0x56038972a760 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa0a40117a0 tx=0x7fa0a40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acdc1ba099c5dc65 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:21.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.247+0000 7fa0bad9d640 1 --2- 172.21.15.120:0/3318183904 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa0a80a3340 0x7fa0a80a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:21.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.248+0000 7fa0b0ff9640 1 -- 172.21.15.120:0/791584128 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a4004030 con 0x560389728370 2024-09-19T09:12:21.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.248+0000 7fa0b0ff9640 1 -- 172.21.15.120:0/791584128 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0a4005500 con 0x560389728370 2024-09-19T09:12:21.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.248+0000 7fa0b0ff9640 1 -- 172.21.15.120:0/791584128 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a4004ce0 con 0x560389728370 2024-09-19T09:12:21.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.249+0000 7fa0c3323740 1 -- 172.21.15.120:0/791584128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560389728370 msgr2=0x56038972a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:21.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.249+0000 7fa0c3323740 1 --2- 172.21.15.120:0/791584128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560389728370 0x56038972a760 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa0a40117a0 tx=0x7fa0a40046c0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.251+0000 7fa0c3323740 1 -- 172.21.15.120:0/791584128 shutdown_connections 2024-09-19T09:12:21.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.251+0000 7fa0c3323740 1 --2- 172.21.15.120:0/791584128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560389728370 0x56038972a760 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.251+0000 7fa0c3323740 1 --2- 172.21.15.120:0/791584128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56038971d5d0 0x56038971da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.251+0000 7fa0c3323740 1 --2- 172.21.15.120:0/791584128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56038971eb40 0x56038971d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.251+0000 7fa0c3323740 1 -- 172.21.15.120:0/791584128 >> 172.21.15.120:0/791584128 conn(0x5603896e6310 msgr2=0x560389719c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:21.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.254+0000 7fa0c3323740 1 -- 172.21.15.120:0/791584128 shutdown_connections 2024-09-19T09:12:21.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.254+0000 7fa0c3323740 1 -- 172.21.15.120:0/791584128 wait complete. 2024-09-19T09:12:21.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.254+0000 7fa0c3323740 1 -- start start 2024-09-19T09:12:21.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.255+0000 7fa0c3323740 1 Processor -- start 2024-09-19T09:12:21.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.257+0000 7fa0bad9d640 1 --2- 172.21.15.120:0/3318183904 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa0a80a3340 0x7fa0a80a5800 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40049d0 tx=0x7fa0b400efd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:21.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.257+0000 7fa0c3323740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56038971d5d0 0x560389727de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:21.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.257+0000 7fa0c3323740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56038971eb40 0x560389725090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:21.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.257+0000 7fa0c3323740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560389728370 0x5603897257d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:21.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.257+0000 7fa0c3323740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56038973c080 con 0x560389728370 2024-09-19T09:12:21.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.257+0000 7fa0c3323740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56038963e060 con 0x56038971d5d0 2024-09-19T09:12:21.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.258+0000 7fa0c3323740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56038973c200 con 0x56038971eb40 2024-09-19T09:12:21.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.258+0000 7fa0c3323740 10 client.0 _ll_register_callbacks cb 0x5603893918c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:21.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.258+0000 7fa0b9d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56038971d5d0 0x560389727de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:21.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.258+0000 7fa0ba59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56038971eb40 0x560389725090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:21.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.258+0000 7fa0b9d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56038971d5d0 0x560389727de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:48128/0 (socket says 172.21.15.120:48128) 2024-09-19T09:12:21.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.258+0000 7fa0b9d9b640 1 -- 172.21.15.120:0/652594110 learned_addr learned my addr 172.21.15.120:0/652594110 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:21.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.258+0000 7fa0b9d9b640 1 --2- 172.21.15.120:0/652594110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560389728370 0x5603897257d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:21.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.259+0000 7fa0b9d9b640 1 -- 172.21.15.120:0/652594110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56038971eb40 msgr2=0x560389725090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:21.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.259+0000 7fa0b9d9b640 1 --2- 172.21.15.120:0/652594110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56038971eb40 0x560389725090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.259+0000 7fa0b9d9b640 1 -- 172.21.15.120:0/652594110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560389728370 msgr2=0x5603897257d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:21.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.259+0000 7fa0b9d9b640 1 --2- 172.21.15.120:0/652594110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560389728370 0x5603897257d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.259+0000 7fa0b9d9b640 1 -- 172.21.15.120:0/652594110 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603893f43c0 con 0x56038971d5d0 2024-09-19T09:12:21.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.259+0000 7fa0b9d9b640 1 --2- 172.21.15.120:0/652594110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56038971d5d0 0x560389727de0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4004e60 tx=0x7fa0a4004980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9b726cde7cf6d3c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:21.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.260+0000 7fa0a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:12:21.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.260+0000 7fa0a1ffb640 1 -- 172.21.15.120:0/652594110 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a4028040 con 0x56038971d5d0 2024-09-19T09:12:21.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.260+0000 7fa0a1ffb640 1 -- 172.21.15.120:0/652594110 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa0a40052b0 con 0x56038971d5d0 2024-09-19T09:12:21.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.260+0000 7fa0c3323740 1 -- 172.21.15.120:0/652594110 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0a4005500 con 0x56038971d5d0 2024-09-19T09:12:21.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.260+0000 7fa0a1ffb640 1 -- 172.21.15.120:0/652594110 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa0a4035050 con 0x56038971d5d0 2024-09-19T09:12:21.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.261+0000 7fa0c3323740 10 client.27941 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:21.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.262+0000 7fa0c3323740 1 -- 172.21.15.120:0/652594110 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560389747c00 con 0x56038971d5d0 2024-09-19T09:12:21.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.262+0000 7fa0a1ffb640 1 -- 172.21.15.120:0/652594110 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0a4005500 con 0x56038971d5d0 2024-09-19T09:12:21.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.262+0000 7fa0a1ffb640 1 -- 172.21.15.120:0/652594110 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa0a401f7e0 con 0x56038971d5d0 2024-09-19T09:12:21.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.263+0000 7fa0c3323740 10 client.27941 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:21.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.263+0000 7fa0c3323740 10 client.27941 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:12:21.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.263+0000 7fa0c3323740 20 client.27941 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:21.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.263+0000 7fa0c3323740 1 --2- 172.21.15.120:0/652594110 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x560389748f50 0x56038974b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:21.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.263+0000 7fa0c3323740 4 client.27941 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:12:21.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.263+0000 7fa0c3323740 1 -- 172.21.15.120:0/652594110 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560389690860 con 0x560389748f50 2024-09-19T09:12:21.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.263+0000 7fa0ba59c640 1 --2- 172.21.15.120:0/652594110 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x560389748f50 0x56038974b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:21.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.264+0000 7fa0ba59c640 1 --2- 172.21.15.120:0/652594110 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x560389748f50 0x56038974b340 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=254e318b4bf51cbe server_cookie=c1d7184e19a047de in_seq=0 out_seq=0 2024-09-19T09:12:21.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.264+0000 7fa0a1ffb640 10 client.27941 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:12:21.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.265+0000 7fa0a1ffb640 1 -- 172.21.15.120:0/652594110 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560389690860 con 0x560389748f50 2024-09-19T09:12:21.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.265+0000 7fa0a1ffb640 10 client.27941 handle_command_reply: tid=0 2024-09-19T09:12:21.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.265+0000 7fa0c3323740 1 client.27941 shutdown 2024-09-19T09:12:21.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.265+0000 7fa0c3323740 1 -- 172.21.15.120:0/652594110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56038971d5d0 msgr2=0x560389727de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:21.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.265+0000 7fa0c3323740 1 --2- 172.21.15.120:0/652594110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56038971d5d0 0x560389727de0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4004e60 tx=0x7fa0a4004980 comp rx=0 tx=0).stop 2024-09-19T09:12:21.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.265+0000 7fa0c3323740 1 -- 172.21.15.120:0/652594110 shutdown_connections 2024-09-19T09:12:21.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.265+0000 7fa0c3323740 1 --2- 172.21.15.120:0/652594110 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x560389748f50 0x56038974b340 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.265+0000 7fa0c3323740 1 --2- 172.21.15.120:0/652594110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560389728370 0x5603897257d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.265+0000 7fa0c3323740 1 --2- 172.21.15.120:0/652594110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56038971eb40 0x560389725090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.265+0000 7fa0c3323740 1 --2- 172.21.15.120:0/652594110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56038971d5d0 0x560389727de0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.265+0000 7fa0c3323740 1 -- 172.21.15.120:0/652594110 >> 172.21.15.120:0/652594110 conn(0x5603896e6310 msgr2=0x56038971f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:21.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.266+0000 7fa0a1ffb640 0 client.27941 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:12:21.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.266+0000 7fa0c3323740 1 -- 172.21.15.120:0/652594110 shutdown_connections 2024-09-19T09:12:21.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.266+0000 7fa0c3323740 1 -- 172.21.15.120:0/652594110 wait complete. 2024-09-19T09:12:21.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.266+0000 7fa0c3323740 20 client.27941 trim_cache size 0 max 16384 2024-09-19T09:12:21.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.281+0000 7fa0c3323740 1 Processor -- start 2024-09-19T09:12:21.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.281+0000 7fa0c3323740 1 -- start start 2024-09-19T09:12:21.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.281+0000 7fa0c3323740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56038971d5d0 0x560389745370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:21.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.281+0000 7fa0c3323740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56038971eb40 0x560389720120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:21.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.281+0000 7fa0c3323740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560389728370 0x560389720660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:21.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.281+0000 7fa0c3323740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc1ad7e0 con 0x56038971eb40 2024-09-19T09:12:21.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.281+0000 7fa0c3323740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc1ad110 con 0x56038971d5d0 2024-09-19T09:12:21.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.281+0000 7fa0c3323740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc1abca0 con 0x560389728370 2024-09-19T09:12:21.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.281+0000 7fa0ba59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56038971eb40 0x560389720120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:21.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.281+0000 7fa0ba59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56038971eb40 0x560389720120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47992/0 (socket says 172.21.15.120:47992) 2024-09-19T09:12:21.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.282+0000 7fa0ba59c640 1 -- 172.21.15.120:0/3183247209 learned_addr learned my addr 172.21.15.120:0/3183247209 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:21.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.282+0000 7fa0b9d9b640 1 --2- 172.21.15.120:0/3183247209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56038971d5d0 0x560389745370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:21.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.282+0000 7fa0b9d9b640 1 --2- 172.21.15.120:0/3183247209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560389728370 0x560389720660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:21.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.282+0000 7fa0b9d9b640 1 -- 172.21.15.120:0/3183247209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560389728370 msgr2=0x560389720660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:21.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.282+0000 7fa0b9d9b640 1 --2- 172.21.15.120:0/3183247209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560389728370 0x560389720660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.282+0000 7fa0b9d9b640 1 -- 172.21.15.120:0/3183247209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56038971eb40 msgr2=0x560389720120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:21.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.282+0000 7fa0b9d9b640 1 --2- 172.21.15.120:0/3183247209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56038971eb40 0x560389720120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.282+0000 7fa0b9d9b640 1 -- 172.21.15.120:0/3183247209 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603897458b0 con 0x56038971d5d0 2024-09-19T09:12:21.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.282+0000 7fa0b9d9b640 1 --2- 172.21.15.120:0/3183247209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56038971d5d0 0x560389745370 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4004700 tx=0x7fa0a4001e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57bb2a8f22e31c92 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:21.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.282+0000 7fa0a3fff640 1 -- 172.21.15.120:0/3183247209 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a4005290 con 0x56038971d5d0 2024-09-19T09:12:21.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.282+0000 7fa0a3fff640 1 -- 172.21.15.120:0/3183247209 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0a40097e0 con 0x56038971d5d0 2024-09-19T09:12:21.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.282+0000 7fa0a3fff640 1 -- 172.21.15.120:0/3183247209 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a401f440 con 0x56038971d5d0 2024-09-19T09:12:21.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.283+0000 7fa0c3323740 1 -- 172.21.15.120:0/3183247209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56038971d5d0 msgr2=0x560389745370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:21.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.283+0000 7fa0c3323740 1 --2- 172.21.15.120:0/3183247209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56038971d5d0 0x560389745370 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4004700 tx=0x7fa0a4001e50 comp rx=0 tx=0).stop 2024-09-19T09:12:21.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.283+0000 7fa0c3323740 1 -- 172.21.15.120:0/3183247209 shutdown_connections 2024-09-19T09:12:21.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.283+0000 7fa0c3323740 1 --2- 172.21.15.120:0/3183247209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560389728370 0x560389720660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.283+0000 7fa0c3323740 1 --2- 172.21.15.120:0/3183247209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56038971eb40 0x560389720120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.283+0000 7fa0c3323740 1 --2- 172.21.15.120:0/3183247209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56038971d5d0 0x560389745370 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.283+0000 7fa0c3323740 1 -- 172.21.15.120:0/3183247209 >> 172.21.15.120:0/3183247209 conn(0x5603896e6310 msgr2=0x560389724180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:21.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.283+0000 7fa0c3323740 1 -- 172.21.15.120:0/3183247209 shutdown_connections 2024-09-19T09:12:21.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.283+0000 7fa0c3323740 1 -- 172.21.15.120:0/3183247209 wait complete. 2024-09-19T09:12:21.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.283+0000 7fa0c3323740 1 -- start start 2024-09-19T09:12:21.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.284+0000 7fa0c3323740 1 Processor -- start 2024-09-19T09:12:21.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.284+0000 7fa0c3323740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56038971d5d0 0x560389741ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:21.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.284+0000 7fa0c3323740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56038971eb40 0x56038973c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:21.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.284+0000 7fa0c3323740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560389728370 0x56038973ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:21.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.284+0000 7fa0c3323740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc1aca40 con 0x560389728370 2024-09-19T09:12:21.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.284+0000 7fa0c3323740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc1ac370 con 0x56038971eb40 2024-09-19T09:12:21.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.284+0000 7fa0c3323740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc072250 con 0x56038971d5d0 2024-09-19T09:12:21.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.284+0000 7fa0c3323740 10 client.0 _ll_register_callbacks cb 0x7fa0a4009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:21.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.284+0000 7fa0b9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560389728370 0x56038973ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:21.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.284+0000 7fa0b9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560389728370 0x56038973ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48004/0 (socket says 172.21.15.120:48004) 2024-09-19T09:12:21.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.284+0000 7fa0b9d9b640 1 -- 172.21.15.120:0/1441448439 learned_addr learned my addr 172.21.15.120:0/1441448439 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:21.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.284+0000 7fa0ba59c640 1 --2- 172.21.15.120:0/1441448439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56038971eb40 0x56038973c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:21.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.284+0000 7fa0b9d9b640 1 --2- 172.21.15.120:0/1441448439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56038971d5d0 0x560389741ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:21.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.284+0000 7fa0ba59c640 1 -- 172.21.15.120:0/1441448439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56038971d5d0 msgr2=0x560389741ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:21.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.284+0000 7fa0ba59c640 1 --2- 172.21.15.120:0/1441448439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56038971d5d0 0x560389741ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.284+0000 7fa0ba59c640 1 -- 172.21.15.120:0/1441448439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560389728370 msgr2=0x56038973ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:21.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.284+0000 7fa0ba59c640 1 --2- 172.21.15.120:0/1441448439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560389728370 0x56038973ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.284+0000 7fa0ba59c640 1 -- 172.21.15.120:0/1441448439 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56038973d2c0 con 0x56038971eb40 2024-09-19T09:12:21.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.284+0000 7fa0b9d9b640 1 --2- 172.21.15.120:0/1441448439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560389728370 0x56038973ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:21.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.285+0000 7fa0ba59c640 1 --2- 172.21.15.120:0/1441448439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56038971eb40 0x56038973c4a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac02a930 tx=0x7fa0ac005ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b02f0ffc02cb54c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:21.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.285+0000 7fa0a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:12:21.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.285+0000 7fa0a1ffb640 1 -- 172.21.15.120:0/1441448439 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac035040 con 0x56038971eb40 2024-09-19T09:12:21.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.285+0000 7fa0c3323740 1 -- 172.21.15.120:0/1441448439 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56038968ad70 con 0x56038971eb40 2024-09-19T09:12:21.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.285+0000 7fa0a1ffb640 1 -- 172.21.15.120:0/1441448439 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa0ac004240 con 0x56038971eb40 2024-09-19T09:12:21.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.285+0000 7fa0a1ffb640 1 -- 172.21.15.120:0/1441448439 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa0ac004440 con 0x56038971eb40 2024-09-19T09:12:21.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.285+0000 7fa0c3323740 10 client.27947 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:21.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.285+0000 7fa0c3323740 1 -- 172.21.15.120:0/1441448439 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5603897b2aa0 con 0x56038971eb40 2024-09-19T09:12:21.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.285+0000 7fa0a1ffb640 1 -- 172.21.15.120:0/1441448439 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56038968ad70 con 0x56038971eb40 2024-09-19T09:12:21.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.285+0000 7fa0a1ffb640 1 -- 172.21.15.120:0/1441448439 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa0ac004bb0 con 0x56038971eb40 2024-09-19T09:12:21.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.285+0000 7fa0c3323740 10 client.27947 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:21.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.285+0000 7fa0c3323740 10 client.27947 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:12:21.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.285+0000 7fa0c3323740 20 client.27947 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:21.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.285+0000 7fa0c3323740 1 --2- 172.21.15.120:0/1441448439 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5603897b3250 0x5603897b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:21.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.285+0000 7fa0c3323740 4 client.27947 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:12:21.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.285+0000 7fa0c3323740 1 -- 172.21.15.120:0/1441448439 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560389607170 con 0x5603897b3250 2024-09-19T09:12:21.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.286+0000 7fa0b9d9b640 1 --2- 172.21.15.120:0/1441448439 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5603897b3250 0x5603897b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:21.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.286+0000 7fa0b9d9b640 1 --2- 172.21.15.120:0/1441448439 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5603897b3250 0x5603897b5670 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c0dcd70ac1e9418 server_cookie=6977dbddd2299af4 in_seq=0 out_seq=0 2024-09-19T09:12:21.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.286+0000 7fa0a1ffb640 10 client.27947 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:12:21.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.286+0000 7fa0a1ffb640 1 -- 172.21.15.120:0/1441448439 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560389607170 con 0x5603897b3250 2024-09-19T09:12:21.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.286+0000 7fa0a1ffb640 10 client.27947 handle_command_reply: tid=0 2024-09-19T09:12:21.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.286+0000 7fa0c3323740 1 client.27947 shutdown 2024-09-19T09:12:21.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.286+0000 7fa0c3323740 1 -- 172.21.15.120:0/1441448439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56038971eb40 msgr2=0x56038973c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:21.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.286+0000 7fa0c3323740 1 --2- 172.21.15.120:0/1441448439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56038971eb40 0x56038973c4a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac02a930 tx=0x7fa0ac005ba0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.286+0000 7fa0c3323740 1 -- 172.21.15.120:0/1441448439 shutdown_connections 2024-09-19T09:12:21.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.286+0000 7fa0c3323740 1 --2- 172.21.15.120:0/1441448439 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5603897b3250 0x5603897b5670 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.286+0000 7fa0c3323740 1 --2- 172.21.15.120:0/1441448439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560389728370 0x56038973ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.286+0000 7fa0c3323740 1 --2- 172.21.15.120:0/1441448439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56038971eb40 0x56038973c4a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.286+0000 7fa0c3323740 1 --2- 172.21.15.120:0/1441448439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56038971d5d0 0x560389741ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.286+0000 7fa0a1ffb640 0 client.27947 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:12:21.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.286+0000 7fa0c3323740 1 -- 172.21.15.120:0/1441448439 >> 172.21.15.120:0/1441448439 conn(0x5603896e6310 msgr2=0x5603897184e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:21.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.287+0000 7fa0c3323740 1 -- 172.21.15.120:0/1441448439 shutdown_connections 2024-09-19T09:12:21.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.287+0000 7fa0c3323740 1 -- 172.21.15.120:0/1441448439 wait complete. 2024-09-19T09:12:21.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.287+0000 7fa0c3323740 20 client.27947 trim_cache size 0 max 16384 2024-09-19T09:12:21.294 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:12:21.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.287+0000 7fa0c0f4a640 1 -- 172.21.15.120:0/3318183904 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa0a80a3340 msgr2=0x7fa0a80a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:21.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.287+0000 7fa0c0f4a640 1 --2- 172.21.15.120:0/3318183904 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa0a80a3340 0x7fa0a80a5800 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40049d0 tx=0x7fa0b400efd0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.287+0000 7fa0c0f4a640 1 -- 172.21.15.120:0/3318183904 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0bc106f20 msgr2=0x7fa0bc074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:21.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.287+0000 7fa0c0f4a640 1 --2- 172.21.15.120:0/3318183904 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0bc106f20 0x7fa0bc074f70 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fa0b400eb10 tx=0x7fa0b400c2d0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.287+0000 7fa0c0f4a640 1 -- 172.21.15.120:0/3318183904 shutdown_connections 2024-09-19T09:12:21.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.287+0000 7fa0c0f4a640 1 --2- 172.21.15.120:0/3318183904 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa0a80a3340 0x7fa0a80a5800 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.287+0000 7fa0c0f4a640 1 --2- 172.21.15.120:0/3318183904 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0bc106f20 0x7fa0bc074f70 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.287+0000 7fa0c0f4a640 1 --2- 172.21.15.120:0/3318183904 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0bc1027a0 0x7fa0bc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.287+0000 7fa0c0f4a640 1 --2- 172.21.15.120:0/3318183904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0bc101df0 0x7fa0bc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:21.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.287+0000 7fa0c0f4a640 1 -- 172.21.15.120:0/3318183904 >> 172.21.15.120:0/3318183904 conn(0x7fa0bc0fd8f0 msgr2=0x7fa0bc0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:21.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.287+0000 7fa0c0f4a640 1 -- 172.21.15.120:0/3318183904 shutdown_connections 2024-09-19T09:12:21.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:21.287+0000 7fa0c0f4a640 1 -- 172.21.15.120:0/3318183904 wait complete. 2024-09-19T09:12:22.066 INFO:tasks.workunit.client.0.smithi120.stdout: 1 919695 9.99 MB/sec execute 377 sec latency 81.129 ms 2024-09-19T09:12:22.088 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:12:22.349 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:12:22.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.823+0000 7f865329c640 1 -- 172.21.15.120:0/4028916526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f864c100a90 msgr2=0x7f864c100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:22.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.823+0000 7f865329c640 1 --2- 172.21.15.120:0/4028916526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f864c100a90 0x7f864c100ef0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f863c009930 tx=0x7f863c02a420 comp rx=0 tx=0).stop 2024-09-19T09:12:22.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.823+0000 7f865329c640 1 -- 172.21.15.120:0/4028916526 shutdown_connections 2024-09-19T09:12:22.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.823+0000 7f865329c640 1 --2- 172.21.15.120:0/4028916526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f864c101430 0x7f864c10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.823+0000 7f865329c640 1 --2- 172.21.15.120:0/4028916526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f864c100a90 0x7f864c100ef0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.823+0000 7f865329c640 1 --2- 172.21.15.120:0/4028916526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f864c106a90 0x7f864c106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.823+0000 7f865329c640 1 -- 172.21.15.120:0/4028916526 >> 172.21.15.120:0/4028916526 conn(0x7f864c0fc7d0 msgr2=0x7f864c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:22.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.823+0000 7f865329c640 1 -- 172.21.15.120:0/4028916526 shutdown_connections 2024-09-19T09:12:22.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.823+0000 7f865329c640 1 -- 172.21.15.120:0/4028916526 wait complete. 2024-09-19T09:12:22.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.824+0000 7f865329c640 1 Processor -- start 2024-09-19T09:12:22.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.824+0000 7f865329c640 1 -- start start 2024-09-19T09:12:22.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.824+0000 7f865329c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f864c100a90 0x7f864c0fce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:22.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.824+0000 7f865329c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f864c101430 0x7f864c0fd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:22.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.824+0000 7f865329c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f864c106a90 0x7f864c0fd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:22.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.824+0000 7f865329c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f864c110e90 con 0x7f864c106a90 2024-09-19T09:12:22.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.824+0000 7f865329c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f864c110d10 con 0x7f864c101430 2024-09-19T09:12:22.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.824+0000 7f865329c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f864c111010 con 0x7f864c100a90 2024-09-19T09:12:22.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.824+0000 7f8651839640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f864c106a90 0x7f864c0fd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:22.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.824+0000 7f8650837640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f864c101430 0x7f864c0fd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:22.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.824+0000 7f8651839640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f864c106a90 0x7f864c0fd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48030/0 (socket says 172.21.15.120:48030) 2024-09-19T09:12:22.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.824+0000 7f8651839640 1 -- 172.21.15.120:0/936059581 learned_addr learned my addr 172.21.15.120:0/936059581 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:22.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.824+0000 7f8651038640 1 --2- 172.21.15.120:0/936059581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f864c100a90 0x7f864c0fce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:22.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.824+0000 7f8651839640 1 -- 172.21.15.120:0/936059581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f864c100a90 msgr2=0x7f864c0fce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:22.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.824+0000 7f8651839640 1 --2- 172.21.15.120:0/936059581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f864c100a90 0x7f864c0fce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.824+0000 7f8651839640 1 -- 172.21.15.120:0/936059581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f864c101430 msgr2=0x7f864c0fd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:22.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.824+0000 7f8651839640 1 --2- 172.21.15.120:0/936059581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f864c101430 0x7f864c0fd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.824+0000 7f8651839640 1 -- 172.21.15.120:0/936059581 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f864c1a6930 con 0x7f864c106a90 2024-09-19T09:12:22.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.825+0000 7f8651839640 1 --2- 172.21.15.120:0/936059581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f864c106a90 0x7f864c0fd890 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f864400eaa0 tx=0x7f864400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:22.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.825+0000 7f86427fc640 1 -- 172.21.15.120:0/936059581 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864400cdb0 con 0x7f864c106a90 2024-09-19T09:12:22.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.825+0000 7f86427fc640 1 -- 172.21.15.120:0/936059581 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8644005020 con 0x7f864c106a90 2024-09-19T09:12:22.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.825+0000 7f86427fc640 1 -- 172.21.15.120:0/936059581 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86440116b0 con 0x7f864c106a90 2024-09-19T09:12:22.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.825+0000 7f865329c640 1 -- 172.21.15.120:0/936059581 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f864c1a6c20 con 0x7f864c106a90 2024-09-19T09:12:22.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.825+0000 7f865329c640 1 -- 172.21.15.120:0/936059581 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f864c1a70e0 con 0x7f864c106a90 2024-09-19T09:12:22.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.827+0000 7f86427fc640 1 -- 172.21.15.120:0/936059581 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f8644005560 con 0x7f864c106a90 2024-09-19T09:12:22.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.828+0000 7f8655675740 1 Processor -- start 2024-09-19T09:12:22.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.828+0000 7f8655675740 1 -- start start 2024-09-19T09:12:22.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.828+0000 7f8655675740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed6925fb40 0x55ed6925e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:22.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.828+0000 7f8655675740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed6925e5d0 0x55ed6925ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:22.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.828+0000 7f8655675740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed69269370 0x55ed6926b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:22.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.828+0000 7f8655675740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ed691c42f0 con 0x55ed6925fb40 2024-09-19T09:12:22.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.828+0000 7f8655675740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ed6916bcf0 con 0x55ed69269370 2024-09-19T09:12:22.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.828+0000 7f8655675740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ed691c02b0 con 0x55ed6925e5d0 2024-09-19T09:12:22.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.828+0000 7f8650837640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed6925fb40 0x55ed6925e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:22.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.828+0000 7f8650837640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed6925fb40 0x55ed6925e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48034/0 (socket says 172.21.15.120:48034) 2024-09-19T09:12:22.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.828+0000 7f8650837640 1 -- 172.21.15.120:0/2443142202 learned_addr learned my addr 172.21.15.120:0/2443142202 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:22.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.828+0000 7f8650837640 1 --2- 172.21.15.120:0/2443142202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed69269370 0x55ed6926b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:22.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.828+0000 7f8651038640 1 --2- 172.21.15.120:0/2443142202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed6925e5d0 0x55ed6925ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:22.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.828+0000 7f86427fc640 1 --2- 172.21.15.120:0/936059581 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f86380a32c0 0x7f86380a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:22.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.828+0000 7f8650837640 1 -- 172.21.15.120:0/2443142202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed6925e5d0 msgr2=0x55ed6925ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:22.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.828+0000 7f8650837640 1 --2- 172.21.15.120:0/2443142202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed6925e5d0 0x55ed6925ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.828+0000 7f8650837640 1 -- 172.21.15.120:0/2443142202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed69269370 msgr2=0x55ed6926b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:22.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.828+0000 7f8650837640 1 --2- 172.21.15.120:0/2443142202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed69269370 0x55ed6926b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.828+0000 7f8650837640 1 -- 172.21.15.120:0/2443142202 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed691503a0 con 0x55ed6925fb40 2024-09-19T09:12:22.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.828+0000 7f8651038640 1 --2- 172.21.15.120:0/2443142202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed6925e5d0 0x55ed6925ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:22.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.829+0000 7f8651839640 1 --2- 172.21.15.120:0/936059581 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f86380a32c0 0x7f86380a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:22.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.829+0000 7f8650837640 1 --2- 172.21.15.120:0/2443142202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed6925fb40 0x55ed6925e090 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f863c02a940 tx=0x7f863c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bdcfe8ac42323d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:22.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.829+0000 7f86427fc640 1 -- 172.21.15.120:0/936059581 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f86440cfa10 con 0x7f864c106a90 2024-09-19T09:12:22.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.829+0000 7f86377fe640 1 -- 172.21.15.120:0/2443142202 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c035040 con 0x55ed6925fb40 2024-09-19T09:12:22.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.829+0000 7f86377fe640 1 -- 172.21.15.120:0/2443142202 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f863c005500 con 0x55ed6925fb40 2024-09-19T09:12:22.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.829+0000 7f86377fe640 1 -- 172.21.15.120:0/2443142202 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c004c60 con 0x55ed6925fb40 2024-09-19T09:12:22.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.830+0000 7f8655675740 1 -- 172.21.15.120:0/2443142202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed6925fb40 msgr2=0x55ed6925e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:22.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.830+0000 7f8651839640 1 --2- 172.21.15.120:0/936059581 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f86380a32c0 0x7f86380a5780 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8644000f80 tx=0x7f8644066fc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:22.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.830+0000 7f8655675740 1 --2- 172.21.15.120:0/2443142202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed6925fb40 0x55ed6925e090 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f863c02a940 tx=0x7f863c000bc0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.830+0000 7f8655675740 1 -- 172.21.15.120:0/2443142202 shutdown_connections 2024-09-19T09:12:22.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.830+0000 7f8655675740 1 --2- 172.21.15.120:0/2443142202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed69269370 0x55ed6926b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.830+0000 7f8655675740 1 --2- 172.21.15.120:0/2443142202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed6925e5d0 0x55ed6925ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.830+0000 7f8655675740 1 --2- 172.21.15.120:0/2443142202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed6925fb40 0x55ed6925e090 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.830+0000 7f8655675740 1 -- 172.21.15.120:0/2443142202 >> 172.21.15.120:0/2443142202 conn(0x55ed69227310 msgr2=0x55ed6925ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:22.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.830+0000 7f8655675740 1 -- 172.21.15.120:0/2443142202 shutdown_connections 2024-09-19T09:12:22.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.830+0000 7f8655675740 1 -- 172.21.15.120:0/2443142202 wait complete. 2024-09-19T09:12:22.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.830+0000 7f8655675740 1 -- start start 2024-09-19T09:12:22.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.830+0000 7f8655675740 1 Processor -- start 2024-09-19T09:12:22.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.831+0000 7f8655675740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed6925e5d0 0x55ed69268d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:22.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.831+0000 7f8655675740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed6925fb40 0x55ed69265fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:22.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.831+0000 7f8655675740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed69269370 0x55ed69266560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:22.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.831+0000 7f8655675740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ed6926bfb0 con 0x55ed6925e5d0 2024-09-19T09:12:22.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.831+0000 7f8655675740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ed6917f060 con 0x55ed6925fb40 2024-09-19T09:12:22.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.831+0000 7f8655675740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ed6926c130 con 0x55ed69269370 2024-09-19T09:12:22.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.831+0000 7f8655675740 10 client.0 _ll_register_callbacks cb 0x55ed68ed28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:22.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.831+0000 7f8650837640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed6925e5d0 0x55ed69268d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:22.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.831+0000 7f8651038640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed6925fb40 0x55ed69265fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:22.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.831+0000 7f8650837640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed6925e5d0 0x55ed69268d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48036/0 (socket says 172.21.15.120:48036) 2024-09-19T09:12:22.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.831+0000 7f8650837640 1 -- 172.21.15.120:0/1071786362 learned_addr learned my addr 172.21.15.120:0/1071786362 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:22.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.831+0000 7f8650837640 1 --2- 172.21.15.120:0/1071786362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed69269370 0x55ed69266560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:22.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.831+0000 7f8651038640 1 -- 172.21.15.120:0/1071786362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed69269370 msgr2=0x55ed69266560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:22.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.831+0000 7f8651038640 1 --2- 172.21.15.120:0/1071786362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed69269370 0x55ed69266560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.831+0000 7f8651038640 1 -- 172.21.15.120:0/1071786362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed6925e5d0 msgr2=0x55ed69268d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:22.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.831+0000 7f8651038640 1 --2- 172.21.15.120:0/1071786362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed6925e5d0 0x55ed69268d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.831+0000 7f8651038640 1 -- 172.21.15.120:0/1071786362 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed68f353c0 con 0x55ed6925fb40 2024-09-19T09:12:22.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.831+0000 7f8650837640 1 --2- 172.21.15.120:0/1071786362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed69269370 0x55ed69266560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:12:22.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.831+0000 7f8651038640 1 --2- 172.21.15.120:0/1071786362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed6925fb40 0x55ed69265fc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f86480126b0 tx=0x7f8648012ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9e2035da31413b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:22.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.831+0000 7f8634ff9640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:12:22.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.832+0000 7f8634ff9640 1 -- 172.21.15.120:0/1071786362 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8648004050 con 0x55ed6925fb40 2024-09-19T09:12:22.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.832+0000 7f8634ff9640 1 -- 172.21.15.120:0/1071786362 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f86480041f0 con 0x55ed6925fb40 2024-09-19T09:12:22.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.832+0000 7f8655675740 1 -- 172.21.15.120:0/1071786362 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f863c005500 con 0x55ed6925fb40 2024-09-19T09:12:22.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.832+0000 7f8634ff9640 1 -- 172.21.15.120:0/1071786362 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f86480043f0 con 0x55ed6925fb40 2024-09-19T09:12:22.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.832+0000 7f8655675740 10 client.27959 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:22.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.832+0000 7f8655675740 1 -- 172.21.15.120:0/1071786362 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ed69277a90 con 0x55ed6925fb40 2024-09-19T09:12:22.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.832+0000 7f8634ff9640 1 -- 172.21.15.120:0/1071786362 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f863c005500 con 0x55ed6925fb40 2024-09-19T09:12:22.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.832+0000 7f8634ff9640 1 -- 172.21.15.120:0/1071786362 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f8648003980 con 0x55ed6925fb40 2024-09-19T09:12:22.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.832+0000 7f8655675740 10 client.27959 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:22.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.832+0000 7f8655675740 10 client.27959 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:12:22.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.832+0000 7f8655675740 20 client.27959 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:22.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.832+0000 7f8655675740 1 --2- 172.21.15.120:0/1071786362 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55ed69278fa0 0x55ed6927b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:22.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.832+0000 7f8655675740 4 client.27959 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:12:22.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.832+0000 7f8655675740 1 -- 172.21.15.120:0/1071786362 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed691d1860 con 0x55ed69278fa0 2024-09-19T09:12:22.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.832+0000 7f8650837640 1 --2- 172.21.15.120:0/1071786362 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55ed69278fa0 0x55ed6927b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:22.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.833+0000 7f8650837640 1 --2- 172.21.15.120:0/1071786362 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55ed69278fa0 0x55ed6927b390 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b84958f9cdbe707d server_cookie=ad32a5c3fb79f581 in_seq=0 out_seq=0 2024-09-19T09:12:22.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.833+0000 7f8634ff9640 10 client.27959 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:12:22.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.833+0000 7f8634ff9640 1 -- 172.21.15.120:0/1071786362 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed691d1860 con 0x55ed69278fa0 2024-09-19T09:12:22.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.833+0000 7f8634ff9640 10 client.27959 handle_command_reply: tid=0 2024-09-19T09:12:22.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.833+0000 7f8655675740 1 client.27959 shutdown 2024-09-19T09:12:22.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.834+0000 7f8655675740 1 -- 172.21.15.120:0/1071786362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed6925fb40 msgr2=0x55ed69265fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:22.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.834+0000 7f8655675740 1 --2- 172.21.15.120:0/1071786362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed6925fb40 0x55ed69265fc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f86480126b0 tx=0x7f8648012ab0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.834+0000 7f8655675740 1 -- 172.21.15.120:0/1071786362 shutdown_connections 2024-09-19T09:12:22.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.834+0000 7f8655675740 1 --2- 172.21.15.120:0/1071786362 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55ed69278fa0 0x55ed6927b390 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.834+0000 7f8655675740 1 --2- 172.21.15.120:0/1071786362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed69269370 0x55ed69266560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.834+0000 7f8655675740 1 --2- 172.21.15.120:0/1071786362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed6925fb40 0x55ed69265fc0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.834+0000 7f8655675740 1 --2- 172.21.15.120:0/1071786362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed6925e5d0 0x55ed69268d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.834+0000 7f8634ff9640 0 client.27959 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:12:22.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.834+0000 7f8655675740 1 -- 172.21.15.120:0/1071786362 >> 172.21.15.120:0/1071786362 conn(0x55ed69227310 msgr2=0x55ed69260930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:22.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.834+0000 7f8655675740 1 -- 172.21.15.120:0/1071786362 shutdown_connections 2024-09-19T09:12:22.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.834+0000 7f8655675740 1 -- 172.21.15.120:0/1071786362 wait complete. 2024-09-19T09:12:22.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.834+0000 7f8655675740 20 client.27959 trim_cache size 0 max 16384 2024-09-19T09:12:22.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.847+0000 7f8655675740 1 Processor -- start 2024-09-19T09:12:22.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.847+0000 7f8655675740 1 -- start start 2024-09-19T09:12:22.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.848+0000 7f8655675740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed6925e5d0 0x55ed69265880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:22.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.848+0000 7f8655675740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed6925fb40 0x55ed69267f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:22.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.848+0000 7f8655675740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed69269370 0x55ed69261120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:22.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.848+0000 7f8655675740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f864c1af7a0 con 0x55ed6925fb40 2024-09-19T09:12:22.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.848+0000 7f8655675740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f864c1b12e0 con 0x55ed6925e5d0 2024-09-19T09:12:22.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.848+0000 7f8655675740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f864c1afe70 con 0x55ed69269370 2024-09-19T09:12:22.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.848+0000 7f8651038640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed6925fb40 0x55ed69267f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:22.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.848+0000 7f8651038640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed6925fb40 0x55ed69267f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48046/0 (socket says 172.21.15.120:48046) 2024-09-19T09:12:22.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.848+0000 7f8651038640 1 -- 172.21.15.120:0/1872695675 learned_addr learned my addr 172.21.15.120:0/1872695675 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:22.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.848+0000 7f8650837640 1 --2- 172.21.15.120:0/1872695675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed6925e5d0 0x55ed69265880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:22.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.848+0000 7f8650837640 1 --2- 172.21.15.120:0/1872695675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed69269370 0x55ed69261120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:22.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.848+0000 7f8651038640 1 -- 172.21.15.120:0/1872695675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed69269370 msgr2=0x55ed69261120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:22.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.848+0000 7f8651038640 1 --2- 172.21.15.120:0/1872695675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed69269370 0x55ed69261120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.848+0000 7f8651038640 1 -- 172.21.15.120:0/1872695675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed6925e5d0 msgr2=0x55ed69265880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:22.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.848+0000 7f8651038640 1 --2- 172.21.15.120:0/1872695675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed6925e5d0 0x55ed69265880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.848+0000 7f8651038640 1 -- 172.21.15.120:0/1872695675 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed69264f70 con 0x55ed6925fb40 2024-09-19T09:12:22.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.848+0000 7f8650837640 1 --2- 172.21.15.120:0/1872695675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed6925e5d0 0x55ed69265880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:12:22.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.848+0000 7f8651038640 1 --2- 172.21.15.120:0/1872695675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed6925fb40 0x55ed69267f30 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f8648003da0 tx=0x7f86480057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb04fc279c7c2897 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:22.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.848+0000 7f8636ffd640 1 -- 172.21.15.120:0/1872695675 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86480105f0 con 0x55ed6925fb40 2024-09-19T09:12:22.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.849+0000 7f8636ffd640 1 -- 172.21.15.120:0/1872695675 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f864802b030 con 0x55ed6925fb40 2024-09-19T09:12:22.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.849+0000 7f8636ffd640 1 -- 172.21.15.120:0/1872695675 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8648005580 con 0x55ed6925fb40 2024-09-19T09:12:22.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.849+0000 7f8655675740 1 -- 172.21.15.120:0/1872695675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed6925fb40 msgr2=0x55ed69267f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:22.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.849+0000 7f8655675740 1 --2- 172.21.15.120:0/1872695675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed6925fb40 0x55ed69267f30 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f8648003da0 tx=0x7f86480057d0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.850+0000 7f8655675740 1 -- 172.21.15.120:0/1872695675 shutdown_connections 2024-09-19T09:12:22.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.850+0000 7f8655675740 1 --2- 172.21.15.120:0/1872695675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed69269370 0x55ed69261120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.850+0000 7f8655675740 1 --2- 172.21.15.120:0/1872695675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed6925fb40 0x55ed69267f30 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.850+0000 7f8655675740 1 --2- 172.21.15.120:0/1872695675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed6925e5d0 0x55ed69265880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.850+0000 7f8655675740 1 -- 172.21.15.120:0/1872695675 >> 172.21.15.120:0/1872695675 conn(0x55ed69227310 msgr2=0x55ed69267820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:22.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.850+0000 7f8655675740 1 -- 172.21.15.120:0/1872695675 shutdown_connections 2024-09-19T09:12:22.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.850+0000 7f8655675740 1 -- 172.21.15.120:0/1872695675 wait complete. 2024-09-19T09:12:22.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.850+0000 7f8655675740 1 -- start start 2024-09-19T09:12:22.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.850+0000 7f8655675740 1 Processor -- start 2024-09-19T09:12:22.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.851+0000 7f8655675740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed6925e5d0 0x55ed69271c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:22.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.851+0000 7f8655675740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed6925fb40 0x55ed69262e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:22.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.851+0000 7f8655675740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed69269370 0x55ed69263420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:22.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.851+0000 7f8655675740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f864c1b0c10 con 0x55ed6925fb40 2024-09-19T09:12:22.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.851+0000 7f8655675740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f864c1b0540 con 0x55ed6925e5d0 2024-09-19T09:12:22.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.851+0000 7f8655675740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f864c1ad760 con 0x55ed69269370 2024-09-19T09:12:22.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.851+0000 7f8655675740 10 client.0 _ll_register_callbacks cb 0x7f8648009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:22.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.851+0000 7f8651038640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed6925fb40 0x55ed69262e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:22.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.851+0000 7f8650837640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed6925e5d0 0x55ed69271c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:22.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.851+0000 7f8650837640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed6925e5d0 0x55ed69271c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:48222/0 (socket says 172.21.15.120:48222) 2024-09-19T09:12:22.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.851+0000 7f8651038640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed6925fb40 0x55ed69262e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48062/0 (socket says 172.21.15.120:48062) 2024-09-19T09:12:22.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.851+0000 7f8650837640 1 -- 172.21.15.120:0/651000483 learned_addr learned my addr 172.21.15.120:0/651000483 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:22.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.851+0000 7f8650837640 1 --2- 172.21.15.120:0/651000483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed69269370 0x55ed69263420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:22.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.851+0000 7f8651038640 1 -- 172.21.15.120:0/651000483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed69269370 msgr2=0x55ed69263420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:22.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.851+0000 7f8651038640 1 --2- 172.21.15.120:0/651000483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed69269370 0x55ed69263420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.851+0000 7f8651038640 1 -- 172.21.15.120:0/651000483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed6925e5d0 msgr2=0x55ed69271c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:22.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.851+0000 7f8651038640 1 --2- 172.21.15.120:0/651000483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed6925e5d0 0x55ed69271c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.851+0000 7f8651038640 1 -- 172.21.15.120:0/651000483 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed69263c20 con 0x55ed6925fb40 2024-09-19T09:12:22.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.851+0000 7f8650837640 1 --2- 172.21.15.120:0/651000483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed6925e5d0 0x55ed69271c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:22.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.851+0000 7f8651038640 1 --2- 172.21.15.120:0/651000483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed6925fb40 0x55ed69262e10 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x55ed692646f0 tx=0x7f8648003fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9d9568a904ba04 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:22.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.851+0000 7f8634ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:22.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.853+0000 7f8634ff9640 1 -- 172.21.15.120:0/651000483 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8648034070 con 0x55ed6925fb40 2024-09-19T09:12:22.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.853+0000 7f8634ff9640 1 -- 172.21.15.120:0/651000483 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f86480052c0 con 0x55ed6925fb40 2024-09-19T09:12:22.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.853+0000 7f8655675740 1 -- 172.21.15.120:0/651000483 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f864802b030 con 0x55ed6925fb40 2024-09-19T09:12:22.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.853+0000 7f8634ff9640 1 -- 172.21.15.120:0/651000483 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f86480054e0 con 0x55ed6925fb40 2024-09-19T09:12:22.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.853+0000 7f8655675740 10 client.18837 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:22.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.853+0000 7f8655675740 1 -- 172.21.15.120:0/651000483 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ed692e3d70 con 0x55ed6925fb40 2024-09-19T09:12:22.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.853+0000 7f8634ff9640 1 -- 172.21.15.120:0/651000483 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f864802b030 con 0x55ed6925fb40 2024-09-19T09:12:22.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.854+0000 7f8634ff9640 1 -- 172.21.15.120:0/651000483 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f864800fb30 con 0x55ed6925fb40 2024-09-19T09:12:22.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.854+0000 7f8655675740 10 client.18837 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:22.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.854+0000 7f8655675740 10 client.18837 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:12:22.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.854+0000 7f8655675740 20 client.18837 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:22.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.854+0000 7f8655675740 1 --2- 172.21.15.120:0/651000483 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55ed692e4370 0x55ed692e6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:22.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.854+0000 7f8655675740 4 client.18837 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:12:22.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.854+0000 7f8655675740 1 -- 172.21.15.120:0/651000483 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed691cbd70 con 0x55ed692e4370 2024-09-19T09:12:22.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.854+0000 7f8650837640 1 --2- 172.21.15.120:0/651000483 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55ed692e4370 0x55ed692e6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:22.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.854+0000 7f8650837640 1 --2- 172.21.15.120:0/651000483 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55ed692e4370 0x55ed692e6760 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1fcdd3bc04d90c44 server_cookie=dc276b22cf3b536e in_seq=0 out_seq=0 2024-09-19T09:12:22.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.854+0000 7f8634ff9640 10 client.18837 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:12:22.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.854+0000 7f8634ff9640 1 -- 172.21.15.120:0/651000483 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed691cbd70 con 0x55ed692e4370 2024-09-19T09:12:22.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.854+0000 7f8634ff9640 10 client.18837 handle_command_reply: tid=0 2024-09-19T09:12:22.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.854+0000 7f8655675740 1 client.18837 shutdown 2024-09-19T09:12:22.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.855+0000 7f8655675740 1 -- 172.21.15.120:0/651000483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed6925fb40 msgr2=0x55ed69262e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:22.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.855+0000 7f8655675740 1 --2- 172.21.15.120:0/651000483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed6925fb40 0x55ed69262e10 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x55ed692646f0 tx=0x7f8648003fc0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.855+0000 7f8655675740 1 -- 172.21.15.120:0/651000483 shutdown_connections 2024-09-19T09:12:22.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.855+0000 7f8655675740 1 --2- 172.21.15.120:0/651000483 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55ed692e4370 0x55ed692e6760 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.855+0000 7f8634ff9640 0 client.18837 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:12:22.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.855+0000 7f8655675740 1 --2- 172.21.15.120:0/651000483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed69269370 0x55ed69263420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.855+0000 7f8655675740 1 --2- 172.21.15.120:0/651000483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed6925fb40 0x55ed69262e10 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.855+0000 7f8655675740 1 --2- 172.21.15.120:0/651000483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed6925e5d0 0x55ed69271c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.855+0000 7f8655675740 1 -- 172.21.15.120:0/651000483 >> 172.21.15.120:0/651000483 conn(0x55ed69227310 msgr2=0x55ed69268040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:22.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.855+0000 7f8655675740 1 -- 172.21.15.120:0/651000483 shutdown_connections 2024-09-19T09:12:22.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.855+0000 7f8655675740 1 -- 172.21.15.120:0/651000483 wait complete. 2024-09-19T09:12:22.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.855+0000 7f8655675740 20 client.18837 trim_cache size 0 max 16384 2024-09-19T09:12:22.872 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:12:22.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.856+0000 7f865329c640 1 -- 172.21.15.120:0/936059581 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f86380a32c0 msgr2=0x7f86380a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:22.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.856+0000 7f865329c640 1 --2- 172.21.15.120:0/936059581 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f86380a32c0 0x7f86380a5780 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8644000f80 tx=0x7f8644066fc0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.856+0000 7f865329c640 1 -- 172.21.15.120:0/936059581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f864c106a90 msgr2=0x7f864c0fd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:22.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.856+0000 7f865329c640 1 --2- 172.21.15.120:0/936059581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f864c106a90 0x7f864c0fd890 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f864400eaa0 tx=0x7f864400ef60 comp rx=0 tx=0).stop 2024-09-19T09:12:22.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.856+0000 7f865329c640 1 -- 172.21.15.120:0/936059581 shutdown_connections 2024-09-19T09:12:22.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.856+0000 7f865329c640 1 --2- 172.21.15.120:0/936059581 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f86380a32c0 0x7f86380a5780 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.856+0000 7f865329c640 1 --2- 172.21.15.120:0/936059581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f864c106a90 0x7f864c0fd890 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.856+0000 7f865329c640 1 --2- 172.21.15.120:0/936059581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f864c101430 0x7f864c0fd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.856+0000 7f865329c640 1 --2- 172.21.15.120:0/936059581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f864c100a90 0x7f864c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:22.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.856+0000 7f865329c640 1 -- 172.21.15.120:0/936059581 >> 172.21.15.120:0/936059581 conn(0x7f864c0fc7d0 msgr2=0x7f864c06a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:22.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.856+0000 7f865329c640 1 -- 172.21.15.120:0/936059581 shutdown_connections 2024-09-19T09:12:22.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:22.856+0000 7f865329c640 1 -- 172.21.15.120:0/936059581 wait complete. 2024-09-19T09:12:23.066 INFO:tasks.workunit.client.0.smithi120.stdout: 1 922241 9.99 MB/sec execute 378 sec latency 31.434 ms 2024-09-19T09:12:23.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:22 smithi120 ceph-mon[31150]: pgmap v628: 97 pgs: 97 active+clean; 46 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.7 MiB/s wr, 346 op/s 2024-09-19T09:12:23.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:22 smithi196 ceph-mon[35150]: pgmap v628: 97 pgs: 97 active+clean; 46 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.7 MiB/s wr, 346 op/s 2024-09-19T09:12:23.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:22 smithi153 ceph-mon[35071]: pgmap v628: 97 pgs: 97 active+clean; 46 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.7 MiB/s wr, 346 op/s 2024-09-19T09:12:23.576 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:12:23.830 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:12:24.066 INFO:tasks.workunit.client.0.smithi120.stdout: 1 924768 10.00 MB/sec execute 379 sec latency 81.723 ms 2024-09-19T09:12:24.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.332+0000 7ffbc165b640 1 -- 172.21.15.120:0/4183051484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffbbc101df0 msgr2=0x7ffbbc1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:24.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.332+0000 7ffbc165b640 1 --2- 172.21.15.120:0/4183051484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffbbc101df0 0x7ffbbc1021d0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7ffbac009900 tx=0x7ffbac02a410 comp rx=0 tx=0).stop 2024-09-19T09:12:24.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.332+0000 7ffbc165b640 1 -- 172.21.15.120:0/4183051484 shutdown_connections 2024-09-19T09:12:24.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.332+0000 7ffbc165b640 1 --2- 172.21.15.120:0/4183051484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbbc106f20 0x7ffbbc10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.332+0000 7ffbc165b640 1 --2- 172.21.15.120:0/4183051484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbbc1027a0 0x7ffbbc1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.332+0000 7ffbc165b640 1 --2- 172.21.15.120:0/4183051484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffbbc101df0 0x7ffbbc1021d0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.332+0000 7ffbc165b640 1 -- 172.21.15.120:0/4183051484 >> 172.21.15.120:0/4183051484 conn(0x7ffbbc0fd8b0 msgr2=0x7ffbbc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:24.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.332+0000 7ffbc165b640 1 -- 172.21.15.120:0/4183051484 shutdown_connections 2024-09-19T09:12:24.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.332+0000 7ffbc165b640 1 -- 172.21.15.120:0/4183051484 wait complete. 2024-09-19T09:12:24.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.333+0000 7ffbc165b640 1 Processor -- start 2024-09-19T09:12:24.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.333+0000 7ffbc165b640 1 -- start start 2024-09-19T09:12:24.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.333+0000 7ffbc165b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbbc101df0 0x7ffbbc198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:24.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.333+0000 7ffbc165b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffbbc1027a0 0x7ffbbc198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:24.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.333+0000 7ffbc165b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbbc106f20 0x7ffbbc19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:24.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.334+0000 7ffbc165b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffbbc111f80 con 0x7ffbbc1027a0 2024-09-19T09:12:24.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.334+0000 7ffbc165b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffbbc111e00 con 0x7ffbbc106f20 2024-09-19T09:12:24.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.334+0000 7ffbc165b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffbbc112100 con 0x7ffbbc101df0 2024-09-19T09:12:24.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.334+0000 7ffbba7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffbbc1027a0 0x7ffbbc198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:24.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.334+0000 7ffbbaffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbbc101df0 0x7ffbbc198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:24.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.334+0000 7ffbbaffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbbc101df0 0x7ffbbc198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:48016/0 (socket says 172.21.15.120:48016) 2024-09-19T09:12:24.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.334+0000 7ffbbb7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbbc106f20 0x7ffbbc19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:24.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.334+0000 7ffbbaffd640 1 -- 172.21.15.120:0/3629994541 learned_addr learned my addr 172.21.15.120:0/3629994541 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:24.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.334+0000 7ffbba7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffbbc1027a0 0x7ffbbc198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48076/0 (socket says 172.21.15.120:48076) 2024-09-19T09:12:24.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.334+0000 7ffbbb7fe640 1 -- 172.21.15.120:0/3629994541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbbc101df0 msgr2=0x7ffbbc198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:24.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.334+0000 7ffbbb7fe640 1 --2- 172.21.15.120:0/3629994541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbbc101df0 0x7ffbbc198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.334+0000 7ffbbb7fe640 1 -- 172.21.15.120:0/3629994541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffbbc1027a0 msgr2=0x7ffbbc198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:24.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.334+0000 7ffbbb7fe640 1 --2- 172.21.15.120:0/3629994541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffbbc1027a0 0x7ffbbc198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.334+0000 7ffbbb7fe640 1 -- 172.21.15.120:0/3629994541 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbbc19d3f0 con 0x7ffbbc106f20 2024-09-19T09:12:24.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.334+0000 7ffbba7fc640 1 --2- 172.21.15.120:0/3629994541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffbbc1027a0 0x7ffbbc198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:24.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.334+0000 7ffbbaffd640 1 --2- 172.21.15.120:0/3629994541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbbc101df0 0x7ffbbc198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:24.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.334+0000 7ffbbb7fe640 1 --2- 172.21.15.120:0/3629994541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbbc106f20 0x7ffbbc19cd10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4009fd0 tx=0x7ffbb400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:24.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.334+0000 7ffba3fff640 1 -- 172.21.15.120:0/3629994541 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb400ce20 con 0x7ffbbc106f20 2024-09-19T09:12:24.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.334+0000 7ffbc165b640 1 -- 172.21.15.120:0/3629994541 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbbc19d6e0 con 0x7ffbbc106f20 2024-09-19T09:12:24.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.334+0000 7ffba3fff640 1 -- 172.21.15.120:0/3629994541 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbb4005020 con 0x7ffbbc106f20 2024-09-19T09:12:24.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.335+0000 7ffba3fff640 1 -- 172.21.15.120:0/3629994541 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4010720 con 0x7ffbbc106f20 2024-09-19T09:12:24.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.335+0000 7ffbc165b640 1 -- 172.21.15.120:0/3629994541 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbbc1a4fc0 con 0x7ffbbc106f20 2024-09-19T09:12:24.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.337+0000 7ffba3fff640 1 -- 172.21.15.120:0/3629994541 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7ffbb4005540 con 0x7ffbbc106f20 2024-09-19T09:12:24.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.337+0000 7ffbc3a34740 1 Processor -- start 2024-09-19T09:12:24.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.337+0000 7ffbc3a34740 1 -- start start 2024-09-19T09:12:24.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.337+0000 7ffbc3a34740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ded6e8b40 0x560ded6e7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:24.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.337+0000 7ffbc3a34740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ded6e75d0 0x560ded6e7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:24.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.337+0000 7ffbc3a34740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ded6f2370 0x560ded6f4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:24.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.337+0000 7ffbc3a34740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560ded64d2f0 con 0x560ded6f2370 2024-09-19T09:12:24.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.337+0000 7ffbc3a34740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560ded5f4cf0 con 0x560ded6e75d0 2024-09-19T09:12:24.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.337+0000 7ffbc3a34740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560ded6492b0 con 0x560ded6e8b40 2024-09-19T09:12:24.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.337+0000 7ffba3fff640 1 --2- 172.21.15.120:0/3629994541 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ffba40a3240 0x7ffba40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:24.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.338+0000 7ffbba7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ded6e8b40 0x560ded6e7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:24.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.338+0000 7ffbba7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ded6e8b40 0x560ded6e7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:48030/0 (socket says 172.21.15.120:48030) 2024-09-19T09:12:24.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.338+0000 7ffbba7fc640 1 -- 172.21.15.120:0/1547935506 learned_addr learned my addr 172.21.15.120:0/1547935506 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:24.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.338+0000 7ffbba7fc640 1 --2- 172.21.15.120:0/1547935506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ded6f2370 0x560ded6f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:24.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.338+0000 7ffbbaffd640 1 --2- 172.21.15.120:0/1547935506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ded6e75d0 0x560ded6e7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:24.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.338+0000 7ffba3fff640 1 -- 172.21.15.120:0/3629994541 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ffbb40cffd0 con 0x7ffbbc106f20 2024-09-19T09:12:24.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.338+0000 7ffbbb7fe640 1 --2- 172.21.15.120:0/3629994541 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ffba40a3240 0x7ffba40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:24.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.338+0000 7ffbba7fc640 1 -- 172.21.15.120:0/1547935506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ded6e75d0 msgr2=0x560ded6e7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:24.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.338+0000 7ffbba7fc640 1 --2- 172.21.15.120:0/1547935506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ded6e75d0 0x560ded6e7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.338+0000 7ffbba7fc640 1 -- 172.21.15.120:0/1547935506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ded6f2370 msgr2=0x560ded6f4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:24.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.338+0000 7ffbba7fc640 1 --2- 172.21.15.120:0/1547935506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ded6f2370 0x560ded6f4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.338+0000 7ffbba7fc640 1 -- 172.21.15.120:0/1547935506 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ded5d93a0 con 0x560ded6e8b40 2024-09-19T09:12:24.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.338+0000 7ffbbb7fe640 1 --2- 172.21.15.120:0/3629994541 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ffba40a3240 0x7ffba40a5700 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4005ba0 tx=0x7ffbb4005940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:24.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.338+0000 7ffbba7fc640 1 --2- 172.21.15.120:0/1547935506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ded6e8b40 0x560ded6e7090 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0011ad0 tx=0x7ffbb000aa90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f27635dc13bef06f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:24.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.338+0000 7ffba17fa640 1 -- 172.21.15.120:0/1547935506 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb000fa90 con 0x560ded6e8b40 2024-09-19T09:12:24.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.338+0000 7ffba17fa640 1 -- 172.21.15.120:0/1547935506 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbb000fc30 con 0x560ded6e8b40 2024-09-19T09:12:24.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.338+0000 7ffba17fa640 1 -- 172.21.15.120:0/1547935506 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb00047a0 con 0x560ded6e8b40 2024-09-19T09:12:24.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.339+0000 7ffbc3a34740 1 -- 172.21.15.120:0/1547935506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ded6e8b40 msgr2=0x560ded6e7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:24.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.339+0000 7ffbc3a34740 1 --2- 172.21.15.120:0/1547935506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ded6e8b40 0x560ded6e7090 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0011ad0 tx=0x7ffbb000aa90 comp rx=0 tx=0).stop 2024-09-19T09:12:24.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.339+0000 7ffbc3a34740 1 -- 172.21.15.120:0/1547935506 shutdown_connections 2024-09-19T09:12:24.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.339+0000 7ffbc3a34740 1 --2- 172.21.15.120:0/1547935506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ded6f2370 0x560ded6f4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.339+0000 7ffbc3a34740 1 --2- 172.21.15.120:0/1547935506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ded6e75d0 0x560ded6e7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.339+0000 7ffbc3a34740 1 --2- 172.21.15.120:0/1547935506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ded6e8b40 0x560ded6e7090 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.339+0000 7ffbc3a34740 1 -- 172.21.15.120:0/1547935506 >> 172.21.15.120:0/1547935506 conn(0x560ded6b0310 msgr2=0x560ded6e3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:24.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.339+0000 7ffbc3a34740 1 -- 172.21.15.120:0/1547935506 shutdown_connections 2024-09-19T09:12:24.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.339+0000 7ffbc3a34740 1 -- 172.21.15.120:0/1547935506 wait complete. 2024-09-19T09:12:24.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.339+0000 7ffbc3a34740 1 -- start start 2024-09-19T09:12:24.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.339+0000 7ffbc3a34740 1 Processor -- start 2024-09-19T09:12:24.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.340+0000 7ffbc3a34740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ded6e75d0 0x560ded6f1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:24.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.340+0000 7ffbc3a34740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ded6e8b40 0x560ded6eefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:24.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.340+0000 7ffbc3a34740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ded6f2370 0x560ded6ef570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:24.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.340+0000 7ffbc3a34740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560ded6f5070 con 0x560ded6e75d0 2024-09-19T09:12:24.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.340+0000 7ffbc3a34740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560ded608060 con 0x560ded6e8b40 2024-09-19T09:12:24.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.340+0000 7ffbc3a34740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560ded6f51f0 con 0x560ded6f2370 2024-09-19T09:12:24.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.340+0000 7ffbba7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ded6e75d0 0x560ded6f1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:24.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.340+0000 7ffbc3a34740 10 client.0 _ll_register_callbacks cb 0x560ded35b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:24.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.340+0000 7ffbba7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ded6e75d0 0x560ded6f1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48096/0 (socket says 172.21.15.120:48096) 2024-09-19T09:12:24.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.340+0000 7ffbba7fc640 1 -- 172.21.15.120:0/509063459 learned_addr learned my addr 172.21.15.120:0/509063459 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:24.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.340+0000 7ffbbaffd640 1 --2- 172.21.15.120:0/509063459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ded6e8b40 0x560ded6eefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:24.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.340+0000 7ffbba7fc640 1 --2- 172.21.15.120:0/509063459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ded6f2370 0x560ded6ef570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:24.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.340+0000 7ffbba7fc640 1 -- 172.21.15.120:0/509063459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ded6f2370 msgr2=0x560ded6ef570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:24.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.340+0000 7ffbba7fc640 1 --2- 172.21.15.120:0/509063459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ded6f2370 0x560ded6ef570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.340+0000 7ffbba7fc640 1 -- 172.21.15.120:0/509063459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ded6e8b40 msgr2=0x560ded6eefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:24.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.340+0000 7ffbba7fc640 1 --2- 172.21.15.120:0/509063459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ded6e8b40 0x560ded6eefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.340+0000 7ffbba7fc640 1 -- 172.21.15.120:0/509063459 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ded3be3c0 con 0x560ded6e75d0 2024-09-19T09:12:24.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.340+0000 7ffbbaffd640 1 --2- 172.21.15.120:0/509063459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ded6e8b40 0x560ded6eefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:24.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.340+0000 7ffbba7fc640 1 --2- 172.21.15.120:0/509063459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ded6e75d0 0x560ded6f1d20 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000aa20 tx=0x7ffbb00054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789bf6e8ac705e50 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:24.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.340+0000 7ffb9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:24.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.341+0000 7ffb9eff5640 1 -- 172.21.15.120:0/509063459 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0004220 con 0x560ded6e75d0 2024-09-19T09:12:24.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.341+0000 7ffbc3a34740 1 -- 172.21.15.120:0/509063459 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbb000fc30 con 0x560ded6e75d0 2024-09-19T09:12:24.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.341+0000 7ffb9eff5640 1 -- 172.21.15.120:0/509063459 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ffbb00043c0 con 0x560ded6e75d0 2024-09-19T09:12:24.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.341+0000 7ffb9eff5640 1 -- 172.21.15.120:0/509063459 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ffbb0031050 con 0x560ded6e75d0 2024-09-19T09:12:24.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.341+0000 7ffbc3a34740 10 client.18849 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:24.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.341+0000 7ffbc3a34740 1 -- 172.21.15.120:0/509063459 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560ded700ac0 con 0x560ded6e75d0 2024-09-19T09:12:24.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.341+0000 7ffb9eff5640 1 -- 172.21.15.120:0/509063459 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbb000fc30 con 0x560ded6e75d0 2024-09-19T09:12:24.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.341+0000 7ffb9eff5640 1 -- 172.21.15.120:0/509063459 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ffbb001e6e0 con 0x560ded6e75d0 2024-09-19T09:12:24.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.341+0000 7ffbc3a34740 10 client.18849 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:24.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.341+0000 7ffbc3a34740 10 client.18849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:12:24.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.341+0000 7ffbc3a34740 20 client.18849 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:24.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.341+0000 7ffbc3a34740 1 --2- 172.21.15.120:0/509063459 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560ded701e10 0x560ded704200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:24.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.341+0000 7ffbc3a34740 4 client.18849 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:12:24.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.341+0000 7ffbc3a34740 1 -- 172.21.15.120:0/509063459 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ded65a860 con 0x560ded701e10 2024-09-19T09:12:24.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.341+0000 7ffbbaffd640 1 --2- 172.21.15.120:0/509063459 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560ded701e10 0x560ded704200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:24.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.342+0000 7ffb9eff5640 10 client.18849 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:24.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.342+0000 7ffbbaffd640 1 --2- 172.21.15.120:0/509063459 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560ded701e10 0x560ded704200 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1368686c099f946 server_cookie=9338d7e4a4eab81b in_seq=0 out_seq=0 2024-09-19T09:12:24.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.343+0000 7ffb9eff5640 1 -- 172.21.15.120:0/509063459 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560ded65a860 con 0x560ded701e10 2024-09-19T09:12:24.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.343+0000 7ffb9eff5640 10 client.18849 handle_command_reply: tid=0 2024-09-19T09:12:24.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.343+0000 7ffbc3a34740 1 client.18849 shutdown 2024-09-19T09:12:24.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.343+0000 7ffbc3a34740 1 -- 172.21.15.120:0/509063459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ded6e75d0 msgr2=0x560ded6f1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:24.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.343+0000 7ffbc3a34740 1 --2- 172.21.15.120:0/509063459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ded6e75d0 0x560ded6f1d20 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000aa20 tx=0x7ffbb00054a0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.343+0000 7ffbc3a34740 1 -- 172.21.15.120:0/509063459 shutdown_connections 2024-09-19T09:12:24.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.343+0000 7ffbc3a34740 1 --2- 172.21.15.120:0/509063459 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560ded701e10 0x560ded704200 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.343+0000 7ffbc3a34740 1 --2- 172.21.15.120:0/509063459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ded6f2370 0x560ded6ef570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.343+0000 7ffb9eff5640 0 client.18849 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:24.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.343+0000 7ffbc3a34740 1 --2- 172.21.15.120:0/509063459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ded6e8b40 0x560ded6eefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.343+0000 7ffbc3a34740 1 --2- 172.21.15.120:0/509063459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ded6e75d0 0x560ded6f1d20 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.343+0000 7ffbc3a34740 1 -- 172.21.15.120:0/509063459 >> 172.21.15.120:0/509063459 conn(0x560ded6b0310 msgr2=0x560ded6e9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:24.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.343+0000 7ffbc3a34740 1 -- 172.21.15.120:0/509063459 shutdown_connections 2024-09-19T09:12:24.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.343+0000 7ffbc3a34740 1 -- 172.21.15.120:0/509063459 wait complete. 2024-09-19T09:12:24.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.343+0000 7ffbc3a34740 20 client.18849 trim_cache size 0 max 16384 2024-09-19T09:12:24.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.357+0000 7ffbc3a34740 1 Processor -- start 2024-09-19T09:12:24.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.357+0000 7ffbc3a34740 1 -- start start 2024-09-19T09:12:24.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.357+0000 7ffbc3a34740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ded6e75d0 0x560ded6fe310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:24.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.357+0000 7ffbc3a34740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ded6e8b40 0x560ded6ee890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:24.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.357+0000 7ffbc3a34740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ded6f2370 0x560ded6edfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:24.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.357+0000 7ffbc3a34740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffbbc1ad610 con 0x560ded6e75d0 2024-09-19T09:12:24.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.357+0000 7ffbc3a34740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffbbc1af150 con 0x560ded6e8b40 2024-09-19T09:12:24.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.357+0000 7ffbc3a34740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffbbc1adce0 con 0x560ded6f2370 2024-09-19T09:12:24.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.357+0000 7ffbba7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ded6e75d0 0x560ded6fe310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:24.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.357+0000 7ffbba7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ded6e75d0 0x560ded6fe310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48112/0 (socket says 172.21.15.120:48112) 2024-09-19T09:12:24.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.357+0000 7ffbba7fc640 1 -- 172.21.15.120:0/953917491 learned_addr learned my addr 172.21.15.120:0/953917491 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:24.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.357+0000 7ffbbaffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ded6e8b40 0x560ded6ee890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:24.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.357+0000 7ffbba7fc640 1 --2- 172.21.15.120:0/953917491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ded6f2370 0x560ded6edfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:24.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.357+0000 7ffbbaffd640 1 -- 172.21.15.120:0/953917491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ded6f2370 msgr2=0x560ded6edfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:24.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.357+0000 7ffbbaffd640 1 --2- 172.21.15.120:0/953917491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ded6f2370 0x560ded6edfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.357+0000 7ffbbaffd640 1 -- 172.21.15.120:0/953917491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ded6e75d0 msgr2=0x560ded6fe310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:24.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.357+0000 7ffbbaffd640 1 --2- 172.21.15.120:0/953917491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ded6e75d0 0x560ded6fe310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.357+0000 7ffbbaffd640 1 -- 172.21.15.120:0/953917491 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ded6e8300 con 0x560ded6e8b40 2024-09-19T09:12:24.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.357+0000 7ffbba7fc640 1 --2- 172.21.15.120:0/953917491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ded6e75d0 0x560ded6fe310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:24.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.357+0000 7ffbbaffd640 1 --2- 172.21.15.120:0/953917491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ded6e8b40 0x560ded6ee890 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffbac009230 tx=0x7ffbac005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=506a06f70037c17b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:24.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.358+0000 7ffba0ff9640 1 -- 172.21.15.120:0/953917491 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac035040 con 0x560ded6e8b40 2024-09-19T09:12:24.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.358+0000 7ffba0ff9640 1 -- 172.21.15.120:0/953917491 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbac005040 con 0x560ded6e8b40 2024-09-19T09:12:24.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.358+0000 7ffba0ff9640 1 -- 172.21.15.120:0/953917491 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac004a50 con 0x560ded6e8b40 2024-09-19T09:12:24.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.358+0000 7ffbc3a34740 1 -- 172.21.15.120:0/953917491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ded6e8b40 msgr2=0x560ded6ee890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:24.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.358+0000 7ffbc3a34740 1 --2- 172.21.15.120:0/953917491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ded6e8b40 0x560ded6ee890 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffbac009230 tx=0x7ffbac005a20 comp rx=0 tx=0).stop 2024-09-19T09:12:24.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.359+0000 7ffbc3a34740 1 -- 172.21.15.120:0/953917491 shutdown_connections 2024-09-19T09:12:24.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.359+0000 7ffbc3a34740 1 --2- 172.21.15.120:0/953917491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ded6f2370 0x560ded6edfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.359+0000 7ffbc3a34740 1 --2- 172.21.15.120:0/953917491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ded6e8b40 0x560ded6ee890 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.359+0000 7ffbc3a34740 1 --2- 172.21.15.120:0/953917491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ded6e75d0 0x560ded6fe310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.359+0000 7ffbc3a34740 1 -- 172.21.15.120:0/953917491 >> 172.21.15.120:0/953917491 conn(0x560ded6b0310 msgr2=0x560ded6ed8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:24.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.359+0000 7ffbc3a34740 1 -- 172.21.15.120:0/953917491 shutdown_connections 2024-09-19T09:12:24.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.359+0000 7ffbc3a34740 1 -- 172.21.15.120:0/953917491 wait complete. 2024-09-19T09:12:24.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.359+0000 7ffbc3a34740 1 -- start start 2024-09-19T09:12:24.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.359+0000 7ffbc3a34740 1 Processor -- start 2024-09-19T09:12:24.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.359+0000 7ffbc3a34740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ded6e75d0 0x560ded769010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:24.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.359+0000 7ffbc3a34740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ded6e8b40 0x560ded769550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:24.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.359+0000 7ffbc3a34740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ded6f2370 0x560ded6f5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:24.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.359+0000 7ffbc3a34740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffbbc1aea80 con 0x560ded6e8b40 2024-09-19T09:12:24.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.359+0000 7ffbc3a34740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffbbc1ae3b0 con 0x560ded6e75d0 2024-09-19T09:12:24.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.359+0000 7ffbc3a34740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffbbc1ab5d0 con 0x560ded6f2370 2024-09-19T09:12:24.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.359+0000 7ffbc3a34740 10 client.0 _ll_register_callbacks cb 0x7ffbb00115c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:24.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.359+0000 7ffbbaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ded6e8b40 0x560ded769550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:24.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.359+0000 7ffbba7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ded6e75d0 0x560ded769010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:24.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.359+0000 7ffbbaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ded6e8b40 0x560ded769550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48118/0 (socket says 172.21.15.120:48118) 2024-09-19T09:12:24.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.359+0000 7ffbba7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ded6e75d0 0x560ded769010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:48282/0 (socket says 172.21.15.120:48282) 2024-09-19T09:12:24.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.360+0000 7ffbbaffd640 1 -- 172.21.15.120:0/717441674 learned_addr learned my addr 172.21.15.120:0/717441674 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:24.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.360+0000 7ffbba7fc640 1 --2- 172.21.15.120:0/717441674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ded6f2370 0x560ded6f5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:24.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.360+0000 7ffbbaffd640 1 -- 172.21.15.120:0/717441674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ded6f2370 msgr2=0x560ded6f5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:24.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.360+0000 7ffbbaffd640 1 --2- 172.21.15.120:0/717441674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ded6f2370 0x560ded6f5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.360+0000 7ffbbaffd640 1 -- 172.21.15.120:0/717441674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ded6e75d0 msgr2=0x560ded769010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:24.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.360+0000 7ffbbaffd640 1 --2- 172.21.15.120:0/717441674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ded6e75d0 0x560ded769010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.360+0000 7ffbbaffd640 1 -- 172.21.15.120:0/717441674 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ded6f60f0 con 0x560ded6e8b40 2024-09-19T09:12:24.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.360+0000 7ffbbaffd640 1 --2- 172.21.15.120:0/717441674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ded6e8b40 0x560ded769550 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7ffbac031550 tx=0x7ffbac004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1436fc787b16fab8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:24.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.360+0000 7ffb9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:24.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.360+0000 7ffb9eff5640 1 -- 172.21.15.120:0/717441674 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac035040 con 0x560ded6e8b40 2024-09-19T09:12:24.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.360+0000 7ffb9eff5640 1 -- 172.21.15.120:0/717441674 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ffbac0051f0 con 0x560ded6e8b40 2024-09-19T09:12:24.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.360+0000 7ffbc3a34740 1 -- 172.21.15.120:0/717441674 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbac005040 con 0x560ded6e8b40 2024-09-19T09:12:24.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.360+0000 7ffb9eff5640 1 -- 172.21.15.120:0/717441674 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ffbac0053f0 con 0x560ded6e8b40 2024-09-19T09:12:24.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.360+0000 7ffbc3a34740 10 client.18855 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:24.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.360+0000 7ffbc3a34740 1 -- 172.21.15.120:0/717441674 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560ded770810 con 0x560ded6e8b40 2024-09-19T09:12:24.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.361+0000 7ffb9eff5640 1 -- 172.21.15.120:0/717441674 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbac005040 con 0x560ded6e8b40 2024-09-19T09:12:24.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.361+0000 7ffb9eff5640 1 -- 172.21.15.120:0/717441674 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ffbac003840 con 0x560ded6e8b40 2024-09-19T09:12:24.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.361+0000 7ffbc3a34740 10 client.18855 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:24.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.361+0000 7ffbc3a34740 10 client.18855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:12:24.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.361+0000 7ffbc3a34740 20 client.18855 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:24.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.361+0000 7ffbc3a34740 1 --2- 172.21.15.120:0/717441674 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560ded771010 0x560ded773430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:24.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.361+0000 7ffbc3a34740 4 client.18855 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:12:24.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.361+0000 7ffbc3a34740 1 -- 172.21.15.120:0/717441674 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560ded654d70 con 0x560ded771010 2024-09-19T09:12:24.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.361+0000 7ffbba7fc640 1 --2- 172.21.15.120:0/717441674 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560ded771010 0x560ded773430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:24.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.361+0000 7ffbba7fc640 1 --2- 172.21.15.120:0/717441674 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560ded771010 0x560ded773430 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=370e4e643c421aa server_cookie=254c7a90ead75e5 in_seq=0 out_seq=0 2024-09-19T09:12:24.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.361+0000 7ffb9eff5640 10 client.18855 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:24.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.362+0000 7ffb9eff5640 1 -- 172.21.15.120:0/717441674 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560ded654d70 con 0x560ded771010 2024-09-19T09:12:24.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.362+0000 7ffb9eff5640 10 client.18855 handle_command_reply: tid=0 2024-09-19T09:12:24.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.362+0000 7ffbc3a34740 1 client.18855 shutdown 2024-09-19T09:12:24.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.362+0000 7ffbc3a34740 1 -- 172.21.15.120:0/717441674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ded6e8b40 msgr2=0x560ded769550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:24.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.362+0000 7ffbc3a34740 1 --2- 172.21.15.120:0/717441674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ded6e8b40 0x560ded769550 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7ffbac031550 tx=0x7ffbac004040 comp rx=0 tx=0).stop 2024-09-19T09:12:24.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.362+0000 7ffbc3a34740 1 -- 172.21.15.120:0/717441674 shutdown_connections 2024-09-19T09:12:24.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.362+0000 7ffbc3a34740 1 --2- 172.21.15.120:0/717441674 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560ded771010 0x560ded773430 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.362+0000 7ffbc3a34740 1 --2- 172.21.15.120:0/717441674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ded6f2370 0x560ded6f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.362+0000 7ffbc3a34740 1 --2- 172.21.15.120:0/717441674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ded6e8b40 0x560ded769550 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.362+0000 7ffb9eff5640 0 client.18855 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:24.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.362+0000 7ffbc3a34740 1 --2- 172.21.15.120:0/717441674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560ded6e75d0 0x560ded769010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.362+0000 7ffbc3a34740 1 -- 172.21.15.120:0/717441674 >> 172.21.15.120:0/717441674 conn(0x560ded6b0310 msgr2=0x560ded6f0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:24.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.362+0000 7ffbc3a34740 1 -- 172.21.15.120:0/717441674 shutdown_connections 2024-09-19T09:12:24.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.362+0000 7ffbc3a34740 1 -- 172.21.15.120:0/717441674 wait complete. 2024-09-19T09:12:24.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.362+0000 7ffbc3a34740 20 client.18855 trim_cache size 0 max 16384 2024-09-19T09:12:24.378 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:12:24.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.362+0000 7ffbc165b640 1 -- 172.21.15.120:0/3629994541 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ffba40a3240 msgr2=0x7ffba40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:24.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.362+0000 7ffbc165b640 1 --2- 172.21.15.120:0/3629994541 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ffba40a3240 0x7ffba40a5700 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4005ba0 tx=0x7ffbb4005940 comp rx=0 tx=0).stop 2024-09-19T09:12:24.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.362+0000 7ffbc165b640 1 -- 172.21.15.120:0/3629994541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbbc106f20 msgr2=0x7ffbbc19cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:24.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.362+0000 7ffbc165b640 1 --2- 172.21.15.120:0/3629994541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbbc106f20 0x7ffbbc19cd10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4009fd0 tx=0x7ffbb400ef10 comp rx=0 tx=0).stop 2024-09-19T09:12:24.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.363+0000 7ffbc165b640 1 -- 172.21.15.120:0/3629994541 shutdown_connections 2024-09-19T09:12:24.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.363+0000 7ffbc165b640 1 --2- 172.21.15.120:0/3629994541 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ffba40a3240 0x7ffba40a5700 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.363+0000 7ffbc165b640 1 --2- 172.21.15.120:0/3629994541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbbc106f20 0x7ffbbc19cd10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.363+0000 7ffbc165b640 1 --2- 172.21.15.120:0/3629994541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffbbc1027a0 0x7ffbbc198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.363+0000 7ffbc165b640 1 --2- 172.21.15.120:0/3629994541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbbc101df0 0x7ffbbc198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:24.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.363+0000 7ffbc165b640 1 -- 172.21.15.120:0/3629994541 >> 172.21.15.120:0/3629994541 conn(0x7ffbbc0fd8b0 msgr2=0x7ffbbc0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:24.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.363+0000 7ffbc165b640 1 -- 172.21.15.120:0/3629994541 shutdown_connections 2024-09-19T09:12:24.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:24.363+0000 7ffbc165b640 1 -- 172.21.15.120:0/3629994541 wait complete. 2024-09-19T09:12:25.066 INFO:tasks.workunit.client.0.smithi120.stdout: 1 927792 10.03 MB/sec execute 380 sec latency 11.760 ms 2024-09-19T09:12:25.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:24 smithi196 ceph-mon[35150]: pgmap v629: 97 pgs: 97 active+clean; 33 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 424 op/s 2024-09-19T09:12:25.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:24 smithi153 ceph-mon[35071]: pgmap v629: 97 pgs: 97 active+clean; 33 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 424 op/s 2024-09-19T09:12:25.294 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:24 smithi120 ceph-mon[31150]: pgmap v629: 97 pgs: 97 active+clean; 33 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 424 op/s 2024-09-19T09:12:26.066 INFO:tasks.workunit.client.0.smithi120.stdout: 1 931119 10.05 MB/sec execute 381 sec latency 16.832 ms 2024-09-19T09:12:26.324 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:12:26.325 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:12:26.611 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:12:26.984 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:26 smithi120 ceph-mon[31150]: pgmap v630: 97 pgs: 97 active+clean; 30 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 376 op/s 2024-09-19T09:12:27.066 INFO:tasks.workunit.client.0.smithi120.stdout: 1 933649 10.05 MB/sec execute 382 sec latency 60.732 ms 2024-09-19T09:12:27.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:26 smithi196 ceph-mon[35150]: pgmap v630: 97 pgs: 97 active+clean; 30 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 376 op/s 2024-09-19T09:12:27.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:26 smithi153 ceph-mon[35071]: pgmap v630: 97 pgs: 97 active+clean; 30 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 376 op/s 2024-09-19T09:12:27.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.195+0000 7f5c6b040640 1 -- 172.21.15.120:0/2491681076 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c64102570 msgr2=0x7f5c641029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:27.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.195+0000 7f5c6b040640 1 --2- 172.21.15.120:0/2491681076 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c64102570 0x7f5c641029b0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009900 tx=0x7f5c5002a480 comp rx=0 tx=0).stop 2024-09-19T09:12:27.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.195+0000 7f5c6b040640 1 -- 172.21.15.120:0/2491681076 shutdown_connections 2024-09-19T09:12:27.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.195+0000 7f5c6b040640 1 --2- 172.21.15.120:0/2491681076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c64102ef0 0x7f5c6410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.195+0000 7f5c6b040640 1 --2- 172.21.15.120:0/2491681076 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c64102570 0x7f5c641029b0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.195+0000 7f5c6b040640 1 --2- 172.21.15.120:0/2491681076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c64100e20 0x7f5c64101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.195+0000 7f5c6b040640 1 -- 172.21.15.120:0/2491681076 >> 172.21.15.120:0/2491681076 conn(0x7f5c640fc7d0 msgr2=0x7f5c640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:27.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.195+0000 7f5c6b040640 1 -- 172.21.15.120:0/2491681076 shutdown_connections 2024-09-19T09:12:27.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.195+0000 7f5c6b040640 1 -- 172.21.15.120:0/2491681076 wait complete. 2024-09-19T09:12:27.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.196+0000 7f5c6b040640 1 Processor -- start 2024-09-19T09:12:27.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.196+0000 7f5c6b040640 1 -- start start 2024-09-19T09:12:27.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.196+0000 7f5c6b040640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c64100e20 0x7f5c64199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:27.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.196+0000 7f5c6b040640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c64102570 0x7f5c64199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:27.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.196+0000 7f5c6b040640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c64102ef0 0x7f5c641a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:27.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.196+0000 7f5c6b040640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5c64110f80 con 0x7f5c64102570 2024-09-19T09:12:27.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.196+0000 7f5c6b040640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5c64110e00 con 0x7f5c64102ef0 2024-09-19T09:12:27.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.196+0000 7f5c6b040640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5c64111100 con 0x7f5c64100e20 2024-09-19T09:12:27.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.196+0000 7f5c63fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c64102570 0x7f5c64199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:27.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.196+0000 7f5c68ddc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c64100e20 0x7f5c64199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:27.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.196+0000 7f5c63fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c64102570 0x7f5c64199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48150/0 (socket says 172.21.15.120:48150) 2024-09-19T09:12:27.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.196+0000 7f5c63fff640 1 -- 172.21.15.120:0/4089168014 learned_addr learned my addr 172.21.15.120:0/4089168014 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:27.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.196+0000 7f5c695dd640 1 --2- 172.21.15.120:0/4089168014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c64102ef0 0x7f5c641a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:27.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.196+0000 7f5c68ddc640 1 -- 172.21.15.120:0/4089168014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c64102ef0 msgr2=0x7f5c641a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:27.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.196+0000 7f5c68ddc640 1 --2- 172.21.15.120:0/4089168014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c64102ef0 0x7f5c641a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.196+0000 7f5c68ddc640 1 -- 172.21.15.120:0/4089168014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c64102570 msgr2=0x7f5c64199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:27.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.196+0000 7f5c68ddc640 1 --2- 172.21.15.120:0/4089168014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c64102570 0x7f5c64199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.196+0000 7f5c68ddc640 1 -- 172.21.15.120:0/4089168014 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c641a48d0 con 0x7f5c64100e20 2024-09-19T09:12:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.196+0000 7f5c63fff640 1 --2- 172.21.15.120:0/4089168014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c64102570 0x7f5c64199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.197+0000 7f5c68ddc640 1 --2- 172.21.15.120:0/4089168014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c64100e20 0x7f5c64199310 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5c5800e9f0 tx=0x7f5c5800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.197+0000 7f5c61ffb640 1 -- 172.21.15.120:0/4089168014 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5800ccc0 con 0x7f5c64100e20 2024-09-19T09:12:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.197+0000 7f5c6b040640 1 -- 172.21.15.120:0/4089168014 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c641a4aa0 con 0x7f5c64100e20 2024-09-19T09:12:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.197+0000 7f5c61ffb640 1 -- 172.21.15.120:0/4089168014 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c58005020 con 0x7f5c64100e20 2024-09-19T09:12:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.197+0000 7f5c61ffb640 1 -- 172.21.15.120:0/4089168014 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c580049e0 con 0x7f5c64100e20 2024-09-19T09:12:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.197+0000 7f5c6b040640 1 -- 172.21.15.120:0/4089168014 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c641a4fe0 con 0x7f5c64100e20 2024-09-19T09:12:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.199+0000 7f5c61ffb640 1 -- 172.21.15.120:0/4089168014 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5c58005560 con 0x7f5c64100e20 2024-09-19T09:12:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.200+0000 7f5c6d419740 1 Processor -- start 2024-09-19T09:12:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.200+0000 7f5c6d419740 1 -- start start 2024-09-19T09:12:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.200+0000 7f5c61ffb640 1 --2- 172.21.15.120:0/4089168014 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5c540a3240 0x7f5c540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.200+0000 7f5c6d419740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b3a2d3b60 0x558b3a2d3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.200+0000 7f5c6d419740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3a2d4480 0x558b3a2de800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.200+0000 7f5c6d419740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b3a2ded40 0x558b3a2e1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.200+0000 7f5c6d419740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558b3a239280 con 0x558b3a2d4480 2024-09-19T09:12:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.200+0000 7f5c6d419740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558b3a1e0b10 con 0x558b3a2ded40 2024-09-19T09:12:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.200+0000 7f5c6d419740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558b3a235240 con 0x558b3a2d3b60 2024-09-19T09:12:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.200+0000 7f5c63fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3a2d4480 0x558b3a2de800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.200+0000 7f5c61ffb640 1 -- 172.21.15.120:0/4089168014 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f5c580cf8d0 con 0x7f5c64100e20 2024-09-19T09:12:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.200+0000 7f5c695dd640 1 --2- 172.21.15.120:0/4089168014 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5c540a3240 0x7f5c540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.200+0000 7f5c68ddc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b3a2d3b60 0x558b3a2d3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.200+0000 7f5c63fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3a2d4480 0x558b3a2de800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48166/0 (socket says 172.21.15.120:48166) 2024-09-19T09:12:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.200+0000 7f5c68ddc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b3a2d3b60 0x558b3a2d3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:48104/0 (socket says 172.21.15.120:48104) 2024-09-19T09:12:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.200+0000 7f5c63fff640 1 -- 172.21.15.120:0/64837052 learned_addr learned my addr 172.21.15.120:0/64837052 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.200+0000 7f5c695dd640 1 --2- 172.21.15.120:0/64837052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b3a2ded40 0x558b3a2e1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.200+0000 7f5c68ddc640 1 -- 172.21.15.120:0/64837052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b3a2ded40 msgr2=0x558b3a2e1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.200+0000 7f5c68ddc640 1 --2- 172.21.15.120:0/64837052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b3a2ded40 0x558b3a2e1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.200+0000 7f5c68ddc640 1 -- 172.21.15.120:0/64837052 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3a2d4480 msgr2=0x558b3a2de800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.200+0000 7f5c68ddc640 1 --2- 172.21.15.120:0/64837052 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3a2d4480 0x558b3a2de800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.200+0000 7f5c68ddc640 1 -- 172.21.15.120:0/64837052 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b3a1ed1f0 con 0x558b3a2d3b60 2024-09-19T09:12:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.200+0000 7f5c63fff640 1 --2- 172.21.15.120:0/64837052 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3a2d4480 0x558b3a2de800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.201+0000 7f5c695dd640 1 --2- 172.21.15.120:0/4089168014 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5c540a3240 0x7f5c540a5700 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c005fd0 tx=0x7f5c5c011b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.201+0000 7f5c68ddc640 1 --2- 172.21.15.120:0/64837052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b3a2d3b60 0x558b3a2d3f40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5c5800c6a0 tx=0x7f5c5809bdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=670f111300195a73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.201+0000 7f5c4effd640 1 -- 172.21.15.120:0/64837052 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5801c070 con 0x558b3a2d3b60 2024-09-19T09:12:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.201+0000 7f5c4effd640 1 -- 172.21.15.120:0/64837052 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c5809beb0 con 0x558b3a2d3b60 2024-09-19T09:12:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.201+0000 7f5c4effd640 1 -- 172.21.15.120:0/64837052 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c58098520 con 0x558b3a2d3b60 2024-09-19T09:12:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.202+0000 7f5c6d419740 1 -- 172.21.15.120:0/64837052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b3a2d3b60 msgr2=0x558b3a2d3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.202+0000 7f5c6d419740 1 --2- 172.21.15.120:0/64837052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b3a2d3b60 0x558b3a2d3f40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5c5800c6a0 tx=0x7f5c5809bdb0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.202+0000 7f5c6d419740 1 -- 172.21.15.120:0/64837052 shutdown_connections 2024-09-19T09:12:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.202+0000 7f5c6d419740 1 --2- 172.21.15.120:0/64837052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b3a2ded40 0x558b3a2e1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.202+0000 7f5c6d419740 1 --2- 172.21.15.120:0/64837052 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3a2d4480 0x558b3a2de800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.202+0000 7f5c6d419740 1 --2- 172.21.15.120:0/64837052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b3a2d3b60 0x558b3a2d3f40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.202+0000 7f5c6d419740 1 -- 172.21.15.120:0/64837052 >> 172.21.15.120:0/64837052 conn(0x558b3a2d00f0 msgr2=0x558b3a2d04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:27.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.202+0000 7f5c6d419740 1 -- 172.21.15.120:0/64837052 shutdown_connections 2024-09-19T09:12:27.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.202+0000 7f5c6d419740 1 -- 172.21.15.120:0/64837052 wait complete. 2024-09-19T09:12:27.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.202+0000 7f5c6d419740 1 -- start start 2024-09-19T09:12:27.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.202+0000 7f5c6d419740 1 Processor -- start 2024-09-19T09:12:27.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.203+0000 7f5c6d419740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b3a2d3b60 0x558b3a2dbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:27.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.203+0000 7f5c6d419740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b3a2d4480 0x558b3a2dc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:27.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.203+0000 7f5c6d419740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3a2ded40 0x558b3a2dc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:27.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.203+0000 7f5c6d419740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558b3a2e1910 con 0x558b3a2ded40 2024-09-19T09:12:27.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.203+0000 7f5c6d419740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558b3a1f3f10 con 0x558b3a2d3b60 2024-09-19T09:12:27.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.203+0000 7f5c6d419740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558b3a2e1a90 con 0x558b3a2d4480 2024-09-19T09:12:27.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.203+0000 7f5c6d419740 10 client.0 _ll_register_callbacks cb 0x558b3a1bca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:27.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.203+0000 7f5c695dd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3a2ded40 0x558b3a2dc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:27.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.203+0000 7f5c695dd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3a2ded40 0x558b3a2dc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48168/0 (socket says 172.21.15.120:48168) 2024-09-19T09:12:27.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.203+0000 7f5c695dd640 1 -- 172.21.15.120:0/4035950151 learned_addr learned my addr 172.21.15.120:0/4035950151 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:27.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.203+0000 7f5c68ddc640 1 --2- 172.21.15.120:0/4035950151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b3a2d3b60 0x558b3a2dbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:27.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.203+0000 7f5c63fff640 1 --2- 172.21.15.120:0/4035950151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b3a2d4480 0x558b3a2dc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:27.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.203+0000 7f5c695dd640 1 -- 172.21.15.120:0/4035950151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b3a2d4480 msgr2=0x558b3a2dc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:27.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.203+0000 7f5c695dd640 1 --2- 172.21.15.120:0/4035950151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b3a2d4480 0x558b3a2dc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.203+0000 7f5c695dd640 1 -- 172.21.15.120:0/4035950151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b3a2d3b60 msgr2=0x558b3a2dbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.203+0000 7f5c695dd640 1 --2- 172.21.15.120:0/4035950151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b3a2d3b60 0x558b3a2dbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.203+0000 7f5c695dd640 1 -- 172.21.15.120:0/4035950151 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b39faa6a0 con 0x558b3a2ded40 2024-09-19T09:12:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.203+0000 7f5c68ddc640 1 --2- 172.21.15.120:0/4035950151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b3a2d3b60 0x558b3a2dbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.203+0000 7f5c695dd640 1 --2- 172.21.15.120:0/4035950151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3a2ded40 0x558b3a2dc700 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00fbb0 tx=0x7f5c5c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2868183cf610d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.203+0000 7f5c43fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.204+0000 7f5c43fff640 1 -- 172.21.15.120:0/4035950151 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c01a040 con 0x558b3a2ded40 2024-09-19T09:12:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.204+0000 7f5c43fff640 1 -- 172.21.15.120:0/4035950151 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f5c5c0045e0 con 0x558b3a2ded40 2024-09-19T09:12:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.204+0000 7f5c6d419740 1 -- 172.21.15.120:0/4035950151 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c5809beb0 con 0x558b3a2ded40 2024-09-19T09:12:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.204+0000 7f5c43fff640 1 -- 172.21.15.120:0/4035950151 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5c5c0047e0 con 0x558b3a2ded40 2024-09-19T09:12:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.204+0000 7f5c6d419740 10 client.18867 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.204+0000 7f5c6d419740 1 -- 172.21.15.120:0/4035950151 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558b3a2ed570 con 0x558b3a2ded40 2024-09-19T09:12:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.204+0000 7f5c43fff640 1 -- 172.21.15.120:0/4035950151 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c5809beb0 con 0x558b3a2ded40 2024-09-19T09:12:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.204+0000 7f5c43fff640 1 -- 172.21.15.120:0/4035950151 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f5c5c028590 con 0x558b3a2ded40 2024-09-19T09:12:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.204+0000 7f5c6d419740 10 client.18867 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.205+0000 7f5c6d419740 10 client.18867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:12:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.205+0000 7f5c6d419740 20 client.18867 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.205+0000 7f5c6d419740 1 --2- 172.21.15.120:0/4035950151 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558b3a2eea30 0x558b3a2f0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.205+0000 7f5c6d419740 4 client.18867 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:12:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.205+0000 7f5c6d419740 1 -- 172.21.15.120:0/4035950151 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b3a2467f0 con 0x558b3a2eea30 2024-09-19T09:12:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.205+0000 7f5c68ddc640 1 --2- 172.21.15.120:0/4035950151 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558b3a2eea30 0x558b3a2f0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.205+0000 7f5c43fff640 10 client.18867 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.205+0000 7f5c68ddc640 1 --2- 172.21.15.120:0/4035950151 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558b3a2eea30 0x558b3a2f0e40 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f637861195c298a server_cookie=fbf08ec06c9999b5 in_seq=0 out_seq=0 2024-09-19T09:12:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.207+0000 7f5c43fff640 1 -- 172.21.15.120:0/4035950151 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558b3a2467f0 con 0x558b3a2eea30 2024-09-19T09:12:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.207+0000 7f5c43fff640 10 client.18867 handle_command_reply: tid=0 2024-09-19T09:12:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.207+0000 7f5c6d419740 1 client.18867 shutdown 2024-09-19T09:12:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.207+0000 7f5c6d419740 1 -- 172.21.15.120:0/4035950151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3a2ded40 msgr2=0x558b3a2dc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.207+0000 7f5c6d419740 1 --2- 172.21.15.120:0/4035950151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3a2ded40 0x558b3a2dc700 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00fbb0 tx=0x7f5c5c004ab0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.207+0000 7f5c6d419740 1 -- 172.21.15.120:0/4035950151 shutdown_connections 2024-09-19T09:12:27.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.207+0000 7f5c6d419740 1 --2- 172.21.15.120:0/4035950151 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558b3a2eea30 0x558b3a2f0e40 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.207+0000 7f5c6d419740 1 --2- 172.21.15.120:0/4035950151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3a2ded40 0x558b3a2dc700 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.207+0000 7f5c6d419740 1 --2- 172.21.15.120:0/4035950151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b3a2d4480 0x558b3a2dc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.207+0000 7f5c6d419740 1 --2- 172.21.15.120:0/4035950151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b3a2d3b60 0x558b3a2dbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.207+0000 7f5c43fff640 0 client.18867 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:27.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.207+0000 7f5c6d419740 1 -- 172.21.15.120:0/4035950151 >> 172.21.15.120:0/4035950151 conn(0x558b3a2d00f0 msgr2=0x558b3a2cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:27.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.207+0000 7f5c6d419740 1 -- 172.21.15.120:0/4035950151 shutdown_connections 2024-09-19T09:12:27.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.207+0000 7f5c6d419740 1 -- 172.21.15.120:0/4035950151 wait complete. 2024-09-19T09:12:27.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.207+0000 7f5c6d419740 20 client.18867 trim_cache size 0 max 16384 2024-09-19T09:12:27.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.225+0000 7f5c6d419740 1 Processor -- start 2024-09-19T09:12:27.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.225+0000 7f5c6d419740 1 -- start start 2024-09-19T09:12:27.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.225+0000 7f5c6d419740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b3a2d3b60 0x558b3a2ddae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:27.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.225+0000 7f5c6d419740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b3a2d4480 0x558b3a2f6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:27.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.225+0000 7f5c6d419740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3a2ded40 0x558b3a2f6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:27.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.225+0000 7f5c6d419740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5c641ab350 con 0x558b3a2ded40 2024-09-19T09:12:27.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.225+0000 7f5c6d419740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5c641ace90 con 0x558b3a2d4480 2024-09-19T09:12:27.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.225+0000 7f5c6d419740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5c641aba20 con 0x558b3a2d3b60 2024-09-19T09:12:27.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.225+0000 7f5c695dd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3a2ded40 0x558b3a2f6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:27.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.225+0000 7f5c695dd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3a2ded40 0x558b3a2f6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48176/0 (socket says 172.21.15.120:48176) 2024-09-19T09:12:27.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.225+0000 7f5c695dd640 1 -- 172.21.15.120:0/4027322496 learned_addr learned my addr 172.21.15.120:0/4027322496 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:27.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.225+0000 7f5c63fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b3a2d4480 0x558b3a2f6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:27.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.225+0000 7f5c68ddc640 1 --2- 172.21.15.120:0/4027322496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b3a2d3b60 0x558b3a2ddae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:27.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.225+0000 7f5c695dd640 1 -- 172.21.15.120:0/4027322496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b3a2d3b60 msgr2=0x558b3a2ddae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:27.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.226+0000 7f5c695dd640 1 --2- 172.21.15.120:0/4027322496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b3a2d3b60 0x558b3a2ddae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.226+0000 7f5c695dd640 1 -- 172.21.15.120:0/4027322496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b3a2d4480 msgr2=0x558b3a2f6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:27.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.226+0000 7f5c695dd640 1 --2- 172.21.15.120:0/4027322496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b3a2d4480 0x558b3a2f6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.226+0000 7f5c695dd640 1 -- 172.21.15.120:0/4027322496 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b3a2dac30 con 0x558b3a2ded40 2024-09-19T09:12:27.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.226+0000 7f5c695dd640 1 --2- 172.21.15.120:0/4027322496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3a2ded40 0x558b3a2f6c50 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00ffd0 tx=0x7f5c5c009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2affb94349e3942 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:27.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.226+0000 7f5c4e7fc640 1 -- 172.21.15.120:0/4027322496 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c01a030 con 0x558b3a2ded40 2024-09-19T09:12:27.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.226+0000 7f5c4e7fc640 1 -- 172.21.15.120:0/4027322496 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c5c018040 con 0x558b3a2ded40 2024-09-19T09:12:27.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.226+0000 7f5c4e7fc640 1 -- 172.21.15.120:0/4027322496 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c033660 con 0x558b3a2ded40 2024-09-19T09:12:27.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.227+0000 7f5c6d419740 1 -- 172.21.15.120:0/4027322496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3a2ded40 msgr2=0x558b3a2f6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:27.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.227+0000 7f5c6d419740 1 --2- 172.21.15.120:0/4027322496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3a2ded40 0x558b3a2f6c50 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00ffd0 tx=0x7f5c5c009630 comp rx=0 tx=0).stop 2024-09-19T09:12:27.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.227+0000 7f5c6d419740 1 -- 172.21.15.120:0/4027322496 shutdown_connections 2024-09-19T09:12:27.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.227+0000 7f5c6d419740 1 --2- 172.21.15.120:0/4027322496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3a2ded40 0x558b3a2f6c50 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.227+0000 7f5c6d419740 1 --2- 172.21.15.120:0/4027322496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b3a2d4480 0x558b3a2f6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.227+0000 7f5c6d419740 1 --2- 172.21.15.120:0/4027322496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b3a2d3b60 0x558b3a2ddae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.227+0000 7f5c6d419740 1 -- 172.21.15.120:0/4027322496 >> 172.21.15.120:0/4027322496 conn(0x558b3a2d00f0 msgr2=0x558b3a2dd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:27.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.227+0000 7f5c6d419740 1 -- 172.21.15.120:0/4027322496 shutdown_connections 2024-09-19T09:12:27.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.227+0000 7f5c6d419740 1 -- 172.21.15.120:0/4027322496 wait complete. 2024-09-19T09:12:27.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.227+0000 7f5c6d419740 1 -- start start 2024-09-19T09:12:27.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.227+0000 7f5c6d419740 1 Processor -- start 2024-09-19T09:12:27.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.227+0000 7f5c6d419740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b3a2d3b60 0x558b3a2e2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:27.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.228+0000 7f5c6d419740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b3a2d4480 0x558b3a2e3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:27.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.228+0000 7f5c6d419740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3a2ded40 0x558b3a2e3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:27.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.228+0000 7f5c6d419740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5c641ac7c0 con 0x558b3a2ded40 2024-09-19T09:12:27.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.228+0000 7f5c6d419740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5c641ac0f0 con 0x558b3a2d3b60 2024-09-19T09:12:27.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.228+0000 7f5c6d419740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5c641a9310 con 0x558b3a2d4480 2024-09-19T09:12:27.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.228+0000 7f5c6d419740 10 client.0 _ll_register_callbacks cb 0x7f5c5c009bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:27.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.228+0000 7f5c695dd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3a2ded40 0x558b3a2e3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:27.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.228+0000 7f5c695dd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3a2ded40 0x558b3a2e3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48188/0 (socket says 172.21.15.120:48188) 2024-09-19T09:12:27.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.228+0000 7f5c695dd640 1 -- 172.21.15.120:0/795198960 learned_addr learned my addr 172.21.15.120:0/795198960 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:27.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.228+0000 7f5c68ddc640 1 --2- 172.21.15.120:0/795198960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b3a2d3b60 0x558b3a2e2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:27.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.228+0000 7f5c63fff640 1 --2- 172.21.15.120:0/795198960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b3a2d4480 0x558b3a2e3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:27.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.228+0000 7f5c695dd640 1 -- 172.21.15.120:0/795198960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b3a2d4480 msgr2=0x558b3a2e3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:27.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.228+0000 7f5c695dd640 1 --2- 172.21.15.120:0/795198960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b3a2d4480 0x558b3a2e3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.228+0000 7f5c695dd640 1 -- 172.21.15.120:0/795198960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b3a2d3b60 msgr2=0x558b3a2e2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:27.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.228+0000 7f5c695dd640 1 --2- 172.21.15.120:0/795198960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b3a2d3b60 0x558b3a2e2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.228+0000 7f5c695dd640 1 -- 172.21.15.120:0/795198960 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b3a358980 con 0x558b3a2ded40 2024-09-19T09:12:27.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.228+0000 7f5c68ddc640 1 --2- 172.21.15.120:0/795198960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b3a2d3b60 0x558b3a2e2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:27.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.228+0000 7f5c695dd640 1 --2- 172.21.15.120:0/795198960 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3a2ded40 0x558b3a2e3780 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c028a30 tx=0x7f5c5c0288b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb41a01435e6974a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:27.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.228+0000 7f5c43fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:27.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.228+0000 7f5c43fff640 1 -- 172.21.15.120:0/795198960 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c01a040 con 0x558b3a2ded40 2024-09-19T09:12:27.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.228+0000 7f5c43fff640 1 -- 172.21.15.120:0/795198960 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f5c5c0043e0 con 0x558b3a2ded40 2024-09-19T09:12:27.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.228+0000 7f5c6d419740 1 -- 172.21.15.120:0/795198960 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c5c018040 con 0x558b3a2ded40 2024-09-19T09:12:27.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.229+0000 7f5c43fff640 1 -- 172.21.15.120:0/795198960 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5c5c00fa40 con 0x558b3a2ded40 2024-09-19T09:12:27.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.229+0000 7f5c6d419740 10 client.18873 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:27.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.229+0000 7f5c6d419740 1 -- 172.21.15.120:0/795198960 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558b3a359cc0 con 0x558b3a2ded40 2024-09-19T09:12:27.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.229+0000 7f5c43fff640 1 -- 172.21.15.120:0/795198960 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c5c018040 con 0x558b3a2ded40 2024-09-19T09:12:27.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.229+0000 7f5c43fff640 1 -- 172.21.15.120:0/795198960 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f5c5c005670 con 0x558b3a2ded40 2024-09-19T09:12:27.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.229+0000 7f5c6d419740 10 client.18873 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:27.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.229+0000 7f5c6d419740 10 client.18873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:12:27.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.229+0000 7f5c6d419740 20 client.18873 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:27.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.229+0000 7f5c6d419740 1 --2- 172.21.15.120:0/795198960 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558b3a35a2c0 0x558b3a35c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:27.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.229+0000 7f5c6d419740 4 client.18873 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:12:27.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.229+0000 7f5c6d419740 1 -- 172.21.15.120:0/795198960 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558b3a240d00 con 0x558b3a35a2c0 2024-09-19T09:12:27.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.229+0000 7f5c68ddc640 1 --2- 172.21.15.120:0/795198960 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558b3a35a2c0 0x558b3a35c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:27.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.230+0000 7f5c68ddc640 1 --2- 172.21.15.120:0/795198960 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558b3a35a2c0 0x558b3a35c6b0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e93fbb5e09510490 server_cookie=3cbdff0cc870225d in_seq=0 out_seq=0 2024-09-19T09:12:27.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.230+0000 7f5c43fff640 10 client.18873 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:27.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.233+0000 7f5c43fff640 1 -- 172.21.15.120:0/795198960 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558b3a240d00 con 0x558b3a35a2c0 2024-09-19T09:12:27.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.233+0000 7f5c43fff640 10 client.18873 handle_command_reply: tid=0 2024-09-19T09:12:27.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.233+0000 7f5c6d419740 1 client.18873 shutdown 2024-09-19T09:12:27.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.233+0000 7f5c6d419740 1 -- 172.21.15.120:0/795198960 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3a2ded40 msgr2=0x558b3a2e3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:27.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.233+0000 7f5c6d419740 1 --2- 172.21.15.120:0/795198960 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3a2ded40 0x558b3a2e3780 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c028a30 tx=0x7f5c5c0288b0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.233+0000 7f5c6d419740 1 -- 172.21.15.120:0/795198960 shutdown_connections 2024-09-19T09:12:27.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.233+0000 7f5c6d419740 1 --2- 172.21.15.120:0/795198960 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558b3a35a2c0 0x558b3a35c6b0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.233+0000 7f5c6d419740 1 --2- 172.21.15.120:0/795198960 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b3a2ded40 0x558b3a2e3780 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.233+0000 7f5c6d419740 1 --2- 172.21.15.120:0/795198960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b3a2d4480 0x558b3a2e3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.233+0000 7f5c6d419740 1 --2- 172.21.15.120:0/795198960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b3a2d3b60 0x558b3a2e2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.233+0000 7f5c6d419740 1 -- 172.21.15.120:0/795198960 >> 172.21.15.120:0/795198960 conn(0x558b3a2d00f0 msgr2=0x558b3a2d5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:27.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.233+0000 7f5c43fff640 0 client.18873 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:27.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.233+0000 7f5c6d419740 1 -- 172.21.15.120:0/795198960 shutdown_connections 2024-09-19T09:12:27.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.233+0000 7f5c6d419740 1 -- 172.21.15.120:0/795198960 wait complete. 2024-09-19T09:12:27.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.233+0000 7f5c6d419740 20 client.18873 trim_cache size 0 max 16384 2024-09-19T09:12:27.238 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:12:27.238 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:12:27.238 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "29c1bc05-29ca-41ed-a0a9-4d69de0d79f6", 2024-09-19T09:12:27.238 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:12:27.238 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:12:27.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.234+0000 7f5c6b040640 1 -- 172.21.15.120:0/4089168014 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5c540a3240 msgr2=0x7f5c540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:27.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.234+0000 7f5c6b040640 1 --2- 172.21.15.120:0/4089168014 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5c540a3240 0x7f5c540a5700 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c005fd0 tx=0x7f5c5c011b40 comp rx=0 tx=0).stop 2024-09-19T09:12:27.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.234+0000 7f5c6b040640 1 -- 172.21.15.120:0/4089168014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c64100e20 msgr2=0x7f5c64199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:27.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.234+0000 7f5c6b040640 1 --2- 172.21.15.120:0/4089168014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c64100e20 0x7f5c64199310 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5c5800e9f0 tx=0x7f5c5800eeb0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.234+0000 7f5c6b040640 1 -- 172.21.15.120:0/4089168014 shutdown_connections 2024-09-19T09:12:27.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.234+0000 7f5c6b040640 1 --2- 172.21.15.120:0/4089168014 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5c540a3240 0x7f5c540a5700 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.234+0000 7f5c6b040640 1 --2- 172.21.15.120:0/4089168014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c64102ef0 0x7f5c641a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.234+0000 7f5c6b040640 1 --2- 172.21.15.120:0/4089168014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c64102570 0x7f5c64199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.234+0000 7f5c6b040640 1 --2- 172.21.15.120:0/4089168014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c64100e20 0x7f5c64199310 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:27.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.234+0000 7f5c6b040640 1 -- 172.21.15.120:0/4089168014 >> 172.21.15.120:0/4089168014 conn(0x7f5c640fc7d0 msgr2=0x7f5c640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:27.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.234+0000 7f5c6b040640 1 -- 172.21.15.120:0/4089168014 shutdown_connections 2024-09-19T09:12:27.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:27.234+0000 7f5c6b040640 1 -- 172.21.15.120:0/4089168014 wait complete. 2024-09-19T09:12:28.007 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:12:28.066 INFO:tasks.workunit.client.0.smithi120.stdout: 1 936023 10.06 MB/sec execute 383 sec latency 21.144 ms 2024-09-19T09:12:28.268 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:12:28.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.795+0000 7f5177f31640 1 -- 172.21.15.120:0/1424009518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5170100e20 msgr2=0x7f5170101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:28.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.795+0000 7f5177f31640 1 --2- 172.21.15.120:0/1424009518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5170100e20 0x7f5170101200 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f516c009900 tx=0x7f516c02a480 comp rx=0 tx=0).stop 2024-09-19T09:12:28.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.795+0000 7f5177f31640 1 -- 172.21.15.120:0/1424009518 shutdown_connections 2024-09-19T09:12:28.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.795+0000 7f5177f31640 1 --2- 172.21.15.120:0/1424009518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5170102ef0 0x7f517010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.795+0000 7f5177f31640 1 --2- 172.21.15.120:0/1424009518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5170102570 0x7f51701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.795+0000 7f5177f31640 1 --2- 172.21.15.120:0/1424009518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5170100e20 0x7f5170101200 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.795+0000 7f5177f31640 1 -- 172.21.15.120:0/1424009518 >> 172.21.15.120:0/1424009518 conn(0x7f51700fc7d0 msgr2=0x7f51700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:28.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.795+0000 7f5177f31640 1 -- 172.21.15.120:0/1424009518 shutdown_connections 2024-09-19T09:12:28.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.795+0000 7f5177f31640 1 -- 172.21.15.120:0/1424009518 wait complete. 2024-09-19T09:12:28.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.796+0000 7f5177f31640 1 Processor -- start 2024-09-19T09:12:28.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.796+0000 7f5177f31640 1 -- start start 2024-09-19T09:12:28.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.796+0000 7f5177f31640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5170100e20 0x7f5170199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:28.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.796+0000 7f5177f31640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5170102570 0x7f51701998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:28.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.796+0000 7f5177f31640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5170102ef0 0x7f51701a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:28.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.796+0000 7f5177f31640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5170110fa0 con 0x7f5170100e20 2024-09-19T09:12:28.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.796+0000 7f5177f31640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5170110e20 con 0x7f5170102ef0 2024-09-19T09:12:28.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.796+0000 7f5177f31640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5170111120 con 0x7f5170102570 2024-09-19T09:12:28.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.796+0000 7f5175ccd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5170100e20 0x7f5170199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:28.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.796+0000 7f5175ccd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5170100e20 0x7f5170199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48218/0 (socket says 172.21.15.120:48218) 2024-09-19T09:12:28.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.796+0000 7f5175ccd640 1 -- 172.21.15.120:0/4247331123 learned_addr learned my addr 172.21.15.120:0/4247331123 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:28.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.796+0000 7f51754cc640 1 --2- 172.21.15.120:0/4247331123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5170102570 0x7f51701998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:28.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.796+0000 7f51764ce640 1 --2- 172.21.15.120:0/4247331123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5170102ef0 0x7f51701a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:28.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.796+0000 7f5175ccd640 1 -- 172.21.15.120:0/4247331123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5170102570 msgr2=0x7f51701998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:28.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.796+0000 7f5175ccd640 1 --2- 172.21.15.120:0/4247331123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5170102570 0x7f51701998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.796+0000 7f5175ccd640 1 -- 172.21.15.120:0/4247331123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5170102ef0 msgr2=0x7f51701a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:28.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.796+0000 7f5175ccd640 1 --2- 172.21.15.120:0/4247331123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5170102ef0 0x7f51701a43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.796+0000 7f5175ccd640 1 -- 172.21.15.120:0/4247331123 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51701a4930 con 0x7f5170100e20 2024-09-19T09:12:28.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.796+0000 7f51754cc640 1 --2- 172.21.15.120:0/4247331123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5170102570 0x7f51701998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:12:28.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.796+0000 7f51764ce640 1 --2- 172.21.15.120:0/4247331123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5170102ef0 0x7f51701a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:12:28.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.796+0000 7f5175ccd640 1 --2- 172.21.15.120:0/4247331123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5170100e20 0x7f5170199370 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f516c006010 tx=0x7f516c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:28.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.797+0000 7f5166ffd640 1 -- 172.21.15.120:0/4247331123 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c03f070 con 0x7f5170100e20 2024-09-19T09:12:28.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.797+0000 7f5166ffd640 1 -- 172.21.15.120:0/4247331123 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f516c031e30 con 0x7f5170100e20 2024-09-19T09:12:28.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.797+0000 7f5177f31640 1 -- 172.21.15.120:0/4247331123 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51701a4b00 con 0x7f5170100e20 2024-09-19T09:12:28.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.797+0000 7f5166ffd640 1 -- 172.21.15.120:0/4247331123 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c035030 con 0x7f5170100e20 2024-09-19T09:12:28.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.797+0000 7f5177f31640 1 -- 172.21.15.120:0/4247331123 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51701a4f50 con 0x7f5170100e20 2024-09-19T09:12:28.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.799+0000 7f5166ffd640 1 -- 172.21.15.120:0/4247331123 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f516c042070 con 0x7f5170100e20 2024-09-19T09:12:28.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.799+0000 7f517a30a740 1 Processor -- start 2024-09-19T09:12:28.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.799+0000 7f517a30a740 1 -- start start 2024-09-19T09:12:28.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.800+0000 7f517a30a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6dc7b84a0 0x55a6dc7b8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:28.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.800+0000 7f517a30a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6dc7b8dc0 0x55a6dc7c3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:28.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.800+0000 7f517a30a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6dc7c3680 0x55a6dc7c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:28.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.800+0000 7f517a30a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a6dc71e2e0 con 0x55a6dc7b8dc0 2024-09-19T09:12:28.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.800+0000 7f517a30a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a6dc6c5ce0 con 0x55a6dc7b84a0 2024-09-19T09:12:28.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.800+0000 7f517a30a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a6dc71a2a0 con 0x55a6dc7c3680 2024-09-19T09:12:28.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.800+0000 7f5175ccd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6dc7b8dc0 0x55a6dc7c3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:28.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.800+0000 7f51764ce640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6dc7b84a0 0x55a6dc7b8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:28.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.800+0000 7f5166ffd640 1 --2- 172.21.15.120:0/4247331123 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f515c0a3240 0x7f515c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:28.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.800+0000 7f5175ccd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6dc7b8dc0 0x55a6dc7c3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48234/0 (socket says 172.21.15.120:48234) 2024-09-19T09:12:28.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.800+0000 7f51764ce640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6dc7b84a0 0x55a6dc7b8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:48354/0 (socket says 172.21.15.120:48354) 2024-09-19T09:12:28.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.800+0000 7f5175ccd640 1 -- 172.21.15.120:0/2239969286 learned_addr learned my addr 172.21.15.120:0/2239969286 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:28.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.800+0000 7f51754cc640 1 --2- 172.21.15.120:0/2239969286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6dc7c3680 0x55a6dc7c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:28.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.800+0000 7f51764ce640 1 --2- 172.21.15.120:0/4247331123 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f515c0a3240 0x7f515c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:28.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.800+0000 7f5166ffd640 1 -- 172.21.15.120:0/4247331123 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f516c0f0310 con 0x7f5170100e20 2024-09-19T09:12:28.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.800+0000 7f5175ccd640 1 -- 172.21.15.120:0/2239969286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6dc7c3680 msgr2=0x55a6dc7c5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:28.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.800+0000 7f5175ccd640 1 --2- 172.21.15.120:0/2239969286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6dc7c3680 0x55a6dc7c5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.800+0000 7f5175ccd640 1 -- 172.21.15.120:0/2239969286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6dc7b84a0 msgr2=0x55a6dc7b8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:28.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.800+0000 7f5175ccd640 1 --2- 172.21.15.120:0/2239969286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6dc7b84a0 0x55a6dc7b8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.800+0000 7f5175ccd640 1 -- 172.21.15.120:0/2239969286 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6dc6aa440 con 0x55a6dc7b8dc0 2024-09-19T09:12:28.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.800+0000 7f51754cc640 1 --2- 172.21.15.120:0/2239969286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6dc7c3680 0x55a6dc7c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:28.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.800+0000 7f51764ce640 1 --2- 172.21.15.120:0/2239969286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6dc7b84a0 0x55a6dc7b8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:28.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.800+0000 7f5175ccd640 1 --2- 172.21.15.120:0/2239969286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6dc7b8dc0 0x55a6dc7c3140 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f516c0ede10 tx=0x7f516c034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2679ac7162a5d4e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:28.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.801+0000 7f515bfff640 1 -- 172.21.15.120:0/2239969286 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c049070 con 0x55a6dc7b8dc0 2024-09-19T09:12:28.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.801+0000 7f51764ce640 1 --2- 172.21.15.120:0/4247331123 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f515c0a3240 0x7f515c0a5700 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5168005fd0 tx=0x7f5168004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:28.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.801+0000 7f515bfff640 1 -- 172.21.15.120:0/2239969286 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f516c0bd030 con 0x55a6dc7b8dc0 2024-09-19T09:12:28.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.801+0000 7f515bfff640 1 -- 172.21.15.120:0/2239969286 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c0b8fa0 con 0x55a6dc7b8dc0 2024-09-19T09:12:28.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.801+0000 7f517a30a740 1 -- 172.21.15.120:0/2239969286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6dc7b8dc0 msgr2=0x55a6dc7c3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:28.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.801+0000 7f517a30a740 1 --2- 172.21.15.120:0/2239969286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6dc7b8dc0 0x55a6dc7c3140 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f516c0ede10 tx=0x7f516c034000 comp rx=0 tx=0).stop 2024-09-19T09:12:28.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.801+0000 7f517a30a740 1 -- 172.21.15.120:0/2239969286 shutdown_connections 2024-09-19T09:12:28.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.801+0000 7f517a30a740 1 --2- 172.21.15.120:0/2239969286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6dc7c3680 0x55a6dc7c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.801+0000 7f517a30a740 1 --2- 172.21.15.120:0/2239969286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6dc7b8dc0 0x55a6dc7c3140 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.801+0000 7f517a30a740 1 --2- 172.21.15.120:0/2239969286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6dc7b84a0 0x55a6dc7b8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.801+0000 7f517a30a740 1 -- 172.21.15.120:0/2239969286 >> 172.21.15.120:0/2239969286 conn(0x55a6dc7b4a30 msgr2=0x55a6dc7b4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:28.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.802+0000 7f517a30a740 1 -- 172.21.15.120:0/2239969286 shutdown_connections 2024-09-19T09:12:28.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.802+0000 7f517a30a740 1 -- 172.21.15.120:0/2239969286 wait complete. 2024-09-19T09:12:28.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.802+0000 7f517a30a740 1 -- start start 2024-09-19T09:12:28.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.802+0000 7f517a30a740 1 Processor -- start 2024-09-19T09:12:28.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.802+0000 7f517a30a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6dc7b84a0 0x55a6dc7c3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:28.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.802+0000 7f517a30a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6dc7b8dc0 0x55a6dc7c02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:28.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.802+0000 7f517a30a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6dc7c3680 0x55a6dc7c0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:28.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.802+0000 7f517a30a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a6dc7c6010 con 0x55a6dc7b84a0 2024-09-19T09:12:28.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.803+0000 7f517a30a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a6dc6d9050 con 0x55a6dc7b8dc0 2024-09-19T09:12:28.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.803+0000 7f517a30a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a6dc7c6190 con 0x55a6dc7c3680 2024-09-19T09:12:28.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.803+0000 7f517a30a740 10 client.0 _ll_register_callbacks cb 0x55a6dc719160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:28.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.803+0000 7f5175ccd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6dc7b84a0 0x55a6dc7c3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:28.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.803+0000 7f5175ccd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6dc7b84a0 0x55a6dc7c3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48246/0 (socket says 172.21.15.120:48246) 2024-09-19T09:12:28.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.803+0000 7f5175ccd640 1 -- 172.21.15.120:0/3218982015 learned_addr learned my addr 172.21.15.120:0/3218982015 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:28.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.803+0000 7f51764ce640 1 --2- 172.21.15.120:0/3218982015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6dc7c3680 0x55a6dc7c0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:28.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.803+0000 7f51754cc640 1 --2- 172.21.15.120:0/3218982015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6dc7b8dc0 0x55a6dc7c02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:28.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.803+0000 7f5175ccd640 1 -- 172.21.15.120:0/3218982015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6dc7c3680 msgr2=0x55a6dc7c0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:28.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.803+0000 7f5175ccd640 1 --2- 172.21.15.120:0/3218982015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6dc7c3680 0x55a6dc7c0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.803+0000 7f5175ccd640 1 -- 172.21.15.120:0/3218982015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6dc7b8dc0 msgr2=0x55a6dc7c02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:28.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.803+0000 7f5175ccd640 1 --2- 172.21.15.120:0/3218982015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6dc7b8dc0 0x55a6dc7c02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.803+0000 7f5175ccd640 1 -- 172.21.15.120:0/3218982015 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6dc48f3c0 con 0x55a6dc7b84a0 2024-09-19T09:12:28.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.803+0000 7f51754cc640 1 --2- 172.21.15.120:0/3218982015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6dc7b8dc0 0x55a6dc7c02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:12:28.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.803+0000 7f5175ccd640 1 --2- 172.21.15.120:0/3218982015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6dc7b84a0 0x55a6dc7c3010 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f516c0c5070 tx=0x7f516c0f0600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7baa3ab9db6dbb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:28.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.803+0000 7f51597fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:28.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.803+0000 7f51597fa640 1 -- 172.21.15.120:0/3218982015 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c049070 con 0x55a6dc7b84a0 2024-09-19T09:12:28.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.803+0000 7f51597fa640 1 -- 172.21.15.120:0/3218982015 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f516c03b5f0 con 0x55a6dc7b84a0 2024-09-19T09:12:28.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.803+0000 7f517a30a740 1 -- 172.21.15.120:0/3218982015 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f516c0bd030 con 0x55a6dc7b84a0 2024-09-19T09:12:28.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.803+0000 7f51597fa640 1 -- 172.21.15.120:0/3218982015 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f516c0c55a0 con 0x55a6dc7b84a0 2024-09-19T09:12:28.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.803+0000 7f517a30a740 10 client.18885 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:28.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.803+0000 7f517a30a740 1 -- 172.21.15.120:0/3218982015 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a6dc7c6010 con 0x55a6dc7b84a0 2024-09-19T09:12:28.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.803+0000 7f51597fa640 1 -- 172.21.15.120:0/3218982015 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f516c0bd030 con 0x55a6dc7b84a0 2024-09-19T09:12:28.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.804+0000 7f51597fa640 1 -- 172.21.15.120:0/3218982015 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f516c0c5210 con 0x55a6dc7b84a0 2024-09-19T09:12:28.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.804+0000 7f517a30a740 10 client.18885 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:28.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.804+0000 7f517a30a740 10 client.18885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:12:28.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.804+0000 7f517a30a740 20 client.18885 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:28.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.804+0000 7f517a30a740 1 --2- 172.21.15.120:0/3218982015 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a6dc7d2c20 0x55a6dc7d5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:28.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.804+0000 7f517a30a740 4 client.18885 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:12:28.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.804+0000 7f517a30a740 1 -- 172.21.15.120:0/3218982015 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6dc72b850 con 0x55a6dc7d2c20 2024-09-19T09:12:28.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.818+0000 7f51754cc640 1 --2- 172.21.15.120:0/3218982015 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a6dc7d2c20 0x55a6dc7d5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:28.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.818+0000 7f51754cc640 1 --2- 172.21.15.120:0/3218982015 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a6dc7d2c20 0x55a6dc7d5040 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b70dd5e0fe74beff server_cookie=655243cee27bf55d in_seq=0 out_seq=0 2024-09-19T09:12:28.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.818+0000 7f51597fa640 10 client.18885 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:28.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.819+0000 7f51597fa640 1 -- 172.21.15.120:0/3218982015 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a6dc72b850 con 0x55a6dc7d2c20 2024-09-19T09:12:28.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.819+0000 7f51597fa640 10 client.18885 handle_command_reply: tid=0 2024-09-19T09:12:28.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.819+0000 7f517a30a740 1 client.18885 shutdown 2024-09-19T09:12:28.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.819+0000 7f517a30a740 1 -- 172.21.15.120:0/3218982015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6dc7b84a0 msgr2=0x55a6dc7c3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:28.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.819+0000 7f517a30a740 1 --2- 172.21.15.120:0/3218982015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6dc7b84a0 0x55a6dc7c3010 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f516c0c5070 tx=0x7f516c0f0600 comp rx=0 tx=0).stop 2024-09-19T09:12:28.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.819+0000 7f517a30a740 1 -- 172.21.15.120:0/3218982015 shutdown_connections 2024-09-19T09:12:28.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.819+0000 7f517a30a740 1 --2- 172.21.15.120:0/3218982015 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a6dc7d2c20 0x55a6dc7d5040 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.819+0000 7f517a30a740 1 --2- 172.21.15.120:0/3218982015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6dc7c3680 0x55a6dc7c0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.819+0000 7f517a30a740 1 --2- 172.21.15.120:0/3218982015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6dc7b8dc0 0x55a6dc7c02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.819+0000 7f51597fa640 0 client.18885 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:28.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.819+0000 7f517a30a740 1 --2- 172.21.15.120:0/3218982015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6dc7b84a0 0x55a6dc7c3010 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.819+0000 7f517a30a740 1 -- 172.21.15.120:0/3218982015 >> 172.21.15.120:0/3218982015 conn(0x55a6dc7b4a30 msgr2=0x55a6dc7ba760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:28.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.819+0000 7f517a30a740 1 -- 172.21.15.120:0/3218982015 shutdown_connections 2024-09-19T09:12:28.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.819+0000 7f517a30a740 1 -- 172.21.15.120:0/3218982015 wait complete. 2024-09-19T09:12:28.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.819+0000 7f517a30a740 20 client.18885 trim_cache size 0 max 16384 2024-09-19T09:12:28.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.837+0000 7f517a30a740 1 Processor -- start 2024-09-19T09:12:28.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.837+0000 7f517a30a740 1 -- start start 2024-09-19T09:12:28.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.838+0000 7f517a30a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6dc7b84a0 0x55a6dc7da910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:28.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.838+0000 7f517a30a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6dc7b8dc0 0x55a6dc7dae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:28.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.838+0000 7f517a30a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6dc7c3680 0x55a6dc7bd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:28.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.838+0000 7f517a30a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f51701ad560 con 0x55a6dc7b84a0 2024-09-19T09:12:28.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.838+0000 7f517a30a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f51701af0a0 con 0x55a6dc7c3680 2024-09-19T09:12:28.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.838+0000 7f517a30a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f51701adc30 con 0x55a6dc7b8dc0 2024-09-19T09:12:28.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.838+0000 7f5175ccd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6dc7b84a0 0x55a6dc7da910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:28.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.838+0000 7f5175ccd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6dc7b84a0 0x55a6dc7da910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48256/0 (socket says 172.21.15.120:48256) 2024-09-19T09:12:28.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.838+0000 7f5175ccd640 1 -- 172.21.15.120:0/2303995776 learned_addr learned my addr 172.21.15.120:0/2303995776 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:28.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.838+0000 7f51764ce640 1 --2- 172.21.15.120:0/2303995776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6dc7c3680 0x55a6dc7bd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:28.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.838+0000 7f51754cc640 1 --2- 172.21.15.120:0/2303995776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6dc7b8dc0 0x55a6dc7dae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:28.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.838+0000 7f5175ccd640 1 -- 172.21.15.120:0/2303995776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6dc7b8dc0 msgr2=0x55a6dc7dae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:28.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.838+0000 7f5175ccd640 1 --2- 172.21.15.120:0/2303995776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6dc7b8dc0 0x55a6dc7dae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.838+0000 7f5175ccd640 1 -- 172.21.15.120:0/2303995776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6dc7c3680 msgr2=0x55a6dc7bd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:28.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.838+0000 7f5175ccd640 1 --2- 172.21.15.120:0/2303995776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6dc7c3680 0x55a6dc7bd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.838+0000 7f5175ccd640 1 -- 172.21.15.120:0/2303995776 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6dc7c0bb0 con 0x55a6dc7b84a0 2024-09-19T09:12:28.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.838+0000 7f5175ccd640 1 --2- 172.21.15.120:0/2303995776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6dc7b84a0 0x55a6dc7da910 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f516c0b9300 tx=0x7f516c034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=298afeb22283635d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:28.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.838+0000 7f515b7fe640 1 -- 172.21.15.120:0/2303995776 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c049070 con 0x55a6dc7b84a0 2024-09-19T09:12:28.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.838+0000 7f515b7fe640 1 -- 172.21.15.120:0/2303995776 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f516c0d5070 con 0x55a6dc7b84a0 2024-09-19T09:12:28.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.838+0000 7f515b7fe640 1 -- 172.21.15.120:0/2303995776 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c0bace0 con 0x55a6dc7b84a0 2024-09-19T09:12:28.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.839+0000 7f517a30a740 1 -- 172.21.15.120:0/2303995776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6dc7b84a0 msgr2=0x55a6dc7da910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:28.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.839+0000 7f517a30a740 1 --2- 172.21.15.120:0/2303995776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6dc7b84a0 0x55a6dc7da910 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f516c0b9300 tx=0x7f516c034000 comp rx=0 tx=0).stop 2024-09-19T09:12:28.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.839+0000 7f517a30a740 1 -- 172.21.15.120:0/2303995776 shutdown_connections 2024-09-19T09:12:28.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.839+0000 7f517a30a740 1 --2- 172.21.15.120:0/2303995776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6dc7c3680 0x55a6dc7bd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.839+0000 7f517a30a740 1 --2- 172.21.15.120:0/2303995776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6dc7b8dc0 0x55a6dc7dae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.839+0000 7f517a30a740 1 --2- 172.21.15.120:0/2303995776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6dc7b84a0 0x55a6dc7da910 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.839+0000 7f517a30a740 1 -- 172.21.15.120:0/2303995776 >> 172.21.15.120:0/2303995776 conn(0x55a6dc7b4a30 msgr2=0x55a6dc7b3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:28.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.839+0000 7f517a30a740 1 -- 172.21.15.120:0/2303995776 shutdown_connections 2024-09-19T09:12:28.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.839+0000 7f517a30a740 1 -- 172.21.15.120:0/2303995776 wait complete. 2024-09-19T09:12:28.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.839+0000 7f517a30a740 1 -- start start 2024-09-19T09:12:28.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.840+0000 7f517a30a740 1 Processor -- start 2024-09-19T09:12:28.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.840+0000 7f517a30a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6dc7b84a0 0x55a6dc7c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:28.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.840+0000 7f517a30a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6dc7b8dc0 0x55a6dc7c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:28.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.840+0000 7f517a30a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6dc7c3680 0x55a6dc7c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:28.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.840+0000 7f517a30a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f51701ae9d0 con 0x55a6dc7b84a0 2024-09-19T09:12:28.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.840+0000 7f517a30a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f51701ae300 con 0x55a6dc7b8dc0 2024-09-19T09:12:28.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.840+0000 7f517a30a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f51701ab520 con 0x55a6dc7c3680 2024-09-19T09:12:28.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.840+0000 7f517a30a740 10 client.0 _ll_register_callbacks cb 0x7f5168008650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:28.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.840+0000 7f5175ccd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6dc7b84a0 0x55a6dc7c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:28.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.840+0000 7f5175ccd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6dc7b84a0 0x55a6dc7c90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48258/0 (socket says 172.21.15.120:48258) 2024-09-19T09:12:28.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.840+0000 7f5175ccd640 1 -- 172.21.15.120:0/797228287 learned_addr learned my addr 172.21.15.120:0/797228287 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:28.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.840+0000 7f51754cc640 1 --2- 172.21.15.120:0/797228287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6dc7b8dc0 0x55a6dc7c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:28.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.840+0000 7f51764ce640 1 --2- 172.21.15.120:0/797228287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6dc7c3680 0x55a6dc7c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:28.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.841+0000 7f5175ccd640 1 -- 172.21.15.120:0/797228287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6dc7c3680 msgr2=0x55a6dc7c9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:28.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.841+0000 7f5175ccd640 1 --2- 172.21.15.120:0/797228287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6dc7c3680 0x55a6dc7c9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.841+0000 7f5175ccd640 1 -- 172.21.15.120:0/797228287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6dc7b8dc0 msgr2=0x55a6dc7c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:28.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.841+0000 7f5175ccd640 1 --2- 172.21.15.120:0/797228287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6dc7b8dc0 0x55a6dc7c95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.841+0000 7f5175ccd640 1 -- 172.21.15.120:0/797228287 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6dc83c8f0 con 0x55a6dc7b84a0 2024-09-19T09:12:28.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.841+0000 7f51754cc640 1 --2- 172.21.15.120:0/797228287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6dc7b8dc0 0x55a6dc7c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:28.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.841+0000 7f5175ccd640 1 --2- 172.21.15.120:0/797228287 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6dc7b84a0 0x55a6dc7c90a0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f516c0d5b30 tx=0x7f516c0bbe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f2fb9a10fcac5d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:28.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.841+0000 7f51597fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:28.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.841+0000 7f51597fa640 1 -- 172.21.15.120:0/797228287 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c049070 con 0x55a6dc7b84a0 2024-09-19T09:12:28.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.841+0000 7f51597fa640 1 -- 172.21.15.120:0/797228287 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f516c03b5f0 con 0x55a6dc7b84a0 2024-09-19T09:12:28.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.841+0000 7f517a30a740 1 -- 172.21.15.120:0/797228287 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f516c0d5070 con 0x55a6dc7b84a0 2024-09-19T09:12:28.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.841+0000 7f51597fa640 1 -- 172.21.15.120:0/797228287 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f516c0a2050 con 0x55a6dc7b84a0 2024-09-19T09:12:28.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.841+0000 7f517a30a740 10 client.18891 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:28.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.841+0000 7f517a30a740 1 -- 172.21.15.120:0/797228287 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a6dc83c8f0 con 0x55a6dc7b84a0 2024-09-19T09:12:28.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.841+0000 7f51597fa640 1 -- 172.21.15.120:0/797228287 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f516c0d5070 con 0x55a6dc7b84a0 2024-09-19T09:12:28.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.841+0000 7f51597fa640 1 -- 172.21.15.120:0/797228287 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f516c0c5110 con 0x55a6dc7b84a0 2024-09-19T09:12:28.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.841+0000 7f517a30a740 10 client.18891 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:28.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.841+0000 7f517a30a740 10 client.18891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:12:28.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.841+0000 7f517a30a740 20 client.18891 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:28.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.842+0000 7f517a30a740 1 --2- 172.21.15.120:0/797228287 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a6dc83dd60 0x55a6dc840150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:28.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.842+0000 7f517a30a740 4 client.18891 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:12:28.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.842+0000 7f517a30a740 1 -- 172.21.15.120:0/797228287 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a6dc725d60 con 0x55a6dc83dd60 2024-09-19T09:12:28.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.842+0000 7f51754cc640 1 --2- 172.21.15.120:0/797228287 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a6dc83dd60 0x55a6dc840150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:28.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.842+0000 7f51754cc640 1 --2- 172.21.15.120:0/797228287 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a6dc83dd60 0x55a6dc840150 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32e8a2f0c2a1242f server_cookie=4f41ff732ecd0823 in_seq=0 out_seq=0 2024-09-19T09:12:28.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.842+0000 7f51597fa640 10 client.18891 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:28.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.843+0000 7f51597fa640 1 -- 172.21.15.120:0/797228287 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a6dc725d60 con 0x55a6dc83dd60 2024-09-19T09:12:28.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.843+0000 7f51597fa640 10 client.18891 handle_command_reply: tid=0 2024-09-19T09:12:28.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.843+0000 7f517a30a740 1 client.18891 shutdown 2024-09-19T09:12:28.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.844+0000 7f517a30a740 1 -- 172.21.15.120:0/797228287 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6dc7b84a0 msgr2=0x55a6dc7c90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:28.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.844+0000 7f517a30a740 1 --2- 172.21.15.120:0/797228287 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6dc7b84a0 0x55a6dc7c90a0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f516c0d5b30 tx=0x7f516c0bbe00 comp rx=0 tx=0).stop 2024-09-19T09:12:28.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.844+0000 7f517a30a740 1 -- 172.21.15.120:0/797228287 shutdown_connections 2024-09-19T09:12:28.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.844+0000 7f517a30a740 1 --2- 172.21.15.120:0/797228287 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a6dc83dd60 0x55a6dc840150 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.844+0000 7f517a30a740 1 --2- 172.21.15.120:0/797228287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6dc7c3680 0x55a6dc7c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.844+0000 7f517a30a740 1 --2- 172.21.15.120:0/797228287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6dc7b8dc0 0x55a6dc7c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.844+0000 7f51597fa640 0 client.18891 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:28.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.844+0000 7f517a30a740 1 --2- 172.21.15.120:0/797228287 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6dc7b84a0 0x55a6dc7c90a0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.844+0000 7f517a30a740 1 -- 172.21.15.120:0/797228287 >> 172.21.15.120:0/797228287 conn(0x55a6dc7b4a30 msgr2=0x55a6dc7c1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:28.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.844+0000 7f517a30a740 1 -- 172.21.15.120:0/797228287 shutdown_connections 2024-09-19T09:12:28.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.844+0000 7f517a30a740 1 -- 172.21.15.120:0/797228287 wait complete. 2024-09-19T09:12:28.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.844+0000 7f517a30a740 20 client.18891 trim_cache size 0 max 16384 2024-09-19T09:12:28.849 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:12:28.849 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:12:28.849 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:12:28.849 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:12:28.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.844+0000 7f5177f31640 1 -- 172.21.15.120:0/4247331123 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f515c0a3240 msgr2=0x7f515c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:28.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.844+0000 7f5177f31640 1 --2- 172.21.15.120:0/4247331123 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f515c0a3240 0x7f515c0a5700 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5168005fd0 tx=0x7f5168004ab0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.844+0000 7f5177f31640 1 -- 172.21.15.120:0/4247331123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5170100e20 msgr2=0x7f5170199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:28.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.844+0000 7f5177f31640 1 --2- 172.21.15.120:0/4247331123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5170100e20 0x7f5170199370 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f516c006010 tx=0x7f516c003c80 comp rx=0 tx=0).stop 2024-09-19T09:12:28.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.845+0000 7f5177f31640 1 -- 172.21.15.120:0/4247331123 shutdown_connections 2024-09-19T09:12:28.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.845+0000 7f5177f31640 1 --2- 172.21.15.120:0/4247331123 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f515c0a3240 0x7f515c0a5700 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.845+0000 7f5177f31640 1 --2- 172.21.15.120:0/4247331123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5170102ef0 0x7f51701a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.845+0000 7f5177f31640 1 --2- 172.21.15.120:0/4247331123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5170102570 0x7f51701998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.845+0000 7f5177f31640 1 --2- 172.21.15.120:0/4247331123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5170100e20 0x7f5170199370 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:28.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.845+0000 7f5177f31640 1 -- 172.21.15.120:0/4247331123 >> 172.21.15.120:0/4247331123 conn(0x7f51700fc7d0 msgr2=0x7f51700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:28.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.845+0000 7f5177f31640 1 -- 172.21.15.120:0/4247331123 shutdown_connections 2024-09-19T09:12:28.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:28.845+0000 7f5177f31640 1 -- 172.21.15.120:0/4247331123 wait complete. 2024-09-19T09:12:29.066 INFO:tasks.workunit.client.0.smithi120.stdout: 1 938679 10.07 MB/sec execute 384 sec latency 83.836 ms 2024-09-19T09:12:29.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:28 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:12:29.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:28 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:12:29.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:28 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:12:29.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:28 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:12:29.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:28 smithi120 ceph-mon[31150]: pgmap v631: 97 pgs: 97 active+clean; 36 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 441 op/s 2024-09-19T09:12:29.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:28 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:12:29.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:28 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:12:29.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:28 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:12:29.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:28 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:12:29.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:28 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:12:29.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:28 smithi196 ceph-mon[35150]: pgmap v631: 97 pgs: 97 active+clean; 36 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 441 op/s 2024-09-19T09:12:29.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:28 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:12:29.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:28 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:12:29.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:28 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:12:29.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:28 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:12:29.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:28 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:12:29.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:28 smithi153 ceph-mon[35071]: pgmap v631: 97 pgs: 97 active+clean; 36 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 441 op/s 2024-09-19T09:12:29.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:28 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:12:29.658 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:12:29.659 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:12:29.920 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:12:30.066 INFO:tasks.workunit.client.0.smithi120.stdout: 1 941945 10.09 MB/sec execute 385 sec latency 13.997 ms 2024-09-19T09:12:30.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.477+0000 7fdc8f0f1640 1 -- 172.21.15.120:0/1883533014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc881082a0 msgr2=0x7fdc8810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:30.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.477+0000 7fdc8f0f1640 1 --2- 172.21.15.120:0/1883533014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc881082a0 0x7fdc8810ec40 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fdc80009cd0 tx=0x7fdc8002a8e0 comp rx=0 tx=0).stop 2024-09-19T09:12:30.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.477+0000 7fdc8f0f1640 1 -- 172.21.15.120:0/1883533014 shutdown_connections 2024-09-19T09:12:30.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.477+0000 7fdc8f0f1640 1 --2- 172.21.15.120:0/1883533014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc881082a0 0x7fdc8810ec40 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:30.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.477+0000 7fdc8f0f1640 1 --2- 172.21.15.120:0/1883533014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc881036e0 0x7fdc88107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:30.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.477+0000 7fdc8f0f1640 1 --2- 172.21.15.120:0/1883533014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc88102d30 0x7fdc88103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.477+0000 7fdc8f0f1640 1 -- 172.21.15.120:0/1883533014 >> 172.21.15.120:0/1883533014 conn(0x7fdc880fc7d0 msgr2=0x7fdc880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.477+0000 7fdc8f0f1640 1 -- 172.21.15.120:0/1883533014 shutdown_connections 2024-09-19T09:12:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.477+0000 7fdc8f0f1640 1 -- 172.21.15.120:0/1883533014 wait complete. 2024-09-19T09:12:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.477+0000 7fdc8f0f1640 1 Processor -- start 2024-09-19T09:12:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.477+0000 7fdc8f0f1640 1 -- start start 2024-09-19T09:12:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.478+0000 7fdc8f0f1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc88102d30 0x7fdc881985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.478+0000 7fdc8f0f1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc881036e0 0x7fdc88198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.478+0000 7fdc8f0f1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc881082a0 0x7fdc8819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.478+0000 7fdc8f0f1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881120d0 con 0x7fdc881082a0 2024-09-19T09:12:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.478+0000 7fdc8f0f1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdc88111f50 con 0x7fdc881036e0 2024-09-19T09:12:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.478+0000 7fdc8f0f1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc88112250 con 0x7fdc88102d30 2024-09-19T09:12:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.478+0000 7fdc8ce8d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc88102d30 0x7fdc881985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.478+0000 7fdc8d68e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc881082a0 0x7fdc8819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.478+0000 7fdc8ce8d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc88102d30 0x7fdc881985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:48208/0 (socket says 172.21.15.120:48208) 2024-09-19T09:12:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.478+0000 7fdc8ce8d640 1 -- 172.21.15.120:0/3962311226 learned_addr learned my addr 172.21.15.120:0/3962311226 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.478+0000 7fdc87fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc881036e0 0x7fdc88198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.478+0000 7fdc8ce8d640 1 -- 172.21.15.120:0/3962311226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc881036e0 msgr2=0x7fdc88198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.478+0000 7fdc8ce8d640 1 --2- 172.21.15.120:0/3962311226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc881036e0 0x7fdc88198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.478+0000 7fdc8ce8d640 1 -- 172.21.15.120:0/3962311226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc881082a0 msgr2=0x7fdc8819ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.478+0000 7fdc8ce8d640 1 --2- 172.21.15.120:0/3962311226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc881082a0 0x7fdc8819ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.478+0000 7fdc8ce8d640 1 -- 172.21.15.120:0/3962311226 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc8819d560 con 0x7fdc88102d30 2024-09-19T09:12:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.478+0000 7fdc8ce8d640 1 --2- 172.21.15.120:0/3962311226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc88102d30 0x7fdc881985b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c00e9f0 tx=0x7fdc7c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.478+0000 7fdc85ffb640 1 -- 172.21.15.120:0/3962311226 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c00cd80 con 0x7fdc88102d30 2024-09-19T09:12:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.478+0000 7fdc85ffb640 1 -- 172.21.15.120:0/3962311226 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc7c010910 con 0x7fdc88102d30 2024-09-19T09:12:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.478+0000 7fdc8f0f1640 1 -- 172.21.15.120:0/3962311226 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc8819d850 con 0x7fdc88102d30 2024-09-19T09:12:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.478+0000 7fdc85ffb640 1 -- 172.21.15.120:0/3962311226 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c018990 con 0x7fdc88102d30 2024-09-19T09:12:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.479+0000 7fdc8f0f1640 1 -- 172.21.15.120:0/3962311226 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc881a5130 con 0x7fdc88102d30 2024-09-19T09:12:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.480+0000 7fdc8f0f1640 1 -- 172.21.15.120:0/3962311226 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc88103110 con 0x7fdc88102d30 2024-09-19T09:12:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.481+0000 7fdc85ffb640 1 -- 172.21.15.120:0/3962311226 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fdc7c010ac0 con 0x7fdc88102d30 2024-09-19T09:12:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.482+0000 7fdc85ffb640 1 --2- 172.21.15.120:0/3962311226 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdc780a3240 0x7fdc780a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.482+0000 7fdc87fff640 1 --2- 172.21.15.120:0/3962311226 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdc780a3240 0x7fdc780a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.482+0000 7fdc85ffb640 1 -- 172.21.15.120:0/3962311226 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fdc7c0cfff0 con 0x7fdc88102d30 2024-09-19T09:12:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.482+0000 7fdc87fff640 1 --2- 172.21.15.120:0/3962311226 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdc780a3240 0x7fdc780a5700 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdc880fecd0 tx=0x7fdc74005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.485+0000 7fdc85ffb640 1 -- 172.21.15.120:0/3962311226 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdc7c09c680 con 0x7fdc88102d30 2024-09-19T09:12:30.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.712+0000 7fdc8f0f1640 1 -- 172.21.15.120:0/3962311226 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc88199910 con 0x7fdc88102d30 2024-09-19T09:12:30.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.713+0000 7fdc85ffb640 1 -- 172.21.15.120:0/3962311226 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11988 (secure 0 0 0) 0x7fdc7c0a0980 con 0x7fdc88102d30 2024-09-19T09:12:30.714 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:12:30.714 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 32 2024-09-19T09:12:30.714 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":32,"btime":"2024-09-19T09:11:12:904684+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:11:12.904670+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27518":{"gid":27518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3214776406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3214776406},{"type":"v1","addr":"172.21.15.153:6835","nonce":3214776406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:12:30.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.715+0000 7fdc8f0f1640 1 -- 172.21.15.120:0/3962311226 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdc780a3240 msgr2=0x7fdc780a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:30.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.715+0000 7fdc8f0f1640 1 --2- 172.21.15.120:0/3962311226 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdc780a3240 0x7fdc780a5700 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdc880fecd0 tx=0x7fdc74005f90 comp rx=0 tx=0).stop 2024-09-19T09:12:30.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.715+0000 7fdc8f0f1640 1 -- 172.21.15.120:0/3962311226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc88102d30 msgr2=0x7fdc881985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:30.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.715+0000 7fdc8f0f1640 1 --2- 172.21.15.120:0/3962311226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc88102d30 0x7fdc881985b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c00e9f0 tx=0x7fdc7c00eeb0 comp rx=0 tx=0).stop 2024-09-19T09:12:30.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.715+0000 7fdc8f0f1640 1 -- 172.21.15.120:0/3962311226 shutdown_connections 2024-09-19T09:12:30.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.715+0000 7fdc8f0f1640 1 --2- 172.21.15.120:0/3962311226 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdc780a3240 0x7fdc780a5700 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:30.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.715+0000 7fdc8f0f1640 1 --2- 172.21.15.120:0/3962311226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc881082a0 0x7fdc8819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:30.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.715+0000 7fdc8f0f1640 1 --2- 172.21.15.120:0/3962311226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc881036e0 0x7fdc88198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:30.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.715+0000 7fdc8f0f1640 1 --2- 172.21.15.120:0/3962311226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc88102d30 0x7fdc881985b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:30.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.715+0000 7fdc8f0f1640 1 -- 172.21.15.120:0/3962311226 >> 172.21.15.120:0/3962311226 conn(0x7fdc880fc7d0 msgr2=0x7fdc88106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:30.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.715+0000 7fdc8f0f1640 1 -- 172.21.15.120:0/3962311226 shutdown_connections 2024-09-19T09:12:30.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:30.715+0000 7fdc8f0f1640 1 -- 172.21.15.120:0/3962311226 wait complete. 2024-09-19T09:12:31.066 INFO:tasks.workunit.client.0.smithi120.stdout: 1 945212 10.11 MB/sec execute 386 sec latency 23.529 ms 2024-09-19T09:12:31.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:30 smithi120 ceph-mon[31150]: pgmap v632: 97 pgs: 97 active+clean; 36 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.9 MiB/s wr, 446 op/s 2024-09-19T09:12:31.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:30 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3962311226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:12:31.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:30 smithi196 ceph-mon[35150]: pgmap v632: 97 pgs: 97 active+clean; 36 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.9 MiB/s wr, 446 op/s 2024-09-19T09:12:31.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:30 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3962311226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:12:31.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:30 smithi153 ceph-mon[35071]: pgmap v632: 97 pgs: 97 active+clean; 36 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.9 MiB/s wr, 446 op/s 2024-09-19T09:12:31.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:30 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3962311226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:12:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.697+0000 7f5944959640 1 -- 172.21.15.120:0/1525016339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5940101c40 msgr2=0x7f59401020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.697+0000 7f5944959640 1 --2- 172.21.15.120:0/1525016339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5940101c40 0x7f59401020c0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f5934009900 tx=0x7f593402a410 comp rx=0 tx=0).stop 2024-09-19T09:12:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.697+0000 7f5944959640 1 -- 172.21.15.120:0/1525016339 shutdown_connections 2024-09-19T09:12:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.697+0000 7f5944959640 1 --2- 172.21.15.120:0/1525016339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5940102600 0x7f5940108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.697+0000 7f5944959640 1 --2- 172.21.15.120:0/1525016339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5940101c40 0x7f59401020c0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.697+0000 7f5944959640 1 --2- 172.21.15.120:0/1525016339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5940100a50 0x7f5940100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.697+0000 7f5944959640 1 -- 172.21.15.120:0/1525016339 >> 172.21.15.120:0/1525016339 conn(0x7f59400fc1b0 msgr2=0x7f59400fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.697+0000 7f5944959640 1 -- 172.21.15.120:0/1525016339 shutdown_connections 2024-09-19T09:12:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.697+0000 7f5944959640 1 -- 172.21.15.120:0/1525016339 wait complete. 2024-09-19T09:12:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.698+0000 7f5944959640 1 Processor -- start 2024-09-19T09:12:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.698+0000 7f5944959640 1 -- start start 2024-09-19T09:12:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.698+0000 7f5944959640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5940100a50 0x7f5940196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.698+0000 7f5944959640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5940101c40 0x7f59401971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.698+0000 7f5944959640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5940102600 0x7f594019e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.698+0000 7f5944959640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f594010c700 con 0x7f5940101c40 2024-09-19T09:12:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.698+0000 7f5944959640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f594010c580 con 0x7f5940100a50 2024-09-19T09:12:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.698+0000 7f5944959640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f594010c880 con 0x7f5940102600 2024-09-19T09:12:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.698+0000 7f593dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5940101c40 0x7f59401971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.698+0000 7f593dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5940101c40 0x7f59401971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47654/0 (socket says 172.21.15.120:47654) 2024-09-19T09:12:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.698+0000 7f593dd9b640 1 -- 172.21.15.120:0/4066362854 learned_addr learned my addr 172.21.15.120:0/4066362854 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.698+0000 7f593e59c640 1 --2- 172.21.15.120:0/4066362854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5940100a50 0x7f5940196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.698+0000 7f593ed9d640 1 --2- 172.21.15.120:0/4066362854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5940102600 0x7f594019e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.698+0000 7f593dd9b640 1 -- 172.21.15.120:0/4066362854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5940102600 msgr2=0x7f594019e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.698+0000 7f593dd9b640 1 --2- 172.21.15.120:0/4066362854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5940102600 0x7f594019e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.698+0000 7f593dd9b640 1 -- 172.21.15.120:0/4066362854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5940100a50 msgr2=0x7f5940196ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.698+0000 7f593dd9b640 1 --2- 172.21.15.120:0/4066362854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5940100a50 0x7f5940196ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.698+0000 7f593dd9b640 1 -- 172.21.15.120:0/4066362854 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f594019e8e0 con 0x7f5940101c40 2024-09-19T09:12:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.698+0000 7f593e59c640 1 --2- 172.21.15.120:0/4066362854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5940100a50 0x7f5940196ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.699+0000 7f593dd9b640 1 --2- 172.21.15.120:0/4066362854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5940101c40 0x7f59401971e0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f5934031440 tx=0x7f5934032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.699+0000 7f592f7fe640 1 -- 172.21.15.120:0/4066362854 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593403f030 con 0x7f5940101c40 2024-09-19T09:12:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.699+0000 7f592f7fe640 1 -- 172.21.15.120:0/4066362854 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5934003ce0 con 0x7f5940101c40 2024-09-19T09:12:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.699+0000 7f5944959640 1 -- 172.21.15.120:0/4066362854 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f594019eb70 con 0x7f5940101c40 2024-09-19T09:12:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.699+0000 7f592f7fe640 1 -- 172.21.15.120:0/4066362854 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934035070 con 0x7f5940101c40 2024-09-19T09:12:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.699+0000 7f5944959640 1 -- 172.21.15.120:0/4066362854 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f594019f060 con 0x7f5940101c40 2024-09-19T09:12:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.702+0000 7f59477d2740 1 Processor -- start 2024-09-19T09:12:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.702+0000 7f59477d2740 1 -- start start 2024-09-19T09:12:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.702+0000 7f592f7fe640 1 -- 172.21.15.120:0/4066362854 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5934042030 con 0x7f5940101c40 2024-09-19T09:12:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.702+0000 7f59477d2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c1cb629a0 0x562c1cb62da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.702+0000 7f59477d2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c1cb632e0 0x562c1cb67da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.702+0000 7f59477d2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c1cb68370 0x562c1cb6a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.702+0000 7f59477d2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562c1cac62f0 con 0x562c1cb68370 2024-09-19T09:12:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.702+0000 7f59477d2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562c1ca6e270 con 0x562c1cb629a0 2024-09-19T09:12:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.702+0000 7f59477d2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562c1cac22b0 con 0x562c1cb632e0 2024-09-19T09:12:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.703+0000 7f593e59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c1cb632e0 0x562c1cb67da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.703+0000 7f593dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c1cb68370 0x562c1cb6a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.703+0000 7f593ed9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c1cb629a0 0x562c1cb62da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.703+0000 7f593e59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c1cb632e0 0x562c1cb67da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:47484/0 (socket says 172.21.15.120:47484) 2024-09-19T09:12:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.703+0000 7f593ed9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c1cb629a0 0x562c1cb62da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52728/0 (socket says 172.21.15.120:52728) 2024-09-19T09:12:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.703+0000 7f593e59c640 1 -- 172.21.15.120:0/856343414 learned_addr learned my addr 172.21.15.120:0/856343414 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.703+0000 7f593e59c640 1 -- 172.21.15.120:0/856343414 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c1cb629a0 msgr2=0x562c1cb62da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.703+0000 7f593e59c640 1 --2- 172.21.15.120:0/856343414 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c1cb629a0 0x562c1cb62da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.703+0000 7f593e59c640 1 -- 172.21.15.120:0/856343414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c1cb68370 msgr2=0x562c1cb6a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.703+0000 7f593e59c640 1 --2- 172.21.15.120:0/856343414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c1cb68370 0x562c1cb6a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.703+0000 7f593e59c640 1 -- 172.21.15.120:0/856343414 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c1ca6d340 con 0x562c1cb632e0 2024-09-19T09:12:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.703+0000 7f593e59c640 1 --2- 172.21.15.120:0/856343414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c1cb632e0 0x562c1cb67da0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f59300114f0 tx=0x7f59300119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=531a622d85df057e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.703+0000 7f592f7fe640 1 --2- 172.21.15.120:0/4066362854 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f59280a32f0 0x7f59280a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.703+0000 7f592cff9640 1 -- 172.21.15.120:0/856343414 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5930004310 con 0x562c1cb632e0 2024-09-19T09:12:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.703+0000 7f593ed9d640 1 --2- 172.21.15.120:0/4066362854 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f59280a32f0 0x7f59280a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.703+0000 7f592cff9640 1 -- 172.21.15.120:0/856343414 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59300044b0 con 0x562c1cb632e0 2024-09-19T09:12:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.703+0000 7f592cff9640 1 -- 172.21.15.120:0/856343414 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59300037f0 con 0x562c1cb632e0 2024-09-19T09:12:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.703+0000 7f592f7fe640 1 -- 172.21.15.120:0/4066362854 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f59340efd70 con 0x7f5940101c40 2024-09-19T09:12:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.704+0000 7f59477d2740 1 -- 172.21.15.120:0/856343414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c1cb632e0 msgr2=0x562c1cb67da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.704+0000 7f593ed9d640 1 --2- 172.21.15.120:0/4066362854 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f59280a32f0 0x7f59280a57b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5938005fd0 tx=0x7f5938009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.704+0000 7f59477d2740 1 --2- 172.21.15.120:0/856343414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c1cb632e0 0x562c1cb67da0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f59300114f0 tx=0x7f59300119b0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.704+0000 7f59477d2740 1 -- 172.21.15.120:0/856343414 shutdown_connections 2024-09-19T09:12:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.704+0000 7f59477d2740 1 --2- 172.21.15.120:0/856343414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c1cb68370 0x562c1cb6a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.704+0000 7f59477d2740 1 --2- 172.21.15.120:0/856343414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c1cb632e0 0x562c1cb67da0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.704+0000 7f59477d2740 1 --2- 172.21.15.120:0/856343414 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c1cb629a0 0x562c1cb62da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.704+0000 7f59477d2740 1 -- 172.21.15.120:0/856343414 >> 172.21.15.120:0/856343414 conn(0x562c1cb5e010 msgr2=0x562c1cb5e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.704+0000 7f59477d2740 1 -- 172.21.15.120:0/856343414 shutdown_connections 2024-09-19T09:12:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.704+0000 7f59477d2740 1 -- 172.21.15.120:0/856343414 wait complete. 2024-09-19T09:12:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.705+0000 7f59477d2740 1 -- start start 2024-09-19T09:12:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.705+0000 7f59477d2740 1 Processor -- start 2024-09-19T09:12:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.705+0000 7f59477d2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c1cb629a0 0x562c1cb6cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.705+0000 7f59477d2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c1cb632e0 0x562c1cb6d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.705+0000 7f59477d2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c1cb68370 0x562c1cb740e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.705+0000 7f59477d2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562c1cb6b340 con 0x562c1cb632e0 2024-09-19T09:12:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.705+0000 7f59477d2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562c1ca4f290 con 0x562c1cb68370 2024-09-19T09:12:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.705+0000 7f59477d2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562c1cb6b4c0 con 0x562c1cb629a0 2024-09-19T09:12:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.705+0000 7f59477d2740 10 client.0 _ll_register_callbacks cb 0x562c1ca80140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.705+0000 7f593dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c1cb632e0 0x562c1cb6d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.705+0000 7f593e59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c1cb629a0 0x562c1cb6cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.705+0000 7f593dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c1cb632e0 0x562c1cb6d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47674/0 (socket says 172.21.15.120:47674) 2024-09-19T09:12:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.705+0000 7f593dd9b640 1 -- 172.21.15.120:0/1905571863 learned_addr learned my addr 172.21.15.120:0/1905571863 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.705+0000 7f593ed9d640 1 --2- 172.21.15.120:0/1905571863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c1cb68370 0x562c1cb740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.706+0000 7f593dd9b640 1 -- 172.21.15.120:0/1905571863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c1cb629a0 msgr2=0x562c1cb6cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.706+0000 7f593dd9b640 1 --2- 172.21.15.120:0/1905571863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c1cb629a0 0x562c1cb6cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.706+0000 7f593dd9b640 1 -- 172.21.15.120:0/1905571863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c1cb68370 msgr2=0x562c1cb740e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.706+0000 7f593dd9b640 1 --2- 172.21.15.120:0/1905571863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c1cb68370 0x562c1cb740e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.706+0000 7f593dd9b640 1 -- 172.21.15.120:0/1905571863 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c1c7d9210 con 0x562c1cb632e0 2024-09-19T09:12:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.706+0000 7f593ed9d640 1 --2- 172.21.15.120:0/1905571863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c1cb68370 0x562c1cb740e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:12:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.706+0000 7f593dd9b640 1 --2- 172.21.15.120:0/1905571863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c1cb632e0 0x562c1cb6d0a0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f59340c4ad0 tx=0x7f5934032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71737087ddf5dbee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.706+0000 7f5925ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.706+0000 7f5925ffb640 1 -- 172.21.15.120:0/1905571863 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934049070 con 0x562c1cb632e0 2024-09-19T09:12:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.706+0000 7f5925ffb640 1 -- 172.21.15.120:0/1905571863 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f59340b8a00 con 0x562c1cb632e0 2024-09-19T09:12:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.706+0000 7f59477d2740 1 -- 172.21.15.120:0/1905571863 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59300044b0 con 0x562c1cb632e0 2024-09-19T09:12:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.706+0000 7f5925ffb640 1 -- 172.21.15.120:0/1905571863 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f59340bc160 con 0x562c1cb632e0 2024-09-19T09:12:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.706+0000 7f59477d2740 10 client.18909 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.706+0000 7f59477d2740 1 -- 172.21.15.120:0/1905571863 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562c1c8a5620 con 0x562c1cb632e0 2024-09-19T09:12:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.706+0000 7f5925ffb640 1 -- 172.21.15.120:0/1905571863 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59300044b0 con 0x562c1cb632e0 2024-09-19T09:12:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.707+0000 7f5925ffb640 1 -- 172.21.15.120:0/1905571863 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f593403bc60 con 0x562c1cb632e0 2024-09-19T09:12:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.707+0000 7f59477d2740 10 client.18909 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.707+0000 7f59477d2740 10 client.18909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:12:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.707+0000 7f59477d2740 20 client.18909 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.707+0000 7f59477d2740 1 --2- 172.21.15.120:0/1905571863 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562c1cb78770 0x562c1cb7ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.707+0000 7f59477d2740 4 client.18909 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:12:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.707+0000 7f59477d2740 1 -- 172.21.15.120:0/1905571863 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c1cad3a20 con 0x562c1cb78770 2024-09-19T09:12:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.707+0000 7f593e59c640 1 --2- 172.21.15.120:0/1905571863 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562c1cb78770 0x562c1cb7ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.707+0000 7f5925ffb640 10 client.18909 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.707+0000 7f593e59c640 1 --2- 172.21.15.120:0/1905571863 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562c1cb78770 0x562c1cb7ab80 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b58860d842fd8f7b server_cookie=90bf3500af66f3a9 in_seq=0 out_seq=0 2024-09-19T09:12:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.709+0000 7f5925ffb640 1 -- 172.21.15.120:0/1905571863 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562c1cad3a20 con 0x562c1cb78770 2024-09-19T09:12:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.709+0000 7f5925ffb640 10 client.18909 handle_command_reply: tid=0 2024-09-19T09:12:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.709+0000 7f59477d2740 1 client.18909 shutdown 2024-09-19T09:12:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.709+0000 7f59477d2740 1 -- 172.21.15.120:0/1905571863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c1cb632e0 msgr2=0x562c1cb6d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.709+0000 7f59477d2740 1 --2- 172.21.15.120:0/1905571863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c1cb632e0 0x562c1cb6d0a0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f59340c4ad0 tx=0x7f5934032f90 comp rx=0 tx=0).stop 2024-09-19T09:12:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.709+0000 7f59477d2740 1 -- 172.21.15.120:0/1905571863 shutdown_connections 2024-09-19T09:12:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.709+0000 7f59477d2740 1 --2- 172.21.15.120:0/1905571863 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562c1cb78770 0x562c1cb7ab80 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.709+0000 7f59477d2740 1 --2- 172.21.15.120:0/1905571863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c1cb68370 0x562c1cb740e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.709+0000 7f5925ffb640 0 client.18909 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.709+0000 7f59477d2740 1 --2- 172.21.15.120:0/1905571863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c1cb632e0 0x562c1cb6d0a0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.709+0000 7f59477d2740 1 --2- 172.21.15.120:0/1905571863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c1cb629a0 0x562c1cb6cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.709+0000 7f59477d2740 1 -- 172.21.15.120:0/1905571863 >> 172.21.15.120:0/1905571863 conn(0x562c1cb5e010 msgr2=0x562c1cb5f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.709+0000 7f59477d2740 1 -- 172.21.15.120:0/1905571863 shutdown_connections 2024-09-19T09:12:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.709+0000 7f59477d2740 1 -- 172.21.15.120:0/1905571863 wait complete. 2024-09-19T09:12:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.709+0000 7f59477d2740 20 client.18909 trim_cache size 0 max 16384 2024-09-19T09:12:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.725+0000 7f59477d2740 1 Processor -- start 2024-09-19T09:12:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.725+0000 7f59477d2740 1 -- start start 2024-09-19T09:12:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.725+0000 7f59477d2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c1cb629a0 0x562c1cb5eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.725+0000 7f59477d2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c1cb632e0 0x562c1cb740a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.725+0000 7f59477d2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c1cb68370 0x562c1cb5f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.725+0000 7f59477d2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f59401a7fb0 con 0x562c1cb68370 2024-09-19T09:12:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.725+0000 7f59477d2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f59401a9af0 con 0x562c1cb632e0 2024-09-19T09:12:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.725+0000 7f59477d2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f59401a8680 con 0x562c1cb629a0 2024-09-19T09:12:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.726+0000 7f593e59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c1cb629a0 0x562c1cb5eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.726+0000 7f593ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c1cb68370 0x562c1cb5f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.726+0000 7f593e59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c1cb629a0 0x562c1cb5eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:47500/0 (socket says 172.21.15.120:47500) 2024-09-19T09:12:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.726+0000 7f593e59c640 1 -- 172.21.15.120:0/1995936174 learned_addr learned my addr 172.21.15.120:0/1995936174 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.726+0000 7f593dd9b640 1 --2- 172.21.15.120:0/1995936174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c1cb632e0 0x562c1cb740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.726+0000 7f593e59c640 1 -- 172.21.15.120:0/1995936174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c1cb632e0 msgr2=0x562c1cb740a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.726+0000 7f593e59c640 1 --2- 172.21.15.120:0/1995936174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c1cb632e0 0x562c1cb740a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.726+0000 7f593e59c640 1 -- 172.21.15.120:0/1995936174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c1cb68370 msgr2=0x562c1cb5f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.726+0000 7f593e59c640 1 --2- 172.21.15.120:0/1995936174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c1cb68370 0x562c1cb5f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.726+0000 7f593e59c640 1 -- 172.21.15.120:0/1995936174 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c1cb680d0 con 0x562c1cb629a0 2024-09-19T09:12:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.726+0000 7f593e59c640 1 --2- 172.21.15.120:0/1995936174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c1cb629a0 0x562c1cb5eec0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5930014040 tx=0x7f5930004930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fff1ff195181a28b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.726+0000 7f5927fff640 1 -- 172.21.15.120:0/1995936174 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5930004390 con 0x562c1cb629a0 2024-09-19T09:12:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.726+0000 7f5927fff640 1 -- 172.21.15.120:0/1995936174 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5930003ad0 con 0x562c1cb629a0 2024-09-19T09:12:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.726+0000 7f5927fff640 1 -- 172.21.15.120:0/1995936174 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593000f8e0 con 0x562c1cb629a0 2024-09-19T09:12:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.727+0000 7f59477d2740 1 -- 172.21.15.120:0/1995936174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c1cb629a0 msgr2=0x562c1cb5eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.727+0000 7f59477d2740 1 --2- 172.21.15.120:0/1995936174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c1cb629a0 0x562c1cb5eec0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5930014040 tx=0x7f5930004930 comp rx=0 tx=0).stop 2024-09-19T09:12:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.727+0000 7f59477d2740 1 -- 172.21.15.120:0/1995936174 shutdown_connections 2024-09-19T09:12:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.727+0000 7f59477d2740 1 --2- 172.21.15.120:0/1995936174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c1cb68370 0x562c1cb5f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.727+0000 7f59477d2740 1 --2- 172.21.15.120:0/1995936174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c1cb632e0 0x562c1cb740a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.727+0000 7f59477d2740 1 --2- 172.21.15.120:0/1995936174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c1cb629a0 0x562c1cb5eec0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.727+0000 7f59477d2740 1 -- 172.21.15.120:0/1995936174 >> 172.21.15.120:0/1995936174 conn(0x562c1cb5e010 msgr2=0x562c1cb64420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.727+0000 7f59477d2740 1 -- 172.21.15.120:0/1995936174 shutdown_connections 2024-09-19T09:12:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.727+0000 7f59477d2740 1 -- 172.21.15.120:0/1995936174 wait complete. 2024-09-19T09:12:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.727+0000 7f59477d2740 1 -- start start 2024-09-19T09:12:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.728+0000 7f59477d2740 1 Processor -- start 2024-09-19T09:12:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.728+0000 7f59477d2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c1cb629a0 0x562c1cb73820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.728+0000 7f59477d2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c1cb632e0 0x562c1cb73d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.728+0000 7f59477d2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c1cb68370 0x562c1cb719d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.728+0000 7f59477d2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f59401a9420 con 0x562c1cb632e0 2024-09-19T09:12:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.728+0000 7f59477d2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f59401a8d50 con 0x562c1cb68370 2024-09-19T09:12:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.728+0000 7f59477d2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f59401a5f70 con 0x562c1cb629a0 2024-09-19T09:12:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.728+0000 7f59477d2740 10 client.0 _ll_register_callbacks cb 0x7f5934005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.728+0000 7f593e59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c1cb629a0 0x562c1cb73820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.728+0000 7f593e59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c1cb629a0 0x562c1cb73820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:47516/0 (socket says 172.21.15.120:47516) 2024-09-19T09:12:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.728+0000 7f593e59c640 1 -- 172.21.15.120:0/3321630314 learned_addr learned my addr 172.21.15.120:0/3321630314 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.728+0000 7f593dd9b640 1 --2- 172.21.15.120:0/3321630314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c1cb632e0 0x562c1cb73d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.728+0000 7f593ed9d640 1 --2- 172.21.15.120:0/3321630314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c1cb68370 0x562c1cb719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.728+0000 7f593e59c640 1 -- 172.21.15.120:0/3321630314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c1cb68370 msgr2=0x562c1cb719d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.728+0000 7f593e59c640 1 --2- 172.21.15.120:0/3321630314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c1cb68370 0x562c1cb719d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.728+0000 7f593e59c640 1 -- 172.21.15.120:0/3321630314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c1cb632e0 msgr2=0x562c1cb73d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.728+0000 7f593e59c640 1 --2- 172.21.15.120:0/3321630314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c1cb632e0 0x562c1cb73d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.728+0000 7f593e59c640 1 -- 172.21.15.120:0/3321630314 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c1cb71fe0 con 0x562c1cb629a0 2024-09-19T09:12:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.728+0000 7f593e59c640 1 --2- 172.21.15.120:0/3321630314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c1cb629a0 0x562c1cb73820 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5930009f70 tx=0x7f5930003eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f145ad4665e670d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.728+0000 7f5925ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:12:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.729+0000 7f5925ffb640 1 -- 172.21.15.120:0/3321630314 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59300035f0 con 0x562c1cb629a0 2024-09-19T09:12:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.729+0000 7f5925ffb640 1 -- 172.21.15.120:0/3321630314 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f5930003790 con 0x562c1cb629a0 2024-09-19T09:12:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.729+0000 7f59477d2740 1 -- 172.21.15.120:0/3321630314 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5930003ad0 con 0x562c1cb629a0 2024-09-19T09:12:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.729+0000 7f5925ffb640 1 -- 172.21.15.120:0/3321630314 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f593000f660 con 0x562c1cb629a0 2024-09-19T09:12:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.729+0000 7f59477d2740 10 client.27859 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.729+0000 7f59477d2740 1 -- 172.21.15.120:0/3321630314 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562c1cbe7870 con 0x562c1cb629a0 2024-09-19T09:12:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.729+0000 7f5925ffb640 1 -- 172.21.15.120:0/3321630314 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5930003ad0 con 0x562c1cb629a0 2024-09-19T09:12:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.730+0000 7f5925ffb640 1 -- 172.21.15.120:0/3321630314 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f593000f800 con 0x562c1cb629a0 2024-09-19T09:12:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.730+0000 7f59477d2740 10 client.27859 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.730+0000 7f59477d2740 10 client.27859 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:12:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.730+0000 7f59477d2740 20 client.27859 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.730+0000 7f59477d2740 1 --2- 172.21.15.120:0/3321630314 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562c1cbe7e70 0x562c1cbea260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.730+0000 7f59477d2740 4 client.27859 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:12:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.730+0000 7f59477d2740 1 -- 172.21.15.120:0/3321630314 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562c1cacdd70 con 0x562c1cbe7e70 2024-09-19T09:12:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.730+0000 7f593dd9b640 1 --2- 172.21.15.120:0/3321630314 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562c1cbe7e70 0x562c1cbea260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.730+0000 7f5925ffb640 10 client.27859 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.730+0000 7f593dd9b640 1 --2- 172.21.15.120:0/3321630314 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562c1cbe7e70 0x562c1cbea260 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2a1e03b188c5a30 server_cookie=b2e094e4211e7a94 in_seq=0 out_seq=0 2024-09-19T09:12:31.868 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:12:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.909+0000 7f5925ffb640 1 -- 172.21.15.120:0/3321630314 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562c1cacdd70 con 0x562c1cbe7e70 2024-09-19T09:12:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.909+0000 7f5925ffb640 10 client.27859 handle_command_reply: tid=0 2024-09-19T09:12:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.909+0000 7f59477d2740 1 client.27859 shutdown 2024-09-19T09:12:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.909+0000 7f59477d2740 1 -- 172.21.15.120:0/3321630314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c1cb629a0 msgr2=0x562c1cb73820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.909+0000 7f59477d2740 1 --2- 172.21.15.120:0/3321630314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c1cb629a0 0x562c1cb73820 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5930009f70 tx=0x7f5930003eb0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.912+0000 7f59477d2740 1 -- 172.21.15.120:0/3321630314 shutdown_connections 2024-09-19T09:12:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.912+0000 7f59477d2740 1 --2- 172.21.15.120:0/3321630314 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562c1cbe7e70 0x562c1cbea260 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.912+0000 7f59477d2740 1 --2- 172.21.15.120:0/3321630314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562c1cb68370 0x562c1cb719d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.912+0000 7f59477d2740 1 --2- 172.21.15.120:0/3321630314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562c1cb632e0 0x562c1cb73d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.912+0000 7f59477d2740 1 --2- 172.21.15.120:0/3321630314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562c1cb629a0 0x562c1cb73820 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.912+0000 7f59477d2740 1 -- 172.21.15.120:0/3321630314 >> 172.21.15.120:0/3321630314 conn(0x562c1cb5e010 msgr2=0x562c1cb5c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.912+0000 7f5925ffb640 0 client.27859 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.912+0000 7f59477d2740 1 -- 172.21.15.120:0/3321630314 shutdown_connections 2024-09-19T09:12:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.912+0000 7f59477d2740 1 -- 172.21.15.120:0/3321630314 wait complete. 2024-09-19T09:12:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.913+0000 7f59477d2740 20 client.27859 trim_cache size 0 max 16384 2024-09-19T09:12:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:12:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:12:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:12:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:12:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.914+0000 7f5944959640 1 -- 172.21.15.120:0/4066362854 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f59280a32f0 msgr2=0x7f59280a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.914+0000 7f5944959640 1 --2- 172.21.15.120:0/4066362854 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f59280a32f0 0x7f59280a57b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5938005fd0 tx=0x7f5938009500 comp rx=0 tx=0).stop 2024-09-19T09:12:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.914+0000 7f5944959640 1 -- 172.21.15.120:0/4066362854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5940101c40 msgr2=0x7f59401971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.914+0000 7f5944959640 1 --2- 172.21.15.120:0/4066362854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5940101c40 0x7f59401971e0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f5934031440 tx=0x7f5934032ce0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.914+0000 7f5944959640 1 -- 172.21.15.120:0/4066362854 shutdown_connections 2024-09-19T09:12:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.914+0000 7f5944959640 1 --2- 172.21.15.120:0/4066362854 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f59280a32f0 0x7f59280a57b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.914+0000 7f5944959640 1 --2- 172.21.15.120:0/4066362854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5940102600 0x7f594019e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.914+0000 7f5944959640 1 --2- 172.21.15.120:0/4066362854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5940101c40 0x7f59401971e0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.914+0000 7f5944959640 1 --2- 172.21.15.120:0/4066362854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5940100a50 0x7f5940196ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.914+0000 7f5944959640 1 -- 172.21.15.120:0/4066362854 >> 172.21.15.120:0/4066362854 conn(0x7f59400fc1b0 msgr2=0x7f5940104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:31.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.915+0000 7f5944959640 1 -- 172.21.15.120:0/4066362854 shutdown_connections 2024-09-19T09:12:31.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:31.915+0000 7f5944959640 1 -- 172.21.15.120:0/4066362854 wait complete. 2024-09-19T09:12:32.069 INFO:tasks.workunit.client.0.smithi120.stdout: 1 946745 10.10 MB/sec execute 387 sec latency 103.589 ms 2024-09-19T09:12:32.148 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:12:32.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.659+0000 7fdaa0f6e640 1 -- 172.21.15.120:0/2703910748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda9c1027c0 msgr2=0x7fda9c106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:32.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.659+0000 7fdaa0f6e640 1 --2- 172.21.15.120:0/2703910748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda9c1027c0 0x7fda9c106810 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fda84009900 tx=0x7fda8402a410 comp rx=0 tx=0).stop 2024-09-19T09:12:32.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.659+0000 7fdaa0f6e640 1 -- 172.21.15.120:0/2703910748 shutdown_connections 2024-09-19T09:12:32.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.659+0000 7fdaa0f6e640 1 --2- 172.21.15.120:0/2703910748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda9c106f40 0x7fda9c10eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.659+0000 7fdaa0f6e640 1 --2- 172.21.15.120:0/2703910748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda9c1027c0 0x7fda9c106810 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.659+0000 7fdaa0f6e640 1 --2- 172.21.15.120:0/2703910748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda9c101e10 0x7fda9c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.659+0000 7fdaa0f6e640 1 -- 172.21.15.120:0/2703910748 >> 172.21.15.120:0/2703910748 conn(0x7fda9c0fd8d0 msgr2=0x7fda9c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:32.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.659+0000 7fdaa0f6e640 1 -- 172.21.15.120:0/2703910748 shutdown_connections 2024-09-19T09:12:32.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.660+0000 7fdaa0f6e640 1 -- 172.21.15.120:0/2703910748 wait complete. 2024-09-19T09:12:32.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.660+0000 7fdaa0f6e640 1 Processor -- start 2024-09-19T09:12:32.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.660+0000 7fdaa0f6e640 1 -- start start 2024-09-19T09:12:32.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.660+0000 7fdaa0f6e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda9c101e10 0x7fda9c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:32.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.661+0000 7fdaa0f6e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda9c1027c0 0x7fda9c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:32.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.661+0000 7fdaa0f6e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda9c106f40 0x7fda9c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:32.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.661+0000 7fdaa0f6e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c111fa0 con 0x7fda9c1027c0 2024-09-19T09:12:32.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.661+0000 7fdaa0f6e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c111e20 con 0x7fda9c101e10 2024-09-19T09:12:32.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.661+0000 7fdaa0f6e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c112120 con 0x7fda9c106f40 2024-09-19T09:12:32.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.661+0000 7fda93fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda9c1027c0 0x7fda9c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:32.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.661+0000 7fda93fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda9c1027c0 0x7fda9c1989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47718/0 (socket says 172.21.15.120:47718) 2024-09-19T09:12:32.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.661+0000 7fda93fff640 1 -- 172.21.15.120:0/849375662 learned_addr learned my addr 172.21.15.120:0/849375662 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:32.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.661+0000 7fda9a59c640 1 --2- 172.21.15.120:0/849375662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda9c101e10 0x7fda9c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:32.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.661+0000 7fda9ad9d640 1 --2- 172.21.15.120:0/849375662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda9c106f40 0x7fda9c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:32.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.661+0000 7fda9a59c640 1 -- 172.21.15.120:0/849375662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda9c106f40 msgr2=0x7fda9c19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:32.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.661+0000 7fda9a59c640 1 --2- 172.21.15.120:0/849375662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda9c106f40 0x7fda9c19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.661+0000 7fda9a59c640 1 -- 172.21.15.120:0/849375662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda9c1027c0 msgr2=0x7fda9c1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:32.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.661+0000 7fda9a59c640 1 --2- 172.21.15.120:0/849375662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda9c1027c0 0x7fda9c1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.661+0000 7fda9a59c640 1 -- 172.21.15.120:0/849375662 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda9c19d410 con 0x7fda9c101e10 2024-09-19T09:12:32.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.661+0000 7fda93fff640 1 --2- 172.21.15.120:0/849375662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda9c1027c0 0x7fda9c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:32.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.661+0000 7fda9a59c640 1 --2- 172.21.15.120:0/849375662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda9c101e10 0x7fda9c198460 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00e9f0 tx=0x7fda8c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:32.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.661+0000 7fda937fe640 1 -- 172.21.15.120:0/849375662 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8c00ccc0 con 0x7fda9c101e10 2024-09-19T09:12:32.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.662+0000 7fdaa0f6e640 1 -- 172.21.15.120:0/849375662 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda9c19d700 con 0x7fda9c101e10 2024-09-19T09:12:32.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.662+0000 7fda937fe640 1 -- 172.21.15.120:0/849375662 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda8c005020 con 0x7fda9c101e10 2024-09-19T09:12:32.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.662+0000 7fdaa0f6e640 1 -- 172.21.15.120:0/849375662 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda9c1a4fe0 con 0x7fda9c101e10 2024-09-19T09:12:32.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.662+0000 7fda937fe640 1 -- 172.21.15.120:0/849375662 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8c0049e0 con 0x7fda9c101e10 2024-09-19T09:12:32.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.664+0000 7fda937fe640 1 -- 172.21.15.120:0/849375662 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fda8c005560 con 0x7fda9c101e10 2024-09-19T09:12:32.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.664+0000 7fdaa3347740 1 Processor -- start 2024-09-19T09:12:32.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.664+0000 7fda937fe640 1 --2- 172.21.15.120:0/849375662 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fda880a3240 0x7fda880a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:32.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.664+0000 7fda937fe640 1 -- 172.21.15.120:0/849375662 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fda8c0cf880 con 0x7fda9c101e10 2024-09-19T09:12:32.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.664+0000 7fda9ad9d640 1 --2- 172.21.15.120:0/849375662 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fda880a3240 0x7fda880a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:32.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.665+0000 7fdaa3347740 1 -- start start 2024-09-19T09:12:32.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.665+0000 7fdaa3347740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622a7d76b40 0x5622a7d75090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:32.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.665+0000 7fdaa3347740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a7d755d0 0x5622a7d75a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:32.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.665+0000 7fdaa3347740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622a7d80370 0x5622a7d82760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:32.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.665+0000 7fdaa3347740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5622a7cdb2f0 con 0x5622a7d80370 2024-09-19T09:12:32.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.665+0000 7fdaa3347740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5622a7c82cf0 con 0x5622a7d76b40 2024-09-19T09:12:32.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.665+0000 7fdaa3347740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5622a7cd72b0 con 0x5622a7d755d0 2024-09-19T09:12:32.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.665+0000 7fda9a59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622a7d76b40 0x5622a7d75090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:32.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.665+0000 7fda9ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622a7d80370 0x5622a7d82760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:32.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.665+0000 7fda93fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a7d755d0 0x5622a7d75a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:32.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.665+0000 7fda9ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622a7d80370 0x5622a7d82760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47734/0 (socket says 172.21.15.120:47734) 2024-09-19T09:12:32.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.665+0000 7fda9ad9d640 1 -- 172.21.15.120:0/1115435067 learned_addr learned my addr 172.21.15.120:0/1115435067 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:32.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.665+0000 7fda9ad9d640 1 --2- 172.21.15.120:0/849375662 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fda880a3240 0x7fda880a5700 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fda94005fd0 tx=0x7fda94009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:32.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.665+0000 7fda93fff640 1 -- 172.21.15.120:0/1115435067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622a7d76b40 msgr2=0x5622a7d75090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:32.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.665+0000 7fda93fff640 1 --2- 172.21.15.120:0/1115435067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622a7d76b40 0x5622a7d75090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.665+0000 7fda93fff640 1 -- 172.21.15.120:0/1115435067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622a7d80370 msgr2=0x5622a7d82760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:32.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.665+0000 7fda93fff640 1 --2- 172.21.15.120:0/1115435067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622a7d80370 0x5622a7d82760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.665+0000 7fda93fff640 1 -- 172.21.15.120:0/1115435067 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622a7c673a0 con 0x5622a7d755d0 2024-09-19T09:12:32.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.665+0000 7fda9ad9d640 1 --2- 172.21.15.120:0/1115435067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622a7d80370 0x5622a7d82760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:32.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.665+0000 7fda93fff640 1 --2- 172.21.15.120:0/1115435067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a7d755d0 0x5622a7d75a10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fda84009230 tx=0x7fda84005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7ac7cb47bafc6fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:32.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.666+0000 7fda90ff9640 1 -- 172.21.15.120:0/1115435067 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda84040070 con 0x5622a7d755d0 2024-09-19T09:12:32.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.666+0000 7fda90ff9640 1 -- 172.21.15.120:0/1115435067 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda84031970 con 0x5622a7d755d0 2024-09-19T09:12:32.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.666+0000 7fda90ff9640 1 -- 172.21.15.120:0/1115435067 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda84035040 con 0x5622a7d755d0 2024-09-19T09:12:32.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.666+0000 7fdaa3347740 1 -- 172.21.15.120:0/1115435067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a7d755d0 msgr2=0x5622a7d75a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:32.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.666+0000 7fdaa3347740 1 --2- 172.21.15.120:0/1115435067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a7d755d0 0x5622a7d75a10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fda84009230 tx=0x7fda84005dd0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.666+0000 7fdaa3347740 1 -- 172.21.15.120:0/1115435067 shutdown_connections 2024-09-19T09:12:32.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.666+0000 7fdaa3347740 1 --2- 172.21.15.120:0/1115435067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622a7d80370 0x5622a7d82760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.666+0000 7fdaa3347740 1 --2- 172.21.15.120:0/1115435067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a7d755d0 0x5622a7d75a10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.666+0000 7fdaa3347740 1 --2- 172.21.15.120:0/1115435067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622a7d76b40 0x5622a7d75090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.666+0000 7fdaa3347740 1 -- 172.21.15.120:0/1115435067 >> 172.21.15.120:0/1115435067 conn(0x5622a7d3e310 msgr2=0x5622a7d71c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:32.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.667+0000 7fdaa3347740 1 -- 172.21.15.120:0/1115435067 shutdown_connections 2024-09-19T09:12:32.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.667+0000 7fdaa3347740 1 -- 172.21.15.120:0/1115435067 wait complete. 2024-09-19T09:12:32.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.667+0000 7fdaa3347740 1 -- start start 2024-09-19T09:12:32.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.667+0000 7fdaa3347740 1 Processor -- start 2024-09-19T09:12:32.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.667+0000 7fdaa3347740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a7d755d0 0x5622a7d7fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:32.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.667+0000 7fdaa3347740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622a7d76b40 0x5622a7d7d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:32.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.667+0000 7fdaa3347740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622a7d80370 0x5622a7d7d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:32.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.667+0000 7fdaa3347740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5622a7d83010 con 0x5622a7d76b40 2024-09-19T09:12:32.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.667+0000 7fdaa3347740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5622a7c96060 con 0x5622a7d80370 2024-09-19T09:12:32.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.667+0000 7fdaa3347740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5622a7d83190 con 0x5622a7d755d0 2024-09-19T09:12:32.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.667+0000 7fdaa3347740 10 client.0 _ll_register_callbacks cb 0x5622a79e98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:32.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.667+0000 7fda93fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622a7d76b40 0x5622a7d7d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:32.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.667+0000 7fda93fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622a7d76b40 0x5622a7d7d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47748/0 (socket says 172.21.15.120:47748) 2024-09-19T09:12:32.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.667+0000 7fda93fff640 1 -- 172.21.15.120:0/1343209456 learned_addr learned my addr 172.21.15.120:0/1343209456 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:32.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.667+0000 7fda9ad9d640 1 --2- 172.21.15.120:0/1343209456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622a7d80370 0x5622a7d7d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:32.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.667+0000 7fda9a59c640 1 --2- 172.21.15.120:0/1343209456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a7d755d0 0x5622a7d7fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:32.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.668+0000 7fda93fff640 1 -- 172.21.15.120:0/1343209456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a7d755d0 msgr2=0x5622a7d7fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:32.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.668+0000 7fda93fff640 1 --2- 172.21.15.120:0/1343209456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a7d755d0 0x5622a7d7fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.668+0000 7fda93fff640 1 -- 172.21.15.120:0/1343209456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622a7d80370 msgr2=0x5622a7d7d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:32.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.668+0000 7fda93fff640 1 --2- 172.21.15.120:0/1343209456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622a7d80370 0x5622a7d7d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.668+0000 7fda93fff640 1 -- 172.21.15.120:0/1343209456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622a7a4c3c0 con 0x5622a7d76b40 2024-09-19T09:12:32.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.668+0000 7fda9ad9d640 1 --2- 172.21.15.120:0/1343209456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622a7d80370 0x5622a7d7d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:32.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.668+0000 7fda93fff640 1 --2- 172.21.15.120:0/1343209456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622a7d76b40 0x5622a7d7d040 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fda84031550 tx=0x7fda8402abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308d4946555738b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:32.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.668+0000 7fda81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:32.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.668+0000 7fda81ffb640 1 -- 172.21.15.120:0/1343209456 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8404b070 con 0x5622a7d76b40 2024-09-19T09:12:32.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.668+0000 7fda81ffb640 1 -- 172.21.15.120:0/1343209456 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fda84004450 con 0x5622a7d76b40 2024-09-19T09:12:32.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.668+0000 7fdaa3347740 1 -- 172.21.15.120:0/1343209456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda84031970 con 0x5622a7d76b40 2024-09-19T09:12:32.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.668+0000 7fda81ffb640 1 -- 172.21.15.120:0/1343209456 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fda84057050 con 0x5622a7d76b40 2024-09-19T09:12:32.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.668+0000 7fdaa3347740 10 client.18927 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:32.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.668+0000 7fdaa3347740 1 -- 172.21.15.120:0/1343209456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5622a7d8eae0 con 0x5622a7d76b40 2024-09-19T09:12:32.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.668+0000 7fda81ffb640 1 -- 172.21.15.120:0/1343209456 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda84031970 con 0x5622a7d76b40 2024-09-19T09:12:32.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.669+0000 7fda81ffb640 1 -- 172.21.15.120:0/1343209456 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fda8403c6a0 con 0x5622a7d76b40 2024-09-19T09:12:32.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.669+0000 7fdaa3347740 10 client.18927 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:32.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.669+0000 7fdaa3347740 10 client.18927 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:12:32.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.669+0000 7fdaa3347740 20 client.18927 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:32.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.669+0000 7fdaa3347740 1 --2- 172.21.15.120:0/1343209456 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5622a7d8fe30 0x5622a7d92220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:32.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.669+0000 7fdaa3347740 4 client.18927 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:12:32.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.669+0000 7fdaa3347740 1 -- 172.21.15.120:0/1343209456 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622a7ce8860 con 0x5622a7d8fe30 2024-09-19T09:12:32.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.669+0000 7fda9a59c640 1 --2- 172.21.15.120:0/1343209456 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5622a7d8fe30 0x5622a7d92220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:32.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.669+0000 7fda81ffb640 10 client.18927 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:12:32.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.669+0000 7fda9a59c640 1 --2- 172.21.15.120:0/1343209456 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5622a7d8fe30 0x5622a7d92220 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9fe5611018fbefd4 server_cookie=3818fe14813eac44 in_seq=0 out_seq=0 2024-09-19T09:12:32.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.670+0000 7fda81ffb640 1 -- 172.21.15.120:0/1343209456 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5622a7ce8860 con 0x5622a7d8fe30 2024-09-19T09:12:32.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.670+0000 7fda81ffb640 10 client.18927 handle_command_reply: tid=0 2024-09-19T09:12:32.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.670+0000 7fdaa3347740 1 client.18927 shutdown 2024-09-19T09:12:32.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.670+0000 7fdaa3347740 1 -- 172.21.15.120:0/1343209456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622a7d76b40 msgr2=0x5622a7d7d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:32.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.670+0000 7fdaa3347740 1 --2- 172.21.15.120:0/1343209456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622a7d76b40 0x5622a7d7d040 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fda84031550 tx=0x7fda8402abc0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.670+0000 7fdaa3347740 1 -- 172.21.15.120:0/1343209456 shutdown_connections 2024-09-19T09:12:32.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.670+0000 7fdaa3347740 1 --2- 172.21.15.120:0/1343209456 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5622a7d8fe30 0x5622a7d92220 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.670+0000 7fdaa3347740 1 --2- 172.21.15.120:0/1343209456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622a7d80370 0x5622a7d7d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.670+0000 7fda81ffb640 0 client.18927 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:12:32.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.670+0000 7fdaa3347740 1 --2- 172.21.15.120:0/1343209456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622a7d76b40 0x5622a7d7d040 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.670+0000 7fdaa3347740 1 --2- 172.21.15.120:0/1343209456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a7d755d0 0x5622a7d7fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.670+0000 7fdaa3347740 1 -- 172.21.15.120:0/1343209456 >> 172.21.15.120:0/1343209456 conn(0x5622a7d3e310 msgr2=0x5622a7d77960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:32.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.670+0000 7fdaa3347740 1 -- 172.21.15.120:0/1343209456 shutdown_connections 2024-09-19T09:12:32.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.670+0000 7fdaa3347740 1 -- 172.21.15.120:0/1343209456 wait complete. 2024-09-19T09:12:32.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.670+0000 7fdaa3347740 20 client.18927 trim_cache size 0 max 16384 2024-09-19T09:12:32.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.685+0000 7fdaa3347740 1 Processor -- start 2024-09-19T09:12:32.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.685+0000 7fdaa3347740 1 -- start start 2024-09-19T09:12:32.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.686+0000 7fdaa3347740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622a7d755d0 0x5622a7d76300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:32.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.686+0000 7fdaa3347740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622a7d76b40 0x5622a7d6cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:32.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.686+0000 7fdaa3347740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a7d80370 0x5622a7d7efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:32.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.686+0000 7fdaa3347740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c1ad5f0 con 0x5622a7d755d0 2024-09-19T09:12:32.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.686+0000 7fdaa3347740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c1af130 con 0x5622a7d76b40 2024-09-19T09:12:32.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.686+0000 7fdaa3347740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c1adcc0 con 0x5622a7d80370 2024-09-19T09:12:32.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.686+0000 7fda9a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622a7d755d0 0x5622a7d76300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:32.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.686+0000 7fda93fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622a7d76b40 0x5622a7d6cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:32.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.686+0000 7fda9ad9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a7d80370 0x5622a7d7efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:32.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.686+0000 7fda93fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622a7d76b40 0x5622a7d6cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52782/0 (socket says 172.21.15.120:52782) 2024-09-19T09:12:32.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.686+0000 7fda9ad9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a7d80370 0x5622a7d7efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:47546/0 (socket says 172.21.15.120:47546) 2024-09-19T09:12:32.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.686+0000 7fda93fff640 1 -- 172.21.15.120:0/2305298340 learned_addr learned my addr 172.21.15.120:0/2305298340 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:32.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.686+0000 7fda9ad9d640 1 -- 172.21.15.120:0/2305298340 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622a7d76b40 msgr2=0x5622a7d6cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:32.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.686+0000 7fda9ad9d640 1 --2- 172.21.15.120:0/2305298340 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622a7d76b40 0x5622a7d6cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.686+0000 7fda9ad9d640 1 -- 172.21.15.120:0/2305298340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622a7d755d0 msgr2=0x5622a7d76300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:32.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.686+0000 7fda9ad9d640 1 --2- 172.21.15.120:0/2305298340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622a7d755d0 0x5622a7d76300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.686+0000 7fda9ad9d640 1 -- 172.21.15.120:0/2305298340 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622a7d72960 con 0x5622a7d80370 2024-09-19T09:12:32.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.686+0000 7fda93fff640 1 --2- 172.21.15.120:0/2305298340 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622a7d76b40 0x5622a7d6cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:32.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.686+0000 7fda9ad9d640 1 --2- 172.21.15.120:0/2305298340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a7d80370 0x5622a7d7efb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fda94013010 tx=0x7fda94014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef84997b16e21660 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:32.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.686+0000 7fda83fff640 1 -- 172.21.15.120:0/2305298340 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9401f8d0 con 0x5622a7d80370 2024-09-19T09:12:32.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.687+0000 7fda83fff640 1 -- 172.21.15.120:0/2305298340 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda9401e3f0 con 0x5622a7d80370 2024-09-19T09:12:32.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.687+0000 7fda83fff640 1 -- 172.21.15.120:0/2305298340 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9400e3b0 con 0x5622a7d80370 2024-09-19T09:12:32.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.687+0000 7fdaa3347740 1 -- 172.21.15.120:0/2305298340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a7d80370 msgr2=0x5622a7d7efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:32.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.687+0000 7fdaa3347740 1 --2- 172.21.15.120:0/2305298340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a7d80370 0x5622a7d7efb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fda94013010 tx=0x7fda94014000 comp rx=0 tx=0).stop 2024-09-19T09:12:32.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.687+0000 7fdaa3347740 1 -- 172.21.15.120:0/2305298340 shutdown_connections 2024-09-19T09:12:32.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.687+0000 7fdaa3347740 1 --2- 172.21.15.120:0/2305298340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a7d80370 0x5622a7d7efb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.687+0000 7fdaa3347740 1 --2- 172.21.15.120:0/2305298340 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622a7d76b40 0x5622a7d6cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.687+0000 7fdaa3347740 1 --2- 172.21.15.120:0/2305298340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622a7d755d0 0x5622a7d76300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.687+0000 7fdaa3347740 1 -- 172.21.15.120:0/2305298340 >> 172.21.15.120:0/2305298340 conn(0x5622a7d3e310 msgr2=0x5622a7d7e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:32.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.687+0000 7fdaa3347740 1 -- 172.21.15.120:0/2305298340 shutdown_connections 2024-09-19T09:12:32.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.687+0000 7fdaa3347740 1 -- 172.21.15.120:0/2305298340 wait complete. 2024-09-19T09:12:32.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.688+0000 7fdaa3347740 1 -- start start 2024-09-19T09:12:32.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.688+0000 7fdaa3347740 1 Processor -- start 2024-09-19T09:12:32.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.688+0000 7fdaa3347740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a7d755d0 0x5622a7d88980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:32.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.688+0000 7fdaa3347740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622a7d76b40 0x5622a7d79bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:32.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.688+0000 7fdaa3347740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622a7d80370 0x5622a7d7a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:32.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.688+0000 7fdaa3347740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c1aea60 con 0x5622a7d76b40 2024-09-19T09:12:32.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.688+0000 7fdaa3347740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c1ae390 con 0x5622a7d80370 2024-09-19T09:12:32.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.688+0000 7fdaa3347740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c1ab5b0 con 0x5622a7d755d0 2024-09-19T09:12:32.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.688+0000 7fdaa3347740 10 client.0 _ll_register_callbacks cb 0x7fda94009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:32.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.688+0000 7fda93fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622a7d76b40 0x5622a7d79bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:32.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.688+0000 7fda9a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a7d755d0 0x5622a7d88980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:32.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.688+0000 7fda9a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a7d755d0 0x5622a7d88980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:47548/0 (socket says 172.21.15.120:47548) 2024-09-19T09:12:32.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.688+0000 7fda93fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622a7d76b40 0x5622a7d79bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47774/0 (socket says 172.21.15.120:47774) 2024-09-19T09:12:32.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.688+0000 7fda9a59c640 1 -- 172.21.15.120:0/1470079887 learned_addr learned my addr 172.21.15.120:0/1470079887 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:32.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.689+0000 7fda9ad9d640 1 --2- 172.21.15.120:0/1470079887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622a7d80370 0x5622a7d7a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:32.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.689+0000 7fda9a59c640 1 -- 172.21.15.120:0/1470079887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622a7d80370 msgr2=0x5622a7d7a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:32.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.689+0000 7fda9a59c640 1 --2- 172.21.15.120:0/1470079887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622a7d80370 0x5622a7d7a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.689+0000 7fda9a59c640 1 -- 172.21.15.120:0/1470079887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622a7d76b40 msgr2=0x5622a7d79bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:32.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.689+0000 7fda9a59c640 1 --2- 172.21.15.120:0/1470079887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622a7d76b40 0x5622a7d79bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.689+0000 7fda9a59c640 1 -- 172.21.15.120:0/1470079887 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622a7d7a750 con 0x5622a7d755d0 2024-09-19T09:12:32.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.689+0000 7fda93fff640 1 --2- 172.21.15.120:0/1470079887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622a7d76b40 0x5622a7d79bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:12:32.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.689+0000 7fda9a59c640 1 --2- 172.21.15.120:0/1470079887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a7d755d0 0x5622a7d88980 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fda8c098300 tx=0x7fda8c09c690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a366b7244b6b1134 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:32.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.689+0000 7fda81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:12:32.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.689+0000 7fda81ffb640 1 -- 172.21.15.120:0/1470079887 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8c07f060 con 0x5622a7d755d0 2024-09-19T09:12:32.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.689+0000 7fda81ffb640 1 -- 172.21.15.120:0/1470079887 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fda8c09a1a0 con 0x5622a7d755d0 2024-09-19T09:12:32.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.689+0000 7fdaa3347740 1 -- 172.21.15.120:0/1470079887 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda9401e3f0 con 0x5622a7d755d0 2024-09-19T09:12:32.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.689+0000 7fda81ffb640 1 -- 172.21.15.120:0/1470079887 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fda8c01c040 con 0x5622a7d755d0 2024-09-19T09:12:32.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.689+0000 7fdaa3347740 10 client.27877 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:32.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.689+0000 7fdaa3347740 1 -- 172.21.15.120:0/1470079887 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5622a7df9820 con 0x5622a7d755d0 2024-09-19T09:12:32.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.689+0000 7fda81ffb640 1 -- 172.21.15.120:0/1470079887 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda9401e3f0 con 0x5622a7d755d0 2024-09-19T09:12:32.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.690+0000 7fda81ffb640 1 -- 172.21.15.120:0/1470079887 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fda8c084070 con 0x5622a7d755d0 2024-09-19T09:12:32.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.690+0000 7fdaa3347740 10 client.27877 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:32.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.690+0000 7fdaa3347740 10 client.27877 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:12:32.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.690+0000 7fdaa3347740 20 client.27877 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:32.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.690+0000 7fdaa3347740 1 --2- 172.21.15.120:0/1470079887 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5622a7df9f50 0x5622a7dfc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:32.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.690+0000 7fdaa3347740 4 client.27877 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:12:32.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.690+0000 7fdaa3347740 1 -- 172.21.15.120:0/1470079887 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5622a7ce2d70 con 0x5622a7df9f50 2024-09-19T09:12:32.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.690+0000 7fda93fff640 1 --2- 172.21.15.120:0/1470079887 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5622a7df9f50 0x5622a7dfc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:32.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.690+0000 7fda93fff640 1 --2- 172.21.15.120:0/1470079887 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5622a7df9f50 0x5622a7dfc370 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=602200b0b0302f5f server_cookie=d2b450f9ad927d3d in_seq=0 out_seq=0 2024-09-19T09:12:32.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.690+0000 7fda81ffb640 10 client.27877 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:12:32.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.690+0000 7fda81ffb640 1 -- 172.21.15.120:0/1470079887 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5622a7ce2d70 con 0x5622a7df9f50 2024-09-19T09:12:32.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.690+0000 7fda81ffb640 10 client.27877 handle_command_reply: tid=0 2024-09-19T09:12:32.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.690+0000 7fdaa3347740 1 client.27877 shutdown 2024-09-19T09:12:32.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.690+0000 7fdaa3347740 1 -- 172.21.15.120:0/1470079887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a7d755d0 msgr2=0x5622a7d88980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:32.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.690+0000 7fdaa3347740 1 --2- 172.21.15.120:0/1470079887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a7d755d0 0x5622a7d88980 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fda8c098300 tx=0x7fda8c09c690 comp rx=0 tx=0).stop 2024-09-19T09:12:32.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.691+0000 7fdaa3347740 1 -- 172.21.15.120:0/1470079887 shutdown_connections 2024-09-19T09:12:32.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.691+0000 7fdaa3347740 1 --2- 172.21.15.120:0/1470079887 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5622a7df9f50 0x5622a7dfc370 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.691+0000 7fdaa3347740 1 --2- 172.21.15.120:0/1470079887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622a7d80370 0x5622a7d7a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.691+0000 7fda81ffb640 0 client.27877 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:12:32.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.691+0000 7fdaa3347740 1 --2- 172.21.15.120:0/1470079887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622a7d76b40 0x5622a7d79bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.691+0000 7fdaa3347740 1 --2- 172.21.15.120:0/1470079887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a7d755d0 0x5622a7d88980 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.691+0000 7fdaa3347740 1 -- 172.21.15.120:0/1470079887 >> 172.21.15.120:0/1470079887 conn(0x5622a7d3e310 msgr2=0x5622a7d70d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:32.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.691+0000 7fdaa3347740 1 -- 172.21.15.120:0/1470079887 shutdown_connections 2024-09-19T09:12:32.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.691+0000 7fdaa3347740 1 -- 172.21.15.120:0/1470079887 wait complete. 2024-09-19T09:12:32.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.691+0000 7fdaa3347740 20 client.27877 trim_cache size 0 max 16384 2024-09-19T09:12:32.707 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:12:32.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.691+0000 7fdaa0f6e640 1 -- 172.21.15.120:0/849375662 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fda880a3240 msgr2=0x7fda880a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:32.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.691+0000 7fdaa0f6e640 1 --2- 172.21.15.120:0/849375662 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fda880a3240 0x7fda880a5700 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fda94005fd0 tx=0x7fda94009500 comp rx=0 tx=0).stop 2024-09-19T09:12:32.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.691+0000 7fdaa0f6e640 1 -- 172.21.15.120:0/849375662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda9c101e10 msgr2=0x7fda9c198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:32.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.691+0000 7fdaa0f6e640 1 --2- 172.21.15.120:0/849375662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda9c101e10 0x7fda9c198460 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00e9f0 tx=0x7fda8c00eeb0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.691+0000 7fdaa0f6e640 1 -- 172.21.15.120:0/849375662 shutdown_connections 2024-09-19T09:12:32.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.691+0000 7fdaa0f6e640 1 --2- 172.21.15.120:0/849375662 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fda880a3240 0x7fda880a5700 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.691+0000 7fdaa0f6e640 1 --2- 172.21.15.120:0/849375662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda9c106f40 0x7fda9c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.692+0000 7fdaa0f6e640 1 --2- 172.21.15.120:0/849375662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda9c1027c0 0x7fda9c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.692+0000 7fdaa0f6e640 1 --2- 172.21.15.120:0/849375662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda9c101e10 0x7fda9c198460 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:32.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.692+0000 7fdaa0f6e640 1 -- 172.21.15.120:0/849375662 >> 172.21.15.120:0/849375662 conn(0x7fda9c0fd8d0 msgr2=0x7fda9c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:32.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.692+0000 7fdaa0f6e640 1 -- 172.21.15.120:0/849375662 shutdown_connections 2024-09-19T09:12:32.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:32.692+0000 7fdaa0f6e640 1 -- 172.21.15.120:0/849375662 wait complete. 2024-09-19T09:12:33.066 INFO:tasks.workunit.client.0.smithi120.stdout: 1 949555 10.12 MB/sec execute 388 sec latency 18.734 ms 2024-09-19T09:12:33.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:32 smithi196 ceph-mon[35150]: pgmap v633: 97 pgs: 97 active+clean; 37 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.5 MiB/s wr, 355 op/s 2024-09-19T09:12:33.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:32 smithi153 ceph-mon[35071]: pgmap v633: 97 pgs: 97 active+clean; 37 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.5 MiB/s wr, 355 op/s 2024-09-19T09:12:33.315 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:32 smithi120 ceph-mon[31150]: pgmap v633: 97 pgs: 97 active+clean; 37 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.5 MiB/s wr, 355 op/s 2024-09-19T09:12:33.496 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:12:33.803 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:12:34.066 INFO:tasks.workunit.client.0.smithi120.stdout: 1 952629 10.14 MB/sec execute 389 sec latency 53.808 ms 2024-09-19T09:12:34.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.309+0000 7f11ee0b2640 1 -- 172.21.15.120:0/3385660481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11e81008c0 msgr2=0x7f11e8100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:34.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.309+0000 7f11ee0b2640 1 --2- 172.21.15.120:0/3385660481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11e81008c0 0x7f11e8100ca0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009900 tx=0x7f11d802a410 comp rx=0 tx=0).stop 2024-09-19T09:12:34.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.309+0000 7f11ee0b2640 1 -- 172.21.15.120:0/3385660481 shutdown_connections 2024-09-19T09:12:34.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.309+0000 7f11ee0b2640 1 --2- 172.21.15.120:0/3385660481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11e8105ec0 0x7f11e8106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.309+0000 7f11ee0b2640 1 --2- 172.21.15.120:0/3385660481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11e81011e0 0x7f11e8105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.309+0000 7f11ee0b2640 1 --2- 172.21.15.120:0/3385660481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11e81008c0 0x7f11e8100ca0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.309+0000 7f11ee0b2640 1 -- 172.21.15.120:0/3385660481 >> 172.21.15.120:0/3385660481 conn(0x7f11e80fc7d0 msgr2=0x7f11e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:34.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.309+0000 7f11ee0b2640 1 -- 172.21.15.120:0/3385660481 shutdown_connections 2024-09-19T09:12:34.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.309+0000 7f11ee0b2640 1 -- 172.21.15.120:0/3385660481 wait complete. 2024-09-19T09:12:34.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.309+0000 7f11ee0b2640 1 Processor -- start 2024-09-19T09:12:34.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.310+0000 7f11ee0b2640 1 -- start start 2024-09-19T09:12:34.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.310+0000 7f11ee0b2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11e81008c0 0x7f11e81a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:34.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.310+0000 7f11ee0b2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11e81011e0 0x7f11e81a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:34.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.310+0000 7f11ee0b2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11e8105ec0 0x7f11e819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:34.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.310+0000 7f11ee0b2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f11e8118630 con 0x7f11e81011e0 2024-09-19T09:12:34.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.310+0000 7f11ee0b2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f11e81184b0 con 0x7f11e81008c0 2024-09-19T09:12:34.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.310+0000 7f11ee0b2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f11e81187b0 con 0x7f11e8105ec0 2024-09-19T09:12:34.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.310+0000 7f11e6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11e81011e0 0x7f11e81a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:34.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.310+0000 7f11e6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11e81011e0 0x7f11e81a4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47806/0 (socket says 172.21.15.120:47806) 2024-09-19T09:12:34.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.310+0000 7f11e6ffd640 1 -- 172.21.15.120:0/22595101 learned_addr learned my addr 172.21.15.120:0/22595101 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:34.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.310+0000 7f11e77fe640 1 --2- 172.21.15.120:0/22595101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11e81008c0 0x7f11e81a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:34.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.310+0000 7f11e7fff640 1 --2- 172.21.15.120:0/22595101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11e8105ec0 0x7f11e819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:34.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.310+0000 7f11e6ffd640 1 -- 172.21.15.120:0/22595101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11e8105ec0 msgr2=0x7f11e819e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:34.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.310+0000 7f11e6ffd640 1 --2- 172.21.15.120:0/22595101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11e8105ec0 0x7f11e819e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.310+0000 7f11e6ffd640 1 -- 172.21.15.120:0/22595101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11e81008c0 msgr2=0x7f11e81a4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:34.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.310+0000 7f11e6ffd640 1 --2- 172.21.15.120:0/22595101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11e81008c0 0x7f11e81a4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.310+0000 7f11e6ffd640 1 -- 172.21.15.120:0/22595101 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e819f070 con 0x7f11e81011e0 2024-09-19T09:12:34.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.311+0000 7f11e6ffd640 1 --2- 172.21.15.120:0/22595101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11e81011e0 0x7f11e81a4cc0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f11dc00e9f0 tx=0x7f11dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:34.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.311+0000 7f11e4ff9640 1 -- 172.21.15.120:0/22595101 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc00cd80 con 0x7f11e81011e0 2024-09-19T09:12:34.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.311+0000 7f11ee0b2640 1 -- 172.21.15.120:0/22595101 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e819f360 con 0x7f11e81011e0 2024-09-19T09:12:34.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.311+0000 7f11e4ff9640 1 -- 172.21.15.120:0/22595101 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11dc010910 con 0x7f11e81011e0 2024-09-19T09:12:34.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.311+0000 7f11e4ff9640 1 -- 172.21.15.120:0/22595101 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc0188f0 con 0x7f11e81011e0 2024-09-19T09:12:34.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.311+0000 7f11ee0b2640 1 -- 172.21.15.120:0/22595101 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e81ab650 con 0x7f11e81011e0 2024-09-19T09:12:34.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.313+0000 7f11e4ff9640 1 -- 172.21.15.120:0/22595101 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f11dc010ac0 con 0x7f11e81011e0 2024-09-19T09:12:34.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.314+0000 7f11e4ff9640 1 --2- 172.21.15.120:0/22595101 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f11d00a32c0 0x7f11d00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:34.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.314+0000 7f11e4ff9640 1 -- 172.21.15.120:0/22595101 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f11dc0cfc30 con 0x7f11e81011e0 2024-09-19T09:12:34.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.315+0000 7f11f048b740 1 Processor -- start 2024-09-19T09:12:34.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.315+0000 7f11f048b740 1 -- start start 2024-09-19T09:12:34.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.315+0000 7f11e77fe640 1 --2- 172.21.15.120:0/22595101 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f11d00a32c0 0x7f11d00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:34.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.315+0000 7f11f048b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eaefe60b40 0x55eaefe5f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:34.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.315+0000 7f11f048b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaefe5f5d0 0x55eaefe5fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:34.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.315+0000 7f11f048b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eaefe6a370 0x55eaefe6c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:34.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.315+0000 7f11f048b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55eaefdc52f0 con 0x55eaefe60b40 2024-09-19T09:12:34.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.315+0000 7f11f048b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55eaefd6ccf0 con 0x55eaefe6a370 2024-09-19T09:12:34.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.315+0000 7f11f048b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55eaefdc12b0 con 0x55eaefe5f5d0 2024-09-19T09:12:34.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.315+0000 7f11e6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaefe5f5d0 0x55eaefe5fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:34.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.315+0000 7f11e6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaefe5f5d0 0x55eaefe5fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:47586/0 (socket says 172.21.15.120:47586) 2024-09-19T09:12:34.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.315+0000 7f11e6ffd640 1 -- 172.21.15.120:0/204589065 learned_addr learned my addr 172.21.15.120:0/204589065 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:34.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.315+0000 7f11e7fff640 1 --2- 172.21.15.120:0/204589065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eaefe6a370 0x55eaefe6c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:34.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.315+0000 7f11e77fe640 1 --2- 172.21.15.120:0/204589065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eaefe60b40 0x55eaefe5f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:34.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.316+0000 7f11e6ffd640 1 -- 172.21.15.120:0/204589065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eaefe6a370 msgr2=0x55eaefe6c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:34.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.316+0000 7f11e6ffd640 1 --2- 172.21.15.120:0/204589065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eaefe6a370 0x55eaefe6c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.316+0000 7f11e6ffd640 1 -- 172.21.15.120:0/204589065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eaefe60b40 msgr2=0x55eaefe5f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:34.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.316+0000 7f11e6ffd640 1 --2- 172.21.15.120:0/204589065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eaefe60b40 0x55eaefe5f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.316+0000 7f11e6ffd640 1 -- 172.21.15.120:0/204589065 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eaefd513a0 con 0x55eaefe5f5d0 2024-09-19T09:12:34.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.316+0000 7f11e77fe640 1 --2- 172.21.15.120:0/22595101 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f11d00a32c0 0x7f11d00a5780 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f11e819fa80 tx=0x7f11d8004370 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:34.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.316+0000 7f11e6ffd640 1 --2- 172.21.15.120:0/204589065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaefe5f5d0 0x55eaefe5fa10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f11dc098760 tx=0x7f11dc09bf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5824633399496865 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:34.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.316+0000 7f11cdffb640 1 -- 172.21.15.120:0/204589065 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc0a4ec0 con 0x55eaefe5f5d0 2024-09-19T09:12:34.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.316+0000 7f11cdffb640 1 -- 172.21.15.120:0/204589065 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11dc09a600 con 0x55eaefe5f5d0 2024-09-19T09:12:34.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.316+0000 7f11cdffb640 1 -- 172.21.15.120:0/204589065 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc0a0380 con 0x55eaefe5f5d0 2024-09-19T09:12:34.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.317+0000 7f11f048b740 1 -- 172.21.15.120:0/204589065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaefe5f5d0 msgr2=0x55eaefe5fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:34.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.317+0000 7f11f048b740 1 --2- 172.21.15.120:0/204589065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaefe5f5d0 0x55eaefe5fa10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f11dc098760 tx=0x7f11dc09bf70 comp rx=0 tx=0).stop 2024-09-19T09:12:34.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.317+0000 7f11f048b740 1 -- 172.21.15.120:0/204589065 shutdown_connections 2024-09-19T09:12:34.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.317+0000 7f11f048b740 1 --2- 172.21.15.120:0/204589065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eaefe6a370 0x55eaefe6c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.317+0000 7f11f048b740 1 --2- 172.21.15.120:0/204589065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaefe5f5d0 0x55eaefe5fa10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.317+0000 7f11f048b740 1 --2- 172.21.15.120:0/204589065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eaefe60b40 0x55eaefe5f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.317+0000 7f11f048b740 1 -- 172.21.15.120:0/204589065 >> 172.21.15.120:0/204589065 conn(0x55eaefe28310 msgr2=0x55eaefe5bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:34.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.317+0000 7f11f048b740 1 -- 172.21.15.120:0/204589065 shutdown_connections 2024-09-19T09:12:34.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.317+0000 7f11f048b740 1 -- 172.21.15.120:0/204589065 wait complete. 2024-09-19T09:12:34.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.318+0000 7f11f048b740 1 -- start start 2024-09-19T09:12:34.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.318+0000 7f11f048b740 1 Processor -- start 2024-09-19T09:12:34.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.318+0000 7f11f048b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaefe5f5d0 0x55eaefe69ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:34.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.318+0000 7f11f048b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eaefe60b40 0x55eaefe66f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:34.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.318+0000 7f11f048b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eaefe6a370 0x55eaefe674f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:34.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.318+0000 7f11f048b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55eaefe6cfb0 con 0x55eaefe60b40 2024-09-19T09:12:34.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.318+0000 7f11f048b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55eaefd80060 con 0x55eaefe6a370 2024-09-19T09:12:34.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.318+0000 7f11f048b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55eaefe6d130 con 0x55eaefe5f5d0 2024-09-19T09:12:34.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.318+0000 7f11f048b740 10 client.0 _ll_register_callbacks cb 0x55eaefad38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:34.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.318+0000 7f11e6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eaefe60b40 0x55eaefe66f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:34.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.318+0000 7f11e77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaefe5f5d0 0x55eaefe69ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:34.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.318+0000 7f11e7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eaefe6a370 0x55eaefe674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:34.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.318+0000 7f11e77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaefe5f5d0 0x55eaefe69ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:47600/0 (socket says 172.21.15.120:47600) 2024-09-19T09:12:34.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.318+0000 7f11e6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eaefe60b40 0x55eaefe66f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47820/0 (socket says 172.21.15.120:47820) 2024-09-19T09:12:34.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.318+0000 7f11e77fe640 1 -- 172.21.15.120:0/4078617441 learned_addr learned my addr 172.21.15.120:0/4078617441 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:34.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.319+0000 7f11e77fe640 1 -- 172.21.15.120:0/4078617441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eaefe6a370 msgr2=0x55eaefe674f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:34.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.319+0000 7f11e77fe640 1 --2- 172.21.15.120:0/4078617441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eaefe6a370 0x55eaefe674f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.319+0000 7f11e77fe640 1 -- 172.21.15.120:0/4078617441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eaefe60b40 msgr2=0x55eaefe66f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:34.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.319+0000 7f11e77fe640 1 --2- 172.21.15.120:0/4078617441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eaefe60b40 0x55eaefe66f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.319+0000 7f11e77fe640 1 -- 172.21.15.120:0/4078617441 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eaefb363c0 con 0x55eaefe5f5d0 2024-09-19T09:12:34.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.319+0000 7f11e77fe640 1 --2- 172.21.15.120:0/4078617441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaefe5f5d0 0x55eaefe69ce0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009230 tx=0x7f11d802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fea96d8ad00ba289 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:34.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.319+0000 7f11c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:12:34.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.319+0000 7f11c6ffd640 1 -- 172.21.15.120:0/4078617441 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d8035050 con 0x55eaefe5f5d0 2024-09-19T09:12:34.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.319+0000 7f11c6ffd640 1 -- 172.21.15.120:0/4078617441 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f11d8031c90 con 0x55eaefe5f5d0 2024-09-19T09:12:34.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.319+0000 7f11f048b740 1 -- 172.21.15.120:0/4078617441 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11dc09a600 con 0x55eaefe5f5d0 2024-09-19T09:12:34.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.319+0000 7f11c6ffd640 1 -- 172.21.15.120:0/4078617441 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f11d80035f0 con 0x55eaefe5f5d0 2024-09-19T09:12:34.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.319+0000 7f11f048b740 10 client.27889 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:34.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.319+0000 7f11f048b740 1 -- 172.21.15.120:0/4078617441 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55eaefe78a40 con 0x55eaefe5f5d0 2024-09-19T09:12:34.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.320+0000 7f11c6ffd640 1 -- 172.21.15.120:0/4078617441 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11dc09a600 con 0x55eaefe5f5d0 2024-09-19T09:12:34.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.320+0000 7f11c6ffd640 1 -- 172.21.15.120:0/4078617441 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f11d804b810 con 0x55eaefe5f5d0 2024-09-19T09:12:34.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.320+0000 7f11f048b740 10 client.27889 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:34.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.320+0000 7f11f048b740 10 client.27889 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:12:34.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.320+0000 7f11f048b740 20 client.27889 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:34.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.320+0000 7f11f048b740 1 --2- 172.21.15.120:0/4078617441 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55eaefe79f50 0x55eaefe7c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:34.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.320+0000 7f11f048b740 4 client.27889 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:12:34.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.320+0000 7f11f048b740 1 -- 172.21.15.120:0/4078617441 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eaefdd2860 con 0x55eaefe79f50 2024-09-19T09:12:34.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.320+0000 7f11e6ffd640 1 --2- 172.21.15.120:0/4078617441 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55eaefe79f50 0x55eaefe7c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:34.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.320+0000 7f11c6ffd640 10 client.27889 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:12:34.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.321+0000 7f11e6ffd640 1 --2- 172.21.15.120:0/4078617441 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55eaefe79f50 0x55eaefe7c360 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d976bed51ffdd9c5 server_cookie=2e5f69ab3aa7bf4b in_seq=0 out_seq=0 2024-09-19T09:12:34.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.321+0000 7f11c6ffd640 1 -- 172.21.15.120:0/4078617441 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eaefdd2860 con 0x55eaefe79f50 2024-09-19T09:12:34.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.321+0000 7f11c6ffd640 10 client.27889 handle_command_reply: tid=0 2024-09-19T09:12:34.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.321+0000 7f11f048b740 1 client.27889 shutdown 2024-09-19T09:12:34.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.321+0000 7f11f048b740 1 -- 172.21.15.120:0/4078617441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaefe5f5d0 msgr2=0x55eaefe69ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:34.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.321+0000 7f11f048b740 1 --2- 172.21.15.120:0/4078617441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaefe5f5d0 0x55eaefe69ce0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009230 tx=0x7f11d802a410 comp rx=0 tx=0).stop 2024-09-19T09:12:34.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.321+0000 7f11f048b740 1 -- 172.21.15.120:0/4078617441 shutdown_connections 2024-09-19T09:12:34.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.321+0000 7f11f048b740 1 --2- 172.21.15.120:0/4078617441 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55eaefe79f50 0x55eaefe7c360 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.321+0000 7f11f048b740 1 --2- 172.21.15.120:0/4078617441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eaefe6a370 0x55eaefe674f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.321+0000 7f11f048b740 1 --2- 172.21.15.120:0/4078617441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eaefe60b40 0x55eaefe66f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.321+0000 7f11c6ffd640 0 client.27889 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:12:34.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.321+0000 7f11f048b740 1 --2- 172.21.15.120:0/4078617441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaefe5f5d0 0x55eaefe69ce0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.321+0000 7f11f048b740 1 -- 172.21.15.120:0/4078617441 >> 172.21.15.120:0/4078617441 conn(0x55eaefe28310 msgr2=0x55eaefe61930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:34.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.322+0000 7f11f048b740 1 -- 172.21.15.120:0/4078617441 shutdown_connections 2024-09-19T09:12:34.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.322+0000 7f11f048b740 1 -- 172.21.15.120:0/4078617441 wait complete. 2024-09-19T09:12:34.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.322+0000 7f11f048b740 20 client.27889 trim_cache size 0 max 16384 2024-09-19T09:12:34.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.339+0000 7f11f048b740 1 Processor -- start 2024-09-19T09:12:34.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.339+0000 7f11f048b740 1 -- start start 2024-09-19T09:12:34.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.339+0000 7f11f048b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eaefe5f5d0 0x55eaefe5abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:34.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.339+0000 7f11f048b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eaefe60b40 0x55eaefe62120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:34.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.339+0000 7f11f048b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaefe6a370 0x55eaefe62660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:34.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.339+0000 7f11f048b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f11e81b3bb0 con 0x55eaefe60b40 2024-09-19T09:12:34.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.339+0000 7f11f048b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f11e81b56f0 con 0x55eaefe5f5d0 2024-09-19T09:12:34.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.339+0000 7f11f048b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f11e81b4280 con 0x55eaefe6a370 2024-09-19T09:12:34.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.339+0000 7f11e6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eaefe60b40 0x55eaefe62120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:34.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.339+0000 7f11e6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eaefe60b40 0x55eaefe62120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47832/0 (socket says 172.21.15.120:47832) 2024-09-19T09:12:34.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.339+0000 7f11e6ffd640 1 -- 172.21.15.120:0/2851535451 learned_addr learned my addr 172.21.15.120:0/2851535451 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:34.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.339+0000 7f11e77fe640 1 --2- 172.21.15.120:0/2851535451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eaefe5f5d0 0x55eaefe5abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:34.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.339+0000 7f11e7fff640 1 --2- 172.21.15.120:0/2851535451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaefe6a370 0x55eaefe62660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:34.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.340+0000 7f11e6ffd640 1 -- 172.21.15.120:0/2851535451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaefe6a370 msgr2=0x55eaefe62660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:34.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.340+0000 7f11e6ffd640 1 --2- 172.21.15.120:0/2851535451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaefe6a370 0x55eaefe62660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.340+0000 7f11e6ffd640 1 -- 172.21.15.120:0/2851535451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eaefe5f5d0 msgr2=0x55eaefe5abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:34.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.340+0000 7f11e6ffd640 1 --2- 172.21.15.120:0/2851535451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eaefe5f5d0 0x55eaefe5abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.340+0000 7f11e6ffd640 1 -- 172.21.15.120:0/2851535451 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eaefe5cdd0 con 0x55eaefe60b40 2024-09-19T09:12:34.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.340+0000 7f11e6ffd640 1 --2- 172.21.15.120:0/2851535451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eaefe60b40 0x55eaefe62120 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f11dc01e000 tx=0x7f11dc0986d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5207cab4b2fc506a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:34.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.340+0000 7f11cd7fa640 1 -- 172.21.15.120:0/2851535451 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc09a600 con 0x55eaefe60b40 2024-09-19T09:12:34.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.340+0000 7f11cd7fa640 1 -- 172.21.15.120:0/2851535451 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11dc098970 con 0x55eaefe60b40 2024-09-19T09:12:34.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.340+0000 7f11cd7fa640 1 -- 172.21.15.120:0/2851535451 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc0a0120 con 0x55eaefe60b40 2024-09-19T09:12:34.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.341+0000 7f11f048b740 1 -- 172.21.15.120:0/2851535451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eaefe60b40 msgr2=0x55eaefe62120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:34.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.341+0000 7f11f048b740 1 --2- 172.21.15.120:0/2851535451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eaefe60b40 0x55eaefe62120 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f11dc01e000 tx=0x7f11dc0986d0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.341+0000 7f11f048b740 1 -- 172.21.15.120:0/2851535451 shutdown_connections 2024-09-19T09:12:34.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.341+0000 7f11f048b740 1 --2- 172.21.15.120:0/2851535451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaefe6a370 0x55eaefe62660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.341+0000 7f11f048b740 1 --2- 172.21.15.120:0/2851535451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eaefe60b40 0x55eaefe62120 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.341+0000 7f11f048b740 1 --2- 172.21.15.120:0/2851535451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eaefe5f5d0 0x55eaefe5abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.341+0000 7f11f048b740 1 -- 172.21.15.120:0/2851535451 >> 172.21.15.120:0/2851535451 conn(0x55eaefe28310 msgr2=0x55eaefe5a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:34.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.341+0000 7f11f048b740 1 -- 172.21.15.120:0/2851535451 shutdown_connections 2024-09-19T09:12:34.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.341+0000 7f11f048b740 1 -- 172.21.15.120:0/2851535451 wait complete. 2024-09-19T09:12:34.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.341+0000 7f11f048b740 1 -- start start 2024-09-19T09:12:34.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.341+0000 7f11f048b740 1 Processor -- start 2024-09-19T09:12:34.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.342+0000 7f11f048b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eaefe5f5d0 0x55eaefe63fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:34.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.342+0000 7f11f048b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaefe60b40 0x55eaefe644f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:34.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.342+0000 7f11f048b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eaefe6a370 0x55eaefe64a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:34.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.342+0000 7f11f048b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f11e81b5020 con 0x55eaefe5f5d0 2024-09-19T09:12:34.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.342+0000 7f11f048b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f11e81b4950 con 0x55eaefe6a370 2024-09-19T09:12:34.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.342+0000 7f11f048b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f11e81b1b70 con 0x55eaefe60b40 2024-09-19T09:12:34.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.342+0000 7f11f048b740 10 client.0 _ll_register_callbacks cb 0x7f11e00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:34.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.342+0000 7f11e77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eaefe5f5d0 0x55eaefe63fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:34.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.342+0000 7f11e77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eaefe5f5d0 0x55eaefe63fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47834/0 (socket says 172.21.15.120:47834) 2024-09-19T09:12:34.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.342+0000 7f11e77fe640 1 -- 172.21.15.120:0/1486496224 learned_addr learned my addr 172.21.15.120:0/1486496224 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:34.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.342+0000 7f11e7fff640 1 --2- 172.21.15.120:0/1486496224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eaefe6a370 0x55eaefe64a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:34.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.342+0000 7f11e6ffd640 1 --2- 172.21.15.120:0/1486496224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaefe60b40 0x55eaefe644f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:34.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.342+0000 7f11e77fe640 1 -- 172.21.15.120:0/1486496224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaefe60b40 msgr2=0x55eaefe644f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:34.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.342+0000 7f11e77fe640 1 --2- 172.21.15.120:0/1486496224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaefe60b40 0x55eaefe644f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.342+0000 7f11e77fe640 1 -- 172.21.15.120:0/1486496224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eaefe6a370 msgr2=0x55eaefe64a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:34.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.342+0000 7f11e77fe640 1 --2- 172.21.15.120:0/1486496224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eaefe6a370 0x55eaefe64a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.342+0000 7f11e77fe640 1 -- 172.21.15.120:0/1486496224 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eaefee3c00 con 0x55eaefe5f5d0 2024-09-19T09:12:34.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.342+0000 7f11e77fe640 1 --2- 172.21.15.120:0/1486496224 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eaefe5f5d0 0x55eaefe63fb0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f11d804be20 tx=0x7f11d8009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99c5d97ab1aa185 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:34.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.342+0000 7f11c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:34.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.343+0000 7f11c6ffd640 1 -- 172.21.15.120:0/1486496224 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d8035030 con 0x55eaefe5f5d0 2024-09-19T09:12:34.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.343+0000 7f11c6ffd640 1 -- 172.21.15.120:0/1486496224 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f11d8005020 con 0x55eaefe5f5d0 2024-09-19T09:12:34.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.343+0000 7f11f048b740 1 -- 172.21.15.120:0/1486496224 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11dc098970 con 0x55eaefe5f5d0 2024-09-19T09:12:34.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.343+0000 7f11c6ffd640 1 -- 172.21.15.120:0/1486496224 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f11d8005d50 con 0x55eaefe5f5d0 2024-09-19T09:12:34.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.343+0000 7f11f048b740 10 client.18951 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:34.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.343+0000 7f11f048b740 1 -- 172.21.15.120:0/1486496224 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55eaefee4ee0 con 0x55eaefe5f5d0 2024-09-19T09:12:34.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.343+0000 7f11c6ffd640 1 -- 172.21.15.120:0/1486496224 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11dc098970 con 0x55eaefe5f5d0 2024-09-19T09:12:34.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.343+0000 7f11c6ffd640 1 -- 172.21.15.120:0/1486496224 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f11d8004cc0 con 0x55eaefe5f5d0 2024-09-19T09:12:34.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.343+0000 7f11f048b740 10 client.18951 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:34.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.343+0000 7f11f048b740 10 client.18951 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:12:34.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.343+0000 7f11f048b740 20 client.18951 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:34.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.343+0000 7f11f048b740 1 --2- 172.21.15.120:0/1486496224 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55eaefee55c0 0x55eaefee79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:34.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.343+0000 7f11f048b740 4 client.18951 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:12:34.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.343+0000 7f11f048b740 1 -- 172.21.15.120:0/1486496224 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eaefdccd70 con 0x55eaefee55c0 2024-09-19T09:12:34.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.343+0000 7f11e6ffd640 1 --2- 172.21.15.120:0/1486496224 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55eaefee55c0 0x55eaefee79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:34.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.344+0000 7f11c6ffd640 10 client.18951 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:12:34.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.344+0000 7f11e6ffd640 1 --2- 172.21.15.120:0/1486496224 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55eaefee55c0 0x55eaefee79e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c0f2e0788f15e2bd server_cookie=96de3d10557f7e91 in_seq=0 out_seq=0 2024-09-19T09:12:34.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.344+0000 7f11c6ffd640 1 -- 172.21.15.120:0/1486496224 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eaefdccd70 con 0x55eaefee55c0 2024-09-19T09:12:34.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.344+0000 7f11c6ffd640 10 client.18951 handle_command_reply: tid=0 2024-09-19T09:12:34.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.344+0000 7f11f048b740 1 client.18951 shutdown 2024-09-19T09:12:34.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.344+0000 7f11f048b740 1 -- 172.21.15.120:0/1486496224 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eaefe5f5d0 msgr2=0x55eaefe63fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:34.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.344+0000 7f11f048b740 1 --2- 172.21.15.120:0/1486496224 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eaefe5f5d0 0x55eaefe63fb0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f11d804be20 tx=0x7f11d8009680 comp rx=0 tx=0).stop 2024-09-19T09:12:34.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.344+0000 7f11f048b740 1 -- 172.21.15.120:0/1486496224 shutdown_connections 2024-09-19T09:12:34.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.344+0000 7f11f048b740 1 --2- 172.21.15.120:0/1486496224 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55eaefee55c0 0x55eaefee79e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.344+0000 7f11f048b740 1 --2- 172.21.15.120:0/1486496224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eaefe6a370 0x55eaefe64a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.344+0000 7f11f048b740 1 --2- 172.21.15.120:0/1486496224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eaefe60b40 0x55eaefe644f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.344+0000 7f11c6ffd640 0 client.18951 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:12:34.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.344+0000 7f11f048b740 1 --2- 172.21.15.120:0/1486496224 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eaefe5f5d0 0x55eaefe63fb0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.344+0000 7f11f048b740 1 -- 172.21.15.120:0/1486496224 >> 172.21.15.120:0/1486496224 conn(0x55eaefe28310 msgr2=0x55eaefe5ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:34.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.344+0000 7f11f048b740 1 -- 172.21.15.120:0/1486496224 shutdown_connections 2024-09-19T09:12:34.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.344+0000 7f11f048b740 1 -- 172.21.15.120:0/1486496224 wait complete. 2024-09-19T09:12:34.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.345+0000 7f11f048b740 20 client.18951 trim_cache size 0 max 16384 2024-09-19T09:12:34.350 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:12:34.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.345+0000 7f11ee0b2640 1 -- 172.21.15.120:0/22595101 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f11d00a32c0 msgr2=0x7f11d00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:34.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.345+0000 7f11ee0b2640 1 --2- 172.21.15.120:0/22595101 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f11d00a32c0 0x7f11d00a5780 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f11e819fa80 tx=0x7f11d8004370 comp rx=0 tx=0).stop 2024-09-19T09:12:34.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.345+0000 7f11ee0b2640 1 -- 172.21.15.120:0/22595101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11e81011e0 msgr2=0x7f11e81a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:34.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.345+0000 7f11ee0b2640 1 --2- 172.21.15.120:0/22595101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11e81011e0 0x7f11e81a4cc0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f11dc00e9f0 tx=0x7f11dc00eeb0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.345+0000 7f11ee0b2640 1 -- 172.21.15.120:0/22595101 shutdown_connections 2024-09-19T09:12:34.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.345+0000 7f11ee0b2640 1 --2- 172.21.15.120:0/22595101 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f11d00a32c0 0x7f11d00a5780 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.345+0000 7f11ee0b2640 1 --2- 172.21.15.120:0/22595101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11e8105ec0 0x7f11e819e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.345+0000 7f11ee0b2640 1 --2- 172.21.15.120:0/22595101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11e81011e0 0x7f11e81a4cc0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.345+0000 7f11ee0b2640 1 --2- 172.21.15.120:0/22595101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11e81008c0 0x7f11e81a4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:34.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.345+0000 7f11ee0b2640 1 -- 172.21.15.120:0/22595101 >> 172.21.15.120:0/22595101 conn(0x7f11e80fc7d0 msgr2=0x7f11e8103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:34.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.345+0000 7f11ee0b2640 1 -- 172.21.15.120:0/22595101 shutdown_connections 2024-09-19T09:12:34.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:34.345+0000 7f11ee0b2640 1 -- 172.21.15.120:0/22595101 wait complete. 2024-09-19T09:12:35.066 INFO:tasks.workunit.client.0.smithi120.stdout: 1 955850 10.16 MB/sec execute 390 sec latency 59.160 ms 2024-09-19T09:12:35.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:34 smithi196 ceph-mon[35150]: pgmap v634: 97 pgs: 97 active+clean; 33 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 439 op/s 2024-09-19T09:12:35.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:34 smithi153 ceph-mon[35071]: pgmap v634: 97 pgs: 97 active+clean; 33 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 439 op/s 2024-09-19T09:12:35.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:34 smithi120 ceph-mon[31150]: pgmap v634: 97 pgs: 97 active+clean; 33 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 439 op/s 2024-09-19T09:12:35.406 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:12:35.687 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:12:36.066 INFO:tasks.workunit.client.0.smithi120.stdout: 1 958946 10.17 MB/sec execute 391 sec latency 10.452 ms 2024-09-19T09:12:36.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.217+0000 7f89a293e640 1 -- 172.21.15.120:0/1996404369 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f899c102570 msgr2=0x7f899c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:36.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.217+0000 7f89a293e640 1 --2- 172.21.15.120:0/1996404369 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f899c102570 0x7f899c1029b0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f8990009900 tx=0x7f899002a480 comp rx=0 tx=0).stop 2024-09-19T09:12:36.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.217+0000 7f89a293e640 1 -- 172.21.15.120:0/1996404369 shutdown_connections 2024-09-19T09:12:36.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.217+0000 7f89a293e640 1 --2- 172.21.15.120:0/1996404369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f899c102ef0 0x7f899c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.217+0000 7f89a293e640 1 --2- 172.21.15.120:0/1996404369 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f899c102570 0x7f899c1029b0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.217+0000 7f89a293e640 1 --2- 172.21.15.120:0/1996404369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f899c100e20 0x7f899c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.217+0000 7f89a293e640 1 -- 172.21.15.120:0/1996404369 >> 172.21.15.120:0/1996404369 conn(0x7f899c0fc7d0 msgr2=0x7f899c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:36.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.217+0000 7f89a293e640 1 -- 172.21.15.120:0/1996404369 shutdown_connections 2024-09-19T09:12:36.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.218+0000 7f89a293e640 1 -- 172.21.15.120:0/1996404369 wait complete. 2024-09-19T09:12:36.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.218+0000 7f89a293e640 1 Processor -- start 2024-09-19T09:12:36.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.218+0000 7f89a293e640 1 -- start start 2024-09-19T09:12:36.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.218+0000 7f89a293e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f899c100e20 0x7f899c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:36.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.218+0000 7f89a293e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f899c102570 0x7f899c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:36.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.218+0000 7f89a293e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f899c102ef0 0x7f899c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:36.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.218+0000 7f89a293e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f899c110f80 con 0x7f899c102ef0 2024-09-19T09:12:36.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.218+0000 7f89a293e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f899c110e00 con 0x7f899c100e20 2024-09-19T09:12:36.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.218+0000 7f89a293e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f899c111100 con 0x7f899c102570 2024-09-19T09:12:36.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.219+0000 7f89a0edb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f899c102ef0 0x7f899c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:36.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.219+0000 7f899bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f899c100e20 0x7f899c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:36.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.219+0000 7f899b7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f899c102570 0x7f899c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:36.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.219+0000 7f89a0edb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f899c102ef0 0x7f899c0fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47856/0 (socket says 172.21.15.120:47856) 2024-09-19T09:12:36.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.219+0000 7f89a0edb640 1 -- 172.21.15.120:0/4109776435 learned_addr learned my addr 172.21.15.120:0/4109776435 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:36.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.219+0000 7f899bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f899c100e20 0x7f899c0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52852/0 (socket says 172.21.15.120:52852) 2024-09-19T09:12:36.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.219+0000 7f899b7fe640 1 -- 172.21.15.120:0/4109776435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f899c100e20 msgr2=0x7f899c0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:36.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.219+0000 7f899b7fe640 1 --2- 172.21.15.120:0/4109776435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f899c100e20 0x7f899c0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.219+0000 7f899b7fe640 1 -- 172.21.15.120:0/4109776435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f899c102ef0 msgr2=0x7f899c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:36.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.219+0000 7f899b7fe640 1 --2- 172.21.15.120:0/4109776435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f899c102ef0 0x7f899c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.219+0000 7f899b7fe640 1 -- 172.21.15.120:0/4109776435 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f899c1a6a70 con 0x7f899c102570 2024-09-19T09:12:36.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.219+0000 7f89a0edb640 1 --2- 172.21.15.120:0/4109776435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f899c102ef0 0x7f899c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:36.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.219+0000 7f899b7fe640 1 --2- 172.21.15.120:0/4109776435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f899c102570 0x7f899c0fd1c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8990005940 tx=0x7f8990003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:36.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.219+0000 7f89997fa640 1 -- 172.21.15.120:0/4109776435 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899003f070 con 0x7f899c102570 2024-09-19T09:12:36.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.219+0000 7f89997fa640 1 -- 172.21.15.120:0/4109776435 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8990031c30 con 0x7f899c102570 2024-09-19T09:12:36.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.219+0000 7f89997fa640 1 -- 172.21.15.120:0/4109776435 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8990035030 con 0x7f899c102570 2024-09-19T09:12:36.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.219+0000 7f89a293e640 1 -- 172.21.15.120:0/4109776435 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f899c1a6d00 con 0x7f899c102570 2024-09-19T09:12:36.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.219+0000 7f89a293e640 1 -- 172.21.15.120:0/4109776435 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f899c1a71d0 con 0x7f899c102570 2024-09-19T09:12:36.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.222+0000 7f89997fa640 1 -- 172.21.15.120:0/4109776435 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f8990042070 con 0x7f899c102570 2024-09-19T09:12:36.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.222+0000 7f89a4d17740 1 Processor -- start 2024-09-19T09:12:36.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.222+0000 7f89a4d17740 1 -- start start 2024-09-19T09:12:36.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.222+0000 7f89a4d17740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563707b71b40 0x563707b70090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:36.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.222+0000 7f89a4d17740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563707b705d0 0x563707b70a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:36.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.222+0000 7f89a4d17740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563707b7b370 0x563707b7d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:36.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.222+0000 7f89a4d17740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563707ad62f0 con 0x563707b705d0 2024-09-19T09:12:36.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.222+0000 7f89a4d17740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563707a7dcf0 con 0x563707b7b370 2024-09-19T09:12:36.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.222+0000 7f89a4d17740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563707ad22b0 con 0x563707b71b40 2024-09-19T09:12:36.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.222+0000 7f89997fa640 1 --2- 172.21.15.120:0/4109776435 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f89840a32c0 0x7f89840a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:36.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.222+0000 7f899bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563707b705d0 0x563707b70a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:36.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.222+0000 7f899b7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563707b7b370 0x563707b7d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:36.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.222+0000 7f89a0edb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563707b71b40 0x563707b70090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:36.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.223+0000 7f89a0edb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563707b71b40 0x563707b70090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:47642/0 (socket says 172.21.15.120:47642) 2024-09-19T09:12:36.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.223+0000 7f899b7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563707b7b370 0x563707b7d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52856/0 (socket says 172.21.15.120:52856) 2024-09-19T09:12:36.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.223+0000 7f89a0edb640 1 -- 172.21.15.120:0/3662989731 learned_addr learned my addr 172.21.15.120:0/3662989731 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:36.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.223+0000 7f899bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563707b705d0 0x563707b70a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47868/0 (socket says 172.21.15.120:47868) 2024-09-19T09:12:36.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.223+0000 7f89997fa640 1 -- 172.21.15.120:0/4109776435 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f89900efe30 con 0x7f899c102570 2024-09-19T09:12:36.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.223+0000 7f89a0edb640 1 --2- 172.21.15.120:0/4109776435 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f89840a32c0 0x7f89840a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:36.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.223+0000 7f899bfff640 1 -- 172.21.15.120:0/3662989731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563707b71b40 msgr2=0x563707b70090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:36.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.223+0000 7f899bfff640 1 --2- 172.21.15.120:0/3662989731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563707b71b40 0x563707b70090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.223+0000 7f899bfff640 1 -- 172.21.15.120:0/3662989731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563707b7b370 msgr2=0x563707b7d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:36.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.223+0000 7f899bfff640 1 --2- 172.21.15.120:0/3662989731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563707b7b370 0x563707b7d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.223+0000 7f899bfff640 1 -- 172.21.15.120:0/3662989731 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563707a623a0 con 0x563707b705d0 2024-09-19T09:12:36.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.223+0000 7f899b7fe640 1 --2- 172.21.15.120:0/3662989731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563707b7b370 0x563707b7d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:36.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.223+0000 7f899bfff640 1 --2- 172.21.15.120:0/3662989731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563707b705d0 0x563707b70a10 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f898c009c00 tx=0x7f898c011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94aa85763b989bcb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:36.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.223+0000 7f89a0edb640 1 --2- 172.21.15.120:0/4109776435 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f89840a32c0 0x7f89840a5780 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8994009aa0 tx=0x7f8994004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:36.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.223+0000 7f89827fc640 1 -- 172.21.15.120:0/3662989731 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c004310 con 0x563707b705d0 2024-09-19T09:12:36.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.223+0000 7f89827fc640 1 -- 172.21.15.120:0/3662989731 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f898c0044b0 con 0x563707b705d0 2024-09-19T09:12:36.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.223+0000 7f89827fc640 1 -- 172.21.15.120:0/3662989731 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c0035f0 con 0x563707b705d0 2024-09-19T09:12:36.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.224+0000 7f89a4d17740 1 -- 172.21.15.120:0/3662989731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563707b705d0 msgr2=0x563707b70a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:36.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.224+0000 7f89a4d17740 1 --2- 172.21.15.120:0/3662989731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563707b705d0 0x563707b70a10 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f898c009c00 tx=0x7f898c011440 comp rx=0 tx=0).stop 2024-09-19T09:12:36.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.224+0000 7f89a4d17740 1 -- 172.21.15.120:0/3662989731 shutdown_connections 2024-09-19T09:12:36.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.224+0000 7f89a4d17740 1 --2- 172.21.15.120:0/3662989731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563707b7b370 0x563707b7d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.224+0000 7f89a4d17740 1 --2- 172.21.15.120:0/3662989731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563707b705d0 0x563707b70a10 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.224+0000 7f89a4d17740 1 --2- 172.21.15.120:0/3662989731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563707b71b40 0x563707b70090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.224+0000 7f89a4d17740 1 -- 172.21.15.120:0/3662989731 >> 172.21.15.120:0/3662989731 conn(0x563707b39310 msgr2=0x563707b6cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:36.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.224+0000 7f89a4d17740 1 -- 172.21.15.120:0/3662989731 shutdown_connections 2024-09-19T09:12:36.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.224+0000 7f89a4d17740 1 -- 172.21.15.120:0/3662989731 wait complete. 2024-09-19T09:12:36.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.224+0000 7f89a4d17740 1 -- start start 2024-09-19T09:12:36.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.225+0000 7f89a4d17740 1 Processor -- start 2024-09-19T09:12:36.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.225+0000 7f89a4d17740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563707b705d0 0x563707b7ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:36.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.225+0000 7f89a4d17740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563707b71b40 0x563707b77fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:36.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.225+0000 7f89a4d17740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563707b7b370 0x563707b78560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:36.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.225+0000 7f89a4d17740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563707b7e070 con 0x563707b7b370 2024-09-19T09:12:36.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.225+0000 7f89a4d17740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563707a91060 con 0x563707b71b40 2024-09-19T09:12:36.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.225+0000 7f89a4d17740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563707b7e1f0 con 0x563707b705d0 2024-09-19T09:12:36.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.225+0000 7f89a4d17740 10 client.0 _ll_register_callbacks cb 0x5637077e48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:36.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.225+0000 7f89a0edb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563707b7b370 0x563707b78560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:36.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.225+0000 7f899bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563707b705d0 0x563707b7ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:36.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.225+0000 7f89a0edb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563707b7b370 0x563707b78560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47880/0 (socket says 172.21.15.120:47880) 2024-09-19T09:12:36.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.225+0000 7f89a0edb640 1 -- 172.21.15.120:0/1288738944 learned_addr learned my addr 172.21.15.120:0/1288738944 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:36.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.225+0000 7f899b7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563707b71b40 0x563707b77fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:36.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.225+0000 7f89a0edb640 1 -- 172.21.15.120:0/1288738944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563707b705d0 msgr2=0x563707b7ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:36.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.225+0000 7f89a0edb640 1 --2- 172.21.15.120:0/1288738944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563707b705d0 0x563707b7ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.225+0000 7f89a0edb640 1 -- 172.21.15.120:0/1288738944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563707b71b40 msgr2=0x563707b77fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:36.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.225+0000 7f89a0edb640 1 --2- 172.21.15.120:0/1288738944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563707b71b40 0x563707b77fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.225+0000 7f89a0edb640 1 -- 172.21.15.120:0/1288738944 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637078473c0 con 0x563707b7b370 2024-09-19T09:12:36.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.226+0000 7f89a0edb640 1 --2- 172.21.15.120:0/1288738944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563707b7b370 0x563707b78560 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f8994011670 tx=0x7f8994004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c1401fb7aeab4f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:36.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.226+0000 7f897b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:36.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.226+0000 7f897b7fe640 1 -- 172.21.15.120:0/1288738944 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8994026070 con 0x563707b7b370 2024-09-19T09:12:36.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.226+0000 7f89a4d17740 1 -- 172.21.15.120:0/1288738944 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f898c0044b0 con 0x563707b7b370 2024-09-19T09:12:36.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.226+0000 7f897b7fe640 1 -- 172.21.15.120:0/1288738944 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f8994004d20 con 0x563707b7b370 2024-09-19T09:12:36.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.226+0000 7f897b7fe640 1 -- 172.21.15.120:0/1288738944 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8994031050 con 0x563707b7b370 2024-09-19T09:12:36.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.226+0000 7f89a4d17740 10 client.18963 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:36.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.226+0000 7f89a4d17740 1 -- 172.21.15.120:0/1288738944 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563707b89b70 con 0x563707b7b370 2024-09-19T09:12:36.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.226+0000 7f897b7fe640 1 -- 172.21.15.120:0/1288738944 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f898c0044b0 con 0x563707b7b370 2024-09-19T09:12:36.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.226+0000 7f897b7fe640 1 -- 172.21.15.120:0/1288738944 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f899401f050 con 0x563707b7b370 2024-09-19T09:12:36.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.226+0000 7f89a4d17740 10 client.18963 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:36.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.226+0000 7f89a4d17740 10 client.18963 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:12:36.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.226+0000 7f89a4d17740 20 client.18963 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:36.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.226+0000 7f89a4d17740 1 --2- 172.21.15.120:0/1288738944 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x563707b8af80 0x563707b8d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:36.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.226+0000 7f89a4d17740 4 client.18963 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:12:36.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.226+0000 7f89a4d17740 1 -- 172.21.15.120:0/1288738944 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563707ae3860 con 0x563707b8af80 2024-09-19T09:12:36.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.227+0000 7f899bfff640 1 --2- 172.21.15.120:0/1288738944 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x563707b8af80 0x563707b8d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:36.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.227+0000 7f897b7fe640 10 client.18963 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:12:36.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.227+0000 7f899bfff640 1 --2- 172.21.15.120:0/1288738944 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x563707b8af80 0x563707b8d370 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f44b6e943767c126 server_cookie=2b251d8f76dba989 in_seq=0 out_seq=0 2024-09-19T09:12:36.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.228+0000 7f897b7fe640 1 -- 172.21.15.120:0/1288738944 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563707ae3860 con 0x563707b8af80 2024-09-19T09:12:36.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.228+0000 7f897b7fe640 10 client.18963 handle_command_reply: tid=0 2024-09-19T09:12:36.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.228+0000 7f89a4d17740 1 client.18963 shutdown 2024-09-19T09:12:36.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.228+0000 7f89a4d17740 1 -- 172.21.15.120:0/1288738944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563707b7b370 msgr2=0x563707b78560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:36.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.228+0000 7f89a4d17740 1 --2- 172.21.15.120:0/1288738944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563707b7b370 0x563707b78560 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f8994011670 tx=0x7f8994004a40 comp rx=0 tx=0).stop 2024-09-19T09:12:36.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.228+0000 7f89a4d17740 1 -- 172.21.15.120:0/1288738944 shutdown_connections 2024-09-19T09:12:36.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.228+0000 7f89a4d17740 1 --2- 172.21.15.120:0/1288738944 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x563707b8af80 0x563707b8d370 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.228+0000 7f897b7fe640 0 client.18963 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:12:36.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.228+0000 7f89a4d17740 1 --2- 172.21.15.120:0/1288738944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563707b7b370 0x563707b78560 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.228+0000 7f89a4d17740 1 --2- 172.21.15.120:0/1288738944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563707b71b40 0x563707b77fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.228+0000 7f89a4d17740 1 --2- 172.21.15.120:0/1288738944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563707b705d0 0x563707b7ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.228+0000 7f89a4d17740 1 -- 172.21.15.120:0/1288738944 >> 172.21.15.120:0/1288738944 conn(0x563707b39310 msgr2=0x563707b72960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:36.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.228+0000 7f89a4d17740 1 -- 172.21.15.120:0/1288738944 shutdown_connections 2024-09-19T09:12:36.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.228+0000 7f89a4d17740 1 -- 172.21.15.120:0/1288738944 wait complete. 2024-09-19T09:12:36.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.228+0000 7f89a4d17740 20 client.18963 trim_cache size 0 max 16384 2024-09-19T09:12:36.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.242+0000 7f89a4d17740 1 Processor -- start 2024-09-19T09:12:36.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.242+0000 7f89a4d17740 1 -- start start 2024-09-19T09:12:36.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.242+0000 7f89a4d17740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563707b705d0 0x563707b6ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:36.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.242+0000 7f89a4d17740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563707b71b40 0x563707b87310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:36.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.242+0000 7f89a4d17740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563707b7b370 0x563707b77880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:36.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.242+0000 7f89a4d17740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f899c1af850 con 0x563707b705d0 2024-09-19T09:12:36.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.242+0000 7f89a4d17740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f899c1b1390 con 0x563707b71b40 2024-09-19T09:12:36.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.242+0000 7f89a4d17740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f899c1aff20 con 0x563707b7b370 2024-09-19T09:12:36.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.242+0000 7f899bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563707b705d0 0x563707b6ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:36.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.242+0000 7f89a0edb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563707b7b370 0x563707b77880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:36.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.242+0000 7f899bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563707b705d0 0x563707b6ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47888/0 (socket says 172.21.15.120:47888) 2024-09-19T09:12:36.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.242+0000 7f89a0edb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563707b7b370 0x563707b77880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:47650/0 (socket says 172.21.15.120:47650) 2024-09-19T09:12:36.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.242+0000 7f899bfff640 1 -- 172.21.15.120:0/1063425838 learned_addr learned my addr 172.21.15.120:0/1063425838 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:36.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.242+0000 7f899b7fe640 1 --2- 172.21.15.120:0/1063425838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563707b71b40 0x563707b87310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:36.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.242+0000 7f899bfff640 1 -- 172.21.15.120:0/1063425838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563707b7b370 msgr2=0x563707b77880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:36.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.242+0000 7f899bfff640 1 --2- 172.21.15.120:0/1063425838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563707b7b370 0x563707b77880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.242+0000 7f899bfff640 1 -- 172.21.15.120:0/1063425838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563707b71b40 msgr2=0x563707b87310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:36.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.242+0000 7f899bfff640 1 --2- 172.21.15.120:0/1063425838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563707b71b40 0x563707b87310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.242+0000 7f899bfff640 1 -- 172.21.15.120:0/1063425838 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563707b8d6a0 con 0x563707b705d0 2024-09-19T09:12:36.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.242+0000 7f89a0edb640 1 --2- 172.21.15.120:0/1063425838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563707b7b370 0x563707b77880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:36.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.242+0000 7f899bfff640 1 --2- 172.21.15.120:0/1063425838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563707b705d0 0x563707b6ce10 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f898c009b20 tx=0x7f898c0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c3721959e213f05 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:36.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.243+0000 7f8981ffb640 1 -- 172.21.15.120:0/1063425838 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c004460 con 0x563707b705d0 2024-09-19T09:12:36.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.243+0000 7f8981ffb640 1 -- 172.21.15.120:0/1063425838 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f898c01f040 con 0x563707b705d0 2024-09-19T09:12:36.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.243+0000 7f8981ffb640 1 -- 172.21.15.120:0/1063425838 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c004e30 con 0x563707b705d0 2024-09-19T09:12:36.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.243+0000 7f89a4d17740 1 -- 172.21.15.120:0/1063425838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563707b705d0 msgr2=0x563707b6ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:36.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.243+0000 7f89a4d17740 1 --2- 172.21.15.120:0/1063425838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563707b705d0 0x563707b6ce10 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f898c009b20 tx=0x7f898c0096a0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.243+0000 7f89a4d17740 1 -- 172.21.15.120:0/1063425838 shutdown_connections 2024-09-19T09:12:36.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.243+0000 7f89a4d17740 1 --2- 172.21.15.120:0/1063425838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563707b7b370 0x563707b77880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.243+0000 7f89a4d17740 1 --2- 172.21.15.120:0/1063425838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563707b71b40 0x563707b87310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.243+0000 7f89a4d17740 1 --2- 172.21.15.120:0/1063425838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563707b705d0 0x563707b6ce10 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.243+0000 7f89a4d17740 1 -- 172.21.15.120:0/1063425838 >> 172.21.15.120:0/1063425838 conn(0x563707b39310 msgr2=0x563707b6c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:36.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.243+0000 7f89a4d17740 1 -- 172.21.15.120:0/1063425838 shutdown_connections 2024-09-19T09:12:36.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.243+0000 7f89a4d17740 1 -- 172.21.15.120:0/1063425838 wait complete. 2024-09-19T09:12:36.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.244+0000 7f89a4d17740 1 -- start start 2024-09-19T09:12:36.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.244+0000 7f89a4d17740 1 Processor -- start 2024-09-19T09:12:36.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.244+0000 7f89a4d17740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563707b705d0 0x563707b83840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:36.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.244+0000 7f89a4d17740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563707b71b40 0x563707b83d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:36.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.244+0000 7f89a4d17740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563707b7b370 0x563707b74ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:36.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.244+0000 7f89a4d17740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f899c1b0cc0 con 0x563707b705d0 2024-09-19T09:12:36.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.244+0000 7f89a4d17740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f899c1b05f0 con 0x563707b71b40 2024-09-19T09:12:36.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.244+0000 7f89a4d17740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f899c1ad810 con 0x563707b7b370 2024-09-19T09:12:36.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.244+0000 7f899bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563707b705d0 0x563707b83840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:36.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.244+0000 7f899b7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563707b71b40 0x563707b83d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:36.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.244+0000 7f899bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563707b705d0 0x563707b83840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47902/0 (socket says 172.21.15.120:47902) 2024-09-19T09:12:36.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.244+0000 7f899bfff640 1 -- 172.21.15.120:0/278408427 learned_addr learned my addr 172.21.15.120:0/278408427 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:36.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.244+0000 7f89a4d17740 10 client.0 _ll_register_callbacks cb 0x7f899400ff50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:36.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.244+0000 7f89a0edb640 1 --2- 172.21.15.120:0/278408427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563707b7b370 0x563707b74ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:36.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.245+0000 7f899bfff640 1 -- 172.21.15.120:0/278408427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563707b7b370 msgr2=0x563707b74ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:36.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.245+0000 7f899bfff640 1 --2- 172.21.15.120:0/278408427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563707b7b370 0x563707b74ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.245+0000 7f899bfff640 1 -- 172.21.15.120:0/278408427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563707b71b40 msgr2=0x563707b83d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:36.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.245+0000 7f899bfff640 1 --2- 172.21.15.120:0/278408427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563707b71b40 0x563707b83d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.245+0000 7f899bfff640 1 -- 172.21.15.120:0/278408427 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563707b753a0 con 0x563707b705d0 2024-09-19T09:12:36.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.245+0000 7f899b7fe640 1 --2- 172.21.15.120:0/278408427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563707b71b40 0x563707b83d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:36.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.245+0000 7f899bfff640 1 --2- 172.21.15.120:0/278408427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563707b705d0 0x563707b83840 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f898c0043b0 tx=0x7f898c009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d073d25c86238a8c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:36.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.245+0000 7f897b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:36.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.245+0000 7f897b7fe640 1 -- 172.21.15.120:0/278408427 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c01e040 con 0x563707b705d0 2024-09-19T09:12:36.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.245+0000 7f897b7fe640 1 -- 172.21.15.120:0/278408427 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f898c005050 con 0x563707b705d0 2024-09-19T09:12:36.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.245+0000 7f89a4d17740 1 -- 172.21.15.120:0/278408427 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f898c01f040 con 0x563707b705d0 2024-09-19T09:12:36.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.245+0000 7f897b7fe640 1 -- 172.21.15.120:0/278408427 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f898c03e050 con 0x563707b705d0 2024-09-19T09:12:36.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.245+0000 7f89a4d17740 10 client.18969 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:36.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.245+0000 7f89a4d17740 1 -- 172.21.15.120:0/278408427 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563707bf58c0 con 0x563707b705d0 2024-09-19T09:12:36.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.245+0000 7f897b7fe640 1 -- 172.21.15.120:0/278408427 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f898c01f040 con 0x563707b705d0 2024-09-19T09:12:36.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.246+0000 7f897b7fe640 1 -- 172.21.15.120:0/278408427 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f898c018070 con 0x563707b705d0 2024-09-19T09:12:36.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.246+0000 7f89a4d17740 10 client.18969 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:36.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.246+0000 7f89a4d17740 10 client.18969 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:12:36.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.246+0000 7f89a4d17740 20 client.18969 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:36.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.246+0000 7f89a4d17740 1 --2- 172.21.15.120:0/278408427 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x563707b8af80 0x563707bf7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:36.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.246+0000 7f89a4d17740 4 client.18969 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:12:36.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.246+0000 7f89a4d17740 1 -- 172.21.15.120:0/278408427 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563707addd70 con 0x563707b8af80 2024-09-19T09:12:36.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.246+0000 7f899b7fe640 1 --2- 172.21.15.120:0/278408427 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x563707b8af80 0x563707bf7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:36.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.246+0000 7f899b7fe640 1 --2- 172.21.15.120:0/278408427 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x563707b8af80 0x563707bf7d00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e8bd6ff2370ef2d2 server_cookie=353864fabdfa5b72 in_seq=0 out_seq=0 2024-09-19T09:12:36.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.246+0000 7f897b7fe640 10 client.18969 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:12:36.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.247+0000 7f897b7fe640 1 -- 172.21.15.120:0/278408427 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563707addd70 con 0x563707b8af80 2024-09-19T09:12:36.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.247+0000 7f897b7fe640 10 client.18969 handle_command_reply: tid=0 2024-09-19T09:12:36.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.247+0000 7f89a4d17740 1 client.18969 shutdown 2024-09-19T09:12:36.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.247+0000 7f89a4d17740 1 -- 172.21.15.120:0/278408427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563707b705d0 msgr2=0x563707b83840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:36.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.247+0000 7f89a4d17740 1 --2- 172.21.15.120:0/278408427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563707b705d0 0x563707b83840 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f898c0043b0 tx=0x7f898c009940 comp rx=0 tx=0).stop 2024-09-19T09:12:36.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.247+0000 7f89a4d17740 1 -- 172.21.15.120:0/278408427 shutdown_connections 2024-09-19T09:12:36.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.247+0000 7f89a4d17740 1 --2- 172.21.15.120:0/278408427 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x563707b8af80 0x563707bf7d00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.247+0000 7f89a4d17740 1 --2- 172.21.15.120:0/278408427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563707b7b370 0x563707b74ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.247+0000 7f89a4d17740 1 --2- 172.21.15.120:0/278408427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563707b71b40 0x563707b83d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.247+0000 7f897b7fe640 0 client.18969 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:12:36.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.247+0000 7f89a4d17740 1 --2- 172.21.15.120:0/278408427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563707b705d0 0x563707b83840 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.247+0000 7f89a4d17740 1 -- 172.21.15.120:0/278408427 >> 172.21.15.120:0/278408427 conn(0x563707b39310 msgr2=0x563707b6c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:36.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.247+0000 7f89a4d17740 1 -- 172.21.15.120:0/278408427 shutdown_connections 2024-09-19T09:12:36.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.247+0000 7f89a4d17740 1 -- 172.21.15.120:0/278408427 wait complete. 2024-09-19T09:12:36.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.247+0000 7f89a4d17740 20 client.18969 trim_cache size 0 max 16384 2024-09-19T09:12:36.256 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:12:36.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.248+0000 7f89a293e640 1 -- 172.21.15.120:0/4109776435 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f89840a32c0 msgr2=0x7f89840a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:36.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.248+0000 7f89a293e640 1 --2- 172.21.15.120:0/4109776435 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f89840a32c0 0x7f89840a5780 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8994009aa0 tx=0x7f8994004ab0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.248+0000 7f89a293e640 1 -- 172.21.15.120:0/4109776435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f899c102570 msgr2=0x7f899c0fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:36.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.248+0000 7f89a293e640 1 --2- 172.21.15.120:0/4109776435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f899c102570 0x7f899c0fd1c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8990005940 tx=0x7f8990003c10 comp rx=0 tx=0).stop 2024-09-19T09:12:36.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.248+0000 7f89a293e640 1 -- 172.21.15.120:0/4109776435 shutdown_connections 2024-09-19T09:12:36.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.248+0000 7f89a293e640 1 --2- 172.21.15.120:0/4109776435 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f89840a32c0 0x7f89840a5780 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.248+0000 7f89a293e640 1 --2- 172.21.15.120:0/4109776435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f899c102ef0 0x7f899c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.248+0000 7f89a293e640 1 --2- 172.21.15.120:0/4109776435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f899c102570 0x7f899c0fd1c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.248+0000 7f89a293e640 1 --2- 172.21.15.120:0/4109776435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f899c100e20 0x7f899c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:36.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.248+0000 7f89a293e640 1 -- 172.21.15.120:0/4109776435 >> 172.21.15.120:0/4109776435 conn(0x7f899c0fc7d0 msgr2=0x7f899c06a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:36.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.248+0000 7f89a293e640 1 -- 172.21.15.120:0/4109776435 shutdown_connections 2024-09-19T09:12:36.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:36.248+0000 7f89a293e640 1 -- 172.21.15.120:0/4109776435 wait complete. 2024-09-19T09:12:37.066 INFO:tasks.workunit.client.0.smithi120.stdout: 1 961632 10.18 MB/sec execute 392 sec latency 54.608 ms 2024-09-19T09:12:37.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:36 smithi196 ceph-mon[35150]: pgmap v635: 97 pgs: 97 active+clean; 34 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 3.4 MiB/s wr, 367 op/s 2024-09-19T09:12:37.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:36 smithi153 ceph-mon[35071]: pgmap v635: 97 pgs: 97 active+clean; 34 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 3.4 MiB/s wr, 367 op/s 2024-09-19T09:12:37.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:36 smithi120 ceph-mon[31150]: pgmap v635: 97 pgs: 97 active+clean; 34 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 3.4 MiB/s wr, 367 op/s 2024-09-19T09:12:37.552 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:12:37.811 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:12:38.067 INFO:tasks.workunit.client.0.smithi120.stdout: 1 963977 10.19 MB/sec execute 393 sec latency 17.679 ms 2024-09-19T09:12:38.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.301+0000 7f573febe640 1 -- 172.21.15.120:0/3058339300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5738100e20 msgr2=0x7f5738101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:38.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.301+0000 7f573febe640 1 --2- 172.21.15.120:0/3058339300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5738100e20 0x7f5738101200 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f5734009900 tx=0x7f573402a480 comp rx=0 tx=0).stop 2024-09-19T09:12:38.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.301+0000 7f573febe640 1 -- 172.21.15.120:0/3058339300 shutdown_connections 2024-09-19T09:12:38.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.301+0000 7f573febe640 1 --2- 172.21.15.120:0/3058339300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5738102ef0 0x7f573810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.301+0000 7f573febe640 1 --2- 172.21.15.120:0/3058339300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5738102570 0x7f57381029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.301+0000 7f573febe640 1 --2- 172.21.15.120:0/3058339300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5738100e20 0x7f5738101200 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.301+0000 7f573febe640 1 -- 172.21.15.120:0/3058339300 >> 172.21.15.120:0/3058339300 conn(0x7f57380fc7d0 msgr2=0x7f57380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:38.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.301+0000 7f573febe640 1 -- 172.21.15.120:0/3058339300 shutdown_connections 2024-09-19T09:12:38.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.301+0000 7f573febe640 1 -- 172.21.15.120:0/3058339300 wait complete. 2024-09-19T09:12:38.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.302+0000 7f573febe640 1 Processor -- start 2024-09-19T09:12:38.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.302+0000 7f573febe640 1 -- start start 2024-09-19T09:12:38.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.302+0000 7f573febe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5738100e20 0x7f57381971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:38.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.302+0000 7f573febe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5738102570 0x7f5738197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:38.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.302+0000 7f573febe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5738102ef0 0x7f5738197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:38.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.302+0000 7f573febe640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5738110fd0 con 0x7f5738102570 2024-09-19T09:12:38.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.302+0000 7f573febe640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5738110e50 con 0x7f5738100e20 2024-09-19T09:12:38.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.302+0000 7f573febe640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5738111150 con 0x7f5738102ef0 2024-09-19T09:12:38.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.302+0000 7f573d459640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5738102570 0x7f5738197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:38.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.302+0000 7f573d459640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5738102570 0x7f5738197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47914/0 (socket says 172.21.15.120:47914) 2024-09-19T09:12:38.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.302+0000 7f573d459640 1 -- 172.21.15.120:0/243502760 learned_addr learned my addr 172.21.15.120:0/243502760 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:38.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.302+0000 7f573dc5a640 1 --2- 172.21.15.120:0/243502760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5738100e20 0x7f57381971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:38.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.302+0000 7f573e45b640 1 --2- 172.21.15.120:0/243502760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5738102ef0 0x7f5738197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:38.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.302+0000 7f573d459640 1 -- 172.21.15.120:0/243502760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5738102ef0 msgr2=0x7f5738197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:38.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.302+0000 7f573d459640 1 --2- 172.21.15.120:0/243502760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5738102ef0 0x7f5738197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.302+0000 7f573d459640 1 -- 172.21.15.120:0/243502760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5738100e20 msgr2=0x7f57381971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:38.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.302+0000 7f573d459640 1 --2- 172.21.15.120:0/243502760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5738100e20 0x7f57381971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.302+0000 7f573d459640 1 -- 172.21.15.120:0/243502760 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57381a6a80 con 0x7f5738102570 2024-09-19T09:12:38.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.303+0000 7f573d459640 1 --2- 172.21.15.120:0/243502760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5738102570 0x7f5738197710 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f572800e9f0 tx=0x7f572800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:38.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.303+0000 7f572effd640 1 -- 172.21.15.120:0/243502760 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572800cd80 con 0x7f5738102570 2024-09-19T09:12:38.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.303+0000 7f572effd640 1 -- 172.21.15.120:0/243502760 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5728010910 con 0x7f5738102570 2024-09-19T09:12:38.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.303+0000 7f573febe640 1 -- 172.21.15.120:0/243502760 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57381a6d70 con 0x7f5738102570 2024-09-19T09:12:38.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.303+0000 7f572effd640 1 -- 172.21.15.120:0/243502760 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5728018990 con 0x7f5738102570 2024-09-19T09:12:38.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.303+0000 7f573febe640 1 -- 172.21.15.120:0/243502760 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57381a7270 con 0x7f5738102570 2024-09-19T09:12:38.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.305+0000 7f572effd640 1 -- 172.21.15.120:0/243502760 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5728010ac0 con 0x7f5738102570 2024-09-19T09:12:38.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.306+0000 7f5742297740 1 Processor -- start 2024-09-19T09:12:38.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.306+0000 7f5742297740 1 -- start start 2024-09-19T09:12:38.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.306+0000 7f5742297740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615ff2f4b40 0x5615ff2f3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:38.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.306+0000 7f5742297740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615ff2f35d0 0x5615ff2f3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:38.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.306+0000 7f5742297740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615ff2fe370 0x5615ff300760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:38.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.306+0000 7f5742297740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5615ff2592f0 con 0x5615ff2f35d0 2024-09-19T09:12:38.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.306+0000 7f5742297740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5615ff200cf0 con 0x5615ff2f4b40 2024-09-19T09:12:38.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.306+0000 7f5742297740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5615ff2552b0 con 0x5615ff2fe370 2024-09-19T09:12:38.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.306+0000 7f572effd640 1 --2- 172.21.15.120:0/243502760 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f57240a32c0 0x7f57240a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:38.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.306+0000 7f573dc5a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615ff2f35d0 0x5615ff2f3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:38.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.306+0000 7f573e45b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615ff2f4b40 0x5615ff2f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:38.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.306+0000 7f573dc5a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615ff2f35d0 0x5615ff2f3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47916/0 (socket says 172.21.15.120:47916) 2024-09-19T09:12:38.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.306+0000 7f573e45b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615ff2f4b40 0x5615ff2f3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52906/0 (socket says 172.21.15.120:52906) 2024-09-19T09:12:38.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.306+0000 7f573dc5a640 1 -- 172.21.15.120:0/978416617 learned_addr learned my addr 172.21.15.120:0/978416617 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:38.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.306+0000 7f573e45b640 1 --2- 172.21.15.120:0/243502760 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f57240a32c0 0x7f57240a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:38.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.306+0000 7f572effd640 1 -- 172.21.15.120:0/243502760 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f57280cfc40 con 0x7f5738102570 2024-09-19T09:12:38.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.306+0000 7f573d459640 1 --2- 172.21.15.120:0/978416617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615ff2fe370 0x5615ff300760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:38.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.307+0000 7f573dc5a640 1 -- 172.21.15.120:0/978416617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615ff2fe370 msgr2=0x5615ff300760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:38.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.307+0000 7f573dc5a640 1 --2- 172.21.15.120:0/978416617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615ff2fe370 0x5615ff300760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.307+0000 7f573dc5a640 1 -- 172.21.15.120:0/978416617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615ff2f4b40 msgr2=0x5615ff2f3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:38.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.307+0000 7f573dc5a640 1 --2- 172.21.15.120:0/978416617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615ff2f4b40 0x5615ff2f3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.307+0000 7f573dc5a640 1 -- 172.21.15.120:0/978416617 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615ff1e53a0 con 0x5615ff2f35d0 2024-09-19T09:12:38.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.307+0000 7f573e45b640 1 --2- 172.21.15.120:0/978416617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615ff2f4b40 0x5615ff2f3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:38.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.307+0000 7f573e45b640 1 --2- 172.21.15.120:0/243502760 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f57240a32c0 0x7f57240a5780 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5730005c40 tx=0x7f5730009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:38.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.307+0000 7f573dc5a640 1 --2- 172.21.15.120:0/978416617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615ff2f35d0 0x5615ff2f3a10 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f5734009230 tx=0x7f5734005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801474eafdc01a34 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:38.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.307+0000 7f5723fff640 1 -- 172.21.15.120:0/978416617 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5734035040 con 0x5615ff2f35d0 2024-09-19T09:12:38.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.307+0000 7f5723fff640 1 -- 172.21.15.120:0/978416617 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5734004400 con 0x5615ff2f35d0 2024-09-19T09:12:38.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.307+0000 7f5723fff640 1 -- 172.21.15.120:0/978416617 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57340035f0 con 0x5615ff2f35d0 2024-09-19T09:12:38.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.308+0000 7f5742297740 1 -- 172.21.15.120:0/978416617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615ff2f35d0 msgr2=0x5615ff2f3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:38.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.308+0000 7f5742297740 1 --2- 172.21.15.120:0/978416617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615ff2f35d0 0x5615ff2f3a10 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f5734009230 tx=0x7f5734005e00 comp rx=0 tx=0).stop 2024-09-19T09:12:38.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.308+0000 7f5742297740 1 -- 172.21.15.120:0/978416617 shutdown_connections 2024-09-19T09:12:38.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.308+0000 7f5742297740 1 --2- 172.21.15.120:0/978416617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615ff2fe370 0x5615ff300760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.308+0000 7f5742297740 1 --2- 172.21.15.120:0/978416617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615ff2f35d0 0x5615ff2f3a10 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.308+0000 7f5742297740 1 --2- 172.21.15.120:0/978416617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615ff2f4b40 0x5615ff2f3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.308+0000 7f5742297740 1 -- 172.21.15.120:0/978416617 >> 172.21.15.120:0/978416617 conn(0x5615ff2bc310 msgr2=0x5615ff2efc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:38.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.308+0000 7f5742297740 1 -- 172.21.15.120:0/978416617 shutdown_connections 2024-09-19T09:12:38.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.308+0000 7f5742297740 1 -- 172.21.15.120:0/978416617 wait complete. 2024-09-19T09:12:38.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.308+0000 7f5742297740 1 -- start start 2024-09-19T09:12:38.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.308+0000 7f5742297740 1 Processor -- start 2024-09-19T09:12:38.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.309+0000 7f5742297740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615ff2f35d0 0x5615ff2fdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:38.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.309+0000 7f5742297740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615ff2f4b40 0x5615ff2fafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:38.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.309+0000 7f5742297740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615ff2fe370 0x5615ff2fb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:38.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.309+0000 7f5742297740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5615ff301010 con 0x5615ff2f35d0 2024-09-19T09:12:38.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.309+0000 7f5742297740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5615ff214060 con 0x5615ff2f4b40 2024-09-19T09:12:38.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.309+0000 7f5742297740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5615ff301190 con 0x5615ff2fe370 2024-09-19T09:12:38.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.309+0000 7f5742297740 10 client.0 _ll_register_callbacks cb 0x5615fef678c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:38.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.309+0000 7f573dc5a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615ff2f35d0 0x5615ff2fdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:38.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.309+0000 7f573dc5a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615ff2f35d0 0x5615ff2fdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47922/0 (socket says 172.21.15.120:47922) 2024-09-19T09:12:38.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.309+0000 7f573d459640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615ff2f4b40 0x5615ff2fafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:38.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.309+0000 7f573dc5a640 1 -- 172.21.15.120:0/3426583959 learned_addr learned my addr 172.21.15.120:0/3426583959 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:38.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.309+0000 7f573e45b640 1 --2- 172.21.15.120:0/3426583959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615ff2fe370 0x5615ff2fb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:38.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.309+0000 7f573dc5a640 1 -- 172.21.15.120:0/3426583959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615ff2fe370 msgr2=0x5615ff2fb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:38.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.309+0000 7f573dc5a640 1 --2- 172.21.15.120:0/3426583959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615ff2fe370 0x5615ff2fb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.309+0000 7f573dc5a640 1 -- 172.21.15.120:0/3426583959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615ff2f4b40 msgr2=0x5615ff2fafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:38.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.309+0000 7f573dc5a640 1 --2- 172.21.15.120:0/3426583959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615ff2f4b40 0x5615ff2fafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.309+0000 7f573dc5a640 1 -- 172.21.15.120:0/3426583959 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615fefca3c0 con 0x5615ff2f35d0 2024-09-19T09:12:38.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.309+0000 7f573d459640 1 --2- 172.21.15.120:0/3426583959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615ff2f4b40 0x5615ff2fafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:38.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.309+0000 7f573dc5a640 1 --2- 172.21.15.120:0/3426583959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615ff2f35d0 0x5615ff2fdd00 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f5734006010 tx=0x7f5734009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2d5b63a4e0f67d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:38.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.309+0000 7f57217fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:38.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.310+0000 7f57217fa640 1 -- 172.21.15.120:0/3426583959 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5734053060 con 0x5615ff2f35d0 2024-09-19T09:12:38.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.310+0000 7f57217fa640 1 -- 172.21.15.120:0/3426583959 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f5734003800 con 0x5615ff2f35d0 2024-09-19T09:12:38.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.310+0000 7f5742297740 1 -- 172.21.15.120:0/3426583959 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5734004400 con 0x5615ff2f35d0 2024-09-19T09:12:38.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.310+0000 7f57217fa640 1 -- 172.21.15.120:0/3426583959 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5734035050 con 0x5615ff2f35d0 2024-09-19T09:12:38.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.310+0000 7f5742297740 10 client.18981 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:38.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.310+0000 7f5742297740 1 -- 172.21.15.120:0/3426583959 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5615ff30ca40 con 0x5615ff2f35d0 2024-09-19T09:12:38.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.310+0000 7f57217fa640 1 -- 172.21.15.120:0/3426583959 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5734004400 con 0x5615ff2f35d0 2024-09-19T09:12:38.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.310+0000 7f57217fa640 1 -- 172.21.15.120:0/3426583959 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f5734058070 con 0x5615ff2f35d0 2024-09-19T09:12:38.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.310+0000 7f5742297740 10 client.18981 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:38.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.310+0000 7f5742297740 10 client.18981 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:12:38.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.310+0000 7f5742297740 20 client.18981 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:38.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.310+0000 7f5742297740 1 --2- 172.21.15.120:0/3426583959 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5615ff30dd90 0x5615ff310180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:38.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.310+0000 7f5742297740 4 client.18981 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:12:38.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.310+0000 7f5742297740 1 -- 172.21.15.120:0/3426583959 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615ff266860 con 0x5615ff30dd90 2024-09-19T09:12:38.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.310+0000 7f573d459640 1 --2- 172.21.15.120:0/3426583959 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5615ff30dd90 0x5615ff310180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:38.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.311+0000 7f573d459640 1 --2- 172.21.15.120:0/3426583959 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5615ff30dd90 0x5615ff310180 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3e5ef28e9d920d9d server_cookie=995a2ee7fa14afe8 in_seq=0 out_seq=0 2024-09-19T09:12:38.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.311+0000 7f57217fa640 10 client.18981 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:12:38.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.311+0000 7f57217fa640 1 -- 172.21.15.120:0/3426583959 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5615ff266860 con 0x5615ff30dd90 2024-09-19T09:12:38.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.311+0000 7f57217fa640 10 client.18981 handle_command_reply: tid=0 2024-09-19T09:12:38.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.311+0000 7f5742297740 1 client.18981 shutdown 2024-09-19T09:12:38.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.311+0000 7f5742297740 1 -- 172.21.15.120:0/3426583959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615ff2f35d0 msgr2=0x5615ff2fdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:38.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.311+0000 7f5742297740 1 --2- 172.21.15.120:0/3426583959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615ff2f35d0 0x5615ff2fdd00 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f5734006010 tx=0x7f5734009260 comp rx=0 tx=0).stop 2024-09-19T09:12:38.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.312+0000 7f5742297740 1 -- 172.21.15.120:0/3426583959 shutdown_connections 2024-09-19T09:12:38.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.312+0000 7f5742297740 1 --2- 172.21.15.120:0/3426583959 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5615ff30dd90 0x5615ff310180 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.312+0000 7f5742297740 1 --2- 172.21.15.120:0/3426583959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615ff2fe370 0x5615ff2fb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.312+0000 7f5742297740 1 --2- 172.21.15.120:0/3426583959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615ff2f4b40 0x5615ff2fafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.312+0000 7f5742297740 1 --2- 172.21.15.120:0/3426583959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615ff2f35d0 0x5615ff2fdd00 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.312+0000 7f5742297740 1 -- 172.21.15.120:0/3426583959 >> 172.21.15.120:0/3426583959 conn(0x5615ff2bc310 msgr2=0x5615ff2f5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:38.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.312+0000 7f57217fa640 0 client.18981 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:12:38.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.312+0000 7f5742297740 1 -- 172.21.15.120:0/3426583959 shutdown_connections 2024-09-19T09:12:38.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.312+0000 7f5742297740 1 -- 172.21.15.120:0/3426583959 wait complete. 2024-09-19T09:12:38.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.312+0000 7f5742297740 20 client.18981 trim_cache size 0 max 16384 2024-09-19T09:12:38.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.325+0000 7f5742297740 1 Processor -- start 2024-09-19T09:12:38.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.325+0000 7f5742297740 1 -- start start 2024-09-19T09:12:38.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.326+0000 7f5742297740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615ff2f35d0 0x5615ff2efe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:38.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.326+0000 7f5742297740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615ff2f4b40 0x5615ff30a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:38.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.326+0000 7f5742297740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615ff2fe370 0x5615ff315750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:38.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.326+0000 7f5742297740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f57381af690 con 0x5615ff2f35d0 2024-09-19T09:12:38.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.326+0000 7f5742297740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f57381b11d0 con 0x5615ff2f4b40 2024-09-19T09:12:38.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.326+0000 7f5742297740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f57381afd60 con 0x5615ff2fe370 2024-09-19T09:12:38.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.326+0000 7f573dc5a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615ff2f35d0 0x5615ff2efe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:38.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.326+0000 7f573dc5a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615ff2f35d0 0x5615ff2efe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47926/0 (socket says 172.21.15.120:47926) 2024-09-19T09:12:38.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.326+0000 7f573dc5a640 1 -- 172.21.15.120:0/1300892255 learned_addr learned my addr 172.21.15.120:0/1300892255 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:38.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.326+0000 7f573d459640 1 --2- 172.21.15.120:0/1300892255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615ff2f4b40 0x5615ff30a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:38.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.326+0000 7f573e45b640 1 --2- 172.21.15.120:0/1300892255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615ff2fe370 0x5615ff315750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:38.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.326+0000 7f573dc5a640 1 -- 172.21.15.120:0/1300892255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615ff2fe370 msgr2=0x5615ff315750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:38.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.326+0000 7f573dc5a640 1 --2- 172.21.15.120:0/1300892255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615ff2fe370 0x5615ff315750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.326+0000 7f573dc5a640 1 -- 172.21.15.120:0/1300892255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615ff2f4b40 msgr2=0x5615ff30a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:38.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.326+0000 7f573dc5a640 1 --2- 172.21.15.120:0/1300892255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615ff2f4b40 0x5615ff30a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.326+0000 7f573dc5a640 1 -- 172.21.15.120:0/1300892255 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615ff2fe030 con 0x5615ff2f35d0 2024-09-19T09:12:38.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.326+0000 7f573d459640 1 --2- 172.21.15.120:0/1300892255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615ff2f4b40 0x5615ff30a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:38.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.326+0000 7f573dc5a640 1 --2- 172.21.15.120:0/1300892255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615ff2f35d0 0x5615ff2efe10 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f5734005e40 tx=0x7f57340317a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ec65041009fa0f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:38.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.326+0000 7f57237fe640 1 -- 172.21.15.120:0/1300892255 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573405e070 con 0x5615ff2f35d0 2024-09-19T09:12:38.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.326+0000 7f57237fe640 1 -- 172.21.15.120:0/1300892255 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f573404a070 con 0x5615ff2f35d0 2024-09-19T09:12:38.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.326+0000 7f57237fe640 1 -- 172.21.15.120:0/1300892255 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5734053070 con 0x5615ff2f35d0 2024-09-19T09:12:38.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.327+0000 7f5742297740 1 -- 172.21.15.120:0/1300892255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615ff2f35d0 msgr2=0x5615ff2efe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:38.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.327+0000 7f5742297740 1 --2- 172.21.15.120:0/1300892255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615ff2f35d0 0x5615ff2efe10 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f5734005e40 tx=0x7f57340317a0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.327+0000 7f5742297740 1 -- 172.21.15.120:0/1300892255 shutdown_connections 2024-09-19T09:12:38.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.327+0000 7f5742297740 1 --2- 172.21.15.120:0/1300892255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615ff2fe370 0x5615ff315750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.327+0000 7f5742297740 1 --2- 172.21.15.120:0/1300892255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615ff2f4b40 0x5615ff30a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.327+0000 7f5742297740 1 --2- 172.21.15.120:0/1300892255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615ff2f35d0 0x5615ff2efe10 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.327+0000 7f5742297740 1 -- 172.21.15.120:0/1300892255 >> 172.21.15.120:0/1300892255 conn(0x5615ff2bc310 msgr2=0x5615ff2ef700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:38.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.327+0000 7f5742297740 1 -- 172.21.15.120:0/1300892255 shutdown_connections 2024-09-19T09:12:38.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.327+0000 7f5742297740 1 -- 172.21.15.120:0/1300892255 wait complete. 2024-09-19T09:12:38.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.328+0000 7f5742297740 1 -- start start 2024-09-19T09:12:38.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.328+0000 7f5742297740 1 Processor -- start 2024-09-19T09:12:38.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.328+0000 7f5742297740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615ff2f35d0 0x5615ff3014e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:38.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.328+0000 7f5742297740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615ff2f4b40 0x5615ff301a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:38.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.328+0000 7f5742297740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615ff2fe370 0x5615ff376710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:38.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.328+0000 7f573dc5a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615ff2f35d0 0x5615ff3014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:38.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.328+0000 7f5742297740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f57381b0b00 con 0x5615ff2fe370 2024-09-19T09:12:38.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.328+0000 7f5742297740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f57381b0430 con 0x5615ff2f35d0 2024-09-19T09:12:38.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.328+0000 7f5742297740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f57381ad650 con 0x5615ff2f4b40 2024-09-19T09:12:38.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.328+0000 7f573dc5a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615ff2f35d0 0x5615ff3014e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52940/0 (socket says 172.21.15.120:52940) 2024-09-19T09:12:38.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.328+0000 7f573dc5a640 1 -- 172.21.15.120:0/2081099957 learned_addr learned my addr 172.21.15.120:0/2081099957 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:38.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.329+0000 7f573e45b640 1 --2- 172.21.15.120:0/2081099957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615ff2fe370 0x5615ff376710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:38.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.329+0000 7f573d459640 1 --2- 172.21.15.120:0/2081099957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615ff2f4b40 0x5615ff301a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:38.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.329+0000 7f5742297740 10 client.0 _ll_register_callbacks cb 0x7f5730009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:38.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.329+0000 7f573dc5a640 1 -- 172.21.15.120:0/2081099957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615ff2f4b40 msgr2=0x5615ff301a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:38.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.329+0000 7f573dc5a640 1 --2- 172.21.15.120:0/2081099957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615ff2f4b40 0x5615ff301a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.329+0000 7f573dc5a640 1 -- 172.21.15.120:0/2081099957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615ff2fe370 msgr2=0x5615ff376710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:38.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.329+0000 7f573dc5a640 1 --2- 172.21.15.120:0/2081099957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615ff2fe370 0x5615ff376710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.329+0000 7f573dc5a640 1 -- 172.21.15.120:0/2081099957 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615ff3022f0 con 0x5615ff2f35d0 2024-09-19T09:12:38.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.329+0000 7f5742297740 1 -- 172.21.15.120:0/2081099957 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f573404a070 con 0x5615ff2f35d0 2024-09-19T09:12:38.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.329+0000 7f573dc5a640 1 --2- 172.21.15.120:0/2081099957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615ff2f35d0 0x5615ff3014e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5734009400 tx=0x7f5734031f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70a1e374e6325a99 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:38.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.329+0000 7f57217fa640 10 client.28109 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:12:38.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.329+0000 7f57217fa640 1 -- 172.21.15.120:0/2081099957 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5734067070 con 0x5615ff2f35d0 2024-09-19T09:12:38.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.329+0000 7f57217fa640 1 -- 172.21.15.120:0/2081099957 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f57340048d0 con 0x5615ff2f35d0 2024-09-19T09:12:38.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.329+0000 7f57217fa640 1 -- 172.21.15.120:0/2081099957 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f573405e070 con 0x5615ff2f35d0 2024-09-19T09:12:38.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.329+0000 7f5742297740 10 client.28109 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:38.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.329+0000 7f5742297740 1 -- 172.21.15.120:0/2081099957 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({fsmap=32}) -- 0x5615ff377c50 con 0x5615ff2f35d0 2024-09-19T09:12:38.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.330+0000 7f57217fa640 1 -- 172.21.15.120:0/2081099957 <== mon.2 v2:172.21.15.153:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f5734058040 con 0x5615ff2f35d0 2024-09-19T09:12:38.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.330+0000 7f5742297740 10 client.28109 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:38.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.330+0000 7f5742297740 10 client.28109 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:12:38.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.330+0000 7f5742297740 20 client.28109 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:38.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.330+0000 7f5742297740 1 --2- 172.21.15.120:0/2081099957 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5615ff3781b0 0x5615ff37a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:38.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.330+0000 7f5742297740 4 client.28109 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:12:38.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.330+0000 7f5742297740 1 -- 172.21.15.120:0/2081099957 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5615ff260d70 con 0x5615ff3781b0 2024-09-19T09:12:38.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.330+0000 7f573d459640 1 --2- 172.21.15.120:0/2081099957 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5615ff3781b0 0x5615ff37a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:38.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.330+0000 7f573d459640 1 --2- 172.21.15.120:0/2081099957 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5615ff3781b0 0x5615ff37a5a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=91c43c998dbe71ef server_cookie=633030d254eb40b0 in_seq=0 out_seq=0 2024-09-19T09:12:38.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.330+0000 7f57217fa640 10 client.28109 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:12:38.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.331+0000 7f57217fa640 1 -- 172.21.15.120:0/2081099957 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5615ff260d70 con 0x5615ff3781b0 2024-09-19T09:12:38.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.331+0000 7f57217fa640 10 client.28109 handle_command_reply: tid=0 2024-09-19T09:12:38.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.331+0000 7f5742297740 1 client.28109 shutdown 2024-09-19T09:12:38.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.331+0000 7f5742297740 1 -- 172.21.15.120:0/2081099957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615ff2f35d0 msgr2=0x5615ff3014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:38.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.331+0000 7f5742297740 1 --2- 172.21.15.120:0/2081099957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615ff2f35d0 0x5615ff3014e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5734009400 tx=0x7f5734031f20 comp rx=0 tx=0).stop 2024-09-19T09:12:38.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.331+0000 7f5742297740 1 -- 172.21.15.120:0/2081099957 shutdown_connections 2024-09-19T09:12:38.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.331+0000 7f5742297740 1 --2- 172.21.15.120:0/2081099957 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5615ff3781b0 0x5615ff37a5a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.331+0000 7f5742297740 1 --2- 172.21.15.120:0/2081099957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615ff2fe370 0x5615ff376710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.331+0000 7f5742297740 1 --2- 172.21.15.120:0/2081099957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615ff2f4b40 0x5615ff301a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.331+0000 7f57217fa640 0 client.28109 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:12:38.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.331+0000 7f5742297740 1 --2- 172.21.15.120:0/2081099957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615ff2f35d0 0x5615ff3014e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.331+0000 7f5742297740 1 -- 172.21.15.120:0/2081099957 >> 172.21.15.120:0/2081099957 conn(0x5615ff2bc310 msgr2=0x5615ff2f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:38.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.331+0000 7f5742297740 1 -- 172.21.15.120:0/2081099957 shutdown_connections 2024-09-19T09:12:38.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.331+0000 7f5742297740 1 -- 172.21.15.120:0/2081099957 wait complete. 2024-09-19T09:12:38.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.331+0000 7f5742297740 20 client.28109 trim_cache size 0 max 16384 2024-09-19T09:12:38.338 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:12:38.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.332+0000 7f573febe640 1 -- 172.21.15.120:0/243502760 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f57240a32c0 msgr2=0x7f57240a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:38.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.332+0000 7f573febe640 1 --2- 172.21.15.120:0/243502760 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f57240a32c0 0x7f57240a5780 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5730005c40 tx=0x7f5730009500 comp rx=0 tx=0).stop 2024-09-19T09:12:38.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.332+0000 7f573febe640 1 -- 172.21.15.120:0/243502760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5738102570 msgr2=0x7f5738197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:38.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.332+0000 7f573febe640 1 --2- 172.21.15.120:0/243502760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5738102570 0x7f5738197710 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f572800e9f0 tx=0x7f572800eeb0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.332+0000 7f573febe640 1 -- 172.21.15.120:0/243502760 shutdown_connections 2024-09-19T09:12:38.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.332+0000 7f573febe640 1 --2- 172.21.15.120:0/243502760 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f57240a32c0 0x7f57240a5780 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.332+0000 7f573febe640 1 --2- 172.21.15.120:0/243502760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5738102ef0 0x7f5738197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.332+0000 7f573febe640 1 --2- 172.21.15.120:0/243502760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5738102570 0x7f5738197710 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.332+0000 7f573febe640 1 --2- 172.21.15.120:0/243502760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5738100e20 0x7f57381971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:38.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.332+0000 7f573febe640 1 -- 172.21.15.120:0/243502760 >> 172.21.15.120:0/243502760 conn(0x7f57380fc7d0 msgr2=0x7f57381068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:38.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.332+0000 7f573febe640 1 -- 172.21.15.120:0/243502760 shutdown_connections 2024-09-19T09:12:38.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:38.332+0000 7f573febe640 1 -- 172.21.15.120:0/243502760 wait complete. 2024-09-19T09:12:39.036 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:12:39.066 INFO:tasks.workunit.client.0.smithi120.stdout: 1 967157 10.20 MB/sec execute 394 sec latency 10.712 ms 2024-09-19T09:12:39.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:38 smithi196 ceph-mon[35150]: pgmap v636: 97 pgs: 97 active+clean; 29 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 3.8 MiB/s wr, 428 op/s 2024-09-19T09:12:39.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:38 smithi153 ceph-mon[35071]: pgmap v636: 97 pgs: 97 active+clean; 29 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 3.8 MiB/s wr, 428 op/s 2024-09-19T09:12:39.297 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:12:39.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:38 smithi120 ceph-mon[31150]: pgmap v636: 97 pgs: 97 active+clean; 29 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 3.8 MiB/s wr, 428 op/s 2024-09-19T09:12:39.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.785+0000 7f54d35f1640 1 -- 172.21.15.120:0/1779267797 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54cc101df0 msgr2=0x7f54cc1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:39.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.785+0000 7f54d35f1640 1 --2- 172.21.15.120:0/1779267797 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54cc101df0 0x7f54cc1021d0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f54c8009900 tx=0x7f54c802a410 comp rx=0 tx=0).stop 2024-09-19T09:12:39.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.785+0000 7f54d35f1640 1 -- 172.21.15.120:0/1779267797 shutdown_connections 2024-09-19T09:12:39.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.785+0000 7f54d35f1640 1 --2- 172.21.15.120:0/1779267797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54cc106f20 0x7f54cc10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.785+0000 7f54d35f1640 1 --2- 172.21.15.120:0/1779267797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54cc1027a0 0x7f54cc1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.785+0000 7f54d35f1640 1 --2- 172.21.15.120:0/1779267797 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54cc101df0 0x7f54cc1021d0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.785+0000 7f54d35f1640 1 -- 172.21.15.120:0/1779267797 >> 172.21.15.120:0/1779267797 conn(0x7f54cc0fd8b0 msgr2=0x7f54cc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:39.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.785+0000 7f54d35f1640 1 -- 172.21.15.120:0/1779267797 shutdown_connections 2024-09-19T09:12:39.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.785+0000 7f54d35f1640 1 -- 172.21.15.120:0/1779267797 wait complete. 2024-09-19T09:12:39.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.786+0000 7f54d35f1640 1 Processor -- start 2024-09-19T09:12:39.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.786+0000 7f54d35f1640 1 -- start start 2024-09-19T09:12:39.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.786+0000 7f54d35f1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54cc101df0 0x7f54cc198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:39.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.786+0000 7f54d35f1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54cc1027a0 0x7f54cc198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:39.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.786+0000 7f54d35f1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54cc106f20 0x7f54cc19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:39.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.786+0000 7f54d35f1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f54cc111f80 con 0x7f54cc1027a0 2024-09-19T09:12:39.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.786+0000 7f54d35f1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f54cc111e00 con 0x7f54cc106f20 2024-09-19T09:12:39.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.786+0000 7f54d35f1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f54cc112100 con 0x7f54cc101df0 2024-09-19T09:12:39.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.786+0000 7f54d0b8c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54cc1027a0 0x7f54cc198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:39.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.786+0000 7f54d0b8c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54cc1027a0 0x7f54cc198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47944/0 (socket says 172.21.15.120:47944) 2024-09-19T09:12:39.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.786+0000 7f54d0b8c640 1 -- 172.21.15.120:0/2932141936 learned_addr learned my addr 172.21.15.120:0/2932141936 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:39.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.786+0000 7f54d1b8e640 1 --2- 172.21.15.120:0/2932141936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54cc106f20 0x7f54cc19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:39.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.787+0000 7f54d138d640 1 --2- 172.21.15.120:0/2932141936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54cc101df0 0x7f54cc198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:39.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.787+0000 7f54d0b8c640 1 -- 172.21.15.120:0/2932141936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54cc101df0 msgr2=0x7f54cc198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:39.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.787+0000 7f54d0b8c640 1 --2- 172.21.15.120:0/2932141936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54cc101df0 0x7f54cc198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.787+0000 7f54d0b8c640 1 -- 172.21.15.120:0/2932141936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54cc106f20 msgr2=0x7f54cc19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:39.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.787+0000 7f54d0b8c640 1 --2- 172.21.15.120:0/2932141936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54cc106f20 0x7f54cc19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.787+0000 7f54d0b8c640 1 -- 172.21.15.120:0/2932141936 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54cc19d3b0 con 0x7f54cc1027a0 2024-09-19T09:12:39.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.787+0000 7f54d0b8c640 1 --2- 172.21.15.120:0/2932141936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54cc1027a0 0x7f54cc198940 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f54bc00e9f0 tx=0x7f54bc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:39.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.787+0000 7f54c27fc640 1 -- 172.21.15.120:0/2932141936 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc00cd80 con 0x7f54cc1027a0 2024-09-19T09:12:39.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.787+0000 7f54d35f1640 1 -- 172.21.15.120:0/2932141936 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54cc19d6a0 con 0x7f54cc1027a0 2024-09-19T09:12:39.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.787+0000 7f54c27fc640 1 -- 172.21.15.120:0/2932141936 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54bc005500 con 0x7f54cc1027a0 2024-09-19T09:12:39.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.787+0000 7f54c27fc640 1 -- 172.21.15.120:0/2932141936 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc010690 con 0x7f54cc1027a0 2024-09-19T09:12:39.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.787+0000 7f54d35f1640 1 -- 172.21.15.120:0/2932141936 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54cc1a4f80 con 0x7f54cc1027a0 2024-09-19T09:12:39.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.789+0000 7f54c27fc640 1 -- 172.21.15.120:0/2932141936 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f54bc0046c0 con 0x7f54cc1027a0 2024-09-19T09:12:39.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.790+0000 7f54d59ca740 1 Processor -- start 2024-09-19T09:12:39.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.790+0000 7f54d59ca740 1 -- start start 2024-09-19T09:12:39.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.790+0000 7f54d59ca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a00fc50b40 0x55a00fc4f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:39.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.790+0000 7f54d59ca740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a00fc4f5d0 0x55a00fc4fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:39.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.790+0000 7f54d59ca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a00fc5a370 0x55a00fc5c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:39.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.790+0000 7f54d59ca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a00fbb52f0 con 0x55a00fc5a370 2024-09-19T09:12:39.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.790+0000 7f54d59ca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a00fb5ccf0 con 0x55a00fc50b40 2024-09-19T09:12:39.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.790+0000 7f54d59ca740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a00fbb12b0 con 0x55a00fc4f5d0 2024-09-19T09:12:39.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.790+0000 7f54d1b8e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a00fc50b40 0x55a00fc4f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:39.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.790+0000 7f54d0b8c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a00fc5a370 0x55a00fc5c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:39.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.790+0000 7f54d1b8e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a00fc50b40 0x55a00fc4f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52964/0 (socket says 172.21.15.120:52964) 2024-09-19T09:12:39.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.790+0000 7f54d1b8e640 1 -- 172.21.15.120:0/440323748 learned_addr learned my addr 172.21.15.120:0/440323748 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:39.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.790+0000 7f54d138d640 1 --2- 172.21.15.120:0/440323748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a00fc4f5d0 0x55a00fc4fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:39.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.790+0000 7f54d1b8e640 1 -- 172.21.15.120:0/440323748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a00fc4f5d0 msgr2=0x55a00fc4fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:39.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.790+0000 7f54d1b8e640 1 --2- 172.21.15.120:0/440323748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a00fc4f5d0 0x55a00fc4fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.790+0000 7f54d1b8e640 1 -- 172.21.15.120:0/440323748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a00fc5a370 msgr2=0x55a00fc5c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:39.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.790+0000 7f54d1b8e640 1 --2- 172.21.15.120:0/440323748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a00fc5a370 0x55a00fc5c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.790+0000 7f54d1b8e640 1 -- 172.21.15.120:0/440323748 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a00fb413a0 con 0x55a00fc50b40 2024-09-19T09:12:39.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.790+0000 7f54c27fc640 1 --2- 172.21.15.120:0/2932141936 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f54b80a3240 0x7f54b80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:39.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.791+0000 7f54d138d640 1 --2- 172.21.15.120:0/2932141936 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f54b80a3240 0x7f54b80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:39.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.791+0000 7f54d1b8e640 1 --2- 172.21.15.120:0/440323748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a00fc50b40 0x55a00fc4f090 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f54c4009f50 tx=0x7f54c40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19a3d0acbd2334f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:39.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.791+0000 7f54c27fc640 1 -- 172.21.15.120:0/2932141936 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f54bc0cfde0 con 0x7f54cc1027a0 2024-09-19T09:12:39.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.791+0000 7f54b77fe640 1 -- 172.21.15.120:0/440323748 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c4004d20 con 0x55a00fc50b40 2024-09-19T09:12:39.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.791+0000 7f54b77fe640 1 -- 172.21.15.120:0/440323748 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54c4004050 con 0x55a00fc50b40 2024-09-19T09:12:39.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.791+0000 7f54b77fe640 1 -- 172.21.15.120:0/440323748 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c4005240 con 0x55a00fc50b40 2024-09-19T09:12:39.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.792+0000 7f54d59ca740 1 -- 172.21.15.120:0/440323748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a00fc50b40 msgr2=0x55a00fc4f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:39.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.792+0000 7f54d138d640 1 --2- 172.21.15.120:0/2932141936 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f54b80a3240 0x7f54b80a5700 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f54c8005940 tx=0x7f54c80058b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:39.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.792+0000 7f54d59ca740 1 --2- 172.21.15.120:0/440323748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a00fc50b40 0x55a00fc4f090 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f54c4009f50 tx=0x7f54c40118d0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.792+0000 7f54d59ca740 1 -- 172.21.15.120:0/440323748 shutdown_connections 2024-09-19T09:12:39.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.792+0000 7f54d59ca740 1 --2- 172.21.15.120:0/440323748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a00fc5a370 0x55a00fc5c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.792+0000 7f54d59ca740 1 --2- 172.21.15.120:0/440323748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a00fc4f5d0 0x55a00fc4fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.792+0000 7f54d59ca740 1 --2- 172.21.15.120:0/440323748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a00fc50b40 0x55a00fc4f090 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.792+0000 7f54d59ca740 1 -- 172.21.15.120:0/440323748 >> 172.21.15.120:0/440323748 conn(0x55a00fc18310 msgr2=0x55a00fc4bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:39.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.792+0000 7f54d59ca740 1 -- 172.21.15.120:0/440323748 shutdown_connections 2024-09-19T09:12:39.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.792+0000 7f54d59ca740 1 -- 172.21.15.120:0/440323748 wait complete. 2024-09-19T09:12:39.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.792+0000 7f54d59ca740 1 -- start start 2024-09-19T09:12:39.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.792+0000 7f54d59ca740 1 Processor -- start 2024-09-19T09:12:39.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.793+0000 7f54d59ca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a00fc4f5d0 0x55a00fc59d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:39.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.793+0000 7f54d59ca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a00fc50b40 0x55a00fc57000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:39.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.793+0000 7f54d59ca740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a00fc5a370 0x55a00fc575d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:39.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.793+0000 7f54d59ca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a00fc5d070 con 0x55a00fc50b40 2024-09-19T09:12:39.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.793+0000 7f54d59ca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a00fb70060 con 0x55a00fc4f5d0 2024-09-19T09:12:39.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.793+0000 7f54d59ca740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a00fc5d1f0 con 0x55a00fc5a370 2024-09-19T09:12:39.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.793+0000 7f54d59ca740 10 client.0 _ll_register_callbacks cb 0x55a00f8c38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:39.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.793+0000 7f54d0b8c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a00fc50b40 0x55a00fc57000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:39.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.793+0000 7f54d0b8c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a00fc50b40 0x55a00fc57000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47950/0 (socket says 172.21.15.120:47950) 2024-09-19T09:12:39.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.793+0000 7f54d0b8c640 1 -- 172.21.15.120:0/3553646126 learned_addr learned my addr 172.21.15.120:0/3553646126 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:39.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.793+0000 7f54d138d640 1 --2- 172.21.15.120:0/3553646126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a00fc4f5d0 0x55a00fc59d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:39.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.793+0000 7f54d1b8e640 1 --2- 172.21.15.120:0/3553646126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a00fc5a370 0x55a00fc575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:39.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.793+0000 7f54d0b8c640 1 -- 172.21.15.120:0/3553646126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a00fc5a370 msgr2=0x55a00fc575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:39.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.793+0000 7f54d0b8c640 1 --2- 172.21.15.120:0/3553646126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a00fc5a370 0x55a00fc575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.793+0000 7f54d0b8c640 1 -- 172.21.15.120:0/3553646126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a00fc4f5d0 msgr2=0x55a00fc59d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:39.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.793+0000 7f54d0b8c640 1 --2- 172.21.15.120:0/3553646126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a00fc4f5d0 0x55a00fc59d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.793+0000 7f54d0b8c640 1 -- 172.21.15.120:0/3553646126 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a00f9263c0 con 0x55a00fc50b40 2024-09-19T09:12:39.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.793+0000 7f54d1b8e640 1 --2- 172.21.15.120:0/3553646126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a00fc5a370 0x55a00fc575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:12:39.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.793+0000 7f54d0b8c640 1 --2- 172.21.15.120:0/3553646126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a00fc50b40 0x55a00fc57000 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f54bc09c350 tx=0x7f54bc004ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a81cfa1f6cf72a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:39.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.793+0000 7f54b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:39.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.793+0000 7f54b4ff9640 1 -- 172.21.15.120:0/3553646126 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc09bf00 con 0x55a00fc50b40 2024-09-19T09:12:39.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.793+0000 7f54b4ff9640 1 -- 172.21.15.120:0/3553646126 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f54bc0a4ba0 con 0x55a00fc50b40 2024-09-19T09:12:39.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.793+0000 7f54d59ca740 1 -- 172.21.15.120:0/3553646126 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54c4004050 con 0x55a00fc50b40 2024-09-19T09:12:39.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.794+0000 7f54b4ff9640 1 -- 172.21.15.120:0/3553646126 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f54bc09bd20 con 0x55a00fc50b40 2024-09-19T09:12:39.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.794+0000 7f54d59ca740 10 client.18999 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:39.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.794+0000 7f54d59ca740 1 -- 172.21.15.120:0/3553646126 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a00fc68b50 con 0x55a00fc50b40 2024-09-19T09:12:39.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.794+0000 7f54b4ff9640 1 -- 172.21.15.120:0/3553646126 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54c4004050 con 0x55a00fc50b40 2024-09-19T09:12:39.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.794+0000 7f54b4ff9640 1 -- 172.21.15.120:0/3553646126 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f54bc0b50f0 con 0x55a00fc50b40 2024-09-19T09:12:39.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.794+0000 7f54d59ca740 10 client.18999 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:39.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.794+0000 7f54d59ca740 10 client.18999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:12:39.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.794+0000 7f54d59ca740 20 client.18999 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:39.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.794+0000 7f54d59ca740 1 --2- 172.21.15.120:0/3553646126 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a00fc69f60 0x55a00fc6c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:39.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.794+0000 7f54d59ca740 4 client.18999 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:12:39.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.794+0000 7f54d59ca740 1 -- 172.21.15.120:0/3553646126 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a00fbc2860 con 0x55a00fc69f60 2024-09-19T09:12:39.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.794+0000 7f54d138d640 1 --2- 172.21.15.120:0/3553646126 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a00fc69f60 0x55a00fc6c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:39.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.795+0000 7f54b4ff9640 10 client.18999 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:39.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.795+0000 7f54d138d640 1 --2- 172.21.15.120:0/3553646126 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a00fc69f60 0x55a00fc6c370 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3d6bb1f9dd69f63 server_cookie=7d6b36f51e1f4c5c in_seq=0 out_seq=0 2024-09-19T09:12:39.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.796+0000 7f54b4ff9640 1 -- 172.21.15.120:0/3553646126 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a00fbc2860 con 0x55a00fc69f60 2024-09-19T09:12:39.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.796+0000 7f54b4ff9640 10 client.18999 handle_command_reply: tid=0 2024-09-19T09:12:39.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.796+0000 7f54d59ca740 1 client.18999 shutdown 2024-09-19T09:12:39.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.796+0000 7f54d59ca740 1 -- 172.21.15.120:0/3553646126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a00fc50b40 msgr2=0x55a00fc57000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:39.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.796+0000 7f54d59ca740 1 --2- 172.21.15.120:0/3553646126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a00fc50b40 0x55a00fc57000 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f54bc09c350 tx=0x7f54bc004ae0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.796+0000 7f54d59ca740 1 -- 172.21.15.120:0/3553646126 shutdown_connections 2024-09-19T09:12:39.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.796+0000 7f54d59ca740 1 --2- 172.21.15.120:0/3553646126 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a00fc69f60 0x55a00fc6c370 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.796+0000 7f54d59ca740 1 --2- 172.21.15.120:0/3553646126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a00fc5a370 0x55a00fc575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.796+0000 7f54d59ca740 1 --2- 172.21.15.120:0/3553646126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a00fc50b40 0x55a00fc57000 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.796+0000 7f54b4ff9640 0 client.18999 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:39.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.796+0000 7f54d59ca740 1 --2- 172.21.15.120:0/3553646126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a00fc4f5d0 0x55a00fc59d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.796+0000 7f54d59ca740 1 -- 172.21.15.120:0/3553646126 >> 172.21.15.120:0/3553646126 conn(0x55a00fc18310 msgr2=0x55a00fc51960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:39.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.796+0000 7f54d59ca740 1 -- 172.21.15.120:0/3553646126 shutdown_connections 2024-09-19T09:12:39.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.796+0000 7f54d59ca740 1 -- 172.21.15.120:0/3553646126 wait complete. 2024-09-19T09:12:39.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.797+0000 7f54d59ca740 20 client.18999 trim_cache size 0 max 16384 2024-09-19T09:12:39.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.810+0000 7f54d59ca740 1 Processor -- start 2024-09-19T09:12:39.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.810+0000 7f54d59ca740 1 -- start start 2024-09-19T09:12:39.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.811+0000 7f54d59ca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a00fc4f5d0 0x55a00fc66310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:39.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.811+0000 7f54d59ca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a00fc50b40 0x55a00fc52120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:39.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.811+0000 7f54d59ca740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a00fc5a370 0x55a00fc52660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:39.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.811+0000 7f54d59ca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f54cc1ad630 con 0x55a00fc4f5d0 2024-09-19T09:12:39.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.811+0000 7f54d59ca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f54cc1af170 con 0x55a00fc50b40 2024-09-19T09:12:39.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.811+0000 7f54d59ca740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f54cc1add00 con 0x55a00fc5a370 2024-09-19T09:12:39.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.811+0000 7f54d138d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a00fc4f5d0 0x55a00fc66310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:39.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.811+0000 7f54d138d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a00fc4f5d0 0x55a00fc66310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47964/0 (socket says 172.21.15.120:47964) 2024-09-19T09:12:39.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.811+0000 7f54d1b8e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a00fc5a370 0x55a00fc52660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:39.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.811+0000 7f54d138d640 1 -- 172.21.15.120:0/3729203457 learned_addr learned my addr 172.21.15.120:0/3729203457 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:39.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.811+0000 7f54d0b8c640 1 --2- 172.21.15.120:0/3729203457 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a00fc50b40 0x55a00fc52120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:39.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.811+0000 7f54d138d640 1 -- 172.21.15.120:0/3729203457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a00fc5a370 msgr2=0x55a00fc52660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:39.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.811+0000 7f54d138d640 1 --2- 172.21.15.120:0/3729203457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a00fc5a370 0x55a00fc52660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.811+0000 7f54d138d640 1 -- 172.21.15.120:0/3729203457 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a00fc50b40 msgr2=0x55a00fc52120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:39.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.811+0000 7f54d138d640 1 --2- 172.21.15.120:0/3729203457 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a00fc50b40 0x55a00fc52120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.811+0000 7f54d138d640 1 -- 172.21.15.120:0/3729203457 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a00fc59970 con 0x55a00fc4f5d0 2024-09-19T09:12:39.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.811+0000 7f54d1b8e640 1 --2- 172.21.15.120:0/3729203457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a00fc5a370 0x55a00fc52660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:39.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.811+0000 7f54d138d640 1 --2- 172.21.15.120:0/3729203457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a00fc4f5d0 0x55a00fc66310 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f54c8009900 tx=0x7f54c803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5c9a98a1e16855 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:39.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.811+0000 7f54b6ffd640 1 -- 172.21.15.120:0/3729203457 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8035040 con 0x55a00fc4f5d0 2024-09-19T09:12:39.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.812+0000 7f54b6ffd640 1 -- 172.21.15.120:0/3729203457 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54c8055070 con 0x55a00fc4f5d0 2024-09-19T09:12:39.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.812+0000 7f54b6ffd640 1 -- 172.21.15.120:0/3729203457 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8004920 con 0x55a00fc4f5d0 2024-09-19T09:12:39.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.812+0000 7f54d59ca740 1 -- 172.21.15.120:0/3729203457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a00fc4f5d0 msgr2=0x55a00fc66310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:39.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.812+0000 7f54d59ca740 1 --2- 172.21.15.120:0/3729203457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a00fc4f5d0 0x55a00fc66310 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f54c8009900 tx=0x7f54c803e000 comp rx=0 tx=0).stop 2024-09-19T09:12:39.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.812+0000 7f54d59ca740 1 -- 172.21.15.120:0/3729203457 shutdown_connections 2024-09-19T09:12:39.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.812+0000 7f54d59ca740 1 --2- 172.21.15.120:0/3729203457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a00fc5a370 0x55a00fc52660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.812+0000 7f54d59ca740 1 --2- 172.21.15.120:0/3729203457 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a00fc50b40 0x55a00fc52120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.812+0000 7f54d59ca740 1 --2- 172.21.15.120:0/3729203457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a00fc4f5d0 0x55a00fc66310 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.812+0000 7f54d59ca740 1 -- 172.21.15.120:0/3729203457 >> 172.21.15.120:0/3729203457 conn(0x55a00fc18310 msgr2=0x55a00fc560f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:39.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.812+0000 7f54d59ca740 1 -- 172.21.15.120:0/3729203457 shutdown_connections 2024-09-19T09:12:39.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.812+0000 7f54d59ca740 1 -- 172.21.15.120:0/3729203457 wait complete. 2024-09-19T09:12:39.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.813+0000 7f54d59ca740 1 -- start start 2024-09-19T09:12:39.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.813+0000 7f54d59ca740 1 Processor -- start 2024-09-19T09:12:39.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.813+0000 7f54d59ca740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a00fc4f5d0 0x55a00fc62d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:39.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.813+0000 7f54d59ca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a00fc50b40 0x55a00fc53e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:39.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.813+0000 7f54d59ca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a00fc5a370 0x55a00fc54600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:39.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.813+0000 7f54d59ca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f54cc1aeaa0 con 0x55a00fc50b40 2024-09-19T09:12:39.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.813+0000 7f54d59ca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f54cc1ae3d0 con 0x55a00fc5a370 2024-09-19T09:12:39.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.813+0000 7f54d59ca740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f54cc1ab5f0 con 0x55a00fc4f5d0 2024-09-19T09:12:39.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.813+0000 7f54d59ca740 10 client.0 _ll_register_callbacks cb 0x7f54bc0d3510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:39.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.813+0000 7f54d0b8c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a00fc50b40 0x55a00fc53e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:39.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.813+0000 7f54d0b8c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a00fc50b40 0x55a00fc53e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47980/0 (socket says 172.21.15.120:47980) 2024-09-19T09:12:39.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.813+0000 7f54d0b8c640 1 -- 172.21.15.120:0/1267128898 learned_addr learned my addr 172.21.15.120:0/1267128898 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:39.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.813+0000 7f54d138d640 1 --2- 172.21.15.120:0/1267128898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a00fc4f5d0 0x55a00fc62d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:39.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.813+0000 7f54d1b8e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a00fc5a370 0x55a00fc54600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:39.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.814+0000 7f54d0b8c640 1 -- 172.21.15.120:0/1267128898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a00fc4f5d0 msgr2=0x55a00fc62d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:39.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.814+0000 7f54d0b8c640 1 --2- 172.21.15.120:0/1267128898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a00fc4f5d0 0x55a00fc62d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.814+0000 7f54d0b8c640 1 -- 172.21.15.120:0/1267128898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a00fc5a370 msgr2=0x55a00fc54600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:39.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.814+0000 7f54d0b8c640 1 --2- 172.21.15.120:0/1267128898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a00fc5a370 0x55a00fc54600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.814+0000 7f54d0b8c640 1 -- 172.21.15.120:0/1267128898 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a00fcd3c10 con 0x55a00fc50b40 2024-09-19T09:12:39.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.814+0000 7f54d138d640 1 --2- 172.21.15.120:0/1267128898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a00fc4f5d0 0x55a00fc62d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:39.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.814+0000 7f54d0b8c640 1 --2- 172.21.15.120:0/1267128898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a00fc50b40 0x55a00fc53e70 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f54bc0b5530 tx=0x7f54bc00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27942d48442c2c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:39.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.814+0000 7f54b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:39.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.814+0000 7f54b4ff9640 1 -- 172.21.15.120:0/1267128898 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc09bf00 con 0x55a00fc50b40 2024-09-19T09:12:39.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.814+0000 7f54b4ff9640 1 -- 172.21.15.120:0/1267128898 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f54bc098110 con 0x55a00fc50b40 2024-09-19T09:12:39.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.814+0000 7f54d59ca740 1 -- 172.21.15.120:0/1267128898 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54c8055070 con 0x55a00fc50b40 2024-09-19T09:12:39.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.814+0000 7f54b4ff9640 1 -- 172.21.15.120:0/1267128898 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f54bc098660 con 0x55a00fc50b40 2024-09-19T09:12:39.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.814+0000 7f54d59ca740 10 client.19005 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:39.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.814+0000 7f54d59ca740 1 -- 172.21.15.120:0/1267128898 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a00fcd4d90 con 0x55a00fc50b40 2024-09-19T09:12:39.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.814+0000 7f54b4ff9640 1 -- 172.21.15.120:0/1267128898 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54c8055070 con 0x55a00fc50b40 2024-09-19T09:12:39.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.815+0000 7f54b4ff9640 1 -- 172.21.15.120:0/1267128898 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f54bc098310 con 0x55a00fc50b40 2024-09-19T09:12:39.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.815+0000 7f54d59ca740 10 client.19005 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:39.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.815+0000 7f54d59ca740 10 client.19005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:12:39.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.815+0000 7f54d59ca740 20 client.19005 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:39.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.815+0000 7f54d59ca740 1 --2- 172.21.15.120:0/1267128898 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a00fcd5510 0x55a00fcd7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:39.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.815+0000 7f54d59ca740 4 client.19005 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:12:39.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.815+0000 7f54d59ca740 1 -- 172.21.15.120:0/1267128898 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a00fbbcd70 con 0x55a00fcd5510 2024-09-19T09:12:39.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.815+0000 7f54d138d640 1 --2- 172.21.15.120:0/1267128898 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a00fcd5510 0x55a00fcd7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:39.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.815+0000 7f54b4ff9640 10 client.19005 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:39.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.815+0000 7f54d138d640 1 --2- 172.21.15.120:0/1267128898 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a00fcd5510 0x55a00fcd7930 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cb345f748a0a19b server_cookie=82dc9a13b69ed5e9 in_seq=0 out_seq=0 2024-09-19T09:12:39.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.816+0000 7f54b4ff9640 1 -- 172.21.15.120:0/1267128898 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a00fbbcd70 con 0x55a00fcd5510 2024-09-19T09:12:39.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.816+0000 7f54b4ff9640 10 client.19005 handle_command_reply: tid=0 2024-09-19T09:12:39.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.816+0000 7f54d59ca740 1 client.19005 shutdown 2024-09-19T09:12:39.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.816+0000 7f54d59ca740 1 -- 172.21.15.120:0/1267128898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a00fc50b40 msgr2=0x55a00fc53e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:39.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.816+0000 7f54d59ca740 1 --2- 172.21.15.120:0/1267128898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a00fc50b40 0x55a00fc53e70 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f54bc0b5530 tx=0x7f54bc00cf20 comp rx=0 tx=0).stop 2024-09-19T09:12:39.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.816+0000 7f54d59ca740 1 -- 172.21.15.120:0/1267128898 shutdown_connections 2024-09-19T09:12:39.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.816+0000 7f54d59ca740 1 --2- 172.21.15.120:0/1267128898 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a00fcd5510 0x55a00fcd7930 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.816+0000 7f54d59ca740 1 --2- 172.21.15.120:0/1267128898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a00fc5a370 0x55a00fc54600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.816+0000 7f54d59ca740 1 --2- 172.21.15.120:0/1267128898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a00fc50b40 0x55a00fc53e70 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.816+0000 7f54d59ca740 1 --2- 172.21.15.120:0/1267128898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a00fc4f5d0 0x55a00fc62d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.816+0000 7f54d59ca740 1 -- 172.21.15.120:0/1267128898 >> 172.21.15.120:0/1267128898 conn(0x55a00fc18310 msgr2=0x55a00fc58860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:39.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.816+0000 7f54b4ff9640 0 client.19005 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:39.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.816+0000 7f54d59ca740 1 -- 172.21.15.120:0/1267128898 shutdown_connections 2024-09-19T09:12:39.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.816+0000 7f54d59ca740 1 -- 172.21.15.120:0/1267128898 wait complete. 2024-09-19T09:12:39.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.816+0000 7f54d59ca740 20 client.19005 trim_cache size 0 max 16384 2024-09-19T09:12:39.831 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:12:39.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.817+0000 7f54d35f1640 1 -- 172.21.15.120:0/2932141936 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f54b80a3240 msgr2=0x7f54b80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:39.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.817+0000 7f54d35f1640 1 --2- 172.21.15.120:0/2932141936 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f54b80a3240 0x7f54b80a5700 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f54c8005940 tx=0x7f54c80058b0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.817+0000 7f54d35f1640 1 -- 172.21.15.120:0/2932141936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54cc1027a0 msgr2=0x7f54cc198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:39.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.817+0000 7f54d35f1640 1 --2- 172.21.15.120:0/2932141936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54cc1027a0 0x7f54cc198940 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f54bc00e9f0 tx=0x7f54bc00edf0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.817+0000 7f54d35f1640 1 -- 172.21.15.120:0/2932141936 shutdown_connections 2024-09-19T09:12:39.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.817+0000 7f54d35f1640 1 --2- 172.21.15.120:0/2932141936 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f54b80a3240 0x7f54b80a5700 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.817+0000 7f54d35f1640 1 --2- 172.21.15.120:0/2932141936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54cc106f20 0x7f54cc19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.817+0000 7f54d35f1640 1 --2- 172.21.15.120:0/2932141936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54cc1027a0 0x7f54cc198940 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.817+0000 7f54d35f1640 1 --2- 172.21.15.120:0/2932141936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54cc101df0 0x7f54cc198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:39.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.817+0000 7f54d35f1640 1 -- 172.21.15.120:0/2932141936 >> 172.21.15.120:0/2932141936 conn(0x7f54cc0fd8b0 msgr2=0x7f54cc0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:39.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.817+0000 7f54d35f1640 1 -- 172.21.15.120:0/2932141936 shutdown_connections 2024-09-19T09:12:39.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:39.817+0000 7f54d35f1640 1 -- 172.21.15.120:0/2932141936 wait complete. 2024-09-19T09:12:40.067 INFO:tasks.workunit.client.0.smithi120.stdout: 1 970310 10.23 MB/sec execute 395 sec latency 12.012 ms 2024-09-19T09:12:40.538 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T09:12:40.538 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-19T09:12:40.538 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:12:40.895 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:12:41.067 INFO:tasks.workunit.client.0.smithi120.stdout: 1 973541 10.24 MB/sec execute 396 sec latency 15.796 ms 2024-09-19T09:12:41.069 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:40 smithi120 ceph-mon[31150]: pgmap v637: 97 pgs: 97 active+clean; 31 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 3.8 MiB/s wr, 422 op/s 2024-09-19T09:12:41.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:40 smithi196 ceph-mon[35150]: pgmap v637: 97 pgs: 97 active+clean; 31 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 3.8 MiB/s wr, 422 op/s 2024-09-19T09:12:41.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:40 smithi153 ceph-mon[35071]: pgmap v637: 97 pgs: 97 active+clean; 31 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 3.8 MiB/s wr, 422 op/s 2024-09-19T09:12:41.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.386+0000 7fa37b7ef640 1 -- 172.21.15.120:0/3007217895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa374102d90 msgr2=0x7fa374103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:41.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.386+0000 7fa37b7ef640 1 --2- 172.21.15.120:0/3007217895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa374102d90 0x7fa374103170 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fa370009930 tx=0x7fa37002a420 comp rx=0 tx=0).stop 2024-09-19T09:12:41.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.386+0000 7fa37b7ef640 1 -- 172.21.15.120:0/3007217895 shutdown_connections 2024-09-19T09:12:41.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.386+0000 7fa37b7ef640 1 --2- 172.21.15.120:0/3007217895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa374108290 0x7fa37410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:41.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.386+0000 7fa37b7ef640 1 --2- 172.21.15.120:0/3007217895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa374103740 0x7fa374107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:41.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.386+0000 7fa37b7ef640 1 --2- 172.21.15.120:0/3007217895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa374102d90 0x7fa374103170 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:41.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.386+0000 7fa37b7ef640 1 -- 172.21.15.120:0/3007217895 >> 172.21.15.120:0/3007217895 conn(0x7fa3740fc7d0 msgr2=0x7fa3740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:41.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.386+0000 7fa37b7ef640 1 -- 172.21.15.120:0/3007217895 shutdown_connections 2024-09-19T09:12:41.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.386+0000 7fa37b7ef640 1 -- 172.21.15.120:0/3007217895 wait complete. 2024-09-19T09:12:41.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.387+0000 7fa37b7ef640 1 Processor -- start 2024-09-19T09:12:41.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.387+0000 7fa37b7ef640 1 -- start start 2024-09-19T09:12:41.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.387+0000 7fa37b7ef640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa374102d90 0x7fa3741009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:41.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.387+0000 7fa37b7ef640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa374103740 0x7fa374100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:41.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.387+0000 7fa37b7ef640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa374108290 0x7fa374101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:41.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.387+0000 7fa37b7ef640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa3741120c0 con 0x7fa374102d90 2024-09-19T09:12:41.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.387+0000 7fa37b7ef640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa374111f40 con 0x7fa374103740 2024-09-19T09:12:41.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.387+0000 7fa37b7ef640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa374112240 con 0x7fa374108290 2024-09-19T09:12:41.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.387+0000 7fa37958b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa374102d90 0x7fa3741009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:41.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.387+0000 7fa37958b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa374102d90 0x7fa3741009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53170/0 (socket says 172.21.15.120:53170) 2024-09-19T09:12:41.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.387+0000 7fa37958b640 1 -- 172.21.15.120:0/2995483569 learned_addr learned my addr 172.21.15.120:0/2995483569 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:41.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.387+0000 7fa378d8a640 1 --2- 172.21.15.120:0/2995483569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa374103740 0x7fa374100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:41.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.387+0000 7fa379d8c640 1 --2- 172.21.15.120:0/2995483569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa374108290 0x7fa374101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:41.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.388+0000 7fa37958b640 1 -- 172.21.15.120:0/2995483569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa374108290 msgr2=0x7fa374101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:41.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.388+0000 7fa37958b640 1 --2- 172.21.15.120:0/2995483569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa374108290 0x7fa374101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:41.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.388+0000 7fa37958b640 1 -- 172.21.15.120:0/2995483569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa374103740 msgr2=0x7fa374100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:41.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.388+0000 7fa37958b640 1 --2- 172.21.15.120:0/2995483569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa374103740 0x7fa374100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:41.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.388+0000 7fa37958b640 1 -- 172.21.15.120:0/2995483569 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3741a6a50 con 0x7fa374102d90 2024-09-19T09:12:41.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.388+0000 7fa378d8a640 1 --2- 172.21.15.120:0/2995483569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa374103740 0x7fa374100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:41.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.388+0000 7fa37958b640 1 --2- 172.21.15.120:0/2995483569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa374102d90 0x7fa3741009f0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fa37002a940 tx=0x7fa370003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:41.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.388+0000 7fa36a7fc640 1 -- 172.21.15.120:0/2995483569 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37003f070 con 0x7fa374102d90 2024-09-19T09:12:41.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.388+0000 7fa37b7ef640 1 -- 172.21.15.120:0/2995483569 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3741a6ce0 con 0x7fa374102d90 2024-09-19T09:12:41.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.388+0000 7fa36a7fc640 1 -- 172.21.15.120:0/2995483569 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa370031c90 con 0x7fa374102d90 2024-09-19T09:12:41.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.388+0000 7fa36a7fc640 1 -- 172.21.15.120:0/2995483569 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa370035030 con 0x7fa374102d90 2024-09-19T09:12:41.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.388+0000 7fa37b7ef640 1 -- 172.21.15.120:0/2995483569 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3741a71c0 con 0x7fa374102d90 2024-09-19T09:12:41.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.389+0000 7fa37b7ef640 1 -- 172.21.15.120:0/2995483569 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa37406a7f0 con 0x7fa374102d90 2024-09-19T09:12:41.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.390+0000 7fa36a7fc640 1 -- 172.21.15.120:0/2995483569 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa370042070 con 0x7fa374102d90 2024-09-19T09:12:41.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.391+0000 7fa36a7fc640 1 --2- 172.21.15.120:0/2995483569 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa3600a3240 0x7fa3600a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:41.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.391+0000 7fa378d8a640 1 --2- 172.21.15.120:0/2995483569 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa3600a3240 0x7fa3600a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:41.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.391+0000 7fa36a7fc640 1 -- 172.21.15.120:0/2995483569 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa3700efbc0 con 0x7fa374102d90 2024-09-19T09:12:41.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.392+0000 7fa378d8a640 1 --2- 172.21.15.120:0/2995483569 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa3600a3240 0x7fa3600a5700 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa374102480 tx=0x7fa364005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:41.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.395+0000 7fa36a7fc640 1 -- 172.21.15.120:0/2995483569 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa3700bc290 con 0x7fa374102d90 2024-09-19T09:12:41.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.633+0000 7fa37b7ef640 1 -- 172.21.15.120:0/2995483569 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa37410a2a0 con 0x7fa374102d90 2024-09-19T09:12:41.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.633+0000 7fa36a7fc640 1 -- 172.21.15.120:0/2995483569 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11988 (secure 0 0 0) 0x7fa3700dae20 con 0x7fa374102d90 2024-09-19T09:12:41.634 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:12:41.634 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":32,"btime":"2024-09-19T09:11:12:904684+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:11:12.904670+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27518":{"gid":27518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3214776406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3214776406},{"type":"v1","addr":"172.21.15.153:6835","nonce":3214776406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:12:41.634 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 32 2024-09-19T09:12:41.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.635+0000 7fa37b7ef640 1 -- 172.21.15.120:0/2995483569 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa3600a3240 msgr2=0x7fa3600a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:41.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.635+0000 7fa37b7ef640 1 --2- 172.21.15.120:0/2995483569 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa3600a3240 0x7fa3600a5700 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa374102480 tx=0x7fa364005f90 comp rx=0 tx=0).stop 2024-09-19T09:12:41.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.635+0000 7fa37b7ef640 1 -- 172.21.15.120:0/2995483569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa374102d90 msgr2=0x7fa3741009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:41.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.635+0000 7fa37b7ef640 1 --2- 172.21.15.120:0/2995483569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa374102d90 0x7fa3741009f0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fa37002a940 tx=0x7fa370003b20 comp rx=0 tx=0).stop 2024-09-19T09:12:41.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.635+0000 7fa37b7ef640 1 -- 172.21.15.120:0/2995483569 shutdown_connections 2024-09-19T09:12:41.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.635+0000 7fa37b7ef640 1 --2- 172.21.15.120:0/2995483569 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa3600a3240 0x7fa3600a5700 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:41.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.635+0000 7fa37b7ef640 1 --2- 172.21.15.120:0/2995483569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa374108290 0x7fa374101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:41.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.635+0000 7fa37b7ef640 1 --2- 172.21.15.120:0/2995483569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa374103740 0x7fa374100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:41.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.635+0000 7fa37b7ef640 1 --2- 172.21.15.120:0/2995483569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa374102d90 0x7fa3741009f0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:41.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.636+0000 7fa37b7ef640 1 -- 172.21.15.120:0/2995483569 >> 172.21.15.120:0/2995483569 conn(0x7fa3740fc7d0 msgr2=0x7fa374106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:41.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.636+0000 7fa37b7ef640 1 -- 172.21.15.120:0/2995483569 shutdown_connections 2024-09-19T09:12:41.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:41.636+0000 7fa37b7ef640 1 -- 172.21.15.120:0/2995483569 wait complete. 2024-09-19T09:12:41.848 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:12:41.848 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:12:42.067 INFO:tasks.workunit.client.0.smithi120.stdout: 1 975163 10.24 MB/sec execute 397 sec latency 64.465 ms 2024-09-19T09:12:42.073 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:41 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2995483569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:12:42.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:41 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2995483569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:12:42.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:41 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2995483569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:12:42.966 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-19T09:12:42.966 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-19T09:12:43.035 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:12:43.067 INFO:tasks.workunit.client.0.smithi120.stdout: 1 977802 10.25 MB/sec execute 398 sec latency 13.576 ms 2024-09-19T09:12:43.157 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:42 smithi120 ceph-mon[31150]: pgmap v638: 97 pgs: 97 active+clean; 29 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 3.7 MiB/s wr, 411 op/s 2024-09-19T09:12:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:42 smithi196 ceph-mon[35150]: pgmap v638: 97 pgs: 97 active+clean; 29 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 3.7 MiB/s wr, 411 op/s 2024-09-19T09:12:43.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:42 smithi153 ceph-mon[35071]: pgmap v638: 97 pgs: 97 active+clean; 29 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 3.7 MiB/s wr, 411 op/s 2024-09-19T09:12:43.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.564+0000 7fecb1256640 1 -- 172.21.15.120:0/4012771532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fecac10bfc0 msgr2=0x7fecac10e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:43.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.564+0000 7fecb1256640 1 --2- 172.21.15.120:0/4012771532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fecac10bfc0 0x7fecac10e460 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7feca4009ca0 tx=0x7feca402a950 comp rx=0 tx=0).stop 2024-09-19T09:12:43.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.564+0000 7fecb1256640 1 -- 172.21.15.120:0/4012771532 shutdown_connections 2024-09-19T09:12:43.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.564+0000 7fecb1256640 1 --2- 172.21.15.120:0/4012771532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fecac10bfc0 0x7fecac10e460 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.564+0000 7fecb1256640 1 --2- 172.21.15.120:0/4012771532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecac1034b0 0x7fecac10b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.564+0000 7fecb1256640 1 --2- 172.21.15.120:0/4012771532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecac102b90 0x7fecac102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.564+0000 7fecb1256640 1 -- 172.21.15.120:0/4012771532 >> 172.21.15.120:0/4012771532 conn(0x7fecac0fc610 msgr2=0x7fecac0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:43.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.564+0000 7fecb1256640 1 -- 172.21.15.120:0/4012771532 shutdown_connections 2024-09-19T09:12:43.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.564+0000 7fecb1256640 1 -- 172.21.15.120:0/4012771532 wait complete. 2024-09-19T09:12:43.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.564+0000 7fecb1256640 1 Processor -- start 2024-09-19T09:12:43.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.564+0000 7fecb1256640 1 -- start start 2024-09-19T09:12:43.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.565+0000 7fecb1256640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fecac102b90 0x7fecac198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:43.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.565+0000 7fecb1256640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecac1034b0 0x7fecac198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:43.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.565+0000 7fecb1256640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecac10bfc0 0x7fecac19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:43.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.565+0000 7fecb1256640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fecac111db0 con 0x7fecac102b90 2024-09-19T09:12:43.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.565+0000 7fecb1256640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fecac111c30 con 0x7fecac10bfc0 2024-09-19T09:12:43.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.565+0000 7fecb1256640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fecac111f30 con 0x7fecac1034b0 2024-09-19T09:12:43.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.565+0000 7fecabfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fecac102b90 0x7fecac198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:43.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.565+0000 7fecabfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fecac102b90 0x7fecac198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53178/0 (socket says 172.21.15.120:53178) 2024-09-19T09:12:43.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.565+0000 7fecabfff640 1 -- 172.21.15.120:0/2287287238 learned_addr learned my addr 172.21.15.120:0/2287287238 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:43.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.565+0000 7fecb0a55640 1 --2- 172.21.15.120:0/2287287238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecac10bfc0 0x7fecac19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:43.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.565+0000 7fecab7fe640 1 --2- 172.21.15.120:0/2287287238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecac1034b0 0x7fecac198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:43.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.565+0000 7fecabfff640 1 -- 172.21.15.120:0/2287287238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecac1034b0 msgr2=0x7fecac198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:43.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.565+0000 7fecabfff640 1 --2- 172.21.15.120:0/2287287238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecac1034b0 0x7fecac198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.565+0000 7fecabfff640 1 -- 172.21.15.120:0/2287287238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecac10bfc0 msgr2=0x7fecac19cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:43.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.565+0000 7fecabfff640 1 --2- 172.21.15.120:0/2287287238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecac10bfc0 0x7fecac19cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.565+0000 7fecabfff640 1 -- 172.21.15.120:0/2287287238 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecac19d1c0 con 0x7fecac102b90 2024-09-19T09:12:43.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.565+0000 7fecab7fe640 1 --2- 172.21.15.120:0/2287287238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecac1034b0 0x7fecac198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:12:43.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.565+0000 7fecabfff640 1 --2- 172.21.15.120:0/2287287238 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fecac102b90 0x7fecac198210 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00e9f0 tx=0x7fec9c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:43.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.565+0000 7feca97fa640 1 -- 172.21.15.120:0/2287287238 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c00cd80 con 0x7fecac102b90 2024-09-19T09:12:43.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.566+0000 7fecb1256640 1 -- 172.21.15.120:0/2287287238 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecac19d4b0 con 0x7fecac102b90 2024-09-19T09:12:43.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.566+0000 7feca97fa640 1 -- 172.21.15.120:0/2287287238 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec9c010910 con 0x7fecac102b90 2024-09-19T09:12:43.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.566+0000 7feca97fa640 1 -- 172.21.15.120:0/2287287238 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c0188f0 con 0x7fecac102b90 2024-09-19T09:12:43.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.566+0000 7fecb1256640 1 -- 172.21.15.120:0/2287287238 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecac1a4d90 con 0x7fecac102b90 2024-09-19T09:12:43.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.568+0000 7feca97fa640 1 -- 172.21.15.120:0/2287287238 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fec9c010ac0 con 0x7fecac102b90 2024-09-19T09:12:43.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.568+0000 7fecb4891740 1 Processor -- start 2024-09-19T09:12:43.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.568+0000 7fecb4891740 1 -- start start 2024-09-19T09:12:43.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.569+0000 7fecb4891740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55674774cd70 0x55674774d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:43.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.569+0000 7fecb4891740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55674774d690 0x556747757a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:43.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.569+0000 7fecb4891740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556747757f50 0x55674775a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:43.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.569+0000 7fecb4891740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5567476b2280 con 0x55674774d690 2024-09-19T09:12:43.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.569+0000 7fecb4891740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556747659b10 con 0x556747757f50 2024-09-19T09:12:43.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.569+0000 7fecb4891740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5567476ae240 con 0x55674774cd70 2024-09-19T09:12:43.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.569+0000 7feca97fa640 1 --2- 172.21.15.120:0/2287287238 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fec940a3240 0x7fec940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:43.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.569+0000 7fecabfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55674774d690 0x556747757a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:43.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.569+0000 7fecabfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55674774d690 0x556747757a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53194/0 (socket says 172.21.15.120:53194) 2024-09-19T09:12:43.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.569+0000 7fecabfff640 1 -- 172.21.15.120:0/3797883229 learned_addr learned my addr 172.21.15.120:0/3797883229 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:43.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.569+0000 7fecb0a55640 1 --2- 172.21.15.120:0/3797883229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55674774cd70 0x55674774d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:43.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.569+0000 7fecab7fe640 1 --2- 172.21.15.120:0/3797883229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556747757f50 0x55674775a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:43.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.569+0000 7fecb0a55640 1 --2- 172.21.15.120:0/2287287238 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fec940a3240 0x7fec940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:43.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.569+0000 7feca97fa640 1 -- 172.21.15.120:0/2287287238 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fec9c0cfd90 con 0x7fecac102b90 2024-09-19T09:12:43.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.569+0000 7fecab7fe640 1 -- 172.21.15.120:0/3797883229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55674774cd70 msgr2=0x55674774d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:43.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.569+0000 7fecab7fe640 1 --2- 172.21.15.120:0/3797883229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55674774cd70 0x55674774d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.569+0000 7fecab7fe640 1 -- 172.21.15.120:0/3797883229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55674774d690 msgr2=0x556747757a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:43.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.569+0000 7fecab7fe640 1 --2- 172.21.15.120:0/3797883229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55674774d690 0x556747757a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.569+0000 7fecab7fe640 1 -- 172.21.15.120:0/3797883229 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567476661f0 con 0x556747757f50 2024-09-19T09:12:43.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.569+0000 7fecabfff640 1 --2- 172.21.15.120:0/3797883229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55674774d690 0x556747757a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:43.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.569+0000 7fecab7fe640 1 --2- 172.21.15.120:0/3797883229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556747757f50 0x55674775a340 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7feca0009b80 tx=0x7feca0010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ca7e4d7d2a35f8b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:43.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.569+0000 7fecb0a55640 1 --2- 172.21.15.120:0/2287287238 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fec940a3240 0x7fec940a5700 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7feca4009430 tx=0x7feca4005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:43.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.570+0000 7fec927fc640 1 -- 172.21.15.120:0/3797883229 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca0004030 con 0x556747757f50 2024-09-19T09:12:43.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.570+0000 7fec927fc640 1 -- 172.21.15.120:0/3797883229 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feca00041d0 con 0x556747757f50 2024-09-19T09:12:43.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.570+0000 7fec927fc640 1 -- 172.21.15.120:0/3797883229 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca00126b0 con 0x556747757f50 2024-09-19T09:12:43.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.570+0000 7fecb4891740 1 -- 172.21.15.120:0/3797883229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556747757f50 msgr2=0x55674775a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:43.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.570+0000 7fecb4891740 1 --2- 172.21.15.120:0/3797883229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556747757f50 0x55674775a340 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7feca0009b80 tx=0x7feca0010740 comp rx=0 tx=0).stop 2024-09-19T09:12:43.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.570+0000 7fecb4891740 1 -- 172.21.15.120:0/3797883229 shutdown_connections 2024-09-19T09:12:43.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.570+0000 7fecb4891740 1 --2- 172.21.15.120:0/3797883229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556747757f50 0x55674775a340 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.570+0000 7fecb4891740 1 --2- 172.21.15.120:0/3797883229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55674774d690 0x556747757a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.570+0000 7fecb4891740 1 --2- 172.21.15.120:0/3797883229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55674774cd70 0x55674774d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.570+0000 7fecb4891740 1 -- 172.21.15.120:0/3797883229 >> 172.21.15.120:0/3797883229 conn(0x556747749300 msgr2=0x5567477496e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:43.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.570+0000 7fecb4891740 1 -- 172.21.15.120:0/3797883229 shutdown_connections 2024-09-19T09:12:43.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.570+0000 7fecb4891740 1 -- 172.21.15.120:0/3797883229 wait complete. 2024-09-19T09:12:43.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.571+0000 7fecb4891740 1 -- start start 2024-09-19T09:12:43.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.571+0000 7fecb4891740 1 Processor -- start 2024-09-19T09:12:43.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.571+0000 7fecb4891740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55674774cd70 0x556747757a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:43.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.571+0000 7fecb4891740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55674774d690 0x556747754d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:43.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.571+0000 7fecb4891740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556747757f50 0x5567477554f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:43.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.571+0000 7fecb4891740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55674775ab20 con 0x55674774cd70 2024-09-19T09:12:43.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.571+0000 7fecb4891740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55674766cf10 con 0x55674774d690 2024-09-19T09:12:43.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.571+0000 7fecb4891740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55674775aca0 con 0x556747757f50 2024-09-19T09:12:43.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.571+0000 7fecb4891740 10 client.0 _ll_register_callbacks cb 0x556747635a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:43.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.571+0000 7fecabfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55674774cd70 0x556747757a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:43.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.571+0000 7fecabfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55674774cd70 0x556747757a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53204/0 (socket says 172.21.15.120:53204) 2024-09-19T09:12:43.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.571+0000 7fecabfff640 1 -- 172.21.15.120:0/3523681782 learned_addr learned my addr 172.21.15.120:0/3523681782 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:43.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.571+0000 7fecab7fe640 1 --2- 172.21.15.120:0/3523681782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55674774d690 0x556747754d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:43.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.572+0000 7fecb0a55640 1 --2- 172.21.15.120:0/3523681782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556747757f50 0x5567477554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:43.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.572+0000 7fecabfff640 1 -- 172.21.15.120:0/3523681782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556747757f50 msgr2=0x5567477554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:43.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.572+0000 7fecabfff640 1 --2- 172.21.15.120:0/3523681782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556747757f50 0x5567477554f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.572+0000 7fecabfff640 1 -- 172.21.15.120:0/3523681782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55674774d690 msgr2=0x556747754d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:43.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.572+0000 7fecabfff640 1 --2- 172.21.15.120:0/3523681782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55674774d690 0x556747754d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.572+0000 7fecabfff640 1 -- 172.21.15.120:0/3523681782 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567474236a0 con 0x55674774cd70 2024-09-19T09:12:43.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.572+0000 7fecabfff640 1 --2- 172.21.15.120:0/3523681782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55674774cd70 0x556747757a10 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00c6a0 tx=0x7fec9c0a4fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a02e800f18fb62d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:43.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.572+0000 7fec8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:43.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.572+0000 7fec8b7fe640 1 -- 172.21.15.120:0/3523681782 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c09b7c0 con 0x55674774cd70 2024-09-19T09:12:43.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.572+0000 7fec8b7fe640 1 -- 172.21.15.120:0/3523681782 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fec9c09b960 con 0x55674774cd70 2024-09-19T09:12:43.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.572+0000 7fecb4891740 1 -- 172.21.15.120:0/3523681782 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feca00041d0 con 0x55674774cd70 2024-09-19T09:12:43.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.572+0000 7fec8b7fe640 1 -- 172.21.15.120:0/3523681782 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fec9c0b5140 con 0x55674774cd70 2024-09-19T09:12:43.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.572+0000 7fecb4891740 10 client.19023 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:43.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.572+0000 7fecb4891740 1 -- 172.21.15.120:0/3523681782 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5567477666a0 con 0x55674774cd70 2024-09-19T09:12:43.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.572+0000 7fec8b7fe640 1 -- 172.21.15.120:0/3523681782 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feca00041d0 con 0x55674774cd70 2024-09-19T09:12:43.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.572+0000 7fec8b7fe640 1 -- 172.21.15.120:0/3523681782 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fec9c0bc070 con 0x55674774cd70 2024-09-19T09:12:43.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.572+0000 7fecb4891740 10 client.19023 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:43.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.572+0000 7fecb4891740 10 client.19023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:12:43.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.572+0000 7fecb4891740 20 client.19023 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:43.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.573+0000 7fecb4891740 1 --2- 172.21.15.120:0/3523681782 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556747767ab0 0x556747769ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:43.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.573+0000 7fecb4891740 4 client.19023 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:12:43.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.573+0000 7fecb4891740 1 -- 172.21.15.120:0/3523681782 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567476bf7f0 con 0x556747767ab0 2024-09-19T09:12:43.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.573+0000 7fecab7fe640 1 --2- 172.21.15.120:0/3523681782 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556747767ab0 0x556747769ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:43.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.573+0000 7fec8b7fe640 10 client.19023 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:43.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.573+0000 7fecab7fe640 1 --2- 172.21.15.120:0/3523681782 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556747767ab0 0x556747769ea0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be14e41222c23504 server_cookie=5a35cb459a92f75e in_seq=0 out_seq=0 2024-09-19T09:12:43.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.574+0000 7fec8b7fe640 1 -- 172.21.15.120:0/3523681782 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5567476bf7f0 con 0x556747767ab0 2024-09-19T09:12:43.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.574+0000 7fec8b7fe640 10 client.19023 handle_command_reply: tid=0 2024-09-19T09:12:43.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.574+0000 7fecb4891740 1 client.19023 shutdown 2024-09-19T09:12:43.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.574+0000 7fecb4891740 1 -- 172.21.15.120:0/3523681782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55674774cd70 msgr2=0x556747757a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:43.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.574+0000 7fecb4891740 1 --2- 172.21.15.120:0/3523681782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55674774cd70 0x556747757a10 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00c6a0 tx=0x7fec9c0a4fe0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.574+0000 7fecb4891740 1 -- 172.21.15.120:0/3523681782 shutdown_connections 2024-09-19T09:12:43.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.574+0000 7fecb4891740 1 --2- 172.21.15.120:0/3523681782 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556747767ab0 0x556747769ea0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.574+0000 7fecb4891740 1 --2- 172.21.15.120:0/3523681782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556747757f50 0x5567477554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.574+0000 7fec8b7fe640 0 client.19023 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:43.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.574+0000 7fecb4891740 1 --2- 172.21.15.120:0/3523681782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55674774d690 0x556747754d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.574+0000 7fecb4891740 1 --2- 172.21.15.120:0/3523681782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55674774cd70 0x556747757a10 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.574+0000 7fecb4891740 1 -- 172.21.15.120:0/3523681782 >> 172.21.15.120:0/3523681782 conn(0x556747749300 msgr2=0x55674774f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:43.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.574+0000 7fecb4891740 1 -- 172.21.15.120:0/3523681782 shutdown_connections 2024-09-19T09:12:43.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.574+0000 7fecb4891740 1 -- 172.21.15.120:0/3523681782 wait complete. 2024-09-19T09:12:43.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.575+0000 7fecb4891740 20 client.19023 trim_cache size 0 max 16384 2024-09-19T09:12:43.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.592+0000 7fecb4891740 1 Processor -- start 2024-09-19T09:12:43.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.592+0000 7fecb4891740 1 -- start start 2024-09-19T09:12:43.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.592+0000 7fecb4891740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55674774cd70 0x55674776f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:43.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.592+0000 7fecb4891740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55674774d690 0x55674776fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:43.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.592+0000 7fecb4891740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556747757f50 0x556747751fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:43.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.592+0000 7fecb4891740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fecac1ad330 con 0x55674774d690 2024-09-19T09:12:43.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.592+0000 7fecb4891740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fecac1aee70 con 0x556747757f50 2024-09-19T09:12:43.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.592+0000 7fecb4891740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fecac1ada00 con 0x55674774cd70 2024-09-19T09:12:43.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.592+0000 7fecabfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55674774cd70 0x55674776f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:43.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.592+0000 7fecab7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55674774d690 0x55674776fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:43.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.592+0000 7fecabfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55674774cd70 0x55674776f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:33552/0 (socket says 172.21.15.120:33552) 2024-09-19T09:12:43.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.592+0000 7fecab7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55674774d690 0x55674776fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53210/0 (socket says 172.21.15.120:53210) 2024-09-19T09:12:43.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.592+0000 7fecabfff640 1 -- 172.21.15.120:0/2005884734 learned_addr learned my addr 172.21.15.120:0/2005884734 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:43.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.592+0000 7fecb0a55640 1 --2- 172.21.15.120:0/2005884734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556747757f50 0x556747751fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:43.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.592+0000 7fecabfff640 1 -- 172.21.15.120:0/2005884734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556747757f50 msgr2=0x556747751fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:43.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.592+0000 7fecabfff640 1 --2- 172.21.15.120:0/2005884734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556747757f50 0x556747751fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.592+0000 7fecabfff640 1 -- 172.21.15.120:0/2005884734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55674774d690 msgr2=0x55674776fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:43.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.592+0000 7fecabfff640 1 --2- 172.21.15.120:0/2005884734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55674774d690 0x55674776fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.592+0000 7fecabfff640 1 -- 172.21.15.120:0/2005884734 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556747757d80 con 0x55674774cd70 2024-09-19T09:12:43.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.592+0000 7fecab7fe640 1 --2- 172.21.15.120:0/2005884734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55674774d690 0x55674776fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:43.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.592+0000 7fecabfff640 1 --2- 172.21.15.120:0/2005884734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55674774cd70 0x55674776f6d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fec9c09c300 tx=0x7fec9c01f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0609b926b8cb88d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:43.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.593+0000 7fec91ffb640 1 -- 172.21.15.120:0/2005884734 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c0bc0b0 con 0x55674774cd70 2024-09-19T09:12:43.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.593+0000 7fec91ffb640 1 -- 172.21.15.120:0/2005884734 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec9c0d1070 con 0x55674774cd70 2024-09-19T09:12:43.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.593+0000 7fec91ffb640 1 -- 172.21.15.120:0/2005884734 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c098320 con 0x55674774cd70 2024-09-19T09:12:43.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.593+0000 7fecb4891740 1 -- 172.21.15.120:0/2005884734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55674774cd70 msgr2=0x55674776f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:43.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.593+0000 7fecb4891740 1 --2- 172.21.15.120:0/2005884734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55674774cd70 0x55674776f6d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fec9c09c300 tx=0x7fec9c01f000 comp rx=0 tx=0).stop 2024-09-19T09:12:43.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.593+0000 7fecb4891740 1 -- 172.21.15.120:0/2005884734 shutdown_connections 2024-09-19T09:12:43.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.593+0000 7fecb4891740 1 --2- 172.21.15.120:0/2005884734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556747757f50 0x556747751fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.593+0000 7fecb4891740 1 --2- 172.21.15.120:0/2005884734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55674774d690 0x55674776fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.593+0000 7fecb4891740 1 --2- 172.21.15.120:0/2005884734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55674774cd70 0x55674776f6d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.593+0000 7fecb4891740 1 -- 172.21.15.120:0/2005884734 >> 172.21.15.120:0/2005884734 conn(0x556747749300 msgr2=0x556747753630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:43.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.593+0000 7fecb4891740 1 -- 172.21.15.120:0/2005884734 shutdown_connections 2024-09-19T09:12:43.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.594+0000 7fecb4891740 1 -- 172.21.15.120:0/2005884734 wait complete. 2024-09-19T09:12:43.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.594+0000 7fecb4891740 1 -- start start 2024-09-19T09:12:43.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.594+0000 7fecb4891740 1 Processor -- start 2024-09-19T09:12:43.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.594+0000 7fecb4891740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55674774cd70 0x5567477600e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:43.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.594+0000 7fecb4891740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55674774d690 0x556747760620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:43.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.594+0000 7fecb4891740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556747757f50 0x55674775aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:43.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.594+0000 7fecb4891740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fecac1ae7a0 con 0x556747757f50 2024-09-19T09:12:43.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.594+0000 7fecb4891740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fecac1ae0d0 con 0x55674774d690 2024-09-19T09:12:43.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.594+0000 7fecb4891740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fecac1ab2f0 con 0x55674774cd70 2024-09-19T09:12:43.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.594+0000 7fecb4891740 10 client.0 _ll_register_callbacks cb 0x7fec9c00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:43.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.595+0000 7fecb0a55640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556747757f50 0x55674775aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:43.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.595+0000 7fecabfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55674774cd70 0x5567477600e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:43.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.595+0000 7fecab7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55674774d690 0x556747760620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:43.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.595+0000 7fecabfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55674774cd70 0x5567477600e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:33554/0 (socket says 172.21.15.120:33554) 2024-09-19T09:12:43.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.595+0000 7fecb0a55640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556747757f50 0x55674775aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53214/0 (socket says 172.21.15.120:53214) 2024-09-19T09:12:43.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.595+0000 7fecabfff640 1 -- 172.21.15.120:0/2596217277 learned_addr learned my addr 172.21.15.120:0/2596217277 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:43.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.595+0000 7fecab7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55674774d690 0x556747760620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:45670/0 (socket says 172.21.15.120:45670) 2024-09-19T09:12:43.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.595+0000 7fecabfff640 1 -- 172.21.15.120:0/2596217277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55674774d690 msgr2=0x556747760620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:43.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.595+0000 7fecabfff640 1 --2- 172.21.15.120:0/2596217277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55674774d690 0x556747760620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.595+0000 7fecabfff640 1 -- 172.21.15.120:0/2596217277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556747757f50 msgr2=0x55674775aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:43.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.595+0000 7fecabfff640 1 --2- 172.21.15.120:0/2596217277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556747757f50 0x55674775aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.595+0000 7fecabfff640 1 -- 172.21.15.120:0/2596217277 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55674775b6c0 con 0x55674774cd70 2024-09-19T09:12:43.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.595+0000 7fecab7fe640 1 --2- 172.21.15.120:0/2596217277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55674774d690 0x556747760620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:43.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.595+0000 7fecabfff640 1 --2- 172.21.15.120:0/2596217277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55674774cd70 0x5567477600e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00e9c0 tx=0x7fec9c0cd8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2b002c71851e72 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:43.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.595+0000 7fec8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:12:43.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.595+0000 7fec8b7fe640 1 -- 172.21.15.120:0/2596217277 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c09b5a0 con 0x55674774cd70 2024-09-19T09:12:43.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.595+0000 7fecb4891740 1 -- 172.21.15.120:0/2596217277 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec9c0d1070 con 0x55674774cd70 2024-09-19T09:12:43.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.596+0000 7fec8b7fe640 1 -- 172.21.15.120:0/2596217277 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fec9c010430 con 0x55674774cd70 2024-09-19T09:12:43.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.596+0000 7fec8b7fe640 1 -- 172.21.15.120:0/2596217277 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fec9c010630 con 0x55674774cd70 2024-09-19T09:12:43.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.596+0000 7fecb4891740 10 client.27967 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:43.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.596+0000 7fecb4891740 1 -- 172.21.15.120:0/2596217277 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5567477d31e0 con 0x55674774cd70 2024-09-19T09:12:43.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.596+0000 7fec8b7fe640 1 -- 172.21.15.120:0/2596217277 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec9c0d1070 con 0x55674774cd70 2024-09-19T09:12:43.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.596+0000 7fec8b7fe640 1 -- 172.21.15.120:0/2596217277 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fec9c0a18b0 con 0x55674774cd70 2024-09-19T09:12:43.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.596+0000 7fecb4891740 10 client.27967 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:43.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.596+0000 7fecb4891740 10 client.27967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:12:43.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.596+0000 7fecb4891740 20 client.27967 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:43.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.596+0000 7fecb4891740 1 --2- 172.21.15.120:0/2596217277 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5567477d3660 0x5567477d5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:43.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.596+0000 7fecb4891740 4 client.27967 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:12:43.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.596+0000 7fecb4891740 1 -- 172.21.15.120:0/2596217277 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5567476b9d00 con 0x5567477d3660 2024-09-19T09:12:43.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.596+0000 7fecab7fe640 1 --2- 172.21.15.120:0/2596217277 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5567477d3660 0x5567477d5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:43.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.597+0000 7fec8b7fe640 10 client.27967 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:43.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.597+0000 7fecab7fe640 1 --2- 172.21.15.120:0/2596217277 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5567477d3660 0x5567477d5a50 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f69654e2b4e881b1 server_cookie=70cf7b34c8077848 in_seq=0 out_seq=0 2024-09-19T09:12:43.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.598+0000 7fec8b7fe640 1 -- 172.21.15.120:0/2596217277 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5567476b9d00 con 0x5567477d3660 2024-09-19T09:12:43.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.598+0000 7fec8b7fe640 10 client.27967 handle_command_reply: tid=0 2024-09-19T09:12:43.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.598+0000 7fecb4891740 1 client.27967 shutdown 2024-09-19T09:12:43.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.598+0000 7fecb4891740 1 -- 172.21.15.120:0/2596217277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55674774cd70 msgr2=0x5567477600e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:43.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.598+0000 7fecb4891740 1 --2- 172.21.15.120:0/2596217277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55674774cd70 0x5567477600e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00e9c0 tx=0x7fec9c0cd8d0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.598+0000 7fecb4891740 1 -- 172.21.15.120:0/2596217277 shutdown_connections 2024-09-19T09:12:43.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.598+0000 7fecb4891740 1 --2- 172.21.15.120:0/2596217277 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5567477d3660 0x5567477d5a50 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.598+0000 7fecb4891740 1 --2- 172.21.15.120:0/2596217277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556747757f50 0x55674775aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.598+0000 7fecb4891740 1 --2- 172.21.15.120:0/2596217277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55674774d690 0x556747760620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.598+0000 7fecb4891740 1 --2- 172.21.15.120:0/2596217277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55674774cd70 0x5567477600e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.598+0000 7fecb4891740 1 -- 172.21.15.120:0/2596217277 >> 172.21.15.120:0/2596217277 conn(0x556747749300 msgr2=0x55674774e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:43.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.598+0000 7fec8b7fe640 0 client.27967 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:43.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.598+0000 7fecb4891740 1 -- 172.21.15.120:0/2596217277 shutdown_connections 2024-09-19T09:12:43.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.598+0000 7fecb4891740 1 -- 172.21.15.120:0/2596217277 wait complete. 2024-09-19T09:12:43.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.598+0000 7fecb4891740 20 client.27967 trim_cache size 0 max 16384 2024-09-19T09:12:43.625 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:12:43.625 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:12:43.625 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "f8ec7f9f-6aef-4c34-85ef-1ee9f90ed97e", 2024-09-19T09:12:43.625 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:12:43.626 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:12:43.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.599+0000 7fecb1256640 1 -- 172.21.15.120:0/2287287238 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fec940a3240 msgr2=0x7fec940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:43.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.599+0000 7fecb1256640 1 --2- 172.21.15.120:0/2287287238 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fec940a3240 0x7fec940a5700 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7feca4009430 tx=0x7feca4005c50 comp rx=0 tx=0).stop 2024-09-19T09:12:43.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.599+0000 7fecb1256640 1 -- 172.21.15.120:0/2287287238 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fecac102b90 msgr2=0x7fecac198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:43.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.599+0000 7fecb1256640 1 --2- 172.21.15.120:0/2287287238 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fecac102b90 0x7fecac198210 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00e9f0 tx=0x7fec9c00eeb0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.599+0000 7fecb1256640 1 -- 172.21.15.120:0/2287287238 shutdown_connections 2024-09-19T09:12:43.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.599+0000 7fecb1256640 1 --2- 172.21.15.120:0/2287287238 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fec940a3240 0x7fec940a5700 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.599+0000 7fecb1256640 1 --2- 172.21.15.120:0/2287287238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecac10bfc0 0x7fecac19cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.599+0000 7fecb1256640 1 --2- 172.21.15.120:0/2287287238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecac1034b0 0x7fecac198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.599+0000 7fecb1256640 1 --2- 172.21.15.120:0/2287287238 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fecac102b90 0x7fecac198210 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:43.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.599+0000 7fecb1256640 1 -- 172.21.15.120:0/2287287238 >> 172.21.15.120:0/2287287238 conn(0x7fecac0fc610 msgr2=0x7fecac0fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:43.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.599+0000 7fecb1256640 1 -- 172.21.15.120:0/2287287238 shutdown_connections 2024-09-19T09:12:43.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:43.599+0000 7fecb1256640 1 -- 172.21.15.120:0/2287287238 wait complete. 2024-09-19T09:12:44.067 INFO:tasks.workunit.client.0.smithi120.stdout: 1 980946 10.27 MB/sec execute 399 sec latency 9.900 ms 2024-09-19T09:12:44.509 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:12:44.579 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:12:45.000 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:44 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:12:45.000 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:44 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:12:45.000 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:44 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:12:45.000 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:44 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:12:45.000 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:44 smithi120 ceph-mon[31150]: pgmap v639: 97 pgs: 97 active+clean; 30 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 3.8 MiB/s wr, 431 op/s 2024-09-19T09:12:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.008+0000 7f9f7f59e640 1 -- 172.21.15.120:0/2715794959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f80107620 msgr2=0x7f9f80109a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.008+0000 7f9f7f59e640 1 --2- 172.21.15.120:0/2715794959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f80107620 0x7f9f80109a90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009be0 tx=0x7f9f7802a6c0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.008+0000 7f9f7f59e640 1 -- 172.21.15.120:0/2715794959 shutdown_connections 2024-09-19T09:12:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.008+0000 7f9f7f59e640 1 --2- 172.21.15.120:0/2715794959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f80107620 0x7f9f80109a90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.008+0000 7f9f7f59e640 1 --2- 172.21.15.120:0/2715794959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f80104cf0 0x7f9f801070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.008+0000 7f9f7f59e640 1 --2- 172.21.15.120:0/2715794959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f801023c0 0x7f9f801047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.008+0000 7f9f7f59e640 1 -- 172.21.15.120:0/2715794959 >> 172.21.15.120:0/2715794959 conn(0x7f9f800fbfa0 msgr2=0x7f9f800fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.008+0000 7f9f7f59e640 1 -- 172.21.15.120:0/2715794959 shutdown_connections 2024-09-19T09:12:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.008+0000 7f9f7f59e640 1 -- 172.21.15.120:0/2715794959 wait complete. 2024-09-19T09:12:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.008+0000 7f9f7f59e640 1 Processor -- start 2024-09-19T09:12:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.008+0000 7f9f7f59e640 1 -- start start 2024-09-19T09:12:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.009+0000 7f9f7f59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f801023c0 0x7f9f80197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.009+0000 7f9f7f59e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f80104cf0 0x7f9f80198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.009+0000 7f9f7f59e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f80107620 0x7f9f8019f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.009+0000 7f9f7f59e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8010d590 con 0x7f9f801023c0 2024-09-19T09:12:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.009+0000 7f9f7f59e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8010d410 con 0x7f9f80104cf0 2024-09-19T09:12:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.009+0000 7f9f7f59e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8010d710 con 0x7f9f80107620 2024-09-19T09:12:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.009+0000 7f9f7e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f801023c0 0x7f9f80197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.009+0000 7f9f7e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f801023c0 0x7f9f80197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53228/0 (socket says 172.21.15.120:53228) 2024-09-19T09:12:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.009+0000 7f9f7e59c640 1 -- 172.21.15.120:0/3075985884 learned_addr learned my addr 172.21.15.120:0/3075985884 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.009+0000 7f9f77fff640 1 --2- 172.21.15.120:0/3075985884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f80104cf0 0x7f9f80198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.009+0000 7f9f7ed9d640 1 --2- 172.21.15.120:0/3075985884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f80107620 0x7f9f8019f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.009+0000 7f9f7e59c640 1 -- 172.21.15.120:0/3075985884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f80107620 msgr2=0x7f9f8019f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.009+0000 7f9f7e59c640 1 --2- 172.21.15.120:0/3075985884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f80107620 0x7f9f8019f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.009+0000 7f9f7e59c640 1 -- 172.21.15.120:0/3075985884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f80104cf0 msgr2=0x7f9f80198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.009+0000 7f9f7e59c640 1 --2- 172.21.15.120:0/3075985884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f80104cf0 0x7f9f80198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.009+0000 7f9f7e59c640 1 -- 172.21.15.120:0/3075985884 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8019f900 con 0x7f9f801023c0 2024-09-19T09:12:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.009+0000 7f9f7e59c640 1 --2- 172.21.15.120:0/3075985884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f801023c0 0x7f9f80197cf0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000e9c0 tx=0x7f9f7000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.009+0000 7f9f777fe640 1 -- 172.21.15.120:0/3075985884 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7000cde0 con 0x7f9f801023c0 2024-09-19T09:12:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.009+0000 7f9f7f59e640 1 -- 172.21.15.120:0/3075985884 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f8019fbf0 con 0x7f9f801023c0 2024-09-19T09:12:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.009+0000 7f9f777fe640 1 -- 172.21.15.120:0/3075985884 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f700035f0 con 0x7f9f801023c0 2024-09-19T09:12:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.009+0000 7f9f777fe640 1 -- 172.21.15.120:0/3075985884 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f70010650 con 0x7f9f801023c0 2024-09-19T09:12:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.010+0000 7f9f7f59e640 1 -- 172.21.15.120:0/3075985884 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f801a0190 con 0x7f9f801023c0 2024-09-19T09:12:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.012+0000 7f9f777fe640 1 -- 172.21.15.120:0/3075985884 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f9f700037a0 con 0x7f9f801023c0 2024-09-19T09:12:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.013+0000 7f9f777fe640 1 --2- 172.21.15.120:0/3075985884 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9f6c0a3240 0x7f9f6c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.013+0000 7f9f77fff640 1 --2- 172.21.15.120:0/3075985884 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9f6c0a3240 0x7f9f6c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.013+0000 7f9f777fe640 1 -- 172.21.15.120:0/3075985884 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f9f700cfd00 con 0x7f9f801023c0 2024-09-19T09:12:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.013+0000 7f9f87679740 1 Processor -- start 2024-09-19T09:12:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.013+0000 7f9f77fff640 1 --2- 172.21.15.120:0/3075985884 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9f6c0a3240 0x7f9f6c0a5700 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9f680095f0 tx=0x7f9f68005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.013+0000 7f9f87679740 1 -- start start 2024-09-19T09:12:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.014+0000 7f9f87679740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600199a2b70 0x5600199a2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.014+0000 7f9f87679740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600199a34b0 0x5600199a7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.014+0000 7f9f87679740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600199a8540 0x5600199aaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.014+0000 7f9f87679740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5600199062f0 con 0x5600199a2b70 2024-09-19T09:12:45.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.014+0000 7f9f87679740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5600198ae270 con 0x5600199a34b0 2024-09-19T09:12:45.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.014+0000 7f9f87679740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5600199022b0 con 0x5600199a8540 2024-09-19T09:12:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.014+0000 7f9f7e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600199a2b70 0x5600199a2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.014+0000 7f9f7e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600199a2b70 0x5600199a2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53244/0 (socket says 172.21.15.120:53244) 2024-09-19T09:12:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.014+0000 7f9f77fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600199a34b0 0x5600199a7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.014+0000 7f9f7e59c640 1 -- 172.21.15.120:0/968217506 learned_addr learned my addr 172.21.15.120:0/968217506 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.014+0000 7f9f7ed9d640 1 --2- 172.21.15.120:0/968217506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600199a8540 0x5600199aaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.014+0000 7f9f7ed9d640 1 -- 172.21.15.120:0/968217506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600199a34b0 msgr2=0x5600199a7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.014+0000 7f9f7ed9d640 1 --2- 172.21.15.120:0/968217506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600199a34b0 0x5600199a7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.014+0000 7f9f7ed9d640 1 -- 172.21.15.120:0/968217506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600199a2b70 msgr2=0x5600199a2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.014+0000 7f9f7ed9d640 1 --2- 172.21.15.120:0/968217506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600199a2b70 0x5600199a2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.014+0000 7f9f7ed9d640 1 -- 172.21.15.120:0/968217506 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600198ad340 con 0x5600199a8540 2024-09-19T09:12:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.014+0000 7f9f7e59c640 1 --2- 172.21.15.120:0/968217506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600199a2b70 0x5600199a2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.014+0000 7f9f7ed9d640 1 --2- 172.21.15.120:0/968217506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600199a8540 0x5600199aaa00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9f78034040 tx=0x7f9f78005ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=305a44beab460f76 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.014+0000 7f9f74ff9640 1 -- 172.21.15.120:0/968217506 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78042030 con 0x5600199a8540 2024-09-19T09:12:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.014+0000 7f9f74ff9640 1 -- 172.21.15.120:0/968217506 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f78037040 con 0x5600199a8540 2024-09-19T09:12:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.015+0000 7f9f74ff9640 1 -- 172.21.15.120:0/968217506 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78036070 con 0x5600199a8540 2024-09-19T09:12:45.022 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.015+0000 7f9f87679740 1 -- 172.21.15.120:0/968217506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600199a8540 msgr2=0x5600199aaa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.015+0000 7f9f87679740 1 --2- 172.21.15.120:0/968217506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600199a8540 0x5600199aaa00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9f78034040 tx=0x7f9f78005ec0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.016+0000 7f9f87679740 1 -- 172.21.15.120:0/968217506 shutdown_connections 2024-09-19T09:12:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.016+0000 7f9f87679740 1 --2- 172.21.15.120:0/968217506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600199a8540 0x5600199aaa00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.016+0000 7f9f87679740 1 --2- 172.21.15.120:0/968217506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600199a34b0 0x5600199a7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.016+0000 7f9f87679740 1 --2- 172.21.15.120:0/968217506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600199a2b70 0x5600199a2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.016+0000 7f9f87679740 1 -- 172.21.15.120:0/968217506 >> 172.21.15.120:0/968217506 conn(0x56001999e1e0 msgr2=0x56001999e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.016+0000 7f9f87679740 1 -- 172.21.15.120:0/968217506 shutdown_connections 2024-09-19T09:12:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.016+0000 7f9f87679740 1 -- 172.21.15.120:0/968217506 wait complete. 2024-09-19T09:12:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.016+0000 7f9f87679740 1 -- start start 2024-09-19T09:12:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.017+0000 7f9f87679740 1 Processor -- start 2024-09-19T09:12:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.017+0000 7f9f87679740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600199a2b70 0x5600199acd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.017+0000 7f9f87679740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600199a34b0 0x5600199ad240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.017+0000 7f9f87679740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600199a8540 0x5600199b4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.017+0000 7f9f7e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600199a2b70 0x5600199acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.017+0000 7f9f7e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600199a2b70 0x5600199acd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:45700/0 (socket says 172.21.15.120:45700) 2024-09-19T09:12:45.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.017+0000 7f9f77fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600199a34b0 0x5600199ad240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.017+0000 7f9f77fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600199a34b0 0x5600199ad240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53250/0 (socket says 172.21.15.120:53250) 2024-09-19T09:12:45.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.017+0000 7f9f7e59c640 1 -- 172.21.15.120:0/476492674 learned_addr learned my addr 172.21.15.120:0/476492674 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:45.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.017+0000 7f9f87679740 1 -- 172.21.15.120:0/476492674 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5600199ab4b0 con 0x5600199a34b0 2024-09-19T09:12:45.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.017+0000 7f9f87679740 1 -- 172.21.15.120:0/476492674 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56001988f290 con 0x5600199a2b70 2024-09-19T09:12:45.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.018+0000 7f9f7ed9d640 1 --2- 172.21.15.120:0/476492674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600199a8540 0x5600199b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.018+0000 7f9f87679740 1 -- 172.21.15.120:0/476492674 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5600199ab630 con 0x5600199a8540 2024-09-19T09:12:45.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.018+0000 7f9f87679740 10 client.0 _ll_register_callbacks cb 0x5600198c0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:45.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.018+0000 7f9f7e59c640 1 -- 172.21.15.120:0/476492674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600199a8540 msgr2=0x5600199b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:45.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.018+0000 7f9f7e59c640 1 --2- 172.21.15.120:0/476492674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600199a8540 0x5600199b4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.018+0000 7f9f7e59c640 1 -- 172.21.15.120:0/476492674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600199a34b0 msgr2=0x5600199ad240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.018+0000 7f9f7e59c640 1 --2- 172.21.15.120:0/476492674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600199a34b0 0x5600199ad240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.018+0000 7f9f7e59c640 1 -- 172.21.15.120:0/476492674 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560019619210 con 0x5600199a2b70 2024-09-19T09:12:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.018+0000 7f9f7ed9d640 1 --2- 172.21.15.120:0/476492674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600199a8540 0x5600199b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.018+0000 7f9f77fff640 1 --2- 172.21.15.120:0/476492674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600199a34b0 0x5600199ad240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.018+0000 7f9f7e59c640 1 --2- 172.21.15.120:0/476492674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600199a2b70 0x5600199acd00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9f70098c20 tx=0x7f9f7009a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a28766f644d65903 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.018+0000 7f9f65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:12:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.018+0000 7f9f65ffb640 1 -- 172.21.15.120:0/476492674 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f700b4020 con 0x5600199a2b70 2024-09-19T09:12:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.018+0000 7f9f65ffb640 1 -- 172.21.15.120:0/476492674 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f9f700a0170 con 0x5600199a2b70 2024-09-19T09:12:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.018+0000 7f9f87679740 1 -- 172.21.15.120:0/476492674 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f78037040 con 0x5600199a2b70 2024-09-19T09:12:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.018+0000 7f9f65ffb640 1 -- 172.21.15.120:0/476492674 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9f700a0690 con 0x5600199a2b70 2024-09-19T09:12:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.018+0000 7f9f87679740 10 client.28163 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.019+0000 7f9f87679740 1 -- 172.21.15.120:0/476492674 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5600196e5620 con 0x5600199a2b70 2024-09-19T09:12:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.019+0000 7f9f65ffb640 1 -- 172.21.15.120:0/476492674 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f78037040 con 0x5600199a2b70 2024-09-19T09:12:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.019+0000 7f9f65ffb640 1 -- 172.21.15.120:0/476492674 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f9f7009b730 con 0x5600199a2b70 2024-09-19T09:12:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.019+0000 7f9f87679740 10 client.28163 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.019+0000 7f9f87679740 10 client.28163 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:12:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.019+0000 7f9f87679740 20 client.28163 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.019+0000 7f9f87679740 1 --2- 172.21.15.120:0/476492674 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5600199b89a0 0x5600199badd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.019+0000 7f9f87679740 4 client.28163 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:12:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.019+0000 7f9f87679740 1 -- 172.21.15.120:0/476492674 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560019913a20 con 0x5600199b89a0 2024-09-19T09:12:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.019+0000 7f9f77fff640 1 --2- 172.21.15.120:0/476492674 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5600199b89a0 0x5600199badd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.019+0000 7f9f77fff640 1 --2- 172.21.15.120:0/476492674 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5600199b89a0 0x5600199badd0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bff9674f1df85d97 server_cookie=aa2ac9481560c28d in_seq=0 out_seq=0 2024-09-19T09:12:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.019+0000 7f9f65ffb640 10 client.28163 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.021+0000 7f9f65ffb640 1 -- 172.21.15.120:0/476492674 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560019913a20 con 0x5600199b89a0 2024-09-19T09:12:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.021+0000 7f9f65ffb640 10 client.28163 handle_command_reply: tid=0 2024-09-19T09:12:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.021+0000 7f9f87679740 1 client.28163 shutdown 2024-09-19T09:12:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.021+0000 7f9f87679740 1 -- 172.21.15.120:0/476492674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600199a2b70 msgr2=0x5600199acd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.021+0000 7f9f87679740 1 --2- 172.21.15.120:0/476492674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600199a2b70 0x5600199acd00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9f70098c20 tx=0x7f9f7009a420 comp rx=0 tx=0).stop 2024-09-19T09:12:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.021+0000 7f9f87679740 1 -- 172.21.15.120:0/476492674 shutdown_connections 2024-09-19T09:12:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.021+0000 7f9f87679740 1 --2- 172.21.15.120:0/476492674 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5600199b89a0 0x5600199badd0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.021+0000 7f9f87679740 1 --2- 172.21.15.120:0/476492674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600199a8540 0x5600199b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.021+0000 7f9f65ffb640 0 client.28163 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.021+0000 7f9f87679740 1 --2- 172.21.15.120:0/476492674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600199a34b0 0x5600199ad240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.021+0000 7f9f87679740 1 --2- 172.21.15.120:0/476492674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600199a2b70 0x5600199acd00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.021+0000 7f9f87679740 1 -- 172.21.15.120:0/476492674 >> 172.21.15.120:0/476492674 conn(0x56001999e1e0 msgr2=0x56001999f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.021+0000 7f9f87679740 1 -- 172.21.15.120:0/476492674 shutdown_connections 2024-09-19T09:12:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.021+0000 7f9f87679740 1 -- 172.21.15.120:0/476492674 wait complete. 2024-09-19T09:12:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.021+0000 7f9f87679740 20 client.28163 trim_cache size 0 max 16384 2024-09-19T09:12:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.042+0000 7f9f87679740 1 Processor -- start 2024-09-19T09:12:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.042+0000 7f9f87679740 1 -- start start 2024-09-19T09:12:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.043+0000 7f9f87679740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600199a2b70 0x5600199ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.043+0000 7f9f87679740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600199a34b0 0x5600199b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.043+0000 7f9f87679740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600199a8540 0x56001999f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.043+0000 7f9f87679740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9f801a8e30 con 0x5600199a8540 2024-09-19T09:12:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.043+0000 7f9f87679740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9f801aa970 con 0x5600199a2b70 2024-09-19T09:12:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.043+0000 7f9f87679740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9f801a9500 con 0x5600199a34b0 2024-09-19T09:12:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.043+0000 7f9f77fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600199a34b0 0x5600199b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.043+0000 7f9f7ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600199a8540 0x56001999f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.043+0000 7f9f7e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600199a2b70 0x5600199ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.043+0000 7f9f77fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600199a34b0 0x5600199b4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:33606/0 (socket says 172.21.15.120:33606) 2024-09-19T09:12:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.043+0000 7f9f77fff640 1 -- 172.21.15.120:0/3614894286 learned_addr learned my addr 172.21.15.120:0/3614894286 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.043+0000 7f9f77fff640 1 -- 172.21.15.120:0/3614894286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600199a2b70 msgr2=0x5600199ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.043+0000 7f9f77fff640 1 --2- 172.21.15.120:0/3614894286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600199a2b70 0x5600199ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.043+0000 7f9f77fff640 1 -- 172.21.15.120:0/3614894286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600199a8540 msgr2=0x56001999f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.043+0000 7f9f77fff640 1 --2- 172.21.15.120:0/3614894286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600199a8540 0x56001999f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.043+0000 7f9f77fff640 1 -- 172.21.15.120:0/3614894286 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600199bb450 con 0x5600199a34b0 2024-09-19T09:12:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.043+0000 7f9f77fff640 1 --2- 172.21.15.120:0/3614894286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600199a34b0 0x5600199b4210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9f680099c0 tx=0x7f9f68014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94d3aaa3558361f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.043+0000 7f9f67fff640 1 -- 172.21.15.120:0/3614894286 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f68037070 con 0x5600199a34b0 2024-09-19T09:12:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.043+0000 7f9f67fff640 1 -- 172.21.15.120:0/3614894286 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f68032420 con 0x5600199a34b0 2024-09-19T09:12:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.043+0000 7f9f67fff640 1 -- 172.21.15.120:0/3614894286 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6802a660 con 0x5600199a34b0 2024-09-19T09:12:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.045+0000 7f9f87679740 1 -- 172.21.15.120:0/3614894286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600199a34b0 msgr2=0x5600199b4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.045+0000 7f9f87679740 1 --2- 172.21.15.120:0/3614894286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600199a34b0 0x5600199b4210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9f680099c0 tx=0x7f9f68014000 comp rx=0 tx=0).stop 2024-09-19T09:12:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.045+0000 7f9f87679740 1 -- 172.21.15.120:0/3614894286 shutdown_connections 2024-09-19T09:12:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.045+0000 7f9f87679740 1 --2- 172.21.15.120:0/3614894286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600199a8540 0x56001999f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.045+0000 7f9f87679740 1 --2- 172.21.15.120:0/3614894286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600199a34b0 0x5600199b4210 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.045+0000 7f9f87679740 1 --2- 172.21.15.120:0/3614894286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600199a2b70 0x5600199ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.045+0000 7f9f87679740 1 -- 172.21.15.120:0/3614894286 >> 172.21.15.120:0/3614894286 conn(0x56001999e1e0 msgr2=0x56001999f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.045+0000 7f9f87679740 1 -- 172.21.15.120:0/3614894286 shutdown_connections 2024-09-19T09:12:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.045+0000 7f9f87679740 1 -- 172.21.15.120:0/3614894286 wait complete. 2024-09-19T09:12:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.045+0000 7f9f87679740 1 -- start start 2024-09-19T09:12:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.045+0000 7f9f87679740 1 Processor -- start 2024-09-19T09:12:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.046+0000 7f9f87679740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600199a2b70 0x5600199b1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.046+0000 7f9f87679740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600199a34b0 0x5600199b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.046+0000 7f9f87679740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600199a8540 0x560019a216a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.046+0000 7f9f87679740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9f801aa2a0 con 0x5600199a2b70 2024-09-19T09:12:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.046+0000 7f9f87679740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9f801a9bd0 con 0x5600199a8540 2024-09-19T09:12:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.046+0000 7f9f87679740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9f801a6df0 con 0x5600199a34b0 2024-09-19T09:12:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.046+0000 7f9f87679740 10 client.0 _ll_register_callbacks cb 0x7f9f7009be20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.046+0000 7f9f7e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600199a2b70 0x5600199b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.046+0000 7f9f77fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600199a34b0 0x5600199b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.046+0000 7f9f7ed9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600199a8540 0x560019a216a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.046+0000 7f9f7e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600199a2b70 0x5600199b1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53270/0 (socket says 172.21.15.120:53270) 2024-09-19T09:12:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.046+0000 7f9f77fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600199a34b0 0x5600199b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:33608/0 (socket says 172.21.15.120:33608) 2024-09-19T09:12:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.046+0000 7f9f7e59c640 1 -- 172.21.15.120:0/205170764 learned_addr learned my addr 172.21.15.120:0/205170764 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.046+0000 7f9f7e59c640 1 -- 172.21.15.120:0/205170764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600199a34b0 msgr2=0x5600199b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.046+0000 7f9f7e59c640 1 --2- 172.21.15.120:0/205170764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600199a34b0 0x5600199b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.046+0000 7f9f7e59c640 1 -- 172.21.15.120:0/205170764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600199a8540 msgr2=0x560019a216a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.046+0000 7f9f7e59c640 1 --2- 172.21.15.120:0/205170764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600199a8540 0x560019a216a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.046+0000 7f9f7e59c640 1 -- 172.21.15.120:0/205170764 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600199a4140 con 0x5600199a2b70 2024-09-19T09:12:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.046+0000 7f9f77fff640 1 --2- 172.21.15.120:0/205170764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600199a34b0 0x5600199b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.046+0000 7f9f7e59c640 1 --2- 172.21.15.120:0/205170764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600199a2b70 0x5600199b1100 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f9f700a4a20 tx=0x7f9f700bc3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175a1ae2dbce4839 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.046+0000 7f9f65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.046+0000 7f9f65ffb640 1 -- 172.21.15.120:0/205170764 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f700b4070 con 0x5600199a2b70 2024-09-19T09:12:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.046+0000 7f9f65ffb640 1 -- 172.21.15.120:0/205170764 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f9f700a0170 con 0x5600199a2b70 2024-09-19T09:12:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.046+0000 7f9f87679740 1 -- 172.21.15.120:0/205170764 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f68032420 con 0x5600199a2b70 2024-09-19T09:12:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.046+0000 7f9f65ffb640 1 -- 172.21.15.120:0/205170764 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9f7009c0b0 con 0x5600199a2b70 2024-09-19T09:12:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.046+0000 7f9f87679740 10 client.19047 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.046+0000 7f9f87679740 1 -- 172.21.15.120:0/205170764 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560019a22ca0 con 0x5600199a2b70 2024-09-19T09:12:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.047+0000 7f9f65ffb640 1 -- 172.21.15.120:0/205170764 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f68032420 con 0x5600199a2b70 2024-09-19T09:12:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.047+0000 7f9f65ffb640 1 -- 172.21.15.120:0/205170764 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f9f70085050 con 0x5600199a2b70 2024-09-19T09:12:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.047+0000 7f9f87679740 10 client.19047 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.047+0000 7f9f87679740 10 client.19047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:12:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.047+0000 7f9f87679740 20 client.19047 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.047+0000 7f9f87679740 1 --2- 172.21.15.120:0/205170764 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560019a234f0 0x560019a25910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.047+0000 7f9f87679740 4 client.19047 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:12:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.047+0000 7f9f87679740 1 -- 172.21.15.120:0/205170764 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56001990dd70 con 0x560019a234f0 2024-09-19T09:12:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.047+0000 7f9f77fff640 1 --2- 172.21.15.120:0/205170764 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560019a234f0 0x560019a25910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.047+0000 7f9f77fff640 1 --2- 172.21.15.120:0/205170764 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560019a234f0 0x560019a25910 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16eb5a810f5bdaf server_cookie=11d45fdf5473690c in_seq=0 out_seq=0 2024-09-19T09:12:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.047+0000 7f9f65ffb640 10 client.19047 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:45.067 INFO:tasks.workunit.client.0.smithi120.stdout: 1 984105 10.28 MB/sec execute 400 sec latency 19.215 ms 2024-09-19T09:12:45.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.125+0000 7f9705140640 1 -- 172.21.15.120:0/1639974907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9700109450 msgr2=0x7f9700109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:45.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.125+0000 7f9705140640 1 --2- 172.21.15.120:0/1639974907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9700109450 0x7f9700109830 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f96f000b880 tx=0x7f96f0005f90 comp rx=0 tx=0).stop 2024-09-19T09:12:45.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.126+0000 7f9705140640 1 -- 172.21.15.120:0/1639974907 shutdown_connections 2024-09-19T09:12:45.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.126+0000 7f9705140640 1 --2- 172.21.15.120:0/1639974907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f970010ea80 0x7f97001154f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.126+0000 7f9705140640 1 --2- 172.21.15.120:0/1639974907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9700109e00 0x7f970010e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.126+0000 7f9705140640 1 --2- 172.21.15.120:0/1639974907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9700109450 0x7f9700109830 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.126+0000 7f9705140640 1 -- 172.21.15.120:0/1639974907 >> 172.21.15.120:0/1639974907 conn(0x7f970006e2d0 msgr2=0x7f970006e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:45.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.134+0000 7f9705140640 1 -- 172.21.15.120:0/1639974907 shutdown_connections 2024-09-19T09:12:45.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.136+0000 7f9705140640 1 -- 172.21.15.120:0/1639974907 wait complete. 2024-09-19T09:12:45.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.137+0000 7f9705140640 1 Processor -- start 2024-09-19T09:12:45.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.142+0000 7f9705140640 1 -- start start 2024-09-19T09:12:45.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.142+0000 7f9705140640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9700109450 0x7f97001a72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.142+0000 7f9705140640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9700109e00 0x7f97001a7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.142+0000 7f9705140640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f970010ea80 0x7f97001abbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.142+0000 7f9705140640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9700118610 con 0x7f9700109e00 2024-09-19T09:12:45.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.142+0000 7f9705140640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9700118490 con 0x7f9700109450 2024-09-19T09:12:45.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.142+0000 7f9705140640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9700118790 con 0x7f970010ea80 2024-09-19T09:12:45.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.143+0000 7f96fed9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9700109450 0x7f97001a72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.143+0000 7f96fed9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9700109450 0x7f97001a72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:45728/0 (socket says 172.21.15.120:45728) 2024-09-19T09:12:45.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.143+0000 7f96fed9d640 1 -- 172.21.15.120:0/4076564946 learned_addr learned my addr 172.21.15.120:0/4076564946 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:45.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.143+0000 7f96ff59e640 1 --2- 172.21.15.120:0/4076564946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f970010ea80 0x7f97001abbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.145+0000 7f96ff59e640 1 -- 172.21.15.120:0/4076564946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9700109450 msgr2=0x7f97001a72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:45.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.145+0000 7f96ff59e640 1 --2- 172.21.15.120:0/4076564946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9700109450 0x7f97001a72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.145+0000 7f96ff59e640 1 -- 172.21.15.120:0/4076564946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9700109e00 msgr2=0x7f97001a7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:45.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.145+0000 7f96ff59e640 1 --2- 172.21.15.120:0/4076564946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9700109e00 0x7f97001a7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.145+0000 7f96ff59e640 1 -- 172.21.15.120:0/4076564946 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97001ac2b0 con 0x7f970010ea80 2024-09-19T09:12:45.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.148+0000 7f96ff59e640 1 --2- 172.21.15.120:0/4076564946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f970010ea80 0x7f97001abbb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f96f800be20 tx=0x7f96f800bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:45.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.150+0000 7f96e7fff640 1 -- 172.21.15.120:0/4076564946 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f8019070 con 0x7f970010ea80 2024-09-19T09:12:45.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.150+0000 7f96e7fff640 1 -- 172.21.15.120:0/4076564946 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f8014920 con 0x7f970010ea80 2024-09-19T09:12:45.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.150+0000 7f96e7fff640 1 -- 172.21.15.120:0/4076564946 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f8012910 con 0x7f970010ea80 2024-09-19T09:12:45.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.152+0000 7f9705140640 1 -- 172.21.15.120:0/4076564946 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97001ac5a0 con 0x7f970010ea80 2024-09-19T09:12:45.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.152+0000 7f9705140640 1 -- 172.21.15.120:0/4076564946 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f970019e830 con 0x7f970010ea80 2024-09-19T09:12:45.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.159+0000 7f9707519740 1 Processor -- start 2024-09-19T09:12:45.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.159+0000 7f9707519740 1 -- start start 2024-09-19T09:12:45.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.159+0000 7f9707519740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b68d509760 0x55b68d509b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.159+0000 7f9707519740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d50a080 0x55b68d514400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.159+0000 7f9707519740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b68d514940 0x55b68d516d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.160+0000 7f9707519740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b68d46f2a0 con 0x55b68d509760 2024-09-19T09:12:45.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.160+0000 7f9707519740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b68d416b10 con 0x55b68d514940 2024-09-19T09:12:45.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.160+0000 7f9707519740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b68d46b260 con 0x55b68d50a080 2024-09-19T09:12:45.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.160+0000 7f96fed9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d50a080 0x55b68d514400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.160+0000 7f96fed9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d50a080 0x55b68d514400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:33646/0 (socket says 172.21.15.120:33646) 2024-09-19T09:12:45.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.161+0000 7f96fed9d640 1 -- 172.21.15.120:0/2759289336 learned_addr learned my addr 172.21.15.120:0/2759289336 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:45.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.161+0000 7f96fe59c640 1 --2- 172.21.15.120:0/2759289336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b68d509760 0x55b68d509b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.161+0000 7f96fe59c640 1 --2- 172.21.15.120:0/2759289336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b68d514940 0x55b68d516d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.162+0000 7f96fed9d640 1 -- 172.21.15.120:0/2759289336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b68d514940 msgr2=0x55b68d516d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:45.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.162+0000 7f96fed9d640 1 --2- 172.21.15.120:0/2759289336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b68d514940 0x55b68d516d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.162+0000 7f96fed9d640 1 -- 172.21.15.120:0/2759289336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b68d509760 msgr2=0x55b68d509b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:45.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.162+0000 7f96fed9d640 1 --2- 172.21.15.120:0/2759289336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b68d509760 0x55b68d509b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.162+0000 7f96fed9d640 1 -- 172.21.15.120:0/2759289336 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b68d1e0b90 con 0x55b68d50a080 2024-09-19T09:12:45.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.163+0000 7f96e7fff640 1 -- 172.21.15.120:0/4076564946 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f96f8014440 con 0x7f970010ea80 2024-09-19T09:12:45.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.163+0000 7f96fed9d640 1 --2- 172.21.15.120:0/2759289336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d50a080 0x55b68d514400 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f96f000b9a0 tx=0x7f96f000ac40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8479ebb0aeddd33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:45.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.164+0000 7f96e57fa640 1 -- 172.21.15.120:0/2759289336 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f0041070 con 0x55b68d50a080 2024-09-19T09:12:45.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.164+0000 7f96e57fa640 1 -- 172.21.15.120:0/2759289336 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f002ac30 con 0x55b68d50a080 2024-09-19T09:12:45.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.164+0000 7f96e57fa640 1 -- 172.21.15.120:0/2759289336 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f00035f0 con 0x55b68d50a080 2024-09-19T09:12:45.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.164+0000 7f96e7fff640 1 --2- 172.21.15.120:0/4076564946 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f96e80a3450 0x7f96e80a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.164+0000 7f96e7fff640 1 -- 172.21.15.120:0/4076564946 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f96f80cf0d0 con 0x7f970010ea80 2024-09-19T09:12:45.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.164+0000 7f96fe59c640 1 --2- 172.21.15.120:0/4076564946 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f96e80a3450 0x7f96e80a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.165+0000 7f9707519740 1 -- 172.21.15.120:0/2759289336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d50a080 msgr2=0x55b68d514400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:45.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.165+0000 7f9707519740 1 --2- 172.21.15.120:0/2759289336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d50a080 0x55b68d514400 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f96f000b9a0 tx=0x7f96f000ac40 comp rx=0 tx=0).stop 2024-09-19T09:12:45.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.165+0000 7f96fe59c640 1 --2- 172.21.15.120:0/4076564946 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f96e80a3450 0x7f96e80a5910 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f96f400ab50 tx=0x7f96f400c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:45.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.169+0000 7f9707519740 1 -- 172.21.15.120:0/2759289336 shutdown_connections 2024-09-19T09:12:45.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.169+0000 7f9707519740 1 --2- 172.21.15.120:0/2759289336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b68d514940 0x55b68d516d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.169+0000 7f9707519740 1 --2- 172.21.15.120:0/2759289336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d50a080 0x55b68d514400 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.169+0000 7f9707519740 1 --2- 172.21.15.120:0/2759289336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b68d509760 0x55b68d509b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.169+0000 7f9707519740 1 -- 172.21.15.120:0/2759289336 >> 172.21.15.120:0/2759289336 conn(0x55b68d505cf0 msgr2=0x55b68d5060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:45.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.169+0000 7f9707519740 1 -- 172.21.15.120:0/2759289336 shutdown_connections 2024-09-19T09:12:45.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.170+0000 7f9707519740 1 -- 172.21.15.120:0/2759289336 wait complete. 2024-09-19T09:12:45.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.170+0000 7f9707519740 1 -- start start 2024-09-19T09:12:45.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.170+0000 7f9707519740 1 Processor -- start 2024-09-19T09:12:45.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.171+0000 7f9707519740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d509760 0x55b68d5116f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.171+0000 7f9707519740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b68d50a080 0x55b68d511c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.171+0000 7f9707519740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b68d514940 0x55b68d512170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.171+0000 7f9707519740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b68d517560 con 0x55b68d50a080 2024-09-19T09:12:45.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.171+0000 7f9707519740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b68d3f76e0 con 0x55b68d514940 2024-09-19T09:12:45.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.171+0000 7f9707519740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b68d5176e0 con 0x55b68d509760 2024-09-19T09:12:45.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.171+0000 7f9707519740 10 client.0 _ll_register_callbacks cb 0x55b68d41ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:45.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.171+0000 7f96fe59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b68d50a080 0x55b68d511c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.171+0000 7f96fe59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b68d50a080 0x55b68d511c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53298/0 (socket says 172.21.15.120:53298) 2024-09-19T09:12:45.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.171+0000 7f96fe59c640 1 -- 172.21.15.120:0/3056984775 learned_addr learned my addr 172.21.15.120:0/3056984775 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:45.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.171+0000 7f96fed9d640 1 --2- 172.21.15.120:0/3056984775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d509760 0x55b68d5116f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.172+0000 7f96fe59c640 1 -- 172.21.15.120:0/3056984775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d509760 msgr2=0x55b68d5116f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:45.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.172+0000 7f96fe59c640 1 --2- 172.21.15.120:0/3056984775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d509760 0x55b68d5116f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.172+0000 7f96fe59c640 1 -- 172.21.15.120:0/3056984775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b68d514940 msgr2=0x55b68d512170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:12:45.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.172+0000 7f96fe59c640 1 --2- 172.21.15.120:0/3056984775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b68d514940 0x55b68d512170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.172+0000 7f96fe59c640 1 -- 172.21.15.120:0/3056984775 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b68d520a60 con 0x55b68d50a080 2024-09-19T09:12:45.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.173+0000 7f96fe59c640 1 --2- 172.21.15.120:0/3056984775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b68d50a080 0x55b68d511c30 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f96f401f9b0 tx=0x7f96f400ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cfba2cfe50e41fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:45.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.173+0000 7f96e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:45.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.173+0000 7f96e2ff5640 1 -- 172.21.15.120:0/3056984775 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f401d730 con 0x55b68d50a080 2024-09-19T09:12:45.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.173+0000 7f96e2ff5640 1 -- 172.21.15.120:0/3056984775 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f96f401d8d0 con 0x55b68d50a080 2024-09-19T09:12:45.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.173+0000 7f9707519740 1 -- 172.21.15.120:0/3056984775 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96f002ac30 con 0x55b68d50a080 2024-09-19T09:12:45.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.173+0000 7f96e2ff5640 1 -- 172.21.15.120:0/3056984775 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f96f401dad0 con 0x55b68d50a080 2024-09-19T09:12:45.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.174+0000 7f9707519740 10 client.19053 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:45.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.174+0000 7f9707519740 1 -- 172.21.15.120:0/3056984775 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b68d523380 con 0x55b68d50a080 2024-09-19T09:12:45.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.174+0000 7f96e2ff5640 1 -- 172.21.15.120:0/3056984775 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f002ac30 con 0x55b68d50a080 2024-09-19T09:12:45.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.174+0000 7f96e2ff5640 1 -- 172.21.15.120:0/3056984775 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f96f401c070 con 0x55b68d50a080 2024-09-19T09:12:45.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.174+0000 7f9707519740 10 client.19053 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:45.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.174+0000 7f9707519740 10 client.19053 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:12:45.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.174+0000 7f9707519740 20 client.19053 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:45.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.174+0000 7f9707519740 1 --2- 172.21.15.120:0/3056984775 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b68d524d60 0x55b68d527200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.174+0000 7f9707519740 4 client.19053 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:12:45.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.174+0000 7f9707519740 1 -- 172.21.15.120:0/3056984775 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b68d47c9d0 con 0x55b68d524d60 2024-09-19T09:12:45.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.175+0000 7f96fed9d640 1 --2- 172.21.15.120:0/3056984775 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b68d524d60 0x55b68d527200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.177+0000 7f96fed9d640 1 --2- 172.21.15.120:0/3056984775 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b68d524d60 0x55b68d527200 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b154c9b225bb268a server_cookie=d142bd9aa454454c in_seq=0 out_seq=0 2024-09-19T09:12:45.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.178+0000 7f96e2ff5640 10 client.19053 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:45.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.179+0000 7f96e2ff5640 1 -- 172.21.15.120:0/3056984775 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b68d47c9d0 con 0x55b68d524d60 2024-09-19T09:12:45.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.179+0000 7f96e2ff5640 10 client.19053 handle_command_reply: tid=0 2024-09-19T09:12:45.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.179+0000 7f9707519740 1 client.19053 shutdown 2024-09-19T09:12:45.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.180+0000 7f9707519740 1 -- 172.21.15.120:0/3056984775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b68d50a080 msgr2=0x55b68d511c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:45.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.180+0000 7f9707519740 1 --2- 172.21.15.120:0/3056984775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b68d50a080 0x55b68d511c30 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f96f401f9b0 tx=0x7f96f400ac00 comp rx=0 tx=0).stop 2024-09-19T09:12:45.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.180+0000 7f9707519740 1 -- 172.21.15.120:0/3056984775 shutdown_connections 2024-09-19T09:12:45.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.180+0000 7f9707519740 1 --2- 172.21.15.120:0/3056984775 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b68d524d60 0x55b68d527200 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.180+0000 7f9707519740 1 --2- 172.21.15.120:0/3056984775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b68d514940 0x55b68d512170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.180+0000 7f9707519740 1 --2- 172.21.15.120:0/3056984775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b68d50a080 0x55b68d511c30 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.180+0000 7f9707519740 1 --2- 172.21.15.120:0/3056984775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d509760 0x55b68d5116f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.180+0000 7f9707519740 1 -- 172.21.15.120:0/3056984775 >> 172.21.15.120:0/3056984775 conn(0x55b68d505cf0 msgr2=0x55b68d504e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:45.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.181+0000 7f96e2ff5640 0 client.19053 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:45.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.181+0000 7f9707519740 1 -- 172.21.15.120:0/3056984775 shutdown_connections 2024-09-19T09:12:45.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.181+0000 7f9707519740 1 -- 172.21.15.120:0/3056984775 wait complete. 2024-09-19T09:12:45.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.181+0000 7f9707519740 20 client.19053 trim_cache size 0 max 16384 2024-09-19T09:12:45.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:44 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:12:45.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:44 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:12:45.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:44 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:12:45.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:44 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:12:45.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:44 smithi196 ceph-mon[35150]: pgmap v639: 97 pgs: 97 active+clean; 30 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 3.8 MiB/s wr, 431 op/s 2024-09-19T09:12:45.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:44 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:12:45.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:44 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:12:45.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:44 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:12:45.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:44 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:12:45.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:44 smithi153 ceph-mon[35071]: pgmap v639: 97 pgs: 97 active+clean; 30 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 3.8 MiB/s wr, 431 op/s 2024-09-19T09:12:45.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.205+0000 7f9707519740 1 Processor -- start 2024-09-19T09:12:45.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.205+0000 7f9707519740 1 -- start start 2024-09-19T09:12:45.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.205+0000 7f9707519740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d509760 0x55b68d504d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.205+0000 7f9707519740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b68d50a080 0x55b68d52cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.205+0000 7f9707519740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b68d514940 0x55b68d52d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.205+0000 7f9707519740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec00e4a0 con 0x55b68d514940 2024-09-19T09:12:45.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.205+0000 7f9707519740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec00ffe0 con 0x55b68d50a080 2024-09-19T09:12:45.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.205+0000 7f9707519740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec00eb70 con 0x55b68d509760 2024-09-19T09:12:45.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.205+0000 7f96ff59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b68d514940 0x55b68d52d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.205+0000 7f96fe59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b68d50a080 0x55b68d52cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.205+0000 7f96ff59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b68d514940 0x55b68d52d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53302/0 (socket says 172.21.15.120:53302) 2024-09-19T09:12:45.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.205+0000 7f96ff59e640 1 -- 172.21.15.120:0/1287780923 learned_addr learned my addr 172.21.15.120:0/1287780923 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:45.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.205+0000 7f96fed9d640 1 --2- 172.21.15.120:0/1287780923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d509760 0x55b68d504d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.206+0000 7f96ff59e640 1 -- 172.21.15.120:0/1287780923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d509760 msgr2=0x55b68d504d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:45.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.206+0000 7f96ff59e640 1 --2- 172.21.15.120:0/1287780923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d509760 0x55b68d504d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.206+0000 7f96ff59e640 1 -- 172.21.15.120:0/1287780923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b68d50a080 msgr2=0x55b68d52cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:45.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.206+0000 7f96ff59e640 1 --2- 172.21.15.120:0/1287780923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b68d50a080 0x55b68d52cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.206+0000 7f96ff59e640 1 -- 172.21.15.120:0/1287780923 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b68d505b20 con 0x55b68d514940 2024-09-19T09:12:45.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.206+0000 7f96fe59c640 1 --2- 172.21.15.120:0/1287780923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b68d50a080 0x55b68d52cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:45.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.206+0000 7f96ff59e640 1 --2- 172.21.15.120:0/1287780923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b68d514940 0x55b68d52d050 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f96f8098080 tx=0x7f96f800cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9de3cd0335fb5bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:45.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.206+0000 7f96e4ff9640 1 -- 172.21.15.120:0/1287780923 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f809b6b0 con 0x55b68d514940 2024-09-19T09:12:45.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.206+0000 7f96e4ff9640 1 -- 172.21.15.120:0/1287780923 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f80b4070 con 0x55b68d514940 2024-09-19T09:12:45.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.206+0000 7f96e4ff9640 1 -- 172.21.15.120:0/1287780923 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f80afbd0 con 0x55b68d514940 2024-09-19T09:12:45.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.207+0000 7f9707519740 1 -- 172.21.15.120:0/1287780923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b68d514940 msgr2=0x55b68d52d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:45.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.207+0000 7f9707519740 1 --2- 172.21.15.120:0/1287780923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b68d514940 0x55b68d52d050 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f96f8098080 tx=0x7f96f800cef0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.207+0000 7f9707519740 1 -- 172.21.15.120:0/1287780923 shutdown_connections 2024-09-19T09:12:45.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.207+0000 7f9707519740 1 --2- 172.21.15.120:0/1287780923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b68d514940 0x55b68d52d050 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.207+0000 7f9707519740 1 --2- 172.21.15.120:0/1287780923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b68d50a080 0x55b68d52cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.207+0000 7f9707519740 1 --2- 172.21.15.120:0/1287780923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d509760 0x55b68d504d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.207+0000 7f9707519740 1 -- 172.21.15.120:0/1287780923 >> 172.21.15.120:0/1287780923 conn(0x55b68d505cf0 msgr2=0x55b68d504650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:45.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.207+0000 7f9707519740 1 -- 172.21.15.120:0/1287780923 shutdown_connections 2024-09-19T09:12:45.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.207+0000 7f9707519740 1 -- 172.21.15.120:0/1287780923 wait complete. 2024-09-19T09:12:45.210 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:12:45.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.207+0000 7f9707519740 1 -- start start 2024-09-19T09:12:45.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.207+0000 7f9707519740 1 Processor -- start 2024-09-19T09:12:45.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.208+0000 7f9707519740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b68d509760 0x55b68d51c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.208+0000 7f9707519740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d50a080 0x55b68d51cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.208+0000 7f9707519740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b68d514940 0x55b68d515c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.208+0000 7f9707519740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec00f910 con 0x55b68d514940 2024-09-19T09:12:45.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.208+0000 7f9707519740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec00f240 con 0x55b68d509760 2024-09-19T09:12:45.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.208+0000 7f9707519740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec00c460 con 0x55b68d50a080 2024-09-19T09:12:45.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.208+0000 7f9707519740 10 client.0 _ll_register_callbacks cb 0x7f96f401f630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:45.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.208+0000 7f96fed9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b68d509760 0x55b68d51c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.208+0000 7f96ff59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b68d514940 0x55b68d515c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.208+0000 7f96fed9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b68d509760 0x55b68d51c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:45758/0 (socket says 172.21.15.120:45758) 2024-09-19T09:12:45.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.208+0000 7f96ff59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b68d514940 0x55b68d515c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53318/0 (socket says 172.21.15.120:53318) 2024-09-19T09:12:45.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.208+0000 7f96fed9d640 1 -- 172.21.15.120:0/3568168131 learned_addr learned my addr 172.21.15.120:0/3568168131 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:45.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.208+0000 7f96fe59c640 1 --2- 172.21.15.120:0/3568168131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d50a080 0x55b68d51cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.208+0000 7f96fed9d640 1 -- 172.21.15.120:0/3568168131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d50a080 msgr2=0x55b68d51cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:45.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.208+0000 7f96fed9d640 1 --2- 172.21.15.120:0/3568168131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d50a080 0x55b68d51cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.208+0000 7f96fed9d640 1 -- 172.21.15.120:0/3568168131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b68d514940 msgr2=0x55b68d515c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:45.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.208+0000 7f96fed9d640 1 --2- 172.21.15.120:0/3568168131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b68d514940 0x55b68d515c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.208+0000 7f96fed9d640 1 -- 172.21.15.120:0/3568168131 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b68d501260 con 0x55b68d509760 2024-09-19T09:12:45.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.209+0000 7f96fed9d640 1 --2- 172.21.15.120:0/3568168131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b68d509760 0x55b68d51c9c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009250 tx=0x7f96f000a650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2aa8beccae6a7bab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:45.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.209+0000 7f96e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:12:45.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.209+0000 7f96e2ff5640 1 -- 172.21.15.120:0/3568168131 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f0041040 con 0x55b68d509760 2024-09-19T09:12:45.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.209+0000 7f96e2ff5640 1 -- 172.21.15.120:0/3568168131 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f96f000a8c0 con 0x55b68d509760 2024-09-19T09:12:45.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.209+0000 7f9707519740 1 -- 172.21.15.120:0/3568168131 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96f80b4070 con 0x55b68d509760 2024-09-19T09:12:45.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.209+0000 7f96e2ff5640 1 -- 172.21.15.120:0/3568168131 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f96f0061050 con 0x55b68d509760 2024-09-19T09:12:45.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.209+0000 7f9707519740 10 client.28184 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:45.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.209+0000 7f9707519740 1 -- 172.21.15.120:0/3568168131 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b68d58fdd0 con 0x55b68d509760 2024-09-19T09:12:45.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.210+0000 7f96e2ff5640 1 -- 172.21.15.120:0/3568168131 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f0005530 con 0x55b68d509760 2024-09-19T09:12:45.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.210+0000 7f96e2ff5640 1 -- 172.21.15.120:0/3568168131 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f96f004f3f0 con 0x55b68d509760 2024-09-19T09:12:45.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.210+0000 7f9707519740 10 client.28184 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:45.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.210+0000 7f9707519740 10 client.28184 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:12:45.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.210+0000 7f9707519740 20 client.28184 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:45.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.210+0000 7f9707519740 1 --2- 172.21.15.120:0/3568168131 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b68d590180 0x55b68d592570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:45.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.210+0000 7f9707519740 4 client.28184 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-19T09:12:45.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.210+0000 7f9707519740 1 -- 172.21.15.120:0/3568168131 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x7f96f80b4070 con 0x55b68d590180 2024-09-19T09:12:45.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.210+0000 7f96fe59c640 1 --2- 172.21.15.120:0/3568168131 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b68d590180 0x55b68d592570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:45.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.210+0000 7f96e2ff5640 10 client.28184 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:45.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.210+0000 7f96fe59c640 1 --2- 172.21.15.120:0/3568168131 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b68d590180 0x55b68d592570 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e77d7ce621338c12 server_cookie=12758bca7c835c9f in_seq=0 out_seq=0 2024-09-19T09:12:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.222+0000 7f9f65ffb640 1 -- 172.21.15.120:0/205170764 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56001990dd70 con 0x560019a234f0 2024-09-19T09:12:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.222+0000 7f9f65ffb640 10 client.19047 handle_command_reply: tid=0 2024-09-19T09:12:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.222+0000 7f9f87679740 1 client.19047 shutdown 2024-09-19T09:12:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.222+0000 7f9f87679740 1 -- 172.21.15.120:0/205170764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600199a2b70 msgr2=0x5600199b1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.222+0000 7f9f87679740 1 --2- 172.21.15.120:0/205170764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600199a2b70 0x5600199b1100 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f9f700a4a20 tx=0x7f9f700bc3c0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.222+0000 7f9f87679740 1 -- 172.21.15.120:0/205170764 shutdown_connections 2024-09-19T09:12:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.222+0000 7f9f87679740 1 --2- 172.21.15.120:0/205170764 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560019a234f0 0x560019a25910 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.222+0000 7f9f87679740 1 --2- 172.21.15.120:0/205170764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600199a8540 0x560019a216a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.222+0000 7f9f87679740 1 --2- 172.21.15.120:0/205170764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5600199a34b0 0x5600199b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.222+0000 7f9f87679740 1 --2- 172.21.15.120:0/205170764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600199a2b70 0x5600199b1100 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.222+0000 7f9f87679740 1 -- 172.21.15.120:0/205170764 >> 172.21.15.120:0/205170764 conn(0x56001999e1e0 msgr2=0x56001999f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.223+0000 7f9f65ffb640 0 client.19047 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.223+0000 7f9f87679740 1 -- 172.21.15.120:0/205170764 shutdown_connections 2024-09-19T09:12:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.223+0000 7f9f87679740 1 -- 172.21.15.120:0/205170764 wait complete. 2024-09-19T09:12:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.224+0000 7f9f87679740 20 client.19047 trim_cache size 0 max 16384 2024-09-19T09:12:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:12:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:12:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:12:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:12:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.225+0000 7f9f7f59e640 1 -- 172.21.15.120:0/3075985884 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9f6c0a3240 msgr2=0x7f9f6c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.225+0000 7f9f7f59e640 1 --2- 172.21.15.120:0/3075985884 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9f6c0a3240 0x7f9f6c0a5700 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9f680095f0 tx=0x7f9f68005f90 comp rx=0 tx=0).stop 2024-09-19T09:12:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.225+0000 7f9f7f59e640 1 -- 172.21.15.120:0/3075985884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f801023c0 msgr2=0x7f9f80197cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.225+0000 7f9f7f59e640 1 --2- 172.21.15.120:0/3075985884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f801023c0 0x7f9f80197cf0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000e9c0 tx=0x7f9f7000ee80 comp rx=0 tx=0).stop 2024-09-19T09:12:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.225+0000 7f9f7f59e640 1 -- 172.21.15.120:0/3075985884 shutdown_connections 2024-09-19T09:12:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.225+0000 7f9f7f59e640 1 --2- 172.21.15.120:0/3075985884 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9f6c0a3240 0x7f9f6c0a5700 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.225+0000 7f9f7f59e640 1 --2- 172.21.15.120:0/3075985884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f80107620 0x7f9f8019f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.225+0000 7f9f7f59e640 1 --2- 172.21.15.120:0/3075985884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f80104cf0 0x7f9f80198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.225+0000 7f9f7f59e640 1 --2- 172.21.15.120:0/3075985884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f801023c0 0x7f9f80197cf0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.225+0000 7f9f7f59e640 1 -- 172.21.15.120:0/3075985884 >> 172.21.15.120:0/3075985884 conn(0x7f9f800fbfa0 msgr2=0x7f9f80102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.225+0000 7f9f7f59e640 1 -- 172.21.15.120:0/3075985884 shutdown_connections 2024-09-19T09:12:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:45.225+0000 7f9f7f59e640 1 -- 172.21.15.120:0/3075985884 wait complete. 2024-09-19T09:12:45.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.371+0000 7f96e2ff5640 1 -- 172.21.15.120:0/3568168131 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f96f80b4070 con 0x55b68d590180 2024-09-19T09:12:45.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.371+0000 7f96e2ff5640 10 client.28184 handle_command_reply: tid=0 2024-09-19T09:12:45.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.371+0000 7f9707519740 1 client.28184 shutdown 2024-09-19T09:12:45.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.371+0000 7f9707519740 1 -- 172.21.15.120:0/3568168131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b68d509760 msgr2=0x55b68d51c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:45.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.371+0000 7f9707519740 1 --2- 172.21.15.120:0/3568168131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b68d509760 0x55b68d51c9c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009250 tx=0x7f96f000a650 comp rx=0 tx=0).stop 2024-09-19T09:12:45.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.372+0000 7f9707519740 1 -- 172.21.15.120:0/3568168131 shutdown_connections 2024-09-19T09:12:45.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.372+0000 7f9707519740 1 --2- 172.21.15.120:0/3568168131 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b68d590180 0x55b68d592570 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.372+0000 7f9707519740 1 --2- 172.21.15.120:0/3568168131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b68d514940 0x55b68d515c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.372+0000 7f96e2ff5640 0 client.28184 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:45.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.372+0000 7f9707519740 1 --2- 172.21.15.120:0/3568168131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d50a080 0x55b68d51cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.372+0000 7f9707519740 1 --2- 172.21.15.120:0/3568168131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b68d509760 0x55b68d51c9c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.372+0000 7f9707519740 1 -- 172.21.15.120:0/3568168131 >> 172.21.15.120:0/3568168131 conn(0x55b68d505cf0 msgr2=0x55b68d504e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:45.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.372+0000 7f9707519740 1 -- 172.21.15.120:0/3568168131 shutdown_connections 2024-09-19T09:12:45.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.372+0000 7f9707519740 1 -- 172.21.15.120:0/3568168131 wait complete. 2024-09-19T09:12:45.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.372+0000 7f9707519740 20 client.28184 trim_cache size 0 max 16384 2024-09-19T09:12:45.378 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:12:45.378 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 32, 2024-09-19T09:12:45.378 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:12:45.378 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 34, 2024-09-19T09:12:45.378 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:12:45.379 INFO:teuthology.orchestra.run.smithi120.stdout: "b8a09801": { 2024-09-19T09:12:45.379 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 34, 2024-09-19T09:12:45.379 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:12:45.379 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:12:45.379 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:12:45.379 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:12:45.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:12:45.379 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:12:45.379 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 150.0, 2024-09-19T09:12:45.379 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:12:45.379 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:12:45.379 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:12:45.379 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:12:45.380 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:12:45.380 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:12:45.380 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:12:45.380 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:12:45.380 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:12:45.380 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:12:45.380 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:12:45.380 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:12:45.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.372+0000 7f9705140640 1 -- 172.21.15.120:0/4076564946 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f96e80a3450 msgr2=0x7f96e80a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:45.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.372+0000 7f9705140640 1 --2- 172.21.15.120:0/4076564946 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f96e80a3450 0x7f96e80a5910 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f96f400ab50 tx=0x7f96f400c040 comp rx=0 tx=0).stop 2024-09-19T09:12:45.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.372+0000 7f9705140640 1 -- 172.21.15.120:0/4076564946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f970010ea80 msgr2=0x7f97001abbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:45.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.372+0000 7f9705140640 1 --2- 172.21.15.120:0/4076564946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f970010ea80 0x7f97001abbb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f96f800be20 tx=0x7f96f800bf10 comp rx=0 tx=0).stop 2024-09-19T09:12:45.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.373+0000 7f9705140640 1 -- 172.21.15.120:0/4076564946 shutdown_connections 2024-09-19T09:12:45.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.373+0000 7f9705140640 1 --2- 172.21.15.120:0/4076564946 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f96e80a3450 0x7f96e80a5910 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.373+0000 7f9705140640 1 --2- 172.21.15.120:0/4076564946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f970010ea80 0x7f97001abbb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.373+0000 7f9705140640 1 --2- 172.21.15.120:0/4076564946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9700109e00 0x7f97001a7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.373+0000 7f9705140640 1 --2- 172.21.15.120:0/4076564946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9700109450 0x7f97001a72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:45.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.373+0000 7f9705140640 1 -- 172.21.15.120:0/4076564946 >> 172.21.15.120:0/4076564946 conn(0x7f970006e2d0 msgr2=0x7f970010c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:45.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.373+0000 7f9705140640 1 -- 172.21.15.120:0/4076564946 shutdown_connections 2024-09-19T09:12:45.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:45.373+0000 7f9705140640 1 -- 172.21.15.120:0/4076564946 wait complete. 2024-09-19T09:12:46.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.014+0000 7fd7992e7640 1 -- 172.21.15.120:0/677365252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd794102ef0 msgr2=0x7fd79410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:46.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.014+0000 7fd7992e7640 1 --2- 172.21.15.120:0/677365252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd794102ef0 0x7fd79410da40 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fd78c009ca0 tx=0x7fd78c02a950 comp rx=0 tx=0).stop 2024-09-19T09:12:46.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.014+0000 7fd7992e7640 1 -- 172.21.15.120:0/677365252 shutdown_connections 2024-09-19T09:12:46.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.014+0000 7fd7992e7640 1 --2- 172.21.15.120:0/677365252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd794102ef0 0x7fd79410da40 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.014+0000 7fd7992e7640 1 --2- 172.21.15.120:0/677365252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd794102570 0x7fd7941029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.014+0000 7fd7992e7640 1 --2- 172.21.15.120:0/677365252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd794100e20 0x7fd794101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.014+0000 7fd7992e7640 1 -- 172.21.15.120:0/677365252 >> 172.21.15.120:0/677365252 conn(0x7fd7940fc7d0 msgr2=0x7fd7940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:46.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.014+0000 7fd7992e7640 1 -- 172.21.15.120:0/677365252 shutdown_connections 2024-09-19T09:12:46.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.014+0000 7fd7992e7640 1 -- 172.21.15.120:0/677365252 wait complete. 2024-09-19T09:12:46.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.015+0000 7fd7992e7640 1 Processor -- start 2024-09-19T09:12:46.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.015+0000 7fd7992e7640 1 -- start start 2024-09-19T09:12:46.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.015+0000 7fd7992e7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd794100e20 0x7fd7941973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:46.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.015+0000 7fd7992e7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd794102570 0x7fd794197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:46.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.015+0000 7fd7992e7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd794102ef0 0x7fd79419bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:46.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.015+0000 7fd7992e7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd794110fa0 con 0x7fd794102ef0 2024-09-19T09:12:46.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.015+0000 7fd7992e7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd794110e20 con 0x7fd794100e20 2024-09-19T09:12:46.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.015+0000 7fd7992e7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd794111120 con 0x7fd794102570 2024-09-19T09:12:46.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.015+0000 7fd7937fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd794102ef0 0x7fd79419bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:46.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.015+0000 7fd7937fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd794102ef0 0x7fd79419bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53328/0 (socket says 172.21.15.120:53328) 2024-09-19T09:12:46.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.015+0000 7fd7937fe640 1 -- 172.21.15.120:0/1938545011 learned_addr learned my addr 172.21.15.120:0/1938545011 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:46.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.015+0000 7fd792ffd640 1 --2- 172.21.15.120:0/1938545011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd794100e20 0x7fd7941973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:46.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.015+0000 7fd78a7fc640 1 --2- 172.21.15.120:0/1938545011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd794102570 0x7fd794197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:46.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.016+0000 7fd7937fe640 1 -- 172.21.15.120:0/1938545011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd794102570 msgr2=0x7fd794197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:46.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.016+0000 7fd7937fe640 1 --2- 172.21.15.120:0/1938545011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd794102570 0x7fd794197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.016+0000 7fd7937fe640 1 -- 172.21.15.120:0/1938545011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd794100e20 msgr2=0x7fd7941973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:46.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.016+0000 7fd7937fe640 1 --2- 172.21.15.120:0/1938545011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd794100e20 0x7fd7941973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.016+0000 7fd7937fe640 1 -- 172.21.15.120:0/1938545011 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd79419c390 con 0x7fd794102ef0 2024-09-19T09:12:46.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.016+0000 7fd7937fe640 1 --2- 172.21.15.120:0/1938545011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd794102ef0 0x7fd79419bcb0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fd78c02ae70 tx=0x7fd78c02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:46.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.016+0000 7fd790ff9640 1 -- 172.21.15.120:0/1938545011 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c034040 con 0x7fd794102ef0 2024-09-19T09:12:46.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.016+0000 7fd790ff9640 1 -- 172.21.15.120:0/1938545011 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd78c030920 con 0x7fd794102ef0 2024-09-19T09:12:46.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.016+0000 7fd7992e7640 1 -- 172.21.15.120:0/1938545011 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd79419c620 con 0x7fd794102ef0 2024-09-19T09:12:46.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.016+0000 7fd790ff9640 1 -- 172.21.15.120:0/1938545011 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c045480 con 0x7fd794102ef0 2024-09-19T09:12:46.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.016+0000 7fd7992e7640 1 -- 172.21.15.120:0/1938545011 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7941a3f60 con 0x7fd794102ef0 2024-09-19T09:12:46.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.018+0000 7fd790ff9640 1 -- 172.21.15.120:0/1938545011 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd78c037020 con 0x7fd794102ef0 2024-09-19T09:12:46.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.018+0000 7fd79b6c0740 1 Processor -- start 2024-09-19T09:12:46.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.018+0000 7fd79b6c0740 1 -- start start 2024-09-19T09:12:46.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.019+0000 7fd79b6c0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56416e1ea3c0 0x56416e1ea7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:46.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.019+0000 7fd79b6c0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56416e1eace0 0x56416e1f7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:46.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.019+0000 7fd79b6c0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56416e1f7890 0x56416e1f9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:46.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.019+0000 7fd79b6c0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56416e1502e0 con 0x56416e1f7890 2024-09-19T09:12:46.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.019+0000 7fd79b6c0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56416e0f7ce0 con 0x56416e1ea3c0 2024-09-19T09:12:46.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.019+0000 7fd79b6c0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56416e14c2a0 con 0x56416e1eace0 2024-09-19T09:12:46.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.019+0000 7fd78a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56416e1f7890 0x56416e1f9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:46.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.019+0000 7fd78a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56416e1f7890 0x56416e1f9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53336/0 (socket says 172.21.15.120:53336) 2024-09-19T09:12:46.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.019+0000 7fd78a7fc640 1 -- 172.21.15.120:0/219376189 learned_addr learned my addr 172.21.15.120:0/219376189 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:46.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.019+0000 7fd790ff9640 1 --2- 172.21.15.120:0/1938545011 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd7800a32c0 0x7fd7800a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:46.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.019+0000 7fd78a7fc640 1 --2- 172.21.15.120:0/219376189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56416e1ea3c0 0x56416e1ea7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:46.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.019+0000 7fd792ffd640 1 --2- 172.21.15.120:0/219376189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56416e1eace0 0x56416e1f7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:46.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.019+0000 7fd7937fe640 1 --2- 172.21.15.120:0/1938545011 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd7800a32c0 0x7fd7800a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:46.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.019+0000 7fd78a7fc640 1 -- 172.21.15.120:0/219376189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56416e1eace0 msgr2=0x56416e1f7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:46.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.019+0000 7fd78a7fc640 1 --2- 172.21.15.120:0/219376189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56416e1eace0 0x56416e1f7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.019+0000 7fd78a7fc640 1 -- 172.21.15.120:0/219376189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56416e1ea3c0 msgr2=0x56416e1ea7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:46.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.019+0000 7fd78a7fc640 1 --2- 172.21.15.120:0/219376189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56416e1ea3c0 0x56416e1ea7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.019+0000 7fd78a7fc640 1 -- 172.21.15.120:0/219376189 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56416e0dc440 con 0x56416e1f7890 2024-09-19T09:12:46.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.019+0000 7fd790ff9640 1 -- 172.21.15.120:0/1938545011 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fd78c0f07c0 con 0x7fd794102ef0 2024-09-19T09:12:46.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.019+0000 7fd78a7fc640 1 --2- 172.21.15.120:0/219376189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56416e1f7890 0x56416e1f9c80 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fd77c016530 tx=0x7fd77c00a2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c050ec26003f0abe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:46.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.020+0000 7fd7897fa640 1 -- 172.21.15.120:0/219376189 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c018430 con 0x56416e1f7890 2024-09-19T09:12:46.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.020+0000 7fd7897fa640 1 -- 172.21.15.120:0/219376189 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd77c018ab0 con 0x56416e1f7890 2024-09-19T09:12:46.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.020+0000 7fd7897fa640 1 -- 172.21.15.120:0/219376189 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c017a00 con 0x56416e1f7890 2024-09-19T09:12:46.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.020+0000 7fd79b6c0740 1 -- 172.21.15.120:0/219376189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56416e1f7890 msgr2=0x56416e1f9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:46.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.020+0000 7fd7937fe640 1 --2- 172.21.15.120:0/1938545011 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd7800a32c0 0x7fd7800a5780 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd78c0b9240 tx=0x7fd78c08bb20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:46.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.020+0000 7fd79b6c0740 1 --2- 172.21.15.120:0/219376189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56416e1f7890 0x56416e1f9c80 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fd77c016530 tx=0x7fd77c00a2b0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.020+0000 7fd79b6c0740 1 -- 172.21.15.120:0/219376189 shutdown_connections 2024-09-19T09:12:46.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.020+0000 7fd79b6c0740 1 --2- 172.21.15.120:0/219376189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56416e1f7890 0x56416e1f9c80 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.020+0000 7fd79b6c0740 1 --2- 172.21.15.120:0/219376189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56416e1eace0 0x56416e1f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.020+0000 7fd79b6c0740 1 --2- 172.21.15.120:0/219376189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56416e1ea3c0 0x56416e1ea7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.020+0000 7fd79b6c0740 1 -- 172.21.15.120:0/219376189 >> 172.21.15.120:0/219376189 conn(0x56416e1e6a30 msgr2=0x56416e1e6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:46.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.020+0000 7fd79b6c0740 1 -- 172.21.15.120:0/219376189 shutdown_connections 2024-09-19T09:12:46.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.020+0000 7fd79b6c0740 1 -- 172.21.15.120:0/219376189 wait complete. 2024-09-19T09:12:46.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.021+0000 7fd79b6c0740 1 -- start start 2024-09-19T09:12:46.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.021+0000 7fd79b6c0740 1 Processor -- start 2024-09-19T09:12:46.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.021+0000 7fd79b6c0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56416e1ea3c0 0x56416e1f72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:46.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.021+0000 7fd79b6c0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56416e1eace0 0x56416e1f4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:46.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.021+0000 7fd79b6c0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56416e1f7890 0x56416e1f4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:46.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.021+0000 7fd79b6c0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56416e1fa340 con 0x56416e1f7890 2024-09-19T09:12:46.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.021+0000 7fd79b6c0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56416e10b050 con 0x56416e1ea3c0 2024-09-19T09:12:46.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.021+0000 7fd79b6c0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56416e1fa4c0 con 0x56416e1eace0 2024-09-19T09:12:46.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.021+0000 7fd79b6c0740 10 client.0 _ll_register_callbacks cb 0x56416e14b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:46.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.021+0000 7fd78a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56416e1f7890 0x56416e1f4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:46.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.021+0000 7fd78a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56416e1f7890 0x56416e1f4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53340/0 (socket says 172.21.15.120:53340) 2024-09-19T09:12:46.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.021+0000 7fd78a7fc640 1 -- 172.21.15.120:0/2947710732 learned_addr learned my addr 172.21.15.120:0/2947710732 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:46.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.021+0000 7fd78a7fc640 1 --2- 172.21.15.120:0/2947710732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56416e1ea3c0 0x56416e1f72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:46.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.021+0000 7fd792ffd640 1 --2- 172.21.15.120:0/2947710732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56416e1eace0 0x56416e1f4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:46.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.022+0000 7fd78a7fc640 1 -- 172.21.15.120:0/2947710732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56416e1eace0 msgr2=0x56416e1f4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:46.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.022+0000 7fd78a7fc640 1 --2- 172.21.15.120:0/2947710732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56416e1eace0 0x56416e1f4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.022+0000 7fd78a7fc640 1 -- 172.21.15.120:0/2947710732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56416e1ea3c0 msgr2=0x56416e1f72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:46.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.022+0000 7fd78a7fc640 1 --2- 172.21.15.120:0/2947710732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56416e1ea3c0 0x56416e1f72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.022+0000 7fd78a7fc640 1 -- 172.21.15.120:0/2947710732 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56416dec13c0 con 0x56416e1f7890 2024-09-19T09:12:46.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.022+0000 7fd78a7fc640 1 --2- 172.21.15.120:0/2947710732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56416e1f7890 0x56416e1f4b60 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fd77c017720 tx=0x7fd77c017620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f731fc260880739 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:46.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.022+0000 7fd77a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:46.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.022+0000 7fd77a7fc640 1 -- 172.21.15.120:0/2947710732 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c02a060 con 0x56416e1f7890 2024-09-19T09:12:46.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.022+0000 7fd77a7fc640 1 -- 172.21.15.120:0/2947710732 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fd77c0177c0 con 0x56416e1f7890 2024-09-19T09:12:46.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.022+0000 7fd79b6c0740 1 -- 172.21.15.120:0/2947710732 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd77c018ab0 con 0x56416e1f7890 2024-09-19T09:12:46.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.022+0000 7fd77a7fc640 1 -- 172.21.15.120:0/2947710732 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd77c018900 con 0x56416e1f7890 2024-09-19T09:12:46.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.022+0000 7fd79b6c0740 10 client.19071 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:46.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.022+0000 7fd79b6c0740 1 -- 172.21.15.120:0/2947710732 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56416e205e10 con 0x56416e1f7890 2024-09-19T09:12:46.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.022+0000 7fd77a7fc640 1 -- 172.21.15.120:0/2947710732 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd77c018ab0 con 0x56416e1f7890 2024-09-19T09:12:46.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.022+0000 7fd77a7fc640 1 -- 172.21.15.120:0/2947710732 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd77c02f070 con 0x56416e1f7890 2024-09-19T09:12:46.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.022+0000 7fd79b6c0740 10 client.19071 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:46.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.022+0000 7fd79b6c0740 10 client.19071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:12:46.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.022+0000 7fd79b6c0740 20 client.19071 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:46.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.022+0000 7fd79b6c0740 1 --2- 172.21.15.120:0/2947710732 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56416e207210 0x56416e209600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:46.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.022+0000 7fd79b6c0740 4 client.19071 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:12:46.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.022+0000 7fd79b6c0740 1 -- 172.21.15.120:0/2947710732 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56416e15d850 con 0x56416e207210 2024-09-19T09:12:46.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.023+0000 7fd792ffd640 1 --2- 172.21.15.120:0/2947710732 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56416e207210 0x56416e209600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:46.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.023+0000 7fd77a7fc640 10 client.19071 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:46.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.023+0000 7fd792ffd640 1 --2- 172.21.15.120:0/2947710732 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56416e207210 0x56416e209600 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6b55faba065746 server_cookie=6408a279d5a4d279 in_seq=0 out_seq=0 2024-09-19T09:12:46.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.024+0000 7fd77a7fc640 1 -- 172.21.15.120:0/2947710732 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56416e15d850 con 0x56416e207210 2024-09-19T09:12:46.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.024+0000 7fd77a7fc640 10 client.19071 handle_command_reply: tid=0 2024-09-19T09:12:46.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.024+0000 7fd79b6c0740 1 client.19071 shutdown 2024-09-19T09:12:46.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.024+0000 7fd79b6c0740 1 -- 172.21.15.120:0/2947710732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56416e1f7890 msgr2=0x56416e1f4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:46.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.024+0000 7fd79b6c0740 1 --2- 172.21.15.120:0/2947710732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56416e1f7890 0x56416e1f4b60 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fd77c017720 tx=0x7fd77c017620 comp rx=0 tx=0).stop 2024-09-19T09:12:46.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.024+0000 7fd79b6c0740 1 -- 172.21.15.120:0/2947710732 shutdown_connections 2024-09-19T09:12:46.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.024+0000 7fd79b6c0740 1 --2- 172.21.15.120:0/2947710732 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56416e207210 0x56416e209600 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.024+0000 7fd79b6c0740 1 --2- 172.21.15.120:0/2947710732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56416e1f7890 0x56416e1f4b60 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.024+0000 7fd79b6c0740 1 --2- 172.21.15.120:0/2947710732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56416e1eace0 0x56416e1f4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.024+0000 7fd77a7fc640 0 client.19071 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:46.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.024+0000 7fd79b6c0740 1 --2- 172.21.15.120:0/2947710732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56416e1ea3c0 0x56416e1f72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.024+0000 7fd79b6c0740 1 -- 172.21.15.120:0/2947710732 >> 172.21.15.120:0/2947710732 conn(0x56416e1e6a30 msgr2=0x56416e1e5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:46.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.024+0000 7fd79b6c0740 1 -- 172.21.15.120:0/2947710732 shutdown_connections 2024-09-19T09:12:46.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.024+0000 7fd79b6c0740 1 -- 172.21.15.120:0/2947710732 wait complete. 2024-09-19T09:12:46.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.024+0000 7fd79b6c0740 20 client.19071 trim_cache size 0 max 16384 2024-09-19T09:12:46.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.041+0000 7fd79b6c0740 1 Processor -- start 2024-09-19T09:12:46.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.041+0000 7fd79b6c0740 1 -- start start 2024-09-19T09:12:46.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.042+0000 7fd79b6c0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56416e1ea3c0 0x56416e203760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:46.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.042+0000 7fd79b6c0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56416e1eace0 0x56416e20c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:46.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.042+0000 7fd79b6c0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56416e1f7890 0x56416e20cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:46.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.042+0000 7fd79b6c0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd7941ab2e0 con 0x56416e1f7890 2024-09-19T09:12:46.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.042+0000 7fd79b6c0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd7941ace20 con 0x56416e1eace0 2024-09-19T09:12:46.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.042+0000 7fd79b6c0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7941ab9b0 con 0x56416e1ea3c0 2024-09-19T09:12:46.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.042+0000 7fd78a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56416e1f7890 0x56416e20cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:46.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.042+0000 7fd78a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56416e1f7890 0x56416e20cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53346/0 (socket says 172.21.15.120:53346) 2024-09-19T09:12:46.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.042+0000 7fd78a7fc640 1 -- 172.21.15.120:0/4261973946 learned_addr learned my addr 172.21.15.120:0/4261973946 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:46.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.042+0000 7fd792ffd640 1 --2- 172.21.15.120:0/4261973946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56416e1eace0 0x56416e20c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:46.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.042+0000 7fd78a7fc640 1 --2- 172.21.15.120:0/4261973946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56416e1ea3c0 0x56416e203760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:46.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.042+0000 7fd78a7fc640 1 -- 172.21.15.120:0/4261973946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56416e1ea3c0 msgr2=0x56416e203760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:46.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.042+0000 7fd78a7fc640 1 --2- 172.21.15.120:0/4261973946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56416e1ea3c0 0x56416e203760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.042+0000 7fd78a7fc640 1 -- 172.21.15.120:0/4261973946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56416e1eace0 msgr2=0x56416e20c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:46.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.042+0000 7fd78a7fc640 1 --2- 172.21.15.120:0/4261973946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56416e1eace0 0x56416e20c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.042+0000 7fd78a7fc640 1 -- 172.21.15.120:0/4261973946 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56416e1f2aa0 con 0x56416e1f7890 2024-09-19T09:12:46.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.042+0000 7fd792ffd640 1 --2- 172.21.15.120:0/4261973946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56416e1eace0 0x56416e20c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:46.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.042+0000 7fd78a7fc640 1 --2- 172.21.15.120:0/4261973946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56416e1f7890 0x56416e20cb60 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fd77c025a70 tx=0x7fd77c027040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f0e116e8d08b26 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:46.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.042+0000 7fd788ff9640 1 -- 172.21.15.120:0/4261973946 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c035040 con 0x56416e1f7890 2024-09-19T09:12:46.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.042+0000 7fd788ff9640 1 -- 172.21.15.120:0/4261973946 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd77c036040 con 0x56416e1f7890 2024-09-19T09:12:46.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.042+0000 7fd788ff9640 1 -- 172.21.15.120:0/4261973946 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c0189f0 con 0x56416e1f7890 2024-09-19T09:12:46.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.043+0000 7fd79b6c0740 1 -- 172.21.15.120:0/4261973946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56416e1f7890 msgr2=0x56416e20cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:46.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.043+0000 7fd79b6c0740 1 --2- 172.21.15.120:0/4261973946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56416e1f7890 0x56416e20cb60 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fd77c025a70 tx=0x7fd77c027040 comp rx=0 tx=0).stop 2024-09-19T09:12:46.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.043+0000 7fd79b6c0740 1 -- 172.21.15.120:0/4261973946 shutdown_connections 2024-09-19T09:12:46.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.043+0000 7fd79b6c0740 1 --2- 172.21.15.120:0/4261973946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56416e1f7890 0x56416e20cb60 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.043+0000 7fd79b6c0740 1 --2- 172.21.15.120:0/4261973946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56416e1eace0 0x56416e20c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.043+0000 7fd79b6c0740 1 --2- 172.21.15.120:0/4261973946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56416e1ea3c0 0x56416e203760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.043+0000 7fd79b6c0740 1 -- 172.21.15.120:0/4261973946 >> 172.21.15.120:0/4261973946 conn(0x56416e1e6a30 msgr2=0x56416e1efb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:46.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.043+0000 7fd79b6c0740 1 -- 172.21.15.120:0/4261973946 shutdown_connections 2024-09-19T09:12:46.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.043+0000 7fd79b6c0740 1 -- 172.21.15.120:0/4261973946 wait complete. 2024-09-19T09:12:46.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.044+0000 7fd79b6c0740 1 -- start start 2024-09-19T09:12:46.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.044+0000 7fd79b6c0740 1 Processor -- start 2024-09-19T09:12:46.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.044+0000 7fd79b6c0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56416e1ea3c0 0x56416e20c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:46.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.044+0000 7fd79b6c0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56416e1eace0 0x56416e20cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:46.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.044+0000 7fd79b6c0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56416e1f7890 0x56416e1ed210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:46.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.044+0000 7fd79b6c0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd7941ac750 con 0x56416e1f7890 2024-09-19T09:12:46.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.044+0000 7fd79b6c0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd7941ac080 con 0x56416e1eace0 2024-09-19T09:12:46.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.044+0000 7fd79b6c0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7941a92a0 con 0x56416e1ea3c0 2024-09-19T09:12:46.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.044+0000 7fd79b6c0740 10 client.0 _ll_register_callbacks cb 0x7fd77c00ac60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:46.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.044+0000 7fd78a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56416e1f7890 0x56416e1ed210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:46.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.044+0000 7fd78a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56416e1f7890 0x56416e1ed210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53348/0 (socket says 172.21.15.120:53348) 2024-09-19T09:12:46.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.044+0000 7fd78a7fc640 1 -- 172.21.15.120:0/2883015272 learned_addr learned my addr 172.21.15.120:0/2883015272 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:46.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.044+0000 7fd792ffd640 1 --2- 172.21.15.120:0/2883015272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56416e1eace0 0x56416e20cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:46.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.044+0000 7fd78a7fc640 1 --2- 172.21.15.120:0/2883015272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56416e1ea3c0 0x56416e20c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:46.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.045+0000 7fd792ffd640 1 -- 172.21.15.120:0/2883015272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56416e1ea3c0 msgr2=0x56416e20c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:46.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.045+0000 7fd792ffd640 1 --2- 172.21.15.120:0/2883015272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56416e1ea3c0 0x56416e20c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.045+0000 7fd792ffd640 1 -- 172.21.15.120:0/2883015272 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56416e1f7890 msgr2=0x56416e1ed210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:46.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.045+0000 7fd792ffd640 1 --2- 172.21.15.120:0/2883015272 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56416e1f7890 0x56416e1ed210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.045+0000 7fd792ffd640 1 -- 172.21.15.120:0/2883015272 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56416e20d160 con 0x56416e1eace0 2024-09-19T09:12:46.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.045+0000 7fd78a7fc640 1 --2- 172.21.15.120:0/2883015272 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56416e1f7890 0x56416e1ed210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:12:46.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.045+0000 7fd792ffd640 1 --2- 172.21.15.120:0/2883015272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56416e1eace0 0x56416e20cb70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x56416e1fa610 tx=0x7fd78400be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8e3656d2ea4d2ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:46.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.045+0000 7fd77a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:12:46.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.045+0000 7fd77a7fc640 1 -- 172.21.15.120:0/2883015272 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78401d380 con 0x56416e1eace0 2024-09-19T09:12:46.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.045+0000 7fd79b6c0740 1 -- 172.21.15.120:0/2883015272 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd77c036040 con 0x56416e1eace0 2024-09-19T09:12:46.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.045+0000 7fd77a7fc640 1 -- 172.21.15.120:0/2883015272 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fd78401d520 con 0x56416e1eace0 2024-09-19T09:12:46.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.046+0000 7fd77a7fc640 1 -- 172.21.15.120:0/2883015272 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd78401d720 con 0x56416e1eace0 2024-09-19T09:12:46.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.046+0000 7fd79b6c0740 10 client.28202 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:46.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.046+0000 7fd79b6c0740 1 -- 172.21.15.120:0/2883015272 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56416e274490 con 0x56416e1eace0 2024-09-19T09:12:46.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.046+0000 7fd77a7fc640 1 -- 172.21.15.120:0/2883015272 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd77c036040 con 0x56416e1eace0 2024-09-19T09:12:46.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.046+0000 7fd77a7fc640 1 -- 172.21.15.120:0/2883015272 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd784026be0 con 0x56416e1eace0 2024-09-19T09:12:46.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.046+0000 7fd79b6c0740 10 client.28202 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:46.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.046+0000 7fd79b6c0740 10 client.28202 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:12:46.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.046+0000 7fd79b6c0740 20 client.28202 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:46.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.046+0000 7fd79b6c0740 1 --2- 172.21.15.120:0/2883015272 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56416e274b60 0x56416e276f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:46.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.046+0000 7fd79b6c0740 4 client.28202 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:12:46.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.046+0000 7fd79b6c0740 1 -- 172.21.15.120:0/2883015272 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56416e157d60 con 0x56416e274b60 2024-09-19T09:12:46.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.046+0000 7fd78a7fc640 1 --2- 172.21.15.120:0/2883015272 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56416e274b60 0x56416e276f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:46.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.047+0000 7fd77a7fc640 10 client.28202 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:46.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.047+0000 7fd78a7fc640 1 --2- 172.21.15.120:0/2883015272 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56416e274b60 0x56416e276f50 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=636523de7c98e804 server_cookie=ab71066c64f7dc5 in_seq=0 out_seq=0 2024-09-19T09:12:46.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.047+0000 7fd77a7fc640 1 -- 172.21.15.120:0/2883015272 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56416e157d60 con 0x56416e274b60 2024-09-19T09:12:46.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.047+0000 7fd77a7fc640 10 client.28202 handle_command_reply: tid=0 2024-09-19T09:12:46.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.047+0000 7fd79b6c0740 1 client.28202 shutdown 2024-09-19T09:12:46.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.047+0000 7fd79b6c0740 1 -- 172.21.15.120:0/2883015272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56416e1eace0 msgr2=0x56416e20cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:46.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.047+0000 7fd79b6c0740 1 --2- 172.21.15.120:0/2883015272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56416e1eace0 0x56416e20cb70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x56416e1fa610 tx=0x7fd78400be20 comp rx=0 tx=0).stop 2024-09-19T09:12:46.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.047+0000 7fd79b6c0740 1 -- 172.21.15.120:0/2883015272 shutdown_connections 2024-09-19T09:12:46.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.047+0000 7fd79b6c0740 1 --2- 172.21.15.120:0/2883015272 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56416e274b60 0x56416e276f50 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.047+0000 7fd79b6c0740 1 --2- 172.21.15.120:0/2883015272 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56416e1f7890 0x56416e1ed210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.047+0000 7fd79b6c0740 1 --2- 172.21.15.120:0/2883015272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56416e1eace0 0x56416e20cb70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.047+0000 7fd77a7fc640 0 client.28202 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:46.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.047+0000 7fd79b6c0740 1 --2- 172.21.15.120:0/2883015272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56416e1ea3c0 0x56416e20c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.047+0000 7fd79b6c0740 1 -- 172.21.15.120:0/2883015272 >> 172.21.15.120:0/2883015272 conn(0x56416e1e6a30 msgr2=0x56416e1e5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:46.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.047+0000 7fd79b6c0740 1 -- 172.21.15.120:0/2883015272 shutdown_connections 2024-09-19T09:12:46.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.047+0000 7fd79b6c0740 1 -- 172.21.15.120:0/2883015272 wait complete. 2024-09-19T09:12:46.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.047+0000 7fd79b6c0740 20 client.28202 trim_cache size 0 max 16384 2024-09-19T09:12:46.055 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:12:46.055 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:12:46.055 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-19T09:12:46.055 INFO:teuthology.orchestra.run.smithi120.stdout: "f8ec7f9f-6aef-4c34-85ef-1ee9f90ed97e": { 2024-09-19T09:12:46.055 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-19T09:12:46.056 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "f8ec7f9f-6aef-4c34-85ef-1ee9f90ed97e", 2024-09-19T09:12:46.056 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-19T09:12:46.056 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:12:46.056 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:12:46.056 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:12:46.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.048+0000 7fd7992e7640 1 -- 172.21.15.120:0/1938545011 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd7800a32c0 msgr2=0x7fd7800a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:46.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.048+0000 7fd7992e7640 1 --2- 172.21.15.120:0/1938545011 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd7800a32c0 0x7fd7800a5780 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd78c0b9240 tx=0x7fd78c08bb20 comp rx=0 tx=0).stop 2024-09-19T09:12:46.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.048+0000 7fd7992e7640 1 -- 172.21.15.120:0/1938545011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd794102ef0 msgr2=0x7fd79419bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:46.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.048+0000 7fd7992e7640 1 --2- 172.21.15.120:0/1938545011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd794102ef0 0x7fd79419bcb0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fd78c02ae70 tx=0x7fd78c02f5d0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.048+0000 7fd7992e7640 1 -- 172.21.15.120:0/1938545011 shutdown_connections 2024-09-19T09:12:46.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.048+0000 7fd7992e7640 1 --2- 172.21.15.120:0/1938545011 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd7800a32c0 0x7fd7800a5780 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.048+0000 7fd7992e7640 1 --2- 172.21.15.120:0/1938545011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd794102ef0 0x7fd79419bcb0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.048+0000 7fd7992e7640 1 --2- 172.21.15.120:0/1938545011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd794102570 0x7fd794197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.048+0000 7fd7992e7640 1 --2- 172.21.15.120:0/1938545011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd794100e20 0x7fd7941973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:46.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.048+0000 7fd7992e7640 1 -- 172.21.15.120:0/1938545011 >> 172.21.15.120:0/1938545011 conn(0x7fd7940fc7d0 msgr2=0x7fd794106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:46.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.048+0000 7fd7992e7640 1 -- 172.21.15.120:0/1938545011 shutdown_connections 2024-09-19T09:12:46.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:12:46.048+0000 7fd7992e7640 1 -- 172.21.15.120:0/1938545011 wait complete. 2024-09-19T09:12:46.067 INFO:tasks.workunit.client.0.smithi120.stdout: 1 984176 10.26 MB/sec execute 401 sec latency 696.409 ms 2024-09-19T09:12:46.643 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b8a09801', quiesce took 6.1 seconds. Will release after: 23.9 2024-09-19T09:12:47.018 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:46 smithi120 ceph-mon[31150]: pgmap v640: 97 pgs: 97 active+clean; 33 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 3.3 MiB/s wr, 338 op/s 2024-09-19T09:12:47.067 INFO:tasks.workunit.client.0.smithi120.stdout: 1 984176 10.23 MB/sec execute 402 sec latency 1696.490 ms 2024-09-19T09:12:47.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:46 smithi196 ceph-mon[35150]: pgmap v640: 97 pgs: 97 active+clean; 33 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 3.3 MiB/s wr, 338 op/s 2024-09-19T09:12:47.190 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:46 smithi153 ceph-mon[35071]: pgmap v640: 97 pgs: 97 active+clean; 33 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 3.3 MiB/s wr, 338 op/s 2024-09-19T09:12:47.371 INFO:tasks.cephfs.filesystem:scrub status for tag:f8ec7f9f-6aef-4c34-85ef-1ee9f90ed97e - {'path': '/', 'tag': 'f8ec7f9f-6aef-4c34-85ef-1ee9f90ed97e', 'options': 'recursive,force'} 2024-09-19T09:12:48.067 INFO:tasks.workunit.client.0.smithi120.stdout: 1 984176 10.21 MB/sec execute 403 sec latency 2696.597 ms 2024-09-19T09:12:49.067 INFO:tasks.workunit.client.0.smithi120.stdout: 1 984176 10.18 MB/sec execute 404 sec latency 3696.703 ms 2024-09-19T09:12:49.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:48 smithi196 ceph-mon[35150]: pgmap v641: 97 pgs: 97 active+clean; 30 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 3.7 MiB/s wr, 389 op/s 2024-09-19T09:12:49.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:48 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:12:49.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:48 smithi153 ceph-mon[35071]: pgmap v641: 97 pgs: 97 active+clean; 30 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 3.7 MiB/s wr, 389 op/s 2024-09-19T09:12:49.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:48 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:12:49.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:48 smithi120 ceph-mon[31150]: pgmap v641: 97 pgs: 97 active+clean; 30 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 3.7 MiB/s wr, 389 op/s 2024-09-19T09:12:49.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:48 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:12:50.067 INFO:tasks.workunit.client.0.smithi120.stdout: 1 984176 10.16 MB/sec execute 405 sec latency 4696.803 ms 2024-09-19T09:12:51.067 INFO:tasks.workunit.client.0.smithi120.stdout: 1 984176 10.13 MB/sec execute 406 sec latency 5696.840 ms 2024-09-19T09:12:51.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:50 smithi196 ceph-mon[35150]: pgmap v642: 97 pgs: 97 active+clean; 29 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 4.1 MiB/s wr, 398 op/s 2024-09-19T09:12:51.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:50 smithi153 ceph-mon[35071]: pgmap v642: 97 pgs: 97 active+clean; 29 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 4.1 MiB/s wr, 398 op/s 2024-09-19T09:12:51.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:50 smithi120 ceph-mon[31150]: pgmap v642: 97 pgs: 97 active+clean; 29 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 4.1 MiB/s wr, 398 op/s 2024-09-19T09:12:52.067 INFO:tasks.workunit.client.0.smithi120.stdout: 1 984176 10.11 MB/sec execute 407 sec latency 6696.894 ms 2024-09-19T09:12:53.067 INFO:tasks.workunit.client.0.smithi120.stdout: 1 984176 10.08 MB/sec execute 408 sec latency 7696.978 ms 2024-09-19T09:12:53.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:52 smithi196 ceph-mon[35150]: pgmap v643: 97 pgs: 97 active+clean; 28 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 3.6 MiB/s wr, 333 op/s 2024-09-19T09:12:53.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:52 smithi153 ceph-mon[35071]: pgmap v643: 97 pgs: 97 active+clean; 28 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 3.6 MiB/s wr, 333 op/s 2024-09-19T09:12:53.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:52 smithi120 ceph-mon[31150]: pgmap v643: 97 pgs: 97 active+clean; 28 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 3.6 MiB/s wr, 333 op/s 2024-09-19T09:12:54.068 INFO:tasks.workunit.client.0.smithi120.stdout: 1 984176 10.06 MB/sec execute 409 sec latency 8697.101 ms 2024-09-19T09:12:55.068 INFO:tasks.workunit.client.0.smithi120.stdout: 1 984176 10.03 MB/sec execute 410 sec latency 9697.149 ms 2024-09-19T09:12:55.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:54 smithi196 ceph-mon[35150]: pgmap v644: 97 pgs: 97 active+clean; 34 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 4.1 MiB/s wr, 332 op/s 2024-09-19T09:12:55.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:54 smithi153 ceph-mon[35071]: pgmap v644: 97 pgs: 97 active+clean; 34 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 4.1 MiB/s wr, 332 op/s 2024-09-19T09:12:55.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:54 smithi120 ceph-mon[31150]: pgmap v644: 97 pgs: 97 active+clean; 34 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 4.1 MiB/s wr, 332 op/s 2024-09-19T09:12:56.068 INFO:tasks.workunit.client.0.smithi120.stdout: 1 984176 10.01 MB/sec execute 411 sec latency 10697.255 ms 2024-09-19T09:12:57.068 INFO:tasks.workunit.client.0.smithi120.stdout: 1 984176 9.99 MB/sec execute 412 sec latency 11697.315 ms 2024-09-19T09:12:57.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:56 smithi196 ceph-mon[35150]: pgmap v645: 97 pgs: 97 active+clean; 36 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 3.7 MiB/s wr, 291 op/s 2024-09-19T09:12:57.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:56 smithi153 ceph-mon[35071]: pgmap v645: 97 pgs: 97 active+clean; 36 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 3.7 MiB/s wr, 291 op/s 2024-09-19T09:12:57.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:56 smithi120 ceph-mon[31150]: pgmap v645: 97 pgs: 97 active+clean; 36 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 3.7 MiB/s wr, 291 op/s 2024-09-19T09:12:58.068 INFO:tasks.workunit.client.0.smithi120.stdout: 1 984176 9.96 MB/sec execute 413 sec latency 12697.415 ms 2024-09-19T09:12:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.322+0000 7fb16ef59640 1 -- 172.21.15.120:0/3593963028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb168100a40 msgr2=0x7fb168100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.322+0000 7fb16ef59640 1 --2- 172.21.15.120:0/3593963028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb168100a40 0x7fb168100e40 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fb15c009900 tx=0x7fb15c02a410 comp rx=0 tx=0).stop 2024-09-19T09:12:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.322+0000 7fb16ef59640 1 -- 172.21.15.120:0/3593963028 shutdown_connections 2024-09-19T09:12:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.322+0000 7fb16ef59640 1 --2- 172.21.15.120:0/3593963028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1681025f0 0x7fb168108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.322+0000 7fb16ef59640 1 --2- 172.21.15.120:0/3593963028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb168101c30 0x7fb1681020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.322+0000 7fb16ef59640 1 --2- 172.21.15.120:0/3593963028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb168100a40 0x7fb168100e40 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.322+0000 7fb16ef59640 1 -- 172.21.15.120:0/3593963028 >> 172.21.15.120:0/3593963028 conn(0x7fb1680fc1e0 msgr2=0x7fb1680fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.322+0000 7fb16ef59640 1 -- 172.21.15.120:0/3593963028 shutdown_connections 2024-09-19T09:12:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.322+0000 7fb16ef59640 1 -- 172.21.15.120:0/3593963028 wait complete. 2024-09-19T09:12:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.323+0000 7fb16ef59640 1 Processor -- start 2024-09-19T09:12:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.323+0000 7fb16ef59640 1 -- start start 2024-09-19T09:12:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.323+0000 7fb16ef59640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb168100a40 0x7fb168197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.323+0000 7fb16ef59640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb168101c30 0x7fb168198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.323+0000 7fb16ef59640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1681025f0 0x7fb16819f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.323+0000 7fb16ef59640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb16810c6c0 con 0x7fb168101c30 2024-09-19T09:12:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.323+0000 7fb16ef59640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb16810c540 con 0x7fb1681025f0 2024-09-19T09:12:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.323+0000 7fb16ef59640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb16810c840 con 0x7fb168100a40 2024-09-19T09:12:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.323+0000 7fb167fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb168101c30 0x7fb168198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.323+0000 7fb167fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb168101c30 0x7fb168198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56948/0 (socket says 172.21.15.120:56948) 2024-09-19T09:12:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.323+0000 7fb167fff640 1 -- 172.21.15.120:0/3520352913 learned_addr learned my addr 172.21.15.120:0/3520352913 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.323+0000 7fb16ccf5640 1 --2- 172.21.15.120:0/3520352913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb168100a40 0x7fb168197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.323+0000 7fb16d4f6640 1 --2- 172.21.15.120:0/3520352913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1681025f0 0x7fb16819f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.323+0000 7fb167fff640 1 -- 172.21.15.120:0/3520352913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb168100a40 msgr2=0x7fb168197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.323+0000 7fb167fff640 1 --2- 172.21.15.120:0/3520352913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb168100a40 0x7fb168197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.323+0000 7fb167fff640 1 -- 172.21.15.120:0/3520352913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1681025f0 msgr2=0x7fb16819f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.323+0000 7fb167fff640 1 --2- 172.21.15.120:0/3520352913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1681025f0 0x7fb16819f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.323+0000 7fb167fff640 1 -- 172.21.15.120:0/3520352913 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16819fa10 con 0x7fb168101c30 2024-09-19T09:12:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.323+0000 7fb167fff640 1 --2- 172.21.15.120:0/3520352913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb168101c30 0x7fb168198330 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fb15400d770 tx=0x7fb15400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.324+0000 7fb165ffb640 1 -- 172.21.15.120:0/3520352913 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154017070 con 0x7fb168101c30 2024-09-19T09:12:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.324+0000 7fb165ffb640 1 -- 172.21.15.120:0/3520352913 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb154005500 con 0x7fb168101c30 2024-09-19T09:12:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.324+0000 7fb16ef59640 1 -- 172.21.15.120:0/3520352913 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb16819fd00 con 0x7fb168101c30 2024-09-19T09:12:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.324+0000 7fb165ffb640 1 -- 172.21.15.120:0/3520352913 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154004e00 con 0x7fb168101c30 2024-09-19T09:12:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.324+0000 7fb16ef59640 1 -- 172.21.15.120:0/3520352913 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1681a01f0 con 0x7fb168101c30 2024-09-19T09:12:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.326+0000 7fb165ffb640 1 -- 172.21.15.120:0/3520352913 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fb154004fa0 con 0x7fb168101c30 2024-09-19T09:12:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.326+0000 7fb171dd0740 1 Processor -- start 2024-09-19T09:12:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.326+0000 7fb171dd0740 1 -- start start 2024-09-19T09:12:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.327+0000 7fb171dd0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8926429a0 0x55f892642da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.327+0000 7fb171dd0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f8926432e0 0x55f892647da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.327+0000 7fb165ffb640 1 --2- 172.21.15.120:0/3520352913 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb1580a3370 0x7fb1580a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.327+0000 7fb171dd0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f892648370 0x55f89264a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.327+0000 7fb171dd0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f8925a62f0 con 0x55f8926432e0 2024-09-19T09:12:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.327+0000 7fb171dd0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f89254e270 con 0x55f8926429a0 2024-09-19T09:12:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.327+0000 7fb171dd0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f8925a22b0 con 0x55f892648370 2024-09-19T09:12:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.327+0000 7fb16ccf5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f8926432e0 0x55f892647da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.327+0000 7fb16ccf5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f8926432e0 0x55f892647da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56950/0 (socket says 172.21.15.120:56950) 2024-09-19T09:12:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.327+0000 7fb16ccf5640 1 -- 172.21.15.120:0/1350325193 learned_addr learned my addr 172.21.15.120:0/1350325193 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.327+0000 7fb167fff640 1 --2- 172.21.15.120:0/3520352913 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb1580a3370 0x7fb1580a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.327+0000 7fb16d4f6640 1 --2- 172.21.15.120:0/1350325193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8926429a0 0x55f892642da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.327+0000 7fb165ffb640 1 -- 172.21.15.120:0/3520352913 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb1540cf320 con 0x7fb168101c30 2024-09-19T09:12:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.327+0000 7fb16d4f6640 1 --2- 172.21.15.120:0/1350325193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f892648370 0x55f89264a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.327+0000 7fb16ccf5640 1 -- 172.21.15.120:0/1350325193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f892648370 msgr2=0x55f89264a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.327+0000 7fb16ccf5640 1 --2- 172.21.15.120:0/1350325193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f892648370 0x55f89264a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.327+0000 7fb16ccf5640 1 -- 172.21.15.120:0/1350325193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8926429a0 msgr2=0x55f892642da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.327+0000 7fb16ccf5640 1 --2- 172.21.15.120:0/1350325193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8926429a0 0x55f892642da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.327+0000 7fb16ccf5640 1 -- 172.21.15.120:0/1350325193 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f89254d340 con 0x55f8926432e0 2024-09-19T09:12:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.327+0000 7fb16d4f6640 1 --2- 172.21.15.120:0/1350325193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f892648370 0x55f89264a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:12:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.327+0000 7fb16ccf5640 1 --2- 172.21.15.120:0/1350325193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f8926432e0 0x55f892647da0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fb15c031610 tx=0x7fb15c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567d236b094ee73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.327+0000 7fb167fff640 1 --2- 172.21.15.120:0/3520352913 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb1580a3370 0x7fb1580a5830 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb15400d740 tx=0x7fb154090e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.328+0000 7fb152ffd640 1 -- 172.21.15.120:0/1350325193 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c040070 con 0x55f8926432e0 2024-09-19T09:12:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.328+0000 7fb152ffd640 1 -- 172.21.15.120:0/1350325193 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb15c0318b0 con 0x55f8926432e0 2024-09-19T09:12:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.328+0000 7fb152ffd640 1 -- 172.21.15.120:0/1350325193 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c035040 con 0x55f8926432e0 2024-09-19T09:12:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.328+0000 7fb171dd0740 1 -- 172.21.15.120:0/1350325193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f8926432e0 msgr2=0x55f892647da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.328+0000 7fb171dd0740 1 --2- 172.21.15.120:0/1350325193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f8926432e0 0x55f892647da0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fb15c031610 tx=0x7fb15c005c00 comp rx=0 tx=0).stop 2024-09-19T09:12:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.328+0000 7fb171dd0740 1 -- 172.21.15.120:0/1350325193 shutdown_connections 2024-09-19T09:12:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.328+0000 7fb171dd0740 1 --2- 172.21.15.120:0/1350325193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f892648370 0x55f89264a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.328+0000 7fb171dd0740 1 --2- 172.21.15.120:0/1350325193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f8926432e0 0x55f892647da0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.328+0000 7fb171dd0740 1 --2- 172.21.15.120:0/1350325193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8926429a0 0x55f892642da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.328+0000 7fb171dd0740 1 -- 172.21.15.120:0/1350325193 >> 172.21.15.120:0/1350325193 conn(0x55f89263e010 msgr2=0x55f89263e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.328+0000 7fb171dd0740 1 -- 172.21.15.120:0/1350325193 shutdown_connections 2024-09-19T09:12:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.328+0000 7fb171dd0740 1 -- 172.21.15.120:0/1350325193 wait complete. 2024-09-19T09:12:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.329+0000 7fb171dd0740 1 -- start start 2024-09-19T09:12:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.329+0000 7fb171dd0740 1 Processor -- start 2024-09-19T09:12:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.329+0000 7fb171dd0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8926429a0 0x55f89264cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.329+0000 7fb171dd0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8926432e0 0x55f89264d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.329+0000 7fb171dd0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f892648370 0x55f8926540b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.329+0000 7fb171dd0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f89264b2e0 con 0x55f892648370 2024-09-19T09:12:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.329+0000 7fb171dd0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f89252f290 con 0x55f8926432e0 2024-09-19T09:12:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.329+0000 7fb171dd0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f89264b460 con 0x55f8926429a0 2024-09-19T09:12:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.329+0000 7fb171dd0740 10 client.0 _ll_register_callbacks cb 0x55f892560140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.329+0000 7fb16d4f6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f892648370 0x55f8926540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.329+0000 7fb16d4f6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f892648370 0x55f8926540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56960/0 (socket says 172.21.15.120:56960) 2024-09-19T09:12:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.329+0000 7fb16d4f6640 1 -- 172.21.15.120:0/1603759720 learned_addr learned my addr 172.21.15.120:0/1603759720 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.330+0000 7fb16ccf5640 1 --2- 172.21.15.120:0/1603759720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8926432e0 0x55f89264d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.330+0000 7fb16d4f6640 1 --2- 172.21.15.120:0/1603759720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8926429a0 0x55f89264cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.330+0000 7fb16d4f6640 1 -- 172.21.15.120:0/1603759720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8926429a0 msgr2=0x55f89264cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.330+0000 7fb16d4f6640 1 --2- 172.21.15.120:0/1603759720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8926429a0 0x55f89264cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.330+0000 7fb16d4f6640 1 -- 172.21.15.120:0/1603759720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8926432e0 msgr2=0x55f89264d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.330+0000 7fb16d4f6640 1 --2- 172.21.15.120:0/1603759720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8926432e0 0x55f89264d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.330+0000 7fb16d4f6640 1 -- 172.21.15.120:0/1603759720 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8922b9210 con 0x55f892648370 2024-09-19T09:12:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.330+0000 7fb16d4f6640 1 --2- 172.21.15.120:0/1603759720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f892648370 0x55f8926540b0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb160013ba0 tx=0x7fb160009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee91ed5fa2e4378 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.330+0000 7fb147fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.330+0000 7fb147fff640 1 -- 172.21.15.120:0/1603759720 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16000f600 con 0x55f892648370 2024-09-19T09:12:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.330+0000 7fb147fff640 1 -- 172.21.15.120:0/1603759720 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb16000f7a0 con 0x55f892648370 2024-09-19T09:12:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.330+0000 7fb171dd0740 1 -- 172.21.15.120:0/1603759720 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb15c0318b0 con 0x55f892648370 2024-09-19T09:12:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.330+0000 7fb147fff640 1 -- 172.21.15.120:0/1603759720 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb16000f9a0 con 0x55f892648370 2024-09-19T09:12:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.330+0000 7fb171dd0740 10 client.19089 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.330+0000 7fb171dd0740 1 -- 172.21.15.120:0/1603759720 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f892385620 con 0x55f892648370 2024-09-19T09:12:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.330+0000 7fb147fff640 1 -- 172.21.15.120:0/1603759720 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb15c0318b0 con 0x55f892648370 2024-09-19T09:12:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.331+0000 7fb147fff640 1 -- 172.21.15.120:0/1603759720 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb16000c070 con 0x55f892648370 2024-09-19T09:12:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.331+0000 7fb171dd0740 10 client.19089 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.331+0000 7fb171dd0740 10 client.19089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:12:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.331+0000 7fb171dd0740 20 client.19089 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.331+0000 7fb171dd0740 1 --2- 172.21.15.120:0/1603759720 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f8926587c0 0x55f89265ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.331+0000 7fb171dd0740 4 client.19089 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:12:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.331+0000 7fb171dd0740 1 -- 172.21.15.120:0/1603759720 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8925b3a20 con 0x55f8926587c0 2024-09-19T09:12:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.331+0000 7fb16ccf5640 1 --2- 172.21.15.120:0/1603759720 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f8926587c0 0x55f89265ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.331+0000 7fb147fff640 10 client.19089 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.331+0000 7fb16ccf5640 1 --2- 172.21.15.120:0/1603759720 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f8926587c0 0x55f89265ac60 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5498110c44afb07c server_cookie=131c5f750b0ccc13 in_seq=0 out_seq=0 2024-09-19T09:12:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.332+0000 7fb147fff640 1 -- 172.21.15.120:0/1603759720 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f8925b3a20 con 0x55f8926587c0 2024-09-19T09:12:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.332+0000 7fb147fff640 10 client.19089 handle_command_reply: tid=0 2024-09-19T09:12:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.332+0000 7fb171dd0740 1 client.19089 shutdown 2024-09-19T09:12:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.332+0000 7fb171dd0740 1 -- 172.21.15.120:0/1603759720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f892648370 msgr2=0x55f8926540b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.332+0000 7fb171dd0740 1 --2- 172.21.15.120:0/1603759720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f892648370 0x55f8926540b0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb160013ba0 tx=0x7fb160009f90 comp rx=0 tx=0).stop 2024-09-19T09:12:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.332+0000 7fb171dd0740 1 -- 172.21.15.120:0/1603759720 shutdown_connections 2024-09-19T09:12:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.332+0000 7fb171dd0740 1 --2- 172.21.15.120:0/1603759720 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f8926587c0 0x55f89265ac60 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.332+0000 7fb171dd0740 1 --2- 172.21.15.120:0/1603759720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f892648370 0x55f8926540b0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.332+0000 7fb171dd0740 1 --2- 172.21.15.120:0/1603759720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8926432e0 0x55f89264d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.332+0000 7fb147fff640 0 client.19089 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.332+0000 7fb171dd0740 1 --2- 172.21.15.120:0/1603759720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8926429a0 0x55f89264cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.332+0000 7fb171dd0740 1 -- 172.21.15.120:0/1603759720 >> 172.21.15.120:0/1603759720 conn(0x55f89263e010 msgr2=0x55f89263f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.332+0000 7fb171dd0740 1 -- 172.21.15.120:0/1603759720 shutdown_connections 2024-09-19T09:12:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.332+0000 7fb171dd0740 1 -- 172.21.15.120:0/1603759720 wait complete. 2024-09-19T09:12:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.332+0000 7fb171dd0740 20 client.19089 trim_cache size 0 max 16384 2024-09-19T09:12:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.346+0000 7fb171dd0740 1 Processor -- start 2024-09-19T09:12:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.347+0000 7fb171dd0740 1 -- start start 2024-09-19T09:12:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.347+0000 7fb171dd0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8926429a0 0x55f89263eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.347+0000 7fb171dd0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8926432e0 0x55f89263f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.347+0000 7fb171dd0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f892648370 0x55f89263f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.347+0000 7fb171dd0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb1681a8f50 con 0x55f892648370 2024-09-19T09:12:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.347+0000 7fb171dd0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb1681aaa90 con 0x55f8926429a0 2024-09-19T09:12:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.347+0000 7fb171dd0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb1681a9620 con 0x55f8926432e0 2024-09-19T09:12:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.347+0000 7fb16d4f6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f892648370 0x55f89263f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.347+0000 7fb16d4f6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f892648370 0x55f89263f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56976/0 (socket says 172.21.15.120:56976) 2024-09-19T09:12:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.347+0000 7fb16d4f6640 1 -- 172.21.15.120:0/950917854 learned_addr learned my addr 172.21.15.120:0/950917854 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.347+0000 7fb16d4f6640 1 --2- 172.21.15.120:0/950917854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8926429a0 0x55f89263eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.347+0000 7fb16ccf5640 1 --2- 172.21.15.120:0/950917854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8926432e0 0x55f89263f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.347+0000 7fb16d4f6640 1 -- 172.21.15.120:0/950917854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8926432e0 msgr2=0x55f89263f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.347+0000 7fb16d4f6640 1 --2- 172.21.15.120:0/950917854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8926432e0 0x55f89263f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.347+0000 7fb16d4f6640 1 -- 172.21.15.120:0/950917854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8926429a0 msgr2=0x55f89263eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.347+0000 7fb16d4f6640 1 --2- 172.21.15.120:0/950917854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8926429a0 0x55f89263eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.347+0000 7fb16d4f6640 1 -- 172.21.15.120:0/950917854 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f89264b520 con 0x55f892648370 2024-09-19T09:12:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.347+0000 7fb16ccf5640 1 --2- 172.21.15.120:0/950917854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8926432e0 0x55f89263f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:12:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.347+0000 7fb16d4f6640 1 --2- 172.21.15.120:0/950917854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f892648370 0x55f89263f940 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb160013720 tx=0x7fb1600021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b830cd593732f27f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.347+0000 7fb1527fc640 1 -- 172.21.15.120:0/950917854 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16002e030 con 0x55f892648370 2024-09-19T09:12:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.348+0000 7fb1527fc640 1 -- 172.21.15.120:0/950917854 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb16000f600 con 0x55f892648370 2024-09-19T09:12:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.348+0000 7fb1527fc640 1 -- 172.21.15.120:0/950917854 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16000fd80 con 0x55f892648370 2024-09-19T09:12:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.348+0000 7fb171dd0740 1 -- 172.21.15.120:0/950917854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f892648370 msgr2=0x55f89263f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.348+0000 7fb171dd0740 1 --2- 172.21.15.120:0/950917854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f892648370 0x55f89263f940 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb160013720 tx=0x7fb1600021d0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.348+0000 7fb171dd0740 1 -- 172.21.15.120:0/950917854 shutdown_connections 2024-09-19T09:12:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.348+0000 7fb171dd0740 1 --2- 172.21.15.120:0/950917854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f892648370 0x55f89263f940 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.348+0000 7fb171dd0740 1 --2- 172.21.15.120:0/950917854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8926432e0 0x55f89263f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.348+0000 7fb171dd0740 1 --2- 172.21.15.120:0/950917854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8926429a0 0x55f89263eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.348+0000 7fb171dd0740 1 -- 172.21.15.120:0/950917854 >> 172.21.15.120:0/950917854 conn(0x55f89263e010 msgr2=0x55f89264cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.348+0000 7fb171dd0740 1 -- 172.21.15.120:0/950917854 shutdown_connections 2024-09-19T09:12:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.348+0000 7fb171dd0740 1 -- 172.21.15.120:0/950917854 wait complete. 2024-09-19T09:12:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.348+0000 7fb171dd0740 1 -- start start 2024-09-19T09:12:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.349+0000 7fb171dd0740 1 Processor -- start 2024-09-19T09:12:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.349+0000 7fb171dd0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8926429a0 0x55f8926be9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.349+0000 7fb171dd0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8926432e0 0x55f8926bef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.349+0000 7fb171dd0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f892648370 0x55f8926c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.349+0000 7fb171dd0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb1681aa3c0 con 0x55f892648370 2024-09-19T09:12:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.349+0000 7fb171dd0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb1681a9cf0 con 0x55f8926429a0 2024-09-19T09:12:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.349+0000 7fb171dd0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb1681a6f10 con 0x55f8926432e0 2024-09-19T09:12:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.349+0000 7fb171dd0740 10 client.0 _ll_register_callbacks cb 0x7fb1600137e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:12:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.349+0000 7fb16d4f6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f892648370 0x55f8926c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.349+0000 7fb16d4f6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f892648370 0x55f8926c5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56984/0 (socket says 172.21.15.120:56984) 2024-09-19T09:12:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.349+0000 7fb16d4f6640 1 -- 172.21.15.120:0/4121123416 learned_addr learned my addr 172.21.15.120:0/4121123416 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:12:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.349+0000 7fb16d4f6640 1 --2- 172.21.15.120:0/4121123416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8926429a0 0x55f8926be9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.349+0000 7fb16ccf5640 1 --2- 172.21.15.120:0/4121123416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8926432e0 0x55f8926bef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.349+0000 7fb16d4f6640 1 -- 172.21.15.120:0/4121123416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8926432e0 msgr2=0x55f8926bef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.349+0000 7fb16d4f6640 1 --2- 172.21.15.120:0/4121123416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8926432e0 0x55f8926bef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.349+0000 7fb16d4f6640 1 -- 172.21.15.120:0/4121123416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8926429a0 msgr2=0x55f8926be9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:12:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.349+0000 7fb16d4f6640 1 --2- 172.21.15.120:0/4121123416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8926429a0 0x55f8926be9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.349+0000 7fb16d4f6640 1 -- 172.21.15.120:0/4121123416 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8926c6570 con 0x55f892648370 2024-09-19T09:12:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.349+0000 7fb16d4f6640 1 --2- 172.21.15.120:0/4121123416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f892648370 0x55f8926c5f60 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fb160009be0 tx=0x7fb1600135d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7879a0e510b71ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:12:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.349+0000 7fb147fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:12:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.350+0000 7fb147fff640 1 -- 172.21.15.120:0/4121123416 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16003c060 con 0x55f892648370 2024-09-19T09:12:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.350+0000 7fb147fff640 1 -- 172.21.15.120:0/4121123416 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb16000fd40 con 0x55f892648370 2024-09-19T09:12:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.350+0000 7fb171dd0740 1 -- 172.21.15.120:0/4121123416 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb16000f600 con 0x55f892648370 2024-09-19T09:12:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.350+0000 7fb147fff640 1 -- 172.21.15.120:0/4121123416 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb16002e040 con 0x55f892648370 2024-09-19T09:12:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.350+0000 7fb171dd0740 10 client.19095 fetch_fsmap learned FSMap version 32 2024-09-19T09:12:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.350+0000 7fb171dd0740 1 -- 172.21.15.120:0/4121123416 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f8926c7880 con 0x55f892648370 2024-09-19T09:12:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.350+0000 7fb147fff640 1 -- 172.21.15.120:0/4121123416 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb16000f600 con 0x55f892648370 2024-09-19T09:12:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.350+0000 7fb147fff640 1 -- 172.21.15.120:0/4121123416 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb16000c070 con 0x55f892648370 2024-09-19T09:12:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.350+0000 7fb171dd0740 10 client.19095 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:12:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.350+0000 7fb171dd0740 10 client.19095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:12:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.350+0000 7fb171dd0740 20 client.19095 populate_metadata read hostname 'smithi120' 2024-09-19T09:12:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.350+0000 7fb171dd0740 1 --2- 172.21.15.120:0/4121123416 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f8926c81b0 0x55f8926ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:12:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.350+0000 7fb171dd0740 4 client.19095 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:12:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.350+0000 7fb171dd0740 1 -- 172.21.15.120:0/4121123416 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f8925add70 con 0x55f8926c81b0 2024-09-19T09:12:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.351+0000 7fb16ccf5640 1 --2- 172.21.15.120:0/4121123416 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f8926c81b0 0x55f8926ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:12:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.351+0000 7fb147fff640 10 client.19095 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.351+0000 7fb16ccf5640 1 --2- 172.21.15.120:0/4121123416 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f8926c81b0 0x55f8926ca5d0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a523b97b048376 server_cookie=5e9f108dbf74dab4 in_seq=0 out_seq=0 2024-09-19T09:12:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.405+0000 7fb147fff640 1 -- 172.21.15.120:0/4121123416 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f8925add70 con 0x55f8926c81b0 2024-09-19T09:12:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.405+0000 7fb147fff640 10 client.19095 handle_command_reply: tid=0 2024-09-19T09:12:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.405+0000 7fb171dd0740 1 client.19095 shutdown 2024-09-19T09:12:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.405+0000 7fb171dd0740 1 -- 172.21.15.120:0/4121123416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f892648370 msgr2=0x55f8926c5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.405+0000 7fb171dd0740 1 --2- 172.21.15.120:0/4121123416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f892648370 0x55f8926c5f60 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fb160009be0 tx=0x7fb1600135d0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.405+0000 7fb171dd0740 1 -- 172.21.15.120:0/4121123416 shutdown_connections 2024-09-19T09:12:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.405+0000 7fb171dd0740 1 --2- 172.21.15.120:0/4121123416 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f8926c81b0 0x55f8926ca5d0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.405+0000 7fb171dd0740 1 --2- 172.21.15.120:0/4121123416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f892648370 0x55f8926c5f60 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.405+0000 7fb171dd0740 1 --2- 172.21.15.120:0/4121123416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8926432e0 0x55f8926bef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.405+0000 7fb147fff640 0 client.19095 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:12:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.405+0000 7fb171dd0740 1 --2- 172.21.15.120:0/4121123416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8926429a0 0x55f8926be9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.405+0000 7fb171dd0740 1 -- 172.21.15.120:0/4121123416 >> 172.21.15.120:0/4121123416 conn(0x55f89263e010 msgr2=0x55f89263c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.405+0000 7fb171dd0740 1 -- 172.21.15.120:0/4121123416 shutdown_connections 2024-09-19T09:12:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.405+0000 7fb171dd0740 1 -- 172.21.15.120:0/4121123416 wait complete. 2024-09-19T09:12:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.405+0000 7fb171dd0740 20 client.19095 trim_cache size 0 max 16384 2024-09-19T09:12:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:12:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:12:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:12:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:12:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.406+0000 7fb16ef59640 1 -- 172.21.15.120:0/3520352913 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb1580a3370 msgr2=0x7fb1580a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.406+0000 7fb16ef59640 1 --2- 172.21.15.120:0/3520352913 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb1580a3370 0x7fb1580a5830 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb15400d740 tx=0x7fb154090e20 comp rx=0 tx=0).stop 2024-09-19T09:12:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.406+0000 7fb16ef59640 1 -- 172.21.15.120:0/3520352913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb168101c30 msgr2=0x7fb168198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:12:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.406+0000 7fb16ef59640 1 --2- 172.21.15.120:0/3520352913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb168101c30 0x7fb168198330 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fb15400d770 tx=0x7fb15400dc30 comp rx=0 tx=0).stop 2024-09-19T09:12:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.406+0000 7fb16ef59640 1 -- 172.21.15.120:0/3520352913 shutdown_connections 2024-09-19T09:12:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.406+0000 7fb16ef59640 1 --2- 172.21.15.120:0/3520352913 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb1580a3370 0x7fb1580a5830 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.406+0000 7fb16ef59640 1 --2- 172.21.15.120:0/3520352913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1681025f0 0x7fb16819f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.406+0000 7fb16ef59640 1 --2- 172.21.15.120:0/3520352913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb168101c30 0x7fb168198330 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.406+0000 7fb16ef59640 1 --2- 172.21.15.120:0/3520352913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb168100a40 0x7fb168197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:12:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.406+0000 7fb16ef59640 1 -- 172.21.15.120:0/3520352913 >> 172.21.15.120:0/3520352913 conn(0x7fb1680fc1e0 msgr2=0x7fb1680698a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:12:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.406+0000 7fb16ef59640 1 -- 172.21.15.120:0/3520352913 shutdown_connections 2024-09-19T09:12:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:12:58.406+0000 7fb16ef59640 1 -- 172.21.15.120:0/3520352913 wait complete. 2024-09-19T09:12:59.068 INFO:tasks.workunit.client.0.smithi120.stdout: 1 984176 9.94 MB/sec execute 414 sec latency 13697.509 ms 2024-09-19T09:12:59.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:12:58 smithi196 ceph-mon[35150]: pgmap v646: 97 pgs: 97 active+clean; 40 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 3.9 MiB/s wr, 244 op/s 2024-09-19T09:12:59.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:12:58 smithi153 ceph-mon[35071]: pgmap v646: 97 pgs: 97 active+clean; 40 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 3.9 MiB/s wr, 244 op/s 2024-09-19T09:12:59.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:12:58 smithi120 ceph-mon[31150]: pgmap v646: 97 pgs: 97 active+clean; 40 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 3.9 MiB/s wr, 244 op/s 2024-09-19T09:13:00.068 INFO:tasks.workunit.client.0.smithi120.stdout: 1 984176 9.91 MB/sec execute 415 sec latency 14697.568 ms 2024-09-19T09:13:01.068 INFO:tasks.workunit.client.0.smithi120.stdout: 1 984176 9.89 MB/sec execute 416 sec latency 15697.655 ms 2024-09-19T09:13:01.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:00 smithi196 ceph-mon[35150]: pgmap v647: 97 pgs: 97 active+clean; 42 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 4.0 MiB/s wr, 172 op/s 2024-09-19T09:13:01.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:00 smithi153 ceph-mon[35071]: pgmap v647: 97 pgs: 97 active+clean; 42 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 4.0 MiB/s wr, 172 op/s 2024-09-19T09:13:01.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:00 smithi120 ceph-mon[31150]: pgmap v647: 97 pgs: 97 active+clean; 42 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 4.0 MiB/s wr, 172 op/s 2024-09-19T09:13:02.068 INFO:tasks.workunit.client.0.smithi120.stdout: 1 984176 9.87 MB/sec execute 417 sec latency 16697.742 ms 2024-09-19T09:13:03.068 INFO:tasks.workunit.client.0.smithi120.stdout: 1 984176 9.84 MB/sec execute 418 sec latency 17697.824 ms 2024-09-19T09:13:03.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:02 smithi196 ceph-mon[35150]: pgmap v648: 97 pgs: 97 active+clean; 41 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 3.5 MiB/s wr, 119 op/s 2024-09-19T09:13:03.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:02 smithi153 ceph-mon[35071]: pgmap v648: 97 pgs: 97 active+clean; 41 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 3.5 MiB/s wr, 119 op/s 2024-09-19T09:13:03.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:02 smithi120 ceph-mon[31150]: pgmap v648: 97 pgs: 97 active+clean; 41 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 3.5 MiB/s wr, 119 op/s 2024-09-19T09:13:04.068 INFO:tasks.workunit.client.0.smithi120.stdout: 1 984176 9.82 MB/sec execute 419 sec latency 18697.895 ms 2024-09-19T09:13:05.069 INFO:tasks.workunit.client.0.smithi120.stdout: 1 984176 9.80 MB/sec execute 420 sec latency 19698.001 ms 2024-09-19T09:13:05.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:04 smithi196 ceph-mon[35150]: pgmap v649: 97 pgs: 97 active+clean; 39 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 3.4 MiB/s wr, 120 op/s 2024-09-19T09:13:05.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:04 smithi153 ceph-mon[35071]: pgmap v649: 97 pgs: 97 active+clean; 39 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 3.4 MiB/s wr, 120 op/s 2024-09-19T09:13:05.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:04 smithi120 ceph-mon[31150]: pgmap v649: 97 pgs: 97 active+clean; 39 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 3.4 MiB/s wr, 120 op/s 2024-09-19T09:13:06.069 INFO:tasks.workunit.client.0.smithi120.stdout: 1 984176 9.77 MB/sec execute 421 sec latency 20698.072 ms 2024-09-19T09:13:06.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:05 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:13:06.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:05 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:13:06.216 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:05 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:13:07.069 INFO:tasks.workunit.client.0.smithi120.stdout: 1 984176 9.75 MB/sec execute 422 sec latency 21698.102 ms 2024-09-19T09:13:07.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:06 smithi196 ceph-mon[35150]: pgmap v650: 97 pgs: 97 active+clean; 38 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 2.7 MiB/s wr, 107 op/s 2024-09-19T09:13:07.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:06 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:13:07.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:06 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:13:07.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:06 smithi153 ceph-mon[35071]: pgmap v650: 97 pgs: 97 active+clean; 38 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 2.7 MiB/s wr, 107 op/s 2024-09-19T09:13:07.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:06 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:13:07.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:06 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:13:07.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:06 smithi120 ceph-mon[31150]: pgmap v650: 97 pgs: 97 active+clean; 38 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 2.7 MiB/s wr, 107 op/s 2024-09-19T09:13:07.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:06 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:13:07.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:06 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:13:08.069 INFO:tasks.workunit.client.0.smithi120.stdout: 1 984176 9.73 MB/sec execute 423 sec latency 22698.191 ms 2024-09-19T09:13:08.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:07 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:13:08.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:07 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:13:08.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:07 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:13:08.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:07 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:13:08.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:07 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:13:08.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:07 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:13:08.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:07 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:13:08.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:07 smithi196 ceph-mon[35150]: pgmap v651: 97 pgs: 97 active+clean; 37 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 2.8 MiB/s wr, 119 op/s 2024-09-19T09:13:08.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:07 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:13:08.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:07 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:13:08.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:07 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:13:08.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:07 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:13:08.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:07 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:13:08.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:07 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:13:08.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:07 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:13:08.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:07 smithi153 ceph-mon[35071]: pgmap v651: 97 pgs: 97 active+clean; 37 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 2.8 MiB/s wr, 119 op/s 2024-09-19T09:13:08.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:07 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:13:08.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:07 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:13:08.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:07 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:13:08.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:07 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:13:08.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:07 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:13:08.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:07 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:13:08.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:07 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:13:08.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:07 smithi120 ceph-mon[31150]: pgmap v651: 97 pgs: 97 active+clean; 37 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 2.8 MiB/s wr, 119 op/s 2024-09-19T09:13:09.069 INFO:tasks.workunit.client.0.smithi120.stdout: 1 984176 9.70 MB/sec execute 424 sec latency 23698.276 ms 2024-09-19T09:13:10.069 INFO:tasks.workunit.client.0.smithi120.stdout: 1 984176 9.68 MB/sec execute 425 sec latency 24698.372 ms 2024-09-19T09:13:10.544 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:13:10.804 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:13:11.069 INFO:tasks.workunit.client.0.smithi120.stdout: 1 984176 9.66 MB/sec execute 426 sec latency 25698.439 ms 2024-09-19T09:13:11.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:10 smithi196 ceph-mon[35150]: pgmap v652: 97 pgs: 97 active+clean; 39 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 2.5 MiB/s wr, 121 op/s 2024-09-19T09:13:11.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:10 smithi153 ceph-mon[35071]: pgmap v652: 97 pgs: 97 active+clean; 39 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 2.5 MiB/s wr, 121 op/s 2024-09-19T09:13:11.221 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:10 smithi120 ceph-mon[31150]: pgmap v652: 97 pgs: 97 active+clean; 39 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 2.5 MiB/s wr, 121 op/s 2024-09-19T09:13:11.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.301+0000 7f92e5bed640 1 -- 172.21.15.120:0/9931148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92e010c1b0 msgr2=0x7f92e010e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:11.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.301+0000 7f92e5bed640 1 --2- 172.21.15.120:0/9931148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92e010c1b0 0x7f92e010e620 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009900 tx=0x7f92d802a480 comp rx=0 tx=0).stop 2024-09-19T09:13:11.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.301+0000 7f92e5bed640 1 -- 172.21.15.120:0/9931148 shutdown_connections 2024-09-19T09:13:11.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.301+0000 7f92e5bed640 1 --2- 172.21.15.120:0/9931148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92e010c1b0 0x7f92e010e620 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.301+0000 7f92e5bed640 1 --2- 172.21.15.120:0/9931148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92e01036f0 0x7f92e010bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.301+0000 7f92e5bed640 1 --2- 172.21.15.120:0/9931148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92e0102dd0 0x7f92e01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.301+0000 7f92e5bed640 1 -- 172.21.15.120:0/9931148 >> 172.21.15.120:0/9931148 conn(0x7f92e00fc7d0 msgr2=0x7f92e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:11.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.301+0000 7f92e5bed640 1 -- 172.21.15.120:0/9931148 shutdown_connections 2024-09-19T09:13:11.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.301+0000 7f92e5bed640 1 -- 172.21.15.120:0/9931148 wait complete. 2024-09-19T09:13:11.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.302+0000 7f92e5bed640 1 Processor -- start 2024-09-19T09:13:11.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.302+0000 7f92e5bed640 1 -- start start 2024-09-19T09:13:11.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.302+0000 7f92e5bed640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92e0102dd0 0x7f92e0100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:11.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.302+0000 7f92e5bed640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92e01036f0 0x7f92e0100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:11.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.302+0000 7f92e5bed640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92e010c1b0 0x7f92e01014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:11.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.302+0000 7f92e5bed640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f92e0111f40 con 0x7f92e01036f0 2024-09-19T09:13:11.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.302+0000 7f92e5bed640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f92e0111dc0 con 0x7f92e010c1b0 2024-09-19T09:13:11.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.302+0000 7f92e5bed640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f92e01120c0 con 0x7f92e0102dd0 2024-09-19T09:13:11.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.302+0000 7f92deffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92e01036f0 0x7f92e0100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:11.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.302+0000 7f92deffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92e01036f0 0x7f92e0100f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39742/0 (socket says 172.21.15.120:39742) 2024-09-19T09:13:11.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.302+0000 7f92deffd640 1 -- 172.21.15.120:0/1563912140 learned_addr learned my addr 172.21.15.120:0/1563912140 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:11.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.303+0000 7f92dffff640 1 --2- 172.21.15.120:0/1563912140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92e010c1b0 0x7f92e01014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:11.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.303+0000 7f92df7fe640 1 --2- 172.21.15.120:0/1563912140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92e0102dd0 0x7f92e0100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:11.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.303+0000 7f92deffd640 1 -- 172.21.15.120:0/1563912140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92e0102dd0 msgr2=0x7f92e0100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:11.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.303+0000 7f92deffd640 1 --2- 172.21.15.120:0/1563912140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92e0102dd0 0x7f92e0100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.303+0000 7f92deffd640 1 -- 172.21.15.120:0/1563912140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92e010c1b0 msgr2=0x7f92e01014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:11.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.303+0000 7f92deffd640 1 --2- 172.21.15.120:0/1563912140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92e010c1b0 0x7f92e01014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.303+0000 7f92deffd640 1 -- 172.21.15.120:0/1563912140 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e01a68a0 con 0x7f92e01036f0 2024-09-19T09:13:11.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.303+0000 7f92df7fe640 1 --2- 172.21.15.120:0/1563912140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92e0102dd0 0x7f92e0100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:13:11.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.303+0000 7f92dffff640 1 --2- 172.21.15.120:0/1563912140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92e010c1b0 0x7f92e01014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:13:11.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.303+0000 7f92deffd640 1 --2- 172.21.15.120:0/1563912140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92e01036f0 0x7f92e0100f70 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f92d400c890 tx=0x7f92d400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:11.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.303+0000 7f92dcff9640 1 -- 172.21.15.120:0/1563912140 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d4004220 con 0x7f92e01036f0 2024-09-19T09:13:11.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.303+0000 7f92dcff9640 1 -- 172.21.15.120:0/1563912140 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92d40043c0 con 0x7f92e01036f0 2024-09-19T09:13:11.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.303+0000 7f92e5bed640 1 -- 172.21.15.120:0/1563912140 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92e01a6b90 con 0x7f92e01036f0 2024-09-19T09:13:11.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.303+0000 7f92dcff9640 1 -- 172.21.15.120:0/1563912140 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d40049e0 con 0x7f92e01036f0 2024-09-19T09:13:11.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.303+0000 7f92e5bed640 1 -- 172.21.15.120:0/1563912140 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92e01a7120 con 0x7f92e01036f0 2024-09-19T09:13:11.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.304+0000 7f92e5bed640 1 -- 172.21.15.120:0/1563912140 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92e01031b0 con 0x7f92e01036f0 2024-09-19T09:13:11.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.305+0000 7f92dcff9640 1 -- 172.21.15.120:0/1563912140 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f92d40079b0 con 0x7f92e01036f0 2024-09-19T09:13:11.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.306+0000 7f92dcff9640 1 --2- 172.21.15.120:0/1563912140 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92c80a32c0 0x7f92c80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:11.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.306+0000 7f92df7fe640 1 --2- 172.21.15.120:0/1563912140 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92c80a32c0 0x7f92c80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:11.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.306+0000 7f92dcff9640 1 -- 172.21.15.120:0/1563912140 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f92d40cf920 con 0x7f92e01036f0 2024-09-19T09:13:11.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.306+0000 7f92df7fe640 1 --2- 172.21.15.120:0/1563912140 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92c80a32c0 0x7f92c80a5780 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f92d0008e00 tx=0x7f92d0008350 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:11.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.309+0000 7f92dcff9640 1 -- 172.21.15.120:0/1563912140 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f92d4014030 con 0x7f92e01036f0 2024-09-19T09:13:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.495+0000 7f417a01e640 1 -- 172.21.15.120:0/2242079242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41741025f0 msgr2=0x7f4174108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.495+0000 7f417a01e640 1 --2- 172.21.15.120:0/2242079242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41741025f0 0x7f4174108e80 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f416c009ca0 tx=0x7f416c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.495+0000 7f417a01e640 1 -- 172.21.15.120:0/2242079242 shutdown_connections 2024-09-19T09:13:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.495+0000 7f417a01e640 1 --2- 172.21.15.120:0/2242079242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41741025f0 0x7f4174108e80 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.495+0000 7f417a01e640 1 --2- 172.21.15.120:0/2242079242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4174101c30 0x7f41741020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.495+0000 7f417a01e640 1 --2- 172.21.15.120:0/2242079242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4174100a40 0x7f4174100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.495+0000 7f417a01e640 1 -- 172.21.15.120:0/2242079242 >> 172.21.15.120:0/2242079242 conn(0x7f41740fc1e0 msgr2=0x7f41740fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.495+0000 7f417a01e640 1 -- 172.21.15.120:0/2242079242 shutdown_connections 2024-09-19T09:13:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.495+0000 7f417a01e640 1 -- 172.21.15.120:0/2242079242 wait complete. 2024-09-19T09:13:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.495+0000 7f417a01e640 1 Processor -- start 2024-09-19T09:13:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.495+0000 7f417a01e640 1 -- start start 2024-09-19T09:13:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.496+0000 7f417a01e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4174100a40 0x7f4174196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.496+0000 7f417a01e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4174101c30 0x7f41741971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.496+0000 7f417a01e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41741025f0 0x7f417419e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.496+0000 7f417a01e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f417410c6c0 con 0x7f4174100a40 2024-09-19T09:13:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.496+0000 7f417a01e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f417410c540 con 0x7f4174101c30 2024-09-19T09:13:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.496+0000 7f417a01e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f417410c840 con 0x7f41741025f0 2024-09-19T09:13:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.496+0000 7f41737fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4174100a40 0x7f4174196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.496+0000 7f41737fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4174100a40 0x7f4174196c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39748/0 (socket says 172.21.15.120:39748) 2024-09-19T09:13:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.496+0000 7f41737fe640 1 -- 172.21.15.120:0/1619150993 learned_addr learned my addr 172.21.15.120:0/1619150993 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.496+0000 7f4173fff640 1 --2- 172.21.15.120:0/1619150993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41741025f0 0x7f417419e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.496+0000 7f41737fe640 1 -- 172.21.15.120:0/1619150993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41741025f0 msgr2=0x7f417419e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.496+0000 7f41737fe640 1 --2- 172.21.15.120:0/1619150993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41741025f0 0x7f417419e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.496+0000 7f41737fe640 1 -- 172.21.15.120:0/1619150993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4174101c30 msgr2=0x7f41741971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.496+0000 7f4172ffd640 1 --2- 172.21.15.120:0/1619150993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4174101c30 0x7f41741971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.496+0000 7f41737fe640 1 --2- 172.21.15.120:0/1619150993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4174101c30 0x7f41741971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.496+0000 7f41737fe640 1 -- 172.21.15.120:0/1619150993 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f417419e8c0 con 0x7f4174100a40 2024-09-19T09:13:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.496+0000 7f4172ffd640 1 --2- 172.21.15.120:0/1619150993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4174101c30 0x7f41741971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:13:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.496+0000 7f41737fe640 1 --2- 172.21.15.120:0/1619150993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4174100a40 0x7f4174196c80 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f416400c970 tx=0x7f416400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.496+0000 7f4170ff9640 1 -- 172.21.15.120:0/1619150993 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4164004220 con 0x7f4174100a40 2024-09-19T09:13:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.496+0000 7f4170ff9640 1 -- 172.21.15.120:0/1619150993 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41640043c0 con 0x7f4174100a40 2024-09-19T09:13:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.496+0000 7f417a01e640 1 -- 172.21.15.120:0/1619150993 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f417419ebb0 con 0x7f4174100a40 2024-09-19T09:13:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.496+0000 7f4170ff9640 1 -- 172.21.15.120:0/1619150993 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416400f450 con 0x7f4174100a40 2024-09-19T09:13:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.496+0000 7f417a01e640 1 -- 172.21.15.120:0/1619150993 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f417419f100 con 0x7f4174100a40 2024-09-19T09:13:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.499+0000 7f4170ff9640 1 -- 172.21.15.120:0/1619150993 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f416400f5f0 con 0x7f4174100a40 2024-09-19T09:13:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.499+0000 7f417ce90740 1 Processor -- start 2024-09-19T09:13:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.499+0000 7f417ce90740 1 -- start start 2024-09-19T09:13:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.499+0000 7f417ce90740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55720bc00bb0 0x55720bc00fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.499+0000 7f417ce90740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55720bc014f0 0x55720bc05fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.499+0000 7f417ce90740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55720bc06580 0x55720bc08a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.499+0000 7f417ce90740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55720bb622f0 con 0x55720bc06580 2024-09-19T09:13:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.499+0000 7f417ce90740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55720bb0a270 con 0x55720bc014f0 2024-09-19T09:13:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.499+0000 7f417ce90740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55720bb5e2b0 con 0x55720bc00bb0 2024-09-19T09:13:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.499+0000 7f4172ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55720bc06580 0x55720bc08a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.499+0000 7f4172ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55720bc06580 0x55720bc08a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39762/0 (socket says 172.21.15.120:39762) 2024-09-19T09:13:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.499+0000 7f4172ffd640 1 -- 172.21.15.120:0/3199567878 learned_addr learned my addr 172.21.15.120:0/3199567878 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.499+0000 7f4170ff9640 1 --2- 172.21.15.120:0/1619150993 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f41600a32f0 0x7f41600a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.499+0000 7f41737fe640 1 --2- 172.21.15.120:0/3199567878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55720bc014f0 0x55720bc05fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.499+0000 7f4173fff640 1 --2- 172.21.15.120:0/3199567878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55720bc00bb0 0x55720bc00fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.499+0000 7f4173fff640 1 --2- 172.21.15.120:0/1619150993 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f41600a32f0 0x7f41600a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.500+0000 7f4172ffd640 1 -- 172.21.15.120:0/3199567878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55720bc00bb0 msgr2=0x55720bc00fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.500+0000 7f4172ffd640 1 --2- 172.21.15.120:0/3199567878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55720bc00bb0 0x55720bc00fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.500+0000 7f4172ffd640 1 -- 172.21.15.120:0/3199567878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55720bc014f0 msgr2=0x55720bc05fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.500+0000 7f4172ffd640 1 --2- 172.21.15.120:0/3199567878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55720bc014f0 0x55720bc05fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.500+0000 7f4172ffd640 1 -- 172.21.15.120:0/3199567878 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55720bb09340 con 0x55720bc06580 2024-09-19T09:13:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.500+0000 7f41737fe640 1 --2- 172.21.15.120:0/3199567878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55720bc014f0 0x55720bc05fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:13:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.500+0000 7f4170ff9640 1 -- 172.21.15.120:0/1619150993 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f41640cf6f0 con 0x7f4174100a40 2024-09-19T09:13:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.500+0000 7f4172ffd640 1 --2- 172.21.15.120:0/3199567878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55720bc06580 0x55720bc08a40 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f415c00bb40 tx=0x7f415c00d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f3d72c879bcae3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.500+0000 7f4169ffb640 1 -- 172.21.15.120:0/3199567878 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415c017420 con 0x55720bc06580 2024-09-19T09:13:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.500+0000 7f4169ffb640 1 -- 172.21.15.120:0/3199567878 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f415c017aa0 con 0x55720bc06580 2024-09-19T09:13:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.500+0000 7f4169ffb640 1 -- 172.21.15.120:0/3199567878 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415c00fb90 con 0x55720bc06580 2024-09-19T09:13:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.501+0000 7f417ce90740 1 -- 172.21.15.120:0/3199567878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55720bc06580 msgr2=0x55720bc08a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.501+0000 7f417ce90740 1 --2- 172.21.15.120:0/3199567878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55720bc06580 0x55720bc08a40 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f415c00bb40 tx=0x7f415c00d340 comp rx=0 tx=0).stop 2024-09-19T09:13:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.501+0000 7f4173fff640 1 --2- 172.21.15.120:0/1619150993 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f41600a32f0 0x7f41600a57b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f416c009430 tx=0x7f416c009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.501+0000 7f417ce90740 1 -- 172.21.15.120:0/3199567878 shutdown_connections 2024-09-19T09:13:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.501+0000 7f417ce90740 1 --2- 172.21.15.120:0/3199567878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55720bc06580 0x55720bc08a40 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.501+0000 7f417ce90740 1 --2- 172.21.15.120:0/3199567878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55720bc014f0 0x55720bc05fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.501+0000 7f417ce90740 1 --2- 172.21.15.120:0/3199567878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55720bc00bb0 0x55720bc00fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.501+0000 7f417ce90740 1 -- 172.21.15.120:0/3199567878 >> 172.21.15.120:0/3199567878 conn(0x55720bbfa010 msgr2=0x55720bbfa410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.501+0000 7f417ce90740 1 -- 172.21.15.120:0/3199567878 shutdown_connections 2024-09-19T09:13:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.501+0000 7f417ce90740 1 -- 172.21.15.120:0/3199567878 wait complete. 2024-09-19T09:13:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.501+0000 7f417ce90740 1 -- start start 2024-09-19T09:13:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.501+0000 7f417ce90740 1 Processor -- start 2024-09-19T09:13:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.501+0000 7f417ce90740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55720bc00bb0 0x55720bc0ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.501+0000 7f417ce90740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55720bc014f0 0x55720bc0b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.501+0000 7f417ce90740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55720bc06580 0x55720bc122f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.502+0000 7f417ce90740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55720bc09550 con 0x55720bc014f0 2024-09-19T09:13:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.502+0000 7f417ce90740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55720baeb290 con 0x55720bc06580 2024-09-19T09:13:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.502+0000 7f417ce90740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55720bc096d0 con 0x55720bc00bb0 2024-09-19T09:13:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.502+0000 7f417ce90740 10 client.0 _ll_register_callbacks cb 0x55720bb1c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.502+0000 7f4172ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55720bc014f0 0x55720bc0b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.502+0000 7f4172ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55720bc014f0 0x55720bc0b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39776/0 (socket says 172.21.15.120:39776) 2024-09-19T09:13:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.502+0000 7f4172ffd640 1 -- 172.21.15.120:0/636140898 learned_addr learned my addr 172.21.15.120:0/636140898 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.502+0000 7f4173fff640 1 --2- 172.21.15.120:0/636140898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55720bc06580 0x55720bc122f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.502+0000 7f41737fe640 1 --2- 172.21.15.120:0/636140898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55720bc00bb0 0x55720bc0ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.502+0000 7f4172ffd640 1 -- 172.21.15.120:0/636140898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55720bc00bb0 msgr2=0x55720bc0ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.502+0000 7f4172ffd640 1 --2- 172.21.15.120:0/636140898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55720bc00bb0 0x55720bc0ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.502+0000 7f4172ffd640 1 -- 172.21.15.120:0/636140898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55720bc06580 msgr2=0x55720bc122f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.502+0000 7f4172ffd640 1 --2- 172.21.15.120:0/636140898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55720bc06580 0x55720bc122f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.502+0000 7f4172ffd640 1 -- 172.21.15.120:0/636140898 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55720b875210 con 0x55720bc014f0 2024-09-19T09:13:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.502+0000 7f4173fff640 1 --2- 172.21.15.120:0/636140898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55720bc06580 0x55720bc122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:13:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.502+0000 7f4172ffd640 1 --2- 172.21.15.120:0/636140898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55720bc014f0 0x55720bc0b2b0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f415c00bb10 tx=0x7f415c017650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e95bf2b382fbf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.502+0000 7f4152ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.502+0000 7f4152ffd640 1 -- 172.21.15.120:0/636140898 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415c026070 con 0x55720bc014f0 2024-09-19T09:13:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.502+0000 7f417ce90740 1 -- 172.21.15.120:0/636140898 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f415c017aa0 con 0x55720bc014f0 2024-09-19T09:13:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.502+0000 7f4152ffd640 1 -- 172.21.15.120:0/636140898 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f415c00f8f0 con 0x55720bc014f0 2024-09-19T09:13:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.502+0000 7f4152ffd640 1 -- 172.21.15.120:0/636140898 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f415c032050 con 0x55720bc014f0 2024-09-19T09:13:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.502+0000 7f417ce90740 10 client.19113 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.502+0000 7f417ce90740 1 -- 172.21.15.120:0/636140898 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55720b941620 con 0x55720bc014f0 2024-09-19T09:13:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.503+0000 7f4152ffd640 1 -- 172.21.15.120:0/636140898 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f415c017aa0 con 0x55720bc014f0 2024-09-19T09:13:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.503+0000 7f4152ffd640 1 -- 172.21.15.120:0/636140898 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f415c022c10 con 0x55720bc014f0 2024-09-19T09:13:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.503+0000 7f417ce90740 10 client.19113 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.503+0000 7f417ce90740 10 client.19113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:13:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.503+0000 7f417ce90740 20 client.19113 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.503+0000 7f417ce90740 1 --2- 172.21.15.120:0/636140898 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55720bc16710 0x55720bc18b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.503+0000 7f417ce90740 4 client.19113 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:13:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.503+0000 7f417ce90740 1 -- 172.21.15.120:0/636140898 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55720bb6fa20 con 0x55720bc16710 2024-09-19T09:13:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.503+0000 7f41737fe640 1 --2- 172.21.15.120:0/636140898 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55720bc16710 0x55720bc18b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.504+0000 7f4152ffd640 10 client.19113 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.504+0000 7f41737fe640 1 --2- 172.21.15.120:0/636140898 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55720bc16710 0x55720bc18b00 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7441e15a515e3328 server_cookie=3264ddab2e974b6b in_seq=0 out_seq=0 2024-09-19T09:13:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.504+0000 7f4152ffd640 1 -- 172.21.15.120:0/636140898 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55720bb6fa20 con 0x55720bc16710 2024-09-19T09:13:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.504+0000 7f4152ffd640 10 client.19113 handle_command_reply: tid=0 2024-09-19T09:13:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.504+0000 7f417ce90740 1 client.19113 shutdown 2024-09-19T09:13:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.505+0000 7f417ce90740 1 -- 172.21.15.120:0/636140898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55720bc014f0 msgr2=0x55720bc0b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.505+0000 7f417ce90740 1 --2- 172.21.15.120:0/636140898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55720bc014f0 0x55720bc0b2b0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f415c00bb10 tx=0x7f415c017650 comp rx=0 tx=0).stop 2024-09-19T09:13:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.505+0000 7f417ce90740 1 -- 172.21.15.120:0/636140898 shutdown_connections 2024-09-19T09:13:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.505+0000 7f417ce90740 1 --2- 172.21.15.120:0/636140898 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55720bc16710 0x55720bc18b00 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.505+0000 7f417ce90740 1 --2- 172.21.15.120:0/636140898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55720bc06580 0x55720bc122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.505+0000 7f417ce90740 1 --2- 172.21.15.120:0/636140898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55720bc014f0 0x55720bc0b2b0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.505+0000 7f4152ffd640 0 client.19113 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.505+0000 7f417ce90740 1 --2- 172.21.15.120:0/636140898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55720bc00bb0 0x55720bc0ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.505+0000 7f417ce90740 1 -- 172.21.15.120:0/636140898 >> 172.21.15.120:0/636140898 conn(0x55720bbfa010 msgr2=0x55720bbfb820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.505+0000 7f417ce90740 1 -- 172.21.15.120:0/636140898 shutdown_connections 2024-09-19T09:13:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.505+0000 7f417ce90740 1 -- 172.21.15.120:0/636140898 wait complete. 2024-09-19T09:13:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.505+0000 7f417ce90740 20 client.19113 trim_cache size 0 max 16384 2024-09-19T09:13:11.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.533+0000 7f92e5bed640 1 -- 172.21.15.120:0/1563912140 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92e0101f60 con 0x7f92e01036f0 2024-09-19T09:13:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.534+0000 7f417ce90740 1 Processor -- start 2024-09-19T09:13:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.534+0000 7f417ce90740 1 -- start start 2024-09-19T09:13:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.534+0000 7f417ce90740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55720bc00bb0 0x55720bbf90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.534+0000 7f417ce90740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55720bc014f0 0x55720bc122b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.534+0000 7f417ce90740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55720bc06580 0x55720bbfb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.534+0000 7f417ce90740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f41741a6d80 con 0x55720bc00bb0 2024-09-19T09:13:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.534+0000 7f417ce90740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f41741a88c0 con 0x55720bc06580 2024-09-19T09:13:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.534+0000 7f417ce90740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41741a7450 con 0x55720bc014f0 2024-09-19T09:13:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.534+0000 7f41737fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55720bc00bb0 0x55720bbf90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.534+0000 7f41737fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55720bc00bb0 0x55720bbf90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39786/0 (socket says 172.21.15.120:39786) 2024-09-19T09:13:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.534+0000 7f41737fe640 1 -- 172.21.15.120:0/364053209 learned_addr learned my addr 172.21.15.120:0/364053209 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.534+0000 7f4172ffd640 1 --2- 172.21.15.120:0/364053209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55720bc014f0 0x55720bc122b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.534+0000 7f4173fff640 1 --2- 172.21.15.120:0/364053209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55720bc06580 0x55720bbfb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.534+0000 7f41737fe640 1 -- 172.21.15.120:0/364053209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55720bc014f0 msgr2=0x55720bc122b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.534+0000 7f41737fe640 1 --2- 172.21.15.120:0/364053209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55720bc014f0 0x55720bc122b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.534+0000 7f41737fe640 1 -- 172.21.15.120:0/364053209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55720bc06580 msgr2=0x55720bbfb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.534+0000 7f41737fe640 1 --2- 172.21.15.120:0/364053209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55720bc06580 0x55720bbfb400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.534+0000 7f41737fe640 1 -- 172.21.15.120:0/364053209 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55720bbf52d0 con 0x55720bc00bb0 2024-09-19T09:13:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.534+0000 7f41737fe640 1 --2- 172.21.15.120:0/364053209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55720bc00bb0 0x55720bbf90b0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f4164098170 tx=0x7f41640bbf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71be5b7649d5d0ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:11.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.535+0000 7f92dcff9640 1 -- 172.21.15.120:0/1563912140 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11988 (secure 0 0 0) 0x7f92d409c680 con 0x7f92e01036f0 2024-09-19T09:13:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.535+0000 7f41697fa640 1 -- 172.21.15.120:0/364053209 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416401d030 con 0x55720bc00bb0 2024-09-19T09:13:11.537 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:13:11.538 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":32,"btime":"2024-09-19T09:11:12:904684+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:11:12.904670+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27518":{"gid":27518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3214776406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3214776406},{"type":"v1","addr":"172.21.15.153:6835","nonce":3214776406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:13:11.538 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 32 2024-09-19T09:13:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.535+0000 7f41697fa640 1 -- 172.21.15.120:0/364053209 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41640a4980 con 0x55720bc00bb0 2024-09-19T09:13:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.535+0000 7f41697fa640 1 -- 172.21.15.120:0/364053209 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41640079b0 con 0x55720bc00bb0 2024-09-19T09:13:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.535+0000 7f417ce90740 1 -- 172.21.15.120:0/364053209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55720bc00bb0 msgr2=0x55720bbf90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.535+0000 7f417ce90740 1 --2- 172.21.15.120:0/364053209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55720bc00bb0 0x55720bbf90b0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f4164098170 tx=0x7f41640bbf40 comp rx=0 tx=0).stop 2024-09-19T09:13:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.536+0000 7f417ce90740 1 -- 172.21.15.120:0/364053209 shutdown_connections 2024-09-19T09:13:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.536+0000 7f417ce90740 1 --2- 172.21.15.120:0/364053209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55720bc06580 0x55720bbfb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.536+0000 7f417ce90740 1 --2- 172.21.15.120:0/364053209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55720bc014f0 0x55720bc122b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.536+0000 7f417ce90740 1 --2- 172.21.15.120:0/364053209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55720bc00bb0 0x55720bbf90b0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.536+0000 7f417ce90740 1 -- 172.21.15.120:0/364053209 >> 172.21.15.120:0/364053209 conn(0x55720bbfa010 msgr2=0x55720bbf89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.536+0000 7f417ce90740 1 -- 172.21.15.120:0/364053209 shutdown_connections 2024-09-19T09:13:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.536+0000 7f417ce90740 1 -- 172.21.15.120:0/364053209 wait complete. 2024-09-19T09:13:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.536+0000 7f417ce90740 1 -- start start 2024-09-19T09:13:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.536+0000 7f417ce90740 1 Processor -- start 2024-09-19T09:13:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.536+0000 7f417ce90740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55720bc00bb0 0x55720bc0d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.536+0000 7f417ce90740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55720bc014f0 0x55720bc0dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.536+0000 7f417ce90740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55720bc06580 0x55720bc10d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.536+0000 7f417ce90740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f41741a81f0 con 0x55720bc014f0 2024-09-19T09:13:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.536+0000 7f417ce90740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f41741a7b20 con 0x55720bc06580 2024-09-19T09:13:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.536+0000 7f417ce90740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41741a4d40 con 0x55720bc00bb0 2024-09-19T09:13:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.537+0000 7f417ce90740 10 client.0 _ll_register_callbacks cb 0x7f415c00b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.537+0000 7f4172ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55720bc014f0 0x55720bc0dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.537+0000 7f4172ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55720bc014f0 0x55720bc0dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39790/0 (socket says 172.21.15.120:39790) 2024-09-19T09:13:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.537+0000 7f4172ffd640 1 -- 172.21.15.120:0/3792732889 learned_addr learned my addr 172.21.15.120:0/3792732889 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.537+0000 7f41737fe640 1 --2- 172.21.15.120:0/3792732889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55720bc00bb0 0x55720bc0d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.537+0000 7f4173fff640 1 --2- 172.21.15.120:0/3792732889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55720bc06580 0x55720bc10d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.537+0000 7f4172ffd640 1 -- 172.21.15.120:0/3792732889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55720bc00bb0 msgr2=0x55720bc0d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.537+0000 7f4172ffd640 1 --2- 172.21.15.120:0/3792732889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55720bc00bb0 0x55720bc0d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.537+0000 7f4172ffd640 1 -- 172.21.15.120:0/3792732889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55720bc06580 msgr2=0x55720bc10d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.537+0000 7f4172ffd640 1 --2- 172.21.15.120:0/3792732889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55720bc06580 0x55720bc10d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.537+0000 7f4172ffd640 1 -- 172.21.15.120:0/3792732889 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55720bc062e0 con 0x55720bc014f0 2024-09-19T09:13:11.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.537+0000 7f92e5bed640 1 -- 172.21.15.120:0/1563912140 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92c80a32c0 msgr2=0x7f92c80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:11.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.537+0000 7f92e5bed640 1 --2- 172.21.15.120:0/1563912140 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92c80a32c0 0x7f92c80a5780 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f92d0008e00 tx=0x7f92d0008350 comp rx=0 tx=0).stop 2024-09-19T09:13:11.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.537+0000 7f92e5bed640 1 -- 172.21.15.120:0/1563912140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92e01036f0 msgr2=0x7f92e0100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:11.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.537+0000 7f92e5bed640 1 --2- 172.21.15.120:0/1563912140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92e01036f0 0x7f92e0100f70 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f92d400c890 tx=0x7f92d400cd50 comp rx=0 tx=0).stop 2024-09-19T09:13:11.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.537+0000 7f92e5bed640 1 -- 172.21.15.120:0/1563912140 shutdown_connections 2024-09-19T09:13:11.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.537+0000 7f92e5bed640 1 --2- 172.21.15.120:0/1563912140 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92c80a32c0 0x7f92c80a5780 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.537+0000 7f92e5bed640 1 --2- 172.21.15.120:0/1563912140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92e010c1b0 0x7f92e01014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.537+0000 7f92e5bed640 1 --2- 172.21.15.120:0/1563912140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92e01036f0 0x7f92e0100f70 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.537+0000 7f92e5bed640 1 --2- 172.21.15.120:0/1563912140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92e0102dd0 0x7f92e0100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.537+0000 7f92e5bed640 1 -- 172.21.15.120:0/1563912140 >> 172.21.15.120:0/1563912140 conn(0x7f92e00fc7d0 msgr2=0x7f92e00fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.537+0000 7f4172ffd640 1 --2- 172.21.15.120:0/3792732889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55720bc014f0 0x55720bc0dd10 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x55720bc0c250 tx=0x7f415c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c17f1fadfbfc206 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.537+0000 7f4152ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:11.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.537+0000 7f92e5bed640 1 -- 172.21.15.120:0/1563912140 shutdown_connections 2024-09-19T09:13:11.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:11.537+0000 7f92e5bed640 1 -- 172.21.15.120:0/1563912140 wait complete. 2024-09-19T09:13:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.539+0000 7f4152ffd640 1 -- 172.21.15.120:0/3792732889 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415c026040 con 0x55720bc014f0 2024-09-19T09:13:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.539+0000 7f4152ffd640 1 -- 172.21.15.120:0/3792732889 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f415c020a70 con 0x55720bc014f0 2024-09-19T09:13:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.539+0000 7f417ce90740 1 -- 172.21.15.120:0/3792732889 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41640a4980 con 0x55720bc014f0 2024-09-19T09:13:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.539+0000 7f4152ffd640 1 -- 172.21.15.120:0/3792732889 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f415c005b00 con 0x55720bc014f0 2024-09-19T09:13:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.539+0000 7f417ce90740 10 client.19119 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.539+0000 7f417ce90740 1 -- 172.21.15.120:0/3792732889 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55720bc12480 con 0x55720bc014f0 2024-09-19T09:13:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.539+0000 7f4152ffd640 1 -- 172.21.15.120:0/3792732889 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41640a4980 con 0x55720bc014f0 2024-09-19T09:13:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.540+0000 7f4152ffd640 1 -- 172.21.15.120:0/3792732889 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f415c02b440 con 0x55720bc014f0 2024-09-19T09:13:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.540+0000 7f417ce90740 10 client.19119 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.540+0000 7f417ce90740 10 client.19119 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:13:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.540+0000 7f417ce90740 20 client.19119 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.540+0000 7f417ce90740 1 --2- 172.21.15.120:0/3792732889 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55720bc16710 0x55720bc81940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.540+0000 7f417ce90740 4 client.19119 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:13:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.540+0000 7f417ce90740 1 -- 172.21.15.120:0/3792732889 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55720bb69d70 con 0x55720bc16710 2024-09-19T09:13:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.540+0000 7f41737fe640 1 --2- 172.21.15.120:0/3792732889 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55720bc16710 0x55720bc81940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.540+0000 7f41737fe640 1 --2- 172.21.15.120:0/3792732889 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55720bc16710 0x55720bc81940 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d55485c1eae9ffb server_cookie=98776d4a2f751ef9 in_seq=0 out_seq=0 2024-09-19T09:13:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.540+0000 7f4152ffd640 10 client.19119 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.597+0000 7f4152ffd640 1 -- 172.21.15.120:0/3792732889 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55720bb69d70 con 0x55720bc16710 2024-09-19T09:13:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.597+0000 7f4152ffd640 10 client.19119 handle_command_reply: tid=0 2024-09-19T09:13:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.597+0000 7f417ce90740 1 client.19119 shutdown 2024-09-19T09:13:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.597+0000 7f417ce90740 1 -- 172.21.15.120:0/3792732889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55720bc014f0 msgr2=0x55720bc0dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.597+0000 7f417ce90740 1 --2- 172.21.15.120:0/3792732889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55720bc014f0 0x55720bc0dd10 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x55720bc0c250 tx=0x7f415c00f580 comp rx=0 tx=0).stop 2024-09-19T09:13:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.597+0000 7f417ce90740 1 -- 172.21.15.120:0/3792732889 shutdown_connections 2024-09-19T09:13:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.597+0000 7f417ce90740 1 --2- 172.21.15.120:0/3792732889 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55720bc16710 0x55720bc81940 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.597+0000 7f417ce90740 1 --2- 172.21.15.120:0/3792732889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55720bc06580 0x55720bc10d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.597+0000 7f417ce90740 1 --2- 172.21.15.120:0/3792732889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55720bc014f0 0x55720bc0dd10 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.597+0000 7f4152ffd640 0 client.19119 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.597+0000 7f417ce90740 1 --2- 172.21.15.120:0/3792732889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55720bc00bb0 0x55720bc0d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.597+0000 7f417ce90740 1 -- 172.21.15.120:0/3792732889 >> 172.21.15.120:0/3792732889 conn(0x55720bbfa010 msgr2=0x55720bbf91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.597+0000 7f417ce90740 1 -- 172.21.15.120:0/3792732889 shutdown_connections 2024-09-19T09:13:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.597+0000 7f417ce90740 1 -- 172.21.15.120:0/3792732889 wait complete. 2024-09-19T09:13:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.598+0000 7f417ce90740 20 client.19119 trim_cache size 0 max 16384 2024-09-19T09:13:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:13:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:13:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:13:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:13:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.598+0000 7f417a01e640 1 -- 172.21.15.120:0/1619150993 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f41600a32f0 msgr2=0x7f41600a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.598+0000 7f417a01e640 1 --2- 172.21.15.120:0/1619150993 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f41600a32f0 0x7f41600a57b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f416c009430 tx=0x7f416c009360 comp rx=0 tx=0).stop 2024-09-19T09:13:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.598+0000 7f417a01e640 1 -- 172.21.15.120:0/1619150993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4174100a40 msgr2=0x7f4174196c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.598+0000 7f417a01e640 1 --2- 172.21.15.120:0/1619150993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4174100a40 0x7f4174196c80 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f416400c970 tx=0x7f416400ce30 comp rx=0 tx=0).stop 2024-09-19T09:13:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.598+0000 7f417a01e640 1 -- 172.21.15.120:0/1619150993 shutdown_connections 2024-09-19T09:13:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.598+0000 7f417a01e640 1 --2- 172.21.15.120:0/1619150993 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f41600a32f0 0x7f41600a57b0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.598+0000 7f417a01e640 1 --2- 172.21.15.120:0/1619150993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41741025f0 0x7f417419e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.598+0000 7f417a01e640 1 --2- 172.21.15.120:0/1619150993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4174101c30 0x7f41741971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.598+0000 7f417a01e640 1 --2- 172.21.15.120:0/1619150993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4174100a40 0x7f4174196c80 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.598+0000 7f417a01e640 1 -- 172.21.15.120:0/1619150993 >> 172.21.15.120:0/1619150993 conn(0x7f41740fc1e0 msgr2=0x7f4174104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.598+0000 7f417a01e640 1 -- 172.21.15.120:0/1619150993 shutdown_connections 2024-09-19T09:13:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:11.598+0000 7f417a01e640 1 -- 172.21.15.120:0/1619150993 wait complete. 2024-09-19T09:13:12.069 INFO:tasks.workunit.client.0.smithi120.stdout: 1 984176 9.63 MB/sec execute 427 sec latency 26698.479 ms 2024-09-19T09:13:12.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:11 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1563912140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:13:12.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:11 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1563912140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:13:12.312 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:11 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1563912140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:13:12.346 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db --set-id b8a09801 --release --await' 2024-09-19T09:13:12.347 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db --set-id b8a09801 --release --await 2024-09-19T09:13:12.629 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:13:13.067 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:12 smithi120 ceph-mon[31150]: pgmap v653: 97 pgs: 97 active+clean; 39 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 2.4 MiB/s wr, 116 op/s 2024-09-19T09:13:13.069 INFO:tasks.workunit.client.0.smithi120.stdout: 1 984176 9.61 MB/sec execute 428 sec latency 27698.537 ms 2024-09-19T09:13:13.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.153+0000 7f7ed2a3e640 1 -- 172.21.15.120:0/1769475007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ecc100e20 msgr2=0x7f7ecc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:13.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.153+0000 7f7ed2a3e640 1 --2- 172.21.15.120:0/1769475007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ecc100e20 0x7f7ecc101200 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009900 tx=0x7f7ebc02a480 comp rx=0 tx=0).stop 2024-09-19T09:13:13.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.153+0000 7f7ed2a3e640 1 -- 172.21.15.120:0/1769475007 shutdown_connections 2024-09-19T09:13:13.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.153+0000 7f7ed2a3e640 1 --2- 172.21.15.120:0/1769475007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ecc102ef0 0x7f7ecc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.153+0000 7f7ed2a3e640 1 --2- 172.21.15.120:0/1769475007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ecc102570 0x7f7ecc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.153+0000 7f7ed2a3e640 1 --2- 172.21.15.120:0/1769475007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ecc100e20 0x7f7ecc101200 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.153+0000 7f7ed2a3e640 1 -- 172.21.15.120:0/1769475007 >> 172.21.15.120:0/1769475007 conn(0x7f7ecc0fc7d0 msgr2=0x7f7ecc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:13.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.153+0000 7f7ed2a3e640 1 -- 172.21.15.120:0/1769475007 shutdown_connections 2024-09-19T09:13:13.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.153+0000 7f7ed2a3e640 1 -- 172.21.15.120:0/1769475007 wait complete. 2024-09-19T09:13:13.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.153+0000 7f7ed2a3e640 1 Processor -- start 2024-09-19T09:13:13.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.153+0000 7f7ed2a3e640 1 -- start start 2024-09-19T09:13:13.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.153+0000 7f7ed2a3e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ecc100e20 0x7f7ecc1994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:13.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.153+0000 7f7ed2a3e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ecc102570 0x7f7ecc1999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:13.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.153+0000 7f7ed2a3e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ecc102ef0 0x7f7ecc1a4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:13.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.153+0000 7f7ed2a3e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc111020 con 0x7f7ecc102570 2024-09-19T09:13:13.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.153+0000 7f7ed2a3e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc110ea0 con 0x7f7ecc102ef0 2024-09-19T09:13:13.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.153+0000 7f7ed2a3e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1111a0 con 0x7f7ecc100e20 2024-09-19T09:13:13.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.154+0000 7f7ec3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ecc102570 0x7f7ecc1999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:13.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.154+0000 7f7ec3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ecc102570 0x7f7ecc1999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39808/0 (socket says 172.21.15.120:39808) 2024-09-19T09:13:13.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.154+0000 7f7ec3fff640 1 -- 172.21.15.120:0/2629036368 learned_addr learned my addr 172.21.15.120:0/2629036368 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:13.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.154+0000 7f7ecbfff640 1 --2- 172.21.15.120:0/2629036368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ecc100e20 0x7f7ecc1994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:13.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.154+0000 7f7ed0fdb640 1 --2- 172.21.15.120:0/2629036368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ecc102ef0 0x7f7ecc1a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:13.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.154+0000 7f7ec3fff640 1 -- 172.21.15.120:0/2629036368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ecc100e20 msgr2=0x7f7ecc1994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:13.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.154+0000 7f7ec3fff640 1 --2- 172.21.15.120:0/2629036368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ecc100e20 0x7f7ecc1994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.154+0000 7f7ec3fff640 1 -- 172.21.15.120:0/2629036368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ecc102ef0 msgr2=0x7f7ecc1a4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:13.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.154+0000 7f7ec3fff640 1 --2- 172.21.15.120:0/2629036368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ecc102ef0 0x7f7ecc1a4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.154+0000 7f7ec3fff640 1 -- 172.21.15.120:0/2629036368 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ecc1a4a70 con 0x7f7ecc102570 2024-09-19T09:13:13.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.154+0000 7f7ed0fdb640 1 --2- 172.21.15.120:0/2629036368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ecc102ef0 0x7f7ecc1a4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:13:13.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.154+0000 7f7ec3fff640 1 --2- 172.21.15.120:0/2629036368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ecc102570 0x7f7ecc1999f0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400b4a0 tx=0x7f7eb400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:13.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.154+0000 7f7ec9ffb640 1 -- 172.21.15.120:0/2629036368 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4014070 con 0x7f7ecc102570 2024-09-19T09:13:13.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.154+0000 7f7ec9ffb640 1 -- 172.21.15.120:0/2629036368 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eb4010af0 con 0x7f7ecc102570 2024-09-19T09:13:13.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.154+0000 7f7ed2a3e640 1 -- 172.21.15.120:0/2629036368 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ecc1a4c40 con 0x7f7ecc102570 2024-09-19T09:13:13.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.154+0000 7f7ec9ffb640 1 -- 172.21.15.120:0/2629036368 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4017d10 con 0x7f7ecc102570 2024-09-19T09:13:13.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.154+0000 7f7ed2a3e640 1 -- 172.21.15.120:0/2629036368 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ecc1a5150 con 0x7f7ecc102570 2024-09-19T09:13:13.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.156+0000 7f7ec9ffb640 1 -- 172.21.15.120:0/2629036368 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f7eb4017070 con 0x7f7ecc102570 2024-09-19T09:13:13.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.157+0000 7f7ed4e17740 1 Processor -- start 2024-09-19T09:13:13.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.157+0000 7f7ed4e17740 1 -- start start 2024-09-19T09:13:13.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.157+0000 7f7ed4e17740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d721005060 0x55d721005440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:13.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.157+0000 7f7ec9ffb640 1 --2- 172.21.15.120:0/2629036368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7eb80a3240 0x7f7eb80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:13.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.157+0000 7f7ed4e17740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d721005980 0x55d72100fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:13.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.157+0000 7f7ed4e17740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d721010240 0x55d721012630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:13.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.157+0000 7f7ed4e17740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d720f6a620 con 0x55d721010240 2024-09-19T09:13:13.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.157+0000 7f7ed4e17740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d720f11e50 con 0x55d721005980 2024-09-19T09:13:13.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.157+0000 7f7ed4e17740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d720f665e0 con 0x55d721005060 2024-09-19T09:13:13.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.157+0000 7f7ecbfff640 1 --2- 172.21.15.120:0/2629036368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7eb80a3240 0x7f7eb80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:13.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.157+0000 7f7ed0fdb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d721010240 0x55d721012630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:13.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.157+0000 7f7ec9ffb640 1 -- 172.21.15.120:0/2629036368 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f7eb40ced70 con 0x7f7ecc102570 2024-09-19T09:13:13.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.158+0000 7f7ed0fdb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d721010240 0x55d721012630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39818/0 (socket says 172.21.15.120:39818) 2024-09-19T09:13:13.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.158+0000 7f7ed0fdb640 1 -- 172.21.15.120:0/2760531416 learned_addr learned my addr 172.21.15.120:0/2760531416 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:13.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.158+0000 7f7ec3fff640 1 --2- 172.21.15.120:0/2760531416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d721005980 0x55d72100fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:13.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.158+0000 7f7ed0fdb640 1 --2- 172.21.15.120:0/2760531416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d721005060 0x55d721005440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:13.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.158+0000 7f7ed0fdb640 1 -- 172.21.15.120:0/2760531416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d721005060 msgr2=0x55d721005440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:13.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.158+0000 7f7ed0fdb640 1 --2- 172.21.15.120:0/2760531416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d721005060 0x55d721005440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.158+0000 7f7ed0fdb640 1 -- 172.21.15.120:0/2760531416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d721005980 msgr2=0x55d72100fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:13.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.158+0000 7f7ed0fdb640 1 --2- 172.21.15.120:0/2760531416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d721005980 0x55d72100fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.158+0000 7f7ed0fdb640 1 -- 172.21.15.120:0/2760531416 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d720f02d40 con 0x55d721010240 2024-09-19T09:13:13.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.158+0000 7f7ec3fff640 1 --2- 172.21.15.120:0/2760531416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d721005980 0x55d72100fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:13:13.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.158+0000 7f7ed0fdb640 1 --2- 172.21.15.120:0/2760531416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d721010240 0x55d721012630 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400af50 tx=0x7f7ec400a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6956657ba926c82 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:13.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.158+0000 7f7ecbfff640 1 --2- 172.21.15.120:0/2629036368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7eb80a3240 0x7f7eb80a5700 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc005ab0 tx=0x7f7ebc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:13.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.158+0000 7f7ec27fc640 1 -- 172.21.15.120:0/2760531416 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec400daf0 con 0x55d721010240 2024-09-19T09:13:13.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.158+0000 7f7ec27fc640 1 -- 172.21.15.120:0/2760531416 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ec400dc90 con 0x55d721010240 2024-09-19T09:13:13.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.158+0000 7f7ec27fc640 1 -- 172.21.15.120:0/2760531416 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec401e600 con 0x55d721010240 2024-09-19T09:13:13.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.159+0000 7f7ed4e17740 1 -- 172.21.15.120:0/2760531416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d721010240 msgr2=0x55d721012630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:13.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.159+0000 7f7ed4e17740 1 --2- 172.21.15.120:0/2760531416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d721010240 0x55d721012630 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400af50 tx=0x7f7ec400a890 comp rx=0 tx=0).stop 2024-09-19T09:13:13.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.159+0000 7f7ed4e17740 1 -- 172.21.15.120:0/2760531416 shutdown_connections 2024-09-19T09:13:13.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.159+0000 7f7ed4e17740 1 --2- 172.21.15.120:0/2760531416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d721010240 0x55d721012630 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.159+0000 7f7ed4e17740 1 --2- 172.21.15.120:0/2760531416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d721005980 0x55d72100fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.159+0000 7f7ed4e17740 1 --2- 172.21.15.120:0/2760531416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d721005060 0x55d721005440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.159+0000 7f7ed4e17740 1 -- 172.21.15.120:0/2760531416 >> 172.21.15.120:0/2760531416 conn(0x55d7210015f0 msgr2=0x55d7210019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:13.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.159+0000 7f7ed4e17740 1 -- 172.21.15.120:0/2760531416 shutdown_connections 2024-09-19T09:13:13.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.159+0000 7f7ed4e17740 1 -- 172.21.15.120:0/2760531416 wait complete. 2024-09-19T09:13:13.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.159+0000 7f7ed4e17740 1 -- start start 2024-09-19T09:13:13.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.159+0000 7f7ed4e17740 1 Processor -- start 2024-09-19T09:13:13.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.160+0000 7f7ed4e17740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d721005060 0x55d72100d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:13.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.160+0000 7f7ed4e17740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d721005980 0x55d72100d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:13.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.160+0000 7f7ed4e17740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d721010240 0x55d72100dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:13.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.160+0000 7f7ed4e17740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d721012e10 con 0x55d721005060 2024-09-19T09:13:13.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.160+0000 7f7ed4e17740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d720ef2f10 con 0x55d721010240 2024-09-19T09:13:13.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.160+0000 7f7ed4e17740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d721012f90 con 0x55d721005980 2024-09-19T09:13:13.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.160+0000 7f7ed4e17740 10 client.0 _ll_register_callbacks cb 0x55d720f16db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:13.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.160+0000 7f7ecbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d721005060 0x55d72100d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:13.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.160+0000 7f7ecbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d721005060 0x55d72100d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39834/0 (socket says 172.21.15.120:39834) 2024-09-19T09:13:13.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.160+0000 7f7ecbfff640 1 -- 172.21.15.120:0/3660456240 learned_addr learned my addr 172.21.15.120:0/3660456240 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:13.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.160+0000 7f7ed0fdb640 1 --2- 172.21.15.120:0/3660456240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d721010240 0x55d72100dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:13.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.160+0000 7f7ec3fff640 1 --2- 172.21.15.120:0/3660456240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d721005980 0x55d72100d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:13.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.160+0000 7f7ecbfff640 1 -- 172.21.15.120:0/3660456240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d721005980 msgr2=0x55d72100d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:13.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.160+0000 7f7ecbfff640 1 --2- 172.21.15.120:0/3660456240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d721005980 0x55d72100d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.160+0000 7f7ecbfff640 1 -- 172.21.15.120:0/3660456240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d721010240 msgr2=0x55d72100dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:13.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.160+0000 7f7ecbfff640 1 --2- 172.21.15.120:0/3660456240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d721010240 0x55d72100dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.160+0000 7f7ecbfff640 1 -- 172.21.15.120:0/3660456240 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d720c79580 con 0x55d721005060 2024-09-19T09:13:13.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.160+0000 7f7ecbfff640 1 --2- 172.21.15.120:0/3660456240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d721005060 0x55d72100d1f0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0096c0 tx=0x7f7ebc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49969e02586604a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:13.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.160+0000 7f7eab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:13.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.161+0000 7f7eab7fe640 1 -- 172.21.15.120:0/3660456240 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc04d020 con 0x55d721005060 2024-09-19T09:13:13.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.161+0000 7f7eab7fe640 1 -- 172.21.15.120:0/3660456240 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f7ebc031540 con 0x55d721005060 2024-09-19T09:13:13.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.161+0000 7f7ed4e17740 1 -- 172.21.15.120:0/3660456240 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ec400dc90 con 0x55d721005060 2024-09-19T09:13:13.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.161+0000 7f7eab7fe640 1 -- 172.21.15.120:0/3660456240 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7ebc031740 con 0x55d721005060 2024-09-19T09:13:13.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.161+0000 7f7ed4e17740 10 client.19131 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:13.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.161+0000 7f7ed4e17740 1 -- 172.21.15.120:0/3660456240 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d720d43960 con 0x55d721005060 2024-09-19T09:13:13.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.161+0000 7f7eab7fe640 1 -- 172.21.15.120:0/3660456240 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ec400dc90 con 0x55d721005060 2024-09-19T09:13:13.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.161+0000 7f7eab7fe640 1 -- 172.21.15.120:0/3660456240 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f7ebc049a10 con 0x55d721005060 2024-09-19T09:13:13.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.161+0000 7f7ed4e17740 10 client.19131 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:13.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.161+0000 7f7ed4e17740 10 client.19131 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:13:13.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.161+0000 7f7ed4e17740 20 client.19131 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:13.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.161+0000 7f7ed4e17740 1 --2- 172.21.15.120:0/3660456240 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d72101fdc0 0x55d7210221f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:13.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.161+0000 7f7ed4e17740 4 client.19131 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:13:13.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.161+0000 7f7ed4e17740 1 -- 172.21.15.120:0/3660456240 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d720f77b90 con 0x55d72101fdc0 2024-09-19T09:13:13.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.161+0000 7f7ec3fff640 1 --2- 172.21.15.120:0/3660456240 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d72101fdc0 0x55d7210221f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:13.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.162+0000 7f7ec3fff640 1 --2- 172.21.15.120:0/3660456240 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d72101fdc0 0x55d7210221f0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e80d2421cd4dd9b server_cookie=d0a269b3eb0d1087 in_seq=0 out_seq=0 2024-09-19T09:13:13.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.162+0000 7f7eab7fe640 10 client.19131 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:13.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.162+0000 7f7eab7fe640 1 -- 172.21.15.120:0/3660456240 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d720f77b90 con 0x55d72101fdc0 2024-09-19T09:13:13.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.162+0000 7f7eab7fe640 10 client.19131 handle_command_reply: tid=0 2024-09-19T09:13:13.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.162+0000 7f7ed4e17740 1 client.19131 shutdown 2024-09-19T09:13:13.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.163+0000 7f7ed4e17740 1 -- 172.21.15.120:0/3660456240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d721005060 msgr2=0x55d72100d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:13.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.163+0000 7f7ed4e17740 1 --2- 172.21.15.120:0/3660456240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d721005060 0x55d72100d1f0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0096c0 tx=0x7f7ebc009260 comp rx=0 tx=0).stop 2024-09-19T09:13:13.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.163+0000 7f7ed4e17740 1 -- 172.21.15.120:0/3660456240 shutdown_connections 2024-09-19T09:13:13.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.163+0000 7f7ed4e17740 1 --2- 172.21.15.120:0/3660456240 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d72101fdc0 0x55d7210221f0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.163+0000 7f7ed4e17740 1 --2- 172.21.15.120:0/3660456240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d721010240 0x55d72100dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.163+0000 7f7ed4e17740 1 --2- 172.21.15.120:0/3660456240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d721005980 0x55d72100d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.163+0000 7f7ed4e17740 1 --2- 172.21.15.120:0/3660456240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d721005060 0x55d72100d1f0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.163+0000 7f7ed4e17740 1 -- 172.21.15.120:0/3660456240 >> 172.21.15.120:0/3660456240 conn(0x55d7210015f0 msgr2=0x55d7210072d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:13.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.163+0000 7f7eab7fe640 0 client.19131 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:13.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.163+0000 7f7ed4e17740 1 -- 172.21.15.120:0/3660456240 shutdown_connections 2024-09-19T09:13:13.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.163+0000 7f7ed4e17740 1 -- 172.21.15.120:0/3660456240 wait complete. 2024-09-19T09:13:13.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.163+0000 7f7ed4e17740 20 client.19131 trim_cache size 0 max 16384 2024-09-19T09:13:13.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.180+0000 7f7ed4e17740 1 Processor -- start 2024-09-19T09:13:13.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.180+0000 7f7ed4e17740 1 -- start start 2024-09-19T09:13:13.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.180+0000 7f7ed4e17740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d721005060 0x55d7210086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:13.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.180+0000 7f7ed4e17740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d721005980 0x55d721000640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:13.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.180+0000 7f7ed4e17740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d721010240 0x55d721027b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:13.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.180+0000 7f7ed4e17740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1ab5b0 con 0x55d721005060 2024-09-19T09:13:13.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.180+0000 7f7ed4e17740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1ad0f0 con 0x55d721010240 2024-09-19T09:13:13.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.180+0000 7f7ed4e17740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1abc80 con 0x55d721005980 2024-09-19T09:13:13.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.181+0000 7f7ecbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d721005060 0x55d7210086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:13.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.181+0000 7f7ecbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d721005060 0x55d7210086f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39842/0 (socket says 172.21.15.120:39842) 2024-09-19T09:13:13.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.181+0000 7f7ecbfff640 1 -- 172.21.15.120:0/2731470445 learned_addr learned my addr 172.21.15.120:0/2731470445 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:13.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.181+0000 7f7ecbfff640 1 -- 172.21.15.120:0/2731470445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d721005980 msgr2=0x55d721000640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:13.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.181+0000 7f7ec3fff640 1 --2- 172.21.15.120:0/2731470445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d721005980 0x55d721000640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:13.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.181+0000 7f7ed0fdb640 1 --2- 172.21.15.120:0/2731470445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d721010240 0x55d721027b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:13.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.181+0000 7f7ecbfff640 1 --2- 172.21.15.120:0/2731470445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d721005980 0x55d721000640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.181+0000 7f7ecbfff640 1 -- 172.21.15.120:0/2731470445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d721010240 msgr2=0x55d721027b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:13.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.181+0000 7f7ecbfff640 1 --2- 172.21.15.120:0/2731470445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d721010240 0x55d721027b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.181+0000 7f7ecbfff640 1 -- 172.21.15.120:0/2731470445 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d72100fb60 con 0x55d721005060 2024-09-19T09:13:13.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.181+0000 7f7ec3fff640 1 --2- 172.21.15.120:0/2731470445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d721005980 0x55d721000640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:13:13.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.181+0000 7f7ecbfff640 1 --2- 172.21.15.120:0/2731470445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d721005060 0x55d7210086f0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc006010 tx=0x7f7ebc03c5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7287cb55184f034b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:13.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.181+0000 7f7ec1ffb640 1 -- 172.21.15.120:0/2731470445 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc04d070 con 0x55d721005060 2024-09-19T09:13:13.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.181+0000 7f7ec1ffb640 1 -- 172.21.15.120:0/2731470445 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ebc031dc0 con 0x55d721005060 2024-09-19T09:13:13.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.181+0000 7f7ec1ffb640 1 -- 172.21.15.120:0/2731470445 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc03c620 con 0x55d721005060 2024-09-19T09:13:13.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.182+0000 7f7ed4e17740 1 -- 172.21.15.120:0/2731470445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d721005060 msgr2=0x55d7210086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:13.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.182+0000 7f7ed4e17740 1 --2- 172.21.15.120:0/2731470445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d721005060 0x55d7210086f0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc006010 tx=0x7f7ebc03c5b0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.182+0000 7f7ed4e17740 1 -- 172.21.15.120:0/2731470445 shutdown_connections 2024-09-19T09:13:13.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.182+0000 7f7ed4e17740 1 --2- 172.21.15.120:0/2731470445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d721010240 0x55d721027b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.182+0000 7f7ed4e17740 1 --2- 172.21.15.120:0/2731470445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d721005980 0x55d721000640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.182+0000 7f7ed4e17740 1 --2- 172.21.15.120:0/2731470445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d721005060 0x55d7210086f0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.182+0000 7f7ed4e17740 1 -- 172.21.15.120:0/2731470445 >> 172.21.15.120:0/2731470445 conn(0x55d7210015f0 msgr2=0x55d720ffff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:13.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.182+0000 7f7ed4e17740 1 -- 172.21.15.120:0/2731470445 shutdown_connections 2024-09-19T09:13:13.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.182+0000 7f7ed4e17740 1 -- 172.21.15.120:0/2731470445 wait complete. 2024-09-19T09:13:13.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.182+0000 7f7ed4e17740 1 -- start start 2024-09-19T09:13:13.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.183+0000 7f7ed4e17740 1 Processor -- start 2024-09-19T09:13:13.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.183+0000 7f7ed4e17740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d721005060 0x55d721014a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:13.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.183+0000 7f7ed4e17740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d721005980 0x55d721014f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:13.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.183+0000 7f7ed4e17740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d721010240 0x55d721082aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:13.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.183+0000 7f7ed4e17740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1aca20 con 0x55d721010240 2024-09-19T09:13:13.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.183+0000 7f7ed4e17740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1ac350 con 0x55d721005060 2024-09-19T09:13:13.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.183+0000 7f7ed4e17740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1a9570 con 0x55d721005980 2024-09-19T09:13:13.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.183+0000 7f7ed4e17740 10 client.0 _ll_register_callbacks cb 0x7f7ebc004400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:13.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.183+0000 7f7ed0fdb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d721010240 0x55d721082aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:13.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.183+0000 7f7ed0fdb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d721010240 0x55d721082aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39846/0 (socket says 172.21.15.120:39846) 2024-09-19T09:13:13.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.183+0000 7f7ed0fdb640 1 -- 172.21.15.120:0/3851513246 learned_addr learned my addr 172.21.15.120:0/3851513246 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:13.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.183+0000 7f7ec3fff640 1 --2- 172.21.15.120:0/3851513246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d721005980 0x55d721014f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:13.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.183+0000 7f7ecbfff640 1 --2- 172.21.15.120:0/3851513246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d721005060 0x55d721014a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:13.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.183+0000 7f7ed0fdb640 1 -- 172.21.15.120:0/3851513246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d721005980 msgr2=0x55d721014f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:13.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.183+0000 7f7ed0fdb640 1 --2- 172.21.15.120:0/3851513246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d721005980 0x55d721014f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.183+0000 7f7ed0fdb640 1 -- 172.21.15.120:0/3851513246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d721005060 msgr2=0x55d721014a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:13.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.183+0000 7f7ed0fdb640 1 --2- 172.21.15.120:0/3851513246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d721005060 0x55d721014a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.183+0000 7f7ed0fdb640 1 -- 172.21.15.120:0/3851513246 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d721083140 con 0x55d721010240 2024-09-19T09:13:13.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.184+0000 7f7ed0fdb640 1 --2- 172.21.15.120:0/3851513246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d721010240 0x55d721082aa0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4004b90 tx=0x7f7ec40049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed960450f1967841 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:13.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.184+0000 7f7eab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:13.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.184+0000 7f7eab7fe640 1 -- 172.21.15.120:0/3851513246 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4013270 con 0x55d721010240 2024-09-19T09:13:13.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.184+0000 7f7eab7fe640 1 -- 172.21.15.120:0/3851513246 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f7ec4013410 con 0x55d721010240 2024-09-19T09:13:13.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.184+0000 7f7ed4e17740 1 -- 172.21.15.120:0/3851513246 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ebc031dc0 con 0x55d721010240 2024-09-19T09:13:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.184+0000 7f7eab7fe640 1 -- 172.21.15.120:0/3851513246 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7ec4013610 con 0x55d721010240 2024-09-19T09:13:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.184+0000 7f7ed4e17740 10 client.19137 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.184+0000 7f7ed4e17740 1 -- 172.21.15.120:0/3851513246 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d72108b610 con 0x55d721010240 2024-09-19T09:13:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.184+0000 7f7eab7fe640 1 -- 172.21.15.120:0/3851513246 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ebc031dc0 con 0x55d721010240 2024-09-19T09:13:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.184+0000 7f7eab7fe640 1 -- 172.21.15.120:0/3851513246 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f7ec4032050 con 0x55d721010240 2024-09-19T09:13:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.184+0000 7f7ed4e17740 10 client.19137 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.184+0000 7f7ed4e17740 10 client.19137 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:13:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.184+0000 7f7ed4e17740 20 client.19137 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.184+0000 7f7ed4e17740 1 --2- 172.21.15.120:0/3851513246 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d72108bb40 0x55d72108df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.184+0000 7f7ed4e17740 4 client.19137 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b8a09801", "release": true, "await": true}] 2024-09-19T09:13:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.184+0000 7f7ed4e17740 1 -- 172.21.15.120:0/3851513246 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b8a09801", "release": true, "await": true}) -- 0x55d720f720a0 con 0x55d72108bb40 2024-09-19T09:13:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.185+0000 7f7ecbfff640 1 --2- 172.21.15.120:0/3851513246 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d72108bb40 0x55d72108df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.185+0000 7f7eab7fe640 10 client.19137 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:13.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.185+0000 7f7ecbfff640 1 --2- 172.21.15.120:0/3851513246 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d72108bb40 0x55d72108df30 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c6035f24f6d0b22 server_cookie=9981b0b9bceff51 in_seq=0 out_seq=0 2024-09-19T09:13:13.195 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:12 smithi153 ceph-mon[35071]: pgmap v653: 97 pgs: 97 active+clean; 39 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 2.4 MiB/s wr, 116 op/s 2024-09-19T09:13:13.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:12 smithi196 ceph-mon[35150]: pgmap v653: 97 pgs: 97 active+clean; 39 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 2.4 MiB/s wr, 116 op/s 2024-09-19T09:13:13.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.195+0000 7f7eab7fe640 1 -- 172.21.15.120:0/3851513246 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d720f720a0 con 0x55d72108bb40 2024-09-19T09:13:13.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.195+0000 7f7eab7fe640 10 client.19137 handle_command_reply: tid=0 2024-09-19T09:13:13.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.195+0000 7f7ed4e17740 1 client.19137 shutdown 2024-09-19T09:13:13.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.196+0000 7f7ed4e17740 1 -- 172.21.15.120:0/3851513246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d721010240 msgr2=0x55d721082aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:13.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.196+0000 7f7ed4e17740 1 --2- 172.21.15.120:0/3851513246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d721010240 0x55d721082aa0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4004b90 tx=0x7f7ec40049e0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.196+0000 7f7ed4e17740 1 -- 172.21.15.120:0/3851513246 shutdown_connections 2024-09-19T09:13:13.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.196+0000 7f7ed4e17740 1 --2- 172.21.15.120:0/3851513246 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d72108bb40 0x55d72108df30 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.196+0000 7f7ed4e17740 1 --2- 172.21.15.120:0/3851513246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d721010240 0x55d721082aa0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.196+0000 7f7ed4e17740 1 --2- 172.21.15.120:0/3851513246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d721005980 0x55d721014f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.196+0000 7f7ed4e17740 1 --2- 172.21.15.120:0/3851513246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d721005060 0x55d721014a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.196+0000 7f7ed4e17740 1 -- 172.21.15.120:0/3851513246 >> 172.21.15.120:0/3851513246 conn(0x55d7210015f0 msgr2=0x55d7210072d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:13.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.196+0000 7f7eab7fe640 0 client.19137 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:13.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.196+0000 7f7ed4e17740 1 -- 172.21.15.120:0/3851513246 shutdown_connections 2024-09-19T09:13:13.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.196+0000 7f7ed4e17740 1 -- 172.21.15.120:0/3851513246 wait complete. 2024-09-19T09:13:13.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.196+0000 7f7ed4e17740 20 client.19137 trim_cache size 0 max 16384 2024-09-19T09:13:13.198 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:13:13.198 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 32, 2024-09-19T09:13:13.198 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:13:13.198 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 36, 2024-09-19T09:13:13.198 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:13:13.198 INFO:teuthology.orchestra.run.smithi120.stdout: "b8a09801": { 2024-09-19T09:13:13.198 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 36, 2024-09-19T09:13:13.198 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:13:13.198 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:13:13.198 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:13:13.198 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:13:13.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:13:13.199 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:13:13.199 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 150.0, 2024-09-19T09:13:13.199 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:13:13.199 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:13:13.199 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:13:13.199 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:13:13.199 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:13:13.199 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:13:13.199 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:13:13.199 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:13:13.199 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:13:13.199 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:13:13.199 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:13:13.199 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:13:13.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.196+0000 7f7ed2a3e640 1 -- 172.21.15.120:0/2629036368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7eb80a3240 msgr2=0x7f7eb80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:13.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.196+0000 7f7ed2a3e640 1 --2- 172.21.15.120:0/2629036368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7eb80a3240 0x7f7eb80a5700 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc005ab0 tx=0x7f7ebc005a20 comp rx=0 tx=0).stop 2024-09-19T09:13:13.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.196+0000 7f7ed2a3e640 1 -- 172.21.15.120:0/2629036368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ecc102570 msgr2=0x7f7ecc1999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:13.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.196+0000 7f7ed2a3e640 1 --2- 172.21.15.120:0/2629036368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ecc102570 0x7f7ecc1999f0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400b4a0 tx=0x7f7eb400b960 comp rx=0 tx=0).stop 2024-09-19T09:13:13.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.196+0000 7f7ed2a3e640 1 -- 172.21.15.120:0/2629036368 shutdown_connections 2024-09-19T09:13:13.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.196+0000 7f7ed2a3e640 1 --2- 172.21.15.120:0/2629036368 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7eb80a3240 0x7f7eb80a5700 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.196+0000 7f7ed2a3e640 1 --2- 172.21.15.120:0/2629036368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ecc102ef0 0x7f7ecc1a4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.196+0000 7f7ed2a3e640 1 --2- 172.21.15.120:0/2629036368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ecc102570 0x7f7ecc1999f0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.196+0000 7f7ed2a3e640 1 --2- 172.21.15.120:0/2629036368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ecc100e20 0x7f7ecc1994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:13.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.196+0000 7f7ed2a3e640 1 -- 172.21.15.120:0/2629036368 >> 172.21.15.120:0/2629036368 conn(0x7f7ecc0fc7d0 msgr2=0x7f7ecc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:13.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.196+0000 7f7ed2a3e640 1 -- 172.21.15.120:0/2629036368 shutdown_connections 2024-09-19T09:13:13.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:13.197+0000 7f7ed2a3e640 1 -- 172.21.15.120:0/2629036368 wait complete. 2024-09-19T09:13:14.015 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b8a09801', total seconds elapsed: 33.5 2024-09-19T09:13:14.015 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-19T09:13:14.015 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:13:14.069 INFO:tasks.workunit.client.0.smithi120.stdout: 1 985934 9.61 MB/sec execute 429 sec latency 27830.527 ms 2024-09-19T09:13:14.273 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:13:14.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.790+0000 7f58a425d640 1 -- 172.21.15.120:0/260704927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f589c101df0 msgr2=0x7f589c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:14.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.790+0000 7f58a425d640 1 --2- 172.21.15.120:0/260704927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f589c101df0 0x7f589c1021d0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f5898009ca0 tx=0x7f589802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:13:14.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.790+0000 7f58a425d640 1 -- 172.21.15.120:0/260704927 shutdown_connections 2024-09-19T09:13:14.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.790+0000 7f58a425d640 1 --2- 172.21.15.120:0/260704927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f589c106f20 0x7f589c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:14.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.790+0000 7f58a425d640 1 --2- 172.21.15.120:0/260704927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f589c1027a0 0x7f589c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:14.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.790+0000 7f58a425d640 1 --2- 172.21.15.120:0/260704927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f589c101df0 0x7f589c1021d0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:14.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.790+0000 7f58a425d640 1 -- 172.21.15.120:0/260704927 >> 172.21.15.120:0/260704927 conn(0x7f589c0fd8b0 msgr2=0x7f589c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:14.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.790+0000 7f58a425d640 1 -- 172.21.15.120:0/260704927 shutdown_connections 2024-09-19T09:13:14.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.791+0000 7f58a425d640 1 -- 172.21.15.120:0/260704927 wait complete. 2024-09-19T09:13:14.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.791+0000 7f58a425d640 1 Processor -- start 2024-09-19T09:13:14.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.791+0000 7f58a425d640 1 -- start start 2024-09-19T09:13:14.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.791+0000 7f58a425d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f589c101df0 0x7f589c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:14.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.791+0000 7f58a425d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f589c1027a0 0x7f589c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:14.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.791+0000 7f58a425d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f589c106f20 0x7f589c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:14.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.791+0000 7f58a425d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f589c111f80 con 0x7f589c1027a0 2024-09-19T09:13:14.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.791+0000 7f58a425d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f589c111e00 con 0x7f589c101df0 2024-09-19T09:13:14.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.791+0000 7f58a425d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f589c112100 con 0x7f589c106f20 2024-09-19T09:13:14.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.792+0000 7f58a27fa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f589c106f20 0x7f589c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:14.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.792+0000 7f58a1ff9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f589c101df0 0x7f589c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:14.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.792+0000 7f58a17f8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f589c1027a0 0x7f589c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:14.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.792+0000 7f58a1ff9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f589c101df0 0x7f589c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:43004/0 (socket says 172.21.15.120:43004) 2024-09-19T09:13:14.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.792+0000 7f58a27fa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f589c106f20 0x7f589c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:55058/0 (socket says 172.21.15.120:55058) 2024-09-19T09:13:14.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.792+0000 7f58a17f8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f589c1027a0 0x7f589c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39876/0 (socket says 172.21.15.120:39876) 2024-09-19T09:13:14.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.792+0000 7f58a1ff9640 1 -- 172.21.15.120:0/2024274164 learned_addr learned my addr 172.21.15.120:0/2024274164 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:14.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.792+0000 7f58a17f8640 1 -- 172.21.15.120:0/2024274164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f589c106f20 msgr2=0x7f589c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:14.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.792+0000 7f58a17f8640 1 --2- 172.21.15.120:0/2024274164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f589c106f20 0x7f589c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:14.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.792+0000 7f58a17f8640 1 -- 172.21.15.120:0/2024274164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f589c101df0 msgr2=0x7f589c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:14.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.792+0000 7f58a17f8640 1 --2- 172.21.15.120:0/2024274164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f589c101df0 0x7f589c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:14.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.792+0000 7f58a17f8640 1 -- 172.21.15.120:0/2024274164 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f589c1a6900 con 0x7f589c1027a0 2024-09-19T09:13:14.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.792+0000 7f58a1ff9640 1 --2- 172.21.15.120:0/2024274164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f589c101df0 0x7f589c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:13:14.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.792+0000 7f58a17f8640 1 --2- 172.21.15.120:0/2024274164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f589c1027a0 0x7f589c074a30 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f589000e9f0 tx=0x7f589000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:14.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.792+0000 7f588effd640 1 -- 172.21.15.120:0/2024274164 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589000ccc0 con 0x7f589c1027a0 2024-09-19T09:13:14.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.792+0000 7f588effd640 1 -- 172.21.15.120:0/2024274164 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5890010910 con 0x7f589c1027a0 2024-09-19T09:13:14.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.792+0000 7f58a425d640 1 -- 172.21.15.120:0/2024274164 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f589c1a6bf0 con 0x7f589c1027a0 2024-09-19T09:13:14.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.792+0000 7f588effd640 1 -- 172.21.15.120:0/2024274164 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5890018960 con 0x7f589c1027a0 2024-09-19T09:13:14.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.792+0000 7f58a425d640 1 -- 172.21.15.120:0/2024274164 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f589c1a7180 con 0x7f589c1027a0 2024-09-19T09:13:14.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.793+0000 7f58a425d640 1 -- 172.21.15.120:0/2024274164 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f589c1021d0 con 0x7f589c1027a0 2024-09-19T09:13:14.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.794+0000 7f588effd640 1 -- 172.21.15.120:0/2024274164 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5890010ac0 con 0x7f589c1027a0 2024-09-19T09:13:14.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.795+0000 7f588effd640 1 --2- 172.21.15.120:0/2024274164 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f58880a32c0 0x7f58880a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:14.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.795+0000 7f58a1ff9640 1 --2- 172.21.15.120:0/2024274164 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f58880a32c0 0x7f58880a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:14.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.795+0000 7f588effd640 1 -- 172.21.15.120:0/2024274164 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f58900cfc20 con 0x7f589c1027a0 2024-09-19T09:13:14.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.796+0000 7f58a1ff9640 1 --2- 172.21.15.120:0/2024274164 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f58880a32c0 0x7f58880a5780 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5898031f80 tx=0x7f5898004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:14.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:14.799+0000 7f588effd640 1 -- 172.21.15.120:0/2024274164 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f589009c2f0 con 0x7f589c1027a0 2024-09-19T09:13:15.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:15.017+0000 7f58a425d640 1 -- 172.21.15.120:0/2024274164 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f589c109b70 con 0x7f589c1027a0 2024-09-19T09:13:15.018 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:14 smithi120 ceph-mon[31150]: pgmap v654: 97 pgs: 97 active+clean; 35 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 2.4 MiB/s wr, 117 op/s 2024-09-19T09:13:15.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:15.018+0000 7f588effd640 1 -- 172.21.15.120:0/2024274164 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11988 (secure 0 0 0) 0x7f58900a05f0 con 0x7f589c1027a0 2024-09-19T09:13:15.019 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:13:15.019 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":32,"btime":"2024-09-19T09:11:12:904684+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:11:12.904670+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27518":{"gid":27518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3214776406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3214776406},{"type":"v1","addr":"172.21.15.153:6835","nonce":3214776406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:13:15.020 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 32 2024-09-19T09:13:15.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:15.021+0000 7f58a425d640 1 -- 172.21.15.120:0/2024274164 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f58880a32c0 msgr2=0x7f58880a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:15.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:15.021+0000 7f58a425d640 1 --2- 172.21.15.120:0/2024274164 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f58880a32c0 0x7f58880a5780 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5898031f80 tx=0x7f5898004a40 comp rx=0 tx=0).stop 2024-09-19T09:13:15.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:15.021+0000 7f58a425d640 1 -- 172.21.15.120:0/2024274164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f589c1027a0 msgr2=0x7f589c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:15.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:15.021+0000 7f58a425d640 1 --2- 172.21.15.120:0/2024274164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f589c1027a0 0x7f589c074a30 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f589000e9f0 tx=0x7f589000eeb0 comp rx=0 tx=0).stop 2024-09-19T09:13:15.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:15.021+0000 7f58a425d640 1 -- 172.21.15.120:0/2024274164 shutdown_connections 2024-09-19T09:13:15.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:15.021+0000 7f58a425d640 1 --2- 172.21.15.120:0/2024274164 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f58880a32c0 0x7f58880a5780 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:15.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:15.021+0000 7f58a425d640 1 --2- 172.21.15.120:0/2024274164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f589c106f20 0x7f589c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:15.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:15.021+0000 7f58a425d640 1 --2- 172.21.15.120:0/2024274164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f589c1027a0 0x7f589c074a30 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:15.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:15.021+0000 7f58a425d640 1 --2- 172.21.15.120:0/2024274164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f589c101df0 0x7f589c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:15.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:15.021+0000 7f58a425d640 1 -- 172.21.15.120:0/2024274164 >> 172.21.15.120:0/2024274164 conn(0x7f589c0fd8b0 msgr2=0x7f589c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:15.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:15.021+0000 7f58a425d640 1 -- 172.21.15.120:0/2024274164 shutdown_connections 2024-09-19T09:13:15.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:15.021+0000 7f58a425d640 1 -- 172.21.15.120:0/2024274164 wait complete. 2024-09-19T09:13:15.069 INFO:tasks.workunit.client.0.smithi120.stdout: 1 988598 9.62 MB/sec execute 430 sec latency 16.424 ms 2024-09-19T09:13:15.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:14 smithi196 ceph-mon[35150]: pgmap v654: 97 pgs: 97 active+clean; 35 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 2.4 MiB/s wr, 117 op/s 2024-09-19T09:13:15.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:14 smithi153 ceph-mon[35071]: pgmap v654: 97 pgs: 97 active+clean; 35 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 2.4 MiB/s wr, 117 op/s 2024-09-19T09:13:15.715 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-19T09:13:15.716 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-19T09:13:15.976 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:13:16.069 INFO:tasks.workunit.client.0.smithi120.stdout: 1 991686 9.64 MB/sec execute 431 sec latency 12.228 ms 2024-09-19T09:13:16.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:15 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2024274164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:13:16.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:15 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2024274164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:13:16.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:15 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2024274164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:13:16.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.567+0000 7f91400a0640 1 -- 172.21.15.120:0/54524167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9138102ef0 msgr2=0x7f913810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:16.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.567+0000 7f91400a0640 1 --2- 172.21.15.120:0/54524167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9138102ef0 0x7f913810da40 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f9130009ca0 tx=0x7f913002a950 comp rx=0 tx=0).stop 2024-09-19T09:13:16.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.567+0000 7f91400a0640 1 -- 172.21.15.120:0/54524167 shutdown_connections 2024-09-19T09:13:16.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.567+0000 7f91400a0640 1 --2- 172.21.15.120:0/54524167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9138102ef0 0x7f913810da40 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.567+0000 7f91400a0640 1 --2- 172.21.15.120:0/54524167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9138102570 0x7f91381029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.567+0000 7f91400a0640 1 --2- 172.21.15.120:0/54524167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9138100e20 0x7f9138101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.567+0000 7f91400a0640 1 -- 172.21.15.120:0/54524167 >> 172.21.15.120:0/54524167 conn(0x7f91380fc7d0 msgr2=0x7f91380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:16.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.567+0000 7f91400a0640 1 -- 172.21.15.120:0/54524167 shutdown_connections 2024-09-19T09:13:16.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.567+0000 7f91400a0640 1 -- 172.21.15.120:0/54524167 wait complete. 2024-09-19T09:13:16.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.568+0000 7f91400a0640 1 Processor -- start 2024-09-19T09:13:16.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.568+0000 7f91400a0640 1 -- start start 2024-09-19T09:13:16.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.569+0000 7f91400a0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9138100e20 0x7f9138199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:16.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.569+0000 7f91400a0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9138102570 0x7f91381998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:16.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.569+0000 7f91400a0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9138102ef0 0x7f91381a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:16.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.569+0000 7f913de3c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9138100e20 0x7f9138199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:16.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.569+0000 7f913de3c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9138100e20 0x7f9138199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39892/0 (socket says 172.21.15.120:39892) 2024-09-19T09:13:16.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.569+0000 7f913de3c640 1 -- 172.21.15.120:0/2566192042 learned_addr learned my addr 172.21.15.120:0/2566192042 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:16.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.569+0000 7f91400a0640 1 -- 172.21.15.120:0/2566192042 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9138110f80 con 0x7f9138100e20 2024-09-19T09:13:16.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.569+0000 7f913d63b640 1 --2- 172.21.15.120:0/2566192042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9138102570 0x7f91381998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:16.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.569+0000 7f91400a0640 1 -- 172.21.15.120:0/2566192042 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9138110e00 con 0x7f9138102ef0 2024-09-19T09:13:16.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.569+0000 7f91400a0640 1 -- 172.21.15.120:0/2566192042 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9138111100 con 0x7f9138102570 2024-09-19T09:13:16.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.569+0000 7f913e63d640 1 --2- 172.21.15.120:0/2566192042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9138102ef0 0x7f91381a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:16.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.570+0000 7f913de3c640 1 -- 172.21.15.120:0/2566192042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9138102570 msgr2=0x7f91381998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:16.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.570+0000 7f913de3c640 1 --2- 172.21.15.120:0/2566192042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9138102570 0x7f91381998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.570+0000 7f913de3c640 1 -- 172.21.15.120:0/2566192042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9138102ef0 msgr2=0x7f91381a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:16.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.570+0000 7f913de3c640 1 --2- 172.21.15.120:0/2566192042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9138102ef0 0x7f91381a43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.570+0000 7f913de3c640 1 -- 172.21.15.120:0/2566192042 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91381a4930 con 0x7f9138100e20 2024-09-19T09:13:16.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.570+0000 7f913d63b640 1 --2- 172.21.15.120:0/2566192042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9138102570 0x7f91381998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:13:16.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.570+0000 7f913de3c640 1 --2- 172.21.15.120:0/2566192042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9138100e20 0x7f9138199370 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f913400d770 tx=0x7f913400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:16.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.570+0000 7f912effd640 1 -- 172.21.15.120:0/2566192042 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134017070 con 0x7f9138100e20 2024-09-19T09:13:16.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.570+0000 7f912effd640 1 -- 172.21.15.120:0/2566192042 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9134005500 con 0x7f9138100e20 2024-09-19T09:13:16.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.570+0000 7f91400a0640 1 -- 172.21.15.120:0/2566192042 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91381a4b00 con 0x7f9138100e20 2024-09-19T09:13:16.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.570+0000 7f912effd640 1 -- 172.21.15.120:0/2566192042 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134004e00 con 0x7f9138100e20 2024-09-19T09:13:16.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.570+0000 7f91400a0640 1 -- 172.21.15.120:0/2566192042 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91381a5010 con 0x7f9138100e20 2024-09-19T09:13:16.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.573+0000 7f912effd640 1 -- 172.21.15.120:0/2566192042 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f9134004fa0 con 0x7f9138100e20 2024-09-19T09:13:16.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.573+0000 7f9142479740 1 Processor -- start 2024-09-19T09:13:16.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.573+0000 7f9142479740 1 -- start start 2024-09-19T09:13:16.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.574+0000 7f9142479740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d294bab70 0x558d294baf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:16.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.574+0000 7f9142479740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d294bb490 0x558d294c5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:16.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.574+0000 7f9142479740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d294c5d50 0x558d294c8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:16.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.574+0000 7f9142479740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558d2941c2a0 con 0x558d294bab70 2024-09-19T09:13:16.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.574+0000 7f9142479740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558d293c3b10 con 0x558d294c5d50 2024-09-19T09:13:16.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.574+0000 7f9142479740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558d29418260 con 0x558d294bb490 2024-09-19T09:13:16.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.574+0000 7f912effd640 1 --2- 172.21.15.120:0/2566192042 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f91240a32c0 0x7f91240a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:16.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.574+0000 7f913e63d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d294bab70 0x558d294baf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:16.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.574+0000 7f913e63d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d294bab70 0x558d294baf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39894/0 (socket says 172.21.15.120:39894) 2024-09-19T09:13:16.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.574+0000 7f913e63d640 1 -- 172.21.15.120:0/862039320 learned_addr learned my addr 172.21.15.120:0/862039320 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:16.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.574+0000 7f913e63d640 1 --2- 172.21.15.120:0/2566192042 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f91240a32c0 0x7f91240a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:16.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.574+0000 7f913d63b640 1 --2- 172.21.15.120:0/862039320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d294c5d50 0x558d294c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:16.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.574+0000 7f912effd640 1 -- 172.21.15.120:0/2566192042 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f91340cf270 con 0x7f9138100e20 2024-09-19T09:13:16.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.574+0000 7f913de3c640 1 --2- 172.21.15.120:0/862039320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d294bb490 0x558d294c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:16.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.574+0000 7f913d63b640 1 -- 172.21.15.120:0/862039320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d294bb490 msgr2=0x558d294c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:16.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.575+0000 7f913d63b640 1 --2- 172.21.15.120:0/862039320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d294bb490 0x558d294c5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.575+0000 7f913d63b640 1 -- 172.21.15.120:0/862039320 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d294bab70 msgr2=0x558d294baf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:16.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.575+0000 7f913d63b640 1 --2- 172.21.15.120:0/862039320 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d294bab70 0x558d294baf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.575+0000 7f913d63b640 1 -- 172.21.15.120:0/862039320 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d2918db90 con 0x558d294c5d50 2024-09-19T09:13:16.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.575+0000 7f913e63d640 1 --2- 172.21.15.120:0/862039320 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d294bab70 0x558d294baf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:13:16.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.575+0000 7f913de3c640 1 --2- 172.21.15.120:0/862039320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d294bb490 0x558d294c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:13:16.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.575+0000 7f913e63d640 1 --2- 172.21.15.120:0/2566192042 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f91240a32c0 0x7f91240a5780 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9130005fe0 tx=0x7f9130005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:16.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.575+0000 7f913d63b640 1 --2- 172.21.15.120:0/862039320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d294c5d50 0x558d294c8140 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9128009f20 tx=0x7f91280118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5f44a813ac0e7d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:16.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.575+0000 7f9123fff640 1 -- 172.21.15.120:0/862039320 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9128004030 con 0x558d294c5d50 2024-09-19T09:13:16.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.575+0000 7f9123fff640 1 -- 172.21.15.120:0/862039320 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9128005500 con 0x558d294c5d50 2024-09-19T09:13:16.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.575+0000 7f9123fff640 1 -- 172.21.15.120:0/862039320 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9128004d40 con 0x558d294c5d50 2024-09-19T09:13:16.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.576+0000 7f9142479740 1 -- 172.21.15.120:0/862039320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d294c5d50 msgr2=0x558d294c8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:16.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.576+0000 7f9142479740 1 --2- 172.21.15.120:0/862039320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d294c5d50 0x558d294c8140 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9128009f20 tx=0x7f91280118d0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.576+0000 7f9142479740 1 -- 172.21.15.120:0/862039320 shutdown_connections 2024-09-19T09:13:16.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.576+0000 7f9142479740 1 --2- 172.21.15.120:0/862039320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d294c5d50 0x558d294c8140 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.576+0000 7f9142479740 1 --2- 172.21.15.120:0/862039320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d294bb490 0x558d294c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.576+0000 7f9142479740 1 --2- 172.21.15.120:0/862039320 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d294bab70 0x558d294baf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.577+0000 7f9142479740 1 -- 172.21.15.120:0/862039320 >> 172.21.15.120:0/862039320 conn(0x558d294b7100 msgr2=0x558d294b74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:16.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.577+0000 7f9142479740 1 -- 172.21.15.120:0/862039320 shutdown_connections 2024-09-19T09:13:16.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.577+0000 7f9142479740 1 -- 172.21.15.120:0/862039320 wait complete. 2024-09-19T09:13:16.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.578+0000 7f9142479740 1 -- start start 2024-09-19T09:13:16.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.578+0000 7f9142479740 1 Processor -- start 2024-09-19T09:13:16.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.578+0000 7f9142479740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d294bab70 0x558d294c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:16.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.578+0000 7f9142479740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d294bb490 0x558d294c30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:16.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.578+0000 7f9142479740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d294c5d50 0x558d294c3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:16.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.578+0000 7f9142479740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558d294c8920 con 0x558d294bab70 2024-09-19T09:13:16.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.578+0000 7f9142479740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558d293a46e0 con 0x558d294c5d50 2024-09-19T09:13:16.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.578+0000 7f9142479740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558d294c8aa0 con 0x558d294bb490 2024-09-19T09:13:16.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.578+0000 7f9142479740 10 client.0 _ll_register_callbacks cb 0x558d293c8a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:16.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.578+0000 7f913de3c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d294bab70 0x558d294c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:16.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.578+0000 7f913de3c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d294bab70 0x558d294c2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39910/0 (socket says 172.21.15.120:39910) 2024-09-19T09:13:16.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.578+0000 7f913de3c640 1 -- 172.21.15.120:0/3387299255 learned_addr learned my addr 172.21.15.120:0/3387299255 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:16.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.578+0000 7f913e63d640 1 --2- 172.21.15.120:0/3387299255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d294c5d50 0x558d294c3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:16.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.578+0000 7f913d63b640 1 --2- 172.21.15.120:0/3387299255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d294bb490 0x558d294c30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:16.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.579+0000 7f913de3c640 1 -- 172.21.15.120:0/3387299255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d294bb490 msgr2=0x558d294c30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:16.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.579+0000 7f913de3c640 1 --2- 172.21.15.120:0/3387299255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d294bb490 0x558d294c30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.579+0000 7f913de3c640 1 -- 172.21.15.120:0/3387299255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d294c5d50 msgr2=0x558d294c3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:16.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.579+0000 7f913de3c640 1 --2- 172.21.15.120:0/3387299255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d294c5d50 0x558d294c3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.579+0000 7f913de3c640 1 -- 172.21.15.120:0/3387299255 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d294d1df0 con 0x558d294bab70 2024-09-19T09:13:16.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.579+0000 7f913d63b640 1 --2- 172.21.15.120:0/3387299255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d294bb490 0x558d294c30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:13:16.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.579+0000 7f913de3c640 1 --2- 172.21.15.120:0/3387299255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d294bab70 0x558d294c2b90 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f9134097da0 tx=0x7f91340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e31b51f2a0dda3b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:16.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.579+0000 7f91217fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:16.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.579+0000 7f91217fa640 1 -- 172.21.15.120:0/3387299255 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134020040 con 0x558d294bab70 2024-09-19T09:13:16.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.579+0000 7f91217fa640 1 -- 172.21.15.120:0/3387299255 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f91340cce80 con 0x558d294bab70 2024-09-19T09:13:16.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.579+0000 7f9142479740 1 -- 172.21.15.120:0/3387299255 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9128005500 con 0x558d294bab70 2024-09-19T09:13:16.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.579+0000 7f91217fa640 1 -- 172.21.15.120:0/3387299255 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f91340b4050 con 0x558d294bab70 2024-09-19T09:13:16.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.579+0000 7f9142479740 10 client.19155 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:16.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.580+0000 7f9142479740 1 -- 172.21.15.120:0/3387299255 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558d294d46c0 con 0x558d294bab70 2024-09-19T09:13:16.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.580+0000 7f91217fa640 1 -- 172.21.15.120:0/3387299255 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9128005500 con 0x558d294bab70 2024-09-19T09:13:16.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.580+0000 7f91217fa640 1 -- 172.21.15.120:0/3387299255 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f91340c9650 con 0x558d294bab70 2024-09-19T09:13:16.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.580+0000 7f9142479740 10 client.19155 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:16.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.580+0000 7f9142479740 10 client.19155 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:13:16.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.580+0000 7f9142479740 20 client.19155 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:16.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.580+0000 7f9142479740 1 --2- 172.21.15.120:0/3387299255 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558d294d5ad0 0x558d294d7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:16.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.580+0000 7f9142479740 4 client.19155 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:13:16.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.580+0000 7f9142479740 1 -- 172.21.15.120:0/3387299255 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d294299d0 con 0x558d294d5ad0 2024-09-19T09:13:16.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.580+0000 7f913d63b640 1 --2- 172.21.15.120:0/3387299255 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558d294d5ad0 0x558d294d7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:16.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.581+0000 7f91217fa640 10 client.19155 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:16.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.581+0000 7f913d63b640 1 --2- 172.21.15.120:0/3387299255 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558d294d5ad0 0x558d294d7f40 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c704cf14fd789c9 server_cookie=b0eed5dadc7ba4fc in_seq=0 out_seq=0 2024-09-19T09:13:16.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.583+0000 7f91217fa640 1 -- 172.21.15.120:0/3387299255 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558d294299d0 con 0x558d294d5ad0 2024-09-19T09:13:16.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.583+0000 7f91217fa640 10 client.19155 handle_command_reply: tid=0 2024-09-19T09:13:16.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.583+0000 7f9142479740 1 client.19155 shutdown 2024-09-19T09:13:16.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.583+0000 7f9142479740 1 -- 172.21.15.120:0/3387299255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d294bab70 msgr2=0x558d294c2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:16.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.583+0000 7f9142479740 1 --2- 172.21.15.120:0/3387299255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d294bab70 0x558d294c2b90 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f9134097da0 tx=0x7f91340046c0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.583+0000 7f9142479740 1 -- 172.21.15.120:0/3387299255 shutdown_connections 2024-09-19T09:13:16.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.583+0000 7f9142479740 1 --2- 172.21.15.120:0/3387299255 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558d294d5ad0 0x558d294d7f40 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.583+0000 7f9142479740 1 --2- 172.21.15.120:0/3387299255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d294c5d50 0x558d294c3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.583+0000 7f9142479740 1 --2- 172.21.15.120:0/3387299255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d294bb490 0x558d294c30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.583+0000 7f9142479740 1 --2- 172.21.15.120:0/3387299255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d294bab70 0x558d294c2b90 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.583+0000 7f9142479740 1 -- 172.21.15.120:0/3387299255 >> 172.21.15.120:0/3387299255 conn(0x558d294b7100 msgr2=0x558d294b1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:16.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.583+0000 7f91217fa640 0 client.19155 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:16.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.583+0000 7f9142479740 1 -- 172.21.15.120:0/3387299255 shutdown_connections 2024-09-19T09:13:16.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.583+0000 7f9142479740 1 -- 172.21.15.120:0/3387299255 wait complete. 2024-09-19T09:13:16.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.583+0000 7f9142479740 20 client.19155 trim_cache size 0 max 16384 2024-09-19T09:13:16.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.601+0000 7f9142479740 1 Processor -- start 2024-09-19T09:13:16.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.601+0000 7f9142479740 1 -- start start 2024-09-19T09:13:16.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.601+0000 7f9142479740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d294bab70 0x558d294c1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:16.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.601+0000 7f9142479740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d294bb490 0x558d294dd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:16.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.601+0000 7f9142479740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d294c5d50 0x558d294ddd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:16.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.601+0000 7f9142479740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f91381ab330 con 0x558d294bb490 2024-09-19T09:13:16.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.601+0000 7f9142479740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f91381ace70 con 0x558d294bab70 2024-09-19T09:13:16.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.601+0000 7f9142479740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f91381aba00 con 0x558d294c5d50 2024-09-19T09:13:16.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.601+0000 7f913d63b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d294bb490 0x558d294dd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:16.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.601+0000 7f913de3c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d294bab70 0x558d294c1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:16.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.601+0000 7f913d63b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d294bb490 0x558d294dd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39918/0 (socket says 172.21.15.120:39918) 2024-09-19T09:13:16.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.601+0000 7f913d63b640 1 -- 172.21.15.120:0/4081139004 learned_addr learned my addr 172.21.15.120:0/4081139004 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:16.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.602+0000 7f913e63d640 1 --2- 172.21.15.120:0/4081139004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d294c5d50 0x558d294ddd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:16.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.602+0000 7f913de3c640 1 -- 172.21.15.120:0/4081139004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d294c5d50 msgr2=0x558d294ddd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:16.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.602+0000 7f913de3c640 1 --2- 172.21.15.120:0/4081139004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d294c5d50 0x558d294ddd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.602+0000 7f913de3c640 1 -- 172.21.15.120:0/4081139004 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d294bb490 msgr2=0x558d294dd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:16.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.602+0000 7f913de3c640 1 --2- 172.21.15.120:0/4081139004 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d294bb490 0x558d294dd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.602+0000 7f913de3c640 1 -- 172.21.15.120:0/4081139004 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d294b6f30 con 0x558d294bab70 2024-09-19T09:13:16.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.602+0000 7f913d63b640 1 --2- 172.21.15.120:0/4081139004 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d294bb490 0x558d294dd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:13:16.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.602+0000 7f913de3c640 1 --2- 172.21.15.120:0/4081139004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d294bab70 0x558d294c1ad0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9134009610 tx=0x7f913409b550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=706f3e8bed2b740b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:16.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.602+0000 7f91237fe640 1 -- 172.21.15.120:0/4081139004 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134020070 con 0x558d294bab70 2024-09-19T09:13:16.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.602+0000 7f91237fe640 1 -- 172.21.15.120:0/4081139004 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91340ba290 con 0x558d294bab70 2024-09-19T09:13:16.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.602+0000 7f91237fe640 1 -- 172.21.15.120:0/4081139004 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91340b9080 con 0x558d294bab70 2024-09-19T09:13:16.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.603+0000 7f9142479740 1 -- 172.21.15.120:0/4081139004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d294bab70 msgr2=0x558d294c1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:16.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.603+0000 7f9142479740 1 --2- 172.21.15.120:0/4081139004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d294bab70 0x558d294c1ad0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9134009610 tx=0x7f913409b550 comp rx=0 tx=0).stop 2024-09-19T09:13:16.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.603+0000 7f9142479740 1 -- 172.21.15.120:0/4081139004 shutdown_connections 2024-09-19T09:13:16.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.603+0000 7f9142479740 1 --2- 172.21.15.120:0/4081139004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d294c5d50 0x558d294ddd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.603+0000 7f9142479740 1 --2- 172.21.15.120:0/4081139004 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d294bb490 0x558d294dd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.603+0000 7f9142479740 1 --2- 172.21.15.120:0/4081139004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d294bab70 0x558d294c1ad0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.603+0000 7f9142479740 1 -- 172.21.15.120:0/4081139004 >> 172.21.15.120:0/4081139004 conn(0x558d294b7100 msgr2=0x558d294b1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:16.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.603+0000 7f9142479740 1 -- 172.21.15.120:0/4081139004 shutdown_connections 2024-09-19T09:13:16.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.603+0000 7f9142479740 1 -- 172.21.15.120:0/4081139004 wait complete. 2024-09-19T09:13:16.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.603+0000 7f9142479740 1 -- start start 2024-09-19T09:13:16.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.604+0000 7f9142479740 1 Processor -- start 2024-09-19T09:13:16.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.604+0000 7f9142479740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d294bab70 0x558d294ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:16.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.604+0000 7f9142479740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d294bb490 0x558d294c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:16.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.604+0000 7f9142479740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d294c5d50 0x558d294ca130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:16.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.604+0000 7f9142479740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f91381ac7a0 con 0x558d294bab70 2024-09-19T09:13:16.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.604+0000 7f9142479740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f91381ac0d0 con 0x558d294bb490 2024-09-19T09:13:16.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.604+0000 7f9142479740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f91381a9310 con 0x558d294c5d50 2024-09-19T09:13:16.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.604+0000 7f9142479740 10 client.0 _ll_register_callbacks cb 0x7f913409b4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:16.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.604+0000 7f913de3c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d294bab70 0x558d294ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:16.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.604+0000 7f913de3c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d294bab70 0x558d294ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39932/0 (socket says 172.21.15.120:39932) 2024-09-19T09:13:16.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.604+0000 7f913de3c640 1 -- 172.21.15.120:0/1281220463 learned_addr learned my addr 172.21.15.120:0/1281220463 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:16.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.604+0000 7f913d63b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d294bb490 0x558d294c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:16.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.605+0000 7f913e63d640 1 --2- 172.21.15.120:0/1281220463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d294c5d50 0x558d294ca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:16.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.605+0000 7f913de3c640 1 -- 172.21.15.120:0/1281220463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d294c5d50 msgr2=0x558d294ca130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:16.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.605+0000 7f913de3c640 1 --2- 172.21.15.120:0/1281220463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d294c5d50 0x558d294ca130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.605+0000 7f913de3c640 1 -- 172.21.15.120:0/1281220463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d294bb490 msgr2=0x558d294c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:16.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.605+0000 7f913de3c640 1 --2- 172.21.15.120:0/1281220463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d294bb490 0x558d294c99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.605+0000 7f913de3c640 1 -- 172.21.15.120:0/1281220463 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d294ae260 con 0x558d294bab70 2024-09-19T09:13:16.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.605+0000 7f913de3c640 1 --2- 172.21.15.120:0/1281220463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d294bab70 0x558d294ce670 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f9134099bf0 tx=0x7f91340c9560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f823935feeb68a6a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:16.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.605+0000 7f91217fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:16.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.605+0000 7f91217fa640 1 -- 172.21.15.120:0/1281220463 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91340b9070 con 0x558d294bab70 2024-09-19T09:13:16.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.605+0000 7f91217fa640 1 -- 172.21.15.120:0/1281220463 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f9134099c40 con 0x558d294bab70 2024-09-19T09:13:16.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.605+0000 7f9142479740 1 -- 172.21.15.120:0/1281220463 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558d29423d20 con 0x558d294bab70 2024-09-19T09:13:16.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.605+0000 7f91217fa640 1 -- 172.21.15.120:0/1281220463 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f91340a9050 con 0x558d294bab70 2024-09-19T09:13:16.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.605+0000 7f9142479740 10 client.19161 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:16.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.605+0000 7f9142479740 1 -- 172.21.15.120:0/1281220463 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558d295407f0 con 0x558d294bab70 2024-09-19T09:13:16.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.605+0000 7f91217fa640 1 -- 172.21.15.120:0/1281220463 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558d29423d20 con 0x558d294bab70 2024-09-19T09:13:16.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.606+0000 7f91217fa640 1 -- 172.21.15.120:0/1281220463 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f91340b4a90 con 0x558d294bab70 2024-09-19T09:13:16.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.606+0000 7f9142479740 10 client.19161 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:16.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.606+0000 7f9142479740 10 client.19161 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:13:16.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.606+0000 7f9142479740 20 client.19161 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:16.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.606+0000 7f9142479740 1 --2- 172.21.15.120:0/1281220463 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558d29540d30 0x558d29543120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:16.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.606+0000 7f9142479740 4 client.19161 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-19T09:13:16.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.606+0000 7f9142479740 1 -- 172.21.15.120:0/1281220463 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x558d2939fec0 con 0x558d29540d30 2024-09-19T09:13:16.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.606+0000 7f913d63b640 1 --2- 172.21.15.120:0/1281220463 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558d29540d30 0x558d29543120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:16.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.606+0000 7f913d63b640 1 --2- 172.21.15.120:0/1281220463 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558d29540d30 0x558d29543120 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cca68c84622fdc5 server_cookie=1e1d9ef35ce6ed6e in_seq=0 out_seq=0 2024-09-19T09:13:16.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.606+0000 7f91217fa640 10 client.19161 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:16.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.693+0000 7f91217fa640 1 -- 172.21.15.120:0/1281220463 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x558d2939fec0 con 0x558d29540d30 2024-09-19T09:13:16.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.693+0000 7f91217fa640 10 client.19161 handle_command_reply: tid=0 2024-09-19T09:13:16.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.693+0000 7f9142479740 1 client.19161 shutdown 2024-09-19T09:13:16.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.693+0000 7f9142479740 1 -- 172.21.15.120:0/1281220463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d294bab70 msgr2=0x558d294ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:16.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.693+0000 7f9142479740 1 --2- 172.21.15.120:0/1281220463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d294bab70 0x558d294ce670 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f9134099bf0 tx=0x7f91340c9560 comp rx=0 tx=0).stop 2024-09-19T09:13:16.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.693+0000 7f9142479740 1 -- 172.21.15.120:0/1281220463 shutdown_connections 2024-09-19T09:13:16.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.693+0000 7f9142479740 1 --2- 172.21.15.120:0/1281220463 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558d29540d30 0x558d29543120 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.693+0000 7f9142479740 1 --2- 172.21.15.120:0/1281220463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d294c5d50 0x558d294ca130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.693+0000 7f9142479740 1 --2- 172.21.15.120:0/1281220463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d294bb490 0x558d294c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.693+0000 7f9142479740 1 --2- 172.21.15.120:0/1281220463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d294bab70 0x558d294ce670 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.693+0000 7f91217fa640 0 client.19161 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:16.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.693+0000 7f9142479740 1 -- 172.21.15.120:0/1281220463 >> 172.21.15.120:0/1281220463 conn(0x558d294b7100 msgr2=0x558d294bc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:16.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.693+0000 7f9142479740 1 -- 172.21.15.120:0/1281220463 shutdown_connections 2024-09-19T09:13:16.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.693+0000 7f9142479740 1 -- 172.21.15.120:0/1281220463 wait complete. 2024-09-19T09:13:16.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.693+0000 7f9142479740 20 client.19161 trim_cache size 0 max 16384 2024-09-19T09:13:16.695 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:13:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 32, 2024-09-19T09:13:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:13:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 38, 2024-09-19T09:13:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:13:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "13a5f0": { 2024-09-19T09:13:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 38, 2024-09-19T09:13:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:13:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:13:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:13:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:13:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:13:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:13:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 150.0, 2024-09-19T09:13:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:13:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:13:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:13:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:13:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:13:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:13:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:13:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:13:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:13:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:13:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:13:16.697 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:13:16.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.694+0000 7f91400a0640 1 -- 172.21.15.120:0/2566192042 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f91240a32c0 msgr2=0x7f91240a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:16.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.694+0000 7f91400a0640 1 --2- 172.21.15.120:0/2566192042 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f91240a32c0 0x7f91240a5780 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9130005fe0 tx=0x7f9130005ef0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.694+0000 7f91400a0640 1 -- 172.21.15.120:0/2566192042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9138100e20 msgr2=0x7f9138199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:16.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.694+0000 7f91400a0640 1 --2- 172.21.15.120:0/2566192042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9138100e20 0x7f9138199370 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f913400d770 tx=0x7f913400dc30 comp rx=0 tx=0).stop 2024-09-19T09:13:16.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.694+0000 7f91400a0640 1 -- 172.21.15.120:0/2566192042 shutdown_connections 2024-09-19T09:13:16.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.694+0000 7f91400a0640 1 --2- 172.21.15.120:0/2566192042 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f91240a32c0 0x7f91240a5780 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.694+0000 7f91400a0640 1 --2- 172.21.15.120:0/2566192042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9138102ef0 0x7f91381a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.694+0000 7f91400a0640 1 --2- 172.21.15.120:0/2566192042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9138102570 0x7f91381998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.694+0000 7f91400a0640 1 --2- 172.21.15.120:0/2566192042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9138100e20 0x7f9138199370 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:16.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.694+0000 7f91400a0640 1 -- 172.21.15.120:0/2566192042 >> 172.21.15.120:0/2566192042 conn(0x7f91380fc7d0 msgr2=0x7f9138106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:16.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.694+0000 7f91400a0640 1 -- 172.21.15.120:0/2566192042 shutdown_connections 2024-09-19T09:13:16.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:16.694+0000 7f91400a0640 1 -- 172.21.15.120:0/2566192042 wait complete. 2024-09-19T09:13:17.069 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.63 MB/sec execute 432 sec latency 376.144 ms 2024-09-19T09:13:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:16 smithi196 ceph-mon[35150]: pgmap v655: 97 pgs: 97 active+clean; 36 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 2.3 MiB/s wr, 124 op/s 2024-09-19T09:13:17.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:16 smithi153 ceph-mon[35071]: pgmap v655: 97 pgs: 97 active+clean; 36 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 2.3 MiB/s wr, 124 op/s 2024-09-19T09:13:17.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:16 smithi120 ceph-mon[31150]: pgmap v655: 97 pgs: 97 active+clean; 36 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 2.3 MiB/s wr, 124 op/s 2024-09-19T09:13:17.372 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:13:17.543 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '13a5f0', quiesce took 3.5 seconds. Will release after: 26.5 2024-09-19T09:13:17.636 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:13:18.069 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.61 MB/sec execute 433 sec latency 1376.184 ms 2024-09-19T09:13:18.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.142+0000 7fd676d2e640 1 -- 172.21.15.120:0/2423487377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd670100a80 msgr2=0x7fd670100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:18.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.142+0000 7fd676d2e640 1 --2- 172.21.15.120:0/2423487377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd670100a80 0x7fd670100ee0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009930 tx=0x7fd65c02a420 comp rx=0 tx=0).stop 2024-09-19T09:13:18.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.142+0000 7fd676d2e640 1 -- 172.21.15.120:0/2423487377 shutdown_connections 2024-09-19T09:13:18.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.142+0000 7fd676d2e640 1 --2- 172.21.15.120:0/2423487377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd670101420 0x7fd67010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.142+0000 7fd676d2e640 1 --2- 172.21.15.120:0/2423487377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd670100a80 0x7fd670100ee0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.142+0000 7fd676d2e640 1 --2- 172.21.15.120:0/2423487377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd670106a80 0x7fd670106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.142+0000 7fd676d2e640 1 -- 172.21.15.120:0/2423487377 >> 172.21.15.120:0/2423487377 conn(0x7fd6700fc7a0 msgr2=0x7fd6700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:18.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.142+0000 7fd676d2e640 1 -- 172.21.15.120:0/2423487377 shutdown_connections 2024-09-19T09:13:18.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.142+0000 7fd676d2e640 1 -- 172.21.15.120:0/2423487377 wait complete. 2024-09-19T09:13:18.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.143+0000 7fd676d2e640 1 Processor -- start 2024-09-19T09:13:18.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.143+0000 7fd676d2e640 1 -- start start 2024-09-19T09:13:18.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.143+0000 7fd676d2e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd670100a80 0x7fd6701974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:18.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.143+0000 7fd676d2e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd670101420 0x7fd670197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:18.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.143+0000 7fd676d2e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd670106a80 0x7fd67019bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:18.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.143+0000 7fd676d2e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd670110f20 con 0x7fd670101420 2024-09-19T09:13:18.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.143+0000 7fd676d2e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd670110da0 con 0x7fd670106a80 2024-09-19T09:13:18.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.143+0000 7fd676d2e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd6701110a0 con 0x7fd670100a80 2024-09-19T09:13:18.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.143+0000 7fd66ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd670101420 0x7fd670197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:18.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.143+0000 7fd66ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd670101420 0x7fd670197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39944/0 (socket says 172.21.15.120:39944) 2024-09-19T09:13:18.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.143+0000 7fd66ffff640 1 -- 172.21.15.120:0/3769277151 learned_addr learned my addr 172.21.15.120:0/3769277151 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:18.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.143+0000 7fd674aca640 1 --2- 172.21.15.120:0/3769277151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd670100a80 0x7fd6701974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:18.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.143+0000 7fd6752cb640 1 --2- 172.21.15.120:0/3769277151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd670106a80 0x7fd67019bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:18.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.143+0000 7fd66ffff640 1 -- 172.21.15.120:0/3769277151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd670100a80 msgr2=0x7fd6701974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:18.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.143+0000 7fd66ffff640 1 --2- 172.21.15.120:0/3769277151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd670100a80 0x7fd6701974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.143+0000 7fd66ffff640 1 -- 172.21.15.120:0/3769277151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd670106a80 msgr2=0x7fd67019bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:18.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.143+0000 7fd66ffff640 1 --2- 172.21.15.120:0/3769277151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd670106a80 0x7fd67019bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.143+0000 7fd66ffff640 1 -- 172.21.15.120:0/3769277151 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67019c470 con 0x7fd670101420 2024-09-19T09:13:18.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.144+0000 7fd66ffff640 1 --2- 172.21.15.120:0/3769277151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd670101420 0x7fd670197a00 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009a50 tx=0x7fd65c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:18.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.144+0000 7fd66dffb640 1 -- 172.21.15.120:0/3769277151 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c03f030 con 0x7fd670101420 2024-09-19T09:13:18.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.144+0000 7fd676d2e640 1 -- 172.21.15.120:0/3769277151 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd67019c700 con 0x7fd670101420 2024-09-19T09:13:18.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.144+0000 7fd66dffb640 1 -- 172.21.15.120:0/3769277151 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd65c003ad0 con 0x7fd670101420 2024-09-19T09:13:18.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.144+0000 7fd66dffb640 1 -- 172.21.15.120:0/3769277151 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c035070 con 0x7fd670101420 2024-09-19T09:13:18.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.144+0000 7fd676d2e640 1 -- 172.21.15.120:0/3769277151 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6701a4040 con 0x7fd670101420 2024-09-19T09:13:18.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.146+0000 7fd66dffb640 1 -- 172.21.15.120:0/3769277151 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd65c042030 con 0x7fd670101420 2024-09-19T09:13:18.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.147+0000 7fd679107740 1 Processor -- start 2024-09-19T09:13:18.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.147+0000 7fd679107740 1 -- start start 2024-09-19T09:13:18.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.147+0000 7fd679107740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfccac24a0 0x55cfccac2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:18.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.147+0000 7fd679107740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfccac2dc0 0x55cfccacd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:18.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.147+0000 7fd679107740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfccacd680 0x55cfccacfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:18.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.147+0000 7fd679107740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cfcca282e0 con 0x55cfccac2dc0 2024-09-19T09:13:18.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.147+0000 7fd679107740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cfcc9cfce0 con 0x55cfccac24a0 2024-09-19T09:13:18.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.147+0000 7fd679107740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cfcca242a0 con 0x55cfccacd680 2024-09-19T09:13:18.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.147+0000 7fd66dffb640 1 --2- 172.21.15.120:0/3769277151 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd6600a3240 0x7fd6600a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:18.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.147+0000 7fd674aca640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfccac2dc0 0x55cfccacd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:18.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.147+0000 7fd674aca640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfccac2dc0 0x55cfccacd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39958/0 (socket says 172.21.15.120:39958) 2024-09-19T09:13:18.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.147+0000 7fd674aca640 1 -- 172.21.15.120:0/3479745399 learned_addr learned my addr 172.21.15.120:0/3479745399 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:18.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.147+0000 7fd6752cb640 1 --2- 172.21.15.120:0/3769277151 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd6600a3240 0x7fd6600a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:18.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.147+0000 7fd66dffb640 1 -- 172.21.15.120:0/3769277151 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fd65c0f00f0 con 0x7fd670101420 2024-09-19T09:13:18.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.147+0000 7fd6752cb640 1 --2- 172.21.15.120:0/3479745399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfccac24a0 0x55cfccac2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:18.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.147+0000 7fd66ffff640 1 --2- 172.21.15.120:0/3479745399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfccacd680 0x55cfccacfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:18.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.148+0000 7fd674aca640 1 -- 172.21.15.120:0/3479745399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfccacd680 msgr2=0x55cfccacfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:18.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.148+0000 7fd674aca640 1 --2- 172.21.15.120:0/3479745399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfccacd680 0x55cfccacfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.148+0000 7fd674aca640 1 -- 172.21.15.120:0/3479745399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfccac24a0 msgr2=0x55cfccac2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:18.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.148+0000 7fd674aca640 1 --2- 172.21.15.120:0/3479745399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfccac24a0 0x55cfccac2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.148+0000 7fd674aca640 1 -- 172.21.15.120:0/3479745399 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfcc9b4440 con 0x55cfccac2dc0 2024-09-19T09:13:18.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.148+0000 7fd6752cb640 1 --2- 172.21.15.120:0/3769277151 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd6600a3240 0x7fd6600a5700 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fd668009fd0 tx=0x7fd668009d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:18.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.148+0000 7fd674aca640 1 --2- 172.21.15.120:0/3479745399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfccac2dc0 0x55cfccacd140 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fd664009a70 tx=0x7fd664009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec16548aabbb13d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:18.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.148+0000 7fd65affd640 1 -- 172.21.15.120:0/3479745399 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664017070 con 0x55cfccac2dc0 2024-09-19T09:13:18.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.148+0000 7fd65affd640 1 -- 172.21.15.120:0/3479745399 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd664013470 con 0x55cfccac2dc0 2024-09-19T09:13:18.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.148+0000 7fd65affd640 1 -- 172.21.15.120:0/3479745399 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664012610 con 0x55cfccac2dc0 2024-09-19T09:13:18.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.149+0000 7fd679107740 1 -- 172.21.15.120:0/3479745399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfccac2dc0 msgr2=0x55cfccacd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:18.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.149+0000 7fd679107740 1 --2- 172.21.15.120:0/3479745399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfccac2dc0 0x55cfccacd140 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fd664009a70 tx=0x7fd664009f30 comp rx=0 tx=0).stop 2024-09-19T09:13:18.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.149+0000 7fd679107740 1 -- 172.21.15.120:0/3479745399 shutdown_connections 2024-09-19T09:13:18.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.149+0000 7fd679107740 1 --2- 172.21.15.120:0/3479745399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfccacd680 0x55cfccacfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.149+0000 7fd679107740 1 --2- 172.21.15.120:0/3479745399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfccac2dc0 0x55cfccacd140 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.149+0000 7fd679107740 1 --2- 172.21.15.120:0/3479745399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfccac24a0 0x55cfccac2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.149+0000 7fd679107740 1 -- 172.21.15.120:0/3479745399 >> 172.21.15.120:0/3479745399 conn(0x55cfccabea30 msgr2=0x55cfccabee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:18.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.149+0000 7fd679107740 1 -- 172.21.15.120:0/3479745399 shutdown_connections 2024-09-19T09:13:18.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.149+0000 7fd679107740 1 -- 172.21.15.120:0/3479745399 wait complete. 2024-09-19T09:13:18.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.149+0000 7fd679107740 1 -- start start 2024-09-19T09:13:18.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.149+0000 7fd679107740 1 Processor -- start 2024-09-19T09:13:18.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.149+0000 7fd679107740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfccac24a0 0x55cfccacc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:18.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.149+0000 7fd679107740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfccac2dc0 0x55cfccaccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:18.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.149+0000 7fd679107740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfccacd680 0x55cfccad3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:18.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.149+0000 7fd679107740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cfccad0250 con 0x55cfccacd680 2024-09-19T09:13:18.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.149+0000 7fd679107740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cfcc9e3050 con 0x55cfccac24a0 2024-09-19T09:13:18.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.149+0000 7fd679107740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cfccad03d0 con 0x55cfccac2dc0 2024-09-19T09:13:18.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.150+0000 7fd679107740 10 client.0 _ll_register_callbacks cb 0x55cfcca23160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:18.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.150+0000 7fd6752cb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfccacd680 0x55cfccad3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:18.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.150+0000 7fd6752cb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfccacd680 0x55cfccad3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39966/0 (socket says 172.21.15.120:39966) 2024-09-19T09:13:18.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.150+0000 7fd6752cb640 1 -- 172.21.15.120:0/597847211 learned_addr learned my addr 172.21.15.120:0/597847211 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:18.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.150+0000 7fd66ffff640 1 --2- 172.21.15.120:0/597847211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfccac2dc0 0x55cfccaccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:18.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.150+0000 7fd674aca640 1 --2- 172.21.15.120:0/597847211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfccac24a0 0x55cfccacc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:18.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.150+0000 7fd6752cb640 1 -- 172.21.15.120:0/597847211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfccac2dc0 msgr2=0x55cfccaccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:18.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.150+0000 7fd6752cb640 1 --2- 172.21.15.120:0/597847211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfccac2dc0 0x55cfccaccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.150+0000 7fd6752cb640 1 -- 172.21.15.120:0/597847211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfccac24a0 msgr2=0x55cfccacc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:18.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.150+0000 7fd6752cb640 1 --2- 172.21.15.120:0/597847211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfccac24a0 0x55cfccacc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.150+0000 7fd6752cb640 1 -- 172.21.15.120:0/597847211 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfcc7993c0 con 0x55cfccacd680 2024-09-19T09:13:18.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.150+0000 7fd674aca640 1 --2- 172.21.15.120:0/597847211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfccac24a0 0x55cfccacc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:13:18.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.150+0000 7fd6752cb640 1 --2- 172.21.15.120:0/597847211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfccacd680 0x55cfccad3ce0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fd668009c40 tx=0x7fd6680093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=855f7ef2eb935f98 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:18.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.150+0000 7fd64ffff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:18.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.150+0000 7fd64ffff640 1 -- 172.21.15.120:0/597847211 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668028060 con 0x55cfccacd680 2024-09-19T09:13:18.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.150+0000 7fd64ffff640 1 -- 172.21.15.120:0/597847211 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fd6680235f0 con 0x55cfccacd680 2024-09-19T09:13:18.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.150+0000 7fd679107740 1 -- 172.21.15.120:0/597847211 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd664013470 con 0x55cfccacd680 2024-09-19T09:13:18.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.150+0000 7fd64ffff640 1 -- 172.21.15.120:0/597847211 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd668013040 con 0x55cfccacd680 2024-09-19T09:13:18.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.150+0000 7fd679107740 10 client.19173 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:18.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.150+0000 7fd679107740 1 -- 172.21.15.120:0/597847211 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cfccaddd60 con 0x55cfccacd680 2024-09-19T09:13:18.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.151+0000 7fd64ffff640 1 -- 172.21.15.120:0/597847211 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd664013470 con 0x55cfccacd680 2024-09-19T09:13:18.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.151+0000 7fd64ffff640 1 -- 172.21.15.120:0/597847211 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd66802d070 con 0x55cfccacd680 2024-09-19T09:13:18.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.151+0000 7fd679107740 10 client.19173 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:18.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.151+0000 7fd679107740 10 client.19173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:13:18.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.151+0000 7fd679107740 20 client.19173 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:18.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.151+0000 7fd679107740 1 --2- 172.21.15.120:0/597847211 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cfccadf220 0x55cfccae16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:18.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.151+0000 7fd679107740 4 client.19173 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:13:18.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.151+0000 7fd679107740 1 -- 172.21.15.120:0/597847211 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfcca35850 con 0x55cfccadf220 2024-09-19T09:13:18.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.151+0000 7fd674aca640 1 --2- 172.21.15.120:0/597847211 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cfccadf220 0x55cfccae16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:18.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.152+0000 7fd674aca640 1 --2- 172.21.15.120:0/597847211 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cfccadf220 0x55cfccae16c0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8511f43427298712 server_cookie=46a92dd4ff5d110 in_seq=0 out_seq=0 2024-09-19T09:13:18.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.152+0000 7fd64ffff640 10 client.19173 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:18.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.152+0000 7fd64ffff640 1 -- 172.21.15.120:0/597847211 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cfcca35850 con 0x55cfccadf220 2024-09-19T09:13:18.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.152+0000 7fd64ffff640 10 client.19173 handle_command_reply: tid=0 2024-09-19T09:13:18.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.152+0000 7fd679107740 1 client.19173 shutdown 2024-09-19T09:13:18.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.152+0000 7fd679107740 1 -- 172.21.15.120:0/597847211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfccacd680 msgr2=0x55cfccad3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:18.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.152+0000 7fd679107740 1 --2- 172.21.15.120:0/597847211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfccacd680 0x55cfccad3ce0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fd668009c40 tx=0x7fd6680093d0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.152+0000 7fd679107740 1 -- 172.21.15.120:0/597847211 shutdown_connections 2024-09-19T09:13:18.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.152+0000 7fd679107740 1 --2- 172.21.15.120:0/597847211 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cfccadf220 0x55cfccae16c0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.153+0000 7fd679107740 1 --2- 172.21.15.120:0/597847211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfccacd680 0x55cfccad3ce0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.153+0000 7fd679107740 1 --2- 172.21.15.120:0/597847211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfccac2dc0 0x55cfccaccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.153+0000 7fd679107740 1 --2- 172.21.15.120:0/597847211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfccac24a0 0x55cfccacc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.153+0000 7fd679107740 1 -- 172.21.15.120:0/597847211 >> 172.21.15.120:0/597847211 conn(0x55cfccabea30 msgr2=0x55cfccac4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:18.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.153+0000 7fd64ffff640 0 client.19173 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:18.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.153+0000 7fd679107740 1 -- 172.21.15.120:0/597847211 shutdown_connections 2024-09-19T09:13:18.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.153+0000 7fd679107740 1 -- 172.21.15.120:0/597847211 wait complete. 2024-09-19T09:13:18.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.153+0000 7fd679107740 20 client.19173 trim_cache size 0 max 16384 2024-09-19T09:13:18.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.170+0000 7fd679107740 1 Processor -- start 2024-09-19T09:13:18.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.170+0000 7fd679107740 1 -- start start 2024-09-19T09:13:18.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.170+0000 7fd679107740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfccac24a0 0x55cfccac8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:18.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.170+0000 7fd679107740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfccac2dc0 0x55cfccae6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:18.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.170+0000 7fd679107740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfccacd680 0x55cfccae74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:18.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.170+0000 7fd679107740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd6701ab440 con 0x55cfccacd680 2024-09-19T09:13:18.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.170+0000 7fd679107740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd6701acf80 con 0x55cfccac24a0 2024-09-19T09:13:18.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.170+0000 7fd679107740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd6701abb10 con 0x55cfccac2dc0 2024-09-19T09:13:18.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.170+0000 7fd6752cb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfccacd680 0x55cfccae74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:18.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.170+0000 7fd6752cb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfccacd680 0x55cfccae74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39970/0 (socket says 172.21.15.120:39970) 2024-09-19T09:13:18.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.170+0000 7fd6752cb640 1 -- 172.21.15.120:0/2585860428 learned_addr learned my addr 172.21.15.120:0/2585860428 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:18.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.170+0000 7fd66ffff640 1 --2- 172.21.15.120:0/2585860428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfccac2dc0 0x55cfccae6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:18.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.170+0000 7fd674aca640 1 --2- 172.21.15.120:0/2585860428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfccac24a0 0x55cfccac8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:18.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.170+0000 7fd6752cb640 1 -- 172.21.15.120:0/2585860428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfccac2dc0 msgr2=0x55cfccae6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:18.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.170+0000 7fd6752cb640 1 --2- 172.21.15.120:0/2585860428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfccac2dc0 0x55cfccae6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.170+0000 7fd6752cb640 1 -- 172.21.15.120:0/2585860428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfccac24a0 msgr2=0x55cfccac8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:18.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.170+0000 7fd6752cb640 1 --2- 172.21.15.120:0/2585860428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfccac24a0 0x55cfccac8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.170+0000 7fd6752cb640 1 -- 172.21.15.120:0/2585860428 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfccabe860 con 0x55cfccacd680 2024-09-19T09:13:18.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.171+0000 7fd6752cb640 1 --2- 172.21.15.120:0/2585860428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfccacd680 0x55cfccae74d0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fd668009440 tx=0x7fd66800a790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b421ca4b5cb7d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:18.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.171+0000 7fd65a7fc640 1 -- 172.21.15.120:0/2585860428 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668033030 con 0x55cfccacd680 2024-09-19T09:13:18.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.171+0000 7fd65a7fc640 1 -- 172.21.15.120:0/2585860428 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd668009a90 con 0x55cfccacd680 2024-09-19T09:13:18.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.171+0000 7fd65a7fc640 1 -- 172.21.15.120:0/2585860428 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668028050 con 0x55cfccacd680 2024-09-19T09:13:18.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.171+0000 7fd679107740 1 -- 172.21.15.120:0/2585860428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfccacd680 msgr2=0x55cfccae74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:18.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.171+0000 7fd679107740 1 --2- 172.21.15.120:0/2585860428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfccacd680 0x55cfccae74d0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fd668009440 tx=0x7fd66800a790 comp rx=0 tx=0).stop 2024-09-19T09:13:18.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.171+0000 7fd679107740 1 -- 172.21.15.120:0/2585860428 shutdown_connections 2024-09-19T09:13:18.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.171+0000 7fd679107740 1 --2- 172.21.15.120:0/2585860428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfccacd680 0x55cfccae74d0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.171+0000 7fd679107740 1 --2- 172.21.15.120:0/2585860428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfccac2dc0 0x55cfccae6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.171+0000 7fd679107740 1 --2- 172.21.15.120:0/2585860428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfccac24a0 0x55cfccac8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.171+0000 7fd679107740 1 -- 172.21.15.120:0/2585860428 >> 172.21.15.120:0/2585860428 conn(0x55cfccabea30 msgr2=0x55cfccad5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:18.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.171+0000 7fd679107740 1 -- 172.21.15.120:0/2585860428 shutdown_connections 2024-09-19T09:13:18.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.171+0000 7fd679107740 1 -- 172.21.15.120:0/2585860428 wait complete. 2024-09-19T09:13:18.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.172+0000 7fd679107740 1 -- start start 2024-09-19T09:13:18.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.172+0000 7fd679107740 1 Processor -- start 2024-09-19T09:13:18.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.172+0000 7fd679107740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfccac24a0 0x55cfccadb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:18.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.172+0000 7fd679107740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfccac2dc0 0x55cfccad8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:18.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.172+0000 7fd679107740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfccacd680 0x55cfccad8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:18.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.172+0000 7fd679107740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd6701ac8b0 con 0x55cfccac24a0 2024-09-19T09:13:18.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.172+0000 7fd679107740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd6701ac1e0 con 0x55cfccac2dc0 2024-09-19T09:13:18.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.172+0000 7fd679107740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd6701a9400 con 0x55cfccacd680 2024-09-19T09:13:18.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.172+0000 7fd679107740 10 client.0 _ll_register_callbacks cb 0x7fd66800a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:18.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.172+0000 7fd674aca640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfccac24a0 0x55cfccadb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:18.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.172+0000 7fd674aca640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfccac24a0 0x55cfccadb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39984/0 (socket says 172.21.15.120:39984) 2024-09-19T09:13:18.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.172+0000 7fd674aca640 1 -- 172.21.15.120:0/1218699020 learned_addr learned my addr 172.21.15.120:0/1218699020 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:18.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.172+0000 7fd66ffff640 1 --2- 172.21.15.120:0/1218699020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfccac2dc0 0x55cfccad8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:18.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.173+0000 7fd6752cb640 1 --2- 172.21.15.120:0/1218699020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfccacd680 0x55cfccad8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:18.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.173+0000 7fd674aca640 1 -- 172.21.15.120:0/1218699020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfccacd680 msgr2=0x55cfccad8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:18.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.173+0000 7fd674aca640 1 --2- 172.21.15.120:0/1218699020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfccacd680 0x55cfccad8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.173+0000 7fd674aca640 1 -- 172.21.15.120:0/1218699020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfccac2dc0 msgr2=0x55cfccad8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:18.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.173+0000 7fd674aca640 1 --2- 172.21.15.120:0/1218699020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfccac2dc0 0x55cfccad8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.173+0000 7fd674aca640 1 -- 172.21.15.120:0/1218699020 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfccad9810 con 0x55cfccac24a0 2024-09-19T09:13:18.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.173+0000 7fd6752cb640 1 --2- 172.21.15.120:0/1218699020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfccacd680 0x55cfccad8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:13:18.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.173+0000 7fd674aca640 1 --2- 172.21.15.120:0/1218699020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfccac24a0 0x55cfccadb710 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fd664009830 tx=0x7fd664009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af9c7c4ed6db1d29 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:18.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.173+0000 7fd64ffff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:18.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.173+0000 7fd64ffff640 1 -- 172.21.15.120:0/1218699020 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664017040 con 0x55cfccac24a0 2024-09-19T09:13:18.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.173+0000 7fd64ffff640 1 -- 172.21.15.120:0/1218699020 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fd664025810 con 0x55cfccac24a0 2024-09-19T09:13:18.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.173+0000 7fd679107740 1 -- 172.21.15.120:0/1218699020 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd668009a90 con 0x55cfccac24a0 2024-09-19T09:13:18.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.173+0000 7fd64ffff640 1 -- 172.21.15.120:0/1218699020 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd664025a10 con 0x55cfccac24a0 2024-09-19T09:13:18.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.173+0000 7fd679107740 10 client.19179 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:18.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.173+0000 7fd679107740 1 -- 172.21.15.120:0/1218699020 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cfccb4a020 con 0x55cfccac24a0 2024-09-19T09:13:18.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.173+0000 7fd64ffff640 1 -- 172.21.15.120:0/1218699020 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd668009a90 con 0x55cfccac24a0 2024-09-19T09:13:18.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.173+0000 7fd64ffff640 1 -- 172.21.15.120:0/1218699020 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd66401c450 con 0x55cfccac24a0 2024-09-19T09:13:18.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.173+0000 7fd679107740 10 client.19179 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:18.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.173+0000 7fd679107740 10 client.19179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:13:18.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.173+0000 7fd679107740 20 client.19179 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:18.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.173+0000 7fd679107740 1 --2- 172.21.15.120:0/1218699020 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cfccb4a470 0x55cfccb4c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:18.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.173+0000 7fd679107740 4 client.19179 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:13:18.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.173+0000 7fd679107740 1 -- 172.21.15.120:0/1218699020 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cfcca2fd60 con 0x55cfccb4a470 2024-09-19T09:13:18.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.174+0000 7fd66ffff640 1 --2- 172.21.15.120:0/1218699020 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cfccb4a470 0x55cfccb4c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:18.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.174+0000 7fd66ffff640 1 --2- 172.21.15.120:0/1218699020 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cfccb4a470 0x55cfccb4c860 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fedd2efedb4049 server_cookie=e8b01434dd40b723 in_seq=0 out_seq=0 2024-09-19T09:13:18.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.174+0000 7fd64ffff640 10 client.19179 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:18.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.174+0000 7fd64ffff640 1 -- 172.21.15.120:0/1218699020 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cfcca2fd60 con 0x55cfccb4a470 2024-09-19T09:13:18.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.174+0000 7fd64ffff640 10 client.19179 handle_command_reply: tid=0 2024-09-19T09:13:18.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.174+0000 7fd679107740 1 client.19179 shutdown 2024-09-19T09:13:18.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.174+0000 7fd679107740 1 -- 172.21.15.120:0/1218699020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfccac24a0 msgr2=0x55cfccadb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:18.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.174+0000 7fd679107740 1 --2- 172.21.15.120:0/1218699020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfccac24a0 0x55cfccadb710 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fd664009830 tx=0x7fd664009280 comp rx=0 tx=0).stop 2024-09-19T09:13:18.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.174+0000 7fd679107740 1 -- 172.21.15.120:0/1218699020 shutdown_connections 2024-09-19T09:13:18.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.174+0000 7fd679107740 1 --2- 172.21.15.120:0/1218699020 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cfccb4a470 0x55cfccb4c860 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.174+0000 7fd679107740 1 --2- 172.21.15.120:0/1218699020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfccacd680 0x55cfccad8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.174+0000 7fd679107740 1 --2- 172.21.15.120:0/1218699020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfccac2dc0 0x55cfccad8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.174+0000 7fd64ffff640 0 client.19179 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:18.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.174+0000 7fd679107740 1 --2- 172.21.15.120:0/1218699020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfccac24a0 0x55cfccadb710 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.174+0000 7fd679107740 1 -- 172.21.15.120:0/1218699020 >> 172.21.15.120:0/1218699020 conn(0x55cfccabea30 msgr2=0x55cfccad4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:18.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.174+0000 7fd679107740 1 -- 172.21.15.120:0/1218699020 shutdown_connections 2024-09-19T09:13:18.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.174+0000 7fd679107740 1 -- 172.21.15.120:0/1218699020 wait complete. 2024-09-19T09:13:18.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.175+0000 7fd679107740 20 client.19179 trim_cache size 0 max 16384 2024-09-19T09:13:18.205 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:13:18.205 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:13:18.205 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:13:18.205 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:13:18.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.175+0000 7fd676d2e640 1 -- 172.21.15.120:0/3769277151 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd6600a3240 msgr2=0x7fd6600a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:18.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.175+0000 7fd676d2e640 1 --2- 172.21.15.120:0/3769277151 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd6600a3240 0x7fd6600a5700 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fd668009fd0 tx=0x7fd668009d80 comp rx=0 tx=0).stop 2024-09-19T09:13:18.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.175+0000 7fd676d2e640 1 -- 172.21.15.120:0/3769277151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd670101420 msgr2=0x7fd670197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:18.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.175+0000 7fd676d2e640 1 --2- 172.21.15.120:0/3769277151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd670101420 0x7fd670197a00 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009a50 tx=0x7fd65c004270 comp rx=0 tx=0).stop 2024-09-19T09:13:18.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.175+0000 7fd676d2e640 1 -- 172.21.15.120:0/3769277151 shutdown_connections 2024-09-19T09:13:18.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.175+0000 7fd676d2e640 1 --2- 172.21.15.120:0/3769277151 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd6600a3240 0x7fd6600a5700 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.175+0000 7fd676d2e640 1 --2- 172.21.15.120:0/3769277151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd670106a80 0x7fd67019bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.175+0000 7fd676d2e640 1 --2- 172.21.15.120:0/3769277151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd670101420 0x7fd670197a00 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.175+0000 7fd676d2e640 1 --2- 172.21.15.120:0/3769277151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd670100a80 0x7fd6701974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:18.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.175+0000 7fd676d2e640 1 -- 172.21.15.120:0/3769277151 >> 172.21.15.120:0/3769277151 conn(0x7fd6700fc7a0 msgr2=0x7fd6700fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:18.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.175+0000 7fd676d2e640 1 -- 172.21.15.120:0/3769277151 shutdown_connections 2024-09-19T09:13:18.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:18.175+0000 7fd676d2e640 1 -- 172.21.15.120:0/3769277151 wait complete. 2024-09-19T09:13:19.054 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:18 smithi120 ceph-mon[31150]: pgmap v656: 97 pgs: 97 active+clean; 36 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 2.8 MiB/s wr, 144 op/s 2024-09-19T09:13:19.055 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:13:19.055 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:13:19.069 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.58 MB/sec execute 434 sec latency 2376.210 ms 2024-09-19T09:13:19.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:18 smithi196 ceph-mon[35150]: pgmap v656: 97 pgs: 97 active+clean; 36 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 2.8 MiB/s wr, 144 op/s 2024-09-19T09:13:19.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:18 smithi153 ceph-mon[35071]: pgmap v656: 97 pgs: 97 active+clean; 36 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 2.8 MiB/s wr, 144 op/s 2024-09-19T09:13:19.297 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:13:19.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.759+0000 7f2a2dc1b640 1 -- 172.21.15.120:0/1119911124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a28102570 msgr2=0x7f2a281029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:19.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.759+0000 7f2a2dc1b640 1 --2- 172.21.15.120:0/1119911124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a28102570 0x7f2a281029b0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c009900 tx=0x7f2a1c02a480 comp rx=0 tx=0).stop 2024-09-19T09:13:19.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.760+0000 7f2a2dc1b640 1 -- 172.21.15.120:0/1119911124 shutdown_connections 2024-09-19T09:13:19.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.760+0000 7f2a2dc1b640 1 --2- 172.21.15.120:0/1119911124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a28102ef0 0x7f2a2810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:19.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.760+0000 7f2a2dc1b640 1 --2- 172.21.15.120:0/1119911124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a28102570 0x7f2a281029b0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:19.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.760+0000 7f2a2dc1b640 1 --2- 172.21.15.120:0/1119911124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a28100e20 0x7f2a28101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:19.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.760+0000 7f2a2dc1b640 1 -- 172.21.15.120:0/1119911124 >> 172.21.15.120:0/1119911124 conn(0x7f2a280fc7d0 msgr2=0x7f2a280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:19.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.760+0000 7f2a2dc1b640 1 -- 172.21.15.120:0/1119911124 shutdown_connections 2024-09-19T09:13:19.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.760+0000 7f2a2dc1b640 1 -- 172.21.15.120:0/1119911124 wait complete. 2024-09-19T09:13:19.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.760+0000 7f2a2dc1b640 1 Processor -- start 2024-09-19T09:13:19.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.760+0000 7f2a2dc1b640 1 -- start start 2024-09-19T09:13:19.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.761+0000 7f2a2dc1b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a28100e20 0x7f2a2819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:19.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.761+0000 7f2a2dc1b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a28102570 0x7f2a2819ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:19.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.761+0000 7f2a2dc1b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a28102ef0 0x7f2a2819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:19.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.761+0000 7f2a2dc1b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2a28110f80 con 0x7f2a28102570 2024-09-19T09:13:19.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.761+0000 7f2a2dc1b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2a28110e00 con 0x7f2a28100e20 2024-09-19T09:13:19.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.761+0000 7f2a2dc1b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a28111100 con 0x7f2a28102ef0 2024-09-19T09:13:19.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.761+0000 7f2a26ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a28102570 0x7f2a2819ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:19.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.761+0000 7f2a26ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a28102570 0x7f2a2819ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39994/0 (socket says 172.21.15.120:39994) 2024-09-19T09:13:19.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.761+0000 7f2a26ffd640 1 -- 172.21.15.120:0/1840291748 learned_addr learned my addr 172.21.15.120:0/1840291748 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:19.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.761+0000 7f2a277fe640 1 --2- 172.21.15.120:0/1840291748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a28100e20 0x7f2a2819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:19.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.761+0000 7f2a27fff640 1 --2- 172.21.15.120:0/1840291748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a28102ef0 0x7f2a2819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:19.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.761+0000 7f2a26ffd640 1 -- 172.21.15.120:0/1840291748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a28102ef0 msgr2=0x7f2a2819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:19.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.761+0000 7f2a26ffd640 1 --2- 172.21.15.120:0/1840291748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a28102ef0 0x7f2a2819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:19.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.761+0000 7f2a26ffd640 1 -- 172.21.15.120:0/1840291748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a28100e20 msgr2=0x7f2a2819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:19.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.761+0000 7f2a26ffd640 1 --2- 172.21.15.120:0/1840291748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a28100e20 0x7f2a2819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:19.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.761+0000 7f2a26ffd640 1 -- 172.21.15.120:0/1840291748 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a2819f680 con 0x7f2a28102570 2024-09-19T09:13:19.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.761+0000 7f2a277fe640 1 --2- 172.21.15.120:0/1840291748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a28100e20 0x7f2a2819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:13:19.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.761+0000 7f2a26ffd640 1 --2- 172.21.15.120:0/1840291748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a28102570 0x7f2a2819ac10 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c006010 tx=0x7f2a1c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:19.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.761+0000 7f2a24ff9640 1 -- 172.21.15.120:0/1840291748 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c03f070 con 0x7f2a28102570 2024-09-19T09:13:19.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.761+0000 7f2a2dc1b640 1 -- 172.21.15.120:0/1840291748 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a2819f910 con 0x7f2a28102570 2024-09-19T09:13:19.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.761+0000 7f2a24ff9640 1 -- 172.21.15.120:0/1840291748 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a1c031c30 con 0x7f2a28102570 2024-09-19T09:13:19.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.761+0000 7f2a24ff9640 1 -- 172.21.15.120:0/1840291748 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c035030 con 0x7f2a28102570 2024-09-19T09:13:19.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.761+0000 7f2a2dc1b640 1 -- 172.21.15.120:0/1840291748 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a281a7250 con 0x7f2a28102570 2024-09-19T09:13:19.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.762+0000 7f2a2dc1b640 1 -- 172.21.15.120:0/1840291748 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a281021b0 con 0x7f2a28102570 2024-09-19T09:13:19.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.763+0000 7f2a24ff9640 1 -- 172.21.15.120:0/1840291748 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f2a1c042070 con 0x7f2a28102570 2024-09-19T09:13:19.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.764+0000 7f2a24ff9640 1 --2- 172.21.15.120:0/1840291748 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2a100a3240 0x7f2a100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:19.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.764+0000 7f2a24ff9640 1 -- 172.21.15.120:0/1840291748 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f2a1c0f0680 con 0x7f2a28102570 2024-09-19T09:13:19.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.764+0000 7f2a277fe640 1 --2- 172.21.15.120:0/1840291748 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2a100a3240 0x7f2a100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:19.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.765+0000 7f2a277fe640 1 --2- 172.21.15.120:0/1840291748 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2a100a3240 0x7f2a100a5700 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2a18005fd0 tx=0x7f2a18009520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:19.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.767+0000 7f2a24ff9640 1 -- 172.21.15.120:0/1840291748 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2a1c031de0 con 0x7f2a28102570 2024-09-19T09:13:19.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.969+0000 7f2a2dc1b640 1 -- 172.21.15.120:0/1840291748 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a281a7420 con 0x7f2a28102570 2024-09-19T09:13:19.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.969+0000 7f2a24ff9640 1 -- 172.21.15.120:0/1840291748 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11988 (secure 0 0 0) 0x7f2a1c0bd460 con 0x7f2a28102570 2024-09-19T09:13:19.970 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:13:19.970 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":32,"btime":"2024-09-19T09:11:12:904684+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:11:12.904670+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27518":{"gid":27518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3214776406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3214776406},{"type":"v1","addr":"172.21.15.153:6835","nonce":3214776406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:13:19.971 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 32 2024-09-19T09:13:19.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.972+0000 7f2a2dc1b640 1 -- 172.21.15.120:0/1840291748 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2a100a3240 msgr2=0x7f2a100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:19.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.972+0000 7f2a2dc1b640 1 --2- 172.21.15.120:0/1840291748 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2a100a3240 0x7f2a100a5700 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2a18005fd0 tx=0x7f2a18009520 comp rx=0 tx=0).stop 2024-09-19T09:13:19.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.972+0000 7f2a2dc1b640 1 -- 172.21.15.120:0/1840291748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a28102570 msgr2=0x7f2a2819ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:19.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.972+0000 7f2a2dc1b640 1 --2- 172.21.15.120:0/1840291748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a28102570 0x7f2a2819ac10 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c006010 tx=0x7f2a1c003c10 comp rx=0 tx=0).stop 2024-09-19T09:13:19.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.972+0000 7f2a2dc1b640 1 -- 172.21.15.120:0/1840291748 shutdown_connections 2024-09-19T09:13:19.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.972+0000 7f2a2dc1b640 1 --2- 172.21.15.120:0/1840291748 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2a100a3240 0x7f2a100a5700 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:19.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.972+0000 7f2a2dc1b640 1 --2- 172.21.15.120:0/1840291748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a28102ef0 0x7f2a2819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:19.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.972+0000 7f2a2dc1b640 1 --2- 172.21.15.120:0/1840291748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a28102570 0x7f2a2819ac10 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:19.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.972+0000 7f2a2dc1b640 1 --2- 172.21.15.120:0/1840291748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a28100e20 0x7f2a2819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:19.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.972+0000 7f2a2dc1b640 1 -- 172.21.15.120:0/1840291748 >> 172.21.15.120:0/1840291748 conn(0x7f2a280fc7d0 msgr2=0x7f2a2806a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:19.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.972+0000 7f2a2dc1b640 1 -- 172.21.15.120:0/1840291748 shutdown_connections 2024-09-19T09:13:19.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:19.972+0000 7f2a2dc1b640 1 -- 172.21.15.120:0/1840291748 wait complete. 2024-09-19T09:13:20.069 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.56 MB/sec execute 435 sec latency 3376.282 ms 2024-09-19T09:13:20.760 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:13:21.048 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:13:21.069 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.54 MB/sec execute 436 sec latency 4376.308 ms 2024-09-19T09:13:21.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:20 smithi120 ceph-mon[31150]: pgmap v657: 97 pgs: 97 active+clean; 38 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 177 op/s 2024-09-19T09:13:21.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:20 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1840291748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:13:21.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:20 smithi196 ceph-mon[35150]: pgmap v657: 97 pgs: 97 active+clean; 38 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 177 op/s 2024-09-19T09:13:21.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:20 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1840291748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:13:21.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:20 smithi153 ceph-mon[35071]: pgmap v657: 97 pgs: 97 active+clean; 38 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 177 op/s 2024-09-19T09:13:21.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:20 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1840291748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:13:21.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.667+0000 7f19cdefd640 1 -- 172.21.15.120:0/2137826507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19c810c170 msgr2=0x7f19c810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:21.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.667+0000 7f19cdefd640 1 --2- 172.21.15.120:0/2137826507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19c810c170 0x7f19c810e610 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f19c0009ca0 tx=0x7f19c002a950 comp rx=0 tx=0).stop 2024-09-19T09:13:21.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.667+0000 7f19cdefd640 1 -- 172.21.15.120:0/2137826507 shutdown_connections 2024-09-19T09:13:21.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.667+0000 7f19cdefd640 1 --2- 172.21.15.120:0/2137826507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19c810c170 0x7f19c810e610 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.667+0000 7f19cdefd640 1 --2- 172.21.15.120:0/2137826507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19c8103660 0x7f19c810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.667+0000 7f19cdefd640 1 --2- 172.21.15.120:0/2137826507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19c8102d40 0x7f19c8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.667+0000 7f19cdefd640 1 -- 172.21.15.120:0/2137826507 >> 172.21.15.120:0/2137826507 conn(0x7f19c80fc7a0 msgr2=0x7f19c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:21.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.667+0000 7f19cdefd640 1 -- 172.21.15.120:0/2137826507 shutdown_connections 2024-09-19T09:13:21.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.667+0000 7f19cdefd640 1 -- 172.21.15.120:0/2137826507 wait complete. 2024-09-19T09:13:21.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.668+0000 7f19cdefd640 1 Processor -- start 2024-09-19T09:13:21.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.668+0000 7f19cdefd640 1 -- start start 2024-09-19T09:13:21.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.668+0000 7f19cdefd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19c8102d40 0x7f19c8198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:21.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.668+0000 7f19cdefd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19c8103660 0x7f19c81988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:21.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.668+0000 7f19cdefd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19c810c170 0x7f19c819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:21.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.668+0000 7f19cdefd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f19c8111f70 con 0x7f19c8103660 2024-09-19T09:13:21.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.668+0000 7f19cdefd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f19c8111df0 con 0x7f19c810c170 2024-09-19T09:13:21.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.668+0000 7f19cdefd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f19c81120f0 con 0x7f19c8102d40 2024-09-19T09:13:21.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.668+0000 7f19bffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19c8103660 0x7f19c81988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:21.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.668+0000 7f19bffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19c8103660 0x7f19c81988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56230/0 (socket says 172.21.15.120:56230) 2024-09-19T09:13:21.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.668+0000 7f19bffff640 1 -- 172.21.15.120:0/659618747 learned_addr learned my addr 172.21.15.120:0/659618747 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:21.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.668+0000 7f19c77fe640 1 --2- 172.21.15.120:0/659618747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19c8102d40 0x7f19c8198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:21.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.668+0000 7f19bffff640 1 -- 172.21.15.120:0/659618747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19c8102d40 msgr2=0x7f19c8198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:21.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.668+0000 7f19bffff640 1 --2- 172.21.15.120:0/659618747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19c8102d40 0x7f19c8198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.668+0000 7f19c7fff640 1 --2- 172.21.15.120:0/659618747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19c810c170 0x7f19c819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:21.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.668+0000 7f19bffff640 1 -- 172.21.15.120:0/659618747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19c810c170 msgr2=0x7f19c819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:21.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.668+0000 7f19bffff640 1 --2- 172.21.15.120:0/659618747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19c810c170 0x7f19c819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.668+0000 7f19bffff640 1 -- 172.21.15.120:0/659618747 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c819d320 con 0x7f19c8103660 2024-09-19T09:13:21.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.668+0000 7f19c7fff640 1 --2- 172.21.15.120:0/659618747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19c810c170 0x7f19c819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:13:21.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.669+0000 7f19bffff640 1 --2- 172.21.15.120:0/659618747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19c8103660 0x7f19c81988b0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f19b800c970 tx=0x7f19b800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:21.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.669+0000 7f19c57fa640 1 -- 172.21.15.120:0/659618747 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b8007bb0 con 0x7f19c8103660 2024-09-19T09:13:21.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.669+0000 7f19cdefd640 1 -- 172.21.15.120:0/659618747 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19c819d610 con 0x7f19c8103660 2024-09-19T09:13:21.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.669+0000 7f19c57fa640 1 -- 172.21.15.120:0/659618747 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19b8007d50 con 0x7f19c8103660 2024-09-19T09:13:21.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.669+0000 7f19c57fa640 1 -- 172.21.15.120:0/659618747 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b8018820 con 0x7f19c8103660 2024-09-19T09:13:21.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.669+0000 7f19cdefd640 1 -- 172.21.15.120:0/659618747 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19c81a4ef0 con 0x7f19c8103660 2024-09-19T09:13:21.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.671+0000 7f19c57fa640 1 -- 172.21.15.120:0/659618747 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f19b80189c0 con 0x7f19c8103660 2024-09-19T09:13:21.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.671+0000 7f19d02d6740 1 Processor -- start 2024-09-19T09:13:21.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.671+0000 7f19d02d6740 1 -- start start 2024-09-19T09:13:21.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.672+0000 7f19d02d6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e1f171b40 0x560e1f170090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:21.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.672+0000 7f19d02d6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e1f1705d0 0x560e1f170a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:21.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.672+0000 7f19d02d6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e1f17f780 0x560e1f181b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:21.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.672+0000 7f19d02d6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560e1f0d62f0 con 0x560e1f17f780 2024-09-19T09:13:21.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.672+0000 7f19d02d6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560e1f07dcf0 con 0x560e1f171b40 2024-09-19T09:13:21.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.672+0000 7f19d02d6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560e1f0d22b0 con 0x560e1f1705d0 2024-09-19T09:13:21.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.672+0000 7f19bffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e1f17f780 0x560e1f181b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:21.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.672+0000 7f19bffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e1f17f780 0x560e1f181b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56242/0 (socket says 172.21.15.120:56242) 2024-09-19T09:13:21.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.672+0000 7f19bffff640 1 -- 172.21.15.120:0/323416691 learned_addr learned my addr 172.21.15.120:0/323416691 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:21.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.672+0000 7f19c57fa640 1 --2- 172.21.15.120:0/659618747 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f19b40a3240 0x7f19b40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:21.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.672+0000 7f19c77fe640 1 --2- 172.21.15.120:0/323416691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e1f1705d0 0x560e1f170a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:21.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.672+0000 7f19c7fff640 1 --2- 172.21.15.120:0/323416691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e1f171b40 0x560e1f170090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:21.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.672+0000 7f19c7fff640 1 --2- 172.21.15.120:0/659618747 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f19b40a3240 0x7f19b40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:21.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.672+0000 7f19c57fa640 1 -- 172.21.15.120:0/659618747 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f19b80cf940 con 0x7f19c8103660 2024-09-19T09:13:21.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.672+0000 7f19c77fe640 1 -- 172.21.15.120:0/323416691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e1f171b40 msgr2=0x560e1f170090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:21.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.672+0000 7f19c77fe640 1 --2- 172.21.15.120:0/323416691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e1f171b40 0x560e1f170090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.672+0000 7f19c77fe640 1 -- 172.21.15.120:0/323416691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e1f17f780 msgr2=0x560e1f181b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:21.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.672+0000 7f19c77fe640 1 --2- 172.21.15.120:0/323416691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e1f17f780 0x560e1f181b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.672+0000 7f19c77fe640 1 -- 172.21.15.120:0/323416691 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e1f0623a0 con 0x560e1f1705d0 2024-09-19T09:13:21.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.673+0000 7f19c77fe640 1 --2- 172.21.15.120:0/323416691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e1f1705d0 0x560e1f170a10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f19b0009c20 tx=0x7f19b0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=478cb5d9afbe7c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:21.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.673+0000 7f19c7fff640 1 --2- 172.21.15.120:0/659618747 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f19b40a3240 0x7f19b40a5700 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f19c0009ca0 tx=0x7f19c00095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:21.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.673+0000 7f19bdffb640 1 -- 172.21.15.120:0/323416691 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0004050 con 0x560e1f1705d0 2024-09-19T09:13:21.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.673+0000 7f19bdffb640 1 -- 172.21.15.120:0/323416691 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19b0005500 con 0x560e1f1705d0 2024-09-19T09:13:21.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.673+0000 7f19bdffb640 1 -- 172.21.15.120:0/323416691 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0004dd0 con 0x560e1f1705d0 2024-09-19T09:13:21.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.673+0000 7f19d02d6740 1 -- 172.21.15.120:0/323416691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e1f1705d0 msgr2=0x560e1f170a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:21.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.673+0000 7f19d02d6740 1 --2- 172.21.15.120:0/323416691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e1f1705d0 0x560e1f170a10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f19b0009c20 tx=0x7f19b0011810 comp rx=0 tx=0).stop 2024-09-19T09:13:21.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.673+0000 7f19d02d6740 1 -- 172.21.15.120:0/323416691 shutdown_connections 2024-09-19T09:13:21.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.673+0000 7f19d02d6740 1 --2- 172.21.15.120:0/323416691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e1f17f780 0x560e1f181b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.673+0000 7f19d02d6740 1 --2- 172.21.15.120:0/323416691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e1f1705d0 0x560e1f170a10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.673+0000 7f19d02d6740 1 --2- 172.21.15.120:0/323416691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e1f171b40 0x560e1f170090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.673+0000 7f19d02d6740 1 -- 172.21.15.120:0/323416691 >> 172.21.15.120:0/323416691 conn(0x560e1f139310 msgr2=0x560e1f16cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:21.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.673+0000 7f19d02d6740 1 -- 172.21.15.120:0/323416691 shutdown_connections 2024-09-19T09:13:21.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.674+0000 7f19d02d6740 1 -- 172.21.15.120:0/323416691 wait complete. 2024-09-19T09:13:21.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.674+0000 7f19d02d6740 1 -- start start 2024-09-19T09:13:21.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.674+0000 7f19d02d6740 1 Processor -- start 2024-09-19T09:13:21.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.674+0000 7f19d02d6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e1f1705d0 0x560e1f17f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:21.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.674+0000 7f19d02d6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e1f171b40 0x560e1f17c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:21.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.674+0000 7f19d02d6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e1f17f780 0x560e1f17c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:21.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.674+0000 7f19d02d6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560e1f182480 con 0x560e1f17f780 2024-09-19T09:13:21.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.674+0000 7f19d02d6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560e1f091060 con 0x560e1f171b40 2024-09-19T09:13:21.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.674+0000 7f19d02d6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560e1f182600 con 0x560e1f1705d0 2024-09-19T09:13:21.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.674+0000 7f19d02d6740 10 client.0 _ll_register_callbacks cb 0x560e1ede48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:21.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.674+0000 7f19c7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e1f17f780 0x560e1f17c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:21.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.674+0000 7f19c77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e1f1705d0 0x560e1f17f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:21.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.674+0000 7f19c77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e1f1705d0 0x560e1f17f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:49432/0 (socket says 172.21.15.120:49432) 2024-09-19T09:13:21.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.674+0000 7f19c7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e1f17f780 0x560e1f17c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56254/0 (socket says 172.21.15.120:56254) 2024-09-19T09:13:21.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.674+0000 7f19c77fe640 1 -- 172.21.15.120:0/1015060911 learned_addr learned my addr 172.21.15.120:0/1015060911 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:21.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.675+0000 7f19bffff640 1 --2- 172.21.15.120:0/1015060911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e1f171b40 0x560e1f17c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:21.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.675+0000 7f19c7fff640 1 -- 172.21.15.120:0/1015060911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e1f1705d0 msgr2=0x560e1f17f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:21.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.675+0000 7f19c7fff640 1 --2- 172.21.15.120:0/1015060911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e1f1705d0 0x560e1f17f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.675+0000 7f19c7fff640 1 -- 172.21.15.120:0/1015060911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e1f171b40 msgr2=0x560e1f17c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:21.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.675+0000 7f19c7fff640 1 --2- 172.21.15.120:0/1015060911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e1f171b40 0x560e1f17c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.675+0000 7f19c7fff640 1 -- 172.21.15.120:0/1015060911 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e1ee473c0 con 0x560e1f17f780 2024-09-19T09:13:21.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.675+0000 7f19bffff640 1 --2- 172.21.15.120:0/1015060911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e1f171b40 0x560e1f17c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:13:21.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.675+0000 7f19c7fff640 1 --2- 172.21.15.120:0/1015060911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e1f17f780 0x560e1f17c890 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f19c0003bf0 tx=0x7f19c002a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb7f2f5c280e0be server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:21.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.675+0000 7f19a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:21.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.675+0000 7f19a6ffd640 1 -- 172.21.15.120:0/1015060911 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c0050070 con 0x560e1f17f780 2024-09-19T09:13:21.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.675+0000 7f19d02d6740 1 -- 172.21.15.120:0/1015060911 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19b0005500 con 0x560e1f17f780 2024-09-19T09:13:21.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.675+0000 7f19a6ffd640 1 -- 172.21.15.120:0/1015060911 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f19c0004440 con 0x560e1f17f780 2024-09-19T09:13:21.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.675+0000 7f19a6ffd640 1 -- 172.21.15.120:0/1015060911 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f19c0034040 con 0x560e1f17f780 2024-09-19T09:13:21.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.675+0000 7f19d02d6740 10 client.19197 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:21.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.675+0000 7f19d02d6740 1 -- 172.21.15.120:0/1015060911 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560e1f18de90 con 0x560e1f17f780 2024-09-19T09:13:21.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.675+0000 7f19a6ffd640 1 -- 172.21.15.120:0/1015060911 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19b0005500 con 0x560e1f17f780 2024-09-19T09:13:21.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.676+0000 7f19a6ffd640 1 -- 172.21.15.120:0/1015060911 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f19c0005500 con 0x560e1f17f780 2024-09-19T09:13:21.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.676+0000 7f19d02d6740 10 client.19197 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:21.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.676+0000 7f19d02d6740 10 client.19197 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:13:21.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.676+0000 7f19d02d6740 20 client.19197 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:21.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.676+0000 7f19d02d6740 1 --2- 172.21.15.120:0/1015060911 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560e1f18f2a0 0x560e1f1916f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:21.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.676+0000 7f19d02d6740 4 client.19197 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:13:21.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.676+0000 7f19d02d6740 1 -- 172.21.15.120:0/1015060911 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e1f0e3860 con 0x560e1f18f2a0 2024-09-19T09:13:21.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.676+0000 7f19c77fe640 1 --2- 172.21.15.120:0/1015060911 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560e1f18f2a0 0x560e1f1916f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:21.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.676+0000 7f19c77fe640 1 --2- 172.21.15.120:0/1015060911 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560e1f18f2a0 0x560e1f1916f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8ba02e2e53f17fb5 server_cookie=f666bcc64c4ff3c3 in_seq=0 out_seq=0 2024-09-19T09:13:21.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.676+0000 7f19a6ffd640 10 client.19197 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:13:21.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.677+0000 7f19a6ffd640 1 -- 172.21.15.120:0/1015060911 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e1f0e3860 con 0x560e1f18f2a0 2024-09-19T09:13:21.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.677+0000 7f19a6ffd640 10 client.19197 handle_command_reply: tid=0 2024-09-19T09:13:21.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.677+0000 7f19d02d6740 1 client.19197 shutdown 2024-09-19T09:13:21.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.677+0000 7f19d02d6740 1 -- 172.21.15.120:0/1015060911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e1f17f780 msgr2=0x560e1f17c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:21.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.677+0000 7f19d02d6740 1 --2- 172.21.15.120:0/1015060911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e1f17f780 0x560e1f17c890 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f19c0003bf0 tx=0x7f19c002a950 comp rx=0 tx=0).stop 2024-09-19T09:13:21.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.677+0000 7f19d02d6740 1 -- 172.21.15.120:0/1015060911 shutdown_connections 2024-09-19T09:13:21.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.677+0000 7f19d02d6740 1 --2- 172.21.15.120:0/1015060911 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560e1f18f2a0 0x560e1f1916f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.677+0000 7f19a6ffd640 0 client.19197 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:13:21.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.677+0000 7f19d02d6740 1 --2- 172.21.15.120:0/1015060911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e1f17f780 0x560e1f17c890 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.677+0000 7f19d02d6740 1 --2- 172.21.15.120:0/1015060911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e1f171b40 0x560e1f17c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.677+0000 7f19d02d6740 1 --2- 172.21.15.120:0/1015060911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e1f1705d0 0x560e1f17f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.677+0000 7f19d02d6740 1 -- 172.21.15.120:0/1015060911 >> 172.21.15.120:0/1015060911 conn(0x560e1f139310 msgr2=0x560e1f172960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:21.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.677+0000 7f19d02d6740 1 -- 172.21.15.120:0/1015060911 shutdown_connections 2024-09-19T09:13:21.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.677+0000 7f19d02d6740 1 -- 172.21.15.120:0/1015060911 wait complete. 2024-09-19T09:13:21.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.677+0000 7f19d02d6740 20 client.19197 trim_cache size 0 max 16384 2024-09-19T09:13:21.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.690+0000 7f19d02d6740 1 Processor -- start 2024-09-19T09:13:21.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.690+0000 7f19d02d6740 1 -- start start 2024-09-19T09:13:21.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.690+0000 7f19d02d6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e1f1705d0 0x560e1f18b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:21.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.690+0000 7f19d02d6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e1f171b40 0x560e1f180b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:21.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.690+0000 7f19d02d6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e1f17f780 0x560e1f1810b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:21.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.690+0000 7f19d02d6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f19c81ab390 con 0x560e1f17f780 2024-09-19T09:13:21.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.690+0000 7f19d02d6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f19c81aced0 con 0x560e1f171b40 2024-09-19T09:13:21.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.690+0000 7f19d02d6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f19c81aba60 con 0x560e1f1705d0 2024-09-19T09:13:21.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.690+0000 7f19c7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e1f17f780 0x560e1f1810b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:21.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.690+0000 7f19c7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e1f17f780 0x560e1f1810b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56266/0 (socket says 172.21.15.120:56266) 2024-09-19T09:13:21.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.690+0000 7f19c7fff640 1 -- 172.21.15.120:0/413451388 learned_addr learned my addr 172.21.15.120:0/413451388 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:21.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.690+0000 7f19c77fe640 1 --2- 172.21.15.120:0/413451388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e1f1705d0 0x560e1f18b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:21.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.690+0000 7f19bffff640 1 --2- 172.21.15.120:0/413451388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e1f171b40 0x560e1f180b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:21.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.690+0000 7f19c7fff640 1 -- 172.21.15.120:0/413451388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e1f1705d0 msgr2=0x560e1f18b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:21.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.690+0000 7f19c7fff640 1 --2- 172.21.15.120:0/413451388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e1f1705d0 0x560e1f18b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.691+0000 7f19c7fff640 1 -- 172.21.15.120:0/413451388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e1f171b40 msgr2=0x560e1f180b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:21.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.691+0000 7f19c7fff640 1 --2- 172.21.15.120:0/413451388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e1f171b40 0x560e1f180b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.691+0000 7f19c7fff640 1 -- 172.21.15.120:0/413451388 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e1f17cb30 con 0x560e1f17f780 2024-09-19T09:13:21.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.691+0000 7f19bffff640 1 --2- 172.21.15.120:0/413451388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e1f171b40 0x560e1f180b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:13:21.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.691+0000 7f19c7fff640 1 --2- 172.21.15.120:0/413451388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e1f17f780 0x560e1f1810b0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f19c002f3c0 tx=0x7f19c002ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6821bfeddef9b635 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:21.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.691+0000 7f19bd7fa640 1 -- 172.21.15.120:0/413451388 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c0050070 con 0x560e1f17f780 2024-09-19T09:13:21.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.691+0000 7f19bd7fa640 1 -- 172.21.15.120:0/413451388 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19c002f700 con 0x560e1f17f780 2024-09-19T09:13:21.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.691+0000 7f19bd7fa640 1 -- 172.21.15.120:0/413451388 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c0039c60 con 0x560e1f17f780 2024-09-19T09:13:21.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.691+0000 7f19d02d6740 1 -- 172.21.15.120:0/413451388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e1f17f780 msgr2=0x560e1f1810b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:21.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.691+0000 7f19d02d6740 1 --2- 172.21.15.120:0/413451388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e1f17f780 0x560e1f1810b0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f19c002f3c0 tx=0x7f19c002ff70 comp rx=0 tx=0).stop 2024-09-19T09:13:21.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.691+0000 7f19d02d6740 1 -- 172.21.15.120:0/413451388 shutdown_connections 2024-09-19T09:13:21.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.691+0000 7f19d02d6740 1 --2- 172.21.15.120:0/413451388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e1f17f780 0x560e1f1810b0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.691+0000 7f19d02d6740 1 --2- 172.21.15.120:0/413451388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e1f171b40 0x560e1f180b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.691+0000 7f19d02d6740 1 --2- 172.21.15.120:0/413451388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e1f1705d0 0x560e1f18b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.691+0000 7f19d02d6740 1 -- 172.21.15.120:0/413451388 >> 172.21.15.120:0/413451388 conn(0x560e1f139310 msgr2=0x560e1f17ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:21.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.692+0000 7f19d02d6740 1 -- 172.21.15.120:0/413451388 shutdown_connections 2024-09-19T09:13:21.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.692+0000 7f19d02d6740 1 -- 172.21.15.120:0/413451388 wait complete. 2024-09-19T09:13:21.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.692+0000 7f19d02d6740 1 -- start start 2024-09-19T09:13:21.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.692+0000 7f19d02d6740 1 Processor -- start 2024-09-19T09:13:21.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.692+0000 7f19d02d6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e1f1705d0 0x560e1f188080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:21.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.692+0000 7f19d02d6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e1f171b40 0x560e1f196910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:21.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.692+0000 7f19d02d6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e1f17f780 0x560e1f196e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:21.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.692+0000 7f19d02d6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f19c81ac800 con 0x560e1f1705d0 2024-09-19T09:13:21.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.692+0000 7f19d02d6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f19c81ac130 con 0x560e1f17f780 2024-09-19T09:13:21.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.692+0000 7f19d02d6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f19c81a9350 con 0x560e1f171b40 2024-09-19T09:13:21.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.692+0000 7f19d02d6740 10 client.0 _ll_register_callbacks cb 0x7f19c0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:21.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.692+0000 7f19c77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e1f1705d0 0x560e1f188080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:21.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.692+0000 7f19c77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e1f1705d0 0x560e1f188080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56282/0 (socket says 172.21.15.120:56282) 2024-09-19T09:13:21.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.692+0000 7f19c77fe640 1 -- 172.21.15.120:0/3668099136 learned_addr learned my addr 172.21.15.120:0/3668099136 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:21.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.692+0000 7f19bffff640 1 --2- 172.21.15.120:0/3668099136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e1f171b40 0x560e1f196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:21.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.693+0000 7f19c77fe640 1 -- 172.21.15.120:0/3668099136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e1f171b40 msgr2=0x560e1f196910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:21.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.693+0000 7f19c77fe640 1 --2- 172.21.15.120:0/3668099136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e1f171b40 0x560e1f196910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.693+0000 7f19c77fe640 1 -- 172.21.15.120:0/3668099136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e1f17f780 msgr2=0x560e1f196e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:21.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.693+0000 7f19c77fe640 1 --2- 172.21.15.120:0/3668099136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e1f17f780 0x560e1f196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.693+0000 7f19c77fe640 1 -- 172.21.15.120:0/3668099136 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e1f197680 con 0x560e1f1705d0 2024-09-19T09:13:21.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.693+0000 7f19c77fe640 1 --2- 172.21.15.120:0/3668099136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e1f1705d0 0x560e1f188080 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f19b0009c20 tx=0x7f19b00096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b629be52085ccb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:21.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.693+0000 7f19a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:21.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.693+0000 7f19a6ffd640 1 -- 172.21.15.120:0/3668099136 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0004450 con 0x560e1f1705d0 2024-09-19T09:13:21.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.693+0000 7f19a6ffd640 1 -- 172.21.15.120:0/3668099136 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f19b00035f0 con 0x560e1f1705d0 2024-09-19T09:13:21.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.693+0000 7f19d02d6740 1 -- 172.21.15.120:0/3668099136 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19c002f700 con 0x560e1f1705d0 2024-09-19T09:13:21.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.693+0000 7f19a6ffd640 1 -- 172.21.15.120:0/3668099136 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f19b00037f0 con 0x560e1f1705d0 2024-09-19T09:13:21.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.693+0000 7f19d02d6740 10 client.19203 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:21.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.693+0000 7f19d02d6740 1 -- 172.21.15.120:0/3668099136 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560e1f1f9ff0 con 0x560e1f1705d0 2024-09-19T09:13:21.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.693+0000 7f19a6ffd640 1 -- 172.21.15.120:0/3668099136 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19c002f700 con 0x560e1f1705d0 2024-09-19T09:13:21.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.694+0000 7f19a6ffd640 1 -- 172.21.15.120:0/3668099136 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f19b0033050 con 0x560e1f1705d0 2024-09-19T09:13:21.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.694+0000 7f19d02d6740 10 client.19203 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:21.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.694+0000 7f19d02d6740 10 client.19203 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:13:21.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.694+0000 7f19d02d6740 20 client.19203 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:21.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.694+0000 7f19d02d6740 1 --2- 172.21.15.120:0/3668099136 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560e1f18f2a0 0x560e1f1fc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:21.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.694+0000 7f19d02d6740 4 client.19203 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:13:21.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.694+0000 7f19d02d6740 1 -- 172.21.15.120:0/3668099136 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560e1f0ddd70 con 0x560e1f18f2a0 2024-09-19T09:13:21.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.694+0000 7f19bffff640 1 --2- 172.21.15.120:0/3668099136 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560e1f18f2a0 0x560e1f1fc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:21.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.694+0000 7f19a6ffd640 10 client.19203 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:13:21.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.694+0000 7f19bffff640 1 --2- 172.21.15.120:0/3668099136 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560e1f18f2a0 0x560e1f1fc580 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5d64d27396d797ee server_cookie=d3f1ba1d76204eeb in_seq=0 out_seq=0 2024-09-19T09:13:21.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.694+0000 7f19a6ffd640 1 -- 172.21.15.120:0/3668099136 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560e1f0ddd70 con 0x560e1f18f2a0 2024-09-19T09:13:21.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.694+0000 7f19a6ffd640 10 client.19203 handle_command_reply: tid=0 2024-09-19T09:13:21.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.695+0000 7f19d02d6740 1 client.19203 shutdown 2024-09-19T09:13:21.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.695+0000 7f19d02d6740 1 -- 172.21.15.120:0/3668099136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e1f1705d0 msgr2=0x560e1f188080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:21.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.695+0000 7f19d02d6740 1 --2- 172.21.15.120:0/3668099136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e1f1705d0 0x560e1f188080 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f19b0009c20 tx=0x7f19b00096a0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.695+0000 7f19d02d6740 1 -- 172.21.15.120:0/3668099136 shutdown_connections 2024-09-19T09:13:21.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.695+0000 7f19d02d6740 1 --2- 172.21.15.120:0/3668099136 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560e1f18f2a0 0x560e1f1fc580 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.695+0000 7f19d02d6740 1 --2- 172.21.15.120:0/3668099136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560e1f17f780 0x560e1f196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.695+0000 7f19d02d6740 1 --2- 172.21.15.120:0/3668099136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e1f171b40 0x560e1f196910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.695+0000 7f19d02d6740 1 --2- 172.21.15.120:0/3668099136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560e1f1705d0 0x560e1f188080 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.695+0000 7f19d02d6740 1 -- 172.21.15.120:0/3668099136 >> 172.21.15.120:0/3668099136 conn(0x560e1f139310 msgr2=0x560e1f17dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:21.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.695+0000 7f19a6ffd640 0 client.19203 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:13:21.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.695+0000 7f19d02d6740 1 -- 172.21.15.120:0/3668099136 shutdown_connections 2024-09-19T09:13:21.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.695+0000 7f19d02d6740 1 -- 172.21.15.120:0/3668099136 wait complete. 2024-09-19T09:13:21.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.695+0000 7f19d02d6740 20 client.19203 trim_cache size 0 max 16384 2024-09-19T09:13:21.707 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:13:21.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.695+0000 7f19cdefd640 1 -- 172.21.15.120:0/659618747 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f19b40a3240 msgr2=0x7f19b40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:21.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.695+0000 7f19cdefd640 1 --2- 172.21.15.120:0/659618747 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f19b40a3240 0x7f19b40a5700 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f19c0009ca0 tx=0x7f19c00095c0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.695+0000 7f19cdefd640 1 -- 172.21.15.120:0/659618747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19c8103660 msgr2=0x7f19c81988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:21.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.695+0000 7f19cdefd640 1 --2- 172.21.15.120:0/659618747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19c8103660 0x7f19c81988b0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f19b800c970 tx=0x7f19b800ce30 comp rx=0 tx=0).stop 2024-09-19T09:13:21.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.696+0000 7f19cdefd640 1 -- 172.21.15.120:0/659618747 shutdown_connections 2024-09-19T09:13:21.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.696+0000 7f19cdefd640 1 --2- 172.21.15.120:0/659618747 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f19b40a3240 0x7f19b40a5700 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.696+0000 7f19cdefd640 1 --2- 172.21.15.120:0/659618747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19c810c170 0x7f19c819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.696+0000 7f19cdefd640 1 --2- 172.21.15.120:0/659618747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19c8103660 0x7f19c81988b0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.696+0000 7f19cdefd640 1 --2- 172.21.15.120:0/659618747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19c8102d40 0x7f19c8198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:21.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.696+0000 7f19cdefd640 1 -- 172.21.15.120:0/659618747 >> 172.21.15.120:0/659618747 conn(0x7f19c80fc7a0 msgr2=0x7f19c80fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:21.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.696+0000 7f19cdefd640 1 -- 172.21.15.120:0/659618747 shutdown_connections 2024-09-19T09:13:21.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:21.696+0000 7f19cdefd640 1 -- 172.21.15.120:0/659618747 wait complete. 2024-09-19T09:13:22.069 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.52 MB/sec execute 437 sec latency 5376.367 ms 2024-09-19T09:13:22.640 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:13:22.887 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:13:23.069 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.50 MB/sec execute 438 sec latency 6376.442 ms 2024-09-19T09:13:23.136 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:22 smithi120 ceph-mon[31150]: pgmap v658: 97 pgs: 97 active+clean; 37 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 166 op/s 2024-09-19T09:13:23.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:22 smithi196 ceph-mon[35150]: pgmap v658: 97 pgs: 97 active+clean; 37 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 166 op/s 2024-09-19T09:13:23.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:22 smithi153 ceph-mon[35071]: pgmap v658: 97 pgs: 97 active+clean; 37 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 166 op/s 2024-09-19T09:13:23.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.353+0000 7fe6a4d39640 1 -- 172.21.15.120:0/3406048844 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6a0106a80 msgr2=0x7fe6a0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:23.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.353+0000 7fe6a4d39640 1 --2- 172.21.15.120:0/3406048844 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6a0106a80 0x7fe6a0106e60 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fe688009930 tx=0x7fe68802a420 comp rx=0 tx=0).stop 2024-09-19T09:13:23.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.353+0000 7fe6a4d39640 1 -- 172.21.15.120:0/3406048844 shutdown_connections 2024-09-19T09:13:23.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.353+0000 7fe6a4d39640 1 --2- 172.21.15.120:0/3406048844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6a0101420 0x7fe6a010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.353+0000 7fe6a4d39640 1 --2- 172.21.15.120:0/3406048844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6a0100a80 0x7fe6a0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.353+0000 7fe6a4d39640 1 --2- 172.21.15.120:0/3406048844 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6a0106a80 0x7fe6a0106e60 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.353+0000 7fe6a4d39640 1 -- 172.21.15.120:0/3406048844 >> 172.21.15.120:0/3406048844 conn(0x7fe6a00fc7a0 msgr2=0x7fe6a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:23.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.353+0000 7fe6a4d39640 1 -- 172.21.15.120:0/3406048844 shutdown_connections 2024-09-19T09:13:23.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.353+0000 7fe6a4d39640 1 -- 172.21.15.120:0/3406048844 wait complete. 2024-09-19T09:13:23.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.354+0000 7fe6a4d39640 1 Processor -- start 2024-09-19T09:13:23.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.354+0000 7fe6a4d39640 1 -- start start 2024-09-19T09:13:23.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.354+0000 7fe6a4d39640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6a0100a80 0x7fe6a0199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:23.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.354+0000 7fe6a4d39640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6a0101420 0x7fe6a0199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:23.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.354+0000 7fe6a4d39640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6a0106a80 0x7fe6a01a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:23.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.354+0000 7fe6a4d39640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a0110f20 con 0x7fe6a0100a80 2024-09-19T09:13:23.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.354+0000 7fe6a4d39640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a0110da0 con 0x7fe6a0101420 2024-09-19T09:13:23.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.354+0000 7fe6a4d39640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a01110a0 con 0x7fe6a0106a80 2024-09-19T09:13:23.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.354+0000 7fe69e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6a0100a80 0x7fe6a0199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:23.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.354+0000 7fe69e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6a0100a80 0x7fe6a0199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56306/0 (socket says 172.21.15.120:56306) 2024-09-19T09:13:23.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.354+0000 7fe69e59c640 1 -- 172.21.15.120:0/1536708540 learned_addr learned my addr 172.21.15.120:0/1536708540 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:23.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.354+0000 7fe69dd9b640 1 --2- 172.21.15.120:0/1536708540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6a0101420 0x7fe6a0199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:23.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.354+0000 7fe69e59c640 1 -- 172.21.15.120:0/1536708540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6a0106a80 msgr2=0x7fe6a01a44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:23.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.354+0000 7fe69ed9d640 1 --2- 172.21.15.120:0/1536708540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6a0106a80 0x7fe6a01a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:23.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.354+0000 7fe69e59c640 1 --2- 172.21.15.120:0/1536708540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6a0106a80 0x7fe6a01a44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.354+0000 7fe69e59c640 1 -- 172.21.15.120:0/1536708540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6a0101420 msgr2=0x7fe6a0199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:23.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.354+0000 7fe69e59c640 1 --2- 172.21.15.120:0/1536708540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6a0101420 0x7fe6a0199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.354+0000 7fe69e59c640 1 -- 172.21.15.120:0/1536708540 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a01a4a10 con 0x7fe6a0100a80 2024-09-19T09:13:23.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.354+0000 7fe69dd9b640 1 --2- 172.21.15.120:0/1536708540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6a0101420 0x7fe6a0199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:13:23.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.355+0000 7fe69e59c640 1 --2- 172.21.15.120:0/1536708540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6a0100a80 0x7fe6a0199450 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fe688009a50 tx=0x7fe688004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:23.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.355+0000 7fe6977fe640 1 -- 172.21.15.120:0/1536708540 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68803f030 con 0x7fe6a0100a80 2024-09-19T09:13:23.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.355+0000 7fe6a4d39640 1 -- 172.21.15.120:0/1536708540 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a01a4be0 con 0x7fe6a0100a80 2024-09-19T09:13:23.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.355+0000 7fe6977fe640 1 -- 172.21.15.120:0/1536708540 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe688003ce0 con 0x7fe6a0100a80 2024-09-19T09:13:23.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.355+0000 7fe6977fe640 1 -- 172.21.15.120:0/1536708540 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688035070 con 0x7fe6a0100a80 2024-09-19T09:13:23.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.355+0000 7fe6a4d39640 1 -- 172.21.15.120:0/1536708540 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a01a5030 con 0x7fe6a0100a80 2024-09-19T09:13:23.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.357+0000 7fe6977fe640 1 -- 172.21.15.120:0/1536708540 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe688042030 con 0x7fe6a0100a80 2024-09-19T09:13:23.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.357+0000 7fe6a7112740 1 Processor -- start 2024-09-19T09:13:23.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.357+0000 7fe6a7112740 1 -- start start 2024-09-19T09:13:23.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.358+0000 7fe6a7112740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ee103fb40 0x559ee103e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:23.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.358+0000 7fe6a7112740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559ee103e5d0 0x559ee103ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:23.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.358+0000 7fe6a7112740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ee104d780 0x559ee104fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:23.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.358+0000 7fe6a7112740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559ee0fa42f0 con 0x559ee103e5d0 2024-09-19T09:13:23.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.358+0000 7fe6a7112740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559ee0f4bcf0 con 0x559ee103fb40 2024-09-19T09:13:23.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.358+0000 7fe6a7112740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559ee0fa02b0 con 0x559ee104d780 2024-09-19T09:13:23.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.358+0000 7fe69e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559ee103e5d0 0x559ee103ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:23.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.358+0000 7fe69e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559ee103e5d0 0x559ee103ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56308/0 (socket says 172.21.15.120:56308) 2024-09-19T09:13:23.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.358+0000 7fe69e59c640 1 -- 172.21.15.120:0/3985714884 learned_addr learned my addr 172.21.15.120:0/3985714884 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:23.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.358+0000 7fe69ed9d640 1 --2- 172.21.15.120:0/3985714884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ee103fb40 0x559ee103e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:23.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.358+0000 7fe6977fe640 1 --2- 172.21.15.120:0/1536708540 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe68c0a3240 0x7fe68c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:23.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.358+0000 7fe69dd9b640 1 --2- 172.21.15.120:0/3985714884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ee104d780 0x559ee104fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:23.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.358+0000 7fe69e59c640 1 -- 172.21.15.120:0/3985714884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ee104d780 msgr2=0x559ee104fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:23.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.358+0000 7fe69ed9d640 1 --2- 172.21.15.120:0/1536708540 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe68c0a3240 0x7fe68c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:23.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.358+0000 7fe69e59c640 1 --2- 172.21.15.120:0/3985714884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ee104d780 0x559ee104fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.358+0000 7fe69e59c640 1 -- 172.21.15.120:0/3985714884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ee103fb40 msgr2=0x559ee103e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:23.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.358+0000 7fe69e59c640 1 --2- 172.21.15.120:0/3985714884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ee103fb40 0x559ee103e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.358+0000 7fe69e59c640 1 -- 172.21.15.120:0/3985714884 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ee0f303a0 con 0x559ee103e5d0 2024-09-19T09:13:23.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.358+0000 7fe69dd9b640 1 --2- 172.21.15.120:0/3985714884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ee104d780 0x559ee104fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:13:23.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.358+0000 7fe6977fe640 1 -- 172.21.15.120:0/1536708540 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fe6880eff40 con 0x7fe6a0100a80 2024-09-19T09:13:23.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.359+0000 7fe69e59c640 1 --2- 172.21.15.120:0/3985714884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559ee103e5d0 0x559ee103ea10 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fe68803de10 tx=0x7fe68803b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db70ee6cb8aa2417 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:23.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.359+0000 7fe69ed9d640 1 --2- 172.21.15.120:0/1536708540 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe68c0a3240 0x7fe68c0a5700 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe698013700 tx=0x7fe698004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:23.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.359+0000 7fe694ff9640 1 -- 172.21.15.120:0/3985714884 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6880bc330 con 0x559ee103e5d0 2024-09-19T09:13:23.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.359+0000 7fe694ff9640 1 -- 172.21.15.120:0/3985714884 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6880f34f0 con 0x559ee103e5d0 2024-09-19T09:13:23.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.359+0000 7fe694ff9640 1 -- 172.21.15.120:0/3985714884 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688003800 con 0x559ee103e5d0 2024-09-19T09:13:23.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.359+0000 7fe6a7112740 1 -- 172.21.15.120:0/3985714884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559ee103e5d0 msgr2=0x559ee103ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:23.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.359+0000 7fe6a7112740 1 --2- 172.21.15.120:0/3985714884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559ee103e5d0 0x559ee103ea10 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fe68803de10 tx=0x7fe68803b800 comp rx=0 tx=0).stop 2024-09-19T09:13:23.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.359+0000 7fe6a7112740 1 -- 172.21.15.120:0/3985714884 shutdown_connections 2024-09-19T09:13:23.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.359+0000 7fe6a7112740 1 --2- 172.21.15.120:0/3985714884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ee104d780 0x559ee104fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.360+0000 7fe6a7112740 1 --2- 172.21.15.120:0/3985714884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559ee103e5d0 0x559ee103ea10 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.360+0000 7fe6a7112740 1 --2- 172.21.15.120:0/3985714884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ee103fb40 0x559ee103e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.360+0000 7fe6a7112740 1 -- 172.21.15.120:0/3985714884 >> 172.21.15.120:0/3985714884 conn(0x559ee1007310 msgr2=0x559ee103ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:23.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.360+0000 7fe6a7112740 1 -- 172.21.15.120:0/3985714884 shutdown_connections 2024-09-19T09:13:23.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.360+0000 7fe6a7112740 1 -- 172.21.15.120:0/3985714884 wait complete. 2024-09-19T09:13:23.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.360+0000 7fe6a7112740 1 -- start start 2024-09-19T09:13:23.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.360+0000 7fe6a7112740 1 Processor -- start 2024-09-19T09:13:23.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.360+0000 7fe6a7112740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ee103e5d0 0x559ee104d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:23.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.360+0000 7fe6a7112740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ee103fb40 0x559ee104a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:23.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.360+0000 7fe6a7112740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559ee104d780 0x559ee104a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:23.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.360+0000 7fe6a7112740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559ee0fa42f0 con 0x559ee104d780 2024-09-19T09:13:23.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.360+0000 7fe6a7112740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559ee0f5f060 con 0x559ee103e5d0 2024-09-19T09:13:23.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.360+0000 7fe6a7112740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559ee1050240 con 0x559ee103fb40 2024-09-19T09:13:23.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.360+0000 7fe6a7112740 10 client.0 _ll_register_callbacks cb 0x559ee0cb28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:23.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.360+0000 7fe69ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559ee104d780 0x559ee104a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:23.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.360+0000 7fe69ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559ee104d780 0x559ee104a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56318/0 (socket says 172.21.15.120:56318) 2024-09-19T09:13:23.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.360+0000 7fe69ed9d640 1 -- 172.21.15.120:0/1746166699 learned_addr learned my addr 172.21.15.120:0/1746166699 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:23.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.360+0000 7fe69e59c640 1 --2- 172.21.15.120:0/1746166699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ee103e5d0 0x559ee104d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:23.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.361+0000 7fe69dd9b640 1 --2- 172.21.15.120:0/1746166699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ee103fb40 0x559ee104a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:23.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.361+0000 7fe69ed9d640 1 -- 172.21.15.120:0/1746166699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ee103fb40 msgr2=0x559ee104a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:23.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.361+0000 7fe69ed9d640 1 --2- 172.21.15.120:0/1746166699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ee103fb40 0x559ee104a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.361+0000 7fe69ed9d640 1 -- 172.21.15.120:0/1746166699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ee103e5d0 msgr2=0x559ee104d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:23.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.361+0000 7fe69ed9d640 1 --2- 172.21.15.120:0/1746166699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ee103e5d0 0x559ee104d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.361+0000 7fe69ed9d640 1 -- 172.21.15.120:0/1746166699 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ee0d153c0 con 0x559ee104d780 2024-09-19T09:13:23.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.361+0000 7fe69e59c640 1 --2- 172.21.15.120:0/1746166699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ee103e5d0 0x559ee104d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:13:23.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.361+0000 7fe69ed9d640 1 --2- 172.21.15.120:0/1746166699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559ee104d780 0x559ee104a900 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fe698017dd0 tx=0x7fe698004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c312388ab67044 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:23.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.361+0000 7fe685ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:23.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.361+0000 7fe685ffb640 1 -- 172.21.15.120:0/1746166699 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698027060 con 0x559ee104d780 2024-09-19T09:13:23.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.361+0000 7fe685ffb640 1 -- 172.21.15.120:0/1746166699 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fe698022600 con 0x559ee104d780 2024-09-19T09:13:23.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.361+0000 7fe6a7112740 1 -- 172.21.15.120:0/1746166699 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559ee0fb1860 con 0x559ee104d780 2024-09-19T09:13:23.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.361+0000 7fe685ffb640 1 -- 172.21.15.120:0/1746166699 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe698022800 con 0x559ee104d780 2024-09-19T09:13:23.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.361+0000 7fe6a7112740 10 client.19215 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:23.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.361+0000 7fe6a7112740 1 -- 172.21.15.120:0/1746166699 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559ee105bbb0 con 0x559ee104d780 2024-09-19T09:13:23.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.361+0000 7fe685ffb640 1 -- 172.21.15.120:0/1746166699 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559ee0fb1860 con 0x559ee104d780 2024-09-19T09:13:23.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.361+0000 7fe685ffb640 1 -- 172.21.15.120:0/1746166699 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe69802c070 con 0x559ee104d780 2024-09-19T09:13:23.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.361+0000 7fe6a7112740 10 client.19215 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:23.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.362+0000 7fe6a7112740 10 client.19215 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:13:23.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.362+0000 7fe6a7112740 20 client.19215 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:23.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.362+0000 7fe6a7112740 1 --2- 172.21.15.120:0/1746166699 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x559ee105d070 0x559ee105f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:23.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.362+0000 7fe6a7112740 4 client.19215 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:13:23.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.362+0000 7fe6a7112740 1 -- 172.21.15.120:0/1746166699 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ee0fabd70 con 0x559ee105d070 2024-09-19T09:13:23.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.362+0000 7fe69e59c640 1 --2- 172.21.15.120:0/1746166699 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x559ee105d070 0x559ee105f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:23.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.362+0000 7fe685ffb640 10 client.19215 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:13:23.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.362+0000 7fe69e59c640 1 --2- 172.21.15.120:0/1746166699 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x559ee105d070 0x559ee105f4c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8a87ce6289e0699d server_cookie=eb6613914789638 in_seq=0 out_seq=0 2024-09-19T09:13:23.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.363+0000 7fe685ffb640 1 -- 172.21.15.120:0/1746166699 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559ee0fabd70 con 0x559ee105d070 2024-09-19T09:13:23.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.363+0000 7fe685ffb640 10 client.19215 handle_command_reply: tid=0 2024-09-19T09:13:23.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.363+0000 7fe6a7112740 1 client.19215 shutdown 2024-09-19T09:13:23.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.363+0000 7fe6a7112740 1 -- 172.21.15.120:0/1746166699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559ee104d780 msgr2=0x559ee104a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:23.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.363+0000 7fe6a7112740 1 --2- 172.21.15.120:0/1746166699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559ee104d780 0x559ee104a900 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fe698017dd0 tx=0x7fe698004b50 comp rx=0 tx=0).stop 2024-09-19T09:13:23.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.363+0000 7fe6a7112740 1 -- 172.21.15.120:0/1746166699 shutdown_connections 2024-09-19T09:13:23.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.363+0000 7fe6a7112740 1 --2- 172.21.15.120:0/1746166699 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x559ee105d070 0x559ee105f4c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.363+0000 7fe6a7112740 1 --2- 172.21.15.120:0/1746166699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559ee104d780 0x559ee104a900 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.363+0000 7fe6a7112740 1 --2- 172.21.15.120:0/1746166699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ee103fb40 0x559ee104a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.363+0000 7fe6a7112740 1 --2- 172.21.15.120:0/1746166699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ee103e5d0 0x559ee104d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.363+0000 7fe6a7112740 1 -- 172.21.15.120:0/1746166699 >> 172.21.15.120:0/1746166699 conn(0x559ee1007310 msgr2=0x559ee1040930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:23.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.363+0000 7fe685ffb640 0 client.19215 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:13:23.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.363+0000 7fe6a7112740 1 -- 172.21.15.120:0/1746166699 shutdown_connections 2024-09-19T09:13:23.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.363+0000 7fe6a7112740 1 -- 172.21.15.120:0/1746166699 wait complete. 2024-09-19T09:13:23.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.363+0000 7fe6a7112740 20 client.19215 trim_cache size 0 max 16384 2024-09-19T09:13:23.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.376+0000 7fe6a7112740 1 Processor -- start 2024-09-19T09:13:23.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.376+0000 7fe6a7112740 1 -- start start 2024-09-19T09:13:23.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.376+0000 7fe6a7112740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559ee103e5d0 0x559ee103ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:23.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.376+0000 7fe6a7112740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ee103fb40 0x559ee1059390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:23.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.376+0000 7fe6a7112740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ee104d780 0x559ee1064d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:23.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.376+0000 7fe6a7112740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a01ab490 con 0x559ee103e5d0 2024-09-19T09:13:23.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.376+0000 7fe6a7112740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a01acfd0 con 0x559ee103fb40 2024-09-19T09:13:23.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.376+0000 7fe6a7112740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a01abb60 con 0x559ee104d780 2024-09-19T09:13:23.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.376+0000 7fe69e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559ee103e5d0 0x559ee103ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:23.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.377+0000 7fe69e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559ee103e5d0 0x559ee103ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56322/0 (socket says 172.21.15.120:56322) 2024-09-19T09:13:23.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.377+0000 7fe69e59c640 1 -- 172.21.15.120:0/3817301596 learned_addr learned my addr 172.21.15.120:0/3817301596 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:23.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.377+0000 7fe69dd9b640 1 --2- 172.21.15.120:0/3817301596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ee103fb40 0x559ee1059390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:23.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.377+0000 7fe69ed9d640 1 --2- 172.21.15.120:0/3817301596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ee104d780 0x559ee1064d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:23.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.377+0000 7fe69e59c640 1 -- 172.21.15.120:0/3817301596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ee104d780 msgr2=0x559ee1064d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:23.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.377+0000 7fe69e59c640 1 --2- 172.21.15.120:0/3817301596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ee104d780 0x559ee1064d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.377+0000 7fe69e59c640 1 -- 172.21.15.120:0/3817301596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ee103fb40 msgr2=0x559ee1059390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:23.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.377+0000 7fe69e59c640 1 --2- 172.21.15.120:0/3817301596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ee103fb40 0x559ee1059390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.377+0000 7fe69e59c640 1 -- 172.21.15.120:0/3817301596 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ee1041120 con 0x559ee103e5d0 2024-09-19T09:13:23.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.377+0000 7fe69dd9b640 1 --2- 172.21.15.120:0/3817301596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ee103fb40 0x559ee1059390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:13:23.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.377+0000 7fe69e59c640 1 --2- 172.21.15.120:0/3817301596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559ee103e5d0 0x559ee103ae10 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fe6880bc4f0 tx=0x7fe6880f39c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c953bf860d5475e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:23.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.377+0000 7fe687fff640 1 -- 172.21.15.120:0/3817301596 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688049040 con 0x559ee103e5d0 2024-09-19T09:13:23.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.377+0000 7fe687fff640 1 -- 172.21.15.120:0/3817301596 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6880c51d0 con 0x559ee103e5d0 2024-09-19T09:13:23.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.377+0000 7fe687fff640 1 -- 172.21.15.120:0/3817301596 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6880bbec0 con 0x559ee103e5d0 2024-09-19T09:13:23.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.378+0000 7fe6a7112740 1 -- 172.21.15.120:0/3817301596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559ee103e5d0 msgr2=0x559ee103ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:23.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.378+0000 7fe6a7112740 1 --2- 172.21.15.120:0/3817301596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559ee103e5d0 0x559ee103ae10 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fe6880bc4f0 tx=0x7fe6880f39c0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.378+0000 7fe6a7112740 1 -- 172.21.15.120:0/3817301596 shutdown_connections 2024-09-19T09:13:23.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.378+0000 7fe6a7112740 1 --2- 172.21.15.120:0/3817301596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ee104d780 0x559ee1064d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.378+0000 7fe6a7112740 1 --2- 172.21.15.120:0/3817301596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ee103fb40 0x559ee1059390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.378+0000 7fe6a7112740 1 --2- 172.21.15.120:0/3817301596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559ee103e5d0 0x559ee103ae10 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.378+0000 7fe6a7112740 1 -- 172.21.15.120:0/3817301596 >> 172.21.15.120:0/3817301596 conn(0x559ee1007310 msgr2=0x559ee103a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:23.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.378+0000 7fe6a7112740 1 -- 172.21.15.120:0/3817301596 shutdown_connections 2024-09-19T09:13:23.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.378+0000 7fe6a7112740 1 -- 172.21.15.120:0/3817301596 wait complete. 2024-09-19T09:13:23.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.378+0000 7fe6a7112740 1 -- start start 2024-09-19T09:13:23.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.378+0000 7fe6a7112740 1 Processor -- start 2024-09-19T09:13:23.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.379+0000 7fe6a7112740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559ee103e5d0 0x559ee10556e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:23.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.379+0000 7fe6a7112740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ee103fb40 0x559ee1055c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:23.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.379+0000 7fe6a7112740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ee104d780 0x559ee1046c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:23.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.379+0000 7fe6a7112740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a01ac900 con 0x559ee103e5d0 2024-09-19T09:13:23.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.379+0000 7fe6a7112740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a01ac230 con 0x559ee103fb40 2024-09-19T09:13:23.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.379+0000 7fe6a7112740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a01a9450 con 0x559ee104d780 2024-09-19T09:13:23.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.379+0000 7fe69e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559ee103e5d0 0x559ee10556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:23.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.379+0000 7fe6a7112740 10 client.0 _ll_register_callbacks cb 0x7fe698007da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:23.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.379+0000 7fe69e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559ee103e5d0 0x559ee10556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56324/0 (socket says 172.21.15.120:56324) 2024-09-19T09:13:23.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.379+0000 7fe69e59c640 1 -- 172.21.15.120:0/583062012 learned_addr learned my addr 172.21.15.120:0/583062012 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:23.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.379+0000 7fe69dd9b640 1 --2- 172.21.15.120:0/583062012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ee103fb40 0x559ee1055c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:23.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.379+0000 7fe69ed9d640 1 --2- 172.21.15.120:0/583062012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ee104d780 0x559ee1046c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:23.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.379+0000 7fe69e59c640 1 -- 172.21.15.120:0/583062012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ee104d780 msgr2=0x559ee1046c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:23.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.379+0000 7fe69e59c640 1 --2- 172.21.15.120:0/583062012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ee104d780 0x559ee1046c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.379+0000 7fe69e59c640 1 -- 172.21.15.120:0/583062012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ee103fb40 msgr2=0x559ee1055c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:23.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.379+0000 7fe69e59c640 1 --2- 172.21.15.120:0/583062012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ee103fb40 0x559ee1055c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.379+0000 7fe69e59c640 1 -- 172.21.15.120:0/583062012 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ee1047430 con 0x559ee103e5d0 2024-09-19T09:13:23.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.379+0000 7fe69e59c640 1 --2- 172.21.15.120:0/583062012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559ee103e5d0 0x559ee10556e0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fe6880f3040 tx=0x7fe68802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1157a4fcde29bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:23.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.379+0000 7fe685ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:23.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.379+0000 7fe685ffb640 1 -- 172.21.15.120:0/583062012 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688049040 con 0x559ee103e5d0 2024-09-19T09:13:23.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.379+0000 7fe685ffb640 1 -- 172.21.15.120:0/583062012 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fe6880ba600 con 0x559ee103e5d0 2024-09-19T09:13:23.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.379+0000 7fe6a7112740 1 -- 172.21.15.120:0/583062012 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6880c51d0 con 0x559ee103e5d0 2024-09-19T09:13:23.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.380+0000 7fe685ffb640 1 -- 172.21.15.120:0/583062012 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe6880bc0a0 con 0x559ee103e5d0 2024-09-19T09:13:23.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.380+0000 7fe6a7112740 10 client.19221 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:23.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.380+0000 7fe6a7112740 1 -- 172.21.15.120:0/583062012 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559ee1047430 con 0x559ee103e5d0 2024-09-19T09:13:23.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.380+0000 7fe685ffb640 1 -- 172.21.15.120:0/583062012 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6880c51d0 con 0x559ee103e5d0 2024-09-19T09:13:23.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.380+0000 7fe685ffb640 1 -- 172.21.15.120:0/583062012 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe6880edac0 con 0x559ee103e5d0 2024-09-19T09:13:23.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.380+0000 7fe6a7112740 10 client.19221 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:23.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.380+0000 7fe6a7112740 10 client.19221 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:13:23.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.380+0000 7fe6a7112740 20 client.19221 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:23.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.380+0000 7fe6a7112740 1 --2- 172.21.15.120:0/583062012 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x559ee10c78c0 0x559ee10c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:23.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.380+0000 7fe6a7112740 4 client.19221 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:13:23.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.380+0000 7fe6a7112740 1 -- 172.21.15.120:0/583062012 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559ee0f28170 con 0x559ee10c78c0 2024-09-19T09:13:23.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.380+0000 7fe69dd9b640 1 --2- 172.21.15.120:0/583062012 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x559ee10c78c0 0x559ee10c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:23.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.381+0000 7fe685ffb640 10 client.19221 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:13:23.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.381+0000 7fe69dd9b640 1 --2- 172.21.15.120:0/583062012 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x559ee10c78c0 0x559ee10c9cb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=645cd2d702baa657 server_cookie=1d2371448f356c7d in_seq=0 out_seq=0 2024-09-19T09:13:23.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.381+0000 7fe685ffb640 1 -- 172.21.15.120:0/583062012 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559ee0f28170 con 0x559ee10c78c0 2024-09-19T09:13:23.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.381+0000 7fe685ffb640 10 client.19221 handle_command_reply: tid=0 2024-09-19T09:13:23.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.381+0000 7fe6a7112740 1 client.19221 shutdown 2024-09-19T09:13:23.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.381+0000 7fe6a7112740 1 -- 172.21.15.120:0/583062012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559ee103e5d0 msgr2=0x559ee10556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:23.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.381+0000 7fe6a7112740 1 --2- 172.21.15.120:0/583062012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559ee103e5d0 0x559ee10556e0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fe6880f3040 tx=0x7fe68802a420 comp rx=0 tx=0).stop 2024-09-19T09:13:23.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.381+0000 7fe6a7112740 1 -- 172.21.15.120:0/583062012 shutdown_connections 2024-09-19T09:13:23.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.381+0000 7fe6a7112740 1 --2- 172.21.15.120:0/583062012 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x559ee10c78c0 0x559ee10c9cb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.381+0000 7fe6a7112740 1 --2- 172.21.15.120:0/583062012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ee104d780 0x559ee1046c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.381+0000 7fe6a7112740 1 --2- 172.21.15.120:0/583062012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559ee103fb40 0x559ee1055c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.381+0000 7fe6a7112740 1 --2- 172.21.15.120:0/583062012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559ee103e5d0 0x559ee10556e0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.381+0000 7fe6a7112740 1 -- 172.21.15.120:0/583062012 >> 172.21.15.120:0/583062012 conn(0x559ee1007310 msgr2=0x559ee104bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:23.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.381+0000 7fe685ffb640 0 client.19221 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:13:23.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.381+0000 7fe6a7112740 1 -- 172.21.15.120:0/583062012 shutdown_connections 2024-09-19T09:13:23.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.381+0000 7fe6a7112740 1 -- 172.21.15.120:0/583062012 wait complete. 2024-09-19T09:13:23.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.381+0000 7fe6a7112740 20 client.19221 trim_cache size 0 max 16384 2024-09-19T09:13:23.406 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:13:23.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.382+0000 7fe6a4d39640 1 -- 172.21.15.120:0/1536708540 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe68c0a3240 msgr2=0x7fe68c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:23.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.382+0000 7fe6a4d39640 1 --2- 172.21.15.120:0/1536708540 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe68c0a3240 0x7fe68c0a5700 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe698013700 tx=0x7fe698004a40 comp rx=0 tx=0).stop 2024-09-19T09:13:23.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.382+0000 7fe6a4d39640 1 -- 172.21.15.120:0/1536708540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6a0100a80 msgr2=0x7fe6a0199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:23.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.382+0000 7fe6a4d39640 1 --2- 172.21.15.120:0/1536708540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6a0100a80 0x7fe6a0199450 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fe688009a50 tx=0x7fe688004270 comp rx=0 tx=0).stop 2024-09-19T09:13:23.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.382+0000 7fe6a4d39640 1 -- 172.21.15.120:0/1536708540 shutdown_connections 2024-09-19T09:13:23.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.382+0000 7fe6a4d39640 1 --2- 172.21.15.120:0/1536708540 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe68c0a3240 0x7fe68c0a5700 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.382+0000 7fe6a4d39640 1 --2- 172.21.15.120:0/1536708540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6a0106a80 0x7fe6a01a44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.382+0000 7fe6a4d39640 1 --2- 172.21.15.120:0/1536708540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6a0101420 0x7fe6a0199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.382+0000 7fe6a4d39640 1 --2- 172.21.15.120:0/1536708540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6a0100a80 0x7fe6a0199450 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:23.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.382+0000 7fe6a4d39640 1 -- 172.21.15.120:0/1536708540 >> 172.21.15.120:0/1536708540 conn(0x7fe6a00fc7a0 msgr2=0x7fe6a00fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:23.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.382+0000 7fe6a4d39640 1 -- 172.21.15.120:0/1536708540 shutdown_connections 2024-09-19T09:13:23.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:23.382+0000 7fe6a4d39640 1 -- 172.21.15.120:0/1536708540 wait complete. 2024-09-19T09:13:24.070 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.47 MB/sec execute 439 sec latency 7376.534 ms 2024-09-19T09:13:24.104 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:13:24.352 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:13:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd979f640 1 -- 172.21.15.120:0/2414780072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdd4104f90 msgr2=0x7efdd4107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd979f640 1 --2- 172.21.15.120:0/2414780072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdd4104f90 0x7efdd4107380 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7efdbc009900 tx=0x7efdbc02a410 comp rx=0 tx=0).stop 2024-09-19T09:13:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd979f640 1 -- 172.21.15.120:0/2414780072 shutdown_connections 2024-09-19T09:13:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd979f640 1 --2- 172.21.15.120:0/2414780072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efdd41078c0 0x7efdd4109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd979f640 1 --2- 172.21.15.120:0/2414780072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdd4104f90 0x7efdd4107380 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd979f640 1 --2- 172.21.15.120:0/2414780072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdd4102660 0x7efdd4104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd979f640 1 -- 172.21.15.120:0/2414780072 >> 172.21.15.120:0/2414780072 conn(0x7efdd40fc1e0 msgr2=0x7efdd40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd979f640 1 -- 172.21.15.120:0/2414780072 shutdown_connections 2024-09-19T09:13:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd979f640 1 -- 172.21.15.120:0/2414780072 wait complete. 2024-09-19T09:13:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd979f640 1 Processor -- start 2024-09-19T09:13:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd979f640 1 -- start start 2024-09-19T09:13:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd979f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdd4102660 0x7efdd4197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd979f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efdd4104f90 0x7efdd4198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd979f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdd41078c0 0x7efdd419f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd979f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efdd410d7d0 con 0x7efdd41078c0 2024-09-19T09:13:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd979f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efdd410d650 con 0x7efdd4102660 2024-09-19T09:13:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd979f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efdd410d950 con 0x7efdd4104f90 2024-09-19T09:13:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdd41078c0 0x7efdd419f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdd41078c0 0x7efdd419f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56336/0 (socket says 172.21.15.120:56336) 2024-09-19T09:13:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd37fe640 1 -- 172.21.15.120:0/143644259 learned_addr learned my addr 172.21.15.120:0/143644259 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd2ffd640 1 --2- 172.21.15.120:0/143644259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdd4102660 0x7efdd4197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd27fc640 1 --2- 172.21.15.120:0/143644259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efdd4104f90 0x7efdd4198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd37fe640 1 -- 172.21.15.120:0/143644259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efdd4104f90 msgr2=0x7efdd4198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd37fe640 1 --2- 172.21.15.120:0/143644259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efdd4104f90 0x7efdd4198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd37fe640 1 -- 172.21.15.120:0/143644259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdd4102660 msgr2=0x7efdd4197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd37fe640 1 --2- 172.21.15.120:0/143644259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdd4102660 0x7efdd4197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd37fe640 1 -- 172.21.15.120:0/143644259 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd419fa60 con 0x7efdd41078c0 2024-09-19T09:13:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd37fe640 1 --2- 172.21.15.120:0/143644259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdd41078c0 0x7efdd419f3f0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7efdc400e9d0 tx=0x7efdc400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdcbfff640 1 -- 172.21.15.120:0/143644259 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc400cd80 con 0x7efdd41078c0 2024-09-19T09:13:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd979f640 1 -- 172.21.15.120:0/143644259 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd419fd50 con 0x7efdd41078c0 2024-09-19T09:13:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdcbfff640 1 -- 172.21.15.120:0/143644259 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdc4005020 con 0x7efdd41078c0 2024-09-19T09:13:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdcbfff640 1 -- 172.21.15.120:0/143644259 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc40116d0 con 0x7efdd41078c0 2024-09-19T09:13:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdd979f640 1 -- 172.21.15.120:0/143644259 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd41a0210 con 0x7efdd41078c0 2024-09-19T09:13:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efdcbfff640 1 -- 172.21.15.120:0/143644259 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7efdc40051d0 con 0x7efdd41078c0 2024-09-19T09:13:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efddc615740 1 Processor -- start 2024-09-19T09:13:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efddc615740 1 -- start start 2024-09-19T09:13:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efddc615740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626c06f99a0 0x5626c06f9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efddc615740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626c06fa2e0 0x5626c06feda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efddc615740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626c06ff370 0x5626c0701830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efddc615740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5626c065d2f0 con 0x5626c06f99a0 2024-09-19T09:13:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efddc615740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5626c0605270 con 0x5626c06ff370 2024-09-19T09:13:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.689+0000 7efddc615740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5626c06592b0 con 0x5626c06fa2e0 2024-09-19T09:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efdd27fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626c06f99a0 0x5626c06f9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efdcbfff640 1 --2- 172.21.15.120:0/143644259 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efdc00a32f0 0x7efdc00a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efdd27fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626c06f99a0 0x5626c06f9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56340/0 (socket says 172.21.15.120:56340) 2024-09-19T09:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efdd27fc640 1 -- 172.21.15.120:0/3089222736 learned_addr learned my addr 172.21.15.120:0/3089222736 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efdd2ffd640 1 --2- 172.21.15.120:0/3089222736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626c06fa2e0 0x5626c06feda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efdd37fe640 1 --2- 172.21.15.120:0/143644259 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efdc00a32f0 0x7efdc00a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efdd27fc640 1 --2- 172.21.15.120:0/3089222736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626c06ff370 0x5626c0701830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efdcbfff640 1 -- 172.21.15.120:0/143644259 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7efdc40cfe70 con 0x7efdd41078c0 2024-09-19T09:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efdd27fc640 1 -- 172.21.15.120:0/3089222736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626c06fa2e0 msgr2=0x5626c06feda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efdd27fc640 1 --2- 172.21.15.120:0/3089222736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626c06fa2e0 0x5626c06feda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efdd27fc640 1 -- 172.21.15.120:0/3089222736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626c06ff370 msgr2=0x5626c0701830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efdd27fc640 1 --2- 172.21.15.120:0/3089222736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626c06ff370 0x5626c0701830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efdd27fc640 1 -- 172.21.15.120:0/3089222736 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626c0604340 con 0x5626c06f99a0 2024-09-19T09:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efdd27fc640 1 --2- 172.21.15.120:0/3089222736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626c06f99a0 0x5626c06f9da0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7efdbc031440 tx=0x7efdbc009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447f7b5d08fb8cbe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efdd37fe640 1 --2- 172.21.15.120:0/143644259 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efdc00a32f0 0x7efdc00a57b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7efdc4098e60 tx=0x7efdc4066ed0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efdc97fa640 1 -- 172.21.15.120:0/3089222736 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc035070 con 0x5626c06f99a0 2024-09-19T09:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efdc97fa640 1 -- 172.21.15.120:0/3089222736 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdbc044070 con 0x5626c06f99a0 2024-09-19T09:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efdc97fa640 1 -- 172.21.15.120:0/3089222736 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc004920 con 0x5626c06f99a0 2024-09-19T09:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efddc615740 1 -- 172.21.15.120:0/3089222736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626c06f99a0 msgr2=0x5626c06f9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efddc615740 1 --2- 172.21.15.120:0/3089222736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626c06f99a0 0x5626c06f9da0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7efdbc031440 tx=0x7efdbc009620 comp rx=0 tx=0).stop 2024-09-19T09:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efddc615740 1 -- 172.21.15.120:0/3089222736 shutdown_connections 2024-09-19T09:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efddc615740 1 --2- 172.21.15.120:0/3089222736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626c06ff370 0x5626c0701830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efddc615740 1 --2- 172.21.15.120:0/3089222736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626c06fa2e0 0x5626c06feda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efddc615740 1 --2- 172.21.15.120:0/3089222736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626c06f99a0 0x5626c06f9da0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efddc615740 1 -- 172.21.15.120:0/3089222736 >> 172.21.15.120:0/3089222736 conn(0x5626c06f5010 msgr2=0x5626c06f5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efddc615740 1 -- 172.21.15.120:0/3089222736 shutdown_connections 2024-09-19T09:13:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efddc615740 1 -- 172.21.15.120:0/3089222736 wait complete. 2024-09-19T09:13:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efddc615740 1 -- start start 2024-09-19T09:13:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efddc615740 1 Processor -- start 2024-09-19T09:13:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efddc615740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626c06f99a0 0x5626c0703b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efddc615740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626c06fa2e0 0x5626c07040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efddc615740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626c06ff370 0x5626c070b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efddc615740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5626c07022e0 con 0x5626c06ff370 2024-09-19T09:13:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efddc615740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5626c05e6290 con 0x5626c06fa2e0 2024-09-19T09:13:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efddc615740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5626c0702460 con 0x5626c06f99a0 2024-09-19T09:13:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efddc615740 10 client.0 _ll_register_callbacks cb 0x5626c0617140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efdd27fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626c06ff370 0x5626c070b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efdd27fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626c06ff370 0x5626c070b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56342/0 (socket says 172.21.15.120:56342) 2024-09-19T09:13:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efdd27fc640 1 -- 172.21.15.120:0/3747532447 learned_addr learned my addr 172.21.15.120:0/3747532447 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.694+0000 7efdd27fc640 1 --2- 172.21.15.120:0/3747532447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626c06f99a0 0x5626c0703b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.696+0000 7efdd2ffd640 1 --2- 172.21.15.120:0/3747532447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626c06fa2e0 0x5626c07040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.696+0000 7efdd27fc640 1 -- 172.21.15.120:0/3747532447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626c06f99a0 msgr2=0x5626c0703b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.696+0000 7efdd27fc640 1 --2- 172.21.15.120:0/3747532447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626c06f99a0 0x5626c0703b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.696+0000 7efdd27fc640 1 -- 172.21.15.120:0/3747532447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626c06fa2e0 msgr2=0x5626c07040a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.696+0000 7efdd27fc640 1 --2- 172.21.15.120:0/3747532447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626c06fa2e0 0x5626c07040a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.696+0000 7efdd27fc640 1 -- 172.21.15.120:0/3747532447 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626c0370210 con 0x5626c06ff370 2024-09-19T09:13:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.696+0000 7efdd2ffd640 1 --2- 172.21.15.120:0/3747532447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626c06fa2e0 0x5626c07040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:13:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.696+0000 7efdd27fc640 1 --2- 172.21.15.120:0/3747532447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626c06ff370 0x5626c070b0e0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x5626c0703090 tx=0x7efdbc03f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608d34042aaf12 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.696+0000 7efdba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.696+0000 7efdba7fc640 1 -- 172.21.15.120:0/3747532447 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc035030 con 0x5626c06ff370 2024-09-19T09:13:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.696+0000 7efdba7fc640 1 -- 172.21.15.120:0/3747532447 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7efdbc0048c0 con 0x5626c06ff370 2024-09-19T09:13:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.696+0000 7efddc615740 1 -- 172.21.15.120:0/3747532447 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdbc044070 con 0x5626c06ff370 2024-09-19T09:13:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.696+0000 7efdba7fc640 1 -- 172.21.15.120:0/3747532447 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efdbc0044e0 con 0x5626c06ff370 2024-09-19T09:13:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.696+0000 7efddc615740 10 client.19233 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.696+0000 7efddc615740 1 -- 172.21.15.120:0/3747532447 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5626c043c620 con 0x5626c06ff370 2024-09-19T09:13:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.697+0000 7efdba7fc640 1 -- 172.21.15.120:0/3747532447 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdbc044070 con 0x5626c06ff370 2024-09-19T09:13:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.697+0000 7efdba7fc640 1 -- 172.21.15.120:0/3747532447 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7efdbc03d450 con 0x5626c06ff370 2024-09-19T09:13:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.697+0000 7efddc615740 10 client.19233 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.697+0000 7efddc615740 10 client.19233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:13:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.697+0000 7efddc615740 20 client.19233 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.697+0000 7efddc615740 1 --2- 172.21.15.120:0/3747532447 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5626c070f3e0 0x5626c07117d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.697+0000 7efddc615740 4 client.19233 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:13:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.697+0000 7efddc615740 1 -- 172.21.15.120:0/3747532447 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626c066aa20 con 0x5626c070f3e0 2024-09-19T09:13:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.697+0000 7efdd2ffd640 1 --2- 172.21.15.120:0/3747532447 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5626c070f3e0 0x5626c07117d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.697+0000 7efdba7fc640 10 client.19233 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.697+0000 7efdd2ffd640 1 --2- 172.21.15.120:0/3747532447 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5626c070f3e0 0x5626c07117d0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae8999bcdd74e7db server_cookie=a07be8b2cbe74ce7 in_seq=0 out_seq=0 2024-09-19T09:13:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.698+0000 7efdba7fc640 1 -- 172.21.15.120:0/3747532447 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5626c066aa20 con 0x5626c070f3e0 2024-09-19T09:13:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.698+0000 7efdba7fc640 10 client.19233 handle_command_reply: tid=0 2024-09-19T09:13:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.698+0000 7efddc615740 1 client.19233 shutdown 2024-09-19T09:13:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.698+0000 7efddc615740 1 -- 172.21.15.120:0/3747532447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626c06ff370 msgr2=0x5626c070b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.698+0000 7efddc615740 1 --2- 172.21.15.120:0/3747532447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626c06ff370 0x5626c070b0e0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x5626c0703090 tx=0x7efdbc03f040 comp rx=0 tx=0).stop 2024-09-19T09:13:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.698+0000 7efddc615740 1 -- 172.21.15.120:0/3747532447 shutdown_connections 2024-09-19T09:13:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.698+0000 7efddc615740 1 --2- 172.21.15.120:0/3747532447 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5626c070f3e0 0x5626c07117d0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.698+0000 7efddc615740 1 --2- 172.21.15.120:0/3747532447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626c06ff370 0x5626c070b0e0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.698+0000 7efddc615740 1 --2- 172.21.15.120:0/3747532447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626c06fa2e0 0x5626c07040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.698+0000 7efdba7fc640 0 client.19233 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.698+0000 7efddc615740 1 --2- 172.21.15.120:0/3747532447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626c06f99a0 0x5626c0703b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.698+0000 7efddc615740 1 -- 172.21.15.120:0/3747532447 >> 172.21.15.120:0/3747532447 conn(0x5626c06f5010 msgr2=0x5626c06f6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.698+0000 7efddc615740 1 -- 172.21.15.120:0/3747532447 shutdown_connections 2024-09-19T09:13:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.698+0000 7efddc615740 1 -- 172.21.15.120:0/3747532447 wait complete. 2024-09-19T09:13:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.699+0000 7efddc615740 20 client.19233 trim_cache size 0 max 16384 2024-09-19T09:13:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.713+0000 7efddc615740 1 Processor -- start 2024-09-19T09:13:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.713+0000 7efddc615740 1 -- start start 2024-09-19T09:13:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.713+0000 7efddc615740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626c06f99a0 0x5626c06f39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.713+0000 7efddc615740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626c06fa2e0 0x5626c070b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.713+0000 7efddc615740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626c06ff370 0x5626c06f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.713+0000 7efddc615740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efdd41a8fd0 con 0x5626c06f99a0 2024-09-19T09:13:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.713+0000 7efddc615740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efdd41aab10 con 0x5626c06fa2e0 2024-09-19T09:13:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.713+0000 7efddc615740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efdd41a96a0 con 0x5626c06ff370 2024-09-19T09:13:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.713+0000 7efdd27fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626c06f99a0 0x5626c06f39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.713+0000 7efdd27fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626c06f99a0 0x5626c06f39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56358/0 (socket says 172.21.15.120:56358) 2024-09-19T09:13:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.713+0000 7efdd27fc640 1 -- 172.21.15.120:0/649301904 learned_addr learned my addr 172.21.15.120:0/649301904 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.713+0000 7efdd2ffd640 1 --2- 172.21.15.120:0/649301904 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626c06fa2e0 0x5626c070b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.713+0000 7efdd27fc640 1 -- 172.21.15.120:0/649301904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626c06ff370 msgr2=0x5626c06f3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.713+0000 7efdd27fc640 1 --2- 172.21.15.120:0/649301904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626c06ff370 0x5626c06f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.713+0000 7efdd27fc640 1 -- 172.21.15.120:0/649301904 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626c06fa2e0 msgr2=0x5626c070b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.713+0000 7efdd27fc640 1 --2- 172.21.15.120:0/649301904 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626c06fa2e0 0x5626c070b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.713+0000 7efdd27fc640 1 -- 172.21.15.120:0/649301904 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626c06fa7c0 con 0x5626c06f99a0 2024-09-19T09:13:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.713+0000 7efdd2ffd640 1 --2- 172.21.15.120:0/649301904 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626c06fa2e0 0x5626c070b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:13:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.714+0000 7efdd27fc640 1 --2- 172.21.15.120:0/649301904 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626c06f99a0 0x5626c06f39a0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0092a0 tx=0x7efdbc005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0fd2a0c75b5f3f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.714+0000 7efdc8ff9640 1 -- 172.21.15.120:0/649301904 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc035070 con 0x5626c06f99a0 2024-09-19T09:13:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.714+0000 7efdc8ff9640 1 -- 172.21.15.120:0/649301904 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdbc044030 con 0x5626c06f99a0 2024-09-19T09:13:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.714+0000 7efdc8ff9640 1 -- 172.21.15.120:0/649301904 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc05e070 con 0x5626c06f99a0 2024-09-19T09:13:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.714+0000 7efddc615740 1 -- 172.21.15.120:0/649301904 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626c06f99a0 msgr2=0x5626c06f39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.714+0000 7efddc615740 1 --2- 172.21.15.120:0/649301904 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626c06f99a0 0x5626c06f39a0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0092a0 tx=0x7efdbc005e60 comp rx=0 tx=0).stop 2024-09-19T09:13:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.714+0000 7efddc615740 1 -- 172.21.15.120:0/649301904 shutdown_connections 2024-09-19T09:13:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.714+0000 7efddc615740 1 --2- 172.21.15.120:0/649301904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626c06ff370 0x5626c06f3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.714+0000 7efddc615740 1 --2- 172.21.15.120:0/649301904 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626c06fa2e0 0x5626c070b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.714+0000 7efddc615740 1 --2- 172.21.15.120:0/649301904 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626c06f99a0 0x5626c06f39a0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.714+0000 7efddc615740 1 -- 172.21.15.120:0/649301904 >> 172.21.15.120:0/649301904 conn(0x5626c06f5010 msgr2=0x5626c0703b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.714+0000 7efddc615740 1 -- 172.21.15.120:0/649301904 shutdown_connections 2024-09-19T09:13:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.715+0000 7efddc615740 1 -- 172.21.15.120:0/649301904 wait complete. 2024-09-19T09:13:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.715+0000 7efddc615740 1 -- start start 2024-09-19T09:13:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.715+0000 7efddc615740 1 Processor -- start 2024-09-19T09:13:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.715+0000 7efddc615740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626c06f99a0 0x5626c0708610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.715+0000 7efddc615740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626c06fa2e0 0x5626c0708b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.715+0000 7efddc615740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626c06ff370 0x5626c0709af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.715+0000 7efddc615740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efdd41aa440 con 0x5626c06f99a0 2024-09-19T09:13:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.715+0000 7efddc615740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efdd41a9d70 con 0x5626c06ff370 2024-09-19T09:13:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.715+0000 7efddc615740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efdd41a6f90 con 0x5626c06fa2e0 2024-09-19T09:13:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.715+0000 7efddc615740 10 client.0 _ll_register_callbacks cb 0x7efdcc00b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.715+0000 7efdd27fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626c06f99a0 0x5626c0708610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.715+0000 7efdd27fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626c06f99a0 0x5626c0708610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56372/0 (socket says 172.21.15.120:56372) 2024-09-19T09:13:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.715+0000 7efdd27fc640 1 -- 172.21.15.120:0/1183047338 learned_addr learned my addr 172.21.15.120:0/1183047338 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.715+0000 7efdd2ffd640 1 --2- 172.21.15.120:0/1183047338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626c06fa2e0 0x5626c0708b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.715+0000 7efdd27fc640 1 --2- 172.21.15.120:0/1183047338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626c06ff370 0x5626c0709af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.715+0000 7efdd27fc640 1 -- 172.21.15.120:0/1183047338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626c06fa2e0 msgr2=0x5626c0708b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.715+0000 7efdd27fc640 1 --2- 172.21.15.120:0/1183047338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626c06fa2e0 0x5626c0708b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.715+0000 7efdd27fc640 1 -- 172.21.15.120:0/1183047338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626c06ff370 msgr2=0x5626c0709af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.715+0000 7efdd27fc640 1 --2- 172.21.15.120:0/1183047338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626c06ff370 0x5626c0709af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.715+0000 7efdd27fc640 1 -- 172.21.15.120:0/1183047338 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626c070a100 con 0x5626c06f99a0 2024-09-19T09:13:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.716+0000 7efdd2ffd640 1 --2- 172.21.15.120:0/1183047338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626c06fa2e0 0x5626c0708b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:13:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.716+0000 7efdd27fc640 1 --2- 172.21.15.120:0/1183047338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626c06f99a0 0x5626c0708610 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0096c0 tx=0x7efdbc03efa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29db15c3b578033e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.716+0000 7efdba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.716+0000 7efdba7fc640 1 -- 172.21.15.120:0/1183047338 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc035030 con 0x5626c06f99a0 2024-09-19T09:13:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.716+0000 7efdba7fc640 1 -- 172.21.15.120:0/1183047338 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7efdbc005620 con 0x5626c06f99a0 2024-09-19T09:13:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.716+0000 7efddc615740 1 -- 172.21.15.120:0/1183047338 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdbc044030 con 0x5626c06f99a0 2024-09-19T09:13:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.716+0000 7efdba7fc640 1 -- 172.21.15.120:0/1183047338 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efdbc05e070 con 0x5626c06f99a0 2024-09-19T09:13:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.716+0000 7efddc615740 10 client.19239 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.716+0000 7efddc615740 1 -- 172.21.15.120:0/1183047338 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5626c070b410 con 0x5626c06f99a0 2024-09-19T09:13:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.716+0000 7efdba7fc640 1 -- 172.21.15.120:0/1183047338 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdbc044030 con 0x5626c06f99a0 2024-09-19T09:13:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.716+0000 7efdba7fc640 1 -- 172.21.15.120:0/1183047338 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7efdbc06d060 con 0x5626c06f99a0 2024-09-19T09:13:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.716+0000 7efddc615740 10 client.19239 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.716+0000 7efddc615740 10 client.19239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:13:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.716+0000 7efddc615740 20 client.19239 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.716+0000 7efddc615740 1 --2- 172.21.15.120:0/1183047338 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5626c077a6e0 0x5626c077cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.716+0000 7efddc615740 4 client.19239 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:13:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.716+0000 7efddc615740 1 -- 172.21.15.120:0/1183047338 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5626c0664d70 con 0x5626c077a6e0 2024-09-19T09:13:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.717+0000 7efdd2ffd640 1 --2- 172.21.15.120:0/1183047338 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5626c077a6e0 0x5626c077cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.717+0000 7efdba7fc640 10 client.19239 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.717+0000 7efdd2ffd640 1 --2- 172.21.15.120:0/1183047338 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5626c077a6e0 0x5626c077cb00 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2c9a84ddf6e5bb9 server_cookie=cd1b9912e9525b5f in_seq=0 out_seq=0 2024-09-19T09:13:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.783+0000 7efdba7fc640 1 -- 172.21.15.120:0/1183047338 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5626c0664d70 con 0x5626c077a6e0 2024-09-19T09:13:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.783+0000 7efdba7fc640 10 client.19239 handle_command_reply: tid=0 2024-09-19T09:13:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.783+0000 7efddc615740 1 client.19239 shutdown 2024-09-19T09:13:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.783+0000 7efddc615740 1 -- 172.21.15.120:0/1183047338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626c06f99a0 msgr2=0x5626c0708610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.783+0000 7efddc615740 1 --2- 172.21.15.120:0/1183047338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626c06f99a0 0x5626c0708610 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0096c0 tx=0x7efdbc03efa0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.783+0000 7efddc615740 1 -- 172.21.15.120:0/1183047338 shutdown_connections 2024-09-19T09:13:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.783+0000 7efddc615740 1 --2- 172.21.15.120:0/1183047338 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5626c077a6e0 0x5626c077cb00 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.783+0000 7efddc615740 1 --2- 172.21.15.120:0/1183047338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626c06ff370 0x5626c0709af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.783+0000 7efddc615740 1 --2- 172.21.15.120:0/1183047338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626c06fa2e0 0x5626c0708b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.783+0000 7efdba7fc640 0 client.19239 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.783+0000 7efddc615740 1 --2- 172.21.15.120:0/1183047338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626c06f99a0 0x5626c0708610 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.784+0000 7efddc615740 1 -- 172.21.15.120:0/1183047338 >> 172.21.15.120:0/1183047338 conn(0x5626c06f5010 msgr2=0x5626c06f5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.784+0000 7efddc615740 1 -- 172.21.15.120:0/1183047338 shutdown_connections 2024-09-19T09:13:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.784+0000 7efddc615740 1 -- 172.21.15.120:0/1183047338 wait complete. 2024-09-19T09:13:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.784+0000 7efddc615740 20 client.19239 trim_cache size 0 max 16384 2024-09-19T09:13:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:13:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:13:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:13:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:13:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.784+0000 7efdd979f640 1 -- 172.21.15.120:0/143644259 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efdc00a32f0 msgr2=0x7efdc00a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.784+0000 7efdd979f640 1 --2- 172.21.15.120:0/143644259 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efdc00a32f0 0x7efdc00a57b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7efdc4098e60 tx=0x7efdc4066ed0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.784+0000 7efdd979f640 1 -- 172.21.15.120:0/143644259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdd41078c0 msgr2=0x7efdd419f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.784+0000 7efdd979f640 1 --2- 172.21.15.120:0/143644259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdd41078c0 0x7efdd419f3f0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7efdc400e9d0 tx=0x7efdc400ee90 comp rx=0 tx=0).stop 2024-09-19T09:13:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.784+0000 7efdd979f640 1 -- 172.21.15.120:0/143644259 shutdown_connections 2024-09-19T09:13:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.784+0000 7efdd979f640 1 --2- 172.21.15.120:0/143644259 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efdc00a32f0 0x7efdc00a57b0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.784+0000 7efdd979f640 1 --2- 172.21.15.120:0/143644259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdd41078c0 0x7efdd419f3f0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.784+0000 7efdd979f640 1 --2- 172.21.15.120:0/143644259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efdd4104f90 0x7efdd4198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.784+0000 7efdd979f640 1 --2- 172.21.15.120:0/143644259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdd4102660 0x7efdd4197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.784+0000 7efdd979f640 1 -- 172.21.15.120:0/143644259 >> 172.21.15.120:0/143644259 conn(0x7efdd40fc1e0 msgr2=0x7efdd4102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.784+0000 7efdd979f640 1 -- 172.21.15.120:0/143644259 shutdown_connections 2024-09-19T09:13:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:24.784+0000 7efdd979f640 1 -- 172.21.15.120:0/143644259 wait complete. 2024-09-19T09:13:24.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.851+0000 7fe08d7a1640 1 -- 172.21.15.120:0/3889384659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe08810bfc0 msgr2=0x7fe08810e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:24.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.851+0000 7fe08d7a1640 1 --2- 172.21.15.120:0/3889384659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe08810bfc0 0x7fe08810e460 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fe080009ca0 tx=0x7fe08002a950 comp rx=0 tx=0).stop 2024-09-19T09:13:24.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.852+0000 7fe08d7a1640 1 -- 172.21.15.120:0/3889384659 shutdown_connections 2024-09-19T09:13:24.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.852+0000 7fe08d7a1640 1 --2- 172.21.15.120:0/3889384659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe08810bfc0 0x7fe08810e460 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.852+0000 7fe08d7a1640 1 --2- 172.21.15.120:0/3889384659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0881034b0 0x7fe08810b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.852+0000 7fe08d7a1640 1 --2- 172.21.15.120:0/3889384659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe088102b90 0x7fe088102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.852+0000 7fe08d7a1640 1 -- 172.21.15.120:0/3889384659 >> 172.21.15.120:0/3889384659 conn(0x7fe0880fc630 msgr2=0x7fe0880fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:24.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.852+0000 7fe08d7a1640 1 -- 172.21.15.120:0/3889384659 shutdown_connections 2024-09-19T09:13:24.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.852+0000 7fe08d7a1640 1 -- 172.21.15.120:0/3889384659 wait complete. 2024-09-19T09:13:24.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.852+0000 7fe08d7a1640 1 Processor -- start 2024-09-19T09:13:24.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.852+0000 7fe08d7a1640 1 -- start start 2024-09-19T09:13:24.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.853+0000 7fe08d7a1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe088102b90 0x7fe088198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.853+0000 7fe08d7a1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0881034b0 0x7fe088198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.853+0000 7fe08d7a1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe08810bfc0 0x7fe08819cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.853+0000 7fe08d7a1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe088111db0 con 0x7fe0881034b0 2024-09-19T09:13:24.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.853+0000 7fe08d7a1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe088111c30 con 0x7fe08810bfc0 2024-09-19T09:13:24.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.853+0000 7fe08d7a1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe088111f30 con 0x7fe088102b90 2024-09-19T09:13:24.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.853+0000 7fe0877fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0881034b0 0x7fe088198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.853+0000 7fe0877fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0881034b0 0x7fe088198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56386/0 (socket says 172.21.15.120:56386) 2024-09-19T09:13:24.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.853+0000 7fe0877fe640 1 -- 172.21.15.120:0/771530456 learned_addr learned my addr 172.21.15.120:0/771530456 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:24.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.853+0000 7fe08cfa0640 1 --2- 172.21.15.120:0/771530456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe08810bfc0 0x7fe08819cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.853+0000 7fe087fff640 1 --2- 172.21.15.120:0/771530456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe088102b90 0x7fe088198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.853+0000 7fe0877fe640 1 -- 172.21.15.120:0/771530456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe088102b90 msgr2=0x7fe088198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:24.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.853+0000 7fe0877fe640 1 --2- 172.21.15.120:0/771530456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe088102b90 0x7fe088198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.853+0000 7fe0877fe640 1 -- 172.21.15.120:0/771530456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe08810bfc0 msgr2=0x7fe08819cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:24.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.853+0000 7fe0877fe640 1 --2- 172.21.15.120:0/771530456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe08810bfc0 0x7fe08819cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.853+0000 7fe0877fe640 1 -- 172.21.15.120:0/771530456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe08819d1c0 con 0x7fe0881034b0 2024-09-19T09:13:24.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.853+0000 7fe0877fe640 1 --2- 172.21.15.120:0/771530456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0881034b0 0x7fe088198750 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00e9f0 tx=0x7fe07c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:24.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.854+0000 7fe0857fa640 1 -- 172.21.15.120:0/771530456 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c00ccc0 con 0x7fe0881034b0 2024-09-19T09:13:24.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.854+0000 7fe0857fa640 1 -- 172.21.15.120:0/771530456 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe07c00ce60 con 0x7fe0881034b0 2024-09-19T09:13:24.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.854+0000 7fe08d7a1640 1 -- 172.21.15.120:0/771530456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe08819d4b0 con 0x7fe0881034b0 2024-09-19T09:13:24.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.854+0000 7fe0857fa640 1 -- 172.21.15.120:0/771530456 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c010690 con 0x7fe0881034b0 2024-09-19T09:13:24.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.854+0000 7fe08d7a1640 1 -- 172.21.15.120:0/771530456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0881a4d90 con 0x7fe0881034b0 2024-09-19T09:13:24.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.857+0000 7fe090ddc740 1 Processor -- start 2024-09-19T09:13:24.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.857+0000 7fe090ddc740 1 -- start start 2024-09-19T09:13:24.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.857+0000 7fe0857fa640 1 -- 172.21.15.120:0/771530456 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe07c0046c0 con 0x7fe0881034b0 2024-09-19T09:13:24.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.857+0000 7fe090ddc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc1a0c9d20 0x55fc1a0c8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.857+0000 7fe090ddc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc1a0c87b0 0x55fc1a0c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.857+0000 7fe090ddc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc1a0d3550 0x55fc1a0d5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.857+0000 7fe090ddc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fc1a02e2f0 con 0x55fc1a0c87b0 2024-09-19T09:13:24.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.857+0000 7fe090ddc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fc19fd5cf0 con 0x55fc1a0d3550 2024-09-19T09:13:24.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.857+0000 7fe090ddc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fc1a02a2b0 con 0x55fc1a0c9d20 2024-09-19T09:13:24.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.857+0000 7fe087fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc1a0c87b0 0x55fc1a0c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.857+0000 7fe087fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc1a0c87b0 0x55fc1a0c8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56394/0 (socket says 172.21.15.120:56394) 2024-09-19T09:13:24.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.857+0000 7fe087fff640 1 -- 172.21.15.120:0/1705829472 learned_addr learned my addr 172.21.15.120:0/1705829472 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:24.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.857+0000 7fe08cfa0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc1a0c9d20 0x55fc1a0c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.857+0000 7fe0877fe640 1 --2- 172.21.15.120:0/1705829472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc1a0d3550 0x55fc1a0d5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.857+0000 7fe087fff640 1 -- 172.21.15.120:0/1705829472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc1a0c9d20 msgr2=0x55fc1a0c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:24.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.857+0000 7fe087fff640 1 --2- 172.21.15.120:0/1705829472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc1a0c9d20 0x55fc1a0c8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.857+0000 7fe087fff640 1 -- 172.21.15.120:0/1705829472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc1a0d3550 msgr2=0x55fc1a0d5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:24.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.857+0000 7fe087fff640 1 --2- 172.21.15.120:0/1705829472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc1a0d3550 0x55fc1a0d5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.857+0000 7fe087fff640 1 -- 172.21.15.120:0/1705829472 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc19fba3a0 con 0x55fc1a0c87b0 2024-09-19T09:13:24.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.858+0000 7fe087fff640 1 --2- 172.21.15.120:0/1705829472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc1a0c87b0 0x55fc1a0c8bf0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fe078009c80 tx=0x7fe0780117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9c0a46fe62bd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:24.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.858+0000 7fe0857fa640 1 --2- 172.21.15.120:0/771530456 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe0700a3240 0x7fe0700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.858+0000 7fe06e7fc640 1 -- 172.21.15.120:0/1705829472 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe078012450 con 0x55fc1a0c87b0 2024-09-19T09:13:24.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.858+0000 7fe08cfa0640 1 --2- 172.21.15.120:0/771530456 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe0700a3240 0x7fe0700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.858+0000 7fe0857fa640 1 -- 172.21.15.120:0/771530456 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fe07c0cfcd0 con 0x7fe0881034b0 2024-09-19T09:13:24.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.858+0000 7fe06e7fc640 1 -- 172.21.15.120:0/1705829472 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe078012ad0 con 0x55fc1a0c87b0 2024-09-19T09:13:24.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.858+0000 7fe06e7fc640 1 -- 172.21.15.120:0/1705829472 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07800fc50 con 0x55fc1a0c87b0 2024-09-19T09:13:24.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.859+0000 7fe090ddc740 1 -- 172.21.15.120:0/1705829472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc1a0c87b0 msgr2=0x55fc1a0c8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:24.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.859+0000 7fe090ddc740 1 --2- 172.21.15.120:0/1705829472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc1a0c87b0 0x55fc1a0c8bf0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fe078009c80 tx=0x7fe0780117b0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.859+0000 7fe08cfa0640 1 --2- 172.21.15.120:0/771530456 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe0700a3240 0x7fe0700a5700 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe080005e30 tx=0x7fe080005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:24.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.859+0000 7fe090ddc740 1 -- 172.21.15.120:0/1705829472 shutdown_connections 2024-09-19T09:13:24.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.859+0000 7fe090ddc740 1 --2- 172.21.15.120:0/1705829472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc1a0d3550 0x55fc1a0d5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.859+0000 7fe090ddc740 1 --2- 172.21.15.120:0/1705829472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc1a0c87b0 0x55fc1a0c8bf0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.859+0000 7fe090ddc740 1 --2- 172.21.15.120:0/1705829472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc1a0c9d20 0x55fc1a0c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.859+0000 7fe090ddc740 1 -- 172.21.15.120:0/1705829472 >> 172.21.15.120:0/1705829472 conn(0x55fc1a091310 msgr2=0x55fc1a0c4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:24.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.859+0000 7fe090ddc740 1 -- 172.21.15.120:0/1705829472 shutdown_connections 2024-09-19T09:13:24.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.859+0000 7fe090ddc740 1 -- 172.21.15.120:0/1705829472 wait complete. 2024-09-19T09:13:24.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.859+0000 7fe090ddc740 1 -- start start 2024-09-19T09:13:24.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.859+0000 7fe090ddc740 1 Processor -- start 2024-09-19T09:13:24.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.859+0000 7fe090ddc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc1a0c87b0 0x55fc1a0d2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.859+0000 7fe090ddc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc1a0c9d20 0x55fc1a0d0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.859+0000 7fe090ddc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc1a0d3550 0x55fc1a0d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.859+0000 7fe090ddc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fc1a0d6250 con 0x55fc1a0c9d20 2024-09-19T09:13:24.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.859+0000 7fe090ddc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fc19fe9060 con 0x55fc1a0c87b0 2024-09-19T09:13:24.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.859+0000 7fe090ddc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fc1a0d63d0 con 0x55fc1a0d3550 2024-09-19T09:13:24.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.859+0000 7fe090ddc740 10 client.0 _ll_register_callbacks cb 0x55fc19d3c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:24.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.860+0000 7fe0877fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc1a0c9d20 0x55fc1a0d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.860+0000 7fe0877fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc1a0c9d20 0x55fc1a0d0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56400/0 (socket says 172.21.15.120:56400) 2024-09-19T09:13:24.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.860+0000 7fe08cfa0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc1a0d3550 0x55fc1a0d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.860+0000 7fe0877fe640 1 -- 172.21.15.120:0/1091538439 learned_addr learned my addr 172.21.15.120:0/1091538439 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:24.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.860+0000 7fe087fff640 1 --2- 172.21.15.120:0/1091538439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc1a0c87b0 0x55fc1a0d2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.860+0000 7fe0877fe640 1 -- 172.21.15.120:0/1091538439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc1a0d3550 msgr2=0x55fc1a0d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:24.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.860+0000 7fe0877fe640 1 --2- 172.21.15.120:0/1091538439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc1a0d3550 0x55fc1a0d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.860+0000 7fe0877fe640 1 -- 172.21.15.120:0/1091538439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc1a0c87b0 msgr2=0x55fc1a0d2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:24.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.860+0000 7fe0877fe640 1 --2- 172.21.15.120:0/1091538439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc1a0c87b0 0x55fc1a0d2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.860+0000 7fe0877fe640 1 -- 172.21.15.120:0/1091538439 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc19d9f3c0 con 0x55fc1a0c9d20 2024-09-19T09:13:24.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.860+0000 7fe0877fe640 1 --2- 172.21.15.120:0/1091538439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc1a0c9d20 0x55fc1a0d0120 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00e9c0 tx=0x7fe07c09bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5509c1de933e5f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:24.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.860+0000 7fe0677fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:24.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.860+0000 7fe0677fe640 1 -- 172.21.15.120:0/1091538439 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c096e60 con 0x55fc1a0c9d20 2024-09-19T09:13:24.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.860+0000 7fe0677fe640 1 -- 172.21.15.120:0/1091538439 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fe07c0a4cd0 con 0x55fc1a0c9d20 2024-09-19T09:13:24.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.860+0000 7fe090ddc740 1 -- 172.21.15.120:0/1091538439 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe078012ad0 con 0x55fc1a0c9d20 2024-09-19T09:13:24.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.860+0000 7fe0677fe640 1 -- 172.21.15.120:0/1091538439 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe07c09c2f0 con 0x55fc1a0c9d20 2024-09-19T09:13:24.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.860+0000 7fe090ddc740 10 client.19251 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:24.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.860+0000 7fe090ddc740 1 -- 172.21.15.120:0/1091538439 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fc1a0e1cc0 con 0x55fc1a0c9d20 2024-09-19T09:13:24.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.861+0000 7fe0677fe640 1 -- 172.21.15.120:0/1091538439 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe078012ad0 con 0x55fc1a0c9d20 2024-09-19T09:13:24.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.861+0000 7fe0677fe640 1 -- 172.21.15.120:0/1091538439 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe07c0d5030 con 0x55fc1a0c9d20 2024-09-19T09:13:24.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.861+0000 7fe090ddc740 10 client.19251 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:24.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.861+0000 7fe090ddc740 10 client.19251 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:13:24.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.861+0000 7fe090ddc740 20 client.19251 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:24.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.861+0000 7fe090ddc740 1 --2- 172.21.15.120:0/1091538439 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55fc1a0e3180 0x55fc1a0e5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.861+0000 7fe090ddc740 4 client.19251 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:13:24.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.861+0000 7fe090ddc740 1 -- 172.21.15.120:0/1091538439 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc1a03b860 con 0x55fc1a0e3180 2024-09-19T09:13:24.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.861+0000 7fe087fff640 1 --2- 172.21.15.120:0/1091538439 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55fc1a0e3180 0x55fc1a0e5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.861+0000 7fe0677fe640 10 client.19251 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:13:24.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.861+0000 7fe087fff640 1 --2- 172.21.15.120:0/1091538439 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55fc1a0e3180 0x55fc1a0e5570 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=de55d455b07ad769 server_cookie=590193d5091c1c77 in_seq=0 out_seq=0 2024-09-19T09:13:24.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.862+0000 7fe0677fe640 1 -- 172.21.15.120:0/1091538439 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fc1a03b860 con 0x55fc1a0e3180 2024-09-19T09:13:24.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.862+0000 7fe0677fe640 10 client.19251 handle_command_reply: tid=0 2024-09-19T09:13:24.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.862+0000 7fe090ddc740 1 client.19251 shutdown 2024-09-19T09:13:24.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.862+0000 7fe090ddc740 1 -- 172.21.15.120:0/1091538439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc1a0c9d20 msgr2=0x55fc1a0d0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:24.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.862+0000 7fe090ddc740 1 --2- 172.21.15.120:0/1091538439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc1a0c9d20 0x55fc1a0d0120 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00e9c0 tx=0x7fe07c09bef0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.862+0000 7fe090ddc740 1 -- 172.21.15.120:0/1091538439 shutdown_connections 2024-09-19T09:13:24.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.862+0000 7fe090ddc740 1 --2- 172.21.15.120:0/1091538439 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55fc1a0e3180 0x55fc1a0e5570 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.862+0000 7fe090ddc740 1 --2- 172.21.15.120:0/1091538439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc1a0d3550 0x55fc1a0d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.862+0000 7fe090ddc740 1 --2- 172.21.15.120:0/1091538439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc1a0c9d20 0x55fc1a0d0120 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.862+0000 7fe090ddc740 1 --2- 172.21.15.120:0/1091538439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc1a0c87b0 0x55fc1a0d2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.862+0000 7fe090ddc740 1 -- 172.21.15.120:0/1091538439 >> 172.21.15.120:0/1091538439 conn(0x55fc1a091310 msgr2=0x55fc1a0cab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:24.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.862+0000 7fe0677fe640 0 client.19251 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:13:24.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.862+0000 7fe090ddc740 1 -- 172.21.15.120:0/1091538439 shutdown_connections 2024-09-19T09:13:24.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.862+0000 7fe090ddc740 1 -- 172.21.15.120:0/1091538439 wait complete. 2024-09-19T09:13:24.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.863+0000 7fe090ddc740 20 client.19251 trim_cache size 0 max 16384 2024-09-19T09:13:24.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.875+0000 7fe090ddc740 1 Processor -- start 2024-09-19T09:13:24.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.875+0000 7fe090ddc740 1 -- start start 2024-09-19T09:13:24.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.876+0000 7fe090ddc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc1a0c87b0 0x55fc1a0df4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.876+0000 7fe090ddc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc1a0c9d20 0x55fc1a0e5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.876+0000 7fe090ddc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc1a0d3550 0x55fc1a0d4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.876+0000 7fe090ddc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe0881ad3d0 con 0x55fc1a0d3550 2024-09-19T09:13:24.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.876+0000 7fe090ddc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe0881aef10 con 0x55fc1a0c87b0 2024-09-19T09:13:24.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.876+0000 7fe090ddc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe0881adaa0 con 0x55fc1a0c9d20 2024-09-19T09:13:24.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.876+0000 7fe08cfa0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc1a0d3550 0x55fc1a0d4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.876+0000 7fe08cfa0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc1a0d3550 0x55fc1a0d4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56402/0 (socket says 172.21.15.120:56402) 2024-09-19T09:13:24.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.876+0000 7fe0877fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc1a0c9d20 0x55fc1a0e5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.876+0000 7fe08cfa0640 1 -- 172.21.15.120:0/1335826441 learned_addr learned my addr 172.21.15.120:0/1335826441 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:24.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.876+0000 7fe087fff640 1 --2- 172.21.15.120:0/1335826441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc1a0c87b0 0x55fc1a0df4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.876+0000 7fe08cfa0640 1 -- 172.21.15.120:0/1335826441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc1a0c9d20 msgr2=0x55fc1a0e5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:24.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.876+0000 7fe08cfa0640 1 --2- 172.21.15.120:0/1335826441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc1a0c9d20 0x55fc1a0e5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.876+0000 7fe08cfa0640 1 -- 172.21.15.120:0/1335826441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc1a0c87b0 msgr2=0x55fc1a0df4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:24.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.876+0000 7fe08cfa0640 1 --2- 172.21.15.120:0/1335826441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc1a0c87b0 0x55fc1a0df4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.876+0000 7fe08cfa0640 1 -- 172.21.15.120:0/1335826441 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc1a0cbcf0 con 0x55fc1a0d3550 2024-09-19T09:13:24.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.876+0000 7fe0877fe640 1 --2- 172.21.15.120:0/1335826441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc1a0c9d20 0x55fc1a0e5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:13:24.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.876+0000 7fe08cfa0640 1 --2- 172.21.15.120:0/1335826441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc1a0d3550 0x55fc1a0d4940 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fe08002fc60 tx=0x7fe080045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e8218f8049ed47 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:24.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.876+0000 7fe06dffb640 1 -- 172.21.15.120:0/1335826441 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe080034040 con 0x55fc1a0d3550 2024-09-19T09:13:24.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.876+0000 7fe06dffb640 1 -- 172.21.15.120:0/1335826441 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe080003c30 con 0x55fc1a0d3550 2024-09-19T09:13:24.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.876+0000 7fe06dffb640 1 -- 172.21.15.120:0/1335826441 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe080030980 con 0x55fc1a0d3550 2024-09-19T09:13:24.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.877+0000 7fe090ddc740 1 -- 172.21.15.120:0/1335826441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc1a0d3550 msgr2=0x55fc1a0d4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:24.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.877+0000 7fe090ddc740 1 --2- 172.21.15.120:0/1335826441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc1a0d3550 0x55fc1a0d4940 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fe08002fc60 tx=0x7fe080045000 comp rx=0 tx=0).stop 2024-09-19T09:13:24.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.877+0000 7fe090ddc740 1 -- 172.21.15.120:0/1335826441 shutdown_connections 2024-09-19T09:13:24.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.877+0000 7fe090ddc740 1 --2- 172.21.15.120:0/1335826441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc1a0d3550 0x55fc1a0d4940 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.877+0000 7fe090ddc740 1 --2- 172.21.15.120:0/1335826441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc1a0c9d20 0x55fc1a0e5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.877+0000 7fe090ddc740 1 --2- 172.21.15.120:0/1335826441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc1a0c87b0 0x55fc1a0df4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.877+0000 7fe090ddc740 1 -- 172.21.15.120:0/1335826441 >> 172.21.15.120:0/1335826441 conn(0x55fc1a091310 msgr2=0x55fc1a0d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:24.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.877+0000 7fe090ddc740 1 -- 172.21.15.120:0/1335826441 shutdown_connections 2024-09-19T09:13:24.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.877+0000 7fe090ddc740 1 -- 172.21.15.120:0/1335826441 wait complete. 2024-09-19T09:13:24.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.877+0000 7fe090ddc740 1 -- start start 2024-09-19T09:13:24.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.878+0000 7fe090ddc740 1 Processor -- start 2024-09-19T09:13:24.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.878+0000 7fe090ddc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc1a0c87b0 0x55fc1a0dbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.878+0000 7fe090ddc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc1a0c9d20 0x55fc1a0ea4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.878+0000 7fe090ddc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc1a0d3550 0x55fc1a0eaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.878+0000 7fe090ddc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe0881ae840 con 0x55fc1a0c87b0 2024-09-19T09:13:24.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.878+0000 7fe090ddc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe0881ae170 con 0x55fc1a0d3550 2024-09-19T09:13:24.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.878+0000 7fe090ddc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe0881ab390 con 0x55fc1a0c9d20 2024-09-19T09:13:24.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.878+0000 7fe090ddc740 10 client.0 _ll_register_callbacks cb 0x7fe07c009670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:24.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.878+0000 7fe087fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc1a0c87b0 0x55fc1a0dbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.878+0000 7fe087fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc1a0c87b0 0x55fc1a0dbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56418/0 (socket says 172.21.15.120:56418) 2024-09-19T09:13:24.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.878+0000 7fe087fff640 1 -- 172.21.15.120:0/1701658157 learned_addr learned my addr 172.21.15.120:0/1701658157 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:24.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.878+0000 7fe08cfa0640 1 --2- 172.21.15.120:0/1701658157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc1a0d3550 0x55fc1a0eaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.878+0000 7fe0877fe640 1 --2- 172.21.15.120:0/1701658157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc1a0c9d20 0x55fc1a0ea4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.878+0000 7fe087fff640 1 -- 172.21.15.120:0/1701658157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc1a0c9d20 msgr2=0x55fc1a0ea4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:24.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.878+0000 7fe087fff640 1 --2- 172.21.15.120:0/1701658157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc1a0c9d20 0x55fc1a0ea4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.878+0000 7fe087fff640 1 -- 172.21.15.120:0/1701658157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc1a0d3550 msgr2=0x55fc1a0eaa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:24.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.878+0000 7fe087fff640 1 --2- 172.21.15.120:0/1701658157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc1a0d3550 0x55fc1a0eaa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.878+0000 7fe087fff640 1 -- 172.21.15.120:0/1701658157 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc1a0eafa0 con 0x55fc1a0c87b0 2024-09-19T09:13:24.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.878+0000 7fe08cfa0640 1 --2- 172.21.15.120:0/1701658157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc1a0d3550 0x55fc1a0eaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:13:24.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.879+0000 7fe087fff640 1 --2- 172.21.15.120:0/1701658157 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc1a0c87b0 0x55fc1a0dbb80 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fe078009c80 tx=0x7fe07800f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b413966a145a0e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:24.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.879+0000 7fe0677fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:24.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.879+0000 7fe0677fe640 1 -- 172.21.15.120:0/1701658157 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0780050b0 con 0x55fc1a0c87b0 2024-09-19T09:13:24.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.879+0000 7fe0677fe640 1 -- 172.21.15.120:0/1701658157 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fe078005250 con 0x55fc1a0c87b0 2024-09-19T09:13:24.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.879+0000 7fe090ddc740 1 -- 172.21.15.120:0/1701658157 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe080003c30 con 0x55fc1a0c87b0 2024-09-19T09:13:24.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.879+0000 7fe0677fe640 1 -- 172.21.15.120:0/1701658157 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe078005450 con 0x55fc1a0c87b0 2024-09-19T09:13:24.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.879+0000 7fe090ddc740 10 client.19257 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:24.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.879+0000 7fe090ddc740 1 -- 172.21.15.120:0/1701658157 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fc1a0da3c0 con 0x55fc1a0c87b0 2024-09-19T09:13:24.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.879+0000 7fe0677fe640 1 -- 172.21.15.120:0/1701658157 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe080003c30 con 0x55fc1a0c87b0 2024-09-19T09:13:24.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.879+0000 7fe0677fe640 1 -- 172.21.15.120:0/1701658157 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe078037050 con 0x55fc1a0c87b0 2024-09-19T09:13:24.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.879+0000 7fe090ddc740 10 client.19257 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:24.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.879+0000 7fe090ddc740 10 client.19257 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:13:24.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.879+0000 7fe090ddc740 20 client.19257 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:24.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.879+0000 7fe090ddc740 1 --2- 172.21.15.120:0/1701658157 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55fc1a1522f0 0x55fc1a1546e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:24.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.879+0000 7fe090ddc740 4 client.19257 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:13:24.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.879+0000 7fe090ddc740 1 -- 172.21.15.120:0/1701658157 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc1a035d70 con 0x55fc1a1522f0 2024-09-19T09:13:24.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.880+0000 7fe0877fe640 1 --2- 172.21.15.120:0/1701658157 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55fc1a1522f0 0x55fc1a1546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:24.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.880+0000 7fe0677fe640 10 client.19257 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:13:24.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.880+0000 7fe0877fe640 1 --2- 172.21.15.120:0/1701658157 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55fc1a1522f0 0x55fc1a1546e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7776d7de58d3049a server_cookie=6b0e2ceab316b157 in_seq=0 out_seq=0 2024-09-19T09:13:24.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.880+0000 7fe0677fe640 1 -- 172.21.15.120:0/1701658157 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc1a035d70 con 0x55fc1a1522f0 2024-09-19T09:13:24.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.880+0000 7fe0677fe640 10 client.19257 handle_command_reply: tid=0 2024-09-19T09:13:24.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.880+0000 7fe090ddc740 1 client.19257 shutdown 2024-09-19T09:13:24.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.880+0000 7fe090ddc740 1 -- 172.21.15.120:0/1701658157 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc1a0c87b0 msgr2=0x55fc1a0dbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:24.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.880+0000 7fe090ddc740 1 --2- 172.21.15.120:0/1701658157 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc1a0c87b0 0x55fc1a0dbb80 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fe078009c80 tx=0x7fe07800f440 comp rx=0 tx=0).stop 2024-09-19T09:13:24.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.880+0000 7fe090ddc740 1 -- 172.21.15.120:0/1701658157 shutdown_connections 2024-09-19T09:13:24.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.880+0000 7fe090ddc740 1 --2- 172.21.15.120:0/1701658157 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55fc1a1522f0 0x55fc1a1546e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.880+0000 7fe090ddc740 1 --2- 172.21.15.120:0/1701658157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc1a0d3550 0x55fc1a0eaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.880+0000 7fe090ddc740 1 --2- 172.21.15.120:0/1701658157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc1a0c9d20 0x55fc1a0ea4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.880+0000 7fe090ddc740 1 --2- 172.21.15.120:0/1701658157 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc1a0c87b0 0x55fc1a0dbb80 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.880+0000 7fe090ddc740 1 -- 172.21.15.120:0/1701658157 >> 172.21.15.120:0/1701658157 conn(0x55fc1a091310 msgr2=0x55fc1a0ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:24.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.880+0000 7fe0677fe640 0 client.19257 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:13:24.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.880+0000 7fe090ddc740 1 -- 172.21.15.120:0/1701658157 shutdown_connections 2024-09-19T09:13:24.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.880+0000 7fe090ddc740 1 -- 172.21.15.120:0/1701658157 wait complete. 2024-09-19T09:13:24.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.881+0000 7fe090ddc740 20 client.19257 trim_cache size 0 max 16384 2024-09-19T09:13:24.897 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:13:24.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.881+0000 7fe08d7a1640 1 -- 172.21.15.120:0/771530456 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe0700a3240 msgr2=0x7fe0700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:24.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.881+0000 7fe08d7a1640 1 --2- 172.21.15.120:0/771530456 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe0700a3240 0x7fe0700a5700 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe080005e30 tx=0x7fe080005c50 comp rx=0 tx=0).stop 2024-09-19T09:13:24.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.881+0000 7fe08d7a1640 1 -- 172.21.15.120:0/771530456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0881034b0 msgr2=0x7fe088198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:24.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.881+0000 7fe08d7a1640 1 --2- 172.21.15.120:0/771530456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0881034b0 0x7fe088198750 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00e9f0 tx=0x7fe07c00eeb0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.881+0000 7fe08d7a1640 1 -- 172.21.15.120:0/771530456 shutdown_connections 2024-09-19T09:13:24.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.881+0000 7fe08d7a1640 1 --2- 172.21.15.120:0/771530456 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe0700a3240 0x7fe0700a5700 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.881+0000 7fe08d7a1640 1 --2- 172.21.15.120:0/771530456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe08810bfc0 0x7fe08819cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.881+0000 7fe08d7a1640 1 --2- 172.21.15.120:0/771530456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0881034b0 0x7fe088198750 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.881+0000 7fe08d7a1640 1 --2- 172.21.15.120:0/771530456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe088102b90 0x7fe088198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:24.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.881+0000 7fe08d7a1640 1 -- 172.21.15.120:0/771530456 >> 172.21.15.120:0/771530456 conn(0x7fe0880fc630 msgr2=0x7fe0880fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:24.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.881+0000 7fe08d7a1640 1 -- 172.21.15.120:0/771530456 shutdown_connections 2024-09-19T09:13:24.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:24.881+0000 7fe08d7a1640 1 -- 172.21.15.120:0/771530456 wait complete. 2024-09-19T09:13:25.070 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.45 MB/sec execute 440 sec latency 8376.630 ms 2024-09-19T09:13:25.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:24 smithi120 ceph-mon[31150]: pgmap v659: 97 pgs: 97 active+clean; 33 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 156 op/s 2024-09-19T09:13:25.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:24 smithi196 ceph-mon[35150]: pgmap v659: 97 pgs: 97 active+clean; 33 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 156 op/s 2024-09-19T09:13:25.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:24 smithi153 ceph-mon[35071]: pgmap v659: 97 pgs: 97 active+clean; 33 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 156 op/s 2024-09-19T09:13:25.695 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:13:25.979 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:13:26.070 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.43 MB/sec execute 441 sec latency 9376.704 ms 2024-09-19T09:13:26.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.460+0000 7febc4f32640 1 -- 172.21.15.120:0/4143979900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febc0102790 msgr2=0x7febc01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:26.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.460+0000 7febc4f32640 1 --2- 172.21.15.120:0/4143979900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febc0102790 0x7febc01067e0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7feba8009900 tx=0x7feba802a410 comp rx=0 tx=0).stop 2024-09-19T09:13:26.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.460+0000 7febc4f32640 1 -- 172.21.15.120:0/4143979900 shutdown_connections 2024-09-19T09:13:26.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.460+0000 7febc4f32640 1 --2- 172.21.15.120:0/4143979900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febc0106f10 0x7febc010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.460+0000 7febc4f32640 1 --2- 172.21.15.120:0/4143979900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febc0102790 0x7febc01067e0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.460+0000 7febc4f32640 1 --2- 172.21.15.120:0/4143979900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febc0101de0 0x7febc01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.460+0000 7febc4f32640 1 -- 172.21.15.120:0/4143979900 >> 172.21.15.120:0/4143979900 conn(0x7febc00fd8c0 msgr2=0x7febc00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:26.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.460+0000 7febc4f32640 1 -- 172.21.15.120:0/4143979900 shutdown_connections 2024-09-19T09:13:26.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.460+0000 7febc4f32640 1 -- 172.21.15.120:0/4143979900 wait complete. 2024-09-19T09:13:26.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.461+0000 7febc4f32640 1 Processor -- start 2024-09-19T09:13:26.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.461+0000 7febc4f32640 1 -- start start 2024-09-19T09:13:26.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.461+0000 7febc4f32640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febc0101de0 0x7febc00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:26.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.461+0000 7febc4f32640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febc0102790 0x7febc0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:26.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.461+0000 7febc4f32640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febc0106f10 0x7febc0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:26.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.461+0000 7febc4f32640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7febc0111f70 con 0x7febc0102790 2024-09-19T09:13:26.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.461+0000 7febc4f32640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7febc0111df0 con 0x7febc0101de0 2024-09-19T09:13:26.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.461+0000 7febc4f32640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7febc01120f0 con 0x7febc0106f10 2024-09-19T09:13:26.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.461+0000 7febbdd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febc0102790 0x7febc0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:26.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.461+0000 7febbdd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febc0102790 0x7febc0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56446/0 (socket says 172.21.15.120:56446) 2024-09-19T09:13:26.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.461+0000 7febbdd9b640 1 -- 172.21.15.120:0/949139900 learned_addr learned my addr 172.21.15.120:0/949139900 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:26.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.461+0000 7febbe59c640 1 --2- 172.21.15.120:0/949139900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febc0101de0 0x7febc00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:26.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.461+0000 7febbed9d640 1 --2- 172.21.15.120:0/949139900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febc0106f10 0x7febc0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:26.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.461+0000 7febbdd9b640 1 -- 172.21.15.120:0/949139900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febc0106f10 msgr2=0x7febc0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:26.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.461+0000 7febbdd9b640 1 --2- 172.21.15.120:0/949139900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febc0106f10 0x7febc0074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.461+0000 7febbdd9b640 1 -- 172.21.15.120:0/949139900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febc0101de0 msgr2=0x7febc00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:26.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.461+0000 7febbdd9b640 1 --2- 172.21.15.120:0/949139900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febc0101de0 0x7febc00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.461+0000 7febbdd9b640 1 -- 172.21.15.120:0/949139900 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc01a6930 con 0x7febc0102790 2024-09-19T09:13:26.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.462+0000 7febbdd9b640 1 --2- 172.21.15.120:0/949139900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febc0102790 0x7febc0074a30 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7feba8006010 tx=0x7feba80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:26.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.462+0000 7febb77fe640 1 -- 172.21.15.120:0/949139900 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba803f030 con 0x7febc0102790 2024-09-19T09:13:26.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.462+0000 7febb77fe640 1 -- 172.21.15.120:0/949139900 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feba8003ce0 con 0x7febc0102790 2024-09-19T09:13:26.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.462+0000 7febc4f32640 1 -- 172.21.15.120:0/949139900 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc01a6bc0 con 0x7febc0102790 2024-09-19T09:13:26.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.462+0000 7febb77fe640 1 -- 172.21.15.120:0/949139900 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba8035070 con 0x7febc0102790 2024-09-19T09:13:26.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.462+0000 7febc4f32640 1 -- 172.21.15.120:0/949139900 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc01a70a0 con 0x7febc0102790 2024-09-19T09:13:26.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.464+0000 7febb77fe640 1 -- 172.21.15.120:0/949139900 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7feba8042030 con 0x7febc0102790 2024-09-19T09:13:26.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.464+0000 7febc730b740 1 Processor -- start 2024-09-19T09:13:26.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.465+0000 7febc730b740 1 -- start start 2024-09-19T09:13:26.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.465+0000 7febc730b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db06b89b40 0x55db06b88090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:26.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.465+0000 7febc730b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db06b885d0 0x55db06b88a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:26.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.465+0000 7febc730b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db06b93370 0x55db06b95760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:26.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.465+0000 7febc730b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55db06aee2f0 con 0x55db06b93370 2024-09-19T09:13:26.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.465+0000 7febc730b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55db06a95cf0 con 0x55db06b89b40 2024-09-19T09:13:26.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.465+0000 7febc730b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55db06aea2b0 con 0x55db06b885d0 2024-09-19T09:13:26.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.465+0000 7febb77fe640 1 --2- 172.21.15.120:0/949139900 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7febac0a3240 0x7febac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:26.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.465+0000 7febbdd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db06b93370 0x55db06b95760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:26.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.465+0000 7febbdd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db06b93370 0x55db06b95760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56458/0 (socket says 172.21.15.120:56458) 2024-09-19T09:13:26.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.465+0000 7febbdd9b640 1 -- 172.21.15.120:0/4294663644 learned_addr learned my addr 172.21.15.120:0/4294663644 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:26.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.465+0000 7febbed9d640 1 --2- 172.21.15.120:0/4294663644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db06b89b40 0x55db06b88090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:26.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.465+0000 7febbed9d640 1 --2- 172.21.15.120:0/949139900 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7febac0a3240 0x7febac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:26.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.465+0000 7febbe59c640 1 --2- 172.21.15.120:0/4294663644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db06b885d0 0x55db06b88a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:26.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.465+0000 7febb77fe640 1 -- 172.21.15.120:0/949139900 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7feba80efe20 con 0x7febc0102790 2024-09-19T09:13:26.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.465+0000 7febbdd9b640 1 -- 172.21.15.120:0/4294663644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db06b885d0 msgr2=0x55db06b88a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:26.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.465+0000 7febbdd9b640 1 --2- 172.21.15.120:0/4294663644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db06b885d0 0x55db06b88a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.465+0000 7febbdd9b640 1 -- 172.21.15.120:0/4294663644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db06b89b40 msgr2=0x55db06b88090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:26.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.465+0000 7febbdd9b640 1 --2- 172.21.15.120:0/4294663644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db06b89b40 0x55db06b88090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.465+0000 7febbdd9b640 1 -- 172.21.15.120:0/4294663644 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db06a7a3a0 con 0x55db06b93370 2024-09-19T09:13:26.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.466+0000 7febbed9d640 1 --2- 172.21.15.120:0/949139900 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7febac0a3240 0x7febac0a5700 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7febb8009370 tx=0x7febb8009860 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:26.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.466+0000 7febbdd9b640 1 --2- 172.21.15.120:0/4294663644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db06b93370 0x55db06b95760 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7feba8005ab0 tx=0x7feba8003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed93af7b3fa7295 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:26.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.466+0000 7febb4ff9640 1 -- 172.21.15.120:0/4294663644 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba8049070 con 0x55db06b93370 2024-09-19T09:13:26.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.466+0000 7febb4ff9640 1 -- 172.21.15.120:0/4294663644 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feba803b580 con 0x55db06b93370 2024-09-19T09:13:26.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.466+0000 7febb4ff9640 1 -- 172.21.15.120:0/4294663644 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba80c5170 con 0x55db06b93370 2024-09-19T09:13:26.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.466+0000 7febc730b740 1 -- 172.21.15.120:0/4294663644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db06b93370 msgr2=0x55db06b95760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:26.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.466+0000 7febc730b740 1 --2- 172.21.15.120:0/4294663644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db06b93370 0x55db06b95760 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7feba8005ab0 tx=0x7feba8003f50 comp rx=0 tx=0).stop 2024-09-19T09:13:26.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.466+0000 7febc730b740 1 -- 172.21.15.120:0/4294663644 shutdown_connections 2024-09-19T09:13:26.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.466+0000 7febc730b740 1 --2- 172.21.15.120:0/4294663644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db06b93370 0x55db06b95760 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.466+0000 7febc730b740 1 --2- 172.21.15.120:0/4294663644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db06b885d0 0x55db06b88a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.466+0000 7febc730b740 1 --2- 172.21.15.120:0/4294663644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db06b89b40 0x55db06b88090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.466+0000 7febc730b740 1 -- 172.21.15.120:0/4294663644 >> 172.21.15.120:0/4294663644 conn(0x55db06b51310 msgr2=0x55db06b84c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:26.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.467+0000 7febc730b740 1 -- 172.21.15.120:0/4294663644 shutdown_connections 2024-09-19T09:13:26.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.467+0000 7febc730b740 1 -- 172.21.15.120:0/4294663644 wait complete. 2024-09-19T09:13:26.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.467+0000 7febc730b740 1 -- start start 2024-09-19T09:13:26.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.467+0000 7febc730b740 1 Processor -- start 2024-09-19T09:13:26.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.467+0000 7febc730b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db06b885d0 0x55db06b92d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:26.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.467+0000 7febc730b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db06b89b40 0x55db06b8ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:26.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.467+0000 7febc730b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db06b93370 0x55db06b90560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:26.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.467+0000 7febc730b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55db06aee2f0 con 0x55db06b89b40 2024-09-19T09:13:26.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.467+0000 7febc730b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55db06aa9060 con 0x55db06b885d0 2024-09-19T09:13:26.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.467+0000 7febc730b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55db06b95e30 con 0x55db06b93370 2024-09-19T09:13:26.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.467+0000 7febc730b740 10 client.0 _ll_register_callbacks cb 0x55db067fc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:26.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.467+0000 7febbdd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db06b89b40 0x55db06b8ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:26.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.467+0000 7febbdd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db06b89b40 0x55db06b8ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56464/0 (socket says 172.21.15.120:56464) 2024-09-19T09:13:26.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.467+0000 7febbdd9b640 1 -- 172.21.15.120:0/2431503252 learned_addr learned my addr 172.21.15.120:0/2431503252 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:26.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.467+0000 7febbe59c640 1 --2- 172.21.15.120:0/2431503252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db06b885d0 0x55db06b92d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:26.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.468+0000 7febbed9d640 1 --2- 172.21.15.120:0/2431503252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db06b93370 0x55db06b90560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:26.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.468+0000 7febbdd9b640 1 -- 172.21.15.120:0/2431503252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db06b93370 msgr2=0x55db06b90560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:26.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.468+0000 7febbdd9b640 1 --2- 172.21.15.120:0/2431503252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db06b93370 0x55db06b90560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.468+0000 7febbdd9b640 1 -- 172.21.15.120:0/2431503252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db06b885d0 msgr2=0x55db06b92d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:26.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.468+0000 7febbdd9b640 1 --2- 172.21.15.120:0/2431503252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db06b885d0 0x55db06b92d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.468+0000 7febbdd9b640 1 -- 172.21.15.120:0/2431503252 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db0685f3c0 con 0x55db06b89b40 2024-09-19T09:13:26.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.468+0000 7febbed9d640 1 --2- 172.21.15.120:0/2431503252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db06b93370 0x55db06b90560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:13:26.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.468+0000 7febbdd9b640 1 --2- 172.21.15.120:0/2431503252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db06b89b40 0x55db06b8ffc0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7feba80ba7b0 tx=0x7feba80b8160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201e216c33d09176 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:26.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.468+0000 7feba5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:26.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.468+0000 7feba5ffb640 1 -- 172.21.15.120:0/2431503252 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba8049040 con 0x55db06b89b40 2024-09-19T09:13:26.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.468+0000 7feba5ffb640 1 -- 172.21.15.120:0/2431503252 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7feba80bc760 con 0x55db06b89b40 2024-09-19T09:13:26.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.468+0000 7febc730b740 1 -- 172.21.15.120:0/2431503252 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55db06afb860 con 0x55db06b89b40 2024-09-19T09:13:26.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.468+0000 7feba5ffb640 1 -- 172.21.15.120:0/2431503252 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feba80bcc00 con 0x55db06b89b40 2024-09-19T09:13:26.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.468+0000 7febc730b740 10 client.19269 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:26.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.468+0000 7febc730b740 1 -- 172.21.15.120:0/2431503252 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55db06ba1730 con 0x55db06b89b40 2024-09-19T09:13:26.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.468+0000 7feba5ffb640 1 -- 172.21.15.120:0/2431503252 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55db06afb860 con 0x55db06b89b40 2024-09-19T09:13:26.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.468+0000 7feba5ffb640 1 -- 172.21.15.120:0/2431503252 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7feba80f2070 con 0x55db06b89b40 2024-09-19T09:13:26.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.468+0000 7febc730b740 10 client.19269 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:26.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.468+0000 7febc730b740 10 client.19269 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:13:26.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.468+0000 7febc730b740 20 client.19269 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:26.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.468+0000 7febc730b740 1 --2- 172.21.15.120:0/2431503252 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55db06ba2a80 0x55db06ba4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:26.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.468+0000 7febc730b740 4 client.19269 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:13:26.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.468+0000 7febc730b740 1 -- 172.21.15.120:0/2431503252 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db06af5d70 con 0x55db06ba2a80 2024-09-19T09:13:26.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.469+0000 7febbe59c640 1 --2- 172.21.15.120:0/2431503252 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55db06ba2a80 0x55db06ba4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:26.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.469+0000 7feba5ffb640 10 client.19269 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:13:26.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.469+0000 7febbe59c640 1 --2- 172.21.15.120:0/2431503252 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55db06ba2a80 0x55db06ba4e70 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=904b2260a851b0f2 server_cookie=d782239063d4a6d1 in_seq=0 out_seq=0 2024-09-19T09:13:26.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.470+0000 7feba5ffb640 1 -- 172.21.15.120:0/2431503252 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55db06af5d70 con 0x55db06ba2a80 2024-09-19T09:13:26.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.470+0000 7feba5ffb640 10 client.19269 handle_command_reply: tid=0 2024-09-19T09:13:26.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.470+0000 7febc730b740 1 client.19269 shutdown 2024-09-19T09:13:26.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.470+0000 7febc730b740 1 -- 172.21.15.120:0/2431503252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db06b89b40 msgr2=0x55db06b8ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:26.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.470+0000 7febc730b740 1 --2- 172.21.15.120:0/2431503252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db06b89b40 0x55db06b8ffc0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7feba80ba7b0 tx=0x7feba80b8160 comp rx=0 tx=0).stop 2024-09-19T09:13:26.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.470+0000 7febc730b740 1 -- 172.21.15.120:0/2431503252 shutdown_connections 2024-09-19T09:13:26.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.470+0000 7febc730b740 1 --2- 172.21.15.120:0/2431503252 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55db06ba2a80 0x55db06ba4e70 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.470+0000 7febc730b740 1 --2- 172.21.15.120:0/2431503252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db06b93370 0x55db06b90560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.470+0000 7feba5ffb640 0 client.19269 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:13:26.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.470+0000 7febc730b740 1 --2- 172.21.15.120:0/2431503252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db06b89b40 0x55db06b8ffc0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.470+0000 7febc730b740 1 --2- 172.21.15.120:0/2431503252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db06b885d0 0x55db06b92d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.470+0000 7febc730b740 1 -- 172.21.15.120:0/2431503252 >> 172.21.15.120:0/2431503252 conn(0x55db06b51310 msgr2=0x55db06b8a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:26.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.470+0000 7febc730b740 1 -- 172.21.15.120:0/2431503252 shutdown_connections 2024-09-19T09:13:26.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.470+0000 7febc730b740 1 -- 172.21.15.120:0/2431503252 wait complete. 2024-09-19T09:13:26.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.470+0000 7febc730b740 20 client.19269 trim_cache size 0 max 16384 2024-09-19T09:13:26.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.483+0000 7febc730b740 1 Processor -- start 2024-09-19T09:13:26.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.483+0000 7febc730b740 1 -- start start 2024-09-19T09:13:26.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.483+0000 7febc730b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db06b885d0 0x55db06b9ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:26.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.483+0000 7febc730b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db06b89b40 0x55db06baa630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:26.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.483+0000 7febc730b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db06b93370 0x55db06baab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:26.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.483+0000 7febc730b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7febc01af840 con 0x55db06b89b40 2024-09-19T09:13:26.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.483+0000 7febc730b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7febc01b1380 con 0x55db06b885d0 2024-09-19T09:13:26.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.483+0000 7febc730b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7febc01aff10 con 0x55db06b93370 2024-09-19T09:13:26.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.483+0000 7febbdd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db06b89b40 0x55db06baa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:26.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.483+0000 7febbe59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db06b885d0 0x55db06b9ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:26.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.483+0000 7febbdd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db06b89b40 0x55db06baa630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56476/0 (socket says 172.21.15.120:56476) 2024-09-19T09:13:26.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.483+0000 7febbdd9b640 1 -- 172.21.15.120:0/3510105888 learned_addr learned my addr 172.21.15.120:0/3510105888 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:26.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.484+0000 7febbed9d640 1 --2- 172.21.15.120:0/3510105888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db06b93370 0x55db06baab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:26.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.484+0000 7febbdd9b640 1 -- 172.21.15.120:0/3510105888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db06b93370 msgr2=0x55db06baab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:26.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.484+0000 7febbdd9b640 1 --2- 172.21.15.120:0/3510105888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db06b93370 0x55db06baab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.484+0000 7febbdd9b640 1 -- 172.21.15.120:0/3510105888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db06b885d0 msgr2=0x55db06b9ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:26.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.484+0000 7febbdd9b640 1 --2- 172.21.15.120:0/3510105888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db06b885d0 0x55db06b9ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.484+0000 7febbdd9b640 1 -- 172.21.15.120:0/3510105888 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db06b85730 con 0x55db06b89b40 2024-09-19T09:13:26.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.484+0000 7febbe59c640 1 --2- 172.21.15.120:0/3510105888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db06b885d0 0x55db06b9ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:13:26.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.484+0000 7febbdd9b640 1 --2- 172.21.15.120:0/3510105888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db06b89b40 0x55db06baa630 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7feba80c4c40 tx=0x7feba802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b973ec51ed7f604 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:26.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.484+0000 7feba7fff640 1 -- 172.21.15.120:0/3510105888 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba8049040 con 0x55db06b89b40 2024-09-19T09:13:26.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.484+0000 7feba7fff640 1 -- 172.21.15.120:0/3510105888 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feba80c5170 con 0x55db06b89b40 2024-09-19T09:13:26.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.484+0000 7feba7fff640 1 -- 172.21.15.120:0/3510105888 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba80ca070 con 0x55db06b89b40 2024-09-19T09:13:26.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.485+0000 7febc730b740 1 -- 172.21.15.120:0/3510105888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db06b89b40 msgr2=0x55db06baa630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:26.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.485+0000 7febc730b740 1 --2- 172.21.15.120:0/3510105888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db06b89b40 0x55db06baa630 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7feba80c4c40 tx=0x7feba802a410 comp rx=0 tx=0).stop 2024-09-19T09:13:26.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.485+0000 7febc730b740 1 -- 172.21.15.120:0/3510105888 shutdown_connections 2024-09-19T09:13:26.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.485+0000 7febc730b740 1 --2- 172.21.15.120:0/3510105888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db06b93370 0x55db06baab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.485+0000 7febc730b740 1 --2- 172.21.15.120:0/3510105888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db06b89b40 0x55db06baa630 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.485+0000 7febc730b740 1 --2- 172.21.15.120:0/3510105888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db06b885d0 0x55db06b9ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.485+0000 7febc730b740 1 -- 172.21.15.120:0/3510105888 >> 172.21.15.120:0/3510105888 conn(0x55db06b51310 msgr2=0x55db06b8f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:26.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.485+0000 7febc730b740 1 -- 172.21.15.120:0/3510105888 shutdown_connections 2024-09-19T09:13:26.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.485+0000 7febc730b740 1 -- 172.21.15.120:0/3510105888 wait complete. 2024-09-19T09:13:26.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.485+0000 7febc730b740 1 -- start start 2024-09-19T09:13:26.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.485+0000 7febc730b740 1 Processor -- start 2024-09-19T09:13:26.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.485+0000 7febc730b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db06b885d0 0x55db06b8cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:26.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.485+0000 7febc730b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db06b89b40 0x55db06b8d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:26.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.485+0000 7febc730b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db06b93370 0x55db06b8da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:26.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.485+0000 7febc730b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7febc01b0cb0 con 0x55db06b93370 2024-09-19T09:13:26.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.485+0000 7febc730b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7febc01b05e0 con 0x55db06b885d0 2024-09-19T09:13:26.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.485+0000 7febc730b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7febc01ad800 con 0x55db06b89b40 2024-09-19T09:13:26.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.486+0000 7febc730b740 10 client.0 _ll_register_callbacks cb 0x7feba80dc070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:26.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.486+0000 7febbed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db06b93370 0x55db06b8da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:26.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.486+0000 7febbed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db06b93370 0x55db06b8da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56488/0 (socket says 172.21.15.120:56488) 2024-09-19T09:13:26.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.486+0000 7febbed9d640 1 -- 172.21.15.120:0/373892483 learned_addr learned my addr 172.21.15.120:0/373892483 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:26.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.486+0000 7febbdd9b640 1 --2- 172.21.15.120:0/373892483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db06b89b40 0x55db06b8d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:26.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.486+0000 7febbe59c640 1 --2- 172.21.15.120:0/373892483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db06b885d0 0x55db06b8cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:26.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.486+0000 7febbed9d640 1 -- 172.21.15.120:0/373892483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db06b89b40 msgr2=0x55db06b8d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:26.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.486+0000 7febbed9d640 1 --2- 172.21.15.120:0/373892483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db06b89b40 0x55db06b8d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.486+0000 7febbed9d640 1 -- 172.21.15.120:0/373892483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db06b885d0 msgr2=0x55db06b8cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:26.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.486+0000 7febbed9d640 1 --2- 172.21.15.120:0/373892483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db06b885d0 0x55db06b8cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.486+0000 7febbed9d640 1 -- 172.21.15.120:0/373892483 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db06c0c600 con 0x55db06b93370 2024-09-19T09:13:26.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.486+0000 7febbe59c640 1 --2- 172.21.15.120:0/373892483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db06b885d0 0x55db06b8cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:13:26.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.486+0000 7febbed9d640 1 --2- 172.21.15.120:0/373892483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db06b93370 0x55db06b8da60 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7febb80132d0 tx=0x7febb80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e623e250e3325af8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:26.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.486+0000 7feba5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:26.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.486+0000 7feba5ffb640 1 -- 172.21.15.120:0/373892483 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb801fd80 con 0x55db06b93370 2024-09-19T09:13:26.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.486+0000 7feba5ffb640 1 -- 172.21.15.120:0/373892483 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7febb8004b20 con 0x55db06b93370 2024-09-19T09:13:26.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.486+0000 7febc730b740 1 -- 172.21.15.120:0/373892483 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feba80c5170 con 0x55db06b93370 2024-09-19T09:13:26.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.486+0000 7feba5ffb640 1 -- 172.21.15.120:0/373892483 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7febb8010e50 con 0x55db06b93370 2024-09-19T09:13:26.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.486+0000 7febc730b740 10 client.19275 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:26.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.486+0000 7febc730b740 1 -- 172.21.15.120:0/373892483 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55db06c0d8e0 con 0x55db06b93370 2024-09-19T09:13:26.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.487+0000 7feba5ffb640 1 -- 172.21.15.120:0/373892483 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feba80c5170 con 0x55db06b93370 2024-09-19T09:13:26.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.487+0000 7feba5ffb640 1 -- 172.21.15.120:0/373892483 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7febb800f040 con 0x55db06b93370 2024-09-19T09:13:26.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.487+0000 7febc730b740 10 client.19275 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:26.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.487+0000 7febc730b740 10 client.19275 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:13:26.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.487+0000 7febc730b740 20 client.19275 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:26.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.487+0000 7febc730b740 1 --2- 172.21.15.120:0/373892483 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55db06c0e0e0 0x55db06c10500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:26.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.487+0000 7febc730b740 4 client.19275 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:13:26.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.487+0000 7febc730b740 1 -- 172.21.15.120:0/373892483 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55db06a72170 con 0x55db06c0e0e0 2024-09-19T09:13:26.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.487+0000 7febbe59c640 1 --2- 172.21.15.120:0/373892483 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55db06c0e0e0 0x55db06c10500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:26.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.488+0000 7feba5ffb640 10 client.19275 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:13:26.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.488+0000 7febbe59c640 1 --2- 172.21.15.120:0/373892483 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55db06c0e0e0 0x55db06c10500 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d3ec2fc69ebbccbf server_cookie=7fedf57a3f687061 in_seq=0 out_seq=0 2024-09-19T09:13:26.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.488+0000 7feba5ffb640 1 -- 172.21.15.120:0/373892483 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55db06a72170 con 0x55db06c0e0e0 2024-09-19T09:13:26.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.488+0000 7feba5ffb640 10 client.19275 handle_command_reply: tid=0 2024-09-19T09:13:26.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.488+0000 7febc730b740 1 client.19275 shutdown 2024-09-19T09:13:26.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.488+0000 7febc730b740 1 -- 172.21.15.120:0/373892483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db06b93370 msgr2=0x55db06b8da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:26.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.488+0000 7febc730b740 1 --2- 172.21.15.120:0/373892483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db06b93370 0x55db06b8da60 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7febb80132d0 tx=0x7febb80095c0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.488+0000 7febc730b740 1 -- 172.21.15.120:0/373892483 shutdown_connections 2024-09-19T09:13:26.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.488+0000 7febc730b740 1 --2- 172.21.15.120:0/373892483 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55db06c0e0e0 0x55db06c10500 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.488+0000 7febc730b740 1 --2- 172.21.15.120:0/373892483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db06b93370 0x55db06b8da60 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.488+0000 7febc730b740 1 --2- 172.21.15.120:0/373892483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db06b89b40 0x55db06b8d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.488+0000 7febc730b740 1 --2- 172.21.15.120:0/373892483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db06b885d0 0x55db06b8cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.488+0000 7feba5ffb640 0 client.19275 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:13:26.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.488+0000 7febc730b740 1 -- 172.21.15.120:0/373892483 >> 172.21.15.120:0/373892483 conn(0x55db06b51310 msgr2=0x55db06b91820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:26.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.488+0000 7febc730b740 1 -- 172.21.15.120:0/373892483 shutdown_connections 2024-09-19T09:13:26.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.488+0000 7febc730b740 1 -- 172.21.15.120:0/373892483 wait complete. 2024-09-19T09:13:26.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.488+0000 7febc730b740 20 client.19275 trim_cache size 0 max 16384 2024-09-19T09:13:26.501 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:13:26.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.489+0000 7febc4f32640 1 -- 172.21.15.120:0/949139900 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7febac0a3240 msgr2=0x7febac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:26.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.489+0000 7febc4f32640 1 --2- 172.21.15.120:0/949139900 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7febac0a3240 0x7febac0a5700 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7febb8009370 tx=0x7febb8009860 comp rx=0 tx=0).stop 2024-09-19T09:13:26.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.489+0000 7febc4f32640 1 -- 172.21.15.120:0/949139900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febc0102790 msgr2=0x7febc0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:26.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.489+0000 7febc4f32640 1 --2- 172.21.15.120:0/949139900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febc0102790 0x7febc0074a30 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7feba8006010 tx=0x7feba80042f0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.489+0000 7febc4f32640 1 -- 172.21.15.120:0/949139900 shutdown_connections 2024-09-19T09:13:26.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.489+0000 7febc4f32640 1 --2- 172.21.15.120:0/949139900 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7febac0a3240 0x7febac0a5700 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.489+0000 7febc4f32640 1 --2- 172.21.15.120:0/949139900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febc0106f10 0x7febc0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.489+0000 7febc4f32640 1 --2- 172.21.15.120:0/949139900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febc0102790 0x7febc0074a30 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.489+0000 7febc4f32640 1 --2- 172.21.15.120:0/949139900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febc0101de0 0x7febc00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:26.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.489+0000 7febc4f32640 1 -- 172.21.15.120:0/949139900 >> 172.21.15.120:0/949139900 conn(0x7febc00fd8c0 msgr2=0x7febc00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:26.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.489+0000 7febc4f32640 1 -- 172.21.15.120:0/949139900 shutdown_connections 2024-09-19T09:13:26.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:26.489+0000 7febc4f32640 1 -- 172.21.15.120:0/949139900 wait complete. 2024-09-19T09:13:27.070 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.41 MB/sec execute 442 sec latency 10376.779 ms 2024-09-19T09:13:27.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:26 smithi196 ceph-mon[35150]: pgmap v660: 97 pgs: 97 active+clean; 33 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-09-19T09:13:27.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:26 smithi153 ceph-mon[35071]: pgmap v660: 97 pgs: 97 active+clean; 33 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-09-19T09:13:27.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:26 smithi120 ceph-mon[31150]: pgmap v660: 97 pgs: 97 active+clean; 33 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-09-19T09:13:27.491 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:13:27.736 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:13:28.070 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.39 MB/sec execute 443 sec latency 11376.840 ms 2024-09-19T09:13:28.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.230+0000 7f55fd60d640 1 -- 172.21.15.120:0/2623567971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55f8106f10 msgr2=0x7f55f810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:28.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.230+0000 7f55fd60d640 1 --2- 172.21.15.120:0/2623567971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55f8106f10 0x7f55f810ea90 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009900 tx=0x7f55f002a410 comp rx=0 tx=0).stop 2024-09-19T09:13:28.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.230+0000 7f55fd60d640 1 -- 172.21.15.120:0/2623567971 shutdown_connections 2024-09-19T09:13:28.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.230+0000 7f55fd60d640 1 --2- 172.21.15.120:0/2623567971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55f8106f10 0x7f55f810ea90 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.230+0000 7f55fd60d640 1 --2- 172.21.15.120:0/2623567971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55f8102790 0x7f55f81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.230+0000 7f55fd60d640 1 --2- 172.21.15.120:0/2623567971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55f8101de0 0x7f55f81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.230+0000 7f55fd60d640 1 -- 172.21.15.120:0/2623567971 >> 172.21.15.120:0/2623567971 conn(0x7f55f80fd8c0 msgr2=0x7f55f80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:28.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.230+0000 7f55fd60d640 1 -- 172.21.15.120:0/2623567971 shutdown_connections 2024-09-19T09:13:28.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.230+0000 7f55fd60d640 1 -- 172.21.15.120:0/2623567971 wait complete. 2024-09-19T09:13:28.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.231+0000 7f55fd60d640 1 Processor -- start 2024-09-19T09:13:28.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.231+0000 7f55fd60d640 1 -- start start 2024-09-19T09:13:28.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.231+0000 7f55fd60d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55f8101de0 0x7f55f8198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:28.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.231+0000 7f55fd60d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55f8102790 0x7f55f8198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:28.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.231+0000 7f55fd60d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55f8106f10 0x7f55f819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:28.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.231+0000 7f55fd60d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f55f8111f90 con 0x7f55f8106f10 2024-09-19T09:13:28.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.231+0000 7f55fd60d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f55f8111e10 con 0x7f55f8102790 2024-09-19T09:13:28.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.231+0000 7f55fd60d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f55f8112110 con 0x7f55f8101de0 2024-09-19T09:13:28.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.231+0000 7f55f77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55f8106f10 0x7f55f819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:28.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.231+0000 7f55f77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55f8106f10 0x7f55f819cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56504/0 (socket says 172.21.15.120:56504) 2024-09-19T09:13:28.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.231+0000 7f55f77fe640 1 -- 172.21.15.120:0/965401458 learned_addr learned my addr 172.21.15.120:0/965401458 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:28.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.231+0000 7f55f6ffd640 1 --2- 172.21.15.120:0/965401458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55f8101de0 0x7f55f8198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:28.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.231+0000 7f55f67fc640 1 --2- 172.21.15.120:0/965401458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55f8102790 0x7f55f8198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:28.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.232+0000 7f55f77fe640 1 -- 172.21.15.120:0/965401458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55f8101de0 msgr2=0x7f55f8198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:28.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.232+0000 7f55f77fe640 1 --2- 172.21.15.120:0/965401458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55f8101de0 0x7f55f8198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.232+0000 7f55f77fe640 1 -- 172.21.15.120:0/965401458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55f8102790 msgr2=0x7f55f8198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.232+0000 7f55f77fe640 1 --2- 172.21.15.120:0/965401458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55f8102790 0x7f55f8198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.232+0000 7f55f77fe640 1 -- 172.21.15.120:0/965401458 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f819d3c0 con 0x7f55f8106f10 2024-09-19T09:13:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.232+0000 7f55f67fc640 1 --2- 172.21.15.120:0/965401458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55f8102790 0x7f55f8198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:13:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.232+0000 7f55f77fe640 1 --2- 172.21.15.120:0/965401458 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55f8106f10 0x7f55f819cce0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009230 tx=0x7f55f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.232+0000 7f55dffff640 1 -- 172.21.15.120:0/965401458 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f003f030 con 0x7f55f8106f10 2024-09-19T09:13:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.232+0000 7f55fd60d640 1 -- 172.21.15.120:0/965401458 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f819d650 con 0x7f55f8106f10 2024-09-19T09:13:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.232+0000 7f55dffff640 1 -- 172.21.15.120:0/965401458 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55f0003ce0 con 0x7f55f8106f10 2024-09-19T09:13:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.232+0000 7f55dffff640 1 -- 172.21.15.120:0/965401458 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f0035070 con 0x7f55f8106f10 2024-09-19T09:13:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.232+0000 7f55fd60d640 1 -- 172.21.15.120:0/965401458 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f81a4f90 con 0x7f55f8106f10 2024-09-19T09:13:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.234+0000 7f55dffff640 1 -- 172.21.15.120:0/965401458 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f55f0042030 con 0x7f55f8106f10 2024-09-19T09:13:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.235+0000 7f55ff9e6740 1 Processor -- start 2024-09-19T09:13:28.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.235+0000 7f55ff9e6740 1 -- start start 2024-09-19T09:13:28.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.235+0000 7f55ff9e6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ba68fdb40 0x558ba68fc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:28.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.235+0000 7f55ff9e6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ba68fc5d0 0x558ba68fca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:28.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.235+0000 7f55ff9e6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ba6907370 0x558ba6909760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:28.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.235+0000 7f55ff9e6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558ba68622f0 con 0x558ba6907370 2024-09-19T09:13:28.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.235+0000 7f55ff9e6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558ba6809cf0 con 0x558ba68fc5d0 2024-09-19T09:13:28.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.235+0000 7f55ff9e6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558ba685e2b0 con 0x558ba68fdb40 2024-09-19T09:13:28.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.235+0000 7f55dffff640 1 --2- 172.21.15.120:0/965401458 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f55e00a3240 0x7f55e00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:28.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.235+0000 7f55f67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ba6907370 0x558ba6909760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:28.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.235+0000 7f55f67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ba6907370 0x558ba6909760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56516/0 (socket says 172.21.15.120:56516) 2024-09-19T09:13:28.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.235+0000 7f55f67fc640 1 -- 172.21.15.120:0/4241311109 learned_addr learned my addr 172.21.15.120:0/4241311109 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:28.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.235+0000 7f55f77fe640 1 --2- 172.21.15.120:0/965401458 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f55e00a3240 0x7f55e00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:28.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.235+0000 7f55f67fc640 1 --2- 172.21.15.120:0/4241311109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ba68fdb40 0x558ba68fc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:28.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.235+0000 7f55dffff640 1 -- 172.21.15.120:0/965401458 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f55f00f02f0 con 0x7f55f8106f10 2024-09-19T09:13:28.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.235+0000 7f55f6ffd640 1 --2- 172.21.15.120:0/4241311109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ba68fc5d0 0x558ba68fca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:28.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.235+0000 7f55f67fc640 1 -- 172.21.15.120:0/4241311109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ba68fdb40 msgr2=0x558ba68fc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:28.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.235+0000 7f55f67fc640 1 --2- 172.21.15.120:0/4241311109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ba68fdb40 0x558ba68fc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.235+0000 7f55f67fc640 1 -- 172.21.15.120:0/4241311109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ba68fc5d0 msgr2=0x558ba68fca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:28.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.235+0000 7f55f67fc640 1 --2- 172.21.15.120:0/4241311109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ba68fc5d0 0x558ba68fca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.235+0000 7f55f67fc640 1 -- 172.21.15.120:0/4241311109 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ba67ee3a0 con 0x558ba6907370 2024-09-19T09:13:28.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.236+0000 7f55f67fc640 1 --2- 172.21.15.120:0/4241311109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ba6907370 0x558ba6909760 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f55ec012880 tx=0x7f55ec00d440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03153ceac27a71c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:28.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.236+0000 7f55f77fe640 1 --2- 172.21.15.120:0/965401458 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f55e00a3240 0x7f55e00a5700 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f55f00b8d70 tx=0x7f55f00872a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:28.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.236+0000 7f55dd7fa640 1 -- 172.21.15.120:0/4241311109 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec00aa60 con 0x558ba6907370 2024-09-19T09:13:28.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.236+0000 7f55dd7fa640 1 -- 172.21.15.120:0/4241311109 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55ec005500 con 0x558ba6907370 2024-09-19T09:13:28.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.236+0000 7f55dd7fa640 1 -- 172.21.15.120:0/4241311109 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec004bc0 con 0x558ba6907370 2024-09-19T09:13:28.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.236+0000 7f55ff9e6740 1 -- 172.21.15.120:0/4241311109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ba6907370 msgr2=0x558ba6909760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:28.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.236+0000 7f55ff9e6740 1 --2- 172.21.15.120:0/4241311109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ba6907370 0x558ba6909760 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f55ec012880 tx=0x7f55ec00d440 comp rx=0 tx=0).stop 2024-09-19T09:13:28.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.236+0000 7f55ff9e6740 1 -- 172.21.15.120:0/4241311109 shutdown_connections 2024-09-19T09:13:28.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.236+0000 7f55ff9e6740 1 --2- 172.21.15.120:0/4241311109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ba6907370 0x558ba6909760 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.236+0000 7f55ff9e6740 1 --2- 172.21.15.120:0/4241311109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ba68fc5d0 0x558ba68fca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.236+0000 7f55ff9e6740 1 --2- 172.21.15.120:0/4241311109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ba68fdb40 0x558ba68fc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.236+0000 7f55ff9e6740 1 -- 172.21.15.120:0/4241311109 >> 172.21.15.120:0/4241311109 conn(0x558ba68c5310 msgr2=0x558ba68f8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:28.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.236+0000 7f55ff9e6740 1 -- 172.21.15.120:0/4241311109 shutdown_connections 2024-09-19T09:13:28.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.236+0000 7f55ff9e6740 1 -- 172.21.15.120:0/4241311109 wait complete. 2024-09-19T09:13:28.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.237+0000 7f55ff9e6740 1 -- start start 2024-09-19T09:13:28.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.237+0000 7f55ff9e6740 1 Processor -- start 2024-09-19T09:13:28.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.237+0000 7f55ff9e6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ba68fc5d0 0x558ba6906d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:28.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.237+0000 7f55ff9e6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ba68fdb40 0x558ba6904020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:28.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.237+0000 7f55ff9e6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ba6907370 0x558ba6904610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:28.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.237+0000 7f55ff9e6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558ba690a070 con 0x558ba68fdb40 2024-09-19T09:13:28.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.237+0000 7f55ff9e6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558ba681d060 con 0x558ba6907370 2024-09-19T09:13:28.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.237+0000 7f55ff9e6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558ba690a1f0 con 0x558ba68fc5d0 2024-09-19T09:13:28.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.237+0000 7f55ff9e6740 10 client.0 _ll_register_callbacks cb 0x558ba65708c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:28.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.237+0000 7f55f6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ba68fdb40 0x558ba6904020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:28.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.237+0000 7f55f6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ba68fdb40 0x558ba6904020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56524/0 (socket says 172.21.15.120:56524) 2024-09-19T09:13:28.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.237+0000 7f55f6ffd640 1 -- 172.21.15.120:0/2052207405 learned_addr learned my addr 172.21.15.120:0/2052207405 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:28.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.237+0000 7f55f67fc640 1 --2- 172.21.15.120:0/2052207405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ba68fc5d0 0x558ba6906d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:28.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.237+0000 7f55f6ffd640 1 -- 172.21.15.120:0/2052207405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ba68fc5d0 msgr2=0x558ba6906d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:28.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.237+0000 7f55f6ffd640 1 --2- 172.21.15.120:0/2052207405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ba68fc5d0 0x558ba6906d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.237+0000 7f55f6ffd640 1 -- 172.21.15.120:0/2052207405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ba6907370 msgr2=0x558ba6904610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:28.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.237+0000 7f55f6ffd640 1 --2- 172.21.15.120:0/2052207405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ba6907370 0x558ba6904610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.237+0000 7f55f6ffd640 1 -- 172.21.15.120:0/2052207405 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ba65d33c0 con 0x558ba68fdb40 2024-09-19T09:13:28.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.238+0000 7f55f6ffd640 1 --2- 172.21.15.120:0/2052207405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ba68fdb40 0x558ba6904020 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f55e800abc0 tx=0x7f55e8007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12dac86067c7a263 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:28.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.238+0000 7f55daff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:28.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.238+0000 7f55daff5640 1 -- 172.21.15.120:0/2052207405 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e801b060 con 0x558ba68fdb40 2024-09-19T09:13:28.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.238+0000 7f55daff5640 1 -- 172.21.15.120:0/2052207405 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f55e8016640 con 0x558ba68fdb40 2024-09-19T09:13:28.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.238+0000 7f55ff9e6740 1 -- 172.21.15.120:0/2052207405 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55ec005500 con 0x558ba68fdb40 2024-09-19T09:13:28.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.238+0000 7f55daff5640 1 -- 172.21.15.120:0/2052207405 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f55e8016840 con 0x558ba68fdb40 2024-09-19T09:13:28.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.238+0000 7f55ff9e6740 10 client.19287 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:28.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.238+0000 7f55ff9e6740 1 -- 172.21.15.120:0/2052207405 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558ba6915b70 con 0x558ba68fdb40 2024-09-19T09:13:28.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.238+0000 7f55daff5640 1 -- 172.21.15.120:0/2052207405 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55ec005500 con 0x558ba68fdb40 2024-09-19T09:13:28.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.238+0000 7f55daff5640 1 -- 172.21.15.120:0/2052207405 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f55e8020070 con 0x558ba68fdb40 2024-09-19T09:13:28.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.238+0000 7f55ff9e6740 10 client.19287 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:28.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.238+0000 7f55ff9e6740 10 client.19287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:13:28.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.238+0000 7f55ff9e6740 20 client.19287 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:28.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.238+0000 7f55ff9e6740 1 --2- 172.21.15.120:0/2052207405 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558ba6917080 0x558ba69194a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:28.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.238+0000 7f55ff9e6740 4 client.19287 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:13:28.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.238+0000 7f55ff9e6740 1 -- 172.21.15.120:0/2052207405 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ba686f860 con 0x558ba6917080 2024-09-19T09:13:28.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.239+0000 7f55f67fc640 1 --2- 172.21.15.120:0/2052207405 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558ba6917080 0x558ba69194a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:28.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.239+0000 7f55f67fc640 1 --2- 172.21.15.120:0/2052207405 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558ba6917080 0x558ba69194a0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ebccebb80602311 server_cookie=f263ea91b1ee0d53 in_seq=0 out_seq=0 2024-09-19T09:13:28.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.239+0000 7f55daff5640 10 client.19287 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:28.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.240+0000 7f55daff5640 1 -- 172.21.15.120:0/2052207405 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558ba686f860 con 0x558ba6917080 2024-09-19T09:13:28.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.240+0000 7f55daff5640 10 client.19287 handle_command_reply: tid=0 2024-09-19T09:13:28.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.240+0000 7f55ff9e6740 1 client.19287 shutdown 2024-09-19T09:13:28.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.240+0000 7f55ff9e6740 1 -- 172.21.15.120:0/2052207405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ba68fdb40 msgr2=0x558ba6904020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:28.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.240+0000 7f55ff9e6740 1 --2- 172.21.15.120:0/2052207405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ba68fdb40 0x558ba6904020 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f55e800abc0 tx=0x7f55e8007680 comp rx=0 tx=0).stop 2024-09-19T09:13:28.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.240+0000 7f55ff9e6740 1 -- 172.21.15.120:0/2052207405 shutdown_connections 2024-09-19T09:13:28.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.240+0000 7f55ff9e6740 1 --2- 172.21.15.120:0/2052207405 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558ba6917080 0x558ba69194a0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.240+0000 7f55ff9e6740 1 --2- 172.21.15.120:0/2052207405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ba6907370 0x558ba6904610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.240+0000 7f55daff5640 0 client.19287 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:28.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.240+0000 7f55ff9e6740 1 --2- 172.21.15.120:0/2052207405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ba68fdb40 0x558ba6904020 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.240+0000 7f55ff9e6740 1 --2- 172.21.15.120:0/2052207405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ba68fc5d0 0x558ba6906d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.240+0000 7f55ff9e6740 1 -- 172.21.15.120:0/2052207405 >> 172.21.15.120:0/2052207405 conn(0x558ba68c5310 msgr2=0x558ba68fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:28.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.240+0000 7f55ff9e6740 1 -- 172.21.15.120:0/2052207405 shutdown_connections 2024-09-19T09:13:28.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.240+0000 7f55ff9e6740 1 -- 172.21.15.120:0/2052207405 wait complete. 2024-09-19T09:13:28.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.240+0000 7f55ff9e6740 20 client.19287 trim_cache size 0 max 16384 2024-09-19T09:13:28.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.253+0000 7f55ff9e6740 1 Processor -- start 2024-09-19T09:13:28.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.253+0000 7f55ff9e6740 1 -- start start 2024-09-19T09:13:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.253+0000 7f55ff9e6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ba68fc5d0 0x558ba6913310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.253+0000 7f55ff9e6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ba68fdb40 0x558ba6903020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.253+0000 7f55ff9e6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ba6907370 0x558ba690f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.253+0000 7f55ff9e6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81ad4f0 con 0x558ba68fc5d0 2024-09-19T09:13:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.254+0000 7f55ff9e6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81af030 con 0x558ba68fdb40 2024-09-19T09:13:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.254+0000 7f55ff9e6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81adbc0 con 0x558ba6907370 2024-09-19T09:13:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.254+0000 7f55f67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ba68fc5d0 0x558ba6913310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.254+0000 7f55f67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ba68fc5d0 0x558ba6913310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56538/0 (socket says 172.21.15.120:56538) 2024-09-19T09:13:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.254+0000 7f55f67fc640 1 -- 172.21.15.120:0/4053554770 learned_addr learned my addr 172.21.15.120:0/4053554770 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.254+0000 7f55f67fc640 1 --2- 172.21.15.120:0/4053554770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ba6907370 0x558ba690f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:28.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.254+0000 7f55f67fc640 1 -- 172.21.15.120:0/4053554770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ba6907370 msgr2=0x558ba690f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:28.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.254+0000 7f55f67fc640 1 --2- 172.21.15.120:0/4053554770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ba6907370 0x558ba690f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.254+0000 7f55f67fc640 1 -- 172.21.15.120:0/4053554770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ba68fdb40 msgr2=0x558ba6903020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:28.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.254+0000 7f55f67fc640 1 --2- 172.21.15.120:0/4053554770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ba68fdb40 0x558ba6903020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.254+0000 7f55f67fc640 1 -- 172.21.15.120:0/4053554770 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ba6904980 con 0x558ba68fc5d0 2024-09-19T09:13:28.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.254+0000 7f55f67fc640 1 --2- 172.21.15.120:0/4053554770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ba68fc5d0 0x558ba6913310 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f55ec001eb0 tx=0x7f55ec005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c01ae7cf4c80dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:28.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.254+0000 7f55dcff9640 1 -- 172.21.15.120:0/4053554770 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec00a860 con 0x558ba68fc5d0 2024-09-19T09:13:28.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.254+0000 7f55dcff9640 1 -- 172.21.15.120:0/4053554770 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55ec02b3b0 con 0x558ba68fc5d0 2024-09-19T09:13:28.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.254+0000 7f55dcff9640 1 -- 172.21.15.120:0/4053554770 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec034690 con 0x558ba68fc5d0 2024-09-19T09:13:28.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.255+0000 7f55ff9e6740 1 -- 172.21.15.120:0/4053554770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ba68fc5d0 msgr2=0x558ba6913310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:28.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.255+0000 7f55ff9e6740 1 --2- 172.21.15.120:0/4053554770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ba68fc5d0 0x558ba6913310 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f55ec001eb0 tx=0x7f55ec005c30 comp rx=0 tx=0).stop 2024-09-19T09:13:28.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.255+0000 7f55ff9e6740 1 -- 172.21.15.120:0/4053554770 shutdown_connections 2024-09-19T09:13:28.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.255+0000 7f55ff9e6740 1 --2- 172.21.15.120:0/4053554770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ba6907370 0x558ba690f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.255+0000 7f55ff9e6740 1 --2- 172.21.15.120:0/4053554770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ba68fdb40 0x558ba6903020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.255+0000 7f55ff9e6740 1 --2- 172.21.15.120:0/4053554770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ba68fc5d0 0x558ba6913310 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.255+0000 7f55ff9e6740 1 -- 172.21.15.120:0/4053554770 >> 172.21.15.120:0/4053554770 conn(0x558ba68c5310 msgr2=0x558ba6902910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.255+0000 7f55ff9e6740 1 -- 172.21.15.120:0/4053554770 shutdown_connections 2024-09-19T09:13:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.255+0000 7f55ff9e6740 1 -- 172.21.15.120:0/4053554770 wait complete. 2024-09-19T09:13:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.255+0000 7f55ff9e6740 1 -- start start 2024-09-19T09:13:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.255+0000 7f55ff9e6740 1 Processor -- start 2024-09-19T09:13:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.255+0000 7f55ff9e6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ba68fc5d0 0x558ba691ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.255+0000 7f55ff9e6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ba68fdb40 0x558ba691ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.255+0000 7f55ff9e6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ba6907370 0x558ba691f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.255+0000 7f55ff9e6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81ae960 con 0x558ba68fdb40 2024-09-19T09:13:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.255+0000 7f55ff9e6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81ae290 con 0x558ba6907370 2024-09-19T09:13:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.255+0000 7f55ff9e6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81ab4b0 con 0x558ba68fc5d0 2024-09-19T09:13:28.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.255+0000 7f55ff9e6740 10 client.0 _ll_register_callbacks cb 0x7f55e8009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.256+0000 7f55f6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ba68fdb40 0x558ba691ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.256+0000 7f55f6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ba68fdb40 0x558ba691ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56554/0 (socket says 172.21.15.120:56554) 2024-09-19T09:13:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.256+0000 7f55f6ffd640 1 -- 172.21.15.120:0/3242384994 learned_addr learned my addr 172.21.15.120:0/3242384994 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.256+0000 7f55f67fc640 1 --2- 172.21.15.120:0/3242384994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ba6907370 0x558ba691f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.256+0000 7f55f6ffd640 1 -- 172.21.15.120:0/3242384994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ba68fc5d0 msgr2=0x558ba691ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.256+0000 7f55f6ffd640 1 --2- 172.21.15.120:0/3242384994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ba68fc5d0 0x558ba691ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.256+0000 7f55f6ffd640 1 -- 172.21.15.120:0/3242384994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ba6907370 msgr2=0x558ba691f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.256+0000 7f55f6ffd640 1 --2- 172.21.15.120:0/3242384994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ba6907370 0x558ba691f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.256+0000 7f55f6ffd640 1 -- 172.21.15.120:0/3242384994 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ba6984d70 con 0x558ba68fdb40 2024-09-19T09:13:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.256+0000 7f55f67fc640 1 --2- 172.21.15.120:0/3242384994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ba6907370 0x558ba691f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:13:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.256+0000 7f55f6ffd640 1 --2- 172.21.15.120:0/3242384994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ba68fdb40 0x558ba691ef70 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f55e8002230 tx=0x7f55e8021f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf60413f935d54d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:28.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.256+0000 7f55daff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:28.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.256+0000 7f55daff5640 1 -- 172.21.15.120:0/3242384994 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e802f060 con 0x558ba68fdb40 2024-09-19T09:13:28.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.256+0000 7f55daff5640 1 -- 172.21.15.120:0/3242384994 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f55e8016be0 con 0x558ba68fdb40 2024-09-19T09:13:28.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.256+0000 7f55ff9e6740 1 -- 172.21.15.120:0/3242384994 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55ec02b3b0 con 0x558ba68fdb40 2024-09-19T09:13:28.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.256+0000 7f55daff5640 1 -- 172.21.15.120:0/3242384994 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f55e8026070 con 0x558ba68fdb40 2024-09-19T09:13:28.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.256+0000 7f55ff9e6740 10 client.19293 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:28.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.256+0000 7f55ff9e6740 1 -- 172.21.15.120:0/3242384994 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558ba6985fb0 con 0x558ba68fdb40 2024-09-19T09:13:28.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.257+0000 7f55daff5640 1 -- 172.21.15.120:0/3242384994 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55ec02b3b0 con 0x558ba68fdb40 2024-09-19T09:13:28.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.257+0000 7f55daff5640 1 -- 172.21.15.120:0/3242384994 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f55e8020050 con 0x558ba68fdb40 2024-09-19T09:13:28.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.257+0000 7f55ff9e6740 10 client.19293 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:28.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.257+0000 7f55ff9e6740 10 client.19293 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:13:28.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.257+0000 7f55ff9e6740 20 client.19293 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:28.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.257+0000 7f55ff9e6740 1 --2- 172.21.15.120:0/3242384994 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558ba6917080 0x558ba6988710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:28.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.257+0000 7f55ff9e6740 4 client.19293 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:13:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.257+0000 7f55ff9e6740 1 -- 172.21.15.120:0/3242384994 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ba6869d70 con 0x558ba6917080 2024-09-19T09:13:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.257+0000 7f55f67fc640 1 --2- 172.21.15.120:0/3242384994 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558ba6917080 0x558ba6988710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.257+0000 7f55daff5640 10 client.19293 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.257+0000 7f55f67fc640 1 --2- 172.21.15.120:0/3242384994 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558ba6917080 0x558ba6988710 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb1545d316e488cb server_cookie=dac7e6c788e43d12 in_seq=0 out_seq=0 2024-09-19T09:13:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.258+0000 7f55daff5640 1 -- 172.21.15.120:0/3242384994 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ba6869d70 con 0x558ba6917080 2024-09-19T09:13:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.258+0000 7f55daff5640 10 client.19293 handle_command_reply: tid=0 2024-09-19T09:13:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.258+0000 7f55ff9e6740 1 client.19293 shutdown 2024-09-19T09:13:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.258+0000 7f55ff9e6740 1 -- 172.21.15.120:0/3242384994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ba68fdb40 msgr2=0x558ba691ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.258+0000 7f55ff9e6740 1 --2- 172.21.15.120:0/3242384994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ba68fdb40 0x558ba691ef70 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f55e8002230 tx=0x7f55e8021f60 comp rx=0 tx=0).stop 2024-09-19T09:13:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.258+0000 7f55ff9e6740 1 -- 172.21.15.120:0/3242384994 shutdown_connections 2024-09-19T09:13:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.258+0000 7f55ff9e6740 1 --2- 172.21.15.120:0/3242384994 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558ba6917080 0x558ba6988710 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.258+0000 7f55ff9e6740 1 --2- 172.21.15.120:0/3242384994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ba6907370 0x558ba691f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.258+0000 7f55daff5640 0 client.19293 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:28.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.258+0000 7f55ff9e6740 1 --2- 172.21.15.120:0/3242384994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ba68fdb40 0x558ba691ef70 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.258+0000 7f55ff9e6740 1 --2- 172.21.15.120:0/3242384994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ba68fc5d0 0x558ba691ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.258+0000 7f55ff9e6740 1 -- 172.21.15.120:0/3242384994 >> 172.21.15.120:0/3242384994 conn(0x558ba68c5310 msgr2=0x558ba6903130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:28.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.258+0000 7f55ff9e6740 1 -- 172.21.15.120:0/3242384994 shutdown_connections 2024-09-19T09:13:28.283 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:13:28.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.258+0000 7f55ff9e6740 1 -- 172.21.15.120:0/3242384994 wait complete. 2024-09-19T09:13:28.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.258+0000 7f55ff9e6740 20 client.19293 trim_cache size 0 max 16384 2024-09-19T09:13:28.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.258+0000 7f55fd60d640 1 -- 172.21.15.120:0/965401458 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f55e00a3240 msgr2=0x7f55e00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:28.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.258+0000 7f55fd60d640 1 --2- 172.21.15.120:0/965401458 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f55e00a3240 0x7f55e00a5700 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f55f00b8d70 tx=0x7f55f00872a0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.258+0000 7f55fd60d640 1 -- 172.21.15.120:0/965401458 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55f8106f10 msgr2=0x7f55f819cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:28.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.258+0000 7f55fd60d640 1 --2- 172.21.15.120:0/965401458 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55f8106f10 0x7f55f819cce0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009230 tx=0x7f55f00042f0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.258+0000 7f55fd60d640 1 -- 172.21.15.120:0/965401458 shutdown_connections 2024-09-19T09:13:28.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.258+0000 7f55fd60d640 1 --2- 172.21.15.120:0/965401458 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f55e00a3240 0x7f55e00a5700 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.258+0000 7f55fd60d640 1 --2- 172.21.15.120:0/965401458 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55f8106f10 0x7f55f819cce0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.258+0000 7f55fd60d640 1 --2- 172.21.15.120:0/965401458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55f8102790 0x7f55f8198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.258+0000 7f55fd60d640 1 --2- 172.21.15.120:0/965401458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55f8101de0 0x7f55f8198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:28.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.258+0000 7f55fd60d640 1 -- 172.21.15.120:0/965401458 >> 172.21.15.120:0/965401458 conn(0x7f55f80fd8c0 msgr2=0x7f55f80ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:28.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.259+0000 7f55fd60d640 1 -- 172.21.15.120:0/965401458 shutdown_connections 2024-09-19T09:13:28.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:28.259+0000 7f55fd60d640 1 -- 172.21.15.120:0/965401458 wait complete. 2024-09-19T09:13:29.070 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.37 MB/sec execute 444 sec latency 12376.940 ms 2024-09-19T09:13:29.155 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:28 smithi120 ceph-mon[31150]: pgmap v661: 97 pgs: 97 active+clean; 29 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-09-19T09:13:29.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:28 smithi196 ceph-mon[35150]: pgmap v661: 97 pgs: 97 active+clean; 29 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-09-19T09:13:29.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:28 smithi153 ceph-mon[35071]: pgmap v661: 97 pgs: 97 active+clean; 29 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-09-19T09:13:30.070 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.35 MB/sec execute 445 sec latency 13377.056 ms 2024-09-19T09:13:30.191 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:13:30.192 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:13:30.450 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:13:31.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.993+0000 7f2e5cf77640 1 -- 172.21.15.120:0/895986817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e58102790 msgr2=0x7f2e581067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:31.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.993+0000 7f2e5cf77640 1 --2- 172.21.15.120:0/895986817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e58102790 0x7f2e581067e0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c009900 tx=0x7f2e4c02a410 comp rx=0 tx=0).stop 2024-09-19T09:13:31.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.993+0000 7f2e5cf77640 1 -- 172.21.15.120:0/895986817 shutdown_connections 2024-09-19T09:13:31.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.993+0000 7f2e5cf77640 1 --2- 172.21.15.120:0/895986817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e58106f10 0x7f2e5810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.993+0000 7f2e5cf77640 1 --2- 172.21.15.120:0/895986817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e58102790 0x7f2e581067e0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.993+0000 7f2e5cf77640 1 --2- 172.21.15.120:0/895986817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e58101de0 0x7f2e581021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.993+0000 7f2e5cf77640 1 -- 172.21.15.120:0/895986817 >> 172.21.15.120:0/895986817 conn(0x7f2e580fd8c0 msgr2=0x7f2e580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:31.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.993+0000 7f2e5cf77640 1 -- 172.21.15.120:0/895986817 shutdown_connections 2024-09-19T09:13:31.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.993+0000 7f2e5cf77640 1 -- 172.21.15.120:0/895986817 wait complete. 2024-09-19T09:13:31.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.994+0000 7f2e5cf77640 1 Processor -- start 2024-09-19T09:13:31.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.994+0000 7f2e5cf77640 1 -- start start 2024-09-19T09:13:31.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.994+0000 7f2e5cf77640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e58101de0 0x7f2e58198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:31.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.994+0000 7f2e5cf77640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e58102790 0x7f2e581989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:31.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.994+0000 7f2e5cf77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e58106f10 0x7f2e5819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:31.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.994+0000 7f2e5cf77640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2e58111f70 con 0x7f2e58106f10 2024-09-19T09:13:31.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.994+0000 7f2e5cf77640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2e58111df0 con 0x7f2e58101de0 2024-09-19T09:13:31.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.994+0000 7f2e5cf77640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2e581120f0 con 0x7f2e58102790 2024-09-19T09:13:31.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.994+0000 7f2e56d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e58106f10 0x7f2e5819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:31.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.994+0000 7f2e56d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e58106f10 0x7f2e5819cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56580/0 (socket says 172.21.15.120:56580) 2024-09-19T09:13:31.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.994+0000 7f2e56d9d640 1 -- 172.21.15.120:0/2213146474 learned_addr learned my addr 172.21.15.120:0/2213146474 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:31.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.994+0000 7f2e5659c640 1 --2- 172.21.15.120:0/2213146474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e58101de0 0x7f2e58198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:31.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.994+0000 7f2e55d9b640 1 --2- 172.21.15.120:0/2213146474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e58102790 0x7f2e581989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:31.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.994+0000 7f2e56d9d640 1 -- 172.21.15.120:0/2213146474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e58102790 msgr2=0x7f2e581989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:31.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.994+0000 7f2e56d9d640 1 --2- 172.21.15.120:0/2213146474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e58102790 0x7f2e581989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.994+0000 7f2e56d9d640 1 -- 172.21.15.120:0/2213146474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e58101de0 msgr2=0x7f2e58198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:31.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.994+0000 7f2e56d9d640 1 --2- 172.21.15.120:0/2213146474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e58101de0 0x7f2e58198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.994+0000 7f2e56d9d640 1 -- 172.21.15.120:0/2213146474 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e5819d440 con 0x7f2e58106f10 2024-09-19T09:13:31.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.995+0000 7f2e56d9d640 1 --2- 172.21.15.120:0/2213146474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e58106f10 0x7f2e5819cd60 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f2e5000eaa0 tx=0x7f2e5000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:31.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.995+0000 7f2e477fe640 1 -- 172.21.15.120:0/2213146474 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5000cdb0 con 0x7f2e58106f10 2024-09-19T09:13:31.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.995+0000 7f2e477fe640 1 -- 172.21.15.120:0/2213146474 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e50005020 con 0x7f2e58106f10 2024-09-19T09:13:31.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.995+0000 7f2e5cf77640 1 -- 172.21.15.120:0/2213146474 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e5819d730 con 0x7f2e58106f10 2024-09-19T09:13:31.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.995+0000 7f2e477fe640 1 -- 172.21.15.120:0/2213146474 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e500116b0 con 0x7f2e58106f10 2024-09-19T09:13:31.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.995+0000 7f2e5cf77640 1 -- 172.21.15.120:0/2213146474 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e581a5010 con 0x7f2e58106f10 2024-09-19T09:13:31.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.997+0000 7f2e477fe640 1 -- 172.21.15.120:0/2213146474 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f2e50005560 con 0x7f2e58106f10 2024-09-19T09:13:31.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.997+0000 7f2e5f350740 1 Processor -- start 2024-09-19T09:13:31.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.997+0000 7f2e5f350740 1 -- start start 2024-09-19T09:13:31.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.998+0000 7f2e5f350740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea560fca20 0x55ea560fce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:31.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.998+0000 7f2e5f350740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea560fd340 0x55ea5610bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:31.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.998+0000 7f2e5f350740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea5610c150 0x55ea5610e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:31.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.998+0000 7f2e5f350740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ea56062280 con 0x55ea560fd340 2024-09-19T09:13:31.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.998+0000 7f2e5f350740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ea56009b10 con 0x55ea5610c150 2024-09-19T09:13:31.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.998+0000 7f2e5f350740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ea5605e240 con 0x55ea560fca20 2024-09-19T09:13:31.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.998+0000 7f2e5659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea560fd340 0x55ea5610bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:31.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.998+0000 7f2e5659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea560fd340 0x55ea5610bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56584/0 (socket says 172.21.15.120:56584) 2024-09-19T09:13:31.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.998+0000 7f2e5659c640 1 -- 172.21.15.120:0/726860284 learned_addr learned my addr 172.21.15.120:0/726860284 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:31.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.998+0000 7f2e477fe640 1 --2- 172.21.15.120:0/2213146474 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2e400a3240 0x7f2e400a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:31.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.998+0000 7f2e55d9b640 1 --2- 172.21.15.120:0/726860284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea560fca20 0x55ea560fce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:31.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.998+0000 7f2e55d9b640 1 --2- 172.21.15.120:0/726860284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea5610c150 0x55ea5610e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:31.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.998+0000 7f2e56d9d640 1 --2- 172.21.15.120:0/2213146474 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2e400a3240 0x7f2e400a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:31.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.998+0000 7f2e5659c640 1 -- 172.21.15.120:0/726860284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea560fca20 msgr2=0x55ea560fce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:31.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.998+0000 7f2e5659c640 1 --2- 172.21.15.120:0/726860284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea560fca20 0x55ea560fce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.998+0000 7f2e5659c640 1 -- 172.21.15.120:0/726860284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea5610c150 msgr2=0x55ea5610e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:31.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.998+0000 7f2e5659c640 1 --2- 172.21.15.120:0/726860284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea5610c150 0x55ea5610e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.998+0000 7f2e5659c640 1 -- 172.21.15.120:0/726860284 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea560161f0 con 0x55ea560fd340 2024-09-19T09:13:31.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.998+0000 7f2e477fe640 1 -- 172.21.15.120:0/2213146474 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f2e500cfa40 con 0x7f2e58106f10 2024-09-19T09:13:31.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.998+0000 7f2e5659c640 1 --2- 172.21.15.120:0/726860284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea560fd340 0x55ea5610bc10 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f2e4800bcd0 tx=0x7f2e480107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d36c1c9573301f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:31.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.998+0000 7f2e56d9d640 1 --2- 172.21.15.120:0/2213146474 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2e400a3240 0x7f2e400a5700 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2e50098ae0 tx=0x7f2e500c6e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:31.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.998+0000 7f2e44ff9640 1 -- 172.21.15.120:0/726860284 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e48005220 con 0x55ea560fd340 2024-09-19T09:13:31.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.999+0000 7f2e44ff9640 1 -- 172.21.15.120:0/726860284 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e48004c80 con 0x55ea560fd340 2024-09-19T09:13:31.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.999+0000 7f2e44ff9640 1 -- 172.21.15.120:0/726860284 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e480117c0 con 0x55ea560fd340 2024-09-19T09:13:31.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.999+0000 7f2e5f350740 1 -- 172.21.15.120:0/726860284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea560fd340 msgr2=0x55ea5610bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:31.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.999+0000 7f2e5f350740 1 --2- 172.21.15.120:0/726860284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea560fd340 0x55ea5610bc10 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f2e4800bcd0 tx=0x7f2e480107a0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.999+0000 7f2e5f350740 1 -- 172.21.15.120:0/726860284 shutdown_connections 2024-09-19T09:13:31.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.999+0000 7f2e5f350740 1 --2- 172.21.15.120:0/726860284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea5610c150 0x55ea5610e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.999+0000 7f2e5f350740 1 --2- 172.21.15.120:0/726860284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea560fd340 0x55ea5610bc10 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.999+0000 7f2e5f350740 1 --2- 172.21.15.120:0/726860284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea560fca20 0x55ea560fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.999+0000 7f2e5f350740 1 -- 172.21.15.120:0/726860284 >> 172.21.15.120:0/726860284 conn(0x55ea560f90f0 msgr2=0x55ea560f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:31.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.999+0000 7f2e5f350740 1 -- 172.21.15.120:0/726860284 shutdown_connections 2024-09-19T09:13:31.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:30.999+0000 7f2e5f350740 1 -- 172.21.15.120:0/726860284 wait complete. 2024-09-19T09:13:31.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.000+0000 7f2e5f350740 1 -- start start 2024-09-19T09:13:31.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.000+0000 7f2e5f350740 1 Processor -- start 2024-09-19T09:13:31.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.000+0000 7f2e5f350740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea560fca20 0x55ea561090b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:31.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.000+0000 7f2e5f350740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea560fd340 0x55ea561095f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:31.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.000+0000 7f2e5f350740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea5610c150 0x55ea56109b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:31.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.000+0000 7f2e5f350740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ea5610ed20 con 0x55ea560fca20 2024-09-19T09:13:31.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.000+0000 7f2e5f350740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ea5601cf10 con 0x55ea5610c150 2024-09-19T09:13:31.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.000+0000 7f2e5f350740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ea5610eea0 con 0x55ea560fd340 2024-09-19T09:13:31.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.000+0000 7f2e5f350740 10 client.0 _ll_register_callbacks cb 0x55ea55fe5a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:31.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.000+0000 7f2e55d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea560fca20 0x55ea561090b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:31.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.000+0000 7f2e55d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea560fca20 0x55ea561090b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56596/0 (socket says 172.21.15.120:56596) 2024-09-19T09:13:31.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.000+0000 7f2e55d9b640 1 -- 172.21.15.120:0/914334701 learned_addr learned my addr 172.21.15.120:0/914334701 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:31.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.000+0000 7f2e5659c640 1 --2- 172.21.15.120:0/914334701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea560fd340 0x55ea561095f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:31.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.000+0000 7f2e55d9b640 1 --2- 172.21.15.120:0/914334701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea5610c150 0x55ea56109b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:31.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.000+0000 7f2e55d9b640 1 -- 172.21.15.120:0/914334701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea560fd340 msgr2=0x55ea561095f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:31.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.000+0000 7f2e55d9b640 1 --2- 172.21.15.120:0/914334701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea560fd340 0x55ea561095f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.000+0000 7f2e55d9b640 1 -- 172.21.15.120:0/914334701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea5610c150 msgr2=0x55ea56109b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:31.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.000+0000 7f2e55d9b640 1 --2- 172.21.15.120:0/914334701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea5610c150 0x55ea56109b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.000+0000 7f2e55d9b640 1 -- 172.21.15.120:0/914334701 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea55dd36a0 con 0x55ea560fca20 2024-09-19T09:13:31.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.001+0000 7f2e55d9b640 1 --2- 172.21.15.120:0/914334701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea560fca20 0x55ea561090b0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c031470 tx=0x7f2e4c040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0a249eebadaa07 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:31.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.001+0000 7f2e3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:31.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.001+0000 7f2e3dffb640 1 -- 172.21.15.120:0/914334701 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c035030 con 0x55ea560fca20 2024-09-19T09:13:31.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.001+0000 7f2e5f350740 1 -- 172.21.15.120:0/914334701 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e48004c80 con 0x55ea560fca20 2024-09-19T09:13:31.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.001+0000 7f2e3dffb640 1 -- 172.21.15.120:0/914334701 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f2e4c004260 con 0x55ea560fca20 2024-09-19T09:13:31.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.001+0000 7f2e3dffb640 1 -- 172.21.15.120:0/914334701 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2e4c052050 con 0x55ea560fca20 2024-09-19T09:13:31.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.001+0000 7f2e5f350740 10 client.19305 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:31.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.001+0000 7f2e5f350740 1 -- 172.21.15.120:0/914334701 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ea5611a9e0 con 0x55ea560fca20 2024-09-19T09:13:31.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.001+0000 7f2e3dffb640 1 -- 172.21.15.120:0/914334701 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e48004c80 con 0x55ea560fca20 2024-09-19T09:13:31.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.001+0000 7f2e3dffb640 1 -- 172.21.15.120:0/914334701 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f2e4c03d450 con 0x55ea560fca20 2024-09-19T09:13:31.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.001+0000 7f2e5f350740 10 client.19305 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:31.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.001+0000 7f2e5f350740 10 client.19305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:13:31.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.001+0000 7f2e5f350740 20 client.19305 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:31.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.001+0000 7f2e5f350740 1 --2- 172.21.15.120:0/914334701 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ea5611bdf0 0x55ea5611e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:31.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.001+0000 7f2e5f350740 4 client.19305 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:13:31.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.001+0000 7f2e5f350740 1 -- 172.21.15.120:0/914334701 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea5606f7f0 con 0x55ea5611bdf0 2024-09-19T09:13:31.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.002+0000 7f2e5659c640 1 --2- 172.21.15.120:0/914334701 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ea5611bdf0 0x55ea5611e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:31.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.002+0000 7f2e3dffb640 10 client.19305 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:31.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.002+0000 7f2e5659c640 1 --2- 172.21.15.120:0/914334701 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ea5611bdf0 0x55ea5611e1e0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d22cb24da045fe3 server_cookie=540367ac070a587e in_seq=0 out_seq=0 2024-09-19T09:13:31.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.003+0000 7f2e3dffb640 1 -- 172.21.15.120:0/914334701 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ea5606f7f0 con 0x55ea5611bdf0 2024-09-19T09:13:31.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.003+0000 7f2e3dffb640 10 client.19305 handle_command_reply: tid=0 2024-09-19T09:13:31.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.003+0000 7f2e5f350740 1 client.19305 shutdown 2024-09-19T09:13:31.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.003+0000 7f2e5f350740 1 -- 172.21.15.120:0/914334701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea560fca20 msgr2=0x55ea561090b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:31.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.003+0000 7f2e5f350740 1 --2- 172.21.15.120:0/914334701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea560fca20 0x55ea561090b0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c031470 tx=0x7f2e4c040040 comp rx=0 tx=0).stop 2024-09-19T09:13:31.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.003+0000 7f2e5f350740 1 -- 172.21.15.120:0/914334701 shutdown_connections 2024-09-19T09:13:31.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.003+0000 7f2e5f350740 1 --2- 172.21.15.120:0/914334701 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ea5611bdf0 0x55ea5611e1e0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.003+0000 7f2e5f350740 1 --2- 172.21.15.120:0/914334701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea5610c150 0x55ea56109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.003+0000 7f2e3dffb640 0 client.19305 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:31.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.003+0000 7f2e5f350740 1 --2- 172.21.15.120:0/914334701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea560fd340 0x55ea561095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.003+0000 7f2e5f350740 1 --2- 172.21.15.120:0/914334701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea560fca20 0x55ea561090b0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.003+0000 7f2e5f350740 1 -- 172.21.15.120:0/914334701 >> 172.21.15.120:0/914334701 conn(0x55ea560f90f0 msgr2=0x55ea560fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:31.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.003+0000 7f2e5f350740 1 -- 172.21.15.120:0/914334701 shutdown_connections 2024-09-19T09:13:31.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.003+0000 7f2e5f350740 1 -- 172.21.15.120:0/914334701 wait complete. 2024-09-19T09:13:31.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.003+0000 7f2e5f350740 20 client.19305 trim_cache size 0 max 16384 2024-09-19T09:13:31.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.020+0000 7f2e5f350740 1 Processor -- start 2024-09-19T09:13:31.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.020+0000 7f2e5f350740 1 -- start start 2024-09-19T09:13:31.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.020+0000 7f2e5f350740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea560fca20 0x55ea56123a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:31.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.020+0000 7f2e5f350740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea560fd340 0x55ea56123fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:31.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.020+0000 7f2e5f350740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea5610c150 0x55ea56106310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:31.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.020+0000 7f2e5f350740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2e581ab460 con 0x55ea560fca20 2024-09-19T09:13:31.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.020+0000 7f2e5f350740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2e581acfa0 con 0x55ea560fd340 2024-09-19T09:13:31.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.020+0000 7f2e5f350740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2e581abb30 con 0x55ea5610c150 2024-09-19T09:13:31.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.020+0000 7f2e55d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea560fca20 0x55ea56123a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:31.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.020+0000 7f2e55d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea560fca20 0x55ea56123a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56600/0 (socket says 172.21.15.120:56600) 2024-09-19T09:13:31.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.020+0000 7f2e55d9b640 1 -- 172.21.15.120:0/1165560644 learned_addr learned my addr 172.21.15.120:0/1165560644 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:31.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.020+0000 7f2e5659c640 1 --2- 172.21.15.120:0/1165560644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea560fd340 0x55ea56123fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:31.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.020+0000 7f2e55d9b640 1 -- 172.21.15.120:0/1165560644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea5610c150 msgr2=0x55ea56106310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:31.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.020+0000 7f2e55d9b640 1 --2- 172.21.15.120:0/1165560644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea5610c150 0x55ea56106310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.020+0000 7f2e55d9b640 1 -- 172.21.15.120:0/1165560644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea560fd340 msgr2=0x55ea56123fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:31.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.020+0000 7f2e55d9b640 1 --2- 172.21.15.120:0/1165560644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea560fd340 0x55ea56123fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.020+0000 7f2e55d9b640 1 -- 172.21.15.120:0/1165560644 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea56108060 con 0x55ea560fca20 2024-09-19T09:13:31.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.021+0000 7f2e55d9b640 1 --2- 172.21.15.120:0/1165560644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea560fca20 0x55ea56123a70 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c005c40 tx=0x7f2e4c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c29a8ebf57cfd5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:31.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.021+0000 7f2e3ffff640 1 -- 172.21.15.120:0/1165560644 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c05b070 con 0x55ea560fca20 2024-09-19T09:13:31.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.021+0000 7f2e3ffff640 1 -- 172.21.15.120:0/1165560644 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e4c005850 con 0x55ea560fca20 2024-09-19T09:13:31.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.021+0000 7f2e3ffff640 1 -- 172.21.15.120:0/1165560644 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c035040 con 0x55ea560fca20 2024-09-19T09:13:31.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.021+0000 7f2e5f350740 1 -- 172.21.15.120:0/1165560644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea560fca20 msgr2=0x55ea56123a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:31.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.021+0000 7f2e5f350740 1 --2- 172.21.15.120:0/1165560644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea560fca20 0x55ea56123a70 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c005c40 tx=0x7f2e4c02a410 comp rx=0 tx=0).stop 2024-09-19T09:13:31.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.021+0000 7f2e5f350740 1 -- 172.21.15.120:0/1165560644 shutdown_connections 2024-09-19T09:13:31.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.021+0000 7f2e5f350740 1 --2- 172.21.15.120:0/1165560644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea5610c150 0x55ea56106310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.021+0000 7f2e5f350740 1 --2- 172.21.15.120:0/1165560644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea560fd340 0x55ea56123fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.021+0000 7f2e5f350740 1 --2- 172.21.15.120:0/1165560644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea560fca20 0x55ea56123a70 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.021+0000 7f2e5f350740 1 -- 172.21.15.120:0/1165560644 >> 172.21.15.120:0/1165560644 conn(0x55ea560f90f0 msgr2=0x55ea5610b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:31.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.022+0000 7f2e5f350740 1 -- 172.21.15.120:0/1165560644 shutdown_connections 2024-09-19T09:13:31.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.022+0000 7f2e5f350740 1 -- 172.21.15.120:0/1165560644 wait complete. 2024-09-19T09:13:31.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.022+0000 7f2e5f350740 1 -- start start 2024-09-19T09:13:31.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.022+0000 7f2e5f350740 1 Processor -- start 2024-09-19T09:13:31.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.022+0000 7f2e5f350740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea560fca20 0x55ea56110590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:31.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.022+0000 7f2e5f350740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea560fd340 0x55ea56110ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:31.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.022+0000 7f2e5f350740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea5610c150 0x55ea56185a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:31.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.022+0000 7f2e5f350740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2e581ac8d0 con 0x55ea560fca20 2024-09-19T09:13:31.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.022+0000 7f2e5f350740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2e581ac200 con 0x55ea5610c150 2024-09-19T09:13:31.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.022+0000 7f2e5f350740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2e581a9420 con 0x55ea560fd340 2024-09-19T09:13:31.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.022+0000 7f2e5f350740 10 client.0 _ll_register_callbacks cb 0x7f2e4c032ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:31.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.022+0000 7f2e55d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea560fca20 0x55ea56110590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:31.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.022+0000 7f2e55d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea560fca20 0x55ea56110590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56602/0 (socket says 172.21.15.120:56602) 2024-09-19T09:13:31.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.022+0000 7f2e55d9b640 1 -- 172.21.15.120:0/3881494263 learned_addr learned my addr 172.21.15.120:0/3881494263 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:31.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.022+0000 7f2e5659c640 1 --2- 172.21.15.120:0/3881494263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea560fd340 0x55ea56110ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:31.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.022+0000 7f2e55d9b640 1 --2- 172.21.15.120:0/3881494263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea5610c150 0x55ea56185a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:31.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.023+0000 7f2e55d9b640 1 -- 172.21.15.120:0/3881494263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea560fd340 msgr2=0x55ea56110ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:31.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.023+0000 7f2e55d9b640 1 --2- 172.21.15.120:0/3881494263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea560fd340 0x55ea56110ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.023+0000 7f2e55d9b640 1 -- 172.21.15.120:0/3881494263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea5610c150 msgr2=0x55ea56185a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:31.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.023+0000 7f2e55d9b640 1 --2- 172.21.15.120:0/3881494263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea5610c150 0x55ea56185a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.023+0000 7f2e55d9b640 1 -- 172.21.15.120:0/3881494263 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea560fd170 con 0x55ea560fca20 2024-09-19T09:13:31.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.023+0000 7f2e55d9b640 1 --2- 172.21.15.120:0/3881494263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea560fca20 0x55ea56110590 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c005dd0 tx=0x7f2e4c03d4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c28b25c570a38f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:31.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.023+0000 7f2e3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:31.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.023+0000 7f2e3dffb640 1 -- 172.21.15.120:0/3881494263 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c05b070 con 0x55ea560fca20 2024-09-19T09:13:31.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.023+0000 7f2e3dffb640 1 -- 172.21.15.120:0/3881494263 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f2e4c004460 con 0x55ea560fca20 2024-09-19T09:13:31.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.023+0000 7f2e5f350740 1 -- 172.21.15.120:0/3881494263 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e4c005850 con 0x55ea560fca20 2024-09-19T09:13:31.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.023+0000 7f2e3dffb640 1 -- 172.21.15.120:0/3881494263 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2e4c035070 con 0x55ea560fca20 2024-09-19T09:13:31.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.023+0000 7f2e5f350740 10 client.19311 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:31.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.023+0000 7f2e5f350740 1 -- 172.21.15.120:0/3881494263 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ea561870c0 con 0x55ea560fca20 2024-09-19T09:13:31.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.023+0000 7f2e3dffb640 1 -- 172.21.15.120:0/3881494263 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e4c005850 con 0x55ea560fca20 2024-09-19T09:13:31.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.023+0000 7f2e3dffb640 1 -- 172.21.15.120:0/3881494263 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f2e4c004750 con 0x55ea560fca20 2024-09-19T09:13:31.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.023+0000 7f2e5f350740 10 client.19311 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:31.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.023+0000 7f2e5f350740 10 client.19311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:13:31.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.023+0000 7f2e5f350740 20 client.19311 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:31.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.023+0000 7f2e5f350740 1 --2- 172.21.15.120:0/3881494263 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ea56187720 0x55ea56189b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:31.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.023+0000 7f2e5f350740 4 client.19311 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:13:31.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.023+0000 7f2e5f350740 1 -- 172.21.15.120:0/3881494263 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ea56069d00 con 0x55ea56187720 2024-09-19T09:13:31.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.024+0000 7f2e5659c640 1 --2- 172.21.15.120:0/3881494263 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ea56187720 0x55ea56189b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:31.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.024+0000 7f2e5659c640 1 --2- 172.21.15.120:0/3881494263 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ea56187720 0x55ea56189b10 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6598c6418b9e83e4 server_cookie=dd68e68b2e4f7a6d in_seq=0 out_seq=0 2024-09-19T09:13:31.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.024+0000 7f2e3dffb640 10 client.19311 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:31.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.025+0000 7f2e3dffb640 1 -- 172.21.15.120:0/3881494263 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ea56069d00 con 0x55ea56187720 2024-09-19T09:13:31.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.025+0000 7f2e3dffb640 10 client.19311 handle_command_reply: tid=0 2024-09-19T09:13:31.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.025+0000 7f2e5f350740 1 client.19311 shutdown 2024-09-19T09:13:31.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.025+0000 7f2e5f350740 1 -- 172.21.15.120:0/3881494263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea560fca20 msgr2=0x55ea56110590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:31.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.025+0000 7f2e5f350740 1 --2- 172.21.15.120:0/3881494263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea560fca20 0x55ea56110590 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c005dd0 tx=0x7f2e4c03d4f0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.025+0000 7f2e5f350740 1 -- 172.21.15.120:0/3881494263 shutdown_connections 2024-09-19T09:13:31.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.025+0000 7f2e5f350740 1 --2- 172.21.15.120:0/3881494263 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ea56187720 0x55ea56189b10 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.025+0000 7f2e5f350740 1 --2- 172.21.15.120:0/3881494263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea5610c150 0x55ea56185a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.025+0000 7f2e3dffb640 0 client.19311 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:31.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.025+0000 7f2e5f350740 1 --2- 172.21.15.120:0/3881494263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea560fd340 0x55ea56110ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.025+0000 7f2e5f350740 1 --2- 172.21.15.120:0/3881494263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea560fca20 0x55ea56110590 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.025+0000 7f2e5f350740 1 -- 172.21.15.120:0/3881494263 >> 172.21.15.120:0/3881494263 conn(0x55ea560f90f0 msgr2=0x55ea5610a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:31.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.025+0000 7f2e5f350740 1 -- 172.21.15.120:0/3881494263 shutdown_connections 2024-09-19T09:13:31.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.025+0000 7f2e5f350740 1 -- 172.21.15.120:0/3881494263 wait complete. 2024-09-19T09:13:31.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.026+0000 7f2e5f350740 20 client.19311 trim_cache size 0 max 16384 2024-09-19T09:13:31.031 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:13:31.031 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:13:31.031 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "d3d6728f-1d1f-49ef-831f-19392a6e5c2f", 2024-09-19T09:13:31.031 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:13:31.031 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:13:31.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.026+0000 7f2e5cf77640 1 -- 172.21.15.120:0/2213146474 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2e400a3240 msgr2=0x7f2e400a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:31.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.026+0000 7f2e5cf77640 1 --2- 172.21.15.120:0/2213146474 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2e400a3240 0x7f2e400a5700 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2e50098ae0 tx=0x7f2e500c6e30 comp rx=0 tx=0).stop 2024-09-19T09:13:31.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.026+0000 7f2e5cf77640 1 -- 172.21.15.120:0/2213146474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e58106f10 msgr2=0x7f2e5819cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:31.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.026+0000 7f2e5cf77640 1 --2- 172.21.15.120:0/2213146474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e58106f10 0x7f2e5819cd60 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f2e5000eaa0 tx=0x7f2e5000ef60 comp rx=0 tx=0).stop 2024-09-19T09:13:31.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.026+0000 7f2e5cf77640 1 -- 172.21.15.120:0/2213146474 shutdown_connections 2024-09-19T09:13:31.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.026+0000 7f2e5cf77640 1 --2- 172.21.15.120:0/2213146474 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2e400a3240 0x7f2e400a5700 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.026+0000 7f2e5cf77640 1 --2- 172.21.15.120:0/2213146474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e58106f10 0x7f2e5819cd60 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.026+0000 7f2e5cf77640 1 --2- 172.21.15.120:0/2213146474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e58102790 0x7f2e581989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.026+0000 7f2e5cf77640 1 --2- 172.21.15.120:0/2213146474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e58101de0 0x7f2e58198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:31.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.026+0000 7f2e5cf77640 1 -- 172.21.15.120:0/2213146474 >> 172.21.15.120:0/2213146474 conn(0x7f2e580fd8c0 msgr2=0x7f2e580fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:31.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.026+0000 7f2e5cf77640 1 -- 172.21.15.120:0/2213146474 shutdown_connections 2024-09-19T09:13:31.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:31.026+0000 7f2e5cf77640 1 -- 172.21.15.120:0/2213146474 wait complete. 2024-09-19T09:13:31.070 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.33 MB/sec execute 446 sec latency 14377.111 ms 2024-09-19T09:13:31.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:30 smithi196 ceph-mon[35150]: pgmap v662: 97 pgs: 97 active+clean; 29 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 687 KiB/s wr, 108 op/s 2024-09-19T09:13:31.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:30 smithi153 ceph-mon[35071]: pgmap v662: 97 pgs: 97 active+clean; 29 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 687 KiB/s wr, 108 op/s 2024-09-19T09:13:31.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:30 smithi120 ceph-mon[31150]: pgmap v662: 97 pgs: 97 active+clean; 29 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 687 KiB/s wr, 108 op/s 2024-09-19T09:13:31.964 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:13:32.070 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.30 MB/sec execute 447 sec latency 15377.178 ms 2024-09-19T09:13:32.208 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:13:32.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.901+0000 7fa9cb460640 1 -- 172.21.15.120:0/1208918265 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9c4102790 msgr2=0x7fa9c41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:32.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.901+0000 7fa9cb460640 1 --2- 172.21.15.120:0/1208918265 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9c4102790 0x7fa9c41067e0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4009900 tx=0x7fa9b402a410 comp rx=0 tx=0).stop 2024-09-19T09:13:32.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.901+0000 7fa9cb460640 1 -- 172.21.15.120:0/1208918265 shutdown_connections 2024-09-19T09:13:32.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.901+0000 7fa9cb460640 1 --2- 172.21.15.120:0/1208918265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9c4106f10 0x7fa9c410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.901+0000 7fa9cb460640 1 --2- 172.21.15.120:0/1208918265 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9c4102790 0x7fa9c41067e0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.901+0000 7fa9cb460640 1 --2- 172.21.15.120:0/1208918265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9c4101de0 0x7fa9c41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.901+0000 7fa9cb460640 1 -- 172.21.15.120:0/1208918265 >> 172.21.15.120:0/1208918265 conn(0x7fa9c40fd880 msgr2=0x7fa9c40ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:32.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.901+0000 7fa9cb460640 1 -- 172.21.15.120:0/1208918265 shutdown_connections 2024-09-19T09:13:32.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.901+0000 7fa9cb460640 1 -- 172.21.15.120:0/1208918265 wait complete. 2024-09-19T09:13:32.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.901+0000 7fa9cb460640 1 Processor -- start 2024-09-19T09:13:32.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.902+0000 7fa9cb460640 1 -- start start 2024-09-19T09:13:32.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.902+0000 7fa9cb460640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9c4101de0 0x7fa9c4198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:32.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.902+0000 7fa9cb460640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9c4102790 0x7fa9c4198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:32.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.902+0000 7fa9cb460640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9c4106f10 0x7fa9c419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:32.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.902+0000 7fa9cb460640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c4111f70 con 0x7fa9c4102790 2024-09-19T09:13:32.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.902+0000 7fa9cb460640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c4111df0 con 0x7fa9c4101de0 2024-09-19T09:13:32.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.902+0000 7fa9cb460640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41120f0 con 0x7fa9c4106f10 2024-09-19T09:13:32.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.902+0000 7fa9c89fb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9c4102790 0x7fa9c4198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:32.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.902+0000 7fa9c91fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9c4101de0 0x7fa9c4198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:32.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.902+0000 7fa9c91fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9c4101de0 0x7fa9c4198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:55582/0 (socket says 172.21.15.120:55582) 2024-09-19T09:13:32.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.902+0000 7fa9c91fc640 1 -- 172.21.15.120:0/2861173342 learned_addr learned my addr 172.21.15.120:0/2861173342 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:32.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.902+0000 7fa9c89fb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9c4102790 0x7fa9c4198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51396/0 (socket says 172.21.15.120:51396) 2024-09-19T09:13:32.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.902+0000 7fa9c99fd640 1 --2- 172.21.15.120:0/2861173342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9c4106f10 0x7fa9c419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:32.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.902+0000 7fa9c89fb640 1 -- 172.21.15.120:0/2861173342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9c4106f10 msgr2=0x7fa9c419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:32.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.902+0000 7fa9c89fb640 1 --2- 172.21.15.120:0/2861173342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9c4106f10 0x7fa9c419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.902+0000 7fa9c89fb640 1 -- 172.21.15.120:0/2861173342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9c4101de0 msgr2=0x7fa9c4198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:32.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.902+0000 7fa9c89fb640 1 --2- 172.21.15.120:0/2861173342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9c4101de0 0x7fa9c4198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.902+0000 7fa9c89fb640 1 -- 172.21.15.120:0/2861173342 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c419d400 con 0x7fa9c4102790 2024-09-19T09:13:32.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.902+0000 7fa9c99fd640 1 --2- 172.21.15.120:0/2861173342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9c4106f10 0x7fa9c419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:13:32.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.902+0000 7fa9c91fc640 1 --2- 172.21.15.120:0/2861173342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9c4101de0 0x7fa9c4198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:13:32.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.903+0000 7fa9c89fb640 1 --2- 172.21.15.120:0/2861173342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9c4102790 0x7fa9c4198990 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4005940 tx=0x7fa9b4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:32.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.903+0000 7fa9ba7fc640 1 -- 172.21.15.120:0/2861173342 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b403f030 con 0x7fa9c4102790 2024-09-19T09:13:32.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.903+0000 7fa9ba7fc640 1 -- 172.21.15.120:0/2861173342 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9b4003ad0 con 0x7fa9c4102790 2024-09-19T09:13:32.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.903+0000 7fa9cb460640 1 -- 172.21.15.120:0/2861173342 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c419d690 con 0x7fa9c4102790 2024-09-19T09:13:32.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.903+0000 7fa9ba7fc640 1 -- 172.21.15.120:0/2861173342 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4035070 con 0x7fa9c4102790 2024-09-19T09:13:32.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.903+0000 7fa9cb460640 1 -- 172.21.15.120:0/2861173342 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c41a4fd0 con 0x7fa9c4102790 2024-09-19T09:13:32.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.905+0000 7fa9ba7fc640 1 -- 172.21.15.120:0/2861173342 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa9b4042030 con 0x7fa9c4102790 2024-09-19T09:13:32.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.906+0000 7fa9cd839740 1 Processor -- start 2024-09-19T09:13:32.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.906+0000 7fa9cd839740 1 -- start start 2024-09-19T09:13:32.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.906+0000 7fa9ba7fc640 1 --2- 172.21.15.120:0/2861173342 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9b00a3240 0x7fa9b00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:32.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.906+0000 7fa9cd839740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa3fc03c0 0x555aa3fc07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:32.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.906+0000 7fa9cd839740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa3fc0ce0 0x555aa3fcf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:32.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.906+0000 7fa9cd839740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555aa3fcfa90 0x555aa3fd1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:32.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.906+0000 7fa9cd839740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555aa3f262e0 con 0x555aa3fcfa90 2024-09-19T09:13:32.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.906+0000 7fa9cd839740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555aa3ecdce0 con 0x555aa3fc0ce0 2024-09-19T09:13:32.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.906+0000 7fa9cd839740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555aa3f222a0 con 0x555aa3fc03c0 2024-09-19T09:13:32.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.906+0000 7fa9c99fd640 1 --2- 172.21.15.120:0/2861173342 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9b00a3240 0x7fa9b00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:32.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.906+0000 7fa9ba7fc640 1 -- 172.21.15.120:0/2861173342 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa9b40eff20 con 0x7fa9c4102790 2024-09-19T09:13:32.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.906+0000 7fa9c99fd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555aa3fcfa90 0x555aa3fd1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:32.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.906+0000 7fa9c99fd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555aa3fcfa90 0x555aa3fd1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51398/0 (socket says 172.21.15.120:51398) 2024-09-19T09:13:32.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.906+0000 7fa9c99fd640 1 -- 172.21.15.120:0/1651723255 learned_addr learned my addr 172.21.15.120:0/1651723255 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:32.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.906+0000 7fa9c89fb640 1 --2- 172.21.15.120:0/1651723255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa3fc0ce0 0x555aa3fcf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:32.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.906+0000 7fa9c91fc640 1 --2- 172.21.15.120:0/1651723255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa3fc03c0 0x555aa3fc07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:32.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.906+0000 7fa9c99fd640 1 -- 172.21.15.120:0/1651723255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa3fc03c0 msgr2=0x555aa3fc07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:32.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.906+0000 7fa9c99fd640 1 --2- 172.21.15.120:0/1651723255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa3fc03c0 0x555aa3fc07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.906+0000 7fa9c99fd640 1 -- 172.21.15.120:0/1651723255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa3fc0ce0 msgr2=0x555aa3fcf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:32.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.906+0000 7fa9c99fd640 1 --2- 172.21.15.120:0/1651723255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa3fc0ce0 0x555aa3fcf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.906+0000 7fa9c99fd640 1 -- 172.21.15.120:0/1651723255 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aa3eb2440 con 0x555aa3fcfa90 2024-09-19T09:13:32.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.906+0000 7fa9c89fb640 1 --2- 172.21.15.120:0/1651723255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa3fc0ce0 0x555aa3fcf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:13:32.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.907+0000 7fa9c99fd640 1 --2- 172.21.15.120:0/1651723255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555aa3fcfa90 0x555aa3fd1e80 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0014ea0 tx=0x7fa9c0014f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb12a7f67f6deac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:32.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.907+0000 7fa9c99fd640 1 --2- 172.21.15.120:0/2861173342 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9b00a3240 0x7fa9b00a5700 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0011c40 tx=0x7fa9c001c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:32.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.907+0000 7fa9af7fe640 1 -- 172.21.15.120:0/1651723255 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0022070 con 0x555aa3fcfa90 2024-09-19T09:13:32.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.907+0000 7fa9af7fe640 1 -- 172.21.15.120:0/1651723255 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9c00141a0 con 0x555aa3fcfa90 2024-09-19T09:13:32.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.907+0000 7fa9af7fe640 1 -- 172.21.15.120:0/1651723255 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c00053f0 con 0x555aa3fcfa90 2024-09-19T09:13:32.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.907+0000 7fa9cd839740 1 -- 172.21.15.120:0/1651723255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555aa3fcfa90 msgr2=0x555aa3fd1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:32.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.907+0000 7fa9cd839740 1 --2- 172.21.15.120:0/1651723255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555aa3fcfa90 0x555aa3fd1e80 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0014ea0 tx=0x7fa9c0014f90 comp rx=0 tx=0).stop 2024-09-19T09:13:32.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.907+0000 7fa9cd839740 1 -- 172.21.15.120:0/1651723255 shutdown_connections 2024-09-19T09:13:32.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.907+0000 7fa9cd839740 1 --2- 172.21.15.120:0/1651723255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555aa3fcfa90 0x555aa3fd1e80 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.907+0000 7fa9cd839740 1 --2- 172.21.15.120:0/1651723255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa3fc0ce0 0x555aa3fcf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.907+0000 7fa9cd839740 1 --2- 172.21.15.120:0/1651723255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa3fc03c0 0x555aa3fc07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.908+0000 7fa9cd839740 1 -- 172.21.15.120:0/1651723255 >> 172.21.15.120:0/1651723255 conn(0x555aa3fbca30 msgr2=0x555aa3fbce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:32.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.908+0000 7fa9cd839740 1 -- 172.21.15.120:0/1651723255 shutdown_connections 2024-09-19T09:13:32.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.908+0000 7fa9cd839740 1 -- 172.21.15.120:0/1651723255 wait complete. 2024-09-19T09:13:32.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.908+0000 7fa9cd839740 1 -- start start 2024-09-19T09:13:32.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.908+0000 7fa9cd839740 1 Processor -- start 2024-09-19T09:13:32.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.908+0000 7fa9cd839740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa3fc03c0 0x555aa3fcf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:32.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.908+0000 7fa9cd839740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa3fc0ce0 0x555aa3fcc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:32.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.908+0000 7fa9cd839740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555aa3fcfa90 0x555aa3fccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:32.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.908+0000 7fa9cd839740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555aa3fd25a0 con 0x555aa3fcfa90 2024-09-19T09:13:32.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.908+0000 7fa9cd839740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555aa3ee1050 con 0x555aa3fc03c0 2024-09-19T09:13:32.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.908+0000 7fa9cd839740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555aa3fd2720 con 0x555aa3fc0ce0 2024-09-19T09:13:32.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.908+0000 7fa9cd839740 10 client.0 _ll_register_callbacks cb 0x555aa3f21160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:32.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.908+0000 7fa9c99fd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555aa3fcfa90 0x555aa3fccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:32.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.909+0000 7fa9c99fd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555aa3fcfa90 0x555aa3fccd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51400/0 (socket says 172.21.15.120:51400) 2024-09-19T09:13:32.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.909+0000 7fa9c99fd640 1 -- 172.21.15.120:0/3635323710 learned_addr learned my addr 172.21.15.120:0/3635323710 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:32.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.909+0000 7fa9c91fc640 1 --2- 172.21.15.120:0/3635323710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa3fc03c0 0x555aa3fcf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:32.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.909+0000 7fa9c89fb640 1 --2- 172.21.15.120:0/3635323710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa3fc0ce0 0x555aa3fcc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:32.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.909+0000 7fa9c99fd640 1 -- 172.21.15.120:0/3635323710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa3fc0ce0 msgr2=0x555aa3fcc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:32.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.909+0000 7fa9c99fd640 1 --2- 172.21.15.120:0/3635323710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa3fc0ce0 0x555aa3fcc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.909+0000 7fa9c99fd640 1 -- 172.21.15.120:0/3635323710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa3fc03c0 msgr2=0x555aa3fcf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:32.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.909+0000 7fa9c99fd640 1 --2- 172.21.15.120:0/3635323710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa3fc03c0 0x555aa3fcf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.909+0000 7fa9c99fd640 1 -- 172.21.15.120:0/3635323710 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aa3c973c0 con 0x555aa3fcfa90 2024-09-19T09:13:32.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.909+0000 7fa9c89fb640 1 --2- 172.21.15.120:0/3635323710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa3fc0ce0 0x555aa3fcc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:13:32.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.909+0000 7fa9c91fc640 1 --2- 172.21.15.120:0/3635323710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa3fc03c0 0x555aa3fcf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:13:32.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.909+0000 7fa9c99fd640 1 --2- 172.21.15.120:0/3635323710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555aa3fcfa90 0x555aa3fccd30 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0006010 tx=0x7fa9c0014be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52df74444d1804e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:32.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.909+0000 7fa9acff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:32.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.909+0000 7fa9acff9640 1 -- 172.21.15.120:0/3635323710 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0022040 con 0x555aa3fcfa90 2024-09-19T09:13:32.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.909+0000 7fa9cd839740 1 -- 172.21.15.120:0/3635323710 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9c00141a0 con 0x555aa3fcfa90 2024-09-19T09:13:32.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.909+0000 7fa9acff9640 1 -- 172.21.15.120:0/3635323710 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa9c0005660 con 0x555aa3fcfa90 2024-09-19T09:13:32.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.909+0000 7fa9acff9640 1 -- 172.21.15.120:0/3635323710 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa9c0005860 con 0x555aa3fcfa90 2024-09-19T09:13:32.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.909+0000 7fa9cd839740 10 client.19323 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:32.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.909+0000 7fa9cd839740 1 -- 172.21.15.120:0/3635323710 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555aa3fde040 con 0x555aa3fcfa90 2024-09-19T09:13:32.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.909+0000 7fa9acff9640 1 -- 172.21.15.120:0/3635323710 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9c00141a0 con 0x555aa3fcfa90 2024-09-19T09:13:32.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.910+0000 7fa9acff9640 1 -- 172.21.15.120:0/3635323710 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa9c001d070 con 0x555aa3fcfa90 2024-09-19T09:13:32.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.910+0000 7fa9cd839740 10 client.19323 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:32.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.910+0000 7fa9cd839740 10 client.19323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:13:32.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.910+0000 7fa9cd839740 20 client.19323 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:32.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.910+0000 7fa9cd839740 1 --2- 172.21.15.120:0/3635323710 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555aa3fdf450 0x555aa3fe1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:32.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.910+0000 7fa9cd839740 4 client.19323 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:13:32.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.910+0000 7fa9cd839740 1 -- 172.21.15.120:0/3635323710 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555aa3f33850 con 0x555aa3fdf450 2024-09-19T09:13:32.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.910+0000 7fa9c91fc640 1 --2- 172.21.15.120:0/3635323710 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555aa3fdf450 0x555aa3fe1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:32.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.910+0000 7fa9c91fc640 1 --2- 172.21.15.120:0/3635323710 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555aa3fdf450 0x555aa3fe1840 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddbc1c935b033112 server_cookie=516c6a5d83c5f1a5 in_seq=0 out_seq=0 2024-09-19T09:13:32.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.910+0000 7fa9acff9640 10 client.19323 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:32.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.911+0000 7fa9acff9640 1 -- 172.21.15.120:0/3635323710 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555aa3f33850 con 0x555aa3fdf450 2024-09-19T09:13:32.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.911+0000 7fa9acff9640 10 client.19323 handle_command_reply: tid=0 2024-09-19T09:13:32.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.911+0000 7fa9cd839740 1 client.19323 shutdown 2024-09-19T09:13:32.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.911+0000 7fa9cd839740 1 -- 172.21.15.120:0/3635323710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555aa3fcfa90 msgr2=0x555aa3fccd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:32.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.911+0000 7fa9cd839740 1 --2- 172.21.15.120:0/3635323710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555aa3fcfa90 0x555aa3fccd30 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0006010 tx=0x7fa9c0014be0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.911+0000 7fa9cd839740 1 -- 172.21.15.120:0/3635323710 shutdown_connections 2024-09-19T09:13:32.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.911+0000 7fa9cd839740 1 --2- 172.21.15.120:0/3635323710 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555aa3fdf450 0x555aa3fe1840 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.911+0000 7fa9acff9640 0 client.19323 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:32.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.911+0000 7fa9cd839740 1 --2- 172.21.15.120:0/3635323710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555aa3fcfa90 0x555aa3fccd30 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.911+0000 7fa9cd839740 1 --2- 172.21.15.120:0/3635323710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa3fc0ce0 0x555aa3fcc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.911+0000 7fa9cd839740 1 --2- 172.21.15.120:0/3635323710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa3fc03c0 0x555aa3fcf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.911+0000 7fa9cd839740 1 -- 172.21.15.120:0/3635323710 >> 172.21.15.120:0/3635323710 conn(0x555aa3fbca30 msgr2=0x555aa3fc26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:32.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.911+0000 7fa9cd839740 1 -- 172.21.15.120:0/3635323710 shutdown_connections 2024-09-19T09:13:32.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.911+0000 7fa9cd839740 1 -- 172.21.15.120:0/3635323710 wait complete. 2024-09-19T09:13:32.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.911+0000 7fa9cd839740 20 client.19323 trim_cache size 0 max 16384 2024-09-19T09:13:32.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.928+0000 7fa9cd839740 1 Processor -- start 2024-09-19T09:13:32.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.928+0000 7fa9cd839740 1 -- start start 2024-09-19T09:13:32.941 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:32 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:13:32.941 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:32 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:13:32.941 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:32 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:13:32.941 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:32 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:13:32.942 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:32 smithi120 ceph-mon[31150]: pgmap v663: 97 pgs: 97 active+clean; 29 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 130 KiB/s wr, 60 op/s 2024-09-19T09:13:32.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.929+0000 7fa9cd839740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa3fc03c0 0x555aa3fdb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:32.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.929+0000 7fa9cd839740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa3fc0ce0 0x555aa3fe6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:32.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.929+0000 7fa9cd839740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555aa3fcfa90 0x555aa3fe73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:32.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.929+0000 7fa9cd839740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41ab3b0 con 0x555aa3fcfa90 2024-09-19T09:13:32.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.929+0000 7fa9cd839740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41acef0 con 0x555aa3fc03c0 2024-09-19T09:13:32.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.929+0000 7fa9cd839740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41aba80 con 0x555aa3fc0ce0 2024-09-19T09:13:32.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.929+0000 7fa9c99fd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555aa3fcfa90 0x555aa3fe73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:32.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.929+0000 7fa9c99fd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555aa3fcfa90 0x555aa3fe73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51408/0 (socket says 172.21.15.120:51408) 2024-09-19T09:13:32.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.929+0000 7fa9c99fd640 1 -- 172.21.15.120:0/4055643735 learned_addr learned my addr 172.21.15.120:0/4055643735 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:32.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.929+0000 7fa9c91fc640 1 --2- 172.21.15.120:0/4055643735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa3fc03c0 0x555aa3fdb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:32.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.929+0000 7fa9c89fb640 1 --2- 172.21.15.120:0/4055643735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa3fc0ce0 0x555aa3fe6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:32.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.929+0000 7fa9c99fd640 1 -- 172.21.15.120:0/4055643735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa3fc0ce0 msgr2=0x555aa3fe6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:32.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.929+0000 7fa9c99fd640 1 --2- 172.21.15.120:0/4055643735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa3fc0ce0 0x555aa3fe6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.929+0000 7fa9c99fd640 1 -- 172.21.15.120:0/4055643735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa3fc03c0 msgr2=0x555aa3fdb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:32.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.929+0000 7fa9c99fd640 1 --2- 172.21.15.120:0/4055643735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa3fc03c0 0x555aa3fdb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.929+0000 7fa9c99fd640 1 -- 172.21.15.120:0/4055643735 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aa3fcac90 con 0x555aa3fcfa90 2024-09-19T09:13:32.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.929+0000 7fa9c89fb640 1 --2- 172.21.15.120:0/4055643735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa3fc0ce0 0x555aa3fe6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:13:32.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.929+0000 7fa9c91fc640 1 --2- 172.21.15.120:0/4055643735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa3fc03c0 0x555aa3fdb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:13:32.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.929+0000 7fa9c99fd640 1 --2- 172.21.15.120:0/4055643735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555aa3fcfa90 0x555aa3fe73e0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fa9c00140b0 tx=0x7fa9c0011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1ec4cd7d9bd68a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:32.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.930+0000 7fa9aeffd640 1 -- 172.21.15.120:0/4055643735 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0022040 con 0x555aa3fcfa90 2024-09-19T09:13:32.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.930+0000 7fa9aeffd640 1 -- 172.21.15.120:0/4055643735 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9c002c070 con 0x555aa3fcfa90 2024-09-19T09:13:32.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.930+0000 7fa9aeffd640 1 -- 172.21.15.120:0/4055643735 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0004c40 con 0x555aa3fcfa90 2024-09-19T09:13:32.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.930+0000 7fa9cd839740 1 -- 172.21.15.120:0/4055643735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555aa3fcfa90 msgr2=0x555aa3fe73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:32.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.930+0000 7fa9cd839740 1 --2- 172.21.15.120:0/4055643735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555aa3fcfa90 0x555aa3fe73e0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fa9c00140b0 tx=0x7fa9c0011f90 comp rx=0 tx=0).stop 2024-09-19T09:13:32.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.930+0000 7fa9cd839740 1 -- 172.21.15.120:0/4055643735 shutdown_connections 2024-09-19T09:13:32.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.930+0000 7fa9cd839740 1 --2- 172.21.15.120:0/4055643735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555aa3fcfa90 0x555aa3fe73e0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.930+0000 7fa9cd839740 1 --2- 172.21.15.120:0/4055643735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa3fc0ce0 0x555aa3fe6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.930+0000 7fa9cd839740 1 --2- 172.21.15.120:0/4055643735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa3fc03c0 0x555aa3fdb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.930+0000 7fa9cd839740 1 -- 172.21.15.120:0/4055643735 >> 172.21.15.120:0/4055643735 conn(0x555aa3fbca30 msgr2=0x555aa3fc7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:32.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.930+0000 7fa9cd839740 1 -- 172.21.15.120:0/4055643735 shutdown_connections 2024-09-19T09:13:32.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.930+0000 7fa9cd839740 1 -- 172.21.15.120:0/4055643735 wait complete. 2024-09-19T09:13:32.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.931+0000 7fa9cd839740 1 -- start start 2024-09-19T09:13:32.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.931+0000 7fa9cd839740 1 Processor -- start 2024-09-19T09:13:32.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.931+0000 7fa9cd839740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa3fc03c0 0x555aa3fc7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:32.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.931+0000 7fa9cd839740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555aa3fc0ce0 0x555aa3fd79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:32.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.931+0000 7fa9cd839740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa3fcfa90 0x555aa3fd7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:32.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.931+0000 7fa9cd839740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41ac820 con 0x555aa3fc0ce0 2024-09-19T09:13:32.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.931+0000 7fa9cd839740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41ac150 con 0x555aa3fcfa90 2024-09-19T09:13:32.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.931+0000 7fa9cd839740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41a9370 con 0x555aa3fc03c0 2024-09-19T09:13:32.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.931+0000 7fa9cd839740 10 client.0 _ll_register_callbacks cb 0x7fa9c0014cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:32.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.931+0000 7fa9c89fb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555aa3fc0ce0 0x555aa3fd79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:32.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.931+0000 7fa9c89fb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555aa3fc0ce0 0x555aa3fd79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51420/0 (socket says 172.21.15.120:51420) 2024-09-19T09:13:32.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.931+0000 7fa9c89fb640 1 -- 172.21.15.120:0/103294778 learned_addr learned my addr 172.21.15.120:0/103294778 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:32.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.931+0000 7fa9c99fd640 1 --2- 172.21.15.120:0/103294778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa3fcfa90 0x555aa3fd7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:32.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.932+0000 7fa9c91fc640 1 --2- 172.21.15.120:0/103294778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa3fc03c0 0x555aa3fc7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:32.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.932+0000 7fa9c89fb640 1 -- 172.21.15.120:0/103294778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa3fc03c0 msgr2=0x555aa3fc7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:32.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.932+0000 7fa9c89fb640 1 --2- 172.21.15.120:0/103294778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa3fc03c0 0x555aa3fc7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.932+0000 7fa9c89fb640 1 -- 172.21.15.120:0/103294778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa3fcfa90 msgr2=0x555aa3fd7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:32.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.932+0000 7fa9c89fb640 1 --2- 172.21.15.120:0/103294778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa3fcfa90 0x555aa3fd7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.932+0000 7fa9c89fb640 1 -- 172.21.15.120:0/103294778 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aa3fd84e0 con 0x555aa3fc0ce0 2024-09-19T09:13:32.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.932+0000 7fa9c89fb640 1 --2- 172.21.15.120:0/103294778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555aa3fc0ce0 0x555aa3fd79b0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fa9b40b8f10 tx=0x7fa9b40bc310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9b5ebb1a662568 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:32.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.932+0000 7fa9acff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:32.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.932+0000 7fa9acff9640 1 -- 172.21.15.120:0/103294778 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4044ea0 con 0x555aa3fc0ce0 2024-09-19T09:13:32.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.932+0000 7fa9acff9640 1 -- 172.21.15.120:0/103294778 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa9b40edb30 con 0x555aa3fc0ce0 2024-09-19T09:13:32.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.932+0000 7fa9cd839740 1 -- 172.21.15.120:0/103294778 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9c002c070 con 0x555aa3fc0ce0 2024-09-19T09:13:32.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.932+0000 7fa9acff9640 1 -- 172.21.15.120:0/103294778 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa9b40bcc40 con 0x555aa3fc0ce0 2024-09-19T09:13:32.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.932+0000 7fa9cd839740 10 client.19329 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:32.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.932+0000 7fa9cd839740 1 -- 172.21.15.120:0/103294778 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555aa404e660 con 0x555aa3fc0ce0 2024-09-19T09:13:32.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.932+0000 7fa9acff9640 1 -- 172.21.15.120:0/103294778 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9c002c070 con 0x555aa3fc0ce0 2024-09-19T09:13:32.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.933+0000 7fa9acff9640 1 -- 172.21.15.120:0/103294778 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa9b40b8bb0 con 0x555aa3fc0ce0 2024-09-19T09:13:32.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.933+0000 7fa9cd839740 10 client.19329 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:32.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.933+0000 7fa9cd839740 10 client.19329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:13:32.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.933+0000 7fa9cd839740 20 client.19329 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:32.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.933+0000 7fa9cd839740 1 --2- 172.21.15.120:0/103294778 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555aa3fdf450 0x555aa4050dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:32.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.933+0000 7fa9cd839740 4 client.19329 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:13:32.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.933+0000 7fa9cd839740 1 -- 172.21.15.120:0/103294778 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555aa3f2dd60 con 0x555aa3fdf450 2024-09-19T09:13:32.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.933+0000 7fa9c91fc640 1 --2- 172.21.15.120:0/103294778 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555aa3fdf450 0x555aa4050dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:32.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.933+0000 7fa9c91fc640 1 --2- 172.21.15.120:0/103294778 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555aa3fdf450 0x555aa4050dd0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e039770eb88c8dd3 server_cookie=b428fbaa0ca38321 in_seq=0 out_seq=0 2024-09-19T09:13:32.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.933+0000 7fa9acff9640 10 client.19329 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:32.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.933+0000 7fa9acff9640 1 -- 172.21.15.120:0/103294778 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555aa3f2dd60 con 0x555aa3fdf450 2024-09-19T09:13:32.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.933+0000 7fa9acff9640 10 client.19329 handle_command_reply: tid=0 2024-09-19T09:13:32.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.933+0000 7fa9cd839740 1 client.19329 shutdown 2024-09-19T09:13:32.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.934+0000 7fa9cd839740 1 -- 172.21.15.120:0/103294778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555aa3fc0ce0 msgr2=0x555aa3fd79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:32.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.934+0000 7fa9cd839740 1 --2- 172.21.15.120:0/103294778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555aa3fc0ce0 0x555aa3fd79b0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fa9b40b8f10 tx=0x7fa9b40bc310 comp rx=0 tx=0).stop 2024-09-19T09:13:32.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.934+0000 7fa9cd839740 1 -- 172.21.15.120:0/103294778 shutdown_connections 2024-09-19T09:13:32.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.934+0000 7fa9cd839740 1 --2- 172.21.15.120:0/103294778 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555aa3fdf450 0x555aa4050dd0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.934+0000 7fa9cd839740 1 --2- 172.21.15.120:0/103294778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa3fcfa90 0x555aa3fd7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.934+0000 7fa9cd839740 1 --2- 172.21.15.120:0/103294778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555aa3fc0ce0 0x555aa3fd79b0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.934+0000 7fa9cd839740 1 --2- 172.21.15.120:0/103294778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa3fc03c0 0x555aa3fc7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.934+0000 7fa9acff9640 0 client.19329 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:32.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.934+0000 7fa9cd839740 1 -- 172.21.15.120:0/103294778 >> 172.21.15.120:0/103294778 conn(0x555aa3fbca30 msgr2=0x555aa3fc1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:32.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.934+0000 7fa9cd839740 1 -- 172.21.15.120:0/103294778 shutdown_connections 2024-09-19T09:13:32.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.934+0000 7fa9cd839740 1 -- 172.21.15.120:0/103294778 wait complete. 2024-09-19T09:13:32.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.934+0000 7fa9cd839740 20 client.19329 trim_cache size 0 max 16384 2024-09-19T09:13:32.966 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:13:32.966 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:13:32.966 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-19T09:13:32.967 INFO:teuthology.orchestra.run.smithi120.stdout: "d3d6728f-1d1f-49ef-831f-19392a6e5c2f": { 2024-09-19T09:13:32.967 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-19T09:13:32.967 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "d3d6728f-1d1f-49ef-831f-19392a6e5c2f", 2024-09-19T09:13:32.967 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-19T09:13:32.967 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:13:32.967 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:13:32.968 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:13:32.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.935+0000 7fa9cb460640 1 -- 172.21.15.120:0/2861173342 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9b00a3240 msgr2=0x7fa9b00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:32.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.935+0000 7fa9cb460640 1 --2- 172.21.15.120:0/2861173342 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9b00a3240 0x7fa9b00a5700 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0011c40 tx=0x7fa9c001c000 comp rx=0 tx=0).stop 2024-09-19T09:13:32.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.935+0000 7fa9cb460640 1 -- 172.21.15.120:0/2861173342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9c4102790 msgr2=0x7fa9c4198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:32.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.935+0000 7fa9cb460640 1 --2- 172.21.15.120:0/2861173342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9c4102790 0x7fa9c4198990 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4005940 tx=0x7fa9b4004270 comp rx=0 tx=0).stop 2024-09-19T09:13:32.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.935+0000 7fa9cb460640 1 -- 172.21.15.120:0/2861173342 shutdown_connections 2024-09-19T09:13:32.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.935+0000 7fa9cb460640 1 --2- 172.21.15.120:0/2861173342 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9b00a3240 0x7fa9b00a5700 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.935+0000 7fa9cb460640 1 --2- 172.21.15.120:0/2861173342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9c4106f10 0x7fa9c419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.935+0000 7fa9cb460640 1 --2- 172.21.15.120:0/2861173342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9c4102790 0x7fa9c4198990 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.935+0000 7fa9cb460640 1 --2- 172.21.15.120:0/2861173342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9c4101de0 0x7fa9c4198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:32.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.935+0000 7fa9cb460640 1 -- 172.21.15.120:0/2861173342 >> 172.21.15.120:0/2861173342 conn(0x7fa9c40fd880 msgr2=0x7fa9c40feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:32.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.935+0000 7fa9cb460640 1 -- 172.21.15.120:0/2861173342 shutdown_connections 2024-09-19T09:13:32.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:32.935+0000 7fa9cb460640 1 -- 172.21.15.120:0/2861173342 wait complete. 2024-09-19T09:13:33.070 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.28 MB/sec execute 448 sec latency 16377.221 ms 2024-09-19T09:13:33.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:32 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:13:33.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:32 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:13:33.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:32 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:13:33.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:32 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:13:33.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:32 smithi196 ceph-mon[35150]: pgmap v663: 97 pgs: 97 active+clean; 29 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 130 KiB/s wr, 60 op/s 2024-09-19T09:13:33.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:32 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:13:33.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:32 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:13:33.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:32 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:13:33.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:32 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:13:33.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:32 smithi153 ceph-mon[35071]: pgmap v663: 97 pgs: 97 active+clean; 29 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 130 KiB/s wr, 60 op/s 2024-09-19T09:13:33.663 INFO:tasks.cephfs.filesystem:scrub status for tag:d3d6728f-1d1f-49ef-831f-19392a6e5c2f - {'path': '/', 'tag': 'd3d6728f-1d1f-49ef-831f-19392a6e5c2f', 'options': 'recursive,force'} 2024-09-19T09:13:34.070 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.26 MB/sec execute 449 sec latency 17377.304 ms 2024-09-19T09:13:34.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:33 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:13:34.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:33 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:13:34.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:33 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:13:35.070 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.24 MB/sec execute 450 sec latency 18377.352 ms 2024-09-19T09:13:35.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:34 smithi196 ceph-mon[35150]: pgmap v664: 97 pgs: 97 active+clean; 29 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 54 op/s 2024-09-19T09:13:35.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:34 smithi153 ceph-mon[35071]: pgmap v664: 97 pgs: 97 active+clean; 29 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 54 op/s 2024-09-19T09:13:35.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:34 smithi120 ceph-mon[31150]: pgmap v664: 97 pgs: 97 active+clean; 29 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 54 op/s 2024-09-19T09:13:36.071 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.22 MB/sec execute 451 sec latency 19377.438 ms 2024-09-19T09:13:37.071 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.20 MB/sec execute 452 sec latency 20377.522 ms 2024-09-19T09:13:37.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:36 smithi196 ceph-mon[35150]: pgmap v665: 97 pgs: 97 active+clean; 29 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 KiB/s wr, 52 op/s 2024-09-19T09:13:37.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:36 smithi153 ceph-mon[35071]: pgmap v665: 97 pgs: 97 active+clean; 29 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 KiB/s wr, 52 op/s 2024-09-19T09:13:37.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:36 smithi120 ceph-mon[31150]: pgmap v665: 97 pgs: 97 active+clean; 29 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 KiB/s wr, 52 op/s 2024-09-19T09:13:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.880+0000 7fa0df78d640 1 -- 172.21.15.120:0/2632506555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0d81003c0 msgr2=0x7fa0d81027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.880+0000 7fa0df78d640 1 --2- 172.21.15.120:0/2632506555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0d81003c0 0x7fa0d81027d0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4009900 tx=0x7fa0d402a410 comp rx=0 tx=0).stop 2024-09-19T09:13:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.880+0000 7fa0df78d640 1 -- 172.21.15.120:0/2632506555 shutdown_connections 2024-09-19T09:13:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.881+0000 7fa0df78d640 1 --2- 172.21.15.120:0/2632506555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0d81056c0 0x7fa0d8105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.881+0000 7fa0df78d640 1 --2- 172.21.15.120:0/2632506555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0d8102d10 0x7fa0d8105180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.881+0000 7fa0df78d640 1 --2- 172.21.15.120:0/2632506555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0d81003c0 0x7fa0d81027d0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.881+0000 7fa0df78d640 1 -- 172.21.15.120:0/2632506555 >> 172.21.15.120:0/2632506555 conn(0x7fa0d80fc1b0 msgr2=0x7fa0d80fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.881+0000 7fa0df78d640 1 -- 172.21.15.120:0/2632506555 shutdown_connections 2024-09-19T09:13:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.881+0000 7fa0df78d640 1 -- 172.21.15.120:0/2632506555 wait complete. 2024-09-19T09:13:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.881+0000 7fa0df78d640 1 Processor -- start 2024-09-19T09:13:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.881+0000 7fa0df78d640 1 -- start start 2024-09-19T09:13:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.881+0000 7fa0df78d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0d81003c0 0x7fa0d819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.881+0000 7fa0df78d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0d8102d10 0x7fa0d819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.881+0000 7fa0df78d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0d81056c0 0x7fa0d81a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.881+0000 7fa0df78d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d8113e00 con 0x7fa0d8102d10 2024-09-19T09:13:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.881+0000 7fa0df78d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d8113c80 con 0x7fa0d81056c0 2024-09-19T09:13:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.881+0000 7fa0df78d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d8113f80 con 0x7fa0d81003c0 2024-09-19T09:13:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.882+0000 7fa0dcd28640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0d8102d10 0x7fa0d819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.882+0000 7fa0dcd28640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0d8102d10 0x7fa0d819e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51448/0 (socket says 172.21.15.120:51448) 2024-09-19T09:13:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.882+0000 7fa0dcd28640 1 -- 172.21.15.120:0/2919833489 learned_addr learned my addr 172.21.15.120:0/2919833489 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.882+0000 7fa0dd529640 1 --2- 172.21.15.120:0/2919833489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0d81003c0 0x7fa0d819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.882+0000 7fa0ddd2a640 1 --2- 172.21.15.120:0/2919833489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0d81056c0 0x7fa0d81a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.882+0000 7fa0dcd28640 1 -- 172.21.15.120:0/2919833489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0d81003c0 msgr2=0x7fa0d819e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.882+0000 7fa0dcd28640 1 --2- 172.21.15.120:0/2919833489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0d81003c0 0x7fa0d819e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.882+0000 7fa0dcd28640 1 -- 172.21.15.120:0/2919833489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0d81056c0 msgr2=0x7fa0d81a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.882+0000 7fa0dcd28640 1 --2- 172.21.15.120:0/2919833489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0d81056c0 0x7fa0d81a5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.882+0000 7fa0dcd28640 1 -- 172.21.15.120:0/2919833489 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d81a6000 con 0x7fa0d8102d10 2024-09-19T09:13:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.882+0000 7fa0ddd2a640 1 --2- 172.21.15.120:0/2919833489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0d81056c0 0x7fa0d81a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:13:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.882+0000 7fa0dcd28640 1 --2- 172.21.15.120:0/2919833489 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0d8102d10 0x7fa0d819e920 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fa0c800d950 tx=0x7fa0c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.882+0000 7fa0ce7fc640 1 -- 172.21.15.120:0/2919833489 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8017070 con 0x7fa0d8102d10 2024-09-19T09:13:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.882+0000 7fa0df78d640 1 -- 172.21.15.120:0/2919833489 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0d81a62f0 con 0x7fa0d8102d10 2024-09-19T09:13:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.882+0000 7fa0ce7fc640 1 -- 172.21.15.120:0/2919833489 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0c8009c30 con 0x7fa0d8102d10 2024-09-19T09:13:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.882+0000 7fa0ce7fc640 1 -- 172.21.15.120:0/2919833489 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c800f6b0 con 0x7fa0d8102d10 2024-09-19T09:13:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.882+0000 7fa0df78d640 1 -- 172.21.15.120:0/2919833489 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0d81a6890 con 0x7fa0d8102d10 2024-09-19T09:13:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.885+0000 7fa0ce7fc640 1 -- 172.21.15.120:0/2919833489 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa0c80109d0 con 0x7fa0d8102d10 2024-09-19T09:13:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.885+0000 7fa0e2602740 1 Processor -- start 2024-09-19T09:13:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.885+0000 7fa0e2602740 1 -- start start 2024-09-19T09:13:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.885+0000 7fa0e2602740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561453f39a0 0x5561453f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.885+0000 7fa0e2602740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561453f42e0 0x5561453f8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.885+0000 7fa0e2602740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561453f9370 0x5561453fb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.885+0000 7fa0e2602740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5561453572f0 con 0x5561453f42e0 2024-09-19T09:13:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.885+0000 7fa0e2602740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5561452ff270 con 0x5561453f9370 2024-09-19T09:13:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.885+0000 7fa0e2602740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5561453532b0 con 0x5561453f39a0 2024-09-19T09:13:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.885+0000 7fa0dd529640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561453f42e0 0x5561453f8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.885+0000 7fa0dd529640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561453f42e0 0x5561453f8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51464/0 (socket says 172.21.15.120:51464) 2024-09-19T09:13:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.885+0000 7fa0dd529640 1 -- 172.21.15.120:0/560616525 learned_addr learned my addr 172.21.15.120:0/560616525 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.886+0000 7fa0dcd28640 1 --2- 172.21.15.120:0/560616525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561453f9370 0x5561453fb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.886+0000 7fa0ddd2a640 1 --2- 172.21.15.120:0/560616525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561453f39a0 0x5561453f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.886+0000 7fa0dd529640 1 -- 172.21.15.120:0/560616525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561453f39a0 msgr2=0x5561453f3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.886+0000 7fa0dd529640 1 --2- 172.21.15.120:0/560616525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561453f39a0 0x5561453f3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.886+0000 7fa0dd529640 1 -- 172.21.15.120:0/560616525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561453f9370 msgr2=0x5561453fb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.886+0000 7fa0dd529640 1 --2- 172.21.15.120:0/560616525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561453f9370 0x5561453fb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.886+0000 7fa0dd529640 1 -- 172.21.15.120:0/560616525 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561452fe340 con 0x5561453f42e0 2024-09-19T09:13:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.886+0000 7fa0ce7fc640 1 --2- 172.21.15.120:0/2919833489 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa0c40a32f0 0x7fa0c40a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.886+0000 7fa0ddd2a640 1 --2- 172.21.15.120:0/2919833489 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa0c40a32f0 0x7fa0c40a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.886+0000 7fa0dd529640 1 --2- 172.21.15.120:0/560616525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561453f42e0 0x5561453f8da0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fa0d402a930 tx=0x7fa0d4032d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69760df66546e12d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.886+0000 7fa0ce7fc640 1 -- 172.21.15.120:0/2919833489 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa0c80cf630 con 0x7fa0d8102d10 2024-09-19T09:13:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.886+0000 7fa0c37fe640 1 -- 172.21.15.120:0/560616525 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d403f070 con 0x5561453f42e0 2024-09-19T09:13:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.886+0000 7fa0c37fe640 1 -- 172.21.15.120:0/560616525 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0d40318b0 con 0x5561453f42e0 2024-09-19T09:13:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.886+0000 7fa0c37fe640 1 -- 172.21.15.120:0/560616525 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4035070 con 0x5561453f42e0 2024-09-19T09:13:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.887+0000 7fa0e2602740 1 -- 172.21.15.120:0/560616525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561453f42e0 msgr2=0x5561453f8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.887+0000 7fa0e2602740 1 --2- 172.21.15.120:0/560616525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561453f42e0 0x5561453f8da0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fa0d402a930 tx=0x7fa0d4032d60 comp rx=0 tx=0).stop 2024-09-19T09:13:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.887+0000 7fa0ddd2a640 1 --2- 172.21.15.120:0/2919833489 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa0c40a32f0 0x7fa0c40a57b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0005fd0 tx=0x7fa0d00083f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.887+0000 7fa0e2602740 1 -- 172.21.15.120:0/560616525 shutdown_connections 2024-09-19T09:13:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.887+0000 7fa0e2602740 1 --2- 172.21.15.120:0/560616525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561453f9370 0x5561453fb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.887+0000 7fa0e2602740 1 --2- 172.21.15.120:0/560616525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561453f42e0 0x5561453f8da0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.887+0000 7fa0e2602740 1 --2- 172.21.15.120:0/560616525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561453f39a0 0x5561453f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.887+0000 7fa0e2602740 1 -- 172.21.15.120:0/560616525 >> 172.21.15.120:0/560616525 conn(0x5561453ef010 msgr2=0x5561453ef410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.887+0000 7fa0e2602740 1 -- 172.21.15.120:0/560616525 shutdown_connections 2024-09-19T09:13:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.887+0000 7fa0e2602740 1 -- 172.21.15.120:0/560616525 wait complete. 2024-09-19T09:13:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.887+0000 7fa0e2602740 1 -- start start 2024-09-19T09:13:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.888+0000 7fa0e2602740 1 Processor -- start 2024-09-19T09:13:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.888+0000 7fa0e2602740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561453f39a0 0x5561453fdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.888+0000 7fa0e2602740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561453f42e0 0x5561453fe170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.888+0000 7fa0e2602740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561453f9370 0x5561454051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.888+0000 7fa0e2602740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5561453fc2e0 con 0x5561453f39a0 2024-09-19T09:13:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.888+0000 7fa0e2602740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5561452e0290 con 0x5561453f42e0 2024-09-19T09:13:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.888+0000 7fa0e2602740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5561453fc460 con 0x5561453f9370 2024-09-19T09:13:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.888+0000 7fa0e2602740 10 client.0 _ll_register_callbacks cb 0x556145311140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.888+0000 7fa0dd529640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561453f39a0 0x5561453fdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.888+0000 7fa0dd529640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561453f39a0 0x5561453fdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51480/0 (socket says 172.21.15.120:51480) 2024-09-19T09:13:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.888+0000 7fa0dd529640 1 -- 172.21.15.120:0/3198192118 learned_addr learned my addr 172.21.15.120:0/3198192118 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.888+0000 7fa0dcd28640 1 --2- 172.21.15.120:0/3198192118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561453f42e0 0x5561453fe170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.888+0000 7fa0ddd2a640 1 --2- 172.21.15.120:0/3198192118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561453f9370 0x5561454051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.888+0000 7fa0dd529640 1 -- 172.21.15.120:0/3198192118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561453f9370 msgr2=0x5561454051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.888+0000 7fa0dd529640 1 --2- 172.21.15.120:0/3198192118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561453f9370 0x5561454051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.888+0000 7fa0dd529640 1 -- 172.21.15.120:0/3198192118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561453f42e0 msgr2=0x5561453fe170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.888+0000 7fa0dd529640 1 --2- 172.21.15.120:0/3198192118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561453f42e0 0x5561453fe170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.888+0000 7fa0dd529640 1 -- 172.21.15.120:0/3198192118 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55614506a210 con 0x5561453f39a0 2024-09-19T09:13:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.888+0000 7fa0ddd2a640 1 --2- 172.21.15.120:0/3198192118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561453f9370 0x5561454051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:13:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.888+0000 7fa0dcd28640 1 --2- 172.21.15.120:0/3198192118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561453f42e0 0x5561453fe170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:13:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.889+0000 7fa0dd529640 1 --2- 172.21.15.120:0/3198192118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561453f39a0 0x5561453fdc30 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40314e0 tx=0x7fa0d4003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f4a4df131787b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.889+0000 7fa0c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.889+0000 7fa0c0ff9640 1 -- 172.21.15.120:0/3198192118 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d404a020 con 0x5561453f39a0 2024-09-19T09:13:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.889+0000 7fa0c0ff9640 1 -- 172.21.15.120:0/3198192118 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa0d4003c20 con 0x5561453f39a0 2024-09-19T09:13:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.889+0000 7fa0e2602740 1 -- 172.21.15.120:0/3198192118 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0d40318b0 con 0x5561453f39a0 2024-09-19T09:13:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.889+0000 7fa0c0ff9640 1 -- 172.21.15.120:0/3198192118 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa0d403f070 con 0x5561453f39a0 2024-09-19T09:13:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.889+0000 7fa0e2602740 10 client.19341 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.889+0000 7fa0e2602740 1 -- 172.21.15.120:0/3198192118 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556145136620 con 0x5561453f39a0 2024-09-19T09:13:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.889+0000 7fa0c0ff9640 1 -- 172.21.15.120:0/3198192118 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0d40318b0 con 0x5561453f39a0 2024-09-19T09:13:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.889+0000 7fa0c0ff9640 1 -- 172.21.15.120:0/3198192118 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa0d403bce0 con 0x5561453f39a0 2024-09-19T09:13:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.889+0000 7fa0e2602740 10 client.19341 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.889+0000 7fa0e2602740 10 client.19341 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:13:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.889+0000 7fa0e2602740 20 client.19341 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.889+0000 7fa0e2602740 1 --2- 172.21.15.120:0/3198192118 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5561454094b0 0x55614540b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.889+0000 7fa0e2602740 4 client.19341 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:13:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.889+0000 7fa0e2602740 1 -- 172.21.15.120:0/3198192118 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556145364a20 con 0x5561454094b0 2024-09-19T09:13:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.890+0000 7fa0dcd28640 1 --2- 172.21.15.120:0/3198192118 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5561454094b0 0x55614540b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.890+0000 7fa0dcd28640 1 --2- 172.21.15.120:0/3198192118 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5561454094b0 0x55614540b8a0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ad5c0a0c35c7f21 server_cookie=cd02f004c3af1194 in_seq=0 out_seq=0 2024-09-19T09:13:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.890+0000 7fa0c0ff9640 10 client.19341 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.891+0000 7fa0c0ff9640 1 -- 172.21.15.120:0/3198192118 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556145364a20 con 0x5561454094b0 2024-09-19T09:13:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.891+0000 7fa0c0ff9640 10 client.19341 handle_command_reply: tid=0 2024-09-19T09:13:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.891+0000 7fa0e2602740 1 client.19341 shutdown 2024-09-19T09:13:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.891+0000 7fa0e2602740 1 -- 172.21.15.120:0/3198192118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561453f39a0 msgr2=0x5561453fdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.891+0000 7fa0e2602740 1 --2- 172.21.15.120:0/3198192118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561453f39a0 0x5561453fdc30 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40314e0 tx=0x7fa0d4003820 comp rx=0 tx=0).stop 2024-09-19T09:13:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.891+0000 7fa0e2602740 1 -- 172.21.15.120:0/3198192118 shutdown_connections 2024-09-19T09:13:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.891+0000 7fa0e2602740 1 --2- 172.21.15.120:0/3198192118 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5561454094b0 0x55614540b8a0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.891+0000 7fa0e2602740 1 --2- 172.21.15.120:0/3198192118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561453f9370 0x5561454051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.891+0000 7fa0e2602740 1 --2- 172.21.15.120:0/3198192118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561453f42e0 0x5561453fe170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.891+0000 7fa0e2602740 1 --2- 172.21.15.120:0/3198192118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561453f39a0 0x5561453fdc30 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.891+0000 7fa0c0ff9640 0 client.19341 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.891+0000 7fa0e2602740 1 -- 172.21.15.120:0/3198192118 >> 172.21.15.120:0/3198192118 conn(0x5561453ef010 msgr2=0x5561453f0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.891+0000 7fa0e2602740 1 -- 172.21.15.120:0/3198192118 shutdown_connections 2024-09-19T09:13:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.891+0000 7fa0e2602740 1 -- 172.21.15.120:0/3198192118 wait complete. 2024-09-19T09:13:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.891+0000 7fa0e2602740 20 client.19341 trim_cache size 0 max 16384 2024-09-19T09:13:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.906+0000 7fa0e2602740 1 Processor -- start 2024-09-19T09:13:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.906+0000 7fa0e2602740 1 -- start start 2024-09-19T09:13:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.906+0000 7fa0e2602740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561453f39a0 0x5561453ee0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.906+0000 7fa0e2602740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561453f42e0 0x556145405170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.906+0000 7fa0e2602740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561453f9370 0x5561453f0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.906+0000 7fa0e2602740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d81af590 con 0x5561453f42e0 2024-09-19T09:13:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.906+0000 7fa0e2602740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d81b10d0 con 0x5561453f9370 2024-09-19T09:13:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.906+0000 7fa0e2602740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d81afc60 con 0x5561453f39a0 2024-09-19T09:13:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.906+0000 7fa0dcd28640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561453f42e0 0x556145405170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.906+0000 7fa0dcd28640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561453f42e0 0x556145405170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51482/0 (socket says 172.21.15.120:51482) 2024-09-19T09:13:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.906+0000 7fa0dcd28640 1 -- 172.21.15.120:0/1542243619 learned_addr learned my addr 172.21.15.120:0/1542243619 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.906+0000 7fa0dd529640 1 --2- 172.21.15.120:0/1542243619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561453f39a0 0x5561453ee0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.906+0000 7fa0ddd2a640 1 --2- 172.21.15.120:0/1542243619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561453f9370 0x5561453f0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.906+0000 7fa0dcd28640 1 -- 172.21.15.120:0/1542243619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561453f39a0 msgr2=0x5561453ee0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.906+0000 7fa0dcd28640 1 --2- 172.21.15.120:0/1542243619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561453f39a0 0x5561453ee0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.906+0000 7fa0dcd28640 1 -- 172.21.15.120:0/1542243619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561453f9370 msgr2=0x5561453f0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.906+0000 7fa0dcd28640 1 --2- 172.21.15.120:0/1542243619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561453f9370 0x5561453f0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.906+0000 7fa0dcd28640 1 -- 172.21.15.120:0/1542243619 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55614540bc10 con 0x5561453f42e0 2024-09-19T09:13:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.906+0000 7fa0dcd28640 1 --2- 172.21.15.120:0/1542243619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561453f42e0 0x556145405170 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fa0c809b950 tx=0x7fa0c80d5110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4321283b5f6b2c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.907+0000 7fa0c2ffd640 1 -- 172.21.15.120:0/1542243619 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8020040 con 0x5561453f42e0 2024-09-19T09:13:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.907+0000 7fa0c2ffd640 1 -- 172.21.15.120:0/1542243619 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0c809c350 con 0x5561453f42e0 2024-09-19T09:13:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.907+0000 7fa0c2ffd640 1 -- 172.21.15.120:0/1542243619 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8099fe0 con 0x5561453f42e0 2024-09-19T09:13:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.907+0000 7fa0e2602740 1 -- 172.21.15.120:0/1542243619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561453f42e0 msgr2=0x556145405170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.907+0000 7fa0e2602740 1 --2- 172.21.15.120:0/1542243619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561453f42e0 0x556145405170 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fa0c809b950 tx=0x7fa0c80d5110 comp rx=0 tx=0).stop 2024-09-19T09:13:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.907+0000 7fa0e2602740 1 -- 172.21.15.120:0/1542243619 shutdown_connections 2024-09-19T09:13:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.907+0000 7fa0e2602740 1 --2- 172.21.15.120:0/1542243619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561453f9370 0x5561453f0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.907+0000 7fa0e2602740 1 --2- 172.21.15.120:0/1542243619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561453f42e0 0x556145405170 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.907+0000 7fa0e2602740 1 --2- 172.21.15.120:0/1542243619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561453f39a0 0x5561453ee0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.907+0000 7fa0e2602740 1 -- 172.21.15.120:0/1542243619 >> 172.21.15.120:0/1542243619 conn(0x5561453ef010 msgr2=0x5561453ed9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.907+0000 7fa0e2602740 1 -- 172.21.15.120:0/1542243619 shutdown_connections 2024-09-19T09:13:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.907+0000 7fa0e2602740 1 -- 172.21.15.120:0/1542243619 wait complete. 2024-09-19T09:13:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.908+0000 7fa0e2602740 1 -- start start 2024-09-19T09:13:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.908+0000 7fa0e2602740 1 Processor -- start 2024-09-19T09:13:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.908+0000 7fa0e2602740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561453f39a0 0x556145402910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.908+0000 7fa0e2602740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561453f42e0 0x556145402e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.908+0000 7fa0e2602740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561453f9370 0x556145403e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.908+0000 7fa0e2602740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d81b0a00 con 0x5561453f42e0 2024-09-19T09:13:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.908+0000 7fa0e2602740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d81b0330 con 0x5561453f39a0 2024-09-19T09:13:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.908+0000 7fa0e2602740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d81ad550 con 0x5561453f9370 2024-09-19T09:13:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.908+0000 7fa0e2602740 10 client.0 _ll_register_callbacks cb 0x7fa0c80d5060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.908+0000 7fa0dcd28640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561453f42e0 0x556145402e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.908+0000 7fa0dcd28640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561453f42e0 0x556145402e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51484/0 (socket says 172.21.15.120:51484) 2024-09-19T09:13:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.908+0000 7fa0dcd28640 1 -- 172.21.15.120:0/271696260 learned_addr learned my addr 172.21.15.120:0/271696260 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.908+0000 7fa0dd529640 1 --2- 172.21.15.120:0/271696260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561453f39a0 0x556145402910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.908+0000 7fa0ddd2a640 1 --2- 172.21.15.120:0/271696260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561453f9370 0x556145403e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.909+0000 7fa0dcd28640 1 -- 172.21.15.120:0/271696260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561453f9370 msgr2=0x556145403e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.909+0000 7fa0dcd28640 1 --2- 172.21.15.120:0/271696260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561453f9370 0x556145403e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.909+0000 7fa0dcd28640 1 -- 172.21.15.120:0/271696260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561453f39a0 msgr2=0x556145402910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.909+0000 7fa0dcd28640 1 --2- 172.21.15.120:0/271696260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561453f39a0 0x556145402910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.909+0000 7fa0dcd28640 1 -- 172.21.15.120:0/271696260 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556145404470 con 0x5561453f42e0 2024-09-19T09:13:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.909+0000 7fa0dcd28640 1 --2- 172.21.15.120:0/271696260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561453f42e0 0x556145402e50 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8010fd0 tx=0x7fa0c8005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99989a317d064fe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.909+0000 7fa0c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.909+0000 7fa0c0ff9640 1 -- 172.21.15.120:0/271696260 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c80b1070 con 0x5561453f42e0 2024-09-19T09:13:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.909+0000 7fa0c0ff9640 1 -- 172.21.15.120:0/271696260 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa0c80982c0 con 0x5561453f42e0 2024-09-19T09:13:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.909+0000 7fa0e2602740 1 -- 172.21.15.120:0/271696260 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0c809c350 con 0x5561453f42e0 2024-09-19T09:13:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.909+0000 7fa0c0ff9640 1 -- 172.21.15.120:0/271696260 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa0c8020070 con 0x5561453f42e0 2024-09-19T09:13:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.909+0000 7fa0e2602740 10 client.19347 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.909+0000 7fa0e2602740 1 -- 172.21.15.120:0/271696260 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556145474190 con 0x5561453f42e0 2024-09-19T09:13:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.909+0000 7fa0c0ff9640 1 -- 172.21.15.120:0/271696260 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0c809c350 con 0x5561453f42e0 2024-09-19T09:13:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.909+0000 7fa0c0ff9640 1 -- 172.21.15.120:0/271696260 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa0c80d1ac0 con 0x5561453f42e0 2024-09-19T09:13:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.909+0000 7fa0e2602740 10 client.19347 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.909+0000 7fa0e2602740 10 client.19347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:13:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.909+0000 7fa0e2602740 20 client.19347 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.909+0000 7fa0e2602740 1 --2- 172.21.15.120:0/271696260 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556145474b40 0x556145476f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.909+0000 7fa0e2602740 4 client.19347 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:13:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.909+0000 7fa0e2602740 1 -- 172.21.15.120:0/271696260 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55614535ed70 con 0x556145474b40 2024-09-19T09:13:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.910+0000 7fa0dd529640 1 --2- 172.21.15.120:0/271696260 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556145474b40 0x556145476f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.910+0000 7fa0c0ff9640 10 client.19347 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.910+0000 7fa0dd529640 1 --2- 172.21.15.120:0/271696260 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556145474b40 0x556145476f60 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69269dd6a7f0179b server_cookie=276bd1ff91a4f6e in_seq=0 out_seq=0 2024-09-19T09:13:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.930+0000 7fa0c0ff9640 1 -- 172.21.15.120:0/271696260 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55614535ed70 con 0x556145474b40 2024-09-19T09:13:37.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.930+0000 7fa0c0ff9640 10 client.19347 handle_command_reply: tid=0 2024-09-19T09:13:37.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.930+0000 7fa0e2602740 1 client.19347 shutdown 2024-09-19T09:13:37.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.930+0000 7fa0e2602740 1 -- 172.21.15.120:0/271696260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561453f42e0 msgr2=0x556145402e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:37.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.930+0000 7fa0e2602740 1 --2- 172.21.15.120:0/271696260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561453f42e0 0x556145402e50 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8010fd0 tx=0x7fa0c8005b20 comp rx=0 tx=0).stop 2024-09-19T09:13:37.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.930+0000 7fa0e2602740 1 -- 172.21.15.120:0/271696260 shutdown_connections 2024-09-19T09:13:37.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.930+0000 7fa0e2602740 1 --2- 172.21.15.120:0/271696260 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556145474b40 0x556145476f60 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.930+0000 7fa0e2602740 1 --2- 172.21.15.120:0/271696260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561453f9370 0x556145403e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.930+0000 7fa0c0ff9640 0 client.19347 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:37.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.930+0000 7fa0e2602740 1 --2- 172.21.15.120:0/271696260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561453f42e0 0x556145402e50 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.930+0000 7fa0e2602740 1 --2- 172.21.15.120:0/271696260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561453f39a0 0x556145402910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.930+0000 7fa0e2602740 1 -- 172.21.15.120:0/271696260 >> 172.21.15.120:0/271696260 conn(0x5561453ef010 msgr2=0x5561453ed9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:37.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.931+0000 7fa0e2602740 1 -- 172.21.15.120:0/271696260 shutdown_connections 2024-09-19T09:13:37.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.931+0000 7fa0e2602740 1 -- 172.21.15.120:0/271696260 wait complete. 2024-09-19T09:13:37.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.931+0000 7fa0e2602740 20 client.19347 trim_cache size 0 max 16384 2024-09-19T09:13:37.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:13:37.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:13:37.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:13:37.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:13:37.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.931+0000 7fa0df78d640 1 -- 172.21.15.120:0/2919833489 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa0c40a32f0 msgr2=0x7fa0c40a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:37.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.931+0000 7fa0df78d640 1 --2- 172.21.15.120:0/2919833489 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa0c40a32f0 0x7fa0c40a57b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0005fd0 tx=0x7fa0d00083f0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.931+0000 7fa0df78d640 1 -- 172.21.15.120:0/2919833489 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0d8102d10 msgr2=0x7fa0d819e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:37.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.931+0000 7fa0df78d640 1 --2- 172.21.15.120:0/2919833489 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0d8102d10 0x7fa0d819e920 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fa0c800d950 tx=0x7fa0c800de10 comp rx=0 tx=0).stop 2024-09-19T09:13:37.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.931+0000 7fa0df78d640 1 -- 172.21.15.120:0/2919833489 shutdown_connections 2024-09-19T09:13:37.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.931+0000 7fa0df78d640 1 --2- 172.21.15.120:0/2919833489 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa0c40a32f0 0x7fa0c40a57b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.931+0000 7fa0df78d640 1 --2- 172.21.15.120:0/2919833489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0d81056c0 0x7fa0d81a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.931+0000 7fa0df78d640 1 --2- 172.21.15.120:0/2919833489 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0d8102d10 0x7fa0d819e920 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.931+0000 7fa0df78d640 1 --2- 172.21.15.120:0/2919833489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0d81003c0 0x7fa0d819e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.931+0000 7fa0df78d640 1 -- 172.21.15.120:0/2919833489 >> 172.21.15.120:0/2919833489 conn(0x7fa0d80fc1b0 msgr2=0x7fa0d8103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.931+0000 7fa0df78d640 1 -- 172.21.15.120:0/2919833489 shutdown_connections 2024-09-19T09:13:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:37.932+0000 7fa0df78d640 1 -- 172.21.15.120:0/2919833489 wait complete. 2024-09-19T09:13:38.071 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.18 MB/sec execute 453 sec latency 21377.555 ms 2024-09-19T09:13:39.071 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.16 MB/sec execute 454 sec latency 22377.638 ms 2024-09-19T09:13:39.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:38 smithi196 ceph-mon[35150]: pgmap v666: 97 pgs: 97 active+clean; 29 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 49 op/s 2024-09-19T09:13:39.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:38 smithi153 ceph-mon[35071]: pgmap v666: 97 pgs: 97 active+clean; 29 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 49 op/s 2024-09-19T09:13:39.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:38 smithi120 ceph-mon[31150]: pgmap v666: 97 pgs: 97 active+clean; 29 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 49 op/s 2024-09-19T09:13:40.071 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.14 MB/sec execute 455 sec latency 23377.693 ms 2024-09-19T09:13:41.071 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.12 MB/sec execute 456 sec latency 24377.790 ms 2024-09-19T09:13:41.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:40 smithi196 ceph-mon[35150]: pgmap v667: 97 pgs: 97 active+clean; 31 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 190 KiB/s wr, 35 op/s 2024-09-19T09:13:41.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:40 smithi153 ceph-mon[35071]: pgmap v667: 97 pgs: 97 active+clean; 31 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 190 KiB/s wr, 35 op/s 2024-09-19T09:13:41.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:40 smithi120 ceph-mon[31150]: pgmap v667: 97 pgs: 97 active+clean; 31 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 190 KiB/s wr, 35 op/s 2024-09-19T09:13:42.071 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.10 MB/sec execute 457 sec latency 25377.887 ms 2024-09-19T09:13:43.071 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.08 MB/sec execute 458 sec latency 26378.006 ms 2024-09-19T09:13:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:42 smithi196 ceph-mon[35150]: pgmap v668: 97 pgs: 97 active+clean; 33 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 392 KiB/s wr, 30 op/s 2024-09-19T09:13:43.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:42 smithi153 ceph-mon[35071]: pgmap v668: 97 pgs: 97 active+clean; 33 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 392 KiB/s wr, 30 op/s 2024-09-19T09:13:43.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:42 smithi120 ceph-mon[31150]: pgmap v668: 97 pgs: 97 active+clean; 33 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 392 KiB/s wr, 30 op/s 2024-09-19T09:13:44.045 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:13:44.071 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.06 MB/sec execute 459 sec latency 27378.057 ms 2024-09-19T09:13:44.304 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:13:44.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.790+0000 7f5d9d562640 1 -- 172.21.15.120:0/3897509548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d9810c180 msgr2=0x7f5d9810e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:44.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.790+0000 7f5d9d562640 1 --2- 172.21.15.120:0/3897509548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d9810c180 0x7f5d9810e620 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009900 tx=0x7f5d9002a480 comp rx=0 tx=0).stop 2024-09-19T09:13:44.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.790+0000 7f5d9d562640 1 -- 172.21.15.120:0/3897509548 shutdown_connections 2024-09-19T09:13:44.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.790+0000 7f5d9d562640 1 --2- 172.21.15.120:0/3897509548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d9810c180 0x7f5d9810e620 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:44.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.790+0000 7f5d9d562640 1 --2- 172.21.15.120:0/3897509548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d98103670 0x7f5d9810bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:44.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.790+0000 7f5d9d562640 1 --2- 172.21.15.120:0/3897509548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d98102d50 0x7f5d98103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:44.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.790+0000 7f5d9d562640 1 -- 172.21.15.120:0/3897509548 >> 172.21.15.120:0/3897509548 conn(0x7f5d980fc7d0 msgr2=0x7f5d980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:44.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.791+0000 7f5d9d562640 1 -- 172.21.15.120:0/3897509548 shutdown_connections 2024-09-19T09:13:44.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.791+0000 7f5d9d562640 1 -- 172.21.15.120:0/3897509548 wait complete. 2024-09-19T09:13:44.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.791+0000 7f5d9d562640 1 Processor -- start 2024-09-19T09:13:44.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.791+0000 7f5d9d562640 1 -- start start 2024-09-19T09:13:44.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.792+0000 7f5d9d562640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d98102d50 0x7f5d98198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:44.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.792+0000 7f5d9d562640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d98103670 0x7f5d981989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:44.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.792+0000 7f5d9d562640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d9810c180 0x7f5d9819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:44.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.792+0000 7f5d9d562640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5d98111fc0 con 0x7f5d9810c180 2024-09-19T09:13:44.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.792+0000 7f5d9d562640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5d98111e40 con 0x7f5d98102d50 2024-09-19T09:13:44.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.792+0000 7f5d9d562640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5d98112140 con 0x7f5d98103670 2024-09-19T09:13:44.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.792+0000 7f5d977fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d9810c180 0x7f5d9819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:44.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.792+0000 7f5d977fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d9810c180 0x7f5d9819cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37662/0 (socket says 172.21.15.120:37662) 2024-09-19T09:13:44.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.792+0000 7f5d977fe640 1 -- 172.21.15.120:0/887714875 learned_addr learned my addr 172.21.15.120:0/887714875 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:44.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.792+0000 7f5d967fc640 1 --2- 172.21.15.120:0/887714875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d98103670 0x7f5d981989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:44.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.792+0000 7f5d96ffd640 1 --2- 172.21.15.120:0/887714875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d98102d50 0x7f5d98198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:44.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.792+0000 7f5d977fe640 1 -- 172.21.15.120:0/887714875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d98103670 msgr2=0x7f5d981989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:44.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.792+0000 7f5d977fe640 1 --2- 172.21.15.120:0/887714875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d98103670 0x7f5d981989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:44.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.792+0000 7f5d977fe640 1 -- 172.21.15.120:0/887714875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d98102d50 msgr2=0x7f5d98198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:44.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.792+0000 7f5d977fe640 1 --2- 172.21.15.120:0/887714875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d98102d50 0x7f5d98198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:44.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.792+0000 7f5d977fe640 1 -- 172.21.15.120:0/887714875 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d9819d430 con 0x7f5d9810c180 2024-09-19T09:13:44.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.792+0000 7f5d967fc640 1 --2- 172.21.15.120:0/887714875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d98103670 0x7f5d981989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:13:44.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.792+0000 7f5d977fe640 1 --2- 172.21.15.120:0/887714875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d9810c180 0x7f5d9819cd50 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f5d90005ab0 tx=0x7f5d900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:44.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.792+0000 7f5d7ffff640 1 -- 172.21.15.120:0/887714875 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9003f030 con 0x7f5d9810c180 2024-09-19T09:13:44.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.793+0000 7f5d7ffff640 1 -- 172.21.15.120:0/887714875 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d90003ce0 con 0x7f5d9810c180 2024-09-19T09:13:44.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.793+0000 7f5d9d562640 1 -- 172.21.15.120:0/887714875 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d9819d6c0 con 0x7f5d9810c180 2024-09-19T09:13:44.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.793+0000 7f5d7ffff640 1 -- 172.21.15.120:0/887714875 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90035070 con 0x7f5d9810c180 2024-09-19T09:13:44.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.793+0000 7f5d9d562640 1 -- 172.21.15.120:0/887714875 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d981a5000 con 0x7f5d9810c180 2024-09-19T09:13:44.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.794+0000 7f5d9d562640 1 -- 172.21.15.120:0/887714875 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d9806b070 con 0x7f5d9810c180 2024-09-19T09:13:44.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.795+0000 7f5d7ffff640 1 -- 172.21.15.120:0/887714875 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5d90042030 con 0x7f5d9810c180 2024-09-19T09:13:44.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.795+0000 7f5d7ffff640 1 --2- 172.21.15.120:0/887714875 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5d800a3240 0x7f5d800a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:44.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.795+0000 7f5d96ffd640 1 --2- 172.21.15.120:0/887714875 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5d800a3240 0x7f5d800a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:44.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.796+0000 7f5d7ffff640 1 -- 172.21.15.120:0/887714875 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f5d900f0440 con 0x7f5d9810c180 2024-09-19T09:13:44.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.796+0000 7f5d96ffd640 1 --2- 172.21.15.120:0/887714875 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5d800a3240 0x7f5d800a5700 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5d88009f20 tx=0x7f5d880094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:44.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.799+0000 7f5d7ffff640 1 -- 172.21.15.120:0/887714875 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5d900bd160 con 0x7f5d9810c180 2024-09-19T09:13:44.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.998+0000 7f5d9d562640 1 -- 172.21.15.120:0/887714875 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d981a51d0 con 0x7f5d9810c180 2024-09-19T09:13:45.000 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:44 smithi120 ceph-mon[31150]: pgmap v669: 97 pgs: 97 active+clean; 35 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 592 KiB/s wr, 32 op/s 2024-09-19T09:13:45.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:44.999+0000 7f5d7ffff640 1 -- 172.21.15.120:0/887714875 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11988 (secure 0 0 0) 0x7f5d900bca60 con 0x7f5d9810c180 2024-09-19T09:13:45.001 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:13:45.001 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":32,"btime":"2024-09-19T09:11:12:904684+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:11:12.904670+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27518":{"gid":27518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3214776406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3214776406},{"type":"v1","addr":"172.21.15.153:6835","nonce":3214776406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:13:45.001 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 32 2024-09-19T09:13:45.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:45.001+0000 7f5d9d562640 1 -- 172.21.15.120:0/887714875 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5d800a3240 msgr2=0x7f5d800a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:45.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:45.001+0000 7f5d9d562640 1 --2- 172.21.15.120:0/887714875 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5d800a3240 0x7f5d800a5700 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5d88009f20 tx=0x7f5d880094f0 comp rx=0 tx=0).stop 2024-09-19T09:13:45.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:45.001+0000 7f5d9d562640 1 -- 172.21.15.120:0/887714875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d9810c180 msgr2=0x7f5d9819cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:45.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:45.001+0000 7f5d9d562640 1 --2- 172.21.15.120:0/887714875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d9810c180 0x7f5d9819cd50 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f5d90005ab0 tx=0x7f5d900042f0 comp rx=0 tx=0).stop 2024-09-19T09:13:45.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:45.001+0000 7f5d9d562640 1 -- 172.21.15.120:0/887714875 shutdown_connections 2024-09-19T09:13:45.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:45.001+0000 7f5d9d562640 1 --2- 172.21.15.120:0/887714875 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5d800a3240 0x7f5d800a5700 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:45.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:45.001+0000 7f5d9d562640 1 --2- 172.21.15.120:0/887714875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d9810c180 0x7f5d9819cd50 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:45.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:45.001+0000 7f5d9d562640 1 --2- 172.21.15.120:0/887714875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d98103670 0x7f5d981989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:45.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:45.001+0000 7f5d9d562640 1 --2- 172.21.15.120:0/887714875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d98102d50 0x7f5d98198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:45.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:45.001+0000 7f5d9d562640 1 -- 172.21.15.120:0/887714875 >> 172.21.15.120:0/887714875 conn(0x7f5d980fc7d0 msgr2=0x7f5d980fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:45.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:45.001+0000 7f5d9d562640 1 -- 172.21.15.120:0/887714875 shutdown_connections 2024-09-19T09:13:45.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:45.001+0000 7f5d9d562640 1 -- 172.21.15.120:0/887714875 wait complete. 2024-09-19T09:13:45.071 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.04 MB/sec execute 460 sec latency 28378.120 ms 2024-09-19T09:13:45.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:44 smithi196 ceph-mon[35150]: pgmap v669: 97 pgs: 97 active+clean; 35 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 592 KiB/s wr, 32 op/s 2024-09-19T09:13:45.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:44 smithi153 ceph-mon[35071]: pgmap v669: 97 pgs: 97 active+clean; 35 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 592 KiB/s wr, 32 op/s 2024-09-19T09:13:45.713 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db --set-id 13a5f0 --release --await' 2024-09-19T09:13:45.713 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db --set-id 13a5f0 --release --await 2024-09-19T09:13:45.996 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:13:46.071 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992995 9.02 MB/sec execute 461 sec latency 29378.207 ms 2024-09-19T09:13:46.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:45 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/887714875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:13:46.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:45 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/887714875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:13:46.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:45 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/887714875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:13:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.508+0000 7f6606bbf640 1 -- 172.21.15.120:0/1240444548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66001062b0 msgr2=0x7f6600106690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.508+0000 7f6606bbf640 1 --2- 172.21.15.120:0/1240444548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66001062b0 0x7f6600106690 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f65f400aa40 tx=0x7f65f402b750 comp rx=0 tx=0).stop 2024-09-19T09:13:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.508+0000 7f6606bbf640 1 -- 172.21.15.120:0/1240444548 shutdown_connections 2024-09-19T09:13:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.508+0000 7f6606bbf640 1 --2- 172.21.15.120:0/1240444548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6600100c00 0x7f660010d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.508+0000 7f6606bbf640 1 --2- 172.21.15.120:0/1240444548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66001002a0 0x7f66001006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.508+0000 7f6606bbf640 1 --2- 172.21.15.120:0/1240444548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66001062b0 0x7f6600106690 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.508+0000 7f6606bbf640 1 -- 172.21.15.120:0/1240444548 >> 172.21.15.120:0/1240444548 conn(0x7f66000fc5c0 msgr2=0x7f66000fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.508+0000 7f6606bbf640 1 -- 172.21.15.120:0/1240444548 shutdown_connections 2024-09-19T09:13:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.508+0000 7f6606bbf640 1 -- 172.21.15.120:0/1240444548 wait complete. 2024-09-19T09:13:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.509+0000 7f6606bbf640 1 Processor -- start 2024-09-19T09:13:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.509+0000 7f6606bbf640 1 -- start start 2024-09-19T09:13:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.509+0000 7f6606bbf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66001002a0 0x7f6600198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.509+0000 7f6606bbf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6600100c00 0x7f66001989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.509+0000 7f6606bbf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66001062b0 0x7f660019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.509+0000 7f6606bbf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f660010fd90 con 0x7f66001062b0 2024-09-19T09:13:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.509+0000 7f6606bbf640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f660010fc10 con 0x7f6600100c00 2024-09-19T09:13:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.509+0000 7f6606bbf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f660010ff10 con 0x7f66001002a0 2024-09-19T09:13:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.509+0000 7f660515c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66001062b0 0x7f660019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.509+0000 7f660515c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66001062b0 0x7f660019cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37676/0 (socket says 172.21.15.120:37676) 2024-09-19T09:13:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.509+0000 7f660515c640 1 -- 172.21.15.120:0/3914820670 learned_addr learned my addr 172.21.15.120:0/3914820670 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.509+0000 7f65fffff640 1 --2- 172.21.15.120:0/3914820670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6600100c00 0x7f66001989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.509+0000 7f660495b640 1 --2- 172.21.15.120:0/3914820670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66001002a0 0x7f6600198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.509+0000 7f660515c640 1 -- 172.21.15.120:0/3914820670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66001002a0 msgr2=0x7f6600198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.509+0000 7f660515c640 1 --2- 172.21.15.120:0/3914820670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66001002a0 0x7f6600198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.509+0000 7f660515c640 1 -- 172.21.15.120:0/3914820670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6600100c00 msgr2=0x7f66001989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.509+0000 7f660515c640 1 --2- 172.21.15.120:0/3914820670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6600100c00 0x7f66001989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.509+0000 7f660515c640 1 -- 172.21.15.120:0/3914820670 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f660019d430 con 0x7f66001062b0 2024-09-19T09:13:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.509+0000 7f660495b640 1 --2- 172.21.15.120:0/3914820670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66001002a0 0x7f6600198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:13:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.509+0000 7f65fffff640 1 --2- 172.21.15.120:0/3914820670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6600100c00 0x7f66001989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:13:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.510+0000 7f660515c640 1 --2- 172.21.15.120:0/3914820670 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66001062b0 0x7f660019cd50 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f65f800eaa0 tx=0x7f65f800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.510+0000 7f65fdffb640 1 -- 172.21.15.120:0/3914820670 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f800cdb0 con 0x7f66001062b0 2024-09-19T09:13:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.510+0000 7f65fdffb640 1 -- 172.21.15.120:0/3914820670 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65f8005020 con 0x7f66001062b0 2024-09-19T09:13:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.510+0000 7f6606bbf640 1 -- 172.21.15.120:0/3914820670 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f660019d720 con 0x7f66001062b0 2024-09-19T09:13:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.510+0000 7f65fdffb640 1 -- 172.21.15.120:0/3914820670 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f80116b0 con 0x7f66001062b0 2024-09-19T09:13:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.510+0000 7f6606bbf640 1 -- 172.21.15.120:0/3914820670 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66001a5000 con 0x7f66001062b0 2024-09-19T09:13:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.512+0000 7f65fdffb640 1 -- 172.21.15.120:0/3914820670 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f65f8005560 con 0x7f66001062b0 2024-09-19T09:13:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.513+0000 7f6608f98740 1 Processor -- start 2024-09-19T09:13:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.513+0000 7f6608f98740 1 -- start start 2024-09-19T09:13:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.513+0000 7f65fdffb640 1 --2- 172.21.15.120:0/3914820670 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f65f00a32c0 0x7f65f00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.513+0000 7f6608f98740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9e5ee1060 0x55a9e5ee1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.513+0000 7f6608f98740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9e5ee1980 0x55a9e5eebd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.513+0000 7f6608f98740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9e5eec240 0x55a9e5eee630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.513+0000 7f6608f98740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a9e5e46620 con 0x55a9e5ee1980 2024-09-19T09:13:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.513+0000 7f6608f98740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a9e5dede50 con 0x55a9e5eec240 2024-09-19T09:13:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.513+0000 7f6608f98740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a9e5e425e0 con 0x55a9e5ee1060 2024-09-19T09:13:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.513+0000 7f65fffff640 1 --2- 172.21.15.120:0/3914820670 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f65f00a32c0 0x7f65f00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.513+0000 7f65fdffb640 1 -- 172.21.15.120:0/3914820670 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f65f80cfb80 con 0x7f66001062b0 2024-09-19T09:13:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.513+0000 7f65fffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9e5ee1980 0x55a9e5eebd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.513+0000 7f65fffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9e5ee1980 0x55a9e5eebd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37692/0 (socket says 172.21.15.120:37692) 2024-09-19T09:13:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.513+0000 7f65fffff640 1 -- 172.21.15.120:0/822820853 learned_addr learned my addr 172.21.15.120:0/822820853 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.513+0000 7f660495b640 1 --2- 172.21.15.120:0/822820853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9e5ee1060 0x55a9e5ee1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.514+0000 7f65fffff640 1 -- 172.21.15.120:0/822820853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9e5ee1060 msgr2=0x55a9e5ee1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.514+0000 7f65fffff640 1 --2- 172.21.15.120:0/822820853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9e5ee1060 0x55a9e5ee1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.514+0000 7f660515c640 1 --2- 172.21.15.120:0/822820853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9e5eec240 0x55a9e5eee630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.514+0000 7f65fffff640 1 -- 172.21.15.120:0/822820853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9e5eec240 msgr2=0x55a9e5eee630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.514+0000 7f65fffff640 1 --2- 172.21.15.120:0/822820853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9e5eec240 0x55a9e5eee630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.514+0000 7f65fffff640 1 -- 172.21.15.120:0/822820853 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9e5dded40 con 0x55a9e5ee1980 2024-09-19T09:13:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.514+0000 7f660515c640 1 --2- 172.21.15.120:0/822820853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9e5eec240 0x55a9e5eee630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:13:46.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.514+0000 7f65fffff640 1 --2- 172.21.15.120:0/3914820670 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f65f00a32c0 0x7f65f00a5780 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f65ec00cfd0 tx=0x7f65ec00cc60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:46.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.514+0000 7f65fffff640 1 --2- 172.21.15.120:0/822820853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9e5ee1980 0x55a9e5eebd00 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f65ec011ce0 tx=0x7f65ec01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c63dab28dc856ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:46.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.514+0000 7f65eaffd640 1 -- 172.21.15.120:0/822820853 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec022070 con 0x55a9e5ee1980 2024-09-19T09:13:46.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.514+0000 7f65eaffd640 1 -- 172.21.15.120:0/822820853 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65ec00fc60 con 0x55a9e5ee1980 2024-09-19T09:13:46.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.514+0000 7f65eaffd640 1 -- 172.21.15.120:0/822820853 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec012a90 con 0x55a9e5ee1980 2024-09-19T09:13:46.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.515+0000 7f6608f98740 1 -- 172.21.15.120:0/822820853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9e5ee1980 msgr2=0x55a9e5eebd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:46.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.515+0000 7f6608f98740 1 --2- 172.21.15.120:0/822820853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9e5ee1980 0x55a9e5eebd00 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f65ec011ce0 tx=0x7f65ec01b000 comp rx=0 tx=0).stop 2024-09-19T09:13:46.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.515+0000 7f6608f98740 1 -- 172.21.15.120:0/822820853 shutdown_connections 2024-09-19T09:13:46.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.515+0000 7f6608f98740 1 --2- 172.21.15.120:0/822820853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9e5eec240 0x55a9e5eee630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.515+0000 7f6608f98740 1 --2- 172.21.15.120:0/822820853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9e5ee1980 0x55a9e5eebd00 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.515+0000 7f6608f98740 1 --2- 172.21.15.120:0/822820853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9e5ee1060 0x55a9e5ee1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.515+0000 7f6608f98740 1 -- 172.21.15.120:0/822820853 >> 172.21.15.120:0/822820853 conn(0x55a9e5edd5f0 msgr2=0x55a9e5edd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:46.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.515+0000 7f6608f98740 1 -- 172.21.15.120:0/822820853 shutdown_connections 2024-09-19T09:13:46.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.515+0000 7f6608f98740 1 -- 172.21.15.120:0/822820853 wait complete. 2024-09-19T09:13:46.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.515+0000 7f6608f98740 1 -- start start 2024-09-19T09:13:46.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.515+0000 7f6608f98740 1 Processor -- start 2024-09-19T09:13:46.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.515+0000 7f6608f98740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9e5ee1060 0x55a9e5ee9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:46.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.516+0000 7f6608f98740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9e5ee1980 0x55a9e5ee96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:46.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.516+0000 7f6608f98740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9e5eec240 0x55a9e5ee9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:46.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.516+0000 7f6608f98740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a9e5eeee10 con 0x55a9e5eec240 2024-09-19T09:13:46.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.516+0000 7f6608f98740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a9e5dcef10 con 0x55a9e5ee1980 2024-09-19T09:13:46.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.516+0000 7f6608f98740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a9e5eeef90 con 0x55a9e5ee1060 2024-09-19T09:13:46.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.516+0000 7f6608f98740 10 client.0 _ll_register_callbacks cb 0x55a9e5df2db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:46.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.516+0000 7f660515c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9e5eec240 0x55a9e5ee9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:46.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.516+0000 7f660515c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9e5eec240 0x55a9e5ee9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37700/0 (socket says 172.21.15.120:37700) 2024-09-19T09:13:46.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.516+0000 7f660515c640 1 -- 172.21.15.120:0/394715765 learned_addr learned my addr 172.21.15.120:0/394715765 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:46.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.516+0000 7f660495b640 1 --2- 172.21.15.120:0/394715765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9e5ee1060 0x55a9e5ee9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:46.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.516+0000 7f65fffff640 1 --2- 172.21.15.120:0/394715765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9e5ee1980 0x55a9e5ee96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:46.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.516+0000 7f660515c640 1 -- 172.21.15.120:0/394715765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9e5ee1060 msgr2=0x55a9e5ee9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:46.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.516+0000 7f660515c640 1 --2- 172.21.15.120:0/394715765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9e5ee1060 0x55a9e5ee9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.516+0000 7f660515c640 1 -- 172.21.15.120:0/394715765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9e5ee1980 msgr2=0x55a9e5ee96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:46.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.516+0000 7f660515c640 1 --2- 172.21.15.120:0/394715765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9e5ee1980 0x55a9e5ee96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.516+0000 7f660515c640 1 -- 172.21.15.120:0/394715765 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9e5b55580 con 0x55a9e5eec240 2024-09-19T09:13:46.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.516+0000 7f660515c640 1 --2- 172.21.15.120:0/394715765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9e5eec240 0x55a9e5ee9c00 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f65f800efd0 tx=0x7f65f809c220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504f1679af5e0498 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:46.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.516+0000 7f65dffff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:46.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.516+0000 7f65dffff640 1 -- 172.21.15.120:0/394715765 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f801f070 con 0x55a9e5eec240 2024-09-19T09:13:46.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.516+0000 7f65dffff640 1 -- 172.21.15.120:0/394715765 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f65f80bc0c0 con 0x55a9e5eec240 2024-09-19T09:13:46.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.516+0000 7f6608f98740 1 -- 172.21.15.120:0/394715765 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65ec00fc60 con 0x55a9e5eec240 2024-09-19T09:13:46.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.516+0000 7f65dffff640 1 -- 172.21.15.120:0/394715765 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f65f80a0120 con 0x55a9e5eec240 2024-09-19T09:13:46.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.516+0000 7f6608f98740 10 client.19365 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:46.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.516+0000 7f6608f98740 1 -- 172.21.15.120:0/394715765 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a9e5c1f960 con 0x55a9e5eec240 2024-09-19T09:13:46.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.517+0000 7f65dffff640 1 -- 172.21.15.120:0/394715765 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65ec00fc60 con 0x55a9e5eec240 2024-09-19T09:13:46.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.517+0000 7f65dffff640 1 -- 172.21.15.120:0/394715765 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f65f800f040 con 0x55a9e5eec240 2024-09-19T09:13:46.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.517+0000 7f6608f98740 10 client.19365 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:46.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.517+0000 7f6608f98740 10 client.19365 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:13:46.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.517+0000 7f6608f98740 20 client.19365 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:46.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.517+0000 7f6608f98740 1 --2- 172.21.15.120:0/394715765 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a9e5efbd10 0x55a9e5efe140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:46.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.517+0000 7f6608f98740 4 client.19365 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:13:46.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.517+0000 7f6608f98740 1 -- 172.21.15.120:0/394715765 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9e5e53b90 con 0x55a9e5efbd10 2024-09-19T09:13:46.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.517+0000 7f660495b640 1 --2- 172.21.15.120:0/394715765 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a9e5efbd10 0x55a9e5efe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:46.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.517+0000 7f65dffff640 10 client.19365 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:46.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.517+0000 7f660495b640 1 --2- 172.21.15.120:0/394715765 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a9e5efbd10 0x55a9e5efe140 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdca84b7fe785ba6 server_cookie=81d03d7695a7b874 in_seq=0 out_seq=0 2024-09-19T09:13:46.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.518+0000 7f65dffff640 1 -- 172.21.15.120:0/394715765 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a9e5e53b90 con 0x55a9e5efbd10 2024-09-19T09:13:46.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.518+0000 7f65dffff640 10 client.19365 handle_command_reply: tid=0 2024-09-19T09:13:46.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.518+0000 7f6608f98740 1 client.19365 shutdown 2024-09-19T09:13:46.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.518+0000 7f6608f98740 1 -- 172.21.15.120:0/394715765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9e5eec240 msgr2=0x55a9e5ee9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:46.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.518+0000 7f6608f98740 1 --2- 172.21.15.120:0/394715765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9e5eec240 0x55a9e5ee9c00 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f65f800efd0 tx=0x7f65f809c220 comp rx=0 tx=0).stop 2024-09-19T09:13:46.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.518+0000 7f6608f98740 1 -- 172.21.15.120:0/394715765 shutdown_connections 2024-09-19T09:13:46.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.518+0000 7f6608f98740 1 --2- 172.21.15.120:0/394715765 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a9e5efbd10 0x55a9e5efe140 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.518+0000 7f6608f98740 1 --2- 172.21.15.120:0/394715765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9e5eec240 0x55a9e5ee9c00 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.518+0000 7f6608f98740 1 --2- 172.21.15.120:0/394715765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9e5ee1980 0x55a9e5ee96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.518+0000 7f6608f98740 1 --2- 172.21.15.120:0/394715765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9e5ee1060 0x55a9e5ee9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.518+0000 7f65dffff640 0 client.19365 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:46.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.518+0000 7f6608f98740 1 -- 172.21.15.120:0/394715765 >> 172.21.15.120:0/394715765 conn(0x55a9e5edd5f0 msgr2=0x55a9e5ee3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:46.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.518+0000 7f6608f98740 1 -- 172.21.15.120:0/394715765 shutdown_connections 2024-09-19T09:13:46.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.518+0000 7f6608f98740 1 -- 172.21.15.120:0/394715765 wait complete. 2024-09-19T09:13:46.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.519+0000 7f6608f98740 20 client.19365 trim_cache size 0 max 16384 2024-09-19T09:13:46.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.535+0000 7f6608f98740 1 Processor -- start 2024-09-19T09:13:46.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.535+0000 7f6608f98740 1 -- start start 2024-09-19T09:13:46.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.535+0000 7f6608f98740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9e5ee1060 0x55a9e5ee3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:46.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.535+0000 7f6608f98740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9e5ee1980 0x55a9e5edc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:46.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.535+0000 7f6608f98740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9e5eec240 0x55a9e5ef4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:46.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.535+0000 7f6608f98740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f66001ad620 con 0x55a9e5eec240 2024-09-19T09:13:46.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.535+0000 7f6608f98740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f66001af160 con 0x55a9e5ee1060 2024-09-19T09:13:46.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.535+0000 7f6608f98740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f66001adcf0 con 0x55a9e5ee1980 2024-09-19T09:13:46.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.535+0000 7f660515c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9e5eec240 0x55a9e5ef4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:46.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.535+0000 7f660515c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9e5eec240 0x55a9e5ef4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37704/0 (socket says 172.21.15.120:37704) 2024-09-19T09:13:46.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.535+0000 7f660515c640 1 -- 172.21.15.120:0/2742494205 learned_addr learned my addr 172.21.15.120:0/2742494205 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:46.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.536+0000 7f660495b640 1 --2- 172.21.15.120:0/2742494205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9e5ee1060 0x55a9e5ee3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:46.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.536+0000 7f660515c640 1 -- 172.21.15.120:0/2742494205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9e5ee1980 msgr2=0x55a9e5edc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:46.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.536+0000 7f660515c640 1 --2- 172.21.15.120:0/2742494205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9e5ee1980 0x55a9e5edc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.536+0000 7f660515c640 1 -- 172.21.15.120:0/2742494205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9e5ee1060 msgr2=0x55a9e5ee3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:46.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.536+0000 7f660515c640 1 --2- 172.21.15.120:0/2742494205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9e5ee1060 0x55a9e5ee3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.536+0000 7f660515c640 1 -- 172.21.15.120:0/2742494205 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9e5ede280 con 0x55a9e5eec240 2024-09-19T09:13:46.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.536+0000 7f660495b640 1 --2- 172.21.15.120:0/2742494205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9e5ee1060 0x55a9e5ee3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:13:46.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.536+0000 7f660515c640 1 --2- 172.21.15.120:0/2742494205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9e5eec240 0x55a9e5ef4610 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f65f80a48a0 tx=0x7f65f809a670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7169555d2f230075 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:46.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.536+0000 7f65ea7fc640 1 -- 172.21.15.120:0/2742494205 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f801f040 con 0x55a9e5eec240 2024-09-19T09:13:46.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.536+0000 7f65ea7fc640 1 -- 172.21.15.120:0/2742494205 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65f80a0510 con 0x55a9e5eec240 2024-09-19T09:13:46.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.536+0000 7f65ea7fc640 1 -- 172.21.15.120:0/2742494205 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f8097eb0 con 0x55a9e5eec240 2024-09-19T09:13:46.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.537+0000 7f6608f98740 1 -- 172.21.15.120:0/2742494205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9e5eec240 msgr2=0x55a9e5ef4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:46.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.537+0000 7f6608f98740 1 --2- 172.21.15.120:0/2742494205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9e5eec240 0x55a9e5ef4610 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f65f80a48a0 tx=0x7f65f809a670 comp rx=0 tx=0).stop 2024-09-19T09:13:46.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.537+0000 7f6608f98740 1 -- 172.21.15.120:0/2742494205 shutdown_connections 2024-09-19T09:13:46.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.537+0000 7f6608f98740 1 --2- 172.21.15.120:0/2742494205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9e5eec240 0x55a9e5ef4610 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.537+0000 7f6608f98740 1 --2- 172.21.15.120:0/2742494205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9e5ee1980 0x55a9e5edc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.537+0000 7f6608f98740 1 --2- 172.21.15.120:0/2742494205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9e5ee1060 0x55a9e5ee3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.537+0000 7f6608f98740 1 -- 172.21.15.120:0/2742494205 >> 172.21.15.120:0/2742494205 conn(0x55a9e5edd5f0 msgr2=0x55a9e5edbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:46.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.537+0000 7f6608f98740 1 -- 172.21.15.120:0/2742494205 shutdown_connections 2024-09-19T09:13:46.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.537+0000 7f6608f98740 1 -- 172.21.15.120:0/2742494205 wait complete. 2024-09-19T09:13:46.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.537+0000 7f6608f98740 1 -- start start 2024-09-19T09:13:46.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.537+0000 7f6608f98740 1 Processor -- start 2024-09-19T09:13:46.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.537+0000 7f6608f98740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9e5ee1060 0x55a9e5f5ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:46.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.537+0000 7f6608f98740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9e5ee1980 0x55a9e5f5f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:46.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.537+0000 7f6608f98740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9e5eec240 0x55a9e5f63ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:46.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.537+0000 7f6608f98740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f66001aea90 con 0x55a9e5ee1060 2024-09-19T09:13:46.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.537+0000 7f6608f98740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f66001ae3c0 con 0x55a9e5eec240 2024-09-19T09:13:46.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.537+0000 7f6608f98740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f66001ab5e0 con 0x55a9e5ee1980 2024-09-19T09:13:46.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.538+0000 7f6608f98740 10 client.0 _ll_register_callbacks cb 0x7f65f809c8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:46.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.538+0000 7f660495b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9e5ee1060 0x55a9e5f5ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:46.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.538+0000 7f660495b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9e5ee1060 0x55a9e5f5ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37708/0 (socket says 172.21.15.120:37708) 2024-09-19T09:13:46.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.538+0000 7f660495b640 1 -- 172.21.15.120:0/638557167 learned_addr learned my addr 172.21.15.120:0/638557167 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:46.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.538+0000 7f660515c640 1 --2- 172.21.15.120:0/638557167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9e5eec240 0x55a9e5f63ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:46.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.538+0000 7f65fffff640 1 --2- 172.21.15.120:0/638557167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9e5ee1980 0x55a9e5f5f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:46.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.538+0000 7f660495b640 1 -- 172.21.15.120:0/638557167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9e5ee1980 msgr2=0x55a9e5f5f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:46.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.538+0000 7f660495b640 1 --2- 172.21.15.120:0/638557167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9e5ee1980 0x55a9e5f5f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.538+0000 7f660495b640 1 -- 172.21.15.120:0/638557167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9e5eec240 msgr2=0x55a9e5f63ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:46.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.538+0000 7f660495b640 1 --2- 172.21.15.120:0/638557167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9e5eec240 0x55a9e5f63ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.538+0000 7f660495b640 1 -- 172.21.15.120:0/638557167 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9e5f64150 con 0x55a9e5ee1060 2024-09-19T09:13:46.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.538+0000 7f660495b640 1 --2- 172.21.15.120:0/638557167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9e5ee1060 0x55a9e5f5ebd0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f65f400aa10 tx=0x7f65f40041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40680ed4ddac4f96 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:46.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.538+0000 7f65dffff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:46.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.538+0000 7f65dffff640 1 -- 172.21.15.120:0/638557167 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f404c060 con 0x55a9e5ee1060 2024-09-19T09:13:46.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.538+0000 7f65dffff640 1 -- 172.21.15.120:0/638557167 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f65f40308c0 con 0x55a9e5ee1060 2024-09-19T09:13:46.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.538+0000 7f6608f98740 1 -- 172.21.15.120:0/638557167 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65f80a0510 con 0x55a9e5ee1060 2024-09-19T09:13:46.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.538+0000 7f65dffff640 1 -- 172.21.15.120:0/638557167 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f65f4036040 con 0x55a9e5ee1060 2024-09-19T09:13:46.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.538+0000 7f6608f98740 10 client.19371 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:46.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.538+0000 7f6608f98740 1 -- 172.21.15.120:0/638557167 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a9e5f6b580 con 0x55a9e5ee1060 2024-09-19T09:13:46.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.539+0000 7f65dffff640 1 -- 172.21.15.120:0/638557167 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65f80a0510 con 0x55a9e5ee1060 2024-09-19T09:13:46.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.539+0000 7f65dffff640 1 -- 172.21.15.120:0/638557167 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f65f4051070 con 0x55a9e5ee1060 2024-09-19T09:13:46.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.539+0000 7f6608f98740 10 client.19371 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:46.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.539+0000 7f6608f98740 10 client.19371 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:13:46.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.539+0000 7f6608f98740 20 client.19371 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:46.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.539+0000 7f6608f98740 1 --2- 172.21.15.120:0/638557167 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a9e5f6bd00 0x55a9e5f6e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:46.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.539+0000 7f6608f98740 4 client.19371 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "13a5f0", "release": true, "await": true}] 2024-09-19T09:13:46.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.539+0000 7f6608f98740 1 -- 172.21.15.120:0/638557167 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "set_id": "13a5f0", "release": true, "await": true}) -- 0x55a9e5e4e0a0 con 0x55a9e5f6bd00 2024-09-19T09:13:46.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.539+0000 7f65fffff640 1 --2- 172.21.15.120:0/638557167 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a9e5f6bd00 0x55a9e5f6e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:46.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.539+0000 7f65fffff640 1 --2- 172.21.15.120:0/638557167 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a9e5f6bd00 0x55a9e5f6e120 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcf6622a737792ed server_cookie=b9347a6ae3d2ef18 in_seq=0 out_seq=0 2024-09-19T09:13:46.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.539+0000 7f65dffff640 10 client.19371 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:46.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.560+0000 7f65dffff640 1 -- 172.21.15.120:0/638557167 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55a9e5e4e0a0 con 0x55a9e5f6bd00 2024-09-19T09:13:46.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.560+0000 7f65dffff640 10 client.19371 handle_command_reply: tid=0 2024-09-19T09:13:46.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.560+0000 7f6608f98740 1 client.19371 shutdown 2024-09-19T09:13:46.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.560+0000 7f6608f98740 1 -- 172.21.15.120:0/638557167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9e5ee1060 msgr2=0x55a9e5f5ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:46.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.560+0000 7f6608f98740 1 --2- 172.21.15.120:0/638557167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9e5ee1060 0x55a9e5f5ebd0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f65f400aa10 tx=0x7f65f40041a0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.560+0000 7f6608f98740 1 -- 172.21.15.120:0/638557167 shutdown_connections 2024-09-19T09:13:46.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.560+0000 7f6608f98740 1 --2- 172.21.15.120:0/638557167 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a9e5f6bd00 0x55a9e5f6e120 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.560+0000 7f65dffff640 0 client.19371 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:46.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.560+0000 7f6608f98740 1 --2- 172.21.15.120:0/638557167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9e5eec240 0x55a9e5f63ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.560+0000 7f6608f98740 1 --2- 172.21.15.120:0/638557167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9e5ee1980 0x55a9e5f5f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.560+0000 7f6608f98740 1 --2- 172.21.15.120:0/638557167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9e5ee1060 0x55a9e5f5ebd0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.560+0000 7f6608f98740 1 -- 172.21.15.120:0/638557167 >> 172.21.15.120:0/638557167 conn(0x55a9e5edd5f0 msgr2=0x55a9e5ee9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:46.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.560+0000 7f6608f98740 1 -- 172.21.15.120:0/638557167 shutdown_connections 2024-09-19T09:13:46.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.560+0000 7f6608f98740 1 -- 172.21.15.120:0/638557167 wait complete. 2024-09-19T09:13:46.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.560+0000 7f6608f98740 20 client.19371 trim_cache size 0 max 16384 2024-09-19T09:13:46.563 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:13:46.563 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 32, 2024-09-19T09:13:46.563 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:13:46.563 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 40, 2024-09-19T09:13:46.563 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:13:46.563 INFO:teuthology.orchestra.run.smithi120.stdout: "13a5f0": { 2024-09-19T09:13:46.563 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 40, 2024-09-19T09:13:46.563 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:13:46.563 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:13:46.563 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:13:46.563 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:13:46.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:13:46.563 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:13:46.564 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 150.0, 2024-09-19T09:13:46.564 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:13:46.564 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:13:46.564 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:13:46.564 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:13:46.564 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:13:46.564 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:13:46.564 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:13:46.564 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:13:46.564 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:13:46.564 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:13:46.564 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:13:46.564 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:13:46.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.560+0000 7f6606bbf640 1 -- 172.21.15.120:0/3914820670 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f65f00a32c0 msgr2=0x7f65f00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:46.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.560+0000 7f6606bbf640 1 --2- 172.21.15.120:0/3914820670 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f65f00a32c0 0x7f65f00a5780 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f65ec00cfd0 tx=0x7f65ec00cc60 comp rx=0 tx=0).stop 2024-09-19T09:13:46.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.560+0000 7f6606bbf640 1 -- 172.21.15.120:0/3914820670 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66001062b0 msgr2=0x7f660019cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:46.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.560+0000 7f6606bbf640 1 --2- 172.21.15.120:0/3914820670 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66001062b0 0x7f660019cd50 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f65f800eaa0 tx=0x7f65f800ef60 comp rx=0 tx=0).stop 2024-09-19T09:13:46.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.560+0000 7f6606bbf640 1 -- 172.21.15.120:0/3914820670 shutdown_connections 2024-09-19T09:13:46.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.560+0000 7f6606bbf640 1 --2- 172.21.15.120:0/3914820670 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f65f00a32c0 0x7f65f00a5780 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.560+0000 7f6606bbf640 1 --2- 172.21.15.120:0/3914820670 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66001062b0 0x7f660019cd50 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.560+0000 7f6606bbf640 1 --2- 172.21.15.120:0/3914820670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6600100c00 0x7f66001989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.560+0000 7f6606bbf640 1 --2- 172.21.15.120:0/3914820670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66001002a0 0x7f6600198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:46.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.560+0000 7f6606bbf640 1 -- 172.21.15.120:0/3914820670 >> 172.21.15.120:0/3914820670 conn(0x7f66000fc5c0 msgr2=0x7f66000feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:46.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.560+0000 7f6606bbf640 1 -- 172.21.15.120:0/3914820670 shutdown_connections 2024-09-19T09:13:46.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:13:46.560+0000 7f6606bbf640 1 -- 172.21.15.120:0/3914820670 wait complete. 2024-09-19T09:13:47.071 INFO:tasks.workunit.client.0.smithi120.stdout: 1 994773 9.03 MB/sec execute 462 sec latency 29870.660 ms 2024-09-19T09:13:47.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:46 smithi196 ceph-mon[35150]: pgmap v670: 97 pgs: 97 active+clean; 34 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 617 KiB/s wr, 30 op/s 2024-09-19T09:13:47.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:46 smithi153 ceph-mon[35071]: pgmap v670: 97 pgs: 97 active+clean; 34 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 617 KiB/s wr, 30 op/s 2024-09-19T09:13:47.312 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:46 smithi120 ceph-mon[31150]: pgmap v670: 97 pgs: 97 active+clean; 34 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 617 KiB/s wr, 30 op/s 2024-09-19T09:13:47.342 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '13a5f0', total seconds elapsed: 33.3 2024-09-19T09:13:48.071 INFO:tasks.workunit.client.0.smithi120.stdout: 1 997216 9.03 MB/sec execute 463 sec latency 14.690 ms 2024-09-19T09:13:49.072 INFO:tasks.workunit.client.0.smithi120.stdout: 1 999561 9.04 MB/sec execute 464 sec latency 20.107 ms 2024-09-19T09:13:49.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:48 smithi196 ceph-mon[35150]: pgmap v671: 97 pgs: 97 active+clean; 30 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 636 KiB/s wr, 36 op/s 2024-09-19T09:13:49.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:48 smithi153 ceph-mon[35071]: pgmap v671: 97 pgs: 97 active+clean; 30 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 636 KiB/s wr, 36 op/s 2024-09-19T09:13:49.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:48 smithi120 ceph-mon[31150]: pgmap v671: 97 pgs: 97 active+clean; 30 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 636 KiB/s wr, 36 op/s 2024-09-19T09:13:50.071 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1002466 9.06 MB/sec execute 465 sec latency 7.714 ms 2024-09-19T09:13:50.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:49 smithi120 ceph-mon[31150]: pgmap v672: 97 pgs: 97 active+clean; 30 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 826 KiB/s wr, 69 op/s 2024-09-19T09:13:50.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:49 smithi196 ceph-mon[35150]: pgmap v672: 97 pgs: 97 active+clean; 30 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 826 KiB/s wr, 69 op/s 2024-09-19T09:13:50.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:49 smithi153 ceph-mon[35071]: pgmap v672: 97 pgs: 97 active+clean; 30 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 826 KiB/s wr, 69 op/s 2024-09-19T09:13:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.025+0000 7fa36d8f1640 1 -- 172.21.15.120:0/2640814695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa368102d50 msgr2=0x7fa3681031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.025+0000 7fa36d8f1640 1 --2- 172.21.15.120:0/2640814695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa368102d50 0x7fa3681031d0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fa35c009900 tx=0x7fa35c02a410 comp rx=0 tx=0).stop 2024-09-19T09:13:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.025+0000 7fa36d8f1640 1 -- 172.21.15.120:0/2640814695 shutdown_connections 2024-09-19T09:13:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.025+0000 7fa36d8f1640 1 --2- 172.21.15.120:0/2640814695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa368103710 0x7fa368109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.025+0000 7fa36d8f1640 1 --2- 172.21.15.120:0/2640814695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa368102d50 0x7fa3681031d0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.025+0000 7fa36d8f1640 1 --2- 172.21.15.120:0/2640814695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa368101b60 0x7fa368101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.025+0000 7fa36d8f1640 1 -- 172.21.15.120:0/2640814695 >> 172.21.15.120:0/2640814695 conn(0x7fa3680fd2e0 msgr2=0x7fa3680ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.025+0000 7fa36d8f1640 1 -- 172.21.15.120:0/2640814695 shutdown_connections 2024-09-19T09:13:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.025+0000 7fa36d8f1640 1 -- 172.21.15.120:0/2640814695 wait complete. 2024-09-19T09:13:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.026+0000 7fa36d8f1640 1 Processor -- start 2024-09-19T09:13:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.026+0000 7fa36d8f1640 1 -- start start 2024-09-19T09:13:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.026+0000 7fa36d8f1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa368101b60 0x7fa368197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.026+0000 7fa36d8f1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa368102d50 0x7fa368198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.026+0000 7fa36d8f1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa368103710 0x7fa36819f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.026+0000 7fa36d8f1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa36810d830 con 0x7fa368103710 2024-09-19T09:13:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.026+0000 7fa36d8f1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa36810d6b0 con 0x7fa368101b60 2024-09-19T09:13:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.026+0000 7fa36d8f1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa36810d9b0 con 0x7fa368102d50 2024-09-19T09:13:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.026+0000 7fa3677fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa368103710 0x7fa36819f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.026+0000 7fa3677fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa368103710 0x7fa36819f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37720/0 (socket says 172.21.15.120:37720) 2024-09-19T09:13:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.026+0000 7fa3677fe640 1 -- 172.21.15.120:0/2589945681 learned_addr learned my addr 172.21.15.120:0/2589945681 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.026+0000 7fa3667fc640 1 --2- 172.21.15.120:0/2589945681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa368102d50 0x7fa368198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.026+0000 7fa366ffd640 1 --2- 172.21.15.120:0/2589945681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa368101b60 0x7fa368197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.027+0000 7fa3677fe640 1 -- 172.21.15.120:0/2589945681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa368102d50 msgr2=0x7fa368198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.027+0000 7fa3677fe640 1 --2- 172.21.15.120:0/2589945681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa368102d50 0x7fa368198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.027+0000 7fa3677fe640 1 -- 172.21.15.120:0/2589945681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa368101b60 msgr2=0x7fa368197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.027+0000 7fa3677fe640 1 --2- 172.21.15.120:0/2589945681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa368101b60 0x7fa368197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.027+0000 7fa3677fe640 1 -- 172.21.15.120:0/2589945681 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36819fa10 con 0x7fa368103710 2024-09-19T09:13:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.027+0000 7fa3677fe640 1 --2- 172.21.15.120:0/2589945681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa368103710 0x7fa36819f3a0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fa36000eb10 tx=0x7fa36000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.027+0000 7fa36c8ef640 1 -- 172.21.15.120:0/2589945681 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa360004b20 con 0x7fa368103710 2024-09-19T09:13:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.027+0000 7fa36c8ef640 1 -- 172.21.15.120:0/2589945681 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa360004cc0 con 0x7fa368103710 2024-09-19T09:13:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.027+0000 7fa36d8f1640 1 -- 172.21.15.120:0/2589945681 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa36819fd00 con 0x7fa368103710 2024-09-19T09:13:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.027+0000 7fa36c8ef640 1 -- 172.21.15.120:0/2589945681 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa360010760 con 0x7fa368103710 2024-09-19T09:13:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.027+0000 7fa36d8f1640 1 -- 172.21.15.120:0/2589945681 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3681a01d0 con 0x7fa368103710 2024-09-19T09:13:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.029+0000 7fa36c8ef640 1 -- 172.21.15.120:0/2589945681 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa360005020 con 0x7fa368103710 2024-09-19T09:13:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.029+0000 7fa370690740 1 Processor -- start 2024-09-19T09:13:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.029+0000 7fa370690740 1 -- start start 2024-09-19T09:13:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.030+0000 7fa370690740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deed47c9a0 0x55deed47cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.030+0000 7fa370690740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deed47d2e0 0x55deed481da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.030+0000 7fa370690740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deed482370 0x55deed484830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.030+0000 7fa370690740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55deed3e02f0 con 0x55deed47d2e0 2024-09-19T09:13:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.030+0000 7fa370690740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55deed388270 con 0x55deed47c9a0 2024-09-19T09:13:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.030+0000 7fa370690740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55deed3dc2b0 con 0x55deed482370 2024-09-19T09:13:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.030+0000 7fa366ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deed47d2e0 0x55deed481da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.030+0000 7fa3667fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deed47c9a0 0x55deed47cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.030+0000 7fa366ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deed47d2e0 0x55deed481da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37734/0 (socket says 172.21.15.120:37734) 2024-09-19T09:13:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.030+0000 7fa366ffd640 1 -- 172.21.15.120:0/743325424 learned_addr learned my addr 172.21.15.120:0/743325424 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.030+0000 7fa3667fc640 1 --2- 172.21.15.120:0/743325424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deed482370 0x55deed484830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.030+0000 7fa36c8ef640 1 --2- 172.21.15.120:0/2589945681 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa3500a32f0 0x7fa3500a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.030+0000 7fa366ffd640 1 -- 172.21.15.120:0/743325424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deed482370 msgr2=0x55deed484830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.030+0000 7fa366ffd640 1 --2- 172.21.15.120:0/743325424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deed482370 0x55deed484830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.030+0000 7fa366ffd640 1 -- 172.21.15.120:0/743325424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deed47c9a0 msgr2=0x55deed47cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.030+0000 7fa366ffd640 1 --2- 172.21.15.120:0/743325424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deed47c9a0 0x55deed47cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.030+0000 7fa366ffd640 1 -- 172.21.15.120:0/743325424 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deed387340 con 0x55deed47d2e0 2024-09-19T09:13:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.030+0000 7fa3677fe640 1 --2- 172.21.15.120:0/2589945681 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa3500a32f0 0x7fa3500a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.030+0000 7fa36c8ef640 1 -- 172.21.15.120:0/2589945681 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa3600cfec0 con 0x7fa368103710 2024-09-19T09:13:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.030+0000 7fa366ffd640 1 --2- 172.21.15.120:0/743325424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deed47d2e0 0x55deed481da0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fa358009ca0 tx=0x7fa358011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e3e0771dbc6c6f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.030+0000 7fa34dffb640 1 -- 172.21.15.120:0/743325424 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa358009e50 con 0x55deed47d2e0 2024-09-19T09:13:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.030+0000 7fa34dffb640 1 -- 172.21.15.120:0/743325424 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3580035f0 con 0x55deed47d2e0 2024-09-19T09:13:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.030+0000 7fa34dffb640 1 -- 172.21.15.120:0/743325424 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa358019610 con 0x55deed47d2e0 2024-09-19T09:13:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.031+0000 7fa370690740 1 -- 172.21.15.120:0/743325424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deed47d2e0 msgr2=0x55deed481da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.031+0000 7fa370690740 1 --2- 172.21.15.120:0/743325424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deed47d2e0 0x55deed481da0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fa358009ca0 tx=0x7fa358011810 comp rx=0 tx=0).stop 2024-09-19T09:13:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.031+0000 7fa3677fe640 1 --2- 172.21.15.120:0/2589945681 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa3500a32f0 0x7fa3500a57b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa3600a4c20 tx=0x7fa360066f20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.031+0000 7fa370690740 1 -- 172.21.15.120:0/743325424 shutdown_connections 2024-09-19T09:13:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.031+0000 7fa370690740 1 --2- 172.21.15.120:0/743325424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deed482370 0x55deed484830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.031+0000 7fa370690740 1 --2- 172.21.15.120:0/743325424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deed47d2e0 0x55deed481da0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.031+0000 7fa370690740 1 --2- 172.21.15.120:0/743325424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deed47c9a0 0x55deed47cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.031+0000 7fa370690740 1 -- 172.21.15.120:0/743325424 >> 172.21.15.120:0/743325424 conn(0x55deed478010 msgr2=0x55deed478410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.031+0000 7fa370690740 1 -- 172.21.15.120:0/743325424 shutdown_connections 2024-09-19T09:13:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.031+0000 7fa370690740 1 -- 172.21.15.120:0/743325424 wait complete. 2024-09-19T09:13:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.032+0000 7fa370690740 1 -- start start 2024-09-19T09:13:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.032+0000 7fa370690740 1 Processor -- start 2024-09-19T09:13:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.032+0000 7fa370690740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deed47c9a0 0x55deed486c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.032+0000 7fa370690740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deed47d2e0 0x55deed487140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.032+0000 7fa370690740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deed482370 0x55deed48e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.032+0000 7fa370690740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55deed485340 con 0x55deed47d2e0 2024-09-19T09:13:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.032+0000 7fa370690740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55deed369290 con 0x55deed482370 2024-09-19T09:13:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.032+0000 7fa370690740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55deed4854c0 con 0x55deed47c9a0 2024-09-19T09:13:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.032+0000 7fa370690740 10 client.0 _ll_register_callbacks cb 0x55deed39a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.032+0000 7fa366ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deed47d2e0 0x55deed487140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.032+0000 7fa366ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deed47d2e0 0x55deed487140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37740/0 (socket says 172.21.15.120:37740) 2024-09-19T09:13:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.032+0000 7fa3667fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deed47c9a0 0x55deed486c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.032+0000 7fa366ffd640 1 -- 172.21.15.120:0/3643918511 learned_addr learned my addr 172.21.15.120:0/3643918511 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.032+0000 7fa3667fc640 1 --2- 172.21.15.120:0/3643918511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deed482370 0x55deed48e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.032+0000 7fa366ffd640 1 -- 172.21.15.120:0/3643918511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deed47c9a0 msgr2=0x55deed486c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.032+0000 7fa366ffd640 1 --2- 172.21.15.120:0/3643918511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deed47c9a0 0x55deed486c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.032+0000 7fa366ffd640 1 -- 172.21.15.120:0/3643918511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deed482370 msgr2=0x55deed48e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.032+0000 7fa366ffd640 1 --2- 172.21.15.120:0/3643918511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deed482370 0x55deed48e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.032+0000 7fa366ffd640 1 -- 172.21.15.120:0/3643918511 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deed0f3210 con 0x55deed47d2e0 2024-09-19T09:13:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.033+0000 7fa3667fc640 1 --2- 172.21.15.120:0/3643918511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deed47c9a0 0x55deed486c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:13:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.033+0000 7fa366ffd640 1 --2- 172.21.15.120:0/3643918511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deed47d2e0 0x55deed487140 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fa35801b040 tx=0x7fa3580041c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293129bf10937bc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.033+0000 7fa346ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.033+0000 7fa346ffd640 1 -- 172.21.15.120:0/3643918511 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa358028020 con 0x55deed47d2e0 2024-09-19T09:13:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.033+0000 7fa346ffd640 1 -- 172.21.15.120:0/3643918511 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa358016040 con 0x55deed47d2e0 2024-09-19T09:13:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.033+0000 7fa370690740 1 -- 172.21.15.120:0/3643918511 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3580035f0 con 0x55deed47d2e0 2024-09-19T09:13:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.033+0000 7fa346ffd640 1 -- 172.21.15.120:0/3643918511 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa358009e50 con 0x55deed47d2e0 2024-09-19T09:13:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.033+0000 7fa370690740 10 client.19383 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.033+0000 7fa370690740 1 -- 172.21.15.120:0/3643918511 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55deed1bf620 con 0x55deed47d2e0 2024-09-19T09:13:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.033+0000 7fa346ffd640 1 -- 172.21.15.120:0/3643918511 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3580035f0 con 0x55deed47d2e0 2024-09-19T09:13:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.033+0000 7fa346ffd640 1 -- 172.21.15.120:0/3643918511 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa358004bb0 con 0x55deed47d2e0 2024-09-19T09:13:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.033+0000 7fa370690740 10 client.19383 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.033+0000 7fa370690740 10 client.19383 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:13:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.033+0000 7fa370690740 20 client.19383 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.033+0000 7fa370690740 1 --2- 172.21.15.120:0/3643918511 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55deed4926b0 0x55deed494aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.033+0000 7fa370690740 4 client.19383 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:13:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.033+0000 7fa370690740 1 -- 172.21.15.120:0/3643918511 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55deed3eda20 con 0x55deed4926b0 2024-09-19T09:13:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.034+0000 7fa3667fc640 1 --2- 172.21.15.120:0/3643918511 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55deed4926b0 0x55deed494aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.034+0000 7fa346ffd640 10 client.19383 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.034+0000 7fa3667fc640 1 --2- 172.21.15.120:0/3643918511 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55deed4926b0 0x55deed494aa0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72f8accb7c4fd380 server_cookie=b1087b05836a22df in_seq=0 out_seq=0 2024-09-19T09:13:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.035+0000 7fa346ffd640 1 -- 172.21.15.120:0/3643918511 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55deed3eda20 con 0x55deed4926b0 2024-09-19T09:13:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.035+0000 7fa346ffd640 10 client.19383 handle_command_reply: tid=0 2024-09-19T09:13:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.035+0000 7fa370690740 1 client.19383 shutdown 2024-09-19T09:13:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.035+0000 7fa370690740 1 -- 172.21.15.120:0/3643918511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deed47d2e0 msgr2=0x55deed487140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.035+0000 7fa370690740 1 --2- 172.21.15.120:0/3643918511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deed47d2e0 0x55deed487140 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fa35801b040 tx=0x7fa3580041c0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.035+0000 7fa370690740 1 -- 172.21.15.120:0/3643918511 shutdown_connections 2024-09-19T09:13:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.035+0000 7fa370690740 1 --2- 172.21.15.120:0/3643918511 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55deed4926b0 0x55deed494aa0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.035+0000 7fa370690740 1 --2- 172.21.15.120:0/3643918511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deed482370 0x55deed48e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.035+0000 7fa370690740 1 --2- 172.21.15.120:0/3643918511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deed47d2e0 0x55deed487140 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.035+0000 7fa346ffd640 0 client.19383 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.035+0000 7fa370690740 1 --2- 172.21.15.120:0/3643918511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deed47c9a0 0x55deed486c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.035+0000 7fa370690740 1 -- 172.21.15.120:0/3643918511 >> 172.21.15.120:0/3643918511 conn(0x55deed478010 msgr2=0x55deed479820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.035+0000 7fa370690740 1 -- 172.21.15.120:0/3643918511 shutdown_connections 2024-09-19T09:13:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.035+0000 7fa370690740 1 -- 172.21.15.120:0/3643918511 wait complete. 2024-09-19T09:13:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.035+0000 7fa370690740 20 client.19383 trim_cache size 0 max 16384 2024-09-19T09:13:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.050+0000 7fa370690740 1 Processor -- start 2024-09-19T09:13:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.050+0000 7fa370690740 1 -- start start 2024-09-19T09:13:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.050+0000 7fa370690740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deed47c9a0 0x55deed4770b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.050+0000 7fa370690740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deed47d2e0 0x55deed48e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.050+0000 7fa370690740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deed482370 0x55deed479400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.050+0000 7fa370690740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa3681a8ff0 con 0x55deed47c9a0 2024-09-19T09:13:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.050+0000 7fa370690740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa3681aab30 con 0x55deed47d2e0 2024-09-19T09:13:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.050+0000 7fa370690740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa3681a96c0 con 0x55deed482370 2024-09-19T09:13:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.050+0000 7fa3667fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deed47c9a0 0x55deed4770b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.050+0000 7fa3667fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deed47c9a0 0x55deed4770b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37750/0 (socket says 172.21.15.120:37750) 2024-09-19T09:13:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.050+0000 7fa3667fc640 1 -- 172.21.15.120:0/97474477 learned_addr learned my addr 172.21.15.120:0/97474477 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.050+0000 7fa366ffd640 1 --2- 172.21.15.120:0/97474477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deed47d2e0 0x55deed48e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.050+0000 7fa3667fc640 1 --2- 172.21.15.120:0/97474477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deed482370 0x55deed479400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.050+0000 7fa3667fc640 1 -- 172.21.15.120:0/97474477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deed482370 msgr2=0x55deed479400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.050+0000 7fa3667fc640 1 --2- 172.21.15.120:0/97474477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deed482370 0x55deed479400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.050+0000 7fa3667fc640 1 -- 172.21.15.120:0/97474477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deed47d2e0 msgr2=0x55deed48e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.050+0000 7fa3667fc640 1 --2- 172.21.15.120:0/97474477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deed47d2e0 0x55deed48e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.050+0000 7fa3667fc640 1 -- 172.21.15.120:0/97474477 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deed494e10 con 0x55deed47c9a0 2024-09-19T09:13:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.050+0000 7fa3667fc640 1 --2- 172.21.15.120:0/97474477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deed47c9a0 0x55deed4770b0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fa35c000c00 tx=0x7fa35c0315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7218056745f1bb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.051+0000 7fa34d7fa640 1 -- 172.21.15.120:0/97474477 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c035040 con 0x55deed47c9a0 2024-09-19T09:13:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.051+0000 7fa34d7fa640 1 -- 172.21.15.120:0/97474477 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa35c0035f0 con 0x55deed47c9a0 2024-09-19T09:13:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.051+0000 7fa34d7fa640 1 -- 172.21.15.120:0/97474477 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c04f630 con 0x55deed47c9a0 2024-09-19T09:13:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.051+0000 7fa370690740 1 -- 172.21.15.120:0/97474477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deed47c9a0 msgr2=0x55deed4770b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.051+0000 7fa370690740 1 --2- 172.21.15.120:0/97474477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deed47c9a0 0x55deed4770b0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fa35c000c00 tx=0x7fa35c0315b0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.051+0000 7fa370690740 1 -- 172.21.15.120:0/97474477 shutdown_connections 2024-09-19T09:13:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.051+0000 7fa370690740 1 --2- 172.21.15.120:0/97474477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deed482370 0x55deed479400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.051+0000 7fa370690740 1 --2- 172.21.15.120:0/97474477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deed47d2e0 0x55deed48e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.051+0000 7fa370690740 1 --2- 172.21.15.120:0/97474477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deed47c9a0 0x55deed4770b0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.051+0000 7fa370690740 1 -- 172.21.15.120:0/97474477 >> 172.21.15.120:0/97474477 conn(0x55deed478010 msgr2=0x55deed4769a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.051+0000 7fa370690740 1 -- 172.21.15.120:0/97474477 shutdown_connections 2024-09-19T09:13:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.051+0000 7fa370690740 1 -- 172.21.15.120:0/97474477 wait complete. 2024-09-19T09:13:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.052+0000 7fa370690740 1 -- start start 2024-09-19T09:13:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.052+0000 7fa370690740 1 Processor -- start 2024-09-19T09:13:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.052+0000 7fa370690740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deed47c9a0 0x55deed48b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.052+0000 7fa370690740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deed47d2e0 0x55deed48bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.052+0000 7fa370690740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deed482370 0x55deed48cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.052+0000 7fa370690740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa3681aa460 con 0x55deed47c9a0 2024-09-19T09:13:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.052+0000 7fa370690740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa3681a9d90 con 0x55deed47d2e0 2024-09-19T09:13:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.052+0000 7fa370690740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa3681a6fb0 con 0x55deed482370 2024-09-19T09:13:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.052+0000 7fa370690740 10 client.0 _ll_register_callbacks cb 0x7fa358005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:13:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.052+0000 7fa3667fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deed47c9a0 0x55deed48b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.052+0000 7fa3667fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deed47c9a0 0x55deed48b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37766/0 (socket says 172.21.15.120:37766) 2024-09-19T09:13:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.052+0000 7fa3667fc640 1 -- 172.21.15.120:0/1667538597 learned_addr learned my addr 172.21.15.120:0/1667538597 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:13:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.052+0000 7fa366ffd640 1 --2- 172.21.15.120:0/1667538597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deed47d2e0 0x55deed48bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.052+0000 7fa3667fc640 1 --2- 172.21.15.120:0/1667538597 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deed482370 0x55deed48cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.053+0000 7fa3667fc640 1 -- 172.21.15.120:0/1667538597 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deed482370 msgr2=0x55deed48cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.053+0000 7fa3667fc640 1 --2- 172.21.15.120:0/1667538597 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deed482370 0x55deed48cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.053+0000 7fa3667fc640 1 -- 172.21.15.120:0/1667538597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deed47d2e0 msgr2=0x55deed48bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:13:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.053+0000 7fa3667fc640 1 --2- 172.21.15.120:0/1667538597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deed47d2e0 0x55deed48bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.053+0000 7fa3667fc640 1 -- 172.21.15.120:0/1667538597 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deed48d230 con 0x55deed47c9a0 2024-09-19T09:13:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.053+0000 7fa366ffd640 1 --2- 172.21.15.120:0/1667538597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deed47d2e0 0x55deed48bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:13:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.053+0000 7fa3667fc640 1 --2- 172.21.15.120:0/1667538597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deed47c9a0 0x55deed48b6d0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fa35c009850 tx=0x7fa35c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3e5491a34c8f79 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:13:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.053+0000 7fa346ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:13:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.053+0000 7fa346ffd640 1 -- 172.21.15.120:0/1667538597 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c035070 con 0x55deed47c9a0 2024-09-19T09:13:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.053+0000 7fa346ffd640 1 -- 172.21.15.120:0/1667538597 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa35c0037a0 con 0x55deed47c9a0 2024-09-19T09:13:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.053+0000 7fa370690740 1 -- 172.21.15.120:0/1667538597 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa35c0035f0 con 0x55deed47c9a0 2024-09-19T09:13:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.053+0000 7fa346ffd640 1 -- 172.21.15.120:0/1667538597 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa35c042070 con 0x55deed47c9a0 2024-09-19T09:13:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.053+0000 7fa370690740 10 client.19389 fetch_fsmap learned FSMap version 32 2024-09-19T09:13:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.053+0000 7fa370690740 1 -- 172.21.15.120:0/1667538597 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55deed48e540 con 0x55deed47c9a0 2024-09-19T09:13:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.053+0000 7fa346ffd640 1 -- 172.21.15.120:0/1667538597 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa35c0035f0 con 0x55deed47c9a0 2024-09-19T09:13:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.053+0000 7fa346ffd640 1 -- 172.21.15.120:0/1667538597 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa35c04e040 con 0x55deed47c9a0 2024-09-19T09:13:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.053+0000 7fa370690740 10 client.19389 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:13:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.053+0000 7fa370690740 10 client.19389 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:13:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.053+0000 7fa370690740 20 client.19389 populate_metadata read hostname 'smithi120' 2024-09-19T09:13:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.053+0000 7fa370690740 1 --2- 172.21.15.120:0/1667538597 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55deed4fd9d0 0x55deed4ffdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:13:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.053+0000 7fa370690740 4 client.19389 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:13:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.053+0000 7fa370690740 1 -- 172.21.15.120:0/1667538597 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55deed3e7d70 con 0x55deed4fd9d0 2024-09-19T09:13:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.054+0000 7fa366ffd640 1 --2- 172.21.15.120:0/1667538597 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55deed4fd9d0 0x55deed4ffdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:13:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.054+0000 7fa366ffd640 1 --2- 172.21.15.120:0/1667538597 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55deed4fd9d0 0x55deed4ffdf0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bc377d56cd43887 server_cookie=8c6992809c7e7637 in_seq=0 out_seq=0 2024-09-19T09:13:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.054+0000 7fa346ffd640 10 client.19389 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:51.072 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1005639 9.08 MB/sec execute 466 sec latency 17.341 ms 2024-09-19T09:13:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.132+0000 7fa346ffd640 1 -- 172.21.15.120:0/1667538597 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55deed3e7d70 con 0x55deed4fd9d0 2024-09-19T09:13:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.132+0000 7fa346ffd640 10 client.19389 handle_command_reply: tid=0 2024-09-19T09:13:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.132+0000 7fa370690740 1 client.19389 shutdown 2024-09-19T09:13:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.132+0000 7fa370690740 1 -- 172.21.15.120:0/1667538597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deed47c9a0 msgr2=0x55deed48b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.132+0000 7fa370690740 1 --2- 172.21.15.120:0/1667538597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deed47c9a0 0x55deed48b6d0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fa35c009850 tx=0x7fa35c005780 comp rx=0 tx=0).stop 2024-09-19T09:13:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.132+0000 7fa370690740 1 -- 172.21.15.120:0/1667538597 shutdown_connections 2024-09-19T09:13:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.132+0000 7fa370690740 1 --2- 172.21.15.120:0/1667538597 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55deed4fd9d0 0x55deed4ffdf0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.132+0000 7fa370690740 1 --2- 172.21.15.120:0/1667538597 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deed482370 0x55deed48cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.132+0000 7fa370690740 1 --2- 172.21.15.120:0/1667538597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deed47d2e0 0x55deed48bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.132+0000 7fa346ffd640 0 client.19389 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:13:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.132+0000 7fa370690740 1 --2- 172.21.15.120:0/1667538597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deed47c9a0 0x55deed48b6d0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.132+0000 7fa370690740 1 -- 172.21.15.120:0/1667538597 >> 172.21.15.120:0/1667538597 conn(0x55deed478010 msgr2=0x55deed4771c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.132+0000 7fa370690740 1 -- 172.21.15.120:0/1667538597 shutdown_connections 2024-09-19T09:13:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.132+0000 7fa370690740 1 -- 172.21.15.120:0/1667538597 wait complete. 2024-09-19T09:13:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.132+0000 7fa370690740 20 client.19389 trim_cache size 0 max 16384 2024-09-19T09:13:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:13:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:13:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:13:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:13:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.133+0000 7fa36d8f1640 1 -- 172.21.15.120:0/2589945681 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa3500a32f0 msgr2=0x7fa3500a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.133+0000 7fa36d8f1640 1 --2- 172.21.15.120:0/2589945681 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa3500a32f0 0x7fa3500a57b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa3600a4c20 tx=0x7fa360066f20 comp rx=0 tx=0).stop 2024-09-19T09:13:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.133+0000 7fa36d8f1640 1 -- 172.21.15.120:0/2589945681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa368103710 msgr2=0x7fa36819f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:13:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.133+0000 7fa36d8f1640 1 --2- 172.21.15.120:0/2589945681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa368103710 0x7fa36819f3a0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fa36000eb10 tx=0x7fa36000c2d0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.133+0000 7fa36d8f1640 1 -- 172.21.15.120:0/2589945681 shutdown_connections 2024-09-19T09:13:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.133+0000 7fa36d8f1640 1 --2- 172.21.15.120:0/2589945681 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa3500a32f0 0x7fa3500a57b0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.133+0000 7fa36d8f1640 1 --2- 172.21.15.120:0/2589945681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa368103710 0x7fa36819f3a0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.133+0000 7fa36d8f1640 1 --2- 172.21.15.120:0/2589945681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa368102d50 0x7fa368198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.133+0000 7fa36d8f1640 1 --2- 172.21.15.120:0/2589945681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa368101b60 0x7fa368197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:13:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.133+0000 7fa36d8f1640 1 -- 172.21.15.120:0/2589945681 >> 172.21.15.120:0/2589945681 conn(0x7fa3680fd2e0 msgr2=0x7fa368106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:13:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.133+0000 7fa36d8f1640 1 -- 172.21.15.120:0/2589945681 shutdown_connections 2024-09-19T09:13:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:13:51.133+0000 7fa36d8f1640 1 -- 172.21.15.120:0/2589945681 wait complete. 2024-09-19T09:13:52.072 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1008173 9.08 MB/sec execute 467 sec latency 48.341 ms 2024-09-19T09:13:53.072 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1010410 9.09 MB/sec execute 468 sec latency 55.381 ms 2024-09-19T09:13:53.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:52 smithi196 ceph-mon[35150]: pgmap v673: 97 pgs: 97 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 768 KiB/s wr, 88 op/s 2024-09-19T09:13:53.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:52 smithi153 ceph-mon[35071]: pgmap v673: 97 pgs: 97 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 768 KiB/s wr, 88 op/s 2024-09-19T09:13:53.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:52 smithi120 ceph-mon[31150]: pgmap v673: 97 pgs: 97 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 768 KiB/s wr, 88 op/s 2024-09-19T09:13:54.072 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1013097 9.10 MB/sec execute 469 sec latency 29.246 ms 2024-09-19T09:13:55.073 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1016304 9.12 MB/sec execute 470 sec latency 14.603 ms 2024-09-19T09:13:55.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:54 smithi196 ceph-mon[35150]: pgmap v674: 97 pgs: 97 active+clean; 35 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 970 KiB/s wr, 157 op/s 2024-09-19T09:13:55.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:54 smithi153 ceph-mon[35071]: pgmap v674: 97 pgs: 97 active+clean; 35 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 970 KiB/s wr, 157 op/s 2024-09-19T09:13:55.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:54 smithi120 ceph-mon[31150]: pgmap v674: 97 pgs: 97 active+clean; 35 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 970 KiB/s wr, 157 op/s 2024-09-19T09:13:56.073 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1019069 9.13 MB/sec execute 471 sec latency 12.141 ms 2024-09-19T09:13:57.073 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1021848 9.14 MB/sec execute 472 sec latency 12.805 ms 2024-09-19T09:13:57.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:56 smithi196 ceph-mon[35150]: pgmap v675: 97 pgs: 97 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 802 KiB/s wr, 167 op/s 2024-09-19T09:13:57.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:56 smithi153 ceph-mon[35071]: pgmap v675: 97 pgs: 97 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 802 KiB/s wr, 167 op/s 2024-09-19T09:13:57.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:56 smithi120 ceph-mon[31150]: pgmap v675: 97 pgs: 97 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 802 KiB/s wr, 167 op/s 2024-09-19T09:13:58.073 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1024197 9.15 MB/sec execute 473 sec latency 18.022 ms 2024-09-19T09:13:59.073 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1027273 9.17 MB/sec execute 474 sec latency 7.694 ms 2024-09-19T09:13:59.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:13:58 smithi196 ceph-mon[35150]: pgmap v676: 97 pgs: 97 active+clean; 36 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.3 MiB/s wr, 246 op/s 2024-09-19T09:13:59.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:13:58 smithi153 ceph-mon[35071]: pgmap v676: 97 pgs: 97 active+clean; 36 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.3 MiB/s wr, 246 op/s 2024-09-19T09:13:59.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:13:58 smithi120 ceph-mon[31150]: pgmap v676: 97 pgs: 97 active+clean; 36 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.3 MiB/s wr, 246 op/s 2024-09-19T09:14:00.073 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1030389 9.19 MB/sec execute 475 sec latency 34.839 ms 2024-09-19T09:14:01.073 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1033104 9.19 MB/sec execute 476 sec latency 11.951 ms 2024-09-19T09:14:01.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:00 smithi196 ceph-mon[35150]: pgmap v677: 97 pgs: 97 active+clean; 37 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.4 MiB/s wr, 252 op/s 2024-09-19T09:14:01.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:00 smithi153 ceph-mon[35071]: pgmap v677: 97 pgs: 97 active+clean; 37 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.4 MiB/s wr, 252 op/s 2024-09-19T09:14:01.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:00 smithi120 ceph-mon[31150]: pgmap v677: 97 pgs: 97 active+clean; 37 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.4 MiB/s wr, 252 op/s 2024-09-19T09:14:02.073 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1035349 9.20 MB/sec execute 477 sec latency 11.349 ms 2024-09-19T09:14:03.073 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1038071 9.21 MB/sec execute 478 sec latency 32.996 ms 2024-09-19T09:14:03.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:02 smithi196 ceph-mon[35150]: pgmap v678: 97 pgs: 97 active+clean; 41 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 276 op/s 2024-09-19T09:14:03.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:02 smithi153 ceph-mon[35071]: pgmap v678: 97 pgs: 97 active+clean; 41 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 276 op/s 2024-09-19T09:14:03.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:02 smithi120 ceph-mon[31150]: pgmap v678: 97 pgs: 97 active+clean; 41 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 276 op/s 2024-09-19T09:14:03.663 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:14:03.931 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:14:04.073 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1041161 9.23 MB/sec execute 479 sec latency 16.761 ms 2024-09-19T09:14:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.225+0000 7f2395d23640 1 -- 172.21.15.120:0/3692125491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2390103d60 msgr2=0x7f2390104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.225+0000 7f2395d23640 1 --2- 172.21.15.120:0/3692125491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2390103d60 0x7f2390104160 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f2380009900 tx=0x7f238002a410 comp rx=0 tx=0).stop 2024-09-19T09:14:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.225+0000 7f2395d23640 1 -- 172.21.15.120:0/3692125491 shutdown_connections 2024-09-19T09:14:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.225+0000 7f2395d23640 1 --2- 172.21.15.120:0/3692125491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2390105910 0x7f239010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.225+0000 7f2395d23640 1 --2- 172.21.15.120:0/3692125491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2390104f50 0x7f23901053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.225+0000 7f2395d23640 1 --2- 172.21.15.120:0/3692125491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2390103d60 0x7f2390104160 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.225+0000 7f2395d23640 1 -- 172.21.15.120:0/3692125491 >> 172.21.15.120:0/3692125491 conn(0x7f23900ff4e0 msgr2=0x7f2390101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.225+0000 7f2395d23640 1 -- 172.21.15.120:0/3692125491 shutdown_connections 2024-09-19T09:14:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.225+0000 7f2395d23640 1 -- 172.21.15.120:0/3692125491 wait complete. 2024-09-19T09:14:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.226+0000 7f2395d23640 1 Processor -- start 2024-09-19T09:14:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.226+0000 7f2395d23640 1 -- start start 2024-09-19T09:14:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.226+0000 7f2395d23640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2390103d60 0x7f2390199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.226+0000 7f2395d23640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2390104f50 0x7f239019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.226+0000 7f2395d23640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2390105910 0x7f23901a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.226+0000 7f2395d23640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f239010fa10 con 0x7f2390103d60 2024-09-19T09:14:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.226+0000 7f2395d23640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f239010f890 con 0x7f2390104f50 2024-09-19T09:14:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.226+0000 7f2395d23640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f239010fb90 con 0x7f2390105910 2024-09-19T09:14:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.226+0000 7f238f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2390103d60 0x7f2390199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.226+0000 7f238f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2390103d60 0x7f2390199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59292/0 (socket says 172.21.15.120:59292) 2024-09-19T09:14:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.226+0000 7f238f7fe640 1 -- 172.21.15.120:0/309852666 learned_addr learned my addr 172.21.15.120:0/309852666 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.226+0000 7f238ffff640 1 --2- 172.21.15.120:0/309852666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2390105910 0x7f23901a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.226+0000 7f238effd640 1 --2- 172.21.15.120:0/309852666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2390104f50 0x7f239019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.226+0000 7f238f7fe640 1 -- 172.21.15.120:0/309852666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2390105910 msgr2=0x7f23901a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.226+0000 7f238f7fe640 1 --2- 172.21.15.120:0/309852666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2390105910 0x7f23901a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.226+0000 7f238f7fe640 1 -- 172.21.15.120:0/309852666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2390104f50 msgr2=0x7f239019a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.226+0000 7f238f7fe640 1 --2- 172.21.15.120:0/309852666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2390104f50 0x7f239019a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.226+0000 7f238f7fe640 1 -- 172.21.15.120:0/309852666 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23901a1be0 con 0x7f2390103d60 2024-09-19T09:14:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.227+0000 7f238f7fe640 1 --2- 172.21.15.120:0/309852666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2390103d60 0x7f2390199fc0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f2380031440 tx=0x7f2380032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.227+0000 7f238cff9640 1 -- 172.21.15.120:0/309852666 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238003f030 con 0x7f2390103d60 2024-09-19T09:14:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.227+0000 7f238cff9640 1 -- 172.21.15.120:0/309852666 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2380003ce0 con 0x7f2390103d60 2024-09-19T09:14:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.227+0000 7f2395d23640 1 -- 172.21.15.120:0/309852666 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23901a1e70 con 0x7f2390103d60 2024-09-19T09:14:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.227+0000 7f238cff9640 1 -- 172.21.15.120:0/309852666 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380035070 con 0x7f2390103d60 2024-09-19T09:14:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.227+0000 7f2395d23640 1 -- 172.21.15.120:0/309852666 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23901a2280 con 0x7f2390103d60 2024-09-19T09:14:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.230+0000 7f238cff9640 1 -- 172.21.15.120:0/309852666 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f2380042030 con 0x7f2390103d60 2024-09-19T09:14:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.230+0000 7f2398aba740 1 Processor -- start 2024-09-19T09:14:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.230+0000 7f2398aba740 1 -- start start 2024-09-19T09:14:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.230+0000 7f2398aba740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d49b6aa9a0 0x55d49b6aada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.230+0000 7f2398aba740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d49b6ab2e0 0x55d49b6afda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.230+0000 7f2398aba740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d49b6b0370 0x55d49b6b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.230+0000 7f2398aba740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d49b60e2f0 con 0x55d49b6b0370 2024-09-19T09:14:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.230+0000 7f2398aba740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d49b5b6270 con 0x55d49b6ab2e0 2024-09-19T09:14:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.230+0000 7f2398aba740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d49b60a2b0 con 0x55d49b6aa9a0 2024-09-19T09:14:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.231+0000 7f238effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d49b6b0370 0x55d49b6b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.231+0000 7f238ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d49b6aa9a0 0x55d49b6aada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.231+0000 7f238effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d49b6b0370 0x55d49b6b2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59298/0 (socket says 172.21.15.120:59298) 2024-09-19T09:14:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.231+0000 7f238f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d49b6ab2e0 0x55d49b6afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.231+0000 7f238effd640 1 -- 172.21.15.120:0/385248898 learned_addr learned my addr 172.21.15.120:0/385248898 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.231+0000 7f238ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d49b6aa9a0 0x55d49b6aada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:60086/0 (socket says 172.21.15.120:60086) 2024-09-19T09:14:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.231+0000 7f238cff9640 1 --2- 172.21.15.120:0/309852666 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f23780a32f0 0x7f23780a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.231+0000 7f238ffff640 1 -- 172.21.15.120:0/385248898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d49b6ab2e0 msgr2=0x55d49b6afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.231+0000 7f238ffff640 1 --2- 172.21.15.120:0/385248898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d49b6ab2e0 0x55d49b6afda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.231+0000 7f238ffff640 1 -- 172.21.15.120:0/385248898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d49b6b0370 msgr2=0x55d49b6b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.231+0000 7f238ffff640 1 --2- 172.21.15.120:0/385248898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d49b6b0370 0x55d49b6b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.231+0000 7f238ffff640 1 -- 172.21.15.120:0/385248898 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d49b5b5340 con 0x55d49b6aa9a0 2024-09-19T09:14:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.231+0000 7f238effd640 1 --2- 172.21.15.120:0/385248898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d49b6b0370 0x55d49b6b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.231+0000 7f238cff9640 1 -- 172.21.15.120:0/309852666 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f23800efeb0 con 0x7f2390103d60 2024-09-19T09:14:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.231+0000 7f238ffff640 1 --2- 172.21.15.120:0/309852666 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f23780a32f0 0x7f23780a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.231+0000 7f238ffff640 1 --2- 172.21.15.120:0/385248898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d49b6aa9a0 0x55d49b6aada0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2388009f00 tx=0x7f2388011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2f577cc77459f15 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.231+0000 7f2375ffb640 1 -- 172.21.15.120:0/385248898 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388003cb0 con 0x55d49b6aa9a0 2024-09-19T09:14:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.231+0000 7f238ffff640 1 --2- 172.21.15.120:0/309852666 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f23780a32f0 0x7f23780a57b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2388020000 tx=0x7f2388021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.232+0000 7f2375ffb640 1 -- 172.21.15.120:0/385248898 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2388025bd0 con 0x55d49b6aa9a0 2024-09-19T09:14:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.232+0000 7f2375ffb640 1 -- 172.21.15.120:0/385248898 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238801bc20 con 0x55d49b6aa9a0 2024-09-19T09:14:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.232+0000 7f2398aba740 1 -- 172.21.15.120:0/385248898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d49b6aa9a0 msgr2=0x55d49b6aada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.232+0000 7f2398aba740 1 --2- 172.21.15.120:0/385248898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d49b6aa9a0 0x55d49b6aada0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2388009f00 tx=0x7f2388011810 comp rx=0 tx=0).stop 2024-09-19T09:14:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.232+0000 7f2398aba740 1 -- 172.21.15.120:0/385248898 shutdown_connections 2024-09-19T09:14:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.232+0000 7f2398aba740 1 --2- 172.21.15.120:0/385248898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d49b6b0370 0x55d49b6b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.232+0000 7f2398aba740 1 --2- 172.21.15.120:0/385248898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d49b6ab2e0 0x55d49b6afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.232+0000 7f2398aba740 1 --2- 172.21.15.120:0/385248898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d49b6aa9a0 0x55d49b6aada0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.232+0000 7f2398aba740 1 -- 172.21.15.120:0/385248898 >> 172.21.15.120:0/385248898 conn(0x55d49b6a6010 msgr2=0x55d49b6a6410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.232+0000 7f2398aba740 1 -- 172.21.15.120:0/385248898 shutdown_connections 2024-09-19T09:14:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.232+0000 7f2398aba740 1 -- 172.21.15.120:0/385248898 wait complete. 2024-09-19T09:14:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.233+0000 7f2398aba740 1 -- start start 2024-09-19T09:14:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.233+0000 7f2398aba740 1 Processor -- start 2024-09-19T09:14:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.233+0000 7f2398aba740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d49b6aa9a0 0x55d49b6b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.233+0000 7f2398aba740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d49b6ab2e0 0x55d49b6b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.233+0000 7f2398aba740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d49b6b0370 0x55d49b6bc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.233+0000 7f2398aba740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d49b6b3340 con 0x55d49b6aa9a0 2024-09-19T09:14:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.233+0000 7f2398aba740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d49b597290 con 0x55d49b6b0370 2024-09-19T09:14:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.233+0000 7f2398aba740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d49b6b34c0 con 0x55d49b6ab2e0 2024-09-19T09:14:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.233+0000 7f2398aba740 10 client.0 _ll_register_callbacks cb 0x55d49b5c8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.233+0000 7f238f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d49b6aa9a0 0x55d49b6b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.233+0000 7f238f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d49b6aa9a0 0x55d49b6b4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59312/0 (socket says 172.21.15.120:59312) 2024-09-19T09:14:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.233+0000 7f238f7fe640 1 -- 172.21.15.120:0/4076189389 learned_addr learned my addr 172.21.15.120:0/4076189389 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.233+0000 7f238ffff640 1 --2- 172.21.15.120:0/4076189389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d49b6b0370 0x55d49b6bc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.233+0000 7f238effd640 1 --2- 172.21.15.120:0/4076189389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d49b6ab2e0 0x55d49b6b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.234+0000 7f238f7fe640 1 -- 172.21.15.120:0/4076189389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d49b6ab2e0 msgr2=0x55d49b6b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.234+0000 7f238f7fe640 1 --2- 172.21.15.120:0/4076189389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d49b6ab2e0 0x55d49b6b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.234+0000 7f238f7fe640 1 -- 172.21.15.120:0/4076189389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d49b6b0370 msgr2=0x55d49b6bc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.234+0000 7f238f7fe640 1 --2- 172.21.15.120:0/4076189389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d49b6b0370 0x55d49b6bc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.234+0000 7f238f7fe640 1 -- 172.21.15.120:0/4076189389 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d49b321210 con 0x55d49b6aa9a0 2024-09-19T09:14:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.234+0000 7f238f7fe640 1 --2- 172.21.15.120:0/4076189389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d49b6aa9a0 0x55d49b6b4ca0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f23800b8ea0 tx=0x7f238003b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc83d68a74e8b8b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.234+0000 7f236effd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:14:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.234+0000 7f236effd640 1 -- 172.21.15.120:0/4076189389 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380049070 con 0x55d49b6aa9a0 2024-09-19T09:14:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.234+0000 7f236effd640 1 -- 172.21.15.120:0/4076189389 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f23800b8b40 con 0x55d49b6aa9a0 2024-09-19T09:14:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.234+0000 7f2398aba740 1 -- 172.21.15.120:0/4076189389 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2388025bd0 con 0x55d49b6aa9a0 2024-09-19T09:14:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.234+0000 7f236effd640 1 -- 172.21.15.120:0/4076189389 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f23800bc2a0 con 0x55d49b6aa9a0 2024-09-19T09:14:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.234+0000 7f2398aba740 10 client.19401 fetch_fsmap learned FSMap version 32 2024-09-19T09:14:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.234+0000 7f2398aba740 1 -- 172.21.15.120:0/4076189389 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d49b3ed620 con 0x55d49b6aa9a0 2024-09-19T09:14:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.234+0000 7f236effd640 1 -- 172.21.15.120:0/4076189389 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2388025bd0 con 0x55d49b6aa9a0 2024-09-19T09:14:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.234+0000 7f236effd640 1 -- 172.21.15.120:0/4076189389 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f238003b460 con 0x55d49b6aa9a0 2024-09-19T09:14:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.234+0000 7f2398aba740 10 client.19401 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:14:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.234+0000 7f2398aba740 10 client.19401 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:14:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.235+0000 7f2398aba740 20 client.19401 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.235+0000 7f2398aba740 1 --2- 172.21.15.120:0/4076189389 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d49b6c0890 0x55d49b6c2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.235+0000 7f2398aba740 4 client.19401 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:14:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.235+0000 7f2398aba740 1 -- 172.21.15.120:0/4076189389 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d49b61ba20 con 0x55d49b6c0890 2024-09-19T09:14:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.235+0000 7f238effd640 1 --2- 172.21.15.120:0/4076189389 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d49b6c0890 0x55d49b6c2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.235+0000 7f236effd640 10 client.19401 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.235+0000 7f238effd640 1 --2- 172.21.15.120:0/4076189389 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d49b6c0890 0x55d49b6c2ca0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6465ede32e60ffa server_cookie=da628c4c8e4b241c in_seq=0 out_seq=0 2024-09-19T09:14:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.236+0000 7f236effd640 1 -- 172.21.15.120:0/4076189389 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d49b61ba20 con 0x55d49b6c0890 2024-09-19T09:14:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.236+0000 7f236effd640 10 client.19401 handle_command_reply: tid=0 2024-09-19T09:14:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.236+0000 7f2398aba740 1 client.19401 shutdown 2024-09-19T09:14:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.236+0000 7f2398aba740 1 -- 172.21.15.120:0/4076189389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d49b6aa9a0 msgr2=0x55d49b6b4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.236+0000 7f2398aba740 1 --2- 172.21.15.120:0/4076189389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d49b6aa9a0 0x55d49b6b4ca0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f23800b8ea0 tx=0x7f238003b800 comp rx=0 tx=0).stop 2024-09-19T09:14:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.236+0000 7f2398aba740 1 -- 172.21.15.120:0/4076189389 shutdown_connections 2024-09-19T09:14:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.236+0000 7f2398aba740 1 --2- 172.21.15.120:0/4076189389 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d49b6c0890 0x55d49b6c2ca0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.236+0000 7f2398aba740 1 --2- 172.21.15.120:0/4076189389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d49b6b0370 0x55d49b6bc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.236+0000 7f236effd640 0 client.19401 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.236+0000 7f2398aba740 1 --2- 172.21.15.120:0/4076189389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d49b6ab2e0 0x55d49b6b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.236+0000 7f2398aba740 1 --2- 172.21.15.120:0/4076189389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d49b6aa9a0 0x55d49b6b4ca0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.236+0000 7f2398aba740 1 -- 172.21.15.120:0/4076189389 >> 172.21.15.120:0/4076189389 conn(0x55d49b6a6010 msgr2=0x55d49b6a7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.236+0000 7f2398aba740 1 -- 172.21.15.120:0/4076189389 shutdown_connections 2024-09-19T09:14:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.236+0000 7f2398aba740 1 -- 172.21.15.120:0/4076189389 wait complete. 2024-09-19T09:14:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.236+0000 7f2398aba740 20 client.19401 trim_cache size 0 max 16384 2024-09-19T09:14:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.253+0000 7f2398aba740 1 Processor -- start 2024-09-19T09:14:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.253+0000 7f2398aba740 1 -- start start 2024-09-19T09:14:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.254+0000 7f2398aba740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d49b6a6010 0x55d49b6b5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.254+0000 7f2398aba740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d49b6aa9a0 0x55d49b6bc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.254+0000 7f2398aba740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d49b6ab2e0 0x55d49b6a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.254+0000 7f2398aba740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f23901ab1c0 con 0x55d49b6a6010 2024-09-19T09:14:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.254+0000 7f2398aba740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f23901acd00 con 0x55d49b6ab2e0 2024-09-19T09:14:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.254+0000 7f2398aba740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f23901ab890 con 0x55d49b6aa9a0 2024-09-19T09:14:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.254+0000 7f238f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d49b6a6010 0x55d49b6b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.254+0000 7f238f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d49b6a6010 0x55d49b6b5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59318/0 (socket says 172.21.15.120:59318) 2024-09-19T09:14:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.254+0000 7f238f7fe640 1 -- 172.21.15.120:0/578190209 learned_addr learned my addr 172.21.15.120:0/578190209 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.254+0000 7f238effd640 1 --2- 172.21.15.120:0/578190209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d49b6aa9a0 0x55d49b6bc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.254+0000 7f238ffff640 1 --2- 172.21.15.120:0/578190209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d49b6ab2e0 0x55d49b6a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.254+0000 7f238f7fe640 1 -- 172.21.15.120:0/578190209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d49b6aa9a0 msgr2=0x55d49b6bc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.254+0000 7f238f7fe640 1 --2- 172.21.15.120:0/578190209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d49b6aa9a0 0x55d49b6bc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.254+0000 7f238f7fe640 1 -- 172.21.15.120:0/578190209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d49b6ab2e0 msgr2=0x55d49b6a50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.254+0000 7f238f7fe640 1 --2- 172.21.15.120:0/578190209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d49b6ab2e0 0x55d49b6a50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.254+0000 7f238f7fe640 1 -- 172.21.15.120:0/578190209 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d49b6b2760 con 0x55d49b6a6010 2024-09-19T09:14:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.254+0000 7f238f7fe640 1 --2- 172.21.15.120:0/578190209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d49b6a6010 0x55d49b6b5270 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f2380003fb0 tx=0x7f238002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a609d367061a825 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.254+0000 7f23757fa640 1 -- 172.21.15.120:0/578190209 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380049070 con 0x55d49b6a6010 2024-09-19T09:14:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.254+0000 7f23757fa640 1 -- 172.21.15.120:0/578190209 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23800bcbd0 con 0x55d49b6a6010 2024-09-19T09:14:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.254+0000 7f23757fa640 1 -- 172.21.15.120:0/578190209 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380005d50 con 0x55d49b6a6010 2024-09-19T09:14:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.255+0000 7f2398aba740 1 -- 172.21.15.120:0/578190209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d49b6a6010 msgr2=0x55d49b6b5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.255+0000 7f2398aba740 1 --2- 172.21.15.120:0/578190209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d49b6a6010 0x55d49b6b5270 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f2380003fb0 tx=0x7f238002a410 comp rx=0 tx=0).stop 2024-09-19T09:14:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.255+0000 7f2398aba740 1 -- 172.21.15.120:0/578190209 shutdown_connections 2024-09-19T09:14:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.255+0000 7f2398aba740 1 --2- 172.21.15.120:0/578190209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d49b6ab2e0 0x55d49b6a50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.255+0000 7f2398aba740 1 --2- 172.21.15.120:0/578190209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d49b6aa9a0 0x55d49b6bc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.255+0000 7f2398aba740 1 --2- 172.21.15.120:0/578190209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d49b6a6010 0x55d49b6b5270 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.255+0000 7f2398aba740 1 -- 172.21.15.120:0/578190209 >> 172.21.15.120:0/578190209 conn(0x55d49b6aff00 msgr2=0x55d49b6a49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.255+0000 7f2398aba740 1 -- 172.21.15.120:0/578190209 shutdown_connections 2024-09-19T09:14:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.255+0000 7f2398aba740 1 -- 172.21.15.120:0/578190209 wait complete. 2024-09-19T09:14:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.255+0000 7f2398aba740 1 -- start start 2024-09-19T09:14:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.256+0000 7f2398aba740 1 Processor -- start 2024-09-19T09:14:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.256+0000 7f2398aba740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d49b6a6010 0x55d49b6b9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.256+0000 7f2398aba740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d49b6aa9a0 0x55d49b6ba230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.256+0000 7f2398aba740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d49b6ab2e0 0x55d49b6bb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.256+0000 7f2398aba740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f23901ac630 con 0x55d49b6a6010 2024-09-19T09:14:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.256+0000 7f2398aba740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f23901abf60 con 0x55d49b6aa9a0 2024-09-19T09:14:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.256+0000 7f2398aba740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f23901a9180 con 0x55d49b6ab2e0 2024-09-19T09:14:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.256+0000 7f2398aba740 10 client.0 _ll_register_callbacks cb 0x7f23800f3620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.256+0000 7f238f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d49b6a6010 0x55d49b6b9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.256+0000 7f238f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d49b6a6010 0x55d49b6b9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59324/0 (socket says 172.21.15.120:59324) 2024-09-19T09:14:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.256+0000 7f238f7fe640 1 -- 172.21.15.120:0/3543880501 learned_addr learned my addr 172.21.15.120:0/3543880501 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.256+0000 7f238effd640 1 --2- 172.21.15.120:0/3543880501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d49b6aa9a0 0x55d49b6ba230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.256+0000 7f238ffff640 1 --2- 172.21.15.120:0/3543880501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d49b6ab2e0 0x55d49b6bb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.256+0000 7f238f7fe640 1 -- 172.21.15.120:0/3543880501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d49b6ab2e0 msgr2=0x55d49b6bb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.256+0000 7f238f7fe640 1 --2- 172.21.15.120:0/3543880501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d49b6ab2e0 0x55d49b6bb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.256+0000 7f238f7fe640 1 -- 172.21.15.120:0/3543880501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d49b6aa9a0 msgr2=0x55d49b6ba230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.256+0000 7f238f7fe640 1 --2- 172.21.15.120:0/3543880501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d49b6aa9a0 0x55d49b6ba230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.256+0000 7f238f7fe640 1 -- 172.21.15.120:0/3543880501 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d49b6bb850 con 0x55d49b6a6010 2024-09-19T09:14:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.256+0000 7f238ffff640 1 --2- 172.21.15.120:0/3543880501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d49b6ab2e0 0x55d49b6bb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:14:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.256+0000 7f238effd640 1 --2- 172.21.15.120:0/3543880501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d49b6aa9a0 0x55d49b6ba230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.256+0000 7f238f7fe640 1 --2- 172.21.15.120:0/3543880501 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d49b6a6010 0x55d49b6b9cf0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f23800f36d0 tx=0x7f2380009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef90f2dc6c444e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.256+0000 7f236effd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:14:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.257+0000 7f236effd640 1 -- 172.21.15.120:0/3543880501 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380049070 con 0x55d49b6a6010 2024-09-19T09:14:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.257+0000 7f2398aba740 1 -- 172.21.15.120:0/3543880501 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23800bcbd0 con 0x55d49b6a6010 2024-09-19T09:14:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.257+0000 7f236effd640 1 -- 172.21.15.120:0/3543880501 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f23800f3030 con 0x55d49b6a6010 2024-09-19T09:14:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.257+0000 7f236effd640 1 -- 172.21.15.120:0/3543880501 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f23800bbe30 con 0x55d49b6a6010 2024-09-19T09:14:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.257+0000 7f2398aba740 10 client.19407 fetch_fsmap learned FSMap version 32 2024-09-19T09:14:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.257+0000 7f2398aba740 1 -- 172.21.15.120:0/3543880501 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d49b72b980 con 0x55d49b6a6010 2024-09-19T09:14:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.257+0000 7f236effd640 1 -- 172.21.15.120:0/3543880501 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23800bcbd0 con 0x55d49b6a6010 2024-09-19T09:14:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.257+0000 7f236effd640 1 -- 172.21.15.120:0/3543880501 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f23800c4cb0 con 0x55d49b6a6010 2024-09-19T09:14:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.257+0000 7f2398aba740 10 client.19407 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:14:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.257+0000 7f2398aba740 10 client.19407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:14:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.257+0000 7f2398aba740 20 client.19407 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.257+0000 7f2398aba740 1 --2- 172.21.15.120:0/3543880501 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d49b6b0370 0x55d49b72e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.257+0000 7f2398aba740 4 client.19407 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:14:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.257+0000 7f2398aba740 1 -- 172.21.15.120:0/3543880501 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d49b615d70 con 0x55d49b6b0370 2024-09-19T09:14:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.257+0000 7f238effd640 1 --2- 172.21.15.120:0/3543880501 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d49b6b0370 0x55d49b72e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.258+0000 7f236effd640 10 client.19407 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.258+0000 7f238effd640 1 --2- 172.21.15.120:0/3543880501 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d49b6b0370 0x55d49b72e090 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a77f6dc4a0238d server_cookie=d7157c182ab9d5b2 in_seq=0 out_seq=0 2024-09-19T09:14:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.489+0000 7f236effd640 1 -- 172.21.15.120:0/3543880501 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d49b615d70 con 0x55d49b6b0370 2024-09-19T09:14:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.489+0000 7f236effd640 10 client.19407 handle_command_reply: tid=0 2024-09-19T09:14:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.489+0000 7f2398aba740 1 client.19407 shutdown 2024-09-19T09:14:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.490+0000 7f2398aba740 1 -- 172.21.15.120:0/3543880501 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d49b6a6010 msgr2=0x55d49b6b9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.490+0000 7f2398aba740 1 --2- 172.21.15.120:0/3543880501 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d49b6a6010 0x55d49b6b9cf0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f23800f36d0 tx=0x7f2380009620 comp rx=0 tx=0).stop 2024-09-19T09:14:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.490+0000 7f2398aba740 1 -- 172.21.15.120:0/3543880501 shutdown_connections 2024-09-19T09:14:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.490+0000 7f2398aba740 1 --2- 172.21.15.120:0/3543880501 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d49b6b0370 0x55d49b72e090 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.490+0000 7f2398aba740 1 --2- 172.21.15.120:0/3543880501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d49b6ab2e0 0x55d49b6bb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.490+0000 7f2398aba740 1 --2- 172.21.15.120:0/3543880501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d49b6aa9a0 0x55d49b6ba230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.490+0000 7f236effd640 0 client.19407 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.490+0000 7f2398aba740 1 --2- 172.21.15.120:0/3543880501 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d49b6a6010 0x55d49b6b9cf0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.490+0000 7f2398aba740 1 -- 172.21.15.120:0/3543880501 >> 172.21.15.120:0/3543880501 conn(0x55d49b6aff00 msgr2=0x55d49b6a51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.490+0000 7f2398aba740 1 -- 172.21.15.120:0/3543880501 shutdown_connections 2024-09-19T09:14:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.490+0000 7f2398aba740 1 -- 172.21.15.120:0/3543880501 wait complete. 2024-09-19T09:14:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.490+0000 7f2398aba740 20 client.19407 trim_cache size 0 max 16384 2024-09-19T09:14:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:14:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:14:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:14:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:14:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.491+0000 7f2395d23640 1 -- 172.21.15.120:0/309852666 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f23780a32f0 msgr2=0x7f23780a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.491+0000 7f2395d23640 1 --2- 172.21.15.120:0/309852666 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f23780a32f0 0x7f23780a57b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2388020000 tx=0x7f2388021000 comp rx=0 tx=0).stop 2024-09-19T09:14:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.491+0000 7f2395d23640 1 -- 172.21.15.120:0/309852666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2390103d60 msgr2=0x7f2390199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.491+0000 7f2395d23640 1 --2- 172.21.15.120:0/309852666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2390103d60 0x7f2390199fc0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f2380031440 tx=0x7f2380032ce0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.491+0000 7f2395d23640 1 -- 172.21.15.120:0/309852666 shutdown_connections 2024-09-19T09:14:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.491+0000 7f2395d23640 1 --2- 172.21.15.120:0/309852666 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f23780a32f0 0x7f23780a57b0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.491+0000 7f2395d23640 1 --2- 172.21.15.120:0/309852666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2390105910 0x7f23901a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.491+0000 7f2395d23640 1 --2- 172.21.15.120:0/309852666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2390104f50 0x7f239019a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.491+0000 7f2395d23640 1 --2- 172.21.15.120:0/309852666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2390103d60 0x7f2390199fc0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.491+0000 7f2395d23640 1 -- 172.21.15.120:0/309852666 >> 172.21.15.120:0/309852666 conn(0x7f23900ff4e0 msgr2=0x7f2390108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.491+0000 7f2395d23640 1 -- 172.21.15.120:0/309852666 shutdown_connections 2024-09-19T09:14:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:04.491+0000 7f2395d23640 1 -- 172.21.15.120:0/309852666 wait complete. 2024-09-19T09:14:04.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.551+0000 7ff94006a640 1 -- 172.21.15.120:0/2364446807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff938102d50 msgr2=0x7ff938103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:04.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.551+0000 7ff94006a640 1 --2- 172.21.15.120:0/2364446807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff938102d50 0x7ff938103130 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7ff934009900 tx=0x7ff93402a480 comp rx=0 tx=0).stop 2024-09-19T09:14:04.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.551+0000 7ff94006a640 1 -- 172.21.15.120:0/2364446807 shutdown_connections 2024-09-19T09:14:04.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.551+0000 7ff94006a640 1 --2- 172.21.15.120:0/2364446807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff93810c180 0x7ff93810e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.551+0000 7ff94006a640 1 --2- 172.21.15.120:0/2364446807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff938103670 0x7ff93810bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.551+0000 7ff94006a640 1 --2- 172.21.15.120:0/2364446807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff938102d50 0x7ff938103130 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.551+0000 7ff94006a640 1 -- 172.21.15.120:0/2364446807 >> 172.21.15.120:0/2364446807 conn(0x7ff9380fc7d0 msgr2=0x7ff9380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:04.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.551+0000 7ff94006a640 1 -- 172.21.15.120:0/2364446807 shutdown_connections 2024-09-19T09:14:04.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.551+0000 7ff94006a640 1 -- 172.21.15.120:0/2364446807 wait complete. 2024-09-19T09:14:04.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.552+0000 7ff94006a640 1 Processor -- start 2024-09-19T09:14:04.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.552+0000 7ff94006a640 1 -- start start 2024-09-19T09:14:04.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.552+0000 7ff94006a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff938102d50 0x7ff938198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.552+0000 7ff94006a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff938103670 0x7ff938198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.552+0000 7ff94006a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff93810c180 0x7ff93819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.552+0000 7ff94006a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff938111f80 con 0x7ff938102d50 2024-09-19T09:14:04.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.552+0000 7ff94006a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff938111e00 con 0x7ff938103670 2024-09-19T09:14:04.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.552+0000 7ff94006a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff938112100 con 0x7ff93810c180 2024-09-19T09:14:04.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.552+0000 7ff93de06640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff938102d50 0x7ff938198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.552+0000 7ff93de06640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff938102d50 0x7ff938198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59352/0 (socket says 172.21.15.120:59352) 2024-09-19T09:14:04.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.552+0000 7ff93de06640 1 -- 172.21.15.120:0/3994747630 learned_addr learned my addr 172.21.15.120:0/3994747630 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:04.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.552+0000 7ff93d605640 1 --2- 172.21.15.120:0/3994747630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff938103670 0x7ff938198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.552+0000 7ff93e607640 1 --2- 172.21.15.120:0/3994747630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff93810c180 0x7ff93819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.552+0000 7ff93de06640 1 -- 172.21.15.120:0/3994747630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff93810c180 msgr2=0x7ff93819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:04.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.552+0000 7ff93de06640 1 --2- 172.21.15.120:0/3994747630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff93810c180 0x7ff93819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.552+0000 7ff93de06640 1 -- 172.21.15.120:0/3994747630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff938103670 msgr2=0x7ff938198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:04.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.552+0000 7ff93de06640 1 --2- 172.21.15.120:0/3994747630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff938103670 0x7ff938198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.552+0000 7ff93de06640 1 -- 172.21.15.120:0/3994747630 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff93819d3c0 con 0x7ff938102d50 2024-09-19T09:14:04.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.552+0000 7ff93de06640 1 --2- 172.21.15.120:0/3994747630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff938102d50 0x7ff938198410 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7ff934006010 tx=0x7ff934004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:04.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.553+0000 7ff92effd640 1 -- 172.21.15.120:0/3994747630 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93403f030 con 0x7ff938102d50 2024-09-19T09:14:04.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.553+0000 7ff94006a640 1 -- 172.21.15.120:0/3994747630 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff93819d650 con 0x7ff938102d50 2024-09-19T09:14:04.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.553+0000 7ff92effd640 1 -- 172.21.15.120:0/3994747630 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff934004440 con 0x7ff938102d50 2024-09-19T09:14:04.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.553+0000 7ff92effd640 1 -- 172.21.15.120:0/3994747630 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff934035070 con 0x7ff938102d50 2024-09-19T09:14:04.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.553+0000 7ff94006a640 1 -- 172.21.15.120:0/3994747630 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9381a4f90 con 0x7ff938102d50 2024-09-19T09:14:04.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.555+0000 7ff942443740 1 Processor -- start 2024-09-19T09:14:04.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.555+0000 7ff942443740 1 -- start start 2024-09-19T09:14:04.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.556+0000 7ff942443740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566e237e4a0 0x5566e237e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.556+0000 7ff942443740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566e237edc0 0x5566e2389140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.556+0000 7ff92effd640 1 -- 172.21.15.120:0/3994747630 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7ff934042030 con 0x7ff938102d50 2024-09-19T09:14:04.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.556+0000 7ff942443740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566e2389680 0x5566e238ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.556+0000 7ff942443740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5566e22e42e0 con 0x5566e237edc0 2024-09-19T09:14:04.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.556+0000 7ff942443740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5566e228bce0 con 0x5566e237e4a0 2024-09-19T09:14:04.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.556+0000 7ff942443740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5566e22e02a0 con 0x5566e2389680 2024-09-19T09:14:04.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.556+0000 7ff93e607640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566e237e4a0 0x5566e237e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.556+0000 7ff93e607640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566e237e4a0 0x5566e237e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:57588/0 (socket says 172.21.15.120:57588) 2024-09-19T09:14:04.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.556+0000 7ff93e607640 1 -- 172.21.15.120:0/1933257456 learned_addr learned my addr 172.21.15.120:0/1933257456 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:04.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.556+0000 7ff93de06640 1 --2- 172.21.15.120:0/1933257456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566e237edc0 0x5566e2389140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.556+0000 7ff93d605640 1 --2- 172.21.15.120:0/1933257456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566e2389680 0x5566e238ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.556+0000 7ff93e607640 1 -- 172.21.15.120:0/1933257456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566e2389680 msgr2=0x5566e238ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:04.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.556+0000 7ff93e607640 1 --2- 172.21.15.120:0/1933257456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566e2389680 0x5566e238ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.556+0000 7ff93e607640 1 -- 172.21.15.120:0/1933257456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566e237edc0 msgr2=0x5566e2389140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:04.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.556+0000 7ff93e607640 1 --2- 172.21.15.120:0/1933257456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566e237edc0 0x5566e2389140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.556+0000 7ff93e607640 1 -- 172.21.15.120:0/1933257456 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566e2270440 con 0x5566e237e4a0 2024-09-19T09:14:04.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.556+0000 7ff93e607640 1 --2- 172.21.15.120:0/1933257456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566e237e4a0 0x5566e237e880 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff930009f50 tx=0x7ff9300118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0db44c6f396cb30 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:04.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.557+0000 7ff92effd640 1 --2- 172.21.15.120:0/3994747630 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff9240a3240 0x7ff9240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.557+0000 7ff923fff640 1 -- 172.21.15.120:0/1933257456 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff930004d20 con 0x5566e237e4a0 2024-09-19T09:14:04.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.557+0000 7ff923fff640 1 -- 172.21.15.120:0/1933257456 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff930004050 con 0x5566e237e4a0 2024-09-19T09:14:04.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.557+0000 7ff923fff640 1 -- 172.21.15.120:0/1933257456 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff930005260 con 0x5566e237e4a0 2024-09-19T09:14:04.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.557+0000 7ff92effd640 1 -- 172.21.15.120:0/3994747630 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ff9340efc60 con 0x7ff938102d50 2024-09-19T09:14:04.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.557+0000 7ff93de06640 1 --2- 172.21.15.120:0/3994747630 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff9240a3240 0x7ff9240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.557+0000 7ff942443740 1 -- 172.21.15.120:0/1933257456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566e237e4a0 msgr2=0x5566e237e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:04.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.557+0000 7ff93de06640 1 --2- 172.21.15.120:0/3994747630 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff9240a3240 0x7ff9240a5700 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff934005ab0 tx=0x7ff9340f36e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:04.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.558+0000 7ff942443740 1 --2- 172.21.15.120:0/1933257456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566e237e4a0 0x5566e237e880 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff930009f50 tx=0x7ff9300118d0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.558+0000 7ff942443740 1 -- 172.21.15.120:0/1933257456 shutdown_connections 2024-09-19T09:14:04.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.558+0000 7ff942443740 1 --2- 172.21.15.120:0/1933257456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566e2389680 0x5566e238ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.558+0000 7ff942443740 1 --2- 172.21.15.120:0/1933257456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566e237edc0 0x5566e2389140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.558+0000 7ff942443740 1 --2- 172.21.15.120:0/1933257456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566e237e4a0 0x5566e237e880 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.558+0000 7ff942443740 1 -- 172.21.15.120:0/1933257456 >> 172.21.15.120:0/1933257456 conn(0x5566e237aa30 msgr2=0x5566e237ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:04.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.558+0000 7ff942443740 1 -- 172.21.15.120:0/1933257456 shutdown_connections 2024-09-19T09:14:04.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.558+0000 7ff942443740 1 -- 172.21.15.120:0/1933257456 wait complete. 2024-09-19T09:14:04.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.558+0000 7ff942443740 1 -- start start 2024-09-19T09:14:04.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.558+0000 7ff942443740 1 Processor -- start 2024-09-19T09:14:04.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.559+0000 7ff942443740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566e237e4a0 0x5566e2388f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.559+0000 7ff942443740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566e237edc0 0x5566e23861e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.559+0000 7ff942443740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566e2389680 0x5566e2386720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.559+0000 7ff942443740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5566e238c250 con 0x5566e2389680 2024-09-19T09:14:04.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.559+0000 7ff942443740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5566e229f050 con 0x5566e237e4a0 2024-09-19T09:14:04.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.559+0000 7ff942443740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5566e238c3d0 con 0x5566e237edc0 2024-09-19T09:14:04.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.559+0000 7ff942443740 10 client.0 _ll_register_callbacks cb 0x5566e22df160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:04.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.559+0000 7ff93e607640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566e237e4a0 0x5566e2388f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.559+0000 7ff93e607640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566e237e4a0 0x5566e2388f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:57592/0 (socket says 172.21.15.120:57592) 2024-09-19T09:14:04.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.559+0000 7ff93e607640 1 -- 172.21.15.120:0/1417369938 learned_addr learned my addr 172.21.15.120:0/1417369938 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:04.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.559+0000 7ff93e607640 1 --2- 172.21.15.120:0/1417369938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566e2389680 0x5566e2386720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.559+0000 7ff93d605640 1 --2- 172.21.15.120:0/1417369938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566e237edc0 0x5566e23861e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.559+0000 7ff93e607640 1 -- 172.21.15.120:0/1417369938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566e237edc0 msgr2=0x5566e23861e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:04.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.559+0000 7ff93e607640 1 --2- 172.21.15.120:0/1417369938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566e237edc0 0x5566e23861e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.559+0000 7ff93e607640 1 -- 172.21.15.120:0/1417369938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566e2389680 msgr2=0x5566e2386720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:04.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.559+0000 7ff93e607640 1 --2- 172.21.15.120:0/1417369938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566e2389680 0x5566e2386720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.559+0000 7ff93e607640 1 -- 172.21.15.120:0/1417369938 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566e20553c0 con 0x5566e237e4a0 2024-09-19T09:14:04.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.559+0000 7ff93e607640 1 --2- 172.21.15.120:0/1417369938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566e237e4a0 0x5566e2388f30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff930004920 tx=0x7ff930005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59fca8c0cd64edb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:04.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.559+0000 7ff9217fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:14:04.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.560+0000 7ff9217fa640 1 -- 172.21.15.120:0/1417369938 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff930004c50 con 0x5566e237e4a0 2024-09-19T09:14:04.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.560+0000 7ff942443740 1 -- 172.21.15.120:0/1417369938 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff930004050 con 0x5566e237e4a0 2024-09-19T09:14:04.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.560+0000 7ff9217fa640 1 -- 172.21.15.120:0/1417369938 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ff930004df0 con 0x5566e237e4a0 2024-09-19T09:14:04.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.560+0000 7ff9217fa640 1 -- 172.21.15.120:0/1417369938 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff930028450 con 0x5566e237e4a0 2024-09-19T09:14:04.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.560+0000 7ff942443740 10 client.28472 fetch_fsmap learned FSMap version 32 2024-09-19T09:14:04.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.560+0000 7ff942443740 1 -- 172.21.15.120:0/1417369938 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5566e2397b70 con 0x5566e237e4a0 2024-09-19T09:14:04.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.560+0000 7ff9217fa640 1 -- 172.21.15.120:0/1417369938 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff930004050 con 0x5566e237e4a0 2024-09-19T09:14:04.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.560+0000 7ff9217fa640 1 -- 172.21.15.120:0/1417369938 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff9300274b0 con 0x5566e237e4a0 2024-09-19T09:14:04.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.560+0000 7ff942443740 10 client.28472 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:14:04.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.560+0000 7ff942443740 10 client.28472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:14:04.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.560+0000 7ff942443740 20 client.28472 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:04.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.560+0000 7ff942443740 1 --2- 172.21.15.120:0/1417369938 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5566e2398f80 0x5566e239b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.560+0000 7ff942443740 4 client.28472 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:14:04.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.560+0000 7ff942443740 1 -- 172.21.15.120:0/1417369938 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566e22f1850 con 0x5566e2398f80 2024-09-19T09:14:04.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.561+0000 7ff93d605640 1 --2- 172.21.15.120:0/1417369938 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5566e2398f80 0x5566e239b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.561+0000 7ff9217fa640 10 client.28472 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:04.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.561+0000 7ff93d605640 1 --2- 172.21.15.120:0/1417369938 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5566e2398f80 0x5566e239b370 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30cf27b91d35dbd7 server_cookie=6c35473e8add4ace in_seq=0 out_seq=0 2024-09-19T09:14:04.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.565+0000 7ff9217fa640 1 -- 172.21.15.120:0/1417369938 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5566e22f1850 con 0x5566e2398f80 2024-09-19T09:14:04.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.565+0000 7ff9217fa640 10 client.28472 handle_command_reply: tid=0 2024-09-19T09:14:04.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.565+0000 7ff942443740 1 client.28472 shutdown 2024-09-19T09:14:04.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.565+0000 7ff942443740 1 -- 172.21.15.120:0/1417369938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566e237e4a0 msgr2=0x5566e2388f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:04.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.565+0000 7ff942443740 1 --2- 172.21.15.120:0/1417369938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566e237e4a0 0x5566e2388f30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff930004920 tx=0x7ff930005bd0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.566+0000 7ff942443740 1 -- 172.21.15.120:0/1417369938 shutdown_connections 2024-09-19T09:14:04.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.566+0000 7ff942443740 1 --2- 172.21.15.120:0/1417369938 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5566e2398f80 0x5566e239b370 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.566+0000 7ff9217fa640 0 client.28472 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:04.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.566+0000 7ff942443740 1 --2- 172.21.15.120:0/1417369938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566e2389680 0x5566e2386720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.566+0000 7ff942443740 1 --2- 172.21.15.120:0/1417369938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566e237edc0 0x5566e23861e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.566+0000 7ff942443740 1 --2- 172.21.15.120:0/1417369938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566e237e4a0 0x5566e2388f30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.566+0000 7ff942443740 1 -- 172.21.15.120:0/1417369938 >> 172.21.15.120:0/1417369938 conn(0x5566e237aa30 msgr2=0x5566e2380760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:04.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.566+0000 7ff942443740 1 -- 172.21.15.120:0/1417369938 shutdown_connections 2024-09-19T09:14:04.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.566+0000 7ff942443740 1 -- 172.21.15.120:0/1417369938 wait complete. 2024-09-19T09:14:04.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.566+0000 7ff942443740 20 client.28472 trim_cache size 0 max 16384 2024-09-19T09:14:04.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.598+0000 7ff942443740 1 Processor -- start 2024-09-19T09:14:04.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.598+0000 7ff942443740 1 -- start start 2024-09-19T09:14:04.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.598+0000 7ff942443740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566e237e4a0 0x5566e23954b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.598+0000 7ff942443740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566e237edc0 0x5566e23a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.598+0000 7ff942443740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566e2389680 0x5566e23a1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.598+0000 7ff942443740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff9381ad610 con 0x5566e237edc0 2024-09-19T09:14:04.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.598+0000 7ff942443740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff9381af150 con 0x5566e237e4a0 2024-09-19T09:14:04.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.598+0000 7ff942443740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff9381adce0 con 0x5566e2389680 2024-09-19T09:14:04.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.599+0000 7ff93d605640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566e237edc0 0x5566e23a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.599+0000 7ff93d605640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566e237edc0 0x5566e23a0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59366/0 (socket says 172.21.15.120:59366) 2024-09-19T09:14:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.599+0000 7ff93d605640 1 -- 172.21.15.120:0/997318104 learned_addr learned my addr 172.21.15.120:0/997318104 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.599+0000 7ff93e607640 1 --2- 172.21.15.120:0/997318104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566e237e4a0 0x5566e23954b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.599+0000 7ff93e607640 1 --2- 172.21.15.120:0/997318104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566e2389680 0x5566e23a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.599+0000 7ff93d605640 1 -- 172.21.15.120:0/997318104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566e2389680 msgr2=0x5566e23a1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.599+0000 7ff93d605640 1 --2- 172.21.15.120:0/997318104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566e2389680 0x5566e23a1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.599+0000 7ff93d605640 1 -- 172.21.15.120:0/997318104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566e237e4a0 msgr2=0x5566e23954b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.599+0000 7ff93d605640 1 --2- 172.21.15.120:0/997318104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566e237e4a0 0x5566e23954b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.599+0000 7ff93d605640 1 -- 172.21.15.120:0/997318104 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566e2384730 con 0x5566e237edc0 2024-09-19T09:14:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.599+0000 7ff93d605640 1 --2- 172.21.15.120:0/997318104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566e237edc0 0x5566e23a0b30 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x5566e2385a70 tx=0x7ff928005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b4fceb9bbfef2a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.599+0000 7ff9237fe640 1 -- 172.21.15.120:0/997318104 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff928004230 con 0x5566e237edc0 2024-09-19T09:14:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.599+0000 7ff9237fe640 1 -- 172.21.15.120:0/997318104 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9280043d0 con 0x5566e237edc0 2024-09-19T09:14:04.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.599+0000 7ff9237fe640 1 -- 172.21.15.120:0/997318104 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92801e9a0 con 0x5566e237edc0 2024-09-19T09:14:04.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.600+0000 7ff942443740 1 -- 172.21.15.120:0/997318104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566e237edc0 msgr2=0x5566e23a0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:04.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.600+0000 7ff942443740 1 --2- 172.21.15.120:0/997318104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566e237edc0 0x5566e23a0b30 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x5566e2385a70 tx=0x7ff928005f90 comp rx=0 tx=0).stop 2024-09-19T09:14:04.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.600+0000 7ff942443740 1 -- 172.21.15.120:0/997318104 shutdown_connections 2024-09-19T09:14:04.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.600+0000 7ff942443740 1 --2- 172.21.15.120:0/997318104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566e2389680 0x5566e23a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.600+0000 7ff942443740 1 --2- 172.21.15.120:0/997318104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566e237edc0 0x5566e23a0b30 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.600+0000 7ff942443740 1 --2- 172.21.15.120:0/997318104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566e237e4a0 0x5566e23954b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.600+0000 7ff942443740 1 -- 172.21.15.120:0/997318104 >> 172.21.15.120:0/997318104 conn(0x5566e237aa30 msgr2=0x5566e2379b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:04.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.600+0000 7ff942443740 1 -- 172.21.15.120:0/997318104 shutdown_connections 2024-09-19T09:14:04.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.600+0000 7ff942443740 1 -- 172.21.15.120:0/997318104 wait complete. 2024-09-19T09:14:04.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.600+0000 7ff942443740 1 -- start start 2024-09-19T09:14:04.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.601+0000 7ff942443740 1 Processor -- start 2024-09-19T09:14:04.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.601+0000 7ff942443740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566e237e4a0 0x5566e238d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.601+0000 7ff942443740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566e237edc0 0x5566e238db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.601+0000 7ff942443740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566e2389680 0x5566e238e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.601+0000 7ff942443740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff9381aea80 con 0x5566e237e4a0 2024-09-19T09:14:04.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.601+0000 7ff942443740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff9381ae3b0 con 0x5566e237edc0 2024-09-19T09:14:04.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.601+0000 7ff942443740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff9381ab5d0 con 0x5566e2389680 2024-09-19T09:14:04.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.601+0000 7ff942443740 10 client.0 _ll_register_callbacks cb 0x7ff9340f35b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:04.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.601+0000 7ff93e607640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566e237e4a0 0x5566e238d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.601+0000 7ff93e607640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566e237e4a0 0x5566e238d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59380/0 (socket says 172.21.15.120:59380) 2024-09-19T09:14:04.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.601+0000 7ff93e607640 1 -- 172.21.15.120:0/1194978533 learned_addr learned my addr 172.21.15.120:0/1194978533 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:04.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.601+0000 7ff93d605640 1 --2- 172.21.15.120:0/1194978533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566e237edc0 0x5566e238db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.601+0000 7ff93e607640 1 --2- 172.21.15.120:0/1194978533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566e2389680 0x5566e238e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.601+0000 7ff93d605640 1 -- 172.21.15.120:0/1194978533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566e2389680 msgr2=0x5566e238e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:04.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.601+0000 7ff93d605640 1 --2- 172.21.15.120:0/1194978533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566e2389680 0x5566e238e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.601+0000 7ff93d605640 1 -- 172.21.15.120:0/1194978533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566e237e4a0 msgr2=0x5566e238d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:04.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.601+0000 7ff93d605640 1 --2- 172.21.15.120:0/1194978533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566e237e4a0 0x5566e238d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.601+0000 7ff93d605640 1 -- 172.21.15.120:0/1194978533 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566e2407080 con 0x5566e237edc0 2024-09-19T09:14:04.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.601+0000 7ff93e607640 1 --2- 172.21.15.120:0/1194978533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566e237e4a0 0x5566e238d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:04.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.602+0000 7ff93d605640 1 --2- 172.21.15.120:0/1194978533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566e237edc0 0x5566e238db90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x5566e2376080 tx=0x7ff928009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2c9aa34406789cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:04.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.602+0000 7ff9217fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:14:04.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.602+0000 7ff9217fa640 1 -- 172.21.15.120:0/1194978533 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92801eb30 con 0x5566e237edc0 2024-09-19T09:14:04.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.602+0000 7ff9217fa640 1 -- 172.21.15.120:0/1194978533 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ff92801ecd0 con 0x5566e237edc0 2024-09-19T09:14:04.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.602+0000 7ff942443740 1 -- 172.21.15.120:0/1194978533 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9280043d0 con 0x5566e237edc0 2024-09-19T09:14:04.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.602+0000 7ff9217fa640 1 -- 172.21.15.120:0/1194978533 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff928033450 con 0x5566e237edc0 2024-09-19T09:14:04.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.602+0000 7ff942443740 10 client.28478 fetch_fsmap learned FSMap version 32 2024-09-19T09:14:04.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.602+0000 7ff942443740 1 -- 172.21.15.120:0/1194978533 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5566e2408330 con 0x5566e237edc0 2024-09-19T09:14:04.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.602+0000 7ff9217fa640 1 -- 172.21.15.120:0/1194978533 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9280043d0 con 0x5566e237edc0 2024-09-19T09:14:04.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.602+0000 7ff9217fa640 1 -- 172.21.15.120:0/1194978533 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff92803e650 con 0x5566e237edc0 2024-09-19T09:14:04.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.602+0000 7ff942443740 10 client.28478 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:14:04.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.602+0000 7ff942443740 10 client.28478 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:14:04.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.602+0000 7ff942443740 20 client.28478 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:04.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.602+0000 7ff942443740 1 --2- 172.21.15.120:0/1194978533 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5566e2408a50 0x5566e240ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:04.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.602+0000 7ff942443740 4 client.28478 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:14:04.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.602+0000 7ff942443740 1 -- 172.21.15.120:0/1194978533 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5566e22ebd60 con 0x5566e2408a50 2024-09-19T09:14:04.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.603+0000 7ff93e607640 1 --2- 172.21.15.120:0/1194978533 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5566e2408a50 0x5566e240ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:04.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.603+0000 7ff9217fa640 10 client.28478 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:04.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.603+0000 7ff93e607640 1 --2- 172.21.15.120:0/1194978533 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5566e2408a50 0x5566e240ae70 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cba35aeaf4c1ea4d server_cookie=a6434dd00048a5dc in_seq=0 out_seq=0 2024-09-19T09:14:04.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.603+0000 7ff9217fa640 1 -- 172.21.15.120:0/1194978533 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5566e22ebd60 con 0x5566e2408a50 2024-09-19T09:14:04.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.603+0000 7ff9217fa640 10 client.28478 handle_command_reply: tid=0 2024-09-19T09:14:04.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.603+0000 7ff942443740 1 client.28478 shutdown 2024-09-19T09:14:04.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.603+0000 7ff942443740 1 -- 172.21.15.120:0/1194978533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566e237edc0 msgr2=0x5566e238db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:04.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.603+0000 7ff942443740 1 --2- 172.21.15.120:0/1194978533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566e237edc0 0x5566e238db90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x5566e2376080 tx=0x7ff928009b80 comp rx=0 tx=0).stop 2024-09-19T09:14:04.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.603+0000 7ff942443740 1 -- 172.21.15.120:0/1194978533 shutdown_connections 2024-09-19T09:14:04.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.603+0000 7ff942443740 1 --2- 172.21.15.120:0/1194978533 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5566e2408a50 0x5566e240ae70 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.603+0000 7ff942443740 1 --2- 172.21.15.120:0/1194978533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566e2389680 0x5566e238e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.603+0000 7ff9217fa640 0 client.28478 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:04.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.604+0000 7ff942443740 1 --2- 172.21.15.120:0/1194978533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5566e237edc0 0x5566e238db90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.604+0000 7ff942443740 1 --2- 172.21.15.120:0/1194978533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566e237e4a0 0x5566e238d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.604+0000 7ff942443740 1 -- 172.21.15.120:0/1194978533 >> 172.21.15.120:0/1194978533 conn(0x5566e237aa30 msgr2=0x5566e2384980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:04.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.604+0000 7ff942443740 1 -- 172.21.15.120:0/1194978533 shutdown_connections 2024-09-19T09:14:04.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.604+0000 7ff942443740 1 -- 172.21.15.120:0/1194978533 wait complete. 2024-09-19T09:14:04.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.604+0000 7ff942443740 20 client.28478 trim_cache size 0 max 16384 2024-09-19T09:14:04.609 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:14:04.610 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:14:04.610 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:14:04.610 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:14:04.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.604+0000 7ff94006a640 1 -- 172.21.15.120:0/3994747630 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff9240a3240 msgr2=0x7ff9240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:04.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.604+0000 7ff94006a640 1 --2- 172.21.15.120:0/3994747630 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff9240a3240 0x7ff9240a5700 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff934005ab0 tx=0x7ff9340f36e0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.604+0000 7ff94006a640 1 -- 172.21.15.120:0/3994747630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff938102d50 msgr2=0x7ff938198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:04.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.604+0000 7ff94006a640 1 --2- 172.21.15.120:0/3994747630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff938102d50 0x7ff938198410 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7ff934006010 tx=0x7ff934004040 comp rx=0 tx=0).stop 2024-09-19T09:14:04.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.604+0000 7ff94006a640 1 -- 172.21.15.120:0/3994747630 shutdown_connections 2024-09-19T09:14:04.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.604+0000 7ff94006a640 1 --2- 172.21.15.120:0/3994747630 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff9240a3240 0x7ff9240a5700 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.604+0000 7ff94006a640 1 --2- 172.21.15.120:0/3994747630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff93810c180 0x7ff93819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.604+0000 7ff94006a640 1 --2- 172.21.15.120:0/3994747630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff938103670 0x7ff938198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.604+0000 7ff94006a640 1 --2- 172.21.15.120:0/3994747630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff938102d50 0x7ff938198410 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:04.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.604+0000 7ff94006a640 1 -- 172.21.15.120:0/3994747630 >> 172.21.15.120:0/3994747630 conn(0x7ff9380fc7d0 msgr2=0x7ff9380fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:04.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.605+0000 7ff94006a640 1 -- 172.21.15.120:0/3994747630 shutdown_connections 2024-09-19T09:14:04.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:04.605+0000 7ff94006a640 1 -- 172.21.15.120:0/3994747630 wait complete. 2024-09-19T09:14:05.073 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1043456 9.23 MB/sec execute 480 sec latency 72.289 ms 2024-09-19T09:14:05.149 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:04 smithi120 ceph-mon[31150]: pgmap v679: 97 pgs: 97 active+clean; 42 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.6 MiB/s wr, 263 op/s 2024-09-19T09:14:05.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:04 smithi196 ceph-mon[35150]: pgmap v679: 97 pgs: 97 active+clean; 42 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.6 MiB/s wr, 263 op/s 2024-09-19T09:14:05.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:04 smithi153 ceph-mon[35071]: pgmap v679: 97 pgs: 97 active+clean; 42 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.6 MiB/s wr, 263 op/s 2024-09-19T09:14:05.442 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:14:05.443 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:14:05.709 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:14:06.073 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1045598 9.24 MB/sec execute 481 sec latency 21.766 ms 2024-09-19T09:14:06.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.259+0000 7efc605e3640 1 -- 172.21.15.120:0/2145340119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc58102d30 msgr2=0x7efc58103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:06.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.259+0000 7efc605e3640 1 --2- 172.21.15.120:0/2145340119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc58102d30 0x7efc58103110 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7efc54009930 tx=0x7efc5402a420 comp rx=0 tx=0).stop 2024-09-19T09:14:06.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.260+0000 7efc605e3640 1 -- 172.21.15.120:0/2145340119 shutdown_connections 2024-09-19T09:14:06.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.260+0000 7efc605e3640 1 --2- 172.21.15.120:0/2145340119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc581082a0 0x7efc5810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:06.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.260+0000 7efc605e3640 1 --2- 172.21.15.120:0/2145340119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc581036e0 0x7efc58107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:06.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.260+0000 7efc605e3640 1 --2- 172.21.15.120:0/2145340119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc58102d30 0x7efc58103110 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:06.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.260+0000 7efc605e3640 1 -- 172.21.15.120:0/2145340119 >> 172.21.15.120:0/2145340119 conn(0x7efc580fc7d0 msgr2=0x7efc580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:06.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.260+0000 7efc605e3640 1 -- 172.21.15.120:0/2145340119 shutdown_connections 2024-09-19T09:14:06.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.260+0000 7efc605e3640 1 -- 172.21.15.120:0/2145340119 wait complete. 2024-09-19T09:14:06.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.260+0000 7efc605e3640 1 Processor -- start 2024-09-19T09:14:06.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.260+0000 7efc605e3640 1 -- start start 2024-09-19T09:14:06.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.260+0000 7efc605e3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc58102d30 0x7efc581985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:06.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.260+0000 7efc605e3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc581036e0 0x7efc58198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:06.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.260+0000 7efc605e3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc581082a0 0x7efc5819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:06.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.260+0000 7efc605e3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efc581120a0 con 0x7efc581036e0 2024-09-19T09:14:06.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.260+0000 7efc605e3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efc58111f20 con 0x7efc58102d30 2024-09-19T09:14:06.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.260+0000 7efc605e3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efc58112220 con 0x7efc581082a0 2024-09-19T09:14:06.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.260+0000 7efc5db7e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc581036e0 0x7efc58198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:06.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.261+0000 7efc5e37f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc58102d30 0x7efc581985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:06.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.261+0000 7efc5db7e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc581036e0 0x7efc58198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59396/0 (socket says 172.21.15.120:59396) 2024-09-19T09:14:06.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.261+0000 7efc5e37f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc58102d30 0x7efc581985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:57618/0 (socket says 172.21.15.120:57618) 2024-09-19T09:14:06.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.261+0000 7efc5db7e640 1 -- 172.21.15.120:0/3271191797 learned_addr learned my addr 172.21.15.120:0/3271191797 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:06.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.261+0000 7efc5eb80640 1 --2- 172.21.15.120:0/3271191797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc581082a0 0x7efc5819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:06.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.261+0000 7efc5db7e640 1 -- 172.21.15.120:0/3271191797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc581082a0 msgr2=0x7efc5819ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:06.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.261+0000 7efc5db7e640 1 --2- 172.21.15.120:0/3271191797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc581082a0 0x7efc5819ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:06.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.261+0000 7efc5db7e640 1 -- 172.21.15.120:0/3271191797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc58102d30 msgr2=0x7efc581985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:06.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.261+0000 7efc5db7e640 1 --2- 172.21.15.120:0/3271191797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc58102d30 0x7efc581985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:06.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.261+0000 7efc5db7e640 1 -- 172.21.15.120:0/3271191797 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5819d570 con 0x7efc581036e0 2024-09-19T09:14:06.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.261+0000 7efc5eb80640 1 --2- 172.21.15.120:0/3271191797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc581082a0 0x7efc5819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:06.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.261+0000 7efc5db7e640 1 --2- 172.21.15.120:0/3271191797 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc581036e0 0x7efc58198b00 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7efc4800d950 tx=0x7efc4800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:06.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.261+0000 7efc4f7fe640 1 -- 172.21.15.120:0/3271191797 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48017070 con 0x7efc581036e0 2024-09-19T09:14:06.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.261+0000 7efc605e3640 1 -- 172.21.15.120:0/3271191797 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5819d860 con 0x7efc581036e0 2024-09-19T09:14:06.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.261+0000 7efc4f7fe640 1 -- 172.21.15.120:0/3271191797 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc48009c30 con 0x7efc581036e0 2024-09-19T09:14:06.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.261+0000 7efc4f7fe640 1 -- 172.21.15.120:0/3271191797 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc480047a0 con 0x7efc581036e0 2024-09-19T09:14:06.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.261+0000 7efc605e3640 1 -- 172.21.15.120:0/3271191797 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc581a5140 con 0x7efc581036e0 2024-09-19T09:14:06.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.262+0000 7efc605e3640 1 -- 172.21.15.120:0/3271191797 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc5806a850 con 0x7efc581036e0 2024-09-19T09:14:06.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.264+0000 7efc4f7fe640 1 -- 172.21.15.120:0/3271191797 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7efc48004020 con 0x7efc581036e0 2024-09-19T09:14:06.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.264+0000 7efc4f7fe640 1 --2- 172.21.15.120:0/3271191797 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efc440a3240 0x7efc440a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:06.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.264+0000 7efc5e37f640 1 --2- 172.21.15.120:0/3271191797 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efc440a3240 0x7efc440a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:06.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.264+0000 7efc4f7fe640 1 -- 172.21.15.120:0/3271191797 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7efc480cf910 con 0x7efc581036e0 2024-09-19T09:14:06.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.265+0000 7efc5e37f640 1 --2- 172.21.15.120:0/3271191797 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efc440a3240 0x7efc440a5700 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7efc54009900 tx=0x7efc540058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:06.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.268+0000 7efc4f7fe640 1 -- 172.21.15.120:0/3271191797 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efc48020020 con 0x7efc581036e0 2024-09-19T09:14:06.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.492+0000 7efc605e3640 1 -- 172.21.15.120:0/3271191797 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc5810a2b0 con 0x7efc581036e0 2024-09-19T09:14:06.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.493+0000 7efc4f7fe640 1 -- 172.21.15.120:0/3271191797 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11988 (secure 0 0 0) 0x7efc4809c630 con 0x7efc581036e0 2024-09-19T09:14:06.494 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:14:06.494 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":32,"btime":"2024-09-19T09:11:12:904684+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:11:12.904670+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27518":{"gid":27518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3214776406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3214776406},{"type":"v1","addr":"172.21.15.153:6835","nonce":3214776406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:14:06.495 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 32 2024-09-19T09:14:06.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.495+0000 7efc605e3640 1 -- 172.21.15.120:0/3271191797 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efc440a3240 msgr2=0x7efc440a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:06.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.495+0000 7efc605e3640 1 --2- 172.21.15.120:0/3271191797 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efc440a3240 0x7efc440a5700 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7efc54009900 tx=0x7efc540058d0 comp rx=0 tx=0).stop 2024-09-19T09:14:06.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.495+0000 7efc605e3640 1 -- 172.21.15.120:0/3271191797 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc581036e0 msgr2=0x7efc58198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:06.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.495+0000 7efc605e3640 1 --2- 172.21.15.120:0/3271191797 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc581036e0 0x7efc58198b00 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7efc4800d950 tx=0x7efc4800de10 comp rx=0 tx=0).stop 2024-09-19T09:14:06.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.495+0000 7efc605e3640 1 -- 172.21.15.120:0/3271191797 shutdown_connections 2024-09-19T09:14:06.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.495+0000 7efc605e3640 1 --2- 172.21.15.120:0/3271191797 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efc440a3240 0x7efc440a5700 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:06.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.495+0000 7efc605e3640 1 --2- 172.21.15.120:0/3271191797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc581082a0 0x7efc5819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:06.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.495+0000 7efc605e3640 1 --2- 172.21.15.120:0/3271191797 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc581036e0 0x7efc58198b00 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:06.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.495+0000 7efc605e3640 1 --2- 172.21.15.120:0/3271191797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc58102d30 0x7efc581985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:06.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.495+0000 7efc605e3640 1 -- 172.21.15.120:0/3271191797 >> 172.21.15.120:0/3271191797 conn(0x7efc580fc7d0 msgr2=0x7efc58105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:06.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.496+0000 7efc605e3640 1 -- 172.21.15.120:0/3271191797 shutdown_connections 2024-09-19T09:14:06.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:06.496+0000 7efc605e3640 1 -- 172.21.15.120:0/3271191797 wait complete. 2024-09-19T09:14:07.073 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1048431 9.25 MB/sec execute 482 sec latency 16.720 ms 2024-09-19T09:14:07.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:06 smithi196 ceph-mon[35150]: pgmap v680: 97 pgs: 97 active+clean; 39 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.4 MiB/s wr, 233 op/s 2024-09-19T09:14:07.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:06 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3271191797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:14:07.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:06 smithi153 ceph-mon[35071]: pgmap v680: 97 pgs: 97 active+clean; 39 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.4 MiB/s wr, 233 op/s 2024-09-19T09:14:07.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:06 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3271191797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:14:07.301 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:06 smithi120 ceph-mon[31150]: pgmap v680: 97 pgs: 97 active+clean; 39 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.4 MiB/s wr, 233 op/s 2024-09-19T09:14:07.301 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:06 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3271191797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:14:07.330 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:14:07.590 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:14:08.004 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:07 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:14:08.073 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1051586 9.27 MB/sec execute 483 sec latency 25.203 ms 2024-09-19T09:14:08.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:07 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:14:08.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:07 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:14:08.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.268+0000 7f500e885640 1 -- 172.21.15.120:0/253721460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5008102570 msgr2=0x7f50081029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:08.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.268+0000 7f500e885640 1 --2- 172.21.15.120:0/253721460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5008102570 0x7f50081029b0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0009900 tx=0x7f4ff002a480 comp rx=0 tx=0).stop 2024-09-19T09:14:08.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.269+0000 7f500e885640 1 -- 172.21.15.120:0/253721460 shutdown_connections 2024-09-19T09:14:08.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.269+0000 7f500e885640 1 --2- 172.21.15.120:0/253721460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5008102ef0 0x7f500810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.269+0000 7f500e885640 1 --2- 172.21.15.120:0/253721460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5008102570 0x7f50081029b0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.269+0000 7f500e885640 1 --2- 172.21.15.120:0/253721460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5008100e20 0x7f5008101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.269+0000 7f500e885640 1 -- 172.21.15.120:0/253721460 >> 172.21.15.120:0/253721460 conn(0x7f50080fc7d0 msgr2=0x7f50080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:08.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.269+0000 7f500e885640 1 -- 172.21.15.120:0/253721460 shutdown_connections 2024-09-19T09:14:08.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.269+0000 7f500e885640 1 -- 172.21.15.120:0/253721460 wait complete. 2024-09-19T09:14:08.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.269+0000 7f500e885640 1 Processor -- start 2024-09-19T09:14:08.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.269+0000 7f500e885640 1 -- start start 2024-09-19T09:14:08.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.270+0000 7f500e885640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5008100e20 0x7f5008197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:08.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.270+0000 7f500e885640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5008102570 0x7f50081979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:08.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.270+0000 7f500e885640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5008102ef0 0x7f500819bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:08.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.270+0000 7f500e885640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5008110f80 con 0x7f5008102ef0 2024-09-19T09:14:08.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.270+0000 7f500e885640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5008110e00 con 0x7f5008102570 2024-09-19T09:14:08.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.270+0000 7f500e885640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5008111100 con 0x7f5008100e20 2024-09-19T09:14:08.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.270+0000 7f5007fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5008100e20 0x7f5008197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:08.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.270+0000 7f50077fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5008102570 0x7f50081979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:08.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.270+0000 7f5007fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5008100e20 0x7f5008197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:60204/0 (socket says 172.21.15.120:60204) 2024-09-19T09:14:08.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.270+0000 7f5007fff640 1 -- 172.21.15.120:0/4193218942 learned_addr learned my addr 172.21.15.120:0/4193218942 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:08.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.270+0000 7f500ce22640 1 --2- 172.21.15.120:0/4193218942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5008102ef0 0x7f500819bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:08.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.270+0000 7f500ce22640 1 -- 172.21.15.120:0/4193218942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5008100e20 msgr2=0x7f5008197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:08.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.270+0000 7f500ce22640 1 --2- 172.21.15.120:0/4193218942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5008100e20 0x7f5008197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.270+0000 7f500ce22640 1 -- 172.21.15.120:0/4193218942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5008102570 msgr2=0x7f50081979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:08.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.270+0000 7f500ce22640 1 --2- 172.21.15.120:0/4193218942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5008102570 0x7f50081979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.270+0000 7f500ce22640 1 -- 172.21.15.120:0/4193218942 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f500819c410 con 0x7f5008102ef0 2024-09-19T09:14:08.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.270+0000 7f50077fe640 1 --2- 172.21.15.120:0/4193218942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5008102570 0x7f50081979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:08.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.270+0000 7f5007fff640 1 --2- 172.21.15.120:0/4193218942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5008100e20 0x7f5008197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:08.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.271+0000 7f500ce22640 1 --2- 172.21.15.120:0/4193218942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5008102ef0 0x7f500819bd30 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f5000009fd0 tx=0x7f500000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:08.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.271+0000 7f50057fa640 1 -- 172.21.15.120:0/4193218942 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500000ce20 con 0x7f5008102ef0 2024-09-19T09:14:08.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.271+0000 7f500e885640 1 -- 172.21.15.120:0/4193218942 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f500819c700 con 0x7f5008102ef0 2024-09-19T09:14:08.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.271+0000 7f50057fa640 1 -- 172.21.15.120:0/4193218942 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5000005020 con 0x7f5008102ef0 2024-09-19T09:14:08.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.271+0000 7f50057fa640 1 -- 172.21.15.120:0/4193218942 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5000010720 con 0x7f5008102ef0 2024-09-19T09:14:08.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.271+0000 7f500e885640 1 -- 172.21.15.120:0/4193218942 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5008069f70 con 0x7f5008102ef0 2024-09-19T09:14:08.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.273+0000 7f50057fa640 1 -- 172.21.15.120:0/4193218942 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5000005540 con 0x7f5008102ef0 2024-09-19T09:14:08.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.274+0000 7f5010c5e740 1 Processor -- start 2024-09-19T09:14:08.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.274+0000 7f5010c5e740 1 -- start start 2024-09-19T09:14:08.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.274+0000 7f5010c5e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6dffe7b40 0x55e6dffe6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:08.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.274+0000 7f5010c5e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e6dffe65d0 0x55e6dffe6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:08.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.274+0000 7f50057fa640 1 --2- 172.21.15.120:0/4193218942 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4ff40a32c0 0x7f4ff40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:08.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.274+0000 7f5010c5e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6dfff1370 0x55e6dfff3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:08.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.274+0000 7f5010c5e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e6dff4c2f0 con 0x55e6dffe7b40 2024-09-19T09:14:08.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.274+0000 7f5010c5e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e6dfef3cf0 con 0x55e6dffe65d0 2024-09-19T09:14:08.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.274+0000 7f5010c5e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e6dff482b0 con 0x55e6dfff1370 2024-09-19T09:14:08.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.274+0000 7f50077fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6dffe7b40 0x55e6dffe6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:08.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.274+0000 7f5007fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e6dffe65d0 0x55e6dffe6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:08.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.274+0000 7f50077fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6dffe7b40 0x55e6dffe6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59420/0 (socket says 172.21.15.120:59420) 2024-09-19T09:14:08.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.274+0000 7f50077fe640 1 -- 172.21.15.120:0/4213975161 learned_addr learned my addr 172.21.15.120:0/4213975161 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:08.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.274+0000 7f500ce22640 1 --2- 172.21.15.120:0/4213975161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6dfff1370 0x55e6dfff3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:08.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.274+0000 7f50077fe640 1 --2- 172.21.15.120:0/4193218942 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4ff40a32c0 0x7f4ff40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:08.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.274+0000 7f50057fa640 1 -- 172.21.15.120:0/4193218942 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f50000cff30 con 0x7f5008102ef0 2024-09-19T09:14:08.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.274+0000 7f5007fff640 1 -- 172.21.15.120:0/4213975161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6dfff1370 msgr2=0x55e6dfff3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:08.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.274+0000 7f5007fff640 1 --2- 172.21.15.120:0/4213975161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6dfff1370 0x55e6dfff3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.274+0000 7f5007fff640 1 -- 172.21.15.120:0/4213975161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6dffe7b40 msgr2=0x55e6dffe6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:08.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.274+0000 7f5007fff640 1 --2- 172.21.15.120:0/4213975161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6dffe7b40 0x55e6dffe6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.274+0000 7f5007fff640 1 -- 172.21.15.120:0/4213975161 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6dfed83a0 con 0x55e6dffe65d0 2024-09-19T09:14:08.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.275+0000 7f5007fff640 1 --2- 172.21.15.120:0/4213975161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e6dffe65d0 0x55e6dffe6a10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8013820 tx=0x7f4ff8013c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e7c578850c0504 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:08.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.275+0000 7f50077fe640 1 --2- 172.21.15.120:0/4193218942 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4ff40a32c0 0x7f4ff40a5780 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00314e0 tx=0x7f4ff0009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:08.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.275+0000 7f4ffe7fc640 1 -- 172.21.15.120:0/4213975161 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff800e990 con 0x55e6dffe65d0 2024-09-19T09:14:08.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.275+0000 7f4ffe7fc640 1 -- 172.21.15.120:0/4213975161 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ff801c400 con 0x55e6dffe65d0 2024-09-19T09:14:08.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.275+0000 7f4ffe7fc640 1 -- 172.21.15.120:0/4213975161 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff801b610 con 0x55e6dffe65d0 2024-09-19T09:14:08.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.276+0000 7f5010c5e740 1 -- 172.21.15.120:0/4213975161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e6dffe65d0 msgr2=0x55e6dffe6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:08.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.276+0000 7f5010c5e740 1 --2- 172.21.15.120:0/4213975161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e6dffe65d0 0x55e6dffe6a10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8013820 tx=0x7f4ff8013c20 comp rx=0 tx=0).stop 2024-09-19T09:14:08.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.276+0000 7f5010c5e740 1 -- 172.21.15.120:0/4213975161 shutdown_connections 2024-09-19T09:14:08.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.276+0000 7f5010c5e740 1 --2- 172.21.15.120:0/4213975161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6dfff1370 0x55e6dfff3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.276+0000 7f5010c5e740 1 --2- 172.21.15.120:0/4213975161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e6dffe65d0 0x55e6dffe6a10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.276+0000 7f5010c5e740 1 --2- 172.21.15.120:0/4213975161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6dffe7b40 0x55e6dffe6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.276+0000 7f5010c5e740 1 -- 172.21.15.120:0/4213975161 >> 172.21.15.120:0/4213975161 conn(0x55e6dffaf310 msgr2=0x55e6dffe2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:08.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.276+0000 7f5010c5e740 1 -- 172.21.15.120:0/4213975161 shutdown_connections 2024-09-19T09:14:08.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.276+0000 7f5010c5e740 1 -- 172.21.15.120:0/4213975161 wait complete. 2024-09-19T09:14:08.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.276+0000 7f5010c5e740 1 -- start start 2024-09-19T09:14:08.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.276+0000 7f5010c5e740 1 Processor -- start 2024-09-19T09:14:08.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.277+0000 7f5010c5e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6dffe65d0 0x55e6dfff0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:08.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.277+0000 7f5010c5e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e6dffe7b40 0x55e6dffedf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:08.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.277+0000 7f5010c5e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6dfff1370 0x55e6dffee4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:08.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.277+0000 7f5010c5e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e6dfff4070 con 0x55e6dffe65d0 2024-09-19T09:14:08.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.277+0000 7f5010c5e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e6dff07060 con 0x55e6dffe7b40 2024-09-19T09:14:08.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.277+0000 7f5010c5e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e6dfff41f0 con 0x55e6dfff1370 2024-09-19T09:14:08.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.277+0000 7f5010c5e740 10 client.0 _ll_register_callbacks cb 0x55e6dfc5a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:08.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.277+0000 7f5007fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6dffe65d0 0x55e6dfff0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:08.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.277+0000 7f50077fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e6dffe7b40 0x55e6dffedf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:08.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.277+0000 7f500ce22640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6dfff1370 0x55e6dffee4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:08.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.277+0000 7f500ce22640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6dfff1370 0x55e6dffee4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:60236/0 (socket says 172.21.15.120:60236) 2024-09-19T09:14:08.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.277+0000 7f5007fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6dffe65d0 0x55e6dfff0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59428/0 (socket says 172.21.15.120:59428) 2024-09-19T09:14:08.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.277+0000 7f500ce22640 1 -- 172.21.15.120:0/3838268652 learned_addr learned my addr 172.21.15.120:0/3838268652 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:08.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.277+0000 7f5007fff640 1 -- 172.21.15.120:0/3838268652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6dfff1370 msgr2=0x55e6dffee4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:08.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.277+0000 7f5007fff640 1 --2- 172.21.15.120:0/3838268652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6dfff1370 0x55e6dffee4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.277+0000 7f5007fff640 1 -- 172.21.15.120:0/3838268652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e6dffe7b40 msgr2=0x55e6dffedf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:08.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.277+0000 7f5007fff640 1 --2- 172.21.15.120:0/3838268652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e6dffe7b40 0x55e6dffedf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.277+0000 7f5007fff640 1 -- 172.21.15.120:0/3838268652 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6dfcbd3c0 con 0x55e6dffe65d0 2024-09-19T09:14:08.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.277+0000 7f500ce22640 1 --2- 172.21.15.120:0/3838268652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6dfff1370 0x55e6dffee4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:08.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.277+0000 7f5007fff640 1 --2- 172.21.15.120:0/3838268652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6dffe65d0 0x55e6dfff0cc0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8013820 tx=0x7f4ff800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912446f584ce83e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:08.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.277+0000 7f4fe77fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:14:08.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.277+0000 7f4fe77fe640 1 -- 172.21.15.120:0/3838268652 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8029020 con 0x55e6dffe65d0 2024-09-19T09:14:08.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.278+0000 7f4fe77fe640 1 -- 172.21.15.120:0/3838268652 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f4ff8022040 con 0x55e6dffe65d0 2024-09-19T09:14:08.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.278+0000 7f5010c5e740 1 -- 172.21.15.120:0/3838268652 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ff801c400 con 0x55e6dffe65d0 2024-09-19T09:14:08.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.278+0000 7f4fe77fe640 1 -- 172.21.15.120:0/3838268652 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4ff8009cf0 con 0x55e6dffe65d0 2024-09-19T09:14:08.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.278+0000 7f5010c5e740 10 client.19443 fetch_fsmap learned FSMap version 32 2024-09-19T09:14:08.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.278+0000 7f5010c5e740 1 -- 172.21.15.120:0/3838268652 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e6dffffa80 con 0x55e6dffe65d0 2024-09-19T09:14:08.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.278+0000 7f4fe77fe640 1 -- 172.21.15.120:0/3838268652 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ff801c400 con 0x55e6dffe65d0 2024-09-19T09:14:08.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.278+0000 7f4fe77fe640 1 -- 172.21.15.120:0/3838268652 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f4ff80048a0 con 0x55e6dffe65d0 2024-09-19T09:14:08.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.278+0000 7f5010c5e740 10 client.19443 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:14:08.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.278+0000 7f5010c5e740 10 client.19443 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:14:08.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.278+0000 7f5010c5e740 20 client.19443 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:08.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.278+0000 7f5010c5e740 1 --2- 172.21.15.120:0/3838268652 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55e6e0000e90 0x55e6e0003280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:08.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.278+0000 7f5010c5e740 4 client.19443 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:14:08.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.278+0000 7f5010c5e740 1 -- 172.21.15.120:0/3838268652 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6dff59860 con 0x55e6e0000e90 2024-09-19T09:14:08.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.278+0000 7f50077fe640 1 --2- 172.21.15.120:0/3838268652 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55e6e0000e90 0x55e6e0003280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:08.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.279+0000 7f4fe77fe640 10 client.19443 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:14:08.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.279+0000 7f50077fe640 1 --2- 172.21.15.120:0/3838268652 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55e6e0000e90 0x55e6e0003280 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a54efb5033714be1 server_cookie=efc4e29e8ee46565 in_seq=0 out_seq=0 2024-09-19T09:14:08.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.279+0000 7f4fe77fe640 1 -- 172.21.15.120:0/3838268652 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e6dff59860 con 0x55e6e0000e90 2024-09-19T09:14:08.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.279+0000 7f4fe77fe640 10 client.19443 handle_command_reply: tid=0 2024-09-19T09:14:08.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.279+0000 7f5010c5e740 1 client.19443 shutdown 2024-09-19T09:14:08.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.280+0000 7f5010c5e740 1 -- 172.21.15.120:0/3838268652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6dffe65d0 msgr2=0x55e6dfff0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:08.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.280+0000 7f5010c5e740 1 --2- 172.21.15.120:0/3838268652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6dffe65d0 0x55e6dfff0cc0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8013820 tx=0x7f4ff800ec40 comp rx=0 tx=0).stop 2024-09-19T09:14:08.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.280+0000 7f5010c5e740 1 -- 172.21.15.120:0/3838268652 shutdown_connections 2024-09-19T09:14:08.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.280+0000 7f5010c5e740 1 --2- 172.21.15.120:0/3838268652 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55e6e0000e90 0x55e6e0003280 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.280+0000 7f4fe77fe640 0 client.19443 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:14:08.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.280+0000 7f5010c5e740 1 --2- 172.21.15.120:0/3838268652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6dfff1370 0x55e6dffee4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.280+0000 7f5010c5e740 1 --2- 172.21.15.120:0/3838268652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e6dffe7b40 0x55e6dffedf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.280+0000 7f5010c5e740 1 --2- 172.21.15.120:0/3838268652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6dffe65d0 0x55e6dfff0cc0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.280+0000 7f5010c5e740 1 -- 172.21.15.120:0/3838268652 >> 172.21.15.120:0/3838268652 conn(0x55e6dffaf310 msgr2=0x55e6dffe8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:08.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.280+0000 7f5010c5e740 1 -- 172.21.15.120:0/3838268652 shutdown_connections 2024-09-19T09:14:08.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.280+0000 7f5010c5e740 1 -- 172.21.15.120:0/3838268652 wait complete. 2024-09-19T09:14:08.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.280+0000 7f5010c5e740 20 client.19443 trim_cache size 0 max 16384 2024-09-19T09:14:08.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.298+0000 7f5010c5e740 1 Processor -- start 2024-09-19T09:14:08.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.298+0000 7f5010c5e740 1 -- start start 2024-09-19T09:14:08.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.298+0000 7f5010c5e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6dffe65d0 0x55e6dfffd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:08.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.298+0000 7f5010c5e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6dffe7b40 0x55e6dffed870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:08.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.298+0000 7f5010c5e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e6dfff1370 0x55e6dffe1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:08.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.298+0000 7f5010c5e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f50081ab2c0 con 0x55e6dffe65d0 2024-09-19T09:14:08.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.298+0000 7f5010c5e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f50081ace00 con 0x55e6dfff1370 2024-09-19T09:14:08.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.298+0000 7f5010c5e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f50081ab990 con 0x55e6dffe7b40 2024-09-19T09:14:08.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.298+0000 7f5007fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6dffe65d0 0x55e6dfffd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:08.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.298+0000 7f5007fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6dffe65d0 0x55e6dfffd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59434/0 (socket says 172.21.15.120:59434) 2024-09-19T09:14:08.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.298+0000 7f5007fff640 1 -- 172.21.15.120:0/282342644 learned_addr learned my addr 172.21.15.120:0/282342644 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:08.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.298+0000 7f50077fe640 1 --2- 172.21.15.120:0/282342644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6dffe7b40 0x55e6dffed870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:08.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.299+0000 7f500ce22640 1 --2- 172.21.15.120:0/282342644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e6dfff1370 0x55e6dffe1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:08.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.299+0000 7f5007fff640 1 -- 172.21.15.120:0/282342644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6dffe7b40 msgr2=0x55e6dffed870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:08.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.299+0000 7f5007fff640 1 --2- 172.21.15.120:0/282342644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6dffe7b40 0x55e6dffed870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.299+0000 7f5007fff640 1 -- 172.21.15.120:0/282342644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e6dfff1370 msgr2=0x55e6dffe1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:08.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.299+0000 7f5007fff640 1 --2- 172.21.15.120:0/282342644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e6dfff1370 0x55e6dffe1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.299+0000 7f5007fff640 1 -- 172.21.15.120:0/282342644 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6dffec4e0 con 0x55e6dffe65d0 2024-09-19T09:14:08.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.299+0000 7f5007fff640 1 --2- 172.21.15.120:0/282342644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6dffe65d0 0x55e6dfffd310 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8032040 tx=0x7f4ff8005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6531cff1c13a53 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:08.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.299+0000 7f4ffdffb640 1 -- 172.21.15.120:0/282342644 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff803d070 con 0x55e6dffe65d0 2024-09-19T09:14:08.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.299+0000 7f4ffdffb640 1 -- 172.21.15.120:0/282342644 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ff8009cf0 con 0x55e6dffe65d0 2024-09-19T09:14:08.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.299+0000 7f4ffdffb640 1 -- 172.21.15.120:0/282342644 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff801c8c0 con 0x55e6dffe65d0 2024-09-19T09:14:08.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.300+0000 7f5010c5e740 1 -- 172.21.15.120:0/282342644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6dffe65d0 msgr2=0x55e6dfffd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:08.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.300+0000 7f5010c5e740 1 --2- 172.21.15.120:0/282342644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6dffe65d0 0x55e6dfffd310 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8032040 tx=0x7f4ff8005db0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.300+0000 7f5010c5e740 1 -- 172.21.15.120:0/282342644 shutdown_connections 2024-09-19T09:14:08.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.300+0000 7f5010c5e740 1 --2- 172.21.15.120:0/282342644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e6dfff1370 0x55e6dffe1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.300+0000 7f5010c5e740 1 --2- 172.21.15.120:0/282342644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6dffe7b40 0x55e6dffed870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.300+0000 7f5010c5e740 1 --2- 172.21.15.120:0/282342644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6dffe65d0 0x55e6dfffd310 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.300+0000 7f5010c5e740 1 -- 172.21.15.120:0/282342644 >> 172.21.15.120:0/282342644 conn(0x55e6dffaf310 msgr2=0x55e6dffe14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:08.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.300+0000 7f5010c5e740 1 -- 172.21.15.120:0/282342644 shutdown_connections 2024-09-19T09:14:08.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.300+0000 7f5010c5e740 1 -- 172.21.15.120:0/282342644 wait complete. 2024-09-19T09:14:08.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.301+0000 7f5010c5e740 1 -- start start 2024-09-19T09:14:08.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.301+0000 7f5010c5e740 1 Processor -- start 2024-09-19T09:14:08.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.301+0000 7f5010c5e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6dffe65d0 0x55e6dfff9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:08.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.301+0000 7f5010c5e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e6dffe7b40 0x55e6dfff99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:08.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.301+0000 7f5010c5e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6dfff1370 0x55e6dffea5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:08.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.301+0000 7f5010c5e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f50081ac730 con 0x55e6dffe65d0 2024-09-19T09:14:08.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.301+0000 7f5010c5e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f50081ac060 con 0x55e6dffe7b40 2024-09-19T09:14:08.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.301+0000 7f5010c5e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f50081a9280 con 0x55e6dfff1370 2024-09-19T09:14:08.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.301+0000 7f5010c5e740 10 client.0 _ll_register_callbacks cb 0x7f4ff8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:08.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.301+0000 7f5007fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6dffe65d0 0x55e6dfff9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:08.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.301+0000 7f5007fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6dffe65d0 0x55e6dfff9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59438/0 (socket says 172.21.15.120:59438) 2024-09-19T09:14:08.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.301+0000 7f5007fff640 1 -- 172.21.15.120:0/3578026400 learned_addr learned my addr 172.21.15.120:0/3578026400 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:08.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.301+0000 7f50077fe640 1 --2- 172.21.15.120:0/3578026400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e6dffe7b40 0x55e6dfff99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:08.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.302+0000 7f500ce22640 1 --2- 172.21.15.120:0/3578026400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6dfff1370 0x55e6dffea5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:08.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.302+0000 7f5007fff640 1 -- 172.21.15.120:0/3578026400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6dfff1370 msgr2=0x55e6dffea5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:08.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.302+0000 7f5007fff640 1 --2- 172.21.15.120:0/3578026400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6dfff1370 0x55e6dffea5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.302+0000 7f5007fff640 1 -- 172.21.15.120:0/3578026400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e6dffe7b40 msgr2=0x55e6dfff99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:08.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.302+0000 7f5007fff640 1 --2- 172.21.15.120:0/3578026400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e6dffe7b40 0x55e6dfff99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.302+0000 7f5007fff640 1 -- 172.21.15.120:0/3578026400 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6dffeab00 con 0x55e6dffe65d0 2024-09-19T09:14:08.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.302+0000 7f50077fe640 1 --2- 172.21.15.120:0/3578026400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e6dffe7b40 0x55e6dfff99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:08.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.302+0000 7f5007fff640 1 --2- 172.21.15.120:0/3578026400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6dffe65d0 0x55e6dfff9480 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f4ff800e780 tx=0x7f4ff8005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfad80cd5113c96b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:08.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.302+0000 7f4fe77fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:14:08.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.302+0000 7f4fe77fe640 1 -- 172.21.15.120:0/3578026400 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8004890 con 0x55e6dffe65d0 2024-09-19T09:14:08.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.302+0000 7f4fe77fe640 1 -- 172.21.15.120:0/3578026400 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f4ff8044980 con 0x55e6dffe65d0 2024-09-19T09:14:08.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.302+0000 7f5010c5e740 1 -- 172.21.15.120:0/3578026400 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ff8009cf0 con 0x55e6dffe65d0 2024-09-19T09:14:08.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.302+0000 7f4fe77fe640 1 -- 172.21.15.120:0/3578026400 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4ff8004a30 con 0x55e6dffe65d0 2024-09-19T09:14:08.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.302+0000 7f5010c5e740 10 client.19449 fetch_fsmap learned FSMap version 32 2024-09-19T09:14:08.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.302+0000 7f5010c5e740 1 -- 172.21.15.120:0/3578026400 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e6e006b070 con 0x55e6dffe65d0 2024-09-19T09:14:08.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.302+0000 7f4fe77fe640 1 -- 172.21.15.120:0/3578026400 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ff8009cf0 con 0x55e6dffe65d0 2024-09-19T09:14:08.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.303+0000 7f4fe77fe640 1 -- 172.21.15.120:0/3578026400 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f4ff8037030 con 0x55e6dffe65d0 2024-09-19T09:14:08.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.303+0000 7f5010c5e740 10 client.19449 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:14:08.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.303+0000 7f5010c5e740 10 client.19449 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:14:08.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.303+0000 7f5010c5e740 20 client.19449 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:08.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.303+0000 7f5010c5e740 1 --2- 172.21.15.120:0/3578026400 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55e6e0000e90 0x55e6e006d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:08.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.303+0000 7f5010c5e740 4 client.19449 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:14:08.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.303+0000 7f5010c5e740 1 -- 172.21.15.120:0/3578026400 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e6dff53d70 con 0x55e6e0000e90 2024-09-19T09:14:08.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.303+0000 7f50077fe640 1 --2- 172.21.15.120:0/3578026400 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55e6e0000e90 0x55e6e006d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:08.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.303+0000 7f4fe77fe640 10 client.19449 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:14:08.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.303+0000 7f50077fe640 1 --2- 172.21.15.120:0/3578026400 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55e6e0000e90 0x55e6e006d680 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3cd754d246a6fe0e server_cookie=fe92063bcf6a589b in_seq=0 out_seq=0 2024-09-19T09:14:08.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.303+0000 7f4fe77fe640 1 -- 172.21.15.120:0/3578026400 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e6dff53d70 con 0x55e6e0000e90 2024-09-19T09:14:08.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.303+0000 7f4fe77fe640 10 client.19449 handle_command_reply: tid=0 2024-09-19T09:14:08.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.303+0000 7f5010c5e740 1 client.19449 shutdown 2024-09-19T09:14:08.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.304+0000 7f5010c5e740 1 -- 172.21.15.120:0/3578026400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6dffe65d0 msgr2=0x55e6dfff9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:08.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.304+0000 7f5010c5e740 1 --2- 172.21.15.120:0/3578026400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6dffe65d0 0x55e6dfff9480 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f4ff800e780 tx=0x7f4ff8005590 comp rx=0 tx=0).stop 2024-09-19T09:14:08.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.304+0000 7f5010c5e740 1 -- 172.21.15.120:0/3578026400 shutdown_connections 2024-09-19T09:14:08.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.304+0000 7f5010c5e740 1 --2- 172.21.15.120:0/3578026400 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55e6e0000e90 0x55e6e006d680 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.304+0000 7f5010c5e740 1 --2- 172.21.15.120:0/3578026400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6dfff1370 0x55e6dffea5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.304+0000 7f5010c5e740 1 --2- 172.21.15.120:0/3578026400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e6dffe7b40 0x55e6dfff99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.304+0000 7f5010c5e740 1 --2- 172.21.15.120:0/3578026400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6dffe65d0 0x55e6dfff9480 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.304+0000 7f5010c5e740 1 -- 172.21.15.120:0/3578026400 >> 172.21.15.120:0/3578026400 conn(0x55e6dffaf310 msgr2=0x55e6dffe1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:08.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.304+0000 7f4fe77fe640 0 client.19449 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:14:08.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.304+0000 7f5010c5e740 1 -- 172.21.15.120:0/3578026400 shutdown_connections 2024-09-19T09:14:08.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.304+0000 7f5010c5e740 1 -- 172.21.15.120:0/3578026400 wait complete. 2024-09-19T09:14:08.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.304+0000 7f5010c5e740 20 client.19449 trim_cache size 0 max 16384 2024-09-19T09:14:08.311 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:14:08.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.304+0000 7f500e885640 1 -- 172.21.15.120:0/4193218942 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4ff40a32c0 msgr2=0x7f4ff40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:08.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.304+0000 7f500e885640 1 --2- 172.21.15.120:0/4193218942 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4ff40a32c0 0x7f4ff40a5780 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00314e0 tx=0x7f4ff0009560 comp rx=0 tx=0).stop 2024-09-19T09:14:08.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.304+0000 7f500e885640 1 -- 172.21.15.120:0/4193218942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5008102ef0 msgr2=0x7f500819bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:08.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.304+0000 7f500e885640 1 --2- 172.21.15.120:0/4193218942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5008102ef0 0x7f500819bd30 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f5000009fd0 tx=0x7f500000ef10 comp rx=0 tx=0).stop 2024-09-19T09:14:08.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.305+0000 7f500e885640 1 -- 172.21.15.120:0/4193218942 shutdown_connections 2024-09-19T09:14:08.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.305+0000 7f500e885640 1 --2- 172.21.15.120:0/4193218942 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4ff40a32c0 0x7f4ff40a5780 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.305+0000 7f500e885640 1 --2- 172.21.15.120:0/4193218942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5008102ef0 0x7f500819bd30 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.305+0000 7f500e885640 1 --2- 172.21.15.120:0/4193218942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5008102570 0x7f50081979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.305+0000 7f500e885640 1 --2- 172.21.15.120:0/4193218942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5008100e20 0x7f5008197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:08.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.305+0000 7f500e885640 1 -- 172.21.15.120:0/4193218942 >> 172.21.15.120:0/4193218942 conn(0x7f50080fc7d0 msgr2=0x7f50080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:08.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.305+0000 7f500e885640 1 -- 172.21.15.120:0/4193218942 shutdown_connections 2024-09-19T09:14:08.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:08.305+0000 7f500e885640 1 -- 172.21.15.120:0/4193218942 wait complete. 2024-09-19T09:14:09.073 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1054079 9.27 MB/sec execute 484 sec latency 38.316 ms 2024-09-19T09:14:09.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:08 smithi120 ceph-mon[31150]: pgmap v681: 97 pgs: 97 active+clean; 41 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.9 MiB/s wr, 354 op/s 2024-09-19T09:14:09.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:08 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:14:09.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:08 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:14:09.079 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:14:09.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:08 smithi196 ceph-mon[35150]: pgmap v681: 97 pgs: 97 active+clean; 41 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.9 MiB/s wr, 354 op/s 2024-09-19T09:14:09.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:08 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:14:09.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:08 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:14:09.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:08 smithi153 ceph-mon[35071]: pgmap v681: 97 pgs: 97 active+clean; 41 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.9 MiB/s wr, 354 op/s 2024-09-19T09:14:09.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:08 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:14:09.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:08 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:14:09.340 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:14:09.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.860+0000 7f97c7edf640 1 -- 172.21.15.120:0/2424172740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97c0102ef0 msgr2=0x7f97c010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:09.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.860+0000 7f97c7edf640 1 --2- 172.21.15.120:0/2424172740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97c0102ef0 0x7f97c010da40 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f97b8009ca0 tx=0x7f97b802a950 comp rx=0 tx=0).stop 2024-09-19T09:14:09.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.860+0000 7f97c7edf640 1 -- 172.21.15.120:0/2424172740 shutdown_connections 2024-09-19T09:14:09.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.860+0000 7f97c7edf640 1 --2- 172.21.15.120:0/2424172740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97c0102ef0 0x7f97c010da40 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.860+0000 7f97c7edf640 1 --2- 172.21.15.120:0/2424172740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97c0102570 0x7f97c01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.860+0000 7f97c7edf640 1 --2- 172.21.15.120:0/2424172740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97c0100e20 0x7f97c0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.860+0000 7f97c7edf640 1 -- 172.21.15.120:0/2424172740 >> 172.21.15.120:0/2424172740 conn(0x7f97c00fc7d0 msgr2=0x7f97c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:09.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.860+0000 7f97c7edf640 1 -- 172.21.15.120:0/2424172740 shutdown_connections 2024-09-19T09:14:09.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.861+0000 7f97c7edf640 1 -- 172.21.15.120:0/2424172740 wait complete. 2024-09-19T09:14:09.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.861+0000 7f97c7edf640 1 Processor -- start 2024-09-19T09:14:09.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.861+0000 7f97c7edf640 1 -- start start 2024-09-19T09:14:09.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.861+0000 7f97c7edf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97c0100e20 0x7f97c019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:09.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.861+0000 7f97c7edf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97c0102570 0x7f97c019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:09.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.861+0000 7f97c7edf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97c0102ef0 0x7f97c019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:09.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.861+0000 7f97c7edf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f97c0110f80 con 0x7f97c0102570 2024-09-19T09:14:09.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.861+0000 7f97c7edf640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f97c0110e00 con 0x7f97c0102ef0 2024-09-19T09:14:09.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.861+0000 7f97c7edf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f97c0111100 con 0x7f97c0100e20 2024-09-19T09:14:09.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.861+0000 7f97c547a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97c0102570 0x7f97c019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:09.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.861+0000 7f97c547a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97c0102570 0x7f97c019abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59458/0 (socket says 172.21.15.120:59458) 2024-09-19T09:14:09.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.861+0000 7f97c5c7b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97c0100e20 0x7f97c019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:09.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.861+0000 7f97c547a640 1 -- 172.21.15.120:0/3619888645 learned_addr learned my addr 172.21.15.120:0/3619888645 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:09.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.861+0000 7f97c647c640 1 --2- 172.21.15.120:0/3619888645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97c0102ef0 0x7f97c019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:09.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.862+0000 7f97c547a640 1 -- 172.21.15.120:0/3619888645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97c0100e20 msgr2=0x7f97c019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:09.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.862+0000 7f97c547a640 1 --2- 172.21.15.120:0/3619888645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97c0100e20 0x7f97c019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.862+0000 7f97c547a640 1 -- 172.21.15.120:0/3619888645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97c0102ef0 msgr2=0x7f97c019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:09.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.862+0000 7f97c547a640 1 --2- 172.21.15.120:0/3619888645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97c0102ef0 0x7f97c019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.862+0000 7f97c547a640 1 -- 172.21.15.120:0/3619888645 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c019f660 con 0x7f97c0102570 2024-09-19T09:14:09.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.862+0000 7f97c5c7b640 1 --2- 172.21.15.120:0/3619888645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97c0100e20 0x7f97c019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:09.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.862+0000 7f97c547a640 1 --2- 172.21.15.120:0/3619888645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97c0102570 0x7f97c019abf0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f97b000e9f0 tx=0x7f97b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:09.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.862+0000 7f97b6ffd640 1 -- 172.21.15.120:0/3619888645 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b000cd80 con 0x7f97c0102570 2024-09-19T09:14:09.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.862+0000 7f97c7edf640 1 -- 172.21.15.120:0/3619888645 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97c019f950 con 0x7f97c0102570 2024-09-19T09:14:09.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.862+0000 7f97b6ffd640 1 -- 172.21.15.120:0/3619888645 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97b0010910 con 0x7f97c0102570 2024-09-19T09:14:09.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.862+0000 7f97b6ffd640 1 -- 172.21.15.120:0/3619888645 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b00188f0 con 0x7f97c0102570 2024-09-19T09:14:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.862+0000 7f97c7edf640 1 -- 172.21.15.120:0/3619888645 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97c01a7230 con 0x7f97c0102570 2024-09-19T09:14:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.865+0000 7f97ca2b8740 1 Processor -- start 2024-09-19T09:14:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.865+0000 7f97b6ffd640 1 -- 172.21.15.120:0/3619888645 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f97b0010ac0 con 0x7f97c0102570 2024-09-19T09:14:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.865+0000 7f97ca2b8740 1 -- start start 2024-09-19T09:14:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.866+0000 7f97ca2b8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56268534ab40 0x562685349090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.866+0000 7f97ca2b8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626853495d0 0x562685349a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.866+0000 7f97ca2b8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562685354370 0x562685356760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.866+0000 7f97ca2b8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5626852af2f0 con 0x56268534ab40 2024-09-19T09:14:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.866+0000 7f97ca2b8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562685256cf0 con 0x5626853495d0 2024-09-19T09:14:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.866+0000 7f97ca2b8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5626852ab2b0 con 0x562685354370 2024-09-19T09:14:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.866+0000 7f97c647c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56268534ab40 0x562685349090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.866+0000 7f97c5c7b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626853495d0 0x562685349a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.866+0000 7f97c547a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562685354370 0x562685356760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.866+0000 7f97c647c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56268534ab40 0x562685349090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59470/0 (socket says 172.21.15.120:59470) 2024-09-19T09:14:09.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.866+0000 7f97c647c640 1 -- 172.21.15.120:0/3282096455 learned_addr learned my addr 172.21.15.120:0/3282096455 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:09.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.866+0000 7f97c5c7b640 1 -- 172.21.15.120:0/3282096455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562685354370 msgr2=0x562685356760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:09.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.866+0000 7f97c5c7b640 1 --2- 172.21.15.120:0/3282096455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562685354370 0x562685356760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.866+0000 7f97c5c7b640 1 -- 172.21.15.120:0/3282096455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56268534ab40 msgr2=0x562685349090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:09.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.866+0000 7f97c5c7b640 1 --2- 172.21.15.120:0/3282096455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56268534ab40 0x562685349090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.866+0000 7f97c5c7b640 1 -- 172.21.15.120:0/3282096455 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56268523b3a0 con 0x5626853495d0 2024-09-19T09:14:09.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.866+0000 7f97c647c640 1 --2- 172.21.15.120:0/3282096455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56268534ab40 0x562685349090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:09.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.866+0000 7f97b6ffd640 1 --2- 172.21.15.120:0/3619888645 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f97ac0a3240 0x7f97ac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:09.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.866+0000 7f97c5c7b640 1 --2- 172.21.15.120:0/3282096455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626853495d0 0x562685349a10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f97bc009fd0 tx=0x7f97bc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f219c4b3ab1e80db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:09.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.866+0000 7f97c647c640 1 --2- 172.21.15.120:0/3619888645 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f97ac0a3240 0x7f97ac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:09.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.866+0000 7f97b6ffd640 1 -- 172.21.15.120:0/3619888645 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f97b00cfca0 con 0x7f97c0102570 2024-09-19T09:14:09.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.867+0000 7f97abfff640 1 -- 172.21.15.120:0/3282096455 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc004310 con 0x5626853495d0 2024-09-19T09:14:09.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.867+0000 7f97abfff640 1 -- 172.21.15.120:0/3282096455 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97bc0044b0 con 0x5626853495d0 2024-09-19T09:14:09.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.867+0000 7f97abfff640 1 -- 172.21.15.120:0/3282096455 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc0035f0 con 0x5626853495d0 2024-09-19T09:14:09.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.867+0000 7f97ca2b8740 1 -- 172.21.15.120:0/3282096455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626853495d0 msgr2=0x562685349a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:09.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.867+0000 7f97c647c640 1 --2- 172.21.15.120:0/3619888645 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f97ac0a3240 0x7f97ac0a5700 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f97b8009ca0 tx=0x7f97b8005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:09.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.867+0000 7f97ca2b8740 1 --2- 172.21.15.120:0/3282096455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626853495d0 0x562685349a10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f97bc009fd0 tx=0x7f97bc0118d0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.867+0000 7f97ca2b8740 1 -- 172.21.15.120:0/3282096455 shutdown_connections 2024-09-19T09:14:09.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.867+0000 7f97ca2b8740 1 --2- 172.21.15.120:0/3282096455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562685354370 0x562685356760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.867+0000 7f97ca2b8740 1 --2- 172.21.15.120:0/3282096455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626853495d0 0x562685349a10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.867+0000 7f97ca2b8740 1 --2- 172.21.15.120:0/3282096455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56268534ab40 0x562685349090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.867+0000 7f97ca2b8740 1 -- 172.21.15.120:0/3282096455 >> 172.21.15.120:0/3282096455 conn(0x562685312310 msgr2=0x562685345c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:09.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.868+0000 7f97ca2b8740 1 -- 172.21.15.120:0/3282096455 shutdown_connections 2024-09-19T09:14:09.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.868+0000 7f97ca2b8740 1 -- 172.21.15.120:0/3282096455 wait complete. 2024-09-19T09:14:09.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.868+0000 7f97ca2b8740 1 -- start start 2024-09-19T09:14:09.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.868+0000 7f97ca2b8740 1 Processor -- start 2024-09-19T09:14:09.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.868+0000 7f97ca2b8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626853495d0 0x562685353cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:09.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.868+0000 7f97ca2b8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56268534ab40 0x562685350f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:09.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.868+0000 7f97ca2b8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562685354370 0x5626853514d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:09.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.868+0000 7f97ca2b8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562685357070 con 0x562685354370 2024-09-19T09:14:09.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.868+0000 7f97ca2b8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56268526a060 con 0x5626853495d0 2024-09-19T09:14:09.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.868+0000 7f97ca2b8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5626853571f0 con 0x56268534ab40 2024-09-19T09:14:09.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.868+0000 7f97ca2b8740 10 client.0 _ll_register_callbacks cb 0x562684fbd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:09.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.869+0000 7f97c647c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562685354370 0x5626853514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:09.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.869+0000 7f97c547a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56268534ab40 0x562685350f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:09.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.869+0000 7f97c5c7b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626853495d0 0x562685353cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:09.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.869+0000 7f97c547a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56268534ab40 0x562685350f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:60292/0 (socket says 172.21.15.120:60292) 2024-09-19T09:14:09.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.869+0000 7f97c547a640 1 -- 172.21.15.120:0/2718212265 learned_addr learned my addr 172.21.15.120:0/2718212265 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:09.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.869+0000 7f97c547a640 1 -- 172.21.15.120:0/2718212265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626853495d0 msgr2=0x562685353cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:09.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.869+0000 7f97c547a640 1 --2- 172.21.15.120:0/2718212265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626853495d0 0x562685353cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.869+0000 7f97c547a640 1 -- 172.21.15.120:0/2718212265 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562685354370 msgr2=0x5626853514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:09.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.869+0000 7f97c547a640 1 --2- 172.21.15.120:0/2718212265 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562685354370 0x5626853514d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.869+0000 7f97c547a640 1 -- 172.21.15.120:0/2718212265 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626850203c0 con 0x56268534ab40 2024-09-19T09:14:09.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.869+0000 7f97c547a640 1 --2- 172.21.15.120:0/2718212265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56268534ab40 0x562685350f70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f97b009c070 tx=0x7f97b009c140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd3a7213bb485723 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:09.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.869+0000 7f97a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:14:09.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.869+0000 7f97a97fa640 1 -- 172.21.15.120:0/2718212265 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b009c370 con 0x56268534ab40 2024-09-19T09:14:09.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.869+0000 7f97a97fa640 1 -- 172.21.15.120:0/2718212265 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f97b00bc0e0 con 0x56268534ab40 2024-09-19T09:14:09.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.869+0000 7f97ca2b8740 1 -- 172.21.15.120:0/2718212265 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97bc0044b0 con 0x56268534ab40 2024-09-19T09:14:09.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.869+0000 7f97a97fa640 1 -- 172.21.15.120:0/2718212265 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f97b00a0670 con 0x56268534ab40 2024-09-19T09:14:09.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.869+0000 7f97ca2b8740 10 client.28318 fetch_fsmap learned FSMap version 32 2024-09-19T09:14:09.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.870+0000 7f97ca2b8740 1 -- 172.21.15.120:0/2718212265 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562685362ad0 con 0x56268534ab40 2024-09-19T09:14:09.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.870+0000 7f97a97fa640 1 -- 172.21.15.120:0/2718212265 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97bc0044b0 con 0x56268534ab40 2024-09-19T09:14:09.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.870+0000 7f97a97fa640 1 -- 172.21.15.120:0/2718212265 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f97b009b7c0 con 0x56268534ab40 2024-09-19T09:14:09.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.870+0000 7f97ca2b8740 10 client.28318 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:14:09.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.870+0000 7f97ca2b8740 10 client.28318 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:14:09.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.870+0000 7f97ca2b8740 20 client.28318 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:09.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.870+0000 7f97ca2b8740 1 --2- 172.21.15.120:0/2718212265 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x562685363f90 0x5626853663a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:09.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.870+0000 7f97ca2b8740 4 client.28318 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:14:09.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.870+0000 7f97ca2b8740 1 -- 172.21.15.120:0/2718212265 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626852bc860 con 0x562685363f90 2024-09-19T09:14:09.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.870+0000 7f97c5c7b640 1 --2- 172.21.15.120:0/2718212265 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x562685363f90 0x5626853663a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:09.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.871+0000 7f97a97fa640 10 client.28318 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:14:09.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.871+0000 7f97c5c7b640 1 --2- 172.21.15.120:0/2718212265 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x562685363f90 0x5626853663a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e2ffd48d07bb52bd server_cookie=21d89ae19fdecfb3 in_seq=0 out_seq=0 2024-09-19T09:14:09.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.871+0000 7f97a97fa640 1 -- 172.21.15.120:0/2718212265 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5626852bc860 con 0x562685363f90 2024-09-19T09:14:09.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.871+0000 7f97a97fa640 10 client.28318 handle_command_reply: tid=0 2024-09-19T09:14:09.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.871+0000 7f97ca2b8740 1 client.28318 shutdown 2024-09-19T09:14:09.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.871+0000 7f97ca2b8740 1 -- 172.21.15.120:0/2718212265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56268534ab40 msgr2=0x562685350f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:09.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.871+0000 7f97ca2b8740 1 --2- 172.21.15.120:0/2718212265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56268534ab40 0x562685350f70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f97b009c070 tx=0x7f97b009c140 comp rx=0 tx=0).stop 2024-09-19T09:14:09.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.872+0000 7f97ca2b8740 1 -- 172.21.15.120:0/2718212265 shutdown_connections 2024-09-19T09:14:09.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.872+0000 7f97ca2b8740 1 --2- 172.21.15.120:0/2718212265 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x562685363f90 0x5626853663a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.872+0000 7f97ca2b8740 1 --2- 172.21.15.120:0/2718212265 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562685354370 0x5626853514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.872+0000 7f97ca2b8740 1 --2- 172.21.15.120:0/2718212265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56268534ab40 0x562685350f70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.872+0000 7f97ca2b8740 1 --2- 172.21.15.120:0/2718212265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626853495d0 0x562685353cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.872+0000 7f97ca2b8740 1 -- 172.21.15.120:0/2718212265 >> 172.21.15.120:0/2718212265 conn(0x562685312310 msgr2=0x56268534b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:09.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.872+0000 7f97a97fa640 0 client.28318 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:14:09.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.872+0000 7f97ca2b8740 1 -- 172.21.15.120:0/2718212265 shutdown_connections 2024-09-19T09:14:09.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.872+0000 7f97ca2b8740 1 -- 172.21.15.120:0/2718212265 wait complete. 2024-09-19T09:14:09.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.872+0000 7f97ca2b8740 20 client.28318 trim_cache size 0 max 16384 2024-09-19T09:14:09.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.885+0000 7f97ca2b8740 1 Processor -- start 2024-09-19T09:14:09.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.885+0000 7f97ca2b8740 1 -- start start 2024-09-19T09:14:09.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.885+0000 7f97ca2b8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626853495d0 0x562685345e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:09.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.885+0000 7f97ca2b8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56268534ab40 0x562685360310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:09.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.885+0000 7f97ca2b8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562685354370 0x56268534a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:09.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.885+0000 7f97ca2b8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f97c01af6f0 con 0x562685354370 2024-09-19T09:14:09.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.885+0000 7f97ca2b8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f97c01b1230 con 0x5626853495d0 2024-09-19T09:14:09.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.885+0000 7f97ca2b8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f97c01afdc0 con 0x56268534ab40 2024-09-19T09:14:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.886+0000 7f97c547a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56268534ab40 0x562685360310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.886+0000 7f97c5c7b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626853495d0 0x562685345e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.886+0000 7f97c547a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56268534ab40 0x562685360310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:60302/0 (socket says 172.21.15.120:60302) 2024-09-19T09:14:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.886+0000 7f97c647c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562685354370 0x56268534a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.886+0000 7f97c5c7b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626853495d0 0x562685345e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:57712/0 (socket says 172.21.15.120:57712) 2024-09-19T09:14:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.886+0000 7f97c547a640 1 -- 172.21.15.120:0/3857423649 learned_addr learned my addr 172.21.15.120:0/3857423649 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.886+0000 7f97c647c640 1 -- 172.21.15.120:0/3857423649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56268534ab40 msgr2=0x562685360310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.886+0000 7f97c647c640 1 --2- 172.21.15.120:0/3857423649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56268534ab40 0x562685360310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.886+0000 7f97c647c640 1 -- 172.21.15.120:0/3857423649 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626853495d0 msgr2=0x562685345e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.886+0000 7f97c647c640 1 --2- 172.21.15.120:0/3857423649 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626853495d0 0x562685345e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.886+0000 7f97c647c640 1 -- 172.21.15.120:0/3857423649 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56268534f4e0 con 0x562685354370 2024-09-19T09:14:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.886+0000 7f97c647c640 1 --2- 172.21.15.120:0/3857423649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562685354370 0x56268534a300 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f97b8005cc0 tx=0x7f97b8045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb7c448a0938c05 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.886+0000 7f97ab7fe640 1 -- 172.21.15.120:0/3857423649 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b8034040 con 0x562685354370 2024-09-19T09:14:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.886+0000 7f97ab7fe640 1 -- 172.21.15.120:0/3857423649 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97b8003c30 con 0x562685354370 2024-09-19T09:14:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.886+0000 7f97ab7fe640 1 -- 172.21.15.120:0/3857423649 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b80307c0 con 0x562685354370 2024-09-19T09:14:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.887+0000 7f97ca2b8740 1 -- 172.21.15.120:0/3857423649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562685354370 msgr2=0x56268534a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.887+0000 7f97ca2b8740 1 --2- 172.21.15.120:0/3857423649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562685354370 0x56268534a300 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f97b8005cc0 tx=0x7f97b8045000 comp rx=0 tx=0).stop 2024-09-19T09:14:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.887+0000 7f97ca2b8740 1 -- 172.21.15.120:0/3857423649 shutdown_connections 2024-09-19T09:14:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.887+0000 7f97ca2b8740 1 --2- 172.21.15.120:0/3857423649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562685354370 0x56268534a300 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.887+0000 7f97ca2b8740 1 --2- 172.21.15.120:0/3857423649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56268534ab40 0x562685360310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.887+0000 7f97ca2b8740 1 --2- 172.21.15.120:0/3857423649 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626853495d0 0x562685345e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.887+0000 7f97ca2b8740 1 -- 172.21.15.120:0/3857423649 >> 172.21.15.120:0/3857423649 conn(0x562685312310 msgr2=0x562685345700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.887+0000 7f97ca2b8740 1 -- 172.21.15.120:0/3857423649 shutdown_connections 2024-09-19T09:14:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.887+0000 7f97ca2b8740 1 -- 172.21.15.120:0/3857423649 wait complete. 2024-09-19T09:14:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.888+0000 7f97ca2b8740 1 -- start start 2024-09-19T09:14:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.888+0000 7f97ca2b8740 1 Processor -- start 2024-09-19T09:14:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.888+0000 7f97ca2b8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626853495d0 0x56268535c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.888+0000 7f97ca2b8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56268534ab40 0x56268534d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.888+0000 7f97ca2b8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562685354370 0x56268534df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.888+0000 7f97ca2b8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f97c01b0b60 con 0x56268534ab40 2024-09-19T09:14:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.888+0000 7f97ca2b8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f97c01b0490 con 0x5626853495d0 2024-09-19T09:14:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.888+0000 7f97ca2b8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f97c01ad6b0 con 0x562685354370 2024-09-19T09:14:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.888+0000 7f97ca2b8740 10 client.0 _ll_register_callbacks cb 0x7f97b00d3410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.888+0000 7f97c547a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56268534ab40 0x56268534d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.888+0000 7f97c547a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56268534ab40 0x56268534d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59492/0 (socket says 172.21.15.120:59492) 2024-09-19T09:14:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.888+0000 7f97c5c7b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626853495d0 0x56268535c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.888+0000 7f97c547a640 1 -- 172.21.15.120:0/4034197924 learned_addr learned my addr 172.21.15.120:0/4034197924 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.888+0000 7f97c647c640 1 --2- 172.21.15.120:0/4034197924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562685354370 0x56268534df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.889+0000 7f97c547a640 1 -- 172.21.15.120:0/4034197924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562685354370 msgr2=0x56268534df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.889+0000 7f97c547a640 1 --2- 172.21.15.120:0/4034197924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562685354370 0x56268534df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.889+0000 7f97c547a640 1 -- 172.21.15.120:0/4034197924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626853495d0 msgr2=0x56268535c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.889+0000 7f97c547a640 1 --2- 172.21.15.120:0/4034197924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626853495d0 0x56268535c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.889+0000 7f97c547a640 1 -- 172.21.15.120:0/4034197924 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56268534e4f0 con 0x56268534ab40 2024-09-19T09:14:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.889+0000 7f97c5c7b640 1 --2- 172.21.15.120:0/4034197924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626853495d0 0x56268535c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.889+0000 7f97c647c640 1 --2- 172.21.15.120:0/4034197924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562685354370 0x56268534df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.889+0000 7f97c547a640 1 --2- 172.21.15.120:0/4034197924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56268534ab40 0x56268534d9f0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f97b00a4a00 tx=0x7f97b00d3210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6c3f1487931cc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.889+0000 7f97a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:14:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.889+0000 7f97a97fa640 1 -- 172.21.15.120:0/4034197924 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b009beb0 con 0x56268534ab40 2024-09-19T09:14:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.889+0000 7f97a97fa640 1 -- 172.21.15.120:0/4034197924 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f97b00bc0b0 con 0x56268534ab40 2024-09-19T09:14:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.889+0000 7f97ca2b8740 1 -- 172.21.15.120:0/4034197924 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97b8003c30 con 0x56268534ab40 2024-09-19T09:14:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.889+0000 7f97a97fa640 1 -- 172.21.15.120:0/4034197924 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f97b009c370 con 0x56268534ab40 2024-09-19T09:14:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.889+0000 7f97ca2b8740 10 client.19467 fetch_fsmap learned FSMap version 32 2024-09-19T09:14:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.889+0000 7f97ca2b8740 1 -- 172.21.15.120:0/4034197924 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5626853ce9f0 con 0x56268534ab40 2024-09-19T09:14:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.889+0000 7f97a97fa640 1 -- 172.21.15.120:0/4034197924 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97b8003c30 con 0x56268534ab40 2024-09-19T09:14:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.889+0000 7f97a97fa640 1 -- 172.21.15.120:0/4034197924 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f97b00b5140 con 0x56268534ab40 2024-09-19T09:14:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.889+0000 7f97ca2b8740 10 client.19467 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:14:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.889+0000 7f97ca2b8740 10 client.19467 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:14:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.890+0000 7f97ca2b8740 20 client.19467 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.890+0000 7f97ca2b8740 1 --2- 172.21.15.120:0/4034197924 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5626853cf140 0x5626853d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.890+0000 7f97ca2b8740 4 client.19467 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:14:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.890+0000 7f97ca2b8740 1 -- 172.21.15.120:0/4034197924 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5626852b6d70 con 0x5626853cf140 2024-09-19T09:14:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.890+0000 7f97c5c7b640 1 --2- 172.21.15.120:0/4034197924 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5626853cf140 0x5626853d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.890+0000 7f97c5c7b640 1 --2- 172.21.15.120:0/4034197924 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5626853cf140 0x5626853d1560 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=944234edbc3b7a6b server_cookie=7a27fd86c64d2392 in_seq=0 out_seq=0 2024-09-19T09:14:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.890+0000 7f97a97fa640 10 client.19467 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:14:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.890+0000 7f97a97fa640 1 -- 172.21.15.120:0/4034197924 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5626852b6d70 con 0x5626853cf140 2024-09-19T09:14:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.890+0000 7f97a97fa640 10 client.19467 handle_command_reply: tid=0 2024-09-19T09:14:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.890+0000 7f97ca2b8740 1 client.19467 shutdown 2024-09-19T09:14:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.890+0000 7f97ca2b8740 1 -- 172.21.15.120:0/4034197924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56268534ab40 msgr2=0x56268534d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.890+0000 7f97ca2b8740 1 --2- 172.21.15.120:0/4034197924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56268534ab40 0x56268534d9f0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f97b00a4a00 tx=0x7f97b00d3210 comp rx=0 tx=0).stop 2024-09-19T09:14:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.891+0000 7f97ca2b8740 1 -- 172.21.15.120:0/4034197924 shutdown_connections 2024-09-19T09:14:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.891+0000 7f97ca2b8740 1 --2- 172.21.15.120:0/4034197924 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5626853cf140 0x5626853d1560 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.891+0000 7f97ca2b8740 1 --2- 172.21.15.120:0/4034197924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562685354370 0x56268534df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.891+0000 7f97ca2b8740 1 --2- 172.21.15.120:0/4034197924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56268534ab40 0x56268534d9f0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.891+0000 7f97a97fa640 0 client.19467 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:14:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.891+0000 7f97ca2b8740 1 --2- 172.21.15.120:0/4034197924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626853495d0 0x56268535c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.891+0000 7f97ca2b8740 1 -- 172.21.15.120:0/4034197924 >> 172.21.15.120:0/4034197924 conn(0x562685312310 msgr2=0x562685345700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.891+0000 7f97ca2b8740 1 -- 172.21.15.120:0/4034197924 shutdown_connections 2024-09-19T09:14:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.891+0000 7f97ca2b8740 1 -- 172.21.15.120:0/4034197924 wait complete. 2024-09-19T09:14:09.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.891+0000 7f97ca2b8740 20 client.19467 trim_cache size 0 max 16384 2024-09-19T09:14:09.907 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:14:09.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.891+0000 7f97c7edf640 1 -- 172.21.15.120:0/3619888645 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f97ac0a3240 msgr2=0x7f97ac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:09.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.891+0000 7f97c7edf640 1 --2- 172.21.15.120:0/3619888645 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f97ac0a3240 0x7f97ac0a5700 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f97b8009ca0 tx=0x7f97b8005c50 comp rx=0 tx=0).stop 2024-09-19T09:14:09.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.891+0000 7f97c7edf640 1 -- 172.21.15.120:0/3619888645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97c0102570 msgr2=0x7f97c019abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:09.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.891+0000 7f97c7edf640 1 --2- 172.21.15.120:0/3619888645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97c0102570 0x7f97c019abf0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f97b000e9f0 tx=0x7f97b000eeb0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.891+0000 7f97c7edf640 1 -- 172.21.15.120:0/3619888645 shutdown_connections 2024-09-19T09:14:09.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.891+0000 7f97c7edf640 1 --2- 172.21.15.120:0/3619888645 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f97ac0a3240 0x7f97ac0a5700 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.891+0000 7f97c7edf640 1 --2- 172.21.15.120:0/3619888645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97c0102ef0 0x7f97c019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.891+0000 7f97c7edf640 1 --2- 172.21.15.120:0/3619888645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97c0102570 0x7f97c019abf0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.891+0000 7f97c7edf640 1 --2- 172.21.15.120:0/3619888645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97c0100e20 0x7f97c019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:09.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.892+0000 7f97c7edf640 1 -- 172.21.15.120:0/3619888645 >> 172.21.15.120:0/3619888645 conn(0x7f97c00fc7d0 msgr2=0x7f97c0069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:09.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.892+0000 7f97c7edf640 1 -- 172.21.15.120:0/3619888645 shutdown_connections 2024-09-19T09:14:09.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:09.892+0000 7f97c7edf640 1 -- 172.21.15.120:0/3619888645 wait complete. 2024-09-19T09:14:10.073 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1056238 9.28 MB/sec execute 485 sec latency 29.999 ms 2024-09-19T09:14:10.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:09 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:14:10.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:09 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:14:10.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:09 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:14:10.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:09 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:14:10.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:09 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:14:10.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:09 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:14:10.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:09 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:14:10.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:09 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:14:10.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:09 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:14:10.719 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:14:10.972 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:14:11.073 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1059259 9.29 MB/sec execute 486 sec latency 10.576 ms 2024-09-19T09:14:11.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:10 smithi120 ceph-mon[31150]: pgmap v682: 97 pgs: 97 active+clean; 40 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.7 MiB/s wr, 356 op/s 2024-09-19T09:14:11.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:10 smithi196 ceph-mon[35150]: pgmap v682: 97 pgs: 97 active+clean; 40 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.7 MiB/s wr, 356 op/s 2024-09-19T09:14:11.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:10 smithi153 ceph-mon[35071]: pgmap v682: 97 pgs: 97 active+clean; 40 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.7 MiB/s wr, 356 op/s 2024-09-19T09:14:11.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.498+0000 7f102c66e640 1 -- 172.21.15.120:0/2975375951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1024106f10 msgr2=0x7f102410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:11.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.498+0000 7f102c66e640 1 --2- 172.21.15.120:0/2975375951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1024106f10 0x7f102410ea90 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f101c009ca0 tx=0x7f101c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.498+0000 7f102c66e640 1 -- 172.21.15.120:0/2975375951 shutdown_connections 2024-09-19T09:14:11.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.498+0000 7f102c66e640 1 --2- 172.21.15.120:0/2975375951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1024106f10 0x7f102410ea90 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.498+0000 7f102c66e640 1 --2- 172.21.15.120:0/2975375951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1024102790 0x7f10241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.498+0000 7f102c66e640 1 --2- 172.21.15.120:0/2975375951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1024101de0 0x7f10241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.498+0000 7f102c66e640 1 -- 172.21.15.120:0/2975375951 >> 172.21.15.120:0/2975375951 conn(0x7f10240fd8c0 msgr2=0x7f10240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:11.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.498+0000 7f102c66e640 1 -- 172.21.15.120:0/2975375951 shutdown_connections 2024-09-19T09:14:11.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.498+0000 7f102c66e640 1 -- 172.21.15.120:0/2975375951 wait complete. 2024-09-19T09:14:11.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.499+0000 7f102c66e640 1 Processor -- start 2024-09-19T09:14:11.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.499+0000 7f102c66e640 1 -- start start 2024-09-19T09:14:11.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.499+0000 7f102c66e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1024101de0 0x7f10241983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:11.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.499+0000 7f102c66e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1024102790 0x7f1024198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:11.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.499+0000 7f102c66e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1024106f10 0x7f102419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:11.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.499+0000 7f102c66e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1024111f70 con 0x7f1024102790 2024-09-19T09:14:11.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.499+0000 7f102c66e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1024111df0 con 0x7f1024101de0 2024-09-19T09:14:11.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.499+0000 7f102c66e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10241120f0 con 0x7f1024106f10 2024-09-19T09:14:11.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.499+0000 7f1029c09640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1024102790 0x7f1024198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:11.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.499+0000 7f1029c09640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1024102790 0x7f1024198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47636/0 (socket says 172.21.15.120:47636) 2024-09-19T09:14:11.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.499+0000 7f102a40a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1024101de0 0x7f10241983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:11.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.499+0000 7f1029c09640 1 -- 172.21.15.120:0/3038126464 learned_addr learned my addr 172.21.15.120:0/3038126464 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:11.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.499+0000 7f102ac0b640 1 --2- 172.21.15.120:0/3038126464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1024106f10 0x7f102419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:11.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.500+0000 7f1029c09640 1 -- 172.21.15.120:0/3038126464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1024106f10 msgr2=0x7f102419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:11.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.500+0000 7f1029c09640 1 --2- 172.21.15.120:0/3038126464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1024106f10 0x7f102419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.500+0000 7f1029c09640 1 -- 172.21.15.120:0/3038126464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1024101de0 msgr2=0x7f10241983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:11.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.500+0000 7f1029c09640 1 --2- 172.21.15.120:0/3038126464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1024101de0 0x7f10241983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.500+0000 7f1029c09640 1 -- 172.21.15.120:0/3038126464 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f102419d3a0 con 0x7f1024102790 2024-09-19T09:14:11.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.500+0000 7f102a40a640 1 --2- 172.21.15.120:0/3038126464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1024101de0 0x7f10241983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:11.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.500+0000 7f1029c09640 1 --2- 172.21.15.120:0/3038126464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1024102790 0x7f1024198930 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f101400e9f0 tx=0x7f101400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:11.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.500+0000 7f101b7fe640 1 -- 172.21.15.120:0/3038126464 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101400cd80 con 0x7f1024102790 2024-09-19T09:14:11.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.500+0000 7f101b7fe640 1 -- 172.21.15.120:0/3038126464 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1014005500 con 0x7f1024102790 2024-09-19T09:14:11.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.500+0000 7f102c66e640 1 -- 172.21.15.120:0/3038126464 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f102419d690 con 0x7f1024102790 2024-09-19T09:14:11.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.500+0000 7f101b7fe640 1 -- 172.21.15.120:0/3038126464 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1014010690 con 0x7f1024102790 2024-09-19T09:14:11.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.500+0000 7f102c66e640 1 -- 172.21.15.120:0/3038126464 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1024074420 con 0x7f1024102790 2024-09-19T09:14:11.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.502+0000 7f101b7fe640 1 -- 172.21.15.120:0/3038126464 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f10140046c0 con 0x7f1024102790 2024-09-19T09:14:11.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.503+0000 7f102ea47740 1 Processor -- start 2024-09-19T09:14:11.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.503+0000 7f102ea47740 1 -- start start 2024-09-19T09:14:11.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.503+0000 7f101b7fe640 1 --2- 172.21.15.120:0/3038126464 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f10100a76d0 0x7f10100a9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:11.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.503+0000 7f102ac0b640 1 --2- 172.21.15.120:0/3038126464 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f10100a76d0 0x7f10100a9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:11.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.503+0000 7f102ea47740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572162e4b40 0x5572162e3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:11.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.503+0000 7f102ea47740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572162e35d0 0x5572162e3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:11.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.503+0000 7f101b7fe640 1 -- 172.21.15.120:0/3038126464 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f10140cfc00 con 0x7f1024102790 2024-09-19T09:14:11.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.503+0000 7f102ea47740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572162ee370 0x5572162f0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:11.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.503+0000 7f102ea47740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5572162492f0 con 0x5572162ee370 2024-09-19T09:14:11.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.503+0000 7f102ea47740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5572161f0cf0 con 0x5572162e4b40 2024-09-19T09:14:11.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.503+0000 7f102ea47740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5572162452b0 con 0x5572162e35d0 2024-09-19T09:14:11.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.504+0000 7f1029c09640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572162e35d0 0x5572162e3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:11.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.504+0000 7f102a40a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572162e4b40 0x5572162e3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:11.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.504+0000 7f102ac0b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572162ee370 0x5572162f0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:11.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.504+0000 7f1029c09640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572162e35d0 0x5572162e3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36384/0 (socket says 172.21.15.120:36384) 2024-09-19T09:14:11.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.504+0000 7f102a40a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572162e4b40 0x5572162e3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52022/0 (socket says 172.21.15.120:52022) 2024-09-19T09:14:11.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.504+0000 7f1029c09640 1 -- 172.21.15.120:0/3234081825 learned_addr learned my addr 172.21.15.120:0/3234081825 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:11.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.504+0000 7f1029c09640 1 -- 172.21.15.120:0/3234081825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572162e4b40 msgr2=0x5572162e3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:11.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.504+0000 7f1029c09640 1 --2- 172.21.15.120:0/3234081825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572162e4b40 0x5572162e3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.504+0000 7f1029c09640 1 -- 172.21.15.120:0/3234081825 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572162ee370 msgr2=0x5572162f0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:11.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.504+0000 7f1029c09640 1 --2- 172.21.15.120:0/3234081825 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572162ee370 0x5572162f0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.504+0000 7f1029c09640 1 -- 172.21.15.120:0/3234081825 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572161d53a0 con 0x5572162e35d0 2024-09-19T09:14:11.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.504+0000 7f102a40a640 1 --2- 172.21.15.120:0/3234081825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572162e4b40 0x5572162e3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:11.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.504+0000 7f102ac0b640 1 --2- 172.21.15.120:0/3038126464 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f10100a76d0 0x7f10100a9b90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f101c004af0 tx=0x7f101c031c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:11.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.504+0000 7f1029c09640 1 --2- 172.21.15.120:0/3234081825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572162e35d0 0x5572162e3a10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1014098bf0 tx=0x7f101409be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e7ca7a9ba0103 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:11.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.504+0000 7f1018ff9640 1 -- 172.21.15.120:0/3234081825 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10140b5070 con 0x5572162e35d0 2024-09-19T09:14:11.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.504+0000 7f1018ff9640 1 -- 172.21.15.120:0/3234081825 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10140b5210 con 0x5572162e35d0 2024-09-19T09:14:11.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.504+0000 7f1018ff9640 1 -- 172.21.15.120:0/3234081825 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10140bae60 con 0x5572162e35d0 2024-09-19T09:14:11.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.505+0000 7f102ea47740 1 -- 172.21.15.120:0/3234081825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572162e35d0 msgr2=0x5572162e3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:11.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.505+0000 7f102ea47740 1 --2- 172.21.15.120:0/3234081825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572162e35d0 0x5572162e3a10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1014098bf0 tx=0x7f101409be20 comp rx=0 tx=0).stop 2024-09-19T09:14:11.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.505+0000 7f102ea47740 1 -- 172.21.15.120:0/3234081825 shutdown_connections 2024-09-19T09:14:11.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.505+0000 7f102ea47740 1 --2- 172.21.15.120:0/3234081825 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572162ee370 0x5572162f0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.505+0000 7f102ea47740 1 --2- 172.21.15.120:0/3234081825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572162e35d0 0x5572162e3a10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.505+0000 7f102ea47740 1 --2- 172.21.15.120:0/3234081825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572162e4b40 0x5572162e3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.505+0000 7f102ea47740 1 -- 172.21.15.120:0/3234081825 >> 172.21.15.120:0/3234081825 conn(0x5572162ac310 msgr2=0x5572162dfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:11.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.505+0000 7f102ea47740 1 -- 172.21.15.120:0/3234081825 shutdown_connections 2024-09-19T09:14:11.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.505+0000 7f102ea47740 1 -- 172.21.15.120:0/3234081825 wait complete. 2024-09-19T09:14:11.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.505+0000 7f102ea47740 1 -- start start 2024-09-19T09:14:11.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.505+0000 7f102ea47740 1 Processor -- start 2024-09-19T09:14:11.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.506+0000 7f102ea47740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572162e35d0 0x5572162ede00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:11.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.506+0000 7f102ea47740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572162e4b40 0x5572162eb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:11.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.506+0000 7f102ea47740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572162ee370 0x5572162eb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:11.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.506+0000 7f102ea47740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5572162f1070 con 0x5572162e4b40 2024-09-19T09:14:11.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.506+0000 7f102ea47740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557216204060 con 0x5572162e35d0 2024-09-19T09:14:11.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.506+0000 7f102ea47740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5572162f11f0 con 0x5572162ee370 2024-09-19T09:14:11.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.506+0000 7f102ea47740 10 client.0 _ll_register_callbacks cb 0x557215f578c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:11.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.506+0000 7f1029c09640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572162e4b40 0x5572162eb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:11.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.506+0000 7f1029c09640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572162e4b40 0x5572162eb0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47666/0 (socket says 172.21.15.120:47666) 2024-09-19T09:14:11.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.506+0000 7f102a40a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572162e35d0 0x5572162ede00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:11.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.506+0000 7f1029c09640 1 -- 172.21.15.120:0/252136277 learned_addr learned my addr 172.21.15.120:0/252136277 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:11.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.506+0000 7f102ac0b640 1 --2- 172.21.15.120:0/252136277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572162ee370 0x5572162eb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:11.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.506+0000 7f1029c09640 1 -- 172.21.15.120:0/252136277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572162ee370 msgr2=0x5572162eb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:11.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.506+0000 7f1029c09640 1 --2- 172.21.15.120:0/252136277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572162ee370 0x5572162eb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.506+0000 7f1029c09640 1 -- 172.21.15.120:0/252136277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572162e35d0 msgr2=0x5572162ede00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:11.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.506+0000 7f1029c09640 1 --2- 172.21.15.120:0/252136277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572162e35d0 0x5572162ede00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.506+0000 7f1029c09640 1 -- 172.21.15.120:0/252136277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557215fba3c0 con 0x5572162e4b40 2024-09-19T09:14:11.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.506+0000 7f102a40a640 1 --2- 172.21.15.120:0/252136277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572162e35d0 0x5572162ede00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:11.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.506+0000 7f1029c09640 1 --2- 172.21.15.120:0/252136277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572162e4b40 0x5572162eb0b0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f101409bf20 tx=0x7f1014098890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f880a467a9548f1a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:11.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.506+0000 7f100dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:14:11.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.507+0000 7f100dffb640 1 -- 172.21.15.120:0/252136277 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10140b5040 con 0x5572162e4b40 2024-09-19T09:14:11.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.507+0000 7f100dffb640 1 -- 172.21.15.120:0/252136277 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f10140a4a00 con 0x5572162e4b40 2024-09-19T09:14:11.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.507+0000 7f102ea47740 1 -- 172.21.15.120:0/252136277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10140b5210 con 0x5572162e4b40 2024-09-19T09:14:11.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.507+0000 7f100dffb640 1 -- 172.21.15.120:0/252136277 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f10140bae60 con 0x5572162e4b40 2024-09-19T09:14:11.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.507+0000 7f102ea47740 10 client.19479 fetch_fsmap learned FSMap version 32 2024-09-19T09:14:11.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.507+0000 7f102ea47740 1 -- 172.21.15.120:0/252136277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5572162fcbc0 con 0x5572162e4b40 2024-09-19T09:14:11.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.507+0000 7f100dffb640 1 -- 172.21.15.120:0/252136277 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10140b5210 con 0x5572162e4b40 2024-09-19T09:14:11.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.507+0000 7f100dffb640 1 -- 172.21.15.120:0/252136277 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f10140cdbc0 con 0x5572162e4b40 2024-09-19T09:14:11.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.507+0000 7f102ea47740 10 client.19479 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:14:11.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.507+0000 7f102ea47740 10 client.19479 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:14:11.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.507+0000 7f102ea47740 20 client.19479 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:11.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.507+0000 7f102ea47740 1 --2- 172.21.15.120:0/252136277 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5572162fdf10 0x557216300300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:11.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.507+0000 7f102ea47740 4 client.19479 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:14:11.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.507+0000 7f102ea47740 1 -- 172.21.15.120:0/252136277 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557216256860 con 0x5572162fdf10 2024-09-19T09:14:11.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.507+0000 7f102a40a640 1 --2- 172.21.15.120:0/252136277 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5572162fdf10 0x557216300300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:11.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.508+0000 7f100dffb640 10 client.19479 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:14:11.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.508+0000 7f102a40a640 1 --2- 172.21.15.120:0/252136277 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5572162fdf10 0x557216300300 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cfcf58332c14fd7 server_cookie=d382916574286149 in_seq=0 out_seq=0 2024-09-19T09:14:11.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.509+0000 7f100dffb640 1 -- 172.21.15.120:0/252136277 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557216256860 con 0x5572162fdf10 2024-09-19T09:14:11.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.509+0000 7f100dffb640 10 client.19479 handle_command_reply: tid=0 2024-09-19T09:14:11.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.509+0000 7f102ea47740 1 client.19479 shutdown 2024-09-19T09:14:11.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.509+0000 7f102ea47740 1 -- 172.21.15.120:0/252136277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572162e4b40 msgr2=0x5572162eb0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:11.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.509+0000 7f102ea47740 1 --2- 172.21.15.120:0/252136277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572162e4b40 0x5572162eb0b0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f101409bf20 tx=0x7f1014098890 comp rx=0 tx=0).stop 2024-09-19T09:14:11.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.509+0000 7f102ea47740 1 -- 172.21.15.120:0/252136277 shutdown_connections 2024-09-19T09:14:11.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.509+0000 7f102ea47740 1 --2- 172.21.15.120:0/252136277 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5572162fdf10 0x557216300300 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.509+0000 7f102ea47740 1 --2- 172.21.15.120:0/252136277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572162ee370 0x5572162eb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.509+0000 7f102ea47740 1 --2- 172.21.15.120:0/252136277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572162e4b40 0x5572162eb0b0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.509+0000 7f102ea47740 1 --2- 172.21.15.120:0/252136277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572162e35d0 0x5572162ede00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.509+0000 7f102ea47740 1 -- 172.21.15.120:0/252136277 >> 172.21.15.120:0/252136277 conn(0x5572162ac310 msgr2=0x5572162e5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:11.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.509+0000 7f100dffb640 0 client.19479 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:14:11.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.509+0000 7f102ea47740 1 -- 172.21.15.120:0/252136277 shutdown_connections 2024-09-19T09:14:11.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.509+0000 7f102ea47740 1 -- 172.21.15.120:0/252136277 wait complete. 2024-09-19T09:14:11.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.509+0000 7f102ea47740 20 client.19479 trim_cache size 0 max 16384 2024-09-19T09:14:11.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.523+0000 7f102ea47740 1 Processor -- start 2024-09-19T09:14:11.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.523+0000 7f102ea47740 1 -- start start 2024-09-19T09:14:11.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.523+0000 7f102ea47740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572162e35d0 0x5572162ea090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:11.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.523+0000 7f102ea47740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572162e4b40 0x5572162e60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:11.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.523+0000 7f102ea47740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572162ee370 0x5572162e6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:11.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.523+0000 7f102ea47740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f10241ab490 con 0x5572162e4b40 2024-09-19T09:14:11.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.523+0000 7f102ea47740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f10241acfd0 con 0x5572162e35d0 2024-09-19T09:14:11.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.523+0000 7f102ea47740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10241abb60 con 0x5572162ee370 2024-09-19T09:14:11.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.523+0000 7f1029c09640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572162e4b40 0x5572162e60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:11.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.523+0000 7f1029c09640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572162e4b40 0x5572162e60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47672/0 (socket says 172.21.15.120:47672) 2024-09-19T09:14:11.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.523+0000 7f1029c09640 1 -- 172.21.15.120:0/904105845 learned_addr learned my addr 172.21.15.120:0/904105845 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:11.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.523+0000 7f102a40a640 1 --2- 172.21.15.120:0/904105845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572162e35d0 0x5572162ea090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:11.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.523+0000 7f102ac0b640 1 --2- 172.21.15.120:0/904105845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572162ee370 0x5572162e6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:11.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.523+0000 7f1029c09640 1 -- 172.21.15.120:0/904105845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572162ee370 msgr2=0x5572162e6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:11.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.523+0000 7f1029c09640 1 --2- 172.21.15.120:0/904105845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572162ee370 0x5572162e6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.523+0000 7f1029c09640 1 -- 172.21.15.120:0/904105845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572162e35d0 msgr2=0x5572162ea090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:11.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.523+0000 7f1029c09640 1 --2- 172.21.15.120:0/904105845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572162e35d0 0x5572162ea090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.523+0000 7f1029c09640 1 -- 172.21.15.120:0/904105845 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572162e0dd0 con 0x5572162e4b40 2024-09-19T09:14:11.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.524+0000 7f1029c09640 1 --2- 172.21.15.120:0/904105845 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572162e4b40 0x5572162e60f0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f10140049d0 tx=0x7f101409a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d11962766ee0a93 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:11.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.524+0000 7f100ffff640 1 -- 172.21.15.120:0/904105845 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10140a05d0 con 0x5572162e4b40 2024-09-19T09:14:11.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.524+0000 7f100ffff640 1 -- 172.21.15.120:0/904105845 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10140a4e90 con 0x5572162e4b40 2024-09-19T09:14:11.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.524+0000 7f100ffff640 1 -- 172.21.15.120:0/904105845 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10140b5040 con 0x5572162e4b40 2024-09-19T09:14:11.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.524+0000 7f102ea47740 1 -- 172.21.15.120:0/904105845 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572162e4b40 msgr2=0x5572162e60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:11.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.524+0000 7f102ea47740 1 --2- 172.21.15.120:0/904105845 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572162e4b40 0x5572162e60f0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f10140049d0 tx=0x7f101409a730 comp rx=0 tx=0).stop 2024-09-19T09:14:11.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.525+0000 7f102ea47740 1 -- 172.21.15.120:0/904105845 shutdown_connections 2024-09-19T09:14:11.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.525+0000 7f102ea47740 1 --2- 172.21.15.120:0/904105845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572162ee370 0x5572162e6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.525+0000 7f102ea47740 1 --2- 172.21.15.120:0/904105845 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572162e4b40 0x5572162e60f0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.525+0000 7f102ea47740 1 --2- 172.21.15.120:0/904105845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572162e35d0 0x5572162ea090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.525+0000 7f102ea47740 1 -- 172.21.15.120:0/904105845 >> 172.21.15.120:0/904105845 conn(0x5572162ac310 msgr2=0x5572162e9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:11.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.525+0000 7f102ea47740 1 -- 172.21.15.120:0/904105845 shutdown_connections 2024-09-19T09:14:11.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.525+0000 7f102ea47740 1 -- 172.21.15.120:0/904105845 wait complete. 2024-09-19T09:14:11.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.525+0000 7f102ea47740 1 -- start start 2024-09-19T09:14:11.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.525+0000 7f102ea47740 1 Processor -- start 2024-09-19T09:14:11.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.525+0000 7f102ea47740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572162e35d0 0x5572162fff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:11.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.525+0000 7f102ea47740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572162e4b40 0x5572163004c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:11.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.525+0000 7f102ea47740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572162ee370 0x557216300a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:11.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.525+0000 7f102ea47740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f10241ac900 con 0x5572162ee370 2024-09-19T09:14:11.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.525+0000 7f102ea47740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f10241ac230 con 0x5572162e35d0 2024-09-19T09:14:11.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.525+0000 7f102ea47740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10241a9450 con 0x5572162e4b40 2024-09-19T09:14:11.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.525+0000 7f102ea47740 10 client.0 _ll_register_callbacks cb 0x7f10140b3070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:11.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.525+0000 7f102a40a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572162e35d0 0x5572162fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:11.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.525+0000 7f102ac0b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572162ee370 0x557216300a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:11.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.525+0000 7f1029c09640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572162e4b40 0x5572163004c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:11.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.525+0000 7f1029c09640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572162e4b40 0x5572163004c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36414/0 (socket says 172.21.15.120:36414) 2024-09-19T09:14:11.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.525+0000 7f102ac0b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572162ee370 0x557216300a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47678/0 (socket says 172.21.15.120:47678) 2024-09-19T09:14:11.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.525+0000 7f102ac0b640 1 -- 172.21.15.120:0/3240960570 learned_addr learned my addr 172.21.15.120:0/3240960570 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:11.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.525+0000 7f102a40a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572162e35d0 0x5572162fff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52048/0 (socket says 172.21.15.120:52048) 2024-09-19T09:14:11.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.526+0000 7f102a40a640 1 -- 172.21.15.120:0/3240960570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572162e4b40 msgr2=0x5572163004c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:11.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.526+0000 7f102a40a640 1 --2- 172.21.15.120:0/3240960570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572162e4b40 0x5572163004c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.526+0000 7f102a40a640 1 -- 172.21.15.120:0/3240960570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572162ee370 msgr2=0x557216300a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:11.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.526+0000 7f102a40a640 1 --2- 172.21.15.120:0/3240960570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572162ee370 0x557216300a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.526+0000 7f102a40a640 1 -- 172.21.15.120:0/3240960570 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557216366ae0 con 0x5572162e35d0 2024-09-19T09:14:11.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.526+0000 7f102ac0b640 1 --2- 172.21.15.120:0/3240960570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572162ee370 0x557216300a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:11.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.526+0000 7f102a40a640 1 --2- 172.21.15.120:0/3240960570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572162e35d0 0x5572162fff80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1020004700 tx=0x7f1020012f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71bffb031ef2d4ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:11.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.526+0000 7f100dffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:14:11.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.526+0000 7f100dffb640 1 -- 172.21.15.120:0/3240960570 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1020020d10 con 0x5572162e35d0 2024-09-19T09:14:11.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.526+0000 7f102ea47740 1 -- 172.21.15.120:0/3240960570 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10140a4e90 con 0x5572162e35d0 2024-09-19T09:14:11.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.526+0000 7f100dffb640 1 -- 172.21.15.120:0/3240960570 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f1020004050 con 0x5572162e35d0 2024-09-19T09:14:11.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.526+0000 7f100dffb640 1 -- 172.21.15.120:0/3240960570 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1020004250 con 0x5572162e35d0 2024-09-19T09:14:11.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.526+0000 7f102ea47740 10 client.28538 fetch_fsmap learned FSMap version 32 2024-09-19T09:14:11.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.526+0000 7f102ea47740 1 -- 172.21.15.120:0/3240960570 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557216367e40 con 0x5572162e35d0 2024-09-19T09:14:11.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.527+0000 7f100dffb640 1 -- 172.21.15.120:0/3240960570 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10140a4e90 con 0x5572162e35d0 2024-09-19T09:14:11.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.527+0000 7f100dffb640 1 -- 172.21.15.120:0/3240960570 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f10200104c0 con 0x5572162e35d0 2024-09-19T09:14:11.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.527+0000 7f102ea47740 10 client.28538 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:14:11.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.527+0000 7f102ea47740 10 client.28538 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:14:11.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.527+0000 7f102ea47740 20 client.28538 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:11.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.527+0000 7f102ea47740 1 --2- 172.21.15.120:0/3240960570 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557216368540 0x55721636a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:11.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.527+0000 7f102ea47740 4 client.28538 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:14:11.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.527+0000 7f102ea47740 1 -- 172.21.15.120:0/3240960570 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557216250d70 con 0x557216368540 2024-09-19T09:14:11.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.527+0000 7f1029c09640 1 --2- 172.21.15.120:0/3240960570 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557216368540 0x55721636a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:11.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.527+0000 7f1029c09640 1 --2- 172.21.15.120:0/3240960570 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557216368540 0x55721636a9a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4b0431b954a99fab server_cookie=653c0a12fc859ae in_seq=0 out_seq=0 2024-09-19T09:14:11.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.527+0000 7f100dffb640 10 client.28538 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:14:11.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.527+0000 7f100dffb640 1 -- 172.21.15.120:0/3240960570 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557216250d70 con 0x557216368540 2024-09-19T09:14:11.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.527+0000 7f100dffb640 10 client.28538 handle_command_reply: tid=0 2024-09-19T09:14:11.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.528+0000 7f102ea47740 1 client.28538 shutdown 2024-09-19T09:14:11.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.528+0000 7f102ea47740 1 -- 172.21.15.120:0/3240960570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572162e35d0 msgr2=0x5572162fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:11.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.528+0000 7f102ea47740 1 --2- 172.21.15.120:0/3240960570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572162e35d0 0x5572162fff80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1020004700 tx=0x7f1020012f90 comp rx=0 tx=0).stop 2024-09-19T09:14:11.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.528+0000 7f102ea47740 1 -- 172.21.15.120:0/3240960570 shutdown_connections 2024-09-19T09:14:11.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.528+0000 7f102ea47740 1 --2- 172.21.15.120:0/3240960570 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557216368540 0x55721636a9a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.528+0000 7f102ea47740 1 --2- 172.21.15.120:0/3240960570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572162ee370 0x557216300a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.528+0000 7f102ea47740 1 --2- 172.21.15.120:0/3240960570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572162e4b40 0x5572163004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.528+0000 7f100dffb640 0 client.28538 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:14:11.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.528+0000 7f102ea47740 1 --2- 172.21.15.120:0/3240960570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572162e35d0 0x5572162fff80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.528+0000 7f102ea47740 1 -- 172.21.15.120:0/3240960570 >> 172.21.15.120:0/3240960570 conn(0x5572162ac310 msgr2=0x5572162ea1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:11.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.528+0000 7f102ea47740 1 -- 172.21.15.120:0/3240960570 shutdown_connections 2024-09-19T09:14:11.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.528+0000 7f102ea47740 1 -- 172.21.15.120:0/3240960570 wait complete. 2024-09-19T09:14:11.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.528+0000 7f102ea47740 20 client.28538 trim_cache size 0 max 16384 2024-09-19T09:14:11.542 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:14:11.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.528+0000 7f102c66e640 1 -- 172.21.15.120:0/3038126464 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f10100a76d0 msgr2=0x7f10100a9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:11.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.528+0000 7f102c66e640 1 --2- 172.21.15.120:0/3038126464 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f10100a76d0 0x7f10100a9b90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f101c004af0 tx=0x7f101c031c90 comp rx=0 tx=0).stop 2024-09-19T09:14:11.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.528+0000 7f102c66e640 1 -- 172.21.15.120:0/3038126464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1024102790 msgr2=0x7f1024198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:11.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.528+0000 7f102c66e640 1 --2- 172.21.15.120:0/3038126464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1024102790 0x7f1024198930 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f101400e9f0 tx=0x7f101400edf0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.529+0000 7f102c66e640 1 -- 172.21.15.120:0/3038126464 shutdown_connections 2024-09-19T09:14:11.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.529+0000 7f102c66e640 1 --2- 172.21.15.120:0/3038126464 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f10100a76d0 0x7f10100a9b90 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.529+0000 7f102c66e640 1 --2- 172.21.15.120:0/3038126464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1024106f10 0x7f102419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.529+0000 7f102c66e640 1 --2- 172.21.15.120:0/3038126464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1024102790 0x7f1024198930 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.529+0000 7f102c66e640 1 --2- 172.21.15.120:0/3038126464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1024101de0 0x7f10241983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:11.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.529+0000 7f102c66e640 1 -- 172.21.15.120:0/3038126464 >> 172.21.15.120:0/3038126464 conn(0x7f10240fd8c0 msgr2=0x7f10240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:11.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.529+0000 7f102c66e640 1 -- 172.21.15.120:0/3038126464 shutdown_connections 2024-09-19T09:14:11.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:11.529+0000 7f102c66e640 1 -- 172.21.15.120:0/3038126464 wait complete. 2024-09-19T09:14:12.073 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1062371 9.31 MB/sec execute 487 sec latency 36.889 ms 2024-09-19T09:14:12.422 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:14:12.691 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:14:13.063 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:12 smithi120 ceph-mon[31150]: pgmap v683: 97 pgs: 97 active+clean; 41 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.8 MiB/s wr, 355 op/s 2024-09-19T09:14:13.073 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1065067 9.32 MB/sec execute 488 sec latency 22.861 ms 2024-09-19T09:14:13.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:12 smithi196 ceph-mon[35150]: pgmap v683: 97 pgs: 97 active+clean; 41 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.8 MiB/s wr, 355 op/s 2024-09-19T09:14:13.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:12 smithi153 ceph-mon[35071]: pgmap v683: 97 pgs: 97 active+clean; 41 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.8 MiB/s wr, 355 op/s 2024-09-19T09:14:13.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.291+0000 7ff3efd59640 1 -- 172.21.15.120:0/2103233271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3e8102570 msgr2=0x7ff3e81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:13.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.291+0000 7ff3efd59640 1 --2- 172.21.15.120:0/2103233271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3e8102570 0x7ff3e81029b0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8009900 tx=0x7ff3d802a480 comp rx=0 tx=0).stop 2024-09-19T09:14:13.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.291+0000 7ff3efd59640 1 -- 172.21.15.120:0/2103233271 shutdown_connections 2024-09-19T09:14:13.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.291+0000 7ff3efd59640 1 --2- 172.21.15.120:0/2103233271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3e8102ef0 0x7ff3e810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.291+0000 7ff3efd59640 1 --2- 172.21.15.120:0/2103233271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3e8102570 0x7ff3e81029b0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.291+0000 7ff3efd59640 1 --2- 172.21.15.120:0/2103233271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3e8100e20 0x7ff3e8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.291+0000 7ff3efd59640 1 -- 172.21.15.120:0/2103233271 >> 172.21.15.120:0/2103233271 conn(0x7ff3e80fc7d0 msgr2=0x7ff3e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:13.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.291+0000 7ff3efd59640 1 -- 172.21.15.120:0/2103233271 shutdown_connections 2024-09-19T09:14:13.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.291+0000 7ff3efd59640 1 -- 172.21.15.120:0/2103233271 wait complete. 2024-09-19T09:14:13.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.292+0000 7ff3efd59640 1 Processor -- start 2024-09-19T09:14:13.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.292+0000 7ff3efd59640 1 -- start start 2024-09-19T09:14:13.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.292+0000 7ff3efd59640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3e8100e20 0x7ff3e81993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:13.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.292+0000 7ff3efd59640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3e8102570 0x7ff3e8199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:13.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.292+0000 7ff3efd59640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3e8102ef0 0x7ff3e81a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:13.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.292+0000 7ff3efd59640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e8110f80 con 0x7ff3e8102ef0 2024-09-19T09:14:13.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.292+0000 7ff3efd59640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e8110e00 con 0x7ff3e8102570 2024-09-19T09:14:13.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.292+0000 7ff3efd59640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e8111100 con 0x7ff3e8100e20 2024-09-19T09:14:13.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.292+0000 7ff3ee2f6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3e8102ef0 0x7ff3e81a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:13.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.292+0000 7ff3edaf5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3e8100e20 0x7ff3e81993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:13.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.292+0000 7ff3ee2f6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3e8102ef0 0x7ff3e81a4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47696/0 (socket says 172.21.15.120:47696) 2024-09-19T09:14:13.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.292+0000 7ff3ed2f4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3e8102570 0x7ff3e8199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:13.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.292+0000 7ff3ee2f6640 1 -- 172.21.15.120:0/2305560123 learned_addr learned my addr 172.21.15.120:0/2305560123 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:13.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.293+0000 7ff3ee2f6640 1 -- 172.21.15.120:0/2305560123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3e8100e20 msgr2=0x7ff3e81993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:13.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.293+0000 7ff3ee2f6640 1 --2- 172.21.15.120:0/2305560123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3e8100e20 0x7ff3e81993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.293+0000 7ff3ee2f6640 1 -- 172.21.15.120:0/2305560123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3e8102570 msgr2=0x7ff3e8199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:13.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.293+0000 7ff3ee2f6640 1 --2- 172.21.15.120:0/2305560123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3e8102570 0x7ff3e8199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.293+0000 7ff3ee2f6640 1 -- 172.21.15.120:0/2305560123 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e81a4990 con 0x7ff3e8102ef0 2024-09-19T09:14:13.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.293+0000 7ff3edaf5640 1 --2- 172.21.15.120:0/2305560123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3e8100e20 0x7ff3e81993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:13.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.293+0000 7ff3ed2f4640 1 --2- 172.21.15.120:0/2305560123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3e8102570 0x7ff3e8199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:13.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.293+0000 7ff3ee2f6640 1 --2- 172.21.15.120:0/2305560123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3e8102ef0 0x7ff3e81a4450 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7ff3e000eb10 tx=0x7ff3e000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:13.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.293+0000 7ff3deffd640 1 -- 172.21.15.120:0/2305560123 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e0004b20 con 0x7ff3e8102ef0 2024-09-19T09:14:13.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.293+0000 7ff3deffd640 1 -- 172.21.15.120:0/2305560123 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3e0004cc0 con 0x7ff3e8102ef0 2024-09-19T09:14:13.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.293+0000 7ff3efd59640 1 -- 172.21.15.120:0/2305560123 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3e81a4b60 con 0x7ff3e8102ef0 2024-09-19T09:14:13.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.293+0000 7ff3deffd640 1 -- 172.21.15.120:0/2305560123 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e0010760 con 0x7ff3e8102ef0 2024-09-19T09:14:13.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.293+0000 7ff3efd59640 1 -- 172.21.15.120:0/2305560123 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3e81a5020 con 0x7ff3e8102ef0 2024-09-19T09:14:13.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.295+0000 7ff3deffd640 1 -- 172.21.15.120:0/2305560123 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7ff3e0005020 con 0x7ff3e8102ef0 2024-09-19T09:14:13.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.296+0000 7ff3f2132740 1 Processor -- start 2024-09-19T09:14:13.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.296+0000 7ff3f2132740 1 -- start start 2024-09-19T09:14:13.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.296+0000 7ff3f2132740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55653d4b3b40 0x55653d4b2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:13.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.296+0000 7ff3f2132740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55653d4b25d0 0x55653d4b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:13.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.296+0000 7ff3f2132740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55653d4bd370 0x55653d4bf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:13.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.296+0000 7ff3f2132740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55653d4182f0 con 0x55653d4bd370 2024-09-19T09:14:13.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.296+0000 7ff3f2132740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55653d3bfcf0 con 0x55653d4b25d0 2024-09-19T09:14:13.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.296+0000 7ff3f2132740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55653d4142b0 con 0x55653d4b3b40 2024-09-19T09:14:13.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.296+0000 7ff3ed2f4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55653d4b3b40 0x55653d4b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:13.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.296+0000 7ff3edaf5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55653d4b25d0 0x55653d4b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:13.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.296+0000 7ff3ed2f4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55653d4b3b40 0x55653d4b2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36454/0 (socket says 172.21.15.120:36454) 2024-09-19T09:14:13.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.296+0000 7ff3ed2f4640 1 -- 172.21.15.120:0/218534994 learned_addr learned my addr 172.21.15.120:0/218534994 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:13.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.296+0000 7ff3ed2f4640 1 --2- 172.21.15.120:0/218534994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55653d4bd370 0x55653d4bf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:13.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.296+0000 7ff3deffd640 1 --2- 172.21.15.120:0/2305560123 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff3d40a3240 0x7ff3d40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:13.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.296+0000 7ff3ee2f6640 1 --2- 172.21.15.120:0/2305560123 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff3d40a3240 0x7ff3d40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:13.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.296+0000 7ff3edaf5640 1 -- 172.21.15.120:0/218534994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55653d4b3b40 msgr2=0x55653d4b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:13.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.296+0000 7ff3edaf5640 1 --2- 172.21.15.120:0/218534994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55653d4b3b40 0x55653d4b2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.296+0000 7ff3edaf5640 1 -- 172.21.15.120:0/218534994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55653d4bd370 msgr2=0x55653d4bf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:13.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.296+0000 7ff3edaf5640 1 --2- 172.21.15.120:0/218534994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55653d4bd370 0x55653d4bf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.296+0000 7ff3edaf5640 1 -- 172.21.15.120:0/218534994 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55653d3a43a0 con 0x55653d4b25d0 2024-09-19T09:14:13.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.296+0000 7ff3ed2f4640 1 --2- 172.21.15.120:0/218534994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55653d4b3b40 0x55653d4b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:13.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.297+0000 7ff3deffd640 1 -- 172.21.15.120:0/2305560123 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ff3e00cfe00 con 0x7ff3e8102ef0 2024-09-19T09:14:13.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.297+0000 7ff3edaf5640 1 --2- 172.21.15.120:0/218534994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55653d4b25d0 0x55653d4b2a10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4009fd0 tx=0x7ff3e40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c89e498efd71036 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:13.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.297+0000 7ff3ee2f6640 1 --2- 172.21.15.120:0/2305560123 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff3d40a3240 0x7ff3d40a5700 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff3e00a4b60 tx=0x7ff3e0098020 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:13.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.297+0000 7ff3d3fff640 1 -- 172.21.15.120:0/218534994 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4004310 con 0x55653d4b25d0 2024-09-19T09:14:13.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.297+0000 7ff3d3fff640 1 -- 172.21.15.120:0/218534994 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3e40044b0 con 0x55653d4b25d0 2024-09-19T09:14:13.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.297+0000 7ff3d3fff640 1 -- 172.21.15.120:0/218534994 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e40035f0 con 0x55653d4b25d0 2024-09-19T09:14:13.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.298+0000 7ff3f2132740 1 -- 172.21.15.120:0/218534994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55653d4b25d0 msgr2=0x55653d4b2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:13.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.298+0000 7ff3f2132740 1 --2- 172.21.15.120:0/218534994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55653d4b25d0 0x55653d4b2a10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4009fd0 tx=0x7ff3e40118d0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.298+0000 7ff3f2132740 1 -- 172.21.15.120:0/218534994 shutdown_connections 2024-09-19T09:14:13.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.298+0000 7ff3f2132740 1 --2- 172.21.15.120:0/218534994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55653d4bd370 0x55653d4bf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.298+0000 7ff3f2132740 1 --2- 172.21.15.120:0/218534994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55653d4b25d0 0x55653d4b2a10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.298+0000 7ff3f2132740 1 --2- 172.21.15.120:0/218534994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55653d4b3b40 0x55653d4b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.298+0000 7ff3f2132740 1 -- 172.21.15.120:0/218534994 >> 172.21.15.120:0/218534994 conn(0x55653d47b310 msgr2=0x55653d4aec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:13.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.298+0000 7ff3f2132740 1 -- 172.21.15.120:0/218534994 shutdown_connections 2024-09-19T09:14:13.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.298+0000 7ff3f2132740 1 -- 172.21.15.120:0/218534994 wait complete. 2024-09-19T09:14:13.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.298+0000 7ff3f2132740 1 -- start start 2024-09-19T09:14:13.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.298+0000 7ff3f2132740 1 Processor -- start 2024-09-19T09:14:13.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.299+0000 7ff3f2132740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55653d4b25d0 0x55653d4bccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:13.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.299+0000 7ff3f2132740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55653d4b3b40 0x55653d4b9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:13.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.299+0000 7ff3f2132740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55653d4bd370 0x55653d4ba4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:13.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.299+0000 7ff3f2132740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55653d4c0070 con 0x55653d4b25d0 2024-09-19T09:14:13.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.299+0000 7ff3f2132740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55653d3d3060 con 0x55653d4b3b40 2024-09-19T09:14:13.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.299+0000 7ff3f2132740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55653d4c01f0 con 0x55653d4bd370 2024-09-19T09:14:13.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.299+0000 7ff3f2132740 10 client.0 _ll_register_callbacks cb 0x55653d1268c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:13.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.299+0000 7ff3edaf5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55653d4b3b40 0x55653d4b9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:13.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.299+0000 7ff3ed2f4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55653d4b25d0 0x55653d4bccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:13.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.299+0000 7ff3ed2f4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55653d4b25d0 0x55653d4bccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47704/0 (socket says 172.21.15.120:47704) 2024-09-19T09:14:13.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.299+0000 7ff3edaf5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55653d4b3b40 0x55653d4b9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52080/0 (socket says 172.21.15.120:52080) 2024-09-19T09:14:13.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.299+0000 7ff3ed2f4640 1 -- 172.21.15.120:0/2139337031 learned_addr learned my addr 172.21.15.120:0/2139337031 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:13.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.299+0000 7ff3ed2f4640 1 --2- 172.21.15.120:0/2139337031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55653d4bd370 0x55653d4ba4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:13.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.299+0000 7ff3ed2f4640 1 -- 172.21.15.120:0/2139337031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55653d4bd370 msgr2=0x55653d4ba4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:13.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.299+0000 7ff3ed2f4640 1 --2- 172.21.15.120:0/2139337031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55653d4bd370 0x55653d4ba4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.299+0000 7ff3ed2f4640 1 -- 172.21.15.120:0/2139337031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55653d4b3b40 msgr2=0x55653d4b9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:13.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.299+0000 7ff3ed2f4640 1 --2- 172.21.15.120:0/2139337031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55653d4b3b40 0x55653d4b9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.299+0000 7ff3ed2f4640 1 -- 172.21.15.120:0/2139337031 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55653d1893c0 con 0x55653d4b25d0 2024-09-19T09:14:13.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.299+0000 7ff3edaf5640 1 --2- 172.21.15.120:0/2139337031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55653d4b3b40 0x55653d4b9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:13.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.299+0000 7ff3ed2f4640 1 --2- 172.21.15.120:0/2139337031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55653d4b25d0 0x55653d4bccc0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7ff3d802a9a0 tx=0x7ff3d8005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e90e7bdfc85222 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:13.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.299+0000 7ff3d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:14:13.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.299+0000 7ff3d17fa640 1 -- 172.21.15.120:0/2139337031 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8035040 con 0x55653d4b25d0 2024-09-19T09:14:13.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.299+0000 7ff3d17fa640 1 -- 172.21.15.120:0/2139337031 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ff3d8005020 con 0x55653d4b25d0 2024-09-19T09:14:13.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.299+0000 7ff3f2132740 1 -- 172.21.15.120:0/2139337031 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3e40044b0 con 0x55653d4b25d0 2024-09-19T09:14:13.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.300+0000 7ff3d17fa640 1 -- 172.21.15.120:0/2139337031 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff3d8005220 con 0x55653d4b25d0 2024-09-19T09:14:13.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.300+0000 7ff3f2132740 10 client.19497 fetch_fsmap learned FSMap version 32 2024-09-19T09:14:13.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.300+0000 7ff3f2132740 1 -- 172.21.15.120:0/2139337031 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55653d4cfe90 con 0x55653d4b25d0 2024-09-19T09:14:13.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.300+0000 7ff3d17fa640 1 -- 172.21.15.120:0/2139337031 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3e40044b0 con 0x55653d4b25d0 2024-09-19T09:14:13.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.300+0000 7ff3d17fa640 1 -- 172.21.15.120:0/2139337031 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff3d8004750 con 0x55653d4b25d0 2024-09-19T09:14:13.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.300+0000 7ff3f2132740 10 client.19497 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:14:13.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.300+0000 7ff3f2132740 10 client.19497 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:14:13.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.300+0000 7ff3f2132740 20 client.19497 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:13.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.300+0000 7ff3f2132740 1 --2- 172.21.15.120:0/2139337031 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55653d4d12a0 0x55653d4d3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:13.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.300+0000 7ff3f2132740 4 client.19497 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:14:13.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.300+0000 7ff3f2132740 1 -- 172.21.15.120:0/2139337031 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55653d425860 con 0x55653d4d12a0 2024-09-19T09:14:13.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.300+0000 7ff3edaf5640 1 --2- 172.21.15.120:0/2139337031 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55653d4d12a0 0x55653d4d3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:13.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.301+0000 7ff3edaf5640 1 --2- 172.21.15.120:0/2139337031 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55653d4d12a0 0x55653d4d3690 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f5c248e1757843e8 server_cookie=ceaa1e024a8b58b3 in_seq=0 out_seq=0 2024-09-19T09:14:13.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.301+0000 7ff3d17fa640 10 client.19497 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:14:13.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.301+0000 7ff3d17fa640 1 -- 172.21.15.120:0/2139337031 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55653d425860 con 0x55653d4d12a0 2024-09-19T09:14:13.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.301+0000 7ff3d17fa640 10 client.19497 handle_command_reply: tid=0 2024-09-19T09:14:13.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.301+0000 7ff3f2132740 1 client.19497 shutdown 2024-09-19T09:14:13.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.301+0000 7ff3f2132740 1 -- 172.21.15.120:0/2139337031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55653d4b25d0 msgr2=0x55653d4bccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:13.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.301+0000 7ff3f2132740 1 --2- 172.21.15.120:0/2139337031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55653d4b25d0 0x55653d4bccc0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7ff3d802a9a0 tx=0x7ff3d8005dd0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.302+0000 7ff3f2132740 1 -- 172.21.15.120:0/2139337031 shutdown_connections 2024-09-19T09:14:13.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.302+0000 7ff3f2132740 1 --2- 172.21.15.120:0/2139337031 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55653d4d12a0 0x55653d4d3690 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.302+0000 7ff3f2132740 1 --2- 172.21.15.120:0/2139337031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55653d4bd370 0x55653d4ba4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.302+0000 7ff3f2132740 1 --2- 172.21.15.120:0/2139337031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55653d4b3b40 0x55653d4b9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.302+0000 7ff3d17fa640 0 client.19497 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:14:13.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.302+0000 7ff3f2132740 1 --2- 172.21.15.120:0/2139337031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55653d4b25d0 0x55653d4bccc0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.302+0000 7ff3f2132740 1 -- 172.21.15.120:0/2139337031 >> 172.21.15.120:0/2139337031 conn(0x55653d47b310 msgr2=0x55653d4b4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:13.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.302+0000 7ff3f2132740 1 -- 172.21.15.120:0/2139337031 shutdown_connections 2024-09-19T09:14:13.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.302+0000 7ff3f2132740 1 -- 172.21.15.120:0/2139337031 wait complete. 2024-09-19T09:14:13.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.302+0000 7ff3f2132740 20 client.19497 trim_cache size 0 max 16384 2024-09-19T09:14:13.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.315+0000 7ff3f2132740 1 Processor -- start 2024-09-19T09:14:13.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.315+0000 7ff3f2132740 1 -- start start 2024-09-19T09:14:13.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.315+0000 7ff3f2132740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55653d4b25d0 0x55653d4cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:13.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.315+0000 7ff3f2132740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55653d4b3b40 0x55653d4b5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:13.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.315+0000 7ff3f2132740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55653d4bd370 0x55653d4b5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:13.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.315+0000 7ff3f2132740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e81ab300 con 0x55653d4b3b40 2024-09-19T09:14:13.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.315+0000 7ff3f2132740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e81ace40 con 0x55653d4bd370 2024-09-19T09:14:13.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.315+0000 7ff3f2132740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e81ab9d0 con 0x55653d4b25d0 2024-09-19T09:14:13.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.315+0000 7ff3edaf5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55653d4b3b40 0x55653d4b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:13.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.315+0000 7ff3edaf5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55653d4b3b40 0x55653d4b5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47718/0 (socket says 172.21.15.120:47718) 2024-09-19T09:14:13.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.315+0000 7ff3edaf5640 1 -- 172.21.15.120:0/603147080 learned_addr learned my addr 172.21.15.120:0/603147080 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:13.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.315+0000 7ff3ed2f4640 1 --2- 172.21.15.120:0/603147080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55653d4b25d0 0x55653d4cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:13.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.315+0000 7ff3ed2f4640 1 --2- 172.21.15.120:0/603147080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55653d4bd370 0x55653d4b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:13.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.315+0000 7ff3edaf5640 1 -- 172.21.15.120:0/603147080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55653d4b25d0 msgr2=0x55653d4cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:13.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.315+0000 7ff3edaf5640 1 --2- 172.21.15.120:0/603147080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55653d4b25d0 0x55653d4cd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.315+0000 7ff3edaf5640 1 -- 172.21.15.120:0/603147080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55653d4bd370 msgr2=0x55653d4b5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:13.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.315+0000 7ff3edaf5640 1 --2- 172.21.15.120:0/603147080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55653d4bd370 0x55653d4b5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.315+0000 7ff3edaf5640 1 -- 172.21.15.120:0/603147080 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55653d4aeed0 con 0x55653d4b3b40 2024-09-19T09:14:13.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.316+0000 7ff3edaf5640 1 --2- 172.21.15.120:0/603147080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55653d4b3b40 0x55653d4b5120 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7ff3e40049c0 tx=0x7ff3e4004b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2117539400838a06 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:13.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.316+0000 7ff3d37fe640 1 -- 172.21.15.120:0/603147080 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4003870 con 0x55653d4b3b40 2024-09-19T09:14:13.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.316+0000 7ff3d37fe640 1 -- 172.21.15.120:0/603147080 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3e40046d0 con 0x55653d4b3b40 2024-09-19T09:14:13.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.316+0000 7ff3d37fe640 1 -- 172.21.15.120:0/603147080 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e402e8f0 con 0x55653d4b3b40 2024-09-19T09:14:13.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.317+0000 7ff3f2132740 1 -- 172.21.15.120:0/603147080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55653d4b3b40 msgr2=0x55653d4b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:13.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.317+0000 7ff3f2132740 1 --2- 172.21.15.120:0/603147080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55653d4b3b40 0x55653d4b5120 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7ff3e40049c0 tx=0x7ff3e4004b00 comp rx=0 tx=0).stop 2024-09-19T09:14:13.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.317+0000 7ff3f2132740 1 -- 172.21.15.120:0/603147080 shutdown_connections 2024-09-19T09:14:13.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.317+0000 7ff3f2132740 1 --2- 172.21.15.120:0/603147080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55653d4bd370 0x55653d4b5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.317+0000 7ff3f2132740 1 --2- 172.21.15.120:0/603147080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55653d4b3b40 0x55653d4b5120 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.317+0000 7ff3f2132740 1 --2- 172.21.15.120:0/603147080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55653d4b25d0 0x55653d4cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.317+0000 7ff3f2132740 1 -- 172.21.15.120:0/603147080 >> 172.21.15.120:0/603147080 conn(0x55653d47b310 msgr2=0x55653d4b94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:13.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.317+0000 7ff3f2132740 1 -- 172.21.15.120:0/603147080 shutdown_connections 2024-09-19T09:14:13.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.317+0000 7ff3f2132740 1 -- 172.21.15.120:0/603147080 wait complete. 2024-09-19T09:14:13.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.318+0000 7ff3f2132740 1 -- start start 2024-09-19T09:14:13.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.318+0000 7ff3f2132740 1 Processor -- start 2024-09-19T09:14:13.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.319+0000 7ff3f2132740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55653d4b25d0 0x55653d4ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:13.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.319+0000 7ff3f2132740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55653d4b3b40 0x55653d4d88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:13.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.319+0000 7ff3f2132740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55653d4bd370 0x55653d4d8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:13.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.319+0000 7ff3f2132740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e81ac770 con 0x55653d4bd370 2024-09-19T09:14:13.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.319+0000 7ff3f2132740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e81ac0a0 con 0x55653d4b25d0 2024-09-19T09:14:13.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.319+0000 7ff3f2132740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e81a92c0 con 0x55653d4b3b40 2024-09-19T09:14:13.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.319+0000 7ff3f2132740 10 client.0 _ll_register_callbacks cb 0x7ff3d8009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:13.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.319+0000 7ff3ed2f4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55653d4b25d0 0x55653d4ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:13.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.319+0000 7ff3edaf5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55653d4b3b40 0x55653d4d88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:13.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.319+0000 7ff3ed2f4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55653d4bd370 0x55653d4d8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:13.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.319+0000 7ff3edaf5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55653d4b3b40 0x55653d4d88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36472/0 (socket says 172.21.15.120:36472) 2024-09-19T09:14:13.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.319+0000 7ff3edaf5640 1 -- 172.21.15.120:0/2535666118 learned_addr learned my addr 172.21.15.120:0/2535666118 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:13.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.319+0000 7ff3ed2f4640 1 -- 172.21.15.120:0/2535666118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55653d4b3b40 msgr2=0x55653d4d88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:13.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.319+0000 7ff3ed2f4640 1 --2- 172.21.15.120:0/2535666118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55653d4b3b40 0x55653d4d88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.319+0000 7ff3ed2f4640 1 -- 172.21.15.120:0/2535666118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55653d4b25d0 msgr2=0x55653d4ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:13.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.319+0000 7ff3ed2f4640 1 --2- 172.21.15.120:0/2535666118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55653d4b25d0 0x55653d4ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.319+0000 7ff3ed2f4640 1 -- 172.21.15.120:0/2535666118 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55653d4d9610 con 0x55653d4bd370 2024-09-19T09:14:13.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.319+0000 7ff3edaf5640 1 --2- 172.21.15.120:0/2535666118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55653d4b3b40 0x55653d4d88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:13.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.319+0000 7ff3ed2f4640 1 --2- 172.21.15.120:0/2535666118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55653d4bd370 0x55653d4d8de0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8003650 tx=0x7ff3d8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db9359f7dfa55b68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:13.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.319+0000 7ff3d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:14:13.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.320+0000 7ff3d17fa640 1 -- 172.21.15.120:0/2535666118 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8035040 con 0x55653d4bd370 2024-09-19T09:14:13.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.320+0000 7ff3d17fa640 1 -- 172.21.15.120:0/2535666118 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ff3d8004f10 con 0x55653d4bd370 2024-09-19T09:14:13.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.320+0000 7ff3f2132740 1 -- 172.21.15.120:0/2535666118 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3e40046d0 con 0x55653d4bd370 2024-09-19T09:14:13.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.320+0000 7ff3d17fa640 1 -- 172.21.15.120:0/2535666118 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff3d8031750 con 0x55653d4bd370 2024-09-19T09:14:13.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.320+0000 7ff3f2132740 10 client.19503 fetch_fsmap learned FSMap version 32 2024-09-19T09:14:13.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.320+0000 7ff3f2132740 1 -- 172.21.15.120:0/2535666118 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55653d53bf00 con 0x55653d4bd370 2024-09-19T09:14:13.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.320+0000 7ff3d17fa640 1 -- 172.21.15.120:0/2535666118 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3e40046d0 con 0x55653d4bd370 2024-09-19T09:14:13.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.320+0000 7ff3d17fa640 1 -- 172.21.15.120:0/2535666118 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff3d8060030 con 0x55653d4bd370 2024-09-19T09:14:13.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.320+0000 7ff3f2132740 10 client.19503 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:14:13.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.320+0000 7ff3f2132740 10 client.19503 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:14:13.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.320+0000 7ff3f2132740 20 client.19503 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:13.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.320+0000 7ff3f2132740 1 --2- 172.21.15.120:0/2535666118 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55653d53c560 0x55653d53e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:13.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.320+0000 7ff3f2132740 4 client.19503 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:14:13.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.320+0000 7ff3f2132740 1 -- 172.21.15.120:0/2535666118 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55653d41fd70 con 0x55653d53c560 2024-09-19T09:14:13.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.321+0000 7ff3edaf5640 1 --2- 172.21.15.120:0/2535666118 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55653d53c560 0x55653d53e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:13.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.321+0000 7ff3edaf5640 1 --2- 172.21.15.120:0/2535666118 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55653d53c560 0x55653d53e950 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=80230e53fa792d9d server_cookie=1de5232b653bcd64 in_seq=0 out_seq=0 2024-09-19T09:14:13.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.321+0000 7ff3d17fa640 10 client.19503 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:14:13.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.321+0000 7ff3d17fa640 1 -- 172.21.15.120:0/2535666118 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55653d41fd70 con 0x55653d53c560 2024-09-19T09:14:13.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.321+0000 7ff3d17fa640 10 client.19503 handle_command_reply: tid=0 2024-09-19T09:14:13.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.321+0000 7ff3f2132740 1 client.19503 shutdown 2024-09-19T09:14:13.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.321+0000 7ff3f2132740 1 -- 172.21.15.120:0/2535666118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55653d4bd370 msgr2=0x55653d4d8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:13.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.321+0000 7ff3f2132740 1 --2- 172.21.15.120:0/2535666118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55653d4bd370 0x55653d4d8de0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8003650 tx=0x7ff3d8000bc0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.322+0000 7ff3f2132740 1 -- 172.21.15.120:0/2535666118 shutdown_connections 2024-09-19T09:14:13.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.322+0000 7ff3f2132740 1 --2- 172.21.15.120:0/2535666118 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55653d53c560 0x55653d53e950 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.322+0000 7ff3f2132740 1 --2- 172.21.15.120:0/2535666118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55653d4bd370 0x55653d4d8de0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.322+0000 7ff3d17fa640 0 client.19503 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:14:13.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.322+0000 7ff3f2132740 1 --2- 172.21.15.120:0/2535666118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55653d4b3b40 0x55653d4d88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.322+0000 7ff3f2132740 1 --2- 172.21.15.120:0/2535666118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55653d4b25d0 0x55653d4ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.322+0000 7ff3f2132740 1 -- 172.21.15.120:0/2535666118 >> 172.21.15.120:0/2535666118 conn(0x55653d47b310 msgr2=0x55653d4bb7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:13.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.322+0000 7ff3f2132740 1 -- 172.21.15.120:0/2535666118 shutdown_connections 2024-09-19T09:14:13.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.322+0000 7ff3f2132740 1 -- 172.21.15.120:0/2535666118 wait complete. 2024-09-19T09:14:13.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.322+0000 7ff3f2132740 20 client.19503 trim_cache size 0 max 16384 2024-09-19T09:14:13.343 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:14:13.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.323+0000 7ff3efd59640 1 -- 172.21.15.120:0/2305560123 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff3d40a3240 msgr2=0x7ff3d40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:13.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.323+0000 7ff3efd59640 1 --2- 172.21.15.120:0/2305560123 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff3d40a3240 0x7ff3d40a5700 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff3e00a4b60 tx=0x7ff3e0098020 comp rx=0 tx=0).stop 2024-09-19T09:14:13.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.323+0000 7ff3efd59640 1 -- 172.21.15.120:0/2305560123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3e8102ef0 msgr2=0x7ff3e81a4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:13.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.323+0000 7ff3efd59640 1 --2- 172.21.15.120:0/2305560123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3e8102ef0 0x7ff3e81a4450 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7ff3e000eb10 tx=0x7ff3e000c2d0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.323+0000 7ff3efd59640 1 -- 172.21.15.120:0/2305560123 shutdown_connections 2024-09-19T09:14:13.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.323+0000 7ff3efd59640 1 --2- 172.21.15.120:0/2305560123 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff3d40a3240 0x7ff3d40a5700 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.323+0000 7ff3efd59640 1 --2- 172.21.15.120:0/2305560123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3e8102ef0 0x7ff3e81a4450 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.323+0000 7ff3efd59640 1 --2- 172.21.15.120:0/2305560123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3e8102570 0x7ff3e8199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.323+0000 7ff3efd59640 1 --2- 172.21.15.120:0/2305560123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3e8100e20 0x7ff3e81993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:13.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.323+0000 7ff3efd59640 1 -- 172.21.15.120:0/2305560123 >> 172.21.15.120:0/2305560123 conn(0x7ff3e80fc7d0 msgr2=0x7ff3e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:13.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.323+0000 7ff3efd59640 1 -- 172.21.15.120:0/2305560123 shutdown_connections 2024-09-19T09:14:13.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:13.323+0000 7ff3efd59640 1 -- 172.21.15.120:0/2305560123 wait complete. 2024-09-19T09:14:14.073 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1067347 9.33 MB/sec execute 489 sec latency 21.479 ms 2024-09-19T09:14:14.368 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:14:14.624 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:14:15.073 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1069858 9.34 MB/sec execute 490 sec latency 48.060 ms 2024-09-19T09:14:15.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:14 smithi120 ceph-mon[31150]: pgmap v684: 97 pgs: 97 active+clean; 42 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.5 MiB/s wr, 307 op/s 2024-09-19T09:14:15.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:14 smithi196 ceph-mon[35150]: pgmap v684: 97 pgs: 97 active+clean; 42 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.5 MiB/s wr, 307 op/s 2024-09-19T09:14:15.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:14 smithi153 ceph-mon[35071]: pgmap v684: 97 pgs: 97 active+clean; 42 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.5 MiB/s wr, 307 op/s 2024-09-19T09:14:15.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.185+0000 7ff032ca1640 1 -- 172.21.15.120:0/2746080138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff02c100500 msgr2=0x7ff02c1008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:15.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.185+0000 7ff032ca1640 1 --2- 172.21.15.120:0/2746080138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff02c100500 0x7ff02c1008e0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7ff014009900 tx=0x7ff01402a410 comp rx=0 tx=0).stop 2024-09-19T09:14:15.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.185+0000 7ff032ca1640 1 -- 172.21.15.120:0/2746080138 shutdown_connections 2024-09-19T09:14:15.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.185+0000 7ff032ca1640 1 --2- 172.21.15.120:0/2746080138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff02c106f70 0x7ff02c10eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.185+0000 7ff032ca1640 1 --2- 172.21.15.120:0/2746080138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff02c100eb0 0x7ff02c106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.185+0000 7ff032ca1640 1 --2- 172.21.15.120:0/2746080138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff02c100500 0x7ff02c1008e0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.185+0000 7ff032ca1640 1 -- 172.21.15.120:0/2746080138 >> 172.21.15.120:0/2746080138 conn(0x7ff02c0fc5c0 msgr2=0x7ff02c0fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:15.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.185+0000 7ff032ca1640 1 -- 172.21.15.120:0/2746080138 shutdown_connections 2024-09-19T09:14:15.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.185+0000 7ff032ca1640 1 -- 172.21.15.120:0/2746080138 wait complete. 2024-09-19T09:14:15.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.185+0000 7ff032ca1640 1 Processor -- start 2024-09-19T09:14:15.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.185+0000 7ff032ca1640 1 -- start start 2024-09-19T09:14:15.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.185+0000 7ff032ca1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff02c100500 0x7ff02c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:15.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.185+0000 7ff032ca1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff02c100eb0 0x7ff02c19ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:15.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.185+0000 7ff032ca1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff02c106f70 0x7ff02c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:15.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.185+0000 7ff032ca1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c111f90 con 0x7ff02c106f70 2024-09-19T09:14:15.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.185+0000 7ff032ca1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c111e10 con 0x7ff02c100eb0 2024-09-19T09:14:15.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.185+0000 7ff032ca1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c112110 con 0x7ff02c100500 2024-09-19T09:14:15.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.186+0000 7ff03123e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff02c106f70 0x7ff02c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:15.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.186+0000 7ff030a3d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff02c100500 0x7ff02c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:15.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.186+0000 7ff03123e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff02c106f70 0x7ff02c19ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47740/0 (socket says 172.21.15.120:47740) 2024-09-19T09:14:15.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.186+0000 7ff02bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff02c100eb0 0x7ff02c19ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:15.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.186+0000 7ff03123e640 1 -- 172.21.15.120:0/3799229162 learned_addr learned my addr 172.21.15.120:0/3799229162 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:15.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.186+0000 7ff03123e640 1 -- 172.21.15.120:0/3799229162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff02c100500 msgr2=0x7ff02c19a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:15.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.186+0000 7ff03123e640 1 --2- 172.21.15.120:0/3799229162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff02c100500 0x7ff02c19a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.186+0000 7ff03123e640 1 -- 172.21.15.120:0/3799229162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff02c100eb0 msgr2=0x7ff02c19ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:15.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.186+0000 7ff03123e640 1 --2- 172.21.15.120:0/3799229162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff02c100eb0 0x7ff02c19ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.186+0000 7ff03123e640 1 -- 172.21.15.120:0/3799229162 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff02c19f670 con 0x7ff02c106f70 2024-09-19T09:14:15.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.186+0000 7ff02bfff640 1 --2- 172.21.15.120:0/3799229162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff02c100eb0 0x7ff02c19ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:15.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.186+0000 7ff03123e640 1 --2- 172.21.15.120:0/3799229162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff02c106f70 0x7ff02c19ef90 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7ff02400eb10 tx=0x7ff02400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:15.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.186+0000 7ff029ffb640 1 -- 172.21.15.120:0/3799229162 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024004b20 con 0x7ff02c106f70 2024-09-19T09:14:15.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.186+0000 7ff029ffb640 1 -- 172.21.15.120:0/3799229162 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff024004cc0 con 0x7ff02c106f70 2024-09-19T09:14:15.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.186+0000 7ff029ffb640 1 -- 172.21.15.120:0/3799229162 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024010760 con 0x7ff02c106f70 2024-09-19T09:14:15.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.186+0000 7ff032ca1640 1 -- 172.21.15.120:0/3799229162 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff02c19f960 con 0x7ff02c106f70 2024-09-19T09:14:15.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.187+0000 7ff032ca1640 1 -- 172.21.15.120:0/3799229162 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff02c1a7240 con 0x7ff02c106f70 2024-09-19T09:14:15.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.189+0000 7ff03507a740 1 Processor -- start 2024-09-19T09:14:15.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.189+0000 7ff029ffb640 1 -- 172.21.15.120:0/3799229162 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7ff024005020 con 0x7ff02c106f70 2024-09-19T09:14:15.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.189+0000 7ff03507a740 1 -- start start 2024-09-19T09:14:15.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.189+0000 7ff03507a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd385a5b40 0x55fd385a4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:15.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.189+0000 7ff03507a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd385a45d0 0x55fd385a4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:15.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.190+0000 7ff03507a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd385af370 0x55fd385b1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:15.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.190+0000 7ff03507a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fd3850a2f0 con 0x55fd385af370 2024-09-19T09:14:15.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.190+0000 7ff03507a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fd384b1cf0 con 0x55fd385a45d0 2024-09-19T09:14:15.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.190+0000 7ff03507a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fd385062b0 con 0x55fd385a5b40 2024-09-19T09:14:15.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.190+0000 7ff02bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd385a5b40 0x55fd385a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:15.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.190+0000 7ff030a3d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd385a45d0 0x55fd385a4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:15.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.190+0000 7ff030a3d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd385a45d0 0x55fd385a4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52124/0 (socket says 172.21.15.120:52124) 2024-09-19T09:14:15.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.190+0000 7ff030a3d640 1 -- 172.21.15.120:0/1383397835 learned_addr learned my addr 172.21.15.120:0/1383397835 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:15.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.190+0000 7ff02bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd385a5b40 0x55fd385a4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36488/0 (socket says 172.21.15.120:36488) 2024-09-19T09:14:15.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.190+0000 7ff02bfff640 1 --2- 172.21.15.120:0/1383397835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd385af370 0x55fd385b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:15.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.190+0000 7ff029ffb640 1 --2- 172.21.15.120:0/3799229162 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff0180a3240 0x7ff0180a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:15.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.190+0000 7ff030a3d640 1 -- 172.21.15.120:0/1383397835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd385a5b40 msgr2=0x55fd385a4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:15.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.190+0000 7ff030a3d640 1 --2- 172.21.15.120:0/1383397835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd385a5b40 0x55fd385a4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.190+0000 7ff030a3d640 1 -- 172.21.15.120:0/1383397835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd385af370 msgr2=0x55fd385b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:15.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.190+0000 7ff030a3d640 1 --2- 172.21.15.120:0/1383397835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd385af370 0x55fd385b1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.190+0000 7ff030a3d640 1 -- 172.21.15.120:0/1383397835 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd384963a0 con 0x55fd385a45d0 2024-09-19T09:14:15.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.190+0000 7ff029ffb640 1 -- 172.21.15.120:0/3799229162 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ff0240cfe00 con 0x7ff02c106f70 2024-09-19T09:14:15.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.190+0000 7ff030a3d640 1 --2- 172.21.15.120:0/1383397835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd385a45d0 0x55fd385a4a10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff02c069fc0 tx=0x7ff014005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76650e6f6f9afc6b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:15.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.190+0000 7ff03123e640 1 --2- 172.21.15.120:0/3799229162 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff0180a3240 0x7ff0180a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:15.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.190+0000 7ff022ffd640 1 -- 172.21.15.120:0/1383397835 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff014040070 con 0x55fd385a45d0 2024-09-19T09:14:15.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.191+0000 7ff022ffd640 1 -- 172.21.15.120:0/1383397835 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff014031870 con 0x55fd385a45d0 2024-09-19T09:14:15.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.191+0000 7ff022ffd640 1 -- 172.21.15.120:0/1383397835 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff014035040 con 0x55fd385a45d0 2024-09-19T09:14:15.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.191+0000 7ff03507a740 1 -- 172.21.15.120:0/1383397835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd385a45d0 msgr2=0x55fd385a4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:15.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.191+0000 7ff03507a740 1 --2- 172.21.15.120:0/1383397835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd385a45d0 0x55fd385a4a10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff02c069fc0 tx=0x7ff014005bd0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.191+0000 7ff03123e640 1 --2- 172.21.15.120:0/3799229162 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff0180a3240 0x7ff0180a5700 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff024004f90 tx=0x7ff02400efd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:15.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.192+0000 7ff03507a740 1 -- 172.21.15.120:0/1383397835 shutdown_connections 2024-09-19T09:14:15.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.192+0000 7ff03507a740 1 --2- 172.21.15.120:0/1383397835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd385af370 0x55fd385b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.192+0000 7ff03507a740 1 --2- 172.21.15.120:0/1383397835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd385a45d0 0x55fd385a4a10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.192+0000 7ff03507a740 1 --2- 172.21.15.120:0/1383397835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd385a5b40 0x55fd385a4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.192+0000 7ff03507a740 1 -- 172.21.15.120:0/1383397835 >> 172.21.15.120:0/1383397835 conn(0x55fd3856d310 msgr2=0x55fd385a0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:15.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.192+0000 7ff03507a740 1 -- 172.21.15.120:0/1383397835 shutdown_connections 2024-09-19T09:14:15.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.192+0000 7ff03507a740 1 -- 172.21.15.120:0/1383397835 wait complete. 2024-09-19T09:14:15.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.192+0000 7ff03507a740 1 -- start start 2024-09-19T09:14:15.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.192+0000 7ff03507a740 1 Processor -- start 2024-09-19T09:14:15.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.192+0000 7ff03507a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd385a45d0 0x55fd385aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:15.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.192+0000 7ff03507a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd385a5b40 0x55fd385abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:15.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.192+0000 7ff03507a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd385af370 0x55fd385ac4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:15.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.193+0000 7ff03507a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fd385b2010 con 0x55fd385a5b40 2024-09-19T09:14:15.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.193+0000 7ff03507a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fd384c5060 con 0x55fd385af370 2024-09-19T09:14:15.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.193+0000 7ff03507a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fd385b2190 con 0x55fd385a45d0 2024-09-19T09:14:15.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.193+0000 7ff030a3d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd385a5b40 0x55fd385abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:15.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.193+0000 7ff03507a740 10 client.0 _ll_register_callbacks cb 0x55fd382188c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:15.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.193+0000 7ff030a3d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd385a5b40 0x55fd385abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47758/0 (socket says 172.21.15.120:47758) 2024-09-19T09:14:15.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.193+0000 7ff02bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd385a45d0 0x55fd385aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:15.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.193+0000 7ff030a3d640 1 -- 172.21.15.120:0/2851591516 learned_addr learned my addr 172.21.15.120:0/2851591516 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:15.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.193+0000 7ff02bfff640 1 --2- 172.21.15.120:0/2851591516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd385af370 0x55fd385ac4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:15.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.193+0000 7ff02bfff640 1 -- 172.21.15.120:0/2851591516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd385af370 msgr2=0x55fd385ac4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:15.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.193+0000 7ff02bfff640 1 --2- 172.21.15.120:0/2851591516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd385af370 0x55fd385ac4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.193+0000 7ff02bfff640 1 -- 172.21.15.120:0/2851591516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd385a5b40 msgr2=0x55fd385abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:15.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.193+0000 7ff02bfff640 1 --2- 172.21.15.120:0/2851591516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd385a5b40 0x55fd385abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.193+0000 7ff02bfff640 1 -- 172.21.15.120:0/2851591516 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd3827b3c0 con 0x55fd385a45d0 2024-09-19T09:14:15.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.193+0000 7ff02bfff640 1 --2- 172.21.15.120:0/2851591516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd385a45d0 0x55fd385aece0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff01c0058c0 tx=0x7ff01c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b127cb2f127c8b7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:15.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.193+0000 7ff00bfff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:14:15.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.193+0000 7ff00bfff640 1 -- 172.21.15.120:0/2851591516 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01c011260 con 0x55fd385a45d0 2024-09-19T09:14:15.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.193+0000 7ff03507a740 1 -- 172.21.15.120:0/2851591516 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff014031870 con 0x55fd385a45d0 2024-09-19T09:14:15.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.193+0000 7ff00bfff640 1 -- 172.21.15.120:0/2851591516 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ff01c005020 con 0x55fd385a45d0 2024-09-19T09:14:15.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.194+0000 7ff00bfff640 1 -- 172.21.15.120:0/2851591516 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff01c005220 con 0x55fd385a45d0 2024-09-19T09:14:15.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.194+0000 7ff03507a740 10 client.28372 fetch_fsmap learned FSMap version 32 2024-09-19T09:14:15.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.194+0000 7ff03507a740 1 -- 172.21.15.120:0/2851591516 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fd385bda20 con 0x55fd385a45d0 2024-09-19T09:14:15.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.194+0000 7ff00bfff640 1 -- 172.21.15.120:0/2851591516 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff014031870 con 0x55fd385a45d0 2024-09-19T09:14:15.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.194+0000 7ff00bfff640 1 -- 172.21.15.120:0/2851591516 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff01c022430 con 0x55fd385a45d0 2024-09-19T09:14:15.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.194+0000 7ff03507a740 10 client.28372 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:14:15.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.194+0000 7ff03507a740 10 client.28372 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:14:15.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.194+0000 7ff03507a740 20 client.28372 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:15.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.194+0000 7ff03507a740 1 --2- 172.21.15.120:0/2851591516 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fd385bee30 0x55fd385c1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:15.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.194+0000 7ff03507a740 4 client.28372 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:14:15.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.194+0000 7ff03507a740 1 -- 172.21.15.120:0/2851591516 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd38517860 con 0x55fd385bee30 2024-09-19T09:14:15.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.194+0000 7ff030a3d640 1 --2- 172.21.15.120:0/2851591516 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fd385bee30 0x55fd385c1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:15.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.195+0000 7ff00bfff640 10 client.28372 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:15.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.195+0000 7ff030a3d640 1 --2- 172.21.15.120:0/2851591516 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fd385bee30 0x55fd385c1260 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5d8cf4b82c35ca3 server_cookie=c78477caf59aded3 in_seq=0 out_seq=0 2024-09-19T09:14:15.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.196+0000 7ff00bfff640 1 -- 172.21.15.120:0/2851591516 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fd38517860 con 0x55fd385bee30 2024-09-19T09:14:15.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.196+0000 7ff00bfff640 10 client.28372 handle_command_reply: tid=0 2024-09-19T09:14:15.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.196+0000 7ff03507a740 1 client.28372 shutdown 2024-09-19T09:14:15.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.196+0000 7ff03507a740 1 -- 172.21.15.120:0/2851591516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd385a45d0 msgr2=0x55fd385aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:15.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.196+0000 7ff03507a740 1 --2- 172.21.15.120:0/2851591516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd385a45d0 0x55fd385aece0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff01c0058c0 tx=0x7ff01c004730 comp rx=0 tx=0).stop 2024-09-19T09:14:15.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.196+0000 7ff03507a740 1 -- 172.21.15.120:0/2851591516 shutdown_connections 2024-09-19T09:14:15.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.196+0000 7ff03507a740 1 --2- 172.21.15.120:0/2851591516 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fd385bee30 0x55fd385c1260 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.196+0000 7ff03507a740 1 --2- 172.21.15.120:0/2851591516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd385af370 0x55fd385ac4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.196+0000 7ff03507a740 1 --2- 172.21.15.120:0/2851591516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd385a5b40 0x55fd385abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.196+0000 7ff03507a740 1 --2- 172.21.15.120:0/2851591516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd385a45d0 0x55fd385aece0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.196+0000 7ff00bfff640 0 client.28372 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:15.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.196+0000 7ff03507a740 1 -- 172.21.15.120:0/2851591516 >> 172.21.15.120:0/2851591516 conn(0x55fd3856d310 msgr2=0x55fd385a6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:15.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.196+0000 7ff03507a740 1 -- 172.21.15.120:0/2851591516 shutdown_connections 2024-09-19T09:14:15.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.196+0000 7ff03507a740 1 -- 172.21.15.120:0/2851591516 wait complete. 2024-09-19T09:14:15.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.196+0000 7ff03507a740 20 client.28372 trim_cache size 0 max 16384 2024-09-19T09:14:15.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.209+0000 7ff03507a740 1 Processor -- start 2024-09-19T09:14:15.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.209+0000 7ff03507a740 1 -- start start 2024-09-19T09:14:15.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.209+0000 7ff03507a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd385a45d0 0x55fd385adf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:15.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.209+0000 7ff03507a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd385a5b40 0x55fd385a7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:15.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.209+0000 7ff03507a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd385af370 0x55fd385a7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:15.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.209+0000 7ff03507a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1af8c0 con 0x55fd385a5b40 2024-09-19T09:14:15.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.209+0000 7ff03507a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1b1400 con 0x55fd385af370 2024-09-19T09:14:15.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.209+0000 7ff03507a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1aff90 con 0x55fd385a45d0 2024-09-19T09:14:15.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.209+0000 7ff030a3d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd385a5b40 0x55fd385a7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:15.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.209+0000 7ff030a3d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd385a5b40 0x55fd385a7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47764/0 (socket says 172.21.15.120:47764) 2024-09-19T09:14:15.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.209+0000 7ff030a3d640 1 -- 172.21.15.120:0/4040970277 learned_addr learned my addr 172.21.15.120:0/4040970277 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:15.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.210+0000 7ff02bfff640 1 --2- 172.21.15.120:0/4040970277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd385af370 0x55fd385a7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:15.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.210+0000 7ff02bfff640 1 --2- 172.21.15.120:0/4040970277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd385a45d0 0x55fd385adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:15.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.210+0000 7ff030a3d640 1 -- 172.21.15.120:0/4040970277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd385a45d0 msgr2=0x55fd385adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:15.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.210+0000 7ff030a3d640 1 --2- 172.21.15.120:0/4040970277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd385a45d0 0x55fd385adf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.210+0000 7ff030a3d640 1 -- 172.21.15.120:0/4040970277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd385af370 msgr2=0x55fd385a7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:15.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.210+0000 7ff030a3d640 1 --2- 172.21.15.120:0/4040970277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd385af370 0x55fd385a7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.210+0000 7ff030a3d640 1 -- 172.21.15.120:0/4040970277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd385a1960 con 0x55fd385a5b40 2024-09-19T09:14:15.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.210+0000 7ff02bfff640 1 --2- 172.21.15.120:0/4040970277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd385af370 0x55fd385a7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:14:15.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.210+0000 7ff030a3d640 1 --2- 172.21.15.120:0/4040970277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd385a5b40 0x55fd385a7120 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7ff014009230 tx=0x7ff014009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa7890061893dc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:15.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.210+0000 7ff0227fc640 1 -- 172.21.15.120:0/4040970277 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff014040040 con 0x55fd385a5b40 2024-09-19T09:14:15.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.210+0000 7ff0227fc640 1 -- 172.21.15.120:0/4040970277 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0140054e0 con 0x55fd385a5b40 2024-09-19T09:14:15.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.210+0000 7ff0227fc640 1 -- 172.21.15.120:0/4040970277 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff014035040 con 0x55fd385a5b40 2024-09-19T09:14:15.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.211+0000 7ff03507a740 1 -- 172.21.15.120:0/4040970277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd385a5b40 msgr2=0x55fd385a7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:15.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.211+0000 7ff03507a740 1 --2- 172.21.15.120:0/4040970277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd385a5b40 0x55fd385a7120 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7ff014009230 tx=0x7ff014009260 comp rx=0 tx=0).stop 2024-09-19T09:14:15.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.211+0000 7ff03507a740 1 -- 172.21.15.120:0/4040970277 shutdown_connections 2024-09-19T09:14:15.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.211+0000 7ff03507a740 1 --2- 172.21.15.120:0/4040970277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd385af370 0x55fd385a7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.211+0000 7ff03507a740 1 --2- 172.21.15.120:0/4040970277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd385a5b40 0x55fd385a7120 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.211+0000 7ff03507a740 1 --2- 172.21.15.120:0/4040970277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd385a45d0 0x55fd385adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.211+0000 7ff03507a740 1 -- 172.21.15.120:0/4040970277 >> 172.21.15.120:0/4040970277 conn(0x55fd3856d310 msgr2=0x55fd385ad7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:15.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.211+0000 7ff03507a740 1 -- 172.21.15.120:0/4040970277 shutdown_connections 2024-09-19T09:14:15.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.211+0000 7ff03507a740 1 -- 172.21.15.120:0/4040970277 wait complete. 2024-09-19T09:14:15.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.211+0000 7ff03507a740 1 -- start start 2024-09-19T09:14:15.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.211+0000 7ff03507a740 1 Processor -- start 2024-09-19T09:14:15.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.212+0000 7ff03507a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd385a45d0 0x55fd385b7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:15.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.212+0000 7ff03507a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd385a5b40 0x55fd385b7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:15.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.212+0000 7ff03507a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd385af370 0x55fd385b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:15.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.212+0000 7ff03507a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1b0d30 con 0x55fd385a5b40 2024-09-19T09:14:15.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.212+0000 7ff03507a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1b0660 con 0x55fd385a45d0 2024-09-19T09:14:15.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.212+0000 7ff03507a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1ad880 con 0x55fd385af370 2024-09-19T09:14:15.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.212+0000 7ff03507a740 10 client.0 _ll_register_callbacks cb 0x7ff01c011910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:15.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.212+0000 7ff030a3d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd385a5b40 0x55fd385b7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:15.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.212+0000 7ff030a3d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd385a5b40 0x55fd385b7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47780/0 (socket says 172.21.15.120:47780) 2024-09-19T09:14:15.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.212+0000 7ff02bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd385a45d0 0x55fd385b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:15.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.212+0000 7ff030a3d640 1 -- 172.21.15.120:0/4188601708 learned_addr learned my addr 172.21.15.120:0/4188601708 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:15.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.212+0000 7ff02bfff640 1 --2- 172.21.15.120:0/4188601708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd385af370 0x55fd385b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:15.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.212+0000 7ff030a3d640 1 -- 172.21.15.120:0/4188601708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd385af370 msgr2=0x55fd385b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:15.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.212+0000 7ff030a3d640 1 --2- 172.21.15.120:0/4188601708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd385af370 0x55fd385b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.212+0000 7ff030a3d640 1 -- 172.21.15.120:0/4188601708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd385a45d0 msgr2=0x55fd385b7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:15.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.212+0000 7ff030a3d640 1 --2- 172.21.15.120:0/4188601708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd385a45d0 0x55fd385b7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.212+0000 7ff030a3d640 1 -- 172.21.15.120:0/4188601708 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd385b80a0 con 0x55fd385a5b40 2024-09-19T09:14:15.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.212+0000 7ff030a3d640 1 --2- 172.21.15.120:0/4188601708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd385a5b40 0x55fd385b7960 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7ff014005940 tx=0x7ff014005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be823aafb5afd77b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:15.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.212+0000 7ff00bfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:14:15.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.212+0000 7ff00bfff640 1 -- 172.21.15.120:0/4188601708 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff014035030 con 0x55fd385a5b40 2024-09-19T09:14:15.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.213+0000 7ff00bfff640 1 -- 172.21.15.120:0/4188601708 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ff01404f640 con 0x55fd385a5b40 2024-09-19T09:14:15.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.213+0000 7ff03507a740 1 -- 172.21.15.120:0/4188601708 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0140054e0 con 0x55fd385a5b40 2024-09-19T09:14:15.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.213+0000 7ff00bfff640 1 -- 172.21.15.120:0/4188601708 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff014040040 con 0x55fd385a5b40 2024-09-19T09:14:15.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.213+0000 7ff03507a740 10 client.19521 fetch_fsmap learned FSMap version 32 2024-09-19T09:14:15.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.213+0000 7ff03507a740 1 -- 172.21.15.120:0/4188601708 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fd3862a110 con 0x55fd385a5b40 2024-09-19T09:14:15.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.213+0000 7ff00bfff640 1 -- 172.21.15.120:0/4188601708 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0140054e0 con 0x55fd385a5b40 2024-09-19T09:14:15.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.213+0000 7ff00bfff640 1 -- 172.21.15.120:0/4188601708 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff01404e3e0 con 0x55fd385a5b40 2024-09-19T09:14:15.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.213+0000 7ff03507a740 10 client.19521 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:14:15.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.213+0000 7ff03507a740 10 client.19521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:14:15.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.213+0000 7ff03507a740 20 client.19521 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:15.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.213+0000 7ff03507a740 1 --2- 172.21.15.120:0/4188601708 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fd3862a450 0x55fd3862c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:15.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.213+0000 7ff03507a740 4 client.19521 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:14:15.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.213+0000 7ff03507a740 1 -- 172.21.15.120:0/4188601708 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd38511d70 con 0x55fd3862a450 2024-09-19T09:14:15.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.213+0000 7ff02bfff640 1 --2- 172.21.15.120:0/4188601708 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fd3862a450 0x55fd3862c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:15.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.214+0000 7ff02bfff640 1 --2- 172.21.15.120:0/4188601708 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fd3862a450 0x55fd3862c840 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70c97abd340ded91 server_cookie=5c3af5f9d4eb6d78 in_seq=0 out_seq=0 2024-09-19T09:14:15.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.214+0000 7ff00bfff640 10 client.19521 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:15.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.215+0000 7ff00bfff640 1 -- 172.21.15.120:0/4188601708 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd38511d70 con 0x55fd3862a450 2024-09-19T09:14:15.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.215+0000 7ff00bfff640 10 client.19521 handle_command_reply: tid=0 2024-09-19T09:14:15.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.215+0000 7ff03507a740 1 client.19521 shutdown 2024-09-19T09:14:15.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.215+0000 7ff03507a740 1 -- 172.21.15.120:0/4188601708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd385a5b40 msgr2=0x55fd385b7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:15.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.215+0000 7ff03507a740 1 --2- 172.21.15.120:0/4188601708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd385a5b40 0x55fd385b7960 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7ff014005940 tx=0x7ff014005b50 comp rx=0 tx=0).stop 2024-09-19T09:14:15.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.215+0000 7ff03507a740 1 -- 172.21.15.120:0/4188601708 shutdown_connections 2024-09-19T09:14:15.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.215+0000 7ff03507a740 1 --2- 172.21.15.120:0/4188601708 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fd3862a450 0x55fd3862c840 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.215+0000 7ff03507a740 1 --2- 172.21.15.120:0/4188601708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fd385af370 0x55fd385b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.215+0000 7ff03507a740 1 --2- 172.21.15.120:0/4188601708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd385a5b40 0x55fd385b7960 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.215+0000 7ff00bfff640 0 client.19521 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:15.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.215+0000 7ff03507a740 1 --2- 172.21.15.120:0/4188601708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd385a45d0 0x55fd385b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.215+0000 7ff03507a740 1 -- 172.21.15.120:0/4188601708 >> 172.21.15.120:0/4188601708 conn(0x55fd3856d310 msgr2=0x55fd385ae010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:15.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.215+0000 7ff03507a740 1 -- 172.21.15.120:0/4188601708 shutdown_connections 2024-09-19T09:14:15.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.215+0000 7ff03507a740 1 -- 172.21.15.120:0/4188601708 wait complete. 2024-09-19T09:14:15.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.215+0000 7ff03507a740 20 client.19521 trim_cache size 0 max 16384 2024-09-19T09:14:15.226 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:14:15.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.216+0000 7ff032ca1640 1 -- 172.21.15.120:0/3799229162 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff0180a3240 msgr2=0x7ff0180a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:15.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.216+0000 7ff032ca1640 1 --2- 172.21.15.120:0/3799229162 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff0180a3240 0x7ff0180a5700 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff024004f90 tx=0x7ff02400efd0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.216+0000 7ff032ca1640 1 -- 172.21.15.120:0/3799229162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff02c106f70 msgr2=0x7ff02c19ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:15.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.216+0000 7ff032ca1640 1 --2- 172.21.15.120:0/3799229162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff02c106f70 0x7ff02c19ef90 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7ff02400eb10 tx=0x7ff02400c2d0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.216+0000 7ff032ca1640 1 -- 172.21.15.120:0/3799229162 shutdown_connections 2024-09-19T09:14:15.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.216+0000 7ff032ca1640 1 --2- 172.21.15.120:0/3799229162 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff0180a3240 0x7ff0180a5700 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.216+0000 7ff032ca1640 1 --2- 172.21.15.120:0/3799229162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff02c106f70 0x7ff02c19ef90 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.216+0000 7ff032ca1640 1 --2- 172.21.15.120:0/3799229162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff02c100eb0 0x7ff02c19ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.216+0000 7ff032ca1640 1 --2- 172.21.15.120:0/3799229162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff02c100500 0x7ff02c19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:15.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.216+0000 7ff032ca1640 1 -- 172.21.15.120:0/3799229162 >> 172.21.15.120:0/3799229162 conn(0x7ff02c0fc5c0 msgr2=0x7ff02c1057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:15.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.216+0000 7ff032ca1640 1 -- 172.21.15.120:0/3799229162 shutdown_connections 2024-09-19T09:14:15.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:15.216+0000 7ff032ca1640 1 -- 172.21.15.120:0/3799229162 wait complete. 2024-09-19T09:14:16.073 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1073003 9.36 MB/sec execute 491 sec latency 16.731 ms 2024-09-19T09:14:17.074 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1075709 9.36 MB/sec execute 492 sec latency 14.826 ms 2024-09-19T09:14:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:16 smithi196 ceph-mon[35150]: pgmap v685: 97 pgs: 97 active+clean; 47 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.8 MiB/s wr, 371 op/s 2024-09-19T09:14:17.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:16 smithi153 ceph-mon[35071]: pgmap v685: 97 pgs: 97 active+clean; 47 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.8 MiB/s wr, 371 op/s 2024-09-19T09:14:17.203 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:14:17.203 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:14:17.225 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:16 smithi120 ceph-mon[31150]: pgmap v685: 97 pgs: 97 active+clean; 47 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.8 MiB/s wr, 371 op/s 2024-09-19T09:14:17.476 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:14:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.589+0000 7efed09c4640 1 -- 172.21.15.120:0/2373844177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efecc1025f0 msgr2=0x7efecc108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.589+0000 7efed09c4640 1 --2- 172.21.15.120:0/2373844177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efecc1025f0 0x7efecc108e80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efec4009ca0 tx=0x7efec402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.589+0000 7efed09c4640 1 -- 172.21.15.120:0/2373844177 shutdown_connections 2024-09-19T09:14:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.589+0000 7efed09c4640 1 --2- 172.21.15.120:0/2373844177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efecc1025f0 0x7efecc108e80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.589+0000 7efed09c4640 1 --2- 172.21.15.120:0/2373844177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efecc101c30 0x7efecc1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.589+0000 7efed09c4640 1 --2- 172.21.15.120:0/2373844177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efecc100a40 0x7efecc100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.589+0000 7efed09c4640 1 -- 172.21.15.120:0/2373844177 >> 172.21.15.120:0/2373844177 conn(0x7efecc0fc1e0 msgr2=0x7efecc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.589+0000 7efed09c4640 1 -- 172.21.15.120:0/2373844177 shutdown_connections 2024-09-19T09:14:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.589+0000 7efed09c4640 1 -- 172.21.15.120:0/2373844177 wait complete. 2024-09-19T09:14:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.590+0000 7efed09c4640 1 Processor -- start 2024-09-19T09:14:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.590+0000 7efed09c4640 1 -- start start 2024-09-19T09:14:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.590+0000 7efed09c4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efecc100a40 0x7efecc06a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.590+0000 7efed09c4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efecc101c30 0x7efecc19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.590+0000 7efed09c4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efecc1025f0 0x7efecc1a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.590+0000 7efed09c4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efecc10c6c0 con 0x7efecc101c30 2024-09-19T09:14:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.590+0000 7efed09c4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efecc10c540 con 0x7efecc1025f0 2024-09-19T09:14:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.590+0000 7efed09c4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efecc10c840 con 0x7efecc100a40 2024-09-19T09:14:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.590+0000 7efec9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efecc101c30 0x7efecc19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.590+0000 7efec9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efecc101c30 0x7efecc19ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47796/0 (socket says 172.21.15.120:47796) 2024-09-19T09:14:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.590+0000 7efec9d9b640 1 -- 172.21.15.120:0/1913901734 learned_addr learned my addr 172.21.15.120:0/1913901734 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.590+0000 7efecad9d640 1 --2- 172.21.15.120:0/1913901734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efecc1025f0 0x7efecc1a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.590+0000 7efeca59c640 1 --2- 172.21.15.120:0/1913901734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efecc100a40 0x7efecc06a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.591+0000 7efec9d9b640 1 -- 172.21.15.120:0/1913901734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efecc100a40 msgr2=0x7efecc06a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.591+0000 7efec9d9b640 1 --2- 172.21.15.120:0/1913901734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efecc100a40 0x7efecc06a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.591+0000 7efec9d9b640 1 -- 172.21.15.120:0/1913901734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efecc1025f0 msgr2=0x7efecc1a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.591+0000 7efec9d9b640 1 --2- 172.21.15.120:0/1913901734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efecc1025f0 0x7efecc1a13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.591+0000 7efec9d9b640 1 -- 172.21.15.120:0/1913901734 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efecc1a1b80 con 0x7efecc101c30 2024-09-19T09:14:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.591+0000 7efec9d9b640 1 --2- 172.21.15.120:0/1913901734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efecc101c30 0x7efecc19ee60 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7efec000e9f0 tx=0x7efec000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.591+0000 7efebb7fe640 1 -- 172.21.15.120:0/1913901734 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec000cd80 con 0x7efecc101c30 2024-09-19T09:14:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.591+0000 7efed09c4640 1 -- 172.21.15.120:0/1913901734 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efecc1a1e70 con 0x7efecc101c30 2024-09-19T09:14:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.591+0000 7efebb7fe640 1 -- 172.21.15.120:0/1913901734 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efec0010910 con 0x7efecc101c30 2024-09-19T09:14:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.591+0000 7efebb7fe640 1 -- 172.21.15.120:0/1913901734 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec00188f0 con 0x7efecc101c30 2024-09-19T09:14:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.591+0000 7efed09c4640 1 -- 172.21.15.120:0/1913901734 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efecc1a2330 con 0x7efecc101c30 2024-09-19T09:14:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.594+0000 7efed3840740 1 Processor -- start 2024-09-19T09:14:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.594+0000 7efebb7fe640 1 -- 172.21.15.120:0/1913901734 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7efec0010ac0 con 0x7efecc101c30 2024-09-19T09:14:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.595+0000 7efed3840740 1 -- start start 2024-09-19T09:14:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.595+0000 7efed3840740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563496cb59a0 0x563496cb5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.595+0000 7efed3840740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563496cb62e0 0x563496cbada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.595+0000 7efed3840740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563496cbb370 0x563496cbd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.595+0000 7efed3840740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563496c192f0 con 0x563496cb59a0 2024-09-19T09:14:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.595+0000 7efed3840740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563496bc1270 con 0x563496cb62e0 2024-09-19T09:14:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.595+0000 7efed3840740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563496c152b0 con 0x563496cbb370 2024-09-19T09:14:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.595+0000 7efecad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563496cb59a0 0x563496cb5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.595+0000 7efeca59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563496cb62e0 0x563496cbada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.595+0000 7efec9d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563496cbb370 0x563496cbd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.595+0000 7efeca59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563496cb62e0 0x563496cbada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52182/0 (socket says 172.21.15.120:52182) 2024-09-19T09:14:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.595+0000 7efeca59c640 1 -- 172.21.15.120:0/1247160440 learned_addr learned my addr 172.21.15.120:0/1247160440 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.595+0000 7efeca59c640 1 -- 172.21.15.120:0/1247160440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563496cbb370 msgr2=0x563496cbd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.595+0000 7efeca59c640 1 --2- 172.21.15.120:0/1247160440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563496cbb370 0x563496cbd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.595+0000 7efebb7fe640 1 --2- 172.21.15.120:0/1913901734 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efeb40a3370 0x7efeb40a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.595+0000 7efeca59c640 1 -- 172.21.15.120:0/1247160440 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563496cb59a0 msgr2=0x563496cb5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.595+0000 7efeca59c640 1 --2- 172.21.15.120:0/1247160440 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563496cb59a0 0x563496cb5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.595+0000 7efeca59c640 1 -- 172.21.15.120:0/1247160440 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563496bc0340 con 0x563496cb62e0 2024-09-19T09:14:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.596+0000 7efec9d9b640 1 --2- 172.21.15.120:0/1247160440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563496cbb370 0x563496cbd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.596+0000 7efecad9d640 1 --2- 172.21.15.120:0/1913901734 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efeb40a3370 0x7efeb40a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.596+0000 7efebb7fe640 1 -- 172.21.15.120:0/1913901734 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7efec00cfd00 con 0x7efecc101c30 2024-09-19T09:14:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.596+0000 7efeca59c640 1 --2- 172.21.15.120:0/1247160440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563496cb62e0 0x563496cbada0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efebc009ca0 tx=0x7efebc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f97fa77f71a07b6a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.596+0000 7efeb8ff9640 1 -- 172.21.15.120:0/1247160440 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc004220 con 0x563496cb62e0 2024-09-19T09:14:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.596+0000 7efeb8ff9640 1 -- 172.21.15.120:0/1247160440 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efebc0043c0 con 0x563496cb62e0 2024-09-19T09:14:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.596+0000 7efeb8ff9640 1 -- 172.21.15.120:0/1247160440 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc0198b0 con 0x563496cb62e0 2024-09-19T09:14:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.597+0000 7efed3840740 1 -- 172.21.15.120:0/1247160440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563496cb62e0 msgr2=0x563496cbada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.597+0000 7efecad9d640 1 --2- 172.21.15.120:0/1913901734 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efeb40a3370 0x7efeb40a5830 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7efec4005e30 tx=0x7efec40095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.597+0000 7efed3840740 1 --2- 172.21.15.120:0/1247160440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563496cb62e0 0x563496cbada0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efebc009ca0 tx=0x7efebc011810 comp rx=0 tx=0).stop 2024-09-19T09:14:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.597+0000 7efed3840740 1 -- 172.21.15.120:0/1247160440 shutdown_connections 2024-09-19T09:14:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.597+0000 7efed3840740 1 --2- 172.21.15.120:0/1247160440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563496cbb370 0x563496cbd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.597+0000 7efed3840740 1 --2- 172.21.15.120:0/1247160440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563496cb62e0 0x563496cbada0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.597+0000 7efed3840740 1 --2- 172.21.15.120:0/1247160440 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563496cb59a0 0x563496cb5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.597+0000 7efed3840740 1 -- 172.21.15.120:0/1247160440 >> 172.21.15.120:0/1247160440 conn(0x563496cb1010 msgr2=0x563496cb1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.597+0000 7efed3840740 1 -- 172.21.15.120:0/1247160440 shutdown_connections 2024-09-19T09:14:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.597+0000 7efed3840740 1 -- 172.21.15.120:0/1247160440 wait complete. 2024-09-19T09:14:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.597+0000 7efed3840740 1 -- start start 2024-09-19T09:14:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.598+0000 7efed3840740 1 Processor -- start 2024-09-19T09:14:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.598+0000 7efed3840740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563496cb59a0 0x563496cbfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.598+0000 7efed3840740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563496cb62e0 0x563496cc0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.598+0000 7efed3840740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563496cbb370 0x563496cc7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.598+0000 7efed3840740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563496cbe340 con 0x563496cb59a0 2024-09-19T09:14:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.598+0000 7efed3840740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563496ba2290 con 0x563496cb62e0 2024-09-19T09:14:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.598+0000 7efed3840740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563496cbe4c0 con 0x563496cbb370 2024-09-19T09:14:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.598+0000 7efed3840740 10 client.0 _ll_register_callbacks cb 0x563496bd3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.598+0000 7efecad9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563496cbb370 0x563496cc7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.598+0000 7efeca59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563496cb59a0 0x563496cbfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.598+0000 7efecad9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563496cbb370 0x563496cc7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36574/0 (socket says 172.21.15.120:36574) 2024-09-19T09:14:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.598+0000 7efecad9d640 1 -- 172.21.15.120:0/3926208798 learned_addr learned my addr 172.21.15.120:0/3926208798 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.598+0000 7efec9d9b640 1 --2- 172.21.15.120:0/3926208798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563496cb62e0 0x563496cc0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.599+0000 7efecad9d640 1 -- 172.21.15.120:0/3926208798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563496cb62e0 msgr2=0x563496cc0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.599+0000 7efecad9d640 1 --2- 172.21.15.120:0/3926208798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563496cb62e0 0x563496cc0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.599+0000 7efecad9d640 1 -- 172.21.15.120:0/3926208798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563496cb59a0 msgr2=0x563496cbfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.599+0000 7efecad9d640 1 --2- 172.21.15.120:0/3926208798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563496cb59a0 0x563496cbfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.599+0000 7efecad9d640 1 -- 172.21.15.120:0/3926208798 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56349692c210 con 0x563496cbb370 2024-09-19T09:14:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.599+0000 7efeca59c640 1 --2- 172.21.15.120:0/3926208798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563496cb59a0 0x563496cbfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.599+0000 7efecad9d640 1 --2- 172.21.15.120:0/3926208798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563496cbb370 0x563496cc7140 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efec4009ca0 tx=0x7efec402a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19a672c396d792f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.599+0000 7efeb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:14:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.599+0000 7efeb1ffb640 1 -- 172.21.15.120:0/3926208798 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec40305c0 con 0x563496cbb370 2024-09-19T09:14:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.599+0000 7efeb1ffb640 1 -- 172.21.15.120:0/3926208798 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7efec4030760 con 0x563496cbb370 2024-09-19T09:14:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.599+0000 7efed3840740 1 -- 172.21.15.120:0/3926208798 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efebc0043c0 con 0x563496cbb370 2024-09-19T09:14:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.599+0000 7efeb1ffb640 1 -- 172.21.15.120:0/3926208798 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efec4030960 con 0x563496cbb370 2024-09-19T09:14:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.600+0000 7efed3840740 10 client.28390 fetch_fsmap learned FSMap version 32 2024-09-19T09:14:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.600+0000 7efed3840740 1 -- 172.21.15.120:0/3926208798 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5634969f8620 con 0x563496cbb370 2024-09-19T09:14:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.600+0000 7efeb1ffb640 1 -- 172.21.15.120:0/3926208798 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efebc0043c0 con 0x563496cbb370 2024-09-19T09:14:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.600+0000 7efeb1ffb640 1 -- 172.21.15.120:0/3926208798 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7efec4045070 con 0x563496cbb370 2024-09-19T09:14:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.600+0000 7efed3840740 10 client.28390 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:14:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.600+0000 7efed3840740 10 client.28390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:14:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.600+0000 7efed3840740 20 client.28390 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.600+0000 7efed3840740 1 --2- 172.21.15.120:0/3926208798 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x563496ccb830 0x563496ccdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.600+0000 7efed3840740 4 client.28390 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:14:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.600+0000 7efed3840740 1 -- 172.21.15.120:0/3926208798 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563496c26a20 con 0x563496ccb830 2024-09-19T09:14:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.600+0000 7efeca59c640 1 --2- 172.21.15.120:0/3926208798 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x563496ccb830 0x563496ccdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.601+0000 7efeb1ffb640 10 client.28390 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.601+0000 7efeca59c640 1 --2- 172.21.15.120:0/3926208798 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x563496ccb830 0x563496ccdc40 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90216af7c3683313 server_cookie=a81463eb85091e2c in_seq=0 out_seq=0 2024-09-19T09:14:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.602+0000 7efeb1ffb640 1 -- 172.21.15.120:0/3926208798 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563496c26a20 con 0x563496ccb830 2024-09-19T09:14:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.602+0000 7efeb1ffb640 10 client.28390 handle_command_reply: tid=0 2024-09-19T09:14:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.602+0000 7efed3840740 1 client.28390 shutdown 2024-09-19T09:14:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.602+0000 7efed3840740 1 -- 172.21.15.120:0/3926208798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563496cbb370 msgr2=0x563496cc7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.602+0000 7efed3840740 1 --2- 172.21.15.120:0/3926208798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563496cbb370 0x563496cc7140 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efec4009ca0 tx=0x7efec402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.602+0000 7efed3840740 1 -- 172.21.15.120:0/3926208798 shutdown_connections 2024-09-19T09:14:17.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.602+0000 7efed3840740 1 --2- 172.21.15.120:0/3926208798 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x563496ccb830 0x563496ccdc40 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.602+0000 7efed3840740 1 --2- 172.21.15.120:0/3926208798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563496cbb370 0x563496cc7140 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.602+0000 7efeb1ffb640 0 client.28390 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:17.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.602+0000 7efed3840740 1 --2- 172.21.15.120:0/3926208798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563496cb62e0 0x563496cc0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.602+0000 7efed3840740 1 --2- 172.21.15.120:0/3926208798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563496cb59a0 0x563496cbfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.602+0000 7efed3840740 1 -- 172.21.15.120:0/3926208798 >> 172.21.15.120:0/3926208798 conn(0x563496cb1010 msgr2=0x563496cb2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:17.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.602+0000 7efed3840740 1 -- 172.21.15.120:0/3926208798 shutdown_connections 2024-09-19T09:14:17.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.602+0000 7efed3840740 1 -- 172.21.15.120:0/3926208798 wait complete. 2024-09-19T09:14:17.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.602+0000 7efed3840740 20 client.28390 trim_cache size 0 max 16384 2024-09-19T09:14:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.622+0000 7efed3840740 1 Processor -- start 2024-09-19T09:14:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.622+0000 7efed3840740 1 -- start start 2024-09-19T09:14:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.622+0000 7efed3840740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563496cb59a0 0x563496cb1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.622+0000 7efed3840740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563496cb62e0 0x563496cc7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.622+0000 7efed3840740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563496cbb370 0x563496cb2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.622+0000 7efed3840740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efecc1ab260 con 0x563496cb59a0 2024-09-19T09:14:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.622+0000 7efed3840740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efecc1acda0 con 0x563496cbb370 2024-09-19T09:14:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.622+0000 7efed3840740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efecc1ab930 con 0x563496cb62e0 2024-09-19T09:14:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.623+0000 7efeca59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563496cb59a0 0x563496cb1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.623+0000 7efeca59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563496cb59a0 0x563496cb1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47822/0 (socket says 172.21.15.120:47822) 2024-09-19T09:14:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.623+0000 7efeca59c640 1 -- 172.21.15.120:0/4194838589 learned_addr learned my addr 172.21.15.120:0/4194838589 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.623+0000 7efec9d9b640 1 --2- 172.21.15.120:0/4194838589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563496cb62e0 0x563496cc7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.623+0000 7efeca59c640 1 -- 172.21.15.120:0/4194838589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563496cb62e0 msgr2=0x563496cc7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.623+0000 7efeca59c640 1 --2- 172.21.15.120:0/4194838589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563496cb62e0 0x563496cc7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.623+0000 7efeca59c640 1 -- 172.21.15.120:0/4194838589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563496cbb370 msgr2=0x563496cb2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.623+0000 7efeca59c640 1 --2- 172.21.15.120:0/4194838589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563496cbb370 0x563496cb2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.623+0000 7efeca59c640 1 -- 172.21.15.120:0/4194838589 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563496cca200 con 0x563496cb59a0 2024-09-19T09:14:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.623+0000 7efeca59c640 1 --2- 172.21.15.120:0/4194838589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563496cb59a0 0x563496cb1ec0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7efebc013010 tx=0x7efebc009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffbcd7ec8f1889f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.623+0000 7efeb3fff640 1 -- 172.21.15.120:0/4194838589 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc005410 con 0x563496cb59a0 2024-09-19T09:14:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.623+0000 7efeb3fff640 1 -- 172.21.15.120:0/4194838589 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efebc0055b0 con 0x563496cb59a0 2024-09-19T09:14:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.623+0000 7efeb3fff640 1 -- 172.21.15.120:0/4194838589 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc019840 con 0x563496cb59a0 2024-09-19T09:14:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.624+0000 7efed3840740 1 -- 172.21.15.120:0/4194838589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563496cb59a0 msgr2=0x563496cb1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.624+0000 7efed3840740 1 --2- 172.21.15.120:0/4194838589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563496cb59a0 0x563496cb1ec0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7efebc013010 tx=0x7efebc009f10 comp rx=0 tx=0).stop 2024-09-19T09:14:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.624+0000 7efed3840740 1 -- 172.21.15.120:0/4194838589 shutdown_connections 2024-09-19T09:14:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.624+0000 7efed3840740 1 --2- 172.21.15.120:0/4194838589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563496cbb370 0x563496cb2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.624+0000 7efed3840740 1 --2- 172.21.15.120:0/4194838589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563496cb62e0 0x563496cc7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.624+0000 7efed3840740 1 --2- 172.21.15.120:0/4194838589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563496cb59a0 0x563496cb1ec0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.624+0000 7efed3840740 1 -- 172.21.15.120:0/4194838589 >> 172.21.15.120:0/4194838589 conn(0x563496cb1010 msgr2=0x563496cbfbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.624+0000 7efed3840740 1 -- 172.21.15.120:0/4194838589 shutdown_connections 2024-09-19T09:14:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.624+0000 7efed3840740 1 -- 172.21.15.120:0/4194838589 wait complete. 2024-09-19T09:14:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.625+0000 7efed3840740 1 -- start start 2024-09-19T09:14:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.625+0000 7efed3840740 1 Processor -- start 2024-09-19T09:14:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.625+0000 7efed3840740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563496cb59a0 0x563496cc4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.625+0000 7efed3840740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563496cb62e0 0x563496cc4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.625+0000 7efed3840740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563496cbb370 0x563496cc5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.625+0000 7efed3840740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efecc1ac6d0 con 0x563496cb59a0 2024-09-19T09:14:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.625+0000 7efed3840740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efecc1ac000 con 0x563496cbb370 2024-09-19T09:14:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.625+0000 7efed3840740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efecc1a9220 con 0x563496cb62e0 2024-09-19T09:14:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.625+0000 7efed3840740 10 client.0 _ll_register_callbacks cb 0x7efec4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.625+0000 7efeca59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563496cb59a0 0x563496cc4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.625+0000 7efeca59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563496cb59a0 0x563496cc4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47832/0 (socket says 172.21.15.120:47832) 2024-09-19T09:14:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.625+0000 7efeca59c640 1 -- 172.21.15.120:0/3334314087 learned_addr learned my addr 172.21.15.120:0/3334314087 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.625+0000 7efecad9d640 1 --2- 172.21.15.120:0/3334314087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563496cbb370 0x563496cc5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.625+0000 7efeca59c640 1 -- 172.21.15.120:0/3334314087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563496cb62e0 msgr2=0x563496cc4750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:14:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.625+0000 7efeca59c640 1 --2- 172.21.15.120:0/3334314087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563496cb62e0 0x563496cc4750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.625+0000 7efeca59c640 1 -- 172.21.15.120:0/3334314087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563496cbb370 msgr2=0x563496cc5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.625+0000 7efeca59c640 1 --2- 172.21.15.120:0/3334314087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563496cbb370 0x563496cc5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.625+0000 7efeca59c640 1 -- 172.21.15.120:0/3334314087 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563496cc5d70 con 0x563496cb59a0 2024-09-19T09:14:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.625+0000 7efecad9d640 1 --2- 172.21.15.120:0/3334314087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563496cbb370 0x563496cc5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:14:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.626+0000 7efeca59c640 1 --2- 172.21.15.120:0/3334314087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563496cb59a0 0x563496cc4210 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7efebc006010 tx=0x7efebc011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91b37a3cf597a69 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.626+0000 7efeb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:14:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.626+0000 7efeb1ffb640 1 -- 172.21.15.120:0/3334314087 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc005410 con 0x563496cb59a0 2024-09-19T09:14:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.626+0000 7efed3840740 1 -- 172.21.15.120:0/3334314087 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efebc0055b0 con 0x563496cb59a0 2024-09-19T09:14:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.626+0000 7efeb1ffb640 1 -- 172.21.15.120:0/3334314087 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7efebc003880 con 0x563496cb59a0 2024-09-19T09:14:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.626+0000 7efeb1ffb640 1 -- 172.21.15.120:0/3334314087 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efebc003a80 con 0x563496cb59a0 2024-09-19T09:14:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.626+0000 7efed3840740 10 client.19539 fetch_fsmap learned FSMap version 32 2024-09-19T09:14:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.626+0000 7efed3840740 1 -- 172.21.15.120:0/3334314087 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563496cc7080 con 0x563496cb59a0 2024-09-19T09:14:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.627+0000 7efeb1ffb640 1 -- 172.21.15.120:0/3334314087 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efebc0055b0 con 0x563496cb59a0 2024-09-19T09:14:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.627+0000 7efeb1ffb640 1 -- 172.21.15.120:0/3334314087 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7efebc0369a0 con 0x563496cb59a0 2024-09-19T09:14:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.627+0000 7efed3840740 10 client.19539 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:14:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.627+0000 7efed3840740 10 client.19539 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:14:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.627+0000 7efed3840740 20 client.19539 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.627+0000 7efed3840740 1 --2- 172.21.15.120:0/3334314087 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x563496d36670 0x563496d38a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.627+0000 7efed3840740 4 client.19539 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:14:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.627+0000 7efed3840740 1 -- 172.21.15.120:0/3334314087 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563496cbb0c0 con 0x563496d36670 2024-09-19T09:14:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.627+0000 7efec9d9b640 1 --2- 172.21.15.120:0/3334314087 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x563496d36670 0x563496d38a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.628+0000 7efeb1ffb640 10 client.19539 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.628+0000 7efec9d9b640 1 --2- 172.21.15.120:0/3334314087 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x563496d36670 0x563496d38a60 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4718d99dc48bcf8d server_cookie=eba29b0fd25dc677 in_seq=0 out_seq=0 2024-09-19T09:14:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.834+0000 7efeb1ffb640 1 -- 172.21.15.120:0/3334314087 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563496cbb0c0 con 0x563496d36670 2024-09-19T09:14:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.834+0000 7efeb1ffb640 10 client.19539 handle_command_reply: tid=0 2024-09-19T09:14:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.834+0000 7efed3840740 1 client.19539 shutdown 2024-09-19T09:14:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.834+0000 7efed3840740 1 -- 172.21.15.120:0/3334314087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563496cb59a0 msgr2=0x563496cc4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.834+0000 7efed3840740 1 --2- 172.21.15.120:0/3334314087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563496cb59a0 0x563496cc4210 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7efebc006010 tx=0x7efebc011e50 comp rx=0 tx=0).stop 2024-09-19T09:14:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:14:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:14:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:14:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:14:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.834+0000 7efed3840740 1 -- 172.21.15.120:0/3334314087 shutdown_connections 2024-09-19T09:14:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.834+0000 7efed3840740 1 --2- 172.21.15.120:0/3334314087 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x563496d36670 0x563496d38a60 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.834+0000 7efed3840740 1 --2- 172.21.15.120:0/3334314087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563496cbb370 0x563496cc5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.834+0000 7efeb1ffb640 0 client.19539 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.834+0000 7efed3840740 1 --2- 172.21.15.120:0/3334314087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563496cb62e0 0x563496cc4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.834+0000 7efed3840740 1 --2- 172.21.15.120:0/3334314087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563496cb59a0 0x563496cc4210 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.834+0000 7efed3840740 1 -- 172.21.15.120:0/3334314087 >> 172.21.15.120:0/3334314087 conn(0x563496cb1010 msgr2=0x563496caf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.834+0000 7efed3840740 1 -- 172.21.15.120:0/3334314087 shutdown_connections 2024-09-19T09:14:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.834+0000 7efed3840740 1 -- 172.21.15.120:0/3334314087 wait complete. 2024-09-19T09:14:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.835+0000 7efed3840740 20 client.19539 trim_cache size 0 max 16384 2024-09-19T09:14:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.835+0000 7efed09c4640 1 -- 172.21.15.120:0/1913901734 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efeb40a3370 msgr2=0x7efeb40a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.835+0000 7efed09c4640 1 --2- 172.21.15.120:0/1913901734 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efeb40a3370 0x7efeb40a5830 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7efec4005e30 tx=0x7efec40095c0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.835+0000 7efed09c4640 1 -- 172.21.15.120:0/1913901734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efecc101c30 msgr2=0x7efecc19ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.835+0000 7efed09c4640 1 --2- 172.21.15.120:0/1913901734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efecc101c30 0x7efecc19ee60 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7efec000e9f0 tx=0x7efec000eeb0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.835+0000 7efed09c4640 1 -- 172.21.15.120:0/1913901734 shutdown_connections 2024-09-19T09:14:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.835+0000 7efed09c4640 1 --2- 172.21.15.120:0/1913901734 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efeb40a3370 0x7efeb40a5830 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.835+0000 7efed09c4640 1 --2- 172.21.15.120:0/1913901734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efecc1025f0 0x7efecc1a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.835+0000 7efed09c4640 1 --2- 172.21.15.120:0/1913901734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efecc101c30 0x7efecc19ee60 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.835+0000 7efed09c4640 1 --2- 172.21.15.120:0/1913901734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efecc100a40 0x7efecc06a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.835+0000 7efed09c4640 1 -- 172.21.15.120:0/1913901734 >> 172.21.15.120:0/1913901734 conn(0x7efecc0fc1e0 msgr2=0x7efecc104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.835+0000 7efed09c4640 1 -- 172.21.15.120:0/1913901734 shutdown_connections 2024-09-19T09:14:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:17.835+0000 7efed09c4640 1 -- 172.21.15.120:0/1913901734 wait complete. 2024-09-19T09:14:18.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.000+0000 7ff440c12640 1 -- 172.21.15.120:0/2921852727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff43c102ef0 msgr2=0x7ff43c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:18.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.000+0000 7ff440c12640 1 --2- 172.21.15.120:0/2921852727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff43c102ef0 0x7ff43c10da40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff434009c10 tx=0x7ff43402a770 comp rx=0 tx=0).stop 2024-09-19T09:14:18.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.000+0000 7ff440c12640 1 -- 172.21.15.120:0/2921852727 shutdown_connections 2024-09-19T09:14:18.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.000+0000 7ff440c12640 1 --2- 172.21.15.120:0/2921852727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff43c102ef0 0x7ff43c10da40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.000+0000 7ff440c12640 1 --2- 172.21.15.120:0/2921852727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff43c102570 0x7ff43c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.000+0000 7ff440c12640 1 --2- 172.21.15.120:0/2921852727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff43c100e20 0x7ff43c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.000+0000 7ff440c12640 1 -- 172.21.15.120:0/2921852727 >> 172.21.15.120:0/2921852727 conn(0x7ff43c0fc7d0 msgr2=0x7ff43c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:18.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.000+0000 7ff440c12640 1 -- 172.21.15.120:0/2921852727 shutdown_connections 2024-09-19T09:14:18.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.000+0000 7ff440c12640 1 -- 172.21.15.120:0/2921852727 wait complete. 2024-09-19T09:14:18.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.001+0000 7ff440c12640 1 Processor -- start 2024-09-19T09:14:18.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.001+0000 7ff440c12640 1 -- start start 2024-09-19T09:14:18.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.001+0000 7ff440c12640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff43c100e20 0x7ff43c1973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:18.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.001+0000 7ff440c12640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff43c102570 0x7ff43c197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:18.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.001+0000 7ff440c12640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff43c102ef0 0x7ff43c19bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:18.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.002+0000 7ff440c12640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c110f30 con 0x7ff43c102ef0 2024-09-19T09:14:18.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.002+0000 7ff440c12640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c110db0 con 0x7ff43c102570 2024-09-19T09:14:18.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.002+0000 7ff440c12640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1110b0 con 0x7ff43c100e20 2024-09-19T09:14:18.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.002+0000 7ff43a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff43c100e20 0x7ff43c1973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:18.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.002+0000 7ff43ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff43c102ef0 0x7ff43c19bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:18.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.002+0000 7ff43a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff43c100e20 0x7ff43c1973f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36594/0 (socket says 172.21.15.120:36594) 2024-09-19T09:14:18.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.002+0000 7ff43a59c640 1 -- 172.21.15.120:0/1888954361 learned_addr learned my addr 172.21.15.120:0/1888954361 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:18.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.002+0000 7ff433fff640 1 --2- 172.21.15.120:0/1888954361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff43c102570 0x7ff43c197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:18.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.002+0000 7ff43ad9d640 1 -- 172.21.15.120:0/1888954361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff43c100e20 msgr2=0x7ff43c1973f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:18.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.002+0000 7ff43ad9d640 1 --2- 172.21.15.120:0/1888954361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff43c100e20 0x7ff43c1973f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.002+0000 7ff43ad9d640 1 -- 172.21.15.120:0/1888954361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff43c102570 msgr2=0x7ff43c197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:18.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.002+0000 7ff43ad9d640 1 --2- 172.21.15.120:0/1888954361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff43c102570 0x7ff43c197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.002+0000 7ff43ad9d640 1 -- 172.21.15.120:0/1888954361 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43c19c3a0 con 0x7ff43c102ef0 2024-09-19T09:14:18.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.002+0000 7ff433fff640 1 --2- 172.21.15.120:0/1888954361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff43c102570 0x7ff43c197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:14:18.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.002+0000 7ff43ad9d640 1 --2- 172.21.15.120:0/1888954361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff43c102ef0 0x7ff43c19bcc0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7ff434034040 tx=0x7ff4340305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:18.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.002+0000 7ff4337fe640 1 -- 172.21.15.120:0/1888954361 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff434041020 con 0x7ff43c102ef0 2024-09-19T09:14:18.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.002+0000 7ff440c12640 1 -- 172.21.15.120:0/1888954361 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff43c19c630 con 0x7ff43c102ef0 2024-09-19T09:14:18.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.002+0000 7ff4337fe640 1 -- 172.21.15.120:0/1888954361 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff434037040 con 0x7ff43c102ef0 2024-09-19T09:14:18.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.002+0000 7ff4337fe640 1 -- 172.21.15.120:0/1888954361 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff434036070 con 0x7ff43c102ef0 2024-09-19T09:14:18.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.003+0000 7ff440c12640 1 -- 172.21.15.120:0/1888954361 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff43c1a3f70 con 0x7ff43c102ef0 2024-09-19T09:14:18.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.005+0000 7ff442feb740 1 Processor -- start 2024-09-19T09:14:18.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.006+0000 7ff4337fe640 1 -- 172.21.15.120:0/1888954361 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7ff434044020 con 0x7ff43c102ef0 2024-09-19T09:14:18.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.006+0000 7ff442feb740 1 -- start start 2024-09-19T09:14:18.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.006+0000 7ff442feb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a128291a80 0x55a128291e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:18.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.006+0000 7ff442feb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1282923a0 0x55a12829ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:18.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.006+0000 7ff442feb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a12829ef50 0x55a1282a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:18.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.006+0000 7ff442feb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a1281f7280 con 0x55a12829ef50 2024-09-19T09:14:18.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.006+0000 7ff442feb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a12819eb10 con 0x55a128291a80 2024-09-19T09:14:18.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.006+0000 7ff442feb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a1281f3240 con 0x55a1282923a0 2024-09-19T09:14:18.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.006+0000 7ff43a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1282923a0 0x55a12829ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:18.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.006+0000 7ff433fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a12829ef50 0x55a1282a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:18.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.006+0000 7ff43a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1282923a0 0x55a12829ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36598/0 (socket says 172.21.15.120:36598) 2024-09-19T09:14:18.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.006+0000 7ff43a59c640 1 -- 172.21.15.120:0/1701312067 learned_addr learned my addr 172.21.15.120:0/1701312067 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:18.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.006+0000 7ff433fff640 1 --2- 172.21.15.120:0/1701312067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a128291a80 0x55a128291e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:18.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.007+0000 7ff4337fe640 1 --2- 172.21.15.120:0/1888954361 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff4280a3240 0x7ff4280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:18.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.007+0000 7ff43a59c640 1 -- 172.21.15.120:0/1701312067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a128291a80 msgr2=0x55a128291e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:18.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.007+0000 7ff43a59c640 1 --2- 172.21.15.120:0/1701312067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a128291a80 0x55a128291e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.007+0000 7ff43a59c640 1 -- 172.21.15.120:0/1701312067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a12829ef50 msgr2=0x55a1282a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:18.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.007+0000 7ff43a59c640 1 --2- 172.21.15.120:0/1701312067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a12829ef50 0x55a1282a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.007+0000 7ff43a59c640 1 -- 172.21.15.120:0/1701312067 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1281ab1f0 con 0x55a1282923a0 2024-09-19T09:14:18.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.007+0000 7ff43ad9d640 1 --2- 172.21.15.120:0/1888954361 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff4280a3240 0x7ff4280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:18.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.007+0000 7ff43a59c640 1 --2- 172.21.15.120:0/1701312067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1282923a0 0x55a12829ea10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009dc0 tx=0x7ff42c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dcbdf747049a294 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:18.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.007+0000 7ff4337fe640 1 -- 172.21.15.120:0/1888954361 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ff4340f0260 con 0x7ff43c102ef0 2024-09-19T09:14:18.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.007+0000 7ff430ff9640 1 -- 172.21.15.120:0/1701312067 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c005220 con 0x55a1282923a0 2024-09-19T09:14:18.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.007+0000 7ff430ff9640 1 -- 172.21.15.120:0/1701312067 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff42c004c80 con 0x55a1282923a0 2024-09-19T09:14:18.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.007+0000 7ff430ff9640 1 -- 172.21.15.120:0/1701312067 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c019840 con 0x55a1282923a0 2024-09-19T09:14:18.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.008+0000 7ff442feb740 1 -- 172.21.15.120:0/1701312067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1282923a0 msgr2=0x55a12829ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:18.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.008+0000 7ff43ad9d640 1 --2- 172.21.15.120:0/1888954361 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff4280a3240 0x7ff4280a5700 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff43403f000 tx=0x7ff4340c0040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:18.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.008+0000 7ff442feb740 1 --2- 172.21.15.120:0/1701312067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1282923a0 0x55a12829ea10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009dc0 tx=0x7ff42c011810 comp rx=0 tx=0).stop 2024-09-19T09:14:18.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.008+0000 7ff442feb740 1 -- 172.21.15.120:0/1701312067 shutdown_connections 2024-09-19T09:14:18.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.008+0000 7ff442feb740 1 --2- 172.21.15.120:0/1701312067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a12829ef50 0x55a1282a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.008+0000 7ff442feb740 1 --2- 172.21.15.120:0/1701312067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1282923a0 0x55a12829ea10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.008+0000 7ff442feb740 1 --2- 172.21.15.120:0/1701312067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a128291a80 0x55a128291e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.008+0000 7ff442feb740 1 -- 172.21.15.120:0/1701312067 >> 172.21.15.120:0/1701312067 conn(0x55a12828e0f0 msgr2=0x55a12828e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:18.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.008+0000 7ff442feb740 1 -- 172.21.15.120:0/1701312067 shutdown_connections 2024-09-19T09:14:18.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.008+0000 7ff442feb740 1 -- 172.21.15.120:0/1701312067 wait complete. 2024-09-19T09:14:18.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.008+0000 7ff442feb740 1 -- start start 2024-09-19T09:14:18.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.008+0000 7ff442feb740 1 Processor -- start 2024-09-19T09:14:18.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.009+0000 7ff442feb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a128291a80 0x55a12829a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:18.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.009+0000 7ff442feb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1282923a0 0x55a12829aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:18.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.009+0000 7ff442feb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a12829ef50 0x55a1282accd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:18.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.009+0000 7ff442feb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a1282949e0 con 0x55a1282923a0 2024-09-19T09:14:18.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.009+0000 7ff442feb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a1281b1f10 con 0x55a12829ef50 2024-09-19T09:14:18.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.009+0000 7ff442feb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a128294b60 con 0x55a128291a80 2024-09-19T09:14:18.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.009+0000 7ff442feb740 10 client.0 _ll_register_callbacks cb 0x55a12817aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:18.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.009+0000 7ff43a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1282923a0 0x55a12829aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:18.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.009+0000 7ff43a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1282923a0 0x55a12829aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47860/0 (socket says 172.21.15.120:47860) 2024-09-19T09:14:18.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.009+0000 7ff43a59c640 1 -- 172.21.15.120:0/2116566888 learned_addr learned my addr 172.21.15.120:0/2116566888 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:18.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.009+0000 7ff433fff640 1 --2- 172.21.15.120:0/2116566888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a128291a80 0x55a12829a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:18.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.009+0000 7ff433fff640 1 --2- 172.21.15.120:0/2116566888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a12829ef50 0x55a1282accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:18.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.009+0000 7ff43a59c640 1 -- 172.21.15.120:0/2116566888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a128291a80 msgr2=0x55a12829a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:18.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.009+0000 7ff43a59c640 1 --2- 172.21.15.120:0/2116566888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a128291a80 0x55a12829a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.009+0000 7ff43a59c640 1 -- 172.21.15.120:0/2116566888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a12829ef50 msgr2=0x55a1282accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:18.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.009+0000 7ff43a59c640 1 --2- 172.21.15.120:0/2116566888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a12829ef50 0x55a1282accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.009+0000 7ff43a59c640 1 -- 172.21.15.120:0/2116566888 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a127f686a0 con 0x55a1282923a0 2024-09-19T09:14:18.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.009+0000 7ff43a59c640 1 --2- 172.21.15.120:0/2116566888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1282923a0 0x55a12829aa10 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009730 tx=0x7ff42c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62efd1e3d1bbc3fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:18.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.009+0000 7ff421ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:14:18.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.009+0000 7ff421ffb640 1 -- 172.21.15.120:0/2116566888 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c027070 con 0x55a1282923a0 2024-09-19T09:14:18.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.009+0000 7ff421ffb640 1 -- 172.21.15.120:0/2116566888 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ff42c005240 con 0x55a1282923a0 2024-09-19T09:14:18.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.009+0000 7ff442feb740 1 -- 172.21.15.120:0/2116566888 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff42c004c80 con 0x55a1282923a0 2024-09-19T09:14:18.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.010+0000 7ff421ffb640 1 -- 172.21.15.120:0/2116566888 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff42c033050 con 0x55a1282923a0 2024-09-19T09:14:18.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.010+0000 7ff442feb740 10 client.19551 fetch_fsmap learned FSMap version 32 2024-09-19T09:14:18.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.010+0000 7ff442feb740 1 -- 172.21.15.120:0/2116566888 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a1282af950 con 0x55a1282923a0 2024-09-19T09:14:18.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.010+0000 7ff421ffb640 1 -- 172.21.15.120:0/2116566888 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff42c004c80 con 0x55a1282923a0 2024-09-19T09:14:18.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.010+0000 7ff421ffb640 1 -- 172.21.15.120:0/2116566888 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff42c003bf0 con 0x55a1282923a0 2024-09-19T09:14:18.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.010+0000 7ff442feb740 10 client.19551 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:14:18.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.010+0000 7ff442feb740 10 client.19551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:14:18.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.010+0000 7ff442feb740 20 client.19551 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:18.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.010+0000 7ff442feb740 1 --2- 172.21.15.120:0/2116566888 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a1282b0d60 0x55a1282b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:18.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.010+0000 7ff442feb740 4 client.19551 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:14:18.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.010+0000 7ff442feb740 1 -- 172.21.15.120:0/2116566888 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1282047f0 con 0x55a1282b0d60 2024-09-19T09:14:18.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.010+0000 7ff433fff640 1 --2- 172.21.15.120:0/2116566888 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a1282b0d60 0x55a1282b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:18.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.011+0000 7ff421ffb640 10 client.19551 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:18.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.011+0000 7ff433fff640 1 --2- 172.21.15.120:0/2116566888 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a1282b0d60 0x55a1282b3150 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6724122f45bd00a0 server_cookie=1b727c3604a7e76a in_seq=0 out_seq=0 2024-09-19T09:14:18.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.011+0000 7ff421ffb640 1 -- 172.21.15.120:0/2116566888 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a1282047f0 con 0x55a1282b0d60 2024-09-19T09:14:18.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.011+0000 7ff421ffb640 10 client.19551 handle_command_reply: tid=0 2024-09-19T09:14:18.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.012+0000 7ff442feb740 1 client.19551 shutdown 2024-09-19T09:14:18.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.012+0000 7ff442feb740 1 -- 172.21.15.120:0/2116566888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1282923a0 msgr2=0x55a12829aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:18.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.012+0000 7ff442feb740 1 --2- 172.21.15.120:0/2116566888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1282923a0 0x55a12829aa10 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009730 tx=0x7ff42c0042f0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.012+0000 7ff442feb740 1 -- 172.21.15.120:0/2116566888 shutdown_connections 2024-09-19T09:14:18.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.012+0000 7ff442feb740 1 --2- 172.21.15.120:0/2116566888 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a1282b0d60 0x55a1282b3150 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.012+0000 7ff442feb740 1 --2- 172.21.15.120:0/2116566888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a12829ef50 0x55a1282accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.012+0000 7ff421ffb640 0 client.19551 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:18.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.012+0000 7ff442feb740 1 --2- 172.21.15.120:0/2116566888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1282923a0 0x55a12829aa10 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.012+0000 7ff442feb740 1 --2- 172.21.15.120:0/2116566888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a128291a80 0x55a12829a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.012+0000 7ff442feb740 1 -- 172.21.15.120:0/2116566888 >> 172.21.15.120:0/2116566888 conn(0x55a12828e0f0 msgr2=0x55a12828d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:18.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.012+0000 7ff442feb740 1 -- 172.21.15.120:0/2116566888 shutdown_connections 2024-09-19T09:14:18.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.012+0000 7ff442feb740 1 -- 172.21.15.120:0/2116566888 wait complete. 2024-09-19T09:14:18.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.012+0000 7ff442feb740 20 client.19551 trim_cache size 0 max 16384 2024-09-19T09:14:18.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.030+0000 7ff442feb740 1 Processor -- start 2024-09-19T09:14:18.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.030+0000 7ff442feb740 1 -- start start 2024-09-19T09:14:18.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.030+0000 7ff442feb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a128291a80 0x55a12828d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:18.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.030+0000 7ff442feb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1282923a0 0x55a1282b8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:18.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.031+0000 7ff442feb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a12829ef50 0x55a1282b8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:18.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.031+0000 7ff442feb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1ab3c0 con 0x55a12829ef50 2024-09-19T09:14:18.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.031+0000 7ff442feb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1acf00 con 0x55a1282923a0 2024-09-19T09:14:18.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.031+0000 7ff442feb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1aba90 con 0x55a128291a80 2024-09-19T09:14:18.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.031+0000 7ff43a59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1282923a0 0x55a1282b8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:18.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.031+0000 7ff433fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a128291a80 0x55a12828d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:18.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.031+0000 7ff43a59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1282923a0 0x55a1282b8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52240/0 (socket says 172.21.15.120:52240) 2024-09-19T09:14:18.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.031+0000 7ff43a59c640 1 -- 172.21.15.120:0/1811119348 learned_addr learned my addr 172.21.15.120:0/1811119348 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:18.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.031+0000 7ff433fff640 1 --2- 172.21.15.120:0/1811119348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a12829ef50 0x55a1282b8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:18.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.031+0000 7ff43a59c640 1 -- 172.21.15.120:0/1811119348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a128291a80 msgr2=0x55a12828d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:18.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.031+0000 7ff43a59c640 1 --2- 172.21.15.120:0/1811119348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a128291a80 0x55a12828d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.031+0000 7ff43a59c640 1 -- 172.21.15.120:0/1811119348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a12829ef50 msgr2=0x55a1282b8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:18.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.031+0000 7ff43a59c640 1 --2- 172.21.15.120:0/1811119348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a12829ef50 0x55a1282b8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.031+0000 7ff43a59c640 1 -- 172.21.15.120:0/1811119348 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1282b36f0 con 0x55a1282923a0 2024-09-19T09:14:18.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.031+0000 7ff43a59c640 1 --2- 172.21.15.120:0/1811119348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1282923a0 0x55a1282b8980 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009700 tx=0x7ff42c009760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed2799ebd13a079e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:18.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.031+0000 7ff423fff640 1 -- 172.21.15.120:0/1811119348 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c027040 con 0x55a1282923a0 2024-09-19T09:14:18.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.031+0000 7ff423fff640 1 -- 172.21.15.120:0/1811119348 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff42c0099d0 con 0x55a1282923a0 2024-09-19T09:14:18.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.031+0000 7ff423fff640 1 -- 172.21.15.120:0/1811119348 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c019be0 con 0x55a1282923a0 2024-09-19T09:14:18.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.032+0000 7ff442feb740 1 -- 172.21.15.120:0/1811119348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1282923a0 msgr2=0x55a1282b8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:18.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.032+0000 7ff442feb740 1 --2- 172.21.15.120:0/1811119348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1282923a0 0x55a1282b8980 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009700 tx=0x7ff42c009760 comp rx=0 tx=0).stop 2024-09-19T09:14:18.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.032+0000 7ff442feb740 1 -- 172.21.15.120:0/1811119348 shutdown_connections 2024-09-19T09:14:18.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.032+0000 7ff442feb740 1 --2- 172.21.15.120:0/1811119348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a12829ef50 0x55a1282b8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.032+0000 7ff442feb740 1 --2- 172.21.15.120:0/1811119348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1282923a0 0x55a1282b8980 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.032+0000 7ff442feb740 1 --2- 172.21.15.120:0/1811119348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a128291a80 0x55a12828d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.032+0000 7ff442feb740 1 -- 172.21.15.120:0/1811119348 >> 172.21.15.120:0/1811119348 conn(0x55a12828e0f0 msgr2=0x55a12828ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:18.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.032+0000 7ff442feb740 1 -- 172.21.15.120:0/1811119348 shutdown_connections 2024-09-19T09:14:18.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.032+0000 7ff442feb740 1 -- 172.21.15.120:0/1811119348 wait complete. 2024-09-19T09:14:18.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.033+0000 7ff442feb740 1 -- start start 2024-09-19T09:14:18.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.033+0000 7ff442feb740 1 Processor -- start 2024-09-19T09:14:18.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.033+0000 7ff442feb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a128291a80 0x55a1282ad460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:18.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.033+0000 7ff442feb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1282923a0 0x55a1282a8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:18.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.033+0000 7ff442feb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a12829ef50 0x55a1282a8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:18.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.033+0000 7ff442feb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1ac830 con 0x55a1282923a0 2024-09-19T09:14:18.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.033+0000 7ff442feb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1ac160 con 0x55a128291a80 2024-09-19T09:14:18.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.033+0000 7ff442feb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1a9380 con 0x55a12829ef50 2024-09-19T09:14:18.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.033+0000 7ff442feb740 10 client.0 _ll_register_callbacks cb 0x7ff424009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:18.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.033+0000 7ff433fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a128291a80 0x55a1282ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:18.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.033+0000 7ff433fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a128291a80 0x55a1282ad460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52250/0 (socket says 172.21.15.120:52250) 2024-09-19T09:14:18.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.033+0000 7ff433fff640 1 -- 172.21.15.120:0/2507272989 learned_addr learned my addr 172.21.15.120:0/2507272989 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:18.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.033+0000 7ff433fff640 1 --2- 172.21.15.120:0/2507272989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a12829ef50 0x55a1282a8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:18.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.033+0000 7ff43a59c640 1 --2- 172.21.15.120:0/2507272989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1282923a0 0x55a1282a8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:18.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.034+0000 7ff433fff640 1 -- 172.21.15.120:0/2507272989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a12829ef50 msgr2=0x55a1282a8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:18.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.034+0000 7ff433fff640 1 --2- 172.21.15.120:0/2507272989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a12829ef50 0x55a1282a8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.034+0000 7ff433fff640 1 -- 172.21.15.120:0/2507272989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1282923a0 msgr2=0x55a1282a8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:18.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.034+0000 7ff433fff640 1 --2- 172.21.15.120:0/2507272989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1282923a0 0x55a1282a8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.034+0000 7ff433fff640 1 -- 172.21.15.120:0/2507272989 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a12831a950 con 0x55a128291a80 2024-09-19T09:14:18.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.034+0000 7ff433fff640 1 --2- 172.21.15.120:0/2507272989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a128291a80 0x55a1282ad460 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff424009a40 tx=0x7ff4240046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b35fd16be1df9267 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:18.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.034+0000 7ff421ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:14:18.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.034+0000 7ff421ffb640 1 -- 172.21.15.120:0/2507272989 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424013d50 con 0x55a128291a80 2024-09-19T09:14:18.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.034+0000 7ff442feb740 1 -- 172.21.15.120:0/2507272989 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff42c0099d0 con 0x55a128291a80 2024-09-19T09:14:18.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.034+0000 7ff421ffb640 1 -- 172.21.15.120:0/2507272989 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ff424005050 con 0x55a128291a80 2024-09-19T09:14:18.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.034+0000 7ff421ffb640 1 -- 172.21.15.120:0/2507272989 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff424005250 con 0x55a128291a80 2024-09-19T09:14:18.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.034+0000 7ff442feb740 10 client.28607 fetch_fsmap learned FSMap version 32 2024-09-19T09:14:18.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.034+0000 7ff442feb740 1 -- 172.21.15.120:0/2507272989 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a12831baf0 con 0x55a128291a80 2024-09-19T09:14:18.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.034+0000 7ff421ffb640 1 -- 172.21.15.120:0/2507272989 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff42c0099d0 con 0x55a128291a80 2024-09-19T09:14:18.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.034+0000 7ff421ffb640 1 -- 172.21.15.120:0/2507272989 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff424024450 con 0x55a128291a80 2024-09-19T09:14:18.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.035+0000 7ff442feb740 10 client.28607 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:14:18.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.035+0000 7ff442feb740 10 client.28607 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:14:18.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.035+0000 7ff442feb740 20 client.28607 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:18.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.035+0000 7ff442feb740 1 --2- 172.21.15.120:0/2507272989 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a1282b0d60 0x55a12831e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:18.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.035+0000 7ff442feb740 4 client.28607 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:14:18.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.035+0000 7ff442feb740 1 -- 172.21.15.120:0/2507272989 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a1281fed00 con 0x55a1282b0d60 2024-09-19T09:14:18.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.035+0000 7ff43a59c640 1 --2- 172.21.15.120:0/2507272989 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a1282b0d60 0x55a12831e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:18.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.035+0000 7ff43a59c640 1 --2- 172.21.15.120:0/2507272989 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a1282b0d60 0x55a12831e230 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=799e488aa6e1de2d server_cookie=3f32a817b0a9a310 in_seq=0 out_seq=0 2024-09-19T09:14:18.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.035+0000 7ff421ffb640 10 client.28607 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:18.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.037+0000 7ff421ffb640 1 -- 172.21.15.120:0/2507272989 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a1281fed00 con 0x55a1282b0d60 2024-09-19T09:14:18.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.037+0000 7ff421ffb640 10 client.28607 handle_command_reply: tid=0 2024-09-19T09:14:18.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.037+0000 7ff442feb740 1 client.28607 shutdown 2024-09-19T09:14:18.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.037+0000 7ff442feb740 1 -- 172.21.15.120:0/2507272989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a128291a80 msgr2=0x55a1282ad460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:18.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.037+0000 7ff442feb740 1 --2- 172.21.15.120:0/2507272989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a128291a80 0x55a1282ad460 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff424009a40 tx=0x7ff4240046c0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.037+0000 7ff442feb740 1 -- 172.21.15.120:0/2507272989 shutdown_connections 2024-09-19T09:14:18.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.037+0000 7ff442feb740 1 --2- 172.21.15.120:0/2507272989 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a1282b0d60 0x55a12831e230 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.037+0000 7ff442feb740 1 --2- 172.21.15.120:0/2507272989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a12829ef50 0x55a1282a8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.037+0000 7ff442feb740 1 --2- 172.21.15.120:0/2507272989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1282923a0 0x55a1282a8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.037+0000 7ff442feb740 1 --2- 172.21.15.120:0/2507272989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a128291a80 0x55a1282ad460 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.037+0000 7ff421ffb640 0 client.28607 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:18.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.037+0000 7ff442feb740 1 -- 172.21.15.120:0/2507272989 >> 172.21.15.120:0/2507272989 conn(0x55a12828e0f0 msgr2=0x55a12828d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:18.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.037+0000 7ff442feb740 1 -- 172.21.15.120:0/2507272989 shutdown_connections 2024-09-19T09:14:18.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.037+0000 7ff442feb740 1 -- 172.21.15.120:0/2507272989 wait complete. 2024-09-19T09:14:18.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.038+0000 7ff442feb740 20 client.28607 trim_cache size 0 max 16384 2024-09-19T09:14:18.041 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:14:18.041 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:14:18.041 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "ce8451b8-4827-4de9-8c72-79cbeb7dde0e", 2024-09-19T09:14:18.041 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:14:18.041 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:14:18.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.038+0000 7ff440c12640 1 -- 172.21.15.120:0/1888954361 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff4280a3240 msgr2=0x7ff4280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:18.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.038+0000 7ff440c12640 1 --2- 172.21.15.120:0/1888954361 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff4280a3240 0x7ff4280a5700 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff43403f000 tx=0x7ff4340c0040 comp rx=0 tx=0).stop 2024-09-19T09:14:18.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.038+0000 7ff440c12640 1 -- 172.21.15.120:0/1888954361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff43c102ef0 msgr2=0x7ff43c19bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:18.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.038+0000 7ff440c12640 1 --2- 172.21.15.120:0/1888954361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff43c102ef0 0x7ff43c19bcc0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7ff434034040 tx=0x7ff4340305d0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.038+0000 7ff440c12640 1 -- 172.21.15.120:0/1888954361 shutdown_connections 2024-09-19T09:14:18.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.038+0000 7ff440c12640 1 --2- 172.21.15.120:0/1888954361 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff4280a3240 0x7ff4280a5700 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.038+0000 7ff440c12640 1 --2- 172.21.15.120:0/1888954361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff43c102ef0 0x7ff43c19bcc0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.038+0000 7ff440c12640 1 --2- 172.21.15.120:0/1888954361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff43c102570 0x7ff43c197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.038+0000 7ff440c12640 1 --2- 172.21.15.120:0/1888954361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff43c100e20 0x7ff43c1973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:18.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.038+0000 7ff440c12640 1 -- 172.21.15.120:0/1888954361 >> 172.21.15.120:0/1888954361 conn(0x7ff43c0fc7d0 msgr2=0x7ff43c104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:18.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.038+0000 7ff440c12640 1 -- 172.21.15.120:0/1888954361 shutdown_connections 2024-09-19T09:14:18.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:18.038+0000 7ff440c12640 1 -- 172.21.15.120:0/1888954361 wait complete. 2024-09-19T09:14:18.073 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1077482 9.36 MB/sec execute 493 sec latency 34.056 ms 2024-09-19T09:14:18.873 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:14:19.073 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1080358 9.38 MB/sec execute 494 sec latency 9.113 ms 2024-09-19T09:14:19.167 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:14:19.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:18 smithi196 ceph-mon[35150]: pgmap v686: 97 pgs: 97 active+clean; 49 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.0 MiB/s wr, 365 op/s 2024-09-19T09:14:19.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:18 smithi153 ceph-mon[35071]: pgmap v686: 97 pgs: 97 active+clean; 49 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.0 MiB/s wr, 365 op/s 2024-09-19T09:14:19.289 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:18 smithi120 ceph-mon[31150]: pgmap v686: 97 pgs: 97 active+clean; 49 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.0 MiB/s wr, 365 op/s 2024-09-19T09:14:19.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.900+0000 7fcfe646a640 1 -- 172.21.15.120:0/2145086737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcfe0102ef0 msgr2=0x7fcfe010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:19.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.900+0000 7fcfe646a640 1 --2- 172.21.15.120:0/2145086737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcfe0102ef0 0x7fcfe010da40 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8009ca0 tx=0x7fcfd802a950 comp rx=0 tx=0).stop 2024-09-19T09:14:19.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.900+0000 7fcfe646a640 1 -- 172.21.15.120:0/2145086737 shutdown_connections 2024-09-19T09:14:19.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.900+0000 7fcfe646a640 1 --2- 172.21.15.120:0/2145086737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcfe0102ef0 0x7fcfe010da40 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.900+0000 7fcfe646a640 1 --2- 172.21.15.120:0/2145086737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfe0102570 0x7fcfe01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.900+0000 7fcfe646a640 1 --2- 172.21.15.120:0/2145086737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcfe0100e20 0x7fcfe0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.900+0000 7fcfe646a640 1 -- 172.21.15.120:0/2145086737 >> 172.21.15.120:0/2145086737 conn(0x7fcfe00fc7d0 msgr2=0x7fcfe00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:19.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.900+0000 7fcfe646a640 1 -- 172.21.15.120:0/2145086737 shutdown_connections 2024-09-19T09:14:19.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.900+0000 7fcfe646a640 1 -- 172.21.15.120:0/2145086737 wait complete. 2024-09-19T09:14:19.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.901+0000 7fcfe646a640 1 Processor -- start 2024-09-19T09:14:19.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.901+0000 7fcfe646a640 1 -- start start 2024-09-19T09:14:19.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.901+0000 7fcfe646a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcfe0100e20 0x7fcfe00fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:19.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.901+0000 7fcfe646a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcfe0102570 0x7fcfe00fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:19.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.901+0000 7fcfe646a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfe0102ef0 0x7fcfe00fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:19.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.901+0000 7fcfe646a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe0110f80 con 0x7fcfe0100e20 2024-09-19T09:14:19.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.901+0000 7fcfe646a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe0110e00 con 0x7fcfe0102570 2024-09-19T09:14:19.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.901+0000 7fcfe646a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe0111100 con 0x7fcfe0102ef0 2024-09-19T09:14:19.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.901+0000 7fcfdffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcfe0100e20 0x7fcfe00fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:19.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.901+0000 7fcfdf7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcfe0102570 0x7fcfe00fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:19.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.901+0000 7fcfdffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcfe0100e20 0x7fcfe00fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47894/0 (socket says 172.21.15.120:47894) 2024-09-19T09:14:19.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.901+0000 7fcfdffff640 1 -- 172.21.15.120:0/3726998599 learned_addr learned my addr 172.21.15.120:0/3726998599 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:19.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.901+0000 7fcfe4a07640 1 --2- 172.21.15.120:0/3726998599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfe0102ef0 0x7fcfe00fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:19.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.902+0000 7fcfdffff640 1 -- 172.21.15.120:0/3726998599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfe0102ef0 msgr2=0x7fcfe00fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:19.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.902+0000 7fcfdffff640 1 --2- 172.21.15.120:0/3726998599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfe0102ef0 0x7fcfe00fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.902+0000 7fcfdffff640 1 -- 172.21.15.120:0/3726998599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcfe0102570 msgr2=0x7fcfe00fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:19.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.902+0000 7fcfdffff640 1 --2- 172.21.15.120:0/3726998599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcfe0102570 0x7fcfe00fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.902+0000 7fcfdffff640 1 -- 172.21.15.120:0/3726998599 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe01a6930 con 0x7fcfe0100e20 2024-09-19T09:14:19.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.902+0000 7fcfdf7fe640 1 --2- 172.21.15.120:0/3726998599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcfe0102570 0x7fcfe00fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:19.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.902+0000 7fcfdffff640 1 --2- 172.21.15.120:0/3726998599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcfe0100e20 0x7fcfe00fcc80 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000e9f0 tx=0x7fcfd000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:19.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.902+0000 7fcfdd7fa640 1 -- 172.21.15.120:0/3726998599 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd000cd80 con 0x7fcfe0100e20 2024-09-19T09:14:19.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.902+0000 7fcfdd7fa640 1 -- 172.21.15.120:0/3726998599 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfd0010910 con 0x7fcfe0100e20 2024-09-19T09:14:19.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.902+0000 7fcfe646a640 1 -- 172.21.15.120:0/3726998599 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfe01a6c20 con 0x7fcfe0100e20 2024-09-19T09:14:19.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.902+0000 7fcfdd7fa640 1 -- 172.21.15.120:0/3726998599 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd00188f0 con 0x7fcfe0100e20 2024-09-19T09:14:19.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.902+0000 7fcfe646a640 1 -- 172.21.15.120:0/3726998599 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfe01a7160 con 0x7fcfe0100e20 2024-09-19T09:14:19.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.905+0000 7fcfdd7fa640 1 -- 172.21.15.120:0/3726998599 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fcfd0010ac0 con 0x7fcfe0100e20 2024-09-19T09:14:19.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.905+0000 7fcfe8843740 1 Processor -- start 2024-09-19T09:14:19.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.905+0000 7fcfe8843740 1 -- start start 2024-09-19T09:14:19.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.905+0000 7fcfe8843740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617023604a0 0x561702360880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:19.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.905+0000 7fcfe8843740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561702360dc0 0x56170236b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:19.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.905+0000 7fcfe8843740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56170236b680 0x56170236da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:19.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.905+0000 7fcfe8843740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5617022c62e0 con 0x5617023604a0 2024-09-19T09:14:19.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.905+0000 7fcfe8843740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56170226dce0 con 0x56170236b680 2024-09-19T09:14:19.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.905+0000 7fcfe8843740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5617022c22a0 con 0x561702360dc0 2024-09-19T09:14:19.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.905+0000 7fcfdd7fa640 1 --2- 172.21.15.120:0/3726998599 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcfc80a3240 0x7fcfc80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:19.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.905+0000 7fcfe4a07640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617023604a0 0x561702360880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:19.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.905+0000 7fcfdffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561702360dc0 0x56170236b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:19.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.905+0000 7fcfe4a07640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617023604a0 0x561702360880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47910/0 (socket says 172.21.15.120:47910) 2024-09-19T09:14:19.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.905+0000 7fcfe4a07640 1 -- 172.21.15.120:0/3760454732 learned_addr learned my addr 172.21.15.120:0/3760454732 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:19.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.905+0000 7fcfdf7fe640 1 --2- 172.21.15.120:0/3760454732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56170236b680 0x56170236da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:19.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.906+0000 7fcfe4a07640 1 --2- 172.21.15.120:0/3726998599 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcfc80a3240 0x7fcfc80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:19.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.906+0000 7fcfdd7fa640 1 -- 172.21.15.120:0/3726998599 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fcfd00cfcd0 con 0x7fcfe0100e20 2024-09-19T09:14:19.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.906+0000 7fcfdf7fe640 1 -- 172.21.15.120:0/3760454732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561702360dc0 msgr2=0x56170236b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:19.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.906+0000 7fcfdf7fe640 1 --2- 172.21.15.120:0/3760454732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561702360dc0 0x56170236b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.906+0000 7fcfdf7fe640 1 -- 172.21.15.120:0/3760454732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617023604a0 msgr2=0x561702360880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:19.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.906+0000 7fcfdf7fe640 1 --2- 172.21.15.120:0/3760454732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617023604a0 0x561702360880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.906+0000 7fcfdf7fe640 1 -- 172.21.15.120:0/3760454732 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561702252440 con 0x56170236b680 2024-09-19T09:14:19.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.906+0000 7fcfdffff640 1 --2- 172.21.15.120:0/3760454732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561702360dc0 0x56170236b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:19.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.906+0000 7fcfdf7fe640 1 --2- 172.21.15.120:0/3760454732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56170236b680 0x56170236da70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4005fd0 tx=0x7fcfd4010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98b573e07ec1c894 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:19.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.906+0000 7fcfe4a07640 1 --2- 172.21.15.120:0/3726998599 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcfc80a3240 0x7fcfc80a5700 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8030040 tx=0x7fcfd8005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:19.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.907+0000 7fcfc67fc640 1 -- 172.21.15.120:0/3760454732 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4011700 con 0x56170236b680 2024-09-19T09:14:19.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.907+0000 7fcfc67fc640 1 -- 172.21.15.120:0/3760454732 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfd4011d80 con 0x56170236b680 2024-09-19T09:14:19.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.907+0000 7fcfc67fc640 1 -- 172.21.15.120:0/3760454732 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd401a380 con 0x56170236b680 2024-09-19T09:14:19.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.907+0000 7fcfe8843740 1 -- 172.21.15.120:0/3760454732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56170236b680 msgr2=0x56170236da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:19.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.907+0000 7fcfe8843740 1 --2- 172.21.15.120:0/3760454732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56170236b680 0x56170236da70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4005fd0 tx=0x7fcfd4010860 comp rx=0 tx=0).stop 2024-09-19T09:14:19.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.908+0000 7fcfe8843740 1 -- 172.21.15.120:0/3760454732 shutdown_connections 2024-09-19T09:14:19.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.908+0000 7fcfe8843740 1 --2- 172.21.15.120:0/3760454732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56170236b680 0x56170236da70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.908+0000 7fcfe8843740 1 --2- 172.21.15.120:0/3760454732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561702360dc0 0x56170236b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.908+0000 7fcfe8843740 1 --2- 172.21.15.120:0/3760454732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617023604a0 0x561702360880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.908+0000 7fcfe8843740 1 -- 172.21.15.120:0/3760454732 >> 172.21.15.120:0/3760454732 conn(0x56170235ca30 msgr2=0x56170235ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:19.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.908+0000 7fcfe8843740 1 -- 172.21.15.120:0/3760454732 shutdown_connections 2024-09-19T09:14:19.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.908+0000 7fcfe8843740 1 -- 172.21.15.120:0/3760454732 wait complete. 2024-09-19T09:14:19.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.908+0000 7fcfe8843740 1 -- start start 2024-09-19T09:14:19.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.908+0000 7fcfe8843740 1 Processor -- start 2024-09-19T09:14:19.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.909+0000 7fcfe8843740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617023604a0 0x56170236aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:19.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.909+0000 7fcfe8843740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561702360dc0 0x5617023681a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:19.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.909+0000 7fcfe8843740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56170236b680 0x5617023686e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:19.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.909+0000 7fcfe8843740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56170236e250 con 0x5617023604a0 2024-09-19T09:14:19.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.909+0000 7fcfe8843740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561702281050 con 0x56170236b680 2024-09-19T09:14:19.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.909+0000 7fcfe8843740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56170236e3d0 con 0x561702360dc0 2024-09-19T09:14:19.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.909+0000 7fcfe8843740 10 client.0 _ll_register_callbacks cb 0x5617022c1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:19.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.909+0000 7fcfdffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617023604a0 0x56170236aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:19.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.909+0000 7fcfdffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617023604a0 0x56170236aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47914/0 (socket says 172.21.15.120:47914) 2024-09-19T09:14:19.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.909+0000 7fcfdffff640 1 -- 172.21.15.120:0/607764019 learned_addr learned my addr 172.21.15.120:0/607764019 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:19.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.909+0000 7fcfdf7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561702360dc0 0x5617023681a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:19.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.909+0000 7fcfe4a07640 1 --2- 172.21.15.120:0/607764019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56170236b680 0x5617023686e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:19.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.909+0000 7fcfdffff640 1 -- 172.21.15.120:0/607764019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561702360dc0 msgr2=0x5617023681a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:19.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.909+0000 7fcfdffff640 1 --2- 172.21.15.120:0/607764019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561702360dc0 0x5617023681a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.909+0000 7fcfdffff640 1 -- 172.21.15.120:0/607764019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56170236b680 msgr2=0x5617023686e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:19.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.909+0000 7fcfdffff640 1 --2- 172.21.15.120:0/607764019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56170236b680 0x5617023686e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.909+0000 7fcfdffff640 1 -- 172.21.15.120:0/607764019 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617020373c0 con 0x5617023604a0 2024-09-19T09:14:19.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.909+0000 7fcfdf7fe640 1 --2- 172.21.15.120:0/607764019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561702360dc0 0x5617023681a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:19.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.909+0000 7fcfdffff640 1 --2- 172.21.15.120:0/607764019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617023604a0 0x56170236aef0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000c6a0 tx=0x7fcfd009bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=796b1170e2e1a274 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:19.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.909+0000 7fcfbf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:14:19.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.912+0000 7fcfbf7fe640 1 -- 172.21.15.120:0/607764019 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0026040 con 0x5617023604a0 2024-09-19T09:14:19.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.912+0000 7fcfbf7fe640 1 -- 172.21.15.120:0/607764019 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fcfd009b9f0 con 0x5617023604a0 2024-09-19T09:14:19.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.912+0000 7fcfe8843740 1 -- 172.21.15.120:0/607764019 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfd4011d80 con 0x5617023604a0 2024-09-19T09:14:19.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.913+0000 7fcfbf7fe640 1 -- 172.21.15.120:0/607764019 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcfd00a4f60 con 0x5617023604a0 2024-09-19T09:14:19.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.913+0000 7fcfe8843740 10 client.19569 fetch_fsmap learned FSMap version 32 2024-09-19T09:14:19.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.913+0000 7fcfe8843740 1 -- 172.21.15.120:0/607764019 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561702379b70 con 0x5617023604a0 2024-09-19T09:14:19.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.913+0000 7fcfbf7fe640 1 -- 172.21.15.120:0/607764019 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfd4011d80 con 0x5617023604a0 2024-09-19T09:14:19.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.913+0000 7fcfbf7fe640 1 -- 172.21.15.120:0/607764019 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fcfd00980f0 con 0x5617023604a0 2024-09-19T09:14:19.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.913+0000 7fcfe8843740 10 client.19569 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:14:19.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.913+0000 7fcfe8843740 10 client.19569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:14:19.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.913+0000 7fcfe8843740 20 client.19569 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:19.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.913+0000 7fcfe8843740 1 --2- 172.21.15.120:0/607764019 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56170237b250 0x56170237d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:19.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.913+0000 7fcfe8843740 4 client.19569 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:14:19.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.913+0000 7fcfe8843740 1 -- 172.21.15.120:0/607764019 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617022d3850 con 0x56170237b250 2024-09-19T09:14:19.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.913+0000 7fcfdf7fe640 1 --2- 172.21.15.120:0/607764019 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56170237b250 0x56170237d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:19.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.914+0000 7fcfdf7fe640 1 --2- 172.21.15.120:0/607764019 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56170237b250 0x56170237d640 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b3a90fc572082aa server_cookie=5480f704ebb053b9 in_seq=0 out_seq=0 2024-09-19T09:14:19.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.914+0000 7fcfbf7fe640 10 client.19569 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:19.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.914+0000 7fcfbf7fe640 1 -- 172.21.15.120:0/607764019 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5617022d3850 con 0x56170237b250 2024-09-19T09:14:19.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.914+0000 7fcfbf7fe640 10 client.19569 handle_command_reply: tid=0 2024-09-19T09:14:19.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.915+0000 7fcfe8843740 1 client.19569 shutdown 2024-09-19T09:14:19.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.915+0000 7fcfe8843740 1 -- 172.21.15.120:0/607764019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617023604a0 msgr2=0x56170236aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:19.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.915+0000 7fcfe8843740 1 --2- 172.21.15.120:0/607764019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617023604a0 0x56170236aef0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000c6a0 tx=0x7fcfd009bdc0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.915+0000 7fcfe8843740 1 -- 172.21.15.120:0/607764019 shutdown_connections 2024-09-19T09:14:19.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.915+0000 7fcfe8843740 1 --2- 172.21.15.120:0/607764019 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56170237b250 0x56170237d640 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.915+0000 7fcfe8843740 1 --2- 172.21.15.120:0/607764019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56170236b680 0x5617023686e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.915+0000 7fcfe8843740 1 --2- 172.21.15.120:0/607764019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561702360dc0 0x5617023681a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.915+0000 7fcfe8843740 1 --2- 172.21.15.120:0/607764019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617023604a0 0x56170236aef0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.915+0000 7fcfbf7fe640 0 client.19569 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:19.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.915+0000 7fcfe8843740 1 -- 172.21.15.120:0/607764019 >> 172.21.15.120:0/607764019 conn(0x56170235ca30 msgr2=0x561702362760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:19.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.915+0000 7fcfe8843740 1 -- 172.21.15.120:0/607764019 shutdown_connections 2024-09-19T09:14:19.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.915+0000 7fcfe8843740 1 -- 172.21.15.120:0/607764019 wait complete. 2024-09-19T09:14:19.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.915+0000 7fcfe8843740 20 client.19569 trim_cache size 0 max 16384 2024-09-19T09:14:19.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.934+0000 7fcfe8843740 1 Processor -- start 2024-09-19T09:14:19.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.934+0000 7fcfe8843740 1 -- start start 2024-09-19T09:14:19.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.934+0000 7fcfe8843740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617023604a0 0x56170235ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:19.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.934+0000 7fcfe8843740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561702360dc0 0x561702382be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:19.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.934+0000 7fcfe8843740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56170236b680 0x561702383120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:19.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.934+0000 7fcfe8843740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe01af710 con 0x56170236b680 2024-09-19T09:14:19.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.934+0000 7fcfe8843740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe01b1250 con 0x5617023604a0 2024-09-19T09:14:19.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.934+0000 7fcfe8843740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe01afde0 con 0x561702360dc0 2024-09-19T09:14:19.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.934+0000 7fcfe4a07640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56170236b680 0x561702383120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:19.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.934+0000 7fcfe4a07640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56170236b680 0x561702383120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47918/0 (socket says 172.21.15.120:47918) 2024-09-19T09:14:19.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.934+0000 7fcfe4a07640 1 -- 172.21.15.120:0/3949201122 learned_addr learned my addr 172.21.15.120:0/3949201122 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:19.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.934+0000 7fcfdffff640 1 --2- 172.21.15.120:0/3949201122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617023604a0 0x56170235ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:19.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.934+0000 7fcfdf7fe640 1 --2- 172.21.15.120:0/3949201122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561702360dc0 0x561702382be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:19.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.934+0000 7fcfdffff640 1 -- 172.21.15.120:0/3949201122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561702360dc0 msgr2=0x561702382be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:19.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.934+0000 7fcfdffff640 1 --2- 172.21.15.120:0/3949201122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561702360dc0 0x561702382be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.934+0000 7fcfdffff640 1 -- 172.21.15.120:0/3949201122 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56170236b680 msgr2=0x561702383120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:19.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.934+0000 7fcfdffff640 1 --2- 172.21.15.120:0/3949201122 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56170236b680 0x561702383120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.934+0000 7fcfdffff640 1 -- 172.21.15.120:0/3949201122 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56170235c860 con 0x5617023604a0 2024-09-19T09:14:19.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.934+0000 7fcfe4a07640 1 --2- 172.21.15.120:0/3949201122 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56170236b680 0x561702383120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:19.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.935+0000 7fcfdffff640 1 --2- 172.21.15.120:0/3949201122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617023604a0 0x56170235ba60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcfd009b280 tx=0x7fcfd009be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f79712d360517bbd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:19.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.935+0000 7fcfc5ffb640 1 -- 172.21.15.120:0/3949201122 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd00980f0 con 0x5617023604a0 2024-09-19T09:14:19.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.935+0000 7fcfc5ffb640 1 -- 172.21.15.120:0/3949201122 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfd0098290 con 0x5617023604a0 2024-09-19T09:14:19.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.935+0000 7fcfc5ffb640 1 -- 172.21.15.120:0/3949201122 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0010690 con 0x5617023604a0 2024-09-19T09:14:19.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.936+0000 7fcfe8843740 1 -- 172.21.15.120:0/3949201122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617023604a0 msgr2=0x56170235ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:19.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.936+0000 7fcfe8843740 1 --2- 172.21.15.120:0/3949201122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617023604a0 0x56170235ba60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcfd009b280 tx=0x7fcfd009be30 comp rx=0 tx=0).stop 2024-09-19T09:14:19.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.936+0000 7fcfe8843740 1 -- 172.21.15.120:0/3949201122 shutdown_connections 2024-09-19T09:14:19.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.936+0000 7fcfe8843740 1 --2- 172.21.15.120:0/3949201122 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56170236b680 0x561702383120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.936+0000 7fcfe8843740 1 --2- 172.21.15.120:0/3949201122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561702360dc0 0x561702382be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.936+0000 7fcfe8843740 1 --2- 172.21.15.120:0/3949201122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617023604a0 0x56170235ba60 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.936+0000 7fcfe8843740 1 -- 172.21.15.120:0/3949201122 >> 172.21.15.120:0/3949201122 conn(0x56170235ca30 msgr2=0x56170235b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:19.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.936+0000 7fcfe8843740 1 -- 172.21.15.120:0/3949201122 shutdown_connections 2024-09-19T09:14:19.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.936+0000 7fcfe8843740 1 -- 172.21.15.120:0/3949201122 wait complete. 2024-09-19T09:14:19.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.936+0000 7fcfe8843740 1 -- start start 2024-09-19T09:14:19.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.936+0000 7fcfe8843740 1 Processor -- start 2024-09-19T09:14:19.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.936+0000 7fcfe8843740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617023604a0 0x561702370800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:19.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.936+0000 7fcfe8843740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561702360dc0 0x561702370d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:19.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.936+0000 7fcfe8843740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56170236b680 0x5617023e3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:19.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.936+0000 7fcfe8843740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe01b0b80 con 0x561702360dc0 2024-09-19T09:14:19.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.936+0000 7fcfe8843740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe01b04b0 con 0x5617023604a0 2024-09-19T09:14:19.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.936+0000 7fcfe8843740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe01ad6d0 con 0x56170236b680 2024-09-19T09:14:19.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.937+0000 7fcfe8843740 10 client.0 _ll_register_callbacks cb 0x7fcfd000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:19.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.937+0000 7fcfdffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617023604a0 0x561702370800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:19.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.937+0000 7fcfdf7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561702360dc0 0x561702370d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:19.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.937+0000 7fcfdffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617023604a0 0x561702370800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52320/0 (socket says 172.21.15.120:52320) 2024-09-19T09:14:19.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.937+0000 7fcfdffff640 1 -- 172.21.15.120:0/1916713890 learned_addr learned my addr 172.21.15.120:0/1916713890 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:19.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.937+0000 7fcfdf7fe640 1 --2- 172.21.15.120:0/1916713890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561702360dc0 0x561702370d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47934/0 (socket says 172.21.15.120:47934) 2024-09-19T09:14:19.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.937+0000 7fcfe4a07640 1 --2- 172.21.15.120:0/1916713890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56170236b680 0x5617023e3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:19.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.937+0000 7fcfdffff640 1 -- 172.21.15.120:0/1916713890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56170236b680 msgr2=0x5617023e3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:19.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.937+0000 7fcfdffff640 1 --2- 172.21.15.120:0/1916713890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56170236b680 0x5617023e3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.937+0000 7fcfdffff640 1 -- 172.21.15.120:0/1916713890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561702360dc0 msgr2=0x561702370d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:19.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.937+0000 7fcfdffff640 1 --2- 172.21.15.120:0/1916713890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561702360dc0 0x561702370d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.937+0000 7fcfdffff640 1 -- 172.21.15.120:0/1916713890 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617023e40f0 con 0x5617023604a0 2024-09-19T09:14:19.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.937+0000 7fcfdf7fe640 1 --2- 172.21.15.120:0/1916713890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561702360dc0 0x561702370d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:19.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.937+0000 7fcfdffff640 1 --2- 172.21.15.120:0/1916713890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617023604a0 0x561702370800 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0098d70 tx=0x7fcfd009bdc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30998ff1210ca3d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:19.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.937+0000 7fcfbf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:14:19.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.937+0000 7fcfbf7fe640 1 -- 172.21.15.120:0/1916713890 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd00980f0 con 0x5617023604a0 2024-09-19T09:14:19.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.937+0000 7fcfbf7fe640 1 -- 172.21.15.120:0/1916713890 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fcfd0010430 con 0x5617023604a0 2024-09-19T09:14:19.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.937+0000 7fcfe8843740 1 -- 172.21.15.120:0/1916713890 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfd0098290 con 0x5617023604a0 2024-09-19T09:14:19.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.938+0000 7fcfbf7fe640 1 -- 172.21.15.120:0/1916713890 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcfd0010630 con 0x5617023604a0 2024-09-19T09:14:19.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.938+0000 7fcfe8843740 10 client.28625 fetch_fsmap learned FSMap version 32 2024-09-19T09:14:19.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.938+0000 7fcfe8843740 1 -- 172.21.15.120:0/1916713890 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5617023e52e0 con 0x5617023604a0 2024-09-19T09:14:19.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.938+0000 7fcfbf7fe640 1 -- 172.21.15.120:0/1916713890 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfd0098290 con 0x5617023604a0 2024-09-19T09:14:19.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.938+0000 7fcfbf7fe640 1 -- 172.21.15.120:0/1916713890 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fcfd00a1930 con 0x5617023604a0 2024-09-19T09:14:19.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.938+0000 7fcfe8843740 10 client.28625 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:14:19.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.938+0000 7fcfe8843740 10 client.28625 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:14:19.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.938+0000 7fcfe8843740 20 client.28625 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:19.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.938+0000 7fcfe8843740 1 --2- 172.21.15.120:0/1916713890 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5617023e5710 0x5617023e7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:19.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.938+0000 7fcfe8843740 4 client.28625 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:14:19.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.938+0000 7fcfe8843740 1 -- 172.21.15.120:0/1916713890 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5617022cdd60 con 0x5617023e5710 2024-09-19T09:14:19.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.938+0000 7fcfdf7fe640 1 --2- 172.21.15.120:0/1916713890 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5617023e5710 0x5617023e7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:19.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.939+0000 7fcfbf7fe640 10 client.28625 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:19.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.939+0000 7fcfdf7fe640 1 --2- 172.21.15.120:0/1916713890 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5617023e5710 0x5617023e7b00 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8038b96c174a2c49 server_cookie=3d2514b0a64f502f in_seq=0 out_seq=0 2024-09-19T09:14:19.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.940+0000 7fcfbf7fe640 1 -- 172.21.15.120:0/1916713890 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5617022cdd60 con 0x5617023e5710 2024-09-19T09:14:19.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.940+0000 7fcfbf7fe640 10 client.28625 handle_command_reply: tid=0 2024-09-19T09:14:19.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.941+0000 7fcfe8843740 1 client.28625 shutdown 2024-09-19T09:14:19.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.941+0000 7fcfe8843740 1 -- 172.21.15.120:0/1916713890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617023604a0 msgr2=0x561702370800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:19.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.941+0000 7fcfe8843740 1 --2- 172.21.15.120:0/1916713890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617023604a0 0x561702370800 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0098d70 tx=0x7fcfd009bdc0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.941+0000 7fcfe8843740 1 -- 172.21.15.120:0/1916713890 shutdown_connections 2024-09-19T09:14:19.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.941+0000 7fcfe8843740 1 --2- 172.21.15.120:0/1916713890 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5617023e5710 0x5617023e7b00 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.941+0000 7fcfe8843740 1 --2- 172.21.15.120:0/1916713890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56170236b680 0x5617023e3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.941+0000 7fcfe8843740 1 --2- 172.21.15.120:0/1916713890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561702360dc0 0x561702370d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.941+0000 7fcfbf7fe640 0 client.28625 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:19.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.941+0000 7fcfe8843740 1 --2- 172.21.15.120:0/1916713890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617023604a0 0x561702370800 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.941+0000 7fcfe8843740 1 -- 172.21.15.120:0/1916713890 >> 172.21.15.120:0/1916713890 conn(0x56170235ca30 msgr2=0x56170235b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:19.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.941+0000 7fcfe8843740 1 -- 172.21.15.120:0/1916713890 shutdown_connections 2024-09-19T09:14:19.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.941+0000 7fcfe8843740 1 -- 172.21.15.120:0/1916713890 wait complete. 2024-09-19T09:14:19.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.941+0000 7fcfe8843740 20 client.28625 trim_cache size 0 max 16384 2024-09-19T09:14:19.946 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:14:19.946 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:14:19.946 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-19T09:14:19.947 INFO:teuthology.orchestra.run.smithi120.stdout: "ce8451b8-4827-4de9-8c72-79cbeb7dde0e": { 2024-09-19T09:14:19.947 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-19T09:14:19.947 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "ce8451b8-4827-4de9-8c72-79cbeb7dde0e", 2024-09-19T09:14:19.947 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-19T09:14:19.947 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:14:19.947 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:14:19.947 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:14:19.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.941+0000 7fcfe646a640 1 -- 172.21.15.120:0/3726998599 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcfc80a3240 msgr2=0x7fcfc80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:19.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.941+0000 7fcfe646a640 1 --2- 172.21.15.120:0/3726998599 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcfc80a3240 0x7fcfc80a5700 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8030040 tx=0x7fcfd8005c50 comp rx=0 tx=0).stop 2024-09-19T09:14:19.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.941+0000 7fcfe646a640 1 -- 172.21.15.120:0/3726998599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcfe0100e20 msgr2=0x7fcfe00fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:19.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.941+0000 7fcfe646a640 1 --2- 172.21.15.120:0/3726998599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcfe0100e20 0x7fcfe00fcc80 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000e9f0 tx=0x7fcfd000eeb0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.942+0000 7fcfe646a640 1 -- 172.21.15.120:0/3726998599 shutdown_connections 2024-09-19T09:14:19.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.942+0000 7fcfe646a640 1 --2- 172.21.15.120:0/3726998599 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcfc80a3240 0x7fcfc80a5700 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.942+0000 7fcfe646a640 1 --2- 172.21.15.120:0/3726998599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfe0102ef0 0x7fcfe00fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.942+0000 7fcfe646a640 1 --2- 172.21.15.120:0/3726998599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcfe0102570 0x7fcfe00fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.942+0000 7fcfe646a640 1 --2- 172.21.15.120:0/3726998599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcfe0100e20 0x7fcfe00fcc80 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:19.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.942+0000 7fcfe646a640 1 -- 172.21.15.120:0/3726998599 >> 172.21.15.120:0/3726998599 conn(0x7fcfe00fc7d0 msgr2=0x7fcfe006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:19.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.942+0000 7fcfe646a640 1 -- 172.21.15.120:0/3726998599 shutdown_connections 2024-09-19T09:14:19.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:19.942+0000 7fcfe646a640 1 -- 172.21.15.120:0/3726998599 wait complete. 2024-09-19T09:14:20.073 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1083561 9.39 MB/sec execute 495 sec latency 24.311 ms 2024-09-19T09:14:20.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:19 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:14:20.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:19 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:14:20.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:19 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:14:20.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:19 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:14:20.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:19 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:14:20.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:19 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:14:20.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:19 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:14:20.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:19 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:14:20.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:19 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:14:20.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:19 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:14:20.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:19 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:14:20.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:19 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:14:21.019 INFO:tasks.cephfs.filesystem:scrub status for tag:ce8451b8-4827-4de9-8c72-79cbeb7dde0e - {'path': '/', 'tag': 'ce8451b8-4827-4de9-8c72-79cbeb7dde0e', 'options': 'recursive,force'} 2024-09-19T09:14:21.074 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1085888 9.39 MB/sec execute 496 sec latency 69.500 ms 2024-09-19T09:14:21.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:20 smithi196 ceph-mon[35150]: pgmap v687: 97 pgs: 97 active+clean; 45 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.7 MiB/s wr, 290 op/s 2024-09-19T09:14:21.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:20 smithi153 ceph-mon[35071]: pgmap v687: 97 pgs: 97 active+clean; 45 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.7 MiB/s wr, 290 op/s 2024-09-19T09:14:21.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:20 smithi120 ceph-mon[31150]: pgmap v687: 97 pgs: 97 active+clean; 45 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.7 MiB/s wr, 290 op/s 2024-09-19T09:14:22.074 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1088371 9.41 MB/sec execute 497 sec latency 14.568 ms 2024-09-19T09:14:23.074 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1090910 9.42 MB/sec execute 498 sec latency 68.292 ms 2024-09-19T09:14:23.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:22 smithi196 ceph-mon[35150]: pgmap v688: 97 pgs: 97 active+clean; 40 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.7 MiB/s wr, 288 op/s 2024-09-19T09:14:23.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:22 smithi153 ceph-mon[35071]: pgmap v688: 97 pgs: 97 active+clean; 40 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.7 MiB/s wr, 288 op/s 2024-09-19T09:14:23.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:22 smithi120 ceph-mon[31150]: pgmap v688: 97 pgs: 97 active+clean; 40 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.7 MiB/s wr, 288 op/s 2024-09-19T09:14:24.074 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1093867 9.43 MB/sec execute 499 sec latency 16.103 ms 2024-09-19T09:14:24.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:23 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:14:24.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:23 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:14:24.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:23 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:14:25.074 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1096422 9.43 MB/sec execute 500 sec latency 12.687 ms 2024-09-19T09:14:25.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:24 smithi196 ceph-mon[35150]: pgmap v689: 97 pgs: 97 active+clean; 49 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.4 MiB/s wr, 370 op/s 2024-09-19T09:14:25.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:24 smithi153 ceph-mon[35071]: pgmap v689: 97 pgs: 97 active+clean; 49 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.4 MiB/s wr, 370 op/s 2024-09-19T09:14:25.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:24 smithi120 ceph-mon[31150]: pgmap v689: 97 pgs: 97 active+clean; 49 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.4 MiB/s wr, 370 op/s 2024-09-19T09:14:26.074 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1098548 9.44 MB/sec execute 501 sec latency 12.135 ms 2024-09-19T09:14:27.074 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1101586 9.46 MB/sec execute 502 sec latency 8.087 ms 2024-09-19T09:14:27.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:26 smithi196 ceph-mon[35150]: pgmap v690: 97 pgs: 97 active+clean; 49 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.6 MiB/s wr, 371 op/s 2024-09-19T09:14:27.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:26 smithi153 ceph-mon[35071]: pgmap v690: 97 pgs: 97 active+clean; 49 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.6 MiB/s wr, 371 op/s 2024-09-19T09:14:27.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:26 smithi120 ceph-mon[31150]: pgmap v690: 97 pgs: 97 active+clean; 49 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.6 MiB/s wr, 371 op/s 2024-09-19T09:14:28.074 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1104500 9.47 MB/sec execute 503 sec latency 61.326 ms 2024-09-19T09:14:29.074 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1106757 9.47 MB/sec execute 504 sec latency 23.025 ms 2024-09-19T09:14:29.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:28 smithi196 ceph-mon[35150]: pgmap v691: 97 pgs: 97 active+clean; 49 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.7 MiB/s wr, 327 op/s 2024-09-19T09:14:29.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:28 smithi153 ceph-mon[35071]: pgmap v691: 97 pgs: 97 active+clean; 49 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.7 MiB/s wr, 327 op/s 2024-09-19T09:14:29.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:28 smithi120 ceph-mon[31150]: pgmap v691: 97 pgs: 97 active+clean; 49 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.7 MiB/s wr, 327 op/s 2024-09-19T09:14:30.074 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1109147 9.48 MB/sec execute 505 sec latency 11.566 ms 2024-09-19T09:14:30.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.929+0000 7fb60eb2a640 1 -- 172.21.15.120:0/16124426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb608100a40 msgr2=0x7fb608100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:30.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.929+0000 7fb60eb2a640 1 --2- 172.21.15.120:0/16124426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb608100a40 0x7fb608100e40 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc009900 tx=0x7fb5fc02a410 comp rx=0 tx=0).stop 2024-09-19T09:14:30.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.929+0000 7fb60eb2a640 1 -- 172.21.15.120:0/16124426 shutdown_connections 2024-09-19T09:14:30.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.929+0000 7fb60eb2a640 1 --2- 172.21.15.120:0/16124426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6081025f0 0x7fb608108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:30.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.929+0000 7fb60eb2a640 1 --2- 172.21.15.120:0/16124426 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb608101c30 0x7fb6081020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:30.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.929+0000 7fb60eb2a640 1 --2- 172.21.15.120:0/16124426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb608100a40 0x7fb608100e40 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:30.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.929+0000 7fb60eb2a640 1 -- 172.21.15.120:0/16124426 >> 172.21.15.120:0/16124426 conn(0x7fb6080fc1e0 msgr2=0x7fb6080fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:30.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.929+0000 7fb60eb2a640 1 -- 172.21.15.120:0/16124426 shutdown_connections 2024-09-19T09:14:30.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.929+0000 7fb60eb2a640 1 -- 172.21.15.120:0/16124426 wait complete. 2024-09-19T09:14:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.929+0000 7fb60eb2a640 1 Processor -- start 2024-09-19T09:14:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.929+0000 7fb60eb2a640 1 -- start start 2024-09-19T09:14:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.929+0000 7fb60eb2a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb608100a40 0x7fb60806a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.929+0000 7fb60eb2a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb608101c30 0x7fb60819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.929+0000 7fb60eb2a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6081025f0 0x7fb60819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.930+0000 7fb60eb2a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb60810c710 con 0x7fb608101c30 2024-09-19T09:14:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.930+0000 7fb60eb2a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb60810c590 con 0x7fb608100a40 2024-09-19T09:14:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.930+0000 7fb60eb2a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb60810c890 con 0x7fb6081025f0 2024-09-19T09:14:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.930+0000 7fb607fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb608101c30 0x7fb60819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.930+0000 7fb607fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb608101c30 0x7fb60819cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49342/0 (socket says 172.21.15.120:49342) 2024-09-19T09:14:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.930+0000 7fb607fff640 1 -- 172.21.15.120:0/4228186722 learned_addr learned my addr 172.21.15.120:0/4228186722 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.930+0000 7fb60c8c6640 1 --2- 172.21.15.120:0/4228186722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb608100a40 0x7fb60806a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.930+0000 7fb60d0c7640 1 --2- 172.21.15.120:0/4228186722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6081025f0 0x7fb60819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.930+0000 7fb607fff640 1 -- 172.21.15.120:0/4228186722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6081025f0 msgr2=0x7fb60819f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.930+0000 7fb607fff640 1 --2- 172.21.15.120:0/4228186722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6081025f0 0x7fb60819f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.930+0000 7fb607fff640 1 -- 172.21.15.120:0/4228186722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb608100a40 msgr2=0x7fb60806a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.930+0000 7fb607fff640 1 --2- 172.21.15.120:0/4228186722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb608100a40 0x7fb60806a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.930+0000 7fb607fff640 1 -- 172.21.15.120:0/4228186722 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60819fa50 con 0x7fb608101c30 2024-09-19T09:14:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.930+0000 7fb607fff640 1 --2- 172.21.15.120:0/4228186722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb608101c30 0x7fb60819cb20 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400e9f0 tx=0x7fb5f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.930+0000 7fb605ffb640 1 -- 172.21.15.120:0/4228186722 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f400cd80 con 0x7fb608101c30 2024-09-19T09:14:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.930+0000 7fb605ffb640 1 -- 172.21.15.120:0/4228186722 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5f4010910 con 0x7fb608101c30 2024-09-19T09:14:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.930+0000 7fb60eb2a640 1 -- 172.21.15.120:0/4228186722 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb60819fce0 con 0x7fb608101c30 2024-09-19T09:14:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.930+0000 7fb605ffb640 1 -- 172.21.15.120:0/4228186722 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4018990 con 0x7fb608101c30 2024-09-19T09:14:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.930+0000 7fb60eb2a640 1 -- 172.21.15.120:0/4228186722 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6081a01d0 con 0x7fb608101c30 2024-09-19T09:14:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.933+0000 7fb605ffb640 1 -- 172.21.15.120:0/4228186722 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fb5f4010ac0 con 0x7fb608101c30 2024-09-19T09:14:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.933+0000 7fb6118c3740 1 Processor -- start 2024-09-19T09:14:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.933+0000 7fb6118c3740 1 -- start start 2024-09-19T09:14:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.933+0000 7fb6118c3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b65724e9a0 0x55b65724eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.933+0000 7fb6118c3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b65724f2e0 0x55b657253da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.933+0000 7fb6118c3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b657254370 0x55b657256830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.933+0000 7fb6118c3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b6571b22f0 con 0x55b65724e9a0 2024-09-19T09:14:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.933+0000 7fb6118c3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b65715a270 con 0x55b657254370 2024-09-19T09:14:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.933+0000 7fb6118c3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b6571ae2b0 con 0x55b65724f2e0 2024-09-19T09:14:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.933+0000 7fb60d0c7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b65724e9a0 0x55b65724eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.933+0000 7fb607fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b657254370 0x55b657256830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.933+0000 7fb60d0c7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b65724e9a0 0x55b65724eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49346/0 (socket says 172.21.15.120:49346) 2024-09-19T09:14:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.933+0000 7fb60d0c7640 1 -- 172.21.15.120:0/609168512 learned_addr learned my addr 172.21.15.120:0/609168512 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.933+0000 7fb60c8c6640 1 --2- 172.21.15.120:0/609168512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b65724f2e0 0x55b657253da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.934+0000 7fb60d0c7640 1 -- 172.21.15.120:0/609168512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b65724f2e0 msgr2=0x55b657253da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.934+0000 7fb60d0c7640 1 --2- 172.21.15.120:0/609168512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b65724f2e0 0x55b657253da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.934+0000 7fb60d0c7640 1 -- 172.21.15.120:0/609168512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b657254370 msgr2=0x55b657256830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.934+0000 7fb60d0c7640 1 --2- 172.21.15.120:0/609168512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b657254370 0x55b657256830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.934+0000 7fb60d0c7640 1 -- 172.21.15.120:0/609168512 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b657159340 con 0x55b65724e9a0 2024-09-19T09:14:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.934+0000 7fb605ffb640 1 --2- 172.21.15.120:0/4228186722 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb5f80a32f0 0x7fb5f80a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.934+0000 7fb60c8c6640 1 --2- 172.21.15.120:0/4228186722 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb5f80a32f0 0x7fb5f80a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.934+0000 7fb605ffb640 1 -- 172.21.15.120:0/4228186722 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb5f40cfc00 con 0x7fb608101c30 2024-09-19T09:14:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.934+0000 7fb60d0c7640 1 --2- 172.21.15.120:0/609168512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b65724e9a0 0x55b65724eda0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fb600009fb0 tx=0x7fb6000118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f636e683c3d7fcd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.934+0000 7fb5f2ffd640 1 -- 172.21.15.120:0/609168512 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb600004d20 con 0x55b65724e9a0 2024-09-19T09:14:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.934+0000 7fb5f2ffd640 1 -- 172.21.15.120:0/609168512 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb600004050 con 0x55b65724e9a0 2024-09-19T09:14:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.934+0000 7fb5f2ffd640 1 -- 172.21.15.120:0/609168512 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb600005240 con 0x55b65724e9a0 2024-09-19T09:14:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.935+0000 7fb6118c3740 1 -- 172.21.15.120:0/609168512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b65724e9a0 msgr2=0x55b65724eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.935+0000 7fb60c8c6640 1 --2- 172.21.15.120:0/4228186722 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb5f80a32f0 0x7fb5f80a57b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc02aa00 tx=0x7fb5fc0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.935+0000 7fb6118c3740 1 --2- 172.21.15.120:0/609168512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b65724e9a0 0x55b65724eda0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fb600009fb0 tx=0x7fb6000118d0 comp rx=0 tx=0).stop 2024-09-19T09:14:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.935+0000 7fb6118c3740 1 -- 172.21.15.120:0/609168512 shutdown_connections 2024-09-19T09:14:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.935+0000 7fb6118c3740 1 --2- 172.21.15.120:0/609168512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b657254370 0x55b657256830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.935+0000 7fb6118c3740 1 --2- 172.21.15.120:0/609168512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b65724f2e0 0x55b657253da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.935+0000 7fb6118c3740 1 --2- 172.21.15.120:0/609168512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b65724e9a0 0x55b65724eda0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.935+0000 7fb6118c3740 1 -- 172.21.15.120:0/609168512 >> 172.21.15.120:0/609168512 conn(0x55b65724a010 msgr2=0x55b65724a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.935+0000 7fb6118c3740 1 -- 172.21.15.120:0/609168512 shutdown_connections 2024-09-19T09:14:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.935+0000 7fb6118c3740 1 -- 172.21.15.120:0/609168512 wait complete. 2024-09-19T09:14:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.935+0000 7fb6118c3740 1 -- start start 2024-09-19T09:14:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.935+0000 7fb6118c3740 1 Processor -- start 2024-09-19T09:14:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.936+0000 7fb6118c3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b65724e9a0 0x55b657258b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.936+0000 7fb6118c3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b65724f2e0 0x55b6572590b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.936+0000 7fb6118c3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b657254370 0x55b6572600f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.936+0000 7fb6118c3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b657257340 con 0x55b65724f2e0 2024-09-19T09:14:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.936+0000 7fb6118c3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b65713b290 con 0x55b65724e9a0 2024-09-19T09:14:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.936+0000 7fb6118c3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b6572574c0 con 0x55b657254370 2024-09-19T09:14:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.936+0000 7fb6118c3740 10 client.0 _ll_register_callbacks cb 0x55b65716c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.936+0000 7fb60c8c6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b65724e9a0 0x55b657258b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.936+0000 7fb607fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b65724f2e0 0x55b6572590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.936+0000 7fb60d0c7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b657254370 0x55b6572600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.936+0000 7fb60c8c6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b65724e9a0 0x55b657258b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38072/0 (socket says 172.21.15.120:38072) 2024-09-19T09:14:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.936+0000 7fb60c8c6640 1 -- 172.21.15.120:0/2630185889 learned_addr learned my addr 172.21.15.120:0/2630185889 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.936+0000 7fb60c8c6640 1 -- 172.21.15.120:0/2630185889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b657254370 msgr2=0x55b6572600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.936+0000 7fb60c8c6640 1 --2- 172.21.15.120:0/2630185889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b657254370 0x55b6572600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.936+0000 7fb60c8c6640 1 -- 172.21.15.120:0/2630185889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b65724f2e0 msgr2=0x55b6572590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.936+0000 7fb60c8c6640 1 --2- 172.21.15.120:0/2630185889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b65724f2e0 0x55b6572590b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.936+0000 7fb60c8c6640 1 -- 172.21.15.120:0/2630185889 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b656ec5210 con 0x55b65724e9a0 2024-09-19T09:14:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.936+0000 7fb60d0c7640 1 --2- 172.21.15.120:0/2630185889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b657254370 0x55b6572600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.936+0000 7fb60c8c6640 1 --2- 172.21.15.120:0/2630185889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b65724e9a0 0x55b657258b70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb608101a90 tx=0x7fb5fc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=623be95e8dbe7dd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.936+0000 7fb5e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:14:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.937+0000 7fb5e7fff640 1 -- 172.21.15.120:0/2630185889 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc04e070 con 0x55b65724e9a0 2024-09-19T09:14:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.937+0000 7fb5e7fff640 1 -- 172.21.15.120:0/2630185889 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb5fc004970 con 0x55b65724e9a0 2024-09-19T09:14:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.937+0000 7fb6118c3740 1 -- 172.21.15.120:0/2630185889 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb600004050 con 0x55b65724e9a0 2024-09-19T09:14:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.937+0000 7fb5e7fff640 1 -- 172.21.15.120:0/2630185889 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb5fc05a050 con 0x55b65724e9a0 2024-09-19T09:14:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.937+0000 7fb6118c3740 10 client.28637 fetch_fsmap learned FSMap version 32 2024-09-19T09:14:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.937+0000 7fb6118c3740 1 -- 172.21.15.120:0/2630185889 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b656f91620 con 0x55b65724e9a0 2024-09-19T09:14:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.937+0000 7fb5e7fff640 1 -- 172.21.15.120:0/2630185889 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb600004050 con 0x55b65724e9a0 2024-09-19T09:14:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.937+0000 7fb5e7fff640 1 -- 172.21.15.120:0/2630185889 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb5fc0035f0 con 0x55b65724e9a0 2024-09-19T09:14:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.937+0000 7fb6118c3740 10 client.28637 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:14:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.937+0000 7fb6118c3740 10 client.28637 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:14:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.937+0000 7fb6118c3740 20 client.28637 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.937+0000 7fb6118c3740 1 --2- 172.21.15.120:0/2630185889 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b657264830 0x55b657266ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.937+0000 7fb6118c3740 4 client.28637 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:14:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.937+0000 7fb6118c3740 1 -- 172.21.15.120:0/2630185889 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6571bfa20 con 0x55b657264830 2024-09-19T09:14:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.938+0000 7fb607fff640 1 --2- 172.21.15.120:0/2630185889 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b657264830 0x55b657266ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.938+0000 7fb5e7fff640 10 client.28637 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.938+0000 7fb607fff640 1 --2- 172.21.15.120:0/2630185889 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b657264830 0x55b657266ca0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56d024509ac1106a server_cookie=f7c7aff54ec74dc3 in_seq=0 out_seq=0 2024-09-19T09:14:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.938+0000 7fb5e7fff640 1 -- 172.21.15.120:0/2630185889 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b6571bfa20 con 0x55b657264830 2024-09-19T09:14:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.938+0000 7fb5e7fff640 10 client.28637 handle_command_reply: tid=0 2024-09-19T09:14:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.939+0000 7fb6118c3740 1 client.28637 shutdown 2024-09-19T09:14:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.939+0000 7fb6118c3740 1 -- 172.21.15.120:0/2630185889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b65724e9a0 msgr2=0x55b657258b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.939+0000 7fb6118c3740 1 --2- 172.21.15.120:0/2630185889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b65724e9a0 0x55b657258b70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb608101a90 tx=0x7fb5fc009260 comp rx=0 tx=0).stop 2024-09-19T09:14:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.939+0000 7fb6118c3740 1 -- 172.21.15.120:0/2630185889 shutdown_connections 2024-09-19T09:14:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.939+0000 7fb6118c3740 1 --2- 172.21.15.120:0/2630185889 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b657264830 0x55b657266ca0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.939+0000 7fb6118c3740 1 --2- 172.21.15.120:0/2630185889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b657254370 0x55b6572600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.939+0000 7fb5e7fff640 0 client.28637 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.939+0000 7fb6118c3740 1 --2- 172.21.15.120:0/2630185889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b65724f2e0 0x55b6572590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.939+0000 7fb6118c3740 1 --2- 172.21.15.120:0/2630185889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b65724e9a0 0x55b657258b70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.939+0000 7fb6118c3740 1 -- 172.21.15.120:0/2630185889 >> 172.21.15.120:0/2630185889 conn(0x55b65724a010 msgr2=0x55b65724b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.939+0000 7fb6118c3740 1 -- 172.21.15.120:0/2630185889 shutdown_connections 2024-09-19T09:14:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.939+0000 7fb6118c3740 1 -- 172.21.15.120:0/2630185889 wait complete. 2024-09-19T09:14:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.939+0000 7fb6118c3740 20 client.28637 trim_cache size 0 max 16384 2024-09-19T09:14:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.954+0000 7fb6118c3740 1 Processor -- start 2024-09-19T09:14:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.954+0000 7fb6118c3740 1 -- start start 2024-09-19T09:14:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.954+0000 7fb6118c3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b65724e9a0 0x55b657258f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.954+0000 7fb6118c3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b65724f2e0 0x55b657260010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.954+0000 7fb6118c3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b657254370 0x55b65724b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.954+0000 7fb6118c3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb6081a8f40 con 0x55b657254370 2024-09-19T09:14:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.954+0000 7fb6118c3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb6081aaa80 con 0x55b65724f2e0 2024-09-19T09:14:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.954+0000 7fb6118c3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb6081a9610 con 0x55b65724e9a0 2024-09-19T09:14:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.955+0000 7fb60d0c7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b657254370 0x55b65724b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.955+0000 7fb60d0c7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b657254370 0x55b65724b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49364/0 (socket says 172.21.15.120:49364) 2024-09-19T09:14:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.955+0000 7fb60d0c7640 1 -- 172.21.15.120:0/3295574970 learned_addr learned my addr 172.21.15.120:0/3295574970 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.955+0000 7fb607fff640 1 --2- 172.21.15.120:0/3295574970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b65724f2e0 0x55b657260010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.955+0000 7fb60c8c6640 1 --2- 172.21.15.120:0/3295574970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b65724e9a0 0x55b657258f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.955+0000 7fb60d0c7640 1 -- 172.21.15.120:0/3295574970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b65724e9a0 msgr2=0x55b657258f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.955+0000 7fb60d0c7640 1 --2- 172.21.15.120:0/3295574970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b65724e9a0 0x55b657258f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.955+0000 7fb60d0c7640 1 -- 172.21.15.120:0/3295574970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b65724f2e0 msgr2=0x55b657260010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.955+0000 7fb60d0c7640 1 --2- 172.21.15.120:0/3295574970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b65724f2e0 0x55b657260010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.955+0000 7fb60d0c7640 1 -- 172.21.15.120:0/3295574970 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6572645d0 con 0x55b657254370 2024-09-19T09:14:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.955+0000 7fb607fff640 1 --2- 172.21.15.120:0/3295574970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b65724f2e0 0x55b657260010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.955+0000 7fb60d0c7640 1 --2- 172.21.15.120:0/3295574970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b657254370 0x55b65724b400 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fb600000f80 tx=0x7fb600009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3176be838fd1128c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.955+0000 7fb5f27fc640 1 -- 172.21.15.120:0/3295574970 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb600003a30 con 0x55b657254370 2024-09-19T09:14:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.955+0000 7fb5f27fc640 1 -- 172.21.15.120:0/3295574970 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb600003bd0 con 0x55b657254370 2024-09-19T09:14:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.955+0000 7fb5f27fc640 1 -- 172.21.15.120:0/3295574970 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb600005670 con 0x55b657254370 2024-09-19T09:14:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.956+0000 7fb6118c3740 1 -- 172.21.15.120:0/3295574970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b657254370 msgr2=0x55b65724b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.956+0000 7fb6118c3740 1 --2- 172.21.15.120:0/3295574970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b657254370 0x55b65724b400 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fb600000f80 tx=0x7fb600009b50 comp rx=0 tx=0).stop 2024-09-19T09:14:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.956+0000 7fb6118c3740 1 -- 172.21.15.120:0/3295574970 shutdown_connections 2024-09-19T09:14:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.956+0000 7fb6118c3740 1 --2- 172.21.15.120:0/3295574970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b657254370 0x55b65724b400 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.956+0000 7fb6118c3740 1 --2- 172.21.15.120:0/3295574970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b65724f2e0 0x55b657260010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.956+0000 7fb6118c3740 1 --2- 172.21.15.120:0/3295574970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b65724e9a0 0x55b657258f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.956+0000 7fb6118c3740 1 -- 172.21.15.120:0/3295574970 >> 172.21.15.120:0/3295574970 conn(0x55b65724a010 msgr2=0x55b6572491c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.956+0000 7fb6118c3740 1 -- 172.21.15.120:0/3295574970 shutdown_connections 2024-09-19T09:14:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.956+0000 7fb6118c3740 1 -- 172.21.15.120:0/3295574970 wait complete. 2024-09-19T09:14:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.956+0000 7fb6118c3740 1 -- start start 2024-09-19T09:14:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.956+0000 7fb6118c3740 1 Processor -- start 2024-09-19T09:14:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.957+0000 7fb6118c3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b65724e9a0 0x55b65725d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.957+0000 7fb6118c3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b65724f2e0 0x55b65725d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.957+0000 7fb6118c3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b657254370 0x55b65725e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.957+0000 7fb6118c3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb6081aa3b0 con 0x55b65724f2e0 2024-09-19T09:14:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.957+0000 7fb6118c3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb6081a9ce0 con 0x55b65724e9a0 2024-09-19T09:14:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.957+0000 7fb6118c3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb6081a6f00 con 0x55b657254370 2024-09-19T09:14:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.957+0000 7fb6118c3740 10 client.0 _ll_register_callbacks cb 0x7fb5fc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.957+0000 7fb607fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b65724f2e0 0x55b65725d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.957+0000 7fb60c8c6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b65724e9a0 0x55b65725d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.957+0000 7fb60c8c6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b65724e9a0 0x55b65725d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38078/0 (socket says 172.21.15.120:38078) 2024-09-19T09:14:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.957+0000 7fb607fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b65724f2e0 0x55b65725d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49380/0 (socket says 172.21.15.120:49380) 2024-09-19T09:14:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.957+0000 7fb60d0c7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b657254370 0x55b65725e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.957+0000 7fb60c8c6640 1 -- 172.21.15.120:0/2056157829 learned_addr learned my addr 172.21.15.120:0/2056157829 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.957+0000 7fb60c8c6640 1 -- 172.21.15.120:0/2056157829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b657254370 msgr2=0x55b65725e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.957+0000 7fb60c8c6640 1 --2- 172.21.15.120:0/2056157829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b657254370 0x55b65725e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.957+0000 7fb60c8c6640 1 -- 172.21.15.120:0/2056157829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b65724f2e0 msgr2=0x55b65725d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.957+0000 7fb60c8c6640 1 --2- 172.21.15.120:0/2056157829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b65724f2e0 0x55b65725d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.957+0000 7fb60c8c6640 1 -- 172.21.15.120:0/2056157829 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b65725ef90 con 0x55b65724e9a0 2024-09-19T09:14:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.957+0000 7fb60d0c7640 1 --2- 172.21.15.120:0/2056157829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b657254370 0x55b65725e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.957+0000 7fb607fff640 1 --2- 172.21.15.120:0/2056157829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b65724f2e0 0x55b65725d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.957+0000 7fb60c8c6640 1 --2- 172.21.15.120:0/2056157829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b65724e9a0 0x55b65725d430 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc009230 tx=0x7fb5fc004210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad534b86b9bc675b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.957+0000 7fb5e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:14:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.958+0000 7fb5e7fff640 1 -- 172.21.15.120:0/2056157829 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc04e040 con 0x55b65724e9a0 2024-09-19T09:14:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.958+0000 7fb5e7fff640 1 -- 172.21.15.120:0/2056157829 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb5fc004770 con 0x55b65724e9a0 2024-09-19T09:14:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.958+0000 7fb6118c3740 1 -- 172.21.15.120:0/2056157829 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb600003bd0 con 0x55b65724e9a0 2024-09-19T09:14:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.958+0000 7fb5e7fff640 1 -- 172.21.15.120:0/2056157829 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb5fc035040 con 0x55b65724e9a0 2024-09-19T09:14:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.958+0000 7fb6118c3740 10 client.28643 fetch_fsmap learned FSMap version 32 2024-09-19T09:14:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.958+0000 7fb6118c3740 1 -- 172.21.15.120:0/2056157829 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b6572602a0 con 0x55b65724e9a0 2024-09-19T09:14:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.958+0000 7fb5e7fff640 1 -- 172.21.15.120:0/2056157829 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb600003bd0 con 0x55b65724e9a0 2024-09-19T09:14:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.958+0000 7fb5e7fff640 1 -- 172.21.15.120:0/2056157829 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb5fc0329d0 con 0x55b65724e9a0 2024-09-19T09:14:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.958+0000 7fb6118c3740 10 client.28643 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T09:14:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.958+0000 7fb6118c3740 10 client.28643 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:14:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.958+0000 7fb6118c3740 20 client.28643 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.958+0000 7fb6118c3740 1 --2- 172.21.15.120:0/2056157829 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b657264830 0x55b6572d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.958+0000 7fb6118c3740 4 client.28643 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:14:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.958+0000 7fb6118c3740 1 -- 172.21.15.120:0/2056157829 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b6571b9d70 con 0x55b657264830 2024-09-19T09:14:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.958+0000 7fb607fff640 1 --2- 172.21.15.120:0/2056157829 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b657264830 0x55b6572d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.959+0000 7fb5e7fff640 10 client.28643 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:30.959+0000 7fb607fff640 1 --2- 172.21.15.120:0/2056157829 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b657264830 0x55b6572d1c50 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b6ceead2ba231a0 server_cookie=da2721d015a0a662 in_seq=0 out_seq=0 2024-09-19T09:14:31.074 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1112100 9.49 MB/sec execute 506 sec latency 60.152 ms 2024-09-19T09:14:31.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:31.132+0000 7fb5e7fff640 1 -- 172.21.15.120:0/2056157829 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b6571b9d70 con 0x55b657264830 2024-09-19T09:14:31.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:31.132+0000 7fb5e7fff640 10 client.28643 handle_command_reply: tid=0 2024-09-19T09:14:31.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:31.132+0000 7fb6118c3740 1 client.28643 shutdown 2024-09-19T09:14:31.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:31.132+0000 7fb6118c3740 1 -- 172.21.15.120:0/2056157829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b65724e9a0 msgr2=0x55b65725d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:31.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:31.132+0000 7fb6118c3740 1 --2- 172.21.15.120:0/2056157829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b65724e9a0 0x55b65725d430 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc009230 tx=0x7fb5fc004210 comp rx=0 tx=0).stop 2024-09-19T09:14:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:31.132+0000 7fb6118c3740 1 -- 172.21.15.120:0/2056157829 shutdown_connections 2024-09-19T09:14:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:31.132+0000 7fb6118c3740 1 --2- 172.21.15.120:0/2056157829 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b657264830 0x55b6572d1c50 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:31.132+0000 7fb6118c3740 1 --2- 172.21.15.120:0/2056157829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b657254370 0x55b65725e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:31.132+0000 7fb6118c3740 1 --2- 172.21.15.120:0/2056157829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b65724f2e0 0x55b65725d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:31.132+0000 7fb6118c3740 1 --2- 172.21.15.120:0/2056157829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b65724e9a0 0x55b65725d430 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:31.132+0000 7fb5e7fff640 0 client.28643 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:31.132+0000 7fb6118c3740 1 -- 172.21.15.120:0/2056157829 >> 172.21.15.120:0/2056157829 conn(0x55b65724a010 msgr2=0x55b6572491c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:31.132+0000 7fb6118c3740 1 -- 172.21.15.120:0/2056157829 shutdown_connections 2024-09-19T09:14:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:31.132+0000 7fb6118c3740 1 -- 172.21.15.120:0/2056157829 wait complete. 2024-09-19T09:14:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:31.132+0000 7fb6118c3740 20 client.28643 trim_cache size 0 max 16384 2024-09-19T09:14:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:14:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:14:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:14:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:14:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:31.133+0000 7fb60eb2a640 1 -- 172.21.15.120:0/4228186722 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb5f80a32f0 msgr2=0x7fb5f80a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:31.133+0000 7fb60eb2a640 1 --2- 172.21.15.120:0/4228186722 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb5f80a32f0 0x7fb5f80a57b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc02aa00 tx=0x7fb5fc0315b0 comp rx=0 tx=0).stop 2024-09-19T09:14:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:31.133+0000 7fb60eb2a640 1 -- 172.21.15.120:0/4228186722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb608101c30 msgr2=0x7fb60819cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:31.133+0000 7fb60eb2a640 1 --2- 172.21.15.120:0/4228186722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb608101c30 0x7fb60819cb20 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400e9f0 tx=0x7fb5f400eeb0 comp rx=0 tx=0).stop 2024-09-19T09:14:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:31.133+0000 7fb60eb2a640 1 -- 172.21.15.120:0/4228186722 shutdown_connections 2024-09-19T09:14:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:31.133+0000 7fb60eb2a640 1 --2- 172.21.15.120:0/4228186722 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb5f80a32f0 0x7fb5f80a57b0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:31.133+0000 7fb60eb2a640 1 --2- 172.21.15.120:0/4228186722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6081025f0 0x7fb60819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:31.133+0000 7fb60eb2a640 1 --2- 172.21.15.120:0/4228186722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb608101c30 0x7fb60819cb20 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:31.133+0000 7fb60eb2a640 1 --2- 172.21.15.120:0/4228186722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb608100a40 0x7fb60806a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:31.133+0000 7fb60eb2a640 1 -- 172.21.15.120:0/4228186722 >> 172.21.15.120:0/4228186722 conn(0x7fb6080fc1e0 msgr2=0x7fb608104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:31.133+0000 7fb60eb2a640 1 -- 172.21.15.120:0/4228186722 shutdown_connections 2024-09-19T09:14:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:31.133+0000 7fb60eb2a640 1 -- 172.21.15.120:0/4228186722 wait complete. 2024-09-19T09:14:31.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:30 smithi196 ceph-mon[35150]: pgmap v692: 97 pgs: 97 active+clean; 55 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.3 MiB/s wr, 374 op/s 2024-09-19T09:14:31.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:30 smithi153 ceph-mon[35071]: pgmap v692: 97 pgs: 97 active+clean; 55 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.3 MiB/s wr, 374 op/s 2024-09-19T09:14:31.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:30 smithi120 ceph-mon[31150]: pgmap v692: 97 pgs: 97 active+clean; 55 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.3 MiB/s wr, 374 op/s 2024-09-19T09:14:31.689 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:14:31 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-e[63911]: ignoring --setuser ceph since I am not root 2024-09-19T09:14:31.689 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:14:31 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-e[63911]: ignoring --setgroup ceph since I am not root 2024-09-19T09:14:31.689 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:14:31 smithi153 ceph-mds[63915]: -- 172.21.15.153:0/1520222263 <== mon.2 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f9fca35c20 con 0x55f9fca13400 2024-09-19T09:14:31.689 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:14:31 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-e[63911]: starting mds.e at 2024-09-19T09:14:32.074 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1115145 9.50 MB/sec execute 507 sec latency 32.329 ms 2024-09-19T09:14:32.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:31 smithi196 ceph-mon[35150]: daemon mds.e restarted 2024-09-19T09:14:32.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:31 smithi196 ceph-mon[35150]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T09:14:32.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:31 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.153:6834/2810080307,v1:172.21.15.153:6835/2810080307] up:boot 2024-09-19T09:14:32.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:31 smithi196 ceph-mon[35150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:14:32.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:31 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:14:32.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:31 smithi153 ceph-mon[35071]: daemon mds.e restarted 2024-09-19T09:14:32.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:31 smithi153 ceph-mon[35071]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T09:14:32.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:31 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.153:6834/2810080307,v1:172.21.15.153:6835/2810080307] up:boot 2024-09-19T09:14:32.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:31 smithi153 ceph-mon[35071]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:14:32.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:31 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:14:32.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:31 smithi120 ceph-mon[31150]: daemon mds.e restarted 2024-09-19T09:14:32.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:31 smithi120 ceph-mon[31150]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T09:14:32.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:31 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.153:6834/2810080307,v1:172.21.15.153:6835/2810080307] up:boot 2024-09-19T09:14:32.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:31 smithi120 ceph-mon[31150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:14:32.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:31 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:14:33.074 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1116976 9.50 MB/sec execute 508 sec latency 75.913 ms 2024-09-19T09:14:33.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:32 smithi196 ceph-mon[35150]: pgmap v694: 97 pgs: 97 active+clean; 55 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 344 op/s 2024-09-19T09:14:33.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:32 smithi153 ceph-mon[35071]: pgmap v694: 97 pgs: 97 active+clean; 55 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 344 op/s 2024-09-19T09:14:33.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:32 smithi120 ceph-mon[31150]: pgmap v694: 97 pgs: 97 active+clean; 55 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 344 op/s 2024-09-19T09:14:34.074 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1119595 9.51 MB/sec execute 509 sec latency 60.964 ms 2024-09-19T09:14:35.075 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1122787 9.53 MB/sec execute 510 sec latency 29.682 ms 2024-09-19T09:14:35.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:34 smithi196 ceph-mon[35150]: pgmap v695: 97 pgs: 97 active+clean; 44 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 395 op/s 2024-09-19T09:14:35.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:34 smithi153 ceph-mon[35071]: pgmap v695: 97 pgs: 97 active+clean; 44 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 395 op/s 2024-09-19T09:14:35.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:34 smithi120 ceph-mon[31150]: pgmap v695: 97 pgs: 97 active+clean; 44 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 395 op/s 2024-09-19T09:14:36.074 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1125972 9.55 MB/sec execute 511 sec latency 10.819 ms 2024-09-19T09:14:37.074 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1128723 9.55 MB/sec execute 512 sec latency 51.756 ms 2024-09-19T09:14:37.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:36 smithi196 ceph-mon[35150]: pgmap v696: 97 pgs: 97 active+clean; 44 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 424 op/s 2024-09-19T09:14:37.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:36 smithi153 ceph-mon[35071]: pgmap v696: 97 pgs: 97 active+clean; 44 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 424 op/s 2024-09-19T09:14:37.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:36 smithi120 ceph-mon[31150]: pgmap v696: 97 pgs: 97 active+clean; 44 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 424 op/s 2024-09-19T09:14:38.074 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1130745 9.55 MB/sec execute 513 sec latency 57.024 ms 2024-09-19T09:14:39.074 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1133730 9.57 MB/sec execute 514 sec latency 14.834 ms 2024-09-19T09:14:39.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:38 smithi196 ceph-mon[35150]: pgmap v697: 97 pgs: 97 active+clean; 47 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 484 op/s 2024-09-19T09:14:39.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:38 smithi153 ceph-mon[35071]: pgmap v697: 97 pgs: 97 active+clean; 47 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 484 op/s 2024-09-19T09:14:39.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:38 smithi120 ceph-mon[31150]: pgmap v697: 97 pgs: 97 active+clean; 47 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 484 op/s 2024-09-19T09:14:40.074 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1136974 9.59 MB/sec execute 515 sec latency 10.684 ms 2024-09-19T09:14:40.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:39 smithi153 ceph-mon[35071]: pgmap v698: 97 pgs: 97 active+clean; 46 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.4 MiB/s wr, 410 op/s 2024-09-19T09:14:40.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:39 smithi120 ceph-mon[31150]: pgmap v698: 97 pgs: 97 active+clean; 46 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.4 MiB/s wr, 410 op/s 2024-09-19T09:14:40.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:39 smithi196 ceph-mon[35150]: pgmap v698: 97 pgs: 97 active+clean; 46 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.4 MiB/s wr, 410 op/s 2024-09-19T09:14:41.074 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1140282 9.60 MB/sec execute 516 sec latency 21.094 ms 2024-09-19T09:14:42.074 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1143100 9.61 MB/sec execute 517 sec latency 10.658 ms 2024-09-19T09:14:43.074 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1145523 9.62 MB/sec execute 518 sec latency 17.028 ms 2024-09-19T09:14:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:42 smithi196 ceph-mon[35150]: pgmap v699: 97 pgs: 97 active+clean; 49 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.0 MiB/s wr, 450 op/s 2024-09-19T09:14:43.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:42 smithi153 ceph-mon[35071]: pgmap v699: 97 pgs: 97 active+clean; 49 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.0 MiB/s wr, 450 op/s 2024-09-19T09:14:43.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:42 smithi120 ceph-mon[31150]: pgmap v699: 97 pgs: 97 active+clean; 49 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.0 MiB/s wr, 450 op/s 2024-09-19T09:14:44.075 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1147886 9.62 MB/sec execute 519 sec latency 51.463 ms 2024-09-19T09:14:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.223+0000 7f98c0fc0640 1 -- 172.21.15.120:0/2516255747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98bc100580 msgr2=0x7f98bc10a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.223+0000 7f98c0fc0640 1 --2- 172.21.15.120:0/2516255747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98bc100580 0x7f98bc10a130 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009cd0 tx=0x7f98b402a8e0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.223+0000 7f98c0fc0640 1 -- 172.21.15.120:0/2516255747 shutdown_connections 2024-09-19T09:14:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.223+0000 7f98c0fc0640 1 --2- 172.21.15.120:0/2516255747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98bc100580 0x7f98bc10a130 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.223+0000 7f98c0fc0640 1 --2- 172.21.15.120:0/2516255747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98bc0ffbc0 0x7f98bc100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.223+0000 7f98c0fc0640 1 --2- 172.21.15.120:0/2516255747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98bc103150 0x7f98bc103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.223+0000 7f98c0fc0640 1 -- 172.21.15.120:0/2516255747 >> 172.21.15.120:0/2516255747 conn(0x7f98bc0fb970 msgr2=0x7f98bc0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.223+0000 7f98c0fc0640 1 -- 172.21.15.120:0/2516255747 shutdown_connections 2024-09-19T09:14:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.223+0000 7f98c0fc0640 1 -- 172.21.15.120:0/2516255747 wait complete. 2024-09-19T09:14:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.224+0000 7f98c0fc0640 1 Processor -- start 2024-09-19T09:14:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.224+0000 7f98c0fc0640 1 -- start start 2024-09-19T09:14:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.224+0000 7f98c0fc0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98bc0ffbc0 0x7f98bc197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.224+0000 7f98c0fc0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98bc100580 0x7f98bc1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.224+0000 7f98c0fc0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98bc103150 0x7f98bc19f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.224+0000 7f98c0fc0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc10d7f0 con 0x7f98bc0ffbc0 2024-09-19T09:14:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.224+0000 7f98c0fc0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc10d670 con 0x7f98bc100580 2024-09-19T09:14:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.224+0000 7f98c0fc0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc10d970 con 0x7f98bc103150 2024-09-19T09:14:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.225+0000 7f98ba59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98bc0ffbc0 0x7f98bc197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.225+0000 7f98b9d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98bc100580 0x7f98bc1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.225+0000 7f98ba59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98bc0ffbc0 0x7f98bc197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47236/0 (socket says 172.21.15.120:47236) 2024-09-19T09:14:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.225+0000 7f98ba59c640 1 -- 172.21.15.120:0/1843753239 learned_addr learned my addr 172.21.15.120:0/1843753239 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.225+0000 7f98bad9d640 1 --2- 172.21.15.120:0/1843753239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98bc103150 0x7f98bc19f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.225+0000 7f98ba59c640 1 -- 172.21.15.120:0/1843753239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98bc103150 msgr2=0x7f98bc19f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.225+0000 7f98ba59c640 1 --2- 172.21.15.120:0/1843753239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98bc103150 0x7f98bc19f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.225+0000 7f98ba59c640 1 -- 172.21.15.120:0/1843753239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98bc100580 msgr2=0x7f98bc1982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.225+0000 7f98ba59c640 1 --2- 172.21.15.120:0/1843753239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98bc100580 0x7f98bc1982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.225+0000 7f98ba59c640 1 -- 172.21.15.120:0/1843753239 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98bc19f9d0 con 0x7f98bc0ffbc0 2024-09-19T09:14:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.225+0000 7f98ba59c640 1 --2- 172.21.15.120:0/1843753239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98bc0ffbc0 0x7f98bc197db0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f98b000d770 tx=0x7f98b000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.225+0000 7f98ab7fe640 1 -- 172.21.15.120:0/1843753239 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b0017070 con 0x7f98bc0ffbc0 2024-09-19T09:14:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.225+0000 7f98ab7fe640 1 -- 172.21.15.120:0/1843753239 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98b0009c30 con 0x7f98bc0ffbc0 2024-09-19T09:14:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.225+0000 7f98c0fc0640 1 -- 172.21.15.120:0/1843753239 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98bc19fcc0 con 0x7f98bc0ffbc0 2024-09-19T09:14:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.225+0000 7f98ab7fe640 1 -- 172.21.15.120:0/1843753239 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b0003dd0 con 0x7f98bc0ffbc0 2024-09-19T09:14:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.225+0000 7f98c0fc0640 1 -- 172.21.15.120:0/1843753239 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98bc1a01b0 con 0x7f98bc0ffbc0 2024-09-19T09:14:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.227+0000 7f98ab7fe640 1 -- 172.21.15.120:0/1843753239 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f98b0003fb0 con 0x7f98bc0ffbc0 2024-09-19T09:14:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.228+0000 7f98c3e2f740 1 Processor -- start 2024-09-19T09:14:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.228+0000 7f98c3e2f740 1 -- start start 2024-09-19T09:14:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.228+0000 7f98c3e2f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c40c679a0 0x561c40c67da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.228+0000 7f98c3e2f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c40c682e0 0x561c40c6cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.228+0000 7f98c3e2f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c40c6d370 0x561c40c6f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.228+0000 7f98c3e2f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561c40bcb2f0 con 0x561c40c6d370 2024-09-19T09:14:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.228+0000 7f98c3e2f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561c40b73270 con 0x561c40c682e0 2024-09-19T09:14:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.228+0000 7f98c3e2f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561c40bc72b0 con 0x561c40c679a0 2024-09-19T09:14:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.228+0000 7f98b9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c40c6d370 0x561c40c6f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.228+0000 7f98b9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c40c6d370 0x561c40c6f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47238/0 (socket says 172.21.15.120:47238) 2024-09-19T09:14:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.228+0000 7f98b9d9b640 1 -- 172.21.15.120:0/2191349963 learned_addr learned my addr 172.21.15.120:0/2191349963 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.228+0000 7f98ab7fe640 1 --2- 172.21.15.120:0/1843753239 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f98a40a3370 0x7f98a40a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.228+0000 7f98ba59c640 1 --2- 172.21.15.120:0/2191349963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c40c682e0 0x561c40c6cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.228+0000 7f98bad9d640 1 --2- 172.21.15.120:0/2191349963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c40c679a0 0x561c40c67da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.229+0000 7f98bad9d640 1 --2- 172.21.15.120:0/1843753239 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f98a40a3370 0x7f98a40a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.229+0000 7f98b9d9b640 1 -- 172.21.15.120:0/2191349963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c40c679a0 msgr2=0x561c40c67da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.229+0000 7f98b9d9b640 1 --2- 172.21.15.120:0/2191349963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c40c679a0 0x561c40c67da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.229+0000 7f98b9d9b640 1 -- 172.21.15.120:0/2191349963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c40c682e0 msgr2=0x561c40c6cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.229+0000 7f98b9d9b640 1 --2- 172.21.15.120:0/2191349963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c40c682e0 0x561c40c6cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.229+0000 7f98ab7fe640 1 -- 172.21.15.120:0/1843753239 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f98b00cf1f0 con 0x7f98bc0ffbc0 2024-09-19T09:14:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.229+0000 7f98b9d9b640 1 -- 172.21.15.120:0/2191349963 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c40b72340 con 0x561c40c6d370 2024-09-19T09:14:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.229+0000 7f98bad9d640 1 --2- 172.21.15.120:0/2191349963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c40c679a0 0x561c40c67da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:14:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.229+0000 7f98b9d9b640 1 --2- 172.21.15.120:0/2191349963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c40c6d370 0x561c40c6f830 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00b740 tx=0x7f98ac00bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547bb9f9bbf00d86 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.229+0000 7f98bad9d640 1 --2- 172.21.15.120:0/1843753239 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f98a40a3370 0x7f98a40a5830 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009430 tx=0x7f98b4004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.229+0000 7f98a8ff9640 1 -- 172.21.15.120:0/2191349963 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac003dd0 con 0x561c40c6d370 2024-09-19T09:14:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.229+0000 7f98a8ff9640 1 -- 172.21.15.120:0/2191349963 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98ac015070 con 0x561c40c6d370 2024-09-19T09:14:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.229+0000 7f98a8ff9640 1 -- 172.21.15.120:0/2191349963 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac011440 con 0x561c40c6d370 2024-09-19T09:14:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.230+0000 7f98c3e2f740 1 -- 172.21.15.120:0/2191349963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c40c6d370 msgr2=0x561c40c6f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.230+0000 7f98c3e2f740 1 --2- 172.21.15.120:0/2191349963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c40c6d370 0x561c40c6f830 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00b740 tx=0x7f98ac00bc00 comp rx=0 tx=0).stop 2024-09-19T09:14:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.230+0000 7f98c3e2f740 1 -- 172.21.15.120:0/2191349963 shutdown_connections 2024-09-19T09:14:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.230+0000 7f98c3e2f740 1 --2- 172.21.15.120:0/2191349963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c40c6d370 0x561c40c6f830 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.230+0000 7f98c3e2f740 1 --2- 172.21.15.120:0/2191349963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c40c682e0 0x561c40c6cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.230+0000 7f98c3e2f740 1 --2- 172.21.15.120:0/2191349963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c40c679a0 0x561c40c67da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.230+0000 7f98c3e2f740 1 -- 172.21.15.120:0/2191349963 >> 172.21.15.120:0/2191349963 conn(0x561c40c63010 msgr2=0x561c40c63410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.230+0000 7f98c3e2f740 1 -- 172.21.15.120:0/2191349963 shutdown_connections 2024-09-19T09:14:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.230+0000 7f98c3e2f740 1 -- 172.21.15.120:0/2191349963 wait complete. 2024-09-19T09:14:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.230+0000 7f98c3e2f740 1 -- start start 2024-09-19T09:14:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.230+0000 7f98c3e2f740 1 Processor -- start 2024-09-19T09:14:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.230+0000 7f98c3e2f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c40c679a0 0x561c40c71bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.230+0000 7f98c3e2f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c40c682e0 0x561c40c72110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.230+0000 7f98c3e2f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c40c6d370 0x561c40c79150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.230+0000 7f98c3e2f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561c40c70340 con 0x561c40c679a0 2024-09-19T09:14:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.230+0000 7f98c3e2f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561c40b54290 con 0x561c40c6d370 2024-09-19T09:14:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.230+0000 7f98c3e2f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561c40c704c0 con 0x561c40c682e0 2024-09-19T09:14:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.230+0000 7f98c3e2f740 10 client.0 _ll_register_callbacks cb 0x561c40b85140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.230+0000 7f98ba59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c40c679a0 0x561c40c71bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.231+0000 7f98ba59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c40c679a0 0x561c40c71bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47244/0 (socket says 172.21.15.120:47244) 2024-09-19T09:14:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.231+0000 7f98ba59c640 1 -- 172.21.15.120:0/550216731 learned_addr learned my addr 172.21.15.120:0/550216731 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.231+0000 7f98bad9d640 1 --2- 172.21.15.120:0/550216731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c40c6d370 0x561c40c79150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.231+0000 7f98b9d9b640 1 --2- 172.21.15.120:0/550216731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c40c682e0 0x561c40c72110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.231+0000 7f98ba59c640 1 -- 172.21.15.120:0/550216731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c40c682e0 msgr2=0x561c40c72110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.231+0000 7f98ba59c640 1 --2- 172.21.15.120:0/550216731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c40c682e0 0x561c40c72110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.231+0000 7f98ba59c640 1 -- 172.21.15.120:0/550216731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c40c6d370 msgr2=0x561c40c79150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.231+0000 7f98ba59c640 1 --2- 172.21.15.120:0/550216731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c40c6d370 0x561c40c79150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.231+0000 7f98ba59c640 1 -- 172.21.15.120:0/550216731 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c408de210 con 0x561c40c679a0 2024-09-19T09:14:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.231+0000 7f98ba59c640 1 --2- 172.21.15.120:0/550216731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c40c679a0 0x561c40c71bd0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f98b0097d20 tx=0x7f98b0004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2a2495b4e9c117 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.231+0000 7f98a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:14:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.231+0000 7f98a1ffb640 1 -- 172.21.15.120:0/550216731 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b0020040 con 0x561c40c679a0 2024-09-19T09:14:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.231+0000 7f98a1ffb640 1 -- 172.21.15.120:0/550216731 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f98b0097f30 con 0x561c40c679a0 2024-09-19T09:14:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.231+0000 7f98c3e2f740 1 -- 172.21.15.120:0/550216731 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98ac015070 con 0x561c40c679a0 2024-09-19T09:14:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.231+0000 7f98a1ffb640 1 -- 172.21.15.120:0/550216731 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f98b00860b0 con 0x561c40c679a0 2024-09-19T09:14:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.231+0000 7f98c3e2f740 10 client.19608 fetch_fsmap learned FSMap version 33 2024-09-19T09:14:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.231+0000 7f98c3e2f740 1 -- 172.21.15.120:0/550216731 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561c409aa620 con 0x561c40c679a0 2024-09-19T09:14:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.231+0000 7f98a1ffb640 1 -- 172.21.15.120:0/550216731 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98ac015070 con 0x561c40c679a0 2024-09-19T09:14:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.231+0000 7f98a1ffb640 1 -- 172.21.15.120:0/550216731 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f98b009ad10 con 0x561c40c679a0 2024-09-19T09:14:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.232+0000 7f98c3e2f740 10 client.19608 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:14:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.232+0000 7f98c3e2f740 10 client.19608 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:14:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.232+0000 7f98c3e2f740 20 client.19608 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.232+0000 7f98c3e2f740 1 --2- 172.21.15.120:0/550216731 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561c40c7d7f0 0x561c40c7fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.232+0000 7f98c3e2f740 4 client.19608 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:14:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.232+0000 7f98c3e2f740 1 -- 172.21.15.120:0/550216731 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c40bd8a20 con 0x561c40c7d7f0 2024-09-19T09:14:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.248+0000 7f98b9d9b640 1 --2- 172.21.15.120:0/550216731 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561c40c7d7f0 0x561c40c7fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.248+0000 7f98b9d9b640 1 --2- 172.21.15.120:0/550216731 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561c40c7d7f0 0x561c40c7fbe0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70cffd0f8a9fd0f1 server_cookie=ea002d6fbb50f2e4 in_seq=0 out_seq=0 2024-09-19T09:14:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.248+0000 7f98a1ffb640 10 client.19608 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.250+0000 7f98a1ffb640 1 -- 172.21.15.120:0/550216731 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c40bd8a20 con 0x561c40c7d7f0 2024-09-19T09:14:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.250+0000 7f98a1ffb640 10 client.19608 handle_command_reply: tid=0 2024-09-19T09:14:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.250+0000 7f98c3e2f740 1 client.19608 shutdown 2024-09-19T09:14:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.251+0000 7f98c3e2f740 1 -- 172.21.15.120:0/550216731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c40c679a0 msgr2=0x561c40c71bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.251+0000 7f98c3e2f740 1 --2- 172.21.15.120:0/550216731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c40c679a0 0x561c40c71bd0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f98b0097d20 tx=0x7f98b0004590 comp rx=0 tx=0).stop 2024-09-19T09:14:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.251+0000 7f98c3e2f740 1 -- 172.21.15.120:0/550216731 shutdown_connections 2024-09-19T09:14:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.251+0000 7f98c3e2f740 1 --2- 172.21.15.120:0/550216731 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561c40c7d7f0 0x561c40c7fbe0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.251+0000 7f98c3e2f740 1 --2- 172.21.15.120:0/550216731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c40c6d370 0x561c40c79150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.251+0000 7f98c3e2f740 1 --2- 172.21.15.120:0/550216731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c40c682e0 0x561c40c72110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.251+0000 7f98c3e2f740 1 --2- 172.21.15.120:0/550216731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c40c679a0 0x561c40c71bd0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.251+0000 7f98c3e2f740 1 -- 172.21.15.120:0/550216731 >> 172.21.15.120:0/550216731 conn(0x561c40c63010 msgr2=0x561c40c64820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.251+0000 7f98a1ffb640 0 client.19608 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.251+0000 7f98c3e2f740 1 -- 172.21.15.120:0/550216731 shutdown_connections 2024-09-19T09:14:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.251+0000 7f98c3e2f740 1 -- 172.21.15.120:0/550216731 wait complete. 2024-09-19T09:14:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.251+0000 7f98c3e2f740 20 client.19608 trim_cache size 0 max 16384 2024-09-19T09:14:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.267+0000 7f98c3e2f740 1 Processor -- start 2024-09-19T09:14:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.267+0000 7f98c3e2f740 1 -- start start 2024-09-19T09:14:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.267+0000 7f98c3e2f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c40c679a0 0x561c40c72090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.267+0000 7f98c3e2f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c40c682e0 0x561c40c79110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.267+0000 7f98c3e2f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c40c6d370 0x561c40c63ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.267+0000 7f98c3e2f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc1a8f50 con 0x561c40c682e0 2024-09-19T09:14:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.267+0000 7f98c3e2f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc1aaa90 con 0x561c40c6d370 2024-09-19T09:14:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.267+0000 7f98c3e2f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc1a9620 con 0x561c40c679a0 2024-09-19T09:14:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.267+0000 7f98b9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c40c682e0 0x561c40c79110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.267+0000 7f98b9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c40c682e0 0x561c40c79110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47260/0 (socket says 172.21.15.120:47260) 2024-09-19T09:14:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.267+0000 7f98b9d9b640 1 -- 172.21.15.120:0/3850743742 learned_addr learned my addr 172.21.15.120:0/3850743742 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.267+0000 7f98ba59c640 1 --2- 172.21.15.120:0/3850743742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c40c679a0 0x561c40c72090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.267+0000 7f98bad9d640 1 --2- 172.21.15.120:0/3850743742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c40c6d370 0x561c40c63ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.268+0000 7f98b9d9b640 1 -- 172.21.15.120:0/3850743742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c40c679a0 msgr2=0x561c40c72090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.268+0000 7f98b9d9b640 1 --2- 172.21.15.120:0/3850743742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c40c679a0 0x561c40c72090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.268+0000 7f98b9d9b640 1 -- 172.21.15.120:0/3850743742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c40c6d370 msgr2=0x561c40c63ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.268+0000 7f98b9d9b640 1 --2- 172.21.15.120:0/3850743742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c40c6d370 0x561c40c63ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.268+0000 7f98b9d9b640 1 -- 172.21.15.120:0/3850743742 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c40c71710 con 0x561c40c682e0 2024-09-19T09:14:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.268+0000 7f98ba59c640 1 --2- 172.21.15.120:0/3850743742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c40c679a0 0x561c40c72090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.268+0000 7f98b9d9b640 1 --2- 172.21.15.120:0/3850743742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c40c682e0 0x561c40c79110 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x561c40c63e90 tx=0x7f98ac00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1e93f6e4920239 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.268+0000 7f98a3fff640 1 -- 172.21.15.120:0/3850743742 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac004680 con 0x561c40c682e0 2024-09-19T09:14:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.268+0000 7f98a3fff640 1 -- 172.21.15.120:0/3850743742 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98ac015040 con 0x561c40c682e0 2024-09-19T09:14:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.268+0000 7f98a3fff640 1 -- 172.21.15.120:0/3850743742 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac027d70 con 0x561c40c682e0 2024-09-19T09:14:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.269+0000 7f98c3e2f740 1 -- 172.21.15.120:0/3850743742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c40c682e0 msgr2=0x561c40c79110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.269+0000 7f98c3e2f740 1 --2- 172.21.15.120:0/3850743742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c40c682e0 0x561c40c79110 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x561c40c63e90 tx=0x7f98ac00b390 comp rx=0 tx=0).stop 2024-09-19T09:14:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.269+0000 7f98c3e2f740 1 -- 172.21.15.120:0/3850743742 shutdown_connections 2024-09-19T09:14:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.269+0000 7f98c3e2f740 1 --2- 172.21.15.120:0/3850743742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c40c6d370 0x561c40c63ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.269+0000 7f98c3e2f740 1 --2- 172.21.15.120:0/3850743742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c40c682e0 0x561c40c79110 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.269+0000 7f98c3e2f740 1 --2- 172.21.15.120:0/3850743742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c40c679a0 0x561c40c72090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.269+0000 7f98c3e2f740 1 -- 172.21.15.120:0/3850743742 >> 172.21.15.120:0/3850743742 conn(0x561c40c63010 msgr2=0x561c40c71980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.269+0000 7f98c3e2f740 1 -- 172.21.15.120:0/3850743742 shutdown_connections 2024-09-19T09:14:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.269+0000 7f98c3e2f740 1 -- 172.21.15.120:0/3850743742 wait complete. 2024-09-19T09:14:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.269+0000 7f98c3e2f740 1 -- start start 2024-09-19T09:14:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.269+0000 7f98c3e2f740 1 Processor -- start 2024-09-19T09:14:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.269+0000 7f98c3e2f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c40c679a0 0x561c40c76230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.269+0000 7f98c3e2f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c40c682e0 0x561c40c76770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.269+0000 7f98c3e2f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c40c6d370 0x561c40c77780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.269+0000 7f98c3e2f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc1aa3c0 con 0x561c40c682e0 2024-09-19T09:14:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.269+0000 7f98c3e2f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc1a9cf0 con 0x561c40c679a0 2024-09-19T09:14:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.269+0000 7f98c3e2f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc1a6f10 con 0x561c40c6d370 2024-09-19T09:14:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.269+0000 7f98c3e2f740 10 client.0 _ll_register_callbacks cb 0x7f98b00046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.270+0000 7f98b9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c40c682e0 0x561c40c76770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.270+0000 7f98ba59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c40c679a0 0x561c40c76230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.270+0000 7f98b9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c40c682e0 0x561c40c76770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47276/0 (socket says 172.21.15.120:47276) 2024-09-19T09:14:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.270+0000 7f98ba59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c40c679a0 0x561c40c76230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:33006/0 (socket says 172.21.15.120:33006) 2024-09-19T09:14:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.270+0000 7f98b9d9b640 1 -- 172.21.15.120:0/2761542410 learned_addr learned my addr 172.21.15.120:0/2761542410 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.270+0000 7f98bad9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c40c6d370 0x561c40c77780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.270+0000 7f98b9d9b640 1 -- 172.21.15.120:0/2761542410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c40c6d370 msgr2=0x561c40c77780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.270+0000 7f98b9d9b640 1 --2- 172.21.15.120:0/2761542410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c40c6d370 0x561c40c77780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.270+0000 7f98b9d9b640 1 -- 172.21.15.120:0/2761542410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c40c679a0 msgr2=0x561c40c76230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.270+0000 7f98b9d9b640 1 --2- 172.21.15.120:0/2761542410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c40c679a0 0x561c40c76230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.270+0000 7f98b9d9b640 1 -- 172.21.15.120:0/2761542410 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c40c77d90 con 0x561c40c682e0 2024-09-19T09:14:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.270+0000 7f98ba59c640 1 --2- 172.21.15.120:0/2761542410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c40c679a0 0x561c40c76230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.270+0000 7f98b9d9b640 1 --2- 172.21.15.120:0/2761542410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c40c682e0 0x561c40c76770 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00b620 tx=0x7f98ac0043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549a411263e4a43a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.270+0000 7f98a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:14:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.270+0000 7f98a1ffb640 1 -- 172.21.15.120:0/2761542410 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac027670 con 0x561c40c682e0 2024-09-19T09:14:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.270+0000 7f98c3e2f740 1 -- 172.21.15.120:0/2761542410 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98ac015040 con 0x561c40c682e0 2024-09-19T09:14:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.270+0000 7f98a1ffb640 1 -- 172.21.15.120:0/2761542410 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f98ac027af0 con 0x561c40c682e0 2024-09-19T09:14:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.270+0000 7f98a1ffb640 1 -- 172.21.15.120:0/2761542410 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f98ac027cf0 con 0x561c40c682e0 2024-09-19T09:14:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.270+0000 7f98c3e2f740 10 client.19614 fetch_fsmap learned FSMap version 33 2024-09-19T09:14:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.270+0000 7f98c3e2f740 1 -- 172.21.15.120:0/2761542410 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561c40c790a0 con 0x561c40c682e0 2024-09-19T09:14:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.270+0000 7f98a1ffb640 1 -- 172.21.15.120:0/2761542410 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98ac015040 con 0x561c40c682e0 2024-09-19T09:14:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.271+0000 7f98a1ffb640 1 -- 172.21.15.120:0/2761542410 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f98ac030a40 con 0x561c40c682e0 2024-09-19T09:14:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.271+0000 7f98c3e2f740 10 client.19614 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:14:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.271+0000 7f98c3e2f740 10 client.19614 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:14:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.271+0000 7f98c3e2f740 20 client.19614 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.271+0000 7f98c3e2f740 1 --2- 172.21.15.120:0/2761542410 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561c40ce85c0 0x561c40cea9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.271+0000 7f98c3e2f740 4 client.19614 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:14:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.271+0000 7f98c3e2f740 1 -- 172.21.15.120:0/2761542410 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561c40bd2d70 con 0x561c40ce85c0 2024-09-19T09:14:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.271+0000 7f98ba59c640 1 --2- 172.21.15.120:0/2761542410 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561c40ce85c0 0x561c40cea9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.271+0000 7f98ba59c640 1 --2- 172.21.15.120:0/2761542410 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561c40ce85c0 0x561c40cea9b0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=948944ea2df44f31 server_cookie=8240e83cdd9c683 in_seq=0 out_seq=0 2024-09-19T09:14:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.271+0000 7f98a1ffb640 10 client.19614 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.520+0000 7f98a1ffb640 1 -- 172.21.15.120:0/2761542410 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561c40bd2d70 con 0x561c40ce85c0 2024-09-19T09:14:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.520+0000 7f98a1ffb640 10 client.19614 handle_command_reply: tid=0 2024-09-19T09:14:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.520+0000 7f98c3e2f740 1 client.19614 shutdown 2024-09-19T09:14:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.520+0000 7f98c3e2f740 1 -- 172.21.15.120:0/2761542410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c40c682e0 msgr2=0x561c40c76770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.520+0000 7f98c3e2f740 1 --2- 172.21.15.120:0/2761542410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c40c682e0 0x561c40c76770 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00b620 tx=0x7f98ac0043d0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.520+0000 7f98c3e2f740 1 -- 172.21.15.120:0/2761542410 shutdown_connections 2024-09-19T09:14:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.520+0000 7f98c3e2f740 1 --2- 172.21.15.120:0/2761542410 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561c40ce85c0 0x561c40cea9b0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.520+0000 7f98c3e2f740 1 --2- 172.21.15.120:0/2761542410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c40c6d370 0x561c40c77780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.520+0000 7f98c3e2f740 1 --2- 172.21.15.120:0/2761542410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c40c682e0 0x561c40c76770 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.520+0000 7f98c3e2f740 1 --2- 172.21.15.120:0/2761542410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c40c679a0 0x561c40c76230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.520+0000 7f98c3e2f740 1 -- 172.21.15.120:0/2761542410 >> 172.21.15.120:0/2761542410 conn(0x561c40c63010 msgr2=0x561c40c619a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.520+0000 7f98a1ffb640 0 client.19614 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.520+0000 7f98c3e2f740 1 -- 172.21.15.120:0/2761542410 shutdown_connections 2024-09-19T09:14:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.520+0000 7f98c3e2f740 1 -- 172.21.15.120:0/2761542410 wait complete. 2024-09-19T09:14:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.520+0000 7f98c3e2f740 20 client.19614 trim_cache size 0 max 16384 2024-09-19T09:14:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:14:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:14:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:14:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:14:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.521+0000 7f98c0fc0640 1 -- 172.21.15.120:0/1843753239 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f98a40a3370 msgr2=0x7f98a40a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.521+0000 7f98c0fc0640 1 --2- 172.21.15.120:0/1843753239 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f98a40a3370 0x7f98a40a5830 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009430 tx=0x7f98b4004a40 comp rx=0 tx=0).stop 2024-09-19T09:14:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.521+0000 7f98c0fc0640 1 -- 172.21.15.120:0/1843753239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98bc0ffbc0 msgr2=0x7f98bc197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.521+0000 7f98c0fc0640 1 --2- 172.21.15.120:0/1843753239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98bc0ffbc0 0x7f98bc197db0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f98b000d770 tx=0x7f98b000dc30 comp rx=0 tx=0).stop 2024-09-19T09:14:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.521+0000 7f98c0fc0640 1 -- 172.21.15.120:0/1843753239 shutdown_connections 2024-09-19T09:14:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.521+0000 7f98c0fc0640 1 --2- 172.21.15.120:0/1843753239 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f98a40a3370 0x7f98a40a5830 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.521+0000 7f98c0fc0640 1 --2- 172.21.15.120:0/1843753239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98bc103150 0x7f98bc19f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.521+0000 7f98c0fc0640 1 --2- 172.21.15.120:0/1843753239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98bc100580 0x7f98bc1982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.521+0000 7f98c0fc0640 1 --2- 172.21.15.120:0/1843753239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98bc0ffbc0 0x7f98bc197db0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.521+0000 7f98c0fc0640 1 -- 172.21.15.120:0/1843753239 >> 172.21.15.120:0/1843753239 conn(0x7f98bc0fb970 msgr2=0x7f98bc1061e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.521+0000 7f98c0fc0640 1 -- 172.21.15.120:0/1843753239 shutdown_connections 2024-09-19T09:14:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:44.521+0000 7f98c0fc0640 1 -- 172.21.15.120:0/1843753239 wait complete. 2024-09-19T09:14:45.075 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1149783 9.62 MB/sec execute 520 sec latency 178.148 ms 2024-09-19T09:14:45.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:44 smithi196 ceph-mon[35150]: pgmap v700: 97 pgs: 97 active+clean; 52 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.0 MiB/s wr, 429 op/s 2024-09-19T09:14:45.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:44 smithi153 ceph-mon[35071]: pgmap v700: 97 pgs: 97 active+clean; 52 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.0 MiB/s wr, 429 op/s 2024-09-19T09:14:45.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:44 smithi120 ceph-mon[31150]: pgmap v700: 97 pgs: 97 active+clean; 52 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.0 MiB/s wr, 429 op/s 2024-09-19T09:14:46.075 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1152245 9.63 MB/sec execute 521 sec latency 13.565 ms 2024-09-19T09:14:47.075 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1154972 9.64 MB/sec execute 522 sec latency 28.862 ms 2024-09-19T09:14:47.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:46 smithi196 ceph-mon[35150]: pgmap v701: 97 pgs: 97 active+clean; 51 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.4 MiB/s wr, 328 op/s 2024-09-19T09:14:47.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:46 smithi153 ceph-mon[35071]: pgmap v701: 97 pgs: 97 active+clean; 51 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.4 MiB/s wr, 328 op/s 2024-09-19T09:14:47.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:46 smithi120 ceph-mon[31150]: pgmap v701: 97 pgs: 97 active+clean; 51 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.4 MiB/s wr, 328 op/s 2024-09-19T09:14:48.075 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1158182 9.66 MB/sec execute 523 sec latency 11.140 ms 2024-09-19T09:14:49.075 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1160396 9.66 MB/sec execute 524 sec latency 48.528 ms 2024-09-19T09:14:49.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:48 smithi196 ceph-mon[35150]: pgmap v702: 97 pgs: 97 active+clean; 42 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.0 MiB/s wr, 428 op/s 2024-09-19T09:14:49.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:48 smithi153 ceph-mon[35071]: pgmap v702: 97 pgs: 97 active+clean; 42 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.0 MiB/s wr, 428 op/s 2024-09-19T09:14:49.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:48 smithi120 ceph-mon[31150]: pgmap v702: 97 pgs: 97 active+clean; 42 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.0 MiB/s wr, 428 op/s 2024-09-19T09:14:50.075 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1162582 9.66 MB/sec execute 525 sec latency 17.129 ms 2024-09-19T09:14:51.020 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:14:51.075 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1165416 9.68 MB/sec execute 526 sec latency 7.956 ms 2024-09-19T09:14:51.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:50 smithi196 ceph-mon[35150]: pgmap v703: 97 pgs: 97 active+clean; 42 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.6 MiB/s wr, 403 op/s 2024-09-19T09:14:51.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:50 smithi153 ceph-mon[35071]: pgmap v703: 97 pgs: 97 active+clean; 42 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.6 MiB/s wr, 403 op/s 2024-09-19T09:14:51.279 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:14:51.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:50 smithi120 ceph-mon[31150]: pgmap v703: 97 pgs: 97 active+clean; 42 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.6 MiB/s wr, 403 op/s 2024-09-19T09:14:52.075 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1168366 9.69 MB/sec execute 527 sec latency 42.276 ms 2024-09-19T09:14:52.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.101+0000 7f6ecd483640 1 -- 172.21.15.120:0/2407280297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ec81082a0 msgr2=0x7f6ec810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:52.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.101+0000 7f6ecd483640 1 --2- 172.21.15.120:0/2407280297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ec81082a0 0x7f6ec810ec40 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0009cd0 tx=0x7f6ec002a8e0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.101+0000 7f6ecd483640 1 -- 172.21.15.120:0/2407280297 shutdown_connections 2024-09-19T09:14:52.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.101+0000 7f6ecd483640 1 --2- 172.21.15.120:0/2407280297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ec81082a0 0x7f6ec810ec40 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.101+0000 7f6ecd483640 1 --2- 172.21.15.120:0/2407280297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ec81036e0 0x7f6ec8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.101+0000 7f6ecd483640 1 --2- 172.21.15.120:0/2407280297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ec8102d30 0x7f6ec8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.101+0000 7f6ecd483640 1 -- 172.21.15.120:0/2407280297 >> 172.21.15.120:0/2407280297 conn(0x7f6ec80fc7d0 msgr2=0x7f6ec80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:52.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.101+0000 7f6ecd483640 1 -- 172.21.15.120:0/2407280297 shutdown_connections 2024-09-19T09:14:52.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.101+0000 7f6ecd483640 1 -- 172.21.15.120:0/2407280297 wait complete. 2024-09-19T09:14:52.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.101+0000 7f6ecd483640 1 Processor -- start 2024-09-19T09:14:52.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.102+0000 7f6ecd483640 1 -- start start 2024-09-19T09:14:52.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.102+0000 7f6ecd483640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ec8102d30 0x7f6ec81a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:52.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.102+0000 7f6ecd483640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ec81036e0 0x7f6ec81a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:52.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.102+0000 7f6ecd483640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ec81082a0 0x7f6ec819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:52.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.102+0000 7f6ecd483640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec81120d0 con 0x7f6ec81082a0 2024-09-19T09:14:52.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.102+0000 7f6ecd483640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec8111f50 con 0x7f6ec81036e0 2024-09-19T09:14:52.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.102+0000 7f6ecd483640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec8112250 con 0x7f6ec8102d30 2024-09-19T09:14:52.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.102+0000 7f6ebe7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ec81036e0 0x7f6ec81a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:52.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.102+0000 7f6ec77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ec81082a0 0x7f6ec819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:52.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.102+0000 7f6ec77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ec81082a0 0x7f6ec819a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37838/0 (socket says 172.21.15.120:37838) 2024-09-19T09:14:52.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.102+0000 7f6ebe7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ec81036e0 0x7f6ec81a0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:41032/0 (socket says 172.21.15.120:41032) 2024-09-19T09:14:52.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.102+0000 7f6ec77fe640 1 -- 172.21.15.120:0/2446419875 learned_addr learned my addr 172.21.15.120:0/2446419875 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:52.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.102+0000 7f6ec6ffd640 1 --2- 172.21.15.120:0/2446419875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ec8102d30 0x7f6ec81a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:52.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.102+0000 7f6ec77fe640 1 -- 172.21.15.120:0/2446419875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ec8102d30 msgr2=0x7f6ec81a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:52.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.102+0000 7f6ec77fe640 1 --2- 172.21.15.120:0/2446419875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ec8102d30 0x7f6ec81a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.102+0000 7f6ec77fe640 1 -- 172.21.15.120:0/2446419875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ec81036e0 msgr2=0x7f6ec81a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:52.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.102+0000 7f6ec77fe640 1 --2- 172.21.15.120:0/2446419875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ec81036e0 0x7f6ec81a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.102+0000 7f6ec77fe640 1 -- 172.21.15.120:0/2446419875 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec819ad00 con 0x7f6ec81082a0 2024-09-19T09:14:52.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.102+0000 7f6ebe7fc640 1 --2- 172.21.15.120:0/2446419875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ec81036e0 0x7f6ec81a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:52.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.103+0000 7f6ec77fe640 1 --2- 172.21.15.120:0/2446419875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ec81082a0 0x7f6ec819a5d0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0033040 tx=0x7f6ec0031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:52.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.103+0000 7f6ec4ff9640 1 -- 172.21.15.120:0/2446419875 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0004040 con 0x7f6ec81082a0 2024-09-19T09:14:52.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.103+0000 7f6ec4ff9640 1 -- 172.21.15.120:0/2446419875 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ec00046c0 con 0x7f6ec81082a0 2024-09-19T09:14:52.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.103+0000 7f6ecd483640 1 -- 172.21.15.120:0/2446419875 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ec819af90 con 0x7f6ec81082a0 2024-09-19T09:14:52.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.103+0000 7f6ec4ff9640 1 -- 172.21.15.120:0/2446419875 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0046480 con 0x7f6ec81082a0 2024-09-19T09:14:52.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.103+0000 7f6ecd483640 1 -- 172.21.15.120:0/2446419875 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ec81a72e0 con 0x7f6ec81082a0 2024-09-19T09:14:52.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.106+0000 7f6ec4ff9640 1 -- 172.21.15.120:0/2446419875 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f6ec00305a0 con 0x7f6ec81082a0 2024-09-19T09:14:52.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.106+0000 7f6ecf85c740 1 Processor -- start 2024-09-19T09:14:52.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.106+0000 7f6ecf85c740 1 -- start start 2024-09-19T09:14:52.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.106+0000 7f6ecf85c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb396fa4a0 0x55fb396fa880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:52.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.106+0000 7f6ecf85c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb396fadc0 0x55fb39705140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:52.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.106+0000 7f6ecf85c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb39705680 0x55fb39707a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:52.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.106+0000 7f6ecf85c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fb396602e0 con 0x55fb39705680 2024-09-19T09:14:52.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.106+0000 7f6ecf85c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fb39607ce0 con 0x55fb396fadc0 2024-09-19T09:14:52.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.106+0000 7f6ecf85c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fb3965c2a0 con 0x55fb396fa4a0 2024-09-19T09:14:52.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.106+0000 7f6ec6ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb396fadc0 0x55fb39705140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:52.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.106+0000 7f6ebe7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb39705680 0x55fb39707a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:52.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.106+0000 7f6ec6ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb396fadc0 0x55fb39705140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:41038/0 (socket says 172.21.15.120:41038) 2024-09-19T09:14:52.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.106+0000 7f6ec6ffd640 1 -- 172.21.15.120:0/1030561136 learned_addr learned my addr 172.21.15.120:0/1030561136 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:52.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.106+0000 7f6ebe7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb39705680 0x55fb39707a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37848/0 (socket says 172.21.15.120:37848) 2024-09-19T09:14:52.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.107+0000 7f6ebe7fc640 1 --2- 172.21.15.120:0/1030561136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb396fa4a0 0x55fb396fa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:52.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.107+0000 7f6ec4ff9640 1 --2- 172.21.15.120:0/2446419875 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6eb40a3240 0x7f6eb40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:52.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.107+0000 7f6ec6ffd640 1 -- 172.21.15.120:0/1030561136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb396fa4a0 msgr2=0x55fb396fa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:52.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.107+0000 7f6ec6ffd640 1 --2- 172.21.15.120:0/1030561136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb396fa4a0 0x55fb396fa880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.107+0000 7f6ec6ffd640 1 -- 172.21.15.120:0/1030561136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb39705680 msgr2=0x55fb39707a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:52.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.107+0000 7f6ec6ffd640 1 --2- 172.21.15.120:0/1030561136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb39705680 0x55fb39707a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.107+0000 7f6ec6ffd640 1 -- 172.21.15.120:0/1030561136 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb395ec440 con 0x55fb396fadc0 2024-09-19T09:14:52.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.107+0000 7f6ec77fe640 1 --2- 172.21.15.120:0/2446419875 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6eb40a3240 0x7f6eb40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:52.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.107+0000 7f6ec4ff9640 1 -- 172.21.15.120:0/2446419875 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f6ec002e030 con 0x7f6ec81082a0 2024-09-19T09:14:52.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.107+0000 7f6ec6ffd640 1 --2- 172.21.15.120:0/1030561136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb396fadc0 0x55fb39705140 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8009c80 tx=0x7f6eb80117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=948fb53cfb4afe90 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:52.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.107+0000 7f6ebd7fa640 1 -- 172.21.15.120:0/1030561136 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8012450 con 0x55fb396fadc0 2024-09-19T09:14:52.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.107+0000 7f6ebd7fa640 1 -- 172.21.15.120:0/1030561136 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eb8012ad0 con 0x55fb396fadc0 2024-09-19T09:14:52.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.107+0000 7f6ec77fe640 1 --2- 172.21.15.120:0/2446419875 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6eb40a3240 0x7f6eb40a5700 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0088420 tx=0x7f6ec00904a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:52.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.107+0000 7f6ebd7fa640 1 -- 172.21.15.120:0/1030561136 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb800fc30 con 0x55fb396fadc0 2024-09-19T09:14:52.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.108+0000 7f6ecf85c740 1 -- 172.21.15.120:0/1030561136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb396fadc0 msgr2=0x55fb39705140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:52.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.108+0000 7f6ecf85c740 1 --2- 172.21.15.120:0/1030561136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb396fadc0 0x55fb39705140 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8009c80 tx=0x7f6eb80117b0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.108+0000 7f6ecf85c740 1 -- 172.21.15.120:0/1030561136 shutdown_connections 2024-09-19T09:14:52.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.108+0000 7f6ecf85c740 1 --2- 172.21.15.120:0/1030561136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb39705680 0x55fb39707a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.108+0000 7f6ecf85c740 1 --2- 172.21.15.120:0/1030561136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb396fadc0 0x55fb39705140 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.108+0000 7f6ecf85c740 1 --2- 172.21.15.120:0/1030561136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb396fa4a0 0x55fb396fa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.108+0000 7f6ecf85c740 1 -- 172.21.15.120:0/1030561136 >> 172.21.15.120:0/1030561136 conn(0x55fb396f6a30 msgr2=0x55fb396f6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:52.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.108+0000 7f6ecf85c740 1 -- 172.21.15.120:0/1030561136 shutdown_connections 2024-09-19T09:14:52.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.108+0000 7f6ecf85c740 1 -- 172.21.15.120:0/1030561136 wait complete. 2024-09-19T09:14:52.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.108+0000 7f6ecf85c740 1 -- start start 2024-09-19T09:14:52.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.109+0000 7f6ecf85c740 1 Processor -- start 2024-09-19T09:14:52.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.109+0000 7f6ecf85c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb396fa4a0 0x55fb39704f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:52.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.109+0000 7f6ecf85c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb396fadc0 0x55fb39702220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:52.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.109+0000 7f6ecf85c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb39705680 0x55fb39702760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:52.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.109+0000 7f6ecf85c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fb39708250 con 0x55fb39705680 2024-09-19T09:14:52.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.109+0000 7f6ecf85c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fb3961b050 con 0x55fb396fadc0 2024-09-19T09:14:52.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.109+0000 7f6ecf85c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fb397083d0 con 0x55fb396fa4a0 2024-09-19T09:14:52.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.109+0000 7f6ecf85c740 10 client.0 _ll_register_callbacks cb 0x55fb3965b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:52.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.109+0000 7f6ebe7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb396fa4a0 0x55fb39704f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:52.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.109+0000 7f6ec6ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb396fadc0 0x55fb39702220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:52.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.109+0000 7f6ec6ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb396fadc0 0x55fb39702220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:41044/0 (socket says 172.21.15.120:41044) 2024-09-19T09:14:52.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.109+0000 7f6ebe7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb396fa4a0 0x55fb39704f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:57988/0 (socket says 172.21.15.120:57988) 2024-09-19T09:14:52.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.109+0000 7f6ec6ffd640 1 -- 172.21.15.120:0/1975088391 learned_addr learned my addr 172.21.15.120:0/1975088391 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:52.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.109+0000 7f6ebe7fc640 1 --2- 172.21.15.120:0/1975088391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb39705680 0x55fb39702760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:52.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.109+0000 7f6ec6ffd640 1 -- 172.21.15.120:0/1975088391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb396fa4a0 msgr2=0x55fb39704f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:52.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.109+0000 7f6ec6ffd640 1 --2- 172.21.15.120:0/1975088391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb396fa4a0 0x55fb39704f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.109+0000 7f6ec6ffd640 1 -- 172.21.15.120:0/1975088391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb39705680 msgr2=0x55fb39702760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:52.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.109+0000 7f6ec6ffd640 1 --2- 172.21.15.120:0/1975088391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb39705680 0x55fb39702760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.109+0000 7f6ec6ffd640 1 -- 172.21.15.120:0/1975088391 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb393d13c0 con 0x55fb396fadc0 2024-09-19T09:14:52.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.110+0000 7f6ec6ffd640 1 --2- 172.21.15.120:0/1975088391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb396fadc0 0x55fb39702220 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8011cb0 tx=0x7f6eb8005890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ea8ebb0cd711bdc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:52.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.110+0000 7f6eae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:14:52.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.110+0000 7f6eae7fc640 1 -- 172.21.15.120:0/1975088391 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8028020 con 0x55fb396fadc0 2024-09-19T09:14:52.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.110+0000 7f6eae7fc640 1 -- 172.21.15.120:0/1975088391 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f6eb8012450 con 0x55fb396fadc0 2024-09-19T09:14:52.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.110+0000 7f6ecf85c740 1 -- 172.21.15.120:0/1975088391 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6eb8012ad0 con 0x55fb396fadc0 2024-09-19T09:14:52.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.110+0000 7f6eae7fc640 1 -- 172.21.15.120:0/1975088391 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6eb8009860 con 0x55fb396fadc0 2024-09-19T09:14:52.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.110+0000 7f6ecf85c740 10 client.28679 fetch_fsmap learned FSMap version 33 2024-09-19T09:14:52.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.110+0000 7f6ecf85c740 1 -- 172.21.15.120:0/1975088391 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fb39713bc0 con 0x55fb396fadc0 2024-09-19T09:14:52.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.110+0000 7f6eae7fc640 1 -- 172.21.15.120:0/1975088391 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eb8012ad0 con 0x55fb396fadc0 2024-09-19T09:14:52.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.110+0000 7f6eae7fc640 1 -- 172.21.15.120:0/1975088391 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f6eb80035f0 con 0x55fb396fadc0 2024-09-19T09:14:52.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.110+0000 7f6ecf85c740 10 client.28679 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:14:52.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.110+0000 7f6ecf85c740 10 client.28679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:14:52.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.110+0000 7f6ecf85c740 20 client.28679 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:52.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.110+0000 7f6ecf85c740 1 --2- 172.21.15.120:0/1975088391 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fb39714f70 0x55fb39717360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:52.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.110+0000 7f6ecf85c740 4 client.28679 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:14:52.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.110+0000 7f6ecf85c740 1 -- 172.21.15.120:0/1975088391 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb3966d850 con 0x55fb39714f70 2024-09-19T09:14:52.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.111+0000 7f6ebe7fc640 1 --2- 172.21.15.120:0/1975088391 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fb39714f70 0x55fb39717360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:52.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.111+0000 7f6eae7fc640 10 client.28679 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:52.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.111+0000 7f6ebe7fc640 1 --2- 172.21.15.120:0/1975088391 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fb39714f70 0x55fb39717360 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a74beed9b72f102a server_cookie=7f6f8288e609a190 in_seq=0 out_seq=0 2024-09-19T09:14:52.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.112+0000 7f6eae7fc640 1 -- 172.21.15.120:0/1975088391 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb3966d850 con 0x55fb39714f70 2024-09-19T09:14:52.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.112+0000 7f6eae7fc640 10 client.28679 handle_command_reply: tid=0 2024-09-19T09:14:52.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.112+0000 7f6ecf85c740 1 client.28679 shutdown 2024-09-19T09:14:52.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.112+0000 7f6ecf85c740 1 -- 172.21.15.120:0/1975088391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb396fadc0 msgr2=0x55fb39702220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:52.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.112+0000 7f6ecf85c740 1 --2- 172.21.15.120:0/1975088391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb396fadc0 0x55fb39702220 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8011cb0 tx=0x7f6eb8005890 comp rx=0 tx=0).stop 2024-09-19T09:14:52.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.112+0000 7f6ecf85c740 1 -- 172.21.15.120:0/1975088391 shutdown_connections 2024-09-19T09:14:52.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.112+0000 7f6ecf85c740 1 --2- 172.21.15.120:0/1975088391 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fb39714f70 0x55fb39717360 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.112+0000 7f6ecf85c740 1 --2- 172.21.15.120:0/1975088391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb39705680 0x55fb39702760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.112+0000 7f6ecf85c740 1 --2- 172.21.15.120:0/1975088391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb396fadc0 0x55fb39702220 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.112+0000 7f6eae7fc640 0 client.28679 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:52.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.112+0000 7f6ecf85c740 1 --2- 172.21.15.120:0/1975088391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb396fa4a0 0x55fb39704f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.112+0000 7f6ecf85c740 1 -- 172.21.15.120:0/1975088391 >> 172.21.15.120:0/1975088391 conn(0x55fb396f6a30 msgr2=0x55fb396fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:52.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.112+0000 7f6ecf85c740 1 -- 172.21.15.120:0/1975088391 shutdown_connections 2024-09-19T09:14:52.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.112+0000 7f6ecf85c740 1 -- 172.21.15.120:0/1975088391 wait complete. 2024-09-19T09:14:52.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.112+0000 7f6ecf85c740 20 client.28679 trim_cache size 0 max 16384 2024-09-19T09:14:52.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.129+0000 7f6ecf85c740 1 Processor -- start 2024-09-19T09:14:52.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.130+0000 7f6ecf85c740 1 -- start start 2024-09-19T09:14:52.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.130+0000 7f6ecf85c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb396fa4a0 0x55fb397114b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:52.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.130+0000 7f6ecf85c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb396fadc0 0x55fb39700af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:52.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.130+0000 7f6ecf85c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb39705680 0x55fb3971c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:52.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.130+0000 7f6ecf85c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec81af900 con 0x55fb396fa4a0 2024-09-19T09:14:52.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.130+0000 7f6ecf85c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec81b1440 con 0x55fb39705680 2024-09-19T09:14:52.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.130+0000 7f6ecf85c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec81affd0 con 0x55fb396fadc0 2024-09-19T09:14:52.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.130+0000 7f6ebe7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb396fa4a0 0x55fb397114b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:52.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.130+0000 7f6ec6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb396fadc0 0x55fb39700af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:52.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.130+0000 7f6ebe7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb396fa4a0 0x55fb397114b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37862/0 (socket says 172.21.15.120:37862) 2024-09-19T09:14:52.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.130+0000 7f6ec6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb396fadc0 0x55fb39700af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:57992/0 (socket says 172.21.15.120:57992) 2024-09-19T09:14:52.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.130+0000 7f6ebe7fc640 1 -- 172.21.15.120:0/573803679 learned_addr learned my addr 172.21.15.120:0/573803679 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:52.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.130+0000 7f6ebe7fc640 1 --2- 172.21.15.120:0/573803679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb39705680 0x55fb3971c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:52.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.130+0000 7f6ebe7fc640 1 -- 172.21.15.120:0/573803679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb396fadc0 msgr2=0x55fb39700af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:52.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.130+0000 7f6ebe7fc640 1 --2- 172.21.15.120:0/573803679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb396fadc0 0x55fb39700af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.130+0000 7f6ebe7fc640 1 -- 172.21.15.120:0/573803679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb39705680 msgr2=0x55fb3971c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:52.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.130+0000 7f6ebe7fc640 1 --2- 172.21.15.120:0/573803679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb39705680 0x55fb3971c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.130+0000 7f6ebe7fc640 1 -- 172.21.15.120:0/573803679 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb397052e0 con 0x55fb396fa4a0 2024-09-19T09:14:52.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.130+0000 7f6ec6ffd640 1 --2- 172.21.15.120:0/573803679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb396fadc0 0x55fb39700af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:52.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.131+0000 7f6ebe7fc640 1 --2- 172.21.15.120:0/573803679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb396fa4a0 0x55fb397114b0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0014010 tx=0x7f6eb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948dcdca014e3e50 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:52.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.131+0000 7f6ebcff9640 1 -- 172.21.15.120:0/573803679 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0005590 con 0x55fb396fa4a0 2024-09-19T09:14:52.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.131+0000 7f6ebcff9640 1 -- 172.21.15.120:0/573803679 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eb0004c00 con 0x55fb396fa4a0 2024-09-19T09:14:52.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.131+0000 7f6ebcff9640 1 -- 172.21.15.120:0/573803679 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0003c20 con 0x55fb396fa4a0 2024-09-19T09:14:52.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.132+0000 7f6ecf85c740 1 -- 172.21.15.120:0/573803679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb396fa4a0 msgr2=0x55fb397114b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:52.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.132+0000 7f6ecf85c740 1 --2- 172.21.15.120:0/573803679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb396fa4a0 0x55fb397114b0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0014010 tx=0x7f6eb00046c0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.132+0000 7f6ecf85c740 1 -- 172.21.15.120:0/573803679 shutdown_connections 2024-09-19T09:14:52.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.132+0000 7f6ecf85c740 1 --2- 172.21.15.120:0/573803679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb39705680 0x55fb3971c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.132+0000 7f6ecf85c740 1 --2- 172.21.15.120:0/573803679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb396fadc0 0x55fb39700af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.132+0000 7f6ecf85c740 1 --2- 172.21.15.120:0/573803679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb396fa4a0 0x55fb397114b0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.132+0000 7f6ecf85c740 1 -- 172.21.15.120:0/573803679 >> 172.21.15.120:0/573803679 conn(0x55fb396f6a30 msgr2=0x55fb397042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:52.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.132+0000 7f6ecf85c740 1 -- 172.21.15.120:0/573803679 shutdown_connections 2024-09-19T09:14:52.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.132+0000 7f6ecf85c740 1 -- 172.21.15.120:0/573803679 wait complete. 2024-09-19T09:14:52.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.132+0000 7f6ecf85c740 1 -- start start 2024-09-19T09:14:52.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.132+0000 7f6ecf85c740 1 Processor -- start 2024-09-19T09:14:52.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.132+0000 7f6ecf85c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb396fa4a0 0x55fb3970dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:52.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.132+0000 7f6ecf85c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb396fadc0 0x55fb39708fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:52.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.132+0000 7f6ecf85c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb39705680 0x55fb39709510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:52.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.132+0000 7f6ecf85c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec81b0d70 con 0x55fb396fa4a0 2024-09-19T09:14:52.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.132+0000 7f6ecf85c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec81b06a0 con 0x55fb396fadc0 2024-09-19T09:14:52.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.132+0000 7f6ecf85c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec81ad8c0 con 0x55fb39705680 2024-09-19T09:14:52.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.132+0000 7f6ecf85c740 10 client.0 _ll_register_callbacks cb 0x7f6eb000bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:52.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.132+0000 7f6ebe7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb396fa4a0 0x55fb3970dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:52.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.132+0000 7f6ebe7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb396fa4a0 0x55fb3970dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37868/0 (socket says 172.21.15.120:37868) 2024-09-19T09:14:52.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.132+0000 7f6ebe7fc640 1 -- 172.21.15.120:0/1985608984 learned_addr learned my addr 172.21.15.120:0/1985608984 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:52.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.133+0000 7f6ec6ffd640 1 --2- 172.21.15.120:0/1985608984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb396fadc0 0x55fb39708fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:52.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.133+0000 7f6ebe7fc640 1 --2- 172.21.15.120:0/1985608984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb39705680 0x55fb39709510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:52.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.133+0000 7f6ebe7fc640 1 -- 172.21.15.120:0/1985608984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb39705680 msgr2=0x55fb39709510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:52.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.133+0000 7f6ebe7fc640 1 --2- 172.21.15.120:0/1985608984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb39705680 0x55fb39709510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.133+0000 7f6ebe7fc640 1 -- 172.21.15.120:0/1985608984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb396fadc0 msgr2=0x55fb39708fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:52.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.133+0000 7f6ebe7fc640 1 --2- 172.21.15.120:0/1985608984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb396fadc0 0x55fb39708fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.133+0000 7f6ebe7fc640 1 -- 172.21.15.120:0/1985608984 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb39709c00 con 0x55fb396fa4a0 2024-09-19T09:14:52.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.133+0000 7f6ebe7fc640 1 --2- 172.21.15.120:0/1985608984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb396fa4a0 0x55fb3970dc50 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000bad0 tx=0x7f6eb00038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d61c11cc415723d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:52.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.133+0000 7f6eae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:14:52.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.133+0000 7f6eae7fc640 1 -- 172.21.15.120:0/1985608984 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0032040 con 0x55fb396fa4a0 2024-09-19T09:14:52.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.133+0000 7f6eae7fc640 1 -- 172.21.15.120:0/1985608984 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f6eb0004db0 con 0x55fb396fa4a0 2024-09-19T09:14:52.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.133+0000 7f6ecf85c740 1 -- 172.21.15.120:0/1985608984 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6eb0004c00 con 0x55fb396fa4a0 2024-09-19T09:14:52.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.133+0000 7f6eae7fc640 1 -- 172.21.15.120:0/1985608984 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6eb0010510 con 0x55fb396fa4a0 2024-09-19T09:14:52.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.133+0000 7f6ecf85c740 10 client.19632 fetch_fsmap learned FSMap version 33 2024-09-19T09:14:52.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.133+0000 7f6ecf85c740 1 -- 172.21.15.120:0/1985608984 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fb3977f840 con 0x55fb396fa4a0 2024-09-19T09:14:52.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.133+0000 7f6eae7fc640 1 -- 172.21.15.120:0/1985608984 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eb0004c00 con 0x55fb396fa4a0 2024-09-19T09:14:52.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.134+0000 7f6eae7fc640 1 -- 172.21.15.120:0/1985608984 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f6eb002c070 con 0x55fb396fa4a0 2024-09-19T09:14:52.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.134+0000 7f6ecf85c740 10 client.19632 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:14:52.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.134+0000 7f6ecf85c740 10 client.19632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:14:52.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.134+0000 7f6ecf85c740 20 client.19632 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:52.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.134+0000 7f6ecf85c740 1 --2- 172.21.15.120:0/1985608984 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fb3977fea0 0x55fb397822c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:52.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.134+0000 7f6ecf85c740 4 client.19632 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:14:52.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.134+0000 7f6ecf85c740 1 -- 172.21.15.120:0/1985608984 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fb39667d60 con 0x55fb3977fea0 2024-09-19T09:14:52.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.134+0000 7f6ec6ffd640 1 --2- 172.21.15.120:0/1985608984 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fb3977fea0 0x55fb397822c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:52.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.134+0000 7f6eae7fc640 10 client.19632 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:52.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.134+0000 7f6ec6ffd640 1 --2- 172.21.15.120:0/1985608984 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fb3977fea0 0x55fb397822c0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66b1e2d5b68083cc server_cookie=b33e61a5bc02f109 in_seq=0 out_seq=0 2024-09-19T09:14:52.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.135+0000 7f6eae7fc640 1 -- 172.21.15.120:0/1985608984 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fb39667d60 con 0x55fb3977fea0 2024-09-19T09:14:52.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.135+0000 7f6eae7fc640 10 client.19632 handle_command_reply: tid=0 2024-09-19T09:14:52.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.135+0000 7f6ecf85c740 1 client.19632 shutdown 2024-09-19T09:14:52.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.135+0000 7f6ecf85c740 1 -- 172.21.15.120:0/1985608984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb396fa4a0 msgr2=0x55fb3970dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:52.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.135+0000 7f6ecf85c740 1 --2- 172.21.15.120:0/1985608984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb396fa4a0 0x55fb3970dc50 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000bad0 tx=0x7f6eb00038a0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.135+0000 7f6ecf85c740 1 -- 172.21.15.120:0/1985608984 shutdown_connections 2024-09-19T09:14:52.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.135+0000 7f6ecf85c740 1 --2- 172.21.15.120:0/1985608984 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fb3977fea0 0x55fb397822c0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.135+0000 7f6eae7fc640 0 client.19632 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:52.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.135+0000 7f6ecf85c740 1 --2- 172.21.15.120:0/1985608984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb39705680 0x55fb39709510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.135+0000 7f6ecf85c740 1 --2- 172.21.15.120:0/1985608984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb396fadc0 0x55fb39708fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.135+0000 7f6ecf85c740 1 --2- 172.21.15.120:0/1985608984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb396fa4a0 0x55fb3970dc50 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.135+0000 7f6ecf85c740 1 -- 172.21.15.120:0/1985608984 >> 172.21.15.120:0/1985608984 conn(0x55fb396f6a30 msgr2=0x55fb396f5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:52.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.135+0000 7f6ecf85c740 1 -- 172.21.15.120:0/1985608984 shutdown_connections 2024-09-19T09:14:52.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.135+0000 7f6ecf85c740 1 -- 172.21.15.120:0/1985608984 wait complete. 2024-09-19T09:14:52.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.135+0000 7f6ecf85c740 20 client.19632 trim_cache size 0 max 16384 2024-09-19T09:14:52.144 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:14:52.144 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:14:52.144 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:14:52.144 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:14:52.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.135+0000 7f6ecd483640 1 -- 172.21.15.120:0/2446419875 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6eb40a3240 msgr2=0x7f6eb40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:52.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.135+0000 7f6ecd483640 1 --2- 172.21.15.120:0/2446419875 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6eb40a3240 0x7f6eb40a5700 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0088420 tx=0x7f6ec00904a0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.136+0000 7f6ecd483640 1 -- 172.21.15.120:0/2446419875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ec81082a0 msgr2=0x7f6ec819a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:52.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.136+0000 7f6ecd483640 1 --2- 172.21.15.120:0/2446419875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ec81082a0 0x7f6ec819a5d0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0033040 tx=0x7f6ec0031e60 comp rx=0 tx=0).stop 2024-09-19T09:14:52.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.136+0000 7f6ecd483640 1 -- 172.21.15.120:0/2446419875 shutdown_connections 2024-09-19T09:14:52.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.136+0000 7f6ecd483640 1 --2- 172.21.15.120:0/2446419875 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6eb40a3240 0x7f6eb40a5700 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.136+0000 7f6ecd483640 1 --2- 172.21.15.120:0/2446419875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ec81082a0 0x7f6ec819a5d0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.136+0000 7f6ecd483640 1 --2- 172.21.15.120:0/2446419875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ec81036e0 0x7f6ec81a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.136+0000 7f6ecd483640 1 --2- 172.21.15.120:0/2446419875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ec8102d30 0x7f6ec81a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:52.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.136+0000 7f6ecd483640 1 -- 172.21.15.120:0/2446419875 >> 172.21.15.120:0/2446419875 conn(0x7f6ec80fc7d0 msgr2=0x7f6ec8101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:52.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.136+0000 7f6ecd483640 1 -- 172.21.15.120:0/2446419875 shutdown_connections 2024-09-19T09:14:52.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:52.136+0000 7f6ecd483640 1 -- 172.21.15.120:0/2446419875 wait complete. 2024-09-19T09:14:53.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:52 smithi120 ceph-mon[31150]: pgmap v704: 97 pgs: 97 active+clean; 46 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 448 op/s 2024-09-19T09:14:53.076 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1170886 9.69 MB/sec execute 528 sec latency 12.311 ms 2024-09-19T09:14:53.126 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:14:53.126 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:14:53.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:52 smithi196 ceph-mon[35150]: pgmap v704: 97 pgs: 97 active+clean; 46 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 448 op/s 2024-09-19T09:14:53.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:52 smithi153 ceph-mon[35071]: pgmap v704: 97 pgs: 97 active+clean; 46 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 448 op/s 2024-09-19T09:14:53.408 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:14:53.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.936+0000 7f2a4e076640 1 -- 172.21.15.120:0/693270065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a48100e20 msgr2=0x7f2a48101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:53.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.936+0000 7f2a4e076640 1 --2- 172.21.15.120:0/693270065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a48100e20 0x7f2a48101200 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f2a38009900 tx=0x7f2a3802a480 comp rx=0 tx=0).stop 2024-09-19T09:14:53.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.936+0000 7f2a4e076640 1 -- 172.21.15.120:0/693270065 shutdown_connections 2024-09-19T09:14:53.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.936+0000 7f2a4e076640 1 --2- 172.21.15.120:0/693270065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a48102ef0 0x7f2a4810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:53.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.936+0000 7f2a4e076640 1 --2- 172.21.15.120:0/693270065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a48102570 0x7f2a481029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:53.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.936+0000 7f2a4e076640 1 --2- 172.21.15.120:0/693270065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a48100e20 0x7f2a48101200 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:53.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.936+0000 7f2a4e076640 1 -- 172.21.15.120:0/693270065 >> 172.21.15.120:0/693270065 conn(0x7f2a480fc7d0 msgr2=0x7f2a480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:53.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.936+0000 7f2a4e076640 1 -- 172.21.15.120:0/693270065 shutdown_connections 2024-09-19T09:14:53.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.936+0000 7f2a4e076640 1 -- 172.21.15.120:0/693270065 wait complete. 2024-09-19T09:14:53.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.936+0000 7f2a4e076640 1 Processor -- start 2024-09-19T09:14:53.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.937+0000 7f2a4e076640 1 -- start start 2024-09-19T09:14:53.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.937+0000 7f2a4e076640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a48100e20 0x7f2a481984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:53.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.937+0000 7f2a4e076640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a48102570 0x7f2a481989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:53.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.937+0000 7f2a4e076640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a48102ef0 0x7f2a4819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:53.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.937+0000 7f2a4e076640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2a48110f50 con 0x7f2a48102570 2024-09-19T09:14:53.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.937+0000 7f2a4e076640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2a48110dd0 con 0x7f2a48100e20 2024-09-19T09:14:53.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.937+0000 7f2a4e076640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a481110d0 con 0x7f2a48102ef0 2024-09-19T09:14:53.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.937+0000 7f2a46ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a48102570 0x7f2a481989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:53.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.937+0000 7f2a477fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a48100e20 0x7f2a481984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:53.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.937+0000 7f2a477fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a48100e20 0x7f2a481984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:41074/0 (socket says 172.21.15.120:41074) 2024-09-19T09:14:53.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.937+0000 7f2a477fe640 1 -- 172.21.15.120:0/2837440059 learned_addr learned my addr 172.21.15.120:0/2837440059 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:53.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.937+0000 7f2a46ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a48102570 0x7f2a481989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37892/0 (socket says 172.21.15.120:37892) 2024-09-19T09:14:53.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.937+0000 7f2a47fff640 1 --2- 172.21.15.120:0/2837440059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a48102ef0 0x7f2a4819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:53.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.937+0000 7f2a46ffd640 1 -- 172.21.15.120:0/2837440059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a48102ef0 msgr2=0x7f2a4819cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:53.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.937+0000 7f2a46ffd640 1 --2- 172.21.15.120:0/2837440059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a48102ef0 0x7f2a4819cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:53.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.937+0000 7f2a46ffd640 1 -- 172.21.15.120:0/2837440059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a48100e20 msgr2=0x7f2a481984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:53.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.937+0000 7f2a46ffd640 1 --2- 172.21.15.120:0/2837440059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a48100e20 0x7f2a481984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:53.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.937+0000 7f2a46ffd640 1 -- 172.21.15.120:0/2837440059 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a4819d450 con 0x7f2a48102570 2024-09-19T09:14:53.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.937+0000 7f2a477fe640 1 --2- 172.21.15.120:0/2837440059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a48100e20 0x7f2a481984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:53.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.938+0000 7f2a46ffd640 1 --2- 172.21.15.120:0/2837440059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a48102570 0x7f2a481989e0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c005fd0 tx=0x7f2a3c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:53.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.938+0000 7f2a44ff9640 1 -- 172.21.15.120:0/2837440059 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c009c30 con 0x7f2a48102570 2024-09-19T09:14:53.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.938+0000 7f2a44ff9640 1 -- 172.21.15.120:0/2837440059 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a3c009dd0 con 0x7f2a48102570 2024-09-19T09:14:53.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.938+0000 7f2a44ff9640 1 -- 172.21.15.120:0/2837440059 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c010460 con 0x7f2a48102570 2024-09-19T09:14:53.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.938+0000 7f2a4e076640 1 -- 172.21.15.120:0/2837440059 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a4819d740 con 0x7f2a48102570 2024-09-19T09:14:53.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.938+0000 7f2a4e076640 1 -- 172.21.15.120:0/2837440059 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a481a5020 con 0x7f2a48102570 2024-09-19T09:14:53.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.939+0000 7f2a4e076640 1 -- 172.21.15.120:0/2837440059 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a48101200 con 0x7f2a48102570 2024-09-19T09:14:53.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.940+0000 7f2a44ff9640 1 -- 172.21.15.120:0/2837440059 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f2a3c003f50 con 0x7f2a48102570 2024-09-19T09:14:53.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.940+0000 7f2a44ff9640 1 --2- 172.21.15.120:0/2837440059 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2a300a3240 0x7f2a300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:53.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.941+0000 7f2a477fe640 1 --2- 172.21.15.120:0/2837440059 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2a300a3240 0x7f2a300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:53.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.941+0000 7f2a44ff9640 1 -- 172.21.15.120:0/2837440059 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f2a3c0cf840 con 0x7f2a48102570 2024-09-19T09:14:53.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.941+0000 7f2a477fe640 1 --2- 172.21.15.120:0/2837440059 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2a300a3240 0x7f2a300a5700 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2a380096c0 tx=0x7f2a38005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:53.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:53.945+0000 7f2a44ff9640 1 -- 172.21.15.120:0/2837440059 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2a3c017020 con 0x7f2a48102570 2024-09-19T09:14:54.075 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1172690 9.70 MB/sec execute 529 sec latency 67.421 ms 2024-09-19T09:14:54.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:54.158+0000 7f2a4e076640 1 -- 172.21.15.120:0/2837440059 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a481a5220 con 0x7f2a48102570 2024-09-19T09:14:54.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:54.159+0000 7f2a44ff9640 1 -- 172.21.15.120:0/2837440059 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11988 (secure 0 0 0) 0x7f2a3c09bf10 con 0x7f2a48102570 2024-09-19T09:14:54.159 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:14:54.160 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":33,"btime":"2024-09-19T09:14:31:550518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:14:31.550494+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28649":{"gid":28649,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/2810080307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2810080307},{"type":"v1","addr":"172.21.15.153:6835","nonce":2810080307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:14:54.160 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 33 2024-09-19T09:14:54.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:54.161+0000 7f2a4e076640 1 -- 172.21.15.120:0/2837440059 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2a300a3240 msgr2=0x7f2a300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:54.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:54.161+0000 7f2a4e076640 1 --2- 172.21.15.120:0/2837440059 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2a300a3240 0x7f2a300a5700 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2a380096c0 tx=0x7f2a38005a20 comp rx=0 tx=0).stop 2024-09-19T09:14:54.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:54.161+0000 7f2a4e076640 1 -- 172.21.15.120:0/2837440059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a48102570 msgr2=0x7f2a481989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:54.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:54.161+0000 7f2a4e076640 1 --2- 172.21.15.120:0/2837440059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a48102570 0x7f2a481989e0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c005fd0 tx=0x7f2a3c00dde0 comp rx=0 tx=0).stop 2024-09-19T09:14:54.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:54.161+0000 7f2a4e076640 1 -- 172.21.15.120:0/2837440059 shutdown_connections 2024-09-19T09:14:54.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:54.161+0000 7f2a4e076640 1 --2- 172.21.15.120:0/2837440059 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2a300a3240 0x7f2a300a5700 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:54.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:54.161+0000 7f2a4e076640 1 --2- 172.21.15.120:0/2837440059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a48102ef0 0x7f2a4819cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:54.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:54.161+0000 7f2a4e076640 1 --2- 172.21.15.120:0/2837440059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a48102570 0x7f2a481989e0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:54.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:54.161+0000 7f2a4e076640 1 --2- 172.21.15.120:0/2837440059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a48100e20 0x7f2a481984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:54.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:54.161+0000 7f2a4e076640 1 -- 172.21.15.120:0/2837440059 >> 172.21.15.120:0/2837440059 conn(0x7f2a480fc7d0 msgr2=0x7f2a48069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:54.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:54.161+0000 7f2a4e076640 1 -- 172.21.15.120:0/2837440059 shutdown_connections 2024-09-19T09:14:54.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:54.161+0000 7f2a4e076640 1 -- 172.21.15.120:0/2837440059 wait complete. 2024-09-19T09:14:54.916 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:14:55.075 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1175903 9.71 MB/sec execute 530 sec latency 13.501 ms 2024-09-19T09:14:55.175 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:14:55.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:54 smithi196 ceph-mon[35150]: pgmap v705: 97 pgs: 97 active+clean; 46 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 408 op/s 2024-09-19T09:14:55.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:54 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2837440059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:14:55.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:54 smithi153 ceph-mon[35071]: pgmap v705: 97 pgs: 97 active+clean; 46 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 408 op/s 2024-09-19T09:14:55.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:54 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2837440059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:14:55.296 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:54 smithi120 ceph-mon[31150]: pgmap v705: 97 pgs: 97 active+clean; 46 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 408 op/s 2024-09-19T09:14:55.296 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:54 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2837440059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:14:55.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.665+0000 7fdf72a6c640 1 -- 172.21.15.120:0/2757395806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf6c102ef0 msgr2=0x7fdf6c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:55.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.665+0000 7fdf72a6c640 1 --2- 172.21.15.120:0/2757395806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf6c102ef0 0x7fdf6c10da40 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009900 tx=0x7fdf6402a480 comp rx=0 tx=0).stop 2024-09-19T09:14:55.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.666+0000 7fdf72a6c640 1 -- 172.21.15.120:0/2757395806 shutdown_connections 2024-09-19T09:14:55.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.666+0000 7fdf72a6c640 1 --2- 172.21.15.120:0/2757395806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf6c102ef0 0x7fdf6c10da40 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.666+0000 7fdf72a6c640 1 --2- 172.21.15.120:0/2757395806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf6c102570 0x7fdf6c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.666+0000 7fdf72a6c640 1 --2- 172.21.15.120:0/2757395806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf6c100e20 0x7fdf6c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.666+0000 7fdf72a6c640 1 -- 172.21.15.120:0/2757395806 >> 172.21.15.120:0/2757395806 conn(0x7fdf6c0fc7d0 msgr2=0x7fdf6c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:55.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.666+0000 7fdf72a6c640 1 -- 172.21.15.120:0/2757395806 shutdown_connections 2024-09-19T09:14:55.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.666+0000 7fdf72a6c640 1 -- 172.21.15.120:0/2757395806 wait complete. 2024-09-19T09:14:55.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.666+0000 7fdf72a6c640 1 Processor -- start 2024-09-19T09:14:55.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.666+0000 7fdf72a6c640 1 -- start start 2024-09-19T09:14:55.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.666+0000 7fdf72a6c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf6c100e20 0x7fdf6c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:55.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.666+0000 7fdf72a6c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf6c102570 0x7fdf6c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:55.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.666+0000 7fdf72a6c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf6c102ef0 0x7fdf6c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:55.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.666+0000 7fdf72a6c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c110f80 con 0x7fdf6c102570 2024-09-19T09:14:55.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.666+0000 7fdf72a6c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c110e00 con 0x7fdf6c102ef0 2024-09-19T09:14:55.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.666+0000 7fdf72a6c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c111100 con 0x7fdf6c100e20 2024-09-19T09:14:55.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.667+0000 7fdf6bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf6c102570 0x7fdf6c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:55.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.667+0000 7fdf6bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf6c102570 0x7fdf6c0fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37900/0 (socket says 172.21.15.120:37900) 2024-09-19T09:14:55.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.667+0000 7fdf6bfff640 1 -- 172.21.15.120:0/2478372424 learned_addr learned my addr 172.21.15.120:0/2478372424 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:55.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.667+0000 7fdf71009640 1 --2- 172.21.15.120:0/2478372424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf6c102ef0 0x7fdf6c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:55.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.667+0000 7fdf70808640 1 --2- 172.21.15.120:0/2478372424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf6c100e20 0x7fdf6c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:55.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.667+0000 7fdf6bfff640 1 -- 172.21.15.120:0/2478372424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf6c100e20 msgr2=0x7fdf6c0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:55.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.667+0000 7fdf6bfff640 1 --2- 172.21.15.120:0/2478372424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf6c100e20 0x7fdf6c0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.667+0000 7fdf6bfff640 1 -- 172.21.15.120:0/2478372424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf6c102ef0 msgr2=0x7fdf6c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:55.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.667+0000 7fdf6bfff640 1 --2- 172.21.15.120:0/2478372424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf6c102ef0 0x7fdf6c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.667+0000 7fdf6bfff640 1 -- 172.21.15.120:0/2478372424 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf6c1a6910 con 0x7fdf6c102570 2024-09-19T09:14:55.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.667+0000 7fdf71009640 1 --2- 172.21.15.120:0/2478372424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf6c102ef0 0x7fdf6c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:55.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.667+0000 7fdf6bfff640 1 --2- 172.21.15.120:0/2478372424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf6c102570 0x7fdf6c0fd1c0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00eab0 tx=0x7fdf5c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:55.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.667+0000 7fdf69ffb640 1 -- 172.21.15.120:0/2478372424 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c00ce40 con 0x7fdf6c102570 2024-09-19T09:14:55.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.667+0000 7fdf69ffb640 1 -- 172.21.15.120:0/2478372424 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf5c011950 con 0x7fdf6c102570 2024-09-19T09:14:55.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.667+0000 7fdf69ffb640 1 -- 172.21.15.120:0/2478372424 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c010ae0 con 0x7fdf6c102570 2024-09-19T09:14:55.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.667+0000 7fdf72a6c640 1 -- 172.21.15.120:0/2478372424 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf6c1a6c00 con 0x7fdf6c102570 2024-09-19T09:14:55.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.668+0000 7fdf72a6c640 1 -- 172.21.15.120:0/2478372424 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf6c1a7140 con 0x7fdf6c102570 2024-09-19T09:14:55.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.670+0000 7fdf69ffb640 1 -- 172.21.15.120:0/2478372424 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fdf5c011b00 con 0x7fdf6c102570 2024-09-19T09:14:55.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.671+0000 7fdf74e45740 1 Processor -- start 2024-09-19T09:14:55.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.671+0000 7fdf74e45740 1 -- start start 2024-09-19T09:14:55.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.671+0000 7fdf74e45740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56466b450b40 0x56466b44f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:55.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.671+0000 7fdf74e45740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56466b44f5d0 0x56466b44fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:55.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.671+0000 7fdf74e45740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56466b45a370 0x56466b45c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:55.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.671+0000 7fdf74e45740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56466b3b52f0 con 0x56466b44f5d0 2024-09-19T09:14:55.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.671+0000 7fdf74e45740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56466b35ccf0 con 0x56466b450b40 2024-09-19T09:14:55.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.671+0000 7fdf74e45740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56466b3b12b0 con 0x56466b45a370 2024-09-19T09:14:55.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.671+0000 7fdf70808640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56466b44f5d0 0x56466b44fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:55.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.671+0000 7fdf70808640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56466b44f5d0 0x56466b44fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37906/0 (socket says 172.21.15.120:37906) 2024-09-19T09:14:55.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.671+0000 7fdf71009640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56466b450b40 0x56466b44f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:55.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.671+0000 7fdf6bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56466b45a370 0x56466b45c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:55.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.671+0000 7fdf70808640 1 -- 172.21.15.120:0/698210560 learned_addr learned my addr 172.21.15.120:0/698210560 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:55.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.671+0000 7fdf69ffb640 1 --2- 172.21.15.120:0/2478372424 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdf580a3240 0x7fdf580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:55.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.671+0000 7fdf70808640 1 -- 172.21.15.120:0/698210560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56466b45a370 msgr2=0x56466b45c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:55.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.671+0000 7fdf70808640 1 --2- 172.21.15.120:0/698210560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56466b45a370 0x56466b45c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.671+0000 7fdf70808640 1 -- 172.21.15.120:0/698210560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56466b450b40 msgr2=0x56466b44f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:55.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.671+0000 7fdf70808640 1 --2- 172.21.15.120:0/698210560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56466b450b40 0x56466b44f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.671+0000 7fdf70808640 1 -- 172.21.15.120:0/698210560 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56466b3413a0 con 0x56466b44f5d0 2024-09-19T09:14:55.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.672+0000 7fdf71009640 1 --2- 172.21.15.120:0/2478372424 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdf580a3240 0x7fdf580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:55.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.672+0000 7fdf69ffb640 1 -- 172.21.15.120:0/2478372424 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fdf5c0cff90 con 0x7fdf6c102570 2024-09-19T09:14:55.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.672+0000 7fdf70808640 1 --2- 172.21.15.120:0/698210560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56466b44f5d0 0x56466b44fa10 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009c80 tx=0x7fdf540117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0fda9763710558 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:55.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.672+0000 7fdf62ffd640 1 -- 172.21.15.120:0/698210560 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf54012450 con 0x56466b44f5d0 2024-09-19T09:14:55.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.672+0000 7fdf62ffd640 1 -- 172.21.15.120:0/698210560 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf54012ad0 con 0x56466b44f5d0 2024-09-19T09:14:55.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.672+0000 7fdf62ffd640 1 -- 172.21.15.120:0/698210560 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5400fba0 con 0x56466b44f5d0 2024-09-19T09:14:55.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.673+0000 7fdf74e45740 1 -- 172.21.15.120:0/698210560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56466b44f5d0 msgr2=0x56466b44fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:55.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.673+0000 7fdf71009640 1 --2- 172.21.15.120:0/2478372424 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdf580a3240 0x7fdf580a5700 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdf640314e0 tx=0x7fdf64009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:55.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.673+0000 7fdf74e45740 1 --2- 172.21.15.120:0/698210560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56466b44f5d0 0x56466b44fa10 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009c80 tx=0x7fdf540117b0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.673+0000 7fdf74e45740 1 -- 172.21.15.120:0/698210560 shutdown_connections 2024-09-19T09:14:55.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.673+0000 7fdf74e45740 1 --2- 172.21.15.120:0/698210560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56466b45a370 0x56466b45c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.673+0000 7fdf74e45740 1 --2- 172.21.15.120:0/698210560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56466b44f5d0 0x56466b44fa10 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.673+0000 7fdf74e45740 1 --2- 172.21.15.120:0/698210560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56466b450b40 0x56466b44f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.673+0000 7fdf74e45740 1 -- 172.21.15.120:0/698210560 >> 172.21.15.120:0/698210560 conn(0x56466b418310 msgr2=0x56466b44bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:55.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.673+0000 7fdf74e45740 1 -- 172.21.15.120:0/698210560 shutdown_connections 2024-09-19T09:14:55.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.673+0000 7fdf74e45740 1 -- 172.21.15.120:0/698210560 wait complete. 2024-09-19T09:14:55.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.673+0000 7fdf74e45740 1 -- start start 2024-09-19T09:14:55.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.673+0000 7fdf74e45740 1 Processor -- start 2024-09-19T09:14:55.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.674+0000 7fdf74e45740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56466b44f5d0 0x56466b459c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:55.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.674+0000 7fdf74e45740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56466b450b40 0x56466b456f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:55.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.674+0000 7fdf74e45740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56466b45a370 0x56466b457460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:55.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.674+0000 7fdf74e45740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56466b45d070 con 0x56466b450b40 2024-09-19T09:14:55.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.674+0000 7fdf74e45740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56466b370060 con 0x56466b44f5d0 2024-09-19T09:14:55.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.674+0000 7fdf74e45740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56466b45d1f0 con 0x56466b45a370 2024-09-19T09:14:55.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.674+0000 7fdf74e45740 10 client.0 _ll_register_callbacks cb 0x56466b0c38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:55.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.674+0000 7fdf6bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56466b450b40 0x56466b456f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:55.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.674+0000 7fdf6bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56466b450b40 0x56466b456f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37920/0 (socket says 172.21.15.120:37920) 2024-09-19T09:14:55.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.674+0000 7fdf6bfff640 1 -- 172.21.15.120:0/3468492434 learned_addr learned my addr 172.21.15.120:0/3468492434 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:55.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.674+0000 7fdf70808640 1 --2- 172.21.15.120:0/3468492434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56466b44f5d0 0x56466b459c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:55.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.674+0000 7fdf71009640 1 --2- 172.21.15.120:0/3468492434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56466b45a370 0x56466b457460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:55.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.674+0000 7fdf6bfff640 1 -- 172.21.15.120:0/3468492434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56466b45a370 msgr2=0x56466b457460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:55.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.674+0000 7fdf6bfff640 1 --2- 172.21.15.120:0/3468492434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56466b45a370 0x56466b457460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.674+0000 7fdf6bfff640 1 -- 172.21.15.120:0/3468492434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56466b44f5d0 msgr2=0x56466b459c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:55.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.674+0000 7fdf6bfff640 1 --2- 172.21.15.120:0/3468492434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56466b44f5d0 0x56466b459c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.674+0000 7fdf6bfff640 1 -- 172.21.15.120:0/3468492434 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56466b1263c0 con 0x56466b450b40 2024-09-19T09:14:55.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.674+0000 7fdf6bfff640 1 --2- 172.21.15.120:0/3468492434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56466b450b40 0x56466b456f20 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c0a4cf0 tx=0x7fdf5c011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df77534ca846de6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:55.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.674+0000 7fdf4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:14:55.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.674+0000 7fdf4bfff640 1 -- 172.21.15.120:0/3468492434 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c09c380 con 0x56466b450b40 2024-09-19T09:14:55.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.674+0000 7fdf4bfff640 1 -- 172.21.15.120:0/3468492434 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fdf5c09ca10 con 0x56466b450b40 2024-09-19T09:14:55.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.674+0000 7fdf74e45740 1 -- 172.21.15.120:0/3468492434 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf54012ad0 con 0x56466b450b40 2024-09-19T09:14:55.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.675+0000 7fdf4bfff640 1 -- 172.21.15.120:0/3468492434 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdf5c09bf10 con 0x56466b450b40 2024-09-19T09:14:55.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.675+0000 7fdf74e45740 10 client.19650 fetch_fsmap learned FSMap version 33 2024-09-19T09:14:55.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.675+0000 7fdf74e45740 1 -- 172.21.15.120:0/3468492434 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56466b468ae0 con 0x56466b450b40 2024-09-19T09:14:55.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.675+0000 7fdf4bfff640 1 -- 172.21.15.120:0/3468492434 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf54012ad0 con 0x56466b450b40 2024-09-19T09:14:55.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.675+0000 7fdf4bfff640 1 -- 172.21.15.120:0/3468492434 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fdf5c00f040 con 0x56466b450b40 2024-09-19T09:14:55.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.675+0000 7fdf74e45740 10 client.19650 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:14:55.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.675+0000 7fdf74e45740 10 client.19650 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:14:55.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.675+0000 7fdf74e45740 20 client.19650 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:55.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.675+0000 7fdf74e45740 1 --2- 172.21.15.120:0/3468492434 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x56466b469f90 0x56466b46c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:55.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.675+0000 7fdf74e45740 4 client.19650 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:14:55.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.675+0000 7fdf74e45740 1 -- 172.21.15.120:0/3468492434 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56466b3c2860 con 0x56466b469f90 2024-09-19T09:14:55.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.675+0000 7fdf70808640 1 --2- 172.21.15.120:0/3468492434 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x56466b469f90 0x56466b46c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:55.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.676+0000 7fdf4bfff640 10 client.19650 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:14:55.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.676+0000 7fdf70808640 1 --2- 172.21.15.120:0/3468492434 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x56466b469f90 0x56466b46c410 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a740410b39de26ed server_cookie=ef28a0b96c012993 in_seq=0 out_seq=0 2024-09-19T09:14:55.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.676+0000 7fdf4bfff640 1 -- 172.21.15.120:0/3468492434 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56466b3c2860 con 0x56466b469f90 2024-09-19T09:14:55.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.676+0000 7fdf4bfff640 10 client.19650 handle_command_reply: tid=0 2024-09-19T09:14:55.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.676+0000 7fdf74e45740 1 client.19650 shutdown 2024-09-19T09:14:55.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.676+0000 7fdf74e45740 1 -- 172.21.15.120:0/3468492434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56466b450b40 msgr2=0x56466b456f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:55.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.676+0000 7fdf74e45740 1 --2- 172.21.15.120:0/3468492434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56466b450b40 0x56466b456f20 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c0a4cf0 tx=0x7fdf5c011ef0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.676+0000 7fdf74e45740 1 -- 172.21.15.120:0/3468492434 shutdown_connections 2024-09-19T09:14:55.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.676+0000 7fdf74e45740 1 --2- 172.21.15.120:0/3468492434 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x56466b469f90 0x56466b46c410 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.676+0000 7fdf74e45740 1 --2- 172.21.15.120:0/3468492434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56466b45a370 0x56466b457460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.676+0000 7fdf74e45740 1 --2- 172.21.15.120:0/3468492434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56466b450b40 0x56466b456f20 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.676+0000 7fdf74e45740 1 --2- 172.21.15.120:0/3468492434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56466b44f5d0 0x56466b459c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.676+0000 7fdf4bfff640 0 client.19650 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:14:55.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.676+0000 7fdf74e45740 1 -- 172.21.15.120:0/3468492434 >> 172.21.15.120:0/3468492434 conn(0x56466b418310 msgr2=0x56466b451930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:55.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.677+0000 7fdf74e45740 1 -- 172.21.15.120:0/3468492434 shutdown_connections 2024-09-19T09:14:55.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.677+0000 7fdf74e45740 1 -- 172.21.15.120:0/3468492434 wait complete. 2024-09-19T09:14:55.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.677+0000 7fdf74e45740 20 client.19650 trim_cache size 0 max 16384 2024-09-19T09:14:55.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.690+0000 7fdf74e45740 1 Processor -- start 2024-09-19T09:14:55.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.690+0000 7fdf74e45740 1 -- start start 2024-09-19T09:14:55.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.691+0000 7fdf74e45740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56466b44f5d0 0x56466b44abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:55.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.691+0000 7fdf74e45740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56466b450b40 0x56466b452120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:55.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.691+0000 7fdf74e45740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56466b45a370 0x56466b452660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:55.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.691+0000 7fdf74e45740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c1af7c0 con 0x56466b450b40 2024-09-19T09:14:55.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.691+0000 7fdf74e45740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c1b1300 con 0x56466b44f5d0 2024-09-19T09:14:55.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.691+0000 7fdf74e45740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c1afe90 con 0x56466b45a370 2024-09-19T09:14:55.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.691+0000 7fdf70808640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56466b44f5d0 0x56466b44abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:55.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.691+0000 7fdf6bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56466b450b40 0x56466b452120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:55.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.691+0000 7fdf70808640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56466b44f5d0 0x56466b44abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:41124/0 (socket says 172.21.15.120:41124) 2024-09-19T09:14:55.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.691+0000 7fdf70808640 1 -- 172.21.15.120:0/3842976092 learned_addr learned my addr 172.21.15.120:0/3842976092 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:55.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.691+0000 7fdf71009640 1 --2- 172.21.15.120:0/3842976092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56466b45a370 0x56466b452660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:55.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.691+0000 7fdf70808640 1 -- 172.21.15.120:0/3842976092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56466b45a370 msgr2=0x56466b452660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:55.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.691+0000 7fdf70808640 1 --2- 172.21.15.120:0/3842976092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56466b45a370 0x56466b452660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.691+0000 7fdf70808640 1 -- 172.21.15.120:0/3842976092 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56466b450b40 msgr2=0x56466b452120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:55.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.691+0000 7fdf70808640 1 --2- 172.21.15.120:0/3842976092 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56466b450b40 0x56466b452120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.691+0000 7fdf70808640 1 -- 172.21.15.120:0/3842976092 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56466b459890 con 0x56466b44f5d0 2024-09-19T09:14:55.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.691+0000 7fdf70808640 1 --2- 172.21.15.120:0/3842976092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56466b44f5d0 0x56466b44abf0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdf5400f750 tx=0x7fdf5400f6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0929cd010a2866f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:55.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.692+0000 7fdf627fc640 1 -- 172.21.15.120:0/3842976092 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf540052b0 con 0x56466b44f5d0 2024-09-19T09:14:55.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.692+0000 7fdf627fc640 1 -- 172.21.15.120:0/3842976092 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf54011cb0 con 0x56466b44f5d0 2024-09-19T09:14:55.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.692+0000 7fdf627fc640 1 -- 172.21.15.120:0/3842976092 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf540035f0 con 0x56466b44f5d0 2024-09-19T09:14:55.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.692+0000 7fdf74e45740 1 -- 172.21.15.120:0/3842976092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56466b44f5d0 msgr2=0x56466b44abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:55.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.692+0000 7fdf74e45740 1 --2- 172.21.15.120:0/3842976092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56466b44f5d0 0x56466b44abf0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdf5400f750 tx=0x7fdf5400f6e0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.692+0000 7fdf74e45740 1 -- 172.21.15.120:0/3842976092 shutdown_connections 2024-09-19T09:14:55.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.692+0000 7fdf74e45740 1 --2- 172.21.15.120:0/3842976092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56466b45a370 0x56466b452660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.692+0000 7fdf74e45740 1 --2- 172.21.15.120:0/3842976092 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56466b450b40 0x56466b452120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.692+0000 7fdf74e45740 1 --2- 172.21.15.120:0/3842976092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56466b44f5d0 0x56466b44abf0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.692+0000 7fdf74e45740 1 -- 172.21.15.120:0/3842976092 >> 172.21.15.120:0/3842976092 conn(0x56466b418310 msgr2=0x56466b44a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:55.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.692+0000 7fdf74e45740 1 -- 172.21.15.120:0/3842976092 shutdown_connections 2024-09-19T09:14:55.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.692+0000 7fdf74e45740 1 -- 172.21.15.120:0/3842976092 wait complete. 2024-09-19T09:14:55.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.693+0000 7fdf74e45740 1 -- start start 2024-09-19T09:14:55.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.693+0000 7fdf74e45740 1 Processor -- start 2024-09-19T09:14:55.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.693+0000 7fdf74e45740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56466b44f5d0 0x56466b462a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:55.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.693+0000 7fdf74e45740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56466b450b40 0x56466b453a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:55.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.693+0000 7fdf74e45740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56466b45a370 0x56466b453fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:55.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.693+0000 7fdf74e45740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c1b0c30 con 0x56466b450b40 2024-09-19T09:14:55.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.693+0000 7fdf74e45740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c1b0560 con 0x56466b44f5d0 2024-09-19T09:14:55.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.693+0000 7fdf74e45740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c1ad780 con 0x56466b45a370 2024-09-19T09:14:55.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.693+0000 7fdf74e45740 10 client.0 _ll_register_callbacks cb 0x7fdf5c098190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:55.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.693+0000 7fdf6bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56466b450b40 0x56466b453a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:55.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.694+0000 7fdf70808640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56466b44f5d0 0x56466b462a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:55.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.694+0000 7fdf6bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56466b450b40 0x56466b453a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37946/0 (socket says 172.21.15.120:37946) 2024-09-19T09:14:55.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.694+0000 7fdf6bfff640 1 -- 172.21.15.120:0/1824050751 learned_addr learned my addr 172.21.15.120:0/1824050751 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:55.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.694+0000 7fdf71009640 1 --2- 172.21.15.120:0/1824050751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56466b45a370 0x56466b453fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:55.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.694+0000 7fdf6bfff640 1 -- 172.21.15.120:0/1824050751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56466b45a370 msgr2=0x56466b453fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:55.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.694+0000 7fdf6bfff640 1 --2- 172.21.15.120:0/1824050751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56466b45a370 0x56466b453fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.694+0000 7fdf6bfff640 1 -- 172.21.15.120:0/1824050751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56466b44f5d0 msgr2=0x56466b462a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:55.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.694+0000 7fdf6bfff640 1 --2- 172.21.15.120:0/1824050751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56466b44f5d0 0x56466b462a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.694+0000 7fdf6bfff640 1 -- 172.21.15.120:0/1824050751 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56466b454610 con 0x56466b450b40 2024-09-19T09:14:55.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.694+0000 7fdf6bfff640 1 --2- 172.21.15.120:0/1824050751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56466b450b40 0x56466b453a60 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c098f80 tx=0x7fdf5c0d3990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f138ee20dc53821c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:55.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.694+0000 7fdf4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:14:55.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.694+0000 7fdf4bfff640 1 -- 172.21.15.120:0/1824050751 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c09c0f0 con 0x56466b450b40 2024-09-19T09:14:55.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.694+0000 7fdf4bfff640 1 -- 172.21.15.120:0/1824050751 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fdf5c011470 con 0x56466b450b40 2024-09-19T09:14:55.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.694+0000 7fdf74e45740 1 -- 172.21.15.120:0/1824050751 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf54011cb0 con 0x56466b450b40 2024-09-19T09:14:55.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.694+0000 7fdf4bfff640 1 -- 172.21.15.120:0/1824050751 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdf5c09ccb0 con 0x56466b450b40 2024-09-19T09:14:55.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.694+0000 7fdf74e45740 10 client.19656 fetch_fsmap learned FSMap version 33 2024-09-19T09:14:55.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.694+0000 7fdf74e45740 1 -- 172.21.15.120:0/1824050751 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56466b4d4b60 con 0x56466b450b40 2024-09-19T09:14:55.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.694+0000 7fdf4bfff640 1 -- 172.21.15.120:0/1824050751 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf54011cb0 con 0x56466b450b40 2024-09-19T09:14:55.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.695+0000 7fdf4bfff640 1 -- 172.21.15.120:0/1824050751 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fdf5c0672b0 con 0x56466b450b40 2024-09-19T09:14:55.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.695+0000 7fdf74e45740 10 client.19656 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:14:55.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.695+0000 7fdf74e45740 10 client.19656 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:14:55.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.695+0000 7fdf74e45740 20 client.19656 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:55.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.695+0000 7fdf74e45740 1 --2- 172.21.15.120:0/1824050751 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x56466b4d5280 0x56466b4d76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:55.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.695+0000 7fdf74e45740 4 client.19656 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:14:55.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.695+0000 7fdf74e45740 1 -- 172.21.15.120:0/1824050751 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56466b3bcd70 con 0x56466b4d5280 2024-09-19T09:14:55.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.695+0000 7fdf70808640 1 --2- 172.21.15.120:0/1824050751 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x56466b4d5280 0x56466b4d76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:55.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.695+0000 7fdf70808640 1 --2- 172.21.15.120:0/1824050751 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x56466b4d5280 0x56466b4d76a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cfb89494ff6697a1 server_cookie=2af3d8173210eb1 in_seq=0 out_seq=0 2024-09-19T09:14:55.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.695+0000 7fdf4bfff640 10 client.19656 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:14:55.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.695+0000 7fdf4bfff640 1 -- 172.21.15.120:0/1824050751 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56466b3bcd70 con 0x56466b4d5280 2024-09-19T09:14:55.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.695+0000 7fdf4bfff640 10 client.19656 handle_command_reply: tid=0 2024-09-19T09:14:55.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.695+0000 7fdf74e45740 1 client.19656 shutdown 2024-09-19T09:14:55.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.696+0000 7fdf74e45740 1 -- 172.21.15.120:0/1824050751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56466b450b40 msgr2=0x56466b453a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:55.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.696+0000 7fdf74e45740 1 --2- 172.21.15.120:0/1824050751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56466b450b40 0x56466b453a60 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c098f80 tx=0x7fdf5c0d3990 comp rx=0 tx=0).stop 2024-09-19T09:14:55.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.696+0000 7fdf74e45740 1 -- 172.21.15.120:0/1824050751 shutdown_connections 2024-09-19T09:14:55.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.696+0000 7fdf74e45740 1 --2- 172.21.15.120:0/1824050751 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x56466b4d5280 0x56466b4d76a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.696+0000 7fdf74e45740 1 --2- 172.21.15.120:0/1824050751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56466b45a370 0x56466b453fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.696+0000 7fdf74e45740 1 --2- 172.21.15.120:0/1824050751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56466b450b40 0x56466b453a60 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.696+0000 7fdf4bfff640 0 client.19656 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:14:55.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.696+0000 7fdf74e45740 1 --2- 172.21.15.120:0/1824050751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56466b44f5d0 0x56466b462a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.696+0000 7fdf74e45740 1 -- 172.21.15.120:0/1824050751 >> 172.21.15.120:0/1824050751 conn(0x56466b418310 msgr2=0x56466b458fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:55.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.696+0000 7fdf74e45740 1 -- 172.21.15.120:0/1824050751 shutdown_connections 2024-09-19T09:14:55.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.696+0000 7fdf74e45740 1 -- 172.21.15.120:0/1824050751 wait complete. 2024-09-19T09:14:55.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.696+0000 7fdf74e45740 20 client.19656 trim_cache size 0 max 16384 2024-09-19T09:14:55.710 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:14:55.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.696+0000 7fdf72a6c640 1 -- 172.21.15.120:0/2478372424 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdf580a3240 msgr2=0x7fdf580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:55.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.696+0000 7fdf72a6c640 1 --2- 172.21.15.120:0/2478372424 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdf580a3240 0x7fdf580a5700 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdf640314e0 tx=0x7fdf64009260 comp rx=0 tx=0).stop 2024-09-19T09:14:55.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.696+0000 7fdf72a6c640 1 -- 172.21.15.120:0/2478372424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf6c102570 msgr2=0x7fdf6c0fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:55.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.696+0000 7fdf72a6c640 1 --2- 172.21.15.120:0/2478372424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf6c102570 0x7fdf6c0fd1c0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00eab0 tx=0x7fdf5c00ef70 comp rx=0 tx=0).stop 2024-09-19T09:14:55.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.697+0000 7fdf72a6c640 1 -- 172.21.15.120:0/2478372424 shutdown_connections 2024-09-19T09:14:55.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.697+0000 7fdf72a6c640 1 --2- 172.21.15.120:0/2478372424 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdf580a3240 0x7fdf580a5700 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.697+0000 7fdf72a6c640 1 --2- 172.21.15.120:0/2478372424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf6c102ef0 0x7fdf6c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.697+0000 7fdf72a6c640 1 --2- 172.21.15.120:0/2478372424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf6c102570 0x7fdf6c0fd1c0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.697+0000 7fdf72a6c640 1 --2- 172.21.15.120:0/2478372424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf6c100e20 0x7fdf6c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:55.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.697+0000 7fdf72a6c640 1 -- 172.21.15.120:0/2478372424 >> 172.21.15.120:0/2478372424 conn(0x7fdf6c0fc7d0 msgr2=0x7fdf6c06a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:55.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.697+0000 7fdf72a6c640 1 -- 172.21.15.120:0/2478372424 shutdown_connections 2024-09-19T09:14:55.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:55.697+0000 7fdf72a6c640 1 -- 172.21.15.120:0/2478372424 wait complete. 2024-09-19T09:14:56.076 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1178624 9.72 MB/sec execute 531 sec latency 30.792 ms 2024-09-19T09:14:56.474 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:14:56.726 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:14:57.075 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1180759 9.72 MB/sec execute 532 sec latency 37.373 ms 2024-09-19T09:14:57.150 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:56 smithi120 ceph-mon[31150]: pgmap v706: 97 pgs: 97 active+clean; 49 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 387 op/s 2024-09-19T09:14:57.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:56 smithi196 ceph-mon[35150]: pgmap v706: 97 pgs: 97 active+clean; 49 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 387 op/s 2024-09-19T09:14:57.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:56 smithi153 ceph-mon[35071]: pgmap v706: 97 pgs: 97 active+clean; 49 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 387 op/s 2024-09-19T09:14:57.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.240+0000 7f4d004b6640 1 -- 172.21.15.120:0/274062137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4cf8103700 msgr2=0x7f4cf810bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:57.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.240+0000 7f4d004b6640 1 --2- 172.21.15.120:0/274062137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4cf8103700 0x7f4cf810bbf0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009900 tx=0x7f4ce802a480 comp rx=0 tx=0).stop 2024-09-19T09:14:57.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.240+0000 7f4d004b6640 1 -- 172.21.15.120:0/274062137 shutdown_connections 2024-09-19T09:14:57.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.240+0000 7f4d004b6640 1 --2- 172.21.15.120:0/274062137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cf810c180 0x7f4cf810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.240+0000 7f4d004b6640 1 --2- 172.21.15.120:0/274062137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4cf8103700 0x7f4cf810bbf0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.240+0000 7f4d004b6640 1 --2- 172.21.15.120:0/274062137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4cf8102de0 0x7f4cf81031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.240+0000 7f4d004b6640 1 -- 172.21.15.120:0/274062137 >> 172.21.15.120:0/274062137 conn(0x7f4cf80fc7a0 msgr2=0x7f4cf80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:57.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.240+0000 7f4d004b6640 1 -- 172.21.15.120:0/274062137 shutdown_connections 2024-09-19T09:14:57.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.240+0000 7f4d004b6640 1 -- 172.21.15.120:0/274062137 wait complete. 2024-09-19T09:14:57.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.241+0000 7f4d004b6640 1 Processor -- start 2024-09-19T09:14:57.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.241+0000 7f4d004b6640 1 -- start start 2024-09-19T09:14:57.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.241+0000 7f4d004b6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4cf8102de0 0x7f4cf8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.241+0000 7f4d004b6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cf8103700 0x7f4cf8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.241+0000 7f4d004b6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4cf810c180 0x7f4cf819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.241+0000 7f4d004b6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf8111f70 con 0x7f4cf810c180 2024-09-19T09:14:57.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.241+0000 7f4d004b6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf8111df0 con 0x7f4cf8102de0 2024-09-19T09:14:57.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.241+0000 7f4d004b6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf81120f0 con 0x7f4cf8103700 2024-09-19T09:14:57.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.241+0000 7f4cfea53640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4cf810c180 0x7f4cf819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.241+0000 7f4cfe252640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4cf8102de0 0x7f4cf8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.241+0000 7f4cfea53640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4cf810c180 0x7f4cf819cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37954/0 (socket says 172.21.15.120:37954) 2024-09-19T09:14:57.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.241+0000 7f4cfea53640 1 -- 172.21.15.120:0/1135796452 learned_addr learned my addr 172.21.15.120:0/1135796452 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:57.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.241+0000 7f4cfda51640 1 --2- 172.21.15.120:0/1135796452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cf8103700 0x7f4cf8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.241+0000 7f4cfe252640 1 -- 172.21.15.120:0/1135796452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cf8103700 msgr2=0x7f4cf8198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:57.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.241+0000 7f4cfe252640 1 --2- 172.21.15.120:0/1135796452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cf8103700 0x7f4cf8198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.241+0000 7f4cfe252640 1 -- 172.21.15.120:0/1135796452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4cf810c180 msgr2=0x7f4cf819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:57.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.241+0000 7f4cfe252640 1 --2- 172.21.15.120:0/1135796452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4cf810c180 0x7f4cf819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.241+0000 7f4cfe252640 1 -- 172.21.15.120:0/1135796452 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf819d3e0 con 0x7f4cf8102de0 2024-09-19T09:14:57.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.242+0000 7f4cfe252640 1 --2- 172.21.15.120:0/1135796452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4cf8102de0 0x7f4cf8198430 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4cf400e9f0 tx=0x7f4cf400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:57.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.242+0000 7f4cef7fe640 1 -- 172.21.15.120:0/1135796452 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf400ccc0 con 0x7f4cf8102de0 2024-09-19T09:14:57.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.242+0000 7f4d004b6640 1 -- 172.21.15.120:0/1135796452 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf819d6d0 con 0x7f4cf8102de0 2024-09-19T09:14:57.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.242+0000 7f4cef7fe640 1 -- 172.21.15.120:0/1135796452 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cf4010910 con 0x7f4cf8102de0 2024-09-19T09:14:57.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.242+0000 7f4cef7fe640 1 -- 172.21.15.120:0/1135796452 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4018960 con 0x7f4cf8102de0 2024-09-19T09:14:57.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.242+0000 7f4d004b6640 1 -- 172.21.15.120:0/1135796452 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf81a4fb0 con 0x7f4cf8102de0 2024-09-19T09:14:57.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.244+0000 7f4d0288f740 1 Processor -- start 2024-09-19T09:14:57.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.244+0000 7f4cef7fe640 1 -- 172.21.15.120:0/1135796452 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f4cf4010ac0 con 0x7f4cf8102de0 2024-09-19T09:14:57.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.244+0000 7f4d0288f740 1 -- start start 2024-09-19T09:14:57.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.245+0000 7f4d0288f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dcf4bbb40 0x555dcf4ba090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.245+0000 7f4d0288f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555dcf4ba5d0 0x555dcf4baa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.245+0000 7f4d0288f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcf4c5370 0x555dcf4c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.245+0000 7f4d0288f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555dcf4202f0 con 0x555dcf4c5370 2024-09-19T09:14:57.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.245+0000 7f4d0288f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555dcf3c7cf0 con 0x555dcf4ba5d0 2024-09-19T09:14:57.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.245+0000 7f4d0288f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555dcf41c2b0 con 0x555dcf4bbb40 2024-09-19T09:14:57.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.245+0000 7f4cfea53640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dcf4bbb40 0x555dcf4ba090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.245+0000 7f4cfe252640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555dcf4ba5d0 0x555dcf4baa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.245+0000 7f4cfea53640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dcf4bbb40 0x555dcf4ba090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:58088/0 (socket says 172.21.15.120:58088) 2024-09-19T09:14:57.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.245+0000 7f4cfea53640 1 -- 172.21.15.120:0/1657784142 learned_addr learned my addr 172.21.15.120:0/1657784142 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:57.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.245+0000 7f4cfda51640 1 --2- 172.21.15.120:0/1657784142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcf4c5370 0x555dcf4c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.245+0000 7f4cfea53640 1 -- 172.21.15.120:0/1657784142 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555dcf4ba5d0 msgr2=0x555dcf4baa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:57.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.245+0000 7f4cfea53640 1 --2- 172.21.15.120:0/1657784142 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555dcf4ba5d0 0x555dcf4baa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.245+0000 7f4cfea53640 1 -- 172.21.15.120:0/1657784142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcf4c5370 msgr2=0x555dcf4c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:57.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.245+0000 7f4cfea53640 1 --2- 172.21.15.120:0/1657784142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcf4c5370 0x555dcf4c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.245+0000 7f4cfea53640 1 -- 172.21.15.120:0/1657784142 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dcf3ac3a0 con 0x555dcf4bbb40 2024-09-19T09:14:57.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.245+0000 7f4cfda51640 1 --2- 172.21.15.120:0/1657784142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcf4c5370 0x555dcf4c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:14:57.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.245+0000 7f4cef7fe640 1 --2- 172.21.15.120:0/1135796452 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4ce40a3240 0x7f4ce40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.246+0000 7f4cfe252640 1 --2- 172.21.15.120:0/1135796452 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4ce40a3240 0x7f4ce40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.246+0000 7f4cfea53640 1 --2- 172.21.15.120:0/1657784142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dcf4bbb40 0x555dcf4ba090 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009fb0 tx=0x7f4cf0011870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf02a7e392b10507 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:57.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.246+0000 7f4cef7fe640 1 -- 172.21.15.120:0/1135796452 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f4cf40cfc80 con 0x7f4cf8102de0 2024-09-19T09:14:57.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.246+0000 7f4cecff9640 1 -- 172.21.15.120:0/1657784142 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf00127d0 con 0x555dcf4bbb40 2024-09-19T09:14:57.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.246+0000 7f4cecff9640 1 -- 172.21.15.120:0/1657784142 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cf0012e50 con 0x555dcf4bbb40 2024-09-19T09:14:57.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.246+0000 7f4cecff9640 1 -- 172.21.15.120:0/1657784142 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf00193a0 con 0x555dcf4bbb40 2024-09-19T09:14:57.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.247+0000 7f4d0288f740 1 -- 172.21.15.120:0/1657784142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dcf4bbb40 msgr2=0x555dcf4ba090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:57.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.247+0000 7f4d0288f740 1 --2- 172.21.15.120:0/1657784142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dcf4bbb40 0x555dcf4ba090 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009fb0 tx=0x7f4cf0011870 comp rx=0 tx=0).stop 2024-09-19T09:14:57.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.247+0000 7f4cfe252640 1 --2- 172.21.15.120:0/1135796452 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4ce40a3240 0x7f4ce40a5700 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4cf400c6a0 tx=0x7f4cf40d3700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:57.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.247+0000 7f4d0288f740 1 -- 172.21.15.120:0/1657784142 shutdown_connections 2024-09-19T09:14:57.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.247+0000 7f4d0288f740 1 --2- 172.21.15.120:0/1657784142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcf4c5370 0x555dcf4c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.247+0000 7f4d0288f740 1 --2- 172.21.15.120:0/1657784142 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555dcf4ba5d0 0x555dcf4baa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.247+0000 7f4d0288f740 1 --2- 172.21.15.120:0/1657784142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dcf4bbb40 0x555dcf4ba090 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.247+0000 7f4d0288f740 1 -- 172.21.15.120:0/1657784142 >> 172.21.15.120:0/1657784142 conn(0x555dcf483310 msgr2=0x555dcf4b6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:57.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.247+0000 7f4d0288f740 1 -- 172.21.15.120:0/1657784142 shutdown_connections 2024-09-19T09:14:57.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.247+0000 7f4d0288f740 1 -- 172.21.15.120:0/1657784142 wait complete. 2024-09-19T09:14:57.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.247+0000 7f4d0288f740 1 -- start start 2024-09-19T09:14:57.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.247+0000 7f4d0288f740 1 Processor -- start 2024-09-19T09:14:57.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.248+0000 7f4d0288f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dcf4ba5d0 0x555dcf4c4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.248+0000 7f4d0288f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555dcf4bbb40 0x555dcf4c1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.248+0000 7f4d0288f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcf4c5370 0x555dcf4c2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.248+0000 7f4d0288f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555dcf4cc520 con 0x555dcf4c5370 2024-09-19T09:14:57.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.248+0000 7f4d0288f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555dcf3db060 con 0x555dcf4bbb40 2024-09-19T09:14:57.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.248+0000 7f4d0288f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555dcf4cc6a0 con 0x555dcf4ba5d0 2024-09-19T09:14:57.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.248+0000 7f4d0288f740 10 client.0 _ll_register_callbacks cb 0x555dcf12e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:57.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.248+0000 7f4cfea53640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcf4c5370 0x555dcf4c2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.248+0000 7f4cfda51640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555dcf4bbb40 0x555dcf4c1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.248+0000 7f4cfea53640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcf4c5370 0x555dcf4c2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37972/0 (socket says 172.21.15.120:37972) 2024-09-19T09:14:57.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.248+0000 7f4cfea53640 1 -- 172.21.15.120:0/3175936650 learned_addr learned my addr 172.21.15.120:0/3175936650 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:57.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.248+0000 7f4cfea53640 1 --2- 172.21.15.120:0/3175936650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dcf4ba5d0 0x555dcf4c4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.248+0000 7f4cfda51640 1 -- 172.21.15.120:0/3175936650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dcf4ba5d0 msgr2=0x555dcf4c4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:57.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.248+0000 7f4cfda51640 1 --2- 172.21.15.120:0/3175936650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dcf4ba5d0 0x555dcf4c4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.248+0000 7f4cfda51640 1 -- 172.21.15.120:0/3175936650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcf4c5370 msgr2=0x555dcf4c2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:57.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.248+0000 7f4cfda51640 1 --2- 172.21.15.120:0/3175936650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcf4c5370 0x555dcf4c2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.248+0000 7f4cfda51640 1 -- 172.21.15.120:0/3175936650 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dcf1913c0 con 0x555dcf4bbb40 2024-09-19T09:14:57.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.248+0000 7f4cfda51640 1 --2- 172.21.15.120:0/3175936650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555dcf4bbb40 0x555dcf4c1f00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8005940 tx=0x7f4ce80317e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10d0c5b231838e5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:57.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.248+0000 7f4ce1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:14:57.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.248+0000 7f4ce1ffb640 1 -- 172.21.15.120:0/3175936650 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8040070 con 0x555dcf4bbb40 2024-09-19T09:14:57.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.248+0000 7f4ce1ffb640 1 -- 172.21.15.120:0/3175936650 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f4ce8004450 con 0x555dcf4bbb40 2024-09-19T09:14:57.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.248+0000 7f4d0288f740 1 -- 172.21.15.120:0/3175936650 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cf0012e50 con 0x555dcf4bbb40 2024-09-19T09:14:57.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.249+0000 7f4ce1ffb640 1 -- 172.21.15.120:0/3175936650 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4ce8035040 con 0x555dcf4bbb40 2024-09-19T09:14:57.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.249+0000 7f4d0288f740 10 client.28721 fetch_fsmap learned FSMap version 33 2024-09-19T09:14:57.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.249+0000 7f4d0288f740 1 -- 172.21.15.120:0/3175936650 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555dcf4d7e40 con 0x555dcf4bbb40 2024-09-19T09:14:57.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.249+0000 7f4ce1ffb640 1 -- 172.21.15.120:0/3175936650 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cf0012e50 con 0x555dcf4bbb40 2024-09-19T09:14:57.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.249+0000 7f4ce1ffb640 1 -- 172.21.15.120:0/3175936650 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f4ce8045660 con 0x555dcf4bbb40 2024-09-19T09:14:57.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.249+0000 7f4d0288f740 10 client.28721 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:14:57.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.249+0000 7f4d0288f740 10 client.28721 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:14:57.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.249+0000 7f4d0288f740 20 client.28721 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:57.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.249+0000 7f4d0288f740 1 --2- 172.21.15.120:0/3175936650 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x555dcf4d9300 0x555dcf4db6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.249+0000 7f4d0288f740 4 client.28721 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:14:57.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.249+0000 7f4d0288f740 1 -- 172.21.15.120:0/3175936650 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555dcf42d860 con 0x555dcf4d9300 2024-09-19T09:14:57.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.250+0000 7f4cfea53640 1 --2- 172.21.15.120:0/3175936650 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x555dcf4d9300 0x555dcf4db6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.250+0000 7f4ce1ffb640 10 client.28721 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:14:57.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.250+0000 7f4cfea53640 1 --2- 172.21.15.120:0/3175936650 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x555dcf4d9300 0x555dcf4db6f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d7e3819fd31d502 server_cookie=d3d613cfc80893e8 in_seq=0 out_seq=0 2024-09-19T09:14:57.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.250+0000 7f4ce1ffb640 1 -- 172.21.15.120:0/3175936650 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555dcf42d860 con 0x555dcf4d9300 2024-09-19T09:14:57.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.250+0000 7f4ce1ffb640 10 client.28721 handle_command_reply: tid=0 2024-09-19T09:14:57.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.251+0000 7f4d0288f740 1 client.28721 shutdown 2024-09-19T09:14:57.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.251+0000 7f4d0288f740 1 -- 172.21.15.120:0/3175936650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555dcf4bbb40 msgr2=0x555dcf4c1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:57.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.251+0000 7f4d0288f740 1 --2- 172.21.15.120:0/3175936650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555dcf4bbb40 0x555dcf4c1f00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8005940 tx=0x7f4ce80317e0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.251+0000 7f4d0288f740 1 -- 172.21.15.120:0/3175936650 shutdown_connections 2024-09-19T09:14:57.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.251+0000 7f4d0288f740 1 --2- 172.21.15.120:0/3175936650 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x555dcf4d9300 0x555dcf4db6f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.251+0000 7f4d0288f740 1 --2- 172.21.15.120:0/3175936650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcf4c5370 0x555dcf4c2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.251+0000 7f4ce1ffb640 0 client.28721 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:14:57.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.251+0000 7f4d0288f740 1 --2- 172.21.15.120:0/3175936650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555dcf4bbb40 0x555dcf4c1f00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.251+0000 7f4d0288f740 1 --2- 172.21.15.120:0/3175936650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dcf4ba5d0 0x555dcf4c4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.251+0000 7f4d0288f740 1 -- 172.21.15.120:0/3175936650 >> 172.21.15.120:0/3175936650 conn(0x555dcf483310 msgr2=0x555dcf4bc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:57.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.251+0000 7f4d0288f740 1 -- 172.21.15.120:0/3175936650 shutdown_connections 2024-09-19T09:14:57.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.251+0000 7f4d0288f740 1 -- 172.21.15.120:0/3175936650 wait complete. 2024-09-19T09:14:57.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.251+0000 7f4d0288f740 20 client.28721 trim_cache size 0 max 16384 2024-09-19T09:14:57.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.264+0000 7f4d0288f740 1 Processor -- start 2024-09-19T09:14:57.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.264+0000 7f4d0288f740 1 -- start start 2024-09-19T09:14:57.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.264+0000 7f4d0288f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dcf4ba5d0 0x555dcf4d57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.264+0000 7f4d0288f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcf4bbb40 0x555dcf4e0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.264+0000 7f4d0288f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555dcf4c5370 0x555dcf4e1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.264+0000 7f4d0288f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf81ab3a0 con 0x555dcf4bbb40 2024-09-19T09:14:57.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.264+0000 7f4d0288f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf81acee0 con 0x555dcf4c5370 2024-09-19T09:14:57.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.264+0000 7f4d0288f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf81aba70 con 0x555dcf4ba5d0 2024-09-19T09:14:57.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.264+0000 7f4cfda51640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcf4bbb40 0x555dcf4e0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.264+0000 7f4cfda51640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcf4bbb40 0x555dcf4e0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37980/0 (socket says 172.21.15.120:37980) 2024-09-19T09:14:57.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.264+0000 7f4cfda51640 1 -- 172.21.15.120:0/2540588755 learned_addr learned my addr 172.21.15.120:0/2540588755 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:57.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.264+0000 7f4cfea53640 1 --2- 172.21.15.120:0/2540588755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dcf4ba5d0 0x555dcf4d57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.264+0000 7f4cfea53640 1 --2- 172.21.15.120:0/2540588755 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555dcf4c5370 0x555dcf4e1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.265+0000 7f4cfda51640 1 -- 172.21.15.120:0/2540588755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dcf4ba5d0 msgr2=0x555dcf4d57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:57.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.265+0000 7f4cfda51640 1 --2- 172.21.15.120:0/2540588755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dcf4ba5d0 0x555dcf4d57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.265+0000 7f4cfda51640 1 -- 172.21.15.120:0/2540588755 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555dcf4c5370 msgr2=0x555dcf4e1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:57.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.265+0000 7f4cfda51640 1 --2- 172.21.15.120:0/2540588755 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555dcf4c5370 0x555dcf4e1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.265+0000 7f4cfda51640 1 -- 172.21.15.120:0/2540588755 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dcf4bd120 con 0x555dcf4bbb40 2024-09-19T09:14:57.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.265+0000 7f4cfea53640 1 --2- 172.21.15.120:0/2540588755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dcf4ba5d0 0x555dcf4d57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:57.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.265+0000 7f4cfda51640 1 --2- 172.21.15.120:0/2540588755 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcf4bbb40 0x555dcf4e0fc0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8045a80 tx=0x7f4ce8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5752063736e592b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:57.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.265+0000 7f4ce3fff640 1 -- 172.21.15.120:0/2540588755 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8040040 con 0x555dcf4bbb40 2024-09-19T09:14:57.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.265+0000 7f4ce3fff640 1 -- 172.21.15.120:0/2540588755 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ce8004450 con 0x555dcf4bbb40 2024-09-19T09:14:57.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.265+0000 7f4ce3fff640 1 -- 172.21.15.120:0/2540588755 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce80049b0 con 0x555dcf4bbb40 2024-09-19T09:14:57.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.266+0000 7f4d0288f740 1 -- 172.21.15.120:0/2540588755 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcf4bbb40 msgr2=0x555dcf4e0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:57.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.266+0000 7f4d0288f740 1 --2- 172.21.15.120:0/2540588755 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcf4bbb40 0x555dcf4e0fc0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8045a80 tx=0x7f4ce8000bc0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.266+0000 7f4d0288f740 1 -- 172.21.15.120:0/2540588755 shutdown_connections 2024-09-19T09:14:57.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.266+0000 7f4d0288f740 1 --2- 172.21.15.120:0/2540588755 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555dcf4c5370 0x555dcf4e1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.266+0000 7f4d0288f740 1 --2- 172.21.15.120:0/2540588755 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcf4bbb40 0x555dcf4e0fc0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.266+0000 7f4d0288f740 1 --2- 172.21.15.120:0/2540588755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dcf4ba5d0 0x555dcf4d57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.266+0000 7f4d0288f740 1 -- 172.21.15.120:0/2540588755 >> 172.21.15.120:0/2540588755 conn(0x555dcf483310 msgr2=0x555dcf4c3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:57.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.266+0000 7f4d0288f740 1 -- 172.21.15.120:0/2540588755 shutdown_connections 2024-09-19T09:14:57.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.266+0000 7f4d0288f740 1 -- 172.21.15.120:0/2540588755 wait complete. 2024-09-19T09:14:57.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.266+0000 7f4d0288f740 1 -- start start 2024-09-19T09:14:57.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.266+0000 7f4d0288f740 1 Processor -- start 2024-09-19T09:14:57.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.267+0000 7f4d0288f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcf4ba5d0 0x555dcf4d2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.267+0000 7f4d0288f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dcf4bbb40 0x555dcf4cd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.267+0000 7f4d0288f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555dcf4c5370 0x555dcf4cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.267+0000 7f4d0288f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf81ac810 con 0x555dcf4ba5d0 2024-09-19T09:14:57.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.267+0000 7f4d0288f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf81ac140 con 0x555dcf4c5370 2024-09-19T09:14:57.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.267+0000 7f4d0288f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf81a9380 con 0x555dcf4bbb40 2024-09-19T09:14:57.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.267+0000 7f4d0288f740 10 client.0 _ll_register_callbacks cb 0x7f4ce8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:57.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.267+0000 7f4cfea53640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcf4ba5d0 0x555dcf4d2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.267+0000 7f4cfea53640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcf4ba5d0 0x555dcf4d2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37990/0 (socket says 172.21.15.120:37990) 2024-09-19T09:14:57.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.267+0000 7f4cfea53640 1 -- 172.21.15.120:0/1811885400 learned_addr learned my addr 172.21.15.120:0/1811885400 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:57.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.267+0000 7f4cfea53640 1 --2- 172.21.15.120:0/1811885400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555dcf4c5370 0x555dcf4cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.267+0000 7f4cfda51640 1 --2- 172.21.15.120:0/1811885400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dcf4bbb40 0x555dcf4cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.267+0000 7f4cfea53640 1 -- 172.21.15.120:0/1811885400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dcf4bbb40 msgr2=0x555dcf4cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:57.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.267+0000 7f4cfea53640 1 --2- 172.21.15.120:0/1811885400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dcf4bbb40 0x555dcf4cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.267+0000 7f4cfea53640 1 -- 172.21.15.120:0/1811885400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555dcf4c5370 msgr2=0x555dcf4cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:57.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.267+0000 7f4cfea53640 1 --2- 172.21.15.120:0/1811885400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555dcf4c5370 0x555dcf4cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.267+0000 7f4cfea53640 1 -- 172.21.15.120:0/1811885400 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dcf542f90 con 0x555dcf4ba5d0 2024-09-19T09:14:57.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.267+0000 7f4cfea53640 1 --2- 172.21.15.120:0/1811885400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcf4ba5d0 0x555dcf4d2210 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009cc0 tx=0x7f4ce8009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e70457ea02b1d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:57.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.267+0000 7f4ce1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:14:57.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.267+0000 7f4ce1ffb640 1 -- 172.21.15.120:0/1811885400 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf002d030 con 0x555dcf4ba5d0 2024-09-19T09:14:57.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.267+0000 7f4ce1ffb640 1 -- 172.21.15.120:0/1811885400 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f4cf001a6f0 con 0x555dcf4ba5d0 2024-09-19T09:14:57.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.267+0000 7f4d0288f740 1 -- 172.21.15.120:0/1811885400 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ce8004450 con 0x555dcf4ba5d0 2024-09-19T09:14:57.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.267+0000 7f4ce1ffb640 1 -- 172.21.15.120:0/1811885400 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4cf0004a00 con 0x555dcf4ba5d0 2024-09-19T09:14:57.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.268+0000 7f4d0288f740 10 client.19671 fetch_fsmap learned FSMap version 33 2024-09-19T09:14:57.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.268+0000 7f4d0288f740 1 -- 172.21.15.120:0/1811885400 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555dcf544130 con 0x555dcf4ba5d0 2024-09-19T09:14:57.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.268+0000 7f4ce1ffb640 1 -- 172.21.15.120:0/1811885400 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ce8004450 con 0x555dcf4ba5d0 2024-09-19T09:14:57.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.268+0000 7f4ce1ffb640 1 -- 172.21.15.120:0/1811885400 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f4cf0026040 con 0x555dcf4ba5d0 2024-09-19T09:14:57.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.268+0000 7f4d0288f740 10 client.19671 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:14:57.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.268+0000 7f4d0288f740 10 client.19671 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:14:57.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.268+0000 7f4d0288f740 20 client.19671 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:57.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.268+0000 7f4d0288f740 1 --2- 172.21.15.120:0/1811885400 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x555dcf5446b0 0x555dcf546aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.268+0000 7f4d0288f740 4 client.19671 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:14:57.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.268+0000 7f4d0288f740 1 -- 172.21.15.120:0/1811885400 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555dcf427d70 con 0x555dcf5446b0 2024-09-19T09:14:57.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.268+0000 7f4cfda51640 1 --2- 172.21.15.120:0/1811885400 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x555dcf5446b0 0x555dcf546aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.269+0000 7f4ce1ffb640 10 client.19671 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:14:57.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.269+0000 7f4cfda51640 1 --2- 172.21.15.120:0/1811885400 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x555dcf5446b0 0x555dcf546aa0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=93d9725c8262c770 server_cookie=d31d74869927eb6b in_seq=0 out_seq=0 2024-09-19T09:14:57.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.269+0000 7f4ce1ffb640 1 -- 172.21.15.120:0/1811885400 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555dcf427d70 con 0x555dcf5446b0 2024-09-19T09:14:57.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.269+0000 7f4ce1ffb640 10 client.19671 handle_command_reply: tid=0 2024-09-19T09:14:57.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.269+0000 7f4d0288f740 1 client.19671 shutdown 2024-09-19T09:14:57.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.269+0000 7f4d0288f740 1 -- 172.21.15.120:0/1811885400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcf4ba5d0 msgr2=0x555dcf4d2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:57.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.269+0000 7f4d0288f740 1 --2- 172.21.15.120:0/1811885400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcf4ba5d0 0x555dcf4d2210 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009cc0 tx=0x7f4ce8009630 comp rx=0 tx=0).stop 2024-09-19T09:14:57.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.269+0000 7f4d0288f740 1 -- 172.21.15.120:0/1811885400 shutdown_connections 2024-09-19T09:14:57.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.269+0000 7f4d0288f740 1 --2- 172.21.15.120:0/1811885400 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x555dcf5446b0 0x555dcf546aa0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.269+0000 7f4d0288f740 1 --2- 172.21.15.120:0/1811885400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555dcf4c5370 0x555dcf4cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.269+0000 7f4d0288f740 1 --2- 172.21.15.120:0/1811885400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dcf4bbb40 0x555dcf4cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.269+0000 7f4d0288f740 1 --2- 172.21.15.120:0/1811885400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcf4ba5d0 0x555dcf4d2210 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.269+0000 7f4d0288f740 1 -- 172.21.15.120:0/1811885400 >> 172.21.15.120:0/1811885400 conn(0x555dcf483310 msgr2=0x555dcf4b54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:57.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.269+0000 7f4ce1ffb640 0 client.19671 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:14:57.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.269+0000 7f4d0288f740 1 -- 172.21.15.120:0/1811885400 shutdown_connections 2024-09-19T09:14:57.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.269+0000 7f4d0288f740 1 -- 172.21.15.120:0/1811885400 wait complete. 2024-09-19T09:14:57.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.269+0000 7f4d0288f740 20 client.19671 trim_cache size 0 max 16384 2024-09-19T09:14:57.281 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:14:57.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.270+0000 7f4d004b6640 1 -- 172.21.15.120:0/1135796452 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4ce40a3240 msgr2=0x7f4ce40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:57.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.270+0000 7f4d004b6640 1 --2- 172.21.15.120:0/1135796452 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4ce40a3240 0x7f4ce40a5700 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4cf400c6a0 tx=0x7f4cf40d3700 comp rx=0 tx=0).stop 2024-09-19T09:14:57.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.270+0000 7f4d004b6640 1 -- 172.21.15.120:0/1135796452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4cf8102de0 msgr2=0x7f4cf8198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:57.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.270+0000 7f4d004b6640 1 --2- 172.21.15.120:0/1135796452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4cf8102de0 0x7f4cf8198430 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4cf400e9f0 tx=0x7f4cf400eeb0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.270+0000 7f4d004b6640 1 -- 172.21.15.120:0/1135796452 shutdown_connections 2024-09-19T09:14:57.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.270+0000 7f4d004b6640 1 --2- 172.21.15.120:0/1135796452 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4ce40a3240 0x7f4ce40a5700 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.270+0000 7f4d004b6640 1 --2- 172.21.15.120:0/1135796452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4cf810c180 0x7f4cf819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.270+0000 7f4d004b6640 1 --2- 172.21.15.120:0/1135796452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cf8103700 0x7f4cf8198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.270+0000 7f4d004b6640 1 --2- 172.21.15.120:0/1135796452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4cf8102de0 0x7f4cf8198430 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.270+0000 7f4d004b6640 1 -- 172.21.15.120:0/1135796452 >> 172.21.15.120:0/1135796452 conn(0x7f4cf80fc7a0 msgr2=0x7f4cf80fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:57.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.270+0000 7f4d004b6640 1 -- 172.21.15.120:0/1135796452 shutdown_connections 2024-09-19T09:14:57.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:57.270+0000 7f4d004b6640 1 -- 172.21.15.120:0/1135796452 wait complete. 2024-09-19T09:14:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.613+0000 7fd6f1d98640 1 -- 172.21.15.120:0/3661026368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6ec102640 msgr2=0x7fd6ec108ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.613+0000 7fd6f1d98640 1 --2- 172.21.15.120:0/3661026368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6ec102640 0x7fd6ec108ed0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009ca0 tx=0x7fd6e402a950 comp rx=0 tx=0).stop 2024-09-19T09:14:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.613+0000 7fd6f1d98640 1 -- 172.21.15.120:0/3661026368 shutdown_connections 2024-09-19T09:14:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.613+0000 7fd6f1d98640 1 --2- 172.21.15.120:0/3661026368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6ec102640 0x7fd6ec108ed0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.613+0000 7fd6f1d98640 1 --2- 172.21.15.120:0/3661026368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6ec101c80 0x7fd6ec102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.613+0000 7fd6f1d98640 1 --2- 172.21.15.120:0/3661026368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6ec100f70 0x7fd6ec101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.613+0000 7fd6f1d98640 1 -- 172.21.15.120:0/3661026368 >> 172.21.15.120:0/3661026368 conn(0x7fd6ec0fb930 msgr2=0x7fd6ec0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.613+0000 7fd6f1d98640 1 -- 172.21.15.120:0/3661026368 shutdown_connections 2024-09-19T09:14:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.613+0000 7fd6f1d98640 1 -- 172.21.15.120:0/3661026368 wait complete. 2024-09-19T09:14:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.614+0000 7fd6f1d98640 1 Processor -- start 2024-09-19T09:14:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.614+0000 7fd6f1d98640 1 -- start start 2024-09-19T09:14:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.614+0000 7fd6f1d98640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6ec100f70 0x7fd6ec196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.614+0000 7fd6f1d98640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6ec101c80 0x7fd6ec197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.614+0000 7fd6f1d98640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6ec102640 0x7fd6ec19e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.614+0000 7fd6f1d98640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ec10c6f0 con 0x7fd6ec100f70 2024-09-19T09:14:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.614+0000 7fd6f1d98640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ec10c570 con 0x7fd6ec102640 2024-09-19T09:14:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.614+0000 7fd6f1d98640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ec10c870 con 0x7fd6ec101c80 2024-09-19T09:14:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.614+0000 7fd6eb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6ec100f70 0x7fd6ec196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.614+0000 7fd6eb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6ec100f70 0x7fd6ec196d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38008/0 (socket says 172.21.15.120:38008) 2024-09-19T09:14:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.614+0000 7fd6eb7fe640 1 -- 172.21.15.120:0/2866312995 learned_addr learned my addr 172.21.15.120:0/2866312995 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.614+0000 7fd6eaffd640 1 --2- 172.21.15.120:0/2866312995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6ec101c80 0x7fd6ec197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.614+0000 7fd6ebfff640 1 --2- 172.21.15.120:0/2866312995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6ec102640 0x7fd6ec19e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.615+0000 7fd6eb7fe640 1 -- 172.21.15.120:0/2866312995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6ec101c80 msgr2=0x7fd6ec197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.615+0000 7fd6eb7fe640 1 --2- 172.21.15.120:0/2866312995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6ec101c80 0x7fd6ec197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.615+0000 7fd6eb7fe640 1 -- 172.21.15.120:0/2866312995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6ec102640 msgr2=0x7fd6ec19e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.615+0000 7fd6eb7fe640 1 --2- 172.21.15.120:0/2866312995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6ec102640 0x7fd6ec19e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.615+0000 7fd6eb7fe640 1 -- 172.21.15.120:0/2866312995 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6ec19e910 con 0x7fd6ec100f70 2024-09-19T09:14:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.615+0000 7fd6eb7fe640 1 --2- 172.21.15.120:0/2866312995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6ec100f70 0x7fd6ec196d00 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00d950 tx=0x7fd6dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.615+0000 7fd6e8ff9640 1 -- 172.21.15.120:0/2866312995 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc017070 con 0x7fd6ec100f70 2024-09-19T09:14:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.615+0000 7fd6e8ff9640 1 -- 172.21.15.120:0/2866312995 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6dc009c30 con 0x7fd6ec100f70 2024-09-19T09:14:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.615+0000 7fd6f1d98640 1 -- 172.21.15.120:0/2866312995 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6ec19ec00 con 0x7fd6ec100f70 2024-09-19T09:14:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.615+0000 7fd6e8ff9640 1 -- 172.21.15.120:0/2866312995 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc00f600 con 0x7fd6ec100f70 2024-09-19T09:14:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.615+0000 7fd6f1d98640 1 -- 172.21.15.120:0/2866312995 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6ec19f150 con 0x7fd6ec100f70 2024-09-19T09:14:57.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.617+0000 7fd6e8ff9640 1 -- 172.21.15.120:0/2866312995 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd6dc01a070 con 0x7fd6ec100f70 2024-09-19T09:14:57.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.617+0000 7fd6f4c0f740 1 Processor -- start 2024-09-19T09:14:57.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.618+0000 7fd6f4c0f740 1 -- start start 2024-09-19T09:14:57.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.618+0000 7fd6f4c0f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff431d3bb0 0x55ff431d3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.618+0000 7fd6f4c0f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff431d44f0 0x55ff431d8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.618+0000 7fd6f4c0f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ff431d9580 0x55ff431dba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.618+0000 7fd6f4c0f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ff431352f0 con 0x55ff431d9580 2024-09-19T09:14:57.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.618+0000 7fd6f4c0f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ff430dd270 con 0x55ff431d3bb0 2024-09-19T09:14:57.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.618+0000 7fd6f4c0f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ff431312b0 con 0x55ff431d44f0 2024-09-19T09:14:57.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.618+0000 7fd6ebfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff431d3bb0 0x55ff431d3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.618+0000 7fd6ebfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff431d3bb0 0x55ff431d3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:41224/0 (socket says 172.21.15.120:41224) 2024-09-19T09:14:57.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.618+0000 7fd6ebfff640 1 -- 172.21.15.120:0/463210443 learned_addr learned my addr 172.21.15.120:0/463210443 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:57.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.618+0000 7fd6eaffd640 1 --2- 172.21.15.120:0/463210443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ff431d9580 0x55ff431dba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.618+0000 7fd6eb7fe640 1 --2- 172.21.15.120:0/463210443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff431d44f0 0x55ff431d8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.618+0000 7fd6ebfff640 1 -- 172.21.15.120:0/463210443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff431d44f0 msgr2=0x55ff431d8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.618+0000 7fd6ebfff640 1 --2- 172.21.15.120:0/463210443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff431d44f0 0x55ff431d8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.618+0000 7fd6ebfff640 1 -- 172.21.15.120:0/463210443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ff431d9580 msgr2=0x55ff431dba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.618+0000 7fd6ebfff640 1 --2- 172.21.15.120:0/463210443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ff431d9580 0x55ff431dba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.618+0000 7fd6ebfff640 1 -- 172.21.15.120:0/463210443 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff430dc340 con 0x55ff431d3bb0 2024-09-19T09:14:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.618+0000 7fd6e8ff9640 1 --2- 172.21.15.120:0/2866312995 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd6d40a3370 0x7fd6d40a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.618+0000 7fd6eb7fe640 1 --2- 172.21.15.120:0/463210443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff431d44f0 0x55ff431d8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:14:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.618+0000 7fd6eb7fe640 1 --2- 172.21.15.120:0/2866312995 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd6d40a3370 0x7fd6d40a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.618+0000 7fd6ebfff640 1 --2- 172.21.15.120:0/463210443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff431d3bb0 0x55ff431d3fb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4030030 tx=0x7fd6e4009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c890d56a172218b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.619+0000 7fd6e8ff9640 1 -- 172.21.15.120:0/2866312995 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fd6dc0cf310 con 0x7fd6ec100f70 2024-09-19T09:14:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.619+0000 7fd6d1ffb640 1 -- 172.21.15.120:0/463210443 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4048070 con 0x55ff431d3bb0 2024-09-19T09:14:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.619+0000 7fd6d1ffb640 1 -- 172.21.15.120:0/463210443 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6e4003e50 con 0x55ff431d3bb0 2024-09-19T09:14:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.619+0000 7fd6d1ffb640 1 -- 172.21.15.120:0/463210443 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4034040 con 0x55ff431d3bb0 2024-09-19T09:14:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.619+0000 7fd6f4c0f740 1 -- 172.21.15.120:0/463210443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff431d3bb0 msgr2=0x55ff431d3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.619+0000 7fd6eb7fe640 1 --2- 172.21.15.120:0/2866312995 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd6d40a3370 0x7fd6d40a5830 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc0a4070 tx=0x7fd6dc010ce0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.619+0000 7fd6f4c0f740 1 --2- 172.21.15.120:0/463210443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff431d3bb0 0x55ff431d3fb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4030030 tx=0x7fd6e4009800 comp rx=0 tx=0).stop 2024-09-19T09:14:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.619+0000 7fd6f4c0f740 1 -- 172.21.15.120:0/463210443 shutdown_connections 2024-09-19T09:14:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.619+0000 7fd6f4c0f740 1 --2- 172.21.15.120:0/463210443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ff431d9580 0x55ff431dba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.620+0000 7fd6f4c0f740 1 --2- 172.21.15.120:0/463210443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff431d44f0 0x55ff431d8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.620+0000 7fd6f4c0f740 1 --2- 172.21.15.120:0/463210443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff431d3bb0 0x55ff431d3fb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.620+0000 7fd6f4c0f740 1 -- 172.21.15.120:0/463210443 >> 172.21.15.120:0/463210443 conn(0x55ff431cf220 msgr2=0x55ff431cf620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.620+0000 7fd6f4c0f740 1 -- 172.21.15.120:0/463210443 shutdown_connections 2024-09-19T09:14:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.620+0000 7fd6f4c0f740 1 -- 172.21.15.120:0/463210443 wait complete. 2024-09-19T09:14:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.620+0000 7fd6f4c0f740 1 -- start start 2024-09-19T09:14:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.620+0000 7fd6f4c0f740 1 Processor -- start 2024-09-19T09:14:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.620+0000 7fd6f4c0f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff431d3bb0 0x55ff431ddde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.620+0000 7fd6f4c0f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ff431d44f0 0x55ff431de320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.620+0000 7fd6f4c0f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff431d9580 0x55ff431e52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.620+0000 7fd6f4c0f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ff431dc4f0 con 0x55ff431d44f0 2024-09-19T09:14:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.620+0000 7fd6f4c0f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ff430be290 con 0x55ff431d9580 2024-09-19T09:14:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.620+0000 7fd6f4c0f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ff431dc670 con 0x55ff431d3bb0 2024-09-19T09:14:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.620+0000 7fd6f4c0f740 10 client.0 _ll_register_callbacks cb 0x55ff430ef140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.620+0000 7fd6eaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ff431d44f0 0x55ff431de320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.620+0000 7fd6ebfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff431d3bb0 0x55ff431ddde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.620+0000 7fd6eaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ff431d44f0 0x55ff431de320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38022/0 (socket says 172.21.15.120:38022) 2024-09-19T09:14:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.621+0000 7fd6eaffd640 1 -- 172.21.15.120:0/3433473648 learned_addr learned my addr 172.21.15.120:0/3433473648 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.621+0000 7fd6ebfff640 1 --2- 172.21.15.120:0/3433473648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff431d9580 0x55ff431e52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.621+0000 7fd6eaffd640 1 -- 172.21.15.120:0/3433473648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff431d3bb0 msgr2=0x55ff431ddde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.621+0000 7fd6eaffd640 1 --2- 172.21.15.120:0/3433473648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff431d3bb0 0x55ff431ddde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.621+0000 7fd6eaffd640 1 -- 172.21.15.120:0/3433473648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff431d9580 msgr2=0x55ff431e52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.621+0000 7fd6eaffd640 1 --2- 172.21.15.120:0/3433473648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff431d9580 0x55ff431e52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.621+0000 7fd6eaffd640 1 -- 172.21.15.120:0/3433473648 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff42e48210 con 0x55ff431d44f0 2024-09-19T09:14:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.621+0000 7fd6eaffd640 1 --2- 172.21.15.120:0/3433473648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ff431d44f0 0x55ff431de320 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0012740 tx=0x7fd6e0012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d19dd9606174f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.621+0000 7fd6caffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:14:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.621+0000 7fd6caffd640 1 -- 172.21.15.120:0/3433473648 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e000f920 con 0x55ff431d44f0 2024-09-19T09:14:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.621+0000 7fd6caffd640 1 -- 172.21.15.120:0/3433473648 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fd6e000fac0 con 0x55ff431d44f0 2024-09-19T09:14:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.621+0000 7fd6f4c0f740 1 -- 172.21.15.120:0/3433473648 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6e4003e50 con 0x55ff431d44f0 2024-09-19T09:14:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.621+0000 7fd6caffd640 1 -- 172.21.15.120:0/3433473648 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd6e000fcc0 con 0x55ff431d44f0 2024-09-19T09:14:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.621+0000 7fd6f4c0f740 10 client.19683 fetch_fsmap learned FSMap version 33 2024-09-19T09:14:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.621+0000 7fd6f4c0f740 1 -- 172.21.15.120:0/3433473648 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ff42f14620 con 0x55ff431d44f0 2024-09-19T09:14:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.621+0000 7fd6caffd640 1 -- 172.21.15.120:0/3433473648 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6e4003e50 con 0x55ff431d44f0 2024-09-19T09:14:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.622+0000 7fd6caffd640 1 -- 172.21.15.120:0/3433473648 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fd6e001c420 con 0x55ff431d44f0 2024-09-19T09:14:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.622+0000 7fd6f4c0f740 10 client.19683 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:14:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.622+0000 7fd6f4c0f740 10 client.19683 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:14:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.622+0000 7fd6f4c0f740 20 client.19683 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.622+0000 7fd6f4c0f740 1 --2- 172.21.15.120:0/3433473648 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ff431e9b00 0x55ff431ebf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.622+0000 7fd6f4c0f740 4 client.19683 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:14:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.622+0000 7fd6f4c0f740 1 -- 172.21.15.120:0/3433473648 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff43142a20 con 0x55ff431e9b00 2024-09-19T09:14:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.622+0000 7fd6ebfff640 1 --2- 172.21.15.120:0/3433473648 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ff431e9b00 0x55ff431ebf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.622+0000 7fd6caffd640 10 client.19683 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.622+0000 7fd6ebfff640 1 --2- 172.21.15.120:0/3433473648 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ff431e9b00 0x55ff431ebf20 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26135313e519a87d server_cookie=c38f687569eb69e7 in_seq=0 out_seq=0 2024-09-19T09:14:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.624+0000 7fd6caffd640 1 -- 172.21.15.120:0/3433473648 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ff43142a20 con 0x55ff431e9b00 2024-09-19T09:14:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.624+0000 7fd6caffd640 10 client.19683 handle_command_reply: tid=0 2024-09-19T09:14:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.624+0000 7fd6f4c0f740 1 client.19683 shutdown 2024-09-19T09:14:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.624+0000 7fd6f4c0f740 1 -- 172.21.15.120:0/3433473648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ff431d44f0 msgr2=0x55ff431de320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.624+0000 7fd6f4c0f740 1 --2- 172.21.15.120:0/3433473648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ff431d44f0 0x55ff431de320 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0012740 tx=0x7fd6e0012c00 comp rx=0 tx=0).stop 2024-09-19T09:14:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.624+0000 7fd6f4c0f740 1 -- 172.21.15.120:0/3433473648 shutdown_connections 2024-09-19T09:14:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.624+0000 7fd6f4c0f740 1 --2- 172.21.15.120:0/3433473648 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ff431e9b00 0x55ff431ebf20 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.624+0000 7fd6f4c0f740 1 --2- 172.21.15.120:0/3433473648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff431d9580 0x55ff431e52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.624+0000 7fd6caffd640 0 client.19683 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.624+0000 7fd6f4c0f740 1 --2- 172.21.15.120:0/3433473648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ff431d44f0 0x55ff431de320 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.624+0000 7fd6f4c0f740 1 --2- 172.21.15.120:0/3433473648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff431d3bb0 0x55ff431ddde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.624+0000 7fd6f4c0f740 1 -- 172.21.15.120:0/3433473648 >> 172.21.15.120:0/3433473648 conn(0x55ff431cf220 msgr2=0x55ff431cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.624+0000 7fd6f4c0f740 1 -- 172.21.15.120:0/3433473648 shutdown_connections 2024-09-19T09:14:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.624+0000 7fd6f4c0f740 1 -- 172.21.15.120:0/3433473648 wait complete. 2024-09-19T09:14:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.624+0000 7fd6f4c0f740 20 client.19683 trim_cache size 0 max 16384 2024-09-19T09:14:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.638+0000 7fd6f4c0f740 1 Processor -- start 2024-09-19T09:14:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.638+0000 7fd6f4c0f740 1 -- start start 2024-09-19T09:14:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.638+0000 7fd6f4c0f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ff431d3bb0 0x55ff431cff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.638+0000 7fd6f4c0f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff431d44f0 0x55ff431e52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.638+0000 7fd6f4c0f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff431d9580 0x55ff431d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.638+0000 7fd6f4c0f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ec1a6cb0 con 0x55ff431d3bb0 2024-09-19T09:14:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.638+0000 7fd6f4c0f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ec1a87f0 con 0x55ff431d44f0 2024-09-19T09:14:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.638+0000 7fd6f4c0f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ec1a7380 con 0x55ff431d9580 2024-09-19T09:14:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.638+0000 7fd6ebfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ff431d3bb0 0x55ff431cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.638+0000 7fd6ebfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ff431d3bb0 0x55ff431cff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38032/0 (socket says 172.21.15.120:38032) 2024-09-19T09:14:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.638+0000 7fd6ebfff640 1 -- 172.21.15.120:0/1575259924 learned_addr learned my addr 172.21.15.120:0/1575259924 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.639+0000 7fd6eaffd640 1 --2- 172.21.15.120:0/1575259924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff431d44f0 0x55ff431e52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.639+0000 7fd6ebfff640 1 -- 172.21.15.120:0/1575259924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff431d9580 msgr2=0x55ff431d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.639+0000 7fd6ebfff640 1 --2- 172.21.15.120:0/1575259924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff431d9580 0x55ff431d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.639+0000 7fd6ebfff640 1 -- 172.21.15.120:0/1575259924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff431d44f0 msgr2=0x55ff431e52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.639+0000 7fd6ebfff640 1 --2- 172.21.15.120:0/1575259924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff431d44f0 0x55ff431e52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.639+0000 7fd6ebfff640 1 -- 172.21.15.120:0/1575259924 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff431cf040 con 0x55ff431d3bb0 2024-09-19T09:14:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.639+0000 7fd6eaffd640 1 --2- 172.21.15.120:0/1575259924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff431d44f0 0x55ff431e52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:14:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.639+0000 7fd6ebfff640 1 --2- 172.21.15.120:0/1575259924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ff431d3bb0 0x55ff431cff60 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4046040 tx=0x7fd6e402f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a555106cd200fba4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.639+0000 7fd6d17fa640 1 -- 172.21.15.120:0/1575259924 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4048040 con 0x55ff431d3bb0 2024-09-19T09:14:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.639+0000 7fd6d17fa640 1 -- 172.21.15.120:0/1575259924 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6e4049040 con 0x55ff431d3bb0 2024-09-19T09:14:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.639+0000 7fd6d17fa640 1 -- 172.21.15.120:0/1575259924 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4003d90 con 0x55ff431d3bb0 2024-09-19T09:14:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.640+0000 7fd6f4c0f740 1 -- 172.21.15.120:0/1575259924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ff431d3bb0 msgr2=0x55ff431cff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.640+0000 7fd6f4c0f740 1 --2- 172.21.15.120:0/1575259924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ff431d3bb0 0x55ff431cff60 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4046040 tx=0x7fd6e402f5c0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.640+0000 7fd6f4c0f740 1 -- 172.21.15.120:0/1575259924 shutdown_connections 2024-09-19T09:14:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.640+0000 7fd6f4c0f740 1 --2- 172.21.15.120:0/1575259924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff431d9580 0x55ff431d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.640+0000 7fd6f4c0f740 1 --2- 172.21.15.120:0/1575259924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff431d44f0 0x55ff431e52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.640+0000 7fd6f4c0f740 1 --2- 172.21.15.120:0/1575259924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ff431d3bb0 0x55ff431cff60 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.640+0000 7fd6f4c0f740 1 -- 172.21.15.120:0/1575259924 >> 172.21.15.120:0/1575259924 conn(0x55ff431cf220 msgr2=0x55ff431ddde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.640+0000 7fd6f4c0f740 1 -- 172.21.15.120:0/1575259924 shutdown_connections 2024-09-19T09:14:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.640+0000 7fd6f4c0f740 1 -- 172.21.15.120:0/1575259924 wait complete. 2024-09-19T09:14:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.640+0000 7fd6f4c0f740 1 -- start start 2024-09-19T09:14:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.640+0000 7fd6f4c0f740 1 Processor -- start 2024-09-19T09:14:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.641+0000 7fd6f4c0f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff431d3bb0 0x55ff431e2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.641+0000 7fd6f4c0f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ff431d44f0 0x55ff431e2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.641+0000 7fd6f4c0f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff431d9580 0x55ff431e3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.641+0000 7fd6f4c0f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ec1a8120 con 0x55ff431d44f0 2024-09-19T09:14:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.641+0000 7fd6f4c0f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ec1a7a50 con 0x55ff431d3bb0 2024-09-19T09:14:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.641+0000 7fd6f4c0f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ec1a4c70 con 0x55ff431d9580 2024-09-19T09:14:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.641+0000 7fd6f4c0f740 10 client.0 _ll_register_callbacks cb 0x7fd6e0009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.641+0000 7fd6eaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ff431d44f0 0x55ff431e2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.641+0000 7fd6eaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ff431d44f0 0x55ff431e2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38048/0 (socket says 172.21.15.120:38048) 2024-09-19T09:14:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.641+0000 7fd6eaffd640 1 -- 172.21.15.120:0/3943787244 learned_addr learned my addr 172.21.15.120:0/3943787244 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.641+0000 7fd6ebfff640 1 --2- 172.21.15.120:0/3943787244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff431d9580 0x55ff431e3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.641+0000 7fd6eaffd640 1 -- 172.21.15.120:0/3943787244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff431d9580 msgr2=0x55ff431e3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.641+0000 7fd6eaffd640 1 --2- 172.21.15.120:0/3943787244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff431d9580 0x55ff431e3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.641+0000 7fd6eaffd640 1 -- 172.21.15.120:0/3943787244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff431d3bb0 msgr2=0x55ff431e2a90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:14:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.641+0000 7fd6eaffd640 1 --2- 172.21.15.120:0/3943787244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff431d3bb0 0x55ff431e2a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.641+0000 7fd6eaffd640 1 -- 172.21.15.120:0/3943787244 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff431e45f0 con 0x55ff431d44f0 2024-09-19T09:14:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.641+0000 7fd6eaffd640 1 --2- 172.21.15.120:0/3943787244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ff431d44f0 0x55ff431e2fd0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0012740 tx=0x7fd6e000fe60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2248e9613a4936 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.641+0000 7fd6caffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:14:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.641+0000 7fd6caffd640 1 -- 172.21.15.120:0/3943787244 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e001ce60 con 0x55ff431d44f0 2024-09-19T09:14:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.641+0000 7fd6caffd640 1 -- 172.21.15.120:0/3943787244 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fd6e000f920 con 0x55ff431d44f0 2024-09-19T09:14:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.641+0000 7fd6f4c0f740 1 -- 172.21.15.120:0/3943787244 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6e4049040 con 0x55ff431d44f0 2024-09-19T09:14:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.642+0000 7fd6caffd640 1 -- 172.21.15.120:0/3943787244 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd6e0037050 con 0x55ff431d44f0 2024-09-19T09:14:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.642+0000 7fd6f4c0f740 10 client.19689 fetch_fsmap learned FSMap version 33 2024-09-19T09:14:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.642+0000 7fd6f4c0f740 1 -- 172.21.15.120:0/3943787244 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ff43254990 con 0x55ff431d44f0 2024-09-19T09:14:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.642+0000 7fd6caffd640 1 -- 172.21.15.120:0/3943787244 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6e4049040 con 0x55ff431d44f0 2024-09-19T09:14:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.642+0000 7fd6caffd640 1 -- 172.21.15.120:0/3943787244 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fd6e0003680 con 0x55ff431d44f0 2024-09-19T09:14:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.642+0000 7fd6f4c0f740 10 client.19689 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:14:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.642+0000 7fd6f4c0f740 10 client.19689 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:14:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.642+0000 7fd6f4c0f740 20 client.19689 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.642+0000 7fd6f4c0f740 1 --2- 172.21.15.120:0/3943787244 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ff432553a0 0x55ff432577c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.642+0000 7fd6f4c0f740 4 client.19689 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:14:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.642+0000 7fd6f4c0f740 1 -- 172.21.15.120:0/3943787244 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ff4313cd70 con 0x55ff432553a0 2024-09-19T09:14:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.642+0000 7fd6ebfff640 1 --2- 172.21.15.120:0/3943787244 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ff432553a0 0x55ff432577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.643+0000 7fd6ebfff640 1 --2- 172.21.15.120:0/3943787244 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ff432553a0 0x55ff432577c0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af4b1b0b9a8f74d2 server_cookie=e1bbb4db4dd211fc in_seq=0 out_seq=0 2024-09-19T09:14:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.643+0000 7fd6caffd640 10 client.19689 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.797+0000 7fd6caffd640 1 -- 172.21.15.120:0/3943787244 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ff4313cd70 con 0x55ff432553a0 2024-09-19T09:14:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.797+0000 7fd6caffd640 10 client.19689 handle_command_reply: tid=0 2024-09-19T09:14:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.797+0000 7fd6f4c0f740 1 client.19689 shutdown 2024-09-19T09:14:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.797+0000 7fd6f4c0f740 1 -- 172.21.15.120:0/3943787244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ff431d44f0 msgr2=0x55ff431e2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.797+0000 7fd6f4c0f740 1 --2- 172.21.15.120:0/3943787244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ff431d44f0 0x55ff431e2fd0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0012740 tx=0x7fd6e000fe60 comp rx=0 tx=0).stop 2024-09-19T09:14:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.797+0000 7fd6f4c0f740 1 -- 172.21.15.120:0/3943787244 shutdown_connections 2024-09-19T09:14:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.797+0000 7fd6f4c0f740 1 --2- 172.21.15.120:0/3943787244 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ff432553a0 0x55ff432577c0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.797+0000 7fd6f4c0f740 1 --2- 172.21.15.120:0/3943787244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff431d9580 0x55ff431e3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.797+0000 7fd6f4c0f740 1 --2- 172.21.15.120:0/3943787244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ff431d44f0 0x55ff431e2fd0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.797+0000 7fd6caffd640 0 client.19689 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:14:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.797+0000 7fd6f4c0f740 1 --2- 172.21.15.120:0/3943787244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ff431d3bb0 0x55ff431e2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.797+0000 7fd6f4c0f740 1 -- 172.21.15.120:0/3943787244 >> 172.21.15.120:0/3943787244 conn(0x55ff431cf220 msgr2=0x55ff431ddd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.797+0000 7fd6f4c0f740 1 -- 172.21.15.120:0/3943787244 shutdown_connections 2024-09-19T09:14:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.797+0000 7fd6f4c0f740 1 -- 172.21.15.120:0/3943787244 wait complete. 2024-09-19T09:14:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.797+0000 7fd6f4c0f740 20 client.19689 trim_cache size 0 max 16384 2024-09-19T09:14:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:14:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:14:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:14:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:14:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.798+0000 7fd6f1d98640 1 -- 172.21.15.120:0/2866312995 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd6d40a3370 msgr2=0x7fd6d40a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.798+0000 7fd6f1d98640 1 --2- 172.21.15.120:0/2866312995 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd6d40a3370 0x7fd6d40a5830 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc0a4070 tx=0x7fd6dc010ce0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.798+0000 7fd6f1d98640 1 -- 172.21.15.120:0/2866312995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6ec100f70 msgr2=0x7fd6ec196d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.798+0000 7fd6f1d98640 1 --2- 172.21.15.120:0/2866312995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6ec100f70 0x7fd6ec196d00 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00d950 tx=0x7fd6dc00de10 comp rx=0 tx=0).stop 2024-09-19T09:14:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.798+0000 7fd6f1d98640 1 -- 172.21.15.120:0/2866312995 shutdown_connections 2024-09-19T09:14:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.798+0000 7fd6f1d98640 1 --2- 172.21.15.120:0/2866312995 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd6d40a3370 0x7fd6d40a5830 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.798+0000 7fd6f1d98640 1 --2- 172.21.15.120:0/2866312995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6ec102640 0x7fd6ec19e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.798+0000 7fd6f1d98640 1 --2- 172.21.15.120:0/2866312995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6ec101c80 0x7fd6ec197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.798+0000 7fd6f1d98640 1 --2- 172.21.15.120:0/2866312995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6ec100f70 0x7fd6ec196d00 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:57.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.798+0000 7fd6f1d98640 1 -- 172.21.15.120:0/2866312995 >> 172.21.15.120:0/2866312995 conn(0x7fd6ec0fb930 msgr2=0x7fd6ec104f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:57.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.798+0000 7fd6f1d98640 1 -- 172.21.15.120:0/2866312995 shutdown_connections 2024-09-19T09:14:57.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:14:57.798+0000 7fd6f1d98640 1 -- 172.21.15.120:0/2866312995 wait complete. 2024-09-19T09:14:58.069 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:14:58.075 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1183216 9.73 MB/sec execute 533 sec latency 103.751 ms 2024-09-19T09:14:58.342 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:14:58.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.809+0000 7fa7ca05d640 1 -- 172.21.15.120:0/2983121133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7c41083f0 msgr2=0x7fa7c41087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:58.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.809+0000 7fa7ca05d640 1 --2- 172.21.15.120:0/2983121133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7c41083f0 0x7fa7c41087d0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc009ca0 tx=0x7fa7bc02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.809+0000 7fa7ca05d640 1 -- 172.21.15.120:0/2983121133 shutdown_connections 2024-09-19T09:14:58.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.809+0000 7fa7ca05d640 1 --2- 172.21.15.120:0/2983121133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7c410d520 0x7fa7c41150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.809+0000 7fa7ca05d640 1 --2- 172.21.15.120:0/2983121133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7c4108da0 0x7fa7c410cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.809+0000 7fa7ca05d640 1 --2- 172.21.15.120:0/2983121133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7c41083f0 0x7fa7c41087d0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.809+0000 7fa7ca05d640 1 -- 172.21.15.120:0/2983121133 >> 172.21.15.120:0/2983121133 conn(0x7fa7c4077af0 msgr2=0x7fa7c4079f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:58.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.809+0000 7fa7ca05d640 1 -- 172.21.15.120:0/2983121133 shutdown_connections 2024-09-19T09:14:58.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.809+0000 7fa7ca05d640 1 -- 172.21.15.120:0/2983121133 wait complete. 2024-09-19T09:14:58.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.810+0000 7fa7ca05d640 1 Processor -- start 2024-09-19T09:14:58.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.810+0000 7fa7ca05d640 1 -- start start 2024-09-19T09:14:58.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.810+0000 7fa7ca05d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7c41083f0 0x7fa7c41a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:58.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.810+0000 7fa7ca05d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7c4108da0 0x7fa7c41a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:58.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.810+0000 7fa7ca05d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7c410d520 0x7fa7c419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:58.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.811+0000 7fa7ca05d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c4118540 con 0x7fa7c4108da0 2024-09-19T09:14:58.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.811+0000 7fa7ca05d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41183c0 con 0x7fa7c410d520 2024-09-19T09:14:58.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.811+0000 7fa7ca05d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41186c0 con 0x7fa7c41083f0 2024-09-19T09:14:58.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.811+0000 7fa7c2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7c4108da0 0x7fa7c41a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:58.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.811+0000 7fa7c2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7c4108da0 0x7fa7c41a4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38066/0 (socket says 172.21.15.120:38066) 2024-09-19T09:14:58.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.811+0000 7fa7c2ffd640 1 -- 172.21.15.120:0/72650716 learned_addr learned my addr 172.21.15.120:0/72650716 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:58.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.811+0000 7fa7c3fff640 1 --2- 172.21.15.120:0/72650716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7c410d520 0x7fa7c419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:58.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.811+0000 7fa7c37fe640 1 --2- 172.21.15.120:0/72650716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7c41083f0 0x7fa7c41a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:58.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.811+0000 7fa7c2ffd640 1 -- 172.21.15.120:0/72650716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7c41083f0 msgr2=0x7fa7c41a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:58.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.811+0000 7fa7c2ffd640 1 --2- 172.21.15.120:0/72650716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7c41083f0 0x7fa7c41a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.811+0000 7fa7c2ffd640 1 -- 172.21.15.120:0/72650716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7c410d520 msgr2=0x7fa7c419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:58.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.811+0000 7fa7c2ffd640 1 --2- 172.21.15.120:0/72650716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7c410d520 0x7fa7c419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.811+0000 7fa7c2ffd640 1 -- 172.21.15.120:0/72650716 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c419efc0 con 0x7fa7c4108da0 2024-09-19T09:14:58.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.811+0000 7fa7c37fe640 1 --2- 172.21.15.120:0/72650716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7c41083f0 0x7fa7c41a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:14:58.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.811+0000 7fa7c2ffd640 1 --2- 172.21.15.120:0/72650716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7c4108da0 0x7fa7c41a4c10 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00d770 tx=0x7fa7ac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:58.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.812+0000 7fa7c0ff9640 1 -- 172.21.15.120:0/72650716 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac017070 con 0x7fa7c4108da0 2024-09-19T09:14:58.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.812+0000 7fa7ca05d640 1 -- 172.21.15.120:0/72650716 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c419f2b0 con 0x7fa7c4108da0 2024-09-19T09:14:58.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.812+0000 7fa7ca05d640 1 -- 172.21.15.120:0/72650716 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c41ab5a0 con 0x7fa7c4108da0 2024-09-19T09:14:58.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.812+0000 7fa7c0ff9640 1 -- 172.21.15.120:0/72650716 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7ac009c30 con 0x7fa7c4108da0 2024-09-19T09:14:58.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.812+0000 7fa7c0ff9640 1 -- 172.21.15.120:0/72650716 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac00f620 con 0x7fa7c4108da0 2024-09-19T09:14:58.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.815+0000 7fa7c0ff9640 1 -- 172.21.15.120:0/72650716 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa7ac01a070 con 0x7fa7c4108da0 2024-09-19T09:14:58.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.815+0000 7fa7cc436740 1 Processor -- start 2024-09-19T09:14:58.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.815+0000 7fa7cc436740 1 -- start start 2024-09-19T09:14:58.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.815+0000 7fa7cc436740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bfb353b40 0x562bfb352090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:58.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.815+0000 7fa7cc436740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562bfb3525d0 0x562bfb352a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:58.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.815+0000 7fa7cc436740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562bfb35d370 0x562bfb35f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:58.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.815+0000 7fa7cc436740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562bfb2b82f0 con 0x562bfb35d370 2024-09-19T09:14:58.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.815+0000 7fa7cc436740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562bfb25fcf0 con 0x562bfb3525d0 2024-09-19T09:14:58.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.815+0000 7fa7cc436740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562bfb2b42b0 con 0x562bfb353b40 2024-09-19T09:14:58.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.815+0000 7fa7c0ff9640 1 --2- 172.21.15.120:0/72650716 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa7b00a32c0 0x7fa7b00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:58.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.816+0000 7fa7c3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bfb353b40 0x562bfb352090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:58.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.816+0000 7fa7c3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bfb353b40 0x562bfb352090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:58198/0 (socket says 172.21.15.120:58198) 2024-09-19T09:14:58.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.816+0000 7fa7c3fff640 1 -- 172.21.15.120:0/1366015950 learned_addr learned my addr 172.21.15.120:0/1366015950 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:58.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.816+0000 7fa7c2ffd640 1 --2- 172.21.15.120:0/1366015950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562bfb35d370 0x562bfb35f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:58.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.816+0000 7fa7c3fff640 1 --2- 172.21.15.120:0/72650716 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa7b00a32c0 0x7fa7b00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:58.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.816+0000 7fa7c37fe640 1 --2- 172.21.15.120:0/1366015950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562bfb3525d0 0x562bfb352a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:58.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.816+0000 7fa7c0ff9640 1 -- 172.21.15.120:0/72650716 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fa7ac0cf760 con 0x7fa7c4108da0 2024-09-19T09:14:58.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.816+0000 7fa7c3fff640 1 -- 172.21.15.120:0/1366015950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562bfb3525d0 msgr2=0x562bfb352a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:58.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.816+0000 7fa7c3fff640 1 --2- 172.21.15.120:0/1366015950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562bfb3525d0 0x562bfb352a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.816+0000 7fa7c3fff640 1 -- 172.21.15.120:0/1366015950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562bfb35d370 msgr2=0x562bfb35f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:58.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.816+0000 7fa7c3fff640 1 --2- 172.21.15.120:0/1366015950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562bfb35d370 0x562bfb35f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.816+0000 7fa7c3fff640 1 -- 172.21.15.120:0/1366015950 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bfb2443a0 con 0x562bfb353b40 2024-09-19T09:14:58.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.816+0000 7fa7c3fff640 1 --2- 172.21.15.120:0/1366015950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bfb353b40 0x562bfb352090 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa7b40129b0 tx=0x7fa7b4012e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=500fc87038229918 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:58.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.816+0000 7fa7c3fff640 1 --2- 172.21.15.120:0/72650716 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa7b00a32c0 0x7fa7b00a5780 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4010740 tx=0x7fa7b4017000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:58.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.817+0000 7fa7b9ffb640 1 -- 172.21.15.120:0/1366015950 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4024070 con 0x562bfb353b40 2024-09-19T09:14:58.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.817+0000 7fa7b9ffb640 1 -- 172.21.15.120:0/1366015950 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7b4012510 con 0x562bfb353b40 2024-09-19T09:14:58.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.817+0000 7fa7b9ffb640 1 -- 172.21.15.120:0/1366015950 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b400ddf0 con 0x562bfb353b40 2024-09-19T09:14:58.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.818+0000 7fa7cc436740 1 -- 172.21.15.120:0/1366015950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bfb353b40 msgr2=0x562bfb352090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:58.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.818+0000 7fa7cc436740 1 --2- 172.21.15.120:0/1366015950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bfb353b40 0x562bfb352090 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa7b40129b0 tx=0x7fa7b4012e70 comp rx=0 tx=0).stop 2024-09-19T09:14:58.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.818+0000 7fa7cc436740 1 -- 172.21.15.120:0/1366015950 shutdown_connections 2024-09-19T09:14:58.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.818+0000 7fa7cc436740 1 --2- 172.21.15.120:0/1366015950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562bfb35d370 0x562bfb35f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.818+0000 7fa7cc436740 1 --2- 172.21.15.120:0/1366015950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562bfb3525d0 0x562bfb352a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.818+0000 7fa7cc436740 1 --2- 172.21.15.120:0/1366015950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bfb353b40 0x562bfb352090 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.818+0000 7fa7cc436740 1 -- 172.21.15.120:0/1366015950 >> 172.21.15.120:0/1366015950 conn(0x562bfb31b310 msgr2=0x562bfb34ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:58.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.818+0000 7fa7cc436740 1 -- 172.21.15.120:0/1366015950 shutdown_connections 2024-09-19T09:14:58.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.818+0000 7fa7cc436740 1 -- 172.21.15.120:0/1366015950 wait complete. 2024-09-19T09:14:58.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.818+0000 7fa7cc436740 1 -- start start 2024-09-19T09:14:58.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.818+0000 7fa7cc436740 1 Processor -- start 2024-09-19T09:14:58.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.819+0000 7fa7cc436740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562bfb3525d0 0x562bfb35a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:58.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.819+0000 7fa7cc436740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562bfb353b40 0x562bfb35a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:58.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.819+0000 7fa7cc436740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bfb35d370 0x562bfb35ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:58.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.819+0000 7fa7cc436740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562bfb360010 con 0x562bfb353b40 2024-09-19T09:14:58.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.819+0000 7fa7cc436740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562bfb273060 con 0x562bfb3525d0 2024-09-19T09:14:58.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.819+0000 7fa7cc436740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562bfb360190 con 0x562bfb35d370 2024-09-19T09:14:58.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.819+0000 7fa7cc436740 10 client.0 _ll_register_callbacks cb 0x562bfafc68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:58.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.819+0000 7fa7c37fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562bfb3525d0 0x562bfb35a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:58.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.819+0000 7fa7c3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bfb35d370 0x562bfb35ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:58.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.819+0000 7fa7c37fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562bfb3525d0 0x562bfb35a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:41282/0 (socket says 172.21.15.120:41282) 2024-09-19T09:14:58.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.819+0000 7fa7c37fe640 1 -- 172.21.15.120:0/3199047275 learned_addr learned my addr 172.21.15.120:0/3199047275 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:58.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.819+0000 7fa7c3fff640 1 --2- 172.21.15.120:0/3199047275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bfb35d370 0x562bfb35ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:58200/0 (socket says 172.21.15.120:58200) 2024-09-19T09:14:58.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.819+0000 7fa7c2ffd640 1 --2- 172.21.15.120:0/3199047275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562bfb353b40 0x562bfb35a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:58.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.819+0000 7fa7c3fff640 1 -- 172.21.15.120:0/3199047275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562bfb3525d0 msgr2=0x562bfb35a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:58.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.819+0000 7fa7c3fff640 1 --2- 172.21.15.120:0/3199047275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562bfb3525d0 0x562bfb35a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.819+0000 7fa7c3fff640 1 -- 172.21.15.120:0/3199047275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562bfb353b40 msgr2=0x562bfb35a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:58.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.819+0000 7fa7c3fff640 1 --2- 172.21.15.120:0/3199047275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562bfb353b40 0x562bfb35a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.819+0000 7fa7c3fff640 1 -- 172.21.15.120:0/3199047275 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bfb0293c0 con 0x562bfb35d370 2024-09-19T09:14:58.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.819+0000 7fa7c37fe640 1 --2- 172.21.15.120:0/3199047275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562bfb3525d0 0x562bfb35a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:58.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.820+0000 7fa7c2ffd640 1 --2- 172.21.15.120:0/3199047275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562bfb353b40 0x562bfb35a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:14:58.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.820+0000 7fa7c3fff640 1 --2- 172.21.15.120:0/3199047275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bfb35d370 0x562bfb35ab90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400d9d0 tx=0x7fa7b400b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39396a7cec177d21 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:58.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.820+0000 7fa7a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:14:58.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.820+0000 7fa7a2ffd640 1 -- 172.21.15.120:0/3199047275 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4024050 con 0x562bfb35d370 2024-09-19T09:14:58.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.820+0000 7fa7a2ffd640 1 -- 172.21.15.120:0/3199047275 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fa7b4005040 con 0x562bfb35d370 2024-09-19T09:14:58.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.820+0000 7fa7cc436740 1 -- 172.21.15.120:0/3199047275 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7b4012510 con 0x562bfb35d370 2024-09-19T09:14:58.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.820+0000 7fa7a2ffd640 1 -- 172.21.15.120:0/3199047275 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa7b4005240 con 0x562bfb35d370 2024-09-19T09:14:58.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.820+0000 7fa7cc436740 10 client.28555 fetch_fsmap learned FSMap version 33 2024-09-19T09:14:58.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.820+0000 7fa7cc436740 1 -- 172.21.15.120:0/3199047275 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562bfb36bc10 con 0x562bfb35d370 2024-09-19T09:14:58.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.821+0000 7fa7a2ffd640 1 -- 172.21.15.120:0/3199047275 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7b4012510 con 0x562bfb35d370 2024-09-19T09:14:58.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.821+0000 7fa7a2ffd640 1 -- 172.21.15.120:0/3199047275 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fa7b4020430 con 0x562bfb35d370 2024-09-19T09:14:58.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.821+0000 7fa7cc436740 10 client.28555 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:14:58.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.821+0000 7fa7cc436740 10 client.28555 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:14:58.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.821+0000 7fa7cc436740 20 client.28555 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:58.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.821+0000 7fa7cc436740 1 --2- 172.21.15.120:0/3199047275 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x562bfb36d0d0 0x562bfb36f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:58.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.821+0000 7fa7cc436740 4 client.28555 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:14:58.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.821+0000 7fa7cc436740 1 -- 172.21.15.120:0/3199047275 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bfb2c5860 con 0x562bfb36d0d0 2024-09-19T09:14:58.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.821+0000 7fa7c37fe640 1 --2- 172.21.15.120:0/3199047275 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x562bfb36d0d0 0x562bfb36f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:58.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.822+0000 7fa7c37fe640 1 --2- 172.21.15.120:0/3199047275 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x562bfb36d0d0 0x562bfb36f4c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=98f54057b6d409e1 server_cookie=6aa52fa3db22670b in_seq=0 out_seq=0 2024-09-19T09:14:58.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.822+0000 7fa7a2ffd640 10 client.28555 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:14:58.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.822+0000 7fa7a2ffd640 1 -- 172.21.15.120:0/3199047275 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562bfb2c5860 con 0x562bfb36d0d0 2024-09-19T09:14:58.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.822+0000 7fa7a2ffd640 10 client.28555 handle_command_reply: tid=0 2024-09-19T09:14:58.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.822+0000 7fa7cc436740 1 client.28555 shutdown 2024-09-19T09:14:58.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.823+0000 7fa7cc436740 1 -- 172.21.15.120:0/3199047275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bfb35d370 msgr2=0x562bfb35ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:58.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.823+0000 7fa7cc436740 1 --2- 172.21.15.120:0/3199047275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bfb35d370 0x562bfb35ab90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400d9d0 tx=0x7fa7b400b980 comp rx=0 tx=0).stop 2024-09-19T09:14:58.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.823+0000 7fa7cc436740 1 -- 172.21.15.120:0/3199047275 shutdown_connections 2024-09-19T09:14:58.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.823+0000 7fa7cc436740 1 --2- 172.21.15.120:0/3199047275 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x562bfb36d0d0 0x562bfb36f4c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.823+0000 7fa7a2ffd640 0 client.28555 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:14:58.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.823+0000 7fa7cc436740 1 --2- 172.21.15.120:0/3199047275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bfb35d370 0x562bfb35ab90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.823+0000 7fa7cc436740 1 --2- 172.21.15.120:0/3199047275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562bfb353b40 0x562bfb35a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.823+0000 7fa7cc436740 1 --2- 172.21.15.120:0/3199047275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562bfb3525d0 0x562bfb35a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.823+0000 7fa7cc436740 1 -- 172.21.15.120:0/3199047275 >> 172.21.15.120:0/3199047275 conn(0x562bfb31b310 msgr2=0x562bfb354960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:58.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.823+0000 7fa7cc436740 1 -- 172.21.15.120:0/3199047275 shutdown_connections 2024-09-19T09:14:58.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.824+0000 7fa7cc436740 1 -- 172.21.15.120:0/3199047275 wait complete. 2024-09-19T09:14:58.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.824+0000 7fa7cc436740 20 client.28555 trim_cache size 0 max 16384 2024-09-19T09:14:58.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.840+0000 7fa7cc436740 1 Processor -- start 2024-09-19T09:14:58.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.840+0000 7fa7cc436740 1 -- start start 2024-09-19T09:14:58.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.841+0000 7fa7cc436740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562bfb3525d0 0x562bfb35c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:58.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.841+0000 7fa7cc436740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562bfb353b40 0x562bfb355120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:58.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.841+0000 7fa7c2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562bfb353b40 0x562bfb355120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:58.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.841+0000 7fa7cc436740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bfb35d370 0x562bfb355660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:58.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.841+0000 7fa7c2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562bfb353b40 0x562bfb355120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38078/0 (socket says 172.21.15.120:38078) 2024-09-19T09:14:58.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.841+0000 7fa7c2ffd640 1 -- 172.21.15.120:0/3456242289 learned_addr learned my addr 172.21.15.120:0/3456242289 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:58.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.841+0000 7fa7cc436740 1 -- 172.21.15.120:0/3456242289 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41b3bb0 con 0x562bfb353b40 2024-09-19T09:14:58.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.841+0000 7fa7cc436740 1 -- 172.21.15.120:0/3456242289 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41b56f0 con 0x562bfb3525d0 2024-09-19T09:14:58.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.841+0000 7fa7c3fff640 1 --2- 172.21.15.120:0/3456242289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bfb35d370 0x562bfb355660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:58.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.841+0000 7fa7c37fe640 1 --2- 172.21.15.120:0/3456242289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562bfb3525d0 0x562bfb35c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:58.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.841+0000 7fa7cc436740 1 -- 172.21.15.120:0/3456242289 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41b4280 con 0x562bfb35d370 2024-09-19T09:14:58.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.842+0000 7fa7c2ffd640 1 -- 172.21.15.120:0/3456242289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bfb35d370 msgr2=0x562bfb355660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:58.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.842+0000 7fa7c2ffd640 1 --2- 172.21.15.120:0/3456242289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bfb35d370 0x562bfb355660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.842+0000 7fa7c2ffd640 1 -- 172.21.15.120:0/3456242289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562bfb3525d0 msgr2=0x562bfb35c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:58.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.842+0000 7fa7c2ffd640 1 --2- 172.21.15.120:0/3456242289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562bfb3525d0 0x562bfb35c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.842+0000 7fa7c2ffd640 1 -- 172.21.15.120:0/3456242289 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bfb34e4f0 con 0x562bfb353b40 2024-09-19T09:14:58.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.842+0000 7fa7c2ffd640 1 --2- 172.21.15.120:0/3456242289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562bfb353b40 0x562bfb355120 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac09ba80 tx=0x7fa7ac06ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d165f835b7646e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:58.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.843+0000 7fa7b97fa640 1 -- 172.21.15.120:0/3456242289 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac021040 con 0x562bfb353b40 2024-09-19T09:14:58.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.843+0000 7fa7b97fa640 1 -- 172.21.15.120:0/3456242289 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7ac0a49f0 con 0x562bfb353b40 2024-09-19T09:14:58.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.843+0000 7fa7b97fa640 1 -- 172.21.15.120:0/3456242289 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac0a4560 con 0x562bfb353b40 2024-09-19T09:14:58.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.844+0000 7fa7cc436740 1 -- 172.21.15.120:0/3456242289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562bfb353b40 msgr2=0x562bfb355120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:58.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.844+0000 7fa7cc436740 1 --2- 172.21.15.120:0/3456242289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562bfb353b40 0x562bfb355120 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac09ba80 tx=0x7fa7ac06ec00 comp rx=0 tx=0).stop 2024-09-19T09:14:58.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.844+0000 7fa7cc436740 1 -- 172.21.15.120:0/3456242289 shutdown_connections 2024-09-19T09:14:58.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.844+0000 7fa7cc436740 1 --2- 172.21.15.120:0/3456242289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bfb35d370 0x562bfb355660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.844+0000 7fa7cc436740 1 --2- 172.21.15.120:0/3456242289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562bfb353b40 0x562bfb355120 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.844+0000 7fa7cc436740 1 --2- 172.21.15.120:0/3456242289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562bfb3525d0 0x562bfb35c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.844+0000 7fa7cc436740 1 -- 172.21.15.120:0/3456242289 >> 172.21.15.120:0/3456242289 conn(0x562bfb31b310 msgr2=0x562bfb368c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:58.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.844+0000 7fa7cc436740 1 -- 172.21.15.120:0/3456242289 shutdown_connections 2024-09-19T09:14:58.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.844+0000 7fa7cc436740 1 -- 172.21.15.120:0/3456242289 wait complete. 2024-09-19T09:14:58.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.845+0000 7fa7cc436740 1 -- start start 2024-09-19T09:14:58.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.845+0000 7fa7cc436740 1 Processor -- start 2024-09-19T09:14:58.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.845+0000 7fa7cc436740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562bfb3525d0 0x562bfb36fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:58.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.845+0000 7fa7cc436740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562bfb353b40 0x562bfb36ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:58.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.845+0000 7fa7cc436740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bfb35d370 0x562bfb3d5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:58.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.846+0000 7fa7cc436740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41b5020 con 0x562bfb3525d0 2024-09-19T09:14:58.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.846+0000 7fa7cc436740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41b4950 con 0x562bfb353b40 2024-09-19T09:14:58.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.846+0000 7fa7cc436740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41b1b70 con 0x562bfb35d370 2024-09-19T09:14:58.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.846+0000 7fa7cc436740 10 client.0 _ll_register_callbacks cb 0x7fa7ac010bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:14:58.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.846+0000 7fa7c37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562bfb3525d0 0x562bfb36fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:58.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.846+0000 7fa7c37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562bfb3525d0 0x562bfb36fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38080/0 (socket says 172.21.15.120:38080) 2024-09-19T09:14:58.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.846+0000 7fa7c37fe640 1 -- 172.21.15.120:0/2307915482 learned_addr learned my addr 172.21.15.120:0/2307915482 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:14:58.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.846+0000 7fa7c3fff640 1 --2- 172.21.15.120:0/2307915482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bfb35d370 0x562bfb3d5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:58.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.846+0000 7fa7c37fe640 1 -- 172.21.15.120:0/2307915482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bfb35d370 msgr2=0x562bfb3d5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:58.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.846+0000 7fa7c37fe640 1 --2- 172.21.15.120:0/2307915482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bfb35d370 0x562bfb3d5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.846+0000 7fa7c37fe640 1 -- 172.21.15.120:0/2307915482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562bfb353b40 msgr2=0x562bfb36ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:14:58.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.846+0000 7fa7c37fe640 1 --2- 172.21.15.120:0/2307915482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562bfb353b40 0x562bfb36ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.846+0000 7fa7c37fe640 1 -- 172.21.15.120:0/2307915482 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bfb3d5f40 con 0x562bfb3525d0 2024-09-19T09:14:58.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.846+0000 7fa7c37fe640 1 --2- 172.21.15.120:0/2307915482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562bfb3525d0 0x562bfb36fa60 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc009980 tx=0x7fa7bc031450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0b0978150d3d0e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:14:58.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.846+0000 7fa7a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:14:58.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.846+0000 7fa7a2ffd640 1 -- 172.21.15.120:0/2307915482 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc02f610 con 0x562bfb3525d0 2024-09-19T09:14:58.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.846+0000 7fa7a2ffd640 1 -- 172.21.15.120:0/2307915482 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fa7bc02f7b0 con 0x562bfb3525d0 2024-09-19T09:14:58.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.846+0000 7fa7cc436740 1 -- 172.21.15.120:0/2307915482 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7ac0a49f0 con 0x562bfb3525d0 2024-09-19T09:14:58.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.846+0000 7fa7a2ffd640 1 -- 172.21.15.120:0/2307915482 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa7bc02f9b0 con 0x562bfb3525d0 2024-09-19T09:14:58.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.846+0000 7fa7cc436740 10 client.19707 fetch_fsmap learned FSMap version 33 2024-09-19T09:14:58.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.847+0000 7fa7cc436740 1 -- 172.21.15.120:0/2307915482 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562bfb3d71e0 con 0x562bfb3525d0 2024-09-19T09:14:58.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.847+0000 7fa7a2ffd640 1 -- 172.21.15.120:0/2307915482 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7bc04d070 con 0x562bfb3525d0 2024-09-19T09:14:58.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.847+0000 7fa7a2ffd640 1 -- 172.21.15.120:0/2307915482 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fa7bc051420 con 0x562bfb3525d0 2024-09-19T09:14:58.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.847+0000 7fa7cc436740 10 client.19707 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:14:58.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.847+0000 7fa7cc436740 10 client.19707 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:14:58.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.847+0000 7fa7cc436740 20 client.19707 populate_metadata read hostname 'smithi120' 2024-09-19T09:14:58.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.847+0000 7fa7cc436740 1 --2- 172.21.15.120:0/2307915482 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x562bfb36d0d0 0x562bfb3d9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:14:58.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.847+0000 7fa7cc436740 4 client.19707 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:14:58.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.847+0000 7fa7cc436740 1 -- 172.21.15.120:0/2307915482 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa7ac0a49f0 con 0x562bfb36d0d0 2024-09-19T09:14:58.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.847+0000 7fa7c2ffd640 1 --2- 172.21.15.120:0/2307915482 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x562bfb36d0d0 0x562bfb3d9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:14:58.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.848+0000 7fa7c2ffd640 1 --2- 172.21.15.120:0/2307915482 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x562bfb36d0d0 0x562bfb3d9930 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ae7732b36d694289 server_cookie=b2a3ae076560e919 in_seq=0 out_seq=0 2024-09-19T09:14:58.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.848+0000 7fa7a2ffd640 10 client.19707 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:14:58.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.848+0000 7fa7a2ffd640 1 -- 172.21.15.120:0/2307915482 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa7ac0a49f0 con 0x562bfb36d0d0 2024-09-19T09:14:58.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.848+0000 7fa7a2ffd640 10 client.19707 handle_command_reply: tid=0 2024-09-19T09:14:58.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.848+0000 7fa7cc436740 1 client.19707 shutdown 2024-09-19T09:14:58.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.848+0000 7fa7cc436740 1 -- 172.21.15.120:0/2307915482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562bfb3525d0 msgr2=0x562bfb36fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:58.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.848+0000 7fa7cc436740 1 --2- 172.21.15.120:0/2307915482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562bfb3525d0 0x562bfb36fa60 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc009980 tx=0x7fa7bc031450 comp rx=0 tx=0).stop 2024-09-19T09:14:58.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.848+0000 7fa7cc436740 1 -- 172.21.15.120:0/2307915482 shutdown_connections 2024-09-19T09:14:58.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.848+0000 7fa7cc436740 1 --2- 172.21.15.120:0/2307915482 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x562bfb36d0d0 0x562bfb3d9930 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.848+0000 7fa7cc436740 1 --2- 172.21.15.120:0/2307915482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bfb35d370 0x562bfb3d5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.848+0000 7fa7cc436740 1 --2- 172.21.15.120:0/2307915482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562bfb353b40 0x562bfb36ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.848+0000 7fa7cc436740 1 --2- 172.21.15.120:0/2307915482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562bfb3525d0 0x562bfb36fa60 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.848+0000 7fa7a2ffd640 0 client.19707 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:14:58.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.848+0000 7fa7cc436740 1 -- 172.21.15.120:0/2307915482 >> 172.21.15.120:0/2307915482 conn(0x562bfb31b310 msgr2=0x562bfb3588d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:58.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.848+0000 7fa7cc436740 1 -- 172.21.15.120:0/2307915482 shutdown_connections 2024-09-19T09:14:58.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.848+0000 7fa7cc436740 1 -- 172.21.15.120:0/2307915482 wait complete. 2024-09-19T09:14:58.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.849+0000 7fa7cc436740 20 client.19707 trim_cache size 0 max 16384 2024-09-19T09:14:58.856 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:14:58.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.849+0000 7fa7ca05d640 1 -- 172.21.15.120:0/72650716 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa7b00a32c0 msgr2=0x7fa7b00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:58.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.849+0000 7fa7ca05d640 1 --2- 172.21.15.120:0/72650716 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa7b00a32c0 0x7fa7b00a5780 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4010740 tx=0x7fa7b4017000 comp rx=0 tx=0).stop 2024-09-19T09:14:58.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.849+0000 7fa7ca05d640 1 -- 172.21.15.120:0/72650716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7c4108da0 msgr2=0x7fa7c41a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:14:58.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.849+0000 7fa7ca05d640 1 --2- 172.21.15.120:0/72650716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7c4108da0 0x7fa7c41a4c10 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00d770 tx=0x7fa7ac00dc30 comp rx=0 tx=0).stop 2024-09-19T09:14:58.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.849+0000 7fa7ca05d640 1 -- 172.21.15.120:0/72650716 shutdown_connections 2024-09-19T09:14:58.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.849+0000 7fa7ca05d640 1 --2- 172.21.15.120:0/72650716 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa7b00a32c0 0x7fa7b00a5780 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.849+0000 7fa7ca05d640 1 --2- 172.21.15.120:0/72650716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7c410d520 0x7fa7c419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.849+0000 7fa7ca05d640 1 --2- 172.21.15.120:0/72650716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7c4108da0 0x7fa7c41a4c10 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.849+0000 7fa7ca05d640 1 --2- 172.21.15.120:0/72650716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7c41083f0 0x7fa7c41a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:14:58.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.849+0000 7fa7ca05d640 1 -- 172.21.15.120:0/72650716 >> 172.21.15.120:0/72650716 conn(0x7fa7c4077af0 msgr2=0x7fa7c4078e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:14:58.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.850+0000 7fa7ca05d640 1 -- 172.21.15.120:0/72650716 shutdown_connections 2024-09-19T09:14:58.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:14:58.850+0000 7fa7ca05d640 1 -- 172.21.15.120:0/72650716 wait complete. 2024-09-19T09:14:59.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:14:58 smithi120 ceph-mon[31150]: pgmap v707: 97 pgs: 97 active+clean; 53 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.1 MiB/s wr, 449 op/s 2024-09-19T09:14:59.075 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1185663 9.74 MB/sec execute 534 sec latency 39.521 ms 2024-09-19T09:14:59.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:14:58 smithi196 ceph-mon[35150]: pgmap v707: 97 pgs: 97 active+clean; 53 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.1 MiB/s wr, 449 op/s 2024-09-19T09:14:59.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:14:58 smithi153 ceph-mon[35071]: pgmap v707: 97 pgs: 97 active+clean; 53 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.1 MiB/s wr, 449 op/s 2024-09-19T09:14:59.714 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:15:00.019 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:15:00.076 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1187978 9.74 MB/sec execute 535 sec latency 11.553 ms 2024-09-19T09:15:00.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.537+0000 7f72221a8640 1 -- 172.21.15.120:0/4216039119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f721c101df0 msgr2=0x7f721c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:00.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.537+0000 7f72221a8640 1 --2- 172.21.15.120:0/4216039119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f721c101df0 0x7f721c1021d0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f720c009900 tx=0x7f720c02a410 comp rx=0 tx=0).stop 2024-09-19T09:15:00.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.537+0000 7f72221a8640 1 -- 172.21.15.120:0/4216039119 shutdown_connections 2024-09-19T09:15:00.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.537+0000 7f72221a8640 1 --2- 172.21.15.120:0/4216039119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f721c106f20 0x7f721c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.537+0000 7f72221a8640 1 --2- 172.21.15.120:0/4216039119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f721c1027a0 0x7f721c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.537+0000 7f72221a8640 1 --2- 172.21.15.120:0/4216039119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f721c101df0 0x7f721c1021d0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.537+0000 7f72221a8640 1 -- 172.21.15.120:0/4216039119 >> 172.21.15.120:0/4216039119 conn(0x7f721c0fd8f0 msgr2=0x7f721c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:00.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.537+0000 7f72221a8640 1 -- 172.21.15.120:0/4216039119 shutdown_connections 2024-09-19T09:15:00.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.537+0000 7f72221a8640 1 -- 172.21.15.120:0/4216039119 wait complete. 2024-09-19T09:15:00.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.538+0000 7f72221a8640 1 Processor -- start 2024-09-19T09:15:00.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.538+0000 7f72221a8640 1 -- start start 2024-09-19T09:15:00.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.538+0000 7f72221a8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f721c101df0 0x7f721c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:00.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.538+0000 7f72221a8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f721c1027a0 0x7f721c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:00.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.538+0000 7f72221a8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f721c106f20 0x7f721c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:00.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.538+0000 7f72221a8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f721c111f80 con 0x7f721c1027a0 2024-09-19T09:15:00.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.538+0000 7f72221a8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f721c111e00 con 0x7f721c106f20 2024-09-19T09:15:00.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.538+0000 7f72221a8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f721c112100 con 0x7f721c101df0 2024-09-19T09:15:00.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.538+0000 7f7213fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f721c1027a0 0x7f721c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:00.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.538+0000 7f721b7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f721c101df0 0x7f721c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:00.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.538+0000 7f7213fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f721c1027a0 0x7f721c1988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38096/0 (socket says 172.21.15.120:38096) 2024-09-19T09:15:00.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.538+0000 7f7213fff640 1 -- 172.21.15.120:0/1677451936 learned_addr learned my addr 172.21.15.120:0/1677451936 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:00.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.538+0000 7f721bfff640 1 --2- 172.21.15.120:0/1677451936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f721c106f20 0x7f721c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:00.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.538+0000 7f7213fff640 1 -- 172.21.15.120:0/1677451936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f721c101df0 msgr2=0x7f721c1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:00.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.538+0000 7f7213fff640 1 --2- 172.21.15.120:0/1677451936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f721c101df0 0x7f721c1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.538+0000 7f7213fff640 1 -- 172.21.15.120:0/1677451936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f721c106f20 msgr2=0x7f721c19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:00.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.538+0000 7f7213fff640 1 --2- 172.21.15.120:0/1677451936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f721c106f20 0x7f721c19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.538+0000 7f7213fff640 1 -- 172.21.15.120:0/1677451936 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721c19d360 con 0x7f721c1027a0 2024-09-19T09:15:00.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.539+0000 7f7213fff640 1 --2- 172.21.15.120:0/1677451936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f721c1027a0 0x7f721c1988f0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f720400e9f0 tx=0x7f720400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:00.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.539+0000 7f72197fa640 1 -- 172.21.15.120:0/1677451936 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720400cd80 con 0x7f721c1027a0 2024-09-19T09:15:00.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.539+0000 7f72197fa640 1 -- 172.21.15.120:0/1677451936 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7204010910 con 0x7f721c1027a0 2024-09-19T09:15:00.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.539+0000 7f72221a8640 1 -- 172.21.15.120:0/1677451936 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f721c19d650 con 0x7f721c1027a0 2024-09-19T09:15:00.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.539+0000 7f72197fa640 1 -- 172.21.15.120:0/1677451936 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7204018990 con 0x7f721c1027a0 2024-09-19T09:15:00.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.539+0000 7f72221a8640 1 -- 172.21.15.120:0/1677451936 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f721c1a4f30 con 0x7f721c1027a0 2024-09-19T09:15:00.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.541+0000 7f72197fa640 1 -- 172.21.15.120:0/1677451936 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f7204010ac0 con 0x7f721c1027a0 2024-09-19T09:15:00.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.541+0000 7f7224581740 1 Processor -- start 2024-09-19T09:15:00.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.541+0000 7f7224581740 1 -- start start 2024-09-19T09:15:00.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.542+0000 7f7224581740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614f0110b40 0x5614f010f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:00.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.542+0000 7f72197fa640 1 --2- 172.21.15.120:0/1677451936 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f72080a3240 0x7f72080a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:00.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.542+0000 7f7224581740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f010f5d0 0x5614f010fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:00.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.542+0000 7f7224581740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614f011a370 0x5614f011c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:00.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.542+0000 7f7224581740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5614f00752f0 con 0x5614f011a370 2024-09-19T09:15:00.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.542+0000 7f7224581740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5614f001ccf0 con 0x5614f0110b40 2024-09-19T09:15:00.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.542+0000 7f7224581740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5614f00712b0 con 0x5614f010f5d0 2024-09-19T09:15:00.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.542+0000 7f721bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614f0110b40 0x5614f010f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:00.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.542+0000 7f721bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614f0110b40 0x5614f010f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:41332/0 (socket says 172.21.15.120:41332) 2024-09-19T09:15:00.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.542+0000 7f721bfff640 1 -- 172.21.15.120:0/2391482937 learned_addr learned my addr 172.21.15.120:0/2391482937 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:00.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.542+0000 7f721b7fe640 1 --2- 172.21.15.120:0/1677451936 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f72080a3240 0x7f72080a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:00.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.542+0000 7f72197fa640 1 -- 172.21.15.120:0/1677451936 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f72040cfc20 con 0x7f721c1027a0 2024-09-19T09:15:00.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.542+0000 7f721bfff640 1 --2- 172.21.15.120:0/2391482937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614f011a370 0x5614f011c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:00.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.542+0000 7f7213fff640 1 --2- 172.21.15.120:0/2391482937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f010f5d0 0x5614f010fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:00.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.542+0000 7f721bfff640 1 -- 172.21.15.120:0/2391482937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f010f5d0 msgr2=0x5614f010fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:00.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.542+0000 7f721bfff640 1 --2- 172.21.15.120:0/2391482937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f010f5d0 0x5614f010fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.542+0000 7f721bfff640 1 -- 172.21.15.120:0/2391482937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614f011a370 msgr2=0x5614f011c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:00.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.542+0000 7f721bfff640 1 --2- 172.21.15.120:0/2391482937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614f011a370 0x5614f011c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.542+0000 7f721bfff640 1 -- 172.21.15.120:0/2391482937 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614f00013a0 con 0x5614f0110b40 2024-09-19T09:15:00.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.542+0000 7f721b7fe640 1 --2- 172.21.15.120:0/1677451936 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f72080a3240 0x7f72080a5700 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f720c009230 tx=0x7f720c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:00.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.542+0000 7f721bfff640 1 --2- 172.21.15.120:0/2391482937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614f0110b40 0x5614f010f090 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7214007310 tx=0x7f72140079b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dca0d49513496684 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:00.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.542+0000 7f7211ffb640 1 -- 172.21.15.120:0/2391482937 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7214014e20 con 0x5614f0110b40 2024-09-19T09:15:00.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.543+0000 7f7211ffb640 1 -- 172.21.15.120:0/2391482937 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72140140d0 con 0x5614f0110b40 2024-09-19T09:15:00.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.543+0000 7f7211ffb640 1 -- 172.21.15.120:0/2391482937 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72140056d0 con 0x5614f0110b40 2024-09-19T09:15:00.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.543+0000 7f7224581740 1 -- 172.21.15.120:0/2391482937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614f0110b40 msgr2=0x5614f010f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:00.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.543+0000 7f7224581740 1 --2- 172.21.15.120:0/2391482937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614f0110b40 0x5614f010f090 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7214007310 tx=0x7f72140079b0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.543+0000 7f7224581740 1 -- 172.21.15.120:0/2391482937 shutdown_connections 2024-09-19T09:15:00.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.543+0000 7f7224581740 1 --2- 172.21.15.120:0/2391482937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614f011a370 0x5614f011c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.543+0000 7f7224581740 1 --2- 172.21.15.120:0/2391482937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f010f5d0 0x5614f010fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.543+0000 7f7224581740 1 --2- 172.21.15.120:0/2391482937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614f0110b40 0x5614f010f090 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.543+0000 7f7224581740 1 -- 172.21.15.120:0/2391482937 >> 172.21.15.120:0/2391482937 conn(0x5614f00d8310 msgr2=0x5614f010bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:00.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.543+0000 7f7224581740 1 -- 172.21.15.120:0/2391482937 shutdown_connections 2024-09-19T09:15:00.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.543+0000 7f7224581740 1 -- 172.21.15.120:0/2391482937 wait complete. 2024-09-19T09:15:00.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.544+0000 7f7224581740 1 -- start start 2024-09-19T09:15:00.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.544+0000 7f7224581740 1 Processor -- start 2024-09-19T09:15:00.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.544+0000 7f7224581740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614f010f5d0 0x5614f0119d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:00.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.544+0000 7f7224581740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614f0110b40 0x5614f0116fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:00.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.544+0000 7f7224581740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f011a370 0x5614f0117570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:00.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.544+0000 7f7224581740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5614f011d010 con 0x5614f010f5d0 2024-09-19T09:15:00.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.544+0000 7f7224581740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5614f0030060 con 0x5614f0110b40 2024-09-19T09:15:00.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.544+0000 7f7224581740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5614f011d190 con 0x5614f011a370 2024-09-19T09:15:00.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.544+0000 7f7224581740 10 client.0 _ll_register_callbacks cb 0x5614efd838c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:00.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.544+0000 7f721b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614f010f5d0 0x5614f0119d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:00.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.544+0000 7f721b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614f010f5d0 0x5614f0119d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38118/0 (socket says 172.21.15.120:38118) 2024-09-19T09:15:00.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.544+0000 7f721b7fe640 1 -- 172.21.15.120:0/1293166155 learned_addr learned my addr 172.21.15.120:0/1293166155 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:00.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.544+0000 7f7213fff640 1 --2- 172.21.15.120:0/1293166155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614f0110b40 0x5614f0116fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:00.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.544+0000 7f721bfff640 1 --2- 172.21.15.120:0/1293166155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f011a370 0x5614f0117570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:00.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.545+0000 7f7213fff640 1 -- 172.21.15.120:0/1293166155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f011a370 msgr2=0x5614f0117570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:00.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.545+0000 7f7213fff640 1 --2- 172.21.15.120:0/1293166155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f011a370 0x5614f0117570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.545+0000 7f7213fff640 1 -- 172.21.15.120:0/1293166155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614f010f5d0 msgr2=0x5614f0119d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:00.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.545+0000 7f7213fff640 1 --2- 172.21.15.120:0/1293166155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614f010f5d0 0x5614f0119d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.545+0000 7f7213fff640 1 -- 172.21.15.120:0/1293166155 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614efde63c0 con 0x5614f0110b40 2024-09-19T09:15:00.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.545+0000 7f721b7fe640 1 --2- 172.21.15.120:0/1293166155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614f010f5d0 0x5614f0119d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:15:00.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.545+0000 7f7213fff640 1 --2- 172.21.15.120:0/1293166155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614f0110b40 0x5614f0116fd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f720409bff0 tx=0x7f720409cab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96e54fcdf621e1df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:00.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.545+0000 7f71faffd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:15:00.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.545+0000 7f71faffd640 1 -- 172.21.15.120:0/1293166155 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7204096e60 con 0x5614f0110b40 2024-09-19T09:15:00.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.545+0000 7f71faffd640 1 -- 172.21.15.120:0/1293166155 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f72040b5190 con 0x5614f0110b40 2024-09-19T09:15:00.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.545+0000 7f7224581740 1 -- 172.21.15.120:0/1293166155 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72140140d0 con 0x5614f0110b40 2024-09-19T09:15:00.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.545+0000 7f71faffd640 1 -- 172.21.15.120:0/1293166155 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f720409c790 con 0x5614f0110b40 2024-09-19T09:15:00.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.545+0000 7f7224581740 10 client.28766 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:00.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.545+0000 7f7224581740 1 -- 172.21.15.120:0/1293166155 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5614f0128ac0 con 0x5614f0110b40 2024-09-19T09:15:00.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.545+0000 7f71faffd640 1 -- 172.21.15.120:0/1293166155 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72140140d0 con 0x5614f0110b40 2024-09-19T09:15:00.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.546+0000 7f71faffd640 1 -- 172.21.15.120:0/1293166155 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f72040a0170 con 0x5614f0110b40 2024-09-19T09:15:00.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.546+0000 7f7224581740 10 client.28766 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:00.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.546+0000 7f7224581740 10 client.28766 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:15:00.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.546+0000 7f7224581740 20 client.28766 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:00.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.546+0000 7f7224581740 1 --2- 172.21.15.120:0/1293166155 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5614f0129e60 0x5614f012c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:00.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.546+0000 7f7224581740 4 client.28766 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:15:00.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.546+0000 7f7224581740 1 -- 172.21.15.120:0/1293166155 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614f0082860 con 0x5614f0129e60 2024-09-19T09:15:00.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.546+0000 7f721b7fe640 1 --2- 172.21.15.120:0/1293166155 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5614f0129e60 0x5614f012c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:00.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.547+0000 7f71faffd640 10 client.28766 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:15:00.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.547+0000 7f721b7fe640 1 --2- 172.21.15.120:0/1293166155 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5614f0129e60 0x5614f012c290 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fd435ac62cefa6cd server_cookie=35d9a79971aeb968 in_seq=0 out_seq=0 2024-09-19T09:15:00.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.548+0000 7f71faffd640 1 -- 172.21.15.120:0/1293166155 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5614f0082860 con 0x5614f0129e60 2024-09-19T09:15:00.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.548+0000 7f71faffd640 10 client.28766 handle_command_reply: tid=0 2024-09-19T09:15:00.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.548+0000 7f7224581740 1 client.28766 shutdown 2024-09-19T09:15:00.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.548+0000 7f7224581740 1 -- 172.21.15.120:0/1293166155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614f0110b40 msgr2=0x5614f0116fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:00.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.548+0000 7f7224581740 1 --2- 172.21.15.120:0/1293166155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614f0110b40 0x5614f0116fd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f720409bff0 tx=0x7f720409cab0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.548+0000 7f7224581740 1 -- 172.21.15.120:0/1293166155 shutdown_connections 2024-09-19T09:15:00.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.548+0000 7f7224581740 1 --2- 172.21.15.120:0/1293166155 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5614f0129e60 0x5614f012c290 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.548+0000 7f7224581740 1 --2- 172.21.15.120:0/1293166155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f011a370 0x5614f0117570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.548+0000 7f7224581740 1 --2- 172.21.15.120:0/1293166155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614f0110b40 0x5614f0116fd0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.548+0000 7f71faffd640 0 client.28766 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:15:00.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.548+0000 7f7224581740 1 --2- 172.21.15.120:0/1293166155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614f010f5d0 0x5614f0119d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.548+0000 7f7224581740 1 -- 172.21.15.120:0/1293166155 >> 172.21.15.120:0/1293166155 conn(0x5614f00d8310 msgr2=0x5614f0111960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:00.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.548+0000 7f7224581740 1 -- 172.21.15.120:0/1293166155 shutdown_connections 2024-09-19T09:15:00.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.548+0000 7f7224581740 1 -- 172.21.15.120:0/1293166155 wait complete. 2024-09-19T09:15:00.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.548+0000 7f7224581740 20 client.28766 trim_cache size 0 max 16384 2024-09-19T09:15:00.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.562+0000 7f7224581740 1 Processor -- start 2024-09-19T09:15:00.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.562+0000 7f7224581740 1 -- start start 2024-09-19T09:15:00.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.562+0000 7f7224581740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614f010f5d0 0x5614f0116890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:00.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.562+0000 7f7224581740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614f0110b40 0x5614f0118f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:00.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.562+0000 7f7224581740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f011a370 0x5614f0112120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:00.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.562+0000 7f7224581740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f721c1ad560 con 0x5614f010f5d0 2024-09-19T09:15:00.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.562+0000 7f7224581740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f721c1af0a0 con 0x5614f0110b40 2024-09-19T09:15:00.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.562+0000 7f7224581740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f721c1adc30 con 0x5614f011a370 2024-09-19T09:15:00.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.562+0000 7f721b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614f010f5d0 0x5614f0116890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:00.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.562+0000 7f721b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614f010f5d0 0x5614f0116890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38122/0 (socket says 172.21.15.120:38122) 2024-09-19T09:15:00.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.562+0000 7f721b7fe640 1 -- 172.21.15.120:0/3627693955 learned_addr learned my addr 172.21.15.120:0/3627693955 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:00.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.562+0000 7f7213fff640 1 --2- 172.21.15.120:0/3627693955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614f0110b40 0x5614f0118f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:00.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.562+0000 7f721bfff640 1 --2- 172.21.15.120:0/3627693955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f011a370 0x5614f0112120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:00.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.562+0000 7f721b7fe640 1 -- 172.21.15.120:0/3627693955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f011a370 msgr2=0x5614f0112120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:00.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.562+0000 7f721b7fe640 1 --2- 172.21.15.120:0/3627693955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f011a370 0x5614f0112120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.562+0000 7f721b7fe640 1 -- 172.21.15.120:0/3627693955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614f0110b40 msgr2=0x5614f0118f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:00.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.562+0000 7f721b7fe640 1 --2- 172.21.15.120:0/3627693955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614f0110b40 0x5614f0118f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.563+0000 7f721b7fe640 1 -- 172.21.15.120:0/3627693955 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614f010c960 con 0x5614f010f5d0 2024-09-19T09:15:00.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.563+0000 7f721bfff640 1 --2- 172.21.15.120:0/3627693955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f011a370 0x5614f0112120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:15:00.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.563+0000 7f721b7fe640 1 --2- 172.21.15.120:0/3627693955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614f010f5d0 0x5614f0116890 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f720c004770 tx=0x7f720c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36fb1a9463680de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:00.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.563+0000 7f72117fa640 1 -- 172.21.15.120:0/3627693955 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720c035040 con 0x5614f010f5d0 2024-09-19T09:15:00.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.563+0000 7f72117fa640 1 -- 172.21.15.120:0/3627693955 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f720c004340 con 0x5614f010f5d0 2024-09-19T09:15:00.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.563+0000 7f72117fa640 1 -- 172.21.15.120:0/3627693955 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720c03c440 con 0x5614f010f5d0 2024-09-19T09:15:00.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.564+0000 7f7224581740 1 -- 172.21.15.120:0/3627693955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614f010f5d0 msgr2=0x5614f0116890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:00.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.564+0000 7f7224581740 1 --2- 172.21.15.120:0/3627693955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614f010f5d0 0x5614f0116890 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f720c004770 tx=0x7f720c040000 comp rx=0 tx=0).stop 2024-09-19T09:15:00.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.564+0000 7f7224581740 1 -- 172.21.15.120:0/3627693955 shutdown_connections 2024-09-19T09:15:00.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.564+0000 7f7224581740 1 --2- 172.21.15.120:0/3627693955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f011a370 0x5614f0112120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.564+0000 7f7224581740 1 --2- 172.21.15.120:0/3627693955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614f0110b40 0x5614f0118f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.564+0000 7f7224581740 1 --2- 172.21.15.120:0/3627693955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614f010f5d0 0x5614f0116890 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.564+0000 7f7224581740 1 -- 172.21.15.120:0/3627693955 >> 172.21.15.120:0/3627693955 conn(0x5614f00d8310 msgr2=0x5614f0118830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:00.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.564+0000 7f7224581740 1 -- 172.21.15.120:0/3627693955 shutdown_connections 2024-09-19T09:15:00.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.564+0000 7f7224581740 1 -- 172.21.15.120:0/3627693955 wait complete. 2024-09-19T09:15:00.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.564+0000 7f7224581740 1 -- start start 2024-09-19T09:15:00.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.565+0000 7f7224581740 1 Processor -- start 2024-09-19T09:15:00.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.565+0000 7f7224581740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614f010f5d0 0x5614f0122d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:00.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.565+0000 7f7224581740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f0110b40 0x5614f011e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:00.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.565+0000 7f7224581740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614f011a370 0x5614f011e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:00.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.565+0000 7f7224581740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f721c1ae9d0 con 0x5614f010f5d0 2024-09-19T09:15:00.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.565+0000 7f7224581740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f721c1ae300 con 0x5614f011a370 2024-09-19T09:15:00.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.565+0000 7f7224581740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f721c1ab520 con 0x5614f0110b40 2024-09-19T09:15:00.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.565+0000 7f721b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614f010f5d0 0x5614f0122d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:00.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.565+0000 7f721b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614f010f5d0 0x5614f0122d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38124/0 (socket says 172.21.15.120:38124) 2024-09-19T09:15:00.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.565+0000 7f721b7fe640 1 -- 172.21.15.120:0/2834760506 learned_addr learned my addr 172.21.15.120:0/2834760506 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:00.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.565+0000 7f721bfff640 1 --2- 172.21.15.120:0/2834760506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614f011a370 0x5614f011e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:00.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.565+0000 7f7213fff640 1 --2- 172.21.15.120:0/2834760506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f0110b40 0x5614f011e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:00.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.565+0000 7f7224581740 10 client.0 _ll_register_callbacks cb 0x7f720400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:00.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.565+0000 7f721bfff640 1 -- 172.21.15.120:0/2834760506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f0110b40 msgr2=0x5614f011e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:00.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.565+0000 7f721bfff640 1 --2- 172.21.15.120:0/2834760506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f0110b40 0x5614f011e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.565+0000 7f721bfff640 1 -- 172.21.15.120:0/2834760506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614f010f5d0 msgr2=0x5614f0122d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:00.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.565+0000 7f721bfff640 1 --2- 172.21.15.120:0/2834760506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614f010f5d0 0x5614f0122d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.565+0000 7f721bfff640 1 -- 172.21.15.120:0/2834760506 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614f010b4f0 con 0x5614f011a370 2024-09-19T09:15:00.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.566+0000 7f721bfff640 1 --2- 172.21.15.120:0/2834760506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614f011a370 0x5614f011e7f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7214005430 tx=0x7f72140145f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea1d7bc242ddce5b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:00.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.566+0000 7f71faffd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:15:00.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.566+0000 7f71faffd640 1 -- 172.21.15.120:0/2834760506 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7214005660 con 0x5614f011a370 2024-09-19T09:15:00.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.566+0000 7f71faffd640 1 -- 172.21.15.120:0/2834760506 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f7214005800 con 0x5614f011a370 2024-09-19T09:15:00.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.566+0000 7f7224581740 1 -- 172.21.15.120:0/2834760506 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f720c004340 con 0x5614f011a370 2024-09-19T09:15:00.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.566+0000 7f71faffd640 1 -- 172.21.15.120:0/2834760506 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7214004a00 con 0x5614f011a370 2024-09-19T09:15:00.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.566+0000 7f7224581740 10 client.28772 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:00.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.566+0000 7f7224581740 1 -- 172.21.15.120:0/2834760506 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5614f0194a40 con 0x5614f011a370 2024-09-19T09:15:00.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.566+0000 7f71faffd640 1 -- 172.21.15.120:0/2834760506 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7214005080 con 0x5614f011a370 2024-09-19T09:15:00.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.567+0000 7f71faffd640 1 -- 172.21.15.120:0/2834760506 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f7214029020 con 0x5614f011a370 2024-09-19T09:15:00.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.567+0000 7f7224581740 10 client.28772 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:00.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.567+0000 7f7224581740 10 client.28772 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:15:00.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.567+0000 7f7224581740 20 client.28772 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:00.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.567+0000 7f7224581740 1 --2- 172.21.15.120:0/2834760506 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5614f0195000 0x5614f0197420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:00.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.567+0000 7f7224581740 4 client.28772 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:15:00.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.567+0000 7f7224581740 1 -- 172.21.15.120:0/2834760506 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f720c004340 con 0x5614f0195000 2024-09-19T09:15:00.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.567+0000 7f721b7fe640 1 --2- 172.21.15.120:0/2834760506 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5614f0195000 0x5614f0197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:00.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.567+0000 7f721b7fe640 1 --2- 172.21.15.120:0/2834760506 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5614f0195000 0x5614f0197420 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3af42ad1fc7e9679 server_cookie=a3ce38f9e0c8222 in_seq=0 out_seq=0 2024-09-19T09:15:00.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.567+0000 7f71faffd640 10 client.28772 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:15:00.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.568+0000 7f71faffd640 1 -- 172.21.15.120:0/2834760506 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f720c004340 con 0x5614f0195000 2024-09-19T09:15:00.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.568+0000 7f71faffd640 10 client.28772 handle_command_reply: tid=0 2024-09-19T09:15:00.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.568+0000 7f7224581740 1 client.28772 shutdown 2024-09-19T09:15:00.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.568+0000 7f7224581740 1 -- 172.21.15.120:0/2834760506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614f011a370 msgr2=0x5614f011e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:00.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.568+0000 7f7224581740 1 --2- 172.21.15.120:0/2834760506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614f011a370 0x5614f011e7f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7214005430 tx=0x7f72140145f0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.568+0000 7f7224581740 1 -- 172.21.15.120:0/2834760506 shutdown_connections 2024-09-19T09:15:00.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.568+0000 7f7224581740 1 --2- 172.21.15.120:0/2834760506 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5614f0195000 0x5614f0197420 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.568+0000 7f7224581740 1 --2- 172.21.15.120:0/2834760506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614f011a370 0x5614f011e7f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.568+0000 7f7224581740 1 --2- 172.21.15.120:0/2834760506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614f0110b40 0x5614f011e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.568+0000 7f7224581740 1 --2- 172.21.15.120:0/2834760506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614f010f5d0 0x5614f0122d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.568+0000 7f7224581740 1 -- 172.21.15.120:0/2834760506 >> 172.21.15.120:0/2834760506 conn(0x5614f00d8310 msgr2=0x5614f010ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:00.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.568+0000 7f71faffd640 0 client.28772 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:15:00.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.568+0000 7f7224581740 1 -- 172.21.15.120:0/2834760506 shutdown_connections 2024-09-19T09:15:00.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.568+0000 7f7224581740 1 -- 172.21.15.120:0/2834760506 wait complete. 2024-09-19T09:15:00.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.568+0000 7f7224581740 20 client.28772 trim_cache size 0 max 16384 2024-09-19T09:15:00.579 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:15:00.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.569+0000 7f72221a8640 1 -- 172.21.15.120:0/1677451936 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f72080a3240 msgr2=0x7f72080a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:00.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.569+0000 7f72221a8640 1 --2- 172.21.15.120:0/1677451936 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f72080a3240 0x7f72080a5700 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f720c009230 tx=0x7f720c0058d0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.569+0000 7f72221a8640 1 -- 172.21.15.120:0/1677451936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f721c1027a0 msgr2=0x7f721c1988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:00.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.569+0000 7f72221a8640 1 --2- 172.21.15.120:0/1677451936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f721c1027a0 0x7f721c1988f0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f720400e9f0 tx=0x7f720400eeb0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.569+0000 7f72221a8640 1 -- 172.21.15.120:0/1677451936 shutdown_connections 2024-09-19T09:15:00.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.569+0000 7f72221a8640 1 --2- 172.21.15.120:0/1677451936 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f72080a3240 0x7f72080a5700 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.569+0000 7f72221a8640 1 --2- 172.21.15.120:0/1677451936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f721c106f20 0x7f721c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.569+0000 7f72221a8640 1 --2- 172.21.15.120:0/1677451936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f721c1027a0 0x7f721c1988f0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.569+0000 7f72221a8640 1 --2- 172.21.15.120:0/1677451936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f721c101df0 0x7f721c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:00.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.569+0000 7f72221a8640 1 -- 172.21.15.120:0/1677451936 >> 172.21.15.120:0/1677451936 conn(0x7f721c0fd8f0 msgr2=0x7f721c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:00.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.569+0000 7f72221a8640 1 -- 172.21.15.120:0/1677451936 shutdown_connections 2024-09-19T09:15:00.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:00.569+0000 7f72221a8640 1 -- 172.21.15.120:0/1677451936 wait complete. 2024-09-19T09:15:01.076 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1190428 9.75 MB/sec execute 536 sec latency 35.453 ms 2024-09-19T09:15:01.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:00 smithi196 ceph-mon[35150]: pgmap v708: 97 pgs: 97 active+clean; 51 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.7 MiB/s wr, 377 op/s 2024-09-19T09:15:01.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:00 smithi153 ceph-mon[35071]: pgmap v708: 97 pgs: 97 active+clean; 51 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.7 MiB/s wr, 377 op/s 2024-09-19T09:15:01.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:00 smithi120 ceph-mon[31150]: pgmap v708: 97 pgs: 97 active+clean; 51 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.7 MiB/s wr, 377 op/s 2024-09-19T09:15:01.291 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:15:01.551 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:15:02.076 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1193541 9.76 MB/sec execute 537 sec latency 16.121 ms 2024-09-19T09:15:02.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.069+0000 7f8aa9ff1640 1 -- 172.21.15.120:0/1851683028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8aa410c170 msgr2=0x7f8aa410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:02.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.069+0000 7f8aa9ff1640 1 --2- 172.21.15.120:0/1851683028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8aa410c170 0x7f8aa410e610 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c009ca0 tx=0x7f8a9c02a950 comp rx=0 tx=0).stop 2024-09-19T09:15:02.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.069+0000 7f8aa9ff1640 1 -- 172.21.15.120:0/1851683028 shutdown_connections 2024-09-19T09:15:02.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.069+0000 7f8aa9ff1640 1 --2- 172.21.15.120:0/1851683028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8aa410c170 0x7f8aa410e610 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.069+0000 7f8aa9ff1640 1 --2- 172.21.15.120:0/1851683028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8aa4103660 0x7f8aa410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.069+0000 7f8aa9ff1640 1 --2- 172.21.15.120:0/1851683028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aa4102d40 0x7f8aa4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.069+0000 7f8aa9ff1640 1 -- 172.21.15.120:0/1851683028 >> 172.21.15.120:0/1851683028 conn(0x7f8aa40fc7a0 msgr2=0x7f8aa40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:02.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.069+0000 7f8aa9ff1640 1 -- 172.21.15.120:0/1851683028 shutdown_connections 2024-09-19T09:15:02.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.069+0000 7f8aa9ff1640 1 -- 172.21.15.120:0/1851683028 wait complete. 2024-09-19T09:15:02.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.070+0000 7f8aa9ff1640 1 Processor -- start 2024-09-19T09:15:02.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.070+0000 7f8aa9ff1640 1 -- start start 2024-09-19T09:15:02.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.070+0000 7f8aa9ff1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8aa4102d40 0x7f8aa4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:02.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.070+0000 7f8aa9ff1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aa4103660 0x7f8aa4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:02.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.070+0000 7f8aa9ff1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8aa410c170 0x7f8aa419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:02.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.070+0000 7f8aa9ff1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa4111f70 con 0x7f8aa410c170 2024-09-19T09:15:02.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.070+0000 7f8aa9ff1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa4111df0 con 0x7f8aa4102d40 2024-09-19T09:15:02.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.070+0000 7f8aa9ff1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa41120f0 con 0x7f8aa4103660 2024-09-19T09:15:02.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.070+0000 7f8aa3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8aa410c170 0x7f8aa419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:02.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.070+0000 7f8aa37fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8aa4102d40 0x7f8aa4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:02.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.070+0000 7f8aa3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8aa410c170 0x7f8aa419cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51558/0 (socket says 172.21.15.120:51558) 2024-09-19T09:15:02.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.070+0000 7f8aa3fff640 1 -- 172.21.15.120:0/3620179580 learned_addr learned my addr 172.21.15.120:0/3620179580 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:02.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.070+0000 7f8aa2ffd640 1 --2- 172.21.15.120:0/3620179580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aa4103660 0x7f8aa4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:02.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.071+0000 7f8aa3fff640 1 -- 172.21.15.120:0/3620179580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aa4103660 msgr2=0x7f8aa4198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:02.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.071+0000 7f8aa3fff640 1 --2- 172.21.15.120:0/3620179580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aa4103660 0x7f8aa4198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.071+0000 7f8aa3fff640 1 -- 172.21.15.120:0/3620179580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8aa4102d40 msgr2=0x7f8aa4198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:02.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.071+0000 7f8aa3fff640 1 --2- 172.21.15.120:0/3620179580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8aa4102d40 0x7f8aa4198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.071+0000 7f8aa3fff640 1 -- 172.21.15.120:0/3620179580 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa419d3c0 con 0x7f8aa410c170 2024-09-19T09:15:02.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.071+0000 7f8aa37fe640 1 --2- 172.21.15.120:0/3620179580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8aa4102d40 0x7f8aa4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:15:02.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.071+0000 7f8aa3fff640 1 --2- 172.21.15.120:0/3620179580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8aa410c170 0x7f8aa419cce0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c004af0 tx=0x7f8a9c02f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:02.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.071+0000 7f8aa0ff9640 1 -- 172.21.15.120:0/3620179580 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c034040 con 0x7f8aa410c170 2024-09-19T09:15:02.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.071+0000 7f8aa9ff1640 1 -- 172.21.15.120:0/3620179580 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa419d650 con 0x7f8aa410c170 2024-09-19T09:15:02.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.071+0000 7f8aa0ff9640 1 -- 172.21.15.120:0/3620179580 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a9c030920 con 0x7f8aa410c170 2024-09-19T09:15:02.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.071+0000 7f8aa9ff1640 1 -- 172.21.15.120:0/3620179580 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa41a4f90 con 0x7f8aa410c170 2024-09-19T09:15:02.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.071+0000 7f8aa0ff9640 1 -- 172.21.15.120:0/3620179580 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c004610 con 0x7f8aa410c170 2024-09-19T09:15:02.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.073+0000 7f8aa0ff9640 1 -- 172.21.15.120:0/3620179580 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f8a9c037020 con 0x7f8aa410c170 2024-09-19T09:15:02.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.074+0000 7f8aac3ca740 1 Processor -- start 2024-09-19T09:15:02.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.074+0000 7f8aac3ca740 1 -- start start 2024-09-19T09:15:02.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.074+0000 7f8aac3ca740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595318c7b40 0x5595318c6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:02.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.074+0000 7f8aac3ca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595318c65d0 0x5595318c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:02.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.074+0000 7f8aac3ca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595318d5780 0x5595318d7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:02.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.074+0000 7f8aac3ca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55953182c2f0 con 0x5595318d5780 2024-09-19T09:15:02.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.074+0000 7f8aac3ca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5595317d3cf0 con 0x5595318c65d0 2024-09-19T09:15:02.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.074+0000 7f8aac3ca740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5595318282b0 con 0x5595318c7b40 2024-09-19T09:15:02.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.074+0000 7f8aa0ff9640 1 --2- 172.21.15.120:0/3620179580 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a8c0a3240 0x7f8a8c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:02.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.074+0000 7f8aa2ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595318c7b40 0x5595318c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:02.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.074+0000 7f8aa2ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595318c7b40 0x5595318c6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:59968/0 (socket says 172.21.15.120:59968) 2024-09-19T09:15:02.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.074+0000 7f8aa2ffd640 1 -- 172.21.15.120:0/3840102880 learned_addr learned my addr 172.21.15.120:0/3840102880 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:02.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.074+0000 7f8aa3fff640 1 --2- 172.21.15.120:0/3620179580 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a8c0a3240 0x7f8a8c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:02.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.074+0000 7f8aa2ffd640 1 --2- 172.21.15.120:0/3840102880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595318d5780 0x5595318d7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:02.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.074+0000 7f8aa37fe640 1 --2- 172.21.15.120:0/3840102880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595318c65d0 0x5595318c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:02.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.075+0000 7f8aa0ff9640 1 -- 172.21.15.120:0/3620179580 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f8a9c0f0630 con 0x7f8aa410c170 2024-09-19T09:15:02.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.075+0000 7f8aa2ffd640 1 -- 172.21.15.120:0/3840102880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595318c65d0 msgr2=0x5595318c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:02.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.075+0000 7f8aa2ffd640 1 --2- 172.21.15.120:0/3840102880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595318c65d0 0x5595318c6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.075+0000 7f8aa2ffd640 1 -- 172.21.15.120:0/3840102880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595318d5780 msgr2=0x5595318d7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:02.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.075+0000 7f8aa2ffd640 1 --2- 172.21.15.120:0/3840102880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595318d5780 0x5595318d7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.075+0000 7f8aa2ffd640 1 -- 172.21.15.120:0/3840102880 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595317b83a0 con 0x5595318c7b40 2024-09-19T09:15:02.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.075+0000 7f8aa2ffd640 1 --2- 172.21.15.120:0/3840102880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595318c7b40 0x5595318c6090 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009730 tx=0x7f8a980096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3728655d9d8ce167 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:02.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.075+0000 7f8aa3fff640 1 --2- 172.21.15.120:0/3620179580 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a8c0a3240 0x7f8a8c0a5700 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c04d000 tx=0x7f8a9c0b20f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:02.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.075+0000 7f8a89ffb640 1 -- 172.21.15.120:0/3840102880 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a98004030 con 0x5595318c7b40 2024-09-19T09:15:02.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.075+0000 7f8a89ffb640 1 -- 172.21.15.120:0/3840102880 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a9800fca0 con 0x5595318c7b40 2024-09-19T09:15:02.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.075+0000 7f8a89ffb640 1 -- 172.21.15.120:0/3840102880 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a98003950 con 0x5595318c7b40 2024-09-19T09:15:02.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.076+0000 7f8aac3ca740 1 -- 172.21.15.120:0/3840102880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595318c7b40 msgr2=0x5595318c6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:02.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.076+0000 7f8aac3ca740 1 --2- 172.21.15.120:0/3840102880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595318c7b40 0x5595318c6090 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009730 tx=0x7f8a980096c0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.076+0000 7f8aac3ca740 1 -- 172.21.15.120:0/3840102880 shutdown_connections 2024-09-19T09:15:02.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.076+0000 7f8aac3ca740 1 --2- 172.21.15.120:0/3840102880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595318d5780 0x5595318d7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.076+0000 7f8aac3ca740 1 --2- 172.21.15.120:0/3840102880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595318c65d0 0x5595318c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.076+0000 7f8aac3ca740 1 --2- 172.21.15.120:0/3840102880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595318c7b40 0x5595318c6090 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.076+0000 7f8aac3ca740 1 -- 172.21.15.120:0/3840102880 >> 172.21.15.120:0/3840102880 conn(0x55953188f310 msgr2=0x5595318c2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:02.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.076+0000 7f8aac3ca740 1 -- 172.21.15.120:0/3840102880 shutdown_connections 2024-09-19T09:15:02.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.076+0000 7f8aac3ca740 1 -- 172.21.15.120:0/3840102880 wait complete. 2024-09-19T09:15:02.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.076+0000 7f8aac3ca740 1 -- start start 2024-09-19T09:15:02.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.076+0000 7f8aac3ca740 1 Processor -- start 2024-09-19T09:15:02.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.077+0000 7f8aac3ca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595318c65d0 0x5595318d5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:02.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.077+0000 7f8aac3ca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595318c7b40 0x5595318d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:02.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.077+0000 7f8aac3ca740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595318d5780 0x5595318d2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:02.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.077+0000 7f8aac3ca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5595318d8480 con 0x5595318c65d0 2024-09-19T09:15:02.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.077+0000 7f8aac3ca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5595317e7060 con 0x5595318c7b40 2024-09-19T09:15:02.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.077+0000 7f8aac3ca740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5595318d8600 con 0x5595318d5780 2024-09-19T09:15:02.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.077+0000 7f8aac3ca740 10 client.0 _ll_register_callbacks cb 0x55953153a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:02.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.077+0000 7f8aa2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595318c65d0 0x5595318d5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:02.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.077+0000 7f8aa2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595318c65d0 0x5595318d5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51586/0 (socket says 172.21.15.120:51586) 2024-09-19T09:15:02.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.077+0000 7f8aa37fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595318c7b40 0x5595318d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:02.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.077+0000 7f8aa2ffd640 1 -- 172.21.15.120:0/3993530262 learned_addr learned my addr 172.21.15.120:0/3993530262 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:02.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.077+0000 7f8aa2ffd640 1 --2- 172.21.15.120:0/3993530262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595318d5780 0x5595318d2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:02.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.077+0000 7f8aa37fe640 1 -- 172.21.15.120:0/3993530262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595318d5780 msgr2=0x5595318d2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:02.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.077+0000 7f8aa37fe640 1 --2- 172.21.15.120:0/3993530262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595318d5780 0x5595318d2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.077+0000 7f8aa37fe640 1 -- 172.21.15.120:0/3993530262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595318c65d0 msgr2=0x5595318d5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:02.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.077+0000 7f8aa37fe640 1 --2- 172.21.15.120:0/3993530262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595318c65d0 0x5595318d5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.077+0000 7f8aa37fe640 1 -- 172.21.15.120:0/3993530262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55953159d3c0 con 0x5595318c7b40 2024-09-19T09:15:02.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.077+0000 7f8aa2ffd640 1 --2- 172.21.15.120:0/3993530262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595318d5780 0x5595318d2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:15:02.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.077+0000 7f8aa37fe640 1 --2- 172.21.15.120:0/3993530262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595318c7b40 0x5595318d23e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8a94012cf0 tx=0x7f8a9400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8ce231a879013a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:02.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.077+0000 7f8a82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:15:02.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.078+0000 7f8a82ffd640 1 -- 172.21.15.120:0/3993530262 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a94012110 con 0x5595318c7b40 2024-09-19T09:15:02.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.078+0000 7f8aac3ca740 1 -- 172.21.15.120:0/3993530262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a9800fca0 con 0x5595318c7b40 2024-09-19T09:15:02.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.078+0000 7f8a82ffd640 1 -- 172.21.15.120:0/3993530262 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f8a94004050 con 0x5595318c7b40 2024-09-19T09:15:02.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.078+0000 7f8a82ffd640 1 -- 172.21.15.120:0/3993530262 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8a940122b0 con 0x5595318c7b40 2024-09-19T09:15:02.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.078+0000 7f8aac3ca740 10 client.28784 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:02.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.078+0000 7f8aac3ca740 1 -- 172.21.15.120:0/3993530262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5595318e3f30 con 0x5595318c7b40 2024-09-19T09:15:02.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.078+0000 7f8a82ffd640 1 -- 172.21.15.120:0/3993530262 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a9800fca0 con 0x5595318c7b40 2024-09-19T09:15:02.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.078+0000 7f8a82ffd640 1 -- 172.21.15.120:0/3993530262 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f8a940104c0 con 0x5595318c7b40 2024-09-19T09:15:02.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.078+0000 7f8aac3ca740 10 client.28784 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:02.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.078+0000 7f8aac3ca740 10 client.28784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:15:02.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.078+0000 7f8aac3ca740 20 client.28784 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:02.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.078+0000 7f8aac3ca740 1 --2- 172.21.15.120:0/3993530262 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5595318e5440 0x5595318e7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:02.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.078+0000 7f8aac3ca740 4 client.28784 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:15:02.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.078+0000 7f8aac3ca740 1 -- 172.21.15.120:0/3993530262 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559531839860 con 0x5595318e5440 2024-09-19T09:15:02.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.078+0000 7f8aa2ffd640 1 --2- 172.21.15.120:0/3993530262 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5595318e5440 0x5595318e7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:02.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.079+0000 7f8a82ffd640 10 client.28784 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:02.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.079+0000 7f8aa2ffd640 1 --2- 172.21.15.120:0/3993530262 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5595318e5440 0x5595318e7830 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c752c4ccfa48c server_cookie=9caf5b210654a6cd in_seq=0 out_seq=0 2024-09-19T09:15:02.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.080+0000 7f8a82ffd640 1 -- 172.21.15.120:0/3993530262 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559531839860 con 0x5595318e5440 2024-09-19T09:15:02.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.080+0000 7f8a82ffd640 10 client.28784 handle_command_reply: tid=0 2024-09-19T09:15:02.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.080+0000 7f8aac3ca740 1 client.28784 shutdown 2024-09-19T09:15:02.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.080+0000 7f8aac3ca740 1 -- 172.21.15.120:0/3993530262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595318c7b40 msgr2=0x5595318d23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:02.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.080+0000 7f8aac3ca740 1 --2- 172.21.15.120:0/3993530262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595318c7b40 0x5595318d23e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8a94012cf0 tx=0x7f8a9400f790 comp rx=0 tx=0).stop 2024-09-19T09:15:02.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.080+0000 7f8aac3ca740 1 -- 172.21.15.120:0/3993530262 shutdown_connections 2024-09-19T09:15:02.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.080+0000 7f8aac3ca740 1 --2- 172.21.15.120:0/3993530262 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5595318e5440 0x5595318e7830 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.080+0000 7f8aac3ca740 1 --2- 172.21.15.120:0/3993530262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595318d5780 0x5595318d2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.080+0000 7f8aac3ca740 1 --2- 172.21.15.120:0/3993530262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595318c7b40 0x5595318d23e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.080+0000 7f8a82ffd640 0 client.28784 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:02.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.080+0000 7f8aac3ca740 1 --2- 172.21.15.120:0/3993530262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595318c65d0 0x5595318d5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.080+0000 7f8aac3ca740 1 -- 172.21.15.120:0/3993530262 >> 172.21.15.120:0/3993530262 conn(0x55953188f310 msgr2=0x5595318c8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:02.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.080+0000 7f8aac3ca740 1 -- 172.21.15.120:0/3993530262 shutdown_connections 2024-09-19T09:15:02.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.080+0000 7f8aac3ca740 1 -- 172.21.15.120:0/3993530262 wait complete. 2024-09-19T09:15:02.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.081+0000 7f8aac3ca740 20 client.28784 trim_cache size 0 max 16384 2024-09-19T09:15:02.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.093+0000 7f8aac3ca740 1 Processor -- start 2024-09-19T09:15:02.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.093+0000 7f8aac3ca740 1 -- start start 2024-09-19T09:15:02.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.094+0000 7f8aac3ca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595318c65d0 0x5595318d23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:02.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.094+0000 7f8aac3ca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595318c7b40 0x5595318e1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:02.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.094+0000 7f8aac3ca740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595318d5780 0x5595318ed140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:02.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.094+0000 7f8aac3ca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa41ab310 con 0x5595318c7b40 2024-09-19T09:15:02.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.094+0000 7f8aac3ca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa41ace50 con 0x5595318c65d0 2024-09-19T09:15:02.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.094+0000 7f8aac3ca740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa41ab9e0 con 0x5595318d5780 2024-09-19T09:15:02.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.094+0000 7f8aa37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595318c7b40 0x5595318e1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:02.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.094+0000 7f8aa37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595318c7b40 0x5595318e1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51596/0 (socket says 172.21.15.120:51596) 2024-09-19T09:15:02.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.094+0000 7f8aa37fe640 1 -- 172.21.15.120:0/578280586 learned_addr learned my addr 172.21.15.120:0/578280586 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:02.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.094+0000 7f8aa2ffd640 1 --2- 172.21.15.120:0/578280586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595318c65d0 0x5595318d23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:02.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.094+0000 7f8aa2ffd640 1 --2- 172.21.15.120:0/578280586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595318d5780 0x5595318ed140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:02.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.094+0000 7f8aa37fe640 1 -- 172.21.15.120:0/578280586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595318d5780 msgr2=0x5595318ed140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:02.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.094+0000 7f8aa37fe640 1 --2- 172.21.15.120:0/578280586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595318d5780 0x5595318ed140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.094+0000 7f8aa37fe640 1 -- 172.21.15.120:0/578280586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595318c65d0 msgr2=0x5595318d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:02.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.094+0000 7f8aa37fe640 1 --2- 172.21.15.120:0/578280586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595318c65d0 0x5595318d23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.094+0000 7f8aa37fe640 1 -- 172.21.15.120:0/578280586 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595318d54a0 con 0x5595318c7b40 2024-09-19T09:15:02.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.094+0000 7f8aa37fe640 1 --2- 172.21.15.120:0/578280586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595318c7b40 0x5595318e1720 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f8a94010ad0 tx=0x7f8a94005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba91c2b084d8a0c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:02.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.094+0000 7f8a897fa640 1 -- 172.21.15.120:0/578280586 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a94000e60 con 0x5595318c7b40 2024-09-19T09:15:02.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.094+0000 7f8a897fa640 1 -- 172.21.15.120:0/578280586 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a94010430 con 0x5595318c7b40 2024-09-19T09:15:02.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.094+0000 7f8a897fa640 1 -- 172.21.15.120:0/578280586 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a94027ba0 con 0x5595318c7b40 2024-09-19T09:15:02.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.095+0000 7f8aac3ca740 1 -- 172.21.15.120:0/578280586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595318c7b40 msgr2=0x5595318e1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:02.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.095+0000 7f8aac3ca740 1 --2- 172.21.15.120:0/578280586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595318c7b40 0x5595318e1720 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f8a94010ad0 tx=0x7f8a94005880 comp rx=0 tx=0).stop 2024-09-19T09:15:02.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.095+0000 7f8aac3ca740 1 -- 172.21.15.120:0/578280586 shutdown_connections 2024-09-19T09:15:02.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.095+0000 7f8aac3ca740 1 --2- 172.21.15.120:0/578280586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595318d5780 0x5595318ed140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.095+0000 7f8aac3ca740 1 --2- 172.21.15.120:0/578280586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595318c7b40 0x5595318e1720 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.095+0000 7f8aac3ca740 1 --2- 172.21.15.120:0/578280586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595318c65d0 0x5595318d23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.095+0000 7f8aac3ca740 1 -- 172.21.15.120:0/578280586 >> 172.21.15.120:0/578280586 conn(0x55953188f310 msgr2=0x5595318d1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:02.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.095+0000 7f8aac3ca740 1 -- 172.21.15.120:0/578280586 shutdown_connections 2024-09-19T09:15:02.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.095+0000 7f8aac3ca740 1 -- 172.21.15.120:0/578280586 wait complete. 2024-09-19T09:15:02.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.096+0000 7f8aac3ca740 1 -- start start 2024-09-19T09:15:02.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.096+0000 7f8aac3ca740 1 Processor -- start 2024-09-19T09:15:02.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.096+0000 7f8aac3ca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595318c65d0 0x5595318d9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:02.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.096+0000 7f8aac3ca740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595318c7b40 0x5595318d9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:02.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.096+0000 7f8aac3ca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595318d5780 0x5595318da210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:02.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.096+0000 7f8aac3ca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa41ac780 con 0x5595318d5780 2024-09-19T09:15:02.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.096+0000 7f8aac3ca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa41ac0b0 con 0x5595318c65d0 2024-09-19T09:15:02.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.096+0000 7f8aac3ca740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa41a92f0 con 0x5595318c7b40 2024-09-19T09:15:02.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.096+0000 7f8aac3ca740 10 client.0 _ll_register_callbacks cb 0x7f8a94009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:02.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.096+0000 7f8aa2ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595318c65d0 0x5595318d9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:02.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.096+0000 7f8aa2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595318d5780 0x5595318da210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:02.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.096+0000 7f8aa2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595318d5780 0x5595318da210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51610/0 (socket says 172.21.15.120:51610) 2024-09-19T09:15:02.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.096+0000 7f8aa2ffd640 1 -- 172.21.15.120:0/2131274404 learned_addr learned my addr 172.21.15.120:0/2131274404 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:02.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.096+0000 7f8aa37fe640 1 --2- 172.21.15.120:0/2131274404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595318c7b40 0x5595318d9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:02.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.097+0000 7f8aa2ffd640 1 -- 172.21.15.120:0/2131274404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595318c7b40 msgr2=0x5595318d9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:02.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.097+0000 7f8aa2ffd640 1 --2- 172.21.15.120:0/2131274404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595318c7b40 0x5595318d9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.097+0000 7f8aa2ffd640 1 -- 172.21.15.120:0/2131274404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595318c65d0 msgr2=0x5595318d9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:02.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.097+0000 7f8aa2ffd640 1 --2- 172.21.15.120:0/2131274404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595318c65d0 0x5595318d9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.097+0000 7f8aa2ffd640 1 -- 172.21.15.120:0/2131274404 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55953194f2b0 con 0x5595318d5780 2024-09-19T09:15:02.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.097+0000 7f8aa37fe640 1 --2- 172.21.15.120:0/2131274404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595318c7b40 0x5595318d9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:15:02.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.097+0000 7f8aa2ffd640 1 --2- 172.21.15.120:0/2131274404 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595318d5780 0x5595318da210 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f8a98011730 tx=0x7f8a98024040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4211a098214cea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:02.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.097+0000 7f8a82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:15:02.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.097+0000 7f8a82ffd640 1 -- 172.21.15.120:0/2131274404 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a98005020 con 0x5595318d5780 2024-09-19T09:15:02.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.097+0000 7f8aac3ca740 1 -- 172.21.15.120:0/2131274404 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a94010430 con 0x5595318d5780 2024-09-19T09:15:02.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.097+0000 7f8a82ffd640 1 -- 172.21.15.120:0/2131274404 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f8a9802b040 con 0x5595318d5780 2024-09-19T09:15:02.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.097+0000 7f8a82ffd640 1 -- 172.21.15.120:0/2131274404 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8a980051c0 con 0x5595318d5780 2024-09-19T09:15:02.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.097+0000 7f8aac3ca740 10 client.19743 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:02.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.097+0000 7f8aac3ca740 1 -- 172.21.15.120:0/2131274404 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559531950590 con 0x5595318d5780 2024-09-19T09:15:02.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.097+0000 7f8a82ffd640 1 -- 172.21.15.120:0/2131274404 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a94010430 con 0x5595318d5780 2024-09-19T09:15:02.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.097+0000 7f8a82ffd640 1 -- 172.21.15.120:0/2131274404 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f8a9803c470 con 0x5595318d5780 2024-09-19T09:15:02.132 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:15:02.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.097+0000 7f8aac3ca740 10 client.19743 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:02.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.097+0000 7f8aac3ca740 10 client.19743 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:15:02.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.097+0000 7f8aac3ca740 20 client.19743 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:02.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.097+0000 7f8aac3ca740 1 --2- 172.21.15.120:0/2131274404 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559531950ab0 0x559531952ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:02.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.097+0000 7f8aac3ca740 4 client.19743 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:15:02.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.097+0000 7f8aac3ca740 1 -- 172.21.15.120:0/2131274404 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559531833d70 con 0x559531950ab0 2024-09-19T09:15:02.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.098+0000 7f8aa37fe640 1 --2- 172.21.15.120:0/2131274404 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559531950ab0 0x559531952ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:02.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.098+0000 7f8a82ffd640 10 client.19743 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:02.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.098+0000 7f8aa37fe640 1 --2- 172.21.15.120:0/2131274404 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559531950ab0 0x559531952ea0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34422c4777ca1c12 server_cookie=2f5ca29517f9f1f3 in_seq=0 out_seq=0 2024-09-19T09:15:02.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.102+0000 7f8a82ffd640 1 -- 172.21.15.120:0/2131274404 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559531833d70 con 0x559531950ab0 2024-09-19T09:15:02.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.102+0000 7f8a82ffd640 10 client.19743 handle_command_reply: tid=0 2024-09-19T09:15:02.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.102+0000 7f8aac3ca740 1 client.19743 shutdown 2024-09-19T09:15:02.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.102+0000 7f8aac3ca740 1 -- 172.21.15.120:0/2131274404 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595318d5780 msgr2=0x5595318da210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:02.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.102+0000 7f8aac3ca740 1 --2- 172.21.15.120:0/2131274404 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595318d5780 0x5595318da210 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f8a98011730 tx=0x7f8a98024040 comp rx=0 tx=0).stop 2024-09-19T09:15:02.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.102+0000 7f8aac3ca740 1 -- 172.21.15.120:0/2131274404 shutdown_connections 2024-09-19T09:15:02.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.102+0000 7f8aac3ca740 1 --2- 172.21.15.120:0/2131274404 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559531950ab0 0x559531952ea0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.102+0000 7f8aac3ca740 1 --2- 172.21.15.120:0/2131274404 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595318d5780 0x5595318da210 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.102+0000 7f8aac3ca740 1 --2- 172.21.15.120:0/2131274404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595318c7b40 0x5595318d9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.102+0000 7f8aac3ca740 1 --2- 172.21.15.120:0/2131274404 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595318c65d0 0x5595318d9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.102+0000 7f8aac3ca740 1 -- 172.21.15.120:0/2131274404 >> 172.21.15.120:0/2131274404 conn(0x55953188f310 msgr2=0x5595318d3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:02.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.102+0000 7f8a82ffd640 0 client.19743 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:02.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.103+0000 7f8aac3ca740 1 -- 172.21.15.120:0/2131274404 shutdown_connections 2024-09-19T09:15:02.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.103+0000 7f8aac3ca740 1 -- 172.21.15.120:0/2131274404 wait complete. 2024-09-19T09:15:02.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.103+0000 7f8aac3ca740 20 client.19743 trim_cache size 0 max 16384 2024-09-19T09:15:02.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.103+0000 7f8aa9ff1640 1 -- 172.21.15.120:0/3620179580 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a8c0a3240 msgr2=0x7f8a8c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:02.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.103+0000 7f8aa9ff1640 1 --2- 172.21.15.120:0/3620179580 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a8c0a3240 0x7f8a8c0a5700 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c04d000 tx=0x7f8a9c0b20f0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.103+0000 7f8aa9ff1640 1 -- 172.21.15.120:0/3620179580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8aa410c170 msgr2=0x7f8aa419cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:02.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.103+0000 7f8aa9ff1640 1 --2- 172.21.15.120:0/3620179580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8aa410c170 0x7f8aa419cce0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c004af0 tx=0x7f8a9c02f3a0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.103+0000 7f8aa9ff1640 1 -- 172.21.15.120:0/3620179580 shutdown_connections 2024-09-19T09:15:02.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.103+0000 7f8aa9ff1640 1 --2- 172.21.15.120:0/3620179580 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a8c0a3240 0x7f8a8c0a5700 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.103+0000 7f8aa9ff1640 1 --2- 172.21.15.120:0/3620179580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8aa410c170 0x7f8aa419cce0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.103+0000 7f8aa9ff1640 1 --2- 172.21.15.120:0/3620179580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aa4103660 0x7f8aa4198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.103+0000 7f8aa9ff1640 1 --2- 172.21.15.120:0/3620179580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8aa4102d40 0x7f8aa4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:02.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.103+0000 7f8aa9ff1640 1 -- 172.21.15.120:0/3620179580 >> 172.21.15.120:0/3620179580 conn(0x7f8aa40fc7a0 msgr2=0x7f8aa40fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:02.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.103+0000 7f8aa9ff1640 1 -- 172.21.15.120:0/3620179580 shutdown_connections 2024-09-19T09:15:02.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:02.103+0000 7f8aa9ff1640 1 -- 172.21.15.120:0/3620179580 wait complete. 2024-09-19T09:15:02.990 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:02 smithi120 ceph-mon[31150]: pgmap v709: 97 pgs: 97 active+clean; 43 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.8 MiB/s wr, 392 op/s 2024-09-19T09:15:03.076 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1196176 9.77 MB/sec execute 538 sec latency 11.529 ms 2024-09-19T09:15:03.188 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:02 smithi196 ceph-mon[35150]: pgmap v709: 97 pgs: 97 active+clean; 43 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.8 MiB/s wr, 392 op/s 2024-09-19T09:15:03.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:02 smithi153 ceph-mon[35071]: pgmap v709: 97 pgs: 97 active+clean; 43 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.8 MiB/s wr, 392 op/s 2024-09-19T09:15:04.021 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:15:04.021 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:15:04.076 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1197843 9.77 MB/sec execute 539 sec latency 128.880 ms 2024-09-19T09:15:04.279 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:15:04.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.833+0000 7fdc41110640 1 -- 172.21.15.120:0/2810102519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc3c100e20 msgr2=0x7fdc3c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:04.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.833+0000 7fdc41110640 1 --2- 172.21.15.120:0/2810102519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc3c100e20 0x7fdc3c101200 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c009900 tx=0x7fdc2c02a4a0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.833+0000 7fdc41110640 1 -- 172.21.15.120:0/2810102519 shutdown_connections 2024-09-19T09:15:04.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.833+0000 7fdc41110640 1 --2- 172.21.15.120:0/2810102519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc3c102ef0 0x7fdc3c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.833+0000 7fdc41110640 1 --2- 172.21.15.120:0/2810102519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc3c102570 0x7fdc3c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.833+0000 7fdc41110640 1 --2- 172.21.15.120:0/2810102519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc3c100e20 0x7fdc3c101200 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.833+0000 7fdc41110640 1 -- 172.21.15.120:0/2810102519 >> 172.21.15.120:0/2810102519 conn(0x7fdc3c0fc7d0 msgr2=0x7fdc3c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:04.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.834+0000 7fdc41110640 1 -- 172.21.15.120:0/2810102519 shutdown_connections 2024-09-19T09:15:04.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.834+0000 7fdc41110640 1 -- 172.21.15.120:0/2810102519 wait complete. 2024-09-19T09:15:04.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.834+0000 7fdc41110640 1 Processor -- start 2024-09-19T09:15:04.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.835+0000 7fdc41110640 1 -- start start 2024-09-19T09:15:04.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.835+0000 7fdc41110640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc3c100e20 0x7fdc3c198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:04.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.835+0000 7fdc41110640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc3c102570 0x7fdc3c198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:04.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.835+0000 7fdc41110640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc3c102ef0 0x7fdc3c19ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:04.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.835+0000 7fdc41110640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3c110fe0 con 0x7fdc3c102570 2024-09-19T09:15:04.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.835+0000 7fdc41110640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3c110e60 con 0x7fdc3c100e20 2024-09-19T09:15:04.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.835+0000 7fdc41110640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3c111160 con 0x7fdc3c102ef0 2024-09-19T09:15:04.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.835+0000 7fdc3a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc3c102570 0x7fdc3c198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:04.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.835+0000 7fdc3a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc3c102570 0x7fdc3c198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51634/0 (socket says 172.21.15.120:51634) 2024-09-19T09:15:04.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.835+0000 7fdc3a59c640 1 -- 172.21.15.120:0/1485188899 learned_addr learned my addr 172.21.15.120:0/1485188899 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:04.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.835+0000 7fdc3ad9d640 1 --2- 172.21.15.120:0/1485188899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc3c100e20 0x7fdc3c198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:04.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.835+0000 7fdc3a59c640 1 -- 172.21.15.120:0/1485188899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc3c102ef0 msgr2=0x7fdc3c19ce50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T09:15:04.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.835+0000 7fdc3a59c640 1 --2- 172.21.15.120:0/1485188899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc3c102ef0 0x7fdc3c19ce50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.835+0000 7fdc3a59c640 1 -- 172.21.15.120:0/1485188899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc3c100e20 msgr2=0x7fdc3c198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:04.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.835+0000 7fdc3a59c640 1 --2- 172.21.15.120:0/1485188899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc3c100e20 0x7fdc3c198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.835+0000 7fdc3a59c640 1 -- 172.21.15.120:0/1485188899 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc3c19d530 con 0x7fdc3c102570 2024-09-19T09:15:04.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.835+0000 7fdc3a59c640 1 --2- 172.21.15.120:0/1485188899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc3c102570 0x7fdc3c198ac0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000b450 tx=0x7fdc3000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:04.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.836+0000 7fdc23fff640 1 -- 172.21.15.120:0/1485188899 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc30015020 con 0x7fdc3c102570 2024-09-19T09:15:04.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.836+0000 7fdc41110640 1 -- 172.21.15.120:0/1485188899 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc3c19d820 con 0x7fdc3c102570 2024-09-19T09:15:04.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.836+0000 7fdc41110640 1 -- 172.21.15.120:0/1485188899 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc3c1a5100 con 0x7fdc3c102570 2024-09-19T09:15:04.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.836+0000 7fdc23fff640 1 -- 172.21.15.120:0/1485188899 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc30011d70 con 0x7fdc3c102570 2024-09-19T09:15:04.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.836+0000 7fdc23fff640 1 -- 172.21.15.120:0/1485188899 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc300103e0 con 0x7fdc3c102570 2024-09-19T09:15:04.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.838+0000 7fdc23fff640 1 -- 172.21.15.120:0/1485188899 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fdc30018020 con 0x7fdc3c102570 2024-09-19T09:15:04.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.838+0000 7fdc434e9740 1 Processor -- start 2024-09-19T09:15:04.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.838+0000 7fdc434e9740 1 -- start start 2024-09-19T09:15:04.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.838+0000 7fdc434e9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fb9e41b60 0x560fb9e41f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:04.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.838+0000 7fdc434e9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fb9e42480 0x560fb9e4c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:04.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.838+0000 7fdc434e9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fb9e4cd40 0x560fb9e4f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:04.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.839+0000 7fdc434e9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560fb9da7280 con 0x560fb9e41b60 2024-09-19T09:15:04.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.839+0000 7fdc434e9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560fb9d4eb10 con 0x560fb9e4cd40 2024-09-19T09:15:04.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.839+0000 7fdc434e9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560fb9da3240 con 0x560fb9e42480 2024-09-19T09:15:04.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.839+0000 7fdc3b59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fb9e41b60 0x560fb9e41f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:04.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.839+0000 7fdc3b59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fb9e41b60 0x560fb9e41f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51636/0 (socket says 172.21.15.120:51636) 2024-09-19T09:15:04.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.839+0000 7fdc3b59e640 1 -- 172.21.15.120:0/406676897 learned_addr learned my addr 172.21.15.120:0/406676897 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:04.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.839+0000 7fdc3a59c640 1 --2- 172.21.15.120:0/406676897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fb9e4cd40 0x560fb9e4f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:04.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.839+0000 7fdc3ad9d640 1 --2- 172.21.15.120:0/406676897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fb9e42480 0x560fb9e4c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:04.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.839+0000 7fdc23fff640 1 --2- 172.21.15.120:0/1485188899 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdc240a3240 0x7fdc240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:04.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.839+0000 7fdc3b59e640 1 -- 172.21.15.120:0/406676897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fb9e42480 msgr2=0x560fb9e4c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:04.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.839+0000 7fdc3b59e640 1 --2- 172.21.15.120:0/406676897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fb9e42480 0x560fb9e4c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.839+0000 7fdc3b59e640 1 -- 172.21.15.120:0/406676897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fb9e4cd40 msgr2=0x560fb9e4f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:04.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.839+0000 7fdc3b59e640 1 --2- 172.21.15.120:0/406676897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fb9e4cd40 0x560fb9e4f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.839+0000 7fdc3b59e640 1 -- 172.21.15.120:0/406676897 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fb9d5b1f0 con 0x560fb9e41b60 2024-09-19T09:15:04.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.839+0000 7fdc3a59c640 1 --2- 172.21.15.120:0/406676897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fb9e4cd40 0x560fb9e4f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:15:04.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.839+0000 7fdc23fff640 1 -- 172.21.15.120:0/1485188899 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fdc300cf420 con 0x7fdc3c102570 2024-09-19T09:15:04.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.839+0000 7fdc3b59e640 1 --2- 172.21.15.120:0/1485188899 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdc240a3240 0x7fdc240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:04.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.839+0000 7fdc3b59e640 1 --2- 172.21.15.120:0/406676897 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fb9e41b60 0x560fb9e41f40 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fdc34009e30 tx=0x7fdc3400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc265011fe0f2963 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:04.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.840+0000 7fdc217fa640 1 -- 172.21.15.120:0/406676897 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3401a610 con 0x560fb9e41b60 2024-09-19T09:15:04.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.840+0000 7fdc217fa640 1 -- 172.21.15.120:0/406676897 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc34010e30 con 0x560fb9e41b60 2024-09-19T09:15:04.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.840+0000 7fdc217fa640 1 -- 172.21.15.120:0/406676897 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc34019af0 con 0x560fb9e41b60 2024-09-19T09:15:04.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.840+0000 7fdc434e9740 1 -- 172.21.15.120:0/406676897 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fb9e41b60 msgr2=0x560fb9e41f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:04.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.840+0000 7fdc3b59e640 1 --2- 172.21.15.120:0/1485188899 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdc240a3240 0x7fdc240a5700 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdc3400f010 tx=0x7fdc34021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:04.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.840+0000 7fdc434e9740 1 --2- 172.21.15.120:0/406676897 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fb9e41b60 0x560fb9e41f40 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fdc34009e30 tx=0x7fdc3400e720 comp rx=0 tx=0).stop 2024-09-19T09:15:04.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.840+0000 7fdc434e9740 1 -- 172.21.15.120:0/406676897 shutdown_connections 2024-09-19T09:15:04.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.840+0000 7fdc434e9740 1 --2- 172.21.15.120:0/406676897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fb9e4cd40 0x560fb9e4f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.840+0000 7fdc434e9740 1 --2- 172.21.15.120:0/406676897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fb9e42480 0x560fb9e4c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.840+0000 7fdc434e9740 1 --2- 172.21.15.120:0/406676897 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fb9e41b60 0x560fb9e41f40 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.841+0000 7fdc434e9740 1 -- 172.21.15.120:0/406676897 >> 172.21.15.120:0/406676897 conn(0x560fb9e3e0f0 msgr2=0x560fb9e3e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:04.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.841+0000 7fdc434e9740 1 -- 172.21.15.120:0/406676897 shutdown_connections 2024-09-19T09:15:04.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.841+0000 7fdc434e9740 1 -- 172.21.15.120:0/406676897 wait complete. 2024-09-19T09:15:04.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.841+0000 7fdc434e9740 1 -- start start 2024-09-19T09:15:04.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.841+0000 7fdc434e9740 1 Processor -- start 2024-09-19T09:15:04.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.841+0000 7fdc434e9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fb9e41b60 0x560fb9e49bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:04.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.841+0000 7fdc434e9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fb9e42480 0x560fb9e4a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:04.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.841+0000 7fdc434e9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fb9e4cd40 0x560fb9e4a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:04.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.841+0000 7fdc434e9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560fb9e53d20 con 0x560fb9e4cd40 2024-09-19T09:15:04.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.841+0000 7fdc434e9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560fb9d61f10 con 0x560fb9e42480 2024-09-19T09:15:04.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.841+0000 7fdc434e9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560fb9e53ea0 con 0x560fb9e41b60 2024-09-19T09:15:04.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.841+0000 7fdc434e9740 10 client.0 _ll_register_callbacks cb 0x560fb9d2aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:04.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.841+0000 7fdc3b59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fb9e4cd40 0x560fb9e4a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:04.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.841+0000 7fdc3ad9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fb9e41b60 0x560fb9e49bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:04.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.841+0000 7fdc3b59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fb9e4cd40 0x560fb9e4a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51652/0 (socket says 172.21.15.120:51652) 2024-09-19T09:15:04.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.841+0000 7fdc3b59e640 1 -- 172.21.15.120:0/2643355065 learned_addr learned my addr 172.21.15.120:0/2643355065 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:04.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.841+0000 7fdc3a59c640 1 --2- 172.21.15.120:0/2643355065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fb9e42480 0x560fb9e4a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:04.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.842+0000 7fdc3b59e640 1 -- 172.21.15.120:0/2643355065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fb9e41b60 msgr2=0x560fb9e49bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:04.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.842+0000 7fdc3b59e640 1 --2- 172.21.15.120:0/2643355065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fb9e41b60 0x560fb9e49bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.842+0000 7fdc3b59e640 1 -- 172.21.15.120:0/2643355065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fb9e42480 msgr2=0x560fb9e4a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:04.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.842+0000 7fdc3b59e640 1 --2- 172.21.15.120:0/2643355065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fb9e42480 0x560fb9e4a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.842+0000 7fdc3b59e640 1 -- 172.21.15.120:0/2643355065 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fb9b186a0 con 0x560fb9e4cd40 2024-09-19T09:15:04.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.842+0000 7fdc3ad9d640 1 --2- 172.21.15.120:0/2643355065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fb9e41b60 0x560fb9e49bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:15:04.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.842+0000 7fdc3b59e640 1 --2- 172.21.15.120:0/2643355065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fb9e4cd40 0x560fb9e4a640 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fdc3401a4b0 tx=0x7fdc34019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3ca4daa5e5d41a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:04.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.842+0000 7fdc1eff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:15:04.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.842+0000 7fdc1eff5640 1 -- 172.21.15.120:0/2643355065 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc34034060 con 0x560fb9e4cd40 2024-09-19T09:15:04.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.842+0000 7fdc434e9740 1 -- 172.21.15.120:0/2643355065 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc34010e30 con 0x560fb9e4cd40 2024-09-19T09:15:04.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.842+0000 7fdc1eff5640 1 -- 172.21.15.120:0/2643355065 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fdc34009a50 con 0x560fb9e4cd40 2024-09-19T09:15:04.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.842+0000 7fdc1eff5640 1 -- 172.21.15.120:0/2643355065 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdc34019da0 con 0x560fb9e4cd40 2024-09-19T09:15:04.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.842+0000 7fdc434e9740 10 client.19755 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:04.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.842+0000 7fdc434e9740 1 -- 172.21.15.120:0/2643355065 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560fb9e5f8d0 con 0x560fb9e4cd40 2024-09-19T09:15:04.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.842+0000 7fdc1eff5640 1 -- 172.21.15.120:0/2643355065 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc34010e30 con 0x560fb9e4cd40 2024-09-19T09:15:04.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.843+0000 7fdc1eff5640 1 -- 172.21.15.120:0/2643355065 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fdc34039070 con 0x560fb9e4cd40 2024-09-19T09:15:04.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.843+0000 7fdc434e9740 10 client.19755 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:04.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.843+0000 7fdc434e9740 10 client.19755 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:15:04.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.843+0000 7fdc434e9740 20 client.19755 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:04.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.843+0000 7fdc434e9740 1 --2- 172.21.15.120:0/2643355065 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560fb9e60d30 0x560fb9e63120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:04.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.843+0000 7fdc434e9740 4 client.19755 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:15:04.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.843+0000 7fdc434e9740 1 -- 172.21.15.120:0/2643355065 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fb9db47f0 con 0x560fb9e60d30 2024-09-19T09:15:04.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.843+0000 7fdc3ad9d640 1 --2- 172.21.15.120:0/2643355065 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560fb9e60d30 0x560fb9e63120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:04.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.843+0000 7fdc3ad9d640 1 --2- 172.21.15.120:0/2643355065 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560fb9e60d30 0x560fb9e63120 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7dc294faecb5f72 server_cookie=bfc60976461931a8 in_seq=0 out_seq=0 2024-09-19T09:15:04.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.843+0000 7fdc1eff5640 10 client.19755 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:04.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.844+0000 7fdc1eff5640 1 -- 172.21.15.120:0/2643355065 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560fb9db47f0 con 0x560fb9e60d30 2024-09-19T09:15:04.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.844+0000 7fdc1eff5640 10 client.19755 handle_command_reply: tid=0 2024-09-19T09:15:04.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.844+0000 7fdc434e9740 1 client.19755 shutdown 2024-09-19T09:15:04.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.844+0000 7fdc434e9740 1 -- 172.21.15.120:0/2643355065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fb9e4cd40 msgr2=0x560fb9e4a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:04.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.844+0000 7fdc434e9740 1 --2- 172.21.15.120:0/2643355065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fb9e4cd40 0x560fb9e4a640 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fdc3401a4b0 tx=0x7fdc34019810 comp rx=0 tx=0).stop 2024-09-19T09:15:04.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.844+0000 7fdc434e9740 1 -- 172.21.15.120:0/2643355065 shutdown_connections 2024-09-19T09:15:04.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.844+0000 7fdc434e9740 1 --2- 172.21.15.120:0/2643355065 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560fb9e60d30 0x560fb9e63120 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.844+0000 7fdc434e9740 1 --2- 172.21.15.120:0/2643355065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fb9e4cd40 0x560fb9e4a640 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.844+0000 7fdc434e9740 1 --2- 172.21.15.120:0/2643355065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fb9e42480 0x560fb9e4a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.844+0000 7fdc434e9740 1 --2- 172.21.15.120:0/2643355065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fb9e41b60 0x560fb9e49bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.844+0000 7fdc434e9740 1 -- 172.21.15.120:0/2643355065 >> 172.21.15.120:0/2643355065 conn(0x560fb9e3e0f0 msgr2=0x560fb9e3d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:04.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.844+0000 7fdc1eff5640 0 client.19755 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:04.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.844+0000 7fdc434e9740 1 -- 172.21.15.120:0/2643355065 shutdown_connections 2024-09-19T09:15:04.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.844+0000 7fdc434e9740 1 -- 172.21.15.120:0/2643355065 wait complete. 2024-09-19T09:15:04.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.844+0000 7fdc434e9740 20 client.19755 trim_cache size 0 max 16384 2024-09-19T09:15:04.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.863+0000 7fdc434e9740 1 Processor -- start 2024-09-19T09:15:04.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.863+0000 7fdc434e9740 1 -- start start 2024-09-19T09:15:04.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.863+0000 7fdc434e9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fb9e41b60 0x560fb9e445c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:04.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.863+0000 7fdc434e9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fb9e42480 0x560fb9e68780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:04.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.863+0000 7fdc434e9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fb9e4cd40 0x560fb9e68cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:04.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.863+0000 7fdc434e9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3c1ab4a0 con 0x560fb9e42480 2024-09-19T09:15:04.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.863+0000 7fdc434e9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3c1acfe0 con 0x560fb9e4cd40 2024-09-19T09:15:04.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.863+0000 7fdc434e9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3c1abb70 con 0x560fb9e41b60 2024-09-19T09:15:04.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.863+0000 7fdc3a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fb9e42480 0x560fb9e68780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:04.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.863+0000 7fdc3ad9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fb9e41b60 0x560fb9e445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:04.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.863+0000 7fdc3a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fb9e42480 0x560fb9e68780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51662/0 (socket says 172.21.15.120:51662) 2024-09-19T09:15:04.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.863+0000 7fdc3a59c640 1 -- 172.21.15.120:0/592621197 learned_addr learned my addr 172.21.15.120:0/592621197 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:04.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.863+0000 7fdc3b59e640 1 --2- 172.21.15.120:0/592621197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fb9e4cd40 0x560fb9e68cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:04.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.863+0000 7fdc3a59c640 1 -- 172.21.15.120:0/592621197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fb9e41b60 msgr2=0x560fb9e445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:04.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.863+0000 7fdc3a59c640 1 --2- 172.21.15.120:0/592621197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fb9e41b60 0x560fb9e445c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.863+0000 7fdc3a59c640 1 -- 172.21.15.120:0/592621197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fb9e4cd40 msgr2=0x560fb9e68cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:04.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.863+0000 7fdc3a59c640 1 --2- 172.21.15.120:0/592621197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fb9e4cd40 0x560fb9e68cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.863+0000 7fdc3a59c640 1 -- 172.21.15.120:0/592621197 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fb9e53e50 con 0x560fb9e42480 2024-09-19T09:15:04.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.864+0000 7fdc3ad9d640 1 --2- 172.21.15.120:0/592621197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fb9e41b60 0x560fb9e445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:15:04.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.864+0000 7fdc3a59c640 1 --2- 172.21.15.120:0/592621197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fb9e42480 0x560fb9e68780 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fdc3009b740 tx=0x7fdc3001a900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc7a8d8db20492a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:04.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.864+0000 7fdc20ff9640 1 -- 172.21.15.120:0/592621197 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3009ba40 con 0x560fb9e42480 2024-09-19T09:15:04.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.864+0000 7fdc20ff9640 1 -- 172.21.15.120:0/592621197 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc3009fdf0 con 0x560fb9e42480 2024-09-19T09:15:04.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.864+0000 7fdc20ff9640 1 -- 172.21.15.120:0/592621197 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc300a4480 con 0x560fb9e42480 2024-09-19T09:15:04.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.865+0000 7fdc434e9740 1 -- 172.21.15.120:0/592621197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fb9e42480 msgr2=0x560fb9e68780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:04.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.865+0000 7fdc434e9740 1 --2- 172.21.15.120:0/592621197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fb9e42480 0x560fb9e68780 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fdc3009b740 tx=0x7fdc3001a900 comp rx=0 tx=0).stop 2024-09-19T09:15:04.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.865+0000 7fdc434e9740 1 -- 172.21.15.120:0/592621197 shutdown_connections 2024-09-19T09:15:04.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.865+0000 7fdc434e9740 1 --2- 172.21.15.120:0/592621197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fb9e4cd40 0x560fb9e68cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.865+0000 7fdc434e9740 1 --2- 172.21.15.120:0/592621197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fb9e42480 0x560fb9e68780 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.865+0000 7fdc434e9740 1 --2- 172.21.15.120:0/592621197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fb9e41b60 0x560fb9e445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.865+0000 7fdc434e9740 1 -- 172.21.15.120:0/592621197 >> 172.21.15.120:0/592621197 conn(0x560fb9e3e0f0 msgr2=0x560fb9e4bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:04.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.865+0000 7fdc434e9740 1 -- 172.21.15.120:0/592621197 shutdown_connections 2024-09-19T09:15:04.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.865+0000 7fdc434e9740 1 -- 172.21.15.120:0/592621197 wait complete. 2024-09-19T09:15:04.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.865+0000 7fdc434e9740 1 -- start start 2024-09-19T09:15:04.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.865+0000 7fdc434e9740 1 Processor -- start 2024-09-19T09:15:04.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.865+0000 7fdc434e9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fb9e41b60 0x560fb9e592d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:04.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.865+0000 7fdc434e9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fb9e42480 0x560fb9e59810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:04.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.865+0000 7fdc434e9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fb9e4cd40 0x560fb9e463d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:04.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.865+0000 7fdc434e9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3c1ac910 con 0x560fb9e42480 2024-09-19T09:15:04.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.865+0000 7fdc434e9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3c1ac240 con 0x560fb9e4cd40 2024-09-19T09:15:04.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.865+0000 7fdc434e9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3c1a9480 con 0x560fb9e41b60 2024-09-19T09:15:04.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.865+0000 7fdc434e9740 10 client.0 _ll_register_callbacks cb 0x7fdc3401d020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:04.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.866+0000 7fdc3a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fb9e42480 0x560fb9e59810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:04.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.866+0000 7fdc3a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fb9e42480 0x560fb9e59810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51670/0 (socket says 172.21.15.120:51670) 2024-09-19T09:15:04.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.866+0000 7fdc3a59c640 1 -- 172.21.15.120:0/1451531598 learned_addr learned my addr 172.21.15.120:0/1451531598 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:04.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.866+0000 7fdc3ad9d640 1 --2- 172.21.15.120:0/1451531598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fb9e41b60 0x560fb9e592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:04.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.866+0000 7fdc3b59e640 1 --2- 172.21.15.120:0/1451531598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fb9e4cd40 0x560fb9e463d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:04.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.866+0000 7fdc3a59c640 1 -- 172.21.15.120:0/1451531598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fb9e41b60 msgr2=0x560fb9e592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:04.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.866+0000 7fdc3a59c640 1 --2- 172.21.15.120:0/1451531598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fb9e41b60 0x560fb9e592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.866+0000 7fdc3a59c640 1 -- 172.21.15.120:0/1451531598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fb9e4cd40 msgr2=0x560fb9e463d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:04.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.866+0000 7fdc3a59c640 1 --2- 172.21.15.120:0/1451531598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fb9e4cd40 0x560fb9e463d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.866+0000 7fdc3a59c640 1 -- 172.21.15.120:0/1451531598 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fb9e46ac0 con 0x560fb9e42480 2024-09-19T09:15:04.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.866+0000 7fdc3a59c640 1 --2- 172.21.15.120:0/1451531598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fb9e42480 0x560fb9e59810 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fdc3009b9d0 tx=0x7fdc30098290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4feeeca75e2b7782 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:04.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.866+0000 7fdc1eff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:15:04.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.866+0000 7fdc1eff5640 1 -- 172.21.15.120:0/1451531598 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3009ba40 con 0x560fb9e42480 2024-09-19T09:15:04.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.866+0000 7fdc1eff5640 1 -- 172.21.15.120:0/1451531598 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fdc300cd030 con 0x560fb9e42480 2024-09-19T09:15:04.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.866+0000 7fdc434e9740 1 -- 172.21.15.120:0/1451531598 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc3009fdf0 con 0x560fb9e42480 2024-09-19T09:15:04.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.866+0000 7fdc1eff5640 1 -- 172.21.15.120:0/1451531598 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdc300ba170 con 0x560fb9e42480 2024-09-19T09:15:04.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.866+0000 7fdc434e9740 10 client.19761 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:04.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.866+0000 7fdc434e9740 1 -- 172.21.15.120:0/1451531598 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560fb9ecb170 con 0x560fb9e42480 2024-09-19T09:15:04.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.867+0000 7fdc1eff5640 1 -- 172.21.15.120:0/1451531598 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc3009fdf0 con 0x560fb9e42480 2024-09-19T09:15:04.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.867+0000 7fdc1eff5640 1 -- 172.21.15.120:0/1451531598 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fdc30097790 con 0x560fb9e42480 2024-09-19T09:15:04.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.867+0000 7fdc434e9740 10 client.19761 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:04.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.867+0000 7fdc434e9740 10 client.19761 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:15:04.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.867+0000 7fdc434e9740 20 client.19761 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:04.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.867+0000 7fdc434e9740 1 --2- 172.21.15.120:0/1451531598 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560fb9ecb7b0 0x560fb9ecdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:04.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.867+0000 7fdc434e9740 4 client.19761 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:15:04.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.867+0000 7fdc434e9740 1 -- 172.21.15.120:0/1451531598 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560fb9daed00 con 0x560fb9ecb7b0 2024-09-19T09:15:04.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.867+0000 7fdc3ad9d640 1 --2- 172.21.15.120:0/1451531598 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560fb9ecb7b0 0x560fb9ecdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:04.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.867+0000 7fdc1eff5640 10 client.19761 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:04.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.867+0000 7fdc3ad9d640 1 --2- 172.21.15.120:0/1451531598 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560fb9ecb7b0 0x560fb9ecdbd0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c164d991827faf6c server_cookie=64b58e09042b5926 in_seq=0 out_seq=0 2024-09-19T09:15:04.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.868+0000 7fdc1eff5640 1 -- 172.21.15.120:0/1451531598 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560fb9daed00 con 0x560fb9ecb7b0 2024-09-19T09:15:04.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.868+0000 7fdc1eff5640 10 client.19761 handle_command_reply: tid=0 2024-09-19T09:15:04.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.868+0000 7fdc434e9740 1 client.19761 shutdown 2024-09-19T09:15:04.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.869+0000 7fdc434e9740 1 -- 172.21.15.120:0/1451531598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fb9e42480 msgr2=0x560fb9e59810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:04.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.869+0000 7fdc434e9740 1 --2- 172.21.15.120:0/1451531598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fb9e42480 0x560fb9e59810 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fdc3009b9d0 tx=0x7fdc30098290 comp rx=0 tx=0).stop 2024-09-19T09:15:04.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.869+0000 7fdc434e9740 1 -- 172.21.15.120:0/1451531598 shutdown_connections 2024-09-19T09:15:04.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.869+0000 7fdc434e9740 1 --2- 172.21.15.120:0/1451531598 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560fb9ecb7b0 0x560fb9ecdbd0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.869+0000 7fdc434e9740 1 --2- 172.21.15.120:0/1451531598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560fb9e4cd40 0x560fb9e463d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.869+0000 7fdc434e9740 1 --2- 172.21.15.120:0/1451531598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560fb9e42480 0x560fb9e59810 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.869+0000 7fdc434e9740 1 --2- 172.21.15.120:0/1451531598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fb9e41b60 0x560fb9e592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.869+0000 7fdc434e9740 1 -- 172.21.15.120:0/1451531598 >> 172.21.15.120:0/1451531598 conn(0x560fb9e3e0f0 msgr2=0x560fb9e49b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:04.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.869+0000 7fdc1eff5640 0 client.19761 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:04.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.869+0000 7fdc434e9740 1 -- 172.21.15.120:0/1451531598 shutdown_connections 2024-09-19T09:15:04.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.869+0000 7fdc434e9740 1 -- 172.21.15.120:0/1451531598 wait complete. 2024-09-19T09:15:04.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.869+0000 7fdc434e9740 20 client.19761 trim_cache size 0 max 16384 2024-09-19T09:15:04.876 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:15:04.876 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:15:04.876 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "eeb47af6-4c74-481d-95ce-743485e303b4", 2024-09-19T09:15:04.876 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:15:04.876 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:15:04.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.870+0000 7fdc41110640 1 -- 172.21.15.120:0/1485188899 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdc240a3240 msgr2=0x7fdc240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:04.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.870+0000 7fdc41110640 1 --2- 172.21.15.120:0/1485188899 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdc240a3240 0x7fdc240a5700 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdc3400f010 tx=0x7fdc34021000 comp rx=0 tx=0).stop 2024-09-19T09:15:04.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.870+0000 7fdc41110640 1 -- 172.21.15.120:0/1485188899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc3c102570 msgr2=0x7fdc3c198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:04.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.870+0000 7fdc41110640 1 --2- 172.21.15.120:0/1485188899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc3c102570 0x7fdc3c198ac0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000b450 tx=0x7fdc3000b910 comp rx=0 tx=0).stop 2024-09-19T09:15:04.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.870+0000 7fdc41110640 1 -- 172.21.15.120:0/1485188899 shutdown_connections 2024-09-19T09:15:04.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.870+0000 7fdc41110640 1 --2- 172.21.15.120:0/1485188899 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdc240a3240 0x7fdc240a5700 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.870+0000 7fdc41110640 1 --2- 172.21.15.120:0/1485188899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc3c102ef0 0x7fdc3c19ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.870+0000 7fdc41110640 1 --2- 172.21.15.120:0/1485188899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc3c102570 0x7fdc3c198ac0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.870+0000 7fdc41110640 1 --2- 172.21.15.120:0/1485188899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc3c100e20 0x7fdc3c198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:04.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.870+0000 7fdc41110640 1 -- 172.21.15.120:0/1485188899 >> 172.21.15.120:0/1485188899 conn(0x7fdc3c0fc7d0 msgr2=0x7fdc3c069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:04.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.870+0000 7fdc41110640 1 -- 172.21.15.120:0/1485188899 shutdown_connections 2024-09-19T09:15:04.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:04.870+0000 7fdc41110640 1 -- 172.21.15.120:0/1485188899 wait complete. 2024-09-19T09:15:05.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:04 smithi120 ceph-mon[31150]: pgmap v710: 97 pgs: 97 active+clean; 47 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.0 MiB/s wr, 420 op/s 2024-09-19T09:15:05.075 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1200847 9.78 MB/sec execute 540 sec latency 30.944 ms 2024-09-19T09:15:05.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:04 smithi196 ceph-mon[35150]: pgmap v710: 97 pgs: 97 active+clean; 47 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.0 MiB/s wr, 420 op/s 2024-09-19T09:15:05.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:04 smithi153 ceph-mon[35071]: pgmap v710: 97 pgs: 97 active+clean; 47 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.0 MiB/s wr, 420 op/s 2024-09-19T09:15:05.631 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:15:05.905 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:15:06.076 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1203667 9.79 MB/sec execute 541 sec latency 11.418 ms 2024-09-19T09:15:06.182 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:05 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:15:06.182 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:05 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:15:06.183 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:05 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:15:06.183 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:05 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:15:06.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:05 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:15:06.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:05 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:15:06.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:05 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:15:06.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:05 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:15:06.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:05 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:15:06.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:05 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:15:06.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:05 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:15:06.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:05 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:15:06.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.416+0000 7f1ade030640 1 -- 172.21.15.120:0/334066714 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ad8102790 msgr2=0x7f1ad81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:06.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.416+0000 7f1ade030640 1 --2- 172.21.15.120:0/334066714 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ad8102790 0x7f1ad81067e0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009900 tx=0x7f1acc02a410 comp rx=0 tx=0).stop 2024-09-19T09:15:06.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.416+0000 7f1ade030640 1 -- 172.21.15.120:0/334066714 shutdown_connections 2024-09-19T09:15:06.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.416+0000 7f1ade030640 1 --2- 172.21.15.120:0/334066714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ad8106f10 0x7f1ad810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.416+0000 7f1ade030640 1 --2- 172.21.15.120:0/334066714 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ad8102790 0x7f1ad81067e0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.417+0000 7f1ade030640 1 --2- 172.21.15.120:0/334066714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ad8101de0 0x7f1ad81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.417+0000 7f1ade030640 1 -- 172.21.15.120:0/334066714 >> 172.21.15.120:0/334066714 conn(0x7f1ad80fd8c0 msgr2=0x7f1ad80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:06.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.417+0000 7f1ade030640 1 -- 172.21.15.120:0/334066714 shutdown_connections 2024-09-19T09:15:06.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.417+0000 7f1ade030640 1 -- 172.21.15.120:0/334066714 wait complete. 2024-09-19T09:15:06.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.417+0000 7f1ade030640 1 Processor -- start 2024-09-19T09:15:06.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.417+0000 7f1ade030640 1 -- start start 2024-09-19T09:15:06.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.418+0000 7f1ade030640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ad8101de0 0x7f1ad81984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:06.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.418+0000 7f1ade030640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ad8102790 0x7f1ad8198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:06.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.418+0000 7f1ade030640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ad8106f10 0x7f1ad819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:06.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.418+0000 7f1ade030640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad8111fb0 con 0x7f1ad8106f10 2024-09-19T09:15:06.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.418+0000 7f1ade030640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad8111e30 con 0x7f1ad8101de0 2024-09-19T09:15:06.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.418+0000 7f1ade030640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad8112130 con 0x7f1ad8102790 2024-09-19T09:15:06.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.418+0000 7f1ad7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ad8106f10 0x7f1ad819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:06.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.418+0000 7f1ad7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ad8106f10 0x7f1ad819cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51684/0 (socket says 172.21.15.120:51684) 2024-09-19T09:15:06.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.418+0000 7f1ad7fff640 1 -- 172.21.15.120:0/667868921 learned_addr learned my addr 172.21.15.120:0/667868921 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:06.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.418+0000 7f1ad77fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ad8101de0 0x7f1ad81984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:06.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.418+0000 7f1ad6ffd640 1 --2- 172.21.15.120:0/667868921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ad8102790 0x7f1ad8198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:06.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.418+0000 7f1ad7fff640 1 -- 172.21.15.120:0/667868921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ad8102790 msgr2=0x7f1ad8198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:06.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.418+0000 7f1ad7fff640 1 --2- 172.21.15.120:0/667868921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ad8102790 0x7f1ad8198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.418+0000 7f1ad7fff640 1 -- 172.21.15.120:0/667868921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ad8101de0 msgr2=0x7f1ad81984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:06.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.418+0000 7f1ad7fff640 1 --2- 172.21.15.120:0/667868921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ad8101de0 0x7f1ad81984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.418+0000 7f1ad7fff640 1 -- 172.21.15.120:0/667868921 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad819d480 con 0x7f1ad8106f10 2024-09-19T09:15:06.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.418+0000 7f1ad7fff640 1 --2- 172.21.15.120:0/667868921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ad8106f10 0x7f1ad819cda0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000b990 tx=0x7f1ad000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:06.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.418+0000 7f1ad4ff9640 1 -- 172.21.15.120:0/667868921 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0019070 con 0x7f1ad8106f10 2024-09-19T09:15:06.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.418+0000 7f1ade030640 1 -- 172.21.15.120:0/667868921 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ad819d770 con 0x7f1ad8106f10 2024-09-19T09:15:06.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.418+0000 7f1ad4ff9640 1 -- 172.21.15.120:0/667868921 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ad000cba0 con 0x7f1ad8106f10 2024-09-19T09:15:06.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.418+0000 7f1ad4ff9640 1 -- 172.21.15.120:0/667868921 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0007610 con 0x7f1ad8106f10 2024-09-19T09:15:06.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.418+0000 7f1ade030640 1 -- 172.21.15.120:0/667868921 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ad81a5050 con 0x7f1ad8106f10 2024-09-19T09:15:06.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.421+0000 7f1ad4ff9640 1 -- 172.21.15.120:0/667868921 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f1ad001c070 con 0x7f1ad8106f10 2024-09-19T09:15:06.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.421+0000 7f1ae0409740 1 Processor -- start 2024-09-19T09:15:06.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.421+0000 7f1ae0409740 1 -- start start 2024-09-19T09:15:06.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.421+0000 7f1ae0409740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d666f804a0 0x55d666f80880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:06.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.421+0000 7f1ae0409740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d666f80dc0 0x55d666f8b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:06.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.421+0000 7f1ad4ff9640 1 --2- 172.21.15.120:0/667868921 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1ac00a3240 0x7f1ac00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:06.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.421+0000 7f1ae0409740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d666f8b680 0x55d666f8da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:06.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.421+0000 7f1ae0409740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d666ee62e0 con 0x55d666f8b680 2024-09-19T09:15:06.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.421+0000 7f1ae0409740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d666e8dce0 con 0x55d666f80dc0 2024-09-19T09:15:06.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.421+0000 7f1ae0409740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d666ee22a0 con 0x55d666f804a0 2024-09-19T09:15:06.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.421+0000 7f1ad77fe640 1 --2- 172.21.15.120:0/667868921 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1ac00a3240 0x7f1ac00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:06.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.421+0000 7f1ad4ff9640 1 -- 172.21.15.120:0/667868921 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f1ad00ceea0 con 0x7f1ad8106f10 2024-09-19T09:15:06.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.421+0000 7f1ad7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d666f8b680 0x55d666f8da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:06.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.421+0000 7f1ad6ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d666f80dc0 0x55d666f8b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:06.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.421+0000 7f1ad7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d666f8b680 0x55d666f8da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51698/0 (socket says 172.21.15.120:51698) 2024-09-19T09:15:06.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.422+0000 7f1ad7fff640 1 -- 172.21.15.120:0/672841888 learned_addr learned my addr 172.21.15.120:0/672841888 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:06.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.422+0000 7f1ad6ffd640 1 --2- 172.21.15.120:0/672841888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d666f804a0 0x55d666f80880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:06.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.422+0000 7f1ad77fe640 1 --2- 172.21.15.120:0/667868921 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1ac00a3240 0x7f1ac00a5700 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8009bd0 tx=0x7f1ac8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:06.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.422+0000 7f1ad6ffd640 1 -- 172.21.15.120:0/672841888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d666f804a0 msgr2=0x55d666f80880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:06.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.422+0000 7f1ad6ffd640 1 --2- 172.21.15.120:0/672841888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d666f804a0 0x55d666f80880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.422+0000 7f1ad6ffd640 1 -- 172.21.15.120:0/672841888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d666f8b680 msgr2=0x55d666f8da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:06.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.422+0000 7f1ad6ffd640 1 --2- 172.21.15.120:0/672841888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d666f8b680 0x55d666f8da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.422+0000 7f1ad6ffd640 1 -- 172.21.15.120:0/672841888 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d666e72440 con 0x55d666f80dc0 2024-09-19T09:15:06.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.422+0000 7f1ad6ffd640 1 --2- 172.21.15.120:0/672841888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d666f80dc0 0x55d666f8b140 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009230 tx=0x7f1acc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99858e4bb86eb0b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:06.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.422+0000 7f1abdffb640 1 -- 172.21.15.120:0/672841888 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc035040 con 0x55d666f80dc0 2024-09-19T09:15:06.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.422+0000 7f1abdffb640 1 -- 172.21.15.120:0/672841888 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1acc0319d0 con 0x55d666f80dc0 2024-09-19T09:15:06.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.422+0000 7f1abdffb640 1 -- 172.21.15.120:0/672841888 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc003d80 con 0x55d666f80dc0 2024-09-19T09:15:06.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.423+0000 7f1ae0409740 1 -- 172.21.15.120:0/672841888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d666f80dc0 msgr2=0x55d666f8b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:06.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.423+0000 7f1ae0409740 1 --2- 172.21.15.120:0/672841888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d666f80dc0 0x55d666f8b140 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009230 tx=0x7f1acc009260 comp rx=0 tx=0).stop 2024-09-19T09:15:06.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.423+0000 7f1ae0409740 1 -- 172.21.15.120:0/672841888 shutdown_connections 2024-09-19T09:15:06.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.423+0000 7f1ae0409740 1 --2- 172.21.15.120:0/672841888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d666f8b680 0x55d666f8da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.423+0000 7f1ae0409740 1 --2- 172.21.15.120:0/672841888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d666f80dc0 0x55d666f8b140 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.423+0000 7f1ae0409740 1 --2- 172.21.15.120:0/672841888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d666f804a0 0x55d666f80880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.423+0000 7f1ae0409740 1 -- 172.21.15.120:0/672841888 >> 172.21.15.120:0/672841888 conn(0x55d666f7ca30 msgr2=0x55d666f7ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:06.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.423+0000 7f1ae0409740 1 -- 172.21.15.120:0/672841888 shutdown_connections 2024-09-19T09:15:06.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.423+0000 7f1ae0409740 1 -- 172.21.15.120:0/672841888 wait complete. 2024-09-19T09:15:06.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.424+0000 7f1ae0409740 1 -- start start 2024-09-19T09:15:06.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.424+0000 7f1ae0409740 1 Processor -- start 2024-09-19T09:15:06.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.424+0000 7f1ae0409740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d666f804a0 0x55d666f8b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:06.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.424+0000 7f1ae0409740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d666f80dc0 0x55d666f88310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:06.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.424+0000 7f1ae0409740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d666f8b680 0x55d666f888e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:06.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.424+0000 7f1ae0409740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d666f8e250 con 0x55d666f80dc0 2024-09-19T09:15:06.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.424+0000 7f1ae0409740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d666ea1050 con 0x55d666f8b680 2024-09-19T09:15:06.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.424+0000 7f1ae0409740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d666f8e3d0 con 0x55d666f804a0 2024-09-19T09:15:06.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.424+0000 7f1ae0409740 10 client.0 _ll_register_callbacks cb 0x55d666ee1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:06.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.424+0000 7f1ad6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d666f80dc0 0x55d666f88310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:06.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.425+0000 7f1ad6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d666f80dc0 0x55d666f88310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51714/0 (socket says 172.21.15.120:51714) 2024-09-19T09:15:06.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.425+0000 7f1ad77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d666f804a0 0x55d666f8b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:06.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.425+0000 7f1ad6ffd640 1 -- 172.21.15.120:0/3845129288 learned_addr learned my addr 172.21.15.120:0/3845129288 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:06.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.425+0000 7f1ad7fff640 1 --2- 172.21.15.120:0/3845129288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d666f8b680 0x55d666f888e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:06.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.425+0000 7f1ad77fe640 1 -- 172.21.15.120:0/3845129288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d666f8b680 msgr2=0x55d666f888e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:06.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.425+0000 7f1ad77fe640 1 --2- 172.21.15.120:0/3845129288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d666f8b680 0x55d666f888e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.425+0000 7f1ad77fe640 1 -- 172.21.15.120:0/3845129288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d666f80dc0 msgr2=0x55d666f88310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:06.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.425+0000 7f1ad77fe640 1 --2- 172.21.15.120:0/3845129288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d666f80dc0 0x55d666f88310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.425+0000 7f1ad77fe640 1 -- 172.21.15.120:0/3845129288 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d666c573c0 con 0x55d666f804a0 2024-09-19T09:15:06.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.425+0000 7f1ad6ffd640 1 --2- 172.21.15.120:0/3845129288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d666f80dc0 0x55d666f88310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:15:06.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.425+0000 7f1ad77fe640 1 --2- 172.21.15.120:0/3845129288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d666f804a0 0x55d666f8b060 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800ec10 tx=0x7f1ac8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55bdcf856a10aba3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:06.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.425+0000 7f1ab6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:15:06.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.425+0000 7f1ab6ffd640 1 -- 172.21.15.120:0/3845129288 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac801caa0 con 0x55d666f804a0 2024-09-19T09:15:06.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.425+0000 7f1ab6ffd640 1 -- 172.21.15.120:0/3845129288 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f1ac801cc40 con 0x55d666f804a0 2024-09-19T09:15:06.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.425+0000 7f1ae0409740 1 -- 172.21.15.120:0/3845129288 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1acc0319d0 con 0x55d666f804a0 2024-09-19T09:15:06.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.425+0000 7f1ab6ffd640 1 -- 172.21.15.120:0/3845129288 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1ac801ce40 con 0x55d666f804a0 2024-09-19T09:15:06.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.425+0000 7f1ae0409740 10 client.28615 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:06.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.425+0000 7f1ae0409740 1 -- 172.21.15.120:0/3845129288 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d666f99cf0 con 0x55d666f804a0 2024-09-19T09:15:06.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.426+0000 7f1ab6ffd640 1 -- 172.21.15.120:0/3845129288 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1acc0319d0 con 0x55d666f804a0 2024-09-19T09:15:06.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.426+0000 7f1ab6ffd640 1 -- 172.21.15.120:0/3845129288 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f1ac801d740 con 0x55d666f804a0 2024-09-19T09:15:06.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.426+0000 7f1ae0409740 10 client.28615 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:06.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.426+0000 7f1ae0409740 10 client.28615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:15:06.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.426+0000 7f1ae0409740 20 client.28615 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:06.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.426+0000 7f1ae0409740 1 --2- 172.21.15.120:0/3845129288 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d666f9b1b0 0x55d666f9d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:06.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.426+0000 7f1ae0409740 4 client.28615 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:15:06.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.426+0000 7f1ae0409740 1 -- 172.21.15.120:0/3845129288 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d666ef3850 con 0x55d666f9b1b0 2024-09-19T09:15:06.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.426+0000 7f1ad6ffd640 1 --2- 172.21.15.120:0/3845129288 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d666f9b1b0 0x55d666f9d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:06.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.426+0000 7f1ab6ffd640 10 client.28615 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:06.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.426+0000 7f1ad6ffd640 1 --2- 172.21.15.120:0/3845129288 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d666f9b1b0 0x55d666f9d5a0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e304bdff0cbfc8e server_cookie=673f72754feaa6f8 in_seq=0 out_seq=0 2024-09-19T09:15:06.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.427+0000 7f1ab6ffd640 1 -- 172.21.15.120:0/3845129288 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d666ef3850 con 0x55d666f9b1b0 2024-09-19T09:15:06.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.427+0000 7f1ab6ffd640 10 client.28615 handle_command_reply: tid=0 2024-09-19T09:15:06.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.427+0000 7f1ae0409740 1 client.28615 shutdown 2024-09-19T09:15:06.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.427+0000 7f1ae0409740 1 -- 172.21.15.120:0/3845129288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d666f804a0 msgr2=0x55d666f8b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:06.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.427+0000 7f1ae0409740 1 --2- 172.21.15.120:0/3845129288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d666f804a0 0x55d666f8b060 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800ec10 tx=0x7f1ac8004730 comp rx=0 tx=0).stop 2024-09-19T09:15:06.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.427+0000 7f1ae0409740 1 -- 172.21.15.120:0/3845129288 shutdown_connections 2024-09-19T09:15:06.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.427+0000 7f1ae0409740 1 --2- 172.21.15.120:0/3845129288 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d666f9b1b0 0x55d666f9d5a0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.427+0000 7f1ae0409740 1 --2- 172.21.15.120:0/3845129288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d666f8b680 0x55d666f888e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.427+0000 7f1ae0409740 1 --2- 172.21.15.120:0/3845129288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d666f80dc0 0x55d666f88310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.427+0000 7f1ae0409740 1 --2- 172.21.15.120:0/3845129288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d666f804a0 0x55d666f8b060 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.427+0000 7f1ae0409740 1 -- 172.21.15.120:0/3845129288 >> 172.21.15.120:0/3845129288 conn(0x55d666f7ca30 msgr2=0x55d666f82710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:06.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.427+0000 7f1ab6ffd640 0 client.28615 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:06.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.427+0000 7f1ae0409740 1 -- 172.21.15.120:0/3845129288 shutdown_connections 2024-09-19T09:15:06.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.427+0000 7f1ae0409740 1 -- 172.21.15.120:0/3845129288 wait complete. 2024-09-19T09:15:06.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.427+0000 7f1ae0409740 20 client.28615 trim_cache size 0 max 16384 2024-09-19T09:15:06.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.445+0000 7f1ae0409740 1 Processor -- start 2024-09-19T09:15:06.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.445+0000 7f1ae0409740 1 -- start start 2024-09-19T09:15:06.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.445+0000 7f1ae0409740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d666f804a0 0x55d666f974b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:06.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.445+0000 7f1ae0409740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d666f80dc0 0x55d666f7ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:06.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.445+0000 7f1ae0409740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d666f8b680 0x55d666fa2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:06.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.445+0000 7f1ae0409740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad81ab510 con 0x55d666f80dc0 2024-09-19T09:15:06.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.445+0000 7f1ae0409740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad81ad050 con 0x55d666f8b680 2024-09-19T09:15:06.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.445+0000 7f1ae0409740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad81abbe0 con 0x55d666f804a0 2024-09-19T09:15:06.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.445+0000 7f1ad77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d666f804a0 0x55d666f974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:06.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.445+0000 7f1ad77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d666f804a0 0x55d666f974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:60084/0 (socket says 172.21.15.120:60084) 2024-09-19T09:15:06.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.445+0000 7f1ad77fe640 1 -- 172.21.15.120:0/492074773 learned_addr learned my addr 172.21.15.120:0/492074773 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:06.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.445+0000 7f1ad6ffd640 1 --2- 172.21.15.120:0/492074773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d666f80dc0 0x55d666f7ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:06.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.445+0000 7f1ad7fff640 1 --2- 172.21.15.120:0/492074773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d666f8b680 0x55d666fa2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:06.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.446+0000 7f1ad77fe640 1 -- 172.21.15.120:0/492074773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d666f8b680 msgr2=0x55d666fa2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:06.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.446+0000 7f1ad77fe640 1 --2- 172.21.15.120:0/492074773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d666f8b680 0x55d666fa2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.446+0000 7f1ad77fe640 1 -- 172.21.15.120:0/492074773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d666f80dc0 msgr2=0x55d666f7ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:06.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.446+0000 7f1ad77fe640 1 --2- 172.21.15.120:0/492074773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d666f80dc0 0x55d666f7ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.446+0000 7f1ad77fe640 1 -- 172.21.15.120:0/492074773 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d666f872c0 con 0x55d666f804a0 2024-09-19T09:15:06.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.446+0000 7f1ad77fe640 1 --2- 172.21.15.120:0/492074773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d666f804a0 0x55d666f974b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1ac801db30 tx=0x7f1ac800e7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a20a1fa2051fbe3b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:06.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.446+0000 7f1abd7fa640 1 -- 172.21.15.120:0/492074773 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8004040 con 0x55d666f804a0 2024-09-19T09:15:06.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.446+0000 7f1abd7fa640 1 -- 172.21.15.120:0/492074773 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ac801cc50 con 0x55d666f804a0 2024-09-19T09:15:06.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.446+0000 7f1abd7fa640 1 -- 172.21.15.120:0/492074773 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac80035f0 con 0x55d666f804a0 2024-09-19T09:15:06.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.447+0000 7f1ae0409740 1 -- 172.21.15.120:0/492074773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d666f804a0 msgr2=0x55d666f974b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:06.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.447+0000 7f1ae0409740 1 --2- 172.21.15.120:0/492074773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d666f804a0 0x55d666f974b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1ac801db30 tx=0x7f1ac800e7f0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.447+0000 7f1ae0409740 1 -- 172.21.15.120:0/492074773 shutdown_connections 2024-09-19T09:15:06.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.447+0000 7f1ae0409740 1 --2- 172.21.15.120:0/492074773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d666f8b680 0x55d666fa2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.447+0000 7f1ae0409740 1 --2- 172.21.15.120:0/492074773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d666f80dc0 0x55d666f7ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.447+0000 7f1ae0409740 1 --2- 172.21.15.120:0/492074773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d666f804a0 0x55d666f974b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.447+0000 7f1ae0409740 1 -- 172.21.15.120:0/492074773 >> 172.21.15.120:0/492074773 conn(0x55d666f7ca30 msgr2=0x55d666f838b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:06.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.447+0000 7f1ae0409740 1 -- 172.21.15.120:0/492074773 shutdown_connections 2024-09-19T09:15:06.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.447+0000 7f1ae0409740 1 -- 172.21.15.120:0/492074773 wait complete. 2024-09-19T09:15:06.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.447+0000 7f1ae0409740 1 -- start start 2024-09-19T09:15:06.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.448+0000 7f1ae0409740 1 Processor -- start 2024-09-19T09:15:06.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.448+0000 7f1ae0409740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d666f804a0 0x55d666f93d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:06.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.448+0000 7f1ae0409740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d666f80dc0 0x55d666f8f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:06.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.448+0000 7f1ae0409740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d666f8b680 0x55d666f8f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:06.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.448+0000 7f1ae0409740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad81ac980 con 0x55d666f804a0 2024-09-19T09:15:06.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.448+0000 7f1ae0409740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad81ac2b0 con 0x55d666f8b680 2024-09-19T09:15:06.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.448+0000 7f1ae0409740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad81a94d0 con 0x55d666f80dc0 2024-09-19T09:15:06.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.448+0000 7f1ae0409740 10 client.0 _ll_register_callbacks cb 0x7f1ac8009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:06.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.448+0000 7f1ad7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d666f8b680 0x55d666f8f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:06.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.448+0000 7f1ad77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d666f804a0 0x55d666f93d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:06.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.448+0000 7f1ad6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d666f80dc0 0x55d666f8f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:06.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.448+0000 7f1ad7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d666f8b680 0x55d666f8f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:33802/0 (socket says 172.21.15.120:33802) 2024-09-19T09:15:06.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.448+0000 7f1ad7fff640 1 -- 172.21.15.120:0/1757153440 learned_addr learned my addr 172.21.15.120:0/1757153440 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:06.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.448+0000 7f1ad77fe640 1 -- 172.21.15.120:0/1757153440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d666f80dc0 msgr2=0x55d666f8f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:06.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.448+0000 7f1ad77fe640 1 --2- 172.21.15.120:0/1757153440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d666f80dc0 0x55d666f8f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.448+0000 7f1ad77fe640 1 -- 172.21.15.120:0/1757153440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d666f8b680 msgr2=0x55d666f8f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:06.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.448+0000 7f1ad77fe640 1 --2- 172.21.15.120:0/1757153440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d666f8b680 0x55d666f8f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.449+0000 7f1ad77fe640 1 -- 172.21.15.120:0/1757153440 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d666f8fdc0 con 0x55d666f804a0 2024-09-19T09:15:06.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.449+0000 7f1ad6ffd640 1 --2- 172.21.15.120:0/1757153440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d666f80dc0 0x55d666f8f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:15:06.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.449+0000 7f1ad77fe640 1 --2- 172.21.15.120:0/1757153440 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d666f804a0 0x55d666f93d70 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8016040 tx=0x7f1ac8009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3afa0c3f00ad5b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:06.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.449+0000 7f1ab6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:15:06.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.449+0000 7f1ab6ffd640 1 -- 172.21.15.120:0/1757153440 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac803f070 con 0x55d666f804a0 2024-09-19T09:15:06.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.449+0000 7f1ab6ffd640 1 -- 172.21.15.120:0/1757153440 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f1ac8004040 con 0x55d666f804a0 2024-09-19T09:15:06.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.449+0000 7f1ae0409740 1 -- 172.21.15.120:0/1757153440 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ac801cc50 con 0x55d666f804a0 2024-09-19T09:15:06.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.449+0000 7f1ab6ffd640 1 -- 172.21.15.120:0/1757153440 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1ac801de60 con 0x55d666f804a0 2024-09-19T09:15:06.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.449+0000 7f1ae0409740 10 client.19779 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:06.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.449+0000 7f1ae0409740 1 -- 172.21.15.120:0/1757153440 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d667005e40 con 0x55d666f804a0 2024-09-19T09:15:06.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.449+0000 7f1ab6ffd640 1 -- 172.21.15.120:0/1757153440 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ac801cc50 con 0x55d666f804a0 2024-09-19T09:15:06.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.450+0000 7f1ab6ffd640 1 -- 172.21.15.120:0/1757153440 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f1ac8042050 con 0x55d666f804a0 2024-09-19T09:15:06.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.450+0000 7f1ae0409740 10 client.19779 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:06.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.450+0000 7f1ae0409740 10 client.19779 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:15:06.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.450+0000 7f1ae0409740 20 client.19779 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:06.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.450+0000 7f1ae0409740 1 --2- 172.21.15.120:0/1757153440 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d6670064b0 0x55d6670088a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:06.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.450+0000 7f1ae0409740 4 client.19779 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:15:06.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.450+0000 7f1ae0409740 1 -- 172.21.15.120:0/1757153440 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d666eedd60 con 0x55d6670064b0 2024-09-19T09:15:06.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.450+0000 7f1ad6ffd640 1 --2- 172.21.15.120:0/1757153440 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d6670064b0 0x55d6670088a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:06.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.450+0000 7f1ab6ffd640 10 client.19779 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:06.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.450+0000 7f1ad6ffd640 1 --2- 172.21.15.120:0/1757153440 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d6670064b0 0x55d6670088a0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5fa1e1b77b77de4 server_cookie=17208f46d8bb18e6 in_seq=0 out_seq=0 2024-09-19T09:15:06.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.451+0000 7f1ab6ffd640 1 -- 172.21.15.120:0/1757153440 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d666eedd60 con 0x55d6670064b0 2024-09-19T09:15:06.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.451+0000 7f1ab6ffd640 10 client.19779 handle_command_reply: tid=0 2024-09-19T09:15:06.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.451+0000 7f1ae0409740 1 client.19779 shutdown 2024-09-19T09:15:06.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.451+0000 7f1ae0409740 1 -- 172.21.15.120:0/1757153440 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d666f804a0 msgr2=0x55d666f93d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:06.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.451+0000 7f1ae0409740 1 --2- 172.21.15.120:0/1757153440 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d666f804a0 0x55d666f93d70 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8016040 tx=0x7f1ac8009280 comp rx=0 tx=0).stop 2024-09-19T09:15:06.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.451+0000 7f1ae0409740 1 -- 172.21.15.120:0/1757153440 shutdown_connections 2024-09-19T09:15:06.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.451+0000 7f1ae0409740 1 --2- 172.21.15.120:0/1757153440 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d6670064b0 0x55d6670088a0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.451+0000 7f1ae0409740 1 --2- 172.21.15.120:0/1757153440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d666f8b680 0x55d666f8f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.451+0000 7f1ab6ffd640 0 client.19779 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:06.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.451+0000 7f1ae0409740 1 --2- 172.21.15.120:0/1757153440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d666f80dc0 0x55d666f8f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.451+0000 7f1ae0409740 1 --2- 172.21.15.120:0/1757153440 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d666f804a0 0x55d666f93d70 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.451+0000 7f1ae0409740 1 -- 172.21.15.120:0/1757153440 >> 172.21.15.120:0/1757153440 conn(0x55d666f7ca30 msgr2=0x55d666f7b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:06.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.451+0000 7f1ae0409740 1 -- 172.21.15.120:0/1757153440 shutdown_connections 2024-09-19T09:15:06.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.451+0000 7f1ae0409740 1 -- 172.21.15.120:0/1757153440 wait complete. 2024-09-19T09:15:06.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.451+0000 7f1ae0409740 20 client.19779 trim_cache size 0 max 16384 2024-09-19T09:15:06.459 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:15:06.459 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:15:06.459 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-19T09:15:06.459 INFO:teuthology.orchestra.run.smithi120.stdout: "eeb47af6-4c74-481d-95ce-743485e303b4": { 2024-09-19T09:15:06.459 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-19T09:15:06.459 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "eeb47af6-4c74-481d-95ce-743485e303b4", 2024-09-19T09:15:06.459 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-19T09:15:06.459 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:15:06.460 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:15:06.460 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:15:06.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.452+0000 7f1ade030640 1 -- 172.21.15.120:0/667868921 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1ac00a3240 msgr2=0x7f1ac00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:06.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.452+0000 7f1ade030640 1 --2- 172.21.15.120:0/667868921 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1ac00a3240 0x7f1ac00a5700 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8009bd0 tx=0x7f1ac8005f90 comp rx=0 tx=0).stop 2024-09-19T09:15:06.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.452+0000 7f1ade030640 1 -- 172.21.15.120:0/667868921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ad8106f10 msgr2=0x7f1ad819cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:06.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.452+0000 7f1ade030640 1 --2- 172.21.15.120:0/667868921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ad8106f10 0x7f1ad819cda0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000b990 tx=0x7f1ad000be50 comp rx=0 tx=0).stop 2024-09-19T09:15:06.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.452+0000 7f1ade030640 1 -- 172.21.15.120:0/667868921 shutdown_connections 2024-09-19T09:15:06.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.452+0000 7f1ade030640 1 --2- 172.21.15.120:0/667868921 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1ac00a3240 0x7f1ac00a5700 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.452+0000 7f1ade030640 1 --2- 172.21.15.120:0/667868921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ad8106f10 0x7f1ad819cda0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.452+0000 7f1ade030640 1 --2- 172.21.15.120:0/667868921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ad8102790 0x7f1ad8198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.452+0000 7f1ade030640 1 --2- 172.21.15.120:0/667868921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ad8101de0 0x7f1ad81984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:06.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.452+0000 7f1ade030640 1 -- 172.21.15.120:0/667868921 >> 172.21.15.120:0/667868921 conn(0x7f1ad80fd8c0 msgr2=0x7f1ad80fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:06.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.452+0000 7f1ade030640 1 -- 172.21.15.120:0/667868921 shutdown_connections 2024-09-19T09:15:06.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:06.452+0000 7f1ade030640 1 -- 172.21.15.120:0/667868921 wait complete. 2024-09-19T09:15:07.076 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1206003 9.79 MB/sec execute 542 sec latency 15.903 ms 2024-09-19T09:15:07.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:06 smithi196 ceph-mon[35150]: pgmap v711: 97 pgs: 97 active+clean; 49 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.5 MiB/s wr, 424 op/s 2024-09-19T09:15:07.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:06 smithi153 ceph-mon[35071]: pgmap v711: 97 pgs: 97 active+clean; 49 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.5 MiB/s wr, 424 op/s 2024-09-19T09:15:07.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:06 smithi120 ceph-mon[31150]: pgmap v711: 97 pgs: 97 active+clean; 49 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.5 MiB/s wr, 424 op/s 2024-09-19T09:15:07.546 INFO:tasks.cephfs.filesystem:scrub status for tag:eeb47af6-4c74-481d-95ce-743485e303b4 - {'path': '/', 'tag': 'eeb47af6-4c74-481d-95ce-743485e303b4', 'options': 'recursive,force'} 2024-09-19T09:15:08.076 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1208542 9.80 MB/sec execute 543 sec latency 11.249 ms 2024-09-19T09:15:09.076 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1211352 9.81 MB/sec execute 544 sec latency 73.130 ms 2024-09-19T09:15:09.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:08 smithi196 ceph-mon[35150]: pgmap v712: 97 pgs: 97 active+clean; 49 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.5 MiB/s wr, 404 op/s 2024-09-19T09:15:09.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:08 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:15:09.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:08 smithi153 ceph-mon[35071]: pgmap v712: 97 pgs: 97 active+clean; 49 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.5 MiB/s wr, 404 op/s 2024-09-19T09:15:09.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:08 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:15:09.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:08 smithi120 ceph-mon[31150]: pgmap v712: 97 pgs: 97 active+clean; 49 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.5 MiB/s wr, 404 op/s 2024-09-19T09:15:09.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:08 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:15:10.076 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1214090 9.82 MB/sec execute 545 sec latency 12.555 ms 2024-09-19T09:15:10.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:09 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:15:10.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:09 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:15:10.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:09 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:15:10.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:09 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:15:10.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:09 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:15:10.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:09 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:15:10.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:09 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:15:10.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:09 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:15:10.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:09 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:15:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.894+0000 7f8550c80640 1 -- 172.21.15.120:0/3520588740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f854c102600 msgr2=0x7f854c108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.894+0000 7f8550c80640 1 --2- 172.21.15.120:0/3520588740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f854c102600 0x7f854c108e90 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f8544009900 tx=0x7f854402a410 comp rx=0 tx=0).stop 2024-09-19T09:15:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.894+0000 7f8550c80640 1 -- 172.21.15.120:0/3520588740 shutdown_connections 2024-09-19T09:15:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.894+0000 7f8550c80640 1 --2- 172.21.15.120:0/3520588740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f854c102600 0x7f854c108e90 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.894+0000 7f8550c80640 1 --2- 172.21.15.120:0/3520588740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f854c101c40 0x7f854c1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.894+0000 7f8550c80640 1 --2- 172.21.15.120:0/3520588740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f854c100a50 0x7f854c100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.894+0000 7f8550c80640 1 -- 172.21.15.120:0/3520588740 >> 172.21.15.120:0/3520588740 conn(0x7f854c0fc1b0 msgr2=0x7f854c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.894+0000 7f8550c80640 1 -- 172.21.15.120:0/3520588740 shutdown_connections 2024-09-19T09:15:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.894+0000 7f8550c80640 1 -- 172.21.15.120:0/3520588740 wait complete. 2024-09-19T09:15:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.895+0000 7f8550c80640 1 Processor -- start 2024-09-19T09:15:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.895+0000 7f8550c80640 1 -- start start 2024-09-19T09:15:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.895+0000 7f8550c80640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f854c100a50 0x7f854c06ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.895+0000 7f8550c80640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f854c101c40 0x7f854c19ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.895+0000 7f8550c80640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f854c102600 0x7f854c19f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.895+0000 7f8550c80640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f854c10c6b0 con 0x7f854c101c40 2024-09-19T09:15:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.895+0000 7f8550c80640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f854c10c530 con 0x7f854c102600 2024-09-19T09:15:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.895+0000 7f8550c80640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f854c10c830 con 0x7f854c100a50 2024-09-19T09:15:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.895+0000 7f8549d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f854c101c40 0x7f854c19ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.895+0000 7f8549d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f854c101c40 0x7f854c19ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51748/0 (socket says 172.21.15.120:51748) 2024-09-19T09:15:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.895+0000 7f8549d9b640 1 -- 172.21.15.120:0/793768846 learned_addr learned my addr 172.21.15.120:0/793768846 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.895+0000 7f854a59c640 1 --2- 172.21.15.120:0/793768846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f854c100a50 0x7f854c06ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.895+0000 7f854ad9d640 1 --2- 172.21.15.120:0/793768846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f854c102600 0x7f854c19f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.896+0000 7f8549d9b640 1 -- 172.21.15.120:0/793768846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f854c100a50 msgr2=0x7f854c06ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.896+0000 7f8549d9b640 1 --2- 172.21.15.120:0/793768846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f854c100a50 0x7f854c06ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.896+0000 7f8549d9b640 1 -- 172.21.15.120:0/793768846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f854c102600 msgr2=0x7f854c19f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.896+0000 7f8549d9b640 1 --2- 172.21.15.120:0/793768846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f854c102600 0x7f854c19f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.896+0000 7f8549d9b640 1 -- 172.21.15.120:0/793768846 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854c19f990 con 0x7f854c101c40 2024-09-19T09:15:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.896+0000 7f8549d9b640 1 --2- 172.21.15.120:0/793768846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f854c101c40 0x7f854c19ca80 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f853400ea60 tx=0x7f853400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.896+0000 7f85437fe640 1 -- 172.21.15.120:0/793768846 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853400cde0 con 0x7f854c101c40 2024-09-19T09:15:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.896+0000 7f85437fe640 1 -- 172.21.15.120:0/793768846 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85340035f0 con 0x7f854c101c40 2024-09-19T09:15:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.896+0000 7f8550c80640 1 -- 172.21.15.120:0/793768846 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f854c19fc80 con 0x7f854c101c40 2024-09-19T09:15:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.896+0000 7f85437fe640 1 -- 172.21.15.120:0/793768846 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8534010670 con 0x7f854c101c40 2024-09-19T09:15:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.896+0000 7f8550c80640 1 -- 172.21.15.120:0/793768846 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f854c1a01d0 con 0x7f854c101c40 2024-09-19T09:15:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.898+0000 7f85437fe640 1 -- 172.21.15.120:0/793768846 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f85340037a0 con 0x7f854c101c40 2024-09-19T09:15:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.898+0000 7f8553a14740 1 Processor -- start 2024-09-19T09:15:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.899+0000 7f8553a14740 1 -- start start 2024-09-19T09:15:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.899+0000 7f8553a14740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c4c72db0 0x55f0c4c731b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.899+0000 7f8553a14740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c4c736f0 0x55f0c4c781b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.899+0000 7f8553a14740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c4c78780 0x55f0c4c7ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.899+0000 7f8553a14740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f0c4bd22f0 con 0x55f0c4c78780 2024-09-19T09:15:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.899+0000 7f8553a14740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f0c4b7a270 con 0x55f0c4c72db0 2024-09-19T09:15:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.899+0000 7f8553a14740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f0c4bce2b0 con 0x55f0c4c736f0 2024-09-19T09:15:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.899+0000 7f854ad9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c4c72db0 0x55f0c4c731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.899+0000 7f8549d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c4c78780 0x55f0c4c7ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.899+0000 7f854ad9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c4c72db0 0x55f0c4c731b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:33824/0 (socket says 172.21.15.120:33824) 2024-09-19T09:15:10.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.899+0000 7f854ad9d640 1 -- 172.21.15.120:0/1284792306 learned_addr learned my addr 172.21.15.120:0/1284792306 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:10.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.899+0000 7f854a59c640 1 --2- 172.21.15.120:0/1284792306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c4c736f0 0x55f0c4c781b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:10.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.899+0000 7f854ad9d640 1 -- 172.21.15.120:0/1284792306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c4c736f0 msgr2=0x55f0c4c781b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:10.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.899+0000 7f854ad9d640 1 --2- 172.21.15.120:0/1284792306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c4c736f0 0x55f0c4c781b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:10.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.899+0000 7f854ad9d640 1 -- 172.21.15.120:0/1284792306 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c4c78780 msgr2=0x55f0c4c7ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:10.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.899+0000 7f854ad9d640 1 --2- 172.21.15.120:0/1284792306 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c4c78780 0x55f0c4c7ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:10.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.899+0000 7f854ad9d640 1 -- 172.21.15.120:0/1284792306 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0c4b79340 con 0x55f0c4c72db0 2024-09-19T09:15:10.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.899+0000 7f85437fe640 1 --2- 172.21.15.120:0/793768846 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f85380a32f0 0x7f85380a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:10.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.899+0000 7f854a59c640 1 --2- 172.21.15.120:0/793768846 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f85380a32f0 0x7f85380a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:10.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.899+0000 7f854ad9d640 1 --2- 172.21.15.120:0/1284792306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c4c72db0 0x55f0c4c731b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8544009900 tx=0x7f8544005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c961e99950f0c16 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:10.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.899+0000 7f85437fe640 1 -- 172.21.15.120:0/793768846 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f85340cfbe0 con 0x7f854c101c40 2024-09-19T09:15:10.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.900+0000 7f8540ff9640 1 -- 172.21.15.120:0/1284792306 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8544040070 con 0x55f0c4c72db0 2024-09-19T09:15:10.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.900+0000 7f8540ff9640 1 -- 172.21.15.120:0/1284792306 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85440319f0 con 0x55f0c4c72db0 2024-09-19T09:15:10.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.900+0000 7f8540ff9640 1 -- 172.21.15.120:0/1284792306 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8544035040 con 0x55f0c4c72db0 2024-09-19T09:15:10.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.900+0000 7f8553a14740 1 -- 172.21.15.120:0/1284792306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c4c72db0 msgr2=0x55f0c4c731b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:10.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.900+0000 7f854a59c640 1 --2- 172.21.15.120:0/793768846 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f85380a32f0 0x7f85380a57b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x55f0c4c74120 tx=0x7f853c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:10.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.900+0000 7f8553a14740 1 --2- 172.21.15.120:0/1284792306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c4c72db0 0x55f0c4c731b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8544009900 tx=0x7f8544005c00 comp rx=0 tx=0).stop 2024-09-19T09:15:10.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.900+0000 7f8553a14740 1 -- 172.21.15.120:0/1284792306 shutdown_connections 2024-09-19T09:15:10.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.900+0000 7f8553a14740 1 --2- 172.21.15.120:0/1284792306 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c4c78780 0x55f0c4c7ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:10.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.900+0000 7f8553a14740 1 --2- 172.21.15.120:0/1284792306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c4c736f0 0x55f0c4c781b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:10.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.900+0000 7f8553a14740 1 --2- 172.21.15.120:0/1284792306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c4c72db0 0x55f0c4c731b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:10.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.900+0000 7f8553a14740 1 -- 172.21.15.120:0/1284792306 >> 172.21.15.120:0/1284792306 conn(0x55f0c4c6a010 msgr2=0x55f0c4c6a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:10.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.901+0000 7f8553a14740 1 -- 172.21.15.120:0/1284792306 shutdown_connections 2024-09-19T09:15:10.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.901+0000 7f8553a14740 1 -- 172.21.15.120:0/1284792306 wait complete. 2024-09-19T09:15:10.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.901+0000 7f8553a14740 1 -- start start 2024-09-19T09:15:10.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.901+0000 7f8553a14740 1 Processor -- start 2024-09-19T09:15:10.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.901+0000 7f8553a14740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c4c72db0 0x55f0c4c7cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.901+0000 7f8553a14740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c4c736f0 0x55f0c4c7d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.901+0000 7f8553a14740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c4c78780 0x55f0c4c84520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.901+0000 7f8553a14740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f0c4c7b6f0 con 0x55f0c4c736f0 2024-09-19T09:15:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.901+0000 7f8553a14740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f0c4b5b290 con 0x55f0c4c72db0 2024-09-19T09:15:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.901+0000 7f8553a14740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f0c4c7b870 con 0x55f0c4c78780 2024-09-19T09:15:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.901+0000 7f8553a14740 10 client.0 _ll_register_callbacks cb 0x55f0c4b8c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.901+0000 7f8549d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c4c736f0 0x55f0c4c7d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.901+0000 7f8549d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c4c736f0 0x55f0c4c7d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51776/0 (socket says 172.21.15.120:51776) 2024-09-19T09:15:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.901+0000 7f8549d9b640 1 -- 172.21.15.120:0/133173526 learned_addr learned my addr 172.21.15.120:0/133173526 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.901+0000 7f854a59c640 1 --2- 172.21.15.120:0/133173526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c4c72db0 0x55f0c4c7cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.901+0000 7f854ad9d640 1 --2- 172.21.15.120:0/133173526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c4c78780 0x55f0c4c84520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.902+0000 7f8549d9b640 1 -- 172.21.15.120:0/133173526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c4c78780 msgr2=0x55f0c4c84520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.902+0000 7f8549d9b640 1 --2- 172.21.15.120:0/133173526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c4c78780 0x55f0c4c84520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.902+0000 7f8549d9b640 1 -- 172.21.15.120:0/133173526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c4c72db0 msgr2=0x55f0c4c7cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.902+0000 7f8549d9b640 1 --2- 172.21.15.120:0/133173526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c4c72db0 0x55f0c4c7cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.902+0000 7f8549d9b640 1 -- 172.21.15.120:0/133173526 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0c48e5210 con 0x55f0c4c736f0 2024-09-19T09:15:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.902+0000 7f8549d9b640 1 --2- 172.21.15.120:0/133173526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c4c736f0 0x55f0c4c7d4e0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f85340a4940 tx=0x7f8534097e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0224fbaf8c16b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.902+0000 7f8531ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:15:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.902+0000 7f8531ffb640 1 -- 172.21.15.120:0/133173526 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853409bb60 con 0x55f0c4c736f0 2024-09-19T09:15:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.902+0000 7f8531ffb640 1 -- 172.21.15.120:0/133173526 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f8534098870 con 0x55f0c4c736f0 2024-09-19T09:15:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.902+0000 7f8553a14740 1 -- 172.21.15.120:0/133173526 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85440319f0 con 0x55f0c4c736f0 2024-09-19T09:15:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.902+0000 7f8531ffb640 1 -- 172.21.15.120:0/133173526 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f853409c900 con 0x55f0c4c736f0 2024-09-19T09:15:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.902+0000 7f8553a14740 10 client.19791 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.902+0000 7f8553a14740 1 -- 172.21.15.120:0/133173526 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f0c49b1620 con 0x55f0c4c736f0 2024-09-19T09:15:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.902+0000 7f8531ffb640 1 -- 172.21.15.120:0/133173526 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85440319f0 con 0x55f0c4c736f0 2024-09-19T09:15:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.902+0000 7f8531ffb640 1 -- 172.21.15.120:0/133173526 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f85340bc0e0 con 0x55f0c4c736f0 2024-09-19T09:15:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.902+0000 7f8553a14740 10 client.19791 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.902+0000 7f8553a14740 10 client.19791 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:15:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.902+0000 7f8553a14740 20 client.19791 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.902+0000 7f8553a14740 1 --2- 172.21.15.120:0/133173526 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f0c4c88bc0 0x55f0c4c8afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.902+0000 7f8553a14740 4 client.19791 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:15:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.902+0000 7f8553a14740 1 -- 172.21.15.120:0/133173526 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0c4bdfa20 con 0x55f0c4c88bc0 2024-09-19T09:15:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.903+0000 7f854a59c640 1 --2- 172.21.15.120:0/133173526 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f0c4c88bc0 0x55f0c4c8afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.903+0000 7f854a59c640 1 --2- 172.21.15.120:0/133173526 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f0c4c88bc0 0x55f0c4c8afd0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=114ac4ae4242fcb server_cookie=c87398d298a14273 in_seq=0 out_seq=0 2024-09-19T09:15:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.903+0000 7f8531ffb640 10 client.19791 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.904+0000 7f8531ffb640 1 -- 172.21.15.120:0/133173526 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f0c4bdfa20 con 0x55f0c4c88bc0 2024-09-19T09:15:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.904+0000 7f8531ffb640 10 client.19791 handle_command_reply: tid=0 2024-09-19T09:15:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.904+0000 7f8553a14740 1 client.19791 shutdown 2024-09-19T09:15:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.904+0000 7f8553a14740 1 -- 172.21.15.120:0/133173526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c4c736f0 msgr2=0x55f0c4c7d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.904+0000 7f8553a14740 1 --2- 172.21.15.120:0/133173526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c4c736f0 0x55f0c4c7d4e0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f85340a4940 tx=0x7f8534097e40 comp rx=0 tx=0).stop 2024-09-19T09:15:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.904+0000 7f8553a14740 1 -- 172.21.15.120:0/133173526 shutdown_connections 2024-09-19T09:15:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.904+0000 7f8553a14740 1 --2- 172.21.15.120:0/133173526 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f0c4c88bc0 0x55f0c4c8afd0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.904+0000 7f8553a14740 1 --2- 172.21.15.120:0/133173526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c4c78780 0x55f0c4c84520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.904+0000 7f8553a14740 1 --2- 172.21.15.120:0/133173526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c4c736f0 0x55f0c4c7d4e0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.904+0000 7f8553a14740 1 --2- 172.21.15.120:0/133173526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c4c72db0 0x55f0c4c7cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.904+0000 7f8531ffb640 0 client.19791 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.904+0000 7f8553a14740 1 -- 172.21.15.120:0/133173526 >> 172.21.15.120:0/133173526 conn(0x55f0c4c6a010 msgr2=0x55f0c4c6b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.904+0000 7f8553a14740 1 -- 172.21.15.120:0/133173526 shutdown_connections 2024-09-19T09:15:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.904+0000 7f8553a14740 1 -- 172.21.15.120:0/133173526 wait complete. 2024-09-19T09:15:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.904+0000 7f8553a14740 20 client.19791 trim_cache size 0 max 16384 2024-09-19T09:15:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.920+0000 7f8553a14740 1 Processor -- start 2024-09-19T09:15:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.920+0000 7f8553a14740 1 -- start start 2024-09-19T09:15:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.920+0000 7f8553a14740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c4c72db0 0x55f0c4c691c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.920+0000 7f8553a14740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c4c736f0 0x55f0c4c844e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.920+0000 7f8553a14740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c4c78780 0x55f0c4c8f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.920+0000 7f8553a14740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f854c1a6db0 con 0x55f0c4c78780 2024-09-19T09:15:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.920+0000 7f8553a14740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f854c1a88f0 con 0x55f0c4c736f0 2024-09-19T09:15:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.920+0000 7f8553a14740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f854c1a7480 con 0x55f0c4c72db0 2024-09-19T09:15:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.921+0000 7f854ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c4c78780 0x55f0c4c8f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.921+0000 7f854ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c4c78780 0x55f0c4c8f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51780/0 (socket says 172.21.15.120:51780) 2024-09-19T09:15:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.921+0000 7f854ad9d640 1 -- 172.21.15.120:0/2170441231 learned_addr learned my addr 172.21.15.120:0/2170441231 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.921+0000 7f8549d9b640 1 --2- 172.21.15.120:0/2170441231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c4c736f0 0x55f0c4c844e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.921+0000 7f854a59c640 1 --2- 172.21.15.120:0/2170441231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c4c72db0 0x55f0c4c691c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.921+0000 7f854ad9d640 1 -- 172.21.15.120:0/2170441231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c4c72db0 msgr2=0x55f0c4c691c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.921+0000 7f854ad9d640 1 --2- 172.21.15.120:0/2170441231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c4c72db0 0x55f0c4c691c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.921+0000 7f854ad9d640 1 -- 172.21.15.120:0/2170441231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c4c736f0 msgr2=0x55f0c4c844e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.921+0000 7f854ad9d640 1 --2- 172.21.15.120:0/2170441231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c4c736f0 0x55f0c4c844e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.921+0000 7f854ad9d640 1 -- 172.21.15.120:0/2170441231 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0c4c8b620 con 0x55f0c4c78780 2024-09-19T09:15:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.921+0000 7f854ad9d640 1 --2- 172.21.15.120:0/2170441231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c4c78780 0x55f0c4c8f8a0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f8544042000 tx=0x7f854402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47468dd9b4fc9d14 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.921+0000 7f8533fff640 1 -- 172.21.15.120:0/2170441231 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8544040070 con 0x55f0c4c78780 2024-09-19T09:15:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.921+0000 7f8533fff640 1 -- 172.21.15.120:0/2170441231 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f854404c070 con 0x55f0c4c78780 2024-09-19T09:15:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.921+0000 7f8533fff640 1 -- 172.21.15.120:0/2170441231 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8544035040 con 0x55f0c4c78780 2024-09-19T09:15:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.922+0000 7f8553a14740 1 -- 172.21.15.120:0/2170441231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c4c78780 msgr2=0x55f0c4c8f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.922+0000 7f8553a14740 1 --2- 172.21.15.120:0/2170441231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c4c78780 0x55f0c4c8f8a0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f8544042000 tx=0x7f854402a410 comp rx=0 tx=0).stop 2024-09-19T09:15:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.922+0000 7f8553a14740 1 -- 172.21.15.120:0/2170441231 shutdown_connections 2024-09-19T09:15:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.922+0000 7f8553a14740 1 --2- 172.21.15.120:0/2170441231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c4c78780 0x55f0c4c8f8a0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.922+0000 7f8553a14740 1 --2- 172.21.15.120:0/2170441231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c4c736f0 0x55f0c4c844e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.922+0000 7f8553a14740 1 --2- 172.21.15.120:0/2170441231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c4c72db0 0x55f0c4c691c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.922+0000 7f8553a14740 1 -- 172.21.15.120:0/2170441231 >> 172.21.15.120:0/2170441231 conn(0x55f0c4c6a010 msgr2=0x55f0c4c7c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.922+0000 7f8553a14740 1 -- 172.21.15.120:0/2170441231 shutdown_connections 2024-09-19T09:15:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.922+0000 7f8553a14740 1 -- 172.21.15.120:0/2170441231 wait complete. 2024-09-19T09:15:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.922+0000 7f8553a14740 1 -- start start 2024-09-19T09:15:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.922+0000 7f8553a14740 1 Processor -- start 2024-09-19T09:15:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.922+0000 7f8553a14740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c4c72db0 0x55f0c4c81de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.922+0000 7f8553a14740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c4c736f0 0x55f0c4c82320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.922+0000 7f8553a14740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c4c78780 0x55f0c4c832e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.923+0000 7f8553a14740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f854c1a8220 con 0x55f0c4c72db0 2024-09-19T09:15:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.923+0000 7f8553a14740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f854c1a7b50 con 0x55f0c4c736f0 2024-09-19T09:15:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.923+0000 7f8553a14740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f854c1a4d70 con 0x55f0c4c78780 2024-09-19T09:15:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.923+0000 7f8553a14740 10 client.0 _ll_register_callbacks cb 0x7f85340d3350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.923+0000 7f854a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c4c72db0 0x55f0c4c81de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.923+0000 7f854a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c4c72db0 0x55f0c4c81de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51786/0 (socket says 172.21.15.120:51786) 2024-09-19T09:15:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.923+0000 7f854a59c640 1 -- 172.21.15.120:0/737619831 learned_addr learned my addr 172.21.15.120:0/737619831 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.923+0000 7f854ad9d640 1 --2- 172.21.15.120:0/737619831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c4c78780 0x55f0c4c832e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.923+0000 7f8549d9b640 1 --2- 172.21.15.120:0/737619831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c4c736f0 0x55f0c4c82320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.923+0000 7f854a59c640 1 -- 172.21.15.120:0/737619831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c4c78780 msgr2=0x55f0c4c832e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.923+0000 7f854a59c640 1 --2- 172.21.15.120:0/737619831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c4c78780 0x55f0c4c832e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.923+0000 7f854a59c640 1 -- 172.21.15.120:0/737619831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c4c736f0 msgr2=0x55f0c4c82320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.923+0000 7f854a59c640 1 --2- 172.21.15.120:0/737619831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c4c736f0 0x55f0c4c82320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.923+0000 7f854a59c640 1 -- 172.21.15.120:0/737619831 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0c4c652d0 con 0x55f0c4c72db0 2024-09-19T09:15:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.923+0000 7f8549d9b640 1 --2- 172.21.15.120:0/737619831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c4c736f0 0x55f0c4c82320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:15:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.923+0000 7f854a59c640 1 --2- 172.21.15.120:0/737619831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c4c72db0 0x55f0c4c81de0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f853c009810 tx=0x7f853c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946eef4fac893f47 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.923+0000 7f8531ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:15:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.923+0000 7f8531ffb640 1 -- 172.21.15.120:0/737619831 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c00fb50 con 0x55f0c4c72db0 2024-09-19T09:15:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.923+0000 7f8553a14740 1 -- 172.21.15.120:0/737619831 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f854404c070 con 0x55f0c4c72db0 2024-09-19T09:15:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.923+0000 7f8531ffb640 1 -- 172.21.15.120:0/737619831 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f853c00fcf0 con 0x55f0c4c72db0 2024-09-19T09:15:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.923+0000 7f8531ffb640 1 -- 172.21.15.120:0/737619831 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f853c004b20 con 0x55f0c4c72db0 2024-09-19T09:15:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.923+0000 7f8553a14740 10 client.19797 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.923+0000 7f8553a14740 1 -- 172.21.15.120:0/737619831 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f0c4cf39b0 con 0x55f0c4c72db0 2024-09-19T09:15:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.924+0000 7f8531ffb640 1 -- 172.21.15.120:0/737619831 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f854404c070 con 0x55f0c4c72db0 2024-09-19T09:15:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.924+0000 7f8531ffb640 1 -- 172.21.15.120:0/737619831 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f853c031070 con 0x55f0c4c72db0 2024-09-19T09:15:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.924+0000 7f8553a14740 10 client.19797 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.924+0000 7f8553a14740 10 client.19797 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:15:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.924+0000 7f8553a14740 20 client.19797 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.924+0000 7f8553a14740 1 --2- 172.21.15.120:0/737619831 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f0c4cf4340 0x55f0c4cf6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.924+0000 7f8553a14740 4 client.19797 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:15:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.924+0000 7f8553a14740 1 -- 172.21.15.120:0/737619831 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f0c4bd9d70 con 0x55f0c4cf4340 2024-09-19T09:15:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.924+0000 7f8549d9b640 1 --2- 172.21.15.120:0/737619831 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f0c4cf4340 0x55f0c4cf6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.924+0000 7f8531ffb640 10 client.19797 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:10.924+0000 7f8549d9b640 1 --2- 172.21.15.120:0/737619831 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f0c4cf4340 0x55f0c4cf6760 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35a16201160e32 server_cookie=57006ec0dece26a1 in_seq=0 out_seq=0 2024-09-19T09:15:11.076 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1216167 9.82 MB/sec execute 546 sec latency 95.963 ms 2024-09-19T09:15:11.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:11.096+0000 7f8531ffb640 1 -- 172.21.15.120:0/737619831 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f0c4bd9d70 con 0x55f0c4cf4340 2024-09-19T09:15:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:11.096+0000 7f8531ffb640 10 client.19797 handle_command_reply: tid=0 2024-09-19T09:15:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:11.096+0000 7f8553a14740 1 client.19797 shutdown 2024-09-19T09:15:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:11.096+0000 7f8553a14740 1 -- 172.21.15.120:0/737619831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c4c72db0 msgr2=0x55f0c4c81de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:11.096+0000 7f8553a14740 1 --2- 172.21.15.120:0/737619831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c4c72db0 0x55f0c4c81de0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f853c009810 tx=0x7f853c004a40 comp rx=0 tx=0).stop 2024-09-19T09:15:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:11.096+0000 7f8553a14740 1 -- 172.21.15.120:0/737619831 shutdown_connections 2024-09-19T09:15:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:11.096+0000 7f8553a14740 1 --2- 172.21.15.120:0/737619831 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f0c4cf4340 0x55f0c4cf6760 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:11.097+0000 7f8531ffb640 0 client.19797 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:11.097+0000 7f8553a14740 1 --2- 172.21.15.120:0/737619831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c4c78780 0x55f0c4c832e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:11.097+0000 7f8553a14740 1 --2- 172.21.15.120:0/737619831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c4c736f0 0x55f0c4c82320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:11.097+0000 7f8553a14740 1 --2- 172.21.15.120:0/737619831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c4c72db0 0x55f0c4c81de0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:11.097+0000 7f8553a14740 1 -- 172.21.15.120:0/737619831 >> 172.21.15.120:0/737619831 conn(0x55f0c4c6a010 msgr2=0x55f0c4c691c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:11.097+0000 7f8553a14740 1 -- 172.21.15.120:0/737619831 shutdown_connections 2024-09-19T09:15:11.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:11.097+0000 7f8553a14740 1 -- 172.21.15.120:0/737619831 wait complete. 2024-09-19T09:15:11.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:11.097+0000 7f8553a14740 20 client.19797 trim_cache size 0 max 16384 2024-09-19T09:15:11.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:15:11.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:15:11.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:15:11.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:15:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:11.097+0000 7f8550c80640 1 -- 172.21.15.120:0/793768846 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f85380a32f0 msgr2=0x7f85380a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:11.097+0000 7f8550c80640 1 --2- 172.21.15.120:0/793768846 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f85380a32f0 0x7f85380a57b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x55f0c4c74120 tx=0x7f853c009500 comp rx=0 tx=0).stop 2024-09-19T09:15:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:11.097+0000 7f8550c80640 1 -- 172.21.15.120:0/793768846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f854c101c40 msgr2=0x7f854c19ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:11.097+0000 7f8550c80640 1 --2- 172.21.15.120:0/793768846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f854c101c40 0x7f854c19ca80 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f853400ea60 tx=0x7f853400ef20 comp rx=0 tx=0).stop 2024-09-19T09:15:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:11.098+0000 7f8550c80640 1 -- 172.21.15.120:0/793768846 shutdown_connections 2024-09-19T09:15:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:11.098+0000 7f8550c80640 1 --2- 172.21.15.120:0/793768846 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f85380a32f0 0x7f85380a57b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:11.098+0000 7f8550c80640 1 --2- 172.21.15.120:0/793768846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f854c102600 0x7f854c19f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:11.098+0000 7f8550c80640 1 --2- 172.21.15.120:0/793768846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f854c101c40 0x7f854c19ca80 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:11.098+0000 7f8550c80640 1 --2- 172.21.15.120:0/793768846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f854c100a50 0x7f854c06ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:11.098+0000 7f8550c80640 1 -- 172.21.15.120:0/793768846 >> 172.21.15.120:0/793768846 conn(0x7f854c0fc1b0 msgr2=0x7f854c104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:11.098+0000 7f8550c80640 1 -- 172.21.15.120:0/793768846 shutdown_connections 2024-09-19T09:15:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:11.098+0000 7f8550c80640 1 -- 172.21.15.120:0/793768846 wait complete. 2024-09-19T09:15:11.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:10 smithi196 ceph-mon[35150]: pgmap v713: 97 pgs: 97 active+clean; 52 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.4 MiB/s wr, 368 op/s 2024-09-19T09:15:11.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:10 smithi153 ceph-mon[35071]: pgmap v713: 97 pgs: 97 active+clean; 52 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.4 MiB/s wr, 368 op/s 2024-09-19T09:15:11.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:10 smithi120 ceph-mon[31150]: pgmap v713: 97 pgs: 97 active+clean; 52 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.4 MiB/s wr, 368 op/s 2024-09-19T09:15:12.076 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1218878 9.83 MB/sec execute 547 sec latency 28.172 ms 2024-09-19T09:15:13.076 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1222033 9.85 MB/sec execute 548 sec latency 11.165 ms 2024-09-19T09:15:13.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:12 smithi196 ceph-mon[35150]: pgmap v714: 97 pgs: 97 active+clean; 55 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 406 op/s 2024-09-19T09:15:13.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:12 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:15:13.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:12 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:15:13.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:12 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:15:13.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:12 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:15:13.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:12 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:15:13.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:12 smithi153 ceph-mon[35071]: pgmap v714: 97 pgs: 97 active+clean; 55 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 406 op/s 2024-09-19T09:15:13.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:12 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:15:13.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:12 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:15:13.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:12 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:15:13.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:12 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:15:13.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:12 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:15:13.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:12 smithi120 ceph-mon[31150]: pgmap v714: 97 pgs: 97 active+clean; 55 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 406 op/s 2024-09-19T09:15:13.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:15:13.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:15:13.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:15:13.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:15:13.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:15:14.076 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1224519 9.85 MB/sec execute 549 sec latency 88.674 ms 2024-09-19T09:15:15.076 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1226960 9.86 MB/sec execute 550 sec latency 19.466 ms 2024-09-19T09:15:15.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:14 smithi196 ceph-mon[35150]: pgmap v715: 97 pgs: 97 active+clean; 47 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.0 MiB/s wr, 421 op/s 2024-09-19T09:15:15.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:14 smithi153 ceph-mon[35071]: pgmap v715: 97 pgs: 97 active+clean; 47 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.0 MiB/s wr, 421 op/s 2024-09-19T09:15:15.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:14 smithi120 ceph-mon[31150]: pgmap v715: 97 pgs: 97 active+clean; 47 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.0 MiB/s wr, 421 op/s 2024-09-19T09:15:16.076 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1229756 9.87 MB/sec execute 551 sec latency 14.973 ms 2024-09-19T09:15:17.077 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1232800 9.88 MB/sec execute 552 sec latency 10.640 ms 2024-09-19T09:15:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:16 smithi196 ceph-mon[35150]: pgmap v716: 97 pgs: 97 active+clean; 44 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.3 MiB/s wr, 358 op/s 2024-09-19T09:15:17.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:16 smithi153 ceph-mon[35071]: pgmap v716: 97 pgs: 97 active+clean; 44 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.3 MiB/s wr, 358 op/s 2024-09-19T09:15:17.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:16 smithi120 ceph-mon[31150]: pgmap v716: 97 pgs: 97 active+clean; 44 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.3 MiB/s wr, 358 op/s 2024-09-19T09:15:18.076 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1235910 9.89 MB/sec execute 553 sec latency 14.664 ms 2024-09-19T09:15:19.076 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1237640 9.89 MB/sec execute 554 sec latency 66.795 ms 2024-09-19T09:15:19.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:18 smithi196 ceph-mon[35150]: pgmap v717: 97 pgs: 97 active+clean; 48 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.1 MiB/s wr, 431 op/s 2024-09-19T09:15:19.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:18 smithi153 ceph-mon[35071]: pgmap v717: 97 pgs: 97 active+clean; 48 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.1 MiB/s wr, 431 op/s 2024-09-19T09:15:19.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:18 smithi120 ceph-mon[31150]: pgmap v717: 97 pgs: 97 active+clean; 48 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.1 MiB/s wr, 431 op/s 2024-09-19T09:15:20.077 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1240149 9.90 MB/sec execute 555 sec latency 35.689 ms 2024-09-19T09:15:21.076 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1243288 9.91 MB/sec execute 556 sec latency 9.368 ms 2024-09-19T09:15:21.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:20 smithi196 ceph-mon[35150]: pgmap v718: 97 pgs: 97 active+clean; 47 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.4 MiB/s wr, 466 op/s 2024-09-19T09:15:21.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:20 smithi153 ceph-mon[35071]: pgmap v718: 97 pgs: 97 active+clean; 47 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.4 MiB/s wr, 466 op/s 2024-09-19T09:15:21.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:20 smithi120 ceph-mon[31150]: pgmap v718: 97 pgs: 97 active+clean; 47 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.4 MiB/s wr, 466 op/s 2024-09-19T09:15:22.077 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1246401 9.92 MB/sec execute 557 sec latency 11.445 ms 2024-09-19T09:15:23.077 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1249052 9.93 MB/sec execute 558 sec latency 11.653 ms 2024-09-19T09:15:23.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:22 smithi196 ceph-mon[35150]: pgmap v719: 97 pgs: 97 active+clean; 48 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.9 MiB/s wr, 426 op/s 2024-09-19T09:15:23.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:22 smithi153 ceph-mon[35071]: pgmap v719: 97 pgs: 97 active+clean; 48 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.9 MiB/s wr, 426 op/s 2024-09-19T09:15:23.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:22 smithi120 ceph-mon[31150]: pgmap v719: 97 pgs: 97 active+clean; 48 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.9 MiB/s wr, 426 op/s 2024-09-19T09:15:24.077 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1251604 9.94 MB/sec execute 559 sec latency 33.086 ms 2024-09-19T09:15:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.186+0000 7f5564a77640 1 -- 172.21.15.120:0/3325815566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5560102640 msgr2=0x7f5560104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.186+0000 7f5564a77640 1 --2- 172.21.15.120:0/3325815566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5560102640 0x7f5560104a30 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f5550009900 tx=0x7f555002a410 comp rx=0 tx=0).stop 2024-09-19T09:15:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.186+0000 7f5564a77640 1 -- 172.21.15.120:0/3325815566 shutdown_connections 2024-09-19T09:15:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.186+0000 7f5564a77640 1 --2- 172.21.15.120:0/3325815566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55601078a0 0x7f5560109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.186+0000 7f5564a77640 1 --2- 172.21.15.120:0/3325815566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5560104f70 0x7f5560107360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.186+0000 7f5564a77640 1 --2- 172.21.15.120:0/3325815566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5560102640 0x7f5560104a30 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.186+0000 7f5564a77640 1 -- 172.21.15.120:0/3325815566 >> 172.21.15.120:0/3325815566 conn(0x7f55600fc1e0 msgr2=0x7f55600fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.186+0000 7f5564a77640 1 -- 172.21.15.120:0/3325815566 shutdown_connections 2024-09-19T09:15:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.186+0000 7f5564a77640 1 -- 172.21.15.120:0/3325815566 wait complete. 2024-09-19T09:15:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.187+0000 7f5564a77640 1 Processor -- start 2024-09-19T09:15:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.187+0000 7f5564a77640 1 -- start start 2024-09-19T09:15:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.187+0000 7f5564a77640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5560102640 0x7f5560197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.187+0000 7f5564a77640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5560104f70 0x7f55601983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.187+0000 7f5564a77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55601078a0 0x7f556019f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.187+0000 7f5564a77640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f556010d870 con 0x7f55601078a0 2024-09-19T09:15:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.187+0000 7f5564a77640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f556010d6f0 con 0x7f5560102640 2024-09-19T09:15:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.187+0000 7f5564a77640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f556010d9f0 con 0x7f5560104f70 2024-09-19T09:15:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.187+0000 7f555ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55601078a0 0x7f556019f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.187+0000 7f555ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55601078a0 0x7f556019f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45328/0 (socket says 172.21.15.120:45328) 2024-09-19T09:15:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.187+0000 7f555ed9d640 1 -- 172.21.15.120:0/2189251222 learned_addr learned my addr 172.21.15.120:0/2189251222 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.187+0000 7f555e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5560102640 0x7f5560197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.187+0000 7f555dd9b640 1 --2- 172.21.15.120:0/2189251222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5560104f70 0x7f55601983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.187+0000 7f555ed9d640 1 -- 172.21.15.120:0/2189251222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5560104f70 msgr2=0x7f55601983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.187+0000 7f555ed9d640 1 --2- 172.21.15.120:0/2189251222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5560104f70 0x7f55601983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.187+0000 7f555ed9d640 1 -- 172.21.15.120:0/2189251222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5560102640 msgr2=0x7f5560197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.187+0000 7f555ed9d640 1 --2- 172.21.15.120:0/2189251222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5560102640 0x7f5560197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.187+0000 7f555ed9d640 1 -- 172.21.15.120:0/2189251222 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556019fac0 con 0x7f55601078a0 2024-09-19T09:15:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.187+0000 7f555ed9d640 1 --2- 172.21.15.120:0/2189251222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55601078a0 0x7f556019f450 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f555800eb40 tx=0x7f555800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.187+0000 7f554f7fe640 1 -- 172.21.15.120:0/2189251222 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558004b20 con 0x7f55601078a0 2024-09-19T09:15:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.187+0000 7f554f7fe640 1 -- 172.21.15.120:0/2189251222 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5558004cc0 con 0x7f55601078a0 2024-09-19T09:15:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.187+0000 7f5564a77640 1 -- 172.21.15.120:0/2189251222 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f556019fdb0 con 0x7f55601078a0 2024-09-19T09:15:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.187+0000 7f554f7fe640 1 -- 172.21.15.120:0/2189251222 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558010850 con 0x7f55601078a0 2024-09-19T09:15:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.188+0000 7f5564a77640 1 -- 172.21.15.120:0/2189251222 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55601a02a0 con 0x7f55601078a0 2024-09-19T09:15:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.190+0000 7f554f7fe640 1 -- 172.21.15.120:0/2189251222 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5558005020 con 0x7f55601078a0 2024-09-19T09:15:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.190+0000 7f5567812740 1 Processor -- start 2024-09-19T09:15:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.190+0000 7f5567812740 1 -- start start 2024-09-19T09:15:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.190+0000 7f5567812740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b8ef5bd9a0 0x55b8ef5bdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.190+0000 7f5567812740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8ef5be2e0 0x55b8ef5c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.190+0000 7f5567812740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8ef5c3370 0x55b8ef5c5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.190+0000 7f5567812740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b8ef5212f0 con 0x55b8ef5c3370 2024-09-19T09:15:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.190+0000 7f5567812740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b8ef4c9270 con 0x55b8ef5bd9a0 2024-09-19T09:15:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.190+0000 7f5567812740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b8ef51d2b0 con 0x55b8ef5be2e0 2024-09-19T09:15:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.190+0000 7f555dd9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b8ef5bd9a0 0x55b8ef5bdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.190+0000 7f554f7fe640 1 --2- 172.21.15.120:0/2189251222 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f55480a32f0 0x7f55480a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.190+0000 7f555e59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8ef5be2e0 0x55b8ef5c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.190+0000 7f555dd9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b8ef5bd9a0 0x55b8ef5bdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:48092/0 (socket says 172.21.15.120:48092) 2024-09-19T09:15:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.190+0000 7f555dd9b640 1 -- 172.21.15.120:0/405812556 learned_addr learned my addr 172.21.15.120:0/405812556 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.191+0000 7f555dd9b640 1 --2- 172.21.15.120:0/405812556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8ef5c3370 0x55b8ef5c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.191+0000 7f555ed9d640 1 --2- 172.21.15.120:0/2189251222 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f55480a32f0 0x7f55480a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.191+0000 7f555dd9b640 1 -- 172.21.15.120:0/405812556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8ef5be2e0 msgr2=0x55b8ef5c2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.191+0000 7f555dd9b640 1 --2- 172.21.15.120:0/405812556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8ef5be2e0 0x55b8ef5c2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.191+0000 7f554f7fe640 1 -- 172.21.15.120:0/2189251222 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f55580d0170 con 0x7f55601078a0 2024-09-19T09:15:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.191+0000 7f555dd9b640 1 -- 172.21.15.120:0/405812556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8ef5c3370 msgr2=0x55b8ef5c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.191+0000 7f555dd9b640 1 --2- 172.21.15.120:0/405812556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8ef5c3370 0x55b8ef5c5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.191+0000 7f555dd9b640 1 -- 172.21.15.120:0/405812556 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8ef4c8340 con 0x55b8ef5bd9a0 2024-09-19T09:15:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.191+0000 7f555dd9b640 1 --2- 172.21.15.120:0/405812556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b8ef5bd9a0 0x55b8ef5bdda0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f555400cf30 tx=0x7f555400c170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1712ab82b929f630 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.191+0000 7f555ed9d640 1 --2- 172.21.15.120:0/2189251222 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f55480a32f0 0x7f55480a57b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f555800c7d0 tx=0x7f5558087040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.191+0000 7f554cff9640 1 -- 172.21.15.120:0/405812556 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555401c070 con 0x55b8ef5bd9a0 2024-09-19T09:15:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.191+0000 7f554cff9640 1 -- 172.21.15.120:0/405812556 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5554017af0 con 0x55b8ef5bd9a0 2024-09-19T09:15:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.191+0000 7f554cff9640 1 -- 172.21.15.120:0/405812556 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555400f040 con 0x55b8ef5bd9a0 2024-09-19T09:15:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.192+0000 7f5567812740 1 -- 172.21.15.120:0/405812556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b8ef5bd9a0 msgr2=0x55b8ef5bdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.192+0000 7f5567812740 1 --2- 172.21.15.120:0/405812556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b8ef5bd9a0 0x55b8ef5bdda0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f555400cf30 tx=0x7f555400c170 comp rx=0 tx=0).stop 2024-09-19T09:15:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.192+0000 7f5567812740 1 -- 172.21.15.120:0/405812556 shutdown_connections 2024-09-19T09:15:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.192+0000 7f5567812740 1 --2- 172.21.15.120:0/405812556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8ef5c3370 0x55b8ef5c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.192+0000 7f5567812740 1 --2- 172.21.15.120:0/405812556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8ef5be2e0 0x55b8ef5c2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.192+0000 7f5567812740 1 --2- 172.21.15.120:0/405812556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b8ef5bd9a0 0x55b8ef5bdda0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.192+0000 7f5567812740 1 -- 172.21.15.120:0/405812556 >> 172.21.15.120:0/405812556 conn(0x55b8ef5b9010 msgr2=0x55b8ef5b9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.192+0000 7f5567812740 1 -- 172.21.15.120:0/405812556 shutdown_connections 2024-09-19T09:15:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.192+0000 7f5567812740 1 -- 172.21.15.120:0/405812556 wait complete. 2024-09-19T09:15:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.192+0000 7f5567812740 1 -- start start 2024-09-19T09:15:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.193+0000 7f5567812740 1 Processor -- start 2024-09-19T09:15:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.193+0000 7f5567812740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8ef5bd9a0 0x55b8ef5d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.193+0000 7f5567812740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b8ef5be2e0 0x55b8ef5d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.193+0000 7f5567812740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8ef5c3370 0x55b8ef5d7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.193+0000 7f5567812740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b8ef5c6340 con 0x55b8ef5c3370 2024-09-19T09:15:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.193+0000 7f5567812740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b8ef4aa290 con 0x55b8ef5be2e0 2024-09-19T09:15:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.193+0000 7f5567812740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b8ef5c64c0 con 0x55b8ef5bd9a0 2024-09-19T09:15:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.193+0000 7f5567812740 10 client.0 _ll_register_callbacks cb 0x55b8ef4db140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.193+0000 7f555dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8ef5c3370 0x55b8ef5d7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.193+0000 7f555dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8ef5c3370 0x55b8ef5d7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45348/0 (socket says 172.21.15.120:45348) 2024-09-19T09:15:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.193+0000 7f555dd9b640 1 -- 172.21.15.120:0/1274613230 learned_addr learned my addr 172.21.15.120:0/1274613230 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.193+0000 7f555e59c640 1 --2- 172.21.15.120:0/1274613230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b8ef5be2e0 0x55b8ef5d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.193+0000 7f555dd9b640 1 --2- 172.21.15.120:0/1274613230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8ef5bd9a0 0x55b8ef5d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.193+0000 7f555dd9b640 1 -- 172.21.15.120:0/1274613230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8ef5bd9a0 msgr2=0x55b8ef5d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.193+0000 7f555dd9b640 1 --2- 172.21.15.120:0/1274613230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8ef5bd9a0 0x55b8ef5d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.193+0000 7f555dd9b640 1 -- 172.21.15.120:0/1274613230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b8ef5be2e0 msgr2=0x55b8ef5d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.193+0000 7f555dd9b640 1 --2- 172.21.15.120:0/1274613230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b8ef5be2e0 0x55b8ef5d09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.193+0000 7f555dd9b640 1 -- 172.21.15.120:0/1274613230 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8ef234210 con 0x55b8ef5c3370 2024-09-19T09:15:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.193+0000 7f555dd9b640 1 --2- 172.21.15.120:0/1274613230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8ef5c3370 0x55b8ef5d7970 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f555401a040 tx=0x7f555400ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c64b8ba1fce2b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.193+0000 7f5545ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:15:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.193+0000 7f5545ffb640 1 -- 172.21.15.120:0/1274613230 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555401c070 con 0x55b8ef5c3370 2024-09-19T09:15:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.194+0000 7f5545ffb640 1 -- 172.21.15.120:0/1274613230 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f5554016820 con 0x55b8ef5c3370 2024-09-19T09:15:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.194+0000 7f5567812740 1 -- 172.21.15.120:0/1274613230 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5554017af0 con 0x55b8ef5c3370 2024-09-19T09:15:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.194+0000 7f5545ffb640 1 -- 172.21.15.120:0/1274613230 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5554016a20 con 0x55b8ef5c3370 2024-09-19T09:15:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.194+0000 7f5567812740 10 client.19809 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.194+0000 7f5567812740 1 -- 172.21.15.120:0/1274613230 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b8ef300620 con 0x55b8ef5c3370 2024-09-19T09:15:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.194+0000 7f5545ffb640 1 -- 172.21.15.120:0/1274613230 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5554017af0 con 0x55b8ef5c3370 2024-09-19T09:15:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.194+0000 7f5545ffb640 1 -- 172.21.15.120:0/1274613230 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f5554015040 con 0x55b8ef5c3370 2024-09-19T09:15:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.194+0000 7f5567812740 10 client.19809 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.194+0000 7f5567812740 10 client.19809 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:15:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.194+0000 7f5567812740 20 client.19809 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.194+0000 7f5567812740 1 --2- 172.21.15.120:0/1274613230 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b8ef5dbc70 0x55b8ef5de060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.194+0000 7f5567812740 4 client.19809 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:15:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.194+0000 7f5567812740 1 -- 172.21.15.120:0/1274613230 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8ef52ea20 con 0x55b8ef5dbc70 2024-09-19T09:15:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.194+0000 7f555e59c640 1 --2- 172.21.15.120:0/1274613230 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b8ef5dbc70 0x55b8ef5de060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.195+0000 7f5545ffb640 10 client.19809 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.195+0000 7f555e59c640 1 --2- 172.21.15.120:0/1274613230 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b8ef5dbc70 0x55b8ef5de060 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9590440c0e88346d server_cookie=58eb12757e66d845 in_seq=0 out_seq=0 2024-09-19T09:15:24.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.196+0000 7f5545ffb640 1 -- 172.21.15.120:0/1274613230 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b8ef52ea20 con 0x55b8ef5dbc70 2024-09-19T09:15:24.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.196+0000 7f5545ffb640 10 client.19809 handle_command_reply: tid=0 2024-09-19T09:15:24.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.196+0000 7f5567812740 1 client.19809 shutdown 2024-09-19T09:15:24.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.196+0000 7f5567812740 1 -- 172.21.15.120:0/1274613230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8ef5c3370 msgr2=0x55b8ef5d7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:24.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.196+0000 7f5567812740 1 --2- 172.21.15.120:0/1274613230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8ef5c3370 0x55b8ef5d7970 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f555401a040 tx=0x7f555400ce50 comp rx=0 tx=0).stop 2024-09-19T09:15:24.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.196+0000 7f5567812740 1 -- 172.21.15.120:0/1274613230 shutdown_connections 2024-09-19T09:15:24.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.196+0000 7f5567812740 1 --2- 172.21.15.120:0/1274613230 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b8ef5dbc70 0x55b8ef5de060 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.196+0000 7f5567812740 1 --2- 172.21.15.120:0/1274613230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8ef5c3370 0x55b8ef5d7970 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.196+0000 7f5567812740 1 --2- 172.21.15.120:0/1274613230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b8ef5be2e0 0x55b8ef5d09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.196+0000 7f5545ffb640 0 client.19809 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:24.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.196+0000 7f5567812740 1 --2- 172.21.15.120:0/1274613230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8ef5bd9a0 0x55b8ef5d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.196+0000 7f5567812740 1 -- 172.21.15.120:0/1274613230 >> 172.21.15.120:0/1274613230 conn(0x55b8ef5b9010 msgr2=0x55b8ef5ba820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:24.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.197+0000 7f5567812740 1 -- 172.21.15.120:0/1274613230 shutdown_connections 2024-09-19T09:15:24.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.197+0000 7f5567812740 1 -- 172.21.15.120:0/1274613230 wait complete. 2024-09-19T09:15:24.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.197+0000 7f5567812740 20 client.19809 trim_cache size 0 max 16384 2024-09-19T09:15:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.212+0000 7f5567812740 1 Processor -- start 2024-09-19T09:15:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.212+0000 7f5567812740 1 -- start start 2024-09-19T09:15:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.212+0000 7f5567812740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8ef5bd9a0 0x55b8ef5bab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.212+0000 7f5567812740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8ef5be2e0 0x55b8ef5d7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.212+0000 7f5567812740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b8ef5c3370 0x55b8ef5e2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.212+0000 7f5567812740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f55601a6e70 con 0x55b8ef5bd9a0 2024-09-19T09:15:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.212+0000 7f5567812740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f55601a89b0 con 0x55b8ef5c3370 2024-09-19T09:15:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.212+0000 7f5567812740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f55601a7540 con 0x55b8ef5be2e0 2024-09-19T09:15:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.212+0000 7f555dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8ef5bd9a0 0x55b8ef5bab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.212+0000 7f555dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8ef5bd9a0 0x55b8ef5bab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45364/0 (socket says 172.21.15.120:45364) 2024-09-19T09:15:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.212+0000 7f555dd9b640 1 -- 172.21.15.120:0/3757511155 learned_addr learned my addr 172.21.15.120:0/3757511155 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.212+0000 7f555dd9b640 1 --2- 172.21.15.120:0/3757511155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b8ef5c3370 0x55b8ef5e2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.212+0000 7f555e59c640 1 --2- 172.21.15.120:0/3757511155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8ef5be2e0 0x55b8ef5d7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.212+0000 7f555dd9b640 1 -- 172.21.15.120:0/3757511155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8ef5be2e0 msgr2=0x55b8ef5d7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.212+0000 7f555dd9b640 1 --2- 172.21.15.120:0/3757511155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8ef5be2e0 0x55b8ef5d7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.212+0000 7f555dd9b640 1 -- 172.21.15.120:0/3757511155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b8ef5c3370 msgr2=0x55b8ef5e2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.212+0000 7f555dd9b640 1 --2- 172.21.15.120:0/3757511155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b8ef5c3370 0x55b8ef5e2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.212+0000 7f555dd9b640 1 -- 172.21.15.120:0/3757511155 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8ef5cffa0 con 0x55b8ef5bd9a0 2024-09-19T09:15:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.213+0000 7f555dd9b640 1 --2- 172.21.15.120:0/3757511155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8ef5bd9a0 0x55b8ef5bab10 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f5554004810 tx=0x7f555400cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b1d566d99c007 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.213+0000 7f5547fff640 1 -- 172.21.15.120:0/3757511155 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555401c070 con 0x55b8ef5bd9a0 2024-09-19T09:15:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.213+0000 7f5547fff640 1 -- 172.21.15.120:0/3757511155 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5554029070 con 0x55b8ef5bd9a0 2024-09-19T09:15:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.213+0000 7f5547fff640 1 -- 172.21.15.120:0/3757511155 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555400a070 con 0x55b8ef5bd9a0 2024-09-19T09:15:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.213+0000 7f5567812740 1 -- 172.21.15.120:0/3757511155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8ef5bd9a0 msgr2=0x55b8ef5bab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.213+0000 7f5567812740 1 --2- 172.21.15.120:0/3757511155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8ef5bd9a0 0x55b8ef5bab10 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f5554004810 tx=0x7f555400cec0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.213+0000 7f5567812740 1 -- 172.21.15.120:0/3757511155 shutdown_connections 2024-09-19T09:15:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.213+0000 7f5567812740 1 --2- 172.21.15.120:0/3757511155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b8ef5c3370 0x55b8ef5e2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.213+0000 7f5567812740 1 --2- 172.21.15.120:0/3757511155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8ef5be2e0 0x55b8ef5d7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.213+0000 7f5567812740 1 --2- 172.21.15.120:0/3757511155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8ef5bd9a0 0x55b8ef5bab10 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.213+0000 7f5567812740 1 -- 172.21.15.120:0/3757511155 >> 172.21.15.120:0/3757511155 conn(0x55b8ef5b9010 msgr2=0x55b8ef5b93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.213+0000 7f5567812740 1 -- 172.21.15.120:0/3757511155 shutdown_connections 2024-09-19T09:15:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.213+0000 7f5567812740 1 -- 172.21.15.120:0/3757511155 wait complete. 2024-09-19T09:15:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.214+0000 7f5567812740 1 -- start start 2024-09-19T09:15:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.214+0000 7f5567812740 1 Processor -- start 2024-09-19T09:15:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.214+0000 7f5567812740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8ef5bd9a0 0x55b8ef5d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.214+0000 7f5567812740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b8ef5be2e0 0x55b8ef5d5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.214+0000 7f5567812740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8ef5c3370 0x55b8ef5d6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.214+0000 7f5567812740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f55601a82e0 con 0x55b8ef5bd9a0 2024-09-19T09:15:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.214+0000 7f5567812740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f55601a7c10 con 0x55b8ef5be2e0 2024-09-19T09:15:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.214+0000 7f5567812740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f55601a4e30 con 0x55b8ef5c3370 2024-09-19T09:15:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.214+0000 7f5567812740 10 client.0 _ll_register_callbacks cb 0x7f555401a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.214+0000 7f555dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8ef5bd9a0 0x55b8ef5d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.214+0000 7f555dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8ef5bd9a0 0x55b8ef5d5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45368/0 (socket says 172.21.15.120:45368) 2024-09-19T09:15:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.214+0000 7f555dd9b640 1 -- 172.21.15.120:0/3561366430 learned_addr learned my addr 172.21.15.120:0/3561366430 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.214+0000 7f555e59c640 1 --2- 172.21.15.120:0/3561366430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b8ef5be2e0 0x55b8ef5d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.214+0000 7f555dd9b640 1 --2- 172.21.15.120:0/3561366430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8ef5c3370 0x55b8ef5d6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.214+0000 7f555dd9b640 1 -- 172.21.15.120:0/3561366430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8ef5c3370 msgr2=0x55b8ef5d6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.214+0000 7f555dd9b640 1 --2- 172.21.15.120:0/3561366430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8ef5c3370 0x55b8ef5d6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.214+0000 7f555dd9b640 1 -- 172.21.15.120:0/3561366430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b8ef5be2e0 msgr2=0x55b8ef5d5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.214+0000 7f555dd9b640 1 --2- 172.21.15.120:0/3561366430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b8ef5be2e0 0x55b8ef5d5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.214+0000 7f555dd9b640 1 -- 172.21.15.120:0/3561366430 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8ef5d72b0 con 0x55b8ef5bd9a0 2024-09-19T09:15:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.215+0000 7f555dd9b640 1 --2- 172.21.15.120:0/3561366430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8ef5bd9a0 0x55b8ef5d5750 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f5554024be0 tx=0x7f5554004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a910464632d1ca4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.215+0000 7f5545ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:15:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.215+0000 7f5545ffb640 1 -- 172.21.15.120:0/3561366430 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555401c070 con 0x55b8ef5bd9a0 2024-09-19T09:15:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.215+0000 7f5545ffb640 1 -- 172.21.15.120:0/3561366430 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f5554024440 con 0x55b8ef5bd9a0 2024-09-19T09:15:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.215+0000 7f5567812740 1 -- 172.21.15.120:0/3561366430 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5554029070 con 0x55b8ef5bd9a0 2024-09-19T09:15:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.215+0000 7f5545ffb640 1 -- 172.21.15.120:0/3561366430 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f555400a040 con 0x55b8ef5bd9a0 2024-09-19T09:15:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.215+0000 7f5567812740 10 client.19815 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.215+0000 7f5567812740 1 -- 172.21.15.120:0/3561366430 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b8ef647010 con 0x55b8ef5bd9a0 2024-09-19T09:15:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.215+0000 7f5545ffb640 1 -- 172.21.15.120:0/3561366430 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5554029070 con 0x55b8ef5bd9a0 2024-09-19T09:15:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.215+0000 7f5545ffb640 1 -- 172.21.15.120:0/3561366430 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f5554015070 con 0x55b8ef5bd9a0 2024-09-19T09:15:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.215+0000 7f5567812740 10 client.19815 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.215+0000 7f5567812740 10 client.19815 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:15:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.215+0000 7f5567812740 20 client.19815 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.215+0000 7f5567812740 1 --2- 172.21.15.120:0/3561366430 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b8ef647980 0x55b8ef649da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.215+0000 7f5567812740 4 client.19815 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:15:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.215+0000 7f5567812740 1 -- 172.21.15.120:0/3561366430 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b8ef528d70 con 0x55b8ef647980 2024-09-19T09:15:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.215+0000 7f555e59c640 1 --2- 172.21.15.120:0/3561366430 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b8ef647980 0x55b8ef649da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.216+0000 7f5545ffb640 10 client.19815 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.216+0000 7f555e59c640 1 --2- 172.21.15.120:0/3561366430 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b8ef647980 0x55b8ef649da0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dc6f0656885ce09 server_cookie=d6475745f24b7885 in_seq=0 out_seq=0 2024-09-19T09:15:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.391+0000 7f5545ffb640 1 -- 172.21.15.120:0/3561366430 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b8ef528d70 con 0x55b8ef647980 2024-09-19T09:15:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.391+0000 7f5545ffb640 10 client.19815 handle_command_reply: tid=0 2024-09-19T09:15:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.391+0000 7f5567812740 1 client.19815 shutdown 2024-09-19T09:15:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.391+0000 7f5567812740 1 -- 172.21.15.120:0/3561366430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8ef5bd9a0 msgr2=0x55b8ef5d5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.391+0000 7f5567812740 1 --2- 172.21.15.120:0/3561366430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8ef5bd9a0 0x55b8ef5d5750 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f5554024be0 tx=0x7f5554004660 comp rx=0 tx=0).stop 2024-09-19T09:15:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.391+0000 7f5567812740 1 -- 172.21.15.120:0/3561366430 shutdown_connections 2024-09-19T09:15:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.391+0000 7f5567812740 1 --2- 172.21.15.120:0/3561366430 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b8ef647980 0x55b8ef649da0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.391+0000 7f5567812740 1 --2- 172.21.15.120:0/3561366430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8ef5c3370 0x55b8ef5d6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.391+0000 7f5545ffb640 0 client.19815 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.391+0000 7f5567812740 1 --2- 172.21.15.120:0/3561366430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b8ef5be2e0 0x55b8ef5d5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.391+0000 7f5567812740 1 --2- 172.21.15.120:0/3561366430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8ef5bd9a0 0x55b8ef5d5750 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.391+0000 7f5567812740 1 -- 172.21.15.120:0/3561366430 >> 172.21.15.120:0/3561366430 conn(0x55b8ef5b9010 msgr2=0x55b8ef5b9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.391+0000 7f5567812740 1 -- 172.21.15.120:0/3561366430 shutdown_connections 2024-09-19T09:15:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.392+0000 7f5567812740 1 -- 172.21.15.120:0/3561366430 wait complete. 2024-09-19T09:15:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.392+0000 7f5567812740 20 client.19815 trim_cache size 0 max 16384 2024-09-19T09:15:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:15:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:15:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:15:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:15:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.392+0000 7f5564a77640 1 -- 172.21.15.120:0/2189251222 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f55480a32f0 msgr2=0x7f55480a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.392+0000 7f5564a77640 1 --2- 172.21.15.120:0/2189251222 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f55480a32f0 0x7f55480a57b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f555800c7d0 tx=0x7f5558087040 comp rx=0 tx=0).stop 2024-09-19T09:15:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.392+0000 7f5564a77640 1 -- 172.21.15.120:0/2189251222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55601078a0 msgr2=0x7f556019f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.392+0000 7f5564a77640 1 --2- 172.21.15.120:0/2189251222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55601078a0 0x7f556019f450 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f555800eb40 tx=0x7f555800c2d0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.393+0000 7f5564a77640 1 -- 172.21.15.120:0/2189251222 shutdown_connections 2024-09-19T09:15:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.393+0000 7f5564a77640 1 --2- 172.21.15.120:0/2189251222 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f55480a32f0 0x7f55480a57b0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.393+0000 7f5564a77640 1 --2- 172.21.15.120:0/2189251222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55601078a0 0x7f556019f450 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.393+0000 7f5564a77640 1 --2- 172.21.15.120:0/2189251222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5560104f70 0x7f55601983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.393+0000 7f5564a77640 1 --2- 172.21.15.120:0/2189251222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5560102640 0x7f5560197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.393+0000 7f5564a77640 1 -- 172.21.15.120:0/2189251222 >> 172.21.15.120:0/2189251222 conn(0x7f55600fc1e0 msgr2=0x7f5560102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.393+0000 7f5564a77640 1 -- 172.21.15.120:0/2189251222 shutdown_connections 2024-09-19T09:15:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:24.393+0000 7f5564a77640 1 -- 172.21.15.120:0/2189251222 wait complete. 2024-09-19T09:15:25.077 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1253971 9.94 MB/sec execute 560 sec latency 121.839 ms 2024-09-19T09:15:25.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:24 smithi196 ceph-mon[35150]: pgmap v720: 97 pgs: 97 active+clean; 50 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.7 MiB/s wr, 379 op/s 2024-09-19T09:15:25.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:24 smithi153 ceph-mon[35071]: pgmap v720: 97 pgs: 97 active+clean; 50 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.7 MiB/s wr, 379 op/s 2024-09-19T09:15:25.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:24 smithi120 ceph-mon[31150]: pgmap v720: 97 pgs: 97 active+clean; 50 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.7 MiB/s wr, 379 op/s 2024-09-19T09:15:26.077 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1257172 9.96 MB/sec execute 561 sec latency 11.149 ms 2024-09-19T09:15:27.077 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1259685 9.96 MB/sec execute 562 sec latency 11.341 ms 2024-09-19T09:15:27.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:26 smithi196 ceph-mon[35150]: pgmap v721: 97 pgs: 97 active+clean; 46 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.4 MiB/s wr, 340 op/s 2024-09-19T09:15:27.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:26 smithi153 ceph-mon[35071]: pgmap v721: 97 pgs: 97 active+clean; 46 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.4 MiB/s wr, 340 op/s 2024-09-19T09:15:27.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:26 smithi120 ceph-mon[31150]: pgmap v721: 97 pgs: 97 active+clean; 46 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.4 MiB/s wr, 340 op/s 2024-09-19T09:15:28.077 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1262165 9.97 MB/sec execute 563 sec latency 21.588 ms 2024-09-19T09:15:29.077 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1264963 9.98 MB/sec execute 564 sec latency 15.278 ms 2024-09-19T09:15:29.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:28 smithi153 ceph-mon[35071]: pgmap v722: 97 pgs: 97 active+clean; 42 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.2 MiB/s wr, 448 op/s 2024-09-19T09:15:29.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:28 smithi120 ceph-mon[31150]: pgmap v722: 97 pgs: 97 active+clean; 42 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.2 MiB/s wr, 448 op/s 2024-09-19T09:15:29.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:28 smithi196 ceph-mon[35150]: pgmap v722: 97 pgs: 97 active+clean; 42 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.2 MiB/s wr, 448 op/s 2024-09-19T09:15:30.077 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1267840 9.99 MB/sec execute 565 sec latency 107.956 ms 2024-09-19T09:15:30.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:29 smithi120 ceph-mon[31150]: pgmap v723: 97 pgs: 97 active+clean; 42 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.7 MiB/s wr, 402 op/s 2024-09-19T09:15:30.337 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:29 smithi196 ceph-mon[35150]: pgmap v723: 97 pgs: 97 active+clean; 42 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.7 MiB/s wr, 402 op/s 2024-09-19T09:15:30.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:29 smithi153 ceph-mon[35071]: pgmap v723: 97 pgs: 97 active+clean; 42 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.7 MiB/s wr, 402 op/s 2024-09-19T09:15:31.077 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1270289 9.99 MB/sec execute 566 sec latency 11.222 ms 2024-09-19T09:15:32.077 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1272448 9.99 MB/sec execute 567 sec latency 17.063 ms 2024-09-19T09:15:33.077 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1275325 10.01 MB/sec execute 568 sec latency 15.109 ms 2024-09-19T09:15:33.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:32 smithi196 ceph-mon[35150]: pgmap v724: 97 pgs: 97 active+clean; 45 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 394 op/s 2024-09-19T09:15:33.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:32 smithi153 ceph-mon[35071]: pgmap v724: 97 pgs: 97 active+clean; 45 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 394 op/s 2024-09-19T09:15:33.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:32 smithi120 ceph-mon[31150]: pgmap v724: 97 pgs: 97 active+clean; 45 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 394 op/s 2024-09-19T09:15:34.077 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1278422 10.02 MB/sec execute 569 sec latency 12.056 ms 2024-09-19T09:15:35.077 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1280117 10.01 MB/sec execute 570 sec latency 59.536 ms 2024-09-19T09:15:35.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:34 smithi196 ceph-mon[35150]: pgmap v725: 97 pgs: 97 active+clean; 48 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.1 MiB/s wr, 444 op/s 2024-09-19T09:15:35.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:34 smithi153 ceph-mon[35071]: pgmap v725: 97 pgs: 97 active+clean; 48 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.1 MiB/s wr, 444 op/s 2024-09-19T09:15:35.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:34 smithi120 ceph-mon[31150]: pgmap v725: 97 pgs: 97 active+clean; 48 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.1 MiB/s wr, 444 op/s 2024-09-19T09:15:36.077 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1282961 10.02 MB/sec execute 571 sec latency 17.864 ms 2024-09-19T09:15:37.077 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1286129 10.04 MB/sec execute 572 sec latency 11.316 ms 2024-09-19T09:15:37.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:36 smithi196 ceph-mon[35150]: pgmap v726: 97 pgs: 97 active+clean; 47 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 399 op/s 2024-09-19T09:15:37.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:36 smithi153 ceph-mon[35071]: pgmap v726: 97 pgs: 97 active+clean; 47 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 399 op/s 2024-09-19T09:15:37.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:36 smithi120 ceph-mon[31150]: pgmap v726: 97 pgs: 97 active+clean; 47 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 399 op/s 2024-09-19T09:15:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.489+0000 7fb072c4c640 1 -- 172.21.15.120:0/3871026771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb06c100a50 msgr2=0x7fb06c100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.489+0000 7fb072c4c640 1 --2- 172.21.15.120:0/3871026771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb06c100a50 0x7fb06c100e50 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fb060009900 tx=0x7fb06002a410 comp rx=0 tx=0).stop 2024-09-19T09:15:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.489+0000 7fb072c4c640 1 -- 172.21.15.120:0/3871026771 shutdown_connections 2024-09-19T09:15:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.489+0000 7fb072c4c640 1 --2- 172.21.15.120:0/3871026771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb06c102600 0x7fb06c108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.489+0000 7fb072c4c640 1 --2- 172.21.15.120:0/3871026771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb06c101c40 0x7fb06c1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.489+0000 7fb072c4c640 1 --2- 172.21.15.120:0/3871026771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb06c100a50 0x7fb06c100e50 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.489+0000 7fb072c4c640 1 -- 172.21.15.120:0/3871026771 >> 172.21.15.120:0/3871026771 conn(0x7fb06c0fc1b0 msgr2=0x7fb06c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.489+0000 7fb072c4c640 1 -- 172.21.15.120:0/3871026771 shutdown_connections 2024-09-19T09:15:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.489+0000 7fb072c4c640 1 -- 172.21.15.120:0/3871026771 wait complete. 2024-09-19T09:15:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.490+0000 7fb072c4c640 1 Processor -- start 2024-09-19T09:15:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.490+0000 7fb072c4c640 1 -- start start 2024-09-19T09:15:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.490+0000 7fb072c4c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb06c100a50 0x7fb06c1049e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.490+0000 7fb072c4c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb06c101c40 0x7fb06c103020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.490+0000 7fb072c4c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb06c102600 0x7fb06c103560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.490+0000 7fb072c4c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c10c6b0 con 0x7fb06c101c40 2024-09-19T09:15:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.490+0000 7fb072c4c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c10c530 con 0x7fb06c100a50 2024-09-19T09:15:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.490+0000 7fb072c4c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c10c830 con 0x7fb06c102600 2024-09-19T09:15:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.490+0000 7fb06bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb06c101c40 0x7fb06c103020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.490+0000 7fb06bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb06c101c40 0x7fb06c103020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52064/0 (socket says 172.21.15.120:52064) 2024-09-19T09:15:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.490+0000 7fb0709e8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb06c100a50 0x7fb06c1049e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.490+0000 7fb06bfff640 1 -- 172.21.15.120:0/558857162 learned_addr learned my addr 172.21.15.120:0/558857162 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.490+0000 7fb0711e9640 1 --2- 172.21.15.120:0/558857162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb06c102600 0x7fb06c103560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.490+0000 7fb06bfff640 1 -- 172.21.15.120:0/558857162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb06c102600 msgr2=0x7fb06c103560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.490+0000 7fb06bfff640 1 --2- 172.21.15.120:0/558857162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb06c102600 0x7fb06c103560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.490+0000 7fb06bfff640 1 -- 172.21.15.120:0/558857162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb06c100a50 msgr2=0x7fb06c1049e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.490+0000 7fb06bfff640 1 --2- 172.21.15.120:0/558857162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb06c100a50 0x7fb06c1049e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.490+0000 7fb06bfff640 1 -- 172.21.15.120:0/558857162 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06c103df0 con 0x7fb06c101c40 2024-09-19T09:15:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.491+0000 7fb06bfff640 1 --2- 172.21.15.120:0/558857162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb06c101c40 0x7fb06c103020 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fb05800ea60 tx=0x7fb05800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.491+0000 7fb069ffb640 1 -- 172.21.15.120:0/558857162 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05800cde0 con 0x7fb06c101c40 2024-09-19T09:15:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.491+0000 7fb069ffb640 1 -- 172.21.15.120:0/558857162 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0580035f0 con 0x7fb06c101c40 2024-09-19T09:15:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.491+0000 7fb072c4c640 1 -- 172.21.15.120:0/558857162 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb06c1a1f40 con 0x7fb06c101c40 2024-09-19T09:15:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.491+0000 7fb069ffb640 1 -- 172.21.15.120:0/558857162 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb058010670 con 0x7fb06c101c40 2024-09-19T09:15:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.491+0000 7fb072c4c640 1 -- 172.21.15.120:0/558857162 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb06c1a23b0 con 0x7fb06c101c40 2024-09-19T09:15:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.493+0000 7fb069ffb640 1 -- 172.21.15.120:0/558857162 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fb0580037a0 con 0x7fb06c101c40 2024-09-19T09:15:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.494+0000 7fb0759e7740 1 Processor -- start 2024-09-19T09:15:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.494+0000 7fb0759e7740 1 -- start start 2024-09-19T09:15:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.494+0000 7fb0759e7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d871ae79a0 0x55d871ae7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.494+0000 7fb0759e7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d871ae82e0 0x55d871aecda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.494+0000 7fb0759e7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d871aed370 0x55d871aef830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.494+0000 7fb0759e7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d871a4b2f0 con 0x55d871ae82e0 2024-09-19T09:15:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.494+0000 7fb0759e7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d8719f3270 con 0x55d871aed370 2024-09-19T09:15:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.494+0000 7fb069ffb640 1 --2- 172.21.15.120:0/558857162 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb05c0a3370 0x7fb05c0a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.494+0000 7fb0759e7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d871a472b0 con 0x55d871ae79a0 2024-09-19T09:15:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.494+0000 7fb0709e8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d871ae82e0 0x55d871aecda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.494+0000 7fb0709e8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d871ae82e0 0x55d871aecda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52080/0 (socket says 172.21.15.120:52080) 2024-09-19T09:15:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.494+0000 7fb0709e8640 1 -- 172.21.15.120:0/3314259124 learned_addr learned my addr 172.21.15.120:0/3314259124 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.494+0000 7fb06bfff640 1 --2- 172.21.15.120:0/3314259124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d871aed370 0x55d871aef830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.494+0000 7fb0711e9640 1 --2- 172.21.15.120:0/558857162 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb05c0a3370 0x7fb05c0a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.494+0000 7fb0711e9640 1 --2- 172.21.15.120:0/3314259124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d871ae79a0 0x55d871ae7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.494+0000 7fb069ffb640 1 -- 172.21.15.120:0/558857162 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fb0580cfb20 con 0x7fb06c101c40 2024-09-19T09:15:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.494+0000 7fb0709e8640 1 -- 172.21.15.120:0/3314259124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d871ae79a0 msgr2=0x55d871ae7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.494+0000 7fb0709e8640 1 --2- 172.21.15.120:0/3314259124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d871ae79a0 0x55d871ae7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.494+0000 7fb0709e8640 1 -- 172.21.15.120:0/3314259124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d871aed370 msgr2=0x55d871aef830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.494+0000 7fb0709e8640 1 --2- 172.21.15.120:0/3314259124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d871aed370 0x55d871aef830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.494+0000 7fb0709e8640 1 -- 172.21.15.120:0/3314259124 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8719f2340 con 0x55d871ae82e0 2024-09-19T09:15:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.495+0000 7fb0711e9640 1 --2- 172.21.15.120:0/558857162 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb05c0a3370 0x7fb05c0a5830 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb064005fd0 tx=0x7fb064009be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.495+0000 7fb0709e8640 1 --2- 172.21.15.120:0/3314259124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d871ae82e0 0x55d871aecda0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fb060031610 tx=0x7fb060005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5dc9a0674ace36 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.495+0000 7fb056ffd640 1 -- 172.21.15.120:0/3314259124 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060040070 con 0x55d871ae82e0 2024-09-19T09:15:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.495+0000 7fb056ffd640 1 -- 172.21.15.120:0/3314259124 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb060004030 con 0x55d871ae82e0 2024-09-19T09:15:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.495+0000 7fb056ffd640 1 -- 172.21.15.120:0/3314259124 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060035040 con 0x55d871ae82e0 2024-09-19T09:15:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.496+0000 7fb0759e7740 1 -- 172.21.15.120:0/3314259124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d871ae82e0 msgr2=0x55d871aecda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.496+0000 7fb0759e7740 1 --2- 172.21.15.120:0/3314259124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d871ae82e0 0x55d871aecda0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fb060031610 tx=0x7fb060005de0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.496+0000 7fb0759e7740 1 -- 172.21.15.120:0/3314259124 shutdown_connections 2024-09-19T09:15:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.496+0000 7fb0759e7740 1 --2- 172.21.15.120:0/3314259124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d871aed370 0x55d871aef830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.496+0000 7fb0759e7740 1 --2- 172.21.15.120:0/3314259124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d871ae82e0 0x55d871aecda0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.496+0000 7fb0759e7740 1 --2- 172.21.15.120:0/3314259124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d871ae79a0 0x55d871ae7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.496+0000 7fb0759e7740 1 -- 172.21.15.120:0/3314259124 >> 172.21.15.120:0/3314259124 conn(0x55d871ae3010 msgr2=0x55d871ae3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.496+0000 7fb0759e7740 1 -- 172.21.15.120:0/3314259124 shutdown_connections 2024-09-19T09:15:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.496+0000 7fb0759e7740 1 -- 172.21.15.120:0/3314259124 wait complete. 2024-09-19T09:15:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.496+0000 7fb0759e7740 1 -- start start 2024-09-19T09:15:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.496+0000 7fb0759e7740 1 Processor -- start 2024-09-19T09:15:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.496+0000 7fb0759e7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d871ae79a0 0x55d871af1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.496+0000 7fb0759e7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d871ae82e0 0x55d871af20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.497+0000 7fb0759e7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d871aed370 0x55d871af90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.497+0000 7fb0759e7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d871af02e0 con 0x55d871ae79a0 2024-09-19T09:15:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.497+0000 7fb0759e7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d8719d4290 con 0x55d871ae82e0 2024-09-19T09:15:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.497+0000 7fb0759e7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d871af0460 con 0x55d871aed370 2024-09-19T09:15:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.497+0000 7fb0759e7740 10 client.0 _ll_register_callbacks cb 0x55d871a05140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.497+0000 7fb0709e8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d871ae79a0 0x55d871af1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.497+0000 7fb06bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d871ae82e0 0x55d871af20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.497+0000 7fb0709e8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d871ae79a0 0x55d871af1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52082/0 (socket says 172.21.15.120:52082) 2024-09-19T09:15:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.497+0000 7fb0711e9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d871aed370 0x55d871af90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.497+0000 7fb0709e8640 1 -- 172.21.15.120:0/4211285473 learned_addr learned my addr 172.21.15.120:0/4211285473 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.497+0000 7fb0711e9640 1 -- 172.21.15.120:0/4211285473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d871ae82e0 msgr2=0x55d871af20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.497+0000 7fb0711e9640 1 --2- 172.21.15.120:0/4211285473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d871ae82e0 0x55d871af20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.497+0000 7fb0711e9640 1 -- 172.21.15.120:0/4211285473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d871ae79a0 msgr2=0x55d871af1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.497+0000 7fb0711e9640 1 --2- 172.21.15.120:0/4211285473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d871ae79a0 0x55d871af1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.497+0000 7fb0711e9640 1 -- 172.21.15.120:0/4211285473 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d87175e210 con 0x55d871aed370 2024-09-19T09:15:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.497+0000 7fb0711e9640 1 --2- 172.21.15.120:0/4211285473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d871aed370 0x55d871af90e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb064009fd0 tx=0x7fb064004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa5e1917660f522 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.497+0000 7fb04bfff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:15:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.497+0000 7fb04bfff640 1 -- 172.21.15.120:0/4211285473 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb064026070 con 0x55d871aed370 2024-09-19T09:15:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.497+0000 7fb0759e7740 1 -- 172.21.15.120:0/4211285473 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb060004030 con 0x55d871aed370 2024-09-19T09:15:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.497+0000 7fb04bfff640 1 -- 172.21.15.120:0/4211285473 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fb06400fdb0 con 0x55d871aed370 2024-09-19T09:15:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.497+0000 7fb04bfff640 1 -- 172.21.15.120:0/4211285473 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb06400e880 con 0x55d871aed370 2024-09-19T09:15:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.497+0000 7fb0759e7740 10 client.28660 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.497+0000 7fb0759e7740 1 -- 172.21.15.120:0/4211285473 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d87182a620 con 0x55d871aed370 2024-09-19T09:15:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.498+0000 7fb04bfff640 1 -- 172.21.15.120:0/4211285473 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb060004030 con 0x55d871aed370 2024-09-19T09:15:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.498+0000 7fb04bfff640 1 -- 172.21.15.120:0/4211285473 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fb06401d040 con 0x55d871aed370 2024-09-19T09:15:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.498+0000 7fb0759e7740 10 client.28660 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.498+0000 7fb0759e7740 10 client.28660 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:15:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.498+0000 7fb0759e7740 20 client.28660 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.498+0000 7fb0759e7740 1 --2- 172.21.15.120:0/4211285473 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d871afd7f0 0x55d871affc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.498+0000 7fb0759e7740 4 client.28660 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:15:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.498+0000 7fb0759e7740 1 -- 172.21.15.120:0/4211285473 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d871a58a20 con 0x55d871afd7f0 2024-09-19T09:15:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.498+0000 7fb0709e8640 1 --2- 172.21.15.120:0/4211285473 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d871afd7f0 0x55d871affc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.498+0000 7fb04bfff640 10 client.28660 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.498+0000 7fb0709e8640 1 --2- 172.21.15.120:0/4211285473 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d871afd7f0 0x55d871affc80 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92ca509af5bcffeb server_cookie=f9a55f5ec99f00ac in_seq=0 out_seq=0 2024-09-19T09:15:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.499+0000 7fb04bfff640 1 -- 172.21.15.120:0/4211285473 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d871a58a20 con 0x55d871afd7f0 2024-09-19T09:15:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.499+0000 7fb04bfff640 10 client.28660 handle_command_reply: tid=0 2024-09-19T09:15:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.499+0000 7fb0759e7740 1 client.28660 shutdown 2024-09-19T09:15:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.499+0000 7fb0759e7740 1 -- 172.21.15.120:0/4211285473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d871aed370 msgr2=0x55d871af90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.499+0000 7fb0759e7740 1 --2- 172.21.15.120:0/4211285473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d871aed370 0x55d871af90e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb064009fd0 tx=0x7fb064004ab0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.499+0000 7fb0759e7740 1 -- 172.21.15.120:0/4211285473 shutdown_connections 2024-09-19T09:15:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.499+0000 7fb0759e7740 1 --2- 172.21.15.120:0/4211285473 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d871afd7f0 0x55d871affc80 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.499+0000 7fb0759e7740 1 --2- 172.21.15.120:0/4211285473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d871aed370 0x55d871af90e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.499+0000 7fb0759e7740 1 --2- 172.21.15.120:0/4211285473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d871ae82e0 0x55d871af20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.499+0000 7fb0759e7740 1 --2- 172.21.15.120:0/4211285473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d871ae79a0 0x55d871af1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.499+0000 7fb04bfff640 0 client.28660 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.499+0000 7fb0759e7740 1 -- 172.21.15.120:0/4211285473 >> 172.21.15.120:0/4211285473 conn(0x55d871ae3010 msgr2=0x55d871ae4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.500+0000 7fb0759e7740 1 -- 172.21.15.120:0/4211285473 shutdown_connections 2024-09-19T09:15:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.500+0000 7fb0759e7740 1 -- 172.21.15.120:0/4211285473 wait complete. 2024-09-19T09:15:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.500+0000 7fb0759e7740 20 client.28660 trim_cache size 0 max 16384 2024-09-19T09:15:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.515+0000 7fb0759e7740 1 Processor -- start 2024-09-19T09:15:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.515+0000 7fb0759e7740 1 -- start start 2024-09-19T09:15:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.515+0000 7fb0759e7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d871ae79a0 0x55d871ae4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.515+0000 7fb0759e7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d871ae82e0 0x55d871af9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.515+0000 7fb0759e7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d871aed370 0x55d871b04580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.515+0000 7fb0759e7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1ab2b0 con 0x55d871ae82e0 2024-09-19T09:15:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.515+0000 7fb0759e7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1acdf0 con 0x55d871aed370 2024-09-19T09:15:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.515+0000 7fb0759e7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1ab980 con 0x55d871ae79a0 2024-09-19T09:15:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.515+0000 7fb06bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d871ae82e0 0x55d871af9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.515+0000 7fb06bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d871ae82e0 0x55d871af9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52088/0 (socket says 172.21.15.120:52088) 2024-09-19T09:15:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.515+0000 7fb06bfff640 1 -- 172.21.15.120:0/890372361 learned_addr learned my addr 172.21.15.120:0/890372361 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.515+0000 7fb0709e8640 1 --2- 172.21.15.120:0/890372361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d871ae79a0 0x55d871ae4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.515+0000 7fb0711e9640 1 --2- 172.21.15.120:0/890372361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d871aed370 0x55d871b04580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.515+0000 7fb06bfff640 1 -- 172.21.15.120:0/890372361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d871ae79a0 msgr2=0x55d871ae4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.515+0000 7fb06bfff640 1 --2- 172.21.15.120:0/890372361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d871ae79a0 0x55d871ae4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.515+0000 7fb06bfff640 1 -- 172.21.15.120:0/890372361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d871aed370 msgr2=0x55d871b04580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.515+0000 7fb06bfff640 1 --2- 172.21.15.120:0/890372361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d871aed370 0x55d871b04580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.515+0000 7fb06bfff640 1 -- 172.21.15.120:0/890372361 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d871af16a0 con 0x55d871ae82e0 2024-09-19T09:15:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.516+0000 7fb06bfff640 1 --2- 172.21.15.120:0/890372361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d871ae82e0 0x55d871af9000 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fb05800ea30 tx=0x7fb0580bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537f7700dcbf4589 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.516+0000 7fb0567fc640 1 -- 172.21.15.120:0/890372361 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05809c140 con 0x55d871ae82e0 2024-09-19T09:15:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.516+0000 7fb0567fc640 1 -- 172.21.15.120:0/890372361 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0580a04f0 con 0x55d871ae82e0 2024-09-19T09:15:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.516+0000 7fb0567fc640 1 -- 172.21.15.120:0/890372361 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb058098910 con 0x55d871ae82e0 2024-09-19T09:15:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.516+0000 7fb0759e7740 1 -- 172.21.15.120:0/890372361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d871ae82e0 msgr2=0x55d871af9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.516+0000 7fb0759e7740 1 --2- 172.21.15.120:0/890372361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d871ae82e0 0x55d871af9000 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fb05800ea30 tx=0x7fb0580bc040 comp rx=0 tx=0).stop 2024-09-19T09:15:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.517+0000 7fb0759e7740 1 -- 172.21.15.120:0/890372361 shutdown_connections 2024-09-19T09:15:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.517+0000 7fb0759e7740 1 --2- 172.21.15.120:0/890372361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d871aed370 0x55d871b04580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.517+0000 7fb0759e7740 1 --2- 172.21.15.120:0/890372361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d871ae82e0 0x55d871af9000 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.517+0000 7fb0759e7740 1 --2- 172.21.15.120:0/890372361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d871ae79a0 0x55d871ae4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.517+0000 7fb0759e7740 1 -- 172.21.15.120:0/890372361 >> 172.21.15.120:0/890372361 conn(0x55d871ae3010 msgr2=0x55d871ae33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.517+0000 7fb0759e7740 1 -- 172.21.15.120:0/890372361 shutdown_connections 2024-09-19T09:15:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.517+0000 7fb0759e7740 1 -- 172.21.15.120:0/890372361 wait complete. 2024-09-19T09:15:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.517+0000 7fb0759e7740 1 -- start start 2024-09-19T09:15:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.517+0000 7fb0759e7740 1 Processor -- start 2024-09-19T09:15:37.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.517+0000 7fb0759e7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d871ae79a0 0x55d871af6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:37.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.517+0000 7fb0759e7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d871ae82e0 0x55d871af72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:37.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.517+0000 7fb0759e7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d871aed370 0x55d871af82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:37.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.517+0000 7fb0759e7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1ac720 con 0x55d871ae82e0 2024-09-19T09:15:37.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.517+0000 7fb0759e7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1ac050 con 0x55d871aed370 2024-09-19T09:15:37.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.517+0000 7fb0759e7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1a9270 con 0x55d871ae79a0 2024-09-19T09:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.517+0000 7fb0759e7740 10 client.0 _ll_register_callbacks cb 0x7fb064011560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.517+0000 7fb06bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d871ae82e0 0x55d871af72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.518+0000 7fb0709e8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d871ae79a0 0x55d871af6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.518+0000 7fb0711e9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d871aed370 0x55d871af82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.518+0000 7fb0711e9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d871aed370 0x55d871af82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37858/0 (socket says 172.21.15.120:37858) 2024-09-19T09:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.518+0000 7fb06bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d871ae82e0 0x55d871af72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52090/0 (socket says 172.21.15.120:52090) 2024-09-19T09:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.518+0000 7fb0709e8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d871ae79a0 0x55d871af6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:43628/0 (socket says 172.21.15.120:43628) 2024-09-19T09:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.518+0000 7fb06bfff640 1 -- 172.21.15.120:0/1946714413 learned_addr learned my addr 172.21.15.120:0/1946714413 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.518+0000 7fb06bfff640 1 -- 172.21.15.120:0/1946714413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d871ae79a0 msgr2=0x55d871af6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.518+0000 7fb06bfff640 1 --2- 172.21.15.120:0/1946714413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d871ae79a0 0x55d871af6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.518+0000 7fb06bfff640 1 -- 172.21.15.120:0/1946714413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d871aed370 msgr2=0x55d871af82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.518+0000 7fb06bfff640 1 --2- 172.21.15.120:0/1946714413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d871aed370 0x55d871af82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.518+0000 7fb06bfff640 1 -- 172.21.15.120:0/1946714413 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d871af88f0 con 0x55d871ae82e0 2024-09-19T09:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.518+0000 7fb0711e9640 1 --2- 172.21.15.120:0/1946714413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d871aed370 0x55d871af82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.518+0000 7fb0709e8640 1 --2- 172.21.15.120:0/1946714413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d871ae79a0 0x55d871af6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.518+0000 7fb06bfff640 1 --2- 172.21.15.120:0/1946714413 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d871ae82e0 0x55d871af72d0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fb05801f000 tx=0x7fb058003b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44053c7dd7790125 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.518+0000 7fb04bfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.518+0000 7fb04bfff640 1 -- 172.21.15.120:0/1946714413 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05809c140 con 0x55d871ae82e0 2024-09-19T09:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.518+0000 7fb04bfff640 1 -- 172.21.15.120:0/1946714413 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fb0580a0160 con 0x55d871ae82e0 2024-09-19T09:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.518+0000 7fb0759e7740 1 -- 172.21.15.120:0/1946714413 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0580a04f0 con 0x55d871ae82e0 2024-09-19T09:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.518+0000 7fb04bfff640 1 -- 172.21.15.120:0/1946714413 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb0580a4db0 con 0x55d871ae82e0 2024-09-19T09:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.518+0000 7fb0759e7740 10 client.19833 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.518+0000 7fb0759e7740 1 -- 172.21.15.120:0/1946714413 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d871b68a80 con 0x55d871ae82e0 2024-09-19T09:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.519+0000 7fb04bfff640 1 -- 172.21.15.120:0/1946714413 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0580a04f0 con 0x55d871ae82e0 2024-09-19T09:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.519+0000 7fb04bfff640 1 -- 172.21.15.120:0/1946714413 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fb0580cdae0 con 0x55d871ae82e0 2024-09-19T09:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.519+0000 7fb0759e7740 10 client.19833 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.519+0000 7fb0759e7740 10 client.19833 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:15:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.519+0000 7fb0759e7740 20 client.19833 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.519+0000 7fb0759e7740 1 --2- 172.21.15.120:0/1946714413 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d871b693b0 0x55d871b6b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.519+0000 7fb0759e7740 4 client.19833 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:15:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.519+0000 7fb0759e7740 1 -- 172.21.15.120:0/1946714413 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d871a52d70 con 0x55d871b693b0 2024-09-19T09:15:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.519+0000 7fb0709e8640 1 --2- 172.21.15.120:0/1946714413 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d871b693b0 0x55d871b6b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.519+0000 7fb04bfff640 10 client.19833 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.519+0000 7fb0709e8640 1 --2- 172.21.15.120:0/1946714413 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d871b693b0 0x55d871b6b7d0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38ea836d750bc3d5 server_cookie=96bf254463553979 in_seq=0 out_seq=0 2024-09-19T09:15:37.548 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:15:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.713+0000 7fb04bfff640 1 -- 172.21.15.120:0/1946714413 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d871a52d70 con 0x55d871b693b0 2024-09-19T09:15:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.713+0000 7fb04bfff640 10 client.19833 handle_command_reply: tid=0 2024-09-19T09:15:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.713+0000 7fb0759e7740 1 client.19833 shutdown 2024-09-19T09:15:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.714+0000 7fb0759e7740 1 -- 172.21.15.120:0/1946714413 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d871ae82e0 msgr2=0x55d871af72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.714+0000 7fb0759e7740 1 --2- 172.21.15.120:0/1946714413 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d871ae82e0 0x55d871af72d0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fb05801f000 tx=0x7fb058003b50 comp rx=0 tx=0).stop 2024-09-19T09:15:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.714+0000 7fb0759e7740 1 -- 172.21.15.120:0/1946714413 shutdown_connections 2024-09-19T09:15:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.714+0000 7fb0759e7740 1 --2- 172.21.15.120:0/1946714413 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d871b693b0 0x55d871b6b7d0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.714+0000 7fb0759e7740 1 --2- 172.21.15.120:0/1946714413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d871aed370 0x55d871af82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.714+0000 7fb0759e7740 1 --2- 172.21.15.120:0/1946714413 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d871ae82e0 0x55d871af72d0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.714+0000 7fb0759e7740 1 --2- 172.21.15.120:0/1946714413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d871ae79a0 0x55d871af6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.714+0000 7fb0759e7740 1 -- 172.21.15.120:0/1946714413 >> 172.21.15.120:0/1946714413 conn(0x55d871ae3010 msgr2=0x55d871ae33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.714+0000 7fb04bfff640 0 client.19833 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.714+0000 7fb0759e7740 1 -- 172.21.15.120:0/1946714413 shutdown_connections 2024-09-19T09:15:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.714+0000 7fb0759e7740 1 -- 172.21.15.120:0/1946714413 wait complete. 2024-09-19T09:15:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.714+0000 7fb0759e7740 20 client.19833 trim_cache size 0 max 16384 2024-09-19T09:15:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:15:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:15:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:15:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:15:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.715+0000 7fb072c4c640 1 -- 172.21.15.120:0/558857162 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb05c0a3370 msgr2=0x7fb05c0a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.715+0000 7fb072c4c640 1 --2- 172.21.15.120:0/558857162 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb05c0a3370 0x7fb05c0a5830 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb064005fd0 tx=0x7fb064009be0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.716+0000 7fb072c4c640 1 -- 172.21.15.120:0/558857162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb06c101c40 msgr2=0x7fb06c103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.716+0000 7fb072c4c640 1 --2- 172.21.15.120:0/558857162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb06c101c40 0x7fb06c103020 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fb05800ea60 tx=0x7fb05800ef20 comp rx=0 tx=0).stop 2024-09-19T09:15:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.716+0000 7fb072c4c640 1 -- 172.21.15.120:0/558857162 shutdown_connections 2024-09-19T09:15:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.716+0000 7fb072c4c640 1 --2- 172.21.15.120:0/558857162 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb05c0a3370 0x7fb05c0a5830 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.716+0000 7fb072c4c640 1 --2- 172.21.15.120:0/558857162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb06c102600 0x7fb06c103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.716+0000 7fb072c4c640 1 --2- 172.21.15.120:0/558857162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb06c101c40 0x7fb06c103020 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.716+0000 7fb072c4c640 1 --2- 172.21.15.120:0/558857162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb06c100a50 0x7fb06c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.716+0000 7fb072c4c640 1 -- 172.21.15.120:0/558857162 >> 172.21.15.120:0/558857162 conn(0x7fb06c0fc1b0 msgr2=0x7fb06c0fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.716+0000 7fb072c4c640 1 -- 172.21.15.120:0/558857162 shutdown_connections 2024-09-19T09:15:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:37.716+0000 7fb072c4c640 1 -- 172.21.15.120:0/558857162 wait complete. 2024-09-19T09:15:37.865 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:15:38.077 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1288773 10.04 MB/sec execute 573 sec latency 82.554 ms 2024-09-19T09:15:38.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.746+0000 7f8c6af3a640 1 -- 172.21.15.120:0/617795786 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c64100a80 msgr2=0x7f8c64100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:38.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.746+0000 7f8c6af3a640 1 --2- 172.21.15.120:0/617795786 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c64100a80 0x7f8c64100ee0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f8c50009930 tx=0x7f8c5002a420 comp rx=0 tx=0).stop 2024-09-19T09:15:38.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.746+0000 7f8c6af3a640 1 -- 172.21.15.120:0/617795786 shutdown_connections 2024-09-19T09:15:38.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.746+0000 7f8c6af3a640 1 --2- 172.21.15.120:0/617795786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c64101420 0x7f8c6410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.746+0000 7f8c6af3a640 1 --2- 172.21.15.120:0/617795786 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c64100a80 0x7f8c64100ee0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.746+0000 7f8c6af3a640 1 --2- 172.21.15.120:0/617795786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c64106a80 0x7f8c64106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.746+0000 7f8c6af3a640 1 -- 172.21.15.120:0/617795786 >> 172.21.15.120:0/617795786 conn(0x7f8c640fc7a0 msgr2=0x7f8c640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:38.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.746+0000 7f8c6af3a640 1 -- 172.21.15.120:0/617795786 shutdown_connections 2024-09-19T09:15:38.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.746+0000 7f8c6af3a640 1 -- 172.21.15.120:0/617795786 wait complete. 2024-09-19T09:15:38.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.746+0000 7f8c6af3a640 1 Processor -- start 2024-09-19T09:15:38.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.747+0000 7f8c6af3a640 1 -- start start 2024-09-19T09:15:38.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.747+0000 7f8c6af3a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c64100a80 0x7f8c641992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:38.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.747+0000 7f8c6af3a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c64101420 0x7f8c64199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:38.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.747+0000 7f8c6af3a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c64106a80 0x7f8c641a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:38.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.747+0000 7f8c6af3a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8c64110e40 con 0x7f8c64106a80 2024-09-19T09:15:38.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.747+0000 7f8c6af3a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8c64110cc0 con 0x7f8c64100a80 2024-09-19T09:15:38.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.747+0000 7f8c6af3a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8c64110fc0 con 0x7f8c64101420 2024-09-19T09:15:38.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.747+0000 7f8c694d7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c64106a80 0x7f8c641a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:38.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.747+0000 7f8c63fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c64101420 0x7f8c64199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:38.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.747+0000 7f8c68cd6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c64100a80 0x7f8c641992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:38.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.747+0000 7f8c694d7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c64106a80 0x7f8c641a4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52120/0 (socket says 172.21.15.120:52120) 2024-09-19T09:15:38.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.747+0000 7f8c694d7640 1 -- 172.21.15.120:0/2380259312 learned_addr learned my addr 172.21.15.120:0/2380259312 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:38.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.747+0000 7f8c694d7640 1 -- 172.21.15.120:0/2380259312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c64101420 msgr2=0x7f8c64199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:38.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.747+0000 7f8c694d7640 1 --2- 172.21.15.120:0/2380259312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c64101420 0x7f8c64199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.747+0000 7f8c694d7640 1 -- 172.21.15.120:0/2380259312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c64100a80 msgr2=0x7f8c641992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:38.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.747+0000 7f8c694d7640 1 --2- 172.21.15.120:0/2380259312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c64100a80 0x7f8c641992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.747+0000 7f8c694d7640 1 -- 172.21.15.120:0/2380259312 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c641a48b0 con 0x7f8c64106a80 2024-09-19T09:15:38.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.747+0000 7f8c694d7640 1 --2- 172.21.15.120:0/2380259312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c64106a80 0x7f8c641a4370 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c00eb10 tx=0x7f8c5c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:38.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.747+0000 7f8c61ffb640 1 -- 172.21.15.120:0/2380259312 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c004b20 con 0x7f8c64106a80 2024-09-19T09:15:38.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.747+0000 7f8c61ffb640 1 -- 172.21.15.120:0/2380259312 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c5c004cc0 con 0x7f8c64106a80 2024-09-19T09:15:38.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.747+0000 7f8c6af3a640 1 -- 172.21.15.120:0/2380259312 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c641a4a80 con 0x7f8c64106a80 2024-09-19T09:15:38.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.748+0000 7f8c61ffb640 1 -- 172.21.15.120:0/2380259312 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c010760 con 0x7f8c64106a80 2024-09-19T09:15:38.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.748+0000 7f8c6af3a640 1 -- 172.21.15.120:0/2380259312 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c641a4f40 con 0x7f8c64106a80 2024-09-19T09:15:38.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.751+0000 7f8c61ffb640 1 -- 172.21.15.120:0/2380259312 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f8c5c005020 con 0x7f8c64106a80 2024-09-19T09:15:38.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.751+0000 7f8c6d313740 1 Processor -- start 2024-09-19T09:15:38.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.751+0000 7f8c6d313740 1 -- start start 2024-09-19T09:15:38.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.751+0000 7f8c6d313740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556f2b48b3a0 0x556f2b48b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:38.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.751+0000 7f8c6d313740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f2b48bcc0 0x556f2b49a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:38.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.751+0000 7f8c6d313740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f2b49aa90 0x556f2b49ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:38.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.751+0000 7f8c6d313740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556f2b3f12e0 con 0x556f2b48b3a0 2024-09-19T09:15:38.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.751+0000 7f8c6d313740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556f2b398ce0 con 0x556f2b48bcc0 2024-09-19T09:15:38.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.751+0000 7f8c6d313740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556f2b3ed2a0 con 0x556f2b49aa90 2024-09-19T09:15:38.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.751+0000 7f8c63fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556f2b48b3a0 0x556f2b48b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:38.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.751+0000 7f8c68cd6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f2b48bcc0 0x556f2b49a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:38.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.751+0000 7f8c61ffb640 1 --2- 172.21.15.120:0/2380259312 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8c540a3240 0x7f8c540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:38.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.751+0000 7f8c63fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556f2b48b3a0 0x556f2b48b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52122/0 (socket says 172.21.15.120:52122) 2024-09-19T09:15:38.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.751+0000 7f8c63fff640 1 -- 172.21.15.120:0/2922110447 learned_addr learned my addr 172.21.15.120:0/2922110447 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:38.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.751+0000 7f8c63fff640 1 --2- 172.21.15.120:0/2922110447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f2b49aa90 0x556f2b49ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:38.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.751+0000 7f8c694d7640 1 --2- 172.21.15.120:0/2380259312 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8c540a3240 0x7f8c540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:38.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.751+0000 7f8c68cd6640 1 -- 172.21.15.120:0/2922110447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f2b49aa90 msgr2=0x556f2b49ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:38.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.752+0000 7f8c68cd6640 1 --2- 172.21.15.120:0/2922110447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f2b49aa90 0x556f2b49ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.752+0000 7f8c68cd6640 1 -- 172.21.15.120:0/2922110447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556f2b48b3a0 msgr2=0x556f2b48b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:38.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.752+0000 7f8c68cd6640 1 --2- 172.21.15.120:0/2922110447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556f2b48b3a0 0x556f2b48b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.752+0000 7f8c68cd6640 1 -- 172.21.15.120:0/2922110447 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f2b37d440 con 0x556f2b48bcc0 2024-09-19T09:15:38.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.752+0000 7f8c61ffb640 1 -- 172.21.15.120:0/2380259312 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f8c5c0cff00 con 0x7f8c64106a80 2024-09-19T09:15:38.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.752+0000 7f8c63fff640 1 --2- 172.21.15.120:0/2922110447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556f2b48b3a0 0x556f2b48b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:15:38.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.752+0000 7f8c68cd6640 1 --2- 172.21.15.120:0/2922110447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f2b48bcc0 0x556f2b49a550 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8c58009c20 tx=0x7f8c58011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1897eee9db2d2885 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:38.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.752+0000 7f8c694d7640 1 --2- 172.21.15.120:0/2380259312 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8c540a3240 0x7f8c540a5700 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c0049d0 tx=0x7f8c5c06f360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:38.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.752+0000 7f8c4effd640 1 -- 172.21.15.120:0/2922110447 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c58004050 con 0x556f2b48bcc0 2024-09-19T09:15:38.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.752+0000 7f8c4effd640 1 -- 172.21.15.120:0/2922110447 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c58005500 con 0x556f2b48bcc0 2024-09-19T09:15:38.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.752+0000 7f8c4effd640 1 -- 172.21.15.120:0/2922110447 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c58004d50 con 0x556f2b48bcc0 2024-09-19T09:15:38.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.753+0000 7f8c6d313740 1 -- 172.21.15.120:0/2922110447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f2b48bcc0 msgr2=0x556f2b49a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:38.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.753+0000 7f8c6d313740 1 --2- 172.21.15.120:0/2922110447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f2b48bcc0 0x556f2b49a550 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8c58009c20 tx=0x7f8c58011810 comp rx=0 tx=0).stop 2024-09-19T09:15:38.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.753+0000 7f8c6d313740 1 -- 172.21.15.120:0/2922110447 shutdown_connections 2024-09-19T09:15:38.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.753+0000 7f8c6d313740 1 --2- 172.21.15.120:0/2922110447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f2b49aa90 0x556f2b49ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.753+0000 7f8c6d313740 1 --2- 172.21.15.120:0/2922110447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f2b48bcc0 0x556f2b49a550 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.753+0000 7f8c6d313740 1 --2- 172.21.15.120:0/2922110447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556f2b48b3a0 0x556f2b48b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.753+0000 7f8c6d313740 1 -- 172.21.15.120:0/2922110447 >> 172.21.15.120:0/2922110447 conn(0x556f2b487a30 msgr2=0x556f2b487e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:38.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.753+0000 7f8c6d313740 1 -- 172.21.15.120:0/2922110447 shutdown_connections 2024-09-19T09:15:38.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.753+0000 7f8c6d313740 1 -- 172.21.15.120:0/2922110447 wait complete. 2024-09-19T09:15:38.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.753+0000 7f8c6d313740 1 -- start start 2024-09-19T09:15:38.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.753+0000 7f8c6d313740 1 Processor -- start 2024-09-19T09:15:38.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.754+0000 7f8c6d313740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f2b48b3a0 0x556f2b49a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:38.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.754+0000 7f8c6d313740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556f2b48bcc0 0x556f2b497600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:38.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.754+0000 7f8c6d313740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f2b49aa90 0x556f2b497b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:38.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.754+0000 7f8c6d313740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556f2b49d6c0 con 0x556f2b48bcc0 2024-09-19T09:15:38.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.754+0000 7f8c6d313740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556f2b3ac050 con 0x556f2b48b3a0 2024-09-19T09:15:38.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.754+0000 7f8c6d313740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556f2b49d840 con 0x556f2b49aa90 2024-09-19T09:15:38.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.754+0000 7f8c6d313740 10 client.0 _ll_register_callbacks cb 0x556f2b3ec160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:38.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.754+0000 7f8c68cd6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556f2b48bcc0 0x556f2b497600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:38.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.754+0000 7f8c63fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f2b48b3a0 0x556f2b49a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:38.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.754+0000 7f8c68cd6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556f2b48bcc0 0x556f2b497600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52132/0 (socket says 172.21.15.120:52132) 2024-09-19T09:15:38.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.754+0000 7f8c63fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f2b48b3a0 0x556f2b49a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37872/0 (socket says 172.21.15.120:37872) 2024-09-19T09:15:38.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.754+0000 7f8c68cd6640 1 -- 172.21.15.120:0/3281551732 learned_addr learned my addr 172.21.15.120:0/3281551732 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:38.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.754+0000 7f8c63fff640 1 --2- 172.21.15.120:0/3281551732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f2b49aa90 0x556f2b497b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:38.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.754+0000 7f8c68cd6640 1 -- 172.21.15.120:0/3281551732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f2b49aa90 msgr2=0x556f2b497b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:38.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.754+0000 7f8c68cd6640 1 --2- 172.21.15.120:0/3281551732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f2b49aa90 0x556f2b497b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.754+0000 7f8c68cd6640 1 -- 172.21.15.120:0/3281551732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f2b48b3a0 msgr2=0x556f2b49a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:38.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.754+0000 7f8c68cd6640 1 --2- 172.21.15.120:0/3281551732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f2b48b3a0 0x556f2b49a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.754+0000 7f8c68cd6640 1 -- 172.21.15.120:0/3281551732 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f2b1623c0 con 0x556f2b48bcc0 2024-09-19T09:15:38.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.754+0000 7f8c63fff640 1 --2- 172.21.15.120:0/3281551732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f2b48b3a0 0x556f2b49a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:15:38.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.754+0000 7f8c68cd6640 1 --2- 172.21.15.120:0/3281551732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556f2b48bcc0 0x556f2b497600 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f8c58005db0 tx=0x7f8c58004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9be167336a402a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:38.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.754+0000 7f8c43fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:15:38.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.755+0000 7f8c43fff640 1 -- 172.21.15.120:0/3281551732 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c58029070 con 0x556f2b48bcc0 2024-09-19T09:15:38.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.755+0000 7f8c6d313740 1 -- 172.21.15.120:0/3281551732 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c58005500 con 0x556f2b48bcc0 2024-09-19T09:15:38.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.755+0000 7f8c43fff640 1 -- 172.21.15.120:0/3281551732 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f8c58004f00 con 0x556f2b48bcc0 2024-09-19T09:15:38.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.755+0000 7f8c43fff640 1 -- 172.21.15.120:0/3281551732 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8c58035050 con 0x556f2b48bcc0 2024-09-19T09:15:38.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.755+0000 7f8c6d313740 10 client.19845 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:38.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.755+0000 7f8c6d313740 1 -- 172.21.15.120:0/3281551732 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556f2b4a9080 con 0x556f2b48bcc0 2024-09-19T09:15:38.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.755+0000 7f8c43fff640 1 -- 172.21.15.120:0/3281551732 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c58005500 con 0x556f2b48bcc0 2024-09-19T09:15:38.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.755+0000 7f8c43fff640 1 -- 172.21.15.120:0/3281551732 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f8c58023430 con 0x556f2b48bcc0 2024-09-19T09:15:38.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.755+0000 7f8c6d313740 10 client.19845 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:38.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.755+0000 7f8c6d313740 10 client.19845 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:15:38.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.755+0000 7f8c6d313740 20 client.19845 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:38.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.755+0000 7f8c6d313740 1 --2- 172.21.15.120:0/3281551732 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556f2b4aa430 0x556f2b4ac820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:38.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.756+0000 7f8c6d313740 4 client.19845 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:15:38.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.756+0000 7f8c6d313740 1 -- 172.21.15.120:0/3281551732 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f2b3fe850 con 0x556f2b4aa430 2024-09-19T09:15:38.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.756+0000 7f8c63fff640 1 --2- 172.21.15.120:0/3281551732 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556f2b4aa430 0x556f2b4ac820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:38.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.756+0000 7f8c63fff640 1 --2- 172.21.15.120:0/3281551732 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556f2b4aa430 0x556f2b4ac820 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65de8f4355a6631a server_cookie=83cf7fa5484ea771 in_seq=0 out_seq=0 2024-09-19T09:15:38.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.756+0000 7f8c43fff640 10 client.19845 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:38.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.757+0000 7f8c43fff640 1 -- 172.21.15.120:0/3281551732 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556f2b3fe850 con 0x556f2b4aa430 2024-09-19T09:15:38.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.757+0000 7f8c43fff640 10 client.19845 handle_command_reply: tid=0 2024-09-19T09:15:38.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.757+0000 7f8c6d313740 1 client.19845 shutdown 2024-09-19T09:15:38.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.757+0000 7f8c6d313740 1 -- 172.21.15.120:0/3281551732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556f2b48bcc0 msgr2=0x556f2b497600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:38.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.757+0000 7f8c6d313740 1 --2- 172.21.15.120:0/3281551732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556f2b48bcc0 0x556f2b497600 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f8c58005db0 tx=0x7f8c58004550 comp rx=0 tx=0).stop 2024-09-19T09:15:38.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.757+0000 7f8c6d313740 1 -- 172.21.15.120:0/3281551732 shutdown_connections 2024-09-19T09:15:38.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.757+0000 7f8c6d313740 1 --2- 172.21.15.120:0/3281551732 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556f2b4aa430 0x556f2b4ac820 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.757+0000 7f8c6d313740 1 --2- 172.21.15.120:0/3281551732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f2b49aa90 0x556f2b497b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.757+0000 7f8c6d313740 1 --2- 172.21.15.120:0/3281551732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556f2b48bcc0 0x556f2b497600 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.757+0000 7f8c43fff640 0 client.19845 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:38.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.757+0000 7f8c6d313740 1 --2- 172.21.15.120:0/3281551732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f2b48b3a0 0x556f2b49a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.757+0000 7f8c6d313740 1 -- 172.21.15.120:0/3281551732 >> 172.21.15.120:0/3281551732 conn(0x556f2b487a30 msgr2=0x556f2b48d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:38.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.757+0000 7f8c6d313740 1 -- 172.21.15.120:0/3281551732 shutdown_connections 2024-09-19T09:15:38.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.757+0000 7f8c6d313740 1 -- 172.21.15.120:0/3281551732 wait complete. 2024-09-19T09:15:38.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.757+0000 7f8c6d313740 20 client.19845 trim_cache size 0 max 16384 2024-09-19T09:15:38.775 INFO:tasks.quiescer.fs.[cephfs]:Total duration (44.4) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T09:15:38.775 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.2 2024-09-19T09:15:38.775 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:15:38.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.775+0000 7f8c6d313740 1 Processor -- start 2024-09-19T09:15:38.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.775+0000 7f8c6d313740 1 -- start start 2024-09-19T09:15:38.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.775+0000 7f8c6d313740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556f2b48b3a0 0x556f2b4a6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:38.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.775+0000 7f8c6d313740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f2b48bcc0 0x556f2b492d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:38.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.775+0000 7f8c6d313740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f2b49aa90 0x556f2b48d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:38.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.775+0000 7f8c6d313740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8c641ab3b0 con 0x556f2b48b3a0 2024-09-19T09:15:38.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.775+0000 7f8c6d313740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8c641acef0 con 0x556f2b49aa90 2024-09-19T09:15:38.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.775+0000 7f8c6d313740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8c641aba80 con 0x556f2b48bcc0 2024-09-19T09:15:38.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.775+0000 7f8c63fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556f2b48b3a0 0x556f2b4a6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:38.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.775+0000 7f8c63fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556f2b48b3a0 0x556f2b4a6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52144/0 (socket says 172.21.15.120:52144) 2024-09-19T09:15:38.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.775+0000 7f8c63fff640 1 -- 172.21.15.120:0/481074534 learned_addr learned my addr 172.21.15.120:0/481074534 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:38.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.775+0000 7f8c68cd6640 1 --2- 172.21.15.120:0/481074534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f2b48bcc0 0x556f2b492d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:38.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.775+0000 7f8c63fff640 1 --2- 172.21.15.120:0/481074534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f2b49aa90 0x556f2b48d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:38.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.775+0000 7f8c63fff640 1 -- 172.21.15.120:0/481074534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f2b48bcc0 msgr2=0x556f2b492d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:38.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.775+0000 7f8c63fff640 1 --2- 172.21.15.120:0/481074534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f2b48bcc0 0x556f2b492d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.775+0000 7f8c63fff640 1 -- 172.21.15.120:0/481074534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f2b49aa90 msgr2=0x556f2b48d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:38.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.775+0000 7f8c63fff640 1 --2- 172.21.15.120:0/481074534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f2b49aa90 0x556f2b48d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.775+0000 7f8c63fff640 1 -- 172.21.15.120:0/481074534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f2b4acdb0 con 0x556f2b48b3a0 2024-09-19T09:15:38.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.776+0000 7f8c63fff640 1 --2- 172.21.15.120:0/481074534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556f2b48b3a0 0x556f2b4a6990 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f8c50009230 tx=0x7f8c50009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2ee8d510b604cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:38.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.776+0000 7f8c4e7fc640 1 -- 172.21.15.120:0/481074534 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c50041070 con 0x556f2b48b3a0 2024-09-19T09:15:38.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.776+0000 7f8c4e7fc640 1 -- 172.21.15.120:0/481074534 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c50004c50 con 0x556f2b48b3a0 2024-09-19T09:15:38.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.776+0000 7f8c4e7fc640 1 -- 172.21.15.120:0/481074534 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c50035040 con 0x556f2b48b3a0 2024-09-19T09:15:38.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.776+0000 7f8c6d313740 1 -- 172.21.15.120:0/481074534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556f2b48b3a0 msgr2=0x556f2b4a6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:38.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.776+0000 7f8c6d313740 1 --2- 172.21.15.120:0/481074534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556f2b48b3a0 0x556f2b4a6990 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f8c50009230 tx=0x7f8c50009260 comp rx=0 tx=0).stop 2024-09-19T09:15:38.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.777+0000 7f8c6d313740 1 -- 172.21.15.120:0/481074534 shutdown_connections 2024-09-19T09:15:38.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.777+0000 7f8c6d313740 1 --2- 172.21.15.120:0/481074534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f2b49aa90 0x556f2b48d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.777+0000 7f8c6d313740 1 --2- 172.21.15.120:0/481074534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f2b48bcc0 0x556f2b492d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.777+0000 7f8c6d313740 1 --2- 172.21.15.120:0/481074534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556f2b48b3a0 0x556f2b4a6990 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.777+0000 7f8c6d313740 1 -- 172.21.15.120:0/481074534 >> 172.21.15.120:0/481074534 conn(0x556f2b487a30 msgr2=0x556f2b48ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:38.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.777+0000 7f8c6d313740 1 -- 172.21.15.120:0/481074534 shutdown_connections 2024-09-19T09:15:38.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.777+0000 7f8c6d313740 1 -- 172.21.15.120:0/481074534 wait complete. 2024-09-19T09:15:38.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.777+0000 7f8c6d313740 1 -- start start 2024-09-19T09:15:38.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.777+0000 7f8c6d313740 1 Processor -- start 2024-09-19T09:15:38.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.777+0000 7f8c6d313740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f2b48b3a0 0x556f2b4a2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:38.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.777+0000 7f8c6d313740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556f2b48bcc0 0x556f2b49e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:38.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.777+0000 7f8c6d313740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f2b49aa90 0x556f2b49e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:38.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.777+0000 7f8c6d313740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8c641ac820 con 0x556f2b48bcc0 2024-09-19T09:15:38.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.777+0000 7f8c6d313740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8c641ac150 con 0x556f2b48b3a0 2024-09-19T09:15:38.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.777+0000 7f8c6d313740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8c641a9370 con 0x556f2b49aa90 2024-09-19T09:15:38.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.777+0000 7f8c6d313740 10 client.0 _ll_register_callbacks cb 0x7f8c5c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:38.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.778+0000 7f8c68cd6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556f2b48bcc0 0x556f2b49e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:38.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.778+0000 7f8c63fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f2b48b3a0 0x556f2b4a2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:38.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.778+0000 7f8c63fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f2b48b3a0 0x556f2b4a2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37900/0 (socket says 172.21.15.120:37900) 2024-09-19T09:15:38.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.778+0000 7f8c63fff640 1 -- 172.21.15.120:0/4142100035 learned_addr learned my addr 172.21.15.120:0/4142100035 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:38.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.778+0000 7f8c63fff640 1 --2- 172.21.15.120:0/4142100035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f2b49aa90 0x556f2b49e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:38.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.778+0000 7f8c63fff640 1 -- 172.21.15.120:0/4142100035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f2b49aa90 msgr2=0x556f2b49e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:38.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.778+0000 7f8c63fff640 1 --2- 172.21.15.120:0/4142100035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f2b49aa90 0x556f2b49e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.778+0000 7f8c63fff640 1 -- 172.21.15.120:0/4142100035 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556f2b48bcc0 msgr2=0x556f2b49e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:38.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.778+0000 7f8c63fff640 1 --2- 172.21.15.120:0/4142100035 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556f2b48bcc0 0x556f2b49e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.778+0000 7f8c63fff640 1 -- 172.21.15.120:0/4142100035 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f2b487860 con 0x556f2b48b3a0 2024-09-19T09:15:38.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.778+0000 7f8c68cd6640 1 --2- 172.21.15.120:0/4142100035 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556f2b48bcc0 0x556f2b49e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:15:38.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.778+0000 7f8c63fff640 1 --2- 172.21.15.120:0/4142100035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f2b48b3a0 0x556f2b4a2f10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8c50031bf0 tx=0x7f8c5004e510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3bff4af0fb60356 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:38.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.778+0000 7f8c43fff640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:15:38.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.778+0000 7f8c43fff640 1 -- 172.21.15.120:0/4142100035 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c50035030 con 0x556f2b48b3a0 2024-09-19T09:15:38.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.778+0000 7f8c43fff640 1 -- 172.21.15.120:0/4142100035 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f8c50005440 con 0x556f2b48b3a0 2024-09-19T09:15:38.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.779+0000 7f8c6d313740 1 -- 172.21.15.120:0/4142100035 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c50004c50 con 0x556f2b48b3a0 2024-09-19T09:15:38.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.779+0000 7f8c43fff640 1 -- 172.21.15.120:0/4142100035 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8c50041040 con 0x556f2b48b3a0 2024-09-19T09:15:38.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.779+0000 7f8c6d313740 10 client.28895 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:38.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.779+0000 7f8c6d313740 1 -- 172.21.15.120:0/4142100035 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556f2b514ac0 con 0x556f2b48b3a0 2024-09-19T09:15:38.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.779+0000 7f8c43fff640 1 -- 172.21.15.120:0/4142100035 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c50004c50 con 0x556f2b48b3a0 2024-09-19T09:15:38.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.779+0000 7f8c43fff640 1 -- 172.21.15.120:0/4142100035 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f8c5004d040 con 0x556f2b48b3a0 2024-09-19T09:15:38.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.779+0000 7f8c6d313740 10 client.28895 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:38.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.779+0000 7f8c6d313740 10 client.28895 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:15:38.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.779+0000 7f8c6d313740 20 client.28895 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:38.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.779+0000 7f8c6d313740 1 --2- 172.21.15.120:0/4142100035 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556f2b5150c0 0x556f2b5174b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:38.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.779+0000 7f8c6d313740 4 client.28895 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:15:38.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.779+0000 7f8c6d313740 1 -- 172.21.15.120:0/4142100035 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556f2b497ed0 con 0x556f2b5150c0 2024-09-19T09:15:38.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.780+0000 7f8c68cd6640 1 --2- 172.21.15.120:0/4142100035 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556f2b5150c0 0x556f2b5174b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:38.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.780+0000 7f8c43fff640 10 client.28895 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:38.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.780+0000 7f8c68cd6640 1 --2- 172.21.15.120:0/4142100035 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556f2b5150c0 0x556f2b5174b0 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc87f04e0f4670c5 server_cookie=f9b4b032a468da04 in_seq=0 out_seq=0 2024-09-19T09:15:38.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.781+0000 7f8c43fff640 1 -- 172.21.15.120:0/4142100035 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556f2b497ed0 con 0x556f2b5150c0 2024-09-19T09:15:38.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.781+0000 7f8c43fff640 10 client.28895 handle_command_reply: tid=0 2024-09-19T09:15:38.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.781+0000 7f8c6d313740 1 client.28895 shutdown 2024-09-19T09:15:38.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.781+0000 7f8c6d313740 1 -- 172.21.15.120:0/4142100035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f2b48b3a0 msgr2=0x556f2b4a2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:38.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.781+0000 7f8c6d313740 1 --2- 172.21.15.120:0/4142100035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f2b48b3a0 0x556f2b4a2f10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8c50031bf0 tx=0x7f8c5004e510 comp rx=0 tx=0).stop 2024-09-19T09:15:38.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.781+0000 7f8c6d313740 1 -- 172.21.15.120:0/4142100035 shutdown_connections 2024-09-19T09:15:38.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.781+0000 7f8c6d313740 1 --2- 172.21.15.120:0/4142100035 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556f2b5150c0 0x556f2b5174b0 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.781+0000 7f8c6d313740 1 --2- 172.21.15.120:0/4142100035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f2b49aa90 0x556f2b49e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.782+0000 7f8c6d313740 1 --2- 172.21.15.120:0/4142100035 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556f2b48bcc0 0x556f2b49e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.782+0000 7f8c43fff640 0 client.28895 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:38.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.782+0000 7f8c6d313740 1 --2- 172.21.15.120:0/4142100035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f2b48b3a0 0x556f2b4a2f10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.782+0000 7f8c6d313740 1 -- 172.21.15.120:0/4142100035 >> 172.21.15.120:0/4142100035 conn(0x556f2b487a30 msgr2=0x556f2b492c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:38.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.782+0000 7f8c6d313740 1 -- 172.21.15.120:0/4142100035 shutdown_connections 2024-09-19T09:15:38.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.782+0000 7f8c6d313740 1 -- 172.21.15.120:0/4142100035 wait complete. 2024-09-19T09:15:38.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.782+0000 7f8c6d313740 20 client.28895 trim_cache size 0 max 16384 2024-09-19T09:15:38.788 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:15:38.789 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:15:38.789 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:15:38.789 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:15:38.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.782+0000 7f8c6af3a640 1 -- 172.21.15.120:0/2380259312 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8c540a3240 msgr2=0x7f8c540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:38.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.782+0000 7f8c6af3a640 1 --2- 172.21.15.120:0/2380259312 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8c540a3240 0x7f8c540a5700 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c0049d0 tx=0x7f8c5c06f360 comp rx=0 tx=0).stop 2024-09-19T09:15:38.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.782+0000 7f8c6af3a640 1 -- 172.21.15.120:0/2380259312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c64106a80 msgr2=0x7f8c641a4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:38.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.782+0000 7f8c6af3a640 1 --2- 172.21.15.120:0/2380259312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c64106a80 0x7f8c641a4370 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c00eb10 tx=0x7f8c5c00c2d0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.783+0000 7f8c6af3a640 1 -- 172.21.15.120:0/2380259312 shutdown_connections 2024-09-19T09:15:38.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.783+0000 7f8c6af3a640 1 --2- 172.21.15.120:0/2380259312 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8c540a3240 0x7f8c540a5700 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.783+0000 7f8c6af3a640 1 --2- 172.21.15.120:0/2380259312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c64106a80 0x7f8c641a4370 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.783+0000 7f8c6af3a640 1 --2- 172.21.15.120:0/2380259312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c64101420 0x7f8c64199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.783+0000 7f8c6af3a640 1 --2- 172.21.15.120:0/2380259312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c64100a80 0x7f8c641992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:38.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.783+0000 7f8c6af3a640 1 -- 172.21.15.120:0/2380259312 >> 172.21.15.120:0/2380259312 conn(0x7f8c640fc7a0 msgr2=0x7f8c640fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:38.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.783+0000 7f8c6af3a640 1 -- 172.21.15.120:0/2380259312 shutdown_connections 2024-09-19T09:15:38.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:38.783+0000 7f8c6af3a640 1 -- 172.21.15.120:0/2380259312 wait complete. 2024-09-19T09:15:39.078 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1291260 10.04 MB/sec execute 574 sec latency 38.949 ms 2024-09-19T09:15:39.135 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:15:39.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:38 smithi196 ceph-mon[35150]: pgmap v727: 97 pgs: 97 active+clean; 46 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.1 MiB/s wr, 451 op/s 2024-09-19T09:15:39.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:38 smithi153 ceph-mon[35071]: pgmap v727: 97 pgs: 97 active+clean; 46 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.1 MiB/s wr, 451 op/s 2024-09-19T09:15:39.261 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:38 smithi120 ceph-mon[31150]: pgmap v727: 97 pgs: 97 active+clean; 46 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.1 MiB/s wr, 451 op/s 2024-09-19T09:15:39.861 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:15:39.861 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:15:40.077 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1293251 10.05 MB/sec execute 575 sec latency 46.225 ms 2024-09-19T09:15:40.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.408+0000 7fc0fb7c2640 1 -- 172.21.15.120:0/3708921131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0f4101df0 msgr2=0x7fc0f41021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:40.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.408+0000 7fc0fb7c2640 1 --2- 172.21.15.120:0/3708921131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0f4101df0 0x7fc0f41021d0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0009900 tx=0x7fc0f002a410 comp rx=0 tx=0).stop 2024-09-19T09:15:40.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.408+0000 7fc0fb7c2640 1 -- 172.21.15.120:0/3708921131 shutdown_connections 2024-09-19T09:15:40.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.408+0000 7fc0fb7c2640 1 --2- 172.21.15.120:0/3708921131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0f4106f20 0x7fc0f410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:40.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.408+0000 7fc0fb7c2640 1 --2- 172.21.15.120:0/3708921131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0f41027a0 0x7fc0f41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:40.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.408+0000 7fc0fb7c2640 1 --2- 172.21.15.120:0/3708921131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0f4101df0 0x7fc0f41021d0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:40.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.408+0000 7fc0fb7c2640 1 -- 172.21.15.120:0/3708921131 >> 172.21.15.120:0/3708921131 conn(0x7fc0f40fd8f0 msgr2=0x7fc0f40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:40.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.409+0000 7fc0fb7c2640 1 -- 172.21.15.120:0/3708921131 shutdown_connections 2024-09-19T09:15:40.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.409+0000 7fc0fb7c2640 1 -- 172.21.15.120:0/3708921131 wait complete. 2024-09-19T09:15:40.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.409+0000 7fc0fb7c2640 1 Processor -- start 2024-09-19T09:15:40.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.409+0000 7fc0fb7c2640 1 -- start start 2024-09-19T09:15:40.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.409+0000 7fc0fb7c2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0f4101df0 0x7fc0f41984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:40.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.409+0000 7fc0fb7c2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0f41027a0 0x7fc0f41989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:40.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.409+0000 7fc0fb7c2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0f4106f20 0x7fc0f419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:40.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.409+0000 7fc0fb7c2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f4111f80 con 0x7fc0f41027a0 2024-09-19T09:15:40.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.409+0000 7fc0fb7c2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f4111e00 con 0x7fc0f4101df0 2024-09-19T09:15:40.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.409+0000 7fc0fb7c2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f4112100 con 0x7fc0f4106f20 2024-09-19T09:15:40.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.410+0000 7fc0f8d5d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0f41027a0 0x7fc0f41989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:40.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.410+0000 7fc0f955e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0f4101df0 0x7fc0f41984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:40.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.410+0000 7fc0f8d5d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0f41027a0 0x7fc0f41989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52154/0 (socket says 172.21.15.120:52154) 2024-09-19T09:15:40.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.410+0000 7fc0f8d5d640 1 -- 172.21.15.120:0/4119745685 learned_addr learned my addr 172.21.15.120:0/4119745685 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:40.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.410+0000 7fc0f9d5f640 1 --2- 172.21.15.120:0/4119745685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0f4106f20 0x7fc0f419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:40.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.410+0000 7fc0f8d5d640 1 -- 172.21.15.120:0/4119745685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0f4106f20 msgr2=0x7fc0f419cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:40.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.410+0000 7fc0f8d5d640 1 --2- 172.21.15.120:0/4119745685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0f4106f20 0x7fc0f419cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:40.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.410+0000 7fc0f8d5d640 1 -- 172.21.15.120:0/4119745685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0f4101df0 msgr2=0x7fc0f41984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:40.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.410+0000 7fc0f8d5d640 1 --2- 172.21.15.120:0/4119745685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0f4101df0 0x7fc0f41984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:40.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.410+0000 7fc0f8d5d640 1 -- 172.21.15.120:0/4119745685 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f419d450 con 0x7fc0f41027a0 2024-09-19T09:15:40.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.410+0000 7fc0f8d5d640 1 --2- 172.21.15.120:0/4119745685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0f41027a0 0x7fc0f41989e0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fc0e400e9f0 tx=0x7fc0e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:40.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.410+0000 7fc0ea7fc640 1 -- 172.21.15.120:0/4119745685 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e400cd80 con 0x7fc0f41027a0 2024-09-19T09:15:40.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.410+0000 7fc0fb7c2640 1 -- 172.21.15.120:0/4119745685 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0f419d740 con 0x7fc0f41027a0 2024-09-19T09:15:40.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.410+0000 7fc0ea7fc640 1 -- 172.21.15.120:0/4119745685 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0e4010910 con 0x7fc0f41027a0 2024-09-19T09:15:40.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.410+0000 7fc0ea7fc640 1 -- 172.21.15.120:0/4119745685 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e40188f0 con 0x7fc0f41027a0 2024-09-19T09:15:40.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.410+0000 7fc0fb7c2640 1 -- 172.21.15.120:0/4119745685 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0f41a5020 con 0x7fc0f41027a0 2024-09-19T09:15:40.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.411+0000 7fc0fb7c2640 1 -- 172.21.15.120:0/4119745685 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0f41021d0 con 0x7fc0f41027a0 2024-09-19T09:15:40.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.412+0000 7fc0ea7fc640 1 -- 172.21.15.120:0/4119745685 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fc0e4010ac0 con 0x7fc0f41027a0 2024-09-19T09:15:40.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.413+0000 7fc0ea7fc640 1 --2- 172.21.15.120:0/4119745685 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc0e00a32c0 0x7fc0e00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:40.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.413+0000 7fc0f955e640 1 --2- 172.21.15.120:0/4119745685 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc0e00a32c0 0x7fc0e00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:40.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.413+0000 7fc0ea7fc640 1 -- 172.21.15.120:0/4119745685 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fc0e40cfc50 con 0x7fc0f41027a0 2024-09-19T09:15:40.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.414+0000 7fc0f955e640 1 --2- 172.21.15.120:0/4119745685 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc0e00a32c0 0x7fc0e00a5780 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0009230 tx=0x7fc0f00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:40.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.418+0000 7fc0ea7fc640 1 -- 172.21.15.120:0/4119745685 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc0e409c320 con 0x7fc0f41027a0 2024-09-19T09:15:40.475 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:15:40.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.643+0000 7fc0fb7c2640 1 -- 172.21.15.120:0/4119745685 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0f4109b70 con 0x7fc0f41027a0 2024-09-19T09:15:40.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.644+0000 7fc0ea7fc640 1 -- 172.21.15.120:0/4119745685 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11988 (secure 0 0 0) 0x7fc0e40a0620 con 0x7fc0f41027a0 2024-09-19T09:15:40.645 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:15:40.646 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":33,"btime":"2024-09-19T09:14:31:550518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:14:31.550494+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28649":{"gid":28649,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/2810080307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2810080307},{"type":"v1","addr":"172.21.15.153:6835","nonce":2810080307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:15:40.646 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 33 2024-09-19T09:15:40.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.646+0000 7fc0fb7c2640 1 -- 172.21.15.120:0/4119745685 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc0e00a32c0 msgr2=0x7fc0e00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:40.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.646+0000 7fc0fb7c2640 1 --2- 172.21.15.120:0/4119745685 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc0e00a32c0 0x7fc0e00a5780 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0009230 tx=0x7fc0f00058d0 comp rx=0 tx=0).stop 2024-09-19T09:15:40.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.646+0000 7fc0fb7c2640 1 -- 172.21.15.120:0/4119745685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0f41027a0 msgr2=0x7fc0f41989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:40.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.646+0000 7fc0fb7c2640 1 --2- 172.21.15.120:0/4119745685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0f41027a0 0x7fc0f41989e0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fc0e400e9f0 tx=0x7fc0e400eeb0 comp rx=0 tx=0).stop 2024-09-19T09:15:40.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.646+0000 7fc0fb7c2640 1 -- 172.21.15.120:0/4119745685 shutdown_connections 2024-09-19T09:15:40.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.646+0000 7fc0fb7c2640 1 --2- 172.21.15.120:0/4119745685 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc0e00a32c0 0x7fc0e00a5780 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:40.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.646+0000 7fc0fb7c2640 1 --2- 172.21.15.120:0/4119745685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0f4106f20 0x7fc0f419cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:40.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.646+0000 7fc0fb7c2640 1 --2- 172.21.15.120:0/4119745685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0f41027a0 0x7fc0f41989e0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:40.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.646+0000 7fc0fb7c2640 1 --2- 172.21.15.120:0/4119745685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0f4101df0 0x7fc0f41984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:40.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.646+0000 7fc0fb7c2640 1 -- 172.21.15.120:0/4119745685 >> 172.21.15.120:0/4119745685 conn(0x7fc0f40fd8f0 msgr2=0x7fc0f40fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:40.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.646+0000 7fc0fb7c2640 1 -- 172.21.15.120:0/4119745685 shutdown_connections 2024-09-19T09:15:40.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:40.646+0000 7fc0fb7c2640 1 -- 172.21.15.120:0/4119745685 wait complete. 2024-09-19T09:15:41.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:40 smithi120 ceph-mon[31150]: pgmap v728: 97 pgs: 97 active+clean; 44 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.6 MiB/s wr, 390 op/s 2024-09-19T09:15:41.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:40 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/4119745685' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:15:41.077 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1296450 10.07 MB/sec execute 576 sec latency 13.788 ms 2024-09-19T09:15:41.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:40 smithi196 ceph-mon[35150]: pgmap v728: 97 pgs: 97 active+clean; 44 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.6 MiB/s wr, 390 op/s 2024-09-19T09:15:41.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:40 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/4119745685' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:15:41.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:40 smithi153 ceph-mon[35071]: pgmap v728: 97 pgs: 97 active+clean; 44 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.6 MiB/s wr, 390 op/s 2024-09-19T09:15:41.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:40 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/4119745685' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:15:41.476 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db / --timeout 90 --expiration 142.2 --await-for 45.0' 2024-09-19T09:15:41.477 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db / --timeout 90 --expiration 142.2 --await-for 45.0 2024-09-19T09:15:41.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.690+0000 7f3aa97eb640 1 -- 172.21.15.120:0/4229856234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3aa4102790 msgr2=0x7f3aa41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:41.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.690+0000 7f3aa97eb640 1 --2- 172.21.15.120:0/4229856234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3aa4102790 0x7f3aa41067e0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f3a98009900 tx=0x7f3a9802a410 comp rx=0 tx=0).stop 2024-09-19T09:15:41.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.690+0000 7f3aa97eb640 1 -- 172.21.15.120:0/4229856234 shutdown_connections 2024-09-19T09:15:41.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.690+0000 7f3aa97eb640 1 --2- 172.21.15.120:0/4229856234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3aa4106f10 0x7f3aa410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:41.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.690+0000 7f3aa97eb640 1 --2- 172.21.15.120:0/4229856234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3aa4102790 0x7f3aa41067e0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:41.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.690+0000 7f3aa97eb640 1 --2- 172.21.15.120:0/4229856234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa4101de0 0x7f3aa41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:41.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.690+0000 7f3aa97eb640 1 -- 172.21.15.120:0/4229856234 >> 172.21.15.120:0/4229856234 conn(0x7f3aa40fd8c0 msgr2=0x7f3aa40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:41.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.690+0000 7f3aa97eb640 1 -- 172.21.15.120:0/4229856234 shutdown_connections 2024-09-19T09:15:41.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.690+0000 7f3aa97eb640 1 -- 172.21.15.120:0/4229856234 wait complete. 2024-09-19T09:15:41.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.691+0000 7f3aa97eb640 1 Processor -- start 2024-09-19T09:15:41.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.691+0000 7f3aa97eb640 1 -- start start 2024-09-19T09:15:41.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.691+0000 7f3aa97eb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3aa4101de0 0x7f3aa4198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:41.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.691+0000 7f3aa97eb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3aa4102790 0x7f3aa41989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:41.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.691+0000 7f3aa97eb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa4106f10 0x7f3aa419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:41.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.691+0000 7f3aa97eb640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa4111f70 con 0x7f3aa4102790 2024-09-19T09:15:41.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.691+0000 7f3aa97eb640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa4111df0 con 0x7f3aa4106f10 2024-09-19T09:15:41.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.691+0000 7f3aa97eb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa41120f0 con 0x7f3aa4101de0 2024-09-19T09:15:41.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.692+0000 7f3aa27fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3aa4102790 0x7f3aa41989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:41.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.692+0000 7f3aa2ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3aa4101de0 0x7f3aa4198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:41.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.692+0000 7f3aa27fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3aa4102790 0x7f3aa41989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46010/0 (socket says 172.21.15.120:46010) 2024-09-19T09:15:41.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.692+0000 7f3aa27fc640 1 -- 172.21.15.120:0/994233378 learned_addr learned my addr 172.21.15.120:0/994233378 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:41.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.692+0000 7f3aa37fe640 1 --2- 172.21.15.120:0/994233378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa4106f10 0x7f3aa419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:41.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.692+0000 7f3aa2ffd640 1 -- 172.21.15.120:0/994233378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa4106f10 msgr2=0x7f3aa419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:41.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.692+0000 7f3aa2ffd640 1 --2- 172.21.15.120:0/994233378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa4106f10 0x7f3aa419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:41.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.692+0000 7f3aa2ffd640 1 -- 172.21.15.120:0/994233378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3aa4102790 msgr2=0x7f3aa41989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:41.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.692+0000 7f3aa2ffd640 1 --2- 172.21.15.120:0/994233378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3aa4102790 0x7f3aa41989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:41.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.692+0000 7f3aa2ffd640 1 -- 172.21.15.120:0/994233378 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa419d440 con 0x7f3aa4101de0 2024-09-19T09:15:41.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.692+0000 7f3aa27fc640 1 --2- 172.21.15.120:0/994233378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3aa4102790 0x7f3aa41989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:15:41.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.692+0000 7f3aa2ffd640 1 --2- 172.21.15.120:0/994233378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3aa4101de0 0x7f3aa4198490 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3a9400e9f0 tx=0x7f3a9400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:41.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.692+0000 7f3a8bfff640 1 -- 172.21.15.120:0/994233378 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9400ccc0 con 0x7f3aa4101de0 2024-09-19T09:15:41.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.692+0000 7f3aa97eb640 1 -- 172.21.15.120:0/994233378 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa419d730 con 0x7f3aa4101de0 2024-09-19T09:15:41.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.692+0000 7f3a8bfff640 1 -- 172.21.15.120:0/994233378 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a94010910 con 0x7f3aa4101de0 2024-09-19T09:15:41.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.692+0000 7f3a8bfff640 1 -- 172.21.15.120:0/994233378 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94018960 con 0x7f3aa4101de0 2024-09-19T09:15:41.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.692+0000 7f3aa97eb640 1 -- 172.21.15.120:0/994233378 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa41a5010 con 0x7f3aa4101de0 2024-09-19T09:15:41.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.693+0000 7f3aa97eb640 1 -- 172.21.15.120:0/994233378 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aa41021c0 con 0x7f3aa4101de0 2024-09-19T09:15:41.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.695+0000 7f3a8bfff640 1 -- 172.21.15.120:0/994233378 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f3a94010ac0 con 0x7f3aa4101de0 2024-09-19T09:15:41.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.695+0000 7f3a8bfff640 1 --2- 172.21.15.120:0/994233378 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3a8c0a3240 0x7f3a8c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:41.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.695+0000 7f3aa27fc640 1 --2- 172.21.15.120:0/994233378 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3a8c0a3240 0x7f3a8c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:41.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.695+0000 7f3a8bfff640 1 -- 172.21.15.120:0/994233378 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f3a940d01a0 con 0x7f3aa4101de0 2024-09-19T09:15:41.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.696+0000 7f3aa27fc640 1 --2- 172.21.15.120:0/994233378 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3a8c0a3240 0x7f3a8c0a5700 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3a98031550 tx=0x7f3a98005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:41.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.699+0000 7f3a8bfff640 1 -- 172.21.15.120:0/994233378 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3a9409c830 con 0x7f3aa4101de0 2024-09-19T09:15:41.790 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:15:41.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.919+0000 7f3aa97eb640 1 -- 172.21.15.120:0/994233378 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3aa410a0f0 con 0x7f3aa4101de0 2024-09-19T09:15:41.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.920+0000 7f3a8bfff640 1 -- 172.21.15.120:0/994233378 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11988 (secure 0 0 0) 0x7f3a940a0b30 con 0x7f3aa4101de0 2024-09-19T09:15:41.921 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:15:41.921 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":33,"btime":"2024-09-19T09:14:31:550518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:14:31.550494+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28649":{"gid":28649,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/2810080307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2810080307},{"type":"v1","addr":"172.21.15.153:6835","nonce":2810080307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:15:41.921 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 33 2024-09-19T09:15:41.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.922+0000 7f3aa97eb640 1 -- 172.21.15.120:0/994233378 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3a8c0a3240 msgr2=0x7f3a8c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:41.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.922+0000 7f3aa97eb640 1 --2- 172.21.15.120:0/994233378 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3a8c0a3240 0x7f3a8c0a5700 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3a98031550 tx=0x7f3a98005a20 comp rx=0 tx=0).stop 2024-09-19T09:15:41.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.922+0000 7f3aa97eb640 1 -- 172.21.15.120:0/994233378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3aa4101de0 msgr2=0x7f3aa4198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:41.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.922+0000 7f3aa97eb640 1 --2- 172.21.15.120:0/994233378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3aa4101de0 0x7f3aa4198490 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3a9400e9f0 tx=0x7f3a9400eeb0 comp rx=0 tx=0).stop 2024-09-19T09:15:41.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.922+0000 7f3aa97eb640 1 -- 172.21.15.120:0/994233378 shutdown_connections 2024-09-19T09:15:41.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.922+0000 7f3aa97eb640 1 --2- 172.21.15.120:0/994233378 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3a8c0a3240 0x7f3a8c0a5700 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:41.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.922+0000 7f3aa97eb640 1 --2- 172.21.15.120:0/994233378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa4106f10 0x7f3aa419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:41.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.922+0000 7f3aa97eb640 1 --2- 172.21.15.120:0/994233378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3aa4102790 0x7f3aa41989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:41.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.922+0000 7f3aa97eb640 1 --2- 172.21.15.120:0/994233378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3aa4101de0 0x7f3aa4198490 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:41.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.922+0000 7f3aa97eb640 1 -- 172.21.15.120:0/994233378 >> 172.21.15.120:0/994233378 conn(0x7f3aa40fd8c0 msgr2=0x7f3aa40fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:41.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.922+0000 7f3aa97eb640 1 -- 172.21.15.120:0/994233378 shutdown_connections 2024-09-19T09:15:41.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:41.922+0000 7f3aa97eb640 1 -- 172.21.15.120:0/994233378 wait complete. 2024-09-19T09:15:42.078 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1299128 10.07 MB/sec execute 577 sec latency 21.013 ms 2024-09-19T09:15:42.726 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:15:43.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:42 smithi120 ceph-mon[31150]: pgmap v729: 97 pgs: 97 active+clean; 41 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.7 MiB/s wr, 420 op/s 2024-09-19T09:15:43.013 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:42 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/994233378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:15:43.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.011+0000 7f785be7d640 1 -- 172.21.15.120:0/712136751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7854100a90 msgr2=0x7f7854100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:43.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.011+0000 7f785be7d640 1 --2- 172.21.15.120:0/712136751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7854100a90 0x7f7854100ef0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f7844009930 tx=0x7f784402a420 comp rx=0 tx=0).stop 2024-09-19T09:15:43.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.011+0000 7f785be7d640 1 -- 172.21.15.120:0/712136751 shutdown_connections 2024-09-19T09:15:43.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.011+0000 7f785be7d640 1 --2- 172.21.15.120:0/712136751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7854101430 0x7f785410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.011+0000 7f785be7d640 1 --2- 172.21.15.120:0/712136751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7854100a90 0x7f7854100ef0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.011+0000 7f785be7d640 1 --2- 172.21.15.120:0/712136751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7854106a90 0x7f7854106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.011+0000 7f785be7d640 1 -- 172.21.15.120:0/712136751 >> 172.21.15.120:0/712136751 conn(0x7f78540fc7d0 msgr2=0x7f78540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:43.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.012+0000 7f785be7d640 1 -- 172.21.15.120:0/712136751 shutdown_connections 2024-09-19T09:15:43.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.012+0000 7f785be7d640 1 -- 172.21.15.120:0/712136751 wait complete. 2024-09-19T09:15:43.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.012+0000 7f785be7d640 1 Processor -- start 2024-09-19T09:15:43.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.012+0000 7f785be7d640 1 -- start start 2024-09-19T09:15:43.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.013+0000 7f785be7d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7854100a90 0x7f7854197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:43.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.013+0000 7f785be7d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7854101430 0x7f7854197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:43.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.013+0000 7f785be7d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7854106a90 0x7f785419bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:43.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.013+0000 7f785be7d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7854110e90 con 0x7f7854106a90 2024-09-19T09:15:43.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.013+0000 7f785be7d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7854110d10 con 0x7f7854100a90 2024-09-19T09:15:43.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.013+0000 7f785be7d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7854111010 con 0x7f7854101430 2024-09-19T09:15:43.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.013+0000 7f7859c19640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7854100a90 0x7f7854197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:43.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.013+0000 7f7859c19640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7854100a90 0x7f7854197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:45052/0 (socket says 172.21.15.120:45052) 2024-09-19T09:15:43.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.013+0000 7f785a41a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7854106a90 0x7f785419bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:43.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.013+0000 7f7859c19640 1 -- 172.21.15.120:0/3597182240 learned_addr learned my addr 172.21.15.120:0/3597182240 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:43.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.013+0000 7f7859418640 1 --2- 172.21.15.120:0/3597182240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7854101430 0x7f7854197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:43.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.013+0000 7f785a41a640 1 -- 172.21.15.120:0/3597182240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7854101430 msgr2=0x7f7854197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:43.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.013+0000 7f785a41a640 1 --2- 172.21.15.120:0/3597182240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7854101430 0x7f7854197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.013+0000 7f785a41a640 1 -- 172.21.15.120:0/3597182240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7854100a90 msgr2=0x7f7854197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:43.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.013+0000 7f785a41a640 1 --2- 172.21.15.120:0/3597182240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7854100a90 0x7f7854197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.013+0000 7f785a41a640 1 -- 172.21.15.120:0/3597182240 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f785419c2e0 con 0x7f7854106a90 2024-09-19T09:15:43.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.013+0000 7f785a41a640 1 --2- 172.21.15.120:0/3597182240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7854106a90 0x7f785419bc00 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f784c009fd0 tx=0x7f784c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:43.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.013+0000 7f784affd640 1 -- 172.21.15.120:0/3597182240 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c00ce20 con 0x7f7854106a90 2024-09-19T09:15:43.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.013+0000 7f784affd640 1 -- 172.21.15.120:0/3597182240 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f784c005020 con 0x7f7854106a90 2024-09-19T09:15:43.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.013+0000 7f785be7d640 1 -- 172.21.15.120:0/3597182240 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f785419c5d0 con 0x7f7854106a90 2024-09-19T09:15:43.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.013+0000 7f784affd640 1 -- 172.21.15.120:0/3597182240 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c010720 con 0x7f7854106a90 2024-09-19T09:15:43.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.014+0000 7f785be7d640 1 -- 172.21.15.120:0/3597182240 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f785406a790 con 0x7f7854106a90 2024-09-19T09:15:43.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.016+0000 7f784affd640 1 -- 172.21.15.120:0/3597182240 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f784c005540 con 0x7f7854106a90 2024-09-19T09:15:43.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.016+0000 7f785e256740 1 Processor -- start 2024-09-19T09:15:43.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.016+0000 7f785e256740 1 -- start start 2024-09-19T09:15:43.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.017+0000 7f785e256740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708796760 0x557708796b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:43.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.017+0000 7f785e256740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557708797080 0x5577087a1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:43.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.017+0000 7f785e256740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5577087a1940 0x5577087a3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:43.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.017+0000 7f785e256740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5577086fc2a0 con 0x5577087a1940 2024-09-19T09:15:43.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.017+0000 7f785e256740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5577086a3b10 con 0x557708796760 2024-09-19T09:15:43.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.017+0000 7f785e256740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5577086f8260 con 0x557708797080 2024-09-19T09:15:43.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.017+0000 7f784affd640 1 --2- 172.21.15.120:0/3597182240 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f78400a3240 0x7f78400a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:43.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.017+0000 7f7859418640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708796760 0x557708796b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:43.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.017+0000 7f7859c19640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557708797080 0x5577087a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:43.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.017+0000 7f7859418640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5577087a1940 0x5577087a3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:43.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.017+0000 7f7859c19640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557708797080 0x5577087a1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:50510/0 (socket says 172.21.15.120:50510) 2024-09-19T09:15:43.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.017+0000 7f7859c19640 1 -- 172.21.15.120:0/1009732438 learned_addr learned my addr 172.21.15.120:0/1009732438 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:43.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.017+0000 7f785a41a640 1 --2- 172.21.15.120:0/3597182240 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f78400a3240 0x7f78400a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:43.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.017+0000 7f784affd640 1 -- 172.21.15.120:0/3597182240 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f784c0cff00 con 0x7f7854106a90 2024-09-19T09:15:43.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.017+0000 7f7859c19640 1 -- 172.21.15.120:0/1009732438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708796760 msgr2=0x557708796b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:43.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.017+0000 7f7859c19640 1 --2- 172.21.15.120:0/1009732438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708796760 0x557708796b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.017+0000 7f7859c19640 1 -- 172.21.15.120:0/1009732438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5577087a1940 msgr2=0x5577087a3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:43.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.017+0000 7f7859c19640 1 --2- 172.21.15.120:0/1009732438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5577087a1940 0x5577087a3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.017+0000 7f7859c19640 1 -- 172.21.15.120:0/1009732438 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55770846db90 con 0x557708797080 2024-09-19T09:15:43.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.017+0000 7f7859418640 1 --2- 172.21.15.120:0/1009732438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5577087a1940 0x5577087a3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:15:43.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.017+0000 7f785a41a640 1 --2- 172.21.15.120:0/3597182240 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f78400a3240 0x7f78400a5700 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f784c00c760 tx=0x7f784c066f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:43.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.017+0000 7f7859c19640 1 --2- 172.21.15.120:0/1009732438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557708797080 0x5577087a1400 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7850009d40 tx=0x7f7850011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd5cb6f52e70597d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:43.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.018+0000 7f783ffff640 1 -- 172.21.15.120:0/1009732438 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850004220 con 0x557708797080 2024-09-19T09:15:43.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.018+0000 7f783ffff640 1 -- 172.21.15.120:0/1009732438 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78500043c0 con 0x557708797080 2024-09-19T09:15:43.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.018+0000 7f783ffff640 1 -- 172.21.15.120:0/1009732438 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850004e60 con 0x557708797080 2024-09-19T09:15:43.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.018+0000 7f785e256740 1 -- 172.21.15.120:0/1009732438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557708797080 msgr2=0x5577087a1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:43.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.018+0000 7f785e256740 1 --2- 172.21.15.120:0/1009732438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557708797080 0x5577087a1400 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7850009d40 tx=0x7f7850011810 comp rx=0 tx=0).stop 2024-09-19T09:15:43.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.018+0000 7f785e256740 1 -- 172.21.15.120:0/1009732438 shutdown_connections 2024-09-19T09:15:43.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.018+0000 7f785e256740 1 --2- 172.21.15.120:0/1009732438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5577087a1940 0x5577087a3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.018+0000 7f785e256740 1 --2- 172.21.15.120:0/1009732438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557708797080 0x5577087a1400 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.018+0000 7f785e256740 1 --2- 172.21.15.120:0/1009732438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708796760 0x557708796b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.018+0000 7f785e256740 1 -- 172.21.15.120:0/1009732438 >> 172.21.15.120:0/1009732438 conn(0x557708792cf0 msgr2=0x5577087930d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:43.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.019+0000 7f785e256740 1 -- 172.21.15.120:0/1009732438 shutdown_connections 2024-09-19T09:15:43.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.019+0000 7f785e256740 1 -- 172.21.15.120:0/1009732438 wait complete. 2024-09-19T09:15:43.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.019+0000 7f785e256740 1 -- start start 2024-09-19T09:15:43.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.019+0000 7f785e256740 1 Processor -- start 2024-09-19T09:15:43.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.019+0000 7f785e256740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557708796760 0x55770879e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:43.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.019+0000 7f785e256740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557708797080 0x55770879ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:43.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.019+0000 7f785e256740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577087a1940 0x55770879f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:43.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.019+0000 7f785e256740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5577087a8920 con 0x557708797080 2024-09-19T09:15:43.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.019+0000 7f785e256740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5577086846e0 con 0x5577087a1940 2024-09-19T09:15:43.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.019+0000 7f785e256740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5577087a8aa0 con 0x557708796760 2024-09-19T09:15:43.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.019+0000 7f785e256740 10 client.0 _ll_register_callbacks cb 0x5577086a8a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:43.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.019+0000 7f7859c19640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557708797080 0x55770879ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:43.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.019+0000 7f7859c19640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557708797080 0x55770879ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46048/0 (socket says 172.21.15.120:46048) 2024-09-19T09:15:43.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.020+0000 7f7859418640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557708796760 0x55770879e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:43.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.020+0000 7f7859c19640 1 -- 172.21.15.120:0/3413883094 learned_addr learned my addr 172.21.15.120:0/3413883094 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:43.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.020+0000 7f7859418640 1 --2- 172.21.15.120:0/3413883094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577087a1940 0x55770879f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:43.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.020+0000 7f7859c19640 1 -- 172.21.15.120:0/3413883094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557708796760 msgr2=0x55770879e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:43.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.020+0000 7f7859c19640 1 --2- 172.21.15.120:0/3413883094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557708796760 0x55770879e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.020+0000 7f7859c19640 1 -- 172.21.15.120:0/3413883094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577087a1940 msgr2=0x55770879f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:43.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.020+0000 7f7859c19640 1 --2- 172.21.15.120:0/3413883094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577087a1940 0x55770879f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.020+0000 7f7859c19640 1 -- 172.21.15.120:0/3413883094 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577087b1dc0 con 0x557708797080 2024-09-19T09:15:43.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.020+0000 7f7859418640 1 --2- 172.21.15.120:0/3413883094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557708796760 0x55770879e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:15:43.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.020+0000 7f7859c19640 1 --2- 172.21.15.120:0/3413883094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557708797080 0x55770879ed20 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f7850009620 tx=0x7f7850019b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99c8b7f02a5bf6a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:43.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.020+0000 7f783d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:15:43.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.020+0000 7f783d7fa640 1 -- 172.21.15.120:0/3413883094 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850028020 con 0x557708797080 2024-09-19T09:15:43.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.020+0000 7f785e256740 1 -- 172.21.15.120:0/3413883094 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78500043c0 con 0x557708797080 2024-09-19T09:15:43.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.020+0000 7f783d7fa640 1 -- 172.21.15.120:0/3413883094 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f7850019cb0 con 0x557708797080 2024-09-19T09:15:43.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.020+0000 7f783d7fa640 1 -- 172.21.15.120:0/3413883094 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f785001d030 con 0x557708797080 2024-09-19T09:15:43.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.020+0000 7f785e256740 10 client.19875 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:43.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.020+0000 7f785e256740 1 -- 172.21.15.120:0/3413883094 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5577087b4730 con 0x557708797080 2024-09-19T09:15:43.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.020+0000 7f783d7fa640 1 -- 172.21.15.120:0/3413883094 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78500043c0 con 0x557708797080 2024-09-19T09:15:43.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.021+0000 7f783d7fa640 1 -- 172.21.15.120:0/3413883094 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f7850023070 con 0x557708797080 2024-09-19T09:15:43.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.021+0000 7f785e256740 10 client.19875 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:43.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.021+0000 7f785e256740 10 client.19875 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:15:43.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.021+0000 7f785e256740 20 client.19875 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:43.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.021+0000 7f785e256740 1 --2- 172.21.15.120:0/3413883094 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5577087b5b40 0x5577087b7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:43.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.021+0000 7f785e256740 4 client.19875 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:15:43.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.021+0000 7f785e256740 1 -- 172.21.15.120:0/3413883094 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577087099d0 con 0x5577087b5b40 2024-09-19T09:15:43.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.021+0000 7f7859418640 1 --2- 172.21.15.120:0/3413883094 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5577087b5b40 0x5577087b7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:43.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.021+0000 7f7859418640 1 --2- 172.21.15.120:0/3413883094 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5577087b5b40 0x5577087b7f30 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd3d939176538ffc server_cookie=39935af80c25f20b in_seq=0 out_seq=0 2024-09-19T09:15:43.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.021+0000 7f783d7fa640 10 client.19875 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:43.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.022+0000 7f783d7fa640 1 -- 172.21.15.120:0/3413883094 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5577087099d0 con 0x5577087b5b40 2024-09-19T09:15:43.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.022+0000 7f783d7fa640 10 client.19875 handle_command_reply: tid=0 2024-09-19T09:15:43.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.022+0000 7f785e256740 1 client.19875 shutdown 2024-09-19T09:15:43.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.022+0000 7f785e256740 1 -- 172.21.15.120:0/3413883094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557708797080 msgr2=0x55770879ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:43.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.022+0000 7f785e256740 1 --2- 172.21.15.120:0/3413883094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557708797080 0x55770879ed20 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f7850009620 tx=0x7f7850019b40 comp rx=0 tx=0).stop 2024-09-19T09:15:43.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.022+0000 7f785e256740 1 -- 172.21.15.120:0/3413883094 shutdown_connections 2024-09-19T09:15:43.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.022+0000 7f785e256740 1 --2- 172.21.15.120:0/3413883094 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5577087b5b40 0x5577087b7f30 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.022+0000 7f785e256740 1 --2- 172.21.15.120:0/3413883094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5577087a1940 0x55770879f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.022+0000 7f785e256740 1 --2- 172.21.15.120:0/3413883094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557708797080 0x55770879ed20 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.022+0000 7f785e256740 1 --2- 172.21.15.120:0/3413883094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557708796760 0x55770879e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.022+0000 7f783d7fa640 0 client.19875 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:43.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.022+0000 7f785e256740 1 -- 172.21.15.120:0/3413883094 >> 172.21.15.120:0/3413883094 conn(0x557708792cf0 msgr2=0x5577087989d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:43.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.023+0000 7f785e256740 1 -- 172.21.15.120:0/3413883094 shutdown_connections 2024-09-19T09:15:43.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.023+0000 7f785e256740 1 -- 172.21.15.120:0/3413883094 wait complete. 2024-09-19T09:15:43.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.023+0000 7f785e256740 20 client.19875 trim_cache size 0 max 16384 2024-09-19T09:15:43.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.041+0000 7f785e256740 1 Processor -- start 2024-09-19T09:15:43.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.041+0000 7f785e256740 1 -- start start 2024-09-19T09:15:43.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.041+0000 7f785e256740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708796760 0x5577087991c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:43.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.041+0000 7f785e256740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557708797080 0x55770879eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:43.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.041+0000 7f785e256740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577087a1940 0x55770879f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:43.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.041+0000 7f785e256740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f78541ab4a0 con 0x557708797080 2024-09-19T09:15:43.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.041+0000 7f785e256740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f78541acfe0 con 0x557708796760 2024-09-19T09:15:43.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.041+0000 7f785e256740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f78541abb70 con 0x5577087a1940 2024-09-19T09:15:43.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.041+0000 7f7859c19640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557708797080 0x55770879eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:43.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.041+0000 7f7859c19640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557708797080 0x55770879eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46056/0 (socket says 172.21.15.120:46056) 2024-09-19T09:15:43.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.041+0000 7f7859c19640 1 -- 172.21.15.120:0/3959889141 learned_addr learned my addr 172.21.15.120:0/3959889141 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:43.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.041+0000 7f7859418640 1 --2- 172.21.15.120:0/3959889141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708796760 0x5577087991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:43.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.041+0000 7f7859418640 1 --2- 172.21.15.120:0/3959889141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577087a1940 0x55770879f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:43.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.041+0000 7f7859c19640 1 -- 172.21.15.120:0/3959889141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577087a1940 msgr2=0x55770879f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:43.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.041+0000 7f7859c19640 1 --2- 172.21.15.120:0/3959889141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577087a1940 0x55770879f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.041+0000 7f7859c19640 1 -- 172.21.15.120:0/3959889141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708796760 msgr2=0x5577087991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:43.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.041+0000 7f7859c19640 1 --2- 172.21.15.120:0/3959889141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708796760 0x5577087991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.042+0000 7f7859c19640 1 -- 172.21.15.120:0/3959889141 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557708792b20 con 0x557708797080 2024-09-19T09:15:43.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.042+0000 7f7859418640 1 --2- 172.21.15.120:0/3959889141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708796760 0x5577087991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:15:43.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.042+0000 7f7859c19640 1 --2- 172.21.15.120:0/3959889141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557708797080 0x55770879eeb0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f7850011850 tx=0x7f7850009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d94ce8c32e3e13 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:43.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.042+0000 7f783f7fe640 1 -- 172.21.15.120:0/3959889141 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850028040 con 0x557708797080 2024-09-19T09:15:43.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.042+0000 7f783f7fe640 1 -- 172.21.15.120:0/3959889141 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7850009860 con 0x557708797080 2024-09-19T09:15:43.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.042+0000 7f783f7fe640 1 -- 172.21.15.120:0/3959889141 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850005520 con 0x557708797080 2024-09-19T09:15:43.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.042+0000 7f785e256740 1 -- 172.21.15.120:0/3959889141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557708797080 msgr2=0x55770879eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:43.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.042+0000 7f785e256740 1 --2- 172.21.15.120:0/3959889141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557708797080 0x55770879eeb0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f7850011850 tx=0x7f7850009c70 comp rx=0 tx=0).stop 2024-09-19T09:15:43.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.042+0000 7f785e256740 1 -- 172.21.15.120:0/3959889141 shutdown_connections 2024-09-19T09:15:43.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.042+0000 7f785e256740 1 --2- 172.21.15.120:0/3959889141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577087a1940 0x55770879f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.042+0000 7f785e256740 1 --2- 172.21.15.120:0/3959889141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557708797080 0x55770879eeb0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.042+0000 7f785e256740 1 --2- 172.21.15.120:0/3959889141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708796760 0x5577087991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.042+0000 7f785e256740 1 -- 172.21.15.120:0/3959889141 >> 172.21.15.120:0/3959889141 conn(0x557708792cf0 msgr2=0x55770879e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:43.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.043+0000 7f785e256740 1 -- 172.21.15.120:0/3959889141 shutdown_connections 2024-09-19T09:15:43.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.043+0000 7f785e256740 1 -- 172.21.15.120:0/3959889141 wait complete. 2024-09-19T09:15:43.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.043+0000 7f785e256740 1 -- start start 2024-09-19T09:15:43.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.043+0000 7f785e256740 1 Processor -- start 2024-09-19T09:15:43.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.043+0000 7f785e256740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708796760 0x5577087a9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:43.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.043+0000 7f785e256740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557708797080 0x5577087aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:43.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.043+0000 7f785e256740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577087a1940 0x5577087aa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:43.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.043+0000 7f785e256740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f78541ac910 con 0x557708797080 2024-09-19T09:15:43.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.043+0000 7f785e256740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f78541ac240 con 0x557708796760 2024-09-19T09:15:43.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.043+0000 7f785e256740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f78541a9460 con 0x5577087a1940 2024-09-19T09:15:43.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.043+0000 7f785e256740 10 client.0 _ll_register_callbacks cb 0x7f7850009bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:43.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.043+0000 7f7859c19640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557708797080 0x5577087aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:43.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.043+0000 7f7859c19640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557708797080 0x5577087aa2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46070/0 (socket says 172.21.15.120:46070) 2024-09-19T09:15:43.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.043+0000 7f7859c19640 1 -- 172.21.15.120:0/3966144469 learned_addr learned my addr 172.21.15.120:0/3966144469 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:43.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.043+0000 7f7859418640 1 --2- 172.21.15.120:0/3966144469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708796760 0x5577087a9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:43.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.044+0000 7f7859418640 1 --2- 172.21.15.120:0/3966144469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577087a1940 0x5577087aa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:43.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.044+0000 7f7859c19640 1 -- 172.21.15.120:0/3966144469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577087a1940 msgr2=0x5577087aa7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:43.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.044+0000 7f7859c19640 1 --2- 172.21.15.120:0/3966144469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577087a1940 0x5577087aa7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.044+0000 7f7859c19640 1 -- 172.21.15.120:0/3966144469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708796760 msgr2=0x5577087a9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:43.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.044+0000 7f7859c19640 1 --2- 172.21.15.120:0/3966144469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708796760 0x5577087a9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.044+0000 7f7859c19640 1 -- 172.21.15.120:0/3966144469 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55770878e260 con 0x557708797080 2024-09-19T09:15:43.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.044+0000 7f7859c19640 1 --2- 172.21.15.120:0/3966144469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557708797080 0x5577087aa2a0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f7850005160 tx=0x7f7850005530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac30cc84bee11a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:43.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.044+0000 7f783d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:15:43.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.044+0000 7f783d7fa640 1 -- 172.21.15.120:0/3966144469 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850028070 con 0x557708797080 2024-09-19T09:15:43.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.044+0000 7f783d7fa640 1 -- 172.21.15.120:0/3966144469 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f7850004ae0 con 0x557708797080 2024-09-19T09:15:43.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.044+0000 7f785e256740 1 -- 172.21.15.120:0/3966144469 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7850009860 con 0x557708797080 2024-09-19T09:15:43.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.044+0000 7f783d7fa640 1 -- 172.21.15.120:0/3966144469 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7850011d10 con 0x557708797080 2024-09-19T09:15:43.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.044+0000 7f785e256740 10 client.19881 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:43.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.044+0000 7f785e256740 1 -- 172.21.15.120:0/3966144469 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557708820ac0 con 0x557708797080 2024-09-19T09:15:43.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.044+0000 7f783d7fa640 1 -- 172.21.15.120:0/3966144469 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7850009860 con 0x557708797080 2024-09-19T09:15:43.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.045+0000 7f783d7fa640 1 -- 172.21.15.120:0/3966144469 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f785003f060 con 0x557708797080 2024-09-19T09:15:43.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.045+0000 7f785e256740 10 client.19881 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:43.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.045+0000 7f785e256740 10 client.19881 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:15:43.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.045+0000 7f785e256740 20 client.19881 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:43.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.045+0000 7f785e256740 1 --2- 172.21.15.120:0/3966144469 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5577088211e0 0x557708823600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:43.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.045+0000 7f785e256740 4 client.19881 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.2, "await_for": 45.0}] 2024-09-19T09:15:43.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.045+0000 7f785e256740 1 -- 172.21.15.120:0/3966144469 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.2, "await_for": 45.0}) -- 0x557708703d20 con 0x5577088211e0 2024-09-19T09:15:43.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.045+0000 7f7859418640 1 --2- 172.21.15.120:0/3966144469 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5577088211e0 0x557708823600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:43.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.045+0000 7f783d7fa640 10 client.19881 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:43.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.045+0000 7f7859418640 1 --2- 172.21.15.120:0/3966144469 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5577088211e0 0x557708823600 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63c1c7e37f4ee0d5 server_cookie=ff31d6b2c54980a5 in_seq=0 out_seq=0 2024-09-19T09:15:43.095 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1301014 10.07 MB/sec execute 578 sec latency 15.702 ms 2024-09-19T09:15:43.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.123+0000 7f783d7fa640 1 -- 172.21.15.120:0/3966144469 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557708703d20 con 0x5577088211e0 2024-09-19T09:15:43.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.123+0000 7f783d7fa640 10 client.19881 handle_command_reply: tid=0 2024-09-19T09:15:43.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.123+0000 7f785e256740 1 client.19881 shutdown 2024-09-19T09:15:43.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.123+0000 7f785e256740 1 -- 172.21.15.120:0/3966144469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557708797080 msgr2=0x5577087aa2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:43.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.123+0000 7f785e256740 1 --2- 172.21.15.120:0/3966144469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557708797080 0x5577087aa2a0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f7850005160 tx=0x7f7850005530 comp rx=0 tx=0).stop 2024-09-19T09:15:43.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.123+0000 7f785e256740 1 -- 172.21.15.120:0/3966144469 shutdown_connections 2024-09-19T09:15:43.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.123+0000 7f785e256740 1 --2- 172.21.15.120:0/3966144469 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5577088211e0 0x557708823600 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.123+0000 7f785e256740 1 --2- 172.21.15.120:0/3966144469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5577087a1940 0x5577087aa7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.123+0000 7f785e256740 1 --2- 172.21.15.120:0/3966144469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557708797080 0x5577087aa2a0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.123+0000 7f785e256740 1 --2- 172.21.15.120:0/3966144469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557708796760 0x5577087a9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.123+0000 7f783d7fa640 0 client.19881 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:43.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.123+0000 7f785e256740 1 -- 172.21.15.120:0/3966144469 >> 172.21.15.120:0/3966144469 conn(0x557708792cf0 msgr2=0x55770879efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:43.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.123+0000 7f785e256740 1 -- 172.21.15.120:0/3966144469 shutdown_connections 2024-09-19T09:15:43.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.123+0000 7f785e256740 1 -- 172.21.15.120:0/3966144469 wait complete. 2024-09-19T09:15:43.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.123+0000 7f785e256740 20 client.19881 trim_cache size 0 max 16384 2024-09-19T09:15:43.125 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:15:43.125 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 33, 2024-09-19T09:15:43.126 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:15:43.126 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 42, 2024-09-19T09:15:43.126 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:15:43.126 INFO:teuthology.orchestra.run.smithi120.stdout: "ce305c23": { 2024-09-19T09:15:43.126 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 42, 2024-09-19T09:15:43.126 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:15:43.126 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:15:43.126 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:15:43.126 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:15:43.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:15:43.126 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:15:43.126 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 142.2, 2024-09-19T09:15:43.126 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:15:43.126 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:15:43.127 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:15:43.127 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:15:43.127 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:15:43.127 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:15:43.127 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:15:43.127 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:15:43.127 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:15:43.127 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:15:43.127 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:15:43.127 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:15:43.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.124+0000 7f785be7d640 1 -- 172.21.15.120:0/3597182240 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f78400a3240 msgr2=0x7f78400a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:43.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.124+0000 7f785be7d640 1 --2- 172.21.15.120:0/3597182240 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f78400a3240 0x7f78400a5700 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f784c00c760 tx=0x7f784c066f60 comp rx=0 tx=0).stop 2024-09-19T09:15:43.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.124+0000 7f785be7d640 1 -- 172.21.15.120:0/3597182240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7854106a90 msgr2=0x7f785419bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:43.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.124+0000 7f785be7d640 1 --2- 172.21.15.120:0/3597182240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7854106a90 0x7f785419bc00 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f784c009fd0 tx=0x7f784c00ef10 comp rx=0 tx=0).stop 2024-09-19T09:15:43.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.124+0000 7f785be7d640 1 -- 172.21.15.120:0/3597182240 shutdown_connections 2024-09-19T09:15:43.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.124+0000 7f785be7d640 1 --2- 172.21.15.120:0/3597182240 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f78400a3240 0x7f78400a5700 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.124+0000 7f785be7d640 1 --2- 172.21.15.120:0/3597182240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7854106a90 0x7f785419bc00 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.124+0000 7f785be7d640 1 --2- 172.21.15.120:0/3597182240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7854101430 0x7f7854197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.124+0000 7f785be7d640 1 --2- 172.21.15.120:0/3597182240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7854100a90 0x7f7854197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:43.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.124+0000 7f785be7d640 1 -- 172.21.15.120:0/3597182240 >> 172.21.15.120:0/3597182240 conn(0x7f78540fc7d0 msgr2=0x7f78540fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:43.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.124+0000 7f785be7d640 1 -- 172.21.15.120:0/3597182240 shutdown_connections 2024-09-19T09:15:43.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:43.124+0000 7f785be7d640 1 -- 172.21.15.120:0/3597182240 wait complete. 2024-09-19T09:15:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:42 smithi196 ceph-mon[35150]: pgmap v729: 97 pgs: 97 active+clean; 41 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.7 MiB/s wr, 420 op/s 2024-09-19T09:15:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:42 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/994233378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:15:43.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:42 smithi153 ceph-mon[35071]: pgmap v729: 97 pgs: 97 active+clean; 41 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.7 MiB/s wr, 420 op/s 2024-09-19T09:15:43.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:42 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/994233378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:15:43.814 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:15:43.842 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ce305c23', quiesce took 5.1 seconds. Will release after: 17.1 2024-09-19T09:15:44.078 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1301014 10.05 MB/sec execute 579 sec latency 1015.065 ms 2024-09-19T09:15:44.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.460+0000 7f3ea7489640 1 -- 172.21.15.120:0/1136259784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ea0102ef0 msgr2=0x7f3ea010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:44.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.460+0000 7f3ea7489640 1 --2- 172.21.15.120:0/1136259784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ea0102ef0 0x7f3ea010da40 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f3e98009900 tx=0x7f3e9802a480 comp rx=0 tx=0).stop 2024-09-19T09:15:44.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.460+0000 7f3ea7489640 1 -- 172.21.15.120:0/1136259784 shutdown_connections 2024-09-19T09:15:44.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.460+0000 7f3ea7489640 1 --2- 172.21.15.120:0/1136259784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ea0102ef0 0x7f3ea010da40 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.460+0000 7f3ea7489640 1 --2- 172.21.15.120:0/1136259784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ea0102570 0x7f3ea01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.460+0000 7f3ea7489640 1 --2- 172.21.15.120:0/1136259784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ea0100e20 0x7f3ea0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.460+0000 7f3ea7489640 1 -- 172.21.15.120:0/1136259784 >> 172.21.15.120:0/1136259784 conn(0x7f3ea00fc7d0 msgr2=0x7f3ea00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:44.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.461+0000 7f3ea7489640 1 -- 172.21.15.120:0/1136259784 shutdown_connections 2024-09-19T09:15:44.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.461+0000 7f3ea7489640 1 -- 172.21.15.120:0/1136259784 wait complete. 2024-09-19T09:15:44.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.461+0000 7f3ea7489640 1 Processor -- start 2024-09-19T09:15:44.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.461+0000 7f3ea7489640 1 -- start start 2024-09-19T09:15:44.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.461+0000 7f3ea7489640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ea0100e20 0x7f3ea01993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:44.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.461+0000 7f3ea7489640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ea0102570 0x7f3ea0199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:44.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.461+0000 7f3ea7489640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ea0102ef0 0x7f3ea0199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:44.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.461+0000 7f3ea7489640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea0110f80 con 0x7f3ea0102570 2024-09-19T09:15:44.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.461+0000 7f3ea7489640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea0110e00 con 0x7f3ea0100e20 2024-09-19T09:15:44.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.461+0000 7f3ea7489640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea0111100 con 0x7f3ea0102ef0 2024-09-19T09:15:44.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.461+0000 7f3ea4a24640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ea0102570 0x7f3ea0199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:44.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.462+0000 7f3ea4a24640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ea0102570 0x7f3ea0199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46094/0 (socket says 172.21.15.120:46094) 2024-09-19T09:15:44.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.462+0000 7f3ea4a24640 1 -- 172.21.15.120:0/3815124410 learned_addr learned my addr 172.21.15.120:0/3815124410 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:44.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.462+0000 7f3ea5a26640 1 --2- 172.21.15.120:0/3815124410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ea0102ef0 0x7f3ea0199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:44.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.462+0000 7f3ea5225640 1 --2- 172.21.15.120:0/3815124410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ea0100e20 0x7f3ea01993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:44.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.462+0000 7f3ea4a24640 1 -- 172.21.15.120:0/3815124410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ea0102ef0 msgr2=0x7f3ea0199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:44.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.462+0000 7f3ea4a24640 1 --2- 172.21.15.120:0/3815124410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ea0102ef0 0x7f3ea0199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.462+0000 7f3ea4a24640 1 -- 172.21.15.120:0/3815124410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ea0100e20 msgr2=0x7f3ea01993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:44.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.462+0000 7f3ea4a24640 1 --2- 172.21.15.120:0/3815124410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ea0100e20 0x7f3ea01993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.462+0000 7f3ea4a24640 1 -- 172.21.15.120:0/3815124410 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea01a47d0 con 0x7f3ea0102570 2024-09-19T09:15:44.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.462+0000 7f3ea5a26640 1 --2- 172.21.15.120:0/3815124410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ea0102ef0 0x7f3ea0199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:15:44.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.462+0000 7f3ea4a24640 1 --2- 172.21.15.120:0/3815124410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ea0102570 0x7f3ea0199910 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400c890 tx=0x7f3e9400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:44.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.462+0000 7f3e9e7fc640 1 -- 172.21.15.120:0/3815124410 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e94004220 con 0x7f3ea0102570 2024-09-19T09:15:44.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.462+0000 7f3ea7489640 1 -- 172.21.15.120:0/3815124410 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea01a4ac0 con 0x7f3ea0102570 2024-09-19T09:15:44.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.462+0000 7f3e9e7fc640 1 -- 172.21.15.120:0/3815124410 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e940043c0 con 0x7f3ea0102570 2024-09-19T09:15:44.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.462+0000 7f3e9e7fc640 1 -- 172.21.15.120:0/3815124410 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e94004a00 con 0x7f3ea0102570 2024-09-19T09:15:44.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.462+0000 7f3ea7489640 1 -- 172.21.15.120:0/3815124410 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea01a5050 con 0x7f3ea0102570 2024-09-19T09:15:44.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.464+0000 7f3e9e7fc640 1 -- 172.21.15.120:0/3815124410 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f3e940079b0 con 0x7f3ea0102570 2024-09-19T09:15:44.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.465+0000 7f3ea9862740 1 Processor -- start 2024-09-19T09:15:44.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.465+0000 7f3ea9862740 1 -- start start 2024-09-19T09:15:44.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.465+0000 7f3e9e7fc640 1 --2- 172.21.15.120:0/3815124410 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3e880a3240 0x7f3e880a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:44.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.465+0000 7f3ea9862740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556474150b40 0x55647414f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:44.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.465+0000 7f3ea9862740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55647414f5d0 0x55647414fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:44.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.465+0000 7f3ea9862740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55647415e780 0x556474160b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:44.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.465+0000 7f3ea9862740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5564740b52f0 con 0x556474150b40 2024-09-19T09:15:44.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.465+0000 7f3ea9862740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55647405ccf0 con 0x55647415e780 2024-09-19T09:15:44.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.465+0000 7f3ea9862740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5564740b12b0 con 0x55647414f5d0 2024-09-19T09:15:44.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.465+0000 7f3e9e7fc640 1 -- 172.21.15.120:0/3815124410 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f3e940cf650 con 0x7f3ea0102570 2024-09-19T09:15:44.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.465+0000 7f3ea5225640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556474150b40 0x55647414f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:44.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.465+0000 7f3ea5a26640 1 --2- 172.21.15.120:0/3815124410 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3e880a3240 0x7f3e880a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:44.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.465+0000 7f3ea5225640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556474150b40 0x55647414f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46108/0 (socket says 172.21.15.120:46108) 2024-09-19T09:15:44.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.465+0000 7f3ea5225640 1 -- 172.21.15.120:0/124013951 learned_addr learned my addr 172.21.15.120:0/124013951 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:44.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.466+0000 7f3ea4a24640 1 --2- 172.21.15.120:0/124013951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55647414f5d0 0x55647414fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:44.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.466+0000 7f3ea5225640 1 -- 172.21.15.120:0/124013951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55647414f5d0 msgr2=0x55647414fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:44.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.466+0000 7f3ea5225640 1 --2- 172.21.15.120:0/124013951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55647414f5d0 0x55647414fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.466+0000 7f3ea5225640 1 -- 172.21.15.120:0/124013951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55647415e780 msgr2=0x556474160b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:44.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.466+0000 7f3ea5225640 1 --2- 172.21.15.120:0/124013951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55647415e780 0x556474160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.466+0000 7f3ea5225640 1 -- 172.21.15.120:0/124013951 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564740413a0 con 0x556474150b40 2024-09-19T09:15:44.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.466+0000 7f3ea5a26640 1 --2- 172.21.15.120:0/3815124410 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3e880a3240 0x7f3e880a5700 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3e98005940 tx=0x7f3e980058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:44.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.466+0000 7f3ea5225640 1 --2- 172.21.15.120:0/124013951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556474150b40 0x55647414f090 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f3e90009f60 tx=0x7f3e90009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9048142397265c4d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:44.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.466+0000 7f3e877fe640 1 -- 172.21.15.120:0/124013951 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e90016070 con 0x556474150b40 2024-09-19T09:15:44.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.466+0000 7f3e877fe640 1 -- 172.21.15.120:0/124013951 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e90004d20 con 0x556474150b40 2024-09-19T09:15:44.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.466+0000 7f3e877fe640 1 -- 172.21.15.120:0/124013951 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e900127b0 con 0x556474150b40 2024-09-19T09:15:44.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.467+0000 7f3ea9862740 1 -- 172.21.15.120:0/124013951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556474150b40 msgr2=0x55647414f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:44.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.467+0000 7f3ea9862740 1 --2- 172.21.15.120:0/124013951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556474150b40 0x55647414f090 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f3e90009f60 tx=0x7f3e90009f90 comp rx=0 tx=0).stop 2024-09-19T09:15:44.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.467+0000 7f3ea9862740 1 -- 172.21.15.120:0/124013951 shutdown_connections 2024-09-19T09:15:44.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.467+0000 7f3ea9862740 1 --2- 172.21.15.120:0/124013951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55647415e780 0x556474160b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.467+0000 7f3ea9862740 1 --2- 172.21.15.120:0/124013951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55647414f5d0 0x55647414fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.467+0000 7f3ea9862740 1 --2- 172.21.15.120:0/124013951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556474150b40 0x55647414f090 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.467+0000 7f3ea9862740 1 -- 172.21.15.120:0/124013951 >> 172.21.15.120:0/124013951 conn(0x556474118310 msgr2=0x55647414bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:44.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.467+0000 7f3ea9862740 1 -- 172.21.15.120:0/124013951 shutdown_connections 2024-09-19T09:15:44.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.467+0000 7f3ea9862740 1 -- 172.21.15.120:0/124013951 wait complete. 2024-09-19T09:15:44.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.467+0000 7f3ea9862740 1 -- start start 2024-09-19T09:15:44.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.467+0000 7f3ea9862740 1 Processor -- start 2024-09-19T09:15:44.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.468+0000 7f3ea9862740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55647414f5d0 0x55647415e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:44.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.468+0000 7f3ea9862740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556474150b40 0x55647415b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:44.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.468+0000 7f3ea9862740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55647415e780 0x55647415b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:44.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.468+0000 7f3ea9862740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556474161480 con 0x55647414f5d0 2024-09-19T09:15:44.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.468+0000 7f3ea9862740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556474070060 con 0x556474150b40 2024-09-19T09:15:44.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.468+0000 7f3ea9862740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556474161600 con 0x55647415e780 2024-09-19T09:15:44.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.468+0000 7f3ea9862740 10 client.0 _ll_register_callbacks cb 0x556473dc38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:44.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.468+0000 7f3ea5225640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55647414f5d0 0x55647415e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:44.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.468+0000 7f3ea5225640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55647414f5d0 0x55647415e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46118/0 (socket says 172.21.15.120:46118) 2024-09-19T09:15:44.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.468+0000 7f3ea5225640 1 -- 172.21.15.120:0/1864879387 learned_addr learned my addr 172.21.15.120:0/1864879387 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:44.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.468+0000 7f3ea4a24640 1 --2- 172.21.15.120:0/1864879387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556474150b40 0x55647415b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:44.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.468+0000 7f3ea5a26640 1 --2- 172.21.15.120:0/1864879387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55647415e780 0x55647415b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:44.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.468+0000 7f3ea5225640 1 -- 172.21.15.120:0/1864879387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55647415e780 msgr2=0x55647415b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:44.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.468+0000 7f3ea5225640 1 --2- 172.21.15.120:0/1864879387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55647415e780 0x55647415b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.468+0000 7f3ea5225640 1 -- 172.21.15.120:0/1864879387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556474150b40 msgr2=0x55647415b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:44.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.468+0000 7f3ea5225640 1 --2- 172.21.15.120:0/1864879387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556474150b40 0x55647415b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.468+0000 7f3ea5225640 1 -- 172.21.15.120:0/1864879387 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556473e263c0 con 0x55647414f5d0 2024-09-19T09:15:44.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.468+0000 7f3ea5225640 1 --2- 172.21.15.120:0/1864879387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55647414f5d0 0x55647415e160 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f3e90006010 tx=0x7f3e90004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75fa72da432cc3ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:44.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.468+0000 7f3e84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:15:44.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.469+0000 7f3e84ff9640 1 -- 172.21.15.120:0/1864879387 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e90028060 con 0x55647414f5d0 2024-09-19T09:15:44.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.469+0000 7f3e84ff9640 1 -- 172.21.15.120:0/1864879387 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f3e900052f0 con 0x55647414f5d0 2024-09-19T09:15:44.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.469+0000 7f3ea9862740 1 -- 172.21.15.120:0/1864879387 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e90004d20 con 0x55647414f5d0 2024-09-19T09:15:44.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.469+0000 7f3e84ff9640 1 -- 172.21.15.120:0/1864879387 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3e90016070 con 0x55647414f5d0 2024-09-19T09:15:44.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.469+0000 7f3ea9862740 10 client.19893 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:44.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.469+0000 7f3ea9862740 1 -- 172.21.15.120:0/1864879387 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55647416cf30 con 0x55647414f5d0 2024-09-19T09:15:44.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.469+0000 7f3e84ff9640 1 -- 172.21.15.120:0/1864879387 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e90004d20 con 0x55647414f5d0 2024-09-19T09:15:44.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.469+0000 7f3e84ff9640 1 -- 172.21.15.120:0/1864879387 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f3e9002d070 con 0x55647414f5d0 2024-09-19T09:15:44.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.469+0000 7f3ea9862740 10 client.19893 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:44.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.469+0000 7f3ea9862740 10 client.19893 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:15:44.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.469+0000 7f3ea9862740 20 client.19893 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:44.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.469+0000 7f3ea9862740 1 --2- 172.21.15.120:0/1864879387 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55647416e330 0x556474170720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:44.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.469+0000 7f3ea9862740 4 client.19893 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:15:44.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.469+0000 7f3ea9862740 1 -- 172.21.15.120:0/1864879387 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564740c2860 con 0x55647416e330 2024-09-19T09:15:44.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.469+0000 7f3ea4a24640 1 --2- 172.21.15.120:0/1864879387 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55647416e330 0x556474170720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:44.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.470+0000 7f3e84ff9640 10 client.19893 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:15:44.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.470+0000 7f3ea4a24640 1 --2- 172.21.15.120:0/1864879387 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55647416e330 0x556474170720 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2dec6dafc27935a2 server_cookie=f4a18ef5e2df022d in_seq=0 out_seq=0 2024-09-19T09:15:44.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.470+0000 7f3e84ff9640 1 -- 172.21.15.120:0/1864879387 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5564740c2860 con 0x55647416e330 2024-09-19T09:15:44.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.470+0000 7f3e84ff9640 10 client.19893 handle_command_reply: tid=0 2024-09-19T09:15:44.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.470+0000 7f3ea9862740 1 client.19893 shutdown 2024-09-19T09:15:44.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.471+0000 7f3ea9862740 1 -- 172.21.15.120:0/1864879387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55647414f5d0 msgr2=0x55647415e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:44.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.471+0000 7f3ea9862740 1 --2- 172.21.15.120:0/1864879387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55647414f5d0 0x55647415e160 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f3e90006010 tx=0x7f3e90004c70 comp rx=0 tx=0).stop 2024-09-19T09:15:44.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.471+0000 7f3ea9862740 1 -- 172.21.15.120:0/1864879387 shutdown_connections 2024-09-19T09:15:44.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.471+0000 7f3ea9862740 1 --2- 172.21.15.120:0/1864879387 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55647416e330 0x556474170720 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.471+0000 7f3ea9862740 1 --2- 172.21.15.120:0/1864879387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55647415e780 0x55647415b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.471+0000 7f3ea9862740 1 --2- 172.21.15.120:0/1864879387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556474150b40 0x55647415b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.471+0000 7f3ea9862740 1 --2- 172.21.15.120:0/1864879387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55647414f5d0 0x55647415e160 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.471+0000 7f3e84ff9640 0 client.19893 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:15:44.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.471+0000 7f3ea9862740 1 -- 172.21.15.120:0/1864879387 >> 172.21.15.120:0/1864879387 conn(0x556474118310 msgr2=0x556474151930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:44.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.471+0000 7f3ea9862740 1 -- 172.21.15.120:0/1864879387 shutdown_connections 2024-09-19T09:15:44.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.471+0000 7f3ea9862740 1 -- 172.21.15.120:0/1864879387 wait complete. 2024-09-19T09:15:44.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.471+0000 7f3ea9862740 20 client.19893 trim_cache size 0 max 16384 2024-09-19T09:15:44.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.484+0000 7f3ea9862740 1 Processor -- start 2024-09-19T09:15:44.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.484+0000 7f3ea9862740 1 -- start start 2024-09-19T09:15:44.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.484+0000 7f3ea9862740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55647414f5d0 0x55647414ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:44.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.484+0000 7f3ea9862740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556474150b40 0x556474173740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:44.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.484+0000 7f3ea9862740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55647415e780 0x556474173c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:44.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.484+0000 7f3ea9862740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea01ab4b0 con 0x55647414f5d0 2024-09-19T09:15:44.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.484+0000 7f3ea9862740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea01acff0 con 0x556474150b40 2024-09-19T09:15:44.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.484+0000 7f3ea9862740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea01abb80 con 0x55647415e780 2024-09-19T09:15:44.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.484+0000 7f3ea5225640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55647414f5d0 0x55647414ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:44.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.484+0000 7f3ea5225640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55647414f5d0 0x55647414ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46126/0 (socket says 172.21.15.120:46126) 2024-09-19T09:15:44.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.484+0000 7f3ea5225640 1 -- 172.21.15.120:0/1053419227 learned_addr learned my addr 172.21.15.120:0/1053419227 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:44.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.484+0000 7f3ea4a24640 1 --2- 172.21.15.120:0/1053419227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556474150b40 0x556474173740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:44.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.484+0000 7f3ea5a26640 1 --2- 172.21.15.120:0/1053419227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55647415e780 0x556474173c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:44.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.484+0000 7f3ea5225640 1 -- 172.21.15.120:0/1053419227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55647415e780 msgr2=0x556474173c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:44.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.484+0000 7f3ea5225640 1 --2- 172.21.15.120:0/1053419227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55647415e780 0x556474173c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.484+0000 7f3ea5225640 1 -- 172.21.15.120:0/1053419227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556474150b40 msgr2=0x556474173740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:44.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.484+0000 7f3ea5225640 1 --2- 172.21.15.120:0/1053419227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556474150b40 0x556474173740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.484+0000 7f3ea5225640 1 -- 172.21.15.120:0/1053419227 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556474150740 con 0x55647414f5d0 2024-09-19T09:15:44.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.484+0000 7f3ea4a24640 1 --2- 172.21.15.120:0/1053419227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556474150b40 0x556474173740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:15:44.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.484+0000 7f3ea5225640 1 --2- 172.21.15.120:0/1053419227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55647414f5d0 0x55647414ebd0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f3e90005c40 tx=0x7f3e90009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87d0927afd695ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:44.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.484+0000 7f3e86ffd640 1 -- 172.21.15.120:0/1053419227 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e90033070 con 0x55647414f5d0 2024-09-19T09:15:44.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.485+0000 7f3e86ffd640 1 -- 172.21.15.120:0/1053419227 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e900233e0 con 0x55647414f5d0 2024-09-19T09:15:44.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.485+0000 7f3e86ffd640 1 -- 172.21.15.120:0/1053419227 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e90028070 con 0x55647414f5d0 2024-09-19T09:15:44.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.485+0000 7f3ea9862740 1 -- 172.21.15.120:0/1053419227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55647414f5d0 msgr2=0x55647414ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:44.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.485+0000 7f3ea9862740 1 --2- 172.21.15.120:0/1053419227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55647414f5d0 0x55647414ebd0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f3e90005c40 tx=0x7f3e90009c60 comp rx=0 tx=0).stop 2024-09-19T09:15:44.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.485+0000 7f3ea9862740 1 -- 172.21.15.120:0/1053419227 shutdown_connections 2024-09-19T09:15:44.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.485+0000 7f3ea9862740 1 --2- 172.21.15.120:0/1053419227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55647415e780 0x556474173c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.485+0000 7f3ea9862740 1 --2- 172.21.15.120:0/1053419227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556474150b40 0x556474173740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.485+0000 7f3ea9862740 1 --2- 172.21.15.120:0/1053419227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55647414f5d0 0x55647414ebd0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.485+0000 7f3ea9862740 1 -- 172.21.15.120:0/1053419227 >> 172.21.15.120:0/1053419227 conn(0x556474118310 msgr2=0x55647415a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:44.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.485+0000 7f3ea9862740 1 -- 172.21.15.120:0/1053419227 shutdown_connections 2024-09-19T09:15:44.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.485+0000 7f3ea9862740 1 -- 172.21.15.120:0/1053419227 wait complete. 2024-09-19T09:15:44.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.485+0000 7f3ea9862740 1 -- start start 2024-09-19T09:15:44.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.486+0000 7f3ea9862740 1 Processor -- start 2024-09-19T09:15:44.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.486+0000 7f3ea9862740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55647414f5d0 0x556474171580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:44.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.486+0000 7f3ea9862740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556474150b40 0x556474171ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:44.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.486+0000 7f3ea9862740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55647415e780 0x556474172000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:44.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.486+0000 7f3ea9862740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea01ac920 con 0x55647415e780 2024-09-19T09:15:44.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.486+0000 7f3ea9862740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea01ac250 con 0x55647414f5d0 2024-09-19T09:15:44.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.486+0000 7f3ea9862740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea01a9470 con 0x556474150b40 2024-09-19T09:15:44.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.486+0000 7f3ea9862740 10 client.0 _ll_register_callbacks cb 0x7f3e90009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:44.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.486+0000 7f3ea5a26640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55647415e780 0x556474172000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:44.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.486+0000 7f3ea5a26640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55647415e780 0x556474172000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46130/0 (socket says 172.21.15.120:46130) 2024-09-19T09:15:44.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.486+0000 7f3ea5a26640 1 -- 172.21.15.120:0/240774379 learned_addr learned my addr 172.21.15.120:0/240774379 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:44.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.486+0000 7f3ea5225640 1 --2- 172.21.15.120:0/240774379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55647414f5d0 0x556474171580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:44.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.486+0000 7f3ea4a24640 1 --2- 172.21.15.120:0/240774379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556474150b40 0x556474171ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:44.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.486+0000 7f3ea5a26640 1 -- 172.21.15.120:0/240774379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556474150b40 msgr2=0x556474171ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:44.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.486+0000 7f3ea5a26640 1 --2- 172.21.15.120:0/240774379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556474150b40 0x556474171ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.486+0000 7f3ea5a26640 1 -- 172.21.15.120:0/240774379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55647414f5d0 msgr2=0x556474171580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:44.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.486+0000 7f3ea5a26640 1 --2- 172.21.15.120:0/240774379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55647414f5d0 0x556474171580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.486+0000 7f3ea5a26640 1 -- 172.21.15.120:0/240774379 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564741d7d20 con 0x55647415e780 2024-09-19T09:15:44.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.487+0000 7f3ea5a26640 1 --2- 172.21.15.120:0/240774379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55647415e780 0x556474172000 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f3e98009900 tx=0x7f3e98009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa298fa0749dbe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:44.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.487+0000 7f3e84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:15:44.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.487+0000 7f3e84ff9640 1 -- 172.21.15.120:0/240774379 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9804e020 con 0x55647415e780 2024-09-19T09:15:44.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.487+0000 7f3e84ff9640 1 -- 172.21.15.120:0/240774379 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f3e98004cd0 con 0x55647415e780 2024-09-19T09:15:44.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.487+0000 7f3ea9862740 1 -- 172.21.15.120:0/240774379 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e900233e0 con 0x55647415e780 2024-09-19T09:15:44.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.487+0000 7f3e84ff9640 1 -- 172.21.15.120:0/240774379 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3e98035070 con 0x55647415e780 2024-09-19T09:15:44.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.487+0000 7f3ea9862740 10 client.19899 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:44.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.487+0000 7f3ea9862740 1 -- 172.21.15.120:0/240774379 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5564741d8f70 con 0x55647415e780 2024-09-19T09:15:44.591 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:15:44.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.487+0000 7f3e84ff9640 1 -- 172.21.15.120:0/240774379 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e900233e0 con 0x55647415e780 2024-09-19T09:15:44.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.487+0000 7f3e84ff9640 1 -- 172.21.15.120:0/240774379 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f3e9803a040 con 0x55647415e780 2024-09-19T09:15:44.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.487+0000 7f3ea9862740 10 client.19899 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:44.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.487+0000 7f3ea9862740 10 client.19899 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:15:44.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.487+0000 7f3ea9862740 20 client.19899 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:44.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.487+0000 7f3ea9862740 1 --2- 172.21.15.120:0/240774379 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5564741d9740 0x5564741dbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:44.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.487+0000 7f3ea9862740 4 client.19899 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:15:44.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.487+0000 7f3ea9862740 1 -- 172.21.15.120:0/240774379 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5564740bcd70 con 0x5564741d9740 2024-09-19T09:15:44.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.488+0000 7f3ea5225640 1 --2- 172.21.15.120:0/240774379 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5564741d9740 0x5564741dbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:44.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.488+0000 7f3e84ff9640 10 client.19899 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:15:44.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.488+0000 7f3ea5225640 1 --2- 172.21.15.120:0/240774379 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5564741d9740 0x5564741dbb60 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b96794d1191ac2c8 server_cookie=27a3e1bed1e46886 in_seq=0 out_seq=0 2024-09-19T09:15:44.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.489+0000 7f3e84ff9640 1 -- 172.21.15.120:0/240774379 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5564740bcd70 con 0x5564741d9740 2024-09-19T09:15:44.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.489+0000 7f3e84ff9640 10 client.19899 handle_command_reply: tid=0 2024-09-19T09:15:44.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.489+0000 7f3ea9862740 1 client.19899 shutdown 2024-09-19T09:15:44.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.489+0000 7f3ea9862740 1 -- 172.21.15.120:0/240774379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55647415e780 msgr2=0x556474172000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:44.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.489+0000 7f3ea9862740 1 --2- 172.21.15.120:0/240774379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55647415e780 0x556474172000 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f3e98009900 tx=0x7f3e98009260 comp rx=0 tx=0).stop 2024-09-19T09:15:44.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.489+0000 7f3ea9862740 1 -- 172.21.15.120:0/240774379 shutdown_connections 2024-09-19T09:15:44.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.489+0000 7f3ea9862740 1 --2- 172.21.15.120:0/240774379 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5564741d9740 0x5564741dbb60 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.489+0000 7f3ea9862740 1 --2- 172.21.15.120:0/240774379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55647415e780 0x556474172000 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.489+0000 7f3ea9862740 1 --2- 172.21.15.120:0/240774379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556474150b40 0x556474171ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.489+0000 7f3ea9862740 1 --2- 172.21.15.120:0/240774379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55647414f5d0 0x556474171580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.489+0000 7f3ea9862740 1 -- 172.21.15.120:0/240774379 >> 172.21.15.120:0/240774379 conn(0x556474118310 msgr2=0x556474151110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:44.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.489+0000 7f3e84ff9640 0 client.19899 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:15:44.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.489+0000 7f3ea9862740 1 -- 172.21.15.120:0/240774379 shutdown_connections 2024-09-19T09:15:44.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.489+0000 7f3ea9862740 1 -- 172.21.15.120:0/240774379 wait complete. 2024-09-19T09:15:44.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.489+0000 7f3ea9862740 20 client.19899 trim_cache size 0 max 16384 2024-09-19T09:15:44.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.489+0000 7f3ea7489640 1 -- 172.21.15.120:0/3815124410 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3e880a3240 msgr2=0x7f3e880a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:44.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.489+0000 7f3ea7489640 1 --2- 172.21.15.120:0/3815124410 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3e880a3240 0x7f3e880a5700 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3e98005940 tx=0x7f3e980058d0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.489+0000 7f3ea7489640 1 -- 172.21.15.120:0/3815124410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ea0102570 msgr2=0x7f3ea0199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:44.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.489+0000 7f3ea7489640 1 --2- 172.21.15.120:0/3815124410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ea0102570 0x7f3ea0199910 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400c890 tx=0x7f3e9400cd50 comp rx=0 tx=0).stop 2024-09-19T09:15:44.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.490+0000 7f3ea7489640 1 -- 172.21.15.120:0/3815124410 shutdown_connections 2024-09-19T09:15:44.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.490+0000 7f3ea7489640 1 --2- 172.21.15.120:0/3815124410 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3e880a3240 0x7f3e880a5700 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.490+0000 7f3ea7489640 1 --2- 172.21.15.120:0/3815124410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ea0102ef0 0x7f3ea0199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.490+0000 7f3ea7489640 1 --2- 172.21.15.120:0/3815124410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ea0102570 0x7f3ea0199910 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.490+0000 7f3ea7489640 1 --2- 172.21.15.120:0/3815124410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ea0100e20 0x7f3ea01993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:44.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.490+0000 7f3ea7489640 1 -- 172.21.15.120:0/3815124410 >> 172.21.15.120:0/3815124410 conn(0x7f3ea00fc7d0 msgr2=0x7f3ea01068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:44.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.490+0000 7f3ea7489640 1 -- 172.21.15.120:0/3815124410 shutdown_connections 2024-09-19T09:15:44.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:44.490+0000 7f3ea7489640 1 -- 172.21.15.120:0/3815124410 wait complete. 2024-09-19T09:15:45.078 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1301014 10.03 MB/sec execute 580 sec latency 2015.114 ms 2024-09-19T09:15:45.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:44 smithi196 ceph-mon[35150]: pgmap v730: 97 pgs: 97 active+clean; 42 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.6 MiB/s wr, 405 op/s 2024-09-19T09:15:45.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:44 smithi153 ceph-mon[35071]: pgmap v730: 97 pgs: 97 active+clean; 42 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.6 MiB/s wr, 405 op/s 2024-09-19T09:15:45.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:44 smithi120 ceph-mon[31150]: pgmap v730: 97 pgs: 97 active+clean; 42 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.6 MiB/s wr, 405 op/s 2024-09-19T09:15:45.939 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:15:46.078 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1301014 10.02 MB/sec execute 581 sec latency 3015.178 ms 2024-09-19T09:15:46.188 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:15:46.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.676+0000 7f02fc25f640 1 -- 172.21.15.120:0/2358530416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02f4101de0 msgr2=0x7f02f41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:46.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.676+0000 7f02fc25f640 1 --2- 172.21.15.120:0/2358530416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02f4101de0 0x7f02f41021c0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009900 tx=0x7f02f002a410 comp rx=0 tx=0).stop 2024-09-19T09:15:46.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.676+0000 7f02fc25f640 1 -- 172.21.15.120:0/2358530416 shutdown_connections 2024-09-19T09:15:46.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.676+0000 7f02fc25f640 1 --2- 172.21.15.120:0/2358530416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02f4106f10 0x7f02f410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.676+0000 7f02fc25f640 1 --2- 172.21.15.120:0/2358530416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02f4102790 0x7f02f41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.676+0000 7f02fc25f640 1 --2- 172.21.15.120:0/2358530416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02f4101de0 0x7f02f41021c0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.676+0000 7f02fc25f640 1 -- 172.21.15.120:0/2358530416 >> 172.21.15.120:0/2358530416 conn(0x7f02f40fd880 msgr2=0x7f02f40ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:46.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.676+0000 7f02fc25f640 1 -- 172.21.15.120:0/2358530416 shutdown_connections 2024-09-19T09:15:46.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.676+0000 7f02fc25f640 1 -- 172.21.15.120:0/2358530416 wait complete. 2024-09-19T09:15:46.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.676+0000 7f02fc25f640 1 Processor -- start 2024-09-19T09:15:46.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.677+0000 7f02fc25f640 1 -- start start 2024-09-19T09:15:46.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.677+0000 7f02fc25f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02f4101de0 0x7f02f40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:46.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.677+0000 7f02fc25f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02f4102790 0x7f02f4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:46.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.677+0000 7f02fc25f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02f4106f10 0x7f02f4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:46.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.677+0000 7f02fc25f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f02f4111fb0 con 0x7f02f4102790 2024-09-19T09:15:46.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.677+0000 7f02fc25f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f02f4111e30 con 0x7f02f4106f10 2024-09-19T09:15:46.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.677+0000 7f02fc25f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f02f4112130 con 0x7f02f4101de0 2024-09-19T09:15:46.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.677+0000 7f02f97fa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02f4102790 0x7f02f4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:46.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.677+0000 7f02f97fa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02f4102790 0x7f02f4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46142/0 (socket says 172.21.15.120:46142) 2024-09-19T09:15:46.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.677+0000 7f02f97fa640 1 -- 172.21.15.120:0/1286833396 learned_addr learned my addr 172.21.15.120:0/1286833396 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:46.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.677+0000 7f02f9ffb640 1 --2- 172.21.15.120:0/1286833396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02f4101de0 0x7f02f40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:46.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.677+0000 7f02fa7fc640 1 --2- 172.21.15.120:0/1286833396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02f4106f10 0x7f02f4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:46.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.677+0000 7f02f97fa640 1 -- 172.21.15.120:0/1286833396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02f4101de0 msgr2=0x7f02f40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:46.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.677+0000 7f02f97fa640 1 --2- 172.21.15.120:0/1286833396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02f4101de0 0x7f02f40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.677+0000 7f02f97fa640 1 -- 172.21.15.120:0/1286833396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02f4106f10 msgr2=0x7f02f4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:46.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.677+0000 7f02f97fa640 1 --2- 172.21.15.120:0/1286833396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02f4106f10 0x7f02f4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.677+0000 7f02f97fa640 1 -- 172.21.15.120:0/1286833396 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f41a6960 con 0x7f02f4102790 2024-09-19T09:15:46.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.677+0000 7f02f97fa640 1 --2- 172.21.15.120:0/1286833396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02f4102790 0x7f02f4074a30 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f02e400b4a0 tx=0x7f02e400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:46.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.678+0000 7f02eaffd640 1 -- 172.21.15.120:0/1286833396 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e4014070 con 0x7f02f4102790 2024-09-19T09:15:46.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.678+0000 7f02eaffd640 1 -- 172.21.15.120:0/1286833396 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02e4010af0 con 0x7f02f4102790 2024-09-19T09:15:46.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.678+0000 7f02fc25f640 1 -- 172.21.15.120:0/1286833396 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02f41a6c50 con 0x7f02f4102790 2024-09-19T09:15:46.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.678+0000 7f02eaffd640 1 -- 172.21.15.120:0/1286833396 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e4017d10 con 0x7f02f4102790 2024-09-19T09:15:46.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.678+0000 7f02fc25f640 1 -- 172.21.15.120:0/1286833396 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02f41a7190 con 0x7f02f4102790 2024-09-19T09:15:46.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.680+0000 7f02eaffd640 1 -- 172.21.15.120:0/1286833396 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f02e4017070 con 0x7f02f4102790 2024-09-19T09:15:46.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.680+0000 7f02fe638740 1 Processor -- start 2024-09-19T09:15:46.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.680+0000 7f02fe638740 1 -- start start 2024-09-19T09:15:46.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.681+0000 7f02fe638740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706881cb40 0x55706881b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:46.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.681+0000 7f02fe638740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55706881b5d0 0x55706881ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:46.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.681+0000 7f02fe638740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557068826370 0x557068828760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:46.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.681+0000 7f02fe638740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5570687812f0 con 0x55706881b5d0 2024-09-19T09:15:46.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.681+0000 7f02fe638740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557068728cf0 con 0x557068826370 2024-09-19T09:15:46.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.681+0000 7f02fe638740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55706877d2b0 con 0x55706881cb40 2024-09-19T09:15:46.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.681+0000 7f02f9ffb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55706881b5d0 0x55706881ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:46.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.681+0000 7f02f9ffb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55706881b5d0 0x55706881ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46152/0 (socket says 172.21.15.120:46152) 2024-09-19T09:15:46.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.681+0000 7f02f9ffb640 1 -- 172.21.15.120:0/2483330542 learned_addr learned my addr 172.21.15.120:0/2483330542 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:46.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.681+0000 7f02eaffd640 1 --2- 172.21.15.120:0/1286833396 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f02e00a3240 0x7f02e00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:46.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.681+0000 7f02f97fa640 1 --2- 172.21.15.120:0/2483330542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557068826370 0x557068828760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:46.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.681+0000 7f02fa7fc640 1 --2- 172.21.15.120:0/2483330542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706881cb40 0x55706881b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:46.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.681+0000 7f02f9ffb640 1 -- 172.21.15.120:0/2483330542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706881cb40 msgr2=0x55706881b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:46.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.681+0000 7f02f9ffb640 1 --2- 172.21.15.120:0/2483330542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706881cb40 0x55706881b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.681+0000 7f02f9ffb640 1 -- 172.21.15.120:0/2483330542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557068826370 msgr2=0x557068828760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:46.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.681+0000 7f02f9ffb640 1 --2- 172.21.15.120:0/2483330542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557068826370 0x557068828760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.681+0000 7f02f9ffb640 1 -- 172.21.15.120:0/2483330542 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55706870d3a0 con 0x55706881b5d0 2024-09-19T09:15:46.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.681+0000 7f02fa7fc640 1 --2- 172.21.15.120:0/1286833396 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f02e00a3240 0x7f02e00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:46.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.681+0000 7f02eaffd640 1 -- 172.21.15.120:0/1286833396 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f02e40cebd0 con 0x7f02f4102790 2024-09-19T09:15:46.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.681+0000 7f02f9ffb640 1 --2- 172.21.15.120:0/2483330542 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55706881b5d0 0x55706881ba10 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009900 tx=0x7f02f0031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b6774079c63d2f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:46.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.682+0000 7f02dffff640 1 -- 172.21.15.120:0/2483330542 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f0040070 con 0x55706881b5d0 2024-09-19T09:15:46.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.682+0000 7f02fa7fc640 1 --2- 172.21.15.120:0/1286833396 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f02e00a3240 0x7f02e00a5700 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f02ec005fd0 tx=0x7f02ec004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:46.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.682+0000 7f02dffff640 1 -- 172.21.15.120:0/2483330542 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02f0005020 con 0x55706881b5d0 2024-09-19T09:15:46.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.682+0000 7f02dffff640 1 -- 172.21.15.120:0/2483330542 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f0035040 con 0x55706881b5d0 2024-09-19T09:15:46.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.682+0000 7f02fe638740 1 -- 172.21.15.120:0/2483330542 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55706881b5d0 msgr2=0x55706881ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:46.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.682+0000 7f02fe638740 1 --2- 172.21.15.120:0/2483330542 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55706881b5d0 0x55706881ba10 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009900 tx=0x7f02f0031630 comp rx=0 tx=0).stop 2024-09-19T09:15:46.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.682+0000 7f02fe638740 1 -- 172.21.15.120:0/2483330542 shutdown_connections 2024-09-19T09:15:46.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.682+0000 7f02fe638740 1 --2- 172.21.15.120:0/2483330542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557068826370 0x557068828760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.682+0000 7f02fe638740 1 --2- 172.21.15.120:0/2483330542 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55706881b5d0 0x55706881ba10 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.682+0000 7f02fe638740 1 --2- 172.21.15.120:0/2483330542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706881cb40 0x55706881b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.682+0000 7f02fe638740 1 -- 172.21.15.120:0/2483330542 >> 172.21.15.120:0/2483330542 conn(0x5570687e4310 msgr2=0x557068817c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:46.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.682+0000 7f02fe638740 1 -- 172.21.15.120:0/2483330542 shutdown_connections 2024-09-19T09:15:46.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.682+0000 7f02fe638740 1 -- 172.21.15.120:0/2483330542 wait complete. 2024-09-19T09:15:46.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.683+0000 7f02fe638740 1 -- start start 2024-09-19T09:15:46.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.683+0000 7f02fe638740 1 Processor -- start 2024-09-19T09:15:46.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.683+0000 7f02fe638740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55706881b5d0 0x557068825cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:46.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.683+0000 7f02fe638740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55706881cb40 0x557068822f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:46.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.683+0000 7f02fe638740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557068826370 0x5570688234d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:46.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.683+0000 7f02fe638740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557068829010 con 0x55706881cb40 2024-09-19T09:15:46.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.683+0000 7f02fe638740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55706873c060 con 0x55706881b5d0 2024-09-19T09:15:46.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.683+0000 7f02fe638740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557068829190 con 0x557068826370 2024-09-19T09:15:46.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.683+0000 7f02fe638740 10 client.0 _ll_register_callbacks cb 0x55706848f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:46.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.683+0000 7f02f97fa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55706881cb40 0x557068822f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:46.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.683+0000 7f02f97fa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55706881cb40 0x557068822f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46164/0 (socket says 172.21.15.120:46164) 2024-09-19T09:15:46.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.683+0000 7f02f97fa640 1 -- 172.21.15.120:0/1813127223 learned_addr learned my addr 172.21.15.120:0/1813127223 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:46.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.684+0000 7f02fa7fc640 1 --2- 172.21.15.120:0/1813127223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557068826370 0x5570688234d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:46.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.684+0000 7f02f97fa640 1 -- 172.21.15.120:0/1813127223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557068826370 msgr2=0x5570688234d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:46.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.684+0000 7f02f97fa640 1 --2- 172.21.15.120:0/1813127223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557068826370 0x5570688234d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.684+0000 7f02f97fa640 1 -- 172.21.15.120:0/1813127223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55706881b5d0 msgr2=0x557068825cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:46.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.684+0000 7f02f97fa640 1 --2- 172.21.15.120:0/1813127223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55706881b5d0 0x557068825cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.684+0000 7f02f97fa640 1 -- 172.21.15.120:0/1813127223 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570684f23c0 con 0x55706881cb40 2024-09-19T09:15:46.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.684+0000 7f02fa7fc640 1 --2- 172.21.15.120:0/1813127223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557068826370 0x5570688234d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:15:46.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.684+0000 7f02f97fa640 1 --2- 172.21.15.120:0/1813127223 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55706881cb40 0x557068822f70 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f02e4097700 tx=0x7f02e409b1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b5306fac2aba2c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:46.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.684+0000 7f02dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:15:46.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.684+0000 7f02dd7fa640 1 -- 172.21.15.120:0/1813127223 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e401e030 con 0x55706881cb40 2024-09-19T09:15:46.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.684+0000 7f02dd7fa640 1 -- 172.21.15.120:0/1813127223 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f02e40a5a30 con 0x55706881cb40 2024-09-19T09:15:46.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.684+0000 7f02fe638740 1 -- 172.21.15.120:0/1813127223 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02f0005020 con 0x55706881cb40 2024-09-19T09:15:46.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.684+0000 7f02dd7fa640 1 -- 172.21.15.120:0/1813127223 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f02e4009230 con 0x55706881cb40 2024-09-19T09:15:46.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.684+0000 7f02fe638740 10 client.19911 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:46.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.684+0000 7f02fe638740 1 -- 172.21.15.120:0/1813127223 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557068834a50 con 0x55706881cb40 2024-09-19T09:15:46.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.684+0000 7f02dd7fa640 1 -- 172.21.15.120:0/1813127223 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02f0005020 con 0x55706881cb40 2024-09-19T09:15:46.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.684+0000 7f02dd7fa640 1 -- 172.21.15.120:0/1813127223 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f02e4097910 con 0x55706881cb40 2024-09-19T09:15:46.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.685+0000 7f02fe638740 10 client.19911 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:46.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.685+0000 7f02fe638740 10 client.19911 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:15:46.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.685+0000 7f02fe638740 20 client.19911 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:46.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.685+0000 7f02fe638740 1 --2- 172.21.15.120:0/1813127223 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x557068835e60 0x5570688382d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:46.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.685+0000 7f02fe638740 4 client.19911 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:15:46.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.685+0000 7f02fe638740 1 -- 172.21.15.120:0/1813127223 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55706878e860 con 0x557068835e60 2024-09-19T09:15:46.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.685+0000 7f02f9ffb640 1 --2- 172.21.15.120:0/1813127223 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x557068835e60 0x5570688382d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:46.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.685+0000 7f02f9ffb640 1 --2- 172.21.15.120:0/1813127223 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x557068835e60 0x5570688382d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d983e3d55db98bd2 server_cookie=fd830cd3d7760775 in_seq=0 out_seq=0 2024-09-19T09:15:46.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.685+0000 7f02dd7fa640 10 client.19911 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:15:46.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.686+0000 7f02dd7fa640 1 -- 172.21.15.120:0/1813127223 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55706878e860 con 0x557068835e60 2024-09-19T09:15:46.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.686+0000 7f02dd7fa640 10 client.19911 handle_command_reply: tid=0 2024-09-19T09:15:46.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.686+0000 7f02fe638740 1 client.19911 shutdown 2024-09-19T09:15:46.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.686+0000 7f02fe638740 1 -- 172.21.15.120:0/1813127223 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55706881cb40 msgr2=0x557068822f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:46.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.686+0000 7f02fe638740 1 --2- 172.21.15.120:0/1813127223 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55706881cb40 0x557068822f70 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f02e4097700 tx=0x7f02e409b1a0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.686+0000 7f02fe638740 1 -- 172.21.15.120:0/1813127223 shutdown_connections 2024-09-19T09:15:46.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.686+0000 7f02fe638740 1 --2- 172.21.15.120:0/1813127223 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x557068835e60 0x5570688382d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.686+0000 7f02fe638740 1 --2- 172.21.15.120:0/1813127223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557068826370 0x5570688234d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.686+0000 7f02fe638740 1 --2- 172.21.15.120:0/1813127223 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55706881cb40 0x557068822f70 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.686+0000 7f02dd7fa640 0 client.19911 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:15:46.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.686+0000 7f02fe638740 1 --2- 172.21.15.120:0/1813127223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55706881b5d0 0x557068825cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.686+0000 7f02fe638740 1 -- 172.21.15.120:0/1813127223 >> 172.21.15.120:0/1813127223 conn(0x5570687e4310 msgr2=0x55706881d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:46.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.686+0000 7f02fe638740 1 -- 172.21.15.120:0/1813127223 shutdown_connections 2024-09-19T09:15:46.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.686+0000 7f02fe638740 1 -- 172.21.15.120:0/1813127223 wait complete. 2024-09-19T09:15:46.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.686+0000 7f02fe638740 20 client.19911 trim_cache size 0 max 16384 2024-09-19T09:15:46.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.699+0000 7f02fe638740 1 Processor -- start 2024-09-19T09:15:46.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.699+0000 7f02fe638740 1 -- start start 2024-09-19T09:15:46.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.699+0000 7f02fe638740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55706881b5d0 0x557068824ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:46.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.699+0000 7f02fe638740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706881cb40 0x55706881e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:46.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.699+0000 7f02fe638740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557068826370 0x55706881e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:46.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.699+0000 7f02fe638740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f02f41af7d0 con 0x557068826370 2024-09-19T09:15:46.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.699+0000 7f02fe638740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f02f41b1310 con 0x55706881b5d0 2024-09-19T09:15:46.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.699+0000 7f02fe638740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f02f41afea0 con 0x55706881cb40 2024-09-19T09:15:46.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.699+0000 7f02fa7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557068826370 0x55706881e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:46.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.699+0000 7f02fa7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557068826370 0x55706881e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46172/0 (socket says 172.21.15.120:46172) 2024-09-19T09:15:46.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.699+0000 7f02fa7fc640 1 -- 172.21.15.120:0/2681530403 learned_addr learned my addr 172.21.15.120:0/2681530403 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:46.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.700+0000 7f02f9ffb640 1 --2- 172.21.15.120:0/2681530403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55706881b5d0 0x557068824ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:46.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.700+0000 7f02f97fa640 1 --2- 172.21.15.120:0/2681530403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706881cb40 0x55706881e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:46.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.700+0000 7f02fa7fc640 1 -- 172.21.15.120:0/2681530403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706881cb40 msgr2=0x55706881e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:46.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.700+0000 7f02fa7fc640 1 --2- 172.21.15.120:0/2681530403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706881cb40 0x55706881e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.700+0000 7f02fa7fc640 1 -- 172.21.15.120:0/2681530403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55706881b5d0 msgr2=0x557068824ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:46.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.700+0000 7f02fa7fc640 1 --2- 172.21.15.120:0/2681530403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55706881b5d0 0x557068824ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.700+0000 7f02fa7fc640 1 -- 172.21.15.120:0/2681530403 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55706881eba0 con 0x557068826370 2024-09-19T09:15:46.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.700+0000 7f02f9ffb640 1 --2- 172.21.15.120:0/2681530403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55706881b5d0 0x557068824ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:15:46.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.700+0000 7f02fa7fc640 1 --2- 172.21.15.120:0/2681530403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557068826370 0x55706881e660 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f02ec005ee0 tx=0x7f02ec015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc616b9c4e3e05c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:46.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.700+0000 7f02df7fe640 1 -- 172.21.15.120:0/2681530403 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ec01b620 con 0x557068826370 2024-09-19T09:15:46.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.700+0000 7f02df7fe640 1 -- 172.21.15.120:0/2681530403 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02ec01bca0 con 0x557068826370 2024-09-19T09:15:46.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.700+0000 7f02df7fe640 1 -- 172.21.15.120:0/2681530403 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ec01dc80 con 0x557068826370 2024-09-19T09:15:46.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.701+0000 7f02fe638740 1 -- 172.21.15.120:0/2681530403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557068826370 msgr2=0x55706881e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:46.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.701+0000 7f02fe638740 1 --2- 172.21.15.120:0/2681530403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557068826370 0x55706881e660 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f02ec005ee0 tx=0x7f02ec015000 comp rx=0 tx=0).stop 2024-09-19T09:15:46.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.701+0000 7f02fe638740 1 -- 172.21.15.120:0/2681530403 shutdown_connections 2024-09-19T09:15:46.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.701+0000 7f02fe638740 1 --2- 172.21.15.120:0/2681530403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557068826370 0x55706881e660 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.701+0000 7f02fe638740 1 --2- 172.21.15.120:0/2681530403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706881cb40 0x55706881e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.701+0000 7f02fe638740 1 --2- 172.21.15.120:0/2681530403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55706881b5d0 0x557068824ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.701+0000 7f02fe638740 1 -- 172.21.15.120:0/2681530403 >> 172.21.15.120:0/2681530403 conn(0x5570687e4310 msgr2=0x5570688247d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:46.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.701+0000 7f02fe638740 1 -- 172.21.15.120:0/2681530403 shutdown_connections 2024-09-19T09:15:46.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.701+0000 7f02fe638740 1 -- 172.21.15.120:0/2681530403 wait complete. 2024-09-19T09:15:46.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.701+0000 7f02fe638740 1 -- start start 2024-09-19T09:15:46.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.701+0000 7f02fe638740 1 Processor -- start 2024-09-19T09:15:46.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.701+0000 7f02fe638740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55706881b5d0 0x55706882e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:46.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.701+0000 7f02fe638740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706881cb40 0x55706882e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:46.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.701+0000 7f02fe638740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557068826370 0x55706882b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:46.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.701+0000 7f02fe638740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f02f41b0c40 con 0x557068826370 2024-09-19T09:15:46.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.701+0000 7f02fe638740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f02f41b0570 con 0x55706881b5d0 2024-09-19T09:15:46.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.702+0000 7f02fe638740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f02f41ad790 con 0x55706881cb40 2024-09-19T09:15:46.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.702+0000 7f02fe638740 10 client.0 _ll_register_callbacks cb 0x7f02e4017630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:46.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.702+0000 7f02fa7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557068826370 0x55706882b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:46.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.702+0000 7f02fa7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557068826370 0x55706882b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46186/0 (socket says 172.21.15.120:46186) 2024-09-19T09:15:46.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.702+0000 7f02fa7fc640 1 -- 172.21.15.120:0/3866571082 learned_addr learned my addr 172.21.15.120:0/3866571082 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:46.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.702+0000 7f02f97fa640 1 --2- 172.21.15.120:0/3866571082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706881cb40 0x55706882e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:46.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.702+0000 7f02f9ffb640 1 --2- 172.21.15.120:0/3866571082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55706881b5d0 0x55706882e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:46.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.702+0000 7f02fa7fc640 1 -- 172.21.15.120:0/3866571082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706881cb40 msgr2=0x55706882e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:46.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.702+0000 7f02fa7fc640 1 --2- 172.21.15.120:0/3866571082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706881cb40 0x55706882e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.702+0000 7f02fa7fc640 1 -- 172.21.15.120:0/3866571082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55706881b5d0 msgr2=0x55706882e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:46.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.702+0000 7f02fa7fc640 1 --2- 172.21.15.120:0/3866571082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55706881b5d0 0x55706882e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.702+0000 7f02fa7fc640 1 -- 172.21.15.120:0/3866571082 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55706882b960 con 0x557068826370 2024-09-19T09:15:46.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.702+0000 7f02fa7fc640 1 --2- 172.21.15.120:0/3866571082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557068826370 0x55706882b420 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f02ec0096e0 tx=0x7f02ec01dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b08b05122f8b92 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:46.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.702+0000 7f02dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:15:46.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.702+0000 7f02dd7fa640 1 -- 172.21.15.120:0/3866571082 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ec02e070 con 0x557068826370 2024-09-19T09:15:46.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.702+0000 7f02dd7fa640 1 -- 172.21.15.120:0/3866571082 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f02ec01b620 con 0x557068826370 2024-09-19T09:15:46.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.702+0000 7f02fe638740 1 -- 172.21.15.120:0/3866571082 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02ec01bca0 con 0x557068826370 2024-09-19T09:15:46.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.702+0000 7f02dd7fa640 1 -- 172.21.15.120:0/3866571082 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f02ec03a050 con 0x557068826370 2024-09-19T09:15:46.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.702+0000 7f02fe638740 10 client.19917 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:46.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.702+0000 7f02fe638740 1 -- 172.21.15.120:0/3866571082 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5570688a1300 con 0x557068826370 2024-09-19T09:15:46.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.703+0000 7f02dd7fa640 1 -- 172.21.15.120:0/3866571082 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02ec01bca0 con 0x557068826370 2024-09-19T09:15:46.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.703+0000 7f02dd7fa640 1 -- 172.21.15.120:0/3866571082 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f02ec00f040 con 0x557068826370 2024-09-19T09:15:46.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.703+0000 7f02fe638740 10 client.19917 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:46.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.703+0000 7f02fe638740 10 client.19917 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:15:46.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.703+0000 7f02fe638740 20 client.19917 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:46.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.703+0000 7f02fe638740 1 --2- 172.21.15.120:0/3866571082 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5570688a1960 0x5570688a3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:46.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.703+0000 7f02fe638740 4 client.19917 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:15:46.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.703+0000 7f02fe638740 1 -- 172.21.15.120:0/3866571082 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557068788d70 con 0x5570688a1960 2024-09-19T09:15:46.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.703+0000 7f02f9ffb640 1 --2- 172.21.15.120:0/3866571082 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5570688a1960 0x5570688a3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:46.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.704+0000 7f02dd7fa640 10 client.19917 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:15:46.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.704+0000 7f02f9ffb640 1 --2- 172.21.15.120:0/3866571082 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5570688a1960 0x5570688a3d50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b36f7d5feb6522ff server_cookie=6fc097b0db4d270a in_seq=0 out_seq=0 2024-09-19T09:15:46.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.704+0000 7f02dd7fa640 1 -- 172.21.15.120:0/3866571082 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557068788d70 con 0x5570688a1960 2024-09-19T09:15:46.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.704+0000 7f02dd7fa640 10 client.19917 handle_command_reply: tid=0 2024-09-19T09:15:46.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.704+0000 7f02fe638740 1 client.19917 shutdown 2024-09-19T09:15:46.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.704+0000 7f02fe638740 1 -- 172.21.15.120:0/3866571082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557068826370 msgr2=0x55706882b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:46.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.704+0000 7f02fe638740 1 --2- 172.21.15.120:0/3866571082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557068826370 0x55706882b420 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f02ec0096e0 tx=0x7f02ec01dc40 comp rx=0 tx=0).stop 2024-09-19T09:15:46.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.704+0000 7f02fe638740 1 -- 172.21.15.120:0/3866571082 shutdown_connections 2024-09-19T09:15:46.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.704+0000 7f02fe638740 1 --2- 172.21.15.120:0/3866571082 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5570688a1960 0x5570688a3d50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.704+0000 7f02fe638740 1 --2- 172.21.15.120:0/3866571082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557068826370 0x55706882b420 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.704+0000 7f02fe638740 1 --2- 172.21.15.120:0/3866571082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706881cb40 0x55706882e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.704+0000 7f02dd7fa640 0 client.19917 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:15:46.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.704+0000 7f02fe638740 1 --2- 172.21.15.120:0/3866571082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55706881b5d0 0x55706882e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.704+0000 7f02fe638740 1 -- 172.21.15.120:0/3866571082 >> 172.21.15.120:0/3866571082 conn(0x5570687e4310 msgr2=0x557068824ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:46.736 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:15:46.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.704+0000 7f02fe638740 1 -- 172.21.15.120:0/3866571082 shutdown_connections 2024-09-19T09:15:46.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.704+0000 7f02fe638740 1 -- 172.21.15.120:0/3866571082 wait complete. 2024-09-19T09:15:46.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.704+0000 7f02fe638740 20 client.19917 trim_cache size 0 max 16384 2024-09-19T09:15:46.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.705+0000 7f02fc25f640 1 -- 172.21.15.120:0/1286833396 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f02e00a3240 msgr2=0x7f02e00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:46.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.705+0000 7f02fc25f640 1 --2- 172.21.15.120:0/1286833396 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f02e00a3240 0x7f02e00a5700 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f02ec005fd0 tx=0x7f02ec004a40 comp rx=0 tx=0).stop 2024-09-19T09:15:46.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.705+0000 7f02fc25f640 1 -- 172.21.15.120:0/1286833396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02f4102790 msgr2=0x7f02f4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:46.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.705+0000 7f02fc25f640 1 --2- 172.21.15.120:0/1286833396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02f4102790 0x7f02f4074a30 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f02e400b4a0 tx=0x7f02e400b960 comp rx=0 tx=0).stop 2024-09-19T09:15:46.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.705+0000 7f02fc25f640 1 -- 172.21.15.120:0/1286833396 shutdown_connections 2024-09-19T09:15:46.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.705+0000 7f02fc25f640 1 --2- 172.21.15.120:0/1286833396 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f02e00a3240 0x7f02e00a5700 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.705+0000 7f02fc25f640 1 --2- 172.21.15.120:0/1286833396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02f4106f10 0x7f02f4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.705+0000 7f02fc25f640 1 --2- 172.21.15.120:0/1286833396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02f4102790 0x7f02f4074a30 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.705+0000 7f02fc25f640 1 --2- 172.21.15.120:0/1286833396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02f4101de0 0x7f02f40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:46.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.705+0000 7f02fc25f640 1 -- 172.21.15.120:0/1286833396 >> 172.21.15.120:0/1286833396 conn(0x7f02f40fd880 msgr2=0x7f02f40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:46.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.705+0000 7f02fc25f640 1 -- 172.21.15.120:0/1286833396 shutdown_connections 2024-09-19T09:15:46.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:46.705+0000 7f02fc25f640 1 -- 172.21.15.120:0/1286833396 wait complete. 2024-09-19T09:15:47.078 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1301014 10.00 MB/sec execute 582 sec latency 4015.216 ms 2024-09-19T09:15:47.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:46 smithi196 ceph-mon[35150]: pgmap v731: 97 pgs: 97 active+clean; 46 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.3 MiB/s wr, 380 op/s 2024-09-19T09:15:47.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:46 smithi153 ceph-mon[35071]: pgmap v731: 97 pgs: 97 active+clean; 46 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.3 MiB/s wr, 380 op/s 2024-09-19T09:15:47.317 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:46 smithi120 ceph-mon[31150]: pgmap v731: 97 pgs: 97 active+clean; 46 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.3 MiB/s wr, 380 op/s 2024-09-19T09:15:47.520 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:15:47.765 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:15:48.078 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1301014 9.98 MB/sec execute 583 sec latency 5015.281 ms 2024-09-19T09:15:48.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.293+0000 7f43cd66d640 1 -- 172.21.15.120:0/3366141563 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43c8101420 msgr2=0x7f43c810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:48.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.293+0000 7f43cd66d640 1 --2- 172.21.15.120:0/3366141563 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43c8101420 0x7f43c810d950 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f43c0009cd0 tx=0x7f43c002a8e0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.293+0000 7f43cd66d640 1 -- 172.21.15.120:0/3366141563 shutdown_connections 2024-09-19T09:15:48.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.293+0000 7f43cd66d640 1 --2- 172.21.15.120:0/3366141563 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43c8101420 0x7f43c810d950 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.293+0000 7f43cd66d640 1 --2- 172.21.15.120:0/3366141563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43c8100a80 0x7f43c8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.294+0000 7f43cd66d640 1 --2- 172.21.15.120:0/3366141563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43c8106a80 0x7f43c8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.294+0000 7f43cd66d640 1 -- 172.21.15.120:0/3366141563 >> 172.21.15.120:0/3366141563 conn(0x7f43c80fc7a0 msgr2=0x7f43c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:48.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.294+0000 7f43cd66d640 1 -- 172.21.15.120:0/3366141563 shutdown_connections 2024-09-19T09:15:48.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.294+0000 7f43cd66d640 1 -- 172.21.15.120:0/3366141563 wait complete. 2024-09-19T09:15:48.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.294+0000 7f43cd66d640 1 Processor -- start 2024-09-19T09:15:48.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.294+0000 7f43cd66d640 1 -- start start 2024-09-19T09:15:48.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.295+0000 7f43cd66d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43c8100a80 0x7f43c810e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:48.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.295+0000 7f43cd66d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43c8101420 0x7f43c810e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:48.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.295+0000 7f43cd66d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43c8106a80 0x7f43c810ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:48.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.295+0000 7f43cd66d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f43c8111fa0 con 0x7f43c8100a80 2024-09-19T09:15:48.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.295+0000 7f43cd66d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f43c8111e20 con 0x7f43c8101420 2024-09-19T09:15:48.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.295+0000 7f43cd66d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43c8112120 con 0x7f43c8106a80 2024-09-19T09:15:48.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.295+0000 7f43c6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43c8100a80 0x7f43c810e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:48.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.295+0000 7f43c6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43c8100a80 0x7f43c810e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46204/0 (socket says 172.21.15.120:46204) 2024-09-19T09:15:48.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.295+0000 7f43c6ffd640 1 -- 172.21.15.120:0/857768111 learned_addr learned my addr 172.21.15.120:0/857768111 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:48.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.295+0000 7f43c77fe640 1 --2- 172.21.15.120:0/857768111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43c8106a80 0x7f43c810ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:48.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.295+0000 7f43c67fc640 1 --2- 172.21.15.120:0/857768111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43c8101420 0x7f43c810e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:48.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.295+0000 7f43c6ffd640 1 -- 172.21.15.120:0/857768111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43c8106a80 msgr2=0x7f43c810ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:48.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.295+0000 7f43c6ffd640 1 --2- 172.21.15.120:0/857768111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43c8106a80 0x7f43c810ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.295+0000 7f43c6ffd640 1 -- 172.21.15.120:0/857768111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43c8101420 msgr2=0x7f43c810e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:48.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.295+0000 7f43c6ffd640 1 --2- 172.21.15.120:0/857768111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43c8101420 0x7f43c810e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.295+0000 7f43c6ffd640 1 -- 172.21.15.120:0/857768111 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c81a68f0 con 0x7f43c8100a80 2024-09-19T09:15:48.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.295+0000 7f43c6ffd640 1 --2- 172.21.15.120:0/857768111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43c8100a80 0x7f43c810e3e0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f43b800d770 tx=0x7f43b800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:48.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.295+0000 7f43affff640 1 -- 172.21.15.120:0/857768111 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8017070 con 0x7f43c8100a80 2024-09-19T09:15:48.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.295+0000 7f43affff640 1 -- 172.21.15.120:0/857768111 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43b8005500 con 0x7f43c8100a80 2024-09-19T09:15:48.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.295+0000 7f43cd66d640 1 -- 172.21.15.120:0/857768111 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c81a6be0 con 0x7f43c8100a80 2024-09-19T09:15:48.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.295+0000 7f43affff640 1 -- 172.21.15.120:0/857768111 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8004e00 con 0x7f43c8100a80 2024-09-19T09:15:48.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.295+0000 7f43cd66d640 1 -- 172.21.15.120:0/857768111 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c81a70d0 con 0x7f43c8100a80 2024-09-19T09:15:48.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.298+0000 7f43affff640 1 -- 172.21.15.120:0/857768111 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f43b8004fa0 con 0x7f43c8100a80 2024-09-19T09:15:48.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.299+0000 7f43cfa46740 1 Processor -- start 2024-09-19T09:15:48.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.299+0000 7f43cfa46740 1 -- start start 2024-09-19T09:15:48.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.299+0000 7f43cfa46740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b8b037b40 0x558b8b036090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:48.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.299+0000 7f43cfa46740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b8b0365d0 0x558b8b036a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:48.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.299+0000 7f43cfa46740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b8b041370 0x558b8b043760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:48.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.299+0000 7f43cfa46740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558b8af9c2f0 con 0x558b8b041370 2024-09-19T09:15:48.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.299+0000 7f43cfa46740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558b8af43cf0 con 0x558b8b0365d0 2024-09-19T09:15:48.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.299+0000 7f43cfa46740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558b8af982b0 con 0x558b8b037b40 2024-09-19T09:15:48.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.299+0000 7f43affff640 1 --2- 172.21.15.120:0/857768111 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43b00a3240 0x7f43b00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:48.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.299+0000 7f43c67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b8b041370 0x558b8b043760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:48.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.299+0000 7f43c67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b8b041370 0x558b8b043760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46212/0 (socket says 172.21.15.120:46212) 2024-09-19T09:15:48.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.299+0000 7f43c77fe640 1 --2- 172.21.15.120:0/857768111 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43b00a3240 0x7f43b00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:48.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.299+0000 7f43c67fc640 1 -- 172.21.15.120:0/431294390 learned_addr learned my addr 172.21.15.120:0/431294390 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:48.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.299+0000 7f43affff640 1 -- 172.21.15.120:0/857768111 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f43b80cf2e0 con 0x7f43c8100a80 2024-09-19T09:15:48.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.299+0000 7f43c6ffd640 1 --2- 172.21.15.120:0/431294390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b8b0365d0 0x558b8b036a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:48.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.299+0000 7f43c77fe640 1 --2- 172.21.15.120:0/431294390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b8b037b40 0x558b8b036090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:48.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.299+0000 7f43c67fc640 1 -- 172.21.15.120:0/431294390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b8b037b40 msgr2=0x558b8b036090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:48.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.299+0000 7f43c67fc640 1 --2- 172.21.15.120:0/431294390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b8b037b40 0x558b8b036090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.299+0000 7f43c67fc640 1 -- 172.21.15.120:0/431294390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b8b0365d0 msgr2=0x558b8b036a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:48.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.299+0000 7f43c67fc640 1 --2- 172.21.15.120:0/431294390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b8b0365d0 0x558b8b036a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.299+0000 7f43c67fc640 1 -- 172.21.15.120:0/431294390 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b8af283a0 con 0x558b8b041370 2024-09-19T09:15:48.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.300+0000 7f43c67fc640 1 --2- 172.21.15.120:0/431294390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b8b041370 0x558b8b043760 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00cbc0 tx=0x7f43bc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f723b7ccb764d7e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:48.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.300+0000 7f43c77fe640 1 --2- 172.21.15.120:0/857768111 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43b00a3240 0x7f43b00a5700 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f43c0009430 tx=0x7f43c0009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:48.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.300+0000 7f43ad7fa640 1 -- 172.21.15.120:0/431294390 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc010630 con 0x558b8b041370 2024-09-19T09:15:48.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.300+0000 7f43ad7fa640 1 -- 172.21.15.120:0/431294390 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43bc010cb0 con 0x558b8b041370 2024-09-19T09:15:48.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.300+0000 7f43ad7fa640 1 -- 172.21.15.120:0/431294390 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc01ac50 con 0x558b8b041370 2024-09-19T09:15:48.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.300+0000 7f43cfa46740 1 -- 172.21.15.120:0/431294390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b8b041370 msgr2=0x558b8b043760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:48.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.300+0000 7f43cfa46740 1 --2- 172.21.15.120:0/431294390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b8b041370 0x558b8b043760 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00cbc0 tx=0x7f43bc00e720 comp rx=0 tx=0).stop 2024-09-19T09:15:48.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.300+0000 7f43cfa46740 1 -- 172.21.15.120:0/431294390 shutdown_connections 2024-09-19T09:15:48.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.300+0000 7f43cfa46740 1 --2- 172.21.15.120:0/431294390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b8b041370 0x558b8b043760 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.301+0000 7f43cfa46740 1 --2- 172.21.15.120:0/431294390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b8b0365d0 0x558b8b036a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.301+0000 7f43cfa46740 1 --2- 172.21.15.120:0/431294390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b8b037b40 0x558b8b036090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.301+0000 7f43cfa46740 1 -- 172.21.15.120:0/431294390 >> 172.21.15.120:0/431294390 conn(0x558b8afff310 msgr2=0x558b8b032c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:48.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.301+0000 7f43cfa46740 1 -- 172.21.15.120:0/431294390 shutdown_connections 2024-09-19T09:15:48.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.301+0000 7f43cfa46740 1 -- 172.21.15.120:0/431294390 wait complete. 2024-09-19T09:15:48.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.301+0000 7f43cfa46740 1 -- start start 2024-09-19T09:15:48.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.301+0000 7f43cfa46740 1 Processor -- start 2024-09-19T09:15:48.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.301+0000 7f43cfa46740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b8b0365d0 0x558b8b040ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:48.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.301+0000 7f43cfa46740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b8b037b40 0x558b8b03df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:48.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.301+0000 7f43cfa46740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b8b041370 0x558b8b03e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:48.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.301+0000 7f43cfa46740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558b8b044070 con 0x558b8b037b40 2024-09-19T09:15:48.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.301+0000 7f43cfa46740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558b8af57060 con 0x558b8b0365d0 2024-09-19T09:15:48.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.301+0000 7f43cfa46740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558b8b0441f0 con 0x558b8b041370 2024-09-19T09:15:48.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.301+0000 7f43cfa46740 10 client.0 _ll_register_callbacks cb 0x558b8acaa8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:48.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.301+0000 7f43c77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b8b041370 0x558b8b03e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:48.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.301+0000 7f43c67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b8b037b40 0x558b8b03df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:48.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.302+0000 7f43c77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b8b041370 0x558b8b03e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:50660/0 (socket says 172.21.15.120:50660) 2024-09-19T09:15:48.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.302+0000 7f43c77fe640 1 -- 172.21.15.120:0/1634782659 learned_addr learned my addr 172.21.15.120:0/1634782659 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:48.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.302+0000 7f43c6ffd640 1 --2- 172.21.15.120:0/1634782659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b8b0365d0 0x558b8b040ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:48.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.302+0000 7f43c67fc640 1 -- 172.21.15.120:0/1634782659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b8b041370 msgr2=0x558b8b03e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:48.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.302+0000 7f43c67fc640 1 --2- 172.21.15.120:0/1634782659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b8b041370 0x558b8b03e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.302+0000 7f43c67fc640 1 -- 172.21.15.120:0/1634782659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b8b0365d0 msgr2=0x558b8b040ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:48.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.302+0000 7f43c67fc640 1 --2- 172.21.15.120:0/1634782659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b8b0365d0 0x558b8b040ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.302+0000 7f43c67fc640 1 -- 172.21.15.120:0/1634782659 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b8ad0d3c0 con 0x558b8b037b40 2024-09-19T09:15:48.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.302+0000 7f43c6ffd640 1 --2- 172.21.15.120:0/1634782659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b8b0365d0 0x558b8b040ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:15:48.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.302+0000 7f43c67fc640 1 --2- 172.21.15.120:0/1634782659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b8b037b40 0x558b8b03df90 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00cb90 tx=0x7f43bc00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897ce9bda9b9e1d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:48.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.302+0000 7f43aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:15:48.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.302+0000 7f43aaff5640 1 -- 172.21.15.120:0/1634782659 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc028060 con 0x558b8b037b40 2024-09-19T09:15:48.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.302+0000 7f43aaff5640 1 -- 172.21.15.120:0/1634782659 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f43bc005260 con 0x558b8b037b40 2024-09-19T09:15:48.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.302+0000 7f43cfa46740 1 -- 172.21.15.120:0/1634782659 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43bc010cb0 con 0x558b8b037b40 2024-09-19T09:15:48.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.302+0000 7f43aaff5640 1 -- 172.21.15.120:0/1634782659 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f43bc010e60 con 0x558b8b037b40 2024-09-19T09:15:48.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.302+0000 7f43cfa46740 10 client.19929 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:48.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.302+0000 7f43cfa46740 1 -- 172.21.15.120:0/1634782659 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558b8b04fa80 con 0x558b8b037b40 2024-09-19T09:15:48.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.302+0000 7f43aaff5640 1 -- 172.21.15.120:0/1634782659 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43bc010cb0 con 0x558b8b037b40 2024-09-19T09:15:48.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.303+0000 7f43aaff5640 1 -- 172.21.15.120:0/1634782659 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f43bc02d070 con 0x558b8b037b40 2024-09-19T09:15:48.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.303+0000 7f43cfa46740 10 client.19929 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:48.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.303+0000 7f43cfa46740 10 client.19929 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:15:48.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.303+0000 7f43cfa46740 20 client.19929 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:48.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.303+0000 7f43cfa46740 1 --2- 172.21.15.120:0/1634782659 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x558b8b050dd0 0x558b8b0531c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:48.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.303+0000 7f43cfa46740 4 client.19929 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:15:48.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.303+0000 7f43cfa46740 1 -- 172.21.15.120:0/1634782659 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b8afa9860 con 0x558b8b050dd0 2024-09-19T09:15:48.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.303+0000 7f43c6ffd640 1 --2- 172.21.15.120:0/1634782659 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x558b8b050dd0 0x558b8b0531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:48.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.303+0000 7f43aaff5640 10 client.19929 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:15:48.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.303+0000 7f43c6ffd640 1 --2- 172.21.15.120:0/1634782659 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x558b8b050dd0 0x558b8b0531c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=37137c0b641c799e server_cookie=66e1db83d09f7049 in_seq=0 out_seq=0 2024-09-19T09:15:48.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.304+0000 7f43aaff5640 1 -- 172.21.15.120:0/1634782659 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558b8afa9860 con 0x558b8b050dd0 2024-09-19T09:15:48.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.304+0000 7f43aaff5640 10 client.19929 handle_command_reply: tid=0 2024-09-19T09:15:48.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.304+0000 7f43cfa46740 1 client.19929 shutdown 2024-09-19T09:15:48.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.304+0000 7f43cfa46740 1 -- 172.21.15.120:0/1634782659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b8b037b40 msgr2=0x558b8b03df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:48.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.304+0000 7f43cfa46740 1 --2- 172.21.15.120:0/1634782659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b8b037b40 0x558b8b03df90 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00cb90 tx=0x7f43bc00c900 comp rx=0 tx=0).stop 2024-09-19T09:15:48.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.304+0000 7f43cfa46740 1 -- 172.21.15.120:0/1634782659 shutdown_connections 2024-09-19T09:15:48.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.304+0000 7f43cfa46740 1 --2- 172.21.15.120:0/1634782659 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x558b8b050dd0 0x558b8b0531c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.305+0000 7f43cfa46740 1 --2- 172.21.15.120:0/1634782659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b8b041370 0x558b8b03e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.305+0000 7f43cfa46740 1 --2- 172.21.15.120:0/1634782659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b8b037b40 0x558b8b03df90 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.305+0000 7f43aaff5640 0 client.19929 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:15:48.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.305+0000 7f43cfa46740 1 --2- 172.21.15.120:0/1634782659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b8b0365d0 0x558b8b040ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.305+0000 7f43cfa46740 1 -- 172.21.15.120:0/1634782659 >> 172.21.15.120:0/1634782659 conn(0x558b8afff310 msgr2=0x558b8b038960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:48.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.305+0000 7f43cfa46740 1 -- 172.21.15.120:0/1634782659 shutdown_connections 2024-09-19T09:15:48.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.305+0000 7f43cfa46740 1 -- 172.21.15.120:0/1634782659 wait complete. 2024-09-19T09:15:48.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.305+0000 7f43cfa46740 20 client.19929 trim_cache size 0 max 16384 2024-09-19T09:15:48.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.317+0000 7f43cfa46740 1 Processor -- start 2024-09-19T09:15:48.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.317+0000 7f43cfa46740 1 -- start start 2024-09-19T09:15:48.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.318+0000 7f43cfa46740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b8b0365d0 0x558b8b04d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:48.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.318+0000 7f43cfa46740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b8b037b40 0x558b8b039120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:48.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.318+0000 7f43cfa46740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b8b041370 0x558b8b039660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:48.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.318+0000 7f43cfa46740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f43c81af870 con 0x558b8b0365d0 2024-09-19T09:15:48.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.318+0000 7f43cfa46740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f43c81b13b0 con 0x558b8b037b40 2024-09-19T09:15:48.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.318+0000 7f43cfa46740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43c81aff40 con 0x558b8b041370 2024-09-19T09:15:48.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.318+0000 7f43c6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b8b0365d0 0x558b8b04d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:48.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.318+0000 7f43c6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b8b0365d0 0x558b8b04d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46228/0 (socket says 172.21.15.120:46228) 2024-09-19T09:15:48.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.318+0000 7f43c6ffd640 1 -- 172.21.15.120:0/1037902221 learned_addr learned my addr 172.21.15.120:0/1037902221 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:48.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.318+0000 7f43c77fe640 1 --2- 172.21.15.120:0/1037902221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b8b041370 0x558b8b039660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:48.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.318+0000 7f43c67fc640 1 --2- 172.21.15.120:0/1037902221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b8b037b40 0x558b8b039120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:48.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.318+0000 7f43c6ffd640 1 -- 172.21.15.120:0/1037902221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b8b041370 msgr2=0x558b8b039660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:48.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.318+0000 7f43c6ffd640 1 --2- 172.21.15.120:0/1037902221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b8b041370 0x558b8b039660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.318+0000 7f43c6ffd640 1 -- 172.21.15.120:0/1037902221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b8b037b40 msgr2=0x558b8b039120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:48.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.318+0000 7f43c6ffd640 1 --2- 172.21.15.120:0/1037902221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b8b037b40 0x558b8b039120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.318+0000 7f43c6ffd640 1 -- 172.21.15.120:0/1037902221 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b8b041050 con 0x558b8b0365d0 2024-09-19T09:15:48.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.318+0000 7f43c67fc640 1 --2- 172.21.15.120:0/1037902221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b8b037b40 0x558b8b039120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:15:48.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.318+0000 7f43c6ffd640 1 --2- 172.21.15.120:0/1037902221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b8b0365d0 0x558b8b04d310 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f43b8097d60 tx=0x7f43b8009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7135c4f03d53bb64 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:48.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.318+0000 7f43acff9640 1 -- 172.21.15.120:0/1037902221 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b80b9070 con 0x558b8b0365d0 2024-09-19T09:15:48.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.318+0000 7f43acff9640 1 -- 172.21.15.120:0/1037902221 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43b809fcb0 con 0x558b8b0365d0 2024-09-19T09:15:48.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.318+0000 7f43acff9640 1 -- 172.21.15.120:0/1037902221 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b80b4070 con 0x558b8b0365d0 2024-09-19T09:15:48.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.319+0000 7f43cfa46740 1 -- 172.21.15.120:0/1037902221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b8b0365d0 msgr2=0x558b8b04d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:48.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.319+0000 7f43cfa46740 1 --2- 172.21.15.120:0/1037902221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b8b0365d0 0x558b8b04d310 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f43b8097d60 tx=0x7f43b8009c30 comp rx=0 tx=0).stop 2024-09-19T09:15:48.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.319+0000 7f43cfa46740 1 -- 172.21.15.120:0/1037902221 shutdown_connections 2024-09-19T09:15:48.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.319+0000 7f43cfa46740 1 --2- 172.21.15.120:0/1037902221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b8b041370 0x558b8b039660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.319+0000 7f43cfa46740 1 --2- 172.21.15.120:0/1037902221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b8b037b40 0x558b8b039120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.319+0000 7f43cfa46740 1 --2- 172.21.15.120:0/1037902221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b8b0365d0 0x558b8b04d310 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.319+0000 7f43cfa46740 1 -- 172.21.15.120:0/1037902221 >> 172.21.15.120:0/1037902221 conn(0x558b8afff310 msgr2=0x558b8b040010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:48.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.319+0000 7f43cfa46740 1 -- 172.21.15.120:0/1037902221 shutdown_connections 2024-09-19T09:15:48.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.319+0000 7f43cfa46740 1 -- 172.21.15.120:0/1037902221 wait complete. 2024-09-19T09:15:48.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.319+0000 7f43cfa46740 1 -- start start 2024-09-19T09:15:48.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.319+0000 7f43cfa46740 1 Processor -- start 2024-09-19T09:15:48.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.320+0000 7f43cfa46740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b8b0365d0 0x558b8b03b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:48.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.320+0000 7f43cfa46740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b8b037b40 0x558b8b03b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:48.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.320+0000 7f43cfa46740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b8b041370 0x558b8b03bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:48.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.320+0000 7f43cfa46740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f43c81b0ce0 con 0x558b8b037b40 2024-09-19T09:15:48.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.320+0000 7f43cfa46740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f43c81b0610 con 0x558b8b041370 2024-09-19T09:15:48.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.320+0000 7f43cfa46740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43c81ad830 con 0x558b8b0365d0 2024-09-19T09:15:48.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.320+0000 7f43cfa46740 10 client.0 _ll_register_callbacks cb 0x7f43bc014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:48.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.320+0000 7f43c67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b8b037b40 0x558b8b03b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:48.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.320+0000 7f43c67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b8b037b40 0x558b8b03b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46234/0 (socket says 172.21.15.120:46234) 2024-09-19T09:15:48.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.320+0000 7f43c67fc640 1 -- 172.21.15.120:0/410640948 learned_addr learned my addr 172.21.15.120:0/410640948 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:48.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.320+0000 7f43c6ffd640 1 --2- 172.21.15.120:0/410640948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b8b0365d0 0x558b8b03b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:48.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.320+0000 7f43c77fe640 1 --2- 172.21.15.120:0/410640948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b8b041370 0x558b8b03bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:48.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.320+0000 7f43c67fc640 1 -- 172.21.15.120:0/410640948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b8b0365d0 msgr2=0x558b8b03b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:48.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.320+0000 7f43c67fc640 1 --2- 172.21.15.120:0/410640948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b8b0365d0 0x558b8b03b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.320+0000 7f43c67fc640 1 -- 172.21.15.120:0/410640948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b8b041370 msgr2=0x558b8b03bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:48.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.320+0000 7f43c67fc640 1 --2- 172.21.15.120:0/410640948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b8b041370 0x558b8b03bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.320+0000 7f43c67fc640 1 -- 172.21.15.120:0/410640948 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b8b0ba930 con 0x558b8b037b40 2024-09-19T09:15:48.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.320+0000 7f43c77fe640 1 --2- 172.21.15.120:0/410640948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b8b041370 0x558b8b03bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:15:48.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.320+0000 7f43c67fc640 1 --2- 172.21.15.120:0/410640948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b8b037b40 0x558b8b03b6e0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f43bc001eb0 tx=0x7f43bc0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd45a8bd5b7f9535 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:48.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.320+0000 7f43aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:15:48.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.320+0000 7f43aaff5640 1 -- 172.21.15.120:0/410640948 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc033070 con 0x558b8b037b40 2024-09-19T09:15:48.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.320+0000 7f43aaff5640 1 -- 172.21.15.120:0/410640948 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f43bc005470 con 0x558b8b037b40 2024-09-19T09:15:48.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.320+0000 7f43cfa46740 1 -- 172.21.15.120:0/410640948 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43b809fcb0 con 0x558b8b037b40 2024-09-19T09:15:48.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.320+0000 7f43aaff5640 1 -- 172.21.15.120:0/410640948 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f43bc03e050 con 0x558b8b037b40 2024-09-19T09:15:48.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.320+0000 7f43cfa46740 10 client.19935 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:48.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.320+0000 7f43cfa46740 1 -- 172.21.15.120:0/410640948 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558b8b0bbb80 con 0x558b8b037b40 2024-09-19T09:15:48.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.321+0000 7f43aaff5640 1 -- 172.21.15.120:0/410640948 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43b809fcb0 con 0x558b8b037b40 2024-09-19T09:15:48.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.321+0000 7f43aaff5640 1 -- 172.21.15.120:0/410640948 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f43bc02d040 con 0x558b8b037b40 2024-09-19T09:15:48.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.321+0000 7f43cfa46740 10 client.19935 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:48.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.321+0000 7f43cfa46740 10 client.19935 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:15:48.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.321+0000 7f43cfa46740 20 client.19935 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:48.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.321+0000 7f43cfa46740 1 --2- 172.21.15.120:0/410640948 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x558b8b050dd0 0x558b8b0be2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:48.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.321+0000 7f43cfa46740 4 client.19935 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:15:48.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.321+0000 7f43cfa46740 1 -- 172.21.15.120:0/410640948 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558b8afa3d70 con 0x558b8b050dd0 2024-09-19T09:15:48.348 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:15:48.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.321+0000 7f43c6ffd640 1 --2- 172.21.15.120:0/410640948 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x558b8b050dd0 0x558b8b0be2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:48.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.321+0000 7f43c6ffd640 1 --2- 172.21.15.120:0/410640948 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x558b8b050dd0 0x558b8b0be2b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c6349eecb88c51a9 server_cookie=48ca9afab7e04d2c in_seq=0 out_seq=0 2024-09-19T09:15:48.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.321+0000 7f43aaff5640 10 client.19935 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:15:48.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.322+0000 7f43aaff5640 1 -- 172.21.15.120:0/410640948 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558b8afa3d70 con 0x558b8b050dd0 2024-09-19T09:15:48.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.322+0000 7f43aaff5640 10 client.19935 handle_command_reply: tid=0 2024-09-19T09:15:48.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.322+0000 7f43cfa46740 1 client.19935 shutdown 2024-09-19T09:15:48.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.322+0000 7f43cfa46740 1 -- 172.21.15.120:0/410640948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b8b037b40 msgr2=0x558b8b03b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:48.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.322+0000 7f43cfa46740 1 --2- 172.21.15.120:0/410640948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b8b037b40 0x558b8b03b6e0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f43bc001eb0 tx=0x7f43bc0058a0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.322+0000 7f43cfa46740 1 -- 172.21.15.120:0/410640948 shutdown_connections 2024-09-19T09:15:48.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.322+0000 7f43cfa46740 1 --2- 172.21.15.120:0/410640948 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x558b8b050dd0 0x558b8b0be2b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.322+0000 7f43cfa46740 1 --2- 172.21.15.120:0/410640948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b8b041370 0x558b8b03bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.322+0000 7f43aaff5640 0 client.19935 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:15:48.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.322+0000 7f43cfa46740 1 --2- 172.21.15.120:0/410640948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b8b037b40 0x558b8b03b6e0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.322+0000 7f43cfa46740 1 --2- 172.21.15.120:0/410640948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b8b0365d0 0x558b8b03b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.322+0000 7f43cfa46740 1 -- 172.21.15.120:0/410640948 >> 172.21.15.120:0/410640948 conn(0x558b8afff310 msgr2=0x558b8b03c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:48.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.322+0000 7f43cfa46740 1 -- 172.21.15.120:0/410640948 shutdown_connections 2024-09-19T09:15:48.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.322+0000 7f43cfa46740 1 -- 172.21.15.120:0/410640948 wait complete. 2024-09-19T09:15:48.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.322+0000 7f43cfa46740 20 client.19935 trim_cache size 0 max 16384 2024-09-19T09:15:48.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.323+0000 7f43cd66d640 1 -- 172.21.15.120:0/857768111 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43b00a3240 msgr2=0x7f43b00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:48.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.323+0000 7f43cd66d640 1 --2- 172.21.15.120:0/857768111 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43b00a3240 0x7f43b00a5700 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f43c0009430 tx=0x7f43c0009360 comp rx=0 tx=0).stop 2024-09-19T09:15:48.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.323+0000 7f43cd66d640 1 -- 172.21.15.120:0/857768111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43c8100a80 msgr2=0x7f43c810e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:48.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.323+0000 7f43cd66d640 1 --2- 172.21.15.120:0/857768111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43c8100a80 0x7f43c810e3e0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f43b800d770 tx=0x7f43b800dc30 comp rx=0 tx=0).stop 2024-09-19T09:15:48.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.323+0000 7f43cd66d640 1 -- 172.21.15.120:0/857768111 shutdown_connections 2024-09-19T09:15:48.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.323+0000 7f43cd66d640 1 --2- 172.21.15.120:0/857768111 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43b00a3240 0x7f43b00a5700 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.323+0000 7f43cd66d640 1 --2- 172.21.15.120:0/857768111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43c8106a80 0x7f43c810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.323+0000 7f43cd66d640 1 --2- 172.21.15.120:0/857768111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43c8101420 0x7f43c810e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.323+0000 7f43cd66d640 1 --2- 172.21.15.120:0/857768111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43c8100a80 0x7f43c810e3e0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:48.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.323+0000 7f43cd66d640 1 -- 172.21.15.120:0/857768111 >> 172.21.15.120:0/857768111 conn(0x7f43c80fc7a0 msgr2=0x7f43c80fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:48.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.323+0000 7f43cd66d640 1 -- 172.21.15.120:0/857768111 shutdown_connections 2024-09-19T09:15:48.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:48.323+0000 7f43cd66d640 1 -- 172.21.15.120:0/857768111 wait complete. 2024-09-19T09:15:49.029 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:48 smithi120 ceph-mon[31150]: pgmap v732: 97 pgs: 97 active+clean; 46 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 394 op/s 2024-09-19T09:15:49.056 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:15:49.078 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1301014 9.96 MB/sec execute 584 sec latency 6015.313 ms 2024-09-19T09:15:49.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:48 smithi196 ceph-mon[35150]: pgmap v732: 97 pgs: 97 active+clean; 46 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 394 op/s 2024-09-19T09:15:49.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:48 smithi153 ceph-mon[35071]: pgmap v732: 97 pgs: 97 active+clean; 46 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 394 op/s 2024-09-19T09:15:49.299 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:15:49.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.886+0000 7f098800e640 1 -- 172.21.15.120:0/4065521573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0980074eb0 msgr2=0x7f0980112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:49.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.886+0000 7f098800e640 1 --2- 172.21.15.120:0/4065521573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0980074eb0 0x7f0980112650 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f0970009900 tx=0x7f097002a480 comp rx=0 tx=0).stop 2024-09-19T09:15:49.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.886+0000 7f098800e640 1 -- 172.21.15.120:0/4065521573 shutdown_connections 2024-09-19T09:15:49.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.887+0000 7f098800e640 1 --2- 172.21.15.120:0/4065521573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0980112b90 0x7f0980114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.887+0000 7f098800e640 1 --2- 172.21.15.120:0/4065521573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0980074eb0 0x7f0980112650 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.887+0000 7f098800e640 1 --2- 172.21.15.120:0/4065521573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0980074590 0x7f0980074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.887+0000 7f098800e640 1 -- 172.21.15.120:0/4065521573 >> 172.21.15.120:0/4065521573 conn(0x7f09800fd8c0 msgr2=0x7f09800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.887+0000 7f098800e640 1 -- 172.21.15.120:0/4065521573 shutdown_connections 2024-09-19T09:15:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.887+0000 7f098800e640 1 -- 172.21.15.120:0/4065521573 wait complete. 2024-09-19T09:15:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.887+0000 7f098800e640 1 Processor -- start 2024-09-19T09:15:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.887+0000 7f098800e640 1 -- start start 2024-09-19T09:15:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.887+0000 7f098800e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0980074590 0x7f09801a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.887+0000 7f098800e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0980074eb0 0x7f09801a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.887+0000 7f098800e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0980112b90 0x7f098019e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.887+0000 7f098800e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0980118560 con 0x7f0980074590 2024-09-19T09:15:49.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.887+0000 7f098800e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f09801183e0 con 0x7f0980112b90 2024-09-19T09:15:49.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.887+0000 7f098800e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f09801186e0 con 0x7f0980074eb0 2024-09-19T09:15:49.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.888+0000 7f0985daa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0980074590 0x7f09801a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:49.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.888+0000 7f0985daa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0980074590 0x7f09801a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46248/0 (socket says 172.21.15.120:46248) 2024-09-19T09:15:49.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.888+0000 7f0985daa640 1 -- 172.21.15.120:0/1698031079 learned_addr learned my addr 172.21.15.120:0/1698031079 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:49.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.888+0000 7f09855a9640 1 --2- 172.21.15.120:0/1698031079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0980074eb0 0x7f09801a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:49.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.888+0000 7f09865ab640 1 --2- 172.21.15.120:0/1698031079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0980112b90 0x7f098019e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:49.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.888+0000 7f0985daa640 1 -- 172.21.15.120:0/1698031079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0980074eb0 msgr2=0x7f09801a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:49.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.888+0000 7f0985daa640 1 --2- 172.21.15.120:0/1698031079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0980074eb0 0x7f09801a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.888+0000 7f0985daa640 1 -- 172.21.15.120:0/1698031079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0980112b90 msgr2=0x7f098019e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:49.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.888+0000 7f0985daa640 1 --2- 172.21.15.120:0/1698031079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0980112b90 0x7f098019e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.888+0000 7f0985daa640 1 -- 172.21.15.120:0/1698031079 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f098019efe0 con 0x7f0980074590 2024-09-19T09:15:49.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.888+0000 7f0985daa640 1 --2- 172.21.15.120:0/1698031079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0980074590 0x7f09801a46f0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f097c00d770 tx=0x7f097c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:49.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.888+0000 7f0976ffd640 1 -- 172.21.15.120:0/1698031079 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c017070 con 0x7f0980074590 2024-09-19T09:15:49.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.888+0000 7f0976ffd640 1 -- 172.21.15.120:0/1698031079 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f097c005500 con 0x7f0980074590 2024-09-19T09:15:49.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.888+0000 7f098800e640 1 -- 172.21.15.120:0/1698031079 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f098019f2d0 con 0x7f0980074590 2024-09-19T09:15:49.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.888+0000 7f0976ffd640 1 -- 172.21.15.120:0/1698031079 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c004e00 con 0x7f0980074590 2024-09-19T09:15:49.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.888+0000 7f098800e640 1 -- 172.21.15.120:0/1698031079 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09801ab5c0 con 0x7f0980074590 2024-09-19T09:15:49.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.891+0000 7f0976ffd640 1 -- 172.21.15.120:0/1698031079 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f097c004fa0 con 0x7f0980074590 2024-09-19T09:15:49.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.891+0000 7f098a3e7740 1 Processor -- start 2024-09-19T09:15:49.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.891+0000 7f098a3e7740 1 -- start start 2024-09-19T09:15:49.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.891+0000 7f098a3e7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56380a42eb40 0x56380a42d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:49.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.891+0000 7f098a3e7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56380a42d5d0 0x56380a42da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:49.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.891+0000 7f098a3e7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56380a438370 0x56380a43a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:49.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.891+0000 7f098a3e7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56380a3932f0 con 0x56380a42d5d0 2024-09-19T09:15:49.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.891+0000 7f098a3e7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56380a33acf0 con 0x56380a438370 2024-09-19T09:15:49.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.891+0000 7f098a3e7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56380a38f2b0 con 0x56380a42eb40 2024-09-19T09:15:49.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.891+0000 7f0985daa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56380a42d5d0 0x56380a42da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:49.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.891+0000 7f0985daa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56380a42d5d0 0x56380a42da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46260/0 (socket says 172.21.15.120:46260) 2024-09-19T09:15:49.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.891+0000 7f0985daa640 1 -- 172.21.15.120:0/1011064387 learned_addr learned my addr 172.21.15.120:0/1011064387 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:49.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.892+0000 7f09865ab640 1 --2- 172.21.15.120:0/1011064387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56380a42eb40 0x56380a42d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:49.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.892+0000 7f09855a9640 1 --2- 172.21.15.120:0/1011064387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56380a438370 0x56380a43a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:49.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.892+0000 7f0976ffd640 1 --2- 172.21.15.120:0/1698031079 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f096c0a3240 0x7f096c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:49.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.892+0000 7f0985daa640 1 -- 172.21.15.120:0/1011064387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56380a42eb40 msgr2=0x56380a42d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:49.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.892+0000 7f0985daa640 1 --2- 172.21.15.120:0/1011064387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56380a42eb40 0x56380a42d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.892+0000 7f0985daa640 1 -- 172.21.15.120:0/1011064387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56380a438370 msgr2=0x56380a43a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:49.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.892+0000 7f0985daa640 1 --2- 172.21.15.120:0/1011064387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56380a438370 0x56380a43a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.892+0000 7f0985daa640 1 -- 172.21.15.120:0/1011064387 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56380a31f3a0 con 0x56380a42d5d0 2024-09-19T09:15:49.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.892+0000 7f09855a9640 1 --2- 172.21.15.120:0/1011064387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56380a438370 0x56380a43a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:15:49.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.892+0000 7f09865ab640 1 --2- 172.21.15.120:0/1011064387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56380a42eb40 0x56380a42d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:15:49.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.892+0000 7f09865ab640 1 --2- 172.21.15.120:0/1698031079 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f096c0a3240 0x7f096c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:49.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.892+0000 7f0976ffd640 1 -- 172.21.15.120:0/1698031079 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f097c0cf2e0 con 0x7f0980074590 2024-09-19T09:15:49.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.892+0000 7f0985daa640 1 --2- 172.21.15.120:0/1011064387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56380a42d5d0 0x56380a42da10 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f097c0b1cc0 tx=0x7f097c0afa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ccbdfd086b2f5f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:49.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.892+0000 7f096bfff640 1 -- 172.21.15.120:0/1011064387 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c020070 con 0x56380a42d5d0 2024-09-19T09:15:49.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.892+0000 7f096bfff640 1 -- 172.21.15.120:0/1011064387 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f097c0b1cf0 con 0x56380a42d5d0 2024-09-19T09:15:49.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.892+0000 7f096bfff640 1 -- 172.21.15.120:0/1011064387 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c09b310 con 0x56380a42d5d0 2024-09-19T09:15:49.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.893+0000 7f098a3e7740 1 -- 172.21.15.120:0/1011064387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56380a42d5d0 msgr2=0x56380a42da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:49.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.893+0000 7f09865ab640 1 --2- 172.21.15.120:0/1698031079 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f096c0a3240 0x7f096c0a5700 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0978007330 tx=0x7f0978004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:49.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.893+0000 7f098a3e7740 1 --2- 172.21.15.120:0/1011064387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56380a42d5d0 0x56380a42da10 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f097c0b1cc0 tx=0x7f097c0afa60 comp rx=0 tx=0).stop 2024-09-19T09:15:49.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.893+0000 7f098a3e7740 1 -- 172.21.15.120:0/1011064387 shutdown_connections 2024-09-19T09:15:49.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.893+0000 7f098a3e7740 1 --2- 172.21.15.120:0/1011064387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56380a438370 0x56380a43a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.893+0000 7f098a3e7740 1 --2- 172.21.15.120:0/1011064387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56380a42d5d0 0x56380a42da10 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.893+0000 7f098a3e7740 1 --2- 172.21.15.120:0/1011064387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56380a42eb40 0x56380a42d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.893+0000 7f098a3e7740 1 -- 172.21.15.120:0/1011064387 >> 172.21.15.120:0/1011064387 conn(0x56380a3f6310 msgr2=0x56380a429c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:49.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.893+0000 7f098a3e7740 1 -- 172.21.15.120:0/1011064387 shutdown_connections 2024-09-19T09:15:49.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.893+0000 7f098a3e7740 1 -- 172.21.15.120:0/1011064387 wait complete. 2024-09-19T09:15:49.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.893+0000 7f098a3e7740 1 -- start start 2024-09-19T09:15:49.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.894+0000 7f098a3e7740 1 Processor -- start 2024-09-19T09:15:49.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.894+0000 7f098a3e7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56380a42d5d0 0x56380a437d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:49.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.894+0000 7f098a3e7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56380a42eb40 0x56380a435040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:49.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.894+0000 7f098a3e7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56380a438370 0x56380a435650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:49.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.894+0000 7f098a3e7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56380a43b010 con 0x56380a42eb40 2024-09-19T09:15:49.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.894+0000 7f098a3e7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56380a34e060 con 0x56380a438370 2024-09-19T09:15:49.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.894+0000 7f098a3e7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56380a43b190 con 0x56380a42d5d0 2024-09-19T09:15:49.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.894+0000 7f098a3e7740 10 client.0 _ll_register_callbacks cb 0x56380a0a18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:49.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.894+0000 7f09855a9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56380a42eb40 0x56380a435040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:49.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.894+0000 7f09855a9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56380a42eb40 0x56380a435040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46276/0 (socket says 172.21.15.120:46276) 2024-09-19T09:15:49.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.894+0000 7f09855a9640 1 -- 172.21.15.120:0/1357731080 learned_addr learned my addr 172.21.15.120:0/1357731080 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:49.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.894+0000 7f0985daa640 1 --2- 172.21.15.120:0/1357731080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56380a42d5d0 0x56380a437d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:49.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.894+0000 7f09865ab640 1 --2- 172.21.15.120:0/1357731080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56380a438370 0x56380a435650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:49.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.894+0000 7f09855a9640 1 -- 172.21.15.120:0/1357731080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56380a42d5d0 msgr2=0x56380a437d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:49.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.894+0000 7f09855a9640 1 --2- 172.21.15.120:0/1357731080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56380a42d5d0 0x56380a437d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.894+0000 7f09855a9640 1 -- 172.21.15.120:0/1357731080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56380a438370 msgr2=0x56380a435650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:49.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.894+0000 7f09855a9640 1 --2- 172.21.15.120:0/1357731080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56380a438370 0x56380a435650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.894+0000 7f09855a9640 1 -- 172.21.15.120:0/1357731080 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56380a1043c0 con 0x56380a42eb40 2024-09-19T09:15:49.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.894+0000 7f09865ab640 1 --2- 172.21.15.120:0/1357731080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56380a438370 0x56380a435650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:15:49.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.895+0000 7f09855a9640 1 --2- 172.21.15.120:0/1357731080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56380a42eb40 0x56380a435040 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f0970009900 tx=0x7f09700319b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb8b2b770d840b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:49.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.895+0000 7f09697fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:15:49.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.895+0000 7f09697fa640 1 -- 172.21.15.120:0/1357731080 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0970040070 con 0x56380a42eb40 2024-09-19T09:15:49.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.895+0000 7f09697fa640 1 -- 172.21.15.120:0/1357731080 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f0970004450 con 0x56380a42eb40 2024-09-19T09:15:49.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.895+0000 7f098a3e7740 1 -- 172.21.15.120:0/1357731080 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56380a3a0860 con 0x56380a42eb40 2024-09-19T09:15:49.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.895+0000 7f09697fa640 1 -- 172.21.15.120:0/1357731080 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0970035040 con 0x56380a42eb40 2024-09-19T09:15:49.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.895+0000 7f098a3e7740 10 client.19947 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:49.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.895+0000 7f098a3e7740 1 -- 172.21.15.120:0/1357731080 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56380a446a50 con 0x56380a42eb40 2024-09-19T09:15:49.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.895+0000 7f09697fa640 1 -- 172.21.15.120:0/1357731080 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56380a3a0860 con 0x56380a42eb40 2024-09-19T09:15:49.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.895+0000 7f09697fa640 1 -- 172.21.15.120:0/1357731080 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f09700035f0 con 0x56380a42eb40 2024-09-19T09:15:49.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.895+0000 7f098a3e7740 10 client.19947 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:49.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.895+0000 7f098a3e7740 10 client.19947 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:15:49.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.895+0000 7f098a3e7740 20 client.19947 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:49.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.895+0000 7f098a3e7740 1 --2- 172.21.15.120:0/1357731080 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x56380a447f10 0x56380a44a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:49.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.895+0000 7f098a3e7740 4 client.19947 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:15:49.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.895+0000 7f098a3e7740 1 -- 172.21.15.120:0/1357731080 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56380a39ad70 con 0x56380a447f10 2024-09-19T09:15:49.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.896+0000 7f0985daa640 1 --2- 172.21.15.120:0/1357731080 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x56380a447f10 0x56380a44a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:49.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.896+0000 7f09697fa640 10 client.19947 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:15:49.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.896+0000 7f0985daa640 1 --2- 172.21.15.120:0/1357731080 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x56380a447f10 0x56380a44a350 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cb3eb6075749d5a3 server_cookie=ca2e0095fb9c4d3c in_seq=0 out_seq=0 2024-09-19T09:15:49.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.897+0000 7f09697fa640 1 -- 172.21.15.120:0/1357731080 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56380a39ad70 con 0x56380a447f10 2024-09-19T09:15:49.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.897+0000 7f09697fa640 10 client.19947 handle_command_reply: tid=0 2024-09-19T09:15:49.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.897+0000 7f098a3e7740 1 client.19947 shutdown 2024-09-19T09:15:49.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.897+0000 7f098a3e7740 1 -- 172.21.15.120:0/1357731080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56380a42eb40 msgr2=0x56380a435040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:49.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.897+0000 7f098a3e7740 1 --2- 172.21.15.120:0/1357731080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56380a42eb40 0x56380a435040 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f0970009900 tx=0x7f09700319b0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.897+0000 7f098a3e7740 1 -- 172.21.15.120:0/1357731080 shutdown_connections 2024-09-19T09:15:49.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.897+0000 7f098a3e7740 1 --2- 172.21.15.120:0/1357731080 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x56380a447f10 0x56380a44a350 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.897+0000 7f098a3e7740 1 --2- 172.21.15.120:0/1357731080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56380a438370 0x56380a435650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.897+0000 7f098a3e7740 1 --2- 172.21.15.120:0/1357731080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56380a42eb40 0x56380a435040 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.897+0000 7f098a3e7740 1 --2- 172.21.15.120:0/1357731080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56380a42d5d0 0x56380a437d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.897+0000 7f09697fa640 0 client.19947 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:15:49.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.897+0000 7f098a3e7740 1 -- 172.21.15.120:0/1357731080 >> 172.21.15.120:0/1357731080 conn(0x56380a3f6310 msgr2=0x56380a42f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:49.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.897+0000 7f098a3e7740 1 -- 172.21.15.120:0/1357731080 shutdown_connections 2024-09-19T09:15:49.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.897+0000 7f098a3e7740 1 -- 172.21.15.120:0/1357731080 wait complete. 2024-09-19T09:15:49.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.897+0000 7f098a3e7740 20 client.19947 trim_cache size 0 max 16384 2024-09-19T09:15:49.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.910+0000 7f098a3e7740 1 Processor -- start 2024-09-19T09:15:49.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.910+0000 7f098a3e7740 1 -- start start 2024-09-19T09:15:49.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.910+0000 7f098a3e7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56380a42d5d0 0x56380a429d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:49.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.910+0000 7f098a3e7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56380a42eb40 0x56380a4442b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:49.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.910+0000 7f098a3e7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56380a438370 0x56380a434020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:49.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.910+0000 7f098a3e7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f09801b3c50 con 0x56380a42eb40 2024-09-19T09:15:49.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.910+0000 7f098a3e7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f09801b5790 con 0x56380a438370 2024-09-19T09:15:49.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.910+0000 7f098a3e7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f09801b4320 con 0x56380a42d5d0 2024-09-19T09:15:49.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.910+0000 7f09855a9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56380a42eb40 0x56380a4442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:49.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.910+0000 7f09855a9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56380a42eb40 0x56380a4442b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46290/0 (socket says 172.21.15.120:46290) 2024-09-19T09:15:49.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.910+0000 7f09855a9640 1 -- 172.21.15.120:0/1816667861 learned_addr learned my addr 172.21.15.120:0/1816667861 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:49.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.910+0000 7f09865ab640 1 --2- 172.21.15.120:0/1816667861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56380a438370 0x56380a434020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:49.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.910+0000 7f0985daa640 1 --2- 172.21.15.120:0/1816667861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56380a42d5d0 0x56380a429d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:49.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.910+0000 7f09855a9640 1 -- 172.21.15.120:0/1816667861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56380a42d5d0 msgr2=0x56380a429d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:49.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.910+0000 7f09855a9640 1 --2- 172.21.15.120:0/1816667861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56380a42d5d0 0x56380a429d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.910+0000 7f09855a9640 1 -- 172.21.15.120:0/1816667861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56380a438370 msgr2=0x56380a434020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:49.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.910+0000 7f09855a9640 1 --2- 172.21.15.120:0/1816667861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56380a438370 0x56380a434020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.910+0000 7f09855a9640 1 -- 172.21.15.120:0/1816667861 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56380a428310 con 0x56380a42eb40 2024-09-19T09:15:49.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.910+0000 7f0985daa640 1 --2- 172.21.15.120:0/1816667861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56380a42d5d0 0x56380a429d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:15:49.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.910+0000 7f09855a9640 1 --2- 172.21.15.120:0/1816667861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56380a42eb40 0x56380a4442b0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f097002a9a0 tx=0x7f097002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209a895400f25306 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:49.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.911+0000 7f096b7fe640 1 -- 172.21.15.120:0/1816667861 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0970053070 con 0x56380a42eb40 2024-09-19T09:15:49.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.911+0000 7f096b7fe640 1 -- 172.21.15.120:0/1816667861 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0970004c30 con 0x56380a42eb40 2024-09-19T09:15:49.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.911+0000 7f096b7fe640 1 -- 172.21.15.120:0/1816667861 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0970040070 con 0x56380a42eb40 2024-09-19T09:15:49.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.911+0000 7f098a3e7740 1 -- 172.21.15.120:0/1816667861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56380a42eb40 msgr2=0x56380a4442b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:49.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.911+0000 7f098a3e7740 1 --2- 172.21.15.120:0/1816667861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56380a42eb40 0x56380a4442b0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f097002a9a0 tx=0x7f097002a480 comp rx=0 tx=0).stop 2024-09-19T09:15:49.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.911+0000 7f098a3e7740 1 -- 172.21.15.120:0/1816667861 shutdown_connections 2024-09-19T09:15:49.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.911+0000 7f098a3e7740 1 --2- 172.21.15.120:0/1816667861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56380a438370 0x56380a434020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.911+0000 7f098a3e7740 1 --2- 172.21.15.120:0/1816667861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56380a42eb40 0x56380a4442b0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.911+0000 7f098a3e7740 1 --2- 172.21.15.120:0/1816667861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56380a42d5d0 0x56380a429d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.911+0000 7f098a3e7740 1 -- 172.21.15.120:0/1816667861 >> 172.21.15.120:0/1816667861 conn(0x56380a3f6310 msgr2=0x56380a433910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:49.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.911+0000 7f098a3e7740 1 -- 172.21.15.120:0/1816667861 shutdown_connections 2024-09-19T09:15:49.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.911+0000 7f098a3e7740 1 -- 172.21.15.120:0/1816667861 wait complete. 2024-09-19T09:15:49.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.912+0000 7f098a3e7740 1 -- start start 2024-09-19T09:15:49.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.912+0000 7f098a3e7740 1 Processor -- start 2024-09-19T09:15:49.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.912+0000 7f098a3e7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56380a42d5d0 0x56380a440790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:49.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.912+0000 7f098a3e7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56380a42eb40 0x56380a440cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:49.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.912+0000 7f098a3e7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56380a438370 0x56380a431c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:49.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.912+0000 7f098a3e7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f09801b50c0 con 0x56380a438370 2024-09-19T09:15:49.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.912+0000 7f098a3e7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f09801b49f0 con 0x56380a42eb40 2024-09-19T09:15:49.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.912+0000 7f098a3e7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f09801b1c10 con 0x56380a42d5d0 2024-09-19T09:15:49.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.912+0000 7f098a3e7740 10 client.0 _ll_register_callbacks cb 0x7f0970009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:49.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.912+0000 7f09865ab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56380a438370 0x56380a431c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:49.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.912+0000 7f09865ab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56380a438370 0x56380a431c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46302/0 (socket says 172.21.15.120:46302) 2024-09-19T09:15:49.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.912+0000 7f09865ab640 1 -- 172.21.15.120:0/259004339 learned_addr learned my addr 172.21.15.120:0/259004339 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:49.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.912+0000 7f0985daa640 1 --2- 172.21.15.120:0/259004339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56380a42d5d0 0x56380a440790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:49.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.912+0000 7f09855a9640 1 --2- 172.21.15.120:0/259004339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56380a42eb40 0x56380a440cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:49.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.913+0000 7f09865ab640 1 -- 172.21.15.120:0/259004339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56380a42d5d0 msgr2=0x56380a440790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:49.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.913+0000 7f09865ab640 1 --2- 172.21.15.120:0/259004339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56380a42d5d0 0x56380a440790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.913+0000 7f09865ab640 1 -- 172.21.15.120:0/259004339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56380a42eb40 msgr2=0x56380a440cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:49.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.913+0000 7f09865ab640 1 --2- 172.21.15.120:0/259004339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56380a42eb40 0x56380a440cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.913+0000 7f09865ab640 1 -- 172.21.15.120:0/259004339 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56380a429510 con 0x56380a438370 2024-09-19T09:15:49.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.913+0000 7f0985daa640 1 --2- 172.21.15.120:0/259004339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56380a42d5d0 0x56380a440790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:15:49.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.913+0000 7f09865ab640 1 --2- 172.21.15.120:0/259004339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56380a438370 0x56380a431c20 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f097801ef50 tx=0x7f0978014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff31f0e252edbe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:49.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.913+0000 7f09697fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:15:49.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.915+0000 7f09697fa640 1 -- 172.21.15.120:0/259004339 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0978004d20 con 0x56380a438370 2024-09-19T09:15:49.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.915+0000 7f09697fa640 1 -- 172.21.15.120:0/259004339 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f097801b620 con 0x56380a438370 2024-09-19T09:15:49.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.915+0000 7f098a3e7740 1 -- 172.21.15.120:0/259004339 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0970004c30 con 0x56380a438370 2024-09-19T09:15:49.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.915+0000 7f09697fa640 1 -- 172.21.15.120:0/259004339 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f097801b820 con 0x56380a438370 2024-09-19T09:15:49.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.915+0000 7f098a3e7740 10 client.19953 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:49.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.915+0000 7f098a3e7740 1 -- 172.21.15.120:0/259004339 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56380a4b26c0 con 0x56380a438370 2024-09-19T09:15:49.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.915+0000 7f09697fa640 1 -- 172.21.15.120:0/259004339 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0970004c30 con 0x56380a438370 2024-09-19T09:15:49.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.915+0000 7f09697fa640 1 -- 172.21.15.120:0/259004339 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f0978031020 con 0x56380a438370 2024-09-19T09:15:49.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.915+0000 7f098a3e7740 10 client.19953 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:49.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.915+0000 7f098a3e7740 10 client.19953 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:15:49.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.915+0000 7f098a3e7740 20 client.19953 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:49.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.915+0000 7f098a3e7740 1 --2- 172.21.15.120:0/259004339 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x56380a4b2de0 0x56380a4b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:49.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.916+0000 7f098a3e7740 4 client.19953 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:15:49.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.916+0000 7f098a3e7740 1 -- 172.21.15.120:0/259004339 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56380a317170 con 0x56380a4b2de0 2024-09-19T09:15:49.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.916+0000 7f0985daa640 1 --2- 172.21.15.120:0/259004339 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x56380a4b2de0 0x56380a4b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:49.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.916+0000 7f0985daa640 1 --2- 172.21.15.120:0/259004339 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x56380a4b2de0 0x56380a4b5200 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bed078e65623b70c server_cookie=70f8469eb3e94170 in_seq=0 out_seq=0 2024-09-19T09:15:49.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.916+0000 7f09697fa640 10 client.19953 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:15:49.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.916+0000 7f09697fa640 1 -- 172.21.15.120:0/259004339 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56380a317170 con 0x56380a4b2de0 2024-09-19T09:15:49.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.916+0000 7f09697fa640 10 client.19953 handle_command_reply: tid=0 2024-09-19T09:15:49.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.916+0000 7f098a3e7740 1 client.19953 shutdown 2024-09-19T09:15:49.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.916+0000 7f098a3e7740 1 -- 172.21.15.120:0/259004339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56380a438370 msgr2=0x56380a431c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:49.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.916+0000 7f098a3e7740 1 --2- 172.21.15.120:0/259004339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56380a438370 0x56380a431c20 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f097801ef50 tx=0x7f0978014070 comp rx=0 tx=0).stop 2024-09-19T09:15:49.948 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:15:49.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.916+0000 7f098a3e7740 1 -- 172.21.15.120:0/259004339 shutdown_connections 2024-09-19T09:15:49.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.916+0000 7f098a3e7740 1 --2- 172.21.15.120:0/259004339 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x56380a4b2de0 0x56380a4b5200 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.916+0000 7f098a3e7740 1 --2- 172.21.15.120:0/259004339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56380a438370 0x56380a431c20 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.916+0000 7f09697fa640 0 client.19953 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:15:49.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.916+0000 7f098a3e7740 1 --2- 172.21.15.120:0/259004339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56380a42eb40 0x56380a440cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.916+0000 7f098a3e7740 1 --2- 172.21.15.120:0/259004339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56380a42d5d0 0x56380a440790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.916+0000 7f098a3e7740 1 -- 172.21.15.120:0/259004339 >> 172.21.15.120:0/259004339 conn(0x56380a3f6310 msgr2=0x56380a4284e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:49.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.916+0000 7f098a3e7740 1 -- 172.21.15.120:0/259004339 shutdown_connections 2024-09-19T09:15:49.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.916+0000 7f098a3e7740 1 -- 172.21.15.120:0/259004339 wait complete. 2024-09-19T09:15:49.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.917+0000 7f098a3e7740 20 client.19953 trim_cache size 0 max 16384 2024-09-19T09:15:49.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.917+0000 7f098800e640 1 -- 172.21.15.120:0/1698031079 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f096c0a3240 msgr2=0x7f096c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:49.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.917+0000 7f098800e640 1 --2- 172.21.15.120:0/1698031079 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f096c0a3240 0x7f096c0a5700 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0978007330 tx=0x7f0978004a40 comp rx=0 tx=0).stop 2024-09-19T09:15:49.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.917+0000 7f098800e640 1 -- 172.21.15.120:0/1698031079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0980074590 msgr2=0x7f09801a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:49.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.917+0000 7f098800e640 1 --2- 172.21.15.120:0/1698031079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0980074590 0x7f09801a46f0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f097c00d770 tx=0x7f097c00dc30 comp rx=0 tx=0).stop 2024-09-19T09:15:49.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.917+0000 7f098800e640 1 -- 172.21.15.120:0/1698031079 shutdown_connections 2024-09-19T09:15:49.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.917+0000 7f098800e640 1 --2- 172.21.15.120:0/1698031079 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f096c0a3240 0x7f096c0a5700 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.917+0000 7f098800e640 1 --2- 172.21.15.120:0/1698031079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0980112b90 0x7f098019e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.917+0000 7f098800e640 1 --2- 172.21.15.120:0/1698031079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0980074eb0 0x7f09801a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.917+0000 7f098800e640 1 --2- 172.21.15.120:0/1698031079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0980074590 0x7f09801a46f0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:49.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.917+0000 7f098800e640 1 -- 172.21.15.120:0/1698031079 >> 172.21.15.120:0/1698031079 conn(0x7f09800fd8c0 msgr2=0x7f09800ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:49.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.917+0000 7f098800e640 1 -- 172.21.15.120:0/1698031079 shutdown_connections 2024-09-19T09:15:49.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:49.917+0000 7f098800e640 1 -- 172.21.15.120:0/1698031079 wait complete. 2024-09-19T09:15:50.078 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1301014 9.95 MB/sec execute 585 sec latency 7015.368 ms 2024-09-19T09:15:50.611 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:15:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.808+0000 7eff6196f640 1 -- 172.21.15.120:0/2088888764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff5c104ef0 msgr2=0x7eff5c1072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.808+0000 7eff6196f640 1 --2- 172.21.15.120:0/2088888764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff5c104ef0 0x7eff5c1072e0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7eff50009900 tx=0x7eff5002a410 comp rx=0 tx=0).stop 2024-09-19T09:15:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.808+0000 7eff6196f640 1 -- 172.21.15.120:0/2088888764 shutdown_connections 2024-09-19T09:15:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.808+0000 7eff6196f640 1 --2- 172.21.15.120:0/2088888764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff5c107820 0x7eff5c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.808+0000 7eff6196f640 1 --2- 172.21.15.120:0/2088888764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff5c104ef0 0x7eff5c1072e0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.808+0000 7eff6196f640 1 --2- 172.21.15.120:0/2088888764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff5c1025c0 0x7eff5c1049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.808+0000 7eff6196f640 1 -- 172.21.15.120:0/2088888764 >> 172.21.15.120:0/2088888764 conn(0x7eff5c0fc1e0 msgr2=0x7eff5c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.808+0000 7eff6196f640 1 -- 172.21.15.120:0/2088888764 shutdown_connections 2024-09-19T09:15:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.808+0000 7eff6196f640 1 -- 172.21.15.120:0/2088888764 wait complete. 2024-09-19T09:15:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.809+0000 7eff6196f640 1 Processor -- start 2024-09-19T09:15:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.809+0000 7eff6196f640 1 -- start start 2024-09-19T09:15:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.809+0000 7eff6196f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff5c1025c0 0x7eff5c197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.809+0000 7eff6196f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff5c104ef0 0x7eff5c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.809+0000 7eff6196f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff5c107820 0x7eff5c19f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.809+0000 7eff6196f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c10d840 con 0x7eff5c1025c0 2024-09-19T09:15:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.809+0000 7eff6196f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c10d6c0 con 0x7eff5c107820 2024-09-19T09:15:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.809+0000 7eff6196f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c10d9c0 con 0x7eff5c104ef0 2024-09-19T09:15:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.809+0000 7eff5affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff5c1025c0 0x7eff5c197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.809+0000 7eff5affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff5c1025c0 0x7eff5c197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46322/0 (socket says 172.21.15.120:46322) 2024-09-19T09:15:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.809+0000 7eff5affd640 1 -- 172.21.15.120:0/2663193646 learned_addr learned my addr 172.21.15.120:0/2663193646 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.810+0000 7eff5a7fc640 1 --2- 172.21.15.120:0/2663193646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff5c104ef0 0x7eff5c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.810+0000 7eff5b7fe640 1 --2- 172.21.15.120:0/2663193646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff5c107820 0x7eff5c19f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.810+0000 7eff5affd640 1 -- 172.21.15.120:0/2663193646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff5c104ef0 msgr2=0x7eff5c198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.810+0000 7eff5affd640 1 --2- 172.21.15.120:0/2663193646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff5c104ef0 0x7eff5c198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.810+0000 7eff5affd640 1 -- 172.21.15.120:0/2663193646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff5c107820 msgr2=0x7eff5c19f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.810+0000 7eff5affd640 1 --2- 172.21.15.120:0/2663193646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff5c107820 0x7eff5c19f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.810+0000 7eff5affd640 1 -- 172.21.15.120:0/2663193646 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5c19fa60 con 0x7eff5c1025c0 2024-09-19T09:15:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.810+0000 7eff5affd640 1 --2- 172.21.15.120:0/2663193646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff5c1025c0 0x7eff5c197e40 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00d770 tx=0x7eff4c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.810+0000 7eff6096d640 1 -- 172.21.15.120:0/2663193646 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c017070 con 0x7eff5c1025c0 2024-09-19T09:15:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.810+0000 7eff6196f640 1 -- 172.21.15.120:0/2663193646 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff5c19fd50 con 0x7eff5c1025c0 2024-09-19T09:15:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.810+0000 7eff6096d640 1 -- 172.21.15.120:0/2663193646 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff4c005500 con 0x7eff5c1025c0 2024-09-19T09:15:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.810+0000 7eff6096d640 1 -- 172.21.15.120:0/2663193646 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c004e00 con 0x7eff5c1025c0 2024-09-19T09:15:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.810+0000 7eff6196f640 1 -- 172.21.15.120:0/2663193646 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff5c1a01c0 con 0x7eff5c1025c0 2024-09-19T09:15:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.812+0000 7eff6096d640 1 -- 172.21.15.120:0/2663193646 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7eff4c004fa0 con 0x7eff5c1025c0 2024-09-19T09:15:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.813+0000 7eff647e1740 1 Processor -- start 2024-09-19T09:15:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.813+0000 7eff647e1740 1 -- start start 2024-09-19T09:15:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.813+0000 7eff647e1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561da2fa9da0 0x561da2faa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.813+0000 7eff647e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561da2faa6e0 0x561da2faf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.813+0000 7eff647e1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561da2faf770 0x561da2fb1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.813+0000 7eff647e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561da2f092f0 con 0x561da2faa6e0 2024-09-19T09:15:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.813+0000 7eff647e1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561da2eb1270 con 0x561da2fa9da0 2024-09-19T09:15:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.813+0000 7eff647e1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561da2f052b0 con 0x561da2faf770 2024-09-19T09:15:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.813+0000 7eff5affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561da2faa6e0 0x561da2faf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.813+0000 7eff5affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561da2faa6e0 0x561da2faf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46338/0 (socket says 172.21.15.120:46338) 2024-09-19T09:15:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.813+0000 7eff5affd640 1 -- 172.21.15.120:0/1118260083 learned_addr learned my addr 172.21.15.120:0/1118260083 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.813+0000 7eff6096d640 1 --2- 172.21.15.120:0/2663193646 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7eff440a32f0 0x7eff440a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.813+0000 7eff5a7fc640 1 --2- 172.21.15.120:0/1118260083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561da2faf770 0x561da2fb1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.813+0000 7eff5b7fe640 1 --2- 172.21.15.120:0/1118260083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561da2fa9da0 0x561da2faa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.813+0000 7eff5affd640 1 -- 172.21.15.120:0/1118260083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561da2faf770 msgr2=0x561da2fb1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.813+0000 7eff5affd640 1 --2- 172.21.15.120:0/1118260083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561da2faf770 0x561da2fb1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.813+0000 7eff5affd640 1 -- 172.21.15.120:0/1118260083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561da2fa9da0 msgr2=0x561da2faa1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.813+0000 7eff5affd640 1 --2- 172.21.15.120:0/1118260083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561da2fa9da0 0x561da2faa1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.813+0000 7eff5affd640 1 -- 172.21.15.120:0/1118260083 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561da2eb0340 con 0x561da2faa6e0 2024-09-19T09:15:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.813+0000 7eff5a7fc640 1 --2- 172.21.15.120:0/1118260083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561da2faf770 0x561da2fb1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:15:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.813+0000 7eff6096d640 1 -- 172.21.15.120:0/2663193646 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7eff4c0cf440 con 0x7eff5c1025c0 2024-09-19T09:15:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.813+0000 7eff5b7fe640 1 --2- 172.21.15.120:0/2663193646 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7eff440a32f0 0x7eff440a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.814+0000 7eff5affd640 1 --2- 172.21.15.120:0/1118260083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561da2faa6e0 0x561da2faf1a0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7eff4c058c50 tx=0x7eff4c05b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6ddfc7db47c04e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.814+0000 7eff41ffb640 1 -- 172.21.15.120:0/1118260083 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c020070 con 0x561da2faa6e0 2024-09-19T09:15:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.814+0000 7eff41ffb640 1 -- 172.21.15.120:0/1118260083 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff4c0046c0 con 0x561da2faa6e0 2024-09-19T09:15:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.814+0000 7eff41ffb640 1 -- 172.21.15.120:0/1118260083 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c01e120 con 0x561da2faa6e0 2024-09-19T09:15:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.815+0000 7eff647e1740 1 -- 172.21.15.120:0/1118260083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561da2faa6e0 msgr2=0x561da2faf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.815+0000 7eff647e1740 1 --2- 172.21.15.120:0/1118260083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561da2faa6e0 0x561da2faf1a0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7eff4c058c50 tx=0x7eff4c05b890 comp rx=0 tx=0).stop 2024-09-19T09:15:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.815+0000 7eff5b7fe640 1 --2- 172.21.15.120:0/2663193646 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7eff440a32f0 0x7eff440a57b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7eff5c06a980 tx=0x7eff54004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.815+0000 7eff647e1740 1 -- 172.21.15.120:0/1118260083 shutdown_connections 2024-09-19T09:15:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.815+0000 7eff647e1740 1 --2- 172.21.15.120:0/1118260083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561da2faf770 0x561da2fb1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.815+0000 7eff647e1740 1 --2- 172.21.15.120:0/1118260083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561da2faa6e0 0x561da2faf1a0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.815+0000 7eff647e1740 1 --2- 172.21.15.120:0/1118260083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561da2fa9da0 0x561da2faa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.815+0000 7eff647e1740 1 -- 172.21.15.120:0/1118260083 >> 172.21.15.120:0/1118260083 conn(0x561da2fa5410 msgr2=0x561da2fa5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.815+0000 7eff647e1740 1 -- 172.21.15.120:0/1118260083 shutdown_connections 2024-09-19T09:15:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.815+0000 7eff647e1740 1 -- 172.21.15.120:0/1118260083 wait complete. 2024-09-19T09:15:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.815+0000 7eff647e1740 1 -- start start 2024-09-19T09:15:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.815+0000 7eff647e1740 1 Processor -- start 2024-09-19T09:15:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.815+0000 7eff647e1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561da2fa9da0 0x561da2fb3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.815+0000 7eff647e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561da2faa6e0 0x561da2fb4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.815+0000 7eff647e1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561da2faf770 0x561da2fbb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.815+0000 7eff647e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561da2fb26e0 con 0x561da2faa6e0 2024-09-19T09:15:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.815+0000 7eff647e1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561da2e92290 con 0x561da2faf770 2024-09-19T09:15:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.815+0000 7eff647e1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561da2fb2860 con 0x561da2fa9da0 2024-09-19T09:15:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.816+0000 7eff647e1740 10 client.0 _ll_register_callbacks cb 0x561da2ec3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.816+0000 7eff5a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561da2faa6e0 0x561da2fb4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.816+0000 7eff5a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561da2faa6e0 0x561da2fb4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46346/0 (socket says 172.21.15.120:46346) 2024-09-19T09:15:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.816+0000 7eff5a7fc640 1 -- 172.21.15.120:0/3296458246 learned_addr learned my addr 172.21.15.120:0/3296458246 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.816+0000 7eff5affd640 1 --2- 172.21.15.120:0/3296458246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561da2fa9da0 0x561da2fb3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.816+0000 7eff5b7fe640 1 --2- 172.21.15.120:0/3296458246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561da2faf770 0x561da2fbb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.816+0000 7eff5a7fc640 1 -- 172.21.15.120:0/3296458246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561da2fa9da0 msgr2=0x561da2fb3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.816+0000 7eff5a7fc640 1 --2- 172.21.15.120:0/3296458246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561da2fa9da0 0x561da2fb3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.816+0000 7eff5a7fc640 1 -- 172.21.15.120:0/3296458246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561da2faf770 msgr2=0x561da2fbb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.816+0000 7eff5a7fc640 1 --2- 172.21.15.120:0/3296458246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561da2faf770 0x561da2fbb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.816+0000 7eff5a7fc640 1 -- 172.21.15.120:0/3296458246 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561da2c1c210 con 0x561da2faa6e0 2024-09-19T09:15:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.816+0000 7eff5b7fe640 1 --2- 172.21.15.120:0/3296458246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561da2faf770 0x561da2fbb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:15:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.816+0000 7eff5affd640 1 --2- 172.21.15.120:0/3296458246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561da2fa9da0 0x561da2fb3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:15:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.816+0000 7eff5a7fc640 1 --2- 172.21.15.120:0/3296458246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561da2faa6e0 0x561da2fb4500 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7eff50000c00 tx=0x7eff500052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1aaad1d921446c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.816+0000 7eff3affd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:15:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.816+0000 7eff3affd640 1 -- 172.21.15.120:0/3296458246 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50035030 con 0x561da2faa6e0 2024-09-19T09:15:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.816+0000 7eff3affd640 1 -- 172.21.15.120:0/3296458246 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7eff500318b0 con 0x561da2faa6e0 2024-09-19T09:15:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.816+0000 7eff647e1740 1 -- 172.21.15.120:0/3296458246 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561da2f16a20 con 0x561da2faa6e0 2024-09-19T09:15:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.817+0000 7eff3affd640 1 -- 172.21.15.120:0/3296458246 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7eff50005d50 con 0x561da2faa6e0 2024-09-19T09:15:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.817+0000 7eff647e1740 10 client.19965 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.817+0000 7eff647e1740 1 -- 172.21.15.120:0/3296458246 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561da2ce8620 con 0x561da2faa6e0 2024-09-19T09:15:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.817+0000 7eff3affd640 1 -- 172.21.15.120:0/3296458246 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561da2f16a20 con 0x561da2faa6e0 2024-09-19T09:15:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.817+0000 7eff3affd640 1 -- 172.21.15.120:0/3296458246 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7eff500035f0 con 0x561da2faa6e0 2024-09-19T09:15:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.817+0000 7eff647e1740 10 client.19965 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.817+0000 7eff647e1740 10 client.19965 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:15:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.817+0000 7eff647e1740 20 client.19965 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.817+0000 7eff647e1740 1 --2- 172.21.15.120:0/3296458246 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561da2fbfc10 0x561da2fc2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.817+0000 7eff647e1740 4 client.19965 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:15:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.817+0000 7eff647e1740 1 -- 172.21.15.120:0/3296458246 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561da2f10d70 con 0x561da2fbfc10 2024-09-19T09:15:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.817+0000 7eff5affd640 1 --2- 172.21.15.120:0/3296458246 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561da2fbfc10 0x561da2fc2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.818+0000 7eff5affd640 1 --2- 172.21.15.120:0/3296458246 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561da2fbfc10 0x561da2fc2000 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0efaf538c03cf71 server_cookie=fc3a0b451c8fa773 in_seq=0 out_seq=0 2024-09-19T09:15:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.818+0000 7eff3affd640 10 client.19965 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:50.827 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.818+0000 7eff3affd640 1 -- 172.21.15.120:0/3296458246 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561da2f10d70 con 0x561da2fbfc10 2024-09-19T09:15:50.827 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.818+0000 7eff3affd640 10 client.19965 handle_command_reply: tid=0 2024-09-19T09:15:50.827 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.818+0000 7eff647e1740 1 client.19965 shutdown 2024-09-19T09:15:50.827 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.819+0000 7eff647e1740 1 -- 172.21.15.120:0/3296458246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561da2faa6e0 msgr2=0x561da2fb4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:50.827 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.819+0000 7eff647e1740 1 --2- 172.21.15.120:0/3296458246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561da2faa6e0 0x561da2fb4500 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7eff50000c00 tx=0x7eff500052f0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.827 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.819+0000 7eff647e1740 1 -- 172.21.15.120:0/3296458246 shutdown_connections 2024-09-19T09:15:50.827 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.819+0000 7eff647e1740 1 --2- 172.21.15.120:0/3296458246 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561da2fbfc10 0x561da2fc2000 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.827 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.819+0000 7eff647e1740 1 --2- 172.21.15.120:0/3296458246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561da2faf770 0x561da2fbb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.827 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.819+0000 7eff3affd640 0 client.19965 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:50.827 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.819+0000 7eff647e1740 1 --2- 172.21.15.120:0/3296458246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561da2faa6e0 0x561da2fb4500 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.827 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.819+0000 7eff647e1740 1 --2- 172.21.15.120:0/3296458246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561da2fa9da0 0x561da2fb3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.827 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.819+0000 7eff647e1740 1 -- 172.21.15.120:0/3296458246 >> 172.21.15.120:0/3296458246 conn(0x561da2fa5410 msgr2=0x561da2fa0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:50.827 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.819+0000 7eff647e1740 1 -- 172.21.15.120:0/3296458246 shutdown_connections 2024-09-19T09:15:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.819+0000 7eff647e1740 1 -- 172.21.15.120:0/3296458246 wait complete. 2024-09-19T09:15:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.819+0000 7eff647e1740 20 client.19965 trim_cache size 0 max 16384 2024-09-19T09:15:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.849+0000 7eff647e1740 1 Processor -- start 2024-09-19T09:15:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.849+0000 7eff647e1740 1 -- start start 2024-09-19T09:15:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.849+0000 7eff647e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561da2fa9da0 0x561da2fa6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.849+0000 7eff647e1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561da2faa6e0 0x561da2fbb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.849+0000 7eff647e1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561da2faf770 0x561da2fc68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.849+0000 7eff647e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c1a6d50 con 0x561da2fa9da0 2024-09-19T09:15:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.849+0000 7eff647e1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c1a8890 con 0x561da2faf770 2024-09-19T09:15:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.849+0000 7eff647e1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c1a7420 con 0x561da2faa6e0 2024-09-19T09:15:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.849+0000 7eff5affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561da2fa9da0 0x561da2fa6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.849+0000 7eff5affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561da2fa9da0 0x561da2fa6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46360/0 (socket says 172.21.15.120:46360) 2024-09-19T09:15:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.849+0000 7eff5affd640 1 -- 172.21.15.120:0/1679475766 learned_addr learned my addr 172.21.15.120:0/1679475766 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.849+0000 7eff5a7fc640 1 --2- 172.21.15.120:0/1679475766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561da2faa6e0 0x561da2fbb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.849+0000 7eff5b7fe640 1 --2- 172.21.15.120:0/1679475766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561da2faf770 0x561da2fc68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.849+0000 7eff5affd640 1 -- 172.21.15.120:0/1679475766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561da2faa6e0 msgr2=0x561da2fbb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.849+0000 7eff5affd640 1 --2- 172.21.15.120:0/1679475766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561da2faa6e0 0x561da2fbb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.849+0000 7eff5affd640 1 -- 172.21.15.120:0/1679475766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561da2faf770 msgr2=0x561da2fc68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.849+0000 7eff5affd640 1 --2- 172.21.15.120:0/1679475766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561da2faf770 0x561da2fc68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.849+0000 7eff5affd640 1 -- 172.21.15.120:0/1679475766 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561da2fc2370 con 0x561da2fa9da0 2024-09-19T09:15:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.850+0000 7eff5affd640 1 --2- 172.21.15.120:0/1679475766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561da2fa9da0 0x561da2fa6f10 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7eff4c0048e0 tx=0x7eff4c0d2ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dfb7d3ddbc94e54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.850+0000 7eff417fa640 1 -- 172.21.15.120:0/1679475766 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c020040 con 0x561da2fa9da0 2024-09-19T09:15:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.850+0000 7eff417fa640 1 -- 172.21.15.120:0/1679475766 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff4c0d2d10 con 0x561da2fa9da0 2024-09-19T09:15:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.850+0000 7eff417fa640 1 -- 172.21.15.120:0/1679475766 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c005290 con 0x561da2fa9da0 2024-09-19T09:15:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.851+0000 7eff647e1740 1 -- 172.21.15.120:0/1679475766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561da2fa9da0 msgr2=0x561da2fa6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.851+0000 7eff647e1740 1 --2- 172.21.15.120:0/1679475766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561da2fa9da0 0x561da2fa6f10 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7eff4c0048e0 tx=0x7eff4c0d2ec0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.851+0000 7eff647e1740 1 -- 172.21.15.120:0/1679475766 shutdown_connections 2024-09-19T09:15:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.851+0000 7eff647e1740 1 --2- 172.21.15.120:0/1679475766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561da2faf770 0x561da2fc68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.851+0000 7eff647e1740 1 --2- 172.21.15.120:0/1679475766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561da2faa6e0 0x561da2fbb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.851+0000 7eff647e1740 1 --2- 172.21.15.120:0/1679475766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561da2fa9da0 0x561da2fa6f10 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.851+0000 7eff647e1740 1 -- 172.21.15.120:0/1679475766 >> 172.21.15.120:0/1679475766 conn(0x561da2fa5410 msgr2=0x561da2fa57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.851+0000 7eff647e1740 1 -- 172.21.15.120:0/1679475766 shutdown_connections 2024-09-19T09:15:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.851+0000 7eff647e1740 1 -- 172.21.15.120:0/1679475766 wait complete. 2024-09-19T09:15:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.851+0000 7eff647e1740 1 -- start start 2024-09-19T09:15:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.852+0000 7eff647e1740 1 Processor -- start 2024-09-19T09:15:50.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.852+0000 7eff647e1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561da2fa9da0 0x561da2fb8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:50.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.852+0000 7eff647e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561da2faa6e0 0x561da2fb8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:50.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.852+0000 7eff647e1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561da2faf770 0x561da2fb9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:50.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.852+0000 7eff647e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c1a81c0 con 0x561da2faa6e0 2024-09-19T09:15:50.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.852+0000 7eff647e1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c1a7af0 con 0x561da2fa9da0 2024-09-19T09:15:50.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.852+0000 7eff647e1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c1a4d10 con 0x561da2faf770 2024-09-19T09:15:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.852+0000 7eff647e1740 10 client.0 _ll_register_callbacks cb 0x7eff540093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.852+0000 7eff5a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561da2faa6e0 0x561da2fb8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.852+0000 7eff5a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561da2faa6e0 0x561da2fb8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46364/0 (socket says 172.21.15.120:46364) 2024-09-19T09:15:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.852+0000 7eff5a7fc640 1 -- 172.21.15.120:0/1666382011 learned_addr learned my addr 172.21.15.120:0/1666382011 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.852+0000 7eff5affd640 1 --2- 172.21.15.120:0/1666382011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561da2fa9da0 0x561da2fb8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.852+0000 7eff5b7fe640 1 --2- 172.21.15.120:0/1666382011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561da2faf770 0x561da2fb9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.852+0000 7eff5a7fc640 1 -- 172.21.15.120:0/1666382011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561da2faf770 msgr2=0x561da2fb9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.852+0000 7eff5a7fc640 1 --2- 172.21.15.120:0/1666382011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561da2faf770 0x561da2fb9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.852+0000 7eff5a7fc640 1 -- 172.21.15.120:0/1666382011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561da2fa9da0 msgr2=0x561da2fb8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.852+0000 7eff5a7fc640 1 --2- 172.21.15.120:0/1666382011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561da2fa9da0 0x561da2fb8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.852+0000 7eff5a7fc640 1 -- 172.21.15.120:0/1666382011 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561da2fba510 con 0x561da2faa6e0 2024-09-19T09:15:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.852+0000 7eff5b7fe640 1 --2- 172.21.15.120:0/1666382011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561da2faf770 0x561da2fb9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:15:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.853+0000 7eff5a7fc640 1 --2- 172.21.15.120:0/1666382011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561da2faa6e0 0x561da2fb8f60 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7eff50031440 tx=0x7eff50003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ee92a3c00212ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.853+0000 7eff3affd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:15:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.853+0000 7eff3affd640 1 -- 172.21.15.120:0/1666382011 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50035070 con 0x561da2faa6e0 2024-09-19T09:15:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.853+0000 7eff3affd640 1 -- 172.21.15.120:0/1666382011 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7eff50004ad0 con 0x561da2faa6e0 2024-09-19T09:15:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.853+0000 7eff647e1740 1 -- 172.21.15.120:0/1666382011 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff4c0d2d10 con 0x561da2faa6e0 2024-09-19T09:15:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.853+0000 7eff3affd640 1 -- 172.21.15.120:0/1666382011 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7eff50009520 con 0x561da2faa6e0 2024-09-19T09:15:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.853+0000 7eff647e1740 10 client.19971 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.853+0000 7eff647e1740 1 -- 172.21.15.120:0/1666382011 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561da302a8a0 con 0x561da2faa6e0 2024-09-19T09:15:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.853+0000 7eff3affd640 1 -- 172.21.15.120:0/1666382011 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff4c0d2d10 con 0x561da2faa6e0 2024-09-19T09:15:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.853+0000 7eff3affd640 1 -- 172.21.15.120:0/1666382011 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7eff50044470 con 0x561da2faa6e0 2024-09-19T09:15:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.853+0000 7eff647e1740 10 client.19971 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.853+0000 7eff647e1740 10 client.19971 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:15:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.853+0000 7eff647e1740 20 client.19971 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.854+0000 7eff647e1740 1 --2- 172.21.15.120:0/1666382011 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561da2fbfc10 0x561da302d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.854+0000 7eff647e1740 4 client.19971 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:15:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.854+0000 7eff647e1740 1 -- 172.21.15.120:0/1666382011 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561da2e8d900 con 0x561da2fbfc10 2024-09-19T09:15:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.854+0000 7eff5affd640 1 --2- 172.21.15.120:0/1666382011 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561da2fbfc10 0x561da302d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.854+0000 7eff3affd640 10 client.19971 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.854+0000 7eff5affd640 1 --2- 172.21.15.120:0/1666382011 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561da2fbfc10 0x561da302d170 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a128b03de7f51cc6 server_cookie=bdfc6e9c97952074 in_seq=0 out_seq=0 2024-09-19T09:15:50.925 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:15:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.993+0000 7eff3affd640 1 -- 172.21.15.120:0/1666382011 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561da2e8d900 con 0x561da2fbfc10 2024-09-19T09:15:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.993+0000 7eff3affd640 10 client.19971 handle_command_reply: tid=0 2024-09-19T09:15:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.993+0000 7eff647e1740 1 client.19971 shutdown 2024-09-19T09:15:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.993+0000 7eff647e1740 1 -- 172.21.15.120:0/1666382011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561da2faa6e0 msgr2=0x561da2fb8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.993+0000 7eff647e1740 1 --2- 172.21.15.120:0/1666382011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561da2faa6e0 0x561da2fb8f60 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7eff50031440 tx=0x7eff50003ee0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.993+0000 7eff647e1740 1 -- 172.21.15.120:0/1666382011 shutdown_connections 2024-09-19T09:15:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.993+0000 7eff647e1740 1 --2- 172.21.15.120:0/1666382011 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561da2fbfc10 0x561da302d170 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.993+0000 7eff647e1740 1 --2- 172.21.15.120:0/1666382011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561da2faf770 0x561da2fb9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.993+0000 7eff647e1740 1 --2- 172.21.15.120:0/1666382011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561da2faa6e0 0x561da2fb8f60 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.993+0000 7eff647e1740 1 --2- 172.21.15.120:0/1666382011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561da2fa9da0 0x561da2fb8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.993+0000 7eff647e1740 1 -- 172.21.15.120:0/1666382011 >> 172.21.15.120:0/1666382011 conn(0x561da2fa5410 msgr2=0x561da2fa6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.993+0000 7eff3affd640 0 client.19971 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.994+0000 7eff647e1740 1 -- 172.21.15.120:0/1666382011 shutdown_connections 2024-09-19T09:15:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.994+0000 7eff647e1740 1 -- 172.21.15.120:0/1666382011 wait complete. 2024-09-19T09:15:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.994+0000 7eff647e1740 20 client.19971 trim_cache size 0 max 16384 2024-09-19T09:15:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:15:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:15:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:15:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:15:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.996+0000 7eff6196f640 1 -- 172.21.15.120:0/2663193646 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7eff440a32f0 msgr2=0x7eff440a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.996+0000 7eff6196f640 1 --2- 172.21.15.120:0/2663193646 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7eff440a32f0 0x7eff440a57b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7eff5c06a980 tx=0x7eff54004a40 comp rx=0 tx=0).stop 2024-09-19T09:15:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.996+0000 7eff6196f640 1 -- 172.21.15.120:0/2663193646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff5c1025c0 msgr2=0x7eff5c197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.996+0000 7eff6196f640 1 --2- 172.21.15.120:0/2663193646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff5c1025c0 0x7eff5c197e40 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00d770 tx=0x7eff4c00dc30 comp rx=0 tx=0).stop 2024-09-19T09:15:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.996+0000 7eff6196f640 1 -- 172.21.15.120:0/2663193646 shutdown_connections 2024-09-19T09:15:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.996+0000 7eff6196f640 1 --2- 172.21.15.120:0/2663193646 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7eff440a32f0 0x7eff440a57b0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.996+0000 7eff6196f640 1 --2- 172.21.15.120:0/2663193646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff5c107820 0x7eff5c19f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.996+0000 7eff6196f640 1 --2- 172.21.15.120:0/2663193646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff5c104ef0 0x7eff5c198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.996+0000 7eff6196f640 1 --2- 172.21.15.120:0/2663193646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff5c1025c0 0x7eff5c197e40 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.996+0000 7eff6196f640 1 -- 172.21.15.120:0/2663193646 >> 172.21.15.120:0/2663193646 conn(0x7eff5c0fc1e0 msgr2=0x7eff5c102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.996+0000 7eff6196f640 1 -- 172.21.15.120:0/2663193646 shutdown_connections 2024-09-19T09:15:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:15:50.996+0000 7eff6196f640 1 -- 172.21.15.120:0/2663193646 wait complete. 2024-09-19T09:15:51.078 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1301014 9.93 MB/sec execute 586 sec latency 8015.398 ms 2024-09-19T09:15:51.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:50 smithi196 ceph-mon[35150]: pgmap v733: 97 pgs: 97 active+clean; 46 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 308 op/s 2024-09-19T09:15:51.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:50 smithi153 ceph-mon[35071]: pgmap v733: 97 pgs: 97 active+clean; 46 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 308 op/s 2024-09-19T09:15:51.241 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:50 smithi120 ceph-mon[31150]: pgmap v733: 97 pgs: 97 active+clean; 46 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 308 op/s 2024-09-19T09:15:51.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.485+0000 7f16ecabf640 1 -- 172.21.15.120:0/3695713450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16e8103760 msgr2=0x7f16e8107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:51.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.485+0000 7f16ecabf640 1 --2- 172.21.15.120:0/3695713450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16e8103760 0x7f16e8107ba0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f16dc009930 tx=0x7f16dc02a420 comp rx=0 tx=0).stop 2024-09-19T09:15:51.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.485+0000 7f16ecabf640 1 -- 172.21.15.120:0/3695713450 shutdown_connections 2024-09-19T09:15:51.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.485+0000 7f16ecabf640 1 --2- 172.21.15.120:0/3695713450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16e81082e0 0x7f16e810ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.485+0000 7f16ecabf640 1 --2- 172.21.15.120:0/3695713450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16e8103760 0x7f16e8107ba0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.485+0000 7f16ecabf640 1 --2- 172.21.15.120:0/3695713450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16e8102db0 0x7f16e8103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.485+0000 7f16ecabf640 1 -- 172.21.15.120:0/3695713450 >> 172.21.15.120:0/3695713450 conn(0x7f16e80fc7d0 msgr2=0x7f16e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:51.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.485+0000 7f16ecabf640 1 -- 172.21.15.120:0/3695713450 shutdown_connections 2024-09-19T09:15:51.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.485+0000 7f16ecabf640 1 -- 172.21.15.120:0/3695713450 wait complete. 2024-09-19T09:15:51.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.486+0000 7f16ecabf640 1 Processor -- start 2024-09-19T09:15:51.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.486+0000 7f16ecabf640 1 -- start start 2024-09-19T09:15:51.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.486+0000 7f16ecabf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16e8102db0 0x7f16e81985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:51.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.486+0000 7f16ecabf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16e8103760 0x7f16e8198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:51.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.486+0000 7f16ecabf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16e81082e0 0x7f16e819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:51.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.486+0000 7f16ecabf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f16e8112050 con 0x7f16e8103760 2024-09-19T09:15:51.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.486+0000 7f16ecabf640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f16e8111ed0 con 0x7f16e8102db0 2024-09-19T09:15:51.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.486+0000 7f16ecabf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81121d0 con 0x7f16e81082e0 2024-09-19T09:15:51.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.486+0000 7f16e5d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16e8103760 0x7f16e8198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:51.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.486+0000 7f16e5d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16e8103760 0x7f16e8198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36440/0 (socket says 172.21.15.120:36440) 2024-09-19T09:15:51.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.486+0000 7f16e5d9b640 1 -- 172.21.15.120:0/3423380762 learned_addr learned my addr 172.21.15.120:0/3423380762 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:51.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.486+0000 7f16e6d9d640 1 --2- 172.21.15.120:0/3423380762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16e81082e0 0x7f16e819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:51.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.486+0000 7f16e659c640 1 --2- 172.21.15.120:0/3423380762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16e8102db0 0x7f16e81985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:51.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.486+0000 7f16e5d9b640 1 -- 172.21.15.120:0/3423380762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16e81082e0 msgr2=0x7f16e819ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:51.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.486+0000 7f16e5d9b640 1 --2- 172.21.15.120:0/3423380762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16e81082e0 0x7f16e819ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.486+0000 7f16e5d9b640 1 -- 172.21.15.120:0/3423380762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16e8102db0 msgr2=0x7f16e81985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:51.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.486+0000 7f16e5d9b640 1 --2- 172.21.15.120:0/3423380762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16e8102db0 0x7f16e81985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.486+0000 7f16e5d9b640 1 -- 172.21.15.120:0/3423380762 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e819d560 con 0x7f16e8103760 2024-09-19T09:15:51.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.487+0000 7f16e5d9b640 1 --2- 172.21.15.120:0/3423380762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16e8103760 0x7f16e8198af0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f16dc005940 tx=0x7f16dc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:51.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.487+0000 7f16d77fe640 1 -- 172.21.15.120:0/3423380762 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc03f070 con 0x7f16e8103760 2024-09-19T09:15:51.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.487+0000 7f16d77fe640 1 -- 172.21.15.120:0/3423380762 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16dc031c30 con 0x7f16e8103760 2024-09-19T09:15:51.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.487+0000 7f16ecabf640 1 -- 172.21.15.120:0/3423380762 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e819d7f0 con 0x7f16e8103760 2024-09-19T09:15:51.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.487+0000 7f16d77fe640 1 -- 172.21.15.120:0/3423380762 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc035030 con 0x7f16e8103760 2024-09-19T09:15:51.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.487+0000 7f16ecabf640 1 -- 172.21.15.120:0/3423380762 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e81a5130 con 0x7f16e8103760 2024-09-19T09:15:51.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.489+0000 7f16d77fe640 1 -- 172.21.15.120:0/3423380762 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f16dc042070 con 0x7f16e8103760 2024-09-19T09:15:51.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.489+0000 7f16eee98740 1 Processor -- start 2024-09-19T09:15:51.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.490+0000 7f16eee98740 1 -- start start 2024-09-19T09:15:51.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.490+0000 7f16eee98740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c936531b40 0x55c936530090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:51.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.490+0000 7f16eee98740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9365305d0 0x55c936530a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:51.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.490+0000 7f16eee98740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c93653b370 0x55c93653d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:51.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.490+0000 7f16eee98740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c9364962f0 con 0x55c9365305d0 2024-09-19T09:15:51.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.490+0000 7f16eee98740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c93643dcf0 con 0x55c936531b40 2024-09-19T09:15:51.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.490+0000 7f16eee98740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c9364922b0 con 0x55c93653b370 2024-09-19T09:15:51.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.490+0000 7f16e659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9365305d0 0x55c936530a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:51.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.490+0000 7f16e659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9365305d0 0x55c936530a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36456/0 (socket says 172.21.15.120:36456) 2024-09-19T09:15:51.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.490+0000 7f16e659c640 1 -- 172.21.15.120:0/1267668858 learned_addr learned my addr 172.21.15.120:0/1267668858 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:51.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.490+0000 7f16e6d9d640 1 --2- 172.21.15.120:0/1267668858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c936531b40 0x55c936530090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:51.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.490+0000 7f16d77fe640 1 --2- 172.21.15.120:0/3423380762 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16d00a3240 0x7f16d00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:51.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.490+0000 7f16e5d9b640 1 --2- 172.21.15.120:0/1267668858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c93653b370 0x55c93653d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:51.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.490+0000 7f16e6d9d640 1 --2- 172.21.15.120:0/3423380762 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16d00a3240 0x7f16d00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:51.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.491+0000 7f16e659c640 1 -- 172.21.15.120:0/1267668858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c93653b370 msgr2=0x55c93653d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:51.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.491+0000 7f16e659c640 1 --2- 172.21.15.120:0/1267668858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c93653b370 0x55c93653d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.491+0000 7f16d77fe640 1 -- 172.21.15.120:0/3423380762 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f16dc0f01b0 con 0x7f16e8103760 2024-09-19T09:15:51.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.491+0000 7f16e659c640 1 -- 172.21.15.120:0/1267668858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c936531b40 msgr2=0x55c936530090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:51.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.491+0000 7f16e659c640 1 --2- 172.21.15.120:0/1267668858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c936531b40 0x55c936530090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.491+0000 7f16e659c640 1 -- 172.21.15.120:0/1267668858 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9364223a0 con 0x55c9365305d0 2024-09-19T09:15:51.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.491+0000 7f16e659c640 1 --2- 172.21.15.120:0/1267668858 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9365305d0 0x55c936530a10 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f16d800c7b0 tx=0x7f16d800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf329b0629f7f4b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:51.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.491+0000 7f16e6d9d640 1 --2- 172.21.15.120:0/3423380762 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16d00a3240 0x7f16d00a5700 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f16e00106b0 tx=0x7f16e0004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:51.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.491+0000 7f16d4ff9640 1 -- 172.21.15.120:0/1267668858 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d800a830 con 0x55c9365305d0 2024-09-19T09:15:51.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.491+0000 7f16d4ff9640 1 -- 172.21.15.120:0/1267668858 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16d8016050 con 0x55c9365305d0 2024-09-19T09:15:51.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.491+0000 7f16d4ff9640 1 -- 172.21.15.120:0/1267668858 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d801f5a0 con 0x55c9365305d0 2024-09-19T09:15:51.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.492+0000 7f16eee98740 1 -- 172.21.15.120:0/1267668858 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9365305d0 msgr2=0x55c936530a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:51.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.492+0000 7f16eee98740 1 --2- 172.21.15.120:0/1267668858 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9365305d0 0x55c936530a10 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f16d800c7b0 tx=0x7f16d800cc70 comp rx=0 tx=0).stop 2024-09-19T09:15:51.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.492+0000 7f16eee98740 1 -- 172.21.15.120:0/1267668858 shutdown_connections 2024-09-19T09:15:51.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.492+0000 7f16eee98740 1 --2- 172.21.15.120:0/1267668858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c93653b370 0x55c93653d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.492+0000 7f16eee98740 1 --2- 172.21.15.120:0/1267668858 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9365305d0 0x55c936530a10 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.492+0000 7f16eee98740 1 --2- 172.21.15.120:0/1267668858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c936531b40 0x55c936530090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.492+0000 7f16eee98740 1 -- 172.21.15.120:0/1267668858 >> 172.21.15.120:0/1267668858 conn(0x55c9364f9310 msgr2=0x55c93652cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:51.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.492+0000 7f16eee98740 1 -- 172.21.15.120:0/1267668858 shutdown_connections 2024-09-19T09:15:51.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.492+0000 7f16eee98740 1 -- 172.21.15.120:0/1267668858 wait complete. 2024-09-19T09:15:51.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.492+0000 7f16eee98740 1 -- start start 2024-09-19T09:15:51.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.492+0000 7f16eee98740 1 Processor -- start 2024-09-19T09:15:51.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.493+0000 7f16eee98740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9365305d0 0x55c93653acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:51.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.493+0000 7f16eee98740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c936531b40 0x55c936537f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:51.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.493+0000 7f16eee98740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c93653b370 0x55c9365384a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:51.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.493+0000 7f16eee98740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c93653e070 con 0x55c93653b370 2024-09-19T09:15:51.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.493+0000 7f16eee98740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c936451060 con 0x55c9365305d0 2024-09-19T09:15:51.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.493+0000 7f16eee98740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c93653e1f0 con 0x55c936531b40 2024-09-19T09:15:51.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.493+0000 7f16e6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c93653b370 0x55c9365384a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:51.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.493+0000 7f16e5d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c936531b40 0x55c936537f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:51.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.493+0000 7f16eee98740 10 client.0 _ll_register_callbacks cb 0x55c9361a48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:51.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.493+0000 7f16e659c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9365305d0 0x55c93653acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:51.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.493+0000 7f16e6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c93653b370 0x55c9365384a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36470/0 (socket says 172.21.15.120:36470) 2024-09-19T09:15:51.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.493+0000 7f16e5d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c936531b40 0x55c936537f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:47324/0 (socket says 172.21.15.120:47324) 2024-09-19T09:15:51.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.493+0000 7f16e6d9d640 1 -- 172.21.15.120:0/279375818 learned_addr learned my addr 172.21.15.120:0/279375818 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:51.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.493+0000 7f16e6d9d640 1 -- 172.21.15.120:0/279375818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c936531b40 msgr2=0x55c936537f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:51.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.493+0000 7f16e6d9d640 1 --2- 172.21.15.120:0/279375818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c936531b40 0x55c936537f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.493+0000 7f16e6d9d640 1 -- 172.21.15.120:0/279375818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9365305d0 msgr2=0x55c93653acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:51.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.493+0000 7f16e6d9d640 1 --2- 172.21.15.120:0/279375818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9365305d0 0x55c93653acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.493+0000 7f16e6d9d640 1 -- 172.21.15.120:0/279375818 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9362073c0 con 0x55c93653b370 2024-09-19T09:15:51.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.493+0000 7f16e659c640 1 --2- 172.21.15.120:0/279375818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9365305d0 0x55c93653acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:15:51.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.493+0000 7f16e6d9d640 1 --2- 172.21.15.120:0/279375818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c93653b370 0x55c9365384a0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f16e0003bd0 tx=0x7f16e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e28b2af4b3e7a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:51.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.493+0000 7f16cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:15:51.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.494+0000 7f16cdffb640 1 -- 172.21.15.120:0/279375818 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0004830 con 0x55c93653b370 2024-09-19T09:15:51.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.494+0000 7f16cdffb640 1 -- 172.21.15.120:0/279375818 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f16e0004b20 con 0x55c93653b370 2024-09-19T09:15:51.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.494+0000 7f16eee98740 1 -- 172.21.15.120:0/279375818 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16d8016050 con 0x55c93653b370 2024-09-19T09:15:51.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.494+0000 7f16cdffb640 1 -- 172.21.15.120:0/279375818 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f16e0004d20 con 0x55c93653b370 2024-09-19T09:15:51.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.494+0000 7f16eee98740 10 client.19983 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:51.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.494+0000 7f16eee98740 1 -- 172.21.15.120:0/279375818 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c936549a60 con 0x55c93653b370 2024-09-19T09:15:51.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.494+0000 7f16cdffb640 1 -- 172.21.15.120:0/279375818 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16d8016050 con 0x55c93653b370 2024-09-19T09:15:51.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.494+0000 7f16cdffb640 1 -- 172.21.15.120:0/279375818 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f16e001a070 con 0x55c93653b370 2024-09-19T09:15:51.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.494+0000 7f16eee98740 10 client.19983 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:51.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.494+0000 7f16eee98740 10 client.19983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:15:51.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.494+0000 7f16eee98740 20 client.19983 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:51.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.494+0000 7f16eee98740 1 --2- 172.21.15.120:0/279375818 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c93654af20 0x55c93654d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:51.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.494+0000 7f16eee98740 4 client.19983 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:15:51.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.494+0000 7f16eee98740 1 -- 172.21.15.120:0/279375818 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9364a3860 con 0x55c93654af20 2024-09-19T09:15:51.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.495+0000 7f16e659c640 1 --2- 172.21.15.120:0/279375818 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c93654af20 0x55c93654d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:51.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.495+0000 7f16e659c640 1 --2- 172.21.15.120:0/279375818 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c93654af20 0x55c93654d3c0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9ac6045c11a1fd2 server_cookie=e9f2e4deeb580792 in_seq=0 out_seq=0 2024-09-19T09:15:51.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.495+0000 7f16cdffb640 10 client.19983 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:51.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.496+0000 7f16cdffb640 1 -- 172.21.15.120:0/279375818 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c9364a3860 con 0x55c93654af20 2024-09-19T09:15:51.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.496+0000 7f16cdffb640 10 client.19983 handle_command_reply: tid=0 2024-09-19T09:15:51.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.496+0000 7f16eee98740 1 client.19983 shutdown 2024-09-19T09:15:51.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.496+0000 7f16eee98740 1 -- 172.21.15.120:0/279375818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c93653b370 msgr2=0x55c9365384a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:51.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.496+0000 7f16eee98740 1 --2- 172.21.15.120:0/279375818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c93653b370 0x55c9365384a0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f16e0003bd0 tx=0x7f16e0004a40 comp rx=0 tx=0).stop 2024-09-19T09:15:51.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.496+0000 7f16eee98740 1 -- 172.21.15.120:0/279375818 shutdown_connections 2024-09-19T09:15:51.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.496+0000 7f16eee98740 1 --2- 172.21.15.120:0/279375818 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c93654af20 0x55c93654d3c0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.496+0000 7f16eee98740 1 --2- 172.21.15.120:0/279375818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c93653b370 0x55c9365384a0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.496+0000 7f16eee98740 1 --2- 172.21.15.120:0/279375818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c936531b40 0x55c936537f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.496+0000 7f16eee98740 1 --2- 172.21.15.120:0/279375818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9365305d0 0x55c93653acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.496+0000 7f16cdffb640 0 client.19983 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:51.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.496+0000 7f16eee98740 1 -- 172.21.15.120:0/279375818 >> 172.21.15.120:0/279375818 conn(0x55c9364f9310 msgr2=0x55c936532930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:51.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.496+0000 7f16eee98740 1 -- 172.21.15.120:0/279375818 shutdown_connections 2024-09-19T09:15:51.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.496+0000 7f16eee98740 1 -- 172.21.15.120:0/279375818 wait complete. 2024-09-19T09:15:51.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.496+0000 7f16eee98740 20 client.19983 trim_cache size 0 max 16384 2024-09-19T09:15:51.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.509+0000 7f16eee98740 1 Processor -- start 2024-09-19T09:15:51.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.509+0000 7f16eee98740 1 -- start start 2024-09-19T09:15:51.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.509+0000 7f16eee98740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9364f9310 0x55c93652ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:51.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.509+0000 7f16eee98740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9365305d0 0x55c936547310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:51.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.509+0000 7f16eee98740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c936531b40 0x55c936537820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:51.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.509+0000 7f16eee98740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81ab450 con 0x55c936531b40 2024-09-19T09:15:51.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.509+0000 7f16eee98740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81acf90 con 0x55c9365305d0 2024-09-19T09:15:51.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.509+0000 7f16eee98740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81abb20 con 0x55c9364f9310 2024-09-19T09:15:51.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.509+0000 7f16e6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c936531b40 0x55c936537820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:51.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.509+0000 7f16e6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c936531b40 0x55c936537820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36472/0 (socket says 172.21.15.120:36472) 2024-09-19T09:15:51.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.509+0000 7f16e6d9d640 1 -- 172.21.15.120:0/3140008335 learned_addr learned my addr 172.21.15.120:0/3140008335 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:51.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.509+0000 7f16e659c640 1 --2- 172.21.15.120:0/3140008335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9364f9310 0x55c93652ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:51.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.509+0000 7f16e5d9b640 1 --2- 172.21.15.120:0/3140008335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9365305d0 0x55c936547310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:51.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.510+0000 7f16e6d9d640 1 -- 172.21.15.120:0/3140008335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9364f9310 msgr2=0x55c93652ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:51.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.510+0000 7f16e6d9d640 1 --2- 172.21.15.120:0/3140008335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9364f9310 0x55c93652ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.510+0000 7f16e6d9d640 1 -- 172.21.15.120:0/3140008335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9365305d0 msgr2=0x55c936547310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:51.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.510+0000 7f16e6d9d640 1 --2- 172.21.15.120:0/3140008335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9365305d0 0x55c936547310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.510+0000 7f16e6d9d640 1 -- 172.21.15.120:0/3140008335 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c93653a8d0 con 0x55c936531b40 2024-09-19T09:15:51.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.510+0000 7f16e6d9d640 1 --2- 172.21.15.120:0/3140008335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c936531b40 0x55c936537820 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f16e0000f80 tx=0x7f16e00119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1a35905efe2360 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:51.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.510+0000 7f16cffff640 1 -- 172.21.15.120:0/3140008335 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0035070 con 0x55c936531b40 2024-09-19T09:15:51.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.510+0000 7f16cffff640 1 -- 172.21.15.120:0/3140008335 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16e0004b20 con 0x55c936531b40 2024-09-19T09:15:51.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.510+0000 7f16cffff640 1 -- 172.21.15.120:0/3140008335 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0009ba0 con 0x55c936531b40 2024-09-19T09:15:51.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.511+0000 7f16eee98740 1 -- 172.21.15.120:0/3140008335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c936531b40 msgr2=0x55c936537820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:51.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.511+0000 7f16eee98740 1 --2- 172.21.15.120:0/3140008335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c936531b40 0x55c936537820 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f16e0000f80 tx=0x7f16e00119b0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.511+0000 7f16eee98740 1 -- 172.21.15.120:0/3140008335 shutdown_connections 2024-09-19T09:15:51.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.511+0000 7f16eee98740 1 --2- 172.21.15.120:0/3140008335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c936531b40 0x55c936537820 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.511+0000 7f16eee98740 1 --2- 172.21.15.120:0/3140008335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9365305d0 0x55c936547310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.511+0000 7f16eee98740 1 --2- 172.21.15.120:0/3140008335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9364f9310 0x55c93652ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.511+0000 7f16eee98740 1 -- 172.21.15.120:0/3140008335 >> 172.21.15.120:0/3140008335 conn(0x55c936538600 msgr2=0x55c93652c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:51.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.511+0000 7f16eee98740 1 -- 172.21.15.120:0/3140008335 shutdown_connections 2024-09-19T09:15:51.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.511+0000 7f16eee98740 1 -- 172.21.15.120:0/3140008335 wait complete. 2024-09-19T09:15:51.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.511+0000 7f16eee98740 1 -- start start 2024-09-19T09:15:51.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.511+0000 7f16eee98740 1 Processor -- start 2024-09-19T09:15:51.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.512+0000 7f16eee98740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9364f9310 0x55c936543ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:51.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.512+0000 7f16eee98740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9365305d0 0x55c93653efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:51.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.512+0000 7f16eee98740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c936531b40 0x55c93653f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:51.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.512+0000 7f16eee98740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81ac8c0 con 0x55c9365305d0 2024-09-19T09:15:51.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.512+0000 7f16eee98740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81ac1f0 con 0x55c936531b40 2024-09-19T09:15:51.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.512+0000 7f16eee98740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81a9430 con 0x55c9364f9310 2024-09-19T09:15:51.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.512+0000 7f16eee98740 10 client.0 _ll_register_callbacks cb 0x7f16e0011c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:51.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.512+0000 7f16e5d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9365305d0 0x55c93653efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:51.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.512+0000 7f16e5d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9365305d0 0x55c93653efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36482/0 (socket says 172.21.15.120:36482) 2024-09-19T09:15:51.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.512+0000 7f16e5d9b640 1 -- 172.21.15.120:0/753995018 learned_addr learned my addr 172.21.15.120:0/753995018 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:51.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.512+0000 7f16e659c640 1 --2- 172.21.15.120:0/753995018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9364f9310 0x55c936543ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:51.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.512+0000 7f16e6d9d640 1 --2- 172.21.15.120:0/753995018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c936531b40 0x55c93653f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:51.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.512+0000 7f16e5d9b640 1 -- 172.21.15.120:0/753995018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9364f9310 msgr2=0x55c936543ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:51.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.512+0000 7f16e5d9b640 1 --2- 172.21.15.120:0/753995018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9364f9310 0x55c936543ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.512+0000 7f16e5d9b640 1 -- 172.21.15.120:0/753995018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c936531b40 msgr2=0x55c93653f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:51.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.512+0000 7f16e5d9b640 1 --2- 172.21.15.120:0/753995018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c936531b40 0x55c93653f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.512+0000 7f16e5d9b640 1 -- 172.21.15.120:0/753995018 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c93653fda0 con 0x55c9365305d0 2024-09-19T09:15:51.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.512+0000 7f16e5d9b640 1 --2- 172.21.15.120:0/753995018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9365305d0 0x55c93653efd0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f16dc0b91a0 tx=0x7f16dc03b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9358c15d029cde91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:51.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.512+0000 7f16cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:15:51.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.512+0000 7f16cdffb640 1 -- 172.21.15.120:0/753995018 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc0ef040 con 0x55c9365305d0 2024-09-19T09:15:51.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.512+0000 7f16cdffb640 1 -- 172.21.15.120:0/753995018 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f16dc03b4f0 con 0x55c9365305d0 2024-09-19T09:15:51.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.512+0000 7f16eee98740 1 -- 172.21.15.120:0/753995018 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16e0004b20 con 0x55c9365305d0 2024-09-19T09:15:51.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.512+0000 7f16cdffb640 1 -- 172.21.15.120:0/753995018 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f16dc049040 con 0x55c9365305d0 2024-09-19T09:15:51.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.512+0000 7f16eee98740 10 client.19989 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:51.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.512+0000 7f16eee98740 1 -- 172.21.15.120:0/753995018 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c9365b5d80 con 0x55c9365305d0 2024-09-19T09:15:51.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.513+0000 7f16cdffb640 1 -- 172.21.15.120:0/753995018 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16e0004b20 con 0x55c9365305d0 2024-09-19T09:15:51.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.513+0000 7f16cdffb640 1 -- 172.21.15.120:0/753995018 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f16dc0c4fb0 con 0x55c9365305d0 2024-09-19T09:15:51.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.513+0000 7f16eee98740 10 client.19989 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:51.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.513+0000 7f16eee98740 10 client.19989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:15:51.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.513+0000 7f16eee98740 20 client.19989 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:51.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.513+0000 7f16eee98740 1 --2- 172.21.15.120:0/753995018 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c93653b370 0x55c9365b8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:51.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.513+0000 7f16eee98740 4 client.19989 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:15:51.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.513+0000 7f16eee98740 1 -- 172.21.15.120:0/753995018 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c93649dd70 con 0x55c93653b370 2024-09-19T09:15:51.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.513+0000 7f16e659c640 1 --2- 172.21.15.120:0/753995018 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c93653b370 0x55c9365b8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:51.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.513+0000 7f16e659c640 1 --2- 172.21.15.120:0/753995018 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c93653b370 0x55c9365b8540 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfbb72bc97e9579e server_cookie=caec8b11b3c3a5 in_seq=0 out_seq=0 2024-09-19T09:15:51.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.513+0000 7f16cdffb640 10 client.19989 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:51.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.514+0000 7f16cdffb640 1 -- 172.21.15.120:0/753995018 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c93649dd70 con 0x55c93653b370 2024-09-19T09:15:51.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.514+0000 7f16cdffb640 10 client.19989 handle_command_reply: tid=0 2024-09-19T09:15:51.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.514+0000 7f16eee98740 1 client.19989 shutdown 2024-09-19T09:15:51.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.514+0000 7f16eee98740 1 -- 172.21.15.120:0/753995018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9365305d0 msgr2=0x55c93653efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:51.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.514+0000 7f16eee98740 1 --2- 172.21.15.120:0/753995018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9365305d0 0x55c93653efd0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f16dc0b91a0 tx=0x7f16dc03b890 comp rx=0 tx=0).stop 2024-09-19T09:15:51.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.514+0000 7f16eee98740 1 -- 172.21.15.120:0/753995018 shutdown_connections 2024-09-19T09:15:51.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.514+0000 7f16eee98740 1 --2- 172.21.15.120:0/753995018 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c93653b370 0x55c9365b8540 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.514+0000 7f16eee98740 1 --2- 172.21.15.120:0/753995018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c936531b40 0x55c93653f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.514+0000 7f16cdffb640 0 client.19989 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:51.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.514+0000 7f16eee98740 1 --2- 172.21.15.120:0/753995018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9365305d0 0x55c93653efd0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.514+0000 7f16eee98740 1 --2- 172.21.15.120:0/753995018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9364f9310 0x55c936543ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.514+0000 7f16eee98740 1 -- 172.21.15.120:0/753995018 >> 172.21.15.120:0/753995018 conn(0x55c936538600 msgr2=0x55c936532110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:51.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.515+0000 7f16eee98740 1 -- 172.21.15.120:0/753995018 shutdown_connections 2024-09-19T09:15:51.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.515+0000 7f16eee98740 1 -- 172.21.15.120:0/753995018 wait complete. 2024-09-19T09:15:51.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.515+0000 7f16eee98740 20 client.19989 trim_cache size 0 max 16384 2024-09-19T09:15:51.537 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:15:51.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.516+0000 7f16ecabf640 1 -- 172.21.15.120:0/3423380762 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16d00a3240 msgr2=0x7f16d00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:51.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.516+0000 7f16ecabf640 1 --2- 172.21.15.120:0/3423380762 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16d00a3240 0x7f16d00a5700 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f16e00106b0 tx=0x7f16e0004ab0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.516+0000 7f16ecabf640 1 -- 172.21.15.120:0/3423380762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16e8103760 msgr2=0x7f16e8198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:51.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.516+0000 7f16ecabf640 1 --2- 172.21.15.120:0/3423380762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16e8103760 0x7f16e8198af0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f16dc005940 tx=0x7f16dc003c10 comp rx=0 tx=0).stop 2024-09-19T09:15:51.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.516+0000 7f16ecabf640 1 -- 172.21.15.120:0/3423380762 shutdown_connections 2024-09-19T09:15:51.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.516+0000 7f16ecabf640 1 --2- 172.21.15.120:0/3423380762 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f16d00a3240 0x7f16d00a5700 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.516+0000 7f16ecabf640 1 --2- 172.21.15.120:0/3423380762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16e81082e0 0x7f16e819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.516+0000 7f16ecabf640 1 --2- 172.21.15.120:0/3423380762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16e8103760 0x7f16e8198af0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.516+0000 7f16ecabf640 1 --2- 172.21.15.120:0/3423380762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16e8102db0 0x7f16e81985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:51.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.516+0000 7f16ecabf640 1 -- 172.21.15.120:0/3423380762 >> 172.21.15.120:0/3423380762 conn(0x7f16e80fc7d0 msgr2=0x7f16e8105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:51.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.516+0000 7f16ecabf640 1 -- 172.21.15.120:0/3423380762 shutdown_connections 2024-09-19T09:15:51.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:51.516+0000 7f16ecabf640 1 -- 172.21.15.120:0/3423380762 wait complete. 2024-09-19T09:15:52.078 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1301014 9.91 MB/sec execute 587 sec latency 9015.482 ms 2024-09-19T09:15:53.078 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1301014 9.90 MB/sec execute 588 sec latency 10015.559 ms 2024-09-19T09:15:53.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:52 smithi196 ceph-mon[35150]: pgmap v734: 97 pgs: 97 active+clean; 45 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 253 op/s 2024-09-19T09:15:53.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:52 smithi153 ceph-mon[35071]: pgmap v734: 97 pgs: 97 active+clean; 45 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 253 op/s 2024-09-19T09:15:53.246 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:15:53.246 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:15:53.275 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:52 smithi120 ceph-mon[31150]: pgmap v734: 97 pgs: 97 active+clean; 45 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 253 op/s 2024-09-19T09:15:53.505 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:15:54.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.023+0000 7fe036bfa640 1 -- 172.21.15.120:0/2938833224 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe030106a80 msgr2=0x7fe030106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:54.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.023+0000 7fe036bfa640 1 --2- 172.21.15.120:0/2938833224 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe030106a80 0x7fe030106e60 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fe02c009930 tx=0x7fe02c02a420 comp rx=0 tx=0).stop 2024-09-19T09:15:54.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.023+0000 7fe036bfa640 1 -- 172.21.15.120:0/2938833224 shutdown_connections 2024-09-19T09:15:54.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.023+0000 7fe036bfa640 1 --2- 172.21.15.120:0/2938833224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe030101420 0x7fe03010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.023+0000 7fe036bfa640 1 --2- 172.21.15.120:0/2938833224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe030100a80 0x7fe030100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.023+0000 7fe036bfa640 1 --2- 172.21.15.120:0/2938833224 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe030106a80 0x7fe030106e60 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.023+0000 7fe036bfa640 1 -- 172.21.15.120:0/2938833224 >> 172.21.15.120:0/2938833224 conn(0x7fe0300fc7a0 msgr2=0x7fe0300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:54.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.023+0000 7fe036bfa640 1 -- 172.21.15.120:0/2938833224 shutdown_connections 2024-09-19T09:15:54.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.023+0000 7fe036bfa640 1 -- 172.21.15.120:0/2938833224 wait complete. 2024-09-19T09:15:54.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.024+0000 7fe036bfa640 1 Processor -- start 2024-09-19T09:15:54.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.024+0000 7fe036bfa640 1 -- start start 2024-09-19T09:15:54.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.024+0000 7fe036bfa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe030100a80 0x7fe030198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:54.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.024+0000 7fe036bfa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe030101420 0x7fe030198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:54.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.024+0000 7fe036bfa640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe030106a80 0x7fe03019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:54.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.024+0000 7fe036bfa640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe030110ea0 con 0x7fe030101420 2024-09-19T09:15:54.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.024+0000 7fe036bfa640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe030110d20 con 0x7fe030106a80 2024-09-19T09:15:54.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.024+0000 7fe036bfa640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe030111020 con 0x7fe030100a80 2024-09-19T09:15:54.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.024+0000 7fe027fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe030101420 0x7fe030198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:54.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.024+0000 7fe027fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe030101420 0x7fe030198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36512/0 (socket says 172.21.15.120:36512) 2024-09-19T09:15:54.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.024+0000 7fe027fff640 1 -- 172.21.15.120:0/1260095331 learned_addr learned my addr 172.21.15.120:0/1260095331 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:54.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.024+0000 7fe034996640 1 --2- 172.21.15.120:0/1260095331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe030100a80 0x7fe030198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:54.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.025+0000 7fe027fff640 1 -- 172.21.15.120:0/1260095331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe030100a80 msgr2=0x7fe030198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:54.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.025+0000 7fe027fff640 1 --2- 172.21.15.120:0/1260095331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe030100a80 0x7fe030198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.025+0000 7fe035197640 1 --2- 172.21.15.120:0/1260095331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe030106a80 0x7fe03019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:54.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.025+0000 7fe027fff640 1 -- 172.21.15.120:0/1260095331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe030106a80 msgr2=0x7fe03019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:54.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.025+0000 7fe027fff640 1 --2- 172.21.15.120:0/1260095331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe030106a80 0x7fe03019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.025+0000 7fe027fff640 1 -- 172.21.15.120:0/1260095331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe03019d3e0 con 0x7fe030101420 2024-09-19T09:15:54.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.025+0000 7fe035197640 1 --2- 172.21.15.120:0/1260095331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe030106a80 0x7fe03019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:15:54.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.025+0000 7fe027fff640 1 --2- 172.21.15.120:0/1260095331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe030101420 0x7fe030198970 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fe02000c920 tx=0x7fe02000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:54.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.025+0000 7fe025ffb640 1 -- 172.21.15.120:0/1260095331 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe020007c70 con 0x7fe030101420 2024-09-19T09:15:54.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.025+0000 7fe036bfa640 1 -- 172.21.15.120:0/1260095331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe03019d6d0 con 0x7fe030101420 2024-09-19T09:15:54.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.025+0000 7fe025ffb640 1 -- 172.21.15.120:0/1260095331 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe020007e10 con 0x7fe030101420 2024-09-19T09:15:54.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.025+0000 7fe025ffb640 1 -- 172.21.15.120:0/1260095331 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe020005260 con 0x7fe030101420 2024-09-19T09:15:54.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.025+0000 7fe036bfa640 1 -- 172.21.15.120:0/1260095331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0301a4fb0 con 0x7fe030101420 2024-09-19T09:15:54.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.027+0000 7fe025ffb640 1 -- 172.21.15.120:0/1260095331 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe020005400 con 0x7fe030101420 2024-09-19T09:15:54.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.028+0000 7fe038fd3740 1 Processor -- start 2024-09-19T09:15:54.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.028+0000 7fe038fd3740 1 -- start start 2024-09-19T09:15:54.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.028+0000 7fe025ffb640 1 --2- 172.21.15.120:0/1260095331 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe0180a3240 0x7fe0180a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:54.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.028+0000 7fe038fd3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e132d7b60 0x556e132d7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:54.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.028+0000 7fe038fd3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e132d8480 0x556e132e2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:54.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.028+0000 7fe038fd3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e132e2d40 0x556e132e5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:54.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.028+0000 7fe038fd3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556e1323d280 con 0x556e132e2d40 2024-09-19T09:15:54.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.028+0000 7fe038fd3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556e131e4b10 con 0x556e132d8480 2024-09-19T09:15:54.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.028+0000 7fe038fd3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556e13239240 con 0x556e132d7b60 2024-09-19T09:15:54.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.028+0000 7fe025ffb640 1 -- 172.21.15.120:0/1260095331 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fe0200cfb70 con 0x7fe030101420 2024-09-19T09:15:54.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.028+0000 7fe035197640 1 --2- 172.21.15.120:0/1260095331 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe0180a3240 0x7fe0180a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:54.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.028+0000 7fe035197640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e132e2d40 0x556e132e5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:54.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.029+0000 7fe035197640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e132e2d40 0x556e132e5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36518/0 (socket says 172.21.15.120:36518) 2024-09-19T09:15:54.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.029+0000 7fe035197640 1 -- 172.21.15.120:0/2317024897 learned_addr learned my addr 172.21.15.120:0/2317024897 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:54.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.029+0000 7fe034996640 1 --2- 172.21.15.120:0/2317024897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e132d7b60 0x556e132d7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:54.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.029+0000 7fe027fff640 1 --2- 172.21.15.120:0/2317024897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e132d8480 0x556e132e2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:54.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.029+0000 7fe035197640 1 -- 172.21.15.120:0/2317024897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e132d7b60 msgr2=0x556e132d7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:54.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.029+0000 7fe035197640 1 --2- 172.21.15.120:0/2317024897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e132d7b60 0x556e132d7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.029+0000 7fe035197640 1 -- 172.21.15.120:0/2317024897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e132d8480 msgr2=0x556e132e2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:54.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.029+0000 7fe035197640 1 --2- 172.21.15.120:0/2317024897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e132d8480 0x556e132e2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.029+0000 7fe035197640 1 -- 172.21.15.120:0/2317024897 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e131f11f0 con 0x556e132e2d40 2024-09-19T09:15:54.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.029+0000 7fe027fff640 1 --2- 172.21.15.120:0/2317024897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e132d8480 0x556e132e2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:15:54.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.029+0000 7fe035197640 1 --2- 172.21.15.120:0/1260095331 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe0180a3240 0x7fe0180a5700 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe02800ec60 tx=0x7fe028009e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:54.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.029+0000 7fe035197640 1 --2- 172.21.15.120:0/2317024897 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e132e2d40 0x556e132e5130 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fe02800cfd0 tx=0x7fe028018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dffe3a69d77aa4b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:54.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.029+0000 7fe016ffd640 1 -- 172.21.15.120:0/2317024897 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe028020020 con 0x556e132e2d40 2024-09-19T09:15:54.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.029+0000 7fe016ffd640 1 -- 172.21.15.120:0/2317024897 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe02800e510 con 0x556e132e2d40 2024-09-19T09:15:54.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.029+0000 7fe016ffd640 1 -- 172.21.15.120:0/2317024897 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02801a3f0 con 0x556e132e2d40 2024-09-19T09:15:54.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.030+0000 7fe038fd3740 1 -- 172.21.15.120:0/2317024897 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e132e2d40 msgr2=0x556e132e5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:54.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.030+0000 7fe038fd3740 1 --2- 172.21.15.120:0/2317024897 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e132e2d40 0x556e132e5130 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fe02800cfd0 tx=0x7fe028018000 comp rx=0 tx=0).stop 2024-09-19T09:15:54.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.030+0000 7fe038fd3740 1 -- 172.21.15.120:0/2317024897 shutdown_connections 2024-09-19T09:15:54.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.030+0000 7fe038fd3740 1 --2- 172.21.15.120:0/2317024897 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e132e2d40 0x556e132e5130 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.030+0000 7fe038fd3740 1 --2- 172.21.15.120:0/2317024897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e132d8480 0x556e132e2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.030+0000 7fe038fd3740 1 --2- 172.21.15.120:0/2317024897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e132d7b60 0x556e132d7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.030+0000 7fe038fd3740 1 -- 172.21.15.120:0/2317024897 >> 172.21.15.120:0/2317024897 conn(0x556e132d40f0 msgr2=0x556e132d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:54.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.030+0000 7fe038fd3740 1 -- 172.21.15.120:0/2317024897 shutdown_connections 2024-09-19T09:15:54.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.030+0000 7fe038fd3740 1 -- 172.21.15.120:0/2317024897 wait complete. 2024-09-19T09:15:54.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.030+0000 7fe038fd3740 1 -- start start 2024-09-19T09:15:54.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.030+0000 7fe038fd3740 1 Processor -- start 2024-09-19T09:15:54.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.031+0000 7fe038fd3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e132d7b60 0x556e132dfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:54.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.031+0000 7fe038fd3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e132d8480 0x556e132e0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:54.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.031+0000 7fe038fd3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e132e2d40 0x556e132e0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:54.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.031+0000 7fe038fd3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556e132e5910 con 0x556e132d8480 2024-09-19T09:15:54.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.031+0000 7fe038fd3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556e131f7f10 con 0x556e132e2d40 2024-09-19T09:15:54.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.031+0000 7fe038fd3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556e132e5a90 con 0x556e132d7b60 2024-09-19T09:15:54.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.031+0000 7fe038fd3740 10 client.0 _ll_register_callbacks cb 0x556e131c0a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:54.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.031+0000 7fe027fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e132d8480 0x556e132e0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:54.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.031+0000 7fe027fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e132d8480 0x556e132e0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36528/0 (socket says 172.21.15.120:36528) 2024-09-19T09:15:54.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.031+0000 7fe027fff640 1 -- 172.21.15.120:0/696582409 learned_addr learned my addr 172.21.15.120:0/696582409 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:54.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.031+0000 7fe034996640 1 --2- 172.21.15.120:0/696582409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e132d7b60 0x556e132dfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:54.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.031+0000 7fe035197640 1 --2- 172.21.15.120:0/696582409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e132e2d40 0x556e132e0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:54.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.031+0000 7fe027fff640 1 -- 172.21.15.120:0/696582409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e132d7b60 msgr2=0x556e132dfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:54.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.031+0000 7fe027fff640 1 --2- 172.21.15.120:0/696582409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e132d7b60 0x556e132dfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.031+0000 7fe027fff640 1 -- 172.21.15.120:0/696582409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e132e2d40 msgr2=0x556e132e0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:54.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.031+0000 7fe027fff640 1 --2- 172.21.15.120:0/696582409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e132e2d40 0x556e132e0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.031+0000 7fe027fff640 1 -- 172.21.15.120:0/696582409 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e12fae6a0 con 0x556e132d8480 2024-09-19T09:15:54.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.031+0000 7fe027fff640 1 --2- 172.21.15.120:0/696582409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e132d8480 0x556e132e0140 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fe02009c0e0 tx=0x7fe02000fbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aaadd276490e9fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:54.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.031+0000 7fe00ffff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:15:54.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.031+0000 7fe00ffff640 1 -- 172.21.15.120:0/696582409 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02009c850 con 0x556e132d8480 2024-09-19T09:15:54.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.032+0000 7fe00ffff640 1 -- 172.21.15.120:0/696582409 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fe0200a0170 con 0x556e132d8480 2024-09-19T09:15:54.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.032+0000 7fe038fd3740 1 -- 172.21.15.120:0/696582409 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe02800e510 con 0x556e132d8480 2024-09-19T09:15:54.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.032+0000 7fe00ffff640 1 -- 172.21.15.120:0/696582409 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe02000d520 con 0x556e132d8480 2024-09-19T09:15:54.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.032+0000 7fe038fd3740 10 client.20001 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:54.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.032+0000 7fe038fd3740 1 -- 172.21.15.120:0/696582409 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556e132f14b0 con 0x556e132d8480 2024-09-19T09:15:54.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.032+0000 7fe00ffff640 1 -- 172.21.15.120:0/696582409 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe02800e510 con 0x556e132d8480 2024-09-19T09:15:54.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.032+0000 7fe00ffff640 1 -- 172.21.15.120:0/696582409 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fe0200a4930 con 0x556e132d8480 2024-09-19T09:15:54.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.032+0000 7fe038fd3740 10 client.20001 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:54.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.032+0000 7fe038fd3740 10 client.20001 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:15:54.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.032+0000 7fe038fd3740 20 client.20001 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:54.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.032+0000 7fe038fd3740 1 --2- 172.21.15.120:0/696582409 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556e132f2910 0x556e132f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:54.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.032+0000 7fe038fd3740 4 client.20001 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:15:54.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.032+0000 7fe038fd3740 1 -- 172.21.15.120:0/696582409 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e1324a7f0 con 0x556e132f2910 2024-09-19T09:15:54.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.032+0000 7fe034996640 1 --2- 172.21.15.120:0/696582409 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556e132f2910 0x556e132f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:54.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.033+0000 7fe034996640 1 --2- 172.21.15.120:0/696582409 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556e132f2910 0x556e132f4d20 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88d25be931d9243d server_cookie=eb4210d14fb792dd in_seq=0 out_seq=0 2024-09-19T09:15:54.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.033+0000 7fe00ffff640 10 client.20001 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:54.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.033+0000 7fe00ffff640 1 -- 172.21.15.120:0/696582409 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556e1324a7f0 con 0x556e132f2910 2024-09-19T09:15:54.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.034+0000 7fe00ffff640 10 client.20001 handle_command_reply: tid=0 2024-09-19T09:15:54.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.034+0000 7fe038fd3740 1 client.20001 shutdown 2024-09-19T09:15:54.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.034+0000 7fe038fd3740 1 -- 172.21.15.120:0/696582409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e132d8480 msgr2=0x556e132e0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:54.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.034+0000 7fe038fd3740 1 --2- 172.21.15.120:0/696582409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e132d8480 0x556e132e0140 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fe02009c0e0 tx=0x7fe02000fbf0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.034+0000 7fe038fd3740 1 -- 172.21.15.120:0/696582409 shutdown_connections 2024-09-19T09:15:54.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.034+0000 7fe038fd3740 1 --2- 172.21.15.120:0/696582409 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556e132f2910 0x556e132f4d20 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.034+0000 7fe038fd3740 1 --2- 172.21.15.120:0/696582409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e132e2d40 0x556e132e0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.034+0000 7fe038fd3740 1 --2- 172.21.15.120:0/696582409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e132d8480 0x556e132e0140 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.034+0000 7fe038fd3740 1 --2- 172.21.15.120:0/696582409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e132d7b60 0x556e132dfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.034+0000 7fe00ffff640 0 client.20001 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:54.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.034+0000 7fe038fd3740 1 -- 172.21.15.120:0/696582409 >> 172.21.15.120:0/696582409 conn(0x556e132d40f0 msgr2=0x556e132d9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:54.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.034+0000 7fe038fd3740 1 -- 172.21.15.120:0/696582409 shutdown_connections 2024-09-19T09:15:54.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.034+0000 7fe038fd3740 1 -- 172.21.15.120:0/696582409 wait complete. 2024-09-19T09:15:54.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.034+0000 7fe038fd3740 20 client.20001 trim_cache size 0 max 16384 2024-09-19T09:15:54.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.052+0000 7fe038fd3740 1 Processor -- start 2024-09-19T09:15:54.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.052+0000 7fe038fd3740 1 -- start start 2024-09-19T09:15:54.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.052+0000 7fe038fd3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e132d7b60 0x556e132deab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:54.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.052+0000 7fe038fd3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e132d8480 0x556e132fa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:54.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.052+0000 7fe038fd3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e132e2d40 0x556e132fab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:54.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.052+0000 7fe038fd3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe0301ad5e0 con 0x556e132e2d40 2024-09-19T09:15:54.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.052+0000 7fe038fd3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe0301af120 con 0x556e132d7b60 2024-09-19T09:15:54.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.052+0000 7fe038fd3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe0301adcb0 con 0x556e132d8480 2024-09-19T09:15:54.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.052+0000 7fe035197640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e132e2d40 0x556e132fab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:54.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.052+0000 7fe035197640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e132e2d40 0x556e132fab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36530/0 (socket says 172.21.15.120:36530) 2024-09-19T09:15:54.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.052+0000 7fe035197640 1 -- 172.21.15.120:0/2836871961 learned_addr learned my addr 172.21.15.120:0/2836871961 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:54.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.052+0000 7fe034996640 1 --2- 172.21.15.120:0/2836871961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e132d7b60 0x556e132deab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:54.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.052+0000 7fe027fff640 1 --2- 172.21.15.120:0/2836871961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e132d8480 0x556e132fa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:54.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.052+0000 7fe035197640 1 -- 172.21.15.120:0/2836871961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e132d8480 msgr2=0x556e132fa5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:54.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.052+0000 7fe035197640 1 --2- 172.21.15.120:0/2836871961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e132d8480 0x556e132fa5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.052+0000 7fe035197640 1 -- 172.21.15.120:0/2836871961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e132d7b60 msgr2=0x556e132deab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:54.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.052+0000 7fe035197640 1 --2- 172.21.15.120:0/2836871961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e132d7b60 0x556e132deab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.052+0000 7fe035197640 1 -- 172.21.15.120:0/2836871961 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e132cf830 con 0x556e132e2d40 2024-09-19T09:15:54.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.052+0000 7fe027fff640 1 --2- 172.21.15.120:0/2836871961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e132d8480 0x556e132fa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:15:54.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.052+0000 7fe034996640 1 --2- 172.21.15.120:0/2836871961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e132d7b60 0x556e132deab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:15:54.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.052+0000 7fe035197640 1 --2- 172.21.15.120:0/2836871961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e132e2d40 0x556e132fab30 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fe02800cfd0 tx=0x7fe02800e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e52c87ae086ae60 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:54.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.053+0000 7fe0167fc640 1 -- 172.21.15.120:0/2836871961 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02802c070 con 0x556e132e2d40 2024-09-19T09:15:54.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.053+0000 7fe0167fc640 1 -- 172.21.15.120:0/2836871961 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe028025450 con 0x556e132e2d40 2024-09-19T09:15:54.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.053+0000 7fe0167fc640 1 -- 172.21.15.120:0/2836871961 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe028020040 con 0x556e132e2d40 2024-09-19T09:15:54.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.053+0000 7fe038fd3740 1 -- 172.21.15.120:0/2836871961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e132e2d40 msgr2=0x556e132fab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:54.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.053+0000 7fe038fd3740 1 --2- 172.21.15.120:0/2836871961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e132e2d40 0x556e132fab30 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fe02800cfd0 tx=0x7fe02800e770 comp rx=0 tx=0).stop 2024-09-19T09:15:54.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.053+0000 7fe038fd3740 1 -- 172.21.15.120:0/2836871961 shutdown_connections 2024-09-19T09:15:54.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.053+0000 7fe038fd3740 1 --2- 172.21.15.120:0/2836871961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e132e2d40 0x556e132fab30 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.053+0000 7fe038fd3740 1 --2- 172.21.15.120:0/2836871961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e132d8480 0x556e132fa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.053+0000 7fe038fd3740 1 --2- 172.21.15.120:0/2836871961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e132d7b60 0x556e132deab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.053+0000 7fe038fd3740 1 -- 172.21.15.120:0/2836871961 >> 172.21.15.120:0/2836871961 conn(0x556e132d40f0 msgr2=0x556e132de3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:54.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.053+0000 7fe038fd3740 1 -- 172.21.15.120:0/2836871961 shutdown_connections 2024-09-19T09:15:54.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.053+0000 7fe038fd3740 1 -- 172.21.15.120:0/2836871961 wait complete. 2024-09-19T09:15:54.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.054+0000 7fe038fd3740 1 -- start start 2024-09-19T09:15:54.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.054+0000 7fe038fd3740 1 Processor -- start 2024-09-19T09:15:54.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.054+0000 7fe038fd3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e132d7b60 0x556e132dcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:54.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.054+0000 7fe038fd3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e132d8480 0x556e132dd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:54.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.054+0000 7fe038fd3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e132e2d40 0x556e132dd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:54.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.054+0000 7fe038fd3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe0301aea50 con 0x556e132d7b60 2024-09-19T09:15:54.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.054+0000 7fe038fd3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe0301ae380 con 0x556e132d8480 2024-09-19T09:15:54.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.054+0000 7fe038fd3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe0301ab5a0 con 0x556e132e2d40 2024-09-19T09:15:54.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.054+0000 7fe038fd3740 10 client.0 _ll_register_callbacks cb 0x7fe02000cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:54.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.054+0000 7fe034996640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e132d7b60 0x556e132dcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:54.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.054+0000 7fe034996640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e132d7b60 0x556e132dcb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36538/0 (socket says 172.21.15.120:36538) 2024-09-19T09:15:54.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.054+0000 7fe034996640 1 -- 172.21.15.120:0/1778596395 learned_addr learned my addr 172.21.15.120:0/1778596395 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:54.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.055+0000 7fe035197640 1 --2- 172.21.15.120:0/1778596395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e132e2d40 0x556e132dd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:54.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.055+0000 7fe027fff640 1 --2- 172.21.15.120:0/1778596395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e132d8480 0x556e132dd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:54.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.055+0000 7fe034996640 1 -- 172.21.15.120:0/1778596395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e132e2d40 msgr2=0x556e132dd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:54.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.055+0000 7fe034996640 1 --2- 172.21.15.120:0/1778596395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e132e2d40 0x556e132dd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.055+0000 7fe034996640 1 -- 172.21.15.120:0/1778596395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e132d8480 msgr2=0x556e132dd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:54.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.055+0000 7fe034996640 1 --2- 172.21.15.120:0/1778596395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e132d8480 0x556e132dd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.055+0000 7fe034996640 1 -- 172.21.15.120:0/1778596395 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e132d82b0 con 0x556e132d7b60 2024-09-19T09:15:54.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.055+0000 7fe035197640 1 --2- 172.21.15.120:0/1778596395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e132e2d40 0x556e132dd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:15:54.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.055+0000 7fe027fff640 1 --2- 172.21.15.120:0/1778596395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e132d8480 0x556e132dd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:15:54.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.055+0000 7fe034996640 1 --2- 172.21.15.120:0/1778596395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e132d7b60 0x556e132dcb40 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fe02c02a940 tx=0x7fe02c031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a1f8d955293e12b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:54.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.055+0000 7fe00ffff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:15:54.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.055+0000 7fe00ffff640 1 -- 172.21.15.120:0/1778596395 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c04c020 con 0x556e132d7b60 2024-09-19T09:15:54.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.055+0000 7fe00ffff640 1 -- 172.21.15.120:0/1778596395 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fe02c005040 con 0x556e132d7b60 2024-09-19T09:15:54.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.055+0000 7fe038fd3740 1 -- 172.21.15.120:0/1778596395 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe028025450 con 0x556e132d7b60 2024-09-19T09:15:54.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.055+0000 7fe00ffff640 1 -- 172.21.15.120:0/1778596395 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe02c035040 con 0x556e132d7b60 2024-09-19T09:15:54.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.055+0000 7fe038fd3740 10 client.20007 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:54.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.055+0000 7fe038fd3740 1 -- 172.21.15.120:0/1778596395 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556e1335d5a0 con 0x556e132d7b60 2024-09-19T09:15:54.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.055+0000 7fe00ffff640 1 -- 172.21.15.120:0/1778596395 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe028025450 con 0x556e132d7b60 2024-09-19T09:15:54.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.055+0000 7fe00ffff640 1 -- 172.21.15.120:0/1778596395 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fe02c0035f0 con 0x556e132d7b60 2024-09-19T09:15:54.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.056+0000 7fe038fd3740 10 client.20007 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:54.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.056+0000 7fe038fd3740 10 client.20007 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:15:54.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.056+0000 7fe038fd3740 20 client.20007 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:54.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.056+0000 7fe038fd3740 1 --2- 172.21.15.120:0/1778596395 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556e1335dd20 0x556e13360140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:54.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.056+0000 7fe038fd3740 4 client.20007 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:15:54.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.056+0000 7fe038fd3740 1 -- 172.21.15.120:0/1778596395 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556e13244d00 con 0x556e1335dd20 2024-09-19T09:15:54.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.056+0000 7fe027fff640 1 --2- 172.21.15.120:0/1778596395 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556e1335dd20 0x556e13360140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:54.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.056+0000 7fe00ffff640 10 client.20007 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:54.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.056+0000 7fe027fff640 1 --2- 172.21.15.120:0/1778596395 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556e1335dd20 0x556e13360140 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d23dc668626eb24 server_cookie=39690696d6bbead6 in_seq=0 out_seq=0 2024-09-19T09:15:54.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.057+0000 7fe00ffff640 1 -- 172.21.15.120:0/1778596395 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556e13244d00 con 0x556e1335dd20 2024-09-19T09:15:54.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.057+0000 7fe00ffff640 10 client.20007 handle_command_reply: tid=0 2024-09-19T09:15:54.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.057+0000 7fe038fd3740 1 client.20007 shutdown 2024-09-19T09:15:54.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.058+0000 7fe038fd3740 1 -- 172.21.15.120:0/1778596395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e132d7b60 msgr2=0x556e132dcb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:54.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.058+0000 7fe038fd3740 1 --2- 172.21.15.120:0/1778596395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e132d7b60 0x556e132dcb40 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fe02c02a940 tx=0x7fe02c031a20 comp rx=0 tx=0).stop 2024-09-19T09:15:54.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.058+0000 7fe038fd3740 1 -- 172.21.15.120:0/1778596395 shutdown_connections 2024-09-19T09:15:54.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.058+0000 7fe038fd3740 1 --2- 172.21.15.120:0/1778596395 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556e1335dd20 0x556e13360140 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.058+0000 7fe038fd3740 1 --2- 172.21.15.120:0/1778596395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e132e2d40 0x556e132dd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.058+0000 7fe038fd3740 1 --2- 172.21.15.120:0/1778596395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e132d8480 0x556e132dd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.058+0000 7fe038fd3740 1 --2- 172.21.15.120:0/1778596395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e132d7b60 0x556e132dcb40 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.058+0000 7fe038fd3740 1 -- 172.21.15.120:0/1778596395 >> 172.21.15.120:0/1778596395 conn(0x556e132d40f0 msgr2=0x556e132debc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:54.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.058+0000 7fe00ffff640 0 client.20007 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:54.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.058+0000 7fe038fd3740 1 -- 172.21.15.120:0/1778596395 shutdown_connections 2024-09-19T09:15:54.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.058+0000 7fe038fd3740 1 -- 172.21.15.120:0/1778596395 wait complete. 2024-09-19T09:15:54.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.058+0000 7fe038fd3740 20 client.20007 trim_cache size 0 max 16384 2024-09-19T09:15:54.215 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:15:54.216 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:15:54.216 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "e957f9ad-36d6-40e9-91e2-ec8cba3129b3", 2024-09-19T09:15:54.216 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:15:54.216 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:15:54.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.058+0000 7fe036bfa640 1 -- 172.21.15.120:0/1260095331 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe0180a3240 msgr2=0x7fe0180a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:54.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.058+0000 7fe036bfa640 1 --2- 172.21.15.120:0/1260095331 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe0180a3240 0x7fe0180a5700 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe02800ec60 tx=0x7fe028009e30 comp rx=0 tx=0).stop 2024-09-19T09:15:54.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.058+0000 7fe036bfa640 1 -- 172.21.15.120:0/1260095331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe030101420 msgr2=0x7fe030198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:54.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.058+0000 7fe036bfa640 1 --2- 172.21.15.120:0/1260095331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe030101420 0x7fe030198970 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fe02000c920 tx=0x7fe02000cde0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.059+0000 7fe036bfa640 1 -- 172.21.15.120:0/1260095331 shutdown_connections 2024-09-19T09:15:54.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.059+0000 7fe036bfa640 1 --2- 172.21.15.120:0/1260095331 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe0180a3240 0x7fe0180a5700 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.059+0000 7fe036bfa640 1 --2- 172.21.15.120:0/1260095331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe030106a80 0x7fe03019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.059+0000 7fe036bfa640 1 --2- 172.21.15.120:0/1260095331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe030101420 0x7fe030198970 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.059+0000 7fe036bfa640 1 --2- 172.21.15.120:0/1260095331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe030100a80 0x7fe030198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:54.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.059+0000 7fe036bfa640 1 -- 172.21.15.120:0/1260095331 >> 172.21.15.120:0/1260095331 conn(0x7fe0300fc7a0 msgr2=0x7fe03006a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:54.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.059+0000 7fe036bfa640 1 -- 172.21.15.120:0/1260095331 shutdown_connections 2024-09-19T09:15:54.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:54.059+0000 7fe036bfa640 1 -- 172.21.15.120:0/1260095331 wait complete. 2024-09-19T09:15:54.218 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1301014 9.88 MB/sec execute 589 sec latency 11015.593 ms 2024-09-19T09:15:55.078 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1301014 9.86 MB/sec execute 590 sec latency 12015.681 ms 2024-09-19T09:15:55.142 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:15:55.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:54 smithi196 ceph-mon[35150]: pgmap v735: 97 pgs: 97 active+clean; 39 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 212 op/s 2024-09-19T09:15:55.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:54 smithi153 ceph-mon[35071]: pgmap v735: 97 pgs: 97 active+clean; 39 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 212 op/s 2024-09-19T09:15:55.205 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:54 smithi120 ceph-mon[31150]: pgmap v735: 97 pgs: 97 active+clean; 39 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 212 op/s 2024-09-19T09:15:55.427 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:15:56.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.034+0000 7f7c3f68d640 1 -- 172.21.15.120:0/654992775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c381082a0 msgr2=0x7f7c3810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:56.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.034+0000 7f7c3f68d640 1 --2- 172.21.15.120:0/654992775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c381082a0 0x7f7c3810ec40 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009930 tx=0x7f7c3002a420 comp rx=0 tx=0).stop 2024-09-19T09:15:56.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.034+0000 7f7c3f68d640 1 -- 172.21.15.120:0/654992775 shutdown_connections 2024-09-19T09:15:56.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.034+0000 7f7c3f68d640 1 --2- 172.21.15.120:0/654992775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c381082a0 0x7f7c3810ec40 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.034+0000 7f7c3f68d640 1 --2- 172.21.15.120:0/654992775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c381036e0 0x7f7c38107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.034+0000 7f7c3f68d640 1 --2- 172.21.15.120:0/654992775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c38102d30 0x7f7c38103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.034+0000 7f7c3f68d640 1 -- 172.21.15.120:0/654992775 >> 172.21.15.120:0/654992775 conn(0x7f7c380fc7d0 msgr2=0x7f7c380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:56.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.034+0000 7f7c3f68d640 1 -- 172.21.15.120:0/654992775 shutdown_connections 2024-09-19T09:15:56.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.034+0000 7f7c3f68d640 1 -- 172.21.15.120:0/654992775 wait complete. 2024-09-19T09:15:56.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.034+0000 7f7c3f68d640 1 Processor -- start 2024-09-19T09:15:56.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.034+0000 7f7c3f68d640 1 -- start start 2024-09-19T09:15:56.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.035+0000 7f7c3f68d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c38102d30 0x7f7c38198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:56.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.035+0000 7f7c3f68d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c381036e0 0x7f7c38198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:56.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.035+0000 7f7c3f68d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c381082a0 0x7f7c3819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:56.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.035+0000 7f7c3f68d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7c381120a0 con 0x7f7c38102d30 2024-09-19T09:15:56.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.035+0000 7f7c3f68d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7c38111f20 con 0x7f7c381036e0 2024-09-19T09:15:56.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.035+0000 7f7c3f68d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7c38112220 con 0x7f7c381082a0 2024-09-19T09:15:56.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.035+0000 7f7c3d429640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c38102d30 0x7f7c38198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:56.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.035+0000 7f7c3d429640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c38102d30 0x7f7c38198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36566/0 (socket says 172.21.15.120:36566) 2024-09-19T09:15:56.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.035+0000 7f7c3d429640 1 -- 172.21.15.120:0/500880561 learned_addr learned my addr 172.21.15.120:0/500880561 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:56.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.035+0000 7f7c3cc28640 1 --2- 172.21.15.120:0/500880561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c381036e0 0x7f7c38198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:56.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.035+0000 7f7c3dc2a640 1 --2- 172.21.15.120:0/500880561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c381082a0 0x7f7c3819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:56.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.035+0000 7f7c3d429640 1 -- 172.21.15.120:0/500880561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c381082a0 msgr2=0x7f7c3819ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:56.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.035+0000 7f7c3d429640 1 --2- 172.21.15.120:0/500880561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c381082a0 0x7f7c3819ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.035+0000 7f7c3d429640 1 -- 172.21.15.120:0/500880561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c381036e0 msgr2=0x7f7c38198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:56.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.035+0000 7f7c3d429640 1 --2- 172.21.15.120:0/500880561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c381036e0 0x7f7c38198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.035+0000 7f7c3d429640 1 -- 172.21.15.120:0/500880561 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3819d510 con 0x7f7c38102d30 2024-09-19T09:15:56.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.035+0000 7f7c3d429640 1 --2- 172.21.15.120:0/500880561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c38102d30 0x7f7c38198560 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800eab0 tx=0x7f7c2800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:56.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.036+0000 7f7c367fc640 1 -- 172.21.15.120:0/500880561 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2800ce40 con 0x7f7c38102d30 2024-09-19T09:15:56.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.036+0000 7f7c3f68d640 1 -- 172.21.15.120:0/500880561 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c3819d800 con 0x7f7c38102d30 2024-09-19T09:15:56.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.036+0000 7f7c367fc640 1 -- 172.21.15.120:0/500880561 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c28011950 con 0x7f7c38102d30 2024-09-19T09:15:56.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.036+0000 7f7c367fc640 1 -- 172.21.15.120:0/500880561 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c28010ae0 con 0x7f7c38102d30 2024-09-19T09:15:56.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.036+0000 7f7c3f68d640 1 -- 172.21.15.120:0/500880561 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c381a50e0 con 0x7f7c38102d30 2024-09-19T09:15:56.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.039+0000 7f7c367fc640 1 -- 172.21.15.120:0/500880561 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f7c28011b00 con 0x7f7c38102d30 2024-09-19T09:15:56.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.039+0000 7f7c41a66740 1 Processor -- start 2024-09-19T09:15:56.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.039+0000 7f7c41a66740 1 -- start start 2024-09-19T09:15:56.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.039+0000 7f7c41a66740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a9a8db360 0x562a9a8db740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:56.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.039+0000 7f7c41a66740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a9a8dbc80 0x562a9a8ea550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:56.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.039+0000 7f7c41a66740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a9a8eaa90 0x562a9a8ece80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:56.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.039+0000 7f7c41a66740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562a9a8412e0 con 0x562a9a8db360 2024-09-19T09:15:56.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.039+0000 7f7c41a66740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562a9a7e8ce0 con 0x562a9a8dbc80 2024-09-19T09:15:56.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.039+0000 7f7c41a66740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562a9a83d2a0 con 0x562a9a8eaa90 2024-09-19T09:15:56.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.039+0000 7f7c3dc2a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a9a8db360 0x562a9a8db740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:56.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.039+0000 7f7c3dc2a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a9a8db360 0x562a9a8db740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36568/0 (socket says 172.21.15.120:36568) 2024-09-19T09:15:56.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.039+0000 7f7c3dc2a640 1 -- 172.21.15.120:0/4044128333 learned_addr learned my addr 172.21.15.120:0/4044128333 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:56.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.039+0000 7f7c367fc640 1 --2- 172.21.15.120:0/500880561 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7c200a3240 0x7f7c200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:56.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.039+0000 7f7c3d429640 1 --2- 172.21.15.120:0/4044128333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a9a8dbc80 0x562a9a8ea550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:56.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.039+0000 7f7c3cc28640 1 --2- 172.21.15.120:0/4044128333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a9a8eaa90 0x562a9a8ece80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:56.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.039+0000 7f7c3dc2a640 1 --2- 172.21.15.120:0/500880561 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7c200a3240 0x7f7c200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:56.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.040+0000 7f7c367fc640 1 -- 172.21.15.120:0/500880561 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f7c280d0000 con 0x7f7c38102d30 2024-09-19T09:15:56.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.040+0000 7f7c3dc2a640 1 -- 172.21.15.120:0/4044128333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a9a8eaa90 msgr2=0x562a9a8ece80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:56.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.040+0000 7f7c3dc2a640 1 --2- 172.21.15.120:0/4044128333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a9a8eaa90 0x562a9a8ece80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.040+0000 7f7c3dc2a640 1 -- 172.21.15.120:0/4044128333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a9a8dbc80 msgr2=0x562a9a8ea550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:56.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.040+0000 7f7c3dc2a640 1 --2- 172.21.15.120:0/4044128333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a9a8dbc80 0x562a9a8ea550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.040+0000 7f7c3dc2a640 1 -- 172.21.15.120:0/4044128333 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a9a7cd440 con 0x562a9a8db360 2024-09-19T09:15:56.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.040+0000 7f7c3dc2a640 1 --2- 172.21.15.120:0/4044128333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a9a8db360 0x562a9a8db740 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f7c30005940 tx=0x7f7c30031760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4fe998dfa2ec37b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:56.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.040+0000 7f7c1f7fe640 1 -- 172.21.15.120:0/4044128333 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3004c070 con 0x562a9a8db360 2024-09-19T09:15:56.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.040+0000 7f7c1f7fe640 1 -- 172.21.15.120:0/4044128333 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c300044c0 con 0x562a9a8db360 2024-09-19T09:15:56.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.040+0000 7f7c3dc2a640 1 --2- 172.21.15.120:0/500880561 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7c200a3240 0x7f7c200a5700 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7c30005ec0 tx=0x7f7c30047000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:56.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.040+0000 7f7c1f7fe640 1 -- 172.21.15.120:0/4044128333 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30035040 con 0x562a9a8db360 2024-09-19T09:15:56.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.041+0000 7f7c41a66740 1 -- 172.21.15.120:0/4044128333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a9a8db360 msgr2=0x562a9a8db740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:56.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.041+0000 7f7c41a66740 1 --2- 172.21.15.120:0/4044128333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a9a8db360 0x562a9a8db740 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f7c30005940 tx=0x7f7c30031760 comp rx=0 tx=0).stop 2024-09-19T09:15:56.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.041+0000 7f7c41a66740 1 -- 172.21.15.120:0/4044128333 shutdown_connections 2024-09-19T09:15:56.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.041+0000 7f7c41a66740 1 --2- 172.21.15.120:0/4044128333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a9a8eaa90 0x562a9a8ece80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.041+0000 7f7c41a66740 1 --2- 172.21.15.120:0/4044128333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a9a8dbc80 0x562a9a8ea550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.041+0000 7f7c41a66740 1 --2- 172.21.15.120:0/4044128333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a9a8db360 0x562a9a8db740 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.041+0000 7f7c41a66740 1 -- 172.21.15.120:0/4044128333 >> 172.21.15.120:0/4044128333 conn(0x562a9a8d7a30 msgr2=0x562a9a8d7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:56.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.041+0000 7f7c41a66740 1 -- 172.21.15.120:0/4044128333 shutdown_connections 2024-09-19T09:15:56.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.041+0000 7f7c41a66740 1 -- 172.21.15.120:0/4044128333 wait complete. 2024-09-19T09:15:56.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.041+0000 7f7c41a66740 1 -- start start 2024-09-19T09:15:56.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.041+0000 7f7c41a66740 1 Processor -- start 2024-09-19T09:15:56.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.042+0000 7f7c41a66740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a9a8db360 0x562a9a8ea470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:56.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.042+0000 7f7c41a66740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a9a8dbc80 0x562a9a8e7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:56.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.042+0000 7f7c41a66740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a9a8eaa90 0x562a9a8e7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:56.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.042+0000 7f7c41a66740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562a9a8ed660 con 0x562a9a8db360 2024-09-19T09:15:56.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.042+0000 7f7c41a66740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562a9a7fc050 con 0x562a9a8dbc80 2024-09-19T09:15:56.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.042+0000 7f7c41a66740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562a9a8ed7e0 con 0x562a9a8eaa90 2024-09-19T09:15:56.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.042+0000 7f7c41a66740 10 client.0 _ll_register_callbacks cb 0x562a9a83c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:56.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.042+0000 7f7c3d429640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a9a8db360 0x562a9a8ea470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:56.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.042+0000 7f7c3d429640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a9a8db360 0x562a9a8ea470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36572/0 (socket says 172.21.15.120:36572) 2024-09-19T09:15:56.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.042+0000 7f7c3d429640 1 -- 172.21.15.120:0/2168282132 learned_addr learned my addr 172.21.15.120:0/2168282132 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:56.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.042+0000 7f7c3cc28640 1 --2- 172.21.15.120:0/2168282132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a9a8dbc80 0x562a9a8e7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:56.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.042+0000 7f7c3dc2a640 1 --2- 172.21.15.120:0/2168282132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a9a8eaa90 0x562a9a8e7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:56.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.042+0000 7f7c3d429640 1 -- 172.21.15.120:0/2168282132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a9a8eaa90 msgr2=0x562a9a8e7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:56.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.042+0000 7f7c3d429640 1 --2- 172.21.15.120:0/2168282132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a9a8eaa90 0x562a9a8e7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.042+0000 7f7c3d429640 1 -- 172.21.15.120:0/2168282132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a9a8dbc80 msgr2=0x562a9a8e7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:56.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.042+0000 7f7c3d429640 1 --2- 172.21.15.120:0/2168282132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a9a8dbc80 0x562a9a8e7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.042+0000 7f7c3d429640 1 -- 172.21.15.120:0/2168282132 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a9a5b23c0 con 0x562a9a8db360 2024-09-19T09:15:56.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.042+0000 7f7c3dc2a640 1 --2- 172.21.15.120:0/2168282132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a9a8eaa90 0x562a9a8e7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:15:56.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.042+0000 7f7c3d429640 1 --2- 172.21.15.120:0/2168282132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a9a8db360 0x562a9a8ea470 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f7c28098ff0 tx=0x7f7c280094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4884ce48ec6ddb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:56.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.042+0000 7f7c1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:15:56.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.042+0000 7f7c1cff9640 1 -- 172.21.15.120:0/2168282132 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c280a09d0 con 0x562a9a8db360 2024-09-19T09:15:56.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.042+0000 7f7c1cff9640 1 -- 172.21.15.120:0/2168282132 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f7c2809c880 con 0x562a9a8db360 2024-09-19T09:15:56.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.042+0000 7f7c41a66740 1 -- 172.21.15.120:0/2168282132 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562a9a84e850 con 0x562a9a8db360 2024-09-19T09:15:56.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.042+0000 7f7c1cff9640 1 -- 172.21.15.120:0/2168282132 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7c280a5290 con 0x562a9a8db360 2024-09-19T09:15:56.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.043+0000 7f7c41a66740 10 client.20019 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:56.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.043+0000 7f7c41a66740 1 -- 172.21.15.120:0/2168282132 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562a9a8f9140 con 0x562a9a8db360 2024-09-19T09:15:56.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.043+0000 7f7c1cff9640 1 -- 172.21.15.120:0/2168282132 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562a9a84e850 con 0x562a9a8db360 2024-09-19T09:15:56.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.043+0000 7f7c1cff9640 1 -- 172.21.15.120:0/2168282132 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f7c2800f040 con 0x562a9a8db360 2024-09-19T09:15:56.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.043+0000 7f7c41a66740 10 client.20019 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:56.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.043+0000 7f7c41a66740 10 client.20019 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:15:56.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.043+0000 7f7c41a66740 20 client.20019 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:56.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.043+0000 7f7c41a66740 1 --2- 172.21.15.120:0/2168282132 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562a9a8fa5a0 0x562a9a8fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:56.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.043+0000 7f7c41a66740 4 client.20019 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:15:56.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.043+0000 7f7c41a66740 1 -- 172.21.15.120:0/2168282132 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a9a848d60 con 0x562a9a8fa5a0 2024-09-19T09:15:56.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.043+0000 7f7c3cc28640 1 --2- 172.21.15.120:0/2168282132 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562a9a8fa5a0 0x562a9a8fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:56.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.044+0000 7f7c3cc28640 1 --2- 172.21.15.120:0/2168282132 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562a9a8fa5a0 0x562a9a8fc9b0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8c26bb3e55308bd server_cookie=9c7ddd7324b689f4 in_seq=0 out_seq=0 2024-09-19T09:15:56.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.044+0000 7f7c1cff9640 10 client.20019 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:56.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.044+0000 7f7c1cff9640 1 -- 172.21.15.120:0/2168282132 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a9a848d60 con 0x562a9a8fa5a0 2024-09-19T09:15:56.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.044+0000 7f7c1cff9640 10 client.20019 handle_command_reply: tid=0 2024-09-19T09:15:56.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.044+0000 7f7c41a66740 1 client.20019 shutdown 2024-09-19T09:15:56.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.044+0000 7f7c41a66740 1 -- 172.21.15.120:0/2168282132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a9a8db360 msgr2=0x562a9a8ea470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:56.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.044+0000 7f7c41a66740 1 --2- 172.21.15.120:0/2168282132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a9a8db360 0x562a9a8ea470 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f7c28098ff0 tx=0x7f7c280094a0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.044+0000 7f7c41a66740 1 -- 172.21.15.120:0/2168282132 shutdown_connections 2024-09-19T09:15:56.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.044+0000 7f7c41a66740 1 --2- 172.21.15.120:0/2168282132 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562a9a8fa5a0 0x562a9a8fc9b0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.044+0000 7f7c41a66740 1 --2- 172.21.15.120:0/2168282132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a9a8eaa90 0x562a9a8e7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.044+0000 7f7c1cff9640 0 client.20019 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:56.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.044+0000 7f7c41a66740 1 --2- 172.21.15.120:0/2168282132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a9a8dbc80 0x562a9a8e7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.044+0000 7f7c41a66740 1 --2- 172.21.15.120:0/2168282132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a9a8db360 0x562a9a8ea470 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.044+0000 7f7c41a66740 1 -- 172.21.15.120:0/2168282132 >> 172.21.15.120:0/2168282132 conn(0x562a9a8d7a30 msgr2=0x562a9a8dd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:56.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.045+0000 7f7c41a66740 1 -- 172.21.15.120:0/2168282132 shutdown_connections 2024-09-19T09:15:56.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.045+0000 7f7c41a66740 1 -- 172.21.15.120:0/2168282132 wait complete. 2024-09-19T09:15:56.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.045+0000 7f7c41a66740 20 client.20019 trim_cache size 0 max 16384 2024-09-19T09:15:56.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.062+0000 7f7c41a66740 1 Processor -- start 2024-09-19T09:15:56.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.062+0000 7f7c41a66740 1 -- start start 2024-09-19T09:15:56.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.062+0000 7f7c41a66740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a9a8db360 0x562a9a8f6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:56.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.062+0000 7f7c41a66740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a9a8dbc80 0x562a9a902250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:56.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.062+0000 7f7c41a66740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a9a8eaa90 0x562a9a902790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:56.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.062+0000 7f7c41a66740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7c381ab480 con 0x562a9a8eaa90 2024-09-19T09:15:56.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.062+0000 7f7c41a66740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7c381acfc0 con 0x562a9a8db360 2024-09-19T09:15:56.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.062+0000 7f7c41a66740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7c381abb50 con 0x562a9a8dbc80 2024-09-19T09:15:56.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.062+0000 7f7c3dc2a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a9a8eaa90 0x562a9a902790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:56.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.062+0000 7f7c3dc2a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a9a8eaa90 0x562a9a902790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36578/0 (socket says 172.21.15.120:36578) 2024-09-19T09:15:56.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.062+0000 7f7c3dc2a640 1 -- 172.21.15.120:0/962895001 learned_addr learned my addr 172.21.15.120:0/962895001 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:56.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.062+0000 7f7c3d429640 1 --2- 172.21.15.120:0/962895001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a9a8db360 0x562a9a8f6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:56.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.062+0000 7f7c3cc28640 1 --2- 172.21.15.120:0/962895001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a9a8dbc80 0x562a9a902250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:56.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.062+0000 7f7c3dc2a640 1 -- 172.21.15.120:0/962895001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a9a8dbc80 msgr2=0x562a9a902250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:56.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.062+0000 7f7c3dc2a640 1 --2- 172.21.15.120:0/962895001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a9a8dbc80 0x562a9a902250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.062+0000 7f7c3dc2a640 1 -- 172.21.15.120:0/962895001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a9a8db360 msgr2=0x562a9a8f6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:56.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.062+0000 7f7c3dc2a640 1 --2- 172.21.15.120:0/962895001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a9a8db360 0x562a9a8f6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.062+0000 7f7c3dc2a640 1 -- 172.21.15.120:0/962895001 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a9a8d74d0 con 0x562a9a8eaa90 2024-09-19T09:15:56.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.062+0000 7f7c3d429640 1 --2- 172.21.15.120:0/962895001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a9a8db360 0x562a9a8f6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:15:56.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.062+0000 7f7c3dc2a640 1 --2- 172.21.15.120:0/962895001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a9a8eaa90 0x562a9a902790 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f7c30003780 tx=0x7f7c30031bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b3ad5979a4ffa87 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:56.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.062+0000 7f7c1effd640 1 -- 172.21.15.120:0/962895001 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30035070 con 0x562a9a8eaa90 2024-09-19T09:15:56.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.063+0000 7f7c1effd640 1 -- 172.21.15.120:0/962895001 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c30056030 con 0x562a9a8eaa90 2024-09-19T09:15:56.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.063+0000 7f7c1effd640 1 -- 172.21.15.120:0/962895001 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3004c040 con 0x562a9a8eaa90 2024-09-19T09:15:56.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.063+0000 7f7c41a66740 1 -- 172.21.15.120:0/962895001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a9a8eaa90 msgr2=0x562a9a902790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:56.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.063+0000 7f7c41a66740 1 --2- 172.21.15.120:0/962895001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a9a8eaa90 0x562a9a902790 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f7c30003780 tx=0x7f7c30031bb0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.063+0000 7f7c41a66740 1 -- 172.21.15.120:0/962895001 shutdown_connections 2024-09-19T09:15:56.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.063+0000 7f7c41a66740 1 --2- 172.21.15.120:0/962895001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a9a8eaa90 0x562a9a902790 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.063+0000 7f7c41a66740 1 --2- 172.21.15.120:0/962895001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a9a8dbc80 0x562a9a902250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.063+0000 7f7c41a66740 1 --2- 172.21.15.120:0/962895001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a9a8db360 0x562a9a8f6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.063+0000 7f7c41a66740 1 -- 172.21.15.120:0/962895001 >> 172.21.15.120:0/962895001 conn(0x562a9a8d7a30 msgr2=0x562a9a8e2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:56.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.063+0000 7f7c41a66740 1 -- 172.21.15.120:0/962895001 shutdown_connections 2024-09-19T09:15:56.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.063+0000 7f7c41a66740 1 -- 172.21.15.120:0/962895001 wait complete. 2024-09-19T09:15:56.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.064+0000 7f7c41a66740 1 -- start start 2024-09-19T09:15:56.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.064+0000 7f7c41a66740 1 Processor -- start 2024-09-19T09:15:56.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.064+0000 7f7c41a66740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a9a8db360 0x562a9a8e4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:56.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.064+0000 7f7c41a66740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a9a8dbc80 0x562a9a8e4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:56.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.064+0000 7f7c41a66740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a9a8eaa90 0x562a9a8e52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:56.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.064+0000 7f7c41a66740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7c381ac8f0 con 0x562a9a8eaa90 2024-09-19T09:15:56.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.064+0000 7f7c41a66740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7c381ac220 con 0x562a9a8dbc80 2024-09-19T09:15:56.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.064+0000 7f7c41a66740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7c381a9440 con 0x562a9a8db360 2024-09-19T09:15:56.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.064+0000 7f7c41a66740 10 client.0 _ll_register_callbacks cb 0x7f7c2809c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:15:56.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.064+0000 7f7c3dc2a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a9a8eaa90 0x562a9a8e52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:56.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.064+0000 7f7c3dc2a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a9a8eaa90 0x562a9a8e52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36584/0 (socket says 172.21.15.120:36584) 2024-09-19T09:15:56.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.064+0000 7f7c3dc2a640 1 -- 172.21.15.120:0/4111440385 learned_addr learned my addr 172.21.15.120:0/4111440385 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:15:56.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.064+0000 7f7c3d429640 1 --2- 172.21.15.120:0/4111440385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a9a8db360 0x562a9a8e4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:56.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.064+0000 7f7c3dc2a640 1 -- 172.21.15.120:0/4111440385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a9a8db360 msgr2=0x562a9a8e4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:56.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.064+0000 7f7c3dc2a640 1 --2- 172.21.15.120:0/4111440385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a9a8db360 0x562a9a8e4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.064+0000 7f7c3cc28640 1 --2- 172.21.15.120:0/4111440385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a9a8dbc80 0x562a9a8e4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:56.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.064+0000 7f7c3dc2a640 1 -- 172.21.15.120:0/4111440385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a9a8dbc80 msgr2=0x562a9a8e4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:15:56.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.064+0000 7f7c3dc2a640 1 --2- 172.21.15.120:0/4111440385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a9a8dbc80 0x562a9a8e4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.065+0000 7f7c3dc2a640 1 -- 172.21.15.120:0/4111440385 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a9a964270 con 0x562a9a8eaa90 2024-09-19T09:15:56.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.065+0000 7f7c3dc2a640 1 --2- 172.21.15.120:0/4111440385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a9a8eaa90 0x562a9a8e52c0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f7c3002ab10 tx=0x7f7c30005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f2a1130e8f9b87 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:15:56.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.065+0000 7f7c1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:15:56.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.065+0000 7f7c1cff9640 1 -- 172.21.15.120:0/4111440385 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30062070 con 0x562a9a8eaa90 2024-09-19T09:15:56.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.065+0000 7f7c1cff9640 1 -- 172.21.15.120:0/4111440385 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f7c30005050 con 0x562a9a8eaa90 2024-09-19T09:15:56.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.065+0000 7f7c41a66740 1 -- 172.21.15.120:0/4111440385 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c30056030 con 0x562a9a8eaa90 2024-09-19T09:15:56.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.065+0000 7f7c1cff9640 1 -- 172.21.15.120:0/4111440385 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7c3006e050 con 0x562a9a8eaa90 2024-09-19T09:15:56.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.065+0000 7f7c41a66740 10 client.20025 fetch_fsmap learned FSMap version 33 2024-09-19T09:15:56.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.065+0000 7f7c41a66740 1 -- 172.21.15.120:0/4111440385 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562a9a9654f0 con 0x562a9a8eaa90 2024-09-19T09:15:56.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.065+0000 7f7c1cff9640 1 -- 172.21.15.120:0/4111440385 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c30056030 con 0x562a9a8eaa90 2024-09-19T09:15:56.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.065+0000 7f7c1cff9640 1 -- 172.21.15.120:0/4111440385 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f7c30004a90 con 0x562a9a8eaa90 2024-09-19T09:15:56.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.065+0000 7f7c41a66740 10 client.20025 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:15:56.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.065+0000 7f7c41a66740 10 client.20025 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:15:56.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.065+0000 7f7c41a66740 20 client.20025 populate_metadata read hostname 'smithi120' 2024-09-19T09:15:56.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.065+0000 7f7c41a66740 1 --2- 172.21.15.120:0/4111440385 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562a9a9657e0 0x562a9a967c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:15:56.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.065+0000 7f7c41a66740 4 client.20025 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:15:56.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.065+0000 7f7c41a66740 1 -- 172.21.15.120:0/4111440385 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562a9a7c5210 con 0x562a9a9657e0 2024-09-19T09:15:56.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.066+0000 7f7c3d429640 1 --2- 172.21.15.120:0/4111440385 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562a9a9657e0 0x562a9a967c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:15:56.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.066+0000 7f7c3d429640 1 --2- 172.21.15.120:0/4111440385 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562a9a9657e0 0x562a9a967c00 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0cb687c11476090 server_cookie=84612d9afe6fd323 in_seq=0 out_seq=0 2024-09-19T09:15:56.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.066+0000 7f7c1cff9640 10 client.20025 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:56.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.066+0000 7f7c1cff9640 1 -- 172.21.15.120:0/4111440385 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562a9a7c5210 con 0x562a9a9657e0 2024-09-19T09:15:56.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.066+0000 7f7c1cff9640 10 client.20025 handle_command_reply: tid=0 2024-09-19T09:15:56.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.066+0000 7f7c41a66740 1 client.20025 shutdown 2024-09-19T09:15:56.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.066+0000 7f7c41a66740 1 -- 172.21.15.120:0/4111440385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a9a8eaa90 msgr2=0x562a9a8e52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:56.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.066+0000 7f7c41a66740 1 --2- 172.21.15.120:0/4111440385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a9a8eaa90 0x562a9a8e52c0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f7c3002ab10 tx=0x7f7c30005b50 comp rx=0 tx=0).stop 2024-09-19T09:15:56.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.066+0000 7f7c41a66740 1 -- 172.21.15.120:0/4111440385 shutdown_connections 2024-09-19T09:15:56.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.066+0000 7f7c41a66740 1 --2- 172.21.15.120:0/4111440385 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562a9a9657e0 0x562a9a967c00 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.066+0000 7f7c41a66740 1 --2- 172.21.15.120:0/4111440385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a9a8eaa90 0x562a9a8e52c0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.066+0000 7f7c41a66740 1 --2- 172.21.15.120:0/4111440385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a9a8dbc80 0x562a9a8e4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.066+0000 7f7c41a66740 1 --2- 172.21.15.120:0/4111440385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a9a8db360 0x562a9a8e4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.066+0000 7f7c41a66740 1 -- 172.21.15.120:0/4111440385 >> 172.21.15.120:0/4111440385 conn(0x562a9a8d7a30 msgr2=0x562a9a8d6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:56.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.066+0000 7f7c1cff9640 0 client.20025 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:15:56.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.067+0000 7f7c41a66740 1 -- 172.21.15.120:0/4111440385 shutdown_connections 2024-09-19T09:15:56.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.067+0000 7f7c41a66740 1 -- 172.21.15.120:0/4111440385 wait complete. 2024-09-19T09:15:56.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.067+0000 7f7c41a66740 20 client.20025 trim_cache size 0 max 16384 2024-09-19T09:15:56.074 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:15:56.074 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:15:56.074 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-19T09:15:56.074 INFO:teuthology.orchestra.run.smithi120.stdout: "e957f9ad-36d6-40e9-91e2-ec8cba3129b3": { 2024-09-19T09:15:56.074 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-19T09:15:56.074 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "e957f9ad-36d6-40e9-91e2-ec8cba3129b3", 2024-09-19T09:15:56.074 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-19T09:15:56.075 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:15:56.075 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:15:56.075 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:15:56.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.067+0000 7f7c3f68d640 1 -- 172.21.15.120:0/500880561 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7c200a3240 msgr2=0x7f7c200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:56.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.067+0000 7f7c3f68d640 1 --2- 172.21.15.120:0/500880561 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7c200a3240 0x7f7c200a5700 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7c30005ec0 tx=0x7f7c30047000 comp rx=0 tx=0).stop 2024-09-19T09:15:56.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.067+0000 7f7c3f68d640 1 -- 172.21.15.120:0/500880561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c38102d30 msgr2=0x7f7c38198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:15:56.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.067+0000 7f7c3f68d640 1 --2- 172.21.15.120:0/500880561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c38102d30 0x7f7c38198560 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800eab0 tx=0x7f7c2800ef70 comp rx=0 tx=0).stop 2024-09-19T09:15:56.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.067+0000 7f7c3f68d640 1 -- 172.21.15.120:0/500880561 shutdown_connections 2024-09-19T09:15:56.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.067+0000 7f7c3f68d640 1 --2- 172.21.15.120:0/500880561 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7c200a3240 0x7f7c200a5700 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.067+0000 7f7c3f68d640 1 --2- 172.21.15.120:0/500880561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c381082a0 0x7f7c3819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.067+0000 7f7c3f68d640 1 --2- 172.21.15.120:0/500880561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c381036e0 0x7f7c38198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.067+0000 7f7c3f68d640 1 --2- 172.21.15.120:0/500880561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c38102d30 0x7f7c38198560 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:15:56.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.067+0000 7f7c3f68d640 1 -- 172.21.15.120:0/500880561 >> 172.21.15.120:0/500880561 conn(0x7f7c380fc7d0 msgr2=0x7f7c38105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:15:56.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.067+0000 7f7c3f68d640 1 -- 172.21.15.120:0/500880561 shutdown_connections 2024-09-19T09:15:56.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:15:56.067+0000 7f7c3f68d640 1 -- 172.21.15.120:0/500880561 wait complete. 2024-09-19T09:15:56.078 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1301014 9.85 MB/sec execute 591 sec latency 13015.708 ms 2024-09-19T09:15:56.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:55 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:15:56.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:55 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:15:56.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:55 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:15:56.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:55 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:15:56.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:55 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:15:56.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:55 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:15:56.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:55 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:15:56.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:55 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:15:56.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:55 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:15:56.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:55 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:15:56.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:55 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:15:56.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:55 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:15:56.938 INFO:tasks.cephfs.filesystem:scrub status for tag:e957f9ad-36d6-40e9-91e2-ec8cba3129b3 - {'path': '/', 'tag': 'e957f9ad-36d6-40e9-91e2-ec8cba3129b3', 'options': 'recursive,force'} 2024-09-19T09:15:57.078 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1301014 9.83 MB/sec execute 592 sec latency 14015.772 ms 2024-09-19T09:15:57.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:56 smithi153 ceph-mon[35071]: pgmap v736: 97 pgs: 97 active+clean; 33 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-09-19T09:15:57.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:56 smithi120 ceph-mon[31150]: pgmap v736: 97 pgs: 97 active+clean; 33 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-09-19T09:15:57.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:56 smithi196 ceph-mon[35150]: pgmap v736: 97 pgs: 97 active+clean; 33 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-09-19T09:15:58.079 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1301014 9.81 MB/sec execute 593 sec latency 15015.872 ms 2024-09-19T09:15:59.079 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1301014 9.80 MB/sec execute 594 sec latency 16015.933 ms 2024-09-19T09:15:59.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:58 smithi120 ceph-mon[31150]: pgmap v737: 97 pgs: 97 active+clean; 37 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 2.2 MiB/s wr, 157 op/s 2024-09-19T09:15:59.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:15:58 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:15:59.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:58 smithi196 ceph-mon[35150]: pgmap v737: 97 pgs: 97 active+clean; 37 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 2.2 MiB/s wr, 157 op/s 2024-09-19T09:15:59.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:15:58 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:15:59.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:58 smithi153 ceph-mon[35071]: pgmap v737: 97 pgs: 97 active+clean; 37 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 2.2 MiB/s wr, 157 op/s 2024-09-19T09:15:59.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:15:58 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:16:00.079 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1301014 9.78 MB/sec execute 595 sec latency 17015.983 ms 2024-09-19T09:16:00.943 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:16:01.079 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1301014 9.76 MB/sec execute 596 sec latency 18016.049 ms 2024-09-19T09:16:01.202 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:16:01.239 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:00 smithi196 ceph-mon[35150]: pgmap v738: 97 pgs: 97 active+clean; 41 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 2.2 MiB/s wr, 143 op/s 2024-09-19T09:16:01.240 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:00 smithi153 ceph-mon[35071]: pgmap v738: 97 pgs: 97 active+clean; 41 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 2.2 MiB/s wr, 143 op/s 2024-09-19T09:16:01.323 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:00 smithi120 ceph-mon[31150]: pgmap v738: 97 pgs: 97 active+clean; 41 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 2.2 MiB/s wr, 143 op/s 2024-09-19T09:16:01.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.738+0000 7fd4fcd4a640 1 -- 172.21.15.120:0/1049533410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4f8102780 msgr2=0x7fd4f81067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:01.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.738+0000 7fd4fcd4a640 1 --2- 172.21.15.120:0/1049533410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4f8102780 0x7fd4f81067d0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009900 tx=0x7fd4e002a410 comp rx=0 tx=0).stop 2024-09-19T09:16:01.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.738+0000 7fd4fcd4a640 1 -- 172.21.15.120:0/1049533410 shutdown_connections 2024-09-19T09:16:01.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.738+0000 7fd4fcd4a640 1 --2- 172.21.15.120:0/1049533410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4f8106f00 0x7fd4f810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:01.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.738+0000 7fd4fcd4a640 1 --2- 172.21.15.120:0/1049533410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4f8102780 0x7fd4f81067d0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:01.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.738+0000 7fd4fcd4a640 1 --2- 172.21.15.120:0/1049533410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4f8101dd0 0x7fd4f81021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:01.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.738+0000 7fd4fcd4a640 1 -- 172.21.15.120:0/1049533410 >> 172.21.15.120:0/1049533410 conn(0x7fd4f80fd8b0 msgr2=0x7fd4f80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:01.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.738+0000 7fd4fcd4a640 1 -- 172.21.15.120:0/1049533410 shutdown_connections 2024-09-19T09:16:01.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.738+0000 7fd4fcd4a640 1 -- 172.21.15.120:0/1049533410 wait complete. 2024-09-19T09:16:01.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.739+0000 7fd4fcd4a640 1 Processor -- start 2024-09-19T09:16:01.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.739+0000 7fd4fcd4a640 1 -- start start 2024-09-19T09:16:01.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.739+0000 7fd4fcd4a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4f8101dd0 0x7fd4f80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:01.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.739+0000 7fd4fcd4a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4f8102780 0x7fd4f8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:01.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.739+0000 7fd4fcd4a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4f8106f00 0x7fd4f8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:01.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.739+0000 7fd4fcd4a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f8111f80 con 0x7fd4f8102780 2024-09-19T09:16:01.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.739+0000 7fd4fcd4a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f8111e00 con 0x7fd4f8106f00 2024-09-19T09:16:01.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.739+0000 7fd4fcd4a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f8112100 con 0x7fd4f8101dd0 2024-09-19T09:16:01.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.739+0000 7fd4f5d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4f8102780 0x7fd4f8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:01.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.739+0000 7fd4f5d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4f8102780 0x7fd4f8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54468/0 (socket says 172.21.15.120:54468) 2024-09-19T09:16:01.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.739+0000 7fd4f5d9b640 1 -- 172.21.15.120:0/4206345402 learned_addr learned my addr 172.21.15.120:0/4206345402 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:01.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.739+0000 7fd4f6d9d640 1 --2- 172.21.15.120:0/4206345402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4f8106f00 0x7fd4f8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:01.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.739+0000 7fd4f659c640 1 --2- 172.21.15.120:0/4206345402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4f8101dd0 0x7fd4f80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:01.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.739+0000 7fd4f5d9b640 1 -- 172.21.15.120:0/4206345402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4f8101dd0 msgr2=0x7fd4f80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:01.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.739+0000 7fd4f5d9b640 1 --2- 172.21.15.120:0/4206345402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4f8101dd0 0x7fd4f80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:01.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.739+0000 7fd4f5d9b640 1 -- 172.21.15.120:0/4206345402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4f8106f00 msgr2=0x7fd4f8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:01.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.739+0000 7fd4f5d9b640 1 --2- 172.21.15.120:0/4206345402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4f8106f00 0x7fd4f8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:01.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.739+0000 7fd4f5d9b640 1 -- 172.21.15.120:0/4206345402 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f81a68c0 con 0x7fd4f8102780 2024-09-19T09:16:01.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.740+0000 7fd4f5d9b640 1 --2- 172.21.15.120:0/4206345402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4f8102780 0x7fd4f8074a30 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0005ab0 tx=0x7fd4e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:01.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.740+0000 7fd4ef7fe640 1 -- 172.21.15.120:0/4206345402 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e003f030 con 0x7fd4f8102780 2024-09-19T09:16:01.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.740+0000 7fd4fcd4a640 1 -- 172.21.15.120:0/4206345402 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f81a6b50 con 0x7fd4f8102780 2024-09-19T09:16:01.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.740+0000 7fd4ef7fe640 1 -- 172.21.15.120:0/4206345402 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4e0003ce0 con 0x7fd4f8102780 2024-09-19T09:16:01.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.740+0000 7fd4ef7fe640 1 -- 172.21.15.120:0/4206345402 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0035070 con 0x7fd4f8102780 2024-09-19T09:16:01.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.740+0000 7fd4fcd4a640 1 -- 172.21.15.120:0/4206345402 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f81a7030 con 0x7fd4f8102780 2024-09-19T09:16:01.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.741+0000 7fd4fcd4a640 1 -- 172.21.15.120:0/4206345402 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4f81021b0 con 0x7fd4f8102780 2024-09-19T09:16:01.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.742+0000 7fd4ef7fe640 1 -- 172.21.15.120:0/4206345402 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd4e0042030 con 0x7fd4f8102780 2024-09-19T09:16:01.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.743+0000 7fd4ef7fe640 1 --2- 172.21.15.120:0/4206345402 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd4e40a3240 0x7fd4e40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:01.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.743+0000 7fd4f659c640 1 --2- 172.21.15.120:0/4206345402 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd4e40a3240 0x7fd4e40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:01.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.743+0000 7fd4ef7fe640 1 -- 172.21.15.120:0/4206345402 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fd4e00f0220 con 0x7fd4f8102780 2024-09-19T09:16:01.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.743+0000 7fd4f659c640 1 --2- 172.21.15.120:0/4206345402 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd4e40a3240 0x7fd4e40a5700 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0005e10 tx=0x7fd4f0005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:01.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.747+0000 7fd4ef7fe640 1 -- 172.21.15.120:0/4206345402 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd4e00bcf40 con 0x7fd4f8102780 2024-09-19T09:16:01.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.949+0000 7fd4fcd4a640 1 -- 172.21.15.120:0/4206345402 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4f810a0e0 con 0x7fd4f8102780 2024-09-19T09:16:01.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.951+0000 7fd4ef7fe640 1 -- 172.21.15.120:0/4206345402 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11988 (secure 0 0 0) 0x7fd4e00bc840 con 0x7fd4f8102780 2024-09-19T09:16:01.952 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:16:01.952 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":33,"btime":"2024-09-19T09:14:31:550518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:14:31.550494+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28649":{"gid":28649,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/2810080307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2810080307},{"type":"v1","addr":"172.21.15.153:6835","nonce":2810080307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:16:01.952 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 33 2024-09-19T09:16:01.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.953+0000 7fd4fcd4a640 1 -- 172.21.15.120:0/4206345402 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd4e40a3240 msgr2=0x7fd4e40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:01.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.953+0000 7fd4fcd4a640 1 --2- 172.21.15.120:0/4206345402 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd4e40a3240 0x7fd4e40a5700 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0005e10 tx=0x7fd4f0005da0 comp rx=0 tx=0).stop 2024-09-19T09:16:01.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.953+0000 7fd4fcd4a640 1 -- 172.21.15.120:0/4206345402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4f8102780 msgr2=0x7fd4f8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:01.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.953+0000 7fd4fcd4a640 1 --2- 172.21.15.120:0/4206345402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4f8102780 0x7fd4f8074a30 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0005ab0 tx=0x7fd4e00042f0 comp rx=0 tx=0).stop 2024-09-19T09:16:01.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.953+0000 7fd4fcd4a640 1 -- 172.21.15.120:0/4206345402 shutdown_connections 2024-09-19T09:16:01.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.953+0000 7fd4fcd4a640 1 --2- 172.21.15.120:0/4206345402 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd4e40a3240 0x7fd4e40a5700 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:01.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.953+0000 7fd4fcd4a640 1 --2- 172.21.15.120:0/4206345402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4f8106f00 0x7fd4f8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:01.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.953+0000 7fd4fcd4a640 1 --2- 172.21.15.120:0/4206345402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4f8102780 0x7fd4f8074a30 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:01.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.953+0000 7fd4fcd4a640 1 --2- 172.21.15.120:0/4206345402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4f8101dd0 0x7fd4f80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:01.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.953+0000 7fd4fcd4a640 1 -- 172.21.15.120:0/4206345402 >> 172.21.15.120:0/4206345402 conn(0x7fd4f80fd8b0 msgr2=0x7fd4f80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:01.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.953+0000 7fd4fcd4a640 1 -- 172.21.15.120:0/4206345402 shutdown_connections 2024-09-19T09:16:01.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:01.953+0000 7fd4fcd4a640 1 -- 172.21.15.120:0/4206345402 wait complete. 2024-09-19T09:16:02.079 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1301014 9.75 MB/sec execute 597 sec latency 19016.139 ms 2024-09-19T09:16:02.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:01 smithi120 ceph-mon[31150]: pgmap v739: 97 pgs: 97 active+clean; 42 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.2 MiB/s wr, 145 op/s 2024-09-19T09:16:02.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:01 smithi196 ceph-mon[35150]: pgmap v739: 97 pgs: 97 active+clean; 42 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.2 MiB/s wr, 145 op/s 2024-09-19T09:16:02.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:01 smithi153 ceph-mon[35071]: pgmap v739: 97 pgs: 97 active+clean; 42 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.2 MiB/s wr, 145 op/s 2024-09-19T09:16:02.776 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db --set-id ce305c23 --release --await' 2024-09-19T09:16:02.776 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db --set-id ce305c23 --release --await 2024-09-19T09:16:03.021 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:16:03.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:02 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/4206345402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:16:03.078 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1301014 9.73 MB/sec execute 598 sec latency 20016.165 ms 2024-09-19T09:16:03.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:02 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/4206345402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:16:03.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:02 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/4206345402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:16:03.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.547+0000 7f68bb4e8640 1 -- 172.21.15.120:0/2685640699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68b4102d30 msgr2=0x7f68b4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:03.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.547+0000 7f68bb4e8640 1 --2- 172.21.15.120:0/2685640699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68b4102d30 0x7f68b4103110 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009930 tx=0x7f68b002a440 comp rx=0 tx=0).stop 2024-09-19T09:16:03.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.547+0000 7f68bb4e8640 1 -- 172.21.15.120:0/2685640699 shutdown_connections 2024-09-19T09:16:03.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.547+0000 7f68bb4e8640 1 --2- 172.21.15.120:0/2685640699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68b41082a0 0x7f68b410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.547+0000 7f68bb4e8640 1 --2- 172.21.15.120:0/2685640699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68b41036e0 0x7f68b4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.547+0000 7f68bb4e8640 1 --2- 172.21.15.120:0/2685640699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68b4102d30 0x7f68b4103110 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.547+0000 7f68bb4e8640 1 -- 172.21.15.120:0/2685640699 >> 172.21.15.120:0/2685640699 conn(0x7f68b40fc7d0 msgr2=0x7f68b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:03.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.547+0000 7f68bb4e8640 1 -- 172.21.15.120:0/2685640699 shutdown_connections 2024-09-19T09:16:03.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.547+0000 7f68bb4e8640 1 -- 172.21.15.120:0/2685640699 wait complete. 2024-09-19T09:16:03.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.547+0000 7f68bb4e8640 1 Processor -- start 2024-09-19T09:16:03.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.547+0000 7f68bb4e8640 1 -- start start 2024-09-19T09:16:03.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.548+0000 7f68bb4e8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68b4102d30 0x7f68b4198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:03.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.548+0000 7f68bb4e8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68b41036e0 0x7f68b4198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:03.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.548+0000 7f68bb4e8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68b41082a0 0x7f68b419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:03.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.548+0000 7f68bb4e8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41120e0 con 0x7f68b4102d30 2024-09-19T09:16:03.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.548+0000 7f68bb4e8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f68b4111f60 con 0x7f68b41082a0 2024-09-19T09:16:03.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.548+0000 7f68bb4e8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f68b4112260 con 0x7f68b41036e0 2024-09-19T09:16:03.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.548+0000 7f68b9284640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68b4102d30 0x7f68b4198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:03.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.548+0000 7f68b9284640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68b4102d30 0x7f68b4198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54486/0 (socket says 172.21.15.120:54486) 2024-09-19T09:16:03.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.548+0000 7f68b9284640 1 -- 172.21.15.120:0/2387864445 learned_addr learned my addr 172.21.15.120:0/2387864445 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:03.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.548+0000 7f68b8a83640 1 --2- 172.21.15.120:0/2387864445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68b41036e0 0x7f68b4198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:03.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.548+0000 7f68b9284640 1 -- 172.21.15.120:0/2387864445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68b41036e0 msgr2=0x7f68b4198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:03.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.548+0000 7f68b9284640 1 --2- 172.21.15.120:0/2387864445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68b41036e0 0x7f68b4198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.548+0000 7f68b9284640 1 -- 172.21.15.120:0/2387864445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68b41082a0 msgr2=0x7f68b419cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:03.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.548+0000 7f68b9a85640 1 --2- 172.21.15.120:0/2387864445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68b41082a0 0x7f68b419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:03.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.548+0000 7f68b9284640 1 --2- 172.21.15.120:0/2387864445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68b41082a0 0x7f68b419cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.548+0000 7f68b9284640 1 -- 172.21.15.120:0/2387864445 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b419d5d0 con 0x7f68b4102d30 2024-09-19T09:16:03.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.548+0000 7f68b9284640 1 --2- 172.21.15.120:0/2387864445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68b4102d30 0x7f68b4198620 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009a50 tx=0x7f68b0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:03.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.548+0000 7f68aa7fc640 1 -- 172.21.15.120:0/2387864445 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b003f030 con 0x7f68b4102d30 2024-09-19T09:16:03.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.548+0000 7f68bb4e8640 1 -- 172.21.15.120:0/2387864445 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b419d860 con 0x7f68b4102d30 2024-09-19T09:16:03.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.548+0000 7f68aa7fc640 1 -- 172.21.15.120:0/2387864445 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68b0003ce0 con 0x7f68b4102d30 2024-09-19T09:16:03.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.548+0000 7f68aa7fc640 1 -- 172.21.15.120:0/2387864445 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0035070 con 0x7f68b4102d30 2024-09-19T09:16:03.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.548+0000 7f68bb4e8640 1 -- 172.21.15.120:0/2387864445 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b41a51a0 con 0x7f68b4102d30 2024-09-19T09:16:03.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.550+0000 7f68aa7fc640 1 -- 172.21.15.120:0/2387864445 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f68b0042030 con 0x7f68b4102d30 2024-09-19T09:16:03.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.551+0000 7f68bd8c1740 1 Processor -- start 2024-09-19T09:16:03.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.551+0000 7f68bd8c1740 1 -- start start 2024-09-19T09:16:03.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.551+0000 7f68bd8c1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bbda01f20 0x556bbda02300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:03.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.551+0000 7f68bd8c1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bbda02840 0x556bbda11110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:03.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.551+0000 7f68bd8c1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bbda11650 0x556bbda13a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:03.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.551+0000 7f68bd8c1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556bbd967620 con 0x556bbda01f20 2024-09-19T09:16:03.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.551+0000 7f68bd8c1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556bbd90ee50 con 0x556bbda02840 2024-09-19T09:16:03.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.551+0000 7f68bd8c1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556bbd9635e0 con 0x556bbda11650 2024-09-19T09:16:03.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.551+0000 7f68aa7fc640 1 --2- 172.21.15.120:0/2387864445 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f68a00a3240 0x7f68a00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:03.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.551+0000 7f68b9a85640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bbda01f20 0x556bbda02300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:03.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.551+0000 7f68b9a85640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bbda01f20 0x556bbda02300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54492/0 (socket says 172.21.15.120:54492) 2024-09-19T09:16:03.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.551+0000 7f68b9a85640 1 -- 172.21.15.120:0/612009375 learned_addr learned my addr 172.21.15.120:0/612009375 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:03.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.552+0000 7f68b8a83640 1 --2- 172.21.15.120:0/612009375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bbda11650 0x556bbda13a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:03.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.552+0000 7f68b9a85640 1 --2- 172.21.15.120:0/2387864445 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f68a00a3240 0x7f68a00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:03.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.552+0000 7f68aa7fc640 1 -- 172.21.15.120:0/2387864445 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f68b00eff60 con 0x7f68b4102d30 2024-09-19T09:16:03.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.552+0000 7f68b9284640 1 --2- 172.21.15.120:0/612009375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bbda02840 0x556bbda11110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:03.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.552+0000 7f68b9a85640 1 -- 172.21.15.120:0/612009375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bbda11650 msgr2=0x556bbda13a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:03.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.552+0000 7f68b9a85640 1 --2- 172.21.15.120:0/612009375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bbda11650 0x556bbda13a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.552+0000 7f68b9a85640 1 -- 172.21.15.120:0/612009375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bbda02840 msgr2=0x556bbda11110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:03.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.552+0000 7f68b9a85640 1 --2- 172.21.15.120:0/612009375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bbda02840 0x556bbda11110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.552+0000 7f68b9a85640 1 -- 172.21.15.120:0/612009375 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bbd8ffd40 con 0x556bbda01f20 2024-09-19T09:16:03.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.552+0000 7f68b9a85640 1 --2- 172.21.15.120:0/612009375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bbda01f20 0x556bbda02300 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0148e0 tx=0x7f68ac014da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1061429e84e900 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:03.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.552+0000 7f689f7fe640 1 -- 172.21.15.120:0/612009375 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac021070 con 0x556bbda01f20 2024-09-19T09:16:03.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.552+0000 7f689f7fe640 1 -- 172.21.15.120:0/612009375 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68ac011e30 con 0x556bbda01f20 2024-09-19T09:16:03.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.552+0000 7f689f7fe640 1 -- 172.21.15.120:0/612009375 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac00f630 con 0x556bbda01f20 2024-09-19T09:16:03.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.552+0000 7f68b9a85640 1 --2- 172.21.15.120:0/2387864445 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f68a00a3240 0x7f68a00a5700 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f68ac01e000 tx=0x7f68ac01a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:03.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.553+0000 7f68bd8c1740 1 -- 172.21.15.120:0/612009375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bbda01f20 msgr2=0x556bbda02300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:03.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.553+0000 7f68bd8c1740 1 --2- 172.21.15.120:0/612009375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bbda01f20 0x556bbda02300 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0148e0 tx=0x7f68ac014da0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.553+0000 7f68bd8c1740 1 -- 172.21.15.120:0/612009375 shutdown_connections 2024-09-19T09:16:03.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.553+0000 7f68bd8c1740 1 --2- 172.21.15.120:0/612009375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bbda11650 0x556bbda13a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.553+0000 7f68bd8c1740 1 --2- 172.21.15.120:0/612009375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bbda02840 0x556bbda11110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.553+0000 7f68bd8c1740 1 --2- 172.21.15.120:0/612009375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bbda01f20 0x556bbda02300 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.553+0000 7f68bd8c1740 1 -- 172.21.15.120:0/612009375 >> 172.21.15.120:0/612009375 conn(0x556bbd9fe5f0 msgr2=0x556bbd9fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:03.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.553+0000 7f68bd8c1740 1 -- 172.21.15.120:0/612009375 shutdown_connections 2024-09-19T09:16:03.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.553+0000 7f68bd8c1740 1 -- 172.21.15.120:0/612009375 wait complete. 2024-09-19T09:16:03.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.553+0000 7f68bd8c1740 1 -- start start 2024-09-19T09:16:03.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.553+0000 7f68bd8c1740 1 Processor -- start 2024-09-19T09:16:03.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.554+0000 7f68bd8c1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bbda01f20 0x556bbda0e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:03.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.554+0000 7f68bd8c1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bbda02840 0x556bbda0eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:03.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.554+0000 7f68bd8c1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bbda11650 0x556bbda0f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:03.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.554+0000 7f68bd8c1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556bbda14220 con 0x556bbda01f20 2024-09-19T09:16:03.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.554+0000 7f68bd8c1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556bbd8eff10 con 0x556bbda11650 2024-09-19T09:16:03.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.554+0000 7f68bd8c1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556bbda143a0 con 0x556bbda02840 2024-09-19T09:16:03.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.554+0000 7f68bd8c1740 10 client.0 _ll_register_callbacks cb 0x556bbd913db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:03.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.554+0000 7f68b9284640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bbda01f20 0x556bbda0e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:03.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.554+0000 7f68b9284640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bbda01f20 0x556bbda0e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54506/0 (socket says 172.21.15.120:54506) 2024-09-19T09:16:03.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.554+0000 7f68b9284640 1 -- 172.21.15.120:0/869261448 learned_addr learned my addr 172.21.15.120:0/869261448 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:03.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.554+0000 7f68b9a85640 1 --2- 172.21.15.120:0/869261448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bbda11650 0x556bbda0f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:03.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.554+0000 7f68b8a83640 1 --2- 172.21.15.120:0/869261448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bbda02840 0x556bbda0eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:03.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.554+0000 7f68b9284640 1 -- 172.21.15.120:0/869261448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bbda02840 msgr2=0x556bbda0eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:03.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.554+0000 7f68b9284640 1 --2- 172.21.15.120:0/869261448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bbda02840 0x556bbda0eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.554+0000 7f68b9284640 1 -- 172.21.15.120:0/869261448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bbda11650 msgr2=0x556bbda0f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:03.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.554+0000 7f68b9284640 1 --2- 172.21.15.120:0/869261448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bbda11650 0x556bbda0f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.554+0000 7f68b9284640 1 -- 172.21.15.120:0/869261448 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bbd676580 con 0x556bbda01f20 2024-09-19T09:16:03.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.554+0000 7f68b8a83640 1 --2- 172.21.15.120:0/869261448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bbda02840 0x556bbda0eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:16:03.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.554+0000 7f68b9a85640 1 --2- 172.21.15.120:0/869261448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bbda11650 0x556bbda0f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:16:03.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.554+0000 7f68b9284640 1 --2- 172.21.15.120:0/869261448 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bbda01f20 0x556bbda0e5d0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f68b0031550 tx=0x7f68b00bbee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9135700051c50223 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:03.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.554+0000 7f689cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:16:03.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.554+0000 7f689cff9640 1 -- 172.21.15.120:0/869261448 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b00d5060 con 0x556bbda01f20 2024-09-19T09:16:03.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.554+0000 7f689cff9640 1 -- 172.21.15.120:0/869261448 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f68b00b8bf0 con 0x556bbda01f20 2024-09-19T09:16:03.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.554+0000 7f68bd8c1740 1 -- 172.21.15.120:0/869261448 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68ac011e30 con 0x556bbda01f20 2024-09-19T09:16:03.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.555+0000 7f689cff9640 1 -- 172.21.15.120:0/869261448 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f68b0049040 con 0x556bbda01f20 2024-09-19T09:16:03.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.555+0000 7f68bd8c1740 10 client.20043 fetch_fsmap learned FSMap version 33 2024-09-19T09:16:03.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.555+0000 7f68bd8c1740 1 -- 172.21.15.120:0/869261448 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556bbd740960 con 0x556bbda01f20 2024-09-19T09:16:03.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.555+0000 7f689cff9640 1 -- 172.21.15.120:0/869261448 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68ac011e30 con 0x556bbda01f20 2024-09-19T09:16:03.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.555+0000 7f689cff9640 1 -- 172.21.15.120:0/869261448 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f68b00c4d60 con 0x556bbda01f20 2024-09-19T09:16:03.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.555+0000 7f68bd8c1740 10 client.20043 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:16:03.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.555+0000 7f68bd8c1740 10 client.20043 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:16:03.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.555+0000 7f68bd8c1740 20 client.20043 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:03.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.555+0000 7f68bd8c1740 1 --2- 172.21.15.120:0/869261448 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556bbda21220 0x556bbda23610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:03.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.555+0000 7f68bd8c1740 4 client.20043 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:16:03.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.555+0000 7f68bd8c1740 1 -- 172.21.15.120:0/869261448 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bbd974b90 con 0x556bbda21220 2024-09-19T09:16:03.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.555+0000 7f68b8a83640 1 --2- 172.21.15.120:0/869261448 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556bbda21220 0x556bbda23610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:03.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.556+0000 7f689cff9640 10 client.20043 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:03.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.556+0000 7f68b8a83640 1 --2- 172.21.15.120:0/869261448 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556bbda21220 0x556bbda23610 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8346a4df18f5e46 server_cookie=ccd303302fa3de76 in_seq=0 out_seq=0 2024-09-19T09:16:03.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.556+0000 7f689cff9640 1 -- 172.21.15.120:0/869261448 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556bbd974b90 con 0x556bbda21220 2024-09-19T09:16:03.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.556+0000 7f689cff9640 10 client.20043 handle_command_reply: tid=0 2024-09-19T09:16:03.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.556+0000 7f68bd8c1740 1 client.20043 shutdown 2024-09-19T09:16:03.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.556+0000 7f68bd8c1740 1 -- 172.21.15.120:0/869261448 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bbda01f20 msgr2=0x556bbda0e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:03.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.556+0000 7f68bd8c1740 1 --2- 172.21.15.120:0/869261448 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bbda01f20 0x556bbda0e5d0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f68b0031550 tx=0x7f68b00bbee0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.556+0000 7f68bd8c1740 1 -- 172.21.15.120:0/869261448 shutdown_connections 2024-09-19T09:16:03.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.557+0000 7f68bd8c1740 1 --2- 172.21.15.120:0/869261448 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556bbda21220 0x556bbda23610 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.557+0000 7f68bd8c1740 1 --2- 172.21.15.120:0/869261448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bbda11650 0x556bbda0f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.557+0000 7f68bd8c1740 1 --2- 172.21.15.120:0/869261448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bbda02840 0x556bbda0eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.557+0000 7f689cff9640 0 client.20043 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:03.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.557+0000 7f68bd8c1740 1 --2- 172.21.15.120:0/869261448 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bbda01f20 0x556bbda0e5d0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.557+0000 7f68bd8c1740 1 -- 172.21.15.120:0/869261448 >> 172.21.15.120:0/869261448 conn(0x556bbd9fe5f0 msgr2=0x556bbda04350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:03.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.557+0000 7f68bd8c1740 1 -- 172.21.15.120:0/869261448 shutdown_connections 2024-09-19T09:16:03.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.557+0000 7f68bd8c1740 1 -- 172.21.15.120:0/869261448 wait complete. 2024-09-19T09:16:03.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.557+0000 7f68bd8c1740 20 client.20043 trim_cache size 0 max 16384 2024-09-19T09:16:03.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.574+0000 7f68bd8c1740 1 Processor -- start 2024-09-19T09:16:03.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.574+0000 7f68bd8c1740 1 -- start start 2024-09-19T09:16:03.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.574+0000 7f68bd8c1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bbda01f20 0x556bbda04120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:03.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.574+0000 7f68bd8c1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bbda02840 0x556bbda28f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:03.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.574+0000 7f68bd8c1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bbda11650 0x556bbda29460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:03.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.574+0000 7f68bd8c1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41ab480 con 0x556bbda02840 2024-09-19T09:16:03.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.574+0000 7f68bd8c1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41acfc0 con 0x556bbda01f20 2024-09-19T09:16:03.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.574+0000 7f68bd8c1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41abb50 con 0x556bbda11650 2024-09-19T09:16:03.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.574+0000 7f68b8a83640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bbda02840 0x556bbda28f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:03.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.574+0000 7f68b8a83640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bbda02840 0x556bbda28f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54512/0 (socket says 172.21.15.120:54512) 2024-09-19T09:16:03.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.574+0000 7f68b8a83640 1 -- 172.21.15.120:0/1524143260 learned_addr learned my addr 172.21.15.120:0/1524143260 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:03.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.574+0000 7f68b9284640 1 --2- 172.21.15.120:0/1524143260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bbda01f20 0x556bbda04120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:03.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.574+0000 7f68b9a85640 1 --2- 172.21.15.120:0/1524143260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bbda11650 0x556bbda29460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:03.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.574+0000 7f68b8a83640 1 -- 172.21.15.120:0/1524143260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bbda11650 msgr2=0x556bbda29460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:03.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.575+0000 7f68b8a83640 1 --2- 172.21.15.120:0/1524143260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bbda11650 0x556bbda29460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.575+0000 7f68b8a83640 1 -- 172.21.15.120:0/1524143260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bbda01f20 msgr2=0x556bbda04120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:03.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.575+0000 7f68b8a83640 1 --2- 172.21.15.120:0/1524143260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bbda01f20 0x556bbda04120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.575+0000 7f68b8a83640 1 -- 172.21.15.120:0/1524143260 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bbda0d580 con 0x556bbda02840 2024-09-19T09:16:03.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.575+0000 7f68b9a85640 1 --2- 172.21.15.120:0/1524143260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bbda11650 0x556bbda29460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:16:03.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.575+0000 7f68b8a83640 1 --2- 172.21.15.120:0/1524143260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bbda02840 0x556bbda28f20 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f68b40fee30 tx=0x7f68a4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63d5d41f47872ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:03.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.575+0000 7f689effd640 1 -- 172.21.15.120:0/1524143260 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a4020070 con 0x556bbda02840 2024-09-19T09:16:03.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.575+0000 7f689effd640 1 -- 172.21.15.120:0/1524143260 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68a4015c90 con 0x556bbda02840 2024-09-19T09:16:03.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.575+0000 7f689effd640 1 -- 172.21.15.120:0/1524143260 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a40146a0 con 0x556bbda02840 2024-09-19T09:16:03.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.576+0000 7f68bd8c1740 1 -- 172.21.15.120:0/1524143260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bbda02840 msgr2=0x556bbda28f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:03.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.576+0000 7f68bd8c1740 1 --2- 172.21.15.120:0/1524143260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bbda02840 0x556bbda28f20 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f68b40fee30 tx=0x7f68a4008040 comp rx=0 tx=0).stop 2024-09-19T09:16:03.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.576+0000 7f68bd8c1740 1 -- 172.21.15.120:0/1524143260 shutdown_connections 2024-09-19T09:16:03.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.576+0000 7f68bd8c1740 1 --2- 172.21.15.120:0/1524143260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bbda11650 0x556bbda29460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.576+0000 7f68bd8c1740 1 --2- 172.21.15.120:0/1524143260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bbda02840 0x556bbda28f20 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.576+0000 7f68bd8c1740 1 --2- 172.21.15.120:0/1524143260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bbda01f20 0x556bbda04120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.576+0000 7f68bd8c1740 1 -- 172.21.15.120:0/1524143260 >> 172.21.15.120:0/1524143260 conn(0x556bbd9fe5f0 msgr2=0x556bbda03a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:03.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.576+0000 7f68bd8c1740 1 -- 172.21.15.120:0/1524143260 shutdown_connections 2024-09-19T09:16:03.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.576+0000 7f68bd8c1740 1 -- 172.21.15.120:0/1524143260 wait complete. 2024-09-19T09:16:03.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.576+0000 7f68bd8c1740 1 -- start start 2024-09-19T09:16:03.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.576+0000 7f68bd8c1740 1 Processor -- start 2024-09-19T09:16:03.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.577+0000 7f68bd8c1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bbda01f20 0x556bbda19ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:03.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.577+0000 7f68bd8c1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bbda02840 0x556bbda15e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:03.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.577+0000 7f68bd8c1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bbda11650 0x556bbda16350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:03.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.577+0000 7f68bd8c1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41ac8f0 con 0x556bbda01f20 2024-09-19T09:16:03.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.577+0000 7f68bd8c1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41ac220 con 0x556bbda11650 2024-09-19T09:16:03.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.577+0000 7f68bd8c1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41a9440 con 0x556bbda02840 2024-09-19T09:16:03.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.577+0000 7f68bd8c1740 10 client.0 _ll_register_callbacks cb 0x7f68b00bc350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:03.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.577+0000 7f68b9284640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bbda01f20 0x556bbda19ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:03.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.577+0000 7f68b9284640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bbda01f20 0x556bbda19ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54522/0 (socket says 172.21.15.120:54522) 2024-09-19T09:16:03.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.577+0000 7f68b9284640 1 -- 172.21.15.120:0/2717987783 learned_addr learned my addr 172.21.15.120:0/2717987783 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:03.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.577+0000 7f68b8a83640 1 --2- 172.21.15.120:0/2717987783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bbda02840 0x556bbda15e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:03.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.577+0000 7f68b9a85640 1 --2- 172.21.15.120:0/2717987783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bbda11650 0x556bbda16350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:03.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.577+0000 7f68b9284640 1 -- 172.21.15.120:0/2717987783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bbda02840 msgr2=0x556bbda15e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:03.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.577+0000 7f68b9284640 1 --2- 172.21.15.120:0/2717987783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bbda02840 0x556bbda15e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.577+0000 7f68b9284640 1 -- 172.21.15.120:0/2717987783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bbda11650 msgr2=0x556bbda16350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:03.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.577+0000 7f68b9284640 1 --2- 172.21.15.120:0/2717987783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bbda11650 0x556bbda16350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.577+0000 7f68b9284640 1 -- 172.21.15.120:0/2717987783 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bbda16890 con 0x556bbda01f20 2024-09-19T09:16:03.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.577+0000 7f68b9284640 1 --2- 172.21.15.120:0/2717987783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bbda01f20 0x556bbda19ad0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f68b00c4cc0 tx=0x7f68b00314b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91c80abf817f060 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:03.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.577+0000 7f689cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:16:03.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.577+0000 7f689cff9640 1 -- 172.21.15.120:0/2717987783 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b00c51f0 con 0x556bbda01f20 2024-09-19T09:16:03.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.578+0000 7f689cff9640 1 -- 172.21.15.120:0/2717987783 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f68b0003800 con 0x556bbda01f20 2024-09-19T09:16:03.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.578+0000 7f68bd8c1740 1 -- 172.21.15.120:0/2717987783 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68a4015c90 con 0x556bbda01f20 2024-09-19T09:16:03.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.578+0000 7f689cff9640 1 -- 172.21.15.120:0/2717987783 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f68b00bbcf0 con 0x556bbda01f20 2024-09-19T09:16:03.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.578+0000 7f68bd8c1740 10 client.20049 fetch_fsmap learned FSMap version 33 2024-09-19T09:16:03.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.578+0000 7f68bd8c1740 1 -- 172.21.15.120:0/2717987783 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556bbda8cc00 con 0x556bbda01f20 2024-09-19T09:16:03.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.578+0000 7f689cff9640 1 -- 172.21.15.120:0/2717987783 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68a4015c90 con 0x556bbda01f20 2024-09-19T09:16:03.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.578+0000 7f689cff9640 1 -- 172.21.15.120:0/2717987783 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f68b00bb440 con 0x556bbda01f20 2024-09-19T09:16:03.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.578+0000 7f68bd8c1740 10 client.20049 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:16:03.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.578+0000 7f68bd8c1740 10 client.20049 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:16:03.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.578+0000 7f68bd8c1740 20 client.20049 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:03.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.578+0000 7f68bd8c1740 1 --2- 172.21.15.120:0/2717987783 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556bbda8d1c0 0x556bbda8f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:03.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.578+0000 7f68bd8c1740 4 client.20049 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ce305c23", "release": true, "await": true}] 2024-09-19T09:16:03.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.578+0000 7f68bd8c1740 1 -- 172.21.15.120:0/2717987783 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ce305c23", "release": true, "await": true}) -- 0x556bbd96f0a0 con 0x556bbda8d1c0 2024-09-19T09:16:03.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.578+0000 7f68b8a83640 1 --2- 172.21.15.120:0/2717987783 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556bbda8d1c0 0x556bbda8f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:03.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.579+0000 7f689cff9640 10 client.20049 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:03.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.579+0000 7f68b8a83640 1 --2- 172.21.15.120:0/2717987783 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556bbda8d1c0 0x556bbda8f5e0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f55aa804504629f server_cookie=b8b72c06207cd80b in_seq=0 out_seq=0 2024-09-19T09:16:03.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.589+0000 7f689cff9640 1 -- 172.21.15.120:0/2717987783 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556bbd96f0a0 con 0x556bbda8d1c0 2024-09-19T09:16:03.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.589+0000 7f689cff9640 10 client.20049 handle_command_reply: tid=0 2024-09-19T09:16:03.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.589+0000 7f68bd8c1740 1 client.20049 shutdown 2024-09-19T09:16:03.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.589+0000 7f68bd8c1740 1 -- 172.21.15.120:0/2717987783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bbda01f20 msgr2=0x556bbda19ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:03.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.589+0000 7f68bd8c1740 1 --2- 172.21.15.120:0/2717987783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bbda01f20 0x556bbda19ad0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f68b00c4cc0 tx=0x7f68b00314b0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.589+0000 7f68bd8c1740 1 -- 172.21.15.120:0/2717987783 shutdown_connections 2024-09-19T09:16:03.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.589+0000 7f68bd8c1740 1 --2- 172.21.15.120:0/2717987783 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556bbda8d1c0 0x556bbda8f5e0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.589+0000 7f68bd8c1740 1 --2- 172.21.15.120:0/2717987783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bbda11650 0x556bbda16350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.589+0000 7f689cff9640 0 client.20049 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:03.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.589+0000 7f68bd8c1740 1 --2- 172.21.15.120:0/2717987783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bbda02840 0x556bbda15e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.589+0000 7f68bd8c1740 1 --2- 172.21.15.120:0/2717987783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bbda01f20 0x556bbda19ad0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.589+0000 7f68bd8c1740 1 -- 172.21.15.120:0/2717987783 >> 172.21.15.120:0/2717987783 conn(0x556bbd9fe5f0 msgr2=0x556bbda04230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:03.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.589+0000 7f68bd8c1740 1 -- 172.21.15.120:0/2717987783 shutdown_connections 2024-09-19T09:16:03.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.589+0000 7f68bd8c1740 1 -- 172.21.15.120:0/2717987783 wait complete. 2024-09-19T09:16:03.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.589+0000 7f68bd8c1740 20 client.20049 trim_cache size 0 max 16384 2024-09-19T09:16:03.592 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:16:03.592 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 33, 2024-09-19T09:16:03.593 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:16:03.593 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 44, 2024-09-19T09:16:03.593 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:16:03.593 INFO:teuthology.orchestra.run.smithi120.stdout: "ce305c23": { 2024-09-19T09:16:03.593 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 44, 2024-09-19T09:16:03.593 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:16:03.593 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:16:03.593 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:16:03.593 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:16:03.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:16:03.593 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:16:03.593 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 142.2, 2024-09-19T09:16:03.593 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:16:03.594 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:16:03.594 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:16:03.594 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:16:03.594 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:16:03.594 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:16:03.594 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:16:03.594 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:16:03.594 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:16:03.594 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:16:03.594 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:16:03.594 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:16:03.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.590+0000 7f68bb4e8640 1 -- 172.21.15.120:0/2387864445 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f68a00a3240 msgr2=0x7f68a00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:03.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.590+0000 7f68bb4e8640 1 --2- 172.21.15.120:0/2387864445 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f68a00a3240 0x7f68a00a5700 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f68ac01e000 tx=0x7f68ac01a000 comp rx=0 tx=0).stop 2024-09-19T09:16:03.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.590+0000 7f68bb4e8640 1 -- 172.21.15.120:0/2387864445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68b4102d30 msgr2=0x7f68b4198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:03.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.590+0000 7f68bb4e8640 1 --2- 172.21.15.120:0/2387864445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68b4102d30 0x7f68b4198620 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009a50 tx=0x7f68b0004270 comp rx=0 tx=0).stop 2024-09-19T09:16:03.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.590+0000 7f68bb4e8640 1 -- 172.21.15.120:0/2387864445 shutdown_connections 2024-09-19T09:16:03.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.590+0000 7f68bb4e8640 1 --2- 172.21.15.120:0/2387864445 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f68a00a3240 0x7f68a00a5700 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.590+0000 7f68bb4e8640 1 --2- 172.21.15.120:0/2387864445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68b41082a0 0x7f68b419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.590+0000 7f68bb4e8640 1 --2- 172.21.15.120:0/2387864445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68b41036e0 0x7f68b4198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.590+0000 7f68bb4e8640 1 --2- 172.21.15.120:0/2387864445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68b4102d30 0x7f68b4198620 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:03.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.590+0000 7f68bb4e8640 1 -- 172.21.15.120:0/2387864445 >> 172.21.15.120:0/2387864445 conn(0x7f68b40fc7d0 msgr2=0x7f68b4105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:03.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.590+0000 7f68bb4e8640 1 -- 172.21.15.120:0/2387864445 shutdown_connections 2024-09-19T09:16:03.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:03.590+0000 7f68bb4e8640 1 -- 172.21.15.120:0/2387864445 wait complete. 2024-09-19T09:16:04.079 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1302758 9.73 MB/sec execute 599 sec latency 20520.341 ms 2024-09-19T09:16:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.112+0000 7fa8efdcf640 1 -- 172.21.15.120:0/522665752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8e8101c30 msgr2=0x7fa8e81020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.112+0000 7fa8efdcf640 1 --2- 172.21.15.120:0/522665752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8e8101c30 0x7fa8e81020b0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8009900 tx=0x7fa8d802a410 comp rx=0 tx=0).stop 2024-09-19T09:16:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.112+0000 7fa8efdcf640 1 -- 172.21.15.120:0/522665752 shutdown_connections 2024-09-19T09:16:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.112+0000 7fa8efdcf640 1 --2- 172.21.15.120:0/522665752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8e81025f0 0x7fa8e8108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.112+0000 7fa8efdcf640 1 --2- 172.21.15.120:0/522665752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8e8101c30 0x7fa8e81020b0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.112+0000 7fa8efdcf640 1 --2- 172.21.15.120:0/522665752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8e8100a40 0x7fa8e8100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.112+0000 7fa8efdcf640 1 -- 172.21.15.120:0/522665752 >> 172.21.15.120:0/522665752 conn(0x7fa8e80fc1e0 msgr2=0x7fa8e80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.112+0000 7fa8efdcf640 1 -- 172.21.15.120:0/522665752 shutdown_connections 2024-09-19T09:16:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.112+0000 7fa8efdcf640 1 -- 172.21.15.120:0/522665752 wait complete. 2024-09-19T09:16:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.112+0000 7fa8efdcf640 1 Processor -- start 2024-09-19T09:16:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.112+0000 7fa8efdcf640 1 -- start start 2024-09-19T09:16:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.113+0000 7fa8efdcf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8e8100a40 0x7fa8e8196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.113+0000 7fa8efdcf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8e8101c30 0x7fa8e81971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.113+0000 7fa8efdcf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8e81025f0 0x7fa8e819e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.113+0000 7fa8efdcf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e810c6c0 con 0x7fa8e81025f0 2024-09-19T09:16:04.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.113+0000 7fa8efdcf640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e810c540 con 0x7fa8e8101c30 2024-09-19T09:16:04.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.113+0000 7fa8efdcf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e810c840 con 0x7fa8e8100a40 2024-09-19T09:16:04.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.113+0000 7fa8ee36c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8e81025f0 0x7fa8e819e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:04.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.113+0000 7fa8edb6b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8e8100a40 0x7fa8e8196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:04.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.113+0000 7fa8ed36a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8e8101c30 0x7fa8e81971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:04.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.113+0000 7fa8edb6b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8e8100a40 0x7fa8e8196cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:45804/0 (socket says 172.21.15.120:45804) 2024-09-19T09:16:04.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.113+0000 7fa8ee36c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8e81025f0 0x7fa8e819e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54546/0 (socket says 172.21.15.120:54546) 2024-09-19T09:16:04.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.113+0000 7fa8edb6b640 1 -- 172.21.15.120:0/776948477 learned_addr learned my addr 172.21.15.120:0/776948477 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:04.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.113+0000 7fa8edb6b640 1 -- 172.21.15.120:0/776948477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8e8101c30 msgr2=0x7fa8e81971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:04.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.113+0000 7fa8edb6b640 1 --2- 172.21.15.120:0/776948477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8e8101c30 0x7fa8e81971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.113+0000 7fa8edb6b640 1 -- 172.21.15.120:0/776948477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8e81025f0 msgr2=0x7fa8e819e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:04.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.113+0000 7fa8edb6b640 1 --2- 172.21.15.120:0/776948477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8e81025f0 0x7fa8e819e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.113+0000 7fa8edb6b640 1 -- 172.21.15.120:0/776948477 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e819e8f0 con 0x7fa8e8100a40 2024-09-19T09:16:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.113+0000 7fa8ee36c640 1 --2- 172.21.15.120:0/776948477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8e81025f0 0x7fa8e819e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:16:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.113+0000 7fa8edb6b640 1 --2- 172.21.15.120:0/776948477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8e8100a40 0x7fa8e8196cb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa8e400ebc0 tx=0x7fa8e400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.113+0000 7fa8deffd640 1 -- 172.21.15.120:0/776948477 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4003b70 con 0x7fa8e8100a40 2024-09-19T09:16:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.113+0000 7fa8deffd640 1 -- 172.21.15.120:0/776948477 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8e4003d10 con 0x7fa8e8100a40 2024-09-19T09:16:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.113+0000 7fa8efdcf640 1 -- 172.21.15.120:0/776948477 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e819ebe0 con 0x7fa8e8100a40 2024-09-19T09:16:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.113+0000 7fa8deffd640 1 -- 172.21.15.120:0/776948477 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4019980 con 0x7fa8e8100a40 2024-09-19T09:16:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.114+0000 7fa8efdcf640 1 -- 172.21.15.120:0/776948477 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e819f100 con 0x7fa8e8100a40 2024-09-19T09:16:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.116+0000 7fa8deffd640 1 -- 172.21.15.120:0/776948477 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa8e4019b20 con 0x7fa8e8100a40 2024-09-19T09:16:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.116+0000 7fa8f2c44740 1 Processor -- start 2024-09-19T09:16:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.116+0000 7fa8f2c44740 1 -- start start 2024-09-19T09:16:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.117+0000 7fa8f2c44740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8c4cc79a0 0x55e8c4cc7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.117+0000 7fa8f2c44740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8c4cc82e0 0x55e8c4cccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.117+0000 7fa8f2c44740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8c4ccd370 0x55e8c4ccf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.117+0000 7fa8f2c44740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e8c4c2b2f0 con 0x55e8c4cc79a0 2024-09-19T09:16:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.117+0000 7fa8f2c44740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e8c4bd3270 con 0x55e8c4cc82e0 2024-09-19T09:16:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.117+0000 7fa8f2c44740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e8c4c272b0 con 0x55e8c4ccd370 2024-09-19T09:16:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.117+0000 7fa8ee36c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8c4cc79a0 0x55e8c4cc7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.117+0000 7fa8edb6b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8c4cc82e0 0x55e8c4cccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.117+0000 7fa8ee36c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8c4cc79a0 0x55e8c4cc7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54554/0 (socket says 172.21.15.120:54554) 2024-09-19T09:16:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.117+0000 7fa8ed36a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8c4ccd370 0x55e8c4ccf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.117+0000 7fa8deffd640 1 --2- 172.21.15.120:0/776948477 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa8d40a32f0 0x7fa8d40a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.117+0000 7fa8ee36c640 1 -- 172.21.15.120:0/2789134385 learned_addr learned my addr 172.21.15.120:0/2789134385 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.117+0000 7fa8ee36c640 1 --2- 172.21.15.120:0/776948477 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa8d40a32f0 0x7fa8d40a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.117+0000 7fa8ed36a640 1 -- 172.21.15.120:0/2789134385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8c4cc82e0 msgr2=0x55e8c4cccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.117+0000 7fa8ed36a640 1 --2- 172.21.15.120:0/2789134385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8c4cc82e0 0x55e8c4cccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.117+0000 7fa8ed36a640 1 -- 172.21.15.120:0/2789134385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8c4cc79a0 msgr2=0x55e8c4cc7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.117+0000 7fa8ed36a640 1 --2- 172.21.15.120:0/2789134385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8c4cc79a0 0x55e8c4cc7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.117+0000 7fa8ed36a640 1 -- 172.21.15.120:0/2789134385 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8c4bd2340 con 0x55e8c4ccd370 2024-09-19T09:16:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.117+0000 7fa8deffd640 1 -- 172.21.15.120:0/776948477 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fa8e40d0120 con 0x7fa8e8100a40 2024-09-19T09:16:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.117+0000 7fa8ed36a640 1 --2- 172.21.15.120:0/2789134385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8c4ccd370 0x55e8c4ccf830 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8031440 tx=0x7fa8d8005e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=311f786b4db88743 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.117+0000 7fa8ee36c640 1 --2- 172.21.15.120:0/776948477 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa8d40a32f0 0x7fa8d40a57b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0009fd0 tx=0x7fa8e0004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.118+0000 7fa8d3fff640 1 -- 172.21.15.120:0/2789134385 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8040070 con 0x55e8c4ccd370 2024-09-19T09:16:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.118+0000 7fa8d3fff640 1 -- 172.21.15.120:0/2789134385 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8d8004030 con 0x55e8c4ccd370 2024-09-19T09:16:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.118+0000 7fa8d3fff640 1 -- 172.21.15.120:0/2789134385 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8035040 con 0x55e8c4ccd370 2024-09-19T09:16:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.118+0000 7fa8f2c44740 1 -- 172.21.15.120:0/2789134385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8c4ccd370 msgr2=0x55e8c4ccf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.118+0000 7fa8f2c44740 1 --2- 172.21.15.120:0/2789134385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8c4ccd370 0x55e8c4ccf830 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8031440 tx=0x7fa8d8005e00 comp rx=0 tx=0).stop 2024-09-19T09:16:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.118+0000 7fa8f2c44740 1 -- 172.21.15.120:0/2789134385 shutdown_connections 2024-09-19T09:16:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.118+0000 7fa8f2c44740 1 --2- 172.21.15.120:0/2789134385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8c4ccd370 0x55e8c4ccf830 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.118+0000 7fa8f2c44740 1 --2- 172.21.15.120:0/2789134385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8c4cc82e0 0x55e8c4cccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.118+0000 7fa8f2c44740 1 --2- 172.21.15.120:0/2789134385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8c4cc79a0 0x55e8c4cc7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.118+0000 7fa8f2c44740 1 -- 172.21.15.120:0/2789134385 >> 172.21.15.120:0/2789134385 conn(0x55e8c4cc3010 msgr2=0x55e8c4cc3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.118+0000 7fa8f2c44740 1 -- 172.21.15.120:0/2789134385 shutdown_connections 2024-09-19T09:16:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.118+0000 7fa8f2c44740 1 -- 172.21.15.120:0/2789134385 wait complete. 2024-09-19T09:16:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.119+0000 7fa8f2c44740 1 -- start start 2024-09-19T09:16:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.119+0000 7fa8f2c44740 1 Processor -- start 2024-09-19T09:16:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.119+0000 7fa8f2c44740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8c4cc79a0 0x55e8c4cd3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.119+0000 7fa8f2c44740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8c4cc82e0 0x55e8c4cd42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.119+0000 7fa8f2c44740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8c4ccd370 0x55e8c4cdb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.119+0000 7fa8f2c44740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e8c4cd02e0 con 0x55e8c4cc79a0 2024-09-19T09:16:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.119+0000 7fa8f2c44740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e8c4bb4290 con 0x55e8c4cc82e0 2024-09-19T09:16:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.119+0000 7fa8f2c44740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e8c4cd0460 con 0x55e8c4ccd370 2024-09-19T09:16:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.119+0000 7fa8f2c44740 10 client.0 _ll_register_callbacks cb 0x55e8c4be5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.120+0000 7fa8edb6b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8c4cc79a0 0x55e8c4cd3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.120+0000 7fa8edb6b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8c4cc79a0 0x55e8c4cd3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54560/0 (socket says 172.21.15.120:54560) 2024-09-19T09:16:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.120+0000 7fa8edb6b640 1 -- 172.21.15.120:0/1167079612 learned_addr learned my addr 172.21.15.120:0/1167079612 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.120+0000 7fa8ed36a640 1 --2- 172.21.15.120:0/1167079612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8c4cc82e0 0x55e8c4cd42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.120+0000 7fa8ee36c640 1 --2- 172.21.15.120:0/1167079612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8c4ccd370 0x55e8c4cdb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.120+0000 7fa8edb6b640 1 -- 172.21.15.120:0/1167079612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8c4ccd370 msgr2=0x55e8c4cdb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.120+0000 7fa8edb6b640 1 --2- 172.21.15.120:0/1167079612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8c4ccd370 0x55e8c4cdb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.120+0000 7fa8edb6b640 1 -- 172.21.15.120:0/1167079612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8c4cc82e0 msgr2=0x55e8c4cd42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.120+0000 7fa8edb6b640 1 --2- 172.21.15.120:0/1167079612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8c4cc82e0 0x55e8c4cd42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.120+0000 7fa8edb6b640 1 -- 172.21.15.120:0/1167079612 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8c493e210 con 0x55e8c4cc79a0 2024-09-19T09:16:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.120+0000 7fa8edb6b640 1 --2- 172.21.15.120:0/1167079612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8c4cc79a0 0x55e8c4cd3d80 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fa8e400eb90 tx=0x7fa8e400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dabe0878795a459 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.120+0000 7fa8d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:16:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.120+0000 7fa8d17fa640 1 -- 172.21.15.120:0/1167079612 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e409c0a0 con 0x55e8c4cc79a0 2024-09-19T09:16:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.120+0000 7fa8d17fa640 1 -- 172.21.15.120:0/1167079612 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fa8e40a5120 con 0x55e8c4cc79a0 2024-09-19T09:16:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.120+0000 7fa8f2c44740 1 -- 172.21.15.120:0/1167079612 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8d8004030 con 0x55e8c4cc79a0 2024-09-19T09:16:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.120+0000 7fa8d17fa640 1 -- 172.21.15.120:0/1167079612 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa8e409ce40 con 0x55e8c4cc79a0 2024-09-19T09:16:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.120+0000 7fa8f2c44740 10 client.20061 fetch_fsmap learned FSMap version 33 2024-09-19T09:16:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.120+0000 7fa8f2c44740 1 -- 172.21.15.120:0/1167079612 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e8c4a0a620 con 0x55e8c4cc79a0 2024-09-19T09:16:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.120+0000 7fa8d17fa640 1 -- 172.21.15.120:0/1167079612 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8d8004030 con 0x55e8c4cc79a0 2024-09-19T09:16:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.121+0000 7fa8d17fa640 1 -- 172.21.15.120:0/1167079612 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fa8e40d2160 con 0x55e8c4cc79a0 2024-09-19T09:16:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.121+0000 7fa8f2c44740 10 client.20061 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:16:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.121+0000 7fa8f2c44740 10 client.20061 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:16:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.121+0000 7fa8f2c44740 20 client.20061 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.121+0000 7fa8f2c44740 1 --2- 172.21.15.120:0/1167079612 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e8c4cd2d40 0x55e8c4cd31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.121+0000 7fa8f2c44740 4 client.20061 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:16:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.121+0000 7fa8f2c44740 1 -- 172.21.15.120:0/1167079612 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8c4c38a20 con 0x55e8c4cd2d40 2024-09-19T09:16:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.121+0000 7fa8ed36a640 1 --2- 172.21.15.120:0/1167079612 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e8c4cd2d40 0x55e8c4cd31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.121+0000 7fa8ed36a640 1 --2- 172.21.15.120:0/1167079612 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e8c4cd2d40 0x55e8c4cd31d0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e86efd847bd3d65b server_cookie=98e017325ca7f474 in_seq=0 out_seq=0 2024-09-19T09:16:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.121+0000 7fa8d17fa640 10 client.20061 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.122+0000 7fa8d17fa640 1 -- 172.21.15.120:0/1167079612 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e8c4c38a20 con 0x55e8c4cd2d40 2024-09-19T09:16:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.122+0000 7fa8d17fa640 10 client.20061 handle_command_reply: tid=0 2024-09-19T09:16:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.123+0000 7fa8f2c44740 1 client.20061 shutdown 2024-09-19T09:16:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.123+0000 7fa8f2c44740 1 -- 172.21.15.120:0/1167079612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8c4cc79a0 msgr2=0x55e8c4cd3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.123+0000 7fa8f2c44740 1 --2- 172.21.15.120:0/1167079612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8c4cc79a0 0x55e8c4cd3d80 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fa8e400eb90 tx=0x7fa8e400ef40 comp rx=0 tx=0).stop 2024-09-19T09:16:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.123+0000 7fa8f2c44740 1 -- 172.21.15.120:0/1167079612 shutdown_connections 2024-09-19T09:16:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.123+0000 7fa8f2c44740 1 --2- 172.21.15.120:0/1167079612 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e8c4cd2d40 0x55e8c4cd31d0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.123+0000 7fa8f2c44740 1 --2- 172.21.15.120:0/1167079612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8c4ccd370 0x55e8c4cdb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.123+0000 7fa8f2c44740 1 --2- 172.21.15.120:0/1167079612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8c4cc82e0 0x55e8c4cd42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.123+0000 7fa8f2c44740 1 --2- 172.21.15.120:0/1167079612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8c4cc79a0 0x55e8c4cd3d80 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.123+0000 7fa8f2c44740 1 -- 172.21.15.120:0/1167079612 >> 172.21.15.120:0/1167079612 conn(0x55e8c4cc3010 msgr2=0x55e8c4cc4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.123+0000 7fa8d17fa640 0 client.20061 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.123+0000 7fa8f2c44740 1 -- 172.21.15.120:0/1167079612 shutdown_connections 2024-09-19T09:16:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.123+0000 7fa8f2c44740 1 -- 172.21.15.120:0/1167079612 wait complete. 2024-09-19T09:16:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.123+0000 7fa8f2c44740 20 client.20061 trim_cache size 0 max 16384 2024-09-19T09:16:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.138+0000 7fa8f2c44740 1 Processor -- start 2024-09-19T09:16:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.138+0000 7fa8f2c44740 1 -- start start 2024-09-19T09:16:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.138+0000 7fa8f2c44740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8c4cc79a0 0x55e8c4cc20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.138+0000 7fa8f2c44740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8c4cc82e0 0x55e8c4cdb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.138+0000 7fa8f2c44740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8c4ccd370 0x55e8c4ce45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.138+0000 7fa8f2c44740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e81a6da0 con 0x55e8c4cc82e0 2024-09-19T09:16:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.138+0000 7fa8f2c44740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e81a88e0 con 0x55e8c4ccd370 2024-09-19T09:16:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.138+0000 7fa8f2c44740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e81a7470 con 0x55e8c4cc79a0 2024-09-19T09:16:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.138+0000 7fa8ed36a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8c4cc82e0 0x55e8c4cdb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.138+0000 7fa8ed36a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8c4cc82e0 0x55e8c4cdb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54572/0 (socket says 172.21.15.120:54572) 2024-09-19T09:16:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.138+0000 7fa8ed36a640 1 -- 172.21.15.120:0/1135380868 learned_addr learned my addr 172.21.15.120:0/1135380868 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.138+0000 7fa8edb6b640 1 --2- 172.21.15.120:0/1135380868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8c4cc79a0 0x55e8c4cc20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.138+0000 7fa8ee36c640 1 --2- 172.21.15.120:0/1135380868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8c4ccd370 0x55e8c4ce45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.138+0000 7fa8ed36a640 1 -- 172.21.15.120:0/1135380868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8c4cc79a0 msgr2=0x55e8c4cc20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.138+0000 7fa8ed36a640 1 --2- 172.21.15.120:0/1135380868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8c4cc79a0 0x55e8c4cc20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.138+0000 7fa8ed36a640 1 -- 172.21.15.120:0/1135380868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8c4ccd370 msgr2=0x55e8c4ce45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.138+0000 7fa8ed36a640 1 --2- 172.21.15.120:0/1135380868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8c4ccd370 0x55e8c4ce45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.138+0000 7fa8ed36a640 1 -- 172.21.15.120:0/1135380868 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8c4cc87c0 con 0x55e8c4cc82e0 2024-09-19T09:16:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.138+0000 7fa8ed36a640 1 --2- 172.21.15.120:0/1135380868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8c4cc82e0 0x55e8c4cdb2c0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x55e8c4cc3e90 tx=0x7fa8d8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16be51c1f6da410f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.139+0000 7fa8d37fe640 1 -- 172.21.15.120:0/1135380868 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8040050 con 0x55e8c4cc82e0 2024-09-19T09:16:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.139+0000 7fa8d37fe640 1 -- 172.21.15.120:0/1135380868 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8d803c440 con 0x55e8c4cc82e0 2024-09-19T09:16:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.139+0000 7fa8d37fe640 1 -- 172.21.15.120:0/1135380868 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8035040 con 0x55e8c4cc82e0 2024-09-19T09:16:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.139+0000 7fa8f2c44740 1 -- 172.21.15.120:0/1135380868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8c4cc82e0 msgr2=0x55e8c4cdb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.139+0000 7fa8f2c44740 1 --2- 172.21.15.120:0/1135380868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8c4cc82e0 0x55e8c4cdb2c0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x55e8c4cc3e90 tx=0x7fa8d8000bc0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.139+0000 7fa8f2c44740 1 -- 172.21.15.120:0/1135380868 shutdown_connections 2024-09-19T09:16:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.139+0000 7fa8f2c44740 1 --2- 172.21.15.120:0/1135380868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8c4ccd370 0x55e8c4ce45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.139+0000 7fa8f2c44740 1 --2- 172.21.15.120:0/1135380868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8c4cc82e0 0x55e8c4cdb2c0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.139+0000 7fa8f2c44740 1 --2- 172.21.15.120:0/1135380868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8c4cc79a0 0x55e8c4cc20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.139+0000 7fa8f2c44740 1 -- 172.21.15.120:0/1135380868 >> 172.21.15.120:0/1135380868 conn(0x55e8c4cc3010 msgr2=0x55e8c4cc19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.139+0000 7fa8f2c44740 1 -- 172.21.15.120:0/1135380868 shutdown_connections 2024-09-19T09:16:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.139+0000 7fa8f2c44740 1 -- 172.21.15.120:0/1135380868 wait complete. 2024-09-19T09:16:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.140+0000 7fa8f2c44740 1 -- start start 2024-09-19T09:16:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.140+0000 7fa8f2c44740 1 Processor -- start 2024-09-19T09:16:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.140+0000 7fa8f2c44740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8c4cc79a0 0x55e8c4cd8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.140+0000 7fa8f2c44740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8c4cc82e0 0x55e8c4cd8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.140+0000 7fa8f2c44740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8c4ccd370 0x55e8c4cd9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.140+0000 7fa8f2c44740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e81a8210 con 0x55e8c4ccd370 2024-09-19T09:16:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.140+0000 7fa8f2c44740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e81a7b40 con 0x55e8c4cc79a0 2024-09-19T09:16:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.140+0000 7fa8f2c44740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e81a4d60 con 0x55e8c4cc82e0 2024-09-19T09:16:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.140+0000 7fa8f2c44740 10 client.0 _ll_register_callbacks cb 0x7fa8e4004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.140+0000 7fa8ee36c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8c4ccd370 0x55e8c4cd9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.140+0000 7fa8ed36a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8c4cc82e0 0x55e8c4cd8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.140+0000 7fa8edb6b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8c4cc79a0 0x55e8c4cd8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.140+0000 7fa8ee36c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8c4ccd370 0x55e8c4cd9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54588/0 (socket says 172.21.15.120:54588) 2024-09-19T09:16:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.140+0000 7fa8ee36c640 1 -- 172.21.15.120:0/1853316658 learned_addr learned my addr 172.21.15.120:0/1853316658 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.141+0000 7fa8ed36a640 1 -- 172.21.15.120:0/1853316658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8c4cc79a0 msgr2=0x55e8c4cd8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.141+0000 7fa8ed36a640 1 --2- 172.21.15.120:0/1853316658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8c4cc79a0 0x55e8c4cd8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.141+0000 7fa8ed36a640 1 -- 172.21.15.120:0/1853316658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8c4ccd370 msgr2=0x55e8c4cd9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.141+0000 7fa8ed36a640 1 --2- 172.21.15.120:0/1853316658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8c4ccd370 0x55e8c4cd9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.141+0000 7fa8ed36a640 1 -- 172.21.15.120:0/1853316658 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8c4cda570 con 0x55e8c4cc82e0 2024-09-19T09:16:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.141+0000 7fa8ee36c640 1 --2- 172.21.15.120:0/1853316658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8c4ccd370 0x55e8c4cd9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:16:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.141+0000 7fa8edb6b640 1 --2- 172.21.15.120:0/1853316658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8c4cc79a0 0x55e8c4cd8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:16:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.141+0000 7fa8ed36a640 1 --2- 172.21.15.120:0/1853316658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8c4cc82e0 0x55e8c4cd8fc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55e8c4cd4a80 tx=0x7fa8d8009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb2206a82141456a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.141+0000 7fa8d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:16:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.141+0000 7fa8d17fa640 1 -- 172.21.15.120:0/1853316658 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8035040 con 0x55e8c4cc82e0 2024-09-19T09:16:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.141+0000 7fa8f2c44740 1 -- 172.21.15.120:0/1853316658 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8d803c440 con 0x55e8c4cc82e0 2024-09-19T09:16:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.141+0000 7fa8d17fa640 1 -- 172.21.15.120:0/1853316658 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fa8d803c5f0 con 0x55e8c4cc82e0 2024-09-19T09:16:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.141+0000 7fa8d17fa640 1 -- 172.21.15.120:0/1853316658 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa8d8005650 con 0x55e8c4cc82e0 2024-09-19T09:16:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.141+0000 7fa8f2c44740 10 client.28861 fetch_fsmap learned FSMap version 33 2024-09-19T09:16:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.141+0000 7fa8f2c44740 1 -- 172.21.15.120:0/1853316658 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e8c4d48580 con 0x55e8c4cc82e0 2024-09-19T09:16:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.141+0000 7fa8d17fa640 1 -- 172.21.15.120:0/1853316658 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8d803c440 con 0x55e8c4cc82e0 2024-09-19T09:16:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.141+0000 7fa8d17fa640 1 -- 172.21.15.120:0/1853316658 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fa8d8059030 con 0x55e8c4cc82e0 2024-09-19T09:16:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.142+0000 7fa8f2c44740 10 client.28861 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:16:04.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.142+0000 7fa8f2c44740 10 client.28861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:16:04.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.142+0000 7fa8f2c44740 20 client.28861 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:04.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.142+0000 7fa8f2c44740 1 --2- 172.21.15.120:0/1853316658 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e8c4d489d0 0x55e8c4d4adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:04.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.142+0000 7fa8f2c44740 4 client.28861 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:16:04.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.142+0000 7fa8f2c44740 1 -- 172.21.15.120:0/1853316658 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e8c4c32d70 con 0x55e8c4d489d0 2024-09-19T09:16:04.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.142+0000 7fa8edb6b640 1 --2- 172.21.15.120:0/1853316658 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e8c4d489d0 0x55e8c4d4adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:04.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.142+0000 7fa8d17fa640 10 client.28861 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:04.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.142+0000 7fa8edb6b640 1 --2- 172.21.15.120:0/1853316658 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e8c4d489d0 0x55e8c4d4adc0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e92689741d8234e server_cookie=30a964ff3fa6a592 in_seq=0 out_seq=0 2024-09-19T09:16:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.203+0000 7fa8d17fa640 1 -- 172.21.15.120:0/1853316658 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e8c4c32d70 con 0x55e8c4d489d0 2024-09-19T09:16:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.203+0000 7fa8d17fa640 10 client.28861 handle_command_reply: tid=0 2024-09-19T09:16:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.203+0000 7fa8f2c44740 1 client.28861 shutdown 2024-09-19T09:16:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.204+0000 7fa8f2c44740 1 -- 172.21.15.120:0/1853316658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8c4cc82e0 msgr2=0x55e8c4cd8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.204+0000 7fa8f2c44740 1 --2- 172.21.15.120:0/1853316658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8c4cc82e0 0x55e8c4cd8fc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55e8c4cd4a80 tx=0x7fa8d8009520 comp rx=0 tx=0).stop 2024-09-19T09:16:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.204+0000 7fa8f2c44740 1 -- 172.21.15.120:0/1853316658 shutdown_connections 2024-09-19T09:16:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.204+0000 7fa8f2c44740 1 --2- 172.21.15.120:0/1853316658 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e8c4d489d0 0x55e8c4d4adc0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.204+0000 7fa8f2c44740 1 --2- 172.21.15.120:0/1853316658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8c4ccd370 0x55e8c4cd9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.204+0000 7fa8f2c44740 1 --2- 172.21.15.120:0/1853316658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8c4cc82e0 0x55e8c4cd8fc0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.204+0000 7fa8d17fa640 0 client.28861 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.204+0000 7fa8f2c44740 1 --2- 172.21.15.120:0/1853316658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8c4cc79a0 0x55e8c4cd8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.204+0000 7fa8f2c44740 1 -- 172.21.15.120:0/1853316658 >> 172.21.15.120:0/1853316658 conn(0x55e8c4cc3010 msgr2=0x55e8c4cc21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.204+0000 7fa8f2c44740 1 -- 172.21.15.120:0/1853316658 shutdown_connections 2024-09-19T09:16:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.204+0000 7fa8f2c44740 1 -- 172.21.15.120:0/1853316658 wait complete. 2024-09-19T09:16:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.204+0000 7fa8f2c44740 20 client.28861 trim_cache size 0 max 16384 2024-09-19T09:16:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:16:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:16:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:16:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:16:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.204+0000 7fa8efdcf640 1 -- 172.21.15.120:0/776948477 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa8d40a32f0 msgr2=0x7fa8d40a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.204+0000 7fa8efdcf640 1 --2- 172.21.15.120:0/776948477 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa8d40a32f0 0x7fa8d40a57b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0009fd0 tx=0x7fa8e0004730 comp rx=0 tx=0).stop 2024-09-19T09:16:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.204+0000 7fa8efdcf640 1 -- 172.21.15.120:0/776948477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8e8100a40 msgr2=0x7fa8e8196cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.204+0000 7fa8efdcf640 1 --2- 172.21.15.120:0/776948477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8e8100a40 0x7fa8e8196cb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa8e400ebc0 tx=0x7fa8e400c6a0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.204+0000 7fa8efdcf640 1 -- 172.21.15.120:0/776948477 shutdown_connections 2024-09-19T09:16:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.204+0000 7fa8efdcf640 1 --2- 172.21.15.120:0/776948477 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa8d40a32f0 0x7fa8d40a57b0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.204+0000 7fa8efdcf640 1 --2- 172.21.15.120:0/776948477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8e81025f0 0x7fa8e819e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.204+0000 7fa8efdcf640 1 --2- 172.21.15.120:0/776948477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8e8101c30 0x7fa8e81971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.204+0000 7fa8efdcf640 1 --2- 172.21.15.120:0/776948477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8e8100a40 0x7fa8e8196cb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.204+0000 7fa8efdcf640 1 -- 172.21.15.120:0/776948477 >> 172.21.15.120:0/776948477 conn(0x7fa8e80fc1e0 msgr2=0x7fa8e8104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.205+0000 7fa8efdcf640 1 -- 172.21.15.120:0/776948477 shutdown_connections 2024-09-19T09:16:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:04.205+0000 7fa8efdcf640 1 -- 172.21.15.120:0/776948477 wait complete. 2024-09-19T09:16:04.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:03 smithi120 ceph-mon[31150]: pgmap v740: 97 pgs: 97 active+clean; 39 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.3 MiB/s wr, 135 op/s 2024-09-19T09:16:04.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:03 smithi196 ceph-mon[35150]: pgmap v740: 97 pgs: 97 active+clean; 39 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.3 MiB/s wr, 135 op/s 2024-09-19T09:16:04.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:03 smithi153 ceph-mon[35071]: pgmap v740: 97 pgs: 97 active+clean; 39 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.3 MiB/s wr, 135 op/s 2024-09-19T09:16:04.446 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ce305c23', total seconds elapsed: 25.7 2024-09-19T09:16:04.446 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.2 2024-09-19T09:16:04.446 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:16:04.704 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:16:05.079 INFO:tasks.workunit.client.0.smithi120.stdout: 1 cleanup 600 sec 2024-09-19T09:16:05.248 INFO:tasks.workunit.client.0.smithi120.stdout: 0 cleanup 600 sec 2024-09-19T09:16:05.248 INFO:tasks.workunit.client.0.smithi120.stdout: 2024-09-19T09:16:05.248 INFO:tasks.workunit.client.0.smithi120.stdout: Operation Count AvgLat MaxLat 2024-09-19T09:16:05.248 INFO:tasks.workunit.client.0.smithi120.stdout: ---------------------------------------- 2024-09-19T09:16:05.248 INFO:tasks.workunit.client.0.smithi120.stdout: NTCreateX 186264 1.485 31618.994 2024-09-19T09:16:05.248 INFO:tasks.workunit.client.0.smithi120.stdout: Close 136806 0.002 0.109 2024-09-19T09:16:05.248 INFO:tasks.workunit.client.0.smithi120.stdout: Rename 7890 7.267 27830.518 2024-09-19T09:16:05.248 INFO:tasks.workunit.client.0.smithi120.stdout: Unlink 37644 2.176 29870.649 2024-09-19T09:16:05.248 INFO:tasks.workunit.client.0.smithi120.stdout: Deltree 4 64.279 80.868 2024-09-19T09:16:05.248 INFO:tasks.workunit.client.0.smithi120.stdout: Mkdir 2 0.005 0.005 2024-09-19T09:16:05.249 INFO:tasks.workunit.client.0.smithi120.stdout: Qpathinfo 168834 0.592 133.307 2024-09-19T09:16:05.249 INFO:tasks.workunit.client.0.smithi120.stdout: Qfileinfo 29552 0.001 0.034 2024-09-19T09:16:05.249 INFO:tasks.workunit.client.0.smithi120.stdout: Qfsinfo 30964 0.147 16.457 2024-09-19T09:16:05.249 INFO:tasks.workunit.client.0.smithi120.stdout: Sfileinfo 15192 1.830 27502.538 2024-09-19T09:16:05.249 INFO:tasks.workunit.client.0.smithi120.stdout: Find 65265 0.068 60.725 2024-09-19T09:16:05.249 INFO:tasks.workunit.client.0.smithi120.stdout: WriteX 92729 0.015 21.921 2024-09-19T09:16:05.249 INFO:tasks.workunit.client.0.smithi120.stdout: ReadX 291855 0.020 26.930 2024-09-19T09:16:05.249 INFO:tasks.workunit.client.0.smithi120.stdout: LockX 606 0.877 12.458 2024-09-19T09:16:05.249 INFO:tasks.workunit.client.0.smithi120.stdout: UnlockX 606 0.722 3.220 2024-09-19T09:16:05.249 INFO:tasks.workunit.client.0.smithi120.stdout: Flush 13065 2.786 178.141 2024-09-19T09:16:05.249 INFO:tasks.workunit.client.0.smithi120.stdout: 2024-09-19T09:16:05.249 INFO:tasks.workunit.client.0.smithi120.stdout:Throughput 9.7329 MB/sec 1 clients 1 procs max_latency=31618.998 ms 2024-09-19T09:16:05.250 INFO:tasks.workunit.client.0.smithi120.stderr:+ dbench 10 2024-09-19T09:16:05.250 INFO:tasks.workunit.client.0.smithi120.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-19T09:16:05.250 INFO:tasks.workunit.client.0.smithi120.stdout: 2024-09-19T09:16:05.250 INFO:tasks.workunit.client.0.smithi120.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-19T09:16:05.250 INFO:tasks.workunit.client.0.smithi120.stdout:8 of 10 processes prepared for launch 0 sec 2024-09-19T09:16:05.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.273+0000 7f4773e48640 1 -- 172.21.15.120:0/2944567482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f476c105ec0 msgr2=0x7f476c106340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:05.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.273+0000 7f4773e48640 1 --2- 172.21.15.120:0/2944567482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f476c105ec0 0x7f476c106340 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f4764009ca0 tx=0x7f476402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:16:05.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.273+0000 7f4773e48640 1 -- 172.21.15.120:0/2944567482 shutdown_connections 2024-09-19T09:16:05.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.273+0000 7f4773e48640 1 --2- 172.21.15.120:0/2944567482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f476c105ec0 0x7f476c106340 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:05.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.273+0000 7f4773e48640 1 --2- 172.21.15.120:0/2944567482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f476c1011e0 0x7f476c105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:05.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.273+0000 7f4773e48640 1 --2- 172.21.15.120:0/2944567482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f476c1008c0 0x7f476c100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:05.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.273+0000 7f4773e48640 1 -- 172.21.15.120:0/2944567482 >> 172.21.15.120:0/2944567482 conn(0x7f476c0fc7d0 msgr2=0x7f476c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:05.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.273+0000 7f4773e48640 1 -- 172.21.15.120:0/2944567482 shutdown_connections 2024-09-19T09:16:05.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.273+0000 7f4773e48640 1 -- 172.21.15.120:0/2944567482 wait complete. 2024-09-19T09:16:05.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.274+0000 7f4773e48640 1 Processor -- start 2024-09-19T09:16:05.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.274+0000 7f4773e48640 1 -- start start 2024-09-19T09:16:05.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.274+0000 7f4773e48640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f476c1008c0 0x7f476c1a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:05.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.275+0000 7f4773e48640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f476c1011e0 0x7f476c1a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:05.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.275+0000 7f4773e48640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f476c105ec0 0x7f476c19e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:05.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.275+0000 7f4773e48640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f476c118630 con 0x7f476c105ec0 2024-09-19T09:16:05.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.275+0000 7f4773e48640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f476c1184b0 con 0x7f476c1008c0 2024-09-19T09:16:05.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.275+0000 7f4773e48640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f476c1187b0 con 0x7f476c1011e0 2024-09-19T09:16:05.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.275+0000 7f47723e5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f476c105ec0 0x7f476c19e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:05.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.275+0000 7f47723e5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f476c105ec0 0x7f476c19e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54608/0 (socket says 172.21.15.120:54608) 2024-09-19T09:16:05.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.275+0000 7f47723e5640 1 -- 172.21.15.120:0/3066597574 learned_addr learned my addr 172.21.15.120:0/3066597574 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:05.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.275+0000 7f47713e3640 1 --2- 172.21.15.120:0/3066597574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f476c1011e0 0x7f476c1a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:05.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.275+0000 7f4771be4640 1 --2- 172.21.15.120:0/3066597574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f476c1008c0 0x7f476c1a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:05.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.275+0000 7f47723e5640 1 -- 172.21.15.120:0/3066597574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f476c1011e0 msgr2=0x7f476c1a4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:05.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.275+0000 7f47723e5640 1 --2- 172.21.15.120:0/3066597574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f476c1011e0 0x7f476c1a4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:05.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.275+0000 7f47723e5640 1 -- 172.21.15.120:0/3066597574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f476c1008c0 msgr2=0x7f476c1a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:05.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.275+0000 7f47723e5640 1 --2- 172.21.15.120:0/3066597574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f476c1008c0 0x7f476c1a47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:05.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.275+0000 7f47723e5640 1 -- 172.21.15.120:0/3066597574 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476c19f090 con 0x7f476c105ec0 2024-09-19T09:16:05.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.275+0000 7f47723e5640 1 --2- 172.21.15.120:0/3066597574 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f476c105ec0 0x7f476c19e960 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f4764006010 tx=0x7f4764003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:05.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.275+0000 7f4762ffd640 1 -- 172.21.15.120:0/3066597574 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4764031650 con 0x7f476c105ec0 2024-09-19T09:16:05.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.275+0000 7f4762ffd640 1 -- 172.21.15.120:0/3066597574 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47640317f0 con 0x7f476c105ec0 2024-09-19T09:16:05.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.275+0000 7f4762ffd640 1 -- 172.21.15.120:0/3066597574 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47640309b0 con 0x7f476c105ec0 2024-09-19T09:16:05.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.275+0000 7f4773e48640 1 -- 172.21.15.120:0/3066597574 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f476c19f320 con 0x7f476c105ec0 2024-09-19T09:16:05.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.275+0000 7f4773e48640 1 -- 172.21.15.120:0/3066597574 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f476c1ab670 con 0x7f476c105ec0 2024-09-19T09:16:05.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.276+0000 7f4773e48640 1 -- 172.21.15.120:0/3066597574 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f476c105980 con 0x7f476c105ec0 2024-09-19T09:16:05.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.278+0000 7f4762ffd640 1 -- 172.21.15.120:0/3066597574 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f4764030b50 con 0x7f476c105ec0 2024-09-19T09:16:05.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.278+0000 7f4762ffd640 1 --2- 172.21.15.120:0/3066597574 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f47580a3240 0x7f47580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:05.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.278+0000 7f4771be4640 1 --2- 172.21.15.120:0/3066597574 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f47580a3240 0x7f47580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:05.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.278+0000 7f4762ffd640 1 -- 172.21.15.120:0/3066597574 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f47640f0e70 con 0x7f476c105ec0 2024-09-19T09:16:05.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.279+0000 7f4771be4640 1 --2- 172.21.15.120:0/3066597574 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f47580a3240 0x7f47580a5700 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f476c19faa0 tx=0x7f4768005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:05.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.281+0000 7f4762ffd640 1 -- 172.21.15.120:0/3066597574 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f47640bdb90 con 0x7f476c105ec0 2024-09-19T09:16:05.350 INFO:tasks.workunit.client.0.smithi120.stdout:10 of 10 processes prepared for launch 0 sec 2024-09-19T09:16:05.350 INFO:tasks.workunit.client.0.smithi120.stdout:releasing clients 2024-09-19T09:16:05.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.498+0000 7f4773e48640 1 -- 172.21.15.120:0/3066597574 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f476c1ab840 con 0x7f476c105ec0 2024-09-19T09:16:05.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.498+0000 7f4762ffd640 1 -- 172.21.15.120:0/3066597574 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11988 (secure 0 0 0) 0x7f47640bd490 con 0x7f476c105ec0 2024-09-19T09:16:05.499 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:16:05.499 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":33,"btime":"2024-09-19T09:14:31:550518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:14:31.550494+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28649":{"gid":28649,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/2810080307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2810080307},{"type":"v1","addr":"172.21.15.153:6835","nonce":2810080307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:16:05.500 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 33 2024-09-19T09:16:05.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.501+0000 7f4773e48640 1 -- 172.21.15.120:0/3066597574 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f47580a3240 msgr2=0x7f47580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:05.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.501+0000 7f4773e48640 1 --2- 172.21.15.120:0/3066597574 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f47580a3240 0x7f47580a5700 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f476c19faa0 tx=0x7f4768005f90 comp rx=0 tx=0).stop 2024-09-19T09:16:05.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.501+0000 7f4773e48640 1 -- 172.21.15.120:0/3066597574 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f476c105ec0 msgr2=0x7f476c19e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:05.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.501+0000 7f4773e48640 1 --2- 172.21.15.120:0/3066597574 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f476c105ec0 0x7f476c19e960 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f4764006010 tx=0x7f4764003ce0 comp rx=0 tx=0).stop 2024-09-19T09:16:05.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.501+0000 7f4773e48640 1 -- 172.21.15.120:0/3066597574 shutdown_connections 2024-09-19T09:16:05.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.501+0000 7f4773e48640 1 --2- 172.21.15.120:0/3066597574 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f47580a3240 0x7f47580a5700 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:05.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.501+0000 7f4773e48640 1 --2- 172.21.15.120:0/3066597574 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f476c105ec0 0x7f476c19e960 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:05.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.501+0000 7f4773e48640 1 --2- 172.21.15.120:0/3066597574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f476c1011e0 0x7f476c1a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:05.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.501+0000 7f4773e48640 1 --2- 172.21.15.120:0/3066597574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f476c1008c0 0x7f476c1a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:05.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.501+0000 7f4773e48640 1 -- 172.21.15.120:0/3066597574 >> 172.21.15.120:0/3066597574 conn(0x7f476c0fc7d0 msgr2=0x7f476c103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:05.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.501+0000 7f4773e48640 1 -- 172.21.15.120:0/3066597574 shutdown_connections 2024-09-19T09:16:05.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:05.501+0000 7f4773e48640 1 -- 172.21.15.120:0/3066597574 wait complete. 2024-09-19T09:16:05.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:05 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3066597574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:16:05.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:05 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3066597574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:16:05.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:05 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3066597574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:16:06.277 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db / --timeout 90 --expiration 142.2 --await-for 45.0' 2024-09-19T09:16:06.277 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db / --timeout 90 --expiration 142.2 --await-for 45.0 2024-09-19T09:16:06.350 INFO:tasks.workunit.client.0.smithi120.stdout: 10 415 146.84 MB/sec warmup 1 sec latency 52.931 ms 2024-09-19T09:16:06.529 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:16:06.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:06 smithi120 ceph-mon[31150]: pgmap v741: 97 pgs: 97 active+clean; 38 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.9 MiB/s wr, 104 op/s 2024-09-19T09:16:06.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:06 smithi196 ceph-mon[35150]: pgmap v741: 97 pgs: 97 active+clean; 38 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.9 MiB/s wr, 104 op/s 2024-09-19T09:16:06.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:06 smithi153 ceph-mon[35071]: pgmap v741: 97 pgs: 97 active+clean; 38 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.9 MiB/s wr, 104 op/s 2024-09-19T09:16:07.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.070+0000 7f5160750640 1 -- 172.21.15.120:0/3447384014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5158100100 msgr2=0x7f5158100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:07.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.070+0000 7f5160750640 1 --2- 172.21.15.120:0/3447384014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5158100100 0x7f5158100560 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f5148009900 tx=0x7f514802a410 comp rx=0 tx=0).stop 2024-09-19T09:16:07.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.070+0000 7f5160750640 1 -- 172.21.15.120:0/3447384014 shutdown_connections 2024-09-19T09:16:07.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.070+0000 7f5160750640 1 --2- 172.21.15.120:0/3447384014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5158100aa0 0x7f515810ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:07.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.070+0000 7f5160750640 1 --2- 172.21.15.120:0/3447384014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5158100100 0x7f5158100560 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:07.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.070+0000 7f5160750640 1 --2- 172.21.15.120:0/3447384014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5158106b80 0x7f5158106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:07.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.070+0000 7f5160750640 1 -- 172.21.15.120:0/3447384014 >> 172.21.15.120:0/3447384014 conn(0x7f5158077230 msgr2=0x7f51580fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:07.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.070+0000 7f5160750640 1 -- 172.21.15.120:0/3447384014 shutdown_connections 2024-09-19T09:16:07.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.070+0000 7f5160750640 1 -- 172.21.15.120:0/3447384014 wait complete. 2024-09-19T09:16:07.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.071+0000 7f5160750640 1 Processor -- start 2024-09-19T09:16:07.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.071+0000 7f5160750640 1 -- start start 2024-09-19T09:16:07.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.071+0000 7f5160750640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5158100100 0x7f51581984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:07.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.071+0000 7f5160750640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5158100aa0 0x7f5158198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:07.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.071+0000 7f5160750640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5158106b80 0x7f515819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:07.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.071+0000 7f5160750640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f51581120a0 con 0x7f5158100100 2024-09-19T09:16:07.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.071+0000 7f5160750640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5158111f20 con 0x7f5158106b80 2024-09-19T09:16:07.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.071+0000 7f5160750640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5158112220 con 0x7f5158100aa0 2024-09-19T09:16:07.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.071+0000 7f515e4ec640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5158100100 0x7f51581984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:07.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.071+0000 7f515e4ec640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5158100100 0x7f51581984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54620/0 (socket says 172.21.15.120:54620) 2024-09-19T09:16:07.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.071+0000 7f515e4ec640 1 -- 172.21.15.120:0/632212982 learned_addr learned my addr 172.21.15.120:0/632212982 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:07.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.072+0000 7f515eced640 1 --2- 172.21.15.120:0/632212982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5158106b80 0x7f515819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:07.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.072+0000 7f515dceb640 1 --2- 172.21.15.120:0/632212982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5158100aa0 0x7f5158198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:07.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.072+0000 7f515e4ec640 1 -- 172.21.15.120:0/632212982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5158100aa0 msgr2=0x7f5158198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:07.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.072+0000 7f515e4ec640 1 --2- 172.21.15.120:0/632212982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5158100aa0 0x7f5158198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:07.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.072+0000 7f515e4ec640 1 -- 172.21.15.120:0/632212982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5158106b80 msgr2=0x7f515819cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:07.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.072+0000 7f515e4ec640 1 --2- 172.21.15.120:0/632212982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5158106b80 0x7f515819cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:07.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.072+0000 7f515e4ec640 1 -- 172.21.15.120:0/632212982 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515819d470 con 0x7f5158100100 2024-09-19T09:16:07.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.072+0000 7f515dceb640 1 --2- 172.21.15.120:0/632212982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5158100aa0 0x7f5158198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:16:07.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.072+0000 7f515eced640 1 --2- 172.21.15.120:0/632212982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5158106b80 0x7f515819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:16:07.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.072+0000 7f515e4ec640 1 --2- 172.21.15.120:0/632212982 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5158100100 0x7f51581984c0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f515400e9f0 tx=0x7f515400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:07.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.072+0000 7f514f7fe640 1 -- 172.21.15.120:0/632212982 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515400cd80 con 0x7f5158100100 2024-09-19T09:16:07.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.072+0000 7f5160750640 1 -- 172.21.15.120:0/632212982 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f515819d760 con 0x7f5158100100 2024-09-19T09:16:07.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.072+0000 7f5160750640 1 -- 172.21.15.120:0/632212982 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5158101210 con 0x7f5158100100 2024-09-19T09:16:07.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.072+0000 7f514f7fe640 1 -- 172.21.15.120:0/632212982 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5154010910 con 0x7f5158100100 2024-09-19T09:16:07.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.072+0000 7f514f7fe640 1 -- 172.21.15.120:0/632212982 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51540188f0 con 0x7f5158100100 2024-09-19T09:16:07.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.075+0000 7f514f7fe640 1 -- 172.21.15.120:0/632212982 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5154018bb0 con 0x7f5158100100 2024-09-19T09:16:07.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.075+0000 7f5162b29740 1 Processor -- start 2024-09-19T09:16:07.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.075+0000 7f5162b29740 1 -- start start 2024-09-19T09:16:07.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.076+0000 7f5162b29740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628b786a760 0x5628b786ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:07.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.076+0000 7f5162b29740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628b786b080 0x5628b7875400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:07.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.076+0000 7f5162b29740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628b7875940 0x5628b7877d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:07.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.076+0000 7f5162b29740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5628b77d02a0 con 0x5628b7875940 2024-09-19T09:16:07.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.076+0000 7f5162b29740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5628b7777b10 con 0x5628b786a760 2024-09-19T09:16:07.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.076+0000 7f5162b29740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5628b77cc260 con 0x5628b786b080 2024-09-19T09:16:07.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.076+0000 7f514f7fe640 1 --2- 172.21.15.120:0/632212982 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f51440a32c0 0x7f51440a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:07.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.076+0000 7f515e4ec640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628b786b080 0x5628b7875400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:07.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.076+0000 7f515eced640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628b786a760 0x5628b786ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:07.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.076+0000 7f515e4ec640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628b786b080 0x5628b7875400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:45890/0 (socket says 172.21.15.120:45890) 2024-09-19T09:16:07.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.076+0000 7f515e4ec640 1 -- 172.21.15.120:0/1218092080 learned_addr learned my addr 172.21.15.120:0/1218092080 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:07.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.076+0000 7f515dceb640 1 --2- 172.21.15.120:0/1218092080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628b7875940 0x5628b7877d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:07.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.076+0000 7f515eced640 1 --2- 172.21.15.120:0/632212982 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f51440a32c0 0x7f51440a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:07.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.076+0000 7f514f7fe640 1 -- 172.21.15.120:0/632212982 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f51540cfc10 con 0x7f5158100100 2024-09-19T09:16:07.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.076+0000 7f515e4ec640 1 -- 172.21.15.120:0/1218092080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628b786a760 msgr2=0x5628b786ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:07.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.076+0000 7f515e4ec640 1 --2- 172.21.15.120:0/1218092080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628b786a760 0x5628b786ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:07.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.076+0000 7f515e4ec640 1 -- 172.21.15.120:0/1218092080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628b7875940 msgr2=0x5628b7877d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:07.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.076+0000 7f515e4ec640 1 --2- 172.21.15.120:0/1218092080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628b7875940 0x5628b7877d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:07.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.076+0000 7f515e4ec640 1 -- 172.21.15.120:0/1218092080 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628b7541b90 con 0x5628b786b080 2024-09-19T09:16:07.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.076+0000 7f515e4ec640 1 --2- 172.21.15.120:0/1218092080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628b786b080 0x5628b7875400 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f51540cd710 tx=0x7f5154024000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b7b9934f1222f3a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:07.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.077+0000 7f514cff9640 1 -- 172.21.15.120:0/1218092080 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515409c230 con 0x5628b786b080 2024-09-19T09:16:07.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.077+0000 7f514cff9640 1 -- 172.21.15.120:0/1218092080 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51540a05e0 con 0x5628b786b080 2024-09-19T09:16:07.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.077+0000 7f514cff9640 1 -- 172.21.15.120:0/1218092080 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51540cdbd0 con 0x5628b786b080 2024-09-19T09:16:07.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.077+0000 7f515eced640 1 --2- 172.21.15.120:0/632212982 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f51440a32c0 0x7f51440a5780 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5150009270 tx=0x7f5150009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:07.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.077+0000 7f5162b29740 1 -- 172.21.15.120:0/1218092080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628b786b080 msgr2=0x5628b7875400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:07.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.077+0000 7f5162b29740 1 --2- 172.21.15.120:0/1218092080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628b786b080 0x5628b7875400 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f51540cd710 tx=0x7f5154024000 comp rx=0 tx=0).stop 2024-09-19T09:16:07.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.077+0000 7f5162b29740 1 -- 172.21.15.120:0/1218092080 shutdown_connections 2024-09-19T09:16:07.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.077+0000 7f5162b29740 1 --2- 172.21.15.120:0/1218092080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628b7875940 0x5628b7877d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:07.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.077+0000 7f5162b29740 1 --2- 172.21.15.120:0/1218092080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628b786b080 0x5628b7875400 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:07.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.077+0000 7f5162b29740 1 --2- 172.21.15.120:0/1218092080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628b786a760 0x5628b786ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:07.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.078+0000 7f5162b29740 1 -- 172.21.15.120:0/1218092080 >> 172.21.15.120:0/1218092080 conn(0x5628b7866cf0 msgr2=0x5628b78670d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:07.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.078+0000 7f5162b29740 1 -- 172.21.15.120:0/1218092080 shutdown_connections 2024-09-19T09:16:07.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.078+0000 7f5162b29740 1 -- 172.21.15.120:0/1218092080 wait complete. 2024-09-19T09:16:07.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.078+0000 7f5162b29740 1 -- start start 2024-09-19T09:16:07.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.078+0000 7f5162b29740 1 Processor -- start 2024-09-19T09:16:07.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.078+0000 7f5162b29740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628b786a760 0x5628b78727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:07.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.078+0000 7f5162b29740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628b786b080 0x5628b7872cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:07.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.078+0000 7f5162b29740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628b7875940 0x5628b7873230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:07.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.078+0000 7f5162b29740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5628b78784b0 con 0x5628b786b080 2024-09-19T09:16:07.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.078+0000 7f5162b29740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5628b77586e0 con 0x5628b786a760 2024-09-19T09:16:07.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.078+0000 7f5162b29740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5628b7878630 con 0x5628b7875940 2024-09-19T09:16:07.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.078+0000 7f5162b29740 10 client.0 _ll_register_callbacks cb 0x5628b777ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:07.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.078+0000 7f515dceb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628b786b080 0x5628b7872cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:07.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.078+0000 7f515e4ec640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628b786a760 0x5628b78727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:07.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.078+0000 7f515dceb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628b786b080 0x5628b7872cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54634/0 (socket says 172.21.15.120:54634) 2024-09-19T09:16:07.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.078+0000 7f515eced640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628b7875940 0x5628b7873230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:07.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.078+0000 7f515dceb640 1 -- 172.21.15.120:0/1656075599 learned_addr learned my addr 172.21.15.120:0/1656075599 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:07.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.079+0000 7f515dceb640 1 -- 172.21.15.120:0/1656075599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628b7875940 msgr2=0x5628b7873230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:07.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.079+0000 7f515dceb640 1 --2- 172.21.15.120:0/1656075599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628b7875940 0x5628b7873230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:07.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.079+0000 7f515dceb640 1 -- 172.21.15.120:0/1656075599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628b786a760 msgr2=0x5628b78727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:07.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.079+0000 7f515dceb640 1 --2- 172.21.15.120:0/1656075599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628b786a760 0x5628b78727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:07.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.079+0000 7f515dceb640 1 -- 172.21.15.120:0/1656075599 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628b7881980 con 0x5628b786b080 2024-09-19T09:16:07.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.079+0000 7f515eced640 1 --2- 172.21.15.120:0/1656075599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628b7875940 0x5628b7873230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:16:07.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.079+0000 7f515e4ec640 1 --2- 172.21.15.120:0/1656075599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628b786a760 0x5628b78727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:16:07.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.079+0000 7f515dceb640 1 --2- 172.21.15.120:0/1656075599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628b786b080 0x5628b7872cf0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f5148009900 tx=0x7f5148005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf58bd14386af6e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:07.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.079+0000 7f5141ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:16:07.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.079+0000 7f5141ffb640 1 -- 172.21.15.120:0/1656075599 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5148035040 con 0x5628b786b080 2024-09-19T09:16:07.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.079+0000 7f5162b29740 1 -- 172.21.15.120:0/1656075599 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51540a05e0 con 0x5628b786b080 2024-09-19T09:16:07.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.079+0000 7f5141ffb640 1 -- 172.21.15.120:0/1656075599 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f51480048c0 con 0x5628b786b080 2024-09-19T09:16:07.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.079+0000 7f5141ffb640 1 -- 172.21.15.120:0/1656075599 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5148005d50 con 0x5628b786b080 2024-09-19T09:16:07.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.079+0000 7f5162b29740 10 client.20085 fetch_fsmap learned FSMap version 33 2024-09-19T09:16:07.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.079+0000 7f5162b29740 1 -- 172.21.15.120:0/1656075599 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5628b7884340 con 0x5628b786b080 2024-09-19T09:16:07.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.080+0000 7f5141ffb640 1 -- 172.21.15.120:0/1656075599 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51540a05e0 con 0x5628b786b080 2024-09-19T09:16:07.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.080+0000 7f5141ffb640 1 -- 172.21.15.120:0/1656075599 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f51480035f0 con 0x5628b786b080 2024-09-19T09:16:07.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.080+0000 7f5162b29740 10 client.20085 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:16:07.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.080+0000 7f5162b29740 10 client.20085 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:16:07.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.080+0000 7f5162b29740 20 client.20085 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:07.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.080+0000 7f5162b29740 1 --2- 172.21.15.120:0/1656075599 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5628b7885800 0x5628b7887c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:07.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.080+0000 7f5162b29740 4 client.20085 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:16:07.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.080+0000 7f5162b29740 1 -- 172.21.15.120:0/1656075599 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628b77dd9d0 con 0x5628b7885800 2024-09-19T09:16:07.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.080+0000 7f515e4ec640 1 --2- 172.21.15.120:0/1656075599 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5628b7885800 0x5628b7887c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:07.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.080+0000 7f5141ffb640 10 client.20085 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:07.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.080+0000 7f515e4ec640 1 --2- 172.21.15.120:0/1656075599 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5628b7885800 0x5628b7887c60 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd5f9cb52ad6fc68 server_cookie=d793530ab4180fcc in_seq=0 out_seq=0 2024-09-19T09:16:07.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.086+0000 7f5141ffb640 1 -- 172.21.15.120:0/1656075599 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5628b77dd9d0 con 0x5628b7885800 2024-09-19T09:16:07.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.086+0000 7f5141ffb640 10 client.20085 handle_command_reply: tid=0 2024-09-19T09:16:07.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.086+0000 7f5162b29740 1 client.20085 shutdown 2024-09-19T09:16:07.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.086+0000 7f5162b29740 1 -- 172.21.15.120:0/1656075599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628b786b080 msgr2=0x5628b7872cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:07.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.086+0000 7f5162b29740 1 --2- 172.21.15.120:0/1656075599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628b786b080 0x5628b7872cf0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f5148009900 tx=0x7f5148005270 comp rx=0 tx=0).stop 2024-09-19T09:16:07.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.087+0000 7f5162b29740 1 -- 172.21.15.120:0/1656075599 shutdown_connections 2024-09-19T09:16:07.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.087+0000 7f5162b29740 1 --2- 172.21.15.120:0/1656075599 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5628b7885800 0x5628b7887c60 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:07.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.087+0000 7f5162b29740 1 --2- 172.21.15.120:0/1656075599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628b7875940 0x5628b7873230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:07.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.087+0000 7f5162b29740 1 --2- 172.21.15.120:0/1656075599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628b786b080 0x5628b7872cf0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:07.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.087+0000 7f5162b29740 1 --2- 172.21.15.120:0/1656075599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628b786a760 0x5628b78727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:07.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.087+0000 7f5162b29740 1 -- 172.21.15.120:0/1656075599 >> 172.21.15.120:0/1656075599 conn(0x5628b7866cf0 msgr2=0x5628b7865e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:07.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.087+0000 7f5141ffb640 0 client.20085 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:07.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.087+0000 7f5162b29740 1 -- 172.21.15.120:0/1656075599 shutdown_connections 2024-09-19T09:16:07.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.087+0000 7f5162b29740 1 -- 172.21.15.120:0/1656075599 wait complete. 2024-09-19T09:16:07.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.087+0000 7f5162b29740 20 client.20085 trim_cache size 0 max 16384 2024-09-19T09:16:07.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.103+0000 7f5162b29740 1 Processor -- start 2024-09-19T09:16:07.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.104+0000 7f5162b29740 1 -- start start 2024-09-19T09:16:07.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.104+0000 7f5162b29740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628b786a760 0x5628b786d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:07.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.104+0000 7f5162b29740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628b786b080 0x5628b7871080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:07.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.104+0000 7f5162b29740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628b7875940 0x5628b788d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:07.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.104+0000 7f5162b29740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f513c00e4a0 con 0x5628b7875940 2024-09-19T09:16:07.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.104+0000 7f5162b29740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f513c00ffe0 con 0x5628b786b080 2024-09-19T09:16:07.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.104+0000 7f5162b29740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f513c00eb70 con 0x5628b786a760 2024-09-19T09:16:07.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.104+0000 7f515eced640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628b7875940 0x5628b788d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:07.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.104+0000 7f515eced640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628b7875940 0x5628b788d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54642/0 (socket says 172.21.15.120:54642) 2024-09-19T09:16:07.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.104+0000 7f515eced640 1 -- 172.21.15.120:0/1930500324 learned_addr learned my addr 172.21.15.120:0/1930500324 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:07.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.104+0000 7f515dceb640 1 --2- 172.21.15.120:0/1930500324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628b786b080 0x5628b7871080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:07.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.104+0000 7f515e4ec640 1 --2- 172.21.15.120:0/1930500324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628b786a760 0x5628b786d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:07.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.104+0000 7f515eced640 1 -- 172.21.15.120:0/1930500324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628b786a760 msgr2=0x5628b786d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:07.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.104+0000 7f515eced640 1 --2- 172.21.15.120:0/1930500324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628b786a760 0x5628b786d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:07.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.105+0000 7f515eced640 1 -- 172.21.15.120:0/1930500324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628b786b080 msgr2=0x5628b7871080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:07.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.105+0000 7f515eced640 1 --2- 172.21.15.120:0/1930500324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628b786b080 0x5628b7871080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:07.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.105+0000 7f515eced640 1 -- 172.21.15.120:0/1930500324 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628b786e160 con 0x5628b7875940 2024-09-19T09:16:07.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.105+0000 7f515e4ec640 1 --2- 172.21.15.120:0/1930500324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628b786a760 0x5628b786d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:16:07.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.105+0000 7f515eced640 1 --2- 172.21.15.120:0/1930500324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628b7875940 0x5628b788d570 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f515000dc70 tx=0x7f515000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ced0bfee9a9b69 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:07.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.105+0000 7f5143fff640 1 -- 172.21.15.120:0/1930500324 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515002e070 con 0x5628b7875940 2024-09-19T09:16:07.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.105+0000 7f5143fff640 1 -- 172.21.15.120:0/1930500324 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5150029960 con 0x5628b7875940 2024-09-19T09:16:07.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.105+0000 7f5143fff640 1 -- 172.21.15.120:0/1930500324 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515001d8a0 con 0x5628b7875940 2024-09-19T09:16:07.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.106+0000 7f5162b29740 1 -- 172.21.15.120:0/1930500324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628b7875940 msgr2=0x5628b788d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:07.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.106+0000 7f5162b29740 1 --2- 172.21.15.120:0/1930500324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628b7875940 0x5628b788d570 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f515000dc70 tx=0x7f515000f000 comp rx=0 tx=0).stop 2024-09-19T09:16:07.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.106+0000 7f5162b29740 1 -- 172.21.15.120:0/1930500324 shutdown_connections 2024-09-19T09:16:07.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.106+0000 7f5162b29740 1 --2- 172.21.15.120:0/1930500324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628b7875940 0x5628b788d570 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:07.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.106+0000 7f5162b29740 1 --2- 172.21.15.120:0/1930500324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628b786b080 0x5628b7871080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:07.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.106+0000 7f5162b29740 1 --2- 172.21.15.120:0/1930500324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628b786a760 0x5628b786d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:07.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.106+0000 7f5162b29740 1 -- 172.21.15.120:0/1930500324 >> 172.21.15.120:0/1930500324 conn(0x5628b7866cf0 msgr2=0x5628b7874830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:07.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.106+0000 7f5162b29740 1 -- 172.21.15.120:0/1930500324 shutdown_connections 2024-09-19T09:16:07.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.106+0000 7f5162b29740 1 -- 172.21.15.120:0/1930500324 wait complete. 2024-09-19T09:16:07.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.106+0000 7f5162b29740 1 -- start start 2024-09-19T09:16:07.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.106+0000 7f5162b29740 1 Processor -- start 2024-09-19T09:16:07.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.107+0000 7f5162b29740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628b786a760 0x5628b7879a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:07.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.107+0000 7f5162b29740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628b786b080 0x5628b7879f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:07.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.107+0000 7f5162b29740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628b7875940 0x5628b78ef540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:07.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.107+0000 7f5162b29740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f513c00f910 con 0x5628b786b080 2024-09-19T09:16:07.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.107+0000 7f5162b29740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f513c00f240 con 0x5628b786a760 2024-09-19T09:16:07.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.107+0000 7f5162b29740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f513c00c460 con 0x5628b7875940 2024-09-19T09:16:07.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.107+0000 7f5162b29740 10 client.0 _ll_register_callbacks cb 0x7f5148004470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:07.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.107+0000 7f515e4ec640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628b786a760 0x5628b7879a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:07.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.107+0000 7f515e4ec640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628b786a760 0x5628b7879a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:48262/0 (socket says 172.21.15.120:48262) 2024-09-19T09:16:07.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.107+0000 7f515e4ec640 1 -- 172.21.15.120:0/805828486 learned_addr learned my addr 172.21.15.120:0/805828486 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:07.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.107+0000 7f515dceb640 1 --2- 172.21.15.120:0/805828486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628b786b080 0x5628b7879f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:07.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.107+0000 7f515eced640 1 --2- 172.21.15.120:0/805828486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628b7875940 0x5628b78ef540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:07.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.107+0000 7f515e4ec640 1 -- 172.21.15.120:0/805828486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628b7875940 msgr2=0x5628b78ef540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:07.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.107+0000 7f515e4ec640 1 --2- 172.21.15.120:0/805828486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628b7875940 0x5628b78ef540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:07.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.107+0000 7f515e4ec640 1 -- 172.21.15.120:0/805828486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628b786b080 msgr2=0x5628b7879f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:07.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.107+0000 7f515e4ec640 1 --2- 172.21.15.120:0/805828486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628b786b080 0x5628b7879f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:07.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.107+0000 7f515e4ec640 1 -- 172.21.15.120:0/805828486 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628b7871760 con 0x5628b786a760 2024-09-19T09:16:07.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.107+0000 7f515e4ec640 1 --2- 172.21.15.120:0/805828486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628b786a760 0x5628b7879a40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5154098c00 tx=0x7f5154000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=338499c83f06c59a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:07.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.107+0000 7f5141ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:16:07.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.108+0000 7f5141ffb640 1 -- 172.21.15.120:0/805828486 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51540a05e0 con 0x5628b786a760 2024-09-19T09:16:07.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.108+0000 7f5141ffb640 1 -- 172.21.15.120:0/805828486 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f51540bc0b0 con 0x5628b786a760 2024-09-19T09:16:07.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.108+0000 7f5162b29740 1 -- 172.21.15.120:0/805828486 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5150029960 con 0x5628b786a760 2024-09-19T09:16:07.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.108+0000 7f5141ffb640 1 -- 172.21.15.120:0/805828486 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f51540bc2b0 con 0x5628b786a760 2024-09-19T09:16:07.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.108+0000 7f5162b29740 10 client.29069 fetch_fsmap learned FSMap version 33 2024-09-19T09:16:07.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.108+0000 7f5162b29740 1 -- 172.21.15.120:0/805828486 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5628b786aeb0 con 0x5628b786a760 2024-09-19T09:16:07.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.108+0000 7f5141ffb640 1 -- 172.21.15.120:0/805828486 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5150029960 con 0x5628b786a760 2024-09-19T09:16:07.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.108+0000 7f5141ffb640 1 -- 172.21.15.120:0/805828486 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f51540a01a0 con 0x5628b786a760 2024-09-19T09:16:07.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.108+0000 7f5162b29740 10 client.29069 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:16:07.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.108+0000 7f5162b29740 10 client.29069 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:16:07.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.108+0000 7f5162b29740 20 client.29069 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:07.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.108+0000 7f5162b29740 1 --2- 172.21.15.120:0/805828486 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5628b78f0ba0 0x5628b78f2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:07.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.108+0000 7f5162b29740 4 client.29069 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.2, "await_for": 45.0}] 2024-09-19T09:16:07.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.108+0000 7f5162b29740 1 -- 172.21.15.120:0/805828486 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.2, "await_for": 45.0}) -- 0x5628b77d7d20 con 0x5628b78f0ba0 2024-09-19T09:16:07.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.109+0000 7f515dceb640 1 --2- 172.21.15.120:0/805828486 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5628b78f0ba0 0x5628b78f2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:07.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.109+0000 7f515dceb640 1 --2- 172.21.15.120:0/805828486 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5628b78f0ba0 0x5628b78f2f90 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97ba412dab33c87a server_cookie=bb03fe939a701fce in_seq=0 out_seq=0 2024-09-19T09:16:07.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:07.109+0000 7f5141ffb640 10 client.29069 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:07.350 INFO:tasks.workunit.client.0.smithi120.stdout: 10 794 107.18 MB/sec warmup 2 sec latency 245.408 ms 2024-09-19T09:16:08.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:08.183+0000 7f5141ffb640 1 -- 172.21.15.120:0/805828486 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5628b77d7d20 con 0x5628b78f0ba0 2024-09-19T09:16:08.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:08.183+0000 7f5141ffb640 10 client.29069 handle_command_reply: tid=0 2024-09-19T09:16:08.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:08.183+0000 7f5162b29740 1 client.29069 shutdown 2024-09-19T09:16:08.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:08.183+0000 7f5162b29740 1 -- 172.21.15.120:0/805828486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628b786a760 msgr2=0x5628b7879a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:08.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:08.183+0000 7f5162b29740 1 --2- 172.21.15.120:0/805828486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628b786a760 0x5628b7879a40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5154098c00 tx=0x7f5154000bc0 comp rx=0 tx=0).stop 2024-09-19T09:16:08.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:08.183+0000 7f5162b29740 1 -- 172.21.15.120:0/805828486 shutdown_connections 2024-09-19T09:16:08.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:08.183+0000 7f5162b29740 1 --2- 172.21.15.120:0/805828486 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5628b78f0ba0 0x5628b78f2f90 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:08.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:08.183+0000 7f5162b29740 1 --2- 172.21.15.120:0/805828486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628b7875940 0x5628b78ef540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:08.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:08.183+0000 7f5162b29740 1 --2- 172.21.15.120:0/805828486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628b786b080 0x5628b7879f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:08.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:08.183+0000 7f5162b29740 1 --2- 172.21.15.120:0/805828486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628b786a760 0x5628b7879a40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:08.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:08.183+0000 7f5141ffb640 0 client.29069 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:08.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:08.183+0000 7f5162b29740 1 -- 172.21.15.120:0/805828486 >> 172.21.15.120:0/805828486 conn(0x5628b7866cf0 msgr2=0x5628b7865650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:08.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:08.183+0000 7f5162b29740 1 -- 172.21.15.120:0/805828486 shutdown_connections 2024-09-19T09:16:08.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:08.183+0000 7f5162b29740 1 -- 172.21.15.120:0/805828486 wait complete. 2024-09-19T09:16:08.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:08.183+0000 7f5162b29740 20 client.29069 trim_cache size 0 max 16384 2024-09-19T09:16:08.186 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:16:08.186 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 33, 2024-09-19T09:16:08.186 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:16:08.186 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 46, 2024-09-19T09:16:08.186 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:16:08.186 INFO:teuthology.orchestra.run.smithi120.stdout: "1e2662fa": { 2024-09-19T09:16:08.187 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 46, 2024-09-19T09:16:08.187 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:16:08.187 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:16:08.187 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:16:08.187 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:16:08.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:16:08.187 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:16:08.187 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 142.2, 2024-09-19T09:16:08.187 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:16:08.187 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:16:08.187 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:16:08.187 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:16:08.187 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:16:08.188 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:16:08.188 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:16:08.188 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:16:08.188 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:16:08.188 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:16:08.188 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:16:08.188 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:16:08.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:08.184+0000 7f5160750640 1 -- 172.21.15.120:0/632212982 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f51440a32c0 msgr2=0x7f51440a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:08.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:08.184+0000 7f5160750640 1 --2- 172.21.15.120:0/632212982 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f51440a32c0 0x7f51440a5780 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5150009270 tx=0x7f5150009500 comp rx=0 tx=0).stop 2024-09-19T09:16:08.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:08.184+0000 7f5160750640 1 -- 172.21.15.120:0/632212982 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5158100100 msgr2=0x7f51581984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:08.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:08.184+0000 7f5160750640 1 --2- 172.21.15.120:0/632212982 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5158100100 0x7f51581984c0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f515400e9f0 tx=0x7f515400eeb0 comp rx=0 tx=0).stop 2024-09-19T09:16:08.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:08.184+0000 7f5160750640 1 -- 172.21.15.120:0/632212982 shutdown_connections 2024-09-19T09:16:08.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:08.184+0000 7f5160750640 1 --2- 172.21.15.120:0/632212982 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f51440a32c0 0x7f51440a5780 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:08.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:08.184+0000 7f5160750640 1 --2- 172.21.15.120:0/632212982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5158106b80 0x7f515819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:08.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:08.184+0000 7f5160750640 1 --2- 172.21.15.120:0/632212982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5158100aa0 0x7f5158198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:08.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:08.184+0000 7f5160750640 1 --2- 172.21.15.120:0/632212982 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5158100100 0x7f51581984c0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:08.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:08.184+0000 7f5160750640 1 -- 172.21.15.120:0/632212982 >> 172.21.15.120:0/632212982 conn(0x7f5158077230 msgr2=0x7f5158104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:08.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:08.184+0000 7f5160750640 1 -- 172.21.15.120:0/632212982 shutdown_connections 2024-09-19T09:16:08.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:08.184+0000 7f5160750640 1 -- 172.21.15.120:0/632212982 wait complete. 2024-09-19T09:16:08.350 INFO:tasks.workunit.client.0.smithi120.stdout: 10 795 71.45 MB/sec warmup 3 sec latency 1205.025 ms 2024-09-19T09:16:09.092 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1e2662fa', quiesce took 4.6 seconds. Will release after: 17.6 2024-09-19T09:16:09.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:08 smithi196 ceph-mon[35150]: pgmap v742: 97 pgs: 97 active+clean; 33 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 157 op/s 2024-09-19T09:16:09.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:08 smithi153 ceph-mon[35071]: pgmap v742: 97 pgs: 97 active+clean; 33 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 157 op/s 2024-09-19T09:16:09.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:08 smithi120 ceph-mon[31150]: pgmap v742: 97 pgs: 97 active+clean; 33 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 157 op/s 2024-09-19T09:16:09.350 INFO:tasks.workunit.client.0.smithi120.stdout: 10 795 53.59 MB/sec warmup 4 sec latency 2205.108 ms 2024-09-19T09:16:10.350 INFO:tasks.workunit.client.0.smithi120.stdout: 10 795 42.87 MB/sec warmup 5 sec latency 3205.206 ms 2024-09-19T09:16:11.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:10 smithi196 ceph-mon[35150]: pgmap v743: 97 pgs: 97 active+clean; 25 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 244 op/s 2024-09-19T09:16:11.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:10 smithi153 ceph-mon[35071]: pgmap v743: 97 pgs: 97 active+clean; 25 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 244 op/s 2024-09-19T09:16:11.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:10 smithi120 ceph-mon[31150]: pgmap v743: 97 pgs: 97 active+clean; 25 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 244 op/s 2024-09-19T09:16:11.350 INFO:tasks.workunit.client.0.smithi120.stdout: 10 795 35.73 MB/sec warmup 6 sec latency 4205.308 ms 2024-09-19T09:16:12.350 INFO:tasks.workunit.client.0.smithi120.stdout: 10 795 30.62 MB/sec warmup 7 sec latency 5205.407 ms 2024-09-19T09:16:13.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:12 smithi153 ceph-mon[35071]: pgmap v744: 97 pgs: 97 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 250 op/s 2024-09-19T09:16:13.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:12 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:16:13.189 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:12 smithi196 ceph-mon[35150]: pgmap v744: 97 pgs: 97 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 250 op/s 2024-09-19T09:16:13.189 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:12 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:16:13.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:12 smithi120 ceph-mon[31150]: pgmap v744: 97 pgs: 97 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 250 op/s 2024-09-19T09:16:13.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:12 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:16:13.350 INFO:tasks.workunit.client.0.smithi120.stdout: 10 795 26.79 MB/sec warmup 8 sec latency 6205.455 ms 2024-09-19T09:16:14.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:13 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:16:14.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:13 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:16:14.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:13 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:16:14.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:13 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:16:14.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:13 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:16:14.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:13 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:16:14.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:13 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:16:14.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:13 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:16:14.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:13 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:16:14.351 INFO:tasks.workunit.client.0.smithi120.stdout: 10 795 23.82 MB/sec warmup 9 sec latency 7205.505 ms 2024-09-19T09:16:15.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:14 smithi196 ceph-mon[35150]: pgmap v745: 97 pgs: 97 active+clean; 29 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 248 op/s 2024-09-19T09:16:15.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:14 smithi153 ceph-mon[35071]: pgmap v745: 97 pgs: 97 active+clean; 29 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 248 op/s 2024-09-19T09:16:15.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:14 smithi120 ceph-mon[31150]: pgmap v745: 97 pgs: 97 active+clean; 29 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 248 op/s 2024-09-19T09:16:15.351 INFO:tasks.workunit.client.0.smithi120.stdout: 10 795 21.44 MB/sec warmup 10 sec latency 8205.581 ms 2024-09-19T09:16:16.351 INFO:tasks.workunit.client.0.smithi120.stdout: 10 795 19.49 MB/sec warmup 11 sec latency 9205.681 ms 2024-09-19T09:16:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:16 smithi196 ceph-mon[35150]: pgmap v746: 97 pgs: 97 active+clean; 29 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 235 op/s 2024-09-19T09:16:17.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:16 smithi153 ceph-mon[35071]: pgmap v746: 97 pgs: 97 active+clean; 29 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 235 op/s 2024-09-19T09:16:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.300+0000 7f3bbc7e4640 1 -- 172.21.15.120:0/53206053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3bb4100a60 msgr2=0x7f3bb4100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.300+0000 7f3bbc7e4640 1 --2- 172.21.15.120:0/53206053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3bb4100a60 0x7f3bb4100e60 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009900 tx=0x7f3bb002a410 comp rx=0 tx=0).stop 2024-09-19T09:16:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.300+0000 7f3bbc7e4640 1 -- 172.21.15.120:0/53206053 shutdown_connections 2024-09-19T09:16:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.300+0000 7f3bbc7e4640 1 --2- 172.21.15.120:0/53206053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bb4102610 0x7f3bb4108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.300+0000 7f3bbc7e4640 1 --2- 172.21.15.120:0/53206053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3bb4101c50 0x7f3bb41020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.300+0000 7f3bbc7e4640 1 --2- 172.21.15.120:0/53206053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3bb4100a60 0x7f3bb4100e60 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.300+0000 7f3bbc7e4640 1 -- 172.21.15.120:0/53206053 >> 172.21.15.120:0/53206053 conn(0x7f3bb40fc1e0 msgr2=0x7f3bb40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.300+0000 7f3bbc7e4640 1 -- 172.21.15.120:0/53206053 shutdown_connections 2024-09-19T09:16:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.300+0000 7f3bbc7e4640 1 -- 172.21.15.120:0/53206053 wait complete. 2024-09-19T09:16:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.301+0000 7f3bbc7e4640 1 Processor -- start 2024-09-19T09:16:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.301+0000 7f3bbc7e4640 1 -- start start 2024-09-19T09:16:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.301+0000 7f3bbc7e4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3bb4100a60 0x7f3bb4196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.301+0000 7f3bbc7e4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bb4101c50 0x7f3bb4197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.301+0000 7f3bbc7e4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3bb4102610 0x7f3bb419e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.301+0000 7f3bbc7e4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb410c790 con 0x7f3bb4102610 2024-09-19T09:16:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.301+0000 7f3bbc7e4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb410c610 con 0x7f3bb4100a60 2024-09-19T09:16:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.301+0000 7f3bbc7e4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb410c910 con 0x7f3bb4101c50 2024-09-19T09:16:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.301+0000 7f3bbad81640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3bb4102610 0x7f3bb419e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.301+0000 7f3bbad81640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3bb4102610 0x7f3bb419e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54860/0 (socket says 172.21.15.120:54860) 2024-09-19T09:16:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.301+0000 7f3bbad81640 1 -- 172.21.15.120:0/2713856245 learned_addr learned my addr 172.21.15.120:0/2713856245 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.302+0000 7f3bba580640 1 --2- 172.21.15.120:0/2713856245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3bb4100a60 0x7f3bb4196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.302+0000 7f3bb9d7f640 1 --2- 172.21.15.120:0/2713856245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bb4101c50 0x7f3bb4197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.302+0000 7f3bbad81640 1 -- 172.21.15.120:0/2713856245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bb4101c50 msgr2=0x7f3bb4197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.302+0000 7f3bbad81640 1 --2- 172.21.15.120:0/2713856245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bb4101c50 0x7f3bb4197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.302+0000 7f3bbad81640 1 -- 172.21.15.120:0/2713856245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3bb4100a60 msgr2=0x7f3bb4196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.302+0000 7f3bbad81640 1 --2- 172.21.15.120:0/2713856245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3bb4100a60 0x7f3bb4196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.302+0000 7f3bbad81640 1 -- 172.21.15.120:0/2713856245 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb419e970 con 0x7f3bb4102610 2024-09-19T09:16:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.302+0000 7f3bbad81640 1 --2- 172.21.15.120:0/2713856245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3bb4102610 0x7f3bb419e300 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f3bac00db90 tx=0x7f3bac00dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.302+0000 7f3bab7fe640 1 -- 172.21.15.120:0/2713856245 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac00e980 con 0x7f3bb4102610 2024-09-19T09:16:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.302+0000 7f3bab7fe640 1 -- 172.21.15.120:0/2713856245 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bac009580 con 0x7f3bb4102610 2024-09-19T09:16:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.302+0000 7f3bab7fe640 1 -- 172.21.15.120:0/2713856245 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac016650 con 0x7f3bb4102610 2024-09-19T09:16:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.302+0000 7f3bbc7e4640 1 -- 172.21.15.120:0/2713856245 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb406a300 con 0x7f3bb4102610 2024-09-19T09:16:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.302+0000 7f3bbc7e4640 1 -- 172.21.15.120:0/2713856245 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb406a880 con 0x7f3bb4102610 2024-09-19T09:16:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.304+0000 7f3bab7fe640 1 -- 172.21.15.120:0/2713856245 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f3bac009730 con 0x7f3bb4102610 2024-09-19T09:16:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.305+0000 7f3bbf65c740 1 Processor -- start 2024-09-19T09:16:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.305+0000 7f3bbf65c740 1 -- start start 2024-09-19T09:16:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.305+0000 7f3bbf65c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a24b624db0 0x55a24b6251b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.305+0000 7f3bbf65c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a24b6256f0 0x55a24b62a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.305+0000 7f3bbf65c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a24b62a780 0x55a24b62cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.305+0000 7f3bbf65c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a24b5842f0 con 0x55a24b6256f0 2024-09-19T09:16:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.305+0000 7f3bbf65c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a24b52c270 con 0x55a24b62a780 2024-09-19T09:16:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.305+0000 7f3bbf65c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a24b5802b0 con 0x55a24b624db0 2024-09-19T09:16:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.305+0000 7f3bba580640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a24b6256f0 0x55a24b62a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.305+0000 7f3bba580640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a24b6256f0 0x55a24b62a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54874/0 (socket says 172.21.15.120:54874) 2024-09-19T09:16:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.305+0000 7f3bba580640 1 -- 172.21.15.120:0/2940076128 learned_addr learned my addr 172.21.15.120:0/2940076128 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.305+0000 7f3bab7fe640 1 --2- 172.21.15.120:0/2713856245 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3ba00a32f0 0x7f3ba00a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.305+0000 7f3bb9d7f640 1 --2- 172.21.15.120:0/2940076128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a24b624db0 0x55a24b6251b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.305+0000 7f3bb9d7f640 1 --2- 172.21.15.120:0/2940076128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a24b62a780 0x55a24b62cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.306+0000 7f3bbad81640 1 --2- 172.21.15.120:0/2713856245 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3ba00a32f0 0x7f3ba00a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.306+0000 7f3bba580640 1 -- 172.21.15.120:0/2940076128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a24b624db0 msgr2=0x55a24b6251b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.306+0000 7f3bba580640 1 --2- 172.21.15.120:0/2940076128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a24b624db0 0x55a24b6251b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.306+0000 7f3bba580640 1 -- 172.21.15.120:0/2940076128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a24b62a780 msgr2=0x55a24b62cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.306+0000 7f3bba580640 1 --2- 172.21.15.120:0/2940076128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a24b62a780 0x55a24b62cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.306+0000 7f3bba580640 1 -- 172.21.15.120:0/2940076128 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a24b52b340 con 0x55a24b6256f0 2024-09-19T09:16:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.306+0000 7f3bab7fe640 1 -- 172.21.15.120:0/2713856245 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f3bac0d0ff0 con 0x7f3bb4102610 2024-09-19T09:16:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.306+0000 7f3bba580640 1 --2- 172.21.15.120:0/2940076128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a24b6256f0 0x55a24b62a1b0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009900 tx=0x7f3bb0004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ea8248d6755c3c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.306+0000 7f3bbad81640 1 --2- 172.21.15.120:0/2713856245 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3ba00a32f0 0x7f3ba00a57b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3bac0cebe0 tx=0x7f3bac06c3f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.306+0000 7f3ba8ff9640 1 -- 172.21.15.120:0/2940076128 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb003f070 con 0x55a24b6256f0 2024-09-19T09:16:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.306+0000 7f3ba8ff9640 1 -- 172.21.15.120:0/2940076128 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bb0005500 con 0x55a24b6256f0 2024-09-19T09:16:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.306+0000 7f3ba8ff9640 1 -- 172.21.15.120:0/2940076128 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0035070 con 0x55a24b6256f0 2024-09-19T09:16:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.307+0000 7f3bbf65c740 1 -- 172.21.15.120:0/2940076128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a24b6256f0 msgr2=0x55a24b62a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.307+0000 7f3bbf65c740 1 --2- 172.21.15.120:0/2940076128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a24b6256f0 0x55a24b62a1b0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009900 tx=0x7f3bb0004170 comp rx=0 tx=0).stop 2024-09-19T09:16:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.307+0000 7f3bbf65c740 1 -- 172.21.15.120:0/2940076128 shutdown_connections 2024-09-19T09:16:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.307+0000 7f3bbf65c740 1 --2- 172.21.15.120:0/2940076128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a24b62a780 0x55a24b62cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.307+0000 7f3bbf65c740 1 --2- 172.21.15.120:0/2940076128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a24b6256f0 0x55a24b62a1b0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.307+0000 7f3bbf65c740 1 --2- 172.21.15.120:0/2940076128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a24b624db0 0x55a24b6251b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.307+0000 7f3bbf65c740 1 -- 172.21.15.120:0/2940076128 >> 172.21.15.120:0/2940076128 conn(0x55a24b61c010 msgr2=0x55a24b61c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.307+0000 7f3bbf65c740 1 -- 172.21.15.120:0/2940076128 shutdown_connections 2024-09-19T09:16:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.307+0000 7f3bbf65c740 1 -- 172.21.15.120:0/2940076128 wait complete. 2024-09-19T09:16:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.307+0000 7f3bbf65c740 1 -- start start 2024-09-19T09:16:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.307+0000 7f3bbf65c740 1 Processor -- start 2024-09-19T09:16:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.308+0000 7f3bbf65c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a24b624db0 0x55a24b62f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.308+0000 7f3bbf65c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a24b6256f0 0x55a24b62f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.308+0000 7f3bbf65c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a24b62a780 0x55a24b6365c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.308+0000 7f3bbf65c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a24b62d6f0 con 0x55a24b6256f0 2024-09-19T09:16:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.308+0000 7f3bbf65c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a24b50d290 con 0x55a24b62a780 2024-09-19T09:16:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.308+0000 7f3bbf65c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a24b62d870 con 0x55a24b624db0 2024-09-19T09:16:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.308+0000 7f3bbf65c740 10 client.0 _ll_register_callbacks cb 0x55a24b53e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.308+0000 7f3bba580640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a24b6256f0 0x55a24b62f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.308+0000 7f3bba580640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a24b6256f0 0x55a24b62f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54884/0 (socket says 172.21.15.120:54884) 2024-09-19T09:16:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.308+0000 7f3bba580640 1 -- 172.21.15.120:0/3096099813 learned_addr learned my addr 172.21.15.120:0/3096099813 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.308+0000 7f3bb9d7f640 1 --2- 172.21.15.120:0/3096099813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a24b624db0 0x55a24b62f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.308+0000 7f3bb9d7f640 1 --2- 172.21.15.120:0/3096099813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a24b62a780 0x55a24b6365c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.308+0000 7f3bba580640 1 -- 172.21.15.120:0/3096099813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a24b624db0 msgr2=0x55a24b62f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.308+0000 7f3bba580640 1 --2- 172.21.15.120:0/3096099813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a24b624db0 0x55a24b62f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.308+0000 7f3bba580640 1 -- 172.21.15.120:0/3096099813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a24b62a780 msgr2=0x55a24b6365c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.308+0000 7f3bba580640 1 --2- 172.21.15.120:0/3096099813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a24b62a780 0x55a24b6365c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.308+0000 7f3bba580640 1 -- 172.21.15.120:0/3096099813 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a24b297210 con 0x55a24b6256f0 2024-09-19T09:16:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.308+0000 7f3bba580640 1 --2- 172.21.15.120:0/3096099813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a24b6256f0 0x55a24b62f580 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0031440 tx=0x7f3bb0005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eadd95e747ce3ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.308+0000 7f3b9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:16:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.308+0000 7f3b9dffb640 1 -- 172.21.15.120:0/3096099813 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb003f070 con 0x55a24b6256f0 2024-09-19T09:16:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.308+0000 7f3b9dffb640 1 -- 172.21.15.120:0/3096099813 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f3bb00316b0 con 0x55a24b6256f0 2024-09-19T09:16:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.308+0000 7f3bbf65c740 1 -- 172.21.15.120:0/3096099813 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bb0005500 con 0x55a24b6256f0 2024-09-19T09:16:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.309+0000 7f3b9dffb640 1 -- 172.21.15.120:0/3096099813 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3bb0035040 con 0x55a24b6256f0 2024-09-19T09:16:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.309+0000 7f3bbf65c740 10 client.20103 fetch_fsmap learned FSMap version 33 2024-09-19T09:16:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.309+0000 7f3bbf65c740 1 -- 172.21.15.120:0/3096099813 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a24b363620 con 0x55a24b6256f0 2024-09-19T09:16:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.309+0000 7f3b9dffb640 1 -- 172.21.15.120:0/3096099813 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bb0005500 con 0x55a24b6256f0 2024-09-19T09:16:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.309+0000 7f3b9dffb640 1 -- 172.21.15.120:0/3096099813 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f3bb0003b30 con 0x55a24b6256f0 2024-09-19T09:16:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.309+0000 7f3bbf65c740 10 client.20103 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:16:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.309+0000 7f3bbf65c740 10 client.20103 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:16:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.309+0000 7f3bbf65c740 20 client.20103 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.309+0000 7f3bbf65c740 1 --2- 172.21.15.120:0/3096099813 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a24b63a8c0 0x55a24b63ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.309+0000 7f3bbf65c740 4 client.20103 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:16:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.309+0000 7f3bbf65c740 1 -- 172.21.15.120:0/3096099813 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a24b591a20 con 0x55a24b63a8c0 2024-09-19T09:16:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.309+0000 7f3bb9d7f640 1 --2- 172.21.15.120:0/3096099813 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a24b63a8c0 0x55a24b63ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.310+0000 7f3b9dffb640 10 client.20103 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.310+0000 7f3bb9d7f640 1 --2- 172.21.15.120:0/3096099813 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a24b63a8c0 0x55a24b63ccb0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10914d172473a20 server_cookie=81684a0746094b4e in_seq=0 out_seq=0 2024-09-19T09:16:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.310+0000 7f3b9dffb640 1 -- 172.21.15.120:0/3096099813 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a24b591a20 con 0x55a24b63a8c0 2024-09-19T09:16:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.310+0000 7f3b9dffb640 10 client.20103 handle_command_reply: tid=0 2024-09-19T09:16:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.310+0000 7f3bbf65c740 1 client.20103 shutdown 2024-09-19T09:16:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.310+0000 7f3bbf65c740 1 -- 172.21.15.120:0/3096099813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a24b6256f0 msgr2=0x55a24b62f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.310+0000 7f3bbf65c740 1 --2- 172.21.15.120:0/3096099813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a24b6256f0 0x55a24b62f580 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0031440 tx=0x7f3bb0005250 comp rx=0 tx=0).stop 2024-09-19T09:16:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.310+0000 7f3bbf65c740 1 -- 172.21.15.120:0/3096099813 shutdown_connections 2024-09-19T09:16:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.311+0000 7f3bbf65c740 1 --2- 172.21.15.120:0/3096099813 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a24b63a8c0 0x55a24b63ccb0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.311+0000 7f3bbf65c740 1 --2- 172.21.15.120:0/3096099813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a24b62a780 0x55a24b6365c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.311+0000 7f3bbf65c740 1 --2- 172.21.15.120:0/3096099813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a24b6256f0 0x55a24b62f580 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.311+0000 7f3b9dffb640 0 client.20103 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.311+0000 7f3bbf65c740 1 --2- 172.21.15.120:0/3096099813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a24b624db0 0x55a24b62f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.311+0000 7f3bbf65c740 1 -- 172.21.15.120:0/3096099813 >> 172.21.15.120:0/3096099813 conn(0x55a24b61c010 msgr2=0x55a24b61d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.311+0000 7f3bbf65c740 1 -- 172.21.15.120:0/3096099813 shutdown_connections 2024-09-19T09:16:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.311+0000 7f3bbf65c740 1 -- 172.21.15.120:0/3096099813 wait complete. 2024-09-19T09:16:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.311+0000 7f3bbf65c740 20 client.20103 trim_cache size 0 max 16384 2024-09-19T09:16:17.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:16 smithi120 ceph-mon[31150]: pgmap v746: 97 pgs: 97 active+clean; 29 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 235 op/s 2024-09-19T09:16:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.325+0000 7f3bbf65c740 1 Processor -- start 2024-09-19T09:16:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.325+0000 7f3bbf65c740 1 -- start start 2024-09-19T09:16:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.325+0000 7f3bbf65c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a24b61c010 0x55a24b61d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.325+0000 7f3bbf65c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a24b624db0 0x55a24b636580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.325+0000 7f3bbf65c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a24b6256f0 0x55a24b61d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.325+0000 7f3bbf65c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb41a6ed0 con 0x55a24b6256f0 2024-09-19T09:16:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.325+0000 7f3bbf65c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb41a8a10 con 0x55a24b61c010 2024-09-19T09:16:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.325+0000 7f3bbf65c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb41a75a0 con 0x55a24b624db0 2024-09-19T09:16:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.325+0000 7f3bb9d7f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a24b6256f0 0x55a24b61d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.325+0000 7f3bb9d7f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a24b6256f0 0x55a24b61d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54886/0 (socket says 172.21.15.120:54886) 2024-09-19T09:16:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.325+0000 7f3bb9d7f640 1 -- 172.21.15.120:0/892853473 learned_addr learned my addr 172.21.15.120:0/892853473 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.326+0000 7f3bb9d7f640 1 --2- 172.21.15.120:0/892853473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a24b61c010 0x55a24b61d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.326+0000 7f3bb9d7f640 1 -- 172.21.15.120:0/892853473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a24b624db0 msgr2=0x55a24b636580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.326+0000 7f3bb9d7f640 1 --2- 172.21.15.120:0/892853473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a24b624db0 0x55a24b636580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.326+0000 7f3bb9d7f640 1 -- 172.21.15.120:0/892853473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a24b61c010 msgr2=0x55a24b61d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.326+0000 7f3bb9d7f640 1 --2- 172.21.15.120:0/892853473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a24b61c010 0x55a24b61d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.326+0000 7f3bb9d7f640 1 -- 172.21.15.120:0/892853473 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a24b61de80 con 0x55a24b6256f0 2024-09-19T09:16:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.326+0000 7f3bb9d7f640 1 --2- 172.21.15.120:0/892853473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a24b6256f0 0x55a24b61d940 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4198260 tx=0x7f3ba400e410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f5b564acdca6e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.326+0000 7f3b9ffff640 1 -- 172.21.15.120:0/892853473 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4004220 con 0x55a24b6256f0 2024-09-19T09:16:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.326+0000 7f3b9ffff640 1 -- 172.21.15.120:0/892853473 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ba4014e30 con 0x55a24b6256f0 2024-09-19T09:16:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.326+0000 7f3b9ffff640 1 -- 172.21.15.120:0/892853473 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4003d70 con 0x55a24b6256f0 2024-09-19T09:16:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.326+0000 7f3bbf65c740 1 -- 172.21.15.120:0/892853473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a24b6256f0 msgr2=0x55a24b61d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.327+0000 7f3bbf65c740 1 --2- 172.21.15.120:0/892853473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a24b6256f0 0x55a24b61d940 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4198260 tx=0x7f3ba400e410 comp rx=0 tx=0).stop 2024-09-19T09:16:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.327+0000 7f3bbf65c740 1 -- 172.21.15.120:0/892853473 shutdown_connections 2024-09-19T09:16:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.327+0000 7f3bbf65c740 1 --2- 172.21.15.120:0/892853473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a24b6256f0 0x55a24b61d940 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.327+0000 7f3bbf65c740 1 --2- 172.21.15.120:0/892853473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a24b624db0 0x55a24b636580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.327+0000 7f3bbf65c740 1 --2- 172.21.15.120:0/892853473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a24b61c010 0x55a24b61d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.327+0000 7f3bbf65c740 1 -- 172.21.15.120:0/892853473 >> 172.21.15.120:0/892853473 conn(0x55a24b62a380 msgr2=0x55a24b61b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.327+0000 7f3bbf65c740 1 -- 172.21.15.120:0/892853473 shutdown_connections 2024-09-19T09:16:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.327+0000 7f3bbf65c740 1 -- 172.21.15.120:0/892853473 wait complete. 2024-09-19T09:16:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.327+0000 7f3bbf65c740 1 -- start start 2024-09-19T09:16:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.327+0000 7f3bbf65c740 1 Processor -- start 2024-09-19T09:16:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.327+0000 7f3bbf65c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a24b61c010 0x55a24b636000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.327+0000 7f3bbf65c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a24b624db0 0x55a24b636540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.327+0000 7f3bbf65c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a24b6256f0 0x55a24b634700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.327+0000 7f3bbf65c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb41a8340 con 0x55a24b61c010 2024-09-19T09:16:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.327+0000 7f3bbf65c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb41a7c70 con 0x55a24b6256f0 2024-09-19T09:16:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.327+0000 7f3bbf65c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb41a4e90 con 0x55a24b624db0 2024-09-19T09:16:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.327+0000 7f3bbf65c740 10 client.0 _ll_register_callbacks cb 0x7f3ba400ee70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.327+0000 7f3bb9d7f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a24b61c010 0x55a24b636000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.327+0000 7f3bb9d7f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a24b61c010 0x55a24b636000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54894/0 (socket says 172.21.15.120:54894) 2024-09-19T09:16:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.327+0000 7f3bb9d7f640 1 -- 172.21.15.120:0/1203975741 learned_addr learned my addr 172.21.15.120:0/1203975741 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.327+0000 7f3bb9d7f640 1 --2- 172.21.15.120:0/1203975741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a24b6256f0 0x55a24b634700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.328+0000 7f3bba580640 1 --2- 172.21.15.120:0/1203975741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a24b624db0 0x55a24b636540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.328+0000 7f3bb9d7f640 1 -- 172.21.15.120:0/1203975741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a24b624db0 msgr2=0x55a24b636540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.328+0000 7f3bb9d7f640 1 --2- 172.21.15.120:0/1203975741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a24b624db0 0x55a24b636540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.328+0000 7f3bb9d7f640 1 -- 172.21.15.120:0/1203975741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a24b6256f0 msgr2=0x55a24b634700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.328+0000 7f3bb9d7f640 1 --2- 172.21.15.120:0/1203975741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a24b6256f0 0x55a24b634700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.328+0000 7f3bb9d7f640 1 -- 172.21.15.120:0/1203975741 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a24b634f30 con 0x55a24b61c010 2024-09-19T09:16:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.328+0000 7f3bb9d7f640 1 --2- 172.21.15.120:0/1203975741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a24b61c010 0x55a24b636000 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4000c00 tx=0x7f3ba4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e8b38fc128f8d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.328+0000 7f3b9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:16:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.328+0000 7f3b9dffb640 1 -- 172.21.15.120:0/1203975741 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba40041b0 con 0x55a24b61c010 2024-09-19T09:16:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.328+0000 7f3b9dffb640 1 -- 172.21.15.120:0/1203975741 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f3ba4004350 con 0x55a24b61c010 2024-09-19T09:16:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.328+0000 7f3bbf65c740 1 -- 172.21.15.120:0/1203975741 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ba4014e30 con 0x55a24b61c010 2024-09-19T09:16:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.328+0000 7f3b9dffb640 1 -- 172.21.15.120:0/1203975741 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3ba4005030 con 0x55a24b61c010 2024-09-19T09:16:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.328+0000 7f3bbf65c740 10 client.20109 fetch_fsmap learned FSMap version 33 2024-09-19T09:16:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.328+0000 7f3bbf65c740 1 -- 172.21.15.120:0/1203975741 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a24b6a6120 con 0x55a24b61c010 2024-09-19T09:16:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.328+0000 7f3b9dffb640 1 -- 172.21.15.120:0/1203975741 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ba4014e30 con 0x55a24b61c010 2024-09-19T09:16:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.328+0000 7f3b9dffb640 1 -- 172.21.15.120:0/1203975741 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f3bb00042e0 con 0x55a24b61c010 2024-09-19T09:16:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.328+0000 7f3bbf65c740 10 client.20109 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:16:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.328+0000 7f3bbf65c740 10 client.20109 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:16:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.328+0000 7f3bbf65c740 20 client.20109 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.328+0000 7f3bbf65c740 1 --2- 172.21.15.120:0/1203975741 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a24b62a780 0x55a24b6a8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.328+0000 7f3bbf65c740 4 client.20109 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:16:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.328+0000 7f3bbf65c740 1 -- 172.21.15.120:0/1203975741 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a24b58bd70 con 0x55a24b62a780 2024-09-19T09:16:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.329+0000 7f3bba580640 1 --2- 172.21.15.120:0/1203975741 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a24b62a780 0x55a24b6a8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.330+0000 7f3b9dffb640 10 client.20109 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.330+0000 7f3bba580640 1 --2- 172.21.15.120:0/1203975741 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a24b62a780 0x55a24b6a8af0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35f34a8f977bb69e server_cookie=426f3a461ef9f4fa in_seq=0 out_seq=0 2024-09-19T09:16:17.351 INFO:tasks.workunit.client.0.smithi120.stdout: 10 795 17.86 MB/sec warmup 12 sec latency 10205.780 ms 2024-09-19T09:16:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.630+0000 7f3b9dffb640 1 -- 172.21.15.120:0/1203975741 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a24b58bd70 con 0x55a24b62a780 2024-09-19T09:16:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.630+0000 7f3b9dffb640 10 client.20109 handle_command_reply: tid=0 2024-09-19T09:16:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.630+0000 7f3bbf65c740 1 client.20109 shutdown 2024-09-19T09:16:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.630+0000 7f3bbf65c740 1 -- 172.21.15.120:0/1203975741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a24b61c010 msgr2=0x55a24b636000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.630+0000 7f3bbf65c740 1 --2- 172.21.15.120:0/1203975741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a24b61c010 0x55a24b636000 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4000c00 tx=0x7f3ba4005f90 comp rx=0 tx=0).stop 2024-09-19T09:16:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.630+0000 7f3bbf65c740 1 -- 172.21.15.120:0/1203975741 shutdown_connections 2024-09-19T09:16:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.630+0000 7f3bbf65c740 1 --2- 172.21.15.120:0/1203975741 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a24b62a780 0x55a24b6a8af0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.630+0000 7f3bbf65c740 1 --2- 172.21.15.120:0/1203975741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a24b6256f0 0x55a24b634700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.630+0000 7f3bbf65c740 1 --2- 172.21.15.120:0/1203975741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a24b624db0 0x55a24b636540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.630+0000 7f3bbf65c740 1 --2- 172.21.15.120:0/1203975741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a24b61c010 0x55a24b636000 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.630+0000 7f3bbf65c740 1 -- 172.21.15.120:0/1203975741 >> 172.21.15.120:0/1203975741 conn(0x55a24b62a380 msgr2=0x55a24b61c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.630+0000 7f3b9dffb640 0 client.20109 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.630+0000 7f3bbf65c740 1 -- 172.21.15.120:0/1203975741 shutdown_connections 2024-09-19T09:16:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.630+0000 7f3bbf65c740 1 -- 172.21.15.120:0/1203975741 wait complete. 2024-09-19T09:16:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.631+0000 7f3bbf65c740 20 client.20109 trim_cache size 0 max 16384 2024-09-19T09:16:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:16:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:16:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:16:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:16:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.631+0000 7f3bbc7e4640 1 -- 172.21.15.120:0/2713856245 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3ba00a32f0 msgr2=0x7f3ba00a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.631+0000 7f3bbc7e4640 1 --2- 172.21.15.120:0/2713856245 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3ba00a32f0 0x7f3ba00a57b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3bac0cebe0 tx=0x7f3bac06c3f0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.631+0000 7f3bbc7e4640 1 -- 172.21.15.120:0/2713856245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3bb4102610 msgr2=0x7f3bb419e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.631+0000 7f3bbc7e4640 1 --2- 172.21.15.120:0/2713856245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3bb4102610 0x7f3bb419e300 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f3bac00db90 tx=0x7f3bac00dc80 comp rx=0 tx=0).stop 2024-09-19T09:16:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.631+0000 7f3bbc7e4640 1 -- 172.21.15.120:0/2713856245 shutdown_connections 2024-09-19T09:16:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.631+0000 7f3bbc7e4640 1 --2- 172.21.15.120:0/2713856245 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3ba00a32f0 0x7f3ba00a57b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.631+0000 7f3bbc7e4640 1 --2- 172.21.15.120:0/2713856245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3bb4102610 0x7f3bb419e300 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.631+0000 7f3bbc7e4640 1 --2- 172.21.15.120:0/2713856245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bb4101c50 0x7f3bb4197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.631+0000 7f3bbc7e4640 1 --2- 172.21.15.120:0/2713856245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3bb4100a60 0x7f3bb4196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.631+0000 7f3bbc7e4640 1 -- 172.21.15.120:0/2713856245 >> 172.21.15.120:0/2713856245 conn(0x7f3bb40fc1e0 msgr2=0x7f3bb4104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.631+0000 7f3bbc7e4640 1 -- 172.21.15.120:0/2713856245 shutdown_connections 2024-09-19T09:16:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:17.631+0000 7f3bbc7e4640 1 -- 172.21.15.120:0/2713856245 wait complete. 2024-09-19T09:16:18.352 INFO:tasks.workunit.client.0.smithi120.stdout: 10 795 16.49 MB/sec warmup 13 sec latency 11205.868 ms 2024-09-19T09:16:19.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:18 smithi196 ceph-mon[35150]: pgmap v747: 97 pgs: 97 active+clean; 21 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 267 op/s 2024-09-19T09:16:19.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:18 smithi153 ceph-mon[35071]: pgmap v747: 97 pgs: 97 active+clean; 21 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 267 op/s 2024-09-19T09:16:19.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:18 smithi120 ceph-mon[31150]: pgmap v747: 97 pgs: 97 active+clean; 21 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 267 op/s 2024-09-19T09:16:19.351 INFO:tasks.workunit.client.0.smithi120.stdout: 10 795 15.31 MB/sec warmup 14 sec latency 12205.942 ms 2024-09-19T09:16:20.353 INFO:tasks.workunit.client.0.smithi120.stdout: 10 795 14.29 MB/sec warmup 15 sec latency 13205.976 ms 2024-09-19T09:16:21.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:20 smithi196 ceph-mon[35150]: pgmap v748: 97 pgs: 97 active+clean; 18 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 2.2 MiB/s wr, 270 op/s 2024-09-19T09:16:21.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:20 smithi153 ceph-mon[35071]: pgmap v748: 97 pgs: 97 active+clean; 18 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 2.2 MiB/s wr, 270 op/s 2024-09-19T09:16:21.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:20 smithi120 ceph-mon[31150]: pgmap v748: 97 pgs: 97 active+clean; 18 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 2.2 MiB/s wr, 270 op/s 2024-09-19T09:16:21.351 INFO:tasks.workunit.client.0.smithi120.stdout: 10 795 13.40 MB/sec warmup 16 sec latency 14206.087 ms 2024-09-19T09:16:22.351 INFO:tasks.workunit.client.0.smithi120.stdout: 10 795 12.61 MB/sec warmup 17 sec latency 15206.183 ms 2024-09-19T09:16:23.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:22 smithi196 ceph-mon[35150]: pgmap v749: 97 pgs: 97 active+clean; 16 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.6 MiB/s wr, 238 op/s 2024-09-19T09:16:23.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:22 smithi153 ceph-mon[35071]: pgmap v749: 97 pgs: 97 active+clean; 16 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.6 MiB/s wr, 238 op/s 2024-09-19T09:16:23.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:22 smithi120 ceph-mon[31150]: pgmap v749: 97 pgs: 97 active+clean; 16 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.6 MiB/s wr, 238 op/s 2024-09-19T09:16:23.351 INFO:tasks.workunit.client.0.smithi120.stdout: 10 795 11.91 MB/sec warmup 18 sec latency 16206.284 ms 2024-09-19T09:16:24.351 INFO:tasks.workunit.client.0.smithi120.stdout: 10 795 11.28 MB/sec warmup 19 sec latency 17206.383 ms 2024-09-19T09:16:25.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:24 smithi196 ceph-mon[35150]: pgmap v750: 97 pgs: 97 active+clean; 13 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.2 MiB/s wr, 226 op/s 2024-09-19T09:16:25.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:24 smithi153 ceph-mon[35071]: pgmap v750: 97 pgs: 97 active+clean; 13 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.2 MiB/s wr, 226 op/s 2024-09-19T09:16:25.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:24 smithi120 ceph-mon[31150]: pgmap v750: 97 pgs: 97 active+clean; 13 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.2 MiB/s wr, 226 op/s 2024-09-19T09:16:25.351 INFO:tasks.workunit.client.0.smithi120.stdout: 10 795 10.72 MB/sec warmup 20 sec latency 18206.507 ms 2024-09-19T09:16:26.352 INFO:tasks.workunit.client.0.smithi120.stdout: 10 795 10.21 MB/sec warmup 21 sec latency 19206.586 ms 2024-09-19T09:16:26.693 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:16:26.939 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:16:26.954 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:16:27.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:26 smithi120 ceph-mon[31150]: pgmap v751: 97 pgs: 97 active+clean; 13 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 707 KiB/s wr, 213 op/s 2024-09-19T09:16:27.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:26 smithi196 ceph-mon[35150]: pgmap v751: 97 pgs: 97 active+clean; 13 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 707 KiB/s wr, 213 op/s 2024-09-19T09:16:27.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:26 smithi153 ceph-mon[35071]: pgmap v751: 97 pgs: 97 active+clean; 13 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 707 KiB/s wr, 213 op/s 2024-09-19T09:16:27.352 INFO:tasks.workunit.client.0.smithi120.stdout: 10 795 9.74 MB/sec warmup 22 sec latency 20206.621 ms 2024-09-19T09:16:27.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.511+0000 7fadecc24640 1 -- 172.21.15.120:0/2649159349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fade8102d40 msgr2=0x7fade8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:27.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.511+0000 7fadecc24640 1 --2- 172.21.15.120:0/2649159349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fade8102d40 0x7fade8103120 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7faddc009900 tx=0x7faddc02a480 comp rx=0 tx=0).stop 2024-09-19T09:16:27.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.511+0000 7fadecc24640 1 -- 172.21.15.120:0/2649159349 shutdown_connections 2024-09-19T09:16:27.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.511+0000 7fadecc24640 1 --2- 172.21.15.120:0/2649159349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fade810c170 0x7fade810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:27.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.511+0000 7fadecc24640 1 --2- 172.21.15.120:0/2649159349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fade8103660 0x7fade810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:27.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.511+0000 7fadecc24640 1 --2- 172.21.15.120:0/2649159349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fade8102d40 0x7fade8103120 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:27.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.511+0000 7fadecc24640 1 -- 172.21.15.120:0/2649159349 >> 172.21.15.120:0/2649159349 conn(0x7fade80fc7a0 msgr2=0x7fade80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:27.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.511+0000 7fadecc24640 1 -- 172.21.15.120:0/2649159349 shutdown_connections 2024-09-19T09:16:27.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.511+0000 7fadecc24640 1 -- 172.21.15.120:0/2649159349 wait complete. 2024-09-19T09:16:27.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.512+0000 7fadecc24640 1 Processor -- start 2024-09-19T09:16:27.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.512+0000 7fadecc24640 1 -- start start 2024-09-19T09:16:27.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.512+0000 7fadecc24640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fade8102d40 0x7fade81983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:27.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.512+0000 7fadecc24640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fade8103660 0x7fade8198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:27.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.513+0000 7fadecc24640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fade810c170 0x7fade819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:27.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.513+0000 7fadecc24640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fade8111f70 con 0x7fade810c170 2024-09-19T09:16:27.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.513+0000 7fadecc24640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fade8111df0 con 0x7fade8103660 2024-09-19T09:16:27.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.513+0000 7fadecc24640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fade81120f0 con 0x7fade8102d40 2024-09-19T09:16:27.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.513+0000 7fade6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fade810c170 0x7fade819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:27.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.513+0000 7fade6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fade810c170 0x7fade819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33106/0 (socket says 172.21.15.120:33106) 2024-09-19T09:16:27.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.513+0000 7fade6d9d640 1 -- 172.21.15.120:0/1081640881 learned_addr learned my addr 172.21.15.120:0/1081640881 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:27.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.513+0000 7fade659c640 1 --2- 172.21.15.120:0/1081640881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fade8102d40 0x7fade81983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:27.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.513+0000 7fade5d9b640 1 --2- 172.21.15.120:0/1081640881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fade8103660 0x7fade8198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:27.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.513+0000 7fade6d9d640 1 -- 172.21.15.120:0/1081640881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fade8102d40 msgr2=0x7fade81983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:27.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.513+0000 7fade6d9d640 1 --2- 172.21.15.120:0/1081640881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fade8102d40 0x7fade81983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:27.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.513+0000 7fade6d9d640 1 -- 172.21.15.120:0/1081640881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fade8103660 msgr2=0x7fade8198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:27.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.513+0000 7fade6d9d640 1 --2- 172.21.15.120:0/1081640881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fade8103660 0x7fade8198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:27.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.513+0000 7fade6d9d640 1 -- 172.21.15.120:0/1081640881 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade819d380 con 0x7fade810c170 2024-09-19T09:16:27.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.513+0000 7fade5d9b640 1 --2- 172.21.15.120:0/1081640881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fade8103660 0x7fade8198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:16:27.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.513+0000 7fade6d9d640 1 --2- 172.21.15.120:0/1081640881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fade810c170 0x7fade819cca0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fade000cce0 tx=0x7fade000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:27.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.513+0000 7fadd77fe640 1 -- 172.21.15.120:0/1081640881 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade000f450 con 0x7fade810c170 2024-09-19T09:16:27.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.513+0000 7fadd77fe640 1 -- 172.21.15.120:0/1081640881 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fade000fad0 con 0x7fade810c170 2024-09-19T09:16:27.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.513+0000 7fadecc24640 1 -- 172.21.15.120:0/1081640881 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade819d670 con 0x7fade810c170 2024-09-19T09:16:27.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.513+0000 7fadd77fe640 1 -- 172.21.15.120:0/1081640881 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade0018c10 con 0x7fade810c170 2024-09-19T09:16:27.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.513+0000 7fadecc24640 1 -- 172.21.15.120:0/1081640881 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade81a4f50 con 0x7fade810c170 2024-09-19T09:16:27.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.514+0000 7fadecc24640 1 -- 172.21.15.120:0/1081640881 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fade8103120 con 0x7fade810c170 2024-09-19T09:16:27.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.516+0000 7fadd77fe640 1 -- 172.21.15.120:0/1081640881 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fade000f5f0 con 0x7fade810c170 2024-09-19T09:16:27.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.517+0000 7fadd77fe640 1 --2- 172.21.15.120:0/1081640881 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fadd00a3240 0x7fadd00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:27.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.517+0000 7fadd77fe640 1 -- 172.21.15.120:0/1081640881 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fade00d0010 con 0x7fade810c170 2024-09-19T09:16:27.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.517+0000 7fade659c640 1 --2- 172.21.15.120:0/1081640881 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fadd00a3240 0x7fadd00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:27.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.517+0000 7fade659c640 1 --2- 172.21.15.120:0/1081640881 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fadd00a3240 0x7fadd00a5700 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7faddc000c00 tx=0x7faddc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:27.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.520+0000 7fadd77fe640 1 -- 172.21.15.120:0/1081640881 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fade0014030 con 0x7fade810c170 2024-09-19T09:16:27.599 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:16:27.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.733+0000 7fadecc24640 1 -- 172.21.15.120:0/1081640881 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fade8199730 con 0x7fade810c170 2024-09-19T09:16:27.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.734+0000 7fadd77fe640 1 -- 172.21.15.120:0/1081640881 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11988 (secure 0 0 0) 0x7fade009cdc0 con 0x7fade810c170 2024-09-19T09:16:27.736 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:16:27.736 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":33,"btime":"2024-09-19T09:14:31:550518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:14:31.550494+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28649":{"gid":28649,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/2810080307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2810080307},{"type":"v1","addr":"172.21.15.153:6835","nonce":2810080307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:16:27.736 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 33 2024-09-19T09:16:27.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.736+0000 7fadecc24640 1 -- 172.21.15.120:0/1081640881 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fadd00a3240 msgr2=0x7fadd00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:27.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.736+0000 7fadecc24640 1 --2- 172.21.15.120:0/1081640881 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fadd00a3240 0x7fadd00a5700 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7faddc000c00 tx=0x7faddc0058d0 comp rx=0 tx=0).stop 2024-09-19T09:16:27.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.736+0000 7fadecc24640 1 -- 172.21.15.120:0/1081640881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fade810c170 msgr2=0x7fade819cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:27.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.736+0000 7fadecc24640 1 --2- 172.21.15.120:0/1081640881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fade810c170 0x7fade819cca0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fade000cce0 tx=0x7fade000a620 comp rx=0 tx=0).stop 2024-09-19T09:16:27.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.736+0000 7fadecc24640 1 -- 172.21.15.120:0/1081640881 shutdown_connections 2024-09-19T09:16:27.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.736+0000 7fadecc24640 1 --2- 172.21.15.120:0/1081640881 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fadd00a3240 0x7fadd00a5700 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:27.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.736+0000 7fadecc24640 1 --2- 172.21.15.120:0/1081640881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fade810c170 0x7fade819cca0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:27.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.736+0000 7fadecc24640 1 --2- 172.21.15.120:0/1081640881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fade8103660 0x7fade8198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:27.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.736+0000 7fadecc24640 1 --2- 172.21.15.120:0/1081640881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fade8102d40 0x7fade81983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:27.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.736+0000 7fadecc24640 1 -- 172.21.15.120:0/1081640881 >> 172.21.15.120:0/1081640881 conn(0x7fade80fc7a0 msgr2=0x7fade80fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:27.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.736+0000 7fadecc24640 1 -- 172.21.15.120:0/1081640881 shutdown_connections 2024-09-19T09:16:27.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:27.736+0000 7fadecc24640 1 -- 172.21.15.120:0/1081640881 wait complete. 2024-09-19T09:16:28.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:27 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1081640881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:16:28.352 INFO:tasks.workunit.client.0.smithi120.stdout: 10 795 9.32 MB/sec warmup 23 sec latency 21206.659 ms 2024-09-19T09:16:28.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:27 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1081640881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:16:28.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:27 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1081640881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:16:28.704 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db --set-id 1e2662fa --release --await' 2024-09-19T09:16:28.704 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db --set-id 1e2662fa --release --await 2024-09-19T09:16:28.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.953+0000 7f2929220640 1 -- 172.21.15.120:0/2840408851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2924100e20 msgr2=0x7f2924101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:28.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.953+0000 7f2929220640 1 --2- 172.21.15.120:0/2840408851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2924100e20 0x7f2924101200 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f290c009900 tx=0x7f290c02a490 comp rx=0 tx=0).stop 2024-09-19T09:16:28.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.953+0000 7f2929220640 1 -- 172.21.15.120:0/2840408851 shutdown_connections 2024-09-19T09:16:28.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.953+0000 7f2929220640 1 --2- 172.21.15.120:0/2840408851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2924102ef0 0x7f292410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:28.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.953+0000 7f2929220640 1 --2- 172.21.15.120:0/2840408851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2924102570 0x7f29241029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:28.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.953+0000 7f2929220640 1 --2- 172.21.15.120:0/2840408851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2924100e20 0x7f2924101200 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:28.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.953+0000 7f2929220640 1 -- 172.21.15.120:0/2840408851 >> 172.21.15.120:0/2840408851 conn(0x7f29240fc7d0 msgr2=0x7f29240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:28.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.953+0000 7f2929220640 1 -- 172.21.15.120:0/2840408851 shutdown_connections 2024-09-19T09:16:28.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.953+0000 7f2929220640 1 -- 172.21.15.120:0/2840408851 wait complete. 2024-09-19T09:16:28.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.953+0000 7f2929220640 1 Processor -- start 2024-09-19T09:16:28.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.954+0000 7f2929220640 1 -- start start 2024-09-19T09:16:28.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.954+0000 7f2929220640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2924100e20 0x7f29241986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:28.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.954+0000 7f2929220640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2924102570 0x7f2924198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:28.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.954+0000 7f2929220640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2924102ef0 0x7f292419cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:28.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.954+0000 7f2929220640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2924111070 con 0x7f2924102570 2024-09-19T09:16:28.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.954+0000 7f2929220640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2924110ef0 con 0x7f2924102ef0 2024-09-19T09:16:28.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.954+0000 7f2929220640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f29241111f0 con 0x7f2924100e20 2024-09-19T09:16:28.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.954+0000 7f292259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2924102570 0x7f2924198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:28.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.954+0000 7f292259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2924102570 0x7f2924198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33130/0 (socket says 172.21.15.120:33130) 2024-09-19T09:16:28.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.954+0000 7f292259c640 1 -- 172.21.15.120:0/1629216737 learned_addr learned my addr 172.21.15.120:0/1629216737 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:28.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.954+0000 7f292359e640 1 --2- 172.21.15.120:0/1629216737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2924102ef0 0x7f292419cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:28.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.954+0000 7f2922d9d640 1 --2- 172.21.15.120:0/1629216737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2924100e20 0x7f29241986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:28.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.954+0000 7f292259c640 1 -- 172.21.15.120:0/1629216737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2924100e20 msgr2=0x7f29241986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:28.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.954+0000 7f292259c640 1 --2- 172.21.15.120:0/1629216737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2924100e20 0x7f29241986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:28.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.954+0000 7f292259c640 1 -- 172.21.15.120:0/1629216737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2924102ef0 msgr2=0x7f292419cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:28.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.954+0000 7f292259c640 1 --2- 172.21.15.120:0/1629216737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2924102ef0 0x7f292419cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:28.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.954+0000 7f292259c640 1 -- 172.21.15.120:0/1629216737 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292419d690 con 0x7f2924102570 2024-09-19T09:16:28.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.954+0000 7f292359e640 1 --2- 172.21.15.120:0/1629216737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2924102ef0 0x7f292419cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:16:28.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.955+0000 7f292259c640 1 --2- 172.21.15.120:0/1629216737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2924102570 0x7f2924198c20 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f291400b4a0 tx=0x7f291400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:28.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.955+0000 7f291bfff640 1 -- 172.21.15.120:0/1629216737 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914014070 con 0x7f2924102570 2024-09-19T09:16:28.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.955+0000 7f2929220640 1 -- 172.21.15.120:0/1629216737 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f292419d980 con 0x7f2924102570 2024-09-19T09:16:28.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.955+0000 7f291bfff640 1 -- 172.21.15.120:0/1629216737 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2914010af0 con 0x7f2924102570 2024-09-19T09:16:28.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.955+0000 7f291bfff640 1 -- 172.21.15.120:0/1629216737 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914017d10 con 0x7f2924102570 2024-09-19T09:16:28.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.955+0000 7f2929220640 1 -- 172.21.15.120:0/1629216737 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29241a5260 con 0x7f2924102570 2024-09-19T09:16:28.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.957+0000 7f291bfff640 1 -- 172.21.15.120:0/1629216737 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f2914017070 con 0x7f2924102570 2024-09-19T09:16:28.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.957+0000 7f292b5f9740 1 Processor -- start 2024-09-19T09:16:28.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.958+0000 7f292b5f9740 1 -- start start 2024-09-19T09:16:28.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.958+0000 7f292b5f9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d589ae3360 0x55d589ae3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:28.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.958+0000 7f292b5f9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d589ae3c80 0x55d589af2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:28.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.958+0000 7f292b5f9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d589af2a90 0x55d589af4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:28.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.958+0000 7f292b5f9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d589a492e0 con 0x55d589af2a90 2024-09-19T09:16:28.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.958+0000 7f292b5f9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d5899f0ce0 con 0x55d589ae3c80 2024-09-19T09:16:28.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.958+0000 7f292b5f9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d589a452a0 con 0x55d589ae3360 2024-09-19T09:16:28.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.958+0000 7f292259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d589af2a90 0x55d589af4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:28.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.958+0000 7f292259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d589af2a90 0x55d589af4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33140/0 (socket says 172.21.15.120:33140) 2024-09-19T09:16:28.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.958+0000 7f292259c640 1 -- 172.21.15.120:0/2571972274 learned_addr learned my addr 172.21.15.120:0/2571972274 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:28.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.958+0000 7f291bfff640 1 --2- 172.21.15.120:0/1629216737 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f29100a3240 0x7f29100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:28.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.958+0000 7f292359e640 1 --2- 172.21.15.120:0/2571972274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d589ae3360 0x55d589ae3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:28.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.958+0000 7f2922d9d640 1 --2- 172.21.15.120:0/2571972274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d589ae3c80 0x55d589af2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:28.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.959+0000 7f291bfff640 1 -- 172.21.15.120:0/1629216737 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f29140cedd0 con 0x7f2924102570 2024-09-19T09:16:28.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.959+0000 7f292359e640 1 --2- 172.21.15.120:0/1629216737 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f29100a3240 0x7f29100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:28.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.959+0000 7f292259c640 1 -- 172.21.15.120:0/2571972274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d589ae3360 msgr2=0x55d589ae3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:28.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.959+0000 7f292259c640 1 --2- 172.21.15.120:0/2571972274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d589ae3360 0x55d589ae3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:28.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.959+0000 7f292259c640 1 -- 172.21.15.120:0/2571972274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d589ae3c80 msgr2=0x55d589af2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:28.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.959+0000 7f292259c640 1 --2- 172.21.15.120:0/2571972274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d589ae3c80 0x55d589af2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:28.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.959+0000 7f292259c640 1 -- 172.21.15.120:0/2571972274 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5899d5440 con 0x55d589af2a90 2024-09-19T09:16:28.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.959+0000 7f2922d9d640 1 --2- 172.21.15.120:0/2571972274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d589ae3c80 0x55d589af2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:16:28.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.959+0000 7f292259c640 1 --2- 172.21.15.120:0/2571972274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d589af2a90 0x55d589af4e80 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f29140cc890 tx=0x7f29140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40efa73167d5b05e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:28.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.959+0000 7f29197fa640 1 -- 172.21.15.120:0/2571972274 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291401e050 con 0x55d589af2a90 2024-09-19T09:16:28.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.959+0000 7f292359e640 1 --2- 172.21.15.120:0/1629216737 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f29100a3240 0x7f29100a5700 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f291c009fd0 tx=0x7f291c010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:28.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.959+0000 7f29197fa640 1 -- 172.21.15.120:0/2571972274 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f291409b3b0 con 0x55d589af2a90 2024-09-19T09:16:28.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.959+0000 7f29197fa640 1 -- 172.21.15.120:0/2571972274 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291409a800 con 0x55d589af2a90 2024-09-19T09:16:28.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.960+0000 7f292b5f9740 1 -- 172.21.15.120:0/2571972274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d589af2a90 msgr2=0x55d589af4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:28.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.960+0000 7f292b5f9740 1 --2- 172.21.15.120:0/2571972274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d589af2a90 0x55d589af4e80 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f29140cc890 tx=0x7f29140046c0 comp rx=0 tx=0).stop 2024-09-19T09:16:28.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.960+0000 7f292b5f9740 1 -- 172.21.15.120:0/2571972274 shutdown_connections 2024-09-19T09:16:28.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.960+0000 7f292b5f9740 1 --2- 172.21.15.120:0/2571972274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d589af2a90 0x55d589af4e80 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:28.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.960+0000 7f292b5f9740 1 --2- 172.21.15.120:0/2571972274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d589ae3c80 0x55d589af2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:28.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.960+0000 7f292b5f9740 1 --2- 172.21.15.120:0/2571972274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d589ae3360 0x55d589ae3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:28.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.960+0000 7f292b5f9740 1 -- 172.21.15.120:0/2571972274 >> 172.21.15.120:0/2571972274 conn(0x55d589adfa30 msgr2=0x55d589adfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:28.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.960+0000 7f292b5f9740 1 -- 172.21.15.120:0/2571972274 shutdown_connections 2024-09-19T09:16:28.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.960+0000 7f292b5f9740 1 -- 172.21.15.120:0/2571972274 wait complete. 2024-09-19T09:16:28.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.960+0000 7f292b5f9740 1 -- start start 2024-09-19T09:16:28.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.961+0000 7f292b5f9740 1 Processor -- start 2024-09-19T09:16:28.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.961+0000 7f292b5f9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d589ae3360 0x55d589af2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:28.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.961+0000 7f292b5f9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d589ae3c80 0x55d589aef6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:28.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.961+0000 7f292b5f9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d589af2a90 0x55d589aefc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:28.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.961+0000 7f292b5f9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d589af5600 con 0x55d589ae3360 2024-09-19T09:16:28.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.961+0000 7f292b5f9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d589a04050 con 0x55d589af2a90 2024-09-19T09:16:28.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.961+0000 7f292b5f9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d589af5780 con 0x55d589ae3c80 2024-09-19T09:16:28.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.961+0000 7f292b5f9740 10 client.0 _ll_register_callbacks cb 0x55d589a44160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:28.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.961+0000 7f2922d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d589ae3360 0x55d589af2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:28.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.961+0000 7f2922d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d589ae3360 0x55d589af2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33152/0 (socket says 172.21.15.120:33152) 2024-09-19T09:16:28.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.961+0000 7f2922d9d640 1 -- 172.21.15.120:0/2612218576 learned_addr learned my addr 172.21.15.120:0/2612218576 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:28.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.961+0000 7f292359e640 1 --2- 172.21.15.120:0/2612218576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d589af2a90 0x55d589aefc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:28.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.961+0000 7f292259c640 1 --2- 172.21.15.120:0/2612218576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d589ae3c80 0x55d589aef6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:28.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.961+0000 7f2922d9d640 1 -- 172.21.15.120:0/2612218576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d589ae3c80 msgr2=0x55d589aef6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:28.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.961+0000 7f2922d9d640 1 --2- 172.21.15.120:0/2612218576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d589ae3c80 0x55d589aef6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:28.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.961+0000 7f2922d9d640 1 -- 172.21.15.120:0/2612218576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d589af2a90 msgr2=0x55d589aefc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:28.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.961+0000 7f2922d9d640 1 --2- 172.21.15.120:0/2612218576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d589af2a90 0x55d589aefc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:28.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.961+0000 7f2922d9d640 1 -- 172.21.15.120:0/2612218576 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5897ba3c0 con 0x55d589ae3360 2024-09-19T09:16:28.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.962+0000 7f2922d9d640 1 --2- 172.21.15.120:0/2612218576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d589ae3360 0x55d589af2440 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f290c009900 tx=0x7f290c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac17d8aff2c97097 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:28.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.962+0000 7f290a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:16:28.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.964+0000 7f290a7fc640 1 -- 172.21.15.120:0/2612218576 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c035070 con 0x55d589ae3360 2024-09-19T09:16:28.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.964+0000 7f290a7fc640 1 -- 172.21.15.120:0/2612218576 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f290c003b90 con 0x55d589ae3360 2024-09-19T09:16:28.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.964+0000 7f292b5f9740 1 -- 172.21.15.120:0/2612218576 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f291409b3b0 con 0x55d589ae3360 2024-09-19T09:16:28.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.965+0000 7f290a7fc640 1 -- 172.21.15.120:0/2612218576 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f290c003d90 con 0x55d589ae3360 2024-09-19T09:16:28.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.965+0000 7f292b5f9740 10 client.20127 fetch_fsmap learned FSMap version 33 2024-09-19T09:16:28.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.965+0000 7f292b5f9740 1 -- 172.21.15.120:0/2612218576 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d589ae60e0 con 0x55d589ae3360 2024-09-19T09:16:28.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.965+0000 7f290a7fc640 1 -- 172.21.15.120:0/2612218576 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f291409b3b0 con 0x55d589ae3360 2024-09-19T09:16:28.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.965+0000 7f290a7fc640 1 -- 172.21.15.120:0/2612218576 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f290c031540 con 0x55d589ae3360 2024-09-19T09:16:28.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.965+0000 7f292b5f9740 10 client.20127 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:16:28.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.965+0000 7f292b5f9740 10 client.20127 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:16:28.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.965+0000 7f292b5f9740 20 client.20127 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:28.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.965+0000 7f292b5f9740 1 --2- 172.21.15.120:0/2612218576 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d589ae75a0 0x55d589ae99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:28.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.965+0000 7f292b5f9740 4 client.20127 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:16:28.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.965+0000 7f292b5f9740 1 -- 172.21.15.120:0/2612218576 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d589a56850 con 0x55d589ae75a0 2024-09-19T09:16:28.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.965+0000 7f292259c640 1 --2- 172.21.15.120:0/2612218576 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d589ae75a0 0x55d589ae99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:28.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.966+0000 7f292259c640 1 --2- 172.21.15.120:0/2612218576 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d589ae75a0 0x55d589ae99b0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e600e9eb73fadea3 server_cookie=2a51f5582c3da225 in_seq=0 out_seq=0 2024-09-19T09:16:28.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.966+0000 7f290a7fc640 10 client.20127 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:28.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.967+0000 7f290a7fc640 1 -- 172.21.15.120:0/2612218576 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d589a56850 con 0x55d589ae75a0 2024-09-19T09:16:28.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.967+0000 7f290a7fc640 10 client.20127 handle_command_reply: tid=0 2024-09-19T09:16:28.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.967+0000 7f292b5f9740 1 client.20127 shutdown 2024-09-19T09:16:28.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.967+0000 7f292b5f9740 1 -- 172.21.15.120:0/2612218576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d589ae3360 msgr2=0x55d589af2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:28.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.967+0000 7f292b5f9740 1 --2- 172.21.15.120:0/2612218576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d589ae3360 0x55d589af2440 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f290c009900 tx=0x7f290c003820 comp rx=0 tx=0).stop 2024-09-19T09:16:28.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.967+0000 7f292b5f9740 1 -- 172.21.15.120:0/2612218576 shutdown_connections 2024-09-19T09:16:28.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.967+0000 7f292b5f9740 1 --2- 172.21.15.120:0/2612218576 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d589ae75a0 0x55d589ae99b0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:28.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.967+0000 7f292b5f9740 1 --2- 172.21.15.120:0/2612218576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d589af2a90 0x55d589aefc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:28.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.967+0000 7f292b5f9740 1 --2- 172.21.15.120:0/2612218576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d589ae3c80 0x55d589aef6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:28.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.967+0000 7f292b5f9740 1 --2- 172.21.15.120:0/2612218576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d589ae3360 0x55d589af2440 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:28.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.967+0000 7f290a7fc640 0 client.20127 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:28.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.967+0000 7f292b5f9740 1 -- 172.21.15.120:0/2612218576 >> 172.21.15.120:0/2612218576 conn(0x55d589adfa30 msgr2=0x55d589ae5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:28.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.967+0000 7f292b5f9740 1 -- 172.21.15.120:0/2612218576 shutdown_connections 2024-09-19T09:16:28.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.967+0000 7f292b5f9740 1 -- 172.21.15.120:0/2612218576 wait complete. 2024-09-19T09:16:28.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.968+0000 7f292b5f9740 20 client.20127 trim_cache size 0 max 16384 2024-09-19T09:16:28.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.987+0000 7f292b5f9740 1 Processor -- start 2024-09-19T09:16:28.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.987+0000 7f292b5f9740 1 -- start start 2024-09-19T09:16:28.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.987+0000 7f292b5f9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d589ae3360 0x55d589aeefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:28.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.987+0000 7f292b5f9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d589ae3c80 0x55d589aec950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:28.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.987+0000 7f292b5f9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d589af2a90 0x55d589aece90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:28.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.987+0000 7f292b5f9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f29241ab580 con 0x55d589ae3360 2024-09-19T09:16:28.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.987+0000 7f292b5f9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f29241ad0c0 con 0x55d589ae3c80 2024-09-19T09:16:28.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.987+0000 7f292b5f9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f29241abc50 con 0x55d589af2a90 2024-09-19T09:16:28.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.988+0000 7f2922d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d589ae3360 0x55d589aeefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:28.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.988+0000 7f2922d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d589ae3360 0x55d589aeefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33154/0 (socket says 172.21.15.120:33154) 2024-09-19T09:16:28.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.988+0000 7f2922d9d640 1 -- 172.21.15.120:0/1704091160 learned_addr learned my addr 172.21.15.120:0/1704091160 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:28.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.988+0000 7f292259c640 1 --2- 172.21.15.120:0/1704091160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d589ae3c80 0x55d589aec950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:28.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.988+0000 7f292359e640 1 --2- 172.21.15.120:0/1704091160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d589af2a90 0x55d589aece90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:28.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.988+0000 7f2922d9d640 1 -- 172.21.15.120:0/1704091160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d589af2a90 msgr2=0x55d589aece90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:28.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.988+0000 7f2922d9d640 1 --2- 172.21.15.120:0/1704091160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d589af2a90 0x55d589aece90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:28.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.988+0000 7f2922d9d640 1 -- 172.21.15.120:0/1704091160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d589ae3c80 msgr2=0x55d589aec950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:28.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.988+0000 7f2922d9d640 1 --2- 172.21.15.120:0/1704091160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d589ae3c80 0x55d589aec950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:28.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.988+0000 7f2922d9d640 1 -- 172.21.15.120:0/1704091160 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d589adf4d0 con 0x55d589ae3360 2024-09-19T09:16:28.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.988+0000 7f2922d9d640 1 --2- 172.21.15.120:0/1704091160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d589ae3360 0x55d589aeefd0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f290c006010 tx=0x7f290c02ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b110348279a80e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:28.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.988+0000 7f2918ff9640 1 -- 172.21.15.120:0/1704091160 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c035070 con 0x55d589ae3360 2024-09-19T09:16:28.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.988+0000 7f2918ff9640 1 -- 172.21.15.120:0/1704091160 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f290c031b50 con 0x55d589ae3360 2024-09-19T09:16:28.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.988+0000 7f2918ff9640 1 -- 172.21.15.120:0/1704091160 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c04bac0 con 0x55d589ae3360 2024-09-19T09:16:28.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.989+0000 7f292b5f9740 1 -- 172.21.15.120:0/1704091160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d589ae3360 msgr2=0x55d589aeefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:28.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.989+0000 7f292b5f9740 1 --2- 172.21.15.120:0/1704091160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d589ae3360 0x55d589aeefd0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f290c006010 tx=0x7f290c02ab10 comp rx=0 tx=0).stop 2024-09-19T09:16:28.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.989+0000 7f292b5f9740 1 -- 172.21.15.120:0/1704091160 shutdown_connections 2024-09-19T09:16:28.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.989+0000 7f292b5f9740 1 --2- 172.21.15.120:0/1704091160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d589af2a90 0x55d589aece90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:28.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.989+0000 7f292b5f9740 1 --2- 172.21.15.120:0/1704091160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d589ae3c80 0x55d589aec950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:28.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.989+0000 7f292b5f9740 1 --2- 172.21.15.120:0/1704091160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d589ae3360 0x55d589aeefd0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:28.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.989+0000 7f292b5f9740 1 -- 172.21.15.120:0/1704091160 >> 172.21.15.120:0/1704091160 conn(0x55d589adfa30 msgr2=0x55d589adeb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:28.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.989+0000 7f292b5f9740 1 -- 172.21.15.120:0/1704091160 shutdown_connections 2024-09-19T09:16:28.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.989+0000 7f292b5f9740 1 -- 172.21.15.120:0/1704091160 wait complete. 2024-09-19T09:16:28.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.990+0000 7f292b5f9740 1 -- start start 2024-09-19T09:16:28.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.990+0000 7f292b5f9740 1 Processor -- start 2024-09-19T09:16:28.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.990+0000 7f292b5f9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d589ae3360 0x55d589b04160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:28.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.990+0000 7f292b5f9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d589ae3c80 0x55d589b046a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:28.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.990+0000 7f292b5f9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d589af2a90 0x55d589b68060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:28.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.990+0000 7f292b5f9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f29241ac9f0 con 0x55d589ae3c80 2024-09-19T09:16:28.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.990+0000 7f292b5f9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f29241ac320 con 0x55d589af2a90 2024-09-19T09:16:28.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.990+0000 7f292b5f9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f29241a9560 con 0x55d589ae3360 2024-09-19T09:16:28.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.990+0000 7f292259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d589ae3c80 0x55d589b046a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:28.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.990+0000 7f292b5f9740 10 client.0 _ll_register_callbacks cb 0x7f290c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:28.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.990+0000 7f292259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d589ae3c80 0x55d589b046a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33160/0 (socket says 172.21.15.120:33160) 2024-09-19T09:16:28.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.990+0000 7f292259c640 1 -- 172.21.15.120:0/1052905391 learned_addr learned my addr 172.21.15.120:0/1052905391 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:28.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.990+0000 7f292359e640 1 --2- 172.21.15.120:0/1052905391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d589af2a90 0x55d589b68060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:28.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.990+0000 7f2922d9d640 1 --2- 172.21.15.120:0/1052905391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d589ae3360 0x55d589b04160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:28.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.990+0000 7f292259c640 1 -- 172.21.15.120:0/1052905391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d589ae3360 msgr2=0x55d589b04160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:28.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.990+0000 7f292259c640 1 --2- 172.21.15.120:0/1052905391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d589ae3360 0x55d589b04160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:28.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.991+0000 7f292259c640 1 -- 172.21.15.120:0/1052905391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d589af2a90 msgr2=0x55d589b68060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:28.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.991+0000 7f292259c640 1 --2- 172.21.15.120:0/1052905391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d589af2a90 0x55d589b68060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:28.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.991+0000 7f292259c640 1 -- 172.21.15.120:0/1052905391 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d589af5250 con 0x55d589ae3c80 2024-09-19T09:16:28.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.991+0000 7f292259c640 1 --2- 172.21.15.120:0/1052905391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d589ae3c80 0x55d589b046a0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f2914097d80 tx=0x7f2914097020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5cf2fa4137f1ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:29.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.991+0000 7f290a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:16:29.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.991+0000 7f290a7fc640 1 -- 172.21.15.120:0/1052905391 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291401e040 con 0x55d589ae3c80 2024-09-19T09:16:29.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.991+0000 7f290a7fc640 1 -- 172.21.15.120:0/1052905391 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f2914097a20 con 0x55d589ae3c80 2024-09-19T09:16:29.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.991+0000 7f292b5f9740 1 -- 172.21.15.120:0/1052905391 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f290c031b50 con 0x55d589ae3c80 2024-09-19T09:16:29.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.991+0000 7f290a7fc640 1 -- 172.21.15.120:0/1052905391 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2914097650 con 0x55d589ae3c80 2024-09-19T09:16:29.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.991+0000 7f292b5f9740 10 client.20133 fetch_fsmap learned FSMap version 33 2024-09-19T09:16:29.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.991+0000 7f292b5f9740 1 -- 172.21.15.120:0/1052905391 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d589af5540 con 0x55d589ae3c80 2024-09-19T09:16:29.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.991+0000 7f290a7fc640 1 -- 172.21.15.120:0/1052905391 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f290c031b50 con 0x55d589ae3c80 2024-09-19T09:16:29.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.991+0000 7f290a7fc640 1 -- 172.21.15.120:0/1052905391 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f291409a800 con 0x55d589ae3c80 2024-09-19T09:16:29.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.992+0000 7f292b5f9740 10 client.20133 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:16:29.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.992+0000 7f292b5f9740 10 client.20133 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:16:29.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.992+0000 7f292b5f9740 20 client.20133 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:29.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.992+0000 7f292b5f9740 1 --2- 172.21.15.120:0/1052905391 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d589b696b0 0x55d589b6baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:29.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.992+0000 7f292b5f9740 4 client.20133 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:16:29.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.992+0000 7f292b5f9740 1 -- 172.21.15.120:0/1052905391 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d589a50d60 con 0x55d589b696b0 2024-09-19T09:16:29.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.992+0000 7f2922d9d640 1 --2- 172.21.15.120:0/1052905391 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d589b696b0 0x55d589b6baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:29.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.992+0000 7f290a7fc640 10 client.20133 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:29.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.992+0000 7f2922d9d640 1 --2- 172.21.15.120:0/1052905391 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d589b696b0 0x55d589b6baa0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ac9bd815a4c65d7 server_cookie=52e9f6c430e8fc4d in_seq=0 out_seq=0 2024-09-19T09:16:29.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.992+0000 7f290a7fc640 1 -- 172.21.15.120:0/1052905391 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d589a50d60 con 0x55d589b696b0 2024-09-19T09:16:29.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.992+0000 7f290a7fc640 10 client.20133 handle_command_reply: tid=0 2024-09-19T09:16:29.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.992+0000 7f292b5f9740 1 client.20133 shutdown 2024-09-19T09:16:29.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.992+0000 7f292b5f9740 1 -- 172.21.15.120:0/1052905391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d589ae3c80 msgr2=0x55d589b046a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:29.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.992+0000 7f292b5f9740 1 --2- 172.21.15.120:0/1052905391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d589ae3c80 0x55d589b046a0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f2914097d80 tx=0x7f2914097020 comp rx=0 tx=0).stop 2024-09-19T09:16:29.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.993+0000 7f292b5f9740 1 -- 172.21.15.120:0/1052905391 shutdown_connections 2024-09-19T09:16:29.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.993+0000 7f292b5f9740 1 --2- 172.21.15.120:0/1052905391 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d589b696b0 0x55d589b6baa0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:29.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.993+0000 7f292b5f9740 1 --2- 172.21.15.120:0/1052905391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d589af2a90 0x55d589b68060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:29.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.993+0000 7f292b5f9740 1 --2- 172.21.15.120:0/1052905391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d589ae3c80 0x55d589b046a0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:29.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.993+0000 7f292b5f9740 1 --2- 172.21.15.120:0/1052905391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d589ae3360 0x55d589b04160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:29.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.993+0000 7f292b5f9740 1 -- 172.21.15.120:0/1052905391 >> 172.21.15.120:0/1052905391 conn(0x55d589adfa30 msgr2=0x55d589ae4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:29.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.993+0000 7f290a7fc640 0 client.20133 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:29.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.993+0000 7f292b5f9740 1 -- 172.21.15.120:0/1052905391 shutdown_connections 2024-09-19T09:16:29.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.993+0000 7f292b5f9740 1 -- 172.21.15.120:0/1052905391 wait complete. 2024-09-19T09:16:29.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.993+0000 7f292b5f9740 20 client.20133 trim_cache size 0 max 16384 2024-09-19T09:16:29.002 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:16:29.002 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:16:29.002 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:16:29.002 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:16:29.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.993+0000 7f2929220640 1 -- 172.21.15.120:0/1629216737 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f29100a3240 msgr2=0x7f29100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:29.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.993+0000 7f2929220640 1 --2- 172.21.15.120:0/1629216737 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f29100a3240 0x7f29100a5700 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f291c009fd0 tx=0x7f291c010040 comp rx=0 tx=0).stop 2024-09-19T09:16:29.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.993+0000 7f2929220640 1 -- 172.21.15.120:0/1629216737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2924102570 msgr2=0x7f2924198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:29.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.993+0000 7f2929220640 1 --2- 172.21.15.120:0/1629216737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2924102570 0x7f2924198c20 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f291400b4a0 tx=0x7f291400b960 comp rx=0 tx=0).stop 2024-09-19T09:16:29.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.993+0000 7f2929220640 1 -- 172.21.15.120:0/1629216737 shutdown_connections 2024-09-19T09:16:29.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.993+0000 7f2929220640 1 --2- 172.21.15.120:0/1629216737 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f29100a3240 0x7f29100a5700 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:29.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.993+0000 7f2929220640 1 --2- 172.21.15.120:0/1629216737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2924102ef0 0x7f292419cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:29.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.993+0000 7f2929220640 1 --2- 172.21.15.120:0/1629216737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2924102570 0x7f2924198c20 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:29.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.993+0000 7f2929220640 1 --2- 172.21.15.120:0/1629216737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2924100e20 0x7f29241986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:29.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.993+0000 7f2929220640 1 -- 172.21.15.120:0/1629216737 >> 172.21.15.120:0/1629216737 conn(0x7f29240fc7d0 msgr2=0x7f292406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:29.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.994+0000 7f2929220640 1 -- 172.21.15.120:0/1629216737 shutdown_connections 2024-09-19T09:16:29.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:28.994+0000 7f2929220640 1 -- 172.21.15.120:0/1629216737 wait complete. 2024-09-19T09:16:29.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:28 smithi120 ceph-mon[31150]: pgmap v752: 97 pgs: 97 active+clean; 13 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 568 KiB/s wr, 210 op/s 2024-09-19T09:16:29.352 INFO:tasks.workunit.client.0.smithi120.stdout: 10 795 8.93 MB/sec warmup 24 sec latency 22206.707 ms 2024-09-19T09:16:29.365 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:16:29.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:28 smithi196 ceph-mon[35150]: pgmap v752: 97 pgs: 97 active+clean; 13 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 568 KiB/s wr, 210 op/s 2024-09-19T09:16:29.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:28 smithi153 ceph-mon[35071]: pgmap v752: 97 pgs: 97 active+clean; 13 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 568 KiB/s wr, 210 op/s 2024-09-19T09:16:29.962 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:16:29.963 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:16:30.242 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:30 smithi120 ceph-mon[31150]: pgmap v753: 97 pgs: 97 active+clean; 13 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 258 KiB/s wr, 163 op/s 2024-09-19T09:16:30.352 INFO:tasks.workunit.client.0.smithi120.stdout: 10 795 8.57 MB/sec warmup 25 sec latency 23206.787 ms 2024-09-19T09:16:30.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:30 smithi196 ceph-mon[35150]: pgmap v753: 97 pgs: 97 active+clean; 13 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 258 KiB/s wr, 163 op/s 2024-09-19T09:16:30.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:30 smithi153 ceph-mon[35071]: pgmap v753: 97 pgs: 97 active+clean; 13 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 258 KiB/s wr, 163 op/s 2024-09-19T09:16:30.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.696+0000 7f3c3ad9c640 1 -- 172.21.15.120:0/1503131203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c34100980 msgr2=0x7f3c34100d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:30.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.696+0000 7f3c3ad9c640 1 --2- 172.21.15.120:0/1503131203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c34100980 0x7f3c34100d60 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f3c24009900 tx=0x7f3c2402a4b0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.703+0000 7f3c3ad9c640 1 -- 172.21.15.120:0/1503131203 shutdown_connections 2024-09-19T09:16:30.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.703+0000 7f3c3ad9c640 1 --2- 172.21.15.120:0/1503131203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c34105d90 0x7f3c34106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.703+0000 7f3c3ad9c640 1 --2- 172.21.15.120:0/1503131203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c341012a0 0x7f3c34105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.703+0000 7f3c3ad9c640 1 --2- 172.21.15.120:0/1503131203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c34100980 0x7f3c34100d60 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.703+0000 7f3c3ad9c640 1 -- 172.21.15.120:0/1503131203 >> 172.21.15.120:0/1503131203 conn(0x7f3c340fc610 msgr2=0x7f3c340fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:30.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.708+0000 7f3c3ad9c640 1 -- 172.21.15.120:0/1503131203 shutdown_connections 2024-09-19T09:16:30.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.713+0000 7f3c3ad9c640 1 -- 172.21.15.120:0/1503131203 wait complete. 2024-09-19T09:16:30.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.714+0000 7f3c3ad9c640 1 Processor -- start 2024-09-19T09:16:30.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.716+0000 7f3c3ad9c640 1 -- start start 2024-09-19T09:16:30.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.716+0000 7f3c3ad9c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c34100980 0x7f3c34072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.716+0000 7f3c3ad9c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c341012a0 0x7f3c34073470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.716+0000 7f3c3ad9c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c34105d90 0x7f3c34109e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.716+0000 7f3c3ad9c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3c34118480 con 0x7f3c34105d90 2024-09-19T09:16:30.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.716+0000 7f3c3ad9c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c34118300 con 0x7f3c34100980 2024-09-19T09:16:30.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.716+0000 7f3c3ad9c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3c34118600 con 0x7f3c341012a0 2024-09-19T09:16:30.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.717+0000 7f3c3a59b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c34105d90 0x7f3c34109e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:30.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.717+0000 7f3c3a59b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c34105d90 0x7f3c34109e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33188/0 (socket says 172.21.15.120:33188) 2024-09-19T09:16:30.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.717+0000 7f3c3a59b640 1 -- 172.21.15.120:0/2408088763 learned_addr learned my addr 172.21.15.120:0/2408088763 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:30.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.717+0000 7f3c39599640 1 --2- 172.21.15.120:0/2408088763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c341012a0 0x7f3c34073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:30.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.717+0000 7f3c39d9a640 1 --2- 172.21.15.120:0/2408088763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c34100980 0x7f3c34072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:30.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.718+0000 7f3c39d9a640 1 -- 172.21.15.120:0/2408088763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c341012a0 msgr2=0x7f3c34073470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:30.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.718+0000 7f3c39d9a640 1 --2- 172.21.15.120:0/2408088763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c341012a0 0x7f3c34073470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.718+0000 7f3c39d9a640 1 -- 172.21.15.120:0/2408088763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c34105d90 msgr2=0x7f3c34109e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:30.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.718+0000 7f3c39d9a640 1 --2- 172.21.15.120:0/2408088763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c34105d90 0x7f3c34109e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.718+0000 7f3c39d9a640 1 -- 172.21.15.120:0/2408088763 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c3410a5b0 con 0x7f3c34100980 2024-09-19T09:16:30.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.720+0000 7f3c39d9a640 1 --2- 172.21.15.120:0/2408088763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c34100980 0x7f3c34072f30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3c24033040 tx=0x7f3c240052a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:30.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.724+0000 7f3c2affd640 1 -- 172.21.15.120:0/2408088763 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c24040030 con 0x7f3c34100980 2024-09-19T09:16:30.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.724+0000 7f3c2affd640 1 -- 172.21.15.120:0/2408088763 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c2403c920 con 0x7f3c34100980 2024-09-19T09:16:30.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.724+0000 7f3c2affd640 1 -- 172.21.15.120:0/2408088763 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c240319e0 con 0x7f3c34100980 2024-09-19T09:16:30.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.724+0000 7f3c3ad9c640 1 -- 172.21.15.120:0/2408088763 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c3410a840 con 0x7f3c34100980 2024-09-19T09:16:30.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.724+0000 7f3c3ad9c640 1 -- 172.21.15.120:0/2408088763 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c3410aca0 con 0x7f3c34100980 2024-09-19T09:16:30.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.729+0000 7f3c2affd640 1 -- 172.21.15.120:0/2408088763 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f3c24043030 con 0x7f3c34100980 2024-09-19T09:16:30.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.729+0000 7f3c3e3d7740 1 Processor -- start 2024-09-19T09:16:30.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.730+0000 7f3c2affd640 1 --2- 172.21.15.120:0/2408088763 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3c200a3450 0x7f3c200a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.730+0000 7f3c2affd640 1 -- 172.21.15.120:0/2408088763 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f3c240f0f40 con 0x7f3c34100980 2024-09-19T09:16:30.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.733+0000 7f3c3e3d7740 1 -- start start 2024-09-19T09:16:30.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.734+0000 7f3c3e3d7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559416c9f210 0x559416c9f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.734+0000 7f3c3e3d7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559416c9fb30 0x559416ca9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.734+0000 7f3c3e3d7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559416caa3f0 0x559416cac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.734+0000 7f3c3e3d7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559416c04620 con 0x559416c9f210 2024-09-19T09:16:30.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.734+0000 7f3c3e3d7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559416babe50 con 0x559416caa3f0 2024-09-19T09:16:30.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.734+0000 7f3c3e3d7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559416c005e0 con 0x559416c9fb30 2024-09-19T09:16:30.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.734+0000 7f3c39d9a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559416c9f210 0x559416c9f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:30.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.734+0000 7f3c39d9a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559416c9f210 0x559416c9f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33198/0 (socket says 172.21.15.120:33198) 2024-09-19T09:16:30.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.734+0000 7f3c39d9a640 1 -- 172.21.15.120:0/1858809043 learned_addr learned my addr 172.21.15.120:0/1858809043 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:30.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.735+0000 7f3c39d9a640 1 -- 172.21.15.120:0/1858809043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559416c9fb30 msgr2=0x559416ca9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:30.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.735+0000 7f3c39d9a640 1 --2- 172.21.15.120:0/1858809043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559416c9fb30 0x559416ca9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.735+0000 7f3c39d9a640 1 -- 172.21.15.120:0/1858809043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559416caa3f0 msgr2=0x559416cac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:30.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.735+0000 7f3c39d9a640 1 --2- 172.21.15.120:0/1858809043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559416caa3f0 0x559416cac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.735+0000 7f3c39d9a640 1 -- 172.21.15.120:0/1858809043 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559416b9cd40 con 0x559416c9f210 2024-09-19T09:16:30.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.737+0000 7f3c39d9a640 1 --2- 172.21.15.120:0/1858809043 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559416c9f210 0x559416c9f5f0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f3c240b9e60 tx=0x7f3c240bb660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d0a9adff25c169 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:30.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.737+0000 7f3c3a59b640 1 --2- 172.21.15.120:0/2408088763 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3c200a3450 0x7f3c200a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:30.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.741+0000 7f3c1ffff640 1 -- 172.21.15.120:0/1858809043 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2404a040 con 0x559416c9f210 2024-09-19T09:16:30.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.741+0000 7f3c1ffff640 1 -- 172.21.15.120:0/1858809043 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c240c18d0 con 0x559416c9f210 2024-09-19T09:16:30.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.741+0000 7f3c1ffff640 1 -- 172.21.15.120:0/1858809043 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c240bd9e0 con 0x559416c9f210 2024-09-19T09:16:30.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.743+0000 7f3c3e3d7740 1 -- 172.21.15.120:0/1858809043 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559416c9f210 msgr2=0x559416c9f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:30.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.743+0000 7f3c3e3d7740 1 --2- 172.21.15.120:0/1858809043 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559416c9f210 0x559416c9f5f0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f3c240b9e60 tx=0x7f3c240bb660 comp rx=0 tx=0).stop 2024-09-19T09:16:30.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.757+0000 7f3c3e3d7740 1 -- 172.21.15.120:0/1858809043 shutdown_connections 2024-09-19T09:16:30.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.757+0000 7f3c3e3d7740 1 --2- 172.21.15.120:0/1858809043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559416caa3f0 0x559416cac7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.757+0000 7f3c3e3d7740 1 --2- 172.21.15.120:0/1858809043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559416c9fb30 0x559416ca9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.757+0000 7f3c3e3d7740 1 --2- 172.21.15.120:0/1858809043 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559416c9f210 0x559416c9f5f0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.757+0000 7f3c3e3d7740 1 -- 172.21.15.120:0/1858809043 >> 172.21.15.120:0/1858809043 conn(0x559416c9b7a0 msgr2=0x559416c9bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:30.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.760+0000 7f3c3e3d7740 1 -- 172.21.15.120:0/1858809043 shutdown_connections 2024-09-19T09:16:30.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.765+0000 7f3c3a59b640 1 --2- 172.21.15.120:0/2408088763 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3c200a3450 0x7f3c200a5910 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x559416c9cb20 tx=0x7f3c2c0098e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:30.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.766+0000 7f3c3e3d7740 1 -- 172.21.15.120:0/1858809043 wait complete. 2024-09-19T09:16:30.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.766+0000 7f3c3e3d7740 1 -- start start 2024-09-19T09:16:30.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.766+0000 7f3c3e3d7740 1 Processor -- start 2024-09-19T09:16:30.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.772+0000 7f3c3e3d7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559416c9f210 0x559416ca7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.772+0000 7f3c3e3d7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559416c9fb30 0x559416ca7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.772+0000 7f3c3e3d7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559416caa3f0 0x559416ca7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.772+0000 7f3c3e3d7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559416cacd80 con 0x559416c9fb30 2024-09-19T09:16:30.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.772+0000 7f3c3e3d7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559416b8cf10 con 0x559416caa3f0 2024-09-19T09:16:30.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.772+0000 7f3c3e3d7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559416cacf00 con 0x559416c9f210 2024-09-19T09:16:30.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.772+0000 7f3c3e3d7740 10 client.0 _ll_register_callbacks cb 0x559416bb0db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:30.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.772+0000 7f3c39d9a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559416c9f210 0x559416ca7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:30.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.772+0000 7f3c39d9a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559416c9f210 0x559416ca7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:45626/0 (socket says 172.21.15.120:45626) 2024-09-19T09:16:30.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.772+0000 7f3c39d9a640 1 -- 172.21.15.120:0/647304056 learned_addr learned my addr 172.21.15.120:0/647304056 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:30.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.772+0000 7f3c39d9a640 1 -- 172.21.15.120:0/647304056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559416caa3f0 msgr2=0x559416ca7c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:16:30.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.772+0000 7f3c39d9a640 1 --2- 172.21.15.120:0/647304056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559416caa3f0 0x559416ca7c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.772+0000 7f3c39d9a640 1 -- 172.21.15.120:0/647304056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559416c9fb30 msgr2=0x559416ca7740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:16:30.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.772+0000 7f3c39d9a640 1 --2- 172.21.15.120:0/647304056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559416c9fb30 0x559416ca7740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.772+0000 7f3c39d9a640 1 -- 172.21.15.120:0/647304056 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559416913580 con 0x559416c9f210 2024-09-19T09:16:30.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.773+0000 7f3c39d9a640 1 --2- 172.21.15.120:0/647304056 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559416c9f210 0x559416ca7200 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3c24009850 tx=0x7f3c24005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea1d4fef2c642a5d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:30.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.774+0000 7f3c1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:16:30.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.774+0000 7f3c1d7fa640 1 -- 172.21.15.120:0/647304056 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c240db060 con 0x559416c9f210 2024-09-19T09:16:30.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.774+0000 7f3c3e3d7740 1 -- 172.21.15.120:0/647304056 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c240c18d0 con 0x559416c9f210 2024-09-19T09:16:30.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.774+0000 7f3c1d7fa640 1 -- 172.21.15.120:0/647304056 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f3c240d6270 con 0x559416c9f210 2024-09-19T09:16:30.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.779+0000 7f3c1d7fa640 1 -- 172.21.15.120:0/647304056 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3c240d5070 con 0x559416c9f210 2024-09-19T09:16:30.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.785+0000 7f3c3e3d7740 10 client.28924 fetch_fsmap learned FSMap version 33 2024-09-19T09:16:30.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.785+0000 7f3c3e3d7740 1 -- 172.21.15.120:0/647304056 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5594169dd960 con 0x559416c9f210 2024-09-19T09:16:30.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.789+0000 7f3c1d7fa640 1 -- 172.21.15.120:0/647304056 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c240c18d0 con 0x559416c9f210 2024-09-19T09:16:30.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.790+0000 7f3c1d7fa640 1 -- 172.21.15.120:0/647304056 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f3c24031b80 con 0x559416c9f210 2024-09-19T09:16:30.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.790+0000 7f3c3e3d7740 10 client.28924 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:16:30.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.790+0000 7f3c3e3d7740 10 client.28924 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:16:30.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.790+0000 7f3c3e3d7740 20 client.28924 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:30.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.790+0000 7f3c3e3d7740 1 --2- 172.21.15.120:0/647304056 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559416cb9ad0 0x559416cbbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.790+0000 7f3c3e3d7740 4 client.28924 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:16:30.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.790+0000 7f3c3e3d7740 1 -- 172.21.15.120:0/647304056 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559416c11b90 con 0x559416cb9ad0 2024-09-19T09:16:30.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.791+0000 7f3c39599640 1 --2- 172.21.15.120:0/647304056 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559416cb9ad0 0x559416cbbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:30.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.796+0000 7f3c1d7fa640 10 client.28924 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:30.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.796+0000 7f3c39599640 1 --2- 172.21.15.120:0/647304056 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559416cb9ad0 0x559416cbbec0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd286f819ec334ec server_cookie=204fa95f9ab3788c in_seq=0 out_seq=0 2024-09-19T09:16:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.806+0000 7feae0c55640 1 -- 172.21.15.120:0/2446596439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feadc10dbb0 msgr2=0x7feadc110070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.806+0000 7feae0c55640 1 --2- 172.21.15.120:0/2446596439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feadc10dbb0 0x7feadc110070 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fead4009ca0 tx=0x7fead402a8f0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.806+0000 7f3c1d7fa640 1 -- 172.21.15.120:0/647304056 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559416c11b90 con 0x559416cb9ad0 2024-09-19T09:16:30.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.806+0000 7f3c1d7fa640 10 client.28924 handle_command_reply: tid=0 2024-09-19T09:16:30.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.806+0000 7f3c3e3d7740 1 client.28924 shutdown 2024-09-19T09:16:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.807+0000 7feae0c55640 1 -- 172.21.15.120:0/2446596439 shutdown_connections 2024-09-19T09:16:30.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.807+0000 7f3c3e3d7740 1 -- 172.21.15.120:0/647304056 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559416c9f210 msgr2=0x559416ca7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:30.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.807+0000 7f3c3e3d7740 1 --2- 172.21.15.120:0/647304056 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559416c9f210 0x559416ca7200 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3c24009850 tx=0x7f3c24005fd0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.810 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.807+0000 7feae0c55640 1 --2- 172.21.15.120:0/2446596439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feadc10dbb0 0x7feadc110070 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.810 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.807+0000 7feae0c55640 1 --2- 172.21.15.120:0/2446596439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feadc073680 0x7feadc10d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.810 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.807+0000 7feae0c55640 1 --2- 172.21.15.120:0/2446596439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feadc072cb0 0x7feadc0730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.810 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.807+0000 7feae0c55640 1 -- 172.21.15.120:0/2446596439 >> 172.21.15.120:0/2446596439 conn(0x7feadc06eb50 msgr2=0x7feadc070f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:30.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.807+0000 7f3c3e3d7740 1 -- 172.21.15.120:0/647304056 shutdown_connections 2024-09-19T09:16:30.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.807+0000 7f3c3e3d7740 1 --2- 172.21.15.120:0/647304056 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559416cb9ad0 0x559416cbbec0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.807+0000 7f3c3e3d7740 1 --2- 172.21.15.120:0/647304056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559416caa3f0 0x559416ca7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.807+0000 7f3c3e3d7740 1 --2- 172.21.15.120:0/647304056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559416c9fb30 0x559416ca7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.807+0000 7f3c3e3d7740 1 --2- 172.21.15.120:0/647304056 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559416c9f210 0x559416ca7200 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.807+0000 7f3c3e3d7740 1 -- 172.21.15.120:0/647304056 >> 172.21.15.120:0/647304056 conn(0x559416c9b7a0 msgr2=0x559416ca14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:30.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.807+0000 7f3c1d7fa640 0 client.28924 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:30.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.807+0000 7f3c3e3d7740 1 -- 172.21.15.120:0/647304056 shutdown_connections 2024-09-19T09:16:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.808+0000 7feae0c55640 1 -- 172.21.15.120:0/2446596439 shutdown_connections 2024-09-19T09:16:30.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.808+0000 7f3c3e3d7740 1 -- 172.21.15.120:0/647304056 wait complete. 2024-09-19T09:16:30.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.808+0000 7f3c3e3d7740 20 client.28924 trim_cache size 0 max 16384 2024-09-19T09:16:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.808+0000 7feae0c55640 1 -- 172.21.15.120:0/2446596439 wait complete. 2024-09-19T09:16:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.809+0000 7feae0c55640 1 Processor -- start 2024-09-19T09:16:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.809+0000 7feae0c55640 1 -- start start 2024-09-19T09:16:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.809+0000 7feae0c55640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feadc072cb0 0x7feadc118510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.809+0000 7feae0c55640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feadc073680 0x7feadc118a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.809+0000 7feae0c55640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feadc10dbb0 0x7feadc116b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.809+0000 7feae0c55640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feadc111370 con 0x7feadc10dbb0 2024-09-19T09:16:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.809+0000 7feae0c55640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1111f0 con 0x7feadc073680 2024-09-19T09:16:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.809+0000 7feae0c55640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1114f0 con 0x7feadc072cb0 2024-09-19T09:16:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.810+0000 7feadaffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feadc073680 0x7feadc118a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.810+0000 7feadb7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feadc072cb0 0x7feadc118510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.811+0000 7feadb7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feadc072cb0 0x7feadc118510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:45642/0 (socket says 172.21.15.120:45642) 2024-09-19T09:16:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.811+0000 7feadb7fe640 1 -- 172.21.15.120:0/2706167441 learned_addr learned my addr 172.21.15.120:0/2706167441 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.811+0000 7feadbfff640 1 --2- 172.21.15.120:0/2706167441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feadc10dbb0 0x7feadc116b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.812+0000 7feadb7fe640 1 -- 172.21.15.120:0/2706167441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feadc073680 msgr2=0x7feadc118a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.812+0000 7feadb7fe640 1 --2- 172.21.15.120:0/2706167441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feadc073680 0x7feadc118a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.812+0000 7feadb7fe640 1 -- 172.21.15.120:0/2706167441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feadc10dbb0 msgr2=0x7feadc116b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.812+0000 7feadb7fe640 1 --2- 172.21.15.120:0/2706167441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feadc10dbb0 0x7feadc116b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.812+0000 7feadb7fe640 1 -- 172.21.15.120:0/2706167441 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feadc117410 con 0x7feadc072cb0 2024-09-19T09:16:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.813+0000 7feadb7fe640 1 --2- 172.21.15.120:0/2706167441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feadc072cb0 0x7feadc118510 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7feacc00b450 tx=0x7feacc00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.813+0000 7fead8ff9640 1 -- 172.21.15.120:0/2706167441 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc015020 con 0x7feadc072cb0 2024-09-19T09:16:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.813+0000 7fead8ff9640 1 -- 172.21.15.120:0/2706167441 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feacc011d70 con 0x7feadc072cb0 2024-09-19T09:16:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.813+0000 7fead8ff9640 1 -- 172.21.15.120:0/2706167441 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc00f340 con 0x7feadc072cb0 2024-09-19T09:16:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.814+0000 7feae0c55640 1 -- 172.21.15.120:0/2706167441 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feadc117700 con 0x7feadc072cb0 2024-09-19T09:16:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.814+0000 7feae0c55640 1 -- 172.21.15.120:0/2706167441 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feadc1b79d0 con 0x7feadc072cb0 2024-09-19T09:16:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.818+0000 7feae4c53740 1 Processor -- start 2024-09-19T09:16:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.818+0000 7feae4c53740 1 -- start start 2024-09-19T09:16:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.819+0000 7feae4c53740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d86b59fb70 0x55d86b59ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.819+0000 7feae4c53740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d86b5a04b0 0x55d86b5a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.819+0000 7feae4c53740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86b5a5540 0x55d86b5a7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.819+0000 7feae4c53740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d86b5032f0 con 0x55d86b59fb70 2024-09-19T09:16:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.819+0000 7feae4c53740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d86b4ab270 con 0x55d86b5a5540 2024-09-19T09:16:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.819+0000 7feae4c53740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d86b4ff2b0 con 0x55d86b5a04b0 2024-09-19T09:16:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.819+0000 7feadaffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86b5a5540 0x55d86b5a7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.819+0000 7feadaffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86b5a5540 0x55d86b5a7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:58198/0 (socket says 172.21.15.120:58198) 2024-09-19T09:16:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.819+0000 7feadaffd640 1 -- 172.21.15.120:0/2127442264 learned_addr learned my addr 172.21.15.120:0/2127442264 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.819+0000 7feadb7fe640 1 --2- 172.21.15.120:0/2127442264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d86b5a04b0 0x55d86b5a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.819+0000 7feadaffd640 1 -- 172.21.15.120:0/2127442264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d86b5a04b0 msgr2=0x55d86b5a4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.819+0000 7feadaffd640 1 --2- 172.21.15.120:0/2127442264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d86b5a04b0 0x55d86b5a4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.819+0000 7feadaffd640 1 -- 172.21.15.120:0/2127442264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d86b59fb70 msgr2=0x55d86b59ff70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:16:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.819+0000 7feadaffd640 1 --2- 172.21.15.120:0/2127442264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d86b59fb70 0x55d86b59ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.820+0000 7feadaffd640 1 -- 172.21.15.120:0/2127442264 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d86b4aa340 con 0x55d86b5a5540 2024-09-19T09:16:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.820+0000 7feadaffd640 1 --2- 172.21.15.120:0/2127442264 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86b5a5540 0x55d86b5a7a00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fead000a860 tx=0x7fead000ad20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86216881de24cf0f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.822+0000 7feac1ffb640 1 -- 172.21.15.120:0/2127442264 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead0015070 con 0x55d86b5a5540 2024-09-19T09:16:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.822+0000 7feac1ffb640 1 -- 172.21.15.120:0/2127442264 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fead000cb40 con 0x55d86b5a5540 2024-09-19T09:16:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.822+0000 7feac1ffb640 1 -- 172.21.15.120:0/2127442264 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead00185c0 con 0x55d86b5a5540 2024-09-19T09:16:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.823+0000 7feae4c53740 1 -- 172.21.15.120:0/2127442264 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86b5a5540 msgr2=0x55d86b5a7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.823+0000 7feae4c53740 1 --2- 172.21.15.120:0/2127442264 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86b5a5540 0x55d86b5a7a00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fead000a860 tx=0x7fead000ad20 comp rx=0 tx=0).stop 2024-09-19T09:16:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.824+0000 7fead8ff9640 1 -- 172.21.15.120:0/2706167441 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7feacc018020 con 0x7feadc072cb0 2024-09-19T09:16:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.824+0000 7feae4c53740 1 -- 172.21.15.120:0/2127442264 shutdown_connections 2024-09-19T09:16:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.824+0000 7feae4c53740 1 --2- 172.21.15.120:0/2127442264 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86b5a5540 0x55d86b5a7a00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.824+0000 7feae4c53740 1 --2- 172.21.15.120:0/2127442264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d86b5a04b0 0x55d86b5a4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.824+0000 7feae4c53740 1 --2- 172.21.15.120:0/2127442264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d86b59fb70 0x55d86b59ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.824+0000 7feae4c53740 1 -- 172.21.15.120:0/2127442264 >> 172.21.15.120:0/2127442264 conn(0x55d86b59b1e0 msgr2=0x55d86b59b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.824+0000 7feae4c53740 1 -- 172.21.15.120:0/2127442264 shutdown_connections 2024-09-19T09:16:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.824+0000 7feae4c53740 1 -- 172.21.15.120:0/2127442264 wait complete. 2024-09-19T09:16:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.825+0000 7feae4c53740 1 -- start start 2024-09-19T09:16:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.825+0000 7feae4c53740 1 Processor -- start 2024-09-19T09:16:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.826+0000 7feae4c53740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d86b59fb70 0x55d86b5a9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.826+0000 7feae4c53740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86b5a04b0 0x55d86b5aa370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.826+0000 7feae4c53740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d86b5a5540 0x55d86b5b1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.826+0000 7feae4c53740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d86b5a8510 con 0x55d86b59fb70 2024-09-19T09:16:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.827+0000 7fead8ff9640 1 --2- 172.21.15.120:0/2706167441 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7feac40a3360 0x7feac40a5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.827+0000 7feadbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d86b59fb70 0x55d86b5a9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.827+0000 7feadbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d86b59fb70 0x55d86b5a9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33230/0 (socket says 172.21.15.120:33230) 2024-09-19T09:16:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.827+0000 7feadbfff640 1 -- 172.21.15.120:0/3881402764 learned_addr learned my addr 172.21.15.120:0/3881402764 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.827+0000 7feadaffd640 1 --2- 172.21.15.120:0/3881402764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d86b5a5540 0x55d86b5b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.827+0000 7fead8ff9640 1 -- 172.21.15.120:0/2706167441 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7feacc0cfa10 con 0x7feadc072cb0 2024-09-19T09:16:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.827+0000 7feae4c53740 1 -- 172.21.15.120:0/3881402764 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d86b48c290 con 0x55d86b5a04b0 2024-09-19T09:16:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.827+0000 7feae4c53740 1 -- 172.21.15.120:0/3881402764 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d86b5a8690 con 0x55d86b5a5540 2024-09-19T09:16:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.828+0000 7feae4c53740 10 client.0 _ll_register_callbacks cb 0x55d86b4bd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.828+0000 7feadaffd640 1 -- 172.21.15.120:0/3881402764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86b5a04b0 msgr2=0x55d86b5aa370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:16:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.828+0000 7feadaffd640 1 --2- 172.21.15.120:0/3881402764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86b5a04b0 0x55d86b5aa370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.828+0000 7feadaffd640 1 -- 172.21.15.120:0/3881402764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d86b59fb70 msgr2=0x55d86b5a9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.828+0000 7feadaffd640 1 --2- 172.21.15.120:0/3881402764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d86b59fb70 0x55d86b5a9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.828+0000 7feadaffd640 1 -- 172.21.15.120:0/3881402764 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d86b216210 con 0x55d86b5a5540 2024-09-19T09:16:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.828+0000 7feae4c53740 1 -- 172.21.15.120:0/3881402764 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fead000cb40 con 0x55d86b5a5540 2024-09-19T09:16:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.828+0000 7feadaffd640 1 --2- 172.21.15.120:0/3881402764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d86b5a5540 0x55d86b5b1340 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fead0008010 tx=0x7fead0011620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99976793dfc715df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:30.835 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:16:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.830+0000 7feabaffd640 10 client.28933 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:16:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.830+0000 7feadbfff640 1 --2- 172.21.15.120:0/2706167441 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7feac40a3360 0x7feac40a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.830+0000 7feabaffd640 1 -- 172.21.15.120:0/3881402764 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead0029060 con 0x55d86b5a5540 2024-09-19T09:16:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.830+0000 7feabaffd640 1 -- 172.21.15.120:0/3881402764 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fead0011710 con 0x55d86b5a5540 2024-09-19T09:16:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.830+0000 7feabaffd640 1 -- 172.21.15.120:0/3881402764 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fead0015040 con 0x55d86b5a5540 2024-09-19T09:16:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.831+0000 7feae4c53740 10 client.28933 fetch_fsmap learned FSMap version 33 2024-09-19T09:16:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.831+0000 7feae4c53740 1 -- 172.21.15.120:0/3881402764 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({fsmap=33}) -- 0x55d86b2e2620 con 0x55d86b5a5540 2024-09-19T09:16:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.832+0000 7feadbfff640 1 --2- 172.21.15.120:0/2706167441 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7feac40a3360 0x7feac40a5820 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x55d86b5a9360 tx=0x7fead40095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.832+0000 7feabaffd640 1 -- 172.21.15.120:0/3881402764 <== mon.1 v2:172.21.15.196:3300/0 4 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fead001a450 con 0x55d86b5a5540 2024-09-19T09:16:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.832+0000 7feae4c53740 10 client.28933 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:16:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.832+0000 7feae4c53740 10 client.28933 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:16:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.832+0000 7feae4c53740 20 client.28933 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.832+0000 7feae4c53740 1 --2- 172.21.15.120:0/3881402764 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d86b5b5a10 0x55d86b5b7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.832+0000 7feae4c53740 4 client.28933 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:16:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.832+0000 7feae4c53740 1 -- 172.21.15.120:0/3881402764 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d86b510a20 con 0x55d86b5b5a10 2024-09-19T09:16:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.833+0000 7feadb7fe640 1 --2- 172.21.15.120:0/3881402764 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d86b5b5a10 0x55d86b5b7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.836+0000 7feadb7fe640 1 --2- 172.21.15.120:0/3881402764 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d86b5b5a10 0x55d86b5b7e80 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7a433ccdbfc4df1 server_cookie=85b90556d5392336 in_seq=0 out_seq=0 2024-09-19T09:16:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.836+0000 7feabaffd640 10 client.28933 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.839+0000 7feabaffd640 1 -- 172.21.15.120:0/3881402764 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d86b510a20 con 0x55d86b5b5a10 2024-09-19T09:16:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.839+0000 7feabaffd640 10 client.28933 handle_command_reply: tid=0 2024-09-19T09:16:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.839+0000 7feae4c53740 1 client.28933 shutdown 2024-09-19T09:16:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.841+0000 7feae4c53740 1 -- 172.21.15.120:0/3881402764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d86b5a5540 msgr2=0x55d86b5b1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.841+0000 7feae4c53740 1 --2- 172.21.15.120:0/3881402764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d86b5a5540 0x55d86b5b1340 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fead0008010 tx=0x7fead0011620 comp rx=0 tx=0).stop 2024-09-19T09:16:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.841+0000 7feae4c53740 1 -- 172.21.15.120:0/3881402764 shutdown_connections 2024-09-19T09:16:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.841+0000 7feae4c53740 1 --2- 172.21.15.120:0/3881402764 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d86b5b5a10 0x55d86b5b7e80 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.841+0000 7feae4c53740 1 --2- 172.21.15.120:0/3881402764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d86b5a5540 0x55d86b5b1340 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.841+0000 7feae4c53740 1 --2- 172.21.15.120:0/3881402764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86b5a04b0 0x55d86b5aa370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.841+0000 7feae4c53740 1 --2- 172.21.15.120:0/3881402764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d86b59fb70 0x55d86b5a9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.841+0000 7feae4c53740 1 -- 172.21.15.120:0/3881402764 >> 172.21.15.120:0/3881402764 conn(0x55d86b59b1e0 msgr2=0x55d86b59c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.841+0000 7feabaffd640 0 client.28933 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.841+0000 7feae4c53740 1 -- 172.21.15.120:0/3881402764 shutdown_connections 2024-09-19T09:16:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.841+0000 7feae4c53740 1 -- 172.21.15.120:0/3881402764 wait complete. 2024-09-19T09:16:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.842+0000 7feae4c53740 20 client.28933 trim_cache size 0 max 16384 2024-09-19T09:16:30.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.845+0000 7f3c3e3d7740 1 Processor -- start 2024-09-19T09:16:30.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.846+0000 7f3c3e3d7740 1 -- start start 2024-09-19T09:16:30.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.846+0000 7f3c3e3d7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559416c9f210 0x559416cc1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.846+0000 7f3c3e3d7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559416c9fb30 0x559416cc1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.846+0000 7f3c3e3d7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559416caa3f0 0x559416ca4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.846+0000 7f3c3e3d7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1800e440 con 0x559416caa3f0 2024-09-19T09:16:30.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.846+0000 7f3c3e3d7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1800ff80 con 0x559416c9f210 2024-09-19T09:16:30.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.846+0000 7f3c3e3d7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1800eb10 con 0x559416c9fb30 2024-09-19T09:16:30.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.846+0000 7f3c3a59b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559416caa3f0 0x559416ca4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:30.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.846+0000 7f3c3a59b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559416caa3f0 0x559416ca4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33238/0 (socket says 172.21.15.120:33238) 2024-09-19T09:16:30.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.846+0000 7f3c3a59b640 1 -- 172.21.15.120:0/3468782245 learned_addr learned my addr 172.21.15.120:0/3468782245 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:30.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.846+0000 7f3c39d9a640 1 --2- 172.21.15.120:0/3468782245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559416c9f210 0x559416cc1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:30.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.847+0000 7f3c3a59b640 1 -- 172.21.15.120:0/3468782245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559416c9fb30 msgr2=0x559416cc1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:30.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.847+0000 7f3c3a59b640 1 --2- 172.21.15.120:0/3468782245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559416c9fb30 0x559416cc1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.847+0000 7f3c3a59b640 1 -- 172.21.15.120:0/3468782245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559416c9f210 msgr2=0x559416cc1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:30.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.847+0000 7f3c3a59b640 1 --2- 172.21.15.120:0/3468782245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559416c9f210 0x559416cc1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.847+0000 7f3c3a59b640 1 -- 172.21.15.120:0/3468782245 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559416c9c850 con 0x559416caa3f0 2024-09-19T09:16:30.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.847+0000 7f3c3a59b640 1 --2- 172.21.15.120:0/3468782245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559416caa3f0 0x559416ca4460 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c004e30 tx=0x7f3c2c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a69b5e95aa5c0cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:30.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.847+0000 7f3c1f7fe640 1 -- 172.21.15.120:0/3468782245 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c004e60 con 0x559416caa3f0 2024-09-19T09:16:30.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.847+0000 7f3c1f7fe640 1 -- 172.21.15.120:0/3468782245 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c2c025070 con 0x559416caa3f0 2024-09-19T09:16:30.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.847+0000 7f3c1f7fe640 1 -- 172.21.15.120:0/3468782245 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c020aa0 con 0x559416caa3f0 2024-09-19T09:16:30.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.849+0000 7f3c3e3d7740 1 -- 172.21.15.120:0/3468782245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559416caa3f0 msgr2=0x559416ca4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:30.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.849+0000 7f3c3e3d7740 1 --2- 172.21.15.120:0/3468782245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559416caa3f0 0x559416ca4460 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c004e30 tx=0x7f3c2c014000 comp rx=0 tx=0).stop 2024-09-19T09:16:30.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.849+0000 7f3c3e3d7740 1 -- 172.21.15.120:0/3468782245 shutdown_connections 2024-09-19T09:16:30.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.849+0000 7f3c3e3d7740 1 --2- 172.21.15.120:0/3468782245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559416caa3f0 0x559416ca4460 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.849+0000 7f3c3e3d7740 1 --2- 172.21.15.120:0/3468782245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559416c9fb30 0x559416cc1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.849+0000 7f3c3e3d7740 1 --2- 172.21.15.120:0/3468782245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559416c9f210 0x559416cc1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.849+0000 7f3c3e3d7740 1 -- 172.21.15.120:0/3468782245 >> 172.21.15.120:0/3468782245 conn(0x559416c9b7a0 msgr2=0x559416ca5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:30.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.850+0000 7f3c3e3d7740 1 -- 172.21.15.120:0/3468782245 shutdown_connections 2024-09-19T09:16:30.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.850+0000 7f3c3e3d7740 1 -- 172.21.15.120:0/3468782245 wait complete. 2024-09-19T09:16:30.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.851+0000 7f3c3e3d7740 1 -- start start 2024-09-19T09:16:30.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.851+0000 7f3c3e3d7740 1 Processor -- start 2024-09-19T09:16:30.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.852+0000 7f3c3e3d7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559416c9f210 0x559416cae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.852+0000 7f3c3e3d7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559416c9fb30 0x559416cae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.852+0000 7f3c3e3d7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559416caa3f0 0x559416d236f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.852+0000 7f3c39d9a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559416c9f210 0x559416cae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:30.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.852+0000 7f3c39d9a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559416c9f210 0x559416cae2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:58210/0 (socket says 172.21.15.120:58210) 2024-09-19T09:16:30.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.852+0000 7f3c39d9a640 1 -- 172.21.15.120:0/2891425765 learned_addr learned my addr 172.21.15.120:0/2891425765 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:30.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.853+0000 7f3c39599640 1 --2- 172.21.15.120:0/2891425765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559416c9fb30 0x559416cae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:30.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.853+0000 7f3c3e3d7740 1 -- 172.21.15.120:0/2891425765 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1800f8b0 con 0x559416caa3f0 2024-09-19T09:16:30.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.853+0000 7f3c3e3d7740 1 -- 172.21.15.120:0/2891425765 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1800f1e0 con 0x559416c9f210 2024-09-19T09:16:30.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.853+0000 7f3c3e3d7740 1 -- 172.21.15.120:0/2891425765 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1800c4d0 con 0x559416c9fb30 2024-09-19T09:16:30.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.853+0000 7f3c3e3d7740 10 client.0 _ll_register_callbacks cb 0x7f3c240bda90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:30.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.854+0000 7f3c39d9a640 1 -- 172.21.15.120:0/2891425765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559416c9fb30 msgr2=0x559416cae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:30.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.854+0000 7f3c39d9a640 1 --2- 172.21.15.120:0/2891425765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559416c9fb30 0x559416cae7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.854+0000 7f3c39d9a640 1 -- 172.21.15.120:0/2891425765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559416caa3f0 msgr2=0x559416d236f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:30.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.854+0000 7f3c39d9a640 1 --2- 172.21.15.120:0/2891425765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559416caa3f0 0x559416d236f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.854+0000 7f3c39d9a640 1 -- 172.21.15.120:0/2891425765 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559416d23c30 con 0x559416c9f210 2024-09-19T09:16:30.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.854+0000 7f3c39d9a640 1 --2- 172.21.15.120:0/2891425765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559416c9f210 0x559416cae2b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3c24031f80 tx=0x7f3c240bb790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dfa9f0609d3c2ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:30.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.854+0000 7f3c1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:16:30.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.854+0000 7f3c1d7fa640 1 -- 172.21.15.120:0/2891425765 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c240db070 con 0x559416c9f210 2024-09-19T09:16:30.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.854+0000 7f3c1d7fa640 1 -- 172.21.15.120:0/2891425765 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f3c240bcb60 con 0x559416c9f210 2024-09-19T09:16:30.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.855+0000 7f3c3e3d7740 1 -- 172.21.15.120:0/2891425765 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c2c025070 con 0x559416c9f210 2024-09-19T09:16:30.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.855+0000 7f3c1d7fa640 1 -- 172.21.15.120:0/2891425765 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3c2404a070 con 0x559416c9f210 2024-09-19T09:16:30.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.855+0000 7f3c3e3d7740 10 client.29108 fetch_fsmap learned FSMap version 33 2024-09-19T09:16:30.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.855+0000 7f3c3e3d7740 1 -- 172.21.15.120:0/2891425765 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559416d23c30 con 0x559416c9f210 2024-09-19T09:16:30.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.856+0000 7f3c1d7fa640 1 -- 172.21.15.120:0/2891425765 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c240b9520 con 0x559416c9f210 2024-09-19T09:16:30.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.856+0000 7f3c1d7fa640 1 -- 172.21.15.120:0/2891425765 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f3c2403cad0 con 0x559416c9f210 2024-09-19T09:16:30.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.856+0000 7f3c3e3d7740 10 client.29108 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:16:30.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.856+0000 7f3c3e3d7740 10 client.29108 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:16:30.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.856+0000 7f3c3e3d7740 20 client.29108 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:30.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.856+0000 7f3c3e3d7740 1 --2- 172.21.15.120:0/2891425765 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559416d25120 0x559416d27540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.857+0000 7f3c3e3d7740 4 client.29108 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1e2662fa", "release": true, "await": true}] 2024-09-19T09:16:30.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.857+0000 7f3c3e3d7740 1 -- 172.21.15.120:0/2891425765 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1e2662fa", "release": true, "await": true}) -- 0x7f3c2c025070 con 0x559416d25120 2024-09-19T09:16:30.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.857+0000 7f3c39599640 1 --2- 172.21.15.120:0/2891425765 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559416d25120 0x559416d27540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:30.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.857+0000 7f3c39599640 1 --2- 172.21.15.120:0/2891425765 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559416d25120 0x559416d27540 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8826c7c1336fd7fd server_cookie=20404915b7fbc518 in_seq=0 out_seq=0 2024-09-19T09:16:30.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.857+0000 7f3c1d7fa640 10 client.29108 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.868+0000 7feae4c53740 1 Processor -- start 2024-09-19T09:16:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.868+0000 7feae4c53740 1 -- start start 2024-09-19T09:16:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.868+0000 7feae4c53740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d86b59fb70 0x55d86b59b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.868+0000 7feae4c53740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86b5a04b0 0x55d86b5b1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.868+0000 7feae4c53740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d86b5a5540 0x55d86b59bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.868+0000 7feae4c53740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1c06b0 con 0x55d86b5a5540 2024-09-19T09:16:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.868+0000 7feae4c53740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1c21f0 con 0x55d86b5a04b0 2024-09-19T09:16:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.868+0000 7feae4c53740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1c0d80 con 0x55d86b59fb70 2024-09-19T09:16:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.868+0000 7feadaffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86b5a04b0 0x55d86b5b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.868+0000 7feadb7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d86b59fb70 0x55d86b59b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.868+0000 7feadb7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d86b59fb70 0x55d86b59b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:45692/0 (socket says 172.21.15.120:45692) 2024-09-19T09:16:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.868+0000 7feadb7fe640 1 -- 172.21.15.120:0/3804643652 learned_addr learned my addr 172.21.15.120:0/3804643652 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.869+0000 7feadbfff640 1 --2- 172.21.15.120:0/3804643652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d86b5a5540 0x55d86b59bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.869+0000 7feadb7fe640 1 -- 172.21.15.120:0/3804643652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86b5a04b0 msgr2=0x55d86b5b1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.869+0000 7feadb7fe640 1 --2- 172.21.15.120:0/3804643652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86b5a04b0 0x55d86b5b1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.869+0000 7feadb7fe640 1 -- 172.21.15.120:0/3804643652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d86b5a5540 msgr2=0x55d86b59bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.869+0000 7feadb7fe640 1 --2- 172.21.15.120:0/3804643652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d86b5a5540 0x55d86b59bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.869+0000 7feadb7fe640 1 -- 172.21.15.120:0/3804643652 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d86b5a9970 con 0x55d86b59fb70 2024-09-19T09:16:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.869+0000 7feadaffd640 1 --2- 172.21.15.120:0/3804643652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86b5a04b0 0x55d86b5b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:16:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.869+0000 7feadb7fe640 1 --2- 172.21.15.120:0/3804643652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d86b59fb70 0x55d86b59b5c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feacc004790 tx=0x7feacc010a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a8cc7abd6df6aaa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.869+0000 7feac17fa640 1 -- 172.21.15.120:0/3804643652 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc0a4c60 con 0x55d86b59fb70 2024-09-19T09:16:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.869+0000 7feac17fa640 1 -- 172.21.15.120:0/3804643652 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feacc09a2f0 con 0x55d86b59fb70 2024-09-19T09:16:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.869+0000 7feac17fa640 1 -- 172.21.15.120:0/3804643652 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc0a4a30 con 0x55d86b59fb70 2024-09-19T09:16:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.870+0000 7feae4c53740 1 -- 172.21.15.120:0/3804643652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d86b59fb70 msgr2=0x55d86b59b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.870+0000 7feae4c53740 1 --2- 172.21.15.120:0/3804643652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d86b59fb70 0x55d86b59b5c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feacc004790 tx=0x7feacc010a00 comp rx=0 tx=0).stop 2024-09-19T09:16:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.870+0000 7feae4c53740 1 -- 172.21.15.120:0/3804643652 shutdown_connections 2024-09-19T09:16:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.870+0000 7feae4c53740 1 --2- 172.21.15.120:0/3804643652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d86b5a5540 0x55d86b59bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.870+0000 7feae4c53740 1 --2- 172.21.15.120:0/3804643652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86b5a04b0 0x55d86b5b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.870+0000 7feae4c53740 1 --2- 172.21.15.120:0/3804643652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d86b59fb70 0x55d86b59b5c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.870+0000 7feae4c53740 1 -- 172.21.15.120:0/3804643652 >> 172.21.15.120:0/3804643652 conn(0x55d86b59b1e0 msgr2=0x55d86b5a1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.870+0000 7feae4c53740 1 -- 172.21.15.120:0/3804643652 shutdown_connections 2024-09-19T09:16:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.870+0000 7feae4c53740 1 -- 172.21.15.120:0/3804643652 wait complete. 2024-09-19T09:16:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.871+0000 7feae4c53740 1 -- start start 2024-09-19T09:16:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.871+0000 7feae4c53740 1 Processor -- start 2024-09-19T09:16:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.871+0000 7feae4c53740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86b59fb70 0x55d86b5adf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.871+0000 7feae4c53740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d86b5a04b0 0x55d86b5ae480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.871+0000 7feae4c53740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d86b5a5540 0x55d86b61e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.871+0000 7feae4c53740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1c1b20 con 0x55d86b5a04b0 2024-09-19T09:16:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.871+0000 7feae4c53740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1c1450 con 0x55d86b59fb70 2024-09-19T09:16:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.871+0000 7feae4c53740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1be670 con 0x55d86b5a5540 2024-09-19T09:16:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.871+0000 7feae4c53740 10 client.0 _ll_register_callbacks cb 0x7feacc0046e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.872+0000 7feadb7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86b59fb70 0x55d86b5adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.872+0000 7feadb7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86b59fb70 0x55d86b5adf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:58230/0 (socket says 172.21.15.120:58230) 2024-09-19T09:16:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.872+0000 7feadb7fe640 1 -- 172.21.15.120:0/3187567080 learned_addr learned my addr 172.21.15.120:0/3187567080 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.872+0000 7feadaffd640 1 --2- 172.21.15.120:0/3187567080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d86b5a04b0 0x55d86b5ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.872+0000 7feadbfff640 1 --2- 172.21.15.120:0/3187567080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d86b5a5540 0x55d86b61e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.872+0000 7feadb7fe640 1 -- 172.21.15.120:0/3187567080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d86b5a5540 msgr2=0x55d86b61e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.872+0000 7feadb7fe640 1 --2- 172.21.15.120:0/3187567080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d86b5a5540 0x55d86b61e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.872+0000 7feadb7fe640 1 -- 172.21.15.120:0/3187567080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d86b5a04b0 msgr2=0x55d86b5ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.872+0000 7feadb7fe640 1 --2- 172.21.15.120:0/3187567080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d86b5a04b0 0x55d86b5ae480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.872+0000 7feadb7fe640 1 -- 172.21.15.120:0/3187567080 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d86b5b1520 con 0x55d86b59fb70 2024-09-19T09:16:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.872+0000 7feadb7fe640 1 --2- 172.21.15.120:0/3187567080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86b59fb70 0x55d86b5adf40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7feacc00be10 tx=0x7feacc0cfc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c770bbcfd6b4d58b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.872+0000 7feabaffd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:16:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.872+0000 7feabaffd640 1 -- 172.21.15.120:0/3187567080 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc0a8020 con 0x55d86b59fb70 2024-09-19T09:16:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.872+0000 7feae4c53740 1 -- 172.21.15.120:0/3187567080 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feacc09a2f0 con 0x55d86b59fb70 2024-09-19T09:16:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.873+0000 7feabaffd640 1 -- 172.21.15.120:0/3187567080 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7feacc0a49d0 con 0x55d86b59fb70 2024-09-19T09:16:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.873+0000 7feabaffd640 1 -- 172.21.15.120:0/3187567080 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7feacc0bc0b0 con 0x55d86b59fb70 2024-09-19T09:16:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.873+0000 7feae4c53740 10 client.29114 fetch_fsmap learned FSMap version 33 2024-09-19T09:16:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.873+0000 7feae4c53740 1 -- 172.21.15.120:0/3187567080 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d86b61fc20 con 0x55d86b59fb70 2024-09-19T09:16:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.873+0000 7feabaffd640 1 -- 172.21.15.120:0/3187567080 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feacc09a2f0 con 0x55d86b59fb70 2024-09-19T09:16:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.873+0000 7feabaffd640 1 -- 172.21.15.120:0/3187567080 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7feacc09b440 con 0x55d86b59fb70 2024-09-19T09:16:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.873+0000 7feae4c53740 10 client.29114 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T09:16:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.873+0000 7feae4c53740 10 client.29114 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:16:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.873+0000 7feae4c53740 20 client.29114 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.873+0000 7feae4c53740 1 --2- 172.21.15.120:0/3187567080 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d86b6204a0 0x55d86b6228c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.873+0000 7feae4c53740 4 client.29114 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:16:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.873+0000 7feae4c53740 1 -- 172.21.15.120:0/3187567080 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d86b50ad70 con 0x55d86b6204a0 2024-09-19T09:16:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.873+0000 7feadaffd640 1 --2- 172.21.15.120:0/3187567080 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d86b6204a0 0x55d86b6228c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.874+0000 7feabaffd640 10 client.29114 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:30.874+0000 7feadaffd640 1 --2- 172.21.15.120:0/3187567080 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d86b6204a0 0x55d86b6228c0 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0c8f4decf685c86 server_cookie=7b9f88e2df2b81ab in_seq=0 out_seq=0 2024-09-19T09:16:30.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.935+0000 7f3c1d7fa640 1 -- 172.21.15.120:0/2891425765 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f3c2c025070 con 0x559416d25120 2024-09-19T09:16:30.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.935+0000 7f3c1d7fa640 10 client.29108 handle_command_reply: tid=0 2024-09-19T09:16:30.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.935+0000 7f3c3e3d7740 1 client.29108 shutdown 2024-09-19T09:16:30.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.936+0000 7f3c3e3d7740 1 -- 172.21.15.120:0/2891425765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559416c9f210 msgr2=0x559416cae2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:30.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.936+0000 7f3c3e3d7740 1 --2- 172.21.15.120:0/2891425765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559416c9f210 0x559416cae2b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3c24031f80 tx=0x7f3c240bb790 comp rx=0 tx=0).stop 2024-09-19T09:16:30.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.936+0000 7f3c3e3d7740 1 -- 172.21.15.120:0/2891425765 shutdown_connections 2024-09-19T09:16:30.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.936+0000 7f3c3e3d7740 1 --2- 172.21.15.120:0/2891425765 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559416d25120 0x559416d27540 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.936+0000 7f3c3e3d7740 1 --2- 172.21.15.120:0/2891425765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559416caa3f0 0x559416d236f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.936+0000 7f3c3e3d7740 1 --2- 172.21.15.120:0/2891425765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559416c9fb30 0x559416cae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.936+0000 7f3c3e3d7740 1 --2- 172.21.15.120:0/2891425765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559416c9f210 0x559416cae2b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.936+0000 7f3c1d7fa640 0 client.29108 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:30.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.936+0000 7f3c3e3d7740 1 -- 172.21.15.120:0/2891425765 >> 172.21.15.120:0/2891425765 conn(0x559416c9b7a0 msgr2=0x559416ca71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:30.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.936+0000 7f3c3e3d7740 1 -- 172.21.15.120:0/2891425765 shutdown_connections 2024-09-19T09:16:30.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.936+0000 7f3c3e3d7740 1 -- 172.21.15.120:0/2891425765 wait complete. 2024-09-19T09:16:30.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.936+0000 7f3c3e3d7740 20 client.29108 trim_cache size 0 max 16384 2024-09-19T09:16:30.940 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:16:30.940 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 33, 2024-09-19T09:16:30.940 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:16:30.940 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 48, 2024-09-19T09:16:30.940 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:16:30.940 INFO:teuthology.orchestra.run.smithi120.stdout: "1e2662fa": { 2024-09-19T09:16:30.940 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 48, 2024-09-19T09:16:30.940 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:16:30.941 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:16:30.941 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:16:30.941 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:16:30.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:16:30.941 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:16:30.941 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 142.2, 2024-09-19T09:16:30.941 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:16:30.941 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:16:30.941 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:16:30.941 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:16:30.941 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:16:30.941 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:16:30.941 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:16:30.942 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:16:30.942 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:16:30.942 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:16:30.942 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:16:30.942 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:16:30.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.937+0000 7f3c3ad9c640 1 -- 172.21.15.120:0/2408088763 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3c200a3450 msgr2=0x7f3c200a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:30.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.937+0000 7f3c3ad9c640 1 --2- 172.21.15.120:0/2408088763 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3c200a3450 0x7f3c200a5910 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x559416c9cb20 tx=0x7f3c2c0098e0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.937+0000 7f3c3ad9c640 1 -- 172.21.15.120:0/2408088763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c34100980 msgr2=0x7f3c34072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:30.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.937+0000 7f3c3ad9c640 1 --2- 172.21.15.120:0/2408088763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c34100980 0x7f3c34072f30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3c24033040 tx=0x7f3c240052a0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.937+0000 7f3c3ad9c640 1 -- 172.21.15.120:0/2408088763 shutdown_connections 2024-09-19T09:16:30.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.937+0000 7f3c3ad9c640 1 --2- 172.21.15.120:0/2408088763 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3c200a3450 0x7f3c200a5910 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.937+0000 7f3c3ad9c640 1 --2- 172.21.15.120:0/2408088763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c34105d90 0x7f3c34109e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.937+0000 7f3c3ad9c640 1 --2- 172.21.15.120:0/2408088763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c341012a0 0x7f3c34073470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.937+0000 7f3c3ad9c640 1 --2- 172.21.15.120:0/2408088763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c34100980 0x7f3c34072f30 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:30.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.937+0000 7f3c3ad9c640 1 -- 172.21.15.120:0/2408088763 >> 172.21.15.120:0/2408088763 conn(0x7f3c340fc610 msgr2=0x7f3c340fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:30.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.937+0000 7f3c3ad9c640 1 -- 172.21.15.120:0/2408088763 shutdown_connections 2024-09-19T09:16:30.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:30.937+0000 7f3c3ad9c640 1 -- 172.21.15.120:0/2408088763 wait complete. 2024-09-19T09:16:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:31.314+0000 7feabaffd640 1 -- 172.21.15.120:0/3187567080 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d86b50ad70 con 0x55d86b6204a0 2024-09-19T09:16:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:31.314+0000 7feabaffd640 10 client.29114 handle_command_reply: tid=0 2024-09-19T09:16:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:31.314+0000 7feae4c53740 1 client.29114 shutdown 2024-09-19T09:16:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:31.314+0000 7feae4c53740 1 -- 172.21.15.120:0/3187567080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86b59fb70 msgr2=0x55d86b5adf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:31.314+0000 7feae4c53740 1 --2- 172.21.15.120:0/3187567080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86b59fb70 0x55d86b5adf40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7feacc00be10 tx=0x7feacc0cfc60 comp rx=0 tx=0).stop 2024-09-19T09:16:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:31.314+0000 7feae4c53740 1 -- 172.21.15.120:0/3187567080 shutdown_connections 2024-09-19T09:16:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:31.314+0000 7feae4c53740 1 --2- 172.21.15.120:0/3187567080 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d86b6204a0 0x55d86b6228c0 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:31.314+0000 7feae4c53740 1 --2- 172.21.15.120:0/3187567080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d86b5a5540 0x55d86b61e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:31.314+0000 7feabaffd640 0 client.29114 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:31.314+0000 7feae4c53740 1 --2- 172.21.15.120:0/3187567080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d86b5a04b0 0x55d86b5ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:31.314+0000 7feae4c53740 1 --2- 172.21.15.120:0/3187567080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d86b59fb70 0x55d86b5adf40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:31.314+0000 7feae4c53740 1 -- 172.21.15.120:0/3187567080 >> 172.21.15.120:0/3187567080 conn(0x55d86b59b1e0 msgr2=0x55d86b599a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:31.314+0000 7feae4c53740 1 -- 172.21.15.120:0/3187567080 shutdown_connections 2024-09-19T09:16:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:31.314+0000 7feae4c53740 1 -- 172.21.15.120:0/3187567080 wait complete. 2024-09-19T09:16:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:31.315+0000 7feae4c53740 20 client.29114 trim_cache size 0 max 16384 2024-09-19T09:16:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:16:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:16:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:16:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:16:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:31.315+0000 7feae0c55640 1 -- 172.21.15.120:0/2706167441 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7feac40a3360 msgr2=0x7feac40a5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:31.315+0000 7feae0c55640 1 --2- 172.21.15.120:0/2706167441 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7feac40a3360 0x7feac40a5820 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x55d86b5a9360 tx=0x7fead40095c0 comp rx=0 tx=0).stop 2024-09-19T09:16:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:31.315+0000 7feae0c55640 1 -- 172.21.15.120:0/2706167441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feadc072cb0 msgr2=0x7feadc118510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:31.315+0000 7feae0c55640 1 --2- 172.21.15.120:0/2706167441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feadc072cb0 0x7feadc118510 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7feacc00b450 tx=0x7feacc00b910 comp rx=0 tx=0).stop 2024-09-19T09:16:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:31.315+0000 7feae0c55640 1 -- 172.21.15.120:0/2706167441 shutdown_connections 2024-09-19T09:16:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:31.315+0000 7feae0c55640 1 --2- 172.21.15.120:0/2706167441 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7feac40a3360 0x7feac40a5820 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:31.315+0000 7feae0c55640 1 --2- 172.21.15.120:0/2706167441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feadc10dbb0 0x7feadc116b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:31.315+0000 7feae0c55640 1 --2- 172.21.15.120:0/2706167441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feadc073680 0x7feadc118a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:31.315+0000 7feae0c55640 1 --2- 172.21.15.120:0/2706167441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feadc072cb0 0x7feadc118510 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:31.315+0000 7feae0c55640 1 -- 172.21.15.120:0/2706167441 >> 172.21.15.120:0/2706167441 conn(0x7feadc06eb50 msgr2=0x7feadc06f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:31.315+0000 7feae0c55640 1 -- 172.21.15.120:0/2706167441 shutdown_connections 2024-09-19T09:16:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:31.315+0000 7feae0c55640 1 -- 172.21.15.120:0/2706167441 wait complete. 2024-09-19T09:16:31.352 INFO:tasks.workunit.client.0.smithi120.stdout: 10 863 8.31 MB/sec warmup 26 sec latency 23905.648 ms 2024-09-19T09:16:31.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.600+0000 7f0415245640 1 -- 172.21.15.120:0/1882164719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0410102570 msgr2=0x7f04101029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:31.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.600+0000 7f0415245640 1 --2- 172.21.15.120:0/1882164719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0410102570 0x7f04101029b0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f040400d910 tx=0x7f040402c400 comp rx=0 tx=0).stop 2024-09-19T09:16:31.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.600+0000 7f0415245640 1 -- 172.21.15.120:0/1882164719 shutdown_connections 2024-09-19T09:16:31.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.600+0000 7f0415245640 1 --2- 172.21.15.120:0/1882164719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0410102ef0 0x7f041010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:31.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.600+0000 7f0415245640 1 --2- 172.21.15.120:0/1882164719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0410102570 0x7f04101029b0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:31.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.600+0000 7f0415245640 1 --2- 172.21.15.120:0/1882164719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0410100e20 0x7f0410101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:31.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.600+0000 7f0415245640 1 -- 172.21.15.120:0/1882164719 >> 172.21.15.120:0/1882164719 conn(0x7f04100fc7d0 msgr2=0x7f04100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:31.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.600+0000 7f0415245640 1 -- 172.21.15.120:0/1882164719 shutdown_connections 2024-09-19T09:16:31.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.600+0000 7f0415245640 1 -- 172.21.15.120:0/1882164719 wait complete. 2024-09-19T09:16:31.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.601+0000 7f0415245640 1 Processor -- start 2024-09-19T09:16:31.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.601+0000 7f0415245640 1 -- start start 2024-09-19T09:16:31.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.602+0000 7f0415245640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0410100e20 0x7f0410196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:31.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.602+0000 7f0415245640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0410102570 0x7f0410196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:31.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.602+0000 7f0415245640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0410102ef0 0x7f041019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:31.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.602+0000 7f0415245640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f041010fe80 con 0x7f0410100e20 2024-09-19T09:16:31.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.602+0000 7f0415245640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f041010fd00 con 0x7f0410102ef0 2024-09-19T09:16:31.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.602+0000 7f0415245640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0410110000 con 0x7f0410102570 2024-09-19T09:16:31.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.602+0000 7f040e59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0410102570 0x7f0410196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:31.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.602+0000 7f040ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0410100e20 0x7f0410196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:31.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.602+0000 7f040e59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0410102570 0x7f0410196860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:33418/0 (socket says 172.21.15.120:33418) 2024-09-19T09:16:31.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.602+0000 7f040e59c640 1 -- 172.21.15.120:0/886714333 learned_addr learned my addr 172.21.15.120:0/886714333 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:31.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.602+0000 7f040f59e640 1 --2- 172.21.15.120:0/886714333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0410102ef0 0x7f041019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:31.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.602+0000 7f040e59c640 1 -- 172.21.15.120:0/886714333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0410102ef0 msgr2=0x7f041019abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:31.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.602+0000 7f040e59c640 1 --2- 172.21.15.120:0/886714333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0410102ef0 0x7f041019abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:31.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.602+0000 7f040e59c640 1 -- 172.21.15.120:0/886714333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0410100e20 msgr2=0x7f0410196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:31.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.602+0000 7f040e59c640 1 --2- 172.21.15.120:0/886714333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0410100e20 0x7f0410196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:31.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.602+0000 7f040e59c640 1 -- 172.21.15.120:0/886714333 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041019b2d0 con 0x7f0410102570 2024-09-19T09:16:31.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.602+0000 7f040e59c640 1 --2- 172.21.15.120:0/886714333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0410102570 0x7f0410196860 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0404000c00 tx=0x7f040400d110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:31.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.603+0000 7f03f7fff640 1 -- 172.21.15.120:0/886714333 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404042030 con 0x7f0410102570 2024-09-19T09:16:31.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.603+0000 7f0415245640 1 -- 172.21.15.120:0/886714333 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f041019b560 con 0x7f0410102570 2024-09-19T09:16:31.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.603+0000 7f03f7fff640 1 -- 172.21.15.120:0/886714333 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f040400c8f0 con 0x7f0410102570 2024-09-19T09:16:31.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.603+0000 7f03f7fff640 1 -- 172.21.15.120:0/886714333 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404004bf0 con 0x7f0410102570 2024-09-19T09:16:31.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.603+0000 7f0415245640 1 -- 172.21.15.120:0/886714333 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04101a2ea0 con 0x7f0410102570 2024-09-19T09:16:31.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.604+0000 7f0415245640 1 -- 172.21.15.120:0/886714333 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04101021b0 con 0x7f0410102570 2024-09-19T09:16:31.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.605+0000 7f03f7fff640 1 -- 172.21.15.120:0/886714333 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f0404045030 con 0x7f0410102570 2024-09-19T09:16:31.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.606+0000 7f03f7fff640 1 --2- 172.21.15.120:0/886714333 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f03f80a3240 0x7f03f80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:31.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.606+0000 7f03f7fff640 1 -- 172.21.15.120:0/886714333 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f04040f1980 con 0x7f0410102570 2024-09-19T09:16:31.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.606+0000 7f040ed9d640 1 --2- 172.21.15.120:0/886714333 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f03f80a3240 0x7f03f80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:31.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.607+0000 7f040ed9d640 1 --2- 172.21.15.120:0/886714333 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f03f80a3240 0x7f03f80a5700 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f0400009bd0 tx=0x7f0400005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:31.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.610+0000 7f03f7fff640 1 -- 172.21.15.120:0/886714333 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f04040be010 con 0x7f0410102570 2024-09-19T09:16:31.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:31 smithi153 ceph-mon[35071]: daemon mds.e restarted 2024-09-19T09:16:31.688 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:16:31 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-e[63911]: ignoring --setuser ceph since I am not root 2024-09-19T09:16:31.688 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:16:31 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-e[63911]: ignoring --setgroup ceph since I am not root 2024-09-19T09:16:31.688 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:16:31 smithi153 ceph-mds[63915]: -- 172.21.15.153:0/480221827 <== mon.2 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555890c17860 con 0x555890bf5400 2024-09-19T09:16:31.688 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:16:31 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-e[63911]: starting mds.e at 2024-09-19T09:16:31.738 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:31 smithi120 ceph-mon[31150]: daemon mds.e restarted 2024-09-19T09:16:31.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.854+0000 7f0415245640 1 -- 172.21.15.120:0/886714333 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0410109090 con 0x7f0410102570 2024-09-19T09:16:31.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.855+0000 7f03f7fff640 1 -- 172.21.15.120:0/886714333 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11988 (secure 0 0 0) 0x7f04040c2310 con 0x7f0410102570 2024-09-19T09:16:31.856 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:16:31.857 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":35,"btime":"2024-09-19T09:16:31:434332+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:16:31.434317+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:16:31.857 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 35 2024-09-19T09:16:31.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.857+0000 7f0415245640 1 -- 172.21.15.120:0/886714333 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f03f80a3240 msgr2=0x7f03f80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:31.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.857+0000 7f0415245640 1 --2- 172.21.15.120:0/886714333 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f03f80a3240 0x7f03f80a5700 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f0400009bd0 tx=0x7f0400005f90 comp rx=0 tx=0).stop 2024-09-19T09:16:31.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.857+0000 7f0415245640 1 -- 172.21.15.120:0/886714333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0410102570 msgr2=0x7f0410196860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:31.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.857+0000 7f0415245640 1 --2- 172.21.15.120:0/886714333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0410102570 0x7f0410196860 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0404000c00 tx=0x7f040400d110 comp rx=0 tx=0).stop 2024-09-19T09:16:31.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.857+0000 7f0415245640 1 -- 172.21.15.120:0/886714333 shutdown_connections 2024-09-19T09:16:31.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.858+0000 7f0415245640 1 --2- 172.21.15.120:0/886714333 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f03f80a3240 0x7f03f80a5700 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:31.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.858+0000 7f0415245640 1 --2- 172.21.15.120:0/886714333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0410102ef0 0x7f041019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:31.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.858+0000 7f0415245640 1 --2- 172.21.15.120:0/886714333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0410102570 0x7f0410196860 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:31.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.858+0000 7f0415245640 1 --2- 172.21.15.120:0/886714333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0410100e20 0x7f0410196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:31.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.858+0000 7f0415245640 1 -- 172.21.15.120:0/886714333 >> 172.21.15.120:0/886714333 conn(0x7f04100fc7d0 msgr2=0x7f04100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:31.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.858+0000 7f0415245640 1 -- 172.21.15.120:0/886714333 shutdown_connections 2024-09-19T09:16:31.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:31.858+0000 7f0415245640 1 -- 172.21.15.120:0/886714333 wait complete. 2024-09-19T09:16:31.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:31 smithi196 ceph-mon[35150]: daemon mds.e restarted 2024-09-19T09:16:32.146 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1e2662fa', total seconds elapsed: 27.7 2024-09-19T09:16:32.352 INFO:tasks.workunit.client.0.smithi120.stdout: 10 1462 9.65 MB/sec warmup 27 sec latency 41.194 ms 2024-09-19T09:16:32.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:32 smithi120 ceph-mon[31150]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T09:16:32.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:32 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.153:6834/3253383200,v1:172.21.15.153:6835/3253383200] up:boot 2024-09-19T09:16:32.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:32 smithi120 ceph-mon[31150]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-19T09:16:32.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:32 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:16:32.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:32 smithi120 ceph-mon[31150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:16:32.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:32 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/886714333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:16:32.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:32 smithi120 ceph-mon[31150]: pgmap v755: 97 pgs: 97 active+clean; 77 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 6.5 MiB/s wr, 53 op/s 2024-09-19T09:16:32.905 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:16:32.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:32 smithi196 ceph-mon[35150]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T09:16:32.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:32 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.153:6834/3253383200,v1:172.21.15.153:6835/3253383200] up:boot 2024-09-19T09:16:32.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:32 smithi196 ceph-mon[35150]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-19T09:16:32.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:32 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:16:32.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:32 smithi196 ceph-mon[35150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:16:32.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:32 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/886714333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:16:32.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:32 smithi196 ceph-mon[35150]: pgmap v755: 97 pgs: 97 active+clean; 77 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 6.5 MiB/s wr, 53 op/s 2024-09-19T09:16:32.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:32 smithi153 ceph-mon[35071]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T09:16:32.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:32 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.153:6834/3253383200,v1:172.21.15.153:6835/3253383200] up:boot 2024-09-19T09:16:32.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:32 smithi153 ceph-mon[35071]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-19T09:16:32.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:32 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:16:32.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:32 smithi153 ceph-mon[35071]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:16:32.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:32 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/886714333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:16:32.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:32 smithi153 ceph-mon[35071]: pgmap v755: 97 pgs: 97 active+clean; 77 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 6.5 MiB/s wr, 53 op/s 2024-09-19T09:16:33.180 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:16:33.352 INFO:tasks.workunit.client.0.smithi120.stdout: 10 2359 9.94 MB/sec warmup 28 sec latency 55.411 ms 2024-09-19T09:16:33.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.685+0000 7f819f9df640 1 -- 172.21.15.120:0/3813637556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8198108290 msgr2=0x7f819810ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:33.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.685+0000 7f819f9df640 1 --2- 172.21.15.120:0/3813637556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8198108290 0x7f819810ec30 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f8190009cd0 tx=0x7f819002a8e0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.685+0000 7f819f9df640 1 -- 172.21.15.120:0/3813637556 shutdown_connections 2024-09-19T09:16:33.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.685+0000 7f819f9df640 1 --2- 172.21.15.120:0/3813637556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8198108290 0x7f819810ec30 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.685+0000 7f819f9df640 1 --2- 172.21.15.120:0/3813637556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8198103740 0x7f8198107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.685+0000 7f819f9df640 1 --2- 172.21.15.120:0/3813637556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8198102d90 0x7f8198103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.685+0000 7f819f9df640 1 -- 172.21.15.120:0/3813637556 >> 172.21.15.120:0/3813637556 conn(0x7f81980fc7d0 msgr2=0x7f81980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:33.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.685+0000 7f819f9df640 1 -- 172.21.15.120:0/3813637556 shutdown_connections 2024-09-19T09:16:33.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.685+0000 7f819f9df640 1 -- 172.21.15.120:0/3813637556 wait complete. 2024-09-19T09:16:33.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.686+0000 7f819f9df640 1 Processor -- start 2024-09-19T09:16:33.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.686+0000 7f819f9df640 1 -- start start 2024-09-19T09:16:33.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.686+0000 7f819f9df640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8198102d90 0x7f81981009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:33.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.686+0000 7f819f9df640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8198103740 0x7f8198100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:33.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.686+0000 7f819f9df640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8198108290 0x7f8198101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:33.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.686+0000 7f819f9df640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8198112090 con 0x7f8198103740 2024-09-19T09:16:33.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.686+0000 7f819f9df640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8198111f10 con 0x7f8198108290 2024-09-19T09:16:33.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.686+0000 7f819f9df640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8198112210 con 0x7f8198102d90 2024-09-19T09:16:33.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.686+0000 7f819cf7a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8198103740 0x7f8198100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:33.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.686+0000 7f819df7c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8198108290 0x7f8198101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:33.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.686+0000 7f819cf7a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8198103740 0x7f8198100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52214/0 (socket says 172.21.15.120:52214) 2024-09-19T09:16:33.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.686+0000 7f819cf7a640 1 -- 172.21.15.120:0/2558976941 learned_addr learned my addr 172.21.15.120:0/2558976941 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:33.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.686+0000 7f819d77b640 1 --2- 172.21.15.120:0/2558976941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8198102d90 0x7f81981009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:33.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.687+0000 7f819cf7a640 1 -- 172.21.15.120:0/2558976941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8198102d90 msgr2=0x7f81981009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:33.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.687+0000 7f819cf7a640 1 --2- 172.21.15.120:0/2558976941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8198102d90 0x7f81981009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.687+0000 7f819cf7a640 1 -- 172.21.15.120:0/2558976941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8198108290 msgr2=0x7f8198101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:33.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.687+0000 7f819cf7a640 1 --2- 172.21.15.120:0/2558976941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8198108290 0x7f8198101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.687+0000 7f819cf7a640 1 -- 172.21.15.120:0/2558976941 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81981a6a10 con 0x7f8198103740 2024-09-19T09:16:33.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.687+0000 7f819df7c640 1 --2- 172.21.15.120:0/2558976941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8198108290 0x7f8198101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:16:33.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.687+0000 7f819cf7a640 1 --2- 172.21.15.120:0/2558976941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8198103740 0x7f8198100f30 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f818800b2c0 tx=0x7f818800b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:33.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.687+0000 7f818e7fc640 1 -- 172.21.15.120:0/2558976941 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188005020 con 0x7f8198103740 2024-09-19T09:16:33.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.687+0000 7f819f9df640 1 -- 172.21.15.120:0/2558976941 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8198198310 con 0x7f8198103740 2024-09-19T09:16:33.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.687+0000 7f818e7fc640 1 -- 172.21.15.120:0/2558976941 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81880056a0 con 0x7f8198103740 2024-09-19T09:16:33.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.687+0000 7f818e7fc640 1 -- 172.21.15.120:0/2558976941 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818801b8e0 con 0x7f8198103740 2024-09-19T09:16:33.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.687+0000 7f819f9df640 1 -- 172.21.15.120:0/2558976941 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8198198850 con 0x7f8198103740 2024-09-19T09:16:33.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.690+0000 7f818e7fc640 1 -- 172.21.15.120:0/2558976941 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f81880051c0 con 0x7f8198103740 2024-09-19T09:16:33.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.690+0000 7f81a1db8740 1 Processor -- start 2024-09-19T09:16:33.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.690+0000 7f81a1db8740 1 -- start start 2024-09-19T09:16:33.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.690+0000 7f81a1db8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234469ab40 0x562344699090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:33.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.690+0000 7f81a1db8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623446995d0 0x562344699a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:33.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.690+0000 7f81a1db8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623446a4370 0x5623446a6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:33.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.690+0000 7f81a1db8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5623445ff2f0 con 0x56234469ab40 2024-09-19T09:16:33.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.690+0000 7f81a1db8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5623445a6cf0 con 0x5623446a4370 2024-09-19T09:16:33.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.690+0000 7f81a1db8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5623445fb2b0 con 0x5623446995d0 2024-09-19T09:16:33.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.691+0000 7f819df7c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234469ab40 0x562344699090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:33.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.691+0000 7f819df7c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234469ab40 0x562344699090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52224/0 (socket says 172.21.15.120:52224) 2024-09-19T09:16:33.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.691+0000 7f819df7c640 1 -- 172.21.15.120:0/301939905 learned_addr learned my addr 172.21.15.120:0/301939905 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:33.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.691+0000 7f819d77b640 1 --2- 172.21.15.120:0/301939905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623446995d0 0x562344699a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:33.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.691+0000 7f819cf7a640 1 --2- 172.21.15.120:0/301939905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623446a4370 0x5623446a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:33.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.691+0000 7f818e7fc640 1 --2- 172.21.15.120:0/2558976941 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f81840a3240 0x7f81840a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:33.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.691+0000 7f819df7c640 1 -- 172.21.15.120:0/301939905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623446995d0 msgr2=0x562344699a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:33.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.691+0000 7f819df7c640 1 --2- 172.21.15.120:0/301939905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623446995d0 0x562344699a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.691+0000 7f819df7c640 1 -- 172.21.15.120:0/301939905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623446a4370 msgr2=0x5623446a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:33.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.691+0000 7f819df7c640 1 --2- 172.21.15.120:0/301939905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623446a4370 0x5623446a6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.691+0000 7f819df7c640 1 -- 172.21.15.120:0/301939905 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56234458b3a0 con 0x56234469ab40 2024-09-19T09:16:33.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.691+0000 7f818e7fc640 1 -- 172.21.15.120:0/2558976941 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f81880d76d0 con 0x7f8198103740 2024-09-19T09:16:33.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.691+0000 7f819df7c640 1 --2- 172.21.15.120:0/2558976941 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f81840a3240 0x7f81840a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:33.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.691+0000 7f819df7c640 1 --2- 172.21.15.120:0/301939905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234469ab40 0x562344699090 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f8190035000 tx=0x7f81900318c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f5d81426a51725 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:33.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.691+0000 7f81837fe640 1 -- 172.21.15.120:0/301939905 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8190041040 con 0x56234469ab40 2024-09-19T09:16:33.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.691+0000 7f819df7c640 1 --2- 172.21.15.120:0/2558976941 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f81840a3240 0x7f81840a5700 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f819004c000 tx=0x7f819004d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:33.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.691+0000 7f81837fe640 1 -- 172.21.15.120:0/301939905 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8190043070 con 0x56234469ab40 2024-09-19T09:16:33.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.692+0000 7f81837fe640 1 -- 172.21.15.120:0/301939905 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8190047630 con 0x56234469ab40 2024-09-19T09:16:33.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.692+0000 7f81a1db8740 1 -- 172.21.15.120:0/301939905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234469ab40 msgr2=0x562344699090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:33.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.692+0000 7f81a1db8740 1 --2- 172.21.15.120:0/301939905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234469ab40 0x562344699090 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f8190035000 tx=0x7f81900318c0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.692+0000 7f81a1db8740 1 -- 172.21.15.120:0/301939905 shutdown_connections 2024-09-19T09:16:33.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.692+0000 7f81a1db8740 1 --2- 172.21.15.120:0/301939905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623446a4370 0x5623446a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.692+0000 7f81a1db8740 1 --2- 172.21.15.120:0/301939905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623446995d0 0x562344699a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.692+0000 7f81a1db8740 1 --2- 172.21.15.120:0/301939905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234469ab40 0x562344699090 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.692+0000 7f81a1db8740 1 -- 172.21.15.120:0/301939905 >> 172.21.15.120:0/301939905 conn(0x562344662310 msgr2=0x562344695c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:33.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.692+0000 7f81a1db8740 1 -- 172.21.15.120:0/301939905 shutdown_connections 2024-09-19T09:16:33.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.692+0000 7f81a1db8740 1 -- 172.21.15.120:0/301939905 wait complete. 2024-09-19T09:16:33.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.693+0000 7f81a1db8740 1 -- start start 2024-09-19T09:16:33.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.693+0000 7f81a1db8740 1 Processor -- start 2024-09-19T09:16:33.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.693+0000 7f81a1db8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623446995d0 0x5623446a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:33.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.693+0000 7f81a1db8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234469ab40 0x5623446a1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:33.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.693+0000 7f81a1db8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623446a4370 0x5623446a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:33.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.693+0000 7f81a1db8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5623446a7070 con 0x5623446a4370 2024-09-19T09:16:33.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.693+0000 7f81a1db8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5623445ba060 con 0x56234469ab40 2024-09-19T09:16:33.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.693+0000 7f81a1db8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5623446a71f0 con 0x5623446995d0 2024-09-19T09:16:33.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.693+0000 7f81a1db8740 10 client.0 _ll_register_callbacks cb 0x56234430d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:33.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.693+0000 7f819cf7a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234469ab40 0x5623446a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:33.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.693+0000 7f819cf7a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234469ab40 0x5623446a1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:50654/0 (socket says 172.21.15.120:50654) 2024-09-19T09:16:33.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.693+0000 7f819d77b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623446995d0 0x5623446a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:33.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.693+0000 7f819cf7a640 1 -- 172.21.15.120:0/2959648086 learned_addr learned my addr 172.21.15.120:0/2959648086 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:33.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.693+0000 7f819df7c640 1 --2- 172.21.15.120:0/2959648086 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623446a4370 0x5623446a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:33.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.694+0000 7f819cf7a640 1 -- 172.21.15.120:0/2959648086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623446995d0 msgr2=0x5623446a3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:33.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.694+0000 7f819cf7a640 1 --2- 172.21.15.120:0/2959648086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623446995d0 0x5623446a3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.694+0000 7f819cf7a640 1 -- 172.21.15.120:0/2959648086 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623446a4370 msgr2=0x5623446a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:33.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.694+0000 7f819cf7a640 1 --2- 172.21.15.120:0/2959648086 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623446a4370 0x5623446a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.694+0000 7f819cf7a640 1 -- 172.21.15.120:0/2959648086 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623443703c0 con 0x56234469ab40 2024-09-19T09:16:33.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.694+0000 7f819cf7a640 1 --2- 172.21.15.120:0/2959648086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234469ab40 0x5623446a1050 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f81880a0200 tx=0x7f81880a3c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c75ecd7b2496db2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:33.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.694+0000 7f8180ff9640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:16:33.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.694+0000 7f8180ff9640 1 -- 172.21.15.120:0/2959648086 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81880a80a0 con 0x56234469ab40 2024-09-19T09:16:33.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.694+0000 7f81a1db8740 1 -- 172.21.15.120:0/2959648086 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8190043070 con 0x56234469ab40 2024-09-19T09:16:33.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.694+0000 7f8180ff9640 1 -- 172.21.15.120:0/2959648086 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f81880046c0 con 0x56234469ab40 2024-09-19T09:16:33.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.694+0000 7f8180ff9640 1 -- 172.21.15.120:0/2959648086 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f81880ac960 con 0x56234469ab40 2024-09-19T09:16:33.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.694+0000 7f81a1db8740 10 client.29138 fetch_fsmap learned FSMap version 36 2024-09-19T09:16:33.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.694+0000 7f81a1db8740 1 -- 172.21.15.120:0/2959648086 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5623446b2c00 con 0x56234469ab40 2024-09-19T09:16:33.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.694+0000 7f8180ff9640 1 -- 172.21.15.120:0/2959648086 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8190043070 con 0x56234469ab40 2024-09-19T09:16:33.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.694+0000 7f8180ff9640 1 -- 172.21.15.120:0/2959648086 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f81880bb030 con 0x56234469ab40 2024-09-19T09:16:33.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.695+0000 7f81a1db8740 10 client.29138 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:16:33.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.695+0000 7f81a1db8740 10 client.29138 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:16:33.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.695+0000 7f81a1db8740 20 client.29138 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:33.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.695+0000 7f81a1db8740 1 --2- 172.21.15.120:0/2959648086 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5623446b40c0 0x5623446b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:33.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.695+0000 7f81a1db8740 4 client.29138 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:16:33.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.695+0000 7f81a1db8740 1 -- 172.21.15.120:0/2959648086 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56234460c860 con 0x5623446b40c0 2024-09-19T09:16:33.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.695+0000 7f819d77b640 1 --2- 172.21.15.120:0/2959648086 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5623446b40c0 0x5623446b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:33.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.695+0000 7f8180ff9640 10 client.29138 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:16:33.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.695+0000 7f819d77b640 1 --2- 172.21.15.120:0/2959648086 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5623446b40c0 0x5623446b64b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c1109f6250608eda server_cookie=836f5072f45a6c9b in_seq=0 out_seq=0 2024-09-19T09:16:33.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.696+0000 7f8180ff9640 1 -- 172.21.15.120:0/2959648086 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56234460c860 con 0x5623446b40c0 2024-09-19T09:16:33.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.696+0000 7f8180ff9640 10 client.29138 handle_command_reply: tid=0 2024-09-19T09:16:33.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.696+0000 7f81a1db8740 1 client.29138 shutdown 2024-09-19T09:16:33.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.696+0000 7f81a1db8740 1 -- 172.21.15.120:0/2959648086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234469ab40 msgr2=0x5623446a1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:33.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.696+0000 7f81a1db8740 1 --2- 172.21.15.120:0/2959648086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234469ab40 0x5623446a1050 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f81880a0200 tx=0x7f81880a3c80 comp rx=0 tx=0).stop 2024-09-19T09:16:33.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.696+0000 7f81a1db8740 1 -- 172.21.15.120:0/2959648086 shutdown_connections 2024-09-19T09:16:33.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.696+0000 7f81a1db8740 1 --2- 172.21.15.120:0/2959648086 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5623446b40c0 0x5623446b64b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.696+0000 7f81a1db8740 1 --2- 172.21.15.120:0/2959648086 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623446a4370 0x5623446a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.696+0000 7f81a1db8740 1 --2- 172.21.15.120:0/2959648086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234469ab40 0x5623446a1050 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.696+0000 7f81a1db8740 1 --2- 172.21.15.120:0/2959648086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623446995d0 0x5623446a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.696+0000 7f81a1db8740 1 -- 172.21.15.120:0/2959648086 >> 172.21.15.120:0/2959648086 conn(0x562344662310 msgr2=0x56234469b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:33.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.696+0000 7f8180ff9640 0 client.29138 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:16:33.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.696+0000 7f81a1db8740 1 -- 172.21.15.120:0/2959648086 shutdown_connections 2024-09-19T09:16:33.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.696+0000 7f81a1db8740 1 -- 172.21.15.120:0/2959648086 wait complete. 2024-09-19T09:16:33.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.696+0000 7f81a1db8740 20 client.29138 trim_cache size 0 max 16384 2024-09-19T09:16:33.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.710+0000 7f81a1db8740 1 Processor -- start 2024-09-19T09:16:33.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.710+0000 7f81a1db8740 1 -- start start 2024-09-19T09:16:33.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.710+0000 7f81a1db8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623446995d0 0x562344695e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:33.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.710+0000 7f81a1db8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234469ab40 0x5623446bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:33.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.710+0000 7f81a1db8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623446a4370 0x5623446bc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:33.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.710+0000 7f81a1db8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f81981ab290 con 0x5623446995d0 2024-09-19T09:16:33.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.710+0000 7f81a1db8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f81981ace60 con 0x56234469ab40 2024-09-19T09:16:33.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.710+0000 7f81a1db8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f81981ab960 con 0x5623446a4370 2024-09-19T09:16:33.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.711+0000 7f819d77b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623446995d0 0x562344695e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:33.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.711+0000 7f819d77b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623446995d0 0x562344695e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52238/0 (socket says 172.21.15.120:52238) 2024-09-19T09:16:33.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.711+0000 7f819d77b640 1 -- 172.21.15.120:0/2302771362 learned_addr learned my addr 172.21.15.120:0/2302771362 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:33.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.711+0000 7f819cf7a640 1 --2- 172.21.15.120:0/2302771362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234469ab40 0x5623446bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:33.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.711+0000 7f819df7c640 1 --2- 172.21.15.120:0/2302771362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623446a4370 0x5623446bc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:33.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.711+0000 7f819d77b640 1 -- 172.21.15.120:0/2302771362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623446a4370 msgr2=0x5623446bc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:33.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.711+0000 7f819d77b640 1 --2- 172.21.15.120:0/2302771362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623446a4370 0x5623446bc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.711+0000 7f819d77b640 1 -- 172.21.15.120:0/2302771362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234469ab40 msgr2=0x5623446bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:33.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.711+0000 7f819d77b640 1 --2- 172.21.15.120:0/2302771362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234469ab40 0x5623446bbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.711+0000 7f819d77b640 1 -- 172.21.15.120:0/2302771362 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56234469c120 con 0x5623446995d0 2024-09-19T09:16:33.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.711+0000 7f819d77b640 1 --2- 172.21.15.120:0/2302771362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623446995d0 0x562344695e10 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x5623446b6c50 tx=0x7f8194005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76a38153f6ab494 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:33.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.711+0000 7f8182ffd640 1 -- 172.21.15.120:0/2302771362 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8194022030 con 0x5623446995d0 2024-09-19T09:16:33.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.711+0000 7f8182ffd640 1 -- 172.21.15.120:0/2302771362 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8194005500 con 0x5623446995d0 2024-09-19T09:16:33.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.711+0000 7f8182ffd640 1 -- 172.21.15.120:0/2302771362 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819401e680 con 0x5623446995d0 2024-09-19T09:16:33.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.712+0000 7f81a1db8740 1 -- 172.21.15.120:0/2302771362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623446995d0 msgr2=0x562344695e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:33.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.712+0000 7f81a1db8740 1 --2- 172.21.15.120:0/2302771362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623446995d0 0x562344695e10 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x5623446b6c50 tx=0x7f8194005f90 comp rx=0 tx=0).stop 2024-09-19T09:16:33.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.712+0000 7f81a1db8740 1 -- 172.21.15.120:0/2302771362 shutdown_connections 2024-09-19T09:16:33.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.712+0000 7f81a1db8740 1 --2- 172.21.15.120:0/2302771362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623446a4370 0x5623446bc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.712+0000 7f81a1db8740 1 --2- 172.21.15.120:0/2302771362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234469ab40 0x5623446bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.712+0000 7f81a1db8740 1 --2- 172.21.15.120:0/2302771362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623446995d0 0x562344695e10 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.712+0000 7f81a1db8740 1 -- 172.21.15.120:0/2302771362 >> 172.21.15.120:0/2302771362 conn(0x562344662310 msgr2=0x562344695700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:33.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.712+0000 7f81a1db8740 1 -- 172.21.15.120:0/2302771362 shutdown_connections 2024-09-19T09:16:33.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.712+0000 7f81a1db8740 1 -- 172.21.15.120:0/2302771362 wait complete. 2024-09-19T09:16:33.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.712+0000 7f81a1db8740 1 -- start start 2024-09-19T09:16:33.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.713+0000 7f81a1db8740 1 Processor -- start 2024-09-19T09:16:33.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.713+0000 7f81a1db8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623446995d0 0x5623446a85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:33.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.713+0000 7f81a1db8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234469ab40 0x5623446a8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:33.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.713+0000 7f81a1db8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623446a4370 0x56234471dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:33.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.713+0000 7f81a1db8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f81981ac790 con 0x56234469ab40 2024-09-19T09:16:33.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.713+0000 7f81a1db8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f81981ac030 con 0x5623446995d0 2024-09-19T09:16:33.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.713+0000 7f81a1db8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f81981a9250 con 0x5623446a4370 2024-09-19T09:16:33.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.713+0000 7f81a1db8740 10 client.0 _ll_register_callbacks cb 0x7f81880dd220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:33.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.713+0000 7f819cf7a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234469ab40 0x5623446a8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:33.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.713+0000 7f819d77b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623446995d0 0x5623446a85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:33.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.713+0000 7f819cf7a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234469ab40 0x5623446a8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52252/0 (socket says 172.21.15.120:52252) 2024-09-19T09:16:33.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.713+0000 7f819d77b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623446995d0 0x5623446a85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:50670/0 (socket says 172.21.15.120:50670) 2024-09-19T09:16:33.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.713+0000 7f819d77b640 1 -- 172.21.15.120:0/889691422 learned_addr learned my addr 172.21.15.120:0/889691422 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:33.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.713+0000 7f819df7c640 1 --2- 172.21.15.120:0/889691422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623446a4370 0x56234471dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:33.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.713+0000 7f819cf7a640 1 -- 172.21.15.120:0/889691422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623446a4370 msgr2=0x56234471dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:33.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.714+0000 7f819cf7a640 1 --2- 172.21.15.120:0/889691422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623446a4370 0x56234471dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.714+0000 7f819cf7a640 1 -- 172.21.15.120:0/889691422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623446995d0 msgr2=0x5623446a85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:33.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.714+0000 7f819cf7a640 1 --2- 172.21.15.120:0/889691422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623446995d0 0x5623446a85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.714+0000 7f819cf7a640 1 -- 172.21.15.120:0/889691422 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56234471e1f0 con 0x56234469ab40 2024-09-19T09:16:33.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.714+0000 7f819d77b640 1 --2- 172.21.15.120:0/889691422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623446995d0 0x5623446a85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:16:33.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.714+0000 7f819cf7a640 1 --2- 172.21.15.120:0/889691422 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234469ab40 0x5623446a8ae0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f81880a0770 tx=0x7f81880d5250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fbd3ba7a23aa92f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:33.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.714+0000 7f8180ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:16:33.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.714+0000 7f8180ff9640 1 -- 172.21.15.120:0/889691422 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188087060 con 0x56234469ab40 2024-09-19T09:16:33.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.714+0000 7f8180ff9640 1 -- 172.21.15.120:0/889691422 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f81880a3440 con 0x56234469ab40 2024-09-19T09:16:33.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.714+0000 7f81a1db8740 1 -- 172.21.15.120:0/889691422 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8194005500 con 0x56234469ab40 2024-09-19T09:16:33.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.714+0000 7f8180ff9640 1 -- 172.21.15.120:0/889691422 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f81880a3700 con 0x56234469ab40 2024-09-19T09:16:33.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.714+0000 7f81a1db8740 10 client.20190 fetch_fsmap learned FSMap version 36 2024-09-19T09:16:33.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.714+0000 7f81a1db8740 1 -- 172.21.15.120:0/889691422 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56234471f350 con 0x56234469ab40 2024-09-19T09:16:33.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.714+0000 7f8180ff9640 1 -- 172.21.15.120:0/889691422 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8194005500 con 0x56234469ab40 2024-09-19T09:16:33.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.714+0000 7f8180ff9640 1 -- 172.21.15.120:0/889691422 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f81880bb040 con 0x56234469ab40 2024-09-19T09:16:33.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.714+0000 7f81a1db8740 10 client.20190 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:16:33.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.714+0000 7f81a1db8740 10 client.20190 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:16:33.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.714+0000 7f81a1db8740 20 client.20190 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:33.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.715+0000 7f81a1db8740 1 --2- 172.21.15.120:0/889691422 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x56234471fad0 0x562344721ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:33.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.715+0000 7f81a1db8740 4 client.20190 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:16:33.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.715+0000 7f81a1db8740 1 -- 172.21.15.120:0/889691422 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562344606d70 con 0x56234471fad0 2024-09-19T09:16:33.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.715+0000 7f819d77b640 1 --2- 172.21.15.120:0/889691422 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x56234471fad0 0x562344721ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:33.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.715+0000 7f819d77b640 1 --2- 172.21.15.120:0/889691422 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x56234471fad0 0x562344721ef0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cb2a8c90803df428 server_cookie=cb706966d4718df1 in_seq=0 out_seq=0 2024-09-19T09:16:33.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.715+0000 7f8180ff9640 10 client.20190 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:16:33.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.715+0000 7f8180ff9640 1 -- 172.21.15.120:0/889691422 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562344606d70 con 0x56234471fad0 2024-09-19T09:16:33.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.715+0000 7f8180ff9640 10 client.20190 handle_command_reply: tid=0 2024-09-19T09:16:33.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.715+0000 7f81a1db8740 1 client.20190 shutdown 2024-09-19T09:16:33.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.716+0000 7f81a1db8740 1 -- 172.21.15.120:0/889691422 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234469ab40 msgr2=0x5623446a8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:33.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.716+0000 7f81a1db8740 1 --2- 172.21.15.120:0/889691422 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234469ab40 0x5623446a8ae0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f81880a0770 tx=0x7f81880d5250 comp rx=0 tx=0).stop 2024-09-19T09:16:33.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.716+0000 7f81a1db8740 1 -- 172.21.15.120:0/889691422 shutdown_connections 2024-09-19T09:16:33.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.716+0000 7f81a1db8740 1 --2- 172.21.15.120:0/889691422 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x56234471fad0 0x562344721ef0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.716+0000 7f81a1db8740 1 --2- 172.21.15.120:0/889691422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623446a4370 0x56234471dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.716+0000 7f81a1db8740 1 --2- 172.21.15.120:0/889691422 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234469ab40 0x5623446a8ae0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.716+0000 7f81a1db8740 1 --2- 172.21.15.120:0/889691422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623446995d0 0x5623446a85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.716+0000 7f81a1db8740 1 -- 172.21.15.120:0/889691422 >> 172.21.15.120:0/889691422 conn(0x562344662310 msgr2=0x5623446a28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:33.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.716+0000 7f8180ff9640 0 client.20190 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:16:33.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.716+0000 7f81a1db8740 1 -- 172.21.15.120:0/889691422 shutdown_connections 2024-09-19T09:16:33.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.716+0000 7f81a1db8740 1 -- 172.21.15.120:0/889691422 wait complete. 2024-09-19T09:16:33.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.716+0000 7f81a1db8740 20 client.20190 trim_cache size 0 max 16384 2024-09-19T09:16:33.731 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:16:33.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.717+0000 7f819f9df640 1 -- 172.21.15.120:0/2558976941 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f81840a3240 msgr2=0x7f81840a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:33.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.717+0000 7f819f9df640 1 --2- 172.21.15.120:0/2558976941 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f81840a3240 0x7f81840a5700 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f819004c000 tx=0x7f819004d000 comp rx=0 tx=0).stop 2024-09-19T09:16:33.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.717+0000 7f819f9df640 1 -- 172.21.15.120:0/2558976941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8198103740 msgr2=0x7f8198100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:33.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.717+0000 7f819f9df640 1 --2- 172.21.15.120:0/2558976941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8198103740 0x7f8198100f30 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f818800b2c0 tx=0x7f818800b780 comp rx=0 tx=0).stop 2024-09-19T09:16:33.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.717+0000 7f819f9df640 1 -- 172.21.15.120:0/2558976941 shutdown_connections 2024-09-19T09:16:33.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.717+0000 7f819f9df640 1 --2- 172.21.15.120:0/2558976941 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f81840a3240 0x7f81840a5700 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.717+0000 7f819f9df640 1 --2- 172.21.15.120:0/2558976941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8198108290 0x7f8198101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.717+0000 7f819f9df640 1 --2- 172.21.15.120:0/2558976941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8198103740 0x7f8198100f30 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.717+0000 7f819f9df640 1 --2- 172.21.15.120:0/2558976941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8198102d90 0x7f81981009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:33.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.717+0000 7f819f9df640 1 -- 172.21.15.120:0/2558976941 >> 172.21.15.120:0/2558976941 conn(0x7f81980fc7d0 msgr2=0x7f8198106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:33.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.717+0000 7f819f9df640 1 -- 172.21.15.120:0/2558976941 shutdown_connections 2024-09-19T09:16:33.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:33.717+0000 7f819f9df640 1 -- 172.21.15.120:0/2558976941 wait complete. 2024-09-19T09:16:34.352 INFO:tasks.workunit.client.0.smithi120.stdout: 10 3101 11.73 MB/sec warmup 29 sec latency 83.308 ms 2024-09-19T09:16:34.573 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:16:34.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:34 smithi120 ceph-mon[31150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:16:34.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:34 smithi120 ceph-mon[31150]: pgmap v756: 97 pgs: 97 active+clean; 150 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 15 MiB/s wr, 105 op/s 2024-09-19T09:16:34.848 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:16:34.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:34 smithi196 ceph-mon[35150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:16:34.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:34 smithi196 ceph-mon[35150]: pgmap v756: 97 pgs: 97 active+clean; 150 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 15 MiB/s wr, 105 op/s 2024-09-19T09:16:34.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:34 smithi153 ceph-mon[35071]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:16:34.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:34 smithi153 ceph-mon[35071]: pgmap v756: 97 pgs: 97 active+clean; 150 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 15 MiB/s wr, 105 op/s 2024-09-19T09:16:35.352 INFO:tasks.workunit.client.0.smithi120.stdout: 10 4017 13.56 MB/sec warmup 30 sec latency 64.733 ms 2024-09-19T09:16:35.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.393+0000 7fe84033b640 1 -- 172.21.15.120:0/921103777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe838102570 msgr2=0x7fe8381029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:35.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.393+0000 7fe84033b640 1 --2- 172.21.15.120:0/921103777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe838102570 0x7fe8381029b0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fe828009900 tx=0x7fe82802a480 comp rx=0 tx=0).stop 2024-09-19T09:16:35.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.393+0000 7fe84033b640 1 -- 172.21.15.120:0/921103777 shutdown_connections 2024-09-19T09:16:35.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.393+0000 7fe84033b640 1 --2- 172.21.15.120:0/921103777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe838102ef0 0x7fe83810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.393+0000 7fe84033b640 1 --2- 172.21.15.120:0/921103777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe838102570 0x7fe8381029b0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.393+0000 7fe84033b640 1 --2- 172.21.15.120:0/921103777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe838100e20 0x7fe838101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.393+0000 7fe84033b640 1 -- 172.21.15.120:0/921103777 >> 172.21.15.120:0/921103777 conn(0x7fe8380fc7d0 msgr2=0x7fe8380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:35.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.393+0000 7fe84033b640 1 -- 172.21.15.120:0/921103777 shutdown_connections 2024-09-19T09:16:35.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.393+0000 7fe84033b640 1 -- 172.21.15.120:0/921103777 wait complete. 2024-09-19T09:16:35.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.394+0000 7fe84033b640 1 Processor -- start 2024-09-19T09:16:35.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.394+0000 7fe84033b640 1 -- start start 2024-09-19T09:16:35.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.394+0000 7fe84033b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe838100e20 0x7fe838198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:35.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.394+0000 7fe84033b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe838102570 0x7fe838198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:35.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.394+0000 7fe84033b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe838102ef0 0x7fe83819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:35.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.394+0000 7fe84033b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe838110f80 con 0x7fe838102ef0 2024-09-19T09:16:35.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.394+0000 7fe84033b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe838110e00 con 0x7fe838100e20 2024-09-19T09:16:35.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.394+0000 7fe84033b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe838111100 con 0x7fe838102570 2024-09-19T09:16:35.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.394+0000 7fe83e8d8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe838102ef0 0x7fe83819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:35.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.394+0000 7fe83e0d7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe838100e20 0x7fe838198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:35.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.394+0000 7fe83e0d7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe838100e20 0x7fe838198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:50688/0 (socket says 172.21.15.120:50688) 2024-09-19T09:16:35.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.394+0000 7fe83e0d7640 1 -- 172.21.15.120:0/3568076525 learned_addr learned my addr 172.21.15.120:0/3568076525 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:35.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.394+0000 7fe83d8d6640 1 --2- 172.21.15.120:0/3568076525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe838102570 0x7fe838198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:35.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.394+0000 7fe83e8d8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe838102ef0 0x7fe83819ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52282/0 (socket says 172.21.15.120:52282) 2024-09-19T09:16:35.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.395+0000 7fe83e0d7640 1 -- 172.21.15.120:0/3568076525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe838102570 msgr2=0x7fe838198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:35.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.395+0000 7fe83e0d7640 1 --2- 172.21.15.120:0/3568076525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe838102570 0x7fe838198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.395+0000 7fe83e0d7640 1 -- 172.21.15.120:0/3568076525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe838102ef0 msgr2=0x7fe83819ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:35.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.395+0000 7fe83e0d7640 1 --2- 172.21.15.120:0/3568076525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe838102ef0 0x7fe83819ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.395+0000 7fe83e0d7640 1 -- 172.21.15.120:0/3568076525 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83819d540 con 0x7fe838100e20 2024-09-19T09:16:35.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.395+0000 7fe83d8d6640 1 --2- 172.21.15.120:0/3568076525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe838102570 0x7fe838198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:16:35.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.395+0000 7fe83e8d8640 1 --2- 172.21.15.120:0/3568076525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe838102ef0 0x7fe83819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:16:35.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.395+0000 7fe83e0d7640 1 --2- 172.21.15.120:0/3568076525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe838100e20 0x7fe838198590 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe83400d770 tx=0x7fe83400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:35.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.395+0000 7fe82f7fe640 1 -- 172.21.15.120:0/3568076525 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834017070 con 0x7fe838100e20 2024-09-19T09:16:35.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.395+0000 7fe84033b640 1 -- 172.21.15.120:0/3568076525 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe83819d830 con 0x7fe838100e20 2024-09-19T09:16:35.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.395+0000 7fe84033b640 1 -- 172.21.15.120:0/3568076525 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8381a5110 con 0x7fe838100e20 2024-09-19T09:16:35.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.395+0000 7fe82f7fe640 1 -- 172.21.15.120:0/3568076525 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe834005500 con 0x7fe838100e20 2024-09-19T09:16:35.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.395+0000 7fe82f7fe640 1 -- 172.21.15.120:0/3568076525 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834004e00 con 0x7fe838100e20 2024-09-19T09:16:35.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.398+0000 7fe842714740 1 Processor -- start 2024-09-19T09:16:35.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.398+0000 7fe842714740 1 -- start start 2024-09-19T09:16:35.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.398+0000 7fe842714740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca8dac8b40 0x55ca8dac7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:35.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.398+0000 7fe842714740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca8dac75d0 0x55ca8dac7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:35.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.398+0000 7fe842714740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca8dad2370 0x55ca8dad4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:35.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.398+0000 7fe842714740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ca8da2d2f0 con 0x55ca8dad2370 2024-09-19T09:16:35.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.398+0000 7fe842714740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ca8d9d4cf0 con 0x55ca8dac75d0 2024-09-19T09:16:35.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.398+0000 7fe842714740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ca8da292b0 con 0x55ca8dac8b40 2024-09-19T09:16:35.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.398+0000 7fe82f7fe640 1 -- 172.21.15.120:0/3568076525 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe834004fa0 con 0x7fe838100e20 2024-09-19T09:16:35.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.398+0000 7fe83d8d6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca8dad2370 0x55ca8dad4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:35.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.398+0000 7fe83e8d8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca8dac8b40 0x55ca8dac7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:35.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.398+0000 7fe83d8d6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca8dad2370 0x55ca8dad4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52288/0 (socket says 172.21.15.120:52288) 2024-09-19T09:16:35.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.398+0000 7fe83e8d8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca8dac8b40 0x55ca8dac7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:33492/0 (socket says 172.21.15.120:33492) 2024-09-19T09:16:35.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.398+0000 7fe83d8d6640 1 -- 172.21.15.120:0/4142779901 learned_addr learned my addr 172.21.15.120:0/4142779901 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:35.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.398+0000 7fe83e0d7640 1 --2- 172.21.15.120:0/4142779901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca8dac75d0 0x55ca8dac7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:35.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.399+0000 7fe83d8d6640 1 -- 172.21.15.120:0/4142779901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca8dac8b40 msgr2=0x55ca8dac7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:35.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.399+0000 7fe83d8d6640 1 --2- 172.21.15.120:0/4142779901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca8dac8b40 0x55ca8dac7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.399+0000 7fe83d8d6640 1 -- 172.21.15.120:0/4142779901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca8dac75d0 msgr2=0x55ca8dac7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:35.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.399+0000 7fe83d8d6640 1 --2- 172.21.15.120:0/4142779901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca8dac75d0 0x55ca8dac7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.399+0000 7fe83d8d6640 1 -- 172.21.15.120:0/4142779901 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca8d9b93a0 con 0x55ca8dad2370 2024-09-19T09:16:35.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.399+0000 7fe83e0d7640 1 --2- 172.21.15.120:0/4142779901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca8dac75d0 0x55ca8dac7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:16:35.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.399+0000 7fe83d8d6640 1 --2- 172.21.15.120:0/4142779901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca8dad2370 0x55ca8dad4760 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fe82802a9a0 tx=0x7fe828004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76a70db3c885f67 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:35.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.399+0000 7fe82f7fe640 1 --2- 172.21.15.120:0/3568076525 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe8240a3240 0x7fe8240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:35.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.399+0000 7fe82cdea640 1 -- 172.21.15.120:0/4142779901 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe828035030 con 0x55ca8dad2370 2024-09-19T09:16:35.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.399+0000 7fe82cdea640 1 -- 172.21.15.120:0/4142779901 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe828040070 con 0x55ca8dad2370 2024-09-19T09:16:35.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.399+0000 7fe82cdea640 1 -- 172.21.15.120:0/4142779901 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe828031d80 con 0x55ca8dad2370 2024-09-19T09:16:35.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.399+0000 7fe83e8d8640 1 --2- 172.21.15.120:0/3568076525 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe8240a3240 0x7fe8240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:35.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.399+0000 7fe82f7fe640 1 -- 172.21.15.120:0/3568076525 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe8340cf2f0 con 0x7fe838100e20 2024-09-19T09:16:35.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.400+0000 7fe842714740 1 -- 172.21.15.120:0/4142779901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca8dad2370 msgr2=0x55ca8dad4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:35.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.400+0000 7fe842714740 1 --2- 172.21.15.120:0/4142779901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca8dad2370 0x55ca8dad4760 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fe82802a9a0 tx=0x7fe828004470 comp rx=0 tx=0).stop 2024-09-19T09:16:35.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.400+0000 7fe83e8d8640 1 --2- 172.21.15.120:0/3568076525 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe8240a3240 0x7fe8240a5700 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe830005fd0 tx=0x7fe8300096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:35.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.400+0000 7fe842714740 1 -- 172.21.15.120:0/4142779901 shutdown_connections 2024-09-19T09:16:35.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.400+0000 7fe842714740 1 --2- 172.21.15.120:0/4142779901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca8dad2370 0x55ca8dad4760 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.400+0000 7fe842714740 1 --2- 172.21.15.120:0/4142779901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca8dac75d0 0x55ca8dac7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.400+0000 7fe842714740 1 --2- 172.21.15.120:0/4142779901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca8dac8b40 0x55ca8dac7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.400+0000 7fe842714740 1 -- 172.21.15.120:0/4142779901 >> 172.21.15.120:0/4142779901 conn(0x55ca8da90310 msgr2=0x55ca8dac3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:35.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.400+0000 7fe842714740 1 -- 172.21.15.120:0/4142779901 shutdown_connections 2024-09-19T09:16:35.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.400+0000 7fe842714740 1 -- 172.21.15.120:0/4142779901 wait complete. 2024-09-19T09:16:35.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.401+0000 7fe842714740 1 -- start start 2024-09-19T09:16:35.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.401+0000 7fe842714740 1 Processor -- start 2024-09-19T09:16:35.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.401+0000 7fe842714740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca8dac75d0 0x55ca8dad1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:35.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.401+0000 7fe842714740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca8dac8b40 0x55ca8dacef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:35.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.401+0000 7fe842714740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca8dad2370 0x55ca8dacf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:35.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.401+0000 7fe842714740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ca8dad5010 con 0x55ca8dad2370 2024-09-19T09:16:35.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.401+0000 7fe842714740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ca8d9e8060 con 0x55ca8dac8b40 2024-09-19T09:16:35.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.401+0000 7fe842714740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ca8dad5190 con 0x55ca8dac75d0 2024-09-19T09:16:35.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.401+0000 7fe842714740 10 client.0 _ll_register_callbacks cb 0x55ca8d73b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:35.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.401+0000 7fe83e8d8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca8dad2370 0x55ca8dacf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:35.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.401+0000 7fe83e0d7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca8dac75d0 0x55ca8dad1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:35.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.401+0000 7fe83e8d8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca8dad2370 0x55ca8dacf480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52304/0 (socket says 172.21.15.120:52304) 2024-09-19T09:16:35.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.401+0000 7fe83d8d6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca8dac8b40 0x55ca8dacef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:35.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.401+0000 7fe83e8d8640 1 -- 172.21.15.120:0/1789649058 learned_addr learned my addr 172.21.15.120:0/1789649058 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:35.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.401+0000 7fe83e8d8640 1 -- 172.21.15.120:0/1789649058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca8dac75d0 msgr2=0x55ca8dad1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:35.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.401+0000 7fe83e8d8640 1 --2- 172.21.15.120:0/1789649058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca8dac75d0 0x55ca8dad1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.401+0000 7fe83e8d8640 1 -- 172.21.15.120:0/1789649058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca8dac8b40 msgr2=0x55ca8dacef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:35.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.401+0000 7fe83e8d8640 1 --2- 172.21.15.120:0/1789649058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca8dac8b40 0x55ca8dacef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.401+0000 7fe83e8d8640 1 -- 172.21.15.120:0/1789649058 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca8d79e3c0 con 0x55ca8dad2370 2024-09-19T09:16:35.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.402+0000 7fe83e8d8640 1 --2- 172.21.15.120:0/1789649058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca8dad2370 0x55ca8dacf480 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fe830010b70 tx=0x7fe830004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6e8022cd86c936 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:35.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.402+0000 7fe821ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:16:35.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.402+0000 7fe821ffb640 1 -- 172.21.15.120:0/1789649058 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83002c060 con 0x55ca8dad2370 2024-09-19T09:16:35.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.402+0000 7fe821ffb640 1 -- 172.21.15.120:0/1789649058 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe830017030 con 0x55ca8dad2370 2024-09-19T09:16:35.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.402+0000 7fe842714740 1 -- 172.21.15.120:0/1789649058 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe828040070 con 0x55ca8dad2370 2024-09-19T09:16:35.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.402+0000 7fe821ffb640 1 -- 172.21.15.120:0/1789649058 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe83001fd40 con 0x55ca8dad2370 2024-09-19T09:16:35.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.402+0000 7fe842714740 10 client.20202 fetch_fsmap learned FSMap version 36 2024-09-19T09:16:35.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.402+0000 7fe842714740 1 -- 172.21.15.120:0/1789649058 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ca8dae0a30 con 0x55ca8dad2370 2024-09-19T09:16:35.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.402+0000 7fe821ffb640 1 -- 172.21.15.120:0/1789649058 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe828040070 con 0x55ca8dad2370 2024-09-19T09:16:35.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.402+0000 7fe821ffb640 1 -- 172.21.15.120:0/1789649058 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fe830031070 con 0x55ca8dad2370 2024-09-19T09:16:35.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.402+0000 7fe842714740 10 client.20202 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:16:35.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.402+0000 7fe842714740 10 client.20202 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:16:35.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.402+0000 7fe842714740 20 client.20202 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:35.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.402+0000 7fe842714740 1 --2- 172.21.15.120:0/1789649058 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55ca8dae1ef0 0x55ca8dae4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:35.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.402+0000 7fe842714740 4 client.20202 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:16:35.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.402+0000 7fe842714740 1 -- 172.21.15.120:0/1789649058 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca8da3a860 con 0x55ca8dae1ef0 2024-09-19T09:16:35.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.403+0000 7fe83e0d7640 1 --2- 172.21.15.120:0/1789649058 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55ca8dae1ef0 0x55ca8dae4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:35.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.403+0000 7fe83e0d7640 1 --2- 172.21.15.120:0/1789649058 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55ca8dae1ef0 0x55ca8dae4300 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cea6e4eda367199d server_cookie=d9605aa5244053ce in_seq=0 out_seq=0 2024-09-19T09:16:35.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.403+0000 7fe821ffb640 10 client.20202 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:16:35.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.404+0000 7fe821ffb640 1 -- 172.21.15.120:0/1789649058 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ca8da3a860 con 0x55ca8dae1ef0 2024-09-19T09:16:35.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.404+0000 7fe821ffb640 10 client.20202 handle_command_reply: tid=0 2024-09-19T09:16:35.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.404+0000 7fe842714740 1 client.20202 shutdown 2024-09-19T09:16:35.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.404+0000 7fe842714740 1 -- 172.21.15.120:0/1789649058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca8dad2370 msgr2=0x55ca8dacf480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:35.438 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:16:35.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.404+0000 7fe842714740 1 --2- 172.21.15.120:0/1789649058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca8dad2370 0x55ca8dacf480 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fe830010b70 tx=0x7fe830004ab0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.404+0000 7fe842714740 1 -- 172.21.15.120:0/1789649058 shutdown_connections 2024-09-19T09:16:35.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.404+0000 7fe842714740 1 --2- 172.21.15.120:0/1789649058 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55ca8dae1ef0 0x55ca8dae4300 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.404+0000 7fe842714740 1 --2- 172.21.15.120:0/1789649058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca8dad2370 0x55ca8dacf480 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.404+0000 7fe821ffb640 0 client.20202 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:16:35.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.404+0000 7fe842714740 1 --2- 172.21.15.120:0/1789649058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca8dac8b40 0x55ca8dacef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.404+0000 7fe842714740 1 --2- 172.21.15.120:0/1789649058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca8dac75d0 0x55ca8dad1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.404+0000 7fe842714740 1 -- 172.21.15.120:0/1789649058 >> 172.21.15.120:0/1789649058 conn(0x55ca8da90310 msgr2=0x55ca8dac9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:35.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.404+0000 7fe842714740 1 -- 172.21.15.120:0/1789649058 shutdown_connections 2024-09-19T09:16:35.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.404+0000 7fe842714740 1 -- 172.21.15.120:0/1789649058 wait complete. 2024-09-19T09:16:35.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.404+0000 7fe842714740 20 client.20202 trim_cache size 0 max 16384 2024-09-19T09:16:35.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.417+0000 7fe842714740 1 Processor -- start 2024-09-19T09:16:35.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.417+0000 7fe842714740 1 -- start start 2024-09-19T09:16:35.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.418+0000 7fe842714740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca8dac75d0 0x55ca8dade2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:35.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.418+0000 7fe842714740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca8dac8b40 0x55ca8dae9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:35.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.418+0000 7fe842714740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca8dad2370 0x55ca8daea110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:35.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.418+0000 7fe842714740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe8381ad660 con 0x55ca8dad2370 2024-09-19T09:16:35.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.418+0000 7fe842714740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe8381af230 con 0x55ca8dac75d0 2024-09-19T09:16:35.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.418+0000 7fe842714740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe8381add30 con 0x55ca8dac8b40 2024-09-19T09:16:35.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.418+0000 7fe83e8d8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca8dad2370 0x55ca8daea110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:35.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.418+0000 7fe83e8d8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca8dad2370 0x55ca8daea110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52320/0 (socket says 172.21.15.120:52320) 2024-09-19T09:16:35.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.418+0000 7fe83e8d8640 1 -- 172.21.15.120:0/234526014 learned_addr learned my addr 172.21.15.120:0/234526014 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:35.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.418+0000 7fe83d8d6640 1 --2- 172.21.15.120:0/234526014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca8dac8b40 0x55ca8dae9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:35.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.418+0000 7fe83e0d7640 1 --2- 172.21.15.120:0/234526014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca8dac75d0 0x55ca8dade2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:35.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.418+0000 7fe83e8d8640 1 -- 172.21.15.120:0/234526014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca8dac8b40 msgr2=0x55ca8dae9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:35.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.418+0000 7fe83e8d8640 1 --2- 172.21.15.120:0/234526014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca8dac8b40 0x55ca8dae9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.418+0000 7fe83e8d8640 1 -- 172.21.15.120:0/234526014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca8dac75d0 msgr2=0x55ca8dade2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:35.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.418+0000 7fe83e8d8640 1 --2- 172.21.15.120:0/234526014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca8dac75d0 0x55ca8dade2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.418+0000 7fe83e8d8640 1 -- 172.21.15.120:0/234526014 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca8dad1fc0 con 0x55ca8dad2370 2024-09-19T09:16:35.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.418+0000 7fe83e8d8640 1 --2- 172.21.15.120:0/234526014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca8dad2370 0x55ca8daea110 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fe8300325a0 tx=0x7fe830011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de75b05c8752488 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:35.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.419+0000 7fe823fff640 1 -- 172.21.15.120:0/234526014 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe830037030 con 0x55ca8dad2370 2024-09-19T09:16:35.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.419+0000 7fe823fff640 1 -- 172.21.15.120:0/234526014 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe830003e50 con 0x55ca8dad2370 2024-09-19T09:16:35.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.419+0000 7fe823fff640 1 -- 172.21.15.120:0/234526014 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8300042a0 con 0x55ca8dad2370 2024-09-19T09:16:35.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.419+0000 7fe842714740 1 -- 172.21.15.120:0/234526014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca8dad2370 msgr2=0x55ca8daea110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:35.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.419+0000 7fe842714740 1 --2- 172.21.15.120:0/234526014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca8dad2370 0x55ca8daea110 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fe8300325a0 tx=0x7fe830011690 comp rx=0 tx=0).stop 2024-09-19T09:16:35.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.419+0000 7fe842714740 1 -- 172.21.15.120:0/234526014 shutdown_connections 2024-09-19T09:16:35.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.419+0000 7fe842714740 1 --2- 172.21.15.120:0/234526014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca8dad2370 0x55ca8daea110 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.419+0000 7fe842714740 1 --2- 172.21.15.120:0/234526014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca8dac8b40 0x55ca8dae9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.419+0000 7fe842714740 1 --2- 172.21.15.120:0/234526014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca8dac75d0 0x55ca8dade2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.419+0000 7fe842714740 1 -- 172.21.15.120:0/234526014 >> 172.21.15.120:0/234526014 conn(0x55ca8da90310 msgr2=0x55ca8dacd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:35.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.420+0000 7fe842714740 1 -- 172.21.15.120:0/234526014 shutdown_connections 2024-09-19T09:16:35.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.420+0000 7fe842714740 1 -- 172.21.15.120:0/234526014 wait complete. 2024-09-19T09:16:35.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.420+0000 7fe842714740 1 -- start start 2024-09-19T09:16:35.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.420+0000 7fe842714740 1 Processor -- start 2024-09-19T09:16:35.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.420+0000 7fe842714740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca8dac75d0 0x55ca8dada930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:35.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.420+0000 7fe842714740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca8dac8b40 0x55ca8dad5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:35.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.420+0000 7fe842714740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca8dad2370 0x55ca8dad61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:35.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.420+0000 7fe842714740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe8381aeb60 con 0x55ca8dad2370 2024-09-19T09:16:35.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.420+0000 7fe842714740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe8381ae400 con 0x55ca8dac75d0 2024-09-19T09:16:35.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.420+0000 7fe842714740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe8381ab620 con 0x55ca8dac8b40 2024-09-19T09:16:35.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.420+0000 7fe842714740 10 client.0 _ll_register_callbacks cb 0x7fe830010a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:35.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.420+0000 7fe83e8d8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca8dad2370 0x55ca8dad61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:35.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.420+0000 7fe83e8d8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca8dad2370 0x55ca8dad61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52332/0 (socket says 172.21.15.120:52332) 2024-09-19T09:16:35.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.420+0000 7fe83d8d6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca8dac8b40 0x55ca8dad5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:35.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.420+0000 7fe83e8d8640 1 -- 172.21.15.120:0/1178362257 learned_addr learned my addr 172.21.15.120:0/1178362257 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:35.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.420+0000 7fe83e0d7640 1 --2- 172.21.15.120:0/1178362257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca8dac75d0 0x55ca8dada930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:35.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.421+0000 7fe83e8d8640 1 -- 172.21.15.120:0/1178362257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca8dac8b40 msgr2=0x55ca8dad5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:35.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.421+0000 7fe83e8d8640 1 --2- 172.21.15.120:0/1178362257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca8dac8b40 0x55ca8dad5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.421+0000 7fe83e8d8640 1 -- 172.21.15.120:0/1178362257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca8dac75d0 msgr2=0x55ca8dada930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:35.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.421+0000 7fe83e8d8640 1 --2- 172.21.15.120:0/1178362257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca8dac75d0 0x55ca8dada930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.421+0000 7fe83e8d8640 1 -- 172.21.15.120:0/1178362257 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca8dad6760 con 0x55ca8dad2370 2024-09-19T09:16:35.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.421+0000 7fe83e8d8640 1 --2- 172.21.15.120:0/1178362257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca8dad2370 0x55ca8dad61a0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fe830012010 tx=0x7fe830016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f08c5d735c73ac0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:35.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.421+0000 7fe821ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:16:35.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.421+0000 7fe821ffb640 1 -- 172.21.15.120:0/1178362257 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe830040060 con 0x55ca8dad2370 2024-09-19T09:16:35.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.421+0000 7fe821ffb640 1 -- 172.21.15.120:0/1178362257 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe8300052c0 con 0x55ca8dad2370 2024-09-19T09:16:35.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.421+0000 7fe842714740 1 -- 172.21.15.120:0/1178362257 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe830003e50 con 0x55ca8dad2370 2024-09-19T09:16:35.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.421+0000 7fe821ffb640 1 -- 172.21.15.120:0/1178362257 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe83002c040 con 0x55ca8dad2370 2024-09-19T09:16:35.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.421+0000 7fe842714740 10 client.20208 fetch_fsmap learned FSMap version 36 2024-09-19T09:16:35.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.421+0000 7fe842714740 1 -- 172.21.15.120:0/1178362257 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ca8db4c970 con 0x55ca8dad2370 2024-09-19T09:16:35.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.421+0000 7fe821ffb640 1 -- 172.21.15.120:0/1178362257 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe830003e50 con 0x55ca8dad2370 2024-09-19T09:16:35.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.421+0000 7fe821ffb640 1 -- 172.21.15.120:0/1178362257 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fe830045070 con 0x55ca8dad2370 2024-09-19T09:16:35.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.421+0000 7fe842714740 10 client.20208 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:16:35.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.422+0000 7fe842714740 10 client.20208 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:16:35.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.422+0000 7fe842714740 20 client.20208 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:35.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.422+0000 7fe842714740 1 --2- 172.21.15.120:0/1178362257 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55ca8dae1ef0 0x55ca8db4ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:35.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.422+0000 7fe842714740 4 client.20208 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:16:35.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.422+0000 7fe842714740 1 -- 172.21.15.120:0/1178362257 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ca8da34d70 con 0x55ca8dae1ef0 2024-09-19T09:16:35.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.422+0000 7fe83e0d7640 1 --2- 172.21.15.120:0/1178362257 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55ca8dae1ef0 0x55ca8db4ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:35.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.422+0000 7fe83e0d7640 1 --2- 172.21.15.120:0/1178362257 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55ca8dae1ef0 0x55ca8db4ef80 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f19f4aa4db41ccad server_cookie=23b919d20bb894a5 in_seq=0 out_seq=0 2024-09-19T09:16:35.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.422+0000 7fe821ffb640 10 client.20208 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:16:35.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.423+0000 7fe821ffb640 1 -- 172.21.15.120:0/1178362257 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ca8da34d70 con 0x55ca8dae1ef0 2024-09-19T09:16:35.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.423+0000 7fe821ffb640 10 client.20208 handle_command_reply: tid=0 2024-09-19T09:16:35.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.423+0000 7fe842714740 1 client.20208 shutdown 2024-09-19T09:16:35.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.423+0000 7fe842714740 1 -- 172.21.15.120:0/1178362257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca8dad2370 msgr2=0x55ca8dad61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:35.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.423+0000 7fe842714740 1 --2- 172.21.15.120:0/1178362257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca8dad2370 0x55ca8dad61a0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fe830012010 tx=0x7fe830016000 comp rx=0 tx=0).stop 2024-09-19T09:16:35.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.423+0000 7fe842714740 1 -- 172.21.15.120:0/1178362257 shutdown_connections 2024-09-19T09:16:35.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.423+0000 7fe842714740 1 --2- 172.21.15.120:0/1178362257 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55ca8dae1ef0 0x55ca8db4ef80 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.423+0000 7fe842714740 1 --2- 172.21.15.120:0/1178362257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca8dad2370 0x55ca8dad61a0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.423+0000 7fe842714740 1 --2- 172.21.15.120:0/1178362257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ca8dac8b40 0x55ca8dad5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.423+0000 7fe821ffb640 0 client.20208 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:16:35.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.423+0000 7fe842714740 1 --2- 172.21.15.120:0/1178362257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca8dac75d0 0x55ca8dada930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.423+0000 7fe842714740 1 -- 172.21.15.120:0/1178362257 >> 172.21.15.120:0/1178362257 conn(0x55ca8da90310 msgr2=0x55ca8dac9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:35.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.423+0000 7fe842714740 1 -- 172.21.15.120:0/1178362257 shutdown_connections 2024-09-19T09:16:35.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.423+0000 7fe842714740 1 -- 172.21.15.120:0/1178362257 wait complete. 2024-09-19T09:16:35.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.423+0000 7fe842714740 20 client.20208 trim_cache size 0 max 16384 2024-09-19T09:16:35.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.424+0000 7fe84033b640 1 -- 172.21.15.120:0/3568076525 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe8240a3240 msgr2=0x7fe8240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:35.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.424+0000 7fe84033b640 1 --2- 172.21.15.120:0/3568076525 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe8240a3240 0x7fe8240a5700 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe830005fd0 tx=0x7fe8300096a0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.424+0000 7fe84033b640 1 -- 172.21.15.120:0/3568076525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe838100e20 msgr2=0x7fe838198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:35.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.424+0000 7fe84033b640 1 --2- 172.21.15.120:0/3568076525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe838100e20 0x7fe838198590 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe83400d770 tx=0x7fe83400dc30 comp rx=0 tx=0).stop 2024-09-19T09:16:35.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.424+0000 7fe84033b640 1 -- 172.21.15.120:0/3568076525 shutdown_connections 2024-09-19T09:16:35.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.424+0000 7fe84033b640 1 --2- 172.21.15.120:0/3568076525 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe8240a3240 0x7fe8240a5700 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.424+0000 7fe84033b640 1 --2- 172.21.15.120:0/3568076525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe838102ef0 0x7fe83819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.424+0000 7fe84033b640 1 --2- 172.21.15.120:0/3568076525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe838102570 0x7fe838198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.424+0000 7fe84033b640 1 --2- 172.21.15.120:0/3568076525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe838100e20 0x7fe838198590 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:35.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.424+0000 7fe84033b640 1 -- 172.21.15.120:0/3568076525 >> 172.21.15.120:0/3568076525 conn(0x7fe8380fc7d0 msgr2=0x7fe83806a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:35.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.424+0000 7fe84033b640 1 -- 172.21.15.120:0/3568076525 shutdown_connections 2024-09-19T09:16:35.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:35.424+0000 7fe84033b640 1 -- 172.21.15.120:0/3568076525 wait complete. 2024-09-19T09:16:36.149 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:16:36.353 INFO:tasks.workunit.client.0.smithi120.stdout: 10 4603 14.12 MB/sec warmup 31 sec latency 46.637 ms 2024-09-19T09:16:36.425 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:16:36.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.928+0000 7f6645f5e640 1 -- 172.21.15.120:0/996545706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6640100e20 msgr2=0x7f6640101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:36.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.928+0000 7f6645f5e640 1 --2- 172.21.15.120:0/996545706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6640100e20 0x7f6640101200 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f6630009900 tx=0x7f663002a480 comp rx=0 tx=0).stop 2024-09-19T09:16:36.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.928+0000 7f6645f5e640 1 -- 172.21.15.120:0/996545706 shutdown_connections 2024-09-19T09:16:36.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.928+0000 7f6645f5e640 1 --2- 172.21.15.120:0/996545706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6640102ef0 0x7f664010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.928+0000 7f6645f5e640 1 --2- 172.21.15.120:0/996545706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6640102570 0x7f66401029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.928+0000 7f6645f5e640 1 --2- 172.21.15.120:0/996545706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6640100e20 0x7f6640101200 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.928+0000 7f6645f5e640 1 -- 172.21.15.120:0/996545706 >> 172.21.15.120:0/996545706 conn(0x7f66400fc7d0 msgr2=0x7f66400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:36.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.928+0000 7f6645f5e640 1 -- 172.21.15.120:0/996545706 shutdown_connections 2024-09-19T09:16:36.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.928+0000 7f6645f5e640 1 -- 172.21.15.120:0/996545706 wait complete. 2024-09-19T09:16:36.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.929+0000 7f6645f5e640 1 Processor -- start 2024-09-19T09:16:36.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.929+0000 7f6645f5e640 1 -- start start 2024-09-19T09:16:36.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.929+0000 7f6645f5e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6640100e20 0x7f6640197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:36.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.929+0000 7f6645f5e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6640102570 0x7f6640197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:36.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.929+0000 7f6645f5e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6640102ef0 0x7f6640197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:36.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.929+0000 7f6645f5e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6640110f80 con 0x7f6640102ef0 2024-09-19T09:16:36.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.929+0000 7f6645f5e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6640110e00 con 0x7f6640100e20 2024-09-19T09:16:36.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.929+0000 7f6645f5e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6640111100 con 0x7f6640102570 2024-09-19T09:16:36.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.929+0000 7f663f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6640100e20 0x7f6640197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:36.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.929+0000 7f663ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6640102ef0 0x7f6640197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:36.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.929+0000 7f663effd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6640102570 0x7f6640197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:36.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.929+0000 7f663f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6640100e20 0x7f6640197140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:50732/0 (socket says 172.21.15.120:50732) 2024-09-19T09:16:36.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.929+0000 7f663f7fe640 1 -- 172.21.15.120:0/995914780 learned_addr learned my addr 172.21.15.120:0/995914780 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:36.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.930+0000 7f663ffff640 1 -- 172.21.15.120:0/995914780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6640102570 msgr2=0x7f6640197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:36.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.930+0000 7f663ffff640 1 --2- 172.21.15.120:0/995914780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6640102570 0x7f6640197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.930+0000 7f663ffff640 1 -- 172.21.15.120:0/995914780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6640100e20 msgr2=0x7f6640197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:36.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.930+0000 7f663ffff640 1 --2- 172.21.15.120:0/995914780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6640100e20 0x7f6640197140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.930+0000 7f663ffff640 1 -- 172.21.15.120:0/995914780 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66401a69b0 con 0x7f6640102ef0 2024-09-19T09:16:36.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.930+0000 7f663effd640 1 --2- 172.21.15.120:0/995914780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6640102570 0x7f6640197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:16:36.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.930+0000 7f663ffff640 1 --2- 172.21.15.120:0/995914780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6640102ef0 0x7f6640197bc0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f663800da10 tx=0x7f663800ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:36.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.930+0000 7f663cff9640 1 -- 172.21.15.120:0/995914780 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663800bd70 con 0x7f6640102ef0 2024-09-19T09:16:36.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.930+0000 7f6645f5e640 1 -- 172.21.15.120:0/995914780 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66401a6ca0 con 0x7f6640102ef0 2024-09-19T09:16:36.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.930+0000 7f663cff9640 1 -- 172.21.15.120:0/995914780 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6638003e50 con 0x7f6640102ef0 2024-09-19T09:16:36.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.930+0000 7f663cff9640 1 -- 172.21.15.120:0/995914780 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66380106a0 con 0x7f6640102ef0 2024-09-19T09:16:36.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.930+0000 7f6645f5e640 1 -- 172.21.15.120:0/995914780 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66401a71e0 con 0x7f6640102ef0 2024-09-19T09:16:36.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.933+0000 7f663cff9640 1 -- 172.21.15.120:0/995914780 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f6638010860 con 0x7f6640102ef0 2024-09-19T09:16:36.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.933+0000 7f6648337740 1 Processor -- start 2024-09-19T09:16:36.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.933+0000 7f6648337740 1 -- start start 2024-09-19T09:16:36.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.933+0000 7f6648337740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea818e3b40 0x55ea818e2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:36.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.933+0000 7f6648337740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea818e25d0 0x55ea818e2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:36.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.933+0000 7f6648337740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea818ed370 0x55ea818ef760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:36.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.933+0000 7f6648337740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ea818482f0 con 0x55ea818e25d0 2024-09-19T09:16:36.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.933+0000 7f6648337740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ea817efcf0 con 0x55ea818e3b40 2024-09-19T09:16:36.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.933+0000 7f6648337740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ea818442b0 con 0x55ea818ed370 2024-09-19T09:16:36.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.934+0000 7f663f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea818e25d0 0x55ea818e2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:36.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.934+0000 7f663effd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea818e3b40 0x55ea818e2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:36.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.934+0000 7f663effd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea818e3b40 0x55ea818e2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:50736/0 (socket says 172.21.15.120:50736) 2024-09-19T09:16:36.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.934+0000 7f663f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea818e25d0 0x55ea818e2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52358/0 (socket says 172.21.15.120:52358) 2024-09-19T09:16:36.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.934+0000 7f663effd640 1 -- 172.21.15.120:0/3561980101 learned_addr learned my addr 172.21.15.120:0/3561980101 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:36.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.934+0000 7f663effd640 1 --2- 172.21.15.120:0/3561980101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea818ed370 0x55ea818ef760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:36.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.934+0000 7f663f7fe640 1 -- 172.21.15.120:0/3561980101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea818ed370 msgr2=0x55ea818ef760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:36.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.934+0000 7f663f7fe640 1 --2- 172.21.15.120:0/3561980101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea818ed370 0x55ea818ef760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.934+0000 7f663f7fe640 1 -- 172.21.15.120:0/3561980101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea818e3b40 msgr2=0x55ea818e2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:36.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.934+0000 7f663f7fe640 1 --2- 172.21.15.120:0/3561980101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea818e3b40 0x55ea818e2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.934+0000 7f663f7fe640 1 -- 172.21.15.120:0/3561980101 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea817d43a0 con 0x55ea818e25d0 2024-09-19T09:16:36.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.934+0000 7f663effd640 1 --2- 172.21.15.120:0/3561980101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea818e3b40 0x55ea818e2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:16:36.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.934+0000 7f663f7fe640 1 --2- 172.21.15.120:0/3561980101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea818e25d0 0x55ea818e2a10 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f6630009230 tx=0x7f6630004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c578a205d17056 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:36.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.934+0000 7f663cff9640 1 --2- 172.21.15.120:0/995914780 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f66280a3240 0x7f66280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:36.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.934+0000 7f663effd640 1 --2- 172.21.15.120:0/995914780 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f66280a3240 0x7f66280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:36.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.934+0000 7f6625ffb640 1 -- 172.21.15.120:0/3561980101 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6630040070 con 0x55ea818e25d0 2024-09-19T09:16:36.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.934+0000 7f6625ffb640 1 -- 172.21.15.120:0/3561980101 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6630003ad0 con 0x55ea818e25d0 2024-09-19T09:16:36.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.934+0000 7f6625ffb640 1 -- 172.21.15.120:0/3561980101 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6630035040 con 0x55ea818e25d0 2024-09-19T09:16:36.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.934+0000 7f663cff9640 1 -- 172.21.15.120:0/995914780 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f66380cfc60 con 0x7f6640102ef0 2024-09-19T09:16:36.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.935+0000 7f6648337740 1 -- 172.21.15.120:0/3561980101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea818e25d0 msgr2=0x55ea818e2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:36.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.935+0000 7f663effd640 1 --2- 172.21.15.120:0/995914780 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f66280a3240 0x7f66280a5700 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x55ea818e3070 tx=0x7f663400bbf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:36.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.935+0000 7f6648337740 1 --2- 172.21.15.120:0/3561980101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea818e25d0 0x55ea818e2a10 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f6630009230 tx=0x7f6630004170 comp rx=0 tx=0).stop 2024-09-19T09:16:36.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.935+0000 7f6648337740 1 -- 172.21.15.120:0/3561980101 shutdown_connections 2024-09-19T09:16:36.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.935+0000 7f6648337740 1 --2- 172.21.15.120:0/3561980101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea818ed370 0x55ea818ef760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.935+0000 7f6648337740 1 --2- 172.21.15.120:0/3561980101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea818e25d0 0x55ea818e2a10 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.935+0000 7f6648337740 1 --2- 172.21.15.120:0/3561980101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea818e3b40 0x55ea818e2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.935+0000 7f6648337740 1 -- 172.21.15.120:0/3561980101 >> 172.21.15.120:0/3561980101 conn(0x55ea818ab310 msgr2=0x55ea818dec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:36.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.936+0000 7f6648337740 1 -- 172.21.15.120:0/3561980101 shutdown_connections 2024-09-19T09:16:36.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.936+0000 7f6648337740 1 -- 172.21.15.120:0/3561980101 wait complete. 2024-09-19T09:16:36.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.936+0000 7f6648337740 1 -- start start 2024-09-19T09:16:36.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.936+0000 7f6648337740 1 Processor -- start 2024-09-19T09:16:36.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.937+0000 7f6648337740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea818e25d0 0x55ea818ecd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:36.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.937+0000 7f6648337740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea818e3b40 0x55ea818ea030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:36.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.937+0000 7f6648337740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea818ed370 0x55ea818ea640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:36.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.937+0000 7f6648337740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ea818f0010 con 0x55ea818ed370 2024-09-19T09:16:36.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.937+0000 7f6648337740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ea81803060 con 0x55ea818e3b40 2024-09-19T09:16:36.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.937+0000 7f6648337740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ea818f0190 con 0x55ea818e25d0 2024-09-19T09:16:36.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.937+0000 7f663f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea818e25d0 0x55ea818ecd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:36.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.937+0000 7f663effd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea818e3b40 0x55ea818ea030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:36.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.937+0000 7f663ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea818ed370 0x55ea818ea640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:36.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.937+0000 7f663ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea818ed370 0x55ea818ea640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52360/0 (socket says 172.21.15.120:52360) 2024-09-19T09:16:36.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.937+0000 7f663ffff640 1 -- 172.21.15.120:0/3964213390 learned_addr learned my addr 172.21.15.120:0/3964213390 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:36.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.937+0000 7f663f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea818e25d0 0x55ea818ecd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:33574/0 (socket says 172.21.15.120:33574) 2024-09-19T09:16:36.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.937+0000 7f663effd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea818e3b40 0x55ea818ea030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:50740/0 (socket says 172.21.15.120:50740) 2024-09-19T09:16:36.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.937+0000 7f6648337740 10 client.0 _ll_register_callbacks cb 0x55ea815568c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:36.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.937+0000 7f663ffff640 1 -- 172.21.15.120:0/3964213390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea818e25d0 msgr2=0x55ea818ecd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:36.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.937+0000 7f663ffff640 1 --2- 172.21.15.120:0/3964213390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea818e25d0 0x55ea818ecd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.937+0000 7f663ffff640 1 -- 172.21.15.120:0/3964213390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea818e3b40 msgr2=0x55ea818ea030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:36.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.937+0000 7f663ffff640 1 --2- 172.21.15.120:0/3964213390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea818e3b40 0x55ea818ea030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.937+0000 7f663ffff640 1 -- 172.21.15.120:0/3964213390 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea815b93c0 con 0x55ea818ed370 2024-09-19T09:16:36.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.937+0000 7f6648337740 1 -- 172.21.15.120:0/3964213390 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6630003ad0 con 0x55ea818ed370 2024-09-19T09:16:36.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.937+0000 7f663ffff640 1 --2- 172.21.15.120:0/3964213390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea818ed370 0x55ea818ea640 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f663809c440 tx=0x7f66380a4f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660e25353d22913f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:36.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.937+0000 7f661effd640 10 client.20220 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:16:36.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.937+0000 7f661effd640 1 -- 172.21.15.120:0/3964213390 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66380042e0 con 0x55ea818ed370 2024-09-19T09:16:36.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.937+0000 7f661effd640 1 -- 172.21.15.120:0/3964213390 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f6638098b20 con 0x55ea818ed370 2024-09-19T09:16:36.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.938+0000 7f661effd640 1 -- 172.21.15.120:0/3964213390 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f66380bc0c0 con 0x55ea818ed370 2024-09-19T09:16:36.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.938+0000 7f6648337740 10 client.20220 fetch_fsmap learned FSMap version 36 2024-09-19T09:16:36.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.938+0000 7f6648337740 1 -- 172.21.15.120:0/3964213390 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({fsmap=36}) -- 0x55ea818fba00 con 0x55ea818ed370 2024-09-19T09:16:36.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.938+0000 7f661effd640 1 -- 172.21.15.120:0/3964213390 <== mon.0 v2:172.21.15.120:3300/0 4 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f6638024060 con 0x55ea818ed370 2024-09-19T09:16:36.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.938+0000 7f6648337740 10 client.20220 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:16:36.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.938+0000 7f6648337740 10 client.20220 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:16:36.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.938+0000 7f6648337740 20 client.20220 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:36.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.938+0000 7f6648337740 1 --2- 172.21.15.120:0/3964213390 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55ea818fce00 0x55ea818ff270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:36.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.938+0000 7f6648337740 4 client.20220 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:16:36.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.938+0000 7f6648337740 1 -- 172.21.15.120:0/3964213390 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea81855860 con 0x55ea818fce00 2024-09-19T09:16:36.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.938+0000 7f663f7fe640 1 --2- 172.21.15.120:0/3964213390 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55ea818fce00 0x55ea818ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:36.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.939+0000 7f661effd640 10 client.20220 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:16:36.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.939+0000 7f663f7fe640 1 --2- 172.21.15.120:0/3964213390 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55ea818fce00 0x55ea818ff270 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2a0db6902c63240f server_cookie=455eee392159553b in_seq=0 out_seq=0 2024-09-19T09:16:36.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.939+0000 7f661effd640 1 -- 172.21.15.120:0/3964213390 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f6630050e80 con 0x55ea818fce00 2024-09-19T09:16:36.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.939+0000 7f661effd640 10 client.20220 handle_command_reply: tid=0 2024-09-19T09:16:36.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.940+0000 7f6648337740 1 client.20220 shutdown 2024-09-19T09:16:36.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.940+0000 7f6648337740 1 -- 172.21.15.120:0/3964213390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea818ed370 msgr2=0x55ea818ea640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:36.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.940+0000 7f6648337740 1 --2- 172.21.15.120:0/3964213390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea818ed370 0x55ea818ea640 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f663809c440 tx=0x7f66380a4f70 comp rx=0 tx=0).stop 2024-09-19T09:16:36.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.940+0000 7f6648337740 1 -- 172.21.15.120:0/3964213390 shutdown_connections 2024-09-19T09:16:36.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.940+0000 7f6648337740 1 --2- 172.21.15.120:0/3964213390 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55ea818fce00 0x55ea818ff270 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.940+0000 7f6648337740 1 --2- 172.21.15.120:0/3964213390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea818ed370 0x55ea818ea640 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.940+0000 7f6648337740 1 --2- 172.21.15.120:0/3964213390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea818e3b40 0x55ea818ea030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.940+0000 7f661effd640 0 client.20220 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:16:36.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.940+0000 7f6648337740 1 --2- 172.21.15.120:0/3964213390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea818e25d0 0x55ea818ecd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.940+0000 7f6648337740 1 -- 172.21.15.120:0/3964213390 >> 172.21.15.120:0/3964213390 conn(0x55ea818ab310 msgr2=0x55ea818e4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:36.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.940+0000 7f6648337740 1 -- 172.21.15.120:0/3964213390 shutdown_connections 2024-09-19T09:16:36.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.940+0000 7f6648337740 1 -- 172.21.15.120:0/3964213390 wait complete. 2024-09-19T09:16:36.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.940+0000 7f6648337740 20 client.20220 trim_cache size 0 max 16384 2024-09-19T09:16:36.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.954+0000 7f6648337740 1 Processor -- start 2024-09-19T09:16:36.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.954+0000 7f6648337740 1 -- start start 2024-09-19T09:16:36.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.954+0000 7f6648337740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea818e25d0 0x55ea818ebfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:36.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.954+0000 7f6648337740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea818e3b40 0x55ea818e5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:36.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.954+0000 7f6648337740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea818ed370 0x55ea818e5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:36.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.954+0000 7f6648337740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f66401af6c0 con 0x55ea818e25d0 2024-09-19T09:16:36.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.954+0000 7f6648337740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f66401b1290 con 0x55ea818ed370 2024-09-19T09:16:36.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.954+0000 7f6648337740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f66401afd90 con 0x55ea818e3b40 2024-09-19T09:16:36.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.954+0000 7f663f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea818e25d0 0x55ea818ebfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:36.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.954+0000 7f663ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea818ed370 0x55ea818e5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:36.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.954+0000 7f663effd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea818e3b40 0x55ea818e5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:36.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.954+0000 7f663f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea818e25d0 0x55ea818ebfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52372/0 (socket says 172.21.15.120:52372) 2024-09-19T09:16:36.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.954+0000 7f663ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea818ed370 0x55ea818e5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:50748/0 (socket says 172.21.15.120:50748) 2024-09-19T09:16:36.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.954+0000 7f663f7fe640 1 -- 172.21.15.120:0/3770921885 learned_addr learned my addr 172.21.15.120:0/3770921885 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:36.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.954+0000 7f663f7fe640 1 -- 172.21.15.120:0/3770921885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea818e3b40 msgr2=0x55ea818e5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:36.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.954+0000 7f663f7fe640 1 --2- 172.21.15.120:0/3770921885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea818e3b40 0x55ea818e5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.954+0000 7f663f7fe640 1 -- 172.21.15.120:0/3770921885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea818ed370 msgr2=0x55ea818e5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:36.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.954+0000 7f663f7fe640 1 --2- 172.21.15.120:0/3770921885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea818ed370 0x55ea818e5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.955+0000 7f663f7fe640 1 -- 172.21.15.120:0/3770921885 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea818e8fe0 con 0x55ea818e25d0 2024-09-19T09:16:36.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.955+0000 7f663ffff640 1 --2- 172.21.15.120:0/3770921885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea818ed370 0x55ea818e5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:16:36.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.955+0000 7f663f7fe640 1 --2- 172.21.15.120:0/3770921885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea818e25d0 0x55ea818ebfa0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f6630009a20 tx=0x7f66300045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5747b355eb8038fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:36.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.955+0000 7f66257fa640 1 -- 172.21.15.120:0/3770921885 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6630035040 con 0x55ea818e25d0 2024-09-19T09:16:36.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.955+0000 7f66257fa640 1 -- 172.21.15.120:0/3770921885 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f663004b040 con 0x55ea818e25d0 2024-09-19T09:16:36.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.955+0000 7f66257fa640 1 -- 172.21.15.120:0/3770921885 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6630040040 con 0x55ea818e25d0 2024-09-19T09:16:36.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.956+0000 7f6648337740 1 -- 172.21.15.120:0/3770921885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea818e25d0 msgr2=0x55ea818ebfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:36.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.956+0000 7f6648337740 1 --2- 172.21.15.120:0/3770921885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea818e25d0 0x55ea818ebfa0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f6630009a20 tx=0x7f66300045b0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.956+0000 7f6648337740 1 -- 172.21.15.120:0/3770921885 shutdown_connections 2024-09-19T09:16:36.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.956+0000 7f6648337740 1 --2- 172.21.15.120:0/3770921885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea818ed370 0x55ea818e5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.956+0000 7f6648337740 1 --2- 172.21.15.120:0/3770921885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea818e3b40 0x55ea818e5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.956+0000 7f6648337740 1 --2- 172.21.15.120:0/3770921885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea818e25d0 0x55ea818ebfa0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.956+0000 7f6648337740 1 -- 172.21.15.120:0/3770921885 >> 172.21.15.120:0/3770921885 conn(0x55ea818ab310 msgr2=0x55ea818eb890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:36.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.956+0000 7f6648337740 1 -- 172.21.15.120:0/3770921885 shutdown_connections 2024-09-19T09:16:36.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.956+0000 7f6648337740 1 -- 172.21.15.120:0/3770921885 wait complete. 2024-09-19T09:16:36.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.956+0000 7f6648337740 1 -- start start 2024-09-19T09:16:36.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.956+0000 7f6648337740 1 Processor -- start 2024-09-19T09:16:36.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.957+0000 7f6648337740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea818e25d0 0x55ea818e71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:36.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.957+0000 7f6648337740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea818e3b40 0x55ea818e7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:36.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.957+0000 7f6648337740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea818ed370 0x55ea818e7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:36.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.957+0000 7f6648337740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f66401b0bc0 con 0x55ea818e3b40 2024-09-19T09:16:36.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.957+0000 7f6648337740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f66401b0460 con 0x55ea818e25d0 2024-09-19T09:16:36.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.957+0000 7f6648337740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f66401ad680 con 0x55ea818ed370 2024-09-19T09:16:36.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.957+0000 7f6648337740 10 client.0 _ll_register_callbacks cb 0x7f6638098030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:36.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.957+0000 7f663effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea818e3b40 0x55ea818e7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:36.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.957+0000 7f663f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea818e25d0 0x55ea818e71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:36.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.957+0000 7f663effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea818e3b40 0x55ea818e7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52388/0 (socket says 172.21.15.120:52388) 2024-09-19T09:16:36.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.957+0000 7f663effd640 1 -- 172.21.15.120:0/3336431974 learned_addr learned my addr 172.21.15.120:0/3336431974 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:36.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.957+0000 7f663ffff640 1 --2- 172.21.15.120:0/3336431974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea818ed370 0x55ea818e7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:36.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.957+0000 7f663effd640 1 -- 172.21.15.120:0/3336431974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea818ed370 msgr2=0x55ea818e7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:36.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.957+0000 7f663effd640 1 --2- 172.21.15.120:0/3336431974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea818ed370 0x55ea818e7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.957+0000 7f663effd640 1 -- 172.21.15.120:0/3336431974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea818e25d0 msgr2=0x55ea818e71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:36.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.957+0000 7f663effd640 1 --2- 172.21.15.120:0/3336431974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea818e25d0 0x55ea818e71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.957+0000 7f663effd640 1 -- 172.21.15.120:0/3336431974 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea81966c40 con 0x55ea818e3b40 2024-09-19T09:16:36.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.957+0000 7f663ffff640 1 --2- 172.21.15.120:0/3336431974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea818ed370 0x55ea818e7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:16:36.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.958+0000 7f663effd640 1 --2- 172.21.15.120:0/3336431974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea818e3b40 0x55ea818e7700 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f663400bfd0 tx=0x7f6634004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b8e442944035b9a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:36.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.958+0000 7f661effd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:16:36.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.958+0000 7f661effd640 1 -- 172.21.15.120:0/3336431974 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6634023d50 con 0x55ea818e3b40 2024-09-19T09:16:36.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.958+0000 7f661effd640 1 -- 172.21.15.120:0/3336431974 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f663401b030 con 0x55ea818e3b40 2024-09-19T09:16:36.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.958+0000 7f6648337740 1 -- 172.21.15.120:0/3336431974 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f663004b040 con 0x55ea818e3b40 2024-09-19T09:16:36.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.958+0000 7f661effd640 1 -- 172.21.15.120:0/3336431974 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6634004020 con 0x55ea818e3b40 2024-09-19T09:16:36.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.958+0000 7f6648337740 10 client.20226 fetch_fsmap learned FSMap version 36 2024-09-19T09:16:36.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.958+0000 7f6648337740 1 -- 172.21.15.120:0/3336431974 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ea81967f20 con 0x55ea818e3b40 2024-09-19T09:16:36.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.958+0000 7f661effd640 1 -- 172.21.15.120:0/3336431974 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f663004b040 con 0x55ea818e3b40 2024-09-19T09:16:36.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.958+0000 7f661effd640 1 -- 172.21.15.120:0/3336431974 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f663402c410 con 0x55ea818e3b40 2024-09-19T09:16:36.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.958+0000 7f6648337740 10 client.20226 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:16:36.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.958+0000 7f6648337740 10 client.20226 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:16:36.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.958+0000 7f6648337740 20 client.20226 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:36.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.958+0000 7f6648337740 1 --2- 172.21.15.120:0/3336431974 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55ea81968720 0x55ea8196ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:36.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.958+0000 7f6648337740 4 client.20226 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:16:36.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.958+0000 7f6648337740 1 -- 172.21.15.120:0/3336431974 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ea8184fd70 con 0x55ea81968720 2024-09-19T09:16:36.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.959+0000 7f663f7fe640 1 --2- 172.21.15.120:0/3336431974 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55ea81968720 0x55ea8196ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:36.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.959+0000 7f661effd640 10 client.20226 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:16:36.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.959+0000 7f663f7fe640 1 --2- 172.21.15.120:0/3336431974 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55ea81968720 0x55ea8196ab40 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3d52d0ff1ae3ea8f server_cookie=f61f520dac0eb0a5 in_seq=0 out_seq=0 2024-09-19T09:16:36.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.959+0000 7f661effd640 1 -- 172.21.15.120:0/3336431974 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ea8184fd70 con 0x55ea81968720 2024-09-19T09:16:36.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.959+0000 7f661effd640 10 client.20226 handle_command_reply: tid=0 2024-09-19T09:16:36.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.959+0000 7f6648337740 1 client.20226 shutdown 2024-09-19T09:16:36.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.959+0000 7f6648337740 1 -- 172.21.15.120:0/3336431974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea818e3b40 msgr2=0x55ea818e7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:36.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.959+0000 7f6648337740 1 --2- 172.21.15.120:0/3336431974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea818e3b40 0x55ea818e7700 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f663400bfd0 tx=0x7f6634004730 comp rx=0 tx=0).stop 2024-09-19T09:16:36.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.959+0000 7f6648337740 1 -- 172.21.15.120:0/3336431974 shutdown_connections 2024-09-19T09:16:36.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.959+0000 7f6648337740 1 --2- 172.21.15.120:0/3336431974 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55ea81968720 0x55ea8196ab40 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.959+0000 7f6648337740 1 --2- 172.21.15.120:0/3336431974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea818ed370 0x55ea818e7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.959+0000 7f6648337740 1 --2- 172.21.15.120:0/3336431974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ea818e3b40 0x55ea818e7700 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.959+0000 7f6648337740 1 --2- 172.21.15.120:0/3336431974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ea818e25d0 0x55ea818e71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.959+0000 7f6648337740 1 -- 172.21.15.120:0/3336431974 >> 172.21.15.120:0/3336431974 conn(0x55ea818ab310 msgr2=0x55ea818ddd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:36.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.959+0000 7f661effd640 0 client.20226 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:16:36.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.960+0000 7f6648337740 1 -- 172.21.15.120:0/3336431974 shutdown_connections 2024-09-19T09:16:36.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.960+0000 7f6648337740 1 -- 172.21.15.120:0/3336431974 wait complete. 2024-09-19T09:16:36.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.960+0000 7f6648337740 20 client.20226 trim_cache size 0 max 16384 2024-09-19T09:16:36.969 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:16:36.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.960+0000 7f6645f5e640 1 -- 172.21.15.120:0/995914780 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f66280a3240 msgr2=0x7f66280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:36.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.960+0000 7f6645f5e640 1 --2- 172.21.15.120:0/995914780 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f66280a3240 0x7f66280a5700 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x55ea818e3070 tx=0x7f663400bbf0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.960+0000 7f6645f5e640 1 -- 172.21.15.120:0/995914780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6640102ef0 msgr2=0x7f6640197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:36.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.960+0000 7f6645f5e640 1 --2- 172.21.15.120:0/995914780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6640102ef0 0x7f6640197bc0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f663800da10 tx=0x7f663800ded0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.960+0000 7f6645f5e640 1 -- 172.21.15.120:0/995914780 shutdown_connections 2024-09-19T09:16:36.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.960+0000 7f6645f5e640 1 --2- 172.21.15.120:0/995914780 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f66280a3240 0x7f66280a5700 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.960+0000 7f6645f5e640 1 --2- 172.21.15.120:0/995914780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6640102ef0 0x7f6640197bc0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.960+0000 7f6645f5e640 1 --2- 172.21.15.120:0/995914780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6640102570 0x7f6640197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.960+0000 7f6645f5e640 1 --2- 172.21.15.120:0/995914780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6640100e20 0x7f6640197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:36.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.960+0000 7f6645f5e640 1 -- 172.21.15.120:0/995914780 >> 172.21.15.120:0/995914780 conn(0x7f66400fc7d0 msgr2=0x7f66400fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:36.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.960+0000 7f6645f5e640 1 -- 172.21.15.120:0/995914780 shutdown_connections 2024-09-19T09:16:36.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:36.960+0000 7f6645f5e640 1 -- 172.21.15.120:0/995914780 wait complete. 2024-09-19T09:16:37.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:36 smithi196 ceph-mon[35150]: pgmap v757: 97 pgs: 97 active+clean; 187 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 19 MiB/s wr, 173 op/s 2024-09-19T09:16:37.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:36 smithi153 ceph-mon[35071]: pgmap v757: 97 pgs: 97 active+clean; 187 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 19 MiB/s wr, 173 op/s 2024-09-19T09:16:37.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:36 smithi120 ceph-mon[31150]: pgmap v757: 97 pgs: 97 active+clean; 187 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 19 MiB/s wr, 173 op/s 2024-09-19T09:16:37.352 INFO:tasks.workunit.client.0.smithi120.stdout: 10 5078 14.14 MB/sec warmup 32 sec latency 129.666 ms 2024-09-19T09:16:38.038 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:16:38.344 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:16:38.352 INFO:tasks.workunit.client.0.smithi120.stdout: 10 5647 14.27 MB/sec warmup 33 sec latency 88.733 ms 2024-09-19T09:16:38.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.874+0000 7fcfcfd14640 1 -- 172.21.15.120:0/1667789762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcfc8100aa0 msgr2=0x7fcfc810ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:38.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.874+0000 7fcfcfd14640 1 --2- 172.21.15.120:0/1667789762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcfc8100aa0 0x7fcfc810ea20 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009ca0 tx=0x7fcfc002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.874+0000 7fcfcfd14640 1 -- 172.21.15.120:0/1667789762 shutdown_connections 2024-09-19T09:16:38.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.874+0000 7fcfcfd14640 1 --2- 172.21.15.120:0/1667789762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcfc8100aa0 0x7fcfc810ea20 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.874+0000 7fcfcfd14640 1 --2- 172.21.15.120:0/1667789762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfc8100100 0x7fcfc8100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.874+0000 7fcfcfd14640 1 --2- 172.21.15.120:0/1667789762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcfc8106b80 0x7fcfc8106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.874+0000 7fcfcfd14640 1 -- 172.21.15.120:0/1667789762 >> 172.21.15.120:0/1667789762 conn(0x7fcfc8077230 msgr2=0x7fcfc80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:38.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.874+0000 7fcfcfd14640 1 -- 172.21.15.120:0/1667789762 shutdown_connections 2024-09-19T09:16:38.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.874+0000 7fcfcfd14640 1 -- 172.21.15.120:0/1667789762 wait complete. 2024-09-19T09:16:38.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.875+0000 7fcfcfd14640 1 Processor -- start 2024-09-19T09:16:38.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.875+0000 7fcfcfd14640 1 -- start start 2024-09-19T09:16:38.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.875+0000 7fcfcfd14640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcfc8100100 0x7fcfc81984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:38.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.875+0000 7fcfcfd14640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcfc8100aa0 0x7fcfc8198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:38.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.875+0000 7fcfcfd14640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfc8106b80 0x7fcfc819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:38.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.875+0000 7fcfcfd14640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc8112050 con 0x7fcfc8100aa0 2024-09-19T09:16:38.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.875+0000 7fcfcfd14640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc8111ed0 con 0x7fcfc8100100 2024-09-19T09:16:38.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.875+0000 7fcfcfd14640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc81121d0 con 0x7fcfc8106b80 2024-09-19T09:16:38.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.875+0000 7fcfcd2af640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcfc8100aa0 0x7fcfc8198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:38.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.875+0000 7fcfcdab0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcfc8100100 0x7fcfc81984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:38.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.875+0000 7fcfcdab0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcfc8100100 0x7fcfc81984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:50780/0 (socket says 172.21.15.120:50780) 2024-09-19T09:16:38.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.875+0000 7fcfce2b1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfc8106b80 0x7fcfc819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:38.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.875+0000 7fcfcdab0640 1 -- 172.21.15.120:0/200855528 learned_addr learned my addr 172.21.15.120:0/200855528 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:38.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.876+0000 7fcfce2b1640 1 -- 172.21.15.120:0/200855528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcfc8100100 msgr2=0x7fcfc81984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:38.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.876+0000 7fcfce2b1640 1 --2- 172.21.15.120:0/200855528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcfc8100100 0x7fcfc81984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.876+0000 7fcfce2b1640 1 -- 172.21.15.120:0/200855528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcfc8100aa0 msgr2=0x7fcfc8198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:38.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.876+0000 7fcfce2b1640 1 --2- 172.21.15.120:0/200855528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcfc8100aa0 0x7fcfc8198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.876+0000 7fcfce2b1640 1 -- 172.21.15.120:0/200855528 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc819d480 con 0x7fcfc8106b80 2024-09-19T09:16:38.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.876+0000 7fcfcdab0640 1 --2- 172.21.15.120:0/200855528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcfc8100100 0x7fcfc81984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:16:38.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.876+0000 7fcfce2b1640 1 --2- 172.21.15.120:0/200855528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfc8106b80 0x7fcfc819cda0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcfc002ade0 tx=0x7fcfc00319f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:38.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.876+0000 7fcfbeffd640 1 -- 172.21.15.120:0/200855528 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0003b70 con 0x7fcfc8106b80 2024-09-19T09:16:38.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.876+0000 7fcfbeffd640 1 -- 172.21.15.120:0/200855528 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc0003d10 con 0x7fcfc8106b80 2024-09-19T09:16:38.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.876+0000 7fcfcfd14640 1 -- 172.21.15.120:0/200855528 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc819d710 con 0x7fcfc8106b80 2024-09-19T09:16:38.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.876+0000 7fcfbeffd640 1 -- 172.21.15.120:0/200855528 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc002fc10 con 0x7fcfc8106b80 2024-09-19T09:16:38.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.876+0000 7fcfcfd14640 1 -- 172.21.15.120:0/200855528 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc819dbe0 con 0x7fcfc8106b80 2024-09-19T09:16:38.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.880+0000 7fcfbeffd640 1 -- 172.21.15.120:0/200855528 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fcfc0004050 con 0x7fcfc8106b80 2024-09-19T09:16:38.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.880+0000 7fcfd20ed740 1 Processor -- start 2024-09-19T09:16:38.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.880+0000 7fcfd20ed740 1 -- start start 2024-09-19T09:16:38.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.880+0000 7fcfd20ed740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ad25db40 0x5589ad25c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:38.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.880+0000 7fcfd20ed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589ad25c5d0 0x5589ad25ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:38.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.880+0000 7fcfd20ed740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ad267370 0x5589ad269760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:38.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.880+0000 7fcfd20ed740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5589ad1c22f0 con 0x5589ad267370 2024-09-19T09:16:38.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.880+0000 7fcfd20ed740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5589ad169cf0 con 0x5589ad25db40 2024-09-19T09:16:38.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.881+0000 7fcfd20ed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5589ad1be2b0 con 0x5589ad25c5d0 2024-09-19T09:16:38.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.881+0000 7fcfcd2af640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ad25db40 0x5589ad25c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:38.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.881+0000 7fcfcdab0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589ad25c5d0 0x5589ad25ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:38.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.881+0000 7fcfcd2af640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ad25db40 0x5589ad25c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:50784/0 (socket says 172.21.15.120:50784) 2024-09-19T09:16:38.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.881+0000 7fcfcd2af640 1 -- 172.21.15.120:0/700646036 learned_addr learned my addr 172.21.15.120:0/700646036 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:38.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.881+0000 7fcfcd2af640 1 --2- 172.21.15.120:0/700646036 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ad267370 0x5589ad269760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:38.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.881+0000 7fcfbeffd640 1 --2- 172.21.15.120:0/200855528 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcfb40a32c0 0x7fcfb40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:38.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.881+0000 7fcfcdab0640 1 -- 172.21.15.120:0/700646036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ad25db40 msgr2=0x5589ad25c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:38.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.881+0000 7fcfcdab0640 1 --2- 172.21.15.120:0/700646036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ad25db40 0x5589ad25c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.881+0000 7fcfcdab0640 1 -- 172.21.15.120:0/700646036 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ad267370 msgr2=0x5589ad269760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:38.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.881+0000 7fcfcdab0640 1 --2- 172.21.15.120:0/700646036 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ad267370 0x5589ad269760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.881+0000 7fcfcdab0640 1 -- 172.21.15.120:0/700646036 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589ad14e3a0 con 0x5589ad25c5d0 2024-09-19T09:16:38.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.881+0000 7fcfce2b1640 1 --2- 172.21.15.120:0/200855528 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcfb40a32c0 0x7fcfb40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:38.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.881+0000 7fcfbeffd640 1 -- 172.21.15.120:0/200855528 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fcfc00f0950 con 0x7fcfc8106b80 2024-09-19T09:16:38.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.881+0000 7fcfcdab0640 1 --2- 172.21.15.120:0/700646036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589ad25c5d0 0x5589ad25ca10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcfc40096c0 tx=0x7fcfc4005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c4e1ae8e659c382 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:38.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.882+0000 7fcfce2b1640 1 --2- 172.21.15.120:0/200855528 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcfb40a32c0 0x7fcfb40a5780 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009ca0 tx=0x7fcfc0087bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:38.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.882+0000 7fcfb3fff640 1 -- 172.21.15.120:0/700646036 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc401e020 con 0x5589ad25c5d0 2024-09-19T09:16:38.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.882+0000 7fcfb3fff640 1 -- 172.21.15.120:0/700646036 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc400bce0 con 0x5589ad25c5d0 2024-09-19T09:16:38.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.882+0000 7fcfb3fff640 1 -- 172.21.15.120:0/700646036 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc401ad80 con 0x5589ad25c5d0 2024-09-19T09:16:38.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.883+0000 7fcfd20ed740 1 -- 172.21.15.120:0/700646036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589ad25c5d0 msgr2=0x5589ad25ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:38.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.883+0000 7fcfd20ed740 1 --2- 172.21.15.120:0/700646036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589ad25c5d0 0x5589ad25ca10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcfc40096c0 tx=0x7fcfc4005f90 comp rx=0 tx=0).stop 2024-09-19T09:16:38.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.883+0000 7fcfd20ed740 1 -- 172.21.15.120:0/700646036 shutdown_connections 2024-09-19T09:16:38.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.883+0000 7fcfd20ed740 1 --2- 172.21.15.120:0/700646036 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ad267370 0x5589ad269760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.883+0000 7fcfd20ed740 1 --2- 172.21.15.120:0/700646036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589ad25c5d0 0x5589ad25ca10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.883+0000 7fcfd20ed740 1 --2- 172.21.15.120:0/700646036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ad25db40 0x5589ad25c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.883+0000 7fcfd20ed740 1 -- 172.21.15.120:0/700646036 >> 172.21.15.120:0/700646036 conn(0x5589ad225310 msgr2=0x5589ad258c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:38.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.883+0000 7fcfd20ed740 1 -- 172.21.15.120:0/700646036 shutdown_connections 2024-09-19T09:16:38.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.883+0000 7fcfd20ed740 1 -- 172.21.15.120:0/700646036 wait complete. 2024-09-19T09:16:38.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.883+0000 7fcfd20ed740 1 -- start start 2024-09-19T09:16:38.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.883+0000 7fcfd20ed740 1 Processor -- start 2024-09-19T09:16:38.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.884+0000 7fcfd20ed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589ad25c5d0 0x5589ad266d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:38.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.884+0000 7fcfd20ed740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ad25db40 0x5589ad263fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:38.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.884+0000 7fcfd20ed740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ad267370 0x5589ad264530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:38.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.884+0000 7fcfd20ed740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5589ad26a070 con 0x5589ad267370 2024-09-19T09:16:38.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.884+0000 7fcfd20ed740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5589ad17d060 con 0x5589ad25db40 2024-09-19T09:16:38.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.884+0000 7fcfd20ed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5589ad26a1f0 con 0x5589ad25c5d0 2024-09-19T09:16:38.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.884+0000 7fcfd20ed740 10 client.0 _ll_register_callbacks cb 0x5589aced08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:38.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.884+0000 7fcfcd2af640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589ad25c5d0 0x5589ad266d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:38.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.884+0000 7fcfcd2af640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589ad25c5d0 0x5589ad266d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:33618/0 (socket says 172.21.15.120:33618) 2024-09-19T09:16:38.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.884+0000 7fcfcd2af640 1 -- 172.21.15.120:0/2895517172 learned_addr learned my addr 172.21.15.120:0/2895517172 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:38.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.884+0000 7fcfcd2af640 1 --2- 172.21.15.120:0/2895517172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ad267370 0x5589ad264530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:38.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.884+0000 7fcfcdab0640 1 --2- 172.21.15.120:0/2895517172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ad25db40 0x5589ad263fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:38.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.884+0000 7fcfcd2af640 1 -- 172.21.15.120:0/2895517172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ad25db40 msgr2=0x5589ad263fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:38.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.884+0000 7fcfcd2af640 1 --2- 172.21.15.120:0/2895517172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ad25db40 0x5589ad263fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.884+0000 7fcfcd2af640 1 -- 172.21.15.120:0/2895517172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ad267370 msgr2=0x5589ad264530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:38.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.884+0000 7fcfcd2af640 1 --2- 172.21.15.120:0/2895517172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ad267370 0x5589ad264530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.884+0000 7fcfcd2af640 1 -- 172.21.15.120:0/2895517172 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589acf333c0 con 0x5589ad25c5d0 2024-09-19T09:16:38.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.885+0000 7fcfcd2af640 1 --2- 172.21.15.120:0/2895517172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589ad25c5d0 0x5589ad266d00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8011e70 tx=0x7fcfb80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87d6fa9c46649bf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:38.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.885+0000 7fcfb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:16:38.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.885+0000 7fcfb17fa640 1 -- 172.21.15.120:0/2895517172 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8011320 con 0x5589ad25c5d0 2024-09-19T09:16:38.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.885+0000 7fcfb17fa640 1 -- 172.21.15.120:0/2895517172 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fcfb8009040 con 0x5589ad25c5d0 2024-09-19T09:16:38.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.885+0000 7fcfd20ed740 1 -- 172.21.15.120:0/2895517172 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfc400bce0 con 0x5589ad25c5d0 2024-09-19T09:16:38.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.885+0000 7fcfb17fa640 1 -- 172.21.15.120:0/2895517172 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcfb8029050 con 0x5589ad25c5d0 2024-09-19T09:16:38.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.885+0000 7fcfd20ed740 10 client.29017 fetch_fsmap learned FSMap version 36 2024-09-19T09:16:38.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.885+0000 7fcfd20ed740 1 -- 172.21.15.120:0/2895517172 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5589ad275aa0 con 0x5589ad25c5d0 2024-09-19T09:16:38.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.885+0000 7fcfb17fa640 1 -- 172.21.15.120:0/2895517172 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc400bce0 con 0x5589ad25c5d0 2024-09-19T09:16:38.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.885+0000 7fcfb17fa640 1 -- 172.21.15.120:0/2895517172 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fcfb8020040 con 0x5589ad25c5d0 2024-09-19T09:16:38.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.886+0000 7fcfd20ed740 10 client.29017 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:16:38.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.886+0000 7fcfd20ed740 10 client.29017 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:16:38.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.886+0000 7fcfd20ed740 20 client.29017 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:38.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.886+0000 7fcfd20ed740 1 --2- 172.21.15.120:0/2895517172 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5589ad276eb0 0x5589ad2792c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:38.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.886+0000 7fcfd20ed740 4 client.29017 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:16:38.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.886+0000 7fcfd20ed740 1 -- 172.21.15.120:0/2895517172 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589ad1cf860 con 0x5589ad276eb0 2024-09-19T09:16:38.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.886+0000 7fcfcdab0640 1 --2- 172.21.15.120:0/2895517172 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5589ad276eb0 0x5589ad2792c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:38.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.887+0000 7fcfcdab0640 1 --2- 172.21.15.120:0/2895517172 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5589ad276eb0 0x5589ad2792c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a6a09b6ad9f72464 server_cookie=bf78ca75a887b9b4 in_seq=0 out_seq=0 2024-09-19T09:16:38.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.887+0000 7fcfb17fa640 10 client.29017 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:16:38.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.887+0000 7fcfb17fa640 1 -- 172.21.15.120:0/2895517172 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5589ad1cf860 con 0x5589ad276eb0 2024-09-19T09:16:38.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.887+0000 7fcfb17fa640 10 client.29017 handle_command_reply: tid=0 2024-09-19T09:16:38.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.887+0000 7fcfd20ed740 1 client.29017 shutdown 2024-09-19T09:16:38.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.888+0000 7fcfd20ed740 1 -- 172.21.15.120:0/2895517172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589ad25c5d0 msgr2=0x5589ad266d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:38.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.888+0000 7fcfd20ed740 1 --2- 172.21.15.120:0/2895517172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589ad25c5d0 0x5589ad266d00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8011e70 tx=0x7fcfb80046c0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.888+0000 7fcfd20ed740 1 -- 172.21.15.120:0/2895517172 shutdown_connections 2024-09-19T09:16:38.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.888+0000 7fcfd20ed740 1 --2- 172.21.15.120:0/2895517172 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5589ad276eb0 0x5589ad2792c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.888+0000 7fcfb17fa640 0 client.29017 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:16:38.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.888+0000 7fcfd20ed740 1 --2- 172.21.15.120:0/2895517172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ad267370 0x5589ad264530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.888+0000 7fcfd20ed740 1 --2- 172.21.15.120:0/2895517172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ad25db40 0x5589ad263fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.888+0000 7fcfd20ed740 1 --2- 172.21.15.120:0/2895517172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589ad25c5d0 0x5589ad266d00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.888+0000 7fcfd20ed740 1 -- 172.21.15.120:0/2895517172 >> 172.21.15.120:0/2895517172 conn(0x5589ad225310 msgr2=0x5589ad25e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:38.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.888+0000 7fcfd20ed740 1 -- 172.21.15.120:0/2895517172 shutdown_connections 2024-09-19T09:16:38.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.888+0000 7fcfd20ed740 1 -- 172.21.15.120:0/2895517172 wait complete. 2024-09-19T09:16:38.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.888+0000 7fcfd20ed740 20 client.29017 trim_cache size 0 max 16384 2024-09-19T09:16:38.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.903+0000 7fcfd20ed740 1 Processor -- start 2024-09-19T09:16:38.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.903+0000 7fcfd20ed740 1 -- start start 2024-09-19T09:16:38.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.903+0000 7fcfd20ed740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ad25c5d0 0x5589ad257bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:38.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.903+0000 7fcfd20ed740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ad25db40 0x5589ad268760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:38.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.903+0000 7fcfd20ed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589ad267370 0x5589ad268ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:38.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.903+0000 7fcfd20ed740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc81ab380 con 0x5589ad25c5d0 2024-09-19T09:16:38.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.903+0000 7fcfd20ed740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc81acf50 con 0x5589ad25db40 2024-09-19T09:16:38.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.903+0000 7fcfd20ed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc81aba50 con 0x5589ad267370 2024-09-19T09:16:38.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.903+0000 7fcfcd2af640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ad25c5d0 0x5589ad257bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:38.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.903+0000 7fcfcd2af640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ad25c5d0 0x5589ad257bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52442/0 (socket says 172.21.15.120:52442) 2024-09-19T09:16:38.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.903+0000 7fcfcd2af640 1 -- 172.21.15.120:0/2441000664 learned_addr learned my addr 172.21.15.120:0/2441000664 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:38.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.903+0000 7fcfcdab0640 1 --2- 172.21.15.120:0/2441000664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ad25db40 0x5589ad268760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:38.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.904+0000 7fcfcd2af640 1 --2- 172.21.15.120:0/2441000664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589ad267370 0x5589ad268ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:38.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.904+0000 7fcfcdab0640 1 -- 172.21.15.120:0/2441000664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589ad267370 msgr2=0x5589ad268ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:38.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.904+0000 7fcfcdab0640 1 --2- 172.21.15.120:0/2441000664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589ad267370 0x5589ad268ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.904+0000 7fcfcdab0640 1 -- 172.21.15.120:0/2441000664 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ad25c5d0 msgr2=0x5589ad257bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:38.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.904+0000 7fcfcdab0640 1 --2- 172.21.15.120:0/2441000664 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ad25c5d0 0x5589ad257bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.904+0000 7fcfcdab0640 1 -- 172.21.15.120:0/2441000664 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589ad269ba0 con 0x5589ad25db40 2024-09-19T09:16:38.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.904+0000 7fcfcd2af640 1 --2- 172.21.15.120:0/2441000664 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ad25c5d0 0x5589ad257bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:16:38.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.904+0000 7fcfcdab0640 1 --2- 172.21.15.120:0/2441000664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ad25db40 0x5589ad268760 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4000c00 tx=0x7fcfc401a960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90e895a4e7d110e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:38.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.904+0000 7fcfb37fe640 1 -- 172.21.15.120:0/2441000664 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc401e040 con 0x5589ad25db40 2024-09-19T09:16:38.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.904+0000 7fcfb37fe640 1 -- 172.21.15.120:0/2441000664 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc4005040 con 0x5589ad25db40 2024-09-19T09:16:38.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.904+0000 7fcfb37fe640 1 -- 172.21.15.120:0/2441000664 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc400b840 con 0x5589ad25db40 2024-09-19T09:16:38.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.905+0000 7fcfd20ed740 1 -- 172.21.15.120:0/2441000664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ad25db40 msgr2=0x5589ad268760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:38.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.905+0000 7fcfd20ed740 1 --2- 172.21.15.120:0/2441000664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ad25db40 0x5589ad268760 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4000c00 tx=0x7fcfc401a960 comp rx=0 tx=0).stop 2024-09-19T09:16:38.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.905+0000 7fcfd20ed740 1 -- 172.21.15.120:0/2441000664 shutdown_connections 2024-09-19T09:16:38.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.905+0000 7fcfd20ed740 1 --2- 172.21.15.120:0/2441000664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589ad267370 0x5589ad268ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.905+0000 7fcfd20ed740 1 --2- 172.21.15.120:0/2441000664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ad25db40 0x5589ad268760 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.905+0000 7fcfd20ed740 1 --2- 172.21.15.120:0/2441000664 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ad25c5d0 0x5589ad257bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.905+0000 7fcfd20ed740 1 -- 172.21.15.120:0/2441000664 >> 172.21.15.120:0/2441000664 conn(0x5589ad225310 msgr2=0x5589ad2574e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:38.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.905+0000 7fcfd20ed740 1 -- 172.21.15.120:0/2441000664 shutdown_connections 2024-09-19T09:16:38.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.905+0000 7fcfd20ed740 1 -- 172.21.15.120:0/2441000664 wait complete. 2024-09-19T09:16:38.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.906+0000 7fcfd20ed740 1 -- start start 2024-09-19T09:16:38.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.906+0000 7fcfd20ed740 1 Processor -- start 2024-09-19T09:16:38.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.906+0000 7fcfd20ed740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ad25c5d0 0x5589ad26f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:38.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.906+0000 7fcfd20ed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589ad25db40 0x5589ad26f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:38.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.906+0000 7fcfd20ed740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ad267370 0x5589ad26a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:38.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.906+0000 7fcfd20ed740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc81ac880 con 0x5589ad25c5d0 2024-09-19T09:16:38.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.906+0000 7fcfd20ed740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc81ac120 con 0x5589ad267370 2024-09-19T09:16:38.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.906+0000 7fcfd20ed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc81a9340 con 0x5589ad25db40 2024-09-19T09:16:38.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.907+0000 7fcfcd2af640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ad25c5d0 0x5589ad26f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:38.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.907+0000 7fcfd20ed740 10 client.0 _ll_register_callbacks cb 0x7fcfb8009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:38.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.907+0000 7fcfcdab0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589ad25db40 0x5589ad26f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:38.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.907+0000 7fcfcd2af640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ad25c5d0 0x5589ad26f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52458/0 (socket says 172.21.15.120:52458) 2024-09-19T09:16:38.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.907+0000 7fcfcd2af640 1 -- 172.21.15.120:0/1431036105 learned_addr learned my addr 172.21.15.120:0/1431036105 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:38.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.907+0000 7fcfcd2af640 1 --2- 172.21.15.120:0/1431036105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ad267370 0x5589ad26a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:38.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.907+0000 7fcfcdab0640 1 -- 172.21.15.120:0/1431036105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ad267370 msgr2=0x5589ad26a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:38.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.907+0000 7fcfcdab0640 1 --2- 172.21.15.120:0/1431036105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ad267370 0x5589ad26a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.907+0000 7fcfcdab0640 1 -- 172.21.15.120:0/1431036105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ad25c5d0 msgr2=0x5589ad26f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:38.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.907+0000 7fcfcdab0640 1 --2- 172.21.15.120:0/1431036105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ad25c5d0 0x5589ad26f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.907+0000 7fcfcdab0640 1 -- 172.21.15.120:0/1431036105 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589ad26a980 con 0x5589ad25db40 2024-09-19T09:16:38.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.907+0000 7fcfcdab0640 1 --2- 172.21.15.120:0/1431036105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589ad25db40 0x5589ad26f9d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009250 tx=0x7fcfc401a400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54d40bf4076aae24 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:38.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.907+0000 7fcfb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:16:38.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.907+0000 7fcfb17fa640 1 -- 172.21.15.120:0/1431036105 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc403f060 con 0x5589ad25db40 2024-09-19T09:16:38.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.908+0000 7fcfb17fa640 1 -- 172.21.15.120:0/1431036105 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fcfc40051f0 con 0x5589ad25db40 2024-09-19T09:16:38.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.908+0000 7fcfd20ed740 1 -- 172.21.15.120:0/1431036105 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfc4005040 con 0x5589ad25db40 2024-09-19T09:16:38.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.908+0000 7fcfb17fa640 1 -- 172.21.15.120:0/1431036105 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcfc401e040 con 0x5589ad25db40 2024-09-19T09:16:38.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.908+0000 7fcfd20ed740 10 client.29023 fetch_fsmap learned FSMap version 36 2024-09-19T09:16:38.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.908+0000 7fcfd20ed740 1 -- 172.21.15.120:0/1431036105 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5589ad2e2240 con 0x5589ad25db40 2024-09-19T09:16:38.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.908+0000 7fcfb17fa640 1 -- 172.21.15.120:0/1431036105 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc4005040 con 0x5589ad25db40 2024-09-19T09:16:38.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.909+0000 7fcfb17fa640 1 -- 172.21.15.120:0/1431036105 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fcfc40035f0 con 0x5589ad25db40 2024-09-19T09:16:38.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.909+0000 7fcfd20ed740 10 client.29023 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:16:38.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.909+0000 7fcfd20ed740 10 client.29023 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:16:38.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.909+0000 7fcfd20ed740 20 client.29023 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:38.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.909+0000 7fcfd20ed740 1 --2- 172.21.15.120:0/1431036105 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5589ad276eb0 0x5589ad2e46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:38.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.909+0000 7fcfd20ed740 4 client.29023 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:16:38.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.909+0000 7fcfd20ed740 1 -- 172.21.15.120:0/1431036105 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5589ad1c9d70 con 0x5589ad276eb0 2024-09-19T09:16:38.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.909+0000 7fcfcd2af640 1 --2- 172.21.15.120:0/1431036105 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5589ad276eb0 0x5589ad2e46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:38.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.909+0000 7fcfb17fa640 10 client.29023 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:16:38.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.909+0000 7fcfcd2af640 1 --2- 172.21.15.120:0/1431036105 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5589ad276eb0 0x5589ad2e46c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d22bf1da3d7a20bc server_cookie=8350f282e1813376 in_seq=0 out_seq=0 2024-09-19T09:16:38.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.910+0000 7fcfb17fa640 1 -- 172.21.15.120:0/1431036105 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5589ad1c9d70 con 0x5589ad276eb0 2024-09-19T09:16:38.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.910+0000 7fcfb17fa640 10 client.29023 handle_command_reply: tid=0 2024-09-19T09:16:38.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.910+0000 7fcfd20ed740 1 client.29023 shutdown 2024-09-19T09:16:38.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.910+0000 7fcfd20ed740 1 -- 172.21.15.120:0/1431036105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589ad25db40 msgr2=0x5589ad26f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:38.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.910+0000 7fcfd20ed740 1 --2- 172.21.15.120:0/1431036105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589ad25db40 0x5589ad26f9d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009250 tx=0x7fcfc401a400 comp rx=0 tx=0).stop 2024-09-19T09:16:38.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.910+0000 7fcfd20ed740 1 -- 172.21.15.120:0/1431036105 shutdown_connections 2024-09-19T09:16:38.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.910+0000 7fcfd20ed740 1 --2- 172.21.15.120:0/1431036105 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5589ad276eb0 0x5589ad2e46c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.910+0000 7fcfd20ed740 1 --2- 172.21.15.120:0/1431036105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ad267370 0x5589ad26a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.910+0000 7fcfd20ed740 1 --2- 172.21.15.120:0/1431036105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589ad25db40 0x5589ad26f9d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.910+0000 7fcfd20ed740 1 --2- 172.21.15.120:0/1431036105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ad25c5d0 0x5589ad26f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.910+0000 7fcfd20ed740 1 -- 172.21.15.120:0/1431036105 >> 172.21.15.120:0/1431036105 conn(0x5589ad225310 msgr2=0x5589ad257d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:38.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.910+0000 7fcfb17fa640 0 client.29023 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:16:38.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.910+0000 7fcfd20ed740 1 -- 172.21.15.120:0/1431036105 shutdown_connections 2024-09-19T09:16:38.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.910+0000 7fcfd20ed740 1 -- 172.21.15.120:0/1431036105 wait complete. 2024-09-19T09:16:38.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.910+0000 7fcfd20ed740 20 client.29023 trim_cache size 0 max 16384 2024-09-19T09:16:38.919 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:16:38.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.911+0000 7fcfcfd14640 1 -- 172.21.15.120:0/200855528 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcfb40a32c0 msgr2=0x7fcfb40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:38.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.911+0000 7fcfcfd14640 1 --2- 172.21.15.120:0/200855528 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcfb40a32c0 0x7fcfb40a5780 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009ca0 tx=0x7fcfc0087bf0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.911+0000 7fcfcfd14640 1 -- 172.21.15.120:0/200855528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfc8106b80 msgr2=0x7fcfc819cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:38.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.911+0000 7fcfcfd14640 1 --2- 172.21.15.120:0/200855528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfc8106b80 0x7fcfc819cda0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcfc002ade0 tx=0x7fcfc00319f0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.911+0000 7fcfcfd14640 1 -- 172.21.15.120:0/200855528 shutdown_connections 2024-09-19T09:16:38.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.911+0000 7fcfcfd14640 1 --2- 172.21.15.120:0/200855528 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcfb40a32c0 0x7fcfb40a5780 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.911+0000 7fcfcfd14640 1 --2- 172.21.15.120:0/200855528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfc8106b80 0x7fcfc819cda0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.911+0000 7fcfcfd14640 1 --2- 172.21.15.120:0/200855528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcfc8100aa0 0x7fcfc8198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.911+0000 7fcfcfd14640 1 --2- 172.21.15.120:0/200855528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcfc8100100 0x7fcfc81984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:38.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.911+0000 7fcfcfd14640 1 -- 172.21.15.120:0/200855528 >> 172.21.15.120:0/200855528 conn(0x7fcfc8077230 msgr2=0x7fcfc80fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:38.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.912+0000 7fcfcfd14640 1 -- 172.21.15.120:0/200855528 shutdown_connections 2024-09-19T09:16:38.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:38.912+0000 7fcfcfd14640 1 -- 172.21.15.120:0/200855528 wait complete. 2024-09-19T09:16:39.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:38 smithi120 ceph-mon[31150]: pgmap v758: 97 pgs: 97 active+clean; 197 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 20 MiB/s wr, 346 op/s 2024-09-19T09:16:39.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:38 smithi196 ceph-mon[35150]: pgmap v758: 97 pgs: 97 active+clean; 197 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 20 MiB/s wr, 346 op/s 2024-09-19T09:16:39.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:38 smithi153 ceph-mon[35071]: pgmap v758: 97 pgs: 97 active+clean; 197 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 20 MiB/s wr, 346 op/s 2024-09-19T09:16:39.352 INFO:tasks.workunit.client.0.smithi120.stdout: 10 6425 15.33 MB/sec warmup 34 sec latency 52.741 ms 2024-09-19T09:16:39.611 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:16:39.878 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:16:40.353 INFO:tasks.workunit.client.0.smithi120.stdout: 10 7405 15.90 MB/sec warmup 35 sec latency 78.572 ms 2024-09-19T09:16:40.755 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:16:40.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.431+0000 7f8a42e82640 1 -- 172.21.15.120:0/391645632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a3c101400 msgr2=0x7f8a3c10d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:40.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.431+0000 7f8a42e82640 1 --2- 172.21.15.120:0/391645632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a3c101400 0x7f8a3c10d930 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f8a34009cd0 tx=0x7f8a3402a8e0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.431+0000 7f8a42e82640 1 -- 172.21.15.120:0/391645632 shutdown_connections 2024-09-19T09:16:40.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.431+0000 7f8a42e82640 1 --2- 172.21.15.120:0/391645632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a3c101400 0x7f8a3c10d930 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.431+0000 7f8a42e82640 1 --2- 172.21.15.120:0/391645632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a3c100a60 0x7f8a3c100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.431+0000 7f8a42e82640 1 --2- 172.21.15.120:0/391645632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a3c106a60 0x7f8a3c106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.431+0000 7f8a42e82640 1 -- 172.21.15.120:0/391645632 >> 172.21.15.120:0/391645632 conn(0x7f8a3c0fc780 msgr2=0x7f8a3c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:40.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.431+0000 7f8a42e82640 1 -- 172.21.15.120:0/391645632 shutdown_connections 2024-09-19T09:16:40.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.431+0000 7f8a42e82640 1 -- 172.21.15.120:0/391645632 wait complete. 2024-09-19T09:16:40.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.432+0000 7f8a42e82640 1 Processor -- start 2024-09-19T09:16:40.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.432+0000 7f8a42e82640 1 -- start start 2024-09-19T09:16:40.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.432+0000 7f8a42e82640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a3c100a60 0x7f8a3c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:40.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.432+0000 7f8a42e82640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a3c101400 0x7f8a3c199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:40.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.432+0000 7f8a42e82640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a3c106a60 0x7f8a3c1a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:40.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.432+0000 7f8a42e82640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c110e60 con 0x7f8a3c106a60 2024-09-19T09:16:40.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.432+0000 7f8a42e82640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c110ce0 con 0x7f8a3c101400 2024-09-19T09:16:40.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.432+0000 7f8a42e82640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c110fe0 con 0x7f8a3c100a60 2024-09-19T09:16:40.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.432+0000 7f8a40c1e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a3c100a60 0x7f8a3c1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:40.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.432+0000 7f8a4141f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a3c106a60 0x7f8a3c1a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:40.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.432+0000 7f8a40c1e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a3c100a60 0x7f8a3c1992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:33672/0 (socket says 172.21.15.120:33672) 2024-09-19T09:16:40.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.432+0000 7f8a40c1e640 1 -- 172.21.15.120:0/950104266 learned_addr learned my addr 172.21.15.120:0/950104266 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:40.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.432+0000 7f8a3bfff640 1 --2- 172.21.15.120:0/950104266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a3c101400 0x7f8a3c199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:40.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.432+0000 7f8a40c1e640 1 -- 172.21.15.120:0/950104266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a3c101400 msgr2=0x7f8a3c199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:40.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.432+0000 7f8a40c1e640 1 --2- 172.21.15.120:0/950104266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a3c101400 0x7f8a3c199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.432+0000 7f8a40c1e640 1 -- 172.21.15.120:0/950104266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a3c106a60 msgr2=0x7f8a3c1a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:40.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.432+0000 7f8a40c1e640 1 --2- 172.21.15.120:0/950104266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a3c106a60 0x7f8a3c1a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.432+0000 7f8a40c1e640 1 -- 172.21.15.120:0/950104266 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3c1a48b0 con 0x7f8a3c100a60 2024-09-19T09:16:40.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.432+0000 7f8a4141f640 1 --2- 172.21.15.120:0/950104266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a3c106a60 0x7f8a3c1a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:16:40.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.432+0000 7f8a40c1e640 1 --2- 172.21.15.120:0/950104266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a3c100a60 0x7f8a3c1992f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8a3000e9f0 tx=0x7f8a3000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:40.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.433+0000 7f8a39ffb640 1 -- 172.21.15.120:0/950104266 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3000cd80 con 0x7f8a3c100a60 2024-09-19T09:16:40.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.433+0000 7f8a42e82640 1 -- 172.21.15.120:0/950104266 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a3c1a4a80 con 0x7f8a3c100a60 2024-09-19T09:16:40.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.433+0000 7f8a39ffb640 1 -- 172.21.15.120:0/950104266 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a30005500 con 0x7f8a3c100a60 2024-09-19T09:16:40.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.433+0000 7f8a39ffb640 1 -- 172.21.15.120:0/950104266 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30010690 con 0x7f8a3c100a60 2024-09-19T09:16:40.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.433+0000 7f8a42e82640 1 -- 172.21.15.120:0/950104266 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a3c1a4f40 con 0x7f8a3c100a60 2024-09-19T09:16:40.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.435+0000 7f8a39ffb640 1 -- 172.21.15.120:0/950104266 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f8a300046c0 con 0x7f8a3c100a60 2024-09-19T09:16:40.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.435+0000 7f8a4525b740 1 Processor -- start 2024-09-19T09:16:40.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.435+0000 7f8a4525b740 1 -- start start 2024-09-19T09:16:40.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.436+0000 7f8a39ffb640 1 --2- 172.21.15.120:0/950104266 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a2c0a3240 0x7f8a2c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:40.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.436+0000 7f8a4525b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e0b4a4b40 0x562e0b4a3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:40.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.436+0000 7f8a4525b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e0b4a35d0 0x562e0b4a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:40.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.436+0000 7f8a4525b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e0b4b2780 0x562e0b4b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:40.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.436+0000 7f8a4525b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562e0b4092f0 con 0x562e0b4a35d0 2024-09-19T09:16:40.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.436+0000 7f8a4525b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562e0b3b0cf0 con 0x562e0b4b2780 2024-09-19T09:16:40.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.436+0000 7f8a4525b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562e0b4052b0 con 0x562e0b4a4b40 2024-09-19T09:16:40.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.436+0000 7f8a4141f640 1 --2- 172.21.15.120:0/950104266 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a2c0a3240 0x7f8a2c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:40.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.436+0000 7f8a3bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e0b4a35d0 0x562e0b4a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:40.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.436+0000 7f8a4141f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e0b4b2780 0x562e0b4b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:40.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.436+0000 7f8a3bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e0b4a35d0 0x562e0b4a3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52480/0 (socket says 172.21.15.120:52480) 2024-09-19T09:16:40.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.436+0000 7f8a40c1e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e0b4a4b40 0x562e0b4a3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:40.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.436+0000 7f8a3bfff640 1 -- 172.21.15.120:0/1217205420 learned_addr learned my addr 172.21.15.120:0/1217205420 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:40.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.436+0000 7f8a39ffb640 1 -- 172.21.15.120:0/950104266 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f8a300cfcc0 con 0x7f8a3c100a60 2024-09-19T09:16:40.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.436+0000 7f8a40c1e640 1 -- 172.21.15.120:0/1217205420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e0b4b2780 msgr2=0x562e0b4b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:40.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.436+0000 7f8a40c1e640 1 --2- 172.21.15.120:0/1217205420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e0b4b2780 0x562e0b4b4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.436+0000 7f8a40c1e640 1 -- 172.21.15.120:0/1217205420 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e0b4a35d0 msgr2=0x562e0b4a3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:40.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.436+0000 7f8a40c1e640 1 --2- 172.21.15.120:0/1217205420 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e0b4a35d0 0x562e0b4a3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.436+0000 7f8a40c1e640 1 -- 172.21.15.120:0/1217205420 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e0b3953a0 con 0x562e0b4a4b40 2024-09-19T09:16:40.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.436+0000 7f8a3bfff640 1 --2- 172.21.15.120:0/1217205420 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e0b4a35d0 0x562e0b4a3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:16:40.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.436+0000 7f8a4141f640 1 --2- 172.21.15.120:0/950104266 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a2c0a3240 0x7f8a2c0a5700 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8a34009a60 tx=0x7f8a34005f80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:40.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.436+0000 7f8a40c1e640 1 --2- 172.21.15.120:0/1217205420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e0b4a4b40 0x562e0b4a3090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8a3009c270 tx=0x7f8a30098640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12b39e5e51f677a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:40.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.437+0000 7f8a26ffd640 1 -- 172.21.15.120:0/1217205420 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3009bc40 con 0x562e0b4a4b40 2024-09-19T09:16:40.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.437+0000 7f8a26ffd640 1 -- 172.21.15.120:0/1217205420 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a300a5e10 con 0x562e0b4a4b40 2024-09-19T09:16:40.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.437+0000 7f8a26ffd640 1 -- 172.21.15.120:0/1217205420 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a300a9960 con 0x562e0b4a4b40 2024-09-19T09:16:40.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.437+0000 7f8a4525b740 1 -- 172.21.15.120:0/1217205420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e0b4a4b40 msgr2=0x562e0b4a3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:40.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.437+0000 7f8a4525b740 1 --2- 172.21.15.120:0/1217205420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e0b4a4b40 0x562e0b4a3090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8a3009c270 tx=0x7f8a30098640 comp rx=0 tx=0).stop 2024-09-19T09:16:40.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.437+0000 7f8a4525b740 1 -- 172.21.15.120:0/1217205420 shutdown_connections 2024-09-19T09:16:40.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.437+0000 7f8a4525b740 1 --2- 172.21.15.120:0/1217205420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e0b4b2780 0x562e0b4b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.437+0000 7f8a4525b740 1 --2- 172.21.15.120:0/1217205420 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e0b4a35d0 0x562e0b4a3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.437+0000 7f8a4525b740 1 --2- 172.21.15.120:0/1217205420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e0b4a4b40 0x562e0b4a3090 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.437+0000 7f8a4525b740 1 -- 172.21.15.120:0/1217205420 >> 172.21.15.120:0/1217205420 conn(0x562e0b46c310 msgr2=0x562e0b49fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:40.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.438+0000 7f8a4525b740 1 -- 172.21.15.120:0/1217205420 shutdown_connections 2024-09-19T09:16:40.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.438+0000 7f8a4525b740 1 -- 172.21.15.120:0/1217205420 wait complete. 2024-09-19T09:16:40.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.438+0000 7f8a4525b740 1 -- start start 2024-09-19T09:16:40.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.438+0000 7f8a4525b740 1 Processor -- start 2024-09-19T09:16:40.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.438+0000 7f8a4525b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e0b4a35d0 0x562e0b4b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:40.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.438+0000 7f8a4525b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e0b4a4b40 0x562e0b4af480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:40.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.438+0000 7f8a4525b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e0b4b2780 0x562e0b4afc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:40.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.438+0000 7f8a4525b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562e0b4b5420 con 0x562e0b4b2780 2024-09-19T09:16:40.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.438+0000 7f8a4525b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562e0b3c4060 con 0x562e0b4a4b40 2024-09-19T09:16:40.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.438+0000 7f8a4525b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562e0b4b55a0 con 0x562e0b4a35d0 2024-09-19T09:16:40.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.438+0000 7f8a4525b740 10 client.0 _ll_register_callbacks cb 0x562e0b1178c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:40.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.439+0000 7f8a3bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e0b4a4b40 0x562e0b4af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:40.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.439+0000 7f8a40c1e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e0b4a35d0 0x562e0b4b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:40.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.439+0000 7f8a3bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e0b4a4b40 0x562e0b4af480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:50848/0 (socket says 172.21.15.120:50848) 2024-09-19T09:16:40.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.439+0000 7f8a3bfff640 1 -- 172.21.15.120:0/1720210469 learned_addr learned my addr 172.21.15.120:0/1720210469 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:40.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.439+0000 7f8a4141f640 1 --2- 172.21.15.120:0/1720210469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e0b4b2780 0x562e0b4afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:40.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.439+0000 7f8a40c1e640 1 -- 172.21.15.120:0/1720210469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e0b4a4b40 msgr2=0x562e0b4af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:40.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.439+0000 7f8a40c1e640 1 --2- 172.21.15.120:0/1720210469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e0b4a4b40 0x562e0b4af480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.439+0000 7f8a40c1e640 1 -- 172.21.15.120:0/1720210469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e0b4b2780 msgr2=0x562e0b4afc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:40.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.439+0000 7f8a40c1e640 1 --2- 172.21.15.120:0/1720210469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e0b4b2780 0x562e0b4afc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.439+0000 7f8a40c1e640 1 -- 172.21.15.120:0/1720210469 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e0b17a3c0 con 0x562e0b4a35d0 2024-09-19T09:16:40.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.439+0000 7f8a3bfff640 1 --2- 172.21.15.120:0/1720210469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e0b4a4b40 0x562e0b4af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:16:40.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.439+0000 7f8a40c1e640 1 --2- 172.21.15.120:0/1720210469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e0b4a35d0 0x562e0b4b21d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8a30098c60 tx=0x7f8a300a0690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddd7128c2eacb628 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:40.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.439+0000 7f8a1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:16:40.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.439+0000 7f8a1bfff640 1 -- 172.21.15.120:0/1720210469 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3009c9e0 con 0x562e0b4a35d0 2024-09-19T09:16:40.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.439+0000 7f8a1bfff640 1 -- 172.21.15.120:0/1720210469 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f8a300ac0b0 con 0x562e0b4a35d0 2024-09-19T09:16:40.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.439+0000 7f8a4525b740 1 -- 172.21.15.120:0/1720210469 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a300a5e10 con 0x562e0b4a35d0 2024-09-19T09:16:40.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.439+0000 7f8a1bfff640 1 -- 172.21.15.120:0/1720210469 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8a300a4f50 con 0x562e0b4a35d0 2024-09-19T09:16:40.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.439+0000 7f8a4525b740 10 client.29035 fetch_fsmap learned FSMap version 36 2024-09-19T09:16:40.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.439+0000 7f8a4525b740 1 -- 172.21.15.120:0/1720210469 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562e0b4b5420 con 0x562e0b4a35d0 2024-09-19T09:16:40.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.440+0000 7f8a1bfff640 1 -- 172.21.15.120:0/1720210469 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a300a5e10 con 0x562e0b4a35d0 2024-09-19T09:16:40.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.440+0000 7f8a1bfff640 1 -- 172.21.15.120:0/1720210469 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f8a3009b7c0 con 0x562e0b4a35d0 2024-09-19T09:16:40.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.440+0000 7f8a4525b740 10 client.29035 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:16:40.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.440+0000 7f8a4525b740 10 client.29035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:16:40.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.440+0000 7f8a4525b740 20 client.29035 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:40.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.440+0000 7f8a4525b740 1 --2- 172.21.15.120:0/1720210469 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562e0b4c20b0 0x562e0b4c44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:40.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.440+0000 7f8a4525b740 4 client.29035 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:16:40.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.440+0000 7f8a4525b740 1 -- 172.21.15.120:0/1720210469 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e0b416860 con 0x562e0b4c20b0 2024-09-19T09:16:40.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.440+0000 7f8a3bfff640 1 --2- 172.21.15.120:0/1720210469 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562e0b4c20b0 0x562e0b4c44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:40.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.440+0000 7f8a1bfff640 10 client.29035 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:40.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.440+0000 7f8a3bfff640 1 --2- 172.21.15.120:0/1720210469 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562e0b4c20b0 0x562e0b4c44a0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbb94b20950633e server_cookie=cf0d105289245438 in_seq=0 out_seq=0 2024-09-19T09:16:40.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.443+0000 7f8a1bfff640 1 -- 172.21.15.120:0/1720210469 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562e0b416860 con 0x562e0b4c20b0 2024-09-19T09:16:40.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.443+0000 7f8a1bfff640 10 client.29035 handle_command_reply: tid=0 2024-09-19T09:16:40.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.443+0000 7f8a4525b740 1 client.29035 shutdown 2024-09-19T09:16:40.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.443+0000 7f8a4525b740 1 -- 172.21.15.120:0/1720210469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e0b4a35d0 msgr2=0x562e0b4b21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:40.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.443+0000 7f8a4525b740 1 --2- 172.21.15.120:0/1720210469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e0b4a35d0 0x562e0b4b21d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8a30098c60 tx=0x7f8a300a0690 comp rx=0 tx=0).stop 2024-09-19T09:16:40.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.443+0000 7f8a4525b740 1 -- 172.21.15.120:0/1720210469 shutdown_connections 2024-09-19T09:16:40.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.443+0000 7f8a4525b740 1 --2- 172.21.15.120:0/1720210469 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562e0b4c20b0 0x562e0b4c44a0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.443+0000 7f8a4525b740 1 --2- 172.21.15.120:0/1720210469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e0b4b2780 0x562e0b4afc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.443+0000 7f8a1bfff640 0 client.29035 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:40.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.443+0000 7f8a4525b740 1 --2- 172.21.15.120:0/1720210469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e0b4a4b40 0x562e0b4af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.443+0000 7f8a4525b740 1 --2- 172.21.15.120:0/1720210469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e0b4a35d0 0x562e0b4b21d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.443+0000 7f8a4525b740 1 -- 172.21.15.120:0/1720210469 >> 172.21.15.120:0/1720210469 conn(0x562e0b46c310 msgr2=0x562e0b4a5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:40.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.443+0000 7f8a4525b740 1 -- 172.21.15.120:0/1720210469 shutdown_connections 2024-09-19T09:16:40.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.443+0000 7f8a4525b740 1 -- 172.21.15.120:0/1720210469 wait complete. 2024-09-19T09:16:40.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.443+0000 7f8a4525b740 20 client.29035 trim_cache size 0 max 16384 2024-09-19T09:16:40.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.457+0000 7f8a4525b740 1 Processor -- start 2024-09-19T09:16:40.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.457+0000 7f8a4525b740 1 -- start start 2024-09-19T09:16:40.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.457+0000 7f8a4525b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e0b4a35d0 0x562e0b49fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:40.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.457+0000 7f8a4525b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e0b4a4b40 0x562e0b4aed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:40.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.457+0000 7f8a4525b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e0b4b2780 0x562e0b4c44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:40.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.457+0000 7f8a4525b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c1ab450 con 0x562e0b4a35d0 2024-09-19T09:16:40.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.457+0000 7f8a4525b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c1ad020 con 0x562e0b4a4b40 2024-09-19T09:16:40.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.457+0000 7f8a4525b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c1abb20 con 0x562e0b4b2780 2024-09-19T09:16:40.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.457+0000 7f8a40c1e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e0b4a35d0 0x562e0b49fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:40.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.458+0000 7f8a40c1e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e0b4a35d0 0x562e0b49fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52506/0 (socket says 172.21.15.120:52506) 2024-09-19T09:16:40.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.458+0000 7f8a40c1e640 1 -- 172.21.15.120:0/297173694 learned_addr learned my addr 172.21.15.120:0/297173694 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:40.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.458+0000 7f8a3bfff640 1 --2- 172.21.15.120:0/297173694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e0b4a4b40 0x562e0b4aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:40.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.458+0000 7f8a4141f640 1 --2- 172.21.15.120:0/297173694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e0b4b2780 0x562e0b4c44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:40.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.458+0000 7f8a40c1e640 1 -- 172.21.15.120:0/297173694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e0b4b2780 msgr2=0x562e0b4c44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:40.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.458+0000 7f8a40c1e640 1 --2- 172.21.15.120:0/297173694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e0b4b2780 0x562e0b4c44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.458+0000 7f8a40c1e640 1 -- 172.21.15.120:0/297173694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e0b4a4b40 msgr2=0x562e0b4aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:40.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.458+0000 7f8a40c1e640 1 --2- 172.21.15.120:0/297173694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e0b4a4b40 0x562e0b4aed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.458+0000 7f8a40c1e640 1 -- 172.21.15.120:0/297173694 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e0b4a0960 con 0x562e0b4a35d0 2024-09-19T09:16:40.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.458+0000 7f8a4141f640 1 --2- 172.21.15.120:0/297173694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e0b4b2780 0x562e0b4c44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:16:40.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.458+0000 7f8a40c1e640 1 --2- 172.21.15.120:0/297173694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e0b4a35d0 0x562e0b49fe10 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f8a30098cb0 tx=0x7f8a30098950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd255fa14cdd545 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:40.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.458+0000 7f8a267fc640 1 -- 172.21.15.120:0/297173694 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3009c9e0 con 0x562e0b4a35d0 2024-09-19T09:16:40.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.458+0000 7f8a267fc640 1 -- 172.21.15.120:0/297173694 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a30096e50 con 0x562e0b4a35d0 2024-09-19T09:16:40.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.458+0000 7f8a267fc640 1 -- 172.21.15.120:0/297173694 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3009b730 con 0x562e0b4a35d0 2024-09-19T09:16:40.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.459+0000 7f8a4525b740 1 -- 172.21.15.120:0/297173694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e0b4a35d0 msgr2=0x562e0b49fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:40.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.459+0000 7f8a4525b740 1 --2- 172.21.15.120:0/297173694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e0b4a35d0 0x562e0b49fe10 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f8a30098cb0 tx=0x7f8a30098950 comp rx=0 tx=0).stop 2024-09-19T09:16:40.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.459+0000 7f8a4525b740 1 -- 172.21.15.120:0/297173694 shutdown_connections 2024-09-19T09:16:40.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.459+0000 7f8a4525b740 1 --2- 172.21.15.120:0/297173694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e0b4b2780 0x562e0b4c44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.459+0000 7f8a4525b740 1 --2- 172.21.15.120:0/297173694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e0b4a4b40 0x562e0b4aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.459+0000 7f8a4525b740 1 --2- 172.21.15.120:0/297173694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e0b4a35d0 0x562e0b49fe10 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.459+0000 7f8a4525b740 1 -- 172.21.15.120:0/297173694 >> 172.21.15.120:0/297173694 conn(0x562e0b46c310 msgr2=0x562e0b49f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:40.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.459+0000 7f8a4525b740 1 -- 172.21.15.120:0/297173694 shutdown_connections 2024-09-19T09:16:40.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.459+0000 7f8a4525b740 1 -- 172.21.15.120:0/297173694 wait complete. 2024-09-19T09:16:40.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.459+0000 7f8a4525b740 1 -- start start 2024-09-19T09:16:40.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.459+0000 7f8a4525b740 1 Processor -- start 2024-09-19T09:16:40.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.460+0000 7f8a4525b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e0b4a35d0 0x562e0b4b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:40.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.460+0000 7f8a4525b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e0b4a4b40 0x562e0b4b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:40.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.460+0000 7f8a4525b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e0b4b2780 0x562e0b52ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:40.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.460+0000 7f8a4525b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c1ac950 con 0x562e0b4b2780 2024-09-19T09:16:40.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.460+0000 7f8a4525b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c1ac1f0 con 0x562e0b4a35d0 2024-09-19T09:16:40.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.460+0000 7f8a4525b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c1a9410 con 0x562e0b4a4b40 2024-09-19T09:16:40.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.460+0000 7f8a4525b740 10 client.0 _ll_register_callbacks cb 0x7f8a280097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:40.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.460+0000 7f8a4141f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e0b4b2780 0x562e0b52ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:40.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.460+0000 7f8a4141f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e0b4b2780 0x562e0b52ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52518/0 (socket says 172.21.15.120:52518) 2024-09-19T09:16:40.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.460+0000 7f8a4141f640 1 -- 172.21.15.120:0/626093644 learned_addr learned my addr 172.21.15.120:0/626093644 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:40.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.460+0000 7f8a3bfff640 1 --2- 172.21.15.120:0/626093644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e0b4a4b40 0x562e0b4b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:40.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.460+0000 7f8a40c1e640 1 --2- 172.21.15.120:0/626093644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e0b4a35d0 0x562e0b4b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:40.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.460+0000 7f8a4141f640 1 -- 172.21.15.120:0/626093644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e0b4a4b40 msgr2=0x562e0b4b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:40.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.460+0000 7f8a4141f640 1 --2- 172.21.15.120:0/626093644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e0b4a4b40 0x562e0b4b62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.460+0000 7f8a4141f640 1 -- 172.21.15.120:0/626093644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e0b4a35d0 msgr2=0x562e0b4b5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:40.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.460+0000 7f8a4141f640 1 --2- 172.21.15.120:0/626093644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e0b4a35d0 0x562e0b4b5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.460+0000 7f8a4141f640 1 -- 172.21.15.120:0/626093644 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e0b49f510 con 0x562e0b4b2780 2024-09-19T09:16:40.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.460+0000 7f8a4141f640 1 --2- 172.21.15.120:0/626093644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e0b4b2780 0x562e0b52ac10 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f8a34009cd0 tx=0x7f8a3402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ada2e37c8eef02 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:40.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.460+0000 7f8a1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:16:40.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.460+0000 7f8a1bfff640 1 -- 172.21.15.120:0/626093644 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34059060 con 0x562e0b4b2780 2024-09-19T09:16:40.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.460+0000 7f8a1bfff640 1 -- 172.21.15.120:0/626093644 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f8a34046040 con 0x562e0b4b2780 2024-09-19T09:16:40.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.460+0000 7f8a4525b740 1 -- 172.21.15.120:0/626093644 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a30096e50 con 0x562e0b4b2780 2024-09-19T09:16:40.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.461+0000 7f8a1bfff640 1 -- 172.21.15.120:0/626093644 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8a3404b7a0 con 0x562e0b4b2780 2024-09-19T09:16:40.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.461+0000 7f8a4525b740 10 client.20262 fetch_fsmap learned FSMap version 36 2024-09-19T09:16:40.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.461+0000 7f8a4525b740 1 -- 172.21.15.120:0/626093644 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562e0b52c340 con 0x562e0b4b2780 2024-09-19T09:16:40.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.461+0000 7f8a1bfff640 1 -- 172.21.15.120:0/626093644 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a30096e50 con 0x562e0b4b2780 2024-09-19T09:16:40.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.461+0000 7f8a1bfff640 1 -- 172.21.15.120:0/626093644 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f8a3404a050 con 0x562e0b4b2780 2024-09-19T09:16:40.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.461+0000 7f8a4525b740 10 client.20262 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:16:40.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.461+0000 7f8a4525b740 10 client.20262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:16:40.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.461+0000 7f8a4525b740 20 client.20262 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:40.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.461+0000 7f8a4525b740 1 --2- 172.21.15.120:0/626093644 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562e0b4c20b0 0x562e0b52eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:40.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.461+0000 7f8a4525b740 4 client.20262 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:16:40.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.461+0000 7f8a4525b740 1 -- 172.21.15.120:0/626093644 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562e0b410d70 con 0x562e0b4c20b0 2024-09-19T09:16:40.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.461+0000 7f8a40c1e640 1 --2- 172.21.15.120:0/626093644 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562e0b4c20b0 0x562e0b52eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:40.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.462+0000 7f8a1bfff640 10 client.20262 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:40.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.462+0000 7f8a40c1e640 1 --2- 172.21.15.120:0/626093644 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562e0b4c20b0 0x562e0b52eb90 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7ce400d2c693cba server_cookie=a40d0caf1945ed0d in_seq=0 out_seq=0 2024-09-19T09:16:40.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.465+0000 7f8a1bfff640 1 -- 172.21.15.120:0/626093644 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562e0b410d70 con 0x562e0b4c20b0 2024-09-19T09:16:40.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.465+0000 7f8a1bfff640 10 client.20262 handle_command_reply: tid=0 2024-09-19T09:16:40.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.465+0000 7f8a4525b740 1 client.20262 shutdown 2024-09-19T09:16:40.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.465+0000 7f8a4525b740 1 -- 172.21.15.120:0/626093644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e0b4b2780 msgr2=0x562e0b52ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:40.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.465+0000 7f8a4525b740 1 --2- 172.21.15.120:0/626093644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e0b4b2780 0x562e0b52ac10 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f8a34009cd0 tx=0x7f8a3402a8e0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.465+0000 7f8a4525b740 1 -- 172.21.15.120:0/626093644 shutdown_connections 2024-09-19T09:16:40.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.465+0000 7f8a4525b740 1 --2- 172.21.15.120:0/626093644 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562e0b4c20b0 0x562e0b52eb90 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.465+0000 7f8a4525b740 1 --2- 172.21.15.120:0/626093644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e0b4b2780 0x562e0b52ac10 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.465+0000 7f8a1bfff640 0 client.20262 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:40.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.465+0000 7f8a4525b740 1 --2- 172.21.15.120:0/626093644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e0b4a4b40 0x562e0b4b62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.465+0000 7f8a4525b740 1 --2- 172.21.15.120:0/626093644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e0b4a35d0 0x562e0b4b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.465+0000 7f8a4525b740 1 -- 172.21.15.120:0/626093644 >> 172.21.15.120:0/626093644 conn(0x562e0b46c310 msgr2=0x562e0b49e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:40.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.465+0000 7f8a4525b740 1 -- 172.21.15.120:0/626093644 shutdown_connections 2024-09-19T09:16:40.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.465+0000 7f8a4525b740 1 -- 172.21.15.120:0/626093644 wait complete. 2024-09-19T09:16:40.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.465+0000 7f8a4525b740 20 client.20262 trim_cache size 0 max 16384 2024-09-19T09:16:40.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.466+0000 7f8a42e82640 1 -- 172.21.15.120:0/950104266 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a2c0a3240 msgr2=0x7f8a2c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:40.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.466+0000 7f8a42e82640 1 --2- 172.21.15.120:0/950104266 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a2c0a3240 0x7f8a2c0a5700 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8a34009a60 tx=0x7f8a34005f80 comp rx=0 tx=0).stop 2024-09-19T09:16:40.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.466+0000 7f8a42e82640 1 -- 172.21.15.120:0/950104266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a3c100a60 msgr2=0x7f8a3c1992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:40.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.466+0000 7f8a42e82640 1 --2- 172.21.15.120:0/950104266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a3c100a60 0x7f8a3c1992f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8a3000e9f0 tx=0x7f8a3000edf0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.466+0000 7f8a42e82640 1 -- 172.21.15.120:0/950104266 shutdown_connections 2024-09-19T09:16:40.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.466+0000 7f8a42e82640 1 --2- 172.21.15.120:0/950104266 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a2c0a3240 0x7f8a2c0a5700 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.466+0000 7f8a42e82640 1 --2- 172.21.15.120:0/950104266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a3c106a60 0x7f8a3c1a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.466+0000 7f8a42e82640 1 --2- 172.21.15.120:0/950104266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a3c101400 0x7f8a3c199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.466+0000 7f8a42e82640 1 --2- 172.21.15.120:0/950104266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a3c100a60 0x7f8a3c1992f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:40.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.466+0000 7f8a42e82640 1 -- 172.21.15.120:0/950104266 >> 172.21.15.120:0/950104266 conn(0x7f8a3c0fc780 msgr2=0x7f8a3c0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:40.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.466+0000 7f8a42e82640 1 -- 172.21.15.120:0/950104266 shutdown_connections 2024-09-19T09:16:40.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:40.466+0000 7f8a42e82640 1 -- 172.21.15.120:0/950104266 wait complete. 2024-09-19T09:16:41.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:40 smithi120 ceph-mon[31150]: pgmap v759: 97 pgs: 97 active+clean; 200 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 21 MiB/s wr, 372 op/s 2024-09-19T09:16:41.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:40 smithi196 ceph-mon[35150]: pgmap v759: 97 pgs: 97 active+clean; 200 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 21 MiB/s wr, 372 op/s 2024-09-19T09:16:41.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:40 smithi153 ceph-mon[35071]: pgmap v759: 97 pgs: 97 active+clean; 200 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 21 MiB/s wr, 372 op/s 2024-09-19T09:16:41.353 INFO:tasks.workunit.client.0.smithi120.stdout: 10 8286 17.08 MB/sec warmup 36 sec latency 49.980 ms 2024-09-19T09:16:42.353 INFO:tasks.workunit.client.0.smithi120.stdout: 10 9098 18.23 MB/sec warmup 37 sec latency 58.205 ms 2024-09-19T09:16:42.511 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:16:42.511 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:16:42.781 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:16:43.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:42 smithi196 ceph-mon[35150]: pgmap v760: 97 pgs: 97 active+clean; 215 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 18 MiB/s wr, 406 op/s 2024-09-19T09:16:43.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:42 smithi153 ceph-mon[35071]: pgmap v760: 97 pgs: 97 active+clean; 215 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 18 MiB/s wr, 406 op/s 2024-09-19T09:16:43.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:42 smithi120 ceph-mon[31150]: pgmap v760: 97 pgs: 97 active+clean; 215 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 18 MiB/s wr, 406 op/s 2024-09-19T09:16:43.353 INFO:tasks.workunit.client.0.smithi120.stdout: 10 9631 18.59 MB/sec warmup 38 sec latency 52.341 ms 2024-09-19T09:16:43.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.429+0000 7fe373688640 1 -- 172.21.15.120:0/1158692411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe36c101de0 msgr2=0x7fe36c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:43.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.429+0000 7fe373688640 1 --2- 172.21.15.120:0/1158692411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe36c101de0 0x7fe36c1021c0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fe368009900 tx=0x7fe36802a410 comp rx=0 tx=0).stop 2024-09-19T09:16:43.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.429+0000 7fe373688640 1 -- 172.21.15.120:0/1158692411 shutdown_connections 2024-09-19T09:16:43.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.429+0000 7fe373688640 1 --2- 172.21.15.120:0/1158692411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe36c106f10 0x7fe36c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.429+0000 7fe373688640 1 --2- 172.21.15.120:0/1158692411 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe36c102790 0x7fe36c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.429+0000 7fe373688640 1 --2- 172.21.15.120:0/1158692411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe36c101de0 0x7fe36c1021c0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.429+0000 7fe373688640 1 -- 172.21.15.120:0/1158692411 >> 172.21.15.120:0/1158692411 conn(0x7fe36c0fd8c0 msgr2=0x7fe36c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:43.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.429+0000 7fe373688640 1 -- 172.21.15.120:0/1158692411 shutdown_connections 2024-09-19T09:16:43.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.429+0000 7fe373688640 1 -- 172.21.15.120:0/1158692411 wait complete. 2024-09-19T09:16:43.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.430+0000 7fe373688640 1 Processor -- start 2024-09-19T09:16:43.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.430+0000 7fe373688640 1 -- start start 2024-09-19T09:16:43.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.430+0000 7fe373688640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe36c101de0 0x7fe36c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:43.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.430+0000 7fe373688640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe36c102790 0x7fe36c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:43.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.430+0000 7fe373688640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe36c106f10 0x7fe36c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:43.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.430+0000 7fe373688640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c111f70 con 0x7fe36c106f10 2024-09-19T09:16:43.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.430+0000 7fe373688640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c111df0 con 0x7fe36c101de0 2024-09-19T09:16:43.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.430+0000 7fe373688640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c1120f0 con 0x7fe36c102790 2024-09-19T09:16:43.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.431+0000 7fe371c25640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe36c106f10 0x7fe36c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:43.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.431+0000 7fe371c25640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe36c106f10 0x7fe36c19cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60326/0 (socket says 172.21.15.120:60326) 2024-09-19T09:16:43.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.431+0000 7fe371424640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe36c101de0 0x7fe36c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:43.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.431+0000 7fe371c25640 1 -- 172.21.15.120:0/4257475756 learned_addr learned my addr 172.21.15.120:0/4257475756 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:43.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.431+0000 7fe370c23640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe36c102790 0x7fe36c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:43.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.431+0000 7fe371c25640 1 -- 172.21.15.120:0/4257475756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe36c102790 msgr2=0x7fe36c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:43.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.431+0000 7fe371c25640 1 --2- 172.21.15.120:0/4257475756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe36c102790 0x7fe36c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.431+0000 7fe371c25640 1 -- 172.21.15.120:0/4257475756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe36c101de0 msgr2=0x7fe36c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:43.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.431+0000 7fe371c25640 1 --2- 172.21.15.120:0/4257475756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe36c101de0 0x7fe36c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.431+0000 7fe371c25640 1 -- 172.21.15.120:0/4257475756 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36c19d3e0 con 0x7fe36c106f10 2024-09-19T09:16:43.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.431+0000 7fe371424640 1 --2- 172.21.15.120:0/4257475756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe36c101de0 0x7fe36c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:16:43.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.431+0000 7fe371c25640 1 --2- 172.21.15.120:0/4257475756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe36c106f10 0x7fe36c19cd00 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fe36400eb10 tx=0x7fe36400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:43.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.431+0000 7fe3627fc640 1 -- 172.21.15.120:0/4257475756 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364004b20 con 0x7fe36c106f10 2024-09-19T09:16:43.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.431+0000 7fe373688640 1 -- 172.21.15.120:0/4257475756 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe36c19d6d0 con 0x7fe36c106f10 2024-09-19T09:16:43.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.431+0000 7fe3627fc640 1 -- 172.21.15.120:0/4257475756 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe364004cc0 con 0x7fe36c106f10 2024-09-19T09:16:43.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.431+0000 7fe3627fc640 1 -- 172.21.15.120:0/4257475756 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364010760 con 0x7fe36c106f10 2024-09-19T09:16:43.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.431+0000 7fe373688640 1 -- 172.21.15.120:0/4257475756 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe36c1a4fb0 con 0x7fe36c106f10 2024-09-19T09:16:43.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.434+0000 7fe3627fc640 1 -- 172.21.15.120:0/4257475756 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe364005020 con 0x7fe36c106f10 2024-09-19T09:16:43.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.434+0000 7fe375a61740 1 Processor -- start 2024-09-19T09:16:43.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.434+0000 7fe375a61740 1 -- start start 2024-09-19T09:16:43.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.434+0000 7fe375a61740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4136cda40 0x55e4136cde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:43.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.434+0000 7fe375a61740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4136ce360 0x55e4136dcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:43.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.434+0000 7fe375a61740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4136dd150 0x55e4136df540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:43.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.434+0000 7fe375a61740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e413633280 con 0x55e4136ce360 2024-09-19T09:16:43.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.434+0000 7fe375a61740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e4135dab10 con 0x55e4136dd150 2024-09-19T09:16:43.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.434+0000 7fe375a61740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e41362f240 con 0x55e4136cda40 2024-09-19T09:16:43.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.434+0000 7fe371424640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4136ce360 0x55e4136dcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:43.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.434+0000 7fe370c23640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4136cda40 0x55e4136cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:43.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.434+0000 7fe371424640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4136ce360 0x55e4136dcc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60340/0 (socket says 172.21.15.120:60340) 2024-09-19T09:16:43.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.434+0000 7fe370c23640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4136cda40 0x55e4136cde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:42448/0 (socket says 172.21.15.120:42448) 2024-09-19T09:16:43.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.434+0000 7fe371424640 1 -- 172.21.15.120:0/4009256199 learned_addr learned my addr 172.21.15.120:0/4009256199 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:43.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.434+0000 7fe370c23640 1 --2- 172.21.15.120:0/4009256199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4136dd150 0x55e4136df540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:43.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.434+0000 7fe3627fc640 1 --2- 172.21.15.120:0/4257475756 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe3580a3240 0x7fe3580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:43.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.435+0000 7fe370c23640 1 -- 172.21.15.120:0/4009256199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4136dd150 msgr2=0x55e4136df540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:43.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.435+0000 7fe370c23640 1 --2- 172.21.15.120:0/4009256199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4136dd150 0x55e4136df540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.435+0000 7fe370c23640 1 -- 172.21.15.120:0/4009256199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4136ce360 msgr2=0x55e4136dcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:43.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.435+0000 7fe370c23640 1 --2- 172.21.15.120:0/4009256199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4136ce360 0x55e4136dcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.435+0000 7fe370c23640 1 -- 172.21.15.120:0/4009256199 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4135e71f0 con 0x55e4136cda40 2024-09-19T09:16:43.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.435+0000 7fe371424640 1 --2- 172.21.15.120:0/4009256199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4136ce360 0x55e4136dcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:16:43.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.435+0000 7fe3627fc640 1 -- 172.21.15.120:0/4257475756 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe3640cff40 con 0x7fe36c106f10 2024-09-19T09:16:43.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.435+0000 7fe371c25640 1 --2- 172.21.15.120:0/4257475756 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe3580a3240 0x7fe3580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:43.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.435+0000 7fe370c23640 1 --2- 172.21.15.120:0/4009256199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4136cda40 0x55e4136cde20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe35c009fd0 tx=0x7fe35c0096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a82399a051ea01e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:43.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.435+0000 7fe3577fe640 1 -- 172.21.15.120:0/4009256199 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c004050 con 0x55e4136cda40 2024-09-19T09:16:43.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.435+0000 7fe3577fe640 1 -- 172.21.15.120:0/4009256199 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe35c005500 con 0x55e4136cda40 2024-09-19T09:16:43.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.435+0000 7fe371c25640 1 --2- 172.21.15.120:0/4257475756 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe3580a3240 0x7fe3580a5700 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe364005ba0 tx=0x7fe36400efd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:43.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.435+0000 7fe3577fe640 1 -- 172.21.15.120:0/4009256199 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c004d90 con 0x55e4136cda40 2024-09-19T09:16:43.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.436+0000 7fe375a61740 1 -- 172.21.15.120:0/4009256199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4136cda40 msgr2=0x55e4136cde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:43.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.436+0000 7fe375a61740 1 --2- 172.21.15.120:0/4009256199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4136cda40 0x55e4136cde20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe35c009fd0 tx=0x7fe35c0096c0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.436+0000 7fe375a61740 1 -- 172.21.15.120:0/4009256199 shutdown_connections 2024-09-19T09:16:43.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.436+0000 7fe375a61740 1 --2- 172.21.15.120:0/4009256199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4136dd150 0x55e4136df540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.436+0000 7fe375a61740 1 --2- 172.21.15.120:0/4009256199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4136ce360 0x55e4136dcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.436+0000 7fe375a61740 1 --2- 172.21.15.120:0/4009256199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4136cda40 0x55e4136cde20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.436+0000 7fe375a61740 1 -- 172.21.15.120:0/4009256199 >> 172.21.15.120:0/4009256199 conn(0x55e4136ca0f0 msgr2=0x55e4136ca4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:43.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.436+0000 7fe375a61740 1 -- 172.21.15.120:0/4009256199 shutdown_connections 2024-09-19T09:16:43.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.436+0000 7fe375a61740 1 -- 172.21.15.120:0/4009256199 wait complete. 2024-09-19T09:16:43.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.436+0000 7fe375a61740 1 -- start start 2024-09-19T09:16:43.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.436+0000 7fe375a61740 1 Processor -- start 2024-09-19T09:16:43.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.437+0000 7fe375a61740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4136cda40 0x55e4136da070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:43.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.437+0000 7fe375a61740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4136ce360 0x55e4136da5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:43.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.437+0000 7fe375a61740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4136dd150 0x55e4136daaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:43.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.437+0000 7fe375a61740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e4136dfd20 con 0x55e4136ce360 2024-09-19T09:16:43.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.437+0000 7fe375a61740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e4135edf10 con 0x55e4136dd150 2024-09-19T09:16:43.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.437+0000 7fe375a61740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e4136dfea0 con 0x55e4136cda40 2024-09-19T09:16:43.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.437+0000 7fe375a61740 10 client.0 _ll_register_callbacks cb 0x55e4135b6a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:43.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.437+0000 7fe371424640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4136ce360 0x55e4136da5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:43.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.437+0000 7fe371424640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4136ce360 0x55e4136da5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60348/0 (socket says 172.21.15.120:60348) 2024-09-19T09:16:43.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.437+0000 7fe371424640 1 -- 172.21.15.120:0/1971699442 learned_addr learned my addr 172.21.15.120:0/1971699442 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:43.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.437+0000 7fe370c23640 1 --2- 172.21.15.120:0/1971699442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4136dd150 0x55e4136daaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:43.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.437+0000 7fe370c23640 1 --2- 172.21.15.120:0/1971699442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4136cda40 0x55e4136da070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:43.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.437+0000 7fe371424640 1 -- 172.21.15.120:0/1971699442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4136cda40 msgr2=0x55e4136da070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:43.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.437+0000 7fe371424640 1 --2- 172.21.15.120:0/1971699442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4136cda40 0x55e4136da070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.437+0000 7fe371424640 1 -- 172.21.15.120:0/1971699442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4136dd150 msgr2=0x55e4136daaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:43.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.437+0000 7fe371424640 1 --2- 172.21.15.120:0/1971699442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4136dd150 0x55e4136daaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.437+0000 7fe371424640 1 -- 172.21.15.120:0/1971699442 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4133a46a0 con 0x55e4136ce360 2024-09-19T09:16:43.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.437+0000 7fe371424640 1 --2- 172.21.15.120:0/1971699442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4136ce360 0x55e4136da5b0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fe368033000 tx=0x7fe368031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9daca7106d68298c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:43.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.437+0000 7fe354ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:16:43.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.437+0000 7fe354ff9640 1 -- 172.21.15.120:0/1971699442 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368035040 con 0x55e4136ce360 2024-09-19T09:16:43.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.437+0000 7fe354ff9640 1 -- 172.21.15.120:0/1971699442 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe368004440 con 0x55e4136ce360 2024-09-19T09:16:43.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.437+0000 7fe375a61740 1 -- 172.21.15.120:0/1971699442 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe35c005500 con 0x55e4136ce360 2024-09-19T09:16:43.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.438+0000 7fe354ff9640 1 -- 172.21.15.120:0/1971699442 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe368005050 con 0x55e4136ce360 2024-09-19T09:16:43.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.438+0000 7fe375a61740 10 client.20274 fetch_fsmap learned FSMap version 36 2024-09-19T09:16:43.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.438+0000 7fe375a61740 1 -- 172.21.15.120:0/1971699442 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e4136eb9c0 con 0x55e4136ce360 2024-09-19T09:16:43.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.438+0000 7fe354ff9640 1 -- 172.21.15.120:0/1971699442 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe35c005500 con 0x55e4136ce360 2024-09-19T09:16:43.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.438+0000 7fe354ff9640 1 -- 172.21.15.120:0/1971699442 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fe3680035f0 con 0x55e4136ce360 2024-09-19T09:16:43.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.438+0000 7fe375a61740 10 client.20274 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:16:43.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.438+0000 7fe375a61740 10 client.20274 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:16:43.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.438+0000 7fe375a61740 20 client.20274 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:43.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.438+0000 7fe375a61740 1 --2- 172.21.15.120:0/1971699442 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e4136ecf20 0x55e4136ef310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:43.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.438+0000 7fe375a61740 4 client.20274 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:16:43.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.438+0000 7fe375a61740 1 -- 172.21.15.120:0/1971699442 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4136407f0 con 0x55e4136ecf20 2024-09-19T09:16:43.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.438+0000 7fe370c23640 1 --2- 172.21.15.120:0/1971699442 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e4136ecf20 0x55e4136ef310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:43.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.439+0000 7fe354ff9640 10 client.20274 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:43.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.439+0000 7fe370c23640 1 --2- 172.21.15.120:0/1971699442 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e4136ecf20 0x55e4136ef310 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db4e7ac40d853a19 server_cookie=3823971a709f48f in_seq=0 out_seq=0 2024-09-19T09:16:43.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.441+0000 7fe354ff9640 1 -- 172.21.15.120:0/1971699442 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e4136407f0 con 0x55e4136ecf20 2024-09-19T09:16:43.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.441+0000 7fe354ff9640 10 client.20274 handle_command_reply: tid=0 2024-09-19T09:16:43.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.441+0000 7fe375a61740 1 client.20274 shutdown 2024-09-19T09:16:43.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.441+0000 7fe375a61740 1 -- 172.21.15.120:0/1971699442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4136ce360 msgr2=0x55e4136da5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:43.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.441+0000 7fe375a61740 1 --2- 172.21.15.120:0/1971699442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4136ce360 0x55e4136da5b0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fe368033000 tx=0x7fe368031850 comp rx=0 tx=0).stop 2024-09-19T09:16:43.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.441+0000 7fe375a61740 1 -- 172.21.15.120:0/1971699442 shutdown_connections 2024-09-19T09:16:43.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.441+0000 7fe375a61740 1 --2- 172.21.15.120:0/1971699442 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e4136ecf20 0x55e4136ef310 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.441+0000 7fe375a61740 1 --2- 172.21.15.120:0/1971699442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4136dd150 0x55e4136daaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.441+0000 7fe375a61740 1 --2- 172.21.15.120:0/1971699442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4136ce360 0x55e4136da5b0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.441+0000 7fe354ff9640 0 client.20274 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:43.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.441+0000 7fe375a61740 1 --2- 172.21.15.120:0/1971699442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4136cda40 0x55e4136da070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.441+0000 7fe375a61740 1 -- 172.21.15.120:0/1971699442 >> 172.21.15.120:0/1971699442 conn(0x55e4136ca0f0 msgr2=0x55e4136cfe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:43.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.441+0000 7fe375a61740 1 -- 172.21.15.120:0/1971699442 shutdown_connections 2024-09-19T09:16:43.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.441+0000 7fe375a61740 1 -- 172.21.15.120:0/1971699442 wait complete. 2024-09-19T09:16:43.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.441+0000 7fe375a61740 20 client.20274 trim_cache size 0 max 16384 2024-09-19T09:16:43.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.460+0000 7fe375a61740 1 Processor -- start 2024-09-19T09:16:43.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.460+0000 7fe375a61740 1 -- start start 2024-09-19T09:16:43.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.461+0000 7fe375a61740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4136cda40 0x55e4136da940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:43.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.461+0000 7fe375a61740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4136ce360 0x55e4136f4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:43.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.461+0000 7fe375a61740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4136dd150 0x55e4136f5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:43.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.461+0000 7fe375a61740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c1ab410 con 0x55e4136dd150 2024-09-19T09:16:43.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.461+0000 7fe375a61740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c1acfe0 con 0x55e4136cda40 2024-09-19T09:16:43.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.461+0000 7fe375a61740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c1abae0 con 0x55e4136ce360 2024-09-19T09:16:43.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.461+0000 7fe370c23640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4136dd150 0x55e4136f5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:43.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.461+0000 7fe371424640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4136ce360 0x55e4136f4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:43.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.461+0000 7fe370c23640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4136dd150 0x55e4136f5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60362/0 (socket says 172.21.15.120:60362) 2024-09-19T09:16:43.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.461+0000 7fe370c23640 1 -- 172.21.15.120:0/3458619693 learned_addr learned my addr 172.21.15.120:0/3458619693 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:43.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.461+0000 7fe370c23640 1 --2- 172.21.15.120:0/3458619693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4136cda40 0x55e4136da940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:43.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.461+0000 7fe370c23640 1 -- 172.21.15.120:0/3458619693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4136ce360 msgr2=0x55e4136f4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:43.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.461+0000 7fe370c23640 1 --2- 172.21.15.120:0/3458619693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4136ce360 0x55e4136f4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.461+0000 7fe370c23640 1 -- 172.21.15.120:0/3458619693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4136cda40 msgr2=0x55e4136da940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:43.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.461+0000 7fe370c23640 1 --2- 172.21.15.120:0/3458619693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4136cda40 0x55e4136da940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.461+0000 7fe370c23640 1 -- 172.21.15.120:0/3458619693 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4136ce190 con 0x55e4136dd150 2024-09-19T09:16:43.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.461+0000 7fe370c23640 1 --2- 172.21.15.120:0/3458619693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4136dd150 0x55e4136f5120 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fe35c004940 tx=0x7fe35c004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d96c10532668ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:43.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.461+0000 7fe356ffd640 1 -- 172.21.15.120:0/3458619693 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c00f9a0 con 0x55e4136dd150 2024-09-19T09:16:43.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.462+0000 7fe356ffd640 1 -- 172.21.15.120:0/3458619693 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe35c02f3d0 con 0x55e4136dd150 2024-09-19T09:16:43.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.462+0000 7fe356ffd640 1 -- 172.21.15.120:0/3458619693 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c02d3b0 con 0x55e4136dd150 2024-09-19T09:16:43.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.462+0000 7fe375a61740 1 -- 172.21.15.120:0/3458619693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4136dd150 msgr2=0x55e4136f5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:43.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.462+0000 7fe375a61740 1 --2- 172.21.15.120:0/3458619693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4136dd150 0x55e4136f5120 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fe35c004940 tx=0x7fe35c004970 comp rx=0 tx=0).stop 2024-09-19T09:16:43.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.462+0000 7fe375a61740 1 -- 172.21.15.120:0/3458619693 shutdown_connections 2024-09-19T09:16:43.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.462+0000 7fe375a61740 1 --2- 172.21.15.120:0/3458619693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4136dd150 0x55e4136f5120 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.462+0000 7fe375a61740 1 --2- 172.21.15.120:0/3458619693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4136ce360 0x55e4136f4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.462+0000 7fe375a61740 1 --2- 172.21.15.120:0/3458619693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4136cda40 0x55e4136da940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.462+0000 7fe375a61740 1 -- 172.21.15.120:0/3458619693 >> 172.21.15.120:0/3458619693 conn(0x55e4136ca0f0 msgr2=0x55e4136c9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:43.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.462+0000 7fe375a61740 1 -- 172.21.15.120:0/3458619693 shutdown_connections 2024-09-19T09:16:43.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.462+0000 7fe375a61740 1 -- 172.21.15.120:0/3458619693 wait complete. 2024-09-19T09:16:43.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.463+0000 7fe375a61740 1 -- start start 2024-09-19T09:16:43.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.463+0000 7fe375a61740 1 Processor -- start 2024-09-19T09:16:43.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.463+0000 7fe375a61740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4136cda40 0x55e4136e1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:43.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.463+0000 7fe375a61740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4136ce360 0x55e4136e17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:43.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.463+0000 7fe375a61740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4136dd150 0x55e413756bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:43.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.463+0000 7fe375a61740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c1ac910 con 0x55e4136ce360 2024-09-19T09:16:43.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.463+0000 7fe375a61740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c1ac1b0 con 0x55e4136cda40 2024-09-19T09:16:43.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.463+0000 7fe375a61740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c1a93d0 con 0x55e4136dd150 2024-09-19T09:16:43.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.463+0000 7fe375a61740 10 client.0 _ll_register_callbacks cb 0x7fe368009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:43.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.463+0000 7fe371424640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4136ce360 0x55e4136e17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:43.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.463+0000 7fe370c23640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4136cda40 0x55e4136e1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:43.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.463+0000 7fe371424640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4136ce360 0x55e4136e17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60378/0 (socket says 172.21.15.120:60378) 2024-09-19T09:16:43.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.463+0000 7fe371424640 1 -- 172.21.15.120:0/2555751757 learned_addr learned my addr 172.21.15.120:0/2555751757 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:43.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.463+0000 7fe370c23640 1 --2- 172.21.15.120:0/2555751757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4136dd150 0x55e413756bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:43.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.464+0000 7fe371424640 1 -- 172.21.15.120:0/2555751757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4136dd150 msgr2=0x55e413756bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:43.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.464+0000 7fe371424640 1 --2- 172.21.15.120:0/2555751757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4136dd150 0x55e413756bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.464+0000 7fe371424640 1 -- 172.21.15.120:0/2555751757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4136cda40 msgr2=0x55e4136e1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:43.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.464+0000 7fe371424640 1 --2- 172.21.15.120:0/2555751757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4136cda40 0x55e4136e1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.464+0000 7fe371424640 1 -- 172.21.15.120:0/2555751757 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4137570f0 con 0x55e4136ce360 2024-09-19T09:16:43.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.464+0000 7fe371424640 1 --2- 172.21.15.120:0/2555751757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4136ce360 0x55e4136e17b0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fe368003a40 tx=0x7fe368032b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f3f675c7a663d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:43.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.464+0000 7fe354ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:16:43.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.464+0000 7fe354ff9640 1 -- 172.21.15.120:0/2555751757 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368035070 con 0x55e4136ce360 2024-09-19T09:16:43.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.464+0000 7fe354ff9640 1 -- 172.21.15.120:0/2555751757 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe368004440 con 0x55e4136ce360 2024-09-19T09:16:43.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.464+0000 7fe375a61740 1 -- 172.21.15.120:0/2555751757 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe35c02f3d0 con 0x55e4136ce360 2024-09-19T09:16:43.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.464+0000 7fe354ff9640 1 -- 172.21.15.120:0/2555751757 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe3680051f0 con 0x55e4136ce360 2024-09-19T09:16:43.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.464+0000 7fe375a61740 10 client.20280 fetch_fsmap learned FSMap version 36 2024-09-19T09:16:43.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.464+0000 7fe375a61740 1 -- 172.21.15.120:0/2555751757 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e413758280 con 0x55e4136ce360 2024-09-19T09:16:43.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.465+0000 7fe354ff9640 1 -- 172.21.15.120:0/2555751757 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe35c02f3d0 con 0x55e4136ce360 2024-09-19T09:16:43.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.465+0000 7fe354ff9640 1 -- 172.21.15.120:0/2555751757 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fe368059060 con 0x55e4136ce360 2024-09-19T09:16:43.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.465+0000 7fe375a61740 10 client.20280 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:16:43.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.465+0000 7fe375a61740 10 client.20280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:16:43.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.465+0000 7fe375a61740 20 client.20280 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:43.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.465+0000 7fe375a61740 1 --2- 172.21.15.120:0/2555751757 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e413758b10 0x55e41375af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:43.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.465+0000 7fe375a61740 4 client.20280 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:16:43.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.465+0000 7fe375a61740 1 -- 172.21.15.120:0/2555751757 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e41363ad00 con 0x55e413758b10 2024-09-19T09:16:43.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.465+0000 7fe370c23640 1 --2- 172.21.15.120:0/2555751757 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e413758b10 0x55e41375af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:43.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.465+0000 7fe370c23640 1 --2- 172.21.15.120:0/2555751757 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e413758b10 0x55e41375af30 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8ea4b4cf84589d6 server_cookie=a7e2354548a86e57 in_seq=0 out_seq=0 2024-09-19T09:16:43.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.465+0000 7fe354ff9640 10 client.20280 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:43.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.477+0000 7fe354ff9640 1 -- 172.21.15.120:0/2555751757 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e41363ad00 con 0x55e413758b10 2024-09-19T09:16:43.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.477+0000 7fe354ff9640 10 client.20280 handle_command_reply: tid=0 2024-09-19T09:16:43.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.477+0000 7fe375a61740 1 client.20280 shutdown 2024-09-19T09:16:43.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.477+0000 7fe375a61740 1 -- 172.21.15.120:0/2555751757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4136ce360 msgr2=0x55e4136e17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:43.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.477+0000 7fe375a61740 1 --2- 172.21.15.120:0/2555751757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4136ce360 0x55e4136e17b0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fe368003a40 tx=0x7fe368032b00 comp rx=0 tx=0).stop 2024-09-19T09:16:43.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.477+0000 7fe375a61740 1 -- 172.21.15.120:0/2555751757 shutdown_connections 2024-09-19T09:16:43.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.477+0000 7fe375a61740 1 --2- 172.21.15.120:0/2555751757 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e413758b10 0x55e41375af30 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.477+0000 7fe375a61740 1 --2- 172.21.15.120:0/2555751757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4136dd150 0x55e413756bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.477+0000 7fe375a61740 1 --2- 172.21.15.120:0/2555751757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e4136ce360 0x55e4136e17b0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.477+0000 7fe375a61740 1 --2- 172.21.15.120:0/2555751757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e4136cda40 0x55e4136e1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.477+0000 7fe375a61740 1 -- 172.21.15.120:0/2555751757 >> 172.21.15.120:0/2555751757 conn(0x55e4136ca0f0 msgr2=0x55e4136c8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:43.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.477+0000 7fe354ff9640 0 client.20280 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:43.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.477+0000 7fe375a61740 1 -- 172.21.15.120:0/2555751757 shutdown_connections 2024-09-19T09:16:43.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.477+0000 7fe375a61740 1 -- 172.21.15.120:0/2555751757 wait complete. 2024-09-19T09:16:43.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.477+0000 7fe375a61740 20 client.20280 trim_cache size 0 max 16384 2024-09-19T09:16:43.479 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:16:43.479 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:16:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "6a755c3c-4ed2-4563-9a3f-b83035d8d031", 2024-09-19T09:16:43.480 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:16:43.480 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:16:43.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.478+0000 7fe373688640 1 -- 172.21.15.120:0/4257475756 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe3580a3240 msgr2=0x7fe3580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:43.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.478+0000 7fe373688640 1 --2- 172.21.15.120:0/4257475756 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe3580a3240 0x7fe3580a5700 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe364005ba0 tx=0x7fe36400efd0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.478+0000 7fe373688640 1 -- 172.21.15.120:0/4257475756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe36c106f10 msgr2=0x7fe36c19cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:43.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.478+0000 7fe373688640 1 --2- 172.21.15.120:0/4257475756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe36c106f10 0x7fe36c19cd00 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fe36400eb10 tx=0x7fe36400c2d0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.478+0000 7fe373688640 1 -- 172.21.15.120:0/4257475756 shutdown_connections 2024-09-19T09:16:43.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.478+0000 7fe373688640 1 --2- 172.21.15.120:0/4257475756 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe3580a3240 0x7fe3580a5700 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.478+0000 7fe373688640 1 --2- 172.21.15.120:0/4257475756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe36c106f10 0x7fe36c19cd00 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.478+0000 7fe373688640 1 --2- 172.21.15.120:0/4257475756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe36c102790 0x7fe36c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.478+0000 7fe373688640 1 --2- 172.21.15.120:0/4257475756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe36c101de0 0x7fe36c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:43.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.478+0000 7fe373688640 1 -- 172.21.15.120:0/4257475756 >> 172.21.15.120:0/4257475756 conn(0x7fe36c0fd8c0 msgr2=0x7fe36c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:43.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.478+0000 7fe373688640 1 -- 172.21.15.120:0/4257475756 shutdown_connections 2024-09-19T09:16:43.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:43.478+0000 7fe373688640 1 -- 172.21.15.120:0/4257475756 wait complete. 2024-09-19T09:16:44.329 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:16:44.353 INFO:tasks.workunit.client.0.smithi120.stdout: 10 10069 18.60 MB/sec warmup 39 sec latency 148.032 ms 2024-09-19T09:16:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.428+0000 7f3011bd6640 1 -- 172.21.15.120:0/870279167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f300c104f80 msgr2=0x7f300c107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.428+0000 7f3011bd6640 1 --2- 172.21.15.120:0/870279167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f300c104f80 0x7f300c107370 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f3000009900 tx=0x7f300002a410 comp rx=0 tx=0).stop 2024-09-19T09:16:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.428+0000 7f3011bd6640 1 -- 172.21.15.120:0/870279167 shutdown_connections 2024-09-19T09:16:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.428+0000 7f3011bd6640 1 --2- 172.21.15.120:0/870279167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f300c1078b0 0x7f300c109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.428+0000 7f3011bd6640 1 --2- 172.21.15.120:0/870279167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f300c104f80 0x7f300c107370 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.428+0000 7f3011bd6640 1 --2- 172.21.15.120:0/870279167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f300c102650 0x7f300c104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.428+0000 7f3011bd6640 1 -- 172.21.15.120:0/870279167 >> 172.21.15.120:0/870279167 conn(0x7f300c0fc1b0 msgr2=0x7f300c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.429+0000 7f3011bd6640 1 -- 172.21.15.120:0/870279167 shutdown_connections 2024-09-19T09:16:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.429+0000 7f3011bd6640 1 -- 172.21.15.120:0/870279167 wait complete. 2024-09-19T09:16:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.429+0000 7f3011bd6640 1 Processor -- start 2024-09-19T09:16:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.429+0000 7f3011bd6640 1 -- start start 2024-09-19T09:16:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.429+0000 7f3011bd6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f300c102650 0x7f300c101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.429+0000 7f3011bd6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f300c104f80 0x7f300c100600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.429+0000 7f3011bd6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f300c1078b0 0x7f300c100b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.429+0000 7f3011bd6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f300c10d800 con 0x7f300c104f80 2024-09-19T09:16:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.429+0000 7f3011bd6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f300c10d680 con 0x7f300c102650 2024-09-19T09:16:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.429+0000 7f3011bd6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f300c10d980 con 0x7f300c1078b0 2024-09-19T09:16:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.430+0000 7f300affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f300c104f80 0x7f300c100600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.430+0000 7f300bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f300c1078b0 0x7f300c100b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.430+0000 7f300affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f300c104f80 0x7f300c100600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60390/0 (socket says 172.21.15.120:60390) 2024-09-19T09:16:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.430+0000 7f300affd640 1 -- 172.21.15.120:0/3252190877 learned_addr learned my addr 172.21.15.120:0/3252190877 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.430+0000 7f300b7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f300c102650 0x7f300c101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.430+0000 7f300affd640 1 -- 172.21.15.120:0/3252190877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f300c1078b0 msgr2=0x7f300c100b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.430+0000 7f300affd640 1 --2- 172.21.15.120:0/3252190877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f300c1078b0 0x7f300c100b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.430+0000 7f300affd640 1 -- 172.21.15.120:0/3252190877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f300c102650 msgr2=0x7f300c101fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.430+0000 7f300affd640 1 --2- 172.21.15.120:0/3252190877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f300c102650 0x7f300c101fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.430+0000 7f300affd640 1 -- 172.21.15.120:0/3252190877 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300c1013d0 con 0x7f300c104f80 2024-09-19T09:16:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.430+0000 7f300bfff640 1 --2- 172.21.15.120:0/3252190877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f300c1078b0 0x7f300c100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:16:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.430+0000 7f300b7fe640 1 --2- 172.21.15.120:0/3252190877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f300c102650 0x7f300c101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:16:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.430+0000 7f300affd640 1 --2- 172.21.15.120:0/3252190877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f300c104f80 0x7f300c100600 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f3000031610 tx=0x7f3000004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.430+0000 7f3008ff9640 1 -- 172.21.15.120:0/3252190877 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300003f030 con 0x7f300c104f80 2024-09-19T09:16:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.430+0000 7f3011bd6640 1 -- 172.21.15.120:0/3252190877 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f300c1a1f20 con 0x7f300c104f80 2024-09-19T09:16:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.430+0000 7f3008ff9640 1 -- 172.21.15.120:0/3252190877 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3000003ad0 con 0x7f300c104f80 2024-09-19T09:16:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.430+0000 7f3008ff9640 1 -- 172.21.15.120:0/3252190877 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000035070 con 0x7f300c104f80 2024-09-19T09:16:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.431+0000 7f3011bd6640 1 -- 172.21.15.120:0/3252190877 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f300c1a2390 con 0x7f300c104f80 2024-09-19T09:16:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.433+0000 7f3014a53740 1 Processor -- start 2024-09-19T09:16:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.433+0000 7f3014a53740 1 -- start start 2024-09-19T09:16:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.434+0000 7f3014a53740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f7a9dee1b0 0x55f7a9dee5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.434+0000 7f3014a53740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f7a9deeaf0 0x55f7a9df35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.434+0000 7f3014a53740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9df3b80 0x55f7a9df6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.434+0000 7f3014a53740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f7a9d492f0 con 0x55f7a9df3b80 2024-09-19T09:16:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.434+0000 7f3014a53740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f7a9cf1270 con 0x55f7a9dee1b0 2024-09-19T09:16:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.434+0000 7f3014a53740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f7a9d452b0 con 0x55f7a9deeaf0 2024-09-19T09:16:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.434+0000 7f300b7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f7a9deeaf0 0x55f7a9df35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.434+0000 7f300b7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f7a9deeaf0 0x55f7a9df35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:42496/0 (socket says 172.21.15.120:42496) 2024-09-19T09:16:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.434+0000 7f300bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f7a9dee1b0 0x55f7a9dee5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.434+0000 7f300b7fe640 1 -- 172.21.15.120:0/393243275 learned_addr learned my addr 172.21.15.120:0/393243275 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.434+0000 7f300bfff640 1 -- 172.21.15.120:0/393243275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f7a9deeaf0 msgr2=0x55f7a9df35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.434+0000 7f300bfff640 1 --2- 172.21.15.120:0/393243275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f7a9deeaf0 0x55f7a9df35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.434+0000 7f300bfff640 1 -- 172.21.15.120:0/393243275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9df3b80 msgr2=0x55f7a9df6040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:16:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.434+0000 7f300bfff640 1 --2- 172.21.15.120:0/393243275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9df3b80 0x55f7a9df6040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.434+0000 7f300bfff640 1 -- 172.21.15.120:0/393243275 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7a9cf0340 con 0x55f7a9dee1b0 2024-09-19T09:16:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.434+0000 7f3008ff9640 1 -- 172.21.15.120:0/3252190877 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f3000042030 con 0x7f300c104f80 2024-09-19T09:16:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.434+0000 7f300bfff640 1 --2- 172.21.15.120:0/393243275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f7a9dee1b0 0x55f7a9dee5b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f300c06b1a0 tx=0x7f3004011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2d795eca40bef5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.434+0000 7f2ff1ffb640 1 -- 172.21.15.120:0/393243275 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3004004d20 con 0x55f7a9dee1b0 2024-09-19T09:16:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.435+0000 7f2ff1ffb640 1 -- 172.21.15.120:0/393243275 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f300400f970 con 0x55f7a9dee1b0 2024-09-19T09:16:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.435+0000 7f2ff1ffb640 1 -- 172.21.15.120:0/393243275 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30040044b0 con 0x55f7a9dee1b0 2024-09-19T09:16:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.435+0000 7f3008ff9640 1 --2- 172.21.15.120:0/3252190877 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2ff40a32f0 0x7f2ff40a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.435+0000 7f3008ff9640 1 -- 172.21.15.120:0/3252190877 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f30000efc30 con 0x7f300c104f80 2024-09-19T09:16:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.435+0000 7f300b7fe640 1 --2- 172.21.15.120:0/3252190877 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2ff40a32f0 0x7f2ff40a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.435+0000 7f3014a53740 1 -- 172.21.15.120:0/393243275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f7a9dee1b0 msgr2=0x55f7a9dee5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.435+0000 7f3014a53740 1 --2- 172.21.15.120:0/393243275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f7a9dee1b0 0x55f7a9dee5b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f300c06b1a0 tx=0x7f3004011bf0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.435+0000 7f3014a53740 1 -- 172.21.15.120:0/393243275 shutdown_connections 2024-09-19T09:16:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.435+0000 7f3014a53740 1 --2- 172.21.15.120:0/393243275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9df3b80 0x55f7a9df6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.435+0000 7f3014a53740 1 --2- 172.21.15.120:0/393243275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f7a9deeaf0 0x55f7a9df35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.435+0000 7f3014a53740 1 --2- 172.21.15.120:0/393243275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f7a9dee1b0 0x55f7a9dee5b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.435+0000 7f3014a53740 1 -- 172.21.15.120:0/393243275 >> 172.21.15.120:0/393243275 conn(0x55f7a9de1010 msgr2=0x55f7a9de1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.436+0000 7f3014a53740 1 -- 172.21.15.120:0/393243275 shutdown_connections 2024-09-19T09:16:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.436+0000 7f300b7fe640 1 --2- 172.21.15.120:0/3252190877 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2ff40a32f0 0x7f2ff40a57b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x55f7a9def520 tx=0x7f2ffc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.436+0000 7f3014a53740 1 -- 172.21.15.120:0/393243275 wait complete. 2024-09-19T09:16:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.436+0000 7f3014a53740 1 -- start start 2024-09-19T09:16:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.436+0000 7f3014a53740 1 Processor -- start 2024-09-19T09:16:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.436+0000 7f3014a53740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f7a9dee1b0 0x55f7a9df8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.436+0000 7f3014a53740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f7a9deeaf0 0x55f7a9df8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.436+0000 7f3014a53740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9df3b80 0x55f7a9dff9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.437+0000 7f3014a53740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f7a9df6b50 con 0x55f7a9df3b80 2024-09-19T09:16:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.437+0000 7f3014a53740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f7a9cd2290 con 0x55f7a9dee1b0 2024-09-19T09:16:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.437+0000 7f3014a53740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f7a9df6cd0 con 0x55f7a9deeaf0 2024-09-19T09:16:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.437+0000 7f300bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9df3b80 0x55f7a9dff9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.437+0000 7f300affd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f7a9deeaf0 0x55f7a9df8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.437+0000 7f300affd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f7a9deeaf0 0x55f7a9df8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:42500/0 (socket says 172.21.15.120:42500) 2024-09-19T09:16:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.437+0000 7f300affd640 1 -- 172.21.15.120:0/3665662580 learned_addr learned my addr 172.21.15.120:0/3665662580 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.437+0000 7f300b7fe640 1 --2- 172.21.15.120:0/3665662580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f7a9dee1b0 0x55f7a9df8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.437+0000 7f3014a53740 10 client.0 _ll_register_callbacks cb 0x55f7a9d03140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.437+0000 7f300affd640 1 -- 172.21.15.120:0/3665662580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f7a9dee1b0 msgr2=0x55f7a9df8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.437+0000 7f300affd640 1 --2- 172.21.15.120:0/3665662580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f7a9dee1b0 0x55f7a9df8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.437+0000 7f300affd640 1 -- 172.21.15.120:0/3665662580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9df3b80 msgr2=0x55f7a9dff9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.437+0000 7f300affd640 1 --2- 172.21.15.120:0/3665662580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9df3b80 0x55f7a9dff9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.437+0000 7f300affd640 1 -- 172.21.15.120:0/3665662580 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7a9a5c210 con 0x55f7a9deeaf0 2024-09-19T09:16:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.437+0000 7f300affd640 1 --2- 172.21.15.120:0/3665662580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f7a9deeaf0 0x55f7a9df8970 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f30000c4990 tx=0x7f30000bbf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=663e1cf556c76ef0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.437+0000 7f2feaffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:16:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.437+0000 7f2feaffd640 1 -- 172.21.15.120:0/3665662580 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000049040 con 0x55f7a9deeaf0 2024-09-19T09:16:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.437+0000 7f3014a53740 1 -- 172.21.15.120:0/3665662580 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f300400f970 con 0x55f7a9deeaf0 2024-09-19T09:16:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.437+0000 7f2feaffd640 1 -- 172.21.15.120:0/3665662580 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f300003b460 con 0x55f7a9deeaf0 2024-09-19T09:16:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.437+0000 7f2feaffd640 1 -- 172.21.15.120:0/3665662580 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f30000c4ec0 con 0x55f7a9deeaf0 2024-09-19T09:16:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.437+0000 7f3014a53740 10 client.29071 fetch_fsmap learned FSMap version 36 2024-09-19T09:16:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.438+0000 7f3014a53740 1 -- 172.21.15.120:0/3665662580 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f7a9b28620 con 0x55f7a9deeaf0 2024-09-19T09:16:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.438+0000 7f2feaffd640 1 -- 172.21.15.120:0/3665662580 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f300400f970 con 0x55f7a9deeaf0 2024-09-19T09:16:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.438+0000 7f2feaffd640 1 -- 172.21.15.120:0/3665662580 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f300003bcc0 con 0x55f7a9deeaf0 2024-09-19T09:16:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.438+0000 7f3014a53740 10 client.29071 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:16:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.438+0000 7f3014a53740 10 client.29071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:16:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.438+0000 7f3014a53740 20 client.29071 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.438+0000 7f3014a53740 1 --2- 172.21.15.120:0/3665662580 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f7a9e040e0 0x55f7a9e06500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.438+0000 7f3014a53740 4 client.29071 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:16:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.438+0000 7f3014a53740 1 -- 172.21.15.120:0/3665662580 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7a9d56a20 con 0x55f7a9e040e0 2024-09-19T09:16:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.438+0000 7f300b7fe640 1 --2- 172.21.15.120:0/3665662580 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f7a9e040e0 0x55f7a9e06500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.439+0000 7f300b7fe640 1 --2- 172.21.15.120:0/3665662580 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f7a9e040e0 0x55f7a9e06500 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80dd581a841356e3 server_cookie=e5609fa2bfa209e4 in_seq=0 out_seq=0 2024-09-19T09:16:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.439+0000 7f2feaffd640 10 client.29071 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.442+0000 7f2feaffd640 1 -- 172.21.15.120:0/3665662580 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f7a9d56a20 con 0x55f7a9e040e0 2024-09-19T09:16:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.442+0000 7f2feaffd640 10 client.29071 handle_command_reply: tid=0 2024-09-19T09:16:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.442+0000 7f3014a53740 1 client.29071 shutdown 2024-09-19T09:16:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.442+0000 7f3014a53740 1 -- 172.21.15.120:0/3665662580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f7a9deeaf0 msgr2=0x55f7a9df8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.442+0000 7f3014a53740 1 --2- 172.21.15.120:0/3665662580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f7a9deeaf0 0x55f7a9df8970 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f30000c4990 tx=0x7f30000bbf00 comp rx=0 tx=0).stop 2024-09-19T09:16:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.442+0000 7f3014a53740 1 -- 172.21.15.120:0/3665662580 shutdown_connections 2024-09-19T09:16:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.442+0000 7f3014a53740 1 --2- 172.21.15.120:0/3665662580 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f7a9e040e0 0x55f7a9e06500 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.442+0000 7f3014a53740 1 --2- 172.21.15.120:0/3665662580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9df3b80 0x55f7a9dff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.442+0000 7f2feaffd640 0 client.29071 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.442+0000 7f3014a53740 1 --2- 172.21.15.120:0/3665662580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f7a9deeaf0 0x55f7a9df8970 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.442+0000 7f3014a53740 1 --2- 172.21.15.120:0/3665662580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f7a9dee1b0 0x55f7a9df8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.442+0000 7f3014a53740 1 -- 172.21.15.120:0/3665662580 >> 172.21.15.120:0/3665662580 conn(0x55f7a9de1010 msgr2=0x55f7a9de2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.442+0000 7f3014a53740 1 -- 172.21.15.120:0/3665662580 shutdown_connections 2024-09-19T09:16:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.442+0000 7f3014a53740 1 -- 172.21.15.120:0/3665662580 wait complete. 2024-09-19T09:16:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.442+0000 7f3014a53740 20 client.29071 trim_cache size 0 max 16384 2024-09-19T09:16:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.459+0000 7f3014a53740 1 Processor -- start 2024-09-19T09:16:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.459+0000 7f3014a53740 1 -- start start 2024-09-19T09:16:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.459+0000 7f3014a53740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f7a9de1010 0x55f7a9de00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.459+0000 7f3014a53740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f7a9dee1b0 0x55f7a9dff970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.459+0000 7f3014a53740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9deeaf0 0x55f7a9e0ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.459+0000 7f3014a53740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1a6de0 con 0x55f7a9deeaf0 2024-09-19T09:16:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.459+0000 7f3014a53740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1a89b0 con 0x55f7a9de1010 2024-09-19T09:16:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.459+0000 7f3014a53740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1a74b0 con 0x55f7a9dee1b0 2024-09-19T09:16:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.459+0000 7f300bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9deeaf0 0x55f7a9e0ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.459+0000 7f300affd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f7a9dee1b0 0x55f7a9dff970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.459+0000 7f300b7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f7a9de1010 0x55f7a9de00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.460+0000 7f300bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9deeaf0 0x55f7a9e0ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60410/0 (socket says 172.21.15.120:60410) 2024-09-19T09:16:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.460+0000 7f300bfff640 1 -- 172.21.15.120:0/3333669428 learned_addr learned my addr 172.21.15.120:0/3333669428 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.460+0000 7f300bfff640 1 -- 172.21.15.120:0/3333669428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f7a9dee1b0 msgr2=0x55f7a9dff970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.460+0000 7f300bfff640 1 --2- 172.21.15.120:0/3333669428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f7a9dee1b0 0x55f7a9dff970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.460+0000 7f300bfff640 1 -- 172.21.15.120:0/3333669428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f7a9de1010 msgr2=0x55f7a9de00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.460+0000 7f300bfff640 1 --2- 172.21.15.120:0/3333669428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f7a9de1010 0x55f7a9de00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.460+0000 7f300bfff640 1 -- 172.21.15.120:0/3333669428 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7a9df7f70 con 0x55f7a9deeaf0 2024-09-19T09:16:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.460+0000 7f300affd640 1 --2- 172.21.15.120:0/3333669428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f7a9dee1b0 0x55f7a9dff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:16:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.460+0000 7f300bfff640 1 --2- 172.21.15.120:0/3333669428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9deeaf0 0x55f7a9e0ae00 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f3004005fd0 tx=0x7f3004003a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a1e534e71a2949 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.460+0000 7f2ff17fa640 1 -- 172.21.15.120:0/3333669428 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3004028070 con 0x55f7a9deeaf0 2024-09-19T09:16:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.460+0000 7f2ff17fa640 1 -- 172.21.15.120:0/3333669428 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f300401f040 con 0x55f7a9deeaf0 2024-09-19T09:16:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.460+0000 7f2ff17fa640 1 -- 172.21.15.120:0/3333669428 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30040055e0 con 0x55f7a9deeaf0 2024-09-19T09:16:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.461+0000 7f3014a53740 1 -- 172.21.15.120:0/3333669428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9deeaf0 msgr2=0x55f7a9e0ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.461+0000 7f3014a53740 1 --2- 172.21.15.120:0/3333669428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9deeaf0 0x55f7a9e0ae00 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f3004005fd0 tx=0x7f3004003a30 comp rx=0 tx=0).stop 2024-09-19T09:16:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.461+0000 7f3014a53740 1 -- 172.21.15.120:0/3333669428 shutdown_connections 2024-09-19T09:16:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.461+0000 7f3014a53740 1 --2- 172.21.15.120:0/3333669428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9deeaf0 0x55f7a9e0ae00 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.461+0000 7f3014a53740 1 --2- 172.21.15.120:0/3333669428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f7a9dee1b0 0x55f7a9dff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.461+0000 7f3014a53740 1 --2- 172.21.15.120:0/3333669428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f7a9de1010 0x55f7a9de00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.461+0000 7f3014a53740 1 -- 172.21.15.120:0/3333669428 >> 172.21.15.120:0/3333669428 conn(0x55f7a9df36a0 msgr2=0x55f7a9ddf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.461+0000 7f3014a53740 1 -- 172.21.15.120:0/3333669428 shutdown_connections 2024-09-19T09:16:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.461+0000 7f3014a53740 1 -- 172.21.15.120:0/3333669428 wait complete. 2024-09-19T09:16:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.461+0000 7f3014a53740 1 -- start start 2024-09-19T09:16:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.462+0000 7f3014a53740 1 Processor -- start 2024-09-19T09:16:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.462+0000 7f3014a53740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f7a9de1010 0x55f7a9dfd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.462+0000 7f3014a53740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9dee1b0 0x55f7a9dfd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.462+0000 7f3014a53740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f7a9deeaf0 0x55f7a9dfe990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.462+0000 7f3014a53740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1a82e0 con 0x55f7a9dee1b0 2024-09-19T09:16:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.462+0000 7f3014a53740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1a7b80 con 0x55f7a9deeaf0 2024-09-19T09:16:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.462+0000 7f3014a53740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1a4dc0 con 0x55f7a9de1010 2024-09-19T09:16:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.462+0000 7f3014a53740 10 client.0 _ll_register_callbacks cb 0x7f3000031ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.462+0000 7f300b7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f7a9de1010 0x55f7a9dfd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.462+0000 7f300affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9dee1b0 0x55f7a9dfd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.462+0000 7f300b7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f7a9de1010 0x55f7a9dfd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:42514/0 (socket says 172.21.15.120:42514) 2024-09-19T09:16:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.462+0000 7f300b7fe640 1 -- 172.21.15.120:0/1009097024 learned_addr learned my addr 172.21.15.120:0/1009097024 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.462+0000 7f300bfff640 1 --2- 172.21.15.120:0/1009097024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f7a9deeaf0 0x55f7a9dfe990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.462+0000 7f300b7fe640 1 -- 172.21.15.120:0/1009097024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f7a9deeaf0 msgr2=0x55f7a9dfe990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.462+0000 7f300b7fe640 1 --2- 172.21.15.120:0/1009097024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f7a9deeaf0 0x55f7a9dfe990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.462+0000 7f300b7fe640 1 -- 172.21.15.120:0/1009097024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9dee1b0 msgr2=0x55f7a9dfd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.462+0000 7f300b7fe640 1 --2- 172.21.15.120:0/1009097024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9dee1b0 0x55f7a9dfd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.462+0000 7f300b7fe640 1 -- 172.21.15.120:0/1009097024 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7a9ddc2d0 con 0x55f7a9de1010 2024-09-19T09:16:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.463+0000 7f300b7fe640 1 --2- 172.21.15.120:0/1009097024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f7a9de1010 0x55f7a9dfd490 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009460 tx=0x7f2ffc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad1f1cd376c14e68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.463+0000 7f2feaffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:16:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.463+0000 7f2feaffd640 1 -- 172.21.15.120:0/1009097024 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc005020 con 0x55f7a9de1010 2024-09-19T09:16:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.463+0000 7f2feaffd640 1 -- 172.21.15.120:0/1009097024 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f2ffc0051c0 con 0x55f7a9de1010 2024-09-19T09:16:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.463+0000 7f3014a53740 1 -- 172.21.15.120:0/1009097024 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f300401f040 con 0x55f7a9de1010 2024-09-19T09:16:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.463+0000 7f2feaffd640 1 -- 172.21.15.120:0/1009097024 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2ffc0053c0 con 0x55f7a9de1010 2024-09-19T09:16:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.463+0000 7f3014a53740 10 client.29077 fetch_fsmap learned FSMap version 36 2024-09-19T09:16:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.463+0000 7f3014a53740 1 -- 172.21.15.120:0/1009097024 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f7a9e06b80 con 0x55f7a9de1010 2024-09-19T09:16:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.463+0000 7f2feaffd640 1 -- 172.21.15.120:0/1009097024 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f300401f040 con 0x55f7a9de1010 2024-09-19T09:16:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.463+0000 7f2feaffd640 1 -- 172.21.15.120:0/1009097024 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f2ffc02b3a0 con 0x55f7a9de1010 2024-09-19T09:16:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.463+0000 7f3014a53740 10 client.29077 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:16:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.463+0000 7f3014a53740 10 client.29077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:16:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.463+0000 7f3014a53740 20 client.29077 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.463+0000 7f3014a53740 1 --2- 172.21.15.120:0/1009097024 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f7a9df3b80 0x55f7a9e71950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.464+0000 7f3014a53740 4 client.29077 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:16:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.464+0000 7f3014a53740 1 -- 172.21.15.120:0/1009097024 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f7a9d50d70 con 0x55f7a9df3b80 2024-09-19T09:16:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.464+0000 7f300affd640 1 --2- 172.21.15.120:0/1009097024 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f7a9df3b80 0x55f7a9e71950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.464+0000 7f2feaffd640 10 client.29077 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.464+0000 7f300affd640 1 --2- 172.21.15.120:0/1009097024 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f7a9df3b80 0x55f7a9e71950 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5de08646879e4ac5 server_cookie=e895d302e64d4894 in_seq=0 out_seq=0 2024-09-19T09:16:44.623 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:16:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.984+0000 7f2feaffd640 1 -- 172.21.15.120:0/1009097024 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f7a9d50d70 con 0x55f7a9df3b80 2024-09-19T09:16:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.984+0000 7f2feaffd640 10 client.29077 handle_command_reply: tid=0 2024-09-19T09:16:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.984+0000 7f3014a53740 1 client.29077 shutdown 2024-09-19T09:16:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.984+0000 7f3014a53740 1 -- 172.21.15.120:0/1009097024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f7a9de1010 msgr2=0x55f7a9dfd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.984+0000 7f3014a53740 1 --2- 172.21.15.120:0/1009097024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f7a9de1010 0x55f7a9dfd490 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009460 tx=0x7f2ffc0046c0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.984+0000 7f3014a53740 1 -- 172.21.15.120:0/1009097024 shutdown_connections 2024-09-19T09:16:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.984+0000 7f3014a53740 1 --2- 172.21.15.120:0/1009097024 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f7a9df3b80 0x55f7a9e71950 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.984+0000 7f3014a53740 1 --2- 172.21.15.120:0/1009097024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f7a9deeaf0 0x55f7a9dfe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.984+0000 7f2feaffd640 0 client.29077 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.984+0000 7f3014a53740 1 --2- 172.21.15.120:0/1009097024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9dee1b0 0x55f7a9dfd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.984+0000 7f3014a53740 1 --2- 172.21.15.120:0/1009097024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f7a9de1010 0x55f7a9dfd490 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.985+0000 7f3014a53740 1 -- 172.21.15.120:0/1009097024 >> 172.21.15.120:0/1009097024 conn(0x55f7a9df36a0 msgr2=0x55f7a9de01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.985+0000 7f3014a53740 1 -- 172.21.15.120:0/1009097024 shutdown_connections 2024-09-19T09:16:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.985+0000 7f3014a53740 1 -- 172.21.15.120:0/1009097024 wait complete. 2024-09-19T09:16:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.985+0000 7f3014a53740 20 client.29077 trim_cache size 0 max 16384 2024-09-19T09:16:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:16:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:16:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:16:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:16:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.985+0000 7f3011bd6640 1 -- 172.21.15.120:0/3252190877 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2ff40a32f0 msgr2=0x7f2ff40a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.985+0000 7f3011bd6640 1 --2- 172.21.15.120:0/3252190877 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2ff40a32f0 0x7f2ff40a57b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x55f7a9def520 tx=0x7f2ffc005f90 comp rx=0 tx=0).stop 2024-09-19T09:16:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.985+0000 7f3011bd6640 1 -- 172.21.15.120:0/3252190877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f300c104f80 msgr2=0x7f300c100600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.985+0000 7f3011bd6640 1 --2- 172.21.15.120:0/3252190877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f300c104f80 0x7f300c100600 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f3000031610 tx=0x7f3000004270 comp rx=0 tx=0).stop 2024-09-19T09:16:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.985+0000 7f3011bd6640 1 -- 172.21.15.120:0/3252190877 shutdown_connections 2024-09-19T09:16:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.986+0000 7f3011bd6640 1 --2- 172.21.15.120:0/3252190877 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2ff40a32f0 0x7f2ff40a57b0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.986+0000 7f3011bd6640 1 --2- 172.21.15.120:0/3252190877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f300c1078b0 0x7f300c100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.986+0000 7f3011bd6640 1 --2- 172.21.15.120:0/3252190877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f300c104f80 0x7f300c100600 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.986+0000 7f3011bd6640 1 --2- 172.21.15.120:0/3252190877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f300c102650 0x7f300c101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.986+0000 7f3011bd6640 1 -- 172.21.15.120:0/3252190877 >> 172.21.15.120:0/3252190877 conn(0x7f300c0fc1b0 msgr2=0x7f300c102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.986+0000 7f3011bd6640 1 -- 172.21.15.120:0/3252190877 shutdown_connections 2024-09-19T09:16:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:44.986+0000 7f3011bd6640 1 -- 172.21.15.120:0/3252190877 wait complete. 2024-09-19T09:16:45.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:44 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:16:45.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:44 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:16:45.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:44 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:16:45.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:44 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:16:45.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:44 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:16:45.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:44 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:16:45.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:44 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:16:45.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:44 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:16:45.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:44 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:16:45.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:44 smithi120 ceph-mon[31150]: pgmap v761: 97 pgs: 97 active+clean; 229 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 15 MiB/s wr, 452 op/s 2024-09-19T09:16:45.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:44 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:16:45.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:44 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:16:45.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:44 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:16:45.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:44 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:16:45.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:44 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:16:45.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:44 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:16:45.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:44 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:16:45.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:44 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:16:45.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:44 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:16:45.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:44 smithi196 ceph-mon[35150]: pgmap v761: 97 pgs: 97 active+clean; 229 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 15 MiB/s wr, 452 op/s 2024-09-19T09:16:45.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:44 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:16:45.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:44 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:16:45.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:44 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:16:45.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:44 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:16:45.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:44 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:16:45.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:44 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:16:45.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:44 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:16:45.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:44 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:16:45.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:44 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:16:45.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:44 smithi153 ceph-mon[35071]: pgmap v761: 97 pgs: 97 active+clean; 229 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 15 MiB/s wr, 452 op/s 2024-09-19T09:16:45.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.204+0000 7f84a4958640 1 -- 172.21.15.120:0/1002743066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84a0102ef0 msgr2=0x7f84a010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:45.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.204+0000 7f84a4958640 1 --2- 172.21.15.120:0/1002743066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84a0102ef0 0x7f84a010da40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f8498009ca0 tx=0x7f849802a950 comp rx=0 tx=0).stop 2024-09-19T09:16:45.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.204+0000 7f84a4958640 1 -- 172.21.15.120:0/1002743066 shutdown_connections 2024-09-19T09:16:45.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.204+0000 7f84a4958640 1 --2- 172.21.15.120:0/1002743066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84a0102ef0 0x7f84a010da40 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.204+0000 7f84a4958640 1 --2- 172.21.15.120:0/1002743066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84a0102570 0x7f84a01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.204+0000 7f84a4958640 1 --2- 172.21.15.120:0/1002743066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84a0100e20 0x7f84a0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.204+0000 7f84a4958640 1 -- 172.21.15.120:0/1002743066 >> 172.21.15.120:0/1002743066 conn(0x7f84a00fc7d0 msgr2=0x7f84a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:45.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.204+0000 7f84a4958640 1 -- 172.21.15.120:0/1002743066 shutdown_connections 2024-09-19T09:16:45.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.204+0000 7f84a4958640 1 -- 172.21.15.120:0/1002743066 wait complete. 2024-09-19T09:16:45.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.205+0000 7f84a4958640 1 Processor -- start 2024-09-19T09:16:45.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.205+0000 7f84a4958640 1 -- start start 2024-09-19T09:16:45.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.205+0000 7f84a4958640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84a0100e20 0x7f84a0197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:45.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.205+0000 7f84a4958640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84a0102570 0x7f84a0197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:45.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.205+0000 7f84a4958640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84a0102ef0 0x7f84a019bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:45.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.205+0000 7f84a4958640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f84a0110f80 con 0x7f84a0100e20 2024-09-19T09:16:45.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.205+0000 7f84a4958640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f84a0110e00 con 0x7f84a0102ef0 2024-09-19T09:16:45.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.205+0000 7f849e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84a0100e20 0x7f84a0197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:45.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.205+0000 7f849e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84a0100e20 0x7f84a0197400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60438/0 (socket says 172.21.15.120:60438) 2024-09-19T09:16:45.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.205+0000 7f849ed9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84a0102ef0 0x7f84a019bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:45.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.206+0000 7f849ed9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84a0102ef0 0x7f84a019bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37770/0 (socket says 172.21.15.120:37770) 2024-09-19T09:16:45.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.206+0000 7f849dd9b640 1 --2- 172.21.15.120:0/2745140353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84a0102570 0x7f84a0197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:45.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.206+0000 7f849e59c640 1 -- 172.21.15.120:0/2745140353 learned_addr learned my addr 172.21.15.120:0/2745140353 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:45.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.206+0000 7f84a4958640 1 -- 172.21.15.120:0/2745140353 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84a0111100 con 0x7f84a0102570 2024-09-19T09:16:45.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.206+0000 7f849ed9d640 1 -- 172.21.15.120:0/2745140353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84a0102570 msgr2=0x7f84a0197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:45.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.206+0000 7f849ed9d640 1 --2- 172.21.15.120:0/2745140353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84a0102570 0x7f84a0197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.206+0000 7f849ed9d640 1 -- 172.21.15.120:0/2745140353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84a0100e20 msgr2=0x7f84a0197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:45.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.206+0000 7f849ed9d640 1 --2- 172.21.15.120:0/2745140353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84a0100e20 0x7f84a0197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.206+0000 7f849ed9d640 1 -- 172.21.15.120:0/2745140353 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a019c3b0 con 0x7f84a0102ef0 2024-09-19T09:16:45.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.206+0000 7f849e59c640 1 --2- 172.21.15.120:0/2745140353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84a0100e20 0x7f84a0197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:16:45.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.206+0000 7f849dd9b640 1 --2- 172.21.15.120:0/2745140353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84a0102570 0x7f84a0197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:16:45.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.206+0000 7f849ed9d640 1 --2- 172.21.15.120:0/2745140353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84a0102ef0 0x7f84a019bcd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8498005cc0 tx=0x7f849802f3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:45.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.207+0000 7f848f7fe640 1 -- 172.21.15.120:0/2745140353 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8498034040 con 0x7f84a0102ef0 2024-09-19T09:16:45.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.207+0000 7f848f7fe640 1 -- 172.21.15.120:0/2745140353 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8498030920 con 0x7f84a0102ef0 2024-09-19T09:16:45.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.207+0000 7f84a4958640 1 -- 172.21.15.120:0/2745140353 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a019c640 con 0x7f84a0102ef0 2024-09-19T09:16:45.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.207+0000 7f848f7fe640 1 -- 172.21.15.120:0/2745140353 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84980045b0 con 0x7f84a0102ef0 2024-09-19T09:16:45.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.207+0000 7f84a4958640 1 -- 172.21.15.120:0/2745140353 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a01a3f80 con 0x7f84a0102ef0 2024-09-19T09:16:45.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.209+0000 7f848f7fe640 1 -- 172.21.15.120:0/2745140353 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f8498037020 con 0x7f84a0102ef0 2024-09-19T09:16:45.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.210+0000 7f84a6d31740 1 Processor -- start 2024-09-19T09:16:45.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.210+0000 7f84a6d31740 1 -- start start 2024-09-19T09:16:45.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.210+0000 7f848f7fe640 1 --2- 172.21.15.120:0/2745140353 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f84880a32c0 0x7f84880a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:45.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.210+0000 7f84a6d31740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e2e5b7320 0x561e2e5b7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:45.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.210+0000 7f84a6d31740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e2e5b7c40 0x561e2e5c4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:45.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.210+0000 7f84a6d31740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e2e5c4890 0x561e2e5c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:45.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.210+0000 7f84a6d31740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561e2e51d2e0 con 0x561e2e5b7320 2024-09-19T09:16:45.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.210+0000 7f84a6d31740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561e2e4c4ce0 con 0x561e2e5b7c40 2024-09-19T09:16:45.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.210+0000 7f84a6d31740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561e2e5192a0 con 0x561e2e5c4890 2024-09-19T09:16:45.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.210+0000 7f849dd9b640 1 --2- 172.21.15.120:0/2745140353 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f84880a32c0 0x7f84880a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:45.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.210+0000 7f849e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e2e5b7320 0x561e2e5b7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:45.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.211+0000 7f849e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e2e5b7320 0x561e2e5b7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60442/0 (socket says 172.21.15.120:60442) 2024-09-19T09:16:45.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.211+0000 7f849e59c640 1 -- 172.21.15.120:0/3881666535 learned_addr learned my addr 172.21.15.120:0/3881666535 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:45.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.211+0000 7f848f7fe640 1 -- 172.21.15.120:0/2745140353 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f84980f06d0 con 0x7f84a0102ef0 2024-09-19T09:16:45.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.211+0000 7f849ed9d640 1 --2- 172.21.15.120:0/3881666535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e2e5c4890 0x561e2e5c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:45.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.211+0000 7f849dd9b640 1 --2- 172.21.15.120:0/3881666535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e2e5b7c40 0x561e2e5c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:45.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.211+0000 7f849e59c640 1 -- 172.21.15.120:0/3881666535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e2e5c4890 msgr2=0x561e2e5c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:45.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.211+0000 7f849e59c640 1 --2- 172.21.15.120:0/3881666535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e2e5c4890 0x561e2e5c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.211+0000 7f849e59c640 1 -- 172.21.15.120:0/3881666535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e2e5b7c40 msgr2=0x561e2e5c4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:45.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.211+0000 7f849e59c640 1 --2- 172.21.15.120:0/3881666535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e2e5b7c40 0x561e2e5c4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.211+0000 7f849e59c640 1 -- 172.21.15.120:0/3881666535 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e2e4a9440 con 0x561e2e5b7320 2024-09-19T09:16:45.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.211+0000 7f849dd9b640 1 --2- 172.21.15.120:0/3881666535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e2e5b7c40 0x561e2e5c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:16:45.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.211+0000 7f849e59c640 1 --2- 172.21.15.120:0/3881666535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e2e5b7320 0x561e2e5b7700 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f8490011750 tx=0x7f8490011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff2b834f06fd486 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:45.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.211+0000 7f849dd9b640 1 --2- 172.21.15.120:0/2745140353 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f84880a32c0 0x7f84880a5780 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x561e2e5b83f0 tx=0x7f8494009b20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:45.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.211+0000 7f848cff9640 1 -- 172.21.15.120:0/3881666535 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849001e070 con 0x561e2e5b7320 2024-09-19T09:16:45.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.211+0000 7f848cff9640 1 -- 172.21.15.120:0/3881666535 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f849000fba0 con 0x561e2e5b7320 2024-09-19T09:16:45.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.211+0000 7f848cff9640 1 -- 172.21.15.120:0/3881666535 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849000e5f0 con 0x561e2e5b7320 2024-09-19T09:16:45.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.212+0000 7f84a6d31740 1 -- 172.21.15.120:0/3881666535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e2e5b7320 msgr2=0x561e2e5b7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:45.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.212+0000 7f84a6d31740 1 --2- 172.21.15.120:0/3881666535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e2e5b7320 0x561e2e5b7700 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f8490011750 tx=0x7f8490011b50 comp rx=0 tx=0).stop 2024-09-19T09:16:45.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.212+0000 7f84a6d31740 1 -- 172.21.15.120:0/3881666535 shutdown_connections 2024-09-19T09:16:45.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.212+0000 7f84a6d31740 1 --2- 172.21.15.120:0/3881666535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e2e5c4890 0x561e2e5c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.212+0000 7f84a6d31740 1 --2- 172.21.15.120:0/3881666535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e2e5b7c40 0x561e2e5c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.212+0000 7f84a6d31740 1 --2- 172.21.15.120:0/3881666535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e2e5b7320 0x561e2e5b7700 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.212+0000 7f84a6d31740 1 -- 172.21.15.120:0/3881666535 >> 172.21.15.120:0/3881666535 conn(0x561e2e5b3a30 msgr2=0x561e2e5b3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:45.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.212+0000 7f84a6d31740 1 -- 172.21.15.120:0/3881666535 shutdown_connections 2024-09-19T09:16:45.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.212+0000 7f84a6d31740 1 -- 172.21.15.120:0/3881666535 wait complete. 2024-09-19T09:16:45.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.212+0000 7f84a6d31740 1 -- start start 2024-09-19T09:16:45.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.213+0000 7f84a6d31740 1 Processor -- start 2024-09-19T09:16:45.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.213+0000 7f84a6d31740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e2e5b7320 0x561e2e5c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:45.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.213+0000 7f84a6d31740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e2e5b7c40 0x561e2e5c1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:45.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.213+0000 7f84a6d31740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e2e5c4890 0x561e2e5c1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:45.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.213+0000 7f84a6d31740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561e2e5c74c0 con 0x561e2e5b7320 2024-09-19T09:16:45.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.213+0000 7f84a6d31740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561e2e4d8050 con 0x561e2e5b7c40 2024-09-19T09:16:45.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.213+0000 7f84a6d31740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561e2e5c7640 con 0x561e2e5c4890 2024-09-19T09:16:45.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.213+0000 7f84a6d31740 10 client.0 _ll_register_callbacks cb 0x561e2e518160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:45.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.213+0000 7f849e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e2e5b7320 0x561e2e5c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:45.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.213+0000 7f849dd9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e2e5b7c40 0x561e2e5c1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:45.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.213+0000 7f849e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e2e5b7320 0x561e2e5c41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60456/0 (socket says 172.21.15.120:60456) 2024-09-19T09:16:45.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.213+0000 7f849ed9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e2e5c4890 0x561e2e5c1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:45.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.213+0000 7f849e59c640 1 -- 172.21.15.120:0/3313816308 learned_addr learned my addr 172.21.15.120:0/3313816308 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:45.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.213+0000 7f849e59c640 1 -- 172.21.15.120:0/3313816308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e2e5c4890 msgr2=0x561e2e5c1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:45.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.213+0000 7f849e59c640 1 --2- 172.21.15.120:0/3313816308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e2e5c4890 0x561e2e5c1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.213+0000 7f849e59c640 1 -- 172.21.15.120:0/3313816308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e2e5b7c40 msgr2=0x561e2e5c1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:45.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.213+0000 7f849e59c640 1 --2- 172.21.15.120:0/3313816308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e2e5b7c40 0x561e2e5c1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.213+0000 7f849e59c640 1 -- 172.21.15.120:0/3313816308 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e2e28e3c0 con 0x561e2e5b7320 2024-09-19T09:16:45.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.213+0000 7f849ed9d640 1 --2- 172.21.15.120:0/3313816308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e2e5c4890 0x561e2e5c1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:16:45.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.214+0000 7f849e59c640 1 --2- 172.21.15.120:0/3313816308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e2e5b7320 0x561e2e5c41a0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f8490019d10 tx=0x7f849000ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e667c0a9b3cee8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:45.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.214+0000 7f8485ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:16:45.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.214+0000 7f8485ffb640 1 -- 172.21.15.120:0/3313816308 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490026020 con 0x561e2e5b7320 2024-09-19T09:16:45.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.214+0000 7f8485ffb640 1 -- 172.21.15.120:0/3313816308 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f8490019430 con 0x561e2e5b7320 2024-09-19T09:16:45.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.214+0000 7f84a6d31740 1 -- 172.21.15.120:0/3313816308 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f849000fba0 con 0x561e2e5b7320 2024-09-19T09:16:45.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.214+0000 7f8485ffb640 1 -- 172.21.15.120:0/3313816308 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f849001e040 con 0x561e2e5b7320 2024-09-19T09:16:45.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.214+0000 7f84a6d31740 10 client.20307 fetch_fsmap learned FSMap version 36 2024-09-19T09:16:45.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.214+0000 7f84a6d31740 1 -- 172.21.15.120:0/3313816308 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561e2e5d2df0 con 0x561e2e5b7320 2024-09-19T09:16:45.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.214+0000 7f8485ffb640 1 -- 172.21.15.120:0/3313816308 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f849000fba0 con 0x561e2e5b7320 2024-09-19T09:16:45.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.214+0000 7f8485ffb640 1 -- 172.21.15.120:0/3313816308 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f8490021070 con 0x561e2e5b7320 2024-09-19T09:16:45.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.215+0000 7f84a6d31740 10 client.20307 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:16:45.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.215+0000 7f84a6d31740 10 client.20307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:16:45.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.215+0000 7f84a6d31740 20 client.20307 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:45.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.215+0000 7f84a6d31740 1 --2- 172.21.15.120:0/3313816308 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561e2e5d4140 0x561e2e5d6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:45.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.215+0000 7f84a6d31740 4 client.20307 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:16:45.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.215+0000 7f84a6d31740 1 -- 172.21.15.120:0/3313816308 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e2e52a850 con 0x561e2e5d4140 2024-09-19T09:16:45.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.215+0000 7f849dd9b640 1 --2- 172.21.15.120:0/3313816308 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561e2e5d4140 0x561e2e5d6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:45.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.215+0000 7f8485ffb640 10 client.20307 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:45.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.215+0000 7f849dd9b640 1 --2- 172.21.15.120:0/3313816308 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561e2e5d4140 0x561e2e5d6530 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26aad9869ab35f57 server_cookie=b6946573b6d27801 in_seq=0 out_seq=0 2024-09-19T09:16:45.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.217+0000 7f8485ffb640 1 -- 172.21.15.120:0/3313816308 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e2e52a850 con 0x561e2e5d4140 2024-09-19T09:16:45.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.217+0000 7f8485ffb640 10 client.20307 handle_command_reply: tid=0 2024-09-19T09:16:45.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.218+0000 7f84a6d31740 1 client.20307 shutdown 2024-09-19T09:16:45.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.218+0000 7f84a6d31740 1 -- 172.21.15.120:0/3313816308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e2e5b7320 msgr2=0x561e2e5c41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:45.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.218+0000 7f84a6d31740 1 --2- 172.21.15.120:0/3313816308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e2e5b7320 0x561e2e5c41a0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f8490019d10 tx=0x7f849000ff80 comp rx=0 tx=0).stop 2024-09-19T09:16:45.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.218+0000 7f84a6d31740 1 -- 172.21.15.120:0/3313816308 shutdown_connections 2024-09-19T09:16:45.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.218+0000 7f84a6d31740 1 --2- 172.21.15.120:0/3313816308 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561e2e5d4140 0x561e2e5d6530 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.218+0000 7f84a6d31740 1 --2- 172.21.15.120:0/3313816308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e2e5c4890 0x561e2e5c1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.218+0000 7f84a6d31740 1 --2- 172.21.15.120:0/3313816308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e2e5b7c40 0x561e2e5c1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.218+0000 7f84a6d31740 1 --2- 172.21.15.120:0/3313816308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e2e5b7320 0x561e2e5c41a0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.218+0000 7f84a6d31740 1 -- 172.21.15.120:0/3313816308 >> 172.21.15.120:0/3313816308 conn(0x561e2e5b3a30 msgr2=0x561e2e5b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:45.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.218+0000 7f8485ffb640 0 client.20307 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:45.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.218+0000 7f84a6d31740 1 -- 172.21.15.120:0/3313816308 shutdown_connections 2024-09-19T09:16:45.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.218+0000 7f84a6d31740 1 -- 172.21.15.120:0/3313816308 wait complete. 2024-09-19T09:16:45.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.218+0000 7f84a6d31740 20 client.20307 trim_cache size 0 max 16384 2024-09-19T09:16:45.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.238+0000 7f84a6d31740 1 Processor -- start 2024-09-19T09:16:45.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.238+0000 7f84a6d31740 1 -- start start 2024-09-19T09:16:45.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.239+0000 7f84a6d31740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e2e5b7c40 0x561e2e5d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:45.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.239+0000 7f84a6d31740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e2e5c4890 0x561e2e5dbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:45.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.239+0000 7f84a6d31740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e2e5bbe60 0x561e2e5dc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:45.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.239+0000 7f84a6d31740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01ab3a0 con 0x561e2e5b7c40 2024-09-19T09:16:45.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.239+0000 7f84a6d31740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01acf70 con 0x561e2e5bbe60 2024-09-19T09:16:45.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.239+0000 7f84a6d31740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01aba70 con 0x561e2e5c4890 2024-09-19T09:16:45.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.239+0000 7f849dd9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e2e5c4890 0x561e2e5dbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:45.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.239+0000 7f849e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e2e5b7c40 0x561e2e5d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:45.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.239+0000 7f849e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e2e5b7c40 0x561e2e5d0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60466/0 (socket says 172.21.15.120:60466) 2024-09-19T09:16:45.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.239+0000 7f849dd9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e2e5c4890 0x561e2e5dbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:42538/0 (socket says 172.21.15.120:42538) 2024-09-19T09:16:45.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.239+0000 7f849e59c640 1 -- 172.21.15.120:0/1219477663 learned_addr learned my addr 172.21.15.120:0/1219477663 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:45.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.239+0000 7f849ed9d640 1 --2- 172.21.15.120:0/1219477663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e2e5bbe60 0x561e2e5dc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:45.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.239+0000 7f849e59c640 1 -- 172.21.15.120:0/1219477663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e2e5c4890 msgr2=0x561e2e5dbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:45.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.239+0000 7f849e59c640 1 --2- 172.21.15.120:0/1219477663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e2e5c4890 0x561e2e5dbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.239+0000 7f849e59c640 1 -- 172.21.15.120:0/1219477663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e2e5bbe60 msgr2=0x561e2e5dc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:45.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.239+0000 7f849e59c640 1 --2- 172.21.15.120:0/1219477663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e2e5bbe60 0x561e2e5dc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.239+0000 7f849e59c640 1 -- 172.21.15.120:0/1219477663 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e2e5bf9a0 con 0x561e2e5b7c40 2024-09-19T09:16:45.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.240+0000 7f849e59c640 1 --2- 172.21.15.120:0/1219477663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e2e5b7c40 0x561e2e5d0760 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f8490005540 tx=0x7f8490009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32f46ecdcf83a70 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:45.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.240+0000 7f8487fff640 1 -- 172.21.15.120:0/1219477663 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490026070 con 0x561e2e5b7c40 2024-09-19T09:16:45.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.240+0000 7f8487fff640 1 -- 172.21.15.120:0/1219477663 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8490019430 con 0x561e2e5b7c40 2024-09-19T09:16:45.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.240+0000 7f8487fff640 1 -- 172.21.15.120:0/1219477663 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490004410 con 0x561e2e5b7c40 2024-09-19T09:16:45.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.241+0000 7f84a6d31740 1 -- 172.21.15.120:0/1219477663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e2e5b7c40 msgr2=0x561e2e5d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:45.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.241+0000 7f84a6d31740 1 --2- 172.21.15.120:0/1219477663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e2e5b7c40 0x561e2e5d0760 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f8490005540 tx=0x7f8490009dc0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.242+0000 7f84a6d31740 1 -- 172.21.15.120:0/1219477663 shutdown_connections 2024-09-19T09:16:45.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.242+0000 7f84a6d31740 1 --2- 172.21.15.120:0/1219477663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e2e5bbe60 0x561e2e5dc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.242+0000 7f84a6d31740 1 --2- 172.21.15.120:0/1219477663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e2e5c4890 0x561e2e5dbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.242+0000 7f84a6d31740 1 --2- 172.21.15.120:0/1219477663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e2e5b7c40 0x561e2e5d0760 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.242+0000 7f84a6d31740 1 -- 172.21.15.120:0/1219477663 >> 172.21.15.120:0/1219477663 conn(0x561e2e5b3a30 msgr2=0x561e2e5bcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:45.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.242+0000 7f84a6d31740 1 -- 172.21.15.120:0/1219477663 shutdown_connections 2024-09-19T09:16:45.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.242+0000 7f84a6d31740 1 -- 172.21.15.120:0/1219477663 wait complete. 2024-09-19T09:16:45.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.243+0000 7f84a6d31740 1 -- start start 2024-09-19T09:16:45.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.243+0000 7f84a6d31740 1 Processor -- start 2024-09-19T09:16:45.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.243+0000 7f84a6d31740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e2e5b7c40 0x561e2e5bad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:45.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.243+0000 7f84a6d31740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e2e5bbe60 0x561e2e5bb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:45.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.244+0000 7f84a6d31740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e2e5c4890 0x561e2e5bb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:45.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.244+0000 7f84a6d31740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01ac8a0 con 0x561e2e5bbe60 2024-09-19T09:16:45.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.244+0000 7f84a6d31740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01ac140 con 0x561e2e5b7c40 2024-09-19T09:16:45.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.244+0000 7f84a6d31740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01a9380 con 0x561e2e5c4890 2024-09-19T09:16:45.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.244+0000 7f849dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e2e5bbe60 0x561e2e5bb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:45.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.244+0000 7f84a6d31740 10 client.0 _ll_register_callbacks cb 0x7f849001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:45.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.244+0000 7f849dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e2e5bbe60 0x561e2e5bb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60474/0 (socket says 172.21.15.120:60474) 2024-09-19T09:16:45.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.244+0000 7f849dd9b640 1 -- 172.21.15.120:0/272046072 learned_addr learned my addr 172.21.15.120:0/272046072 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:45.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.244+0000 7f849ed9d640 1 --2- 172.21.15.120:0/272046072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e2e5c4890 0x561e2e5bb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:45.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.244+0000 7f849e59c640 1 --2- 172.21.15.120:0/272046072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e2e5b7c40 0x561e2e5bad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:45.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.244+0000 7f849dd9b640 1 -- 172.21.15.120:0/272046072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e2e5c4890 msgr2=0x561e2e5bb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:45.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.244+0000 7f849dd9b640 1 --2- 172.21.15.120:0/272046072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e2e5c4890 0x561e2e5bb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.244+0000 7f849dd9b640 1 -- 172.21.15.120:0/272046072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e2e5b7c40 msgr2=0x561e2e5bad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:45.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.244+0000 7f849dd9b640 1 --2- 172.21.15.120:0/272046072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e2e5b7c40 0x561e2e5bad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.244+0000 7f849dd9b640 1 -- 172.21.15.120:0/272046072 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e2e5cb200 con 0x561e2e5bbe60 2024-09-19T09:16:45.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.245+0000 7f849dd9b640 1 --2- 172.21.15.120:0/272046072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e2e5bbe60 0x561e2e5bb280 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x561e2e5c0ce0 tx=0x7f84940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1692d72f2a8fdb0a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:45.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.245+0000 7f8485ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:16:45.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.245+0000 7f8485ffb640 1 -- 172.21.15.120:0/272046072 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8494032b10 con 0x561e2e5bbe60 2024-09-19T09:16:45.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.245+0000 7f8485ffb640 1 -- 172.21.15.120:0/272046072 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f8494032cb0 con 0x561e2e5bbe60 2024-09-19T09:16:45.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.246+0000 7f849e59c640 1 --2- 172.21.15.120:0/272046072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e2e5b7c40 0x561e2e5bad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:16:45.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.246+0000 7f84a6d31740 1 -- 172.21.15.120:0/272046072 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8490019430 con 0x561e2e5bbe60 2024-09-19T09:16:45.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.246+0000 7f8485ffb640 1 -- 172.21.15.120:0/272046072 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8494022510 con 0x561e2e5bbe60 2024-09-19T09:16:45.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.246+0000 7f84a6d31740 10 client.20313 fetch_fsmap learned FSMap version 36 2024-09-19T09:16:45.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.246+0000 7f84a6d31740 1 -- 172.21.15.120:0/272046072 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561e2e63d270 con 0x561e2e5bbe60 2024-09-19T09:16:45.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.247+0000 7f8485ffb640 1 -- 172.21.15.120:0/272046072 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8490019430 con 0x561e2e5bbe60 2024-09-19T09:16:45.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.247+0000 7f8485ffb640 1 -- 172.21.15.120:0/272046072 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f849401f040 con 0x561e2e5bbe60 2024-09-19T09:16:45.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.247+0000 7f84a6d31740 10 client.20313 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:16:45.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.247+0000 7f84a6d31740 10 client.20313 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:16:45.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.247+0000 7f84a6d31740 20 client.20313 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:45.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.247+0000 7f84a6d31740 1 --2- 172.21.15.120:0/272046072 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561e2e63dab0 0x561e2e63fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:45.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.247+0000 7f84a6d31740 4 client.20313 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:16:45.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.247+0000 7f84a6d31740 1 -- 172.21.15.120:0/272046072 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561e2e524d60 con 0x561e2e63dab0 2024-09-19T09:16:45.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.248+0000 7f849e59c640 1 --2- 172.21.15.120:0/272046072 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561e2e63dab0 0x561e2e63fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:45.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.248+0000 7f849e59c640 1 --2- 172.21.15.120:0/272046072 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561e2e63dab0 0x561e2e63fed0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37f5d703fa17efdb server_cookie=cab3ed82aa8fcfa2 in_seq=0 out_seq=0 2024-09-19T09:16:45.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.248+0000 7f8485ffb640 10 client.20313 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:45.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.250+0000 7f8485ffb640 1 -- 172.21.15.120:0/272046072 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x561e2e524d60 con 0x561e2e63dab0 2024-09-19T09:16:45.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.250+0000 7f8485ffb640 10 client.20313 handle_command_reply: tid=0 2024-09-19T09:16:45.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.250+0000 7f84a6d31740 1 client.20313 shutdown 2024-09-19T09:16:45.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.250+0000 7f84a6d31740 1 -- 172.21.15.120:0/272046072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e2e5bbe60 msgr2=0x561e2e5bb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:45.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.250+0000 7f84a6d31740 1 --2- 172.21.15.120:0/272046072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e2e5bbe60 0x561e2e5bb280 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x561e2e5c0ce0 tx=0x7f84940046c0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.250+0000 7f84a6d31740 1 -- 172.21.15.120:0/272046072 shutdown_connections 2024-09-19T09:16:45.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.250+0000 7f84a6d31740 1 --2- 172.21.15.120:0/272046072 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561e2e63dab0 0x561e2e63fed0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.250+0000 7f84a6d31740 1 --2- 172.21.15.120:0/272046072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e2e5c4890 0x561e2e5bb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.250+0000 7f84a6d31740 1 --2- 172.21.15.120:0/272046072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e2e5bbe60 0x561e2e5bb280 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.250+0000 7f84a6d31740 1 --2- 172.21.15.120:0/272046072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e2e5b7c40 0x561e2e5bad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.250+0000 7f84a6d31740 1 -- 172.21.15.120:0/272046072 >> 172.21.15.120:0/272046072 conn(0x561e2e5b3a30 msgr2=0x561e2e5b2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:45.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.251+0000 7f8485ffb640 0 client.20313 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:45.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.251+0000 7f84a6d31740 1 -- 172.21.15.120:0/272046072 shutdown_connections 2024-09-19T09:16:45.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.251+0000 7f84a6d31740 1 -- 172.21.15.120:0/272046072 wait complete. 2024-09-19T09:16:45.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.251+0000 7f84a6d31740 20 client.20313 trim_cache size 0 max 16384 2024-09-19T09:16:45.253 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:16:45.253 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "scrub active (106 inodes in the stack)", 2024-09-19T09:16:45.253 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-19T09:16:45.253 INFO:teuthology.orchestra.run.smithi120.stdout: "6a755c3c-4ed2-4563-9a3f-b83035d8d031": { 2024-09-19T09:16:45.253 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-19T09:16:45.253 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "6a755c3c-4ed2-4563-9a3f-b83035d8d031", 2024-09-19T09:16:45.253 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-19T09:16:45.254 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:16:45.254 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:16:45.254 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:16:45.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.252+0000 7f84a4958640 1 -- 172.21.15.120:0/2745140353 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f84880a32c0 msgr2=0x7f84880a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:45.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.252+0000 7f84a4958640 1 --2- 172.21.15.120:0/2745140353 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f84880a32c0 0x7f84880a5780 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x561e2e5b83f0 tx=0x7f8494009b20 comp rx=0 tx=0).stop 2024-09-19T09:16:45.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.252+0000 7f84a4958640 1 -- 172.21.15.120:0/2745140353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84a0102ef0 msgr2=0x7f84a019bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:45.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.252+0000 7f84a4958640 1 --2- 172.21.15.120:0/2745140353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84a0102ef0 0x7f84a019bcd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8498005cc0 tx=0x7f849802f3a0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.254+0000 7f84a4958640 1 -- 172.21.15.120:0/2745140353 shutdown_connections 2024-09-19T09:16:45.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.254+0000 7f84a4958640 1 --2- 172.21.15.120:0/2745140353 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f84880a32c0 0x7f84880a5780 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.254+0000 7f84a4958640 1 --2- 172.21.15.120:0/2745140353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84a0102ef0 0x7f84a019bcd0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.254+0000 7f84a4958640 1 --2- 172.21.15.120:0/2745140353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84a0102570 0x7f84a0197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.254+0000 7f84a4958640 1 --2- 172.21.15.120:0/2745140353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84a0100e20 0x7f84a0197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:45.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.254+0000 7f84a4958640 1 -- 172.21.15.120:0/2745140353 >> 172.21.15.120:0/2745140353 conn(0x7f84a00fc7d0 msgr2=0x7f84a0106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:45.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.254+0000 7f84a4958640 1 -- 172.21.15.120:0/2745140353 shutdown_connections 2024-09-19T09:16:45.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:16:45.254+0000 7f84a4958640 1 -- 172.21.15.120:0/2745140353 wait complete. 2024-09-19T09:16:45.353 INFO:tasks.workunit.client.0.smithi120.stdout: 10 10393 18.36 MB/sec warmup 40 sec latency 236.516 ms 2024-09-19T09:16:46.353 INFO:tasks.workunit.client.0.smithi120.stdout: 10 11028 18.84 MB/sec warmup 41 sec latency 79.014 ms 2024-09-19T09:16:46.710 INFO:tasks.cephfs.filesystem:scrub status for tag:6a755c3c-4ed2-4563-9a3f-b83035d8d031 - {'path': '/', 'tag': '6a755c3c-4ed2-4563-9a3f-b83035d8d031', 'options': 'recursive,force'} 2024-09-19T09:16:47.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:46 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:16:47.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:46 smithi120 ceph-mon[31150]: pgmap v762: 97 pgs: 97 active+clean; 237 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.7 MiB/s wr, 497 op/s 2024-09-19T09:16:47.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:46 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:16:47.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:46 smithi196 ceph-mon[35150]: pgmap v762: 97 pgs: 97 active+clean; 237 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.7 MiB/s wr, 497 op/s 2024-09-19T09:16:47.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:46 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:16:47.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:46 smithi153 ceph-mon[35071]: pgmap v762: 97 pgs: 97 active+clean; 237 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.7 MiB/s wr, 497 op/s 2024-09-19T09:16:47.353 INFO:tasks.workunit.client.0.smithi120.stdout: 10 12042 19.39 MB/sec warmup 42 sec latency 120.565 ms 2024-09-19T09:16:48.353 INFO:tasks.workunit.client.0.smithi120.stdout: 10 12930 20.03 MB/sec warmup 43 sec latency 71.995 ms 2024-09-19T09:16:49.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:48 smithi196 ceph-mon[35150]: pgmap v763: 97 pgs: 97 active+clean; 238 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.5 MiB/s wr, 683 op/s 2024-09-19T09:16:49.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:48 smithi153 ceph-mon[35071]: pgmap v763: 97 pgs: 97 active+clean; 238 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.5 MiB/s wr, 683 op/s 2024-09-19T09:16:49.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:48 smithi120 ceph-mon[31150]: pgmap v763: 97 pgs: 97 active+clean; 238 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.5 MiB/s wr, 683 op/s 2024-09-19T09:16:49.353 INFO:tasks.workunit.client.0.smithi120.stdout: 10 13753 20.99 MB/sec warmup 44 sec latency 47.704 ms 2024-09-19T09:16:50.353 INFO:tasks.workunit.client.0.smithi120.stdout: 10 14363 21.39 MB/sec warmup 45 sec latency 55.966 ms 2024-09-19T09:16:51.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:50 smithi120 ceph-mon[31150]: pgmap v764: 97 pgs: 97 active+clean; 237 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 5.7 MiB/s wr, 651 op/s 2024-09-19T09:16:51.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:50 smithi196 ceph-mon[35150]: pgmap v764: 97 pgs: 97 active+clean; 237 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 5.7 MiB/s wr, 651 op/s 2024-09-19T09:16:51.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:50 smithi153 ceph-mon[35071]: pgmap v764: 97 pgs: 97 active+clean; 237 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 5.7 MiB/s wr, 651 op/s 2024-09-19T09:16:51.353 INFO:tasks.workunit.client.0.smithi120.stdout: 10 14747 21.27 MB/sec warmup 46 sec latency 103.746 ms 2024-09-19T09:16:52.353 INFO:tasks.workunit.client.0.smithi120.stdout: 10 15189 21.18 MB/sec warmup 47 sec latency 76.336 ms 2024-09-19T09:16:53.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:52 smithi196 ceph-mon[35150]: pgmap v765: 97 pgs: 97 active+clean; 244 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.1 MiB/s wr, 723 op/s 2024-09-19T09:16:53.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:52 smithi153 ceph-mon[35071]: pgmap v765: 97 pgs: 97 active+clean; 244 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.1 MiB/s wr, 723 op/s 2024-09-19T09:16:53.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:52 smithi120 ceph-mon[31150]: pgmap v765: 97 pgs: 97 active+clean; 244 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.1 MiB/s wr, 723 op/s 2024-09-19T09:16:53.353 INFO:tasks.workunit.client.0.smithi120.stdout: 10 15693 21.18 MB/sec warmup 48 sec latency 104.817 ms 2024-09-19T09:16:54.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:53 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:16:54.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:53 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:16:54.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:53 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:16:54.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:53 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:16:54.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:53 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:16:54.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:53 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:16:54.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:53 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:16:54.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:53 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:16:54.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:53 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:16:54.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:53 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:16:54.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:53 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:16:54.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:53 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:16:54.353 INFO:tasks.workunit.client.0.smithi120.stdout: 10 16532 21.52 MB/sec warmup 49 sec latency 42.806 ms 2024-09-19T09:16:55.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:54 smithi196 ceph-mon[35150]: pgmap v766: 97 pgs: 97 active+clean; 252 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.6 MiB/s wr, 765 op/s 2024-09-19T09:16:55.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:54 smithi153 ceph-mon[35071]: pgmap v766: 97 pgs: 97 active+clean; 252 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.6 MiB/s wr, 765 op/s 2024-09-19T09:16:55.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:54 smithi120 ceph-mon[31150]: pgmap v766: 97 pgs: 97 active+clean; 252 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.6 MiB/s wr, 765 op/s 2024-09-19T09:16:55.353 INFO:tasks.workunit.client.0.smithi120.stdout: 10 17451 22.09 MB/sec warmup 50 sec latency 45.500 ms 2024-09-19T09:16:56.353 INFO:tasks.workunit.client.0.smithi120.stdout: 10 18286 22.72 MB/sec warmup 51 sec latency 51.983 ms 2024-09-19T09:16:57.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:56 smithi196 ceph-mon[35150]: pgmap v767: 97 pgs: 97 active+clean; 258 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 4.6 MiB/s wr, 703 op/s 2024-09-19T09:16:57.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:56 smithi153 ceph-mon[35071]: pgmap v767: 97 pgs: 97 active+clean; 258 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 4.6 MiB/s wr, 703 op/s 2024-09-19T09:16:57.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:56 smithi120 ceph-mon[31150]: pgmap v767: 97 pgs: 97 active+clean; 258 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 4.6 MiB/s wr, 703 op/s 2024-09-19T09:16:57.353 INFO:tasks.workunit.client.0.smithi120.stdout: 10 18961 23.26 MB/sec warmup 52 sec latency 52.633 ms 2024-09-19T09:16:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.079+0000 7fd657fd6640 1 -- 172.21.15.120:0/1386867641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd650102660 msgr2=0x7fd650104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.079+0000 7fd657fd6640 1 --2- 172.21.15.120:0/1386867641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd650102660 0x7fd650104a50 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fd64c009900 tx=0x7fd64c02a410 comp rx=0 tx=0).stop 2024-09-19T09:16:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.079+0000 7fd657fd6640 1 -- 172.21.15.120:0/1386867641 shutdown_connections 2024-09-19T09:16:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.079+0000 7fd657fd6640 1 --2- 172.21.15.120:0/1386867641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6501078c0 0x7fd650109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.079+0000 7fd657fd6640 1 --2- 172.21.15.120:0/1386867641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd650104f90 0x7fd650107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.079+0000 7fd657fd6640 1 --2- 172.21.15.120:0/1386867641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd650102660 0x7fd650104a50 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.079+0000 7fd657fd6640 1 -- 172.21.15.120:0/1386867641 >> 172.21.15.120:0/1386867641 conn(0x7fd6500fc1e0 msgr2=0x7fd6500fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.079+0000 7fd657fd6640 1 -- 172.21.15.120:0/1386867641 shutdown_connections 2024-09-19T09:16:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.079+0000 7fd657fd6640 1 -- 172.21.15.120:0/1386867641 wait complete. 2024-09-19T09:16:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.080+0000 7fd657fd6640 1 Processor -- start 2024-09-19T09:16:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.080+0000 7fd657fd6640 1 -- start start 2024-09-19T09:16:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.080+0000 7fd657fd6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd650102660 0x7fd650197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.080+0000 7fd657fd6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd650104f90 0x7fd650198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.080+0000 7fd657fd6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6501078c0 0x7fd65019f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.080+0000 7fd657fd6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd65010d800 con 0x7fd6501078c0 2024-09-19T09:16:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.080+0000 7fd657fd6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd65010d680 con 0x7fd650102660 2024-09-19T09:16:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.080+0000 7fd657fd6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd65010d980 con 0x7fd650104f90 2024-09-19T09:16:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.080+0000 7fd656573640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6501078c0 0x7fd65019f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.080+0000 7fd655d72640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd650102660 0x7fd650197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.080+0000 7fd656573640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6501078c0 0x7fd65019f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49808/0 (socket says 172.21.15.120:49808) 2024-09-19T09:16:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.080+0000 7fd656573640 1 -- 172.21.15.120:0/3621446852 learned_addr learned my addr 172.21.15.120:0/3621446852 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.080+0000 7fd655571640 1 --2- 172.21.15.120:0/3621446852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd650104f90 0x7fd650198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.081+0000 7fd656573640 1 -- 172.21.15.120:0/3621446852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd650104f90 msgr2=0x7fd650198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.081+0000 7fd656573640 1 --2- 172.21.15.120:0/3621446852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd650104f90 0x7fd650198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.081+0000 7fd656573640 1 -- 172.21.15.120:0/3621446852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd650102660 msgr2=0x7fd650197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.081+0000 7fd656573640 1 --2- 172.21.15.120:0/3621446852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd650102660 0x7fd650197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.081+0000 7fd656573640 1 -- 172.21.15.120:0/3621446852 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd65019fa50 con 0x7fd6501078c0 2024-09-19T09:16:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.081+0000 7fd655d72640 1 --2- 172.21.15.120:0/3621446852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd650102660 0x7fd650197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:16:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.081+0000 7fd656573640 1 --2- 172.21.15.120:0/3621446852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6501078c0 0x7fd65019f3e0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fd64800eb40 tx=0x7fd64800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.081+0000 7fd646ffd640 1 -- 172.21.15.120:0/3621446852 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648004b20 con 0x7fd6501078c0 2024-09-19T09:16:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.081+0000 7fd646ffd640 1 -- 172.21.15.120:0/3621446852 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd648004cc0 con 0x7fd6501078c0 2024-09-19T09:16:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.081+0000 7fd657fd6640 1 -- 172.21.15.120:0/3621446852 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd65019fd40 con 0x7fd6501078c0 2024-09-19T09:16:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.081+0000 7fd646ffd640 1 -- 172.21.15.120:0/3621446852 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648010850 con 0x7fd6501078c0 2024-09-19T09:16:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.081+0000 7fd657fd6640 1 -- 172.21.15.120:0/3621446852 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6501a0280 con 0x7fd6501078c0 2024-09-19T09:16:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.083+0000 7fd646ffd640 1 -- 172.21.15.120:0/3621446852 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd648005020 con 0x7fd6501078c0 2024-09-19T09:16:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.083+0000 7fd65ae52740 1 Processor -- start 2024-09-19T09:16:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.084+0000 7fd65ae52740 1 -- start start 2024-09-19T09:16:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.084+0000 7fd65ae52740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555afcc86db0 0x555afcc871b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.084+0000 7fd65ae52740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555afcc876f0 0x555afcc8c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.084+0000 7fd65ae52740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555afcc8c780 0x555afcc8ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.084+0000 7fd65ae52740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555afcbe62f0 con 0x555afcc8c780 2024-09-19T09:16:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.084+0000 7fd65ae52740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555afcb8e270 con 0x555afcc876f0 2024-09-19T09:16:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.084+0000 7fd65ae52740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555afcbe22b0 con 0x555afcc86db0 2024-09-19T09:16:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.084+0000 7fd655d72640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555afcc876f0 0x555afcc8c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.084+0000 7fd646ffd640 1 --2- 172.21.15.120:0/3621446852 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd63c0a3370 0x7fd63c0a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.084+0000 7fd655571640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555afcc8c780 0x555afcc8ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.084+0000 7fd655d72640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555afcc876f0 0x555afcc8c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:44566/0 (socket says 172.21.15.120:44566) 2024-09-19T09:16:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.084+0000 7fd655d72640 1 -- 172.21.15.120:0/3284688589 learned_addr learned my addr 172.21.15.120:0/3284688589 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.084+0000 7fd655571640 1 --2- 172.21.15.120:0/3284688589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555afcc86db0 0x555afcc871b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.084+0000 7fd656573640 1 --2- 172.21.15.120:0/3621446852 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd63c0a3370 0x7fd63c0a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.084+0000 7fd646ffd640 1 -- 172.21.15.120:0/3621446852 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd6480cffa0 con 0x7fd6501078c0 2024-09-19T09:16:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.084+0000 7fd655d72640 1 -- 172.21.15.120:0/3284688589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555afcc86db0 msgr2=0x555afcc871b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.084+0000 7fd655d72640 1 --2- 172.21.15.120:0/3284688589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555afcc86db0 0x555afcc871b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.084+0000 7fd655d72640 1 -- 172.21.15.120:0/3284688589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555afcc8c780 msgr2=0x555afcc8ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.084+0000 7fd655d72640 1 --2- 172.21.15.120:0/3284688589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555afcc8c780 0x555afcc8ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.084+0000 7fd655d72640 1 -- 172.21.15.120:0/3284688589 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555afcb8d340 con 0x555afcc876f0 2024-09-19T09:16:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.084+0000 7fd655571640 1 --2- 172.21.15.120:0/3284688589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555afcc8c780 0x555afcc8ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:16:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.085+0000 7fd655d72640 1 --2- 172.21.15.120:0/3284688589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555afcc876f0 0x555afcc8c1b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd64c02a930 tx=0x7fd64c009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=114813e26102fa5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.085+0000 7fd656573640 1 --2- 172.21.15.120:0/3621446852 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd63c0a3370 0x7fd63c0a5830 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd648000f80 tx=0x7fd648067000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.085+0000 7fd63bfff640 1 -- 172.21.15.120:0/3284688589 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c040070 con 0x555afcc876f0 2024-09-19T09:16:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.085+0000 7fd63bfff640 1 -- 172.21.15.120:0/3284688589 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd64c0319b0 con 0x555afcc876f0 2024-09-19T09:16:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.085+0000 7fd63bfff640 1 -- 172.21.15.120:0/3284688589 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c035040 con 0x555afcc876f0 2024-09-19T09:16:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.086+0000 7fd65ae52740 1 -- 172.21.15.120:0/3284688589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555afcc876f0 msgr2=0x555afcc8c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.086+0000 7fd65ae52740 1 --2- 172.21.15.120:0/3284688589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555afcc876f0 0x555afcc8c1b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd64c02a930 tx=0x7fd64c009680 comp rx=0 tx=0).stop 2024-09-19T09:16:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.086+0000 7fd65ae52740 1 -- 172.21.15.120:0/3284688589 shutdown_connections 2024-09-19T09:16:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.086+0000 7fd65ae52740 1 --2- 172.21.15.120:0/3284688589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555afcc8c780 0x555afcc8ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.086+0000 7fd65ae52740 1 --2- 172.21.15.120:0/3284688589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555afcc876f0 0x555afcc8c1b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.086+0000 7fd65ae52740 1 --2- 172.21.15.120:0/3284688589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555afcc86db0 0x555afcc871b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.086+0000 7fd65ae52740 1 -- 172.21.15.120:0/3284688589 >> 172.21.15.120:0/3284688589 conn(0x555afcc7e010 msgr2=0x555afcc7e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:58.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.086+0000 7fd65ae52740 1 -- 172.21.15.120:0/3284688589 shutdown_connections 2024-09-19T09:16:58.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.086+0000 7fd65ae52740 1 -- 172.21.15.120:0/3284688589 wait complete. 2024-09-19T09:16:58.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.086+0000 7fd65ae52740 1 -- start start 2024-09-19T09:16:58.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.087+0000 7fd65ae52740 1 Processor -- start 2024-09-19T09:16:58.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.087+0000 7fd65ae52740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555afcc86db0 0x555afcc91000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:58.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.087+0000 7fd65ae52740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555afcc876f0 0x555afcc91540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:58.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.087+0000 7fd65ae52740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555afcc8c780 0x555afcc98580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:58.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.087+0000 7fd65ae52740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555afcc8f6f0 con 0x555afcc876f0 2024-09-19T09:16:58.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.087+0000 7fd65ae52740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555afcb6f290 con 0x555afcc86db0 2024-09-19T09:16:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.087+0000 7fd65ae52740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555afcc8f870 con 0x555afcc8c780 2024-09-19T09:16:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.087+0000 7fd65ae52740 10 client.0 _ll_register_callbacks cb 0x555afcba0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.087+0000 7fd655d72640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555afcc876f0 0x555afcc91540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.087+0000 7fd655571640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555afcc86db0 0x555afcc91000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.087+0000 7fd655d72640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555afcc876f0 0x555afcc91540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49822/0 (socket says 172.21.15.120:49822) 2024-09-19T09:16:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.087+0000 7fd655d72640 1 -- 172.21.15.120:0/98012997 learned_addr learned my addr 172.21.15.120:0/98012997 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.087+0000 7fd655571640 1 --2- 172.21.15.120:0/98012997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555afcc8c780 0x555afcc98580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.087+0000 7fd655d72640 1 -- 172.21.15.120:0/98012997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555afcc8c780 msgr2=0x555afcc98580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.087+0000 7fd655d72640 1 --2- 172.21.15.120:0/98012997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555afcc8c780 0x555afcc98580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.087+0000 7fd655d72640 1 -- 172.21.15.120:0/98012997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555afcc86db0 msgr2=0x555afcc91000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.087+0000 7fd655d72640 1 --2- 172.21.15.120:0/98012997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555afcc86db0 0x555afcc91000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.087+0000 7fd655d72640 1 -- 172.21.15.120:0/98012997 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555afc8f9210 con 0x555afcc876f0 2024-09-19T09:16:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.087+0000 7fd655571640 1 --2- 172.21.15.120:0/98012997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555afcc86db0 0x555afcc91000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:16:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.088+0000 7fd655d72640 1 --2- 172.21.15.120:0/98012997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555afcc876f0 0x555afcc91540 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fd64c000c00 tx=0x7fd64c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0598894cac982bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.088+0000 7fd6397fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:16:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.088+0000 7fd6397fa640 1 -- 172.21.15.120:0/98012997 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c04b020 con 0x555afcc876f0 2024-09-19T09:16:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.088+0000 7fd6397fa640 1 -- 172.21.15.120:0/98012997 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd64c0037f0 con 0x555afcc876f0 2024-09-19T09:16:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.088+0000 7fd65ae52740 1 -- 172.21.15.120:0/98012997 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd64c0319b0 con 0x555afcc876f0 2024-09-19T09:16:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.088+0000 7fd6397fa640 1 -- 172.21.15.120:0/98012997 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd64c035070 con 0x555afcc876f0 2024-09-19T09:16:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.088+0000 7fd65ae52740 10 client.20325 fetch_fsmap learned FSMap version 36 2024-09-19T09:16:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.088+0000 7fd65ae52740 1 -- 172.21.15.120:0/98012997 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555afc9c5620 con 0x555afcc876f0 2024-09-19T09:16:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.088+0000 7fd6397fa640 1 -- 172.21.15.120:0/98012997 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd64c0319b0 con 0x555afcc876f0 2024-09-19T09:16:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.088+0000 7fd6397fa640 1 -- 172.21.15.120:0/98012997 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fd64c03ccd0 con 0x555afcc876f0 2024-09-19T09:16:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.088+0000 7fd65ae52740 10 client.20325 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:16:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.088+0000 7fd65ae52740 10 client.20325 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:16:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.088+0000 7fd65ae52740 20 client.20325 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.088+0000 7fd65ae52740 1 --2- 172.21.15.120:0/98012997 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555afcc9c880 0x555afcc9ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.088+0000 7fd65ae52740 4 client.20325 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:16:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.088+0000 7fd65ae52740 1 -- 172.21.15.120:0/98012997 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555afcbf3a20 con 0x555afcc9c880 2024-09-19T09:16:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.089+0000 7fd655571640 1 --2- 172.21.15.120:0/98012997 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555afcc9c880 0x555afcc9ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.089+0000 7fd6397fa640 10 client.20325 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.089+0000 7fd655571640 1 --2- 172.21.15.120:0/98012997 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555afcc9c880 0x555afcc9ec70 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95084823d151aad2 server_cookie=ef184cf48c6b2390 in_seq=0 out_seq=0 2024-09-19T09:16:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.091+0000 7fd6397fa640 1 -- 172.21.15.120:0/98012997 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555afcbf3a20 con 0x555afcc9c880 2024-09-19T09:16:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.091+0000 7fd6397fa640 10 client.20325 handle_command_reply: tid=0 2024-09-19T09:16:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.091+0000 7fd65ae52740 1 client.20325 shutdown 2024-09-19T09:16:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.091+0000 7fd65ae52740 1 -- 172.21.15.120:0/98012997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555afcc876f0 msgr2=0x555afcc91540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.091+0000 7fd65ae52740 1 --2- 172.21.15.120:0/98012997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555afcc876f0 0x555afcc91540 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fd64c000c00 tx=0x7fd64c004270 comp rx=0 tx=0).stop 2024-09-19T09:16:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.091+0000 7fd65ae52740 1 -- 172.21.15.120:0/98012997 shutdown_connections 2024-09-19T09:16:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.091+0000 7fd65ae52740 1 --2- 172.21.15.120:0/98012997 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555afcc9c880 0x555afcc9ec70 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.091+0000 7fd65ae52740 1 --2- 172.21.15.120:0/98012997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555afcc8c780 0x555afcc98580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.091+0000 7fd65ae52740 1 --2- 172.21.15.120:0/98012997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555afcc876f0 0x555afcc91540 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.091+0000 7fd65ae52740 1 --2- 172.21.15.120:0/98012997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555afcc86db0 0x555afcc91000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.091+0000 7fd6397fa640 0 client.20325 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.091+0000 7fd65ae52740 1 -- 172.21.15.120:0/98012997 >> 172.21.15.120:0/98012997 conn(0x555afcc7e010 msgr2=0x555afcc7f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.091+0000 7fd65ae52740 1 -- 172.21.15.120:0/98012997 shutdown_connections 2024-09-19T09:16:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.091+0000 7fd65ae52740 1 -- 172.21.15.120:0/98012997 wait complete. 2024-09-19T09:16:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.091+0000 7fd65ae52740 20 client.20325 trim_cache size 0 max 16384 2024-09-19T09:16:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.109+0000 7fd65ae52740 1 Processor -- start 2024-09-19T09:16:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.109+0000 7fd65ae52740 1 -- start start 2024-09-19T09:16:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.109+0000 7fd65ae52740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555afcc86db0 0x555afcc7e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.109+0000 7fd65ae52740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555afcc876f0 0x555afcc98540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.109+0000 7fd65ae52740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555afcc8c780 0x555afcc7e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.109+0000 7fd65ae52740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd6501a6e40 con 0x555afcc876f0 2024-09-19T09:16:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.109+0000 7fd65ae52740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd6501a8a10 con 0x555afcc86db0 2024-09-19T09:16:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.109+0000 7fd65ae52740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd6501a7510 con 0x555afcc8c780 2024-09-19T09:16:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.109+0000 7fd655d72640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555afcc876f0 0x555afcc98540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.109+0000 7fd655d72640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555afcc876f0 0x555afcc98540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49824/0 (socket says 172.21.15.120:49824) 2024-09-19T09:16:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.109+0000 7fd655d72640 1 -- 172.21.15.120:0/3433850664 learned_addr learned my addr 172.21.15.120:0/3433850664 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.109+0000 7fd655571640 1 --2- 172.21.15.120:0/3433850664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555afcc86db0 0x555afcc7e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.109+0000 7fd655571640 1 --2- 172.21.15.120:0/3433850664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555afcc8c780 0x555afcc7e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.109+0000 7fd655d72640 1 -- 172.21.15.120:0/3433850664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555afcc8c780 msgr2=0x555afcc7e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.109+0000 7fd655d72640 1 --2- 172.21.15.120:0/3433850664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555afcc8c780 0x555afcc7e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.109+0000 7fd655d72640 1 -- 172.21.15.120:0/3433850664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555afcc86db0 msgr2=0x555afcc7e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.109+0000 7fd655d72640 1 --2- 172.21.15.120:0/3433850664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555afcc86db0 0x555afcc7e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.109+0000 7fd655d72640 1 -- 172.21.15.120:0/3433850664 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555afcc8bc10 con 0x555afcc876f0 2024-09-19T09:16:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.109+0000 7fd655d72640 1 --2- 172.21.15.120:0/3433850664 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555afcc876f0 0x555afcc98540 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fd64c044610 tx=0x7fd64c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b01b284acd806cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.110+0000 7fd63b7fe640 1 -- 172.21.15.120:0/3433850664 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c04b070 con 0x555afcc876f0 2024-09-19T09:16:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.110+0000 7fd63b7fe640 1 -- 172.21.15.120:0/3433850664 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd64c0035f0 con 0x555afcc876f0 2024-09-19T09:16:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.110+0000 7fd63b7fe640 1 -- 172.21.15.120:0/3433850664 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c040040 con 0x555afcc876f0 2024-09-19T09:16:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.110+0000 7fd65ae52740 1 -- 172.21.15.120:0/3433850664 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555afcc876f0 msgr2=0x555afcc98540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.110+0000 7fd65ae52740 1 --2- 172.21.15.120:0/3433850664 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555afcc876f0 0x555afcc98540 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fd64c044610 tx=0x7fd64c005a20 comp rx=0 tx=0).stop 2024-09-19T09:16:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.110+0000 7fd65ae52740 1 -- 172.21.15.120:0/3433850664 shutdown_connections 2024-09-19T09:16:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.110+0000 7fd65ae52740 1 --2- 172.21.15.120:0/3433850664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555afcc8c780 0x555afcc7e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.110+0000 7fd65ae52740 1 --2- 172.21.15.120:0/3433850664 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555afcc876f0 0x555afcc98540 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.110+0000 7fd65ae52740 1 --2- 172.21.15.120:0/3433850664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555afcc86db0 0x555afcc7e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.110+0000 7fd65ae52740 1 -- 172.21.15.120:0/3433850664 >> 172.21.15.120:0/3433850664 conn(0x555afcc7e010 msgr2=0x555afcc90460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.111+0000 7fd65ae52740 1 -- 172.21.15.120:0/3433850664 shutdown_connections 2024-09-19T09:16:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.111+0000 7fd65ae52740 1 -- 172.21.15.120:0/3433850664 wait complete. 2024-09-19T09:16:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.111+0000 7fd65ae52740 1 -- start start 2024-09-19T09:16:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.111+0000 7fd65ae52740 1 Processor -- start 2024-09-19T09:16:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.111+0000 7fd65ae52740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555afcc86db0 0x555afcc957a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.111+0000 7fd65ae52740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555afcc876f0 0x555afcc95ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.111+0000 7fd65ae52740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555afcc8c780 0x555afcc96cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.111+0000 7fd65ae52740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd6501a8340 con 0x555afcc876f0 2024-09-19T09:16:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.111+0000 7fd65ae52740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd6501a7be0 con 0x555afcc8c780 2024-09-19T09:16:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.111+0000 7fd65ae52740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd6501a4e00 con 0x555afcc86db0 2024-09-19T09:16:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.111+0000 7fd65ae52740 10 client.0 _ll_register_callbacks cb 0x7fd640009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:16:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.112+0000 7fd655571640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555afcc86db0 0x555afcc957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.112+0000 7fd655571640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555afcc86db0 0x555afcc957a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:49968/0 (socket says 172.21.15.120:49968) 2024-09-19T09:16:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.112+0000 7fd655571640 1 -- 172.21.15.120:0/15609961 learned_addr learned my addr 172.21.15.120:0/15609961 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:16:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.112+0000 7fd655571640 1 --2- 172.21.15.120:0/15609961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555afcc8c780 0x555afcc96cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.112+0000 7fd655d72640 1 --2- 172.21.15.120:0/15609961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555afcc876f0 0x555afcc95ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.112+0000 7fd655571640 1 -- 172.21.15.120:0/15609961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555afcc8c780 msgr2=0x555afcc96cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.112+0000 7fd655571640 1 --2- 172.21.15.120:0/15609961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555afcc8c780 0x555afcc96cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.112+0000 7fd655571640 1 -- 172.21.15.120:0/15609961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555afcc876f0 msgr2=0x555afcc95ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:16:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.112+0000 7fd655571640 1 --2- 172.21.15.120:0/15609961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555afcc876f0 0x555afcc95ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.112+0000 7fd655571640 1 -- 172.21.15.120:0/15609961 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555afcc97300 con 0x555afcc86db0 2024-09-19T09:16:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.112+0000 7fd655571640 1 --2- 172.21.15.120:0/15609961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555afcc86db0 0x555afcc957a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd6400042e0 tx=0x7fd640005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f739c886085ccee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:16:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.112+0000 7fd6397fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:16:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.112+0000 7fd6397fa640 1 -- 172.21.15.120:0/15609961 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64001a040 con 0x555afcc86db0 2024-09-19T09:16:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.112+0000 7fd65ae52740 1 -- 172.21.15.120:0/15609961 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd64c0035f0 con 0x555afcc86db0 2024-09-19T09:16:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.112+0000 7fd6397fa640 1 -- 172.21.15.120:0/15609961 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd640005220 con 0x555afcc86db0 2024-09-19T09:16:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.112+0000 7fd6397fa640 1 -- 172.21.15.120:0/15609961 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd640005420 con 0x555afcc86db0 2024-09-19T09:16:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.112+0000 7fd65ae52740 10 client.29113 fetch_fsmap learned FSMap version 36 2024-09-19T09:16:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.113+0000 7fd65ae52740 1 -- 172.21.15.120:0/15609961 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555afcc98680 con 0x555afcc86db0 2024-09-19T09:16:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.113+0000 7fd6397fa640 1 -- 172.21.15.120:0/15609961 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd64c0035f0 con 0x555afcc86db0 2024-09-19T09:16:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.113+0000 7fd6397fa640 1 -- 172.21.15.120:0/15609961 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fd640038050 con 0x555afcc86db0 2024-09-19T09:16:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.113+0000 7fd65ae52740 10 client.29113 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:16:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.113+0000 7fd65ae52740 10 client.29113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:16:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.113+0000 7fd65ae52740 20 client.29113 populate_metadata read hostname 'smithi120' 2024-09-19T09:16:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.113+0000 7fd65ae52740 1 --2- 172.21.15.120:0/15609961 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555afcd07a70 0x555afcd09e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:16:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.113+0000 7fd65ae52740 4 client.29113 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:16:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.113+0000 7fd65ae52740 1 -- 172.21.15.120:0/15609961 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555afcbedd70 con 0x555afcd07a70 2024-09-19T09:16:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.113+0000 7fd655d72640 1 --2- 172.21.15.120:0/15609961 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555afcd07a70 0x555afcd09e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:16:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.113+0000 7fd655d72640 1 --2- 172.21.15.120:0/15609961 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555afcd07a70 0x555afcd09e90 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a63f891c91afa1d server_cookie=fab0893b4d44f0fb in_seq=0 out_seq=0 2024-09-19T09:16:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.113+0000 7fd6397fa640 10 client.29113 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:58.353 INFO:tasks.workunit.client.0.smithi120.stdout: 10 19386 23.16 MB/sec warmup 53 sec latency 151.218 ms 2024-09-19T09:16:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.581+0000 7fd6397fa640 1 -- 172.21.15.120:0/15609961 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555afcbedd70 con 0x555afcd07a70 2024-09-19T09:16:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.581+0000 7fd6397fa640 10 client.29113 handle_command_reply: tid=0 2024-09-19T09:16:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.581+0000 7fd65ae52740 1 client.29113 shutdown 2024-09-19T09:16:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.581+0000 7fd65ae52740 1 -- 172.21.15.120:0/15609961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555afcc86db0 msgr2=0x555afcc957a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.581+0000 7fd65ae52740 1 --2- 172.21.15.120:0/15609961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555afcc86db0 0x555afcc957a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd6400042e0 tx=0x7fd640005f90 comp rx=0 tx=0).stop 2024-09-19T09:16:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.581+0000 7fd65ae52740 1 -- 172.21.15.120:0/15609961 shutdown_connections 2024-09-19T09:16:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.581+0000 7fd65ae52740 1 --2- 172.21.15.120:0/15609961 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555afcd07a70 0x555afcd09e90 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.582+0000 7fd6397fa640 0 client.29113 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:16:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.582+0000 7fd65ae52740 1 --2- 172.21.15.120:0/15609961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555afcc8c780 0x555afcc96cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.582+0000 7fd65ae52740 1 --2- 172.21.15.120:0/15609961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555afcc876f0 0x555afcc95ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.582+0000 7fd65ae52740 1 --2- 172.21.15.120:0/15609961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555afcc86db0 0x555afcc957a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.582+0000 7fd65ae52740 1 -- 172.21.15.120:0/15609961 >> 172.21.15.120:0/15609961 conn(0x555afcc7e010 msgr2=0x555afcc7c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.582+0000 7fd65ae52740 1 -- 172.21.15.120:0/15609961 shutdown_connections 2024-09-19T09:16:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.582+0000 7fd65ae52740 1 -- 172.21.15.120:0/15609961 wait complete. 2024-09-19T09:16:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.582+0000 7fd65ae52740 20 client.29113 trim_cache size 0 max 16384 2024-09-19T09:16:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:16:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:16:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:16:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:16:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.583+0000 7fd657fd6640 1 -- 172.21.15.120:0/3621446852 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd63c0a3370 msgr2=0x7fd63c0a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.583+0000 7fd657fd6640 1 --2- 172.21.15.120:0/3621446852 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd63c0a3370 0x7fd63c0a5830 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd648000f80 tx=0x7fd648067000 comp rx=0 tx=0).stop 2024-09-19T09:16:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.583+0000 7fd657fd6640 1 -- 172.21.15.120:0/3621446852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6501078c0 msgr2=0x7fd65019f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:16:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.583+0000 7fd657fd6640 1 --2- 172.21.15.120:0/3621446852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6501078c0 0x7fd65019f3e0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fd64800eb40 tx=0x7fd64800c2d0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.583+0000 7fd657fd6640 1 -- 172.21.15.120:0/3621446852 shutdown_connections 2024-09-19T09:16:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.583+0000 7fd657fd6640 1 --2- 172.21.15.120:0/3621446852 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd63c0a3370 0x7fd63c0a5830 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.583+0000 7fd657fd6640 1 --2- 172.21.15.120:0/3621446852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6501078c0 0x7fd65019f3e0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.583+0000 7fd657fd6640 1 --2- 172.21.15.120:0/3621446852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd650104f90 0x7fd650198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.583+0000 7fd657fd6640 1 --2- 172.21.15.120:0/3621446852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd650102660 0x7fd650197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:16:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.583+0000 7fd657fd6640 1 -- 172.21.15.120:0/3621446852 >> 172.21.15.120:0/3621446852 conn(0x7fd6500fc1e0 msgr2=0x7fd650102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:16:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.583+0000 7fd657fd6640 1 -- 172.21.15.120:0/3621446852 shutdown_connections 2024-09-19T09:16:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:16:58.583+0000 7fd657fd6640 1 -- 172.21.15.120:0/3621446852 wait complete. 2024-09-19T09:16:59.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:16:58 smithi153 ceph-mon[35071]: pgmap v768: 97 pgs: 97 active+clean; 261 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.0 MiB/s wr, 648 op/s 2024-09-19T09:16:59.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:16:58 smithi120 ceph-mon[31150]: pgmap v768: 97 pgs: 97 active+clean; 261 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.0 MiB/s wr, 648 op/s 2024-09-19T09:16:59.353 INFO:tasks.workunit.client.0.smithi120.stdout: 10 19781 23.04 MB/sec warmup 54 sec latency 236.345 ms 2024-09-19T09:16:59.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:16:58 smithi196 ceph-mon[35150]: pgmap v768: 97 pgs: 97 active+clean; 261 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.0 MiB/s wr, 648 op/s 2024-09-19T09:17:00.353 INFO:tasks.workunit.client.0.smithi120.stdout: 10 20410 23.21 MB/sec warmup 55 sec latency 47.292 ms 2024-09-19T09:17:01.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:00 smithi120 ceph-mon[31150]: pgmap v769: 97 pgs: 97 active+clean; 263 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.9 MiB/s wr, 576 op/s 2024-09-19T09:17:01.240 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:00 smithi196 ceph-mon[35150]: pgmap v769: 97 pgs: 97 active+clean; 263 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.9 MiB/s wr, 576 op/s 2024-09-19T09:17:01.353 INFO:tasks.workunit.client.0.smithi120.stdout: 10 21163 23.26 MB/sec warmup 56 sec latency 115.677 ms 2024-09-19T09:17:01.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:00 smithi153 ceph-mon[35071]: pgmap v769: 97 pgs: 97 active+clean; 263 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.9 MiB/s wr, 576 op/s 2024-09-19T09:17:02.353 INFO:tasks.workunit.client.0.smithi120.stdout: 10 22080 23.75 MB/sec warmup 57 sec latency 39.321 ms 2024-09-19T09:17:03.242 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:02 smithi153 ceph-mon[35071]: pgmap v770: 97 pgs: 97 active+clean; 257 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.9 MiB/s wr, 573 op/s 2024-09-19T09:17:03.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:02 smithi120 ceph-mon[31150]: pgmap v770: 97 pgs: 97 active+clean; 257 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.9 MiB/s wr, 573 op/s 2024-09-19T09:17:03.354 INFO:tasks.workunit.client.0.smithi120.stdout: 10 23008 24.32 MB/sec warmup 58 sec latency 101.486 ms 2024-09-19T09:17:03.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:02 smithi196 ceph-mon[35150]: pgmap v770: 97 pgs: 97 active+clean; 257 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.9 MiB/s wr, 573 op/s 2024-09-19T09:17:04.353 INFO:tasks.workunit.client.0.smithi120.stdout: 10 23700 24.73 MB/sec warmup 59 sec latency 46.030 ms 2024-09-19T09:17:05.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:04 smithi120 ceph-mon[31150]: pgmap v771: 97 pgs: 97 active+clean; 266 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 4.5 MiB/s wr, 700 op/s 2024-09-19T09:17:05.354 INFO:tasks.workunit.client.0.smithi120.stdout: 10 24182 24.79 MB/sec warmup 60 sec latency 86.908 ms 2024-09-19T09:17:05.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:04 smithi196 ceph-mon[35150]: pgmap v771: 97 pgs: 97 active+clean; 266 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 4.5 MiB/s wr, 700 op/s 2024-09-19T09:17:05.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:04 smithi153 ceph-mon[35071]: pgmap v771: 97 pgs: 97 active+clean; 266 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 4.5 MiB/s wr, 700 op/s 2024-09-19T09:17:06.354 INFO:tasks.workunit.client.0.smithi120.stdout: 10 24693 24.73 MB/sec warmup 61 sec latency 97.630 ms 2024-09-19T09:17:07.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:06 smithi120 ceph-mon[31150]: pgmap v772: 97 pgs: 97 active+clean; 267 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.7 MiB/s wr, 607 op/s 2024-09-19T09:17:07.354 INFO:tasks.workunit.client.0.smithi120.stdout: 10 24994 24.57 MB/sec warmup 62 sec latency 363.691 ms 2024-09-19T09:17:07.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:06 smithi196 ceph-mon[35150]: pgmap v772: 97 pgs: 97 active+clean; 267 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.7 MiB/s wr, 607 op/s 2024-09-19T09:17:07.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:06 smithi153 ceph-mon[35071]: pgmap v772: 97 pgs: 97 active+clean; 267 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.7 MiB/s wr, 607 op/s 2024-09-19T09:17:08.354 INFO:tasks.workunit.client.0.smithi120.stdout: 10 25525 24.45 MB/sec warmup 63 sec latency 123.847 ms 2024-09-19T09:17:09.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:08 smithi120 ceph-mon[31150]: pgmap v773: 97 pgs: 97 active+clean; 282 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 5.2 MiB/s wr, 680 op/s 2024-09-19T09:17:09.354 INFO:tasks.workunit.client.0.smithi120.stdout: 10 26384 24.77 MB/sec warmup 64 sec latency 35.859 ms 2024-09-19T09:17:09.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:08 smithi196 ceph-mon[35150]: pgmap v773: 97 pgs: 97 active+clean; 282 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 5.2 MiB/s wr, 680 op/s 2024-09-19T09:17:09.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:08 smithi153 ceph-mon[35071]: pgmap v773: 97 pgs: 97 active+clean; 282 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 5.2 MiB/s wr, 680 op/s 2024-09-19T09:17:10.354 INFO:tasks.workunit.client.0.smithi120.stdout: 10 27280 25.28 MB/sec warmup 65 sec latency 35.710 ms 2024-09-19T09:17:11.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:10 smithi120 ceph-mon[31150]: pgmap v774: 97 pgs: 97 active+clean; 283 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 6.5 MiB/s wr, 721 op/s 2024-09-19T09:17:11.354 INFO:tasks.workunit.client.0.smithi120.stdout: 10 28006 25.55 MB/sec warmup 66 sec latency 50.266 ms 2024-09-19T09:17:11.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:10 smithi196 ceph-mon[35150]: pgmap v774: 97 pgs: 97 active+clean; 283 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 6.5 MiB/s wr, 721 op/s 2024-09-19T09:17:11.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:10 smithi153 ceph-mon[35071]: pgmap v774: 97 pgs: 97 active+clean; 283 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 6.5 MiB/s wr, 721 op/s 2024-09-19T09:17:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.714+0000 7f8a27b6d640 1 -- 172.21.15.120:0/170143389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a201001f0 msgr2=0x7f8a201005f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.714+0000 7f8a27b6d640 1 --2- 172.21.15.120:0/170143389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a201001f0 0x7f8a201005f0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00aa10 tx=0x7f8a1c02b720 comp rx=0 tx=0).stop 2024-09-19T09:17:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.714+0000 7f8a27b6d640 1 -- 172.21.15.120:0/170143389 shutdown_connections 2024-09-19T09:17:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.714+0000 7f8a27b6d640 1 --2- 172.21.15.120:0/170143389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a20101da0 0x7f8a20108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.714+0000 7f8a27b6d640 1 --2- 172.21.15.120:0/170143389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a201013e0 0x7f8a20101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.714+0000 7f8a27b6d640 1 --2- 172.21.15.120:0/170143389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a201001f0 0x7f8a201005f0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.714+0000 7f8a27b6d640 1 -- 172.21.15.120:0/170143389 >> 172.21.15.120:0/170143389 conn(0x7f8a200fb970 msgr2=0x7f8a200fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.714+0000 7f8a27b6d640 1 -- 172.21.15.120:0/170143389 shutdown_connections 2024-09-19T09:17:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.714+0000 7f8a27b6d640 1 -- 172.21.15.120:0/170143389 wait complete. 2024-09-19T09:17:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.715+0000 7f8a27b6d640 1 Processor -- start 2024-09-19T09:17:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.715+0000 7f8a27b6d640 1 -- start start 2024-09-19T09:17:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.715+0000 7f8a27b6d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a201001f0 0x7f8a2019a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.715+0000 7f8a27b6d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a201013e0 0x7f8a2019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.715+0000 7f8a27b6d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a20101da0 0x7f8a201a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.715+0000 7f8a27b6d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2010b620 con 0x7f8a201013e0 2024-09-19T09:17:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.715+0000 7f8a27b6d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2010b4a0 con 0x7f8a201001f0 2024-09-19T09:17:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.715+0000 7f8a27b6d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2010b7a0 con 0x7f8a20101da0 2024-09-19T09:17:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.715+0000 7f8a25108640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a201013e0 0x7f8a2019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.715+0000 7f8a25108640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a201013e0 0x7f8a2019a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57288/0 (socket says 172.21.15.120:57288) 2024-09-19T09:17:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.715+0000 7f8a25108640 1 -- 172.21.15.120:0/1045122872 learned_addr learned my addr 172.21.15.120:0/1045122872 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.715+0000 7f8a25909640 1 --2- 172.21.15.120:0/1045122872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a201001f0 0x7f8a2019a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.715+0000 7f8a2610a640 1 --2- 172.21.15.120:0/1045122872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a20101da0 0x7f8a201a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.715+0000 7f8a25108640 1 -- 172.21.15.120:0/1045122872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a20101da0 msgr2=0x7f8a201a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.715+0000 7f8a25108640 1 --2- 172.21.15.120:0/1045122872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a20101da0 0x7f8a201a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.715+0000 7f8a25108640 1 -- 172.21.15.120:0/1045122872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a201001f0 msgr2=0x7f8a2019a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.715+0000 7f8a25108640 1 --2- 172.21.15.120:0/1045122872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a201001f0 0x7f8a2019a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.716+0000 7f8a25108640 1 -- 172.21.15.120:0/1045122872 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a201a1ca0 con 0x7f8a201013e0 2024-09-19T09:17:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.716+0000 7f8a25909640 1 --2- 172.21.15.120:0/1045122872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a201001f0 0x7f8a2019a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:17:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.716+0000 7f8a25108640 1 --2- 172.21.15.120:0/1045122872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a201013e0 0x7f8a2019a5c0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f8a1000e9f0 tx=0x7f8a1000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.716+0000 7f8a16ffd640 1 -- 172.21.15.120:0/1045122872 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1000cd80 con 0x7f8a201013e0 2024-09-19T09:17:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.716+0000 7f8a27b6d640 1 -- 172.21.15.120:0/1045122872 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a201a1f90 con 0x7f8a201013e0 2024-09-19T09:17:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.716+0000 7f8a27b6d640 1 -- 172.21.15.120:0/1045122872 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a201a24a0 con 0x7f8a201013e0 2024-09-19T09:17:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.716+0000 7f8a16ffd640 1 -- 172.21.15.120:0/1045122872 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a10010910 con 0x7f8a201013e0 2024-09-19T09:17:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.716+0000 7f8a16ffd640 1 -- 172.21.15.120:0/1045122872 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a10026060 con 0x7f8a201013e0 2024-09-19T09:17:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.718+0000 7f8a2a9e5740 1 Processor -- start 2024-09-19T09:17:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.718+0000 7f8a16ffd640 1 -- 172.21.15.120:0/1045122872 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f8a10010ac0 con 0x7f8a201013e0 2024-09-19T09:17:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.719+0000 7f8a2a9e5740 1 -- start start 2024-09-19T09:17:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.719+0000 7f8a2a9e5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc3e7679a0 0x55dc3e767da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.719+0000 7f8a2a9e5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc3e7682e0 0x55dc3e76cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.719+0000 7f8a2a9e5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc3e76d370 0x55dc3e76f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.719+0000 7f8a2a9e5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dc3e6cb2f0 con 0x55dc3e76d370 2024-09-19T09:17:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.719+0000 7f8a2a9e5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55dc3e673270 con 0x55dc3e7682e0 2024-09-19T09:17:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.719+0000 7f8a2a9e5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dc3e6c72b0 con 0x55dc3e7679a0 2024-09-19T09:17:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.719+0000 7f8a2610a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc3e7679a0 0x55dc3e767da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.719+0000 7f8a2610a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc3e7679a0 0x55dc3e767da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:51466/0 (socket says 172.21.15.120:51466) 2024-09-19T09:17:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.719+0000 7f8a2610a640 1 -- 172.21.15.120:0/1823681898 learned_addr learned my addr 172.21.15.120:0/1823681898 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.719+0000 7f8a25909640 1 --2- 172.21.15.120:0/1823681898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc3e7682e0 0x55dc3e76cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.719+0000 7f8a25108640 1 --2- 172.21.15.120:0/1823681898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc3e76d370 0x55dc3e76f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.719+0000 7f8a16ffd640 1 --2- 172.21.15.120:0/1045122872 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a0c0a32f0 0x7f8a0c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.719+0000 7f8a2610a640 1 -- 172.21.15.120:0/1823681898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc3e7682e0 msgr2=0x55dc3e76cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.719+0000 7f8a2610a640 1 --2- 172.21.15.120:0/1823681898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc3e7682e0 0x55dc3e76cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.719+0000 7f8a2610a640 1 -- 172.21.15.120:0/1823681898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc3e76d370 msgr2=0x55dc3e76f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.719+0000 7f8a2610a640 1 --2- 172.21.15.120:0/1823681898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc3e76d370 0x55dc3e76f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.719+0000 7f8a2610a640 1 -- 172.21.15.120:0/1823681898 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc3e672340 con 0x55dc3e7679a0 2024-09-19T09:17:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.719+0000 7f8a25108640 1 --2- 172.21.15.120:0/1823681898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc3e76d370 0x55dc3e76f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:17:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.719+0000 7f8a16ffd640 1 -- 172.21.15.120:0/1045122872 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f8a100cfb70 con 0x7f8a201013e0 2024-09-19T09:17:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.719+0000 7f8a2610a640 1 --2- 172.21.15.120:0/1045122872 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a0c0a32f0 0x7f8a0c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.720+0000 7f8a2610a640 1 --2- 172.21.15.120:0/1823681898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc3e7679a0 0x55dc3e767da0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009fd0 tx=0x7f8a180118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab5b8805df1320b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.720+0000 7f8a0bfff640 1 -- 172.21.15.120:0/1823681898 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18004250 con 0x55dc3e7679a0 2024-09-19T09:17:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.720+0000 7f8a0bfff640 1 -- 172.21.15.120:0/1823681898 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a1800f4e0 con 0x55dc3e7679a0 2024-09-19T09:17:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.720+0000 7f8a0bfff640 1 -- 172.21.15.120:0/1823681898 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1801b690 con 0x55dc3e7679a0 2024-09-19T09:17:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.721+0000 7f8a2a9e5740 1 -- 172.21.15.120:0/1823681898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc3e7679a0 msgr2=0x55dc3e767da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.721+0000 7f8a2a9e5740 1 --2- 172.21.15.120:0/1823681898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc3e7679a0 0x55dc3e767da0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009fd0 tx=0x7f8a180118d0 comp rx=0 tx=0).stop 2024-09-19T09:17:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.721+0000 7f8a2610a640 1 --2- 172.21.15.120:0/1045122872 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a0c0a32f0 0x7f8a0c0a57b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8a18020000 tx=0x7f8a1801be10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.721+0000 7f8a2a9e5740 1 -- 172.21.15.120:0/1823681898 shutdown_connections 2024-09-19T09:17:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.721+0000 7f8a2a9e5740 1 --2- 172.21.15.120:0/1823681898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc3e76d370 0x55dc3e76f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.721+0000 7f8a2a9e5740 1 --2- 172.21.15.120:0/1823681898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc3e7682e0 0x55dc3e76cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.721+0000 7f8a2a9e5740 1 --2- 172.21.15.120:0/1823681898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc3e7679a0 0x55dc3e767da0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.721+0000 7f8a2a9e5740 1 -- 172.21.15.120:0/1823681898 >> 172.21.15.120:0/1823681898 conn(0x55dc3e763010 msgr2=0x55dc3e763410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.721+0000 7f8a2a9e5740 1 -- 172.21.15.120:0/1823681898 shutdown_connections 2024-09-19T09:17:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.721+0000 7f8a2a9e5740 1 -- 172.21.15.120:0/1823681898 wait complete. 2024-09-19T09:17:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.721+0000 7f8a2a9e5740 1 -- start start 2024-09-19T09:17:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.721+0000 7f8a2a9e5740 1 Processor -- start 2024-09-19T09:17:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.721+0000 7f8a2a9e5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc3e7679a0 0x55dc3e771ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.721+0000 7f8a2a9e5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc3e7682e0 0x55dc3e772020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.721+0000 7f8a2a9e5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc3e76d370 0x55dc3e779060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.721+0000 7f8a2a9e5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dc3e770340 con 0x55dc3e7679a0 2024-09-19T09:17:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.721+0000 7f8a2a9e5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55dc3e654290 con 0x55dc3e76d370 2024-09-19T09:17:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.721+0000 7f8a2a9e5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dc3e7704c0 con 0x55dc3e7682e0 2024-09-19T09:17:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.722+0000 7f8a2a9e5740 10 client.0 _ll_register_callbacks cb 0x55dc3e685140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:17:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.722+0000 7f8a25909640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc3e7679a0 0x55dc3e771ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.722+0000 7f8a2610a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc3e76d370 0x55dc3e779060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.722+0000 7f8a25909640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc3e7679a0 0x55dc3e771ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57306/0 (socket says 172.21.15.120:57306) 2024-09-19T09:17:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.722+0000 7f8a25909640 1 -- 172.21.15.120:0/3882395269 learned_addr learned my addr 172.21.15.120:0/3882395269 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.722+0000 7f8a25108640 1 --2- 172.21.15.120:0/3882395269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc3e7682e0 0x55dc3e772020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.722+0000 7f8a2610a640 1 -- 172.21.15.120:0/3882395269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc3e7682e0 msgr2=0x55dc3e772020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.722+0000 7f8a2610a640 1 --2- 172.21.15.120:0/3882395269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc3e7682e0 0x55dc3e772020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.722+0000 7f8a2610a640 1 -- 172.21.15.120:0/3882395269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc3e7679a0 msgr2=0x55dc3e771ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.722+0000 7f8a2610a640 1 --2- 172.21.15.120:0/3882395269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc3e7679a0 0x55dc3e771ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.722+0000 7f8a2610a640 1 -- 172.21.15.120:0/3882395269 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc3e3de210 con 0x55dc3e76d370 2024-09-19T09:17:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.722+0000 7f8a2610a640 1 --2- 172.21.15.120:0/3882395269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc3e76d370 0x55dc3e779060 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8a18006010 tx=0x7f8a18017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ddf7273ba29344c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.722+0000 7f8a097fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:17:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.722+0000 7f8a097fa640 1 -- 172.21.15.120:0/3882395269 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18036020 con 0x55dc3e76d370 2024-09-19T09:17:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.722+0000 7f8a097fa640 1 -- 172.21.15.120:0/3882395269 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f8a1800f690 con 0x55dc3e76d370 2024-09-19T09:17:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.722+0000 7f8a2a9e5740 1 -- 172.21.15.120:0/3882395269 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a1800f4e0 con 0x55dc3e76d370 2024-09-19T09:17:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.722+0000 7f8a097fa640 1 -- 172.21.15.120:0/3882395269 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8a1800f890 con 0x55dc3e76d370 2024-09-19T09:17:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.723+0000 7f8a2a9e5740 10 client.29294 fetch_fsmap learned FSMap version 36 2024-09-19T09:17:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.723+0000 7f8a2a9e5740 1 -- 172.21.15.120:0/3882395269 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55dc3e4aa620 con 0x55dc3e76d370 2024-09-19T09:17:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.723+0000 7f8a097fa640 1 -- 172.21.15.120:0/3882395269 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a1800f4e0 con 0x55dc3e76d370 2024-09-19T09:17:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.723+0000 7f8a097fa640 1 -- 172.21.15.120:0/3882395269 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f8a18004390 con 0x55dc3e76d370 2024-09-19T09:17:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.723+0000 7f8a2a9e5740 10 client.29294 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:17:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.723+0000 7f8a2a9e5740 10 client.29294 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:17:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.723+0000 7f8a2a9e5740 20 client.29294 populate_metadata read hostname 'smithi120' 2024-09-19T09:17:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.723+0000 7f8a2a9e5740 1 --2- 172.21.15.120:0/3882395269 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dc3e77d630 0x55dc3e77fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.723+0000 7f8a2a9e5740 4 client.29294 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:17:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.723+0000 7f8a2a9e5740 1 -- 172.21.15.120:0/3882395269 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc3e6d8a20 con 0x55dc3e77d630 2024-09-19T09:17:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.723+0000 7f8a25909640 1 --2- 172.21.15.120:0/3882395269 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dc3e77d630 0x55dc3e77fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.723+0000 7f8a097fa640 10 client.29294 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.723+0000 7f8a25909640 1 --2- 172.21.15.120:0/3882395269 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dc3e77d630 0x55dc3e77fac0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82c8d3e8127f4ac2 server_cookie=d6bf25eee3adb0da in_seq=0 out_seq=0 2024-09-19T09:17:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.725+0000 7f8a097fa640 1 -- 172.21.15.120:0/3882395269 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dc3e6d8a20 con 0x55dc3e77d630 2024-09-19T09:17:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.725+0000 7f8a097fa640 10 client.29294 handle_command_reply: tid=0 2024-09-19T09:17:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.725+0000 7f8a2a9e5740 1 client.29294 shutdown 2024-09-19T09:17:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.725+0000 7f8a2a9e5740 1 -- 172.21.15.120:0/3882395269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc3e76d370 msgr2=0x55dc3e779060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.725+0000 7f8a2a9e5740 1 --2- 172.21.15.120:0/3882395269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc3e76d370 0x55dc3e779060 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8a18006010 tx=0x7f8a18017000 comp rx=0 tx=0).stop 2024-09-19T09:17:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.725+0000 7f8a2a9e5740 1 -- 172.21.15.120:0/3882395269 shutdown_connections 2024-09-19T09:17:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.725+0000 7f8a2a9e5740 1 --2- 172.21.15.120:0/3882395269 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dc3e77d630 0x55dc3e77fac0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.725+0000 7f8a2a9e5740 1 --2- 172.21.15.120:0/3882395269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc3e76d370 0x55dc3e779060 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.725+0000 7f8a2a9e5740 1 --2- 172.21.15.120:0/3882395269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc3e7682e0 0x55dc3e772020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.725+0000 7f8a2a9e5740 1 --2- 172.21.15.120:0/3882395269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc3e7679a0 0x55dc3e771ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.725+0000 7f8a097fa640 0 client.29294 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.725+0000 7f8a2a9e5740 1 -- 172.21.15.120:0/3882395269 >> 172.21.15.120:0/3882395269 conn(0x55dc3e763010 msgr2=0x55dc3e764820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.725+0000 7f8a2a9e5740 1 -- 172.21.15.120:0/3882395269 shutdown_connections 2024-09-19T09:17:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.725+0000 7f8a2a9e5740 1 -- 172.21.15.120:0/3882395269 wait complete. 2024-09-19T09:17:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.725+0000 7f8a2a9e5740 20 client.29294 trim_cache size 0 max 16384 2024-09-19T09:17:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.747+0000 7f8a2a9e5740 1 Processor -- start 2024-09-19T09:17:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.747+0000 7f8a2a9e5740 1 -- start start 2024-09-19T09:17:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.747+0000 7f8a2a9e5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc3e7679a0 0x55dc3e7620b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.747+0000 7f8a2a9e5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc3e7682e0 0x55dc3e778f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.747+0000 7f8a2a9e5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc3e76d370 0x55dc3e784380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.748+0000 7f8a25909640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc3e7679a0 0x55dc3e7620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.748+0000 7f8a25909640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc3e7679a0 0x55dc3e7620b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:50310/0 (socket says 172.21.15.120:50310) 2024-09-19T09:17:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.748+0000 7f8a25108640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc3e7682e0 0x55dc3e778f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.748+0000 7f8a25108640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc3e7682e0 0x55dc3e778f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:51494/0 (socket says 172.21.15.120:51494) 2024-09-19T09:17:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.748+0000 7f8a25909640 1 -- 172.21.15.120:0/1881489108 learned_addr learned my addr 172.21.15.120:0/1881489108 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.748+0000 7f8a2610a640 1 --2- 172.21.15.120:0/1881489108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc3e76d370 0x55dc3e784380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.748+0000 7f8a2a9e5740 1 -- 172.21.15.120:0/1881489108 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0400e4d0 con 0x55dc3e76d370 2024-09-19T09:17:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.748+0000 7f8a2a9e5740 1 -- 172.21.15.120:0/1881489108 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8a040100a0 con 0x55dc3e7679a0 2024-09-19T09:17:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.748+0000 7f8a2a9e5740 1 -- 172.21.15.120:0/1881489108 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0400eba0 con 0x55dc3e7682e0 2024-09-19T09:17:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.748+0000 7f8a25909640 1 -- 172.21.15.120:0/1881489108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc3e7682e0 msgr2=0x55dc3e778f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.748+0000 7f8a25909640 1 --2- 172.21.15.120:0/1881489108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc3e7682e0 0x55dc3e778f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.748+0000 7f8a25909640 1 -- 172.21.15.120:0/1881489108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc3e76d370 msgr2=0x55dc3e784380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.748+0000 7f8a25909640 1 --2- 172.21.15.120:0/1881489108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc3e76d370 0x55dc3e784380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.748+0000 7f8a25909640 1 -- 172.21.15.120:0/1881489108 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc3e7687c0 con 0x55dc3e7679a0 2024-09-19T09:17:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.748+0000 7f8a25108640 1 --2- 172.21.15.120:0/1881489108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc3e7682e0 0x55dc3e778f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:17:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.748+0000 7f8a2610a640 1 --2- 172.21.15.120:0/1881489108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc3e76d370 0x55dc3e784380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:17:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.749+0000 7f8a25909640 1 --2- 172.21.15.120:0/1881489108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc3e7679a0 0x55dc3e7620b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00a340 tx=0x7f8a1c0306c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87840a47bd6fe5b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.749+0000 7f8a0b7fe640 1 -- 172.21.15.120:0/1881489108 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c036040 con 0x55dc3e7679a0 2024-09-19T09:17:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.749+0000 7f8a0b7fe640 1 -- 172.21.15.120:0/1881489108 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a1c005520 con 0x55dc3e7679a0 2024-09-19T09:17:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.749+0000 7f8a0b7fe640 1 -- 172.21.15.120:0/1881489108 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c004e40 con 0x55dc3e7679a0 2024-09-19T09:17:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.750+0000 7f8a2a9e5740 1 -- 172.21.15.120:0/1881489108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc3e7679a0 msgr2=0x55dc3e7620b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.750+0000 7f8a2a9e5740 1 --2- 172.21.15.120:0/1881489108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc3e7679a0 0x55dc3e7620b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00a340 tx=0x7f8a1c0306c0 comp rx=0 tx=0).stop 2024-09-19T09:17:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.750+0000 7f8a2a9e5740 1 -- 172.21.15.120:0/1881489108 shutdown_connections 2024-09-19T09:17:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.750+0000 7f8a2a9e5740 1 --2- 172.21.15.120:0/1881489108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc3e76d370 0x55dc3e784380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.750+0000 7f8a2a9e5740 1 --2- 172.21.15.120:0/1881489108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc3e7682e0 0x55dc3e778f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.750+0000 7f8a2a9e5740 1 --2- 172.21.15.120:0/1881489108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc3e7679a0 0x55dc3e7620b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.750+0000 7f8a2a9e5740 1 -- 172.21.15.120:0/1881489108 >> 172.21.15.120:0/1881489108 conn(0x55dc3e763010 msgr2=0x55dc3e7619a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.750+0000 7f8a2a9e5740 1 -- 172.21.15.120:0/1881489108 shutdown_connections 2024-09-19T09:17:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.750+0000 7f8a2a9e5740 1 -- 172.21.15.120:0/1881489108 wait complete. 2024-09-19T09:17:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.750+0000 7f8a2a9e5740 1 -- start start 2024-09-19T09:17:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.751+0000 7f8a2a9e5740 1 Processor -- start 2024-09-19T09:17:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.751+0000 7f8a2a9e5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc3e7679a0 0x55dc3e776230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.751+0000 7f8a2a9e5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc3e7682e0 0x55dc3e776770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.751+0000 7f8a25909640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc3e7679a0 0x55dc3e776230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.751+0000 7f8a25909640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc3e7679a0 0x55dc3e776230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57320/0 (socket says 172.21.15.120:57320) 2024-09-19T09:17:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.751+0000 7f8a2a9e5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc3e76d370 0x55dc3e777780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.751+0000 7f8a2a9e5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0400f9d0 con 0x55dc3e7679a0 2024-09-19T09:17:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.751+0000 7f8a2a9e5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0400f270 con 0x55dc3e7682e0 2024-09-19T09:17:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.751+0000 7f8a2a9e5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0400c490 con 0x55dc3e76d370 2024-09-19T09:17:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.751+0000 7f8a2a9e5740 10 client.0 _ll_register_callbacks cb 0x7f8a100d34c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:17:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.751+0000 7f8a25909640 1 -- 172.21.15.120:0/2888638510 learned_addr learned my addr 172.21.15.120:0/2888638510 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.752+0000 7f8a2610a640 1 --2- 172.21.15.120:0/2888638510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc3e76d370 0x55dc3e777780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.752+0000 7f8a25108640 1 --2- 172.21.15.120:0/2888638510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc3e7682e0 0x55dc3e776770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.752+0000 7f8a25909640 1 -- 172.21.15.120:0/2888638510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc3e76d370 msgr2=0x55dc3e777780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.752+0000 7f8a25909640 1 --2- 172.21.15.120:0/2888638510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc3e76d370 0x55dc3e777780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.752+0000 7f8a25909640 1 -- 172.21.15.120:0/2888638510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc3e7682e0 msgr2=0x55dc3e776770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.752+0000 7f8a25909640 1 --2- 172.21.15.120:0/2888638510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc3e7682e0 0x55dc3e776770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.752+0000 7f8a25909640 1 -- 172.21.15.120:0/2888638510 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc3e777d90 con 0x55dc3e7679a0 2024-09-19T09:17:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.752+0000 7f8a25909640 1 --2- 172.21.15.120:0/2888638510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc3e7679a0 0x55dc3e776230 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c02bf10 tx=0x7f8a1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5a0ca4a48285b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.752+0000 7f8a097fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:17:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.752+0000 7f8a097fa640 1 -- 172.21.15.120:0/2888638510 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c036070 con 0x55dc3e7679a0 2024-09-19T09:17:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.752+0000 7f8a2a9e5740 1 -- 172.21.15.120:0/2888638510 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a1c005520 con 0x55dc3e7679a0 2024-09-19T09:17:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.752+0000 7f8a097fa640 1 -- 172.21.15.120:0/2888638510 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f8a1c004bc0 con 0x55dc3e7679a0 2024-09-19T09:17:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.753+0000 7f8a097fa640 1 -- 172.21.15.120:0/2888638510 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8a1c004dc0 con 0x55dc3e7679a0 2024-09-19T09:17:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.753+0000 7f8a2a9e5740 10 client.20346 fetch_fsmap learned FSMap version 36 2024-09-19T09:17:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.753+0000 7f8a2a9e5740 1 -- 172.21.15.120:0/2888638510 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55dc3e779060 con 0x55dc3e7679a0 2024-09-19T09:17:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.753+0000 7f8a097fa640 1 -- 172.21.15.120:0/2888638510 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a1c005520 con 0x55dc3e7679a0 2024-09-19T09:17:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.753+0000 7f8a097fa640 1 -- 172.21.15.120:0/2888638510 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f8a1c03b040 con 0x55dc3e7679a0 2024-09-19T09:17:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.753+0000 7f8a2a9e5740 10 client.20346 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:17:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.753+0000 7f8a2a9e5740 10 client.20346 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:17:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.753+0000 7f8a2a9e5740 20 client.20346 populate_metadata read hostname 'smithi120' 2024-09-19T09:17:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.753+0000 7f8a2a9e5740 1 --2- 172.21.15.120:0/2888638510 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dc3e7e8770 0x55dc3e7eab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.753+0000 7f8a2a9e5740 4 client.20346 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:17:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.753+0000 7f8a2a9e5740 1 -- 172.21.15.120:0/2888638510 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dc3e6d2d70 con 0x55dc3e7e8770 2024-09-19T09:17:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.753+0000 7f8a25108640 1 --2- 172.21.15.120:0/2888638510 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dc3e7e8770 0x55dc3e7eab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.754+0000 7f8a097fa640 10 client.20346 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:11.754+0000 7f8a25108640 1 --2- 172.21.15.120:0/2888638510 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dc3e7e8770 0x55dc3e7eab90 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c33b3e3fc0100ab server_cookie=79369f8dfd718da0 in_seq=0 out_seq=0 2024-09-19T09:17:12.354 INFO:tasks.workunit.client.0.smithi120.stdout: 10 28220 25.34 MB/sec warmup 67 sec latency 308.213 ms 2024-09-19T09:17:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:12.387+0000 7f8a097fa640 1 -- 172.21.15.120:0/2888638510 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dc3e6d2d70 con 0x55dc3e7e8770 2024-09-19T09:17:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:12.387+0000 7f8a097fa640 10 client.20346 handle_command_reply: tid=0 2024-09-19T09:17:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:12.387+0000 7f8a2a9e5740 1 client.20346 shutdown 2024-09-19T09:17:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:12.387+0000 7f8a2a9e5740 1 -- 172.21.15.120:0/2888638510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc3e7679a0 msgr2=0x55dc3e776230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:12.387+0000 7f8a2a9e5740 1 --2- 172.21.15.120:0/2888638510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc3e7679a0 0x55dc3e776230 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c02bf10 tx=0x7f8a1c0046c0 comp rx=0 tx=0).stop 2024-09-19T09:17:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:12.387+0000 7f8a2a9e5740 1 -- 172.21.15.120:0/2888638510 shutdown_connections 2024-09-19T09:17:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:12.387+0000 7f8a2a9e5740 1 --2- 172.21.15.120:0/2888638510 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dc3e7e8770 0x55dc3e7eab90 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:12.387+0000 7f8a097fa640 0 client.20346 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:12.387+0000 7f8a2a9e5740 1 --2- 172.21.15.120:0/2888638510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc3e76d370 0x55dc3e777780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:12.388+0000 7f8a2a9e5740 1 --2- 172.21.15.120:0/2888638510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc3e7682e0 0x55dc3e776770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:12.388+0000 7f8a2a9e5740 1 --2- 172.21.15.120:0/2888638510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc3e7679a0 0x55dc3e776230 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:12.388+0000 7f8a2a9e5740 1 -- 172.21.15.120:0/2888638510 >> 172.21.15.120:0/2888638510 conn(0x55dc3e763010 msgr2=0x55dc3e764740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:12.388+0000 7f8a2a9e5740 1 -- 172.21.15.120:0/2888638510 shutdown_connections 2024-09-19T09:17:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:12.388+0000 7f8a2a9e5740 1 -- 172.21.15.120:0/2888638510 wait complete. 2024-09-19T09:17:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:12.388+0000 7f8a2a9e5740 20 client.20346 trim_cache size 0 max 16384 2024-09-19T09:17:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:17:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:17:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:17:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:17:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:12.389+0000 7f8a27b6d640 1 -- 172.21.15.120:0/1045122872 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a0c0a32f0 msgr2=0x7f8a0c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:12.389+0000 7f8a27b6d640 1 --2- 172.21.15.120:0/1045122872 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a0c0a32f0 0x7f8a0c0a57b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8a18020000 tx=0x7f8a1801be10 comp rx=0 tx=0).stop 2024-09-19T09:17:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:12.389+0000 7f8a27b6d640 1 -- 172.21.15.120:0/1045122872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a201013e0 msgr2=0x7f8a2019a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:12.389+0000 7f8a27b6d640 1 --2- 172.21.15.120:0/1045122872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a201013e0 0x7f8a2019a5c0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f8a1000e9f0 tx=0x7f8a1000eeb0 comp rx=0 tx=0).stop 2024-09-19T09:17:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:12.389+0000 7f8a27b6d640 1 -- 172.21.15.120:0/1045122872 shutdown_connections 2024-09-19T09:17:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:12.390+0000 7f8a27b6d640 1 --2- 172.21.15.120:0/1045122872 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a0c0a32f0 0x7f8a0c0a57b0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:12.390+0000 7f8a27b6d640 1 --2- 172.21.15.120:0/1045122872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a20101da0 0x7f8a201a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:12.390+0000 7f8a27b6d640 1 --2- 172.21.15.120:0/1045122872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a201013e0 0x7f8a2019a5c0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:12.390+0000 7f8a27b6d640 1 --2- 172.21.15.120:0/1045122872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a201001f0 0x7f8a2019a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:12.390+0000 7f8a27b6d640 1 -- 172.21.15.120:0/1045122872 >> 172.21.15.120:0/1045122872 conn(0x7f8a200fb970 msgr2=0x7f8a20104760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:12.390+0000 7f8a27b6d640 1 -- 172.21.15.120:0/1045122872 shutdown_connections 2024-09-19T09:17:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:12.390+0000 7f8a27b6d640 1 -- 172.21.15.120:0/1045122872 wait complete. 2024-09-19T09:17:12.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:12 smithi196 ceph-mon[35150]: pgmap v775: 97 pgs: 97 active+clean; 292 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 7.2 MiB/s wr, 683 op/s 2024-09-19T09:17:12.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:12 smithi153 ceph-mon[35071]: pgmap v775: 97 pgs: 97 active+clean; 292 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 7.2 MiB/s wr, 683 op/s 2024-09-19T09:17:13.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:12 smithi120 ceph-mon[31150]: pgmap v775: 97 pgs: 97 active+clean; 292 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 7.2 MiB/s wr, 683 op/s 2024-09-19T09:17:13.354 INFO:tasks.workunit.client.0.smithi120.stdout: 10 28687 25.46 MB/sec warmup 68 sec latency 63.545 ms 2024-09-19T09:17:13.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:13 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:17:13.937 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:13 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:17:14.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:13 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:17:14.354 INFO:tasks.workunit.client.0.smithi120.stdout: 10 29121 25.30 MB/sec warmup 69 sec latency 63.920 ms 2024-09-19T09:17:14.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:14 smithi196 ceph-mon[35150]: pgmap v776: 97 pgs: 97 active+clean; 281 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.9 MiB/s wr, 796 op/s 2024-09-19T09:17:14.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:14 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:17:14.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:14 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:17:14.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:14 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:17:14.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:14 smithi153 ceph-mon[35071]: pgmap v776: 97 pgs: 97 active+clean; 281 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.9 MiB/s wr, 796 op/s 2024-09-19T09:17:14.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:14 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:17:14.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:14 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:17:14.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:14 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:17:15.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:14 smithi120 ceph-mon[31150]: pgmap v776: 97 pgs: 97 active+clean; 281 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.9 MiB/s wr, 796 op/s 2024-09-19T09:17:15.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:14 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:17:15.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:14 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:17:15.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:14 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:17:15.354 INFO:tasks.workunit.client.0.smithi120.stdout: 10 29600 25.27 MB/sec warmup 70 sec latency 104.669 ms 2024-09-19T09:17:16.354 INFO:tasks.workunit.client.0.smithi120.stdout: 10 30399 25.52 MB/sec warmup 71 sec latency 70.970 ms 2024-09-19T09:17:16.712 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:17:17.013 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:17:17.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:16 smithi120 ceph-mon[31150]: pgmap v777: 97 pgs: 97 active+clean; 283 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 8.3 MiB/s wr, 678 op/s 2024-09-19T09:17:17.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:16 smithi196 ceph-mon[35150]: pgmap v777: 97 pgs: 97 active+clean; 283 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 8.3 MiB/s wr, 678 op/s 2024-09-19T09:17:17.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:16 smithi153 ceph-mon[35071]: pgmap v777: 97 pgs: 97 active+clean; 283 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 8.3 MiB/s wr, 678 op/s 2024-09-19T09:17:17.354 INFO:tasks.workunit.client.0.smithi120.stdout: 10 31122 25.48 MB/sec warmup 72 sec latency 178.647 ms 2024-09-19T09:17:17.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.748+0000 7f5193086640 1 -- 172.21.15.120:0/3310456089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f518c106f20 msgr2=0x7f518c10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:17.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.748+0000 7f5193086640 1 --2- 172.21.15.120:0/3310456089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f518c106f20 0x7f518c10eaa0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f5184009ca0 tx=0x7f518402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.748+0000 7f5193086640 1 -- 172.21.15.120:0/3310456089 shutdown_connections 2024-09-19T09:17:17.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.748+0000 7f5193086640 1 --2- 172.21.15.120:0/3310456089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f518c106f20 0x7f518c10eaa0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.748+0000 7f5193086640 1 --2- 172.21.15.120:0/3310456089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f518c1027a0 0x7f518c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.748+0000 7f5193086640 1 --2- 172.21.15.120:0/3310456089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f518c101df0 0x7f518c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.748+0000 7f5193086640 1 -- 172.21.15.120:0/3310456089 >> 172.21.15.120:0/3310456089 conn(0x7f518c0fd8f0 msgr2=0x7f518c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:17.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.748+0000 7f5193086640 1 -- 172.21.15.120:0/3310456089 shutdown_connections 2024-09-19T09:17:17.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.748+0000 7f5193086640 1 -- 172.21.15.120:0/3310456089 wait complete. 2024-09-19T09:17:17.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.749+0000 7f5193086640 1 Processor -- start 2024-09-19T09:17:17.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.749+0000 7f5193086640 1 -- start start 2024-09-19T09:17:17.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.749+0000 7f5193086640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f518c101df0 0x7f518c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:17.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.749+0000 7f5193086640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f518c1027a0 0x7f518c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:17.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.749+0000 7f5193086640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f518c106f20 0x7f518c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:17.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.749+0000 7f5193086640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f518c111f80 con 0x7f518c1027a0 2024-09-19T09:17:17.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.749+0000 7f5193086640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f518c111e00 con 0x7f518c101df0 2024-09-19T09:17:17.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.749+0000 7f5193086640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f518c112100 con 0x7f518c106f20 2024-09-19T09:17:17.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.749+0000 7f5191623640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f518c106f20 0x7f518c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:17.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.749+0000 7f5190e22640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f518c101df0 0x7f518c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:17.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.749+0000 7f5190e22640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f518c101df0 0x7f518c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:50346/0 (socket says 172.21.15.120:50346) 2024-09-19T09:17:17.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.749+0000 7f5191623640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f518c106f20 0x7f518c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:51526/0 (socket says 172.21.15.120:51526) 2024-09-19T09:17:17.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.749+0000 7f5190e22640 1 -- 172.21.15.120:0/970914599 learned_addr learned my addr 172.21.15.120:0/970914599 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:17.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.750+0000 7f518bfff640 1 --2- 172.21.15.120:0/970914599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f518c1027a0 0x7f518c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:17.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.750+0000 7f5191623640 1 -- 172.21.15.120:0/970914599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f518c101df0 msgr2=0x7f518c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:17.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.750+0000 7f5191623640 1 --2- 172.21.15.120:0/970914599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f518c101df0 0x7f518c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.750+0000 7f5191623640 1 -- 172.21.15.120:0/970914599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f518c1027a0 msgr2=0x7f518c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:17.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.750+0000 7f5191623640 1 --2- 172.21.15.120:0/970914599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f518c1027a0 0x7f518c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.750+0000 7f5191623640 1 -- 172.21.15.120:0/970914599 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518c1aad70 con 0x7f518c106f20 2024-09-19T09:17:17.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.750+0000 7f5191623640 1 --2- 172.21.15.120:0/970914599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f518c106f20 0x7f518c074f70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f518402ade0 tx=0x7f51840319f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:17.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.750+0000 7f5189ffb640 1 -- 172.21.15.120:0/970914599 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184003b70 con 0x7f518c106f20 2024-09-19T09:17:17.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.750+0000 7f5189ffb640 1 -- 172.21.15.120:0/970914599 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5184003d10 con 0x7f518c106f20 2024-09-19T09:17:17.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.751+0000 7f5189ffb640 1 -- 172.21.15.120:0/970914599 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518402fc40 con 0x7f518c106f20 2024-09-19T09:17:17.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.751+0000 7f5193086640 1 -- 172.21.15.120:0/970914599 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f518c1aafa0 con 0x7f518c106f20 2024-09-19T09:17:17.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.751+0000 7f5193086640 1 -- 172.21.15.120:0/970914599 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f518c1ab3a0 con 0x7f518c106f20 2024-09-19T09:17:17.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.754+0000 7f5189ffb640 1 -- 172.21.15.120:0/970914599 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5184004050 con 0x7f518c106f20 2024-09-19T09:17:17.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.754+0000 7f519545f740 1 Processor -- start 2024-09-19T09:17:17.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.754+0000 7f5189ffb640 1 --2- 172.21.15.120:0/970914599 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f517c0a32c0 0x7f517c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:17.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.755+0000 7f519545f740 1 -- start start 2024-09-19T09:17:17.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.755+0000 7f5189ffb640 1 -- 172.21.15.120:0/970914599 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f51840f0840 con 0x7f518c106f20 2024-09-19T09:17:17.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.755+0000 7f5190e22640 1 --2- 172.21.15.120:0/970914599 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f517c0a32c0 0x7f517c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:17.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.755+0000 7f519545f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560b71794a0 0x5560b7179880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:17.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.755+0000 7f519545f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560b7179dc0 0x5560b7184140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:17.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.755+0000 7f519545f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560b7184680 0x5560b7186a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:17.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.755+0000 7f519545f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5560b70df2e0 con 0x5560b7184680 2024-09-19T09:17:17.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.755+0000 7f519545f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5560b7086ce0 con 0x5560b7179dc0 2024-09-19T09:17:17.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.755+0000 7f519545f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5560b70db2a0 con 0x5560b71794a0 2024-09-19T09:17:17.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.755+0000 7f5191623640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560b7184680 0x5560b7186a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:17.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.755+0000 7f5191623640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560b7184680 0x5560b7186a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57332/0 (socket says 172.21.15.120:57332) 2024-09-19T09:17:17.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.755+0000 7f5191623640 1 -- 172.21.15.120:0/3432523601 learned_addr learned my addr 172.21.15.120:0/3432523601 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:17.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.755+0000 7f5190e22640 1 --2- 172.21.15.120:0/3432523601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560b71794a0 0x5560b7179880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:17.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.756+0000 7f5191623640 1 -- 172.21.15.120:0/3432523601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560b71794a0 msgr2=0x5560b7179880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:17.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.756+0000 7f5191623640 1 --2- 172.21.15.120:0/3432523601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560b71794a0 0x5560b7179880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.756+0000 7f5191623640 1 -- 172.21.15.120:0/3432523601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560b7179dc0 msgr2=0x5560b7184140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:17.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.756+0000 7f5191623640 1 --2- 172.21.15.120:0/3432523601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560b7179dc0 0x5560b7184140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.756+0000 7f5191623640 1 -- 172.21.15.120:0/3432523601 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560b706b440 con 0x5560b7184680 2024-09-19T09:17:17.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.756+0000 7f5190e22640 1 --2- 172.21.15.120:0/970914599 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f517c0a32c0 0x7f517c0a5780 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5180011630 tx=0x7f5180011a30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:17.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.756+0000 7f5191623640 1 --2- 172.21.15.120:0/3432523601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560b7184680 0x5560b7186a70 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f51840c55a0 tx=0x7f518402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4ab0f1a4261f6d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:17.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.756+0000 7f5176ffd640 1 -- 172.21.15.120:0/3432523601 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51840bc7c0 con 0x5560b7184680 2024-09-19T09:17:17.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.756+0000 7f5176ffd640 1 -- 172.21.15.120:0/3432523601 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5184036040 con 0x5560b7184680 2024-09-19T09:17:17.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.756+0000 7f5176ffd640 1 -- 172.21.15.120:0/3432523601 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184004730 con 0x5560b7184680 2024-09-19T09:17:17.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.757+0000 7f519545f740 1 -- 172.21.15.120:0/3432523601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560b7184680 msgr2=0x5560b7186a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:17.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.757+0000 7f519545f740 1 --2- 172.21.15.120:0/3432523601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560b7184680 0x5560b7186a70 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f51840c55a0 tx=0x7f518402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.757+0000 7f519545f740 1 -- 172.21.15.120:0/3432523601 shutdown_connections 2024-09-19T09:17:17.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.757+0000 7f519545f740 1 --2- 172.21.15.120:0/3432523601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560b7184680 0x5560b7186a70 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.757+0000 7f519545f740 1 --2- 172.21.15.120:0/3432523601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560b7179dc0 0x5560b7184140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.757+0000 7f519545f740 1 --2- 172.21.15.120:0/3432523601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560b71794a0 0x5560b7179880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.757+0000 7f519545f740 1 -- 172.21.15.120:0/3432523601 >> 172.21.15.120:0/3432523601 conn(0x5560b7175a30 msgr2=0x5560b7175e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:17.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.757+0000 7f519545f740 1 -- 172.21.15.120:0/3432523601 shutdown_connections 2024-09-19T09:17:17.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.757+0000 7f519545f740 1 -- 172.21.15.120:0/3432523601 wait complete. 2024-09-19T09:17:17.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.758+0000 7f519545f740 1 -- start start 2024-09-19T09:17:17.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.758+0000 7f519545f740 1 Processor -- start 2024-09-19T09:17:17.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.758+0000 7f519545f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560b71794a0 0x5560b7183f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:17.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.758+0000 7f519545f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560b7179dc0 0x5560b7181200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:17.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.758+0000 7f519545f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560b7184680 0x5560b7181740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:17.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.758+0000 7f519545f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5560b70df2e0 con 0x5560b71794a0 2024-09-19T09:17:17.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.758+0000 7f519545f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5560b709a050 con 0x5560b7179dc0 2024-09-19T09:17:17.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.758+0000 7f519545f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5560b7187010 con 0x5560b7184680 2024-09-19T09:17:17.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.758+0000 7f519545f740 10 client.0 _ll_register_callbacks cb 0x5560b70da160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:17:17.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.758+0000 7f5190e22640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560b71794a0 0x5560b7183f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:17.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.758+0000 7f518bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560b7179dc0 0x5560b7181200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:17.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.758+0000 7f5190e22640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560b71794a0 0x5560b7183f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57344/0 (socket says 172.21.15.120:57344) 2024-09-19T09:17:17.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.758+0000 7f5190e22640 1 -- 172.21.15.120:0/400341260 learned_addr learned my addr 172.21.15.120:0/400341260 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:17.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.759+0000 7f5191623640 1 --2- 172.21.15.120:0/400341260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560b7184680 0x5560b7181740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:17.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.759+0000 7f5190e22640 1 -- 172.21.15.120:0/400341260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560b7184680 msgr2=0x5560b7181740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:17.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.759+0000 7f5190e22640 1 --2- 172.21.15.120:0/400341260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560b7184680 0x5560b7181740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.759+0000 7f5190e22640 1 -- 172.21.15.120:0/400341260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560b7179dc0 msgr2=0x5560b7181200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:17.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.759+0000 7f5190e22640 1 --2- 172.21.15.120:0/400341260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560b7179dc0 0x5560b7181200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.759+0000 7f5190e22640 1 -- 172.21.15.120:0/400341260 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560b6e503c0 con 0x5560b71794a0 2024-09-19T09:17:17.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.759+0000 7f5190e22640 1 --2- 172.21.15.120:0/400341260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560b71794a0 0x5560b7183f50 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f518000f730 tx=0x7f5180004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf60879657657a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:17.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.759+0000 7f516bfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:17:17.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.759+0000 7f516bfff640 1 -- 172.21.15.120:0/400341260 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5180024070 con 0x5560b71794a0 2024-09-19T09:17:17.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.759+0000 7f516bfff640 1 -- 172.21.15.120:0/400341260 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f5180005220 con 0x5560b71794a0 2024-09-19T09:17:17.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.759+0000 7f519545f740 1 -- 172.21.15.120:0/400341260 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5184036040 con 0x5560b71794a0 2024-09-19T09:17:17.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.759+0000 7f516bfff640 1 -- 172.21.15.120:0/400341260 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f518001a040 con 0x5560b71794a0 2024-09-19T09:17:17.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.759+0000 7f519545f740 10 client.20355 fetch_fsmap learned FSMap version 36 2024-09-19T09:17:17.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.759+0000 7f519545f740 1 -- 172.21.15.120:0/400341260 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5560b7192750 con 0x5560b71794a0 2024-09-19T09:17:17.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.760+0000 7f516bfff640 1 -- 172.21.15.120:0/400341260 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5184036040 con 0x5560b71794a0 2024-09-19T09:17:17.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.760+0000 7f516bfff640 1 -- 172.21.15.120:0/400341260 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f5180004a00 con 0x5560b71794a0 2024-09-19T09:17:17.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.760+0000 7f519545f740 10 client.20355 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:17:17.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.760+0000 7f519545f740 10 client.20355 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:17:17.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.760+0000 7f519545f740 20 client.20355 populate_metadata read hostname 'smithi120' 2024-09-19T09:17:17.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.760+0000 7f519545f740 1 --2- 172.21.15.120:0/400341260 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5560b7193c60 0x5560b7196070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:17.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.760+0000 7f519545f740 4 client.20355 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:17:17.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.760+0000 7f519545f740 1 -- 172.21.15.120:0/400341260 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560b70ec850 con 0x5560b7193c60 2024-09-19T09:17:17.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.760+0000 7f518bfff640 1 --2- 172.21.15.120:0/400341260 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5560b7193c60 0x5560b7196070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:17.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.761+0000 7f516bfff640 10 client.20355 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:17.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.761+0000 7f518bfff640 1 --2- 172.21.15.120:0/400341260 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5560b7193c60 0x5560b7196070 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb91c2b1cec731b server_cookie=991f1bc92e388d5b in_seq=0 out_seq=0 2024-09-19T09:17:17.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.762+0000 7f516bfff640 1 -- 172.21.15.120:0/400341260 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5560b70ec850 con 0x5560b7193c60 2024-09-19T09:17:17.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.762+0000 7f516bfff640 10 client.20355 handle_command_reply: tid=0 2024-09-19T09:17:17.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.762+0000 7f519545f740 1 client.20355 shutdown 2024-09-19T09:17:17.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.763+0000 7f519545f740 1 -- 172.21.15.120:0/400341260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560b71794a0 msgr2=0x5560b7183f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:17.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.763+0000 7f519545f740 1 --2- 172.21.15.120:0/400341260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560b71794a0 0x5560b7183f50 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f518000f730 tx=0x7f5180004730 comp rx=0 tx=0).stop 2024-09-19T09:17:17.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.763+0000 7f519545f740 1 -- 172.21.15.120:0/400341260 shutdown_connections 2024-09-19T09:17:17.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.763+0000 7f519545f740 1 --2- 172.21.15.120:0/400341260 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5560b7193c60 0x5560b7196070 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.763+0000 7f516bfff640 0 client.20355 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:17.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.763+0000 7f519545f740 1 --2- 172.21.15.120:0/400341260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560b7184680 0x5560b7181740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.763+0000 7f519545f740 1 --2- 172.21.15.120:0/400341260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560b7179dc0 0x5560b7181200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.763+0000 7f519545f740 1 --2- 172.21.15.120:0/400341260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560b71794a0 0x5560b7183f50 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.763+0000 7f519545f740 1 -- 172.21.15.120:0/400341260 >> 172.21.15.120:0/400341260 conn(0x5560b7175a30 msgr2=0x5560b717b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:17.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.763+0000 7f519545f740 1 -- 172.21.15.120:0/400341260 shutdown_connections 2024-09-19T09:17:17.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.763+0000 7f519545f740 1 -- 172.21.15.120:0/400341260 wait complete. 2024-09-19T09:17:17.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.763+0000 7f519545f740 20 client.20355 trim_cache size 0 max 16384 2024-09-19T09:17:17.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.782+0000 7f519545f740 1 Processor -- start 2024-09-19T09:17:17.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.782+0000 7f519545f740 1 -- start start 2024-09-19T09:17:17.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.782+0000 7f519545f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560b71794a0 0x5560b7190060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:17.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.782+0000 7f519545f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560b7179dc0 0x5560b717c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:17.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.782+0000 7f519545f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560b7184680 0x5560b719b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:17.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.782+0000 7f519545f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1b3bc0 con 0x5560b7184680 2024-09-19T09:17:17.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.782+0000 7f519545f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1b5790 con 0x5560b7179dc0 2024-09-19T09:17:17.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.782+0000 7f519545f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1b4290 con 0x5560b71794a0 2024-09-19T09:17:17.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.782+0000 7f5191623640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560b7184680 0x5560b719b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:17.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.783+0000 7f5191623640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560b7184680 0x5560b719b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57360/0 (socket says 172.21.15.120:57360) 2024-09-19T09:17:17.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.783+0000 7f5190e22640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560b71794a0 0x5560b7190060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:17.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.783+0000 7f5191623640 1 -- 172.21.15.120:0/2564551495 learned_addr learned my addr 172.21.15.120:0/2564551495 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:17.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.783+0000 7f518bfff640 1 --2- 172.21.15.120:0/2564551495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560b7179dc0 0x5560b717c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:17.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.783+0000 7f5191623640 1 -- 172.21.15.120:0/2564551495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560b71794a0 msgr2=0x5560b7190060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:17.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.783+0000 7f5191623640 1 --2- 172.21.15.120:0/2564551495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560b71794a0 0x5560b7190060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.783+0000 7f5191623640 1 -- 172.21.15.120:0/2564551495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560b7179dc0 msgr2=0x5560b717c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:17.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.783+0000 7f5191623640 1 --2- 172.21.15.120:0/2564551495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560b7179dc0 0x5560b717c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.783+0000 7f5191623640 1 -- 172.21.15.120:0/2564551495 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560b7184280 con 0x5560b7184680 2024-09-19T09:17:17.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.783+0000 7f5191623640 1 --2- 172.21.15.120:0/2564551495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560b7184680 0x5560b719b980 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f51840bcdf0 tx=0x7f518402f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241730a84c9ab0e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:17.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.783+0000 7f51767fc640 1 -- 172.21.15.120:0/2564551495 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51840c1210 con 0x5560b7184680 2024-09-19T09:17:17.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.783+0000 7f51767fc640 1 -- 172.21.15.120:0/2564551495 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5184036040 con 0x5560b7184680 2024-09-19T09:17:17.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.783+0000 7f51767fc640 1 -- 172.21.15.120:0/2564551495 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51840b94d0 con 0x5560b7184680 2024-09-19T09:17:17.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.784+0000 7f519545f740 1 -- 172.21.15.120:0/2564551495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560b7184680 msgr2=0x5560b719b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:17.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.784+0000 7f519545f740 1 --2- 172.21.15.120:0/2564551495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560b7184680 0x5560b719b980 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f51840bcdf0 tx=0x7f518402f710 comp rx=0 tx=0).stop 2024-09-19T09:17:17.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.784+0000 7f519545f740 1 -- 172.21.15.120:0/2564551495 shutdown_connections 2024-09-19T09:17:17.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.784+0000 7f519545f740 1 --2- 172.21.15.120:0/2564551495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560b7184680 0x5560b719b980 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.784+0000 7f519545f740 1 --2- 172.21.15.120:0/2564551495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560b7179dc0 0x5560b717c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.784+0000 7f519545f740 1 --2- 172.21.15.120:0/2564551495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560b71794a0 0x5560b7190060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.784+0000 7f519545f740 1 -- 172.21.15.120:0/2564551495 >> 172.21.15.120:0/2564551495 conn(0x5560b7175a30 msgr2=0x5560b717c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:17.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.784+0000 7f519545f740 1 -- 172.21.15.120:0/2564551495 shutdown_connections 2024-09-19T09:17:17.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.784+0000 7f519545f740 1 -- 172.21.15.120:0/2564551495 wait complete. 2024-09-19T09:17:17.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.785+0000 7f519545f740 1 -- start start 2024-09-19T09:17:17.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.785+0000 7f519545f740 1 Processor -- start 2024-09-19T09:17:17.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.785+0000 7f519545f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560b71794a0 0x5560b7187ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:17.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.785+0000 7f519545f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560b7179dc0 0x5560b7188430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:17.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.785+0000 7f519545f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560b7184680 0x5560b7188970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:17.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.785+0000 7f519545f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1b50c0 con 0x5560b7179dc0 2024-09-19T09:17:17.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.785+0000 7f519545f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1b4960 con 0x5560b71794a0 2024-09-19T09:17:17.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.785+0000 7f519545f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1b1b80 con 0x5560b7184680 2024-09-19T09:17:17.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.785+0000 7f518bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560b7179dc0 0x5560b7188430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:17.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.785+0000 7f519545f740 10 client.0 _ll_register_callbacks cb 0x7f5180009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:17:17.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.785+0000 7f518bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560b7179dc0 0x5560b7188430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57370/0 (socket says 172.21.15.120:57370) 2024-09-19T09:17:17.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.785+0000 7f518bfff640 1 -- 172.21.15.120:0/2849862978 learned_addr learned my addr 172.21.15.120:0/2849862978 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:17.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.785+0000 7f5190e22640 1 --2- 172.21.15.120:0/2849862978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560b71794a0 0x5560b7187ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:17.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.786+0000 7f5191623640 1 --2- 172.21.15.120:0/2849862978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560b7184680 0x5560b7188970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:17.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.786+0000 7f5190e22640 1 -- 172.21.15.120:0/2849862978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560b7184680 msgr2=0x5560b7188970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:17.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.786+0000 7f5190e22640 1 --2- 172.21.15.120:0/2849862978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560b7184680 0x5560b7188970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.786+0000 7f5190e22640 1 -- 172.21.15.120:0/2849862978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560b7179dc0 msgr2=0x5560b7188430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:17.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.786+0000 7f5190e22640 1 --2- 172.21.15.120:0/2849862978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560b7179dc0 0x5560b7188430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.786+0000 7f5190e22640 1 -- 172.21.15.120:0/2849862978 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560b71fd950 con 0x5560b71794a0 2024-09-19T09:17:17.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.786+0000 7f518bfff640 1 --2- 172.21.15.120:0/2849862978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560b7179dc0 0x5560b7188430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:17:17.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.786+0000 7f5190e22640 1 --2- 172.21.15.120:0/2849862978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560b71794a0 0x5560b7187ef0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f51800097b0 tx=0x7f51800049d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aabc555e3a22837e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:17.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.786+0000 7f516bfff640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:17:17.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.786+0000 7f516bfff640 1 -- 172.21.15.120:0/2849862978 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5180037070 con 0x5560b71794a0 2024-09-19T09:17:17.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.786+0000 7f519545f740 1 -- 172.21.15.120:0/2849862978 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5184036040 con 0x5560b71794a0 2024-09-19T09:17:17.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.786+0000 7f516bfff640 1 -- 172.21.15.120:0/2849862978 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f518001fb20 con 0x5560b71794a0 2024-09-19T09:17:17.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.786+0000 7f516bfff640 1 -- 172.21.15.120:0/2849862978 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5180024040 con 0x5560b71794a0 2024-09-19T09:17:17.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.786+0000 7f519545f740 10 client.29315 fetch_fsmap learned FSMap version 36 2024-09-19T09:17:17.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.786+0000 7f519545f740 1 -- 172.21.15.120:0/2849862978 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5560b71fec10 con 0x5560b71794a0 2024-09-19T09:17:17.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.787+0000 7f516bfff640 1 -- 172.21.15.120:0/2849862978 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5184036040 con 0x5560b71794a0 2024-09-19T09:17:17.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.787+0000 7f516bfff640 1 -- 172.21.15.120:0/2849862978 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f5180005070 con 0x5560b71794a0 2024-09-19T09:17:17.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.787+0000 7f519545f740 10 client.29315 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:17:17.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.787+0000 7f519545f740 10 client.29315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:17:17.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.787+0000 7f519545f740 20 client.29315 populate_metadata read hostname 'smithi120' 2024-09-19T09:17:17.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.787+0000 7f519545f740 1 --2- 172.21.15.120:0/2849862978 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5560b71ff3a0 0x5560b72017c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:17.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.787+0000 7f519545f740 4 client.29315 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:17:17.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.787+0000 7f519545f740 1 -- 172.21.15.120:0/2849862978 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5560b70e6d60 con 0x5560b71ff3a0 2024-09-19T09:17:17.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.787+0000 7f518bfff640 1 --2- 172.21.15.120:0/2849862978 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5560b71ff3a0 0x5560b72017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:17.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.787+0000 7f516bfff640 10 client.29315 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:17.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.787+0000 7f518bfff640 1 --2- 172.21.15.120:0/2849862978 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5560b71ff3a0 0x5560b72017c0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5c1c847b3848b09 server_cookie=654a5adbd3fe4617 in_seq=0 out_seq=0 2024-09-19T09:17:17.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.791+0000 7f516bfff640 1 -- 172.21.15.120:0/2849862978 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5560b70e6d60 con 0x5560b71ff3a0 2024-09-19T09:17:17.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.791+0000 7f516bfff640 10 client.29315 handle_command_reply: tid=0 2024-09-19T09:17:17.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.791+0000 7f519545f740 1 client.29315 shutdown 2024-09-19T09:17:17.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.791+0000 7f519545f740 1 -- 172.21.15.120:0/2849862978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560b71794a0 msgr2=0x5560b7187ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:17.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.791+0000 7f519545f740 1 --2- 172.21.15.120:0/2849862978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560b71794a0 0x5560b7187ef0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f51800097b0 tx=0x7f51800049d0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.791+0000 7f519545f740 1 -- 172.21.15.120:0/2849862978 shutdown_connections 2024-09-19T09:17:17.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.791+0000 7f519545f740 1 --2- 172.21.15.120:0/2849862978 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5560b71ff3a0 0x5560b72017c0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.791+0000 7f519545f740 1 --2- 172.21.15.120:0/2849862978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560b7184680 0x5560b7188970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.791+0000 7f519545f740 1 --2- 172.21.15.120:0/2849862978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560b7179dc0 0x5560b7188430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.791+0000 7f516bfff640 0 client.29315 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:17.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.791+0000 7f519545f740 1 --2- 172.21.15.120:0/2849862978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560b71794a0 0x5560b7187ef0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.791+0000 7f519545f740 1 -- 172.21.15.120:0/2849862978 >> 172.21.15.120:0/2849862978 conn(0x5560b7175a30 msgr2=0x5560b717c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:17.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.791+0000 7f519545f740 1 -- 172.21.15.120:0/2849862978 shutdown_connections 2024-09-19T09:17:17.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.791+0000 7f519545f740 1 -- 172.21.15.120:0/2849862978 wait complete. 2024-09-19T09:17:17.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.791+0000 7f519545f740 20 client.29315 trim_cache size 0 max 16384 2024-09-19T09:17:17.798 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:17:17.798 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:17:17.798 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:17:17.798 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:17:17.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.792+0000 7f5193086640 1 -- 172.21.15.120:0/970914599 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f517c0a32c0 msgr2=0x7f517c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:17.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.792+0000 7f5193086640 1 --2- 172.21.15.120:0/970914599 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f517c0a32c0 0x7f517c0a5780 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5180011630 tx=0x7f5180011a30 comp rx=0 tx=0).stop 2024-09-19T09:17:17.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.792+0000 7f5193086640 1 -- 172.21.15.120:0/970914599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f518c106f20 msgr2=0x7f518c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:17.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.792+0000 7f5193086640 1 --2- 172.21.15.120:0/970914599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f518c106f20 0x7f518c074f70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f518402ade0 tx=0x7f51840319f0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.792+0000 7f5193086640 1 -- 172.21.15.120:0/970914599 shutdown_connections 2024-09-19T09:17:17.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.792+0000 7f5193086640 1 --2- 172.21.15.120:0/970914599 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f517c0a32c0 0x7f517c0a5780 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.792+0000 7f5193086640 1 --2- 172.21.15.120:0/970914599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f518c106f20 0x7f518c074f70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.792+0000 7f5193086640 1 --2- 172.21.15.120:0/970914599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f518c1027a0 0x7f518c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.792+0000 7f5193086640 1 --2- 172.21.15.120:0/970914599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f518c101df0 0x7f518c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:17.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.792+0000 7f5193086640 1 -- 172.21.15.120:0/970914599 >> 172.21.15.120:0/970914599 conn(0x7f518c0fd8f0 msgr2=0x7f518c0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:17.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.792+0000 7f5193086640 1 -- 172.21.15.120:0/970914599 shutdown_connections 2024-09-19T09:17:17.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:17.792+0000 7f5193086640 1 -- 172.21.15.120:0/970914599 wait complete. 2024-09-19T09:17:18.354 INFO:tasks.workunit.client.0.smithi120.stdout: 10 31815 25.81 MB/sec warmup 73 sec latency 99.779 ms 2024-09-19T09:17:18.934 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:17:18.935 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:17:19.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:18 smithi196 ceph-mon[35150]: pgmap v778: 97 pgs: 97 active+clean; 277 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.9 MiB/s wr, 854 op/s 2024-09-19T09:17:19.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:18 smithi153 ceph-mon[35071]: pgmap v778: 97 pgs: 97 active+clean; 277 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.9 MiB/s wr, 854 op/s 2024-09-19T09:17:19.254 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:17:19.270 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:18 smithi120 ceph-mon[31150]: pgmap v778: 97 pgs: 97 active+clean; 277 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.9 MiB/s wr, 854 op/s 2024-09-19T09:17:19.354 INFO:tasks.workunit.client.0.smithi120.stdout: 10 32697 26.22 MB/sec warmup 74 sec latency 64.899 ms 2024-09-19T09:17:19.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.835+0000 7f86f00c0640 1 -- 172.21.15.120:0/525798930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86e8102ef0 msgr2=0x7f86e810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:19.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.835+0000 7f86f00c0640 1 --2- 172.21.15.120:0/525798930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86e8102ef0 0x7f86e810da40 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009ca0 tx=0x7f86e002a950 comp rx=0 tx=0).stop 2024-09-19T09:17:19.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.835+0000 7f86f00c0640 1 -- 172.21.15.120:0/525798930 shutdown_connections 2024-09-19T09:17:19.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.835+0000 7f86f00c0640 1 --2- 172.21.15.120:0/525798930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86e8102ef0 0x7f86e810da40 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:19.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.835+0000 7f86f00c0640 1 --2- 172.21.15.120:0/525798930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86e8102570 0x7f86e81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:19.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.835+0000 7f86f00c0640 1 --2- 172.21.15.120:0/525798930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86e8100e20 0x7f86e8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:19.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.835+0000 7f86f00c0640 1 -- 172.21.15.120:0/525798930 >> 172.21.15.120:0/525798930 conn(0x7f86e80fc7d0 msgr2=0x7f86e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:19.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.835+0000 7f86f00c0640 1 -- 172.21.15.120:0/525798930 shutdown_connections 2024-09-19T09:17:19.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.835+0000 7f86f00c0640 1 -- 172.21.15.120:0/525798930 wait complete. 2024-09-19T09:17:19.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.835+0000 7f86f00c0640 1 Processor -- start 2024-09-19T09:17:19.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.835+0000 7f86f00c0640 1 -- start start 2024-09-19T09:17:19.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.835+0000 7f86f00c0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86e8100e20 0x7f86e81993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:19.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.835+0000 7f86f00c0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86e8102570 0x7f86e8199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:19.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.835+0000 7f86f00c0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86e8102ef0 0x7f86e8199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:19.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.835+0000 7f86f00c0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f86e8110f80 con 0x7f86e8100e20 2024-09-19T09:17:19.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.835+0000 7f86f00c0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f86e8110e00 con 0x7f86e8102ef0 2024-09-19T09:17:19.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.835+0000 7f86f00c0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f86e8111100 con 0x7f86e8102570 2024-09-19T09:17:19.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.836+0000 7f86ede5c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86e8100e20 0x7f86e81993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:19.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.836+0000 7f86ede5c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86e8100e20 0x7f86e81993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57392/0 (socket says 172.21.15.120:57392) 2024-09-19T09:17:19.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.836+0000 7f86ede5c640 1 -- 172.21.15.120:0/1242048476 learned_addr learned my addr 172.21.15.120:0/1242048476 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:19.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.836+0000 7f86ed65b640 1 --2- 172.21.15.120:0/1242048476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86e8102570 0x7f86e8199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:19.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.836+0000 7f86ee65d640 1 --2- 172.21.15.120:0/1242048476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86e8102ef0 0x7f86e8199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:19.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.836+0000 7f86ed65b640 1 -- 172.21.15.120:0/1242048476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86e8102ef0 msgr2=0x7f86e8199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:19.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.836+0000 7f86ed65b640 1 --2- 172.21.15.120:0/1242048476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86e8102ef0 0x7f86e8199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:19.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.836+0000 7f86ed65b640 1 -- 172.21.15.120:0/1242048476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86e8100e20 msgr2=0x7f86e81993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:19.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.836+0000 7f86ed65b640 1 --2- 172.21.15.120:0/1242048476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86e8100e20 0x7f86e81993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:19.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.836+0000 7f86ed65b640 1 -- 172.21.15.120:0/1242048476 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e81a47d0 con 0x7f86e8102570 2024-09-19T09:17:19.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.836+0000 7f86ed65b640 1 --2- 172.21.15.120:0/1242048476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86e8102570 0x7f86e8199910 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f86d800e9f0 tx=0x7f86d800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:19.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.836+0000 7f86deffd640 1 -- 172.21.15.120:0/1242048476 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d800ccc0 con 0x7f86e8102570 2024-09-19T09:17:19.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.837+0000 7f86f00c0640 1 -- 172.21.15.120:0/1242048476 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e81a4ac0 con 0x7f86e8102570 2024-09-19T09:17:19.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.837+0000 7f86deffd640 1 -- 172.21.15.120:0/1242048476 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86d8010910 con 0x7f86e8102570 2024-09-19T09:17:19.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.837+0000 7f86deffd640 1 -- 172.21.15.120:0/1242048476 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d8018960 con 0x7f86e8102570 2024-09-19T09:17:19.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.837+0000 7f86f00c0640 1 -- 172.21.15.120:0/1242048476 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e81a5050 con 0x7f86e8102570 2024-09-19T09:17:19.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.837+0000 7f86f00c0640 1 -- 172.21.15.120:0/1242048476 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86e8101200 con 0x7f86e8102570 2024-09-19T09:17:19.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.839+0000 7f86deffd640 1 -- 172.21.15.120:0/1242048476 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f86d8010ac0 con 0x7f86e8102570 2024-09-19T09:17:19.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.840+0000 7f86deffd640 1 --2- 172.21.15.120:0/1242048476 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f86d40a3240 0x7f86d40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:19.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.840+0000 7f86ede5c640 1 --2- 172.21.15.120:0/1242048476 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f86d40a3240 0x7f86d40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:19.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.840+0000 7f86deffd640 1 -- 172.21.15.120:0/1242048476 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f86d80cfd60 con 0x7f86e8102570 2024-09-19T09:17:19.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.840+0000 7f86ede5c640 1 --2- 172.21.15.120:0/1242048476 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f86d40a3240 0x7f86d40a5700 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009cd0 tx=0x7f86e4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:19.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:19.843+0000 7f86deffd640 1 -- 172.21.15.120:0/1242048476 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f86d809c430 con 0x7f86e8102570 2024-09-19T09:17:20.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:20.082+0000 7f86f00c0640 1 -- 172.21.15.120:0/1242048476 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86e8109090 con 0x7f86e8102570 2024-09-19T09:17:20.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:20.082+0000 7f86deffd640 1 -- 172.21.15.120:0/1242048476 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11993 (secure 0 0 0) 0x7f86d80a0730 con 0x7f86e8102570 2024-09-19T09:17:20.083 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:17:20.084 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 36 2024-09-19T09:17:20.084 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":36,"btime":"2024-09-19T09:16:33:442503+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:16:33.001618+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:17:20.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:20.084+0000 7f86f00c0640 1 -- 172.21.15.120:0/1242048476 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f86d40a3240 msgr2=0x7f86d40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:20.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:20.084+0000 7f86f00c0640 1 --2- 172.21.15.120:0/1242048476 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f86d40a3240 0x7f86d40a5700 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009cd0 tx=0x7f86e4005f90 comp rx=0 tx=0).stop 2024-09-19T09:17:20.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:20.084+0000 7f86f00c0640 1 -- 172.21.15.120:0/1242048476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86e8102570 msgr2=0x7f86e8199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:20.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:20.084+0000 7f86f00c0640 1 --2- 172.21.15.120:0/1242048476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86e8102570 0x7f86e8199910 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f86d800e9f0 tx=0x7f86d800eeb0 comp rx=0 tx=0).stop 2024-09-19T09:17:20.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:20.085+0000 7f86f00c0640 1 -- 172.21.15.120:0/1242048476 shutdown_connections 2024-09-19T09:17:20.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:20.085+0000 7f86f00c0640 1 --2- 172.21.15.120:0/1242048476 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f86d40a3240 0x7f86d40a5700 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:20.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:20.085+0000 7f86f00c0640 1 --2- 172.21.15.120:0/1242048476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86e8102ef0 0x7f86e8199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:20.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:20.085+0000 7f86f00c0640 1 --2- 172.21.15.120:0/1242048476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86e8102570 0x7f86e8199910 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:20.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:20.085+0000 7f86f00c0640 1 --2- 172.21.15.120:0/1242048476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86e8100e20 0x7f86e81993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:20.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:20.085+0000 7f86f00c0640 1 -- 172.21.15.120:0/1242048476 >> 172.21.15.120:0/1242048476 conn(0x7f86e80fc7d0 msgr2=0x7f86e81068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:20.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:20.085+0000 7f86f00c0640 1 -- 172.21.15.120:0/1242048476 shutdown_connections 2024-09-19T09:17:20.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:20.085+0000 7f86f00c0640 1 -- 172.21.15.120:0/1242048476 wait complete. 2024-09-19T09:17:20.355 INFO:tasks.workunit.client.0.smithi120.stdout: 10 33276 26.31 MB/sec warmup 75 sec latency 56.248 ms 2024-09-19T09:17:21.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:20 smithi120 ceph-mon[31150]: pgmap v779: 97 pgs: 97 active+clean; 281 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.4 MiB/s wr, 821 op/s 2024-09-19T09:17:21.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:20 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1242048476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:17:21.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:20 smithi196 ceph-mon[35150]: pgmap v779: 97 pgs: 97 active+clean; 281 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.4 MiB/s wr, 821 op/s 2024-09-19T09:17:21.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:20 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1242048476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:17:21.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:20 smithi153 ceph-mon[35071]: pgmap v779: 97 pgs: 97 active+clean; 281 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.4 MiB/s wr, 821 op/s 2024-09-19T09:17:21.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:20 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1242048476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:17:21.241 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:17:21.354 INFO:tasks.workunit.client.0.smithi120.stdout: 10 33804 26.34 MB/sec warmup 76 sec latency 66.971 ms 2024-09-19T09:17:21.549 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:17:22.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.092+0000 7f083bde7640 1 -- 172.21.15.120:0/3890998528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0834106f10 msgr2=0x7f083410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:22.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.092+0000 7f083bde7640 1 --2- 172.21.15.120:0/3890998528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0834106f10 0x7f083410ea90 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f082c009ca0 tx=0x7f082c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.093+0000 7f083bde7640 1 -- 172.21.15.120:0/3890998528 shutdown_connections 2024-09-19T09:17:22.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.093+0000 7f083bde7640 1 --2- 172.21.15.120:0/3890998528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0834106f10 0x7f083410ea90 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.093+0000 7f083bde7640 1 --2- 172.21.15.120:0/3890998528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0834102790 0x7f08341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.093+0000 7f083bde7640 1 --2- 172.21.15.120:0/3890998528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0834101de0 0x7f08341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.093+0000 7f083bde7640 1 -- 172.21.15.120:0/3890998528 >> 172.21.15.120:0/3890998528 conn(0x7f08340fd8c0 msgr2=0x7f08340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:22.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.093+0000 7f083bde7640 1 -- 172.21.15.120:0/3890998528 shutdown_connections 2024-09-19T09:17:22.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.093+0000 7f083bde7640 1 -- 172.21.15.120:0/3890998528 wait complete. 2024-09-19T09:17:22.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.093+0000 7f083bde7640 1 Processor -- start 2024-09-19T09:17:22.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.093+0000 7f083bde7640 1 -- start start 2024-09-19T09:17:22.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.094+0000 7f083bde7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0834101de0 0x7f08340744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:22.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.094+0000 7f083bde7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0834102790 0x7f0834074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:22.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.094+0000 7f083bde7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0834106f10 0x7f0834074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:22.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.094+0000 7f083bde7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0834111f70 con 0x7f0834102790 2024-09-19T09:17:22.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.094+0000 7f083bde7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0834111df0 con 0x7f0834106f10 2024-09-19T09:17:22.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.094+0000 7f083bde7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f08341120f0 con 0x7f0834101de0 2024-09-19T09:17:22.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.094+0000 7f0839382640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0834102790 0x7f0834074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:22.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.094+0000 7f0839382640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0834102790 0x7f0834074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36954/0 (socket says 172.21.15.120:36954) 2024-09-19T09:17:22.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.094+0000 7f0839382640 1 -- 172.21.15.120:0/3319727137 learned_addr learned my addr 172.21.15.120:0/3319727137 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:22.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.094+0000 7f0839b83640 1 --2- 172.21.15.120:0/3319727137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0834101de0 0x7f08340744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:22.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.094+0000 7f083a384640 1 --2- 172.21.15.120:0/3319727137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0834106f10 0x7f0834074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:22.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.094+0000 7f0839382640 1 -- 172.21.15.120:0/3319727137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0834101de0 msgr2=0x7f08340744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:22.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.094+0000 7f0839382640 1 --2- 172.21.15.120:0/3319727137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0834101de0 0x7f08340744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.094+0000 7f0839382640 1 -- 172.21.15.120:0/3319727137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0834106f10 msgr2=0x7f0834074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:22.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.094+0000 7f0839382640 1 --2- 172.21.15.120:0/3319727137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0834106f10 0x7f0834074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.094+0000 7f0839382640 1 -- 172.21.15.120:0/3319727137 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08341a6890 con 0x7f0834102790 2024-09-19T09:17:22.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.094+0000 7f0839382640 1 --2- 172.21.15.120:0/3319727137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0834102790 0x7f0834074a30 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f082400d950 tx=0x7f082400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:22.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.094+0000 7f082affd640 1 -- 172.21.15.120:0/3319727137 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0824017030 con 0x7f0834102790 2024-09-19T09:17:22.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.095+0000 7f082affd640 1 -- 172.21.15.120:0/3319727137 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0824004020 con 0x7f0834102790 2024-09-19T09:17:22.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.095+0000 7f083bde7640 1 -- 172.21.15.120:0/3319727137 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08341a6b80 con 0x7f0834102790 2024-09-19T09:17:22.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.095+0000 7f082affd640 1 -- 172.21.15.120:0/3319727137 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08240047a0 con 0x7f0834102790 2024-09-19T09:17:22.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.095+0000 7f083bde7640 1 -- 172.21.15.120:0/3319727137 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08341a7110 con 0x7f0834102790 2024-09-19T09:17:22.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.097+0000 7f082affd640 1 -- 172.21.15.120:0/3319727137 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f08240041d0 con 0x7f0834102790 2024-09-19T09:17:22.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.097+0000 7f083e1c0740 1 Processor -- start 2024-09-19T09:17:22.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.097+0000 7f083e1c0740 1 -- start start 2024-09-19T09:17:22.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.098+0000 7f083e1c0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1f423b40 0x555c1f422090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:22.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.098+0000 7f083e1c0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1f4225d0 0x555c1f422a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:22.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.098+0000 7f083e1c0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c1f42d370 0x555c1f42f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:22.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.098+0000 7f083e1c0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555c1f3882f0 con 0x555c1f42d370 2024-09-19T09:17:22.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.098+0000 7f083e1c0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555c1f32fcf0 con 0x555c1f423b40 2024-09-19T09:17:22.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.098+0000 7f083e1c0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555c1f3842b0 con 0x555c1f4225d0 2024-09-19T09:17:22.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.098+0000 7f082affd640 1 --2- 172.21.15.120:0/3319727137 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f08200a32c0 0x7f08200a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:22.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.098+0000 7f0839b83640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1f4225d0 0x555c1f422a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:22.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.098+0000 7f0839382640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c1f42d370 0x555c1f42f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:22.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.098+0000 7f0839b83640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1f4225d0 0x555c1f422a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:51718/0 (socket says 172.21.15.120:51718) 2024-09-19T09:17:22.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.098+0000 7f0839b83640 1 -- 172.21.15.120:0/2348102103 learned_addr learned my addr 172.21.15.120:0/2348102103 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:22.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.098+0000 7f083a384640 1 --2- 172.21.15.120:0/2348102103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1f423b40 0x555c1f422090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:22.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.098+0000 7f083a384640 1 --2- 172.21.15.120:0/3319727137 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f08200a32c0 0x7f08200a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:22.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.098+0000 7f0839b83640 1 -- 172.21.15.120:0/2348102103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1f423b40 msgr2=0x555c1f422090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:22.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.098+0000 7f0839b83640 1 --2- 172.21.15.120:0/2348102103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1f423b40 0x555c1f422090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.098+0000 7f0839b83640 1 -- 172.21.15.120:0/2348102103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c1f42d370 msgr2=0x555c1f42f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:22.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.098+0000 7f0839b83640 1 --2- 172.21.15.120:0/2348102103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c1f42d370 0x555c1f42f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.098+0000 7f0839b83640 1 -- 172.21.15.120:0/2348102103 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c1f3143a0 con 0x555c1f4225d0 2024-09-19T09:17:22.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.098+0000 7f082affd640 1 -- 172.21.15.120:0/3319727137 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f08240cf130 con 0x7f0834102790 2024-09-19T09:17:22.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.098+0000 7f0839b83640 1 --2- 172.21.15.120:0/2348102103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1f4225d0 0x555c1f422a10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0830009d20 tx=0x7f0830011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8ca4dab903c9486 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:22.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.099+0000 7f083a384640 1 --2- 172.21.15.120:0/3319727137 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f08200a32c0 0x7f08200a5780 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f082c004af0 tx=0x7f082c004a80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:22.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.099+0000 7f081ffff640 1 -- 172.21.15.120:0/2348102103 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083001e070 con 0x555c1f4225d0 2024-09-19T09:17:22.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.099+0000 7f081ffff640 1 -- 172.21.15.120:0/2348102103 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0830019b10 con 0x555c1f4225d0 2024-09-19T09:17:22.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.099+0000 7f081ffff640 1 -- 172.21.15.120:0/2348102103 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083000ebb0 con 0x555c1f4225d0 2024-09-19T09:17:22.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.099+0000 7f083e1c0740 1 -- 172.21.15.120:0/2348102103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1f4225d0 msgr2=0x555c1f422a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:22.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.099+0000 7f083e1c0740 1 --2- 172.21.15.120:0/2348102103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1f4225d0 0x555c1f422a10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0830009d20 tx=0x7f0830011810 comp rx=0 tx=0).stop 2024-09-19T09:17:22.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.099+0000 7f083e1c0740 1 -- 172.21.15.120:0/2348102103 shutdown_connections 2024-09-19T09:17:22.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.099+0000 7f083e1c0740 1 --2- 172.21.15.120:0/2348102103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c1f42d370 0x555c1f42f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.099+0000 7f083e1c0740 1 --2- 172.21.15.120:0/2348102103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1f4225d0 0x555c1f422a10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.099+0000 7f083e1c0740 1 --2- 172.21.15.120:0/2348102103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1f423b40 0x555c1f422090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.099+0000 7f083e1c0740 1 -- 172.21.15.120:0/2348102103 >> 172.21.15.120:0/2348102103 conn(0x555c1f3eb310 msgr2=0x555c1f41ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:22.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.100+0000 7f083e1c0740 1 -- 172.21.15.120:0/2348102103 shutdown_connections 2024-09-19T09:17:22.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.100+0000 7f083e1c0740 1 -- 172.21.15.120:0/2348102103 wait complete. 2024-09-19T09:17:22.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.100+0000 7f083e1c0740 1 -- start start 2024-09-19T09:17:22.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.100+0000 7f083e1c0740 1 Processor -- start 2024-09-19T09:17:22.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.100+0000 7f083e1c0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1f4225d0 0x555c1f42cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:22.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.100+0000 7f083e1c0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c1f423b40 0x555c1f429fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:22.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.100+0000 7f083e1c0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1f42d370 0x555c1f42a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:22.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.100+0000 7f083e1c0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555c1f430070 con 0x555c1f423b40 2024-09-19T09:17:22.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.100+0000 7f083e1c0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555c1f343060 con 0x555c1f4225d0 2024-09-19T09:17:22.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.100+0000 7f083e1c0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555c1f4301f0 con 0x555c1f42d370 2024-09-19T09:17:22.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.100+0000 7f083e1c0740 10 client.0 _ll_register_callbacks cb 0x555c1f0968c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:17:22.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.101+0000 7f0839382640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c1f423b40 0x555c1f429fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:22.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.101+0000 7f0839b83640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1f4225d0 0x555c1f42cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:22.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.101+0000 7f0839382640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c1f423b40 0x555c1f429fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36978/0 (socket says 172.21.15.120:36978) 2024-09-19T09:17:22.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.101+0000 7f0839b83640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1f4225d0 0x555c1f42cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37932/0 (socket says 172.21.15.120:37932) 2024-09-19T09:17:22.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.101+0000 7f0839382640 1 -- 172.21.15.120:0/1230378108 learned_addr learned my addr 172.21.15.120:0/1230378108 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:22.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.101+0000 7f083a384640 1 --2- 172.21.15.120:0/1230378108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1f42d370 0x555c1f42a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:22.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.101+0000 7f0839382640 1 -- 172.21.15.120:0/1230378108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1f42d370 msgr2=0x555c1f42a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:22.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.101+0000 7f0839382640 1 --2- 172.21.15.120:0/1230378108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1f42d370 0x555c1f42a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.101+0000 7f0839382640 1 -- 172.21.15.120:0/1230378108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1f4225d0 msgr2=0x555c1f42cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:22.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.101+0000 7f0839382640 1 --2- 172.21.15.120:0/1230378108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1f4225d0 0x555c1f42cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.101+0000 7f0839382640 1 -- 172.21.15.120:0/1230378108 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c1f0f93c0 con 0x555c1f423b40 2024-09-19T09:17:22.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.101+0000 7f0839b83640 1 --2- 172.21.15.120:0/1230378108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1f4225d0 0x555c1f42cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:17:22.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.101+0000 7f0839382640 1 --2- 172.21.15.120:0/1230378108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c1f423b40 0x555c1f429fb0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f0824009fd0 tx=0x7f0824005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a0f934ea108e5f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:22.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.101+0000 7f081d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:17:22.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.101+0000 7f081d7fa640 1 -- 172.21.15.120:0/1230378108 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082409b750 con 0x555c1f423b40 2024-09-19T09:17:22.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.101+0000 7f081d7fa640 1 -- 172.21.15.120:0/1230378108 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f08240d10f0 con 0x555c1f423b40 2024-09-19T09:17:22.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.101+0000 7f083e1c0740 1 -- 172.21.15.120:0/1230378108 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0830019b10 con 0x555c1f423b40 2024-09-19T09:17:22.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.101+0000 7f081d7fa640 1 -- 172.21.15.120:0/1230378108 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f082409fb00 con 0x555c1f423b40 2024-09-19T09:17:22.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.102+0000 7f083e1c0740 10 client.20379 fetch_fsmap learned FSMap version 36 2024-09-19T09:17:22.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.102+0000 7f083e1c0740 1 -- 172.21.15.120:0/1230378108 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555c1f43bad0 con 0x555c1f423b40 2024-09-19T09:17:22.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.102+0000 7f081d7fa640 1 -- 172.21.15.120:0/1230378108 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0830019b10 con 0x555c1f423b40 2024-09-19T09:17:22.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.102+0000 7f081d7fa640 1 -- 172.21.15.120:0/1230378108 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f082409aba0 con 0x555c1f423b40 2024-09-19T09:17:22.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.102+0000 7f083e1c0740 10 client.20379 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:17:22.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.102+0000 7f083e1c0740 10 client.20379 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:17:22.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.102+0000 7f083e1c0740 20 client.20379 populate_metadata read hostname 'smithi120' 2024-09-19T09:17:22.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.102+0000 7f083e1c0740 1 --2- 172.21.15.120:0/1230378108 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x555c1f43cfe0 0x555c1f43f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:22.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.102+0000 7f083e1c0740 4 client.20379 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:17:22.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.102+0000 7f083e1c0740 1 -- 172.21.15.120:0/1230378108 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c1f395860 con 0x555c1f43cfe0 2024-09-19T09:17:22.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.102+0000 7f0839b83640 1 --2- 172.21.15.120:0/1230378108 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x555c1f43cfe0 0x555c1f43f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:22.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.103+0000 7f0839b83640 1 --2- 172.21.15.120:0/1230378108 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x555c1f43cfe0 0x555c1f43f3d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=304f008f64204cd5 server_cookie=5aa6f288225f588b in_seq=0 out_seq=0 2024-09-19T09:17:22.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.103+0000 7f081d7fa640 10 client.20379 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:17:22.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.103+0000 7f081d7fa640 1 -- 172.21.15.120:0/1230378108 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c1f395860 con 0x555c1f43cfe0 2024-09-19T09:17:22.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.103+0000 7f081d7fa640 10 client.20379 handle_command_reply: tid=0 2024-09-19T09:17:22.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.103+0000 7f083e1c0740 1 client.20379 shutdown 2024-09-19T09:17:22.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.103+0000 7f083e1c0740 1 -- 172.21.15.120:0/1230378108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c1f423b40 msgr2=0x555c1f429fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:22.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.103+0000 7f083e1c0740 1 --2- 172.21.15.120:0/1230378108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c1f423b40 0x555c1f429fb0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f0824009fd0 tx=0x7f0824005d50 comp rx=0 tx=0).stop 2024-09-19T09:17:22.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.104+0000 7f083e1c0740 1 -- 172.21.15.120:0/1230378108 shutdown_connections 2024-09-19T09:17:22.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.104+0000 7f083e1c0740 1 --2- 172.21.15.120:0/1230378108 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x555c1f43cfe0 0x555c1f43f3d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.104+0000 7f083e1c0740 1 --2- 172.21.15.120:0/1230378108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1f42d370 0x555c1f42a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.104+0000 7f083e1c0740 1 --2- 172.21.15.120:0/1230378108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c1f423b40 0x555c1f429fb0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.104+0000 7f083e1c0740 1 --2- 172.21.15.120:0/1230378108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1f4225d0 0x555c1f42cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.104+0000 7f083e1c0740 1 -- 172.21.15.120:0/1230378108 >> 172.21.15.120:0/1230378108 conn(0x555c1f3eb310 msgr2=0x555c1f424960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:22.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.104+0000 7f081d7fa640 0 client.20379 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:17:22.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.104+0000 7f083e1c0740 1 -- 172.21.15.120:0/1230378108 shutdown_connections 2024-09-19T09:17:22.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.104+0000 7f083e1c0740 1 -- 172.21.15.120:0/1230378108 wait complete. 2024-09-19T09:17:22.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.104+0000 7f083e1c0740 20 client.20379 trim_cache size 0 max 16384 2024-09-19T09:17:22.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.117+0000 7f083e1c0740 1 Processor -- start 2024-09-19T09:17:22.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.117+0000 7f083e1c0740 1 -- start start 2024-09-19T09:17:22.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.117+0000 7f083e1c0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1f4225d0 0x555c1f41edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:22.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.117+0000 7f083e1c0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1f423b40 0x555c1f439340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:22.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.117+0000 7f083e1c0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c1f42d370 0x555c1f429870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:22.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.117+0000 7f083e1c0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f08341af790 con 0x555c1f42d370 2024-09-19T09:17:22.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.117+0000 7f083e1c0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f08341b1360 con 0x555c1f4225d0 2024-09-19T09:17:22.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.117+0000 7f083e1c0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f08341afe60 con 0x555c1f423b40 2024-09-19T09:17:22.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.118+0000 7f083a384640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c1f42d370 0x555c1f429870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:22.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.118+0000 7f083a384640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c1f42d370 0x555c1f429870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36988/0 (socket says 172.21.15.120:36988) 2024-09-19T09:17:22.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.118+0000 7f083a384640 1 -- 172.21.15.120:0/2932467188 learned_addr learned my addr 172.21.15.120:0/2932467188 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:22.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.118+0000 7f0839b83640 1 --2- 172.21.15.120:0/2932467188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1f4225d0 0x555c1f41edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:22.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.118+0000 7f0839382640 1 --2- 172.21.15.120:0/2932467188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1f423b40 0x555c1f439340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:22.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.118+0000 7f083a384640 1 -- 172.21.15.120:0/2932467188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1f423b40 msgr2=0x555c1f439340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:22.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.118+0000 7f083a384640 1 --2- 172.21.15.120:0/2932467188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1f423b40 0x555c1f439340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.118+0000 7f083a384640 1 -- 172.21.15.120:0/2932467188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1f4225d0 msgr2=0x555c1f41edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:22.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.118+0000 7f083a384640 1 --2- 172.21.15.120:0/2932467188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1f4225d0 0x555c1f41edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.118+0000 7f083a384640 1 -- 172.21.15.120:0/2932467188 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c1f428f60 con 0x555c1f42d370 2024-09-19T09:17:22.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.118+0000 7f083a384640 1 --2- 172.21.15.120:0/2932467188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c1f42d370 0x555c1f429870 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f082c009db0 tx=0x7f082c036000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6640abd326a0551f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:22.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.118+0000 7f081f7fe640 1 -- 172.21.15.120:0/2932467188 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082c02fd10 con 0x555c1f42d370 2024-09-19T09:17:22.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.118+0000 7f081f7fe640 1 -- 172.21.15.120:0/2932467188 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f082c036070 con 0x555c1f42d370 2024-09-19T09:17:22.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.118+0000 7f081f7fe640 1 -- 172.21.15.120:0/2932467188 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082c04b790 con 0x555c1f42d370 2024-09-19T09:17:22.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.119+0000 7f083e1c0740 1 -- 172.21.15.120:0/2932467188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c1f42d370 msgr2=0x555c1f429870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:22.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.119+0000 7f083e1c0740 1 --2- 172.21.15.120:0/2932467188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c1f42d370 0x555c1f429870 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f082c009db0 tx=0x7f082c036000 comp rx=0 tx=0).stop 2024-09-19T09:17:22.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.119+0000 7f083e1c0740 1 -- 172.21.15.120:0/2932467188 shutdown_connections 2024-09-19T09:17:22.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.119+0000 7f083e1c0740 1 --2- 172.21.15.120:0/2932467188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c1f42d370 0x555c1f429870 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.119+0000 7f083e1c0740 1 --2- 172.21.15.120:0/2932467188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1f423b40 0x555c1f439340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.119+0000 7f083e1c0740 1 --2- 172.21.15.120:0/2932467188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1f4225d0 0x555c1f41edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.119+0000 7f083e1c0740 1 -- 172.21.15.120:0/2932467188 >> 172.21.15.120:0/2932467188 conn(0x555c1f3eb310 msgr2=0x555c1f41e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:22.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.119+0000 7f083e1c0740 1 -- 172.21.15.120:0/2932467188 shutdown_connections 2024-09-19T09:17:22.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.119+0000 7f083e1c0740 1 -- 172.21.15.120:0/2932467188 wait complete. 2024-09-19T09:17:22.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.119+0000 7f083e1c0740 1 -- start start 2024-09-19T09:17:22.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.120+0000 7f083e1c0740 1 Processor -- start 2024-09-19T09:17:22.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.120+0000 7f083e1c0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1f4225d0 0x555c1f431140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:22.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.120+0000 7f083e1c0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1f423b40 0x555c1f431680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:22.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.120+0000 7f083e1c0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c1f42d370 0x555c1f431bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:22.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.120+0000 7f083e1c0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f08341b0c90 con 0x555c1f42d370 2024-09-19T09:17:22.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.120+0000 7f083e1c0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f08341b0530 con 0x555c1f423b40 2024-09-19T09:17:22.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.120+0000 7f083e1c0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f08341ad750 con 0x555c1f4225d0 2024-09-19T09:17:22.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.120+0000 7f083e1c0740 10 client.0 _ll_register_callbacks cb 0x7f08240046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:17:22.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.120+0000 7f083a384640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c1f42d370 0x555c1f431bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:22.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.120+0000 7f0839382640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1f423b40 0x555c1f431680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:22.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.120+0000 7f083a384640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c1f42d370 0x555c1f431bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37002/0 (socket says 172.21.15.120:37002) 2024-09-19T09:17:22.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.120+0000 7f083a384640 1 -- 172.21.15.120:0/3944129820 learned_addr learned my addr 172.21.15.120:0/3944129820 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:22.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.120+0000 7f0839b83640 1 --2- 172.21.15.120:0/3944129820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1f4225d0 0x555c1f431140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:22.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.121+0000 7f083a384640 1 -- 172.21.15.120:0/3944129820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1f4225d0 msgr2=0x555c1f431140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:22.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.121+0000 7f083a384640 1 --2- 172.21.15.120:0/3944129820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1f4225d0 0x555c1f431140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.121+0000 7f083a384640 1 -- 172.21.15.120:0/3944129820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1f423b40 msgr2=0x555c1f431680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:22.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.121+0000 7f083a384640 1 --2- 172.21.15.120:0/3944129820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1f423b40 0x555c1f431680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.121+0000 7f083a384640 1 -- 172.21.15.120:0/3944129820 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c1f42a8a0 con 0x555c1f42d370 2024-09-19T09:17:22.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.121+0000 7f0839b83640 1 --2- 172.21.15.120:0/3944129820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1f4225d0 0x555c1f431140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:17:22.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.121+0000 7f0839382640 1 --2- 172.21.15.120:0/3944129820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1f423b40 0x555c1f431680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:17:22.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.121+0000 7f083a384640 1 --2- 172.21.15.120:0/3944129820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c1f42d370 0x555c1f431bc0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f082c005e30 tx=0x7f082c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9aea76bec31cdb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:22.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.121+0000 7f081d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:17:22.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.121+0000 7f081d7fa640 1 -- 172.21.15.120:0/3944129820 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082c0309d0 con 0x555c1f42d370 2024-09-19T09:17:22.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.121+0000 7f083e1c0740 1 -- 172.21.15.120:0/3944129820 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f082c036070 con 0x555c1f42d370 2024-09-19T09:17:22.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.121+0000 7f081d7fa640 1 -- 172.21.15.120:0/3944129820 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f082c030b70 con 0x555c1f42d370 2024-09-19T09:17:22.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.121+0000 7f081d7fa640 1 -- 172.21.15.120:0/3944129820 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f082c05a480 con 0x555c1f42d370 2024-09-19T09:17:22.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.121+0000 7f083e1c0740 10 client.20385 fetch_fsmap learned FSMap version 36 2024-09-19T09:17:22.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.121+0000 7f083e1c0740 1 -- 172.21.15.120:0/3944129820 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555c1f4a7cc0 con 0x555c1f42d370 2024-09-19T09:17:22.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.121+0000 7f081d7fa640 1 -- 172.21.15.120:0/3944129820 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f082c036070 con 0x555c1f42d370 2024-09-19T09:17:22.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.122+0000 7f081d7fa640 1 -- 172.21.15.120:0/3944129820 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f082c048040 con 0x555c1f42d370 2024-09-19T09:17:22.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.122+0000 7f083e1c0740 10 client.20385 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:17:22.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.122+0000 7f083e1c0740 10 client.20385 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:17:22.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.122+0000 7f083e1c0740 20 client.20385 populate_metadata read hostname 'smithi120' 2024-09-19T09:17:22.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.122+0000 7f083e1c0740 1 --2- 172.21.15.120:0/3944129820 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x555c1f4a8360 0x555c1f4aa780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:22.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.122+0000 7f083e1c0740 4 client.20385 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:17:22.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.122+0000 7f083e1c0740 1 -- 172.21.15.120:0/3944129820 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555c1f38fd70 con 0x555c1f4a8360 2024-09-19T09:17:22.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.122+0000 7f0839b83640 1 --2- 172.21.15.120:0/3944129820 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x555c1f4a8360 0x555c1f4aa780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:22.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.122+0000 7f0839b83640 1 --2- 172.21.15.120:0/3944129820 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x555c1f4a8360 0x555c1f4aa780 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9ac2097f86f567a4 server_cookie=97f97063129f8d0c in_seq=0 out_seq=0 2024-09-19T09:17:22.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.122+0000 7f081d7fa640 10 client.20385 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:17:22.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.122+0000 7f081d7fa640 1 -- 172.21.15.120:0/3944129820 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555c1f38fd70 con 0x555c1f4a8360 2024-09-19T09:17:22.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.122+0000 7f081d7fa640 10 client.20385 handle_command_reply: tid=0 2024-09-19T09:17:22.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.122+0000 7f083e1c0740 1 client.20385 shutdown 2024-09-19T09:17:22.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.123+0000 7f083e1c0740 1 -- 172.21.15.120:0/3944129820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c1f42d370 msgr2=0x555c1f431bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:22.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.123+0000 7f083e1c0740 1 --2- 172.21.15.120:0/3944129820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c1f42d370 0x555c1f431bc0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f082c005e30 tx=0x7f082c0095c0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.123+0000 7f083e1c0740 1 -- 172.21.15.120:0/3944129820 shutdown_connections 2024-09-19T09:17:22.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.123+0000 7f083e1c0740 1 --2- 172.21.15.120:0/3944129820 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x555c1f4a8360 0x555c1f4aa780 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.123+0000 7f083e1c0740 1 --2- 172.21.15.120:0/3944129820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555c1f42d370 0x555c1f431bc0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.123+0000 7f083e1c0740 1 --2- 172.21.15.120:0/3944129820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555c1f423b40 0x555c1f431680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.123+0000 7f081d7fa640 0 client.20385 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:17:22.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.123+0000 7f083e1c0740 1 --2- 172.21.15.120:0/3944129820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555c1f4225d0 0x555c1f431140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.123+0000 7f083e1c0740 1 -- 172.21.15.120:0/3944129820 >> 172.21.15.120:0/3944129820 conn(0x555c1f3eb310 msgr2=0x555c1f41dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:22.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.123+0000 7f083e1c0740 1 -- 172.21.15.120:0/3944129820 shutdown_connections 2024-09-19T09:17:22.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.123+0000 7f083e1c0740 1 -- 172.21.15.120:0/3944129820 wait complete. 2024-09-19T09:17:22.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.123+0000 7f083e1c0740 20 client.20385 trim_cache size 0 max 16384 2024-09-19T09:17:22.134 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:17:22.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.123+0000 7f083bde7640 1 -- 172.21.15.120:0/3319727137 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f08200a32c0 msgr2=0x7f08200a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:22.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.123+0000 7f083bde7640 1 --2- 172.21.15.120:0/3319727137 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f08200a32c0 0x7f08200a5780 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f082c004af0 tx=0x7f082c004a80 comp rx=0 tx=0).stop 2024-09-19T09:17:22.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.124+0000 7f083bde7640 1 -- 172.21.15.120:0/3319727137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0834102790 msgr2=0x7f0834074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:22.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.124+0000 7f083bde7640 1 --2- 172.21.15.120:0/3319727137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0834102790 0x7f0834074a30 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f082400d950 tx=0x7f082400de10 comp rx=0 tx=0).stop 2024-09-19T09:17:22.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.124+0000 7f083bde7640 1 -- 172.21.15.120:0/3319727137 shutdown_connections 2024-09-19T09:17:22.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.124+0000 7f083bde7640 1 --2- 172.21.15.120:0/3319727137 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f08200a32c0 0x7f08200a5780 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.124+0000 7f083bde7640 1 --2- 172.21.15.120:0/3319727137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0834106f10 0x7f0834074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.124+0000 7f083bde7640 1 --2- 172.21.15.120:0/3319727137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0834102790 0x7f0834074a30 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.124+0000 7f083bde7640 1 --2- 172.21.15.120:0/3319727137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0834101de0 0x7f08340744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:22.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.124+0000 7f083bde7640 1 -- 172.21.15.120:0/3319727137 >> 172.21.15.120:0/3319727137 conn(0x7f08340fd8c0 msgr2=0x7f08340fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:22.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.124+0000 7f083bde7640 1 -- 172.21.15.120:0/3319727137 shutdown_connections 2024-09-19T09:17:22.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:22.124+0000 7f083bde7640 1 -- 172.21.15.120:0/3319727137 wait complete. 2024-09-19T09:17:22.354 INFO:tasks.workunit.client.0.smithi120.stdout: 10 34165 26.17 MB/sec warmup 77 sec latency 149.059 ms 2024-09-19T09:17:23.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:22 smithi120 ceph-mon[31150]: pgmap v780: 97 pgs: 97 active+clean; 286 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.0 MiB/s wr, 846 op/s 2024-09-19T09:17:23.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:22 smithi196 ceph-mon[35150]: pgmap v780: 97 pgs: 97 active+clean; 286 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.0 MiB/s wr, 846 op/s 2024-09-19T09:17:23.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:22 smithi153 ceph-mon[35071]: pgmap v780: 97 pgs: 97 active+clean; 286 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.0 MiB/s wr, 846 op/s 2024-09-19T09:17:23.269 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:17:23.354 INFO:tasks.workunit.client.0.smithi120.stdout: 10 34653 26.21 MB/sec warmup 78 sec latency 83.697 ms 2024-09-19T09:17:23.545 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:17:24.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.072+0000 7f7a1a621640 1 -- 172.21.15.120:0/3974089682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a14112b80 msgr2=0x7f7a14114f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:24.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.072+0000 7f7a1a621640 1 --2- 172.21.15.120:0/3974089682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a14112b80 0x7f7a14114f70 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009ca0 tx=0x7f7a0c02a950 comp rx=0 tx=0).stop 2024-09-19T09:17:24.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.072+0000 7f7a1a621640 1 -- 172.21.15.120:0/3974089682 shutdown_connections 2024-09-19T09:17:24.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.072+0000 7f7a1a621640 1 --2- 172.21.15.120:0/3974089682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a14112b80 0x7f7a14114f70 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.072+0000 7f7a1a621640 1 --2- 172.21.15.120:0/3974089682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a14074eb0 0x7f7a14112640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.072+0000 7f7a1a621640 1 --2- 172.21.15.120:0/3974089682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a14074590 0x7f7a14074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.072+0000 7f7a1a621640 1 -- 172.21.15.120:0/3974089682 >> 172.21.15.120:0/3974089682 conn(0x7f7a140fd8b0 msgr2=0x7f7a140ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:24.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.072+0000 7f7a1a621640 1 -- 172.21.15.120:0/3974089682 shutdown_connections 2024-09-19T09:17:24.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.072+0000 7f7a1a621640 1 -- 172.21.15.120:0/3974089682 wait complete. 2024-09-19T09:17:24.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.073+0000 7f7a1a621640 1 Processor -- start 2024-09-19T09:17:24.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.073+0000 7f7a1a621640 1 -- start start 2024-09-19T09:17:24.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.073+0000 7f7a1a621640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a14074590 0x7f7a141a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:24.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.073+0000 7f7a1a621640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a14074eb0 0x7f7a141a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:24.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.073+0000 7f7a1a621640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a14112b80 0x7f7a1419e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:24.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.073+0000 7f7a1a621640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7a14118550 con 0x7f7a14074590 2024-09-19T09:17:24.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.073+0000 7f7a1a621640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7a141183d0 con 0x7f7a14074eb0 2024-09-19T09:17:24.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.073+0000 7f7a1a621640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7a141186d0 con 0x7f7a14112b80 2024-09-19T09:17:24.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.073+0000 7f7a13fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a14074590 0x7f7a141a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:24.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.073+0000 7f7a0bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a14074eb0 0x7f7a141a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:24.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.073+0000 7f7a13fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a14074590 0x7f7a141a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37026/0 (socket says 172.21.15.120:37026) 2024-09-19T09:17:24.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.073+0000 7f7a0bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a14074eb0 0x7f7a141a4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37958/0 (socket says 172.21.15.120:37958) 2024-09-19T09:17:24.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.073+0000 7f7a13fff640 1 -- 172.21.15.120:0/2717613303 learned_addr learned my addr 172.21.15.120:0/2717613303 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:24.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.073+0000 7f7a18bbe640 1 --2- 172.21.15.120:0/2717613303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a14112b80 0x7f7a1419e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:24.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.074+0000 7f7a0bfff640 1 -- 172.21.15.120:0/2717613303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a14112b80 msgr2=0x7f7a1419e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:24.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.074+0000 7f7a0bfff640 1 --2- 172.21.15.120:0/2717613303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a14112b80 0x7f7a1419e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.074+0000 7f7a0bfff640 1 -- 172.21.15.120:0/2717613303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a14074590 msgr2=0x7f7a141a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:24.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.074+0000 7f7a0bfff640 1 --2- 172.21.15.120:0/2717613303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a14074590 0x7f7a141a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.074+0000 7f7a0bfff640 1 -- 172.21.15.120:0/2717613303 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a1419eff0 con 0x7f7a14074eb0 2024-09-19T09:17:24.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.074+0000 7f7a13fff640 1 --2- 172.21.15.120:0/2717613303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a14074590 0x7f7a141a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:17:24.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.074+0000 7f7a0bfff640 1 --2- 172.21.15.120:0/2717613303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a14074eb0 0x7f7a141a4c40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f79fc00e9f0 tx=0x7f79fc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:24.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.074+0000 7f7a11ffb640 1 -- 172.21.15.120:0/2717613303 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79fc00ccc0 con 0x7f7a14074eb0 2024-09-19T09:17:24.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.074+0000 7f7a1a621640 1 -- 172.21.15.120:0/2717613303 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a1419f2e0 con 0x7f7a14074eb0 2024-09-19T09:17:24.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.074+0000 7f7a11ffb640 1 -- 172.21.15.120:0/2717613303 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79fc010910 con 0x7f7a14074eb0 2024-09-19T09:17:24.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.074+0000 7f7a11ffb640 1 -- 172.21.15.120:0/2717613303 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79fc018960 con 0x7f7a14074eb0 2024-09-19T09:17:24.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.074+0000 7f7a1a621640 1 -- 172.21.15.120:0/2717613303 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a141ab5d0 con 0x7f7a14074eb0 2024-09-19T09:17:24.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.076+0000 7f7a11ffb640 1 -- 172.21.15.120:0/2717613303 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f79fc010ac0 con 0x7f7a14074eb0 2024-09-19T09:17:24.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.077+0000 7f7a1c9fa740 1 Processor -- start 2024-09-19T09:17:24.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.077+0000 7f7a1c9fa740 1 -- start start 2024-09-19T09:17:24.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.077+0000 7f7a1c9fa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646163e1b40 0x5646163e0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:24.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.077+0000 7f7a1c9fa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646163e05d0 0x5646163e0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:24.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.077+0000 7f7a1c9fa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646163eb370 0x5646163ed760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:24.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.077+0000 7f7a1c9fa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5646163462f0 con 0x5646163e1b40 2024-09-19T09:17:24.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.077+0000 7f7a1c9fa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5646162edcf0 con 0x5646163eb370 2024-09-19T09:17:24.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.077+0000 7f7a1c9fa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5646163422b0 con 0x5646163e05d0 2024-09-19T09:17:24.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.077+0000 7f7a18bbe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646163e1b40 0x5646163e0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:24.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.077+0000 7f7a18bbe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646163e1b40 0x5646163e0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37040/0 (socket says 172.21.15.120:37040) 2024-09-19T09:17:24.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.077+0000 7f7a18bbe640 1 -- 172.21.15.120:0/1695832164 learned_addr learned my addr 172.21.15.120:0/1695832164 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:24.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.077+0000 7f7a13fff640 1 --2- 172.21.15.120:0/1695832164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646163e05d0 0x5646163e0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:24.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.077+0000 7f7a0bfff640 1 --2- 172.21.15.120:0/1695832164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646163eb370 0x5646163ed760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:24.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.077+0000 7f7a11ffb640 1 --2- 172.21.15.120:0/2717613303 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7a000a3240 0x7f7a000a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:24.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.078+0000 7f7a18bbe640 1 -- 172.21.15.120:0/1695832164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646163e05d0 msgr2=0x5646163e0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:24.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.078+0000 7f7a18bbe640 1 --2- 172.21.15.120:0/1695832164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646163e05d0 0x5646163e0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.078+0000 7f7a18bbe640 1 -- 172.21.15.120:0/1695832164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646163eb370 msgr2=0x5646163ed760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:24.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.078+0000 7f7a18bbe640 1 --2- 172.21.15.120:0/1695832164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646163eb370 0x5646163ed760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.078+0000 7f7a18bbe640 1 -- 172.21.15.120:0/1695832164 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646162d23a0 con 0x5646163e1b40 2024-09-19T09:17:24.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.078+0000 7f7a13fff640 1 --2- 172.21.15.120:0/1695832164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646163e05d0 0x5646163e0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:17:24.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.078+0000 7f7a11ffb640 1 -- 172.21.15.120:0/2717613303 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f79fc0cfc50 con 0x7f7a14074eb0 2024-09-19T09:17:24.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.078+0000 7f7a18bbe640 1 --2- 172.21.15.120:0/2717613303 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7a000a3240 0x7f7a000a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:24.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.078+0000 7f7a18bbe640 1 --2- 172.21.15.120:0/1695832164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646163e1b40 0x5646163e0090 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f7a14076600 tx=0x7f7a0c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3171761d5c8a7b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:24.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.078+0000 7f7a0a7fc640 1 -- 172.21.15.120:0/1695832164 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c034040 con 0x5646163e1b40 2024-09-19T09:17:24.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.078+0000 7f7a0a7fc640 1 -- 172.21.15.120:0/1695832164 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a0c003b70 con 0x5646163e1b40 2024-09-19T09:17:24.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.078+0000 7f7a0a7fc640 1 -- 172.21.15.120:0/1695832164 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c046690 con 0x5646163e1b40 2024-09-19T09:17:24.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.078+0000 7f7a18bbe640 1 --2- 172.21.15.120:0/2717613303 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7a000a3240 0x7f7a000a5700 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c04d000 tx=0x7f7a0c04a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:24.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.079+0000 7f7a1c9fa740 1 -- 172.21.15.120:0/1695832164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646163e1b40 msgr2=0x5646163e0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:24.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.079+0000 7f7a1c9fa740 1 --2- 172.21.15.120:0/1695832164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646163e1b40 0x5646163e0090 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f7a14076600 tx=0x7f7a0c005ef0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.079+0000 7f7a1c9fa740 1 -- 172.21.15.120:0/1695832164 shutdown_connections 2024-09-19T09:17:24.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.079+0000 7f7a1c9fa740 1 --2- 172.21.15.120:0/1695832164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646163eb370 0x5646163ed760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.079+0000 7f7a1c9fa740 1 --2- 172.21.15.120:0/1695832164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646163e05d0 0x5646163e0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.079+0000 7f7a1c9fa740 1 --2- 172.21.15.120:0/1695832164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646163e1b40 0x5646163e0090 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.079+0000 7f7a1c9fa740 1 -- 172.21.15.120:0/1695832164 >> 172.21.15.120:0/1695832164 conn(0x5646163a9310 msgr2=0x5646163dcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:24.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.079+0000 7f7a1c9fa740 1 -- 172.21.15.120:0/1695832164 shutdown_connections 2024-09-19T09:17:24.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.079+0000 7f7a1c9fa740 1 -- 172.21.15.120:0/1695832164 wait complete. 2024-09-19T09:17:24.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.079+0000 7f7a1c9fa740 1 -- start start 2024-09-19T09:17:24.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.079+0000 7f7a1c9fa740 1 Processor -- start 2024-09-19T09:17:24.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.080+0000 7f7a1c9fa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646163e05d0 0x5646163ead30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:24.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.080+0000 7f7a1c9fa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646163e1b40 0x5646163e7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:24.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.080+0000 7f7a1c9fa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646163eb370 0x5646163e85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:24.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.080+0000 7f7a1c9fa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5646163ee070 con 0x5646163e1b40 2024-09-19T09:17:24.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.080+0000 7f7a1c9fa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564616301060 con 0x5646163e05d0 2024-09-19T09:17:24.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.080+0000 7f7a1c9fa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5646163ee1f0 con 0x5646163eb370 2024-09-19T09:17:24.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.080+0000 7f7a1c9fa740 10 client.0 _ll_register_callbacks cb 0x5646160548c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:17:24.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.080+0000 7f7a0bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646163e1b40 0x5646163e7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:24.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.080+0000 7f7a0bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646163e1b40 0x5646163e7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37044/0 (socket says 172.21.15.120:37044) 2024-09-19T09:17:24.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.080+0000 7f7a18bbe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646163eb370 0x5646163e85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:24.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.080+0000 7f7a0bfff640 1 -- 172.21.15.120:0/192083653 learned_addr learned my addr 172.21.15.120:0/192083653 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:24.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.080+0000 7f7a13fff640 1 --2- 172.21.15.120:0/192083653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646163e05d0 0x5646163ead30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:24.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.080+0000 7f7a0bfff640 1 -- 172.21.15.120:0/192083653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646163eb370 msgr2=0x5646163e85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:24.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.080+0000 7f7a0bfff640 1 --2- 172.21.15.120:0/192083653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646163eb370 0x5646163e85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.080+0000 7f7a0bfff640 1 -- 172.21.15.120:0/192083653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646163e05d0 msgr2=0x5646163ead30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:24.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.080+0000 7f7a0bfff640 1 --2- 172.21.15.120:0/192083653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646163e05d0 0x5646163ead30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.080+0000 7f7a0bfff640 1 -- 172.21.15.120:0/192083653 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646160b73c0 con 0x5646163e1b40 2024-09-19T09:17:24.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.080+0000 7f7a0bfff640 1 --2- 172.21.15.120:0/192083653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646163e1b40 0x5646163e7fe0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f79fc000c00 tx=0x7f79fc09bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208d60284112e840 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:24.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.080+0000 7f79f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:17:24.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.081+0000 7f79f37fe640 1 -- 172.21.15.120:0/192083653 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79fc0bc2b0 con 0x5646163e1b40 2024-09-19T09:17:24.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.081+0000 7f79f37fe640 1 -- 172.21.15.120:0/192083653 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f79fc0cdbd0 con 0x5646163e1b40 2024-09-19T09:17:24.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.081+0000 7f7a1c9fa740 1 -- 172.21.15.120:0/192083653 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a0c003b70 con 0x5646163e1b40 2024-09-19T09:17:24.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.081+0000 7f79f37fe640 1 -- 172.21.15.120:0/192083653 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f79fc0a05e0 con 0x5646163e1b40 2024-09-19T09:17:24.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.081+0000 7f7a1c9fa740 10 client.20397 fetch_fsmap learned FSMap version 36 2024-09-19T09:17:24.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.081+0000 7f7a1c9fa740 1 -- 172.21.15.120:0/192083653 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5646163f9b20 con 0x5646163e1b40 2024-09-19T09:17:24.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.081+0000 7f79f37fe640 1 -- 172.21.15.120:0/192083653 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a0c003b70 con 0x5646163e1b40 2024-09-19T09:17:24.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.082+0000 7f79f37fe640 1 -- 172.21.15.120:0/192083653 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f79fc09b730 con 0x5646163e1b40 2024-09-19T09:17:24.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.082+0000 7f7a1c9fa740 10 client.20397 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:17:24.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.082+0000 7f7a1c9fa740 10 client.20397 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:17:24.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.082+0000 7f7a1c9fa740 20 client.20397 populate_metadata read hostname 'smithi120' 2024-09-19T09:17:24.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.082+0000 7f7a1c9fa740 1 --2- 172.21.15.120:0/192083653 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5646163fb030 0x5646163fd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:24.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.082+0000 7f7a1c9fa740 4 client.20397 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:17:24.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.082+0000 7f7a1c9fa740 1 -- 172.21.15.120:0/192083653 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564616353860 con 0x5646163fb030 2024-09-19T09:17:24.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.082+0000 7f7a13fff640 1 --2- 172.21.15.120:0/192083653 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5646163fb030 0x5646163fd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:24.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.082+0000 7f79f37fe640 10 client.20397 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:17:24.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.082+0000 7f7a13fff640 1 --2- 172.21.15.120:0/192083653 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5646163fb030 0x5646163fd420 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=72cd8e015f178ef0 server_cookie=3f5e42388d2313c7 in_seq=0 out_seq=0 2024-09-19T09:17:24.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.083+0000 7f79f37fe640 1 -- 172.21.15.120:0/192083653 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564616353860 con 0x5646163fb030 2024-09-19T09:17:24.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.083+0000 7f79f37fe640 10 client.20397 handle_command_reply: tid=0 2024-09-19T09:17:24.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.083+0000 7f7a1c9fa740 1 client.20397 shutdown 2024-09-19T09:17:24.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.083+0000 7f7a1c9fa740 1 -- 172.21.15.120:0/192083653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646163e1b40 msgr2=0x5646163e7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:24.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.083+0000 7f7a1c9fa740 1 --2- 172.21.15.120:0/192083653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646163e1b40 0x5646163e7fe0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f79fc000c00 tx=0x7f79fc09bc40 comp rx=0 tx=0).stop 2024-09-19T09:17:24.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.083+0000 7f7a1c9fa740 1 -- 172.21.15.120:0/192083653 shutdown_connections 2024-09-19T09:17:24.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.083+0000 7f7a1c9fa740 1 --2- 172.21.15.120:0/192083653 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5646163fb030 0x5646163fd420 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.083+0000 7f7a1c9fa740 1 --2- 172.21.15.120:0/192083653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646163eb370 0x5646163e85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.083+0000 7f7a1c9fa740 1 --2- 172.21.15.120:0/192083653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646163e1b40 0x5646163e7fe0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.083+0000 7f7a1c9fa740 1 --2- 172.21.15.120:0/192083653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646163e05d0 0x5646163ead30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.083+0000 7f7a1c9fa740 1 -- 172.21.15.120:0/192083653 >> 172.21.15.120:0/192083653 conn(0x5646163a9310 msgr2=0x5646163e2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:24.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.083+0000 7f79f37fe640 0 client.20397 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:17:24.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.083+0000 7f7a1c9fa740 1 -- 172.21.15.120:0/192083653 shutdown_connections 2024-09-19T09:17:24.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.083+0000 7f7a1c9fa740 1 -- 172.21.15.120:0/192083653 wait complete. 2024-09-19T09:17:24.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.084+0000 7f7a1c9fa740 20 client.20397 trim_cache size 0 max 16384 2024-09-19T09:17:24.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.097+0000 7f7a1c9fa740 1 Processor -- start 2024-09-19T09:17:24.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.097+0000 7f7a1c9fa740 1 -- start start 2024-09-19T09:17:24.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.097+0000 7f7a1c9fa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646163e05d0 0x5646163f7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:24.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.097+0000 7f7a1c9fa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646163e1b40 0x5646163dfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:24.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.097+0000 7f7a1c9fa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646163eb370 0x5646163e6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:24.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.097+0000 7f7a1c9fa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7a141b3c50 con 0x5646163eb370 2024-09-19T09:17:24.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.097+0000 7f7a1c9fa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7a141b5820 con 0x5646163e05d0 2024-09-19T09:17:24.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.097+0000 7f7a1c9fa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7a141b4320 con 0x5646163e1b40 2024-09-19T09:17:24.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.098+0000 7f7a18bbe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646163eb370 0x5646163e6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:24.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.098+0000 7f7a18bbe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646163eb370 0x5646163e6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37060/0 (socket says 172.21.15.120:37060) 2024-09-19T09:17:24.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.098+0000 7f7a13fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646163e05d0 0x5646163f7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:24.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.098+0000 7f7a18bbe640 1 -- 172.21.15.120:0/1531190344 learned_addr learned my addr 172.21.15.120:0/1531190344 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:24.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.098+0000 7f7a0bfff640 1 --2- 172.21.15.120:0/1531190344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646163e1b40 0x5646163dfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:24.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.098+0000 7f7a18bbe640 1 -- 172.21.15.120:0/1531190344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646163e1b40 msgr2=0x5646163dfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:24.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.098+0000 7f7a18bbe640 1 --2- 172.21.15.120:0/1531190344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646163e1b40 0x5646163dfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.098+0000 7f7a18bbe640 1 -- 172.21.15.120:0/1531190344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646163e05d0 msgr2=0x5646163f7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:24.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.098+0000 7f7a18bbe640 1 --2- 172.21.15.120:0/1531190344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646163e05d0 0x5646163f7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.098+0000 7f7a18bbe640 1 -- 172.21.15.120:0/1531190344 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646163dd960 con 0x5646163eb370 2024-09-19T09:17:24.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.098+0000 7f7a18bbe640 1 --2- 172.21.15.120:0/1531190344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646163eb370 0x5646163e6fc0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c0047f0 tx=0x7f7a0c0048c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f03f9891fd7f020 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:24.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.098+0000 7f7a09ffb640 1 -- 172.21.15.120:0/1531190344 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c046470 con 0x5646163eb370 2024-09-19T09:17:24.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.098+0000 7f7a09ffb640 1 -- 172.21.15.120:0/1531190344 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a0c046610 con 0x5646163eb370 2024-09-19T09:17:24.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.098+0000 7f7a09ffb640 1 -- 172.21.15.120:0/1531190344 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c0042a0 con 0x5646163eb370 2024-09-19T09:17:24.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.099+0000 7f7a1c9fa740 1 -- 172.21.15.120:0/1531190344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646163eb370 msgr2=0x5646163e6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:24.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.099+0000 7f7a1c9fa740 1 --2- 172.21.15.120:0/1531190344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646163eb370 0x5646163e6fc0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c0047f0 tx=0x7f7a0c0048c0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.099+0000 7f7a1c9fa740 1 -- 172.21.15.120:0/1531190344 shutdown_connections 2024-09-19T09:17:24.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.099+0000 7f7a1c9fa740 1 --2- 172.21.15.120:0/1531190344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646163eb370 0x5646163e6fc0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.099+0000 7f7a1c9fa740 1 --2- 172.21.15.120:0/1531190344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646163e1b40 0x5646163dfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.099+0000 7f7a1c9fa740 1 --2- 172.21.15.120:0/1531190344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646163e05d0 0x5646163f7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.099+0000 7f7a1c9fa740 1 -- 172.21.15.120:0/1531190344 >> 172.21.15.120:0/1531190344 conn(0x5646163a9310 msgr2=0x5646163e68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:24.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.099+0000 7f7a1c9fa740 1 -- 172.21.15.120:0/1531190344 shutdown_connections 2024-09-19T09:17:24.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.099+0000 7f7a1c9fa740 1 -- 172.21.15.120:0/1531190344 wait complete. 2024-09-19T09:17:24.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.100+0000 7f7a1c9fa740 1 -- start start 2024-09-19T09:17:24.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.100+0000 7f7a1c9fa740 1 Processor -- start 2024-09-19T09:17:24.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.100+0000 7f7a1c9fa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646163e05d0 0x5646163f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:24.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.100+0000 7f7a1c9fa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646163e1b40 0x5646163eef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:24.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.100+0000 7f7a1c9fa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646163eb370 0x5646163ef460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:24.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.100+0000 7f7a1c9fa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7a141b5150 con 0x5646163e1b40 2024-09-19T09:17:24.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.100+0000 7f7a0bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646163e1b40 0x5646163eef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:24.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.100+0000 7f7a13fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646163e05d0 0x5646163f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:24.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.100+0000 7f7a0bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646163e1b40 0x5646163eef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37070/0 (socket says 172.21.15.120:37070) 2024-09-19T09:17:24.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.100+0000 7f7a0bfff640 1 -- 172.21.15.120:0/852925904 learned_addr learned my addr 172.21.15.120:0/852925904 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:24.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.100+0000 7f7a18bbe640 1 --2- 172.21.15.120:0/852925904 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646163eb370 0x5646163ef460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:24.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.100+0000 7f7a1c9fa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7a141b49f0 con 0x5646163eb370 2024-09-19T09:17:24.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.100+0000 7f7a1c9fa740 1 -- 172.21.15.120:0/852925904 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7a141b1c10 con 0x5646163e05d0 2024-09-19T09:17:24.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.100+0000 7f7a1c9fa740 10 client.0 _ll_register_callbacks cb 0x7f79fc00ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:17:24.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.101+0000 7f7a13fff640 1 -- 172.21.15.120:0/852925904 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646163eb370 msgr2=0x5646163ef460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:24.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.101+0000 7f7a13fff640 1 --2- 172.21.15.120:0/852925904 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646163eb370 0x5646163ef460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.101+0000 7f7a13fff640 1 -- 172.21.15.120:0/852925904 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646163e1b40 msgr2=0x5646163eef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:24.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.101+0000 7f7a13fff640 1 --2- 172.21.15.120:0/852925904 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646163e1b40 0x5646163eef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.101+0000 7f7a13fff640 1 -- 172.21.15.120:0/852925904 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646163efcb0 con 0x5646163e05d0 2024-09-19T09:17:24.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.101+0000 7f7a0bfff640 1 --2- 172.21.15.120:0/852925904 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646163e1b40 0x5646163eef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:17:24.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.101+0000 7f7a13fff640 1 --2- 172.21.15.120:0/852925904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646163e05d0 0x5646163f3bf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x5646163fdc30 tx=0x7f7a04012d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b3fa5c4f23b1e24 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:24.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.101+0000 7f79f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:17:24.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.101+0000 7f79f37fe640 1 -- 172.21.15.120:0/852925904 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a04020b30 con 0x5646163e05d0 2024-09-19T09:17:24.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.101+0000 7f7a1c9fa740 1 -- 172.21.15.120:0/852925904 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a0c046610 con 0x5646163e05d0 2024-09-19T09:17:24.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.101+0000 7f79f37fe640 1 -- 172.21.15.120:0/852925904 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f7a04020cd0 con 0x5646163e05d0 2024-09-19T09:17:24.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.101+0000 7f79f37fe640 1 -- 172.21.15.120:0/852925904 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7a04005020 con 0x5646163e05d0 2024-09-19T09:17:24.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.101+0000 7f7a1c9fa740 10 client.29188 fetch_fsmap learned FSMap version 36 2024-09-19T09:17:24.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.101+0000 7f7a1c9fa740 1 -- 172.21.15.120:0/852925904 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564616465cd0 con 0x5646163e05d0 2024-09-19T09:17:24.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.102+0000 7f79f37fe640 1 -- 172.21.15.120:0/852925904 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a0c046610 con 0x5646163e05d0 2024-09-19T09:17:24.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.102+0000 7f79f37fe640 1 -- 172.21.15.120:0/852925904 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f7a04031050 con 0x5646163e05d0 2024-09-19T09:17:24.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.102+0000 7f7a1c9fa740 10 client.29188 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:17:24.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.102+0000 7f7a1c9fa740 10 client.29188 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:17:24.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.102+0000 7f7a1c9fa740 20 client.29188 populate_metadata read hostname 'smithi120' 2024-09-19T09:17:24.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.102+0000 7f7a1c9fa740 1 --2- 172.21.15.120:0/852925904 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5646164663a0 0x5646164687b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:24.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.102+0000 7f7a1c9fa740 4 client.29188 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:17:24.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.102+0000 7f7a1c9fa740 1 -- 172.21.15.120:0/852925904 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56461634dd70 con 0x5646164663a0 2024-09-19T09:17:24.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.102+0000 7f7a0bfff640 1 --2- 172.21.15.120:0/852925904 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5646164663a0 0x5646164687b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:24.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.102+0000 7f79f37fe640 10 client.29188 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:17:24.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.102+0000 7f7a0bfff640 1 --2- 172.21.15.120:0/852925904 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5646164663a0 0x5646164687b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c205377740e744e7 server_cookie=12bcbb29fe54ed03 in_seq=0 out_seq=0 2024-09-19T09:17:24.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.103+0000 7f79f37fe640 1 -- 172.21.15.120:0/852925904 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56461634dd70 con 0x5646164663a0 2024-09-19T09:17:24.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.103+0000 7f79f37fe640 10 client.29188 handle_command_reply: tid=0 2024-09-19T09:17:24.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.103+0000 7f7a1c9fa740 1 client.29188 shutdown 2024-09-19T09:17:24.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.103+0000 7f7a1c9fa740 1 -- 172.21.15.120:0/852925904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646163e05d0 msgr2=0x5646163f3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:24.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.103+0000 7f7a1c9fa740 1 --2- 172.21.15.120:0/852925904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646163e05d0 0x5646163f3bf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x5646163fdc30 tx=0x7f7a04012d60 comp rx=0 tx=0).stop 2024-09-19T09:17:24.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.103+0000 7f7a1c9fa740 1 -- 172.21.15.120:0/852925904 shutdown_connections 2024-09-19T09:17:24.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.103+0000 7f7a1c9fa740 1 --2- 172.21.15.120:0/852925904 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5646164663a0 0x5646164687b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.103+0000 7f7a1c9fa740 1 --2- 172.21.15.120:0/852925904 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646163eb370 0x5646163ef460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.103+0000 7f7a1c9fa740 1 --2- 172.21.15.120:0/852925904 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646163e1b40 0x5646163eef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.103+0000 7f7a1c9fa740 1 --2- 172.21.15.120:0/852925904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646163e05d0 0x5646163f3bf0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.103+0000 7f79f37fe640 0 client.29188 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:17:24.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.103+0000 7f7a1c9fa740 1 -- 172.21.15.120:0/852925904 >> 172.21.15.120:0/852925904 conn(0x5646163a9310 msgr2=0x5646163e70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:24.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.103+0000 7f7a1c9fa740 1 -- 172.21.15.120:0/852925904 shutdown_connections 2024-09-19T09:17:24.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.103+0000 7f7a1c9fa740 1 -- 172.21.15.120:0/852925904 wait complete. 2024-09-19T09:17:24.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.103+0000 7f7a1c9fa740 20 client.29188 trim_cache size 0 max 16384 2024-09-19T09:17:24.134 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:17:24.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.104+0000 7f7a1a621640 1 -- 172.21.15.120:0/2717613303 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7a000a3240 msgr2=0x7f7a000a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:24.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.104+0000 7f7a1a621640 1 --2- 172.21.15.120:0/2717613303 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7a000a3240 0x7f7a000a5700 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c04d000 tx=0x7f7a0c04a000 comp rx=0 tx=0).stop 2024-09-19T09:17:24.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.104+0000 7f7a1a621640 1 -- 172.21.15.120:0/2717613303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a14074eb0 msgr2=0x7f7a141a4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:24.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.104+0000 7f7a1a621640 1 --2- 172.21.15.120:0/2717613303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a14074eb0 0x7f7a141a4c40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f79fc00e9f0 tx=0x7f79fc00eeb0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.104+0000 7f7a1a621640 1 -- 172.21.15.120:0/2717613303 shutdown_connections 2024-09-19T09:17:24.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.104+0000 7f7a1a621640 1 --2- 172.21.15.120:0/2717613303 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7a000a3240 0x7f7a000a5700 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.104+0000 7f7a1a621640 1 --2- 172.21.15.120:0/2717613303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a14112b80 0x7f7a1419e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.104+0000 7f7a1a621640 1 --2- 172.21.15.120:0/2717613303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a14074eb0 0x7f7a141a4c40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.104+0000 7f7a1a621640 1 --2- 172.21.15.120:0/2717613303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a14074590 0x7f7a141a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:24.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.104+0000 7f7a1a621640 1 -- 172.21.15.120:0/2717613303 >> 172.21.15.120:0/2717613303 conn(0x7f7a140fd8b0 msgr2=0x7f7a140ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:24.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.104+0000 7f7a1a621640 1 -- 172.21.15.120:0/2717613303 shutdown_connections 2024-09-19T09:17:24.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:24.104+0000 7f7a1a621640 1 -- 172.21.15.120:0/2717613303 wait complete. 2024-09-19T09:17:24.357 INFO:tasks.workunit.client.0.smithi120.stdout: 10 35418 26.28 MB/sec warmup 79 sec latency 60.394 ms 2024-09-19T09:17:25.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:24 smithi120 ceph-mon[31150]: pgmap v781: 97 pgs: 97 active+clean; 289 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.1 MiB/s wr, 835 op/s 2024-09-19T09:17:25.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:24 smithi196 ceph-mon[35150]: pgmap v781: 97 pgs: 97 active+clean; 289 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.1 MiB/s wr, 835 op/s 2024-09-19T09:17:25.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:24 smithi153 ceph-mon[35071]: pgmap v781: 97 pgs: 97 active+clean; 289 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.1 MiB/s wr, 835 op/s 2024-09-19T09:17:25.277 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:17:25.354 INFO:tasks.workunit.client.0.smithi120.stdout: 10 36170 26.37 MB/sec warmup 80 sec latency 37.870 ms 2024-09-19T09:17:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.534+0000 7fcf78e4c640 1 -- 172.21.15.120:0/4225807521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf74100a50 msgr2=0x7fcf74100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.534+0000 7fcf78e4c640 1 --2- 172.21.15.120:0/4225807521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf74100a50 0x7fcf74100e50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009860 tx=0x7fcf6402a340 comp rx=0 tx=0).stop 2024-09-19T09:17:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.535+0000 7fcf78e4c640 1 -- 172.21.15.120:0/4225807521 shutdown_connections 2024-09-19T09:17:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.535+0000 7fcf78e4c640 1 --2- 172.21.15.120:0/4225807521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf74102600 0x7fcf74108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.535+0000 7fcf78e4c640 1 --2- 172.21.15.120:0/4225807521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf74101c40 0x7fcf741020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.535+0000 7fcf78e4c640 1 --2- 172.21.15.120:0/4225807521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf74100a50 0x7fcf74100e50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.535+0000 7fcf78e4c640 1 -- 172.21.15.120:0/4225807521 >> 172.21.15.120:0/4225807521 conn(0x7fcf740fc1b0 msgr2=0x7fcf740fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.535+0000 7fcf78e4c640 1 -- 172.21.15.120:0/4225807521 shutdown_connections 2024-09-19T09:17:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.535+0000 7fcf78e4c640 1 -- 172.21.15.120:0/4225807521 wait complete. 2024-09-19T09:17:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.536+0000 7fcf78e4c640 1 Processor -- start 2024-09-19T09:17:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.536+0000 7fcf78e4c640 1 -- start start 2024-09-19T09:17:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.537+0000 7fcf78e4c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf74100a50 0x7fcf74070970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.537+0000 7fcf78e4c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf74101c40 0x7fcf74070eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.537+0000 7fcf78e4c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf74102600 0x7fcf741aa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.537+0000 7fcf7259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf74100a50 0x7fcf74070970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.537+0000 7fcf78e4c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7410b610 con 0x7fcf74102600 2024-09-19T09:17:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.537+0000 7fcf78e4c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7410b490 con 0x7fcf74101c40 2024-09-19T09:17:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.537+0000 7fcf7259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf74100a50 0x7fcf74070970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:51804/0 (socket says 172.21.15.120:51804) 2024-09-19T09:17:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.537+0000 7fcf7259c640 1 -- 172.21.15.120:0/3042937849 learned_addr learned my addr 172.21.15.120:0/3042937849 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.537+0000 7fcf71d9b640 1 --2- 172.21.15.120:0/3042937849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf74101c40 0x7fcf74070eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.537+0000 7fcf78e4c640 1 -- 172.21.15.120:0/3042937849 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7410b790 con 0x7fcf74100a50 2024-09-19T09:17:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.537+0000 7fcf72d9d640 1 --2- 172.21.15.120:0/3042937849 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf74102600 0x7fcf741aa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.537+0000 7fcf7259c640 1 -- 172.21.15.120:0/3042937849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf74101c40 msgr2=0x7fcf74070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.537+0000 7fcf7259c640 1 --2- 172.21.15.120:0/3042937849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf74101c40 0x7fcf74070eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.537+0000 7fcf7259c640 1 -- 172.21.15.120:0/3042937849 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf74102600 msgr2=0x7fcf741aa0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.537+0000 7fcf7259c640 1 --2- 172.21.15.120:0/3042937849 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf74102600 0x7fcf741aa0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.537+0000 7fcf7259c640 1 -- 172.21.15.120:0/3042937849 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf74074030 con 0x7fcf74100a50 2024-09-19T09:17:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.537+0000 7fcf72d9d640 1 --2- 172.21.15.120:0/3042937849 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf74102600 0x7fcf741aa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:17:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.538+0000 7fcf7259c640 1 --2- 172.21.15.120:0/3042937849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf74100a50 0x7fcf74070970 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcf640057c0 tx=0x7fcf64033e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.538+0000 7fcf637fe640 1 -- 172.21.15.120:0/3042937849 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf64037040 con 0x7fcf74100a50 2024-09-19T09:17:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.538+0000 7fcf78e4c640 1 -- 172.21.15.120:0/3042937849 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf741aa610 con 0x7fcf74100a50 2024-09-19T09:17:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.538+0000 7fcf637fe640 1 -- 172.21.15.120:0/3042937849 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf64004050 con 0x7fcf74100a50 2024-09-19T09:17:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.538+0000 7fcf637fe640 1 -- 172.21.15.120:0/3042937849 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf64004920 con 0x7fcf74100a50 2024-09-19T09:17:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.538+0000 7fcf78e4c640 1 -- 172.21.15.120:0/3042937849 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf741aa9c0 con 0x7fcf74100a50 2024-09-19T09:17:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.541+0000 7fcf637fe640 1 -- 172.21.15.120:0/3042937849 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fcf64042020 con 0x7fcf74100a50 2024-09-19T09:17:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.542+0000 7fcf7bbe7740 1 Processor -- start 2024-09-19T09:17:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.542+0000 7fcf7bbe7740 1 -- start start 2024-09-19T09:17:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.542+0000 7fcf637fe640 1 --2- 172.21.15.120:0/3042937849 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcf5c0a3370 0x7fcf5c0a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.543+0000 7fcf72d9d640 1 --2- 172.21.15.120:0/3042937849 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcf5c0a3370 0x7fcf5c0a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.543+0000 7fcf7bbe7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e50e6859a0 0x55e50e685da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.543+0000 7fcf637fe640 1 -- 172.21.15.120:0/3042937849 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fcf640ef280 con 0x7fcf74100a50 2024-09-19T09:17:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.543+0000 7fcf7bbe7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e50e6862e0 0x55e50e68ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.543+0000 7fcf7bbe7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50e68b370 0x55e50e68d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.543+0000 7fcf7259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e50e6859a0 0x55e50e685da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.543+0000 7fcf7259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e50e6859a0 0x55e50e685da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37088/0 (socket says 172.21.15.120:37088) 2024-09-19T09:17:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.543+0000 7fcf7259c640 1 -- 172.21.15.120:0/633064060 learned_addr learned my addr 172.21.15.120:0/633064060 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.543+0000 7fcf7bbe7740 1 -- 172.21.15.120:0/633064060 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e50e5e92f0 con 0x55e50e6859a0 2024-09-19T09:17:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.543+0000 7fcf7bbe7740 1 -- 172.21.15.120:0/633064060 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e50e591270 con 0x55e50e6862e0 2024-09-19T09:17:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.543+0000 7fcf7bbe7740 1 -- 172.21.15.120:0/633064060 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e50e5e52b0 con 0x55e50e68b370 2024-09-19T09:17:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.543+0000 7fcf71d9b640 1 --2- 172.21.15.120:0/633064060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e50e6862e0 0x55e50e68ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.543+0000 7fcf72d9d640 1 --2- 172.21.15.120:0/633064060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50e68b370 0x55e50e68d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.543+0000 7fcf72d9d640 1 --2- 172.21.15.120:0/3042937849 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcf5c0a3370 0x7fcf5c0a5830 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c005fd0 tx=0x7fcf6c00f040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.543+0000 7fcf7259c640 1 -- 172.21.15.120:0/633064060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50e68b370 msgr2=0x55e50e68d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.543+0000 7fcf7259c640 1 --2- 172.21.15.120:0/633064060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50e68b370 0x55e50e68d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.543+0000 7fcf7259c640 1 -- 172.21.15.120:0/633064060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e50e6862e0 msgr2=0x55e50e68ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.543+0000 7fcf7259c640 1 --2- 172.21.15.120:0/633064060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e50e6862e0 0x55e50e68ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.543+0000 7fcf7259c640 1 -- 172.21.15.120:0/633064060 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e50e590340 con 0x55e50e6859a0 2024-09-19T09:17:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.544+0000 7fcf7259c640 1 --2- 172.21.15.120:0/633064060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e50e6859a0 0x55e50e685da0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fcf640c3fe0 tx=0x7fcf6402a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efcd4c40df6115e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.544+0000 7fcf60ff9640 1 -- 172.21.15.120:0/633064060 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6400b0c0 con 0x55e50e6859a0 2024-09-19T09:17:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.544+0000 7fcf60ff9640 1 -- 172.21.15.120:0/633064060 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf640bb930 con 0x55e50e6859a0 2024-09-19T09:17:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.544+0000 7fcf60ff9640 1 -- 172.21.15.120:0/633064060 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf640b7fa0 con 0x55e50e6859a0 2024-09-19T09:17:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.545+0000 7fcf7bbe7740 1 -- 172.21.15.120:0/633064060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e50e6859a0 msgr2=0x55e50e685da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.545+0000 7fcf7bbe7740 1 --2- 172.21.15.120:0/633064060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e50e6859a0 0x55e50e685da0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fcf640c3fe0 tx=0x7fcf6402a340 comp rx=0 tx=0).stop 2024-09-19T09:17:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.545+0000 7fcf7bbe7740 1 -- 172.21.15.120:0/633064060 shutdown_connections 2024-09-19T09:17:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.545+0000 7fcf7bbe7740 1 --2- 172.21.15.120:0/633064060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50e68b370 0x55e50e68d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.545+0000 7fcf7bbe7740 1 --2- 172.21.15.120:0/633064060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e50e6862e0 0x55e50e68ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.545+0000 7fcf7bbe7740 1 --2- 172.21.15.120:0/633064060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e50e6859a0 0x55e50e685da0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.545+0000 7fcf7bbe7740 1 -- 172.21.15.120:0/633064060 >> 172.21.15.120:0/633064060 conn(0x55e50e681010 msgr2=0x55e50e681410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.545+0000 7fcf7bbe7740 1 -- 172.21.15.120:0/633064060 shutdown_connections 2024-09-19T09:17:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.545+0000 7fcf7bbe7740 1 -- 172.21.15.120:0/633064060 wait complete. 2024-09-19T09:17:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.545+0000 7fcf7bbe7740 1 -- start start 2024-09-19T09:17:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.545+0000 7fcf7bbe7740 1 Processor -- start 2024-09-19T09:17:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.546+0000 7fcf7bbe7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50e6859a0 0x55e50e68f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.546+0000 7fcf7bbe7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e50e6862e0 0x55e50e68fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.546+0000 7fcf7bbe7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e50e68b370 0x55e50e696ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.546+0000 7fcf7bbe7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e50e68e100 con 0x55e50e68b370 2024-09-19T09:17:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.546+0000 7fcf7bbe7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e50e572290 con 0x55e50e6862e0 2024-09-19T09:17:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.546+0000 7fcf7bbe7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e50e68e280 con 0x55e50e6859a0 2024-09-19T09:17:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.546+0000 7fcf7259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50e6859a0 0x55e50e68f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.546+0000 7fcf7259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50e6859a0 0x55e50e68f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:51830/0 (socket says 172.21.15.120:51830) 2024-09-19T09:17:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.546+0000 7fcf7259c640 1 -- 172.21.15.120:0/961903746 learned_addr learned my addr 172.21.15.120:0/961903746 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.546+0000 7fcf71d9b640 1 --2- 172.21.15.120:0/961903746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e50e6862e0 0x55e50e68fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.546+0000 7fcf72d9d640 1 --2- 172.21.15.120:0/961903746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e50e68b370 0x55e50e696ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.546+0000 7fcf7bbe7740 10 client.0 _ll_register_callbacks cb 0x55e50e5a3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:17:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.546+0000 7fcf7259c640 1 -- 172.21.15.120:0/961903746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e50e6862e0 msgr2=0x55e50e68fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.546+0000 7fcf7259c640 1 --2- 172.21.15.120:0/961903746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e50e6862e0 0x55e50e68fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.546+0000 7fcf7259c640 1 -- 172.21.15.120:0/961903746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e50e68b370 msgr2=0x55e50e696ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.546+0000 7fcf7259c640 1 --2- 172.21.15.120:0/961903746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e50e68b370 0x55e50e696ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.546+0000 7fcf7259c640 1 -- 172.21.15.120:0/961903746 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e50e2fc210 con 0x55e50e6859a0 2024-09-19T09:17:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.546+0000 7fcf7259c640 1 --2- 172.21.15.120:0/961903746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50e6859a0 0x55e50e68f920 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcf640bb630 tx=0x7fcf64005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f80603ecc5e7e5d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.546+0000 7fcf59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:17:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.546+0000 7fcf59ffb640 1 -- 172.21.15.120:0/961903746 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf640d3020 con 0x55e50e6859a0 2024-09-19T09:17:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.546+0000 7fcf59ffb640 1 -- 172.21.15.120:0/961903746 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fcf640c20e0 con 0x55e50e6859a0 2024-09-19T09:17:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.546+0000 7fcf7bbe7740 1 -- 172.21.15.120:0/961903746 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf640bb930 con 0x55e50e6859a0 2024-09-19T09:17:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.547+0000 7fcf59ffb640 1 -- 172.21.15.120:0/961903746 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcf640bfce0 con 0x55e50e6859a0 2024-09-19T09:17:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.547+0000 7fcf7bbe7740 10 client.29200 fetch_fsmap learned FSMap version 36 2024-09-19T09:17:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.547+0000 7fcf7bbe7740 1 -- 172.21.15.120:0/961903746 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e50e3c8620 con 0x55e50e6859a0 2024-09-19T09:17:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.547+0000 7fcf59ffb640 1 -- 172.21.15.120:0/961903746 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf640bb930 con 0x55e50e6859a0 2024-09-19T09:17:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.547+0000 7fcf59ffb640 1 -- 172.21.15.120:0/961903746 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fcf6403a070 con 0x55e50e6859a0 2024-09-19T09:17:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.547+0000 7fcf7bbe7740 10 client.29200 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:17:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.547+0000 7fcf7bbe7740 10 client.29200 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:17:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.547+0000 7fcf7bbe7740 20 client.29200 populate_metadata read hostname 'smithi120' 2024-09-19T09:17:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.547+0000 7fcf7bbe7740 1 --2- 172.21.15.120:0/961903746 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e50e69b350 0x55e50e69d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.547+0000 7fcf7bbe7740 4 client.29200 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:17:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.547+0000 7fcf7bbe7740 1 -- 172.21.15.120:0/961903746 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e50e5f6a20 con 0x55e50e69b350 2024-09-19T09:17:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.547+0000 7fcf71d9b640 1 --2- 172.21.15.120:0/961903746 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e50e69b350 0x55e50e69d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.548+0000 7fcf59ffb640 10 client.29200 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.548+0000 7fcf71d9b640 1 --2- 172.21.15.120:0/961903746 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e50e69b350 0x55e50e69d780 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fa2a66bc02f7c73 server_cookie=524f4b67de835f17 in_seq=0 out_seq=0 2024-09-19T09:17:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.549+0000 7fcf59ffb640 1 -- 172.21.15.120:0/961903746 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e50e5f6a20 con 0x55e50e69b350 2024-09-19T09:17:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.549+0000 7fcf59ffb640 10 client.29200 handle_command_reply: tid=0 2024-09-19T09:17:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.549+0000 7fcf7bbe7740 1 client.29200 shutdown 2024-09-19T09:17:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.549+0000 7fcf7bbe7740 1 -- 172.21.15.120:0/961903746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50e6859a0 msgr2=0x55e50e68f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.549+0000 7fcf7bbe7740 1 --2- 172.21.15.120:0/961903746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50e6859a0 0x55e50e68f920 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcf640bb630 tx=0x7fcf64005b20 comp rx=0 tx=0).stop 2024-09-19T09:17:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.549+0000 7fcf7bbe7740 1 -- 172.21.15.120:0/961903746 shutdown_connections 2024-09-19T09:17:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.549+0000 7fcf7bbe7740 1 --2- 172.21.15.120:0/961903746 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e50e69b350 0x55e50e69d780 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.549+0000 7fcf7bbe7740 1 --2- 172.21.15.120:0/961903746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e50e68b370 0x55e50e696ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.550+0000 7fcf59ffb640 0 client.29200 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.550+0000 7fcf7bbe7740 1 --2- 172.21.15.120:0/961903746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e50e6862e0 0x55e50e68fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.550+0000 7fcf7bbe7740 1 --2- 172.21.15.120:0/961903746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50e6859a0 0x55e50e68f920 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.550+0000 7fcf7bbe7740 1 -- 172.21.15.120:0/961903746 >> 172.21.15.120:0/961903746 conn(0x55e50e681010 msgr2=0x55e50e682820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.551+0000 7fcf7bbe7740 1 -- 172.21.15.120:0/961903746 shutdown_connections 2024-09-19T09:17:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.551+0000 7fcf7bbe7740 1 -- 172.21.15.120:0/961903746 wait complete. 2024-09-19T09:17:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.551+0000 7fcf7bbe7740 20 client.29200 trim_cache size 0 max 16384 2024-09-19T09:17:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.573+0000 7fcf7bbe7740 1 Processor -- start 2024-09-19T09:17:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.573+0000 7fcf7bbe7740 1 -- start start 2024-09-19T09:17:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.573+0000 7fcf7bbe7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50e6859a0 0x55e50e681ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.573+0000 7fcf7bbe7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e50e6862e0 0x55e50e696dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.573+0000 7fcf7bbe7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e50e68b370 0x55e50e682400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.573+0000 7fcf7bbe7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcf741b3a40 con 0x55e50e6862e0 2024-09-19T09:17:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.573+0000 7fcf7bbe7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcf741b5610 con 0x55e50e68b370 2024-09-19T09:17:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.573+0000 7fcf7bbe7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcf741b4110 con 0x55e50e6859a0 2024-09-19T09:17:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.574+0000 7fcf71d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e50e6862e0 0x55e50e696dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.574+0000 7fcf7259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50e6859a0 0x55e50e681ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.574+0000 7fcf71d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e50e6862e0 0x55e50e696dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37098/0 (socket says 172.21.15.120:37098) 2024-09-19T09:17:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.574+0000 7fcf71d9b640 1 -- 172.21.15.120:0/3160574762 learned_addr learned my addr 172.21.15.120:0/3160574762 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.574+0000 7fcf72d9d640 1 --2- 172.21.15.120:0/3160574762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e50e68b370 0x55e50e682400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.574+0000 7fcf71d9b640 1 -- 172.21.15.120:0/3160574762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50e6859a0 msgr2=0x55e50e681ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.574+0000 7fcf71d9b640 1 --2- 172.21.15.120:0/3160574762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50e6859a0 0x55e50e681ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.574+0000 7fcf71d9b640 1 -- 172.21.15.120:0/3160574762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e50e68b370 msgr2=0x55e50e682400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.574+0000 7fcf71d9b640 1 --2- 172.21.15.120:0/3160574762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e50e68b370 0x55e50e682400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.574+0000 7fcf71d9b640 1 -- 172.21.15.120:0/3160574762 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e50e68f460 con 0x55e50e6862e0 2024-09-19T09:17:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.574+0000 7fcf71d9b640 1 --2- 172.21.15.120:0/3160574762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e50e6862e0 0x55e50e696dc0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x55e50e681e90 tx=0x7fcf68005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808552915fce2be server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.574+0000 7fcf5bfff640 1 -- 172.21.15.120:0/3160574762 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68022070 con 0x55e50e6862e0 2024-09-19T09:17:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.575+0000 7fcf5bfff640 1 -- 172.21.15.120:0/3160574762 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf6801fb10 con 0x55e50e6862e0 2024-09-19T09:17:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.575+0000 7fcf5bfff640 1 -- 172.21.15.120:0/3160574762 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68026df0 con 0x55e50e6862e0 2024-09-19T09:17:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.576+0000 7fcf7bbe7740 1 -- 172.21.15.120:0/3160574762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e50e6862e0 msgr2=0x55e50e696dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.576+0000 7fcf7bbe7740 1 --2- 172.21.15.120:0/3160574762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e50e6862e0 0x55e50e696dc0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x55e50e681e90 tx=0x7fcf68005f90 comp rx=0 tx=0).stop 2024-09-19T09:17:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.576+0000 7fcf7bbe7740 1 -- 172.21.15.120:0/3160574762 shutdown_connections 2024-09-19T09:17:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.576+0000 7fcf7bbe7740 1 --2- 172.21.15.120:0/3160574762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e50e68b370 0x55e50e682400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.576+0000 7fcf7bbe7740 1 --2- 172.21.15.120:0/3160574762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e50e6862e0 0x55e50e696dc0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.576+0000 7fcf7bbe7740 1 --2- 172.21.15.120:0/3160574762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50e6859a0 0x55e50e681ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.576+0000 7fcf7bbe7740 1 -- 172.21.15.120:0/3160574762 >> 172.21.15.120:0/3160574762 conn(0x55e50e681010 msgr2=0x55e50e68f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.576+0000 7fcf7bbe7740 1 -- 172.21.15.120:0/3160574762 shutdown_connections 2024-09-19T09:17:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.576+0000 7fcf7bbe7740 1 -- 172.21.15.120:0/3160574762 wait complete. 2024-09-19T09:17:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.576+0000 7fcf7bbe7740 1 -- start start 2024-09-19T09:17:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.576+0000 7fcf7bbe7740 1 Processor -- start 2024-09-19T09:17:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.577+0000 7fcf7bbe7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50e6859a0 0x55e50e694710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.577+0000 7fcf7bbe7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e50e6862e0 0x55e50e694c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.577+0000 7fcf7bbe7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e50e68b370 0x55e50e695c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.577+0000 7fcf7bbe7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcf741b4f40 con 0x55e50e68b370 2024-09-19T09:17:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.577+0000 7fcf7bbe7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcf741b47e0 con 0x55e50e6862e0 2024-09-19T09:17:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.577+0000 7fcf7bbe7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcf741b1a00 con 0x55e50e6859a0 2024-09-19T09:17:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.577+0000 7fcf72d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e50e68b370 0x55e50e695c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.577+0000 7fcf7bbe7740 10 client.0 _ll_register_callbacks cb 0x7fcf640bb470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:17:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.577+0000 7fcf72d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e50e68b370 0x55e50e695c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37106/0 (socket says 172.21.15.120:37106) 2024-09-19T09:17:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.577+0000 7fcf72d9d640 1 -- 172.21.15.120:0/2352964533 learned_addr learned my addr 172.21.15.120:0/2352964533 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.577+0000 7fcf72d9d640 1 -- 172.21.15.120:0/2352964533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50e6859a0 msgr2=0x55e50e694710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:17:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.577+0000 7fcf72d9d640 1 --2- 172.21.15.120:0/2352964533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50e6859a0 0x55e50e694710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.577+0000 7fcf72d9d640 1 -- 172.21.15.120:0/2352964533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e50e6862e0 msgr2=0x55e50e694c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.577+0000 7fcf71d9b640 1 --2- 172.21.15.120:0/2352964533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e50e6862e0 0x55e50e694c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.577+0000 7fcf72d9d640 1 --2- 172.21.15.120:0/2352964533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e50e6862e0 0x55e50e694c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.577+0000 7fcf72d9d640 1 -- 172.21.15.120:0/2352964533 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e50e696220 con 0x55e50e68b370 2024-09-19T09:17:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.577+0000 7fcf71d9b640 1 --2- 172.21.15.120:0/2352964533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e50e6862e0 0x55e50e694c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:17:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.578+0000 7fcf72d9d640 1 --2- 172.21.15.120:0/2352964533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e50e68b370 0x55e50e695c10 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c025a90 tx=0x7fcf6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17375b5589bd80ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.578+0000 7fcf59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:17:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.578+0000 7fcf59ffb640 1 -- 172.21.15.120:0/2352964533 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c019e40 con 0x55e50e68b370 2024-09-19T09:17:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.578+0000 7fcf59ffb640 1 -- 172.21.15.120:0/2352964533 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fcf6c02d3e0 con 0x55e50e68b370 2024-09-19T09:17:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.578+0000 7fcf7bbe7740 1 -- 172.21.15.120:0/2352964533 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf6801fb10 con 0x55e50e68b370 2024-09-19T09:17:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.578+0000 7fcf59ffb640 1 -- 172.21.15.120:0/2352964533 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcf6c02d620 con 0x55e50e68b370 2024-09-19T09:17:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.578+0000 7fcf7bbe7740 10 client.20421 fetch_fsmap learned FSMap version 36 2024-09-19T09:17:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.578+0000 7fcf7bbe7740 1 -- 172.21.15.120:0/2352964533 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e50e706250 con 0x55e50e68b370 2024-09-19T09:17:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.578+0000 7fcf59ffb640 1 -- 172.21.15.120:0/2352964533 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf6801fb10 con 0x55e50e68b370 2024-09-19T09:17:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.579+0000 7fcf59ffb640 1 -- 172.21.15.120:0/2352964533 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fcf6c02c070 con 0x55e50e68b370 2024-09-19T09:17:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.579+0000 7fcf7bbe7740 10 client.20421 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:17:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.579+0000 7fcf7bbe7740 10 client.20421 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:17:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.579+0000 7fcf7bbe7740 20 client.20421 populate_metadata read hostname 'smithi120' 2024-09-19T09:17:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.579+0000 7fcf7bbe7740 1 --2- 172.21.15.120:0/2352964533 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e50e69b350 0x55e50e708d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.579+0000 7fcf7bbe7740 4 client.20421 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:17:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.579+0000 7fcf7bbe7740 1 -- 172.21.15.120:0/2352964533 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e50e5f0d70 con 0x55e50e69b350 2024-09-19T09:17:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.579+0000 7fcf7259c640 1 --2- 172.21.15.120:0/2352964533 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e50e69b350 0x55e50e708d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.580+0000 7fcf59ffb640 10 client.20421 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:25.580+0000 7fcf7259c640 1 --2- 172.21.15.120:0/2352964533 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e50e69b350 0x55e50e708d70 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec8cef767f4aa088 server_cookie=f2d9b9e69d147b5e in_seq=0 out_seq=0 2024-09-19T09:17:25.624 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:17:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:26.022+0000 7fcf59ffb640 1 -- 172.21.15.120:0/2352964533 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e50e5f0d70 con 0x55e50e69b350 2024-09-19T09:17:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:26.022+0000 7fcf59ffb640 10 client.20421 handle_command_reply: tid=0 2024-09-19T09:17:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:26.022+0000 7fcf7bbe7740 1 client.20421 shutdown 2024-09-19T09:17:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:26.022+0000 7fcf7bbe7740 1 -- 172.21.15.120:0/2352964533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e50e68b370 msgr2=0x55e50e695c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:26.022+0000 7fcf7bbe7740 1 --2- 172.21.15.120:0/2352964533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e50e68b370 0x55e50e695c10 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c025a90 tx=0x7fcf6c004a40 comp rx=0 tx=0).stop 2024-09-19T09:17:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:26.023+0000 7fcf7bbe7740 1 -- 172.21.15.120:0/2352964533 shutdown_connections 2024-09-19T09:17:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:26.023+0000 7fcf7bbe7740 1 --2- 172.21.15.120:0/2352964533 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e50e69b350 0x55e50e708d70 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:26.023+0000 7fcf7bbe7740 1 --2- 172.21.15.120:0/2352964533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e50e68b370 0x55e50e695c10 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:26.023+0000 7fcf7bbe7740 1 --2- 172.21.15.120:0/2352964533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e50e6862e0 0x55e50e694c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:26.023+0000 7fcf59ffb640 0 client.20421 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:26.023+0000 7fcf7bbe7740 1 --2- 172.21.15.120:0/2352964533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50e6859a0 0x55e50e694710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:26.023+0000 7fcf7bbe7740 1 -- 172.21.15.120:0/2352964533 >> 172.21.15.120:0/2352964533 conn(0x55e50e681010 msgr2=0x55e50e67f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:26.023+0000 7fcf7bbe7740 1 -- 172.21.15.120:0/2352964533 shutdown_connections 2024-09-19T09:17:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:26.023+0000 7fcf7bbe7740 1 -- 172.21.15.120:0/2352964533 wait complete. 2024-09-19T09:17:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:26.023+0000 7fcf7bbe7740 20 client.20421 trim_cache size 0 max 16384 2024-09-19T09:17:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:17:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:17:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:17:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:17:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:26.023+0000 7fcf78e4c640 1 -- 172.21.15.120:0/3042937849 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcf5c0a3370 msgr2=0x7fcf5c0a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:26.023+0000 7fcf78e4c640 1 --2- 172.21.15.120:0/3042937849 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcf5c0a3370 0x7fcf5c0a5830 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c005fd0 tx=0x7fcf6c00f040 comp rx=0 tx=0).stop 2024-09-19T09:17:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:26.023+0000 7fcf78e4c640 1 -- 172.21.15.120:0/3042937849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf74100a50 msgr2=0x7fcf74070970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:26.023+0000 7fcf78e4c640 1 --2- 172.21.15.120:0/3042937849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf74100a50 0x7fcf74070970 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcf640057c0 tx=0x7fcf64033e00 comp rx=0 tx=0).stop 2024-09-19T09:17:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:26.023+0000 7fcf78e4c640 1 -- 172.21.15.120:0/3042937849 shutdown_connections 2024-09-19T09:17:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:26.023+0000 7fcf78e4c640 1 --2- 172.21.15.120:0/3042937849 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcf5c0a3370 0x7fcf5c0a5830 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:26.023+0000 7fcf78e4c640 1 --2- 172.21.15.120:0/3042937849 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf74102600 0x7fcf741aa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:26.023+0000 7fcf78e4c640 1 --2- 172.21.15.120:0/3042937849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf74101c40 0x7fcf74070eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:26.023+0000 7fcf78e4c640 1 --2- 172.21.15.120:0/3042937849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf74100a50 0x7fcf74070970 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:26.023+0000 7fcf78e4c640 1 -- 172.21.15.120:0/3042937849 >> 172.21.15.120:0/3042937849 conn(0x7fcf740fc1b0 msgr2=0x7fcf74104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:26.024+0000 7fcf78e4c640 1 -- 172.21.15.120:0/3042937849 shutdown_connections 2024-09-19T09:17:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:26.024+0000 7fcf78e4c640 1 -- 172.21.15.120:0/3042937849 wait complete. 2024-09-19T09:17:26.354 INFO:tasks.workunit.client.0.smithi120.stdout: 10 36747 26.55 MB/sec warmup 81 sec latency 254.251 ms 2024-09-19T09:17:26.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.827+0000 7f3f76636640 1 -- 172.21.15.120:0/4142353735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f70102ef0 msgr2=0x7f3f7010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:26.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.827+0000 7f3f76636640 1 --2- 172.21.15.120:0/4142353735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f70102ef0 0x7f3f7010da40 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f3f68009ca0 tx=0x7f3f6802a950 comp rx=0 tx=0).stop 2024-09-19T09:17:26.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.827+0000 7f3f76636640 1 -- 172.21.15.120:0/4142353735 shutdown_connections 2024-09-19T09:17:26.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.827+0000 7f3f76636640 1 --2- 172.21.15.120:0/4142353735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f70102ef0 0x7f3f7010da40 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.827+0000 7f3f76636640 1 --2- 172.21.15.120:0/4142353735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f70102570 0x7f3f701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.827+0000 7f3f76636640 1 --2- 172.21.15.120:0/4142353735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f70100e20 0x7f3f70101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.827+0000 7f3f76636640 1 -- 172.21.15.120:0/4142353735 >> 172.21.15.120:0/4142353735 conn(0x7f3f700fc7d0 msgr2=0x7f3f700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:26.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.827+0000 7f3f76636640 1 -- 172.21.15.120:0/4142353735 shutdown_connections 2024-09-19T09:17:26.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.827+0000 7f3f76636640 1 -- 172.21.15.120:0/4142353735 wait complete. 2024-09-19T09:17:26.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.827+0000 7f3f76636640 1 Processor -- start 2024-09-19T09:17:26.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.828+0000 7f3f76636640 1 -- start start 2024-09-19T09:17:26.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.828+0000 7f3f76636640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f70100e20 0x7f3f701984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:26.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.828+0000 7f3f76636640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f70102570 0x7f3f701989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:26.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.828+0000 7f3f76636640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f70102ef0 0x7f3f7019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:26.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.828+0000 7f3f76636640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3f701120a0 con 0x7f3f70102ef0 2024-09-19T09:17:26.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.828+0000 7f3f76636640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3f70111f20 con 0x7f3f70102570 2024-09-19T09:17:26.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.828+0000 7f3f76636640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3f70112220 con 0x7f3f70100e20 2024-09-19T09:17:26.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.828+0000 7f3f74bd3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f70102ef0 0x7f3f7019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:26.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.828+0000 7f3f74bd3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f70102ef0 0x7f3f7019cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37122/0 (socket says 172.21.15.120:37122) 2024-09-19T09:17:26.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.828+0000 7f3f74bd3640 1 -- 172.21.15.120:0/3201445289 learned_addr learned my addr 172.21.15.120:0/3201445289 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:26.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.828+0000 7f3f6ffff640 1 --2- 172.21.15.120:0/3201445289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f70100e20 0x7f3f701984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:26.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.828+0000 7f3f6f7fe640 1 --2- 172.21.15.120:0/3201445289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f70102570 0x7f3f701989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:26.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.828+0000 7f3f74bd3640 1 -- 172.21.15.120:0/3201445289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f70100e20 msgr2=0x7f3f701984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:26.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.828+0000 7f3f74bd3640 1 --2- 172.21.15.120:0/3201445289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f70100e20 0x7f3f701984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.828+0000 7f3f74bd3640 1 -- 172.21.15.120:0/3201445289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f70102570 msgr2=0x7f3f701989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:26.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.828+0000 7f3f74bd3640 1 --2- 172.21.15.120:0/3201445289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f70102570 0x7f3f701989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.828+0000 7f3f74bd3640 1 -- 172.21.15.120:0/3201445289 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f7019d450 con 0x7f3f70102ef0 2024-09-19T09:17:26.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.828+0000 7f3f6ffff640 1 --2- 172.21.15.120:0/3201445289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f70100e20 0x7f3f701984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:17:26.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.828+0000 7f3f6f7fe640 1 --2- 172.21.15.120:0/3201445289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f70102570 0x7f3f701989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:17:26.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.829+0000 7f3f74bd3640 1 --2- 172.21.15.120:0/3201445289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f70102ef0 0x7f3f7019cd70 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f3f68009dc0 tx=0x7f3f68003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:26.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.829+0000 7f3f6d7fa640 1 -- 172.21.15.120:0/3201445289 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f68034040 con 0x7f3f70102ef0 2024-09-19T09:17:26.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.829+0000 7f3f76636640 1 -- 172.21.15.120:0/3201445289 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f7019d6e0 con 0x7f3f70102ef0 2024-09-19T09:17:26.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.829+0000 7f3f76636640 1 -- 172.21.15.120:0/3201445289 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f701a5020 con 0x7f3f70102ef0 2024-09-19T09:17:26.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.829+0000 7f3f6d7fa640 1 -- 172.21.15.120:0/3201445289 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f6802f930 con 0x7f3f70102ef0 2024-09-19T09:17:26.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.829+0000 7f3f6d7fa640 1 -- 172.21.15.120:0/3201445289 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f680046a0 con 0x7f3f70102ef0 2024-09-19T09:17:26.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.832+0000 7f3f78a0f740 1 Processor -- start 2024-09-19T09:17:26.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.833+0000 7f3f6d7fa640 1 -- 172.21.15.120:0/3201445289 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f3f68037020 con 0x7f3f70102ef0 2024-09-19T09:17:26.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.833+0000 7f3f78a0f740 1 -- start start 2024-09-19T09:17:26.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.833+0000 7f3f78a0f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56252fd8eb40 0x56252fd8d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:26.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.833+0000 7f3f78a0f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56252fd8d5d0 0x56252fd8da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:26.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.833+0000 7f3f78a0f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56252fd98370 0x56252fd9a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:26.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.833+0000 7f3f78a0f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56252fcf32f0 con 0x56252fd8d5d0 2024-09-19T09:17:26.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.833+0000 7f3f78a0f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56252fc9acf0 con 0x56252fd98370 2024-09-19T09:17:26.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.833+0000 7f3f78a0f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56252fcef2b0 con 0x56252fd8eb40 2024-09-19T09:17:26.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.833+0000 7f3f6ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56252fd8d5d0 0x56252fd8da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:26.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.833+0000 7f3f6f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56252fd8eb40 0x56252fd8d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:26.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.833+0000 7f3f6f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56252fd8eb40 0x56252fd8d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:51882/0 (socket says 172.21.15.120:51882) 2024-09-19T09:17:26.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.833+0000 7f3f6f7fe640 1 -- 172.21.15.120:0/369700523 learned_addr learned my addr 172.21.15.120:0/369700523 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:26.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.833+0000 7f3f6ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56252fd8d5d0 0x56252fd8da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37126/0 (socket says 172.21.15.120:37126) 2024-09-19T09:17:26.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.833+0000 7f3f6f7fe640 1 --2- 172.21.15.120:0/369700523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56252fd98370 0x56252fd9a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:26.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.834+0000 7f3f6ffff640 1 -- 172.21.15.120:0/369700523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56252fd8eb40 msgr2=0x56252fd8d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:26.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.834+0000 7f3f6ffff640 1 --2- 172.21.15.120:0/369700523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56252fd8eb40 0x56252fd8d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.834+0000 7f3f6ffff640 1 -- 172.21.15.120:0/369700523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56252fd98370 msgr2=0x56252fd9a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:26.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.834+0000 7f3f6ffff640 1 --2- 172.21.15.120:0/369700523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56252fd98370 0x56252fd9a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.834+0000 7f3f6ffff640 1 -- 172.21.15.120:0/369700523 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56252fc7f3a0 con 0x56252fd8d5d0 2024-09-19T09:17:26.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.834+0000 7f3f6f7fe640 1 --2- 172.21.15.120:0/369700523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56252fd8eb40 0x56252fd8d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:17:26.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.834+0000 7f3f6d7fa640 1 --2- 172.21.15.120:0/3201445289 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3f580a3240 0x7f3f580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:26.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.834+0000 7f3f6ffff640 1 --2- 172.21.15.120:0/369700523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56252fd8d5d0 0x56252fd8da10 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f3f60009fd0 tx=0x7f3f600118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e3ea01b4a26bc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:26.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.834+0000 7f3f6d7fa640 1 -- 172.21.15.120:0/3201445289 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f3f680f0500 con 0x7f3f70102ef0 2024-09-19T09:17:26.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.834+0000 7f3f6f7fe640 1 --2- 172.21.15.120:0/3201445289 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3f580a3240 0x7f3f580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:26.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.834+0000 7f3f567fc640 1 -- 172.21.15.120:0/369700523 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f60004310 con 0x56252fd8d5d0 2024-09-19T09:17:26.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.834+0000 7f3f567fc640 1 -- 172.21.15.120:0/369700523 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f600044b0 con 0x56252fd8d5d0 2024-09-19T09:17:26.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.834+0000 7f3f567fc640 1 -- 172.21.15.120:0/369700523 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f600035f0 con 0x56252fd8d5d0 2024-09-19T09:17:26.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.835+0000 7f3f78a0f740 1 -- 172.21.15.120:0/369700523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56252fd8d5d0 msgr2=0x56252fd8da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:26.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.835+0000 7f3f6f7fe640 1 --2- 172.21.15.120:0/3201445289 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3f580a3240 0x7f3f580a5700 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x56252fd8e070 tx=0x7f3f64005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:26.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.835+0000 7f3f78a0f740 1 --2- 172.21.15.120:0/369700523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56252fd8d5d0 0x56252fd8da10 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f3f60009fd0 tx=0x7f3f600118d0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.835+0000 7f3f78a0f740 1 -- 172.21.15.120:0/369700523 shutdown_connections 2024-09-19T09:17:26.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.835+0000 7f3f78a0f740 1 --2- 172.21.15.120:0/369700523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56252fd98370 0x56252fd9a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.835+0000 7f3f78a0f740 1 --2- 172.21.15.120:0/369700523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56252fd8d5d0 0x56252fd8da10 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.835+0000 7f3f78a0f740 1 --2- 172.21.15.120:0/369700523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56252fd8eb40 0x56252fd8d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.835+0000 7f3f78a0f740 1 -- 172.21.15.120:0/369700523 >> 172.21.15.120:0/369700523 conn(0x56252fd56310 msgr2=0x56252fd89c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:26.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.836+0000 7f3f78a0f740 1 -- 172.21.15.120:0/369700523 shutdown_connections 2024-09-19T09:17:26.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.836+0000 7f3f78a0f740 1 -- 172.21.15.120:0/369700523 wait complete. 2024-09-19T09:17:26.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.836+0000 7f3f78a0f740 1 -- start start 2024-09-19T09:17:26.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.836+0000 7f3f78a0f740 1 Processor -- start 2024-09-19T09:17:26.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.837+0000 7f3f78a0f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56252fd8d5d0 0x56252fd97ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:26.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.837+0000 7f3f78a0f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56252fd8eb40 0x56252fd94f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:26.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.837+0000 7f3f78a0f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56252fd98370 0x56252fd95490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:26.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.837+0000 7f3f78a0f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56252fd9b070 con 0x56252fd8eb40 2024-09-19T09:17:26.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.837+0000 7f3f78a0f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56252fcae060 con 0x56252fd98370 2024-09-19T09:17:26.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.837+0000 7f3f78a0f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56252fd9b1f0 con 0x56252fd8d5d0 2024-09-19T09:17:26.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.837+0000 7f3f78a0f740 10 client.0 _ll_register_callbacks cb 0x56252fa018c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:17:26.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.837+0000 7f3f6f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56252fd8eb40 0x56252fd94f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:26.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.837+0000 7f3f6f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56252fd8eb40 0x56252fd94f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37140/0 (socket says 172.21.15.120:37140) 2024-09-19T09:17:26.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.837+0000 7f3f6f7fe640 1 -- 172.21.15.120:0/2761399079 learned_addr learned my addr 172.21.15.120:0/2761399079 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:26.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.837+0000 7f3f6ffff640 1 --2- 172.21.15.120:0/2761399079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56252fd8d5d0 0x56252fd97ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:26.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.837+0000 7f3f74bd3640 1 --2- 172.21.15.120:0/2761399079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56252fd98370 0x56252fd95490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:26.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.837+0000 7f3f6f7fe640 1 -- 172.21.15.120:0/2761399079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56252fd8d5d0 msgr2=0x56252fd97ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:26.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.837+0000 7f3f6f7fe640 1 --2- 172.21.15.120:0/2761399079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56252fd8d5d0 0x56252fd97ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.837+0000 7f3f6f7fe640 1 -- 172.21.15.120:0/2761399079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56252fd98370 msgr2=0x56252fd95490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:26.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.837+0000 7f3f6f7fe640 1 --2- 172.21.15.120:0/2761399079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56252fd98370 0x56252fd95490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.837+0000 7f3f6f7fe640 1 -- 172.21.15.120:0/2761399079 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56252fa643c0 con 0x56252fd8eb40 2024-09-19T09:17:26.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.837+0000 7f3f6f7fe640 1 --2- 172.21.15.120:0/2761399079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56252fd8eb40 0x56252fd94f50 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f3f64010e20 tx=0x7f3f640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4efe66830400027e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:26.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.837+0000 7f3f4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:17:26.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.838+0000 7f3f4f7fe640 1 -- 172.21.15.120:0/2761399079 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f64022e40 con 0x56252fd8eb40 2024-09-19T09:17:26.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.838+0000 7f3f4f7fe640 1 -- 172.21.15.120:0/2761399079 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f3f64004020 con 0x56252fd8eb40 2024-09-19T09:17:26.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.838+0000 7f3f78a0f740 1 -- 172.21.15.120:0/2761399079 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f600044b0 con 0x56252fd8eb40 2024-09-19T09:17:26.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.838+0000 7f3f6ffff640 1 --2- 172.21.15.120:0/2761399079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56252fd8d5d0 0x56252fd97ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:17:26.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.838+0000 7f3f4f7fe640 1 -- 172.21.15.120:0/2761399079 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3f64010e50 con 0x56252fd8eb40 2024-09-19T09:17:26.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.838+0000 7f3f78a0f740 10 client.20433 fetch_fsmap learned FSMap version 36 2024-09-19T09:17:26.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.838+0000 7f3f78a0f740 1 -- 172.21.15.120:0/2761399079 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56252fdaf360 con 0x56252fd8eb40 2024-09-19T09:17:26.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.838+0000 7f3f4f7fe640 1 -- 172.21.15.120:0/2761399079 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f600044b0 con 0x56252fd8eb40 2024-09-19T09:17:26.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.838+0000 7f3f4f7fe640 1 -- 172.21.15.120:0/2761399079 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f3f640035f0 con 0x56252fd8eb40 2024-09-19T09:17:26.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.838+0000 7f3f78a0f740 10 client.20433 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:17:26.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.838+0000 7f3f78a0f740 10 client.20433 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:17:26.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.838+0000 7f3f78a0f740 20 client.20433 populate_metadata read hostname 'smithi120' 2024-09-19T09:17:26.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.838+0000 7f3f78a0f740 1 --2- 172.21.15.120:0/2761399079 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56252fdb0880 0x56252fdb2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:26.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.838+0000 7f3f78a0f740 4 client.20433 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:17:26.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.838+0000 7f3f78a0f740 1 -- 172.21.15.120:0/2761399079 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56252fd00860 con 0x56252fdb0880 2024-09-19T09:17:26.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.838+0000 7f3f6ffff640 1 --2- 172.21.15.120:0/2761399079 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56252fdb0880 0x56252fdb2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:26.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.839+0000 7f3f4f7fe640 10 client.20433 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:17:26.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.839+0000 7f3f6ffff640 1 --2- 172.21.15.120:0/2761399079 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56252fdb0880 0x56252fdb2d10 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=61b7fe5c665a4712 server_cookie=38b5c254e6fc690a in_seq=0 out_seq=0 2024-09-19T09:17:26.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.840+0000 7f3f4f7fe640 1 -- 172.21.15.120:0/2761399079 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56252fd00860 con 0x56252fdb0880 2024-09-19T09:17:26.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.840+0000 7f3f4f7fe640 10 client.20433 handle_command_reply: tid=0 2024-09-19T09:17:26.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.840+0000 7f3f78a0f740 1 client.20433 shutdown 2024-09-19T09:17:26.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.841+0000 7f3f78a0f740 1 -- 172.21.15.120:0/2761399079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56252fd8eb40 msgr2=0x56252fd94f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:26.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.841+0000 7f3f78a0f740 1 --2- 172.21.15.120:0/2761399079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56252fd8eb40 0x56252fd94f50 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f3f64010e20 tx=0x7f3f640046c0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.841+0000 7f3f78a0f740 1 -- 172.21.15.120:0/2761399079 shutdown_connections 2024-09-19T09:17:26.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.841+0000 7f3f78a0f740 1 --2- 172.21.15.120:0/2761399079 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56252fdb0880 0x56252fdb2d10 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.841+0000 7f3f78a0f740 1 --2- 172.21.15.120:0/2761399079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56252fd98370 0x56252fd95490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.841+0000 7f3f78a0f740 1 --2- 172.21.15.120:0/2761399079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56252fd8eb40 0x56252fd94f50 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.841+0000 7f3f78a0f740 1 --2- 172.21.15.120:0/2761399079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56252fd8d5d0 0x56252fd97ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.841+0000 7f3f78a0f740 1 -- 172.21.15.120:0/2761399079 >> 172.21.15.120:0/2761399079 conn(0x56252fd56310 msgr2=0x56252fd8f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:26.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.841+0000 7f3f4f7fe640 0 client.20433 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:17:26.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.841+0000 7f3f78a0f740 1 -- 172.21.15.120:0/2761399079 shutdown_connections 2024-09-19T09:17:26.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.841+0000 7f3f78a0f740 1 -- 172.21.15.120:0/2761399079 wait complete. 2024-09-19T09:17:26.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.841+0000 7f3f78a0f740 20 client.20433 trim_cache size 0 max 16384 2024-09-19T09:17:26.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.857+0000 7f3f78a0f740 1 Processor -- start 2024-09-19T09:17:26.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.857+0000 7f3f78a0f740 1 -- start start 2024-09-19T09:17:26.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.857+0000 7f3f78a0f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56252fd56310 0x56252fdacc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:26.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.857+0000 7f3f78a0f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56252fd8d5d0 0x56252fd93f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:26.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.857+0000 7f3f78a0f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56252fd8eb40 0x56252fd90120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:26.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.857+0000 7f3f78a0f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c00e450 con 0x56252fd8d5d0 2024-09-19T09:17:26.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.857+0000 7f3f78a0f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c010020 con 0x56252fd56310 2024-09-19T09:17:26.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.857+0000 7f3f78a0f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c00eb20 con 0x56252fd8eb40 2024-09-19T09:17:26.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.858+0000 7f3f6f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56252fd8d5d0 0x56252fd93f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:26.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.858+0000 7f3f6ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56252fd56310 0x56252fdacc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:26.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.858+0000 7f3f6f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56252fd8d5d0 0x56252fd93f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37150/0 (socket says 172.21.15.120:37150) 2024-09-19T09:17:26.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.858+0000 7f3f6f7fe640 1 -- 172.21.15.120:0/2712768094 learned_addr learned my addr 172.21.15.120:0/2712768094 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:26.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.858+0000 7f3f74bd3640 1 --2- 172.21.15.120:0/2712768094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56252fd8eb40 0x56252fd90120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:26.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.858+0000 7f3f6f7fe640 1 -- 172.21.15.120:0/2712768094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56252fd8eb40 msgr2=0x56252fd90120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:26.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.858+0000 7f3f6f7fe640 1 --2- 172.21.15.120:0/2712768094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56252fd8eb40 0x56252fd90120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.858+0000 7f3f6f7fe640 1 -- 172.21.15.120:0/2712768094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56252fd56310 msgr2=0x56252fdacc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:26.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.858+0000 7f3f6f7fe640 1 --2- 172.21.15.120:0/2712768094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56252fd56310 0x56252fdacc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.858+0000 7f3f6f7fe640 1 -- 172.21.15.120:0/2712768094 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56252fd934a0 con 0x56252fd8d5d0 2024-09-19T09:17:26.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.858+0000 7f3f6f7fe640 1 --2- 172.21.15.120:0/2712768094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56252fd8d5d0 0x56252fd93f30 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f3f64011680 tx=0x7f3f64009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e773b43d1cc5527 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:26.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.858+0000 7f3f55ffb640 1 -- 172.21.15.120:0/2712768094 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f640037f0 con 0x56252fd8d5d0 2024-09-19T09:17:26.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.858+0000 7f3f55ffb640 1 -- 172.21.15.120:0/2712768094 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f6403b030 con 0x56252fd8d5d0 2024-09-19T09:17:26.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.858+0000 7f3f55ffb640 1 -- 172.21.15.120:0/2712768094 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f64004cb0 con 0x56252fd8d5d0 2024-09-19T09:17:26.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.859+0000 7f3f78a0f740 1 -- 172.21.15.120:0/2712768094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56252fd8d5d0 msgr2=0x56252fd93f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:26.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.859+0000 7f3f78a0f740 1 --2- 172.21.15.120:0/2712768094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56252fd8d5d0 0x56252fd93f30 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f3f64011680 tx=0x7f3f64009f40 comp rx=0 tx=0).stop 2024-09-19T09:17:26.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.859+0000 7f3f78a0f740 1 -- 172.21.15.120:0/2712768094 shutdown_connections 2024-09-19T09:17:26.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.859+0000 7f3f78a0f740 1 --2- 172.21.15.120:0/2712768094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56252fd8eb40 0x56252fd90120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.859+0000 7f3f78a0f740 1 --2- 172.21.15.120:0/2712768094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56252fd8d5d0 0x56252fd93f30 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.859+0000 7f3f78a0f740 1 --2- 172.21.15.120:0/2712768094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56252fd56310 0x56252fdacc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.859+0000 7f3f78a0f740 1 -- 172.21.15.120:0/2712768094 >> 172.21.15.120:0/2712768094 conn(0x56252fd955f0 msgr2=0x56252fd93820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:26.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.859+0000 7f3f78a0f740 1 -- 172.21.15.120:0/2712768094 shutdown_connections 2024-09-19T09:17:26.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.859+0000 7f3f78a0f740 1 -- 172.21.15.120:0/2712768094 wait complete. 2024-09-19T09:17:26.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.860+0000 7f3f78a0f740 1 -- start start 2024-09-19T09:17:26.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.860+0000 7f3f78a0f740 1 Processor -- start 2024-09-19T09:17:26.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.860+0000 7f3f78a0f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56252fd56310 0x56252fda8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:26.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.860+0000 7f3f78a0f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56252fd8d5d0 0x56252fda92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:26.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.860+0000 7f3f78a0f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56252fd8eb40 0x56252fd91580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:26.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.860+0000 7f3f78a0f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c00f950 con 0x56252fd8d5d0 2024-09-19T09:17:26.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.860+0000 7f3f78a0f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c00f1f0 con 0x56252fd8eb40 2024-09-19T09:17:26.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.860+0000 7f3f78a0f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c00c410 con 0x56252fd56310 2024-09-19T09:17:26.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.860+0000 7f3f78a0f740 10 client.0 _ll_register_callbacks cb 0x7f3f640115d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:17:26.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.861+0000 7f3f6f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56252fd8d5d0 0x56252fda92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:26.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.861+0000 7f3f6f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56252fd8d5d0 0x56252fda92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37158/0 (socket says 172.21.15.120:37158) 2024-09-19T09:17:26.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.861+0000 7f3f6f7fe640 1 -- 172.21.15.120:0/324748852 learned_addr learned my addr 172.21.15.120:0/324748852 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:26.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.861+0000 7f3f74bd3640 1 --2- 172.21.15.120:0/324748852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56252fd8eb40 0x56252fd91580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:26.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.861+0000 7f3f6ffff640 1 --2- 172.21.15.120:0/324748852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56252fd56310 0x56252fda8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:26.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.861+0000 7f3f6f7fe640 1 -- 172.21.15.120:0/324748852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56252fd56310 msgr2=0x56252fda8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:26.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.861+0000 7f3f6f7fe640 1 --2- 172.21.15.120:0/324748852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56252fd56310 0x56252fda8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.861+0000 7f3f6f7fe640 1 -- 172.21.15.120:0/324748852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56252fd8eb40 msgr2=0x56252fd91580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:26.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.861+0000 7f3f6f7fe640 1 --2- 172.21.15.120:0/324748852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56252fd8eb40 0x56252fd91580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.861+0000 7f3f6f7fe640 1 -- 172.21.15.120:0/324748852 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56252fd91b40 con 0x56252fd8d5d0 2024-09-19T09:17:26.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.861+0000 7f3f74bd3640 1 --2- 172.21.15.120:0/324748852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56252fd8eb40 0x56252fd91580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:17:26.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.861+0000 7f3f6f7fe640 1 --2- 172.21.15.120:0/324748852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56252fd8d5d0 0x56252fda92e0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f3f64003a90 tx=0x7f3f64010d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f30eb178c8d50a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:26.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.861+0000 7f3f4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:17:26.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.861+0000 7f3f4f7fe640 1 -- 172.21.15.120:0/324748852 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f64043020 con 0x56252fd8d5d0 2024-09-19T09:17:26.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.861+0000 7f3f4f7fe640 1 -- 172.21.15.120:0/324748852 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f3f64020830 con 0x56252fd8d5d0 2024-09-19T09:17:26.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.861+0000 7f3f78a0f740 1 -- 172.21.15.120:0/324748852 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f6403b030 con 0x56252fd8d5d0 2024-09-19T09:17:26.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.862+0000 7f3f4f7fe640 1 -- 172.21.15.120:0/324748852 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3f64011760 con 0x56252fd8d5d0 2024-09-19T09:17:26.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.862+0000 7f3f78a0f740 10 client.20439 fetch_fsmap learned FSMap version 36 2024-09-19T09:17:26.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.862+0000 7f3f78a0f740 1 -- 172.21.15.120:0/324748852 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56252fe1ae90 con 0x56252fd8d5d0 2024-09-19T09:17:26.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.862+0000 7f3f4f7fe640 1 -- 172.21.15.120:0/324748852 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f6403b030 con 0x56252fd8d5d0 2024-09-19T09:17:26.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.862+0000 7f3f4f7fe640 1 -- 172.21.15.120:0/324748852 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f3f64004320 con 0x56252fd8d5d0 2024-09-19T09:17:26.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.862+0000 7f3f78a0f740 10 client.20439 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:17:26.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.862+0000 7f3f78a0f740 10 client.20439 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:17:26.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.862+0000 7f3f78a0f740 20 client.20439 populate_metadata read hostname 'smithi120' 2024-09-19T09:17:26.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.862+0000 7f3f78a0f740 1 --2- 172.21.15.120:0/324748852 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56252fd98370 0x56252fe1d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:26.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.862+0000 7f3f78a0f740 4 client.20439 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:17:26.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.862+0000 7f3f78a0f740 1 -- 172.21.15.120:0/324748852 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56252fcfad70 con 0x56252fd98370 2024-09-19T09:17:26.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.863+0000 7f3f6ffff640 1 --2- 172.21.15.120:0/324748852 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56252fd98370 0x56252fe1d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:26.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.863+0000 7f3f4f7fe640 10 client.20439 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:17:26.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.863+0000 7f3f6ffff640 1 --2- 172.21.15.120:0/324748852 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56252fd98370 0x56252fe1d560 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c5fd399bcfb5746 server_cookie=c6f7351083a024d9 in_seq=0 out_seq=0 2024-09-19T09:17:26.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.863+0000 7f3f4f7fe640 1 -- 172.21.15.120:0/324748852 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56252fcfad70 con 0x56252fd98370 2024-09-19T09:17:26.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.864+0000 7f3f4f7fe640 10 client.20439 handle_command_reply: tid=0 2024-09-19T09:17:26.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.864+0000 7f3f78a0f740 1 client.20439 shutdown 2024-09-19T09:17:26.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.864+0000 7f3f78a0f740 1 -- 172.21.15.120:0/324748852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56252fd8d5d0 msgr2=0x56252fda92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:26.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.864+0000 7f3f78a0f740 1 --2- 172.21.15.120:0/324748852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56252fd8d5d0 0x56252fda92e0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f3f64003a90 tx=0x7f3f64010d60 comp rx=0 tx=0).stop 2024-09-19T09:17:26.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.864+0000 7f3f78a0f740 1 -- 172.21.15.120:0/324748852 shutdown_connections 2024-09-19T09:17:26.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.864+0000 7f3f78a0f740 1 --2- 172.21.15.120:0/324748852 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56252fd98370 0x56252fe1d560 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.864+0000 7f3f78a0f740 1 --2- 172.21.15.120:0/324748852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56252fd8eb40 0x56252fd91580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.864+0000 7f3f78a0f740 1 --2- 172.21.15.120:0/324748852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56252fd8d5d0 0x56252fda92e0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.864+0000 7f3f78a0f740 1 --2- 172.21.15.120:0/324748852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56252fd56310 0x56252fda8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.864+0000 7f3f78a0f740 1 -- 172.21.15.120:0/324748852 >> 172.21.15.120:0/324748852 conn(0x56252fd955f0 msgr2=0x56252fd94040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:26.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.864+0000 7f3f4f7fe640 0 client.20439 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:17:26.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.864+0000 7f3f78a0f740 1 -- 172.21.15.120:0/324748852 shutdown_connections 2024-09-19T09:17:26.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.864+0000 7f3f78a0f740 1 -- 172.21.15.120:0/324748852 wait complete. 2024-09-19T09:17:26.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.864+0000 7f3f78a0f740 20 client.20439 trim_cache size 0 max 16384 2024-09-19T09:17:26.870 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:17:26.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.865+0000 7f3f76636640 1 -- 172.21.15.120:0/3201445289 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3f580a3240 msgr2=0x7f3f580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:26.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.865+0000 7f3f76636640 1 --2- 172.21.15.120:0/3201445289 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3f580a3240 0x7f3f580a5700 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x56252fd8e070 tx=0x7f3f64005f90 comp rx=0 tx=0).stop 2024-09-19T09:17:26.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.865+0000 7f3f76636640 1 -- 172.21.15.120:0/3201445289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f70102ef0 msgr2=0x7f3f7019cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:26.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.865+0000 7f3f76636640 1 --2- 172.21.15.120:0/3201445289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f70102ef0 0x7f3f7019cd70 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f3f68009dc0 tx=0x7f3f68003c10 comp rx=0 tx=0).stop 2024-09-19T09:17:26.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.865+0000 7f3f76636640 1 -- 172.21.15.120:0/3201445289 shutdown_connections 2024-09-19T09:17:26.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.865+0000 7f3f76636640 1 --2- 172.21.15.120:0/3201445289 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3f580a3240 0x7f3f580a5700 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.865+0000 7f3f76636640 1 --2- 172.21.15.120:0/3201445289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f70102ef0 0x7f3f7019cd70 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.865+0000 7f3f76636640 1 --2- 172.21.15.120:0/3201445289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f70102570 0x7f3f701989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.865+0000 7f3f76636640 1 --2- 172.21.15.120:0/3201445289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f70100e20 0x7f3f701984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:26.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.865+0000 7f3f76636640 1 -- 172.21.15.120:0/3201445289 >> 172.21.15.120:0/3201445289 conn(0x7f3f700fc7d0 msgr2=0x7f3f70106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:26.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.865+0000 7f3f76636640 1 -- 172.21.15.120:0/3201445289 shutdown_connections 2024-09-19T09:17:26.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:26.865+0000 7f3f76636640 1 -- 172.21.15.120:0/3201445289 wait complete. 2024-09-19T09:17:27.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:26 smithi120 ceph-mon[31150]: pgmap v782: 97 pgs: 97 active+clean; 289 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.9 MiB/s wr, 640 op/s 2024-09-19T09:17:27.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:26 smithi196 ceph-mon[35150]: pgmap v782: 97 pgs: 97 active+clean; 289 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.9 MiB/s wr, 640 op/s 2024-09-19T09:17:27.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:26 smithi153 ceph-mon[35071]: pgmap v782: 97 pgs: 97 active+clean; 289 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.9 MiB/s wr, 640 op/s 2024-09-19T09:17:27.354 INFO:tasks.workunit.client.0.smithi120.stdout: 10 37338 26.78 MB/sec warmup 82 sec latency 242.562 ms 2024-09-19T09:17:27.833 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:17:28.132 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:17:28.355 INFO:tasks.workunit.client.0.smithi120.stdout: 10 37915 26.77 MB/sec warmup 83 sec latency 73.268 ms 2024-09-19T09:17:28.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.675+0000 7f3ef83be640 1 -- 172.21.15.120:0/255237463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ef01082a0 msgr2=0x7f3ef010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:28.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.675+0000 7f3ef83be640 1 --2- 172.21.15.120:0/255237463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ef01082a0 0x7f3ef010ec40 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8009cd0 tx=0x7f3ee802a8e0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.675+0000 7f3ef83be640 1 -- 172.21.15.120:0/255237463 shutdown_connections 2024-09-19T09:17:28.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.675+0000 7f3ef83be640 1 --2- 172.21.15.120:0/255237463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ef01082a0 0x7f3ef010ec40 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.675+0000 7f3ef83be640 1 --2- 172.21.15.120:0/255237463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ef01036e0 0x7f3ef0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.675+0000 7f3ef83be640 1 --2- 172.21.15.120:0/255237463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ef0102d30 0x7f3ef0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.675+0000 7f3ef83be640 1 -- 172.21.15.120:0/255237463 >> 172.21.15.120:0/255237463 conn(0x7f3ef00fc7d0 msgr2=0x7f3ef00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:28.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.675+0000 7f3ef83be640 1 -- 172.21.15.120:0/255237463 shutdown_connections 2024-09-19T09:17:28.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.675+0000 7f3ef83be640 1 -- 172.21.15.120:0/255237463 wait complete. 2024-09-19T09:17:28.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.676+0000 7f3ef83be640 1 Processor -- start 2024-09-19T09:17:28.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.676+0000 7f3ef83be640 1 -- start start 2024-09-19T09:17:28.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.676+0000 7f3ef83be640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ef0102d30 0x7f3ef0100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:28.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.676+0000 7f3ef83be640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ef01036e0 0x7f3ef0100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:28.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.676+0000 7f3ef83be640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ef01082a0 0x7f3ef0101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:28.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.676+0000 7f3ef83be640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef01120a0 con 0x7f3ef01036e0 2024-09-19T09:17:28.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.676+0000 7f3ef83be640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef0111f20 con 0x7f3ef0102d30 2024-09-19T09:17:28.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.676+0000 7f3ef83be640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef0112220 con 0x7f3ef01082a0 2024-09-19T09:17:28.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.676+0000 7f3ef5959640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ef01036e0 0x7f3ef0100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:28.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.676+0000 7f3ef5959640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ef01036e0 0x7f3ef0100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37168/0 (socket says 172.21.15.120:37168) 2024-09-19T09:17:28.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.676+0000 7f3ef5959640 1 -- 172.21.15.120:0/1478180128 learned_addr learned my addr 172.21.15.120:0/1478180128 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:28.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.676+0000 7f3ef695b640 1 --2- 172.21.15.120:0/1478180128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ef01082a0 0x7f3ef0101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:28.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.676+0000 7f3ef615a640 1 --2- 172.21.15.120:0/1478180128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ef0102d30 0x7f3ef0100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:28.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.677+0000 7f3ef615a640 1 -- 172.21.15.120:0/1478180128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ef01082a0 msgr2=0x7f3ef0101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:28.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.677+0000 7f3ef615a640 1 --2- 172.21.15.120:0/1478180128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ef01082a0 0x7f3ef0101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.677+0000 7f3ef615a640 1 -- 172.21.15.120:0/1478180128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ef01036e0 msgr2=0x7f3ef0100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:28.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.677+0000 7f3ef615a640 1 --2- 172.21.15.120:0/1478180128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ef01036e0 0x7f3ef0100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.677+0000 7f3ef615a640 1 -- 172.21.15.120:0/1478180128 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef01a69f0 con 0x7f3ef0102d30 2024-09-19T09:17:28.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.677+0000 7f3ef695b640 1 --2- 172.21.15.120:0/1478180128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ef01082a0 0x7f3ef0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:17:28.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.677+0000 7f3ef615a640 1 --2- 172.21.15.120:0/1478180128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ef0102d30 0x7f3ef0100990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00e9f0 tx=0x7f3eec00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:28.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.677+0000 7f3ee77fe640 1 -- 172.21.15.120:0/1478180128 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec00ccc0 con 0x7f3ef0102d30 2024-09-19T09:17:28.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.677+0000 7f3ef83be640 1 -- 172.21.15.120:0/1478180128 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef01a6ce0 con 0x7f3ef0102d30 2024-09-19T09:17:28.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.677+0000 7f3ee77fe640 1 -- 172.21.15.120:0/1478180128 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3eec005020 con 0x7f3ef0102d30 2024-09-19T09:17:28.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.677+0000 7f3ee77fe640 1 -- 172.21.15.120:0/1478180128 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec0049e0 con 0x7f3ef0102d30 2024-09-19T09:17:28.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.677+0000 7f3ef83be640 1 -- 172.21.15.120:0/1478180128 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef01a71c0 con 0x7f3ef0102d30 2024-09-19T09:17:28.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.680+0000 7f3efa797740 1 Processor -- start 2024-09-19T09:17:28.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.680+0000 7f3ee77fe640 1 -- 172.21.15.120:0/1478180128 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f3eec005560 con 0x7f3ef0102d30 2024-09-19T09:17:28.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.680+0000 7f3efa797740 1 -- start start 2024-09-19T09:17:28.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.680+0000 7f3efa797740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a670988b40 0x55a670987090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:28.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.680+0000 7f3efa797740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6709875d0 0x55a670987a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:28.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.680+0000 7f3efa797740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a670992370 0x55a670994760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:28.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.680+0000 7f3efa797740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a6708ed2f0 con 0x55a670992370 2024-09-19T09:17:28.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.680+0000 7f3efa797740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a670894cf0 con 0x55a670988b40 2024-09-19T09:17:28.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.680+0000 7f3efa797740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a6708e92b0 con 0x55a6709875d0 2024-09-19T09:17:28.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.680+0000 7f3ef615a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6709875d0 0x55a670987a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:28.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.680+0000 7f3ef695b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a670988b40 0x55a670987090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:28.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.680+0000 7f3ef5959640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a670992370 0x55a670994760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:28.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.680+0000 7f3ef695b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a670988b40 0x55a670987090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38152/0 (socket says 172.21.15.120:38152) 2024-09-19T09:17:28.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.680+0000 7f3ef695b640 1 -- 172.21.15.120:0/343337000 learned_addr learned my addr 172.21.15.120:0/343337000 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:28.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.681+0000 7f3ef695b640 1 -- 172.21.15.120:0/343337000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6709875d0 msgr2=0x55a670987a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:28.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.681+0000 7f3ef695b640 1 --2- 172.21.15.120:0/343337000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6709875d0 0x55a670987a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.681+0000 7f3ef695b640 1 -- 172.21.15.120:0/343337000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a670992370 msgr2=0x55a670994760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:28.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.681+0000 7f3ef695b640 1 --2- 172.21.15.120:0/343337000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a670992370 0x55a670994760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.681+0000 7f3ef695b640 1 -- 172.21.15.120:0/343337000 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6708793a0 con 0x55a670988b40 2024-09-19T09:17:28.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.681+0000 7f3ee77fe640 1 --2- 172.21.15.120:0/1478180128 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3edc0a3240 0x7f3edc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:28.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.681+0000 7f3ef5959640 1 --2- 172.21.15.120:0/343337000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a670992370 0x55a670994760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:17:28.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.681+0000 7f3ef615a640 1 --2- 172.21.15.120:0/1478180128 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3edc0a3240 0x7f3edc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:28.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.681+0000 7f3ef695b640 1 --2- 172.21.15.120:0/343337000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a670988b40 0x55a670987090 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8035000 tx=0x7f3ee8009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d51eff1a779fabf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:28.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.681+0000 7f3ee77fe640 1 -- 172.21.15.120:0/1478180128 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f3eec0cf890 con 0x7f3ef0102d30 2024-09-19T09:17:28.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.681+0000 7f3ee4dea640 1 -- 172.21.15.120:0/343337000 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8048070 con 0x55a670988b40 2024-09-19T09:17:28.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.681+0000 7f3ee4dea640 1 -- 172.21.15.120:0/343337000 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ee8036070 con 0x55a670988b40 2024-09-19T09:17:28.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.681+0000 7f3ee4dea640 1 -- 172.21.15.120:0/343337000 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee80046c0 con 0x55a670988b40 2024-09-19T09:17:28.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.682+0000 7f3efa797740 1 -- 172.21.15.120:0/343337000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a670988b40 msgr2=0x55a670987090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:28.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.682+0000 7f3ef615a640 1 --2- 172.21.15.120:0/1478180128 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3edc0a3240 0x7f3edc0a5700 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00c6a0 tx=0x7f3eec0d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:28.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.682+0000 7f3efa797740 1 --2- 172.21.15.120:0/343337000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a670988b40 0x55a670987090 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8035000 tx=0x7f3ee8009a20 comp rx=0 tx=0).stop 2024-09-19T09:17:28.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.682+0000 7f3efa797740 1 -- 172.21.15.120:0/343337000 shutdown_connections 2024-09-19T09:17:28.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.682+0000 7f3efa797740 1 --2- 172.21.15.120:0/343337000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a670992370 0x55a670994760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.682+0000 7f3efa797740 1 --2- 172.21.15.120:0/343337000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6709875d0 0x55a670987a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.682+0000 7f3efa797740 1 --2- 172.21.15.120:0/343337000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a670988b40 0x55a670987090 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.682+0000 7f3efa797740 1 -- 172.21.15.120:0/343337000 >> 172.21.15.120:0/343337000 conn(0x55a670950310 msgr2=0x55a670983c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:28.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.682+0000 7f3efa797740 1 -- 172.21.15.120:0/343337000 shutdown_connections 2024-09-19T09:17:28.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.682+0000 7f3efa797740 1 -- 172.21.15.120:0/343337000 wait complete. 2024-09-19T09:17:28.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.682+0000 7f3efa797740 1 -- start start 2024-09-19T09:17:28.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.683+0000 7f3efa797740 1 Processor -- start 2024-09-19T09:17:28.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.683+0000 7f3efa797740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6709875d0 0x55a670991d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:28.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.683+0000 7f3efa797740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a670988b40 0x55a67098f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:28.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.683+0000 7f3efa797740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a670992370 0x55a67098f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:28.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.683+0000 7f3efa797740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a670995010 con 0x55a670988b40 2024-09-19T09:17:28.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.683+0000 7f3efa797740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a6708a8060 con 0x55a6709875d0 2024-09-19T09:17:28.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.683+0000 7f3efa797740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a670995190 con 0x55a670992370 2024-09-19T09:17:28.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.683+0000 7f3efa797740 10 client.0 _ll_register_callbacks cb 0x55a6705fb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:17:28.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.683+0000 7f3ef695b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6709875d0 0x55a670991d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:28.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.683+0000 7f3ef695b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6709875d0 0x55a670991d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38162/0 (socket says 172.21.15.120:38162) 2024-09-19T09:17:28.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.683+0000 7f3ef695b640 1 -- 172.21.15.120:0/1425305460 learned_addr learned my addr 172.21.15.120:0/1425305460 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:28.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.683+0000 7f3ef5959640 1 --2- 172.21.15.120:0/1425305460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a670988b40 0x55a67098f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:28.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.683+0000 7f3ef695b640 1 --2- 172.21.15.120:0/1425305460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a670992370 0x55a67098f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:28.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.683+0000 7f3ef695b640 1 -- 172.21.15.120:0/1425305460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a670992370 msgr2=0x55a67098f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:28.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.683+0000 7f3ef695b640 1 --2- 172.21.15.120:0/1425305460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a670992370 0x55a67098f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.683+0000 7f3ef695b640 1 -- 172.21.15.120:0/1425305460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a670988b40 msgr2=0x55a67098f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:28.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.683+0000 7f3ef695b640 1 --2- 172.21.15.120:0/1425305460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a670988b40 0x55a67098f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.683+0000 7f3ef695b640 1 -- 172.21.15.120:0/1425305460 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a67065e3c0 con 0x55a6709875d0 2024-09-19T09:17:28.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.684+0000 7f3ef695b640 1 --2- 172.21.15.120:0/1425305460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6709875d0 0x55a670991d50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8009620 tx=0x7f3ee8004300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ab2227fff272eb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:28.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.684+0000 7f3ed9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:17:28.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.684+0000 7f3ed9ffb640 1 -- 172.21.15.120:0/1425305460 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8056060 con 0x55a6709875d0 2024-09-19T09:17:28.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.684+0000 7f3ed9ffb640 1 -- 172.21.15.120:0/1425305460 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f3ee8003970 con 0x55a6709875d0 2024-09-19T09:17:28.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.684+0000 7f3efa797740 1 -- 172.21.15.120:0/1425305460 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ee8036070 con 0x55a6709875d0 2024-09-19T09:17:28.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.684+0000 7f3ed9ffb640 1 -- 172.21.15.120:0/1425305460 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3ee8048040 con 0x55a6709875d0 2024-09-19T09:17:28.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.684+0000 7f3efa797740 10 client.29402 fetch_fsmap learned FSMap version 36 2024-09-19T09:17:28.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.684+0000 7f3efa797740 1 -- 172.21.15.120:0/1425305460 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a670995010 con 0x55a6709875d0 2024-09-19T09:17:28.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.684+0000 7f3ed9ffb640 1 -- 172.21.15.120:0/1425305460 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ee8036070 con 0x55a6709875d0 2024-09-19T09:17:28.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.684+0000 7f3ed9ffb640 1 -- 172.21.15.120:0/1425305460 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f3ee804e070 con 0x55a6709875d0 2024-09-19T09:17:28.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.685+0000 7f3efa797740 10 client.29402 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:17:28.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.685+0000 7f3efa797740 10 client.29402 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:17:28.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.685+0000 7f3efa797740 20 client.29402 populate_metadata read hostname 'smithi120' 2024-09-19T09:17:28.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.685+0000 7f3efa797740 1 --2- 172.21.15.120:0/1425305460 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55a6709a1c70 0x55a6709a4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:28.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.685+0000 7f3efa797740 4 client.29402 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:17:28.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.685+0000 7f3efa797740 1 -- 172.21.15.120:0/1425305460 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6708fa860 con 0x55a6709a1c70 2024-09-19T09:17:28.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.685+0000 7f3ef5959640 1 --2- 172.21.15.120:0/1425305460 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55a6709a1c70 0x55a6709a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:28.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.685+0000 7f3ef5959640 1 --2- 172.21.15.120:0/1425305460 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55a6709a1c70 0x55a6709a4060 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6d434a810495449c server_cookie=93827b9f68fcbd75 in_seq=0 out_seq=0 2024-09-19T09:17:28.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.685+0000 7f3ed9ffb640 10 client.29402 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:17:28.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.686+0000 7f3ed9ffb640 1 -- 172.21.15.120:0/1425305460 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a6708fa860 con 0x55a6709a1c70 2024-09-19T09:17:28.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.686+0000 7f3ed9ffb640 10 client.29402 handle_command_reply: tid=0 2024-09-19T09:17:28.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.686+0000 7f3efa797740 1 client.29402 shutdown 2024-09-19T09:17:28.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.686+0000 7f3efa797740 1 -- 172.21.15.120:0/1425305460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6709875d0 msgr2=0x55a670991d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:28.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.686+0000 7f3efa797740 1 --2- 172.21.15.120:0/1425305460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6709875d0 0x55a670991d50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8009620 tx=0x7f3ee8004300 comp rx=0 tx=0).stop 2024-09-19T09:17:28.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.686+0000 7f3efa797740 1 -- 172.21.15.120:0/1425305460 shutdown_connections 2024-09-19T09:17:28.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.686+0000 7f3efa797740 1 --2- 172.21.15.120:0/1425305460 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55a6709a1c70 0x55a6709a4060 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.686+0000 7f3efa797740 1 --2- 172.21.15.120:0/1425305460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a670992370 0x55a67098f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.686+0000 7f3efa797740 1 --2- 172.21.15.120:0/1425305460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a670988b40 0x55a67098f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.686+0000 7f3efa797740 1 --2- 172.21.15.120:0/1425305460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6709875d0 0x55a670991d50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.686+0000 7f3efa797740 1 -- 172.21.15.120:0/1425305460 >> 172.21.15.120:0/1425305460 conn(0x55a670950310 msgr2=0x55a670989960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:28.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.686+0000 7f3ed9ffb640 0 client.29402 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:17:28.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.686+0000 7f3efa797740 1 -- 172.21.15.120:0/1425305460 shutdown_connections 2024-09-19T09:17:28.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.686+0000 7f3efa797740 1 -- 172.21.15.120:0/1425305460 wait complete. 2024-09-19T09:17:28.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.687+0000 7f3efa797740 20 client.29402 trim_cache size 0 max 16384 2024-09-19T09:17:28.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.702+0000 7f3efa797740 1 Processor -- start 2024-09-19T09:17:28.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.702+0000 7f3efa797740 1 -- start start 2024-09-19T09:17:28.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.702+0000 7f3efa797740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6709875d0 0x55a670983d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:28.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.702+0000 7f3efa797740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a670988b40 0x55a67099e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:28.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.702+0000 7f3efa797740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a670992370 0x55a67098a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:28.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.702+0000 7f3efa797740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef01af7a0 con 0x55a670988b40 2024-09-19T09:17:28.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.702+0000 7f3efa797740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef01b1370 con 0x55a670992370 2024-09-19T09:17:28.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.702+0000 7f3efa797740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef01afe70 con 0x55a6709875d0 2024-09-19T09:17:28.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.703+0000 7f3ef5959640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a670988b40 0x55a67099e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:28.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.703+0000 7f3ef5959640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a670988b40 0x55a67099e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37200/0 (socket says 172.21.15.120:37200) 2024-09-19T09:17:28.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.703+0000 7f3ef5959640 1 -- 172.21.15.120:0/3235765781 learned_addr learned my addr 172.21.15.120:0/3235765781 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:28.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.703+0000 7f3ef695b640 1 --2- 172.21.15.120:0/3235765781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a670992370 0x55a67098a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:28.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.703+0000 7f3ef695b640 1 --2- 172.21.15.120:0/3235765781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6709875d0 0x55a670983d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:28.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.703+0000 7f3ef695b640 1 -- 172.21.15.120:0/3235765781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6709875d0 msgr2=0x55a670983d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:28.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.703+0000 7f3ef695b640 1 --2- 172.21.15.120:0/3235765781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6709875d0 0x55a670983d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.703+0000 7f3ef695b640 1 -- 172.21.15.120:0/3235765781 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a670988b40 msgr2=0x55a67099e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:28.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.703+0000 7f3ef695b640 1 --2- 172.21.15.120:0/3235765781 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a670988b40 0x55a67099e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.703+0000 7f3ef695b640 1 -- 172.21.15.120:0/3235765781 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a67097ee90 con 0x55a670992370 2024-09-19T09:17:28.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.703+0000 7f3ef5959640 1 --2- 172.21.15.120:0/3235765781 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a670988b40 0x55a67099e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:17:28.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.703+0000 7f3ef695b640 1 --2- 172.21.15.120:0/3235765781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a670992370 0x55a67098a120 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8002230 tx=0x7f3ee8009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=824ee26608a9f4b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:28.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.703+0000 7f3edbfff640 1 -- 172.21.15.120:0/3235765781 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8056070 con 0x55a670992370 2024-09-19T09:17:28.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.703+0000 7f3edbfff640 1 -- 172.21.15.120:0/3235765781 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ee8036040 con 0x55a670992370 2024-09-19T09:17:28.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.703+0000 7f3edbfff640 1 -- 172.21.15.120:0/3235765781 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8031b50 con 0x55a670992370 2024-09-19T09:17:28.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.704+0000 7f3efa797740 1 -- 172.21.15.120:0/3235765781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a670992370 msgr2=0x55a67098a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:28.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.704+0000 7f3efa797740 1 --2- 172.21.15.120:0/3235765781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a670992370 0x55a67098a120 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8002230 tx=0x7f3ee8009a20 comp rx=0 tx=0).stop 2024-09-19T09:17:28.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.704+0000 7f3efa797740 1 -- 172.21.15.120:0/3235765781 shutdown_connections 2024-09-19T09:17:28.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.704+0000 7f3efa797740 1 --2- 172.21.15.120:0/3235765781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a670992370 0x55a67098a120 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.704+0000 7f3efa797740 1 --2- 172.21.15.120:0/3235765781 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a670988b40 0x55a67099e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.704+0000 7f3efa797740 1 --2- 172.21.15.120:0/3235765781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6709875d0 0x55a670983d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.704+0000 7f3efa797740 1 -- 172.21.15.120:0/3235765781 >> 172.21.15.120:0/3235765781 conn(0x55a670950310 msgr2=0x55a67098e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:28.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.704+0000 7f3efa797740 1 -- 172.21.15.120:0/3235765781 shutdown_connections 2024-09-19T09:17:28.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.704+0000 7f3efa797740 1 -- 172.21.15.120:0/3235765781 wait complete. 2024-09-19T09:17:28.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.705+0000 7f3efa797740 1 -- start start 2024-09-19T09:17:28.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.705+0000 7f3efa797740 1 Processor -- start 2024-09-19T09:17:28.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.705+0000 7f3efa797740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6709875d0 0x55a6709955e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:28.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.705+0000 7f3efa797740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a670988b40 0x55a670995b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:28.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.705+0000 7f3efa797740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a670992370 0x55a670a0a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:28.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.705+0000 7f3efa797740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef01b0ca0 con 0x55a6709875d0 2024-09-19T09:17:28.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.705+0000 7f3efa797740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef01b0540 con 0x55a670992370 2024-09-19T09:17:28.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.705+0000 7f3efa797740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef01ad780 con 0x55a670988b40 2024-09-19T09:17:28.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.705+0000 7f3efa797740 10 client.0 _ll_register_callbacks cb 0x7f3ee8046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:17:28.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.705+0000 7f3ef695b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6709875d0 0x55a6709955e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:28.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.705+0000 7f3ef695b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6709875d0 0x55a6709955e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37208/0 (socket says 172.21.15.120:37208) 2024-09-19T09:17:28.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.705+0000 7f3ef695b640 1 -- 172.21.15.120:0/4174955407 learned_addr learned my addr 172.21.15.120:0/4174955407 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:28.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.705+0000 7f3ef695b640 1 --2- 172.21.15.120:0/4174955407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a670992370 0x55a670a0a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:28.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.705+0000 7f3ef5959640 1 --2- 172.21.15.120:0/4174955407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a670988b40 0x55a670995b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:28.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.706+0000 7f3ef695b640 1 -- 172.21.15.120:0/4174955407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a670988b40 msgr2=0x55a670995b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:28.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.706+0000 7f3ef695b640 1 --2- 172.21.15.120:0/4174955407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a670988b40 0x55a670995b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.706+0000 7f3ef695b640 1 -- 172.21.15.120:0/4174955407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a670992370 msgr2=0x55a670a0a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:28.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.706+0000 7f3ef695b640 1 --2- 172.21.15.120:0/4174955407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a670992370 0x55a670a0a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.706+0000 7f3ef695b640 1 -- 172.21.15.120:0/4174955407 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a670a0abc0 con 0x55a6709875d0 2024-09-19T09:17:28.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.706+0000 7f3ef695b640 1 --2- 172.21.15.120:0/4174955407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6709875d0 0x55a6709955e0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8004d50 tx=0x7f3ee8004c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaaad05f3e0394b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:28.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.706+0000 7f3ed9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:17:28.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.706+0000 7f3ed9ffb640 1 -- 172.21.15.120:0/4174955407 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8056070 con 0x55a6709875d0 2024-09-19T09:17:28.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.706+0000 7f3efa797740 1 -- 172.21.15.120:0/4174955407 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ee8036040 con 0x55a6709875d0 2024-09-19T09:17:28.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.706+0000 7f3ed9ffb640 1 -- 172.21.15.120:0/4174955407 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f3ee8002030 con 0x55a6709875d0 2024-09-19T09:17:28.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.706+0000 7f3ed9ffb640 1 -- 172.21.15.120:0/4174955407 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3ee8060040 con 0x55a6709875d0 2024-09-19T09:17:28.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.706+0000 7f3efa797740 10 client.20457 fetch_fsmap learned FSMap version 36 2024-09-19T09:17:28.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.706+0000 7f3efa797740 1 -- 172.21.15.120:0/4174955407 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a670a0bd80 con 0x55a6709875d0 2024-09-19T09:17:28.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.707+0000 7f3ed9ffb640 1 -- 172.21.15.120:0/4174955407 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ee8036040 con 0x55a6709875d0 2024-09-19T09:17:28.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.707+0000 7f3ed9ffb640 1 -- 172.21.15.120:0/4174955407 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f3ee8066040 con 0x55a6709875d0 2024-09-19T09:17:28.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.707+0000 7f3efa797740 10 client.20457 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:17:28.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.707+0000 7f3efa797740 10 client.20457 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:17:28.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.707+0000 7f3efa797740 20 client.20457 populate_metadata read hostname 'smithi120' 2024-09-19T09:17:28.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.707+0000 7f3efa797740 1 --2- 172.21.15.120:0/4174955407 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55a670a0c330 0x55a670a0e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:28.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.707+0000 7f3efa797740 4 client.20457 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:17:28.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.707+0000 7f3efa797740 1 -- 172.21.15.120:0/4174955407 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a6708f4d70 con 0x55a670a0c330 2024-09-19T09:17:28.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.707+0000 7f3ef5959640 1 --2- 172.21.15.120:0/4174955407 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55a670a0c330 0x55a670a0e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:28.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.708+0000 7f3ef5959640 1 --2- 172.21.15.120:0/4174955407 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55a670a0c330 0x55a670a0e720 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a555f10dc5344d2f server_cookie=25a6d9a34399fc09 in_seq=0 out_seq=0 2024-09-19T09:17:28.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.708+0000 7f3ed9ffb640 10 client.20457 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:17:28.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.708+0000 7f3ed9ffb640 1 -- 172.21.15.120:0/4174955407 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a6708f4d70 con 0x55a670a0c330 2024-09-19T09:17:28.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.708+0000 7f3ed9ffb640 10 client.20457 handle_command_reply: tid=0 2024-09-19T09:17:28.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.708+0000 7f3efa797740 1 client.20457 shutdown 2024-09-19T09:17:28.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.708+0000 7f3efa797740 1 -- 172.21.15.120:0/4174955407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6709875d0 msgr2=0x55a6709955e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:28.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.708+0000 7f3efa797740 1 --2- 172.21.15.120:0/4174955407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6709875d0 0x55a6709955e0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8004d50 tx=0x7f3ee8004c50 comp rx=0 tx=0).stop 2024-09-19T09:17:28.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.708+0000 7f3efa797740 1 -- 172.21.15.120:0/4174955407 shutdown_connections 2024-09-19T09:17:28.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.708+0000 7f3efa797740 1 --2- 172.21.15.120:0/4174955407 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55a670a0c330 0x55a670a0e720 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.708+0000 7f3efa797740 1 --2- 172.21.15.120:0/4174955407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a670992370 0x55a670a0a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.708+0000 7f3efa797740 1 --2- 172.21.15.120:0/4174955407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a670988b40 0x55a670995b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.708+0000 7f3ed9ffb640 0 client.20457 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:17:28.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.708+0000 7f3efa797740 1 --2- 172.21.15.120:0/4174955407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6709875d0 0x55a6709955e0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.708+0000 7f3efa797740 1 -- 172.21.15.120:0/4174955407 >> 172.21.15.120:0/4174955407 conn(0x55a670950310 msgr2=0x55a6709824e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:28.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.708+0000 7f3efa797740 1 -- 172.21.15.120:0/4174955407 shutdown_connections 2024-09-19T09:17:28.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.708+0000 7f3efa797740 1 -- 172.21.15.120:0/4174955407 wait complete. 2024-09-19T09:17:28.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.708+0000 7f3efa797740 20 client.20457 trim_cache size 0 max 16384 2024-09-19T09:17:28.715 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:17:28.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.709+0000 7f3ef83be640 1 -- 172.21.15.120:0/1478180128 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3edc0a3240 msgr2=0x7f3edc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:28.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.709+0000 7f3ef83be640 1 --2- 172.21.15.120:0/1478180128 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3edc0a3240 0x7f3edc0a5700 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00c6a0 tx=0x7f3eec0d5530 comp rx=0 tx=0).stop 2024-09-19T09:17:28.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.709+0000 7f3ef83be640 1 -- 172.21.15.120:0/1478180128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ef0102d30 msgr2=0x7f3ef0100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:28.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.709+0000 7f3ef83be640 1 --2- 172.21.15.120:0/1478180128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ef0102d30 0x7f3ef0100990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00e9f0 tx=0x7f3eec00eeb0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.709+0000 7f3ef83be640 1 -- 172.21.15.120:0/1478180128 shutdown_connections 2024-09-19T09:17:28.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.709+0000 7f3ef83be640 1 --2- 172.21.15.120:0/1478180128 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3edc0a3240 0x7f3edc0a5700 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.709+0000 7f3ef83be640 1 --2- 172.21.15.120:0/1478180128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ef01082a0 0x7f3ef0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.709+0000 7f3ef83be640 1 --2- 172.21.15.120:0/1478180128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ef01036e0 0x7f3ef0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.709+0000 7f3ef83be640 1 --2- 172.21.15.120:0/1478180128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ef0102d30 0x7f3ef0100990 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:28.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.709+0000 7f3ef83be640 1 -- 172.21.15.120:0/1478180128 >> 172.21.15.120:0/1478180128 conn(0x7f3ef00fc7d0 msgr2=0x7f3ef0105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:28.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.709+0000 7f3ef83be640 1 -- 172.21.15.120:0/1478180128 shutdown_connections 2024-09-19T09:17:28.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:28.709+0000 7f3ef83be640 1 -- 172.21.15.120:0/1478180128 wait complete. 2024-09-19T09:17:29.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:28 smithi196 ceph-mon[35150]: pgmap v783: 97 pgs: 97 active+clean; 286 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 9.5 MiB/s wr, 785 op/s 2024-09-19T09:17:29.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:28 smithi153 ceph-mon[35071]: pgmap v783: 97 pgs: 97 active+clean; 286 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 9.5 MiB/s wr, 785 op/s 2024-09-19T09:17:29.296 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:28 smithi120 ceph-mon[31150]: pgmap v783: 97 pgs: 97 active+clean; 286 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 9.5 MiB/s wr, 785 op/s 2024-09-19T09:17:29.355 INFO:tasks.workunit.client.0.smithi120.stdout: 10 38341 26.72 MB/sec warmup 84 sec latency 108.064 ms 2024-09-19T09:17:29.557 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:17:29.856 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:17:30.355 INFO:tasks.workunit.client.0.smithi120.stdout: 10 38913 26.71 MB/sec warmup 85 sec latency 112.179 ms 2024-09-19T09:17:30.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.360+0000 7f0ff247c640 1 -- 172.21.15.120:0/139586445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fec102570 msgr2=0x7f0fec1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:30.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.360+0000 7f0ff247c640 1 --2- 172.21.15.120:0/139586445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fec102570 0x7f0fec1029b0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0009900 tx=0x7f0fe002a480 comp rx=0 tx=0).stop 2024-09-19T09:17:30.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.360+0000 7f0ff247c640 1 -- 172.21.15.120:0/139586445 shutdown_connections 2024-09-19T09:17:30.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.360+0000 7f0ff247c640 1 --2- 172.21.15.120:0/139586445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fec102ef0 0x7f0fec10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.360+0000 7f0ff247c640 1 --2- 172.21.15.120:0/139586445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fec102570 0x7f0fec1029b0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.360+0000 7f0ff247c640 1 --2- 172.21.15.120:0/139586445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fec100e20 0x7f0fec101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.360+0000 7f0ff247c640 1 -- 172.21.15.120:0/139586445 >> 172.21.15.120:0/139586445 conn(0x7f0fec0fc7d0 msgr2=0x7f0fec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:30.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.360+0000 7f0ff247c640 1 -- 172.21.15.120:0/139586445 shutdown_connections 2024-09-19T09:17:30.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.360+0000 7f0ff247c640 1 -- 172.21.15.120:0/139586445 wait complete. 2024-09-19T09:17:30.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.361+0000 7f0ff247c640 1 Processor -- start 2024-09-19T09:17:30.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.361+0000 7f0ff247c640 1 -- start start 2024-09-19T09:17:30.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.361+0000 7f0ff247c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fec100e20 0x7f0fec199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:30.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.361+0000 7f0ff247c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fec102570 0x7f0fec1998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:30.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.362+0000 7f0ff247c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fec102ef0 0x7f0fec1a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:30.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.362+0000 7f0ff247c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0fec110f80 con 0x7f0fec102570 2024-09-19T09:17:30.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.362+0000 7f0ff247c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0fec110e00 con 0x7f0fec100e20 2024-09-19T09:17:30.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.362+0000 7f0ff247c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0fec111100 con 0x7f0fec102ef0 2024-09-19T09:17:30.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.362+0000 7f0feb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fec102570 0x7f0fec1998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:30.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.362+0000 7f0feb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fec102570 0x7f0fec1998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37220/0 (socket says 172.21.15.120:37220) 2024-09-19T09:17:30.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.362+0000 7f0feb7fe640 1 -- 172.21.15.120:0/1625347951 learned_addr learned my addr 172.21.15.120:0/1625347951 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:30.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.362+0000 7f0ff0a19640 1 --2- 172.21.15.120:0/1625347951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fec102ef0 0x7f0fec1a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:30.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.362+0000 7f0febfff640 1 --2- 172.21.15.120:0/1625347951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fec100e20 0x7f0fec199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:30.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.362+0000 7f0ff0a19640 1 -- 172.21.15.120:0/1625347951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fec100e20 msgr2=0x7f0fec199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:30.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.362+0000 7f0ff0a19640 1 --2- 172.21.15.120:0/1625347951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fec100e20 0x7f0fec199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.362+0000 7f0ff0a19640 1 -- 172.21.15.120:0/1625347951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fec102570 msgr2=0x7f0fec1998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:30.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.362+0000 7f0ff0a19640 1 --2- 172.21.15.120:0/1625347951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fec102570 0x7f0fec1998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.362+0000 7f0ff0a19640 1 -- 172.21.15.120:0/1625347951 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fec1a4950 con 0x7f0fec102ef0 2024-09-19T09:17:30.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.362+0000 7f0feb7fe640 1 --2- 172.21.15.120:0/1625347951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fec102570 0x7f0fec1998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:17:30.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.362+0000 7f0ff0a19640 1 --2- 172.21.15.120:0/1625347951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fec102ef0 0x7f0fec1a4410 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009fd0 tx=0x7f0fe400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:30.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.362+0000 7f0fe97fa640 1 -- 172.21.15.120:0/1625347951 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe400ce20 con 0x7f0fec102ef0 2024-09-19T09:17:30.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.362+0000 7f0ff247c640 1 -- 172.21.15.120:0/1625347951 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fec1a4b20 con 0x7f0fec102ef0 2024-09-19T09:17:30.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.362+0000 7f0fe97fa640 1 -- 172.21.15.120:0/1625347951 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fe4005020 con 0x7f0fec102ef0 2024-09-19T09:17:30.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.362+0000 7f0fe97fa640 1 -- 172.21.15.120:0/1625347951 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4010720 con 0x7f0fec102ef0 2024-09-19T09:17:30.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.363+0000 7f0ff247c640 1 -- 172.21.15.120:0/1625347951 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fec1a4fe0 con 0x7f0fec102ef0 2024-09-19T09:17:30.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.365+0000 7f0fe97fa640 1 -- 172.21.15.120:0/1625347951 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f0fe4005540 con 0x7f0fec102ef0 2024-09-19T09:17:30.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.366+0000 7f0ff4855740 1 Processor -- start 2024-09-19T09:17:30.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.366+0000 7f0ff4855740 1 -- start start 2024-09-19T09:17:30.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.366+0000 7f0fe97fa640 1 --2- 172.21.15.120:0/1625347951 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0fd40a3240 0x7f0fd40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:30.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.366+0000 7f0ff4855740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b104e09b40 0x55b104e08090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:30.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.366+0000 7f0ff4855740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b104e085d0 0x55b104e08a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:30.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.366+0000 7f0ff4855740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b104e17780 0x55b104e19b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:30.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.366+0000 7f0ff4855740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b104d6e2f0 con 0x55b104e09b40 2024-09-19T09:17:30.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.366+0000 7f0ff4855740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b104d15cf0 con 0x55b104e17780 2024-09-19T09:17:30.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.366+0000 7f0ff4855740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b104d6a2b0 con 0x55b104e085d0 2024-09-19T09:17:30.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.366+0000 7f0feb7fe640 1 --2- 172.21.15.120:0/1625347951 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0fd40a3240 0x7f0fd40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:30.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.366+0000 7f0ff0a19640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b104e17780 0x55b104e19b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:30.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.366+0000 7f0febfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b104e09b40 0x55b104e08090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:30.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.366+0000 7f0fe97fa640 1 -- 172.21.15.120:0/1625347951 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f0fe40cff90 con 0x7f0fec102ef0 2024-09-19T09:17:30.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.366+0000 7f0ff0a19640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b104e17780 0x55b104e19b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38196/0 (socket says 172.21.15.120:38196) 2024-09-19T09:17:30.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.366+0000 7f0ff0a19640 1 -- 172.21.15.120:0/3525532533 learned_addr learned my addr 172.21.15.120:0/3525532533 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:30.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.366+0000 7f0feb7fe640 1 --2- 172.21.15.120:0/3525532533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b104e085d0 0x55b104e08a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:30.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.366+0000 7f0ff0a19640 1 -- 172.21.15.120:0/3525532533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b104e085d0 msgr2=0x55b104e08a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:30.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.366+0000 7f0ff0a19640 1 --2- 172.21.15.120:0/3525532533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b104e085d0 0x55b104e08a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.367+0000 7f0ff0a19640 1 -- 172.21.15.120:0/3525532533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b104e09b40 msgr2=0x55b104e08090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:30.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.367+0000 7f0ff0a19640 1 --2- 172.21.15.120:0/3525532533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b104e09b40 0x55b104e08090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.367+0000 7f0ff0a19640 1 -- 172.21.15.120:0/3525532533 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b104cfa3a0 con 0x55b104e17780 2024-09-19T09:17:30.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.367+0000 7f0febfff640 1 --2- 172.21.15.120:0/3525532533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b104e09b40 0x55b104e08090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:17:30.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.367+0000 7f0feb7fe640 1 --2- 172.21.15.120:0/1625347951 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0fd40a3240 0x7f0fd40a5700 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0fec06aac0 tx=0x7f0fe00043f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:30.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.367+0000 7f0ff0a19640 1 --2- 172.21.15.120:0/3525532533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b104e17780 0x55b104e19b70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0fe40a4cf0 tx=0x7f0fe409bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4982945811a93b70 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:30.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.367+0000 7f0fd27fc640 1 -- 172.21.15.120:0/3525532533 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe40a0960 con 0x55b104e17780 2024-09-19T09:17:30.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.367+0000 7f0fd27fc640 1 -- 172.21.15.120:0/3525532533 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fe40a5220 con 0x55b104e17780 2024-09-19T09:17:30.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.367+0000 7f0fd27fc640 1 -- 172.21.15.120:0/3525532533 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4076b30 con 0x55b104e17780 2024-09-19T09:17:30.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.368+0000 7f0ff4855740 1 -- 172.21.15.120:0/3525532533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b104e17780 msgr2=0x55b104e19b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:30.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.368+0000 7f0ff4855740 1 --2- 172.21.15.120:0/3525532533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b104e17780 0x55b104e19b70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0fe40a4cf0 tx=0x7f0fe409bf10 comp rx=0 tx=0).stop 2024-09-19T09:17:30.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.368+0000 7f0ff4855740 1 -- 172.21.15.120:0/3525532533 shutdown_connections 2024-09-19T09:17:30.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.368+0000 7f0ff4855740 1 --2- 172.21.15.120:0/3525532533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b104e17780 0x55b104e19b70 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.368+0000 7f0ff4855740 1 --2- 172.21.15.120:0/3525532533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b104e085d0 0x55b104e08a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.368+0000 7f0ff4855740 1 --2- 172.21.15.120:0/3525532533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b104e09b40 0x55b104e08090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.368+0000 7f0ff4855740 1 -- 172.21.15.120:0/3525532533 >> 172.21.15.120:0/3525532533 conn(0x55b104dd1310 msgr2=0x55b104e04c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:30.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.368+0000 7f0ff4855740 1 -- 172.21.15.120:0/3525532533 shutdown_connections 2024-09-19T09:17:30.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.368+0000 7f0ff4855740 1 -- 172.21.15.120:0/3525532533 wait complete. 2024-09-19T09:17:30.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.368+0000 7f0ff4855740 1 -- start start 2024-09-19T09:17:30.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.369+0000 7f0ff4855740 1 Processor -- start 2024-09-19T09:17:30.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.369+0000 7f0ff4855740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b104e085d0 0x55b104e171b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:30.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.369+0000 7f0ff4855740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b104e09b40 0x55b104e14460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:30.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.369+0000 7f0ff4855740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b104e17780 0x55b104e14bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:30.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.369+0000 7f0ff4855740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b104e1a480 con 0x55b104e17780 2024-09-19T09:17:30.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.369+0000 7f0ff4855740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b104d29060 con 0x55b104e09b40 2024-09-19T09:17:30.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.369+0000 7f0ff4855740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b104e1a600 con 0x55b104e085d0 2024-09-19T09:17:30.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.369+0000 7f0ff4855740 10 client.0 _ll_register_callbacks cb 0x55b104a7c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:17:30.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.369+0000 7f0febfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b104e085d0 0x55b104e171b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:30.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.369+0000 7f0ff0a19640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b104e17780 0x55b104e14bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:30.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.369+0000 7f0ff0a19640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b104e17780 0x55b104e14bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37234/0 (socket says 172.21.15.120:37234) 2024-09-19T09:17:30.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.369+0000 7f0ff0a19640 1 -- 172.21.15.120:0/418497473 learned_addr learned my addr 172.21.15.120:0/418497473 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:30.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.369+0000 7f0feb7fe640 1 --2- 172.21.15.120:0/418497473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b104e09b40 0x55b104e14460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:30.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.370+0000 7f0ff0a19640 1 -- 172.21.15.120:0/418497473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b104e085d0 msgr2=0x55b104e171b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:30.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.370+0000 7f0ff0a19640 1 --2- 172.21.15.120:0/418497473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b104e085d0 0x55b104e171b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.370+0000 7f0ff0a19640 1 -- 172.21.15.120:0/418497473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b104e09b40 msgr2=0x55b104e14460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:30.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.370+0000 7f0ff0a19640 1 --2- 172.21.15.120:0/418497473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b104e09b40 0x55b104e14460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.370+0000 7f0ff0a19640 1 -- 172.21.15.120:0/418497473 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b104adf3c0 con 0x55b104e17780 2024-09-19T09:17:30.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.370+0000 7f0feb7fe640 1 --2- 172.21.15.120:0/418497473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b104e09b40 0x55b104e14460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:17:30.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.370+0000 7f0ff0a19640 1 --2- 172.21.15.120:0/418497473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b104e17780 0x55b104e14bf0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f0fe401a040 tx=0x7f0fe40a09e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e70784aba1f91d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:30.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.370+0000 7f0fcb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:17:30.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.370+0000 7f0fcb7fe640 1 -- 172.21.15.120:0/418497473 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4074ba0 con 0x55b104e17780 2024-09-19T09:17:30.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.370+0000 7f0fcb7fe640 1 -- 172.21.15.120:0/418497473 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f0fe4074d40 con 0x55b104e17780 2024-09-19T09:17:30.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.370+0000 7f0ff4855740 1 -- 172.21.15.120:0/418497473 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fe40a5220 con 0x55b104e17780 2024-09-19T09:17:30.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.371+0000 7f0fcb7fe640 1 -- 172.21.15.120:0/418497473 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0fe4076a40 con 0x55b104e17780 2024-09-19T09:17:30.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.371+0000 7f0ff4855740 10 client.20469 fetch_fsmap learned FSMap version 36 2024-09-19T09:17:30.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.371+0000 7f0ff4855740 1 -- 172.21.15.120:0/418497473 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b104e25f90 con 0x55b104e17780 2024-09-19T09:17:30.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.371+0000 7f0fcb7fe640 1 -- 172.21.15.120:0/418497473 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fe4076be0 con 0x55b104e17780 2024-09-19T09:17:30.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.371+0000 7f0fcb7fe640 1 -- 172.21.15.120:0/418497473 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f0fe409b470 con 0x55b104e17780 2024-09-19T09:17:30.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.371+0000 7f0ff4855740 10 client.20469 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:17:30.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.371+0000 7f0ff4855740 10 client.20469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:17:30.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.371+0000 7f0ff4855740 20 client.20469 populate_metadata read hostname 'smithi120' 2024-09-19T09:17:30.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.371+0000 7f0ff4855740 1 --2- 172.21.15.120:0/418497473 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b104e272e0 0x55b104e296d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:30.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.371+0000 7f0ff4855740 4 client.20469 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:17:30.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.371+0000 7f0ff4855740 1 -- 172.21.15.120:0/418497473 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0fe40a5220 con 0x55b104e272e0 2024-09-19T09:17:30.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.371+0000 7f0febfff640 1 --2- 172.21.15.120:0/418497473 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b104e272e0 0x55b104e296d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:30.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.372+0000 7f0febfff640 1 --2- 172.21.15.120:0/418497473 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b104e272e0 0x55b104e296d0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5a3ff734d226756 server_cookie=fe061c198e294438 in_seq=0 out_seq=0 2024-09-19T09:17:30.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.372+0000 7f0fcb7fe640 10 client.20469 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:30.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.376+0000 7f0fcb7fe640 1 -- 172.21.15.120:0/418497473 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f0fe40a5220 con 0x55b104e272e0 2024-09-19T09:17:30.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.376+0000 7f0fcb7fe640 10 client.20469 handle_command_reply: tid=0 2024-09-19T09:17:30.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.376+0000 7f0ff4855740 1 client.20469 shutdown 2024-09-19T09:17:30.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.377+0000 7f0ff4855740 1 -- 172.21.15.120:0/418497473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b104e17780 msgr2=0x55b104e14bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:30.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.377+0000 7f0ff4855740 1 --2- 172.21.15.120:0/418497473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b104e17780 0x55b104e14bf0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f0fe401a040 tx=0x7f0fe40a09e0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.377+0000 7f0ff4855740 1 -- 172.21.15.120:0/418497473 shutdown_connections 2024-09-19T09:17:30.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.377+0000 7f0ff4855740 1 --2- 172.21.15.120:0/418497473 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b104e272e0 0x55b104e296d0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.377+0000 7f0ff4855740 1 --2- 172.21.15.120:0/418497473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b104e17780 0x55b104e14bf0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.377+0000 7f0ff4855740 1 --2- 172.21.15.120:0/418497473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b104e09b40 0x55b104e14460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.377+0000 7f0ff4855740 1 --2- 172.21.15.120:0/418497473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b104e085d0 0x55b104e171b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.377+0000 7f0fcb7fe640 0 client.20469 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:30.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.377+0000 7f0ff4855740 1 -- 172.21.15.120:0/418497473 >> 172.21.15.120:0/418497473 conn(0x55b104dd1310 msgr2=0x55b104e0a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:30.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.377+0000 7f0ff4855740 1 -- 172.21.15.120:0/418497473 shutdown_connections 2024-09-19T09:17:30.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.377+0000 7f0ff4855740 1 -- 172.21.15.120:0/418497473 wait complete. 2024-09-19T09:17:30.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.377+0000 7f0ff4855740 20 client.20469 trim_cache size 0 max 16384 2024-09-19T09:17:30.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.398+0000 7f0ff4855740 1 Processor -- start 2024-09-19T09:17:30.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.398+0000 7f0ff4855740 1 -- start start 2024-09-19T09:17:30.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.398+0000 7f0ff4855740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b104e085d0 0x55b104e2c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:30.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.398+0000 7f0ff4855740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b104e09b40 0x55b104e2cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:30.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.399+0000 7f0ff4855740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b104e17780 0x55b104e116c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:30.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.399+0000 7f0ff4855740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0fec1ab3e0 con 0x55b104e17780 2024-09-19T09:17:30.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.399+0000 7f0ff4855740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0fec1acfb0 con 0x55b104e085d0 2024-09-19T09:17:30.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.399+0000 7f0ff4855740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0fec1abab0 con 0x55b104e09b40 2024-09-19T09:17:30.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.399+0000 7f0ff0a19640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b104e17780 0x55b104e116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:30.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.399+0000 7f0ff0a19640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b104e17780 0x55b104e116c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37240/0 (socket says 172.21.15.120:37240) 2024-09-19T09:17:30.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.399+0000 7f0ff0a19640 1 -- 172.21.15.120:0/2985733029 learned_addr learned my addr 172.21.15.120:0/2985733029 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:30.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.399+0000 7f0febfff640 1 --2- 172.21.15.120:0/2985733029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b104e085d0 0x55b104e2c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:30.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.399+0000 7f0feb7fe640 1 --2- 172.21.15.120:0/2985733029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b104e09b40 0x55b104e2cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:30.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.399+0000 7f0ff0a19640 1 -- 172.21.15.120:0/2985733029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b104e09b40 msgr2=0x55b104e2cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:30.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.399+0000 7f0ff0a19640 1 --2- 172.21.15.120:0/2985733029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b104e09b40 0x55b104e2cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.399+0000 7f0ff0a19640 1 -- 172.21.15.120:0/2985733029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b104e085d0 msgr2=0x55b104e2c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:30.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.399+0000 7f0ff0a19640 1 --2- 172.21.15.120:0/2985733029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b104e085d0 0x55b104e2c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.399+0000 7f0ff0a19640 1 -- 172.21.15.120:0/2985733029 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b104e08be0 con 0x55b104e17780 2024-09-19T09:17:30.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.399+0000 7f0ff0a19640 1 --2- 172.21.15.120:0/2985733029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b104e17780 0x55b104e116c0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f0fe40054f0 tx=0x7f0fe409c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27f9265a0cdaa3f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:30.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.399+0000 7f0fd1ffb640 1 -- 172.21.15.120:0/2985733029 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe40a5220 con 0x55b104e17780 2024-09-19T09:17:30.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.399+0000 7f0fd1ffb640 1 -- 172.21.15.120:0/2985733029 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fe4076be0 con 0x55b104e17780 2024-09-19T09:17:30.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.399+0000 7f0fd1ffb640 1 -- 172.21.15.120:0/2985733029 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe409bc60 con 0x55b104e17780 2024-09-19T09:17:30.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.400+0000 7f0ff4855740 1 -- 172.21.15.120:0/2985733029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b104e17780 msgr2=0x55b104e116c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:30.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.400+0000 7f0ff4855740 1 --2- 172.21.15.120:0/2985733029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b104e17780 0x55b104e116c0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f0fe40054f0 tx=0x7f0fe409c730 comp rx=0 tx=0).stop 2024-09-19T09:17:30.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.400+0000 7f0ff4855740 1 -- 172.21.15.120:0/2985733029 shutdown_connections 2024-09-19T09:17:30.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.400+0000 7f0ff4855740 1 --2- 172.21.15.120:0/2985733029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b104e17780 0x55b104e116c0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.400+0000 7f0ff4855740 1 --2- 172.21.15.120:0/2985733029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b104e09b40 0x55b104e2cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.400+0000 7f0ff4855740 1 --2- 172.21.15.120:0/2985733029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b104e085d0 0x55b104e2c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.400+0000 7f0ff4855740 1 -- 172.21.15.120:0/2985733029 >> 172.21.15.120:0/2985733029 conn(0x55b104dd1310 msgr2=0x55b104e12d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:30.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.400+0000 7f0ff4855740 1 -- 172.21.15.120:0/2985733029 shutdown_connections 2024-09-19T09:17:30.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.400+0000 7f0ff4855740 1 -- 172.21.15.120:0/2985733029 wait complete. 2024-09-19T09:17:30.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.401+0000 7f0ff4855740 1 -- start start 2024-09-19T09:17:30.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.401+0000 7f0ff4855740 1 Processor -- start 2024-09-19T09:17:30.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.401+0000 7f0ff4855740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b104e085d0 0x55b104e2a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:30.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.401+0000 7f0ff4855740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b104e09b40 0x55b104e2abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:30.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.401+0000 7f0ff4855740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b104e17780 0x55b104e2b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:30.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.401+0000 7f0ff4855740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0fec1ac8e0 con 0x55b104e09b40 2024-09-19T09:17:30.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.401+0000 7f0ff4855740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0fec1ac180 con 0x55b104e17780 2024-09-19T09:17:30.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.401+0000 7f0ff4855740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0fec1a93a0 con 0x55b104e085d0 2024-09-19T09:17:30.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.401+0000 7f0ff4855740 10 client.0 _ll_register_callbacks cb 0x7f0fe40cdf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:17:30.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.402+0000 7f0feb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b104e09b40 0x55b104e2abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:30.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.402+0000 7f0febfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b104e085d0 0x55b104e2a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:30.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.402+0000 7f0febfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b104e085d0 0x55b104e2a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:52016/0 (socket says 172.21.15.120:52016) 2024-09-19T09:17:30.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.402+0000 7f0febfff640 1 -- 172.21.15.120:0/3982909666 learned_addr learned my addr 172.21.15.120:0/3982909666 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:30.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.402+0000 7f0feb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b104e09b40 0x55b104e2abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37252/0 (socket says 172.21.15.120:37252) 2024-09-19T09:17:30.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.402+0000 7f0ff0a19640 1 --2- 172.21.15.120:0/3982909666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b104e17780 0x55b104e2b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:30.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.402+0000 7f0febfff640 1 -- 172.21.15.120:0/3982909666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b104e17780 msgr2=0x55b104e2b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:30.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.402+0000 7f0febfff640 1 --2- 172.21.15.120:0/3982909666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b104e17780 0x55b104e2b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.402+0000 7f0febfff640 1 -- 172.21.15.120:0/3982909666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b104e09b40 msgr2=0x55b104e2abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:30.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.402+0000 7f0febfff640 1 --2- 172.21.15.120:0/3982909666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b104e09b40 0x55b104e2abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.402+0000 7f0febfff640 1 -- 172.21.15.120:0/3982909666 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b104e90e20 con 0x55b104e085d0 2024-09-19T09:17:30.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.402+0000 7f0ff0a19640 1 --2- 172.21.15.120:0/3982909666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b104e17780 0x55b104e2b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:17:30.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.402+0000 7f0febfff640 1 --2- 172.21.15.120:0/3982909666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b104e085d0 0x55b104e2a680 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x55b104e1a5d0 tx=0x7f0fdc010a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e4c9748f38e76d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:30.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.402+0000 7f0fcb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:17:30.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.402+0000 7f0fcb7fe640 1 -- 172.21.15.120:0/3982909666 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc022070 con 0x55b104e085d0 2024-09-19T09:17:30.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.402+0000 7f0fcb7fe640 1 -- 172.21.15.120:0/3982909666 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f0fdc004220 con 0x55b104e085d0 2024-09-19T09:17:30.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.402+0000 7f0ff4855740 1 -- 172.21.15.120:0/3982909666 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fe4076be0 con 0x55b104e085d0 2024-09-19T09:17:30.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.403+0000 7f0fcb7fe640 1 -- 172.21.15.120:0/3982909666 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0fdc004420 con 0x55b104e085d0 2024-09-19T09:17:30.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.403+0000 7f0ff4855740 10 client.29254 fetch_fsmap learned FSMap version 36 2024-09-19T09:17:30.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.403+0000 7f0ff4855740 1 -- 172.21.15.120:0/3982909666 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b104e92100 con 0x55b104e085d0 2024-09-19T09:17:30.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.403+0000 7f0fcb7fe640 1 -- 172.21.15.120:0/3982909666 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fe4076be0 con 0x55b104e085d0 2024-09-19T09:17:30.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.403+0000 7f0fcb7fe640 1 -- 172.21.15.120:0/3982909666 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f0fdc026410 con 0x55b104e085d0 2024-09-19T09:17:30.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.403+0000 7f0ff4855740 10 client.29254 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:17:30.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.403+0000 7f0ff4855740 10 client.29254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:17:30.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.403+0000 7f0ff4855740 20 client.29254 populate_metadata read hostname 'smithi120' 2024-09-19T09:17:30.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.403+0000 7f0ff4855740 1 --2- 172.21.15.120:0/3982909666 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b104e928b0 0x55b104e94cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:30.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.403+0000 7f0ff4855740 4 client.29254 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:17:30.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.403+0000 7f0ff4855740 1 -- 172.21.15.120:0/3982909666 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b104d7b860 con 0x55b104e928b0 2024-09-19T09:17:30.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.403+0000 7f0feb7fe640 1 --2- 172.21.15.120:0/3982909666 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b104e928b0 0x55b104e94cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:30.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.404+0000 7f0fcb7fe640 10 client.29254 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:30.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.404+0000 7f0feb7fe640 1 --2- 172.21.15.120:0/3982909666 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b104e928b0 0x55b104e94cd0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d0d6eda459c3a1d server_cookie=9888998afe10eb2f in_seq=0 out_seq=0 2024-09-19T09:17:30.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.405+0000 7f0fcb7fe640 1 -- 172.21.15.120:0/3982909666 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b104d7b860 con 0x55b104e928b0 2024-09-19T09:17:30.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.405+0000 7f0fcb7fe640 10 client.29254 handle_command_reply: tid=0 2024-09-19T09:17:30.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.405+0000 7f0ff4855740 1 client.29254 shutdown 2024-09-19T09:17:30.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.405+0000 7f0ff4855740 1 -- 172.21.15.120:0/3982909666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b104e085d0 msgr2=0x55b104e2a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:30.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.405+0000 7f0ff4855740 1 --2- 172.21.15.120:0/3982909666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b104e085d0 0x55b104e2a680 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x55b104e1a5d0 tx=0x7f0fdc010a20 comp rx=0 tx=0).stop 2024-09-19T09:17:30.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.405+0000 7f0ff4855740 1 -- 172.21.15.120:0/3982909666 shutdown_connections 2024-09-19T09:17:30.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.405+0000 7f0ff4855740 1 --2- 172.21.15.120:0/3982909666 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b104e928b0 0x55b104e94cd0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.405+0000 7f0ff4855740 1 --2- 172.21.15.120:0/3982909666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b104e17780 0x55b104e2b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.405+0000 7f0ff4855740 1 --2- 172.21.15.120:0/3982909666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b104e09b40 0x55b104e2abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.405+0000 7f0ff4855740 1 --2- 172.21.15.120:0/3982909666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b104e085d0 0x55b104e2a680 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.405+0000 7f0ff4855740 1 -- 172.21.15.120:0/3982909666 >> 172.21.15.120:0/3982909666 conn(0x55b104dd1310 msgr2=0x55b104e164e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:30.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.405+0000 7f0fcb7fe640 0 client.29254 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:30.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.405+0000 7f0ff4855740 1 -- 172.21.15.120:0/3982909666 shutdown_connections 2024-09-19T09:17:30.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.405+0000 7f0ff4855740 1 -- 172.21.15.120:0/3982909666 wait complete. 2024-09-19T09:17:30.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.405+0000 7f0ff4855740 20 client.29254 trim_cache size 0 max 16384 2024-09-19T09:17:30.418 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:17:30.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.406+0000 7f0ff247c640 1 -- 172.21.15.120:0/1625347951 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0fd40a3240 msgr2=0x7f0fd40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:30.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.406+0000 7f0ff247c640 1 --2- 172.21.15.120:0/1625347951 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0fd40a3240 0x7f0fd40a5700 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0fec06aac0 tx=0x7f0fe00043f0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.406+0000 7f0ff247c640 1 -- 172.21.15.120:0/1625347951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fec102ef0 msgr2=0x7f0fec1a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:30.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.406+0000 7f0ff247c640 1 --2- 172.21.15.120:0/1625347951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fec102ef0 0x7f0fec1a4410 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009fd0 tx=0x7f0fe400ef10 comp rx=0 tx=0).stop 2024-09-19T09:17:30.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.406+0000 7f0ff247c640 1 -- 172.21.15.120:0/1625347951 shutdown_connections 2024-09-19T09:17:30.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.406+0000 7f0ff247c640 1 --2- 172.21.15.120:0/1625347951 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0fd40a3240 0x7f0fd40a5700 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.406+0000 7f0ff247c640 1 --2- 172.21.15.120:0/1625347951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fec102ef0 0x7f0fec1a4410 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.406+0000 7f0ff247c640 1 --2- 172.21.15.120:0/1625347951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fec102570 0x7f0fec1998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.406+0000 7f0ff247c640 1 --2- 172.21.15.120:0/1625347951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fec100e20 0x7f0fec199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:30.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.406+0000 7f0ff247c640 1 -- 172.21.15.120:0/1625347951 >> 172.21.15.120:0/1625347951 conn(0x7f0fec0fc7d0 msgr2=0x7f0fec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:30.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.406+0000 7f0ff247c640 1 -- 172.21.15.120:0/1625347951 shutdown_connections 2024-09-19T09:17:30.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:30.406+0000 7f0ff247c640 1 -- 172.21.15.120:0/1625347951 wait complete. 2024-09-19T09:17:31.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:30 smithi120 ceph-mon[31150]: pgmap v784: 97 pgs: 97 active+clean; 291 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.5 MiB/s wr, 785 op/s 2024-09-19T09:17:31.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:30 smithi153 ceph-mon[35071]: pgmap v784: 97 pgs: 97 active+clean; 291 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.5 MiB/s wr, 785 op/s 2024-09-19T09:17:31.360 INFO:tasks.workunit.client.0.smithi120.stdout: 10 39559 26.81 MB/sec warmup 86 sec latency 41.778 ms 2024-09-19T09:17:31.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:30 smithi196 ceph-mon[35150]: pgmap v784: 97 pgs: 97 active+clean; 291 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.5 MiB/s wr, 785 op/s 2024-09-19T09:17:32.229 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:17:32.229 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:17:32.355 INFO:tasks.workunit.client.0.smithi120.stdout: 10 40236 26.83 MB/sec warmup 87 sec latency 217.078 ms 2024-09-19T09:17:32.493 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:17:33.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.012+0000 7efd59f13640 1 -- 172.21.15.120:0/2210929557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd54102570 msgr2=0x7efd541029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:33.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.012+0000 7efd59f13640 1 --2- 172.21.15.120:0/2210929557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd54102570 0x7efd541029b0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7efd48009900 tx=0x7efd4802a480 comp rx=0 tx=0).stop 2024-09-19T09:17:33.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.012+0000 7efd59f13640 1 -- 172.21.15.120:0/2210929557 shutdown_connections 2024-09-19T09:17:33.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.012+0000 7efd59f13640 1 --2- 172.21.15.120:0/2210929557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd54102ef0 0x7efd5410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.012+0000 7efd59f13640 1 --2- 172.21.15.120:0/2210929557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd54102570 0x7efd541029b0 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.012+0000 7efd59f13640 1 --2- 172.21.15.120:0/2210929557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd54100e20 0x7efd54101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.013+0000 7efd59f13640 1 -- 172.21.15.120:0/2210929557 >> 172.21.15.120:0/2210929557 conn(0x7efd540fc7d0 msgr2=0x7efd540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:33.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.013+0000 7efd59f13640 1 -- 172.21.15.120:0/2210929557 shutdown_connections 2024-09-19T09:17:33.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.013+0000 7efd59f13640 1 -- 172.21.15.120:0/2210929557 wait complete. 2024-09-19T09:17:33.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.013+0000 7efd59f13640 1 Processor -- start 2024-09-19T09:17:33.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.013+0000 7efd59f13640 1 -- start start 2024-09-19T09:17:33.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.014+0000 7efd59f13640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd54100e20 0x7efd54199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:33.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.014+0000 7efd59f13640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd54102570 0x7efd54199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:33.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.014+0000 7efd59f13640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd54102ef0 0x7efd541a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:33.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.014+0000 7efd59f13640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efd54110f80 con 0x7efd54102ef0 2024-09-19T09:17:33.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.014+0000 7efd59f13640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efd54110e00 con 0x7efd54100e20 2024-09-19T09:17:33.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.014+0000 7efd59f13640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efd54111100 con 0x7efd54102570 2024-09-19T09:17:33.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.014+0000 7efd52ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd54102570 0x7efd54199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:33.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.014+0000 7efd537fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd54100e20 0x7efd54199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:33.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.014+0000 7efd53fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd54102ef0 0x7efd541a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:33.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.014+0000 7efd52ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd54102570 0x7efd54199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:39896/0 (socket says 172.21.15.120:39896) 2024-09-19T09:17:33.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.014+0000 7efd537fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd54100e20 0x7efd54199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38928/0 (socket says 172.21.15.120:38928) 2024-09-19T09:17:33.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.014+0000 7efd52ffd640 1 -- 172.21.15.120:0/1624134244 learned_addr learned my addr 172.21.15.120:0/1624134244 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:33.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.014+0000 7efd53fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd54102ef0 0x7efd541a43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51356/0 (socket says 172.21.15.120:51356) 2024-09-19T09:17:33.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.014+0000 7efd53fff640 1 -- 172.21.15.120:0/1624134244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd54102570 msgr2=0x7efd54199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:33.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.014+0000 7efd53fff640 1 --2- 172.21.15.120:0/1624134244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd54102570 0x7efd54199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.014+0000 7efd53fff640 1 -- 172.21.15.120:0/1624134244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd54100e20 msgr2=0x7efd54199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:33.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.014+0000 7efd53fff640 1 --2- 172.21.15.120:0/1624134244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd54100e20 0x7efd54199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.014+0000 7efd53fff640 1 -- 172.21.15.120:0/1624134244 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd541a4910 con 0x7efd54102ef0 2024-09-19T09:17:33.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.014+0000 7efd537fe640 1 --2- 172.21.15.120:0/1624134244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd54100e20 0x7efd54199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:17:33.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.014+0000 7efd53fff640 1 --2- 172.21.15.120:0/1624134244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd54102ef0 0x7efd541a43d0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7efd4c009fd0 tx=0x7efd4c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:33.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.014+0000 7efd50ff9640 1 -- 172.21.15.120:0/1624134244 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c00ce20 con 0x7efd54102ef0 2024-09-19T09:17:33.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.014+0000 7efd59f13640 1 -- 172.21.15.120:0/1624134244 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd541a4ae0 con 0x7efd54102ef0 2024-09-19T09:17:33.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.014+0000 7efd50ff9640 1 -- 172.21.15.120:0/1624134244 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd4c005020 con 0x7efd54102ef0 2024-09-19T09:17:33.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.014+0000 7efd50ff9640 1 -- 172.21.15.120:0/1624134244 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c010720 con 0x7efd54102ef0 2024-09-19T09:17:33.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.015+0000 7efd59f13640 1 -- 172.21.15.120:0/1624134244 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd541a4ff0 con 0x7efd54102ef0 2024-09-19T09:17:33.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.018+0000 7efd50ff9640 1 -- 172.21.15.120:0/1624134244 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7efd4c005540 con 0x7efd54102ef0 2024-09-19T09:17:33.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.018+0000 7efd5c2ec740 1 Processor -- start 2024-09-19T09:17:33.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.018+0000 7efd5c2ec740 1 -- start start 2024-09-19T09:17:33.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.018+0000 7efd5c2ec740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d7d919a20 0x556d7d919e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:33.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.018+0000 7efd5c2ec740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d7d91a340 0x556d7d928c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:33.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.018+0000 7efd5c2ec740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d7d929150 0x556d7d92b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:33.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.018+0000 7efd5c2ec740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556d7d87f280 con 0x556d7d919a20 2024-09-19T09:17:33.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.018+0000 7efd5c2ec740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556d7d826b10 con 0x556d7d91a340 2024-09-19T09:17:33.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.018+0000 7efd5c2ec740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556d7d87b240 con 0x556d7d929150 2024-09-19T09:17:33.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.018+0000 7efd537fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d7d91a340 0x556d7d928c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:33.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.018+0000 7efd537fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d7d91a340 0x556d7d928c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38936/0 (socket says 172.21.15.120:38936) 2024-09-19T09:17:33.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.018+0000 7efd537fe640 1 -- 172.21.15.120:0/3762262262 learned_addr learned my addr 172.21.15.120:0/3762262262 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:33.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.018+0000 7efd52ffd640 1 --2- 172.21.15.120:0/3762262262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d7d929150 0x556d7d92b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:33.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.018+0000 7efd50ff9640 1 --2- 172.21.15.120:0/1624134244 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efd3c0a3240 0x7efd3c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:33.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.018+0000 7efd52ffd640 1 --2- 172.21.15.120:0/3762262262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d7d919a20 0x556d7d919e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:33.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.018+0000 7efd537fe640 1 -- 172.21.15.120:0/3762262262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d7d929150 msgr2=0x556d7d92b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:33.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.018+0000 7efd537fe640 1 --2- 172.21.15.120:0/3762262262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d7d929150 0x556d7d92b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.019+0000 7efd537fe640 1 -- 172.21.15.120:0/3762262262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d7d919a20 msgr2=0x556d7d919e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:33.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.019+0000 7efd53fff640 1 --2- 172.21.15.120:0/1624134244 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efd3c0a3240 0x7efd3c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:33.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.019+0000 7efd537fe640 1 --2- 172.21.15.120:0/3762262262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d7d919a20 0x556d7d919e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.019+0000 7efd537fe640 1 -- 172.21.15.120:0/3762262262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d7d8331f0 con 0x556d7d91a340 2024-09-19T09:17:33.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.019+0000 7efd52ffd640 1 --2- 172.21.15.120:0/3762262262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d7d919a20 0x556d7d919e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:17:33.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.019+0000 7efd50ff9640 1 -- 172.21.15.120:0/1624134244 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7efd4c0cff80 con 0x7efd54102ef0 2024-09-19T09:17:33.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.019+0000 7efd537fe640 1 --2- 172.21.15.120:0/3762262262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d7d91a340 0x556d7d928c10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7efd440115c0 tx=0x7efd44011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58f4bcadf34cf624 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:33.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.019+0000 7efd53fff640 1 --2- 172.21.15.120:0/1624134244 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efd3c0a3240 0x7efd3c0a5700 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7efd4c0a4ce0 tx=0x7efd4c067480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:33.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.019+0000 7efd39ffb640 1 -- 172.21.15.120:0/3762262262 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4400f640 con 0x556d7d91a340 2024-09-19T09:17:33.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.019+0000 7efd39ffb640 1 -- 172.21.15.120:0/3762262262 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd4400fcc0 con 0x556d7d91a340 2024-09-19T09:17:33.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.019+0000 7efd39ffb640 1 -- 172.21.15.120:0/3762262262 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd44019d40 con 0x556d7d91a340 2024-09-19T09:17:33.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.020+0000 7efd5c2ec740 1 -- 172.21.15.120:0/3762262262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d7d91a340 msgr2=0x556d7d928c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:33.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.020+0000 7efd5c2ec740 1 --2- 172.21.15.120:0/3762262262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d7d91a340 0x556d7d928c10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7efd440115c0 tx=0x7efd44011a80 comp rx=0 tx=0).stop 2024-09-19T09:17:33.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.020+0000 7efd5c2ec740 1 -- 172.21.15.120:0/3762262262 shutdown_connections 2024-09-19T09:17:33.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.020+0000 7efd5c2ec740 1 --2- 172.21.15.120:0/3762262262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d7d929150 0x556d7d92b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.020+0000 7efd5c2ec740 1 --2- 172.21.15.120:0/3762262262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d7d91a340 0x556d7d928c10 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.020+0000 7efd5c2ec740 1 --2- 172.21.15.120:0/3762262262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d7d919a20 0x556d7d919e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.020+0000 7efd5c2ec740 1 -- 172.21.15.120:0/3762262262 >> 172.21.15.120:0/3762262262 conn(0x556d7d9160f0 msgr2=0x556d7d9164d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:33.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.020+0000 7efd5c2ec740 1 -- 172.21.15.120:0/3762262262 shutdown_connections 2024-09-19T09:17:33.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.020+0000 7efd5c2ec740 1 -- 172.21.15.120:0/3762262262 wait complete. 2024-09-19T09:17:33.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.020+0000 7efd5c2ec740 1 -- start start 2024-09-19T09:17:33.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.021+0000 7efd5c2ec740 1 Processor -- start 2024-09-19T09:17:33.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.021+0000 7efd5c2ec740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d7d919a20 0x556d7d926020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:33.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.021+0000 7efd5c2ec740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d7d91a340 0x556d7d926560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:33.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.021+0000 7efd5c2ec740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d7d929150 0x556d7d926aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:33.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.021+0000 7efd5c2ec740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556d7d92bd20 con 0x556d7d919a20 2024-09-19T09:17:33.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.021+0000 7efd5c2ec740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556d7d839f10 con 0x556d7d929150 2024-09-19T09:17:33.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.021+0000 7efd5c2ec740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556d7d92bea0 con 0x556d7d91a340 2024-09-19T09:17:33.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.021+0000 7efd5c2ec740 10 client.0 _ll_register_callbacks cb 0x556d7d802a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:17:33.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.021+0000 7efd52ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d7d919a20 0x556d7d926020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:33.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.021+0000 7efd52ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d7d919a20 0x556d7d926020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51368/0 (socket says 172.21.15.120:51368) 2024-09-19T09:17:33.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.021+0000 7efd52ffd640 1 -- 172.21.15.120:0/4047409981 learned_addr learned my addr 172.21.15.120:0/4047409981 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:33.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.021+0000 7efd537fe640 1 --2- 172.21.15.120:0/4047409981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d7d91a340 0x556d7d926560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:33.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.021+0000 7efd52ffd640 1 --2- 172.21.15.120:0/4047409981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d7d929150 0x556d7d926aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:33.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.021+0000 7efd52ffd640 1 -- 172.21.15.120:0/4047409981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d7d91a340 msgr2=0x556d7d926560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:33.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.021+0000 7efd52ffd640 1 --2- 172.21.15.120:0/4047409981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d7d91a340 0x556d7d926560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.021+0000 7efd52ffd640 1 -- 172.21.15.120:0/4047409981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d7d929150 msgr2=0x556d7d926aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:33.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.021+0000 7efd52ffd640 1 --2- 172.21.15.120:0/4047409981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d7d929150 0x556d7d926aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.021+0000 7efd52ffd640 1 -- 172.21.15.120:0/4047409981 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d7d5f06a0 con 0x556d7d919a20 2024-09-19T09:17:33.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.021+0000 7efd52ffd640 1 --2- 172.21.15.120:0/4047409981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d7d919a20 0x556d7d926020 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7efd48000c00 tx=0x7efd480056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487a81f323ea3a2f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:33.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.021+0000 7efd32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:17:33.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.022+0000 7efd32ffd640 1 -- 172.21.15.120:0/4047409981 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd48035070 con 0x556d7d919a20 2024-09-19T09:17:33.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.022+0000 7efd5c2ec740 1 -- 172.21.15.120:0/4047409981 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd4400fcc0 con 0x556d7d919a20 2024-09-19T09:17:33.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.022+0000 7efd32ffd640 1 -- 172.21.15.120:0/4047409981 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7efd48003cb0 con 0x556d7d919a20 2024-09-19T09:17:33.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.022+0000 7efd32ffd640 1 -- 172.21.15.120:0/4047409981 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7efd48031540 con 0x556d7d919a20 2024-09-19T09:17:33.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.022+0000 7efd5c2ec740 10 client.20487 fetch_fsmap learned FSMap version 36 2024-09-19T09:17:33.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.022+0000 7efd5c2ec740 1 -- 172.21.15.120:0/4047409981 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556d7d937960 con 0x556d7d919a20 2024-09-19T09:17:33.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.022+0000 7efd32ffd640 1 -- 172.21.15.120:0/4047409981 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd4400fcc0 con 0x556d7d919a20 2024-09-19T09:17:33.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.022+0000 7efd32ffd640 1 -- 172.21.15.120:0/4047409981 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7efd4803d690 con 0x556d7d919a20 2024-09-19T09:17:33.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.022+0000 7efd5c2ec740 10 client.20487 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:17:33.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.022+0000 7efd5c2ec740 10 client.20487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:17:33.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.022+0000 7efd5c2ec740 20 client.20487 populate_metadata read hostname 'smithi120' 2024-09-19T09:17:33.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.022+0000 7efd5c2ec740 1 --2- 172.21.15.120:0/4047409981 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556d7d938e10 0x556d7d93b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:33.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.022+0000 7efd5c2ec740 4 client.20487 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:17:33.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.022+0000 7efd5c2ec740 1 -- 172.21.15.120:0/4047409981 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d7d88c7f0 con 0x556d7d938e10 2024-09-19T09:17:33.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.022+0000 7efd537fe640 1 --2- 172.21.15.120:0/4047409981 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556d7d938e10 0x556d7d93b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:33.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.023+0000 7efd32ffd640 10 client.20487 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:33.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.023+0000 7efd537fe640 1 --2- 172.21.15.120:0/4047409981 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556d7d938e10 0x556d7d93b240 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8349022dcf3a3bdc server_cookie=bb14bc3630511858 in_seq=0 out_seq=0 2024-09-19T09:17:33.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.026+0000 7efd32ffd640 1 -- 172.21.15.120:0/4047409981 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556d7d88c7f0 con 0x556d7d938e10 2024-09-19T09:17:33.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.026+0000 7efd32ffd640 10 client.20487 handle_command_reply: tid=0 2024-09-19T09:17:33.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.026+0000 7efd5c2ec740 1 client.20487 shutdown 2024-09-19T09:17:33.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.027+0000 7efd5c2ec740 1 -- 172.21.15.120:0/4047409981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d7d919a20 msgr2=0x556d7d926020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:33.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.027+0000 7efd5c2ec740 1 --2- 172.21.15.120:0/4047409981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d7d919a20 0x556d7d926020 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7efd48000c00 tx=0x7efd480056d0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.027+0000 7efd5c2ec740 1 -- 172.21.15.120:0/4047409981 shutdown_connections 2024-09-19T09:17:33.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.027+0000 7efd5c2ec740 1 --2- 172.21.15.120:0/4047409981 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556d7d938e10 0x556d7d93b240 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.027+0000 7efd5c2ec740 1 --2- 172.21.15.120:0/4047409981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d7d929150 0x556d7d926aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.027+0000 7efd5c2ec740 1 --2- 172.21.15.120:0/4047409981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d7d91a340 0x556d7d926560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.027+0000 7efd5c2ec740 1 --2- 172.21.15.120:0/4047409981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d7d919a20 0x556d7d926020 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.027+0000 7efd5c2ec740 1 -- 172.21.15.120:0/4047409981 >> 172.21.15.120:0/4047409981 conn(0x556d7d9160f0 msgr2=0x556d7d91be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:33.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.027+0000 7efd32ffd640 0 client.20487 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:33.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.027+0000 7efd5c2ec740 1 -- 172.21.15.120:0/4047409981 shutdown_connections 2024-09-19T09:17:33.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.027+0000 7efd5c2ec740 1 -- 172.21.15.120:0/4047409981 wait complete. 2024-09-19T09:17:33.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.027+0000 7efd5c2ec740 20 client.20487 trim_cache size 0 max 16384 2024-09-19T09:17:33.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.046+0000 7efd5c2ec740 1 Processor -- start 2024-09-19T09:17:33.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.046+0000 7efd5c2ec740 1 -- start start 2024-09-19T09:17:33.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.046+0000 7efd5c2ec740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d7d919a20 0x556d7d921790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:33.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.046+0000 7efd5c2ec740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d7d91a340 0x556d7d925900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:33.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.046+0000 7efd5c2ec740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d7d929150 0x556d7d915160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:33.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.047+0000 7efd5c2ec740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efd541ab310 con 0x556d7d929150 2024-09-19T09:17:33.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.047+0000 7efd5c2ec740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efd541acee0 con 0x556d7d91a340 2024-09-19T09:17:33.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.047+0000 7efd5c2ec740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efd541ab9e0 con 0x556d7d919a20 2024-09-19T09:17:33.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.047+0000 7efd52ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d7d929150 0x556d7d915160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:33.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.047+0000 7efd52ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d7d929150 0x556d7d915160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51372/0 (socket says 172.21.15.120:51372) 2024-09-19T09:17:33.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.047+0000 7efd52ffd640 1 -- 172.21.15.120:0/3038299562 learned_addr learned my addr 172.21.15.120:0/3038299562 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:33.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.047+0000 7efd52ffd640 1 --2- 172.21.15.120:0/3038299562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d7d919a20 0x556d7d921790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:33.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.047+0000 7efd537fe640 1 --2- 172.21.15.120:0/3038299562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d7d91a340 0x556d7d925900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:33.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.047+0000 7efd52ffd640 1 -- 172.21.15.120:0/3038299562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d7d919a20 msgr2=0x556d7d921790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:33.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.047+0000 7efd52ffd640 1 --2- 172.21.15.120:0/3038299562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d7d919a20 0x556d7d921790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.047+0000 7efd52ffd640 1 -- 172.21.15.120:0/3038299562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d7d91a340 msgr2=0x556d7d925900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:33.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.047+0000 7efd52ffd640 1 --2- 172.21.15.120:0/3038299562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d7d91a340 0x556d7d925900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.047+0000 7efd52ffd640 1 -- 172.21.15.120:0/3038299562 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d7d911830 con 0x556d7d929150 2024-09-19T09:17:33.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.047+0000 7efd537fe640 1 --2- 172.21.15.120:0/3038299562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d7d91a340 0x556d7d925900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:17:33.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.047+0000 7efd52ffd640 1 --2- 172.21.15.120:0/3038299562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d7d929150 0x556d7d915160 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7efd4803f040 tx=0x7efd4802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c052a84be6e1ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:33.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.047+0000 7efd397fa640 1 -- 172.21.15.120:0/3038299562 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd48041070 con 0x556d7d929150 2024-09-19T09:17:33.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.047+0000 7efd397fa640 1 -- 172.21.15.120:0/3038299562 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd480044c0 con 0x556d7d929150 2024-09-19T09:17:33.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.047+0000 7efd397fa640 1 -- 172.21.15.120:0/3038299562 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd48035070 con 0x556d7d929150 2024-09-19T09:17:33.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.048+0000 7efd5c2ec740 1 -- 172.21.15.120:0/3038299562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d7d929150 msgr2=0x556d7d915160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:33.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.048+0000 7efd5c2ec740 1 --2- 172.21.15.120:0/3038299562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d7d929150 0x556d7d915160 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7efd4803f040 tx=0x7efd4802a480 comp rx=0 tx=0).stop 2024-09-19T09:17:33.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.048+0000 7efd5c2ec740 1 -- 172.21.15.120:0/3038299562 shutdown_connections 2024-09-19T09:17:33.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.048+0000 7efd5c2ec740 1 --2- 172.21.15.120:0/3038299562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d7d929150 0x556d7d915160 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.048+0000 7efd5c2ec740 1 --2- 172.21.15.120:0/3038299562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d7d91a340 0x556d7d925900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.048+0000 7efd5c2ec740 1 --2- 172.21.15.120:0/3038299562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d7d919a20 0x556d7d921790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.048+0000 7efd5c2ec740 1 -- 172.21.15.120:0/3038299562 >> 172.21.15.120:0/3038299562 conn(0x556d7d9160f0 msgr2=0x556d7d914a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:33.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.048+0000 7efd5c2ec740 1 -- 172.21.15.120:0/3038299562 shutdown_connections 2024-09-19T09:17:33.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.048+0000 7efd5c2ec740 1 -- 172.21.15.120:0/3038299562 wait complete. 2024-09-19T09:17:33.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.049+0000 7efd5c2ec740 1 -- start start 2024-09-19T09:17:33.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.049+0000 7efd5c2ec740 1 Processor -- start 2024-09-19T09:17:33.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.049+0000 7efd5c2ec740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d7d919a20 0x556d7d92c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:33.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.049+0000 7efd5c2ec740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d7d91a340 0x556d7d92cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:33.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.049+0000 7efd5c2ec740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d7d929150 0x556d7d9a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:33.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.049+0000 7efd5c2ec740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efd541ac810 con 0x556d7d929150 2024-09-19T09:17:33.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.049+0000 7efd5c2ec740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efd541ac0b0 con 0x556d7d919a20 2024-09-19T09:17:33.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.049+0000 7efd5c2ec740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efd541a92f0 con 0x556d7d91a340 2024-09-19T09:17:33.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.049+0000 7efd5c2ec740 10 client.0 _ll_register_callbacks cb 0x7efd44005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:17:33.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.049+0000 7efd52ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d7d929150 0x556d7d9a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:33.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.049+0000 7efd52ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d7d929150 0x556d7d9a1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51374/0 (socket says 172.21.15.120:51374) 2024-09-19T09:17:33.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.049+0000 7efd52ffd640 1 -- 172.21.15.120:0/1914618743 learned_addr learned my addr 172.21.15.120:0/1914618743 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:33.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.049+0000 7efd52ffd640 1 --2- 172.21.15.120:0/1914618743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d7d919a20 0x556d7d92c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:33.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.049+0000 7efd537fe640 1 --2- 172.21.15.120:0/1914618743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d7d91a340 0x556d7d92cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:33.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.050+0000 7efd52ffd640 1 -- 172.21.15.120:0/1914618743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d7d91a340 msgr2=0x556d7d92cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:33.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.050+0000 7efd52ffd640 1 --2- 172.21.15.120:0/1914618743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d7d91a340 0x556d7d92cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.050+0000 7efd52ffd640 1 -- 172.21.15.120:0/1914618743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d7d919a20 msgr2=0x556d7d92c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:33.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.050+0000 7efd52ffd640 1 --2- 172.21.15.120:0/1914618743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d7d919a20 0x556d7d92c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.050+0000 7efd52ffd640 1 -- 172.21.15.120:0/1914618743 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d7d9a2050 con 0x556d7d929150 2024-09-19T09:17:33.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.050+0000 7efd52ffd640 1 --2- 172.21.15.120:0/1914618743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d7d929150 0x556d7d9a1b10 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7efd480059a0 tx=0x7efd48031880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec58307772517e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:33.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.050+0000 7efd32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:17:33.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.050+0000 7efd32ffd640 1 -- 172.21.15.120:0/1914618743 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd48035070 con 0x556d7d929150 2024-09-19T09:17:33.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.050+0000 7efd32ffd640 1 -- 172.21.15.120:0/1914618743 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7efd48031a20 con 0x556d7d929150 2024-09-19T09:17:33.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.050+0000 7efd5c2ec740 1 -- 172.21.15.120:0/1914618743 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556d7d886d00 con 0x556d7d929150 2024-09-19T09:17:33.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.050+0000 7efd32ffd640 1 -- 172.21.15.120:0/1914618743 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7efd4805d040 con 0x556d7d929150 2024-09-19T09:17:33.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.050+0000 7efd5c2ec740 10 client.20493 fetch_fsmap learned FSMap version 36 2024-09-19T09:17:33.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.050+0000 7efd5c2ec740 1 -- 172.21.15.120:0/1914618743 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556d7d9a32c0 con 0x556d7d929150 2024-09-19T09:17:33.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.051+0000 7efd32ffd640 1 -- 172.21.15.120:0/1914618743 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd4803d860 con 0x556d7d929150 2024-09-19T09:17:33.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.051+0000 7efd32ffd640 1 -- 172.21.15.120:0/1914618743 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7efd4803c440 con 0x556d7d929150 2024-09-19T09:17:33.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.051+0000 7efd5c2ec740 10 client.20493 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:17:33.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.051+0000 7efd5c2ec740 10 client.20493 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:17:33.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.051+0000 7efd5c2ec740 20 client.20493 populate_metadata read hostname 'smithi120' 2024-09-19T09:17:33.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.051+0000 7efd5c2ec740 1 --2- 172.21.15.120:0/1914618743 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556d7d9a3980 0x556d7d9a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:33.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.051+0000 7efd5c2ec740 4 client.20493 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:17:33.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.051+0000 7efd5c2ec740 1 -- 172.21.15.120:0/1914618743 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556d7d886d00 con 0x556d7d9a3980 2024-09-19T09:17:33.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.051+0000 7efd537fe640 1 --2- 172.21.15.120:0/1914618743 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556d7d9a3980 0x556d7d9a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:33.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.051+0000 7efd32ffd640 10 client.20493 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:33.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.051+0000 7efd537fe640 1 --2- 172.21.15.120:0/1914618743 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556d7d9a3980 0x556d7d9a5da0 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42cc70926cb48b51 server_cookie=ce385d02f3e6cfe3 in_seq=0 out_seq=0 2024-09-19T09:17:33.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.054+0000 7efd32ffd640 1 -- 172.21.15.120:0/1914618743 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556d7d886d00 con 0x556d7d9a3980 2024-09-19T09:17:33.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.054+0000 7efd32ffd640 10 client.20493 handle_command_reply: tid=0 2024-09-19T09:17:33.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.054+0000 7efd5c2ec740 1 client.20493 shutdown 2024-09-19T09:17:33.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.054+0000 7efd5c2ec740 1 -- 172.21.15.120:0/1914618743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d7d929150 msgr2=0x556d7d9a1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:33.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.054+0000 7efd5c2ec740 1 --2- 172.21.15.120:0/1914618743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d7d929150 0x556d7d9a1b10 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7efd480059a0 tx=0x7efd48031880 comp rx=0 tx=0).stop 2024-09-19T09:17:33.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.054+0000 7efd5c2ec740 1 -- 172.21.15.120:0/1914618743 shutdown_connections 2024-09-19T09:17:33.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.054+0000 7efd5c2ec740 1 --2- 172.21.15.120:0/1914618743 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556d7d9a3980 0x556d7d9a5da0 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.054+0000 7efd5c2ec740 1 --2- 172.21.15.120:0/1914618743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556d7d929150 0x556d7d9a1b10 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.054+0000 7efd32ffd640 0 client.20493 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:33.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.054+0000 7efd5c2ec740 1 --2- 172.21.15.120:0/1914618743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d7d91a340 0x556d7d92cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.054+0000 7efd5c2ec740 1 --2- 172.21.15.120:0/1914618743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d7d919a20 0x556d7d92c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.054+0000 7efd5c2ec740 1 -- 172.21.15.120:0/1914618743 >> 172.21.15.120:0/1914618743 conn(0x556d7d9160f0 msgr2=0x556d7d915270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:33.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.054+0000 7efd5c2ec740 1 -- 172.21.15.120:0/1914618743 shutdown_connections 2024-09-19T09:17:33.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.054+0000 7efd5c2ec740 1 -- 172.21.15.120:0/1914618743 wait complete. 2024-09-19T09:17:33.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.054+0000 7efd5c2ec740 20 client.20493 trim_cache size 0 max 16384 2024-09-19T09:17:33.059 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:17:33.059 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:17:33.059 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "209278e4-d9fd-4d5d-ac02-9c918a46a563", 2024-09-19T09:17:33.059 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:17:33.059 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:17:33.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.055+0000 7efd59f13640 1 -- 172.21.15.120:0/1624134244 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efd3c0a3240 msgr2=0x7efd3c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:33.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.055+0000 7efd59f13640 1 --2- 172.21.15.120:0/1624134244 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efd3c0a3240 0x7efd3c0a5700 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7efd4c0a4ce0 tx=0x7efd4c067480 comp rx=0 tx=0).stop 2024-09-19T09:17:33.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.055+0000 7efd59f13640 1 -- 172.21.15.120:0/1624134244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd54102ef0 msgr2=0x7efd541a43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:33.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.055+0000 7efd59f13640 1 --2- 172.21.15.120:0/1624134244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd54102ef0 0x7efd541a43d0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7efd4c009fd0 tx=0x7efd4c00ef10 comp rx=0 tx=0).stop 2024-09-19T09:17:33.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.055+0000 7efd59f13640 1 -- 172.21.15.120:0/1624134244 shutdown_connections 2024-09-19T09:17:33.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.055+0000 7efd59f13640 1 --2- 172.21.15.120:0/1624134244 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efd3c0a3240 0x7efd3c0a5700 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.055+0000 7efd59f13640 1 --2- 172.21.15.120:0/1624134244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd54102ef0 0x7efd541a43d0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.055+0000 7efd59f13640 1 --2- 172.21.15.120:0/1624134244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd54102570 0x7efd54199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.055+0000 7efd59f13640 1 --2- 172.21.15.120:0/1624134244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd54100e20 0x7efd54199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:33.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.055+0000 7efd59f13640 1 -- 172.21.15.120:0/1624134244 >> 172.21.15.120:0/1624134244 conn(0x7efd540fc7d0 msgr2=0x7efd540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:33.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.055+0000 7efd59f13640 1 -- 172.21.15.120:0/1624134244 shutdown_connections 2024-09-19T09:17:33.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:33.055+0000 7efd59f13640 1 -- 172.21.15.120:0/1624134244 wait complete. 2024-09-19T09:17:33.238 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:32 smithi196 ceph-mon[35150]: pgmap v785: 97 pgs: 97 active+clean; 284 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 9.6 MiB/s wr, 808 op/s 2024-09-19T09:17:33.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:32 smithi120 ceph-mon[31150]: pgmap v785: 97 pgs: 97 active+clean; 284 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 9.6 MiB/s wr, 808 op/s 2024-09-19T09:17:33.355 INFO:tasks.workunit.client.0.smithi120.stdout: 10 40846 26.92 MB/sec warmup 88 sec latency 61.835 ms 2024-09-19T09:17:33.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:32 smithi153 ceph-mon[35071]: pgmap v785: 97 pgs: 97 active+clean; 284 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 9.6 MiB/s wr, 808 op/s 2024-09-19T09:17:33.956 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:17:34.230 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:17:34.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:33 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:17:34.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:33 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:17:34.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:33 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:17:34.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:33 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:17:34.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:33 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:17:34.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:33 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:17:34.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:33 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:17:34.339 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:33 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:17:34.339 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:33 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:17:34.339 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:33 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:17:34.339 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:33 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:17:34.339 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:33 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:17:34.339 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:33 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:17:34.339 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:33 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:17:34.355 INFO:tasks.workunit.client.0.smithi120.stdout: 10 41647 27.23 MB/sec warmup 89 sec latency 51.071 ms 2024-09-19T09:17:34.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:33 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:17:34.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:33 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:17:34.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:33 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:17:34.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:33 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:17:34.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:33 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:17:34.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:33 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:17:34.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:33 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:17:34.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.759+0000 7fdec309e640 1 -- 172.21.15.120:0/3092943154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdebc100e20 msgr2=0x7fdebc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:34.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.759+0000 7fdec309e640 1 --2- 172.21.15.120:0/3092943154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdebc100e20 0x7fdebc101200 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0009900 tx=0x7fdeb002a480 comp rx=0 tx=0).stop 2024-09-19T09:17:34.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.759+0000 7fdec309e640 1 -- 172.21.15.120:0/3092943154 shutdown_connections 2024-09-19T09:17:34.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.759+0000 7fdec309e640 1 --2- 172.21.15.120:0/3092943154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdebc102ef0 0x7fdebc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.759+0000 7fdec309e640 1 --2- 172.21.15.120:0/3092943154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdebc102570 0x7fdebc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.759+0000 7fdec309e640 1 --2- 172.21.15.120:0/3092943154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdebc100e20 0x7fdebc101200 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.759+0000 7fdec309e640 1 -- 172.21.15.120:0/3092943154 >> 172.21.15.120:0/3092943154 conn(0x7fdebc0fc7d0 msgr2=0x7fdebc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:34.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.759+0000 7fdec309e640 1 -- 172.21.15.120:0/3092943154 shutdown_connections 2024-09-19T09:17:34.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.759+0000 7fdec309e640 1 -- 172.21.15.120:0/3092943154 wait complete. 2024-09-19T09:17:34.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.760+0000 7fdec309e640 1 Processor -- start 2024-09-19T09:17:34.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.760+0000 7fdec309e640 1 -- start start 2024-09-19T09:17:34.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.760+0000 7fdec309e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdebc100e20 0x7fdebc199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:34.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.760+0000 7fdec309e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdebc102570 0x7fdebc199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:34.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.760+0000 7fdec309e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdebc102ef0 0x7fdebc1a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:34.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.760+0000 7fdec309e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc110fa0 con 0x7fdebc102ef0 2024-09-19T09:17:34.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.760+0000 7fdec309e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc110e20 con 0x7fdebc102570 2024-09-19T09:17:34.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.760+0000 7fdec309e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc111120 con 0x7fdebc100e20 2024-09-19T09:17:34.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.760+0000 7fdec163b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdebc102ef0 0x7fdebc1a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:34.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.760+0000 7fdec0e3a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdebc100e20 0x7fdebc199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:34.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.760+0000 7fdebbfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdebc102570 0x7fdebc199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:34.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.760+0000 7fdec0e3a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdebc100e20 0x7fdebc199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:39952/0 (socket says 172.21.15.120:39952) 2024-09-19T09:17:34.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.760+0000 7fdec0e3a640 1 -- 172.21.15.120:0/2356721000 learned_addr learned my addr 172.21.15.120:0/2356721000 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:34.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.760+0000 7fdebbfff640 1 --2- 172.21.15.120:0/2356721000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdebc102570 0x7fdebc199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38976/0 (socket says 172.21.15.120:38976) 2024-09-19T09:17:34.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.761+0000 7fdec0e3a640 1 -- 172.21.15.120:0/2356721000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdebc102570 msgr2=0x7fdebc199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:34.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.761+0000 7fdec0e3a640 1 --2- 172.21.15.120:0/2356721000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdebc102570 0x7fdebc199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.761+0000 7fdec0e3a640 1 -- 172.21.15.120:0/2356721000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdebc102ef0 msgr2=0x7fdebc1a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:34.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.761+0000 7fdec0e3a640 1 --2- 172.21.15.120:0/2356721000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdebc102ef0 0x7fdebc1a43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.761+0000 7fdec0e3a640 1 -- 172.21.15.120:0/2356721000 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdebc1a48f0 con 0x7fdebc100e20 2024-09-19T09:17:34.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.761+0000 7fdec0e3a640 1 --2- 172.21.15.120:0/2356721000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdebc100e20 0x7fdebc199330 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdeb002aa70 tx=0x7fdeb0003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:34.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.761+0000 7fdeb9ffb640 1 -- 172.21.15.120:0/2356721000 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb003f070 con 0x7fdebc100e20 2024-09-19T09:17:34.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.761+0000 7fdec309e640 1 -- 172.21.15.120:0/2356721000 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdebc1a4ac0 con 0x7fdebc100e20 2024-09-19T09:17:34.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.761+0000 7fdeb9ffb640 1 -- 172.21.15.120:0/2356721000 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb0031c30 con 0x7fdebc100e20 2024-09-19T09:17:34.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.761+0000 7fdeb9ffb640 1 -- 172.21.15.120:0/2356721000 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0035030 con 0x7fdebc100e20 2024-09-19T09:17:34.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.761+0000 7fdec309e640 1 -- 172.21.15.120:0/2356721000 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdebc1a4f70 con 0x7fdebc100e20 2024-09-19T09:17:34.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.764+0000 7fdec5477740 1 Processor -- start 2024-09-19T09:17:34.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.764+0000 7fdec5477740 1 -- start start 2024-09-19T09:17:34.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.764+0000 7fdeb9ffb640 1 -- 172.21.15.120:0/2356721000 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fdeb0042070 con 0x7fdebc100e20 2024-09-19T09:17:34.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.765+0000 7fdec5477740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557762a6c4a0 0x557762a6c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:34.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.765+0000 7fdec5477740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557762a6cdc0 0x557762a77140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:34.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.765+0000 7fdec5477740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557762a77680 0x557762a79a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:34.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.765+0000 7fdec5477740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5577629d22e0 con 0x557762a77680 2024-09-19T09:17:34.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.765+0000 7fdec5477740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557762979ce0 con 0x557762a6c4a0 2024-09-19T09:17:34.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.765+0000 7fdec5477740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5577629ce2a0 con 0x557762a6cdc0 2024-09-19T09:17:34.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.765+0000 7fdec0e3a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557762a6cdc0 0x557762a77140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:34.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.765+0000 7fdebbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557762a77680 0x557762a79a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:34.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.765+0000 7fdec0e3a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557762a6cdc0 0x557762a77140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:39966/0 (socket says 172.21.15.120:39966) 2024-09-19T09:17:34.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.765+0000 7fdec0e3a640 1 -- 172.21.15.120:0/1459499339 learned_addr learned my addr 172.21.15.120:0/1459499339 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:34.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.765+0000 7fdec163b640 1 --2- 172.21.15.120:0/1459499339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557762a6c4a0 0x557762a6c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:34.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.765+0000 7fdec0e3a640 1 -- 172.21.15.120:0/1459499339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557762a6c4a0 msgr2=0x557762a6c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:34.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.765+0000 7fdec0e3a640 1 --2- 172.21.15.120:0/1459499339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557762a6c4a0 0x557762a6c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.765+0000 7fdec0e3a640 1 -- 172.21.15.120:0/1459499339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557762a77680 msgr2=0x557762a79a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:34.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.765+0000 7fdec0e3a640 1 --2- 172.21.15.120:0/1459499339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557762a77680 0x557762a79a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.765+0000 7fdec0e3a640 1 -- 172.21.15.120:0/1459499339 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55776295e440 con 0x557762a6cdc0 2024-09-19T09:17:34.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.766+0000 7fdec0e3a640 1 --2- 172.21.15.120:0/1459499339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557762a6cdc0 0x557762a77140 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdeb00f3fd0 tx=0x7fdeb00f81a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1a61cc2dabedcc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:34.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.766+0000 7fdeb9ffb640 1 --2- 172.21.15.120:0/2356721000 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdeac0a3240 0x7fdeac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:34.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.766+0000 7fdea6ffd640 1 -- 172.21.15.120:0/1459499339 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb00f8de0 con 0x557762a6cdc0 2024-09-19T09:17:34.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.766+0000 7fdea6ffd640 1 -- 172.21.15.120:0/1459499339 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb00fd070 con 0x557762a6cdc0 2024-09-19T09:17:34.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.766+0000 7fdea6ffd640 1 -- 172.21.15.120:0/1459499339 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb00e6800 con 0x557762a6cdc0 2024-09-19T09:17:34.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.766+0000 7fdec163b640 1 --2- 172.21.15.120:0/2356721000 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdeac0a3240 0x7fdeac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:34.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.766+0000 7fdeb9ffb640 1 -- 172.21.15.120:0/2356721000 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fdeb00efef0 con 0x7fdebc100e20 2024-09-19T09:17:34.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.767+0000 7fdec5477740 1 -- 172.21.15.120:0/1459499339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557762a6cdc0 msgr2=0x557762a77140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:34.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.767+0000 7fdec163b640 1 --2- 172.21.15.120:0/2356721000 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdeac0a3240 0x7fdeac0a5700 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4005fd0 tx=0x7fdeb4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:34.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.767+0000 7fdec5477740 1 --2- 172.21.15.120:0/1459499339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557762a6cdc0 0x557762a77140 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdeb00f3fd0 tx=0x7fdeb00f81a0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.767+0000 7fdec5477740 1 -- 172.21.15.120:0/1459499339 shutdown_connections 2024-09-19T09:17:34.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.767+0000 7fdec5477740 1 --2- 172.21.15.120:0/1459499339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557762a77680 0x557762a79a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.767+0000 7fdec5477740 1 --2- 172.21.15.120:0/1459499339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557762a6cdc0 0x557762a77140 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.767+0000 7fdec5477740 1 --2- 172.21.15.120:0/1459499339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557762a6c4a0 0x557762a6c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.767+0000 7fdec5477740 1 -- 172.21.15.120:0/1459499339 >> 172.21.15.120:0/1459499339 conn(0x557762a68a30 msgr2=0x557762a68e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:34.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.767+0000 7fdec5477740 1 -- 172.21.15.120:0/1459499339 shutdown_connections 2024-09-19T09:17:34.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.767+0000 7fdec5477740 1 -- 172.21.15.120:0/1459499339 wait complete. 2024-09-19T09:17:34.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.767+0000 7fdec5477740 1 -- start start 2024-09-19T09:17:34.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.767+0000 7fdec5477740 1 Processor -- start 2024-09-19T09:17:34.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.768+0000 7fdec5477740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557762a6c4a0 0x557762a76f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:34.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.768+0000 7fdec5477740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557762a6cdc0 0x557762a74240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:34.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.768+0000 7fdec5477740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557762a77680 0x557762a74780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:34.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.768+0000 7fdec5477740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557762a7a010 con 0x557762a77680 2024-09-19T09:17:34.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.768+0000 7fdec5477740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55776298d050 con 0x557762a6c4a0 2024-09-19T09:17:34.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.768+0000 7fdec5477740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557762a7a190 con 0x557762a6cdc0 2024-09-19T09:17:34.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.768+0000 7fdec5477740 10 client.0 _ll_register_callbacks cb 0x5577629cd160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:17:34.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.768+0000 7fdebbfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557762a6cdc0 0x557762a74240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:34.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.768+0000 7fdec163b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557762a77680 0x557762a74780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:34.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.768+0000 7fdec0e3a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557762a6c4a0 0x557762a76f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:34.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.768+0000 7fdebbfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557762a6cdc0 0x557762a74240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:39980/0 (socket says 172.21.15.120:39980) 2024-09-19T09:17:34.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.768+0000 7fdec0e3a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557762a6c4a0 0x557762a76f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38996/0 (socket says 172.21.15.120:38996) 2024-09-19T09:17:34.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.768+0000 7fdebbfff640 1 -- 172.21.15.120:0/1669287105 learned_addr learned my addr 172.21.15.120:0/1669287105 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:34.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.768+0000 7fdebbfff640 1 -- 172.21.15.120:0/1669287105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557762a6c4a0 msgr2=0x557762a76f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:34.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.768+0000 7fdebbfff640 1 --2- 172.21.15.120:0/1669287105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557762a6c4a0 0x557762a76f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.768+0000 7fdebbfff640 1 -- 172.21.15.120:0/1669287105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557762a77680 msgr2=0x557762a74780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:34.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.768+0000 7fdebbfff640 1 --2- 172.21.15.120:0/1669287105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557762a77680 0x557762a74780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.768+0000 7fdebbfff640 1 -- 172.21.15.120:0/1669287105 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577627433c0 con 0x557762a6cdc0 2024-09-19T09:17:34.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.768+0000 7fdec0e3a640 1 --2- 172.21.15.120:0/1669287105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557762a6c4a0 0x557762a76f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:17:34.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.768+0000 7fdebbfff640 1 --2- 172.21.15.120:0/1669287105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557762a6cdc0 0x557762a74240 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdea8012700 tx=0x7fdea8012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3788e1e8992c02f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:34.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.768+0000 7fde9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:17:34.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.769+0000 7fde9bfff640 1 -- 172.21.15.120:0/1669287105 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea800f8c0 con 0x557762a6cdc0 2024-09-19T09:17:34.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.769+0000 7fdec5477740 1 -- 172.21.15.120:0/1669287105 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeb00fd070 con 0x557762a6cdc0 2024-09-19T09:17:34.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.769+0000 7fde9bfff640 1 -- 172.21.15.120:0/1669287105 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fdea800fa60 con 0x557762a6cdc0 2024-09-19T09:17:34.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.769+0000 7fde9bfff640 1 -- 172.21.15.120:0/1669287105 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fdea800fc60 con 0x557762a6cdc0 2024-09-19T09:17:34.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.769+0000 7fdec5477740 10 client.29284 fetch_fsmap learned FSMap version 36 2024-09-19T09:17:34.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.769+0000 7fdec5477740 1 -- 172.21.15.120:0/1669287105 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557762a859b0 con 0x557762a6cdc0 2024-09-19T09:17:34.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.769+0000 7fde9bfff640 1 -- 172.21.15.120:0/1669287105 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb00fd070 con 0x557762a6cdc0 2024-09-19T09:17:34.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.769+0000 7fde9bfff640 1 -- 172.21.15.120:0/1669287105 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fdea801c420 con 0x557762a6cdc0 2024-09-19T09:17:34.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.769+0000 7fdec5477740 10 client.29284 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:17:34.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.769+0000 7fdec5477740 10 client.29284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:17:34.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.769+0000 7fdec5477740 20 client.29284 populate_metadata read hostname 'smithi120' 2024-09-19T09:17:34.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.769+0000 7fdec5477740 1 --2- 172.21.15.120:0/1669287105 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557762a872c0 0x557762a89780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:34.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.770+0000 7fdec5477740 4 client.29284 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:17:34.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.770+0000 7fdec5477740 1 -- 172.21.15.120:0/1669287105 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577629df850 con 0x557762a872c0 2024-09-19T09:17:34.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.770+0000 7fdec0e3a640 1 --2- 172.21.15.120:0/1669287105 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557762a872c0 0x557762a89780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:34.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.770+0000 7fdec0e3a640 1 --2- 172.21.15.120:0/1669287105 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557762a872c0 0x557762a89780 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a7008d518e83b66 server_cookie=8d26c9d3a680715 in_seq=0 out_seq=0 2024-09-19T09:17:34.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.770+0000 7fde9bfff640 10 client.29284 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:34.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.774+0000 7fde9bfff640 1 -- 172.21.15.120:0/1669287105 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5577629df850 con 0x557762a872c0 2024-09-19T09:17:34.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.774+0000 7fde9bfff640 10 client.29284 handle_command_reply: tid=0 2024-09-19T09:17:34.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.774+0000 7fdec5477740 1 client.29284 shutdown 2024-09-19T09:17:34.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.774+0000 7fdec5477740 1 -- 172.21.15.120:0/1669287105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557762a6cdc0 msgr2=0x557762a74240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:34.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.774+0000 7fdec5477740 1 --2- 172.21.15.120:0/1669287105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557762a6cdc0 0x557762a74240 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdea8012700 tx=0x7fdea8012bc0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.775+0000 7fdec5477740 1 -- 172.21.15.120:0/1669287105 shutdown_connections 2024-09-19T09:17:34.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.775+0000 7fdec5477740 1 --2- 172.21.15.120:0/1669287105 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557762a872c0 0x557762a89780 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.775+0000 7fdec5477740 1 --2- 172.21.15.120:0/1669287105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557762a77680 0x557762a74780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.775+0000 7fdec5477740 1 --2- 172.21.15.120:0/1669287105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557762a6cdc0 0x557762a74240 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.775+0000 7fdec5477740 1 --2- 172.21.15.120:0/1669287105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557762a6c4a0 0x557762a76f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.775+0000 7fde9bfff640 0 client.29284 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:34.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.775+0000 7fdec5477740 1 -- 172.21.15.120:0/1669287105 >> 172.21.15.120:0/1669287105 conn(0x557762a68a30 msgr2=0x557762a6e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:34.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.775+0000 7fdec5477740 1 -- 172.21.15.120:0/1669287105 shutdown_connections 2024-09-19T09:17:34.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.775+0000 7fdec5477740 1 -- 172.21.15.120:0/1669287105 wait complete. 2024-09-19T09:17:34.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.775+0000 7fdec5477740 20 client.29284 trim_cache size 0 max 16384 2024-09-19T09:17:34.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.795+0000 7fdec5477740 1 Processor -- start 2024-09-19T09:17:34.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.795+0000 7fdec5477740 1 -- start start 2024-09-19T09:17:34.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.795+0000 7fdec5477740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557762a6c4a0 0x557762a83060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:34.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.795+0000 7fdec5477740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557762a77680 0x557762a714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:34.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.795+0000 7fdec5477740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557762a719e0 0x557762a71e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:34.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.795+0000 7fdec5477740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc1ab3e0 con 0x557762a719e0 2024-09-19T09:17:34.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.795+0000 7fdec5477740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc1acfb0 con 0x557762a6c4a0 2024-09-19T09:17:34.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.795+0000 7fdec5477740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc1abab0 con 0x557762a77680 2024-09-19T09:17:34.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.795+0000 7fdec163b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557762a719e0 0x557762a71e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:34.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.795+0000 7fdec163b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557762a719e0 0x557762a71e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51406/0 (socket says 172.21.15.120:51406) 2024-09-19T09:17:34.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.795+0000 7fdec163b640 1 -- 172.21.15.120:0/390851104 learned_addr learned my addr 172.21.15.120:0/390851104 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:34.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.795+0000 7fdebbfff640 1 --2- 172.21.15.120:0/390851104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557762a77680 0x557762a714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:34.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.795+0000 7fdec0e3a640 1 --2- 172.21.15.120:0/390851104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557762a6c4a0 0x557762a83060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:34.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.796+0000 7fdec163b640 1 -- 172.21.15.120:0/390851104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557762a77680 msgr2=0x557762a714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:34.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.796+0000 7fdec163b640 1 --2- 172.21.15.120:0/390851104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557762a77680 0x557762a714a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.796+0000 7fdec163b640 1 -- 172.21.15.120:0/390851104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557762a6c4a0 msgr2=0x557762a83060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:34.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.796+0000 7fdec163b640 1 --2- 172.21.15.120:0/390851104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557762a6c4a0 0x557762a83060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.796+0000 7fdec163b640 1 -- 172.21.15.120:0/390851104 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557762a835a0 con 0x557762a719e0 2024-09-19T09:17:34.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.796+0000 7fdec163b640 1 --2- 172.21.15.120:0/390851104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557762a719e0 0x557762a71e60 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4003f80 tx=0x7fdeb401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3dacfad95b2f6c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:34.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.796+0000 7fdea67fc640 1 -- 172.21.15.120:0/390851104 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4021de0 con 0x557762a719e0 2024-09-19T09:17:34.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.796+0000 7fdea67fc640 1 -- 172.21.15.120:0/390851104 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb4004540 con 0x557762a719e0 2024-09-19T09:17:34.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.796+0000 7fdea67fc640 1 -- 172.21.15.120:0/390851104 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb401f7b0 con 0x557762a719e0 2024-09-19T09:17:34.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.797+0000 7fdec5477740 1 -- 172.21.15.120:0/390851104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557762a719e0 msgr2=0x557762a71e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:34.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.797+0000 7fdec5477740 1 --2- 172.21.15.120:0/390851104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557762a719e0 0x557762a71e60 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4003f80 tx=0x7fdeb401a000 comp rx=0 tx=0).stop 2024-09-19T09:17:34.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.797+0000 7fdec5477740 1 -- 172.21.15.120:0/390851104 shutdown_connections 2024-09-19T09:17:34.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.797+0000 7fdec5477740 1 --2- 172.21.15.120:0/390851104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557762a719e0 0x557762a71e60 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.797+0000 7fdec5477740 1 --2- 172.21.15.120:0/390851104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557762a77680 0x557762a714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.797+0000 7fdec5477740 1 --2- 172.21.15.120:0/390851104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557762a6c4a0 0x557762a83060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.797+0000 7fdec5477740 1 -- 172.21.15.120:0/390851104 >> 172.21.15.120:0/390851104 conn(0x557762a68a30 msgr2=0x557762a6f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:34.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.797+0000 7fdec5477740 1 -- 172.21.15.120:0/390851104 shutdown_connections 2024-09-19T09:17:34.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.797+0000 7fdec5477740 1 -- 172.21.15.120:0/390851104 wait complete. 2024-09-19T09:17:34.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.798+0000 7fdec5477740 1 -- start start 2024-09-19T09:17:34.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.798+0000 7fdec5477740 1 Processor -- start 2024-09-19T09:17:34.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.798+0000 7fdec5477740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557762a6c4a0 0x557762a8c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:34.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.798+0000 7fdec5477740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557762a719e0 0x557762a8cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:34.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.798+0000 7fdec5477740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557762a77680 0x557762a8ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:34.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.798+0000 7fdec5477740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc1ac8e0 con 0x557762a6c4a0 2024-09-19T09:17:34.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.798+0000 7fdec5477740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc1ac180 con 0x557762a77680 2024-09-19T09:17:34.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.798+0000 7fdec5477740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc1a93c0 con 0x557762a719e0 2024-09-19T09:17:34.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.798+0000 7fdec5477740 10 client.0 _ll_register_callbacks cb 0x7fdea8009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:17:34.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.798+0000 7fdebbfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557762a719e0 0x557762a8cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:34.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.798+0000 7fdec0e3a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557762a6c4a0 0x557762a8c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:34.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.798+0000 7fdebbfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557762a719e0 0x557762a8cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:40006/0 (socket says 172.21.15.120:40006) 2024-09-19T09:17:34.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.798+0000 7fdec0e3a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557762a6c4a0 0x557762a8c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51416/0 (socket says 172.21.15.120:51416) 2024-09-19T09:17:34.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.798+0000 7fdebbfff640 1 -- 172.21.15.120:0/1937933294 learned_addr learned my addr 172.21.15.120:0/1937933294 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:34.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.798+0000 7fdec163b640 1 --2- 172.21.15.120:0/1937933294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557762a77680 0x557762a8ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:34.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.799+0000 7fdebbfff640 1 -- 172.21.15.120:0/1937933294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557762a77680 msgr2=0x557762a8ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:34.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.799+0000 7fdebbfff640 1 --2- 172.21.15.120:0/1937933294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557762a77680 0x557762a8ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.799+0000 7fdebbfff640 1 -- 172.21.15.120:0/1937933294 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557762a6c4a0 msgr2=0x557762a8c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:34.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.799+0000 7fdebbfff640 1 --2- 172.21.15.120:0/1937933294 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557762a6c4a0 0x557762a8c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.799+0000 7fdebbfff640 1 -- 172.21.15.120:0/1937933294 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557762a8f350 con 0x557762a719e0 2024-09-19T09:17:34.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.799+0000 7fdec0e3a640 1 --2- 172.21.15.120:0/1937933294 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557762a6c4a0 0x557762a8c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:17:34.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.799+0000 7fdebbfff640 1 --2- 172.21.15.120:0/1937933294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557762a719e0 0x557762a8cc60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdea8012040 tx=0x7fdea8009e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71aaf8c058b825fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:34.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.799+0000 7fde9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:17:34.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.799+0000 7fde9bfff640 1 -- 172.21.15.120:0/1937933294 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea801cab0 con 0x557762a719e0 2024-09-19T09:17:34.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.799+0000 7fde9bfff640 1 -- 172.21.15.120:0/1937933294 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fdea801cc50 con 0x557762a719e0 2024-09-19T09:17:34.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.799+0000 7fdec5477740 1 -- 172.21.15.120:0/1937933294 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeb4004540 con 0x557762a719e0 2024-09-19T09:17:34.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.799+0000 7fde9bfff640 1 -- 172.21.15.120:0/1937933294 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fdea8036050 con 0x557762a719e0 2024-09-19T09:17:34.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.799+0000 7fdec5477740 10 client.29290 fetch_fsmap learned FSMap version 36 2024-09-19T09:17:34.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.799+0000 7fdec5477740 1 -- 172.21.15.120:0/1937933294 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557762af1aa0 con 0x557762a719e0 2024-09-19T09:17:34.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.800+0000 7fde9bfff640 1 -- 172.21.15.120:0/1937933294 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb4004540 con 0x557762a719e0 2024-09-19T09:17:34.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.800+0000 7fde9bfff640 1 -- 172.21.15.120:0/1937933294 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fdea80035f0 con 0x557762a719e0 2024-09-19T09:17:34.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.800+0000 7fdec5477740 10 client.29290 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:17:34.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.800+0000 7fdec5477740 10 client.29290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:17:34.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.800+0000 7fdec5477740 20 client.29290 populate_metadata read hostname 'smithi120' 2024-09-19T09:17:34.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.800+0000 7fdec5477740 1 --2- 172.21.15.120:0/1937933294 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557762af22a0 0x557762af46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:34.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.800+0000 7fdec5477740 4 client.29290 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:17:34.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.800+0000 7fdec5477740 1 -- 172.21.15.120:0/1937933294 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5577629d9d60 con 0x557762af22a0 2024-09-19T09:17:34.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.800+0000 7fdec0e3a640 1 --2- 172.21.15.120:0/1937933294 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557762af22a0 0x557762af46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:34.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.800+0000 7fde9bfff640 10 client.29290 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:34.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.800+0000 7fdec0e3a640 1 --2- 172.21.15.120:0/1937933294 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557762af22a0 0x557762af46c0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=406eb048c4355725 server_cookie=9698ddb3c1a5980d in_seq=0 out_seq=0 2024-09-19T09:17:34.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.803+0000 7fde9bfff640 1 -- 172.21.15.120:0/1937933294 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5577629d9d60 con 0x557762af22a0 2024-09-19T09:17:34.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.803+0000 7fde9bfff640 10 client.29290 handle_command_reply: tid=0 2024-09-19T09:17:34.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.803+0000 7fdec5477740 1 client.29290 shutdown 2024-09-19T09:17:34.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.803+0000 7fdec5477740 1 -- 172.21.15.120:0/1937933294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557762a719e0 msgr2=0x557762a8cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:34.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.803+0000 7fdec5477740 1 --2- 172.21.15.120:0/1937933294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557762a719e0 0x557762a8cc60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdea8012040 tx=0x7fdea8009e40 comp rx=0 tx=0).stop 2024-09-19T09:17:34.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.803+0000 7fdec5477740 1 -- 172.21.15.120:0/1937933294 shutdown_connections 2024-09-19T09:17:34.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.803+0000 7fdec5477740 1 --2- 172.21.15.120:0/1937933294 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557762af22a0 0x557762af46c0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.803+0000 7fdec5477740 1 --2- 172.21.15.120:0/1937933294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557762a77680 0x557762a8ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.803+0000 7fdec5477740 1 --2- 172.21.15.120:0/1937933294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557762a719e0 0x557762a8cc60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.803+0000 7fde9bfff640 0 client.29290 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:34.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.803+0000 7fdec5477740 1 --2- 172.21.15.120:0/1937933294 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557762a6c4a0 0x557762a8c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.803+0000 7fdec5477740 1 -- 172.21.15.120:0/1937933294 >> 172.21.15.120:0/1937933294 conn(0x557762a68a30 msgr2=0x557762a67350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:34.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.803+0000 7fdec5477740 1 -- 172.21.15.120:0/1937933294 shutdown_connections 2024-09-19T09:17:34.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.803+0000 7fdec5477740 1 -- 172.21.15.120:0/1937933294 wait complete. 2024-09-19T09:17:34.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.804+0000 7fdec5477740 20 client.29290 trim_cache size 0 max 16384 2024-09-19T09:17:34.807 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:17:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:17:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-19T09:17:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: "209278e4-d9fd-4d5d-ac02-9c918a46a563": { 2024-09-19T09:17:34.807 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-19T09:17:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "209278e4-d9fd-4d5d-ac02-9c918a46a563", 2024-09-19T09:17:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-19T09:17:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:17:34.808 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:17:34.808 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:17:34.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.804+0000 7fdec309e640 1 -- 172.21.15.120:0/2356721000 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdeac0a3240 msgr2=0x7fdeac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:34.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.804+0000 7fdec309e640 1 --2- 172.21.15.120:0/2356721000 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdeac0a3240 0x7fdeac0a5700 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4005fd0 tx=0x7fdeb4009500 comp rx=0 tx=0).stop 2024-09-19T09:17:34.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.804+0000 7fdec309e640 1 -- 172.21.15.120:0/2356721000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdebc100e20 msgr2=0x7fdebc199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:34.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.804+0000 7fdec309e640 1 --2- 172.21.15.120:0/2356721000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdebc100e20 0x7fdebc199330 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdeb002aa70 tx=0x7fdeb0003c10 comp rx=0 tx=0).stop 2024-09-19T09:17:34.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.804+0000 7fdec309e640 1 -- 172.21.15.120:0/2356721000 shutdown_connections 2024-09-19T09:17:34.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.804+0000 7fdec309e640 1 --2- 172.21.15.120:0/2356721000 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdeac0a3240 0x7fdeac0a5700 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.804+0000 7fdec309e640 1 --2- 172.21.15.120:0/2356721000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdebc102ef0 0x7fdebc1a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.804+0000 7fdec309e640 1 --2- 172.21.15.120:0/2356721000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdebc102570 0x7fdebc199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.804+0000 7fdec309e640 1 --2- 172.21.15.120:0/2356721000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdebc100e20 0x7fdebc199330 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:34.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.804+0000 7fdec309e640 1 -- 172.21.15.120:0/2356721000 >> 172.21.15.120:0/2356721000 conn(0x7fdebc0fc7d0 msgr2=0x7fdebc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:34.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.805+0000 7fdec309e640 1 -- 172.21.15.120:0/2356721000 shutdown_connections 2024-09-19T09:17:34.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:34.805+0000 7fdec309e640 1 -- 172.21.15.120:0/2356721000 wait complete. 2024-09-19T09:17:35.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:34 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:17:35.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:34 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:17:35.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:34 smithi120 ceph-mon[31150]: pgmap v786: 97 pgs: 97 active+clean; 285 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.6 MiB/s wr, 808 op/s 2024-09-19T09:17:35.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:34 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:17:35.355 INFO:tasks.workunit.client.0.smithi120.stdout: 10 42257 27.29 MB/sec warmup 90 sec latency 50.363 ms 2024-09-19T09:17:35.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:34 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:17:35.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:34 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:17:35.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:34 smithi196 ceph-mon[35150]: pgmap v786: 97 pgs: 97 active+clean; 285 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.6 MiB/s wr, 808 op/s 2024-09-19T09:17:35.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:34 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:17:35.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:34 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:17:35.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:34 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:17:35.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:34 smithi153 ceph-mon[35071]: pgmap v786: 97 pgs: 97 active+clean; 285 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.6 MiB/s wr, 808 op/s 2024-09-19T09:17:35.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:34 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:17:35.615 INFO:tasks.cephfs.filesystem:scrub status for tag:209278e4-d9fd-4d5d-ac02-9c918a46a563 - {'path': '/', 'tag': '209278e4-d9fd-4d5d-ac02-9c918a46a563', 'options': 'recursive,force'} 2024-09-19T09:17:36.355 INFO:tasks.workunit.client.0.smithi120.stdout: 10 42832 27.33 MB/sec warmup 91 sec latency 62.327 ms 2024-09-19T09:17:37.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:36 smithi120 ceph-mon[31150]: pgmap v787: 97 pgs: 97 active+clean; 295 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.9 MiB/s wr, 824 op/s 2024-09-19T09:17:37.355 INFO:tasks.workunit.client.0.smithi120.stdout: 10 43253 27.24 MB/sec warmup 92 sec latency 224.492 ms 2024-09-19T09:17:37.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:36 smithi196 ceph-mon[35150]: pgmap v787: 97 pgs: 97 active+clean; 295 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.9 MiB/s wr, 824 op/s 2024-09-19T09:17:37.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:36 smithi153 ceph-mon[35071]: pgmap v787: 97 pgs: 97 active+clean; 295 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.9 MiB/s wr, 824 op/s 2024-09-19T09:17:38.355 INFO:tasks.workunit.client.0.smithi120.stdout: 10 43826 27.32 MB/sec warmup 93 sec latency 236.322 ms 2024-09-19T09:17:39.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.122+0000 7f00d7a5c640 1 -- 172.21.15.120:0/2950635479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00d0101b60 msgr2=0x7f00d0101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:39.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.122+0000 7f00d7a5c640 1 --2- 172.21.15.120:0/2950635479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00d0101b60 0x7f00d0101f60 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f00c0009860 tx=0x7f00c002a340 comp rx=0 tx=0).stop 2024-09-19T09:17:39.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.122+0000 7f00d7a5c640 1 -- 172.21.15.120:0/2950635479 shutdown_connections 2024-09-19T09:17:39.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.122+0000 7f00d7a5c640 1 --2- 172.21.15.120:0/2950635479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00d0103710 0x7f00d0109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.122+0000 7f00d7a5c640 1 --2- 172.21.15.120:0/2950635479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00d0102d50 0x7f00d01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.122+0000 7f00d7a5c640 1 --2- 172.21.15.120:0/2950635479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00d0101b60 0x7f00d0101f60 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.122+0000 7f00d7a5c640 1 -- 172.21.15.120:0/2950635479 >> 172.21.15.120:0/2950635479 conn(0x7f00d00fd2e0 msgr2=0x7f00d00ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:39.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.122+0000 7f00d7a5c640 1 -- 172.21.15.120:0/2950635479 shutdown_connections 2024-09-19T09:17:39.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.122+0000 7f00d7a5c640 1 -- 172.21.15.120:0/2950635479 wait complete. 2024-09-19T09:17:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.123+0000 7f00d7a5c640 1 Processor -- start 2024-09-19T09:17:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.123+0000 7f00d7a5c640 1 -- start start 2024-09-19T09:17:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.123+0000 7f00d7a5c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00d0101b60 0x7f00d0197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.123+0000 7f00d7a5c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00d0102d50 0x7f00d0198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.123+0000 7f00d7a5c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00d0103710 0x7f00d019f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.123+0000 7f00d7a5c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f00d010d7e0 con 0x7f00d0102d50 2024-09-19T09:17:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.123+0000 7f00d7a5c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f00d010d660 con 0x7f00d0103710 2024-09-19T09:17:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.123+0000 7f00d7a5c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f00d010d960 con 0x7f00d0101b60 2024-09-19T09:17:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.123+0000 7f00d4ff7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00d0102d50 0x7f00d0198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.123+0000 7f00d4ff7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00d0102d50 0x7f00d0198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51432/0 (socket says 172.21.15.120:51432) 2024-09-19T09:17:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.123+0000 7f00d4ff7640 1 -- 172.21.15.120:0/1063057560 learned_addr learned my addr 172.21.15.120:0/1063057560 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.123+0000 7f00d5ff9640 1 --2- 172.21.15.120:0/1063057560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00d0103710 0x7f00d019f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.123+0000 7f00d57f8640 1 --2- 172.21.15.120:0/1063057560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00d0101b60 0x7f00d0197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.124+0000 7f00d4ff7640 1 -- 172.21.15.120:0/1063057560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00d0101b60 msgr2=0x7f00d0197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.124+0000 7f00d4ff7640 1 --2- 172.21.15.120:0/1063057560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00d0101b60 0x7f00d0197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.124+0000 7f00d4ff7640 1 -- 172.21.15.120:0/1063057560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00d0103710 msgr2=0x7f00d019f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.124+0000 7f00d4ff7640 1 --2- 172.21.15.120:0/1063057560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00d0103710 0x7f00d019f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.124+0000 7f00d4ff7640 1 -- 172.21.15.120:0/1063057560 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d019fa20 con 0x7f00d0102d50 2024-09-19T09:17:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.124+0000 7f00d4ff7640 1 --2- 172.21.15.120:0/1063057560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00d0102d50 0x7f00d0198340 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00e9c0 tx=0x7f00cc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.124+0000 7f00c67fc640 1 -- 172.21.15.120:0/1063057560 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc00ccc0 con 0x7f00d0102d50 2024-09-19T09:17:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.124+0000 7f00c67fc640 1 -- 172.21.15.120:0/1063057560 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00cc005500 con 0x7f00d0102d50 2024-09-19T09:17:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.124+0000 7f00c67fc640 1 -- 172.21.15.120:0/1063057560 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc010430 con 0x7f00d0102d50 2024-09-19T09:17:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.124+0000 7f00d7a5c640 1 -- 172.21.15.120:0/1063057560 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00d019fd10 con 0x7f00d0102d50 2024-09-19T09:17:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.124+0000 7f00d7a5c640 1 -- 172.21.15.120:0/1063057560 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00d01a0200 con 0x7f00d0102d50 2024-09-19T09:17:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.126+0000 7f00c67fc640 1 -- 172.21.15.120:0/1063057560 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f00cc0046c0 con 0x7f00d0102d50 2024-09-19T09:17:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.127+0000 7f00da7ed740 1 Processor -- start 2024-09-19T09:17:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.127+0000 7f00da7ed740 1 -- start start 2024-09-19T09:17:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.127+0000 7f00da7ed740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a1f6329a0 0x561a1f632da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.127+0000 7f00da7ed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a1f6332e0 0x561a1f637da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.127+0000 7f00da7ed740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1f638370 0x561a1f63a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.127+0000 7f00da7ed740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561a1f5962f0 con 0x561a1f6329a0 2024-09-19T09:17:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.127+0000 7f00da7ed740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561a1f53e270 con 0x561a1f638370 2024-09-19T09:17:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.127+0000 7f00da7ed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561a1f5922b0 con 0x561a1f6332e0 2024-09-19T09:17:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.127+0000 7f00d5ff9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a1f6329a0 0x561a1f632da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.127+0000 7f00d57f8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a1f6332e0 0x561a1f637da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.127+0000 7f00c67fc640 1 --2- 172.21.15.120:0/1063057560 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f00bc0a32f0 0x7f00bc0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.127+0000 7f00d57f8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a1f6332e0 0x561a1f637da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:40018/0 (socket says 172.21.15.120:40018) 2024-09-19T09:17:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.127+0000 7f00d57f8640 1 -- 172.21.15.120:0/309314887 learned_addr learned my addr 172.21.15.120:0/309314887 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.127+0000 7f00d4ff7640 1 --2- 172.21.15.120:0/309314887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1f638370 0x561a1f63a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.128+0000 7f00d5ff9640 1 --2- 172.21.15.120:0/1063057560 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f00bc0a32f0 0x7f00bc0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.128+0000 7f00d57f8640 1 -- 172.21.15.120:0/309314887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1f638370 msgr2=0x561a1f63a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.128+0000 7f00d57f8640 1 --2- 172.21.15.120:0/309314887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1f638370 0x561a1f63a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.128+0000 7f00d57f8640 1 -- 172.21.15.120:0/309314887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a1f6329a0 msgr2=0x561a1f632da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.128+0000 7f00d57f8640 1 --2- 172.21.15.120:0/309314887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a1f6329a0 0x561a1f632da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.128+0000 7f00d57f8640 1 -- 172.21.15.120:0/309314887 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a1f53d340 con 0x561a1f6332e0 2024-09-19T09:17:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.128+0000 7f00c67fc640 1 -- 172.21.15.120:0/1063057560 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f00cc0cfcb0 con 0x7f00d0102d50 2024-09-19T09:17:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.128+0000 7f00d5ff9640 1 --2- 172.21.15.120:0/309314887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a1f6329a0 0x561a1f632da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:17:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.128+0000 7f00d57f8640 1 --2- 172.21.15.120:0/309314887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a1f6332e0 0x561a1f637da0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f00c0005940 tx=0x7f00c0005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc18f06be2403579 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.128+0000 7f00bb7fe640 1 -- 172.21.15.120:0/309314887 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0037040 con 0x561a1f6332e0 2024-09-19T09:17:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.128+0000 7f00d5ff9640 1 --2- 172.21.15.120:0/1063057560 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f00bc0a32f0 0x7f00bc0a57b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f00c8005c40 tx=0x7f00c8004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.128+0000 7f00bb7fe640 1 -- 172.21.15.120:0/309314887 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00c0004030 con 0x561a1f6332e0 2024-09-19T09:17:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.128+0000 7f00bb7fe640 1 -- 172.21.15.120:0/309314887 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c00055e0 con 0x561a1f6332e0 2024-09-19T09:17:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.129+0000 7f00da7ed740 1 -- 172.21.15.120:0/309314887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a1f6332e0 msgr2=0x561a1f637da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.129+0000 7f00da7ed740 1 --2- 172.21.15.120:0/309314887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a1f6332e0 0x561a1f637da0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f00c0005940 tx=0x7f00c0005c90 comp rx=0 tx=0).stop 2024-09-19T09:17:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.129+0000 7f00da7ed740 1 -- 172.21.15.120:0/309314887 shutdown_connections 2024-09-19T09:17:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.129+0000 7f00da7ed740 1 --2- 172.21.15.120:0/309314887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1f638370 0x561a1f63a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.129+0000 7f00da7ed740 1 --2- 172.21.15.120:0/309314887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a1f6332e0 0x561a1f637da0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.129+0000 7f00da7ed740 1 --2- 172.21.15.120:0/309314887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a1f6329a0 0x561a1f632da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.129+0000 7f00da7ed740 1 -- 172.21.15.120:0/309314887 >> 172.21.15.120:0/309314887 conn(0x561a1f62e010 msgr2=0x561a1f62e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.129+0000 7f00da7ed740 1 -- 172.21.15.120:0/309314887 shutdown_connections 2024-09-19T09:17:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.129+0000 7f00da7ed740 1 -- 172.21.15.120:0/309314887 wait complete. 2024-09-19T09:17:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.129+0000 7f00da7ed740 1 -- start start 2024-09-19T09:17:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.130+0000 7f00da7ed740 1 Processor -- start 2024-09-19T09:17:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.130+0000 7f00da7ed740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a1f6329a0 0x561a1f63cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.130+0000 7f00da7ed740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1f6332e0 0x561a1f63d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.130+0000 7f00da7ed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a1f638370 0x561a1f6440c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.130+0000 7f00da7ed740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561a1f63b2e0 con 0x561a1f6329a0 2024-09-19T09:17:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.130+0000 7f00da7ed740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561a1f51f290 con 0x561a1f6332e0 2024-09-19T09:17:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.130+0000 7f00da7ed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561a1f63b460 con 0x561a1f638370 2024-09-19T09:17:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.130+0000 7f00da7ed740 10 client.0 _ll_register_callbacks cb 0x561a1f550140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:17:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.130+0000 7f00d57f8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a1f6329a0 0x561a1f63cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.130+0000 7f00d4ff7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1f6332e0 0x561a1f63d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.130+0000 7f00d57f8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a1f6329a0 0x561a1f63cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51442/0 (socket says 172.21.15.120:51442) 2024-09-19T09:17:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.130+0000 7f00d57f8640 1 -- 172.21.15.120:0/1207139298 learned_addr learned my addr 172.21.15.120:0/1207139298 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.130+0000 7f00d5ff9640 1 --2- 172.21.15.120:0/1207139298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a1f638370 0x561a1f6440c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.130+0000 7f00d57f8640 1 -- 172.21.15.120:0/1207139298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a1f638370 msgr2=0x561a1f6440c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.130+0000 7f00d57f8640 1 --2- 172.21.15.120:0/1207139298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a1f638370 0x561a1f6440c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.130+0000 7f00d57f8640 1 -- 172.21.15.120:0/1207139298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1f6332e0 msgr2=0x561a1f63d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.130+0000 7f00d57f8640 1 --2- 172.21.15.120:0/1207139298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1f6332e0 0x561a1f63d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.130+0000 7f00d57f8640 1 -- 172.21.15.120:0/1207139298 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a1f2a9210 con 0x561a1f6329a0 2024-09-19T09:17:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.130+0000 7f00d4ff7640 1 --2- 172.21.15.120:0/1207139298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1f6332e0 0x561a1f63d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:17:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.131+0000 7f00d57f8640 1 --2- 172.21.15.120:0/1207139298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a1f6329a0 0x561a1f63cb40 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f00c0000c00 tx=0x7f00c0009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f13503bdda54d77 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.131+0000 7f00b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:17:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.131+0000 7f00b8ff9640 1 -- 172.21.15.120:0/1207139298 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0037050 con 0x561a1f6329a0 2024-09-19T09:17:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.131+0000 7f00b8ff9640 1 -- 172.21.15.120:0/1207139298 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f00c0030650 con 0x561a1f6329a0 2024-09-19T09:17:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.131+0000 7f00da7ed740 1 -- 172.21.15.120:0/1207139298 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00c0004030 con 0x561a1f6329a0 2024-09-19T09:17:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.131+0000 7f00b8ff9640 1 -- 172.21.15.120:0/1207139298 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f00c0030850 con 0x561a1f6329a0 2024-09-19T09:17:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.131+0000 7f00da7ed740 10 client.20523 fetch_fsmap learned FSMap version 36 2024-09-19T09:17:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.131+0000 7f00da7ed740 1 -- 172.21.15.120:0/1207139298 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561a1f375620 con 0x561a1f6329a0 2024-09-19T09:17:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.132+0000 7f00b8ff9640 1 -- 172.21.15.120:0/1207139298 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00c0004030 con 0x561a1f6329a0 2024-09-19T09:17:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.132+0000 7f00b8ff9640 1 -- 172.21.15.120:0/1207139298 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f00c0004860 con 0x561a1f6329a0 2024-09-19T09:17:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.132+0000 7f00da7ed740 10 client.20523 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:17:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.132+0000 7f00da7ed740 10 client.20523 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:17:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.132+0000 7f00da7ed740 20 client.20523 populate_metadata read hostname 'smithi120' 2024-09-19T09:17:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.132+0000 7f00da7ed740 1 --2- 172.21.15.120:0/1207139298 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561a1f6483c0 0x561a1f64a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.132+0000 7f00da7ed740 4 client.20523 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:17:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.132+0000 7f00da7ed740 1 -- 172.21.15.120:0/1207139298 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a1f5a3a20 con 0x561a1f6483c0 2024-09-19T09:17:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.132+0000 7f00d4ff7640 1 --2- 172.21.15.120:0/1207139298 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561a1f6483c0 0x561a1f64a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.133+0000 7f00d4ff7640 1 --2- 172.21.15.120:0/1207139298 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561a1f6483c0 0x561a1f64a7b0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=935a86476ae3c9dc server_cookie=1c827abf0287971b in_seq=0 out_seq=0 2024-09-19T09:17:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.133+0000 7f00b8ff9640 10 client.20523 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.134+0000 7f00b8ff9640 1 -- 172.21.15.120:0/1207139298 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561a1f5a3a20 con 0x561a1f6483c0 2024-09-19T09:17:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.134+0000 7f00b8ff9640 10 client.20523 handle_command_reply: tid=0 2024-09-19T09:17:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.134+0000 7f00da7ed740 1 client.20523 shutdown 2024-09-19T09:17:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.134+0000 7f00da7ed740 1 -- 172.21.15.120:0/1207139298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a1f6329a0 msgr2=0x561a1f63cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.134+0000 7f00da7ed740 1 --2- 172.21.15.120:0/1207139298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a1f6329a0 0x561a1f63cb40 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f00c0000c00 tx=0x7f00c0009490 comp rx=0 tx=0).stop 2024-09-19T09:17:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.134+0000 7f00da7ed740 1 -- 172.21.15.120:0/1207139298 shutdown_connections 2024-09-19T09:17:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.134+0000 7f00da7ed740 1 --2- 172.21.15.120:0/1207139298 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561a1f6483c0 0x561a1f64a7b0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.134+0000 7f00b8ff9640 0 client.20523 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.134+0000 7f00da7ed740 1 --2- 172.21.15.120:0/1207139298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a1f638370 0x561a1f6440c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.134+0000 7f00da7ed740 1 --2- 172.21.15.120:0/1207139298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1f6332e0 0x561a1f63d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.134+0000 7f00da7ed740 1 --2- 172.21.15.120:0/1207139298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a1f6329a0 0x561a1f63cb40 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.134+0000 7f00da7ed740 1 -- 172.21.15.120:0/1207139298 >> 172.21.15.120:0/1207139298 conn(0x561a1f62e010 msgr2=0x561a1f62f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.134+0000 7f00da7ed740 1 -- 172.21.15.120:0/1207139298 shutdown_connections 2024-09-19T09:17:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.134+0000 7f00da7ed740 1 -- 172.21.15.120:0/1207139298 wait complete. 2024-09-19T09:17:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.134+0000 7f00da7ed740 20 client.20523 trim_cache size 0 max 16384 2024-09-19T09:17:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.150+0000 7f00da7ed740 1 Processor -- start 2024-09-19T09:17:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.150+0000 7f00da7ed740 1 -- start start 2024-09-19T09:17:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.150+0000 7f00da7ed740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1f6329a0 0x561a1f63c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.150+0000 7f00da7ed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a1f6332e0 0x561a1f644050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.150+0000 7f00da7ed740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a1f638370 0x561a1f62d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.150+0000 7f00da7ed740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f00d01a6db0 con 0x561a1f638370 2024-09-19T09:17:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.150+0000 7f00da7ed740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f00d01a8980 con 0x561a1f6329a0 2024-09-19T09:17:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.150+0000 7f00da7ed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f00d01a7480 con 0x561a1f6332e0 2024-09-19T09:17:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.150+0000 7f00d5ff9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a1f638370 0x561a1f62d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.150+0000 7f00d4ff7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a1f6332e0 0x561a1f644050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.150+0000 7f00d5ff9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a1f638370 0x561a1f62d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51458/0 (socket says 172.21.15.120:51458) 2024-09-19T09:17:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.150+0000 7f00d57f8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1f6329a0 0x561a1f63c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.150+0000 7f00d5ff9640 1 -- 172.21.15.120:0/953420268 learned_addr learned my addr 172.21.15.120:0/953420268 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.151+0000 7f00d5ff9640 1 -- 172.21.15.120:0/953420268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a1f6332e0 msgr2=0x561a1f644050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.151+0000 7f00d5ff9640 1 --2- 172.21.15.120:0/953420268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a1f6332e0 0x561a1f644050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.151+0000 7f00d5ff9640 1 -- 172.21.15.120:0/953420268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1f6329a0 msgr2=0x561a1f63c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.151+0000 7f00d5ff9640 1 --2- 172.21.15.120:0/953420268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1f6329a0 0x561a1f63c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.151+0000 7f00d5ff9640 1 -- 172.21.15.120:0/953420268 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a1f64ab20 con 0x561a1f638370 2024-09-19T09:17:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.151+0000 7f00d5ff9640 1 --2- 172.21.15.120:0/953420268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a1f638370 0x561a1f62d0b0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f00c801e6f0 tx=0x7f00c8005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4b8037a6712cd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.151+0000 7f00baffd640 1 -- 172.21.15.120:0/953420268 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8019070 con 0x561a1f638370 2024-09-19T09:17:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.151+0000 7f00baffd640 1 -- 172.21.15.120:0/953420268 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00c80137c0 con 0x561a1f638370 2024-09-19T09:17:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.151+0000 7f00baffd640 1 -- 172.21.15.120:0/953420268 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8005600 con 0x561a1f638370 2024-09-19T09:17:39.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.152+0000 7f00da7ed740 1 -- 172.21.15.120:0/953420268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a1f638370 msgr2=0x561a1f62d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:39.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.152+0000 7f00da7ed740 1 --2- 172.21.15.120:0/953420268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a1f638370 0x561a1f62d0b0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f00c801e6f0 tx=0x7f00c8005bd0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.152+0000 7f00da7ed740 1 -- 172.21.15.120:0/953420268 shutdown_connections 2024-09-19T09:17:39.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.152+0000 7f00da7ed740 1 --2- 172.21.15.120:0/953420268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a1f638370 0x561a1f62d0b0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.152+0000 7f00da7ed740 1 --2- 172.21.15.120:0/953420268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a1f6332e0 0x561a1f644050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.152+0000 7f00da7ed740 1 --2- 172.21.15.120:0/953420268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1f6329a0 0x561a1f63c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.152+0000 7f00da7ed740 1 -- 172.21.15.120:0/953420268 >> 172.21.15.120:0/953420268 conn(0x561a1f62e010 msgr2=0x561a1f62c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:39.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.152+0000 7f00da7ed740 1 -- 172.21.15.120:0/953420268 shutdown_connections 2024-09-19T09:17:39.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.152+0000 7f00da7ed740 1 -- 172.21.15.120:0/953420268 wait complete. 2024-09-19T09:17:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.152+0000 7f00da7ed740 1 -- start start 2024-09-19T09:17:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.152+0000 7f00da7ed740 1 Processor -- start 2024-09-19T09:17:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.152+0000 7f00da7ed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a1f6329a0 0x561a1f641010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.152+0000 7f00da7ed740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1f6332e0 0x561a1f641550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.153+0000 7f00da7ed740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a1f638370 0x561a1f6b1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.153+0000 7f00da7ed740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f00d01a82b0 con 0x561a1f638370 2024-09-19T09:17:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.153+0000 7f00da7ed740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f00d01a7b50 con 0x561a1f6332e0 2024-09-19T09:17:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.153+0000 7f00da7ed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f00d01a4d70 con 0x561a1f6329a0 2024-09-19T09:17:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.153+0000 7f00da7ed740 10 client.0 _ll_register_callbacks cb 0x7f00cc097ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:17:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.153+0000 7f00d5ff9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a1f638370 0x561a1f6b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.153+0000 7f00d5ff9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a1f638370 0x561a1f6b1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51466/0 (socket says 172.21.15.120:51466) 2024-09-19T09:17:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.153+0000 7f00d5ff9640 1 -- 172.21.15.120:0/2696773456 learned_addr learned my addr 172.21.15.120:0/2696773456 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.153+0000 7f00d4ff7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1f6332e0 0x561a1f641550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.153+0000 7f00d57f8640 1 --2- 172.21.15.120:0/2696773456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a1f6329a0 0x561a1f641010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:39.159 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.153+0000 7f00d5ff9640 1 -- 172.21.15.120:0/2696773456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a1f6329a0 msgr2=0x561a1f641010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:39.159 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.153+0000 7f00d5ff9640 1 --2- 172.21.15.120:0/2696773456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a1f6329a0 0x561a1f641010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.159 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.153+0000 7f00d5ff9640 1 -- 172.21.15.120:0/2696773456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1f6332e0 msgr2=0x561a1f641550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:39.159 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.153+0000 7f00d5ff9640 1 --2- 172.21.15.120:0/2696773456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1f6332e0 0x561a1f641550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.159 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.153+0000 7f00d5ff9640 1 -- 172.21.15.120:0/2696773456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a1f6b15c0 con 0x561a1f638370 2024-09-19T09:17:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.153+0000 7f00d5ff9640 1 --2- 172.21.15.120:0/2696773456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a1f638370 0x561a1f6b1080 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f00c8004a80 tx=0x7f00c8013ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1523fb2685eba3f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.153+0000 7f00b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:17:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.153+0000 7f00b8ff9640 1 -- 172.21.15.120:0/2696773456 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8037060 con 0x561a1f638370 2024-09-19T09:17:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.153+0000 7f00b8ff9640 1 -- 172.21.15.120:0/2696773456 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f00c8005300 con 0x561a1f638370 2024-09-19T09:17:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.153+0000 7f00da7ed740 1 -- 172.21.15.120:0/2696773456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00c80137c0 con 0x561a1f638370 2024-09-19T09:17:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.153+0000 7f00b8ff9640 1 -- 172.21.15.120:0/2696773456 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f00c8019040 con 0x561a1f638370 2024-09-19T09:17:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.153+0000 7f00da7ed740 10 client.20529 fetch_fsmap learned FSMap version 36 2024-09-19T09:17:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.153+0000 7f00da7ed740 1 -- 172.21.15.120:0/2696773456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561a1f6b2880 con 0x561a1f638370 2024-09-19T09:17:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.154+0000 7f00b8ff9640 1 -- 172.21.15.120:0/2696773456 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00c80137c0 con 0x561a1f638370 2024-09-19T09:17:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.154+0000 7f00b8ff9640 1 -- 172.21.15.120:0/2696773456 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f00c803c070 con 0x561a1f638370 2024-09-19T09:17:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.154+0000 7f00da7ed740 10 client.20529 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:17:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.154+0000 7f00da7ed740 10 client.20529 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:17:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.154+0000 7f00da7ed740 20 client.20529 populate_metadata read hostname 'smithi120' 2024-09-19T09:17:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.154+0000 7f00da7ed740 1 --2- 172.21.15.120:0/2696773456 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561a1f6483c0 0x561a1f6b5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.154+0000 7f00da7ed740 4 client.20529 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:17:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.154+0000 7f00da7ed740 1 -- 172.21.15.120:0/2696773456 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561a1f59dd70 con 0x561a1f6483c0 2024-09-19T09:17:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.154+0000 7f00d57f8640 1 --2- 172.21.15.120:0/2696773456 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561a1f6483c0 0x561a1f6b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.154+0000 7f00b8ff9640 10 client.20529 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.154+0000 7f00d57f8640 1 --2- 172.21.15.120:0/2696773456 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561a1f6483c0 0x561a1f6b5220 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7b0cd6192370c4b server_cookie=2bd91dc646ff9d82 in_seq=0 out_seq=0 2024-09-19T09:17:39.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:38 smithi120 ceph-mon[31150]: pgmap v788: 97 pgs: 97 active+clean; 297 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 9.6 MiB/s wr, 905 op/s 2024-09-19T09:17:39.355 INFO:tasks.workunit.client.0.smithi120.stdout: 10 44397 27.27 MB/sec warmup 94 sec latency 126.463 ms 2024-09-19T09:17:39.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:38 smithi196 ceph-mon[35150]: pgmap v788: 97 pgs: 97 active+clean; 297 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 9.6 MiB/s wr, 905 op/s 2024-09-19T09:17:39.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:38 smithi153 ceph-mon[35071]: pgmap v788: 97 pgs: 97 active+clean; 297 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 9.6 MiB/s wr, 905 op/s 2024-09-19T09:17:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.572+0000 7f00b8ff9640 1 -- 172.21.15.120:0/2696773456 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561a1f59dd70 con 0x561a1f6483c0 2024-09-19T09:17:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.572+0000 7f00b8ff9640 10 client.20529 handle_command_reply: tid=0 2024-09-19T09:17:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.572+0000 7f00da7ed740 1 client.20529 shutdown 2024-09-19T09:17:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.572+0000 7f00da7ed740 1 -- 172.21.15.120:0/2696773456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a1f638370 msgr2=0x561a1f6b1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.572+0000 7f00da7ed740 1 --2- 172.21.15.120:0/2696773456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a1f638370 0x561a1f6b1080 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f00c8004a80 tx=0x7f00c8013ba0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.572+0000 7f00da7ed740 1 -- 172.21.15.120:0/2696773456 shutdown_connections 2024-09-19T09:17:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.572+0000 7f00da7ed740 1 --2- 172.21.15.120:0/2696773456 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561a1f6483c0 0x561a1f6b5220 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.572+0000 7f00da7ed740 1 --2- 172.21.15.120:0/2696773456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a1f638370 0x561a1f6b1080 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.572+0000 7f00da7ed740 1 --2- 172.21.15.120:0/2696773456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a1f6332e0 0x561a1f641550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.572+0000 7f00da7ed740 1 --2- 172.21.15.120:0/2696773456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a1f6329a0 0x561a1f641010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.572+0000 7f00da7ed740 1 -- 172.21.15.120:0/2696773456 >> 172.21.15.120:0/2696773456 conn(0x561a1f62e010 msgr2=0x561a1f62c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.572+0000 7f00b8ff9640 0 client.20529 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.572+0000 7f00da7ed740 1 -- 172.21.15.120:0/2696773456 shutdown_connections 2024-09-19T09:17:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.572+0000 7f00da7ed740 1 -- 172.21.15.120:0/2696773456 wait complete. 2024-09-19T09:17:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.572+0000 7f00da7ed740 20 client.20529 trim_cache size 0 max 16384 2024-09-19T09:17:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:17:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:17:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:17:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:17:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.573+0000 7f00d7a5c640 1 -- 172.21.15.120:0/1063057560 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f00bc0a32f0 msgr2=0x7f00bc0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.573+0000 7f00d7a5c640 1 --2- 172.21.15.120:0/1063057560 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f00bc0a32f0 0x7f00bc0a57b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f00c8005c40 tx=0x7f00c8004ab0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.573+0000 7f00d7a5c640 1 -- 172.21.15.120:0/1063057560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00d0102d50 msgr2=0x7f00d0198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.573+0000 7f00d7a5c640 1 --2- 172.21.15.120:0/1063057560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00d0102d50 0x7f00d0198340 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00e9c0 tx=0x7f00cc00ee80 comp rx=0 tx=0).stop 2024-09-19T09:17:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.573+0000 7f00d7a5c640 1 -- 172.21.15.120:0/1063057560 shutdown_connections 2024-09-19T09:17:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.573+0000 7f00d7a5c640 1 --2- 172.21.15.120:0/1063057560 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f00bc0a32f0 0x7f00bc0a57b0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.573+0000 7f00d7a5c640 1 --2- 172.21.15.120:0/1063057560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00d0103710 0x7f00d019f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.573+0000 7f00d7a5c640 1 --2- 172.21.15.120:0/1063057560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00d0102d50 0x7f00d0198340 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.573+0000 7f00d7a5c640 1 --2- 172.21.15.120:0/1063057560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00d0101b60 0x7f00d0197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.573+0000 7f00d7a5c640 1 -- 172.21.15.120:0/1063057560 >> 172.21.15.120:0/1063057560 conn(0x7f00d00fd2e0 msgr2=0x7f00d0106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.573+0000 7f00d7a5c640 1 -- 172.21.15.120:0/1063057560 shutdown_connections 2024-09-19T09:17:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:39.573+0000 7f00d7a5c640 1 -- 172.21.15.120:0/1063057560 wait complete. 2024-09-19T09:17:40.355 INFO:tasks.workunit.client.0.smithi120.stdout: 10 45136 27.35 MB/sec warmup 95 sec latency 270.733 ms 2024-09-19T09:17:41.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:40 smithi120 ceph-mon[31150]: pgmap v789: 97 pgs: 97 active+clean; 298 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.6 MiB/s wr, 747 op/s 2024-09-19T09:17:41.355 INFO:tasks.workunit.client.0.smithi120.stdout: 10 45926 27.54 MB/sec warmup 96 sec latency 59.207 ms 2024-09-19T09:17:41.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:40 smithi196 ceph-mon[35150]: pgmap v789: 97 pgs: 97 active+clean; 298 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.6 MiB/s wr, 747 op/s 2024-09-19T09:17:41.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:40 smithi153 ceph-mon[35071]: pgmap v789: 97 pgs: 97 active+clean; 298 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.6 MiB/s wr, 747 op/s 2024-09-19T09:17:42.355 INFO:tasks.workunit.client.0.smithi120.stdout: 10 46566 27.76 MB/sec warmup 97 sec latency 93.944 ms 2024-09-19T09:17:43.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:42 smithi120 ceph-mon[31150]: pgmap v790: 97 pgs: 97 active+clean; 301 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.8 MiB/s wr, 771 op/s 2024-09-19T09:17:43.355 INFO:tasks.workunit.client.0.smithi120.stdout: 10 47037 27.69 MB/sec warmup 98 sec latency 327.262 ms 2024-09-19T09:17:43.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:42 smithi196 ceph-mon[35150]: pgmap v790: 97 pgs: 97 active+clean; 301 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.8 MiB/s wr, 771 op/s 2024-09-19T09:17:43.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:42 smithi153 ceph-mon[35071]: pgmap v790: 97 pgs: 97 active+clean; 301 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.8 MiB/s wr, 771 op/s 2024-09-19T09:17:44.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:43 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:17:44.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:43 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:17:44.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:43 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:17:44.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:43 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:17:44.355 INFO:tasks.workunit.client.0.smithi120.stdout: 10 47495 27.63 MB/sec warmup 99 sec latency 89.169 ms 2024-09-19T09:17:44.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:43 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:17:44.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:43 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:17:44.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:43 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:17:44.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:43 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:17:44.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:43 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:17:44.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:43 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:17:44.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:43 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:17:44.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:43 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:17:45.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:44 smithi120 ceph-mon[31150]: pgmap v791: 97 pgs: 97 active+clean; 301 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.8 MiB/s wr, 836 op/s 2024-09-19T09:17:45.360 INFO:tasks.workunit.client.0.smithi120.stdout: 10 48108 27.67 MB/sec warmup 100 sec latency 112.087 ms 2024-09-19T09:17:45.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:44 smithi196 ceph-mon[35150]: pgmap v791: 97 pgs: 97 active+clean; 301 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.8 MiB/s wr, 836 op/s 2024-09-19T09:17:45.443 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:44 smithi153 ceph-mon[35071]: pgmap v791: 97 pgs: 97 active+clean; 301 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.8 MiB/s wr, 836 op/s 2024-09-19T09:17:46.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:45 smithi120 ceph-mon[31150]: pgmap v792: 97 pgs: 97 active+clean; 305 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.3 MiB/s wr, 787 op/s 2024-09-19T09:17:46.356 INFO:tasks.workunit.client.0.smithi120.stdout: 10 48796 27.78 MB/sec warmup 101 sec latency 41.656 ms 2024-09-19T09:17:46.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:45 smithi196 ceph-mon[35150]: pgmap v792: 97 pgs: 97 active+clean; 305 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.3 MiB/s wr, 787 op/s 2024-09-19T09:17:46.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:45 smithi153 ceph-mon[35071]: pgmap v792: 97 pgs: 97 active+clean; 305 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.3 MiB/s wr, 787 op/s 2024-09-19T09:17:47.355 INFO:tasks.workunit.client.0.smithi120.stdout: 10 49590 27.83 MB/sec warmup 102 sec latency 82.867 ms 2024-09-19T09:17:48.355 INFO:tasks.workunit.client.0.smithi120.stdout: 10 50073 27.88 MB/sec warmup 103 sec latency 148.054 ms 2024-09-19T09:17:49.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:48 smithi196 ceph-mon[35150]: pgmap v793: 97 pgs: 97 active+clean; 311 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.1 MiB/s wr, 814 op/s 2024-09-19T09:17:49.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:48 smithi153 ceph-mon[35071]: pgmap v793: 97 pgs: 97 active+clean; 311 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.1 MiB/s wr, 814 op/s 2024-09-19T09:17:49.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:48 smithi120 ceph-mon[31150]: pgmap v793: 97 pgs: 97 active+clean; 311 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.1 MiB/s wr, 814 op/s 2024-09-19T09:17:49.356 INFO:tasks.workunit.client.0.smithi120.stdout: 10 50795 28.07 MB/sec warmup 104 sec latency 60.988 ms 2024-09-19T09:17:50.355 INFO:tasks.workunit.client.0.smithi120.stdout: 10 51406 28.09 MB/sec warmup 105 sec latency 55.267 ms 2024-09-19T09:17:51.186 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:50 smithi196 ceph-mon[35150]: pgmap v794: 97 pgs: 97 active+clean; 308 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.9 MiB/s wr, 783 op/s 2024-09-19T09:17:51.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:50 smithi153 ceph-mon[35071]: pgmap v794: 97 pgs: 97 active+clean; 308 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.9 MiB/s wr, 783 op/s 2024-09-19T09:17:51.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:50 smithi120 ceph-mon[31150]: pgmap v794: 97 pgs: 97 active+clean; 308 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.9 MiB/s wr, 783 op/s 2024-09-19T09:17:51.355 INFO:tasks.workunit.client.0.smithi120.stdout: 10 51937 28.07 MB/sec warmup 106 sec latency 92.647 ms 2024-09-19T09:17:52.355 INFO:tasks.workunit.client.0.smithi120.stdout: 10 52519 28.06 MB/sec warmup 107 sec latency 125.228 ms 2024-09-19T09:17:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.685+0000 7f3de65b9640 1 -- 172.21.15.120:0/384804160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3de0104f00 msgr2=0x7f3de01072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.685+0000 7f3de65b9640 1 --2- 172.21.15.120:0/384804160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3de0104f00 0x7f3de01072f0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4009900 tx=0x7f3dd402a410 comp rx=0 tx=0).stop 2024-09-19T09:17:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.685+0000 7f3de65b9640 1 -- 172.21.15.120:0/384804160 shutdown_connections 2024-09-19T09:17:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.685+0000 7f3de65b9640 1 --2- 172.21.15.120:0/384804160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3de0107830 0x7f3de0109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.685+0000 7f3de65b9640 1 --2- 172.21.15.120:0/384804160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3de0104f00 0x7f3de01072f0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.685+0000 7f3de65b9640 1 --2- 172.21.15.120:0/384804160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3de01025d0 0x7f3de01049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.685+0000 7f3de65b9640 1 -- 172.21.15.120:0/384804160 >> 172.21.15.120:0/384804160 conn(0x7f3de00fc1b0 msgr2=0x7f3de00fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.685+0000 7f3de65b9640 1 -- 172.21.15.120:0/384804160 shutdown_connections 2024-09-19T09:17:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.685+0000 7f3de65b9640 1 -- 172.21.15.120:0/384804160 wait complete. 2024-09-19T09:17:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.685+0000 7f3de65b9640 1 Processor -- start 2024-09-19T09:17:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.686+0000 7f3de65b9640 1 -- start start 2024-09-19T09:17:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.686+0000 7f3de65b9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3de01025d0 0x7f3de0197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.686+0000 7f3de65b9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3de0104f00 0x7f3de0198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.686+0000 7f3de65b9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3de0107830 0x7f3de019f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.686+0000 7f3de65b9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3de010d810 con 0x7f3de01025d0 2024-09-19T09:17:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.686+0000 7f3de65b9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3de010d690 con 0x7f3de0104f00 2024-09-19T09:17:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.686+0000 7f3de65b9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3de010d990 con 0x7f3de0107830 2024-09-19T09:17:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.686+0000 7f3ddffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3de01025d0 0x7f3de0197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.686+0000 7f3ddffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3de01025d0 0x7f3de0197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39034/0 (socket says 172.21.15.120:39034) 2024-09-19T09:17:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.686+0000 7f3ddffff640 1 -- 172.21.15.120:0/1070179617 learned_addr learned my addr 172.21.15.120:0/1070179617 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.686+0000 7f3ddf7fe640 1 --2- 172.21.15.120:0/1070179617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3de0104f00 0x7f3de0198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.686+0000 7f3de4b56640 1 --2- 172.21.15.120:0/1070179617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3de0107830 0x7f3de019f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.686+0000 7f3ddffff640 1 -- 172.21.15.120:0/1070179617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3de0107830 msgr2=0x7f3de019f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.686+0000 7f3ddffff640 1 --2- 172.21.15.120:0/1070179617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3de0107830 0x7f3de019f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.686+0000 7f3ddffff640 1 -- 172.21.15.120:0/1070179617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3de0104f00 msgr2=0x7f3de0198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.686+0000 7f3ddffff640 1 --2- 172.21.15.120:0/1070179617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3de0104f00 0x7f3de0198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.686+0000 7f3ddffff640 1 -- 172.21.15.120:0/1070179617 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de019fa40 con 0x7f3de01025d0 2024-09-19T09:17:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.687+0000 7f3ddffff640 1 --2- 172.21.15.120:0/1070179617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3de01025d0 0x7f3de0197e20 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f3dd000e9f0 tx=0x7f3dd000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.687+0000 7f3ddd7fa640 1 -- 172.21.15.120:0/1070179617 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd000cd80 con 0x7f3de01025d0 2024-09-19T09:17:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.687+0000 7f3ddd7fa640 1 -- 172.21.15.120:0/1070179617 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dd0010910 con 0x7f3de01025d0 2024-09-19T09:17:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.687+0000 7f3de65b9640 1 -- 172.21.15.120:0/1070179617 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3de019fd30 con 0x7f3de01025d0 2024-09-19T09:17:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.687+0000 7f3ddd7fa640 1 -- 172.21.15.120:0/1070179617 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0018990 con 0x7f3de01025d0 2024-09-19T09:17:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.687+0000 7f3de65b9640 1 -- 172.21.15.120:0/1070179617 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3de01a01a0 con 0x7f3de01025d0 2024-09-19T09:17:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.689+0000 7f3ddd7fa640 1 -- 172.21.15.120:0/1070179617 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f3dd0010ac0 con 0x7f3de01025d0 2024-09-19T09:17:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.689+0000 7f3de9426740 1 Processor -- start 2024-09-19T09:17:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.689+0000 7f3de9426740 1 -- start start 2024-09-19T09:17:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.689+0000 7f3de9426740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561aa4c1a9a0 0x561aa4c1ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.689+0000 7f3de9426740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aa4c1b2e0 0x561aa4c1fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.689+0000 7f3de9426740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561aa4c20370 0x561aa4c22830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.689+0000 7f3de9426740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561aa4b7e2f0 con 0x561aa4c20370 2024-09-19T09:17:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.689+0000 7f3de9426740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561aa4b26270 con 0x561aa4c1b2e0 2024-09-19T09:17:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.689+0000 7f3de9426740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561aa4b7a2b0 con 0x561aa4c1a9a0 2024-09-19T09:17:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.690+0000 7f3ddffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aa4c1b2e0 0x561aa4c1fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.690+0000 7f3ddf7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561aa4c20370 0x561aa4c22830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.690+0000 7f3ddffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aa4c1b2e0 0x561aa4c1fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:48066/0 (socket says 172.21.15.120:48066) 2024-09-19T09:17:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.690+0000 7f3ddffff640 1 -- 172.21.15.120:0/744036136 learned_addr learned my addr 172.21.15.120:0/744036136 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.690+0000 7f3ddf7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561aa4c20370 0x561aa4c22830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39036/0 (socket says 172.21.15.120:39036) 2024-09-19T09:17:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.690+0000 7f3ddd7fa640 1 --2- 172.21.15.120:0/1070179617 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3dc80a32f0 0x7f3dc80a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.690+0000 7f3de4b56640 1 --2- 172.21.15.120:0/744036136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561aa4c1a9a0 0x561aa4c1ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.690+0000 7f3ddd7fa640 1 -- 172.21.15.120:0/1070179617 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f3dd00cfc20 con 0x7f3de01025d0 2024-09-19T09:17:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.690+0000 7f3ddf7fe640 1 -- 172.21.15.120:0/744036136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561aa4c1a9a0 msgr2=0x561aa4c1ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.690+0000 7f3ddf7fe640 1 --2- 172.21.15.120:0/744036136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561aa4c1a9a0 0x561aa4c1ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.690+0000 7f3ddf7fe640 1 -- 172.21.15.120:0/744036136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aa4c1b2e0 msgr2=0x561aa4c1fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.690+0000 7f3ddf7fe640 1 --2- 172.21.15.120:0/744036136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aa4c1b2e0 0x561aa4c1fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.690+0000 7f3ddf7fe640 1 -- 172.21.15.120:0/744036136 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aa4b25340 con 0x561aa4c20370 2024-09-19T09:17:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.690+0000 7f3de4b56640 1 --2- 172.21.15.120:0/744036136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561aa4c1a9a0 0x561aa4c1ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:17:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.690+0000 7f3ddffff640 1 --2- 172.21.15.120:0/744036136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aa4c1b2e0 0x561aa4c1fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:17:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.690+0000 7f3de4b56640 1 --2- 172.21.15.120:0/1070179617 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3dc80a32f0 0x7f3dc80a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.690+0000 7f3ddf7fe640 1 --2- 172.21.15.120:0/744036136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561aa4c20370 0x561aa4c22830 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4009230 tx=0x7f3dd4005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330fb3844b67697a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.690+0000 7f3dc67fc640 1 -- 172.21.15.120:0/744036136 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4035040 con 0x561aa4c20370 2024-09-19T09:17:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.690+0000 7f3dc67fc640 1 -- 172.21.15.120:0/744036136 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dd4041b60 con 0x561aa4c20370 2024-09-19T09:17:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.690+0000 7f3de4b56640 1 --2- 172.21.15.120:0/1070179617 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3dc80a32f0 0x7f3dc80a57b0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f3de006b380 tx=0x7f3dd8014040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.690+0000 7f3dc67fc640 1 -- 172.21.15.120:0/744036136 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd40052a0 con 0x561aa4c20370 2024-09-19T09:17:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.691+0000 7f3de9426740 1 -- 172.21.15.120:0/744036136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561aa4c20370 msgr2=0x561aa4c22830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.691+0000 7f3de9426740 1 --2- 172.21.15.120:0/744036136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561aa4c20370 0x561aa4c22830 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4009230 tx=0x7f3dd4005e20 comp rx=0 tx=0).stop 2024-09-19T09:17:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.691+0000 7f3de9426740 1 -- 172.21.15.120:0/744036136 shutdown_connections 2024-09-19T09:17:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.691+0000 7f3de9426740 1 --2- 172.21.15.120:0/744036136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561aa4c20370 0x561aa4c22830 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.691+0000 7f3de9426740 1 --2- 172.21.15.120:0/744036136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aa4c1b2e0 0x561aa4c1fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.691+0000 7f3de9426740 1 --2- 172.21.15.120:0/744036136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561aa4c1a9a0 0x561aa4c1ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.691+0000 7f3de9426740 1 -- 172.21.15.120:0/744036136 >> 172.21.15.120:0/744036136 conn(0x561aa4c16010 msgr2=0x561aa4c16410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.691+0000 7f3de9426740 1 -- 172.21.15.120:0/744036136 shutdown_connections 2024-09-19T09:17:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.691+0000 7f3de9426740 1 -- 172.21.15.120:0/744036136 wait complete. 2024-09-19T09:17:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.691+0000 7f3de9426740 1 -- start start 2024-09-19T09:17:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.692+0000 7f3de9426740 1 Processor -- start 2024-09-19T09:17:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.692+0000 7f3de9426740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561aa4c1a9a0 0x561aa4c24b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.692+0000 7f3de9426740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aa4c1b2e0 0x561aa4c250d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.692+0000 7f3de9426740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561aa4c20370 0x561aa4c2c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.692+0000 7f3de9426740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561aa4c232e0 con 0x561aa4c20370 2024-09-19T09:17:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.692+0000 7f3de9426740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561aa4b07290 con 0x561aa4c1b2e0 2024-09-19T09:17:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.692+0000 7f3de9426740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561aa4c23460 con 0x561aa4c1a9a0 2024-09-19T09:17:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.692+0000 7f3de9426740 10 client.0 _ll_register_callbacks cb 0x561aa4b38140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:17:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.692+0000 7f3de4b56640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561aa4c20370 0x561aa4c2c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.692+0000 7f3de4b56640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561aa4c20370 0x561aa4c2c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39052/0 (socket says 172.21.15.120:39052) 2024-09-19T09:17:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.692+0000 7f3ddffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561aa4c1a9a0 0x561aa4c24b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.692+0000 7f3de4b56640 1 -- 172.21.15.120:0/1526805781 learned_addr learned my addr 172.21.15.120:0/1526805781 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.692+0000 7f3ddf7fe640 1 --2- 172.21.15.120:0/1526805781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aa4c1b2e0 0x561aa4c250d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.692+0000 7f3de4b56640 1 -- 172.21.15.120:0/1526805781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561aa4c1a9a0 msgr2=0x561aa4c24b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.692+0000 7f3de4b56640 1 --2- 172.21.15.120:0/1526805781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561aa4c1a9a0 0x561aa4c24b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.692+0000 7f3de4b56640 1 -- 172.21.15.120:0/1526805781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aa4c1b2e0 msgr2=0x561aa4c250d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.692+0000 7f3de4b56640 1 --2- 172.21.15.120:0/1526805781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aa4c1b2e0 0x561aa4c250d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.692+0000 7f3de4b56640 1 -- 172.21.15.120:0/1526805781 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561aa4891210 con 0x561aa4c20370 2024-09-19T09:17:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.692+0000 7f3ddffff640 1 --2- 172.21.15.120:0/1526805781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561aa4c1a9a0 0x561aa4c24b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:17:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.693+0000 7f3de4b56640 1 --2- 172.21.15.120:0/1526805781 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561aa4c20370 0x561aa4c2c110 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8010bd0 tx=0x7f3dd8009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc860a925630db3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.693+0000 7f3dbf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:17:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.693+0000 7f3dbf7fe640 1 -- 172.21.15.120:0/1526805781 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd801eba0 con 0x561aa4c20370 2024-09-19T09:17:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.693+0000 7f3dbf7fe640 1 -- 172.21.15.120:0/1526805781 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f3dd801ed40 con 0x561aa4c20370 2024-09-19T09:17:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.693+0000 7f3de9426740 1 -- 172.21.15.120:0/1526805781 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dd4041b60 con 0x561aa4c20370 2024-09-19T09:17:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.693+0000 7f3dbf7fe640 1 -- 172.21.15.120:0/1526805781 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3dd801c480 con 0x561aa4c20370 2024-09-19T09:17:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.693+0000 7f3de9426740 10 client.20541 fetch_fsmap learned FSMap version 36 2024-09-19T09:17:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.693+0000 7f3de9426740 1 -- 172.21.15.120:0/1526805781 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561aa495d620 con 0x561aa4c20370 2024-09-19T09:17:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.693+0000 7f3dbf7fe640 1 -- 172.21.15.120:0/1526805781 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dd4041b60 con 0x561aa4c20370 2024-09-19T09:17:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.693+0000 7f3dbf7fe640 1 -- 172.21.15.120:0/1526805781 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f3dd8025070 con 0x561aa4c20370 2024-09-19T09:17:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.693+0000 7f3de9426740 10 client.20541 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:17:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.693+0000 7f3de9426740 10 client.20541 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:17:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.693+0000 7f3de9426740 20 client.20541 populate_metadata read hostname 'smithi120' 2024-09-19T09:17:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.693+0000 7f3de9426740 1 --2- 172.21.15.120:0/1526805781 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561aa4c30940 0x561aa4c32d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.693+0000 7f3de9426740 4 client.20541 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:17:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.693+0000 7f3de9426740 1 -- 172.21.15.120:0/1526805781 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561aa4b8ba20 con 0x561aa4c30940 2024-09-19T09:17:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.694+0000 7f3ddffff640 1 --2- 172.21.15.120:0/1526805781 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561aa4c30940 0x561aa4c32d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.694+0000 7f3dbf7fe640 10 client.20541 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.694+0000 7f3ddffff640 1 --2- 172.21.15.120:0/1526805781 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561aa4c30940 0x561aa4c32d50 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be20a211a860df6b server_cookie=556eb3a057ec6c20 in_seq=0 out_seq=0 2024-09-19T09:17:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.695+0000 7f3dbf7fe640 1 -- 172.21.15.120:0/1526805781 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561aa4b8ba20 con 0x561aa4c30940 2024-09-19T09:17:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.695+0000 7f3dbf7fe640 10 client.20541 handle_command_reply: tid=0 2024-09-19T09:17:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.695+0000 7f3de9426740 1 client.20541 shutdown 2024-09-19T09:17:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.695+0000 7f3de9426740 1 -- 172.21.15.120:0/1526805781 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561aa4c20370 msgr2=0x561aa4c2c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.695+0000 7f3de9426740 1 --2- 172.21.15.120:0/1526805781 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561aa4c20370 0x561aa4c2c110 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8010bd0 tx=0x7f3dd8009500 comp rx=0 tx=0).stop 2024-09-19T09:17:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.695+0000 7f3de9426740 1 -- 172.21.15.120:0/1526805781 shutdown_connections 2024-09-19T09:17:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.695+0000 7f3de9426740 1 --2- 172.21.15.120:0/1526805781 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561aa4c30940 0x561aa4c32d50 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.695+0000 7f3de9426740 1 --2- 172.21.15.120:0/1526805781 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561aa4c20370 0x561aa4c2c110 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.695+0000 7f3dbf7fe640 0 client.20541 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.695+0000 7f3de9426740 1 --2- 172.21.15.120:0/1526805781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aa4c1b2e0 0x561aa4c250d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.695+0000 7f3de9426740 1 --2- 172.21.15.120:0/1526805781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561aa4c1a9a0 0x561aa4c24b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.695+0000 7f3de9426740 1 -- 172.21.15.120:0/1526805781 >> 172.21.15.120:0/1526805781 conn(0x561aa4c16010 msgr2=0x561aa4c17820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.695+0000 7f3de9426740 1 -- 172.21.15.120:0/1526805781 shutdown_connections 2024-09-19T09:17:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.695+0000 7f3de9426740 1 -- 172.21.15.120:0/1526805781 wait complete. 2024-09-19T09:17:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.695+0000 7f3de9426740 20 client.20541 trim_cache size 0 max 16384 2024-09-19T09:17:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.710+0000 7f3de9426740 1 Processor -- start 2024-09-19T09:17:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.710+0000 7f3de9426740 1 -- start start 2024-09-19T09:17:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.710+0000 7f3de9426740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aa4c16010 0x561aa4c1ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.710+0000 7f3de9426740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561aa4c1a9a0 0x561aa4c2c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.710+0000 7f3de9426740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561aa4c1b2e0 0x561aa4c16ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.710+0000 7f3de9426740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3de01a6d30 con 0x561aa4c1a9a0 2024-09-19T09:17:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.710+0000 7f3de9426740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3de01a8900 con 0x561aa4c16010 2024-09-19T09:17:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.710+0000 7f3de9426740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3de01a7400 con 0x561aa4c1b2e0 2024-09-19T09:17:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.710+0000 7f3ddf7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561aa4c1a9a0 0x561aa4c2c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.710+0000 7f3ddf7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561aa4c1a9a0 0x561aa4c2c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39054/0 (socket says 172.21.15.120:39054) 2024-09-19T09:17:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.710+0000 7f3ddf7fe640 1 -- 172.21.15.120:0/1605597310 learned_addr learned my addr 172.21.15.120:0/1605597310 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.710+0000 7f3ddffff640 1 --2- 172.21.15.120:0/1605597310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aa4c16010 0x561aa4c1ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.710+0000 7f3de4b56640 1 --2- 172.21.15.120:0/1605597310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561aa4c1b2e0 0x561aa4c16ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.710+0000 7f3ddf7fe640 1 -- 172.21.15.120:0/1605597310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561aa4c1b2e0 msgr2=0x561aa4c16ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.710+0000 7f3ddf7fe640 1 --2- 172.21.15.120:0/1605597310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561aa4c1b2e0 0x561aa4c16ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.710+0000 7f3ddf7fe640 1 -- 172.21.15.120:0/1605597310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aa4c16010 msgr2=0x561aa4c1ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.710+0000 7f3ddf7fe640 1 --2- 172.21.15.120:0/1605597310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aa4c16010 0x561aa4c1ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.710+0000 7f3ddf7fe640 1 -- 172.21.15.120:0/1605597310 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aa4c246d0 con 0x561aa4c1a9a0 2024-09-19T09:17:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.710+0000 7f3ddffff640 1 --2- 172.21.15.120:0/1605597310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aa4c16010 0x561aa4c1ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:17:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.710+0000 7f3ddf7fe640 1 --2- 172.21.15.120:0/1605597310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561aa4c1a9a0 0x561aa4c2c0d0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4005bc0 tx=0x7f3dd4005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22798db6aaa800ea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.711+0000 7f3dc5ffb640 1 -- 172.21.15.120:0/1605597310 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd405a070 con 0x561aa4c1a9a0 2024-09-19T09:17:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.711+0000 7f3dc5ffb640 1 -- 172.21.15.120:0/1605597310 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dd4032df0 con 0x561aa4c1a9a0 2024-09-19T09:17:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.711+0000 7f3dc5ffb640 1 -- 172.21.15.120:0/1605597310 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4035040 con 0x561aa4c1a9a0 2024-09-19T09:17:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.711+0000 7f3de9426740 1 -- 172.21.15.120:0/1605597310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561aa4c1a9a0 msgr2=0x561aa4c2c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.711+0000 7f3de9426740 1 --2- 172.21.15.120:0/1605597310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561aa4c1a9a0 0x561aa4c2c0d0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4005bc0 tx=0x7f3dd4005b50 comp rx=0 tx=0).stop 2024-09-19T09:17:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.711+0000 7f3de9426740 1 -- 172.21.15.120:0/1605597310 shutdown_connections 2024-09-19T09:17:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.711+0000 7f3de9426740 1 --2- 172.21.15.120:0/1605597310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561aa4c1b2e0 0x561aa4c16ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.711+0000 7f3de9426740 1 --2- 172.21.15.120:0/1605597310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561aa4c1a9a0 0x561aa4c2c0d0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.711+0000 7f3de9426740 1 --2- 172.21.15.120:0/1605597310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aa4c16010 0x561aa4c1ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.711+0000 7f3de9426740 1 -- 172.21.15.120:0/1605597310 >> 172.21.15.120:0/1605597310 conn(0x561aa4c1fec0 msgr2=0x561aa4c24b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.711+0000 7f3de9426740 1 -- 172.21.15.120:0/1605597310 shutdown_connections 2024-09-19T09:17:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.711+0000 7f3de9426740 1 -- 172.21.15.120:0/1605597310 wait complete. 2024-09-19T09:17:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.712+0000 7f3de9426740 1 -- start start 2024-09-19T09:17:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.712+0000 7f3de9426740 1 Processor -- start 2024-09-19T09:17:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.712+0000 7f3de9426740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aa4c16010 0x561aa4c29610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.712+0000 7f3de9426740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561aa4c1a9a0 0x561aa4c29b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.712+0000 7f3de9426740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561aa4c1b2e0 0x561aa4c2aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.712+0000 7f3de9426740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3de01a8230 con 0x561aa4c1b2e0 2024-09-19T09:17:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.712+0000 7f3de9426740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3de01a7ad0 con 0x561aa4c16010 2024-09-19T09:17:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.712+0000 7f3de9426740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3de01a4cf0 con 0x561aa4c1a9a0 2024-09-19T09:17:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.712+0000 7f3de9426740 10 client.0 _ll_register_callbacks cb 0x7f3dd8009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:17:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.712+0000 7f3ddffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aa4c16010 0x561aa4c29610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.712+0000 7f3de4b56640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561aa4c1b2e0 0x561aa4c2aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.712+0000 7f3ddf7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561aa4c1a9a0 0x561aa4c29b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.712+0000 7f3ddffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aa4c16010 0x561aa4c29610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:48086/0 (socket says 172.21.15.120:48086) 2024-09-19T09:17:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.712+0000 7f3de4b56640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561aa4c1b2e0 0x561aa4c2aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39064/0 (socket says 172.21.15.120:39064) 2024-09-19T09:17:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.712+0000 7f3ddffff640 1 -- 172.21.15.120:0/2586422245 learned_addr learned my addr 172.21.15.120:0/2586422245 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.712+0000 7f3de4b56640 1 -- 172.21.15.120:0/2586422245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561aa4c1a9a0 msgr2=0x561aa4c29b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.712+0000 7f3de4b56640 1 --2- 172.21.15.120:0/2586422245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561aa4c1a9a0 0x561aa4c29b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.712+0000 7f3de4b56640 1 -- 172.21.15.120:0/2586422245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aa4c16010 msgr2=0x561aa4c29610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.712+0000 7f3de4b56640 1 --2- 172.21.15.120:0/2586422245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aa4c16010 0x561aa4c29610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.712+0000 7f3de4b56640 1 -- 172.21.15.120:0/2586422245 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561aa4c2b100 con 0x561aa4c1b2e0 2024-09-19T09:17:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.713+0000 7f3de4b56640 1 --2- 172.21.15.120:0/2586422245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561aa4c1b2e0 0x561aa4c2aaf0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8010ba0 tx=0x7f3dd8027a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495db63f295a3d10 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.713+0000 7f3dbf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:17:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.713+0000 7f3dbf7fe640 1 -- 172.21.15.120:0/2586422245 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd80107d0 con 0x561aa4c1b2e0 2024-09-19T09:17:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.713+0000 7f3dbf7fe640 1 -- 172.21.15.120:0/2586422245 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f3dd8004950 con 0x561aa4c1b2e0 2024-09-19T09:17:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.713+0000 7f3de9426740 1 -- 172.21.15.120:0/2586422245 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dd4032df0 con 0x561aa4c1b2e0 2024-09-19T09:17:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.713+0000 7f3dbf7fe640 1 -- 172.21.15.120:0/2586422245 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3dd8042050 con 0x561aa4c1b2e0 2024-09-19T09:17:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.713+0000 7f3de9426740 10 client.20547 fetch_fsmap learned FSMap version 36 2024-09-19T09:17:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.713+0000 7f3de9426740 1 -- 172.21.15.120:0/2586422245 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561aa4c2c410 con 0x561aa4c1b2e0 2024-09-19T09:17:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.713+0000 7f3dbf7fe640 1 -- 172.21.15.120:0/2586422245 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dd4032df0 con 0x561aa4c1b2e0 2024-09-19T09:17:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.713+0000 7f3dbf7fe640 1 -- 172.21.15.120:0/2586422245 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f3dd8025050 con 0x561aa4c1b2e0 2024-09-19T09:17:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.713+0000 7f3de9426740 10 client.20547 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:17:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.713+0000 7f3de9426740 10 client.20547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:17:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.713+0000 7f3de9426740 20 client.20547 populate_metadata read hostname 'smithi120' 2024-09-19T09:17:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.713+0000 7f3de9426740 1 --2- 172.21.15.120:0/2586422245 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561aa4c20370 0x561aa4c9deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.713+0000 7f3de9426740 4 client.20547 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:17:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.713+0000 7f3de9426740 1 -- 172.21.15.120:0/2586422245 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561aa4b85d70 con 0x561aa4c20370 2024-09-19T09:17:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.714+0000 7f3ddffff640 1 --2- 172.21.15.120:0/2586422245 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561aa4c20370 0x561aa4c9deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.714+0000 7f3dbf7fe640 10 client.20547 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:52.714+0000 7f3ddffff640 1 --2- 172.21.15.120:0/2586422245 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561aa4c20370 0x561aa4c9deb0 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1aaaf2032f6c2ae server_cookie=a53e91164856e90c in_seq=0 out_seq=0 2024-09-19T09:17:53.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:52 smithi153 ceph-mon[35071]: pgmap v795: 97 pgs: 97 active+clean; 311 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.7 MiB/s wr, 769 op/s 2024-09-19T09:17:53.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:53.249+0000 7f3dbf7fe640 1 -- 172.21.15.120:0/2586422245 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561aa4b85d70 con 0x561aa4c20370 2024-09-19T09:17:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:53.249+0000 7f3dbf7fe640 10 client.20547 handle_command_reply: tid=0 2024-09-19T09:17:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:53.249+0000 7f3de9426740 1 client.20547 shutdown 2024-09-19T09:17:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:53.250+0000 7f3de9426740 1 -- 172.21.15.120:0/2586422245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561aa4c1b2e0 msgr2=0x561aa4c2aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:53.250+0000 7f3de9426740 1 --2- 172.21.15.120:0/2586422245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561aa4c1b2e0 0x561aa4c2aaf0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8010ba0 tx=0x7f3dd8027a70 comp rx=0 tx=0).stop 2024-09-19T09:17:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:53.250+0000 7f3de9426740 1 -- 172.21.15.120:0/2586422245 shutdown_connections 2024-09-19T09:17:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:53.250+0000 7f3de9426740 1 --2- 172.21.15.120:0/2586422245 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561aa4c20370 0x561aa4c9deb0 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:53.250+0000 7f3de9426740 1 --2- 172.21.15.120:0/2586422245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561aa4c1b2e0 0x561aa4c2aaf0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:53.250+0000 7f3dbf7fe640 0 client.20547 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:53.250+0000 7f3de9426740 1 --2- 172.21.15.120:0/2586422245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561aa4c1a9a0 0x561aa4c29b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:53.250+0000 7f3de9426740 1 --2- 172.21.15.120:0/2586422245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aa4c16010 0x561aa4c29610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:53.250+0000 7f3de9426740 1 -- 172.21.15.120:0/2586422245 >> 172.21.15.120:0/2586422245 conn(0x561aa4c1fec0 msgr2=0x561aa4c149a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:53.250+0000 7f3de9426740 1 -- 172.21.15.120:0/2586422245 shutdown_connections 2024-09-19T09:17:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:53.250+0000 7f3de9426740 1 -- 172.21.15.120:0/2586422245 wait complete. 2024-09-19T09:17:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:53.250+0000 7f3de9426740 20 client.20547 trim_cache size 0 max 16384 2024-09-19T09:17:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:17:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:17:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:17:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:17:53.255 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:53.251+0000 7f3de65b9640 1 -- 172.21.15.120:0/1070179617 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3dc80a32f0 msgr2=0x7f3dc80a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:53.255 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:53.251+0000 7f3de65b9640 1 --2- 172.21.15.120:0/1070179617 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3dc80a32f0 0x7f3dc80a57b0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f3de006b380 tx=0x7f3dd8014040 comp rx=0 tx=0).stop 2024-09-19T09:17:53.255 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:53.251+0000 7f3de65b9640 1 -- 172.21.15.120:0/1070179617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3de01025d0 msgr2=0x7f3de0197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:53.251+0000 7f3de65b9640 1 --2- 172.21.15.120:0/1070179617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3de01025d0 0x7f3de0197e20 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f3dd000e9f0 tx=0x7f3dd000eeb0 comp rx=0 tx=0).stop 2024-09-19T09:17:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:53.251+0000 7f3de65b9640 1 -- 172.21.15.120:0/1070179617 shutdown_connections 2024-09-19T09:17:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:53.251+0000 7f3de65b9640 1 --2- 172.21.15.120:0/1070179617 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3dc80a32f0 0x7f3dc80a57b0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:53.251+0000 7f3de65b9640 1 --2- 172.21.15.120:0/1070179617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3de0107830 0x7f3de019f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:53.251+0000 7f3de65b9640 1 --2- 172.21.15.120:0/1070179617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3de0104f00 0x7f3de0198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:53.251+0000 7f3de65b9640 1 --2- 172.21.15.120:0/1070179617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3de01025d0 0x7f3de0197e20 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:53.251+0000 7f3de65b9640 1 -- 172.21.15.120:0/1070179617 >> 172.21.15.120:0/1070179617 conn(0x7f3de00fc1b0 msgr2=0x7f3de0102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:53.251+0000 7f3de65b9640 1 -- 172.21.15.120:0/1070179617 shutdown_connections 2024-09-19T09:17:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:17:53.251+0000 7f3de65b9640 1 -- 172.21.15.120:0/1070179617 wait complete. 2024-09-19T09:17:53.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:52 smithi120 ceph-mon[31150]: pgmap v795: 97 pgs: 97 active+clean; 311 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.7 MiB/s wr, 769 op/s 2024-09-19T09:17:53.355 INFO:tasks.workunit.client.0.smithi120.stdout: 10 52819 27.97 MB/sec warmup 108 sec latency 268.141 ms 2024-09-19T09:17:53.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:52 smithi196 ceph-mon[35150]: pgmap v795: 97 pgs: 97 active+clean; 311 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.7 MiB/s wr, 769 op/s 2024-09-19T09:17:54.356 INFO:tasks.workunit.client.0.smithi120.stdout: 10 53470 28.01 MB/sec warmup 109 sec latency 56.940 ms 2024-09-19T09:17:54.604 INFO:tasks.quiescer.fs.[cephfs]:Total duration (58.9) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T09:17:54.604 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.45 2024-09-19T09:17:54.605 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:17:54.886 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:17:55.307 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:54 smithi120 ceph-mon[31150]: pgmap v796: 97 pgs: 97 active+clean; 324 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.9 MiB/s wr, 718 op/s 2024-09-19T09:17:55.355 INFO:tasks.workunit.client.0.smithi120.stdout: 10 54251 28.11 MB/sec warmup 110 sec latency 50.245 ms 2024-09-19T09:17:55.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.396+0000 7f5e8d278640 1 -- 172.21.15.120:0/768885217 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e88102ef0 msgr2=0x7f5e8810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:55.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.396+0000 7f5e8d278640 1 --2- 172.21.15.120:0/768885217 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e88102ef0 0x7f5e8810da40 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f5e80009ca0 tx=0x7f5e8002a950 comp rx=0 tx=0).stop 2024-09-19T09:17:55.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.396+0000 7f5e8d278640 1 -- 172.21.15.120:0/768885217 shutdown_connections 2024-09-19T09:17:55.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.396+0000 7f5e8d278640 1 --2- 172.21.15.120:0/768885217 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e88102ef0 0x7f5e8810da40 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:55.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.396+0000 7f5e8d278640 1 --2- 172.21.15.120:0/768885217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e88102570 0x7f5e881029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:55.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.396+0000 7f5e8d278640 1 --2- 172.21.15.120:0/768885217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e88100e20 0x7f5e88101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:55.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.396+0000 7f5e8d278640 1 -- 172.21.15.120:0/768885217 >> 172.21.15.120:0/768885217 conn(0x7f5e880fc7d0 msgr2=0x7f5e880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:55.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.396+0000 7f5e8d278640 1 -- 172.21.15.120:0/768885217 shutdown_connections 2024-09-19T09:17:55.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.396+0000 7f5e8d278640 1 -- 172.21.15.120:0/768885217 wait complete. 2024-09-19T09:17:55.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.397+0000 7f5e8d278640 1 Processor -- start 2024-09-19T09:17:55.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.397+0000 7f5e8d278640 1 -- start start 2024-09-19T09:17:55.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.397+0000 7f5e8d278640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e88100e20 0x7f5e881992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:55.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.397+0000 7f5e8d278640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e88102570 0x7f5e881997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:55.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.397+0000 7f5e8d278640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e88102ef0 0x7f5e881a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:55.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.397+0000 7f5e8d278640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5e88110f80 con 0x7f5e88100e20 2024-09-19T09:17:55.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.397+0000 7f5e8d278640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5e88110e00 con 0x7f5e88102570 2024-09-19T09:17:55.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.397+0000 7f5e8d278640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5e88111100 con 0x7f5e88102ef0 2024-09-19T09:17:55.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.397+0000 7f5e86ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e88100e20 0x7f5e881992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:55.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.397+0000 7f5e86ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e88100e20 0x7f5e881992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39080/0 (socket says 172.21.15.120:39080) 2024-09-19T09:17:55.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.397+0000 7f5e86ffd640 1 -- 172.21.15.120:0/1215969288 learned_addr learned my addr 172.21.15.120:0/1215969288 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:55.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.397+0000 7f5e867fc640 1 --2- 172.21.15.120:0/1215969288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e88102570 0x7f5e881997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:55.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.398+0000 7f5e877fe640 1 --2- 172.21.15.120:0/1215969288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e88102ef0 0x7f5e881a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:55.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.398+0000 7f5e86ffd640 1 -- 172.21.15.120:0/1215969288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e88102ef0 msgr2=0x7f5e881a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:55.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.398+0000 7f5e86ffd640 1 --2- 172.21.15.120:0/1215969288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e88102ef0 0x7f5e881a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:55.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.398+0000 7f5e86ffd640 1 -- 172.21.15.120:0/1215969288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e88102570 msgr2=0x7f5e881997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:55.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.398+0000 7f5e86ffd640 1 --2- 172.21.15.120:0/1215969288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e88102570 0x7f5e881997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:55.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.398+0000 7f5e86ffd640 1 -- 172.21.15.120:0/1215969288 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e881a4870 con 0x7f5e88100e20 2024-09-19T09:17:55.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.398+0000 7f5e86ffd640 1 --2- 172.21.15.120:0/1215969288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e88100e20 0x7f5e881992b0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f5e7800e9f0 tx=0x7f5e7800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:55.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.398+0000 7f5e6ffff640 1 -- 172.21.15.120:0/1215969288 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7800cd80 con 0x7f5e88100e20 2024-09-19T09:17:55.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.398+0000 7f5e6ffff640 1 -- 172.21.15.120:0/1215969288 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e78010910 con 0x7f5e88100e20 2024-09-19T09:17:55.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.398+0000 7f5e8d278640 1 -- 172.21.15.120:0/1215969288 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e881a4a40 con 0x7f5e88100e20 2024-09-19T09:17:55.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.398+0000 7f5e6ffff640 1 -- 172.21.15.120:0/1215969288 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78018990 con 0x7f5e88100e20 2024-09-19T09:17:55.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.398+0000 7f5e8d278640 1 -- 172.21.15.120:0/1215969288 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e881a4f80 con 0x7f5e88100e20 2024-09-19T09:17:55.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.399+0000 7f5e8d278640 1 -- 172.21.15.120:0/1215969288 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e88101200 con 0x7f5e88100e20 2024-09-19T09:17:55.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.400+0000 7f5e6ffff640 1 -- 172.21.15.120:0/1215969288 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5e78010ac0 con 0x7f5e88100e20 2024-09-19T09:17:55.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.401+0000 7f5e6ffff640 1 --2- 172.21.15.120:0/1215969288 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5e700a3240 0x7f5e700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:55.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.401+0000 7f5e6ffff640 1 -- 172.21.15.120:0/1215969288 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f5e780cfcb0 con 0x7f5e88100e20 2024-09-19T09:17:55.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.401+0000 7f5e867fc640 1 --2- 172.21.15.120:0/1215969288 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5e700a3240 0x7f5e700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:55.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.401+0000 7f5e867fc640 1 --2- 172.21.15.120:0/1215969288 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5e700a3240 0x7f5e700a5700 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f5e8806ab00 tx=0x7f5e7c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:55.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.405+0000 7f5e6ffff640 1 -- 172.21.15.120:0/1215969288 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5e7809c9d0 con 0x7f5e88100e20 2024-09-19T09:17:55.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:54 smithi196 ceph-mon[35150]: pgmap v796: 97 pgs: 97 active+clean; 324 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.9 MiB/s wr, 718 op/s 2024-09-19T09:17:55.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:54 smithi153 ceph-mon[35071]: pgmap v796: 97 pgs: 97 active+clean; 324 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.9 MiB/s wr, 718 op/s 2024-09-19T09:17:55.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.637+0000 7f5e8d278640 1 -- 172.21.15.120:0/1215969288 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e881a51b0 con 0x7f5e88100e20 2024-09-19T09:17:55.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.638+0000 7f5e6ffff640 1 -- 172.21.15.120:0/1215969288 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11993 (secure 0 0 0) 0x7f5e7809c2d0 con 0x7f5e88100e20 2024-09-19T09:17:55.640 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:17:55.640 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":36,"btime":"2024-09-19T09:16:33:442503+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:16:33.001618+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:17:55.640 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 36 2024-09-19T09:17:55.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.640+0000 7f5e8d278640 1 -- 172.21.15.120:0/1215969288 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5e700a3240 msgr2=0x7f5e700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:55.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.640+0000 7f5e8d278640 1 --2- 172.21.15.120:0/1215969288 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5e700a3240 0x7f5e700a5700 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f5e8806ab00 tx=0x7f5e7c005f90 comp rx=0 tx=0).stop 2024-09-19T09:17:55.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.640+0000 7f5e8d278640 1 -- 172.21.15.120:0/1215969288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e88100e20 msgr2=0x7f5e881992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:55.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.640+0000 7f5e8d278640 1 --2- 172.21.15.120:0/1215969288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e88100e20 0x7f5e881992b0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f5e7800e9f0 tx=0x7f5e7800eeb0 comp rx=0 tx=0).stop 2024-09-19T09:17:55.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.640+0000 7f5e8d278640 1 -- 172.21.15.120:0/1215969288 shutdown_connections 2024-09-19T09:17:55.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.640+0000 7f5e8d278640 1 --2- 172.21.15.120:0/1215969288 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5e700a3240 0x7f5e700a5700 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:55.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.640+0000 7f5e8d278640 1 --2- 172.21.15.120:0/1215969288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e88102ef0 0x7f5e881a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:55.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.640+0000 7f5e8d278640 1 --2- 172.21.15.120:0/1215969288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e88102570 0x7f5e881997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:55.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.640+0000 7f5e8d278640 1 --2- 172.21.15.120:0/1215969288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e88100e20 0x7f5e881992b0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:55.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.640+0000 7f5e8d278640 1 -- 172.21.15.120:0/1215969288 >> 172.21.15.120:0/1215969288 conn(0x7f5e880fc7d0 msgr2=0x7f5e88106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:55.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.640+0000 7f5e8d278640 1 -- 172.21.15.120:0/1215969288 shutdown_connections 2024-09-19T09:17:55.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:55.640+0000 7f5e8d278640 1 -- 172.21.15.120:0/1215969288 wait complete. 2024-09-19T09:17:56.212 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:55 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1215969288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:17:56.355 INFO:tasks.workunit.client.0.smithi120.stdout: 10 54936 28.24 MB/sec warmup 111 sec latency 54.969 ms 2024-09-19T09:17:56.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:55 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1215969288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:17:56.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:55 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1215969288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:17:56.679 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db / --timeout 90 --expiration 149.45 --await-for 45.0' 2024-09-19T09:17:56.680 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db / --timeout 90 --expiration 149.45 --await-for 45.0 2024-09-19T09:17:56.965 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:17:57.229 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:56 smithi120 ceph-mon[31150]: pgmap v797: 97 pgs: 97 active+clean; 315 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.4 MiB/s wr, 648 op/s 2024-09-19T09:17:57.356 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55606 28.36 MB/sec warmup 112 sec latency 77.955 ms 2024-09-19T09:17:57.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:56 smithi196 ceph-mon[35150]: pgmap v797: 97 pgs: 97 active+clean; 315 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.4 MiB/s wr, 648 op/s 2024-09-19T09:17:57.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:56 smithi153 ceph-mon[35071]: pgmap v797: 97 pgs: 97 active+clean; 315 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.4 MiB/s wr, 648 op/s 2024-09-19T09:17:57.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.809+0000 7f2a9bb2d640 1 -- 172.21.15.120:0/492381381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a9410e890 msgr2=0x7f2a9410ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:57.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.809+0000 7f2a9bb2d640 1 --2- 172.21.15.120:0/492381381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a9410e890 0x7f2a9410ed10 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00bd70 tx=0x7f2a8c02c420 comp rx=0 tx=0).stop 2024-09-19T09:17:57.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.809+0000 7f2a9bb2d640 1 -- 172.21.15.120:0/492381381 shutdown_connections 2024-09-19T09:17:57.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.809+0000 7f2a9bb2d640 1 --2- 172.21.15.120:0/492381381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a9410e890 0x7f2a9410ed10 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:57.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.809+0000 7f2a9bb2d640 1 --2- 172.21.15.120:0/492381381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a94109d70 0x7f2a9410e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:57.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.809+0000 7f2a9bb2d640 1 --2- 172.21.15.120:0/492381381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a94109450 0x7f2a94109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:57.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.809+0000 7f2a9bb2d640 1 -- 172.21.15.120:0/492381381 >> 172.21.15.120:0/492381381 conn(0x7f2a9406e2d0 msgr2=0x7f2a9406e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:57.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.809+0000 7f2a9bb2d640 1 -- 172.21.15.120:0/492381381 shutdown_connections 2024-09-19T09:17:57.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.809+0000 7f2a9bb2d640 1 -- 172.21.15.120:0/492381381 wait complete. 2024-09-19T09:17:57.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.810+0000 7f2a9bb2d640 1 Processor -- start 2024-09-19T09:17:57.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.810+0000 7f2a9bb2d640 1 -- start start 2024-09-19T09:17:57.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.810+0000 7f2a9bb2d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a94109450 0x7f2a941a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:57.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.810+0000 7f2a9bb2d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a94109d70 0x7f2a941a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:57.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.810+0000 7f2a9bb2d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a9410e890 0x7f2a9419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:57.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.810+0000 7f2a9bb2d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2a94118570 con 0x7f2a94109d70 2024-09-19T09:17:57.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.810+0000 7f2a9bb2d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2a941183f0 con 0x7f2a94109450 2024-09-19T09:17:57.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.810+0000 7f2a998c9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a94109450 0x7f2a941a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:57.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.810+0000 7f2a9bb2d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a941186f0 con 0x7f2a9410e890 2024-09-19T09:17:57.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.810+0000 7f2a998c9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a94109450 0x7f2a941a46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:48120/0 (socket says 172.21.15.120:48120) 2024-09-19T09:17:57.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.810+0000 7f2a998c9640 1 -- 172.21.15.120:0/2647616519 learned_addr learned my addr 172.21.15.120:0/2647616519 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:57.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.810+0000 7f2a9a0ca640 1 --2- 172.21.15.120:0/2647616519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a9410e890 0x7f2a9419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:57.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.811+0000 7f2a990c8640 1 --2- 172.21.15.120:0/2647616519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a94109d70 0x7f2a941a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:57.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.811+0000 7f2a998c9640 1 -- 172.21.15.120:0/2647616519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a9410e890 msgr2=0x7f2a9419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:57.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.811+0000 7f2a998c9640 1 --2- 172.21.15.120:0/2647616519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a9410e890 0x7f2a9419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:57.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.811+0000 7f2a998c9640 1 -- 172.21.15.120:0/2647616519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a94109d70 msgr2=0x7f2a941a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:57.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.811+0000 7f2a998c9640 1 --2- 172.21.15.120:0/2647616519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a94109d70 0x7f2a941a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:57.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.811+0000 7f2a998c9640 1 -- 172.21.15.120:0/2647616519 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9419ef90 con 0x7f2a94109450 2024-09-19T09:17:57.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.811+0000 7f2a990c8640 1 --2- 172.21.15.120:0/2647616519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a94109d70 0x7f2a941a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:17:57.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.811+0000 7f2a998c9640 1 --2- 172.21.15.120:0/2647616519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a94109450 0x7f2a941a46a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2a9000e9c0 tx=0x7f2a9000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:57.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.811+0000 7f2a8affd640 1 -- 172.21.15.120:0/2647616519 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9000ccc0 con 0x7f2a94109450 2024-09-19T09:17:57.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.811+0000 7f2a9bb2d640 1 -- 172.21.15.120:0/2647616519 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a9419f280 con 0x7f2a94109450 2024-09-19T09:17:57.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.811+0000 7f2a9bb2d640 1 -- 172.21.15.120:0/2647616519 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a941ab570 con 0x7f2a94109450 2024-09-19T09:17:57.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.811+0000 7f2a8affd640 1 -- 172.21.15.120:0/2647616519 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a90005500 con 0x7f2a94109450 2024-09-19T09:17:57.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.811+0000 7f2a8affd640 1 -- 172.21.15.120:0/2647616519 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90010430 con 0x7f2a94109450 2024-09-19T09:17:57.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.814+0000 7f2a8affd640 1 -- 172.21.15.120:0/2647616519 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f2a900046c0 con 0x7f2a94109450 2024-09-19T09:17:57.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.815+0000 7f2a8affd640 1 --2- 172.21.15.120:0/2647616519 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2a800a32c0 0x7f2a800a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:57.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.815+0000 7f2a990c8640 1 --2- 172.21.15.120:0/2647616519 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2a800a32c0 0x7f2a800a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:57.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.815+0000 7f2a8affd640 1 -- 172.21.15.120:0/2647616519 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f2a900cfc40 con 0x7f2a94109450 2024-09-19T09:17:57.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.816+0000 7f2a9df06740 1 Processor -- start 2024-09-19T09:17:57.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.816+0000 7f2a9df06740 1 -- start start 2024-09-19T09:17:57.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.816+0000 7f2a990c8640 1 --2- 172.21.15.120:0/2647616519 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2a800a32c0 0x7f2a800a5780 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2a9406e9f0 tx=0x7f2a84005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:57.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.816+0000 7f2a9df06740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30139ea90 0x55d30139ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:57.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.816+0000 7f2a9df06740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30139f3b0 0x55d3013993e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:57.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.816+0000 7f2a9df06740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d301399a50 0x55d301399ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:57.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.817+0000 7f2a9df06740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d3012f4160 con 0x55d301399a50 2024-09-19T09:17:57.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.817+0000 7f2a9df06740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d30129b720 con 0x55d30139f3b0 2024-09-19T09:17:57.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.817+0000 7f2a9df06740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d3012f0120 con 0x55d30139ea90 2024-09-19T09:17:57.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.817+0000 7f2a990c8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30139f3b0 0x55d3013993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:57.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.817+0000 7f2a9a0ca640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d301399a50 0x55d301399ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:57.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.817+0000 7f2a998c9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30139ea90 0x55d30139ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:57.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.817+0000 7f2a990c8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30139f3b0 0x55d3013993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:48132/0 (socket says 172.21.15.120:48132) 2024-09-19T09:17:57.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.817+0000 7f2a998c9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30139ea90 0x55d30139ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:60222/0 (socket says 172.21.15.120:60222) 2024-09-19T09:17:57.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.817+0000 7f2a9a0ca640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d301399a50 0x55d301399ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39106/0 (socket says 172.21.15.120:39106) 2024-09-19T09:17:57.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.817+0000 7f2a990c8640 1 -- 172.21.15.120:0/2234592077 learned_addr learned my addr 172.21.15.120:0/2234592077 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:57.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.817+0000 7f2a998c9640 1 -- 172.21.15.120:0/2234592077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30139f3b0 msgr2=0x55d3013993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:57.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.817+0000 7f2a998c9640 1 --2- 172.21.15.120:0/2234592077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30139f3b0 0x55d3013993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:57.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.817+0000 7f2a998c9640 1 -- 172.21.15.120:0/2234592077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d301399a50 msgr2=0x55d301399ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:57.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.817+0000 7f2a998c9640 1 --2- 172.21.15.120:0/2234592077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d301399a50 0x55d301399ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:57.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.817+0000 7f2a998c9640 1 -- 172.21.15.120:0/2234592077 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d301065ba0 con 0x55d30139ea90 2024-09-19T09:17:57.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.817+0000 7f2a9a0ca640 1 --2- 172.21.15.120:0/2234592077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d301399a50 0x55d301399ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:17:57.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.818+0000 7f2a998c9640 1 --2- 172.21.15.120:0/2234592077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30139ea90 0x55d30139ee70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2a9009bf60 tx=0x7f2a9009be60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d09c1a162ea09cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:57.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.818+0000 7f2a7ffff640 1 -- 172.21.15.120:0/2234592077 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a900a0610 con 0x55d30139ea90 2024-09-19T09:17:57.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.818+0000 7f2a7ffff640 1 -- 172.21.15.120:0/2234592077 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a900a4ed0 con 0x55d30139ea90 2024-09-19T09:17:57.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.818+0000 7f2a7ffff640 1 -- 172.21.15.120:0/2234592077 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a900a0170 con 0x55d30139ea90 2024-09-19T09:17:57.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.819+0000 7f2a9df06740 1 -- 172.21.15.120:0/2234592077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30139ea90 msgr2=0x55d30139ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:57.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.819+0000 7f2a9df06740 1 --2- 172.21.15.120:0/2234592077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30139ea90 0x55d30139ee70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2a9009bf60 tx=0x7f2a9009be60 comp rx=0 tx=0).stop 2024-09-19T09:17:57.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.819+0000 7f2a9df06740 1 -- 172.21.15.120:0/2234592077 shutdown_connections 2024-09-19T09:17:57.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.819+0000 7f2a9df06740 1 --2- 172.21.15.120:0/2234592077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d301399a50 0x55d301399ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:57.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.819+0000 7f2a9df06740 1 --2- 172.21.15.120:0/2234592077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30139f3b0 0x55d3013993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:57.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.819+0000 7f2a9df06740 1 --2- 172.21.15.120:0/2234592077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30139ea90 0x55d30139ee70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:57.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.819+0000 7f2a9df06740 1 -- 172.21.15.120:0/2234592077 >> 172.21.15.120:0/2234592077 conn(0x55d30138b890 msgr2=0x55d30138bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:57.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.819+0000 7f2a9df06740 1 -- 172.21.15.120:0/2234592077 shutdown_connections 2024-09-19T09:17:57.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.819+0000 7f2a9df06740 1 -- 172.21.15.120:0/2234592077 wait complete. 2024-09-19T09:17:57.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.820+0000 7f2a9df06740 1 -- start start 2024-09-19T09:17:57.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.820+0000 7f2a9df06740 1 Processor -- start 2024-09-19T09:17:57.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.821+0000 7f2a9df06740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d301399a50 0x55d3013a28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:57.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.821+0000 7f2a9df06740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30139ea90 0x55d3013a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:57.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.821+0000 7f2a9df06740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30139f3b0 0x55d3013a3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:57.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.821+0000 7f2a9df06740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d3013a5d10 con 0x55d301399a50 2024-09-19T09:17:57.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.821+0000 7f2a9df06740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d30127c900 con 0x55d30139ea90 2024-09-19T09:17:57.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.821+0000 7f2a9df06740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d3013a5e90 con 0x55d30139f3b0 2024-09-19T09:17:57.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.821+0000 7f2a998c9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d301399a50 0x55d3013a28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:57.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.821+0000 7f2a9df06740 10 client.0 _ll_register_callbacks cb 0x55d30127a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:17:57.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.821+0000 7f2a998c9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d301399a50 0x55d3013a28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39120/0 (socket says 172.21.15.120:39120) 2024-09-19T09:17:57.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.821+0000 7f2a9a0ca640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30139f3b0 0x55d3013a3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:57.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.821+0000 7f2a998c9640 1 -- 172.21.15.120:0/17415272 learned_addr learned my addr 172.21.15.120:0/17415272 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:57.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.821+0000 7f2a990c8640 1 --2- 172.21.15.120:0/17415272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30139ea90 0x55d3013a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:57.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.821+0000 7f2a990c8640 1 -- 172.21.15.120:0/17415272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30139f3b0 msgr2=0x55d3013a3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:57.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.821+0000 7f2a990c8640 1 --2- 172.21.15.120:0/17415272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30139f3b0 0x55d3013a3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:57.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.821+0000 7f2a990c8640 1 -- 172.21.15.120:0/17415272 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d301399a50 msgr2=0x55d3013a28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:57.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.821+0000 7f2a990c8640 1 --2- 172.21.15.120:0/17415272 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d301399a50 0x55d3013a28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:57.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.821+0000 7f2a990c8640 1 -- 172.21.15.120:0/17415272 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3012bcc50 con 0x55d30139ea90 2024-09-19T09:17:57.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.821+0000 7f2a998c9640 1 --2- 172.21.15.120:0/17415272 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d301399a50 0x55d3013a28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:17:57.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.822+0000 7f2a990c8640 1 --2- 172.21.15.120:0/17415272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30139ea90 0x55d3013a2e20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55d30138cce0 tx=0x7f2a840046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8e021eb82291195 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:57.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.822+0000 7f2a7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:17:57.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.822+0000 7f2a7d7fa640 1 -- 172.21.15.120:0/17415272 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84004220 con 0x55d30139ea90 2024-09-19T09:17:57.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.822+0000 7f2a7d7fa640 1 -- 172.21.15.120:0/17415272 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f2a840043c0 con 0x55d30139ea90 2024-09-19T09:17:57.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.822+0000 7f2a9df06740 1 -- 172.21.15.120:0/17415272 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a900a4ed0 con 0x55d30139ea90 2024-09-19T09:17:57.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.823+0000 7f2a7d7fa640 1 -- 172.21.15.120:0/17415272 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2a84020640 con 0x55d30139ea90 2024-09-19T09:17:57.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.823+0000 7f2a9df06740 10 client.29513 fetch_fsmap learned FSMap version 36 2024-09-19T09:17:57.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.823+0000 7f2a9df06740 1 -- 172.21.15.120:0/17415272 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d3013b1b00 con 0x55d30139ea90 2024-09-19T09:17:57.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.823+0000 7f2a7d7fa640 1 -- 172.21.15.120:0/17415272 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a900a4ed0 con 0x55d30139ea90 2024-09-19T09:17:57.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.823+0000 7f2a7d7fa640 1 -- 172.21.15.120:0/17415272 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f2a8400f040 con 0x55d30139ea90 2024-09-19T09:17:57.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.823+0000 7f2a9df06740 10 client.29513 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:17:57.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.823+0000 7f2a9df06740 10 client.29513 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:17:57.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.823+0000 7f2a9df06740 20 client.29513 populate_metadata read hostname 'smithi120' 2024-09-19T09:17:57.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.823+0000 7f2a9df06740 1 --2- 172.21.15.120:0/17415272 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d3013b2fc0 0x55d3013b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:57.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.823+0000 7f2a9df06740 4 client.29513 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:17:57.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.823+0000 7f2a9df06740 1 -- 172.21.15.120:0/17415272 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d301301890 con 0x55d3013b2fc0 2024-09-19T09:17:57.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.824+0000 7f2a998c9640 1 --2- 172.21.15.120:0/17415272 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d3013b2fc0 0x55d3013b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:57.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.824+0000 7f2a998c9640 1 --2- 172.21.15.120:0/17415272 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d3013b2fc0 0x55d3013b53d0 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6c6561bd369075c server_cookie=6982b121e06d4e4d in_seq=0 out_seq=0 2024-09-19T09:17:57.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.824+0000 7f2a7d7fa640 10 client.29513 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:57.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.826+0000 7f2a7d7fa640 1 -- 172.21.15.120:0/17415272 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d301301890 con 0x55d3013b2fc0 2024-09-19T09:17:57.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.826+0000 7f2a7d7fa640 10 client.29513 handle_command_reply: tid=0 2024-09-19T09:17:57.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.826+0000 7f2a9df06740 1 client.29513 shutdown 2024-09-19T09:17:57.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.826+0000 7f2a9df06740 1 -- 172.21.15.120:0/17415272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30139ea90 msgr2=0x55d3013a2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:57.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.826+0000 7f2a9df06740 1 --2- 172.21.15.120:0/17415272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30139ea90 0x55d3013a2e20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55d30138cce0 tx=0x7f2a840046c0 comp rx=0 tx=0).stop 2024-09-19T09:17:57.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.826+0000 7f2a9df06740 1 -- 172.21.15.120:0/17415272 shutdown_connections 2024-09-19T09:17:57.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.826+0000 7f2a9df06740 1 --2- 172.21.15.120:0/17415272 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d3013b2fc0 0x55d3013b53d0 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:57.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.826+0000 7f2a9df06740 1 --2- 172.21.15.120:0/17415272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30139f3b0 0x55d3013a3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:57.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.826+0000 7f2a7d7fa640 0 client.29513 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:57.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.827+0000 7f2a9df06740 1 --2- 172.21.15.120:0/17415272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30139ea90 0x55d3013a2e20 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:57.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.827+0000 7f2a9df06740 1 --2- 172.21.15.120:0/17415272 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d301399a50 0x55d3013a28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:57.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.827+0000 7f2a9df06740 1 -- 172.21.15.120:0/17415272 >> 172.21.15.120:0/17415272 conn(0x55d30138b890 msgr2=0x55d30138af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:57.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.827+0000 7f2a9df06740 1 -- 172.21.15.120:0/17415272 shutdown_connections 2024-09-19T09:17:57.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.827+0000 7f2a9df06740 1 -- 172.21.15.120:0/17415272 wait complete. 2024-09-19T09:17:57.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.827+0000 7f2a9df06740 20 client.29513 trim_cache size 0 max 16384 2024-09-19T09:17:57.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.845+0000 7f2a9df06740 1 Processor -- start 2024-09-19T09:17:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.845+0000 7f2a9df06740 1 -- start start 2024-09-19T09:17:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.846+0000 7f2a9df06740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d301399a50 0x55d3013993e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.846+0000 7f2a9df06740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30139ea90 0x55d30139c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.846+0000 7f2a9df06740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30139f3b0 0x55d30138a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.846+0000 7f2a9df06740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2a941b3cf0 con 0x55d30139f3b0 2024-09-19T09:17:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.846+0000 7f2a9df06740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2a941b58c0 con 0x55d301399a50 2024-09-19T09:17:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.846+0000 7f2a9df06740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a941b43c0 con 0x55d30139ea90 2024-09-19T09:17:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.846+0000 7f2a9a0ca640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30139f3b0 0x55d30138a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.846+0000 7f2a9a0ca640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30139f3b0 0x55d30138a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39126/0 (socket says 172.21.15.120:39126) 2024-09-19T09:17:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.846+0000 7f2a9a0ca640 1 -- 172.21.15.120:0/3505122021 learned_addr learned my addr 172.21.15.120:0/3505122021 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.846+0000 7f2a998c9640 1 --2- 172.21.15.120:0/3505122021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d301399a50 0x55d3013993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.846+0000 7f2a990c8640 1 --2- 172.21.15.120:0/3505122021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30139ea90 0x55d30139c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.846+0000 7f2a9a0ca640 1 -- 172.21.15.120:0/3505122021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30139ea90 msgr2=0x55d30139c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.846+0000 7f2a9a0ca640 1 --2- 172.21.15.120:0/3505122021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30139ea90 0x55d30139c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.846+0000 7f2a9a0ca640 1 -- 172.21.15.120:0/3505122021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d301399a50 msgr2=0x55d3013993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.846+0000 7f2a9a0ca640 1 --2- 172.21.15.120:0/3505122021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d301399a50 0x55d3013993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.846+0000 7f2a9a0ca640 1 -- 172.21.15.120:0/3505122021 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3013a0e70 con 0x55d30139f3b0 2024-09-19T09:17:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.846+0000 7f2a998c9640 1 --2- 172.21.15.120:0/3505122021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d301399a50 0x55d3013993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:17:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.846+0000 7f2a9a0ca640 1 --2- 172.21.15.120:0/3505122021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30139f3b0 0x55d30138a6a0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c036040 tx=0x7f2a8c033580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372f490c260d3f05 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.846+0000 7f2a7f7fe640 1 -- 172.21.15.120:0/3505122021 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c038040 con 0x55d30139f3b0 2024-09-19T09:17:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.846+0000 7f2a7f7fe640 1 -- 172.21.15.120:0/3505122021 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a8c033820 con 0x55d30139f3b0 2024-09-19T09:17:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.847+0000 7f2a7f7fe640 1 -- 172.21.15.120:0/3505122021 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c004040 con 0x55d30139f3b0 2024-09-19T09:17:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.847+0000 7f2a9df06740 1 -- 172.21.15.120:0/3505122021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30139f3b0 msgr2=0x55d30138a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.847+0000 7f2a9df06740 1 --2- 172.21.15.120:0/3505122021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30139f3b0 0x55d30138a6a0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c036040 tx=0x7f2a8c033580 comp rx=0 tx=0).stop 2024-09-19T09:17:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.847+0000 7f2a9df06740 1 -- 172.21.15.120:0/3505122021 shutdown_connections 2024-09-19T09:17:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.847+0000 7f2a9df06740 1 --2- 172.21.15.120:0/3505122021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30139f3b0 0x55d30138a6a0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.847+0000 7f2a9df06740 1 --2- 172.21.15.120:0/3505122021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d30139ea90 0x55d30139c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.847+0000 7f2a9df06740 1 --2- 172.21.15.120:0/3505122021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d301399a50 0x55d3013993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.847+0000 7f2a9df06740 1 -- 172.21.15.120:0/3505122021 >> 172.21.15.120:0/3505122021 conn(0x55d30138b890 msgr2=0x55d3013a11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.847+0000 7f2a9df06740 1 -- 172.21.15.120:0/3505122021 shutdown_connections 2024-09-19T09:17:57.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.847+0000 7f2a9df06740 1 -- 172.21.15.120:0/3505122021 wait complete. 2024-09-19T09:17:57.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.848+0000 7f2a9df06740 1 -- start start 2024-09-19T09:17:57.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.848+0000 7f2a9df06740 1 Processor -- start 2024-09-19T09:17:57.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.848+0000 7f2a9df06740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d301399a50 0x55d3013ab610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:57.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.848+0000 7f2a9df06740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30139ea90 0x55d3013ba3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:57.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.848+0000 7f2a9df06740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30139f3b0 0x55d3013ba920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:57.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.848+0000 7f2a9df06740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2a941b51f0 con 0x55d30139f3b0 2024-09-19T09:17:57.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.848+0000 7f2a9df06740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2a941b4a90 con 0x55d30139ea90 2024-09-19T09:17:57.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.848+0000 7f2a9df06740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a941b1cb0 con 0x55d301399a50 2024-09-19T09:17:57.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.848+0000 7f2a9df06740 10 client.0 _ll_register_callbacks cb 0x7f2a84009370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:17:57.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.848+0000 7f2a990c8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30139ea90 0x55d3013ba3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:57.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.848+0000 7f2a998c9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d301399a50 0x55d3013ab610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:57.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.848+0000 7f2a990c8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30139ea90 0x55d3013ba3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:48164/0 (socket says 172.21.15.120:48164) 2024-09-19T09:17:57.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.848+0000 7f2a998c9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d301399a50 0x55d3013ab610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:60258/0 (socket says 172.21.15.120:60258) 2024-09-19T09:17:57.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.848+0000 7f2a990c8640 1 -- 172.21.15.120:0/2558423398 learned_addr learned my addr 172.21.15.120:0/2558423398 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:17:57.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.848+0000 7f2a9a0ca640 1 --2- 172.21.15.120:0/2558423398 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30139f3b0 0x55d3013ba920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:57.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.849+0000 7f2a990c8640 1 -- 172.21.15.120:0/2558423398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d301399a50 msgr2=0x55d3013ab610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:57.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.849+0000 7f2a990c8640 1 --2- 172.21.15.120:0/2558423398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d301399a50 0x55d3013ab610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:57.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.849+0000 7f2a990c8640 1 -- 172.21.15.120:0/2558423398 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30139f3b0 msgr2=0x55d3013ba920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:17:57.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.849+0000 7f2a990c8640 1 --2- 172.21.15.120:0/2558423398 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30139f3b0 0x55d3013ba920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:57.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.849+0000 7f2a990c8640 1 -- 172.21.15.120:0/2558423398 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3013baee0 con 0x55d30139ea90 2024-09-19T09:17:57.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.849+0000 7f2a990c8640 1 --2- 172.21.15.120:0/2558423398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30139ea90 0x55d3013ba3e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2a84003e00 tx=0x7f2a840042d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe402833d32211f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:57.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.849+0000 7f2a7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:17:57.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.849+0000 7f2a7d7fa640 1 -- 172.21.15.120:0/2558423398 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84010bb0 con 0x55d30139ea90 2024-09-19T09:17:57.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.849+0000 7f2a9df06740 1 -- 172.21.15.120:0/2558423398 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a8c033820 con 0x55d30139ea90 2024-09-19T09:17:57.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.849+0000 7f2a7d7fa640 1 -- 172.21.15.120:0/2558423398 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f2a84005450 con 0x55d30139ea90 2024-09-19T09:17:57.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.849+0000 7f2a7d7fa640 1 -- 172.21.15.120:0/2558423398 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2a840044c0 con 0x55d30139ea90 2024-09-19T09:17:57.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.849+0000 7f2a9df06740 10 client.29519 fetch_fsmap learned FSMap version 36 2024-09-19T09:17:57.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.849+0000 7f2a9df06740 1 -- 172.21.15.120:0/2558423398 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d30141da60 con 0x55d30139ea90 2024-09-19T09:17:57.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.849+0000 7f2a7d7fa640 1 -- 172.21.15.120:0/2558423398 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a8c033820 con 0x55d30139ea90 2024-09-19T09:17:57.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.850+0000 7f2a7d7fa640 1 -- 172.21.15.120:0/2558423398 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f2a84045050 con 0x55d30139ea90 2024-09-19T09:17:57.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.850+0000 7f2a9df06740 10 client.29519 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:17:57.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.850+0000 7f2a9df06740 10 client.29519 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:17:57.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.850+0000 7f2a9df06740 20 client.29519 populate_metadata read hostname 'smithi120' 2024-09-19T09:17:57.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.850+0000 7f2a9df06740 1 --2- 172.21.15.120:0/2558423398 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d3013b2fc0 0x55d3014202f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:57.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.850+0000 7f2a9df06740 4 client.29519 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.45, "await_for": 45.0}] 2024-09-19T09:17:57.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.850+0000 7f2a9df06740 1 -- 172.21.15.120:0/2558423398 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.45, "await_for": 45.0}) -- 0x55d3012fbbe0 con 0x55d3013b2fc0 2024-09-19T09:17:57.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.850+0000 7f2a998c9640 1 --2- 172.21.15.120:0/2558423398 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d3013b2fc0 0x55d3014202f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:57.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.850+0000 7f2a7d7fa640 10 client.29519 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:57.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:57.850+0000 7f2a998c9640 1 --2- 172.21.15.120:0/2558423398 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d3013b2fc0 0x55d3014202f0 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cc0a482e3def603 server_cookie=9f9d9a1b496d0db8 in_seq=0 out_seq=0 2024-09-19T09:17:58.356 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55745 28.19 MB/sec warmup 113 sec latency 590.478 ms 2024-09-19T09:17:58.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:58.806+0000 7f2a7d7fa640 1 -- 172.21.15.120:0/2558423398 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d3012fbbe0 con 0x55d3013b2fc0 2024-09-19T09:17:58.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:58.806+0000 7f2a7d7fa640 10 client.29519 handle_command_reply: tid=0 2024-09-19T09:17:58.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:58.806+0000 7f2a9df06740 1 client.29519 shutdown 2024-09-19T09:17:58.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:58.807+0000 7f2a9df06740 1 -- 172.21.15.120:0/2558423398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30139ea90 msgr2=0x55d3013ba3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:58.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:58.807+0000 7f2a9df06740 1 --2- 172.21.15.120:0/2558423398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30139ea90 0x55d3013ba3e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2a84003e00 tx=0x7f2a840042d0 comp rx=0 tx=0).stop 2024-09-19T09:17:58.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:58.807+0000 7f2a9df06740 1 -- 172.21.15.120:0/2558423398 shutdown_connections 2024-09-19T09:17:58.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:58.807+0000 7f2a9df06740 1 --2- 172.21.15.120:0/2558423398 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d3013b2fc0 0x55d3014202f0 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:58.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:58.807+0000 7f2a9df06740 1 --2- 172.21.15.120:0/2558423398 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d30139f3b0 0x55d3013ba920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:58.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:58.807+0000 7f2a9df06740 1 --2- 172.21.15.120:0/2558423398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d30139ea90 0x55d3013ba3e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:58.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:58.807+0000 7f2a9df06740 1 --2- 172.21.15.120:0/2558423398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d301399a50 0x55d3013ab610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:58.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:58.807+0000 7f2a7d7fa640 0 client.29519 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:17:58.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:58.807+0000 7f2a9df06740 1 -- 172.21.15.120:0/2558423398 >> 172.21.15.120:0/2558423398 conn(0x55d30138b890 msgr2=0x55d3013a4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:58.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:58.807+0000 7f2a9df06740 1 -- 172.21.15.120:0/2558423398 shutdown_connections 2024-09-19T09:17:58.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:58.807+0000 7f2a9df06740 1 -- 172.21.15.120:0/2558423398 wait complete. 2024-09-19T09:17:58.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:58.807+0000 7f2a9df06740 20 client.29519 trim_cache size 0 max 16384 2024-09-19T09:17:58.809 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:17:58.809 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 36, 2024-09-19T09:17:58.810 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:17:58.810 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 50, 2024-09-19T09:17:58.810 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:17:58.810 INFO:teuthology.orchestra.run.smithi120.stdout: "cc768a2c": { 2024-09-19T09:17:58.810 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 50, 2024-09-19T09:17:58.810 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:17:58.810 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:17:58.810 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:17:58.810 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:17:58.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:17:58.810 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:17:58.810 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 149.4, 2024-09-19T09:17:58.811 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:17:58.811 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:17:58.811 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:17:58.811 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:17:58.811 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:17:58.811 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:17:58.811 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:17:58.811 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:17:58.811 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:17:58.811 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:17:58.811 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:17:58.811 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:17:58.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:58.808+0000 7f2a9bb2d640 1 -- 172.21.15.120:0/2647616519 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2a800a32c0 msgr2=0x7f2a800a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:58.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:58.808+0000 7f2a9bb2d640 1 --2- 172.21.15.120:0/2647616519 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2a800a32c0 0x7f2a800a5780 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2a9406e9f0 tx=0x7f2a84005f90 comp rx=0 tx=0).stop 2024-09-19T09:17:58.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:58.808+0000 7f2a9bb2d640 1 -- 172.21.15.120:0/2647616519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a94109450 msgr2=0x7f2a941a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:58.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:58.808+0000 7f2a9bb2d640 1 --2- 172.21.15.120:0/2647616519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a94109450 0x7f2a941a46a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2a9000e9c0 tx=0x7f2a9000ee80 comp rx=0 tx=0).stop 2024-09-19T09:17:58.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:58.808+0000 7f2a9bb2d640 1 -- 172.21.15.120:0/2647616519 shutdown_connections 2024-09-19T09:17:58.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:58.808+0000 7f2a9bb2d640 1 --2- 172.21.15.120:0/2647616519 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2a800a32c0 0x7f2a800a5780 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:58.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:58.808+0000 7f2a9bb2d640 1 --2- 172.21.15.120:0/2647616519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a9410e890 0x7f2a9419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:58.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:58.808+0000 7f2a9bb2d640 1 --2- 172.21.15.120:0/2647616519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a94109d70 0x7f2a941a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:58.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:58.808+0000 7f2a9bb2d640 1 --2- 172.21.15.120:0/2647616519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a94109450 0x7f2a941a46a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:58.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:58.808+0000 7f2a9bb2d640 1 -- 172.21.15.120:0/2647616519 >> 172.21.15.120:0/2647616519 conn(0x7f2a9406e2d0 msgr2=0x7f2a9410c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:58.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:58.808+0000 7f2a9bb2d640 1 -- 172.21.15.120:0/2647616519 shutdown_connections 2024-09-19T09:17:58.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:17:58.808+0000 7f2a9bb2d640 1 -- 172.21.15.120:0/2647616519 wait complete. 2024-09-19T09:17:59.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:17:58 smithi120 ceph-mon[31150]: pgmap v798: 97 pgs: 97 active+clean; 312 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.9 MiB/s wr, 793 op/s 2024-09-19T09:17:59.356 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55751 27.94 MB/sec warmup 114 sec latency 1480.294 ms 2024-09-19T09:17:59.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:17:58 smithi196 ceph-mon[35150]: pgmap v798: 97 pgs: 97 active+clean; 312 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.9 MiB/s wr, 793 op/s 2024-09-19T09:17:59.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:17:58 smithi153 ceph-mon[35071]: pgmap v798: 97 pgs: 97 active+clean; 312 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.9 MiB/s wr, 793 op/s 2024-09-19T09:17:59.512 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'cc768a2c', quiesce took 4.9 seconds. Will release after: 24.549999999999997 2024-09-19T09:18:00.356 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55751 27.70 MB/sec warmup 115 sec latency 2480.378 ms 2024-09-19T09:18:01.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:00 smithi120 ceph-mon[31150]: pgmap v799: 97 pgs: 97 active+clean; 317 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 9.1 MiB/s wr, 722 op/s 2024-09-19T09:18:01.356 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55751 27.46 MB/sec warmup 116 sec latency 3480.514 ms 2024-09-19T09:18:01.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:00 smithi196 ceph-mon[35150]: pgmap v799: 97 pgs: 97 active+clean; 317 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 9.1 MiB/s wr, 722 op/s 2024-09-19T09:18:01.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:00 smithi153 ceph-mon[35071]: pgmap v799: 97 pgs: 97 active+clean; 317 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 9.1 MiB/s wr, 722 op/s 2024-09-19T09:18:02.356 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55751 27.23 MB/sec warmup 117 sec latency 4480.631 ms 2024-09-19T09:18:03.239 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:02 smithi196 ceph-mon[35150]: pgmap v800: 97 pgs: 97 active+clean; 319 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 9.0 MiB/s wr, 719 op/s 2024-09-19T09:18:03.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:02 smithi120 ceph-mon[31150]: pgmap v800: 97 pgs: 97 active+clean; 319 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 9.0 MiB/s wr, 719 op/s 2024-09-19T09:18:03.356 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55751 27.00 MB/sec warmup 118 sec latency 5480.729 ms 2024-09-19T09:18:03.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:02 smithi153 ceph-mon[35071]: pgmap v800: 97 pgs: 97 active+clean; 319 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 9.0 MiB/s wr, 719 op/s 2024-09-19T09:18:04.356 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55751 26.77 MB/sec warmup 119 sec latency 6480.777 ms 2024-09-19T09:18:05.240 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:04 smithi153 ceph-mon[35071]: pgmap v801: 97 pgs: 97 active+clean; 309 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 8.4 MiB/s wr, 705 op/s 2024-09-19T09:18:05.395 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:04 smithi120 ceph-mon[31150]: pgmap v801: 97 pgs: 97 active+clean; 309 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 8.4 MiB/s wr, 705 op/s 2024-09-19T09:18:05.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:04 smithi196 ceph-mon[35150]: pgmap v801: 97 pgs: 97 active+clean; 309 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 8.4 MiB/s wr, 705 op/s 2024-09-19T09:18:05.616 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:18:05.867 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:18:06.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.339+0000 7f9043d66640 1 -- 172.21.15.120:0/1618347979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f903c100a50 msgr2=0x7f903c100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.339+0000 7f9043d66640 1 --2- 172.21.15.120:0/1618347979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f903c100a50 0x7f903c100e50 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f9038009900 tx=0x7f903802a410 comp rx=0 tx=0).stop 2024-09-19T09:18:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.339+0000 7f9043d66640 1 -- 172.21.15.120:0/1618347979 shutdown_connections 2024-09-19T09:18:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.339+0000 7f9043d66640 1 --2- 172.21.15.120:0/1618347979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f903c102600 0x7f903c108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.339+0000 7f9043d66640 1 --2- 172.21.15.120:0/1618347979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f903c101c40 0x7f903c1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.339+0000 7f9043d66640 1 --2- 172.21.15.120:0/1618347979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f903c100a50 0x7f903c100e50 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.339+0000 7f9043d66640 1 -- 172.21.15.120:0/1618347979 >> 172.21.15.120:0/1618347979 conn(0x7f903c0fc1b0 msgr2=0x7f903c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.339+0000 7f9043d66640 1 -- 172.21.15.120:0/1618347979 shutdown_connections 2024-09-19T09:18:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.339+0000 7f9043d66640 1 -- 172.21.15.120:0/1618347979 wait complete. 2024-09-19T09:18:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.340+0000 7f9043d66640 1 Processor -- start 2024-09-19T09:18:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.340+0000 7f9043d66640 1 -- start start 2024-09-19T09:18:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.340+0000 7f9043d66640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f903c100a50 0x7f903c19a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.340+0000 7f9043d66640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f903c101c40 0x7f903c19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.340+0000 7f9043d66640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f903c102600 0x7f903c1a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.340+0000 7f9043d66640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f903c10b5a0 con 0x7f903c100a50 2024-09-19T09:18:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.340+0000 7f9043d66640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f903c10b420 con 0x7f903c102600 2024-09-19T09:18:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.340+0000 7f9043d66640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f903c10b720 con 0x7f903c101c40 2024-09-19T09:18:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.340+0000 7f9041b02640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f903c100a50 0x7f903c19a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.340+0000 7f9041b02640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f903c100a50 0x7f903c19a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60464/0 (socket says 172.21.15.120:60464) 2024-09-19T09:18:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.340+0000 7f9041b02640 1 -- 172.21.15.120:0/3751211187 learned_addr learned my addr 172.21.15.120:0/3751211187 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.340+0000 7f9041301640 1 --2- 172.21.15.120:0/3751211187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f903c101c40 0x7f903c19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.340+0000 7f9042303640 1 --2- 172.21.15.120:0/3751211187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f903c102600 0x7f903c1a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.340+0000 7f9041b02640 1 -- 172.21.15.120:0/3751211187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f903c101c40 msgr2=0x7f903c19a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.340+0000 7f9041b02640 1 --2- 172.21.15.120:0/3751211187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f903c101c40 0x7f903c19a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.340+0000 7f9041b02640 1 -- 172.21.15.120:0/3751211187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f903c102600 msgr2=0x7f903c1a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.340+0000 7f9041b02640 1 --2- 172.21.15.120:0/3751211187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f903c102600 0x7f903c1a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.340+0000 7f9041b02640 1 -- 172.21.15.120:0/3751211187 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f903c1a1c50 con 0x7f903c100a50 2024-09-19T09:18:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.340+0000 7f9041301640 1 --2- 172.21.15.120:0/3751211187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f903c101c40 0x7f903c19a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:18:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.340+0000 7f9042303640 1 --2- 172.21.15.120:0/3751211187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f903c102600 0x7f903c1a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:18:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.341+0000 7f9041b02640 1 --2- 172.21.15.120:0/3751211187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f903c100a50 0x7f903c19a030 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f9038031610 tx=0x7f90380043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.341+0000 7f9032ffd640 1 -- 172.21.15.120:0/3751211187 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903803f030 con 0x7f903c100a50 2024-09-19T09:18:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.341+0000 7f9032ffd640 1 -- 172.21.15.120:0/3751211187 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9038003ce0 con 0x7f903c100a50 2024-09-19T09:18:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.341+0000 7f9032ffd640 1 -- 172.21.15.120:0/3751211187 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9038035070 con 0x7f903c100a50 2024-09-19T09:18:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.341+0000 7f9043d66640 1 -- 172.21.15.120:0/3751211187 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f903c1a1ee0 con 0x7f903c100a50 2024-09-19T09:18:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.341+0000 7f9043d66640 1 -- 172.21.15.120:0/3751211187 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f903c1a22f0 con 0x7f903c100a50 2024-09-19T09:18:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.343+0000 7f9032ffd640 1 -- 172.21.15.120:0/3751211187 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f9038042030 con 0x7f903c100a50 2024-09-19T09:18:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.344+0000 7f9032ffd640 1 --2- 172.21.15.120:0/3751211187 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f90280a32f0 0x7f90280a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.344+0000 7f9041301640 1 --2- 172.21.15.120:0/3751211187 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f90280a32f0 0x7f90280a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.344+0000 7f9032ffd640 1 -- 172.21.15.120:0/3751211187 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f90380eff90 con 0x7f903c100a50 2024-09-19T09:18:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.344+0000 7f9041301640 1 --2- 172.21.15.120:0/3751211187 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f90280a32f0 0x7f90280a57b0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f902c005d00 tx=0x7f902c005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.345+0000 7f9046bdd740 1 Processor -- start 2024-09-19T09:18:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.345+0000 7f9046bdd740 1 -- start start 2024-09-19T09:18:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.345+0000 7f9046bdd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd315a69a0 0x55bd315a6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.345+0000 7f9046bdd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd315a72e0 0x55bd315abda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.345+0000 7f9046bdd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd315ac370 0x55bd315ae830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.345+0000 7f9046bdd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bd3150a2f0 con 0x55bd315a72e0 2024-09-19T09:18:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.345+0000 7f9046bdd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bd314b2270 con 0x55bd315a69a0 2024-09-19T09:18:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.345+0000 7f9046bdd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bd315062b0 con 0x55bd315ac370 2024-09-19T09:18:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.345+0000 7f9041301640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd315a72e0 0x55bd315abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.345+0000 7f9041301640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd315a72e0 0x55bd315abda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60480/0 (socket says 172.21.15.120:60480) 2024-09-19T09:18:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.345+0000 7f9041301640 1 -- 172.21.15.120:0/961091619 learned_addr learned my addr 172.21.15.120:0/961091619 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.345+0000 7f9041b02640 1 --2- 172.21.15.120:0/961091619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd315a69a0 0x55bd315a6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.345+0000 7f9041301640 1 -- 172.21.15.120:0/961091619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd315ac370 msgr2=0x55bd315ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.345+0000 7f9041301640 1 --2- 172.21.15.120:0/961091619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd315ac370 0x55bd315ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.345+0000 7f9041301640 1 -- 172.21.15.120:0/961091619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd315a69a0 msgr2=0x55bd315a6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.345+0000 7f9041301640 1 --2- 172.21.15.120:0/961091619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd315a69a0 0x55bd315a6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.346+0000 7f9041301640 1 -- 172.21.15.120:0/961091619 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd314b1340 con 0x55bd315a72e0 2024-09-19T09:18:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.346+0000 7f9041301640 1 --2- 172.21.15.120:0/961091619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd315a72e0 0x55bd315abda0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f902c00b540 tx=0x7f902c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc32ca4a24360b65 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.346+0000 7f9027fff640 1 -- 172.21.15.120:0/961091619 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c003d70 con 0x55bd315a72e0 2024-09-19T09:18:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.346+0000 7f9027fff640 1 -- 172.21.15.120:0/961091619 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f902c024070 con 0x55bd315a72e0 2024-09-19T09:18:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.346+0000 7f9027fff640 1 -- 172.21.15.120:0/961091619 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c0116a0 con 0x55bd315a72e0 2024-09-19T09:18:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.347+0000 7f9046bdd740 1 -- 172.21.15.120:0/961091619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd315a72e0 msgr2=0x55bd315abda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.347+0000 7f9046bdd740 1 --2- 172.21.15.120:0/961091619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd315a72e0 0x55bd315abda0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f902c00b540 tx=0x7f902c012000 comp rx=0 tx=0).stop 2024-09-19T09:18:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.347+0000 7f9046bdd740 1 -- 172.21.15.120:0/961091619 shutdown_connections 2024-09-19T09:18:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.347+0000 7f9046bdd740 1 --2- 172.21.15.120:0/961091619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd315ac370 0x55bd315ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.347+0000 7f9046bdd740 1 --2- 172.21.15.120:0/961091619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd315a72e0 0x55bd315abda0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.347+0000 7f9046bdd740 1 --2- 172.21.15.120:0/961091619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd315a69a0 0x55bd315a6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.347+0000 7f9046bdd740 1 -- 172.21.15.120:0/961091619 >> 172.21.15.120:0/961091619 conn(0x55bd315a2010 msgr2=0x55bd315a2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.347+0000 7f9046bdd740 1 -- 172.21.15.120:0/961091619 shutdown_connections 2024-09-19T09:18:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.347+0000 7f9046bdd740 1 -- 172.21.15.120:0/961091619 wait complete. 2024-09-19T09:18:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.347+0000 7f9046bdd740 1 -- start start 2024-09-19T09:18:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.347+0000 7f9046bdd740 1 Processor -- start 2024-09-19T09:18:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.347+0000 7f9046bdd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd315a69a0 0x55bd315b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.348+0000 7f9046bdd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd315a72e0 0x55bd315b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.348+0000 7f9046bdd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd315ac370 0x55bd315b8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.348+0000 7f9046bdd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bd315af340 con 0x55bd315a72e0 2024-09-19T09:18:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.348+0000 7f9046bdd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bd31493290 con 0x55bd315a69a0 2024-09-19T09:18:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.348+0000 7f9046bdd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bd315af4c0 con 0x55bd315ac370 2024-09-19T09:18:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.348+0000 7f9046bdd740 10 client.0 _ll_register_callbacks cb 0x55bd314c4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.348+0000 7f9041301640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd315a72e0 0x55bd315b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.348+0000 7f9041301640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd315a72e0 0x55bd315b11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60490/0 (socket says 172.21.15.120:60490) 2024-09-19T09:18:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.348+0000 7f9041301640 1 -- 172.21.15.120:0/1980474872 learned_addr learned my addr 172.21.15.120:0/1980474872 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.348+0000 7f9041b02640 1 --2- 172.21.15.120:0/1980474872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd315a69a0 0x55bd315b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.348+0000 7f9042303640 1 --2- 172.21.15.120:0/1980474872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd315ac370 0x55bd315b8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.348+0000 7f9041301640 1 -- 172.21.15.120:0/1980474872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd315ac370 msgr2=0x55bd315b8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.348+0000 7f9041301640 1 --2- 172.21.15.120:0/1980474872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd315ac370 0x55bd315b8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.348+0000 7f9041301640 1 -- 172.21.15.120:0/1980474872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd315a69a0 msgr2=0x55bd315b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.348+0000 7f9041301640 1 --2- 172.21.15.120:0/1980474872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd315a69a0 0x55bd315b0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.348+0000 7f9041301640 1 -- 172.21.15.120:0/1980474872 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd3121d210 con 0x55bd315a72e0 2024-09-19T09:18:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.348+0000 7f9041b02640 1 --2- 172.21.15.120:0/1980474872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd315a69a0 0x55bd315b0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:18:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.348+0000 7f9041301640 1 --2- 172.21.15.120:0/1980474872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd315a72e0 0x55bd315b11f0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f902c00b5d0 tx=0x7f902c003da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e809a81683d1da27 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.348+0000 7f90257fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:18:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.348+0000 7f90257fa640 1 -- 172.21.15.120:0/1980474872 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c01f870 con 0x55bd315a72e0 2024-09-19T09:18:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.348+0000 7f90257fa640 1 -- 172.21.15.120:0/1980474872 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f902c029480 con 0x55bd315a72e0 2024-09-19T09:18:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.348+0000 7f9046bdd740 1 -- 172.21.15.120:0/1980474872 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f902c024070 con 0x55bd315a72e0 2024-09-19T09:18:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.348+0000 7f90257fa640 1 -- 172.21.15.120:0/1980474872 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f902c029680 con 0x55bd315a72e0 2024-09-19T09:18:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.348+0000 7f9046bdd740 10 client.20580 fetch_fsmap learned FSMap version 36 2024-09-19T09:18:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.348+0000 7f9046bdd740 1 -- 172.21.15.120:0/1980474872 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55bd312e9620 con 0x55bd315a72e0 2024-09-19T09:18:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.349+0000 7f90257fa640 1 -- 172.21.15.120:0/1980474872 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f902c024070 con 0x55bd315a72e0 2024-09-19T09:18:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.349+0000 7f90257fa640 1 -- 172.21.15.120:0/1980474872 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f902c032070 con 0x55bd315a72e0 2024-09-19T09:18:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.349+0000 7f9046bdd740 10 client.20580 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:18:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.349+0000 7f9046bdd740 10 client.20580 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:18:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.349+0000 7f9046bdd740 20 client.20580 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.349+0000 7f9046bdd740 1 --2- 172.21.15.120:0/1980474872 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bd315bc760 0x55bd315beb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.349+0000 7f9046bdd740 4 client.20580 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:18:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.349+0000 7f9046bdd740 1 -- 172.21.15.120:0/1980474872 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd31517a20 con 0x55bd315bc760 2024-09-19T09:18:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.349+0000 7f9041b02640 1 --2- 172.21.15.120:0/1980474872 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bd315bc760 0x55bd315beb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.350+0000 7f90257fa640 10 client.20580 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.350+0000 7f9041b02640 1 --2- 172.21.15.120:0/1980474872 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bd315bc760 0x55bd315beb80 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae9d4d24f7f478c server_cookie=74e8b6b860c6faff in_seq=0 out_seq=0 2024-09-19T09:18:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.350+0000 7f90257fa640 1 -- 172.21.15.120:0/1980474872 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bd31517a20 con 0x55bd315bc760 2024-09-19T09:18:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.350+0000 7f90257fa640 10 client.20580 handle_command_reply: tid=0 2024-09-19T09:18:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.350+0000 7f9046bdd740 1 client.20580 shutdown 2024-09-19T09:18:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.350+0000 7f9046bdd740 1 -- 172.21.15.120:0/1980474872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd315a72e0 msgr2=0x55bd315b11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.350+0000 7f9046bdd740 1 --2- 172.21.15.120:0/1980474872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd315a72e0 0x55bd315b11f0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f902c00b5d0 tx=0x7f902c003da0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.350+0000 7f9046bdd740 1 -- 172.21.15.120:0/1980474872 shutdown_connections 2024-09-19T09:18:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.350+0000 7f9046bdd740 1 --2- 172.21.15.120:0/1980474872 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bd315bc760 0x55bd315beb80 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.350+0000 7f9046bdd740 1 --2- 172.21.15.120:0/1980474872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd315ac370 0x55bd315b8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.350+0000 7f9046bdd740 1 --2- 172.21.15.120:0/1980474872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd315a72e0 0x55bd315b11f0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.350+0000 7f9046bdd740 1 --2- 172.21.15.120:0/1980474872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd315a69a0 0x55bd315b0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.350+0000 7f90257fa640 0 client.20580 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.350+0000 7f9046bdd740 1 -- 172.21.15.120:0/1980474872 >> 172.21.15.120:0/1980474872 conn(0x55bd315a2010 msgr2=0x55bd315a3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.351+0000 7f9046bdd740 1 -- 172.21.15.120:0/1980474872 shutdown_connections 2024-09-19T09:18:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.351+0000 7f9046bdd740 1 -- 172.21.15.120:0/1980474872 wait complete. 2024-09-19T09:18:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.351+0000 7f9046bdd740 20 client.20580 trim_cache size 0 max 16384 2024-09-19T09:18:06.359 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55751 0.00 MB/sec execute 1 sec latency 8480.894 ms 2024-09-19T09:18:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.365+0000 7f9046bdd740 1 Processor -- start 2024-09-19T09:18:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.365+0000 7f9046bdd740 1 -- start start 2024-09-19T09:18:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.365+0000 7f9046bdd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd315a69a0 0x55bd315a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.365+0000 7f9046bdd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd315a72e0 0x55bd315b81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.365+0000 7f9046bdd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd315ac370 0x55bd315c3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.365+0000 7f9046bdd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1ab2a0 con 0x55bd315a72e0 2024-09-19T09:18:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.365+0000 7f9046bdd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1ace70 con 0x55bd315a69a0 2024-09-19T09:18:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.365+0000 7f9046bdd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1ab970 con 0x55bd315ac370 2024-09-19T09:18:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.365+0000 7f9041301640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd315a72e0 0x55bd315b81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.365+0000 7f9041301640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd315a72e0 0x55bd315b81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60506/0 (socket says 172.21.15.120:60506) 2024-09-19T09:18:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.365+0000 7f9041301640 1 -- 172.21.15.120:0/955777641 learned_addr learned my addr 172.21.15.120:0/955777641 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.365+0000 7f9042303640 1 --2- 172.21.15.120:0/955777641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd315ac370 0x55bd315c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.365+0000 7f9041b02640 1 --2- 172.21.15.120:0/955777641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd315a69a0 0x55bd315a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.365+0000 7f9041301640 1 -- 172.21.15.120:0/955777641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd315ac370 msgr2=0x55bd315c3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.365+0000 7f9041301640 1 --2- 172.21.15.120:0/955777641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd315ac370 0x55bd315c3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.365+0000 7f9041301640 1 -- 172.21.15.120:0/955777641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd315a69a0 msgr2=0x55bd315a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.365+0000 7f9041301640 1 --2- 172.21.15.120:0/955777641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd315a69a0 0x55bd315a10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.365+0000 7f9041301640 1 -- 172.21.15.120:0/955777641 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd315b07f0 con 0x55bd315a72e0 2024-09-19T09:18:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.366+0000 7f9041301640 1 --2- 172.21.15.120:0/955777641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd315a72e0 0x55bd315b81f0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f902c004770 tx=0x7f902c00fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7429b7b1ed10eeda server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.366+0000 7f90277fe640 1 -- 172.21.15.120:0/955777641 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c034440 con 0x55bd315a72e0 2024-09-19T09:18:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.366+0000 7f90277fe640 1 -- 172.21.15.120:0/955777641 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f902c024070 con 0x55bd315a72e0 2024-09-19T09:18:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.366+0000 7f90277fe640 1 -- 172.21.15.120:0/955777641 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c029b10 con 0x55bd315a72e0 2024-09-19T09:18:06.371 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.366+0000 7f9046bdd740 1 -- 172.21.15.120:0/955777641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd315a72e0 msgr2=0x55bd315b81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.366+0000 7f9046bdd740 1 --2- 172.21.15.120:0/955777641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd315a72e0 0x55bd315b81f0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f902c004770 tx=0x7f902c00fd60 comp rx=0 tx=0).stop 2024-09-19T09:18:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.366+0000 7f9046bdd740 1 -- 172.21.15.120:0/955777641 shutdown_connections 2024-09-19T09:18:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.366+0000 7f9046bdd740 1 --2- 172.21.15.120:0/955777641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd315ac370 0x55bd315c3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.366+0000 7f9046bdd740 1 --2- 172.21.15.120:0/955777641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd315a72e0 0x55bd315b81f0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.366+0000 7f9046bdd740 1 --2- 172.21.15.120:0/955777641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd315a69a0 0x55bd315a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.366+0000 7f9046bdd740 1 -- 172.21.15.120:0/955777641 >> 172.21.15.120:0/955777641 conn(0x55bd315a2010 msgr2=0x55bd315a09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.366+0000 7f9046bdd740 1 -- 172.21.15.120:0/955777641 shutdown_connections 2024-09-19T09:18:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.367+0000 7f9046bdd740 1 -- 172.21.15.120:0/955777641 wait complete. 2024-09-19T09:18:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.367+0000 7f9046bdd740 1 -- start start 2024-09-19T09:18:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.367+0000 7f9046bdd740 1 Processor -- start 2024-09-19T09:18:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.367+0000 7f9046bdd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd315a69a0 0x55bd315b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.367+0000 7f9046bdd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd315a72e0 0x55bd315b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.367+0000 7f9046bdd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd315ac370 0x55bd315b6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.367+0000 7f9046bdd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1ac7a0 con 0x55bd315a72e0 2024-09-19T09:18:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.367+0000 7f9046bdd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1ac040 con 0x55bd315ac370 2024-09-19T09:18:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.367+0000 7f9046bdd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1a9260 con 0x55bd315a69a0 2024-09-19T09:18:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.367+0000 7f9046bdd740 10 client.0 _ll_register_callbacks cb 0x7f902c00bee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.367+0000 7f9041301640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd315a72e0 0x55bd315b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.367+0000 7f9041301640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd315a72e0 0x55bd315b5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60512/0 (socket says 172.21.15.120:60512) 2024-09-19T09:18:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.367+0000 7f9041301640 1 -- 172.21.15.120:0/231494089 learned_addr learned my addr 172.21.15.120:0/231494089 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.368+0000 7f9041b02640 1 --2- 172.21.15.120:0/231494089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd315a69a0 0x55bd315b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.368+0000 7f9042303640 1 --2- 172.21.15.120:0/231494089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd315ac370 0x55bd315b6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.368+0000 7f9041301640 1 -- 172.21.15.120:0/231494089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd315a69a0 msgr2=0x55bd315b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.368+0000 7f9041301640 1 --2- 172.21.15.120:0/231494089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd315a69a0 0x55bd315b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.368+0000 7f9041301640 1 -- 172.21.15.120:0/231494089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd315ac370 msgr2=0x55bd315b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.368+0000 7f9041301640 1 --2- 172.21.15.120:0/231494089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd315ac370 0x55bd315b6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.374 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.368+0000 7f9041301640 1 -- 172.21.15.120:0/231494089 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd315bf1e0 con 0x55bd315a72e0 2024-09-19T09:18:06.374 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.368+0000 7f9041b02640 1 --2- 172.21.15.120:0/231494089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd315a69a0 0x55bd315b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:18:06.374 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.368+0000 7f9041301640 1 --2- 172.21.15.120:0/231494089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd315a72e0 0x55bd315b5f10 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f902c02d040 tx=0x7f902c00bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b20b56a944c9c99 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:06.374 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.368+0000 7f90257fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:18:06.374 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.368+0000 7f90257fa640 1 -- 172.21.15.120:0/231494089 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c043050 con 0x55bd315a72e0 2024-09-19T09:18:06.374 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.368+0000 7f90257fa640 1 -- 172.21.15.120:0/231494089 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f902c01f870 con 0x55bd315a72e0 2024-09-19T09:18:06.374 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.368+0000 7f9046bdd740 1 -- 172.21.15.120:0/231494089 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f902c024070 con 0x55bd315a72e0 2024-09-19T09:18:06.374 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.368+0000 7f90257fa640 1 -- 172.21.15.120:0/231494089 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f902c01fa70 con 0x55bd315a72e0 2024-09-19T09:18:06.374 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.368+0000 7f9046bdd740 10 client.20586 fetch_fsmap learned FSMap version 36 2024-09-19T09:18:06.374 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.368+0000 7f9046bdd740 1 -- 172.21.15.120:0/231494089 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55bd316273e0 con 0x55bd315a72e0 2024-09-19T09:18:06.374 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.368+0000 7f90257fa640 1 -- 172.21.15.120:0/231494089 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f902c024070 con 0x55bd315a72e0 2024-09-19T09:18:06.374 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.369+0000 7f90257fa640 1 -- 172.21.15.120:0/231494089 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f902c032040 con 0x55bd315a72e0 2024-09-19T09:18:06.374 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.369+0000 7f9046bdd740 10 client.20586 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:18:06.374 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.369+0000 7f9046bdd740 10 client.20586 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:18:06.375 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.369+0000 7f9046bdd740 20 client.20586 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:06.375 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.369+0000 7f9046bdd740 1 --2- 172.21.15.120:0/231494089 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bd31627a30 0x55bd31629e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.375 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.369+0000 7f9046bdd740 4 client.20586 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:18:06.375 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.369+0000 7f9046bdd740 1 -- 172.21.15.120:0/231494089 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bd31511d70 con 0x55bd31627a30 2024-09-19T09:18:06.375 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.369+0000 7f9041b02640 1 --2- 172.21.15.120:0/231494089 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bd31627a30 0x55bd31629e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.375 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.369+0000 7f90257fa640 10 client.20586 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:06.375 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.369+0000 7f9041b02640 1 --2- 172.21.15.120:0/231494089 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bd31627a30 0x55bd31629e50 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75eb5d1406abe4fb server_cookie=c0bd42388f9cac5 in_seq=0 out_seq=0 2024-09-19T09:18:06.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.394+0000 7f3215b86640 1 -- 172.21.15.120:0/4140449923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3210102570 msgr2=0x7f32101029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:06.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.394+0000 7f3215b86640 1 --2- 172.21.15.120:0/4140449923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3210102570 0x7f32101029b0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f3204009900 tx=0x7f320402a480 comp rx=0 tx=0).stop 2024-09-19T09:18:06.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.394+0000 7f3215b86640 1 -- 172.21.15.120:0/4140449923 shutdown_connections 2024-09-19T09:18:06.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.394+0000 7f3215b86640 1 --2- 172.21.15.120:0/4140449923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3210102ef0 0x7f321010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.394+0000 7f3215b86640 1 --2- 172.21.15.120:0/4140449923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3210102570 0x7f32101029b0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.394+0000 7f3215b86640 1 --2- 172.21.15.120:0/4140449923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3210100e20 0x7f3210101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.394+0000 7f3215b86640 1 -- 172.21.15.120:0/4140449923 >> 172.21.15.120:0/4140449923 conn(0x7f32100fc7d0 msgr2=0x7f32100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:06.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.394+0000 7f3215b86640 1 -- 172.21.15.120:0/4140449923 shutdown_connections 2024-09-19T09:18:06.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.394+0000 7f3215b86640 1 -- 172.21.15.120:0/4140449923 wait complete. 2024-09-19T09:18:06.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.395+0000 7f3215b86640 1 Processor -- start 2024-09-19T09:18:06.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.395+0000 7f3215b86640 1 -- start start 2024-09-19T09:18:06.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.395+0000 7f3215b86640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3210100e20 0x7f32100fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.395+0000 7f3215b86640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3210102570 0x7f32100fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.395+0000 7f3215b86640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3210102ef0 0x7f32100fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.395+0000 7f3215b86640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3210110f80 con 0x7f3210102570 2024-09-19T09:18:06.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.395+0000 7f3215b86640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3210110e00 con 0x7f3210102ef0 2024-09-19T09:18:06.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.395+0000 7f3215b86640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3210111100 con 0x7f3210100e20 2024-09-19T09:18:06.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.395+0000 7f320effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3210102570 0x7f32100fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.395+0000 7f320effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3210102570 0x7f32100fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60530/0 (socket says 172.21.15.120:60530) 2024-09-19T09:18:06.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.395+0000 7f320effd640 1 -- 172.21.15.120:0/3794036003 learned_addr learned my addr 172.21.15.120:0/3794036003 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:06.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.395+0000 7f320ffff640 1 --2- 172.21.15.120:0/3794036003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3210102ef0 0x7f32100fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.395+0000 7f320f7fe640 1 --2- 172.21.15.120:0/3794036003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3210100e20 0x7f32100fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.396+0000 7f320effd640 1 -- 172.21.15.120:0/3794036003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3210100e20 msgr2=0x7f32100fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:06.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.396+0000 7f320effd640 1 --2- 172.21.15.120:0/3794036003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3210100e20 0x7f32100fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.396+0000 7f320effd640 1 -- 172.21.15.120:0/3794036003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3210102ef0 msgr2=0x7f32100fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:06.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.396+0000 7f320effd640 1 --2- 172.21.15.120:0/3794036003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3210102ef0 0x7f32100fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.396+0000 7f320effd640 1 -- 172.21.15.120:0/3794036003 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32101a69f0 con 0x7f3210102570 2024-09-19T09:18:06.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.396+0000 7f320effd640 1 --2- 172.21.15.120:0/3794036003 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3210102570 0x7f32100fd1c0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f3204006010 tx=0x7f3204003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:06.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.396+0000 7f320cff9640 1 -- 172.21.15.120:0/3794036003 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320403f070 con 0x7f3210102570 2024-09-19T09:18:06.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.396+0000 7f320cff9640 1 -- 172.21.15.120:0/3794036003 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3204031c30 con 0x7f3210102570 2024-09-19T09:18:06.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.396+0000 7f3215b86640 1 -- 172.21.15.120:0/3794036003 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32101a6c80 con 0x7f3210102570 2024-09-19T09:18:06.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.396+0000 7f320cff9640 1 -- 172.21.15.120:0/3794036003 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3204035030 con 0x7f3210102570 2024-09-19T09:18:06.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.396+0000 7f3215b86640 1 -- 172.21.15.120:0/3794036003 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32101a71b0 con 0x7f3210102570 2024-09-19T09:18:06.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.398+0000 7f320cff9640 1 -- 172.21.15.120:0/3794036003 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f3204042070 con 0x7f3210102570 2024-09-19T09:18:06.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.399+0000 7f3217f5f740 1 Processor -- start 2024-09-19T09:18:06.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.399+0000 7f3217f5f740 1 -- start start 2024-09-19T09:18:06.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.399+0000 7f3217f5f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f846924a0 0x557f84692880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.399+0000 7f3217f5f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f84692dc0 0x557f8469d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.399+0000 7f3217f5f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f8469d680 0x557f8469fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.399+0000 7f3217f5f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557f845f82e0 con 0x557f846924a0 2024-09-19T09:18:06.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.399+0000 7f320ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f846924a0 0x557f84692880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.399+0000 7f320cff9640 1 --2- 172.21.15.120:0/3794036003 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f31f80a32c0 0x7f31f80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.399+0000 7f320ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f846924a0 0x557f84692880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60538/0 (socket says 172.21.15.120:60538) 2024-09-19T09:18:06.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.399+0000 7f320ffff640 1 -- 172.21.15.120:0/114639877 learned_addr learned my addr 172.21.15.120:0/114639877 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:06.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.399+0000 7f3217f5f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557f8459fce0 con 0x557f8469d680 2024-09-19T09:18:06.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.399+0000 7f3217f5f740 1 -- 172.21.15.120:0/114639877 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557f845f42a0 con 0x557f84692dc0 2024-09-19T09:18:06.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.399+0000 7f320effd640 1 --2- 172.21.15.120:0/114639877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f8469d680 0x557f8469fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.400+0000 7f320f7fe640 1 --2- 172.21.15.120:0/114639877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f84692dc0 0x557f8469d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.400+0000 7f320cff9640 1 -- 172.21.15.120:0/3794036003 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f32040f0000 con 0x7f3210102570 2024-09-19T09:18:06.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.400+0000 7f320ffff640 1 --2- 172.21.15.120:0/3794036003 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f31f80a32c0 0x7f31f80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.400+0000 7f320ffff640 1 -- 172.21.15.120:0/114639877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f84692dc0 msgr2=0x557f8469d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:06.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.400+0000 7f320ffff640 1 --2- 172.21.15.120:0/114639877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f84692dc0 0x557f8469d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.400+0000 7f320ffff640 1 -- 172.21.15.120:0/114639877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f8469d680 msgr2=0x557f8469fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:06.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.400+0000 7f320ffff640 1 --2- 172.21.15.120:0/114639877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f8469d680 0x557f8469fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.400+0000 7f320ffff640 1 -- 172.21.15.120:0/114639877 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f84584440 con 0x557f846924a0 2024-09-19T09:18:06.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.400+0000 7f320effd640 1 --2- 172.21.15.120:0/114639877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f8469d680 0x557f8469fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:18:06.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.400+0000 7f320ffff640 1 --2- 172.21.15.120:0/114639877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f846924a0 0x557f84692880 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f3208014a60 tx=0x7f3208014f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f566bccd825389c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:06.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.400+0000 7f31f5ffb640 1 -- 172.21.15.120:0/114639877 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320801ec50 con 0x557f846924a0 2024-09-19T09:18:06.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.400+0000 7f320ffff640 1 --2- 172.21.15.120:0/3794036003 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f31f80a32c0 0x7f31f80a5780 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3208022000 tx=0x7f320801e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:06.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.400+0000 7f31f5ffb640 1 -- 172.21.15.120:0/114639877 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3208023070 con 0x557f846924a0 2024-09-19T09:18:06.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.400+0000 7f31f5ffb640 1 -- 172.21.15.120:0/114639877 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320800e360 con 0x557f846924a0 2024-09-19T09:18:06.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.401+0000 7f3217f5f740 1 -- 172.21.15.120:0/114639877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f846924a0 msgr2=0x557f84692880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:06.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.401+0000 7f3217f5f740 1 --2- 172.21.15.120:0/114639877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f846924a0 0x557f84692880 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f3208014a60 tx=0x7f3208014f20 comp rx=0 tx=0).stop 2024-09-19T09:18:06.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.401+0000 7f3217f5f740 1 -- 172.21.15.120:0/114639877 shutdown_connections 2024-09-19T09:18:06.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.401+0000 7f3217f5f740 1 --2- 172.21.15.120:0/114639877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f8469d680 0x557f8469fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.401+0000 7f3217f5f740 1 --2- 172.21.15.120:0/114639877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f84692dc0 0x557f8469d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.401+0000 7f3217f5f740 1 --2- 172.21.15.120:0/114639877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f846924a0 0x557f84692880 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.401+0000 7f3217f5f740 1 -- 172.21.15.120:0/114639877 >> 172.21.15.120:0/114639877 conn(0x557f8468ea30 msgr2=0x557f8468ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:06.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.401+0000 7f3217f5f740 1 -- 172.21.15.120:0/114639877 shutdown_connections 2024-09-19T09:18:06.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.401+0000 7f3217f5f740 1 -- 172.21.15.120:0/114639877 wait complete. 2024-09-19T09:18:06.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.401+0000 7f3217f5f740 1 -- start start 2024-09-19T09:18:06.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.401+0000 7f3217f5f740 1 Processor -- start 2024-09-19T09:18:06.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.402+0000 7f3217f5f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f846924a0 0x557f8469d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.402+0000 7f3217f5f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f84692dc0 0x557f8469a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.402+0000 7f3217f5f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f8469d680 0x557f8469a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.402+0000 7f3217f5f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557f846a0250 con 0x557f846924a0 2024-09-19T09:18:06.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.402+0000 7f3217f5f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557f845b3050 con 0x557f8469d680 2024-09-19T09:18:06.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.402+0000 7f3217f5f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557f846a03d0 con 0x557f84692dc0 2024-09-19T09:18:06.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.402+0000 7f3217f5f740 10 client.0 _ll_register_callbacks cb 0x557f845f3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:06.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.402+0000 7f320f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f846924a0 0x557f8469d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.402+0000 7f320f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f846924a0 0x557f8469d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60550/0 (socket says 172.21.15.120:60550) 2024-09-19T09:18:06.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.402+0000 7f320f7fe640 1 -- 172.21.15.120:0/899704108 learned_addr learned my addr 172.21.15.120:0/899704108 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:06.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.402+0000 7f320ffff640 1 --2- 172.21.15.120:0/899704108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f8469d680 0x557f8469a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.402+0000 7f320effd640 1 --2- 172.21.15.120:0/899704108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f84692dc0 0x557f8469a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.402+0000 7f320f7fe640 1 -- 172.21.15.120:0/899704108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f84692dc0 msgr2=0x557f8469a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:06.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.402+0000 7f320f7fe640 1 --2- 172.21.15.120:0/899704108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f84692dc0 0x557f8469a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.402+0000 7f320f7fe640 1 -- 172.21.15.120:0/899704108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f8469d680 msgr2=0x557f8469a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:06.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.402+0000 7f320f7fe640 1 --2- 172.21.15.120:0/899704108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f8469d680 0x557f8469a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.402+0000 7f320f7fe640 1 -- 172.21.15.120:0/899704108 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f843693c0 con 0x557f846924a0 2024-09-19T09:18:06.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.402+0000 7f320f7fe640 1 --2- 172.21.15.120:0/899704108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f846924a0 0x557f8469d000 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f320000be70 tx=0x7f3200012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ac978bd770d7b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:06.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.402+0000 7f31eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:18:06.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.402+0000 7f31eeffd640 1 -- 172.21.15.120:0/899704108 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3200005220 con 0x557f846924a0 2024-09-19T09:18:06.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.402+0000 7f31eeffd640 1 -- 172.21.15.120:0/899704108 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f32000053c0 con 0x557f846924a0 2024-09-19T09:18:06.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.403+0000 7f3217f5f740 1 -- 172.21.15.120:0/899704108 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3208023070 con 0x557f846924a0 2024-09-19T09:18:06.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.403+0000 7f31eeffd640 1 -- 172.21.15.120:0/899704108 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f32000055c0 con 0x557f846924a0 2024-09-19T09:18:06.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.403+0000 7f3217f5f740 10 client.20598 fetch_fsmap learned FSMap version 36 2024-09-19T09:18:06.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.403+0000 7f3217f5f740 1 -- 172.21.15.120:0/899704108 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557f846abd20 con 0x557f846924a0 2024-09-19T09:18:06.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.403+0000 7f31eeffd640 1 -- 172.21.15.120:0/899704108 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3208023070 con 0x557f846924a0 2024-09-19T09:18:06.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.403+0000 7f31eeffd640 1 -- 172.21.15.120:0/899704108 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f3200019030 con 0x557f846924a0 2024-09-19T09:18:06.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.403+0000 7f3217f5f740 10 client.20598 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:18:06.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.403+0000 7f3217f5f740 10 client.20598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:18:06.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.403+0000 7f3217f5f740 20 client.20598 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:06.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.403+0000 7f3217f5f740 1 --2- 172.21.15.120:0/899704108 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557f846ad280 0x557f846af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.403+0000 7f3217f5f740 4 client.20598 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:18:06.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.403+0000 7f3217f5f740 1 -- 172.21.15.120:0/899704108 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f84605850 con 0x557f846ad280 2024-09-19T09:18:06.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.403+0000 7f320effd640 1 --2- 172.21.15.120:0/899704108 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557f846ad280 0x557f846af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.404+0000 7f31eeffd640 10 client.20598 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:06.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.404+0000 7f320effd640 1 --2- 172.21.15.120:0/899704108 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557f846ad280 0x557f846af6e0 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d62d70b9a9d52ff3 server_cookie=83acc8377a689927 in_seq=0 out_seq=0 2024-09-19T09:18:06.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.432+0000 7f31eeffd640 1 -- 172.21.15.120:0/899704108 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557f84605850 con 0x557f846ad280 2024-09-19T09:18:06.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.432+0000 7f31eeffd640 10 client.20598 handle_command_reply: tid=0 2024-09-19T09:18:06.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.432+0000 7f3217f5f740 1 client.20598 shutdown 2024-09-19T09:18:06.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.433+0000 7f3217f5f740 1 -- 172.21.15.120:0/899704108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f846924a0 msgr2=0x557f8469d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:06.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.433+0000 7f3217f5f740 1 --2- 172.21.15.120:0/899704108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f846924a0 0x557f8469d000 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f320000be70 tx=0x7f3200012820 comp rx=0 tx=0).stop 2024-09-19T09:18:06.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.433+0000 7f3217f5f740 1 -- 172.21.15.120:0/899704108 shutdown_connections 2024-09-19T09:18:06.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.433+0000 7f3217f5f740 1 --2- 172.21.15.120:0/899704108 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557f846ad280 0x557f846af6e0 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.433+0000 7f3217f5f740 1 --2- 172.21.15.120:0/899704108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f8469d680 0x557f8469a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.433+0000 7f3217f5f740 1 --2- 172.21.15.120:0/899704108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f84692dc0 0x557f8469a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.433+0000 7f3217f5f740 1 --2- 172.21.15.120:0/899704108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f846924a0 0x557f8469d000 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.433+0000 7f31eeffd640 0 client.20598 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:06.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.433+0000 7f3217f5f740 1 -- 172.21.15.120:0/899704108 >> 172.21.15.120:0/899704108 conn(0x557f8468ea30 msgr2=0x557f84694710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:06.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.433+0000 7f3217f5f740 1 -- 172.21.15.120:0/899704108 shutdown_connections 2024-09-19T09:18:06.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.433+0000 7f3217f5f740 1 -- 172.21.15.120:0/899704108 wait complete. 2024-09-19T09:18:06.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.433+0000 7f3217f5f740 20 client.20598 trim_cache size 0 max 16384 2024-09-19T09:18:06.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.465+0000 7f3217f5f740 1 Processor -- start 2024-09-19T09:18:06.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.469+0000 7f3217f5f740 1 -- start start 2024-09-19T09:18:06.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.469+0000 7f3217f5f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f846924a0 0x557f846b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.469+0000 7f3217f5f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f84692dc0 0x557f846b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.469+0000 7f3217f5f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f8469d680 0x557f84697510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.469+0000 7f3217f5f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f32101af6e0 con 0x557f8469d680 2024-09-19T09:18:06.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.469+0000 7f3217f5f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f32101b12b0 con 0x557f84692dc0 2024-09-19T09:18:06.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.469+0000 7f3217f5f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f32101afdb0 con 0x557f846924a0 2024-09-19T09:18:06.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.470+0000 7f320ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f8469d680 0x557f84697510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.470+0000 7f320ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f8469d680 0x557f84697510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60562/0 (socket says 172.21.15.120:60562) 2024-09-19T09:18:06.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.470+0000 7f320ffff640 1 -- 172.21.15.120:0/2349081746 learned_addr learned my addr 172.21.15.120:0/2349081746 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:06.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.471+0000 7f320ffff640 1 -- 172.21.15.120:0/2349081746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f846924a0 msgr2=0x557f846b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:06.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.471+0000 7f320ffff640 1 --2- 172.21.15.120:0/2349081746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f846924a0 0x557f846b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.471+0000 7f320ffff640 1 -- 172.21.15.120:0/2349081746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f84692dc0 msgr2=0x557f846b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:06.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.471+0000 7f320ffff640 1 --2- 172.21.15.120:0/2349081746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f84692dc0 0x557f846b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.471+0000 7f320ffff640 1 -- 172.21.15.120:0/2349081746 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f84699260 con 0x557f8469d680 2024-09-19T09:18:06.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.471+0000 7f320ffff640 1 --2- 172.21.15.120:0/2349081746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f8469d680 0x557f84697510 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f3208012820 tx=0x7f320801d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0089df0cb53c304 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:06.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.472+0000 7f31f57fa640 1 -- 172.21.15.120:0/2349081746 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3208014560 con 0x557f8469d680 2024-09-19T09:18:06.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.472+0000 7f31f57fa640 1 -- 172.21.15.120:0/2349081746 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3208036070 con 0x557f8469d680 2024-09-19T09:18:06.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.472+0000 7f31f57fa640 1 -- 172.21.15.120:0/2349081746 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3208004c00 con 0x557f8469d680 2024-09-19T09:18:06.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.473+0000 7f3217f5f740 1 -- 172.21.15.120:0/2349081746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f8469d680 msgr2=0x557f84697510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:06.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.473+0000 7f3217f5f740 1 --2- 172.21.15.120:0/2349081746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f8469d680 0x557f84697510 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f3208012820 tx=0x7f320801d940 comp rx=0 tx=0).stop 2024-09-19T09:18:06.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.476+0000 7f3217f5f740 1 -- 172.21.15.120:0/2349081746 shutdown_connections 2024-09-19T09:18:06.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.476+0000 7f3217f5f740 1 --2- 172.21.15.120:0/2349081746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f8469d680 0x557f84697510 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.476+0000 7f3217f5f740 1 --2- 172.21.15.120:0/2349081746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f84692dc0 0x557f846b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.476+0000 7f3217f5f740 1 --2- 172.21.15.120:0/2349081746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f846924a0 0x557f846b4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.476+0000 7f3217f5f740 1 -- 172.21.15.120:0/2349081746 >> 172.21.15.120:0/2349081746 conn(0x557f8468ea30 msgr2=0x557f8468db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:06.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.481+0000 7f3217f5f740 1 -- 172.21.15.120:0/2349081746 shutdown_connections 2024-09-19T09:18:06.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.484+0000 7f3217f5f740 1 -- 172.21.15.120:0/2349081746 wait complete. 2024-09-19T09:18:06.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.485+0000 7f3217f5f740 1 -- start start 2024-09-19T09:18:06.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.485+0000 7f3217f5f740 1 Processor -- start 2024-09-19T09:18:06.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.486+0000 7f3217f5f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f846924a0 0x557f846a3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.486+0000 7f3217f5f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f84692dc0 0x557f846a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.486+0000 7f3217f5f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f8469d680 0x557f846a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.486+0000 7f3217f5f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f32101b0be0 con 0x557f8469d680 2024-09-19T09:18:06.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.486+0000 7f3217f5f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f32101b0480 con 0x557f84692dc0 2024-09-19T09:18:06.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.486+0000 7f3217f5f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f32101ad6a0 con 0x557f846924a0 2024-09-19T09:18:06.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.486+0000 7f3217f5f740 10 client.0 _ll_register_callbacks cb 0x7f320000ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:06.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.489+0000 7f320ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f8469d680 0x557f846a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.489+0000 7f320ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f8469d680 0x557f846a3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60572/0 (socket says 172.21.15.120:60572) 2024-09-19T09:18:06.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.489+0000 7f320ffff640 1 -- 172.21.15.120:0/32575774 learned_addr learned my addr 172.21.15.120:0/32575774 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:06.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.489+0000 7f320effd640 1 --2- 172.21.15.120:0/32575774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f84692dc0 0x557f846a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.491+0000 7f320f7fe640 1 --2- 172.21.15.120:0/32575774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f846924a0 0x557f846a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.494+0000 7f320effd640 1 -- 172.21.15.120:0/32575774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f846924a0 msgr2=0x557f846a3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:06.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.495+0000 7f320effd640 1 --2- 172.21.15.120:0/32575774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f846924a0 0x557f846a3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.495+0000 7f320effd640 1 -- 172.21.15.120:0/32575774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f8469d680 msgr2=0x557f846a3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:06.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.495+0000 7f320effd640 1 --2- 172.21.15.120:0/32575774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f8469d680 0x557f846a3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.495+0000 7f320effd640 1 -- 172.21.15.120:0/32575774 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f84717230 con 0x557f84692dc0 2024-09-19T09:18:06.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.495+0000 7f320effd640 1 --2- 172.21.15.120:0/32575774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f84692dc0 0x557f846a3a80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f32040edb90 tx=0x7f320408b510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b04d5a420b8e9f1c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:06.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.498+0000 7f31eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:18:06.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.498+0000 7f31eeffd640 1 -- 172.21.15.120:0/32575774 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3204049040 con 0x557f84692dc0 2024-09-19T09:18:06.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.498+0000 7f31eeffd640 1 -- 172.21.15.120:0/32575774 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f32040b8d80 con 0x557f84692dc0 2024-09-19T09:18:06.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.506+0000 7f3217f5f740 1 -- 172.21.15.120:0/32575774 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3208036070 con 0x557f84692dc0 2024-09-19T09:18:06.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.507+0000 7f31eeffd640 1 -- 172.21.15.120:0/32575774 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f32040bc710 con 0x557f84692dc0 2024-09-19T09:18:06.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.509+0000 7f3217f5f740 10 client.29543 fetch_fsmap learned FSMap version 36 2024-09-19T09:18:06.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.509+0000 7f3217f5f740 1 -- 172.21.15.120:0/32575774 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557f84718540 con 0x557f84692dc0 2024-09-19T09:18:06.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.510+0000 7f31eeffd640 1 -- 172.21.15.120:0/32575774 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3208036070 con 0x557f84692dc0 2024-09-19T09:18:06.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.510+0000 7f31eeffd640 1 -- 172.21.15.120:0/32575774 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f32040c4e40 con 0x557f84692dc0 2024-09-19T09:18:06.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.512+0000 7f3217f5f740 10 client.29543 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:18:06.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.512+0000 7f3217f5f740 10 client.29543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:18:06.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.512+0000 7f3217f5f740 20 client.29543 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:06.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.512+0000 7f3217f5f740 1 --2- 172.21.15.120:0/32575774 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557f84718bc0 0x557f8471afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.512+0000 7f3217f5f740 4 client.29543 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:18:06.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.512+0000 7f3217f5f740 1 -- 172.21.15.120:0/32575774 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557f8468e300 con 0x557f84718bc0 2024-09-19T09:18:06.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.514+0000 7f320f7fe640 1 --2- 172.21.15.120:0/32575774 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557f84718bc0 0x557f8471afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.522+0000 7f320f7fe640 1 --2- 172.21.15.120:0/32575774 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557f84718bc0 0x557f8471afe0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62580d207c2d1a04 server_cookie=a651ca3bee23e123 in_seq=0 out_seq=0 2024-09-19T09:18:06.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.523+0000 7f31eeffd640 10 client.29543 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:06.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.523+0000 7f31eeffd640 1 -- 172.21.15.120:0/32575774 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557f8468e300 con 0x557f84718bc0 2024-09-19T09:18:06.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.523+0000 7f31eeffd640 10 client.29543 handle_command_reply: tid=0 2024-09-19T09:18:06.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.524+0000 7f3217f5f740 1 client.29543 shutdown 2024-09-19T09:18:06.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.526+0000 7f3217f5f740 1 -- 172.21.15.120:0/32575774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f84692dc0 msgr2=0x557f846a3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:06.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.526+0000 7f3217f5f740 1 --2- 172.21.15.120:0/32575774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f84692dc0 0x557f846a3a80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f32040edb90 tx=0x7f320408b510 comp rx=0 tx=0).stop 2024-09-19T09:18:06.529 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:18:06.529 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:18:06.529 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:18:06.529 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:18:06.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.527+0000 7f3217f5f740 1 -- 172.21.15.120:0/32575774 shutdown_connections 2024-09-19T09:18:06.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.527+0000 7f3217f5f740 1 --2- 172.21.15.120:0/32575774 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557f84718bc0 0x557f8471afe0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.527+0000 7f3217f5f740 1 --2- 172.21.15.120:0/32575774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f8469d680 0x557f846a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.527+0000 7f3217f5f740 1 --2- 172.21.15.120:0/32575774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f84692dc0 0x557f846a3a80 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.527+0000 7f3217f5f740 1 --2- 172.21.15.120:0/32575774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f846924a0 0x557f846a3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.527+0000 7f3217f5f740 1 -- 172.21.15.120:0/32575774 >> 172.21.15.120:0/32575774 conn(0x557f8468ea30 msgr2=0x557f8469bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:06.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.527+0000 7f31eeffd640 0 client.29543 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:06.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.527+0000 7f3217f5f740 1 -- 172.21.15.120:0/32575774 shutdown_connections 2024-09-19T09:18:06.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.527+0000 7f3217f5f740 1 -- 172.21.15.120:0/32575774 wait complete. 2024-09-19T09:18:06.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.528+0000 7f3217f5f740 20 client.29543 trim_cache size 0 max 16384 2024-09-19T09:18:06.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.528+0000 7f3215b86640 1 -- 172.21.15.120:0/3794036003 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f31f80a32c0 msgr2=0x7f31f80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:06.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.528+0000 7f3215b86640 1 --2- 172.21.15.120:0/3794036003 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f31f80a32c0 0x7f31f80a5780 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3208022000 tx=0x7f320801e000 comp rx=0 tx=0).stop 2024-09-19T09:18:06.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.528+0000 7f3215b86640 1 -- 172.21.15.120:0/3794036003 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3210102570 msgr2=0x7f32100fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:06.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.528+0000 7f3215b86640 1 --2- 172.21.15.120:0/3794036003 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3210102570 0x7f32100fd1c0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f3204006010 tx=0x7f3204003c10 comp rx=0 tx=0).stop 2024-09-19T09:18:06.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.529+0000 7f3215b86640 1 -- 172.21.15.120:0/3794036003 shutdown_connections 2024-09-19T09:18:06.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.529+0000 7f3215b86640 1 --2- 172.21.15.120:0/3794036003 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f31f80a32c0 0x7f31f80a5780 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.529+0000 7f3215b86640 1 --2- 172.21.15.120:0/3794036003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3210102ef0 0x7f32100fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.529+0000 7f3215b86640 1 --2- 172.21.15.120:0/3794036003 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3210102570 0x7f32100fd1c0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.529+0000 7f3215b86640 1 --2- 172.21.15.120:0/3794036003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3210100e20 0x7f32100fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.529+0000 7f3215b86640 1 -- 172.21.15.120:0/3794036003 >> 172.21.15.120:0/3794036003 conn(0x7f32100fc7d0 msgr2=0x7f321006a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:06.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.529+0000 7f3215b86640 1 -- 172.21.15.120:0/3794036003 shutdown_connections 2024-09-19T09:18:06.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:06.529+0000 7f3215b86640 1 -- 172.21.15.120:0/3794036003 wait complete. 2024-09-19T09:18:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.606+0000 7f90257fa640 1 -- 172.21.15.120:0/231494089 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bd31511d70 con 0x55bd31627a30 2024-09-19T09:18:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.607+0000 7f90257fa640 10 client.20586 handle_command_reply: tid=0 2024-09-19T09:18:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.607+0000 7f9046bdd740 1 client.20586 shutdown 2024-09-19T09:18:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.607+0000 7f9046bdd740 1 -- 172.21.15.120:0/231494089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd315a72e0 msgr2=0x55bd315b5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.607+0000 7f9046bdd740 1 --2- 172.21.15.120:0/231494089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd315a72e0 0x55bd315b5f10 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f902c02d040 tx=0x7f902c00bd40 comp rx=0 tx=0).stop 2024-09-19T09:18:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.607+0000 7f9046bdd740 1 -- 172.21.15.120:0/231494089 shutdown_connections 2024-09-19T09:18:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.607+0000 7f9046bdd740 1 --2- 172.21.15.120:0/231494089 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bd31627a30 0x55bd31629e50 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.607+0000 7f9046bdd740 1 --2- 172.21.15.120:0/231494089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd315ac370 0x55bd315b6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.607+0000 7f9046bdd740 1 --2- 172.21.15.120:0/231494089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd315a72e0 0x55bd315b5f10 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.607+0000 7f9046bdd740 1 --2- 172.21.15.120:0/231494089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd315a69a0 0x55bd315b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.607+0000 7f90257fa640 0 client.20586 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.607+0000 7f9046bdd740 1 -- 172.21.15.120:0/231494089 >> 172.21.15.120:0/231494089 conn(0x55bd315a2010 msgr2=0x55bd315a11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.607+0000 7f9046bdd740 1 -- 172.21.15.120:0/231494089 shutdown_connections 2024-09-19T09:18:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.607+0000 7f9046bdd740 1 -- 172.21.15.120:0/231494089 wait complete. 2024-09-19T09:18:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.607+0000 7f9046bdd740 20 client.20586 trim_cache size 0 max 16384 2024-09-19T09:18:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:18:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:18:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:18:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:18:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.608+0000 7f9043d66640 1 -- 172.21.15.120:0/3751211187 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f90280a32f0 msgr2=0x7f90280a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.608+0000 7f9043d66640 1 --2- 172.21.15.120:0/3751211187 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f90280a32f0 0x7f90280a57b0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f902c005d00 tx=0x7f902c005c70 comp rx=0 tx=0).stop 2024-09-19T09:18:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.608+0000 7f9043d66640 1 -- 172.21.15.120:0/3751211187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f903c100a50 msgr2=0x7f903c19a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.608+0000 7f9043d66640 1 --2- 172.21.15.120:0/3751211187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f903c100a50 0x7f903c19a030 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f9038031610 tx=0x7f90380043b0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.608+0000 7f9043d66640 1 -- 172.21.15.120:0/3751211187 shutdown_connections 2024-09-19T09:18:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.608+0000 7f9043d66640 1 --2- 172.21.15.120:0/3751211187 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f90280a32f0 0x7f90280a57b0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.608+0000 7f9043d66640 1 --2- 172.21.15.120:0/3751211187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f903c102600 0x7f903c1a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.608+0000 7f9043d66640 1 --2- 172.21.15.120:0/3751211187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f903c101c40 0x7f903c19a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.608+0000 7f9043d66640 1 --2- 172.21.15.120:0/3751211187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f903c100a50 0x7f903c19a030 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.608+0000 7f9043d66640 1 -- 172.21.15.120:0/3751211187 >> 172.21.15.120:0/3751211187 conn(0x7f903c0fc1b0 msgr2=0x7f903c0fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.608+0000 7f9043d66640 1 -- 172.21.15.120:0/3751211187 shutdown_connections 2024-09-19T09:18:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:06.608+0000 7f9043d66640 1 -- 172.21.15.120:0/3751211187 wait complete. 2024-09-19T09:18:07.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:06 smithi120 ceph-mon[31150]: pgmap v802: 97 pgs: 97 active+clean; 309 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.8 MiB/s wr, 562 op/s 2024-09-19T09:18:07.356 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55751 0.00 MB/sec execute 2 sec latency 9480.976 ms 2024-09-19T09:18:07.400 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:18:07.400 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:18:07.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:06 smithi196 ceph-mon[35150]: pgmap v802: 97 pgs: 97 active+clean; 309 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.8 MiB/s wr, 562 op/s 2024-09-19T09:18:07.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:06 smithi153 ceph-mon[35071]: pgmap v802: 97 pgs: 97 active+clean; 309 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.8 MiB/s wr, 562 op/s 2024-09-19T09:18:07.645 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:18:08.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:08 smithi120 ceph-mon[31150]: pgmap v803: 97 pgs: 97 active+clean; 301 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.4 MiB/s wr, 532 op/s 2024-09-19T09:18:08.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.176+0000 7f0036eb9640 1 -- 172.21.15.120:0/316022694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0030103740 msgr2=0x7f0030107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:08.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.176+0000 7f0036eb9640 1 --2- 172.21.15.120:0/316022694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0030103740 0x7f0030107b80 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f0024009930 tx=0x7f002402a420 comp rx=0 tx=0).stop 2024-09-19T09:18:08.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.176+0000 7f0036eb9640 1 -- 172.21.15.120:0/316022694 shutdown_connections 2024-09-19T09:18:08.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.176+0000 7f0036eb9640 1 --2- 172.21.15.120:0/316022694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0030108290 0x7f003010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:08.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.176+0000 7f0036eb9640 1 --2- 172.21.15.120:0/316022694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0030103740 0x7f0030107b80 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:08.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.176+0000 7f0036eb9640 1 --2- 172.21.15.120:0/316022694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0030102d90 0x7f0030103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:08.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.176+0000 7f0036eb9640 1 -- 172.21.15.120:0/316022694 >> 172.21.15.120:0/316022694 conn(0x7f00300fc7d0 msgr2=0x7f00300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:08.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.176+0000 7f0036eb9640 1 -- 172.21.15.120:0/316022694 shutdown_connections 2024-09-19T09:18:08.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.176+0000 7f0036eb9640 1 -- 172.21.15.120:0/316022694 wait complete. 2024-09-19T09:18:08.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.176+0000 7f0036eb9640 1 Processor -- start 2024-09-19T09:18:08.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.176+0000 7f0036eb9640 1 -- start start 2024-09-19T09:18:08.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.177+0000 7f0036eb9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0030102d90 0x7f00301009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:08.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.177+0000 7f0036eb9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0030103740 0x7f0030100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:08.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.177+0000 7f0036eb9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0030108290 0x7f0030101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:08.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.177+0000 7f0036eb9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0030112090 con 0x7f0030103740 2024-09-19T09:18:08.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.177+0000 7f0036eb9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0030111f10 con 0x7f0030102d90 2024-09-19T09:18:08.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.177+0000 7f0036eb9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0030112210 con 0x7f0030108290 2024-09-19T09:18:08.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.177+0000 7f002ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0030103740 0x7f0030100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:08.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.177+0000 7f002ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0030103740 0x7f0030100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60596/0 (socket says 172.21.15.120:60596) 2024-09-19T09:18:08.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.177+0000 7f002ffff640 1 -- 172.21.15.120:0/4267437559 learned_addr learned my addr 172.21.15.120:0/4267437559 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:08.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.177+0000 7f0034c55640 1 --2- 172.21.15.120:0/4267437559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0030102d90 0x7f00301009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:08.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.177+0000 7f0035456640 1 --2- 172.21.15.120:0/4267437559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0030108290 0x7f0030101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:08.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.177+0000 7f002ffff640 1 -- 172.21.15.120:0/4267437559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0030108290 msgr2=0x7f0030101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:08.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.177+0000 7f002ffff640 1 --2- 172.21.15.120:0/4267437559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0030108290 0x7f0030101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:08.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.177+0000 7f002ffff640 1 -- 172.21.15.120:0/4267437559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0030102d90 msgr2=0x7f00301009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:08.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.177+0000 7f002ffff640 1 --2- 172.21.15.120:0/4267437559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0030102d90 0x7f00301009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:08.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.177+0000 7f002ffff640 1 -- 172.21.15.120:0/4267437559 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00301a6a70 con 0x7f0030103740 2024-09-19T09:18:08.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.177+0000 7f002ffff640 1 --2- 172.21.15.120:0/4267437559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0030103740 0x7f0030100f30 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f0024005940 tx=0x7f00240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:08.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.177+0000 7f002dffb640 1 -- 172.21.15.120:0/4267437559 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002403f030 con 0x7f0030103740 2024-09-19T09:18:08.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.177+0000 7f0036eb9640 1 -- 172.21.15.120:0/4267437559 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00301a6d00 con 0x7f0030103740 2024-09-19T09:18:08.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.177+0000 7f002dffb640 1 -- 172.21.15.120:0/4267437559 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0024003ce0 con 0x7f0030103740 2024-09-19T09:18:08.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.177+0000 7f002dffb640 1 -- 172.21.15.120:0/4267437559 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024035070 con 0x7f0030103740 2024-09-19T09:18:08.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.178+0000 7f0036eb9640 1 -- 172.21.15.120:0/4267437559 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00301a71d0 con 0x7f0030103740 2024-09-19T09:18:08.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.179+0000 7f0036eb9640 1 -- 172.21.15.120:0/4267437559 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0030103170 con 0x7f0030103740 2024-09-19T09:18:08.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.180+0000 7f002dffb640 1 -- 172.21.15.120:0/4267437559 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f0024042030 con 0x7f0030103740 2024-09-19T09:18:08.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.180+0000 7f002dffb640 1 --2- 172.21.15.120:0/4267437559 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f00180a3240 0x7f00180a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:08.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.181+0000 7f0034c55640 1 --2- 172.21.15.120:0/4267437559 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f00180a3240 0x7f00180a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:08.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.181+0000 7f002dffb640 1 -- 172.21.15.120:0/4267437559 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f00240f0010 con 0x7f0030103740 2024-09-19T09:18:08.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.181+0000 7f0034c55640 1 --2- 172.21.15.120:0/4267437559 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f00180a3240 0x7f00180a5700 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0020005d00 tx=0x7f002000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:08.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.184+0000 7f002dffb640 1 -- 172.21.15.120:0/4267437559 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f00240bc6e0 con 0x7f0030103740 2024-09-19T09:18:08.356 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55751 0.00 MB/sec execute 3 sec latency 10481.038 ms 2024-09-19T09:18:08.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.389+0000 7f0036eb9640 1 -- 172.21.15.120:0/4267437559 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f003010a2a0 con 0x7f0030103740 2024-09-19T09:18:08.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.390+0000 7f002dffb640 1 -- 172.21.15.120:0/4267437559 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11993 (secure 0 0 0) 0x7f00240c09e0 con 0x7f0030103740 2024-09-19T09:18:08.391 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:18:08.391 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":36,"btime":"2024-09-19T09:16:33:442503+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:16:33.001618+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:18:08.391 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 36 2024-09-19T09:18:08.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.392+0000 7f0036eb9640 1 -- 172.21.15.120:0/4267437559 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f00180a3240 msgr2=0x7f00180a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:08.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.392+0000 7f0036eb9640 1 --2- 172.21.15.120:0/4267437559 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f00180a3240 0x7f00180a5700 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0020005d00 tx=0x7f002000b410 comp rx=0 tx=0).stop 2024-09-19T09:18:08.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.392+0000 7f0036eb9640 1 -- 172.21.15.120:0/4267437559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0030103740 msgr2=0x7f0030100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:08.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.392+0000 7f0036eb9640 1 --2- 172.21.15.120:0/4267437559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0030103740 0x7f0030100f30 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f0024005940 tx=0x7f00240042f0 comp rx=0 tx=0).stop 2024-09-19T09:18:08.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.392+0000 7f0036eb9640 1 -- 172.21.15.120:0/4267437559 shutdown_connections 2024-09-19T09:18:08.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.392+0000 7f0036eb9640 1 --2- 172.21.15.120:0/4267437559 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f00180a3240 0x7f00180a5700 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:08.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.392+0000 7f0036eb9640 1 --2- 172.21.15.120:0/4267437559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0030108290 0x7f0030101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:08.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.392+0000 7f0036eb9640 1 --2- 172.21.15.120:0/4267437559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0030103740 0x7f0030100f30 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:08.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.392+0000 7f0036eb9640 1 --2- 172.21.15.120:0/4267437559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0030102d90 0x7f00301009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:08.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.392+0000 7f0036eb9640 1 -- 172.21.15.120:0/4267437559 >> 172.21.15.120:0/4267437559 conn(0x7f00300fc7d0 msgr2=0x7f0030105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:08.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.392+0000 7f0036eb9640 1 -- 172.21.15.120:0/4267437559 shutdown_connections 2024-09-19T09:18:08.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:08.392+0000 7f0036eb9640 1 -- 172.21.15.120:0/4267437559 wait complete. 2024-09-19T09:18:08.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:08 smithi196 ceph-mon[35150]: pgmap v803: 97 pgs: 97 active+clean; 301 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.4 MiB/s wr, 532 op/s 2024-09-19T09:18:08.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:08 smithi153 ceph-mon[35071]: pgmap v803: 97 pgs: 97 active+clean; 301 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.4 MiB/s wr, 532 op/s 2024-09-19T09:18:09.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:09 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/4267437559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:18:09.114 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:18:09.356 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55751 0.00 MB/sec execute 4 sec latency 11481.065 ms 2024-09-19T09:18:09.357 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:18:09.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:09 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/4267437559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:18:09.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:09 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/4267437559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:18:09.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.857+0000 7f93ff505640 1 -- 172.21.15.120:0/1762339319 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93f8102790 msgr2=0x7f93f81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:09.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.857+0000 7f93ff505640 1 --2- 172.21.15.120:0/1762339319 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93f8102790 0x7f93f81067e0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f93f4009900 tx=0x7f93f402a410 comp rx=0 tx=0).stop 2024-09-19T09:18:09.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.857+0000 7f93ff505640 1 -- 172.21.15.120:0/1762339319 shutdown_connections 2024-09-19T09:18:09.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.857+0000 7f93ff505640 1 --2- 172.21.15.120:0/1762339319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93f8106f10 0x7f93f810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.857+0000 7f93ff505640 1 --2- 172.21.15.120:0/1762339319 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93f8102790 0x7f93f81067e0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.857+0000 7f93ff505640 1 --2- 172.21.15.120:0/1762339319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93f8101de0 0x7f93f81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.857+0000 7f93ff505640 1 -- 172.21.15.120:0/1762339319 >> 172.21.15.120:0/1762339319 conn(0x7f93f80fd8c0 msgr2=0x7f93f80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:09.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.857+0000 7f93ff505640 1 -- 172.21.15.120:0/1762339319 shutdown_connections 2024-09-19T09:18:09.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.857+0000 7f93ff505640 1 -- 172.21.15.120:0/1762339319 wait complete. 2024-09-19T09:18:09.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.857+0000 7f93ff505640 1 Processor -- start 2024-09-19T09:18:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.857+0000 7f93ff505640 1 -- start start 2024-09-19T09:18:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.858+0000 7f93ff505640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93f8101de0 0x7f93f80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.858+0000 7f93ff505640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93f8102790 0x7f93f8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.858+0000 7f93ff505640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93f8106f10 0x7f93f8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.858+0000 7f93ff505640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8111f90 con 0x7f93f8102790 2024-09-19T09:18:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.858+0000 7f93ff505640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8111e10 con 0x7f93f8106f10 2024-09-19T09:18:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.858+0000 7f93ff505640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8112110 con 0x7f93f8101de0 2024-09-19T09:18:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.858+0000 7f93fcaa0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93f8102790 0x7f93f8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.858+0000 7f93fcaa0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93f8102790 0x7f93f8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60616/0 (socket says 172.21.15.120:60616) 2024-09-19T09:18:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.858+0000 7f93fcaa0640 1 -- 172.21.15.120:0/894080150 learned_addr learned my addr 172.21.15.120:0/894080150 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.858+0000 7f93fdaa2640 1 --2- 172.21.15.120:0/894080150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93f8106f10 0x7f93f8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.858+0000 7f93fd2a1640 1 --2- 172.21.15.120:0/894080150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93f8101de0 0x7f93f80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.858+0000 7f93fcaa0640 1 -- 172.21.15.120:0/894080150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93f8101de0 msgr2=0x7f93f80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:09.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.858+0000 7f93fcaa0640 1 --2- 172.21.15.120:0/894080150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93f8101de0 0x7f93f80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.858+0000 7f93fcaa0640 1 -- 172.21.15.120:0/894080150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93f8106f10 msgr2=0x7f93f8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:09.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.858+0000 7f93fcaa0640 1 --2- 172.21.15.120:0/894080150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93f8106f10 0x7f93f8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.858+0000 7f93fcaa0640 1 -- 172.21.15.120:0/894080150 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f81a6980 con 0x7f93f8102790 2024-09-19T09:18:09.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.858+0000 7f93fdaa2640 1 --2- 172.21.15.120:0/894080150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93f8106f10 0x7f93f8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:18:09.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.858+0000 7f93fcaa0640 1 --2- 172.21.15.120:0/894080150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93f8102790 0x7f93f8074a30 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f93f4005ab0 tx=0x7f93f40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:09.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.858+0000 7f93ee7fc640 1 -- 172.21.15.120:0/894080150 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f403f030 con 0x7f93f8102790 2024-09-19T09:18:09.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.858+0000 7f93ee7fc640 1 -- 172.21.15.120:0/894080150 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93f4003ce0 con 0x7f93f8102790 2024-09-19T09:18:09.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.858+0000 7f93ff505640 1 -- 172.21.15.120:0/894080150 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93f81a6c10 con 0x7f93f8102790 2024-09-19T09:18:09.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.858+0000 7f93ee7fc640 1 -- 172.21.15.120:0/894080150 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4035070 con 0x7f93f8102790 2024-09-19T09:18:09.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.858+0000 7f93ff505640 1 -- 172.21.15.120:0/894080150 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93f81a70f0 con 0x7f93f8102790 2024-09-19T09:18:09.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.861+0000 7f93ee7fc640 1 -- 172.21.15.120:0/894080150 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f93f4042030 con 0x7f93f8102790 2024-09-19T09:18:09.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.861+0000 7f94018de740 1 Processor -- start 2024-09-19T09:18:09.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.861+0000 7f94018de740 1 -- start start 2024-09-19T09:18:09.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.861+0000 7f94018de740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb20926b40 0x55bb20925090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:09.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.861+0000 7f94018de740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb209255d0 0x55bb20925a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:09.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.861+0000 7f94018de740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb20930370 0x55bb20932760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:09.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.861+0000 7f94018de740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bb2088b2f0 con 0x55bb20930370 2024-09-19T09:18:09.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.861+0000 7f94018de740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bb20832cf0 con 0x55bb20926b40 2024-09-19T09:18:09.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.861+0000 7f94018de740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bb208872b0 con 0x55bb209255d0 2024-09-19T09:18:09.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.861+0000 7f93ee7fc640 1 --2- 172.21.15.120:0/894080150 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93e40a3240 0x7f93e40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:09.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.861+0000 7f93fcaa0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb20930370 0x55bb20932760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:09.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.861+0000 7f93fcaa0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb20930370 0x55bb20932760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60632/0 (socket says 172.21.15.120:60632) 2024-09-19T09:18:09.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.861+0000 7f93fcaa0640 1 -- 172.21.15.120:0/444460494 learned_addr learned my addr 172.21.15.120:0/444460494 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:09.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.862+0000 7f93fdaa2640 1 --2- 172.21.15.120:0/894080150 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93e40a3240 0x7f93e40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:09.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.862+0000 7f93fdaa2640 1 --2- 172.21.15.120:0/444460494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb20926b40 0x55bb20925090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:09.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.862+0000 7f93fd2a1640 1 --2- 172.21.15.120:0/444460494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb209255d0 0x55bb20925a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:09.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.862+0000 7f93ee7fc640 1 -- 172.21.15.120:0/894080150 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f93f40f02d0 con 0x7f93f8102790 2024-09-19T09:18:09.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.862+0000 7f93fcaa0640 1 -- 172.21.15.120:0/444460494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb209255d0 msgr2=0x55bb20925a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:09.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.862+0000 7f93fcaa0640 1 --2- 172.21.15.120:0/444460494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb209255d0 0x55bb20925a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.862+0000 7f93fcaa0640 1 -- 172.21.15.120:0/444460494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb20926b40 msgr2=0x55bb20925090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:09.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.862+0000 7f93fcaa0640 1 --2- 172.21.15.120:0/444460494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb20926b40 0x55bb20925090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.862+0000 7f93fcaa0640 1 -- 172.21.15.120:0/444460494 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb208173a0 con 0x55bb20930370 2024-09-19T09:18:09.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.862+0000 7f93fdaa2640 1 --2- 172.21.15.120:0/444460494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb20926b40 0x55bb20925090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:18:09.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.862+0000 7f93fcaa0640 1 --2- 172.21.15.120:0/444460494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb20930370 0x55bb20932760 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f93f4004590 tx=0x7f93f40e76c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16407ad26d7d90d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:09.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.862+0000 7f93fdaa2640 1 --2- 172.21.15.120:0/894080150 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93e40a3240 0x7f93e40a5700 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f93f000afd0 tx=0x7f93f000a450 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:09.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.862+0000 7f93e37fe640 1 -- 172.21.15.120:0/444460494 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4049070 con 0x55bb20930370 2024-09-19T09:18:09.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.862+0000 7f93e37fe640 1 -- 172.21.15.120:0/444460494 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93f40bcff0 con 0x55bb20930370 2024-09-19T09:18:09.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.862+0000 7f93e37fe640 1 -- 172.21.15.120:0/444460494 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f40babf0 con 0x55bb20930370 2024-09-19T09:18:09.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.863+0000 7f94018de740 1 -- 172.21.15.120:0/444460494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb20930370 msgr2=0x55bb20932760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:09.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.863+0000 7f94018de740 1 --2- 172.21.15.120:0/444460494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb20930370 0x55bb20932760 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f93f4004590 tx=0x7f93f40e76c0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.863+0000 7f94018de740 1 -- 172.21.15.120:0/444460494 shutdown_connections 2024-09-19T09:18:09.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.863+0000 7f94018de740 1 --2- 172.21.15.120:0/444460494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb20930370 0x55bb20932760 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.863+0000 7f94018de740 1 --2- 172.21.15.120:0/444460494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb209255d0 0x55bb20925a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.863+0000 7f94018de740 1 --2- 172.21.15.120:0/444460494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb20926b40 0x55bb20925090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.863+0000 7f94018de740 1 -- 172.21.15.120:0/444460494 >> 172.21.15.120:0/444460494 conn(0x55bb208ee310 msgr2=0x55bb20921c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:09.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.863+0000 7f94018de740 1 -- 172.21.15.120:0/444460494 shutdown_connections 2024-09-19T09:18:09.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.863+0000 7f94018de740 1 -- 172.21.15.120:0/444460494 wait complete. 2024-09-19T09:18:09.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.863+0000 7f94018de740 1 -- start start 2024-09-19T09:18:09.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.863+0000 7f94018de740 1 Processor -- start 2024-09-19T09:18:09.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.864+0000 7f94018de740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb209255d0 0x55bb2092fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:09.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.864+0000 7f94018de740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb20926b40 0x55bb2092d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:09.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.864+0000 7f94018de740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb20930370 0x55bb2092d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:09.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.864+0000 7f94018de740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bb20932e30 con 0x55bb20926b40 2024-09-19T09:18:09.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.864+0000 7f94018de740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bb20846060 con 0x55bb209255d0 2024-09-19T09:18:09.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.864+0000 7f94018de740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bb20932fb0 con 0x55bb20930370 2024-09-19T09:18:09.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.864+0000 7f94018de740 10 client.0 _ll_register_callbacks cb 0x55bb205998c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:09.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.864+0000 7f93fcaa0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb20926b40 0x55bb2092d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:09.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.864+0000 7f93fcaa0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb20926b40 0x55bb2092d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60644/0 (socket says 172.21.15.120:60644) 2024-09-19T09:18:09.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.864+0000 7f93fcaa0640 1 -- 172.21.15.120:0/4001603487 learned_addr learned my addr 172.21.15.120:0/4001603487 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:09.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.864+0000 7f93fd2a1640 1 --2- 172.21.15.120:0/4001603487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb209255d0 0x55bb2092fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:09.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.864+0000 7f93fdaa2640 1 --2- 172.21.15.120:0/4001603487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb20930370 0x55bb2092d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:09.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.864+0000 7f93fcaa0640 1 -- 172.21.15.120:0/4001603487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb20930370 msgr2=0x55bb2092d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:09.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.864+0000 7f93fcaa0640 1 --2- 172.21.15.120:0/4001603487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb20930370 0x55bb2092d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.864+0000 7f93fcaa0640 1 -- 172.21.15.120:0/4001603487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb209255d0 msgr2=0x55bb2092fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:09.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.864+0000 7f93fcaa0640 1 --2- 172.21.15.120:0/4001603487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb209255d0 0x55bb2092fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.864+0000 7f93fcaa0640 1 -- 172.21.15.120:0/4001603487 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb205fc3c0 con 0x55bb20926b40 2024-09-19T09:18:09.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.864+0000 7f93fcaa0640 1 --2- 172.21.15.120:0/4001603487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb20926b40 0x55bb2092d000 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f93f40bc880 tx=0x7f93f40b7310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8256bf3411146e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:09.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.864+0000 7f93e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:18:09.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.865+0000 7f93e0ff9640 1 -- 172.21.15.120:0/4001603487 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4049040 con 0x55bb20926b40 2024-09-19T09:18:09.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.865+0000 7f93e0ff9640 1 -- 172.21.15.120:0/4001603487 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f93f40ede50 con 0x55bb20926b40 2024-09-19T09:18:09.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.865+0000 7f94018de740 1 -- 172.21.15.120:0/4001603487 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93f40bcff0 con 0x55bb20926b40 2024-09-19T09:18:09.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.865+0000 7f93e0ff9640 1 -- 172.21.15.120:0/4001603487 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f93f40bc8f0 con 0x55bb20926b40 2024-09-19T09:18:09.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.865+0000 7f94018de740 10 client.20622 fetch_fsmap learned FSMap version 36 2024-09-19T09:18:09.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.865+0000 7f94018de740 1 -- 172.21.15.120:0/4001603487 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55bb20932e30 con 0x55bb20926b40 2024-09-19T09:18:09.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.865+0000 7f93e0ff9640 1 -- 172.21.15.120:0/4001603487 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93f40bcff0 con 0x55bb20926b40 2024-09-19T09:18:09.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.865+0000 7f93e0ff9640 1 -- 172.21.15.120:0/4001603487 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f93f40bcb50 con 0x55bb20926b40 2024-09-19T09:18:09.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.865+0000 7f94018de740 10 client.20622 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:18:09.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.865+0000 7f94018de740 10 client.20622 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:18:09.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.865+0000 7f94018de740 20 client.20622 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:09.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.865+0000 7f94018de740 1 --2- 172.21.15.120:0/4001603487 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55bb2093fa00 0x55bb20941df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:09.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.865+0000 7f94018de740 4 client.20622 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:18:09.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.865+0000 7f94018de740 1 -- 172.21.15.120:0/4001603487 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb20898860 con 0x55bb2093fa00 2024-09-19T09:18:09.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.865+0000 7f93fd2a1640 1 --2- 172.21.15.120:0/4001603487 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55bb2093fa00 0x55bb20941df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:09.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.866+0000 7f93e0ff9640 10 client.20622 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:18:09.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.866+0000 7f93fd2a1640 1 --2- 172.21.15.120:0/4001603487 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55bb2093fa00 0x55bb20941df0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d9500bf947479205 server_cookie=260fcd06967c41b8 in_seq=0 out_seq=0 2024-09-19T09:18:09.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.867+0000 7f93e0ff9640 1 -- 172.21.15.120:0/4001603487 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bb20898860 con 0x55bb2093fa00 2024-09-19T09:18:09.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.867+0000 7f93e0ff9640 10 client.20622 handle_command_reply: tid=0 2024-09-19T09:18:09.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.867+0000 7f94018de740 1 client.20622 shutdown 2024-09-19T09:18:09.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.867+0000 7f94018de740 1 -- 172.21.15.120:0/4001603487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb20926b40 msgr2=0x55bb2092d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:09.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.867+0000 7f94018de740 1 --2- 172.21.15.120:0/4001603487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb20926b40 0x55bb2092d000 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f93f40bc880 tx=0x7f93f40b7310 comp rx=0 tx=0).stop 2024-09-19T09:18:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.867+0000 7f94018de740 1 -- 172.21.15.120:0/4001603487 shutdown_connections 2024-09-19T09:18:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.867+0000 7f94018de740 1 --2- 172.21.15.120:0/4001603487 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55bb2093fa00 0x55bb20941df0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.867+0000 7f94018de740 1 --2- 172.21.15.120:0/4001603487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb20930370 0x55bb2092d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.867+0000 7f94018de740 1 --2- 172.21.15.120:0/4001603487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb20926b40 0x55bb2092d000 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.867+0000 7f93e0ff9640 0 client.20622 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:18:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.867+0000 7f94018de740 1 --2- 172.21.15.120:0/4001603487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb209255d0 0x55bb2092fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.867+0000 7f94018de740 1 -- 172.21.15.120:0/4001603487 >> 172.21.15.120:0/4001603487 conn(0x55bb208ee310 msgr2=0x55bb20927990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.867+0000 7f94018de740 1 -- 172.21.15.120:0/4001603487 shutdown_connections 2024-09-19T09:18:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.867+0000 7f94018de740 1 -- 172.21.15.120:0/4001603487 wait complete. 2024-09-19T09:18:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.867+0000 7f94018de740 20 client.20622 trim_cache size 0 max 16384 2024-09-19T09:18:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.880+0000 7f94018de740 1 Processor -- start 2024-09-19T09:18:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.880+0000 7f94018de740 1 -- start start 2024-09-19T09:18:09.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.880+0000 7f94018de740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb209255d0 0x55bb2093c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.880+0000 7f94018de740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb20926b40 0x55bb209204e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.880+0000 7f94018de740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb20930370 0x55bb20928120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.880+0000 7f94018de740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81af720 con 0x55bb20926b40 2024-09-19T09:18:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.880+0000 7f94018de740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81b12f0 con 0x55bb20930370 2024-09-19T09:18:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.880+0000 7f94018de740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81afdf0 con 0x55bb209255d0 2024-09-19T09:18:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.880+0000 7f93fcaa0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb20926b40 0x55bb209204e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.880+0000 7f93fcaa0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb20926b40 0x55bb209204e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60656/0 (socket says 172.21.15.120:60656) 2024-09-19T09:18:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.880+0000 7f93fcaa0640 1 -- 172.21.15.120:0/51839388 learned_addr learned my addr 172.21.15.120:0/51839388 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.880+0000 7f93fd2a1640 1 --2- 172.21.15.120:0/51839388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb209255d0 0x55bb2093c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.880+0000 7f93fdaa2640 1 --2- 172.21.15.120:0/51839388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb20930370 0x55bb20928120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.880+0000 7f93fcaa0640 1 -- 172.21.15.120:0/51839388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb209255d0 msgr2=0x55bb2093c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.880+0000 7f93fcaa0640 1 --2- 172.21.15.120:0/51839388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb209255d0 0x55bb2093c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.880+0000 7f93fcaa0640 1 -- 172.21.15.120:0/51839388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb20930370 msgr2=0x55bb20928120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.880+0000 7f93fcaa0640 1 --2- 172.21.15.120:0/51839388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb20930370 0x55bb20928120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.880+0000 7f93fcaa0640 1 -- 172.21.15.120:0/51839388 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb20921390 con 0x55bb20926b40 2024-09-19T09:18:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.880+0000 7f93fd2a1640 1 --2- 172.21.15.120:0/51839388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb209255d0 0x55bb2093c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:18:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.880+0000 7f93fdaa2640 1 --2- 172.21.15.120:0/51839388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb20930370 0x55bb20928120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:18:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.880+0000 7f93fcaa0640 1 --2- 172.21.15.120:0/51839388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb20926b40 0x55bb209204e0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f93f40bcab0 tx=0x7f93f40bae00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc7dfbc6f44d232 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.881+0000 7f93e2ffd640 1 -- 172.21.15.120:0/51839388 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4049070 con 0x55bb20926b40 2024-09-19T09:18:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.881+0000 7f93e2ffd640 1 -- 172.21.15.120:0/51839388 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93f40d9070 con 0x55bb20926b40 2024-09-19T09:18:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.881+0000 7f93e2ffd640 1 -- 172.21.15.120:0/51839388 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f40b8f60 con 0x55bb20926b40 2024-09-19T09:18:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.881+0000 7f94018de740 1 -- 172.21.15.120:0/51839388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb20926b40 msgr2=0x55bb209204e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.881+0000 7f94018de740 1 --2- 172.21.15.120:0/51839388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb20926b40 0x55bb209204e0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f93f40bcab0 tx=0x7f93f40bae00 comp rx=0 tx=0).stop 2024-09-19T09:18:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.881+0000 7f94018de740 1 -- 172.21.15.120:0/51839388 shutdown_connections 2024-09-19T09:18:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.881+0000 7f94018de740 1 --2- 172.21.15.120:0/51839388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb20930370 0x55bb20928120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.881+0000 7f94018de740 1 --2- 172.21.15.120:0/51839388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb20926b40 0x55bb209204e0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.881+0000 7f94018de740 1 --2- 172.21.15.120:0/51839388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb209255d0 0x55bb2093c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.881+0000 7f94018de740 1 -- 172.21.15.120:0/51839388 >> 172.21.15.120:0/51839388 conn(0x55bb208ee310 msgr2=0x55bb2092f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.881+0000 7f94018de740 1 -- 172.21.15.120:0/51839388 shutdown_connections 2024-09-19T09:18:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.881+0000 7f94018de740 1 -- 172.21.15.120:0/51839388 wait complete. 2024-09-19T09:18:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.881+0000 7f94018de740 1 -- start start 2024-09-19T09:18:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.882+0000 7f94018de740 1 Processor -- start 2024-09-19T09:18:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.882+0000 7f94018de740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb209255d0 0x55bb20933380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.882+0000 7f94018de740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb20926b40 0x55bb209338c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.882+0000 7f94018de740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb20930370 0x55bb209a8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.882+0000 7f94018de740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81b0c20 con 0x55bb20930370 2024-09-19T09:18:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.882+0000 7f94018de740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81b04c0 con 0x55bb20926b40 2024-09-19T09:18:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.882+0000 7f94018de740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81ad6e0 con 0x55bb209255d0 2024-09-19T09:18:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.882+0000 7f94018de740 10 client.0 _ll_register_callbacks cb 0x7f93e800b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:09.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.882+0000 7f93fdaa2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb20930370 0x55bb209a8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.882+0000 7f93fdaa2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb20930370 0x55bb209a8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60658/0 (socket says 172.21.15.120:60658) 2024-09-19T09:18:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.882+0000 7f93fdaa2640 1 -- 172.21.15.120:0/161396793 learned_addr learned my addr 172.21.15.120:0/161396793 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.882+0000 7f93fd2a1640 1 --2- 172.21.15.120:0/161396793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb209255d0 0x55bb20933380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.882+0000 7f93fcaa0640 1 --2- 172.21.15.120:0/161396793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb20926b40 0x55bb209338c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.882+0000 7f93fdaa2640 1 -- 172.21.15.120:0/161396793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb209255d0 msgr2=0x55bb20933380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.882+0000 7f93fdaa2640 1 --2- 172.21.15.120:0/161396793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb209255d0 0x55bb20933380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.882+0000 7f93fdaa2640 1 -- 172.21.15.120:0/161396793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb20926b40 msgr2=0x55bb209338c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.882+0000 7f93fdaa2640 1 --2- 172.21.15.120:0/161396793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb20926b40 0x55bb209338c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.882+0000 7f93fdaa2640 1 -- 172.21.15.120:0/161396793 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb209a8b80 con 0x55bb20930370 2024-09-19T09:18:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.883+0000 7f93fdaa2640 1 --2- 172.21.15.120:0/161396793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb20930370 0x55bb209a8640 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f93f000eaf0 tx=0x7f93f0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8241c2dbcdbaf71a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.883+0000 7f93e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:18:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.883+0000 7f93e0ff9640 1 -- 172.21.15.120:0/161396793 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f002c060 con 0x55bb20930370 2024-09-19T09:18:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.883+0000 7f93e0ff9640 1 -- 172.21.15.120:0/161396793 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f93f0011b50 con 0x55bb20930370 2024-09-19T09:18:09.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.883+0000 7f94018de740 1 -- 172.21.15.120:0/161396793 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93f40d9070 con 0x55bb20930370 2024-09-19T09:18:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.883+0000 7f93e0ff9640 1 -- 172.21.15.120:0/161396793 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f93f0011d50 con 0x55bb20930370 2024-09-19T09:18:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.883+0000 7f94018de740 10 client.20628 fetch_fsmap learned FSMap version 36 2024-09-19T09:18:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.883+0000 7f94018de740 1 -- 172.21.15.120:0/161396793 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55bb209a9d90 con 0x55bb20930370 2024-09-19T09:18:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.883+0000 7f93e0ff9640 1 -- 172.21.15.120:0/161396793 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93f40d9070 con 0x55bb20930370 2024-09-19T09:18:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.883+0000 7f93e0ff9640 1 -- 172.21.15.120:0/161396793 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f93f0031070 con 0x55bb20930370 2024-09-19T09:18:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.883+0000 7f94018de740 10 client.20628 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:18:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.883+0000 7f94018de740 10 client.20628 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:18:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.883+0000 7f94018de740 20 client.20628 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.883+0000 7f94018de740 1 --2- 172.21.15.120:0/161396793 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55bb2093fa00 0x55bb209ac520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.883+0000 7f94018de740 4 client.20628 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:18:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.883+0000 7f94018de740 1 -- 172.21.15.120:0/161396793 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb20892d70 con 0x55bb2093fa00 2024-09-19T09:18:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.884+0000 7f93fd2a1640 1 --2- 172.21.15.120:0/161396793 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55bb2093fa00 0x55bb209ac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.884+0000 7f93e0ff9640 10 client.20628 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:18:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.884+0000 7f93fd2a1640 1 --2- 172.21.15.120:0/161396793 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55bb2093fa00 0x55bb209ac520 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fe4c3d9d0594fd5d server_cookie=1e44cd1d74d9c3ba in_seq=0 out_seq=0 2024-09-19T09:18:09.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.884+0000 7f93e0ff9640 1 -- 172.21.15.120:0/161396793 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb20892d70 con 0x55bb2093fa00 2024-09-19T09:18:09.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.884+0000 7f93e0ff9640 10 client.20628 handle_command_reply: tid=0 2024-09-19T09:18:09.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.884+0000 7f94018de740 1 client.20628 shutdown 2024-09-19T09:18:09.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.884+0000 7f94018de740 1 -- 172.21.15.120:0/161396793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb20930370 msgr2=0x55bb209a8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:09.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.884+0000 7f94018de740 1 --2- 172.21.15.120:0/161396793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb20930370 0x55bb209a8640 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f93f000eaf0 tx=0x7f93f0004ab0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.884+0000 7f94018de740 1 -- 172.21.15.120:0/161396793 shutdown_connections 2024-09-19T09:18:09.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.884+0000 7f94018de740 1 --2- 172.21.15.120:0/161396793 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55bb2093fa00 0x55bb209ac520 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.884+0000 7f94018de740 1 --2- 172.21.15.120:0/161396793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb20930370 0x55bb209a8640 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.884+0000 7f94018de740 1 --2- 172.21.15.120:0/161396793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb20926b40 0x55bb209338c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.884+0000 7f93e0ff9640 0 client.20628 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:18:09.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.884+0000 7f94018de740 1 --2- 172.21.15.120:0/161396793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb209255d0 0x55bb20933380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.884+0000 7f94018de740 1 -- 172.21.15.120:0/161396793 >> 172.21.15.120:0/161396793 conn(0x55bb208ee310 msgr2=0x55bb2093b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:09.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.885+0000 7f94018de740 1 -- 172.21.15.120:0/161396793 shutdown_connections 2024-09-19T09:18:09.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.885+0000 7f94018de740 1 -- 172.21.15.120:0/161396793 wait complete. 2024-09-19T09:18:09.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.885+0000 7f94018de740 20 client.20628 trim_cache size 0 max 16384 2024-09-19T09:18:09.908 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:18:09.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.885+0000 7f93ff505640 1 -- 172.21.15.120:0/894080150 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93e40a3240 msgr2=0x7f93e40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:09.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.885+0000 7f93ff505640 1 --2- 172.21.15.120:0/894080150 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93e40a3240 0x7f93e40a5700 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f93f000afd0 tx=0x7f93f000a450 comp rx=0 tx=0).stop 2024-09-19T09:18:09.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.885+0000 7f93ff505640 1 -- 172.21.15.120:0/894080150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93f8102790 msgr2=0x7f93f8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:09.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.885+0000 7f93ff505640 1 --2- 172.21.15.120:0/894080150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93f8102790 0x7f93f8074a30 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f93f4005ab0 tx=0x7f93f40042f0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.885+0000 7f93ff505640 1 -- 172.21.15.120:0/894080150 shutdown_connections 2024-09-19T09:18:09.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.885+0000 7f93ff505640 1 --2- 172.21.15.120:0/894080150 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93e40a3240 0x7f93e40a5700 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.885+0000 7f93ff505640 1 --2- 172.21.15.120:0/894080150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93f8106f10 0x7f93f8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.885+0000 7f93ff505640 1 --2- 172.21.15.120:0/894080150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93f8102790 0x7f93f8074a30 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.885+0000 7f93ff505640 1 --2- 172.21.15.120:0/894080150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93f8101de0 0x7f93f80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.885+0000 7f93ff505640 1 -- 172.21.15.120:0/894080150 >> 172.21.15.120:0/894080150 conn(0x7f93f80fd8c0 msgr2=0x7f93f80fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:09.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.885+0000 7f93ff505640 1 -- 172.21.15.120:0/894080150 shutdown_connections 2024-09-19T09:18:09.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:09.885+0000 7f93ff505640 1 -- 172.21.15.120:0/894080150 wait complete. 2024-09-19T09:18:10.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:10 smithi120 ceph-mon[31150]: pgmap v804: 97 pgs: 97 active+clean; 296 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.1 MiB/s wr, 413 op/s 2024-09-19T09:18:10.357 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55751 0.00 MB/sec execute 5 sec latency 12481.091 ms 2024-09-19T09:18:10.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:10 smithi196 ceph-mon[35150]: pgmap v804: 97 pgs: 97 active+clean; 296 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.1 MiB/s wr, 413 op/s 2024-09-19T09:18:10.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:10 smithi153 ceph-mon[35071]: pgmap v804: 97 pgs: 97 active+clean; 296 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.1 MiB/s wr, 413 op/s 2024-09-19T09:18:11.127 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:18:11.356 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55751 0.00 MB/sec execute 6 sec latency 13481.119 ms 2024-09-19T09:18:11.379 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:18:11.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.876+0000 7f33037f9640 1 -- 172.21.15.120:0/3711577897 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32fc102ef0 msgr2=0x7f32fc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:11.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.876+0000 7f33037f9640 1 --2- 172.21.15.120:0/3711577897 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32fc102ef0 0x7f32fc10da40 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f32f4009ca0 tx=0x7f32f402a950 comp rx=0 tx=0).stop 2024-09-19T09:18:11.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.876+0000 7f33037f9640 1 -- 172.21.15.120:0/3711577897 shutdown_connections 2024-09-19T09:18:11.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.876+0000 7f33037f9640 1 --2- 172.21.15.120:0/3711577897 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32fc102ef0 0x7f32fc10da40 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.876+0000 7f33037f9640 1 --2- 172.21.15.120:0/3711577897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32fc102570 0x7f32fc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.876+0000 7f33037f9640 1 --2- 172.21.15.120:0/3711577897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32fc100e20 0x7f32fc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.876+0000 7f33037f9640 1 -- 172.21.15.120:0/3711577897 >> 172.21.15.120:0/3711577897 conn(0x7f32fc0fc7d0 msgr2=0x7f32fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:11.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.876+0000 7f33037f9640 1 -- 172.21.15.120:0/3711577897 shutdown_connections 2024-09-19T09:18:11.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.876+0000 7f33037f9640 1 -- 172.21.15.120:0/3711577897 wait complete. 2024-09-19T09:18:11.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.876+0000 7f33037f9640 1 Processor -- start 2024-09-19T09:18:11.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.877+0000 7f33037f9640 1 -- start start 2024-09-19T09:18:11.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.877+0000 7f33037f9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32fc100e20 0x7f32fc197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:11.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.877+0000 7f33037f9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32fc102570 0x7f32fc197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:11.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.877+0000 7f33037f9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32fc102ef0 0x7f32fc19bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:11.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.877+0000 7f33037f9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc110fa0 con 0x7f32fc102570 2024-09-19T09:18:11.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.877+0000 7f33037f9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc110e20 con 0x7f32fc102ef0 2024-09-19T09:18:11.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.877+0000 7f33037f9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc111120 con 0x7f32fc100e20 2024-09-19T09:18:11.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.877+0000 7f3300d94640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32fc102570 0x7f32fc197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:11.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.877+0000 7f3300d94640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32fc102570 0x7f32fc197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:32984/0 (socket says 172.21.15.120:32984) 2024-09-19T09:18:11.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.877+0000 7f3300d94640 1 -- 172.21.15.120:0/2204299727 learned_addr learned my addr 172.21.15.120:0/2204299727 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:11.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.877+0000 7f3301595640 1 --2- 172.21.15.120:0/2204299727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32fc100e20 0x7f32fc197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:11.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.877+0000 7f3301d96640 1 --2- 172.21.15.120:0/2204299727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32fc102ef0 0x7f32fc19bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:11.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.877+0000 7f3300d94640 1 -- 172.21.15.120:0/2204299727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32fc100e20 msgr2=0x7f32fc197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:11.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.877+0000 7f3300d94640 1 --2- 172.21.15.120:0/2204299727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32fc100e20 0x7f32fc197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.877+0000 7f3300d94640 1 -- 172.21.15.120:0/2204299727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32fc102ef0 msgr2=0x7f32fc19bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:11.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.877+0000 7f3300d94640 1 --2- 172.21.15.120:0/2204299727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32fc102ef0 0x7f32fc19bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.877+0000 7f3300d94640 1 -- 172.21.15.120:0/2204299727 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32fc19c2e0 con 0x7f32fc102570 2024-09-19T09:18:11.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.877+0000 7f3301d96640 1 --2- 172.21.15.120:0/2204299727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32fc102ef0 0x7f32fc19bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:18:11.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.877+0000 7f3301595640 1 --2- 172.21.15.120:0/2204299727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32fc100e20 0x7f32fc197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:18:11.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.877+0000 7f3300d94640 1 --2- 172.21.15.120:0/2204299727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32fc102570 0x7f32fc197870 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00b4a0 tx=0x7f32ec00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:11.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.878+0000 7f32f27fc640 1 -- 172.21.15.120:0/2204299727 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec014070 con 0x7f32fc102570 2024-09-19T09:18:11.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.878+0000 7f32f27fc640 1 -- 172.21.15.120:0/2204299727 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32ec010af0 con 0x7f32fc102570 2024-09-19T09:18:11.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.878+0000 7f32f27fc640 1 -- 172.21.15.120:0/2204299727 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec017d10 con 0x7f32fc102570 2024-09-19T09:18:11.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.878+0000 7f33037f9640 1 -- 172.21.15.120:0/2204299727 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32fc19c5d0 con 0x7f32fc102570 2024-09-19T09:18:11.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.878+0000 7f33037f9640 1 -- 172.21.15.120:0/2204299727 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32fc1a3eb0 con 0x7f32fc102570 2024-09-19T09:18:11.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.880+0000 7f32f27fc640 1 -- 172.21.15.120:0/2204299727 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f32ec017070 con 0x7f32fc102570 2024-09-19T09:18:11.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.880+0000 7f3305bd2740 1 Processor -- start 2024-09-19T09:18:11.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.880+0000 7f3305bd2740 1 -- start start 2024-09-19T09:18:11.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.880+0000 7f3305bd2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deb9e1ec30 0x55deb9e1d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:11.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.880+0000 7f3305bd2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deb9e1d7e0 0x55deb9e1dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:11.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.880+0000 7f3305bd2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deb9e2a970 0x55deb9e2cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:11.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.881+0000 7f3305bd2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55deb9d812f0 con 0x55deb9e1d7e0 2024-09-19T09:18:11.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.881+0000 7f3305bd2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55deb9d28cf0 con 0x55deb9e1ec30 2024-09-19T09:18:11.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.881+0000 7f3305bd2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55deb9d7d2b0 con 0x55deb9e2a970 2024-09-19T09:18:11.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.881+0000 7f3301595640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deb9e1d7e0 0x55deb9e1dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:11.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.881+0000 7f32f27fc640 1 --2- 172.21.15.120:0/2204299727 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f32e80a32c0 0x7f32e80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:11.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.881+0000 7f3301595640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deb9e1d7e0 0x55deb9e1dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:32994/0 (socket says 172.21.15.120:32994) 2024-09-19T09:18:11.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.881+0000 7f3301595640 1 -- 172.21.15.120:0/786303261 learned_addr learned my addr 172.21.15.120:0/786303261 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:11.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.881+0000 7f3300d94640 1 --2- 172.21.15.120:0/786303261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deb9e2a970 0x55deb9e2cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:11.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.881+0000 7f3301d96640 1 --2- 172.21.15.120:0/2204299727 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f32e80a32c0 0x7f32e80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:11.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.881+0000 7f3301d96640 1 --2- 172.21.15.120:0/786303261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deb9e1ec30 0x55deb9e1d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:11.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.881+0000 7f32f27fc640 1 -- 172.21.15.120:0/2204299727 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f32ec0cecb0 con 0x7f32fc102570 2024-09-19T09:18:11.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.881+0000 7f3301595640 1 -- 172.21.15.120:0/786303261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deb9e2a970 msgr2=0x55deb9e2cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:11.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.881+0000 7f3301595640 1 --2- 172.21.15.120:0/786303261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deb9e2a970 0x55deb9e2cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.881+0000 7f3301595640 1 -- 172.21.15.120:0/786303261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deb9e1ec30 msgr2=0x55deb9e1d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:11.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.881+0000 7f3301595640 1 --2- 172.21.15.120:0/786303261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deb9e1ec30 0x55deb9e1d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.881+0000 7f3301595640 1 -- 172.21.15.120:0/786303261 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deb9d0d3a0 con 0x55deb9e1d7e0 2024-09-19T09:18:11.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.881+0000 7f3301d96640 1 --2- 172.21.15.120:0/786303261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deb9e1ec30 0x55deb9e1d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:18:11.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.881+0000 7f3301595640 1 --2- 172.21.15.120:0/786303261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deb9e1d7e0 0x55deb9e1dc20 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f32f8008c60 tx=0x7f32f800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3279d4b06f0e8398 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:11.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.881+0000 7f3301d96640 1 --2- 172.21.15.120:0/2204299727 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f32e80a32c0 0x7f32e80a5780 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f32f4009a60 tx=0x7f32f4009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:11.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.881+0000 7f32e77fe640 1 -- 172.21.15.120:0/786303261 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f8014070 con 0x55deb9e1d7e0 2024-09-19T09:18:11.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.881+0000 7f32e77fe640 1 -- 172.21.15.120:0/786303261 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32f80047a0 con 0x55deb9e1d7e0 2024-09-19T09:18:11.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.881+0000 7f32e77fe640 1 -- 172.21.15.120:0/786303261 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f8010760 con 0x55deb9e1d7e0 2024-09-19T09:18:11.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.882+0000 7f3305bd2740 1 -- 172.21.15.120:0/786303261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deb9e1d7e0 msgr2=0x55deb9e1dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:11.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.882+0000 7f3305bd2740 1 --2- 172.21.15.120:0/786303261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deb9e1d7e0 0x55deb9e1dc20 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f32f8008c60 tx=0x7f32f800e720 comp rx=0 tx=0).stop 2024-09-19T09:18:11.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.882+0000 7f3305bd2740 1 -- 172.21.15.120:0/786303261 shutdown_connections 2024-09-19T09:18:11.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.882+0000 7f3305bd2740 1 --2- 172.21.15.120:0/786303261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deb9e2a970 0x55deb9e2cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.882+0000 7f3305bd2740 1 --2- 172.21.15.120:0/786303261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deb9e1d7e0 0x55deb9e1dc20 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.882+0000 7f3305bd2740 1 --2- 172.21.15.120:0/786303261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deb9e1ec30 0x55deb9e1d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.882+0000 7f3305bd2740 1 -- 172.21.15.120:0/786303261 >> 172.21.15.120:0/786303261 conn(0x55deb9de4310 msgr2=0x55deb9e19e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:11.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.882+0000 7f3305bd2740 1 -- 172.21.15.120:0/786303261 shutdown_connections 2024-09-19T09:18:11.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.882+0000 7f3305bd2740 1 -- 172.21.15.120:0/786303261 wait complete. 2024-09-19T09:18:11.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.883+0000 7f3305bd2740 1 -- start start 2024-09-19T09:18:11.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.883+0000 7f3305bd2740 1 Processor -- start 2024-09-19T09:18:11.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.883+0000 7f3305bd2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deb9e1d7e0 0x55deb9e2a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:11.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.883+0000 7f3305bd2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deb9e1ec30 0x55deb9e275b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:11.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.883+0000 7f3305bd2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deb9e2a970 0x55deb9e27b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:11.918 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:18:11.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.883+0000 7f3305bd2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55deb9e2d540 con 0x55deb9e1d7e0 2024-09-19T09:18:11.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.883+0000 7f3305bd2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55deb9d3c060 con 0x55deb9e2a970 2024-09-19T09:18:11.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.883+0000 7f3305bd2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55deb9e2d6c0 con 0x55deb9e1ec30 2024-09-19T09:18:11.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.883+0000 7f3305bd2740 10 client.0 _ll_register_callbacks cb 0x55deb9a8f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:11.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.883+0000 7f3301595640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deb9e1d7e0 0x55deb9e2a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:11.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.883+0000 7f3301595640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deb9e1d7e0 0x55deb9e2a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33002/0 (socket says 172.21.15.120:33002) 2024-09-19T09:18:11.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.883+0000 7f3301595640 1 -- 172.21.15.120:0/1297282715 learned_addr learned my addr 172.21.15.120:0/1297282715 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:11.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.883+0000 7f3300d94640 1 --2- 172.21.15.120:0/1297282715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deb9e1ec30 0x55deb9e275b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:11.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.883+0000 7f3301d96640 1 --2- 172.21.15.120:0/1297282715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deb9e2a970 0x55deb9e27b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:11.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.884+0000 7f3301595640 1 -- 172.21.15.120:0/1297282715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deb9e1ec30 msgr2=0x55deb9e275b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:11.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.884+0000 7f3301595640 1 --2- 172.21.15.120:0/1297282715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deb9e1ec30 0x55deb9e275b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.884+0000 7f3301595640 1 -- 172.21.15.120:0/1297282715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deb9e2a970 msgr2=0x55deb9e27b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:11.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.884+0000 7f3301595640 1 --2- 172.21.15.120:0/1297282715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deb9e2a970 0x55deb9e27b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.884+0000 7f3301595640 1 -- 172.21.15.120:0/1297282715 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deb9af23c0 con 0x55deb9e1d7e0 2024-09-19T09:18:11.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.884+0000 7f3301d96640 1 --2- 172.21.15.120:0/1297282715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deb9e2a970 0x55deb9e27b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:18:11.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.884+0000 7f3301595640 1 --2- 172.21.15.120:0/1297282715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deb9e1d7e0 0x55deb9e2a300 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f32f800ec20 tx=0x7f32f80051a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df83ac287d2bf2f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:11.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.884+0000 7f32e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:18:11.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.884+0000 7f32e4ff9640 1 -- 172.21.15.120:0/1297282715 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f8014070 con 0x55deb9e1d7e0 2024-09-19T09:18:11.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.884+0000 7f32e4ff9640 1 -- 172.21.15.120:0/1297282715 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f32f80054a0 con 0x55deb9e1d7e0 2024-09-19T09:18:11.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.884+0000 7f3305bd2740 1 -- 172.21.15.120:0/1297282715 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32f80047a0 con 0x55deb9e1d7e0 2024-09-19T09:18:11.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.884+0000 7f32e4ff9640 1 -- 172.21.15.120:0/1297282715 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f32f8004030 con 0x55deb9e1d7e0 2024-09-19T09:18:11.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.884+0000 7f3305bd2740 10 client.20640 fetch_fsmap learned FSMap version 36 2024-09-19T09:18:11.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.884+0000 7f3305bd2740 1 -- 172.21.15.120:0/1297282715 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55deb9e38f70 con 0x55deb9e1d7e0 2024-09-19T09:18:11.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.884+0000 7f32e4ff9640 1 -- 172.21.15.120:0/1297282715 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32f80047a0 con 0x55deb9e1d7e0 2024-09-19T09:18:11.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.884+0000 7f32e4ff9640 1 -- 172.21.15.120:0/1297282715 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f32f8003bb0 con 0x55deb9e1d7e0 2024-09-19T09:18:11.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.884+0000 7f3305bd2740 10 client.20640 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:18:11.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.884+0000 7f3305bd2740 10 client.20640 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:18:11.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.884+0000 7f3305bd2740 20 client.20640 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:11.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.885+0000 7f3305bd2740 1 --2- 172.21.15.120:0/1297282715 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55deb9e3a370 0x55deb9e3c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:11.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.885+0000 7f3305bd2740 4 client.20640 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:18:11.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.885+0000 7f3305bd2740 1 -- 172.21.15.120:0/1297282715 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55deb9d8e860 con 0x55deb9e3a370 2024-09-19T09:18:11.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.885+0000 7f3300d94640 1 --2- 172.21.15.120:0/1297282715 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55deb9e3a370 0x55deb9e3c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:11.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.885+0000 7f32e4ff9640 10 client.20640 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:18:11.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.885+0000 7f3300d94640 1 --2- 172.21.15.120:0/1297282715 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55deb9e3a370 0x55deb9e3c760 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ed81445c4d5cddff server_cookie=39452485efaddec7 in_seq=0 out_seq=0 2024-09-19T09:18:11.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.886+0000 7f32e4ff9640 1 -- 172.21.15.120:0/1297282715 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55deb9d8e860 con 0x55deb9e3a370 2024-09-19T09:18:11.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.886+0000 7f32e4ff9640 10 client.20640 handle_command_reply: tid=0 2024-09-19T09:18:11.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.886+0000 7f3305bd2740 1 client.20640 shutdown 2024-09-19T09:18:11.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.886+0000 7f3305bd2740 1 -- 172.21.15.120:0/1297282715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deb9e1d7e0 msgr2=0x55deb9e2a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:11.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.886+0000 7f3305bd2740 1 --2- 172.21.15.120:0/1297282715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deb9e1d7e0 0x55deb9e2a300 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f32f800ec20 tx=0x7f32f80051a0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.886+0000 7f3305bd2740 1 -- 172.21.15.120:0/1297282715 shutdown_connections 2024-09-19T09:18:11.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.886+0000 7f3305bd2740 1 --2- 172.21.15.120:0/1297282715 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55deb9e3a370 0x55deb9e3c760 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.886+0000 7f3305bd2740 1 --2- 172.21.15.120:0/1297282715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deb9e2a970 0x55deb9e27b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.886+0000 7f3305bd2740 1 --2- 172.21.15.120:0/1297282715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deb9e1ec30 0x55deb9e275b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.886+0000 7f32e4ff9640 0 client.20640 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:18:11.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.886+0000 7f3305bd2740 1 --2- 172.21.15.120:0/1297282715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deb9e1d7e0 0x55deb9e2a300 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.886+0000 7f3305bd2740 1 -- 172.21.15.120:0/1297282715 >> 172.21.15.120:0/1297282715 conn(0x55deb9de4310 msgr2=0x55deb9e1fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:11.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.886+0000 7f3305bd2740 1 -- 172.21.15.120:0/1297282715 shutdown_connections 2024-09-19T09:18:11.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.886+0000 7f3305bd2740 1 -- 172.21.15.120:0/1297282715 wait complete. 2024-09-19T09:18:11.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.886+0000 7f3305bd2740 20 client.20640 trim_cache size 0 max 16384 2024-09-19T09:18:11.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.899+0000 7f3305bd2740 1 Processor -- start 2024-09-19T09:18:11.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.899+0000 7f3305bd2740 1 -- start start 2024-09-19T09:18:11.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.899+0000 7f3305bd2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deb9e1d7e0 0x55deb9e367a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:11.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.899+0000 7f3305bd2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deb9e1ec30 0x55deb9e26e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:11.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.899+0000 7f3305bd2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deb9e2a970 0x55deb9e41d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:11.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.899+0000 7f3305bd2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc1ab410 con 0x55deb9e1ec30 2024-09-19T09:18:11.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.899+0000 7f3305bd2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc1acfe0 con 0x55deb9e2a970 2024-09-19T09:18:11.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.899+0000 7f3305bd2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc1abae0 con 0x55deb9e1d7e0 2024-09-19T09:18:11.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.899+0000 7f3300d94640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deb9e1ec30 0x55deb9e26e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:11.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.899+0000 7f3300d94640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deb9e1ec30 0x55deb9e26e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33018/0 (socket says 172.21.15.120:33018) 2024-09-19T09:18:11.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.899+0000 7f3300d94640 1 -- 172.21.15.120:0/1242977491 learned_addr learned my addr 172.21.15.120:0/1242977491 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:11.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.899+0000 7f3300d94640 1 -- 172.21.15.120:0/1242977491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deb9e1d7e0 msgr2=0x55deb9e367a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:11.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.899+0000 7f3300d94640 1 --2- 172.21.15.120:0/1242977491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deb9e1d7e0 0x55deb9e367a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.899+0000 7f3300d94640 1 -- 172.21.15.120:0/1242977491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deb9e2a970 msgr2=0x55deb9e41d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:11.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.899+0000 7f3300d94640 1 --2- 172.21.15.120:0/1242977491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deb9e2a970 0x55deb9e41d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.899+0000 7f3300d94640 1 -- 172.21.15.120:0/1242977491 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deb9e2d650 con 0x55deb9e1ec30 2024-09-19T09:18:11.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.900+0000 7f3300d94640 1 --2- 172.21.15.120:0/1242977491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deb9e1ec30 0x55deb9e26e70 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f32ec09afd0 tx=0x7f32ec0bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a785df82ee155de5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:11.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.900+0000 7f32e6ffd640 1 -- 172.21.15.120:0/1242977491 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec01e040 con 0x55deb9e1ec30 2024-09-19T09:18:11.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.900+0000 7f32e6ffd640 1 -- 172.21.15.120:0/1242977491 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32ec0a3f40 con 0x55deb9e1ec30 2024-09-19T09:18:11.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.900+0000 7f32e6ffd640 1 -- 172.21.15.120:0/1242977491 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec010ca0 con 0x55deb9e1ec30 2024-09-19T09:18:11.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.900+0000 7f3305bd2740 1 -- 172.21.15.120:0/1242977491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deb9e1ec30 msgr2=0x55deb9e26e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:11.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.900+0000 7f3305bd2740 1 --2- 172.21.15.120:0/1242977491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deb9e1ec30 0x55deb9e26e70 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f32ec09afd0 tx=0x7f32ec0bb040 comp rx=0 tx=0).stop 2024-09-19T09:18:11.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.900+0000 7f3305bd2740 1 -- 172.21.15.120:0/1242977491 shutdown_connections 2024-09-19T09:18:11.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.900+0000 7f3305bd2740 1 --2- 172.21.15.120:0/1242977491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deb9e2a970 0x55deb9e41d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.900+0000 7f3305bd2740 1 --2- 172.21.15.120:0/1242977491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deb9e1ec30 0x55deb9e26e70 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.900+0000 7f3305bd2740 1 --2- 172.21.15.120:0/1242977491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deb9e1d7e0 0x55deb9e367a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.900+0000 7f3305bd2740 1 -- 172.21.15.120:0/1242977491 >> 172.21.15.120:0/1242977491 conn(0x55deb9de4310 msgr2=0x55deb9e164e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:11.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.900+0000 7f3305bd2740 1 -- 172.21.15.120:0/1242977491 shutdown_connections 2024-09-19T09:18:11.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.900+0000 7f3305bd2740 1 -- 172.21.15.120:0/1242977491 wait complete. 2024-09-19T09:18:11.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.901+0000 7f3305bd2740 1 -- start start 2024-09-19T09:18:11.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.901+0000 7f3305bd2740 1 Processor -- start 2024-09-19T09:18:11.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.901+0000 7f3305bd2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deb9e1d7e0 0x55deb9e41860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:11.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.901+0000 7f3305bd2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deb9e1ec30 0x55deb9e41da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:11.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.901+0000 7f3305bd2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deb9e2a970 0x55deb9e422e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:11.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.901+0000 7f3305bd2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc1ac910 con 0x55deb9e1ec30 2024-09-19T09:18:11.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.901+0000 7f3305bd2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc1ac1b0 con 0x55deb9e1d7e0 2024-09-19T09:18:11.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.901+0000 7f3305bd2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc1a93d0 con 0x55deb9e2a970 2024-09-19T09:18:11.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.901+0000 7f3305bd2740 10 client.0 _ll_register_callbacks cb 0x7f32ec017630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:11.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.901+0000 7f3300d94640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deb9e1ec30 0x55deb9e41da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:11.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.902+0000 7f3300d94640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deb9e1ec30 0x55deb9e41da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33032/0 (socket says 172.21.15.120:33032) 2024-09-19T09:18:11.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.902+0000 7f3300d94640 1 -- 172.21.15.120:0/2269929676 learned_addr learned my addr 172.21.15.120:0/2269929676 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:11.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.902+0000 7f3301595640 1 --2- 172.21.15.120:0/2269929676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deb9e1d7e0 0x55deb9e41860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:11.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.902+0000 7f3301d96640 1 --2- 172.21.15.120:0/2269929676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deb9e2a970 0x55deb9e422e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:11.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.902+0000 7f3300d94640 1 -- 172.21.15.120:0/2269929676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deb9e2a970 msgr2=0x55deb9e422e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:11.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.902+0000 7f3300d94640 1 --2- 172.21.15.120:0/2269929676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deb9e2a970 0x55deb9e422e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.902+0000 7f3300d94640 1 -- 172.21.15.120:0/2269929676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deb9e1d7e0 msgr2=0x55deb9e41860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:11.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.902+0000 7f3300d94640 1 --2- 172.21.15.120:0/2269929676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deb9e1d7e0 0x55deb9e41860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.902+0000 7f3300d94640 1 -- 172.21.15.120:0/2269929676 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deb9ea3cd0 con 0x55deb9e1ec30 2024-09-19T09:18:11.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.902+0000 7f3300d94640 1 --2- 172.21.15.120:0/2269929676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deb9e1ec30 0x55deb9e41da0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f32ec017770 tx=0x7f32ec000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7d8f9b9ccad3a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:11.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.902+0000 7f32e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:18:11.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.902+0000 7f32e4ff9640 1 -- 172.21.15.120:0/2269929676 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec0b3070 con 0x55deb9e1ec30 2024-09-19T09:18:11.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.902+0000 7f32e4ff9640 1 -- 172.21.15.120:0/2269929676 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f32ec0cc8c0 con 0x55deb9e1ec30 2024-09-19T09:18:11.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.902+0000 7f3305bd2740 1 -- 172.21.15.120:0/2269929676 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32ec0a3f40 con 0x55deb9e1ec30 2024-09-19T09:18:11.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.902+0000 7f32e4ff9640 1 -- 172.21.15.120:0/2269929676 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f32ec01e070 con 0x55deb9e1ec30 2024-09-19T09:18:11.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.902+0000 7f3305bd2740 10 client.20646 fetch_fsmap learned FSMap version 36 2024-09-19T09:18:11.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.902+0000 7f3305bd2740 1 -- 172.21.15.120:0/2269929676 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55deb9ea4f30 con 0x55deb9e1ec30 2024-09-19T09:18:11.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.902+0000 7f32e4ff9640 1 -- 172.21.15.120:0/2269929676 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32ec0a3f40 con 0x55deb9e1ec30 2024-09-19T09:18:11.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.903+0000 7f32e4ff9640 1 -- 172.21.15.120:0/2269929676 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f32ec09f180 con 0x55deb9e1ec30 2024-09-19T09:18:11.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.903+0000 7f3305bd2740 10 client.20646 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:18:11.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.903+0000 7f3305bd2740 10 client.20646 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:18:11.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.903+0000 7f3305bd2740 20 client.20646 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:11.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.903+0000 7f3305bd2740 1 --2- 172.21.15.120:0/2269929676 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55deb9ea56a0 0x55deb9ea7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:11.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.903+0000 7f3305bd2740 4 client.20646 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:18:11.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.903+0000 7f3305bd2740 1 -- 172.21.15.120:0/2269929676 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55deb9d88d70 con 0x55deb9ea56a0 2024-09-19T09:18:11.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.903+0000 7f3301595640 1 --2- 172.21.15.120:0/2269929676 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55deb9ea56a0 0x55deb9ea7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:11.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.903+0000 7f32e4ff9640 10 client.20646 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:18:11.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.903+0000 7f3301595640 1 --2- 172.21.15.120:0/2269929676 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55deb9ea56a0 0x55deb9ea7ac0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bbb581c323e15abf server_cookie=67edb401add47fc8 in_seq=0 out_seq=0 2024-09-19T09:18:11.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.903+0000 7f32e4ff9640 1 -- 172.21.15.120:0/2269929676 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55deb9d88d70 con 0x55deb9ea56a0 2024-09-19T09:18:11.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.903+0000 7f32e4ff9640 10 client.20646 handle_command_reply: tid=0 2024-09-19T09:18:11.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.903+0000 7f3305bd2740 1 client.20646 shutdown 2024-09-19T09:18:11.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.904+0000 7f3305bd2740 1 -- 172.21.15.120:0/2269929676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deb9e1ec30 msgr2=0x55deb9e41da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:11.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.904+0000 7f3305bd2740 1 --2- 172.21.15.120:0/2269929676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deb9e1ec30 0x55deb9e41da0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f32ec017770 tx=0x7f32ec000bc0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.904+0000 7f3305bd2740 1 -- 172.21.15.120:0/2269929676 shutdown_connections 2024-09-19T09:18:11.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.904+0000 7f3305bd2740 1 --2- 172.21.15.120:0/2269929676 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55deb9ea56a0 0x55deb9ea7ac0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.904+0000 7f3305bd2740 1 --2- 172.21.15.120:0/2269929676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55deb9e2a970 0x55deb9e422e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.904+0000 7f3305bd2740 1 --2- 172.21.15.120:0/2269929676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55deb9e1ec30 0x55deb9e41da0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.904+0000 7f3305bd2740 1 --2- 172.21.15.120:0/2269929676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55deb9e1d7e0 0x55deb9e41860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.904+0000 7f3305bd2740 1 -- 172.21.15.120:0/2269929676 >> 172.21.15.120:0/2269929676 conn(0x55deb9de4310 msgr2=0x55deb9e25e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:11.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.904+0000 7f32e4ff9640 0 client.20646 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:18:11.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.904+0000 7f3305bd2740 1 -- 172.21.15.120:0/2269929676 shutdown_connections 2024-09-19T09:18:11.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.904+0000 7f3305bd2740 1 -- 172.21.15.120:0/2269929676 wait complete. 2024-09-19T09:18:11.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.904+0000 7f3305bd2740 20 client.20646 trim_cache size 0 max 16384 2024-09-19T09:18:11.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.904+0000 7f33037f9640 1 -- 172.21.15.120:0/2204299727 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f32e80a32c0 msgr2=0x7f32e80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:11.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.904+0000 7f33037f9640 1 --2- 172.21.15.120:0/2204299727 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f32e80a32c0 0x7f32e80a5780 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f32f4009a60 tx=0x7f32f4009360 comp rx=0 tx=0).stop 2024-09-19T09:18:11.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.904+0000 7f33037f9640 1 -- 172.21.15.120:0/2204299727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32fc102570 msgr2=0x7f32fc197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:11.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.904+0000 7f33037f9640 1 --2- 172.21.15.120:0/2204299727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32fc102570 0x7f32fc197870 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00b4a0 tx=0x7f32ec00b960 comp rx=0 tx=0).stop 2024-09-19T09:18:11.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.904+0000 7f33037f9640 1 -- 172.21.15.120:0/2204299727 shutdown_connections 2024-09-19T09:18:11.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.904+0000 7f33037f9640 1 --2- 172.21.15.120:0/2204299727 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f32e80a32c0 0x7f32e80a5780 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.904+0000 7f33037f9640 1 --2- 172.21.15.120:0/2204299727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32fc102ef0 0x7f32fc19bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.904+0000 7f33037f9640 1 --2- 172.21.15.120:0/2204299727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32fc102570 0x7f32fc197870 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.904+0000 7f33037f9640 1 --2- 172.21.15.120:0/2204299727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32fc100e20 0x7f32fc197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:11.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.904+0000 7f33037f9640 1 -- 172.21.15.120:0/2204299727 >> 172.21.15.120:0/2204299727 conn(0x7f32fc0fc7d0 msgr2=0x7f32fc106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:11.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.904+0000 7f33037f9640 1 -- 172.21.15.120:0/2204299727 shutdown_connections 2024-09-19T09:18:11.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:11.905+0000 7f33037f9640 1 -- 172.21.15.120:0/2204299727 wait complete. 2024-09-19T09:18:12.357 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55751 0.00 MB/sec execute 7 sec latency 14481.175 ms 2024-09-19T09:18:12.664 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:18:12.908 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:18:13.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:12 smithi153 ceph-mon[35071]: pgmap v805: 97 pgs: 97 active+clean; 295 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 5.0 MiB/s wr, 381 op/s 2024-09-19T09:18:13.194 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:12 smithi120 ceph-mon[31150]: pgmap v805: 97 pgs: 97 active+clean; 295 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 5.0 MiB/s wr, 381 op/s 2024-09-19T09:18:13.357 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55751 0.00 MB/sec execute 8 sec latency 15481.204 ms 2024-09-19T09:18:13.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.412+0000 7f5df85e5640 1 -- 172.21.15.120:0/2136582523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5df0101bc0 msgr2=0x7f5df010d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:13.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.412+0000 7f5df85e5640 1 --2- 172.21.15.120:0/2136582523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5df0101bc0 0x7f5df010d910 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f5de8009900 tx=0x7f5de802a410 comp rx=0 tx=0).stop 2024-09-19T09:18:13.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.412+0000 7f5df85e5640 1 -- 172.21.15.120:0/2136582523 shutdown_connections 2024-09-19T09:18:13.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.412+0000 7f5df85e5640 1 --2- 172.21.15.120:0/2136582523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5df0101bc0 0x7f5df010d910 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.412+0000 7f5df85e5640 1 --2- 172.21.15.120:0/2136582523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5df0101240 0x7f5df0101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.412+0000 7f5df85e5640 1 --2- 172.21.15.120:0/2136582523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5df0105a90 0x7f5df0105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.412+0000 7f5df85e5640 1 -- 172.21.15.120:0/2136582523 >> 172.21.15.120:0/2136582523 conn(0x7f5df0077210 msgr2=0x7f5df00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:13.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.412+0000 7f5df85e5640 1 -- 172.21.15.120:0/2136582523 shutdown_connections 2024-09-19T09:18:13.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.412+0000 7f5df85e5640 1 -- 172.21.15.120:0/2136582523 wait complete. 2024-09-19T09:18:13.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.413+0000 7f5df85e5640 1 Processor -- start 2024-09-19T09:18:13.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.413+0000 7f5df85e5640 1 -- start start 2024-09-19T09:18:13.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.413+0000 7f5df85e5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5df0101240 0x7f5df0198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:13.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.413+0000 7f5df85e5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5df0101bc0 0x7f5df0198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:13.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.413+0000 7f5df85e5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5df0105a90 0x7f5df019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:13.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.413+0000 7f5df85e5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5df0110fd0 con 0x7f5df0105a90 2024-09-19T09:18:13.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.413+0000 7f5df85e5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5df0110e50 con 0x7f5df0101bc0 2024-09-19T09:18:13.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.413+0000 7f5df85e5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5df0111150 con 0x7f5df0101240 2024-09-19T09:18:13.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.413+0000 7f5df6b82640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5df0105a90 0x7f5df019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:13.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.413+0000 7f5df6b82640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5df0105a90 0x7f5df019ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33038/0 (socket says 172.21.15.120:33038) 2024-09-19T09:18:13.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.413+0000 7f5df6b82640 1 -- 172.21.15.120:0/282692117 learned_addr learned my addr 172.21.15.120:0/282692117 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:13.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.413+0000 7f5df5b80640 1 --2- 172.21.15.120:0/282692117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5df0101bc0 0x7f5df0198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:13.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.413+0000 7f5df6381640 1 --2- 172.21.15.120:0/282692117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5df0101240 0x7f5df0198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:13.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.413+0000 7f5df6b82640 1 -- 172.21.15.120:0/282692117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5df0101240 msgr2=0x7f5df0198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:13.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.413+0000 7f5df6b82640 1 --2- 172.21.15.120:0/282692117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5df0101240 0x7f5df0198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.413+0000 7f5df6b82640 1 -- 172.21.15.120:0/282692117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5df0101bc0 msgr2=0x7f5df0198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:13.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.413+0000 7f5df6b82640 1 --2- 172.21.15.120:0/282692117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5df0101bc0 0x7f5df0198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.413+0000 7f5df6b82640 1 -- 172.21.15.120:0/282692117 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df019d4e0 con 0x7f5df0105a90 2024-09-19T09:18:13.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.413+0000 7f5df6381640 1 --2- 172.21.15.120:0/282692117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5df0101240 0x7f5df0198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:18:13.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.414+0000 7f5df6b82640 1 --2- 172.21.15.120:0/282692117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5df0105a90 0x7f5df019ce00 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f5de8004770 tx=0x7f5de8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:13.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.414+0000 7f5def7fe640 1 -- 172.21.15.120:0/282692117 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de803f070 con 0x7f5df0105a90 2024-09-19T09:18:13.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.414+0000 7f5def7fe640 1 -- 172.21.15.120:0/282692117 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5de8031c30 con 0x7f5df0105a90 2024-09-19T09:18:13.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.414+0000 7f5df85e5640 1 -- 172.21.15.120:0/282692117 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df019d770 con 0x7f5df0105a90 2024-09-19T09:18:13.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.414+0000 7f5def7fe640 1 -- 172.21.15.120:0/282692117 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de8035030 con 0x7f5df0105a90 2024-09-19T09:18:13.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.414+0000 7f5df85e5640 1 -- 172.21.15.120:0/282692117 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df01a50b0 con 0x7f5df0105a90 2024-09-19T09:18:13.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.416+0000 7f5def7fe640 1 -- 172.21.15.120:0/282692117 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5de8042070 con 0x7f5df0105a90 2024-09-19T09:18:13.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.416+0000 7f5dfa9be740 1 Processor -- start 2024-09-19T09:18:13.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.417+0000 7f5dfa9be740 1 -- start start 2024-09-19T09:18:13.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.417+0000 7f5dfa9be740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efc4443b40 0x55efc4442090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:13.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.417+0000 7f5dfa9be740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efc44425d0 0x55efc4442a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:13.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.417+0000 7f5dfa9be740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efc444d370 0x55efc444f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:13.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.417+0000 7f5dfa9be740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55efc43a82f0 con 0x55efc4443b40 2024-09-19T09:18:13.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.417+0000 7f5dfa9be740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55efc434fcf0 con 0x55efc44425d0 2024-09-19T09:18:13.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.417+0000 7f5dfa9be740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55efc43a42b0 con 0x55efc444d370 2024-09-19T09:18:13.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.417+0000 7f5def7fe640 1 --2- 172.21.15.120:0/282692117 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5dd80a32c0 0x7f5dd80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:13.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.417+0000 7f5df5b80640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efc4443b40 0x55efc4442090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:13.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.417+0000 7f5df5b80640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efc4443b40 0x55efc4442090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33040/0 (socket says 172.21.15.120:33040) 2024-09-19T09:18:13.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.417+0000 7f5df5b80640 1 -- 172.21.15.120:0/4294712789 learned_addr learned my addr 172.21.15.120:0/4294712789 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:13.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.417+0000 7f5df6b82640 1 --2- 172.21.15.120:0/282692117 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5dd80a32c0 0x7f5dd80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:13.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.417+0000 7f5df6381640 1 --2- 172.21.15.120:0/4294712789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efc44425d0 0x55efc4442a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:13.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.417+0000 7f5def7fe640 1 -- 172.21.15.120:0/282692117 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f5de80f00f0 con 0x7f5df0105a90 2024-09-19T09:18:13.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.417+0000 7f5df5b80640 1 --2- 172.21.15.120:0/4294712789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efc444d370 0x55efc444f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:13.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.417+0000 7f5df5b80640 1 -- 172.21.15.120:0/4294712789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efc444d370 msgr2=0x55efc444f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:13.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.417+0000 7f5df5b80640 1 --2- 172.21.15.120:0/4294712789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efc444d370 0x55efc444f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.417+0000 7f5df5b80640 1 -- 172.21.15.120:0/4294712789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efc44425d0 msgr2=0x55efc4442a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:13.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.417+0000 7f5df5b80640 1 --2- 172.21.15.120:0/4294712789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efc44425d0 0x55efc4442a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.417+0000 7f5df5b80640 1 -- 172.21.15.120:0/4294712789 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efc43343a0 con 0x55efc4443b40 2024-09-19T09:18:13.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.417+0000 7f5df6381640 1 --2- 172.21.15.120:0/4294712789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efc44425d0 0x55efc4442a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:18:13.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.418+0000 7f5df5b80640 1 --2- 172.21.15.120:0/4294712789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efc4443b40 0x55efc4442090 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f5de4004650 tx=0x7f5de4004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf3cd31207d180 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:13.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.418+0000 7f5df6b82640 1 --2- 172.21.15.120:0/282692117 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5dd80a32c0 0x7f5dd80a5780 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5de80b8b70 tx=0x7f5de808c380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:13.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.418+0000 7f5decff9640 1 -- 172.21.15.120:0/4294712789 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de400be60 con 0x55efc4443b40 2024-09-19T09:18:13.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.418+0000 7f5decff9640 1 -- 172.21.15.120:0/4294712789 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5de4005500 con 0x55efc4443b40 2024-09-19T09:18:13.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.418+0000 7f5decff9640 1 -- 172.21.15.120:0/4294712789 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de40106c0 con 0x55efc4443b40 2024-09-19T09:18:13.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.418+0000 7f5dfa9be740 1 -- 172.21.15.120:0/4294712789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efc4443b40 msgr2=0x55efc4442090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:13.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.418+0000 7f5dfa9be740 1 --2- 172.21.15.120:0/4294712789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efc4443b40 0x55efc4442090 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f5de4004650 tx=0x7f5de4004250 comp rx=0 tx=0).stop 2024-09-19T09:18:13.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.419+0000 7f5dfa9be740 1 -- 172.21.15.120:0/4294712789 shutdown_connections 2024-09-19T09:18:13.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.419+0000 7f5dfa9be740 1 --2- 172.21.15.120:0/4294712789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efc444d370 0x55efc444f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.419+0000 7f5dfa9be740 1 --2- 172.21.15.120:0/4294712789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efc44425d0 0x55efc4442a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.419+0000 7f5dfa9be740 1 --2- 172.21.15.120:0/4294712789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efc4443b40 0x55efc4442090 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.419+0000 7f5dfa9be740 1 -- 172.21.15.120:0/4294712789 >> 172.21.15.120:0/4294712789 conn(0x55efc440b310 msgr2=0x55efc443ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:13.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.419+0000 7f5dfa9be740 1 -- 172.21.15.120:0/4294712789 shutdown_connections 2024-09-19T09:18:13.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.419+0000 7f5dfa9be740 1 -- 172.21.15.120:0/4294712789 wait complete. 2024-09-19T09:18:13.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.419+0000 7f5dfa9be740 1 -- start start 2024-09-19T09:18:13.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.419+0000 7f5dfa9be740 1 Processor -- start 2024-09-19T09:18:13.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.419+0000 7f5dfa9be740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efc44425d0 0x55efc444cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:13.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.419+0000 7f5dfa9be740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efc4443b40 0x55efc4449f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:13.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.419+0000 7f5dfa9be740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efc444d370 0x55efc444a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:13.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.419+0000 7f5dfa9be740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55efc4450010 con 0x55efc444d370 2024-09-19T09:18:13.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.419+0000 7f5dfa9be740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55efc4363060 con 0x55efc44425d0 2024-09-19T09:18:13.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.419+0000 7f5dfa9be740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55efc4450190 con 0x55efc4443b40 2024-09-19T09:18:13.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.419+0000 7f5dfa9be740 10 client.0 _ll_register_callbacks cb 0x55efc40b68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:13.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.420+0000 7f5df5b80640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efc444d370 0x55efc444a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:13.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.420+0000 7f5df5b80640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efc444d370 0x55efc444a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33046/0 (socket says 172.21.15.120:33046) 2024-09-19T09:18:13.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.420+0000 7f5df6381640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efc4443b40 0x55efc4449f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:13.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.420+0000 7f5df5b80640 1 -- 172.21.15.120:0/3388277372 learned_addr learned my addr 172.21.15.120:0/3388277372 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:13.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.420+0000 7f5df5b80640 1 --2- 172.21.15.120:0/3388277372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efc44425d0 0x55efc444cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:13.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.420+0000 7f5df6381640 1 -- 172.21.15.120:0/3388277372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efc44425d0 msgr2=0x55efc444cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:13.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.420+0000 7f5df6381640 1 --2- 172.21.15.120:0/3388277372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efc44425d0 0x55efc444cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.420+0000 7f5df6381640 1 -- 172.21.15.120:0/3388277372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efc444d370 msgr2=0x55efc444a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:13.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.420+0000 7f5df6381640 1 --2- 172.21.15.120:0/3388277372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efc444d370 0x55efc444a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.420+0000 7f5df6381640 1 -- 172.21.15.120:0/3388277372 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efc41193c0 con 0x55efc4443b40 2024-09-19T09:18:13.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.420+0000 7f5df5b80640 1 --2- 172.21.15.120:0/3388277372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efc444d370 0x55efc444a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:18:13.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.420+0000 7f5df6381640 1 --2- 172.21.15.120:0/3388277372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efc4443b40 0x55efc4449f90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5de000ce60 tx=0x7f5de0007680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=382760abe1dab29e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:13.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.420+0000 7f5dd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:18:13.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.420+0000 7f5dd5ffb640 1 -- 172.21.15.120:0/3388277372 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de0009e20 con 0x55efc4443b40 2024-09-19T09:18:13.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.420+0000 7f5dd5ffb640 1 -- 172.21.15.120:0/3388277372 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f5de0016400 con 0x55efc4443b40 2024-09-19T09:18:13.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.420+0000 7f5dfa9be740 1 -- 172.21.15.120:0/3388277372 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5de4005500 con 0x55efc4443b40 2024-09-19T09:18:13.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.420+0000 7f5dd5ffb640 1 -- 172.21.15.120:0/3388277372 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5de0016600 con 0x55efc4443b40 2024-09-19T09:18:13.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.420+0000 7f5dfa9be740 10 client.29416 fetch_fsmap learned FSMap version 36 2024-09-19T09:18:13.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.420+0000 7f5dfa9be740 1 -- 172.21.15.120:0/3388277372 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55efc445ba20 con 0x55efc4443b40 2024-09-19T09:18:13.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.421+0000 7f5dd5ffb640 1 -- 172.21.15.120:0/3388277372 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5de4005500 con 0x55efc4443b40 2024-09-19T09:18:13.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.421+0000 7f5dd5ffb640 1 -- 172.21.15.120:0/3388277372 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f5de001f710 con 0x55efc4443b40 2024-09-19T09:18:13.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.421+0000 7f5dfa9be740 10 client.29416 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:18:13.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.421+0000 7f5dfa9be740 10 client.29416 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:18:13.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.421+0000 7f5dfa9be740 20 client.29416 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:13.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.421+0000 7f5dfa9be740 1 --2- 172.21.15.120:0/3388277372 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55efc445cef0 0x55efc445f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:13.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.421+0000 7f5dfa9be740 4 client.29416 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:18:13.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.421+0000 7f5dfa9be740 1 -- 172.21.15.120:0/3388277372 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efc43b5860 con 0x55efc445cef0 2024-09-19T09:18:13.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.421+0000 7f5df5b80640 1 --2- 172.21.15.120:0/3388277372 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55efc445cef0 0x55efc445f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:13.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.421+0000 7f5dd5ffb640 10 client.29416 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:18:13.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.421+0000 7f5df5b80640 1 --2- 172.21.15.120:0/3388277372 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55efc445cef0 0x55efc445f310 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=558858a0fec55fdf server_cookie=97ebdb219b51e38f in_seq=0 out_seq=0 2024-09-19T09:18:13.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.422+0000 7f5dd5ffb640 1 -- 172.21.15.120:0/3388277372 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55efc43b5860 con 0x55efc445cef0 2024-09-19T09:18:13.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.422+0000 7f5dd5ffb640 10 client.29416 handle_command_reply: tid=0 2024-09-19T09:18:13.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.422+0000 7f5dfa9be740 1 client.29416 shutdown 2024-09-19T09:18:13.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.422+0000 7f5dfa9be740 1 -- 172.21.15.120:0/3388277372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efc4443b40 msgr2=0x55efc4449f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:13.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.422+0000 7f5dfa9be740 1 --2- 172.21.15.120:0/3388277372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efc4443b40 0x55efc4449f90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5de000ce60 tx=0x7f5de0007680 comp rx=0 tx=0).stop 2024-09-19T09:18:13.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.422+0000 7f5dfa9be740 1 -- 172.21.15.120:0/3388277372 shutdown_connections 2024-09-19T09:18:13.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.422+0000 7f5dfa9be740 1 --2- 172.21.15.120:0/3388277372 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55efc445cef0 0x55efc445f310 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.422+0000 7f5dfa9be740 1 --2- 172.21.15.120:0/3388277372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efc444d370 0x55efc444a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.422+0000 7f5dfa9be740 1 --2- 172.21.15.120:0/3388277372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efc4443b40 0x55efc4449f90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.422+0000 7f5dd5ffb640 0 client.29416 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:18:13.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.422+0000 7f5dfa9be740 1 --2- 172.21.15.120:0/3388277372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efc44425d0 0x55efc444cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.422+0000 7f5dfa9be740 1 -- 172.21.15.120:0/3388277372 >> 172.21.15.120:0/3388277372 conn(0x55efc440b310 msgr2=0x55efc4444930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:13.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.422+0000 7f5dfa9be740 1 -- 172.21.15.120:0/3388277372 shutdown_connections 2024-09-19T09:18:13.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.422+0000 7f5dfa9be740 1 -- 172.21.15.120:0/3388277372 wait complete. 2024-09-19T09:18:13.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.422+0000 7f5dfa9be740 20 client.29416 trim_cache size 0 max 16384 2024-09-19T09:18:13.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:12 smithi196 ceph-mon[35150]: pgmap v805: 97 pgs: 97 active+clean; 295 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 5.0 MiB/s wr, 381 op/s 2024-09-19T09:18:13.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.435+0000 7f5dfa9be740 1 Processor -- start 2024-09-19T09:18:13.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.435+0000 7f5dfa9be740 1 -- start start 2024-09-19T09:18:13.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.435+0000 7f5dfa9be740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efc44425d0 0x55efc44592b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:13.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.435+0000 7f5dfa9be740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efc4443b40 0x55efc443dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:13.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.435+0000 7f5dfa9be740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efc444d370 0x55efc4464be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:13.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.435+0000 7f5dfa9be740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5df01ad630 con 0x55efc444d370 2024-09-19T09:18:13.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.435+0000 7f5dfa9be740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5df01af200 con 0x55efc4443b40 2024-09-19T09:18:13.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.435+0000 7f5dfa9be740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5df01add00 con 0x55efc44425d0 2024-09-19T09:18:13.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.435+0000 7f5df5b80640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efc44425d0 0x55efc44592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:13.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.435+0000 7f5df5b80640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efc444d370 0x55efc4464be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:13.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.436+0000 7f5df5b80640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efc444d370 0x55efc4464be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33058/0 (socket says 172.21.15.120:33058) 2024-09-19T09:18:13.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.436+0000 7f5df6381640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efc4443b40 0x55efc443dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:13.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.436+0000 7f5df5b80640 1 -- 172.21.15.120:0/3593594766 learned_addr learned my addr 172.21.15.120:0/3593594766 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:13.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.436+0000 7f5df5b80640 1 -- 172.21.15.120:0/3593594766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efc44425d0 msgr2=0x55efc44592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:13.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.436+0000 7f5df5b80640 1 --2- 172.21.15.120:0/3593594766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efc44425d0 0x55efc44592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.436+0000 7f5df5b80640 1 -- 172.21.15.120:0/3593594766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efc4443b40 msgr2=0x55efc443dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:13.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.436+0000 7f5df5b80640 1 --2- 172.21.15.120:0/3593594766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efc4443b40 0x55efc443dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.436+0000 7f5df5b80640 1 -- 172.21.15.120:0/3593594766 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efc4443530 con 0x55efc444d370 2024-09-19T09:18:13.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.436+0000 7f5df5b80640 1 --2- 172.21.15.120:0/3593594766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efc444d370 0x55efc4464be0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f5de400b910 tx=0x7f5de4004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3493412c8fe6c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:13.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.436+0000 7f5dd7fff640 1 -- 172.21.15.120:0/3593594766 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4004850 con 0x55efc444d370 2024-09-19T09:18:13.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.436+0000 7f5dd7fff640 1 -- 172.21.15.120:0/3593594766 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5de4036020 con 0x55efc444d370 2024-09-19T09:18:13.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.436+0000 7f5dd7fff640 1 -- 172.21.15.120:0/3593594766 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4030560 con 0x55efc444d370 2024-09-19T09:18:13.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.437+0000 7f5dfa9be740 1 -- 172.21.15.120:0/3593594766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efc444d370 msgr2=0x55efc4464be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:13.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.437+0000 7f5dfa9be740 1 --2- 172.21.15.120:0/3593594766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efc444d370 0x55efc4464be0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f5de400b910 tx=0x7f5de4004e00 comp rx=0 tx=0).stop 2024-09-19T09:18:13.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.437+0000 7f5dfa9be740 1 -- 172.21.15.120:0/3593594766 shutdown_connections 2024-09-19T09:18:13.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.437+0000 7f5dfa9be740 1 --2- 172.21.15.120:0/3593594766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efc444d370 0x55efc4464be0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.437+0000 7f5dfa9be740 1 --2- 172.21.15.120:0/3593594766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efc4443b40 0x55efc443dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.437+0000 7f5dfa9be740 1 --2- 172.21.15.120:0/3593594766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efc44425d0 0x55efc44592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.437+0000 7f5dfa9be740 1 -- 172.21.15.120:0/3593594766 >> 172.21.15.120:0/3593594766 conn(0x55efc440b310 msgr2=0x55efc443d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:13.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.437+0000 7f5dfa9be740 1 -- 172.21.15.120:0/3593594766 shutdown_connections 2024-09-19T09:18:13.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.437+0000 7f5dfa9be740 1 -- 172.21.15.120:0/3593594766 wait complete. 2024-09-19T09:18:13.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.437+0000 7f5dfa9be740 1 -- start start 2024-09-19T09:18:13.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.437+0000 7f5dfa9be740 1 Processor -- start 2024-09-19T09:18:13.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.438+0000 7f5dfa9be740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efc44425d0 0x55efc44553f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:13.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.438+0000 7f5dfa9be740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efc4443b40 0x55efc4455930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:13.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.438+0000 7f5dfa9be740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efc444d370 0x55efc44523c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:13.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.438+0000 7f5dfa9be740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5df01aeb30 con 0x55efc44425d0 2024-09-19T09:18:13.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.438+0000 7f5dfa9be740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5df01ae3d0 con 0x55efc4443b40 2024-09-19T09:18:13.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.438+0000 7f5dfa9be740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5df01ab5f0 con 0x55efc444d370 2024-09-19T09:18:13.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.438+0000 7f5dfa9be740 10 client.0 _ll_register_callbacks cb 0x7f5de0009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:13.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.438+0000 7f5df5b80640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efc44425d0 0x55efc44553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:13.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.438+0000 7f5df5b80640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efc44425d0 0x55efc44553f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33066/0 (socket says 172.21.15.120:33066) 2024-09-19T09:18:13.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.438+0000 7f5df5b80640 1 -- 172.21.15.120:0/3581434092 learned_addr learned my addr 172.21.15.120:0/3581434092 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:13.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.438+0000 7f5df6381640 1 --2- 172.21.15.120:0/3581434092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efc4443b40 0x55efc4455930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:13.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.438+0000 7f5df5b80640 1 --2- 172.21.15.120:0/3581434092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efc444d370 0x55efc44523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:13.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.438+0000 7f5df5b80640 1 -- 172.21.15.120:0/3581434092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efc444d370 msgr2=0x55efc44523c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:13.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.438+0000 7f5df5b80640 1 --2- 172.21.15.120:0/3581434092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efc444d370 0x55efc44523c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.438+0000 7f5df5b80640 1 -- 172.21.15.120:0/3581434092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efc4443b40 msgr2=0x55efc4455930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:13.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.438+0000 7f5df5b80640 1 --2- 172.21.15.120:0/3581434092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efc4443b40 0x55efc4455930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.438+0000 7f5df5b80640 1 -- 172.21.15.120:0/3581434092 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efc4445120 con 0x55efc44425d0 2024-09-19T09:18:13.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.438+0000 7f5df6381640 1 --2- 172.21.15.120:0/3581434092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efc4443b40 0x55efc4455930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:18:13.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.438+0000 7f5df5b80640 1 --2- 172.21.15.120:0/3581434092 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efc44425d0 0x55efc44553f0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f5de4005f70 tx=0x7f5de40041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27508664120cb088 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:13.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.438+0000 7f5dd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:18:13.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.438+0000 7f5dd5ffb640 1 -- 172.21.15.120:0/3581434092 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4004aa0 con 0x55efc44425d0 2024-09-19T09:18:13.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.438+0000 7f5dd5ffb640 1 -- 172.21.15.120:0/3581434092 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f5de4030e80 con 0x55efc44425d0 2024-09-19T09:18:13.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.438+0000 7f5dfa9be740 1 -- 172.21.15.120:0/3581434092 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5de4036020 con 0x55efc44425d0 2024-09-19T09:18:13.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.438+0000 7f5dd5ffb640 1 -- 172.21.15.120:0/3581434092 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5de4005080 con 0x55efc44425d0 2024-09-19T09:18:13.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.438+0000 7f5dfa9be740 10 client.20664 fetch_fsmap learned FSMap version 36 2024-09-19T09:18:13.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.439+0000 7f5dfa9be740 1 -- 172.21.15.120:0/3581434092 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55efc443f960 con 0x55efc44425d0 2024-09-19T09:18:13.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.439+0000 7f5dd5ffb640 1 -- 172.21.15.120:0/3581434092 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5de4036020 con 0x55efc44425d0 2024-09-19T09:18:13.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.439+0000 7f5dd5ffb640 1 -- 172.21.15.120:0/3581434092 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f5de401d040 con 0x55efc44425d0 2024-09-19T09:18:13.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.439+0000 7f5dfa9be740 10 client.20664 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:18:13.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.439+0000 7f5dfa9be740 10 client.20664 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:18:13.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.439+0000 7f5dfa9be740 20 client.20664 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:13.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.439+0000 7f5dfa9be740 1 --2- 172.21.15.120:0/3581434092 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55efc44cc710 0x55efc44ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:13.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.439+0000 7f5dfa9be740 4 client.20664 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:18:13.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.439+0000 7f5dfa9be740 1 -- 172.21.15.120:0/3581434092 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55efc43afd70 con 0x55efc44cc710 2024-09-19T09:18:13.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.439+0000 7f5df6381640 1 --2- 172.21.15.120:0/3581434092 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55efc44cc710 0x55efc44ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:13.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.439+0000 7f5dd5ffb640 10 client.20664 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:18:13.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.439+0000 7f5df6381640 1 --2- 172.21.15.120:0/3581434092 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55efc44cc710 0x55efc44ceb00 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7a2f1b6ceca87a15 server_cookie=38678311c5fed65c in_seq=0 out_seq=0 2024-09-19T09:18:13.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.440+0000 7f5dd5ffb640 1 -- 172.21.15.120:0/3581434092 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55efc43afd70 con 0x55efc44cc710 2024-09-19T09:18:13.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.440+0000 7f5dd5ffb640 10 client.20664 handle_command_reply: tid=0 2024-09-19T09:18:13.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.440+0000 7f5dfa9be740 1 client.20664 shutdown 2024-09-19T09:18:13.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.440+0000 7f5dfa9be740 1 -- 172.21.15.120:0/3581434092 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efc44425d0 msgr2=0x55efc44553f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:13.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.440+0000 7f5dfa9be740 1 --2- 172.21.15.120:0/3581434092 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efc44425d0 0x55efc44553f0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f5de4005f70 tx=0x7f5de40041a0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.440+0000 7f5dfa9be740 1 -- 172.21.15.120:0/3581434092 shutdown_connections 2024-09-19T09:18:13.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.440+0000 7f5dfa9be740 1 --2- 172.21.15.120:0/3581434092 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55efc44cc710 0x55efc44ceb00 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.440+0000 7f5dd5ffb640 0 client.20664 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:18:13.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.440+0000 7f5dfa9be740 1 --2- 172.21.15.120:0/3581434092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efc444d370 0x55efc44523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.440+0000 7f5dfa9be740 1 --2- 172.21.15.120:0/3581434092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efc4443b40 0x55efc4455930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.440+0000 7f5dfa9be740 1 --2- 172.21.15.120:0/3581434092 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efc44425d0 0x55efc44553f0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.440+0000 7f5dfa9be740 1 -- 172.21.15.120:0/3581434092 >> 172.21.15.120:0/3581434092 conn(0x55efc440b310 msgr2=0x55efc444c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:13.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.440+0000 7f5dfa9be740 1 -- 172.21.15.120:0/3581434092 shutdown_connections 2024-09-19T09:18:13.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.440+0000 7f5dfa9be740 1 -- 172.21.15.120:0/3581434092 wait complete. 2024-09-19T09:18:13.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.440+0000 7f5dfa9be740 20 client.20664 trim_cache size 0 max 16384 2024-09-19T09:18:13.453 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:18:13.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.440+0000 7f5df85e5640 1 -- 172.21.15.120:0/282692117 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5dd80a32c0 msgr2=0x7f5dd80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:13.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.440+0000 7f5df85e5640 1 --2- 172.21.15.120:0/282692117 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5dd80a32c0 0x7f5dd80a5780 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5de80b8b70 tx=0x7f5de808c380 comp rx=0 tx=0).stop 2024-09-19T09:18:13.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.440+0000 7f5df85e5640 1 -- 172.21.15.120:0/282692117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5df0105a90 msgr2=0x7f5df019ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:13.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.440+0000 7f5df85e5640 1 --2- 172.21.15.120:0/282692117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5df0105a90 0x7f5df019ce00 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f5de8004770 tx=0x7f5de8003c10 comp rx=0 tx=0).stop 2024-09-19T09:18:13.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.441+0000 7f5df85e5640 1 -- 172.21.15.120:0/282692117 shutdown_connections 2024-09-19T09:18:13.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.441+0000 7f5df85e5640 1 --2- 172.21.15.120:0/282692117 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5dd80a32c0 0x7f5dd80a5780 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.441+0000 7f5df85e5640 1 --2- 172.21.15.120:0/282692117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5df0105a90 0x7f5df019ce00 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.441+0000 7f5df85e5640 1 --2- 172.21.15.120:0/282692117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5df0101bc0 0x7f5df0198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.441+0000 7f5df85e5640 1 --2- 172.21.15.120:0/282692117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5df0101240 0x7f5df0198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.441+0000 7f5df85e5640 1 -- 172.21.15.120:0/282692117 >> 172.21.15.120:0/282692117 conn(0x7f5df0077210 msgr2=0x7f5df00fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:13.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.441+0000 7f5df85e5640 1 -- 172.21.15.120:0/282692117 shutdown_connections 2024-09-19T09:18:13.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:13.441+0000 7f5df85e5640 1 -- 172.21.15.120:0/282692117 wait complete. 2024-09-19T09:18:14.357 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55751 0.00 MB/sec execute 9 sec latency 16481.259 ms 2024-09-19T09:18:14.526 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:18:14.780 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:18:15.187 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:14 smithi120 ceph-mon[31150]: pgmap v806: 97 pgs: 97 active+clean; 292 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 4.8 MiB/s wr, 337 op/s 2024-09-19T09:18:15.188 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:14 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:18:15.188 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:14 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:18:15.188 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:14 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:18:15.188 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:14 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:18:15.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.267+0000 7ff4684d1640 1 -- 172.21.15.120:0/1509775632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff460101420 msgr2=0x7ff46010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:15.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.267+0000 7ff4684d1640 1 --2- 172.21.15.120:0/1509775632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff460101420 0x7ff46010d950 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7ff458009cd0 tx=0x7ff45802a8e0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.267+0000 7ff4684d1640 1 -- 172.21.15.120:0/1509775632 shutdown_connections 2024-09-19T09:18:15.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.267+0000 7ff4684d1640 1 --2- 172.21.15.120:0/1509775632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff460101420 0x7ff46010d950 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.267+0000 7ff4684d1640 1 --2- 172.21.15.120:0/1509775632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff460100a80 0x7ff460100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.267+0000 7ff4684d1640 1 --2- 172.21.15.120:0/1509775632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff460106a80 0x7ff460106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.267+0000 7ff4684d1640 1 -- 172.21.15.120:0/1509775632 >> 172.21.15.120:0/1509775632 conn(0x7ff4600fc7a0 msgr2=0x7ff4600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:15.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.267+0000 7ff4684d1640 1 -- 172.21.15.120:0/1509775632 shutdown_connections 2024-09-19T09:18:15.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.267+0000 7ff4684d1640 1 -- 172.21.15.120:0/1509775632 wait complete. 2024-09-19T09:18:15.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.268+0000 7ff4684d1640 1 Processor -- start 2024-09-19T09:18:15.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.268+0000 7ff4684d1640 1 -- start start 2024-09-19T09:18:15.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.268+0000 7ff4684d1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff460100a80 0x7ff460102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:15.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.268+0000 7ff4684d1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff460101420 0x7ff460102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:15.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.268+0000 7ff4684d1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff460106a80 0x7ff4601030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:15.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.268+0000 7ff4684d1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff460110e80 con 0x7ff460101420 2024-09-19T09:18:15.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.268+0000 7ff4684d1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff460110d00 con 0x7ff460100a80 2024-09-19T09:18:15.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.268+0000 7ff4684d1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff460111000 con 0x7ff460106a80 2024-09-19T09:18:15.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.268+0000 7ff465a6c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff460101420 0x7ff460102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:15.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.268+0000 7ff465a6c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff460101420 0x7ff460102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33088/0 (socket says 172.21.15.120:33088) 2024-09-19T09:18:15.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.268+0000 7ff465a6c640 1 -- 172.21.15.120:0/1176630716 learned_addr learned my addr 172.21.15.120:0/1176630716 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:15.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.268+0000 7ff46626d640 1 --2- 172.21.15.120:0/1176630716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff460100a80 0x7ff460102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:15.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.268+0000 7ff465a6c640 1 -- 172.21.15.120:0/1176630716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff460106a80 msgr2=0x7ff4601030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:15.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.268+0000 7ff465a6c640 1 --2- 172.21.15.120:0/1176630716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff460106a80 0x7ff4601030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.268+0000 7ff465a6c640 1 -- 172.21.15.120:0/1176630716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff460100a80 msgr2=0x7ff460102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:15.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.268+0000 7ff465a6c640 1 --2- 172.21.15.120:0/1176630716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff460100a80 0x7ff460102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.268+0000 7ff465a6c640 1 -- 172.21.15.120:0/1176630716 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4601a67d0 con 0x7ff460101420 2024-09-19T09:18:15.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.268+0000 7ff46626d640 1 --2- 172.21.15.120:0/1176630716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff460100a80 0x7ff460102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:18:15.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.269+0000 7ff465a6c640 1 --2- 172.21.15.120:0/1176630716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff460101420 0x7ff460102b90 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7ff45000c920 tx=0x7ff45000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:15.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.269+0000 7ff4577fe640 1 -- 172.21.15.120:0/1176630716 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff450007c70 con 0x7ff460101420 2024-09-19T09:18:15.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.269+0000 7ff4684d1640 1 -- 172.21.15.120:0/1176630716 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4601a6ac0 con 0x7ff460101420 2024-09-19T09:18:15.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.269+0000 7ff4577fe640 1 -- 172.21.15.120:0/1176630716 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff450007e10 con 0x7ff460101420 2024-09-19T09:18:15.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.269+0000 7ff4577fe640 1 -- 172.21.15.120:0/1176630716 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff450005260 con 0x7ff460101420 2024-09-19T09:18:15.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.269+0000 7ff4684d1640 1 -- 172.21.15.120:0/1176630716 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4601a7050 con 0x7ff460101420 2024-09-19T09:18:15.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.271+0000 7ff4577fe640 1 -- 172.21.15.120:0/1176630716 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7ff450005400 con 0x7ff460101420 2024-09-19T09:18:15.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.271+0000 7ff46a8aa740 1 Processor -- start 2024-09-19T09:18:15.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.271+0000 7ff46a8aa740 1 -- start start 2024-09-19T09:18:15.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.271+0000 7ff46a8aa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddf5b6eb40 0x55ddf5b6d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:15.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.272+0000 7ff46a8aa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddf5b6d5d0 0x55ddf5b6da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:15.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.272+0000 7ff46a8aa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf5b78370 0x55ddf5b7a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:15.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.272+0000 7ff46a8aa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ddf5ad32f0 con 0x55ddf5b6d5d0 2024-09-19T09:18:15.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.272+0000 7ff46a8aa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ddf5a7acf0 con 0x55ddf5b6eb40 2024-09-19T09:18:15.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.272+0000 7ff46a8aa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ddf5acf2b0 con 0x55ddf5b78370 2024-09-19T09:18:15.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.272+0000 7ff46626d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddf5b6d5d0 0x55ddf5b6da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:15.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.272+0000 7ff466a6e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddf5b6eb40 0x55ddf5b6d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:15.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.272+0000 7ff46626d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddf5b6d5d0 0x55ddf5b6da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33090/0 (socket says 172.21.15.120:33090) 2024-09-19T09:18:15.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.272+0000 7ff46626d640 1 -- 172.21.15.120:0/2815563849 learned_addr learned my addr 172.21.15.120:0/2815563849 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:15.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.272+0000 7ff4577fe640 1 --2- 172.21.15.120:0/1176630716 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff44c0a3240 0x7ff44c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:15.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.272+0000 7ff466a6e640 1 --2- 172.21.15.120:0/1176630716 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff44c0a3240 0x7ff44c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:15.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.272+0000 7ff465a6c640 1 --2- 172.21.15.120:0/2815563849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf5b78370 0x55ddf5b7a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:15.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.272+0000 7ff4577fe640 1 -- 172.21.15.120:0/1176630716 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff4500cfb60 con 0x7ff460101420 2024-09-19T09:18:15.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.272+0000 7ff46626d640 1 -- 172.21.15.120:0/2815563849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf5b78370 msgr2=0x55ddf5b7a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:15.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.272+0000 7ff46626d640 1 --2- 172.21.15.120:0/2815563849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf5b78370 0x55ddf5b7a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.272+0000 7ff46626d640 1 -- 172.21.15.120:0/2815563849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddf5b6eb40 msgr2=0x55ddf5b6d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:15.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.272+0000 7ff46626d640 1 --2- 172.21.15.120:0/2815563849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddf5b6eb40 0x55ddf5b6d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.272+0000 7ff46626d640 1 -- 172.21.15.120:0/2815563849 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddf5a5f3a0 con 0x55ddf5b6d5d0 2024-09-19T09:18:15.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.272+0000 7ff465a6c640 1 --2- 172.21.15.120:0/2815563849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf5b78370 0x55ddf5b7a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:18:15.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.272+0000 7ff46626d640 1 --2- 172.21.15.120:0/2815563849 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddf5b6d5d0 0x55ddf5b6da10 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7ff45c00bba0 tx=0x7ff45c00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6b723e6688c843 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:15.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.272+0000 7ff466a6e640 1 --2- 172.21.15.120:0/1176630716 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff44c0a3240 0x7ff44c0a5700 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff458009ca0 tx=0x7ff4580095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:15.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.272+0000 7ff454ff9640 1 -- 172.21.15.120:0/2815563849 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c017620 con 0x55ddf5b6d5d0 2024-09-19T09:18:15.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.273+0000 7ff454ff9640 1 -- 172.21.15.120:0/2815563849 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff45c017ca0 con 0x55ddf5b6d5d0 2024-09-19T09:18:15.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.273+0000 7ff454ff9640 1 -- 172.21.15.120:0/2815563849 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c00fd80 con 0x55ddf5b6d5d0 2024-09-19T09:18:15.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.273+0000 7ff46a8aa740 1 -- 172.21.15.120:0/2815563849 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddf5b6d5d0 msgr2=0x55ddf5b6da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:15.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.273+0000 7ff46a8aa740 1 --2- 172.21.15.120:0/2815563849 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddf5b6d5d0 0x55ddf5b6da10 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7ff45c00bba0 tx=0x7ff45c00d710 comp rx=0 tx=0).stop 2024-09-19T09:18:15.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.273+0000 7ff46a8aa740 1 -- 172.21.15.120:0/2815563849 shutdown_connections 2024-09-19T09:18:15.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.273+0000 7ff46a8aa740 1 --2- 172.21.15.120:0/2815563849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf5b78370 0x55ddf5b7a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.273+0000 7ff46a8aa740 1 --2- 172.21.15.120:0/2815563849 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddf5b6d5d0 0x55ddf5b6da10 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.273+0000 7ff46a8aa740 1 --2- 172.21.15.120:0/2815563849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddf5b6eb40 0x55ddf5b6d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.273+0000 7ff46a8aa740 1 -- 172.21.15.120:0/2815563849 >> 172.21.15.120:0/2815563849 conn(0x55ddf5b36310 msgr2=0x55ddf5b69c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:15.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.273+0000 7ff46a8aa740 1 -- 172.21.15.120:0/2815563849 shutdown_connections 2024-09-19T09:18:15.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.273+0000 7ff46a8aa740 1 -- 172.21.15.120:0/2815563849 wait complete. 2024-09-19T09:18:15.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.274+0000 7ff46a8aa740 1 -- start start 2024-09-19T09:18:15.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.274+0000 7ff46a8aa740 1 Processor -- start 2024-09-19T09:18:15.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.274+0000 7ff46a8aa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddf5b6d5d0 0x55ddf5b77ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:15.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.274+0000 7ff46a8aa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf5b6eb40 0x55ddf5b74f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:15.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.274+0000 7ff46a8aa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddf5b78370 0x55ddf5b754f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:15.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.274+0000 7ff46a8aa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ddf5b7b070 con 0x55ddf5b6d5d0 2024-09-19T09:18:15.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.274+0000 7ff46a8aa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ddf5a8e060 con 0x55ddf5b78370 2024-09-19T09:18:15.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.274+0000 7ff46a8aa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ddf5b7b1f0 con 0x55ddf5b6eb40 2024-09-19T09:18:15.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.274+0000 7ff46a8aa740 10 client.0 _ll_register_callbacks cb 0x55ddf57e18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:15.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.274+0000 7ff46626d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddf5b6d5d0 0x55ddf5b77ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:15.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.274+0000 7ff46626d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddf5b6d5d0 0x55ddf5b77ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33094/0 (socket says 172.21.15.120:33094) 2024-09-19T09:18:15.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.274+0000 7ff46626d640 1 -- 172.21.15.120:0/2088802237 learned_addr learned my addr 172.21.15.120:0/2088802237 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:15.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.274+0000 7ff465a6c640 1 --2- 172.21.15.120:0/2088802237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf5b6eb40 0x55ddf5b74f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:15.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.274+0000 7ff466a6e640 1 --2- 172.21.15.120:0/2088802237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddf5b78370 0x55ddf5b754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:15.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.274+0000 7ff46626d640 1 -- 172.21.15.120:0/2088802237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf5b6eb40 msgr2=0x55ddf5b74f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:15.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.274+0000 7ff46626d640 1 --2- 172.21.15.120:0/2088802237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf5b6eb40 0x55ddf5b74f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.274+0000 7ff46626d640 1 -- 172.21.15.120:0/2088802237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddf5b78370 msgr2=0x55ddf5b754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:15.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.274+0000 7ff46626d640 1 --2- 172.21.15.120:0/2088802237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddf5b78370 0x55ddf5b754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.274+0000 7ff46626d640 1 -- 172.21.15.120:0/2088802237 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddf58443c0 con 0x55ddf5b6d5d0 2024-09-19T09:18:15.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.275+0000 7ff46626d640 1 --2- 172.21.15.120:0/2088802237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddf5b6d5d0 0x55ddf5b77ce0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7ff45c019040 tx=0x7ff45c00fdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550b72a970fd3f54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:15.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.275+0000 7ff449ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:18:15.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.275+0000 7ff449ffb640 1 -- 172.21.15.120:0/2088802237 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c026020 con 0x55ddf5b6d5d0 2024-09-19T09:18:15.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.275+0000 7ff449ffb640 1 -- 172.21.15.120:0/2088802237 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff45c005250 con 0x55ddf5b6d5d0 2024-09-19T09:18:15.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.275+0000 7ff46a8aa740 1 -- 172.21.15.120:0/2088802237 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff45c017ca0 con 0x55ddf5b6d5d0 2024-09-19T09:18:15.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.275+0000 7ff449ffb640 1 -- 172.21.15.120:0/2088802237 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff45c017e50 con 0x55ddf5b6d5d0 2024-09-19T09:18:15.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.275+0000 7ff46a8aa740 10 client.20676 fetch_fsmap learned FSMap version 36 2024-09-19T09:18:15.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.275+0000 7ff46a8aa740 1 -- 172.21.15.120:0/2088802237 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ddf5b86a20 con 0x55ddf5b6d5d0 2024-09-19T09:18:15.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.275+0000 7ff449ffb640 1 -- 172.21.15.120:0/2088802237 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff45c017ca0 con 0x55ddf5b6d5d0 2024-09-19T09:18:15.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.275+0000 7ff449ffb640 1 -- 172.21.15.120:0/2088802237 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7ff45c0046c0 con 0x55ddf5b6d5d0 2024-09-19T09:18:15.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.275+0000 7ff46a8aa740 10 client.20676 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:18:15.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.275+0000 7ff46a8aa740 10 client.20676 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:18:15.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.275+0000 7ff46a8aa740 20 client.20676 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:15.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.275+0000 7ff46a8aa740 1 --2- 172.21.15.120:0/2088802237 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55ddf5b87e30 0x55ddf5b8a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:15.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.275+0000 7ff46a8aa740 4 client.20676 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:18:15.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.275+0000 7ff46a8aa740 1 -- 172.21.15.120:0/2088802237 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddf5ae0860 con 0x55ddf5b87e30 2024-09-19T09:18:15.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.276+0000 7ff465a6c640 1 --2- 172.21.15.120:0/2088802237 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55ddf5b87e30 0x55ddf5b8a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:15.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.276+0000 7ff449ffb640 10 client.20676 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:18:15.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.276+0000 7ff465a6c640 1 --2- 172.21.15.120:0/2088802237 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55ddf5b87e30 0x55ddf5b8a220 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d66a7d4ff8eb958d server_cookie=b0c204bbe08454d8 in_seq=0 out_seq=0 2024-09-19T09:18:15.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.276+0000 7ff449ffb640 1 -- 172.21.15.120:0/2088802237 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ddf5ae0860 con 0x55ddf5b87e30 2024-09-19T09:18:15.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.276+0000 7ff449ffb640 10 client.20676 handle_command_reply: tid=0 2024-09-19T09:18:15.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.277+0000 7ff46a8aa740 1 client.20676 shutdown 2024-09-19T09:18:15.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.277+0000 7ff46a8aa740 1 -- 172.21.15.120:0/2088802237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddf5b6d5d0 msgr2=0x55ddf5b77ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:15.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.277+0000 7ff46a8aa740 1 --2- 172.21.15.120:0/2088802237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddf5b6d5d0 0x55ddf5b77ce0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7ff45c019040 tx=0x7ff45c00fdb0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.277+0000 7ff46a8aa740 1 -- 172.21.15.120:0/2088802237 shutdown_connections 2024-09-19T09:18:15.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.277+0000 7ff46a8aa740 1 --2- 172.21.15.120:0/2088802237 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55ddf5b87e30 0x55ddf5b8a220 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.277+0000 7ff46a8aa740 1 --2- 172.21.15.120:0/2088802237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddf5b78370 0x55ddf5b754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.277+0000 7ff46a8aa740 1 --2- 172.21.15.120:0/2088802237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf5b6eb40 0x55ddf5b74f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.277+0000 7ff46a8aa740 1 --2- 172.21.15.120:0/2088802237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddf5b6d5d0 0x55ddf5b77ce0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.277+0000 7ff449ffb640 0 client.20676 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:18:15.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.277+0000 7ff46a8aa740 1 -- 172.21.15.120:0/2088802237 >> 172.21.15.120:0/2088802237 conn(0x55ddf5b36310 msgr2=0x55ddf5b6f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:15.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.277+0000 7ff46a8aa740 1 -- 172.21.15.120:0/2088802237 shutdown_connections 2024-09-19T09:18:15.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.277+0000 7ff46a8aa740 1 -- 172.21.15.120:0/2088802237 wait complete. 2024-09-19T09:18:15.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.277+0000 7ff46a8aa740 20 client.20676 trim_cache size 0 max 16384 2024-09-19T09:18:15.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.290+0000 7ff46a8aa740 1 Processor -- start 2024-09-19T09:18:15.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.290+0000 7ff46a8aa740 1 -- start start 2024-09-19T09:18:15.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.290+0000 7ff46a8aa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf5b6d5d0 0x55ddf5b84340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:15.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.290+0000 7ff46a8aa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddf5b6eb40 0x55ddf5b8f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:15.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.290+0000 7ff46a8aa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddf5b78370 0x55ddf5b8fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:15.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.290+0000 7ff46a8aa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff4601af780 con 0x55ddf5b6eb40 2024-09-19T09:18:15.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.290+0000 7ff46a8aa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff4601b1350 con 0x55ddf5b78370 2024-09-19T09:18:15.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.290+0000 7ff46a8aa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff4601afe50 con 0x55ddf5b6d5d0 2024-09-19T09:18:15.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.290+0000 7ff465a6c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddf5b6eb40 0x55ddf5b8f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:15.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.290+0000 7ff465a6c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddf5b6eb40 0x55ddf5b8f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33096/0 (socket says 172.21.15.120:33096) 2024-09-19T09:18:15.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.290+0000 7ff465a6c640 1 -- 172.21.15.120:0/2197466740 learned_addr learned my addr 172.21.15.120:0/2197466740 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:15.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.290+0000 7ff466a6e640 1 --2- 172.21.15.120:0/2197466740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddf5b78370 0x55ddf5b8fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:15.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.290+0000 7ff46626d640 1 --2- 172.21.15.120:0/2197466740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf5b6d5d0 0x55ddf5b84340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:15.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.290+0000 7ff465a6c640 1 -- 172.21.15.120:0/2197466740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf5b6d5d0 msgr2=0x55ddf5b84340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:15.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.290+0000 7ff465a6c640 1 --2- 172.21.15.120:0/2197466740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf5b6d5d0 0x55ddf5b84340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.290+0000 7ff465a6c640 1 -- 172.21.15.120:0/2197466740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddf5b78370 msgr2=0x55ddf5b8fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:15.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.290+0000 7ff465a6c640 1 --2- 172.21.15.120:0/2197466740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddf5b78370 0x55ddf5b8fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.290+0000 7ff465a6c640 1 -- 172.21.15.120:0/2197466740 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddf5b734e0 con 0x55ddf5b6eb40 2024-09-19T09:18:15.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.290+0000 7ff465a6c640 1 --2- 172.21.15.120:0/2197466740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddf5b6eb40 0x55ddf5b8f880 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7ff4500985e0 tx=0x7ff4500bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cdd134c78c818df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:15.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.291+0000 7ff44bfff640 1 -- 172.21.15.120:0/2197466740 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4500a4df0 con 0x55ddf5b6eb40 2024-09-19T09:18:15.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.291+0000 7ff44bfff640 1 -- 172.21.15.120:0/2197466740 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff450008230 con 0x55ddf5b6eb40 2024-09-19T09:18:15.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.291+0000 7ff44bfff640 1 -- 172.21.15.120:0/2197466740 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4500bc0b0 con 0x55ddf5b6eb40 2024-09-19T09:18:15.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.291+0000 7ff46a8aa740 1 -- 172.21.15.120:0/2197466740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddf5b6eb40 msgr2=0x55ddf5b8f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:15.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.291+0000 7ff46a8aa740 1 --2- 172.21.15.120:0/2197466740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddf5b6eb40 0x55ddf5b8f880 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7ff4500985e0 tx=0x7ff4500bc040 comp rx=0 tx=0).stop 2024-09-19T09:18:15.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.291+0000 7ff46a8aa740 1 -- 172.21.15.120:0/2197466740 shutdown_connections 2024-09-19T09:18:15.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.291+0000 7ff46a8aa740 1 --2- 172.21.15.120:0/2197466740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddf5b78370 0x55ddf5b8fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.291+0000 7ff46a8aa740 1 --2- 172.21.15.120:0/2197466740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddf5b6eb40 0x55ddf5b8f880 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.291+0000 7ff46a8aa740 1 --2- 172.21.15.120:0/2197466740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf5b6d5d0 0x55ddf5b84340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.291+0000 7ff46a8aa740 1 -- 172.21.15.120:0/2197466740 >> 172.21.15.120:0/2197466740 conn(0x55ddf5b36310 msgr2=0x55ddf5b74080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:15.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.291+0000 7ff46a8aa740 1 -- 172.21.15.120:0/2197466740 shutdown_connections 2024-09-19T09:18:15.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.291+0000 7ff46a8aa740 1 -- 172.21.15.120:0/2197466740 wait complete. 2024-09-19T09:18:15.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.291+0000 7ff46a8aa740 1 -- start start 2024-09-19T09:18:15.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.292+0000 7ff46a8aa740 1 Processor -- start 2024-09-19T09:18:15.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.292+0000 7ff46a8aa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf5b6d5d0 0x55ddf5b71e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:15.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.292+0000 7ff46a8aa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddf5b6eb40 0x55ddf5b723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:15.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.292+0000 7ff46a8aa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddf5b78370 0x55ddf5b728f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:15.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.292+0000 7ff46a8aa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff4601b0c80 con 0x55ddf5b78370 2024-09-19T09:18:15.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.292+0000 7ff46a8aa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff4601b0520 con 0x55ddf5b6eb40 2024-09-19T09:18:15.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.292+0000 7ff46a8aa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff4601ad740 con 0x55ddf5b6d5d0 2024-09-19T09:18:15.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.292+0000 7ff46a8aa740 10 client.0 _ll_register_callbacks cb 0x7ff458009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:15.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.292+0000 7ff466a6e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddf5b78370 0x55ddf5b728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:15.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.292+0000 7ff466a6e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddf5b78370 0x55ddf5b728f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33108/0 (socket says 172.21.15.120:33108) 2024-09-19T09:18:15.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.292+0000 7ff466a6e640 1 -- 172.21.15.120:0/1196013922 learned_addr learned my addr 172.21.15.120:0/1196013922 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:15.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.292+0000 7ff46626d640 1 --2- 172.21.15.120:0/1196013922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf5b6d5d0 0x55ddf5b71e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:15.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.292+0000 7ff465a6c640 1 --2- 172.21.15.120:0/1196013922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddf5b6eb40 0x55ddf5b723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:15.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.292+0000 7ff466a6e640 1 -- 172.21.15.120:0/1196013922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf5b6d5d0 msgr2=0x55ddf5b71e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:15.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.292+0000 7ff466a6e640 1 --2- 172.21.15.120:0/1196013922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf5b6d5d0 0x55ddf5b71e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.292+0000 7ff466a6e640 1 -- 172.21.15.120:0/1196013922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddf5b6eb40 msgr2=0x55ddf5b723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:15.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.292+0000 7ff466a6e640 1 --2- 172.21.15.120:0/1196013922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddf5b6eb40 0x55ddf5b723b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.292+0000 7ff466a6e640 1 -- 172.21.15.120:0/1196013922 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddf5bfa060 con 0x55ddf5b78370 2024-09-19T09:18:15.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.292+0000 7ff465a6c640 1 --2- 172.21.15.120:0/1196013922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddf5b6eb40 0x55ddf5b723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:18:15.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.292+0000 7ff466a6e640 1 --2- 172.21.15.120:0/1196013922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddf5b78370 0x55ddf5b728f0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7ff458035000 tx=0x7ff4580093a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6faa138e4f2d4d7a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:15.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.292+0000 7ff449ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:18:15.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.293+0000 7ff449ffb640 1 -- 172.21.15.120:0/1196013922 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4580045c0 con 0x55ddf5b78370 2024-09-19T09:18:15.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.293+0000 7ff449ffb640 1 -- 172.21.15.120:0/1196013922 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff458004760 con 0x55ddf5b78370 2024-09-19T09:18:15.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.293+0000 7ff46a8aa740 1 -- 172.21.15.120:0/1196013922 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff450008230 con 0x55ddf5b78370 2024-09-19T09:18:15.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.293+0000 7ff449ffb640 1 -- 172.21.15.120:0/1196013922 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff4580305a0 con 0x55ddf5b78370 2024-09-19T09:18:15.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.293+0000 7ff46a8aa740 10 client.20682 fetch_fsmap learned FSMap version 36 2024-09-19T09:18:15.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.293+0000 7ff46a8aa740 1 -- 172.21.15.120:0/1196013922 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ddf5bfb240 con 0x55ddf5b78370 2024-09-19T09:18:15.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.293+0000 7ff449ffb640 1 -- 172.21.15.120:0/1196013922 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff450008230 con 0x55ddf5b78370 2024-09-19T09:18:15.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.293+0000 7ff449ffb640 1 -- 172.21.15.120:0/1196013922 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7ff458030070 con 0x55ddf5b78370 2024-09-19T09:18:15.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.293+0000 7ff46a8aa740 10 client.20682 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:18:15.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.293+0000 7ff46a8aa740 10 client.20682 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:18:15.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.293+0000 7ff46a8aa740 20 client.20682 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:15.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.293+0000 7ff46a8aa740 1 --2- 172.21.15.120:0/1196013922 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55ddf5bfb990 0x55ddf5bfddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:15.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.293+0000 7ff46a8aa740 4 client.20682 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:18:15.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.293+0000 7ff46a8aa740 1 -- 172.21.15.120:0/1196013922 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ddf5adad70 con 0x55ddf5bfb990 2024-09-19T09:18:15.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.293+0000 7ff46626d640 1 --2- 172.21.15.120:0/1196013922 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55ddf5bfb990 0x55ddf5bfddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:15.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.294+0000 7ff46626d640 1 --2- 172.21.15.120:0/1196013922 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55ddf5bfb990 0x55ddf5bfddb0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cb9c4389d1be853c server_cookie=da762e5d1fe6b01b in_seq=0 out_seq=0 2024-09-19T09:18:15.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.294+0000 7ff449ffb640 10 client.20682 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:18:15.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.294+0000 7ff449ffb640 1 -- 172.21.15.120:0/1196013922 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ddf5adad70 con 0x55ddf5bfb990 2024-09-19T09:18:15.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.294+0000 7ff449ffb640 10 client.20682 handle_command_reply: tid=0 2024-09-19T09:18:15.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.294+0000 7ff46a8aa740 1 client.20682 shutdown 2024-09-19T09:18:15.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.294+0000 7ff46a8aa740 1 -- 172.21.15.120:0/1196013922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddf5b78370 msgr2=0x55ddf5b728f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:15.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.294+0000 7ff46a8aa740 1 --2- 172.21.15.120:0/1196013922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddf5b78370 0x55ddf5b728f0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7ff458035000 tx=0x7ff4580093a0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.294+0000 7ff46a8aa740 1 -- 172.21.15.120:0/1196013922 shutdown_connections 2024-09-19T09:18:15.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.294+0000 7ff46a8aa740 1 --2- 172.21.15.120:0/1196013922 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55ddf5bfb990 0x55ddf5bfddb0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.294+0000 7ff46a8aa740 1 --2- 172.21.15.120:0/1196013922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddf5b78370 0x55ddf5b728f0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.294+0000 7ff46a8aa740 1 --2- 172.21.15.120:0/1196013922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddf5b6eb40 0x55ddf5b723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.294+0000 7ff46a8aa740 1 --2- 172.21.15.120:0/1196013922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddf5b6d5d0 0x55ddf5b71e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.294+0000 7ff46a8aa740 1 -- 172.21.15.120:0/1196013922 >> 172.21.15.120:0/1196013922 conn(0x55ddf5b36310 msgr2=0x55ddf5b684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:15.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.294+0000 7ff449ffb640 0 client.20682 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:18:15.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.294+0000 7ff46a8aa740 1 -- 172.21.15.120:0/1196013922 shutdown_connections 2024-09-19T09:18:15.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.294+0000 7ff46a8aa740 1 -- 172.21.15.120:0/1196013922 wait complete. 2024-09-19T09:18:15.313 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:18:15.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.294+0000 7ff46a8aa740 20 client.20682 trim_cache size 0 max 16384 2024-09-19T09:18:15.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.295+0000 7ff4684d1640 1 -- 172.21.15.120:0/1176630716 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff44c0a3240 msgr2=0x7ff44c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:15.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.295+0000 7ff4684d1640 1 --2- 172.21.15.120:0/1176630716 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff44c0a3240 0x7ff44c0a5700 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff458009ca0 tx=0x7ff4580095c0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.295+0000 7ff4684d1640 1 -- 172.21.15.120:0/1176630716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff460101420 msgr2=0x7ff460102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:15.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.295+0000 7ff4684d1640 1 --2- 172.21.15.120:0/1176630716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff460101420 0x7ff460102b90 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7ff45000c920 tx=0x7ff45000cde0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.295+0000 7ff4684d1640 1 -- 172.21.15.120:0/1176630716 shutdown_connections 2024-09-19T09:18:15.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.295+0000 7ff4684d1640 1 --2- 172.21.15.120:0/1176630716 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff44c0a3240 0x7ff44c0a5700 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.295+0000 7ff4684d1640 1 --2- 172.21.15.120:0/1176630716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff460106a80 0x7ff4601030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.295+0000 7ff4684d1640 1 --2- 172.21.15.120:0/1176630716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff460101420 0x7ff460102b90 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.295+0000 7ff4684d1640 1 --2- 172.21.15.120:0/1176630716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff460100a80 0x7ff460102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:15.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.295+0000 7ff4684d1640 1 -- 172.21.15.120:0/1176630716 >> 172.21.15.120:0/1176630716 conn(0x7ff4600fc7a0 msgr2=0x7ff4600fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:15.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.295+0000 7ff4684d1640 1 -- 172.21.15.120:0/1176630716 shutdown_connections 2024-09-19T09:18:15.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:15.295+0000 7ff4684d1640 1 -- 172.21.15.120:0/1176630716 wait complete. 2024-09-19T09:18:15.357 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55751 0.00 MB/sec execute 10 sec latency 17481.374 ms 2024-09-19T09:18:15.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:14 smithi196 ceph-mon[35150]: pgmap v806: 97 pgs: 97 active+clean; 292 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 4.8 MiB/s wr, 337 op/s 2024-09-19T09:18:15.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:14 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:18:15.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:14 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:18:15.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:14 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:18:15.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:14 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:18:15.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:14 smithi153 ceph-mon[35071]: pgmap v806: 97 pgs: 97 active+clean; 292 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 4.8 MiB/s wr, 337 op/s 2024-09-19T09:18:15.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:14 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:18:15.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:14 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:18:15.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:14 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:18:15.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:14 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:18:16.227 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:18:16.357 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55751 0.00 MB/sec execute 11 sec latency 18481.460 ms 2024-09-19T09:18:16.474 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:18:17.125 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:16 smithi120 ceph-mon[31150]: pgmap v807: 97 pgs: 97 active+clean; 291 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 4.5 MiB/s wr, 290 op/s 2024-09-19T09:18:17.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.208+0000 7f36cf3de640 1 -- 172.21.15.120:0/2545610913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36c8102ef0 msgr2=0x7f36c810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:17.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.208+0000 7f36cf3de640 1 --2- 172.21.15.120:0/2545610913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36c8102ef0 0x7f36c810da40 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f36b8009900 tx=0x7f36b802a480 comp rx=0 tx=0).stop 2024-09-19T09:18:17.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.208+0000 7f36cf3de640 1 -- 172.21.15.120:0/2545610913 shutdown_connections 2024-09-19T09:18:17.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.208+0000 7f36cf3de640 1 --2- 172.21.15.120:0/2545610913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36c8102ef0 0x7f36c810da40 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.208+0000 7f36cf3de640 1 --2- 172.21.15.120:0/2545610913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36c8102570 0x7f36c81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.208+0000 7f36cf3de640 1 --2- 172.21.15.120:0/2545610913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36c8100e20 0x7f36c8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.208+0000 7f36cf3de640 1 -- 172.21.15.120:0/2545610913 >> 172.21.15.120:0/2545610913 conn(0x7f36c80fc7d0 msgr2=0x7f36c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:17.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.209+0000 7f36cf3de640 1 -- 172.21.15.120:0/2545610913 shutdown_connections 2024-09-19T09:18:17.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.209+0000 7f36cf3de640 1 -- 172.21.15.120:0/2545610913 wait complete. 2024-09-19T09:18:17.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.209+0000 7f36cf3de640 1 Processor -- start 2024-09-19T09:18:17.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.209+0000 7f36cf3de640 1 -- start start 2024-09-19T09:18:17.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.209+0000 7f36cf3de640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36c8100e20 0x7f36c819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:17.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.209+0000 7f36cf3de640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36c8102570 0x7f36c819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:17.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.209+0000 7f36cf3de640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36c8102ef0 0x7f36c819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:17.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.210+0000 7f36cf3de640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f36c8110fd0 con 0x7f36c8102570 2024-09-19T09:18:17.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.210+0000 7f36cf3de640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f36c8110e50 con 0x7f36c8102ef0 2024-09-19T09:18:17.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.210+0000 7f36cf3de640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f36c8111150 con 0x7f36c8100e20 2024-09-19T09:18:17.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.210+0000 7f36cc979640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36c8102570 0x7f36c819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:17.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.210+0000 7f36cc979640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36c8102570 0x7f36c819ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33124/0 (socket says 172.21.15.120:33124) 2024-09-19T09:18:17.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.210+0000 7f36cc979640 1 -- 172.21.15.120:0/3937930388 learned_addr learned my addr 172.21.15.120:0/3937930388 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:17.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.210+0000 7f36cd17a640 1 --2- 172.21.15.120:0/3937930388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36c8100e20 0x7f36c819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:17.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.210+0000 7f36cd97b640 1 --2- 172.21.15.120:0/3937930388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36c8102ef0 0x7f36c819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:17.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.210+0000 7f36cc979640 1 -- 172.21.15.120:0/3937930388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36c8100e20 msgr2=0x7f36c819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:17.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.210+0000 7f36cc979640 1 --2- 172.21.15.120:0/3937930388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36c8100e20 0x7f36c819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.210+0000 7f36cc979640 1 -- 172.21.15.120:0/3937930388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36c8102ef0 msgr2=0x7f36c819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:17.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.210+0000 7f36cc979640 1 --2- 172.21.15.120:0/3937930388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36c8102ef0 0x7f36c819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.210+0000 7f36cc979640 1 -- 172.21.15.120:0/3937930388 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c819f5a0 con 0x7f36c8102570 2024-09-19T09:18:17.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.210+0000 7f36cd97b640 1 --2- 172.21.15.120:0/3937930388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36c8102ef0 0x7f36c819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:18:17.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.210+0000 7f36cc979640 1 --2- 172.21.15.120:0/3937930388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36c8102570 0x7f36c819ab30 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00c920 tx=0x7f36bc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:17.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.210+0000 7f36c67fc640 1 -- 172.21.15.120:0/3937930388 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc007c70 con 0x7f36c8102570 2024-09-19T09:18:17.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.210+0000 7f36c67fc640 1 -- 172.21.15.120:0/3937930388 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36bc007e10 con 0x7f36c8102570 2024-09-19T09:18:17.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.210+0000 7f36c67fc640 1 -- 172.21.15.120:0/3937930388 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc018890 con 0x7f36c8102570 2024-09-19T09:18:17.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.210+0000 7f36cf3de640 1 -- 172.21.15.120:0/3937930388 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c819f890 con 0x7f36c8102570 2024-09-19T09:18:17.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.210+0000 7f36cf3de640 1 -- 172.21.15.120:0/3937930388 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c81a7170 con 0x7f36c8102570 2024-09-19T09:18:17.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.212+0000 7f36c67fc640 1 -- 172.21.15.120:0/3937930388 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f36bc018a70 con 0x7f36c8102570 2024-09-19T09:18:17.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.213+0000 7f36d17b7740 1 Processor -- start 2024-09-19T09:18:17.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.213+0000 7f36d17b7740 1 -- start start 2024-09-19T09:18:17.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.213+0000 7f36d17b7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56058ce4db40 0x56058ce4c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:17.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.213+0000 7f36d17b7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56058ce4c5d0 0x56058ce4ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:17.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.213+0000 7f36d17b7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56058ce57370 0x56058ce59760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:17.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.213+0000 7f36d17b7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56058cdb22f0 con 0x56058ce4c5d0 2024-09-19T09:18:17.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.213+0000 7f36d17b7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56058cd59cf0 con 0x56058ce57370 2024-09-19T09:18:17.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.213+0000 7f36d17b7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56058cdae2b0 con 0x56058ce4db40 2024-09-19T09:18:17.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.213+0000 7f36cd17a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56058ce4c5d0 0x56058ce4ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:17.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.213+0000 7f36cd17a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56058ce4c5d0 0x56058ce4ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33140/0 (socket says 172.21.15.120:33140) 2024-09-19T09:18:17.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.213+0000 7f36cd17a640 1 -- 172.21.15.120:0/2525800719 learned_addr learned my addr 172.21.15.120:0/2525800719 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:17.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.213+0000 7f36c67fc640 1 --2- 172.21.15.120:0/3937930388 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f36b00a32c0 0x7f36b00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:17.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.213+0000 7f36cd97b640 1 --2- 172.21.15.120:0/2525800719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56058ce4db40 0x56058ce4c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:17.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.213+0000 7f36cd17a640 1 -- 172.21.15.120:0/2525800719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56058ce4db40 msgr2=0x56058ce4c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:17.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.213+0000 7f36cd17a640 1 --2- 172.21.15.120:0/2525800719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56058ce4db40 0x56058ce4c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.213+0000 7f36cd17a640 1 -- 172.21.15.120:0/2525800719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56058ce57370 msgr2=0x56058ce59760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:17.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.213+0000 7f36cd17a640 1 --2- 172.21.15.120:0/2525800719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56058ce57370 0x56058ce59760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.213+0000 7f36cd17a640 1 -- 172.21.15.120:0/2525800719 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56058cd3e3a0 con 0x56058ce4c5d0 2024-09-19T09:18:17.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.214+0000 7f36cd97b640 1 --2- 172.21.15.120:0/3937930388 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f36b00a32c0 0x7f36b00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:17.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.214+0000 7f36c67fc640 1 -- 172.21.15.120:0/3937930388 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f36bc0cfae0 con 0x7f36c8102570 2024-09-19T09:18:17.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.214+0000 7f36cd17a640 1 --2- 172.21.15.120:0/2525800719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56058ce4c5d0 0x56058ce4ca10 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f36c000ae90 tx=0x7f36c000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df8223a233e34ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:17.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.214+0000 7f36af7fe640 1 -- 172.21.15.120:0/2525800719 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c0011850 con 0x56058ce4c5d0 2024-09-19T09:18:17.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.214+0000 7f36af7fe640 1 -- 172.21.15.120:0/2525800719 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36c001a480 con 0x56058ce4c5d0 2024-09-19T09:18:17.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.214+0000 7f36af7fe640 1 -- 172.21.15.120:0/2525800719 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c0010630 con 0x56058ce4c5d0 2024-09-19T09:18:17.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.215+0000 7f36d17b7740 1 -- 172.21.15.120:0/2525800719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56058ce4c5d0 msgr2=0x56058ce4ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:17.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.215+0000 7f36cd97b640 1 --2- 172.21.15.120:0/3937930388 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f36b00a32c0 0x7f36b00a5780 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f36b8006010 tx=0x7f36b8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:17.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.215+0000 7f36d17b7740 1 --2- 172.21.15.120:0/2525800719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56058ce4c5d0 0x56058ce4ca10 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f36c000ae90 tx=0x7f36c000f730 comp rx=0 tx=0).stop 2024-09-19T09:18:17.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.215+0000 7f36d17b7740 1 -- 172.21.15.120:0/2525800719 shutdown_connections 2024-09-19T09:18:17.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.215+0000 7f36d17b7740 1 --2- 172.21.15.120:0/2525800719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56058ce57370 0x56058ce59760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.215+0000 7f36d17b7740 1 --2- 172.21.15.120:0/2525800719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56058ce4c5d0 0x56058ce4ca10 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.215+0000 7f36d17b7740 1 --2- 172.21.15.120:0/2525800719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56058ce4db40 0x56058ce4c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.215+0000 7f36d17b7740 1 -- 172.21.15.120:0/2525800719 >> 172.21.15.120:0/2525800719 conn(0x56058ce15310 msgr2=0x56058ce48c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:17.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.215+0000 7f36d17b7740 1 -- 172.21.15.120:0/2525800719 shutdown_connections 2024-09-19T09:18:17.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.215+0000 7f36d17b7740 1 -- 172.21.15.120:0/2525800719 wait complete. 2024-09-19T09:18:17.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.215+0000 7f36d17b7740 1 -- start start 2024-09-19T09:18:17.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.215+0000 7f36d17b7740 1 Processor -- start 2024-09-19T09:18:17.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.215+0000 7f36d17b7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56058ce4c5d0 0x56058ce54100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:17.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.216+0000 7f36d17b7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56058ce4db40 0x56058ce54640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:17.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.216+0000 7f36d17b7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56058ce57370 0x56058ce54b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:17.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.216+0000 7f36d17b7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56058ce5a070 con 0x56058ce4c5d0 2024-09-19T09:18:17.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.216+0000 7f36d17b7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56058cd6d060 con 0x56058ce57370 2024-09-19T09:18:17.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.216+0000 7f36d17b7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56058ce5a1f0 con 0x56058ce4db40 2024-09-19T09:18:17.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.216+0000 7f36d17b7740 10 client.0 _ll_register_callbacks cb 0x56058cac08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:17.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.216+0000 7f36cd17a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56058ce4c5d0 0x56058ce54100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:17.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.216+0000 7f36cd17a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56058ce4c5d0 0x56058ce54100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33150/0 (socket says 172.21.15.120:33150) 2024-09-19T09:18:17.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.216+0000 7f36cd17a640 1 -- 172.21.15.120:0/2683852569 learned_addr learned my addr 172.21.15.120:0/2683852569 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:17.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.216+0000 7f36cc979640 1 --2- 172.21.15.120:0/2683852569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56058ce4db40 0x56058ce54640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:17.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.216+0000 7f36cd97b640 1 --2- 172.21.15.120:0/2683852569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56058ce57370 0x56058ce54b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:17.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.216+0000 7f36cd17a640 1 -- 172.21.15.120:0/2683852569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56058ce4db40 msgr2=0x56058ce54640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:17.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.216+0000 7f36cd17a640 1 --2- 172.21.15.120:0/2683852569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56058ce4db40 0x56058ce54640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.216+0000 7f36cd17a640 1 -- 172.21.15.120:0/2683852569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56058ce57370 msgr2=0x56058ce54b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:17.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.216+0000 7f36cd17a640 1 --2- 172.21.15.120:0/2683852569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56058ce57370 0x56058ce54b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.216+0000 7f36cd17a640 1 -- 172.21.15.120:0/2683852569 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56058cb233c0 con 0x56058ce4c5d0 2024-09-19T09:18:17.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.216+0000 7f36cd97b640 1 --2- 172.21.15.120:0/2683852569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56058ce57370 0x56058ce54b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:18:17.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.216+0000 7f36cd17a640 1 --2- 172.21.15.120:0/2683852569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56058ce4c5d0 0x56058ce54100 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f36c0000f80 tx=0x7f36c001a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1afa4e7ee2860b09 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:17.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.216+0000 7f36acff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:18:17.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.216+0000 7f36acff9640 1 -- 172.21.15.120:0/2683852569 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c0029060 con 0x56058ce4c5d0 2024-09-19T09:18:17.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.216+0000 7f36acff9640 1 -- 172.21.15.120:0/2683852569 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f36c0004c50 con 0x56058ce4c5d0 2024-09-19T09:18:17.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.216+0000 7f36d17b7740 1 -- 172.21.15.120:0/2683852569 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36c001a480 con 0x56058ce4c5d0 2024-09-19T09:18:17.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.216+0000 7f36acff9640 1 -- 172.21.15.120:0/2683852569 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f36c0004e50 con 0x56058ce4c5d0 2024-09-19T09:18:17.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.216+0000 7f36d17b7740 10 client.20694 fetch_fsmap learned FSMap version 36 2024-09-19T09:18:17.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.216+0000 7f36d17b7740 1 -- 172.21.15.120:0/2683852569 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56058ce65c00 con 0x56058ce4c5d0 2024-09-19T09:18:17.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.217+0000 7f36acff9640 1 -- 172.21.15.120:0/2683852569 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36c001a480 con 0x56058ce4c5d0 2024-09-19T09:18:17.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.217+0000 7f36acff9640 1 -- 172.21.15.120:0/2683852569 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f36c002e070 con 0x56058ce4c5d0 2024-09-19T09:18:17.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.217+0000 7f36d17b7740 10 client.20694 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:18:17.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.217+0000 7f36d17b7740 10 client.20694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:18:17.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.217+0000 7f36d17b7740 20 client.20694 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:17.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.217+0000 7f36d17b7740 1 --2- 172.21.15.120:0/2683852569 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56058ce67000 0x56058ce693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:17.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.217+0000 7f36d17b7740 4 client.20694 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:18:17.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.217+0000 7f36d17b7740 1 -- 172.21.15.120:0/2683852569 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56058cdbf860 con 0x56058ce67000 2024-09-19T09:18:17.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.217+0000 7f36cc979640 1 --2- 172.21.15.120:0/2683852569 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56058ce67000 0x56058ce693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:17.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.217+0000 7f36cc979640 1 --2- 172.21.15.120:0/2683852569 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56058ce67000 0x56058ce693f0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55d98ce0a3faf90d server_cookie=97652ac808251ca6 in_seq=0 out_seq=0 2024-09-19T09:18:17.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.217+0000 7f36acff9640 10 client.20694 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:17.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.218+0000 7f36acff9640 1 -- 172.21.15.120:0/2683852569 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56058cdbf860 con 0x56058ce67000 2024-09-19T09:18:17.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.218+0000 7f36acff9640 10 client.20694 handle_command_reply: tid=0 2024-09-19T09:18:17.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.218+0000 7f36d17b7740 1 client.20694 shutdown 2024-09-19T09:18:17.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.218+0000 7f36d17b7740 1 -- 172.21.15.120:0/2683852569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56058ce4c5d0 msgr2=0x56058ce54100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:17.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.218+0000 7f36d17b7740 1 --2- 172.21.15.120:0/2683852569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56058ce4c5d0 0x56058ce54100 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f36c0000f80 tx=0x7f36c001a860 comp rx=0 tx=0).stop 2024-09-19T09:18:17.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.218+0000 7f36d17b7740 1 -- 172.21.15.120:0/2683852569 shutdown_connections 2024-09-19T09:18:17.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.218+0000 7f36d17b7740 1 --2- 172.21.15.120:0/2683852569 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56058ce67000 0x56058ce693f0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.218+0000 7f36d17b7740 1 --2- 172.21.15.120:0/2683852569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56058ce57370 0x56058ce54b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.218+0000 7f36d17b7740 1 --2- 172.21.15.120:0/2683852569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56058ce4db40 0x56058ce54640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.218+0000 7f36d17b7740 1 --2- 172.21.15.120:0/2683852569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56058ce4c5d0 0x56058ce54100 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.218+0000 7f36acff9640 0 client.20694 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:17.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.218+0000 7f36d17b7740 1 -- 172.21.15.120:0/2683852569 >> 172.21.15.120:0/2683852569 conn(0x56058ce15310 msgr2=0x56058ce4e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:17.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.218+0000 7f36d17b7740 1 -- 172.21.15.120:0/2683852569 shutdown_connections 2024-09-19T09:18:17.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.219+0000 7f36d17b7740 1 -- 172.21.15.120:0/2683852569 wait complete. 2024-09-19T09:18:17.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.219+0000 7f36d17b7740 20 client.20694 trim_cache size 0 max 16384 2024-09-19T09:18:17.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.231+0000 7f36d17b7740 1 Processor -- start 2024-09-19T09:18:17.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.231+0000 7f36d17b7740 1 -- start start 2024-09-19T09:18:17.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.231+0000 7f36d17b7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56058ce4c5d0 0x56058ce58760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:17.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.231+0000 7f36d17b7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56058ce4db40 0x56058ce58ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:17.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.231+0000 7f36d17b7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56058ce57370 0x56058ce540c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:17.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.231+0000 7f36d17b7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f36c81af770 con 0x56058ce4c5d0 2024-09-19T09:18:17.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.232+0000 7f36d17b7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f36c81b1340 con 0x56058ce57370 2024-09-19T09:18:17.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.232+0000 7f36d17b7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f36c81afe40 con 0x56058ce4db40 2024-09-19T09:18:17.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.232+0000 7f36cd17a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56058ce4c5d0 0x56058ce58760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:17.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.232+0000 7f36cd17a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56058ce4c5d0 0x56058ce58760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33162/0 (socket says 172.21.15.120:33162) 2024-09-19T09:18:17.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.232+0000 7f36cd17a640 1 -- 172.21.15.120:0/3461747940 learned_addr learned my addr 172.21.15.120:0/3461747940 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:17.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.232+0000 7f36cc979640 1 --2- 172.21.15.120:0/3461747940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56058ce4db40 0x56058ce58ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:17.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.232+0000 7f36cd17a640 1 -- 172.21.15.120:0/3461747940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56058ce4db40 msgr2=0x56058ce58ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:17.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.232+0000 7f36cd17a640 1 --2- 172.21.15.120:0/3461747940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56058ce4db40 0x56058ce58ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.232+0000 7f36cd17a640 1 -- 172.21.15.120:0/3461747940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56058ce57370 msgr2=0x56058ce540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:17.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.232+0000 7f36cd17a640 1 --2- 172.21.15.120:0/3461747940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56058ce57370 0x56058ce540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.232+0000 7f36cd17a640 1 -- 172.21.15.120:0/3461747940 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56058ce49960 con 0x56058ce4c5d0 2024-09-19T09:18:17.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.232+0000 7f36cd17a640 1 --2- 172.21.15.120:0/3461747940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56058ce4c5d0 0x56058ce58760 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f36c000f770 tx=0x7f36c001aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b07a50a51aa280e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:17.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.232+0000 7f36aeffd640 1 -- 172.21.15.120:0/3461747940 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c0034030 con 0x56058ce4c5d0 2024-09-19T09:18:17.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.232+0000 7f36aeffd640 1 -- 172.21.15.120:0/3461747940 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36c001a480 con 0x56058ce4c5d0 2024-09-19T09:18:17.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.232+0000 7f36aeffd640 1 -- 172.21.15.120:0/3461747940 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c0004c50 con 0x56058ce4c5d0 2024-09-19T09:18:17.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.233+0000 7f36d17b7740 1 -- 172.21.15.120:0/3461747940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56058ce4c5d0 msgr2=0x56058ce58760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:17.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.233+0000 7f36d17b7740 1 --2- 172.21.15.120:0/3461747940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56058ce4c5d0 0x56058ce58760 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f36c000f770 tx=0x7f36c001aeb0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.233+0000 7f36d17b7740 1 -- 172.21.15.120:0/3461747940 shutdown_connections 2024-09-19T09:18:17.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.233+0000 7f36d17b7740 1 --2- 172.21.15.120:0/3461747940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56058ce57370 0x56058ce540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.233+0000 7f36d17b7740 1 --2- 172.21.15.120:0/3461747940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56058ce4db40 0x56058ce58ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.233+0000 7f36d17b7740 1 --2- 172.21.15.120:0/3461747940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56058ce4c5d0 0x56058ce58760 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.233+0000 7f36d17b7740 1 -- 172.21.15.120:0/3461747940 >> 172.21.15.120:0/3461747940 conn(0x56058ce15310 msgr2=0x56058ce53210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:17.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.233+0000 7f36d17b7740 1 -- 172.21.15.120:0/3461747940 shutdown_connections 2024-09-19T09:18:17.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.233+0000 7f36d17b7740 1 -- 172.21.15.120:0/3461747940 wait complete. 2024-09-19T09:18:17.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.233+0000 7f36d17b7740 1 -- start start 2024-09-19T09:18:17.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.233+0000 7f36d17b7740 1 Processor -- start 2024-09-19T09:18:17.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.234+0000 7f36d17b7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56058ce4c5d0 0x56058ce51cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:17.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.234+0000 7f36d17b7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56058ce4db40 0x56058ce5f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:17.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.234+0000 7f36d17b7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56058ce57370 0x56058ce5f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:17.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.234+0000 7f36d17b7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f36c81b0c70 con 0x56058ce57370 2024-09-19T09:18:17.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.234+0000 7f36d17b7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f36c81b0510 con 0x56058ce4c5d0 2024-09-19T09:18:17.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.234+0000 7f36d17b7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f36c81ad730 con 0x56058ce4db40 2024-09-19T09:18:17.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.234+0000 7f36cd97b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56058ce57370 0x56058ce5f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:17.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.234+0000 7f36d17b7740 10 client.0 _ll_register_callbacks cb 0x7f36c000ad10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:17.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.234+0000 7f36cd97b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56058ce57370 0x56058ce5f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33168/0 (socket says 172.21.15.120:33168) 2024-09-19T09:18:17.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.234+0000 7f36cd97b640 1 -- 172.21.15.120:0/2780726305 learned_addr learned my addr 172.21.15.120:0/2780726305 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:17.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.234+0000 7f36cd17a640 1 --2- 172.21.15.120:0/2780726305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56058ce4c5d0 0x56058ce51cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:17.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.234+0000 7f36cc979640 1 --2- 172.21.15.120:0/2780726305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56058ce4db40 0x56058ce5f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:17.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.234+0000 7f36cd97b640 1 -- 172.21.15.120:0/2780726305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56058ce4db40 msgr2=0x56058ce5f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:17.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.234+0000 7f36cd97b640 1 --2- 172.21.15.120:0/2780726305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56058ce4db40 0x56058ce5f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.234+0000 7f36cd97b640 1 -- 172.21.15.120:0/2780726305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56058ce4c5d0 msgr2=0x56058ce51cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:17.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.234+0000 7f36cd97b640 1 --2- 172.21.15.120:0/2780726305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56058ce4c5d0 0x56058ce51cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.234+0000 7f36cd97b640 1 -- 172.21.15.120:0/2780726305 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56058ce4f120 con 0x56058ce57370 2024-09-19T09:18:17.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.234+0000 7f36cd97b640 1 --2- 172.21.15.120:0/2780726305 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56058ce57370 0x56058ce5f9c0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f36b8009900 tx=0x7f36b80095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3abb1c1bb97933a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:17.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.235+0000 7f36acff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:18:17.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.235+0000 7f36acff9640 1 -- 172.21.15.120:0/2780726305 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b804e070 con 0x56058ce57370 2024-09-19T09:18:17.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.235+0000 7f36acff9640 1 -- 172.21.15.120:0/2780726305 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f36b80035f0 con 0x56058ce57370 2024-09-19T09:18:17.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.235+0000 7f36d17b7740 1 -- 172.21.15.120:0/2780726305 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36c001a480 con 0x56058ce57370 2024-09-19T09:18:17.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.235+0000 7f36acff9640 1 -- 172.21.15.120:0/2780726305 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f36b8035040 con 0x56058ce57370 2024-09-19T09:18:17.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.235+0000 7f36d17b7740 10 client.20700 fetch_fsmap learned FSMap version 36 2024-09-19T09:18:17.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.235+0000 7f36d17b7740 1 -- 172.21.15.120:0/2780726305 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56058ced19f0 con 0x56058ce57370 2024-09-19T09:18:17.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.235+0000 7f36acff9640 1 -- 172.21.15.120:0/2780726305 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36c001a480 con 0x56058ce57370 2024-09-19T09:18:17.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.235+0000 7f36acff9640 1 -- 172.21.15.120:0/2780726305 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f36b8031aa0 con 0x56058ce57370 2024-09-19T09:18:17.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.235+0000 7f36d17b7740 10 client.20700 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:18:17.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.235+0000 7f36d17b7740 10 client.20700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:18:17.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.235+0000 7f36d17b7740 20 client.20700 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:17.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.235+0000 7f36d17b7740 1 --2- 172.21.15.120:0/2780726305 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56058ced21d0 0x56058ced45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:17.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.235+0000 7f36d17b7740 4 client.20700 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:18:17.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.235+0000 7f36d17b7740 1 -- 172.21.15.120:0/2780726305 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56058cdb9d70 con 0x56058ced21d0 2024-09-19T09:18:17.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.236+0000 7f36cd17a640 1 --2- 172.21.15.120:0/2780726305 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56058ced21d0 0x56058ced45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:17.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.236+0000 7f36cd17a640 1 --2- 172.21.15.120:0/2780726305 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56058ced21d0 0x56058ced45f0 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed9230c67b3d7ff server_cookie=275f9fabd05995c2 in_seq=0 out_seq=0 2024-09-19T09:18:17.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.236+0000 7f36acff9640 10 client.20700 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:17.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.236+0000 7f36acff9640 1 -- 172.21.15.120:0/2780726305 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56058cdb9d70 con 0x56058ced21d0 2024-09-19T09:18:17.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.236+0000 7f36acff9640 10 client.20700 handle_command_reply: tid=0 2024-09-19T09:18:17.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.236+0000 7f36d17b7740 1 client.20700 shutdown 2024-09-19T09:18:17.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.236+0000 7f36d17b7740 1 -- 172.21.15.120:0/2780726305 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56058ce57370 msgr2=0x56058ce5f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:17.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.236+0000 7f36d17b7740 1 --2- 172.21.15.120:0/2780726305 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56058ce57370 0x56058ce5f9c0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f36b8009900 tx=0x7f36b80095d0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.236+0000 7f36d17b7740 1 -- 172.21.15.120:0/2780726305 shutdown_connections 2024-09-19T09:18:17.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.236+0000 7f36d17b7740 1 --2- 172.21.15.120:0/2780726305 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56058ced21d0 0x56058ced45f0 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.236+0000 7f36acff9640 0 client.20700 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:17.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.236+0000 7f36d17b7740 1 --2- 172.21.15.120:0/2780726305 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56058ce57370 0x56058ce5f9c0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.236+0000 7f36d17b7740 1 --2- 172.21.15.120:0/2780726305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56058ce4db40 0x56058ce5f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.236+0000 7f36d17b7740 1 --2- 172.21.15.120:0/2780726305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56058ce4c5d0 0x56058ce51cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.236+0000 7f36d17b7740 1 -- 172.21.15.120:0/2780726305 >> 172.21.15.120:0/2780726305 conn(0x56058ce15310 msgr2=0x56058ce474e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:17.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.236+0000 7f36d17b7740 1 -- 172.21.15.120:0/2780726305 shutdown_connections 2024-09-19T09:18:17.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.236+0000 7f36d17b7740 1 -- 172.21.15.120:0/2780726305 wait complete. 2024-09-19T09:18:17.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.237+0000 7f36d17b7740 20 client.20700 trim_cache size 0 max 16384 2024-09-19T09:18:17.247 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:18:17.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.237+0000 7f36cf3de640 1 -- 172.21.15.120:0/3937930388 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f36b00a32c0 msgr2=0x7f36b00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:17.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.237+0000 7f36cf3de640 1 --2- 172.21.15.120:0/3937930388 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f36b00a32c0 0x7f36b00a5780 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f36b8006010 tx=0x7f36b8009260 comp rx=0 tx=0).stop 2024-09-19T09:18:17.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.237+0000 7f36cf3de640 1 -- 172.21.15.120:0/3937930388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36c8102570 msgr2=0x7f36c819ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:17.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.237+0000 7f36cf3de640 1 --2- 172.21.15.120:0/3937930388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36c8102570 0x7f36c819ab30 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00c920 tx=0x7f36bc00cde0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.237+0000 7f36cf3de640 1 -- 172.21.15.120:0/3937930388 shutdown_connections 2024-09-19T09:18:17.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.237+0000 7f36cf3de640 1 --2- 172.21.15.120:0/3937930388 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f36b00a32c0 0x7f36b00a5780 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.237+0000 7f36cf3de640 1 --2- 172.21.15.120:0/3937930388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36c8102ef0 0x7f36c819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.237+0000 7f36cf3de640 1 --2- 172.21.15.120:0/3937930388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36c8102570 0x7f36c819ab30 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.237+0000 7f36cf3de640 1 --2- 172.21.15.120:0/3937930388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36c8100e20 0x7f36c819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:17.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.237+0000 7f36cf3de640 1 -- 172.21.15.120:0/3937930388 >> 172.21.15.120:0/3937930388 conn(0x7f36c80fc7d0 msgr2=0x7f36c806a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:17.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.237+0000 7f36cf3de640 1 -- 172.21.15.120:0/3937930388 shutdown_connections 2024-09-19T09:18:17.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:17.237+0000 7f36cf3de640 1 -- 172.21.15.120:0/3937930388 wait complete. 2024-09-19T09:18:17.357 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55751 0.00 MB/sec execute 12 sec latency 19481.496 ms 2024-09-19T09:18:17.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:16 smithi196 ceph-mon[35150]: pgmap v807: 97 pgs: 97 active+clean; 291 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 4.5 MiB/s wr, 290 op/s 2024-09-19T09:18:17.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:16 smithi153 ceph-mon[35071]: pgmap v807: 97 pgs: 97 active+clean; 291 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 4.5 MiB/s wr, 290 op/s 2024-09-19T09:18:18.357 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55751 0.00 MB/sec execute 13 sec latency 20481.919 ms 2024-09-19T09:18:19.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:18 smithi120 ceph-mon[31150]: pgmap v808: 97 pgs: 97 active+clean; 290 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 5.3 MiB/s wr, 322 op/s 2024-09-19T09:18:19.357 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55751 0.00 MB/sec execute 14 sec latency 21481.975 ms 2024-09-19T09:18:19.402 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:18:19.402 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:18:19.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:18 smithi196 ceph-mon[35150]: pgmap v808: 97 pgs: 97 active+clean; 290 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 5.3 MiB/s wr, 322 op/s 2024-09-19T09:18:19.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:18 smithi153 ceph-mon[35071]: pgmap v808: 97 pgs: 97 active+clean; 290 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 5.3 MiB/s wr, 322 op/s 2024-09-19T09:18:19.662 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:18:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.716+0000 7fa91d21e640 1 -- 172.21.15.120:0/1167312917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa918100a40 msgr2=0x7fa918100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.716+0000 7fa91d21e640 1 --2- 172.21.15.120:0/1167312917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa918100a40 0x7fa918100e40 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fa908009900 tx=0x7fa90802a410 comp rx=0 tx=0).stop 2024-09-19T09:18:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.716+0000 7fa91d21e640 1 -- 172.21.15.120:0/1167312917 shutdown_connections 2024-09-19T09:18:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.716+0000 7fa91d21e640 1 --2- 172.21.15.120:0/1167312917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9181025f0 0x7fa918108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.716+0000 7fa91d21e640 1 --2- 172.21.15.120:0/1167312917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa918101c30 0x7fa9181020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.716+0000 7fa91d21e640 1 --2- 172.21.15.120:0/1167312917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa918100a40 0x7fa918100e40 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.716+0000 7fa91d21e640 1 -- 172.21.15.120:0/1167312917 >> 172.21.15.120:0/1167312917 conn(0x7fa9180fc1e0 msgr2=0x7fa9180fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.716+0000 7fa91d21e640 1 -- 172.21.15.120:0/1167312917 shutdown_connections 2024-09-19T09:18:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.716+0000 7fa91d21e640 1 -- 172.21.15.120:0/1167312917 wait complete. 2024-09-19T09:18:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.717+0000 7fa91d21e640 1 Processor -- start 2024-09-19T09:18:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.717+0000 7fa91d21e640 1 -- start start 2024-09-19T09:18:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.717+0000 7fa91d21e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa918100a40 0x7fa91806a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.717+0000 7fa91d21e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa918101c30 0x7fa91819cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.717+0000 7fa91d21e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9181025f0 0x7fa91819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.717+0000 7fa91d21e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa91810c720 con 0x7fa918100a40 2024-09-19T09:18:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.717+0000 7fa91d21e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa91810c5a0 con 0x7fa918101c30 2024-09-19T09:18:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.717+0000 7fa91d21e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa91810c8a0 con 0x7fa9181025f0 2024-09-19T09:18:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.717+0000 7fa916d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa918100a40 0x7fa91806a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.717+0000 7fa916d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa918100a40 0x7fa91806a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33182/0 (socket says 172.21.15.120:33182) 2024-09-19T09:18:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.717+0000 7fa916d9d640 1 -- 172.21.15.120:0/469390857 learned_addr learned my addr 172.21.15.120:0/469390857 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.717+0000 7fa91759e640 1 --2- 172.21.15.120:0/469390857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9181025f0 0x7fa91819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.718+0000 7fa91659c640 1 --2- 172.21.15.120:0/469390857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa918101c30 0x7fa91819cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.718+0000 7fa916d9d640 1 -- 172.21.15.120:0/469390857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9181025f0 msgr2=0x7fa91819f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.718+0000 7fa916d9d640 1 --2- 172.21.15.120:0/469390857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9181025f0 0x7fa91819f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.718+0000 7fa916d9d640 1 -- 172.21.15.120:0/469390857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa918101c30 msgr2=0x7fa91819cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.718+0000 7fa916d9d640 1 --2- 172.21.15.120:0/469390857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa918101c30 0x7fa91819cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.718+0000 7fa916d9d640 1 -- 172.21.15.120:0/469390857 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91819fa10 con 0x7fa918100a40 2024-09-19T09:18:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.718+0000 7fa916d9d640 1 --2- 172.21.15.120:0/469390857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa918100a40 0x7fa91806a410 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fa908031610 tx=0x7fa9080043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.718+0000 7fa8fffff640 1 -- 172.21.15.120:0/469390857 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90803f030 con 0x7fa918100a40 2024-09-19T09:18:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.718+0000 7fa91d21e640 1 -- 172.21.15.120:0/469390857 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa91819fc40 con 0x7fa918100a40 2024-09-19T09:18:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.718+0000 7fa8fffff640 1 -- 172.21.15.120:0/469390857 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa908003ce0 con 0x7fa918100a40 2024-09-19T09:18:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.718+0000 7fa8fffff640 1 -- 172.21.15.120:0/469390857 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908035070 con 0x7fa918100a40 2024-09-19T09:18:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.718+0000 7fa91d21e640 1 -- 172.21.15.120:0/469390857 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9181a0180 con 0x7fa918100a40 2024-09-19T09:18:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.720+0000 7fa8fffff640 1 -- 172.21.15.120:0/469390857 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa908042030 con 0x7fa918100a40 2024-09-19T09:18:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.721+0000 7fa92008b740 1 Processor -- start 2024-09-19T09:18:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.721+0000 7fa92008b740 1 -- start start 2024-09-19T09:18:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.721+0000 7fa92008b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f858669a0 0x562f85866da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.721+0000 7fa92008b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f858672e0 0x562f8586bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.721+0000 7fa92008b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f8586c370 0x562f8586e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.721+0000 7fa92008b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562f857ca2f0 con 0x562f8586c370 2024-09-19T09:18:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.721+0000 7fa92008b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562f85772270 con 0x562f858669a0 2024-09-19T09:18:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.721+0000 7fa92008b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562f857c62b0 con 0x562f858672e0 2024-09-19T09:18:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.721+0000 7fa91659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f8586c370 0x562f8586e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.721+0000 7fa91659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f8586c370 0x562f8586e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33184/0 (socket says 172.21.15.120:33184) 2024-09-19T09:18:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.721+0000 7fa91659c640 1 -- 172.21.15.120:0/2126626750 learned_addr learned my addr 172.21.15.120:0/2126626750 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.721+0000 7fa91759e640 1 --2- 172.21.15.120:0/2126626750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f858669a0 0x562f85866da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.721+0000 7fa916d9d640 1 --2- 172.21.15.120:0/2126626750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f858672e0 0x562f8586bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.721+0000 7fa8fffff640 1 --2- 172.21.15.120:0/469390857 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9000a32f0 0x7fa9000a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.721+0000 7fa91659c640 1 -- 172.21.15.120:0/2126626750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f858672e0 msgr2=0x562f8586bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.721+0000 7fa91659c640 1 --2- 172.21.15.120:0/2126626750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f858672e0 0x562f8586bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.722+0000 7fa91659c640 1 -- 172.21.15.120:0/2126626750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f858669a0 msgr2=0x562f85866da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.722+0000 7fa91659c640 1 --2- 172.21.15.120:0/2126626750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f858669a0 0x562f85866da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.722+0000 7fa91659c640 1 -- 172.21.15.120:0/2126626750 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f85771340 con 0x562f8586c370 2024-09-19T09:18:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.722+0000 7fa8fffff640 1 -- 172.21.15.120:0/469390857 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa9080effe0 con 0x7fa918100a40 2024-09-19T09:18:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.722+0000 7fa91759e640 1 --2- 172.21.15.120:0/469390857 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9000a32f0 0x7fa9000a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.722+0000 7fa91659c640 1 --2- 172.21.15.120:0/2126626750 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f8586c370 0x562f8586e830 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00cc40 tx=0x7fa90c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab2b3cbe9ab2312 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.722+0000 7fa8fd7fa640 1 -- 172.21.15.120:0/2126626750 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c014070 con 0x562f8586c370 2024-09-19T09:18:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.722+0000 7fa91759e640 1 --2- 172.21.15.120:0/469390857 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9000a32f0 0x7fa9000a57b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa910009fd0 tx=0x7fa910004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.722+0000 7fa8fd7fa640 1 -- 172.21.15.120:0/2126626750 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa90c0047a0 con 0x562f8586c370 2024-09-19T09:18:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.722+0000 7fa8fd7fa640 1 -- 172.21.15.120:0/2126626750 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c010760 con 0x562f8586c370 2024-09-19T09:18:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.723+0000 7fa92008b740 1 -- 172.21.15.120:0/2126626750 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f8586c370 msgr2=0x562f8586e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.723+0000 7fa92008b740 1 --2- 172.21.15.120:0/2126626750 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f8586c370 0x562f8586e830 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00cc40 tx=0x7fa90c00e720 comp rx=0 tx=0).stop 2024-09-19T09:18:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.723+0000 7fa92008b740 1 -- 172.21.15.120:0/2126626750 shutdown_connections 2024-09-19T09:18:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.723+0000 7fa92008b740 1 --2- 172.21.15.120:0/2126626750 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f8586c370 0x562f8586e830 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.723+0000 7fa92008b740 1 --2- 172.21.15.120:0/2126626750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f858672e0 0x562f8586bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.723+0000 7fa92008b740 1 --2- 172.21.15.120:0/2126626750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f858669a0 0x562f85866da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.723+0000 7fa92008b740 1 -- 172.21.15.120:0/2126626750 >> 172.21.15.120:0/2126626750 conn(0x562f85862010 msgr2=0x562f85862410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.723+0000 7fa92008b740 1 -- 172.21.15.120:0/2126626750 shutdown_connections 2024-09-19T09:18:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.723+0000 7fa92008b740 1 -- 172.21.15.120:0/2126626750 wait complete. 2024-09-19T09:18:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.723+0000 7fa92008b740 1 -- start start 2024-09-19T09:18:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.723+0000 7fa92008b740 1 Processor -- start 2024-09-19T09:18:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.723+0000 7fa92008b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f858669a0 0x562f85870bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.723+0000 7fa92008b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f858672e0 0x562f85871130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.723+0000 7fa92008b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f8586c370 0x562f85878170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.723+0000 7fa92008b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562f8586f340 con 0x562f8586c370 2024-09-19T09:18:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.723+0000 7fa92008b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562f85753290 con 0x562f858669a0 2024-09-19T09:18:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.723+0000 7fa92008b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562f8586f4c0 con 0x562f858672e0 2024-09-19T09:18:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.724+0000 7fa92008b740 10 client.0 _ll_register_callbacks cb 0x562f85784140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.724+0000 7fa91759e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f8586c370 0x562f85878170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.724+0000 7fa91759e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f8586c370 0x562f85878170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33192/0 (socket says 172.21.15.120:33192) 2024-09-19T09:18:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.724+0000 7fa91759e640 1 -- 172.21.15.120:0/2862787105 learned_addr learned my addr 172.21.15.120:0/2862787105 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.724+0000 7fa91659c640 1 --2- 172.21.15.120:0/2862787105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f858672e0 0x562f85871130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.724+0000 7fa916d9d640 1 --2- 172.21.15.120:0/2862787105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f858669a0 0x562f85870bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.724+0000 7fa91759e640 1 -- 172.21.15.120:0/2862787105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f858672e0 msgr2=0x562f85871130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.724+0000 7fa91759e640 1 --2- 172.21.15.120:0/2862787105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f858672e0 0x562f85871130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.724+0000 7fa91759e640 1 -- 172.21.15.120:0/2862787105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f858669a0 msgr2=0x562f85870bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.724+0000 7fa91759e640 1 --2- 172.21.15.120:0/2862787105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f858669a0 0x562f85870bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.724+0000 7fa91759e640 1 -- 172.21.15.120:0/2862787105 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f854dd210 con 0x562f8586c370 2024-09-19T09:18:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.724+0000 7fa91759e640 1 --2- 172.21.15.120:0/2862787105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f8586c370 0x562f85878170 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fa91000af20 tx=0x7fa910004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8755be35ff20f3f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.724+0000 7fa8faff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:18:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.724+0000 7fa8faff5640 1 -- 172.21.15.120:0/2862787105 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910029060 con 0x562f8586c370 2024-09-19T09:18:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.724+0000 7fa8faff5640 1 -- 172.21.15.120:0/2862787105 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa910024600 con 0x562f8586c370 2024-09-19T09:18:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.724+0000 7fa92008b740 1 -- 172.21.15.120:0/2862787105 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa90c0047a0 con 0x562f8586c370 2024-09-19T09:18:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.724+0000 7fa8faff5640 1 -- 172.21.15.120:0/2862787105 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa910019e10 con 0x562f8586c370 2024-09-19T09:18:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.724+0000 7fa92008b740 10 client.20712 fetch_fsmap learned FSMap version 36 2024-09-19T09:18:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.724+0000 7fa92008b740 1 -- 172.21.15.120:0/2862787105 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562f855a9620 con 0x562f8586c370 2024-09-19T09:18:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.725+0000 7fa8faff5640 1 -- 172.21.15.120:0/2862787105 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa90c0047a0 con 0x562f8586c370 2024-09-19T09:18:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.725+0000 7fa8faff5640 1 -- 172.21.15.120:0/2862787105 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fa91002e070 con 0x562f8586c370 2024-09-19T09:18:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.725+0000 7fa92008b740 10 client.20712 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:18:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.725+0000 7fa92008b740 10 client.20712 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:18:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.725+0000 7fa92008b740 20 client.20712 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.725+0000 7fa92008b740 1 --2- 172.21.15.120:0/2862787105 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562f8587c930 0x562f8587ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.725+0000 7fa92008b740 4 client.20712 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:18:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.725+0000 7fa92008b740 1 -- 172.21.15.120:0/2862787105 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f857d7a20 con 0x562f8587c930 2024-09-19T09:18:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.725+0000 7fa916d9d640 1 --2- 172.21.15.120:0/2862787105 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562f8587c930 0x562f8587ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.726+0000 7fa8faff5640 10 client.20712 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.726+0000 7fa916d9d640 1 --2- 172.21.15.120:0/2862787105 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562f8587c930 0x562f8587ed80 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3014d0d20ce08752 server_cookie=1165f6db34cbbfcd in_seq=0 out_seq=0 2024-09-19T09:18:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.726+0000 7fa8faff5640 1 -- 172.21.15.120:0/2862787105 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562f857d7a20 con 0x562f8587c930 2024-09-19T09:18:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.726+0000 7fa8faff5640 10 client.20712 handle_command_reply: tid=0 2024-09-19T09:18:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.726+0000 7fa92008b740 1 client.20712 shutdown 2024-09-19T09:18:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.726+0000 7fa92008b740 1 -- 172.21.15.120:0/2862787105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f8586c370 msgr2=0x562f85878170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.726+0000 7fa92008b740 1 --2- 172.21.15.120:0/2862787105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f8586c370 0x562f85878170 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fa91000af20 tx=0x7fa910004b50 comp rx=0 tx=0).stop 2024-09-19T09:18:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.726+0000 7fa92008b740 1 -- 172.21.15.120:0/2862787105 shutdown_connections 2024-09-19T09:18:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.726+0000 7fa92008b740 1 --2- 172.21.15.120:0/2862787105 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562f8587c930 0x562f8587ed80 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.726+0000 7fa92008b740 1 --2- 172.21.15.120:0/2862787105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f8586c370 0x562f85878170 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.726+0000 7fa92008b740 1 --2- 172.21.15.120:0/2862787105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f858672e0 0x562f85871130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.726+0000 7fa92008b740 1 --2- 172.21.15.120:0/2862787105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f858669a0 0x562f85870bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.726+0000 7fa92008b740 1 -- 172.21.15.120:0/2862787105 >> 172.21.15.120:0/2862787105 conn(0x562f85862010 msgr2=0x562f85863820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.726+0000 7fa8faff5640 0 client.20712 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.726+0000 7fa92008b740 1 -- 172.21.15.120:0/2862787105 shutdown_connections 2024-09-19T09:18:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.726+0000 7fa92008b740 1 -- 172.21.15.120:0/2862787105 wait complete. 2024-09-19T09:18:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.727+0000 7fa92008b740 20 client.20712 trim_cache size 0 max 16384 2024-09-19T09:18:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.741+0000 7fa92008b740 1 Processor -- start 2024-09-19T09:18:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.741+0000 7fa92008b740 1 -- start start 2024-09-19T09:18:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.741+0000 7fa92008b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f858669a0 0x562f85870bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.741+0000 7fa92008b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f858672e0 0x562f85878130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.741+0000 7fa92008b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f8586c370 0x562f85871130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.741+0000 7fa92008b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9181a9000 con 0x562f858669a0 2024-09-19T09:18:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.741+0000 7fa92008b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa9181aabd0 con 0x562f8586c370 2024-09-19T09:18:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.741+0000 7fa92008b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9181a96d0 con 0x562f858672e0 2024-09-19T09:18:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.741+0000 7fa916d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f858669a0 0x562f85870bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.741+0000 7fa916d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f858669a0 0x562f85870bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33200/0 (socket says 172.21.15.120:33200) 2024-09-19T09:18:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.741+0000 7fa916d9d640 1 -- 172.21.15.120:0/581617830 learned_addr learned my addr 172.21.15.120:0/581617830 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.741+0000 7fa91659c640 1 --2- 172.21.15.120:0/581617830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f858672e0 0x562f85878130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.741+0000 7fa91759e640 1 --2- 172.21.15.120:0/581617830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f8586c370 0x562f85871130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.741+0000 7fa916d9d640 1 -- 172.21.15.120:0/581617830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f858672e0 msgr2=0x562f85878130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.741+0000 7fa916d9d640 1 --2- 172.21.15.120:0/581617830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f858672e0 0x562f85878130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.741+0000 7fa916d9d640 1 -- 172.21.15.120:0/581617830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f8586c370 msgr2=0x562f85871130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.741+0000 7fa916d9d640 1 --2- 172.21.15.120:0/581617830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f8586c370 0x562f85871130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.741+0000 7fa916d9d640 1 -- 172.21.15.120:0/581617830 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f85861e30 con 0x562f858669a0 2024-09-19T09:18:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.741+0000 7fa91759e640 1 --2- 172.21.15.120:0/581617830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f8586c370 0x562f85871130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:18:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.742+0000 7fa916d9d640 1 --2- 172.21.15.120:0/581617830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f858669a0 0x562f85870bf0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fa9080bc2e0 tx=0x7fa9080ed990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8ff3a8154ec9e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.742+0000 7fa8fcff9640 1 -- 172.21.15.120:0/581617830 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908049070 con 0x562f858669a0 2024-09-19T09:18:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.742+0000 7fa8fcff9640 1 -- 172.21.15.120:0/581617830 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9080c5250 con 0x562f858669a0 2024-09-19T09:18:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.742+0000 7fa8fcff9640 1 -- 172.21.15.120:0/581617830 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9080f20b0 con 0x562f858669a0 2024-09-19T09:18:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.742+0000 7fa92008b740 1 -- 172.21.15.120:0/581617830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f858669a0 msgr2=0x562f85870bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.742+0000 7fa92008b740 1 --2- 172.21.15.120:0/581617830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f858669a0 0x562f85870bf0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fa9080bc2e0 tx=0x7fa9080ed990 comp rx=0 tx=0).stop 2024-09-19T09:18:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.742+0000 7fa92008b740 1 -- 172.21.15.120:0/581617830 shutdown_connections 2024-09-19T09:18:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.742+0000 7fa92008b740 1 --2- 172.21.15.120:0/581617830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f8586c370 0x562f85871130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.742+0000 7fa92008b740 1 --2- 172.21.15.120:0/581617830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f858672e0 0x562f85878130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.742+0000 7fa92008b740 1 --2- 172.21.15.120:0/581617830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f858669a0 0x562f85870bf0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.743+0000 7fa92008b740 1 -- 172.21.15.120:0/581617830 >> 172.21.15.120:0/581617830 conn(0x562f85862010 msgr2=0x562f85868480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.743+0000 7fa92008b740 1 -- 172.21.15.120:0/581617830 shutdown_connections 2024-09-19T09:18:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.743+0000 7fa92008b740 1 -- 172.21.15.120:0/581617830 wait complete. 2024-09-19T09:18:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.743+0000 7fa92008b740 1 -- start start 2024-09-19T09:18:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.743+0000 7fa92008b740 1 Processor -- start 2024-09-19T09:18:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.743+0000 7fa92008b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f858669a0 0x562f85874e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.743+0000 7fa92008b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f858672e0 0x562f858753c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.743+0000 7fa92008b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f8586c370 0x562f858e5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.743+0000 7fa92008b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9181aa500 con 0x562f858672e0 2024-09-19T09:18:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.743+0000 7fa92008b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa9181a9da0 con 0x562f8586c370 2024-09-19T09:18:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.743+0000 7fa92008b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9181a6fc0 con 0x562f858669a0 2024-09-19T09:18:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.743+0000 7fa92008b740 10 client.0 _ll_register_callbacks cb 0x7fa91000ae70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.743+0000 7fa91659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f858672e0 0x562f858753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.743+0000 7fa91659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f858672e0 0x562f858753c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33212/0 (socket says 172.21.15.120:33212) 2024-09-19T09:18:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.743+0000 7fa91659c640 1 -- 172.21.15.120:0/2310522198 learned_addr learned my addr 172.21.15.120:0/2310522198 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.743+0000 7fa916d9d640 1 --2- 172.21.15.120:0/2310522198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f858669a0 0x562f85874e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.743+0000 7fa91759e640 1 --2- 172.21.15.120:0/2310522198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f8586c370 0x562f858e5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.743+0000 7fa91659c640 1 -- 172.21.15.120:0/2310522198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f858669a0 msgr2=0x562f85874e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.744+0000 7fa91659c640 1 --2- 172.21.15.120:0/2310522198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f858669a0 0x562f85874e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.744+0000 7fa91659c640 1 -- 172.21.15.120:0/2310522198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f8586c370 msgr2=0x562f858e5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.744+0000 7fa91659c640 1 --2- 172.21.15.120:0/2310522198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f8586c370 0x562f858e5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.744+0000 7fa91659c640 1 -- 172.21.15.120:0/2310522198 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f85875a60 con 0x562f858672e0 2024-09-19T09:18:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.744+0000 7fa91759e640 1 --2- 172.21.15.120:0/2310522198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f8586c370 0x562f858e5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:18:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.744+0000 7fa91659c640 1 --2- 172.21.15.120:0/2310522198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f858672e0 0x562f858753c0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00f010 tx=0x7fa90c0048b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b43b553575c3c807 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.744+0000 7fa8faff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:18:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.744+0000 7fa8faff5640 1 -- 172.21.15.120:0/2310522198 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c027020 con 0x562f858672e0 2024-09-19T09:18:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.744+0000 7fa92008b740 1 -- 172.21.15.120:0/2310522198 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9080c5250 con 0x562f858672e0 2024-09-19T09:18:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.744+0000 7fa8faff5640 1 -- 172.21.15.120:0/2310522198 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa90c004db0 con 0x562f858672e0 2024-09-19T09:18:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.744+0000 7fa8faff5640 1 -- 172.21.15.120:0/2310522198 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa90c014050 con 0x562f858672e0 2024-09-19T09:18:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.744+0000 7fa92008b740 10 client.20718 fetch_fsmap learned FSMap version 36 2024-09-19T09:18:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.744+0000 7fa92008b740 1 -- 172.21.15.120:0/2310522198 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562f858e6cd0 con 0x562f858672e0 2024-09-19T09:18:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.744+0000 7fa8faff5640 1 -- 172.21.15.120:0/2310522198 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9080c5250 con 0x562f858672e0 2024-09-19T09:18:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.744+0000 7fa8faff5640 1 -- 172.21.15.120:0/2310522198 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fa90c010430 con 0x562f858672e0 2024-09-19T09:18:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.744+0000 7fa92008b740 10 client.20718 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:18:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.744+0000 7fa92008b740 10 client.20718 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:18:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.744+0000 7fa92008b740 20 client.20718 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.744+0000 7fa92008b740 1 --2- 172.21.15.120:0/2310522198 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562f858e7390 0x562f858e97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.744+0000 7fa92008b740 4 client.20718 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:18:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.744+0000 7fa92008b740 1 -- 172.21.15.120:0/2310522198 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562f857d1d70 con 0x562f858e7390 2024-09-19T09:18:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.745+0000 7fa916d9d640 1 --2- 172.21.15.120:0/2310522198 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562f858e7390 0x562f858e97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.745+0000 7fa8faff5640 10 client.20718 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.745+0000 7fa916d9d640 1 --2- 172.21.15.120:0/2310522198 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562f858e7390 0x562f858e97b0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7b265f043be0094 server_cookie=5557aa7707938989 in_seq=0 out_seq=0 2024-09-19T09:18:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.865+0000 7fa8faff5640 1 -- 172.21.15.120:0/2310522198 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562f857d1d70 con 0x562f858e7390 2024-09-19T09:18:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.865+0000 7fa8faff5640 10 client.20718 handle_command_reply: tid=0 2024-09-19T09:18:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.865+0000 7fa92008b740 1 client.20718 shutdown 2024-09-19T09:18:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.865+0000 7fa92008b740 1 -- 172.21.15.120:0/2310522198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f858672e0 msgr2=0x562f858753c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.865+0000 7fa92008b740 1 --2- 172.21.15.120:0/2310522198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f858672e0 0x562f858753c0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00f010 tx=0x7fa90c0048b0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.865+0000 7fa92008b740 1 -- 172.21.15.120:0/2310522198 shutdown_connections 2024-09-19T09:18:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.865+0000 7fa92008b740 1 --2- 172.21.15.120:0/2310522198 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562f858e7390 0x562f858e97b0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.865+0000 7fa92008b740 1 --2- 172.21.15.120:0/2310522198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f8586c370 0x562f858e5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.865+0000 7fa92008b740 1 --2- 172.21.15.120:0/2310522198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f858672e0 0x562f858753c0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.865+0000 7fa8faff5640 0 client.20718 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.865+0000 7fa92008b740 1 --2- 172.21.15.120:0/2310522198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562f858669a0 0x562f85874e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.865+0000 7fa92008b740 1 -- 172.21.15.120:0/2310522198 >> 172.21.15.120:0/2310522198 conn(0x562f85862010 msgr2=0x562f85868480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.865+0000 7fa92008b740 1 -- 172.21.15.120:0/2310522198 shutdown_connections 2024-09-19T09:18:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.865+0000 7fa92008b740 1 -- 172.21.15.120:0/2310522198 wait complete. 2024-09-19T09:18:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.865+0000 7fa92008b740 20 client.20718 trim_cache size 0 max 16384 2024-09-19T09:18:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:18:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:18:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:18:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:18:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.865+0000 7fa91d21e640 1 -- 172.21.15.120:0/469390857 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9000a32f0 msgr2=0x7fa9000a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.865+0000 7fa91d21e640 1 --2- 172.21.15.120:0/469390857 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9000a32f0 0x7fa9000a57b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa910009fd0 tx=0x7fa910004a40 comp rx=0 tx=0).stop 2024-09-19T09:18:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.865+0000 7fa91d21e640 1 -- 172.21.15.120:0/469390857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa918100a40 msgr2=0x7fa91806a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.865+0000 7fa91d21e640 1 --2- 172.21.15.120:0/469390857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa918100a40 0x7fa91806a410 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fa908031610 tx=0x7fa9080043b0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.866+0000 7fa91d21e640 1 -- 172.21.15.120:0/469390857 shutdown_connections 2024-09-19T09:18:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.866+0000 7fa91d21e640 1 --2- 172.21.15.120:0/469390857 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9000a32f0 0x7fa9000a57b0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.866+0000 7fa91d21e640 1 --2- 172.21.15.120:0/469390857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9181025f0 0x7fa91819f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.866+0000 7fa91d21e640 1 --2- 172.21.15.120:0/469390857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa918101c30 0x7fa91819cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.866+0000 7fa91d21e640 1 --2- 172.21.15.120:0/469390857 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa918100a40 0x7fa91806a410 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.866+0000 7fa91d21e640 1 -- 172.21.15.120:0/469390857 >> 172.21.15.120:0/469390857 conn(0x7fa9180fc1e0 msgr2=0x7fa918104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.866+0000 7fa91d21e640 1 -- 172.21.15.120:0/469390857 shutdown_connections 2024-09-19T09:18:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:19.866+0000 7fa91d21e640 1 -- 172.21.15.120:0/469390857 wait complete. 2024-09-19T09:18:20.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.174+0000 7fc29bf20640 1 -- 172.21.15.120:0/2277666403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc294102d90 msgr2=0x7fc294103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:20.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.174+0000 7fc29bf20640 1 --2- 172.21.15.120:0/2277666403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc294102d90 0x7fc294103170 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009cd0 tx=0x7fc28c02a8e0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.174+0000 7fc29bf20640 1 -- 172.21.15.120:0/2277666403 shutdown_connections 2024-09-19T09:18:20.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.174+0000 7fc29bf20640 1 --2- 172.21.15.120:0/2277666403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc294108290 0x7fc29410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.174+0000 7fc29bf20640 1 --2- 172.21.15.120:0/2277666403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc294103740 0x7fc294107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.174+0000 7fc29bf20640 1 --2- 172.21.15.120:0/2277666403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc294102d90 0x7fc294103170 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.174+0000 7fc29bf20640 1 -- 172.21.15.120:0/2277666403 >> 172.21.15.120:0/2277666403 conn(0x7fc2940fc7d0 msgr2=0x7fc2940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:20.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.174+0000 7fc29bf20640 1 -- 172.21.15.120:0/2277666403 shutdown_connections 2024-09-19T09:18:20.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.174+0000 7fc29bf20640 1 -- 172.21.15.120:0/2277666403 wait complete. 2024-09-19T09:18:20.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.175+0000 7fc29bf20640 1 Processor -- start 2024-09-19T09:18:20.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.175+0000 7fc29bf20640 1 -- start start 2024-09-19T09:18:20.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.175+0000 7fc29bf20640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc294102d90 0x7fc2941009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:20.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.175+0000 7fc29bf20640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc294103740 0x7fc294100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:20.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.175+0000 7fc29bf20640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc294108290 0x7fc294101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:20.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.175+0000 7fc29bf20640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc294112090 con 0x7fc294103740 2024-09-19T09:18:20.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.175+0000 7fc29bf20640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc294111f10 con 0x7fc294102d90 2024-09-19T09:18:20.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.175+0000 7fc29bf20640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc294112210 con 0x7fc294108290 2024-09-19T09:18:20.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.175+0000 7fc2994bb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc294103740 0x7fc294100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:20.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.175+0000 7fc2994bb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc294103740 0x7fc294100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33230/0 (socket says 172.21.15.120:33230) 2024-09-19T09:18:20.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.175+0000 7fc2994bb640 1 -- 172.21.15.120:0/2200744324 learned_addr learned my addr 172.21.15.120:0/2200744324 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:20.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.175+0000 7fc299cbc640 1 --2- 172.21.15.120:0/2200744324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc294102d90 0x7fc2941009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:20.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.175+0000 7fc29a4bd640 1 --2- 172.21.15.120:0/2200744324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc294108290 0x7fc294101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:20.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.176+0000 7fc2994bb640 1 -- 172.21.15.120:0/2200744324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc294108290 msgr2=0x7fc294101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:20.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.176+0000 7fc2994bb640 1 --2- 172.21.15.120:0/2200744324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc294108290 0x7fc294101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.176+0000 7fc2994bb640 1 -- 172.21.15.120:0/2200744324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc294102d90 msgr2=0x7fc2941009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:20.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.176+0000 7fc2994bb640 1 --2- 172.21.15.120:0/2200744324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc294102d90 0x7fc2941009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.176+0000 7fc2994bb640 1 -- 172.21.15.120:0/2200744324 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2941a6ab0 con 0x7fc294103740 2024-09-19T09:18:20.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.176+0000 7fc2994bb640 1 --2- 172.21.15.120:0/2200744324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc294103740 0x7fc294100f30 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fc28800e9f0 tx=0x7fc28800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:20.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.176+0000 7fc292ffd640 1 -- 172.21.15.120:0/2200744324 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28800ccc0 con 0x7fc294103740 2024-09-19T09:18:20.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.176+0000 7fc292ffd640 1 -- 172.21.15.120:0/2200744324 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc288005500 con 0x7fc294103740 2024-09-19T09:18:20.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.176+0000 7fc292ffd640 1 -- 172.21.15.120:0/2200744324 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc288010690 con 0x7fc294103740 2024-09-19T09:18:20.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.176+0000 7fc29bf20640 1 -- 172.21.15.120:0/2200744324 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2941a6da0 con 0x7fc294103740 2024-09-19T09:18:20.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.176+0000 7fc29bf20640 1 -- 172.21.15.120:0/2200744324 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2941a7200 con 0x7fc294103740 2024-09-19T09:18:20.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.178+0000 7fc292ffd640 1 -- 172.21.15.120:0/2200744324 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fc2880046c0 con 0x7fc294103740 2024-09-19T09:18:20.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.179+0000 7fc29e2f9740 1 Processor -- start 2024-09-19T09:18:20.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.179+0000 7fc29e2f9740 1 -- start start 2024-09-19T09:18:20.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.179+0000 7fc29e2f9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e57abbeb60 0x55e57abbef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:20.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.179+0000 7fc29e2f9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e57abbf480 0x55e57abc9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:20.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.179+0000 7fc29e2f9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e57abc9d40 0x55e57abcc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:20.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.179+0000 7fc29e2f9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e57ab24280 con 0x55e57abc9d40 2024-09-19T09:18:20.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.179+0000 7fc29e2f9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e57aacbb10 con 0x55e57abbf480 2024-09-19T09:18:20.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.179+0000 7fc29e2f9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e57ab20240 con 0x55e57abbeb60 2024-09-19T09:18:20.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.179+0000 7fc292ffd640 1 --2- 172.21.15.120:0/2200744324 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc27c0a32c0 0x7fc27c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:20.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.179+0000 7fc299cbc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e57abbf480 0x55e57abc9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:20.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.179+0000 7fc29a4bd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e57abbeb60 0x55e57abbef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:20.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.179+0000 7fc299cbc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e57abbf480 0x55e57abc9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:49484/0 (socket says 172.21.15.120:49484) 2024-09-19T09:18:20.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.179+0000 7fc2994bb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e57abc9d40 0x55e57abcc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:20.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.179+0000 7fc299cbc640 1 -- 172.21.15.120:0/3737194405 learned_addr learned my addr 172.21.15.120:0/3737194405 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:20.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.179+0000 7fc29a4bd640 1 --2- 172.21.15.120:0/2200744324 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc27c0a32c0 0x7fc27c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:20.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.180+0000 7fc292ffd640 1 -- 172.21.15.120:0/2200744324 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc2880cfc70 con 0x7fc294103740 2024-09-19T09:18:20.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.180+0000 7fc299cbc640 1 -- 172.21.15.120:0/3737194405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e57abbeb60 msgr2=0x55e57abbef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:20.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.180+0000 7fc299cbc640 1 --2- 172.21.15.120:0/3737194405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e57abbeb60 0x55e57abbef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.180+0000 7fc299cbc640 1 -- 172.21.15.120:0/3737194405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e57abc9d40 msgr2=0x55e57abcc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:20.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.180+0000 7fc299cbc640 1 --2- 172.21.15.120:0/3737194405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e57abc9d40 0x55e57abcc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.180+0000 7fc299cbc640 1 -- 172.21.15.120:0/3737194405 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e57aad81f0 con 0x55e57abbf480 2024-09-19T09:18:20.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.180+0000 7fc2994bb640 1 --2- 172.21.15.120:0/3737194405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e57abc9d40 0x55e57abcc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:18:20.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.180+0000 7fc29a4bd640 1 --2- 172.21.15.120:0/2200744324 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc27c0a32c0 0x7fc27c0a5780 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc284005e00 tx=0x7fc284005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:20.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.180+0000 7fc299cbc640 1 --2- 172.21.15.120:0/3737194405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e57abbf480 0x55e57abc9800 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009cd0 tx=0x7fc28c009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa6f9f3682c0bc46 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:20.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.180+0000 7fc27bfff640 1 -- 172.21.15.120:0/3737194405 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c048070 con 0x55e57abbf480 2024-09-19T09:18:20.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.180+0000 7fc27bfff640 1 -- 172.21.15.120:0/3737194405 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc28c036070 con 0x55e57abbf480 2024-09-19T09:18:20.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.180+0000 7fc27bfff640 1 -- 172.21.15.120:0/3737194405 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c0044a0 con 0x55e57abbf480 2024-09-19T09:18:20.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.181+0000 7fc29e2f9740 1 -- 172.21.15.120:0/3737194405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e57abbf480 msgr2=0x55e57abc9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:20.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.181+0000 7fc29e2f9740 1 --2- 172.21.15.120:0/3737194405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e57abbf480 0x55e57abc9800 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009cd0 tx=0x7fc28c009a20 comp rx=0 tx=0).stop 2024-09-19T09:18:20.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.181+0000 7fc29e2f9740 1 -- 172.21.15.120:0/3737194405 shutdown_connections 2024-09-19T09:18:20.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.181+0000 7fc29e2f9740 1 --2- 172.21.15.120:0/3737194405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e57abc9d40 0x55e57abcc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.181+0000 7fc29e2f9740 1 --2- 172.21.15.120:0/3737194405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e57abbf480 0x55e57abc9800 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.181+0000 7fc29e2f9740 1 --2- 172.21.15.120:0/3737194405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e57abbeb60 0x55e57abbef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.181+0000 7fc29e2f9740 1 -- 172.21.15.120:0/3737194405 >> 172.21.15.120:0/3737194405 conn(0x55e57abbb0f0 msgr2=0x55e57abbb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:20.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.181+0000 7fc29e2f9740 1 -- 172.21.15.120:0/3737194405 shutdown_connections 2024-09-19T09:18:20.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.181+0000 7fc29e2f9740 1 -- 172.21.15.120:0/3737194405 wait complete. 2024-09-19T09:18:20.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.181+0000 7fc29e2f9740 1 -- start start 2024-09-19T09:18:20.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.181+0000 7fc29e2f9740 1 Processor -- start 2024-09-19T09:18:20.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.182+0000 7fc29e2f9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e57abbeb60 0x55e57abc6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:20.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.182+0000 7fc29e2f9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e57abbf480 0x55e57abc70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:20.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.182+0000 7fc29e2f9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e57abc9d40 0x55e57abc7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:20.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.182+0000 7fc29e2f9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e57abcc910 con 0x55e57abbeb60 2024-09-19T09:18:20.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.182+0000 7fc29e2f9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e57aadef10 con 0x55e57abbf480 2024-09-19T09:18:20.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.182+0000 7fc29e2f9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e57abcca90 con 0x55e57abc9d40 2024-09-19T09:18:20.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.182+0000 7fc29e2f9740 10 client.0 _ll_register_callbacks cb 0x55e57aaa7a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:20.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.182+0000 7fc299cbc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e57abbeb60 0x55e57abc6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:20.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.182+0000 7fc2994bb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e57abbf480 0x55e57abc70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:20.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.182+0000 7fc299cbc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e57abbeb60 0x55e57abc6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33246/0 (socket says 172.21.15.120:33246) 2024-09-19T09:18:20.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.182+0000 7fc299cbc640 1 -- 172.21.15.120:0/2494131892 learned_addr learned my addr 172.21.15.120:0/2494131892 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:20.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.182+0000 7fc29a4bd640 1 --2- 172.21.15.120:0/2494131892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e57abc9d40 0x55e57abc7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:20.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.182+0000 7fc2994bb640 1 -- 172.21.15.120:0/2494131892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e57abc9d40 msgr2=0x55e57abc7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:20.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.182+0000 7fc2994bb640 1 --2- 172.21.15.120:0/2494131892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e57abc9d40 0x55e57abc7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.182+0000 7fc2994bb640 1 -- 172.21.15.120:0/2494131892 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e57abbeb60 msgr2=0x55e57abc6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:20.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.182+0000 7fc2994bb640 1 --2- 172.21.15.120:0/2494131892 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e57abbeb60 0x55e57abc6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.182+0000 7fc2994bb640 1 -- 172.21.15.120:0/2494131892 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e57a8956a0 con 0x55e57abbf480 2024-09-19T09:18:20.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.183+0000 7fc2994bb640 1 --2- 172.21.15.120:0/2494131892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e57abbf480 0x55e57abc70c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc288098c60 tx=0x7fc288098660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=430803b07a38801 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:20.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.183+0000 7fc2797fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:18:20.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.183+0000 7fc2797fa640 1 -- 172.21.15.120:0/2494131892 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2880a17d0 con 0x55e57abbf480 2024-09-19T09:18:20.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.183+0000 7fc2797fa640 1 -- 172.21.15.120:0/2494131892 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc28809c6f0 con 0x55e57abbf480 2024-09-19T09:18:20.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.183+0000 7fc29e2f9740 1 -- 172.21.15.120:0/2494131892 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc28c036070 con 0x55e57abbf480 2024-09-19T09:18:20.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.183+0000 7fc2797fa640 1 -- 172.21.15.120:0/2494131892 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc28809c990 con 0x55e57abbf480 2024-09-19T09:18:20.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.183+0000 7fc29e2f9740 10 client.29633 fetch_fsmap learned FSMap version 36 2024-09-19T09:18:20.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.183+0000 7fc29e2f9740 1 -- 172.21.15.120:0/2494131892 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e57abd8540 con 0x55e57abbf480 2024-09-19T09:18:20.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.183+0000 7fc2797fa640 1 -- 172.21.15.120:0/2494131892 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc28c036070 con 0x55e57abbf480 2024-09-19T09:18:20.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.183+0000 7fc2797fa640 1 -- 172.21.15.120:0/2494131892 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fc2880a4a70 con 0x55e57abbf480 2024-09-19T09:18:20.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.183+0000 7fc29e2f9740 10 client.29633 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:18:20.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.183+0000 7fc29e2f9740 10 client.29633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:18:20.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.183+0000 7fc29e2f9740 20 client.29633 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:20.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.183+0000 7fc29e2f9740 1 --2- 172.21.15.120:0/2494131892 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e57abd9950 0x55e57abdbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:20.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.183+0000 7fc29e2f9740 4 client.29633 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:18:20.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.183+0000 7fc29e2f9740 1 -- 172.21.15.120:0/2494131892 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e57ab317f0 con 0x55e57abd9950 2024-09-19T09:18:20.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.184+0000 7fc299cbc640 1 --2- 172.21.15.120:0/2494131892 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e57abd9950 0x55e57abdbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:20.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.184+0000 7fc299cbc640 1 --2- 172.21.15.120:0/2494131892 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e57abd9950 0x55e57abdbe10 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f70b52f6b4be2ff1 server_cookie=7ad402fcfed21a5e in_seq=0 out_seq=0 2024-09-19T09:18:20.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.184+0000 7fc2797fa640 10 client.29633 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:20.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.184+0000 7fc2797fa640 1 -- 172.21.15.120:0/2494131892 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e57ab317f0 con 0x55e57abd9950 2024-09-19T09:18:20.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.184+0000 7fc2797fa640 10 client.29633 handle_command_reply: tid=0 2024-09-19T09:18:20.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.185+0000 7fc29e2f9740 1 client.29633 shutdown 2024-09-19T09:18:20.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.185+0000 7fc29e2f9740 1 -- 172.21.15.120:0/2494131892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e57abbf480 msgr2=0x55e57abc70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:20.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.185+0000 7fc29e2f9740 1 --2- 172.21.15.120:0/2494131892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e57abbf480 0x55e57abc70c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc288098c60 tx=0x7fc288098660 comp rx=0 tx=0).stop 2024-09-19T09:18:20.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.185+0000 7fc29e2f9740 1 -- 172.21.15.120:0/2494131892 shutdown_connections 2024-09-19T09:18:20.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.185+0000 7fc29e2f9740 1 --2- 172.21.15.120:0/2494131892 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e57abd9950 0x55e57abdbe10 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.185+0000 7fc29e2f9740 1 --2- 172.21.15.120:0/2494131892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e57abc9d40 0x55e57abc7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.185+0000 7fc29e2f9740 1 --2- 172.21.15.120:0/2494131892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e57abbf480 0x55e57abc70c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.185+0000 7fc29e2f9740 1 --2- 172.21.15.120:0/2494131892 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e57abbeb60 0x55e57abc6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.185+0000 7fc29e2f9740 1 -- 172.21.15.120:0/2494131892 >> 172.21.15.120:0/2494131892 conn(0x55e57abbb0f0 msgr2=0x55e57abba270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:20.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.185+0000 7fc2797fa640 0 client.29633 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:20.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.185+0000 7fc29e2f9740 1 -- 172.21.15.120:0/2494131892 shutdown_connections 2024-09-19T09:18:20.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.185+0000 7fc29e2f9740 1 -- 172.21.15.120:0/2494131892 wait complete. 2024-09-19T09:18:20.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.185+0000 7fc29e2f9740 20 client.29633 trim_cache size 0 max 16384 2024-09-19T09:18:20.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.218+0000 7fc29e2f9740 1 Processor -- start 2024-09-19T09:18:20.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.218+0000 7fc29e2f9740 1 -- start start 2024-09-19T09:18:20.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.218+0000 7fc29e2f9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e57abbeb60 0x55e57abc89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:20.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.218+0000 7fc29e2f9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e57abbf480 0x55e57abe16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:20.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.218+0000 7fc29e2f9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e57abc9d40 0x55e57abe1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:20.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.218+0000 7fc29e2f9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc2941af720 con 0x55e57abc9d40 2024-09-19T09:18:20.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.218+0000 7fc29e2f9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc2941b12f0 con 0x55e57abbf480 2024-09-19T09:18:20.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.218+0000 7fc29e2f9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc2941afdf0 con 0x55e57abbeb60 2024-09-19T09:18:20.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.218+0000 7fc29a4bd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e57abc9d40 0x55e57abe1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:20.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.218+0000 7fc29a4bd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e57abc9d40 0x55e57abe1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33252/0 (socket says 172.21.15.120:33252) 2024-09-19T09:18:20.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.218+0000 7fc29a4bd640 1 -- 172.21.15.120:0/3831763556 learned_addr learned my addr 172.21.15.120:0/3831763556 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:20.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.218+0000 7fc2994bb640 1 --2- 172.21.15.120:0/3831763556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e57abbf480 0x55e57abe16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:20.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.219+0000 7fc299cbc640 1 --2- 172.21.15.120:0/3831763556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e57abbeb60 0x55e57abc89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:20.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.219+0000 7fc29a4bd640 1 -- 172.21.15.120:0/3831763556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e57abbeb60 msgr2=0x55e57abc89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:20.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.219+0000 7fc29a4bd640 1 --2- 172.21.15.120:0/3831763556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e57abbeb60 0x55e57abc89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.219+0000 7fc29a4bd640 1 -- 172.21.15.120:0/3831763556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e57abbf480 msgr2=0x55e57abe16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:20.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.219+0000 7fc29a4bd640 1 --2- 172.21.15.120:0/3831763556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e57abbf480 0x55e57abe16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.219+0000 7fc29a4bd640 1 -- 172.21.15.120:0/3831763556 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e57abbf2b0 con 0x55e57abc9d40 2024-09-19T09:18:20.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.219+0000 7fc2994bb640 1 --2- 172.21.15.120:0/3831763556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e57abbf480 0x55e57abe16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:18:20.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.219+0000 7fc29a4bd640 1 --2- 172.21.15.120:0/3831763556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e57abc9d40 0x55e57abe1c20 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fc2840042f0 tx=0x7fc28400f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edff8c5010af4563 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:20.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.219+0000 7fc27b7fe640 1 -- 172.21.15.120:0/3831763556 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc284026070 con 0x55e57abc9d40 2024-09-19T09:18:20.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.219+0000 7fc27b7fe640 1 -- 172.21.15.120:0/3831763556 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2840047a0 con 0x55e57abc9d40 2024-09-19T09:18:20.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.219+0000 7fc27b7fe640 1 -- 172.21.15.120:0/3831763556 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2840038e0 con 0x55e57abc9d40 2024-09-19T09:18:20.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.220+0000 7fc29e2f9740 1 -- 172.21.15.120:0/3831763556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e57abc9d40 msgr2=0x55e57abe1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:20.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.220+0000 7fc29e2f9740 1 --2- 172.21.15.120:0/3831763556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e57abc9d40 0x55e57abe1c20 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fc2840042f0 tx=0x7fc28400f440 comp rx=0 tx=0).stop 2024-09-19T09:18:20.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.220+0000 7fc29e2f9740 1 -- 172.21.15.120:0/3831763556 shutdown_connections 2024-09-19T09:18:20.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.220+0000 7fc29e2f9740 1 --2- 172.21.15.120:0/3831763556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e57abc9d40 0x55e57abe1c20 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.220+0000 7fc29e2f9740 1 --2- 172.21.15.120:0/3831763556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e57abbf480 0x55e57abe16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.220+0000 7fc29e2f9740 1 --2- 172.21.15.120:0/3831763556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e57abbeb60 0x55e57abc89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.220+0000 7fc29e2f9740 1 -- 172.21.15.120:0/3831763556 >> 172.21.15.120:0/3831763556 conn(0x55e57abbb0f0 msgr2=0x55e57abc82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:20.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.220+0000 7fc29e2f9740 1 -- 172.21.15.120:0/3831763556 shutdown_connections 2024-09-19T09:18:20.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.221+0000 7fc29e2f9740 1 -- 172.21.15.120:0/3831763556 wait complete. 2024-09-19T09:18:20.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.221+0000 7fc29e2f9740 1 -- start start 2024-09-19T09:18:20.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.221+0000 7fc29e2f9740 1 Processor -- start 2024-09-19T09:18:20.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.221+0000 7fc29e2f9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e57abbeb60 0x55e57abce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:20.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.221+0000 7fc29e2f9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e57abbf480 0x55e57abce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:20.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.221+0000 7fc29e2f9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e57abc9d40 0x55e57ac3d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:20.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.221+0000 7fc29e2f9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc2941b0c20 con 0x55e57abbf480 2024-09-19T09:18:20.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.221+0000 7fc29e2f9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc2941b04c0 con 0x55e57abc9d40 2024-09-19T09:18:20.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.221+0000 7fc29e2f9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc2941ad6e0 con 0x55e57abbeb60 2024-09-19T09:18:20.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.221+0000 7fc29e2f9740 10 client.0 _ll_register_callbacks cb 0x7fc28800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:20.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.222+0000 7fc2994bb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e57abbf480 0x55e57abce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:20.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.222+0000 7fc29a4bd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e57abc9d40 0x55e57ac3d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:20.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.222+0000 7fc29a4bd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e57abc9d40 0x55e57ac3d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:49510/0 (socket says 172.21.15.120:49510) 2024-09-19T09:18:20.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.222+0000 7fc29a4bd640 1 -- 172.21.15.120:0/2232846721 learned_addr learned my addr 172.21.15.120:0/2232846721 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:20.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.222+0000 7fc299cbc640 1 --2- 172.21.15.120:0/2232846721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e57abbeb60 0x55e57abce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:20.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.222+0000 7fc29a4bd640 1 -- 172.21.15.120:0/2232846721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e57abbeb60 msgr2=0x55e57abce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:20.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.222+0000 7fc29a4bd640 1 --2- 172.21.15.120:0/2232846721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e57abbeb60 0x55e57abce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.222+0000 7fc29a4bd640 1 -- 172.21.15.120:0/2232846721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e57abbf480 msgr2=0x55e57abce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:20.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.222+0000 7fc29a4bd640 1 --2- 172.21.15.120:0/2232846721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e57abbf480 0x55e57abce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.222+0000 7fc29a4bd640 1 -- 172.21.15.120:0/2232846721 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e57ac3dd90 con 0x55e57abc9d40 2024-09-19T09:18:20.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.222+0000 7fc29a4bd640 1 --2- 172.21.15.120:0/2232846721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e57abc9d40 0x55e57ac3d6f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc2840051a0 tx=0x7fc2840051d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42894e627ece5ad2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:20.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.222+0000 7fc2797fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:18:20.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.222+0000 7fc2797fa640 1 -- 172.21.15.120:0/2232846721 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc284038060 con 0x55e57abc9d40 2024-09-19T09:18:20.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.222+0000 7fc2797fa640 1 -- 172.21.15.120:0/2232846721 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc2840035f0 con 0x55e57abc9d40 2024-09-19T09:18:20.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.222+0000 7fc29e2f9740 1 -- 172.21.15.120:0/2232846721 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2840047a0 con 0x55e57abc9d40 2024-09-19T09:18:20.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.223+0000 7fc2797fa640 1 -- 172.21.15.120:0/2232846721 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc284026040 con 0x55e57abc9d40 2024-09-19T09:18:20.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.223+0000 7fc29e2f9740 10 client.29639 fetch_fsmap learned FSMap version 36 2024-09-19T09:18:20.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.223+0000 7fc29e2f9740 1 -- 172.21.15.120:0/2232846721 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e57ac45160 con 0x55e57abc9d40 2024-09-19T09:18:20.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.223+0000 7fc2797fa640 1 -- 172.21.15.120:0/2232846721 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2840047a0 con 0x55e57abc9d40 2024-09-19T09:18:20.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.223+0000 7fc2797fa640 1 -- 172.21.15.120:0/2232846721 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fc28403d070 con 0x55e57abc9d40 2024-09-19T09:18:20.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.223+0000 7fc29e2f9740 10 client.29639 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:18:20.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.223+0000 7fc29e2f9740 10 client.29639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:18:20.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.223+0000 7fc29e2f9740 20 client.29639 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:20.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.223+0000 7fc29e2f9740 1 --2- 172.21.15.120:0/2232846721 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e57ac45890 0x55e57ac47cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:20.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.223+0000 7fc29e2f9740 4 client.29639 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:18:20.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.223+0000 7fc29e2f9740 1 -- 172.21.15.120:0/2232846721 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e57ab2bd00 con 0x55e57ac45890 2024-09-19T09:18:20.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.223+0000 7fc299cbc640 1 --2- 172.21.15.120:0/2232846721 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e57ac45890 0x55e57ac47cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:20.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.224+0000 7fc2797fa640 10 client.29639 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:20.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.224+0000 7fc299cbc640 1 --2- 172.21.15.120:0/2232846721 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e57ac45890 0x55e57ac47cb0 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcf0241effea238c server_cookie=602601153d8bb9f0 in_seq=0 out_seq=0 2024-09-19T09:18:20.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.225+0000 7fc2797fa640 1 -- 172.21.15.120:0/2232846721 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e57ab2bd00 con 0x55e57ac45890 2024-09-19T09:18:20.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.225+0000 7fc2797fa640 10 client.29639 handle_command_reply: tid=0 2024-09-19T09:18:20.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.225+0000 7fc29e2f9740 1 client.29639 shutdown 2024-09-19T09:18:20.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.225+0000 7fc29e2f9740 1 -- 172.21.15.120:0/2232846721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e57abc9d40 msgr2=0x55e57ac3d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:20.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.225+0000 7fc29e2f9740 1 --2- 172.21.15.120:0/2232846721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e57abc9d40 0x55e57ac3d6f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc2840051a0 tx=0x7fc2840051d0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.225+0000 7fc29e2f9740 1 -- 172.21.15.120:0/2232846721 shutdown_connections 2024-09-19T09:18:20.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.225+0000 7fc29e2f9740 1 --2- 172.21.15.120:0/2232846721 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e57ac45890 0x55e57ac47cb0 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.225+0000 7fc2797fa640 0 client.29639 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:20.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.225+0000 7fc29e2f9740 1 --2- 172.21.15.120:0/2232846721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e57abc9d40 0x55e57ac3d6f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.225+0000 7fc29e2f9740 1 --2- 172.21.15.120:0/2232846721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e57abbf480 0x55e57abce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.225+0000 7fc29e2f9740 1 --2- 172.21.15.120:0/2232846721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e57abbeb60 0x55e57abce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.225+0000 7fc29e2f9740 1 -- 172.21.15.120:0/2232846721 >> 172.21.15.120:0/2232846721 conn(0x55e57abbb0f0 msgr2=0x55e57abc7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:20.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.225+0000 7fc29e2f9740 1 -- 172.21.15.120:0/2232846721 shutdown_connections 2024-09-19T09:18:20.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.225+0000 7fc29e2f9740 1 -- 172.21.15.120:0/2232846721 wait complete. 2024-09-19T09:18:20.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.225+0000 7fc29e2f9740 20 client.29639 trim_cache size 0 max 16384 2024-09-19T09:18:20.230 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:18:20.230 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:18:20.230 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "9df7c1a7-50f2-41fe-83f2-644fe6807b2d", 2024-09-19T09:18:20.230 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:18:20.230 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:18:20.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.226+0000 7fc29bf20640 1 -- 172.21.15.120:0/2200744324 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc27c0a32c0 msgr2=0x7fc27c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:20.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.226+0000 7fc29bf20640 1 --2- 172.21.15.120:0/2200744324 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc27c0a32c0 0x7fc27c0a5780 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc284005e00 tx=0x7fc284005c50 comp rx=0 tx=0).stop 2024-09-19T09:18:20.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.226+0000 7fc29bf20640 1 -- 172.21.15.120:0/2200744324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc294103740 msgr2=0x7fc294100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:20.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.226+0000 7fc29bf20640 1 --2- 172.21.15.120:0/2200744324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc294103740 0x7fc294100f30 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fc28800e9f0 tx=0x7fc28800eeb0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.226+0000 7fc29bf20640 1 -- 172.21.15.120:0/2200744324 shutdown_connections 2024-09-19T09:18:20.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.226+0000 7fc29bf20640 1 --2- 172.21.15.120:0/2200744324 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc27c0a32c0 0x7fc27c0a5780 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.226+0000 7fc29bf20640 1 --2- 172.21.15.120:0/2200744324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc294108290 0x7fc294101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.226+0000 7fc29bf20640 1 --2- 172.21.15.120:0/2200744324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc294103740 0x7fc294100f30 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.226+0000 7fc29bf20640 1 --2- 172.21.15.120:0/2200744324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc294102d90 0x7fc2941009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:20.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.226+0000 7fc29bf20640 1 -- 172.21.15.120:0/2200744324 >> 172.21.15.120:0/2200744324 conn(0x7fc2940fc7d0 msgr2=0x7fc294105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:20.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.226+0000 7fc29bf20640 1 -- 172.21.15.120:0/2200744324 shutdown_connections 2024-09-19T09:18:20.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:20.226+0000 7fc29bf20640 1 -- 172.21.15.120:0/2200744324 wait complete. 2024-09-19T09:18:20.357 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55751 0.00 MB/sec execute 15 sec latency 22482.004 ms 2024-09-19T09:18:20.688 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 19 09:18:20 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-b[64524]: 2024-09-19T09:18:20.260+0000 7f1ed1965640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000b60e [...2,head] /volumes/qa/sv_1/f2bee15c-e720-4829-ac9c-1a0401c9d37d/client.0/tmp/clients/client5/~dmtmp/PARADOX/ auth{1=1,2=1,3=1,4=1} v8273 ap=3 RANDEPHEMERALPIN f(v261 m2024-09-19T09:17:56.912760+0000 20=20+0) n(v9 rc2024-09-19T09:17:56.912760+0000 b5201920 21=20+1) (inest sync->mix g=1,2,3,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:1951 nref=3)) caps={15204=pAsLsXsFs/pAsLsXsFsx@727} mcw={1=AsLsXsFs} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x560de9a06680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000b60e:[<0x1000000b49b/PARADOX v268>,<0x1000000b49a/~dmtmp v47>,<0x10000000007/client5 v1645>,<0x10000000006/clients v1215>,<0x10000000005/tmp v1179>,<0x100000001fb/client.0 v1176>,<0x100000001fa/f2bee15c-e720-4829-ac9c-1a0401c9d37d v1163>,<0x10000000001/sv_1 v1171>,<0x10000000000/qa v1139>,<0x1/volumes v1126>]//[]","memoryvalue":"(2)0x1000000b60e:[<0x1000000b49b/PARADOX v8273>,<0x1000000b49a/~dmtmp v770>,<0x10000000007/client5 v3690>,<0x10000000006/clients v1602>,<0x10000000005/tmp v1403>,<0x100000001fb/client.0 v1384>,<0x100000001fa/f2bee15c-e720-4829-ac9c-1a0401c9d37d v1368>,<0x10000000001/sv_1 v1375>,<0x10000000000/qa v1339>,<0x1/volumes v1318>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T09:17:51.657887+0000 24=24+0)","ondisk_value.rstat":"n(v0 rc2024-09-19T09:17:56.912760+0000 b5201920 21=20+1)","memory_value.dirstat":"f(v261 m2024-09-19T09:17:56.912760+0000 20=20+0)","memory_value.rstat":"n(v9 rc2024-09-19T09:17:56.912760+0000 b5201920 21=20+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-19T09:18:21.046 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:18:21.295 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:18:21.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:20 smithi120 ceph-mon[31150]: pgmap v809: 97 pgs: 97 active+clean; 288 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 5.5 MiB/s wr, 278 op/s 2024-09-19T09:18:21.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:20 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:18:21.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:20 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:18:21.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:20 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:18:21.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:20 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:18:21.358 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55751 0.00 MB/sec execute 16 sec latency 23482.086 ms 2024-09-19T09:18:21.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:20 smithi196 ceph-mon[35150]: pgmap v809: 97 pgs: 97 active+clean; 288 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 5.5 MiB/s wr, 278 op/s 2024-09-19T09:18:21.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:20 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:18:21.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:20 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:18:21.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:20 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:18:21.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:20 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:18:21.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:20 smithi153 ceph-mon[35071]: pgmap v809: 97 pgs: 97 active+clean; 288 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 5.5 MiB/s wr, 278 op/s 2024-09-19T09:18:21.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:20 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:18:21.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:20 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:18:21.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:20 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:18:21.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:20 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:18:21.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.840+0000 7f7182f69640 1 -- 172.21.15.120:0/916948691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f717c106f10 msgr2=0x7f717c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:21.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.840+0000 7f7182f69640 1 --2- 172.21.15.120:0/916948691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f717c106f10 0x7f717c10ea90 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f7174009ca0 tx=0x7f717402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.840+0000 7f7182f69640 1 -- 172.21.15.120:0/916948691 shutdown_connections 2024-09-19T09:18:21.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.840+0000 7f7182f69640 1 --2- 172.21.15.120:0/916948691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f717c106f10 0x7f717c10ea90 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.840+0000 7f7182f69640 1 --2- 172.21.15.120:0/916948691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f717c102790 0x7f717c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.840+0000 7f7182f69640 1 --2- 172.21.15.120:0/916948691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f717c101de0 0x7f717c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.840+0000 7f7182f69640 1 -- 172.21.15.120:0/916948691 >> 172.21.15.120:0/916948691 conn(0x7f717c0fd8c0 msgr2=0x7f717c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:21.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.840+0000 7f7182f69640 1 -- 172.21.15.120:0/916948691 shutdown_connections 2024-09-19T09:18:21.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.840+0000 7f7182f69640 1 -- 172.21.15.120:0/916948691 wait complete. 2024-09-19T09:18:21.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.841+0000 7f7182f69640 1 Processor -- start 2024-09-19T09:18:21.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.841+0000 7f7182f69640 1 -- start start 2024-09-19T09:18:21.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.841+0000 7f7182f69640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f717c101de0 0x7f717c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:21.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.841+0000 7f7182f69640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f717c102790 0x7f717c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:21.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.841+0000 7f7182f69640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f717c106f10 0x7f717c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:21.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.841+0000 7f7182f69640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f717c111f70 con 0x7f717c102790 2024-09-19T09:18:21.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.841+0000 7f7182f69640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f717c111df0 con 0x7f717c101de0 2024-09-19T09:18:21.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.841+0000 7f7182f69640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f717c1120f0 con 0x7f717c106f10 2024-09-19T09:18:21.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.841+0000 7f717bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f717c102790 0x7f717c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:21.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.841+0000 7f717bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f717c102790 0x7f717c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46550/0 (socket says 172.21.15.120:46550) 2024-09-19T09:18:21.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.841+0000 7f717bfff640 1 -- 172.21.15.120:0/1118612331 learned_addr learned my addr 172.21.15.120:0/1118612331 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:21.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.841+0000 7f7180d05640 1 --2- 172.21.15.120:0/1118612331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f717c101de0 0x7f717c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:21.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.841+0000 7f7181506640 1 --2- 172.21.15.120:0/1118612331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f717c106f10 0x7f717c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:21.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.841+0000 7f717bfff640 1 -- 172.21.15.120:0/1118612331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f717c106f10 msgr2=0x7f717c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:21.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.841+0000 7f717bfff640 1 --2- 172.21.15.120:0/1118612331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f717c106f10 0x7f717c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.841+0000 7f717bfff640 1 -- 172.21.15.120:0/1118612331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f717c101de0 msgr2=0x7f717c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:21.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.841+0000 7f717bfff640 1 --2- 172.21.15.120:0/1118612331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f717c101de0 0x7f717c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.841+0000 7f717bfff640 1 -- 172.21.15.120:0/1118612331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717c1a67f0 con 0x7f717c102790 2024-09-19T09:18:21.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.841+0000 7f7181506640 1 --2- 172.21.15.120:0/1118612331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f717c106f10 0x7f717c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:18:21.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.841+0000 7f717bfff640 1 --2- 172.21.15.120:0/1118612331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f717c102790 0x7f717c074a30 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f716800c890 tx=0x7f716800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:21.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.842+0000 7f7179ffb640 1 -- 172.21.15.120:0/1118612331 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7168004220 con 0x7f717c102790 2024-09-19T09:18:21.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.842+0000 7f7182f69640 1 -- 172.21.15.120:0/1118612331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f717c1a6ae0 con 0x7f717c102790 2024-09-19T09:18:21.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.842+0000 7f7179ffb640 1 -- 172.21.15.120:0/1118612331 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71680043c0 con 0x7f717c102790 2024-09-19T09:18:21.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.842+0000 7f7179ffb640 1 -- 172.21.15.120:0/1118612331 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7168004a00 con 0x7f717c102790 2024-09-19T09:18:21.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.842+0000 7f7182f69640 1 -- 172.21.15.120:0/1118612331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f717c1a7070 con 0x7f717c102790 2024-09-19T09:18:21.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.844+0000 7f7179ffb640 1 -- 172.21.15.120:0/1118612331 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f71680079b0 con 0x7f717c102790 2024-09-19T09:18:21.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.844+0000 7f7185342740 1 Processor -- start 2024-09-19T09:18:21.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.844+0000 7f7185342740 1 -- start start 2024-09-19T09:18:21.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.845+0000 7f7179ffb640 1 --2- 172.21.15.120:0/1118612331 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f716c0a3240 0x7f716c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:21.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.845+0000 7f7185342740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f011c2b4a0 0x55f011c2b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:21.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.845+0000 7f7185342740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f011c2bdc0 0x55f011c36140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:21.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.845+0000 7f7185342740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f011c36680 0x55f011c38a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:21.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.845+0000 7f7185342740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f011b912e0 con 0x55f011c2bdc0 2024-09-19T09:18:21.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.845+0000 7f7185342740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f011b38ce0 con 0x55f011c2b4a0 2024-09-19T09:18:21.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.845+0000 7f7185342740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f011b8d2a0 con 0x55f011c36680 2024-09-19T09:18:21.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.845+0000 7f7179ffb640 1 -- 172.21.15.120:0/1118612331 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f71680cf560 con 0x7f717c102790 2024-09-19T09:18:21.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.845+0000 7f7181506640 1 --2- 172.21.15.120:0/1118612331 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f716c0a3240 0x7f716c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:21.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.845+0000 7f717bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f011c2bdc0 0x55f011c36140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:21.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.845+0000 7f717bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f011c2bdc0 0x55f011c36140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46560/0 (socket says 172.21.15.120:46560) 2024-09-19T09:18:21.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.845+0000 7f7180d05640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f011c2b4a0 0x55f011c2b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:21.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.845+0000 7f717bfff640 1 -- 172.21.15.120:0/3444819336 learned_addr learned my addr 172.21.15.120:0/3444819336 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:21.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.845+0000 7f7181506640 1 --2- 172.21.15.120:0/3444819336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f011c36680 0x55f011c38a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:21.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.845+0000 7f717bfff640 1 -- 172.21.15.120:0/3444819336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f011c36680 msgr2=0x55f011c38a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:21.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.845+0000 7f717bfff640 1 --2- 172.21.15.120:0/3444819336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f011c36680 0x55f011c38a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.845+0000 7f717bfff640 1 -- 172.21.15.120:0/3444819336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f011c2b4a0 msgr2=0x55f011c2b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:21.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.845+0000 7f717bfff640 1 --2- 172.21.15.120:0/3444819336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f011c2b4a0 0x55f011c2b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.845+0000 7f717bfff640 1 -- 172.21.15.120:0/3444819336 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f011b1d440 con 0x55f011c2bdc0 2024-09-19T09:18:21.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.845+0000 7f7180d05640 1 --2- 172.21.15.120:0/3444819336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f011c2b4a0 0x55f011c2b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:18:21.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.845+0000 7f717bfff640 1 --2- 172.21.15.120:0/3444819336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f011c2bdc0 0x55f011c36140 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f716809b880 tx=0x7f7168004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ca0b0c70115ec6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:21.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.845+0000 7f7181506640 1 --2- 172.21.15.120:0/1118612331 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f716c0a3240 0x7f716c0a5700 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x55f011c28db0 tx=0x7f7174005f80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:21.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.845+0000 7f7166ffd640 1 -- 172.21.15.120:0/3444819336 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716801d040 con 0x55f011c2bdc0 2024-09-19T09:18:21.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.845+0000 7f7166ffd640 1 -- 172.21.15.120:0/3444819336 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7168012110 con 0x55f011c2bdc0 2024-09-19T09:18:21.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.845+0000 7f7166ffd640 1 -- 172.21.15.120:0/3444819336 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71680a47f0 con 0x55f011c2bdc0 2024-09-19T09:18:21.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.846+0000 7f7185342740 1 -- 172.21.15.120:0/3444819336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f011c2bdc0 msgr2=0x55f011c36140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:21.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.846+0000 7f7185342740 1 --2- 172.21.15.120:0/3444819336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f011c2bdc0 0x55f011c36140 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f716809b880 tx=0x7f7168004570 comp rx=0 tx=0).stop 2024-09-19T09:18:21.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.846+0000 7f7185342740 1 -- 172.21.15.120:0/3444819336 shutdown_connections 2024-09-19T09:18:21.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.846+0000 7f7185342740 1 --2- 172.21.15.120:0/3444819336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f011c36680 0x55f011c38a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.846+0000 7f7185342740 1 --2- 172.21.15.120:0/3444819336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f011c2bdc0 0x55f011c36140 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.846+0000 7f7185342740 1 --2- 172.21.15.120:0/3444819336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f011c2b4a0 0x55f011c2b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.846+0000 7f7185342740 1 -- 172.21.15.120:0/3444819336 >> 172.21.15.120:0/3444819336 conn(0x55f011c27a30 msgr2=0x55f011c27e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:21.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.846+0000 7f7185342740 1 -- 172.21.15.120:0/3444819336 shutdown_connections 2024-09-19T09:18:21.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.846+0000 7f7185342740 1 -- 172.21.15.120:0/3444819336 wait complete. 2024-09-19T09:18:21.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.846+0000 7f7185342740 1 -- start start 2024-09-19T09:18:21.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.847+0000 7f7185342740 1 Processor -- start 2024-09-19T09:18:21.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.847+0000 7f7185342740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f011c2b4a0 0x55f011c35fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:21.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.847+0000 7f7185342740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f011c2bdc0 0x55f011c33250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:21.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.847+0000 7f7185342740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f011c36680 0x55f011c33790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:21.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.847+0000 7f7185342740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f011c39250 con 0x55f011c2bdc0 2024-09-19T09:18:21.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.847+0000 7f7185342740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f011b4c050 con 0x55f011c36680 2024-09-19T09:18:21.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.847+0000 7f7185342740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f011c393d0 con 0x55f011c2b4a0 2024-09-19T09:18:21.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.847+0000 7f7185342740 10 client.0 _ll_register_callbacks cb 0x55f011b8c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:21.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.847+0000 7f717bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f011c2bdc0 0x55f011c33250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:21.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.847+0000 7f717bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f011c2bdc0 0x55f011c33250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46576/0 (socket says 172.21.15.120:46576) 2024-09-19T09:18:21.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.847+0000 7f717bfff640 1 -- 172.21.15.120:0/2035558079 learned_addr learned my addr 172.21.15.120:0/2035558079 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:21.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.847+0000 7f7180d05640 1 --2- 172.21.15.120:0/2035558079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f011c2b4a0 0x55f011c35fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:21.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.847+0000 7f717bfff640 1 -- 172.21.15.120:0/2035558079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f011c2b4a0 msgr2=0x55f011c35fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:21.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.847+0000 7f717bfff640 1 --2- 172.21.15.120:0/2035558079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f011c2b4a0 0x55f011c35fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.847+0000 7f717bfff640 1 -- 172.21.15.120:0/2035558079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f011c36680 msgr2=0x55f011c33790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:21.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.847+0000 7f717bfff640 1 --2- 172.21.15.120:0/2035558079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f011c36680 0x55f011c33790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.847+0000 7f717bfff640 1 -- 172.21.15.120:0/2035558079 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0119023c0 con 0x55f011c2bdc0 2024-09-19T09:18:21.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.847+0000 7f717bfff640 1 --2- 172.21.15.120:0/2035558079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f011c2bdc0 0x55f011c33250 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f71680a42c0 tx=0x7f7168000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b75a07c01223270 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:21.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.848+0000 7f715bfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:18:21.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.848+0000 7f715bfff640 1 -- 172.21.15.120:0/2035558079 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716801d040 con 0x55f011c2bdc0 2024-09-19T09:18:21.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.848+0000 7f715bfff640 1 -- 172.21.15.120:0/2035558079 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f7168086110 con 0x55f011c2bdc0 2024-09-19T09:18:21.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.848+0000 7f7185342740 1 -- 172.21.15.120:0/2035558079 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7168012110 con 0x55f011c2bdc0 2024-09-19T09:18:21.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.848+0000 7f715bfff640 1 -- 172.21.15.120:0/2035558079 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7168086350 con 0x55f011c2bdc0 2024-09-19T09:18:21.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.848+0000 7f7185342740 10 client.20748 fetch_fsmap learned FSMap version 36 2024-09-19T09:18:21.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.848+0000 7f7185342740 1 -- 172.21.15.120:0/2035558079 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f011c39250 con 0x55f011c2bdc0 2024-09-19T09:18:21.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.848+0000 7f715bfff640 1 -- 172.21.15.120:0/2035558079 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7168012110 con 0x55f011c2bdc0 2024-09-19T09:18:21.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.848+0000 7f715bfff640 1 -- 172.21.15.120:0/2035558079 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f71680109f0 con 0x55f011c2bdc0 2024-09-19T09:18:21.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.848+0000 7f7185342740 10 client.20748 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:18:21.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.848+0000 7f7185342740 10 client.20748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:18:21.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.848+0000 7f7185342740 20 client.20748 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:21.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.848+0000 7f7185342740 1 --2- 172.21.15.120:0/2035558079 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f011c45d80 0x55f011c48170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:21.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.848+0000 7f7185342740 4 client.20748 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:18:21.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.848+0000 7f7185342740 1 -- 172.21.15.120:0/2035558079 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f011b9e850 con 0x55f011c45d80 2024-09-19T09:18:21.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.848+0000 7f7180d05640 1 --2- 172.21.15.120:0/2035558079 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f011c45d80 0x55f011c48170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:21.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.849+0000 7f715bfff640 10 client.20748 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:21.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.849+0000 7f7180d05640 1 --2- 172.21.15.120:0/2035558079 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f011c45d80 0x55f011c48170 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=979c3f62479c2108 server_cookie=6965afe600693ed7 in_seq=0 out_seq=0 2024-09-19T09:18:21.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.849+0000 7f715bfff640 1 -- 172.21.15.120:0/2035558079 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f011b9e850 con 0x55f011c45d80 2024-09-19T09:18:21.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.849+0000 7f715bfff640 10 client.20748 handle_command_reply: tid=0 2024-09-19T09:18:21.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.849+0000 7f7185342740 1 client.20748 shutdown 2024-09-19T09:18:21.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.850+0000 7f7185342740 1 -- 172.21.15.120:0/2035558079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f011c2bdc0 msgr2=0x55f011c33250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:21.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.850+0000 7f7185342740 1 --2- 172.21.15.120:0/2035558079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f011c2bdc0 0x55f011c33250 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f71680a42c0 tx=0x7f7168000bc0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.850+0000 7f7185342740 1 -- 172.21.15.120:0/2035558079 shutdown_connections 2024-09-19T09:18:21.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.850+0000 7f7185342740 1 --2- 172.21.15.120:0/2035558079 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f011c45d80 0x55f011c48170 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.850+0000 7f7185342740 1 --2- 172.21.15.120:0/2035558079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f011c36680 0x55f011c33790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.850+0000 7f715bfff640 0 client.20748 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:21.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.850+0000 7f7185342740 1 --2- 172.21.15.120:0/2035558079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f011c2bdc0 0x55f011c33250 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.850+0000 7f7185342740 1 --2- 172.21.15.120:0/2035558079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f011c2b4a0 0x55f011c35fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.850+0000 7f7185342740 1 -- 172.21.15.120:0/2035558079 >> 172.21.15.120:0/2035558079 conn(0x55f011c27a30 msgr2=0x55f011c26b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:21.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.850+0000 7f7185342740 1 -- 172.21.15.120:0/2035558079 shutdown_connections 2024-09-19T09:18:21.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.850+0000 7f7185342740 1 -- 172.21.15.120:0/2035558079 wait complete. 2024-09-19T09:18:21.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.850+0000 7f7185342740 20 client.20748 trim_cache size 0 max 16384 2024-09-19T09:18:21.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.867+0000 7f7185342740 1 Processor -- start 2024-09-19T09:18:21.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.867+0000 7f7185342740 1 -- start start 2024-09-19T09:18:21.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.867+0000 7f7185342740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f011c2b4a0 0x55f011c424e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:21.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.867+0000 7f7185342740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f011c2bdc0 0x55f011c4d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:21.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.867+0000 7f7185342740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f011c36680 0x55f011c4dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:21.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.867+0000 7f7185342740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f717c1af7d0 con 0x55f011c36680 2024-09-19T09:18:21.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.867+0000 7f7185342740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f717c1b13a0 con 0x55f011c2bdc0 2024-09-19T09:18:21.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.867+0000 7f7185342740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f717c1afea0 con 0x55f011c2b4a0 2024-09-19T09:18:21.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.867+0000 7f7181506640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f011c36680 0x55f011c4dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:21.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.867+0000 7f7181506640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f011c36680 0x55f011c4dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46580/0 (socket says 172.21.15.120:46580) 2024-09-19T09:18:21.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.867+0000 7f7181506640 1 -- 172.21.15.120:0/1231463081 learned_addr learned my addr 172.21.15.120:0/1231463081 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:21.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.867+0000 7f717bfff640 1 --2- 172.21.15.120:0/1231463081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f011c2bdc0 0x55f011c4d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:21.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.867+0000 7f7180d05640 1 --2- 172.21.15.120:0/1231463081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f011c2b4a0 0x55f011c424e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:21.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.867+0000 7f7181506640 1 -- 172.21.15.120:0/1231463081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f011c2b4a0 msgr2=0x55f011c424e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:21.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.867+0000 7f7181506640 1 --2- 172.21.15.120:0/1231463081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f011c2b4a0 0x55f011c424e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.867+0000 7f7181506640 1 -- 172.21.15.120:0/1231463081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f011c2bdc0 msgr2=0x55f011c4d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:21.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.867+0000 7f7181506640 1 --2- 172.21.15.120:0/1231463081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f011c2bdc0 0x55f011c4d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.867+0000 7f7181506640 1 -- 172.21.15.120:0/1231463081 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f011c2c330 con 0x55f011c36680 2024-09-19T09:18:21.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.868+0000 7f7181506640 1 --2- 172.21.15.120:0/1231463081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f011c36680 0x55f011c4dd60 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f7174003c00 tx=0x7f717402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e6bbfdfee91bb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:21.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.868+0000 7f71667fc640 1 -- 172.21.15.120:0/1231463081 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7174044030 con 0x55f011c36680 2024-09-19T09:18:21.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.868+0000 7f71667fc640 1 -- 172.21.15.120:0/1231463081 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7174050070 con 0x55f011c36680 2024-09-19T09:18:21.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.868+0000 7f71667fc640 1 -- 172.21.15.120:0/1231463081 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717402f990 con 0x55f011c36680 2024-09-19T09:18:21.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.868+0000 7f7185342740 1 -- 172.21.15.120:0/1231463081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f011c36680 msgr2=0x55f011c4dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:21.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.868+0000 7f7185342740 1 --2- 172.21.15.120:0/1231463081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f011c36680 0x55f011c4dd60 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f7174003c00 tx=0x7f717402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.868+0000 7f7185342740 1 -- 172.21.15.120:0/1231463081 shutdown_connections 2024-09-19T09:18:21.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.868+0000 7f7185342740 1 --2- 172.21.15.120:0/1231463081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f011c36680 0x55f011c4dd60 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.868+0000 7f7185342740 1 --2- 172.21.15.120:0/1231463081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f011c2bdc0 0x55f011c4d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.869+0000 7f7185342740 1 --2- 172.21.15.120:0/1231463081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f011c2b4a0 0x55f011c424e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.869+0000 7f7185342740 1 -- 172.21.15.120:0/1231463081 >> 172.21.15.120:0/1231463081 conn(0x55f011c27a30 msgr2=0x55f011c352d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:21.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.869+0000 7f7185342740 1 -- 172.21.15.120:0/1231463081 shutdown_connections 2024-09-19T09:18:21.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.869+0000 7f7185342740 1 -- 172.21.15.120:0/1231463081 wait complete. 2024-09-19T09:18:21.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.869+0000 7f7185342740 1 -- start start 2024-09-19T09:18:21.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.869+0000 7f7185342740 1 Processor -- start 2024-09-19T09:18:21.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.869+0000 7f7185342740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f011c2b4a0 0x55f011c3e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:21.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.869+0000 7f7185342740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f011c2bdc0 0x55f011c3ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:21.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.869+0000 7f7185342740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f011c36680 0x55f011c3afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:21.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.869+0000 7f7185342740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f717c1b0cd0 con 0x55f011c2b4a0 2024-09-19T09:18:21.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.869+0000 7f7185342740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f717c1b0570 con 0x55f011c36680 2024-09-19T09:18:21.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.870+0000 7f7185342740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f717c1ad790 con 0x55f011c2bdc0 2024-09-19T09:18:21.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.870+0000 7f7185342740 10 client.0 _ll_register_callbacks cb 0x7f717000a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:21.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.870+0000 7f7180d05640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f011c2b4a0 0x55f011c3e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:21.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.870+0000 7f7180d05640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f011c2b4a0 0x55f011c3e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46594/0 (socket says 172.21.15.120:46594) 2024-09-19T09:18:21.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.870+0000 7f7180d05640 1 -- 172.21.15.120:0/2863951054 learned_addr learned my addr 172.21.15.120:0/2863951054 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:21.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.870+0000 7f717bfff640 1 --2- 172.21.15.120:0/2863951054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f011c2bdc0 0x55f011c3ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:21.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.870+0000 7f7181506640 1 --2- 172.21.15.120:0/2863951054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f011c36680 0x55f011c3afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:21.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.870+0000 7f7180d05640 1 -- 172.21.15.120:0/2863951054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f011c2bdc0 msgr2=0x55f011c3ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:21.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.870+0000 7f7180d05640 1 --2- 172.21.15.120:0/2863951054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f011c2bdc0 0x55f011c3ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.870+0000 7f7180d05640 1 -- 172.21.15.120:0/2863951054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f011c36680 msgr2=0x55f011c3afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:21.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.870+0000 7f7180d05640 1 --2- 172.21.15.120:0/2863951054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f011c36680 0x55f011c3afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.870+0000 7f7180d05640 1 -- 172.21.15.120:0/2863951054 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f011c3b7e0 con 0x55f011c2b4a0 2024-09-19T09:18:21.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.870+0000 7f7180d05640 1 --2- 172.21.15.120:0/2863951054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f011c2b4a0 0x55f011c3e9d0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f7170004700 tx=0x7f717001fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de720c46a9435630 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:21.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.870+0000 7f715bfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:18:21.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.870+0000 7f715bfff640 1 -- 172.21.15.120:0/2863951054 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717001fcd0 con 0x55f011c2b4a0 2024-09-19T09:18:21.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.870+0000 7f715bfff640 1 -- 172.21.15.120:0/2863951054 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f7170010b70 con 0x55f011c2b4a0 2024-09-19T09:18:21.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.870+0000 7f7185342740 1 -- 172.21.15.120:0/2863951054 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7174050070 con 0x55f011c2b4a0 2024-09-19T09:18:21.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.870+0000 7f715bfff640 1 -- 172.21.15.120:0/2863951054 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7170010d70 con 0x55f011c2b4a0 2024-09-19T09:18:21.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.870+0000 7f7185342740 10 client.20754 fetch_fsmap learned FSMap version 36 2024-09-19T09:18:21.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.870+0000 7f7185342740 1 -- 172.21.15.120:0/2863951054 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f011cb14d0 con 0x55f011c2b4a0 2024-09-19T09:18:21.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.871+0000 7f715bfff640 1 -- 172.21.15.120:0/2863951054 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7174050070 con 0x55f011c2b4a0 2024-09-19T09:18:21.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.871+0000 7f715bfff640 1 -- 172.21.15.120:0/2863951054 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f717002e020 con 0x55f011c2b4a0 2024-09-19T09:18:21.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.871+0000 7f7185342740 10 client.20754 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:18:21.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.871+0000 7f7185342740 10 client.20754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:18:21.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.871+0000 7f7185342740 20 client.20754 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:21.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.871+0000 7f7185342740 1 --2- 172.21.15.120:0/2863951054 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f011cb1cb0 0x55f011cb40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:21.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.871+0000 7f7185342740 4 client.20754 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:18:21.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.871+0000 7f7185342740 1 -- 172.21.15.120:0/2863951054 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f011b98d60 con 0x55f011cb1cb0 2024-09-19T09:18:21.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.871+0000 7f717bfff640 1 --2- 172.21.15.120:0/2863951054 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f011cb1cb0 0x55f011cb40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:21.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.871+0000 7f715bfff640 10 client.20754 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:21.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.871+0000 7f717bfff640 1 --2- 172.21.15.120:0/2863951054 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f011cb1cb0 0x55f011cb40d0 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2502e680aecb0be8 server_cookie=a83953b2f669847d in_seq=0 out_seq=0 2024-09-19T09:18:21.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.872+0000 7f715bfff640 1 -- 172.21.15.120:0/2863951054 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f011b98d60 con 0x55f011cb1cb0 2024-09-19T09:18:21.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.872+0000 7f715bfff640 10 client.20754 handle_command_reply: tid=0 2024-09-19T09:18:21.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.872+0000 7f7185342740 1 client.20754 shutdown 2024-09-19T09:18:21.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.872+0000 7f7185342740 1 -- 172.21.15.120:0/2863951054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f011c2b4a0 msgr2=0x55f011c3e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:21.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.872+0000 7f7185342740 1 --2- 172.21.15.120:0/2863951054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f011c2b4a0 0x55f011c3e9d0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f7170004700 tx=0x7f717001fa20 comp rx=0 tx=0).stop 2024-09-19T09:18:21.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.872+0000 7f7185342740 1 -- 172.21.15.120:0/2863951054 shutdown_connections 2024-09-19T09:18:21.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.872+0000 7f7185342740 1 --2- 172.21.15.120:0/2863951054 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f011cb1cb0 0x55f011cb40d0 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.872+0000 7f7185342740 1 --2- 172.21.15.120:0/2863951054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f011c36680 0x55f011c3afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.872+0000 7f7185342740 1 --2- 172.21.15.120:0/2863951054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f011c2bdc0 0x55f011c3ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.872+0000 7f715bfff640 0 client.20754 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:21.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.872+0000 7f7185342740 1 --2- 172.21.15.120:0/2863951054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f011c2b4a0 0x55f011c3e9d0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.872+0000 7f7185342740 1 -- 172.21.15.120:0/2863951054 >> 172.21.15.120:0/2863951054 conn(0x55f011c27a30 msgr2=0x55f011c26350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:21.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.872+0000 7f7185342740 1 -- 172.21.15.120:0/2863951054 shutdown_connections 2024-09-19T09:18:21.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.872+0000 7f7185342740 1 -- 172.21.15.120:0/2863951054 wait complete. 2024-09-19T09:18:21.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.872+0000 7f7185342740 20 client.20754 trim_cache size 0 max 16384 2024-09-19T09:18:21.889 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:18:21.889 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:18:21.889 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-19T09:18:21.889 INFO:teuthology.orchestra.run.smithi120.stdout: "9df7c1a7-50f2-41fe-83f2-644fe6807b2d": { 2024-09-19T09:18:21.889 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-19T09:18:21.889 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "9df7c1a7-50f2-41fe-83f2-644fe6807b2d", 2024-09-19T09:18:21.889 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-19T09:18:21.889 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:18:21.889 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:18:21.889 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:18:21.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.872+0000 7f7182f69640 1 -- 172.21.15.120:0/1118612331 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f716c0a3240 msgr2=0x7f716c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:21.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.872+0000 7f7182f69640 1 --2- 172.21.15.120:0/1118612331 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f716c0a3240 0x7f716c0a5700 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x55f011c28db0 tx=0x7f7174005f80 comp rx=0 tx=0).stop 2024-09-19T09:18:21.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.873+0000 7f7182f69640 1 -- 172.21.15.120:0/1118612331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f717c102790 msgr2=0x7f717c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:21.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.873+0000 7f7182f69640 1 --2- 172.21.15.120:0/1118612331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f717c102790 0x7f717c074a30 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f716800c890 tx=0x7f716800cc90 comp rx=0 tx=0).stop 2024-09-19T09:18:21.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.873+0000 7f7182f69640 1 -- 172.21.15.120:0/1118612331 shutdown_connections 2024-09-19T09:18:21.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.873+0000 7f7182f69640 1 --2- 172.21.15.120:0/1118612331 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f716c0a3240 0x7f716c0a5700 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.873+0000 7f7182f69640 1 --2- 172.21.15.120:0/1118612331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f717c106f10 0x7f717c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.873+0000 7f7182f69640 1 --2- 172.21.15.120:0/1118612331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f717c102790 0x7f717c074a30 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.873+0000 7f7182f69640 1 --2- 172.21.15.120:0/1118612331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f717c101de0 0x7f717c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.873+0000 7f7182f69640 1 -- 172.21.15.120:0/1118612331 >> 172.21.15.120:0/1118612331 conn(0x7f717c0fd8c0 msgr2=0x7f717c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:21.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.873+0000 7f7182f69640 1 -- 172.21.15.120:0/1118612331 shutdown_connections 2024-09-19T09:18:21.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:21.873+0000 7f7182f69640 1 -- 172.21.15.120:0/1118612331 wait complete. 2024-09-19T09:18:22.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:21 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:18:22.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:21 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:18:22.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:21 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:18:22.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:21 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:18:22.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:21 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:18:22.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:21 smithi120 ceph-mon[31150]: Scrub error on inode 0x1000000b60e (/volumes/qa/sv_1/f2bee15c-e720-4829-ac9c-1a0401c9d37d/client.0/tmp/clients/client5/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-09-19T09:18:22.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:21 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:18:22.358 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55751 0.00 MB/sec execute 17 sec latency 24482.160 ms 2024-09-19T09:18:22.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:21 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:18:22.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:21 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:18:22.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:21 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:18:22.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:21 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:18:22.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:21 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:18:22.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:21 smithi196 ceph-mon[35150]: Scrub error on inode 0x1000000b60e (/volumes/qa/sv_1/f2bee15c-e720-4829-ac9c-1a0401c9d37d/client.0/tmp/clients/client5/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-09-19T09:18:22.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:21 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:18:22.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:21 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:18:22.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:21 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:18:22.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:21 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:18:22.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:21 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:18:22.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:21 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:18:22.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:21 smithi153 ceph-mon[35071]: Scrub error on inode 0x1000000b60e (/volumes/qa/sv_1/f2bee15c-e720-4829-ac9c-1a0401c9d37d/client.0/tmp/clients/client5/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-09-19T09:18:22.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:21 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:18:22.787 INFO:tasks.cephfs.filesystem:scrub status for tag:9df7c1a7-50f2-41fe-83f2-644fe6807b2d - {'path': '/', 'tag': '9df7c1a7-50f2-41fe-83f2-644fe6807b2d', 'options': 'recursive,force'} 2024-09-19T09:18:23.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:22 smithi120 ceph-mon[31150]: pgmap v810: 97 pgs: 97 active+clean; 289 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 5.5 MiB/s wr, 271 op/s 2024-09-19T09:18:23.358 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55751 0.00 MB/sec execute 18 sec latency 25482.188 ms 2024-09-19T09:18:23.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:22 smithi196 ceph-mon[35150]: pgmap v810: 97 pgs: 97 active+clean; 289 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 5.5 MiB/s wr, 271 op/s 2024-09-19T09:18:23.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:22 smithi153 ceph-mon[35071]: pgmap v810: 97 pgs: 97 active+clean; 289 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 5.5 MiB/s wr, 271 op/s 2024-09-19T09:18:24.063 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:18:24.311 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:18:24.358 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55751 0.00 MB/sec execute 19 sec latency 26482.249 ms 2024-09-19T09:18:24.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.808+0000 7fba6843d640 1 -- 172.21.15.120:0/1981549630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba60100e20 msgr2=0x7fba60101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:24.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.809+0000 7fba6843d640 1 --2- 172.21.15.120:0/1981549630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba60100e20 0x7fba60101200 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fba58009900 tx=0x7fba5802a480 comp rx=0 tx=0).stop 2024-09-19T09:18:24.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.809+0000 7fba6843d640 1 -- 172.21.15.120:0/1981549630 shutdown_connections 2024-09-19T09:18:24.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.809+0000 7fba6843d640 1 --2- 172.21.15.120:0/1981549630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba60102ef0 0x7fba6010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:24.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.809+0000 7fba6843d640 1 --2- 172.21.15.120:0/1981549630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba60102570 0x7fba601029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:24.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.809+0000 7fba6843d640 1 --2- 172.21.15.120:0/1981549630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba60100e20 0x7fba60101200 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:24.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.809+0000 7fba6843d640 1 -- 172.21.15.120:0/1981549630 >> 172.21.15.120:0/1981549630 conn(0x7fba600fc7d0 msgr2=0x7fba600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:24.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.809+0000 7fba6843d640 1 -- 172.21.15.120:0/1981549630 shutdown_connections 2024-09-19T09:18:24.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.809+0000 7fba6843d640 1 -- 172.21.15.120:0/1981549630 wait complete. 2024-09-19T09:18:24.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.809+0000 7fba6843d640 1 Processor -- start 2024-09-19T09:18:24.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.809+0000 7fba6843d640 1 -- start start 2024-09-19T09:18:24.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.810+0000 7fba6843d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba60100e20 0x7fba60199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:24.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.810+0000 7fba6843d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba60102570 0x7fba60199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:24.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.810+0000 7fba6843d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba60102ef0 0x7fba601a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:24.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.810+0000 7fba6843d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fba60110fa0 con 0x7fba60100e20 2024-09-19T09:18:24.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.810+0000 7fba6843d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fba60110e20 con 0x7fba60102ef0 2024-09-19T09:18:24.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.810+0000 7fba6843d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fba60111120 con 0x7fba60102570 2024-09-19T09:18:24.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.810+0000 7fba661d9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba60100e20 0x7fba60199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:24.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.810+0000 7fba661d9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba60100e20 0x7fba60199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46606/0 (socket says 172.21.15.120:46606) 2024-09-19T09:18:24.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.810+0000 7fba661d9640 1 -- 172.21.15.120:0/2764926112 learned_addr learned my addr 172.21.15.120:0/2764926112 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:24.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.810+0000 7fba659d8640 1 --2- 172.21.15.120:0/2764926112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba60102570 0x7fba60199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:24.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.810+0000 7fba669da640 1 --2- 172.21.15.120:0/2764926112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba60102ef0 0x7fba601a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:24.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.810+0000 7fba661d9640 1 -- 172.21.15.120:0/2764926112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba60102570 msgr2=0x7fba60199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:24.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.810+0000 7fba661d9640 1 --2- 172.21.15.120:0/2764926112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba60102570 0x7fba60199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:24.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.810+0000 7fba661d9640 1 -- 172.21.15.120:0/2764926112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba60102ef0 msgr2=0x7fba601a44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:24.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.810+0000 7fba661d9640 1 --2- 172.21.15.120:0/2764926112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba60102ef0 0x7fba601a44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:24.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.810+0000 7fba661d9640 1 -- 172.21.15.120:0/2764926112 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba601a49f0 con 0x7fba60100e20 2024-09-19T09:18:24.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.810+0000 7fba661d9640 1 --2- 172.21.15.120:0/2764926112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba60100e20 0x7fba60199430 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fba58006010 tx=0x7fba58003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:24.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.810+0000 7fba5f7fe640 1 -- 172.21.15.120:0/2764926112 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5803f070 con 0x7fba60100e20 2024-09-19T09:18:24.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.810+0000 7fba5f7fe640 1 -- 172.21.15.120:0/2764926112 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba58031c30 con 0x7fba60100e20 2024-09-19T09:18:24.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.810+0000 7fba6843d640 1 -- 172.21.15.120:0/2764926112 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba601a4bc0 con 0x7fba60100e20 2024-09-19T09:18:24.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.810+0000 7fba5f7fe640 1 -- 172.21.15.120:0/2764926112 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba58035030 con 0x7fba60100e20 2024-09-19T09:18:24.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.810+0000 7fba6843d640 1 -- 172.21.15.120:0/2764926112 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba601a5070 con 0x7fba60100e20 2024-09-19T09:18:24.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.811+0000 7fba6843d640 1 -- 172.21.15.120:0/2764926112 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba60101200 con 0x7fba60100e20 2024-09-19T09:18:24.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.812+0000 7fba5f7fe640 1 -- 172.21.15.120:0/2764926112 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fba58042070 con 0x7fba60100e20 2024-09-19T09:18:24.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.813+0000 7fba5f7fe640 1 --2- 172.21.15.120:0/2764926112 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fba480a32c0 0x7fba480a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:24.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.813+0000 7fba659d8640 1 --2- 172.21.15.120:0/2764926112 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fba480a32c0 0x7fba480a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:24.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.813+0000 7fba5f7fe640 1 -- 172.21.15.120:0/2764926112 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fba580f04c0 con 0x7fba60100e20 2024-09-19T09:18:24.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.813+0000 7fba659d8640 1 --2- 172.21.15.120:0/2764926112 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fba480a32c0 0x7fba480a5780 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fba6006aac0 tx=0x7fba54009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:24.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:24.816+0000 7fba5f7fe640 1 -- 172.21.15.120:0/2764926112 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fba580bd1e0 con 0x7fba60100e20 2024-09-19T09:18:25.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:25.018+0000 7fba6843d640 1 -- 172.21.15.120:0/2764926112 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba60109090 con 0x7fba60100e20 2024-09-19T09:18:25.019 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:24 smithi120 ceph-mon[31150]: pgmap v811: 97 pgs: 97 active+clean; 283 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 4.9 MiB/s wr, 297 op/s 2024-09-19T09:18:25.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:25.018+0000 7fba5f7fe640 1 -- 172.21.15.120:0/2764926112 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11993 (secure 0 0 0) 0x7fba58031de0 con 0x7fba60100e20 2024-09-19T09:18:25.020 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:18:25.020 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":36,"btime":"2024-09-19T09:16:33:442503+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:16:33.001618+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:18:25.020 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 36 2024-09-19T09:18:25.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:25.021+0000 7fba6843d640 1 -- 172.21.15.120:0/2764926112 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fba480a32c0 msgr2=0x7fba480a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:25.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:25.021+0000 7fba6843d640 1 --2- 172.21.15.120:0/2764926112 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fba480a32c0 0x7fba480a5780 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fba6006aac0 tx=0x7fba54009110 comp rx=0 tx=0).stop 2024-09-19T09:18:25.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:25.021+0000 7fba6843d640 1 -- 172.21.15.120:0/2764926112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba60100e20 msgr2=0x7fba60199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:25.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:25.021+0000 7fba6843d640 1 --2- 172.21.15.120:0/2764926112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba60100e20 0x7fba60199430 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fba58006010 tx=0x7fba58003c10 comp rx=0 tx=0).stop 2024-09-19T09:18:25.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:25.021+0000 7fba6843d640 1 -- 172.21.15.120:0/2764926112 shutdown_connections 2024-09-19T09:18:25.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:25.021+0000 7fba6843d640 1 --2- 172.21.15.120:0/2764926112 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fba480a32c0 0x7fba480a5780 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:25.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:25.021+0000 7fba6843d640 1 --2- 172.21.15.120:0/2764926112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba60102ef0 0x7fba601a44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:25.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:25.021+0000 7fba6843d640 1 --2- 172.21.15.120:0/2764926112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba60102570 0x7fba60199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:25.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:25.021+0000 7fba6843d640 1 --2- 172.21.15.120:0/2764926112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba60100e20 0x7fba60199430 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:25.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:25.021+0000 7fba6843d640 1 -- 172.21.15.120:0/2764926112 >> 172.21.15.120:0/2764926112 conn(0x7fba600fc7d0 msgr2=0x7fba600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:25.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:25.021+0000 7fba6843d640 1 -- 172.21.15.120:0/2764926112 shutdown_connections 2024-09-19T09:18:25.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:25.021+0000 7fba6843d640 1 -- 172.21.15.120:0/2764926112 wait complete. 2024-09-19T09:18:25.358 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55751 0.00 MB/sec execute 20 sec latency 27482.305 ms 2024-09-19T09:18:25.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:24 smithi196 ceph-mon[35150]: pgmap v811: 97 pgs: 97 active+clean; 283 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 4.9 MiB/s wr, 297 op/s 2024-09-19T09:18:25.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:24 smithi153 ceph-mon[35071]: pgmap v811: 97 pgs: 97 active+clean; 283 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 4.9 MiB/s wr, 297 op/s 2024-09-19T09:18:25.815 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db --set-id cc768a2c --release --await' 2024-09-19T09:18:25.815 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db --set-id cc768a2c --release --await 2024-09-19T09:18:26.060 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:18:26.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:25 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2764926112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:18:26.358 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55751 0.00 MB/sec execute 21 sec latency 28482.327 ms 2024-09-19T09:18:26.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:25 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2764926112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:18:26.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:25 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2764926112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:18:26.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.541+0000 7f77d55ad640 1 -- 172.21.15.120:0/586656193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77d0102ef0 msgr2=0x7f77d010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:26.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.541+0000 7f77d55ad640 1 --2- 172.21.15.120:0/586656193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77d0102ef0 0x7f77d010da40 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009ca0 tx=0x7f77c802a950 comp rx=0 tx=0).stop 2024-09-19T09:18:26.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.541+0000 7f77d55ad640 1 -- 172.21.15.120:0/586656193 shutdown_connections 2024-09-19T09:18:26.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.541+0000 7f77d55ad640 1 --2- 172.21.15.120:0/586656193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77d0102ef0 0x7f77d010da40 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.541+0000 7f77d55ad640 1 --2- 172.21.15.120:0/586656193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77d0102570 0x7f77d01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.541+0000 7f77d55ad640 1 --2- 172.21.15.120:0/586656193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77d0100e20 0x7f77d0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.541+0000 7f77d55ad640 1 -- 172.21.15.120:0/586656193 >> 172.21.15.120:0/586656193 conn(0x7f77d00fc7d0 msgr2=0x7f77d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:26.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.541+0000 7f77d55ad640 1 -- 172.21.15.120:0/586656193 shutdown_connections 2024-09-19T09:18:26.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.541+0000 7f77d55ad640 1 -- 172.21.15.120:0/586656193 wait complete. 2024-09-19T09:18:26.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.542+0000 7f77d55ad640 1 Processor -- start 2024-09-19T09:18:26.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.542+0000 7f77d55ad640 1 -- start start 2024-09-19T09:18:26.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.542+0000 7f77d55ad640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77d0100e20 0x7f77d01992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:26.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.542+0000 7f77d55ad640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77d0102570 0x7f77d0199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:26.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.542+0000 7f77d55ad640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77d0102ef0 0x7f77d01a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:26.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.542+0000 7f77d55ad640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f77d0110f80 con 0x7f77d0102570 2024-09-19T09:18:26.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.542+0000 7f77d55ad640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f77d0110e00 con 0x7f77d0100e20 2024-09-19T09:18:26.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.542+0000 7f77d55ad640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f77d0111100 con 0x7f77d0102ef0 2024-09-19T09:18:26.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.543+0000 7f77ce7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77d0102570 0x7f77d0199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:26.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.543+0000 7f77ceffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77d0100e20 0x7f77d01992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:26.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.543+0000 7f77ceffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77d0100e20 0x7f77d01992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:42674/0 (socket says 172.21.15.120:42674) 2024-09-19T09:18:26.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.543+0000 7f77ceffd640 1 -- 172.21.15.120:0/2043223295 learned_addr learned my addr 172.21.15.120:0/2043223295 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:26.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.543+0000 7f77cf7fe640 1 --2- 172.21.15.120:0/2043223295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77d0102ef0 0x7f77d01a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:26.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.543+0000 7f77ceffd640 1 -- 172.21.15.120:0/2043223295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77d0102ef0 msgr2=0x7f77d01a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:26.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.543+0000 7f77ceffd640 1 --2- 172.21.15.120:0/2043223295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77d0102ef0 0x7f77d01a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.543+0000 7f77ceffd640 1 -- 172.21.15.120:0/2043223295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77d0102570 msgr2=0x7f77d0199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:26.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.543+0000 7f77ceffd640 1 --2- 172.21.15.120:0/2043223295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77d0102570 0x7f77d0199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.543+0000 7f77ceffd640 1 -- 172.21.15.120:0/2043223295 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d01a48b0 con 0x7f77d0100e20 2024-09-19T09:18:26.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.543+0000 7f77cf7fe640 1 --2- 172.21.15.120:0/2043223295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77d0102ef0 0x7f77d01a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:18:26.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.543+0000 7f77ceffd640 1 --2- 172.21.15.120:0/2043223295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77d0100e20 0x7f77d01992f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f77c000e9f0 tx=0x7f77c000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:26.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.543+0000 7f77b7fff640 1 -- 172.21.15.120:0/2043223295 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c000cd80 con 0x7f77d0100e20 2024-09-19T09:18:26.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.543+0000 7f77d55ad640 1 -- 172.21.15.120:0/2043223295 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d01a4a80 con 0x7f77d0100e20 2024-09-19T09:18:26.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.543+0000 7f77b7fff640 1 -- 172.21.15.120:0/2043223295 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77c0005500 con 0x7f77d0100e20 2024-09-19T09:18:26.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.543+0000 7f77b7fff640 1 -- 172.21.15.120:0/2043223295 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0010690 con 0x7f77d0100e20 2024-09-19T09:18:26.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.544+0000 7f77d55ad640 1 -- 172.21.15.120:0/2043223295 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d01a4f90 con 0x7f77d0100e20 2024-09-19T09:18:26.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.546+0000 7f77b7fff640 1 -- 172.21.15.120:0/2043223295 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f77c00046c0 con 0x7f77d0100e20 2024-09-19T09:18:26.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.546+0000 7f77d7986740 1 Processor -- start 2024-09-19T09:18:26.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.546+0000 7f77d7986740 1 -- start start 2024-09-19T09:18:26.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.547+0000 7f77d7986740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55632cfa2f80 0x55632cfa3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:26.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.547+0000 7f77d7986740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55632cfa38a0 0x55632cfb2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:26.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.547+0000 7f77d7986740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55632cfb2650 0x55632cfb4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:26.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.547+0000 7f77d7986740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55632cf08620 con 0x55632cfa38a0 2024-09-19T09:18:26.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.547+0000 7f77d7986740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55632ceafe50 con 0x55632cfa2f80 2024-09-19T09:18:26.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.547+0000 7f77d7986740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55632cf045e0 con 0x55632cfb2650 2024-09-19T09:18:26.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.547+0000 7f77ceffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55632cfa38a0 0x55632cfb2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:26.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.547+0000 7f77ceffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55632cfa38a0 0x55632cfb2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46626/0 (socket says 172.21.15.120:46626) 2024-09-19T09:18:26.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.547+0000 7f77ceffd640 1 -- 172.21.15.120:0/3078673561 learned_addr learned my addr 172.21.15.120:0/3078673561 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:26.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.547+0000 7f77ce7fc640 1 --2- 172.21.15.120:0/3078673561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55632cfb2650 0x55632cfb4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:26.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.547+0000 7f77cf7fe640 1 --2- 172.21.15.120:0/3078673561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55632cfa2f80 0x55632cfa3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:26.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.547+0000 7f77b7fff640 1 --2- 172.21.15.120:0/2043223295 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f77b80a3240 0x7f77b80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:26.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.547+0000 7f77cf7fe640 1 --2- 172.21.15.120:0/2043223295 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f77b80a3240 0x7f77b80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:26.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.548+0000 7f77cf7fe640 1 -- 172.21.15.120:0/3078673561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55632cfb2650 msgr2=0x55632cfb4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:26.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.548+0000 7f77cf7fe640 1 --2- 172.21.15.120:0/3078673561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55632cfb2650 0x55632cfb4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.548+0000 7f77cf7fe640 1 -- 172.21.15.120:0/3078673561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55632cfa38a0 msgr2=0x55632cfb2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:26.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.548+0000 7f77cf7fe640 1 --2- 172.21.15.120:0/3078673561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55632cfa38a0 0x55632cfb2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.548+0000 7f77cf7fe640 1 -- 172.21.15.120:0/3078673561 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55632cea0d40 con 0x55632cfa2f80 2024-09-19T09:18:26.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.548+0000 7f77ceffd640 1 --2- 172.21.15.120:0/3078673561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55632cfa38a0 0x55632cfb2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:18:26.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.548+0000 7f77cf7fe640 1 --2- 172.21.15.120:0/3078673561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55632cfa2f80 0x55632cfa3360 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f77c8005cc0 tx=0x7f77c80099a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdf348cc8697145b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:26.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.549+0000 7f77b7fff640 1 -- 172.21.15.120:0/2043223295 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f77c00cfce0 con 0x7f77d0100e20 2024-09-19T09:18:26.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.549+0000 7f77cf7fe640 1 --2- 172.21.15.120:0/2043223295 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f77b80a3240 0x7f77b80a5700 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f77c8005e30 tx=0x7f77c805d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:26.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.549+0000 7f77b57fa640 1 -- 172.21.15.120:0/3078673561 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8043050 con 0x55632cfa2f80 2024-09-19T09:18:26.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.549+0000 7f77b57fa640 1 -- 172.21.15.120:0/3078673561 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77c8044040 con 0x55632cfa2f80 2024-09-19T09:18:26.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.549+0000 7f77b57fa640 1 -- 172.21.15.120:0/3078673561 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c805e020 con 0x55632cfa2f80 2024-09-19T09:18:26.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.550+0000 7f77d7986740 1 -- 172.21.15.120:0/3078673561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55632cfa2f80 msgr2=0x55632cfa3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:26.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.550+0000 7f77d7986740 1 --2- 172.21.15.120:0/3078673561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55632cfa2f80 0x55632cfa3360 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f77c8005cc0 tx=0x7f77c80099a0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.550+0000 7f77d7986740 1 -- 172.21.15.120:0/3078673561 shutdown_connections 2024-09-19T09:18:26.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.550+0000 7f77d7986740 1 --2- 172.21.15.120:0/3078673561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55632cfb2650 0x55632cfb4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.550+0000 7f77d7986740 1 --2- 172.21.15.120:0/3078673561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55632cfa38a0 0x55632cfb2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.550+0000 7f77d7986740 1 --2- 172.21.15.120:0/3078673561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55632cfa2f80 0x55632cfa3360 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.550+0000 7f77d7986740 1 -- 172.21.15.120:0/3078673561 >> 172.21.15.120:0/3078673561 conn(0x55632cf9f5f0 msgr2=0x55632cf9f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:26.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.550+0000 7f77d7986740 1 -- 172.21.15.120:0/3078673561 shutdown_connections 2024-09-19T09:18:26.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.550+0000 7f77d7986740 1 -- 172.21.15.120:0/3078673561 wait complete. 2024-09-19T09:18:26.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.550+0000 7f77d7986740 1 -- start start 2024-09-19T09:18:26.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.551+0000 7f77d7986740 1 Processor -- start 2024-09-19T09:18:26.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.551+0000 7f77d7986740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55632cfa2f80 0x55632cfb1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:26.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.551+0000 7f77d7986740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55632cfa38a0 0x55632cfb2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:26.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.551+0000 7f77d7986740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55632cfb2650 0x55632cfae090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:26.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.551+0000 7f77d7986740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55632cfa5ee0 con 0x55632cfb2650 2024-09-19T09:18:26.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.551+0000 7f77d7986740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55632ce90f10 con 0x55632cfa38a0 2024-09-19T09:18:26.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.551+0000 7f77d7986740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55632cfa6060 con 0x55632cfa2f80 2024-09-19T09:18:26.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.551+0000 7f77d7986740 10 client.0 _ll_register_callbacks cb 0x55632ceb4db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:26.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.551+0000 7f77ceffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55632cfa2f80 0x55632cfb1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:26.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.551+0000 7f77ceffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55632cfa2f80 0x55632cfb1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:47150/0 (socket says 172.21.15.120:47150) 2024-09-19T09:18:26.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.551+0000 7f77ceffd640 1 -- 172.21.15.120:0/3561935460 learned_addr learned my addr 172.21.15.120:0/3561935460 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:26.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.551+0000 7f77cf7fe640 1 --2- 172.21.15.120:0/3561935460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55632cfb2650 0x55632cfae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:26.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.551+0000 7f77ce7fc640 1 --2- 172.21.15.120:0/3561935460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55632cfa38a0 0x55632cfb2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:26.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.552+0000 7f77cf7fe640 1 -- 172.21.15.120:0/3561935460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55632cfa2f80 msgr2=0x55632cfb1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:26.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.552+0000 7f77cf7fe640 1 --2- 172.21.15.120:0/3561935460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55632cfa2f80 0x55632cfb1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.552+0000 7f77cf7fe640 1 -- 172.21.15.120:0/3561935460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55632cfa38a0 msgr2=0x55632cfb2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:26.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.552+0000 7f77cf7fe640 1 --2- 172.21.15.120:0/3561935460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55632cfa38a0 0x55632cfb2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.552+0000 7f77cf7fe640 1 -- 172.21.15.120:0/3561935460 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55632cc17580 con 0x55632cfb2650 2024-09-19T09:18:26.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.552+0000 7f77cf7fe640 1 --2- 172.21.15.120:0/3561935460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55632cfb2650 0x55632cfae090 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f77c802ae70 tx=0x7f77c805aee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e148a04b490d4f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:26.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.552+0000 7f77b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:18:26.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.552+0000 7f77b2ff5640 1 -- 172.21.15.120:0/3561935460 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c805e070 con 0x55632cfb2650 2024-09-19T09:18:26.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.552+0000 7f77b2ff5640 1 -- 172.21.15.120:0/3561935460 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f77c8004040 con 0x55632cfb2650 2024-09-19T09:18:26.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.552+0000 7f77d7986740 1 -- 172.21.15.120:0/3561935460 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77c8044040 con 0x55632cfb2650 2024-09-19T09:18:26.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.553+0000 7f77b2ff5640 1 -- 172.21.15.120:0/3561935460 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f77c8043040 con 0x55632cfb2650 2024-09-19T09:18:26.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.553+0000 7f77d7986740 10 client.20772 fetch_fsmap learned FSMap version 36 2024-09-19T09:18:26.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.553+0000 7f77d7986740 1 -- 172.21.15.120:0/3561935460 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55632cce1960 con 0x55632cfb2650 2024-09-19T09:18:26.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.553+0000 7f77b2ff5640 1 -- 172.21.15.120:0/3561935460 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77c8044040 con 0x55632cfb2650 2024-09-19T09:18:26.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.553+0000 7f77b2ff5640 1 -- 172.21.15.120:0/3561935460 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f77c80304b0 con 0x55632cfb2650 2024-09-19T09:18:26.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.553+0000 7f77d7986740 10 client.20772 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:18:26.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.553+0000 7f77d7986740 10 client.20772 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:18:26.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.553+0000 7f77d7986740 20 client.20772 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:26.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.553+0000 7f77d7986740 1 --2- 172.21.15.120:0/3561935460 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55632cfbdd50 0x55632cfc0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:26.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.553+0000 7f77d7986740 4 client.20772 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:18:26.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.553+0000 7f77d7986740 1 -- 172.21.15.120:0/3561935460 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55632cf15b90 con 0x55632cfbdd50 2024-09-19T09:18:26.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.553+0000 7f77ceffd640 1 --2- 172.21.15.120:0/3561935460 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55632cfbdd50 0x55632cfc0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:26.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.554+0000 7f77ceffd640 1 --2- 172.21.15.120:0/3561935460 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55632cfbdd50 0x55632cfc0140 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7a5b5dcc1fa26fc server_cookie=93c44e04aaad3dea in_seq=0 out_seq=0 2024-09-19T09:18:26.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.554+0000 7f77b2ff5640 10 client.20772 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:26.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.555+0000 7f77b2ff5640 1 -- 172.21.15.120:0/3561935460 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55632cf15b90 con 0x55632cfbdd50 2024-09-19T09:18:26.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.555+0000 7f77b2ff5640 10 client.20772 handle_command_reply: tid=0 2024-09-19T09:18:26.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.555+0000 7f77d7986740 1 client.20772 shutdown 2024-09-19T09:18:26.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.555+0000 7f77d7986740 1 -- 172.21.15.120:0/3561935460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55632cfb2650 msgr2=0x55632cfae090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:26.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.555+0000 7f77d7986740 1 --2- 172.21.15.120:0/3561935460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55632cfb2650 0x55632cfae090 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f77c802ae70 tx=0x7f77c805aee0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.555+0000 7f77d7986740 1 -- 172.21.15.120:0/3561935460 shutdown_connections 2024-09-19T09:18:26.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.555+0000 7f77d7986740 1 --2- 172.21.15.120:0/3561935460 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55632cfbdd50 0x55632cfc0140 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.555+0000 7f77d7986740 1 --2- 172.21.15.120:0/3561935460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55632cfb2650 0x55632cfae090 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.555+0000 7f77d7986740 1 --2- 172.21.15.120:0/3561935460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55632cfa38a0 0x55632cfb2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.555+0000 7f77d7986740 1 --2- 172.21.15.120:0/3561935460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55632cfa2f80 0x55632cfb1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.555+0000 7f77b2ff5640 0 client.20772 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:26.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.555+0000 7f77d7986740 1 -- 172.21.15.120:0/3561935460 >> 172.21.15.120:0/3561935460 conn(0x55632cf9f5f0 msgr2=0x55632cf9e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:26.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.555+0000 7f77d7986740 1 -- 172.21.15.120:0/3561935460 shutdown_connections 2024-09-19T09:18:26.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.555+0000 7f77d7986740 1 -- 172.21.15.120:0/3561935460 wait complete. 2024-09-19T09:18:26.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.555+0000 7f77d7986740 20 client.20772 trim_cache size 0 max 16384 2024-09-19T09:18:26.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.572+0000 7f77d7986740 1 Processor -- start 2024-09-19T09:18:26.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.572+0000 7f77d7986740 1 -- start start 2024-09-19T09:18:26.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.572+0000 7f77d7986740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55632cfa2f80 0x55632cf9ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:26.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.572+0000 7f77d7986740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55632cfa38a0 0x55632cfb7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:26.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.572+0000 7f77d7986740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55632cfb2650 0x55632cfb7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:26.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.572+0000 7f77d7986740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01ab3b0 con 0x55632cfb2650 2024-09-19T09:18:26.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.572+0000 7f77d7986740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01acf80 con 0x55632cfa38a0 2024-09-19T09:18:26.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.572+0000 7f77d7986740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01aba80 con 0x55632cfa2f80 2024-09-19T09:18:26.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.572+0000 7f77cf7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55632cfb2650 0x55632cfb7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:26.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.573+0000 7f77cf7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55632cfb2650 0x55632cfb7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46646/0 (socket says 172.21.15.120:46646) 2024-09-19T09:18:26.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.573+0000 7f77cf7fe640 1 -- 172.21.15.120:0/1292734980 learned_addr learned my addr 172.21.15.120:0/1292734980 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:26.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.573+0000 7f77ceffd640 1 --2- 172.21.15.120:0/1292734980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55632cfa2f80 0x55632cf9ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:26.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.573+0000 7f77ce7fc640 1 --2- 172.21.15.120:0/1292734980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55632cfa38a0 0x55632cfb7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:26.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.573+0000 7f77cf7fe640 1 -- 172.21.15.120:0/1292734980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55632cfa2f80 msgr2=0x55632cf9ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:26.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.573+0000 7f77cf7fe640 1 --2- 172.21.15.120:0/1292734980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55632cfa2f80 0x55632cf9ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.573+0000 7f77cf7fe640 1 -- 172.21.15.120:0/1292734980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55632cfa38a0 msgr2=0x55632cfb7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:26.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.573+0000 7f77cf7fe640 1 --2- 172.21.15.120:0/1292734980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55632cfa38a0 0x55632cfb7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.573+0000 7f77cf7fe640 1 -- 172.21.15.120:0/1292734980 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55632cfa44f0 con 0x55632cfb2650 2024-09-19T09:18:26.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.573+0000 7f77ce7fc640 1 --2- 172.21.15.120:0/1292734980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55632cfa38a0 0x55632cfb7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:18:26.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.573+0000 7f77ceffd640 1 --2- 172.21.15.120:0/1292734980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55632cfa2f80 0x55632cf9ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:18:26.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.573+0000 7f77cf7fe640 1 --2- 172.21.15.120:0/1292734980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55632cfb2650 0x55632cfb7fb0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f77c8002230 tx=0x7f77c805a4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59242c354741e73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:26.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.573+0000 7f77b4ff9640 1 -- 172.21.15.120:0/1292734980 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c805e040 con 0x55632cfb2650 2024-09-19T09:18:26.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.573+0000 7f77b4ff9640 1 -- 172.21.15.120:0/1292734980 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77c8009820 con 0x55632cfb2650 2024-09-19T09:18:26.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.573+0000 7f77b4ff9640 1 -- 172.21.15.120:0/1292734980 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8034040 con 0x55632cfb2650 2024-09-19T09:18:26.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.574+0000 7f77d7986740 1 -- 172.21.15.120:0/1292734980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55632cfb2650 msgr2=0x55632cfb7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:26.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.574+0000 7f77d7986740 1 --2- 172.21.15.120:0/1292734980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55632cfb2650 0x55632cfb7fb0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f77c8002230 tx=0x7f77c805a4f0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.574+0000 7f77d7986740 1 -- 172.21.15.120:0/1292734980 shutdown_connections 2024-09-19T09:18:26.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.574+0000 7f77d7986740 1 --2- 172.21.15.120:0/1292734980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55632cfb2650 0x55632cfb7fb0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.574+0000 7f77d7986740 1 --2- 172.21.15.120:0/1292734980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55632cfa38a0 0x55632cfb7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.574+0000 7f77d7986740 1 --2- 172.21.15.120:0/1292734980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55632cfa2f80 0x55632cf9ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.574+0000 7f77d7986740 1 -- 172.21.15.120:0/1292734980 >> 172.21.15.120:0/1292734980 conn(0x55632cf9f5f0 msgr2=0x55632cfa5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:26.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.574+0000 7f77d7986740 1 -- 172.21.15.120:0/1292734980 shutdown_connections 2024-09-19T09:18:26.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.574+0000 7f77d7986740 1 -- 172.21.15.120:0/1292734980 wait complete. 2024-09-19T09:18:26.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.574+0000 7f77d7986740 1 -- start start 2024-09-19T09:18:26.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.574+0000 7f77d7986740 1 Processor -- start 2024-09-19T09:18:26.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.574+0000 7f77d7986740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55632cfa2f80 0x55632cfb7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:26.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.574+0000 7f77d7986740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55632cfa38a0 0x55632cfb8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:26.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.574+0000 7f77d7986740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55632cfb2650 0x55632cfb6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:26.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.574+0000 7f77d7986740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01ac8b0 con 0x55632cfa2f80 2024-09-19T09:18:26.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.574+0000 7f77d7986740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01ac150 con 0x55632cfa38a0 2024-09-19T09:18:26.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.575+0000 7f77d7986740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01a9370 con 0x55632cfb2650 2024-09-19T09:18:26.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.575+0000 7f77d7986740 10 client.0 _ll_register_callbacks cb 0x7f77c8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:26.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.575+0000 7f77ceffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55632cfa2f80 0x55632cfb7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:26.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.575+0000 7f77ceffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55632cfa2f80 0x55632cfb7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46654/0 (socket says 172.21.15.120:46654) 2024-09-19T09:18:26.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.575+0000 7f77ceffd640 1 -- 172.21.15.120:0/3311544134 learned_addr learned my addr 172.21.15.120:0/3311544134 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:26.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.575+0000 7f77ce7fc640 1 --2- 172.21.15.120:0/3311544134 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55632cfa38a0 0x55632cfb8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:26.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.575+0000 7f77cf7fe640 1 --2- 172.21.15.120:0/3311544134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55632cfb2650 0x55632cfb6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:26.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.575+0000 7f77ceffd640 1 -- 172.21.15.120:0/3311544134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55632cfb2650 msgr2=0x55632cfb6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:26.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.575+0000 7f77ceffd640 1 --2- 172.21.15.120:0/3311544134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55632cfb2650 0x55632cfb6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.575+0000 7f77ceffd640 1 -- 172.21.15.120:0/3311544134 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55632cfa38a0 msgr2=0x55632cfb8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:26.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.575+0000 7f77ceffd640 1 --2- 172.21.15.120:0/3311544134 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55632cfa38a0 0x55632cfb8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.575+0000 7f77ceffd640 1 -- 172.21.15.120:0/3311544134 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55632cfb6c70 con 0x55632cfa2f80 2024-09-19T09:18:26.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.575+0000 7f77ceffd640 1 --2- 172.21.15.120:0/3311544134 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55632cfa2f80 0x55632cfb7e20 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f77c0098cd0 tx=0x7f77c0098480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45d3407f4947c39 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:26.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.575+0000 7f77b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:18:26.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.575+0000 7f77b2ff5640 1 -- 172.21.15.120:0/3311544134 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c009bc60 con 0x55632cfa2f80 2024-09-19T09:18:26.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.575+0000 7f77b2ff5640 1 -- 172.21.15.120:0/3311544134 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f77c00b5190 con 0x55632cfa2f80 2024-09-19T09:18:26.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.575+0000 7f77d7986740 1 -- 172.21.15.120:0/3311544134 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77c8009820 con 0x55632cfa2f80 2024-09-19T09:18:26.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.575+0000 7f77b2ff5640 1 -- 172.21.15.120:0/3311544134 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f77c009ca00 con 0x55632cfa2f80 2024-09-19T09:18:26.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.575+0000 7f77d7986740 10 client.20778 fetch_fsmap learned FSMap version 36 2024-09-19T09:18:26.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.575+0000 7f77d7986740 1 -- 172.21.15.120:0/3311544134 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55632d0294d0 con 0x55632cfa2f80 2024-09-19T09:18:26.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.576+0000 7f77b2ff5640 1 -- 172.21.15.120:0/3311544134 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77c8009820 con 0x55632cfa2f80 2024-09-19T09:18:26.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.576+0000 7f77b2ff5640 1 -- 172.21.15.120:0/3311544134 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f77c0098970 con 0x55632cfa2f80 2024-09-19T09:18:26.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.576+0000 7f77d7986740 10 client.20778 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:18:26.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.576+0000 7f77d7986740 10 client.20778 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:18:26.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.576+0000 7f77d7986740 20 client.20778 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:26.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.576+0000 7f77d7986740 1 --2- 172.21.15.120:0/3311544134 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55632d029c90 0x55632d02c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:26.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.576+0000 7f77d7986740 4 client.20778 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "cc768a2c", "release": true, "await": true}] 2024-09-19T09:18:26.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.576+0000 7f77d7986740 1 -- 172.21.15.120:0/3311544134 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "set_id": "cc768a2c", "release": true, "await": true}) -- 0x55632cf100a0 con 0x55632d029c90 2024-09-19T09:18:26.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.576+0000 7f77ce7fc640 1 --2- 172.21.15.120:0/3311544134 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55632d029c90 0x55632d02c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:26.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.576+0000 7f77b2ff5640 10 client.20778 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:26.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.576+0000 7f77ce7fc640 1 --2- 172.21.15.120:0/3311544134 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55632d029c90 0x55632d02c0b0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f421879eba1b5d84 server_cookie=9d32c97bedcb1ea2 in_seq=0 out_seq=0 2024-09-19T09:18:26.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.652+0000 7f77b2ff5640 1 -- 172.21.15.120:0/3311544134 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55632cf100a0 con 0x55632d029c90 2024-09-19T09:18:26.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.652+0000 7f77b2ff5640 10 client.20778 handle_command_reply: tid=0 2024-09-19T09:18:26.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.652+0000 7f77d7986740 1 client.20778 shutdown 2024-09-19T09:18:26.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.652+0000 7f77d7986740 1 -- 172.21.15.120:0/3311544134 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55632cfa2f80 msgr2=0x55632cfb7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:26.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.652+0000 7f77d7986740 1 --2- 172.21.15.120:0/3311544134 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55632cfa2f80 0x55632cfb7e20 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f77c0098cd0 tx=0x7f77c0098480 comp rx=0 tx=0).stop 2024-09-19T09:18:26.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.652+0000 7f77d7986740 1 -- 172.21.15.120:0/3311544134 shutdown_connections 2024-09-19T09:18:26.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.652+0000 7f77d7986740 1 --2- 172.21.15.120:0/3311544134 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55632d029c90 0x55632d02c0b0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.652+0000 7f77d7986740 1 --2- 172.21.15.120:0/3311544134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55632cfb2650 0x55632cfb6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.652+0000 7f77d7986740 1 --2- 172.21.15.120:0/3311544134 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55632cfa38a0 0x55632cfb8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.652+0000 7f77d7986740 1 --2- 172.21.15.120:0/3311544134 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55632cfa2f80 0x55632cfb7e20 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.652+0000 7f77d7986740 1 -- 172.21.15.120:0/3311544134 >> 172.21.15.120:0/3311544134 conn(0x55632cf9f5f0 msgr2=0x55632cf9df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:26.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.652+0000 7f77b2ff5640 0 client.20778 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:26.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.652+0000 7f77d7986740 1 -- 172.21.15.120:0/3311544134 shutdown_connections 2024-09-19T09:18:26.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.652+0000 7f77d7986740 1 -- 172.21.15.120:0/3311544134 wait complete. 2024-09-19T09:18:26.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.652+0000 7f77d7986740 20 client.20778 trim_cache size 0 max 16384 2024-09-19T09:18:26.655 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:18:26.655 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 36, 2024-09-19T09:18:26.655 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:18:26.655 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 52, 2024-09-19T09:18:26.655 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:18:26.655 INFO:teuthology.orchestra.run.smithi120.stdout: "cc768a2c": { 2024-09-19T09:18:26.655 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 52, 2024-09-19T09:18:26.655 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:18:26.655 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:18:26.656 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:18:26.656 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:18:26.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:18:26.656 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:18:26.656 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 149.4, 2024-09-19T09:18:26.656 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:18:26.656 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:18:26.656 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:18:26.656 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:18:26.656 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:18:26.656 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:18:26.656 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:18:26.656 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:18:26.656 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:18:26.656 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:18:26.657 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:18:26.657 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:18:26.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.653+0000 7f77d55ad640 1 -- 172.21.15.120:0/2043223295 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f77b80a3240 msgr2=0x7f77b80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:26.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.653+0000 7f77d55ad640 1 --2- 172.21.15.120:0/2043223295 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f77b80a3240 0x7f77b80a5700 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f77c8005e30 tx=0x7f77c805d000 comp rx=0 tx=0).stop 2024-09-19T09:18:26.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.653+0000 7f77d55ad640 1 -- 172.21.15.120:0/2043223295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77d0100e20 msgr2=0x7f77d01992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:26.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.653+0000 7f77d55ad640 1 --2- 172.21.15.120:0/2043223295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77d0100e20 0x7f77d01992f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f77c000e9f0 tx=0x7f77c000edf0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.653+0000 7f77d55ad640 1 -- 172.21.15.120:0/2043223295 shutdown_connections 2024-09-19T09:18:26.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.653+0000 7f77d55ad640 1 --2- 172.21.15.120:0/2043223295 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f77b80a3240 0x7f77b80a5700 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.653+0000 7f77d55ad640 1 --2- 172.21.15.120:0/2043223295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77d0102ef0 0x7f77d01a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.653+0000 7f77d55ad640 1 --2- 172.21.15.120:0/2043223295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77d0102570 0x7f77d0199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.653+0000 7f77d55ad640 1 --2- 172.21.15.120:0/2043223295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77d0100e20 0x7f77d01992f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:26.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.653+0000 7f77d55ad640 1 -- 172.21.15.120:0/2043223295 >> 172.21.15.120:0/2043223295 conn(0x7f77d00fc7d0 msgr2=0x7f77d0105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:26.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.653+0000 7f77d55ad640 1 -- 172.21.15.120:0/2043223295 shutdown_connections 2024-09-19T09:18:26.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:26.653+0000 7f77d55ad640 1 -- 172.21.15.120:0/2043223295 wait complete. 2024-09-19T09:18:27.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:26 smithi120 ceph-mon[31150]: pgmap v812: 97 pgs: 97 active+clean; 283 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 4.5 MiB/s wr, 291 op/s 2024-09-19T09:18:27.358 INFO:tasks.workunit.client.0.smithi120.stdout: 10 56160 0.85 MB/sec execute 22 sec latency 28710.352 ms 2024-09-19T09:18:27.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:26 smithi196 ceph-mon[35150]: pgmap v812: 97 pgs: 97 active+clean; 283 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 4.5 MiB/s wr, 291 op/s 2024-09-19T09:18:27.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:26 smithi153 ceph-mon[35071]: pgmap v812: 97 pgs: 97 active+clean; 283 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 4.5 MiB/s wr, 291 op/s 2024-09-19T09:18:27.679 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'cc768a2c', total seconds elapsed: 33.1 2024-09-19T09:18:27.679 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.45 2024-09-19T09:18:27.679 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:18:27.971 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:18:28.245 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:27 smithi120 ceph-mon[31150]: pgmap v813: 97 pgs: 97 active+clean; 281 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 5.4 MiB/s wr, 333 op/s 2024-09-19T09:18:28.358 INFO:tasks.workunit.client.0.smithi120.stdout: 10 56670 1.81 MB/sec execute 23 sec latency 90.796 ms 2024-09-19T09:18:28.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:27 smithi196 ceph-mon[35150]: pgmap v813: 97 pgs: 97 active+clean; 281 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 5.4 MiB/s wr, 333 op/s 2024-09-19T09:18:28.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:27 smithi153 ceph-mon[35071]: pgmap v813: 97 pgs: 97 active+clean; 281 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 5.4 MiB/s wr, 333 op/s 2024-09-19T09:18:28.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.511+0000 7f436de6b640 1 -- 172.21.15.120:0/2544709041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43681027c0 msgr2=0x7f4368106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:28.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.511+0000 7f436de6b640 1 --2- 172.21.15.120:0/2544709041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43681027c0 0x7f4368106810 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f4350009900 tx=0x7f435002a3f0 comp rx=0 tx=0).stop 2024-09-19T09:18:28.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.511+0000 7f436de6b640 1 -- 172.21.15.120:0/2544709041 shutdown_connections 2024-09-19T09:18:28.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.511+0000 7f436de6b640 1 --2- 172.21.15.120:0/2544709041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4368106f40 0x7f436810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:28.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.511+0000 7f436de6b640 1 --2- 172.21.15.120:0/2544709041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43681027c0 0x7f4368106810 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:28.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.511+0000 7f436de6b640 1 --2- 172.21.15.120:0/2544709041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4368101e10 0x7f43681021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:28.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.511+0000 7f436de6b640 1 -- 172.21.15.120:0/2544709041 >> 172.21.15.120:0/2544709041 conn(0x7f43680fd8d0 msgr2=0x7f43680ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:28.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.511+0000 7f436de6b640 1 -- 172.21.15.120:0/2544709041 shutdown_connections 2024-09-19T09:18:28.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.511+0000 7f436de6b640 1 -- 172.21.15.120:0/2544709041 wait complete. 2024-09-19T09:18:28.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.512+0000 7f436de6b640 1 Processor -- start 2024-09-19T09:18:28.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.512+0000 7f436de6b640 1 -- start start 2024-09-19T09:18:28.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.512+0000 7f436de6b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4368101e10 0x7f43680744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:28.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.512+0000 7f436de6b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43681027c0 0x7f4368074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:28.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.512+0000 7f436de6b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4368106f40 0x7f4368074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:28.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.512+0000 7f436de6b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4368111f60 con 0x7f4368106f40 2024-09-19T09:18:28.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.512+0000 7f436de6b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4368111de0 con 0x7f4368101e10 2024-09-19T09:18:28.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.512+0000 7f436de6b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43681120e0 con 0x7f43681027c0 2024-09-19T09:18:28.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.512+0000 7f4367fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4368106f40 0x7f4368074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:28.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.512+0000 7f43677fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4368101e10 0x7f43680744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:28.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.512+0000 7f4367fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4368106f40 0x7f4368074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46680/0 (socket says 172.21.15.120:46680) 2024-09-19T09:18:28.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.512+0000 7f4367fff640 1 -- 172.21.15.120:0/657577486 learned_addr learned my addr 172.21.15.120:0/657577486 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:28.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.513+0000 7f435ffff640 1 --2- 172.21.15.120:0/657577486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43681027c0 0x7f4368074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:28.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.513+0000 7f4367fff640 1 -- 172.21.15.120:0/657577486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43681027c0 msgr2=0x7f4368074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:28.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.513+0000 7f4367fff640 1 --2- 172.21.15.120:0/657577486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43681027c0 0x7f4368074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:28.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.513+0000 7f4367fff640 1 -- 172.21.15.120:0/657577486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4368101e10 msgr2=0x7f43680744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:28.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.513+0000 7f4367fff640 1 --2- 172.21.15.120:0/657577486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4368101e10 0x7f43680744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:28.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.513+0000 7f4367fff640 1 -- 172.21.15.120:0/657577486 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43681a6940 con 0x7f4368106f40 2024-09-19T09:18:28.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.513+0000 7f435ffff640 1 --2- 172.21.15.120:0/657577486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43681027c0 0x7f4368074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:18:28.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.513+0000 7f4367fff640 1 --2- 172.21.15.120:0/657577486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4368106f40 0x7f4368074f70 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f4360009fd0 tx=0x7f436000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:28.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.513+0000 7f43657fa640 1 -- 172.21.15.120:0/657577486 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360017070 con 0x7f4368106f40 2024-09-19T09:18:28.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.513+0000 7f43657fa640 1 -- 172.21.15.120:0/657577486 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4360003e50 con 0x7f4368106f40 2024-09-19T09:18:28.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.514+0000 7f43657fa640 1 -- 172.21.15.120:0/657577486 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43600106c0 con 0x7f4368106f40 2024-09-19T09:18:28.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.514+0000 7f436de6b640 1 -- 172.21.15.120:0/657577486 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43681a6c30 con 0x7f4368106f40 2024-09-19T09:18:28.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.514+0000 7f436de6b640 1 -- 172.21.15.120:0/657577486 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43681a70f0 con 0x7f4368106f40 2024-09-19T09:18:28.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.515+0000 7f436de6b640 1 -- 172.21.15.120:0/657577486 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43681032f0 con 0x7f4368106f40 2024-09-19T09:18:28.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.516+0000 7f43657fa640 1 -- 172.21.15.120:0/657577486 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f4360004000 con 0x7f4368106f40 2024-09-19T09:18:28.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.517+0000 7f43657fa640 1 --2- 172.21.15.120:0/657577486 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43540a3110 0x7f43540a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:28.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.517+0000 7f43677fe640 1 --2- 172.21.15.120:0/657577486 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43540a3110 0x7f43540a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:28.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.517+0000 7f43657fa640 1 -- 172.21.15.120:0/657577486 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f43600cf530 con 0x7f4368106f40 2024-09-19T09:18:28.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.517+0000 7f43677fe640 1 --2- 172.21.15.120:0/657577486 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43540a3110 0x7f43540a55d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f4358005d00 tx=0x7f435800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:28.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.521+0000 7f43657fa640 1 -- 172.21.15.120:0/657577486 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4360020030 con 0x7f4368106f40 2024-09-19T09:18:28.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.747+0000 7f436de6b640 1 -- 172.21.15.120:0/657577486 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4368075a20 con 0x7f4368106f40 2024-09-19T09:18:28.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.748+0000 7f43657fa640 1 -- 172.21.15.120:0/657577486 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11993 (secure 0 0 0) 0x7f436009c250 con 0x7f4368106f40 2024-09-19T09:18:28.750 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:18:28.750 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":36,"btime":"2024-09-19T09:16:33:442503+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:16:33.001618+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:18:28.750 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 36 2024-09-19T09:18:28.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.750+0000 7f436de6b640 1 -- 172.21.15.120:0/657577486 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43540a3110 msgr2=0x7f43540a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:28.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.750+0000 7f436de6b640 1 --2- 172.21.15.120:0/657577486 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43540a3110 0x7f43540a55d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f4358005d00 tx=0x7f435800b410 comp rx=0 tx=0).stop 2024-09-19T09:18:28.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.750+0000 7f436de6b640 1 -- 172.21.15.120:0/657577486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4368106f40 msgr2=0x7f4368074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:28.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.750+0000 7f436de6b640 1 --2- 172.21.15.120:0/657577486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4368106f40 0x7f4368074f70 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f4360009fd0 tx=0x7f436000b260 comp rx=0 tx=0).stop 2024-09-19T09:18:28.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.750+0000 7f436de6b640 1 -- 172.21.15.120:0/657577486 shutdown_connections 2024-09-19T09:18:28.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.750+0000 7f436de6b640 1 --2- 172.21.15.120:0/657577486 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43540a3110 0x7f43540a55d0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:28.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.750+0000 7f436de6b640 1 --2- 172.21.15.120:0/657577486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4368106f40 0x7f4368074f70 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:28.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.750+0000 7f436de6b640 1 --2- 172.21.15.120:0/657577486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43681027c0 0x7f4368074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:28.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.750+0000 7f436de6b640 1 --2- 172.21.15.120:0/657577486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4368101e10 0x7f43680744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:28.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.750+0000 7f436de6b640 1 -- 172.21.15.120:0/657577486 >> 172.21.15.120:0/657577486 conn(0x7f43680fd8d0 msgr2=0x7f43680fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:28.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.750+0000 7f436de6b640 1 -- 172.21.15.120:0/657577486 shutdown_connections 2024-09-19T09:18:28.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:28.750+0000 7f436de6b640 1 -- 172.21.15.120:0/657577486 wait complete. 2024-09-19T09:18:29.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:28 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:18:29.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:28 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:18:29.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:28 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:18:29.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:28 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:18:29.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:28 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/657577486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:18:29.358 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57059 2.51 MB/sec execute 24 sec latency 174.858 ms 2024-09-19T09:18:29.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:28 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:18:29.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:28 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:18:29.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:28 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:18:29.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:28 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:18:29.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:28 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/657577486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:18:29.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:28 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:18:29.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:28 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:18:29.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:28 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:18:29.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:28 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:18:29.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:28 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/657577486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:18:29.737 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db / --timeout 90 --expiration 149.45 --await-for 45.0' 2024-09-19T09:18:29.738 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db / --timeout 90 --expiration 149.45 --await-for 45.0 2024-09-19T09:18:30.032 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:18:30.317 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:29 smithi120 ceph-mon[31150]: pgmap v814: 97 pgs: 97 active+clean; 279 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.6 MiB/s wr, 336 op/s 2024-09-19T09:18:30.358 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57740 4.14 MB/sec execute 25 sec latency 53.169 ms 2024-09-19T09:18:30.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:29 smithi196 ceph-mon[35150]: pgmap v814: 97 pgs: 97 active+clean; 279 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.6 MiB/s wr, 336 op/s 2024-09-19T09:18:30.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:29 smithi153 ceph-mon[35071]: pgmap v814: 97 pgs: 97 active+clean; 279 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.6 MiB/s wr, 336 op/s 2024-09-19T09:18:30.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.554+0000 7faccce1e640 1 -- 172.21.15.120:0/2419648507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facc8106a70 msgr2=0x7facc8106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:30.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.554+0000 7faccce1e640 1 --2- 172.21.15.120:0/2419648507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facc8106a70 0x7facc8106e50 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7facb8009930 tx=0x7facb802a420 comp rx=0 tx=0).stop 2024-09-19T09:18:30.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.554+0000 7faccce1e640 1 -- 172.21.15.120:0/2419648507 shutdown_connections 2024-09-19T09:18:30.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.554+0000 7faccce1e640 1 --2- 172.21.15.120:0/2419648507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facc8101410 0x7facc810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:30.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.554+0000 7faccce1e640 1 --2- 172.21.15.120:0/2419648507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facc8100a70 0x7facc8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:30.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.554+0000 7faccce1e640 1 --2- 172.21.15.120:0/2419648507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facc8106a70 0x7facc8106e50 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:30.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.554+0000 7faccce1e640 1 -- 172.21.15.120:0/2419648507 >> 172.21.15.120:0/2419648507 conn(0x7facc80fc790 msgr2=0x7facc80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:30.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.554+0000 7faccce1e640 1 -- 172.21.15.120:0/2419648507 shutdown_connections 2024-09-19T09:18:30.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.555+0000 7faccce1e640 1 -- 172.21.15.120:0/2419648507 wait complete. 2024-09-19T09:18:30.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.555+0000 7faccce1e640 1 Processor -- start 2024-09-19T09:18:30.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.555+0000 7faccce1e640 1 -- start start 2024-09-19T09:18:30.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.555+0000 7faccce1e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facc8100a70 0x7facc81970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:30.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.555+0000 7faccce1e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facc8101410 0x7facc8197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:30.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.556+0000 7faccce1e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facc8106a70 0x7facc8197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:30.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.556+0000 7faccce1e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7facc8110e70 con 0x7facc8106a70 2024-09-19T09:18:30.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.556+0000 7faccce1e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7facc8110cf0 con 0x7facc8101410 2024-09-19T09:18:30.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.556+0000 7faccce1e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7facc8110ff0 con 0x7facc8100a70 2024-09-19T09:18:30.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.556+0000 7facc6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facc8106a70 0x7facc8197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:30.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.556+0000 7facbffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facc8101410 0x7facc8197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:30.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.556+0000 7facc659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facc8100a70 0x7facc81970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:30.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.556+0000 7facbffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facc8101410 0x7facc8197630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:42744/0 (socket says 172.21.15.120:42744) 2024-09-19T09:18:30.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.556+0000 7facc659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facc8100a70 0x7facc81970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:47194/0 (socket says 172.21.15.120:47194) 2024-09-19T09:18:30.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.556+0000 7facbffff640 1 -- 172.21.15.120:0/326399489 learned_addr learned my addr 172.21.15.120:0/326399489 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:30.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.556+0000 7facc6d9d640 1 -- 172.21.15.120:0/326399489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facc8100a70 msgr2=0x7facc81970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:30.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.556+0000 7facc6d9d640 1 --2- 172.21.15.120:0/326399489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facc8100a70 0x7facc81970f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:30.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.556+0000 7facc6d9d640 1 -- 172.21.15.120:0/326399489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facc8101410 msgr2=0x7facc8197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:30.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.556+0000 7facc6d9d640 1 --2- 172.21.15.120:0/326399489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facc8101410 0x7facc8197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:30.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.556+0000 7facc6d9d640 1 -- 172.21.15.120:0/326399489 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc81a6960 con 0x7facc8106a70 2024-09-19T09:18:30.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.556+0000 7facc6d9d640 1 --2- 172.21.15.120:0/326399489 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facc8106a70 0x7facc8197b70 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7facc000e940 tx=0x7facc000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:30.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.556+0000 7facbf7fe640 1 -- 172.21.15.120:0/326399489 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc0004d20 con 0x7facc8106a70 2024-09-19T09:18:30.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.556+0000 7facbf7fe640 1 -- 172.21.15.120:0/326399489 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facc0005020 con 0x7facc8106a70 2024-09-19T09:18:30.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.556+0000 7faccce1e640 1 -- 172.21.15.120:0/326399489 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc81a6c50 con 0x7facc8106a70 2024-09-19T09:18:30.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.556+0000 7facbf7fe640 1 -- 172.21.15.120:0/326399489 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc00116b0 con 0x7facc8106a70 2024-09-19T09:18:30.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.556+0000 7faccce1e640 1 -- 172.21.15.120:0/326399489 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc81a7140 con 0x7facc8106a70 2024-09-19T09:18:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.559+0000 7facbf7fe640 1 -- 172.21.15.120:0/326399489 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7facc00054d0 con 0x7facc8106a70 2024-09-19T09:18:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.559+0000 7faccf1f7740 1 Processor -- start 2024-09-19T09:18:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.559+0000 7faccf1f7740 1 -- start start 2024-09-19T09:18:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.560+0000 7faccf1f7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f875976850 0x55f875976c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.560+0000 7faccf1f7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f875977170 0x55f8759814f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.560+0000 7faccf1f7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f875981a30 0x55f875983e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.560+0000 7faccf1f7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f8758db160 con 0x55f875977170 2024-09-19T09:18:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.560+0000 7faccf1f7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f875882720 con 0x55f875981a30 2024-09-19T09:18:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.560+0000 7faccf1f7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f8758d7120 con 0x55f875976850 2024-09-19T09:18:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.560+0000 7facc659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f875977170 0x55f8759814f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.560+0000 7facc659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f875977170 0x55f8759814f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46704/0 (socket says 172.21.15.120:46704) 2024-09-19T09:18:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.560+0000 7facc659c640 1 -- 172.21.15.120:0/3373533909 learned_addr learned my addr 172.21.15.120:0/3373533909 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.560+0000 7facbffff640 1 --2- 172.21.15.120:0/3373533909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f875976850 0x55f875976c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.560+0000 7facbffff640 1 --2- 172.21.15.120:0/3373533909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f875981a30 0x55f875983e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:30.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.560+0000 7facbf7fe640 1 --2- 172.21.15.120:0/326399489 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7facb40a3240 0x7facb40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:30.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.560+0000 7facc659c640 1 -- 172.21.15.120:0/3373533909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f875976850 msgr2=0x55f875976c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:30.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.560+0000 7facc659c640 1 --2- 172.21.15.120:0/3373533909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f875976850 0x55f875976c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:30.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.560+0000 7facc659c640 1 -- 172.21.15.120:0/3373533909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f875981a30 msgr2=0x55f875983e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:30.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.560+0000 7facc659c640 1 --2- 172.21.15.120:0/3373533909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f875981a30 0x55f875983e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:30.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.560+0000 7facc659c640 1 -- 172.21.15.120:0/3373533909 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f87564cba0 con 0x55f875977170 2024-09-19T09:18:30.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.560+0000 7facbffff640 1 --2- 172.21.15.120:0/3373533909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f875976850 0x55f875976c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:18:30.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.560+0000 7facbf7fe640 1 -- 172.21.15.120:0/326399489 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7facc00cf900 con 0x7facc8106a70 2024-09-19T09:18:30.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.560+0000 7facc6d9d640 1 --2- 172.21.15.120:0/326399489 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7facb40a3240 0x7facb40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:30.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.560+0000 7facc659c640 1 --2- 172.21.15.120:0/3373533909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f875977170 0x55f8759814f0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7facb802a940 tx=0x7facb8005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78cdbbf5b2a8fb45 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:30.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.561+0000 7facbcff9640 1 -- 172.21.15.120:0/3373533909 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8040070 con 0x55f875977170 2024-09-19T09:18:30.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.561+0000 7facbcff9640 1 -- 172.21.15.120:0/3373533909 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facb80318f0 con 0x55f875977170 2024-09-19T09:18:30.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.561+0000 7facbcff9640 1 -- 172.21.15.120:0/3373533909 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8035040 con 0x55f875977170 2024-09-19T09:18:30.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.562+0000 7faccf1f7740 1 -- 172.21.15.120:0/3373533909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f875977170 msgr2=0x55f8759814f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:30.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.562+0000 7facc6d9d640 1 --2- 172.21.15.120:0/326399489 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7facb40a3240 0x7facb40a5700 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7facc0005fd0 tx=0x7facc00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:30.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.562+0000 7faccf1f7740 1 --2- 172.21.15.120:0/3373533909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f875977170 0x55f8759814f0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7facb802a940 tx=0x7facb8005c00 comp rx=0 tx=0).stop 2024-09-19T09:18:30.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.562+0000 7faccf1f7740 1 -- 172.21.15.120:0/3373533909 shutdown_connections 2024-09-19T09:18:30.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.562+0000 7faccf1f7740 1 --2- 172.21.15.120:0/3373533909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f875981a30 0x55f875983e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.562+0000 7faccf1f7740 1 --2- 172.21.15.120:0/3373533909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f875977170 0x55f8759814f0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.562+0000 7faccf1f7740 1 --2- 172.21.15.120:0/3373533909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f875976850 0x55f875976c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.562+0000 7faccf1f7740 1 -- 172.21.15.120:0/3373533909 >> 172.21.15.120:0/3373533909 conn(0x55f875972de0 msgr2=0x55f8759731c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.562+0000 7faccf1f7740 1 -- 172.21.15.120:0/3373533909 shutdown_connections 2024-09-19T09:18:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.562+0000 7faccf1f7740 1 -- 172.21.15.120:0/3373533909 wait complete. 2024-09-19T09:18:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.562+0000 7faccf1f7740 1 -- start start 2024-09-19T09:18:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.562+0000 7faccf1f7740 1 Processor -- start 2024-09-19T09:18:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.563+0000 7faccf1f7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f875976850 0x55f87597e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.563+0000 7faccf1f7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f875977170 0x55f87597ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.563+0000 7faccf1f7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f875981a30 0x55f87597f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.563+0000 7faccf1f7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f875984600 con 0x55f875976850 2024-09-19T09:18:30.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.563+0000 7faccf1f7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f875863900 con 0x55f875981a30 2024-09-19T09:18:30.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.563+0000 7faccf1f7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f875984780 con 0x55f875977170 2024-09-19T09:18:30.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.563+0000 7faccf1f7740 10 client.0 _ll_register_callbacks cb 0x55f875861780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:30.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.563+0000 7facbffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f875976850 0x55f87597e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:30.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.563+0000 7facbffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f875976850 0x55f87597e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46718/0 (socket says 172.21.15.120:46718) 2024-09-19T09:18:30.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.563+0000 7facbffff640 1 -- 172.21.15.120:0/383435441 learned_addr learned my addr 172.21.15.120:0/383435441 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:30.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.563+0000 7facc659c640 1 --2- 172.21.15.120:0/383435441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f875977170 0x55f87597ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:30.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.563+0000 7facbffff640 1 --2- 172.21.15.120:0/383435441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f875981a30 0x55f87597f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:30.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.563+0000 7facbffff640 1 -- 172.21.15.120:0/383435441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f875977170 msgr2=0x55f87597ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:30.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.563+0000 7facbffff640 1 --2- 172.21.15.120:0/383435441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f875977170 0x55f87597ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:30.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.563+0000 7facbffff640 1 -- 172.21.15.120:0/383435441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f875981a30 msgr2=0x55f87597f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:30.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.563+0000 7facbffff640 1 --2- 172.21.15.120:0/383435441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f875981a30 0x55f87597f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:30.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.563+0000 7facbffff640 1 -- 172.21.15.120:0/383435441 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8758a3c50 con 0x55f875976850 2024-09-19T09:18:30.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.563+0000 7facbffff640 1 --2- 172.21.15.120:0/383435441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f875976850 0x55f87597e8e0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7facb00117e0 tx=0x7facb0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fca937f9a35c1a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:30.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.563+0000 7facadffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:18:30.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.563+0000 7facadffb640 1 -- 172.21.15.120:0/383435441 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0010c40 con 0x55f875976850 2024-09-19T09:18:30.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.563+0000 7facadffb640 1 -- 172.21.15.120:0/383435441 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7facb0010de0 con 0x55f875976850 2024-09-19T09:18:30.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.563+0000 7faccf1f7740 1 -- 172.21.15.120:0/383435441 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facb80318f0 con 0x55f875976850 2024-09-19T09:18:30.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.564+0000 7facadffb640 1 -- 172.21.15.120:0/383435441 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7facb0005020 con 0x55f875976850 2024-09-19T09:18:30.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.564+0000 7faccf1f7740 10 client.20796 fetch_fsmap learned FSMap version 36 2024-09-19T09:18:30.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.564+0000 7faccf1f7740 1 -- 172.21.15.120:0/383435441 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f875990340 con 0x55f875976850 2024-09-19T09:18:30.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.564+0000 7facadffb640 1 -- 172.21.15.120:0/383435441 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facb80318f0 con 0x55f875976850 2024-09-19T09:18:30.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.564+0000 7facadffb640 1 -- 172.21.15.120:0/383435441 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7facb0004cc0 con 0x55f875976850 2024-09-19T09:18:30.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.564+0000 7faccf1f7740 10 client.20796 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:18:30.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.564+0000 7faccf1f7740 10 client.20796 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:18:30.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.564+0000 7faccf1f7740 20 client.20796 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:30.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.564+0000 7faccf1f7740 1 --2- 172.21.15.120:0/383435441 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f875991800 0x55f875993c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:30.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.564+0000 7faccf1f7740 4 client.20796 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:18:30.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.564+0000 7faccf1f7740 1 -- 172.21.15.120:0/383435441 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8758e8890 con 0x55f875991800 2024-09-19T09:18:30.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.564+0000 7facc659c640 1 --2- 172.21.15.120:0/383435441 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f875991800 0x55f875993c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:30.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.565+0000 7facc659c640 1 --2- 172.21.15.120:0/383435441 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f875991800 0x55f875993c10 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c312f05c147be72e server_cookie=64fc0117a569071f in_seq=0 out_seq=0 2024-09-19T09:18:30.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.565+0000 7facadffb640 10 client.20796 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:30.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.569+0000 7facadffb640 1 -- 172.21.15.120:0/383435441 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f8758e8890 con 0x55f875991800 2024-09-19T09:18:30.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.569+0000 7facadffb640 10 client.20796 handle_command_reply: tid=0 2024-09-19T09:18:30.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.569+0000 7faccf1f7740 1 client.20796 shutdown 2024-09-19T09:18:30.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.570+0000 7faccf1f7740 1 -- 172.21.15.120:0/383435441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f875976850 msgr2=0x55f87597e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:30.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.570+0000 7faccf1f7740 1 --2- 172.21.15.120:0/383435441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f875976850 0x55f87597e8e0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7facb00117e0 tx=0x7facb0005f90 comp rx=0 tx=0).stop 2024-09-19T09:18:30.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.570+0000 7faccf1f7740 1 -- 172.21.15.120:0/383435441 shutdown_connections 2024-09-19T09:18:30.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.570+0000 7faccf1f7740 1 --2- 172.21.15.120:0/383435441 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f875991800 0x55f875993c10 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:30.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.570+0000 7faccf1f7740 1 --2- 172.21.15.120:0/383435441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f875981a30 0x55f87597f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:30.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.570+0000 7faccf1f7740 1 --2- 172.21.15.120:0/383435441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f875977170 0x55f87597ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:30.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.570+0000 7faccf1f7740 1 --2- 172.21.15.120:0/383435441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f875976850 0x55f87597e8e0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:30.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.570+0000 7faccf1f7740 1 -- 172.21.15.120:0/383435441 >> 172.21.15.120:0/383435441 conn(0x55f875972de0 msgr2=0x55f875978ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:30.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.570+0000 7facadffb640 0 client.20796 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:30.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.570+0000 7faccf1f7740 1 -- 172.21.15.120:0/383435441 shutdown_connections 2024-09-19T09:18:30.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.570+0000 7faccf1f7740 1 -- 172.21.15.120:0/383435441 wait complete. 2024-09-19T09:18:30.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.570+0000 7faccf1f7740 20 client.20796 trim_cache size 0 max 16384 2024-09-19T09:18:30.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.589+0000 7faccf1f7740 1 Processor -- start 2024-09-19T09:18:30.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.590+0000 7faccf1f7740 1 -- start start 2024-09-19T09:18:30.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.590+0000 7faccf1f7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f875976850 0x55f8759792b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:30.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.590+0000 7faccf1f7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f875977170 0x55f875999520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:30.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.590+0000 7faccf1f7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f875981a30 0x55f875999a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:30.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.590+0000 7faccf1f7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7facc81af830 con 0x55f875976850 2024-09-19T09:18:30.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.590+0000 7faccf1f7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7facc81b1400 con 0x55f875981a30 2024-09-19T09:18:30.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.590+0000 7faccf1f7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7facc81aff00 con 0x55f875977170 2024-09-19T09:18:30.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.590+0000 7facbffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f875976850 0x55f8759792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:30.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.590+0000 7facbffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f875976850 0x55f8759792b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46726/0 (socket says 172.21.15.120:46726) 2024-09-19T09:18:30.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.590+0000 7facbffff640 1 -- 172.21.15.120:0/294960647 learned_addr learned my addr 172.21.15.120:0/294960647 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:30.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.590+0000 7facc659c640 1 --2- 172.21.15.120:0/294960647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f875977170 0x55f875999520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:30.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.590+0000 7facbffff640 1 --2- 172.21.15.120:0/294960647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f875981a30 0x55f875999a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:30.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.590+0000 7facbffff640 1 -- 172.21.15.120:0/294960647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f875977170 msgr2=0x55f875999520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:30.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.590+0000 7facbffff640 1 --2- 172.21.15.120:0/294960647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f875977170 0x55f875999520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:30.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.590+0000 7facbffff640 1 -- 172.21.15.120:0/294960647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f875981a30 msgr2=0x55f875999a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:30.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.590+0000 7facbffff640 1 --2- 172.21.15.120:0/294960647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f875981a30 0x55f875999a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:30.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.590+0000 7facbffff640 1 -- 172.21.15.120:0/294960647 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f87597ce30 con 0x55f875976850 2024-09-19T09:18:30.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.591+0000 7facbffff640 1 --2- 172.21.15.120:0/294960647 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f875976850 0x55f8759792b0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7facb0000c00 tx=0x7facb0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15326e366efa16a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:30.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.591+0000 7facaffff640 1 -- 172.21.15.120:0/294960647 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb00226b0 con 0x55f875976850 2024-09-19T09:18:30.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.591+0000 7facaffff640 1 -- 172.21.15.120:0/294960647 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facb0022850 con 0x55f875976850 2024-09-19T09:18:30.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.591+0000 7facaffff640 1 -- 172.21.15.120:0/294960647 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb00054d0 con 0x55f875976850 2024-09-19T09:18:30.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.592+0000 7faccf1f7740 1 -- 172.21.15.120:0/294960647 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f875976850 msgr2=0x55f8759792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:30.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.592+0000 7faccf1f7740 1 --2- 172.21.15.120:0/294960647 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f875976850 0x55f8759792b0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7facb0000c00 tx=0x7facb0009220 comp rx=0 tx=0).stop 2024-09-19T09:18:30.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.592+0000 7faccf1f7740 1 -- 172.21.15.120:0/294960647 shutdown_connections 2024-09-19T09:18:30.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.592+0000 7faccf1f7740 1 --2- 172.21.15.120:0/294960647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f875981a30 0x55f875999a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:30.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.592+0000 7faccf1f7740 1 --2- 172.21.15.120:0/294960647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f875977170 0x55f875999520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:30.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.592+0000 7faccf1f7740 1 --2- 172.21.15.120:0/294960647 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f875976850 0x55f8759792b0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:30.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.592+0000 7faccf1f7740 1 -- 172.21.15.120:0/294960647 >> 172.21.15.120:0/294960647 conn(0x55f875972de0 msgr2=0x55f87597f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:30.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.592+0000 7faccf1f7740 1 -- 172.21.15.120:0/294960647 shutdown_connections 2024-09-19T09:18:30.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.592+0000 7faccf1f7740 1 -- 172.21.15.120:0/294960647 wait complete. 2024-09-19T09:18:30.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.592+0000 7faccf1f7740 1 -- start start 2024-09-19T09:18:30.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.592+0000 7faccf1f7740 1 Processor -- start 2024-09-19T09:18:30.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.592+0000 7faccf1f7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f875976850 0x55f875985ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:30.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.592+0000 7faccf1f7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f875977170 0x55f875986000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:30.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.592+0000 7faccf1f7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f875981a30 0x55f8759fb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:30.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.592+0000 7faccf1f7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7facc81b0d30 con 0x55f875981a30 2024-09-19T09:18:30.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.592+0000 7faccf1f7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7facc81b05d0 con 0x55f875976850 2024-09-19T09:18:30.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.592+0000 7faccf1f7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7facc81ad7f0 con 0x55f875977170 2024-09-19T09:18:30.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.593+0000 7faccf1f7740 10 client.0 _ll_register_callbacks cb 0x7facb0009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:30.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.593+0000 7facc659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f875977170 0x55f875986000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:30.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.593+0000 7facc659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f875977170 0x55f875986000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:47226/0 (socket says 172.21.15.120:47226) 2024-09-19T09:18:30.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.593+0000 7facc659c640 1 -- 172.21.15.120:0/429668140 learned_addr learned my addr 172.21.15.120:0/429668140 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:30.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.593+0000 7facc659c640 1 -- 172.21.15.120:0/429668140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f875976850 msgr2=0x55f875985ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:18:30.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.593+0000 7facbffff640 1 --2- 172.21.15.120:0/429668140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f875976850 0x55f875985ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:30.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.593+0000 7facc659c640 1 --2- 172.21.15.120:0/429668140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f875976850 0x55f875985ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:30.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.593+0000 7facc659c640 1 -- 172.21.15.120:0/429668140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f875981a30 msgr2=0x55f8759fb4f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:18:30.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.593+0000 7facc659c640 1 --2- 172.21.15.120:0/429668140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f875981a30 0x55f8759fb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:30.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.593+0000 7facc659c640 1 -- 172.21.15.120:0/429668140 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f875986890 con 0x55f875977170 2024-09-19T09:18:30.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.593+0000 7facc659c640 1 --2- 172.21.15.120:0/429668140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f875977170 0x55f875986000 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7facb8005550 tx=0x7facb80054e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=307afcac72da0464 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:30.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.593+0000 7facadffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:18:30.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.593+0000 7facadffb640 1 -- 172.21.15.120:0/429668140 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8040040 con 0x55f875977170 2024-09-19T09:18:30.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.594+0000 7faccf1f7740 1 -- 172.21.15.120:0/429668140 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facb0022850 con 0x55f875977170 2024-09-19T09:18:30.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.594+0000 7facadffb640 1 -- 172.21.15.120:0/429668140 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7facb8004040 con 0x55f875977170 2024-09-19T09:18:30.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.594+0000 7facadffb640 1 -- 172.21.15.120:0/429668140 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7facb8035040 con 0x55f875977170 2024-09-19T09:18:30.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.594+0000 7faccf1f7740 10 client.29536 fetch_fsmap learned FSMap version 36 2024-09-19T09:18:30.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.594+0000 7faccf1f7740 1 -- 172.21.15.120:0/429668140 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f8759fcae0 con 0x55f875977170 2024-09-19T09:18:30.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.594+0000 7facadffb640 1 -- 172.21.15.120:0/429668140 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facb0022850 con 0x55f875977170 2024-09-19T09:18:30.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.594+0000 7facadffb640 1 -- 172.21.15.120:0/429668140 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7facb8058050 con 0x55f875977170 2024-09-19T09:18:30.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.595+0000 7faccf1f7740 10 client.29536 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:18:30.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.595+0000 7faccf1f7740 10 client.29536 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:18:30.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.595+0000 7faccf1f7740 20 client.29536 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:30.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.595+0000 7faccf1f7740 1 --2- 172.21.15.120:0/429668140 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f8759fcef0 0x55f8759ff2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:30.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.595+0000 7faccf1f7740 4 client.29536 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.45, "await_for": 45.0}] 2024-09-19T09:18:30.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.595+0000 7faccf1f7740 1 -- 172.21.15.120:0/429668140 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.45, "await_for": 45.0}) -- 0x55f8758e2be0 con 0x55f8759fcef0 2024-09-19T09:18:30.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.595+0000 7facbffff640 1 --2- 172.21.15.120:0/429668140 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f8759fcef0 0x55f8759ff2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:30.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.595+0000 7facbffff640 1 --2- 172.21.15.120:0/429668140 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f8759fcef0 0x55f8759ff2e0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10374f2c7d80faa6 server_cookie=3e508575379758 in_seq=0 out_seq=0 2024-09-19T09:18:30.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:30.595+0000 7facadffb640 10 client.29536 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:31.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:31.271+0000 7facadffb640 1 -- 172.21.15.120:0/429668140 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f8758e2be0 con 0x55f8759fcef0 2024-09-19T09:18:31.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:31.271+0000 7facadffb640 10 client.29536 handle_command_reply: tid=0 2024-09-19T09:18:31.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:31.271+0000 7faccf1f7740 1 client.29536 shutdown 2024-09-19T09:18:31.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:31.272+0000 7faccf1f7740 1 -- 172.21.15.120:0/429668140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f875977170 msgr2=0x55f875986000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:31.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:31.272+0000 7faccf1f7740 1 --2- 172.21.15.120:0/429668140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f875977170 0x55f875986000 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7facb8005550 tx=0x7facb80054e0 comp rx=0 tx=0).stop 2024-09-19T09:18:31.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:31.272+0000 7faccf1f7740 1 -- 172.21.15.120:0/429668140 shutdown_connections 2024-09-19T09:18:31.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:31.272+0000 7faccf1f7740 1 --2- 172.21.15.120:0/429668140 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f8759fcef0 0x55f8759ff2e0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:31.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:31.272+0000 7faccf1f7740 1 --2- 172.21.15.120:0/429668140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f875981a30 0x55f8759fb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:31.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:31.272+0000 7faccf1f7740 1 --2- 172.21.15.120:0/429668140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f875977170 0x55f875986000 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:31.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:31.272+0000 7faccf1f7740 1 --2- 172.21.15.120:0/429668140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f875976850 0x55f875985ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:31.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:31.272+0000 7facadffb640 0 client.29536 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:31.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:31.272+0000 7faccf1f7740 1 -- 172.21.15.120:0/429668140 >> 172.21.15.120:0/429668140 conn(0x55f875972de0 msgr2=0x55f8759716a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:31.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:31.272+0000 7faccf1f7740 1 -- 172.21.15.120:0/429668140 shutdown_connections 2024-09-19T09:18:31.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:31.272+0000 7faccf1f7740 1 -- 172.21.15.120:0/429668140 wait complete. 2024-09-19T09:18:31.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:31.272+0000 7faccf1f7740 20 client.29536 trim_cache size 0 max 16384 2024-09-19T09:18:31.276 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:18:31.276 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 36, 2024-09-19T09:18:31.276 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:18:31.276 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 54, 2024-09-19T09:18:31.276 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:18:31.276 INFO:teuthology.orchestra.run.smithi120.stdout: "792ee6f9": { 2024-09-19T09:18:31.276 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 54, 2024-09-19T09:18:31.276 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:18:31.276 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:18:31.276 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:18:31.276 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:18:31.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:18:31.277 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:18:31.277 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 149.4, 2024-09-19T09:18:31.277 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:18:31.277 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:18:31.277 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:18:31.277 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:18:31.277 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:18:31.277 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:18:31.277 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:18:31.277 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:18:31.277 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:18:31.277 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:18:31.277 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:18:31.278 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:18:31.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:31.272+0000 7faccce1e640 1 -- 172.21.15.120:0/326399489 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7facb40a3240 msgr2=0x7facb40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:31.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:31.272+0000 7faccce1e640 1 --2- 172.21.15.120:0/326399489 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7facb40a3240 0x7facb40a5700 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7facc0005fd0 tx=0x7facc00058d0 comp rx=0 tx=0).stop 2024-09-19T09:18:31.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:31.272+0000 7faccce1e640 1 -- 172.21.15.120:0/326399489 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facc8106a70 msgr2=0x7facc8197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:31.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:31.272+0000 7faccce1e640 1 --2- 172.21.15.120:0/326399489 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facc8106a70 0x7facc8197b70 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7facc000e940 tx=0x7facc000ee00 comp rx=0 tx=0).stop 2024-09-19T09:18:31.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:31.273+0000 7faccce1e640 1 -- 172.21.15.120:0/326399489 shutdown_connections 2024-09-19T09:18:31.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:31.273+0000 7faccce1e640 1 --2- 172.21.15.120:0/326399489 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7facb40a3240 0x7facb40a5700 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:31.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:31.273+0000 7faccce1e640 1 --2- 172.21.15.120:0/326399489 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facc8106a70 0x7facc8197b70 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:31.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:31.273+0000 7faccce1e640 1 --2- 172.21.15.120:0/326399489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facc8101410 0x7facc8197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:31.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:31.273+0000 7faccce1e640 1 --2- 172.21.15.120:0/326399489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facc8100a70 0x7facc81970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:31.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:31.273+0000 7faccce1e640 1 -- 172.21.15.120:0/326399489 >> 172.21.15.120:0/326399489 conn(0x7facc80fc790 msgr2=0x7facc80fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:31.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:31.273+0000 7faccce1e640 1 -- 172.21.15.120:0/326399489 shutdown_connections 2024-09-19T09:18:31.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:31.273+0000 7faccce1e640 1 -- 172.21.15.120:0/326399489 wait complete. 2024-09-19T09:18:31.358 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 4.13 MB/sec execute 26 sec latency 689.052 ms 2024-09-19T09:18:32.254 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '792ee6f9', quiesce took 4.6 seconds. Will release after: 24.85 2024-09-19T09:18:32.358 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 3.98 MB/sec execute 27 sec latency 1689.130 ms 2024-09-19T09:18:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.963+0000 7fc75f074640 1 -- 172.21.15.120:0/3746434059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc758101c50 msgr2=0x7fc7581020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.963+0000 7fc75f074640 1 --2- 172.21.15.120:0/3746434059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc758101c50 0x7fc7581020d0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009900 tx=0x7fc74c02a410 comp rx=0 tx=0).stop 2024-09-19T09:18:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.963+0000 7fc75f074640 1 -- 172.21.15.120:0/3746434059 shutdown_connections 2024-09-19T09:18:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.963+0000 7fc75f074640 1 --2- 172.21.15.120:0/3746434059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc758102610 0x7fc758108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.963+0000 7fc75f074640 1 --2- 172.21.15.120:0/3746434059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc758101c50 0x7fc7581020d0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.963+0000 7fc75f074640 1 --2- 172.21.15.120:0/3746434059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc758100a60 0x7fc758100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.963+0000 7fc75f074640 1 -- 172.21.15.120:0/3746434059 >> 172.21.15.120:0/3746434059 conn(0x7fc7580fc1e0 msgr2=0x7fc7580fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.963+0000 7fc75f074640 1 -- 172.21.15.120:0/3746434059 shutdown_connections 2024-09-19T09:18:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.963+0000 7fc75f074640 1 -- 172.21.15.120:0/3746434059 wait complete. 2024-09-19T09:18:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.964+0000 7fc75f074640 1 Processor -- start 2024-09-19T09:18:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.964+0000 7fc75f074640 1 -- start start 2024-09-19T09:18:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.964+0000 7fc75f074640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc758100a60 0x7fc758197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.964+0000 7fc75f074640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc758101c50 0x7fc758198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.964+0000 7fc75f074640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc758102610 0x7fc75819f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.964+0000 7fc75f074640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc75810c6c0 con 0x7fc758102610 2024-09-19T09:18:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.964+0000 7fc75f074640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc75810c540 con 0x7fc758100a60 2024-09-19T09:18:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.964+0000 7fc75f074640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc75810c840 con 0x7fc758101c50 2024-09-19T09:18:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.964+0000 7fc75d611640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc758102610 0x7fc75819f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.964+0000 7fc75d611640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc758102610 0x7fc75819f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52154/0 (socket says 172.21.15.120:52154) 2024-09-19T09:18:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.964+0000 7fc75d611640 1 -- 172.21.15.120:0/452517407 learned_addr learned my addr 172.21.15.120:0/452517407 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.964+0000 7fc757fff640 1 --2- 172.21.15.120:0/452517407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc758101c50 0x7fc758198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.964+0000 7fc75ce10640 1 --2- 172.21.15.120:0/452517407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc758100a60 0x7fc758197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:32.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.964+0000 7fc75d611640 1 -- 172.21.15.120:0/452517407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc758101c50 msgr2=0x7fc758198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:32.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.964+0000 7fc75d611640 1 --2- 172.21.15.120:0/452517407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc758101c50 0x7fc758198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.964+0000 7fc75d611640 1 -- 172.21.15.120:0/452517407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc758100a60 msgr2=0x7fc758197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:32.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.964+0000 7fc75d611640 1 --2- 172.21.15.120:0/452517407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc758100a60 0x7fc758197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.964+0000 7fc75d611640 1 -- 172.21.15.120:0/452517407 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75819f970 con 0x7fc758102610 2024-09-19T09:18:32.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.964+0000 7fc75d611640 1 --2- 172.21.15.120:0/452517407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc758102610 0x7fc75819f300 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fc75000cce0 tx=0x7fc75000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:32.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.965+0000 7fc755ffb640 1 -- 172.21.15.120:0/452517407 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75000f450 con 0x7fc758102610 2024-09-19T09:18:32.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.965+0000 7fc755ffb640 1 -- 172.21.15.120:0/452517407 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc75000fad0 con 0x7fc758102610 2024-09-19T09:18:32.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.965+0000 7fc75f074640 1 -- 172.21.15.120:0/452517407 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc75819fc60 con 0x7fc758102610 2024-09-19T09:18:32.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.965+0000 7fc755ffb640 1 -- 172.21.15.120:0/452517407 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750018c10 con 0x7fc758102610 2024-09-19T09:18:32.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.965+0000 7fc75f074640 1 -- 172.21.15.120:0/452517407 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7581a0180 con 0x7fc758102610 2024-09-19T09:18:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.967+0000 7fc755ffb640 1 -- 172.21.15.120:0/452517407 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fc75000f5f0 con 0x7fc758102610 2024-09-19T09:18:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.968+0000 7fc761e17740 1 Processor -- start 2024-09-19T09:18:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.968+0000 7fc761e17740 1 -- start start 2024-09-19T09:18:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.968+0000 7fc761e17740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55803a0ec9a0 0x55803a0ecda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.968+0000 7fc761e17740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803a0ed2e0 0x55803a0f1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.968+0000 7fc755ffb640 1 --2- 172.21.15.120:0/452517407 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc7400a3370 0x7fc7400a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.968+0000 7fc761e17740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55803a0f2370 0x55803a0f4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.968+0000 7fc761e17740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55803a0502f0 con 0x55803a0f2370 2024-09-19T09:18:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.968+0000 7fc761e17740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558039ff8270 con 0x55803a0ed2e0 2024-09-19T09:18:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.968+0000 7fc761e17740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55803a04c2b0 con 0x55803a0ec9a0 2024-09-19T09:18:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.968+0000 7fc75d611640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55803a0f2370 0x55803a0f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.968+0000 7fc757fff640 1 --2- 172.21.15.120:0/452517407 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc7400a3370 0x7fc7400a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.968+0000 7fc75d611640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55803a0f2370 0x55803a0f4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52170/0 (socket says 172.21.15.120:52170) 2024-09-19T09:18:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.968+0000 7fc75d611640 1 -- 172.21.15.120:0/2398224764 learned_addr learned my addr 172.21.15.120:0/2398224764 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.968+0000 7fc757fff640 1 --2- 172.21.15.120:0/2398224764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55803a0ec9a0 0x55803a0ecda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.968+0000 7fc75ce10640 1 --2- 172.21.15.120:0/2398224764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803a0ed2e0 0x55803a0f1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.968+0000 7fc755ffb640 1 -- 172.21.15.120:0/452517407 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc7500cfed0 con 0x7fc758102610 2024-09-19T09:18:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.969+0000 7fc75d611640 1 -- 172.21.15.120:0/2398224764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55803a0ec9a0 msgr2=0x55803a0ecda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.969+0000 7fc75d611640 1 --2- 172.21.15.120:0/2398224764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55803a0ec9a0 0x55803a0ecda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.969+0000 7fc75d611640 1 -- 172.21.15.120:0/2398224764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803a0ed2e0 msgr2=0x55803a0f1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.969+0000 7fc75d611640 1 --2- 172.21.15.120:0/2398224764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803a0ed2e0 0x55803a0f1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.969+0000 7fc75d611640 1 -- 172.21.15.120:0/2398224764 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558039ff7340 con 0x55803a0f2370 2024-09-19T09:18:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.969+0000 7fc75ce10640 1 --2- 172.21.15.120:0/2398224764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803a0ed2e0 0x55803a0f1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:18:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.969+0000 7fc75d611640 1 --2- 172.21.15.120:0/2398224764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55803a0f2370 0x55803a0f4830 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fc750098ec0 tx=0x7fc7500186b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3288bbe413acb30a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.969+0000 7fc757fff640 1 --2- 172.21.15.120:0/452517407 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc7400a3370 0x7fc7400a5830 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0314e0 tx=0x7fc74c032cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.969+0000 7fc73effd640 1 -- 172.21.15.120:0/2398224764 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7500a2030 con 0x55803a0f2370 2024-09-19T09:18:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.969+0000 7fc73effd640 1 -- 172.21.15.120:0/2398224764 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc75009cbf0 con 0x55803a0f2370 2024-09-19T09:18:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.969+0000 7fc73effd640 1 -- 172.21.15.120:0/2398224764 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750004a40 con 0x55803a0f2370 2024-09-19T09:18:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.970+0000 7fc761e17740 1 -- 172.21.15.120:0/2398224764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55803a0f2370 msgr2=0x55803a0f4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.970+0000 7fc761e17740 1 --2- 172.21.15.120:0/2398224764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55803a0f2370 0x55803a0f4830 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fc750098ec0 tx=0x7fc7500186b0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.970+0000 7fc761e17740 1 -- 172.21.15.120:0/2398224764 shutdown_connections 2024-09-19T09:18:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.970+0000 7fc761e17740 1 --2- 172.21.15.120:0/2398224764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55803a0f2370 0x55803a0f4830 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.970+0000 7fc761e17740 1 --2- 172.21.15.120:0/2398224764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803a0ed2e0 0x55803a0f1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.970+0000 7fc761e17740 1 --2- 172.21.15.120:0/2398224764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55803a0ec9a0 0x55803a0ecda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.970+0000 7fc761e17740 1 -- 172.21.15.120:0/2398224764 >> 172.21.15.120:0/2398224764 conn(0x55803a0e8010 msgr2=0x55803a0e8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.970+0000 7fc761e17740 1 -- 172.21.15.120:0/2398224764 shutdown_connections 2024-09-19T09:18:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.970+0000 7fc761e17740 1 -- 172.21.15.120:0/2398224764 wait complete. 2024-09-19T09:18:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.970+0000 7fc761e17740 1 -- start start 2024-09-19T09:18:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.970+0000 7fc761e17740 1 Processor -- start 2024-09-19T09:18:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.971+0000 7fc761e17740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55803a0ec9a0 0x55803a0f6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.971+0000 7fc761e17740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55803a0ed2e0 0x55803a0f7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.971+0000 7fc761e17740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803a0f2370 0x55803a0fe170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.971+0000 7fc761e17740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55803a0f5340 con 0x55803a0ed2e0 2024-09-19T09:18:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.971+0000 7fc761e17740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558039fd9290 con 0x55803a0f2370 2024-09-19T09:18:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.971+0000 7fc761e17740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55803a0f54c0 con 0x55803a0ec9a0 2024-09-19T09:18:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.971+0000 7fc761e17740 10 client.0 _ll_register_callbacks cb 0x55803a00a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.971+0000 7fc757fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55803a0ed2e0 0x55803a0f7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.971+0000 7fc757fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55803a0ed2e0 0x55803a0f7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52186/0 (socket says 172.21.15.120:52186) 2024-09-19T09:18:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.971+0000 7fc757fff640 1 -- 172.21.15.120:0/2071668019 learned_addr learned my addr 172.21.15.120:0/2071668019 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.971+0000 7fc75ce10640 1 --2- 172.21.15.120:0/2071668019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55803a0ec9a0 0x55803a0f6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.971+0000 7fc75d611640 1 --2- 172.21.15.120:0/2071668019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803a0f2370 0x55803a0fe170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.971+0000 7fc757fff640 1 -- 172.21.15.120:0/2071668019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55803a0ec9a0 msgr2=0x55803a0f6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.971+0000 7fc757fff640 1 --2- 172.21.15.120:0/2071668019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55803a0ec9a0 0x55803a0f6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.971+0000 7fc757fff640 1 -- 172.21.15.120:0/2071668019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803a0f2370 msgr2=0x55803a0fe170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.971+0000 7fc757fff640 1 --2- 172.21.15.120:0/2071668019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803a0f2370 0x55803a0fe170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.971+0000 7fc757fff640 1 -- 172.21.15.120:0/2071668019 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558039d63210 con 0x55803a0ed2e0 2024-09-19T09:18:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.971+0000 7fc757fff640 1 --2- 172.21.15.120:0/2071668019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55803a0ed2e0 0x55803a0f7130 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fc74c033000 tx=0x7fc74c046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b315ffda864b898c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.971+0000 7fc737fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:18:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.971+0000 7fc737fff640 1 -- 172.21.15.120:0/2071668019 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c051070 con 0x55803a0ed2e0 2024-09-19T09:18:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.971+0000 7fc737fff640 1 -- 172.21.15.120:0/2071668019 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc74c003ac0 con 0x55803a0ed2e0 2024-09-19T09:18:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.971+0000 7fc761e17740 1 -- 172.21.15.120:0/2071668019 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc75009cbf0 con 0x55803a0ed2e0 2024-09-19T09:18:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.972+0000 7fc737fff640 1 -- 172.21.15.120:0/2071668019 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc74c05d050 con 0x55803a0ed2e0 2024-09-19T09:18:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.972+0000 7fc761e17740 10 client.20811 fetch_fsmap learned FSMap version 36 2024-09-19T09:18:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.972+0000 7fc761e17740 1 -- 172.21.15.120:0/2071668019 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558039e2f620 con 0x55803a0ed2e0 2024-09-19T09:18:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.972+0000 7fc737fff640 1 -- 172.21.15.120:0/2071668019 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc75009cbf0 con 0x55803a0ed2e0 2024-09-19T09:18:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.972+0000 7fc737fff640 1 -- 172.21.15.120:0/2071668019 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fc74c04b410 con 0x55803a0ed2e0 2024-09-19T09:18:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.972+0000 7fc761e17740 10 client.20811 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:18:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.972+0000 7fc761e17740 10 client.20811 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:18:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.972+0000 7fc761e17740 20 client.20811 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.972+0000 7fc761e17740 1 --2- 172.21.15.120:0/2071668019 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55803a1029e0 0x55803a104e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.972+0000 7fc761e17740 4 client.20811 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:18:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.972+0000 7fc761e17740 1 -- 172.21.15.120:0/2071668019 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55803a05da20 con 0x55803a1029e0 2024-09-19T09:18:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.972+0000 7fc75ce10640 1 --2- 172.21.15.120:0/2071668019 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55803a1029e0 0x55803a104e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.973+0000 7fc737fff640 10 client.20811 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.973+0000 7fc75ce10640 1 --2- 172.21.15.120:0/2071668019 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55803a1029e0 0x55803a104e10 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=838896f5eabd95ed server_cookie=328988f77cd8f52e in_seq=0 out_seq=0 2024-09-19T09:18:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.973+0000 7fc737fff640 1 -- 172.21.15.120:0/2071668019 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55803a05da20 con 0x55803a1029e0 2024-09-19T09:18:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.973+0000 7fc737fff640 10 client.20811 handle_command_reply: tid=0 2024-09-19T09:18:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.973+0000 7fc761e17740 1 client.20811 shutdown 2024-09-19T09:18:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.973+0000 7fc761e17740 1 -- 172.21.15.120:0/2071668019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55803a0ed2e0 msgr2=0x55803a0f7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.973+0000 7fc761e17740 1 --2- 172.21.15.120:0/2071668019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55803a0ed2e0 0x55803a0f7130 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fc74c033000 tx=0x7fc74c046000 comp rx=0 tx=0).stop 2024-09-19T09:18:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.974+0000 7fc761e17740 1 -- 172.21.15.120:0/2071668019 shutdown_connections 2024-09-19T09:18:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.974+0000 7fc761e17740 1 --2- 172.21.15.120:0/2071668019 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55803a1029e0 0x55803a104e10 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.974+0000 7fc761e17740 1 --2- 172.21.15.120:0/2071668019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803a0f2370 0x55803a0fe170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.974+0000 7fc737fff640 0 client.20811 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.974+0000 7fc761e17740 1 --2- 172.21.15.120:0/2071668019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55803a0ed2e0 0x55803a0f7130 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.974+0000 7fc761e17740 1 --2- 172.21.15.120:0/2071668019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55803a0ec9a0 0x55803a0f6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.974+0000 7fc761e17740 1 -- 172.21.15.120:0/2071668019 >> 172.21.15.120:0/2071668019 conn(0x55803a0e8010 msgr2=0x55803a0e9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.974+0000 7fc761e17740 1 -- 172.21.15.120:0/2071668019 shutdown_connections 2024-09-19T09:18:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.974+0000 7fc761e17740 1 -- 172.21.15.120:0/2071668019 wait complete. 2024-09-19T09:18:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.974+0000 7fc761e17740 20 client.20811 trim_cache size 0 max 16384 2024-09-19T09:18:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.988+0000 7fc761e17740 1 Processor -- start 2024-09-19T09:18:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.988+0000 7fc761e17740 1 -- start start 2024-09-19T09:18:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.988+0000 7fc761e17740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55803a0ec9a0 0x55803a0e70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.988+0000 7fc761e17740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803a0ed2e0 0x55803a0fe130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.988+0000 7fc761e17740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55803a0f2370 0x55803a109710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.988+0000 7fc761e17740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc7581a8ee0 con 0x55803a0ec9a0 2024-09-19T09:18:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.988+0000 7fc761e17740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc7581aaab0 con 0x55803a0ed2e0 2024-09-19T09:18:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.988+0000 7fc761e17740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc7581a95b0 con 0x55803a0f2370 2024-09-19T09:18:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.988+0000 7fc75ce10640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55803a0ec9a0 0x55803a0e70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.988+0000 7fc75ce10640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55803a0ec9a0 0x55803a0e70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52196/0 (socket says 172.21.15.120:52196) 2024-09-19T09:18:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.988+0000 7fc75ce10640 1 -- 172.21.15.120:0/1382037560 learned_addr learned my addr 172.21.15.120:0/1382037560 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.989+0000 7fc75d611640 1 --2- 172.21.15.120:0/1382037560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55803a0f2370 0x55803a109710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.989+0000 7fc75ce10640 1 -- 172.21.15.120:0/1382037560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55803a0f2370 msgr2=0x55803a109710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.989+0000 7fc75ce10640 1 --2- 172.21.15.120:0/1382037560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55803a0f2370 0x55803a109710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.989+0000 7fc75ce10640 1 -- 172.21.15.120:0/1382037560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803a0ed2e0 msgr2=0x55803a0fe130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.989+0000 7fc75ce10640 1 --2- 172.21.15.120:0/1382037560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803a0ed2e0 0x55803a0fe130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.989+0000 7fc75ce10640 1 -- 172.21.15.120:0/1382037560 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55803a105490 con 0x55803a0ec9a0 2024-09-19T09:18:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.989+0000 7fc75d611640 1 --2- 172.21.15.120:0/1382037560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55803a0f2370 0x55803a109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:18:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.989+0000 7fc75ce10640 1 --2- 172.21.15.120:0/1382037560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55803a0ec9a0 0x55803a0e70b0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x55803a0edd10 tx=0x7fc74800a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5adcae1db0e51279 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.989+0000 7fc73e7fc640 1 -- 172.21.15.120:0/1382037560 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc748005220 con 0x55803a0ec9a0 2024-09-19T09:18:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.989+0000 7fc73e7fc640 1 -- 172.21.15.120:0/1382037560 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc748004c80 con 0x55803a0ec9a0 2024-09-19T09:18:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.989+0000 7fc73e7fc640 1 -- 172.21.15.120:0/1382037560 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc748027aa0 con 0x55803a0ec9a0 2024-09-19T09:18:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.990+0000 7fc761e17740 1 -- 172.21.15.120:0/1382037560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55803a0ec9a0 msgr2=0x55803a0e70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.990+0000 7fc761e17740 1 --2- 172.21.15.120:0/1382037560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55803a0ec9a0 0x55803a0e70b0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x55803a0edd10 tx=0x7fc74800a2d0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.990+0000 7fc761e17740 1 -- 172.21.15.120:0/1382037560 shutdown_connections 2024-09-19T09:18:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.990+0000 7fc761e17740 1 --2- 172.21.15.120:0/1382037560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55803a0f2370 0x55803a109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.990+0000 7fc761e17740 1 --2- 172.21.15.120:0/1382037560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803a0ed2e0 0x55803a0fe130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.990+0000 7fc761e17740 1 --2- 172.21.15.120:0/1382037560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55803a0ec9a0 0x55803a0e70b0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.990+0000 7fc761e17740 1 -- 172.21.15.120:0/1382037560 >> 172.21.15.120:0/1382037560 conn(0x55803a0e8010 msgr2=0x55803a0e69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.990+0000 7fc761e17740 1 -- 172.21.15.120:0/1382037560 shutdown_connections 2024-09-19T09:18:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.990+0000 7fc761e17740 1 -- 172.21.15.120:0/1382037560 wait complete. 2024-09-19T09:18:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.990+0000 7fc761e17740 1 -- start start 2024-09-19T09:18:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.990+0000 7fc761e17740 1 Processor -- start 2024-09-19T09:18:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.990+0000 7fc761e17740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55803a0ec9a0 0x55803a0fc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.990+0000 7fc761e17740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803a0ed2e0 0x55803a0fc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.990+0000 7fc761e17740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55803a0f2370 0x55803a0fd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.990+0000 7fc761e17740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc7581aa3e0 con 0x55803a0f2370 2024-09-19T09:18:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.990+0000 7fc761e17740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc7581a9c80 con 0x55803a0ed2e0 2024-09-19T09:18:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.990+0000 7fc761e17740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc7581a6ea0 con 0x55803a0ec9a0 2024-09-19T09:18:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.990+0000 7fc761e17740 10 client.0 _ll_register_callbacks cb 0x7fc74c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.991+0000 7fc75d611640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55803a0f2370 0x55803a0fd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.991+0000 7fc75d611640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55803a0f2370 0x55803a0fd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52212/0 (socket says 172.21.15.120:52212) 2024-09-19T09:18:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.991+0000 7fc75d611640 1 -- 172.21.15.120:0/1068972318 learned_addr learned my addr 172.21.15.120:0/1068972318 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.991+0000 7fc757fff640 1 --2- 172.21.15.120:0/1068972318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803a0ed2e0 0x55803a0fc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.991+0000 7fc75ce10640 1 --2- 172.21.15.120:0/1068972318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55803a0ec9a0 0x55803a0fc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.991+0000 7fc75d611640 1 -- 172.21.15.120:0/1068972318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55803a0ec9a0 msgr2=0x55803a0fc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.991+0000 7fc75d611640 1 --2- 172.21.15.120:0/1068972318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55803a0ec9a0 0x55803a0fc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.991+0000 7fc75d611640 1 -- 172.21.15.120:0/1068972318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803a0ed2e0 msgr2=0x55803a0fc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.991+0000 7fc75d611640 1 --2- 172.21.15.120:0/1068972318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803a0ed2e0 0x55803a0fc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.991+0000 7fc75d611640 1 -- 172.21.15.120:0/1068972318 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55803a0fdb70 con 0x55803a0f2370 2024-09-19T09:18:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.991+0000 7fc75ce10640 1 --2- 172.21.15.120:0/1068972318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55803a0ec9a0 0x55803a0fc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:18:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.991+0000 7fc75d611640 1 --2- 172.21.15.120:0/1068972318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55803a0f2370 0x55803a0fd560 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fc75009c1f0 tx=0x7fc750098180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ececba0f9fc09db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.991+0000 7fc737fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:18:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.991+0000 7fc737fff640 1 -- 172.21.15.120:0/1068972318 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750092020 con 0x55803a0f2370 2024-09-19T09:18:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.991+0000 7fc737fff640 1 -- 172.21.15.120:0/1068972318 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc7500a48d0 con 0x55803a0f2370 2024-09-19T09:18:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.991+0000 7fc761e17740 1 -- 172.21.15.120:0/1068972318 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc748004c80 con 0x55803a0f2370 2024-09-19T09:18:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.991+0000 7fc737fff640 1 -- 172.21.15.120:0/1068972318 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc7500a5160 con 0x55803a0f2370 2024-09-19T09:18:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.991+0000 7fc761e17740 10 client.20817 fetch_fsmap learned FSMap version 36 2024-09-19T09:18:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.991+0000 7fc761e17740 1 -- 172.21.15.120:0/1068972318 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55803a16de30 con 0x55803a0f2370 2024-09-19T09:18:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.991+0000 7fc737fff640 1 -- 172.21.15.120:0/1068972318 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc748004c80 con 0x55803a0f2370 2024-09-19T09:18:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.992+0000 7fc737fff640 1 -- 172.21.15.120:0/1068972318 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fc7500b5260 con 0x55803a0f2370 2024-09-19T09:18:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.992+0000 7fc761e17740 10 client.20817 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T09:18:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.992+0000 7fc761e17740 10 client.20817 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:18:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.992+0000 7fc761e17740 20 client.20817 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.992+0000 7fc761e17740 1 --2- 172.21.15.120:0/1068972318 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55803a16e3b0 0x55803a1707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.992+0000 7fc761e17740 4 client.20817 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:18:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.992+0000 7fc761e17740 1 -- 172.21.15.120:0/1068972318 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55803a057d70 con 0x55803a16e3b0 2024-09-19T09:18:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.992+0000 7fc75ce10640 1 --2- 172.21.15.120:0/1068972318 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55803a16e3b0 0x55803a1707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.992+0000 7fc75ce10640 1 --2- 172.21.15.120:0/1068972318 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55803a16e3b0 0x55803a1707a0 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8794fde76f013209 server_cookie=3e6f3c3df7fbe743 in_seq=0 out_seq=0 2024-09-19T09:18:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:32.992+0000 7fc737fff640 10 client.20817 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:33.162+0000 7fc737fff640 1 -- 172.21.15.120:0/1068972318 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55803a057d70 con 0x55803a16e3b0 2024-09-19T09:18:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:33.162+0000 7fc737fff640 10 client.20817 handle_command_reply: tid=0 2024-09-19T09:18:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:33.162+0000 7fc761e17740 1 client.20817 shutdown 2024-09-19T09:18:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:33.163+0000 7fc761e17740 1 -- 172.21.15.120:0/1068972318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55803a0f2370 msgr2=0x55803a0fd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:33.163+0000 7fc761e17740 1 --2- 172.21.15.120:0/1068972318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55803a0f2370 0x55803a0fd560 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fc75009c1f0 tx=0x7fc750098180 comp rx=0 tx=0).stop 2024-09-19T09:18:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:33.163+0000 7fc761e17740 1 -- 172.21.15.120:0/1068972318 shutdown_connections 2024-09-19T09:18:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:33.163+0000 7fc761e17740 1 --2- 172.21.15.120:0/1068972318 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55803a16e3b0 0x55803a1707a0 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:33.163+0000 7fc761e17740 1 --2- 172.21.15.120:0/1068972318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55803a0f2370 0x55803a0fd560 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:33.163+0000 7fc761e17740 1 --2- 172.21.15.120:0/1068972318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55803a0ed2e0 0x55803a0fc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:33.163+0000 7fc737fff640 0 client.20817 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:33.163+0000 7fc761e17740 1 --2- 172.21.15.120:0/1068972318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55803a0ec9a0 0x55803a0fc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:33.163+0000 7fc761e17740 1 -- 172.21.15.120:0/1068972318 >> 172.21.15.120:0/1068972318 conn(0x55803a0e8010 msgr2=0x55803a0e71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:33.163+0000 7fc761e17740 1 -- 172.21.15.120:0/1068972318 shutdown_connections 2024-09-19T09:18:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:33.163+0000 7fc761e17740 1 -- 172.21.15.120:0/1068972318 wait complete. 2024-09-19T09:18:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:33.163+0000 7fc761e17740 20 client.20817 trim_cache size 0 max 16384 2024-09-19T09:18:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:18:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:18:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:18:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:18:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:33.164+0000 7fc75f074640 1 -- 172.21.15.120:0/452517407 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc7400a3370 msgr2=0x7fc7400a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:33.164+0000 7fc75f074640 1 --2- 172.21.15.120:0/452517407 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc7400a3370 0x7fc7400a5830 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0314e0 tx=0x7fc74c032cc0 comp rx=0 tx=0).stop 2024-09-19T09:18:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:33.164+0000 7fc75f074640 1 -- 172.21.15.120:0/452517407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc758102610 msgr2=0x7fc75819f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:33.164+0000 7fc75f074640 1 --2- 172.21.15.120:0/452517407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc758102610 0x7fc75819f300 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fc75000cce0 tx=0x7fc75000a620 comp rx=0 tx=0).stop 2024-09-19T09:18:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:33.164+0000 7fc75f074640 1 -- 172.21.15.120:0/452517407 shutdown_connections 2024-09-19T09:18:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:33.164+0000 7fc75f074640 1 --2- 172.21.15.120:0/452517407 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc7400a3370 0x7fc7400a5830 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:33.164+0000 7fc75f074640 1 --2- 172.21.15.120:0/452517407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc758102610 0x7fc75819f300 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:33.164+0000 7fc75f074640 1 --2- 172.21.15.120:0/452517407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc758101c50 0x7fc758198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:33.164+0000 7fc75f074640 1 --2- 172.21.15.120:0/452517407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc758100a60 0x7fc758197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:33.164+0000 7fc75f074640 1 -- 172.21.15.120:0/452517407 >> 172.21.15.120:0/452517407 conn(0x7fc7580fc1e0 msgr2=0x7fc7580fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:33.164+0000 7fc75f074640 1 -- 172.21.15.120:0/452517407 shutdown_connections 2024-09-19T09:18:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:33.164+0000 7fc75f074640 1 -- 172.21.15.120:0/452517407 wait complete. 2024-09-19T09:18:33.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:32 smithi120 ceph-mon[31150]: pgmap v815: 97 pgs: 97 active+clean; 284 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.2 MiB/s wr, 419 op/s 2024-09-19T09:18:33.358 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 3.83 MB/sec execute 28 sec latency 2689.214 ms 2024-09-19T09:18:33.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:32 smithi196 ceph-mon[35150]: pgmap v815: 97 pgs: 97 active+clean; 284 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.2 MiB/s wr, 419 op/s 2024-09-19T09:18:33.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:32 smithi153 ceph-mon[35071]: pgmap v815: 97 pgs: 97 active+clean; 284 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.2 MiB/s wr, 419 op/s 2024-09-19T09:18:34.358 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 3.70 MB/sec execute 29 sec latency 3689.301 ms 2024-09-19T09:18:35.240 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:34 smithi153 ceph-mon[35071]: pgmap v816: 97 pgs: 97 active+clean; 281 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.4 MiB/s wr, 440 op/s 2024-09-19T09:18:35.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:34 smithi120 ceph-mon[31150]: pgmap v816: 97 pgs: 97 active+clean; 281 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.4 MiB/s wr, 440 op/s 2024-09-19T09:18:35.359 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 3.58 MB/sec execute 30 sec latency 4689.414 ms 2024-09-19T09:18:35.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:34 smithi196 ceph-mon[35150]: pgmap v816: 97 pgs: 97 active+clean; 281 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.4 MiB/s wr, 440 op/s 2024-09-19T09:18:36.359 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 3.46 MB/sec execute 31 sec latency 5689.499 ms 2024-09-19T09:18:37.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:36 smithi120 ceph-mon[31150]: pgmap v817: 97 pgs: 97 active+clean; 280 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.8 MiB/s wr, 372 op/s 2024-09-19T09:18:37.359 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 3.35 MB/sec execute 32 sec latency 6689.591 ms 2024-09-19T09:18:37.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:36 smithi196 ceph-mon[35150]: pgmap v817: 97 pgs: 97 active+clean; 280 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.8 MiB/s wr, 372 op/s 2024-09-19T09:18:37.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:36 smithi153 ceph-mon[35071]: pgmap v817: 97 pgs: 97 active+clean; 280 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.8 MiB/s wr, 372 op/s 2024-09-19T09:18:38.359 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 3.25 MB/sec execute 33 sec latency 7689.688 ms 2024-09-19T09:18:39.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:38 smithi120 ceph-mon[31150]: pgmap v818: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.4 MiB/s wr, 396 op/s 2024-09-19T09:18:39.359 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 3.16 MB/sec execute 34 sec latency 8689.806 ms 2024-09-19T09:18:39.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:38 smithi196 ceph-mon[35150]: pgmap v818: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.4 MiB/s wr, 396 op/s 2024-09-19T09:18:39.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:38 smithi153 ceph-mon[35071]: pgmap v818: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.4 MiB/s wr, 396 op/s 2024-09-19T09:18:40.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:39 smithi120 ceph-mon[31150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:18:40.359 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 3.07 MB/sec execute 35 sec latency 9689.912 ms 2024-09-19T09:18:40.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:39 smithi196 ceph-mon[35150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:18:40.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:39 smithi153 ceph-mon[35071]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:18:41.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:40 smithi120 ceph-mon[31150]: pgmap v819: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.1 MiB/s wr, 344 op/s 2024-09-19T09:18:41.359 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 2.98 MB/sec execute 36 sec latency 10689.997 ms 2024-09-19T09:18:41.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:40 smithi196 ceph-mon[35150]: pgmap v819: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.1 MiB/s wr, 344 op/s 2024-09-19T09:18:41.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:40 smithi153 ceph-mon[35071]: pgmap v819: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.1 MiB/s wr, 344 op/s 2024-09-19T09:18:42.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:41 smithi120 ceph-mon[31150]: pgmap v820: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.7 MiB/s wr, 297 op/s 2024-09-19T09:18:42.359 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 2.90 MB/sec execute 37 sec latency 11690.118 ms 2024-09-19T09:18:42.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:41 smithi196 ceph-mon[35150]: pgmap v820: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.7 MiB/s wr, 297 op/s 2024-09-19T09:18:42.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:41 smithi153 ceph-mon[35071]: pgmap v820: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.7 MiB/s wr, 297 op/s 2024-09-19T09:18:43.359 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 2.82 MB/sec execute 38 sec latency 12690.233 ms 2024-09-19T09:18:44.360 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 2.75 MB/sec execute 39 sec latency 13690.337 ms 2024-09-19T09:18:45.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:44 smithi120 ceph-mon[31150]: pgmap v821: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 180 op/s 2024-09-19T09:18:45.360 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 2.68 MB/sec execute 40 sec latency 14690.430 ms 2024-09-19T09:18:45.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:44 smithi196 ceph-mon[35150]: pgmap v821: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 180 op/s 2024-09-19T09:18:45.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:44 smithi153 ceph-mon[35071]: pgmap v821: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 180 op/s 2024-09-19T09:18:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.263+0000 7f8ff97cd640 1 -- 172.21.15.120:0/3847716267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ff41025d0 msgr2=0x7f8ff41049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.263+0000 7f8ff97cd640 1 --2- 172.21.15.120:0/3847716267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ff41025d0 0x7f8ff41049c0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009ca0 tx=0x7f8fe402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.263+0000 7f8ff97cd640 1 -- 172.21.15.120:0/3847716267 shutdown_connections 2024-09-19T09:18:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.263+0000 7f8ff97cd640 1 --2- 172.21.15.120:0/3847716267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ff4107830 0x7f8ff4109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.263+0000 7f8ff97cd640 1 --2- 172.21.15.120:0/3847716267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ff4104f00 0x7f8ff41072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.263+0000 7f8ff97cd640 1 --2- 172.21.15.120:0/3847716267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ff41025d0 0x7f8ff41049c0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.263+0000 7f8ff97cd640 1 -- 172.21.15.120:0/3847716267 >> 172.21.15.120:0/3847716267 conn(0x7f8ff40fc1b0 msgr2=0x7f8ff40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.263+0000 7f8ff97cd640 1 -- 172.21.15.120:0/3847716267 shutdown_connections 2024-09-19T09:18:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.263+0000 7f8ff97cd640 1 -- 172.21.15.120:0/3847716267 wait complete. 2024-09-19T09:18:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.263+0000 7f8ff97cd640 1 Processor -- start 2024-09-19T09:18:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.264+0000 7f8ff97cd640 1 -- start start 2024-09-19T09:18:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.264+0000 7f8ff97cd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ff41025d0 0x7f8ff4197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.264+0000 7f8ff97cd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ff4104f00 0x7f8ff4198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.264+0000 7f8ff97cd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ff4107830 0x7f8ff419f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.264+0000 7f8ff97cd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff410d810 con 0x7f8ff41025d0 2024-09-19T09:18:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.264+0000 7f8ff97cd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff410d690 con 0x7f8ff4107830 2024-09-19T09:18:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.264+0000 7f8ff97cd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff410d990 con 0x7f8ff4104f00 2024-09-19T09:18:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.264+0000 7f8ff2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ff41025d0 0x7f8ff4197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.264+0000 7f8ff2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ff41025d0 0x7f8ff4197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34546/0 (socket says 172.21.15.120:34546) 2024-09-19T09:18:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.264+0000 7f8ff2ffd640 1 -- 172.21.15.120:0/3550470365 learned_addr learned my addr 172.21.15.120:0/3550470365 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.264+0000 7f8ff37fe640 1 --2- 172.21.15.120:0/3550470365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ff4107830 0x7f8ff419f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.264+0000 7f8ff27fc640 1 --2- 172.21.15.120:0/3550470365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ff4104f00 0x7f8ff4198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.264+0000 7f8ff2ffd640 1 -- 172.21.15.120:0/3550470365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ff4104f00 msgr2=0x7f8ff4198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.264+0000 7f8ff2ffd640 1 --2- 172.21.15.120:0/3550470365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ff4104f00 0x7f8ff4198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.264+0000 7f8ff2ffd640 1 -- 172.21.15.120:0/3550470365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ff4107830 msgr2=0x7f8ff419f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.264+0000 7f8ff2ffd640 1 --2- 172.21.15.120:0/3550470365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ff4107830 0x7f8ff419f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.264+0000 7f8ff2ffd640 1 -- 172.21.15.120:0/3550470365 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff419fa10 con 0x7f8ff41025d0 2024-09-19T09:18:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.264+0000 7f8ff2ffd640 1 --2- 172.21.15.120:0/3550470365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ff41025d0 0x7f8ff4197df0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4031a40 tx=0x7f8fe4031ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.265+0000 7f8febfff640 1 -- 172.21.15.120:0/3550470365 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe402faa0 con 0x7f8ff41025d0 2024-09-19T09:18:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.265+0000 7f8febfff640 1 -- 172.21.15.120:0/3550470365 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fe402fc40 con 0x7f8ff41025d0 2024-09-19T09:18:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.265+0000 7f8ff97cd640 1 -- 172.21.15.120:0/3550470365 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff419fca0 con 0x7f8ff41025d0 2024-09-19T09:18:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.265+0000 7f8febfff640 1 -- 172.21.15.120:0/3550470365 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe403a790 con 0x7f8ff41025d0 2024-09-19T09:18:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.265+0000 7f8ff97cd640 1 -- 172.21.15.120:0/3550470365 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff41a0190 con 0x7f8ff41025d0 2024-09-19T09:18:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.267+0000 7f8febfff640 1 -- 172.21.15.120:0/3550470365 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f8fe4030b30 con 0x7f8ff41025d0 2024-09-19T09:18:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.267+0000 7f8ffc642740 1 Processor -- start 2024-09-19T09:18:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.267+0000 7f8ffc642740 1 -- start start 2024-09-19T09:18:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.268+0000 7f8ffc642740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556093d979a0 0x556093d97da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.268+0000 7f8ffc642740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556093d982e0 0x556093d9cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.268+0000 7f8ffc642740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556093d9d370 0x556093d9f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.268+0000 7f8ffc642740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556093cfb2f0 con 0x556093d979a0 2024-09-19T09:18:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.268+0000 7f8ffc642740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556093ca3270 con 0x556093d982e0 2024-09-19T09:18:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.268+0000 7f8ffc642740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556093cf72b0 con 0x556093d9d370 2024-09-19T09:18:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.268+0000 7f8febfff640 1 --2- 172.21.15.120:0/3550470365 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8fe00a3370 0x7f8fe00a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.268+0000 7f8ff37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556093d979a0 0x556093d97da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.268+0000 7f8ff2ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556093d982e0 0x556093d9cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.268+0000 7f8ff37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556093d979a0 0x556093d97da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34558/0 (socket says 172.21.15.120:34558) 2024-09-19T09:18:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.268+0000 7f8ff37fe640 1 -- 172.21.15.120:0/3819537214 learned_addr learned my addr 172.21.15.120:0/3819537214 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.268+0000 7f8ff37fe640 1 --2- 172.21.15.120:0/3550470365 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8fe00a3370 0x7f8fe00a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.268+0000 7f8ff27fc640 1 --2- 172.21.15.120:0/3819537214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556093d9d370 0x556093d9f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.268+0000 7f8febfff640 1 -- 172.21.15.120:0/3550470365 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f8fe40f0d90 con 0x7f8ff41025d0 2024-09-19T09:18:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.268+0000 7f8ff37fe640 1 -- 172.21.15.120:0/3819537214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556093d9d370 msgr2=0x556093d9f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.268+0000 7f8ff37fe640 1 --2- 172.21.15.120:0/3819537214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556093d9d370 0x556093d9f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.268+0000 7f8ff37fe640 1 -- 172.21.15.120:0/3819537214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556093d982e0 msgr2=0x556093d9cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.268+0000 7f8ff37fe640 1 --2- 172.21.15.120:0/3819537214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556093d982e0 0x556093d9cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.268+0000 7f8ff37fe640 1 -- 172.21.15.120:0/3819537214 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556093ca2340 con 0x556093d979a0 2024-09-19T09:18:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.268+0000 7f8ff27fc640 1 --2- 172.21.15.120:0/3819537214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556093d9d370 0x556093d9f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:18:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.269+0000 7f8ff37fe640 1 --2- 172.21.15.120:0/3819537214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556093d979a0 0x556093d97da0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f8fec012a10 tx=0x7f8fec012ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc4ddc58c6460b12 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.269+0000 7f8ff37fe640 1 --2- 172.21.15.120:0/3550470365 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8fe00a3370 0x7f8fe00a5830 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8fec010740 tx=0x7f8fec017000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.269+0000 7f8fe97fa640 1 -- 172.21.15.120:0/3819537214 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec024070 con 0x556093d979a0 2024-09-19T09:18:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.269+0000 7f8fe97fa640 1 -- 172.21.15.120:0/3819537214 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fec012510 con 0x556093d979a0 2024-09-19T09:18:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.269+0000 7f8fe97fa640 1 -- 172.21.15.120:0/3819537214 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec004bd0 con 0x556093d979a0 2024-09-19T09:18:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.269+0000 7f8ffc642740 1 -- 172.21.15.120:0/3819537214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556093d979a0 msgr2=0x556093d97da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.269+0000 7f8ffc642740 1 --2- 172.21.15.120:0/3819537214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556093d979a0 0x556093d97da0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f8fec012a10 tx=0x7f8fec012ed0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.270+0000 7f8ffc642740 1 -- 172.21.15.120:0/3819537214 shutdown_connections 2024-09-19T09:18:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.270+0000 7f8ffc642740 1 --2- 172.21.15.120:0/3819537214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556093d9d370 0x556093d9f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.270+0000 7f8ffc642740 1 --2- 172.21.15.120:0/3819537214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556093d982e0 0x556093d9cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.270+0000 7f8ffc642740 1 --2- 172.21.15.120:0/3819537214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556093d979a0 0x556093d97da0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.270+0000 7f8ffc642740 1 -- 172.21.15.120:0/3819537214 >> 172.21.15.120:0/3819537214 conn(0x556093d93010 msgr2=0x556093d93410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.270+0000 7f8ffc642740 1 -- 172.21.15.120:0/3819537214 shutdown_connections 2024-09-19T09:18:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.270+0000 7f8ffc642740 1 -- 172.21.15.120:0/3819537214 wait complete. 2024-09-19T09:18:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.270+0000 7f8ffc642740 1 -- start start 2024-09-19T09:18:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.270+0000 7f8ffc642740 1 Processor -- start 2024-09-19T09:18:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.270+0000 7f8ffc642740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556093d979a0 0x556093da1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.270+0000 7f8ffc642740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556093d982e0 0x556093da2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.270+0000 7f8ffc642740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556093d9d370 0x556093da9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.270+0000 7f8ffc642740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556093da02e0 con 0x556093d982e0 2024-09-19T09:18:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.270+0000 7f8ffc642740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556093c84290 con 0x556093d979a0 2024-09-19T09:18:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.270+0000 7f8ffc642740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556093da0460 con 0x556093d9d370 2024-09-19T09:18:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.270+0000 7f8ffc642740 10 client.0 _ll_register_callbacks cb 0x556093cb5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.270+0000 7f8ff27fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556093d982e0 0x556093da2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.270+0000 7f8ff27fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556093d982e0 0x556093da2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34564/0 (socket says 172.21.15.120:34564) 2024-09-19T09:18:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.270+0000 7f8ff27fc640 1 -- 172.21.15.120:0/3500868479 learned_addr learned my addr 172.21.15.120:0/3500868479 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.271+0000 7f8ff2ffd640 1 --2- 172.21.15.120:0/3500868479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556093d979a0 0x556093da1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.271+0000 7f8ff37fe640 1 --2- 172.21.15.120:0/3500868479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556093d9d370 0x556093da9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.271+0000 7f8ff27fc640 1 -- 172.21.15.120:0/3500868479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556093d9d370 msgr2=0x556093da9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.271+0000 7f8ff27fc640 1 --2- 172.21.15.120:0/3500868479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556093d9d370 0x556093da9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.271+0000 7f8ff27fc640 1 -- 172.21.15.120:0/3500868479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556093d979a0 msgr2=0x556093da1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.271+0000 7f8ff27fc640 1 --2- 172.21.15.120:0/3500868479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556093d979a0 0x556093da1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.271+0000 7f8ff27fc640 1 -- 172.21.15.120:0/3500868479 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556093a0e210 con 0x556093d982e0 2024-09-19T09:18:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.271+0000 7f8ff37fe640 1 --2- 172.21.15.120:0/3500868479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556093d9d370 0x556093da9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:18:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.271+0000 7f8ff27fc640 1 --2- 172.21.15.120:0/3500868479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556093d982e0 0x556093da2100 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc00bde0 tx=0x7f8fdc0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1591fa3fef76839 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.271+0000 7f8fda7fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:18:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.271+0000 7f8fda7fc640 1 -- 172.21.15.120:0/3500868479 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc004220 con 0x556093d982e0 2024-09-19T09:18:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.271+0000 7f8fda7fc640 1 -- 172.21.15.120:0/3500868479 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f8fdc0043c0 con 0x556093d982e0 2024-09-19T09:18:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.271+0000 7f8ffc642740 1 -- 172.21.15.120:0/3500868479 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fec012510 con 0x556093d982e0 2024-09-19T09:18:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.271+0000 7f8fda7fc640 1 -- 172.21.15.120:0/3500868479 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8fdc0035f0 con 0x556093d982e0 2024-09-19T09:18:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.271+0000 7f8ffc642740 10 client.20829 fetch_fsmap learned FSMap version 37 2024-09-19T09:18:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.271+0000 7f8ffc642740 1 -- 172.21.15.120:0/3500868479 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556093ada620 con 0x556093d982e0 2024-09-19T09:18:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.271+0000 7f8fda7fc640 1 -- 172.21.15.120:0/3500868479 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fec012510 con 0x556093d982e0 2024-09-19T09:18:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.271+0000 7f8fda7fc640 1 -- 172.21.15.120:0/3500868479 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f8fdc01ba50 con 0x556093d982e0 2024-09-19T09:18:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.271+0000 7f8ffc642740 10 client.20829 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T09:18:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.271+0000 7f8ffc642740 10 client.20829 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:18:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.271+0000 7f8ffc642740 20 client.20829 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.271+0000 7f8ffc642740 1 --2- 172.21.15.120:0/3500868479 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556093dad910 0x556093dafd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.272+0000 7f8ffc642740 4 client.20829 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:18:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.272+0000 7f8ffc642740 1 -- 172.21.15.120:0/3500868479 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556093d08a20 con 0x556093dad910 2024-09-19T09:18:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.272+0000 7f8ff2ffd640 1 --2- 172.21.15.120:0/3500868479 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556093dad910 0x556093dafd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.272+0000 7f8ff2ffd640 1 --2- 172.21.15.120:0/3500868479 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556093dad910 0x556093dafd70 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=514c15756efaae74 server_cookie=48fa69ebbc977207 in_seq=0 out_seq=0 2024-09-19T09:18:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.272+0000 7f8fda7fc640 10 client.20829 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.273+0000 7f8fda7fc640 1 -- 172.21.15.120:0/3500868479 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556093d08a20 con 0x556093dad910 2024-09-19T09:18:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.273+0000 7f8fda7fc640 10 client.20829 handle_command_reply: tid=0 2024-09-19T09:18:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.273+0000 7f8ffc642740 1 client.20829 shutdown 2024-09-19T09:18:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.273+0000 7f8ffc642740 1 -- 172.21.15.120:0/3500868479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556093d982e0 msgr2=0x556093da2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.273+0000 7f8ffc642740 1 --2- 172.21.15.120:0/3500868479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556093d982e0 0x556093da2100 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc00bde0 tx=0x7f8fdc0117b0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.273+0000 7f8ffc642740 1 -- 172.21.15.120:0/3500868479 shutdown_connections 2024-09-19T09:18:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.273+0000 7f8ffc642740 1 --2- 172.21.15.120:0/3500868479 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556093dad910 0x556093dafd70 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.273+0000 7f8ffc642740 1 --2- 172.21.15.120:0/3500868479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556093d9d370 0x556093da9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.273+0000 7f8fda7fc640 0 client.20829 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.273+0000 7f8ffc642740 1 --2- 172.21.15.120:0/3500868479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556093d982e0 0x556093da2100 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.273+0000 7f8ffc642740 1 --2- 172.21.15.120:0/3500868479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556093d979a0 0x556093da1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.273+0000 7f8ffc642740 1 -- 172.21.15.120:0/3500868479 >> 172.21.15.120:0/3500868479 conn(0x556093d93010 msgr2=0x556093d94820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.273+0000 7f8ffc642740 1 -- 172.21.15.120:0/3500868479 shutdown_connections 2024-09-19T09:18:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.273+0000 7f8ffc642740 1 -- 172.21.15.120:0/3500868479 wait complete. 2024-09-19T09:18:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.273+0000 7f8ffc642740 20 client.20829 trim_cache size 0 max 16384 2024-09-19T09:18:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.288+0000 7f8ffc642740 1 Processor -- start 2024-09-19T09:18:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.288+0000 7f8ffc642740 1 -- start start 2024-09-19T09:18:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.288+0000 7f8ffc642740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556093d979a0 0x556093d946b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.288+0000 7f8ffc642740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556093d982e0 0x556093da9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.288+0000 7f8ffc642740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556093d9d370 0x556093db4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.288+0000 7f8ffc642740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41a8f30 con 0x556093d982e0 2024-09-19T09:18:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.288+0000 7f8ffc642740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41aaad0 con 0x556093d9d370 2024-09-19T09:18:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.288+0000 7f8ffc642740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41a9600 con 0x556093d979a0 2024-09-19T09:18:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.288+0000 7f8ff27fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556093d982e0 0x556093da9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.288+0000 7f8ff27fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556093d982e0 0x556093da9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34574/0 (socket says 172.21.15.120:34574) 2024-09-19T09:18:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.288+0000 7f8ff27fc640 1 -- 172.21.15.120:0/4278621385 learned_addr learned my addr 172.21.15.120:0/4278621385 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.288+0000 7f8ff2ffd640 1 --2- 172.21.15.120:0/4278621385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556093d979a0 0x556093d946b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.288+0000 7f8ff37fe640 1 --2- 172.21.15.120:0/4278621385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556093d9d370 0x556093db4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.288+0000 7f8ff27fc640 1 -- 172.21.15.120:0/4278621385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556093d979a0 msgr2=0x556093d946b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.288+0000 7f8ff27fc640 1 --2- 172.21.15.120:0/4278621385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556093d979a0 0x556093d946b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.288+0000 7f8ff27fc640 1 -- 172.21.15.120:0/4278621385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556093d9d370 msgr2=0x556093db4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.288+0000 7f8ff27fc640 1 --2- 172.21.15.120:0/4278621385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556093d9d370 0x556093db4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.288+0000 7f8ff27fc640 1 -- 172.21.15.120:0/4278621385 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556093d9f760 con 0x556093d982e0 2024-09-19T09:18:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.288+0000 7f8ff37fe640 1 --2- 172.21.15.120:0/4278621385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556093d9d370 0x556093db4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:18:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.289+0000 7f8ff27fc640 1 --2- 172.21.15.120:0/4278621385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556093d982e0 0x556093da9100 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc000c00 tx=0x7f8fdc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f934784be63e41 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.289+0000 7f8fe8ff9640 1 -- 172.21.15.120:0/4278621385 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc004000 con 0x556093d982e0 2024-09-19T09:18:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.289+0000 7f8fe8ff9640 1 -- 172.21.15.120:0/4278621385 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fdc02c040 con 0x556093d982e0 2024-09-19T09:18:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.289+0000 7f8fe8ff9640 1 -- 172.21.15.120:0/4278621385 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc012c30 con 0x556093d982e0 2024-09-19T09:18:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.289+0000 7f8ffc642740 1 -- 172.21.15.120:0/4278621385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556093d982e0 msgr2=0x556093da9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.289+0000 7f8ffc642740 1 --2- 172.21.15.120:0/4278621385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556093d982e0 0x556093da9100 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc000c00 tx=0x7f8fdc00b8e0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.290+0000 7f8ffc642740 1 -- 172.21.15.120:0/4278621385 shutdown_connections 2024-09-19T09:18:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.290+0000 7f8ffc642740 1 --2- 172.21.15.120:0/4278621385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556093d9d370 0x556093db4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.290+0000 7f8ffc642740 1 --2- 172.21.15.120:0/4278621385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556093d982e0 0x556093da9100 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.290+0000 7f8ffc642740 1 --2- 172.21.15.120:0/4278621385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556093d979a0 0x556093d946b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.290+0000 7f8ffc642740 1 -- 172.21.15.120:0/4278621385 >> 172.21.15.120:0/4278621385 conn(0x556093d93010 msgr2=0x556093d93fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.290+0000 7f8ffc642740 1 -- 172.21.15.120:0/4278621385 shutdown_connections 2024-09-19T09:18:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.290+0000 7f8ffc642740 1 -- 172.21.15.120:0/4278621385 wait complete. 2024-09-19T09:18:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.290+0000 7f8ffc642740 1 -- start start 2024-09-19T09:18:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.290+0000 7f8ffc642740 1 Processor -- start 2024-09-19T09:18:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.290+0000 7f8ffc642740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556093d979a0 0x556093da8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.290+0000 7f8ffc642740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556093d982e0 0x556093da72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.290+0000 7f8ffc642740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556093d9d370 0x556093da77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.290+0000 7f8ffc642740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41aa400 con 0x556093d979a0 2024-09-19T09:18:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.290+0000 7f8ffc642740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41a9cd0 con 0x556093d9d370 2024-09-19T09:18:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.290+0000 7f8ffc642740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41a6ef0 con 0x556093d982e0 2024-09-19T09:18:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.290+0000 7f8ffc642740 10 client.0 _ll_register_callbacks cb 0x7f8fdc005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.290+0000 7f8ff2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556093d979a0 0x556093da8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.290+0000 7f8ff2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556093d979a0 0x556093da8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34590/0 (socket says 172.21.15.120:34590) 2024-09-19T09:18:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.290+0000 7f8ff2ffd640 1 -- 172.21.15.120:0/3976765554 learned_addr learned my addr 172.21.15.120:0/3976765554 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.290+0000 7f8ff27fc640 1 --2- 172.21.15.120:0/3976765554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556093d982e0 0x556093da72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.291+0000 7f8ff2ffd640 1 -- 172.21.15.120:0/3976765554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556093d982e0 msgr2=0x556093da72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.291+0000 7f8ff2ffd640 1 --2- 172.21.15.120:0/3976765554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556093d982e0 0x556093da72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.291+0000 7f8ff2ffd640 1 -- 172.21.15.120:0/3976765554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556093d9d370 msgr2=0x556093da77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.291+0000 7f8ff37fe640 1 --2- 172.21.15.120:0/3976765554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556093d9d370 0x556093da77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.291+0000 7f8ff2ffd640 1 --2- 172.21.15.120:0/3976765554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556093d9d370 0x556093da77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.291+0000 7f8ff2ffd640 1 -- 172.21.15.120:0/3976765554 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556093da7d30 con 0x556093d979a0 2024-09-19T09:18:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.291+0000 7f8ff37fe640 1 --2- 172.21.15.120:0/3976765554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556093d9d370 0x556093da77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:18:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.291+0000 7f8ff2ffd640 1 --2- 172.21.15.120:0/3976765554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556093d979a0 0x556093da8ca0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f8fe402a900 tx=0x7f8fe408f180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da15c329ad2c5855 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.291+0000 7f8fda7fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:18:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.291+0000 7f8fda7fc640 1 -- 172.21.15.120:0/3976765554 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe40bdab0 con 0x556093d979a0 2024-09-19T09:18:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.291+0000 7f8fda7fc640 1 -- 172.21.15.120:0/3976765554 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f8fe40ee9a0 con 0x556093d979a0 2024-09-19T09:18:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.291+0000 7f8ffc642740 1 -- 172.21.15.120:0/3976765554 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fdc02c040 con 0x556093d979a0 2024-09-19T09:18:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.291+0000 7f8fda7fc640 1 -- 172.21.15.120:0/3976765554 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8fe40bd3b0 con 0x556093d979a0 2024-09-19T09:18:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.291+0000 7f8ffc642740 10 client.20835 fetch_fsmap learned FSMap version 37 2024-09-19T09:18:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.291+0000 7f8ffc642740 1 -- 172.21.15.120:0/3976765554 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556093e19380 con 0x556093d979a0 2024-09-19T09:18:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.291+0000 7f8fda7fc640 1 -- 172.21.15.120:0/3976765554 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fdc02c040 con 0x556093d979a0 2024-09-19T09:18:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.291+0000 7f8fda7fc640 1 -- 172.21.15.120:0/3976765554 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f8fe40d6140 con 0x556093d979a0 2024-09-19T09:18:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.291+0000 7f8ffc642740 10 client.20835 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T09:18:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.291+0000 7f8ffc642740 10 client.20835 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:18:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.291+0000 7f8ffc642740 20 client.20835 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.291+0000 7f8ffc642740 1 --2- 172.21.15.120:0/3976765554 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556093dad910 0x556093e1bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.291+0000 7f8ffc642740 4 client.20835 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:18:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.291+0000 7f8ffc642740 1 -- 172.21.15.120:0/3976765554 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556093d02d70 con 0x556093dad910 2024-09-19T09:18:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.292+0000 7f8ff27fc640 1 --2- 172.21.15.120:0/3976765554 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556093dad910 0x556093e1bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.292+0000 7f8fda7fc640 10 client.20835 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.292+0000 7f8ff27fc640 1 --2- 172.21.15.120:0/3976765554 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556093dad910 0x556093e1bd30 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53636445770be8b9 server_cookie=9b5a644ea977d6bc in_seq=0 out_seq=0 2024-09-19T09:18:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.344+0000 7f8fda7fc640 1 -- 172.21.15.120:0/3976765554 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556093d02d70 con 0x556093dad910 2024-09-19T09:18:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.344+0000 7f8fda7fc640 10 client.20835 handle_command_reply: tid=0 2024-09-19T09:18:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.344+0000 7f8ffc642740 1 client.20835 shutdown 2024-09-19T09:18:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.345+0000 7f8ffc642740 1 -- 172.21.15.120:0/3976765554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556093d979a0 msgr2=0x556093da8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.345+0000 7f8ffc642740 1 --2- 172.21.15.120:0/3976765554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556093d979a0 0x556093da8ca0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f8fe402a900 tx=0x7f8fe408f180 comp rx=0 tx=0).stop 2024-09-19T09:18:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.345+0000 7f8ffc642740 1 -- 172.21.15.120:0/3976765554 shutdown_connections 2024-09-19T09:18:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.345+0000 7f8ffc642740 1 --2- 172.21.15.120:0/3976765554 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556093dad910 0x556093e1bd30 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.345+0000 7f8ffc642740 1 --2- 172.21.15.120:0/3976765554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556093d9d370 0x556093da77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.345+0000 7f8fda7fc640 0 client.20835 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.345+0000 7f8ffc642740 1 --2- 172.21.15.120:0/3976765554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556093d982e0 0x556093da72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.345+0000 7f8ffc642740 1 --2- 172.21.15.120:0/3976765554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556093d979a0 0x556093da8ca0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.345+0000 7f8ffc642740 1 -- 172.21.15.120:0/3976765554 >> 172.21.15.120:0/3976765554 conn(0x556093d93010 msgr2=0x556093d933f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.345+0000 7f8ffc642740 1 -- 172.21.15.120:0/3976765554 shutdown_connections 2024-09-19T09:18:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.345+0000 7f8ffc642740 1 -- 172.21.15.120:0/3976765554 wait complete. 2024-09-19T09:18:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.345+0000 7f8ffc642740 20 client.20835 trim_cache size 0 max 16384 2024-09-19T09:18:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:18:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:18:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:18:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:18:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.345+0000 7f8ff97cd640 1 -- 172.21.15.120:0/3550470365 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8fe00a3370 msgr2=0x7f8fe00a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.345+0000 7f8ff97cd640 1 --2- 172.21.15.120:0/3550470365 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8fe00a3370 0x7f8fe00a5830 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8fec010740 tx=0x7f8fec017000 comp rx=0 tx=0).stop 2024-09-19T09:18:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.345+0000 7f8ff97cd640 1 -- 172.21.15.120:0/3550470365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ff41025d0 msgr2=0x7f8ff4197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.345+0000 7f8ff97cd640 1 --2- 172.21.15.120:0/3550470365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ff41025d0 0x7f8ff4197df0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4031a40 tx=0x7f8fe4031ec0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.345+0000 7f8ff97cd640 1 -- 172.21.15.120:0/3550470365 shutdown_connections 2024-09-19T09:18:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.345+0000 7f8ff97cd640 1 --2- 172.21.15.120:0/3550470365 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8fe00a3370 0x7f8fe00a5830 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.345+0000 7f8ff97cd640 1 --2- 172.21.15.120:0/3550470365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ff4107830 0x7f8ff419f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.346+0000 7f8ff97cd640 1 --2- 172.21.15.120:0/3550470365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ff4104f00 0x7f8ff4198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.346+0000 7f8ff97cd640 1 --2- 172.21.15.120:0/3550470365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ff41025d0 0x7f8ff4197df0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.346+0000 7f8ff97cd640 1 -- 172.21.15.120:0/3550470365 >> 172.21.15.120:0/3550470365 conn(0x7f8ff40fc1b0 msgr2=0x7f8ff4102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.346+0000 7f8ff97cd640 1 -- 172.21.15.120:0/3550470365 shutdown_connections 2024-09-19T09:18:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:46.346+0000 7f8ff97cd640 1 -- 172.21.15.120:0/3550470365 wait complete. 2024-09-19T09:18:46.360 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 2.62 MB/sec execute 41 sec latency 15690.493 ms 2024-09-19T09:18:47.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:46 smithi120 ceph-mon[31150]: pgmap v822: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 50 KiB/s wr, 84 op/s 2024-09-19T09:18:47.360 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 2.56 MB/sec execute 42 sec latency 16690.593 ms 2024-09-19T09:18:47.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:46 smithi196 ceph-mon[35150]: pgmap v822: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 50 KiB/s wr, 84 op/s 2024-09-19T09:18:47.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:46 smithi153 ceph-mon[35071]: pgmap v822: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 50 KiB/s wr, 84 op/s 2024-09-19T09:18:48.360 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 2.50 MB/sec execute 43 sec latency 17690.688 ms 2024-09-19T09:18:49.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:48 smithi120 ceph-mon[31150]: pgmap v823: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 51 KiB/s wr, 74 op/s 2024-09-19T09:18:49.360 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 2.44 MB/sec execute 44 sec latency 18690.802 ms 2024-09-19T09:18:49.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:48 smithi196 ceph-mon[35150]: pgmap v823: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 51 KiB/s wr, 74 op/s 2024-09-19T09:18:49.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:48 smithi153 ceph-mon[35071]: pgmap v823: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 51 KiB/s wr, 74 op/s 2024-09-19T09:18:50.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:49 smithi120 ceph-mon[31150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:18:50.360 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 2.39 MB/sec execute 45 sec latency 19690.869 ms 2024-09-19T09:18:50.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:49 smithi196 ceph-mon[35150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:18:50.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:49 smithi153 ceph-mon[35071]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:18:51.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:50 smithi120 ceph-mon[31150]: pgmap v824: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 27 op/s 2024-09-19T09:18:51.360 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 2.33 MB/sec execute 46 sec latency 20690.925 ms 2024-09-19T09:18:51.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:50 smithi196 ceph-mon[35150]: pgmap v824: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 27 op/s 2024-09-19T09:18:51.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:50 smithi153 ceph-mon[35071]: pgmap v824: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 27 op/s 2024-09-19T09:18:52.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:51 smithi120 ceph-mon[31150]: pgmap v825: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 25 op/s 2024-09-19T09:18:52.360 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 2.28 MB/sec execute 47 sec latency 21691.005 ms 2024-09-19T09:18:52.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:51 smithi196 ceph-mon[35150]: pgmap v825: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 25 op/s 2024-09-19T09:18:52.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:51 smithi153 ceph-mon[35071]: pgmap v825: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 25 op/s 2024-09-19T09:18:52.788 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:18:53.053 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:18:53.360 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 2.24 MB/sec execute 48 sec latency 22691.047 ms 2024-09-19T09:18:53.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.585+0000 7f7a2a66c640 1 -- 172.21.15.120:0/2626854063 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a24106a80 msgr2=0x7f7a24106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:53.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.585+0000 7f7a2a66c640 1 --2- 172.21.15.120:0/2626854063 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a24106a80 0x7f7a24106e60 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f7a14009930 tx=0x7f7a1402a420 comp rx=0 tx=0).stop 2024-09-19T09:18:53.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.585+0000 7f7a2a66c640 1 -- 172.21.15.120:0/2626854063 shutdown_connections 2024-09-19T09:18:53.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.585+0000 7f7a2a66c640 1 --2- 172.21.15.120:0/2626854063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a24101420 0x7f7a2410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.585+0000 7f7a2a66c640 1 --2- 172.21.15.120:0/2626854063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a24100a80 0x7f7a24100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.585+0000 7f7a2a66c640 1 --2- 172.21.15.120:0/2626854063 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a24106a80 0x7f7a24106e60 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.585+0000 7f7a2a66c640 1 -- 172.21.15.120:0/2626854063 >> 172.21.15.120:0/2626854063 conn(0x7f7a240fc7a0 msgr2=0x7f7a240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:53.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.585+0000 7f7a2a66c640 1 -- 172.21.15.120:0/2626854063 shutdown_connections 2024-09-19T09:18:53.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.585+0000 7f7a2a66c640 1 -- 172.21.15.120:0/2626854063 wait complete. 2024-09-19T09:18:53.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.586+0000 7f7a2a66c640 1 Processor -- start 2024-09-19T09:18:53.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.586+0000 7f7a2a66c640 1 -- start start 2024-09-19T09:18:53.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.586+0000 7f7a2a66c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a24100a80 0x7f7a241992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:53.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.586+0000 7f7a2a66c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a24101420 0x7f7a24199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:53.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.586+0000 7f7a2a66c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a24106a80 0x7f7a241a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:53.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.586+0000 7f7a2a66c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7a24110e80 con 0x7f7a24100a80 2024-09-19T09:18:53.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.586+0000 7f7a2a66c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7a24110d00 con 0x7f7a24101420 2024-09-19T09:18:53.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.586+0000 7f7a2a66c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7a24111000 con 0x7f7a24106a80 2024-09-19T09:18:53.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.586+0000 7f7a23fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a24100a80 0x7f7a241992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:53.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.586+0000 7f7a23fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a24100a80 0x7f7a241992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37534/0 (socket says 172.21.15.120:37534) 2024-09-19T09:18:53.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.586+0000 7f7a23fff640 1 -- 172.21.15.120:0/737037558 learned_addr learned my addr 172.21.15.120:0/737037558 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:53.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.586+0000 7f7a23fff640 1 -- 172.21.15.120:0/737037558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a24106a80 msgr2=0x7f7a241a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:53.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.587+0000 7f7a23fff640 1 --2- 172.21.15.120:0/737037558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a24106a80 0x7f7a241a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.587+0000 7f7a23fff640 1 -- 172.21.15.120:0/737037558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a24101420 msgr2=0x7f7a24199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:53.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.587+0000 7f7a237fe640 1 --2- 172.21.15.120:0/737037558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a24101420 0x7f7a24199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:53.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.587+0000 7f7a23fff640 1 --2- 172.21.15.120:0/737037558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a24101420 0x7f7a24199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.587+0000 7f7a23fff640 1 -- 172.21.15.120:0/737037558 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a241a48b0 con 0x7f7a24100a80 2024-09-19T09:18:53.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.587+0000 7f7a23fff640 1 --2- 172.21.15.120:0/737037558 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a24100a80 0x7f7a241992f0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f7a14009a50 tx=0x7f7a14004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:53.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.587+0000 7f7a217fa640 1 -- 172.21.15.120:0/737037558 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1403f030 con 0x7f7a24100a80 2024-09-19T09:18:53.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.587+0000 7f7a2a66c640 1 -- 172.21.15.120:0/737037558 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a241a4a80 con 0x7f7a24100a80 2024-09-19T09:18:53.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.587+0000 7f7a217fa640 1 -- 172.21.15.120:0/737037558 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a14003ce0 con 0x7f7a24100a80 2024-09-19T09:18:53.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.587+0000 7f7a217fa640 1 -- 172.21.15.120:0/737037558 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a14035070 con 0x7f7a24100a80 2024-09-19T09:18:53.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.587+0000 7f7a2a66c640 1 -- 172.21.15.120:0/737037558 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a241a4f30 con 0x7f7a24100a80 2024-09-19T09:18:53.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.589+0000 7f7a217fa640 1 -- 172.21.15.120:0/737037558 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f7a14042030 con 0x7f7a24100a80 2024-09-19T09:18:53.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.589+0000 7f7a2ca45740 1 Processor -- start 2024-09-19T09:18:53.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.589+0000 7f7a2ca45740 1 -- start start 2024-09-19T09:18:53.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.590+0000 7f7a2ca45740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56046e8134a0 0x56046e813880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:53.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.590+0000 7f7a2ca45740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56046e813dc0 0x56046e81e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:53.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.590+0000 7f7a2ca45740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56046e81e680 0x56046e820a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:53.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.590+0000 7f7a2ca45740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56046e7792e0 con 0x56046e813dc0 2024-09-19T09:18:53.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.590+0000 7f7a2ca45740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56046e720ce0 con 0x56046e81e680 2024-09-19T09:18:53.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.590+0000 7f7a2ca45740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56046e7752a0 con 0x56046e8134a0 2024-09-19T09:18:53.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.590+0000 7f7a23fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56046e813dc0 0x56046e81e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:53.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.590+0000 7f7a23fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56046e813dc0 0x56046e81e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37550/0 (socket says 172.21.15.120:37550) 2024-09-19T09:18:53.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.590+0000 7f7a23fff640 1 -- 172.21.15.120:0/2602627403 learned_addr learned my addr 172.21.15.120:0/2602627403 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:53.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.590+0000 7f7a237fe640 1 --2- 172.21.15.120:0/2602627403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56046e81e680 0x56046e820a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:53.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.590+0000 7f7a28c09640 1 --2- 172.21.15.120:0/2602627403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56046e8134a0 0x56046e813880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:53.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.590+0000 7f7a23fff640 1 -- 172.21.15.120:0/2602627403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56046e8134a0 msgr2=0x56046e813880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:53.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.590+0000 7f7a23fff640 1 --2- 172.21.15.120:0/2602627403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56046e8134a0 0x56046e813880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.590+0000 7f7a23fff640 1 -- 172.21.15.120:0/2602627403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56046e81e680 msgr2=0x56046e820a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:53.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.590+0000 7f7a23fff640 1 --2- 172.21.15.120:0/2602627403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56046e81e680 0x56046e820a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.590+0000 7f7a23fff640 1 -- 172.21.15.120:0/2602627403 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56046e705440 con 0x56046e813dc0 2024-09-19T09:18:53.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.590+0000 7f7a217fa640 1 --2- 172.21.15.120:0/737037558 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7a100a3240 0x7f7a100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:53.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.590+0000 7f7a23fff640 1 --2- 172.21.15.120:0/2602627403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56046e813dc0 0x56046e81e140 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f7a140f8290 tx=0x7f7a140f8380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4478ed7fac7e3230 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:53.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.590+0000 7f7a28c09640 1 --2- 172.21.15.120:0/737037558 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7a100a3240 0x7f7a100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:53.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.590+0000 7f7a1a7fc640 1 -- 172.21.15.120:0/2602627403 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a140b8820 con 0x56046e813dc0 2024-09-19T09:18:53.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.591+0000 7f7a217fa640 1 -- 172.21.15.120:0/737037558 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f7a140eff60 con 0x7f7a24100a80 2024-09-19T09:18:53.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.591+0000 7f7a1a7fc640 1 -- 172.21.15.120:0/2602627403 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a140fd070 con 0x56046e813dc0 2024-09-19T09:18:53.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.591+0000 7f7a1a7fc640 1 -- 172.21.15.120:0/2602627403 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1408ec70 con 0x56046e813dc0 2024-09-19T09:18:53.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.591+0000 7f7a2ca45740 1 -- 172.21.15.120:0/2602627403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56046e813dc0 msgr2=0x56046e81e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:53.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.591+0000 7f7a28c09640 1 --2- 172.21.15.120:0/737037558 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7a100a3240 0x7f7a100a5700 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00aca0 tx=0x7f7a1c00a2b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:53.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.591+0000 7f7a2ca45740 1 --2- 172.21.15.120:0/2602627403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56046e813dc0 0x56046e81e140 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f7a140f8290 tx=0x7f7a140f8380 comp rx=0 tx=0).stop 2024-09-19T09:18:53.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.591+0000 7f7a2ca45740 1 -- 172.21.15.120:0/2602627403 shutdown_connections 2024-09-19T09:18:53.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.591+0000 7f7a2ca45740 1 --2- 172.21.15.120:0/2602627403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56046e81e680 0x56046e820a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.591+0000 7f7a2ca45740 1 --2- 172.21.15.120:0/2602627403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56046e813dc0 0x56046e81e140 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.591+0000 7f7a2ca45740 1 --2- 172.21.15.120:0/2602627403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56046e8134a0 0x56046e813880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.591+0000 7f7a2ca45740 1 -- 172.21.15.120:0/2602627403 >> 172.21.15.120:0/2602627403 conn(0x56046e80fa30 msgr2=0x56046e80fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:53.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.591+0000 7f7a2ca45740 1 -- 172.21.15.120:0/2602627403 shutdown_connections 2024-09-19T09:18:53.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.591+0000 7f7a2ca45740 1 -- 172.21.15.120:0/2602627403 wait complete. 2024-09-19T09:18:53.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.592+0000 7f7a2ca45740 1 -- start start 2024-09-19T09:18:53.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.592+0000 7f7a2ca45740 1 Processor -- start 2024-09-19T09:18:53.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.592+0000 7f7a2ca45740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56046e8134a0 0x56046e81df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:53.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.592+0000 7f7a2ca45740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56046e813dc0 0x56046e81b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:53.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.592+0000 7f7a2ca45740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56046e81e680 0x56046e81b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:53.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.592+0000 7f7a2ca45740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56046e821010 con 0x56046e813dc0 2024-09-19T09:18:53.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.592+0000 7f7a2ca45740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56046e734050 con 0x56046e81e680 2024-09-19T09:18:53.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.592+0000 7f7a2ca45740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56046e821190 con 0x56046e8134a0 2024-09-19T09:18:53.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.592+0000 7f7a2ca45740 10 client.0 _ll_register_callbacks cb 0x56046e774160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:53.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.592+0000 7f7a237fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56046e813dc0 0x56046e81b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:53.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.592+0000 7f7a237fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56046e813dc0 0x56046e81b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37552/0 (socket says 172.21.15.120:37552) 2024-09-19T09:18:53.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.592+0000 7f7a237fe640 1 -- 172.21.15.120:0/2904675667 learned_addr learned my addr 172.21.15.120:0/2904675667 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:53.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.592+0000 7f7a28c09640 1 --2- 172.21.15.120:0/2904675667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56046e81e680 0x56046e81b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:53.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.593+0000 7f7a23fff640 1 --2- 172.21.15.120:0/2904675667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56046e8134a0 0x56046e81df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:53.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.593+0000 7f7a237fe640 1 -- 172.21.15.120:0/2904675667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56046e8134a0 msgr2=0x56046e81df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:53.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.593+0000 7f7a237fe640 1 --2- 172.21.15.120:0/2904675667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56046e8134a0 0x56046e81df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.593+0000 7f7a237fe640 1 -- 172.21.15.120:0/2904675667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56046e81e680 msgr2=0x56046e81b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:53.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.593+0000 7f7a237fe640 1 --2- 172.21.15.120:0/2904675667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56046e81e680 0x56046e81b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.593+0000 7f7a237fe640 1 -- 172.21.15.120:0/2904675667 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56046e4ea3c0 con 0x56046e813dc0 2024-09-19T09:18:53.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.593+0000 7f7a23fff640 1 --2- 172.21.15.120:0/2904675667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56046e8134a0 0x56046e81df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:18:53.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.593+0000 7f7a237fe640 1 --2- 172.21.15.120:0/2904675667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56046e813dc0 0x56046e81b200 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c0106e0 tx=0x7f7a0c010ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeca633b974ae22e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:53.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.593+0000 7f7a037fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:18:53.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.593+0000 7f7a037fe640 1 -- 172.21.15.120:0/2904675667 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c01d060 con 0x56046e813dc0 2024-09-19T09:18:53.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.593+0000 7f7a037fe640 1 -- 172.21.15.120:0/2904675667 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f7a0c00eda0 con 0x56046e813dc0 2024-09-19T09:18:53.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.593+0000 7f7a2ca45740 1 -- 172.21.15.120:0/2904675667 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a140fd070 con 0x56046e813dc0 2024-09-19T09:18:53.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.593+0000 7f7a037fe640 1 -- 172.21.15.120:0/2904675667 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7a0c018430 con 0x56046e813dc0 2024-09-19T09:18:53.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.593+0000 7f7a2ca45740 10 client.20847 fetch_fsmap learned FSMap version 39 2024-09-19T09:18:53.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.593+0000 7f7a2ca45740 1 -- 172.21.15.120:0/2904675667 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56046e82c930 con 0x56046e813dc0 2024-09-19T09:18:53.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.593+0000 7f7a037fe640 1 -- 172.21.15.120:0/2904675667 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a140fd070 con 0x56046e813dc0 2024-09-19T09:18:53.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.593+0000 7f7a037fe640 1 -- 172.21.15.120:0/2904675667 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f7a0c022070 con 0x56046e813dc0 2024-09-19T09:18:53.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.593+0000 7f7a2ca45740 10 client.20847 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:18:53.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.593+0000 7f7a2ca45740 10 client.20847 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:18:53.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.593+0000 7f7a2ca45740 20 client.20847 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:53.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.594+0000 7f7a2ca45740 1 --2- 172.21.15.120:0/2904675667 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56046e82de40 0x56046e8302e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:53.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.594+0000 7f7a2ca45740 4 client.20847 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:18:53.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.594+0000 7f7a2ca45740 1 -- 172.21.15.120:0/2904675667 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56046e786850 con 0x56046e82de40 2024-09-19T09:18:53.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.594+0000 7f7a23fff640 1 --2- 172.21.15.120:0/2904675667 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56046e82de40 0x56046e8302e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:53.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.594+0000 7f7a037fe640 10 client.20847 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:53.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.594+0000 7f7a23fff640 1 --2- 172.21.15.120:0/2904675667 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56046e82de40 0x56046e8302e0 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1563c8403e513bbd server_cookie=4762b02f22b04a68 in_seq=0 out_seq=0 2024-09-19T09:18:53.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.595+0000 7f7a037fe640 1 -- 172.21.15.120:0/2904675667 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56046e786850 con 0x56046e82de40 2024-09-19T09:18:53.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.595+0000 7f7a037fe640 10 client.20847 handle_command_reply: tid=0 2024-09-19T09:18:53.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.595+0000 7f7a2ca45740 1 client.20847 shutdown 2024-09-19T09:18:53.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.595+0000 7f7a2ca45740 1 -- 172.21.15.120:0/2904675667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56046e813dc0 msgr2=0x56046e81b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:53.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.595+0000 7f7a2ca45740 1 --2- 172.21.15.120:0/2904675667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56046e813dc0 0x56046e81b200 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c0106e0 tx=0x7f7a0c010ba0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.595+0000 7f7a2ca45740 1 -- 172.21.15.120:0/2904675667 shutdown_connections 2024-09-19T09:18:53.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.595+0000 7f7a2ca45740 1 --2- 172.21.15.120:0/2904675667 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56046e82de40 0x56046e8302e0 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.595+0000 7f7a2ca45740 1 --2- 172.21.15.120:0/2904675667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56046e81e680 0x56046e81b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.595+0000 7f7a2ca45740 1 --2- 172.21.15.120:0/2904675667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56046e813dc0 0x56046e81b200 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.595+0000 7f7a2ca45740 1 --2- 172.21.15.120:0/2904675667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56046e8134a0 0x56046e81df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.595+0000 7f7a2ca45740 1 -- 172.21.15.120:0/2904675667 >> 172.21.15.120:0/2904675667 conn(0x56046e80fa30 msgr2=0x56046e815710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:53.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.595+0000 7f7a037fe640 0 client.20847 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:53.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.595+0000 7f7a2ca45740 1 -- 172.21.15.120:0/2904675667 shutdown_connections 2024-09-19T09:18:53.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.595+0000 7f7a2ca45740 1 -- 172.21.15.120:0/2904675667 wait complete. 2024-09-19T09:18:53.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.595+0000 7f7a2ca45740 20 client.20847 trim_cache size 0 max 16384 2024-09-19T09:18:53.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.612+0000 7f7a2ca45740 1 Processor -- start 2024-09-19T09:18:53.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.612+0000 7f7a2ca45740 1 -- start start 2024-09-19T09:18:53.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.612+0000 7f7a2ca45740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56046e80fa30 0x56046e82a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:53.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.612+0000 7f7a2ca45740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56046e8134a0 0x56046e816900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:53.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.612+0000 7f7a2ca45740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56046e813dc0 0x56046e81d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:53.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.612+0000 7f7a2ca45740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7a241ad6a0 con 0x56046e813dc0 2024-09-19T09:18:53.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.612+0000 7f7a2ca45740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7a241af1e0 con 0x56046e8134a0 2024-09-19T09:18:53.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.612+0000 7f7a2ca45740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7a241add70 con 0x56046e80fa30 2024-09-19T09:18:53.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.612+0000 7f7a28c09640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56046e813dc0 0x56046e81d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:53.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.612+0000 7f7a28c09640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56046e813dc0 0x56046e81d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37564/0 (socket says 172.21.15.120:37564) 2024-09-19T09:18:53.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.612+0000 7f7a28c09640 1 -- 172.21.15.120:0/763583676 learned_addr learned my addr 172.21.15.120:0/763583676 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:53.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.612+0000 7f7a237fe640 1 --2- 172.21.15.120:0/763583676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56046e8134a0 0x56046e816900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:53.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.613+0000 7f7a23fff640 1 --2- 172.21.15.120:0/763583676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56046e80fa30 0x56046e82a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:53.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.613+0000 7f7a28c09640 1 -- 172.21.15.120:0/763583676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56046e80fa30 msgr2=0x56046e82a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:53.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.613+0000 7f7a28c09640 1 --2- 172.21.15.120:0/763583676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56046e80fa30 0x56046e82a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.613+0000 7f7a28c09640 1 -- 172.21.15.120:0/763583676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56046e8134a0 msgr2=0x56046e816900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:53.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.613+0000 7f7a28c09640 1 --2- 172.21.15.120:0/763583676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56046e8134a0 0x56046e816900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.613+0000 7f7a28c09640 1 -- 172.21.15.120:0/763583676 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56046e80f860 con 0x56046e813dc0 2024-09-19T09:18:53.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.613+0000 7f7a28c09640 1 --2- 172.21.15.120:0/763583676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56046e813dc0 0x56046e81d170 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c011540 tx=0x7f7a1c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f5853fca2b0735 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:53.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.613+0000 7f7a19ffb640 1 -- 172.21.15.120:0/763583676 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c01f900 con 0x56046e813dc0 2024-09-19T09:18:53.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.613+0000 7f7a19ffb640 1 -- 172.21.15.120:0/763583676 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a1c023070 con 0x56046e813dc0 2024-09-19T09:18:53.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.613+0000 7f7a19ffb640 1 -- 172.21.15.120:0/763583676 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c0103d0 con 0x56046e813dc0 2024-09-19T09:18:53.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.614+0000 7f7a2ca45740 1 -- 172.21.15.120:0/763583676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56046e813dc0 msgr2=0x56046e81d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:53.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.614+0000 7f7a2ca45740 1 --2- 172.21.15.120:0/763583676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56046e813dc0 0x56046e81d170 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c011540 tx=0x7f7a1c012000 comp rx=0 tx=0).stop 2024-09-19T09:18:53.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.614+0000 7f7a2ca45740 1 -- 172.21.15.120:0/763583676 shutdown_connections 2024-09-19T09:18:53.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.614+0000 7f7a2ca45740 1 --2- 172.21.15.120:0/763583676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56046e813dc0 0x56046e81d170 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.614+0000 7f7a2ca45740 1 --2- 172.21.15.120:0/763583676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56046e8134a0 0x56046e816900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.614+0000 7f7a2ca45740 1 --2- 172.21.15.120:0/763583676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56046e80fa30 0x56046e82a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.614+0000 7f7a2ca45740 1 -- 172.21.15.120:0/763583676 >> 172.21.15.120:0/763583676 conn(0x56046e81b8a0 msgr2=0x56046e81ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:53.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.614+0000 7f7a2ca45740 1 -- 172.21.15.120:0/763583676 shutdown_connections 2024-09-19T09:18:53.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.614+0000 7f7a2ca45740 1 -- 172.21.15.120:0/763583676 wait complete. 2024-09-19T09:18:53.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.614+0000 7f7a2ca45740 1 -- start start 2024-09-19T09:18:53.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.614+0000 7f7a2ca45740 1 Processor -- start 2024-09-19T09:18:53.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.615+0000 7f7a2ca45740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56046e80fa30 0x56046e822100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:53.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.615+0000 7f7a2ca45740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56046e8134a0 0x56046e822640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:53.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.615+0000 7f7a2ca45740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56046e813dc0 0x56046e822b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:53.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.615+0000 7f7a2ca45740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7a241aeb10 con 0x56046e8134a0 2024-09-19T09:18:53.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.615+0000 7f7a2ca45740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7a241ae440 con 0x56046e813dc0 2024-09-19T09:18:53.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.615+0000 7f7a2ca45740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7a241ab660 con 0x56046e80fa30 2024-09-19T09:18:53.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.615+0000 7f7a2ca45740 10 client.0 _ll_register_callbacks cb 0x7f7a0c00b560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:53.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.615+0000 7f7a237fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56046e8134a0 0x56046e822640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:53.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.615+0000 7f7a23fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56046e80fa30 0x56046e822100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:53.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.615+0000 7f7a237fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56046e8134a0 0x56046e822640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37576/0 (socket says 172.21.15.120:37576) 2024-09-19T09:18:53.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.615+0000 7f7a237fe640 1 -- 172.21.15.120:0/4184099922 learned_addr learned my addr 172.21.15.120:0/4184099922 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:53.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.615+0000 7f7a28c09640 1 --2- 172.21.15.120:0/4184099922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56046e813dc0 0x56046e822b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:53.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.615+0000 7f7a237fe640 1 -- 172.21.15.120:0/4184099922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56046e80fa30 msgr2=0x56046e822100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:53.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.615+0000 7f7a237fe640 1 --2- 172.21.15.120:0/4184099922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56046e80fa30 0x56046e822100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.615+0000 7f7a237fe640 1 -- 172.21.15.120:0/4184099922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56046e813dc0 msgr2=0x56046e822b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:53.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.615+0000 7f7a237fe640 1 --2- 172.21.15.120:0/4184099922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56046e813dc0 0x56046e822b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.615+0000 7f7a237fe640 1 -- 172.21.15.120:0/4184099922 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56046e897d20 con 0x56046e8134a0 2024-09-19T09:18:53.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.615+0000 7f7a23fff640 1 --2- 172.21.15.120:0/4184099922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56046e80fa30 0x56046e822100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:18:53.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.615+0000 7f7a237fe640 1 --2- 172.21.15.120:0/4184099922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56046e8134a0 0x56046e822640 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c023850 tx=0x7f7a0c00edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a833318b51292ce6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:53.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.615+0000 7f7a037fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:18:53.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.615+0000 7f7a037fe640 1 -- 172.21.15.120:0/4184099922 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c028040 con 0x56046e8134a0 2024-09-19T09:18:53.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.616+0000 7f7a037fe640 1 -- 172.21.15.120:0/4184099922 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f7a0c023cc0 con 0x56046e8134a0 2024-09-19T09:18:53.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.616+0000 7f7a2ca45740 1 -- 172.21.15.120:0/4184099922 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a1c023070 con 0x56046e8134a0 2024-09-19T09:18:53.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.616+0000 7f7a037fe640 1 -- 172.21.15.120:0/4184099922 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7a0c00e780 con 0x56046e8134a0 2024-09-19T09:18:53.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.616+0000 7f7a2ca45740 10 client.20853 fetch_fsmap learned FSMap version 39 2024-09-19T09:18:53.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.616+0000 7f7a2ca45740 1 -- 172.21.15.120:0/4184099922 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56046e898ff0 con 0x56046e8134a0 2024-09-19T09:18:53.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.616+0000 7f7a037fe640 1 -- 172.21.15.120:0/4184099922 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a1c023070 con 0x56046e8134a0 2024-09-19T09:18:53.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.616+0000 7f7a037fe640 1 -- 172.21.15.120:0/4184099922 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f7a0c034030 con 0x56046e8134a0 2024-09-19T09:18:53.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.616+0000 7f7a2ca45740 10 client.20853 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:18:53.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.616+0000 7f7a2ca45740 10 client.20853 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:18:53.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.616+0000 7f7a2ca45740 20 client.20853 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:53.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.616+0000 7f7a2ca45740 1 --2- 172.21.15.120:0/4184099922 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56046e81e680 0x56046e89b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:53.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.616+0000 7f7a2ca45740 4 client.20853 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:18:53.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.616+0000 7f7a2ca45740 1 -- 172.21.15.120:0/4184099922 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56046e780d60 con 0x56046e81e680 2024-09-19T09:18:53.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.616+0000 7f7a23fff640 1 --2- 172.21.15.120:0/4184099922 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56046e81e680 0x56046e89b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:53.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.617+0000 7f7a037fe640 10 client.20853 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:53.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.617+0000 7f7a23fff640 1 --2- 172.21.15.120:0/4184099922 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56046e81e680 0x56046e89b770 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41b54a374362e2e5 server_cookie=d9ffb924dface3cc in_seq=0 out_seq=0 2024-09-19T09:18:53.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.617+0000 7f7a037fe640 1 -- 172.21.15.120:0/4184099922 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56046e780d60 con 0x56046e81e680 2024-09-19T09:18:53.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.617+0000 7f7a037fe640 10 client.20853 handle_command_reply: tid=0 2024-09-19T09:18:53.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.617+0000 7f7a2ca45740 1 client.20853 shutdown 2024-09-19T09:18:53.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.617+0000 7f7a2ca45740 1 -- 172.21.15.120:0/4184099922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56046e8134a0 msgr2=0x56046e822640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:53.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.617+0000 7f7a2ca45740 1 --2- 172.21.15.120:0/4184099922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56046e8134a0 0x56046e822640 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c023850 tx=0x7f7a0c00edb0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.617+0000 7f7a2ca45740 1 -- 172.21.15.120:0/4184099922 shutdown_connections 2024-09-19T09:18:53.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.617+0000 7f7a2ca45740 1 --2- 172.21.15.120:0/4184099922 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56046e81e680 0x56046e89b770 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.617+0000 7f7a2ca45740 1 --2- 172.21.15.120:0/4184099922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56046e813dc0 0x56046e822b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.617+0000 7f7a2ca45740 1 --2- 172.21.15.120:0/4184099922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56046e8134a0 0x56046e822640 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.617+0000 7f7a2ca45740 1 --2- 172.21.15.120:0/4184099922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56046e80fa30 0x56046e822100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.617+0000 7f7a2ca45740 1 -- 172.21.15.120:0/4184099922 >> 172.21.15.120:0/4184099922 conn(0x56046e81b8a0 msgr2=0x56046e81ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:53.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.617+0000 7f7a037fe640 0 client.20853 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:53.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.617+0000 7f7a2ca45740 1 -- 172.21.15.120:0/4184099922 shutdown_connections 2024-09-19T09:18:53.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.617+0000 7f7a2ca45740 1 -- 172.21.15.120:0/4184099922 wait complete. 2024-09-19T09:18:53.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.617+0000 7f7a2ca45740 20 client.20853 trim_cache size 0 max 16384 2024-09-19T09:18:53.629 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:18:53.629 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:18:53.629 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:18:53.629 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:18:53.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.618+0000 7f7a2a66c640 1 -- 172.21.15.120:0/737037558 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7a100a3240 msgr2=0x7f7a100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:53.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.618+0000 7f7a2a66c640 1 --2- 172.21.15.120:0/737037558 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7a100a3240 0x7f7a100a5700 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00aca0 tx=0x7f7a1c00a2b0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.618+0000 7f7a2a66c640 1 -- 172.21.15.120:0/737037558 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a24100a80 msgr2=0x7f7a241992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:53.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.618+0000 7f7a2a66c640 1 --2- 172.21.15.120:0/737037558 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a24100a80 0x7f7a241992f0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f7a14009a50 tx=0x7f7a14004270 comp rx=0 tx=0).stop 2024-09-19T09:18:53.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.618+0000 7f7a2a66c640 1 -- 172.21.15.120:0/737037558 shutdown_connections 2024-09-19T09:18:53.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.618+0000 7f7a2a66c640 1 --2- 172.21.15.120:0/737037558 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7a100a3240 0x7f7a100a5700 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.618+0000 7f7a2a66c640 1 --2- 172.21.15.120:0/737037558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a24106a80 0x7f7a241a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.618+0000 7f7a2a66c640 1 --2- 172.21.15.120:0/737037558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a24101420 0x7f7a24199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.618+0000 7f7a2a66c640 1 --2- 172.21.15.120:0/737037558 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a24100a80 0x7f7a241992f0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:53.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.618+0000 7f7a2a66c640 1 -- 172.21.15.120:0/737037558 >> 172.21.15.120:0/737037558 conn(0x7f7a240fc7a0 msgr2=0x7f7a240fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:53.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.618+0000 7f7a2a66c640 1 -- 172.21.15.120:0/737037558 shutdown_connections 2024-09-19T09:18:53.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:53.618+0000 7f7a2a66c640 1 -- 172.21.15.120:0/737037558 wait complete. 2024-09-19T09:18:54.360 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 2.19 MB/sec execute 49 sec latency 23691.082 ms 2024-09-19T09:18:54.385 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:54 smithi120 ceph-mon[31150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:18:54.386 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:54 smithi120 ceph-mon[31150]: pgmap v826: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-19T09:18:54.413 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:18:54.413 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:18:54.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:54 smithi196 ceph-mon[35150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:18:54.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:54 smithi196 ceph-mon[35150]: pgmap v826: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-19T09:18:54.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:54 smithi153 ceph-mon[35071]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:18:54.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:54 smithi153 ceph-mon[35071]: pgmap v826: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-19T09:18:54.658 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:18:55.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.134+0000 7f32b8e9b640 1 -- 172.21.15.120:0/698690227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32b4106f10 msgr2=0x7f32b410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:55.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.134+0000 7f32b8e9b640 1 --2- 172.21.15.120:0/698690227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32b4106f10 0x7f32b410ea90 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f32ac009ca0 tx=0x7f32ac02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:18:55.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.134+0000 7f32b8e9b640 1 -- 172.21.15.120:0/698690227 shutdown_connections 2024-09-19T09:18:55.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.134+0000 7f32b8e9b640 1 --2- 172.21.15.120:0/698690227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32b4106f10 0x7f32b410ea90 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:55.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.134+0000 7f32b8e9b640 1 --2- 172.21.15.120:0/698690227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b4102790 0x7f32b41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:55.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.134+0000 7f32b8e9b640 1 --2- 172.21.15.120:0/698690227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32b4101de0 0x7f32b41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:55.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.134+0000 7f32b8e9b640 1 -- 172.21.15.120:0/698690227 >> 172.21.15.120:0/698690227 conn(0x7f32b40fd8c0 msgr2=0x7f32b40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:55.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.134+0000 7f32b8e9b640 1 -- 172.21.15.120:0/698690227 shutdown_connections 2024-09-19T09:18:55.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.134+0000 7f32b8e9b640 1 -- 172.21.15.120:0/698690227 wait complete. 2024-09-19T09:18:55.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.135+0000 7f32b8e9b640 1 Processor -- start 2024-09-19T09:18:55.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.135+0000 7f32b8e9b640 1 -- start start 2024-09-19T09:18:55.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.135+0000 7f32b8e9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b4101de0 0x7f32b4198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:55.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.136+0000 7f32b8e9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32b4102790 0x7f32b4198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:55.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.136+0000 7f32b8e9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32b4106f10 0x7f32b419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:55.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.136+0000 7f32b8e9b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f32b4111f70 con 0x7f32b4102790 2024-09-19T09:18:55.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.136+0000 7f32b8e9b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f32b4111df0 con 0x7f32b4101de0 2024-09-19T09:18:55.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.136+0000 7f32b8e9b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f32b41120f0 con 0x7f32b4106f10 2024-09-19T09:18:55.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.136+0000 7f32b1d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32b4102790 0x7f32b4198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:55.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.136+0000 7f32b1d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32b4102790 0x7f32b4198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37594/0 (socket says 172.21.15.120:37594) 2024-09-19T09:18:55.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.136+0000 7f32b1d9b640 1 -- 172.21.15.120:0/1761022369 learned_addr learned my addr 172.21.15.120:0/1761022369 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:55.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.136+0000 7f32b259c640 1 --2- 172.21.15.120:0/1761022369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b4101de0 0x7f32b4198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:55.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.136+0000 7f32b2d9d640 1 --2- 172.21.15.120:0/1761022369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32b4106f10 0x7f32b419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:55.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.136+0000 7f32b1d9b640 1 -- 172.21.15.120:0/1761022369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32b4106f10 msgr2=0x7f32b419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:55.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.136+0000 7f32b1d9b640 1 --2- 172.21.15.120:0/1761022369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32b4106f10 0x7f32b419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:55.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.136+0000 7f32b1d9b640 1 -- 172.21.15.120:0/1761022369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b4101de0 msgr2=0x7f32b4198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:55.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.136+0000 7f32b1d9b640 1 --2- 172.21.15.120:0/1761022369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b4101de0 0x7f32b4198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:55.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.136+0000 7f32b1d9b640 1 -- 172.21.15.120:0/1761022369 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b419d2f0 con 0x7f32b4102790 2024-09-19T09:18:55.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.136+0000 7f32b2d9d640 1 --2- 172.21.15.120:0/1761022369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32b4106f10 0x7f32b419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:18:55.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.136+0000 7f32b1d9b640 1 --2- 172.21.15.120:0/1761022369 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32b4102790 0x7f32b4198880 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f32a800c920 tx=0x7f32a800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:55.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.136+0000 7f32a37fe640 1 -- 172.21.15.120:0/1761022369 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a8007c70 con 0x7f32b4102790 2024-09-19T09:18:55.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.136+0000 7f32a37fe640 1 -- 172.21.15.120:0/1761022369 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32a8007e10 con 0x7f32b4102790 2024-09-19T09:18:55.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.136+0000 7f32b8e9b640 1 -- 172.21.15.120:0/1761022369 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b419d5e0 con 0x7f32b4102790 2024-09-19T09:18:55.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.136+0000 7f32a37fe640 1 -- 172.21.15.120:0/1761022369 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a8018890 con 0x7f32b4102790 2024-09-19T09:18:55.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.136+0000 7f32b8e9b640 1 -- 172.21.15.120:0/1761022369 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b41a4ec0 con 0x7f32b4102790 2024-09-19T09:18:55.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.137+0000 7f32b8e9b640 1 -- 172.21.15.120:0/1761022369 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32b41021c0 con 0x7f32b4102790 2024-09-19T09:18:55.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.138+0000 7f32a37fe640 1 -- 172.21.15.120:0/1761022369 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f32a8018a70 con 0x7f32b4102790 2024-09-19T09:18:55.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.139+0000 7f32a37fe640 1 --2- 172.21.15.120:0/1761022369 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f329c0a3240 0x7f329c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:55.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.139+0000 7f32b259c640 1 --2- 172.21.15.120:0/1761022369 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f329c0a3240 0x7f329c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:55.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.139+0000 7f32a37fe640 1 -- 172.21.15.120:0/1761022369 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f32a80cfa10 con 0x7f32b4102790 2024-09-19T09:18:55.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.139+0000 7f32b259c640 1 --2- 172.21.15.120:0/1761022369 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f329c0a3240 0x7f329c0a5700 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f32a40058c0 tx=0x7f32a400a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:55.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.143+0000 7f32a37fe640 1 -- 172.21.15.120:0/1761022369 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f32a8014030 con 0x7f32b4102790 2024-09-19T09:18:55.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.346+0000 7f32b8e9b640 1 -- 172.21.15.120:0/1761022369 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32b410a150 con 0x7f32b4102790 2024-09-19T09:18:55.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.347+0000 7f32a37fe640 1 -- 172.21.15.120:0/1761022369 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11988 (secure 0 0 0) 0x7f32a809c730 con 0x7f32b4102790 2024-09-19T09:18:55.349 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:18:55.349 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":39,"btime":"2024-09-19T09:18:53:120858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:18:53.070639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:18:55.349 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 39 2024-09-19T09:18:55.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.349+0000 7f32b8e9b640 1 -- 172.21.15.120:0/1761022369 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f329c0a3240 msgr2=0x7f329c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:55.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.349+0000 7f32b8e9b640 1 --2- 172.21.15.120:0/1761022369 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f329c0a3240 0x7f329c0a5700 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f32a40058c0 tx=0x7f32a400a250 comp rx=0 tx=0).stop 2024-09-19T09:18:55.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.349+0000 7f32b8e9b640 1 -- 172.21.15.120:0/1761022369 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32b4102790 msgr2=0x7f32b4198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:55.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.349+0000 7f32b8e9b640 1 --2- 172.21.15.120:0/1761022369 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32b4102790 0x7f32b4198880 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f32a800c920 tx=0x7f32a800cde0 comp rx=0 tx=0).stop 2024-09-19T09:18:55.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.349+0000 7f32b8e9b640 1 -- 172.21.15.120:0/1761022369 shutdown_connections 2024-09-19T09:18:55.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.349+0000 7f32b8e9b640 1 --2- 172.21.15.120:0/1761022369 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f329c0a3240 0x7f329c0a5700 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:55.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.349+0000 7f32b8e9b640 1 --2- 172.21.15.120:0/1761022369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32b4106f10 0x7f32b419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:55.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.349+0000 7f32b8e9b640 1 --2- 172.21.15.120:0/1761022369 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32b4102790 0x7f32b4198880 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:55.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.349+0000 7f32b8e9b640 1 --2- 172.21.15.120:0/1761022369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b4101de0 0x7f32b4198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:55.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.349+0000 7f32b8e9b640 1 -- 172.21.15.120:0/1761022369 >> 172.21.15.120:0/1761022369 conn(0x7f32b40fd8c0 msgr2=0x7f32b40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:55.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.349+0000 7f32b8e9b640 1 -- 172.21.15.120:0/1761022369 shutdown_connections 2024-09-19T09:18:55.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:55.349+0000 7f32b8e9b640 1 -- 172.21.15.120:0/1761022369 wait complete. 2024-09-19T09:18:55.360 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 2.15 MB/sec execute 50 sec latency 24691.112 ms 2024-09-19T09:18:55.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:55 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1761022369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:18:55.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:55 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1761022369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:18:55.720 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:55 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1761022369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:18:56.082 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:18:56.324 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:18:56.360 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 2.10 MB/sec execute 51 sec latency 25691.185 ms 2024-09-19T09:18:56.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:56 smithi196 ceph-mon[35150]: pgmap v827: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 23 op/s 2024-09-19T09:18:56.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:56 smithi153 ceph-mon[35071]: pgmap v827: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 23 op/s 2024-09-19T09:18:56.760 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:56 smithi120 ceph-mon[31150]: pgmap v827: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 23 op/s 2024-09-19T09:18:56.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.841+0000 7f0b50e81640 1 -- 172.21.15.120:0/4012224803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b4c102790 msgr2=0x7f0b4c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:56.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.841+0000 7f0b50e81640 1 --2- 172.21.15.120:0/4012224803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b4c102790 0x7f0b4c1067e0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009900 tx=0x7f0b4002a410 comp rx=0 tx=0).stop 2024-09-19T09:18:56.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.841+0000 7f0b50e81640 1 -- 172.21.15.120:0/4012224803 shutdown_connections 2024-09-19T09:18:56.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.841+0000 7f0b50e81640 1 --2- 172.21.15.120:0/4012224803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b4c106f10 0x7f0b4c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.841+0000 7f0b50e81640 1 --2- 172.21.15.120:0/4012224803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b4c102790 0x7f0b4c1067e0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.841+0000 7f0b50e81640 1 --2- 172.21.15.120:0/4012224803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b4c101de0 0x7f0b4c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.841+0000 7f0b50e81640 1 -- 172.21.15.120:0/4012224803 >> 172.21.15.120:0/4012224803 conn(0x7f0b4c0fd8c0 msgr2=0x7f0b4c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:56.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.841+0000 7f0b50e81640 1 -- 172.21.15.120:0/4012224803 shutdown_connections 2024-09-19T09:18:56.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.841+0000 7f0b50e81640 1 -- 172.21.15.120:0/4012224803 wait complete. 2024-09-19T09:18:56.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.842+0000 7f0b50e81640 1 Processor -- start 2024-09-19T09:18:56.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.842+0000 7f0b50e81640 1 -- start start 2024-09-19T09:18:56.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.842+0000 7f0b50e81640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b4c101de0 0x7f0b4c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:56.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.842+0000 7f0b50e81640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b4c102790 0x7f0b4c198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:56.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.842+0000 7f0b50e81640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b4c106f10 0x7f0b4c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:56.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.842+0000 7f0b50e81640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4c111f70 con 0x7f0b4c101de0 2024-09-19T09:18:56.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.842+0000 7f0b50e81640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4c111df0 con 0x7f0b4c102790 2024-09-19T09:18:56.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.842+0000 7f0b50e81640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4c1120f0 con 0x7f0b4c106f10 2024-09-19T09:18:56.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.842+0000 7f0b4a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b4c101de0 0x7f0b4c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:56.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.842+0000 7f0b4a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b4c101de0 0x7f0b4c1984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37602/0 (socket says 172.21.15.120:37602) 2024-09-19T09:18:56.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.842+0000 7f0b4a59c640 1 -- 172.21.15.120:0/1810806559 learned_addr learned my addr 172.21.15.120:0/1810806559 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:56.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.842+0000 7f0b49d9b640 1 --2- 172.21.15.120:0/1810806559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b4c102790 0x7f0b4c198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:56.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.843+0000 7f0b4ad9d640 1 --2- 172.21.15.120:0/1810806559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b4c106f10 0x7f0b4c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:56.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.843+0000 7f0b4a59c640 1 -- 172.21.15.120:0/1810806559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b4c106f10 msgr2=0x7f0b4c19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:56.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.843+0000 7f0b4a59c640 1 --2- 172.21.15.120:0/1810806559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b4c106f10 0x7f0b4c19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.843+0000 7f0b4a59c640 1 -- 172.21.15.120:0/1810806559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b4c102790 msgr2=0x7f0b4c198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:56.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.843+0000 7f0b4a59c640 1 --2- 172.21.15.120:0/1810806559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b4c102790 0x7f0b4c198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.843+0000 7f0b4a59c640 1 -- 172.21.15.120:0/1810806559 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b4c19d480 con 0x7f0b4c101de0 2024-09-19T09:18:56.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.843+0000 7f0b4ad9d640 1 --2- 172.21.15.120:0/1810806559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b4c106f10 0x7f0b4c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:18:56.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.843+0000 7f0b4a59c640 1 --2- 172.21.15.120:0/1810806559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b4c101de0 0x7f0b4c1984d0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00d950 tx=0x7f0b3c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:56.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.843+0000 7f0b3b7fe640 1 -- 172.21.15.120:0/1810806559 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c017070 con 0x7f0b4c101de0 2024-09-19T09:18:56.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.843+0000 7f0b3b7fe640 1 -- 172.21.15.120:0/1810806559 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b3c009c30 con 0x7f0b4c101de0 2024-09-19T09:18:56.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.843+0000 7f0b3b7fe640 1 -- 172.21.15.120:0/1810806559 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c00f6b0 con 0x7f0b4c101de0 2024-09-19T09:18:56.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.843+0000 7f0b50e81640 1 -- 172.21.15.120:0/1810806559 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b4c19d770 con 0x7f0b4c101de0 2024-09-19T09:18:56.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.843+0000 7f0b50e81640 1 -- 172.21.15.120:0/1810806559 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b4c1a5050 con 0x7f0b4c101de0 2024-09-19T09:18:56.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.845+0000 7f0b3b7fe640 1 -- 172.21.15.120:0/1810806559 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f0b3c0109d0 con 0x7f0b4c101de0 2024-09-19T09:18:56.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.846+0000 7f0b5325a740 1 Processor -- start 2024-09-19T09:18:56.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.846+0000 7f0b5325a740 1 -- start start 2024-09-19T09:18:56.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.846+0000 7f0b3b7fe640 1 --2- 172.21.15.120:0/1810806559 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0b340a32c0 0x7f0b340a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:56.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.846+0000 7f0b5325a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638e7368b40 0x5638e7367090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:56.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.846+0000 7f0b5325a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e73675d0 0x5638e7367a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:56.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.846+0000 7f0b5325a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e7376780 0x5638e7378b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:56.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.846+0000 7f0b3b7fe640 1 -- 172.21.15.120:0/1810806559 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f0b3c0cf500 con 0x7f0b4c101de0 2024-09-19T09:18:56.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.846+0000 7f0b5325a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5638e72cd2f0 con 0x5638e7376780 2024-09-19T09:18:56.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.846+0000 7f0b5325a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5638e7274cf0 con 0x5638e73675d0 2024-09-19T09:18:56.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.846+0000 7f0b5325a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5638e72c92b0 con 0x5638e7368b40 2024-09-19T09:18:56.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.846+0000 7f0b4ad9d640 1 --2- 172.21.15.120:0/1810806559 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0b340a32c0 0x7f0b340a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:56.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.846+0000 7f0b4ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e7376780 0x5638e7378b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:56.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.846+0000 7f0b4ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e7376780 0x5638e7378b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37608/0 (socket says 172.21.15.120:37608) 2024-09-19T09:18:56.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.846+0000 7f0b4ad9d640 1 -- 172.21.15.120:0/819065739 learned_addr learned my addr 172.21.15.120:0/819065739 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:56.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.847+0000 7f0b4a59c640 1 --2- 172.21.15.120:0/819065739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638e7368b40 0x5638e7367090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:56.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.847+0000 7f0b49d9b640 1 --2- 172.21.15.120:0/819065739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e73675d0 0x5638e7367a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:56.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.847+0000 7f0b4ad9d640 1 -- 172.21.15.120:0/819065739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638e7368b40 msgr2=0x5638e7367090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:56.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.847+0000 7f0b4ad9d640 1 --2- 172.21.15.120:0/819065739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638e7368b40 0x5638e7367090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.847+0000 7f0b4ad9d640 1 -- 172.21.15.120:0/819065739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e73675d0 msgr2=0x5638e7367a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:56.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.847+0000 7f0b4ad9d640 1 --2- 172.21.15.120:0/819065739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e73675d0 0x5638e7367a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.847+0000 7f0b4ad9d640 1 -- 172.21.15.120:0/819065739 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638e72593a0 con 0x5638e7376780 2024-09-19T09:18:56.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.847+0000 7f0b4ad9d640 1 --2- 172.21.15.120:0/1810806559 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0b340a32c0 0x7f0b340a5780 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0b44013bd0 tx=0x7f0b44009de0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:56.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.847+0000 7f0b4ad9d640 1 --2- 172.21.15.120:0/819065739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e7376780 0x5638e7378b70 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f0b44005f30 tx=0x7f0b4401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f5a0f2525f177a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:56.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.847+0000 7f0b38ff9640 1 -- 172.21.15.120:0/819065739 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b44015e30 con 0x5638e7376780 2024-09-19T09:18:56.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.847+0000 7f0b38ff9640 1 -- 172.21.15.120:0/819065739 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b44010710 con 0x5638e7376780 2024-09-19T09:18:56.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.847+0000 7f0b38ff9640 1 -- 172.21.15.120:0/819065739 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b44005020 con 0x5638e7376780 2024-09-19T09:18:56.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.848+0000 7f0b5325a740 1 -- 172.21.15.120:0/819065739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e7376780 msgr2=0x5638e7378b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:56.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.848+0000 7f0b5325a740 1 --2- 172.21.15.120:0/819065739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e7376780 0x5638e7378b70 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f0b44005f30 tx=0x7f0b4401a000 comp rx=0 tx=0).stop 2024-09-19T09:18:56.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.848+0000 7f0b5325a740 1 -- 172.21.15.120:0/819065739 shutdown_connections 2024-09-19T09:18:56.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.848+0000 7f0b5325a740 1 --2- 172.21.15.120:0/819065739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e7376780 0x5638e7378b70 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.848+0000 7f0b5325a740 1 --2- 172.21.15.120:0/819065739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e73675d0 0x5638e7367a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.848+0000 7f0b5325a740 1 --2- 172.21.15.120:0/819065739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638e7368b40 0x5638e7367090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.848+0000 7f0b5325a740 1 -- 172.21.15.120:0/819065739 >> 172.21.15.120:0/819065739 conn(0x5638e7330310 msgr2=0x5638e7363c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:56.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.848+0000 7f0b5325a740 1 -- 172.21.15.120:0/819065739 shutdown_connections 2024-09-19T09:18:56.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.848+0000 7f0b5325a740 1 -- 172.21.15.120:0/819065739 wait complete. 2024-09-19T09:18:56.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.848+0000 7f0b5325a740 1 -- start start 2024-09-19T09:18:56.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.848+0000 7f0b5325a740 1 Processor -- start 2024-09-19T09:18:56.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.849+0000 7f0b5325a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e73675d0 0x5638e7376170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:56.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.849+0000 7f0b5325a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e7368b40 0x5638e7373420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:56.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.849+0000 7f0b5325a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638e7376780 0x5638e7373a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:56.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.849+0000 7f0b5325a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5638e7379420 con 0x5638e7368b40 2024-09-19T09:18:56.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.849+0000 7f0b5325a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5638e7288060 con 0x5638e73675d0 2024-09-19T09:18:56.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.849+0000 7f0b5325a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5638e73795a0 con 0x5638e7376780 2024-09-19T09:18:56.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.849+0000 7f0b5325a740 10 client.0 _ll_register_callbacks cb 0x5638e6fdb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:56.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.849+0000 7f0b49d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e7368b40 0x5638e7373420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:56.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.849+0000 7f0b49d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e7368b40 0x5638e7373420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37612/0 (socket says 172.21.15.120:37612) 2024-09-19T09:18:56.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.849+0000 7f0b49d9b640 1 -- 172.21.15.120:0/605436118 learned_addr learned my addr 172.21.15.120:0/605436118 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:56.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.849+0000 7f0b4a59c640 1 --2- 172.21.15.120:0/605436118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e73675d0 0x5638e7376170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:56.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.849+0000 7f0b49d9b640 1 -- 172.21.15.120:0/605436118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638e7376780 msgr2=0x5638e7373a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:56.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.849+0000 7f0b49d9b640 1 --2- 172.21.15.120:0/605436118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638e7376780 0x5638e7373a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.849+0000 7f0b49d9b640 1 -- 172.21.15.120:0/605436118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e73675d0 msgr2=0x5638e7376170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:56.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.849+0000 7f0b49d9b640 1 --2- 172.21.15.120:0/605436118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e73675d0 0x5638e7376170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.849+0000 7f0b49d9b640 1 -- 172.21.15.120:0/605436118 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638e703e3c0 con 0x5638e7368b40 2024-09-19T09:18:56.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.849+0000 7f0b49d9b640 1 --2- 172.21.15.120:0/605436118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e7368b40 0x5638e7373420 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f0b4002a930 tx=0x7f0b40031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c987005657b7ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:56.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.849+0000 7f0b31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:18:56.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.849+0000 7f0b31ffb640 1 -- 172.21.15.120:0/605436118 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b40035040 con 0x5638e7368b40 2024-09-19T09:18:56.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.849+0000 7f0b31ffb640 1 -- 172.21.15.120:0/605436118 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f0b40004240 con 0x5638e7368b40 2024-09-19T09:18:56.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.849+0000 7f0b5325a740 1 -- 172.21.15.120:0/605436118 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b44010710 con 0x5638e7368b40 2024-09-19T09:18:56.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.850+0000 7f0b31ffb640 1 -- 172.21.15.120:0/605436118 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0b40004440 con 0x5638e7368b40 2024-09-19T09:18:56.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.850+0000 7f0b5325a740 10 client.20871 fetch_fsmap learned FSMap version 39 2024-09-19T09:18:56.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.850+0000 7f0b5325a740 1 -- 172.21.15.120:0/605436118 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5638e7384f60 con 0x5638e7368b40 2024-09-19T09:18:56.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.850+0000 7f0b31ffb640 1 -- 172.21.15.120:0/605436118 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b44010710 con 0x5638e7368b40 2024-09-19T09:18:56.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.850+0000 7f0b31ffb640 1 -- 172.21.15.120:0/605436118 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f0b40004bf0 con 0x5638e7368b40 2024-09-19T09:18:56.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.850+0000 7f0b5325a740 10 client.20871 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:18:56.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.850+0000 7f0b5325a740 10 client.20871 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:18:56.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.850+0000 7f0b5325a740 20 client.20871 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:56.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.850+0000 7f0b5325a740 1 --2- 172.21.15.120:0/605436118 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5638e7386420 0x5638e73888c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:56.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.850+0000 7f0b5325a740 4 client.20871 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:18:56.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.850+0000 7f0b5325a740 1 -- 172.21.15.120:0/605436118 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638e72da860 con 0x5638e7386420 2024-09-19T09:18:56.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.850+0000 7f0b4a59c640 1 --2- 172.21.15.120:0/605436118 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5638e7386420 0x5638e73888c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:56.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.851+0000 7f0b31ffb640 10 client.20871 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:18:56.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.851+0000 7f0b4a59c640 1 --2- 172.21.15.120:0/605436118 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5638e7386420 0x5638e73888c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e8d1fa9d3ff48955 server_cookie=d9730f7abc31da18 in_seq=0 out_seq=0 2024-09-19T09:18:56.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.851+0000 7f0b31ffb640 1 -- 172.21.15.120:0/605436118 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5638e72da860 con 0x5638e7386420 2024-09-19T09:18:56.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.851+0000 7f0b31ffb640 10 client.20871 handle_command_reply: tid=0 2024-09-19T09:18:56.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.851+0000 7f0b5325a740 1 client.20871 shutdown 2024-09-19T09:18:56.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.852+0000 7f0b5325a740 1 -- 172.21.15.120:0/605436118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e7368b40 msgr2=0x5638e7373420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:56.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.852+0000 7f0b5325a740 1 --2- 172.21.15.120:0/605436118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e7368b40 0x5638e7373420 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f0b4002a930 tx=0x7f0b40031850 comp rx=0 tx=0).stop 2024-09-19T09:18:56.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.852+0000 7f0b5325a740 1 -- 172.21.15.120:0/605436118 shutdown_connections 2024-09-19T09:18:56.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.852+0000 7f0b5325a740 1 --2- 172.21.15.120:0/605436118 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5638e7386420 0x5638e73888c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.852+0000 7f0b5325a740 1 --2- 172.21.15.120:0/605436118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638e7376780 0x5638e7373a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.852+0000 7f0b5325a740 1 --2- 172.21.15.120:0/605436118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e7368b40 0x5638e7373420 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.852+0000 7f0b5325a740 1 --2- 172.21.15.120:0/605436118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e73675d0 0x5638e7376170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.852+0000 7f0b31ffb640 0 client.20871 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:18:56.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.852+0000 7f0b5325a740 1 -- 172.21.15.120:0/605436118 >> 172.21.15.120:0/605436118 conn(0x5638e7330310 msgr2=0x5638e7369960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:56.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.852+0000 7f0b5325a740 1 -- 172.21.15.120:0/605436118 shutdown_connections 2024-09-19T09:18:56.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.852+0000 7f0b5325a740 1 -- 172.21.15.120:0/605436118 wait complete. 2024-09-19T09:18:56.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.852+0000 7f0b5325a740 20 client.20871 trim_cache size 0 max 16384 2024-09-19T09:18:56.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.864+0000 7f0b5325a740 1 Processor -- start 2024-09-19T09:18:56.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.864+0000 7f0b5325a740 1 -- start start 2024-09-19T09:18:56.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.865+0000 7f0b5325a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638e73675d0 0x5638e7382710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:56.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.865+0000 7f0b5325a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e7368b40 0x5638e7371cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:56.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.865+0000 7f0b5325a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e7376780 0x5638e7372230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:56.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.865+0000 7f0b5325a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4c1ab3f0 con 0x5638e7376780 2024-09-19T09:18:56.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.865+0000 7f0b5325a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4c1acf30 con 0x5638e7368b40 2024-09-19T09:18:56.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.865+0000 7f0b5325a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4c1abac0 con 0x5638e73675d0 2024-09-19T09:18:56.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.865+0000 7f0b4ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e7376780 0x5638e7372230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:56.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.865+0000 7f0b4ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e7376780 0x5638e7372230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37620/0 (socket says 172.21.15.120:37620) 2024-09-19T09:18:56.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.865+0000 7f0b4ad9d640 1 -- 172.21.15.120:0/3170261877 learned_addr learned my addr 172.21.15.120:0/3170261877 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:56.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.865+0000 7f0b4ad9d640 1 -- 172.21.15.120:0/3170261877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638e73675d0 msgr2=0x5638e7382710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:56.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.865+0000 7f0b4ad9d640 1 --2- 172.21.15.120:0/3170261877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638e73675d0 0x5638e7382710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.865+0000 7f0b4ad9d640 1 -- 172.21.15.120:0/3170261877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e7368b40 msgr2=0x5638e7371cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:56.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.865+0000 7f0b4ad9d640 1 --2- 172.21.15.120:0/3170261877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e7368b40 0x5638e7371cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.865+0000 7f0b4ad9d640 1 -- 172.21.15.120:0/3170261877 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638e7373d80 con 0x5638e7376780 2024-09-19T09:18:56.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.865+0000 7f0b4ad9d640 1 --2- 172.21.15.120:0/3170261877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e7376780 0x5638e7372230 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f0b44013760 tx=0x7f0b44009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f2116316e20b26 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:56.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.865+0000 7f0b33fff640 1 -- 172.21.15.120:0/3170261877 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b44013de0 con 0x5638e7376780 2024-09-19T09:18:56.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.865+0000 7f0b33fff640 1 -- 172.21.15.120:0/3170261877 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b4402e070 con 0x5638e7376780 2024-09-19T09:18:56.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.865+0000 7f0b33fff640 1 -- 172.21.15.120:0/3170261877 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b44010c70 con 0x5638e7376780 2024-09-19T09:18:56.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.866+0000 7f0b5325a740 1 -- 172.21.15.120:0/3170261877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e7376780 msgr2=0x5638e7372230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:56.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.866+0000 7f0b5325a740 1 --2- 172.21.15.120:0/3170261877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e7376780 0x5638e7372230 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f0b44013760 tx=0x7f0b44009520 comp rx=0 tx=0).stop 2024-09-19T09:18:56.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.866+0000 7f0b5325a740 1 -- 172.21.15.120:0/3170261877 shutdown_connections 2024-09-19T09:18:56.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.866+0000 7f0b5325a740 1 --2- 172.21.15.120:0/3170261877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e7376780 0x5638e7372230 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.866+0000 7f0b5325a740 1 --2- 172.21.15.120:0/3170261877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e7368b40 0x5638e7371cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.866+0000 7f0b5325a740 1 --2- 172.21.15.120:0/3170261877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638e73675d0 0x5638e7382710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.866+0000 7f0b5325a740 1 -- 172.21.15.120:0/3170261877 >> 172.21.15.120:0/3170261877 conn(0x5638e7330310 msgr2=0x5638e736e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:56.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.866+0000 7f0b5325a740 1 -- 172.21.15.120:0/3170261877 shutdown_connections 2024-09-19T09:18:56.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.866+0000 7f0b5325a740 1 -- 172.21.15.120:0/3170261877 wait complete. 2024-09-19T09:18:56.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.866+0000 7f0b5325a740 1 -- start start 2024-09-19T09:18:56.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.866+0000 7f0b5325a740 1 Processor -- start 2024-09-19T09:18:56.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.867+0000 7f0b5325a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638e73675d0 0x5638e737ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:56.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.867+0000 7f0b5325a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e7368b40 0x5638e737f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:56.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.867+0000 7f0b5325a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e7376780 0x5638e736fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:56.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.867+0000 7f0b5325a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4c1ac860 con 0x5638e7368b40 2024-09-19T09:18:56.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.867+0000 7f0b5325a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4c1ac190 con 0x5638e7376780 2024-09-19T09:18:56.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.867+0000 7f0b5325a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4c1a93b0 con 0x5638e73675d0 2024-09-19T09:18:56.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.867+0000 7f0b5325a740 10 client.0 _ll_register_callbacks cb 0x7f0b40005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:56.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.867+0000 7f0b49d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e7368b40 0x5638e737f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:56.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.867+0000 7f0b49d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e7368b40 0x5638e737f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37632/0 (socket says 172.21.15.120:37632) 2024-09-19T09:18:56.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.867+0000 7f0b49d9b640 1 -- 172.21.15.120:0/3129232812 learned_addr learned my addr 172.21.15.120:0/3129232812 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:56.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.867+0000 7f0b4ad9d640 1 --2- 172.21.15.120:0/3129232812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e7376780 0x5638e736fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:56.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.867+0000 7f0b4a59c640 1 --2- 172.21.15.120:0/3129232812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638e73675d0 0x5638e737ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:56.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.867+0000 7f0b49d9b640 1 -- 172.21.15.120:0/3129232812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638e73675d0 msgr2=0x5638e737ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:56.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.867+0000 7f0b49d9b640 1 --2- 172.21.15.120:0/3129232812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638e73675d0 0x5638e737ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.867+0000 7f0b49d9b640 1 -- 172.21.15.120:0/3129232812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e7376780 msgr2=0x5638e736fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:56.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.867+0000 7f0b49d9b640 1 --2- 172.21.15.120:0/3129232812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e7376780 0x5638e736fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.867+0000 7f0b49d9b640 1 -- 172.21.15.120:0/3129232812 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638e7370460 con 0x5638e7368b40 2024-09-19T09:18:56.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.867+0000 7f0b49d9b640 1 --2- 172.21.15.120:0/3129232812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e7368b40 0x5638e737f010 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009850 tx=0x7f0b40005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e6b6e7f84d0503 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:56.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.867+0000 7f0b31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:18:56.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.867+0000 7f0b31ffb640 1 -- 172.21.15.120:0/3129232812 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b40004950 con 0x5638e7368b40 2024-09-19T09:18:56.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.868+0000 7f0b31ffb640 1 -- 172.21.15.120:0/3129232812 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f0b40004af0 con 0x5638e7368b40 2024-09-19T09:18:56.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.868+0000 7f0b5325a740 1 -- 172.21.15.120:0/3129232812 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b4402e070 con 0x5638e7368b40 2024-09-19T09:18:56.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.868+0000 7f0b31ffb640 1 -- 172.21.15.120:0/3129232812 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0b40004cf0 con 0x5638e7368b40 2024-09-19T09:18:56.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.868+0000 7f0b5325a740 10 client.20877 fetch_fsmap learned FSMap version 39 2024-09-19T09:18:56.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.868+0000 7f0b5325a740 1 -- 172.21.15.120:0/3129232812 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5638e73f0c20 con 0x5638e7368b40 2024-09-19T09:18:56.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.868+0000 7f0b31ffb640 1 -- 172.21.15.120:0/3129232812 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b4402e070 con 0x5638e7368b40 2024-09-19T09:18:56.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.868+0000 7f0b31ffb640 1 -- 172.21.15.120:0/3129232812 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f0b40003d80 con 0x5638e7368b40 2024-09-19T09:18:56.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.868+0000 7f0b5325a740 10 client.20877 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:18:56.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.868+0000 7f0b5325a740 10 client.20877 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:18:56.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.868+0000 7f0b5325a740 20 client.20877 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:56.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.868+0000 7f0b5325a740 1 --2- 172.21.15.120:0/3129232812 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5638e73f1310 0x5638e73f3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:56.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.868+0000 7f0b5325a740 4 client.20877 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:18:56.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.868+0000 7f0b5325a740 1 -- 172.21.15.120:0/3129232812 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5638e72d4d70 con 0x5638e73f1310 2024-09-19T09:18:56.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.868+0000 7f0b4a59c640 1 --2- 172.21.15.120:0/3129232812 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5638e73f1310 0x5638e73f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:56.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.869+0000 7f0b4a59c640 1 --2- 172.21.15.120:0/3129232812 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5638e73f1310 0x5638e73f3730 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3877b45193e09147 server_cookie=59c07741eeaaa1bd in_seq=0 out_seq=0 2024-09-19T09:18:56.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.869+0000 7f0b31ffb640 10 client.20877 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:18:56.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.869+0000 7f0b31ffb640 1 -- 172.21.15.120:0/3129232812 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5638e72d4d70 con 0x5638e73f1310 2024-09-19T09:18:56.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.869+0000 7f0b31ffb640 10 client.20877 handle_command_reply: tid=0 2024-09-19T09:18:56.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.869+0000 7f0b5325a740 1 client.20877 shutdown 2024-09-19T09:18:56.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.869+0000 7f0b5325a740 1 -- 172.21.15.120:0/3129232812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e7368b40 msgr2=0x5638e737f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:56.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.869+0000 7f0b5325a740 1 --2- 172.21.15.120:0/3129232812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e7368b40 0x5638e737f010 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009850 tx=0x7f0b40005fd0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.869+0000 7f0b5325a740 1 -- 172.21.15.120:0/3129232812 shutdown_connections 2024-09-19T09:18:56.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.869+0000 7f0b5325a740 1 --2- 172.21.15.120:0/3129232812 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x5638e73f1310 0x5638e73f3730 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.869+0000 7f0b5325a740 1 --2- 172.21.15.120:0/3129232812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e7376780 0x5638e736fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.869+0000 7f0b5325a740 1 --2- 172.21.15.120:0/3129232812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e7368b40 0x5638e737f010 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.869+0000 7f0b5325a740 1 --2- 172.21.15.120:0/3129232812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638e73675d0 0x5638e737ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.869+0000 7f0b5325a740 1 -- 172.21.15.120:0/3129232812 >> 172.21.15.120:0/3129232812 conn(0x5638e7330310 msgr2=0x5638e7374c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:56.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.869+0000 7f0b31ffb640 0 client.20877 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:18:56.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.869+0000 7f0b5325a740 1 -- 172.21.15.120:0/3129232812 shutdown_connections 2024-09-19T09:18:56.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.869+0000 7f0b5325a740 1 -- 172.21.15.120:0/3129232812 wait complete. 2024-09-19T09:18:56.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.869+0000 7f0b5325a740 20 client.20877 trim_cache size 0 max 16384 2024-09-19T09:18:56.884 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:18:56.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.870+0000 7f0b50e81640 1 -- 172.21.15.120:0/1810806559 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0b340a32c0 msgr2=0x7f0b340a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:56.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.870+0000 7f0b50e81640 1 --2- 172.21.15.120:0/1810806559 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0b340a32c0 0x7f0b340a5780 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0b44013bd0 tx=0x7f0b44009de0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.870+0000 7f0b50e81640 1 -- 172.21.15.120:0/1810806559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b4c101de0 msgr2=0x7f0b4c1984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:56.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.870+0000 7f0b50e81640 1 --2- 172.21.15.120:0/1810806559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b4c101de0 0x7f0b4c1984d0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00d950 tx=0x7f0b3c00de10 comp rx=0 tx=0).stop 2024-09-19T09:18:56.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.870+0000 7f0b50e81640 1 -- 172.21.15.120:0/1810806559 shutdown_connections 2024-09-19T09:18:56.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.870+0000 7f0b50e81640 1 --2- 172.21.15.120:0/1810806559 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0b340a32c0 0x7f0b340a5780 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.870+0000 7f0b50e81640 1 --2- 172.21.15.120:0/1810806559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b4c106f10 0x7f0b4c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.870+0000 7f0b50e81640 1 --2- 172.21.15.120:0/1810806559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b4c102790 0x7f0b4c198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.870+0000 7f0b50e81640 1 --2- 172.21.15.120:0/1810806559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b4c101de0 0x7f0b4c1984d0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:56.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.870+0000 7f0b50e81640 1 -- 172.21.15.120:0/1810806559 >> 172.21.15.120:0/1810806559 conn(0x7f0b4c0fd8c0 msgr2=0x7f0b4c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:56.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.870+0000 7f0b50e81640 1 -- 172.21.15.120:0/1810806559 shutdown_connections 2024-09-19T09:18:56.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:56.870+0000 7f0b50e81640 1 -- 172.21.15.120:0/1810806559 wait complete. 2024-09-19T09:18:57.105 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:18:57.361 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 2.06 MB/sec execute 52 sec latency 26691.316 ms 2024-09-19T09:18:57.553 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:18:57.683 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:18:58.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.191+0000 7fbde9796640 1 -- 172.21.15.120:0/2496319527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbde4102d40 msgr2=0x7fbde4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:58.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.191+0000 7fbde9796640 1 --2- 172.21.15.120:0/2496319527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbde4102d40 0x7fbde4103120 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8009900 tx=0x7fbdd802a480 comp rx=0 tx=0).stop 2024-09-19T09:18:58.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.191+0000 7fbde9796640 1 -- 172.21.15.120:0/2496319527 shutdown_connections 2024-09-19T09:18:58.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.191+0000 7fbde9796640 1 --2- 172.21.15.120:0/2496319527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbde410c170 0x7fbde410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:58.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.191+0000 7fbde9796640 1 --2- 172.21.15.120:0/2496319527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbde4103660 0x7fbde410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:58.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.191+0000 7fbde9796640 1 --2- 172.21.15.120:0/2496319527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbde4102d40 0x7fbde4103120 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:58.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.191+0000 7fbde9796640 1 -- 172.21.15.120:0/2496319527 >> 172.21.15.120:0/2496319527 conn(0x7fbde40fc7a0 msgr2=0x7fbde40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:58.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.191+0000 7fbde9796640 1 -- 172.21.15.120:0/2496319527 shutdown_connections 2024-09-19T09:18:58.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.191+0000 7fbde9796640 1 -- 172.21.15.120:0/2496319527 wait complete. 2024-09-19T09:18:58.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.191+0000 7fbde9796640 1 Processor -- start 2024-09-19T09:18:58.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.192+0000 7fbde9796640 1 -- start start 2024-09-19T09:18:58.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.192+0000 7fbde9796640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbde4102d40 0x7fbde41009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:58.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.192+0000 7fbde9796640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbde4103660 0x7fbde4100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:58.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.192+0000 7fbde9796640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbde410c170 0x7fbde4101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:58.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.192+0000 7fbde9796640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbde4111fd0 con 0x7fbde4102d40 2024-09-19T09:18:58.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.192+0000 7fbde9796640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbde4111e50 con 0x7fbde410c170 2024-09-19T09:18:58.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.192+0000 7fbde9796640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbde4112150 con 0x7fbde4103660 2024-09-19T09:18:58.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.192+0000 7fbde2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbde4102d40 0x7fbde41009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:58.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.192+0000 7fbde2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbde4102d40 0x7fbde41009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37644/0 (socket says 172.21.15.120:37644) 2024-09-19T09:18:58.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.192+0000 7fbde2ffd640 1 -- 172.21.15.120:0/1980223541 learned_addr learned my addr 172.21.15.120:0/1980223541 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:58.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.192+0000 7fbde37fe640 1 --2- 172.21.15.120:0/1980223541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbde410c170 0x7fbde4101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:58.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.192+0000 7fbde27fc640 1 --2- 172.21.15.120:0/1980223541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbde4103660 0x7fbde4100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:58.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.192+0000 7fbde2ffd640 1 -- 172.21.15.120:0/1980223541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbde4103660 msgr2=0x7fbde4100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:58.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.192+0000 7fbde2ffd640 1 --2- 172.21.15.120:0/1980223541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbde4103660 0x7fbde4100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:58.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.192+0000 7fbde2ffd640 1 -- 172.21.15.120:0/1980223541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbde410c170 msgr2=0x7fbde4101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:58.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.192+0000 7fbde2ffd640 1 --2- 172.21.15.120:0/1980223541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbde410c170 0x7fbde4101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:58.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.192+0000 7fbde2ffd640 1 -- 172.21.15.120:0/1980223541 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde41a6950 con 0x7fbde4102d40 2024-09-19T09:18:58.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.192+0000 7fbde37fe640 1 --2- 172.21.15.120:0/1980223541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbde410c170 0x7fbde4101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:18:58.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.193+0000 7fbde2ffd640 1 --2- 172.21.15.120:0/1980223541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbde4102d40 0x7fbde41009a0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8006010 tx=0x7fbdd80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:58.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.193+0000 7fbdcbfff640 1 -- 172.21.15.120:0/1980223541 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd803f030 con 0x7fbde4102d40 2024-09-19T09:18:58.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.193+0000 7fbdcbfff640 1 -- 172.21.15.120:0/1980223541 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdd8003ce0 con 0x7fbde4102d40 2024-09-19T09:18:58.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.193+0000 7fbde9796640 1 -- 172.21.15.120:0/1980223541 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde41a6be0 con 0x7fbde4102d40 2024-09-19T09:18:58.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.193+0000 7fbdcbfff640 1 -- 172.21.15.120:0/1980223541 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8035070 con 0x7fbde4102d40 2024-09-19T09:18:58.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.193+0000 7fbde9796640 1 -- 172.21.15.120:0/1980223541 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde41a70c0 con 0x7fbde4102d40 2024-09-19T09:18:58.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.194+0000 7fbde9796640 1 -- 172.21.15.120:0/1980223541 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbde4103120 con 0x7fbde4102d40 2024-09-19T09:18:58.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.195+0000 7fbdcbfff640 1 -- 172.21.15.120:0/1980223541 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fbdd8042030 con 0x7fbde4102d40 2024-09-19T09:18:58.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.196+0000 7fbdcbfff640 1 --2- 172.21.15.120:0/1980223541 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbdcc0a3240 0x7fbdcc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:58.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.196+0000 7fbde27fc640 1 --2- 172.21.15.120:0/1980223541 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbdcc0a3240 0x7fbdcc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:58.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.196+0000 7fbdcbfff640 1 -- 172.21.15.120:0/1980223541 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fbdd80f0220 con 0x7fbde4102d40 2024-09-19T09:18:58.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.196+0000 7fbde27fc640 1 --2- 172.21.15.120:0/1980223541 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbdcc0a3240 0x7fbdcc0a5700 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fbde4102430 tx=0x7fbdd4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:58.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.199+0000 7fbdcbfff640 1 -- 172.21.15.120:0/1980223541 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbdd80bd000 con 0x7fbde4102d40 2024-09-19T09:18:58.270 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:18:58.361 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 2.03 MB/sec execute 53 sec latency 27691.366 ms 2024-09-19T09:18:58.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.415+0000 7fbde9796640 1 -- 172.21.15.120:0/1980223541 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbde4107210 con 0x7fbde4102d40 2024-09-19T09:18:58.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.416+0000 7fbdcbfff640 1 -- 172.21.15.120:0/1980223541 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11988 (secure 0 0 0) 0x7fbdd80bc900 con 0x7fbde4102d40 2024-09-19T09:18:58.417 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:18:58.417 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":39,"btime":"2024-09-19T09:18:53:120858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:18:53.070639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:18:58.417 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 39 2024-09-19T09:18:58.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.418+0000 7fbde9796640 1 -- 172.21.15.120:0/1980223541 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbdcc0a3240 msgr2=0x7fbdcc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:58.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.418+0000 7fbde9796640 1 --2- 172.21.15.120:0/1980223541 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbdcc0a3240 0x7fbdcc0a5700 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fbde4102430 tx=0x7fbdd4008040 comp rx=0 tx=0).stop 2024-09-19T09:18:58.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.418+0000 7fbde9796640 1 -- 172.21.15.120:0/1980223541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbde4102d40 msgr2=0x7fbde41009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:58.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.418+0000 7fbde9796640 1 --2- 172.21.15.120:0/1980223541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbde4102d40 0x7fbde41009a0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8006010 tx=0x7fbdd80042f0 comp rx=0 tx=0).stop 2024-09-19T09:18:58.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.418+0000 7fbde9796640 1 -- 172.21.15.120:0/1980223541 shutdown_connections 2024-09-19T09:18:58.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.418+0000 7fbde9796640 1 --2- 172.21.15.120:0/1980223541 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbdcc0a3240 0x7fbdcc0a5700 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:58.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.418+0000 7fbde9796640 1 --2- 172.21.15.120:0/1980223541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbde410c170 0x7fbde4101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:58.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.418+0000 7fbde9796640 1 --2- 172.21.15.120:0/1980223541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbde4103660 0x7fbde4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:58.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.418+0000 7fbde9796640 1 --2- 172.21.15.120:0/1980223541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbde4102d40 0x7fbde41009a0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:58.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.418+0000 7fbde9796640 1 -- 172.21.15.120:0/1980223541 >> 172.21.15.120:0/1980223541 conn(0x7fbde40fc7a0 msgr2=0x7fbde40fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:58.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.418+0000 7fbde9796640 1 -- 172.21.15.120:0/1980223541 shutdown_connections 2024-09-19T09:18:58.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:58.418+0000 7fbde9796640 1 -- 172.21.15.120:0/1980223541 wait complete. 2024-09-19T09:18:59.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:58 smithi120 ceph-mon[31150]: pgmap v828: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 26 op/s 2024-09-19T09:18:59.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:18:58 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1980223541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:18:59.361 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 1.99 MB/sec execute 54 sec latency 28691.419 ms 2024-09-19T09:18:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.434+0000 7eff5b02e640 1 -- 172.21.15.120:0/1556421014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff541025c0 msgr2=0x7eff541049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.434+0000 7eff5b02e640 1 --2- 172.21.15.120:0/1556421014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff541025c0 0x7eff541049b0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7eff44009ca0 tx=0x7eff4402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.434+0000 7eff5b02e640 1 -- 172.21.15.120:0/1556421014 shutdown_connections 2024-09-19T09:18:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.434+0000 7eff5b02e640 1 --2- 172.21.15.120:0/1556421014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff54107820 0x7eff54109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.434+0000 7eff5b02e640 1 --2- 172.21.15.120:0/1556421014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff54104ef0 0x7eff541072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.434+0000 7eff5b02e640 1 --2- 172.21.15.120:0/1556421014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff541025c0 0x7eff541049b0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.434+0000 7eff5b02e640 1 -- 172.21.15.120:0/1556421014 >> 172.21.15.120:0/1556421014 conn(0x7eff540fc1e0 msgr2=0x7eff540fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.434+0000 7eff5b02e640 1 -- 172.21.15.120:0/1556421014 shutdown_connections 2024-09-19T09:18:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.434+0000 7eff5b02e640 1 -- 172.21.15.120:0/1556421014 wait complete. 2024-09-19T09:18:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.434+0000 7eff5b02e640 1 Processor -- start 2024-09-19T09:18:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.434+0000 7eff5b02e640 1 -- start start 2024-09-19T09:18:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.435+0000 7eff5b02e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff541025c0 0x7eff54197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.435+0000 7eff5b02e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff54104ef0 0x7eff54198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.435+0000 7eff5b02e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff54107820 0x7eff5419f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.435+0000 7eff5b02e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7eff5410d880 con 0x7eff541025c0 2024-09-19T09:18:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.435+0000 7eff5b02e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7eff5410d700 con 0x7eff54104ef0 2024-09-19T09:18:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.435+0000 7eff5b02e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7eff5410da00 con 0x7eff54107820 2024-09-19T09:18:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.435+0000 7eff58dca640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff541025c0 0x7eff54197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.435+0000 7eff58dca640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff541025c0 0x7eff54197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37666/0 (socket says 172.21.15.120:37666) 2024-09-19T09:18:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.435+0000 7eff58dca640 1 -- 172.21.15.120:0/2621988183 learned_addr learned my addr 172.21.15.120:0/2621988183 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.435+0000 7eff53fff640 1 --2- 172.21.15.120:0/2621988183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff54104ef0 0x7eff54198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.435+0000 7eff595cb640 1 --2- 172.21.15.120:0/2621988183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff54107820 0x7eff5419f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.435+0000 7eff58dca640 1 -- 172.21.15.120:0/2621988183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff54107820 msgr2=0x7eff5419f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.435+0000 7eff58dca640 1 --2- 172.21.15.120:0/2621988183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff54107820 0x7eff5419f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.435+0000 7eff58dca640 1 -- 172.21.15.120:0/2621988183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff54104ef0 msgr2=0x7eff54198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.435+0000 7eff58dca640 1 --2- 172.21.15.120:0/2621988183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff54104ef0 0x7eff54198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.435+0000 7eff58dca640 1 -- 172.21.15.120:0/2621988183 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5419fa70 con 0x7eff541025c0 2024-09-19T09:18:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.435+0000 7eff58dca640 1 --2- 172.21.15.120:0/2621988183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff541025c0 0x7eff54197e50 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7eff44031a40 tx=0x7eff44031ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.435+0000 7eff51ffb640 1 -- 172.21.15.120:0/2621988183 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4402faa0 con 0x7eff541025c0 2024-09-19T09:18:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.435+0000 7eff51ffb640 1 -- 172.21.15.120:0/2621988183 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff4402fc40 con 0x7eff541025c0 2024-09-19T09:18:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.435+0000 7eff5b02e640 1 -- 172.21.15.120:0/2621988183 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff5419fd00 con 0x7eff541025c0 2024-09-19T09:18:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.435+0000 7eff51ffb640 1 -- 172.21.15.120:0/2621988183 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4403a790 con 0x7eff541025c0 2024-09-19T09:18:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.436+0000 7eff5b02e640 1 -- 172.21.15.120:0/2621988183 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff541a01a0 con 0x7eff541025c0 2024-09-19T09:18:59.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:58 smithi153 ceph-mon[35071]: pgmap v828: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 26 op/s 2024-09-19T09:18:59.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:18:58 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1980223541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:18:59.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:58 smithi196 ceph-mon[35150]: pgmap v828: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 26 op/s 2024-09-19T09:18:59.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:18:58 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1980223541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:18:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.438+0000 7eff51ffb640 1 -- 172.21.15.120:0/2621988183 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7eff44030b30 con 0x7eff541025c0 2024-09-19T09:18:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.438+0000 7eff5ddc9740 1 Processor -- start 2024-09-19T09:18:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.438+0000 7eff5ddc9740 1 -- start start 2024-09-19T09:18:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.438+0000 7eff5ddc9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613c98a79a0 0x5613c98a7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.438+0000 7eff5ddc9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613c98a82e0 0x5613c98acda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.438+0000 7eff5ddc9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613c98ad370 0x5613c98af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.438+0000 7eff5ddc9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5613c980b2f0 con 0x5613c98ad370 2024-09-19T09:18:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.438+0000 7eff5ddc9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5613c97b3270 con 0x5613c98a82e0 2024-09-19T09:18:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.438+0000 7eff5ddc9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5613c98072b0 con 0x5613c98a79a0 2024-09-19T09:18:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.438+0000 7eff53fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613c98ad370 0x5613c98af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.438+0000 7eff53fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613c98ad370 0x5613c98af830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37676/0 (socket says 172.21.15.120:37676) 2024-09-19T09:18:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.438+0000 7eff53fff640 1 -- 172.21.15.120:0/1886326573 learned_addr learned my addr 172.21.15.120:0/1886326573 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.438+0000 7eff58dca640 1 --2- 172.21.15.120:0/1886326573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613c98a82e0 0x5613c98acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.438+0000 7eff595cb640 1 --2- 172.21.15.120:0/1886326573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613c98a79a0 0x5613c98a7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.439+0000 7eff53fff640 1 -- 172.21.15.120:0/1886326573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613c98a79a0 msgr2=0x5613c98a7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.439+0000 7eff53fff640 1 --2- 172.21.15.120:0/1886326573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613c98a79a0 0x5613c98a7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.439+0000 7eff53fff640 1 -- 172.21.15.120:0/1886326573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613c98a82e0 msgr2=0x5613c98acda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.439+0000 7eff53fff640 1 --2- 172.21.15.120:0/1886326573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613c98a82e0 0x5613c98acda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.439+0000 7eff53fff640 1 -- 172.21.15.120:0/1886326573 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613c97b2340 con 0x5613c98ad370 2024-09-19T09:18:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.439+0000 7eff51ffb640 1 --2- 172.21.15.120:0/2621988183 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7eff400a32f0 0x7eff400a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.439+0000 7eff53fff640 1 --2- 172.21.15.120:0/1886326573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613c98ad370 0x5613c98af830 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7eff3c00bcd0 tx=0x7eff3c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6736fc46fae257 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.439+0000 7eff595cb640 1 --2- 172.21.15.120:0/2621988183 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7eff400a32f0 0x7eff400a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.439+0000 7eff51ffb640 1 -- 172.21.15.120:0/2621988183 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7eff440f0f90 con 0x7eff541025c0 2024-09-19T09:18:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.439+0000 7eff4affd640 1 -- 172.21.15.120:0/1886326573 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c01c680 con 0x5613c98ad370 2024-09-19T09:18:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.439+0000 7eff4affd640 1 -- 172.21.15.120:0/1886326573 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff3c01cd00 con 0x5613c98ad370 2024-09-19T09:18:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.439+0000 7eff4affd640 1 -- 172.21.15.120:0/1886326573 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c011d40 con 0x5613c98ad370 2024-09-19T09:18:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.440+0000 7eff5ddc9740 1 -- 172.21.15.120:0/1886326573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613c98ad370 msgr2=0x5613c98af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.440+0000 7eff595cb640 1 --2- 172.21.15.120:0/2621988183 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7eff400a32f0 0x7eff400a57b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7eff5406a980 tx=0x7eff4c0090e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.440+0000 7eff5ddc9740 1 --2- 172.21.15.120:0/1886326573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613c98ad370 0x5613c98af830 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7eff3c00bcd0 tx=0x7eff3c0107a0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.440+0000 7eff5ddc9740 1 -- 172.21.15.120:0/1886326573 shutdown_connections 2024-09-19T09:18:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.440+0000 7eff5ddc9740 1 --2- 172.21.15.120:0/1886326573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613c98ad370 0x5613c98af830 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.440+0000 7eff5ddc9740 1 --2- 172.21.15.120:0/1886326573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613c98a82e0 0x5613c98acda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.440+0000 7eff5ddc9740 1 --2- 172.21.15.120:0/1886326573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613c98a79a0 0x5613c98a7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.440+0000 7eff5ddc9740 1 -- 172.21.15.120:0/1886326573 >> 172.21.15.120:0/1886326573 conn(0x5613c98a3010 msgr2=0x5613c98a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.440+0000 7eff5ddc9740 1 -- 172.21.15.120:0/1886326573 shutdown_connections 2024-09-19T09:18:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.440+0000 7eff5ddc9740 1 -- 172.21.15.120:0/1886326573 wait complete. 2024-09-19T09:18:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.440+0000 7eff5ddc9740 1 -- start start 2024-09-19T09:18:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.441+0000 7eff5ddc9740 1 Processor -- start 2024-09-19T09:18:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.441+0000 7eff5ddc9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613c98a79a0 0x5613c98b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.441+0000 7eff5ddc9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613c98a82e0 0x5613c98b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.441+0000 7eff5ddc9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613c98ad370 0x5613c98b9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.441+0000 7eff5ddc9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5613c98b0340 con 0x5613c98a82e0 2024-09-19T09:18:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.441+0000 7eff5ddc9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5613c9794290 con 0x5613c98ad370 2024-09-19T09:18:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.441+0000 7eff5ddc9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5613c98b04c0 con 0x5613c98a79a0 2024-09-19T09:18:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.441+0000 7eff5ddc9740 10 client.0 _ll_register_callbacks cb 0x5613c97c5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.441+0000 7eff53fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613c98a82e0 0x5613c98b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.441+0000 7eff595cb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613c98ad370 0x5613c98b9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.441+0000 7eff53fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613c98a82e0 0x5613c98b2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37692/0 (socket says 172.21.15.120:37692) 2024-09-19T09:18:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.441+0000 7eff53fff640 1 -- 172.21.15.120:0/3104203120 learned_addr learned my addr 172.21.15.120:0/3104203120 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.441+0000 7eff58dca640 1 --2- 172.21.15.120:0/3104203120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613c98a79a0 0x5613c98b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.442+0000 7eff53fff640 1 -- 172.21.15.120:0/3104203120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613c98a79a0 msgr2=0x5613c98b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.442+0000 7eff53fff640 1 --2- 172.21.15.120:0/3104203120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613c98a79a0 0x5613c98b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.442+0000 7eff53fff640 1 -- 172.21.15.120:0/3104203120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613c98ad370 msgr2=0x5613c98b9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.442+0000 7eff53fff640 1 --2- 172.21.15.120:0/3104203120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613c98ad370 0x5613c98b9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.442+0000 7eff53fff640 1 -- 172.21.15.120:0/3104203120 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613c951e210 con 0x5613c98a82e0 2024-09-19T09:18:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.442+0000 7eff53fff640 1 --2- 172.21.15.120:0/3104203120 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613c98a82e0 0x5613c98b2100 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7eff3c0114f0 tx=0x7eff3c0115c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe67b52e9bbe3d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.442+0000 7eff33fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:18:59.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.442+0000 7eff33fff640 1 -- 172.21.15.120:0/3104203120 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c00b8c0 con 0x5613c98a82e0 2024-09-19T09:18:59.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.442+0000 7eff33fff640 1 -- 172.21.15.120:0/3104203120 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7eff3c004020 con 0x5613c98a82e0 2024-09-19T09:18:59.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.442+0000 7eff5ddc9740 1 -- 172.21.15.120:0/3104203120 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff3c01cd00 con 0x5613c98a82e0 2024-09-19T09:18:59.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.442+0000 7eff33fff640 1 -- 172.21.15.120:0/3104203120 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7eff3c004220 con 0x5613c98a82e0 2024-09-19T09:18:59.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.442+0000 7eff5ddc9740 10 client.20895 fetch_fsmap learned FSMap version 39 2024-09-19T09:18:59.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.442+0000 7eff5ddc9740 1 -- 172.21.15.120:0/3104203120 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5613c95ea620 con 0x5613c98a82e0 2024-09-19T09:18:59.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.442+0000 7eff33fff640 1 -- 172.21.15.120:0/3104203120 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff3c01cd00 con 0x5613c98a82e0 2024-09-19T09:18:59.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.442+0000 7eff33fff640 1 -- 172.21.15.120:0/3104203120 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7eff3c01c5d0 con 0x5613c98a82e0 2024-09-19T09:18:59.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.443+0000 7eff5ddc9740 10 client.20895 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:18:59.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.443+0000 7eff5ddc9740 10 client.20895 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:18:59.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.443+0000 7eff5ddc9740 20 client.20895 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:59.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.443+0000 7eff5ddc9740 1 --2- 172.21.15.120:0/3104203120 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5613c98bd5a0 0x5613c98bf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.443+0000 7eff5ddc9740 4 client.20895 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:18:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.443+0000 7eff5ddc9740 1 -- 172.21.15.120:0/3104203120 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613c9818a20 con 0x5613c98bd5a0 2024-09-19T09:18:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.443+0000 7eff58dca640 1 --2- 172.21.15.120:0/3104203120 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5613c98bd5a0 0x5613c98bf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.443+0000 7eff58dca640 1 --2- 172.21.15.120:0/3104203120 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5613c98bd5a0 0x5613c98bf990 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bf5a96feab9015c server_cookie=dbeac96fb6f5fc72 in_seq=0 out_seq=0 2024-09-19T09:18:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.443+0000 7eff33fff640 10 client.20895 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.444+0000 7eff33fff640 1 -- 172.21.15.120:0/3104203120 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5613c9818a20 con 0x5613c98bd5a0 2024-09-19T09:18:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.444+0000 7eff33fff640 10 client.20895 handle_command_reply: tid=0 2024-09-19T09:18:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.444+0000 7eff5ddc9740 1 client.20895 shutdown 2024-09-19T09:18:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.444+0000 7eff5ddc9740 1 -- 172.21.15.120:0/3104203120 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613c98a82e0 msgr2=0x5613c98b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.444+0000 7eff5ddc9740 1 --2- 172.21.15.120:0/3104203120 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613c98a82e0 0x5613c98b2100 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7eff3c0114f0 tx=0x7eff3c0115c0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.444+0000 7eff5ddc9740 1 -- 172.21.15.120:0/3104203120 shutdown_connections 2024-09-19T09:18:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.444+0000 7eff5ddc9740 1 --2- 172.21.15.120:0/3104203120 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5613c98bd5a0 0x5613c98bf990 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.444+0000 7eff5ddc9740 1 --2- 172.21.15.120:0/3104203120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613c98ad370 0x5613c98b9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.444+0000 7eff5ddc9740 1 --2- 172.21.15.120:0/3104203120 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613c98a82e0 0x5613c98b2100 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.444+0000 7eff33fff640 0 client.20895 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:59.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.444+0000 7eff5ddc9740 1 --2- 172.21.15.120:0/3104203120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613c98a79a0 0x5613c98b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.444+0000 7eff5ddc9740 1 -- 172.21.15.120:0/3104203120 >> 172.21.15.120:0/3104203120 conn(0x5613c98a3010 msgr2=0x5613c98a4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:59.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.444+0000 7eff5ddc9740 1 -- 172.21.15.120:0/3104203120 shutdown_connections 2024-09-19T09:18:59.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.444+0000 7eff5ddc9740 1 -- 172.21.15.120:0/3104203120 wait complete. 2024-09-19T09:18:59.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.444+0000 7eff5ddc9740 20 client.20895 trim_cache size 0 max 16384 2024-09-19T09:18:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.458+0000 7eff5ddc9740 1 Processor -- start 2024-09-19T09:18:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.458+0000 7eff5ddc9740 1 -- start start 2024-09-19T09:18:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.459+0000 7eff5ddc9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613c98a79a0 0x5613c98a20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.459+0000 7eff5ddc9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613c98a82e0 0x5613c98b9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.459+0000 7eff5ddc9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613c98ad370 0x5613c98a4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.459+0000 7eff5ddc9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7eff541ab160 con 0x5613c98ad370 2024-09-19T09:18:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.459+0000 7eff5ddc9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7eff541acca0 con 0x5613c98a82e0 2024-09-19T09:18:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.459+0000 7eff5ddc9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7eff541ab830 con 0x5613c98a79a0 2024-09-19T09:18:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.459+0000 7eff595cb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613c98ad370 0x5613c98a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.459+0000 7eff595cb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613c98ad370 0x5613c98a4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37704/0 (socket says 172.21.15.120:37704) 2024-09-19T09:18:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.459+0000 7eff595cb640 1 -- 172.21.15.120:0/290375761 learned_addr learned my addr 172.21.15.120:0/290375761 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.459+0000 7eff53fff640 1 --2- 172.21.15.120:0/290375761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613c98a82e0 0x5613c98b9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.459+0000 7eff58dca640 1 --2- 172.21.15.120:0/290375761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613c98a79a0 0x5613c98a20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.459+0000 7eff595cb640 1 -- 172.21.15.120:0/290375761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613c98a79a0 msgr2=0x5613c98a20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.459+0000 7eff595cb640 1 --2- 172.21.15.120:0/290375761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613c98a79a0 0x5613c98a20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.459+0000 7eff595cb640 1 -- 172.21.15.120:0/290375761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613c98a82e0 msgr2=0x5613c98b9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.459+0000 7eff595cb640 1 --2- 172.21.15.120:0/290375761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613c98a82e0 0x5613c98b9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.459+0000 7eff595cb640 1 -- 172.21.15.120:0/290375761 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613c989e2d0 con 0x5613c98ad370 2024-09-19T09:18:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.459+0000 7eff53fff640 1 --2- 172.21.15.120:0/290375761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613c98a82e0 0x5613c98b9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:18:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.459+0000 7eff595cb640 1 --2- 172.21.15.120:0/290375761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613c98ad370 0x5613c98a4400 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7eff4c011e20 tx=0x7eff4c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2acac8eb236f041f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.459+0000 7eff4a7fc640 1 -- 172.21.15.120:0/290375761 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c027030 con 0x5613c98ad370 2024-09-19T09:18:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.460+0000 7eff4a7fc640 1 -- 172.21.15.120:0/290375761 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff4c01b070 con 0x5613c98ad370 2024-09-19T09:18:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.460+0000 7eff4a7fc640 1 -- 172.21.15.120:0/290375761 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c0223e0 con 0x5613c98ad370 2024-09-19T09:18:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.460+0000 7eff5ddc9740 1 -- 172.21.15.120:0/290375761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613c98ad370 msgr2=0x5613c98a4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.460+0000 7eff5ddc9740 1 --2- 172.21.15.120:0/290375761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613c98ad370 0x5613c98a4400 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7eff4c011e20 tx=0x7eff4c014000 comp rx=0 tx=0).stop 2024-09-19T09:18:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.460+0000 7eff5ddc9740 1 -- 172.21.15.120:0/290375761 shutdown_connections 2024-09-19T09:18:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.460+0000 7eff5ddc9740 1 --2- 172.21.15.120:0/290375761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613c98ad370 0x5613c98a4400 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.460+0000 7eff5ddc9740 1 --2- 172.21.15.120:0/290375761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613c98a82e0 0x5613c98b9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.460+0000 7eff5ddc9740 1 --2- 172.21.15.120:0/290375761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613c98a79a0 0x5613c98a20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.460+0000 7eff5ddc9740 1 -- 172.21.15.120:0/290375761 >> 172.21.15.120:0/290375761 conn(0x5613c98a3010 msgr2=0x5613c98a19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.460+0000 7eff5ddc9740 1 -- 172.21.15.120:0/290375761 shutdown_connections 2024-09-19T09:18:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.460+0000 7eff5ddc9740 1 -- 172.21.15.120:0/290375761 wait complete. 2024-09-19T09:18:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.461+0000 7eff5ddc9740 1 -- start start 2024-09-19T09:18:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.461+0000 7eff5ddc9740 1 Processor -- start 2024-09-19T09:18:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.461+0000 7eff5ddc9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613c98a79a0 0x5613c98b6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.461+0000 7eff5ddc9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613c98a82e0 0x5613c98b6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.461+0000 7eff5ddc9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613c98ad370 0x5613c98b7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.461+0000 7eff5ddc9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7eff541ac5d0 con 0x5613c98ad370 2024-09-19T09:18:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.461+0000 7eff5ddc9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7eff541abf00 con 0x5613c98a82e0 2024-09-19T09:18:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.461+0000 7eff5ddc9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7eff541a9120 con 0x5613c98a79a0 2024-09-19T09:18:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.461+0000 7eff5ddc9740 10 client.0 _ll_register_callbacks cb 0x7eff440f48e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.461+0000 7eff595cb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613c98ad370 0x5613c98b7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.461+0000 7eff595cb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613c98ad370 0x5613c98b7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37714/0 (socket says 172.21.15.120:37714) 2024-09-19T09:18:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.461+0000 7eff595cb640 1 -- 172.21.15.120:0/4111144919 learned_addr learned my addr 172.21.15.120:0/4111144919 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.461+0000 7eff58dca640 1 --2- 172.21.15.120:0/4111144919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613c98a79a0 0x5613c98b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.461+0000 7eff53fff640 1 --2- 172.21.15.120:0/4111144919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613c98a82e0 0x5613c98b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.462+0000 7eff595cb640 1 -- 172.21.15.120:0/4111144919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613c98a79a0 msgr2=0x5613c98b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.462+0000 7eff595cb640 1 --2- 172.21.15.120:0/4111144919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613c98a79a0 0x5613c98b6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.462+0000 7eff595cb640 1 -- 172.21.15.120:0/4111144919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613c98a82e0 msgr2=0x5613c98b6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.462+0000 7eff595cb640 1 --2- 172.21.15.120:0/4111144919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613c98a82e0 0x5613c98b6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.462+0000 7eff595cb640 1 -- 172.21.15.120:0/4111144919 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613c98b7e50 con 0x5613c98ad370 2024-09-19T09:18:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.462+0000 7eff58dca640 1 --2- 172.21.15.120:0/4111144919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613c98a79a0 0x5613c98b6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:18:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.462+0000 7eff595cb640 1 --2- 172.21.15.120:0/4111144919 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613c98ad370 0x5613c98b7840 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7eff4c014fd0 tx=0x7eff4c0236b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766c7745bc650df4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.462+0000 7eff33fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:18:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.462+0000 7eff33fff640 1 -- 172.21.15.120:0/4111144919 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c027040 con 0x5613c98ad370 2024-09-19T09:18:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.462+0000 7eff33fff640 1 -- 172.21.15.120:0/4111144919 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7eff4c036610 con 0x5613c98ad370 2024-09-19T09:18:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.462+0000 7eff5ddc9740 1 -- 172.21.15.120:0/4111144919 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff4c01b070 con 0x5613c98ad370 2024-09-19T09:18:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.462+0000 7eff33fff640 1 -- 172.21.15.120:0/4111144919 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7eff4c036810 con 0x5613c98ad370 2024-09-19T09:18:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.462+0000 7eff5ddc9740 10 client.20901 fetch_fsmap learned FSMap version 39 2024-09-19T09:18:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.462+0000 7eff5ddc9740 1 -- 172.21.15.120:0/4111144919 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5613c98b9160 con 0x5613c98ad370 2024-09-19T09:18:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.462+0000 7eff33fff640 1 -- 172.21.15.120:0/4111144919 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff4c01b070 con 0x5613c98ad370 2024-09-19T09:18:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.462+0000 7eff33fff640 1 -- 172.21.15.120:0/4111144919 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7eff4c020050 con 0x5613c98ad370 2024-09-19T09:18:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.462+0000 7eff5ddc9740 10 client.20901 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:18:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.462+0000 7eff5ddc9740 10 client.20901 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:18:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.462+0000 7eff5ddc9740 20 client.20901 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.462+0000 7eff5ddc9740 1 --2- 172.21.15.120:0/4111144919 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5613c9928640 0x5613c992aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.462+0000 7eff5ddc9740 4 client.20901 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:18:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.462+0000 7eff5ddc9740 1 -- 172.21.15.120:0/4111144919 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5613c9812d70 con 0x5613c9928640 2024-09-19T09:18:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.463+0000 7eff58dca640 1 --2- 172.21.15.120:0/4111144919 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5613c9928640 0x5613c992aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.463+0000 7eff33fff640 10 client.20901 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.463+0000 7eff58dca640 1 --2- 172.21.15.120:0/4111144919 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5613c9928640 0x5613c992aa60 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3823a1f076675e0 server_cookie=722e09e71061cf23 in_seq=0 out_seq=0 2024-09-19T09:18:59.472 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db --set-id 792ee6f9 --release --await' 2024-09-19T09:18:59.472 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db --set-id 792ee6f9 --release --await 2024-09-19T09:18:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.558+0000 7eff33fff640 1 -- 172.21.15.120:0/4111144919 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5613c9812d70 con 0x5613c9928640 2024-09-19T09:18:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.558+0000 7eff33fff640 10 client.20901 handle_command_reply: tid=0 2024-09-19T09:18:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.558+0000 7eff5ddc9740 1 client.20901 shutdown 2024-09-19T09:18:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.558+0000 7eff5ddc9740 1 -- 172.21.15.120:0/4111144919 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613c98ad370 msgr2=0x5613c98b7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.558+0000 7eff5ddc9740 1 --2- 172.21.15.120:0/4111144919 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613c98ad370 0x5613c98b7840 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7eff4c014fd0 tx=0x7eff4c0236b0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.558+0000 7eff5ddc9740 1 -- 172.21.15.120:0/4111144919 shutdown_connections 2024-09-19T09:18:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.558+0000 7eff5ddc9740 1 --2- 172.21.15.120:0/4111144919 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5613c9928640 0x5613c992aa60 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.558+0000 7eff5ddc9740 1 --2- 172.21.15.120:0/4111144919 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613c98ad370 0x5613c98b7840 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.558+0000 7eff5ddc9740 1 --2- 172.21.15.120:0/4111144919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613c98a82e0 0x5613c98b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.558+0000 7eff5ddc9740 1 --2- 172.21.15.120:0/4111144919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5613c98a79a0 0x5613c98b6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.558+0000 7eff33fff640 0 client.20901 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:18:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.559+0000 7eff5ddc9740 1 -- 172.21.15.120:0/4111144919 >> 172.21.15.120:0/4111144919 conn(0x5613c98a3010 msgr2=0x5613c98a21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.559+0000 7eff5ddc9740 1 -- 172.21.15.120:0/4111144919 shutdown_connections 2024-09-19T09:18:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.559+0000 7eff5ddc9740 1 -- 172.21.15.120:0/4111144919 wait complete. 2024-09-19T09:18:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.559+0000 7eff5ddc9740 20 client.20901 trim_cache size 0 max 16384 2024-09-19T09:18:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:18:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:18:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:18:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:18:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.559+0000 7eff5b02e640 1 -- 172.21.15.120:0/2621988183 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7eff400a32f0 msgr2=0x7eff400a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.559+0000 7eff5b02e640 1 --2- 172.21.15.120:0/2621988183 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7eff400a32f0 0x7eff400a57b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7eff5406a980 tx=0x7eff4c0090e0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.559+0000 7eff5b02e640 1 -- 172.21.15.120:0/2621988183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff541025c0 msgr2=0x7eff54197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.559+0000 7eff5b02e640 1 --2- 172.21.15.120:0/2621988183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff541025c0 0x7eff54197e50 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7eff44031a40 tx=0x7eff44031ec0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.559+0000 7eff5b02e640 1 -- 172.21.15.120:0/2621988183 shutdown_connections 2024-09-19T09:18:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.559+0000 7eff5b02e640 1 --2- 172.21.15.120:0/2621988183 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7eff400a32f0 0x7eff400a57b0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.559+0000 7eff5b02e640 1 --2- 172.21.15.120:0/2621988183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff54107820 0x7eff5419f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.559+0000 7eff5b02e640 1 --2- 172.21.15.120:0/2621988183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff54104ef0 0x7eff54198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.559+0000 7eff5b02e640 1 --2- 172.21.15.120:0/2621988183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff541025c0 0x7eff54197e50 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.559+0000 7eff5b02e640 1 -- 172.21.15.120:0/2621988183 >> 172.21.15.120:0/2621988183 conn(0x7eff540fc1e0 msgr2=0x7eff54102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.559+0000 7eff5b02e640 1 -- 172.21.15.120:0/2621988183 shutdown_connections 2024-09-19T09:18:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:18:59.559+0000 7eff5b02e640 1 -- 172.21.15.120:0/2621988183 wait complete. 2024-09-19T09:18:59.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.681+0000 7fccdca66640 1 -- 172.21.15.120:0/1793642789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccd8100e20 msgr2=0x7fccd8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:59.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.681+0000 7fccdca66640 1 --2- 172.21.15.120:0/1793642789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccd8100e20 0x7fccd8101200 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009900 tx=0x7fccc802a480 comp rx=0 tx=0).stop 2024-09-19T09:18:59.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.682+0000 7fccdca66640 1 -- 172.21.15.120:0/1793642789 shutdown_connections 2024-09-19T09:18:59.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.682+0000 7fccdca66640 1 --2- 172.21.15.120:0/1793642789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccd8102ef0 0x7fccd810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.682+0000 7fccdca66640 1 --2- 172.21.15.120:0/1793642789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccd8102570 0x7fccd81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.682+0000 7fccdca66640 1 --2- 172.21.15.120:0/1793642789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccd8100e20 0x7fccd8101200 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.682+0000 7fccdca66640 1 -- 172.21.15.120:0/1793642789 >> 172.21.15.120:0/1793642789 conn(0x7fccd80fc7d0 msgr2=0x7fccd80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:59.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.682+0000 7fccdca66640 1 -- 172.21.15.120:0/1793642789 shutdown_connections 2024-09-19T09:18:59.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.682+0000 7fccdca66640 1 -- 172.21.15.120:0/1793642789 wait complete. 2024-09-19T09:18:59.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.682+0000 7fccdca66640 1 Processor -- start 2024-09-19T09:18:59.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.682+0000 7fccdca66640 1 -- start start 2024-09-19T09:18:59.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.683+0000 7fccdca66640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccd8100e20 0x7fccd8197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.683+0000 7fccdca66640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccd8102570 0x7fccd8197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.683+0000 7fccdca66640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccd8102ef0 0x7fccd819bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.683+0000 7fccdca66640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8110fa0 con 0x7fccd8100e20 2024-09-19T09:18:59.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.683+0000 7fccdca66640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8110e20 con 0x7fccd8102570 2024-09-19T09:18:59.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.683+0000 7fccdca66640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8111120 con 0x7fccd8102ef0 2024-09-19T09:18:59.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.683+0000 7fccd659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccd8100e20 0x7fccd8197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.683+0000 7fccd659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccd8100e20 0x7fccd8197420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37736/0 (socket says 172.21.15.120:37736) 2024-09-19T09:18:59.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.683+0000 7fccd659c640 1 -- 172.21.15.120:0/4238987461 learned_addr learned my addr 172.21.15.120:0/4238987461 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:59.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.683+0000 7fccd5d9b640 1 --2- 172.21.15.120:0/4238987461 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccd8102570 0x7fccd8197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.683+0000 7fccd6d9d640 1 --2- 172.21.15.120:0/4238987461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccd8102ef0 0x7fccd819bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.683+0000 7fccd659c640 1 -- 172.21.15.120:0/4238987461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccd8102ef0 msgr2=0x7fccd819bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:59.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.683+0000 7fccd659c640 1 --2- 172.21.15.120:0/4238987461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccd8102ef0 0x7fccd819bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.683+0000 7fccd659c640 1 -- 172.21.15.120:0/4238987461 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccd8102570 msgr2=0x7fccd8197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:59.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.683+0000 7fccd659c640 1 --2- 172.21.15.120:0/4238987461 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccd8102570 0x7fccd8197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.683+0000 7fccd659c640 1 -- 172.21.15.120:0/4238987461 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd819c3d0 con 0x7fccd8100e20 2024-09-19T09:18:59.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.683+0000 7fccd659c640 1 --2- 172.21.15.120:0/4238987461 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccd8100e20 0x7fccd8197420 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fccc8006010 tx=0x7fccc8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:59.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.683+0000 7fccc77fe640 1 -- 172.21.15.120:0/4238987461 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc803f070 con 0x7fccd8100e20 2024-09-19T09:18:59.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.683+0000 7fccc77fe640 1 -- 172.21.15.120:0/4238987461 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc8031c30 con 0x7fccd8100e20 2024-09-19T09:18:59.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.683+0000 7fccdca66640 1 -- 172.21.15.120:0/4238987461 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd819c660 con 0x7fccd8100e20 2024-09-19T09:18:59.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.683+0000 7fccc77fe640 1 -- 172.21.15.120:0/4238987461 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8035030 con 0x7fccd8100e20 2024-09-19T09:18:59.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.683+0000 7fccdca66640 1 -- 172.21.15.120:0/4238987461 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd81a3fa0 con 0x7fccd8100e20 2024-09-19T09:18:59.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.685+0000 7fccc77fe640 1 -- 172.21.15.120:0/4238987461 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fccc8042070 con 0x7fccd8100e20 2024-09-19T09:18:59.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.686+0000 7fccdee3f740 1 Processor -- start 2024-09-19T09:18:59.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.686+0000 7fccdee3f740 1 -- start start 2024-09-19T09:18:59.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.686+0000 7fccdee3f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56309e5e0b40 0x56309e5df090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.686+0000 7fccdee3f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56309e5df5d0 0x56309e5dfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.686+0000 7fccdee3f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56309e5ea370 0x56309e5ec760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.686+0000 7fccc77fe640 1 --2- 172.21.15.120:0/4238987461 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fccc00a32c0 0x7fccc00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.686+0000 7fccdee3f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56309e5452f0 con 0x56309e5ea370 2024-09-19T09:18:59.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.686+0000 7fccdee3f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56309e4eccf0 con 0x56309e5df5d0 2024-09-19T09:18:59.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.686+0000 7fccdee3f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56309e5412b0 con 0x56309e5e0b40 2024-09-19T09:18:59.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.687+0000 7fccd5d9b640 1 --2- 172.21.15.120:0/4238987461 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fccc00a32c0 0x7fccc00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.687+0000 7fccc77fe640 1 -- 172.21.15.120:0/4238987461 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fccc80effa0 con 0x7fccd8100e20 2024-09-19T09:18:59.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.687+0000 7fccd659c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56309e5df5d0 0x56309e5dfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.687+0000 7fccd6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56309e5ea370 0x56309e5ec760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.687+0000 7fccd659c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56309e5df5d0 0x56309e5dfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:47730/0 (socket says 172.21.15.120:47730) 2024-09-19T09:18:59.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.687+0000 7fccd659c640 1 -- 172.21.15.120:0/75619616 learned_addr learned my addr 172.21.15.120:0/75619616 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:59.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.687+0000 7fccd659c640 1 -- 172.21.15.120:0/75619616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56309e5e0b40 msgr2=0x56309e5df090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:18:59.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.687+0000 7fccd659c640 1 --2- 172.21.15.120:0/75619616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56309e5e0b40 0x56309e5df090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.687+0000 7fccd659c640 1 -- 172.21.15.120:0/75619616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56309e5ea370 msgr2=0x56309e5ec760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:59.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.687+0000 7fccd659c640 1 --2- 172.21.15.120:0/75619616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56309e5ea370 0x56309e5ec760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.687+0000 7fccd659c640 1 -- 172.21.15.120:0/75619616 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56309e4d13a0 con 0x56309e5df5d0 2024-09-19T09:18:59.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.687+0000 7fccd6d9d640 1 --2- 172.21.15.120:0/75619616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56309e5ea370 0x56309e5ec760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:18:59.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.687+0000 7fccd5d9b640 1 --2- 172.21.15.120:0/4238987461 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fccc00a32c0 0x7fccc00a5780 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fccd80ffdc0 tx=0x7fcccc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:59.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.687+0000 7fccd659c640 1 --2- 172.21.15.120:0/75619616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56309e5df5d0 0x56309e5dfa10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fccc80edb40 tx=0x7fccc80bc250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21499df54b1cad43 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:59.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.687+0000 7fccc4ff9640 1 -- 172.21.15.120:0/75619616 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8049070 con 0x56309e5df5d0 2024-09-19T09:18:59.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.687+0000 7fccc4ff9640 1 -- 172.21.15.120:0/75619616 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc80cc110 con 0x56309e5df5d0 2024-09-19T09:18:59.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.688+0000 7fccc4ff9640 1 -- 172.21.15.120:0/75619616 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc80cc4d0 con 0x56309e5df5d0 2024-09-19T09:18:59.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.688+0000 7fccdee3f740 1 -- 172.21.15.120:0/75619616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56309e5df5d0 msgr2=0x56309e5dfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:59.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.688+0000 7fccdee3f740 1 --2- 172.21.15.120:0/75619616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56309e5df5d0 0x56309e5dfa10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fccc80edb40 tx=0x7fccc80bc250 comp rx=0 tx=0).stop 2024-09-19T09:18:59.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.688+0000 7fccdee3f740 1 -- 172.21.15.120:0/75619616 shutdown_connections 2024-09-19T09:18:59.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.688+0000 7fccdee3f740 1 --2- 172.21.15.120:0/75619616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56309e5ea370 0x56309e5ec760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.688+0000 7fccdee3f740 1 --2- 172.21.15.120:0/75619616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56309e5df5d0 0x56309e5dfa10 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.688+0000 7fccdee3f740 1 --2- 172.21.15.120:0/75619616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56309e5e0b40 0x56309e5df090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.688+0000 7fccdee3f740 1 -- 172.21.15.120:0/75619616 >> 172.21.15.120:0/75619616 conn(0x56309e5a8310 msgr2=0x56309e5dbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:59.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.689+0000 7fccdee3f740 1 -- 172.21.15.120:0/75619616 shutdown_connections 2024-09-19T09:18:59.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.689+0000 7fccdee3f740 1 -- 172.21.15.120:0/75619616 wait complete. 2024-09-19T09:18:59.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.689+0000 7fccdee3f740 1 -- start start 2024-09-19T09:18:59.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.689+0000 7fccdee3f740 1 Processor -- start 2024-09-19T09:18:59.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.689+0000 7fccdee3f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56309e5df5d0 0x56309e5e9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.689+0000 7fccdee3f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56309e5e0b40 0x56309e5e7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.689+0000 7fccdee3f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56309e5ea370 0x56309e5e77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.689+0000 7fccdee3f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56309e5ece30 con 0x56309e5e0b40 2024-09-19T09:18:59.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.689+0000 7fccdee3f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56309e500060 con 0x56309e5ea370 2024-09-19T09:18:59.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.689+0000 7fccdee3f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56309e5ecfb0 con 0x56309e5df5d0 2024-09-19T09:18:59.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.689+0000 7fccdee3f740 10 client.0 _ll_register_callbacks cb 0x56309e2538c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:59.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.689+0000 7fccd5d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56309e5e0b40 0x56309e5e7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.690+0000 7fccd659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56309e5df5d0 0x56309e5e9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.690+0000 7fccd6d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56309e5ea370 0x56309e5e77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.690+0000 7fccd659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56309e5df5d0 0x56309e5e9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:55684/0 (socket says 172.21.15.120:55684) 2024-09-19T09:18:59.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.690+0000 7fccd659c640 1 -- 172.21.15.120:0/3770748215 learned_addr learned my addr 172.21.15.120:0/3770748215 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:59.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.690+0000 7fccd6d9d640 1 -- 172.21.15.120:0/3770748215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56309e5df5d0 msgr2=0x56309e5e9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:59.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.690+0000 7fccd6d9d640 1 --2- 172.21.15.120:0/3770748215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56309e5df5d0 0x56309e5e9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.690+0000 7fccd6d9d640 1 -- 172.21.15.120:0/3770748215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56309e5e0b40 msgr2=0x56309e5e7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:59.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.690+0000 7fccd6d9d640 1 --2- 172.21.15.120:0/3770748215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56309e5e0b40 0x56309e5e7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.690+0000 7fccd6d9d640 1 -- 172.21.15.120:0/3770748215 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56309e2b63c0 con 0x56309e5ea370 2024-09-19T09:18:59.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.690+0000 7fccd6d9d640 1 --2- 172.21.15.120:0/3770748215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56309e5ea370 0x56309e5e77d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fccd000aa70 tx=0x7fccd000af30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c96b60fbabae1a4b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:59.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.690+0000 7fccbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:18:59.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.690+0000 7fccbdffb640 1 -- 172.21.15.120:0/3770748215 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd0008e30 con 0x56309e5ea370 2024-09-19T09:18:59.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.690+0000 7fccdee3f740 1 -- 172.21.15.120:0/3770748215 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccc80cc110 con 0x56309e5ea370 2024-09-19T09:18:59.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.690+0000 7fccbdffb640 1 -- 172.21.15.120:0/3770748215 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fccd0016460 con 0x56309e5ea370 2024-09-19T09:18:59.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.690+0000 7fccbdffb640 1 -- 172.21.15.120:0/3770748215 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fccd0016660 con 0x56309e5ea370 2024-09-19T09:18:59.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.690+0000 7fccdee3f740 10 client.29780 fetch_fsmap learned FSMap version 39 2024-09-19T09:18:59.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.690+0000 7fccdee3f740 1 -- 172.21.15.120:0/3770748215 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56309e5f8950 con 0x56309e5ea370 2024-09-19T09:18:59.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.691+0000 7fccbdffb640 1 -- 172.21.15.120:0/3770748215 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc80cc110 con 0x56309e5ea370 2024-09-19T09:18:59.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.691+0000 7fccbdffb640 1 -- 172.21.15.120:0/3770748215 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fccd001fa50 con 0x56309e5ea370 2024-09-19T09:18:59.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.691+0000 7fccdee3f740 10 client.29780 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:18:59.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.691+0000 7fccdee3f740 10 client.29780 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:18:59.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.691+0000 7fccdee3f740 20 client.29780 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:59.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.692+0000 7fccdee3f740 1 --2- 172.21.15.120:0/3770748215 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56309e5f9db0 0x56309e5fc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.692+0000 7fccdee3f740 4 client.29780 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:18:59.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.692+0000 7fccdee3f740 1 -- 172.21.15.120:0/3770748215 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56309e552860 con 0x56309e5f9db0 2024-09-19T09:18:59.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.692+0000 7fccd659c640 1 --2- 172.21.15.120:0/3770748215 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56309e5f9db0 0x56309e5fc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.692+0000 7fccd659c640 1 --2- 172.21.15.120:0/3770748215 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56309e5f9db0 0x56309e5fc1a0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ad1f1ab8f5ad233f server_cookie=6e86eba57d9787f5 in_seq=0 out_seq=0 2024-09-19T09:18:59.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.692+0000 7fccbdffb640 10 client.29780 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:18:59.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.693+0000 7fccbdffb640 1 -- 172.21.15.120:0/3770748215 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56309e552860 con 0x56309e5f9db0 2024-09-19T09:18:59.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.693+0000 7fccbdffb640 10 client.29780 handle_command_reply: tid=0 2024-09-19T09:18:59.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.693+0000 7fccdee3f740 1 client.29780 shutdown 2024-09-19T09:18:59.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.693+0000 7fccdee3f740 1 -- 172.21.15.120:0/3770748215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56309e5ea370 msgr2=0x56309e5e77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:59.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.693+0000 7fccdee3f740 1 --2- 172.21.15.120:0/3770748215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56309e5ea370 0x56309e5e77d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fccd000aa70 tx=0x7fccd000af30 comp rx=0 tx=0).stop 2024-09-19T09:18:59.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.693+0000 7fccdee3f740 1 -- 172.21.15.120:0/3770748215 shutdown_connections 2024-09-19T09:18:59.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.693+0000 7fccdee3f740 1 --2- 172.21.15.120:0/3770748215 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56309e5f9db0 0x56309e5fc1a0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.693+0000 7fccdee3f740 1 --2- 172.21.15.120:0/3770748215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56309e5ea370 0x56309e5e77d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.693+0000 7fccdee3f740 1 --2- 172.21.15.120:0/3770748215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56309e5e0b40 0x56309e5e7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.693+0000 7fccdee3f740 1 --2- 172.21.15.120:0/3770748215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56309e5df5d0 0x56309e5e9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.693+0000 7fccdee3f740 1 -- 172.21.15.120:0/3770748215 >> 172.21.15.120:0/3770748215 conn(0x56309e5a8310 msgr2=0x56309e5e19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:59.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.693+0000 7fccbdffb640 0 client.29780 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:18:59.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.693+0000 7fccdee3f740 1 -- 172.21.15.120:0/3770748215 shutdown_connections 2024-09-19T09:18:59.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.693+0000 7fccdee3f740 1 -- 172.21.15.120:0/3770748215 wait complete. 2024-09-19T09:18:59.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.693+0000 7fccdee3f740 20 client.29780 trim_cache size 0 max 16384 2024-09-19T09:18:59.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.716+0000 7fccdee3f740 1 Processor -- start 2024-09-19T09:18:59.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.716+0000 7fccdee3f740 1 -- start start 2024-09-19T09:18:59.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.717+0000 7fccdee3f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56309e5df5d0 0x56309e5f6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.717+0000 7fccdee3f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56309e5e0b40 0x56309e5e6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.717+0000 7fccdee3f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56309e5ea370 0x56309e601a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.717+0000 7fccdee3f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fccd81ab3f0 con 0x56309e5e0b40 2024-09-19T09:18:59.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.717+0000 7fccdee3f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fccd81acf30 con 0x56309e5df5d0 2024-09-19T09:18:59.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.717+0000 7fccdee3f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fccd81abac0 con 0x56309e5ea370 2024-09-19T09:18:59.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.717+0000 7fccd659c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56309e5df5d0 0x56309e5f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.717+0000 7fccd659c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56309e5df5d0 0x56309e5f6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:47752/0 (socket says 172.21.15.120:47752) 2024-09-19T09:18:59.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.717+0000 7fccd659c640 1 -- 172.21.15.120:0/859446105 learned_addr learned my addr 172.21.15.120:0/859446105 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:59.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.717+0000 7fccd6d9d640 1 --2- 172.21.15.120:0/859446105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56309e5ea370 0x56309e601a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.717+0000 7fccd5d9b640 1 --2- 172.21.15.120:0/859446105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56309e5e0b40 0x56309e5e6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.717+0000 7fccd659c640 1 -- 172.21.15.120:0/859446105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56309e5ea370 msgr2=0x56309e601a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:59.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.717+0000 7fccd659c640 1 --2- 172.21.15.120:0/859446105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56309e5ea370 0x56309e601a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.717+0000 7fccd659c640 1 -- 172.21.15.120:0/859446105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56309e5e0b40 msgr2=0x56309e5e6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:59.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.717+0000 7fccd659c640 1 --2- 172.21.15.120:0/859446105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56309e5e0b40 0x56309e5e6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.717+0000 7fccd659c640 1 -- 172.21.15.120:0/859446105 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56309e5dc960 con 0x56309e5df5d0 2024-09-19T09:18:59.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.718+0000 7fccd659c640 1 --2- 172.21.15.120:0/859446105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56309e5df5d0 0x56309e5f6090 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fccc80b8b70 tx=0x7fccc803b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f67e8dae0331ad3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.719+0000 7fccbffff640 1 -- 172.21.15.120:0/859446105 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8049040 con 0x56309e5df5d0 2024-09-19T09:18:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.719+0000 7fccbffff640 1 -- 172.21.15.120:0/859446105 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc80c0a10 con 0x56309e5df5d0 2024-09-19T09:18:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.719+0000 7fccbffff640 1 -- 172.21.15.120:0/859446105 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc80bcbd0 con 0x56309e5df5d0 2024-09-19T09:18:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.720+0000 7fccdee3f740 1 -- 172.21.15.120:0/859446105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56309e5df5d0 msgr2=0x56309e5f6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.720+0000 7fccdee3f740 1 --2- 172.21.15.120:0/859446105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56309e5df5d0 0x56309e5f6090 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fccc80b8b70 tx=0x7fccc803b870 comp rx=0 tx=0).stop 2024-09-19T09:18:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.720+0000 7fccdee3f740 1 -- 172.21.15.120:0/859446105 shutdown_connections 2024-09-19T09:18:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.720+0000 7fccdee3f740 1 --2- 172.21.15.120:0/859446105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56309e5ea370 0x56309e601a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.720+0000 7fccdee3f740 1 --2- 172.21.15.120:0/859446105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56309e5e0b40 0x56309e5e6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.720+0000 7fccdee3f740 1 --2- 172.21.15.120:0/859446105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56309e5df5d0 0x56309e5f6090 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.720+0000 7fccdee3f740 1 -- 172.21.15.120:0/859446105 >> 172.21.15.120:0/859446105 conn(0x56309e5a8310 msgr2=0x56309e5e5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.720+0000 7fccdee3f740 1 -- 172.21.15.120:0/859446105 shutdown_connections 2024-09-19T09:18:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.720+0000 7fccdee3f740 1 -- 172.21.15.120:0/859446105 wait complete. 2024-09-19T09:18:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.720+0000 7fccdee3f740 1 -- start start 2024-09-19T09:18:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.721+0000 7fccdee3f740 1 Processor -- start 2024-09-19T09:18:59.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.721+0000 7fccdee3f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56309e5df5d0 0x56309e5f2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.721+0000 7fccdee3f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56309e5e0b40 0x56309e5eda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.721+0000 7fccdee3f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56309e5ea370 0x56309e5edf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.721+0000 7fccd5d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56309e5e0b40 0x56309e5eda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.721+0000 7fccd5d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56309e5e0b40 0x56309e5eda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:47754/0 (socket says 172.21.15.120:47754) 2024-09-19T09:18:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.721+0000 7fccd6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56309e5ea370 0x56309e5edf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.721+0000 7fccd6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56309e5ea370 0x56309e5edf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37780/0 (socket says 172.21.15.120:37780) 2024-09-19T09:18:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.721+0000 7fccd5d9b640 1 -- 172.21.15.120:0/4294891366 learned_addr learned my addr 172.21.15.120:0/4294891366 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:18:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.721+0000 7fccdee3f740 1 -- 172.21.15.120:0/4294891366 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fccd81ac860 con 0x56309e5ea370 2024-09-19T09:18:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.721+0000 7fccdee3f740 1 -- 172.21.15.120:0/4294891366 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fccd81ac190 con 0x56309e5e0b40 2024-09-19T09:18:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.721+0000 7fccdee3f740 1 -- 172.21.15.120:0/4294891366 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fccd81a93b0 con 0x56309e5df5d0 2024-09-19T09:18:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.722+0000 7fccd659c640 1 --2- 172.21.15.120:0/4294891366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56309e5df5d0 0x56309e5f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.722+0000 7fccdee3f740 10 client.0 _ll_register_callbacks cb 0x7fccd0009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:18:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.722+0000 7fccd5d9b640 1 -- 172.21.15.120:0/4294891366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56309e5df5d0 msgr2=0x56309e5f2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:59.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.722+0000 7fccd5d9b640 1 --2- 172.21.15.120:0/4294891366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56309e5df5d0 0x56309e5f2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.722+0000 7fccd5d9b640 1 -- 172.21.15.120:0/4294891366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56309e5ea370 msgr2=0x56309e5edf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:18:59.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.722+0000 7fccd5d9b640 1 --2- 172.21.15.120:0/4294891366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56309e5ea370 0x56309e5edf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.722+0000 7fccd5d9b640 1 -- 172.21.15.120:0/4294891366 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56309e5ee540 con 0x56309e5e0b40 2024-09-19T09:18:59.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.722+0000 7fccd659c640 1 --2- 172.21.15.120:0/4294891366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56309e5df5d0 0x56309e5f2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:18:59.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.722+0000 7fccd5d9b640 1 --2- 172.21.15.120:0/4294891366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56309e5e0b40 0x56309e5eda40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcccc00b9c0 tx=0x7fcccc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abcb764d465c56aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:59.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.722+0000 7fccbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:18:59.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.723+0000 7fccbdffb640 1 -- 172.21.15.120:0/4294891366 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc011ae0 con 0x56309e5e0b40 2024-09-19T09:18:59.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.723+0000 7fccbdffb640 1 -- 172.21.15.120:0/4294891366 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fcccc011c80 con 0x56309e5e0b40 2024-09-19T09:18:59.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.723+0000 7fccdee3f740 1 -- 172.21.15.120:0/4294891366 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccc80c0a10 con 0x56309e5e0b40 2024-09-19T09:18:59.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.723+0000 7fccbdffb640 1 -- 172.21.15.120:0/4294891366 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcccc0047a0 con 0x56309e5e0b40 2024-09-19T09:18:59.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.723+0000 7fccdee3f740 10 client.29786 fetch_fsmap learned FSMap version 39 2024-09-19T09:18:59.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.723+0000 7fccdee3f740 1 -- 172.21.15.120:0/4294891366 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56309e664790 con 0x56309e5e0b40 2024-09-19T09:18:59.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.723+0000 7fccbdffb640 1 -- 172.21.15.120:0/4294891366 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc80c0a10 con 0x56309e5e0b40 2024-09-19T09:18:59.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.723+0000 7fccbdffb640 1 -- 172.21.15.120:0/4294891366 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fcccc053570 con 0x56309e5e0b40 2024-09-19T09:18:59.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.723+0000 7fccdee3f740 10 client.29786 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:18:59.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.723+0000 7fccdee3f740 10 client.29786 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:18:59.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.723+0000 7fccdee3f740 20 client.29786 populate_metadata read hostname 'smithi120' 2024-09-19T09:18:59.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.723+0000 7fccdee3f740 1 --2- 172.21.15.120:0/4294891366 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56309e664f70 0x56309e667390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.723+0000 7fccdee3f740 4 client.29786 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:18:59.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.723+0000 7fccdee3f740 1 -- 172.21.15.120:0/4294891366 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56309e54cd70 con 0x56309e664f70 2024-09-19T09:18:59.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.724+0000 7fccd659c640 1 --2- 172.21.15.120:0/4294891366 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56309e664f70 0x56309e667390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.724+0000 7fccbdffb640 10 client.29786 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:18:59.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.724+0000 7fccd659c640 1 --2- 172.21.15.120:0/4294891366 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56309e664f70 0x56309e667390 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fdfa902822363bdd server_cookie=df4e6c452e165223 in_seq=0 out_seq=0 2024-09-19T09:18:59.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.725+0000 7fccbdffb640 1 -- 172.21.15.120:0/4294891366 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56309e54cd70 con 0x56309e664f70 2024-09-19T09:18:59.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.725+0000 7fccbdffb640 10 client.29786 handle_command_reply: tid=0 2024-09-19T09:18:59.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.725+0000 7fccdee3f740 1 client.29786 shutdown 2024-09-19T09:18:59.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.725+0000 7fccdee3f740 1 -- 172.21.15.120:0/4294891366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56309e5e0b40 msgr2=0x56309e5eda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:59.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.725+0000 7fccdee3f740 1 --2- 172.21.15.120:0/4294891366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56309e5e0b40 0x56309e5eda40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcccc00b9c0 tx=0x7fcccc004730 comp rx=0 tx=0).stop 2024-09-19T09:18:59.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.725+0000 7fccdee3f740 1 -- 172.21.15.120:0/4294891366 shutdown_connections 2024-09-19T09:18:59.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.725+0000 7fccdee3f740 1 --2- 172.21.15.120:0/4294891366 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56309e664f70 0x56309e667390 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.725+0000 7fccdee3f740 1 --2- 172.21.15.120:0/4294891366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56309e5ea370 0x56309e5edf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.725+0000 7fccdee3f740 1 --2- 172.21.15.120:0/4294891366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56309e5e0b40 0x56309e5eda40 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.725+0000 7fccdee3f740 1 --2- 172.21.15.120:0/4294891366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56309e5df5d0 0x56309e5f2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.725+0000 7fccdee3f740 1 -- 172.21.15.120:0/4294891366 >> 172.21.15.120:0/4294891366 conn(0x56309e5a8310 msgr2=0x56309e5e6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:59.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.725+0000 7fccbdffb640 0 client.29786 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:18:59.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.725+0000 7fccdee3f740 1 -- 172.21.15.120:0/4294891366 shutdown_connections 2024-09-19T09:18:59.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.726+0000 7fccdee3f740 1 -- 172.21.15.120:0/4294891366 wait complete. 2024-09-19T09:18:59.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.726+0000 7fccdee3f740 20 client.29786 trim_cache size 0 max 16384 2024-09-19T09:18:59.950 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:18:59.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.726+0000 7fccdca66640 1 -- 172.21.15.120:0/4238987461 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fccc00a32c0 msgr2=0x7fccc00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:59.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.726+0000 7fccdca66640 1 --2- 172.21.15.120:0/4238987461 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fccc00a32c0 0x7fccc00a5780 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fccd80ffdc0 tx=0x7fcccc00b390 comp rx=0 tx=0).stop 2024-09-19T09:18:59.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.726+0000 7fccdca66640 1 -- 172.21.15.120:0/4238987461 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccd8100e20 msgr2=0x7fccd8197420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:59.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.726+0000 7fccdca66640 1 --2- 172.21.15.120:0/4238987461 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccd8100e20 0x7fccd8197420 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fccc8006010 tx=0x7fccc8003c10 comp rx=0 tx=0).stop 2024-09-19T09:18:59.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.726+0000 7fccdca66640 1 -- 172.21.15.120:0/4238987461 shutdown_connections 2024-09-19T09:18:59.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.726+0000 7fccdca66640 1 --2- 172.21.15.120:0/4238987461 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fccc00a32c0 0x7fccc00a5780 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.726+0000 7fccdca66640 1 --2- 172.21.15.120:0/4238987461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccd8102ef0 0x7fccd819bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.726+0000 7fccdca66640 1 --2- 172.21.15.120:0/4238987461 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccd8102570 0x7fccd8197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.726+0000 7fccdca66640 1 --2- 172.21.15.120:0/4238987461 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccd8100e20 0x7fccd8197420 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.726+0000 7fccdca66640 1 -- 172.21.15.120:0/4238987461 >> 172.21.15.120:0/4238987461 conn(0x7fccd80fc7d0 msgr2=0x7fccd80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:59.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.727+0000 7fccdca66640 1 -- 172.21.15.120:0/4238987461 shutdown_connections 2024-09-19T09:18:59.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:18:59.727+0000 7fccdca66640 1 -- 172.21.15.120:0/4238987461 wait complete. 2024-09-19T09:19:00.361 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57910 1.95 MB/sec execute 55 sec latency 29691.445 ms 2024-09-19T09:19:00.496 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:19:00.515 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:19:01.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.002+0000 7fe5069c2640 1 -- 172.21.15.120:0/3785179929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe500102780 msgr2=0x7fe5001067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:01.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.002+0000 7fe5069c2640 1 --2- 172.21.15.120:0/3785179929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe500102780 0x7fe5001067d0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009900 tx=0x7fe4f402a410 comp rx=0 tx=0).stop 2024-09-19T09:19:01.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.002+0000 7fe5069c2640 1 -- 172.21.15.120:0/3785179929 shutdown_connections 2024-09-19T09:19:01.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.002+0000 7fe5069c2640 1 --2- 172.21.15.120:0/3785179929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe500106f00 0x7fe50010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.002+0000 7fe5069c2640 1 --2- 172.21.15.120:0/3785179929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe500102780 0x7fe5001067d0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.002+0000 7fe5069c2640 1 --2- 172.21.15.120:0/3785179929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe500101dd0 0x7fe5001021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.002+0000 7fe5069c2640 1 -- 172.21.15.120:0/3785179929 >> 172.21.15.120:0/3785179929 conn(0x7fe5000fd8b0 msgr2=0x7fe5000ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:01.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.002+0000 7fe5069c2640 1 -- 172.21.15.120:0/3785179929 shutdown_connections 2024-09-19T09:19:01.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.003+0000 7fe5069c2640 1 -- 172.21.15.120:0/3785179929 wait complete. 2024-09-19T09:19:01.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.003+0000 7fe5069c2640 1 Processor -- start 2024-09-19T09:19:01.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.003+0000 7fe5069c2640 1 -- start start 2024-09-19T09:19:01.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.004+0000 7fe5069c2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe500101dd0 0x7fe5000744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:01.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.004+0000 7fe5069c2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe500102780 0x7fe500074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:01.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.004+0000 7fe5069c2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe500106f00 0x7fe500074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:01.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.004+0000 7fe5069c2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe500112000 con 0x7fe500101dd0 2024-09-19T09:19:01.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.004+0000 7fe5069c2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe500111e80 con 0x7fe500102780 2024-09-19T09:19:01.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.004+0000 7fe5069c2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe500112180 con 0x7fe500106f00 2024-09-19T09:19:01.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.004+0000 7fe4fffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe500101dd0 0x7fe5000744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:01.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.004+0000 7fe4fffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe500101dd0 0x7fe5000744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37784/0 (socket says 172.21.15.120:37784) 2024-09-19T09:19:01.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.004+0000 7fe4fffff640 1 -- 172.21.15.120:0/1184126853 learned_addr learned my addr 172.21.15.120:0/1184126853 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:01.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.004+0000 7fe4ff7fe640 1 --2- 172.21.15.120:0/1184126853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe500102780 0x7fe500074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:01.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.004+0000 7fe504f5f640 1 --2- 172.21.15.120:0/1184126853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe500106f00 0x7fe500074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:01.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.004+0000 7fe4fffff640 1 -- 172.21.15.120:0/1184126853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe500106f00 msgr2=0x7fe500074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:01.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.004+0000 7fe4fffff640 1 --2- 172.21.15.120:0/1184126853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe500106f00 0x7fe500074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.004+0000 7fe4fffff640 1 -- 172.21.15.120:0/1184126853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe500102780 msgr2=0x7fe500074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:01.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.004+0000 7fe4fffff640 1 --2- 172.21.15.120:0/1184126853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe500102780 0x7fe500074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.004+0000 7fe4fffff640 1 -- 172.21.15.120:0/1184126853 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5001a6b30 con 0x7fe500101dd0 2024-09-19T09:19:01.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.004+0000 7fe4fffff640 1 --2- 172.21.15.120:0/1184126853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe500101dd0 0x7fe5000744f0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000b4a0 tx=0x7fe4f000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:01.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.004+0000 7fe4fd7fa640 1 -- 172.21.15.120:0/1184126853 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0014070 con 0x7fe500101dd0 2024-09-19T09:19:01.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.004+0000 7fe5069c2640 1 -- 172.21.15.120:0/1184126853 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5001a6e20 con 0x7fe500101dd0 2024-09-19T09:19:01.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.004+0000 7fe4fd7fa640 1 -- 172.21.15.120:0/1184126853 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4f0010af0 con 0x7fe500101dd0 2024-09-19T09:19:01.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.005+0000 7fe4fd7fa640 1 -- 172.21.15.120:0/1184126853 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0017d10 con 0x7fe500101dd0 2024-09-19T09:19:01.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.005+0000 7fe5069c2640 1 -- 172.21.15.120:0/1184126853 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5001a7320 con 0x7fe500101dd0 2024-09-19T09:19:01.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.008+0000 7fe4fd7fa640 1 -- 172.21.15.120:0/1184126853 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe4f0017070 con 0x7fe500101dd0 2024-09-19T09:19:01.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.008+0000 7fe508d9b740 1 Processor -- start 2024-09-19T09:19:01.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.008+0000 7fe508d9b740 1 -- start start 2024-09-19T09:19:01.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.008+0000 7fe508d9b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555832734060 0x555832734440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:01.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.008+0000 7fe508d9b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555832734980 0x55583273ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:01.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.008+0000 7fe508d9b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583273f240 0x555832741630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:01.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.008+0000 7fe508d9b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555832699620 con 0x555832734980 2024-09-19T09:19:01.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.008+0000 7fe508d9b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555832640e50 con 0x555832734060 2024-09-19T09:19:01.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.008+0000 7fe508d9b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5558326955e0 con 0x55583273f240 2024-09-19T09:19:01.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.009+0000 7fe4fffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555832734980 0x55583273ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:01.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.009+0000 7fe4fffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555832734980 0x55583273ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37790/0 (socket says 172.21.15.120:37790) 2024-09-19T09:19:01.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.009+0000 7fe4fffff640 1 -- 172.21.15.120:0/1625059936 learned_addr learned my addr 172.21.15.120:0/1625059936 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:01.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.009+0000 7fe504f5f640 1 --2- 172.21.15.120:0/1625059936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555832734060 0x555832734440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:01.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.009+0000 7fe4fd7fa640 1 --2- 172.21.15.120:0/1184126853 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe4e80a32c0 0x7fe4e80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:01.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.009+0000 7fe504f5f640 1 --2- 172.21.15.120:0/1184126853 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe4e80a32c0 0x7fe4e80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:01.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.009+0000 7fe4ff7fe640 1 --2- 172.21.15.120:0/1625059936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583273f240 0x555832741630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:01.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.009+0000 7fe4fffff640 1 -- 172.21.15.120:0/1625059936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583273f240 msgr2=0x555832741630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:01.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.009+0000 7fe4fffff640 1 --2- 172.21.15.120:0/1625059936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583273f240 0x555832741630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.009+0000 7fe4fffff640 1 -- 172.21.15.120:0/1625059936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555832734060 msgr2=0x555832734440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:01.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.009+0000 7fe4fffff640 1 --2- 172.21.15.120:0/1625059936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555832734060 0x555832734440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.009+0000 7fe4fd7fa640 1 -- 172.21.15.120:0/1184126853 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe4f00cec60 con 0x7fe500101dd0 2024-09-19T09:19:01.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.009+0000 7fe4fffff640 1 -- 172.21.15.120:0/1625059936 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555832631d40 con 0x555832734980 2024-09-19T09:19:01.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.009+0000 7fe504f5f640 1 --2- 172.21.15.120:0/1625059936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555832734060 0x555832734440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:01.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.010+0000 7fe4fffff640 1 --2- 172.21.15.120:0/1625059936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555832734980 0x55583273ed00 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0097c50 tx=0x7fe4f00515f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e23ff0f61ee729d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:01.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.010+0000 7fe504f5f640 1 --2- 172.21.15.120:0/1184126853 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe4e80a32c0 0x7fe4e80a5780 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8009fd0 tx=0x7fe4f8004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:01.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.010+0000 7fe4e67fc640 1 -- 172.21.15.120:0/1625059936 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f009abe0 con 0x555832734980 2024-09-19T09:19:01.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.010+0000 7fe4e67fc640 1 -- 172.21.15.120:0/1625059936 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4f009b980 con 0x555832734980 2024-09-19T09:19:01.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.010+0000 7fe4e67fc640 1 -- 172.21.15.120:0/1625059936 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0017360 con 0x555832734980 2024-09-19T09:19:01.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.010+0000 7fe508d9b740 1 -- 172.21.15.120:0/1625059936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555832734980 msgr2=0x55583273ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:01.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.010+0000 7fe508d9b740 1 --2- 172.21.15.120:0/1625059936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555832734980 0x55583273ed00 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0097c50 tx=0x7fe4f00515f0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.011+0000 7fe508d9b740 1 -- 172.21.15.120:0/1625059936 shutdown_connections 2024-09-19T09:19:01.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.011+0000 7fe508d9b740 1 --2- 172.21.15.120:0/1625059936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583273f240 0x555832741630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.011+0000 7fe508d9b740 1 --2- 172.21.15.120:0/1625059936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555832734980 0x55583273ed00 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.011+0000 7fe508d9b740 1 --2- 172.21.15.120:0/1625059936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555832734060 0x555832734440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.011+0000 7fe508d9b740 1 -- 172.21.15.120:0/1625059936 >> 172.21.15.120:0/1625059936 conn(0x5558327305f0 msgr2=0x5558327309d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:01.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.011+0000 7fe508d9b740 1 -- 172.21.15.120:0/1625059936 shutdown_connections 2024-09-19T09:19:01.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.011+0000 7fe508d9b740 1 -- 172.21.15.120:0/1625059936 wait complete. 2024-09-19T09:19:01.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.011+0000 7fe508d9b740 1 -- start start 2024-09-19T09:19:01.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.011+0000 7fe508d9b740 1 Processor -- start 2024-09-19T09:19:01.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.012+0000 7fe508d9b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555832734060 0x55583273c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:01.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.012+0000 7fe508d9b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555832734980 0x55583273c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:01.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.012+0000 7fe508d9b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55583273f240 0x55583273cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:01.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.012+0000 7fe508d9b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555832741e10 con 0x55583273f240 2024-09-19T09:19:01.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.012+0000 7fe508d9b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555832621f10 con 0x555832734980 2024-09-19T09:19:01.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.012+0000 7fe508d9b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555832741f90 con 0x555832734060 2024-09-19T09:19:01.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.012+0000 7fe504f5f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55583273f240 0x55583273cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:01.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.012+0000 7fe504f5f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55583273f240 0x55583273cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37792/0 (socket says 172.21.15.120:37792) 2024-09-19T09:19:01.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.012+0000 7fe504f5f640 1 -- 172.21.15.120:0/4185543490 learned_addr learned my addr 172.21.15.120:0/4185543490 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:01.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.012+0000 7fe4fffff640 1 --2- 172.21.15.120:0/4185543490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555832734060 0x55583273c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:01.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.012+0000 7fe4ff7fe640 1 --2- 172.21.15.120:0/4185543490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555832734980 0x55583273c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:01.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.012+0000 7fe508d9b740 10 client.0 _ll_register_callbacks cb 0x555832645db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:01.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.012+0000 7fe504f5f640 1 -- 172.21.15.120:0/4185543490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555832734060 msgr2=0x55583273c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:01.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.012+0000 7fe504f5f640 1 --2- 172.21.15.120:0/4185543490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555832734060 0x55583273c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.012+0000 7fe504f5f640 1 -- 172.21.15.120:0/4185543490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555832734980 msgr2=0x55583273c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:01.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.013+0000 7fe504f5f640 1 --2- 172.21.15.120:0/4185543490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555832734980 0x55583273c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.013+0000 7fe504f5f640 1 -- 172.21.15.120:0/4185543490 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558323a8580 con 0x55583273f240 2024-09-19T09:19:01.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.013+0000 7fe508d9b740 1 -- 172.21.15.120:0/4185543490 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4f009b980 con 0x55583273f240 2024-09-19T09:19:01.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.013+0000 7fe504f5f640 1 --2- 172.21.15.120:0/4185543490 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55583273f240 0x55583273cc20 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fe4f800f040 tx=0x7fe4f8004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167d1142fffc9f16 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:01.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.013+0000 7fe4df7fe640 10 client.20931 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:19:01.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.013+0000 7fe4df7fe640 1 -- 172.21.15.120:0/4185543490 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f8020070 con 0x55583273f240 2024-09-19T09:19:01.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.013+0000 7fe4df7fe640 1 -- 172.21.15.120:0/4185543490 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe4f801c400 con 0x55583273f240 2024-09-19T09:19:01.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.013+0000 7fe4df7fe640 1 -- 172.21.15.120:0/4185543490 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe4f8015050 con 0x55583273f240 2024-09-19T09:19:01.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.013+0000 7fe508d9b740 10 client.20931 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:01.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.013+0000 7fe508d9b740 1 -- 172.21.15.120:0/4185543490 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({fsmap=39}) -- 0x555832472960 con 0x55583273f240 2024-09-19T09:19:01.018 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:19:01.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.014+0000 7fe4df7fe640 1 -- 172.21.15.120:0/4185543490 <== mon.0 v2:172.21.15.120:3300/0 4 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fe4f801c640 con 0x55583273f240 2024-09-19T09:19:01.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.014+0000 7fe508d9b740 10 client.20931 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:01.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.014+0000 7fe508d9b740 10 client.20931 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:19:01.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.014+0000 7fe508d9b740 20 client.20931 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:01.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.014+0000 7fe508d9b740 1 --2- 172.21.15.120:0/4185543490 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55583274ece0 0x5558327510f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:01.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.014+0000 7fe508d9b740 4 client.20931 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:19:01.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.014+0000 7fe508d9b740 1 -- 172.21.15.120:0/4185543490 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558326a6b90 con 0x55583274ece0 2024-09-19T09:19:01.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.014+0000 7fe4fffff640 1 --2- 172.21.15.120:0/4185543490 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55583274ece0 0x5558327510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:01.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.014+0000 7fe4fffff640 1 --2- 172.21.15.120:0/4185543490 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55583274ece0 0x5558327510f0 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51114f5a4a9d996a server_cookie=b3e0285933743c12 in_seq=0 out_seq=0 2024-09-19T09:19:01.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.014+0000 7fe4df7fe640 10 client.20931 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:01.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.015+0000 7fe4df7fe640 1 -- 172.21.15.120:0/4185543490 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5558326a6b90 con 0x55583274ece0 2024-09-19T09:19:01.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.015+0000 7fe4df7fe640 10 client.20931 handle_command_reply: tid=0 2024-09-19T09:19:01.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.015+0000 7fe508d9b740 1 client.20931 shutdown 2024-09-19T09:19:01.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.015+0000 7fe508d9b740 1 -- 172.21.15.120:0/4185543490 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55583273f240 msgr2=0x55583273cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:01.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.015+0000 7fe508d9b740 1 --2- 172.21.15.120:0/4185543490 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55583273f240 0x55583273cc20 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fe4f800f040 tx=0x7fe4f8004b50 comp rx=0 tx=0).stop 2024-09-19T09:19:01.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.015+0000 7fe508d9b740 1 -- 172.21.15.120:0/4185543490 shutdown_connections 2024-09-19T09:19:01.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.015+0000 7fe508d9b740 1 --2- 172.21.15.120:0/4185543490 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55583274ece0 0x5558327510f0 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.015+0000 7fe508d9b740 1 --2- 172.21.15.120:0/4185543490 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55583273f240 0x55583273cc20 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.015+0000 7fe508d9b740 1 --2- 172.21.15.120:0/4185543490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555832734980 0x55583273c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.015+0000 7fe508d9b740 1 --2- 172.21.15.120:0/4185543490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555832734060 0x55583273c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.015+0000 7fe508d9b740 1 -- 172.21.15.120:0/4185543490 >> 172.21.15.120:0/4185543490 conn(0x5558327305f0 msgr2=0x555832736320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:01.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.015+0000 7fe4df7fe640 0 client.20931 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:01.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.015+0000 7fe508d9b740 1 -- 172.21.15.120:0/4185543490 shutdown_connections 2024-09-19T09:19:01.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.015+0000 7fe508d9b740 1 -- 172.21.15.120:0/4185543490 wait complete. 2024-09-19T09:19:01.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.016+0000 7fe508d9b740 20 client.20931 trim_cache size 0 max 16384 2024-09-19T09:19:01.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.038+0000 7fe508d9b740 1 Processor -- start 2024-09-19T09:19:01.040 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:00 smithi120 ceph-mon[31150]: pgmap v829: 97 pgs: 97 active+clean; 273 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 222 KiB/s wr, 23 op/s 2024-09-19T09:19:01.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.038+0000 7fe508d9b740 1 -- start start 2024-09-19T09:19:01.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.039+0000 7fe508d9b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555832734060 0x5558327376b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:01.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.039+0000 7fe508d9b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555832734980 0x555832756a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:01.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.039+0000 7fe508d9b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583273f240 0x555832756f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:01.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.039+0000 7fe508d9b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe5001afa30 con 0x555832734060 2024-09-19T09:19:01.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.039+0000 7fe508d9b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe5001b1570 con 0x555832734980 2024-09-19T09:19:01.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.039+0000 7fe508d9b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5001b0100 con 0x55583273f240 2024-09-19T09:19:01.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.039+0000 7fe4fffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555832734060 0x5558327376b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:01.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.039+0000 7fe4fffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555832734060 0x5558327376b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37794/0 (socket says 172.21.15.120:37794) 2024-09-19T09:19:01.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.039+0000 7fe4fffff640 1 -- 172.21.15.120:0/1809432110 learned_addr learned my addr 172.21.15.120:0/1809432110 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:01.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.039+0000 7fe4ff7fe640 1 --2- 172.21.15.120:0/1809432110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555832734980 0x555832756a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:01.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.039+0000 7fe504f5f640 1 --2- 172.21.15.120:0/1809432110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583273f240 0x555832756f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:01.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.039+0000 7fe4fffff640 1 -- 172.21.15.120:0/1809432110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583273f240 msgr2=0x555832756f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:01.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.039+0000 7fe4fffff640 1 --2- 172.21.15.120:0/1809432110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583273f240 0x555832756f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.039+0000 7fe4fffff640 1 -- 172.21.15.120:0/1809432110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555832734980 msgr2=0x555832756a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:01.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.039+0000 7fe4fffff640 1 --2- 172.21.15.120:0/1809432110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555832734980 0x555832756a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.039+0000 7fe4fffff640 1 -- 172.21.15.120:0/1809432110 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555832730420 con 0x555832734060 2024-09-19T09:19:01.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.039+0000 7fe4fffff640 1 --2- 172.21.15.120:0/1809432110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555832734060 0x5558327376b0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fe4f009af80 tx=0x7fe4f0009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa498175f6cfd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:01.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.040+0000 7fe4e5ffb640 1 -- 172.21.15.120:0/1809432110 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f001e040 con 0x555832734060 2024-09-19T09:19:01.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.040+0000 7fe4e5ffb640 1 -- 172.21.15.120:0/1809432110 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4f00a3ef0 con 0x555832734060 2024-09-19T09:19:01.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.040+0000 7fe4e5ffb640 1 -- 172.21.15.120:0/1809432110 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0017360 con 0x555832734060 2024-09-19T09:19:01.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.041+0000 7fe508d9b740 1 -- 172.21.15.120:0/1809432110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555832734060 msgr2=0x5558327376b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:01.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.041+0000 7fe508d9b740 1 --2- 172.21.15.120:0/1809432110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555832734060 0x5558327376b0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fe4f009af80 tx=0x7fe4f0009390 comp rx=0 tx=0).stop 2024-09-19T09:19:01.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.041+0000 7fe508d9b740 1 -- 172.21.15.120:0/1809432110 shutdown_connections 2024-09-19T09:19:01.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.041+0000 7fe508d9b740 1 --2- 172.21.15.120:0/1809432110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583273f240 0x555832756f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.041+0000 7fe508d9b740 1 --2- 172.21.15.120:0/1809432110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555832734980 0x555832756a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.041+0000 7fe508d9b740 1 --2- 172.21.15.120:0/1809432110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555832734060 0x5558327376b0 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.041+0000 7fe508d9b740 1 -- 172.21.15.120:0/1809432110 >> 172.21.15.120:0/1809432110 conn(0x5558327305f0 msgr2=0x55583273e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:01.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.041+0000 7fe508d9b740 1 -- 172.21.15.120:0/1809432110 shutdown_connections 2024-09-19T09:19:01.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.041+0000 7fe508d9b740 1 -- 172.21.15.120:0/1809432110 wait complete. 2024-09-19T09:19:01.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.041+0000 7fe508d9b740 1 -- start start 2024-09-19T09:19:01.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.042+0000 7fe508d9b740 1 Processor -- start 2024-09-19T09:19:01.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.042+0000 7fe508d9b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555832734060 0x555832743370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:01.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.042+0000 7fe508d9b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555832734980 0x5558327438b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:01.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.042+0000 7fe508d9b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55583273f240 0x5558327b89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:01.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.042+0000 7fe508d9b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe5001b0ea0 con 0x555832734980 2024-09-19T09:19:01.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.042+0000 7fe4ff7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555832734980 0x5558327438b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:01.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.042+0000 7fe4ff7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555832734980 0x5558327438b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37810/0 (socket says 172.21.15.120:37810) 2024-09-19T09:19:01.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.042+0000 7fe4ff7fe640 1 -- 172.21.15.120:0/284560853 learned_addr learned my addr 172.21.15.120:0/284560853 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:01.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.042+0000 7fe508d9b740 1 -- 172.21.15.120:0/284560853 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe5001b07d0 con 0x55583273f240 2024-09-19T09:19:01.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.042+0000 7fe508d9b740 1 -- 172.21.15.120:0/284560853 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5001ad9f0 con 0x555832734060 2024-09-19T09:19:01.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.042+0000 7fe4fffff640 1 --2- 172.21.15.120:0/284560853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555832734060 0x555832743370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:01.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.042+0000 7fe504f5f640 1 --2- 172.21.15.120:0/284560853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55583273f240 0x5558327b89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:01.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.042+0000 7fe508d9b740 10 client.0 _ll_register_callbacks cb 0x7fe4f8007a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:01.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.042+0000 7fe4ff7fe640 1 -- 172.21.15.120:0/284560853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555832734060 msgr2=0x555832743370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:01.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.042+0000 7fe4ff7fe640 1 --2- 172.21.15.120:0/284560853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555832734060 0x555832743370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.043+0000 7fe4ff7fe640 1 -- 172.21.15.120:0/284560853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55583273f240 msgr2=0x5558327b89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:01.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.043+0000 7fe4ff7fe640 1 --2- 172.21.15.120:0/284560853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55583273f240 0x5558327b89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.043+0000 7fe4ff7fe640 1 -- 172.21.15.120:0/284560853 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558327b8f10 con 0x555832734980 2024-09-19T09:19:01.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.043+0000 7fe4ff7fe640 1 --2- 172.21.15.120:0/284560853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555832734980 0x5558327438b0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fe4f402a930 tx=0x7fe4f4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415c7391b3de2633 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:01.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.043+0000 7fe4df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:19:01.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.043+0000 7fe4df7fe640 1 -- 172.21.15.120:0/284560853 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4035040 con 0x555832734980 2024-09-19T09:19:01.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.043+0000 7fe4df7fe640 1 -- 172.21.15.120:0/284560853 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe4f4005020 con 0x555832734980 2024-09-19T09:19:01.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.043+0000 7fe508d9b740 1 -- 172.21.15.120:0/284560853 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4f00a3ef0 con 0x555832734980 2024-09-19T09:19:01.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.043+0000 7fe4df7fe640 1 -- 172.21.15.120:0/284560853 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe4f4005220 con 0x555832734980 2024-09-19T09:19:01.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.043+0000 7fe508d9b740 10 client.20937 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:01.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.043+0000 7fe508d9b740 1 -- 172.21.15.120:0/284560853 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5558327ba140 con 0x555832734980 2024-09-19T09:19:01.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.044+0000 7fe4df7fe640 1 -- 172.21.15.120:0/284560853 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4f4004c80 con 0x555832734980 2024-09-19T09:19:01.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.045+0000 7fe4df7fe640 1 -- 172.21.15.120:0/284560853 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fe4f4003790 con 0x555832734980 2024-09-19T09:19:01.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.045+0000 7fe508d9b740 10 client.20937 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:01.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.045+0000 7fe508d9b740 10 client.20937 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:19:01.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.045+0000 7fe508d9b740 20 client.20937 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:01.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.045+0000 7fe508d9b740 1 --2- 172.21.15.120:0/284560853 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5558327ba940 0x5558327bcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:01.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.045+0000 7fe508d9b740 4 client.20937 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "792ee6f9", "release": true, "await": true}] 2024-09-19T09:19:01.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.045+0000 7fe508d9b740 1 -- 172.21.15.120:0/284560853 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "set_id": "792ee6f9", "release": true, "await": true}) -- 0x7fe4f00a3ef0 con 0x5558327ba940 2024-09-19T09:19:01.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.045+0000 7fe4fffff640 1 --2- 172.21.15.120:0/284560853 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5558327ba940 0x5558327bcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:01.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.045+0000 7fe4fffff640 1 --2- 172.21.15.120:0/284560853 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5558327ba940 0x5558327bcd30 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f203af9a57f09c8b server_cookie=46e1c4e60505ebca in_seq=0 out_seq=0 2024-09-19T09:19:01.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.045+0000 7fe4df7fe640 10 client.20937 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:01.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.111+0000 7fe4df7fe640 1 -- 172.21.15.120:0/284560853 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fe4f00a3ef0 con 0x5558327ba940 2024-09-19T09:19:01.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.111+0000 7fe4df7fe640 10 client.20937 handle_command_reply: tid=0 2024-09-19T09:19:01.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.111+0000 7fe508d9b740 1 client.20937 shutdown 2024-09-19T09:19:01.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.111+0000 7fe508d9b740 1 -- 172.21.15.120:0/284560853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555832734980 msgr2=0x5558327438b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:01.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.111+0000 7fe508d9b740 1 --2- 172.21.15.120:0/284560853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555832734980 0x5558327438b0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fe4f402a930 tx=0x7fe4f4004270 comp rx=0 tx=0).stop 2024-09-19T09:19:01.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.111+0000 7fe508d9b740 1 -- 172.21.15.120:0/284560853 shutdown_connections 2024-09-19T09:19:01.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.111+0000 7fe508d9b740 1 --2- 172.21.15.120:0/284560853 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5558327ba940 0x5558327bcd30 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.111+0000 7fe508d9b740 1 --2- 172.21.15.120:0/284560853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55583273f240 0x5558327b89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.111+0000 7fe508d9b740 1 --2- 172.21.15.120:0/284560853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555832734980 0x5558327438b0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.111+0000 7fe4df7fe640 0 client.20937 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:01.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.111+0000 7fe508d9b740 1 --2- 172.21.15.120:0/284560853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555832734060 0x555832743370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.111+0000 7fe508d9b740 1 -- 172.21.15.120:0/284560853 >> 172.21.15.120:0/284560853 conn(0x5558327305f0 msgr2=0x55583273c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:01.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.111+0000 7fe508d9b740 1 -- 172.21.15.120:0/284560853 shutdown_connections 2024-09-19T09:19:01.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.111+0000 7fe508d9b740 1 -- 172.21.15.120:0/284560853 wait complete. 2024-09-19T09:19:01.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.111+0000 7fe508d9b740 20 client.20937 trim_cache size 0 max 16384 2024-09-19T09:19:01.113 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:19:01.114 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 39, 2024-09-19T09:19:01.114 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:19:01.114 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 56, 2024-09-19T09:19:01.114 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:19:01.114 INFO:teuthology.orchestra.run.smithi120.stdout: "792ee6f9": { 2024-09-19T09:19:01.114 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 56, 2024-09-19T09:19:01.114 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:19:01.114 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:19:01.114 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:19:01.114 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:19:01.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:19:01.114 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:19:01.114 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 149.4, 2024-09-19T09:19:01.115 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:19:01.115 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:19:01.115 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:19:01.115 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:19:01.115 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:19:01.115 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:19:01.115 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:19:01.115 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:19:01.115 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:19:01.115 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:19:01.115 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:19:01.115 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:19:01.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.112+0000 7fe5069c2640 1 -- 172.21.15.120:0/1184126853 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe4e80a32c0 msgr2=0x7fe4e80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:01.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.112+0000 7fe5069c2640 1 --2- 172.21.15.120:0/1184126853 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe4e80a32c0 0x7fe4e80a5780 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8009fd0 tx=0x7fe4f8004a40 comp rx=0 tx=0).stop 2024-09-19T09:19:01.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.112+0000 7fe5069c2640 1 -- 172.21.15.120:0/1184126853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe500101dd0 msgr2=0x7fe5000744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:01.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.112+0000 7fe5069c2640 1 --2- 172.21.15.120:0/1184126853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe500101dd0 0x7fe5000744f0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000b4a0 tx=0x7fe4f000b960 comp rx=0 tx=0).stop 2024-09-19T09:19:01.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.112+0000 7fe5069c2640 1 -- 172.21.15.120:0/1184126853 shutdown_connections 2024-09-19T09:19:01.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.112+0000 7fe5069c2640 1 --2- 172.21.15.120:0/1184126853 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe4e80a32c0 0x7fe4e80a5780 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.112+0000 7fe5069c2640 1 --2- 172.21.15.120:0/1184126853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe500106f00 0x7fe500074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.112+0000 7fe5069c2640 1 --2- 172.21.15.120:0/1184126853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe500102780 0x7fe500074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.112+0000 7fe5069c2640 1 --2- 172.21.15.120:0/1184126853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe500101dd0 0x7fe5000744f0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:01.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.112+0000 7fe5069c2640 1 -- 172.21.15.120:0/1184126853 >> 172.21.15.120:0/1184126853 conn(0x7fe5000fd8b0 msgr2=0x7fe5000fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:01.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.112+0000 7fe5069c2640 1 -- 172.21.15.120:0/1184126853 shutdown_connections 2024-09-19T09:19:01.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:01.112+0000 7fe5069c2640 1 -- 172.21.15.120:0/1184126853 wait complete. 2024-09-19T09:19:01.361 INFO:tasks.workunit.client.0.smithi120.stdout: 10 58039 1.99 MB/sec execute 56 sec latency 30442.656 ms 2024-09-19T09:19:01.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:00 smithi196 ceph-mon[35150]: pgmap v829: 97 pgs: 97 active+clean; 273 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 222 KiB/s wr, 23 op/s 2024-09-19T09:19:01.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:00 smithi153 ceph-mon[35071]: pgmap v829: 97 pgs: 97 active+clean; 273 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 222 KiB/s wr, 23 op/s 2024-09-19T09:19:02.120 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '792ee6f9', total seconds elapsed: 34.4 2024-09-19T09:19:02.361 INFO:tasks.workunit.client.0.smithi120.stdout: 10 58800 2.69 MB/sec execute 57 sec latency 40.671 ms 2024-09-19T09:19:02.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.364+0000 7f9afa04d640 1 -- 172.21.15.120:0/3081559510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9af4101440 msgr2=0x7f9af410df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:02.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.364+0000 7f9afa04d640 1 --2- 172.21.15.120:0/3081559510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9af4101440 0x7f9af410df30 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8009900 tx=0x7f9ae802a410 comp rx=0 tx=0).stop 2024-09-19T09:19:02.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.364+0000 7f9afa04d640 1 -- 172.21.15.120:0/3081559510 shutdown_connections 2024-09-19T09:19:02.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.364+0000 7f9afa04d640 1 --2- 172.21.15.120:0/3081559510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af410e500 0x7f9af4110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.364+0000 7f9afa04d640 1 --2- 172.21.15.120:0/3081559510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9af4101440 0x7f9af410df30 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.364+0000 7f9afa04d640 1 --2- 172.21.15.120:0/3081559510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9af4100b20 0x7f9af4100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.364+0000 7f9afa04d640 1 -- 172.21.15.120:0/3081559510 >> 172.21.15.120:0/3081559510 conn(0x7f9af40fc7d0 msgr2=0x7f9af40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:02.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.364+0000 7f9afa04d640 1 -- 172.21.15.120:0/3081559510 shutdown_connections 2024-09-19T09:19:02.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.364+0000 7f9afa04d640 1 -- 172.21.15.120:0/3081559510 wait complete. 2024-09-19T09:19:02.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.364+0000 7f9afa04d640 1 Processor -- start 2024-09-19T09:19:02.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.365+0000 7f9afa04d640 1 -- start start 2024-09-19T09:19:02.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.365+0000 7f9afa04d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9af4100b20 0x7f9af41a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:02.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.365+0000 7f9afa04d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9af4101440 0x7f9af41a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:02.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.365+0000 7f9afa04d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af410e500 0x7f9af419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.365+0000 7f9afa04d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9af4114250 con 0x7f9af4101440 2024-09-19T09:19:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.365+0000 7f9afa04d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41140d0 con 0x7f9af410e500 2024-09-19T09:19:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.365+0000 7f9afa04d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41143d0 con 0x7f9af4100b20 2024-09-19T09:19:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.365+0000 7f9af2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9af4101440 0x7f9af41a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.365+0000 7f9af2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9af4101440 0x7f9af41a0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48358/0 (socket says 172.21.15.120:48358) 2024-09-19T09:19:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.365+0000 7f9af2ffd640 1 -- 172.21.15.120:0/2421248014 learned_addr learned my addr 172.21.15.120:0/2421248014 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.365+0000 7f9af37fe640 1 --2- 172.21.15.120:0/2421248014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9af4100b20 0x7f9af41a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.365+0000 7f9af3fff640 1 --2- 172.21.15.120:0/2421248014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af410e500 0x7f9af419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.365+0000 7f9af2ffd640 1 -- 172.21.15.120:0/2421248014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9af4100b20 msgr2=0x7f9af41a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.365+0000 7f9af2ffd640 1 --2- 172.21.15.120:0/2421248014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9af4100b20 0x7f9af41a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.365+0000 7f9af2ffd640 1 -- 172.21.15.120:0/2421248014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af410e500 msgr2=0x7f9af419a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.365+0000 7f9af2ffd640 1 --2- 172.21.15.120:0/2421248014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af410e500 0x7f9af419a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.365+0000 7f9af2ffd640 1 -- 172.21.15.120:0/2421248014 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af419acf0 con 0x7f9af4101440 2024-09-19T09:19:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.365+0000 7f9af3fff640 1 --2- 172.21.15.120:0/2421248014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af410e500 0x7f9af419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:02.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.366+0000 7f9af2ffd640 1 --2- 172.21.15.120:0/2421248014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9af4101440 0x7f9af41a0940 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8005940 tx=0x7f9ae80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:02.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.366+0000 7f9af0ff9640 1 -- 172.21.15.120:0/2421248014 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae803f030 con 0x7f9af4101440 2024-09-19T09:19:02.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.366+0000 7f9af0ff9640 1 -- 172.21.15.120:0/2421248014 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ae80044c0 con 0x7f9af4101440 2024-09-19T09:19:02.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.366+0000 7f9af0ff9640 1 -- 172.21.15.120:0/2421248014 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8035070 con 0x7f9af4101440 2024-09-19T09:19:02.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.366+0000 7f9afa04d640 1 -- 172.21.15.120:0/2421248014 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af419af80 con 0x7f9af4101440 2024-09-19T09:19:02.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.366+0000 7f9afa04d640 1 -- 172.21.15.120:0/2421248014 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af41a72d0 con 0x7f9af4101440 2024-09-19T09:19:02.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.368+0000 7f9af0ff9640 1 -- 172.21.15.120:0/2421248014 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f9ae8042030 con 0x7f9af4101440 2024-09-19T09:19:02.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.369+0000 7f9afc426740 1 Processor -- start 2024-09-19T09:19:02.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.369+0000 7f9afc426740 1 -- start start 2024-09-19T09:19:02.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.369+0000 7f9afc426740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a1b2bcb40 0x564a1b2bb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:02.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.369+0000 7f9afc426740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1b2bb5d0 0x564a1b2bba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:02.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.369+0000 7f9afc426740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a1b2c6370 0x564a1b2c8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:02.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.369+0000 7f9afc426740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564a1b2212f0 con 0x564a1b2c6370 2024-09-19T09:19:02.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.369+0000 7f9afc426740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564a1b1c8cf0 con 0x564a1b2bcb40 2024-09-19T09:19:02.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.369+0000 7f9afc426740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564a1b21d2b0 con 0x564a1b2bb5d0 2024-09-19T09:19:02.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.369+0000 7f9af0ff9640 1 --2- 172.21.15.120:0/2421248014 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9adc0a3240 0x7f9adc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:02.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.369+0000 7f9af37fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1b2bb5d0 0x564a1b2bba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:02.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.369+0000 7f9af0ff9640 1 -- 172.21.15.120:0/2421248014 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f9ae80efcb0 con 0x7f9af4101440 2024-09-19T09:19:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.369+0000 7f9af37fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1b2bb5d0 0x564a1b2bba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:52642/0 (socket says 172.21.15.120:52642) 2024-09-19T09:19:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.369+0000 7f9af37fe640 1 -- 172.21.15.120:0/2619776215 learned_addr learned my addr 172.21.15.120:0/2619776215 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.369+0000 7f9af2ffd640 1 --2- 172.21.15.120:0/2619776215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a1b2c6370 0x564a1b2c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.370+0000 7f9af3fff640 1 --2- 172.21.15.120:0/2619776215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a1b2bcb40 0x564a1b2bb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.370+0000 7f9af37fe640 1 -- 172.21.15.120:0/2619776215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a1b2bcb40 msgr2=0x564a1b2bb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.370+0000 7f9af37fe640 1 --2- 172.21.15.120:0/2619776215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a1b2bcb40 0x564a1b2bb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.370+0000 7f9af37fe640 1 -- 172.21.15.120:0/2619776215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a1b2c6370 msgr2=0x564a1b2c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.370+0000 7f9af37fe640 1 --2- 172.21.15.120:0/2619776215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a1b2c6370 0x564a1b2c8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.370+0000 7f9af37fe640 1 -- 172.21.15.120:0/2619776215 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a1b1ad3a0 con 0x564a1b2bb5d0 2024-09-19T09:19:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.370+0000 7f9af2ffd640 1 --2- 172.21.15.120:0/2619776215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a1b2c6370 0x564a1b2c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.370+0000 7f9af3fff640 1 --2- 172.21.15.120:0/2421248014 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9adc0a3240 0x7f9adc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.370+0000 7f9af37fe640 1 --2- 172.21.15.120:0/2619776215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1b2bb5d0 0x564a1b2bba10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4009cf0 tx=0x7f9ae4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3102f2c22d1427a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.370+0000 7f9ad9ffb640 1 -- 172.21.15.120:0/2619776215 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4004230 con 0x564a1b2bb5d0 2024-09-19T09:19:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.370+0000 7f9ad9ffb640 1 -- 172.21.15.120:0/2619776215 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ae40043d0 con 0x564a1b2bb5d0 2024-09-19T09:19:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.370+0000 7f9ad9ffb640 1 -- 172.21.15.120:0/2619776215 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4019960 con 0x564a1b2bb5d0 2024-09-19T09:19:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.372+0000 7f9afc426740 1 -- 172.21.15.120:0/2619776215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1b2bb5d0 msgr2=0x564a1b2bba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.372+0000 7f9afc426740 1 --2- 172.21.15.120:0/2619776215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1b2bb5d0 0x564a1b2bba10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4009cf0 tx=0x7f9ae4011810 comp rx=0 tx=0).stop 2024-09-19T09:19:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.372+0000 7f9af3fff640 1 --2- 172.21.15.120:0/2421248014 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9adc0a3240 0x7f9adc0a5700 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9aec0099e0 tx=0x7f9aec004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.372+0000 7f9afc426740 1 -- 172.21.15.120:0/2619776215 shutdown_connections 2024-09-19T09:19:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.372+0000 7f9afc426740 1 --2- 172.21.15.120:0/2619776215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a1b2c6370 0x564a1b2c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.372+0000 7f9afc426740 1 --2- 172.21.15.120:0/2619776215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1b2bb5d0 0x564a1b2bba10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.372+0000 7f9afc426740 1 --2- 172.21.15.120:0/2619776215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a1b2bcb40 0x564a1b2bb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.372+0000 7f9afc426740 1 -- 172.21.15.120:0/2619776215 >> 172.21.15.120:0/2619776215 conn(0x564a1b284310 msgr2=0x564a1b2b7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.372+0000 7f9afc426740 1 -- 172.21.15.120:0/2619776215 shutdown_connections 2024-09-19T09:19:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.372+0000 7f9afc426740 1 -- 172.21.15.120:0/2619776215 wait complete. 2024-09-19T09:19:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.372+0000 7f9afc426740 1 -- start start 2024-09-19T09:19:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.372+0000 7f9afc426740 1 Processor -- start 2024-09-19T09:19:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.373+0000 7f9afc426740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1b2bb5d0 0x564a1b2c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.373+0000 7f9afc426740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a1b2bcb40 0x564a1b2c3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.373+0000 7f9afc426740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a1b2c6370 0x564a1b2c35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.373+0000 7f9afc426740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564a1b2c9070 con 0x564a1b2c6370 2024-09-19T09:19:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.373+0000 7f9afc426740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564a1b1dc060 con 0x564a1b2bcb40 2024-09-19T09:19:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.373+0000 7f9afc426740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564a1b2c91f0 con 0x564a1b2bb5d0 2024-09-19T09:19:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.373+0000 7f9af37fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1b2bb5d0 0x564a1b2c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.373+0000 7f9af37fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1b2bb5d0 0x564a1b2c5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:52658/0 (socket says 172.21.15.120:52658) 2024-09-19T09:19:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.373+0000 7f9af37fe640 1 -- 172.21.15.120:0/1556329475 learned_addr learned my addr 172.21.15.120:0/1556329475 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.373+0000 7f9af3fff640 1 --2- 172.21.15.120:0/1556329475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a1b2c6370 0x564a1b2c35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.373+0000 7f9afc426740 10 client.0 _ll_register_callbacks cb 0x564a1af2f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.373+0000 7f9af2ffd640 1 --2- 172.21.15.120:0/1556329475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a1b2bcb40 0x564a1b2c3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.373+0000 7f9af37fe640 1 -- 172.21.15.120:0/1556329475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a1b2bcb40 msgr2=0x564a1b2c3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:02.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.373+0000 7f9af37fe640 1 --2- 172.21.15.120:0/1556329475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a1b2bcb40 0x564a1b2c3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.373+0000 7f9af37fe640 1 -- 172.21.15.120:0/1556329475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a1b2c6370 msgr2=0x564a1b2c35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:02.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.373+0000 7f9af37fe640 1 --2- 172.21.15.120:0/1556329475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a1b2c6370 0x564a1b2c35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.373+0000 7f9af37fe640 1 -- 172.21.15.120:0/1556329475 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a1af923c0 con 0x564a1b2bb5d0 2024-09-19T09:19:02.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.373+0000 7f9af3fff640 1 --2- 172.21.15.120:0/1556329475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a1b2c6370 0x564a1b2c35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:02.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.374+0000 7f9af37fe640 1 --2- 172.21.15.120:0/1556329475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1b2bb5d0 0x564a1b2c5d50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40195c0 tx=0x7f9ae4005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=472a66acc4cd42f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:02.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.374+0000 7f9ad2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:19:02.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.374+0000 7f9ad2ffd640 1 -- 172.21.15.120:0/1556329475 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4038070 con 0x564a1b2bb5d0 2024-09-19T09:19:02.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.374+0000 7f9ad2ffd640 1 -- 172.21.15.120:0/1556329475 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f9ae40035f0 con 0x564a1b2bb5d0 2024-09-19T09:19:02.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.374+0000 7f9afc426740 1 -- 172.21.15.120:0/1556329475 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ae40043d0 con 0x564a1b2bb5d0 2024-09-19T09:19:02.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.375+0000 7f9ad2ffd640 1 -- 172.21.15.120:0/1556329475 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9ae4044050 con 0x564a1b2bb5d0 2024-09-19T09:19:02.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.375+0000 7f9afc426740 10 client.29638 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:02.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.375+0000 7f9afc426740 1 -- 172.21.15.120:0/1556329475 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564a1b2d4ad0 con 0x564a1b2bb5d0 2024-09-19T09:19:02.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.375+0000 7f9ad2ffd640 1 -- 172.21.15.120:0/1556329475 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ae40043d0 con 0x564a1b2bb5d0 2024-09-19T09:19:02.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.375+0000 7f9ad2ffd640 1 -- 172.21.15.120:0/1556329475 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f9ae4004a60 con 0x564a1b2bb5d0 2024-09-19T09:19:02.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.375+0000 7f9afc426740 10 client.29638 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:02.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.375+0000 7f9afc426740 10 client.29638 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:19:02.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.375+0000 7f9afc426740 20 client.29638 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:02.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.375+0000 7f9afc426740 1 --2- 172.21.15.120:0/1556329475 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x564a1b2d5e80 0x564a1b2d8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:02.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.375+0000 7f9afc426740 4 client.29638 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:19:02.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.375+0000 7f9afc426740 1 -- 172.21.15.120:0/1556329475 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a1b22e860 con 0x564a1b2d5e80 2024-09-19T09:19:02.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.375+0000 7f9af2ffd640 1 --2- 172.21.15.120:0/1556329475 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x564a1b2d5e80 0x564a1b2d8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:02.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.376+0000 7f9af2ffd640 1 --2- 172.21.15.120:0/1556329475 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x564a1b2d5e80 0x564a1b2d8270 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a5e5d85012242e7c server_cookie=5902718efad2623c in_seq=0 out_seq=0 2024-09-19T09:19:02.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.376+0000 7f9ad2ffd640 10 client.29638 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:19:02.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.377+0000 7f9ad2ffd640 1 -- 172.21.15.120:0/1556329475 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a1b22e860 con 0x564a1b2d5e80 2024-09-19T09:19:02.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.377+0000 7f9ad2ffd640 10 client.29638 handle_command_reply: tid=0 2024-09-19T09:19:02.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.377+0000 7f9afc426740 1 client.29638 shutdown 2024-09-19T09:19:02.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.378+0000 7f9afc426740 1 -- 172.21.15.120:0/1556329475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1b2bb5d0 msgr2=0x564a1b2c5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:02.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.378+0000 7f9afc426740 1 --2- 172.21.15.120:0/1556329475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1b2bb5d0 0x564a1b2c5d50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40195c0 tx=0x7f9ae4005d30 comp rx=0 tx=0).stop 2024-09-19T09:19:02.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.378+0000 7f9afc426740 1 -- 172.21.15.120:0/1556329475 shutdown_connections 2024-09-19T09:19:02.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.378+0000 7f9afc426740 1 --2- 172.21.15.120:0/1556329475 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x564a1b2d5e80 0x564a1b2d8270 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.378+0000 7f9afc426740 1 --2- 172.21.15.120:0/1556329475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a1b2c6370 0x564a1b2c35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.378+0000 7f9afc426740 1 --2- 172.21.15.120:0/1556329475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a1b2bcb40 0x564a1b2c3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.378+0000 7f9afc426740 1 --2- 172.21.15.120:0/1556329475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1b2bb5d0 0x564a1b2c5d50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.378+0000 7f9afc426740 1 -- 172.21.15.120:0/1556329475 >> 172.21.15.120:0/1556329475 conn(0x564a1b284310 msgr2=0x564a1b2bd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:02.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.378+0000 7f9ad2ffd640 0 client.29638 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:19:02.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.378+0000 7f9afc426740 1 -- 172.21.15.120:0/1556329475 shutdown_connections 2024-09-19T09:19:02.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.378+0000 7f9afc426740 1 -- 172.21.15.120:0/1556329475 wait complete. 2024-09-19T09:19:02.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.378+0000 7f9afc426740 20 client.29638 trim_cache size 0 max 16384 2024-09-19T09:19:02.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.392+0000 7f9afc426740 1 Processor -- start 2024-09-19T09:19:02.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.392+0000 7f9afc426740 1 -- start start 2024-09-19T09:19:02.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.392+0000 7f9afc426740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a1b2bb5d0 0x564a1b2d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:02.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.392+0000 7f9afc426740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1b2bcb40 0x564a1b2c28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:02.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.392+0000 7f9afc426740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a1b2c6370 0x564a1b2c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:02.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.392+0000 7f9afc426740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41ab450 con 0x564a1b2bb5d0 2024-09-19T09:19:02.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.392+0000 7f9afc426740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41acf90 con 0x564a1b2c6370 2024-09-19T09:19:02.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.392+0000 7f9afc426740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41abb20 con 0x564a1b2bcb40 2024-09-19T09:19:02.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.392+0000 7f9af37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a1b2bb5d0 0x564a1b2d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:02.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.393+0000 7f9af37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a1b2bb5d0 0x564a1b2d2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48388/0 (socket says 172.21.15.120:48388) 2024-09-19T09:19:02.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.393+0000 7f9af37fe640 1 -- 172.21.15.120:0/1858367885 learned_addr learned my addr 172.21.15.120:0/1858367885 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:02.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.393+0000 7f9af3fff640 1 --2- 172.21.15.120:0/1858367885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a1b2c6370 0x564a1b2c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:02.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.393+0000 7f9af2ffd640 1 --2- 172.21.15.120:0/1858367885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1b2bcb40 0x564a1b2c28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:02.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.393+0000 7f9af37fe640 1 -- 172.21.15.120:0/1858367885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1b2bcb40 msgr2=0x564a1b2c28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:02.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.393+0000 7f9af37fe640 1 --2- 172.21.15.120:0/1858367885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1b2bcb40 0x564a1b2c28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.393+0000 7f9af37fe640 1 -- 172.21.15.120:0/1858367885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a1b2c6370 msgr2=0x564a1b2c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:02.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.393+0000 7f9af37fe640 1 --2- 172.21.15.120:0/1858367885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a1b2c6370 0x564a1b2c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.393+0000 7f9af37fe640 1 -- 172.21.15.120:0/1858367885 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a1b2b8960 con 0x564a1b2bb5d0 2024-09-19T09:19:02.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.393+0000 7f9af37fe640 1 --2- 172.21.15.120:0/1858367885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a1b2bb5d0 0x564a1b2d2310 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4009aa0 tx=0x7f9ae40097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823ce7665006669e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:02.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.393+0000 7f9ad97fa640 1 -- 172.21.15.120:0/1858367885 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4041020 con 0x564a1b2bb5d0 2024-09-19T09:19:02.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.393+0000 7f9ad97fa640 1 -- 172.21.15.120:0/1858367885 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ae4003e30 con 0x564a1b2bb5d0 2024-09-19T09:19:02.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.393+0000 7f9ad97fa640 1 -- 172.21.15.120:0/1858367885 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4038070 con 0x564a1b2bb5d0 2024-09-19T09:19:02.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.394+0000 7f9afc426740 1 -- 172.21.15.120:0/1858367885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a1b2bb5d0 msgr2=0x564a1b2d2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:02.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.394+0000 7f9afc426740 1 --2- 172.21.15.120:0/1858367885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a1b2bb5d0 0x564a1b2d2310 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4009aa0 tx=0x7f9ae40097a0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.394+0000 7f9afc426740 1 -- 172.21.15.120:0/1858367885 shutdown_connections 2024-09-19T09:19:02.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.394+0000 7f9afc426740 1 --2- 172.21.15.120:0/1858367885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a1b2c6370 0x564a1b2c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.394+0000 7f9afc426740 1 --2- 172.21.15.120:0/1858367885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1b2bcb40 0x564a1b2c28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.394+0000 7f9afc426740 1 --2- 172.21.15.120:0/1858367885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a1b2bb5d0 0x564a1b2d2310 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.394+0000 7f9afc426740 1 -- 172.21.15.120:0/1858367885 >> 172.21.15.120:0/1858367885 conn(0x564a1b284310 msgr2=0x564a1b2d7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:02.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.394+0000 7f9afc426740 1 -- 172.21.15.120:0/1858367885 shutdown_connections 2024-09-19T09:19:02.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.394+0000 7f9afc426740 1 -- 172.21.15.120:0/1858367885 wait complete. 2024-09-19T09:19:02.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.395+0000 7f9afc426740 1 -- start start 2024-09-19T09:19:02.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.395+0000 7f9afc426740 1 Processor -- start 2024-09-19T09:19:02.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.395+0000 7f9afc426740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a1b2bb5d0 0x564a1b2dc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:02.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.395+0000 7f9afc426740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a1b2bcb40 0x564a1b2dcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:02.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.395+0000 7f9afc426740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1b2c6370 0x564a1b2dd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:02.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.395+0000 7f9afc426740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41ac8c0 con 0x564a1b2bb5d0 2024-09-19T09:19:02.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.395+0000 7f9afc426740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41ac1f0 con 0x564a1b2bcb40 2024-09-19T09:19:02.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.395+0000 7f9afc426740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41a9410 con 0x564a1b2c6370 2024-09-19T09:19:02.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.395+0000 7f9afc426740 10 client.0 _ll_register_callbacks cb 0x7f9ae8009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:02.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.395+0000 7f9af2ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a1b2bcb40 0x564a1b2dcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:02.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.395+0000 7f9af37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a1b2bb5d0 0x564a1b2dc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:02.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.395+0000 7f9af2ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a1b2bcb40 0x564a1b2dcf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:43260/0 (socket says 172.21.15.120:43260) 2024-09-19T09:19:02.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.395+0000 7f9af2ffd640 1 -- 172.21.15.120:0/409458380 learned_addr learned my addr 172.21.15.120:0/409458380 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:02.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.395+0000 7f9af3fff640 1 --2- 172.21.15.120:0/409458380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1b2c6370 0x564a1b2dd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:02.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.396+0000 7f9af37fe640 1 -- 172.21.15.120:0/409458380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1b2c6370 msgr2=0x564a1b2dd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:02.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.396+0000 7f9af37fe640 1 --2- 172.21.15.120:0/409458380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1b2c6370 0x564a1b2dd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.396+0000 7f9af37fe640 1 -- 172.21.15.120:0/409458380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a1b2bcb40 msgr2=0x564a1b2dcf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:02.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.396+0000 7f9af37fe640 1 --2- 172.21.15.120:0/409458380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a1b2bcb40 0x564a1b2dcf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.396+0000 7f9af37fe640 1 -- 172.21.15.120:0/409458380 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a1b2cd500 con 0x564a1b2bb5d0 2024-09-19T09:19:02.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.396+0000 7f9af2ffd640 1 --2- 172.21.15.120:0/409458380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a1b2bcb40 0x564a1b2dcf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:02.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.396+0000 7f9af37fe640 1 --2- 172.21.15.120:0/409458380 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a1b2bb5d0 0x564a1b2dc9d0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4006010 tx=0x7f9ae40056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa1ed802157db28 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:02.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.396+0000 7f9ad2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:19:02.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.396+0000 7f9ad2ffd640 1 -- 172.21.15.120:0/409458380 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4038070 con 0x564a1b2bb5d0 2024-09-19T09:19:02.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.396+0000 7f9ad2ffd640 1 -- 172.21.15.120:0/409458380 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f9ae40035f0 con 0x564a1b2bb5d0 2024-09-19T09:19:02.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.396+0000 7f9afc426740 1 -- 172.21.15.120:0/409458380 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ae4003e30 con 0x564a1b2bb5d0 2024-09-19T09:19:02.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.396+0000 7f9ad2ffd640 1 -- 172.21.15.120:0/409458380 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9ae4019530 con 0x564a1b2bb5d0 2024-09-19T09:19:02.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.396+0000 7f9afc426740 10 client.20955 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:02.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.396+0000 7f9afc426740 1 -- 172.21.15.120:0/409458380 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564a1b2ce7a0 con 0x564a1b2bb5d0 2024-09-19T09:19:02.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.396+0000 7f9ad2ffd640 1 -- 172.21.15.120:0/409458380 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ae4003e30 con 0x564a1b2bb5d0 2024-09-19T09:19:02.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.397+0000 7f9ad2ffd640 1 -- 172.21.15.120:0/409458380 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f9ae4004430 con 0x564a1b2bb5d0 2024-09-19T09:19:02.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.397+0000 7f9afc426740 10 client.20955 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:02.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.397+0000 7f9afc426740 10 client.20955 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:19:02.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.397+0000 7f9afc426740 20 client.20955 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:02.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.397+0000 7f9afc426740 1 --2- 172.21.15.120:0/409458380 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x564a1b2cee70 0x564a1b3429c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:02.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.397+0000 7f9afc426740 4 client.20955 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:19:02.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.397+0000 7f9afc426740 1 -- 172.21.15.120:0/409458380 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a1b228d70 con 0x564a1b2cee70 2024-09-19T09:19:02.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.397+0000 7f9af2ffd640 1 --2- 172.21.15.120:0/409458380 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x564a1b2cee70 0x564a1b3429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:02.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.397+0000 7f9af2ffd640 1 --2- 172.21.15.120:0/409458380 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x564a1b2cee70 0x564a1b3429c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9e4ca6ea7b94ad34 server_cookie=a9483ec16701e344 in_seq=0 out_seq=0 2024-09-19T09:19:02.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.397+0000 7f9ad2ffd640 10 client.20955 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:19:02.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.397+0000 7f9ad2ffd640 1 -- 172.21.15.120:0/409458380 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a1b228d70 con 0x564a1b2cee70 2024-09-19T09:19:02.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.397+0000 7f9ad2ffd640 10 client.20955 handle_command_reply: tid=0 2024-09-19T09:19:02.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.398+0000 7f9afc426740 1 client.20955 shutdown 2024-09-19T09:19:02.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.398+0000 7f9afc426740 1 -- 172.21.15.120:0/409458380 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a1b2bb5d0 msgr2=0x564a1b2dc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:02.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.398+0000 7f9afc426740 1 --2- 172.21.15.120:0/409458380 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a1b2bb5d0 0x564a1b2dc9d0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4006010 tx=0x7f9ae40056b0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.398+0000 7f9afc426740 1 -- 172.21.15.120:0/409458380 shutdown_connections 2024-09-19T09:19:02.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.398+0000 7f9afc426740 1 --2- 172.21.15.120:0/409458380 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x564a1b2cee70 0x564a1b3429c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.398+0000 7f9ad2ffd640 0 client.20955 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:19:02.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.398+0000 7f9afc426740 1 --2- 172.21.15.120:0/409458380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564a1b2c6370 0x564a1b2dd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.398+0000 7f9afc426740 1 --2- 172.21.15.120:0/409458380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564a1b2bcb40 0x564a1b2dcf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.398+0000 7f9afc426740 1 --2- 172.21.15.120:0/409458380 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a1b2bb5d0 0x564a1b2dc9d0 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.398+0000 7f9afc426740 1 -- 172.21.15.120:0/409458380 >> 172.21.15.120:0/409458380 conn(0x564a1b284310 msgr2=0x564a1b2b6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:02.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.398+0000 7f9afc426740 1 -- 172.21.15.120:0/409458380 shutdown_connections 2024-09-19T09:19:02.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.398+0000 7f9afc426740 1 -- 172.21.15.120:0/409458380 wait complete. 2024-09-19T09:19:02.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.398+0000 7f9afc426740 20 client.20955 trim_cache size 0 max 16384 2024-09-19T09:19:02.407 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:19:02.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.399+0000 7f9afa04d640 1 -- 172.21.15.120:0/2421248014 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9adc0a3240 msgr2=0x7f9adc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:02.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.399+0000 7f9afa04d640 1 --2- 172.21.15.120:0/2421248014 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9adc0a3240 0x7f9adc0a5700 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9aec0099e0 tx=0x7f9aec004a40 comp rx=0 tx=0).stop 2024-09-19T09:19:02.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.399+0000 7f9afa04d640 1 -- 172.21.15.120:0/2421248014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9af4101440 msgr2=0x7f9af41a0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:02.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.399+0000 7f9afa04d640 1 --2- 172.21.15.120:0/2421248014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9af4101440 0x7f9af41a0940 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8005940 tx=0x7f9ae80040c0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.399+0000 7f9afa04d640 1 -- 172.21.15.120:0/2421248014 shutdown_connections 2024-09-19T09:19:02.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.399+0000 7f9afa04d640 1 --2- 172.21.15.120:0/2421248014 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9adc0a3240 0x7f9adc0a5700 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.399+0000 7f9afa04d640 1 --2- 172.21.15.120:0/2421248014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af410e500 0x7f9af419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.399+0000 7f9afa04d640 1 --2- 172.21.15.120:0/2421248014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9af4101440 0x7f9af41a0940 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.399+0000 7f9afa04d640 1 --2- 172.21.15.120:0/2421248014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9af4100b20 0x7f9af41a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:02.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.399+0000 7f9afa04d640 1 -- 172.21.15.120:0/2421248014 >> 172.21.15.120:0/2421248014 conn(0x7f9af40fc7d0 msgr2=0x7f9af40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:02.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.399+0000 7f9afa04d640 1 -- 172.21.15.120:0/2421248014 shutdown_connections 2024-09-19T09:19:02.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:02.399+0000 7f9afa04d640 1 -- 172.21.15.120:0/2421248014 wait complete. 2024-09-19T09:19:03.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:02 smithi120 ceph-mon[31150]: pgmap v830: 97 pgs: 97 active+clean; 276 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 1.1 MiB/s wr, 53 op/s 2024-09-19T09:19:03.226 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:19:03.239 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:02 smithi196 ceph-mon[35150]: pgmap v830: 97 pgs: 97 active+clean; 276 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 1.1 MiB/s wr, 53 op/s 2024-09-19T09:19:03.363 INFO:tasks.workunit.client.0.smithi120.stdout: 10 59434 3.36 MB/sec execute 58 sec latency 80.899 ms 2024-09-19T09:19:03.679 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:02 smithi153 ceph-mon[35071]: pgmap v830: 97 pgs: 97 active+clean; 276 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 1.1 MiB/s wr, 53 op/s 2024-09-19T09:19:03.679 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:19:04.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.982+0000 7ff16d86f640 1 -- 172.21.15.120:0/3042404616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff168102d90 msgr2=0x7ff168103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:04.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.982+0000 7ff16d86f640 1 --2- 172.21.15.120:0/3042404616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff168102d90 0x7ff168103170 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7ff158009930 tx=0x7ff15802a420 comp rx=0 tx=0).stop 2024-09-19T09:19:04.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.982+0000 7ff16d86f640 1 -- 172.21.15.120:0/3042404616 shutdown_connections 2024-09-19T09:19:04.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.982+0000 7ff16d86f640 1 --2- 172.21.15.120:0/3042404616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff168108290 0x7ff16810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.982+0000 7ff16d86f640 1 --2- 172.21.15.120:0/3042404616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff168103740 0x7ff168107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.982+0000 7ff16d86f640 1 --2- 172.21.15.120:0/3042404616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff168102d90 0x7ff168103170 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.982+0000 7ff16d86f640 1 -- 172.21.15.120:0/3042404616 >> 172.21.15.120:0/3042404616 conn(0x7ff1680fc7d0 msgr2=0x7ff1680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:04.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.982+0000 7ff16d86f640 1 -- 172.21.15.120:0/3042404616 shutdown_connections 2024-09-19T09:19:04.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.982+0000 7ff16d86f640 1 -- 172.21.15.120:0/3042404616 wait complete. 2024-09-19T09:19:04.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.983+0000 7ff16d86f640 1 Processor -- start 2024-09-19T09:19:04.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.983+0000 7ff16d86f640 1 -- start start 2024-09-19T09:19:04.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.983+0000 7ff16d86f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff168102d90 0x7ff1681985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:04.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.983+0000 7ff16d86f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff168103740 0x7ff168198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:04.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.983+0000 7ff16d86f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff168108290 0x7ff16819cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:04.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.983+0000 7ff16d86f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff168112090 con 0x7ff168102d90 2024-09-19T09:19:04.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.983+0000 7ff16d86f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff168111f10 con 0x7ff168108290 2024-09-19T09:19:04.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.983+0000 7ff16d86f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff168112210 con 0x7ff168103740 2024-09-19T09:19:04.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.983+0000 7ff166ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff168102d90 0x7ff1681985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:04.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.983+0000 7ff166ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff168102d90 0x7ff1681985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48402/0 (socket says 172.21.15.120:48402) 2024-09-19T09:19:04.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.983+0000 7ff166ffd640 1 -- 172.21.15.120:0/1832921065 learned_addr learned my addr 172.21.15.120:0/1832921065 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:04.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.983+0000 7ff1677fe640 1 --2- 172.21.15.120:0/1832921065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff168108290 0x7ff16819cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:04.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.983+0000 7ff1667fc640 1 --2- 172.21.15.120:0/1832921065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff168103740 0x7ff168198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:04.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.983+0000 7ff166ffd640 1 -- 172.21.15.120:0/1832921065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff168103740 msgr2=0x7ff168198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:04.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.983+0000 7ff166ffd640 1 --2- 172.21.15.120:0/1832921065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff168103740 0x7ff168198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.983+0000 7ff166ffd640 1 -- 172.21.15.120:0/1832921065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff168108290 msgr2=0x7ff16819cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:04.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.983+0000 7ff166ffd640 1 --2- 172.21.15.120:0/1832921065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff168108290 0x7ff16819cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.983+0000 7ff166ffd640 1 -- 172.21.15.120:0/1832921065 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff16819d580 con 0x7ff168102d90 2024-09-19T09:19:04.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.984+0000 7ff166ffd640 1 --2- 172.21.15.120:0/1832921065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff168102d90 0x7ff1681985d0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7ff158009230 tx=0x7ff158004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:04.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.984+0000 7ff16c86d640 1 -- 172.21.15.120:0/1832921065 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15803f030 con 0x7ff168102d90 2024-09-19T09:19:04.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.984+0000 7ff16c86d640 1 -- 172.21.15.120:0/1832921065 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff158003ad0 con 0x7ff168102d90 2024-09-19T09:19:04.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.984+0000 7ff16d86f640 1 -- 172.21.15.120:0/1832921065 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff16819d810 con 0x7ff168102d90 2024-09-19T09:19:04.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.984+0000 7ff16c86d640 1 -- 172.21.15.120:0/1832921065 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff158035070 con 0x7ff168102d90 2024-09-19T09:19:04.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.984+0000 7ff16d86f640 1 -- 172.21.15.120:0/1832921065 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1681a5150 con 0x7ff168102d90 2024-09-19T09:19:04.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.986+0000 7ff16c86d640 1 -- 172.21.15.120:0/1832921065 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7ff158042030 con 0x7ff168102d90 2024-09-19T09:19:04.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.986+0000 7ff16fc48740 1 Processor -- start 2024-09-19T09:19:04.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.986+0000 7ff16fc48740 1 -- start start 2024-09-19T09:19:04.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.987+0000 7ff16fc48740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea05beb40 0x55fea05bd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:04.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.987+0000 7ff16fc48740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea05bd5d0 0x55fea05bda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:04.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.987+0000 7ff16fc48740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fea05c8370 0x55fea05ca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:04.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.987+0000 7ff16fc48740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fea05232f0 con 0x55fea05c8370 2024-09-19T09:19:04.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.987+0000 7ff16fc48740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fea04cacf0 con 0x55fea05bd5d0 2024-09-19T09:19:04.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.987+0000 7ff16fc48740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fea051f2b0 con 0x55fea05beb40 2024-09-19T09:19:04.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.987+0000 7ff166ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea05bd5d0 0x55fea05bda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:04.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.987+0000 7ff1677fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea05beb40 0x55fea05bd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:04.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.987+0000 7ff1677fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea05beb40 0x55fea05bd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:52708/0 (socket says 172.21.15.120:52708) 2024-09-19T09:19:04.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.987+0000 7ff1677fe640 1 -- 172.21.15.120:0/2801401518 learned_addr learned my addr 172.21.15.120:0/2801401518 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:04.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.987+0000 7ff166ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea05bd5d0 0x55fea05bda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:43278/0 (socket says 172.21.15.120:43278) 2024-09-19T09:19:04.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.987+0000 7ff1667fc640 1 --2- 172.21.15.120:0/2801401518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fea05c8370 0x55fea05ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:04.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.987+0000 7ff16c86d640 1 --2- 172.21.15.120:0/1832921065 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff1500a3240 0x7ff1500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:04.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.987+0000 7ff1677fe640 1 -- 172.21.15.120:0/2801401518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea05bd5d0 msgr2=0x55fea05bda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:04.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.987+0000 7ff1677fe640 1 --2- 172.21.15.120:0/2801401518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea05bd5d0 0x55fea05bda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.987+0000 7ff1677fe640 1 -- 172.21.15.120:0/2801401518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fea05c8370 msgr2=0x55fea05ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:04.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.987+0000 7ff1677fe640 1 --2- 172.21.15.120:0/2801401518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fea05c8370 0x55fea05ca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.987+0000 7ff1677fe640 1 -- 172.21.15.120:0/2801401518 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fea04af3a0 con 0x55fea05beb40 2024-09-19T09:19:04.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.987+0000 7ff166ffd640 1 --2- 172.21.15.120:0/2801401518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea05bd5d0 0x55fea05bda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:04.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.987+0000 7ff1667fc640 1 --2- 172.21.15.120:0/2801401518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fea05c8370 0x55fea05ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:04.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.987+0000 7ff16c86d640 1 -- 172.21.15.120:0/1832921065 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff1580effa0 con 0x7ff168102d90 2024-09-19T09:19:04.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.988+0000 7ff1677fe640 1 --2- 172.21.15.120:0/2801401518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea05beb40 0x55fea05bd090 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff160009f50 tx=0x7ff1600118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6153ae9df48cbe1a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:04.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.988+0000 7ff1677fe640 1 --2- 172.21.15.120:0/1832921065 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff1500a3240 0x7ff1500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:04.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.988+0000 7ff14dffb640 1 -- 172.21.15.120:0/2801401518 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16000f680 con 0x55fea05beb40 2024-09-19T09:19:04.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.988+0000 7ff1677fe640 1 --2- 172.21.15.120:0/1832921065 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff1500a3240 0x7ff1500a5700 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff160012fd0 tx=0x7ff160021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:04.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.988+0000 7ff14dffb640 1 -- 172.21.15.120:0/2801401518 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff16000fd00 con 0x55fea05beb40 2024-09-19T09:19:04.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.988+0000 7ff14dffb640 1 -- 172.21.15.120:0/2801401518 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16001bb10 con 0x55fea05beb40 2024-09-19T09:19:04.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.989+0000 7ff16fc48740 1 -- 172.21.15.120:0/2801401518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea05beb40 msgr2=0x55fea05bd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:04.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.989+0000 7ff16fc48740 1 --2- 172.21.15.120:0/2801401518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea05beb40 0x55fea05bd090 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff160009f50 tx=0x7ff1600118d0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.989+0000 7ff16fc48740 1 -- 172.21.15.120:0/2801401518 shutdown_connections 2024-09-19T09:19:04.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.989+0000 7ff16fc48740 1 --2- 172.21.15.120:0/2801401518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fea05c8370 0x55fea05ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.989+0000 7ff16fc48740 1 --2- 172.21.15.120:0/2801401518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea05bd5d0 0x55fea05bda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.989+0000 7ff16fc48740 1 --2- 172.21.15.120:0/2801401518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea05beb40 0x55fea05bd090 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.989+0000 7ff16fc48740 1 -- 172.21.15.120:0/2801401518 >> 172.21.15.120:0/2801401518 conn(0x55fea0586310 msgr2=0x55fea05b9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:04.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.989+0000 7ff16fc48740 1 -- 172.21.15.120:0/2801401518 shutdown_connections 2024-09-19T09:19:04.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.989+0000 7ff16fc48740 1 -- 172.21.15.120:0/2801401518 wait complete. 2024-09-19T09:19:04.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.990+0000 7ff16fc48740 1 -- start start 2024-09-19T09:19:04.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.990+0000 7ff16fc48740 1 Processor -- start 2024-09-19T09:19:04.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.990+0000 7ff16fc48740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea05bd5d0 0x55fea05c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:04.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.990+0000 7ff16fc48740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fea05beb40 0x55fea05c5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:04.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.990+0000 7ff16fc48740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea05c8370 0x55fea05c5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:04.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.990+0000 7ff16fc48740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fea05cb070 con 0x55fea05beb40 2024-09-19T09:19:04.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.990+0000 7ff16fc48740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fea04de060 con 0x55fea05c8370 2024-09-19T09:19:04.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.990+0000 7ff16fc48740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fea05cb1f0 con 0x55fea05bd5d0 2024-09-19T09:19:04.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.990+0000 7ff16fc48740 10 client.0 _ll_register_callbacks cb 0x55fea02318c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:04.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.990+0000 7ff1677fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea05c8370 0x55fea05c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:04.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.990+0000 7ff1667fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fea05beb40 0x55fea05c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:04.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.991+0000 7ff166ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea05bd5d0 0x55fea05c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:04.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.991+0000 7ff1677fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea05c8370 0x55fea05c5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:43286/0 (socket says 172.21.15.120:43286) 2024-09-19T09:19:04.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.991+0000 7ff1677fe640 1 -- 172.21.15.120:0/3793911334 learned_addr learned my addr 172.21.15.120:0/3793911334 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:04.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.991+0000 7ff1667fc640 1 -- 172.21.15.120:0/3793911334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea05bd5d0 msgr2=0x55fea05c7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:04.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.991+0000 7ff1667fc640 1 --2- 172.21.15.120:0/3793911334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea05bd5d0 0x55fea05c7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.991+0000 7ff1667fc640 1 -- 172.21.15.120:0/3793911334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea05c8370 msgr2=0x55fea05c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:04.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.991+0000 7ff1667fc640 1 --2- 172.21.15.120:0/3793911334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea05c8370 0x55fea05c5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.991+0000 7ff1667fc640 1 -- 172.21.15.120:0/3793911334 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fea02943c0 con 0x55fea05beb40 2024-09-19T09:19:04.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.991+0000 7ff1677fe640 1 --2- 172.21.15.120:0/3793911334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea05c8370 0x55fea05c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:04.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.991+0000 7ff1667fc640 1 --2- 172.21.15.120:0/3793911334 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fea05beb40 0x55fea05c5080 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7ff1680fee00 tx=0x7ff15c00de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfad97d4dce34816 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:04.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.991+0000 7ff146ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:19:04.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.991+0000 7ff146ffd640 1 -- 172.21.15.120:0/3793911334 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c011440 con 0x55fea05beb40 2024-09-19T09:19:04.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.991+0000 7ff146ffd640 1 -- 172.21.15.120:0/3793911334 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff15c0115e0 con 0x55fea05beb40 2024-09-19T09:19:04.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.991+0000 7ff16fc48740 1 -- 172.21.15.120:0/3793911334 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff16000fd00 con 0x55fea05beb40 2024-09-19T09:19:04.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.992+0000 7ff146ffd640 1 -- 172.21.15.120:0/3793911334 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff15c0117e0 con 0x55fea05beb40 2024-09-19T09:19:04.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.992+0000 7ff16fc48740 10 client.20967 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:04.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.992+0000 7ff16fc48740 1 -- 172.21.15.120:0/3793911334 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55fea05d6bf0 con 0x55fea05beb40 2024-09-19T09:19:04.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.992+0000 7ff146ffd640 1 -- 172.21.15.120:0/3793911334 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff16000fd00 con 0x55fea05beb40 2024-09-19T09:19:04.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.992+0000 7ff146ffd640 1 -- 172.21.15.120:0/3793911334 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7ff15c019030 con 0x55fea05beb40 2024-09-19T09:19:04.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.992+0000 7ff16fc48740 10 client.20967 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:04.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.992+0000 7ff16fc48740 10 client.20967 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:19:04.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.992+0000 7ff16fc48740 20 client.20967 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:04.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.992+0000 7ff16fc48740 1 --2- 172.21.15.120:0/3793911334 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55fea05d80b0 0x55fea05da4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:04.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.992+0000 7ff16fc48740 4 client.20967 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:19:04.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.992+0000 7ff16fc48740 1 -- 172.21.15.120:0/3793911334 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fea0530860 con 0x55fea05d80b0 2024-09-19T09:19:04.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.992+0000 7ff166ffd640 1 --2- 172.21.15.120:0/3793911334 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55fea05d80b0 0x55fea05da4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:04.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.993+0000 7ff166ffd640 1 --2- 172.21.15.120:0/3793911334 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55fea05d80b0 0x55fea05da4f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=71821e80e53a90f7 server_cookie=aa6a1a958e48ebb5 in_seq=0 out_seq=0 2024-09-19T09:19:04.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.993+0000 7ff146ffd640 10 client.20967 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:19:04.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.993+0000 7ff146ffd640 1 -- 172.21.15.120:0/3793911334 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fea0530860 con 0x55fea05d80b0 2024-09-19T09:19:04.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.993+0000 7ff146ffd640 10 client.20967 handle_command_reply: tid=0 2024-09-19T09:19:04.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.993+0000 7ff16fc48740 1 client.20967 shutdown 2024-09-19T09:19:04.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.994+0000 7ff16fc48740 1 -- 172.21.15.120:0/3793911334 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fea05beb40 msgr2=0x55fea05c5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:04.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.994+0000 7ff16fc48740 1 --2- 172.21.15.120:0/3793911334 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fea05beb40 0x55fea05c5080 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7ff1680fee00 tx=0x7ff15c00de50 comp rx=0 tx=0).stop 2024-09-19T09:19:04.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.994+0000 7ff16fc48740 1 -- 172.21.15.120:0/3793911334 shutdown_connections 2024-09-19T09:19:04.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.994+0000 7ff16fc48740 1 --2- 172.21.15.120:0/3793911334 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55fea05d80b0 0x55fea05da4f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.994+0000 7ff16fc48740 1 --2- 172.21.15.120:0/3793911334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea05c8370 0x55fea05c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.994+0000 7ff16fc48740 1 --2- 172.21.15.120:0/3793911334 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fea05beb40 0x55fea05c5080 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.994+0000 7ff16fc48740 1 --2- 172.21.15.120:0/3793911334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea05bd5d0 0x55fea05c7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.994+0000 7ff146ffd640 0 client.20967 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:19:04.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.994+0000 7ff16fc48740 1 -- 172.21.15.120:0/3793911334 >> 172.21.15.120:0/3793911334 conn(0x55fea0586310 msgr2=0x55fea05bf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:04.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.994+0000 7ff16fc48740 1 -- 172.21.15.120:0/3793911334 shutdown_connections 2024-09-19T09:19:04.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.994+0000 7ff16fc48740 1 -- 172.21.15.120:0/3793911334 wait complete. 2024-09-19T09:19:04.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:03.994+0000 7ff16fc48740 20 client.20967 trim_cache size 0 max 16384 2024-09-19T09:19:04.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.008+0000 7ff16fc48740 1 Processor -- start 2024-09-19T09:19:04.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.008+0000 7ff16fc48740 1 -- start start 2024-09-19T09:19:04.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.008+0000 7ff16fc48740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fea05bd5d0 0x55fea05c5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:04.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.008+0000 7ff16fc48740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea05beb40 0x55fea05be3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:04.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.008+0000 7ff16fc48740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea05c8370 0x55fea05dfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:04.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.008+0000 7ff16fc48740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff1681ab4f0 con 0x55fea05bd5d0 2024-09-19T09:19:04.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.008+0000 7ff16fc48740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff1681ad030 con 0x55fea05c8370 2024-09-19T09:19:04.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.008+0000 7ff16fc48740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff1681abbc0 con 0x55fea05beb40 2024-09-19T09:19:04.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.008+0000 7ff166ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fea05bd5d0 0x55fea05c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:04.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.008+0000 7ff166ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fea05bd5d0 0x55fea05c5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48430/0 (socket says 172.21.15.120:48430) 2024-09-19T09:19:04.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.008+0000 7ff166ffd640 1 -- 172.21.15.120:0/4045938720 learned_addr learned my addr 172.21.15.120:0/4045938720 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:04.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.008+0000 7ff1667fc640 1 --2- 172.21.15.120:0/4045938720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea05beb40 0x55fea05be3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:04.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.008+0000 7ff1677fe640 1 --2- 172.21.15.120:0/4045938720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea05c8370 0x55fea05dfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:04.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.008+0000 7ff166ffd640 1 -- 172.21.15.120:0/4045938720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea05beb40 msgr2=0x55fea05be3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:04.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.008+0000 7ff166ffd640 1 --2- 172.21.15.120:0/4045938720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea05beb40 0x55fea05be3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.008+0000 7ff166ffd640 1 -- 172.21.15.120:0/4045938720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea05c8370 msgr2=0x55fea05dfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:04.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.008+0000 7ff166ffd640 1 --2- 172.21.15.120:0/4045938720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea05c8370 0x55fea05dfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.008+0000 7ff166ffd640 1 -- 172.21.15.120:0/4045938720 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fea05d4310 con 0x55fea05bd5d0 2024-09-19T09:19:04.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.009+0000 7ff166ffd640 1 --2- 172.21.15.120:0/4045938720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fea05bd5d0 0x55fea05c5050 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7ff1580b8a20 tx=0x7ff1580f3a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed935c7e14f0fc40 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:04.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.009+0000 7ff14d7fa640 1 -- 172.21.15.120:0/4045938720 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff158049040 con 0x55fea05bd5d0 2024-09-19T09:19:04.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.009+0000 7ff14d7fa640 1 -- 172.21.15.120:0/4045938720 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff15803b460 con 0x55fea05bd5d0 2024-09-19T09:19:04.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.009+0000 7ff14d7fa640 1 -- 172.21.15.120:0/4045938720 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1580c5230 con 0x55fea05bd5d0 2024-09-19T09:19:04.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.010+0000 7ff16fc48740 1 -- 172.21.15.120:0/4045938720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fea05bd5d0 msgr2=0x55fea05c5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:04.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.010+0000 7ff16fc48740 1 --2- 172.21.15.120:0/4045938720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fea05bd5d0 0x55fea05c5050 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7ff1580b8a20 tx=0x7ff1580f3a20 comp rx=0 tx=0).stop 2024-09-19T09:19:04.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.010+0000 7ff16fc48740 1 -- 172.21.15.120:0/4045938720 shutdown_connections 2024-09-19T09:19:04.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.010+0000 7ff16fc48740 1 --2- 172.21.15.120:0/4045938720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea05c8370 0x55fea05dfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.010+0000 7ff16fc48740 1 --2- 172.21.15.120:0/4045938720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea05beb40 0x55fea05be3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.010+0000 7ff16fc48740 1 --2- 172.21.15.120:0/4045938720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fea05bd5d0 0x55fea05c5050 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.010+0000 7ff16fc48740 1 -- 172.21.15.120:0/4045938720 >> 172.21.15.120:0/4045938720 conn(0x55fea0586310 msgr2=0x55fea05c4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:04.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.010+0000 7ff16fc48740 1 -- 172.21.15.120:0/4045938720 shutdown_connections 2024-09-19T09:19:04.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.010+0000 7ff16fc48740 1 -- 172.21.15.120:0/4045938720 wait complete. 2024-09-19T09:19:04.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.010+0000 7ff16fc48740 1 -- start start 2024-09-19T09:19:04.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.010+0000 7ff16fc48740 1 Processor -- start 2024-09-19T09:19:04.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.011+0000 7ff16fc48740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fea05bd5d0 0x55fea05cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:04.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.011+0000 7ff16fc48740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea05beb40 0x55fea05cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:04.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.011+0000 7ff16fc48740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea05c8370 0x55fea0640d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:04.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.011+0000 7ff16fc48740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff1681ac960 con 0x55fea05bd5d0 2024-09-19T09:19:04.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.011+0000 7ff16fc48740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff1681ac290 con 0x55fea05c8370 2024-09-19T09:19:04.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.011+0000 7ff16fc48740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff1681a94b0 con 0x55fea05beb40 2024-09-19T09:19:04.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.011+0000 7ff16fc48740 10 client.0 _ll_register_callbacks cb 0x7ff15c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:04.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.011+0000 7ff166ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fea05bd5d0 0x55fea05cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:04.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.011+0000 7ff1667fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea05beb40 0x55fea05cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:04.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.011+0000 7ff1667fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea05beb40 0x55fea05cbb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:52742/0 (socket says 172.21.15.120:52742) 2024-09-19T09:19:04.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.011+0000 7ff166ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fea05bd5d0 0x55fea05cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48440/0 (socket says 172.21.15.120:48440) 2024-09-19T09:19:04.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.011+0000 7ff1667fc640 1 -- 172.21.15.120:0/3903530576 learned_addr learned my addr 172.21.15.120:0/3903530576 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:04.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.011+0000 7ff1677fe640 1 --2- 172.21.15.120:0/3903530576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea05c8370 0x55fea0640d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:04.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.011+0000 7ff166ffd640 1 -- 172.21.15.120:0/3903530576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea05beb40 msgr2=0x55fea05cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:04.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.011+0000 7ff166ffd640 1 --2- 172.21.15.120:0/3903530576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea05beb40 0x55fea05cbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.011+0000 7ff166ffd640 1 -- 172.21.15.120:0/3903530576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea05c8370 msgr2=0x55fea0640d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:04.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.011+0000 7ff166ffd640 1 --2- 172.21.15.120:0/3903530576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea05c8370 0x55fea0640d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.011+0000 7ff166ffd640 1 -- 172.21.15.120:0/3903530576 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fea06412c0 con 0x55fea05bd5d0 2024-09-19T09:19:04.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.011+0000 7ff166ffd640 1 --2- 172.21.15.120:0/3903530576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fea05bd5d0 0x55fea05cb620 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7ff1580b8f90 tx=0x7ff1580c09b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b660084c5c0453e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:04.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.011+0000 7ff146ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:19:04.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.011+0000 7ff146ffd640 1 -- 172.21.15.120:0/3903530576 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1580d5020 con 0x55fea05bd5d0 2024-09-19T09:19:04.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.012+0000 7ff146ffd640 1 -- 172.21.15.120:0/3903530576 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff15803bd40 con 0x55fea05bd5d0 2024-09-19T09:19:04.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.012+0000 7ff16fc48740 1 -- 172.21.15.120:0/3903530576 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fea052ad70 con 0x55fea05bd5d0 2024-09-19T09:19:04.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.012+0000 7ff146ffd640 1 -- 172.21.15.120:0/3903530576 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff158049040 con 0x55fea05bd5d0 2024-09-19T09:19:04.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.012+0000 7ff16fc48740 10 client.20973 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:04.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.012+0000 7ff16fc48740 1 -- 172.21.15.120:0/3903530576 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55fea06424c0 con 0x55fea05bd5d0 2024-09-19T09:19:04.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.012+0000 7ff146ffd640 1 -- 172.21.15.120:0/3903530576 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55fea052ad70 con 0x55fea05bd5d0 2024-09-19T09:19:04.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.012+0000 7ff146ffd640 1 -- 172.21.15.120:0/3903530576 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7ff158003c80 con 0x55fea05bd5d0 2024-09-19T09:19:04.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.012+0000 7ff16fc48740 10 client.20973 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:04.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.012+0000 7ff16fc48740 10 client.20973 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:19:04.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.012+0000 7ff16fc48740 20 client.20973 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:04.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.012+0000 7ff16fc48740 1 --2- 172.21.15.120:0/3903530576 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55fea0642bd0 0x55fea0644ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:04.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.012+0000 7ff16fc48740 4 client.20973 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:19:04.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.012+0000 7ff16fc48740 1 -- 172.21.15.120:0/3903530576 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fea04a7170 con 0x55fea0642bd0 2024-09-19T09:19:04.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.013+0000 7ff1667fc640 1 --2- 172.21.15.120:0/3903530576 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55fea0642bd0 0x55fea0644ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:04.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.014+0000 7ff1667fc640 1 --2- 172.21.15.120:0/3903530576 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55fea0642bd0 0x55fea0644ff0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=779cb6f944f0538f server_cookie=924e4bde5b772b6f in_seq=0 out_seq=0 2024-09-19T09:19:04.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.014+0000 7ff146ffd640 10 client.20973 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:19:04.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.014+0000 7ff146ffd640 1 -- 172.21.15.120:0/3903530576 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fea04a7170 con 0x55fea0642bd0 2024-09-19T09:19:04.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.014+0000 7ff146ffd640 10 client.20973 handle_command_reply: tid=0 2024-09-19T09:19:04.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.014+0000 7ff16fc48740 1 client.20973 shutdown 2024-09-19T09:19:04.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.014+0000 7ff16fc48740 1 -- 172.21.15.120:0/3903530576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fea05bd5d0 msgr2=0x55fea05cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:04.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.014+0000 7ff16fc48740 1 --2- 172.21.15.120:0/3903530576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fea05bd5d0 0x55fea05cb620 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7ff1580b8f90 tx=0x7ff1580c09b0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.014+0000 7ff16fc48740 1 -- 172.21.15.120:0/3903530576 shutdown_connections 2024-09-19T09:19:04.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.014+0000 7ff16fc48740 1 --2- 172.21.15.120:0/3903530576 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55fea0642bd0 0x55fea0644ff0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.014+0000 7ff16fc48740 1 --2- 172.21.15.120:0/3903530576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fea05c8370 0x55fea0640d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.015+0000 7ff16fc48740 1 --2- 172.21.15.120:0/3903530576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fea05beb40 0x55fea05cbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.015+0000 7ff146ffd640 0 client.20973 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:19:04.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.015+0000 7ff16fc48740 1 --2- 172.21.15.120:0/3903530576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fea05bd5d0 0x55fea05cb620 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.015+0000 7ff16fc48740 1 -- 172.21.15.120:0/3903530576 >> 172.21.15.120:0/3903530576 conn(0x55fea0586310 msgr2=0x55fea05c3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:04.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.015+0000 7ff16fc48740 1 -- 172.21.15.120:0/3903530576 shutdown_connections 2024-09-19T09:19:04.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.015+0000 7ff16fc48740 1 -- 172.21.15.120:0/3903530576 wait complete. 2024-09-19T09:19:04.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.015+0000 7ff16fc48740 20 client.20973 trim_cache size 0 max 16384 2024-09-19T09:19:04.032 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:19:04.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.015+0000 7ff16d86f640 1 -- 172.21.15.120:0/1832921065 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff1500a3240 msgr2=0x7ff1500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:04.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.015+0000 7ff16d86f640 1 --2- 172.21.15.120:0/1832921065 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff1500a3240 0x7ff1500a5700 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff160012fd0 tx=0x7ff160021000 comp rx=0 tx=0).stop 2024-09-19T09:19:04.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.015+0000 7ff16d86f640 1 -- 172.21.15.120:0/1832921065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff168102d90 msgr2=0x7ff1681985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:04.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.015+0000 7ff16d86f640 1 --2- 172.21.15.120:0/1832921065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff168102d90 0x7ff1681985d0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7ff158009230 tx=0x7ff158004270 comp rx=0 tx=0).stop 2024-09-19T09:19:04.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.015+0000 7ff16d86f640 1 -- 172.21.15.120:0/1832921065 shutdown_connections 2024-09-19T09:19:04.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.015+0000 7ff16d86f640 1 --2- 172.21.15.120:0/1832921065 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff1500a3240 0x7ff1500a5700 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.015+0000 7ff16d86f640 1 --2- 172.21.15.120:0/1832921065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff168108290 0x7ff16819cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.015+0000 7ff16d86f640 1 --2- 172.21.15.120:0/1832921065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff168103740 0x7ff168198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.015+0000 7ff16d86f640 1 --2- 172.21.15.120:0/1832921065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff168102d90 0x7ff1681985d0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:04.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.015+0000 7ff16d86f640 1 -- 172.21.15.120:0/1832921065 >> 172.21.15.120:0/1832921065 conn(0x7ff1680fc7d0 msgr2=0x7ff168106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:04.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.016+0000 7ff16d86f640 1 -- 172.21.15.120:0/1832921065 shutdown_connections 2024-09-19T09:19:04.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:04.016+0000 7ff16d86f640 1 -- 172.21.15.120:0/1832921065 wait complete. 2024-09-19T09:19:04.361 INFO:tasks.workunit.client.0.smithi120.stdout: 10 60074 3.94 MB/sec execute 59 sec latency 63.756 ms 2024-09-19T09:19:04.931 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:19:05.203 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:19:05.239 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:04 smithi153 ceph-mon[35071]: pgmap v831: 97 pgs: 97 active+clean; 282 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 146 op/s 2024-09-19T09:19:05.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:04 smithi120 ceph-mon[31150]: pgmap v831: 97 pgs: 97 active+clean; 282 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 146 op/s 2024-09-19T09:19:05.361 INFO:tasks.workunit.client.0.smithi120.stdout: 10 60588 4.34 MB/sec execute 60 sec latency 101.239 ms 2024-09-19T09:19:05.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:04 smithi196 ceph-mon[35150]: pgmap v831: 97 pgs: 97 active+clean; 282 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 146 op/s 2024-09-19T09:19:05.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.733+0000 7f93837d4640 1 -- 172.21.15.120:0/552670511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f937c102d30 msgr2=0x7f937c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:05.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.733+0000 7f93837d4640 1 --2- 172.21.15.120:0/552670511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f937c102d30 0x7f937c103110 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f9378009930 tx=0x7f937802a420 comp rx=0 tx=0).stop 2024-09-19T09:19:05.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.733+0000 7f93837d4640 1 -- 172.21.15.120:0/552670511 shutdown_connections 2024-09-19T09:19:05.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.733+0000 7f93837d4640 1 --2- 172.21.15.120:0/552670511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f937c1082a0 0x7f937c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.733+0000 7f93837d4640 1 --2- 172.21.15.120:0/552670511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f937c1036e0 0x7f937c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.733+0000 7f93837d4640 1 --2- 172.21.15.120:0/552670511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f937c102d30 0x7f937c103110 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.733+0000 7f93837d4640 1 -- 172.21.15.120:0/552670511 >> 172.21.15.120:0/552670511 conn(0x7f937c0fc7d0 msgr2=0x7f937c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:05.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.733+0000 7f93837d4640 1 -- 172.21.15.120:0/552670511 shutdown_connections 2024-09-19T09:19:05.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.733+0000 7f93837d4640 1 -- 172.21.15.120:0/552670511 wait complete. 2024-09-19T09:19:05.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.734+0000 7f93837d4640 1 Processor -- start 2024-09-19T09:19:05.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.734+0000 7f93837d4640 1 -- start start 2024-09-19T09:19:05.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.734+0000 7f93837d4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f937c102d30 0x7f937c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:05.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.734+0000 7f93837d4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f937c1036e0 0x7f937c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:05.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.734+0000 7f93837d4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f937c1082a0 0x7f937c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:05.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.734+0000 7f93837d4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f937c1120a0 con 0x7f937c1082a0 2024-09-19T09:19:05.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.734+0000 7f93837d4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f937c111f20 con 0x7f937c1036e0 2024-09-19T09:19:05.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.734+0000 7f93837d4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f937c112220 con 0x7f937c102d30 2024-09-19T09:19:05.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.735+0000 7f9380d6f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f937c1036e0 0x7f937c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:05.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.735+0000 7f9381570640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f937c102d30 0x7f937c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:05.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.735+0000 7f9381d71640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f937c1082a0 0x7f937c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:05.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.735+0000 7f9381570640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f937c102d30 0x7f937c100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:52758/0 (socket says 172.21.15.120:52758) 2024-09-19T09:19:05.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.735+0000 7f9380d6f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f937c1036e0 0x7f937c100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:43310/0 (socket says 172.21.15.120:43310) 2024-09-19T09:19:05.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.735+0000 7f9381570640 1 -- 172.21.15.120:0/521004027 learned_addr learned my addr 172.21.15.120:0/521004027 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:05.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.735+0000 7f9381570640 1 -- 172.21.15.120:0/521004027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f937c1036e0 msgr2=0x7f937c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:05.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.735+0000 7f9381570640 1 --2- 172.21.15.120:0/521004027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f937c1036e0 0x7f937c100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.735+0000 7f9381570640 1 -- 172.21.15.120:0/521004027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f937c1082a0 msgr2=0x7f937c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:05.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.735+0000 7f9381570640 1 --2- 172.21.15.120:0/521004027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f937c1082a0 0x7f937c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.735+0000 7f9381570640 1 -- 172.21.15.120:0/521004027 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937c1a6a80 con 0x7f937c102d30 2024-09-19T09:19:05.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.735+0000 7f9381d71640 1 --2- 172.21.15.120:0/521004027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f937c1082a0 0x7f937c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:05.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.735+0000 7f9381570640 1 --2- 172.21.15.120:0/521004027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f937c102d30 0x7f937c100990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9378009230 tx=0x7f9378004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:05.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.735+0000 7f93727fc640 1 -- 172.21.15.120:0/521004027 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937803f030 con 0x7f937c102d30 2024-09-19T09:19:05.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.735+0000 7f93837d4640 1 -- 172.21.15.120:0/521004027 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f937c1a6d10 con 0x7f937c102d30 2024-09-19T09:19:05.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.735+0000 7f93837d4640 1 -- 172.21.15.120:0/521004027 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f937c1a71e0 con 0x7f937c102d30 2024-09-19T09:19:05.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.735+0000 7f93727fc640 1 -- 172.21.15.120:0/521004027 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9378004440 con 0x7f937c102d30 2024-09-19T09:19:05.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.735+0000 7f93727fc640 1 -- 172.21.15.120:0/521004027 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9378035070 con 0x7f937c102d30 2024-09-19T09:19:05.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.738+0000 7f93727fc640 1 -- 172.21.15.120:0/521004027 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f9378042030 con 0x7f937c102d30 2024-09-19T09:19:05.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.738+0000 7f9385bad740 1 Processor -- start 2024-09-19T09:19:05.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.738+0000 7f9385bad740 1 -- start start 2024-09-19T09:19:05.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.739+0000 7f9385bad740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1e137b40 0x55ba1e136090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:05.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.739+0000 7f9385bad740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1e1365d0 0x55ba1e136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:05.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.739+0000 7f9385bad740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba1e141370 0x55ba1e143760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:05.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.739+0000 7f9385bad740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ba1e09c2f0 con 0x55ba1e1365d0 2024-09-19T09:19:05.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.739+0000 7f9385bad740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ba1e043cf0 con 0x55ba1e137b40 2024-09-19T09:19:05.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.739+0000 7f9385bad740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ba1e0982b0 con 0x55ba1e141370 2024-09-19T09:19:05.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.739+0000 7f9381d71640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1e137b40 0x55ba1e136090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:05.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.739+0000 7f9381d71640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1e137b40 0x55ba1e136090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:43326/0 (socket says 172.21.15.120:43326) 2024-09-19T09:19:05.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.739+0000 7f9381d71640 1 -- 172.21.15.120:0/3995604798 learned_addr learned my addr 172.21.15.120:0/3995604798 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:05.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.739+0000 7f9381570640 1 --2- 172.21.15.120:0/3995604798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1e1365d0 0x55ba1e136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:05.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.739+0000 7f9380d6f640 1 --2- 172.21.15.120:0/3995604798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba1e141370 0x55ba1e143760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:05.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.739+0000 7f9381d71640 1 -- 172.21.15.120:0/3995604798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba1e141370 msgr2=0x55ba1e143760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:05.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.739+0000 7f9381d71640 1 --2- 172.21.15.120:0/3995604798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba1e141370 0x55ba1e143760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.739+0000 7f93727fc640 1 --2- 172.21.15.120:0/521004027 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93680a3240 0x7f93680a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:05.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.739+0000 7f9381d71640 1 -- 172.21.15.120:0/3995604798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1e1365d0 msgr2=0x55ba1e136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:05.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.739+0000 7f9381d71640 1 --2- 172.21.15.120:0/3995604798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1e1365d0 0x55ba1e136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.739+0000 7f9381d71640 1 -- 172.21.15.120:0/3995604798 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba1e0283a0 con 0x55ba1e137b40 2024-09-19T09:19:05.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.739+0000 7f9381570640 1 --2- 172.21.15.120:0/3995604798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1e1365d0 0x55ba1e136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:19:05.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.739+0000 7f9380d6f640 1 --2- 172.21.15.120:0/3995604798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba1e141370 0x55ba1e143760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:19:05.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.739+0000 7f93727fc640 1 -- 172.21.15.120:0/521004027 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f93780ef870 con 0x7f937c102d30 2024-09-19T09:19:05.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.740+0000 7f9381d71640 1 --2- 172.21.15.120:0/521004027 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93680a3240 0x7f93680a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:05.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.740+0000 7f9381d71640 1 --2- 172.21.15.120:0/3995604798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1e137b40 0x55ba1e136090 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9374011790 tx=0x7f9374011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaee1c68c1606c3a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:05.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.740+0000 7f93677fe640 1 -- 172.21.15.120:0/3995604798 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9374004510 con 0x55ba1e137b40 2024-09-19T09:19:05.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.740+0000 7f93677fe640 1 -- 172.21.15.120:0/3995604798 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93740046b0 con 0x55ba1e137b40 2024-09-19T09:19:05.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.740+0000 7f93677fe640 1 -- 172.21.15.120:0/3995604798 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937400e610 con 0x55ba1e137b40 2024-09-19T09:19:05.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.740+0000 7f9381d71640 1 --2- 172.21.15.120:0/521004027 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93680a3240 0x7f93680a5700 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f937401f000 tx=0x7f9374020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:05.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.741+0000 7f9385bad740 1 -- 172.21.15.120:0/3995604798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1e137b40 msgr2=0x55ba1e136090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:05.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.741+0000 7f9385bad740 1 --2- 172.21.15.120:0/3995604798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1e137b40 0x55ba1e136090 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9374011790 tx=0x7f9374011c50 comp rx=0 tx=0).stop 2024-09-19T09:19:05.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.741+0000 7f9385bad740 1 -- 172.21.15.120:0/3995604798 shutdown_connections 2024-09-19T09:19:05.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.741+0000 7f9385bad740 1 --2- 172.21.15.120:0/3995604798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba1e141370 0x55ba1e143760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.741+0000 7f9385bad740 1 --2- 172.21.15.120:0/3995604798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1e1365d0 0x55ba1e136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.741+0000 7f9385bad740 1 --2- 172.21.15.120:0/3995604798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1e137b40 0x55ba1e136090 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.741+0000 7f9385bad740 1 -- 172.21.15.120:0/3995604798 >> 172.21.15.120:0/3995604798 conn(0x55ba1e0ff310 msgr2=0x55ba1e132c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:05.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.741+0000 7f9385bad740 1 -- 172.21.15.120:0/3995604798 shutdown_connections 2024-09-19T09:19:05.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.741+0000 7f9385bad740 1 -- 172.21.15.120:0/3995604798 wait complete. 2024-09-19T09:19:05.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.742+0000 7f9385bad740 1 -- start start 2024-09-19T09:19:05.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.742+0000 7f9385bad740 1 Processor -- start 2024-09-19T09:19:05.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.742+0000 7f9385bad740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba1e1365d0 0x55ba1e140da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:05.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.742+0000 7f9385bad740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1e137b40 0x55ba1e13e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:05.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.742+0000 7f9385bad740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1e141370 0x55ba1e13e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:05.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.742+0000 7f9385bad740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ba1e144070 con 0x55ba1e137b40 2024-09-19T09:19:05.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.742+0000 7f9385bad740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ba1e057060 con 0x55ba1e141370 2024-09-19T09:19:05.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.742+0000 7f9385bad740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ba1e1441f0 con 0x55ba1e1365d0 2024-09-19T09:19:05.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.742+0000 7f9385bad740 10 client.0 _ll_register_callbacks cb 0x55ba1ddaa8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:05.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.742+0000 7f9380d6f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1e137b40 0x55ba1e13e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:05.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.742+0000 7f9381570640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba1e1365d0 0x55ba1e140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:05.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.742+0000 7f9381d71640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1e141370 0x55ba1e13e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:05.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.742+0000 7f9380d6f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1e137b40 0x55ba1e13e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48472/0 (socket says 172.21.15.120:48472) 2024-09-19T09:19:05.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.742+0000 7f9380d6f640 1 -- 172.21.15.120:0/3199739241 learned_addr learned my addr 172.21.15.120:0/3199739241 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:05.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.742+0000 7f9381570640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba1e1365d0 0x55ba1e140da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:52788/0 (socket says 172.21.15.120:52788) 2024-09-19T09:19:05.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.743+0000 7f9380d6f640 1 -- 172.21.15.120:0/3199739241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba1e1365d0 msgr2=0x55ba1e140da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:05.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.743+0000 7f9380d6f640 1 --2- 172.21.15.120:0/3199739241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba1e1365d0 0x55ba1e140da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.743+0000 7f9380d6f640 1 -- 172.21.15.120:0/3199739241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1e141370 msgr2=0x55ba1e13e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:05.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.743+0000 7f9380d6f640 1 --2- 172.21.15.120:0/3199739241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1e141370 0x55ba1e13e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.743+0000 7f9380d6f640 1 -- 172.21.15.120:0/3199739241 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba1de0d3c0 con 0x55ba1e137b40 2024-09-19T09:19:05.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.743+0000 7f9380d6f640 1 --2- 172.21.15.120:0/3199739241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1e137b40 0x55ba1e13e050 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f936c013710 tx=0x7f936c013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729754f39038e6ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:05.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.743+0000 7f9364ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:19:05.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.743+0000 7f9364ff9640 1 -- 172.21.15.120:0/3199739241 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c0119c0 con 0x55ba1e137b40 2024-09-19T09:19:05.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.743+0000 7f9364ff9640 1 -- 172.21.15.120:0/3199739241 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f936c011b60 con 0x55ba1e137b40 2024-09-19T09:19:05.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.743+0000 7f9385bad740 1 -- 172.21.15.120:0/3199739241 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93740046b0 con 0x55ba1e137b40 2024-09-19T09:19:05.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.743+0000 7f9364ff9640 1 -- 172.21.15.120:0/3199739241 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f936c011d60 con 0x55ba1e137b40 2024-09-19T09:19:05.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.743+0000 7f9385bad740 10 client.20985 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:05.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.743+0000 7f9385bad740 1 -- 172.21.15.120:0/3199739241 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ba1e14fb20 con 0x55ba1e137b40 2024-09-19T09:19:05.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.743+0000 7f9364ff9640 1 -- 172.21.15.120:0/3199739241 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93740046b0 con 0x55ba1e137b40 2024-09-19T09:19:05.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.744+0000 7f9364ff9640 1 -- 172.21.15.120:0/3199739241 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f936c01a070 con 0x55ba1e137b40 2024-09-19T09:19:05.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.744+0000 7f9385bad740 10 client.20985 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:05.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.744+0000 7f9385bad740 10 client.20985 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:19:05.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.744+0000 7f9385bad740 20 client.20985 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:05.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.744+0000 7f9385bad740 1 --2- 172.21.15.120:0/3199739241 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ba1e150fd0 0x55ba1e153410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:05.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.744+0000 7f9385bad740 4 client.20985 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:19:05.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.744+0000 7f9385bad740 1 -- 172.21.15.120:0/3199739241 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba1e0a9860 con 0x55ba1e150fd0 2024-09-19T09:19:05.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.744+0000 7f9381570640 1 --2- 172.21.15.120:0/3199739241 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ba1e150fd0 0x55ba1e153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:05.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.744+0000 7f9381570640 1 --2- 172.21.15.120:0/3199739241 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ba1e150fd0 0x55ba1e153410 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d39b3ff39cb8f0d server_cookie=3d12d2d3c9542f3a in_seq=0 out_seq=0 2024-09-19T09:19:05.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.744+0000 7f9364ff9640 10 client.20985 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:05.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.747+0000 7f9364ff9640 1 -- 172.21.15.120:0/3199739241 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ba1e0a9860 con 0x55ba1e150fd0 2024-09-19T09:19:05.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.747+0000 7f9364ff9640 10 client.20985 handle_command_reply: tid=0 2024-09-19T09:19:05.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.747+0000 7f9385bad740 1 client.20985 shutdown 2024-09-19T09:19:05.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.747+0000 7f9385bad740 1 -- 172.21.15.120:0/3199739241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1e137b40 msgr2=0x55ba1e13e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:05.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.747+0000 7f9385bad740 1 --2- 172.21.15.120:0/3199739241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1e137b40 0x55ba1e13e050 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f936c013710 tx=0x7f936c013bd0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.748+0000 7f9385bad740 1 -- 172.21.15.120:0/3199739241 shutdown_connections 2024-09-19T09:19:05.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.748+0000 7f9385bad740 1 --2- 172.21.15.120:0/3199739241 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ba1e150fd0 0x55ba1e153410 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.748+0000 7f9385bad740 1 --2- 172.21.15.120:0/3199739241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1e141370 0x55ba1e13e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.748+0000 7f9364ff9640 0 client.20985 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:05.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.748+0000 7f9385bad740 1 --2- 172.21.15.120:0/3199739241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1e137b40 0x55ba1e13e050 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.748+0000 7f9385bad740 1 --2- 172.21.15.120:0/3199739241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba1e1365d0 0x55ba1e140da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.748+0000 7f9385bad740 1 -- 172.21.15.120:0/3199739241 >> 172.21.15.120:0/3199739241 conn(0x55ba1e0ff310 msgr2=0x55ba1e138930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:05.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.748+0000 7f9385bad740 1 -- 172.21.15.120:0/3199739241 shutdown_connections 2024-09-19T09:19:05.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.748+0000 7f9385bad740 1 -- 172.21.15.120:0/3199739241 wait complete. 2024-09-19T09:19:05.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.748+0000 7f9385bad740 20 client.20985 trim_cache size 0 max 16384 2024-09-19T09:19:05.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.761+0000 7f9385bad740 1 Processor -- start 2024-09-19T09:19:05.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.761+0000 7f9385bad740 1 -- start start 2024-09-19T09:19:05.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.762+0000 7f9385bad740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1e1365d0 0x55ba1e131c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:05.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.762+0000 7f9385bad740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1e137b40 0x55ba1e139120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:05.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.762+0000 7f9385bad740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba1e141370 0x55ba1e139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:05.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.762+0000 7f9385bad740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f937c1af8f0 con 0x55ba1e137b40 2024-09-19T09:19:05.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.762+0000 7f9385bad740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f937c1b1430 con 0x55ba1e1365d0 2024-09-19T09:19:05.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.762+0000 7f9385bad740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f937c1affc0 con 0x55ba1e141370 2024-09-19T09:19:05.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.762+0000 7f9380d6f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1e137b40 0x55ba1e139120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:05.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.762+0000 7f9381570640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1e1365d0 0x55ba1e131c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:05.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.762+0000 7f9380d6f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1e137b40 0x55ba1e139120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48480/0 (socket says 172.21.15.120:48480) 2024-09-19T09:19:05.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.762+0000 7f9380d6f640 1 -- 172.21.15.120:0/1721887277 learned_addr learned my addr 172.21.15.120:0/1721887277 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:05.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.762+0000 7f9381570640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1e1365d0 0x55ba1e131c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:43350/0 (socket says 172.21.15.120:43350) 2024-09-19T09:19:05.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.762+0000 7f9381d71640 1 --2- 172.21.15.120:0/1721887277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba1e141370 0x55ba1e139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:05.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.762+0000 7f9380d6f640 1 -- 172.21.15.120:0/1721887277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba1e141370 msgr2=0x55ba1e139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:05.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.762+0000 7f9380d6f640 1 --2- 172.21.15.120:0/1721887277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba1e141370 0x55ba1e139660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.762+0000 7f9380d6f640 1 -- 172.21.15.120:0/1721887277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1e1365d0 msgr2=0x55ba1e131c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:05.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.762+0000 7f9380d6f640 1 --2- 172.21.15.120:0/1721887277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1e1365d0 0x55ba1e131c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.762+0000 7f9380d6f640 1 -- 172.21.15.120:0/1721887277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba1e13eb50 con 0x55ba1e137b40 2024-09-19T09:19:05.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.762+0000 7f9381570640 1 --2- 172.21.15.120:0/1721887277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1e1365d0 0x55ba1e131c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:05.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.762+0000 7f9380d6f640 1 --2- 172.21.15.120:0/1721887277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1e137b40 0x55ba1e139120 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f936c01c970 tx=0x7f936c011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75fbaad6632ce6c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:05.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.762+0000 7f9366ffd640 1 -- 172.21.15.120:0/1721887277 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c005c50 con 0x55ba1e137b40 2024-09-19T09:19:05.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.763+0000 7f9366ffd640 1 -- 172.21.15.120:0/1721887277 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f936c0119c0 con 0x55ba1e137b40 2024-09-19T09:19:05.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.763+0000 7f9366ffd640 1 -- 172.21.15.120:0/1721887277 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c003ad0 con 0x55ba1e137b40 2024-09-19T09:19:05.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.764+0000 7f9385bad740 1 -- 172.21.15.120:0/1721887277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1e137b40 msgr2=0x55ba1e139120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:05.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.764+0000 7f9385bad740 1 --2- 172.21.15.120:0/1721887277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1e137b40 0x55ba1e139120 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f936c01c970 tx=0x7f936c011f00 comp rx=0 tx=0).stop 2024-09-19T09:19:05.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.764+0000 7f9385bad740 1 -- 172.21.15.120:0/1721887277 shutdown_connections 2024-09-19T09:19:05.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.764+0000 7f9385bad740 1 --2- 172.21.15.120:0/1721887277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba1e141370 0x55ba1e139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.764+0000 7f9385bad740 1 --2- 172.21.15.120:0/1721887277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1e137b40 0x55ba1e139120 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.764+0000 7f9385bad740 1 --2- 172.21.15.120:0/1721887277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1e1365d0 0x55ba1e131c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.764+0000 7f9385bad740 1 -- 172.21.15.120:0/1721887277 >> 172.21.15.120:0/1721887277 conn(0x55ba1e0ff310 msgr2=0x55ba1e131500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:05.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.764+0000 7f9385bad740 1 -- 172.21.15.120:0/1721887277 shutdown_connections 2024-09-19T09:19:05.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.764+0000 7f9385bad740 1 -- 172.21.15.120:0/1721887277 wait complete. 2024-09-19T09:19:05.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.764+0000 7f9385bad740 1 -- start start 2024-09-19T09:19:05.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.764+0000 7f9385bad740 1 Processor -- start 2024-09-19T09:19:05.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.765+0000 7f9385bad740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba1e1365d0 0x55ba1e13b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:05.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.765+0000 7f9385bad740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1e137b40 0x55ba1e13b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:05.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.765+0000 7f9385bad740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1e141370 0x55ba1e13bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:05.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.765+0000 7f9385bad740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f937c1b0d60 con 0x55ba1e137b40 2024-09-19T09:19:05.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.765+0000 7f9385bad740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f937c1b0690 con 0x55ba1e141370 2024-09-19T09:19:05.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.765+0000 7f9385bad740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f937c1ad8b0 con 0x55ba1e1365d0 2024-09-19T09:19:05.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.765+0000 7f9385bad740 10 client.0 _ll_register_callbacks cb 0x7f936c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:05.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.765+0000 7f9381d71640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1e141370 0x55ba1e13bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:05.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.765+0000 7f9381d71640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1e141370 0x55ba1e13bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:43354/0 (socket says 172.21.15.120:43354) 2024-09-19T09:19:05.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.765+0000 7f9381d71640 1 -- 172.21.15.120:0/831148326 learned_addr learned my addr 172.21.15.120:0/831148326 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:05.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.765+0000 7f9380d6f640 1 --2- 172.21.15.120:0/831148326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1e137b40 0x55ba1e13b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:05.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.765+0000 7f9381d71640 1 -- 172.21.15.120:0/831148326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba1e1365d0 msgr2=0x55ba1e13b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:05.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.765+0000 7f9381570640 1 --2- 172.21.15.120:0/831148326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba1e1365d0 0x55ba1e13b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:05.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.765+0000 7f9381d71640 1 --2- 172.21.15.120:0/831148326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba1e1365d0 0x55ba1e13b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.765+0000 7f9381d71640 1 -- 172.21.15.120:0/831148326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1e137b40 msgr2=0x55ba1e13b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:05.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.765+0000 7f9381d71640 1 --2- 172.21.15.120:0/831148326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1e137b40 0x55ba1e13b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.765+0000 7f9381d71640 1 -- 172.21.15.120:0/831148326 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba1e1bae10 con 0x55ba1e141370 2024-09-19T09:19:05.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.766+0000 7f9381d71640 1 --2- 172.21.15.120:0/831148326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1e141370 0x55ba1e13bcb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9374011790 tx=0x7f9374005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e34b8b515c2538b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:05.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.766+0000 7f9364ff9640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:19:05.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.766+0000 7f9364ff9640 1 -- 172.21.15.120:0/831148326 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937402b040 con 0x55ba1e141370 2024-09-19T09:19:05.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.766+0000 7f9364ff9640 1 -- 172.21.15.120:0/831148326 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f9374004a00 con 0x55ba1e141370 2024-09-19T09:19:05.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.766+0000 7f9385bad740 1 -- 172.21.15.120:0/831148326 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f936c0119c0 con 0x55ba1e141370 2024-09-19T09:19:05.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.767+0000 7f9364ff9640 1 -- 172.21.15.120:0/831148326 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9374003970 con 0x55ba1e141370 2024-09-19T09:19:05.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.767+0000 7f9385bad740 10 client.29852 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:05.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.767+0000 7f9385bad740 1 -- 172.21.15.120:0/831148326 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ba1e1bc0d0 con 0x55ba1e141370 2024-09-19T09:19:05.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.767+0000 7f9364ff9640 1 -- 172.21.15.120:0/831148326 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f936c0119c0 con 0x55ba1e141370 2024-09-19T09:19:05.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.767+0000 7f9364ff9640 1 -- 172.21.15.120:0/831148326 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f9374018070 con 0x55ba1e141370 2024-09-19T09:19:05.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.767+0000 7f9385bad740 10 client.29852 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:05.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.767+0000 7f9385bad740 10 client.29852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:19:05.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.767+0000 7f9385bad740 20 client.29852 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:05.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.767+0000 7f9385bad740 1 --2- 172.21.15.120:0/831148326 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ba1e1bc670 0x55ba1e1beab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:05.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.767+0000 7f9385bad740 4 client.29852 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:19:05.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.767+0000 7f9385bad740 1 -- 172.21.15.120:0/831148326 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ba1e0a3d70 con 0x55ba1e1bc670 2024-09-19T09:19:05.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.768+0000 7f9381570640 1 --2- 172.21.15.120:0/831148326 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ba1e1bc670 0x55ba1e1beab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:05.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.768+0000 7f9381570640 1 --2- 172.21.15.120:0/831148326 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ba1e1bc670 0x55ba1e1beab0 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43d5a9ac71979d8b server_cookie=6f04940bef16a997 in_seq=0 out_seq=0 2024-09-19T09:19:05.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.768+0000 7f9364ff9640 10 client.29852 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:05.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.771+0000 7f9364ff9640 1 -- 172.21.15.120:0/831148326 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ba1e0a3d70 con 0x55ba1e1bc670 2024-09-19T09:19:05.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.771+0000 7f9364ff9640 10 client.29852 handle_command_reply: tid=0 2024-09-19T09:19:05.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.771+0000 7f9385bad740 1 client.29852 shutdown 2024-09-19T09:19:05.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.771+0000 7f9385bad740 1 -- 172.21.15.120:0/831148326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1e141370 msgr2=0x55ba1e13bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:05.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.771+0000 7f9385bad740 1 --2- 172.21.15.120:0/831148326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1e141370 0x55ba1e13bcb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9374011790 tx=0x7f9374005bd0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.772+0000 7f9385bad740 1 -- 172.21.15.120:0/831148326 shutdown_connections 2024-09-19T09:19:05.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.772+0000 7f9385bad740 1 --2- 172.21.15.120:0/831148326 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ba1e1bc670 0x55ba1e1beab0 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.772+0000 7f9385bad740 1 --2- 172.21.15.120:0/831148326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1e141370 0x55ba1e13bcb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.772+0000 7f9385bad740 1 --2- 172.21.15.120:0/831148326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1e137b40 0x55ba1e13b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.772+0000 7f9385bad740 1 --2- 172.21.15.120:0/831148326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba1e1365d0 0x55ba1e13b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.772+0000 7f9385bad740 1 -- 172.21.15.120:0/831148326 >> 172.21.15.120:0/831148326 conn(0x55ba1e0ff310 msgr2=0x55ba1e1400d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:05.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.772+0000 7f9364ff9640 0 client.29852 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:05.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.772+0000 7f9385bad740 1 -- 172.21.15.120:0/831148326 shutdown_connections 2024-09-19T09:19:05.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.772+0000 7f9385bad740 1 -- 172.21.15.120:0/831148326 wait complete. 2024-09-19T09:19:05.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.772+0000 7f9385bad740 20 client.29852 trim_cache size 0 max 16384 2024-09-19T09:19:05.779 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:19:05.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.773+0000 7f93837d4640 1 -- 172.21.15.120:0/521004027 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93680a3240 msgr2=0x7f93680a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:05.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.773+0000 7f93837d4640 1 --2- 172.21.15.120:0/521004027 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93680a3240 0x7f93680a5700 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f937401f000 tx=0x7f9374020000 comp rx=0 tx=0).stop 2024-09-19T09:19:05.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.773+0000 7f93837d4640 1 -- 172.21.15.120:0/521004027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f937c102d30 msgr2=0x7f937c100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:05.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.773+0000 7f93837d4640 1 --2- 172.21.15.120:0/521004027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f937c102d30 0x7f937c100990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9378009230 tx=0x7f9378004040 comp rx=0 tx=0).stop 2024-09-19T09:19:05.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.773+0000 7f93837d4640 1 -- 172.21.15.120:0/521004027 shutdown_connections 2024-09-19T09:19:05.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.773+0000 7f93837d4640 1 --2- 172.21.15.120:0/521004027 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93680a3240 0x7f93680a5700 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.773+0000 7f93837d4640 1 --2- 172.21.15.120:0/521004027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f937c1082a0 0x7f937c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.773+0000 7f93837d4640 1 --2- 172.21.15.120:0/521004027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f937c1036e0 0x7f937c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.773+0000 7f93837d4640 1 --2- 172.21.15.120:0/521004027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f937c102d30 0x7f937c100990 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.773+0000 7f93837d4640 1 -- 172.21.15.120:0/521004027 >> 172.21.15.120:0/521004027 conn(0x7f937c0fc7d0 msgr2=0x7f937c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:05.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.773+0000 7f93837d4640 1 -- 172.21.15.120:0/521004027 shutdown_connections 2024-09-19T09:19:05.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:05.773+0000 7f93837d4640 1 -- 172.21.15.120:0/521004027 wait complete. 2024-09-19T09:19:06.361 INFO:tasks.workunit.client.0.smithi120.stdout: 10 61191 4.76 MB/sec execute 61 sec latency 61.077 ms 2024-09-19T09:19:07.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:06 smithi120 ceph-mon[31150]: pgmap v832: 97 pgs: 97 active+clean; 281 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.6 MiB/s wr, 155 op/s 2024-09-19T09:19:07.362 INFO:tasks.workunit.client.0.smithi120.stdout: 10 61843 5.08 MB/sec execute 62 sec latency 57.791 ms 2024-09-19T09:19:07.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:06 smithi196 ceph-mon[35150]: pgmap v832: 97 pgs: 97 active+clean; 281 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.6 MiB/s wr, 155 op/s 2024-09-19T09:19:07.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:06 smithi153 ceph-mon[35071]: pgmap v832: 97 pgs: 97 active+clean; 281 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.6 MiB/s wr, 155 op/s 2024-09-19T09:19:07.641 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:19:07.641 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:19:07.930 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:19:08.361 INFO:tasks.workunit.client.0.smithi120.stdout: 10 62508 5.69 MB/sec execute 63 sec latency 61.089 ms 2024-09-19T09:19:08.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.476+0000 7f4f6c7a2640 1 -- 172.21.15.120:0/867067144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f64106a80 msgr2=0x7f4f64106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:08.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.476+0000 7f4f6c7a2640 1 --2- 172.21.15.120:0/867067144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f64106a80 0x7f4f64106e60 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f4f60009930 tx=0x7f4f6002a420 comp rx=0 tx=0).stop 2024-09-19T09:19:08.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.476+0000 7f4f6c7a2640 1 -- 172.21.15.120:0/867067144 shutdown_connections 2024-09-19T09:19:08.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.476+0000 7f4f6c7a2640 1 --2- 172.21.15.120:0/867067144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f64101420 0x7f4f6410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.476+0000 7f4f6c7a2640 1 --2- 172.21.15.120:0/867067144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f64100a80 0x7f4f64100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.476+0000 7f4f6c7a2640 1 --2- 172.21.15.120:0/867067144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f64106a80 0x7f4f64106e60 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.476+0000 7f4f6c7a2640 1 -- 172.21.15.120:0/867067144 >> 172.21.15.120:0/867067144 conn(0x7f4f640fc7a0 msgr2=0x7f4f640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:08.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.476+0000 7f4f6c7a2640 1 -- 172.21.15.120:0/867067144 shutdown_connections 2024-09-19T09:19:08.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.476+0000 7f4f6c7a2640 1 -- 172.21.15.120:0/867067144 wait complete. 2024-09-19T09:19:08.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.477+0000 7f4f6c7a2640 1 Processor -- start 2024-09-19T09:19:08.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.477+0000 7f4f6c7a2640 1 -- start start 2024-09-19T09:19:08.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.477+0000 7f4f6c7a2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f64100a80 0x7f4f640fce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:08.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.477+0000 7f4f6c7a2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f64101420 0x7f4f640fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:08.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.477+0000 7f4f6c7a2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f64106a80 0x7f4f640fd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:08.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.477+0000 7f4f6c7a2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4f64110e80 con 0x7f4f64100a80 2024-09-19T09:19:08.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.477+0000 7f4f6c7a2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4f64110d00 con 0x7f4f64101420 2024-09-19T09:19:08.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.477+0000 7f4f6c7a2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f64111000 con 0x7f4f64106a80 2024-09-19T09:19:08.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.477+0000 7f4f6a53e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f64100a80 0x7f4f640fce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:08.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.477+0000 7f4f69d3d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f64101420 0x7f4f640fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:08.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.477+0000 7f4f69d3d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f64101420 0x7f4f640fd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:43370/0 (socket says 172.21.15.120:43370) 2024-09-19T09:19:08.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.477+0000 7f4f69d3d640 1 -- 172.21.15.120:0/2423162066 learned_addr learned my addr 172.21.15.120:0/2423162066 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:08.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.477+0000 7f4f6ad3f640 1 --2- 172.21.15.120:0/2423162066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f64106a80 0x7f4f640fd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:08.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.477+0000 7f4f6a53e640 1 -- 172.21.15.120:0/2423162066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f64106a80 msgr2=0x7f4f640fd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:08.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.477+0000 7f4f6a53e640 1 --2- 172.21.15.120:0/2423162066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f64106a80 0x7f4f640fd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.477+0000 7f4f6a53e640 1 -- 172.21.15.120:0/2423162066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f64101420 msgr2=0x7f4f640fd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:08.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.477+0000 7f4f6a53e640 1 --2- 172.21.15.120:0/2423162066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f64101420 0x7f4f640fd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.477+0000 7f4f6a53e640 1 -- 172.21.15.120:0/2423162066 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f641a68a0 con 0x7f4f64100a80 2024-09-19T09:19:08.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.477+0000 7f4f69d3d640 1 --2- 172.21.15.120:0/2423162066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f64101420 0x7f4f640fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:08.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.477+0000 7f4f6ad3f640 1 --2- 172.21.15.120:0/2423162066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f64106a80 0x7f4f640fd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:08.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.478+0000 7f4f6a53e640 1 --2- 172.21.15.120:0/2423162066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f64100a80 0x7f4f640fce90 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f4f60009230 tx=0x7f4f60031b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:08.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.478+0000 7f4f5b7fe640 1 -- 172.21.15.120:0/2423162066 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6003f030 con 0x7f4f64100a80 2024-09-19T09:19:08.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.478+0000 7f4f5b7fe640 1 -- 172.21.15.120:0/2423162066 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f60004490 con 0x7f4f64100a80 2024-09-19T09:19:08.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.478+0000 7f4f6c7a2640 1 -- 172.21.15.120:0/2423162066 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f641a6b30 con 0x7f4f64100a80 2024-09-19T09:19:08.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.478+0000 7f4f5b7fe640 1 -- 172.21.15.120:0/2423162066 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f60035070 con 0x7f4f64100a80 2024-09-19T09:19:08.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.478+0000 7f4f6c7a2640 1 -- 172.21.15.120:0/2423162066 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f641a7030 con 0x7f4f64100a80 2024-09-19T09:19:08.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.481+0000 7f4f5b7fe640 1 -- 172.21.15.120:0/2423162066 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f4f60042030 con 0x7f4f64100a80 2024-09-19T09:19:08.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.481+0000 7f4f6eb7b740 1 Processor -- start 2024-09-19T09:19:08.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.481+0000 7f4f6eb7b740 1 -- start start 2024-09-19T09:19:08.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.482+0000 7f4f6eb7b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55594a2f2b60 0x55594a2f2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:08.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.482+0000 7f4f6eb7b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55594a2f3480 0x55594a2fd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:08.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.482+0000 7f4f6eb7b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55594a2fdd40 0x55594a300130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:08.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.482+0000 7f4f6eb7b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55594a258280 con 0x55594a2f3480 2024-09-19T09:19:08.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.482+0000 7f4f6eb7b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55594a1ffb10 con 0x55594a2fdd40 2024-09-19T09:19:08.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.482+0000 7f4f6eb7b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55594a254240 con 0x55594a2f2b60 2024-09-19T09:19:08.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.482+0000 7f4f5b7fe640 1 --2- 172.21.15.120:0/2423162066 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4f500a32c0 0x7f4f500a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:08.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.482+0000 7f4f6ad3f640 1 --2- 172.21.15.120:0/2423162066 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4f500a32c0 0x7f4f500a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:08.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.482+0000 7f4f69d3d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55594a2fdd40 0x55594a300130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:08.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.482+0000 7f4f6ad3f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55594a2f2b60 0x55594a2f2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:08.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.482+0000 7f4f69d3d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55594a2fdd40 0x55594a300130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:43374/0 (socket says 172.21.15.120:43374) 2024-09-19T09:19:08.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.482+0000 7f4f6ad3f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55594a2f2b60 0x55594a2f2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:52832/0 (socket says 172.21.15.120:52832) 2024-09-19T09:19:08.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.482+0000 7f4f69d3d640 1 -- 172.21.15.120:0/177962466 learned_addr learned my addr 172.21.15.120:0/177962466 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:08.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.482+0000 7f4f5b7fe640 1 -- 172.21.15.120:0/2423162066 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f4f600efb10 con 0x7f4f64100a80 2024-09-19T09:19:08.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.482+0000 7f4f6a53e640 1 --2- 172.21.15.120:0/177962466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55594a2f3480 0x55594a2fd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:08.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.482+0000 7f4f69d3d640 1 -- 172.21.15.120:0/177962466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55594a2f2b60 msgr2=0x55594a2f2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:08.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.482+0000 7f4f69d3d640 1 --2- 172.21.15.120:0/177962466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55594a2f2b60 0x55594a2f2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.482+0000 7f4f69d3d640 1 -- 172.21.15.120:0/177962466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55594a2f3480 msgr2=0x55594a2fd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:08.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.482+0000 7f4f69d3d640 1 --2- 172.21.15.120:0/177962466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55594a2f3480 0x55594a2fd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.482+0000 7f4f69d3d640 1 -- 172.21.15.120:0/177962466 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55594a20c1f0 con 0x55594a2fdd40 2024-09-19T09:19:08.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.483+0000 7f4f6ad3f640 1 --2- 172.21.15.120:0/2423162066 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4f500a32c0 0x7f4f500a5780 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c009fd0 tx=0x7f4f5c0116b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:08.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.483+0000 7f4f69d3d640 1 --2- 172.21.15.120:0/177962466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55594a2fdd40 0x55594a300130 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4f540115f0 tx=0x7f4f54011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e1f06899551e909 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:08.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.483+0000 7f4f58ff9640 1 -- 172.21.15.120:0/177962466 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f54004020 con 0x55594a2fdd40 2024-09-19T09:19:08.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.483+0000 7f4f58ff9640 1 -- 172.21.15.120:0/177962466 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f540041c0 con 0x55594a2fdd40 2024-09-19T09:19:08.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.483+0000 7f4f58ff9640 1 -- 172.21.15.120:0/177962466 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f54003a10 con 0x55594a2fdd40 2024-09-19T09:19:08.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.484+0000 7f4f6eb7b740 1 -- 172.21.15.120:0/177962466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55594a2fdd40 msgr2=0x55594a300130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:08.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.484+0000 7f4f6eb7b740 1 --2- 172.21.15.120:0/177962466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55594a2fdd40 0x55594a300130 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4f540115f0 tx=0x7f4f54011ab0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.484+0000 7f4f6eb7b740 1 -- 172.21.15.120:0/177962466 shutdown_connections 2024-09-19T09:19:08.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.484+0000 7f4f6eb7b740 1 --2- 172.21.15.120:0/177962466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55594a2fdd40 0x55594a300130 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.484+0000 7f4f6eb7b740 1 --2- 172.21.15.120:0/177962466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55594a2f3480 0x55594a2fd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.484+0000 7f4f6eb7b740 1 --2- 172.21.15.120:0/177962466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55594a2f2b60 0x55594a2f2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.484+0000 7f4f6eb7b740 1 -- 172.21.15.120:0/177962466 >> 172.21.15.120:0/177962466 conn(0x55594a2ef0f0 msgr2=0x55594a2ef4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:08.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.484+0000 7f4f6eb7b740 1 -- 172.21.15.120:0/177962466 shutdown_connections 2024-09-19T09:19:08.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.484+0000 7f4f6eb7b740 1 -- 172.21.15.120:0/177962466 wait complete. 2024-09-19T09:19:08.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.484+0000 7f4f6eb7b740 1 -- start start 2024-09-19T09:19:08.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.484+0000 7f4f6eb7b740 1 Processor -- start 2024-09-19T09:19:08.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.485+0000 7f4f6eb7b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55594a2f2b60 0x55594a2fab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:08.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.485+0000 7f4f6eb7b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55594a2f3480 0x55594a2fb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:08.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.485+0000 7f4f6eb7b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55594a2fdd40 0x55594a2fb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:08.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.485+0000 7f4f6eb7b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55594a300910 con 0x55594a2f2b60 2024-09-19T09:19:08.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.485+0000 7f4f6eb7b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55594a212f10 con 0x55594a2fdd40 2024-09-19T09:19:08.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.485+0000 7f4f6eb7b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55594a300a90 con 0x55594a2f3480 2024-09-19T09:19:08.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.485+0000 7f4f6eb7b740 10 client.0 _ll_register_callbacks cb 0x55594a1dba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:08.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.485+0000 7f4f69d3d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55594a2f3480 0x55594a2fb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:08.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.485+0000 7f4f6ad3f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55594a2fdd40 0x55594a2fb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:08.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.485+0000 7f4f69d3d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55594a2f3480 0x55594a2fb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:52842/0 (socket says 172.21.15.120:52842) 2024-09-19T09:19:08.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.485+0000 7f4f69d3d640 1 -- 172.21.15.120:0/1369414799 learned_addr learned my addr 172.21.15.120:0/1369414799 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:08.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.485+0000 7f4f6a53e640 1 --2- 172.21.15.120:0/1369414799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55594a2f2b60 0x55594a2fab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:08.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.485+0000 7f4f69d3d640 1 -- 172.21.15.120:0/1369414799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55594a2fdd40 msgr2=0x55594a2fb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:08.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.485+0000 7f4f69d3d640 1 --2- 172.21.15.120:0/1369414799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55594a2fdd40 0x55594a2fb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.485+0000 7f4f69d3d640 1 -- 172.21.15.120:0/1369414799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55594a2f2b60 msgr2=0x55594a2fab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:08.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.485+0000 7f4f69d3d640 1 --2- 172.21.15.120:0/1369414799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55594a2f2b60 0x55594a2fab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.485+0000 7f4f69d3d640 1 -- 172.21.15.120:0/1369414799 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555949fc96a0 con 0x55594a2f3480 2024-09-19T09:19:08.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.485+0000 7f4f6a53e640 1 --2- 172.21.15.120:0/1369414799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55594a2f2b60 0x55594a2fab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:08.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.486+0000 7f4f69d3d640 1 --2- 172.21.15.120:0/1369414799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55594a2f3480 0x55594a2fb0c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4f540115c0 tx=0x7f4f540045a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfb4291d1b20ccdc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:08.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.486+0000 7f4f4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:19:08.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.486+0000 7f4f4dffb640 1 -- 172.21.15.120:0/1369414799 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f54027030 con 0x55594a2f3480 2024-09-19T09:19:08.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.486+0000 7f4f6eb7b740 1 -- 172.21.15.120:0/1369414799 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f540041c0 con 0x55594a2f3480 2024-09-19T09:19:08.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.486+0000 7f4f4dffb640 1 -- 172.21.15.120:0/1369414799 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f4f5401f040 con 0x55594a2f3480 2024-09-19T09:19:08.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.486+0000 7f4f4dffb640 1 -- 172.21.15.120:0/1369414799 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4f5400fa90 con 0x55594a2f3480 2024-09-19T09:19:08.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.486+0000 7f4f6eb7b740 10 client.29686 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:08.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.486+0000 7f4f6eb7b740 1 -- 172.21.15.120:0/1369414799 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55594a30c480 con 0x55594a2f3480 2024-09-19T09:19:08.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.486+0000 7f4f4dffb640 1 -- 172.21.15.120:0/1369414799 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f540041c0 con 0x55594a2f3480 2024-09-19T09:19:08.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.486+0000 7f4f4dffb640 1 -- 172.21.15.120:0/1369414799 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f4f54003790 con 0x55594a2f3480 2024-09-19T09:19:08.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.486+0000 7f4f6eb7b740 10 client.29686 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:08.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.486+0000 7f4f6eb7b740 10 client.29686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:19:08.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.486+0000 7f4f6eb7b740 20 client.29686 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:08.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.486+0000 7f4f6eb7b740 1 --2- 172.21.15.120:0/1369414799 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55594a30d830 0x55594a30fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:08.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.486+0000 7f4f6eb7b740 4 client.29686 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:19:08.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.486+0000 7f4f6eb7b740 1 -- 172.21.15.120:0/1369414799 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55594a2657f0 con 0x55594a30d830 2024-09-19T09:19:08.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.487+0000 7f4f6a53e640 1 --2- 172.21.15.120:0/1369414799 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55594a30d830 0x55594a30fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:08.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.487+0000 7f4f4dffb640 10 client.29686 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:08.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.487+0000 7f4f6a53e640 1 --2- 172.21.15.120:0/1369414799 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55594a30d830 0x55594a30fc20 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a85998b0830e009 server_cookie=89b7bcb8d4b35ce in_seq=0 out_seq=0 2024-09-19T09:19:08.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.489+0000 7f4f4dffb640 1 -- 172.21.15.120:0/1369414799 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55594a2657f0 con 0x55594a30d830 2024-09-19T09:19:08.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.489+0000 7f4f4dffb640 10 client.29686 handle_command_reply: tid=0 2024-09-19T09:19:08.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.489+0000 7f4f6eb7b740 1 client.29686 shutdown 2024-09-19T09:19:08.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.489+0000 7f4f6eb7b740 1 -- 172.21.15.120:0/1369414799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55594a2f3480 msgr2=0x55594a2fb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:08.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.489+0000 7f4f6eb7b740 1 --2- 172.21.15.120:0/1369414799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55594a2f3480 0x55594a2fb0c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4f540115c0 tx=0x7f4f540045a0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.489+0000 7f4f6eb7b740 1 -- 172.21.15.120:0/1369414799 shutdown_connections 2024-09-19T09:19:08.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.489+0000 7f4f6eb7b740 1 --2- 172.21.15.120:0/1369414799 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55594a30d830 0x55594a30fc20 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.489+0000 7f4f6eb7b740 1 --2- 172.21.15.120:0/1369414799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55594a2fdd40 0x55594a2fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.489+0000 7f4f4dffb640 0 client.29686 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:08.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.489+0000 7f4f6eb7b740 1 --2- 172.21.15.120:0/1369414799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55594a2f3480 0x55594a2fb0c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.489+0000 7f4f6eb7b740 1 --2- 172.21.15.120:0/1369414799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55594a2f2b60 0x55594a2fab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.489+0000 7f4f6eb7b740 1 -- 172.21.15.120:0/1369414799 >> 172.21.15.120:0/1369414799 conn(0x55594a2ef0f0 msgr2=0x55594a2f4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:08.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.489+0000 7f4f6eb7b740 1 -- 172.21.15.120:0/1369414799 shutdown_connections 2024-09-19T09:19:08.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.489+0000 7f4f6eb7b740 1 -- 172.21.15.120:0/1369414799 wait complete. 2024-09-19T09:19:08.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.489+0000 7f4f6eb7b740 20 client.29686 trim_cache size 0 max 16384 2024-09-19T09:19:08.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.507+0000 7f4f6eb7b740 1 Processor -- start 2024-09-19T09:19:08.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.507+0000 7f4f6eb7b740 1 -- start start 2024-09-19T09:19:08.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.508+0000 7f4f6eb7b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55594a2f2b60 0x55594a2fc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:08.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.508+0000 7f4f6eb7b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55594a2f3480 0x55594a315390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:08.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.508+0000 7f4f6eb7b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55594a2fdd40 0x55594a3158d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:08.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.508+0000 7f4f6eb7b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4f641af780 con 0x55594a2f3480 2024-09-19T09:19:08.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.508+0000 7f4f6eb7b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4f641b12c0 con 0x55594a2fdd40 2024-09-19T09:19:08.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.508+0000 7f4f6eb7b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f641afe50 con 0x55594a2f2b60 2024-09-19T09:19:08.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.508+0000 7f4f69d3d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55594a2f3480 0x55594a315390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:08.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.508+0000 7f4f69d3d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55594a2f3480 0x55594a315390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48536/0 (socket says 172.21.15.120:48536) 2024-09-19T09:19:08.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.508+0000 7f4f6a53e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55594a2f2b60 0x55594a2fc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:08.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.508+0000 7f4f69d3d640 1 -- 172.21.15.120:0/1945257307 learned_addr learned my addr 172.21.15.120:0/1945257307 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:08.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.508+0000 7f4f6ad3f640 1 --2- 172.21.15.120:0/1945257307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55594a2fdd40 0x55594a3158d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:08.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.508+0000 7f4f69d3d640 1 -- 172.21.15.120:0/1945257307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55594a2f2b60 msgr2=0x55594a2fc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:08.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.508+0000 7f4f69d3d640 1 --2- 172.21.15.120:0/1945257307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55594a2f2b60 0x55594a2fc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.508+0000 7f4f69d3d640 1 -- 172.21.15.120:0/1945257307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55594a2fdd40 msgr2=0x55594a3158d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:08.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.508+0000 7f4f69d3d640 1 --2- 172.21.15.120:0/1945257307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55594a2fdd40 0x55594a3158d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.508+0000 7f4f69d3d640 1 -- 172.21.15.120:0/1945257307 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55594a2eef20 con 0x55594a2f3480 2024-09-19T09:19:08.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.508+0000 7f4f6a53e640 1 --2- 172.21.15.120:0/1945257307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55594a2f2b60 0x55594a2fc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:08.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.508+0000 7f4f69d3d640 1 --2- 172.21.15.120:0/1945257307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55594a2f3480 0x55594a315390 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f4f54003bb0 tx=0x7f4f5400fdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a89564432d9c33d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:08.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.508+0000 7f4f4ffff640 1 -- 172.21.15.120:0/1945257307 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f54027070 con 0x55594a2f3480 2024-09-19T09:19:08.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.509+0000 7f4f4ffff640 1 -- 172.21.15.120:0/1945257307 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f54028040 con 0x55594a2f3480 2024-09-19T09:19:08.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.509+0000 7f4f4ffff640 1 -- 172.21.15.120:0/1945257307 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f54005260 con 0x55594a2f3480 2024-09-19T09:19:08.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.509+0000 7f4f6eb7b740 1 -- 172.21.15.120:0/1945257307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55594a2f3480 msgr2=0x55594a315390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:08.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.509+0000 7f4f6eb7b740 1 --2- 172.21.15.120:0/1945257307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55594a2f3480 0x55594a315390 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f4f54003bb0 tx=0x7f4f5400fdd0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.509+0000 7f4f6eb7b740 1 -- 172.21.15.120:0/1945257307 shutdown_connections 2024-09-19T09:19:08.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.509+0000 7f4f6eb7b740 1 --2- 172.21.15.120:0/1945257307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55594a2fdd40 0x55594a3158d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.509+0000 7f4f6eb7b740 1 --2- 172.21.15.120:0/1945257307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55594a2f3480 0x55594a315390 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.509+0000 7f4f6eb7b740 1 --2- 172.21.15.120:0/1945257307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55594a2f2b60 0x55594a2fc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.509+0000 7f4f6eb7b740 1 -- 172.21.15.120:0/1945257307 >> 172.21.15.120:0/1945257307 conn(0x55594a2ef0f0 msgr2=0x55594a2fc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:08.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.509+0000 7f4f6eb7b740 1 -- 172.21.15.120:0/1945257307 shutdown_connections 2024-09-19T09:19:08.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.509+0000 7f4f6eb7b740 1 -- 172.21.15.120:0/1945257307 wait complete. 2024-09-19T09:19:08.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.510+0000 7f4f6eb7b740 1 -- start start 2024-09-19T09:19:08.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.510+0000 7f4f6eb7b740 1 Processor -- start 2024-09-19T09:19:08.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.510+0000 7f4f6eb7b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55594a2f2b60 0x55594a305cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:08.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.510+0000 7f4f6eb7b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55594a2fdd40 0x55594a306230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:08.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.510+0000 7f4f6eb7b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55594a300ca0 0x55594a301120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:08.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.510+0000 7f4f6eb7b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4f641b0bf0 con 0x55594a300ca0 2024-09-19T09:19:08.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.510+0000 7f4f6eb7b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4f641b0520 con 0x55594a2fdd40 2024-09-19T09:19:08.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.510+0000 7f4f6eb7b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f641ad740 con 0x55594a2f2b60 2024-09-19T09:19:08.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.510+0000 7f4f6eb7b740 10 client.0 _ll_register_callbacks cb 0x7f4f60009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:08.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.510+0000 7f4f6a53e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55594a2f2b60 0x55594a305cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:08.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.510+0000 7f4f6a53e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55594a2f2b60 0x55594a305cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:52858/0 (socket says 172.21.15.120:52858) 2024-09-19T09:19:08.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.510+0000 7f4f6a53e640 1 -- 172.21.15.120:0/3643779436 learned_addr learned my addr 172.21.15.120:0/3643779436 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:08.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.510+0000 7f4f6ad3f640 1 --2- 172.21.15.120:0/3643779436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55594a300ca0 0x55594a301120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:08.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.510+0000 7f4f69d3d640 1 --2- 172.21.15.120:0/3643779436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55594a2fdd40 0x55594a306230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:08.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.510+0000 7f4f6a53e640 1 -- 172.21.15.120:0/3643779436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55594a2fdd40 msgr2=0x55594a306230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:08.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.510+0000 7f4f6a53e640 1 --2- 172.21.15.120:0/3643779436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55594a2fdd40 0x55594a306230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.510+0000 7f4f6a53e640 1 -- 172.21.15.120:0/3643779436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55594a300ca0 msgr2=0x55594a301120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:08.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.510+0000 7f4f6a53e640 1 --2- 172.21.15.120:0/3643779436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55594a300ca0 0x55594a301120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.511+0000 7f4f6a53e640 1 -- 172.21.15.120:0/3643779436 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55594a2efca0 con 0x55594a2f2b60 2024-09-19T09:19:08.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.511+0000 7f4f6a53e640 1 --2- 172.21.15.120:0/3643779436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55594a2f2b60 0x55594a305cf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4f60006010 tx=0x7f4f6003b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8226d22b7cc7541f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:08.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.511+0000 7f4f4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:19:08.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.511+0000 7f4f4dffb640 1 -- 172.21.15.120:0/3643779436 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f600c4d60 con 0x55594a2f2b60 2024-09-19T09:19:08.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.511+0000 7f4f4dffb640 1 -- 172.21.15.120:0/3643779436 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f4f600dc0b0 con 0x55594a2f2b60 2024-09-19T09:19:08.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.511+0000 7f4f6eb7b740 1 -- 172.21.15.120:0/3643779436 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f54028040 con 0x55594a2f2b60 2024-09-19T09:19:08.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.511+0000 7f4f4dffb640 1 -- 172.21.15.120:0/3643779436 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4f600bc0f0 con 0x55594a2f2b60 2024-09-19T09:19:08.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.511+0000 7f4f6eb7b740 10 client.29692 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:08.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.511+0000 7f4f6eb7b740 1 -- 172.21.15.120:0/3643779436 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55594a381650 con 0x55594a2f2b60 2024-09-19T09:19:08.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.511+0000 7f4f4dffb640 1 -- 172.21.15.120:0/3643779436 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f54028040 con 0x55594a2f2b60 2024-09-19T09:19:08.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.512+0000 7f4f4dffb640 1 -- 172.21.15.120:0/3643779436 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f4f600d6070 con 0x55594a2f2b60 2024-09-19T09:19:08.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.512+0000 7f4f6eb7b740 10 client.29692 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:08.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.512+0000 7f4f6eb7b740 10 client.29692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:19:08.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.512+0000 7f4f6eb7b740 20 client.29692 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:08.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.512+0000 7f4f6eb7b740 1 --2- 172.21.15.120:0/3643779436 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55594a381d20 0x55594a384110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:08.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.512+0000 7f4f6eb7b740 4 client.29692 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:19:08.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.512+0000 7f4f6eb7b740 1 -- 172.21.15.120:0/3643779436 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55594a25fd00 con 0x55594a381d20 2024-09-19T09:19:08.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.512+0000 7f4f69d3d640 1 --2- 172.21.15.120:0/3643779436 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55594a381d20 0x55594a384110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:08.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.512+0000 7f4f69d3d640 1 --2- 172.21.15.120:0/3643779436 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55594a381d20 0x55594a384110 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1342783894bbff3d server_cookie=484df6ad4bc525c5 in_seq=0 out_seq=0 2024-09-19T09:19:08.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.512+0000 7f4f4dffb640 10 client.29692 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:08.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.517+0000 7f4f4dffb640 1 -- 172.21.15.120:0/3643779436 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55594a25fd00 con 0x55594a381d20 2024-09-19T09:19:08.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.517+0000 7f4f4dffb640 10 client.29692 handle_command_reply: tid=0 2024-09-19T09:19:08.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.517+0000 7f4f6eb7b740 1 client.29692 shutdown 2024-09-19T09:19:08.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.517+0000 7f4f6eb7b740 1 -- 172.21.15.120:0/3643779436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55594a2f2b60 msgr2=0x55594a305cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:08.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.517+0000 7f4f6eb7b740 1 --2- 172.21.15.120:0/3643779436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55594a2f2b60 0x55594a305cf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4f60006010 tx=0x7f4f6003b460 comp rx=0 tx=0).stop 2024-09-19T09:19:08.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.517+0000 7f4f6eb7b740 1 -- 172.21.15.120:0/3643779436 shutdown_connections 2024-09-19T09:19:08.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.517+0000 7f4f6eb7b740 1 --2- 172.21.15.120:0/3643779436 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55594a381d20 0x55594a384110 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.517+0000 7f4f6eb7b740 1 --2- 172.21.15.120:0/3643779436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55594a300ca0 0x55594a301120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.517+0000 7f4f6eb7b740 1 --2- 172.21.15.120:0/3643779436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55594a2fdd40 0x55594a306230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.517+0000 7f4f6eb7b740 1 --2- 172.21.15.120:0/3643779436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55594a2f2b60 0x55594a305cf0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.517+0000 7f4f4dffb640 0 client.29692 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:08.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.517+0000 7f4f6eb7b740 1 -- 172.21.15.120:0/3643779436 >> 172.21.15.120:0/3643779436 conn(0x55594a2ef0f0 msgr2=0x55594a2fb360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:08.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.518+0000 7f4f6eb7b740 1 -- 172.21.15.120:0/3643779436 shutdown_connections 2024-09-19T09:19:08.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.518+0000 7f4f6eb7b740 1 -- 172.21.15.120:0/3643779436 wait complete. 2024-09-19T09:19:08.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.518+0000 7f4f6eb7b740 20 client.29692 trim_cache size 0 max 16384 2024-09-19T09:19:08.521 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:19:08.521 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:19:08.521 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "d9c22cf6-1191-4b92-904f-9848f28513ef", 2024-09-19T09:19:08.521 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:19:08.522 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:19:08.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.518+0000 7f4f6c7a2640 1 -- 172.21.15.120:0/2423162066 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4f500a32c0 msgr2=0x7f4f500a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:08.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.518+0000 7f4f6c7a2640 1 --2- 172.21.15.120:0/2423162066 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4f500a32c0 0x7f4f500a5780 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c009fd0 tx=0x7f4f5c0116b0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.518+0000 7f4f6c7a2640 1 -- 172.21.15.120:0/2423162066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f64100a80 msgr2=0x7f4f640fce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:08.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.518+0000 7f4f6c7a2640 1 --2- 172.21.15.120:0/2423162066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f64100a80 0x7f4f640fce90 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f4f60009230 tx=0x7f4f60031b50 comp rx=0 tx=0).stop 2024-09-19T09:19:08.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.518+0000 7f4f6c7a2640 1 -- 172.21.15.120:0/2423162066 shutdown_connections 2024-09-19T09:19:08.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.518+0000 7f4f6c7a2640 1 --2- 172.21.15.120:0/2423162066 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4f500a32c0 0x7f4f500a5780 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.518+0000 7f4f6c7a2640 1 --2- 172.21.15.120:0/2423162066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f64106a80 0x7f4f640fd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.518+0000 7f4f6c7a2640 1 --2- 172.21.15.120:0/2423162066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f64101420 0x7f4f640fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.518+0000 7f4f6c7a2640 1 --2- 172.21.15.120:0/2423162066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f64100a80 0x7f4f640fce90 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.518+0000 7f4f6c7a2640 1 -- 172.21.15.120:0/2423162066 >> 172.21.15.120:0/2423162066 conn(0x7f4f640fc7a0 msgr2=0x7f4f6406aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:08.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.519+0000 7f4f6c7a2640 1 -- 172.21.15.120:0/2423162066 shutdown_connections 2024-09-19T09:19:08.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:08.519+0000 7f4f6c7a2640 1 -- 172.21.15.120:0/2423162066 wait complete. 2024-09-19T09:19:09.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:08 smithi120 ceph-mon[31150]: pgmap v833: 97 pgs: 97 active+clean; 286 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.5 MiB/s wr, 276 op/s 2024-09-19T09:19:09.361 INFO:tasks.workunit.client.0.smithi120.stdout: 10 63271 6.20 MB/sec execute 64 sec latency 45.837 ms 2024-09-19T09:19:09.377 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:19:09.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:08 smithi196 ceph-mon[35150]: pgmap v833: 97 pgs: 97 active+clean; 286 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.5 MiB/s wr, 276 op/s 2024-09-19T09:19:09.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:08 smithi153 ceph-mon[35071]: pgmap v833: 97 pgs: 97 active+clean; 286 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.5 MiB/s wr, 276 op/s 2024-09-19T09:19:09.645 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:19:10.044 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:09 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:19:10.044 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:09 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:10.044 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:09 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:19:10.044 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:09 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:19:10.044 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:09 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:10.044 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:09 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:19:10.044 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:09 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:10.044 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:09 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:19:10.045 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:09 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:10.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.133+0000 7f6e5ced6640 1 -- 172.21.15.120:0/3291981316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e58105560 msgr2=0x7f6e5810d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:10.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.133+0000 7f6e5ced6640 1 --2- 172.21.15.120:0/3291981316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e58105560 0x7f6e5810d0e0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f6e50009ca0 tx=0x7f6e5002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.133+0000 7f6e5ced6640 1 -- 172.21.15.120:0/3291981316 shutdown_connections 2024-09-19T09:19:10.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.133+0000 7f6e5ced6640 1 --2- 172.21.15.120:0/3291981316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e58105560 0x7f6e5810d0e0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.133+0000 7f6e5ced6640 1 --2- 172.21.15.120:0/3291981316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e58100de0 0x7f6e58104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.133+0000 7f6e5ced6640 1 --2- 172.21.15.120:0/3291981316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e58100430 0x7f6e58100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.133+0000 7f6e5ced6640 1 -- 172.21.15.120:0/3291981316 >> 172.21.15.120:0/3291981316 conn(0x7f6e58077230 msgr2=0x7f6e580fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:10.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.133+0000 7f6e5ced6640 1 -- 172.21.15.120:0/3291981316 shutdown_connections 2024-09-19T09:19:10.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.133+0000 7f6e5ced6640 1 -- 172.21.15.120:0/3291981316 wait complete. 2024-09-19T09:19:10.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.134+0000 7f6e5ced6640 1 Processor -- start 2024-09-19T09:19:10.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.134+0000 7f6e5ced6640 1 -- start start 2024-09-19T09:19:10.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.134+0000 7f6e5ced6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e58100430 0x7f6e58199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:10.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.134+0000 7f6e5ced6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e58100de0 0x7f6e58199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:10.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.134+0000 7f6e5ced6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e58105560 0x7f6e581a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:10.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.134+0000 7f6e5ced6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6e58110e40 con 0x7f6e58100de0 2024-09-19T09:19:10.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.134+0000 7f6e5ced6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6e58110cc0 con 0x7f6e58105560 2024-09-19T09:19:10.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.134+0000 7f6e5ced6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6e58110fc0 con 0x7f6e58100430 2024-09-19T09:19:10.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.134+0000 7f6e5659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e58100430 0x7f6e58199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:10.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.134+0000 7f6e5659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e58100430 0x7f6e58199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:52882/0 (socket says 172.21.15.120:52882) 2024-09-19T09:19:10.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.134+0000 7f6e5659c640 1 -- 172.21.15.120:0/3584995312 learned_addr learned my addr 172.21.15.120:0/3584995312 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:10.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.134+0000 7f6e56d9d640 1 --2- 172.21.15.120:0/3584995312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e58105560 0x7f6e581a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:10.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.135+0000 7f6e55d9b640 1 --2- 172.21.15.120:0/3584995312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e58100de0 0x7f6e58199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:10.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.135+0000 7f6e5659c640 1 -- 172.21.15.120:0/3584995312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e58105560 msgr2=0x7f6e581a4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:10.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.135+0000 7f6e5659c640 1 --2- 172.21.15.120:0/3584995312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e58105560 0x7f6e581a4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.135+0000 7f6e5659c640 1 -- 172.21.15.120:0/3584995312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e58100de0 msgr2=0x7f6e58199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:10.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.135+0000 7f6e5659c640 1 --2- 172.21.15.120:0/3584995312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e58100de0 0x7f6e58199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.135+0000 7f6e5659c640 1 -- 172.21.15.120:0/3584995312 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e581a47d0 con 0x7f6e58100430 2024-09-19T09:19:10.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.135+0000 7f6e5659c640 1 --2- 172.21.15.120:0/3584995312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e58100430 0x7f6e58199210 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800e9f0 tx=0x7f6e4800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:10.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.135+0000 7f6e477fe640 1 -- 172.21.15.120:0/3584995312 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4800cd80 con 0x7f6e58100430 2024-09-19T09:19:10.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.135+0000 7f6e5ced6640 1 -- 172.21.15.120:0/3584995312 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e581a49a0 con 0x7f6e58100430 2024-09-19T09:19:10.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.135+0000 7f6e5ced6640 1 -- 172.21.15.120:0/3584995312 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e581a4eb0 con 0x7f6e58100430 2024-09-19T09:19:10.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.135+0000 7f6e477fe640 1 -- 172.21.15.120:0/3584995312 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e48005500 con 0x7f6e58100430 2024-09-19T09:19:10.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.135+0000 7f6e477fe640 1 -- 172.21.15.120:0/3584995312 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e48010690 con 0x7f6e58100430 2024-09-19T09:19:10.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.138+0000 7f6e477fe640 1 -- 172.21.15.120:0/3584995312 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f6e48010830 con 0x7f6e58100430 2024-09-19T09:19:10.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.138+0000 7f6e5f2af740 1 Processor -- start 2024-09-19T09:19:10.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.138+0000 7f6e5f2af740 1 -- start start 2024-09-19T09:19:10.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.139+0000 7f6e5f2af740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94502f4a0 0x55b94502f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:10.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.139+0000 7f6e5f2af740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94502fdc0 0x55b94503a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:10.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.139+0000 7f6e5f2af740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b94503a680 0x55b94503ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:10.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.139+0000 7f6e5f2af740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b944f952e0 con 0x55b94502f4a0 2024-09-19T09:19:10.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.139+0000 7f6e5f2af740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b944f3cce0 con 0x55b94502fdc0 2024-09-19T09:19:10.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.139+0000 7f6e5f2af740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b944f912a0 con 0x55b94503a680 2024-09-19T09:19:10.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.139+0000 7f6e477fe640 1 --2- 172.21.15.120:0/3584995312 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6e400a3240 0x7f6e400a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:10.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.139+0000 7f6e56d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94502f4a0 0x55b94502f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:10.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.139+0000 7f6e55d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b94503a680 0x55b94503ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:10.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.139+0000 7f6e56d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94502f4a0 0x55b94502f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48568/0 (socket says 172.21.15.120:48568) 2024-09-19T09:19:10.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.139+0000 7f6e56d9d640 1 -- 172.21.15.120:0/3732269293 learned_addr learned my addr 172.21.15.120:0/3732269293 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:10.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.139+0000 7f6e5659c640 1 --2- 172.21.15.120:0/3732269293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94502fdc0 0x55b94503a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:10.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.139+0000 7f6e55d9b640 1 -- 172.21.15.120:0/3732269293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94502fdc0 msgr2=0x55b94503a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:10.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.139+0000 7f6e55d9b640 1 --2- 172.21.15.120:0/3732269293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94502fdc0 0x55b94503a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.139+0000 7f6e55d9b640 1 -- 172.21.15.120:0/3732269293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94502f4a0 msgr2=0x55b94502f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:10.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.139+0000 7f6e55d9b640 1 --2- 172.21.15.120:0/3732269293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94502f4a0 0x55b94502f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.139+0000 7f6e55d9b640 1 -- 172.21.15.120:0/3732269293 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b944f21440 con 0x55b94503a680 2024-09-19T09:19:10.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.139+0000 7f6e5659c640 1 --2- 172.21.15.120:0/3732269293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94502fdc0 0x55b94503a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:19:10.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.139+0000 7f6e477fe640 1 -- 172.21.15.120:0/3584995312 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f6e480cfd70 con 0x7f6e58100430 2024-09-19T09:19:10.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.139+0000 7f6e56d9d640 1 --2- 172.21.15.120:0/3584995312 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6e400a3240 0x7f6e400a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:10.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.140+0000 7f6e55d9b640 1 --2- 172.21.15.120:0/3732269293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b94503a680 0x55b94503ca70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c013c80 tx=0x7f6e4c009900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71f694071f081c2a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:10.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.140+0000 7f6e44ff9640 1 -- 172.21.15.120:0/3732269293 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c004050 con 0x55b94503a680 2024-09-19T09:19:10.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.140+0000 7f6e44ff9640 1 -- 172.21.15.120:0/3732269293 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e4c0041f0 con 0x55b94503a680 2024-09-19T09:19:10.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.140+0000 7f6e44ff9640 1 -- 172.21.15.120:0/3732269293 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c0035f0 con 0x55b94503a680 2024-09-19T09:19:10.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.141+0000 7f6e5f2af740 1 -- 172.21.15.120:0/3732269293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b94503a680 msgr2=0x55b94503ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:10.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.141+0000 7f6e56d9d640 1 --2- 172.21.15.120:0/3584995312 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6e400a3240 0x7f6e400a5700 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6e5002f540 tx=0x7f6e5002a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:10.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.141+0000 7f6e5f2af740 1 --2- 172.21.15.120:0/3732269293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b94503a680 0x55b94503ca70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c013c80 tx=0x7f6e4c009900 comp rx=0 tx=0).stop 2024-09-19T09:19:10.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.141+0000 7f6e5f2af740 1 -- 172.21.15.120:0/3732269293 shutdown_connections 2024-09-19T09:19:10.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.141+0000 7f6e5f2af740 1 --2- 172.21.15.120:0/3732269293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b94503a680 0x55b94503ca70 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.141+0000 7f6e5f2af740 1 --2- 172.21.15.120:0/3732269293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94502fdc0 0x55b94503a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.141+0000 7f6e5f2af740 1 --2- 172.21.15.120:0/3732269293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94502f4a0 0x55b94502f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.141+0000 7f6e5f2af740 1 -- 172.21.15.120:0/3732269293 >> 172.21.15.120:0/3732269293 conn(0x55b94502ba30 msgr2=0x55b94502be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:10.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.141+0000 7f6e5f2af740 1 -- 172.21.15.120:0/3732269293 shutdown_connections 2024-09-19T09:19:10.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.141+0000 7f6e5f2af740 1 -- 172.21.15.120:0/3732269293 wait complete. 2024-09-19T09:19:10.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.142+0000 7f6e5f2af740 1 -- start start 2024-09-19T09:19:10.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.142+0000 7f6e5f2af740 1 Processor -- start 2024-09-19T09:19:10.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.142+0000 7f6e5f2af740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94502f4a0 0x55b945039f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:10.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.142+0000 7f6e5f2af740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b94502fdc0 0x55b945037200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:10.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.142+0000 7f6e5f2af740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94503a680 0x55b945037740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:10.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.142+0000 7f6e5f2af740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b94503d250 con 0x55b94502f4a0 2024-09-19T09:19:10.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.142+0000 7f6e5f2af740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b944f50050 con 0x55b94503a680 2024-09-19T09:19:10.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.142+0000 7f6e5f2af740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b94503d3d0 con 0x55b94502fdc0 2024-09-19T09:19:10.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.143+0000 7f6e5659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94502f4a0 0x55b945039f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:10.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.143+0000 7f6e5659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94502f4a0 0x55b945039f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48576/0 (socket says 172.21.15.120:48576) 2024-09-19T09:19:10.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.143+0000 7f6e5659c640 1 -- 172.21.15.120:0/2458188512 learned_addr learned my addr 172.21.15.120:0/2458188512 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:10.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.142+0000 7f6e5f2af740 10 client.0 _ll_register_callbacks cb 0x55b944f90160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:10.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.143+0000 7f6e56d9d640 1 --2- 172.21.15.120:0/2458188512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94503a680 0x55b945037740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:10.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.143+0000 7f6e55d9b640 1 --2- 172.21.15.120:0/2458188512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b94502fdc0 0x55b945037200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:10.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.143+0000 7f6e5659c640 1 -- 172.21.15.120:0/2458188512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b94502fdc0 msgr2=0x55b945037200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:10.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.143+0000 7f6e5659c640 1 --2- 172.21.15.120:0/2458188512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b94502fdc0 0x55b945037200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.143+0000 7f6e5659c640 1 -- 172.21.15.120:0/2458188512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94503a680 msgr2=0x55b945037740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:10.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.143+0000 7f6e5659c640 1 --2- 172.21.15.120:0/2458188512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94503a680 0x55b945037740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.143+0000 7f6e5659c640 1 -- 172.21.15.120:0/2458188512 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b944d063c0 con 0x55b94502f4a0 2024-09-19T09:19:10.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.143+0000 7f6e55d9b640 1 --2- 172.21.15.120:0/2458188512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b94502fdc0 0x55b945037200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:10.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.143+0000 7f6e56d9d640 1 --2- 172.21.15.120:0/2458188512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94503a680 0x55b945037740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:10.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.143+0000 7f6e5659c640 1 --2- 172.21.15.120:0/2458188512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94502f4a0 0x55b945039f50 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f6e48018000 tx=0x7f6e4806d0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63cf2fc2df560458 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:10.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.143+0000 7f6e3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:19:10.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.144+0000 7f6e3dffb640 1 -- 172.21.15.120:0/2458188512 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e480682d0 con 0x55b94502f4a0 2024-09-19T09:19:10.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.144+0000 7f6e3dffb640 1 -- 172.21.15.120:0/2458188512 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f6e4809c5b0 con 0x55b94502f4a0 2024-09-19T09:19:10.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.144+0000 7f6e5f2af740 1 -- 172.21.15.120:0/2458188512 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e4c0041f0 con 0x55b94502f4a0 2024-09-19T09:19:10.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.144+0000 7f6e3dffb640 1 -- 172.21.15.120:0/2458188512 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6e4809ca50 con 0x55b94502f4a0 2024-09-19T09:19:10.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.144+0000 7f6e5f2af740 10 client.21021 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:10.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.144+0000 7f6e5f2af740 1 -- 172.21.15.120:0/2458188512 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b945048bd0 con 0x55b94502f4a0 2024-09-19T09:19:10.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.144+0000 7f6e3dffb640 1 -- 172.21.15.120:0/2458188512 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e4c0041f0 con 0x55b94502f4a0 2024-09-19T09:19:10.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.144+0000 7f6e3dffb640 1 -- 172.21.15.120:0/2458188512 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f6e480a4b30 con 0x55b94502f4a0 2024-09-19T09:19:10.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.144+0000 7f6e5f2af740 10 client.21021 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:10.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.144+0000 7f6e5f2af740 10 client.21021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:19:10.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.144+0000 7f6e5f2af740 20 client.21021 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:10.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.144+0000 7f6e5f2af740 1 --2- 172.21.15.120:0/2458188512 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b94504a090 0x55b94504c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:10.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.144+0000 7f6e5f2af740 4 client.21021 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:19:10.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.144+0000 7f6e5f2af740 1 -- 172.21.15.120:0/2458188512 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b944fa2850 con 0x55b94504a090 2024-09-19T09:19:10.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.145+0000 7f6e55d9b640 1 --2- 172.21.15.120:0/2458188512 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b94504a090 0x55b94504c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:10.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.145+0000 7f6e55d9b640 1 --2- 172.21.15.120:0/2458188512 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b94504a090 0x55b94504c480 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68b84e6b15691f97 server_cookie=d98ee86e0442e972 in_seq=0 out_seq=0 2024-09-19T09:19:10.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.145+0000 7f6e3dffb640 10 client.21021 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:10.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.149+0000 7f6e3dffb640 1 -- 172.21.15.120:0/2458188512 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b944fa2850 con 0x55b94504a090 2024-09-19T09:19:10.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.149+0000 7f6e3dffb640 10 client.21021 handle_command_reply: tid=0 2024-09-19T09:19:10.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.149+0000 7f6e5f2af740 1 client.21021 shutdown 2024-09-19T09:19:10.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.150+0000 7f6e5f2af740 1 -- 172.21.15.120:0/2458188512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94502f4a0 msgr2=0x55b945039f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:10.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.150+0000 7f6e5f2af740 1 --2- 172.21.15.120:0/2458188512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94502f4a0 0x55b945039f50 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f6e48018000 tx=0x7f6e4806d0a0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.150+0000 7f6e5f2af740 1 -- 172.21.15.120:0/2458188512 shutdown_connections 2024-09-19T09:19:10.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.150+0000 7f6e5f2af740 1 --2- 172.21.15.120:0/2458188512 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b94504a090 0x55b94504c480 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.150+0000 7f6e5f2af740 1 --2- 172.21.15.120:0/2458188512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94503a680 0x55b945037740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.150+0000 7f6e5f2af740 1 --2- 172.21.15.120:0/2458188512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b94502fdc0 0x55b945037200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.150+0000 7f6e5f2af740 1 --2- 172.21.15.120:0/2458188512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94502f4a0 0x55b945039f50 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.150+0000 7f6e5f2af740 1 -- 172.21.15.120:0/2458188512 >> 172.21.15.120:0/2458188512 conn(0x55b94502ba30 msgr2=0x55b945031760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:10.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.150+0000 7f6e3dffb640 0 client.21021 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:10.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.150+0000 7f6e5f2af740 1 -- 172.21.15.120:0/2458188512 shutdown_connections 2024-09-19T09:19:10.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.150+0000 7f6e5f2af740 1 -- 172.21.15.120:0/2458188512 wait complete. 2024-09-19T09:19:10.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.150+0000 7f6e5f2af740 20 client.21021 trim_cache size 0 max 16384 2024-09-19T09:19:10.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.168+0000 7f6e5f2af740 1 Processor -- start 2024-09-19T09:19:10.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.168+0000 7f6e5f2af740 1 -- start start 2024-09-19T09:19:10.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.168+0000 7f6e5f2af740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b94502ba30 0x55b9450464e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:10.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.168+0000 7f6e5f2af740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94502f4a0 0x55b9450327f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:10.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.168+0000 7f6e5f2af740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94502fdc0 0x55b945051cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:10.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.168+0000 7f6e5f2af740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6e581ab490 con 0x55b94502fdc0 2024-09-19T09:19:10.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.168+0000 7f6e5f2af740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6e581acfd0 con 0x55b94502f4a0 2024-09-19T09:19:10.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.168+0000 7f6e5f2af740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6e581abb60 con 0x55b94502ba30 2024-09-19T09:19:10.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.169+0000 7f6e56d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94502fdc0 0x55b945051cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:10.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.169+0000 7f6e56d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94502fdc0 0x55b945051cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48582/0 (socket says 172.21.15.120:48582) 2024-09-19T09:19:10.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.169+0000 7f6e56d9d640 1 -- 172.21.15.120:0/246270553 learned_addr learned my addr 172.21.15.120:0/246270553 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:10.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.169+0000 7f6e56d9d640 1 -- 172.21.15.120:0/246270553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b94502ba30 msgr2=0x55b9450464e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:10.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.169+0000 7f6e56d9d640 1 --2- 172.21.15.120:0/246270553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b94502ba30 0x55b9450464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.169+0000 7f6e56d9d640 1 -- 172.21.15.120:0/246270553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94502f4a0 msgr2=0x55b9450327f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:10.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.169+0000 7f6e56d9d640 1 --2- 172.21.15.120:0/246270553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94502f4a0 0x55b9450327f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.169+0000 7f6e56d9d640 1 -- 172.21.15.120:0/246270553 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b94502b860 con 0x55b94502fdc0 2024-09-19T09:19:10.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.169+0000 7f6e56d9d640 1 --2- 172.21.15.120:0/246270553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94502fdc0 0x55b945051cb0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f6e50009ca0 tx=0x7f6e50005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897b323d48541532 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:10.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.169+0000 7f6e3ffff640 1 -- 172.21.15.120:0/246270553 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e50045070 con 0x55b94502fdc0 2024-09-19T09:19:10.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.169+0000 7f6e3ffff640 1 -- 172.21.15.120:0/246270553 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e50036040 con 0x55b94502fdc0 2024-09-19T09:19:10.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.169+0000 7f6e3ffff640 1 -- 172.21.15.120:0/246270553 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5004c660 con 0x55b94502fdc0 2024-09-19T09:19:10.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.170+0000 7f6e5f2af740 1 -- 172.21.15.120:0/246270553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94502fdc0 msgr2=0x55b945051cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:10.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.170+0000 7f6e5f2af740 1 --2- 172.21.15.120:0/246270553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94502fdc0 0x55b945051cb0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f6e50009ca0 tx=0x7f6e50005f10 comp rx=0 tx=0).stop 2024-09-19T09:19:10.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.170+0000 7f6e5f2af740 1 -- 172.21.15.120:0/246270553 shutdown_connections 2024-09-19T09:19:10.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.170+0000 7f6e5f2af740 1 --2- 172.21.15.120:0/246270553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94502fdc0 0x55b945051cb0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.170+0000 7f6e5f2af740 1 --2- 172.21.15.120:0/246270553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94502f4a0 0x55b9450327f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.170+0000 7f6e5f2af740 1 --2- 172.21.15.120:0/246270553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b94502ba30 0x55b9450464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.170+0000 7f6e5f2af740 1 -- 172.21.15.120:0/246270553 >> 172.21.15.120:0/246270553 conn(0x55b94504c870 msgr2=0x55b9450320e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:10.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.170+0000 7f6e5f2af740 1 -- 172.21.15.120:0/246270553 shutdown_connections 2024-09-19T09:19:10.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.170+0000 7f6e5f2af740 1 -- 172.21.15.120:0/246270553 wait complete. 2024-09-19T09:19:10.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.171+0000 7f6e5f2af740 1 -- start start 2024-09-19T09:19:10.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.171+0000 7f6e5f2af740 1 Processor -- start 2024-09-19T09:19:10.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.171+0000 7f6e5f2af740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b94502ba30 0x55b94503e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:10.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.171+0000 7f6e5f2af740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94502f4a0 0x55b94503ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:10.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.171+0000 7f6e5f2af740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94502fdc0 0x55b94503ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:10.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.171+0000 7f6e5f2af740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6e581ac900 con 0x55b94502f4a0 2024-09-19T09:19:10.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.171+0000 7f6e5f2af740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6e581ac230 con 0x55b94502fdc0 2024-09-19T09:19:10.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.171+0000 7f6e5f2af740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6e581a9450 con 0x55b94502ba30 2024-09-19T09:19:10.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.171+0000 7f6e55d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94502f4a0 0x55b94503ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:10.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.171+0000 7f6e5f2af740 10 client.0 _ll_register_callbacks cb 0x7f6e48009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:10.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.171+0000 7f6e55d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94502f4a0 0x55b94503ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48590/0 (socket says 172.21.15.120:48590) 2024-09-19T09:19:10.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.171+0000 7f6e55d9b640 1 -- 172.21.15.120:0/3700824822 learned_addr learned my addr 172.21.15.120:0/3700824822 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:10.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.171+0000 7f6e56d9d640 1 --2- 172.21.15.120:0/3700824822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94502fdc0 0x55b94503ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:10.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.171+0000 7f6e55d9b640 1 -- 172.21.15.120:0/3700824822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b94502ba30 msgr2=0x55b94503e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:10.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.171+0000 7f6e55d9b640 1 --2- 172.21.15.120:0/3700824822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b94502ba30 0x55b94503e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.171+0000 7f6e55d9b640 1 -- 172.21.15.120:0/3700824822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94502fdc0 msgr2=0x55b94503ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:10.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.171+0000 7f6e55d9b640 1 --2- 172.21.15.120:0/3700824822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94502fdc0 0x55b94503ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.171+0000 7f6e55d9b640 1 -- 172.21.15.120:0/3700824822 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9450b3de0 con 0x55b94502f4a0 2024-09-19T09:19:10.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.172+0000 7f6e55d9b640 1 --2- 172.21.15.120:0/3700824822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94502f4a0 0x55b94503ea20 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c013c50 tx=0x7f6e4c013650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2730d582515fe913 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:10.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.172+0000 7f6e3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:19:10.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.172+0000 7f6e3dffb640 1 -- 172.21.15.120:0/3700824822 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c0053e0 con 0x55b94502f4a0 2024-09-19T09:19:10.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.172+0000 7f6e5f2af740 1 -- 172.21.15.120:0/3700824822 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e50036040 con 0x55b94502f4a0 2024-09-19T09:19:10.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.172+0000 7f6e3dffb640 1 -- 172.21.15.120:0/3700824822 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f6e4c005580 con 0x55b94502f4a0 2024-09-19T09:19:10.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.172+0000 7f6e3dffb640 1 -- 172.21.15.120:0/3700824822 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6e4c029440 con 0x55b94502f4a0 2024-09-19T09:19:10.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.172+0000 7f6e5f2af740 10 client.21027 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:10.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.172+0000 7f6e5f2af740 1 -- 172.21.15.120:0/3700824822 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b9450b5090 con 0x55b94502f4a0 2024-09-19T09:19:10.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.172+0000 7f6e3dffb640 1 -- 172.21.15.120:0/3700824822 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e50036040 con 0x55b94502f4a0 2024-09-19T09:19:10.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.172+0000 7f6e3dffb640 1 -- 172.21.15.120:0/3700824822 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f6e4c003710 con 0x55b94502f4a0 2024-09-19T09:19:10.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.173+0000 7f6e5f2af740 10 client.21027 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:10.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.173+0000 7f6e5f2af740 10 client.21027 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:19:10.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.173+0000 7f6e5f2af740 20 client.21027 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:10.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.173+0000 7f6e5f2af740 1 --2- 172.21.15.120:0/3700824822 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b94503a680 0x55b9450b7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:10.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.173+0000 7f6e5f2af740 4 client.21027 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:19:10.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.173+0000 7f6e5f2af740 1 -- 172.21.15.120:0/3700824822 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b944f9cd60 con 0x55b94503a680 2024-09-19T09:19:10.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.173+0000 7f6e5659c640 1 --2- 172.21.15.120:0/3700824822 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b94503a680 0x55b9450b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:10.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.173+0000 7f6e3dffb640 10 client.21027 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:10.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.173+0000 7f6e5659c640 1 --2- 172.21.15.120:0/3700824822 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b94503a680 0x55b9450b7710 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dff576c858c065ac server_cookie=5ce8abe8eb3b5240 in_seq=0 out_seq=0 2024-09-19T09:19:10.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.178+0000 7f6e3dffb640 1 -- 172.21.15.120:0/3700824822 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b944f9cd60 con 0x55b94503a680 2024-09-19T09:19:10.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.178+0000 7f6e3dffb640 10 client.21027 handle_command_reply: tid=0 2024-09-19T09:19:10.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.178+0000 7f6e5f2af740 1 client.21027 shutdown 2024-09-19T09:19:10.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.179+0000 7f6e5f2af740 1 -- 172.21.15.120:0/3700824822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94502f4a0 msgr2=0x55b94503ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:10.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.179+0000 7f6e5f2af740 1 --2- 172.21.15.120:0/3700824822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94502f4a0 0x55b94503ea20 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c013c50 tx=0x7f6e4c013650 comp rx=0 tx=0).stop 2024-09-19T09:19:10.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.179+0000 7f6e5f2af740 1 -- 172.21.15.120:0/3700824822 shutdown_connections 2024-09-19T09:19:10.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.179+0000 7f6e5f2af740 1 --2- 172.21.15.120:0/3700824822 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b94503a680 0x55b9450b7710 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.179+0000 7f6e5f2af740 1 --2- 172.21.15.120:0/3700824822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94502fdc0 0x55b94503ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.179+0000 7f6e5f2af740 1 --2- 172.21.15.120:0/3700824822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94502f4a0 0x55b94503ea20 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.179+0000 7f6e5f2af740 1 --2- 172.21.15.120:0/3700824822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b94502ba30 0x55b94503e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.179+0000 7f6e5f2af740 1 -- 172.21.15.120:0/3700824822 >> 172.21.15.120:0/3700824822 conn(0x55b94504c870 msgr2=0x55b94502ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:10.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.179+0000 7f6e3dffb640 0 client.21027 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:10.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.179+0000 7f6e5f2af740 1 -- 172.21.15.120:0/3700824822 shutdown_connections 2024-09-19T09:19:10.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.179+0000 7f6e5f2af740 1 -- 172.21.15.120:0/3700824822 wait complete. 2024-09-19T09:19:10.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.179+0000 7f6e5f2af740 20 client.21027 trim_cache size 0 max 16384 2024-09-19T09:19:10.182 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:19:10.182 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:19:10.182 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-19T09:19:10.182 INFO:teuthology.orchestra.run.smithi120.stdout: "d9c22cf6-1191-4b92-904f-9848f28513ef": { 2024-09-19T09:19:10.182 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-19T09:19:10.182 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "d9c22cf6-1191-4b92-904f-9848f28513ef", 2024-09-19T09:19:10.183 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-19T09:19:10.183 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:19:10.183 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:19:10.183 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:19:10.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.179+0000 7f6e5ced6640 1 -- 172.21.15.120:0/3584995312 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6e400a3240 msgr2=0x7f6e400a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:10.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.179+0000 7f6e5ced6640 1 --2- 172.21.15.120:0/3584995312 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6e400a3240 0x7f6e400a5700 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6e5002f540 tx=0x7f6e5002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.179+0000 7f6e5ced6640 1 -- 172.21.15.120:0/3584995312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e58100430 msgr2=0x7f6e58199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:10.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.179+0000 7f6e5ced6640 1 --2- 172.21.15.120:0/3584995312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e58100430 0x7f6e58199210 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800e9f0 tx=0x7f6e4800edf0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.179+0000 7f6e5ced6640 1 -- 172.21.15.120:0/3584995312 shutdown_connections 2024-09-19T09:19:10.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.179+0000 7f6e5ced6640 1 --2- 172.21.15.120:0/3584995312 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6e400a3240 0x7f6e400a5700 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.179+0000 7f6e5ced6640 1 --2- 172.21.15.120:0/3584995312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e58105560 0x7f6e581a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.179+0000 7f6e5ced6640 1 --2- 172.21.15.120:0/3584995312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e58100de0 0x7f6e58199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.179+0000 7f6e5ced6640 1 --2- 172.21.15.120:0/3584995312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e58100430 0x7f6e58199210 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.179+0000 7f6e5ced6640 1 -- 172.21.15.120:0/3584995312 >> 172.21.15.120:0/3584995312 conn(0x7f6e58077230 msgr2=0x7f6e58103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:10.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.180+0000 7f6e5ced6640 1 -- 172.21.15.120:0/3584995312 shutdown_connections 2024-09-19T09:19:10.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:10.180+0000 7f6e5ced6640 1 -- 172.21.15.120:0/3584995312 wait complete. 2024-09-19T09:19:10.361 INFO:tasks.workunit.client.0.smithi120.stdout: 10 63894 6.64 MB/sec execute 65 sec latency 65.270 ms 2024-09-19T09:19:10.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:09 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:19:10.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:09 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:10.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:09 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:19:10.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:09 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:19:10.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:09 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:10.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:09 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:19:10.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:09 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:10.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:09 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:19:10.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:09 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:10.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:09 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:19:10.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:09 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:10.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:09 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:19:10.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:09 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:19:10.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:09 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:10.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:09 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:19:10.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:09 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:10.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:09 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:19:10.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:09 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:11.026 INFO:tasks.cephfs.filesystem:scrub status for tag:d9c22cf6-1191-4b92-904f-9848f28513ef - {'path': '/', 'tag': 'd9c22cf6-1191-4b92-904f-9848f28513ef', 'options': 'recursive,force'} 2024-09-19T09:19:11.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:10 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:11.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:10 smithi120 ceph-mon[31150]: pgmap v834: 97 pgs: 97 active+clean; 289 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.0 MiB/s wr, 339 op/s 2024-09-19T09:19:11.361 INFO:tasks.workunit.client.0.smithi120.stdout: 10 64536 7.14 MB/sec execute 66 sec latency 45.266 ms 2024-09-19T09:19:11.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:10 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:11.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:10 smithi196 ceph-mon[35150]: pgmap v834: 97 pgs: 97 active+clean; 289 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.0 MiB/s wr, 339 op/s 2024-09-19T09:19:11.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:10 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:11.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:10 smithi153 ceph-mon[35071]: pgmap v834: 97 pgs: 97 active+clean; 289 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.0 MiB/s wr, 339 op/s 2024-09-19T09:19:12.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:11 smithi120 ceph-mon[31150]: pgmap v835: 97 pgs: 97 active+clean; 295 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.5 MiB/s wr, 422 op/s 2024-09-19T09:19:12.361 INFO:tasks.workunit.client.0.smithi120.stdout: 10 65142 7.49 MB/sec execute 67 sec latency 88.066 ms 2024-09-19T09:19:12.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:11 smithi196 ceph-mon[35150]: pgmap v835: 97 pgs: 97 active+clean; 295 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.5 MiB/s wr, 422 op/s 2024-09-19T09:19:12.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:11 smithi153 ceph-mon[35071]: pgmap v835: 97 pgs: 97 active+clean; 295 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.5 MiB/s wr, 422 op/s 2024-09-19T09:19:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.653+0000 7f7907c82640 1 -- 172.21.15.120:0/3102654009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7900104ef0 msgr2=0x7f79001072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.653+0000 7f7907c82640 1 --2- 172.21.15.120:0/3102654009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7900104ef0 0x7f79001072e0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f78f0009900 tx=0x7f78f002a410 comp rx=0 tx=0).stop 2024-09-19T09:19:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.654+0000 7f7907c82640 1 -- 172.21.15.120:0/3102654009 shutdown_connections 2024-09-19T09:19:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.654+0000 7f7907c82640 1 --2- 172.21.15.120:0/3102654009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7900107820 0x7f7900109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.654+0000 7f7907c82640 1 --2- 172.21.15.120:0/3102654009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7900104ef0 0x7f79001072e0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.654+0000 7f7907c82640 1 --2- 172.21.15.120:0/3102654009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79001025c0 0x7f79001049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.654+0000 7f7907c82640 1 -- 172.21.15.120:0/3102654009 >> 172.21.15.120:0/3102654009 conn(0x7f79000fc1e0 msgr2=0x7f79000fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.654+0000 7f7907c82640 1 -- 172.21.15.120:0/3102654009 shutdown_connections 2024-09-19T09:19:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.654+0000 7f7907c82640 1 -- 172.21.15.120:0/3102654009 wait complete. 2024-09-19T09:19:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.654+0000 7f7907c82640 1 Processor -- start 2024-09-19T09:19:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.654+0000 7f7907c82640 1 -- start start 2024-09-19T09:19:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.655+0000 7f7907c82640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79001025c0 0x7f7900197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.655+0000 7f7907c82640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7900104ef0 0x7f7900198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.655+0000 7f7907c82640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7900107820 0x7f790019f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.655+0000 7f7907c82640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f790010d7d0 con 0x7f7900104ef0 2024-09-19T09:19:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.655+0000 7f7907c82640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f790010d650 con 0x7f79001025c0 2024-09-19T09:19:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.655+0000 7f7907c82640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f790010d950 con 0x7f7900107820 2024-09-19T09:19:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.655+0000 7f790521d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7900104ef0 0x7f7900198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.655+0000 7f790521d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7900104ef0 0x7f7900198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57926/0 (socket says 172.21.15.120:57926) 2024-09-19T09:19:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.655+0000 7f790521d640 1 -- 172.21.15.120:0/1232877456 learned_addr learned my addr 172.21.15.120:0/1232877456 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.655+0000 7f7905a1e640 1 --2- 172.21.15.120:0/1232877456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79001025c0 0x7f7900197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.655+0000 7f790621f640 1 --2- 172.21.15.120:0/1232877456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7900107820 0x7f790019f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.655+0000 7f790521d640 1 -- 172.21.15.120:0/1232877456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7900107820 msgr2=0x7f790019f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.655+0000 7f790521d640 1 --2- 172.21.15.120:0/1232877456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7900107820 0x7f790019f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.655+0000 7f790521d640 1 -- 172.21.15.120:0/1232877456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79001025c0 msgr2=0x7f7900197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.655+0000 7f790521d640 1 --2- 172.21.15.120:0/1232877456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79001025c0 0x7f7900197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.655+0000 7f790521d640 1 -- 172.21.15.120:0/1232877456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f790019fa70 con 0x7f7900104ef0 2024-09-19T09:19:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.655+0000 7f790521d640 1 --2- 172.21.15.120:0/1232877456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7900104ef0 0x7f7900198390 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f78f0031610 tx=0x7f78f00043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.655+0000 7f78f6ffd640 1 -- 172.21.15.120:0/1232877456 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f003f030 con 0x7f7900104ef0 2024-09-19T09:19:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.655+0000 7f7907c82640 1 -- 172.21.15.120:0/1232877456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f790019fd00 con 0x7f7900104ef0 2024-09-19T09:19:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.656+0000 7f78f6ffd640 1 -- 172.21.15.120:0/1232877456 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78f0003ce0 con 0x7f7900104ef0 2024-09-19T09:19:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.656+0000 7f78f6ffd640 1 -- 172.21.15.120:0/1232877456 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f0035070 con 0x7f7900104ef0 2024-09-19T09:19:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.656+0000 7f7907c82640 1 -- 172.21.15.120:0/1232877456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79001a01e0 con 0x7f7900104ef0 2024-09-19T09:19:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.658+0000 7f78f6ffd640 1 -- 172.21.15.120:0/1232877456 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f78f0042030 con 0x7f7900104ef0 2024-09-19T09:19:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.658+0000 7f790aa1f740 1 Processor -- start 2024-09-19T09:19:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.659+0000 7f790aa1f740 1 -- start start 2024-09-19T09:19:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.659+0000 7f790aa1f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56387f4af9a0 0x56387f4afda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.659+0000 7f790aa1f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56387f4b02e0 0x56387f4b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.659+0000 7f790aa1f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56387f4b5370 0x56387f4b7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.659+0000 7f790aa1f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56387f4132f0 con 0x56387f4af9a0 2024-09-19T09:19:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.659+0000 7f790aa1f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56387f3bb270 con 0x56387f4b02e0 2024-09-19T09:19:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.659+0000 7f790aa1f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56387f40f2b0 con 0x56387f4b5370 2024-09-19T09:19:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.659+0000 7f790621f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56387f4af9a0 0x56387f4afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.659+0000 7f790621f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56387f4af9a0 0x56387f4afda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57940/0 (socket says 172.21.15.120:57940) 2024-09-19T09:19:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.659+0000 7f790621f640 1 -- 172.21.15.120:0/310345492 learned_addr learned my addr 172.21.15.120:0/310345492 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.659+0000 7f7905a1e640 1 --2- 172.21.15.120:0/310345492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56387f4b02e0 0x56387f4b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.659+0000 7f790521d640 1 --2- 172.21.15.120:0/310345492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56387f4b5370 0x56387f4b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.659+0000 7f78f6ffd640 1 --2- 172.21.15.120:0/1232877456 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f78ec0a32f0 0x7f78ec0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.659+0000 7f790621f640 1 -- 172.21.15.120:0/310345492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56387f4b5370 msgr2=0x56387f4b7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.659+0000 7f790621f640 1 --2- 172.21.15.120:0/310345492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56387f4b5370 0x56387f4b7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.659+0000 7f790621f640 1 -- 172.21.15.120:0/310345492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56387f4b02e0 msgr2=0x56387f4b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.659+0000 7f790621f640 1 --2- 172.21.15.120:0/310345492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56387f4b02e0 0x56387f4b4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.659+0000 7f790621f640 1 -- 172.21.15.120:0/310345492 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56387f3ba340 con 0x56387f4af9a0 2024-09-19T09:19:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.659+0000 7f7905a1e640 1 --2- 172.21.15.120:0/310345492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56387f4b02e0 0x56387f4b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.660+0000 7f78f6ffd640 1 -- 172.21.15.120:0/1232877456 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f78f00efab0 con 0x7f7900104ef0 2024-09-19T09:19:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.660+0000 7f790621f640 1 --2- 172.21.15.120:0/1232877456 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f78ec0a32f0 0x7f78ec0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.660+0000 7f790621f640 1 --2- 172.21.15.120:0/310345492 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56387f4af9a0 0x56387f4afda0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f78f8010470 tx=0x7f78f8010930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3518c9bb0899049 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.660+0000 7f78ebfff640 1 -- 172.21.15.120:0/310345492 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f801d400 con 0x56387f4af9a0 2024-09-19T09:19:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.660+0000 7f790621f640 1 --2- 172.21.15.120:0/1232877456 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f78ec0a32f0 0x7f78ec0a57b0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f78f8021000 tx=0x7f78f801e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.660+0000 7f78ebfff640 1 -- 172.21.15.120:0/310345492 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78f80134c0 con 0x56387f4af9a0 2024-09-19T09:19:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.660+0000 7f78ebfff640 1 -- 172.21.15.120:0/310345492 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f8011a40 con 0x56387f4af9a0 2024-09-19T09:19:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.661+0000 7f790aa1f740 1 -- 172.21.15.120:0/310345492 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56387f4af9a0 msgr2=0x56387f4afda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.661+0000 7f790aa1f740 1 --2- 172.21.15.120:0/310345492 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56387f4af9a0 0x56387f4afda0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f78f8010470 tx=0x7f78f8010930 comp rx=0 tx=0).stop 2024-09-19T09:19:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.661+0000 7f790aa1f740 1 -- 172.21.15.120:0/310345492 shutdown_connections 2024-09-19T09:19:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.661+0000 7f790aa1f740 1 --2- 172.21.15.120:0/310345492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56387f4b5370 0x56387f4b7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.661+0000 7f790aa1f740 1 --2- 172.21.15.120:0/310345492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56387f4b02e0 0x56387f4b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.661+0000 7f790aa1f740 1 --2- 172.21.15.120:0/310345492 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56387f4af9a0 0x56387f4afda0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.661+0000 7f790aa1f740 1 -- 172.21.15.120:0/310345492 >> 172.21.15.120:0/310345492 conn(0x56387f4ab010 msgr2=0x56387f4ab410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.661+0000 7f790aa1f740 1 -- 172.21.15.120:0/310345492 shutdown_connections 2024-09-19T09:19:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.661+0000 7f790aa1f740 1 -- 172.21.15.120:0/310345492 wait complete. 2024-09-19T09:19:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.661+0000 7f790aa1f740 1 -- start start 2024-09-19T09:19:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.661+0000 7f790aa1f740 1 Processor -- start 2024-09-19T09:19:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.662+0000 7f790aa1f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56387f4af9a0 0x56387f4b9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.662+0000 7f790aa1f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56387f4b02e0 0x56387f4ba160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.662+0000 7f790aa1f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56387f4b5370 0x56387f4c11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.662+0000 7f790aa1f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56387f4b8340 con 0x56387f4b02e0 2024-09-19T09:19:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.662+0000 7f790aa1f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56387f39c290 con 0x56387f4b5370 2024-09-19T09:19:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.662+0000 7f790aa1f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56387f4b84c0 con 0x56387f4af9a0 2024-09-19T09:19:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.662+0000 7f790aa1f740 10 client.0 _ll_register_callbacks cb 0x56387f3cd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.662+0000 7f790521d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56387f4b02e0 0x56387f4ba160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.662+0000 7f790521d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56387f4b02e0 0x56387f4ba160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57946/0 (socket says 172.21.15.120:57946) 2024-09-19T09:19:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.662+0000 7f790621f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56387f4b5370 0x56387f4c11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.662+0000 7f7905a1e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56387f4af9a0 0x56387f4b9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.662+0000 7f790521d640 1 -- 172.21.15.120:0/2020361589 learned_addr learned my addr 172.21.15.120:0/2020361589 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.662+0000 7f790521d640 1 -- 172.21.15.120:0/2020361589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56387f4af9a0 msgr2=0x56387f4b9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.662+0000 7f790521d640 1 --2- 172.21.15.120:0/2020361589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56387f4af9a0 0x56387f4b9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.662+0000 7f790521d640 1 -- 172.21.15.120:0/2020361589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56387f4b5370 msgr2=0x56387f4c11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.662+0000 7f790521d640 1 --2- 172.21.15.120:0/2020361589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56387f4b5370 0x56387f4c11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.662+0000 7f790521d640 1 -- 172.21.15.120:0/2020361589 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56387f126210 con 0x56387f4b02e0 2024-09-19T09:19:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.662+0000 7f790521d640 1 --2- 172.21.15.120:0/2020361589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56387f4b02e0 0x56387f4ba160 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f78f00314e0 tx=0x7f78f00f3060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a13ca660a4d0ca9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.662+0000 7f78e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:19:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.662+0000 7f78e97fa640 1 -- 172.21.15.120:0/2020361589 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f00f6020 con 0x56387f4b02e0 2024-09-19T09:19:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.662+0000 7f790aa1f740 1 -- 172.21.15.120:0/2020361589 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78f80134c0 con 0x56387f4b02e0 2024-09-19T09:19:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.663+0000 7f78e97fa640 1 -- 172.21.15.120:0/2020361589 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f78f00dc0c0 con 0x56387f4b02e0 2024-09-19T09:19:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.663+0000 7f78e97fa640 1 -- 172.21.15.120:0/2020361589 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f78f0049040 con 0x56387f4b02e0 2024-09-19T09:19:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.663+0000 7f790aa1f740 10 client.21039 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.663+0000 7f790aa1f740 1 -- 172.21.15.120:0/2020361589 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56387f1f2620 con 0x56387f4b02e0 2024-09-19T09:19:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.663+0000 7f78e97fa640 1 -- 172.21.15.120:0/2020361589 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78f80134c0 con 0x56387f4b02e0 2024-09-19T09:19:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.663+0000 7f78e97fa640 1 -- 172.21.15.120:0/2020361589 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f78f00b87f0 con 0x56387f4b02e0 2024-09-19T09:19:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.663+0000 7f790aa1f740 10 client.21039 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.663+0000 7f790aa1f740 10 client.21039 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:19:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.663+0000 7f790aa1f740 20 client.21039 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.663+0000 7f790aa1f740 1 --2- 172.21.15.120:0/2020361589 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56387f4c5900 0x56387f4c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.663+0000 7f790aa1f740 4 client.21039 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:19:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.663+0000 7f790aa1f740 1 -- 172.21.15.120:0/2020361589 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56387f420a20 con 0x56387f4c5900 2024-09-19T09:19:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.663+0000 7f7905a1e640 1 --2- 172.21.15.120:0/2020361589 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56387f4c5900 0x56387f4c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.664+0000 7f7905a1e640 1 --2- 172.21.15.120:0/2020361589 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56387f4c5900 0x56387f4c7d10 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=310e6e0e107dafaa server_cookie=9fec713ab84c5ba4 in_seq=0 out_seq=0 2024-09-19T09:19:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.664+0000 7f78e97fa640 10 client.21039 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.666+0000 7f78e97fa640 1 -- 172.21.15.120:0/2020361589 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56387f420a20 con 0x56387f4c5900 2024-09-19T09:19:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.666+0000 7f78e97fa640 10 client.21039 handle_command_reply: tid=0 2024-09-19T09:19:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.666+0000 7f790aa1f740 1 client.21039 shutdown 2024-09-19T09:19:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.666+0000 7f790aa1f740 1 -- 172.21.15.120:0/2020361589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56387f4b02e0 msgr2=0x56387f4ba160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.666+0000 7f790aa1f740 1 --2- 172.21.15.120:0/2020361589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56387f4b02e0 0x56387f4ba160 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f78f00314e0 tx=0x7f78f00f3060 comp rx=0 tx=0).stop 2024-09-19T09:19:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.666+0000 7f790aa1f740 1 -- 172.21.15.120:0/2020361589 shutdown_connections 2024-09-19T09:19:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.666+0000 7f790aa1f740 1 --2- 172.21.15.120:0/2020361589 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56387f4c5900 0x56387f4c7d10 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.666+0000 7f790aa1f740 1 --2- 172.21.15.120:0/2020361589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56387f4b5370 0x56387f4c11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.666+0000 7f790aa1f740 1 --2- 172.21.15.120:0/2020361589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56387f4b02e0 0x56387f4ba160 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.666+0000 7f790aa1f740 1 --2- 172.21.15.120:0/2020361589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56387f4af9a0 0x56387f4b9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.666+0000 7f78e97fa640 0 client.21039 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.666+0000 7f790aa1f740 1 -- 172.21.15.120:0/2020361589 >> 172.21.15.120:0/2020361589 conn(0x56387f4ab010 msgr2=0x56387f4ac820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.666+0000 7f790aa1f740 1 -- 172.21.15.120:0/2020361589 shutdown_connections 2024-09-19T09:19:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.666+0000 7f790aa1f740 1 -- 172.21.15.120:0/2020361589 wait complete. 2024-09-19T09:19:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.666+0000 7f790aa1f740 20 client.21039 trim_cache size 0 max 16384 2024-09-19T09:19:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.681+0000 7f790aa1f740 1 Processor -- start 2024-09-19T09:19:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.681+0000 7f790aa1f740 1 -- start start 2024-09-19T09:19:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.681+0000 7f790aa1f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56387f4af9a0 0x56387f4abec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.681+0000 7f790aa1f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56387f4b02e0 0x56387f4c1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.681+0000 7f790aa1f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56387f4b5370 0x56387f4ac400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.681+0000 7f790aa1f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f79001a8fd0 con 0x56387f4b5370 2024-09-19T09:19:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.681+0000 7f790aa1f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f79001aab10 con 0x56387f4b02e0 2024-09-19T09:19:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.681+0000 7f790aa1f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f79001a96a0 con 0x56387f4af9a0 2024-09-19T09:19:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.681+0000 7f790621f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56387f4b5370 0x56387f4ac400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.681+0000 7f790621f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56387f4b5370 0x56387f4ac400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57958/0 (socket says 172.21.15.120:57958) 2024-09-19T09:19:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.681+0000 7f790621f640 1 -- 172.21.15.120:0/2351829081 learned_addr learned my addr 172.21.15.120:0/2351829081 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.681+0000 7f790521d640 1 --2- 172.21.15.120:0/2351829081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56387f4b02e0 0x56387f4c1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.681+0000 7f7905a1e640 1 --2- 172.21.15.120:0/2351829081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56387f4af9a0 0x56387f4abec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.681+0000 7f790621f640 1 -- 172.21.15.120:0/2351829081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56387f4af9a0 msgr2=0x56387f4abec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.681+0000 7f790621f640 1 --2- 172.21.15.120:0/2351829081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56387f4af9a0 0x56387f4abec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.681+0000 7f790621f640 1 -- 172.21.15.120:0/2351829081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56387f4b02e0 msgr2=0x56387f4c1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.681+0000 7f790621f640 1 --2- 172.21.15.120:0/2351829081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56387f4b02e0 0x56387f4c1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.681+0000 7f790621f640 1 -- 172.21.15.120:0/2351829081 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56387f4c8390 con 0x56387f4b5370 2024-09-19T09:19:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.682+0000 7f790621f640 1 --2- 172.21.15.120:0/2351829081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56387f4b5370 0x56387f4ac400 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f78f8009440 tx=0x7f78f801df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8330a4e950753876 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.682+0000 7f78eb7fe640 1 -- 172.21.15.120:0/2351829081 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f80113e0 con 0x56387f4b5370 2024-09-19T09:19:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.682+0000 7f78eb7fe640 1 -- 172.21.15.120:0/2351829081 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78f8035070 con 0x56387f4b5370 2024-09-19T09:19:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.682+0000 7f78eb7fe640 1 -- 172.21.15.120:0/2351829081 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f8011dd0 con 0x56387f4b5370 2024-09-19T09:19:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.682+0000 7f790aa1f740 1 -- 172.21.15.120:0/2351829081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56387f4b5370 msgr2=0x56387f4ac400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.682+0000 7f790aa1f740 1 --2- 172.21.15.120:0/2351829081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56387f4b5370 0x56387f4ac400 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f78f8009440 tx=0x7f78f801df00 comp rx=0 tx=0).stop 2024-09-19T09:19:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.683+0000 7f790aa1f740 1 -- 172.21.15.120:0/2351829081 shutdown_connections 2024-09-19T09:19:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.683+0000 7f790aa1f740 1 --2- 172.21.15.120:0/2351829081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56387f4b5370 0x56387f4ac400 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.683+0000 7f790aa1f740 1 --2- 172.21.15.120:0/2351829081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56387f4b02e0 0x56387f4c1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.683+0000 7f790aa1f740 1 --2- 172.21.15.120:0/2351829081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56387f4af9a0 0x56387f4abec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.683+0000 7f790aa1f740 1 -- 172.21.15.120:0/2351829081 >> 172.21.15.120:0/2351829081 conn(0x56387f4ab010 msgr2=0x56387f4b9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.683+0000 7f790aa1f740 1 -- 172.21.15.120:0/2351829081 shutdown_connections 2024-09-19T09:19:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.683+0000 7f790aa1f740 1 -- 172.21.15.120:0/2351829081 wait complete. 2024-09-19T09:19:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.683+0000 7f790aa1f740 1 -- start start 2024-09-19T09:19:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.683+0000 7f790aa1f740 1 Processor -- start 2024-09-19T09:19:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.683+0000 7f790aa1f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56387f4af9a0 0x56387f4be740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.683+0000 7f790aa1f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56387f4b02e0 0x56387f4bec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.683+0000 7f790aa1f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56387f4b5370 0x56387f4bfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.683+0000 7f790aa1f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f79001aa440 con 0x56387f4b02e0 2024-09-19T09:19:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.683+0000 7f790aa1f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f79001a9d70 con 0x56387f4af9a0 2024-09-19T09:19:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.683+0000 7f790aa1f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f79001a6f90 con 0x56387f4b5370 2024-09-19T09:19:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.683+0000 7f790aa1f740 10 client.0 _ll_register_callbacks cb 0x7f78f00f3220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.683+0000 7f790521d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56387f4b02e0 0x56387f4bec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.683+0000 7f7905a1e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56387f4af9a0 0x56387f4be740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.683+0000 7f7905a1e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56387f4af9a0 0x56387f4be740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:56366/0 (socket says 172.21.15.120:56366) 2024-09-19T09:19:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.684+0000 7f7905a1e640 1 -- 172.21.15.120:0/2552070202 learned_addr learned my addr 172.21.15.120:0/2552070202 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.683+0000 7f790521d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56387f4b02e0 0x56387f4bec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57972/0 (socket says 172.21.15.120:57972) 2024-09-19T09:19:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.684+0000 7f790621f640 1 --2- 172.21.15.120:0/2552070202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56387f4b5370 0x56387f4bfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.684+0000 7f7905a1e640 1 -- 172.21.15.120:0/2552070202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56387f4b5370 msgr2=0x56387f4bfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.684+0000 7f7905a1e640 1 --2- 172.21.15.120:0/2552070202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56387f4b5370 0x56387f4bfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.684+0000 7f7905a1e640 1 -- 172.21.15.120:0/2552070202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56387f4b02e0 msgr2=0x56387f4bec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.684+0000 7f7905a1e640 1 --2- 172.21.15.120:0/2552070202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56387f4b02e0 0x56387f4bec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.684+0000 7f7905a1e640 1 -- 172.21.15.120:0/2552070202 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56387f4c0250 con 0x56387f4af9a0 2024-09-19T09:19:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.684+0000 7f790521d640 1 --2- 172.21.15.120:0/2552070202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56387f4b02e0 0x56387f4bec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.684+0000 7f7905a1e640 1 --2- 172.21.15.120:0/2552070202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56387f4af9a0 0x56387f4be740 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f78fc005940 tx=0x7f78fc012a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=608b745e44dd6872 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.684+0000 7f78e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:19:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.684+0000 7f78e97fa640 1 -- 172.21.15.120:0/2552070202 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc004220 con 0x56387f4af9a0 2024-09-19T09:19:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.684+0000 7f790aa1f740 1 -- 172.21.15.120:0/2552070202 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78f8035070 con 0x56387f4af9a0 2024-09-19T09:19:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.684+0000 7f78e97fa640 1 -- 172.21.15.120:0/2552070202 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f78fc0043c0 con 0x56387f4af9a0 2024-09-19T09:19:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.684+0000 7f78e97fa640 1 -- 172.21.15.120:0/2552070202 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f78fc0108a0 con 0x56387f4af9a0 2024-09-19T09:19:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.684+0000 7f790aa1f740 10 client.29903 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.684+0000 7f790aa1f740 1 -- 172.21.15.120:0/2552070202 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56387f5305f0 con 0x56387f4af9a0 2024-09-19T09:19:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.685+0000 7f78e97fa640 1 -- 172.21.15.120:0/2552070202 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78f8035070 con 0x56387f4af9a0 2024-09-19T09:19:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.685+0000 7f78e97fa640 1 -- 172.21.15.120:0/2552070202 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f78fc0047a0 con 0x56387f4af9a0 2024-09-19T09:19:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.685+0000 7f790aa1f740 10 client.29903 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.685+0000 7f790aa1f740 10 client.29903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:19:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.685+0000 7f790aa1f740 20 client.29903 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.685+0000 7f790aa1f740 1 --2- 172.21.15.120:0/2552070202 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56387f530e20 0x56387f533240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.685+0000 7f790aa1f740 4 client.29903 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:19:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.685+0000 7f790aa1f740 1 -- 172.21.15.120:0/2552070202 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56387f41ad70 con 0x56387f530e20 2024-09-19T09:19:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.685+0000 7f790521d640 1 --2- 172.21.15.120:0/2552070202 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56387f530e20 0x56387f533240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.685+0000 7f78e97fa640 10 client.29903 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:12.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:12.685+0000 7f790521d640 1 --2- 172.21.15.120:0/2552070202 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56387f530e20 0x56387f533240 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=764778a64367d533 server_cookie=8064ce88b2d7e97 in_seq=0 out_seq=0 2024-09-19T09:19:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:13.072+0000 7f78e97fa640 1 -- 172.21.15.120:0/2552070202 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56387f41ad70 con 0x56387f530e20 2024-09-19T09:19:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:13.072+0000 7f78e97fa640 10 client.29903 handle_command_reply: tid=0 2024-09-19T09:19:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:13.072+0000 7f790aa1f740 1 client.29903 shutdown 2024-09-19T09:19:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:13.072+0000 7f790aa1f740 1 -- 172.21.15.120:0/2552070202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56387f4af9a0 msgr2=0x56387f4be740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:13.072+0000 7f790aa1f740 1 --2- 172.21.15.120:0/2552070202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56387f4af9a0 0x56387f4be740 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f78fc005940 tx=0x7f78fc012a20 comp rx=0 tx=0).stop 2024-09-19T09:19:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:13.073+0000 7f790aa1f740 1 -- 172.21.15.120:0/2552070202 shutdown_connections 2024-09-19T09:19:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:13.073+0000 7f790aa1f740 1 --2- 172.21.15.120:0/2552070202 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56387f530e20 0x56387f533240 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:13.073+0000 7f790aa1f740 1 --2- 172.21.15.120:0/2552070202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56387f4b5370 0x56387f4bfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:13.073+0000 7f78e97fa640 0 client.29903 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:13.073+0000 7f790aa1f740 1 --2- 172.21.15.120:0/2552070202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56387f4b02e0 0x56387f4bec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:13.073+0000 7f790aa1f740 1 --2- 172.21.15.120:0/2552070202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56387f4af9a0 0x56387f4be740 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:13.073+0000 7f790aa1f740 1 -- 172.21.15.120:0/2552070202 >> 172.21.15.120:0/2552070202 conn(0x56387f4ab010 msgr2=0x56387f4a99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:13.073+0000 7f790aa1f740 1 -- 172.21.15.120:0/2552070202 shutdown_connections 2024-09-19T09:19:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:13.073+0000 7f790aa1f740 1 -- 172.21.15.120:0/2552070202 wait complete. 2024-09-19T09:19:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:13.073+0000 7f790aa1f740 20 client.29903 trim_cache size 0 max 16384 2024-09-19T09:19:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:19:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:19:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:19:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:19:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:13.074+0000 7f7907c82640 1 -- 172.21.15.120:0/1232877456 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f78ec0a32f0 msgr2=0x7f78ec0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:13.074+0000 7f7907c82640 1 --2- 172.21.15.120:0/1232877456 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f78ec0a32f0 0x7f78ec0a57b0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f78f8021000 tx=0x7f78f801e000 comp rx=0 tx=0).stop 2024-09-19T09:19:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:13.074+0000 7f7907c82640 1 -- 172.21.15.120:0/1232877456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7900104ef0 msgr2=0x7f7900198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:13.074+0000 7f7907c82640 1 --2- 172.21.15.120:0/1232877456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7900104ef0 0x7f7900198390 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f78f0031610 tx=0x7f78f00043b0 comp rx=0 tx=0).stop 2024-09-19T09:19:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:13.074+0000 7f7907c82640 1 -- 172.21.15.120:0/1232877456 shutdown_connections 2024-09-19T09:19:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:13.074+0000 7f7907c82640 1 --2- 172.21.15.120:0/1232877456 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f78ec0a32f0 0x7f78ec0a57b0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:13.074+0000 7f7907c82640 1 --2- 172.21.15.120:0/1232877456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7900107820 0x7f790019f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:13.074+0000 7f7907c82640 1 --2- 172.21.15.120:0/1232877456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7900104ef0 0x7f7900198390 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:13.074+0000 7f7907c82640 1 --2- 172.21.15.120:0/1232877456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79001025c0 0x7f7900197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:13.074+0000 7f7907c82640 1 -- 172.21.15.120:0/1232877456 >> 172.21.15.120:0/1232877456 conn(0x7f79000fc1e0 msgr2=0x7f7900102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:13.074+0000 7f7907c82640 1 -- 172.21.15.120:0/1232877456 shutdown_connections 2024-09-19T09:19:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:13.074+0000 7f7907c82640 1 -- 172.21.15.120:0/1232877456 wait complete. 2024-09-19T09:19:13.361 INFO:tasks.workunit.client.0.smithi120.stdout: 10 65536 7.72 MB/sec execute 68 sec latency 240.198 ms 2024-09-19T09:19:14.361 INFO:tasks.workunit.client.0.smithi120.stdout: 10 66090 7.94 MB/sec execute 69 sec latency 214.605 ms 2024-09-19T09:19:15.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:14 smithi120 ceph-mon[31150]: pgmap v836: 97 pgs: 97 active+clean; 302 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.3 MiB/s wr, 550 op/s 2024-09-19T09:19:15.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:14 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:19:15.361 INFO:tasks.workunit.client.0.smithi120.stdout: 10 66726 8.30 MB/sec execute 70 sec latency 55.391 ms 2024-09-19T09:19:15.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:14 smithi196 ceph-mon[35150]: pgmap v836: 97 pgs: 97 active+clean; 302 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.3 MiB/s wr, 550 op/s 2024-09-19T09:19:15.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:14 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:19:15.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:14 smithi153 ceph-mon[35071]: pgmap v836: 97 pgs: 97 active+clean; 302 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.3 MiB/s wr, 550 op/s 2024-09-19T09:19:15.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:14 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:19:16.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:15 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:16.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:15 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:19:16.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:15 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:19:16.361 INFO:tasks.workunit.client.0.smithi120.stdout: 10 67473 8.78 MB/sec execute 71 sec latency 173.108 ms 2024-09-19T09:19:16.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:15 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:16.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:15 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:19:16.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:15 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:19:16.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:15 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:16.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:15 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:19:16.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:15 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:19:17.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:16 smithi120 ceph-mon[31150]: pgmap v837: 97 pgs: 97 active+clean; 302 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.4 MiB/s wr, 542 op/s 2024-09-19T09:19:17.361 INFO:tasks.workunit.client.0.smithi120.stdout: 10 68153 9.17 MB/sec execute 72 sec latency 88.964 ms 2024-09-19T09:19:17.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:16 smithi196 ceph-mon[35150]: pgmap v837: 97 pgs: 97 active+clean; 302 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.4 MiB/s wr, 542 op/s 2024-09-19T09:19:17.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:16 smithi153 ceph-mon[35071]: pgmap v837: 97 pgs: 97 active+clean; 302 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.4 MiB/s wr, 542 op/s 2024-09-19T09:19:18.361 INFO:tasks.workunit.client.0.smithi120.stdout: 10 68855 9.63 MB/sec execute 73 sec latency 43.956 ms 2024-09-19T09:19:19.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:18 smithi120 ceph-mon[31150]: pgmap v838: 97 pgs: 97 active+clean; 297 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.6 MiB/s wr, 651 op/s 2024-09-19T09:19:19.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:18 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:19:19.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:18 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:19:19.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:18 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:19:19.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:18 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:19:19.361 INFO:tasks.workunit.client.0.smithi120.stdout: 10 69416 9.90 MB/sec execute 74 sec latency 85.015 ms 2024-09-19T09:19:19.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:18 smithi196 ceph-mon[35150]: pgmap v838: 97 pgs: 97 active+clean; 297 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.6 MiB/s wr, 651 op/s 2024-09-19T09:19:19.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:18 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:19:19.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:18 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:19:19.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:18 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:19:19.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:18 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:19:19.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:18 smithi153 ceph-mon[35071]: pgmap v838: 97 pgs: 97 active+clean; 297 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.6 MiB/s wr, 651 op/s 2024-09-19T09:19:19.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:18 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:19:19.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:18 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:19:19.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:18 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:19:19.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:18 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:19:20.362 INFO:tasks.workunit.client.0.smithi120.stdout: 10 69937 10.17 MB/sec execute 75 sec latency 68.573 ms 2024-09-19T09:19:21.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:20 smithi120 ceph-mon[31150]: pgmap v839: 97 pgs: 97 active+clean; 303 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.7 MiB/s wr, 650 op/s 2024-09-19T09:19:21.362 INFO:tasks.workunit.client.0.smithi120.stdout: 10 70500 10.35 MB/sec execute 76 sec latency 60.913 ms 2024-09-19T09:19:21.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:20 smithi196 ceph-mon[35150]: pgmap v839: 97 pgs: 97 active+clean; 303 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.7 MiB/s wr, 650 op/s 2024-09-19T09:19:21.437 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:20 smithi153 ceph-mon[35071]: pgmap v839: 97 pgs: 97 active+clean; 303 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.7 MiB/s wr, 650 op/s 2024-09-19T09:19:22.362 INFO:tasks.workunit.client.0.smithi120.stdout: 10 71126 10.58 MB/sec execute 77 sec latency 65.019 ms 2024-09-19T09:19:23.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:22 smithi120 ceph-mon[31150]: pgmap v840: 97 pgs: 97 active+clean; 307 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.7 MiB/s wr, 653 op/s 2024-09-19T09:19:23.362 INFO:tasks.workunit.client.0.smithi120.stdout: 10 71869 10.96 MB/sec execute 78 sec latency 66.917 ms 2024-09-19T09:19:23.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:22 smithi196 ceph-mon[35150]: pgmap v840: 97 pgs: 97 active+clean; 307 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.7 MiB/s wr, 653 op/s 2024-09-19T09:19:23.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:22 smithi153 ceph-mon[35071]: pgmap v840: 97 pgs: 97 active+clean; 307 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.7 MiB/s wr, 653 op/s 2024-09-19T09:19:24.362 INFO:tasks.workunit.client.0.smithi120.stdout: 10 72601 11.32 MB/sec execute 79 sec latency 48.742 ms 2024-09-19T09:19:25.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:24 smithi120 ceph-mon[31150]: pgmap v841: 97 pgs: 97 active+clean; 310 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.5 MiB/s wr, 598 op/s 2024-09-19T09:19:25.362 INFO:tasks.workunit.client.0.smithi120.stdout: 10 73308 11.80 MB/sec execute 80 sec latency 78.379 ms 2024-09-19T09:19:25.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:24 smithi196 ceph-mon[35150]: pgmap v841: 97 pgs: 97 active+clean; 310 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.5 MiB/s wr, 598 op/s 2024-09-19T09:19:25.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:24 smithi153 ceph-mon[35071]: pgmap v841: 97 pgs: 97 active+clean; 310 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.5 MiB/s wr, 598 op/s 2024-09-19T09:19:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.170+0000 7fd02245e640 1 -- 172.21.15.120:0/1855734258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd01c102d50 msgr2=0x7fd01c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.170+0000 7fd02245e640 1 --2- 172.21.15.120:0/1855734258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd01c102d50 0x7fd01c1031d0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fd010009900 tx=0x7fd01002a410 comp rx=0 tx=0).stop 2024-09-19T09:19:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.170+0000 7fd02245e640 1 -- 172.21.15.120:0/1855734258 shutdown_connections 2024-09-19T09:19:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.170+0000 7fd02245e640 1 --2- 172.21.15.120:0/1855734258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd01c103710 0x7fd01c109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.170+0000 7fd02245e640 1 --2- 172.21.15.120:0/1855734258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd01c102d50 0x7fd01c1031d0 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.170+0000 7fd02245e640 1 --2- 172.21.15.120:0/1855734258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd01c101b60 0x7fd01c101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.170+0000 7fd02245e640 1 -- 172.21.15.120:0/1855734258 >> 172.21.15.120:0/1855734258 conn(0x7fd01c0fd2e0 msgr2=0x7fd01c0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.170+0000 7fd02245e640 1 -- 172.21.15.120:0/1855734258 shutdown_connections 2024-09-19T09:19:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.170+0000 7fd02245e640 1 -- 172.21.15.120:0/1855734258 wait complete. 2024-09-19T09:19:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.171+0000 7fd02245e640 1 Processor -- start 2024-09-19T09:19:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.171+0000 7fd02245e640 1 -- start start 2024-09-19T09:19:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.171+0000 7fd02245e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd01c101b60 0x7fd01c197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.171+0000 7fd02245e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd01c102d50 0x7fd01c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.171+0000 7fd02245e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd01c103710 0x7fd01c19f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.171+0000 7fd02245e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c10d7e0 con 0x7fd01c102d50 2024-09-19T09:19:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.171+0000 7fd02245e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c10d660 con 0x7fd01c101b60 2024-09-19T09:19:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.171+0000 7fd02245e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c10d960 con 0x7fd01c103710 2024-09-19T09:19:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.171+0000 7fd01bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd01c101b60 0x7fd01c197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.171+0000 7fd01b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd01c102d50 0x7fd01c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.171+0000 7fd01bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd01c101b60 0x7fd01c197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:50830/0 (socket says 172.21.15.120:50830) 2024-09-19T09:19:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.171+0000 7fd01bfff640 1 -- 172.21.15.120:0/742978681 learned_addr learned my addr 172.21.15.120:0/742978681 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.171+0000 7fd01b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd01c102d50 0x7fd01c198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46196/0 (socket says 172.21.15.120:46196) 2024-09-19T09:19:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.171+0000 7fd0209fb640 1 --2- 172.21.15.120:0/742978681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd01c103710 0x7fd01c19f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.172+0000 7fd01bfff640 1 -- 172.21.15.120:0/742978681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd01c103710 msgr2=0x7fd01c19f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.172+0000 7fd01bfff640 1 --2- 172.21.15.120:0/742978681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd01c103710 0x7fd01c19f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.172+0000 7fd01bfff640 1 -- 172.21.15.120:0/742978681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd01c102d50 msgr2=0x7fd01c198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.172+0000 7fd01bfff640 1 --2- 172.21.15.120:0/742978681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd01c102d50 0x7fd01c198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.172+0000 7fd01bfff640 1 -- 172.21.15.120:0/742978681 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd01c19f9f0 con 0x7fd01c101b60 2024-09-19T09:19:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.172+0000 7fd01bfff640 1 --2- 172.21.15.120:0/742978681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd01c101b60 0x7fd01c197dd0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd00c00e9c0 tx=0x7fd00c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.172+0000 7fd0197fa640 1 -- 172.21.15.120:0/742978681 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c00cd80 con 0x7fd01c101b60 2024-09-19T09:19:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.172+0000 7fd0197fa640 1 -- 172.21.15.120:0/742978681 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd00c0035f0 con 0x7fd01c101b60 2024-09-19T09:19:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.172+0000 7fd02245e640 1 -- 172.21.15.120:0/742978681 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd01c19fce0 con 0x7fd01c101b60 2024-09-19T09:19:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.172+0000 7fd0197fa640 1 -- 172.21.15.120:0/742978681 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c010690 con 0x7fd01c101b60 2024-09-19T09:19:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.172+0000 7fd02245e640 1 -- 172.21.15.120:0/742978681 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd01c1a01a0 con 0x7fd01c101b60 2024-09-19T09:19:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.175+0000 7fd0197fa640 1 -- 172.21.15.120:0/742978681 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd00c0037a0 con 0x7fd01c101b60 2024-09-19T09:19:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.175+0000 7fd0251fc740 1 Processor -- start 2024-09-19T09:19:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.175+0000 7fd0251fc740 1 -- start start 2024-09-19T09:19:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.176+0000 7fd0251fc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e0d807920 0x561e0d807d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.176+0000 7fd0251fc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e0d808260 0x561e0d80ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.176+0000 7fd0209fb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e0d807920 0x561e0d807d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.176+0000 7fd0251fc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0d80b330 0x561e0d8182e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.176+0000 7fd0209fb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e0d807920 0x561e0d807d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46204/0 (socket says 172.21.15.120:46204) 2024-09-19T09:19:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.176+0000 7fd0209fb640 1 -- 172.21.15.120:0/274682929 learned_addr learned my addr 172.21.15.120:0/274682929 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.176+0000 7fd0251fc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561e0d76b2f0 con 0x561e0d807920 2024-09-19T09:19:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.176+0000 7fd0251fc740 1 -- 172.21.15.120:0/274682929 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561e0d713270 con 0x561e0d808260 2024-09-19T09:19:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.176+0000 7fd0251fc740 1 -- 172.21.15.120:0/274682929 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561e0d7672b0 con 0x561e0d80b330 2024-09-19T09:19:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.176+0000 7fd01bfff640 1 --2- 172.21.15.120:0/274682929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e0d808260 0x561e0d80ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.176+0000 7fd0197fa640 1 --2- 172.21.15.120:0/742978681 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd0040a3370 0x7fd0040a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.176+0000 7fd01b7fe640 1 --2- 172.21.15.120:0/274682929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0d80b330 0x561e0d8182e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.177+0000 7fd0209fb640 1 -- 172.21.15.120:0/274682929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0d80b330 msgr2=0x561e0d8182e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.177+0000 7fd0209fb640 1 --2- 172.21.15.120:0/274682929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0d80b330 0x561e0d8182e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.177+0000 7fd0209fb640 1 -- 172.21.15.120:0/274682929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e0d808260 msgr2=0x561e0d80ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.177+0000 7fd0209fb640 1 --2- 172.21.15.120:0/274682929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e0d808260 0x561e0d80ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.177+0000 7fd0209fb640 1 -- 172.21.15.120:0/274682929 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e0d712340 con 0x561e0d807920 2024-09-19T09:19:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.177+0000 7fd01bfff640 1 --2- 172.21.15.120:0/274682929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e0d808260 0x561e0d80ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.177+0000 7fd01b7fe640 1 --2- 172.21.15.120:0/274682929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0d80b330 0x561e0d8182e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:19:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.177+0000 7fd0197fa640 1 -- 172.21.15.120:0/742978681 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd00c0cfc80 con 0x7fd01c101b60 2024-09-19T09:19:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.177+0000 7fd0209fb640 1 --2- 172.21.15.120:0/742978681 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd0040a3370 0x7fd0040a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.177+0000 7fd0209fb640 1 --2- 172.21.15.120:0/274682929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e0d807920 0x561e0d807d20 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fd0140115c0 tx=0x7fd014011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d230c029b6aa42 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.177+0000 7fd0027fc640 1 -- 172.21.15.120:0/274682929 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd014017050 con 0x561e0d807920 2024-09-19T09:19:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.177+0000 7fd0027fc640 1 -- 172.21.15.120:0/274682929 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd014004720 con 0x561e0d807920 2024-09-19T09:19:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.177+0000 7fd0209fb640 1 --2- 172.21.15.120:0/742978681 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd0040a3370 0x7fd0040a5830 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd014022000 tx=0x7fd014023000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.177+0000 7fd0027fc640 1 -- 172.21.15.120:0/274682929 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01401bce0 con 0x561e0d807920 2024-09-19T09:19:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.178+0000 7fd0251fc740 1 -- 172.21.15.120:0/274682929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e0d807920 msgr2=0x561e0d807d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.178+0000 7fd0251fc740 1 --2- 172.21.15.120:0/274682929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e0d807920 0x561e0d807d20 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fd0140115c0 tx=0x7fd014011a80 comp rx=0 tx=0).stop 2024-09-19T09:19:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.178+0000 7fd0251fc740 1 -- 172.21.15.120:0/274682929 shutdown_connections 2024-09-19T09:19:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.178+0000 7fd0251fc740 1 --2- 172.21.15.120:0/274682929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0d80b330 0x561e0d8182e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.178+0000 7fd0251fc740 1 --2- 172.21.15.120:0/274682929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e0d808260 0x561e0d80ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.178+0000 7fd0251fc740 1 --2- 172.21.15.120:0/274682929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e0d807920 0x561e0d807d20 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.178+0000 7fd0251fc740 1 -- 172.21.15.120:0/274682929 >> 172.21.15.120:0/274682929 conn(0x561e0d803010 msgr2=0x561e0d803410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.178+0000 7fd0251fc740 1 -- 172.21.15.120:0/274682929 shutdown_connections 2024-09-19T09:19:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.178+0000 7fd0251fc740 1 -- 172.21.15.120:0/274682929 wait complete. 2024-09-19T09:19:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.178+0000 7fd0251fc740 1 -- start start 2024-09-19T09:19:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.178+0000 7fd0251fc740 1 Processor -- start 2024-09-19T09:19:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.179+0000 7fd0251fc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e0d807920 0x561e0d81a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.179+0000 7fd0251fc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0d808260 0x561e0d81a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.179+0000 7fd0251fc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e0d80b330 0x561e0d821940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.179+0000 7fd0251fc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561e0d818b70 con 0x561e0d807920 2024-09-19T09:19:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.179+0000 7fd0251fc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561e0d6f4290 con 0x561e0d80b330 2024-09-19T09:19:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.179+0000 7fd0251fc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561e0d818cf0 con 0x561e0d808260 2024-09-19T09:19:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.179+0000 7fd0251fc740 10 client.0 _ll_register_callbacks cb 0x561e0d725140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.179+0000 7fd01bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e0d807920 0x561e0d81a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.179+0000 7fd01bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e0d807920 0x561e0d81a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46220/0 (socket says 172.21.15.120:46220) 2024-09-19T09:19:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.179+0000 7fd01bfff640 1 -- 172.21.15.120:0/972182783 learned_addr learned my addr 172.21.15.120:0/972182783 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.179+0000 7fd0209fb640 1 --2- 172.21.15.120:0/972182783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e0d80b330 0x561e0d821940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.179+0000 7fd01b7fe640 1 --2- 172.21.15.120:0/972182783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0d808260 0x561e0d81a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.179+0000 7fd01bfff640 1 -- 172.21.15.120:0/972182783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0d808260 msgr2=0x561e0d81a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.179+0000 7fd01bfff640 1 --2- 172.21.15.120:0/972182783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0d808260 0x561e0d81a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.179+0000 7fd01bfff640 1 -- 172.21.15.120:0/972182783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e0d80b330 msgr2=0x561e0d821940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.179+0000 7fd01bfff640 1 --2- 172.21.15.120:0/972182783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e0d80b330 0x561e0d821940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.179+0000 7fd01bfff640 1 -- 172.21.15.120:0/972182783 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e0d47e210 con 0x561e0d807920 2024-09-19T09:19:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.179+0000 7fd0209fb640 1 --2- 172.21.15.120:0/972182783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e0d80b330 0x561e0d821940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.179+0000 7fd01bfff640 1 --2- 172.21.15.120:0/972182783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e0d807920 0x561e0d81a3c0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0058c0 tx=0x7fd00c09bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=defd4c1313296d5c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.179+0000 7fcffb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:19:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.179+0000 7fcffb7fe640 1 -- 172.21.15.120:0/972182783 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c0a4f10 con 0x561e0d807920 2024-09-19T09:19:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.179+0000 7fcffb7fe640 1 -- 172.21.15.120:0/972182783 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd00c0bc0b0 con 0x561e0d807920 2024-09-19T09:19:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.179+0000 7fd0251fc740 1 -- 172.21.15.120:0/972182783 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd014004720 con 0x561e0d807920 2024-09-19T09:19:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.180+0000 7fcffb7fe640 1 -- 172.21.15.120:0/972182783 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd00c0a0650 con 0x561e0d807920 2024-09-19T09:19:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.180+0000 7fd0251fc740 10 client.21057 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.180+0000 7fd0251fc740 1 -- 172.21.15.120:0/972182783 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561e0d54a620 con 0x561e0d807920 2024-09-19T09:19:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.180+0000 7fcffb7fe640 1 -- 172.21.15.120:0/972182783 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd014004720 con 0x561e0d807920 2024-09-19T09:19:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.180+0000 7fcffb7fe640 1 -- 172.21.15.120:0/972182783 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fd00c0d6060 con 0x561e0d807920 2024-09-19T09:19:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.180+0000 7fd0251fc740 10 client.21057 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.180+0000 7fd0251fc740 10 client.21057 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:19:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.180+0000 7fd0251fc740 20 client.21057 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.180+0000 7fd0251fc740 1 --2- 172.21.15.120:0/972182783 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561e0d8260e0 0x561e0d8284d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.180+0000 7fd0251fc740 4 client.21057 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:19:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.180+0000 7fd0251fc740 1 -- 172.21.15.120:0/972182783 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e0d778a20 con 0x561e0d8260e0 2024-09-19T09:19:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.181+0000 7fd01b7fe640 1 --2- 172.21.15.120:0/972182783 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561e0d8260e0 0x561e0d8284d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.181+0000 7fcffb7fe640 10 client.21057 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.181+0000 7fd01b7fe640 1 --2- 172.21.15.120:0/972182783 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561e0d8260e0 0x561e0d8284d0 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da252c051448b8b2 server_cookie=69a2fb046f919167 in_seq=0 out_seq=0 2024-09-19T09:19:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.186+0000 7fcffb7fe640 1 -- 172.21.15.120:0/972182783 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e0d778a20 con 0x561e0d8260e0 2024-09-19T09:19:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.186+0000 7fcffb7fe640 10 client.21057 handle_command_reply: tid=0 2024-09-19T09:19:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.186+0000 7fd0251fc740 1 client.21057 shutdown 2024-09-19T09:19:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.186+0000 7fd0251fc740 1 -- 172.21.15.120:0/972182783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e0d807920 msgr2=0x561e0d81a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.186+0000 7fd0251fc740 1 --2- 172.21.15.120:0/972182783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e0d807920 0x561e0d81a3c0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0058c0 tx=0x7fd00c09bde0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.186+0000 7fd0251fc740 1 -- 172.21.15.120:0/972182783 shutdown_connections 2024-09-19T09:19:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.186+0000 7fd0251fc740 1 --2- 172.21.15.120:0/972182783 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561e0d8260e0 0x561e0d8284d0 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.186+0000 7fd0251fc740 1 --2- 172.21.15.120:0/972182783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e0d80b330 0x561e0d821940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.186+0000 7fd0251fc740 1 --2- 172.21.15.120:0/972182783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0d808260 0x561e0d81a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.186+0000 7fd0251fc740 1 --2- 172.21.15.120:0/972182783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e0d807920 0x561e0d81a3c0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.186+0000 7fd0251fc740 1 -- 172.21.15.120:0/972182783 >> 172.21.15.120:0/972182783 conn(0x561e0d803010 msgr2=0x561e0d804820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.186+0000 7fcffb7fe640 0 client.21057 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.186+0000 7fd0251fc740 1 -- 172.21.15.120:0/972182783 shutdown_connections 2024-09-19T09:19:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.187+0000 7fd0251fc740 1 -- 172.21.15.120:0/972182783 wait complete. 2024-09-19T09:19:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.187+0000 7fd0251fc740 20 client.21057 trim_cache size 0 max 16384 2024-09-19T09:19:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.208+0000 7fd0251fc740 1 Processor -- start 2024-09-19T09:19:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.208+0000 7fd0251fc740 1 -- start start 2024-09-19T09:19:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.208+0000 7fd0251fc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e0d808260 0x561e0d81a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.208+0000 7fd0251fc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0d80b330 0x561e0d821900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.208+0000 7fd0251fc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e0d815010 0x561e0d8020b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.208+0000 7fd0251fc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1a6d60 con 0x561e0d815010 2024-09-19T09:19:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.208+0000 7fd0251fc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1a88a0 con 0x561e0d808260 2024-09-19T09:19:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.208+0000 7fd0251fc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1a7430 con 0x561e0d80b330 2024-09-19T09:19:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.208+0000 7fd0209fb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e0d815010 0x561e0d8020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.208+0000 7fd0209fb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e0d815010 0x561e0d8020b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46232/0 (socket says 172.21.15.120:46232) 2024-09-19T09:19:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.208+0000 7fd0209fb640 1 -- 172.21.15.120:0/4209675007 learned_addr learned my addr 172.21.15.120:0/4209675007 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.208+0000 7fd01bfff640 1 --2- 172.21.15.120:0/4209675007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e0d808260 0x561e0d81a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.208+0000 7fd01b7fe640 1 --2- 172.21.15.120:0/4209675007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0d80b330 0x561e0d821900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.209+0000 7fd0209fb640 1 -- 172.21.15.120:0/4209675007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0d80b330 msgr2=0x561e0d821900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.209+0000 7fd0209fb640 1 --2- 172.21.15.120:0/4209675007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0d80b330 0x561e0d821900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.209+0000 7fd0209fb640 1 -- 172.21.15.120:0/4209675007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e0d808260 msgr2=0x561e0d81a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.209+0000 7fd0209fb640 1 --2- 172.21.15.120:0/4209675007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e0d808260 0x561e0d81a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.209+0000 7fd0209fb640 1 -- 172.21.15.120:0/4209675007 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e0d824a00 con 0x561e0d815010 2024-09-19T09:19:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.209+0000 7fd01b7fe640 1 --2- 172.21.15.120:0/4209675007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0d80b330 0x561e0d821900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.209+0000 7fd0209fb640 1 --2- 172.21.15.120:0/4209675007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e0d815010 0x561e0d8020b0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fd01400f570 tx=0x7fd01400ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209bc404a2559b8c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.209+0000 7fd001ffb640 1 -- 172.21.15.120:0/4209675007 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd014035070 con 0x561e0d815010 2024-09-19T09:19:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.209+0000 7fd001ffb640 1 -- 172.21.15.120:0/4209675007 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd01401b430 con 0x561e0d815010 2024-09-19T09:19:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.209+0000 7fd001ffb640 1 -- 172.21.15.120:0/4209675007 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd014004250 con 0x561e0d815010 2024-09-19T09:19:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.210+0000 7fd0251fc740 1 -- 172.21.15.120:0/4209675007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e0d815010 msgr2=0x561e0d8020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.210+0000 7fd0251fc740 1 --2- 172.21.15.120:0/4209675007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e0d815010 0x561e0d8020b0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fd01400f570 tx=0x7fd01400ffd0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.210+0000 7fd0251fc740 1 -- 172.21.15.120:0/4209675007 shutdown_connections 2024-09-19T09:19:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.210+0000 7fd0251fc740 1 --2- 172.21.15.120:0/4209675007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e0d815010 0x561e0d8020b0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.210+0000 7fd0251fc740 1 --2- 172.21.15.120:0/4209675007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0d80b330 0x561e0d821900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.210+0000 7fd0251fc740 1 --2- 172.21.15.120:0/4209675007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e0d808260 0x561e0d81a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.210+0000 7fd0251fc740 1 -- 172.21.15.120:0/4209675007 >> 172.21.15.120:0/4209675007 conn(0x561e0d803010 msgr2=0x561e0d8019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.210+0000 7fd0251fc740 1 -- 172.21.15.120:0/4209675007 shutdown_connections 2024-09-19T09:19:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.210+0000 7fd0251fc740 1 -- 172.21.15.120:0/4209675007 wait complete. 2024-09-19T09:19:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.210+0000 7fd0251fc740 1 -- start start 2024-09-19T09:19:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.211+0000 7fd0251fc740 1 Processor -- start 2024-09-19T09:19:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.211+0000 7fd0251fc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e0d808260 0x561e0d81ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.211+0000 7fd0251fc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0d80b330 0x561e0d81f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.211+0000 7fd0251fc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e0d815010 0x561e0d820380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.211+0000 7fd0251fc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1a81d0 con 0x561e0d808260 2024-09-19T09:19:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.211+0000 7fd0251fc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1a7b00 con 0x561e0d815010 2024-09-19T09:19:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.211+0000 7fd0251fc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1a4d20 con 0x561e0d80b330 2024-09-19T09:19:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.211+0000 7fd0251fc740 10 client.0 _ll_register_callbacks cb 0x7fd00c09c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.211+0000 7fd01bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e0d808260 0x561e0d81ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.211+0000 7fd0209fb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e0d815010 0x561e0d820380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.211+0000 7fd01bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e0d808260 0x561e0d81ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46244/0 (socket says 172.21.15.120:46244) 2024-09-19T09:19:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.211+0000 7fd01bfff640 1 -- 172.21.15.120:0/3627003700 learned_addr learned my addr 172.21.15.120:0/3627003700 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.211+0000 7fd01b7fe640 1 --2- 172.21.15.120:0/3627003700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0d80b330 0x561e0d81f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.211+0000 7fd0209fb640 1 -- 172.21.15.120:0/3627003700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0d80b330 msgr2=0x561e0d81f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.211+0000 7fd0209fb640 1 --2- 172.21.15.120:0/3627003700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0d80b330 0x561e0d81f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.211+0000 7fd0209fb640 1 -- 172.21.15.120:0/3627003700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e0d808260 msgr2=0x561e0d81ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.211+0000 7fd0209fb640 1 --2- 172.21.15.120:0/3627003700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e0d808260 0x561e0d81ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.211+0000 7fd0209fb640 1 -- 172.21.15.120:0/3627003700 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e0d820990 con 0x561e0d815010 2024-09-19T09:19:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.211+0000 7fd01b7fe640 1 --2- 172.21.15.120:0/3627003700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0d80b330 0x561e0d81f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.212+0000 7fd0209fb640 1 --2- 172.21.15.120:0/3627003700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e0d815010 0x561e0d820380 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd01400f540 tx=0x7fd0140046b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c7f2fdc5244174f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.212+0000 7fcffb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:19:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.212+0000 7fcffb7fe640 1 -- 172.21.15.120:0/3627003700 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01403e060 con 0x561e0d815010 2024-09-19T09:19:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.212+0000 7fcffb7fe640 1 -- 172.21.15.120:0/3627003700 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd014005b70 con 0x561e0d815010 2024-09-19T09:19:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.212+0000 7fd0251fc740 1 -- 172.21.15.120:0/3627003700 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd01401b430 con 0x561e0d815010 2024-09-19T09:19:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.212+0000 7fcffb7fe640 1 -- 172.21.15.120:0/3627003700 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd014035040 con 0x561e0d815010 2024-09-19T09:19:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.212+0000 7fd0251fc740 10 client.29921 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.212+0000 7fd0251fc740 1 -- 172.21.15.120:0/3627003700 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561e0d821ca0 con 0x561e0d815010 2024-09-19T09:19:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.212+0000 7fcffb7fe640 1 -- 172.21.15.120:0/3627003700 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd01401b430 con 0x561e0d815010 2024-09-19T09:19:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.213+0000 7fcffb7fe640 1 -- 172.21.15.120:0/3627003700 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fd014043070 con 0x561e0d815010 2024-09-19T09:19:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.213+0000 7fd0251fc740 10 client.29921 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.213+0000 7fd0251fc740 10 client.29921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:19:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.213+0000 7fd0251fc740 20 client.29921 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.213+0000 7fd0251fc740 1 --2- 172.21.15.120:0/3627003700 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561e0d8912b0 0x561e0d8936d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.213+0000 7fd0251fc740 4 client.29921 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:19:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.213+0000 7fd0251fc740 1 -- 172.21.15.120:0/3627003700 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e0d772d70 con 0x561e0d8912b0 2024-09-19T09:19:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.213+0000 7fd01bfff640 1 --2- 172.21.15.120:0/3627003700 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561e0d8912b0 0x561e0d8936d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.213+0000 7fcffb7fe640 10 client.29921 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.213+0000 7fd01bfff640 1 --2- 172.21.15.120:0/3627003700 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561e0d8912b0 0x561e0d8936d0 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2afec03e4a03fb52 server_cookie=5edce9ea77900530 in_seq=0 out_seq=0 2024-09-19T09:19:26.362 INFO:tasks.workunit.client.0.smithi120.stdout: 10 73787 12.00 MB/sec execute 81 sec latency 206.444 ms 2024-09-19T09:19:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.692+0000 7fcffb7fe640 1 -- 172.21.15.120:0/3627003700 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561e0d772d70 con 0x561e0d8912b0 2024-09-19T09:19:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.692+0000 7fcffb7fe640 10 client.29921 handle_command_reply: tid=0 2024-09-19T09:19:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.692+0000 7fd0251fc740 1 client.29921 shutdown 2024-09-19T09:19:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.692+0000 7fd0251fc740 1 -- 172.21.15.120:0/3627003700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e0d815010 msgr2=0x561e0d820380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.692+0000 7fd0251fc740 1 --2- 172.21.15.120:0/3627003700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e0d815010 0x561e0d820380 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd01400f540 tx=0x7fd0140046b0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.692+0000 7fd0251fc740 1 -- 172.21.15.120:0/3627003700 shutdown_connections 2024-09-19T09:19:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.692+0000 7fd0251fc740 1 --2- 172.21.15.120:0/3627003700 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561e0d8912b0 0x561e0d8936d0 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.692+0000 7fd0251fc740 1 --2- 172.21.15.120:0/3627003700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e0d815010 0x561e0d820380 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.692+0000 7fd0251fc740 1 --2- 172.21.15.120:0/3627003700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0d80b330 0x561e0d81f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.692+0000 7fd0251fc740 1 --2- 172.21.15.120:0/3627003700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561e0d808260 0x561e0d81ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.692+0000 7fd0251fc740 1 -- 172.21.15.120:0/3627003700 >> 172.21.15.120:0/3627003700 conn(0x561e0d803010 msgr2=0x561e0d804740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.692+0000 7fcffb7fe640 0 client.29921 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.692+0000 7fd0251fc740 1 -- 172.21.15.120:0/3627003700 shutdown_connections 2024-09-19T09:19:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.692+0000 7fd0251fc740 1 -- 172.21.15.120:0/3627003700 wait complete. 2024-09-19T09:19:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.692+0000 7fd0251fc740 20 client.29921 trim_cache size 0 max 16384 2024-09-19T09:19:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:19:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:19:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:19:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:19:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.693+0000 7fd02245e640 1 -- 172.21.15.120:0/742978681 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd0040a3370 msgr2=0x7fd0040a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.693+0000 7fd02245e640 1 --2- 172.21.15.120:0/742978681 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd0040a3370 0x7fd0040a5830 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd014022000 tx=0x7fd014023000 comp rx=0 tx=0).stop 2024-09-19T09:19:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.693+0000 7fd02245e640 1 -- 172.21.15.120:0/742978681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd01c101b60 msgr2=0x7fd01c197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.693+0000 7fd02245e640 1 --2- 172.21.15.120:0/742978681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd01c101b60 0x7fd01c197dd0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd00c00e9c0 tx=0x7fd00c00ee80 comp rx=0 tx=0).stop 2024-09-19T09:19:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.693+0000 7fd02245e640 1 -- 172.21.15.120:0/742978681 shutdown_connections 2024-09-19T09:19:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.693+0000 7fd02245e640 1 --2- 172.21.15.120:0/742978681 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd0040a3370 0x7fd0040a5830 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.693+0000 7fd02245e640 1 --2- 172.21.15.120:0/742978681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd01c103710 0x7fd01c19f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.693+0000 7fd02245e640 1 --2- 172.21.15.120:0/742978681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd01c102d50 0x7fd01c198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.693+0000 7fd02245e640 1 --2- 172.21.15.120:0/742978681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd01c101b60 0x7fd01c197dd0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.693+0000 7fd02245e640 1 -- 172.21.15.120:0/742978681 >> 172.21.15.120:0/742978681 conn(0x7fd01c0fd2e0 msgr2=0x7fd01c1060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.693+0000 7fd02245e640 1 -- 172.21.15.120:0/742978681 shutdown_connections 2024-09-19T09:19:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:26.694+0000 7fd02245e640 1 -- 172.21.15.120:0/742978681 wait complete. 2024-09-19T09:19:27.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:26 smithi120 ceph-mon[31150]: pgmap v842: 97 pgs: 97 active+clean; 318 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.5 MiB/s wr, 512 op/s 2024-09-19T09:19:27.362 INFO:tasks.workunit.client.0.smithi120.stdout: 10 74248 12.13 MB/sec execute 82 sec latency 252.198 ms 2024-09-19T09:19:27.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:26 smithi196 ceph-mon[35150]: pgmap v842: 97 pgs: 97 active+clean; 318 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.5 MiB/s wr, 512 op/s 2024-09-19T09:19:27.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:26 smithi153 ceph-mon[35071]: pgmap v842: 97 pgs: 97 active+clean; 318 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.5 MiB/s wr, 512 op/s 2024-09-19T09:19:28.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:28 smithi120 ceph-mon[31150]: pgmap v843: 97 pgs: 97 active+clean; 323 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.6 MiB/s wr, 573 op/s 2024-09-19T09:19:28.362 INFO:tasks.workunit.client.0.smithi120.stdout: 10 74739 12.28 MB/sec execute 83 sec latency 49.940 ms 2024-09-19T09:19:28.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:28 smithi196 ceph-mon[35150]: pgmap v843: 97 pgs: 97 active+clean; 323 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.6 MiB/s wr, 573 op/s 2024-09-19T09:19:28.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:28 smithi153 ceph-mon[35071]: pgmap v843: 97 pgs: 97 active+clean; 323 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.6 MiB/s wr, 573 op/s 2024-09-19T09:19:29.362 INFO:tasks.workunit.client.0.smithi120.stdout: 10 75373 12.48 MB/sec execute 84 sec latency 88.535 ms 2024-09-19T09:19:30.362 INFO:tasks.workunit.client.0.smithi120.stdout: 10 76058 12.80 MB/sec execute 85 sec latency 61.103 ms 2024-09-19T09:19:31.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:30 smithi120 ceph-mon[31150]: pgmap v844: 97 pgs: 97 active+clean; 314 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.0 MiB/s wr, 617 op/s 2024-09-19T09:19:31.362 INFO:tasks.workunit.client.0.smithi120.stdout: 10 76742 13.01 MB/sec execute 86 sec latency 64.396 ms 2024-09-19T09:19:31.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:30 smithi196 ceph-mon[35150]: pgmap v844: 97 pgs: 97 active+clean; 314 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.0 MiB/s wr, 617 op/s 2024-09-19T09:19:31.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:30 smithi153 ceph-mon[35071]: pgmap v844: 97 pgs: 97 active+clean; 314 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.0 MiB/s wr, 617 op/s 2024-09-19T09:19:32.362 INFO:tasks.workunit.client.0.smithi120.stdout: 10 77334 13.21 MB/sec execute 87 sec latency 96.945 ms 2024-09-19T09:19:32.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:32 smithi196 ceph-mon[35150]: pgmap v845: 97 pgs: 97 active+clean; 318 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.5 MiB/s wr, 604 op/s 2024-09-19T09:19:32.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:32 smithi153 ceph-mon[35071]: pgmap v845: 97 pgs: 97 active+clean; 318 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.5 MiB/s wr, 604 op/s 2024-09-19T09:19:33.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:32 smithi120 ceph-mon[31150]: pgmap v845: 97 pgs: 97 active+clean; 318 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.5 MiB/s wr, 604 op/s 2024-09-19T09:19:33.362 INFO:tasks.workunit.client.0.smithi120.stdout: 10 78083 13.62 MB/sec execute 88 sec latency 107.470 ms 2024-09-19T09:19:34.362 INFO:tasks.workunit.client.0.smithi120.stdout: 10 78613 13.82 MB/sec execute 89 sec latency 77.604 ms 2024-09-19T09:19:35.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:34 smithi120 ceph-mon[31150]: pgmap v846: 97 pgs: 97 active+clean; 328 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.9 MiB/s wr, 631 op/s 2024-09-19T09:19:35.362 INFO:tasks.workunit.client.0.smithi120.stdout: 10 79162 13.99 MB/sec execute 90 sec latency 61.377 ms 2024-09-19T09:19:35.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:34 smithi196 ceph-mon[35150]: pgmap v846: 97 pgs: 97 active+clean; 328 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.9 MiB/s wr, 631 op/s 2024-09-19T09:19:35.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:34 smithi153 ceph-mon[35071]: pgmap v846: 97 pgs: 97 active+clean; 328 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.9 MiB/s wr, 631 op/s 2024-09-19T09:19:36.362 INFO:tasks.workunit.client.0.smithi120.stdout: 10 79734 14.10 MB/sec execute 91 sec latency 119.518 ms 2024-09-19T09:19:37.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:36 smithi120 ceph-mon[31150]: pgmap v847: 97 pgs: 97 active+clean; 331 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.1 MiB/s wr, 657 op/s 2024-09-19T09:19:37.362 INFO:tasks.workunit.client.0.smithi120.stdout: 10 80252 14.26 MB/sec execute 92 sec latency 286.443 ms 2024-09-19T09:19:37.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:36 smithi196 ceph-mon[35150]: pgmap v847: 97 pgs: 97 active+clean; 331 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.1 MiB/s wr, 657 op/s 2024-09-19T09:19:37.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:36 smithi153 ceph-mon[35071]: pgmap v847: 97 pgs: 97 active+clean; 331 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.1 MiB/s wr, 657 op/s 2024-09-19T09:19:38.362 INFO:tasks.workunit.client.0.smithi120.stdout: 10 80916 14.47 MB/sec execute 93 sec latency 86.029 ms 2024-09-19T09:19:39.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:38 smithi120 ceph-mon[31150]: pgmap v848: 97 pgs: 97 active+clean; 345 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.0 MiB/s wr, 677 op/s 2024-09-19T09:19:39.362 INFO:tasks.workunit.client.0.smithi120.stdout: 10 81671 14.73 MB/sec execute 94 sec latency 43.639 ms 2024-09-19T09:19:39.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:38 smithi196 ceph-mon[35150]: pgmap v848: 97 pgs: 97 active+clean; 345 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.0 MiB/s wr, 677 op/s 2024-09-19T09:19:39.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:38 smithi153 ceph-mon[35071]: pgmap v848: 97 pgs: 97 active+clean; 345 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.0 MiB/s wr, 677 op/s 2024-09-19T09:19:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.795+0000 7fc605766640 1 -- 172.21.15.120:0/2864616744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc600107870 msgr2=0x7fc600109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.795+0000 7fc605766640 1 --2- 172.21.15.120:0/2864616744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc600107870 0x7fc600109ca0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009ca0 tx=0x7fc5f802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:19:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.795+0000 7fc605766640 1 -- 172.21.15.120:0/2864616744 shutdown_connections 2024-09-19T09:19:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.795+0000 7fc605766640 1 --2- 172.21.15.120:0/2864616744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc600107870 0x7fc600109ca0 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.795+0000 7fc605766640 1 --2- 172.21.15.120:0/2864616744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc600104f40 0x7fc600107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.795+0000 7fc605766640 1 --2- 172.21.15.120:0/2864616744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc600102610 0x7fc600104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.795+0000 7fc605766640 1 -- 172.21.15.120:0/2864616744 >> 172.21.15.120:0/2864616744 conn(0x7fc6000fc1b0 msgr2=0x7fc6000fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.795+0000 7fc605766640 1 -- 172.21.15.120:0/2864616744 shutdown_connections 2024-09-19T09:19:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.795+0000 7fc605766640 1 -- 172.21.15.120:0/2864616744 wait complete. 2024-09-19T09:19:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.795+0000 7fc605766640 1 Processor -- start 2024-09-19T09:19:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.796+0000 7fc605766640 1 -- start start 2024-09-19T09:19:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.796+0000 7fc605766640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc600102610 0x7fc600101f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.796+0000 7fc605766640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc600104f40 0x7fc6001005c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.796+0000 7fc605766640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc600107870 0x7fc600100b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.796+0000 7fc605766640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc60010d810 con 0x7fc600102610 2024-09-19T09:19:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.796+0000 7fc605766640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc60010d690 con 0x7fc600104f40 2024-09-19T09:19:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.796+0000 7fc605766640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc60010d990 con 0x7fc600107870 2024-09-19T09:19:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.796+0000 7fc5feffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc600102610 0x7fc600101f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.796+0000 7fc5feffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc600102610 0x7fc600101f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48962/0 (socket says 172.21.15.120:48962) 2024-09-19T09:19:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.796+0000 7fc5feffd640 1 -- 172.21.15.120:0/1081791619 learned_addr learned my addr 172.21.15.120:0/1081791619 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.796+0000 7fc5fe7fc640 1 --2- 172.21.15.120:0/1081791619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc600104f40 0x7fc6001005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.796+0000 7fc5ff7fe640 1 --2- 172.21.15.120:0/1081791619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc600107870 0x7fc600100b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.796+0000 7fc5feffd640 1 -- 172.21.15.120:0/1081791619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc600107870 msgr2=0x7fc600100b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.796+0000 7fc5feffd640 1 --2- 172.21.15.120:0/1081791619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc600107870 0x7fc600100b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.796+0000 7fc5feffd640 1 -- 172.21.15.120:0/1081791619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc600104f40 msgr2=0x7fc6001005c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.796+0000 7fc5feffd640 1 --2- 172.21.15.120:0/1081791619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc600104f40 0x7fc6001005c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.796+0000 7fc5feffd640 1 -- 172.21.15.120:0/1081791619 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc600101390 con 0x7fc600102610 2024-09-19T09:19:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.796+0000 7fc5feffd640 1 --2- 172.21.15.120:0/1081791619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc600102610 0x7fc600101f80 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000e9f0 tx=0x7fc5f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.797+0000 7fc5e7fff640 1 -- 172.21.15.120:0/1081791619 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f000cd80 con 0x7fc600102610 2024-09-19T09:19:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.797+0000 7fc5e7fff640 1 -- 172.21.15.120:0/1081791619 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f0010910 con 0x7fc600102610 2024-09-19T09:19:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.797+0000 7fc605766640 1 -- 172.21.15.120:0/1081791619 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6001a1f00 con 0x7fc600102610 2024-09-19T09:19:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.797+0000 7fc5e7fff640 1 -- 172.21.15.120:0/1081791619 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0018950 con 0x7fc600102610 2024-09-19T09:19:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.797+0000 7fc605766640 1 -- 172.21.15.120:0/1081791619 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6001a2310 con 0x7fc600102610 2024-09-19T09:19:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.799+0000 7fc6085dd740 1 Processor -- start 2024-09-19T09:19:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.799+0000 7fc6085dd740 1 -- start start 2024-09-19T09:19:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.800+0000 7fc6085dd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cb2e79a0 0x5610cb2e7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.800+0000 7fc6085dd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610cb2e82e0 0x5610cb2ecda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.800+0000 7fc6085dd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610cb2ed370 0x5610cb2ef830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.800+0000 7fc6085dd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5610cb24b2f0 con 0x5610cb2e79a0 2024-09-19T09:19:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.800+0000 7fc6085dd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5610cb1f3270 con 0x5610cb2ed370 2024-09-19T09:19:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.800+0000 7fc6085dd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5610cb2472b0 con 0x5610cb2e82e0 2024-09-19T09:19:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.800+0000 7fc5e7fff640 1 -- 172.21.15.120:0/1081791619 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fc5f0018b50 con 0x7fc600102610 2024-09-19T09:19:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.800+0000 7fc5ff7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cb2e79a0 0x5610cb2e7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.800+0000 7fc5ff7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cb2e79a0 0x5610cb2e7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48966/0 (socket says 172.21.15.120:48966) 2024-09-19T09:19:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.800+0000 7fc5ff7fe640 1 -- 172.21.15.120:0/3359286806 learned_addr learned my addr 172.21.15.120:0/3359286806 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.800+0000 7fc5fe7fc640 1 --2- 172.21.15.120:0/3359286806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610cb2ed370 0x5610cb2ef830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.800+0000 7fc5feffd640 1 --2- 172.21.15.120:0/3359286806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610cb2e82e0 0x5610cb2ecda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.800+0000 7fc5fe7fc640 1 -- 172.21.15.120:0/3359286806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610cb2e82e0 msgr2=0x5610cb2ecda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.800+0000 7fc5fe7fc640 1 --2- 172.21.15.120:0/3359286806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610cb2e82e0 0x5610cb2ecda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.800+0000 7fc5fe7fc640 1 -- 172.21.15.120:0/3359286806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cb2e79a0 msgr2=0x5610cb2e7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.800+0000 7fc5fe7fc640 1 --2- 172.21.15.120:0/3359286806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cb2e79a0 0x5610cb2e7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.800+0000 7fc5fe7fc640 1 -- 172.21.15.120:0/3359286806 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610cb1f2340 con 0x5610cb2ed370 2024-09-19T09:19:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.800+0000 7fc5fe7fc640 1 --2- 172.21.15.120:0/3359286806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610cb2ed370 0x5610cb2ef830 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc60006adc0 tx=0x7fc5f4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bab48314574a141 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.801+0000 7fc5e57fa640 1 -- 172.21.15.120:0/3359286806 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4004230 con 0x5610cb2ed370 2024-09-19T09:19:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.801+0000 7fc5e7fff640 1 --2- 172.21.15.120:0/1081791619 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc5e80a32f0 0x7fc5e80a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.801+0000 7fc5e57fa640 1 -- 172.21.15.120:0/3359286806 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f4009e10 con 0x5610cb2ed370 2024-09-19T09:19:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.801+0000 7fc5e57fa640 1 -- 172.21.15.120:0/3359286806 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4004d10 con 0x5610cb2ed370 2024-09-19T09:19:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.801+0000 7fc5ff7fe640 1 --2- 172.21.15.120:0/1081791619 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc5e80a32f0 0x7fc5e80a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.801+0000 7fc5e7fff640 1 -- 172.21.15.120:0/1081791619 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc5f00cfc90 con 0x7fc600102610 2024-09-19T09:19:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.801+0000 7fc6085dd740 1 -- 172.21.15.120:0/3359286806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610cb2ed370 msgr2=0x5610cb2ef830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.801+0000 7fc5ff7fe640 1 --2- 172.21.15.120:0/1081791619 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc5e80a32f0 0x7fc5e80a57b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8005e30 tx=0x7fc5f803d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.801+0000 7fc6085dd740 1 --2- 172.21.15.120:0/3359286806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610cb2ed370 0x5610cb2ef830 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc60006adc0 tx=0x7fc5f4011810 comp rx=0 tx=0).stop 2024-09-19T09:19:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.802+0000 7fc6085dd740 1 -- 172.21.15.120:0/3359286806 shutdown_connections 2024-09-19T09:19:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.802+0000 7fc6085dd740 1 --2- 172.21.15.120:0/3359286806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610cb2ed370 0x5610cb2ef830 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.802+0000 7fc6085dd740 1 --2- 172.21.15.120:0/3359286806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610cb2e82e0 0x5610cb2ecda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.802+0000 7fc6085dd740 1 --2- 172.21.15.120:0/3359286806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cb2e79a0 0x5610cb2e7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.802+0000 7fc6085dd740 1 -- 172.21.15.120:0/3359286806 >> 172.21.15.120:0/3359286806 conn(0x5610cb2e3010 msgr2=0x5610cb2e3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.802+0000 7fc6085dd740 1 -- 172.21.15.120:0/3359286806 shutdown_connections 2024-09-19T09:19:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.802+0000 7fc6085dd740 1 -- 172.21.15.120:0/3359286806 wait complete. 2024-09-19T09:19:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.802+0000 7fc6085dd740 1 -- start start 2024-09-19T09:19:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.802+0000 7fc6085dd740 1 Processor -- start 2024-09-19T09:19:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.802+0000 7fc6085dd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610cb2e79a0 0x5610cb2f1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.802+0000 7fc6085dd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610cb2e82e0 0x5610cb2f2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.802+0000 7fc6085dd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cb2ed370 0x5610cb2f90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.802+0000 7fc6085dd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5610cb2f0340 con 0x5610cb2ed370 2024-09-19T09:19:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.802+0000 7fc6085dd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5610cb1d4290 con 0x5610cb2e79a0 2024-09-19T09:19:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.802+0000 7fc6085dd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5610cb2f04c0 con 0x5610cb2e82e0 2024-09-19T09:19:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.802+0000 7fc6085dd740 10 client.0 _ll_register_callbacks cb 0x5610cb205140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.802+0000 7fc5ff7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cb2ed370 0x5610cb2f90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.802+0000 7fc5ff7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cb2ed370 0x5610cb2f90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48978/0 (socket says 172.21.15.120:48978) 2024-09-19T09:19:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.802+0000 7fc5feffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610cb2e79a0 0x5610cb2f1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.802+0000 7fc5ff7fe640 1 -- 172.21.15.120:0/645762474 learned_addr learned my addr 172.21.15.120:0/645762474 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.802+0000 7fc5fe7fc640 1 --2- 172.21.15.120:0/645762474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610cb2e82e0 0x5610cb2f2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.803+0000 7fc5ff7fe640 1 -- 172.21.15.120:0/645762474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610cb2e82e0 msgr2=0x5610cb2f2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.803+0000 7fc5ff7fe640 1 --2- 172.21.15.120:0/645762474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610cb2e82e0 0x5610cb2f2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.803+0000 7fc5ff7fe640 1 -- 172.21.15.120:0/645762474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610cb2e79a0 msgr2=0x5610cb2f1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.803+0000 7fc5ff7fe640 1 --2- 172.21.15.120:0/645762474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610cb2e79a0 0x5610cb2f1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.803+0000 7fc5ff7fe640 1 -- 172.21.15.120:0/645762474 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610caf5e210 con 0x5610cb2ed370 2024-09-19T09:19:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.803+0000 7fc5ff7fe640 1 --2- 172.21.15.120:0/645762474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cb2ed370 0x5610cb2f90c0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009bf0 tx=0x7fc5f802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80afffbf42c3cbf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.803+0000 7fc5e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:19:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.803+0000 7fc5e2ff5640 1 -- 172.21.15.120:0/645762474 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8056060 con 0x5610cb2ed370 2024-09-19T09:19:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.803+0000 7fc5e2ff5640 1 -- 172.21.15.120:0/645762474 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc5f80305c0 con 0x5610cb2ed370 2024-09-19T09:19:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.803+0000 7fc6085dd740 1 -- 172.21.15.120:0/645762474 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5f4009e10 con 0x5610cb2ed370 2024-09-19T09:19:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.803+0000 7fc5e2ff5640 1 -- 172.21.15.120:0/645762474 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc5f80307c0 con 0x5610cb2ed370 2024-09-19T09:19:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.803+0000 7fc6085dd740 10 client.21075 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.803+0000 7fc6085dd740 1 -- 172.21.15.120:0/645762474 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5610cb02a620 con 0x5610cb2ed370 2024-09-19T09:19:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.803+0000 7fc5e2ff5640 1 -- 172.21.15.120:0/645762474 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f4009e10 con 0x5610cb2ed370 2024-09-19T09:19:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.804+0000 7fc5e2ff5640 1 -- 172.21.15.120:0/645762474 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fc5f805b070 con 0x5610cb2ed370 2024-09-19T09:19:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.804+0000 7fc6085dd740 10 client.21075 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.804+0000 7fc6085dd740 10 client.21075 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:19:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.804+0000 7fc6085dd740 20 client.21075 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.804+0000 7fc6085dd740 1 --2- 172.21.15.120:0/645762474 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5610cb2fd7b0 0x5610cb2ffbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.804+0000 7fc6085dd740 4 client.21075 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:19:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.804+0000 7fc6085dd740 1 -- 172.21.15.120:0/645762474 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610cb258a20 con 0x5610cb2fd7b0 2024-09-19T09:19:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.804+0000 7fc5feffd640 1 --2- 172.21.15.120:0/645762474 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5610cb2fd7b0 0x5610cb2ffbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.805+0000 7fc5feffd640 1 --2- 172.21.15.120:0/645762474 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5610cb2fd7b0 0x5610cb2ffbe0 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=373515cb9f802b40 server_cookie=62358e6dcb5ee122 in_seq=0 out_seq=0 2024-09-19T09:19:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.805+0000 7fc5e2ff5640 10 client.21075 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.808+0000 7fc5e2ff5640 1 -- 172.21.15.120:0/645762474 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5610cb258a20 con 0x5610cb2fd7b0 2024-09-19T09:19:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.808+0000 7fc5e2ff5640 10 client.21075 handle_command_reply: tid=0 2024-09-19T09:19:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.808+0000 7fc6085dd740 1 client.21075 shutdown 2024-09-19T09:19:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.808+0000 7fc6085dd740 1 -- 172.21.15.120:0/645762474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cb2ed370 msgr2=0x5610cb2f90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.808+0000 7fc6085dd740 1 --2- 172.21.15.120:0/645762474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cb2ed370 0x5610cb2f90c0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009bf0 tx=0x7fc5f802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:19:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.808+0000 7fc6085dd740 1 -- 172.21.15.120:0/645762474 shutdown_connections 2024-09-19T09:19:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.808+0000 7fc6085dd740 1 --2- 172.21.15.120:0/645762474 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5610cb2fd7b0 0x5610cb2ffbe0 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.808+0000 7fc6085dd740 1 --2- 172.21.15.120:0/645762474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cb2ed370 0x5610cb2f90c0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.808+0000 7fc5e2ff5640 0 client.21075 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.808+0000 7fc6085dd740 1 --2- 172.21.15.120:0/645762474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610cb2e82e0 0x5610cb2f2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.808+0000 7fc6085dd740 1 --2- 172.21.15.120:0/645762474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610cb2e79a0 0x5610cb2f1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.808+0000 7fc6085dd740 1 -- 172.21.15.120:0/645762474 >> 172.21.15.120:0/645762474 conn(0x5610cb2e3010 msgr2=0x5610cb2e4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.808+0000 7fc6085dd740 1 -- 172.21.15.120:0/645762474 shutdown_connections 2024-09-19T09:19:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.808+0000 7fc6085dd740 1 -- 172.21.15.120:0/645762474 wait complete. 2024-09-19T09:19:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.808+0000 7fc6085dd740 20 client.21075 trim_cache size 0 max 16384 2024-09-19T09:19:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.823+0000 7fc6085dd740 1 Processor -- start 2024-09-19T09:19:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.823+0000 7fc6085dd740 1 -- start start 2024-09-19T09:19:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.823+0000 7fc6085dd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610cb2e79a0 0x5610cb2e3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.823+0000 7fc6085dd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610cb2e82e0 0x5610cb2f9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.823+0000 7fc6085dd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cb2ed370 0x5610cb2e4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.823+0000 7fc6085dd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc6001ab240 con 0x5610cb2ed370 2024-09-19T09:19:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.823+0000 7fc6085dd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc6001acd80 con 0x5610cb2e79a0 2024-09-19T09:19:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.823+0000 7fc6085dd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc6001ab910 con 0x5610cb2e82e0 2024-09-19T09:19:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.823+0000 7fc5ff7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cb2ed370 0x5610cb2e4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.823+0000 7fc5ff7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cb2ed370 0x5610cb2e4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48982/0 (socket says 172.21.15.120:48982) 2024-09-19T09:19:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.823+0000 7fc5ff7fe640 1 -- 172.21.15.120:0/2645470685 learned_addr learned my addr 172.21.15.120:0/2645470685 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.823+0000 7fc5fe7fc640 1 --2- 172.21.15.120:0/2645470685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610cb2e82e0 0x5610cb2f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.823+0000 7fc5feffd640 1 --2- 172.21.15.120:0/2645470685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610cb2e79a0 0x5610cb2e3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.824+0000 7fc5ff7fe640 1 -- 172.21.15.120:0/2645470685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610cb2e82e0 msgr2=0x5610cb2f9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.824+0000 7fc5ff7fe640 1 --2- 172.21.15.120:0/2645470685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610cb2e82e0 0x5610cb2f9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.824+0000 7fc5ff7fe640 1 -- 172.21.15.120:0/2645470685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610cb2e79a0 msgr2=0x5610cb2e3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.824+0000 7fc5ff7fe640 1 --2- 172.21.15.120:0/2645470685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610cb2e79a0 0x5610cb2e3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.824+0000 7fc5ff7fe640 1 -- 172.21.15.120:0/2645470685 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610cb2e87c0 con 0x5610cb2ed370 2024-09-19T09:19:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.824+0000 7fc5ff7fe640 1 --2- 172.21.15.120:0/2645470685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cb2ed370 0x5610cb2e4400 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8031ae0 tx=0x7fc5f802f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=642d886041f70cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.824+0000 7fc5e4ff9640 1 -- 172.21.15.120:0/2645470685 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8061070 con 0x5610cb2ed370 2024-09-19T09:19:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.824+0000 7fc5e4ff9640 1 -- 172.21.15.120:0/2645470685 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f8036040 con 0x5610cb2ed370 2024-09-19T09:19:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.824+0000 7fc5e4ff9640 1 -- 172.21.15.120:0/2645470685 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8056070 con 0x5610cb2ed370 2024-09-19T09:19:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.825+0000 7fc6085dd740 1 -- 172.21.15.120:0/2645470685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cb2ed370 msgr2=0x5610cb2e4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.825+0000 7fc6085dd740 1 --2- 172.21.15.120:0/2645470685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cb2ed370 0x5610cb2e4400 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8031ae0 tx=0x7fc5f802f390 comp rx=0 tx=0).stop 2024-09-19T09:19:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.825+0000 7fc6085dd740 1 -- 172.21.15.120:0/2645470685 shutdown_connections 2024-09-19T09:19:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.825+0000 7fc6085dd740 1 --2- 172.21.15.120:0/2645470685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cb2ed370 0x5610cb2e4400 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.825+0000 7fc6085dd740 1 --2- 172.21.15.120:0/2645470685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610cb2e82e0 0x5610cb2f9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.825+0000 7fc6085dd740 1 --2- 172.21.15.120:0/2645470685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610cb2e79a0 0x5610cb2e3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.825+0000 7fc6085dd740 1 -- 172.21.15.120:0/2645470685 >> 172.21.15.120:0/2645470685 conn(0x5610cb2e3010 msgr2=0x5610cb2f1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.825+0000 7fc6085dd740 1 -- 172.21.15.120:0/2645470685 shutdown_connections 2024-09-19T09:19:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.825+0000 7fc6085dd740 1 -- 172.21.15.120:0/2645470685 wait complete. 2024-09-19T09:19:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.825+0000 7fc6085dd740 1 -- start start 2024-09-19T09:19:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.826+0000 7fc6085dd740 1 Processor -- start 2024-09-19T09:19:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.826+0000 7fc6085dd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cb2e79a0 0x5610cb2f6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.826+0000 7fc6085dd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610cb2e82e0 0x5610cb2f6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.826+0000 7fc6085dd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610cb2ed370 0x5610cb2f7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.826+0000 7fc6085dd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc6001ac6b0 con 0x5610cb2e79a0 2024-09-19T09:19:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.826+0000 7fc6085dd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc6001abfe0 con 0x5610cb2ed370 2024-09-19T09:19:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.826+0000 7fc6085dd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc6001a9200 con 0x5610cb2e82e0 2024-09-19T09:19:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.826+0000 7fc6085dd740 10 client.0 _ll_register_callbacks cb 0x7fc5f8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.826+0000 7fc5feffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cb2e79a0 0x5610cb2f6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.826+0000 7fc5feffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cb2e79a0 0x5610cb2f6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48990/0 (socket says 172.21.15.120:48990) 2024-09-19T09:19:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.826+0000 7fc5feffd640 1 -- 172.21.15.120:0/1542893443 learned_addr learned my addr 172.21.15.120:0/1542893443 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.826+0000 7fc5fe7fc640 1 --2- 172.21.15.120:0/1542893443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610cb2e82e0 0x5610cb2f6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.826+0000 7fc5ff7fe640 1 --2- 172.21.15.120:0/1542893443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610cb2ed370 0x5610cb2f7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.826+0000 7fc5feffd640 1 -- 172.21.15.120:0/1542893443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610cb2e82e0 msgr2=0x5610cb2f6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.826+0000 7fc5feffd640 1 --2- 172.21.15.120:0/1542893443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610cb2e82e0 0x5610cb2f6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.826+0000 7fc5feffd640 1 -- 172.21.15.120:0/1542893443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610cb2ed370 msgr2=0x5610cb2f7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.826+0000 7fc5feffd640 1 --2- 172.21.15.120:0/1542893443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610cb2ed370 0x5610cb2f7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.826+0000 7fc5feffd640 1 -- 172.21.15.120:0/1542893443 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610cb2f80d0 con 0x5610cb2e79a0 2024-09-19T09:19:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.826+0000 7fc5feffd640 1 --2- 172.21.15.120:0/1542893443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cb2e79a0 0x5610cb2f6570 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000c6a0 tx=0x7fc5f00bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b56eadbe188469 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.826+0000 7fc5e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:19:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.826+0000 7fc5e2ff5640 1 -- 172.21.15.120:0/1542893443 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f009c320 con 0x5610cb2e79a0 2024-09-19T09:19:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.826+0000 7fc6085dd740 1 -- 172.21.15.120:0/1542893443 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5f8036040 con 0x5610cb2e79a0 2024-09-19T09:19:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.827+0000 7fc5e2ff5640 1 -- 172.21.15.120:0/1542893443 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc5f0010ac0 con 0x5610cb2e79a0 2024-09-19T09:19:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.827+0000 7fc5e2ff5640 1 -- 172.21.15.120:0/1542893443 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc5f00a0620 con 0x5610cb2e79a0 2024-09-19T09:19:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.827+0000 7fc6085dd740 10 client.21081 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.827+0000 7fc6085dd740 1 -- 172.21.15.120:0/1542893443 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5610cb3684c0 con 0x5610cb2e79a0 2024-09-19T09:19:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.827+0000 7fc5e2ff5640 1 -- 172.21.15.120:0/1542893443 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f8036040 con 0x5610cb2e79a0 2024-09-19T09:19:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.827+0000 7fc5e2ff5640 1 -- 172.21.15.120:0/1542893443 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fc5f00bc0b0 con 0x5610cb2e79a0 2024-09-19T09:19:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.827+0000 7fc6085dd740 10 client.21081 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.827+0000 7fc6085dd740 10 client.21081 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:19:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.827+0000 7fc6085dd740 20 client.21081 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.827+0000 7fc6085dd740 1 --2- 172.21.15.120:0/1542893443 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5610cb368d00 0x5610cb36b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.827+0000 7fc6085dd740 4 client.21081 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:19:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.827+0000 7fc6085dd740 1 -- 172.21.15.120:0/1542893443 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5610cb252d70 con 0x5610cb368d00 2024-09-19T09:19:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.827+0000 7fc5fe7fc640 1 --2- 172.21.15.120:0/1542893443 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5610cb368d00 0x5610cb36b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.828+0000 7fc5e2ff5640 10 client.21081 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:39.828+0000 7fc5fe7fc640 1 --2- 172.21.15.120:0/1542893443 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5610cb368d00 0x5610cb36b120 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51351ee02828e6a5 server_cookie=9fc6fdd3836b4ab8 in_seq=0 out_seq=0 2024-09-19T09:19:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:40.294+0000 7fc5e2ff5640 1 -- 172.21.15.120:0/1542893443 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5610cb252d70 con 0x5610cb368d00 2024-09-19T09:19:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:40.294+0000 7fc5e2ff5640 10 client.21081 handle_command_reply: tid=0 2024-09-19T09:19:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:40.294+0000 7fc6085dd740 1 client.21081 shutdown 2024-09-19T09:19:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:40.294+0000 7fc6085dd740 1 -- 172.21.15.120:0/1542893443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cb2e79a0 msgr2=0x5610cb2f6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:40.294+0000 7fc6085dd740 1 --2- 172.21.15.120:0/1542893443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cb2e79a0 0x5610cb2f6570 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000c6a0 tx=0x7fc5f00bc040 comp rx=0 tx=0).stop 2024-09-19T09:19:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:40.294+0000 7fc6085dd740 1 -- 172.21.15.120:0/1542893443 shutdown_connections 2024-09-19T09:19:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:40.294+0000 7fc6085dd740 1 --2- 172.21.15.120:0/1542893443 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5610cb368d00 0x5610cb36b120 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:40.294+0000 7fc6085dd740 1 --2- 172.21.15.120:0/1542893443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610cb2ed370 0x5610cb2f7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:40.294+0000 7fc5e2ff5640 0 client.21081 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:40.294+0000 7fc6085dd740 1 --2- 172.21.15.120:0/1542893443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610cb2e82e0 0x5610cb2f6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:40.294+0000 7fc6085dd740 1 --2- 172.21.15.120:0/1542893443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610cb2e79a0 0x5610cb2f6570 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:40.294+0000 7fc6085dd740 1 -- 172.21.15.120:0/1542893443 >> 172.21.15.120:0/1542893443 conn(0x5610cb2e3010 msgr2=0x5610cb2e19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:40.294+0000 7fc6085dd740 1 -- 172.21.15.120:0/1542893443 shutdown_connections 2024-09-19T09:19:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:40.295+0000 7fc6085dd740 1 -- 172.21.15.120:0/1542893443 wait complete. 2024-09-19T09:19:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:40.295+0000 7fc6085dd740 20 client.21081 trim_cache size 0 max 16384 2024-09-19T09:19:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:19:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:19:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:19:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:19:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:40.296+0000 7fc605766640 1 -- 172.21.15.120:0/1081791619 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc5e80a32f0 msgr2=0x7fc5e80a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:40.296+0000 7fc605766640 1 --2- 172.21.15.120:0/1081791619 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc5e80a32f0 0x7fc5e80a57b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8005e30 tx=0x7fc5f803d040 comp rx=0 tx=0).stop 2024-09-19T09:19:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:40.296+0000 7fc605766640 1 -- 172.21.15.120:0/1081791619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc600102610 msgr2=0x7fc600101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:40.296+0000 7fc605766640 1 --2- 172.21.15.120:0/1081791619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc600102610 0x7fc600101f80 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000e9f0 tx=0x7fc5f000eeb0 comp rx=0 tx=0).stop 2024-09-19T09:19:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:40.296+0000 7fc605766640 1 -- 172.21.15.120:0/1081791619 shutdown_connections 2024-09-19T09:19:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:40.296+0000 7fc605766640 1 --2- 172.21.15.120:0/1081791619 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc5e80a32f0 0x7fc5e80a57b0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:40.296+0000 7fc605766640 1 --2- 172.21.15.120:0/1081791619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc600107870 0x7fc600100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:40.296+0000 7fc605766640 1 --2- 172.21.15.120:0/1081791619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc600104f40 0x7fc6001005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:40.296+0000 7fc605766640 1 --2- 172.21.15.120:0/1081791619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc600102610 0x7fc600101f80 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:40.296+0000 7fc605766640 1 -- 172.21.15.120:0/1081791619 >> 172.21.15.120:0/1081791619 conn(0x7fc6000fc1b0 msgr2=0x7fc600102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:40.296+0000 7fc605766640 1 -- 172.21.15.120:0/1081791619 shutdown_connections 2024-09-19T09:19:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:40.296+0000 7fc605766640 1 -- 172.21.15.120:0/1081791619 wait complete. 2024-09-19T09:19:40.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:40 smithi120 ceph-mon[31150]: pgmap v849: 97 pgs: 97 active+clean; 347 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 5.8 MiB/s wr, 590 op/s 2024-09-19T09:19:40.363 INFO:tasks.workunit.client.0.smithi120.stdout: 10 82124 14.91 MB/sec execute 95 sec latency 257.360 ms 2024-09-19T09:19:40.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:40 smithi196 ceph-mon[35150]: pgmap v849: 97 pgs: 97 active+clean; 347 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 5.8 MiB/s wr, 590 op/s 2024-09-19T09:19:40.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:40 smithi153 ceph-mon[35071]: pgmap v849: 97 pgs: 97 active+clean; 347 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 5.8 MiB/s wr, 590 op/s 2024-09-19T09:19:41.027 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:19:41.336 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:19:41.362 INFO:tasks.workunit.client.0.smithi120.stdout: 10 82866 15.27 MB/sec execute 96 sec latency 70.759 ms 2024-09-19T09:19:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.888+0000 7fdf9b3d4640 1 -- 172.21.15.120:0/3794951541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf94102570 msgr2=0x7fdf941029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.888+0000 7fdf9b3d4640 1 --2- 172.21.15.120:0/3794951541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf94102570 0x7fdf941029b0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fdf84009900 tx=0x7fdf8402a480 comp rx=0 tx=0).stop 2024-09-19T09:19:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.888+0000 7fdf9b3d4640 1 -- 172.21.15.120:0/3794951541 shutdown_connections 2024-09-19T09:19:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.888+0000 7fdf9b3d4640 1 --2- 172.21.15.120:0/3794951541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf94102ef0 0x7fdf9410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.888+0000 7fdf9b3d4640 1 --2- 172.21.15.120:0/3794951541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf94102570 0x7fdf941029b0 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.888+0000 7fdf9b3d4640 1 --2- 172.21.15.120:0/3794951541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf94100e20 0x7fdf94101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.888+0000 7fdf9b3d4640 1 -- 172.21.15.120:0/3794951541 >> 172.21.15.120:0/3794951541 conn(0x7fdf940fc7d0 msgr2=0x7fdf940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.889+0000 7fdf9b3d4640 1 -- 172.21.15.120:0/3794951541 shutdown_connections 2024-09-19T09:19:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.889+0000 7fdf9b3d4640 1 -- 172.21.15.120:0/3794951541 wait complete. 2024-09-19T09:19:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.889+0000 7fdf9b3d4640 1 Processor -- start 2024-09-19T09:19:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.889+0000 7fdf9b3d4640 1 -- start start 2024-09-19T09:19:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.890+0000 7fdf9b3d4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf94100e20 0x7fdf941984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.890+0000 7fdf9b3d4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf94102570 0x7fdf941989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.890+0000 7fdf9b3d4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf94102ef0 0x7fdf9419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.890+0000 7fdf9b3d4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdf94110f80 con 0x7fdf94102570 2024-09-19T09:19:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.890+0000 7fdf9b3d4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdf94110e00 con 0x7fdf94102ef0 2024-09-19T09:19:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.890+0000 7fdf9b3d4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdf94111100 con 0x7fdf94100e20 2024-09-19T09:19:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.890+0000 7fdf9896f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf94102570 0x7fdf941989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.890+0000 7fdf9896f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf94102570 0x7fdf941989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35070/0 (socket says 172.21.15.120:35070) 2024-09-19T09:19:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.890+0000 7fdf9896f640 1 -- 172.21.15.120:0/3259371267 learned_addr learned my addr 172.21.15.120:0/3259371267 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.890+0000 7fdf99170640 1 --2- 172.21.15.120:0/3259371267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf94100e20 0x7fdf941984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.890+0000 7fdf99971640 1 --2- 172.21.15.120:0/3259371267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf94102ef0 0x7fdf9419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.890+0000 7fdf9896f640 1 -- 172.21.15.120:0/3259371267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf94100e20 msgr2=0x7fdf941984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.890+0000 7fdf9896f640 1 --2- 172.21.15.120:0/3259371267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf94100e20 0x7fdf941984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.890+0000 7fdf9896f640 1 -- 172.21.15.120:0/3259371267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf94102ef0 msgr2=0x7fdf9419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.890+0000 7fdf9896f640 1 --2- 172.21.15.120:0/3259371267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf94102ef0 0x7fdf9419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.890+0000 7fdf9896f640 1 -- 172.21.15.120:0/3259371267 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9419d460 con 0x7fdf94102570 2024-09-19T09:19:41.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.890+0000 7fdf99170640 1 --2- 172.21.15.120:0/3259371267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf94100e20 0x7fdf941984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:41.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.890+0000 7fdf99971640 1 --2- 172.21.15.120:0/3259371267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf94102ef0 0x7fdf9419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:19:41.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.890+0000 7fdf9896f640 1 --2- 172.21.15.120:0/3259371267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf94102570 0x7fdf941989f0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fdf84006010 tx=0x7fdf84003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:41.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.891+0000 7fdf8a7fc640 1 -- 172.21.15.120:0/3259371267 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8403f070 con 0x7fdf94102570 2024-09-19T09:19:41.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.891+0000 7fdf8a7fc640 1 -- 172.21.15.120:0/3259371267 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf84031c30 con 0x7fdf94102570 2024-09-19T09:19:41.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.891+0000 7fdf8a7fc640 1 -- 172.21.15.120:0/3259371267 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf84035030 con 0x7fdf94102570 2024-09-19T09:19:41.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.891+0000 7fdf9b3d4640 1 -- 172.21.15.120:0/3259371267 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf9419d6f0 con 0x7fdf94102570 2024-09-19T09:19:41.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.891+0000 7fdf9b3d4640 1 -- 172.21.15.120:0/3259371267 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf9419db80 con 0x7fdf94102570 2024-09-19T09:19:41.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.894+0000 7fdf9d7ad740 1 Processor -- start 2024-09-19T09:19:41.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.894+0000 7fdf9d7ad740 1 -- start start 2024-09-19T09:19:41.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.894+0000 7fdf9d7ad740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c36dd22a20 0x55c36dd21340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:41.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.894+0000 7fdf9d7ad740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c36dd21880 0x55c36dd21cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:41.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.894+0000 7fdf9d7ad740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c36dd34d20 0x55c36dd37110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:41.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.894+0000 7fdf9d7ad740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c36dc872e0 con 0x55c36dd21880 2024-09-19T09:19:41.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.894+0000 7fdf9d7ad740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c36dc2ece0 con 0x55c36dd22a20 2024-09-19T09:19:41.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.894+0000 7fdf9d7ad740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c36dc832a0 con 0x55c36dd34d20 2024-09-19T09:19:41.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.894+0000 7fdf8a7fc640 1 -- 172.21.15.120:0/3259371267 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fdf84042070 con 0x7fdf94102570 2024-09-19T09:19:41.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.894+0000 7fdf99971640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c36dd22a20 0x55c36dd21340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:41.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.894+0000 7fdf99170640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c36dd21880 0x55c36dd21cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:41.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.894+0000 7fdf99971640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c36dd22a20 0x55c36dd21340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:43788/0 (socket says 172.21.15.120:43788) 2024-09-19T09:19:41.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.894+0000 7fdf99971640 1 -- 172.21.15.120:0/2225710737 learned_addr learned my addr 172.21.15.120:0/2225710737 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:41.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.895+0000 7fdf9896f640 1 --2- 172.21.15.120:0/2225710737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c36dd34d20 0x55c36dd37110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:41.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.895+0000 7fdf99170640 1 -- 172.21.15.120:0/2225710737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c36dd34d20 msgr2=0x55c36dd37110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:41.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.895+0000 7fdf99170640 1 --2- 172.21.15.120:0/2225710737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c36dd34d20 0x55c36dd37110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.895+0000 7fdf99170640 1 -- 172.21.15.120:0/2225710737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c36dd22a20 msgr2=0x55c36dd21340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:41.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.895+0000 7fdf99170640 1 --2- 172.21.15.120:0/2225710737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c36dd22a20 0x55c36dd21340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.895+0000 7fdf99170640 1 -- 172.21.15.120:0/2225710737 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c36dc13440 con 0x55c36dd21880 2024-09-19T09:19:41.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.895+0000 7fdf9896f640 1 --2- 172.21.15.120:0/2225710737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c36dd34d20 0x55c36dd37110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:19:41.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.895+0000 7fdf99971640 1 --2- 172.21.15.120:0/2225710737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c36dd22a20 0x55c36dd21340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:41.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.895+0000 7fdf99170640 1 --2- 172.21.15.120:0/2225710737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c36dd21880 0x55c36dd21cc0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fdf90009fd0 tx=0x7fdf900118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dfb21f9ccfc7953 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:41.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.895+0000 7fdf7f7fe640 1 -- 172.21.15.120:0/2225710737 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90004030 con 0x55c36dd21880 2024-09-19T09:19:41.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.895+0000 7fdf8a7fc640 1 --2- 172.21.15.120:0/3259371267 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdf800a32c0 0x7fdf800a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:41.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.895+0000 7fdf7f7fe640 1 -- 172.21.15.120:0/2225710737 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf90005500 con 0x55c36dd21880 2024-09-19T09:19:41.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.895+0000 7fdf7f7fe640 1 -- 172.21.15.120:0/2225710737 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90004d10 con 0x55c36dd21880 2024-09-19T09:19:41.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.895+0000 7fdf99971640 1 --2- 172.21.15.120:0/3259371267 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdf800a32c0 0x7fdf800a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:41.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.895+0000 7fdf8a7fc640 1 -- 172.21.15.120:0/3259371267 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fdf840eff70 con 0x7fdf94102570 2024-09-19T09:19:41.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.896+0000 7fdf9d7ad740 1 -- 172.21.15.120:0/2225710737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c36dd21880 msgr2=0x55c36dd21cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:41.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.896+0000 7fdf99971640 1 --2- 172.21.15.120:0/3259371267 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdf800a32c0 0x7fdf800a5780 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c005c40 tx=0x7fdf8c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:41.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.896+0000 7fdf9d7ad740 1 --2- 172.21.15.120:0/2225710737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c36dd21880 0x55c36dd21cc0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fdf90009fd0 tx=0x7fdf900118d0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.897+0000 7fdf9d7ad740 1 -- 172.21.15.120:0/2225710737 shutdown_connections 2024-09-19T09:19:41.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.897+0000 7fdf9d7ad740 1 --2- 172.21.15.120:0/2225710737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c36dd34d20 0x55c36dd37110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.897+0000 7fdf9d7ad740 1 --2- 172.21.15.120:0/2225710737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c36dd21880 0x55c36dd21cc0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.897+0000 7fdf9d7ad740 1 --2- 172.21.15.120:0/2225710737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c36dd22a20 0x55c36dd21340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.897+0000 7fdf9d7ad740 1 -- 172.21.15.120:0/2225710737 >> 172.21.15.120:0/2225710737 conn(0x55c36dd1da30 msgr2=0x55c36dd1de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:41.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.897+0000 7fdf9d7ad740 1 -- 172.21.15.120:0/2225710737 shutdown_connections 2024-09-19T09:19:41.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.897+0000 7fdf9d7ad740 1 -- 172.21.15.120:0/2225710737 wait complete. 2024-09-19T09:19:41.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.897+0000 7fdf9d7ad740 1 -- start start 2024-09-19T09:19:41.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.897+0000 7fdf9d7ad740 1 Processor -- start 2024-09-19T09:19:41.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.898+0000 7fdf9d7ad740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c36dd21880 0x55c36dd27b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:41.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.898+0000 7fdf9d7ad740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c36dd22a20 0x55c36dd24df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:41.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.898+0000 7fdf9d7ad740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c36dd34d20 0x55c36dd25370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:41.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.898+0000 7fdf9d7ad740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c36dd37a50 con 0x55c36dd22a20 2024-09-19T09:19:41.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.898+0000 7fdf9d7ad740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c36dc42050 con 0x55c36dd21880 2024-09-19T09:19:41.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.898+0000 7fdf9d7ad740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c36dd37bd0 con 0x55c36dd34d20 2024-09-19T09:19:41.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.898+0000 7fdf99170640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c36dd21880 0x55c36dd27b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:41.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.898+0000 7fdf99170640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c36dd21880 0x55c36dd27b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:43800/0 (socket says 172.21.15.120:43800) 2024-09-19T09:19:41.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.898+0000 7fdf99170640 1 -- 172.21.15.120:0/2217532898 learned_addr learned my addr 172.21.15.120:0/2217532898 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:41.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.898+0000 7fdf9d7ad740 10 client.0 _ll_register_callbacks cb 0x55c36dc82160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:41.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.898+0000 7fdf99971640 1 --2- 172.21.15.120:0/2217532898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c36dd34d20 0x55c36dd25370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:41.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.899+0000 7fdf99170640 1 -- 172.21.15.120:0/2217532898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c36dd34d20 msgr2=0x55c36dd25370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:41.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.899+0000 7fdf99170640 1 --2- 172.21.15.120:0/2217532898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c36dd34d20 0x55c36dd25370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.899+0000 7fdf99170640 1 -- 172.21.15.120:0/2217532898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c36dd22a20 msgr2=0x55c36dd24df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:19:41.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.899+0000 7fdf99170640 1 --2- 172.21.15.120:0/2217532898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c36dd22a20 0x55c36dd24df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.899+0000 7fdf99170640 1 -- 172.21.15.120:0/2217532898 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c36d9f83c0 con 0x55c36dd21880 2024-09-19T09:19:41.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.900+0000 7fdf99170640 1 --2- 172.21.15.120:0/2217532898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c36dd21880 0x55c36dd27b40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdf9001b040 tx=0x7fdf9000fcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c923656e4125fa9a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:41.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.900+0000 7fdf7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:19:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.900+0000 7fdf7cff9640 1 -- 172.21.15.120:0/2217532898 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90029020 con 0x55c36dd21880 2024-09-19T09:19:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.900+0000 7fdf9d7ad740 1 -- 172.21.15.120:0/2217532898 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf90005500 con 0x55c36dd21880 2024-09-19T09:19:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.900+0000 7fdf7cff9640 1 -- 172.21.15.120:0/2217532898 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fdf90004c60 con 0x55c36dd21880 2024-09-19T09:19:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.900+0000 7fdf7cff9640 1 -- 172.21.15.120:0/2217532898 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdf9000fea0 con 0x55c36dd21880 2024-09-19T09:19:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.900+0000 7fdf9d7ad740 10 client.29951 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.900+0000 7fdf9d7ad740 1 -- 172.21.15.120:0/2217532898 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c36dd43490 con 0x55c36dd21880 2024-09-19T09:19:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.901+0000 7fdf7cff9640 1 -- 172.21.15.120:0/2217532898 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf90005500 con 0x55c36dd21880 2024-09-19T09:19:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.901+0000 7fdf7cff9640 1 -- 172.21.15.120:0/2217532898 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fdf90025710 con 0x55c36dd21880 2024-09-19T09:19:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.901+0000 7fdf9d7ad740 10 client.29951 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.901+0000 7fdf9d7ad740 10 client.29951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:19:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.901+0000 7fdf9d7ad740 20 client.29951 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.901+0000 7fdf9d7ad740 1 --2- 172.21.15.120:0/2217532898 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c36dd44840 0x55c36dd46c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.901+0000 7fdf9d7ad740 4 client.29951 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:19:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.901+0000 7fdf9d7ad740 1 -- 172.21.15.120:0/2217532898 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c36dc94850 con 0x55c36dd44840 2024-09-19T09:19:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.901+0000 7fdf9896f640 1 --2- 172.21.15.120:0/2217532898 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c36dd44840 0x55c36dd46c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.902+0000 7fdf9896f640 1 --2- 172.21.15.120:0/2217532898 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c36dd44840 0x55c36dd46c30 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51187c5678b58e49 server_cookie=8be13a03afccea89 in_seq=0 out_seq=0 2024-09-19T09:19:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.902+0000 7fdf7cff9640 10 client.29951 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.909+0000 7fdf7cff9640 1 -- 172.21.15.120:0/2217532898 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c36dc94850 con 0x55c36dd44840 2024-09-19T09:19:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.909+0000 7fdf7cff9640 10 client.29951 handle_command_reply: tid=0 2024-09-19T09:19:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.909+0000 7fdf9d7ad740 1 client.29951 shutdown 2024-09-19T09:19:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.909+0000 7fdf9d7ad740 1 -- 172.21.15.120:0/2217532898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c36dd21880 msgr2=0x55c36dd27b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.909+0000 7fdf9d7ad740 1 --2- 172.21.15.120:0/2217532898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c36dd21880 0x55c36dd27b40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdf9001b040 tx=0x7fdf9000fcb0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.909+0000 7fdf9d7ad740 1 -- 172.21.15.120:0/2217532898 shutdown_connections 2024-09-19T09:19:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.909+0000 7fdf9d7ad740 1 --2- 172.21.15.120:0/2217532898 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c36dd44840 0x55c36dd46c30 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.909+0000 7fdf9d7ad740 1 --2- 172.21.15.120:0/2217532898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c36dd34d20 0x55c36dd25370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.909+0000 7fdf9d7ad740 1 --2- 172.21.15.120:0/2217532898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c36dd22a20 0x55c36dd24df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.909+0000 7fdf9d7ad740 1 --2- 172.21.15.120:0/2217532898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c36dd21880 0x55c36dd27b40 secure :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdf9001b040 tx=0x7fdf9000fcb0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.909+0000 7fdf9d7ad740 1 -- 172.21.15.120:0/2217532898 >> 172.21.15.120:0/2217532898 conn(0x55c36dd1da30 msgr2=0x55c36dd23750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.910+0000 7fdf7cff9640 0 client.29951 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.910+0000 7fdf9d7ad740 1 -- 172.21.15.120:0/2217532898 shutdown_connections 2024-09-19T09:19:41.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.910+0000 7fdf9d7ad740 1 -- 172.21.15.120:0/2217532898 wait complete. 2024-09-19T09:19:41.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.910+0000 7fdf9d7ad740 20 client.29951 trim_cache size 0 max 16384 2024-09-19T09:19:41.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.937+0000 7fdf9d7ad740 1 Processor -- start 2024-09-19T09:19:41.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.937+0000 7fdf9d7ad740 1 -- start start 2024-09-19T09:19:41.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.937+0000 7fdf9d7ad740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c36dd1da30 0x55c36dd40d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:41.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.937+0000 7fdf9d7ad740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c36dd22a20 0x55c36dd30d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:41.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.937+0000 7fdf9d7ad740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c36dd34d20 0x55c36dd31240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:41.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.937+0000 7fdf9d7ad740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdf941ab360 con 0x55c36dd34d20 2024-09-19T09:19:41.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.937+0000 7fdf9d7ad740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdf941acea0 con 0x55c36dd1da30 2024-09-19T09:19:41.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.937+0000 7fdf9d7ad740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdf941aba30 con 0x55c36dd22a20 2024-09-19T09:19:41.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.937+0000 7fdf99971640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c36dd34d20 0x55c36dd31240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:41.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.937+0000 7fdf99170640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c36dd1da30 0x55c36dd40d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:41.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.937+0000 7fdf9896f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c36dd22a20 0x55c36dd30d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:41.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.937+0000 7fdf99971640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c36dd34d20 0x55c36dd31240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35108/0 (socket says 172.21.15.120:35108) 2024-09-19T09:19:41.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.937+0000 7fdf99971640 1 -- 172.21.15.120:0/1153834213 learned_addr learned my addr 172.21.15.120:0/1153834213 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:41.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.938+0000 7fdf99971640 1 -- 172.21.15.120:0/1153834213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c36dd22a20 msgr2=0x55c36dd30d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:41.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.938+0000 7fdf99971640 1 --2- 172.21.15.120:0/1153834213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c36dd22a20 0x55c36dd30d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.938+0000 7fdf99971640 1 -- 172.21.15.120:0/1153834213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c36dd1da30 msgr2=0x55c36dd40d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:41.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.938+0000 7fdf99971640 1 --2- 172.21.15.120:0/1153834213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c36dd1da30 0x55c36dd40d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.938+0000 7fdf99971640 1 -- 172.21.15.120:0/1153834213 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c36dd27eb0 con 0x55c36dd34d20 2024-09-19T09:19:41.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.938+0000 7fdf99971640 1 --2- 172.21.15.120:0/1153834213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c36dd34d20 0x55c36dd31240 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c010b00 tx=0x7fdf8c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee2a3cc675873ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:41.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.938+0000 7fdf7effd640 1 -- 172.21.15.120:0/1153834213 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c004b20 con 0x55c36dd34d20 2024-09-19T09:19:41.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.938+0000 7fdf7effd640 1 -- 172.21.15.120:0/1153834213 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf8c004cc0 con 0x55c36dd34d20 2024-09-19T09:19:41.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.938+0000 7fdf7effd640 1 -- 172.21.15.120:0/1153834213 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c005930 con 0x55c36dd34d20 2024-09-19T09:19:41.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.939+0000 7fdf9d7ad740 1 -- 172.21.15.120:0/1153834213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c36dd34d20 msgr2=0x55c36dd31240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:41.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.939+0000 7fdf9d7ad740 1 --2- 172.21.15.120:0/1153834213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c36dd34d20 0x55c36dd31240 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c010b00 tx=0x7fdf8c019000 comp rx=0 tx=0).stop 2024-09-19T09:19:41.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.940+0000 7fdf9d7ad740 1 -- 172.21.15.120:0/1153834213 shutdown_connections 2024-09-19T09:19:41.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.940+0000 7fdf9d7ad740 1 --2- 172.21.15.120:0/1153834213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c36dd34d20 0x55c36dd31240 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.940+0000 7fdf9d7ad740 1 --2- 172.21.15.120:0/1153834213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c36dd22a20 0x55c36dd30d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.940+0000 7fdf9d7ad740 1 --2- 172.21.15.120:0/1153834213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c36dd1da30 0x55c36dd40d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.940+0000 7fdf9d7ad740 1 -- 172.21.15.120:0/1153834213 >> 172.21.15.120:0/1153834213 conn(0x55c36dd369a0 msgr2=0x55c36dd46850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:41.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.940+0000 7fdf9d7ad740 1 -- 172.21.15.120:0/1153834213 shutdown_connections 2024-09-19T09:19:41.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.940+0000 7fdf9d7ad740 1 -- 172.21.15.120:0/1153834213 wait complete. 2024-09-19T09:19:41.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.940+0000 7fdf9d7ad740 1 -- start start 2024-09-19T09:19:41.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.941+0000 7fdf9d7ad740 1 Processor -- start 2024-09-19T09:19:41.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.941+0000 7fdf9d7ad740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c36dd1da30 0x55c36dd3d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:41.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.941+0000 7fdf9d7ad740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c36dd22a20 0x55c36dd4bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:41.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.942+0000 7fdf99170640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c36dd1da30 0x55c36dd3d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:41.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.942+0000 7fdf99170640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c36dd1da30 0x55c36dd3d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:43816/0 (socket says 172.21.15.120:43816) 2024-09-19T09:19:41.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.942+0000 7fdf9d7ad740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c36dd34d20 0x55c36dd4c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:41.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.942+0000 7fdf9d7ad740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdf941ac7d0 con 0x55c36dd34d20 2024-09-19T09:19:41.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.942+0000 7fdf9d7ad740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdf941ac100 con 0x55c36dd1da30 2024-09-19T09:19:41.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.942+0000 7fdf99971640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c36dd34d20 0x55c36dd4c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:41.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.942+0000 7fdf99971640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c36dd34d20 0x55c36dd4c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35114/0 (socket says 172.21.15.120:35114) 2024-09-19T09:19:41.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.942+0000 7fdf99971640 1 -- 172.21.15.120:0/1540743017 learned_addr learned my addr 172.21.15.120:0/1540743017 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:41.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.942+0000 7fdf9896f640 1 --2- 172.21.15.120:0/1540743017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c36dd22a20 0x55c36dd4bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:41.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.942+0000 7fdf9d7ad740 1 -- 172.21.15.120:0/1540743017 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdf941a9320 con 0x55c36dd22a20 2024-09-19T09:19:41.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.942+0000 7fdf9d7ad740 10 client.0 _ll_register_callbacks cb 0x7fdf8c009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:41.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.943+0000 7fdf99971640 1 -- 172.21.15.120:0/1540743017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c36dd22a20 msgr2=0x55c36dd4bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:41.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.943+0000 7fdf99971640 1 --2- 172.21.15.120:0/1540743017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c36dd22a20 0x55c36dd4bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.943+0000 7fdf99971640 1 -- 172.21.15.120:0/1540743017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c36dd1da30 msgr2=0x55c36dd3d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:41.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.943+0000 7fdf99971640 1 --2- 172.21.15.120:0/1540743017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c36dd1da30 0x55c36dd3d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.943+0000 7fdf99971640 1 -- 172.21.15.120:0/1540743017 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c36dd256e0 con 0x55c36dd34d20 2024-09-19T09:19:41.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.943+0000 7fdf99971640 1 --2- 172.21.15.120:0/1540743017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c36dd34d20 0x55c36dd4c360 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c018040 tx=0x7fdf8c009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d246e611903cddbd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:41.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.943+0000 7fdf7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:19:41.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.943+0000 7fdf7cff9640 1 -- 172.21.15.120:0/1540743017 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c036060 con 0x55c36dd34d20 2024-09-19T09:19:41.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.944+0000 7fdf9d7ad740 1 -- 172.21.15.120:0/1540743017 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf8c004cc0 con 0x55c36dd34d20 2024-09-19T09:19:41.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.944+0000 7fdf7cff9640 1 -- 172.21.15.120:0/1540743017 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fdf8c005670 con 0x55c36dd34d20 2024-09-19T09:19:41.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.944+0000 7fdf7cff9640 1 -- 172.21.15.120:0/1540743017 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdf8c005870 con 0x55c36dd34d20 2024-09-19T09:19:41.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.944+0000 7fdf9d7ad740 10 client.21096 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:41.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.944+0000 7fdf9d7ad740 1 -- 172.21.15.120:0/1540743017 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c36ddaf340 con 0x55c36dd34d20 2024-09-19T09:19:41.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.944+0000 7fdf7cff9640 1 -- 172.21.15.120:0/1540743017 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf8c004cc0 con 0x55c36dd34d20 2024-09-19T09:19:41.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.944+0000 7fdf7cff9640 1 -- 172.21.15.120:0/1540743017 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fdf8c03b070 con 0x55c36dd34d20 2024-09-19T09:19:41.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.944+0000 7fdf9d7ad740 10 client.21096 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:41.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.944+0000 7fdf9d7ad740 10 client.21096 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:19:41.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.944+0000 7fdf9d7ad740 20 client.21096 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:41.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.944+0000 7fdf9d7ad740 1 --2- 172.21.15.120:0/1540743017 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c36ddaf850 0x55c36ddb1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:41.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.944+0000 7fdf9d7ad740 4 client.21096 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:19:41.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.945+0000 7fdf9d7ad740 1 -- 172.21.15.120:0/1540743017 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c36dc8ed60 con 0x55c36ddaf850 2024-09-19T09:19:41.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.945+0000 7fdf99170640 1 --2- 172.21.15.120:0/1540743017 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c36ddaf850 0x55c36ddb1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:41.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.945+0000 7fdf99170640 1 --2- 172.21.15.120:0/1540743017 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c36ddaf850 0x55c36ddb1c70 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c6821c47f61af17 server_cookie=8ab7741ff9764b15 in_seq=0 out_seq=0 2024-09-19T09:19:41.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.945+0000 7fdf7cff9640 10 client.21096 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:41.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.954+0000 7fdf7cff9640 1 -- 172.21.15.120:0/1540743017 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c36dc8ed60 con 0x55c36ddaf850 2024-09-19T09:19:41.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.954+0000 7fdf7cff9640 10 client.21096 handle_command_reply: tid=0 2024-09-19T09:19:41.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.954+0000 7fdf9d7ad740 1 client.21096 shutdown 2024-09-19T09:19:41.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.954+0000 7fdf9d7ad740 1 -- 172.21.15.120:0/1540743017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c36dd34d20 msgr2=0x55c36dd4c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:41.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.954+0000 7fdf9d7ad740 1 --2- 172.21.15.120:0/1540743017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c36dd34d20 0x55c36dd4c360 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c018040 tx=0x7fdf8c009ce0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.954+0000 7fdf9d7ad740 1 -- 172.21.15.120:0/1540743017 shutdown_connections 2024-09-19T09:19:41.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.954+0000 7fdf9d7ad740 1 --2- 172.21.15.120:0/1540743017 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c36ddaf850 0x55c36ddb1c70 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.954+0000 7fdf9d7ad740 1 --2- 172.21.15.120:0/1540743017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c36dd34d20 0x55c36dd4c360 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.954+0000 7fdf9d7ad740 1 --2- 172.21.15.120:0/1540743017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c36dd22a20 0x55c36dd4bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.954+0000 7fdf9d7ad740 1 --2- 172.21.15.120:0/1540743017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c36dd1da30 0x55c36dd3d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.954+0000 7fdf7cff9640 0 client.21096 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:41.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.954+0000 7fdf9d7ad740 1 -- 172.21.15.120:0/1540743017 >> 172.21.15.120:0/1540743017 conn(0x55c36dd369a0 msgr2=0x55c36dd46850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:41.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.954+0000 7fdf9d7ad740 1 -- 172.21.15.120:0/1540743017 shutdown_connections 2024-09-19T09:19:41.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.954+0000 7fdf9d7ad740 1 -- 172.21.15.120:0/1540743017 wait complete. 2024-09-19T09:19:41.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.954+0000 7fdf9d7ad740 20 client.21096 trim_cache size 0 max 16384 2024-09-19T09:19:41.959 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:19:41.960 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:19:41.960 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:19:41.960 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:19:41.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.955+0000 7fdf9b3d4640 1 -- 172.21.15.120:0/3259371267 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdf800a32c0 msgr2=0x7fdf800a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:41.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.955+0000 7fdf9b3d4640 1 --2- 172.21.15.120:0/3259371267 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdf800a32c0 0x7fdf800a5780 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c005c40 tx=0x7fdf8c009500 comp rx=0 tx=0).stop 2024-09-19T09:19:41.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.955+0000 7fdf9b3d4640 1 -- 172.21.15.120:0/3259371267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf94102570 msgr2=0x7fdf941989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:41.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.955+0000 7fdf9b3d4640 1 --2- 172.21.15.120:0/3259371267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf94102570 0x7fdf941989f0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fdf84006010 tx=0x7fdf84003c10 comp rx=0 tx=0).stop 2024-09-19T09:19:41.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.955+0000 7fdf9b3d4640 1 -- 172.21.15.120:0/3259371267 shutdown_connections 2024-09-19T09:19:41.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.955+0000 7fdf9b3d4640 1 --2- 172.21.15.120:0/3259371267 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdf800a32c0 0x7fdf800a5780 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.955+0000 7fdf9b3d4640 1 --2- 172.21.15.120:0/3259371267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf94102ef0 0x7fdf9419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.955+0000 7fdf9b3d4640 1 --2- 172.21.15.120:0/3259371267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf94102570 0x7fdf941989f0 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.955+0000 7fdf9b3d4640 1 --2- 172.21.15.120:0/3259371267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf94100e20 0x7fdf941984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:41.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.955+0000 7fdf9b3d4640 1 -- 172.21.15.120:0/3259371267 >> 172.21.15.120:0/3259371267 conn(0x7fdf940fc7d0 msgr2=0x7fdf9406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:41.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.955+0000 7fdf9b3d4640 1 -- 172.21.15.120:0/3259371267 shutdown_connections 2024-09-19T09:19:41.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:41.955+0000 7fdf9b3d4640 1 -- 172.21.15.120:0/3259371267 wait complete. 2024-09-19T09:19:42.363 INFO:tasks.workunit.client.0.smithi120.stdout: 10 83235 15.24 MB/sec execute 97 sec latency 251.411 ms 2024-09-19T09:19:43.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:42 smithi120 ceph-mon[31150]: pgmap v850: 97 pgs: 97 active+clean; 353 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 7.1 MiB/s wr, 664 op/s 2024-09-19T09:19:43.362 INFO:tasks.workunit.client.0.smithi120.stdout: 10 83678 15.33 MB/sec execute 98 sec latency 305.334 ms 2024-09-19T09:19:43.412 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:19:43.412 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:19:43.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:42 smithi196 ceph-mon[35150]: pgmap v850: 97 pgs: 97 active+clean; 353 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 7.1 MiB/s wr, 664 op/s 2024-09-19T09:19:43.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:42 smithi153 ceph-mon[35071]: pgmap v850: 97 pgs: 97 active+clean; 353 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 7.1 MiB/s wr, 664 op/s 2024-09-19T09:19:43.704 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:19:44.363 INFO:tasks.workunit.client.0.smithi120.stdout: 10 84201 15.44 MB/sec execute 99 sec latency 75.651 ms 2024-09-19T09:19:44.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.375+0000 7f4f1c180640 1 -- 172.21.15.120:0/2089716147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f14102ef0 msgr2=0x7f4f1410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:44.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.375+0000 7f4f1c180640 1 --2- 172.21.15.120:0/2089716147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f14102ef0 0x7f4f1410da40 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c009900 tx=0x7f4f0c02a480 comp rx=0 tx=0).stop 2024-09-19T09:19:44.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.375+0000 7f4f1c180640 1 -- 172.21.15.120:0/2089716147 shutdown_connections 2024-09-19T09:19:44.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.375+0000 7f4f1c180640 1 --2- 172.21.15.120:0/2089716147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f14102ef0 0x7f4f1410da40 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:44.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.375+0000 7f4f1c180640 1 --2- 172.21.15.120:0/2089716147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f14102570 0x7f4f141029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:44.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.375+0000 7f4f1c180640 1 --2- 172.21.15.120:0/2089716147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f14100e20 0x7f4f14101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:44.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.375+0000 7f4f1c180640 1 -- 172.21.15.120:0/2089716147 >> 172.21.15.120:0/2089716147 conn(0x7f4f140fc7d0 msgr2=0x7f4f140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:44.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.375+0000 7f4f1c180640 1 -- 172.21.15.120:0/2089716147 shutdown_connections 2024-09-19T09:19:44.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.375+0000 7f4f1c180640 1 -- 172.21.15.120:0/2089716147 wait complete. 2024-09-19T09:19:44.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.376+0000 7f4f1c180640 1 Processor -- start 2024-09-19T09:19:44.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.376+0000 7f4f1c180640 1 -- start start 2024-09-19T09:19:44.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.376+0000 7f4f1c180640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f14100e20 0x7f4f14198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:44.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.376+0000 7f4f1c180640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f14102570 0x7f4f14198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:44.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.376+0000 7f4f1c180640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f14102ef0 0x7f4f1419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:44.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.376+0000 7f4f1c180640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4f14110f80 con 0x7f4f14102570 2024-09-19T09:19:44.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.376+0000 7f4f1c180640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4f14110e00 con 0x7f4f14102ef0 2024-09-19T09:19:44.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.376+0000 7f4f1c180640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f14111100 con 0x7f4f14100e20 2024-09-19T09:19:44.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.376+0000 7f4f1971b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f14102570 0x7f4f14198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:44.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.376+0000 7f4f1971b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f14102570 0x7f4f14198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35126/0 (socket says 172.21.15.120:35126) 2024-09-19T09:19:44.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.376+0000 7f4f1971b640 1 -- 172.21.15.120:0/3064803686 learned_addr learned my addr 172.21.15.120:0/3064803686 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:44.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.377+0000 7f4f19f1c640 1 --2- 172.21.15.120:0/3064803686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f14100e20 0x7f4f14198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:44.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.377+0000 7f4f1a71d640 1 --2- 172.21.15.120:0/3064803686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f14102ef0 0x7f4f1419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:44.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.377+0000 7f4f1971b640 1 -- 172.21.15.120:0/3064803686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f14100e20 msgr2=0x7f4f14198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:44.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.377+0000 7f4f1971b640 1 --2- 172.21.15.120:0/3064803686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f14100e20 0x7f4f14198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:44.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.377+0000 7f4f1971b640 1 -- 172.21.15.120:0/3064803686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f14102ef0 msgr2=0x7f4f1419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:44.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.377+0000 7f4f1971b640 1 --2- 172.21.15.120:0/3064803686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f14102ef0 0x7f4f1419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:44.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.377+0000 7f4f1971b640 1 -- 172.21.15.120:0/3064803686 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1419d4b0 con 0x7f4f14102570 2024-09-19T09:19:44.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.377+0000 7f4f19f1c640 1 --2- 172.21.15.120:0/3064803686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f14100e20 0x7f4f14198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:44.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.377+0000 7f4f1971b640 1 --2- 172.21.15.120:0/3064803686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f14102570 0x7f4f14198a40 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400e9f0 tx=0x7f4f0400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:44.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.377+0000 7f4f0affd640 1 -- 172.21.15.120:0/3064803686 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0400cd80 con 0x7f4f14102570 2024-09-19T09:19:44.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.377+0000 7f4f0affd640 1 -- 172.21.15.120:0/3064803686 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f04005500 con 0x7f4f14102570 2024-09-19T09:19:44.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.377+0000 7f4f1c180640 1 -- 172.21.15.120:0/3064803686 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f1419d7a0 con 0x7f4f14102570 2024-09-19T09:19:44.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.377+0000 7f4f0affd640 1 -- 172.21.15.120:0/3064803686 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f04010690 con 0x7f4f14102570 2024-09-19T09:19:44.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.377+0000 7f4f1c180640 1 -- 172.21.15.120:0/3064803686 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f140fcbb0 con 0x7f4f14102570 2024-09-19T09:19:44.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.379+0000 7f4f1c180640 1 -- 172.21.15.120:0/3064803686 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f14101200 con 0x7f4f14102570 2024-09-19T09:19:44.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.380+0000 7f4f0affd640 1 -- 172.21.15.120:0/3064803686 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f4f040046c0 con 0x7f4f14102570 2024-09-19T09:19:44.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.380+0000 7f4f0affd640 1 --2- 172.21.15.120:0/3064803686 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4f000a32c0 0x7f4f000a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:44.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.381+0000 7f4f0affd640 1 -- 172.21.15.120:0/3064803686 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f4f040cfcf0 con 0x7f4f14102570 2024-09-19T09:19:44.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.381+0000 7f4f19f1c640 1 --2- 172.21.15.120:0/3064803686 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4f000a32c0 0x7f4f000a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:44.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.381+0000 7f4f19f1c640 1 --2- 172.21.15.120:0/3064803686 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4f000a32c0 0x7f4f000a5780 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4f10005fd0 tx=0x7f4f10009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:44.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.385+0000 7f4f0affd640 1 -- 172.21.15.120:0/3064803686 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4f0409c9d0 con 0x7f4f14102570 2024-09-19T09:19:44.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.629+0000 7f4f1c180640 1 -- 172.21.15.120:0/3064803686 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f1410a0a0 con 0x7f4f14102570 2024-09-19T09:19:44.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.629+0000 7f4f0affd640 1 -- 172.21.15.120:0/3064803686 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11988 (secure 0 0 0) 0x7f4f0409c2d0 con 0x7f4f14102570 2024-09-19T09:19:44.631 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:19:44.631 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":39,"btime":"2024-09-19T09:18:53:120858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:18:53.070639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:19:44.631 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 39 2024-09-19T09:19:44.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.632+0000 7f4f1c180640 1 -- 172.21.15.120:0/3064803686 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4f000a32c0 msgr2=0x7f4f000a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:44.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.632+0000 7f4f1c180640 1 --2- 172.21.15.120:0/3064803686 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4f000a32c0 0x7f4f000a5780 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4f10005fd0 tx=0x7f4f10009500 comp rx=0 tx=0).stop 2024-09-19T09:19:44.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.632+0000 7f4f1c180640 1 -- 172.21.15.120:0/3064803686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f14102570 msgr2=0x7f4f14198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:44.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.632+0000 7f4f1c180640 1 --2- 172.21.15.120:0/3064803686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f14102570 0x7f4f14198a40 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400e9f0 tx=0x7f4f0400edf0 comp rx=0 tx=0).stop 2024-09-19T09:19:44.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.632+0000 7f4f1c180640 1 -- 172.21.15.120:0/3064803686 shutdown_connections 2024-09-19T09:19:44.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.632+0000 7f4f1c180640 1 --2- 172.21.15.120:0/3064803686 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4f000a32c0 0x7f4f000a5780 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:44.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.632+0000 7f4f1c180640 1 --2- 172.21.15.120:0/3064803686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f14102ef0 0x7f4f1419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:44.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.632+0000 7f4f1c180640 1 --2- 172.21.15.120:0/3064803686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f14102570 0x7f4f14198a40 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:44.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.632+0000 7f4f1c180640 1 --2- 172.21.15.120:0/3064803686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f14100e20 0x7f4f14198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:44.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.632+0000 7f4f1c180640 1 -- 172.21.15.120:0/3064803686 >> 172.21.15.120:0/3064803686 conn(0x7f4f140fc7d0 msgr2=0x7f4f1406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:44.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.632+0000 7f4f1c180640 1 -- 172.21.15.120:0/3064803686 shutdown_connections 2024-09-19T09:19:44.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:44.632+0000 7f4f1c180640 1 -- 172.21.15.120:0/3064803686 wait complete. 2024-09-19T09:19:45.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:44 smithi120 ceph-mon[31150]: pgmap v851: 97 pgs: 97 active+clean; 340 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.0 MiB/s wr, 730 op/s 2024-09-19T09:19:45.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:44 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3064803686' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:19:45.363 INFO:tasks.workunit.client.0.smithi120.stdout: 10 84859 15.60 MB/sec execute 100 sec latency 51.422 ms 2024-09-19T09:19:45.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:44 smithi196 ceph-mon[35150]: pgmap v851: 97 pgs: 97 active+clean; 340 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.0 MiB/s wr, 730 op/s 2024-09-19T09:19:45.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:44 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3064803686' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:19:45.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:44 smithi153 ceph-mon[35071]: pgmap v851: 97 pgs: 97 active+clean; 340 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.0 MiB/s wr, 730 op/s 2024-09-19T09:19:45.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:44 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3064803686' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:19:45.692 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:19:45.975 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:19:46.363 INFO:tasks.workunit.client.0.smithi120.stdout: 10 85559 15.80 MB/sec execute 101 sec latency 61.294 ms 2024-09-19T09:19:46.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.484+0000 7f6fe8af2640 1 -- 172.21.15.120:0/878324828 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fe4101410 msgr2=0x7f6fe410d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:46.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.484+0000 7f6fe8af2640 1 --2- 172.21.15.120:0/878324828 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fe4101410 0x7f6fe410d940 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc009cd0 tx=0x7f6fdc02a8e0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.484+0000 7f6fe8af2640 1 -- 172.21.15.120:0/878324828 shutdown_connections 2024-09-19T09:19:46.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.484+0000 7f6fe8af2640 1 --2- 172.21.15.120:0/878324828 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fe4101410 0x7f6fe410d940 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.484+0000 7f6fe8af2640 1 --2- 172.21.15.120:0/878324828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fe4100a70 0x7f6fe4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.484+0000 7f6fe8af2640 1 --2- 172.21.15.120:0/878324828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fe4106a70 0x7f6fe4106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.484+0000 7f6fe8af2640 1 -- 172.21.15.120:0/878324828 >> 172.21.15.120:0/878324828 conn(0x7f6fe40fc790 msgr2=0x7f6fe40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:46.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.484+0000 7f6fe8af2640 1 -- 172.21.15.120:0/878324828 shutdown_connections 2024-09-19T09:19:46.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.484+0000 7f6fe8af2640 1 -- 172.21.15.120:0/878324828 wait complete. 2024-09-19T09:19:46.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.485+0000 7f6fe8af2640 1 Processor -- start 2024-09-19T09:19:46.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.485+0000 7f6fe8af2640 1 -- start start 2024-09-19T09:19:46.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.485+0000 7f6fe8af2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fe4100a70 0x7f6fe4197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:46.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.485+0000 7f6fe8af2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fe4101410 0x7f6fe4197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:46.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.485+0000 7f6fe8af2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fe4106a70 0x7f6fe419bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:46.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.485+0000 7f6fe8af2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe4110e70 con 0x7f6fe4100a70 2024-09-19T09:19:46.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.485+0000 7f6fe8af2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe4110cf0 con 0x7f6fe4101410 2024-09-19T09:19:46.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.485+0000 7f6fe8af2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe4110ff0 con 0x7f6fe4106a70 2024-09-19T09:19:46.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.485+0000 7f6fe259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fe4100a70 0x7f6fe4197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:46.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.485+0000 7f6fe259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fe4100a70 0x7f6fe4197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35142/0 (socket says 172.21.15.120:35142) 2024-09-19T09:19:46.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.485+0000 7f6fe259c640 1 -- 172.21.15.120:0/2928308464 learned_addr learned my addr 172.21.15.120:0/2928308464 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:46.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.485+0000 7f6fd9d9b640 1 --2- 172.21.15.120:0/2928308464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fe4101410 0x7f6fe4197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:46.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.486+0000 7f6fe2d9d640 1 --2- 172.21.15.120:0/2928308464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fe4106a70 0x7f6fe419bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:46.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.486+0000 7f6fe259c640 1 -- 172.21.15.120:0/2928308464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fe4106a70 msgr2=0x7f6fe419bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:46.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.486+0000 7f6fe259c640 1 --2- 172.21.15.120:0/2928308464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fe4106a70 0x7f6fe419bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.486+0000 7f6fe259c640 1 -- 172.21.15.120:0/2928308464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fe4101410 msgr2=0x7f6fe4197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:46.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.486+0000 7f6fe259c640 1 --2- 172.21.15.120:0/2928308464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fe4101410 0x7f6fe4197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.486+0000 7f6fe259c640 1 -- 172.21.15.120:0/2928308464 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fe419c2e0 con 0x7f6fe4100a70 2024-09-19T09:19:46.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.486+0000 7f6fe2d9d640 1 --2- 172.21.15.120:0/2928308464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fe4106a70 0x7f6fe419bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:19:46.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.486+0000 7f6fd9d9b640 1 --2- 172.21.15.120:0/2928308464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fe4101410 0x7f6fe4197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:46.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.486+0000 7f6fe259c640 1 --2- 172.21.15.120:0/2928308464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fe4100a70 0x7f6fe4197330 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f6fd400e9f0 tx=0x7f6fd400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:46.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.486+0000 7f6fdbfff640 1 -- 172.21.15.120:0/2928308464 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd400cd80 con 0x7f6fe4100a70 2024-09-19T09:19:46.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.486+0000 7f6fe8af2640 1 -- 172.21.15.120:0/2928308464 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fe419c5d0 con 0x7f6fe4100a70 2024-09-19T09:19:46.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.486+0000 7f6fdbfff640 1 -- 172.21.15.120:0/2928308464 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fd4005500 con 0x7f6fe4100a70 2024-09-19T09:19:46.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.486+0000 7f6fdbfff640 1 -- 172.21.15.120:0/2928308464 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4010690 con 0x7f6fe4100a70 2024-09-19T09:19:46.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.486+0000 7f6fe8af2640 1 -- 172.21.15.120:0/2928308464 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fe406a790 con 0x7f6fe4100a70 2024-09-19T09:19:46.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.489+0000 7f6fdbfff640 1 -- 172.21.15.120:0/2928308464 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f6fd4010830 con 0x7f6fe4100a70 2024-09-19T09:19:46.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.490+0000 7f6feaecb740 1 Processor -- start 2024-09-19T09:19:46.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.490+0000 7f6feaecb740 1 -- start start 2024-09-19T09:19:46.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.490+0000 7f6feaecb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d26aedb40 0x560d26aec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:46.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.490+0000 7f6feaecb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d26aec5d0 0x560d26aeca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:46.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.490+0000 7f6feaecb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d26af7370 0x560d26af9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:46.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.490+0000 7f6feaecb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560d26a522f0 con 0x560d26aec5d0 2024-09-19T09:19:46.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.490+0000 7f6feaecb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560d269f9cf0 con 0x560d26af7370 2024-09-19T09:19:46.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.490+0000 7f6feaecb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560d26a4e2b0 con 0x560d26aedb40 2024-09-19T09:19:46.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.491+0000 7f6fe2d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d26aedb40 0x560d26aec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:46.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.491+0000 7f6fe259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d26aec5d0 0x560d26aeca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:46.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.491+0000 7f6fe2d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d26aedb40 0x560d26aec090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:34138/0 (socket says 172.21.15.120:34138) 2024-09-19T09:19:46.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.491+0000 7f6fe259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d26aec5d0 0x560d26aeca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35152/0 (socket says 172.21.15.120:35152) 2024-09-19T09:19:46.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.491+0000 7f6fe2d9d640 1 -- 172.21.15.120:0/2906284420 learned_addr learned my addr 172.21.15.120:0/2906284420 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:46.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.491+0000 7f6fd9d9b640 1 --2- 172.21.15.120:0/2906284420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d26af7370 0x560d26af9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:46.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.491+0000 7f6fe259c640 1 -- 172.21.15.120:0/2906284420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d26aedb40 msgr2=0x560d26aec090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:46.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.491+0000 7f6fdbfff640 1 --2- 172.21.15.120:0/2928308464 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6fd00a3240 0x7f6fd00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:46.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.491+0000 7f6fe259c640 1 --2- 172.21.15.120:0/2906284420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d26aedb40 0x560d26aec090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.491+0000 7f6fe259c640 1 -- 172.21.15.120:0/2906284420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d26af7370 msgr2=0x560d26af9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:46.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.491+0000 7f6fe259c640 1 --2- 172.21.15.120:0/2906284420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d26af7370 0x560d26af9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.491+0000 7f6fe259c640 1 -- 172.21.15.120:0/2906284420 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d269de3a0 con 0x560d26aec5d0 2024-09-19T09:19:46.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.491+0000 7f6fe2d9d640 1 --2- 172.21.15.120:0/2928308464 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6fd00a3240 0x7f6fd00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:46.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.491+0000 7f6fdbfff640 1 -- 172.21.15.120:0/2928308464 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f6fd40cfd30 con 0x7f6fe4100a70 2024-09-19T09:19:46.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.492+0000 7f6fe2d9d640 1 --2- 172.21.15.120:0/2928308464 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6fd00a3240 0x7f6fd00a5700 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc009a60 tx=0x7f6fdc03f040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:46.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.492+0000 7f6fe259c640 1 --2- 172.21.15.120:0/2906284420 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d26aec5d0 0x560d26aeca10 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f6fd40cd7d0 tx=0x7f6fd4098490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb929f0edffd526 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:46.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.492+0000 7f6fd8d99640 1 -- 172.21.15.120:0/2906284420 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd409c190 con 0x560d26aec5d0 2024-09-19T09:19:46.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.492+0000 7f6fd8d99640 1 -- 172.21.15.120:0/2906284420 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fd409ca50 con 0x560d26aec5d0 2024-09-19T09:19:46.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.492+0000 7f6fd8d99640 1 -- 172.21.15.120:0/2906284420 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd409bd20 con 0x560d26aec5d0 2024-09-19T09:19:46.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.493+0000 7f6feaecb740 1 -- 172.21.15.120:0/2906284420 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d26aec5d0 msgr2=0x560d26aeca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:46.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.493+0000 7f6feaecb740 1 --2- 172.21.15.120:0/2906284420 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d26aec5d0 0x560d26aeca10 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f6fd40cd7d0 tx=0x7f6fd4098490 comp rx=0 tx=0).stop 2024-09-19T09:19:46.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.493+0000 7f6feaecb740 1 -- 172.21.15.120:0/2906284420 shutdown_connections 2024-09-19T09:19:46.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.493+0000 7f6feaecb740 1 --2- 172.21.15.120:0/2906284420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d26af7370 0x560d26af9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.493+0000 7f6feaecb740 1 --2- 172.21.15.120:0/2906284420 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d26aec5d0 0x560d26aeca10 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.493+0000 7f6feaecb740 1 --2- 172.21.15.120:0/2906284420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d26aedb40 0x560d26aec090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.493+0000 7f6feaecb740 1 -- 172.21.15.120:0/2906284420 >> 172.21.15.120:0/2906284420 conn(0x560d26ab5310 msgr2=0x560d26ae8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:46.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.493+0000 7f6feaecb740 1 -- 172.21.15.120:0/2906284420 shutdown_connections 2024-09-19T09:19:46.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.493+0000 7f6feaecb740 1 -- 172.21.15.120:0/2906284420 wait complete. 2024-09-19T09:19:46.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.494+0000 7f6feaecb740 1 -- start start 2024-09-19T09:19:46.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.494+0000 7f6feaecb740 1 Processor -- start 2024-09-19T09:19:46.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.494+0000 7f6feaecb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d26aec5d0 0x560d26af6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:46.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.494+0000 7f6feaecb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d26aedb40 0x560d26af4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:46.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.494+0000 7f6feaecb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d26af7370 0x560d26af47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:46.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.494+0000 7f6feaecb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560d26afa010 con 0x560d26aedb40 2024-09-19T09:19:46.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.494+0000 7f6feaecb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560d26a0d060 con 0x560d26aec5d0 2024-09-19T09:19:46.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.494+0000 7f6feaecb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560d26afa190 con 0x560d26af7370 2024-09-19T09:19:46.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.494+0000 7f6feaecb740 10 client.0 _ll_register_callbacks cb 0x560d267608c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:46.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.494+0000 7f6fe259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d26aec5d0 0x560d26af6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:46.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.494+0000 7f6fe259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d26aec5d0 0x560d26af6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:43872/0 (socket says 172.21.15.120:43872) 2024-09-19T09:19:46.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.494+0000 7f6fd9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d26aedb40 0x560d26af4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:46.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.494+0000 7f6fe259c640 1 -- 172.21.15.120:0/542984214 learned_addr learned my addr 172.21.15.120:0/542984214 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:46.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.494+0000 7f6fe2d9d640 1 --2- 172.21.15.120:0/542984214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d26af7370 0x560d26af47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:46.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.495+0000 7f6fe2d9d640 1 -- 172.21.15.120:0/542984214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d26aec5d0 msgr2=0x560d26af6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:46.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.495+0000 7f6fe2d9d640 1 --2- 172.21.15.120:0/542984214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d26aec5d0 0x560d26af6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.495+0000 7f6fe2d9d640 1 -- 172.21.15.120:0/542984214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d26aedb40 msgr2=0x560d26af4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:46.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.495+0000 7f6fe2d9d640 1 --2- 172.21.15.120:0/542984214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d26aedb40 0x560d26af4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.495+0000 7f6fe2d9d640 1 -- 172.21.15.120:0/542984214 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d267c33c0 con 0x560d26af7370 2024-09-19T09:19:46.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.495+0000 7f6fe2d9d640 1 --2- 172.21.15.120:0/542984214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d26af7370 0x560d26af47d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc02ae00 tx=0x7f6fdc005d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=305d886357d50c2c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:46.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.495+0000 7f6fc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:19:46.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.495+0000 7f6fc9ffb640 1 -- 172.21.15.120:0/542984214 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc004450 con 0x560d26af7370 2024-09-19T09:19:46.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.495+0000 7f6fc9ffb640 1 -- 172.21.15.120:0/542984214 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f6fdc0045f0 con 0x560d26af7370 2024-09-19T09:19:46.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.496+0000 7f6feaecb740 1 -- 172.21.15.120:0/542984214 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fd409ca50 con 0x560d26af7370 2024-09-19T09:19:46.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.496+0000 7f6fc9ffb640 1 -- 172.21.15.120:0/542984214 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6fdc0047f0 con 0x560d26af7370 2024-09-19T09:19:46.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.496+0000 7f6feaecb740 10 client.29794 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:46.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.496+0000 7f6feaecb740 1 -- 172.21.15.120:0/542984214 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560d26b0e350 con 0x560d26af7370 2024-09-19T09:19:46.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.496+0000 7f6fc9ffb640 1 -- 172.21.15.120:0/542984214 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fd409ca50 con 0x560d26af7370 2024-09-19T09:19:46.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.496+0000 7f6fc9ffb640 1 -- 172.21.15.120:0/542984214 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f6fdc030070 con 0x560d26af7370 2024-09-19T09:19:46.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.497+0000 7f6feaecb740 10 client.29794 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:46.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.497+0000 7f6feaecb740 10 client.29794 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:19:46.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.497+0000 7f6feaecb740 20 client.29794 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:46.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.497+0000 7f6feaecb740 1 --2- 172.21.15.120:0/542984214 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560d26b0fcf0 0x560d26b12130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:46.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.497+0000 7f6feaecb740 4 client.29794 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:19:46.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.497+0000 7f6feaecb740 1 -- 172.21.15.120:0/542984214 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d26a5f860 con 0x560d26b0fcf0 2024-09-19T09:19:46.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.497+0000 7f6fe259c640 1 --2- 172.21.15.120:0/542984214 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560d26b0fcf0 0x560d26b12130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:46.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.498+0000 7f6fe259c640 1 --2- 172.21.15.120:0/542984214 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560d26b0fcf0 0x560d26b12130 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5a3a0383c492f34e server_cookie=69b3ec71a00221fa in_seq=0 out_seq=0 2024-09-19T09:19:46.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.498+0000 7f6fc9ffb640 10 client.29794 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:19:46.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.498+0000 7f6fc9ffb640 1 -- 172.21.15.120:0/542984214 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d26a5f860 con 0x560d26b0fcf0 2024-09-19T09:19:46.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.498+0000 7f6fc9ffb640 10 client.29794 handle_command_reply: tid=0 2024-09-19T09:19:46.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.498+0000 7f6feaecb740 1 client.29794 shutdown 2024-09-19T09:19:46.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.498+0000 7f6feaecb740 1 -- 172.21.15.120:0/542984214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d26af7370 msgr2=0x560d26af47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:46.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.498+0000 7f6feaecb740 1 --2- 172.21.15.120:0/542984214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d26af7370 0x560d26af47d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc02ae00 tx=0x7f6fdc005d80 comp rx=0 tx=0).stop 2024-09-19T09:19:46.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.499+0000 7f6feaecb740 1 -- 172.21.15.120:0/542984214 shutdown_connections 2024-09-19T09:19:46.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.499+0000 7f6feaecb740 1 --2- 172.21.15.120:0/542984214 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560d26b0fcf0 0x560d26b12130 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.499+0000 7f6feaecb740 1 --2- 172.21.15.120:0/542984214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d26af7370 0x560d26af47d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.499+0000 7f6feaecb740 1 --2- 172.21.15.120:0/542984214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d26aedb40 0x560d26af4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.499+0000 7f6fc9ffb640 0 client.29794 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:19:46.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.499+0000 7f6feaecb740 1 --2- 172.21.15.120:0/542984214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d26aec5d0 0x560d26af6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.499+0000 7f6feaecb740 1 -- 172.21.15.120:0/542984214 >> 172.21.15.120:0/542984214 conn(0x560d26ab5310 msgr2=0x560d26aee960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:46.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.499+0000 7f6feaecb740 1 -- 172.21.15.120:0/542984214 shutdown_connections 2024-09-19T09:19:46.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.499+0000 7f6feaecb740 1 -- 172.21.15.120:0/542984214 wait complete. 2024-09-19T09:19:46.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.499+0000 7f6feaecb740 20 client.29794 trim_cache size 0 max 16384 2024-09-19T09:19:46.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.513+0000 7f6feaecb740 1 Processor -- start 2024-09-19T09:19:46.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.513+0000 7f6feaecb740 1 -- start start 2024-09-19T09:19:46.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.513+0000 7f6feaecb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d26aec5d0 0x560d26b0bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:46.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.514+0000 7f6feaecb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d26aedb40 0x560d26b177e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:46.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.514+0000 7f6feaecb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d26af7370 0x560d26b17d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:46.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.514+0000 7f6feaecb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe41ab4e0 con 0x560d26aec5d0 2024-09-19T09:19:46.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.514+0000 7f6feaecb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe41ad020 con 0x560d26af7370 2024-09-19T09:19:46.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.514+0000 7f6feaecb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe41abbb0 con 0x560d26aedb40 2024-09-19T09:19:46.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.514+0000 7f6fe259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d26aec5d0 0x560d26b0bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:46.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.514+0000 7f6fd9d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d26aedb40 0x560d26b177e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:46.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.514+0000 7f6fe259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d26aec5d0 0x560d26b0bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35166/0 (socket says 172.21.15.120:35166) 2024-09-19T09:19:46.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.514+0000 7f6fe259c640 1 -- 172.21.15.120:0/3076603064 learned_addr learned my addr 172.21.15.120:0/3076603064 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:46.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.514+0000 7f6fe2d9d640 1 --2- 172.21.15.120:0/3076603064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d26af7370 0x560d26b17d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:46.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.514+0000 7f6fd9d9b640 1 -- 172.21.15.120:0/3076603064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d26af7370 msgr2=0x560d26b17d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:46.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.514+0000 7f6fd9d9b640 1 --2- 172.21.15.120:0/3076603064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d26af7370 0x560d26b17d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.514+0000 7f6fd9d9b640 1 -- 172.21.15.120:0/3076603064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d26aec5d0 msgr2=0x560d26b0bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:46.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.514+0000 7f6fd9d9b640 1 --2- 172.21.15.120:0/3076603064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d26aec5d0 0x560d26b0bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.514+0000 7f6fd9d9b640 1 -- 172.21.15.120:0/3076603064 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d26aec2d0 con 0x560d26aedb40 2024-09-19T09:19:46.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.514+0000 7f6fe259c640 1 --2- 172.21.15.120:0/3076603064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d26aec5d0 0x560d26b0bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:46.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.514+0000 7f6fe2d9d640 1 --2- 172.21.15.120:0/3076603064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d26af7370 0x560d26b17d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:19:46.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.514+0000 7f6fd9d9b640 1 --2- 172.21.15.120:0/3076603064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d26aedb40 0x560d26b177e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x560d26af3920 tx=0x7f6fcc012870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8239ca82b757a258 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:46.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.514+0000 7f6fcbfff640 1 -- 172.21.15.120:0/3076603064 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc004020 con 0x560d26aedb40 2024-09-19T09:19:46.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.514+0000 7f6fcbfff640 1 -- 172.21.15.120:0/3076603064 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fcc0041c0 con 0x560d26aedb40 2024-09-19T09:19:46.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.514+0000 7f6fcbfff640 1 -- 172.21.15.120:0/3076603064 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc011840 con 0x560d26aedb40 2024-09-19T09:19:46.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.515+0000 7f6feaecb740 1 -- 172.21.15.120:0/3076603064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d26aedb40 msgr2=0x560d26b177e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:46.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.515+0000 7f6feaecb740 1 --2- 172.21.15.120:0/3076603064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d26aedb40 0x560d26b177e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x560d26af3920 tx=0x7f6fcc012870 comp rx=0 tx=0).stop 2024-09-19T09:19:46.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.515+0000 7f6feaecb740 1 -- 172.21.15.120:0/3076603064 shutdown_connections 2024-09-19T09:19:46.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.515+0000 7f6feaecb740 1 --2- 172.21.15.120:0/3076603064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d26af7370 0x560d26b17d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.515+0000 7f6feaecb740 1 --2- 172.21.15.120:0/3076603064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d26aedb40 0x560d26b177e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.515+0000 7f6feaecb740 1 --2- 172.21.15.120:0/3076603064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d26aec5d0 0x560d26b0bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.515+0000 7f6feaecb740 1 -- 172.21.15.120:0/3076603064 >> 172.21.15.120:0/3076603064 conn(0x560d26ab5310 msgr2=0x560d26af31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:46.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.515+0000 7f6feaecb740 1 -- 172.21.15.120:0/3076603064 shutdown_connections 2024-09-19T09:19:46.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.515+0000 7f6feaecb740 1 -- 172.21.15.120:0/3076603064 wait complete. 2024-09-19T09:19:46.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.516+0000 7f6feaecb740 1 -- start start 2024-09-19T09:19:46.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.516+0000 7f6feaecb740 1 Processor -- start 2024-09-19T09:19:46.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.516+0000 7f6feaecb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d26aec5d0 0x560d26b07cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:46.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.516+0000 7f6feaecb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d26aedb40 0x560d26b081f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:46.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.516+0000 7f6feaecb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d26af7370 0x560d26b04c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:46.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.516+0000 7f6feaecb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe41ac950 con 0x560d26aedb40 2024-09-19T09:19:46.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.516+0000 7f6feaecb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe41ac280 con 0x560d26af7370 2024-09-19T09:19:46.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.516+0000 7f6feaecb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe41a94a0 con 0x560d26aec5d0 2024-09-19T09:19:46.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.516+0000 7f6feaecb740 10 client.0 _ll_register_callbacks cb 0x7f6fdc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:46.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.516+0000 7f6fe259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d26aec5d0 0x560d26b07cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:46.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.516+0000 7f6fd9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d26aedb40 0x560d26b081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:46.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.516+0000 7f6fe259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d26aec5d0 0x560d26b07cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:34164/0 (socket says 172.21.15.120:34164) 2024-09-19T09:19:46.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.516+0000 7f6fe259c640 1 -- 172.21.15.120:0/800329731 learned_addr learned my addr 172.21.15.120:0/800329731 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:46.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.516+0000 7f6fe2d9d640 1 --2- 172.21.15.120:0/800329731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d26af7370 0x560d26b04c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:46.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.516+0000 7f6fe259c640 1 -- 172.21.15.120:0/800329731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d26af7370 msgr2=0x560d26b04c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:46.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.516+0000 7f6fe259c640 1 --2- 172.21.15.120:0/800329731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d26af7370 0x560d26b04c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.516+0000 7f6fe259c640 1 -- 172.21.15.120:0/800329731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d26aedb40 msgr2=0x560d26b081f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:46.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.516+0000 7f6fe259c640 1 --2- 172.21.15.120:0/800329731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d26aedb40 0x560d26b081f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.516+0000 7f6fe259c640 1 -- 172.21.15.120:0/800329731 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d26b051c0 con 0x560d26aec5d0 2024-09-19T09:19:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.516+0000 7f6fe2d9d640 1 --2- 172.21.15.120:0/800329731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d26af7370 0x560d26b04c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:19:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.517+0000 7f6fe259c640 1 --2- 172.21.15.120:0/800329731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d26aec5d0 0x560d26b07cb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4098d90 tx=0x7f6fd40d33d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daa7ca6b9118f659 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.517+0000 7f6fc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:19:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.517+0000 7f6fc9ffb640 1 -- 172.21.15.120:0/800329731 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd409bf00 con 0x560d26aec5d0 2024-09-19T09:19:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.517+0000 7f6feaecb740 1 -- 172.21.15.120:0/800329731 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fcc0041c0 con 0x560d26aec5d0 2024-09-19T09:19:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.517+0000 7f6fc9ffb640 1 -- 172.21.15.120:0/800329731 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f6fd409a6c0 con 0x560d26aec5d0 2024-09-19T09:19:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.517+0000 7f6fc9ffb640 1 -- 172.21.15.120:0/800329731 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6fd409bd20 con 0x560d26aec5d0 2024-09-19T09:19:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.517+0000 7f6feaecb740 10 client.29800 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.517+0000 7f6feaecb740 1 -- 172.21.15.120:0/800329731 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560d26b7aeb0 con 0x560d26aec5d0 2024-09-19T09:19:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.517+0000 7f6fc9ffb640 1 -- 172.21.15.120:0/800329731 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fcc0041c0 con 0x560d26aec5d0 2024-09-19T09:19:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.517+0000 7f6fc9ffb640 1 -- 172.21.15.120:0/800329731 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f6fd40cd8b0 con 0x560d26aec5d0 2024-09-19T09:19:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.517+0000 7f6feaecb740 10 client.29800 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.517+0000 7f6feaecb740 10 client.29800 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:19:46.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.517+0000 7f6feaecb740 20 client.29800 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.517+0000 7f6feaecb740 1 --2- 172.21.15.120:0/800329731 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560d26b7b240 0x560d26b7d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.518+0000 7f6feaecb740 4 client.29800 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:19:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.518+0000 7f6feaecb740 1 -- 172.21.15.120:0/800329731 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560d26a59d70 con 0x560d26b7b240 2024-09-19T09:19:46.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.518+0000 7f6fd9d9b640 1 --2- 172.21.15.120:0/800329731 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560d26b7b240 0x560d26b7d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.518+0000 7f6fc9ffb640 10 client.29800 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:19:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.518+0000 7f6fd9d9b640 1 --2- 172.21.15.120:0/800329731 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560d26b7b240 0x560d26b7d630 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7743f2453e9f5b2c server_cookie=d5e9a2ea63ea8332 in_seq=0 out_seq=0 2024-09-19T09:19:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.518+0000 7f6fc9ffb640 1 -- 172.21.15.120:0/800329731 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560d26a59d70 con 0x560d26b7b240 2024-09-19T09:19:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.518+0000 7f6fc9ffb640 10 client.29800 handle_command_reply: tid=0 2024-09-19T09:19:46.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.518+0000 7f6feaecb740 1 client.29800 shutdown 2024-09-19T09:19:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.518+0000 7f6feaecb740 1 -- 172.21.15.120:0/800329731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d26aec5d0 msgr2=0x560d26b07cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.518+0000 7f6feaecb740 1 --2- 172.21.15.120:0/800329731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d26aec5d0 0x560d26b07cb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4098d90 tx=0x7f6fd40d33d0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.518+0000 7f6feaecb740 1 -- 172.21.15.120:0/800329731 shutdown_connections 2024-09-19T09:19:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.518+0000 7f6feaecb740 1 --2- 172.21.15.120:0/800329731 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560d26b7b240 0x560d26b7d630 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.519+0000 7f6feaecb740 1 --2- 172.21.15.120:0/800329731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d26af7370 0x560d26b04c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.519+0000 7f6feaecb740 1 --2- 172.21.15.120:0/800329731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d26aedb40 0x560d26b081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.519+0000 7f6feaecb740 1 --2- 172.21.15.120:0/800329731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d26aec5d0 0x560d26b07cb0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.519+0000 7f6feaecb740 1 -- 172.21.15.120:0/800329731 >> 172.21.15.120:0/800329731 conn(0x560d26ab5310 msgr2=0x560d26ae74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.519+0000 7f6fc9ffb640 0 client.29800 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:19:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.519+0000 7f6feaecb740 1 -- 172.21.15.120:0/800329731 shutdown_connections 2024-09-19T09:19:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.519+0000 7f6feaecb740 1 -- 172.21.15.120:0/800329731 wait complete. 2024-09-19T09:19:46.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.519+0000 7f6feaecb740 20 client.29800 trim_cache size 0 max 16384 2024-09-19T09:19:46.527 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:19:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.520+0000 7f6fe8af2640 1 -- 172.21.15.120:0/2928308464 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6fd00a3240 msgr2=0x7f6fd00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.520+0000 7f6fe8af2640 1 --2- 172.21.15.120:0/2928308464 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6fd00a3240 0x7f6fd00a5700 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc009a60 tx=0x7f6fdc03f040 comp rx=0 tx=0).stop 2024-09-19T09:19:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.520+0000 7f6fe8af2640 1 -- 172.21.15.120:0/2928308464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fe4100a70 msgr2=0x7f6fe4197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.520+0000 7f6fe8af2640 1 --2- 172.21.15.120:0/2928308464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fe4100a70 0x7f6fe4197330 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f6fd400e9f0 tx=0x7f6fd400edf0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.520+0000 7f6fe8af2640 1 -- 172.21.15.120:0/2928308464 shutdown_connections 2024-09-19T09:19:46.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.520+0000 7f6fe8af2640 1 --2- 172.21.15.120:0/2928308464 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6fd00a3240 0x7f6fd00a5700 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.520+0000 7f6fe8af2640 1 --2- 172.21.15.120:0/2928308464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fe4106a70 0x7f6fe419bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.520+0000 7f6fe8af2640 1 --2- 172.21.15.120:0/2928308464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fe4101410 0x7f6fe4197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.520+0000 7f6fe8af2640 1 --2- 172.21.15.120:0/2928308464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fe4100a70 0x7f6fe4197330 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:46.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.520+0000 7f6fe8af2640 1 -- 172.21.15.120:0/2928308464 >> 172.21.15.120:0/2928308464 conn(0x7f6fe40fc790 msgr2=0x7f6fe40fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:46.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.520+0000 7f6fe8af2640 1 -- 172.21.15.120:0/2928308464 shutdown_connections 2024-09-19T09:19:46.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:46.520+0000 7f6fe8af2640 1 -- 172.21.15.120:0/2928308464 wait complete. 2024-09-19T09:19:47.300 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:46 smithi120 ceph-mon[31150]: pgmap v852: 97 pgs: 97 active+clean; 342 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.9 MiB/s wr, 728 op/s 2024-09-19T09:19:47.340 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:19:47.363 INFO:tasks.workunit.client.0.smithi120.stdout: 10 86214 16.00 MB/sec execute 102 sec latency 150.352 ms 2024-09-19T09:19:47.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:46 smithi196 ceph-mon[35150]: pgmap v852: 97 pgs: 97 active+clean; 342 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.9 MiB/s wr, 728 op/s 2024-09-19T09:19:47.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:46 smithi153 ceph-mon[35071]: pgmap v852: 97 pgs: 97 active+clean; 342 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.9 MiB/s wr, 728 op/s 2024-09-19T09:19:47.610 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:19:48.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.137+0000 7f7129089640 1 -- 172.21.15.120:0/423319473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7124106a90 msgr2=0x7f7124106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:48.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.137+0000 7f7129089640 1 --2- 172.21.15.120:0/423319473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7124106a90 0x7f7124106e70 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f7114009890 tx=0x7f711402a380 comp rx=0 tx=0).stop 2024-09-19T09:19:48.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.137+0000 7f7129089640 1 -- 172.21.15.120:0/423319473 shutdown_connections 2024-09-19T09:19:48.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.137+0000 7f7129089640 1 --2- 172.21.15.120:0/423319473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7124101430 0x7f712410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.137+0000 7f7129089640 1 --2- 172.21.15.120:0/423319473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7124100a90 0x7f7124100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.137+0000 7f7129089640 1 --2- 172.21.15.120:0/423319473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7124106a90 0x7f7124106e70 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.137+0000 7f7129089640 1 -- 172.21.15.120:0/423319473 >> 172.21.15.120:0/423319473 conn(0x7f71240fc7d0 msgr2=0x7f71240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:48.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.138+0000 7f7129089640 1 -- 172.21.15.120:0/423319473 shutdown_connections 2024-09-19T09:19:48.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.138+0000 7f7129089640 1 -- 172.21.15.120:0/423319473 wait complete. 2024-09-19T09:19:48.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.139+0000 7f7129089640 1 Processor -- start 2024-09-19T09:19:48.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.139+0000 7f7129089640 1 -- start start 2024-09-19T09:19:48.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.139+0000 7f7129089640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7124100a90 0x7f712410d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:48.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.139+0000 7f7129089640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7124101430 0x7f71241095f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:48.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.139+0000 7f7129089640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7124106a90 0x7f7124109b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:48.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.139+0000 7f7129089640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f712410fd60 con 0x7f7124106a90 2024-09-19T09:19:48.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.139+0000 7f7129089640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f712410fbe0 con 0x7f7124100a90 2024-09-19T09:19:48.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.139+0000 7f7129089640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f712410fee0 con 0x7f7124101430 2024-09-19T09:19:48.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.139+0000 7f712359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7124106a90 0x7f7124109b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:48.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.139+0000 7f7122d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7124100a90 0x7f712410d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:48.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.139+0000 7f712359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7124106a90 0x7f7124109b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35198/0 (socket says 172.21.15.120:35198) 2024-09-19T09:19:48.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.139+0000 7f712359e640 1 -- 172.21.15.120:0/449086424 learned_addr learned my addr 172.21.15.120:0/449086424 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:48.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.140+0000 7f712259c640 1 --2- 172.21.15.120:0/449086424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7124101430 0x7f71241095f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:48.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.140+0000 7f712359e640 1 -- 172.21.15.120:0/449086424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7124101430 msgr2=0x7f71241095f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:48.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.140+0000 7f712359e640 1 --2- 172.21.15.120:0/449086424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7124101430 0x7f71241095f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.140+0000 7f712359e640 1 -- 172.21.15.120:0/449086424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7124100a90 msgr2=0x7f712410d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:48.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.140+0000 7f712359e640 1 --2- 172.21.15.120:0/449086424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7124100a90 0x7f712410d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.140+0000 7f712359e640 1 -- 172.21.15.120:0/449086424 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f712410a0f0 con 0x7f7124106a90 2024-09-19T09:19:48.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.140+0000 7f7122d9d640 1 --2- 172.21.15.120:0/449086424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7124100a90 0x7f712410d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:48.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.140+0000 7f712359e640 1 --2- 172.21.15.120:0/449086424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7124106a90 0x7f7124109b30 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f711c00eb10 tx=0x7f711c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:48.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.140+0000 7f710bfff640 1 -- 172.21.15.120:0/449086424 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c004b20 con 0x7f7124106a90 2024-09-19T09:19:48.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.140+0000 7f7129089640 1 -- 172.21.15.120:0/449086424 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f712410a3e0 con 0x7f7124106a90 2024-09-19T09:19:48.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.140+0000 7f710bfff640 1 -- 172.21.15.120:0/449086424 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f711c004cc0 con 0x7f7124106a90 2024-09-19T09:19:48.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.140+0000 7f710bfff640 1 -- 172.21.15.120:0/449086424 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c010760 con 0x7f7124106a90 2024-09-19T09:19:48.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.140+0000 7f7129089640 1 -- 172.21.15.120:0/449086424 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71241ab530 con 0x7f7124106a90 2024-09-19T09:19:48.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.143+0000 7f712b462740 1 Processor -- start 2024-09-19T09:19:48.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.144+0000 7f710bfff640 1 -- 172.21.15.120:0/449086424 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f711c005020 con 0x7f7124106a90 2024-09-19T09:19:48.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.144+0000 7f712b462740 1 -- start start 2024-09-19T09:19:48.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.145+0000 7f712b462740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55638b171b40 0x55638b170090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:48.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.145+0000 7f712b462740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55638b1705d0 0x55638b170a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:48.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.145+0000 7f712b462740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638b17b370 0x55638b17d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:48.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.145+0000 7f712b462740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55638b0d62f0 con 0x55638b1705d0 2024-09-19T09:19:48.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.145+0000 7f712b462740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55638b07dcf0 con 0x55638b171b40 2024-09-19T09:19:48.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.145+0000 7f712b462740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55638b0d22b0 con 0x55638b17b370 2024-09-19T09:19:48.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.145+0000 7f712259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55638b171b40 0x55638b170090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:48.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.145+0000 7f712259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55638b171b40 0x55638b170090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:43914/0 (socket says 172.21.15.120:43914) 2024-09-19T09:19:48.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.145+0000 7f712259c640 1 -- 172.21.15.120:0/3466526126 learned_addr learned my addr 172.21.15.120:0/3466526126 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:48.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.145+0000 7f712259c640 1 --2- 172.21.15.120:0/3466526126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638b17b370 0x55638b17d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:48.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.145+0000 7f710bfff640 1 --2- 172.21.15.120:0/449086424 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f710c0a3240 0x7f710c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:48.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.145+0000 7f712259c640 1 -- 172.21.15.120:0/3466526126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638b17b370 msgr2=0x55638b17d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:48.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.145+0000 7f712259c640 1 --2- 172.21.15.120:0/3466526126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638b17b370 0x55638b17d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.145+0000 7f712259c640 1 -- 172.21.15.120:0/3466526126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55638b1705d0 msgr2=0x55638b170a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:19:48.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.145+0000 7f712359e640 1 --2- 172.21.15.120:0/449086424 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f710c0a3240 0x7f710c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:48.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.145+0000 7f712259c640 1 --2- 172.21.15.120:0/3466526126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55638b1705d0 0x55638b170a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.145+0000 7f712259c640 1 -- 172.21.15.120:0/3466526126 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55638b0623a0 con 0x55638b171b40 2024-09-19T09:19:48.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.146+0000 7f710bfff640 1 -- 172.21.15.120:0/449086424 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f711c0cfe00 con 0x7f7124106a90 2024-09-19T09:19:48.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.146+0000 7f712259c640 1 --2- 172.21.15.120:0/3466526126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55638b171b40 0x55638b170090 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55638b171070 tx=0x7f711800b650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fedec6ffac617f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:48.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.146+0000 7f712359e640 1 --2- 172.21.15.120:0/449086424 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f710c0a3240 0x7f710c0a5700 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f711c09c120 tx=0x7f711c098040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:48.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.146+0000 7f71097fa640 1 -- 172.21.15.120:0/3466526126 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118003e80 con 0x55638b171b40 2024-09-19T09:19:48.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.146+0000 7f71097fa640 1 -- 172.21.15.120:0/3466526126 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f711800be60 con 0x55638b171b40 2024-09-19T09:19:48.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.146+0000 7f71097fa640 1 -- 172.21.15.120:0/3466526126 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71180107c0 con 0x55638b171b40 2024-09-19T09:19:48.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.147+0000 7f712b462740 1 -- 172.21.15.120:0/3466526126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55638b171b40 msgr2=0x55638b170090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:48.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.147+0000 7f712b462740 1 --2- 172.21.15.120:0/3466526126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55638b171b40 0x55638b170090 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55638b171070 tx=0x7f711800b650 comp rx=0 tx=0).stop 2024-09-19T09:19:48.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.147+0000 7f712b462740 1 -- 172.21.15.120:0/3466526126 shutdown_connections 2024-09-19T09:19:48.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.147+0000 7f712b462740 1 --2- 172.21.15.120:0/3466526126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638b17b370 0x55638b17d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.147+0000 7f712b462740 1 --2- 172.21.15.120:0/3466526126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55638b1705d0 0x55638b170a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.147+0000 7f712b462740 1 --2- 172.21.15.120:0/3466526126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55638b171b40 0x55638b170090 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.147+0000 7f712b462740 1 -- 172.21.15.120:0/3466526126 >> 172.21.15.120:0/3466526126 conn(0x55638b139310 msgr2=0x55638b16cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:48.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.147+0000 7f712b462740 1 -- 172.21.15.120:0/3466526126 shutdown_connections 2024-09-19T09:19:48.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.147+0000 7f712b462740 1 -- 172.21.15.120:0/3466526126 wait complete. 2024-09-19T09:19:48.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.148+0000 7f712b462740 1 -- start start 2024-09-19T09:19:48.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.148+0000 7f712b462740 1 Processor -- start 2024-09-19T09:19:48.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.148+0000 7f712b462740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55638b1705d0 0x55638b17ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:48.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.148+0000 7f712b462740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55638b171b40 0x55638b178010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:48.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.148+0000 7f712b462740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638b17b370 0x55638b178600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:48.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.148+0000 7f712b462740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55638b17e010 con 0x55638b171b40 2024-09-19T09:19:48.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.148+0000 7f712b462740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55638b091060 con 0x55638b1705d0 2024-09-19T09:19:48.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.148+0000 7f712b462740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55638b17e190 con 0x55638b17b370 2024-09-19T09:19:48.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.148+0000 7f712b462740 10 client.0 _ll_register_callbacks cb 0x55638ade48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:48.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.149+0000 7f7122d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55638b171b40 0x55638b178010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:48.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.149+0000 7f712259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55638b1705d0 0x55638b17ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:48.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.149+0000 7f712259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55638b1705d0 0x55638b17ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:43926/0 (socket says 172.21.15.120:43926) 2024-09-19T09:19:48.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.149+0000 7f712259c640 1 -- 172.21.15.120:0/1670048923 learned_addr learned my addr 172.21.15.120:0/1670048923 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:48.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.149+0000 7f712259c640 1 --2- 172.21.15.120:0/1670048923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638b17b370 0x55638b178600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:48.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.149+0000 7f712259c640 1 -- 172.21.15.120:0/1670048923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638b17b370 msgr2=0x55638b178600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:48.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.149+0000 7f712259c640 1 --2- 172.21.15.120:0/1670048923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638b17b370 0x55638b178600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.149+0000 7f712259c640 1 -- 172.21.15.120:0/1670048923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55638b171b40 msgr2=0x55638b178010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:48.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.149+0000 7f712259c640 1 --2- 172.21.15.120:0/1670048923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55638b171b40 0x55638b178010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.149+0000 7f712259c640 1 -- 172.21.15.120:0/1670048923 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55638ae473c0 con 0x55638b1705d0 2024-09-19T09:19:48.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.149+0000 7f712259c640 1 --2- 172.21.15.120:0/1670048923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55638b1705d0 0x55638b17ad60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f711800b510 tx=0x7f7118017040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34f44a5350a6f041 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:48.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.149+0000 7f7106ff5640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:19:48.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.150+0000 7f7106ff5640 1 -- 172.21.15.120:0/1670048923 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118003e50 con 0x55638b1705d0 2024-09-19T09:19:48.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.150+0000 7f7106ff5640 1 -- 172.21.15.120:0/1670048923 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f7118012660 con 0x55638b1705d0 2024-09-19T09:19:48.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.150+0000 7f712b462740 1 -- 172.21.15.120:0/1670048923 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f711800be60 con 0x55638b1705d0 2024-09-19T09:19:48.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.150+0000 7f7106ff5640 1 -- 172.21.15.120:0/1670048923 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7118012860 con 0x55638b1705d0 2024-09-19T09:19:48.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.150+0000 7f712b462740 10 client.29990 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:48.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.150+0000 7f712b462740 1 -- 172.21.15.120:0/1670048923 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55638b189ab0 con 0x55638b1705d0 2024-09-19T09:19:48.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.150+0000 7f7106ff5640 1 -- 172.21.15.120:0/1670048923 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f711800be60 con 0x55638b1705d0 2024-09-19T09:19:48.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.150+0000 7f7106ff5640 1 -- 172.21.15.120:0/1670048923 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f711801fc90 con 0x55638b1705d0 2024-09-19T09:19:48.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.151+0000 7f712b462740 10 client.29990 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:48.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.151+0000 7f712b462740 10 client.29990 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:19:48.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.151+0000 7f712b462740 20 client.29990 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:48.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.151+0000 7f712b462740 1 --2- 172.21.15.120:0/1670048923 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55638b18ae00 0x55638b18d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:48.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.151+0000 7f712b462740 4 client.29990 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:19:48.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.151+0000 7f712b462740 1 -- 172.21.15.120:0/1670048923 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55638b0e3860 con 0x55638b18ae00 2024-09-19T09:19:48.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.151+0000 7f7122d9d640 1 --2- 172.21.15.120:0/1670048923 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55638b18ae00 0x55638b18d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:48.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.151+0000 7f7122d9d640 1 --2- 172.21.15.120:0/1670048923 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55638b18ae00 0x55638b18d1f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=600b2533f743c625 server_cookie=3c389a08df2fe2ac in_seq=0 out_seq=0 2024-09-19T09:19:48.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.151+0000 7f7106ff5640 10 client.29990 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:19:48.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.152+0000 7f7106ff5640 1 -- 172.21.15.120:0/1670048923 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55638b0e3860 con 0x55638b18ae00 2024-09-19T09:19:48.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.152+0000 7f7106ff5640 10 client.29990 handle_command_reply: tid=0 2024-09-19T09:19:48.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.152+0000 7f712b462740 1 client.29990 shutdown 2024-09-19T09:19:48.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.152+0000 7f712b462740 1 -- 172.21.15.120:0/1670048923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55638b1705d0 msgr2=0x55638b17ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:48.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.152+0000 7f712b462740 1 --2- 172.21.15.120:0/1670048923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55638b1705d0 0x55638b17ad60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f711800b510 tx=0x7f7118017040 comp rx=0 tx=0).stop 2024-09-19T09:19:48.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.153+0000 7f712b462740 1 -- 172.21.15.120:0/1670048923 shutdown_connections 2024-09-19T09:19:48.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.153+0000 7f712b462740 1 --2- 172.21.15.120:0/1670048923 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55638b18ae00 0x55638b18d1f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.153+0000 7f712b462740 1 --2- 172.21.15.120:0/1670048923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638b17b370 0x55638b178600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.153+0000 7f712b462740 1 --2- 172.21.15.120:0/1670048923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55638b171b40 0x55638b178010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.153+0000 7f712b462740 1 --2- 172.21.15.120:0/1670048923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55638b1705d0 0x55638b17ad60 secure :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f711800b510 tx=0x7f7118017040 comp rx=0 tx=0).stop 2024-09-19T09:19:48.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.153+0000 7f712b462740 1 -- 172.21.15.120:0/1670048923 >> 172.21.15.120:0/1670048923 conn(0x55638b139310 msgr2=0x55638b172960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:48.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.153+0000 7f7106ff5640 0 client.29990 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:19:48.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.153+0000 7f712b462740 1 -- 172.21.15.120:0/1670048923 shutdown_connections 2024-09-19T09:19:48.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.153+0000 7f712b462740 1 -- 172.21.15.120:0/1670048923 wait complete. 2024-09-19T09:19:48.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.153+0000 7f712b462740 20 client.29990 trim_cache size 0 max 16384 2024-09-19T09:19:48.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.170+0000 7f712b462740 1 Processor -- start 2024-09-19T09:19:48.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.170+0000 7f712b462740 1 -- start start 2024-09-19T09:19:48.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.170+0000 7f712b462740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638b171b40 0x55638b1872b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:48.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.170+0000 7f712b462740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55638b17b370 0x55638b176ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:48.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.170+0000 7f712b462740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55638b173120 0x55638b1735b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:48.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.170+0000 7f712b462740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f71241b3d00 con 0x55638b17b370 2024-09-19T09:19:48.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.170+0000 7f712b462740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f71241b5840 con 0x55638b173120 2024-09-19T09:19:48.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.170+0000 7f712b462740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f71241b43d0 con 0x55638b171b40 2024-09-19T09:19:48.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.170+0000 7f7122d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55638b17b370 0x55638b176ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:48.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.170+0000 7f712259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638b171b40 0x55638b1872b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:48.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.170+0000 7f712259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638b171b40 0x55638b1872b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:34220/0 (socket says 172.21.15.120:34220) 2024-09-19T09:19:48.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.170+0000 7f712259c640 1 -- 172.21.15.120:0/547551163 learned_addr learned my addr 172.21.15.120:0/547551163 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:48.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.170+0000 7f712259c640 1 --2- 172.21.15.120:0/547551163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55638b173120 0x55638b1735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:48.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.170+0000 7f712259c640 1 -- 172.21.15.120:0/547551163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55638b173120 msgr2=0x55638b1735b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:48.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.170+0000 7f712259c640 1 --2- 172.21.15.120:0/547551163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55638b173120 0x55638b1735b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.170+0000 7f712259c640 1 -- 172.21.15.120:0/547551163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55638b17b370 msgr2=0x55638b176ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:48.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.170+0000 7f712259c640 1 --2- 172.21.15.120:0/547551163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55638b17b370 0x55638b176ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.171+0000 7f712259c640 1 -- 172.21.15.120:0/547551163 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55638b177f40 con 0x55638b171b40 2024-09-19T09:19:48.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.171+0000 7f712259c640 1 --2- 172.21.15.120:0/547551163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638b171b40 0x55638b1872b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f711800b510 tx=0x7f7118010530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ac334dc807c771e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:48.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.171+0000 7f7108ff9640 1 -- 172.21.15.120:0/547551163 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711800b540 con 0x55638b171b40 2024-09-19T09:19:48.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.171+0000 7f7108ff9640 1 -- 172.21.15.120:0/547551163 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71180043e0 con 0x55638b171b40 2024-09-19T09:19:48.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.171+0000 7f7108ff9640 1 -- 172.21.15.120:0/547551163 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711802c820 con 0x55638b171b40 2024-09-19T09:19:48.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.172+0000 7f712b462740 1 -- 172.21.15.120:0/547551163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638b171b40 msgr2=0x55638b1872b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:48.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.172+0000 7f712b462740 1 --2- 172.21.15.120:0/547551163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638b171b40 0x55638b1872b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f711800b510 tx=0x7f7118010530 comp rx=0 tx=0).stop 2024-09-19T09:19:48.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.172+0000 7f712b462740 1 -- 172.21.15.120:0/547551163 shutdown_connections 2024-09-19T09:19:48.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.172+0000 7f712b462740 1 --2- 172.21.15.120:0/547551163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55638b173120 0x55638b1735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.172+0000 7f712b462740 1 --2- 172.21.15.120:0/547551163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55638b17b370 0x55638b176ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.172+0000 7f712b462740 1 --2- 172.21.15.120:0/547551163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638b171b40 0x55638b1872b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.172+0000 7f712b462740 1 -- 172.21.15.120:0/547551163 >> 172.21.15.120:0/547551163 conn(0x55638b139310 msgr2=0x55638b1767b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:48.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.172+0000 7f712b462740 1 -- 172.21.15.120:0/547551163 shutdown_connections 2024-09-19T09:19:48.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.172+0000 7f712b462740 1 -- 172.21.15.120:0/547551163 wait complete. 2024-09-19T09:19:48.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.172+0000 7f712b462740 1 -- start start 2024-09-19T09:19:48.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.172+0000 7f712b462740 1 Processor -- start 2024-09-19T09:19:48.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.173+0000 7f712b462740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638b171b40 0x55638b17e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:48.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.173+0000 7f712b462740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55638b173120 0x55638b17eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:48.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.173+0000 7f712b462740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55638b17b370 0x55638b1f3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:48.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.173+0000 7f712b462740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f71241b5170 con 0x55638b17b370 2024-09-19T09:19:48.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.173+0000 7f712b462740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f71241b4aa0 con 0x55638b173120 2024-09-19T09:19:48.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.173+0000 7f712b462740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f71241b1cc0 con 0x55638b171b40 2024-09-19T09:19:48.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.173+0000 7f712259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638b171b40 0x55638b17e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:48.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.173+0000 7f712b462740 10 client.0 _ll_register_callbacks cb 0x7f711800b890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:48.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.173+0000 7f712259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55638b17b370 0x55638b1f3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:48.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.173+0000 7f7122d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55638b173120 0x55638b17eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:48.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.173+0000 7f7122d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55638b173120 0x55638b17eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:43938/0 (socket says 172.21.15.120:43938) 2024-09-19T09:19:48.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.173+0000 7f7122d9d640 1 -- 172.21.15.120:0/341135400 learned_addr learned my addr 172.21.15.120:0/341135400 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:48.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.173+0000 7f712259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55638b17b370 0x55638b1f3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35220/0 (socket says 172.21.15.120:35220) 2024-09-19T09:19:48.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.173+0000 7f7122d9d640 1 -- 172.21.15.120:0/341135400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638b171b40 msgr2=0x55638b17e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:48.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.173+0000 7f7122d9d640 1 --2- 172.21.15.120:0/341135400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638b171b40 0x55638b17e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.173+0000 7f7122d9d640 1 -- 172.21.15.120:0/341135400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55638b17b370 msgr2=0x55638b1f3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:48.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.173+0000 7f7122d9d640 1 --2- 172.21.15.120:0/341135400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55638b17b370 0x55638b1f3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.173+0000 7f7122d9d640 1 -- 172.21.15.120:0/341135400 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55638b1f3d80 con 0x55638b173120 2024-09-19T09:19:48.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.173+0000 7f712259c640 1 --2- 172.21.15.120:0/341135400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55638b17b370 0x55638b1f3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:48.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.174+0000 7f7122d9d640 1 --2- 172.21.15.120:0/341135400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55638b173120 0x55638b17eae0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7114031000 tx=0x7f7114004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf23365744ff4d0b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:48.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.174+0000 7f7106ff5640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:19:48.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.174+0000 7f7106ff5640 1 -- 172.21.15.120:0/341135400 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7114037040 con 0x55638b173120 2024-09-19T09:19:48.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.174+0000 7f712b462740 1 -- 172.21.15.120:0/341135400 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55638b0ddd70 con 0x55638b173120 2024-09-19T09:19:48.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.174+0000 7f7106ff5640 1 -- 172.21.15.120:0/341135400 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f7114030750 con 0x55638b173120 2024-09-19T09:19:48.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.174+0000 7f7106ff5640 1 -- 172.21.15.120:0/341135400 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f71140044a0 con 0x55638b173120 2024-09-19T09:19:48.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.174+0000 7f712b462740 10 client.29996 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:48.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.174+0000 7f712b462740 1 -- 172.21.15.120:0/341135400 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55638b1f4f60 con 0x55638b173120 2024-09-19T09:19:48.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.174+0000 7f7106ff5640 1 -- 172.21.15.120:0/341135400 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55638b0ddd70 con 0x55638b173120 2024-09-19T09:19:48.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.175+0000 7f7106ff5640 1 -- 172.21.15.120:0/341135400 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f7114032560 con 0x55638b173120 2024-09-19T09:19:48.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.175+0000 7f712b462740 10 client.29996 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:48.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.175+0000 7f712b462740 10 client.29996 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:19:48.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.175+0000 7f712b462740 20 client.29996 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:48.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.175+0000 7f712b462740 1 --2- 172.21.15.120:0/341135400 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55638b1f5760 0x55638b1f7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:48.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.175+0000 7f712b462740 4 client.29996 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:19:48.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.175+0000 7f712b462740 1 -- 172.21.15.120:0/341135400 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55638b05a170 con 0x55638b1f5760 2024-09-19T09:19:48.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.175+0000 7f712259c640 1 --2- 172.21.15.120:0/341135400 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55638b1f5760 0x55638b1f7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:48.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.175+0000 7f7106ff5640 10 client.29996 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:19:48.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.175+0000 7f712259c640 1 --2- 172.21.15.120:0/341135400 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55638b1f5760 0x55638b1f7b50 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ff2a30dd59375066 server_cookie=de20283929e48175 in_seq=0 out_seq=0 2024-09-19T09:19:48.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.175+0000 7f7106ff5640 1 -- 172.21.15.120:0/341135400 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55638b05a170 con 0x55638b1f5760 2024-09-19T09:19:48.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.175+0000 7f7106ff5640 10 client.29996 handle_command_reply: tid=0 2024-09-19T09:19:48.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.176+0000 7f712b462740 1 client.29996 shutdown 2024-09-19T09:19:48.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.176+0000 7f712b462740 1 -- 172.21.15.120:0/341135400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55638b173120 msgr2=0x55638b17eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:48.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.176+0000 7f712b462740 1 --2- 172.21.15.120:0/341135400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55638b173120 0x55638b17eae0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7114031000 tx=0x7f7114004270 comp rx=0 tx=0).stop 2024-09-19T09:19:48.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.176+0000 7f712b462740 1 -- 172.21.15.120:0/341135400 shutdown_connections 2024-09-19T09:19:48.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.176+0000 7f712b462740 1 --2- 172.21.15.120:0/341135400 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55638b1f5760 0x55638b1f7b50 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.176+0000 7f712b462740 1 --2- 172.21.15.120:0/341135400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55638b17b370 0x55638b1f3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.176+0000 7f712b462740 1 --2- 172.21.15.120:0/341135400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55638b173120 0x55638b17eae0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.176+0000 7f7106ff5640 0 client.29996 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:19:48.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.176+0000 7f712b462740 1 --2- 172.21.15.120:0/341135400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55638b171b40 0x55638b17e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.176+0000 7f712b462740 1 -- 172.21.15.120:0/341135400 >> 172.21.15.120:0/341135400 conn(0x55638b139310 msgr2=0x55638b1767b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:48.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.176+0000 7f712b462740 1 -- 172.21.15.120:0/341135400 shutdown_connections 2024-09-19T09:19:48.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.176+0000 7f712b462740 1 -- 172.21.15.120:0/341135400 wait complete. 2024-09-19T09:19:48.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.176+0000 7f712b462740 20 client.29996 trim_cache size 0 max 16384 2024-09-19T09:19:48.183 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:19:48.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.176+0000 7f7129089640 1 -- 172.21.15.120:0/449086424 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f710c0a3240 msgr2=0x7f710c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:48.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.176+0000 7f7129089640 1 --2- 172.21.15.120:0/449086424 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f710c0a3240 0x7f710c0a5700 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f711c09c120 tx=0x7f711c098040 comp rx=0 tx=0).stop 2024-09-19T09:19:48.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.176+0000 7f7129089640 1 -- 172.21.15.120:0/449086424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7124106a90 msgr2=0x7f7124109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:48.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.176+0000 7f7129089640 1 --2- 172.21.15.120:0/449086424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7124106a90 0x7f7124109b30 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f711c00eb10 tx=0x7f711c00c2d0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.177+0000 7f7129089640 1 -- 172.21.15.120:0/449086424 shutdown_connections 2024-09-19T09:19:48.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.177+0000 7f7129089640 1 --2- 172.21.15.120:0/449086424 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f710c0a3240 0x7f710c0a5700 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.177+0000 7f7129089640 1 --2- 172.21.15.120:0/449086424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7124106a90 0x7f7124109b30 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.177+0000 7f7129089640 1 --2- 172.21.15.120:0/449086424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7124101430 0x7f71241095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.177+0000 7f7129089640 1 --2- 172.21.15.120:0/449086424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7124100a90 0x7f712410d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:48.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.177+0000 7f7129089640 1 -- 172.21.15.120:0/449086424 >> 172.21.15.120:0/449086424 conn(0x7f71240fc7d0 msgr2=0x7f71240fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:48.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.177+0000 7f7129089640 1 -- 172.21.15.120:0/449086424 shutdown_connections 2024-09-19T09:19:48.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:48.177+0000 7f7129089640 1 -- 172.21.15.120:0/449086424 wait complete. 2024-09-19T09:19:48.363 INFO:tasks.workunit.client.0.smithi120.stdout: 10 86710 16.16 MB/sec execute 103 sec latency 208.693 ms 2024-09-19T09:19:49.017 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:19:49.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:48 smithi120 ceph-mon[31150]: pgmap v853: 97 pgs: 97 active+clean; 344 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.9 MiB/s wr, 761 op/s 2024-09-19T09:19:49.358 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:19:49.363 INFO:tasks.workunit.client.0.smithi120.stdout: 10 87409 16.43 MB/sec execute 104 sec latency 56.000 ms 2024-09-19T09:19:49.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:48 smithi196 ceph-mon[35150]: pgmap v853: 97 pgs: 97 active+clean; 344 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.9 MiB/s wr, 761 op/s 2024-09-19T09:19:49.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:48 smithi153 ceph-mon[35071]: pgmap v853: 97 pgs: 97 active+clean; 344 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.9 MiB/s wr, 761 op/s 2024-09-19T09:19:49.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.870+0000 7f8ed3c99640 1 -- 172.21.15.120:0/3989318352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ecc101290 msgr2=0x7f8ecc107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:49.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.870+0000 7f8ed3c99640 1 --2- 172.21.15.120:0/3989318352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ecc101290 0x7f8ecc107a50 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc009930 tx=0x7f8ebc02a420 comp rx=0 tx=0).stop 2024-09-19T09:19:49.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.870+0000 7f8ed3c99640 1 -- 172.21.15.120:0/3989318352 shutdown_connections 2024-09-19T09:19:49.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.870+0000 7f8ed3c99640 1 --2- 172.21.15.120:0/3989318352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ecc108160 0x7f8ecc10eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.870+0000 7f8ed3c99640 1 --2- 172.21.15.120:0/3989318352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ecc101290 0x7f8ecc107a50 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.870+0000 7f8ed3c99640 1 --2- 172.21.15.120:0/3989318352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ecc1008e0 0x7f8ecc100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.870+0000 7f8ed3c99640 1 -- 172.21.15.120:0/3989318352 >> 172.21.15.120:0/3989318352 conn(0x7f8ecc077210 msgr2=0x7f8ecc077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:49.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.870+0000 7f8ed3c99640 1 -- 172.21.15.120:0/3989318352 shutdown_connections 2024-09-19T09:19:49.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.870+0000 7f8ed3c99640 1 -- 172.21.15.120:0/3989318352 wait complete. 2024-09-19T09:19:49.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.870+0000 7f8ed3c99640 1 Processor -- start 2024-09-19T09:19:49.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.870+0000 7f8ed3c99640 1 -- start start 2024-09-19T09:19:49.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.871+0000 7f8ed3c99640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ecc1008e0 0x7f8ecc101860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:49.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.871+0000 7f8ed3c99640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ecc101290 0x7f8ecc101da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:49.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.871+0000 7f8ed3c99640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ecc108160 0x7f8ecc1022e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:49.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.871+0000 7f8ed3c99640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc111f70 con 0x7f8ecc108160 2024-09-19T09:19:49.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.871+0000 7f8ed3c99640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc111df0 con 0x7f8ecc101290 2024-09-19T09:19:49.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.871+0000 7f8ed3c99640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1120f0 con 0x7f8ecc1008e0 2024-09-19T09:19:49.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.871+0000 7f8ed2236640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ecc108160 0x7f8ecc1022e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:49.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.871+0000 7f8ed2236640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ecc108160 0x7f8ecc1022e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35238/0 (socket says 172.21.15.120:35238) 2024-09-19T09:19:49.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.871+0000 7f8ed2236640 1 -- 172.21.15.120:0/3915586402 learned_addr learned my addr 172.21.15.120:0/3915586402 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:49.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.871+0000 7f8ed1234640 1 --2- 172.21.15.120:0/3915586402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ecc101290 0x7f8ecc101da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:49.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.871+0000 7f8ed1a35640 1 --2- 172.21.15.120:0/3915586402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ecc1008e0 0x7f8ecc101860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:49.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.871+0000 7f8ed2236640 1 -- 172.21.15.120:0/3915586402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ecc1008e0 msgr2=0x7f8ecc101860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:49.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.871+0000 7f8ed2236640 1 --2- 172.21.15.120:0/3915586402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ecc1008e0 0x7f8ecc101860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.871+0000 7f8ed2236640 1 -- 172.21.15.120:0/3915586402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ecc101290 msgr2=0x7f8ecc101da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:49.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.871+0000 7f8ed2236640 1 --2- 172.21.15.120:0/3915586402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ecc101290 0x7f8ecc101da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.871+0000 7f8ed2236640 1 -- 172.21.15.120:0/3915586402 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ecc1a6a30 con 0x7f8ecc108160 2024-09-19T09:19:49.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.871+0000 7f8ed2236640 1 --2- 172.21.15.120:0/3915586402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ecc108160 0x7f8ecc1022e0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400eb10 tx=0x7f8ec400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:49.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.871+0000 7f8ec2ffd640 1 -- 172.21.15.120:0/3915586402 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4004b20 con 0x7f8ecc108160 2024-09-19T09:19:49.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.871+0000 7f8ec2ffd640 1 -- 172.21.15.120:0/3915586402 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec4004cc0 con 0x7f8ecc108160 2024-09-19T09:19:49.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.871+0000 7f8ed3c99640 1 -- 172.21.15.120:0/3915586402 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ecc1a6d20 con 0x7f8ecc108160 2024-09-19T09:19:49.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.871+0000 7f8ec2ffd640 1 -- 172.21.15.120:0/3915586402 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4010760 con 0x7f8ecc108160 2024-09-19T09:19:49.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.871+0000 7f8ed3c99640 1 -- 172.21.15.120:0/3915586402 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ecc1a71e0 con 0x7f8ecc108160 2024-09-19T09:19:49.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.874+0000 7f8ec2ffd640 1 -- 172.21.15.120:0/3915586402 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f8ec4005020 con 0x7f8ecc108160 2024-09-19T09:19:49.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.874+0000 7f8ed6072740 1 Processor -- start 2024-09-19T09:19:49.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.875+0000 7f8ed6072740 1 -- start start 2024-09-19T09:19:49.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.875+0000 7f8ed6072740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e2b625b40 0x563e2b624090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:49.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.875+0000 7f8ed6072740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2b6245d0 0x563e2b624a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:49.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.875+0000 7f8ed6072740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e2b62f370 0x563e2b631760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:49.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.875+0000 7f8ed6072740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563e2b58a2f0 con 0x563e2b62f370 2024-09-19T09:19:49.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.875+0000 7f8ed6072740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563e2b531cf0 con 0x563e2b625b40 2024-09-19T09:19:49.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.875+0000 7f8ed6072740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563e2b5862b0 con 0x563e2b6245d0 2024-09-19T09:19:49.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.875+0000 7f8ed1a35640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2b6245d0 0x563e2b624a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:49.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.875+0000 7f8ed1234640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e2b625b40 0x563e2b624090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:49.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.875+0000 7f8ed1a35640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2b6245d0 0x563e2b624a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:34250/0 (socket says 172.21.15.120:34250) 2024-09-19T09:19:49.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.875+0000 7f8ed1a35640 1 -- 172.21.15.120:0/3974844964 learned_addr learned my addr 172.21.15.120:0/3974844964 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:49.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.875+0000 7f8ed1234640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e2b625b40 0x563e2b624090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:43966/0 (socket says 172.21.15.120:43966) 2024-09-19T09:19:49.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.875+0000 7f8ed1234640 1 --2- 172.21.15.120:0/3974844964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e2b62f370 0x563e2b631760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:49.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.875+0000 7f8ec2ffd640 1 --2- 172.21.15.120:0/3915586402 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8eb80a3240 0x7f8eb80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:49.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.875+0000 7f8ed1a35640 1 -- 172.21.15.120:0/3974844964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e2b625b40 msgr2=0x563e2b624090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:49.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.875+0000 7f8ed1a35640 1 --2- 172.21.15.120:0/3974844964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e2b625b40 0x563e2b624090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.875+0000 7f8ed1a35640 1 -- 172.21.15.120:0/3974844964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e2b62f370 msgr2=0x563e2b631760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:49.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.875+0000 7f8ed1a35640 1 --2- 172.21.15.120:0/3974844964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e2b62f370 0x563e2b631760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.875+0000 7f8ed1a35640 1 -- 172.21.15.120:0/3974844964 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e2b5163a0 con 0x563e2b6245d0 2024-09-19T09:19:49.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.876+0000 7f8ed2236640 1 --2- 172.21.15.120:0/3915586402 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8eb80a3240 0x7f8eb80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:49.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.876+0000 7f8ec2ffd640 1 -- 172.21.15.120:0/3915586402 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f8ec40cfe00 con 0x7f8ecc108160 2024-09-19T09:19:49.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.876+0000 7f8ed1a35640 1 --2- 172.21.15.120:0/3974844964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2b6245d0 0x563e2b624a10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800bcd0 tx=0x7f8ec80107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e84fae61ed64427 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:49.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.876+0000 7f8eb7fff640 1 -- 172.21.15.120:0/3974844964 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8005220 con 0x563e2b6245d0 2024-09-19T09:19:49.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.876+0000 7f8eb7fff640 1 -- 172.21.15.120:0/3974844964 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec8004c80 con 0x563e2b6245d0 2024-09-19T09:19:49.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.876+0000 7f8eb7fff640 1 -- 172.21.15.120:0/3974844964 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8011710 con 0x563e2b6245d0 2024-09-19T09:19:49.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.877+0000 7f8ed6072740 1 -- 172.21.15.120:0/3974844964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2b6245d0 msgr2=0x563e2b624a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:49.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.877+0000 7f8ed6072740 1 --2- 172.21.15.120:0/3974844964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2b6245d0 0x563e2b624a10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800bcd0 tx=0x7f8ec80107a0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.877+0000 7f8ed2236640 1 --2- 172.21.15.120:0/3915586402 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8eb80a3240 0x7f8eb80a5700 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40cd900 tx=0x7f8ec40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:49.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.877+0000 7f8ed6072740 1 -- 172.21.15.120:0/3974844964 shutdown_connections 2024-09-19T09:19:49.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.877+0000 7f8ed6072740 1 --2- 172.21.15.120:0/3974844964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e2b62f370 0x563e2b631760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.877+0000 7f8ed6072740 1 --2- 172.21.15.120:0/3974844964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2b6245d0 0x563e2b624a10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.877+0000 7f8ed6072740 1 --2- 172.21.15.120:0/3974844964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e2b625b40 0x563e2b624090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.877+0000 7f8ed6072740 1 -- 172.21.15.120:0/3974844964 >> 172.21.15.120:0/3974844964 conn(0x563e2b5ed310 msgr2=0x563e2b620c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:49.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.877+0000 7f8ed6072740 1 -- 172.21.15.120:0/3974844964 shutdown_connections 2024-09-19T09:19:49.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.877+0000 7f8ed6072740 1 -- 172.21.15.120:0/3974844964 wait complete. 2024-09-19T09:19:49.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.877+0000 7f8ed6072740 1 -- start start 2024-09-19T09:19:49.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.877+0000 7f8ed6072740 1 Processor -- start 2024-09-19T09:19:49.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.878+0000 7f8ed6072740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2b6245d0 0x563e2b62eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:49.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.878+0000 7f8ed6072740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e2b625b40 0x563e2b62bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:49.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.878+0000 7f8ed6072740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e2b62f370 0x563e2b62c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:49.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.878+0000 7f8ed6072740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563e2b632070 con 0x563e2b62f370 2024-09-19T09:19:49.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.878+0000 7f8ed6072740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563e2b545060 con 0x563e2b625b40 2024-09-19T09:19:49.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.878+0000 7f8ed6072740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563e2b6321f0 con 0x563e2b6245d0 2024-09-19T09:19:49.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.878+0000 7f8ed6072740 10 client.0 _ll_register_callbacks cb 0x563e2b2988c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:49.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.879+0000 7f8ed1234640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e2b62f370 0x563e2b62c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:49.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.879+0000 7f8ed1a35640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e2b625b40 0x563e2b62bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:49.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.879+0000 7f8ed1234640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e2b62f370 0x563e2b62c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35262/0 (socket says 172.21.15.120:35262) 2024-09-19T09:19:49.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.879+0000 7f8ed1234640 1 -- 172.21.15.120:0/2617237902 learned_addr learned my addr 172.21.15.120:0/2617237902 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:49.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.879+0000 7f8ed1234640 1 --2- 172.21.15.120:0/2617237902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2b6245d0 0x563e2b62eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:49.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.879+0000 7f8ed1234640 1 -- 172.21.15.120:0/2617237902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2b6245d0 msgr2=0x563e2b62eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:49.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.879+0000 7f8ed1234640 1 --2- 172.21.15.120:0/2617237902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2b6245d0 0x563e2b62eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.879+0000 7f8ed1234640 1 -- 172.21.15.120:0/2617237902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e2b625b40 msgr2=0x563e2b62bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:49.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.879+0000 7f8ed1234640 1 --2- 172.21.15.120:0/2617237902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e2b625b40 0x563e2b62bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.879+0000 7f8ed1234640 1 -- 172.21.15.120:0/2617237902 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e2b2fb3c0 con 0x563e2b62f370 2024-09-19T09:19:49.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.879+0000 7f8ed1234640 1 --2- 172.21.15.120:0/2617237902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e2b62f370 0x563e2b62c490 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc0096c0 tx=0x7f8ebc02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41694cf4cad2a5de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:49.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.879+0000 7f8eb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:19:49.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.879+0000 7f8eb57fa640 1 -- 172.21.15.120:0/2617237902 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc035040 con 0x563e2b62f370 2024-09-19T09:19:49.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.879+0000 7f8eb57fa640 1 -- 172.21.15.120:0/2617237902 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f8ebc03e7d0 con 0x563e2b62f370 2024-09-19T09:19:49.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.879+0000 7f8ed6072740 1 -- 172.21.15.120:0/2617237902 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec8004c80 con 0x563e2b62f370 2024-09-19T09:19:49.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.879+0000 7f8eb57fa640 1 -- 172.21.15.120:0/2617237902 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8ebc03e9d0 con 0x563e2b62f370 2024-09-19T09:19:49.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.879+0000 7f8ed6072740 10 client.21147 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:49.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.879+0000 7f8ed6072740 1 -- 172.21.15.120:0/2617237902 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563e2b63da90 con 0x563e2b62f370 2024-09-19T09:19:49.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.880+0000 7f8eb57fa640 1 -- 172.21.15.120:0/2617237902 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec8004c80 con 0x563e2b62f370 2024-09-19T09:19:49.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.880+0000 7f8eb57fa640 1 -- 172.21.15.120:0/2617237902 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f8ebc003cb0 con 0x563e2b62f370 2024-09-19T09:19:49.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.880+0000 7f8ed6072740 10 client.21147 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:49.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.880+0000 7f8ed6072740 10 client.21147 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:19:49.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.880+0000 7f8ed6072740 20 client.21147 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:49.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.880+0000 7f8ed6072740 1 --2- 172.21.15.120:0/2617237902 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x563e2b63eea0 0x563e2b641290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:49.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.880+0000 7f8ed6072740 4 client.21147 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:19:49.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.880+0000 7f8ed6072740 1 -- 172.21.15.120:0/2617237902 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e2b597860 con 0x563e2b63eea0 2024-09-19T09:19:49.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.880+0000 7f8ed1a35640 1 --2- 172.21.15.120:0/2617237902 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x563e2b63eea0 0x563e2b641290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:49.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.881+0000 7f8eb57fa640 10 client.21147 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:19:49.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.881+0000 7f8ed1a35640 1 --2- 172.21.15.120:0/2617237902 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x563e2b63eea0 0x563e2b641290 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8313e0880fa6d14f server_cookie=b781805ad02074ba in_seq=0 out_seq=0 2024-09-19T09:19:49.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.881+0000 7f8eb57fa640 1 -- 172.21.15.120:0/2617237902 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e2b597860 con 0x563e2b63eea0 2024-09-19T09:19:49.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.881+0000 7f8eb57fa640 10 client.21147 handle_command_reply: tid=0 2024-09-19T09:19:49.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.881+0000 7f8ed6072740 1 client.21147 shutdown 2024-09-19T09:19:49.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.881+0000 7f8ed6072740 1 -- 172.21.15.120:0/2617237902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e2b62f370 msgr2=0x563e2b62c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:49.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.881+0000 7f8ed6072740 1 --2- 172.21.15.120:0/2617237902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e2b62f370 0x563e2b62c490 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc0096c0 tx=0x7f8ebc02a420 comp rx=0 tx=0).stop 2024-09-19T09:19:49.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.882+0000 7f8ed6072740 1 -- 172.21.15.120:0/2617237902 shutdown_connections 2024-09-19T09:19:49.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.882+0000 7f8ed6072740 1 --2- 172.21.15.120:0/2617237902 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x563e2b63eea0 0x563e2b641290 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.882+0000 7f8ed6072740 1 --2- 172.21.15.120:0/2617237902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e2b62f370 0x563e2b62c490 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.882+0000 7f8ed6072740 1 --2- 172.21.15.120:0/2617237902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e2b625b40 0x563e2b62bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.882+0000 7f8ed6072740 1 --2- 172.21.15.120:0/2617237902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2b6245d0 0x563e2b62eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.882+0000 7f8eb57fa640 0 client.21147 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:19:49.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.882+0000 7f8ed6072740 1 -- 172.21.15.120:0/2617237902 >> 172.21.15.120:0/2617237902 conn(0x563e2b5ed310 msgr2=0x563e2b626960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:49.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.882+0000 7f8ed6072740 1 -- 172.21.15.120:0/2617237902 shutdown_connections 2024-09-19T09:19:49.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.882+0000 7f8ed6072740 1 -- 172.21.15.120:0/2617237902 wait complete. 2024-09-19T09:19:49.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.882+0000 7f8ed6072740 20 client.21147 trim_cache size 0 max 16384 2024-09-19T09:19:49.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.899+0000 7f8ed6072740 1 Processor -- start 2024-09-19T09:19:49.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.899+0000 7f8ed6072740 1 -- start start 2024-09-19T09:19:49.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.899+0000 7f8ed6072740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e2b6245d0 0x563e2b63b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:49.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.899+0000 7f8ed6072740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e2b625b40 0x563e2b62dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:49.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.899+0000 7f8ed6072740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2b62f370 0x563e2b630760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:49.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.899+0000 7f8ed6072740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1af850 con 0x563e2b625b40 2024-09-19T09:19:49.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.899+0000 7f8ed6072740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1b1390 con 0x563e2b6245d0 2024-09-19T09:19:49.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.899+0000 7f8ed6072740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1aff20 con 0x563e2b62f370 2024-09-19T09:19:49.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.899+0000 7f8ed1234640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2b62f370 0x563e2b630760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:49.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.899+0000 7f8ed1234640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2b62f370 0x563e2b630760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:34260/0 (socket says 172.21.15.120:34260) 2024-09-19T09:19:49.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.899+0000 7f8ed1234640 1 -- 172.21.15.120:0/2007783042 learned_addr learned my addr 172.21.15.120:0/2007783042 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:49.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.899+0000 7f8ed1234640 1 --2- 172.21.15.120:0/2007783042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e2b6245d0 0x563e2b63b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:49.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.899+0000 7f8ed1a35640 1 --2- 172.21.15.120:0/2007783042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e2b625b40 0x563e2b62dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:49.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.899+0000 7f8ed1234640 1 -- 172.21.15.120:0/2007783042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e2b6245d0 msgr2=0x563e2b63b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:49.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.900+0000 7f8ed1234640 1 --2- 172.21.15.120:0/2007783042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e2b6245d0 0x563e2b63b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.900+0000 7f8ed1234640 1 -- 172.21.15.120:0/2007783042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e2b625b40 msgr2=0x563e2b62dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:49.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.900+0000 7f8ed1234640 1 --2- 172.21.15.120:0/2007783042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e2b625b40 0x563e2b62dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.900+0000 7f8ed1234640 1 -- 172.21.15.120:0/2007783042 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e2b625300 con 0x563e2b62f370 2024-09-19T09:19:49.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.900+0000 7f8ed1a35640 1 --2- 172.21.15.120:0/2007783042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e2b625b40 0x563e2b62dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:19:49.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.900+0000 7f8ed1234640 1 --2- 172.21.15.120:0/2007783042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2b62f370 0x563e2b630760 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc03e570 tx=0x7f8ebc02a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0fce9e034cc9132 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:49.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.900+0000 7f8eb77fe640 1 -- 172.21.15.120:0/2007783042 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc035040 con 0x563e2b62f370 2024-09-19T09:19:49.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.900+0000 7f8eb77fe640 1 -- 172.21.15.120:0/2007783042 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ebc042030 con 0x563e2b62f370 2024-09-19T09:19:49.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.900+0000 7f8eb77fe640 1 -- 172.21.15.120:0/2007783042 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc03eb70 con 0x563e2b62f370 2024-09-19T09:19:49.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.901+0000 7f8ed6072740 1 -- 172.21.15.120:0/2007783042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2b62f370 msgr2=0x563e2b630760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:49.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.901+0000 7f8ed6072740 1 --2- 172.21.15.120:0/2007783042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2b62f370 0x563e2b630760 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc03e570 tx=0x7f8ebc02a420 comp rx=0 tx=0).stop 2024-09-19T09:19:49.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.901+0000 7f8ed6072740 1 -- 172.21.15.120:0/2007783042 shutdown_connections 2024-09-19T09:19:49.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.901+0000 7f8ed6072740 1 --2- 172.21.15.120:0/2007783042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2b62f370 0x563e2b630760 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.901+0000 7f8ed6072740 1 --2- 172.21.15.120:0/2007783042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e2b625b40 0x563e2b62dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.901+0000 7f8ed6072740 1 --2- 172.21.15.120:0/2007783042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e2b6245d0 0x563e2b63b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.901+0000 7f8ed6072740 1 -- 172.21.15.120:0/2007783042 >> 172.21.15.120:0/2007783042 conn(0x563e2b5ed310 msgr2=0x563e2b62d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:49.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.902+0000 7f8ed6072740 1 -- 172.21.15.120:0/2007783042 shutdown_connections 2024-09-19T09:19:49.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.902+0000 7f8ed6072740 1 -- 172.21.15.120:0/2007783042 wait complete. 2024-09-19T09:19:49.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.902+0000 7f8ed6072740 1 -- start start 2024-09-19T09:19:49.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.902+0000 7f8ed6072740 1 Processor -- start 2024-09-19T09:19:49.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.903+0000 7f8ed6072740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e2b6245d0 0x563e2b6374b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:49.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.903+0000 7f8ed6072740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e2b625b40 0x563e2b6379f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:49.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.903+0000 7f8ed6072740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2b62f370 0x563e2b6a6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:49.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.903+0000 7f8ed6072740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1b0cc0 con 0x563e2b625b40 2024-09-19T09:19:49.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.903+0000 7f8ed6072740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1b05f0 con 0x563e2b6245d0 2024-09-19T09:19:49.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.903+0000 7f8ed6072740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1ad810 con 0x563e2b62f370 2024-09-19T09:19:49.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.903+0000 7f8ed6072740 10 client.0 _ll_register_callbacks cb 0x7f8ebc03e460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:49.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.903+0000 7f8ed1a35640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e2b625b40 0x563e2b6379f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:49.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.903+0000 7f8ed1a35640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e2b625b40 0x563e2b6379f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35282/0 (socket says 172.21.15.120:35282) 2024-09-19T09:19:49.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.903+0000 7f8ed1a35640 1 -- 172.21.15.120:0/3848464018 learned_addr learned my addr 172.21.15.120:0/3848464018 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:49.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.903+0000 7f8ed1234640 1 --2- 172.21.15.120:0/3848464018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e2b6245d0 0x563e2b6374b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:49.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.903+0000 7f8ed1234640 1 --2- 172.21.15.120:0/3848464018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2b62f370 0x563e2b6a6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:49.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.903+0000 7f8ed1a35640 1 -- 172.21.15.120:0/3848464018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2b62f370 msgr2=0x563e2b6a6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:49.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.903+0000 7f8ed1a35640 1 --2- 172.21.15.120:0/3848464018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2b62f370 0x563e2b6a6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.903+0000 7f8ed1a35640 1 -- 172.21.15.120:0/3848464018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e2b6245d0 msgr2=0x563e2b6374b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:49.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.903+0000 7f8ed1a35640 1 --2- 172.21.15.120:0/3848464018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e2b6245d0 0x563e2b6374b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.903+0000 7f8ed1a35640 1 -- 172.21.15.120:0/3848464018 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e2b6a7040 con 0x563e2b625b40 2024-09-19T09:19:49.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.903+0000 7f8ed1234640 1 --2- 172.21.15.120:0/3848464018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e2b6245d0 0x563e2b6374b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:19:49.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.904+0000 7f8ed1a35640 1 --2- 172.21.15.120:0/3848464018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e2b625b40 0x563e2b6379f0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800bcd0 tx=0x7f8ec8010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b696103f939449 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:49.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.904+0000 7f8eb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:19:49.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.904+0000 7f8eb57fa640 1 -- 172.21.15.120:0/3848464018 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8005130 con 0x563e2b625b40 2024-09-19T09:19:49.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.904+0000 7f8eb57fa640 1 -- 172.21.15.120:0/3848464018 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f8ec8004020 con 0x563e2b625b40 2024-09-19T09:19:49.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.904+0000 7f8ed6072740 1 -- 172.21.15.120:0/3848464018 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ebc042030 con 0x563e2b625b40 2024-09-19T09:19:49.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.905+0000 7f8eb57fa640 1 -- 172.21.15.120:0/3848464018 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8ec8037050 con 0x563e2b625b40 2024-09-19T09:19:49.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.905+0000 7f8ed6072740 10 client.21153 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:49.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.905+0000 7f8ed6072740 1 -- 172.21.15.120:0/3848464018 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563e2b628a40 con 0x563e2b625b40 2024-09-19T09:19:49.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.905+0000 7f8eb57fa640 1 -- 172.21.15.120:0/3848464018 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ebc042030 con 0x563e2b625b40 2024-09-19T09:19:49.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.905+0000 7f8eb57fa640 1 -- 172.21.15.120:0/3848464018 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f8ec80035f0 con 0x563e2b625b40 2024-09-19T09:19:49.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.905+0000 7f8ed6072740 10 client.21153 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:49.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.905+0000 7f8ed6072740 10 client.21153 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:19:49.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.905+0000 7f8ed6072740 20 client.21153 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:49.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.905+0000 7f8ed6072740 1 --2- 172.21.15.120:0/3848464018 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x563e2b628ed0 0x563e2b6292e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:49.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.905+0000 7f8ed6072740 4 client.21153 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:19:49.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.905+0000 7f8ed6072740 1 -- 172.21.15.120:0/3848464018 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e2b591d70 con 0x563e2b628ed0 2024-09-19T09:19:49.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.905+0000 7f8ed1234640 1 --2- 172.21.15.120:0/3848464018 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x563e2b628ed0 0x563e2b6292e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:49.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.906+0000 7f8ed1234640 1 --2- 172.21.15.120:0/3848464018 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x563e2b628ed0 0x563e2b6292e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e80dbcb41b8c6832 server_cookie=fd9717ba1ed521ab in_seq=0 out_seq=0 2024-09-19T09:19:49.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.906+0000 7f8eb57fa640 10 client.21153 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:19:49.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.906+0000 7f8eb57fa640 1 -- 172.21.15.120:0/3848464018 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e2b591d70 con 0x563e2b628ed0 2024-09-19T09:19:49.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.906+0000 7f8eb57fa640 10 client.21153 handle_command_reply: tid=0 2024-09-19T09:19:49.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.906+0000 7f8ed6072740 1 client.21153 shutdown 2024-09-19T09:19:49.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.906+0000 7f8ed6072740 1 -- 172.21.15.120:0/3848464018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e2b625b40 msgr2=0x563e2b6379f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:49.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.906+0000 7f8ed6072740 1 --2- 172.21.15.120:0/3848464018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e2b625b40 0x563e2b6379f0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800bcd0 tx=0x7f8ec8010fd0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.906+0000 7f8ed6072740 1 -- 172.21.15.120:0/3848464018 shutdown_connections 2024-09-19T09:19:49.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.906+0000 7f8ed6072740 1 --2- 172.21.15.120:0/3848464018 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x563e2b628ed0 0x563e2b6292e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.906+0000 7f8ed6072740 1 --2- 172.21.15.120:0/3848464018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2b62f370 0x563e2b6a6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.906+0000 7f8ed6072740 1 --2- 172.21.15.120:0/3848464018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e2b625b40 0x563e2b6379f0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.906+0000 7f8ed6072740 1 --2- 172.21.15.120:0/3848464018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e2b6245d0 0x563e2b6374b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.906+0000 7f8eb57fa640 0 client.21153 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:19:49.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.906+0000 7f8ed6072740 1 -- 172.21.15.120:0/3848464018 >> 172.21.15.120:0/3848464018 conn(0x563e2b5ed310 msgr2=0x563e2b62dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:49.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.907+0000 7f8ed6072740 1 -- 172.21.15.120:0/3848464018 shutdown_connections 2024-09-19T09:19:49.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.907+0000 7f8ed6072740 1 -- 172.21.15.120:0/3848464018 wait complete. 2024-09-19T09:19:49.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.907+0000 7f8ed6072740 20 client.21153 trim_cache size 0 max 16384 2024-09-19T09:19:49.925 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:19:49.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.907+0000 7f8ed3c99640 1 -- 172.21.15.120:0/3915586402 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8eb80a3240 msgr2=0x7f8eb80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:49.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.907+0000 7f8ed3c99640 1 --2- 172.21.15.120:0/3915586402 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8eb80a3240 0x7f8eb80a5700 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40cd900 tx=0x7f8ec40058d0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.907+0000 7f8ed3c99640 1 -- 172.21.15.120:0/3915586402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ecc108160 msgr2=0x7f8ecc1022e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:49.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.907+0000 7f8ed3c99640 1 --2- 172.21.15.120:0/3915586402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ecc108160 0x7f8ecc1022e0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400eb10 tx=0x7f8ec400c2d0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.908+0000 7f8ed3c99640 1 -- 172.21.15.120:0/3915586402 shutdown_connections 2024-09-19T09:19:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.908+0000 7f8ed3c99640 1 --2- 172.21.15.120:0/3915586402 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8eb80a3240 0x7f8eb80a5700 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.908+0000 7f8ed3c99640 1 --2- 172.21.15.120:0/3915586402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ecc108160 0x7f8ecc1022e0 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.908+0000 7f8ed3c99640 1 --2- 172.21.15.120:0/3915586402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ecc101290 0x7f8ecc101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.908+0000 7f8ed3c99640 1 --2- 172.21.15.120:0/3915586402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ecc1008e0 0x7f8ecc101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.908+0000 7f8ed3c99640 1 -- 172.21.15.120:0/3915586402 >> 172.21.15.120:0/3915586402 conn(0x7f8ecc077210 msgr2=0x7f8ecc105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.908+0000 7f8ed3c99640 1 -- 172.21.15.120:0/3915586402 shutdown_connections 2024-09-19T09:19:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:49.908+0000 7f8ed3c99640 1 -- 172.21.15.120:0/3915586402 wait complete. 2024-09-19T09:19:50.363 INFO:tasks.workunit.client.0.smithi120.stdout: 10 87893 16.47 MB/sec execute 105 sec latency 62.643 ms 2024-09-19T09:19:50.633 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:19:50.906 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:19:51.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:50 smithi120 ceph-mon[31150]: pgmap v854: 97 pgs: 97 active+clean; 353 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.2 MiB/s wr, 818 op/s 2024-09-19T09:19:51.363 INFO:tasks.workunit.client.0.smithi120.stdout: 10 88406 16.57 MB/sec execute 106 sec latency 79.097 ms 2024-09-19T09:19:51.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:50 smithi196 ceph-mon[35150]: pgmap v854: 97 pgs: 97 active+clean; 353 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.2 MiB/s wr, 818 op/s 2024-09-19T09:19:51.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:50 smithi153 ceph-mon[35071]: pgmap v854: 97 pgs: 97 active+clean; 353 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.2 MiB/s wr, 818 op/s 2024-09-19T09:19:51.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.853+0000 7f571aa11640 1 -- 172.21.15.120:0/4262723937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5714102570 msgr2=0x7f57141029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:51.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.853+0000 7f571aa11640 1 --2- 172.21.15.120:0/4262723937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5714102570 0x7f57141029b0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f5708009900 tx=0x7f570802a480 comp rx=0 tx=0).stop 2024-09-19T09:19:51.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.853+0000 7f571aa11640 1 -- 172.21.15.120:0/4262723937 shutdown_connections 2024-09-19T09:19:51.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.853+0000 7f571aa11640 1 --2- 172.21.15.120:0/4262723937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5714102ef0 0x7f571410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.853+0000 7f571aa11640 1 --2- 172.21.15.120:0/4262723937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5714102570 0x7f57141029b0 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.853+0000 7f571aa11640 1 --2- 172.21.15.120:0/4262723937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5714100e20 0x7f5714101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.853+0000 7f571aa11640 1 -- 172.21.15.120:0/4262723937 >> 172.21.15.120:0/4262723937 conn(0x7f57140fc7d0 msgr2=0x7f57140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:51.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.853+0000 7f571aa11640 1 -- 172.21.15.120:0/4262723937 shutdown_connections 2024-09-19T09:19:51.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.853+0000 7f571aa11640 1 -- 172.21.15.120:0/4262723937 wait complete. 2024-09-19T09:19:51.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.854+0000 7f571aa11640 1 Processor -- start 2024-09-19T09:19:51.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.854+0000 7f571aa11640 1 -- start start 2024-09-19T09:19:51.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.854+0000 7f571aa11640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5714100e20 0x7f57141970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:51.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.854+0000 7f571aa11640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5714102570 0x7f5714197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:51.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.854+0000 7f571aa11640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5714102ef0 0x7f5714197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:51.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.854+0000 7f571aa11640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5714110f80 con 0x7f5714102ef0 2024-09-19T09:19:51.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.855+0000 7f571aa11640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5714110e00 con 0x7f5714102570 2024-09-19T09:19:51.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.855+0000 7f571aa11640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5714111100 con 0x7f5714100e20 2024-09-19T09:19:51.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.855+0000 7f5713fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5714100e20 0x7f57141970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:51.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.855+0000 7f5718fae640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5714102ef0 0x7f5714197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:51.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.855+0000 7f5713fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5714100e20 0x7f57141970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:37344/0 (socket says 172.21.15.120:37344) 2024-09-19T09:19:51.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.855+0000 7f5713fff640 1 -- 172.21.15.120:0/3293610654 learned_addr learned my addr 172.21.15.120:0/3293610654 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:51.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.855+0000 7f57137fe640 1 --2- 172.21.15.120:0/3293610654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5714102570 0x7f5714197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:51.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.855+0000 7f5713fff640 1 -- 172.21.15.120:0/3293610654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5714102570 msgr2=0x7f5714197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:51.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.855+0000 7f5713fff640 1 --2- 172.21.15.120:0/3293610654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5714102570 0x7f5714197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.855+0000 7f5713fff640 1 -- 172.21.15.120:0/3293610654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5714102ef0 msgr2=0x7f5714197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:51.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.855+0000 7f5713fff640 1 --2- 172.21.15.120:0/3293610654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5714102ef0 0x7f5714197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.855+0000 7f5713fff640 1 -- 172.21.15.120:0/3293610654 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57141a6950 con 0x7f5714100e20 2024-09-19T09:19:51.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.855+0000 7f5713fff640 1 --2- 172.21.15.120:0/3293610654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5714100e20 0x7f57141970e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f570400e9f0 tx=0x7f570400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:51.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.855+0000 7f57117fa640 1 -- 172.21.15.120:0/3293610654 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570400cd80 con 0x7f5714100e20 2024-09-19T09:19:51.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.855+0000 7f57117fa640 1 -- 172.21.15.120:0/3293610654 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5704010910 con 0x7f5714100e20 2024-09-19T09:19:51.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.855+0000 7f571aa11640 1 -- 172.21.15.120:0/3293610654 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57141a6c40 con 0x7f5714100e20 2024-09-19T09:19:51.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.855+0000 7f57117fa640 1 -- 172.21.15.120:0/3293610654 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5704018990 con 0x7f5714100e20 2024-09-19T09:19:51.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.855+0000 7f571aa11640 1 -- 172.21.15.120:0/3293610654 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57141a7150 con 0x7f5714100e20 2024-09-19T09:19:51.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.858+0000 7f571cdea740 1 Processor -- start 2024-09-19T09:19:51.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.858+0000 7f571cdea740 1 -- start start 2024-09-19T09:19:51.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.858+0000 7f57117fa640 1 -- 172.21.15.120:0/3293610654 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5704018b30 con 0x7f5714100e20 2024-09-19T09:19:51.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.858+0000 7f571cdea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569dcc85b40 0x5569dcc84090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:51.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.858+0000 7f571cdea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5569dcc845d0 0x5569dcc84a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:51.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.858+0000 7f571cdea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569dcc8f370 0x5569dcc91760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:51.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.858+0000 7f571cdea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5569dcbea2f0 con 0x5569dcc85b40 2024-09-19T09:19:51.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.858+0000 7f571cdea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5569dcb91cf0 con 0x5569dcc8f370 2024-09-19T09:19:51.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.858+0000 7f571cdea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5569dcbe62b0 con 0x5569dcc845d0 2024-09-19T09:19:51.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.858+0000 7f5718fae640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569dcc85b40 0x5569dcc84090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:51.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.858+0000 7f5718fae640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569dcc85b40 0x5569dcc84090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49586/0 (socket says 172.21.15.120:49586) 2024-09-19T09:19:51.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.858+0000 7f5718fae640 1 -- 172.21.15.120:0/3396989734 learned_addr learned my addr 172.21.15.120:0/3396989734 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:51.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.858+0000 7f5713fff640 1 --2- 172.21.15.120:0/3396989734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5569dcc845d0 0x5569dcc84a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:51.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.858+0000 7f57137fe640 1 --2- 172.21.15.120:0/3396989734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569dcc8f370 0x5569dcc91760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:51.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.859+0000 7f5718fae640 1 -- 172.21.15.120:0/3396989734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5569dcc845d0 msgr2=0x5569dcc84a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:51.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.859+0000 7f5718fae640 1 --2- 172.21.15.120:0/3396989734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5569dcc845d0 0x5569dcc84a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.859+0000 7f5718fae640 1 -- 172.21.15.120:0/3396989734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569dcc8f370 msgr2=0x5569dcc91760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:51.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.859+0000 7f5718fae640 1 --2- 172.21.15.120:0/3396989734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569dcc8f370 0x5569dcc91760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.859+0000 7f5718fae640 1 -- 172.21.15.120:0/3396989734 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569dcb763a0 con 0x5569dcc85b40 2024-09-19T09:19:51.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.859+0000 7f5718fae640 1 --2- 172.21.15.120:0/3396989734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569dcc85b40 0x5569dcc84090 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f570c009fd0 tx=0x7f570c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06a7a145b423784 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:51.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.859+0000 7f56fa7fc640 1 -- 172.21.15.120:0/3396989734 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570c004d20 con 0x5569dcc85b40 2024-09-19T09:19:51.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.859+0000 7f57117fa640 1 --2- 172.21.15.120:0/3293610654 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f56fc0a3240 0x7f56fc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:51.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.859+0000 7f56fa7fc640 1 -- 172.21.15.120:0/3396989734 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f570c004050 con 0x5569dcc85b40 2024-09-19T09:19:51.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.859+0000 7f56fa7fc640 1 -- 172.21.15.120:0/3396989734 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570c005240 con 0x5569dcc85b40 2024-09-19T09:19:51.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.859+0000 7f5713fff640 1 --2- 172.21.15.120:0/3293610654 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f56fc0a3240 0x7f56fc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:51.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.859+0000 7f57117fa640 1 -- 172.21.15.120:0/3293610654 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f57040cfc00 con 0x7f5714100e20 2024-09-19T09:19:51.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.860+0000 7f571cdea740 1 -- 172.21.15.120:0/3396989734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569dcc85b40 msgr2=0x5569dcc84090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:51.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.860+0000 7f5713fff640 1 --2- 172.21.15.120:0/3293610654 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f56fc0a3240 0x7f56fc0a5700 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5704098730 tx=0x7f57040d3680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:51.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.860+0000 7f571cdea740 1 --2- 172.21.15.120:0/3396989734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569dcc85b40 0x5569dcc84090 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f570c009fd0 tx=0x7f570c0118d0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.860+0000 7f571cdea740 1 -- 172.21.15.120:0/3396989734 shutdown_connections 2024-09-19T09:19:51.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.860+0000 7f571cdea740 1 --2- 172.21.15.120:0/3396989734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569dcc8f370 0x5569dcc91760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.860+0000 7f571cdea740 1 --2- 172.21.15.120:0/3396989734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5569dcc845d0 0x5569dcc84a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.860+0000 7f571cdea740 1 --2- 172.21.15.120:0/3396989734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569dcc85b40 0x5569dcc84090 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.860+0000 7f571cdea740 1 -- 172.21.15.120:0/3396989734 >> 172.21.15.120:0/3396989734 conn(0x5569dcc4d310 msgr2=0x5569dcc80c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:51.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.860+0000 7f571cdea740 1 -- 172.21.15.120:0/3396989734 shutdown_connections 2024-09-19T09:19:51.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.860+0000 7f571cdea740 1 -- 172.21.15.120:0/3396989734 wait complete. 2024-09-19T09:19:51.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.860+0000 7f571cdea740 1 -- start start 2024-09-19T09:19:51.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.861+0000 7f571cdea740 1 Processor -- start 2024-09-19T09:19:51.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.861+0000 7f571cdea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5569dcc845d0 0x5569dcc8ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:51.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.861+0000 7f571cdea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569dcc85b40 0x5569dcc8bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:51.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.861+0000 7f571cdea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569dcc8f370 0x5569dcc8c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:51.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.861+0000 7f571cdea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5569dcc92070 con 0x5569dcc8f370 2024-09-19T09:19:51.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.861+0000 7f571cdea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5569dcba5060 con 0x5569dcc85b40 2024-09-19T09:19:51.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.861+0000 7f571cdea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5569dcc921f0 con 0x5569dcc845d0 2024-09-19T09:19:51.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.861+0000 7f571cdea740 10 client.0 _ll_register_callbacks cb 0x5569dc8f88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:51.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.861+0000 7f5718fae640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5569dcc845d0 0x5569dcc8ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:51.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.861+0000 7f5718fae640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5569dcc845d0 0x5569dcc8ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:37356/0 (socket says 172.21.15.120:37356) 2024-09-19T09:19:51.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.861+0000 7f5718fae640 1 -- 172.21.15.120:0/1078848234 learned_addr learned my addr 172.21.15.120:0/1078848234 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:51.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.861+0000 7f57137fe640 1 --2- 172.21.15.120:0/1078848234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569dcc85b40 0x5569dcc8bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:51.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.861+0000 7f5718fae640 1 --2- 172.21.15.120:0/1078848234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569dcc8f370 0x5569dcc8c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:51.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.862+0000 7f5718fae640 1 -- 172.21.15.120:0/1078848234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569dcc85b40 msgr2=0x5569dcc8bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:51.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.862+0000 7f5718fae640 1 --2- 172.21.15.120:0/1078848234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569dcc85b40 0x5569dcc8bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.862+0000 7f5718fae640 1 -- 172.21.15.120:0/1078848234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569dcc8f370 msgr2=0x5569dcc8c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:51.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.862+0000 7f5718fae640 1 --2- 172.21.15.120:0/1078848234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569dcc8f370 0x5569dcc8c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.862+0000 7f5718fae640 1 -- 172.21.15.120:0/1078848234 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569dc95b3c0 con 0x5569dcc845d0 2024-09-19T09:19:51.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.862+0000 7f5718fae640 1 --2- 172.21.15.120:0/1078848234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5569dcc845d0 0x5569dcc8ecd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f570c004990 tx=0x7f570c009be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa08abaa965c9a63 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:51.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.862+0000 7f56f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:19:51.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.862+0000 7f56f37fe640 1 -- 172.21.15.120:0/1078848234 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570c02c020 con 0x5569dcc845d0 2024-09-19T09:19:51.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.862+0000 7f571cdea740 1 -- 172.21.15.120:0/1078848234 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f570c004050 con 0x5569dcc845d0 2024-09-19T09:19:51.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.862+0000 7f56f37fe640 1 -- 172.21.15.120:0/1078848234 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f570c00f4e0 con 0x5569dcc845d0 2024-09-19T09:19:51.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.862+0000 7f56f37fe640 1 -- 172.21.15.120:0/1078848234 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f570c00f6e0 con 0x5569dcc845d0 2024-09-19T09:19:51.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.862+0000 7f571cdea740 10 client.29845 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:51.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.862+0000 7f571cdea740 1 -- 172.21.15.120:0/1078848234 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5569dcc9da80 con 0x5569dcc845d0 2024-09-19T09:19:51.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.863+0000 7f56f37fe640 1 -- 172.21.15.120:0/1078848234 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f570c004050 con 0x5569dcc845d0 2024-09-19T09:19:51.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.863+0000 7f56f37fe640 1 -- 172.21.15.120:0/1078848234 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f570c0287a0 con 0x5569dcc845d0 2024-09-19T09:19:51.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.863+0000 7f571cdea740 10 client.29845 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:51.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.863+0000 7f571cdea740 10 client.29845 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:19:51.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.863+0000 7f571cdea740 20 client.29845 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:51.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.863+0000 7f571cdea740 1 --2- 172.21.15.120:0/1078848234 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5569dcc9ee90 0x5569dcca1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:51.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.863+0000 7f571cdea740 4 client.29845 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:19:51.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.863+0000 7f571cdea740 1 -- 172.21.15.120:0/1078848234 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569dcbf7860 con 0x5569dcc9ee90 2024-09-19T09:19:51.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.863+0000 7f57137fe640 1 --2- 172.21.15.120:0/1078848234 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5569dcc9ee90 0x5569dcca1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:51.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.864+0000 7f56f37fe640 10 client.29845 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:19:51.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.864+0000 7f57137fe640 1 --2- 172.21.15.120:0/1078848234 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5569dcc9ee90 0x5569dcca1280 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=68e1276e0844639f server_cookie=8c74d4f962c3983d in_seq=0 out_seq=0 2024-09-19T09:19:51.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.864+0000 7f56f37fe640 1 -- 172.21.15.120:0/1078848234 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5569dcbf7860 con 0x5569dcc9ee90 2024-09-19T09:19:51.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.864+0000 7f56f37fe640 10 client.29845 handle_command_reply: tid=0 2024-09-19T09:19:51.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.865+0000 7f571cdea740 1 client.29845 shutdown 2024-09-19T09:19:51.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.865+0000 7f571cdea740 1 -- 172.21.15.120:0/1078848234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5569dcc845d0 msgr2=0x5569dcc8ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:51.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.865+0000 7f571cdea740 1 --2- 172.21.15.120:0/1078848234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5569dcc845d0 0x5569dcc8ecd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f570c004990 tx=0x7f570c009be0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.865+0000 7f571cdea740 1 -- 172.21.15.120:0/1078848234 shutdown_connections 2024-09-19T09:19:51.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.865+0000 7f571cdea740 1 --2- 172.21.15.120:0/1078848234 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5569dcc9ee90 0x5569dcca1280 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.865+0000 7f571cdea740 1 --2- 172.21.15.120:0/1078848234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569dcc8f370 0x5569dcc8c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.865+0000 7f571cdea740 1 --2- 172.21.15.120:0/1078848234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569dcc85b40 0x5569dcc8bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.865+0000 7f56f37fe640 0 client.29845 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:19:51.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.865+0000 7f571cdea740 1 --2- 172.21.15.120:0/1078848234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5569dcc845d0 0x5569dcc8ecd0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.865+0000 7f571cdea740 1 -- 172.21.15.120:0/1078848234 >> 172.21.15.120:0/1078848234 conn(0x5569dcc4d310 msgr2=0x5569dcc86960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:51.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.865+0000 7f571cdea740 1 -- 172.21.15.120:0/1078848234 shutdown_connections 2024-09-19T09:19:51.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.865+0000 7f571cdea740 1 -- 172.21.15.120:0/1078848234 wait complete. 2024-09-19T09:19:51.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.865+0000 7f571cdea740 20 client.29845 trim_cache size 0 max 16384 2024-09-19T09:19:51.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.880+0000 7f571cdea740 1 Processor -- start 2024-09-19T09:19:51.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.880+0000 7f571cdea740 1 -- start start 2024-09-19T09:19:51.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.880+0000 7f571cdea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5569dcc845d0 0x5569dcc9b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:51.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.880+0000 7f571cdea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569dcc85b40 0x5569dcc8b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:51.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.880+0000 7f571cdea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569dcc8f370 0x5569dcc87120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:51.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.880+0000 7f571cdea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f57141af820 con 0x5569dcc8f370 2024-09-19T09:19:51.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.880+0000 7f571cdea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f57141b1360 con 0x5569dcc85b40 2024-09-19T09:19:51.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.880+0000 7f571cdea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f57141afef0 con 0x5569dcc845d0 2024-09-19T09:19:51.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.880+0000 7f5718fae640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569dcc8f370 0x5569dcc87120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:51.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.880+0000 7f57137fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569dcc85b40 0x5569dcc8b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:51.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.880+0000 7f5718fae640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569dcc8f370 0x5569dcc87120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49608/0 (socket says 172.21.15.120:49608) 2024-09-19T09:19:51.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.880+0000 7f5718fae640 1 -- 172.21.15.120:0/2018424096 learned_addr learned my addr 172.21.15.120:0/2018424096 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:51.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.880+0000 7f57137fe640 1 --2- 172.21.15.120:0/2018424096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569dcc85b40 0x5569dcc8b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52880/0 (socket says 172.21.15.120:52880) 2024-09-19T09:19:51.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.880+0000 7f5718fae640 1 --2- 172.21.15.120:0/2018424096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5569dcc845d0 0x5569dcc9b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:51.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.881+0000 7f5718fae640 1 -- 172.21.15.120:0/2018424096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5569dcc845d0 msgr2=0x5569dcc9b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:51.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.881+0000 7f5718fae640 1 --2- 172.21.15.120:0/2018424096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5569dcc845d0 0x5569dcc9b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.881+0000 7f5718fae640 1 -- 172.21.15.120:0/2018424096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569dcc85b40 msgr2=0x5569dcc8b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:51.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.881+0000 7f5718fae640 1 --2- 172.21.15.120:0/2018424096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569dcc85b40 0x5569dcc8b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.881+0000 7f5718fae640 1 -- 172.21.15.120:0/2018424096 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569dcc8f040 con 0x5569dcc8f370 2024-09-19T09:19:51.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.881+0000 7f5718fae640 1 --2- 172.21.15.120:0/2018424096 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569dcc8f370 0x5569dcc87120 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f570c005d70 tx=0x7f570c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4ab9e596bcf91e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:51.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.881+0000 7f56f9ffb640 1 -- 172.21.15.120:0/2018424096 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570c02c070 con 0x5569dcc8f370 2024-09-19T09:19:51.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.881+0000 7f56f9ffb640 1 -- 172.21.15.120:0/2018424096 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f570c004690 con 0x5569dcc8f370 2024-09-19T09:19:51.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.881+0000 7f56f9ffb640 1 -- 172.21.15.120:0/2018424096 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570c0049c0 con 0x5569dcc8f370 2024-09-19T09:19:51.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.882+0000 7f571cdea740 1 -- 172.21.15.120:0/2018424096 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569dcc8f370 msgr2=0x5569dcc87120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:51.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.882+0000 7f571cdea740 1 --2- 172.21.15.120:0/2018424096 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569dcc8f370 0x5569dcc87120 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f570c005d70 tx=0x7f570c0118d0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.882+0000 7f571cdea740 1 -- 172.21.15.120:0/2018424096 shutdown_connections 2024-09-19T09:19:51.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.882+0000 7f571cdea740 1 --2- 172.21.15.120:0/2018424096 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569dcc8f370 0x5569dcc87120 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.882+0000 7f571cdea740 1 --2- 172.21.15.120:0/2018424096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569dcc85b40 0x5569dcc8b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.882+0000 7f571cdea740 1 --2- 172.21.15.120:0/2018424096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5569dcc845d0 0x5569dcc9b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.882+0000 7f571cdea740 1 -- 172.21.15.120:0/2018424096 >> 172.21.15.120:0/2018424096 conn(0x5569dcc4d310 msgr2=0x5569dcc7fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:51.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.882+0000 7f571cdea740 1 -- 172.21.15.120:0/2018424096 shutdown_connections 2024-09-19T09:19:51.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.882+0000 7f571cdea740 1 -- 172.21.15.120:0/2018424096 wait complete. 2024-09-19T09:19:51.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.882+0000 7f571cdea740 1 -- start start 2024-09-19T09:19:51.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.883+0000 7f571cdea740 1 Processor -- start 2024-09-19T09:19:51.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.883+0000 7f571cdea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569dcc845d0 0x5569dcc92600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:51.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.883+0000 7f571cdea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569dcc85b40 0x5569dcc92b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:51.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.883+0000 7f571cdea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5569dcc8f370 0x5569dcd07960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:51.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.883+0000 7f571cdea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f57141b0c90 con 0x5569dcc85b40 2024-09-19T09:19:51.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.883+0000 7f57137fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569dcc85b40 0x5569dcc92b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:51.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.883+0000 7f5718fae640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569dcc845d0 0x5569dcc92600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:51.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.883+0000 7f57137fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569dcc85b40 0x5569dcc92b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49614/0 (socket says 172.21.15.120:49614) 2024-09-19T09:19:51.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.883+0000 7f57137fe640 1 -- 172.21.15.120:0/534126131 learned_addr learned my addr 172.21.15.120:0/534126131 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:51.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.883+0000 7f571cdea740 1 -- 172.21.15.120:0/534126131 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f57141b05c0 con 0x5569dcc845d0 2024-09-19T09:19:51.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.883+0000 7f571cdea740 1 -- 172.21.15.120:0/534126131 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f57141ad7e0 con 0x5569dcc8f370 2024-09-19T09:19:51.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.883+0000 7f571cdea740 10 client.0 _ll_register_callbacks cb 0x7f5704005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:51.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.883+0000 7f5718fae640 1 --2- 172.21.15.120:0/534126131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5569dcc8f370 0x5569dcd07960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:51.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.884+0000 7f5718fae640 1 -- 172.21.15.120:0/534126131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5569dcc8f370 msgr2=0x5569dcd07960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:51.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.884+0000 7f5718fae640 1 --2- 172.21.15.120:0/534126131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5569dcc8f370 0x5569dcd07960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.884+0000 7f5718fae640 1 -- 172.21.15.120:0/534126131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569dcc85b40 msgr2=0x5569dcc92b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:51.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.884+0000 7f5718fae640 1 --2- 172.21.15.120:0/534126131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569dcc85b40 0x5569dcc92b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.884+0000 7f5718fae640 1 -- 172.21.15.120:0/534126131 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569dcd07ea0 con 0x5569dcc845d0 2024-09-19T09:19:51.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.884+0000 7f57137fe640 1 --2- 172.21.15.120:0/534126131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569dcc85b40 0x5569dcc92b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:51.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.884+0000 7f5718fae640 1 --2- 172.21.15.120:0/534126131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569dcc845d0 0x5569dcc92600 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f570c003bf0 tx=0x7f570c0021d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3af568009f295311 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:51.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.884+0000 7f56f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:19:51.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.884+0000 7f56f37fe640 1 -- 172.21.15.120:0/534126131 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570c02c040 con 0x5569dcc845d0 2024-09-19T09:19:51.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.884+0000 7f56f37fe640 1 -- 172.21.15.120:0/534126131 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f570c00f4e0 con 0x5569dcc845d0 2024-09-19T09:19:51.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.884+0000 7f571cdea740 1 -- 172.21.15.120:0/534126131 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f570c004690 con 0x5569dcc845d0 2024-09-19T09:19:51.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.885+0000 7f56f37fe640 1 -- 172.21.15.120:0/534126131 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f570c011dd0 con 0x5569dcc845d0 2024-09-19T09:19:51.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.885+0000 7f571cdea740 10 client.30032 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:51.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.885+0000 7f571cdea740 1 -- 172.21.15.120:0/534126131 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5569dcd09040 con 0x5569dcc845d0 2024-09-19T09:19:51.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.885+0000 7f56f37fe640 1 -- 172.21.15.120:0/534126131 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f570c004690 con 0x5569dcc845d0 2024-09-19T09:19:51.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.885+0000 7f56f37fe640 1 -- 172.21.15.120:0/534126131 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f570c005030 con 0x5569dcc845d0 2024-09-19T09:19:51.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.885+0000 7f571cdea740 10 client.30032 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:51.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.885+0000 7f571cdea740 10 client.30032 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:19:51.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.885+0000 7f571cdea740 20 client.30032 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:51.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.885+0000 7f571cdea740 1 --2- 172.21.15.120:0/534126131 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5569dcd09380 0x5569dcd0b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:51.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.885+0000 7f571cdea740 4 client.30032 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:19:51.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.885+0000 7f571cdea740 1 -- 172.21.15.120:0/534126131 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5569dcbf1d70 con 0x5569dcd09380 2024-09-19T09:19:51.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.885+0000 7f57137fe640 1 --2- 172.21.15.120:0/534126131 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5569dcd09380 0x5569dcd0b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:51.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.886+0000 7f56f37fe640 10 client.30032 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:19:51.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.886+0000 7f57137fe640 1 --2- 172.21.15.120:0/534126131 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5569dcd09380 0x5569dcd0b770 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4b3302995714a09b server_cookie=991eb2e2786c0594 in_seq=0 out_seq=0 2024-09-19T09:19:51.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.886+0000 7f56f37fe640 1 -- 172.21.15.120:0/534126131 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5569dcbf1d70 con 0x5569dcd09380 2024-09-19T09:19:51.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.886+0000 7f56f37fe640 10 client.30032 handle_command_reply: tid=0 2024-09-19T09:19:51.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.886+0000 7f571cdea740 1 client.30032 shutdown 2024-09-19T09:19:51.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.886+0000 7f571cdea740 1 -- 172.21.15.120:0/534126131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569dcc845d0 msgr2=0x5569dcc92600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:51.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.886+0000 7f571cdea740 1 --2- 172.21.15.120:0/534126131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569dcc845d0 0x5569dcc92600 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f570c003bf0 tx=0x7f570c0021d0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.886+0000 7f571cdea740 1 -- 172.21.15.120:0/534126131 shutdown_connections 2024-09-19T09:19:51.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.886+0000 7f571cdea740 1 --2- 172.21.15.120:0/534126131 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5569dcd09380 0x5569dcd0b770 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.886+0000 7f571cdea740 1 --2- 172.21.15.120:0/534126131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5569dcc8f370 0x5569dcd07960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.886+0000 7f571cdea740 1 --2- 172.21.15.120:0/534126131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569dcc85b40 0x5569dcc92b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.886+0000 7f56f37fe640 0 client.30032 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:19:51.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.886+0000 7f571cdea740 1 --2- 172.21.15.120:0/534126131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569dcc845d0 0x5569dcc92600 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.886+0000 7f571cdea740 1 -- 172.21.15.120:0/534126131 >> 172.21.15.120:0/534126131 conn(0x5569dcc4d310 msgr2=0x5569dcc8a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:51.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.887+0000 7f571cdea740 1 -- 172.21.15.120:0/534126131 shutdown_connections 2024-09-19T09:19:51.910 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:19:51.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.887+0000 7f571cdea740 1 -- 172.21.15.120:0/534126131 wait complete. 2024-09-19T09:19:51.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.887+0000 7f571cdea740 20 client.30032 trim_cache size 0 max 16384 2024-09-19T09:19:51.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.887+0000 7f571aa11640 1 -- 172.21.15.120:0/3293610654 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f56fc0a3240 msgr2=0x7f56fc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:51.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.887+0000 7f571aa11640 1 --2- 172.21.15.120:0/3293610654 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f56fc0a3240 0x7f56fc0a5700 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5704098730 tx=0x7f57040d3680 comp rx=0 tx=0).stop 2024-09-19T09:19:51.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.887+0000 7f571aa11640 1 -- 172.21.15.120:0/3293610654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5714100e20 msgr2=0x7f57141970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:51.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.887+0000 7f571aa11640 1 --2- 172.21.15.120:0/3293610654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5714100e20 0x7f57141970e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f570400e9f0 tx=0x7f570400eeb0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.888+0000 7f571aa11640 1 -- 172.21.15.120:0/3293610654 shutdown_connections 2024-09-19T09:19:51.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.888+0000 7f571aa11640 1 --2- 172.21.15.120:0/3293610654 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f56fc0a3240 0x7f56fc0a5700 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.888+0000 7f571aa11640 1 --2- 172.21.15.120:0/3293610654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5714102ef0 0x7f5714197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.888+0000 7f571aa11640 1 --2- 172.21.15.120:0/3293610654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5714102570 0x7f5714197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.888+0000 7f571aa11640 1 --2- 172.21.15.120:0/3293610654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5714100e20 0x7f57141970e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:51.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.888+0000 7f571aa11640 1 -- 172.21.15.120:0/3293610654 >> 172.21.15.120:0/3293610654 conn(0x7f57140fc7d0 msgr2=0x7f57140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:51.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.888+0000 7f571aa11640 1 -- 172.21.15.120:0/3293610654 shutdown_connections 2024-09-19T09:19:51.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:51.888+0000 7f571aa11640 1 -- 172.21.15.120:0/3293610654 wait complete. 2024-09-19T09:19:52.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:52 smithi120 ceph-mon[31150]: pgmap v855: 97 pgs: 97 active+clean; 353 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.8 MiB/s wr, 805 op/s 2024-09-19T09:19:52.363 INFO:tasks.workunit.client.0.smithi120.stdout: 10 88916 16.66 MB/sec execute 107 sec latency 73.862 ms 2024-09-19T09:19:52.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:52 smithi196 ceph-mon[35150]: pgmap v855: 97 pgs: 97 active+clean; 353 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.8 MiB/s wr, 805 op/s 2024-09-19T09:19:52.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:52 smithi153 ceph-mon[35071]: pgmap v855: 97 pgs: 97 active+clean; 353 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.8 MiB/s wr, 805 op/s 2024-09-19T09:19:53.058 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:19:53.365 INFO:tasks.workunit.client.0.smithi120.stdout: 10 89466 16.75 MB/sec execute 108 sec latency 222.093 ms 2024-09-19T09:19:53.370 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:19:53.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.427+0000 7fdedfeba640 1 -- 172.21.15.120:0/3488244639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fded8107880 msgr2=0x7fded8109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:53.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.427+0000 7fdedfeba640 1 --2- 172.21.15.120:0/3488244639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fded8107880 0x7fded8109c90 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fded0009ca0 tx=0x7fded002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.427+0000 7fdedfeba640 1 -- 172.21.15.120:0/3488244639 shutdown_connections 2024-09-19T09:19:53.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.427+0000 7fdedfeba640 1 --2- 172.21.15.120:0/3488244639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fded8107880 0x7fded8109c90 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.427+0000 7fdedfeba640 1 --2- 172.21.15.120:0/3488244639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fded8104f50 0x7fded8107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.427+0000 7fdedfeba640 1 --2- 172.21.15.120:0/3488244639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fded8102620 0x7fded8104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.427+0000 7fdedfeba640 1 -- 172.21.15.120:0/3488244639 >> 172.21.15.120:0/3488244639 conn(0x7fded80fc1e0 msgr2=0x7fded80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.427+0000 7fdedfeba640 1 -- 172.21.15.120:0/3488244639 shutdown_connections 2024-09-19T09:19:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.427+0000 7fdedfeba640 1 -- 172.21.15.120:0/3488244639 wait complete. 2024-09-19T09:19:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.428+0000 7fdedfeba640 1 Processor -- start 2024-09-19T09:19:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.428+0000 7fdedfeba640 1 -- start start 2024-09-19T09:19:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.428+0000 7fdedfeba640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fded8102620 0x7fded8112600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.428+0000 7fdedfeba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fded8104f50 0x7fded8112b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.428+0000 7fdedfeba640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fded8107880 0x7fded8110c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.428+0000 7fdedfeba640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fded810b5d0 con 0x7fded8102620 2024-09-19T09:19:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.428+0000 7fdedfeba640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fded810b450 con 0x7fded8107880 2024-09-19T09:19:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.428+0000 7fdedfeba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fded810b750 con 0x7fded8104f50 2024-09-19T09:19:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.429+0000 7fdedd455640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fded8104f50 0x7fded8112b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.429+0000 7fdeddc56640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fded8102620 0x7fded8112600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.429+0000 7fdedd455640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fded8104f50 0x7fded8112b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:37386/0 (socket says 172.21.15.120:37386) 2024-09-19T09:19:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.429+0000 7fdedd455640 1 -- 172.21.15.120:0/2165670386 learned_addr learned my addr 172.21.15.120:0/2165670386 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.429+0000 7fdede457640 1 --2- 172.21.15.120:0/2165670386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fded8107880 0x7fded8110c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.429+0000 7fdedd455640 1 -- 172.21.15.120:0/2165670386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fded8107880 msgr2=0x7fded8110c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.429+0000 7fdedd455640 1 --2- 172.21.15.120:0/2165670386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fded8107880 0x7fded8110c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.429+0000 7fdedd455640 1 -- 172.21.15.120:0/2165670386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fded8102620 msgr2=0x7fded8112600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.429+0000 7fdedd455640 1 --2- 172.21.15.120:0/2165670386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fded8102620 0x7fded8112600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.429+0000 7fdedd455640 1 -- 172.21.15.120:0/2165670386 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded81112e0 con 0x7fded8104f50 2024-09-19T09:19:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.429+0000 7fdede457640 1 --2- 172.21.15.120:0/2165670386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fded8107880 0x7fded8110c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:19:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.429+0000 7fdedd455640 1 --2- 172.21.15.120:0/2165670386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fded8104f50 0x7fded8112b40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdec800e9f0 tx=0x7fdec800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.429+0000 7fdeceffd640 1 -- 172.21.15.120:0/2165670386 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec800ccc0 con 0x7fded8104f50 2024-09-19T09:19:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.429+0000 7fdedfeba640 1 -- 172.21.15.120:0/2165670386 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fded81115d0 con 0x7fded8104f50 2024-09-19T09:19:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.429+0000 7fdeceffd640 1 -- 172.21.15.120:0/2165670386 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdec8005500 con 0x7fded8104f50 2024-09-19T09:19:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.429+0000 7fdeceffd640 1 -- 172.21.15.120:0/2165670386 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec8010690 con 0x7fded8104f50 2024-09-19T09:19:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.429+0000 7fdedfeba640 1 -- 172.21.15.120:0/2165670386 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fded81af180 con 0x7fded8104f50 2024-09-19T09:19:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.432+0000 7fdeceffd640 1 -- 172.21.15.120:0/2165670386 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fdec80046c0 con 0x7fded8104f50 2024-09-19T09:19:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.432+0000 7fdee2c59740 1 Processor -- start 2024-09-19T09:19:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.433+0000 7fdee2c59740 1 -- start start 2024-09-19T09:19:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.433+0000 7fdee2c59740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5645674ec9a0 0x5645674ecda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.433+0000 7fdee2c59740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645674ed2e0 0x5645674f1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.433+0000 7fdee2c59740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645674f2370 0x5645674f4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.433+0000 7fdee2c59740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5645674502f0 con 0x5645674ec9a0 2024-09-19T09:19:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.433+0000 7fdee2c59740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5645673f8270 con 0x5645674ed2e0 2024-09-19T09:19:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.433+0000 7fdee2c59740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56456744c2b0 con 0x5645674f2370 2024-09-19T09:19:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.433+0000 7fdede457640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5645674ec9a0 0x5645674ecda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.433+0000 7fdede457640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5645674ec9a0 0x5645674ecda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49636/0 (socket says 172.21.15.120:49636) 2024-09-19T09:19:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.433+0000 7fdede457640 1 -- 172.21.15.120:0/845084698 learned_addr learned my addr 172.21.15.120:0/845084698 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.433+0000 7fdeddc56640 1 --2- 172.21.15.120:0/845084698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645674ed2e0 0x5645674f1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.433+0000 7fdedd455640 1 --2- 172.21.15.120:0/845084698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645674f2370 0x5645674f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.433+0000 7fdeceffd640 1 --2- 172.21.15.120:0/2165670386 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdec40a3310 0x7fdec40a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.434+0000 7fdede457640 1 -- 172.21.15.120:0/845084698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645674f2370 msgr2=0x5645674f4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.434+0000 7fdede457640 1 --2- 172.21.15.120:0/845084698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645674f2370 0x5645674f4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.434+0000 7fdede457640 1 -- 172.21.15.120:0/845084698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645674ed2e0 msgr2=0x5645674f1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.434+0000 7fdede457640 1 --2- 172.21.15.120:0/845084698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645674ed2e0 0x5645674f1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.434+0000 7fdede457640 1 -- 172.21.15.120:0/845084698 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645673f7340 con 0x5645674ec9a0 2024-09-19T09:19:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.434+0000 7fdeddc56640 1 --2- 172.21.15.120:0/845084698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645674ed2e0 0x5645674f1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.434+0000 7fdeceffd640 1 -- 172.21.15.120:0/2165670386 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fdec80cfde0 con 0x7fded8104f50 2024-09-19T09:19:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.434+0000 7fdede457640 1 --2- 172.21.15.120:0/2165670386 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdec40a3310 0x7fdec40a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.434+0000 7fdede457640 1 --2- 172.21.15.120:0/845084698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5645674ec9a0 0x5645674ecda0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fded0005e30 tx=0x7fded0009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ec7db67cb42319 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.434+0000 7fdec3fff640 1 -- 172.21.15.120:0/845084698 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0003e40 con 0x5645674ec9a0 2024-09-19T09:19:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.434+0000 7fdec3fff640 1 -- 172.21.15.120:0/845084698 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fded0036040 con 0x5645674ec9a0 2024-09-19T09:19:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.434+0000 7fdec3fff640 1 -- 172.21.15.120:0/845084698 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded002fa30 con 0x5645674ec9a0 2024-09-19T09:19:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.435+0000 7fdee2c59740 1 -- 172.21.15.120:0/845084698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5645674ec9a0 msgr2=0x5645674ecda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.435+0000 7fdede457640 1 --2- 172.21.15.120:0/2165670386 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdec40a3310 0x7fdec40a57d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fded0048000 tx=0x7fded0049000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.435+0000 7fdee2c59740 1 --2- 172.21.15.120:0/845084698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5645674ec9a0 0x5645674ecda0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fded0005e30 tx=0x7fded0009880 comp rx=0 tx=0).stop 2024-09-19T09:19:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.435+0000 7fdee2c59740 1 -- 172.21.15.120:0/845084698 shutdown_connections 2024-09-19T09:19:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.435+0000 7fdee2c59740 1 --2- 172.21.15.120:0/845084698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645674f2370 0x5645674f4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.435+0000 7fdee2c59740 1 --2- 172.21.15.120:0/845084698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645674ed2e0 0x5645674f1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.435+0000 7fdee2c59740 1 --2- 172.21.15.120:0/845084698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5645674ec9a0 0x5645674ecda0 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.435+0000 7fdee2c59740 1 -- 172.21.15.120:0/845084698 >> 172.21.15.120:0/845084698 conn(0x5645674e8010 msgr2=0x5645674e8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.435+0000 7fdee2c59740 1 -- 172.21.15.120:0/845084698 shutdown_connections 2024-09-19T09:19:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.435+0000 7fdee2c59740 1 -- 172.21.15.120:0/845084698 wait complete. 2024-09-19T09:19:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.436+0000 7fdee2c59740 1 -- start start 2024-09-19T09:19:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.436+0000 7fdee2c59740 1 Processor -- start 2024-09-19T09:19:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.436+0000 7fdee2c59740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645674ec9a0 0x5645674f6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.436+0000 7fdee2c59740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645674ed2e0 0x5645674f6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.436+0000 7fdee2c59740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5645674f2370 0x5645674fdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.436+0000 7fdee2c59740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5645674f51c0 con 0x5645674f2370 2024-09-19T09:19:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.436+0000 7fdee2c59740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5645673d9290 con 0x5645674ec9a0 2024-09-19T09:19:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.436+0000 7fdee2c59740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5645674f5340 con 0x5645674ed2e0 2024-09-19T09:19:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.436+0000 7fdee2c59740 10 client.0 _ll_register_callbacks cb 0x56456740a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.436+0000 7fdeddc56640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645674ec9a0 0x5645674f6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.436+0000 7fdedd455640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645674ed2e0 0x5645674f6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.436+0000 7fdede457640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5645674f2370 0x5645674fdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.436+0000 7fdedd455640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645674ed2e0 0x5645674f6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:37418/0 (socket says 172.21.15.120:37418) 2024-09-19T09:19:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.436+0000 7fdeddc56640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645674ec9a0 0x5645674f6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52920/0 (socket says 172.21.15.120:52920) 2024-09-19T09:19:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.436+0000 7fdedd455640 1 -- 172.21.15.120:0/531644766 learned_addr learned my addr 172.21.15.120:0/531644766 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.437+0000 7fdedd455640 1 -- 172.21.15.120:0/531644766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645674ec9a0 msgr2=0x5645674f6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.437+0000 7fdedd455640 1 --2- 172.21.15.120:0/531644766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645674ec9a0 0x5645674f6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.437+0000 7fdedd455640 1 -- 172.21.15.120:0/531644766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5645674f2370 msgr2=0x5645674fdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.437+0000 7fdedd455640 1 --2- 172.21.15.120:0/531644766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5645674f2370 0x5645674fdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.437+0000 7fdedd455640 1 -- 172.21.15.120:0/531644766 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564567163210 con 0x5645674ed2e0 2024-09-19T09:19:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.437+0000 7fdede457640 1 --2- 172.21.15.120:0/531644766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5645674f2370 0x5645674fdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.437+0000 7fdedd455640 1 --2- 172.21.15.120:0/531644766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645674ed2e0 0x5645674f6f90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdec80987d0 tx=0x7fdec809c290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7cfa8cac569235b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.437+0000 7fdec17fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:19:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.437+0000 7fdec17fa640 1 -- 172.21.15.120:0/531644766 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec809a540 con 0x5645674ed2e0 2024-09-19T09:19:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.437+0000 7fdec17fa640 1 -- 172.21.15.120:0/531644766 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fdec809b850 con 0x5645674ed2e0 2024-09-19T09:19:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.437+0000 7fdee2c59740 1 -- 172.21.15.120:0/531644766 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fded0036040 con 0x5645674ed2e0 2024-09-19T09:19:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.437+0000 7fdec17fa640 1 -- 172.21.15.120:0/531644766 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdec80056b0 con 0x5645674ed2e0 2024-09-19T09:19:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.437+0000 7fdee2c59740 10 client.29863 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.438+0000 7fdee2c59740 1 -- 172.21.15.120:0/531644766 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56456722f620 con 0x5645674ed2e0 2024-09-19T09:19:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.438+0000 7fdec17fa640 1 -- 172.21.15.120:0/531644766 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fded0036040 con 0x5645674ed2e0 2024-09-19T09:19:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.438+0000 7fdec17fa640 1 -- 172.21.15.120:0/531644766 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fdec80b5140 con 0x5645674ed2e0 2024-09-19T09:19:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.438+0000 7fdee2c59740 10 client.29863 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.438+0000 7fdee2c59740 10 client.29863 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:19:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.438+0000 7fdee2c59740 20 client.29863 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.438+0000 7fdee2c59740 1 --2- 172.21.15.120:0/531644766 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5645675026c0 0x564567504af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.438+0000 7fdee2c59740 4 client.29863 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:19:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.438+0000 7fdee2c59740 1 -- 172.21.15.120:0/531644766 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56456745da20 con 0x5645675026c0 2024-09-19T09:19:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.438+0000 7fdeddc56640 1 --2- 172.21.15.120:0/531644766 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5645675026c0 0x564567504af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.439+0000 7fdec17fa640 10 client.29863 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.439+0000 7fdeddc56640 1 --2- 172.21.15.120:0/531644766 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5645675026c0 0x564567504af0 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=952ac182ff645d6b server_cookie=15ba70cbaf9d56e9 in_seq=0 out_seq=0 2024-09-19T09:19:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.441+0000 7fdec17fa640 1 -- 172.21.15.120:0/531644766 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56456745da20 con 0x5645675026c0 2024-09-19T09:19:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.441+0000 7fdec17fa640 10 client.29863 handle_command_reply: tid=0 2024-09-19T09:19:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.441+0000 7fdee2c59740 1 client.29863 shutdown 2024-09-19T09:19:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.442+0000 7fdee2c59740 1 -- 172.21.15.120:0/531644766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645674ed2e0 msgr2=0x5645674f6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.442+0000 7fdee2c59740 1 --2- 172.21.15.120:0/531644766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645674ed2e0 0x5645674f6f90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdec80987d0 tx=0x7fdec809c290 comp rx=0 tx=0).stop 2024-09-19T09:19:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.442+0000 7fdee2c59740 1 -- 172.21.15.120:0/531644766 shutdown_connections 2024-09-19T09:19:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.442+0000 7fdee2c59740 1 --2- 172.21.15.120:0/531644766 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5645675026c0 0x564567504af0 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.442+0000 7fdee2c59740 1 --2- 172.21.15.120:0/531644766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5645674f2370 0x5645674fdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.442+0000 7fdec17fa640 0 client.29863 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.442+0000 7fdee2c59740 1 --2- 172.21.15.120:0/531644766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645674ed2e0 0x5645674f6f90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.442+0000 7fdee2c59740 1 --2- 172.21.15.120:0/531644766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645674ec9a0 0x5645674f6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.442+0000 7fdee2c59740 1 -- 172.21.15.120:0/531644766 >> 172.21.15.120:0/531644766 conn(0x5645674e8010 msgr2=0x5645674e9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.442+0000 7fdee2c59740 1 -- 172.21.15.120:0/531644766 shutdown_connections 2024-09-19T09:19:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.442+0000 7fdee2c59740 1 -- 172.21.15.120:0/531644766 wait complete. 2024-09-19T09:19:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.442+0000 7fdee2c59740 20 client.29863 trim_cache size 0 max 16384 2024-09-19T09:19:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.456+0000 7fdee2c59740 1 Processor -- start 2024-09-19T09:19:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.456+0000 7fdee2c59740 1 -- start start 2024-09-19T09:19:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.456+0000 7fdee2c59740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645674ec9a0 0x5645674e8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.457+0000 7fdee2c59740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645674ed2e0 0x5645674fdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.457+0000 7fdee2c59740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5645674f2370 0x5645674e9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.457+0000 7fdee2c59740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fded81b7e30 con 0x5645674f2370 2024-09-19T09:19:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.457+0000 7fdee2c59740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fded81b9970 con 0x5645674ed2e0 2024-09-19T09:19:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.457+0000 7fdee2c59740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fded81b8500 con 0x5645674ec9a0 2024-09-19T09:19:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.457+0000 7fdede457640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5645674f2370 0x5645674e9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.457+0000 7fdeddc56640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645674ec9a0 0x5645674e8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.457+0000 7fdede457640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5645674f2370 0x5645674e9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49656/0 (socket says 172.21.15.120:49656) 2024-09-19T09:19:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.457+0000 7fdeddc56640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645674ec9a0 0x5645674e8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:37428/0 (socket says 172.21.15.120:37428) 2024-09-19T09:19:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.457+0000 7fdede457640 1 -- 172.21.15.120:0/419881166 learned_addr learned my addr 172.21.15.120:0/419881166 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.457+0000 7fdedd455640 1 --2- 172.21.15.120:0/419881166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645674ed2e0 0x5645674fdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.457+0000 7fdede457640 1 -- 172.21.15.120:0/419881166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645674ec9a0 msgr2=0x5645674e8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.457+0000 7fdede457640 1 --2- 172.21.15.120:0/419881166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645674ec9a0 0x5645674e8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.457+0000 7fdede457640 1 -- 172.21.15.120:0/419881166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645674ed2e0 msgr2=0x5645674fdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.457+0000 7fdede457640 1 --2- 172.21.15.120:0/419881166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645674ed2e0 0x5645674fdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.457+0000 7fdede457640 1 -- 172.21.15.120:0/419881166 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645674f6560 con 0x5645674f2370 2024-09-19T09:19:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.457+0000 7fdeddc56640 1 --2- 172.21.15.120:0/419881166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645674ec9a0 0x5645674e8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.457+0000 7fdede457640 1 --2- 172.21.15.120:0/419881166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5645674f2370 0x5645674e9400 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fded0031ae0 tx=0x7fded0052000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a992bf8a1c71b705 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.457+0000 7fdec37fe640 1 -- 172.21.15.120:0/419881166 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0062070 con 0x5645674f2370 2024-09-19T09:19:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.457+0000 7fdec37fe640 1 -- 172.21.15.120:0/419881166 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fded002f310 con 0x5645674f2370 2024-09-19T09:19:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.457+0000 7fdec37fe640 1 -- 172.21.15.120:0/419881166 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0004780 con 0x5645674f2370 2024-09-19T09:19:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.458+0000 7fdee2c59740 1 -- 172.21.15.120:0/419881166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5645674f2370 msgr2=0x5645674e9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.458+0000 7fdee2c59740 1 --2- 172.21.15.120:0/419881166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5645674f2370 0x5645674e9400 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fded0031ae0 tx=0x7fded0052000 comp rx=0 tx=0).stop 2024-09-19T09:19:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.458+0000 7fdee2c59740 1 -- 172.21.15.120:0/419881166 shutdown_connections 2024-09-19T09:19:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.458+0000 7fdee2c59740 1 --2- 172.21.15.120:0/419881166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5645674f2370 0x5645674e9400 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.458+0000 7fdee2c59740 1 --2- 172.21.15.120:0/419881166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645674ed2e0 0x5645674fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.458+0000 7fdee2c59740 1 --2- 172.21.15.120:0/419881166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645674ec9a0 0x5645674e8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.458+0000 7fdee2c59740 1 -- 172.21.15.120:0/419881166 >> 172.21.15.120:0/419881166 conn(0x5645674e8010 msgr2=0x5645674f6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.459+0000 7fdee2c59740 1 -- 172.21.15.120:0/419881166 shutdown_connections 2024-09-19T09:19:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.459+0000 7fdee2c59740 1 -- 172.21.15.120:0/419881166 wait complete. 2024-09-19T09:19:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.459+0000 7fdee2c59740 1 -- start start 2024-09-19T09:19:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.459+0000 7fdee2c59740 1 Processor -- start 2024-09-19T09:19:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.459+0000 7fdee2c59740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645674ec9a0 0x5645674fb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.460+0000 7fdee2c59740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645674ed2e0 0x5645674fbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.460+0000 7fdee2c59740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5645674f2370 0x5645674fcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.460+0000 7fdee2c59740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fded81b92a0 con 0x5645674f2370 2024-09-19T09:19:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.460+0000 7fdee2c59740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fded81b8bd0 con 0x5645674ed2e0 2024-09-19T09:19:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.460+0000 7fdee2c59740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fded81b5df0 con 0x5645674ec9a0 2024-09-19T09:19:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.460+0000 7fdee2c59740 10 client.0 _ll_register_callbacks cb 0x7fdec8098610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.460+0000 7fdeddc56640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645674ec9a0 0x5645674fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.460+0000 7fdeddc56640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645674ec9a0 0x5645674fb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:37432/0 (socket says 172.21.15.120:37432) 2024-09-19T09:19:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.460+0000 7fdede457640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5645674f2370 0x5645674fcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.460+0000 7fdeddc56640 1 -- 172.21.15.120:0/2549888257 learned_addr learned my addr 172.21.15.120:0/2549888257 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.460+0000 7fdedd455640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645674ed2e0 0x5645674fbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.460+0000 7fdeddc56640 1 -- 172.21.15.120:0/2549888257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645674ed2e0 msgr2=0x5645674fbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.460+0000 7fdeddc56640 1 --2- 172.21.15.120:0/2549888257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645674ed2e0 0x5645674fbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.460+0000 7fdeddc56640 1 -- 172.21.15.120:0/2549888257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5645674f2370 msgr2=0x5645674fcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.460+0000 7fdeddc56640 1 --2- 172.21.15.120:0/2549888257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5645674f2370 0x5645674fcc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.460+0000 7fdeddc56640 1 -- 172.21.15.120:0/2549888257 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645674fd280 con 0x5645674ec9a0 2024-09-19T09:19:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.460+0000 7fdede457640 1 --2- 172.21.15.120:0/2549888257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5645674f2370 0x5645674fcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.460+0000 7fdeddc56640 1 --2- 172.21.15.120:0/2549888257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645674ec9a0 0x5645674fb720 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x5645674e8e90 tx=0x7fded40204d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eab7f7f27d12867f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.460+0000 7fdec17fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:19:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.460+0000 7fdec17fa640 1 -- 172.21.15.120:0/2549888257 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded402b060 con 0x5645674ec9a0 2024-09-19T09:19:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.460+0000 7fdee2c59740 1 -- 172.21.15.120:0/2549888257 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fded002f310 con 0x5645674ec9a0 2024-09-19T09:19:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.460+0000 7fdec17fa640 1 -- 172.21.15.120:0/2549888257 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fded4010840 con 0x5645674ec9a0 2024-09-19T09:19:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.461+0000 7fdec17fa640 1 -- 172.21.15.120:0/2549888257 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fded4010a40 con 0x5645674ec9a0 2024-09-19T09:19:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.461+0000 7fdee2c59740 10 client.29869 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.461+0000 7fdee2c59740 1 -- 172.21.15.120:0/2549888257 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56456756d4a0 con 0x5645674ec9a0 2024-09-19T09:19:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.461+0000 7fdec17fa640 1 -- 172.21.15.120:0/2549888257 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fded002f310 con 0x5645674ec9a0 2024-09-19T09:19:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.461+0000 7fdec17fa640 1 -- 172.21.15.120:0/2549888257 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fded4030070 con 0x5645674ec9a0 2024-09-19T09:19:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.461+0000 7fdee2c59740 10 client.29869 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.461+0000 7fdee2c59740 10 client.29869 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:19:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.461+0000 7fdee2c59740 20 client.29869 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.461+0000 7fdee2c59740 1 --2- 172.21.15.120:0/2549888257 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5645675026c0 0x56456756fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.461+0000 7fdee2c59740 4 client.29869 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:19:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.461+0000 7fdee2c59740 1 -- 172.21.15.120:0/2549888257 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564567457d70 con 0x5645675026c0 2024-09-19T09:19:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.461+0000 7fdedd455640 1 --2- 172.21.15.120:0/2549888257 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5645675026c0 0x56456756fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.461+0000 7fdec17fa640 10 client.29869 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.462+0000 7fdedd455640 1 --2- 172.21.15.120:0/2549888257 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5645675026c0 0x56456756fea0 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9858ef230c1670ee server_cookie=116d85d615258563 in_seq=0 out_seq=0 2024-09-19T09:19:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.911+0000 7fdec17fa640 1 -- 172.21.15.120:0/2549888257 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564567457d70 con 0x5645675026c0 2024-09-19T09:19:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.911+0000 7fdec17fa640 10 client.29869 handle_command_reply: tid=0 2024-09-19T09:19:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.912+0000 7fdee2c59740 1 client.29869 shutdown 2024-09-19T09:19:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.912+0000 7fdee2c59740 1 -- 172.21.15.120:0/2549888257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645674ec9a0 msgr2=0x5645674fb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.912+0000 7fdee2c59740 1 --2- 172.21.15.120:0/2549888257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645674ec9a0 0x5645674fb720 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x5645674e8e90 tx=0x7fded40204d0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.912+0000 7fdee2c59740 1 -- 172.21.15.120:0/2549888257 shutdown_connections 2024-09-19T09:19:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.912+0000 7fdee2c59740 1 --2- 172.21.15.120:0/2549888257 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5645675026c0 0x56456756fea0 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.912+0000 7fdec17fa640 0 client.29869 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.912+0000 7fdee2c59740 1 --2- 172.21.15.120:0/2549888257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5645674f2370 0x5645674fcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.912+0000 7fdee2c59740 1 --2- 172.21.15.120:0/2549888257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645674ed2e0 0x5645674fbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.912+0000 7fdee2c59740 1 --2- 172.21.15.120:0/2549888257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645674ec9a0 0x5645674fb720 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.912+0000 7fdee2c59740 1 -- 172.21.15.120:0/2549888257 >> 172.21.15.120:0/2549888257 conn(0x5645674e8010 msgr2=0x5645674e69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.913+0000 7fdee2c59740 1 -- 172.21.15.120:0/2549888257 shutdown_connections 2024-09-19T09:19:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.913+0000 7fdee2c59740 1 -- 172.21.15.120:0/2549888257 wait complete. 2024-09-19T09:19:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.913+0000 7fdee2c59740 20 client.29869 trim_cache size 0 max 16384 2024-09-19T09:19:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:19:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:19:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:19:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:19:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.914+0000 7fdedfeba640 1 -- 172.21.15.120:0/2165670386 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdec40a3310 msgr2=0x7fdec40a57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.914+0000 7fdedfeba640 1 --2- 172.21.15.120:0/2165670386 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdec40a3310 0x7fdec40a57d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fded0048000 tx=0x7fded0049000 comp rx=0 tx=0).stop 2024-09-19T09:19:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.914+0000 7fdedfeba640 1 -- 172.21.15.120:0/2165670386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fded8104f50 msgr2=0x7fded8112b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.914+0000 7fdedfeba640 1 --2- 172.21.15.120:0/2165670386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fded8104f50 0x7fded8112b40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdec800e9f0 tx=0x7fdec800eeb0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.914+0000 7fdedfeba640 1 -- 172.21.15.120:0/2165670386 shutdown_connections 2024-09-19T09:19:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.914+0000 7fdedfeba640 1 --2- 172.21.15.120:0/2165670386 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdec40a3310 0x7fdec40a57d0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.914+0000 7fdedfeba640 1 --2- 172.21.15.120:0/2165670386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fded8107880 0x7fded8110c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.914+0000 7fdedfeba640 1 --2- 172.21.15.120:0/2165670386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fded8104f50 0x7fded8112b40 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.914+0000 7fdedfeba640 1 --2- 172.21.15.120:0/2165670386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fded8102620 0x7fded8112600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.914+0000 7fdedfeba640 1 -- 172.21.15.120:0/2165670386 >> 172.21.15.120:0/2165670386 conn(0x7fded80fc1e0 msgr2=0x7fded8102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.914+0000 7fdedfeba640 1 -- 172.21.15.120:0/2165670386 shutdown_connections 2024-09-19T09:19:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:19:53.914+0000 7fdedfeba640 1 -- 172.21.15.120:0/2165670386 wait complete. 2024-09-19T09:19:53.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.954+0000 7fceb497d640 1 -- 172.21.15.120:0/478947445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fceb0102ef0 msgr2=0x7fceb010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:53.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.954+0000 7fceb497d640 1 --2- 172.21.15.120:0/478947445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fceb0102ef0 0x7fceb010da40 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fcea8009ca0 tx=0x7fcea802a950 comp rx=0 tx=0).stop 2024-09-19T09:19:53.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.954+0000 7fceb497d640 1 -- 172.21.15.120:0/478947445 shutdown_connections 2024-09-19T09:19:53.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.954+0000 7fceb497d640 1 --2- 172.21.15.120:0/478947445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fceb0102ef0 0x7fceb010da40 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.954+0000 7fceb497d640 1 --2- 172.21.15.120:0/478947445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fceb0102570 0x7fceb01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.954+0000 7fceb497d640 1 --2- 172.21.15.120:0/478947445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fceb0100e20 0x7fceb0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.954+0000 7fceb497d640 1 -- 172.21.15.120:0/478947445 >> 172.21.15.120:0/478947445 conn(0x7fceb00fc7d0 msgr2=0x7fceb00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:53.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.954+0000 7fceb497d640 1 -- 172.21.15.120:0/478947445 shutdown_connections 2024-09-19T09:19:53.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.954+0000 7fceb497d640 1 -- 172.21.15.120:0/478947445 wait complete. 2024-09-19T09:19:53.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.955+0000 7fceb497d640 1 Processor -- start 2024-09-19T09:19:53.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.955+0000 7fceb497d640 1 -- start start 2024-09-19T09:19:53.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.955+0000 7fceb497d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fceb0100e20 0x7fceb01993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.955+0000 7fceb497d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fceb0102570 0x7fceb01998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.955+0000 7fceb497d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fceb0102ef0 0x7fceb0199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.955+0000 7fceb497d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fceb0110f80 con 0x7fceb0100e20 2024-09-19T09:19:53.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.955+0000 7fceb497d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fceb0110e00 con 0x7fceb0102ef0 2024-09-19T09:19:53.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.955+0000 7fceb497d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fceb0111100 con 0x7fceb0102570 2024-09-19T09:19:53.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.955+0000 7fceae59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fceb0100e20 0x7fceb01993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.955+0000 7fceadd9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fceb0102570 0x7fceb01998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.955+0000 7fceae59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fceb0100e20 0x7fceb01993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49674/0 (socket says 172.21.15.120:49674) 2024-09-19T09:19:53.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.956+0000 7fceae59c640 1 -- 172.21.15.120:0/2061758302 learned_addr learned my addr 172.21.15.120:0/2061758302 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:53.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.956+0000 7fceaed9d640 1 --2- 172.21.15.120:0/2061758302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fceb0102ef0 0x7fceb0199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.956+0000 7fceae59c640 1 -- 172.21.15.120:0/2061758302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fceb0102570 msgr2=0x7fceb01998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:53.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.956+0000 7fceae59c640 1 --2- 172.21.15.120:0/2061758302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fceb0102570 0x7fceb01998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.956+0000 7fceae59c640 1 -- 172.21.15.120:0/2061758302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fceb0102ef0 msgr2=0x7fceb0199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:53.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.956+0000 7fceae59c640 1 --2- 172.21.15.120:0/2061758302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fceb0102ef0 0x7fceb0199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.956+0000 7fceae59c640 1 -- 172.21.15.120:0/2061758302 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb01a47b0 con 0x7fceb0100e20 2024-09-19T09:19:53.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.956+0000 7fceadd9b640 1 --2- 172.21.15.120:0/2061758302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fceb0102570 0x7fceb01998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:53.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.956+0000 7fceae59c640 1 --2- 172.21.15.120:0/2061758302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fceb0100e20 0x7fceb01993b0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fcea000e9f0 tx=0x7fcea000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:53.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.956+0000 7fce9f7fe640 1 -- 172.21.15.120:0/2061758302 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea000cd80 con 0x7fceb0100e20 2024-09-19T09:19:53.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.956+0000 7fceb497d640 1 -- 172.21.15.120:0/2061758302 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb01a4aa0 con 0x7fceb0100e20 2024-09-19T09:19:53.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.956+0000 7fce9f7fe640 1 -- 172.21.15.120:0/2061758302 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcea0010910 con 0x7fceb0100e20 2024-09-19T09:19:53.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.956+0000 7fce9f7fe640 1 -- 172.21.15.120:0/2061758302 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea0018990 con 0x7fceb0100e20 2024-09-19T09:19:53.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.956+0000 7fceb497d640 1 -- 172.21.15.120:0/2061758302 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb01a5030 con 0x7fceb0100e20 2024-09-19T09:19:53.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.959+0000 7fce9f7fe640 1 -- 172.21.15.120:0/2061758302 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fcea0010ac0 con 0x7fceb0100e20 2024-09-19T09:19:53.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.959+0000 7fceb6d56740 1 Processor -- start 2024-09-19T09:19:53.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.960+0000 7fceb6d56740 1 -- start start 2024-09-19T09:19:53.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.960+0000 7fceb6d56740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f693eb0b40 0x55f693eaf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.960+0000 7fceb6d56740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f693eaf5d0 0x55f693eafa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.960+0000 7fceb6d56740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f693eba370 0x55f693ebc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.960+0000 7fceb6d56740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f693e152f0 con 0x55f693eaf5d0 2024-09-19T09:19:53.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.960+0000 7fceb6d56740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f693dbccf0 con 0x55f693eba370 2024-09-19T09:19:53.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.960+0000 7fceb6d56740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f693e112b0 con 0x55f693eb0b40 2024-09-19T09:19:53.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.960+0000 7fceaed9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f693eb0b40 0x55f693eaf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.960+0000 7fceaed9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f693eb0b40 0x55f693eaf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:37456/0 (socket says 172.21.15.120:37456) 2024-09-19T09:19:53.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.960+0000 7fceaed9d640 1 -- 172.21.15.120:0/3651540675 learned_addr learned my addr 172.21.15.120:0/3651540675 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:53.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.960+0000 7fceae59c640 1 --2- 172.21.15.120:0/3651540675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f693eaf5d0 0x55f693eafa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.961+0000 7fceadd9b640 1 --2- 172.21.15.120:0/3651540675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f693eba370 0x55f693ebc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.961+0000 7fce9f7fe640 1 --2- 172.21.15.120:0/2061758302 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fce980a32c0 0x7fce980a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.961+0000 7fce9f7fe640 1 -- 172.21.15.120:0/2061758302 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fcea00cfb80 con 0x7fceb0100e20 2024-09-19T09:19:53.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.961+0000 7fceae59c640 1 -- 172.21.15.120:0/3651540675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f693eb0b40 msgr2=0x55f693eaf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:53.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.961+0000 7fceae59c640 1 --2- 172.21.15.120:0/3651540675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f693eb0b40 0x55f693eaf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.961+0000 7fceae59c640 1 -- 172.21.15.120:0/3651540675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f693eba370 msgr2=0x55f693ebc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:53.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.961+0000 7fceae59c640 1 --2- 172.21.15.120:0/3651540675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f693eba370 0x55f693ebc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.961+0000 7fceae59c640 1 -- 172.21.15.120:0/3651540675 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f693da13a0 con 0x55f693eaf5d0 2024-09-19T09:19:53.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.961+0000 7fceaed9d640 1 --2- 172.21.15.120:0/3651540675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f693eb0b40 0x55f693eaf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:53.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.961+0000 7fceaed9d640 1 --2- 172.21.15.120:0/2061758302 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fce980a32c0 0x7fce980a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.962+0000 7fceae59c640 1 --2- 172.21.15.120:0/3651540675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f693eaf5d0 0x55f693eafa10 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fcea00cd620 tx=0x7fcea0018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d82e0c680daeccd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:53.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.962+0000 7fceaed9d640 1 --2- 172.21.15.120:0/2061758302 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fce980a32c0 0x7fce980a5780 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fcea8006010 tx=0x7fcea803c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:53.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.963+0000 7fce9cff9640 1 -- 172.21.15.120:0/3651540675 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea009c250 con 0x55f693eaf5d0 2024-09-19T09:19:53.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.963+0000 7fce9cff9640 1 -- 172.21.15.120:0/3651540675 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcea00a0550 con 0x55f693eaf5d0 2024-09-19T09:19:53.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.963+0000 7fce9cff9640 1 -- 172.21.15.120:0/3651540675 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea00cdb40 con 0x55f693eaf5d0 2024-09-19T09:19:53.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.964+0000 7fceb6d56740 1 -- 172.21.15.120:0/3651540675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f693eaf5d0 msgr2=0x55f693eafa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:53.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.964+0000 7fceb6d56740 1 --2- 172.21.15.120:0/3651540675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f693eaf5d0 0x55f693eafa10 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fcea00cd620 tx=0x7fcea0018420 comp rx=0 tx=0).stop 2024-09-19T09:19:53.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.964+0000 7fceb6d56740 1 -- 172.21.15.120:0/3651540675 shutdown_connections 2024-09-19T09:19:53.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.964+0000 7fceb6d56740 1 --2- 172.21.15.120:0/3651540675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f693eba370 0x55f693ebc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.964+0000 7fceb6d56740 1 --2- 172.21.15.120:0/3651540675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f693eaf5d0 0x55f693eafa10 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.964+0000 7fceb6d56740 1 --2- 172.21.15.120:0/3651540675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f693eb0b40 0x55f693eaf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.964+0000 7fceb6d56740 1 -- 172.21.15.120:0/3651540675 >> 172.21.15.120:0/3651540675 conn(0x55f693e78310 msgr2=0x55f693eabc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:53.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.964+0000 7fceb6d56740 1 -- 172.21.15.120:0/3651540675 shutdown_connections 2024-09-19T09:19:53.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.964+0000 7fceb6d56740 1 -- 172.21.15.120:0/3651540675 wait complete. 2024-09-19T09:19:53.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.965+0000 7fceb6d56740 1 -- start start 2024-09-19T09:19:53.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.965+0000 7fceb6d56740 1 Processor -- start 2024-09-19T09:19:53.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.965+0000 7fceb6d56740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f693eaf5d0 0x55f693eb9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.965+0000 7fceb6d56740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f693eb0b40 0x55f693eb7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.965+0000 7fceadd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f693eb0b40 0x55f693eb7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.965+0000 7fceb6d56740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f693eba370 0x55f693eb77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.965+0000 7fceb6d56740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f693ebcfb0 con 0x55f693eb0b40 2024-09-19T09:19:53.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.965+0000 7fceb6d56740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f693dd0060 con 0x55f693eaf5d0 2024-09-19T09:19:53.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.965+0000 7fceb6d56740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f693ebd130 con 0x55f693eba370 2024-09-19T09:19:53.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.965+0000 7fceadd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f693eb0b40 0x55f693eb7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49694/0 (socket says 172.21.15.120:49694) 2024-09-19T09:19:53.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.965+0000 7fceadd9b640 1 -- 172.21.15.120:0/2093427511 learned_addr learned my addr 172.21.15.120:0/2093427511 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:53.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.965+0000 7fceb6d56740 10 client.0 _ll_register_callbacks cb 0x55f693b238c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:53.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.965+0000 7fceae59c640 1 --2- 172.21.15.120:0/2093427511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f693eaf5d0 0x55f693eb9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.965+0000 7fceaed9d640 1 --2- 172.21.15.120:0/2093427511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f693eba370 0x55f693eb77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.965+0000 7fceadd9b640 1 -- 172.21.15.120:0/2093427511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f693eba370 msgr2=0x55f693eb77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:53.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.965+0000 7fceadd9b640 1 --2- 172.21.15.120:0/2093427511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f693eba370 0x55f693eb77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.965+0000 7fceadd9b640 1 -- 172.21.15.120:0/2093427511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f693eaf5d0 msgr2=0x55f693eb9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:53.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.965+0000 7fceadd9b640 1 --2- 172.21.15.120:0/2093427511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f693eaf5d0 0x55f693eb9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.965+0000 7fceadd9b640 1 -- 172.21.15.120:0/2093427511 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f693b863c0 con 0x55f693eb0b40 2024-09-19T09:19:53.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.966+0000 7fceaed9d640 1 --2- 172.21.15.120:0/2093427511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f693eba370 0x55f693eb77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:19:53.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.966+0000 7fceadd9b640 1 --2- 172.21.15.120:0/2093427511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f693eb0b40 0x55f693eb7060 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fcea40129f0 tx=0x7fcea4012eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf393b7ef034c44 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:53.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.966+0000 7fce95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:19:53.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.966+0000 7fce95ffb640 1 -- 172.21.15.120:0/2093427511 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea400fa40 con 0x55f693eb0b40 2024-09-19T09:19:53.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.966+0000 7fce95ffb640 1 -- 172.21.15.120:0/2093427511 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fcea400fbe0 con 0x55f693eb0b40 2024-09-19T09:19:53.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.966+0000 7fceb6d56740 1 -- 172.21.15.120:0/2093427511 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcea00a0550 con 0x55f693eb0b40 2024-09-19T09:19:53.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.966+0000 7fce95ffb640 1 -- 172.21.15.120:0/2093427511 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcea400fe20 con 0x55f693eb0b40 2024-09-19T09:19:53.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.966+0000 7fceb6d56740 10 client.21201 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:53.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.966+0000 7fceb6d56740 1 -- 172.21.15.120:0/2093427511 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f693ec8b10 con 0x55f693eb0b40 2024-09-19T09:19:53.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.966+0000 7fce95ffb640 1 -- 172.21.15.120:0/2093427511 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcea00a0550 con 0x55f693eb0b40 2024-09-19T09:19:53.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.966+0000 7fce95ffb640 1 -- 172.21.15.120:0/2093427511 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fcea4010430 con 0x55f693eb0b40 2024-09-19T09:19:53.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.966+0000 7fceb6d56740 10 client.21201 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:53.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.966+0000 7fceb6d56740 10 client.21201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:19:53.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.966+0000 7fceb6d56740 20 client.21201 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:53.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.966+0000 7fceb6d56740 1 --2- 172.21.15.120:0/2093427511 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f693eca250 0x55f693ecc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.966+0000 7fceb6d56740 4 client.21201 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:19:53.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.966+0000 7fceb6d56740 1 -- 172.21.15.120:0/2093427511 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f693e22860 con 0x55f693eca250 2024-09-19T09:19:53.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.967+0000 7fceae59c640 1 --2- 172.21.15.120:0/2093427511 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f693eca250 0x55f693ecc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.967+0000 7fceae59c640 1 --2- 172.21.15.120:0/2093427511 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f693eca250 0x55f693ecc640 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61cdc1e88fb5b346 server_cookie=54337516abcb776 in_seq=0 out_seq=0 2024-09-19T09:19:53.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.967+0000 7fce95ffb640 10 client.21201 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:53.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.971+0000 7fce95ffb640 1 -- 172.21.15.120:0/2093427511 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f693e22860 con 0x55f693eca250 2024-09-19T09:19:53.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.971+0000 7fce95ffb640 10 client.21201 handle_command_reply: tid=0 2024-09-19T09:19:53.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.971+0000 7fceb6d56740 1 client.21201 shutdown 2024-09-19T09:19:53.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.971+0000 7fceb6d56740 1 -- 172.21.15.120:0/2093427511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f693eb0b40 msgr2=0x55f693eb7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:53.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.971+0000 7fceb6d56740 1 --2- 172.21.15.120:0/2093427511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f693eb0b40 0x55f693eb7060 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fcea40129f0 tx=0x7fcea4012eb0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.971+0000 7fceb6d56740 1 -- 172.21.15.120:0/2093427511 shutdown_connections 2024-09-19T09:19:53.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.971+0000 7fceb6d56740 1 --2- 172.21.15.120:0/2093427511 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f693eca250 0x55f693ecc640 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.971+0000 7fce95ffb640 0 client.21201 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:53.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.971+0000 7fceb6d56740 1 --2- 172.21.15.120:0/2093427511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f693eba370 0x55f693eb77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.971+0000 7fceb6d56740 1 --2- 172.21.15.120:0/2093427511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f693eb0b40 0x55f693eb7060 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.971+0000 7fceb6d56740 1 --2- 172.21.15.120:0/2093427511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f693eaf5d0 0x55f693eb9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.971+0000 7fceb6d56740 1 -- 172.21.15.120:0/2093427511 >> 172.21.15.120:0/2093427511 conn(0x55f693e78310 msgr2=0x55f693eb1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:53.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.972+0000 7fceb6d56740 1 -- 172.21.15.120:0/2093427511 shutdown_connections 2024-09-19T09:19:53.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.972+0000 7fceb6d56740 1 -- 172.21.15.120:0/2093427511 wait complete. 2024-09-19T09:19:53.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.972+0000 7fceb6d56740 20 client.21201 trim_cache size 0 max 16384 2024-09-19T09:19:53.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.986+0000 7fceb6d56740 1 Processor -- start 2024-09-19T09:19:53.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.986+0000 7fceb6d56740 1 -- start start 2024-09-19T09:19:53.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.986+0000 7fceb6d56740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f693eaf5d0 0x55f693eb9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.986+0000 7fceb6d56740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f693eb0b40 0x55f693eab700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.986+0000 7fceb6d56740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f693eba370 0x55f693eabc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.986+0000 7fceb6d56740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fceb01ad600 con 0x55f693eaf5d0 2024-09-19T09:19:53.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.986+0000 7fceb6d56740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fceb01af140 con 0x55f693eb0b40 2024-09-19T09:19:53.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.986+0000 7fceb6d56740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fceb01adcd0 con 0x55f693eba370 2024-09-19T09:19:53.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.986+0000 7fceae59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f693eaf5d0 0x55f693eb9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.986+0000 7fceae59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f693eaf5d0 0x55f693eb9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49696/0 (socket says 172.21.15.120:49696) 2024-09-19T09:19:53.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.986+0000 7fceae59c640 1 -- 172.21.15.120:0/2974368468 learned_addr learned my addr 172.21.15.120:0/2974368468 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:53.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.987+0000 7fceadd9b640 1 --2- 172.21.15.120:0/2974368468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f693eb0b40 0x55f693eab700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.987+0000 7fceaed9d640 1 --2- 172.21.15.120:0/2974368468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f693eba370 0x55f693eabc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.987+0000 7fceae59c640 1 -- 172.21.15.120:0/2974368468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f693eba370 msgr2=0x55f693eabc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:53.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.987+0000 7fceae59c640 1 --2- 172.21.15.120:0/2974368468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f693eba370 0x55f693eabc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.987+0000 7fceae59c640 1 -- 172.21.15.120:0/2974368468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f693eb0b40 msgr2=0x55f693eab700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:53.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.987+0000 7fceae59c640 1 --2- 172.21.15.120:0/2974368468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f693eb0b40 0x55f693eab700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.987+0000 7fceae59c640 1 -- 172.21.15.120:0/2974368468 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f693eac180 con 0x55f693eaf5d0 2024-09-19T09:19:53.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.987+0000 7fceae59c640 1 --2- 172.21.15.120:0/2974368468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f693eaf5d0 0x55f693eb9db0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fcea0098c20 tx=0x7fcea0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e19cdc3c7ef3f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:53.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.987+0000 7fce97fff640 1 -- 172.21.15.120:0/2974368468 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea00b5070 con 0x55f693eaf5d0 2024-09-19T09:19:53.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.987+0000 7fce97fff640 1 -- 172.21.15.120:0/2974368468 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcea009a610 con 0x55f693eaf5d0 2024-09-19T09:19:53.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.987+0000 7fce97fff640 1 -- 172.21.15.120:0/2974368468 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea00d3170 con 0x55f693eaf5d0 2024-09-19T09:19:53.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.988+0000 7fceb6d56740 1 -- 172.21.15.120:0/2974368468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f693eaf5d0 msgr2=0x55f693eb9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:53.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.988+0000 7fceb6d56740 1 --2- 172.21.15.120:0/2974368468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f693eaf5d0 0x55f693eb9db0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fcea0098c20 tx=0x7fcea0000bc0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.988+0000 7fceb6d56740 1 -- 172.21.15.120:0/2974368468 shutdown_connections 2024-09-19T09:19:53.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.988+0000 7fceb6d56740 1 --2- 172.21.15.120:0/2974368468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f693eba370 0x55f693eabc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.988+0000 7fceb6d56740 1 --2- 172.21.15.120:0/2974368468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f693eb0b40 0x55f693eab700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.988+0000 7fceb6d56740 1 --2- 172.21.15.120:0/2974368468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f693eaf5d0 0x55f693eb9db0 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.988+0000 7fceb6d56740 1 -- 172.21.15.120:0/2974368468 >> 172.21.15.120:0/2974368468 conn(0x55f693e78310 msgr2=0x55f693ea6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:53.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.988+0000 7fceb6d56740 1 -- 172.21.15.120:0/2974368468 shutdown_connections 2024-09-19T09:19:53.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.988+0000 7fceb6d56740 1 -- 172.21.15.120:0/2974368468 wait complete. 2024-09-19T09:19:53.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.988+0000 7fceb6d56740 1 -- start start 2024-09-19T09:19:53.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.989+0000 7fceb6d56740 1 Processor -- start 2024-09-19T09:19:53.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.989+0000 7fceb6d56740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f693eaf5d0 0x55f693ebe240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.989+0000 7fceb6d56740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f693eb0b40 0x55f693ebe780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.989+0000 7fceb6d56740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f693eba370 0x55f693ebecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:53.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.989+0000 7fceb6d56740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fceb01aea70 con 0x55f693eb0b40 2024-09-19T09:19:53.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.989+0000 7fceb6d56740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fceb01ae3a0 con 0x55f693eba370 2024-09-19T09:19:53.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.989+0000 7fceb6d56740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fceb01ab5c0 con 0x55f693eaf5d0 2024-09-19T09:19:53.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.989+0000 7fceb6d56740 10 client.0 _ll_register_callbacks cb 0x7fcea4009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:53.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.989+0000 7fceae59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f693eaf5d0 0x55f693ebe240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.989+0000 7fceaed9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f693eba370 0x55f693ebecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.989+0000 7fceadd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f693eb0b40 0x55f693ebe780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:53.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.989+0000 7fceaed9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f693eba370 0x55f693ebecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:53012/0 (socket says 172.21.15.120:53012) 2024-09-19T09:19:53.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.989+0000 7fceae59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f693eaf5d0 0x55f693ebe240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:37486/0 (socket says 172.21.15.120:37486) 2024-09-19T09:19:53.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.989+0000 7fceaed9d640 1 -- 172.21.15.120:0/2696826176 learned_addr learned my addr 172.21.15.120:0/2696826176 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:53.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.989+0000 7fceadd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f693eb0b40 0x55f693ebe780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49698/0 (socket says 172.21.15.120:49698) 2024-09-19T09:19:53.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.989+0000 7fceadd9b640 1 -- 172.21.15.120:0/2696826176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f693eaf5d0 msgr2=0x55f693ebe240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:53.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.989+0000 7fceadd9b640 1 --2- 172.21.15.120:0/2696826176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f693eaf5d0 0x55f693ebe240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.989+0000 7fceadd9b640 1 -- 172.21.15.120:0/2696826176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f693eba370 msgr2=0x55f693ebecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:53.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.989+0000 7fceadd9b640 1 --2- 172.21.15.120:0/2696826176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f693eba370 0x55f693ebecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:53.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.989+0000 7fceadd9b640 1 -- 172.21.15.120:0/2696826176 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f693f33dc0 con 0x55f693eb0b40 2024-09-19T09:19:53.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.990+0000 7fceadd9b640 1 --2- 172.21.15.120:0/2696826176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f693eb0b40 0x55f693ebe780 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fcea4010a40 tx=0x7fcea4010cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1320068eea9da95 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:53.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.990+0000 7fce95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:19:53.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.990+0000 7fce95ffb640 1 -- 172.21.15.120:0/2696826176 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea400fc50 con 0x55f693eb0b40 2024-09-19T09:19:53.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.990+0000 7fce95ffb640 1 -- 172.21.15.120:0/2696826176 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fcea4005110 con 0x55f693eb0b40 2024-09-19T09:19:53.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.990+0000 7fceb6d56740 1 -- 172.21.15.120:0/2696826176 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f693e1cd70 con 0x55f693eb0b40 2024-09-19T09:19:53.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.990+0000 7fce95ffb640 1 -- 172.21.15.120:0/2696826176 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcea4036050 con 0x55f693eb0b40 2024-09-19T09:19:54.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.990+0000 7fceb6d56740 10 client.21207 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:54.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.990+0000 7fceb6d56740 1 -- 172.21.15.120:0/2696826176 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f693f350a0 con 0x55f693eb0b40 2024-09-19T09:19:54.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.990+0000 7fce95ffb640 1 -- 172.21.15.120:0/2696826176 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f693e1cd70 con 0x55f693eb0b40 2024-09-19T09:19:54.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.990+0000 7fce95ffb640 1 -- 172.21.15.120:0/2696826176 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fcea4003c00 con 0x55f693eb0b40 2024-09-19T09:19:54.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.990+0000 7fceb6d56740 10 client.21207 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:54.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.990+0000 7fceb6d56740 10 client.21207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:19:54.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.990+0000 7fceb6d56740 20 client.21207 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:54.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.990+0000 7fceb6d56740 1 --2- 172.21.15.120:0/2696826176 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f693f358a0 0x55f693f37cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:54.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.990+0000 7fceb6d56740 4 client.21207 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:19:54.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.990+0000 7fceb6d56740 1 -- 172.21.15.120:0/2696826176 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f693d99170 con 0x55f693f358a0 2024-09-19T09:19:54.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.991+0000 7fceae59c640 1 --2- 172.21.15.120:0/2696826176 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f693f358a0 0x55f693f37cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:54.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.991+0000 7fce95ffb640 10 client.21207 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:54.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.991+0000 7fceae59c640 1 --2- 172.21.15.120:0/2696826176 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f693f358a0 0x55f693f37cc0 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2b7c66cf3ac8b24 server_cookie=3e611cc539cc24e0 in_seq=0 out_seq=0 2024-09-19T09:19:54.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.993+0000 7fce95ffb640 1 -- 172.21.15.120:0/2696826176 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f693d99170 con 0x55f693f358a0 2024-09-19T09:19:54.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.993+0000 7fce95ffb640 10 client.21207 handle_command_reply: tid=0 2024-09-19T09:19:54.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.993+0000 7fceb6d56740 1 client.21207 shutdown 2024-09-19T09:19:54.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.993+0000 7fceb6d56740 1 -- 172.21.15.120:0/2696826176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f693eb0b40 msgr2=0x55f693ebe780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:54.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.993+0000 7fceb6d56740 1 --2- 172.21.15.120:0/2696826176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f693eb0b40 0x55f693ebe780 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fcea4010a40 tx=0x7fcea4010cf0 comp rx=0 tx=0).stop 2024-09-19T09:19:54.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.993+0000 7fceb6d56740 1 -- 172.21.15.120:0/2696826176 shutdown_connections 2024-09-19T09:19:54.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.993+0000 7fceb6d56740 1 --2- 172.21.15.120:0/2696826176 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f693f358a0 0x55f693f37cc0 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:54.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.993+0000 7fceb6d56740 1 --2- 172.21.15.120:0/2696826176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f693eba370 0x55f693ebecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:54.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.993+0000 7fceb6d56740 1 --2- 172.21.15.120:0/2696826176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f693eb0b40 0x55f693ebe780 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:54.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.993+0000 7fceb6d56740 1 --2- 172.21.15.120:0/2696826176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f693eaf5d0 0x55f693ebe240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:54.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.993+0000 7fceb6d56740 1 -- 172.21.15.120:0/2696826176 >> 172.21.15.120:0/2696826176 conn(0x55f693e78310 msgr2=0x55f693eb54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:54.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.993+0000 7fce95ffb640 0 client.21207 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:54.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.993+0000 7fceb6d56740 1 -- 172.21.15.120:0/2696826176 shutdown_connections 2024-09-19T09:19:54.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.993+0000 7fceb6d56740 1 -- 172.21.15.120:0/2696826176 wait complete. 2024-09-19T09:19:54.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.993+0000 7fceb6d56740 20 client.21207 trim_cache size 0 max 16384 2024-09-19T09:19:54.002 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:19:54.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.994+0000 7fceb497d640 1 -- 172.21.15.120:0/2061758302 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fce980a32c0 msgr2=0x7fce980a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:54.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.994+0000 7fceb497d640 1 --2- 172.21.15.120:0/2061758302 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fce980a32c0 0x7fce980a5780 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fcea8006010 tx=0x7fcea803c040 comp rx=0 tx=0).stop 2024-09-19T09:19:54.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.994+0000 7fceb497d640 1 -- 172.21.15.120:0/2061758302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fceb0100e20 msgr2=0x7fceb01993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:54.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.994+0000 7fceb497d640 1 --2- 172.21.15.120:0/2061758302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fceb0100e20 0x7fceb01993b0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fcea000e9f0 tx=0x7fcea000eeb0 comp rx=0 tx=0).stop 2024-09-19T09:19:54.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.994+0000 7fceb497d640 1 -- 172.21.15.120:0/2061758302 shutdown_connections 2024-09-19T09:19:54.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.994+0000 7fceb497d640 1 --2- 172.21.15.120:0/2061758302 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fce980a32c0 0x7fce980a5780 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:54.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.994+0000 7fceb497d640 1 --2- 172.21.15.120:0/2061758302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fceb0102ef0 0x7fceb0199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:54.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.994+0000 7fceb497d640 1 --2- 172.21.15.120:0/2061758302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fceb0102570 0x7fceb01998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:54.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.994+0000 7fceb497d640 1 --2- 172.21.15.120:0/2061758302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fceb0100e20 0x7fceb01993b0 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:54.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.994+0000 7fceb497d640 1 -- 172.21.15.120:0/2061758302 >> 172.21.15.120:0/2061758302 conn(0x7fceb00fc7d0 msgr2=0x7fceb01068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:54.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.994+0000 7fceb497d640 1 -- 172.21.15.120:0/2061758302 shutdown_connections 2024-09-19T09:19:54.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:53.994+0000 7fceb497d640 1 -- 172.21.15.120:0/2061758302 wait complete. 2024-09-19T09:19:54.365 INFO:tasks.workunit.client.0.smithi120.stdout: 10 89871 16.82 MB/sec execute 109 sec latency 406.930 ms 2024-09-19T09:19:55.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:54 smithi120 ceph-mon[31150]: pgmap v856: 97 pgs: 97 active+clean; 359 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.5 MiB/s wr, 683 op/s 2024-09-19T09:19:55.365 INFO:tasks.workunit.client.0.smithi120.stdout: 10 90669 16.98 MB/sec execute 110 sec latency 55.119 ms 2024-09-19T09:19:55.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:54 smithi196 ceph-mon[35150]: pgmap v856: 97 pgs: 97 active+clean; 359 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.5 MiB/s wr, 683 op/s 2024-09-19T09:19:55.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:54 smithi153 ceph-mon[35071]: pgmap v856: 97 pgs: 97 active+clean; 359 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.5 MiB/s wr, 683 op/s 2024-09-19T09:19:55.869 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:19:55.869 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:19:56.157 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:19:56.365 INFO:tasks.workunit.client.0.smithi120.stdout: 10 91348 17.29 MB/sec execute 111 sec latency 37.899 ms 2024-09-19T09:19:56.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.648+0000 7fa208553640 1 -- 172.21.15.120:0/670137479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa200102570 msgr2=0x7fa2001029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:56.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.648+0000 7fa208553640 1 --2- 172.21.15.120:0/670137479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa200102570 0x7fa2001029b0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0009900 tx=0x7fa1f002a480 comp rx=0 tx=0).stop 2024-09-19T09:19:56.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.648+0000 7fa208553640 1 -- 172.21.15.120:0/670137479 shutdown_connections 2024-09-19T09:19:56.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.648+0000 7fa208553640 1 --2- 172.21.15.120:0/670137479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa200102ef0 0x7fa20010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.648+0000 7fa208553640 1 --2- 172.21.15.120:0/670137479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa200102570 0x7fa2001029b0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.648+0000 7fa208553640 1 --2- 172.21.15.120:0/670137479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa200100e20 0x7fa200101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.648+0000 7fa208553640 1 -- 172.21.15.120:0/670137479 >> 172.21.15.120:0/670137479 conn(0x7fa2000fc7d0 msgr2=0x7fa2000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:56.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.648+0000 7fa208553640 1 -- 172.21.15.120:0/670137479 shutdown_connections 2024-09-19T09:19:56.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.648+0000 7fa208553640 1 -- 172.21.15.120:0/670137479 wait complete. 2024-09-19T09:19:56.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.649+0000 7fa208553640 1 Processor -- start 2024-09-19T09:19:56.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.649+0000 7fa208553640 1 -- start start 2024-09-19T09:19:56.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.649+0000 7fa208553640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa200100e20 0x7fa200196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:56.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.649+0000 7fa208553640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa200102570 0x7fa2001968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:56.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.649+0000 7fa208553640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa200102ef0 0x7fa20019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:56.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.649+0000 7fa208553640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa20010fe80 con 0x7fa200102ef0 2024-09-19T09:19:56.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.649+0000 7fa208553640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa20010fd00 con 0x7fa200102570 2024-09-19T09:19:56.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.649+0000 7fa208553640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa200110000 con 0x7fa200100e20 2024-09-19T09:19:56.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.650+0000 7fa206af0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa200102ef0 0x7fa20019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:56.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.650+0000 7fa205aee640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa200102570 0x7fa2001968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:56.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.650+0000 7fa2062ef640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa200100e20 0x7fa200196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:56.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.650+0000 7fa205aee640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa200102570 0x7fa2001968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:53038/0 (socket says 172.21.15.120:53038) 2024-09-19T09:19:56.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.650+0000 7fa206af0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa200102ef0 0x7fa20019ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49708/0 (socket says 172.21.15.120:49708) 2024-09-19T09:19:56.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.650+0000 7fa2062ef640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa200100e20 0x7fa200196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:37510/0 (socket says 172.21.15.120:37510) 2024-09-19T09:19:56.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.650+0000 7fa205aee640 1 -- 172.21.15.120:0/4202057925 learned_addr learned my addr 172.21.15.120:0/4202057925 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:56.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.650+0000 7fa206af0640 1 -- 172.21.15.120:0/4202057925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa200100e20 msgr2=0x7fa200196370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:56.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.650+0000 7fa206af0640 1 --2- 172.21.15.120:0/4202057925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa200100e20 0x7fa200196370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.650+0000 7fa206af0640 1 -- 172.21.15.120:0/4202057925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa200102570 msgr2=0x7fa2001968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:56.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.650+0000 7fa206af0640 1 --2- 172.21.15.120:0/4202057925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa200102570 0x7fa2001968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.650+0000 7fa206af0640 1 -- 172.21.15.120:0/4202057925 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20019b320 con 0x7fa200102ef0 2024-09-19T09:19:56.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.650+0000 7fa2062ef640 1 --2- 172.21.15.120:0/4202057925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa200100e20 0x7fa200196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:56.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.650+0000 7fa206af0640 1 --2- 172.21.15.120:0/4202057925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa200102ef0 0x7fa20019ac40 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800da10 tx=0x7fa1f800ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:56.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.650+0000 7fa1f77fe640 1 -- 172.21.15.120:0/4202057925 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f800bd70 con 0x7fa200102ef0 2024-09-19T09:19:56.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.651+0000 7fa208553640 1 -- 172.21.15.120:0/4202057925 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa20019b610 con 0x7fa200102ef0 2024-09-19T09:19:56.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.651+0000 7fa1f77fe640 1 -- 172.21.15.120:0/4202057925 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1f8003e50 con 0x7fa200102ef0 2024-09-19T09:19:56.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.651+0000 7fa1f77fe640 1 -- 172.21.15.120:0/4202057925 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f80106a0 con 0x7fa200102ef0 2024-09-19T09:19:56.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.651+0000 7fa208553640 1 -- 172.21.15.120:0/4202057925 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2001a2ef0 con 0x7fa200102ef0 2024-09-19T09:19:56.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.653+0000 7fa20a92c740 1 Processor -- start 2024-09-19T09:19:56.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.653+0000 7fa1f77fe640 1 -- 172.21.15.120:0/4202057925 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa1f8003970 con 0x7fa200102ef0 2024-09-19T09:19:56.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.654+0000 7fa20a92c740 1 -- start start 2024-09-19T09:19:56.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.654+0000 7fa20a92c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1913d6a40 0x55c1913d6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:56.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.654+0000 7fa20a92c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c1913d7360 0x55c1913e5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:56.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.654+0000 7fa20a92c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c1913e6150 0x55c1913e8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:56.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.654+0000 7fa20a92c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c19133c280 con 0x55c1913e6150 2024-09-19T09:19:56.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.654+0000 7fa20a92c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c1912e3b10 con 0x55c1913d7360 2024-09-19T09:19:56.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.654+0000 7fa20a92c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c191338240 con 0x55c1913d6a40 2024-09-19T09:19:56.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.654+0000 7fa205aee640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1913d6a40 0x55c1913d6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:56.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.654+0000 7fa2062ef640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c1913d7360 0x55c1913e5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:56.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.654+0000 7fa205aee640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1913d6a40 0x55c1913d6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:37522/0 (socket says 172.21.15.120:37522) 2024-09-19T09:19:56.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.654+0000 7fa205aee640 1 -- 172.21.15.120:0/659842384 learned_addr learned my addr 172.21.15.120:0/659842384 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:56.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.654+0000 7fa205aee640 1 --2- 172.21.15.120:0/659842384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c1913e6150 0x55c1913e8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:56.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.655+0000 7fa2062ef640 1 -- 172.21.15.120:0/659842384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1913d6a40 msgr2=0x55c1913d6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:56.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.655+0000 7fa2062ef640 1 --2- 172.21.15.120:0/659842384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1913d6a40 0x55c1913d6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.655+0000 7fa2062ef640 1 -- 172.21.15.120:0/659842384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c1913e6150 msgr2=0x55c1913e8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:56.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.655+0000 7fa1f77fe640 1 --2- 172.21.15.120:0/4202057925 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa1ec0a3240 0x7fa1ec0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:56.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.655+0000 7fa2062ef640 1 --2- 172.21.15.120:0/659842384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c1913e6150 0x55c1913e8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.655+0000 7fa2062ef640 1 -- 172.21.15.120:0/659842384 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1912f01f0 con 0x55c1913d7360 2024-09-19T09:19:56.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.655+0000 7fa205aee640 1 --2- 172.21.15.120:0/4202057925 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa1ec0a3240 0x7fa1ec0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:56.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.655+0000 7fa1f77fe640 1 -- 172.21.15.120:0/4202057925 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa1f80cfc20 con 0x7fa200102ef0 2024-09-19T09:19:56.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.655+0000 7fa2062ef640 1 --2- 172.21.15.120:0/659842384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c1913d7360 0x55c1913e5c10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc009fd0 tx=0x7fa1fc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dc6bea34f022c8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:56.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.655+0000 7fa1f4ff9640 1 -- 172.21.15.120:0/659842384 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc004310 con 0x55c1913d7360 2024-09-19T09:19:56.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.655+0000 7fa205aee640 1 --2- 172.21.15.120:0/4202057925 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa1ec0a3240 0x7fa1ec0a5700 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x55c1913d4470 tx=0x7fa1f002a480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:56.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.655+0000 7fa1f4ff9640 1 -- 172.21.15.120:0/659842384 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1fc0044b0 con 0x55c1913d7360 2024-09-19T09:19:56.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.655+0000 7fa1f4ff9640 1 -- 172.21.15.120:0/659842384 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc003650 con 0x55c1913d7360 2024-09-19T09:19:56.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.656+0000 7fa20a92c740 1 -- 172.21.15.120:0/659842384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c1913d7360 msgr2=0x55c1913e5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:56.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.656+0000 7fa20a92c740 1 --2- 172.21.15.120:0/659842384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c1913d7360 0x55c1913e5c10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc009fd0 tx=0x7fa1fc0118d0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.656+0000 7fa20a92c740 1 -- 172.21.15.120:0/659842384 shutdown_connections 2024-09-19T09:19:56.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.656+0000 7fa20a92c740 1 --2- 172.21.15.120:0/659842384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c1913e6150 0x55c1913e8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.656+0000 7fa20a92c740 1 --2- 172.21.15.120:0/659842384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c1913d7360 0x55c1913e5c10 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.656+0000 7fa20a92c740 1 --2- 172.21.15.120:0/659842384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1913d6a40 0x55c1913d6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.656+0000 7fa20a92c740 1 -- 172.21.15.120:0/659842384 >> 172.21.15.120:0/659842384 conn(0x55c1913d30f0 msgr2=0x55c1913d34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:56.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.656+0000 7fa20a92c740 1 -- 172.21.15.120:0/659842384 shutdown_connections 2024-09-19T09:19:56.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.657+0000 7fa20a92c740 1 -- 172.21.15.120:0/659842384 wait complete. 2024-09-19T09:19:56.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.657+0000 7fa20a92c740 1 -- start start 2024-09-19T09:19:56.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.657+0000 7fa20a92c740 1 Processor -- start 2024-09-19T09:19:56.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.657+0000 7fa20a92c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1913d6a40 0x55c1913e3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:56.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.657+0000 7fa20a92c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c1913d7360 0x55c1913e35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:56.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.657+0000 7fa20a92c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c1913e6150 0x55c1913e3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:56.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.657+0000 7fa20a92c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c1913e8d20 con 0x55c1913e6150 2024-09-19T09:19:56.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.657+0000 7fa20a92c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c1912f6f10 con 0x55c1913d7360 2024-09-19T09:19:56.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.657+0000 7fa20a92c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c1913e8ea0 con 0x55c1913d6a40 2024-09-19T09:19:56.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.657+0000 7fa20a92c740 10 client.0 _ll_register_callbacks cb 0x55c1912bfa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:56.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.657+0000 7fa2062ef640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1913d6a40 0x55c1913e3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:56.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.657+0000 7fa206af0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c1913e6150 0x55c1913e3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:56.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.657+0000 7fa206af0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c1913e6150 0x55c1913e3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49730/0 (socket says 172.21.15.120:49730) 2024-09-19T09:19:56.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.657+0000 7fa206af0640 1 -- 172.21.15.120:0/227099772 learned_addr learned my addr 172.21.15.120:0/227099772 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:56.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.657+0000 7fa205aee640 1 --2- 172.21.15.120:0/227099772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c1913d7360 0x55c1913e35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:56.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.658+0000 7fa2062ef640 1 -- 172.21.15.120:0/227099772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c1913d7360 msgr2=0x55c1913e35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:56.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.658+0000 7fa2062ef640 1 --2- 172.21.15.120:0/227099772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c1913d7360 0x55c1913e35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.658+0000 7fa2062ef640 1 -- 172.21.15.120:0/227099772 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c1913e6150 msgr2=0x55c1913e3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:56.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.658+0000 7fa2062ef640 1 --2- 172.21.15.120:0/227099772 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c1913e6150 0x55c1913e3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.658+0000 7fa2062ef640 1 -- 172.21.15.120:0/227099772 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1910ad6a0 con 0x55c1913d6a40 2024-09-19T09:19:56.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.658+0000 7fa206af0640 1 --2- 172.21.15.120:0/227099772 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c1913e6150 0x55c1913e3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:56.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.658+0000 7fa205aee640 1 --2- 172.21.15.120:0/227099772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c1913d7360 0x55c1913e35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:56.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.658+0000 7fa2062ef640 1 --2- 172.21.15.120:0/227099772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1913d6a40 0x55c1913e3070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc012fd0 tx=0x7fa1fc009c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68075f97580d9999 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:56.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.658+0000 7fa1e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:19:56.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.658+0000 7fa1e9ffb640 1 -- 172.21.15.120:0/227099772 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc005200 con 0x55c1913d6a40 2024-09-19T09:19:56.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.658+0000 7fa20a92c740 1 -- 172.21.15.120:0/227099772 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1fc0044b0 con 0x55c1913d6a40 2024-09-19T09:19:56.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.658+0000 7fa1e9ffb640 1 -- 172.21.15.120:0/227099772 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa1fc0053a0 con 0x55c1913d6a40 2024-09-19T09:19:56.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.659+0000 7fa1e9ffb640 1 -- 172.21.15.120:0/227099772 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa1fc034050 con 0x55c1913d6a40 2024-09-19T09:19:56.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.659+0000 7fa20a92c740 10 client.29896 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:56.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.659+0000 7fa20a92c740 1 -- 172.21.15.120:0/227099772 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c1913f4990 con 0x55c1913d6a40 2024-09-19T09:19:56.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.659+0000 7fa1e9ffb640 1 -- 172.21.15.120:0/227099772 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1fc0044b0 con 0x55c1913d6a40 2024-09-19T09:19:56.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.659+0000 7fa1e9ffb640 1 -- 172.21.15.120:0/227099772 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fa1fc00f480 con 0x55c1913d6a40 2024-09-19T09:19:56.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.660+0000 7fa20a92c740 10 client.29896 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:56.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.660+0000 7fa20a92c740 10 client.29896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:19:56.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.660+0000 7fa20a92c740 20 client.29896 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:56.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.660+0000 7fa20a92c740 1 --2- 172.21.15.120:0/227099772 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c1913f6170 0x55c1913f85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:56.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.660+0000 7fa20a92c740 4 client.29896 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:19:56.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.660+0000 7fa20a92c740 1 -- 172.21.15.120:0/227099772 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1913497f0 con 0x55c1913f6170 2024-09-19T09:19:56.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.660+0000 7fa205aee640 1 --2- 172.21.15.120:0/227099772 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c1913f6170 0x55c1913f85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:56.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.660+0000 7fa205aee640 1 --2- 172.21.15.120:0/227099772 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c1913f6170 0x55c1913f85e0 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2d054098b3c8ffa server_cookie=b9b62537e3b535df in_seq=0 out_seq=0 2024-09-19T09:19:56.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.660+0000 7fa1e9ffb640 10 client.29896 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:56.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.665+0000 7fa1e9ffb640 1 -- 172.21.15.120:0/227099772 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c1913497f0 con 0x55c1913f6170 2024-09-19T09:19:56.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.665+0000 7fa1e9ffb640 10 client.29896 handle_command_reply: tid=0 2024-09-19T09:19:56.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.666+0000 7fa20a92c740 1 client.29896 shutdown 2024-09-19T09:19:56.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.666+0000 7fa20a92c740 1 -- 172.21.15.120:0/227099772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1913d6a40 msgr2=0x55c1913e3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:56.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.666+0000 7fa20a92c740 1 --2- 172.21.15.120:0/227099772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1913d6a40 0x55c1913e3070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc012fd0 tx=0x7fa1fc009c90 comp rx=0 tx=0).stop 2024-09-19T09:19:56.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.666+0000 7fa20a92c740 1 -- 172.21.15.120:0/227099772 shutdown_connections 2024-09-19T09:19:56.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.666+0000 7fa20a92c740 1 --2- 172.21.15.120:0/227099772 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c1913f6170 0x55c1913f85e0 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.666+0000 7fa20a92c740 1 --2- 172.21.15.120:0/227099772 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c1913e6150 0x55c1913e3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.666+0000 7fa20a92c740 1 --2- 172.21.15.120:0/227099772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c1913d7360 0x55c1913e35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.666+0000 7fa1e9ffb640 0 client.29896 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:56.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.666+0000 7fa20a92c740 1 --2- 172.21.15.120:0/227099772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1913d6a40 0x55c1913e3070 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.666+0000 7fa20a92c740 1 -- 172.21.15.120:0/227099772 >> 172.21.15.120:0/227099772 conn(0x55c1913d30f0 msgr2=0x55c1913d8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:56.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.666+0000 7fa20a92c740 1 -- 172.21.15.120:0/227099772 shutdown_connections 2024-09-19T09:19:56.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.666+0000 7fa20a92c740 1 -- 172.21.15.120:0/227099772 wait complete. 2024-09-19T09:19:56.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.666+0000 7fa20a92c740 20 client.29896 trim_cache size 0 max 16384 2024-09-19T09:19:56.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.686+0000 7fa20a92c740 1 Processor -- start 2024-09-19T09:19:56.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.686+0000 7fa20a92c740 1 -- start start 2024-09-19T09:19:56.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.686+0000 7fa20a92c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c1913d6a40 0x55c1913fde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:56.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.686+0000 7fa20a92c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c1913d7360 0x55c1913fe3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:56.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.686+0000 7fa20a92c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1913e6150 0x55c1913e3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:56.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.686+0000 7fa20a92c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa2001ab390 con 0x55c1913d7360 2024-09-19T09:19:56.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.686+0000 7fa20a92c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa2001aced0 con 0x55c1913d6a40 2024-09-19T09:19:56.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.686+0000 7fa20a92c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa2001aba60 con 0x55c1913e6150 2024-09-19T09:19:56.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.687+0000 7fa205aee640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c1913d7360 0x55c1913fe3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:56.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.687+0000 7fa205aee640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c1913d7360 0x55c1913fe3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49736/0 (socket says 172.21.15.120:49736) 2024-09-19T09:19:56.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.687+0000 7fa205aee640 1 -- 172.21.15.120:0/1319525573 learned_addr learned my addr 172.21.15.120:0/1319525573 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:56.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.687+0000 7fa206af0640 1 --2- 172.21.15.120:0/1319525573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1913e6150 0x55c1913e3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:56.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.687+0000 7fa2062ef640 1 --2- 172.21.15.120:0/1319525573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c1913d6a40 0x55c1913fde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:56.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.687+0000 7fa205aee640 1 -- 172.21.15.120:0/1319525573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1913e6150 msgr2=0x55c1913e3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:56.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.687+0000 7fa205aee640 1 --2- 172.21.15.120:0/1319525573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1913e6150 0x55c1913e3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.687+0000 7fa205aee640 1 -- 172.21.15.120:0/1319525573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c1913d6a40 msgr2=0x55c1913fde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:56.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.687+0000 7fa205aee640 1 --2- 172.21.15.120:0/1319525573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c1913d6a40 0x55c1913fde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.687+0000 7fa205aee640 1 -- 172.21.15.120:0/1319525573 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1913ddde0 con 0x55c1913d7360 2024-09-19T09:19:56.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.687+0000 7fa205aee640 1 --2- 172.21.15.120:0/1319525573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c1913d7360 0x55c1913fe3c0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0000c00 tx=0x7fa1f0041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d5e0cb0bdd41f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:56.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.687+0000 7fa1ebfff640 1 -- 172.21.15.120:0/1319525573 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0003e80 con 0x55c1913d7360 2024-09-19T09:19:56.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.687+0000 7fa1ebfff640 1 -- 172.21.15.120:0/1319525573 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1f0031a20 con 0x55c1913d7360 2024-09-19T09:19:56.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.687+0000 7fa1ebfff640 1 -- 172.21.15.120:0/1319525573 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f003ca40 con 0x55c1913d7360 2024-09-19T09:19:56.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.688+0000 7fa20a92c740 1 -- 172.21.15.120:0/1319525573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c1913d7360 msgr2=0x55c1913fe3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:56.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.688+0000 7fa20a92c740 1 --2- 172.21.15.120:0/1319525573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c1913d7360 0x55c1913fe3c0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0000c00 tx=0x7fa1f0041040 comp rx=0 tx=0).stop 2024-09-19T09:19:56.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.688+0000 7fa20a92c740 1 -- 172.21.15.120:0/1319525573 shutdown_connections 2024-09-19T09:19:56.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.688+0000 7fa20a92c740 1 --2- 172.21.15.120:0/1319525573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1913e6150 0x55c1913e3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.688+0000 7fa20a92c740 1 --2- 172.21.15.120:0/1319525573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c1913d7360 0x55c1913fe3c0 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.688+0000 7fa20a92c740 1 --2- 172.21.15.120:0/1319525573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c1913d6a40 0x55c1913fde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.688+0000 7fa20a92c740 1 -- 172.21.15.120:0/1319525573 >> 172.21.15.120:0/1319525573 conn(0x55c1913d30f0 msgr2=0x55c1913e1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:56.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.688+0000 7fa20a92c740 1 -- 172.21.15.120:0/1319525573 shutdown_connections 2024-09-19T09:19:56.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.688+0000 7fa20a92c740 1 -- 172.21.15.120:0/1319525573 wait complete. 2024-09-19T09:19:56.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.689+0000 7fa20a92c740 1 -- start start 2024-09-19T09:19:56.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.689+0000 7fa20a92c740 1 Processor -- start 2024-09-19T09:19:56.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.689+0000 7fa20a92c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c1913d6a40 0x55c1913ee5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:56.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.689+0000 7fa20a92c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c1913d7360 0x55c1913e9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:56.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.689+0000 7fa20a92c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1913e6150 0x55c1913e9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:56.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.689+0000 7fa20a92c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa2001ac800 con 0x55c1913d7360 2024-09-19T09:19:56.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.689+0000 7fa20a92c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa2001ac130 con 0x55c1913d6a40 2024-09-19T09:19:56.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.689+0000 7fa20a92c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa2001a9350 con 0x55c1913e6150 2024-09-19T09:19:56.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.689+0000 7fa205aee640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c1913d7360 0x55c1913e9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:56.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.689+0000 7fa20a92c740 10 client.0 _ll_register_callbacks cb 0x7fa1f00041d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:56.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.689+0000 7fa205aee640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c1913d7360 0x55c1913e9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49748/0 (socket says 172.21.15.120:49748) 2024-09-19T09:19:56.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.689+0000 7fa205aee640 1 -- 172.21.15.120:0/424383378 learned_addr learned my addr 172.21.15.120:0/424383378 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:56.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.689+0000 7fa206af0640 1 --2- 172.21.15.120:0/424383378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1913e6150 0x55c1913e9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:56.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.689+0000 7fa2062ef640 1 --2- 172.21.15.120:0/424383378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c1913d6a40 0x55c1913ee5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:56.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.690+0000 7fa206af0640 1 -- 172.21.15.120:0/424383378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c1913d6a40 msgr2=0x55c1913ee5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:56.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.690+0000 7fa206af0640 1 --2- 172.21.15.120:0/424383378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c1913d6a40 0x55c1913ee5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.690+0000 7fa206af0640 1 -- 172.21.15.120:0/424383378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c1913d7360 msgr2=0x55c1913e9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:56.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.690+0000 7fa206af0640 1 --2- 172.21.15.120:0/424383378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c1913d7360 0x55c1913e9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.690+0000 7fa206af0640 1 -- 172.21.15.120:0/424383378 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1913ea3c0 con 0x55c1913e6150 2024-09-19T09:19:56.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.690+0000 7fa205aee640 1 --2- 172.21.15.120:0/424383378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c1913d7360 0x55c1913e9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:56.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.690+0000 7fa206af0640 1 --2- 172.21.15.120:0/424383378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1913e6150 0x55c1913e9e50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa1f809bf40 tx=0x7fa1f809bc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79bc5fb2156d8491 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:56.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.690+0000 7fa1e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:19:56.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.690+0000 7fa1e9ffb640 1 -- 172.21.15.120:0/424383378 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f80a4eb0 con 0x55c1913e6150 2024-09-19T09:19:56.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.690+0000 7fa1e9ffb640 1 -- 172.21.15.120:0/424383378 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa1f80042e0 con 0x55c1913e6150 2024-09-19T09:19:56.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.690+0000 7fa20a92c740 1 -- 172.21.15.120:0/424383378 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1f0031a20 con 0x55c1913e6150 2024-09-19T09:19:56.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.691+0000 7fa1e9ffb640 1 -- 172.21.15.120:0/424383378 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa1f8003c60 con 0x55c1913e6150 2024-09-19T09:19:56.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.691+0000 7fa20a92c740 10 client.29902 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:56.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.691+0000 7fa20a92c740 1 -- 172.21.15.120:0/424383378 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c191460c50 con 0x55c1913e6150 2024-09-19T09:19:56.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.691+0000 7fa1e9ffb640 1 -- 172.21.15.120:0/424383378 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1f0031a20 con 0x55c1913e6150 2024-09-19T09:19:56.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.691+0000 7fa1e9ffb640 1 -- 172.21.15.120:0/424383378 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fa1f8024060 con 0x55c1913e6150 2024-09-19T09:19:56.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.691+0000 7fa20a92c740 10 client.29902 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:56.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.691+0000 7fa20a92c740 10 client.29902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:19:56.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.691+0000 7fa20a92c740 20 client.29902 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:56.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.691+0000 7fa20a92c740 1 --2- 172.21.15.120:0/424383378 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c191461380 0x55c1914637a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:56.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.691+0000 7fa20a92c740 4 client.29902 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:19:56.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.691+0000 7fa20a92c740 1 -- 172.21.15.120:0/424383378 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c191343d00 con 0x55c191461380 2024-09-19T09:19:56.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.691+0000 7fa2062ef640 1 --2- 172.21.15.120:0/424383378 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c191461380 0x55c1914637a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:56.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.692+0000 7fa1e9ffb640 10 client.29902 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:56.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.692+0000 7fa2062ef640 1 --2- 172.21.15.120:0/424383378 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c191461380 0x55c1914637a0 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ada4c4d98276cc0 server_cookie=76a0f8f729f05ff9 in_seq=0 out_seq=0 2024-09-19T09:19:56.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.694+0000 7fa1e9ffb640 1 -- 172.21.15.120:0/424383378 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c191343d00 con 0x55c191461380 2024-09-19T09:19:56.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.694+0000 7fa1e9ffb640 10 client.29902 handle_command_reply: tid=0 2024-09-19T09:19:56.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.694+0000 7fa20a92c740 1 client.29902 shutdown 2024-09-19T09:19:56.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.694+0000 7fa20a92c740 1 -- 172.21.15.120:0/424383378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1913e6150 msgr2=0x55c1913e9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:56.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.694+0000 7fa20a92c740 1 --2- 172.21.15.120:0/424383378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1913e6150 0x55c1913e9e50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa1f809bf40 tx=0x7fa1f809bc60 comp rx=0 tx=0).stop 2024-09-19T09:19:56.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.694+0000 7fa20a92c740 1 -- 172.21.15.120:0/424383378 shutdown_connections 2024-09-19T09:19:56.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.694+0000 7fa20a92c740 1 --2- 172.21.15.120:0/424383378 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c191461380 0x55c1914637a0 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.694+0000 7fa20a92c740 1 --2- 172.21.15.120:0/424383378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1913e6150 0x55c1913e9e50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.694+0000 7fa20a92c740 1 --2- 172.21.15.120:0/424383378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c1913d7360 0x55c1913e9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.694+0000 7fa20a92c740 1 --2- 172.21.15.120:0/424383378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c1913d6a40 0x55c1913ee5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.694+0000 7fa20a92c740 1 -- 172.21.15.120:0/424383378 >> 172.21.15.120:0/424383378 conn(0x55c1913d30f0 msgr2=0x55c1913d1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:56.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.694+0000 7fa1e9ffb640 0 client.29902 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:56.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.694+0000 7fa20a92c740 1 -- 172.21.15.120:0/424383378 shutdown_connections 2024-09-19T09:19:56.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.694+0000 7fa20a92c740 1 -- 172.21.15.120:0/424383378 wait complete. 2024-09-19T09:19:56.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.695+0000 7fa20a92c740 20 client.29902 trim_cache size 0 max 16384 2024-09-19T09:19:56.701 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:19:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:19:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "cf55f3fb-e778-4b8b-9a8c-d2834cc2cbb0", 2024-09-19T09:19:56.701 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:19:56.701 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:19:56.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.695+0000 7fa208553640 1 -- 172.21.15.120:0/4202057925 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa1ec0a3240 msgr2=0x7fa1ec0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:56.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.695+0000 7fa208553640 1 --2- 172.21.15.120:0/4202057925 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa1ec0a3240 0x7fa1ec0a5700 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x55c1913d4470 tx=0x7fa1f002a480 comp rx=0 tx=0).stop 2024-09-19T09:19:56.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.695+0000 7fa208553640 1 -- 172.21.15.120:0/4202057925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa200102ef0 msgr2=0x7fa20019ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:56.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.695+0000 7fa208553640 1 --2- 172.21.15.120:0/4202057925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa200102ef0 0x7fa20019ac40 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800da10 tx=0x7fa1f800ded0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.695+0000 7fa208553640 1 -- 172.21.15.120:0/4202057925 shutdown_connections 2024-09-19T09:19:56.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.696+0000 7fa208553640 1 --2- 172.21.15.120:0/4202057925 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa1ec0a3240 0x7fa1ec0a5700 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.696+0000 7fa208553640 1 --2- 172.21.15.120:0/4202057925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa200102ef0 0x7fa20019ac40 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.696+0000 7fa208553640 1 --2- 172.21.15.120:0/4202057925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa200102570 0x7fa2001968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.696+0000 7fa208553640 1 --2- 172.21.15.120:0/4202057925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa200100e20 0x7fa200196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:56.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.696+0000 7fa208553640 1 -- 172.21.15.120:0/4202057925 >> 172.21.15.120:0/4202057925 conn(0x7fa2000fc7d0 msgr2=0x7fa2000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:56.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.696+0000 7fa208553640 1 -- 172.21.15.120:0/4202057925 shutdown_connections 2024-09-19T09:19:56.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:56.696+0000 7fa208553640 1 -- 172.21.15.120:0/4202057925 wait complete. 2024-09-19T09:19:57.308 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:56 smithi120 ceph-mon[31150]: pgmap v857: 97 pgs: 97 active+clean; 362 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.9 MiB/s wr, 610 op/s 2024-09-19T09:19:57.365 INFO:tasks.workunit.client.0.smithi120.stdout: 10 92016 17.47 MB/sec execute 112 sec latency 64.381 ms 2024-09-19T09:19:57.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:56 smithi196 ceph-mon[35150]: pgmap v857: 97 pgs: 97 active+clean; 362 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.9 MiB/s wr, 610 op/s 2024-09-19T09:19:57.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:56 smithi153 ceph-mon[35071]: pgmap v857: 97 pgs: 97 active+clean; 362 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.9 MiB/s wr, 610 op/s 2024-09-19T09:19:57.577 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:19:57.881 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:19:58.296 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:57 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:19:58.296 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:57 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:58.296 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:57 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:19:58.296 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:57 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:19:58.296 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:57 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:58.296 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:57 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:19:58.296 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:57 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:58.296 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:57 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:19:58.296 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:57 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:58.365 INFO:tasks.workunit.client.0.smithi120.stdout: 10 92314 17.43 MB/sec execute 113 sec latency 231.936 ms 2024-09-19T09:19:58.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.381+0000 7f5085b16640 1 -- 172.21.15.120:0/3548092537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5080100e20 msgr2=0x7f5080101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:58.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.381+0000 7f5085b16640 1 --2- 172.21.15.120:0/3548092537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5080100e20 0x7f5080101200 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f5070009900 tx=0x7f507002a480 comp rx=0 tx=0).stop 2024-09-19T09:19:58.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.381+0000 7f5085b16640 1 -- 172.21.15.120:0/3548092537 shutdown_connections 2024-09-19T09:19:58.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.381+0000 7f5085b16640 1 --2- 172.21.15.120:0/3548092537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5080102ef0 0x7f508010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.381+0000 7f5085b16640 1 --2- 172.21.15.120:0/3548092537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5080102570 0x7f50801029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.381+0000 7f5085b16640 1 --2- 172.21.15.120:0/3548092537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5080100e20 0x7f5080101200 secure :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f5070009900 tx=0x7f507002a480 comp rx=0 tx=0).stop 2024-09-19T09:19:58.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.381+0000 7f5085b16640 1 -- 172.21.15.120:0/3548092537 >> 172.21.15.120:0/3548092537 conn(0x7f50800fc7d0 msgr2=0x7f50800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:58.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.382+0000 7f5085b16640 1 -- 172.21.15.120:0/3548092537 shutdown_connections 2024-09-19T09:19:58.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.382+0000 7f5085b16640 1 -- 172.21.15.120:0/3548092537 wait complete. 2024-09-19T09:19:58.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.384+0000 7f5085b16640 1 Processor -- start 2024-09-19T09:19:58.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.384+0000 7f5085b16640 1 -- start start 2024-09-19T09:19:58.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.384+0000 7f5085b16640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5080102570 0x7f508010d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:58.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.384+0000 7f5085b16640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5080102ef0 0x7f5080109a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:58.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.384+0000 7f5085b16640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f508010a0e0 0x7f50801ab000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:58.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.384+0000 7f5085b16640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f508010fed0 con 0x7f5080102ef0 2024-09-19T09:19:58.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.384+0000 7f5085b16640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f508010fd50 con 0x7f5080102570 2024-09-19T09:19:58.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.384+0000 7f5085b16640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5080110050 con 0x7f508010a0e0 2024-09-19T09:19:58.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.385+0000 7f507f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5080102570 0x7f508010d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:58.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.385+0000 7f507f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5080102570 0x7f508010d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:53104/0 (socket says 172.21.15.120:53104) 2024-09-19T09:19:58.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.385+0000 7f507f7fe640 1 -- 172.21.15.120:0/566072437 learned_addr learned my addr 172.21.15.120:0/566072437 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:58.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.385+0000 7f507ffff640 1 --2- 172.21.15.120:0/566072437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f508010a0e0 0x7f50801ab000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:58.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.385+0000 7f507effd640 1 --2- 172.21.15.120:0/566072437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5080102ef0 0x7f5080109a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:58.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.385+0000 7f507f7fe640 1 -- 172.21.15.120:0/566072437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f508010a0e0 msgr2=0x7f50801ab000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:58.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.385+0000 7f507f7fe640 1 --2- 172.21.15.120:0/566072437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f508010a0e0 0x7f50801ab000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.385+0000 7f507f7fe640 1 -- 172.21.15.120:0/566072437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5080102ef0 msgr2=0x7f5080109a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:58.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.385+0000 7f507f7fe640 1 --2- 172.21.15.120:0/566072437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5080102ef0 0x7f5080109a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.385+0000 7f507f7fe640 1 -- 172.21.15.120:0/566072437 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50801ab540 con 0x7f5080102570 2024-09-19T09:19:58.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.385+0000 7f507ffff640 1 --2- 172.21.15.120:0/566072437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f508010a0e0 0x7f50801ab000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:58.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.386+0000 7f507f7fe640 1 --2- 172.21.15.120:0/566072437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5080102570 0x7f508010d9f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f507002aa70 tx=0x7f5070003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:58.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.386+0000 7f507cff9640 1 -- 172.21.15.120:0/566072437 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507003f020 con 0x7f5080102570 2024-09-19T09:19:58.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.386+0000 7f507cff9640 1 -- 172.21.15.120:0/566072437 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5070031c30 con 0x7f5080102570 2024-09-19T09:19:58.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.386+0000 7f507cff9640 1 -- 172.21.15.120:0/566072437 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5070035070 con 0x7f5080102570 2024-09-19T09:19:58.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.387+0000 7f5085b16640 1 -- 172.21.15.120:0/566072437 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50801ab710 con 0x7f5080102570 2024-09-19T09:19:58.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.387+0000 7f5085b16640 1 -- 172.21.15.120:0/566072437 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50801abb10 con 0x7f5080102570 2024-09-19T09:19:58.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.390+0000 7f5087eef740 1 Processor -- start 2024-09-19T09:19:58.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.390+0000 7f507cff9640 1 -- 172.21.15.120:0/566072437 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5070042020 con 0x7f5080102570 2024-09-19T09:19:58.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.390+0000 7f5087eef740 1 -- start start 2024-09-19T09:19:58.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.390+0000 7f5087eef740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbc9d594a0 0x55dbc9d59880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:58.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.390+0000 7f5087eef740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbc9d59dc0 0x55dbc9d64140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:58.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.391+0000 7f5087eef740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbc9d64680 0x55dbc9d66a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:58.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.391+0000 7f507f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbc9d59dc0 0x55dbc9d64140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:58.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.391+0000 7f507f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbc9d59dc0 0x55dbc9d64140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:37558/0 (socket says 172.21.15.120:37558) 2024-09-19T09:19:58.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.391+0000 7f507ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbc9d594a0 0x55dbc9d59880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:58.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.391+0000 7f507effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbc9d64680 0x55dbc9d66a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:58.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.391+0000 7f507ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbc9d594a0 0x55dbc9d59880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:53110/0 (socket says 172.21.15.120:53110) 2024-09-19T09:19:58.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.391+0000 7f507effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbc9d64680 0x55dbc9d66a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49782/0 (socket says 172.21.15.120:49782) 2024-09-19T09:19:58.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.391+0000 7f507f7fe640 1 -- 172.21.15.120:0/3080601998 learned_addr learned my addr 172.21.15.120:0/3080601998 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:58.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.391+0000 7f5087eef740 1 -- 172.21.15.120:0/3080601998 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dbc9cbf2e0 con 0x55dbc9d64680 2024-09-19T09:19:58.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.391+0000 7f5087eef740 1 -- 172.21.15.120:0/3080601998 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55dbc9c66ce0 con 0x55dbc9d594a0 2024-09-19T09:19:58.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.391+0000 7f5087eef740 1 -- 172.21.15.120:0/3080601998 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dbc9cbb2a0 con 0x55dbc9d59dc0 2024-09-19T09:19:58.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.391+0000 7f507f7fe640 1 -- 172.21.15.120:0/3080601998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbc9d594a0 msgr2=0x55dbc9d59880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:58.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.391+0000 7f507f7fe640 1 --2- 172.21.15.120:0/3080601998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbc9d594a0 0x55dbc9d59880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.391+0000 7f507f7fe640 1 -- 172.21.15.120:0/3080601998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbc9d64680 msgr2=0x55dbc9d66a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:58.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.391+0000 7f507f7fe640 1 --2- 172.21.15.120:0/3080601998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbc9d64680 0x55dbc9d66a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.391+0000 7f507f7fe640 1 -- 172.21.15.120:0/3080601998 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbc9c4b440 con 0x55dbc9d59dc0 2024-09-19T09:19:58.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.392+0000 7f507f7fe640 1 --2- 172.21.15.120:0/3080601998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbc9d59dc0 0x55dbc9d64140 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f50700f3c70 tx=0x7f50700f3d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbb028c725cd5201 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:58.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.392+0000 7f507cff9640 1 --2- 172.21.15.120:0/566072437 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f50680a3410 0x7f50680a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:58.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.392+0000 7f507ffff640 1 --2- 172.21.15.120:0/566072437 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f50680a3410 0x7f50680a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:58.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.392+0000 7f507cff9640 1 -- 172.21.15.120:0/566072437 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f50700efbc0 con 0x7f5080102570 2024-09-19T09:19:58.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.392+0000 7f5065ffb640 1 -- 172.21.15.120:0/3080601998 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50700b8440 con 0x55dbc9d59dc0 2024-09-19T09:19:58.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.393+0000 7f5065ffb640 1 -- 172.21.15.120:0/3080601998 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50700bc8a0 con 0x55dbc9d59dc0 2024-09-19T09:19:58.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.393+0000 7f5065ffb640 1 -- 172.21.15.120:0/3080601998 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50700bbb00 con 0x55dbc9d59dc0 2024-09-19T09:19:58.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.393+0000 7f5087eef740 1 -- 172.21.15.120:0/3080601998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbc9d59dc0 msgr2=0x55dbc9d64140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:58.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.393+0000 7f507ffff640 1 --2- 172.21.15.120:0/566072437 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f50680a3410 0x7f50680a58d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5078005fd0 tx=0x7f5078009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:58.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.393+0000 7f5087eef740 1 --2- 172.21.15.120:0/3080601998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbc9d59dc0 0x55dbc9d64140 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f50700f3c70 tx=0x7f50700f3d60 comp rx=0 tx=0).stop 2024-09-19T09:19:58.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.394+0000 7f5087eef740 1 -- 172.21.15.120:0/3080601998 shutdown_connections 2024-09-19T09:19:58.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.394+0000 7f5087eef740 1 --2- 172.21.15.120:0/3080601998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbc9d64680 0x55dbc9d66a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.394+0000 7f5087eef740 1 --2- 172.21.15.120:0/3080601998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbc9d59dc0 0x55dbc9d64140 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.394+0000 7f5087eef740 1 --2- 172.21.15.120:0/3080601998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbc9d594a0 0x55dbc9d59880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.394+0000 7f5087eef740 1 -- 172.21.15.120:0/3080601998 >> 172.21.15.120:0/3080601998 conn(0x55dbc9d55a30 msgr2=0x55dbc9d55e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:58.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.394+0000 7f5087eef740 1 -- 172.21.15.120:0/3080601998 shutdown_connections 2024-09-19T09:19:58.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.394+0000 7f5087eef740 1 -- 172.21.15.120:0/3080601998 wait complete. 2024-09-19T09:19:58.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.394+0000 7f5087eef740 1 -- start start 2024-09-19T09:19:58.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.394+0000 7f5087eef740 1 Processor -- start 2024-09-19T09:19:58.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.395+0000 7f5087eef740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbc9d594a0 0x55dbc9d63f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:58.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.395+0000 7f5087eef740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbc9d59dc0 0x55dbc9d61200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:58.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.395+0000 7f5087eef740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbc9d64680 0x55dbc9d61740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:58.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.395+0000 7f5087eef740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dbc9cbb2a0 con 0x55dbc9d59dc0 2024-09-19T09:19:58.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.395+0000 7f5087eef740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55dbc9c7a050 con 0x55dbc9d594a0 2024-09-19T09:19:58.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.395+0000 7f5087eef740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dbc9d67010 con 0x55dbc9d64680 2024-09-19T09:19:58.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.395+0000 7f5087eef740 10 client.0 _ll_register_callbacks cb 0x55dbc9cba160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:58.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.395+0000 7f507ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbc9d64680 0x55dbc9d61740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:58.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.395+0000 7f507effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbc9d59dc0 0x55dbc9d61200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:58.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.395+0000 7f507f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbc9d594a0 0x55dbc9d63f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:58.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.395+0000 7f507ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbc9d64680 0x55dbc9d61740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:37568/0 (socket says 172.21.15.120:37568) 2024-09-19T09:19:58.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.395+0000 7f507f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbc9d594a0 0x55dbc9d63f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:53118/0 (socket says 172.21.15.120:53118) 2024-09-19T09:19:58.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.395+0000 7f507ffff640 1 -- 172.21.15.120:0/3693287854 learned_addr learned my addr 172.21.15.120:0/3693287854 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:58.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.395+0000 7f507ffff640 1 -- 172.21.15.120:0/3693287854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbc9d594a0 msgr2=0x55dbc9d63f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:58.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.395+0000 7f507ffff640 1 --2- 172.21.15.120:0/3693287854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbc9d594a0 0x55dbc9d63f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.395+0000 7f507ffff640 1 -- 172.21.15.120:0/3693287854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbc9d59dc0 msgr2=0x55dbc9d61200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:58.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.395+0000 7f507ffff640 1 --2- 172.21.15.120:0/3693287854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbc9d59dc0 0x55dbc9d61200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.395+0000 7f507ffff640 1 -- 172.21.15.120:0/3693287854 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbc9a303c0 con 0x55dbc9d64680 2024-09-19T09:19:58.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.395+0000 7f507effd640 1 --2- 172.21.15.120:0/3693287854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbc9d59dc0 0x55dbc9d61200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:19:58.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.396+0000 7f507ffff640 1 --2- 172.21.15.120:0/3693287854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbc9d64680 0x55dbc9d61740 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5078003f20 tx=0x7f5078004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=113c83b672e4c996 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:58.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.396+0000 7f505effd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:19:58.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.396+0000 7f505effd640 1 -- 172.21.15.120:0/3693287854 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50780047d0 con 0x55dbc9d64680 2024-09-19T09:19:58.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.396+0000 7f505effd640 1 -- 172.21.15.120:0/3693287854 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f507801f430 con 0x55dbc9d64680 2024-09-19T09:19:58.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.396+0000 7f5087eef740 1 -- 172.21.15.120:0/3693287854 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50700bc8a0 con 0x55dbc9d64680 2024-09-19T09:19:58.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.397+0000 7f505effd640 1 -- 172.21.15.120:0/3693287854 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f507801f630 con 0x55dbc9d64680 2024-09-19T09:19:58.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.397+0000 7f5087eef740 10 client.29914 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:58.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.397+0000 7f5087eef740 1 -- 172.21.15.120:0/3693287854 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55dbc9d72800 con 0x55dbc9d64680 2024-09-19T09:19:58.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.397+0000 7f505effd640 1 -- 172.21.15.120:0/3693287854 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50700bc8a0 con 0x55dbc9d64680 2024-09-19T09:19:58.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.397+0000 7f505effd640 1 -- 172.21.15.120:0/3693287854 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f5078027070 con 0x55dbc9d64680 2024-09-19T09:19:58.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.397+0000 7f5087eef740 10 client.29914 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:58.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.397+0000 7f5087eef740 10 client.29914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:19:58.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.397+0000 7f5087eef740 20 client.29914 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:58.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.397+0000 7f5087eef740 1 --2- 172.21.15.120:0/3693287854 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dbc9d73cb0 0x55dbc9d760e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:58.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.397+0000 7f5087eef740 4 client.29914 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:19:58.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.397+0000 7f5087eef740 1 -- 172.21.15.120:0/3693287854 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbc9ccc850 con 0x55dbc9d73cb0 2024-09-19T09:19:58.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.397+0000 7f507f7fe640 1 --2- 172.21.15.120:0/3693287854 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dbc9d73cb0 0x55dbc9d760e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:58.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.398+0000 7f505effd640 10 client.29914 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:58.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.398+0000 7f507f7fe640 1 --2- 172.21.15.120:0/3693287854 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dbc9d73cb0 0x55dbc9d760e0 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5478be853466c056 server_cookie=9ec6bce6181efc8a in_seq=0 out_seq=0 2024-09-19T09:19:58.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.401+0000 7f505effd640 1 -- 172.21.15.120:0/3693287854 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dbc9ccc850 con 0x55dbc9d73cb0 2024-09-19T09:19:58.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.401+0000 7f505effd640 10 client.29914 handle_command_reply: tid=0 2024-09-19T09:19:58.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.401+0000 7f5087eef740 1 client.29914 shutdown 2024-09-19T09:19:58.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.401+0000 7f5087eef740 1 -- 172.21.15.120:0/3693287854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbc9d64680 msgr2=0x55dbc9d61740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:58.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.401+0000 7f5087eef740 1 --2- 172.21.15.120:0/3693287854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbc9d64680 0x55dbc9d61740 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5078003f20 tx=0x7f5078004ab0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.401+0000 7f5087eef740 1 -- 172.21.15.120:0/3693287854 shutdown_connections 2024-09-19T09:19:58.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.401+0000 7f5087eef740 1 --2- 172.21.15.120:0/3693287854 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dbc9d73cb0 0x55dbc9d760e0 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.401+0000 7f5087eef740 1 --2- 172.21.15.120:0/3693287854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbc9d64680 0x55dbc9d61740 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.401+0000 7f5087eef740 1 --2- 172.21.15.120:0/3693287854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbc9d59dc0 0x55dbc9d61200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.401+0000 7f5087eef740 1 --2- 172.21.15.120:0/3693287854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbc9d594a0 0x55dbc9d63f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.401+0000 7f5087eef740 1 -- 172.21.15.120:0/3693287854 >> 172.21.15.120:0/3693287854 conn(0x55dbc9d55a30 msgr2=0x55dbc9d54b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:58.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.402+0000 7f505effd640 0 client.29914 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:58.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.402+0000 7f5087eef740 1 -- 172.21.15.120:0/3693287854 shutdown_connections 2024-09-19T09:19:58.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.402+0000 7f5087eef740 1 -- 172.21.15.120:0/3693287854 wait complete. 2024-09-19T09:19:58.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.402+0000 7f5087eef740 20 client.29914 trim_cache size 0 max 16384 2024-09-19T09:19:58.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.424+0000 7f5087eef740 1 Processor -- start 2024-09-19T09:19:58.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.424+0000 7f5087eef740 1 -- start start 2024-09-19T09:19:58.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.424+0000 7f5087eef740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbc9d594a0 0x55dbc9d6ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:58.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.425+0000 7f5087eef740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbc9d59dc0 0x55dbc9d7b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:58.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.425+0000 7f5087eef740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbc9d64680 0x55dbc9d7bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:58.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.425+0000 7f5087eef740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f50801b4140 con 0x55dbc9d64680 2024-09-19T09:19:58.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.425+0000 7f5087eef740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f50801b5c80 con 0x55dbc9d594a0 2024-09-19T09:19:58.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.425+0000 7f5087eef740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f50801b4810 con 0x55dbc9d59dc0 2024-09-19T09:19:58.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.425+0000 7f507effd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbc9d59dc0 0x55dbc9d7b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:58.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.425+0000 7f507effd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbc9d59dc0 0x55dbc9d7b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:37572/0 (socket says 172.21.15.120:37572) 2024-09-19T09:19:58.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.425+0000 7f507effd640 1 -- 172.21.15.120:0/1221948890 learned_addr learned my addr 172.21.15.120:0/1221948890 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:58.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.425+0000 7f507f7fe640 1 --2- 172.21.15.120:0/1221948890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbc9d594a0 0x55dbc9d6ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:58.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.425+0000 7f507ffff640 1 --2- 172.21.15.120:0/1221948890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbc9d64680 0x55dbc9d7bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:58.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.425+0000 7f507effd640 1 -- 172.21.15.120:0/1221948890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbc9d594a0 msgr2=0x55dbc9d6ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:58.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.425+0000 7f507effd640 1 --2- 172.21.15.120:0/1221948890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbc9d594a0 0x55dbc9d6ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.425+0000 7f507effd640 1 -- 172.21.15.120:0/1221948890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbc9d64680 msgr2=0x55dbc9d7bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:58.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.425+0000 7f507effd640 1 --2- 172.21.15.120:0/1221948890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbc9d64680 0x55dbc9d7bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.425+0000 7f507effd640 1 -- 172.21.15.120:0/1221948890 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbc9d554d0 con 0x55dbc9d59dc0 2024-09-19T09:19:58.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.426+0000 7f507effd640 1 --2- 172.21.15.120:0/1221948890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbc9d59dc0 0x55dbc9d7b9f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f508010b2e0 tx=0x7f5074005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f27dc5caf495d88d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:58.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.426+0000 7f50657fa640 1 -- 172.21.15.120:0/1221948890 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5074022070 con 0x55dbc9d59dc0 2024-09-19T09:19:58.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.426+0000 7f50657fa640 1 -- 172.21.15.120:0/1221948890 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50740035f0 con 0x55dbc9d59dc0 2024-09-19T09:19:58.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.426+0000 7f50657fa640 1 -- 172.21.15.120:0/1221948890 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507401d6c0 con 0x55dbc9d59dc0 2024-09-19T09:19:58.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.427+0000 7f5087eef740 1 -- 172.21.15.120:0/1221948890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbc9d59dc0 msgr2=0x55dbc9d7b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:58.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.427+0000 7f5087eef740 1 --2- 172.21.15.120:0/1221948890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbc9d59dc0 0x55dbc9d7b9f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f508010b2e0 tx=0x7f5074005f90 comp rx=0 tx=0).stop 2024-09-19T09:19:58.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.427+0000 7f5087eef740 1 -- 172.21.15.120:0/1221948890 shutdown_connections 2024-09-19T09:19:58.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.427+0000 7f5087eef740 1 --2- 172.21.15.120:0/1221948890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbc9d64680 0x55dbc9d7bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.427+0000 7f5087eef740 1 --2- 172.21.15.120:0/1221948890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbc9d59dc0 0x55dbc9d7b9f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.427+0000 7f5087eef740 1 --2- 172.21.15.120:0/1221948890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbc9d594a0 0x55dbc9d6ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.427+0000 7f5087eef740 1 -- 172.21.15.120:0/1221948890 >> 172.21.15.120:0/1221948890 conn(0x55dbc9d55a30 msgr2=0x55dbc9d602f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:58.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.427+0000 7f5087eef740 1 -- 172.21.15.120:0/1221948890 shutdown_connections 2024-09-19T09:19:58.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.428+0000 7f5087eef740 1 -- 172.21.15.120:0/1221948890 wait complete. 2024-09-19T09:19:58.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.428+0000 7f5087eef740 1 -- start start 2024-09-19T09:19:58.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.429+0000 7f5087eef740 1 Processor -- start 2024-09-19T09:19:58.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.429+0000 7f5087eef740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbc9d594a0 0x55dbc9d67810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:58.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.429+0000 7f5087eef740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbc9d59dc0 0x55dbc9d67d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:58.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.429+0000 7f5087eef740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbc9d64680 0x55dbc9dd69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:58.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.429+0000 7f5087eef740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f50801b55b0 con 0x55dbc9d59dc0 2024-09-19T09:19:58.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.429+0000 7f5087eef740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f50801b4ee0 con 0x55dbc9d594a0 2024-09-19T09:19:58.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.429+0000 7f5087eef740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f50801b2100 con 0x55dbc9d64680 2024-09-19T09:19:58.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.429+0000 7f5087eef740 10 client.0 _ll_register_callbacks cb 0x7f5078012f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:19:58.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.429+0000 7f507f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbc9d594a0 0x55dbc9d67810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:58.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.429+0000 7f507f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbc9d594a0 0x55dbc9d67810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:53136/0 (socket says 172.21.15.120:53136) 2024-09-19T09:19:58.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.429+0000 7f507f7fe640 1 -- 172.21.15.120:0/4249242846 learned_addr learned my addr 172.21.15.120:0/4249242846 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:19:58.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.430+0000 7f507ffff640 1 --2- 172.21.15.120:0/4249242846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbc9d64680 0x55dbc9dd69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:58.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.430+0000 7f507f7fe640 1 -- 172.21.15.120:0/4249242846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbc9d64680 msgr2=0x55dbc9dd69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:58.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.430+0000 7f507f7fe640 1 --2- 172.21.15.120:0/4249242846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbc9d64680 0x55dbc9dd69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.430+0000 7f507f7fe640 1 -- 172.21.15.120:0/4249242846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbc9d59dc0 msgr2=0x55dbc9d67d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:19:58.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.430+0000 7f507f7fe640 1 --2- 172.21.15.120:0/4249242846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbc9d59dc0 0x55dbc9d67d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.430+0000 7f507f7fe640 1 -- 172.21.15.120:0/4249242846 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbc9dd6fe0 con 0x55dbc9d594a0 2024-09-19T09:19:58.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.430+0000 7f507f7fe640 1 --2- 172.21.15.120:0/4249242846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbc9d594a0 0x55dbc9d67810 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f50700b8b70 tx=0x7f5070005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=885bb6d45913acb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:58.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.430+0000 7f505effd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:19:58.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.430+0000 7f505effd640 1 -- 172.21.15.120:0/4249242846 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50700bbce0 con 0x55dbc9d594a0 2024-09-19T09:19:58.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.431+0000 7f505effd640 1 -- 172.21.15.120:0/4249242846 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f50700f37f0 con 0x55dbc9d594a0 2024-09-19T09:19:58.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.431+0000 7f5087eef740 1 -- 172.21.15.120:0/4249242846 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50740035f0 con 0x55dbc9d594a0 2024-09-19T09:19:58.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.431+0000 7f505effd640 1 -- 172.21.15.120:0/4249242846 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f50700bbf70 con 0x55dbc9d594a0 2024-09-19T09:19:58.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.431+0000 7f5087eef740 10 client.30104 fetch_fsmap learned FSMap version 39 2024-09-19T09:19:58.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.431+0000 7f5087eef740 1 -- 172.21.15.120:0/4249242846 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55dbc9dd6fe0 con 0x55dbc9d594a0 2024-09-19T09:19:58.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.431+0000 7f505effd640 1 -- 172.21.15.120:0/4249242846 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50740035f0 con 0x55dbc9d594a0 2024-09-19T09:19:58.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.431+0000 7f505effd640 1 -- 172.21.15.120:0/4249242846 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f507004c060 con 0x55dbc9d594a0 2024-09-19T09:19:58.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.432+0000 7f5087eef740 10 client.30104 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:19:58.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.432+0000 7f5087eef740 10 client.30104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:19:58.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.432+0000 7f5087eef740 20 client.30104 populate_metadata read hostname 'smithi120' 2024-09-19T09:19:58.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.432+0000 7f5087eef740 1 --2- 172.21.15.120:0/4249242846 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dbc9dde5d0 0x55dbc9de09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:58.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.432+0000 7f5087eef740 4 client.30104 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:19:58.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.432+0000 7f5087eef740 1 -- 172.21.15.120:0/4249242846 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dbc9cc6d60 con 0x55dbc9dde5d0 2024-09-19T09:19:58.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.432+0000 7f507effd640 1 --2- 172.21.15.120:0/4249242846 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dbc9dde5d0 0x55dbc9de09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:58.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.432+0000 7f505effd640 10 client.30104 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:58.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.432+0000 7f507effd640 1 --2- 172.21.15.120:0/4249242846 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dbc9dde5d0 0x55dbc9de09e0 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cca8f5e79eb2c00 server_cookie=a0a23e8ff4ec35f0 in_seq=0 out_seq=0 2024-09-19T09:19:58.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.434+0000 7f505effd640 1 -- 172.21.15.120:0/4249242846 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55dbc9cc6d60 con 0x55dbc9dde5d0 2024-09-19T09:19:58.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.434+0000 7f505effd640 10 client.30104 handle_command_reply: tid=0 2024-09-19T09:19:58.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.434+0000 7f5087eef740 1 client.30104 shutdown 2024-09-19T09:19:58.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.434+0000 7f5087eef740 1 -- 172.21.15.120:0/4249242846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbc9d594a0 msgr2=0x55dbc9d67810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:58.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.434+0000 7f5087eef740 1 --2- 172.21.15.120:0/4249242846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbc9d594a0 0x55dbc9d67810 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f50700b8b70 tx=0x7f5070005c90 comp rx=0 tx=0).stop 2024-09-19T09:19:58.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.435+0000 7f5087eef740 1 -- 172.21.15.120:0/4249242846 shutdown_connections 2024-09-19T09:19:58.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.435+0000 7f5087eef740 1 --2- 172.21.15.120:0/4249242846 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dbc9dde5d0 0x55dbc9de09e0 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.435+0000 7f505effd640 0 client.30104 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:19:58.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.435+0000 7f5087eef740 1 --2- 172.21.15.120:0/4249242846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbc9d64680 0x55dbc9dd69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.435+0000 7f5087eef740 1 --2- 172.21.15.120:0/4249242846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbc9d59dc0 0x55dbc9d67d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.435+0000 7f5087eef740 1 --2- 172.21.15.120:0/4249242846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbc9d594a0 0x55dbc9d67810 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.435+0000 7f5087eef740 1 -- 172.21.15.120:0/4249242846 >> 172.21.15.120:0/4249242846 conn(0x55dbc9d55a30 msgr2=0x55dbc9d62a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:58.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.435+0000 7f5087eef740 1 -- 172.21.15.120:0/4249242846 shutdown_connections 2024-09-19T09:19:58.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.435+0000 7f5087eef740 1 -- 172.21.15.120:0/4249242846 wait complete. 2024-09-19T09:19:58.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.435+0000 7f5087eef740 20 client.30104 trim_cache size 0 max 16384 2024-09-19T09:19:58.438 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:19:58.438 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:19:58.439 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-19T09:19:58.439 INFO:teuthology.orchestra.run.smithi120.stdout: "cf55f3fb-e778-4b8b-9a8c-d2834cc2cbb0": { 2024-09-19T09:19:58.439 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-19T09:19:58.439 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "cf55f3fb-e778-4b8b-9a8c-d2834cc2cbb0", 2024-09-19T09:19:58.439 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-19T09:19:58.439 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:19:58.439 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:19:58.439 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:19:58.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.436+0000 7f5085b16640 1 -- 172.21.15.120:0/566072437 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f50680a3410 msgr2=0x7f50680a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:58.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.436+0000 7f5085b16640 1 --2- 172.21.15.120:0/566072437 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f50680a3410 0x7f50680a58d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5078005fd0 tx=0x7f5078009500 comp rx=0 tx=0).stop 2024-09-19T09:19:58.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.436+0000 7f5085b16640 1 -- 172.21.15.120:0/566072437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5080102570 msgr2=0x7f508010d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:58.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.436+0000 7f5085b16640 1 --2- 172.21.15.120:0/566072437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5080102570 0x7f508010d9f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f507002aa70 tx=0x7f5070003c10 comp rx=0 tx=0).stop 2024-09-19T09:19:58.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.436+0000 7f5085b16640 1 -- 172.21.15.120:0/566072437 shutdown_connections 2024-09-19T09:19:58.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.436+0000 7f5085b16640 1 --2- 172.21.15.120:0/566072437 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f50680a3410 0x7f50680a58d0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.436+0000 7f5085b16640 1 --2- 172.21.15.120:0/566072437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f508010a0e0 0x7f50801ab000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.436+0000 7f5085b16640 1 --2- 172.21.15.120:0/566072437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5080102ef0 0x7f5080109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:58.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.436+0000 7f5085b16640 1 --2- 172.21.15.120:0/566072437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5080102570 0x7f508010d9f0 secure :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f507002aa70 tx=0x7f5070003c10 comp rx=0 tx=0).stop 2024-09-19T09:19:58.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.436+0000 7f5085b16640 1 -- 172.21.15.120:0/566072437 >> 172.21.15.120:0/566072437 conn(0x7f50800fc7d0 msgr2=0x7f50800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:58.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.436+0000 7f5085b16640 1 -- 172.21.15.120:0/566072437 shutdown_connections 2024-09-19T09:19:58.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:19:58.436+0000 7f5085b16640 1 -- 172.21.15.120:0/566072437 wait complete. 2024-09-19T09:19:58.442 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:57 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:19:58.442 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:57 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:58.442 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:57 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:19:58.442 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:57 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:19:58.442 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:57 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:58.442 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:57 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:19:58.442 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:57 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:58.442 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:57 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:19:58.442 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:57 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:58.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:57 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:19:58.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:57 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:58.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:57 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:19:58.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:57 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:19:58.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:57 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:58.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:57 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:19:58.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:57 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:58.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:57 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:19:58.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:57 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:59.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:58 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:59.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:58 smithi120 ceph-mon[31150]: pgmap v858: 97 pgs: 97 active+clean; 352 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.7 MiB/s wr, 695 op/s 2024-09-19T09:19:59.365 INFO:tasks.workunit.client.0.smithi120.stdout: 10 92765 17.46 MB/sec execute 114 sec latency 158.700 ms 2024-09-19T09:19:59.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:58 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:59.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:58 smithi196 ceph-mon[35150]: pgmap v858: 97 pgs: 97 active+clean; 352 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.7 MiB/s wr, 695 op/s 2024-09-19T09:19:59.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:58 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:19:59.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:58 smithi153 ceph-mon[35071]: pgmap v858: 97 pgs: 97 active+clean; 352 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.7 MiB/s wr, 695 op/s 2024-09-19T09:19:59.627 INFO:tasks.cephfs.filesystem:scrub status for tag:cf55f3fb-e778-4b8b-9a8c-d2834cc2cbb0 - {'path': '/', 'tag': 'cf55f3fb-e778-4b8b-9a8c-d2834cc2cbb0', 'options': 'recursive,force'} 2024-09-19T09:20:00.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:19:59 smithi120 ceph-mon[31150]: pgmap v859: 97 pgs: 97 active+clean; 355 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.2 MiB/s wr, 810 op/s 2024-09-19T09:20:00.365 INFO:tasks.workunit.client.0.smithi120.stdout: 10 93274 17.56 MB/sec execute 115 sec latency 176.072 ms 2024-09-19T09:20:00.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:19:59 smithi196 ceph-mon[35150]: pgmap v859: 97 pgs: 97 active+clean; 355 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.2 MiB/s wr, 810 op/s 2024-09-19T09:20:00.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:19:59 smithi153 ceph-mon[35071]: pgmap v859: 97 pgs: 97 active+clean; 355 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.2 MiB/s wr, 810 op/s 2024-09-19T09:20:01.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:00 smithi120 ceph-mon[31150]: overall HEALTH_OK 2024-09-19T09:20:01.365 INFO:tasks.workunit.client.0.smithi120.stdout: 10 94043 17.71 MB/sec execute 116 sec latency 77.182 ms 2024-09-19T09:20:01.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:00 smithi196 ceph-mon[35150]: overall HEALTH_OK 2024-09-19T09:20:01.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:00 smithi153 ceph-mon[35071]: overall HEALTH_OK 2024-09-19T09:20:02.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:01 smithi120 ceph-mon[31150]: pgmap v860: 97 pgs: 97 active+clean; 355 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 8.1 MiB/s wr, 727 op/s 2024-09-19T09:20:02.365 INFO:tasks.workunit.client.0.smithi120.stdout: 10 94816 17.89 MB/sec execute 117 sec latency 50.621 ms 2024-09-19T09:20:02.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:01 smithi196 ceph-mon[35150]: pgmap v860: 97 pgs: 97 active+clean; 355 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 8.1 MiB/s wr, 727 op/s 2024-09-19T09:20:02.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:01 smithi153 ceph-mon[35071]: pgmap v860: 97 pgs: 97 active+clean; 355 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 8.1 MiB/s wr, 727 op/s 2024-09-19T09:20:03.365 INFO:tasks.workunit.client.0.smithi120.stdout: 10 95368 17.97 MB/sec execute 118 sec latency 235.262 ms 2024-09-19T09:20:03.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:03 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:20:03.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:03 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:20:03.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:03 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:20:03.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:03 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:20:03.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:03 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:20:03.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:03 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:20:04.365 INFO:tasks.workunit.client.0.smithi120.stdout: 10 96022 18.18 MB/sec execute 119 sec latency 120.975 ms 2024-09-19T09:20:04.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:04 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:20:04.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:04 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:20:04.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:04 smithi196 ceph-mon[35150]: pgmap v861: 97 pgs: 97 active+clean; 362 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.4 MiB/s wr, 845 op/s 2024-09-19T09:20:04.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:04 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:20:04.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:04 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:20:04.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:04 smithi153 ceph-mon[35071]: pgmap v861: 97 pgs: 97 active+clean; 362 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.4 MiB/s wr, 845 op/s 2024-09-19T09:20:04.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:04 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:20:04.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:04 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:20:04.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:04 smithi120 ceph-mon[31150]: pgmap v861: 97 pgs: 97 active+clean; 362 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.4 MiB/s wr, 845 op/s 2024-09-19T09:20:05.365 INFO:tasks.workunit.client.0.smithi120.stdout: 10 96732 18.43 MB/sec execute 120 sec latency 56.087 ms 2024-09-19T09:20:06.365 INFO:tasks.workunit.client.0.smithi120.stdout: 10 97290 18.47 MB/sec execute 121 sec latency 61.144 ms 2024-09-19T09:20:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.021+0000 7fad463c0640 1 -- 172.21.15.120:0/204885540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad401025f0 msgr2=0x7fad40108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.021+0000 7fad463c0640 1 --2- 172.21.15.120:0/204885540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad401025f0 0x7fad40108e80 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fad38009ca0 tx=0x7fad3802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.021+0000 7fad463c0640 1 -- 172.21.15.120:0/204885540 shutdown_connections 2024-09-19T09:20:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.021+0000 7fad463c0640 1 --2- 172.21.15.120:0/204885540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad401025f0 0x7fad40108e80 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.021+0000 7fad463c0640 1 --2- 172.21.15.120:0/204885540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad40101c30 0x7fad401020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.021+0000 7fad463c0640 1 --2- 172.21.15.120:0/204885540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad40100a40 0x7fad40100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.021+0000 7fad463c0640 1 -- 172.21.15.120:0/204885540 >> 172.21.15.120:0/204885540 conn(0x7fad400fc1e0 msgr2=0x7fad400fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.021+0000 7fad463c0640 1 -- 172.21.15.120:0/204885540 shutdown_connections 2024-09-19T09:20:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.021+0000 7fad463c0640 1 -- 172.21.15.120:0/204885540 wait complete. 2024-09-19T09:20:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.022+0000 7fad463c0640 1 Processor -- start 2024-09-19T09:20:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.022+0000 7fad463c0640 1 -- start start 2024-09-19T09:20:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.022+0000 7fad463c0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad40100a40 0x7fad4006a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.022+0000 7fad463c0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad40101c30 0x7fad4019cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.022+0000 7fad463c0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad401025f0 0x7fad4019f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.022+0000 7fad463c0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fad4010c6a0 con 0x7fad401025f0 2024-09-19T09:20:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.022+0000 7fad463c0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fad4010c520 con 0x7fad40100a40 2024-09-19T09:20:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.022+0000 7fad463c0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fad4010c820 con 0x7fad40101c30 2024-09-19T09:20:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.022+0000 7fad4495d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad401025f0 0x7fad4019f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.022+0000 7fad4495d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad401025f0 0x7fad4019f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47722/0 (socket says 172.21.15.120:47722) 2024-09-19T09:20:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.022+0000 7fad4495d640 1 -- 172.21.15.120:0/4195308438 learned_addr learned my addr 172.21.15.120:0/4195308438 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.022+0000 7fad3ffff640 1 --2- 172.21.15.120:0/4195308438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad40100a40 0x7fad4006a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.022+0000 7fad3f7fe640 1 --2- 172.21.15.120:0/4195308438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad40101c30 0x7fad4019cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.023+0000 7fad4495d640 1 -- 172.21.15.120:0/4195308438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad40101c30 msgr2=0x7fad4019cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.023+0000 7fad4495d640 1 --2- 172.21.15.120:0/4195308438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad40101c30 0x7fad4019cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.023+0000 7fad4495d640 1 -- 172.21.15.120:0/4195308438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad40100a40 msgr2=0x7fad4006a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.023+0000 7fad4495d640 1 --2- 172.21.15.120:0/4195308438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad40100a40 0x7fad4006a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.023+0000 7fad4495d640 1 -- 172.21.15.120:0/4195308438 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4019f9e0 con 0x7fad401025f0 2024-09-19T09:20:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.023+0000 7fad4495d640 1 --2- 172.21.15.120:0/4195308438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad401025f0 0x7fad4019f0d0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fad38031ae0 tx=0x7fad38031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.023+0000 7fad3d7fa640 1 -- 172.21.15.120:0/4195308438 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad38003c30 con 0x7fad401025f0 2024-09-19T09:20:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.023+0000 7fad3d7fa640 1 -- 172.21.15.120:0/4195308438 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad38003dd0 con 0x7fad401025f0 2024-09-19T09:20:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.023+0000 7fad463c0640 1 -- 172.21.15.120:0/4195308438 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad4019fc10 con 0x7fad401025f0 2024-09-19T09:20:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.023+0000 7fad3d7fa640 1 -- 172.21.15.120:0/4195308438 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3802fcb0 con 0x7fad401025f0 2024-09-19T09:20:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.023+0000 7fad463c0640 1 -- 172.21.15.120:0/4195308438 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad401a0160 con 0x7fad401025f0 2024-09-19T09:20:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.025+0000 7fad3d7fa640 1 -- 172.21.15.120:0/4195308438 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fad38004050 con 0x7fad401025f0 2024-09-19T09:20:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.026+0000 7fad4922f740 1 Processor -- start 2024-09-19T09:20:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.026+0000 7fad4922f740 1 -- start start 2024-09-19T09:20:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.026+0000 7fad4922f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c4e781db0 0x561c4e7821b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.026+0000 7fad4922f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c4e7826f0 0x561c4e7871b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.026+0000 7fad4922f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4e787780 0x561c4e789c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.026+0000 7fad4922f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561c4e6e12f0 con 0x561c4e7826f0 2024-09-19T09:20:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.026+0000 7fad4922f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561c4e689270 con 0x561c4e781db0 2024-09-19T09:20:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.026+0000 7fad4922f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561c4e6dd2b0 con 0x561c4e787780 2024-09-19T09:20:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.026+0000 7fad3ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c4e7826f0 0x561c4e7871b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.026+0000 7fad3ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c4e7826f0 0x561c4e7871b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47738/0 (socket says 172.21.15.120:47738) 2024-09-19T09:20:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.026+0000 7fad3ffff640 1 -- 172.21.15.120:0/3505505133 learned_addr learned my addr 172.21.15.120:0/3505505133 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.026+0000 7fad3f7fe640 1 --2- 172.21.15.120:0/3505505133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c4e781db0 0x561c4e7821b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.026+0000 7fad3d7fa640 1 --2- 172.21.15.120:0/4195308438 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fad280a32f0 0x7fad280a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.026+0000 7fad3f7fe640 1 --2- 172.21.15.120:0/3505505133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4e787780 0x561c4e789c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.026+0000 7fad4495d640 1 --2- 172.21.15.120:0/4195308438 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fad280a32f0 0x7fad280a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.026+0000 7fad3ffff640 1 -- 172.21.15.120:0/3505505133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4e787780 msgr2=0x561c4e789c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.026+0000 7fad3ffff640 1 --2- 172.21.15.120:0/3505505133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4e787780 0x561c4e789c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.026+0000 7fad3ffff640 1 -- 172.21.15.120:0/3505505133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c4e781db0 msgr2=0x561c4e7821b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.026+0000 7fad3ffff640 1 --2- 172.21.15.120:0/3505505133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c4e781db0 0x561c4e7821b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.026+0000 7fad3ffff640 1 -- 172.21.15.120:0/3505505133 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c4e688340 con 0x561c4e7826f0 2024-09-19T09:20:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.026+0000 7fad3d7fa640 1 -- 172.21.15.120:0/4195308438 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fad380f0b30 con 0x7fad401025f0 2024-09-19T09:20:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.026+0000 7fad3ffff640 1 --2- 172.21.15.120:0/3505505133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c4e7826f0 0x561c4e7871b0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fad30009ca0 tx=0x7fad30011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7812dd57dcb9c6d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.027+0000 7fad267fc640 1 -- 172.21.15.120:0/3505505133 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad30004220 con 0x561c4e7826f0 2024-09-19T09:20:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.027+0000 7fad267fc640 1 -- 172.21.15.120:0/3505505133 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad300043c0 con 0x561c4e7826f0 2024-09-19T09:20:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.027+0000 7fad4495d640 1 --2- 172.21.15.120:0/4195308438 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fad280a32f0 0x7fad280a57b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fad380b9b20 tx=0x7fad380e7f20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.027+0000 7fad267fc640 1 -- 172.21.15.120:0/3505505133 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad30019870 con 0x561c4e7826f0 2024-09-19T09:20:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.027+0000 7fad4922f740 1 -- 172.21.15.120:0/3505505133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c4e7826f0 msgr2=0x561c4e7871b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.027+0000 7fad4922f740 1 --2- 172.21.15.120:0/3505505133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c4e7826f0 0x561c4e7871b0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fad30009ca0 tx=0x7fad30011810 comp rx=0 tx=0).stop 2024-09-19T09:20:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.027+0000 7fad4922f740 1 -- 172.21.15.120:0/3505505133 shutdown_connections 2024-09-19T09:20:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.027+0000 7fad4922f740 1 --2- 172.21.15.120:0/3505505133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4e787780 0x561c4e789c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.027+0000 7fad4922f740 1 --2- 172.21.15.120:0/3505505133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c4e7826f0 0x561c4e7871b0 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.027+0000 7fad4922f740 1 --2- 172.21.15.120:0/3505505133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c4e781db0 0x561c4e7821b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.027+0000 7fad4922f740 1 -- 172.21.15.120:0/3505505133 >> 172.21.15.120:0/3505505133 conn(0x561c4e779010 msgr2=0x561c4e779410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.027+0000 7fad4922f740 1 -- 172.21.15.120:0/3505505133 shutdown_connections 2024-09-19T09:20:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.027+0000 7fad4922f740 1 -- 172.21.15.120:0/3505505133 wait complete. 2024-09-19T09:20:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.028+0000 7fad4922f740 1 -- start start 2024-09-19T09:20:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.028+0000 7fad4922f740 1 Processor -- start 2024-09-19T09:20:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.028+0000 7fad4922f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c4e781db0 0x561c4e78bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.028+0000 7fad4922f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4e7826f0 0x561c4e78c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.028+0000 7fad4922f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c4e787780 0x561c4e7934f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.028+0000 7fad4922f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561c4e78a750 con 0x561c4e781db0 2024-09-19T09:20:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.028+0000 7fad4922f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561c4e66a290 con 0x561c4e787780 2024-09-19T09:20:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.028+0000 7fad4922f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561c4e78a8d0 con 0x561c4e7826f0 2024-09-19T09:20:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.028+0000 7fad3f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c4e781db0 0x561c4e78bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.028+0000 7fad3f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c4e781db0 0x561c4e78bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47748/0 (socket says 172.21.15.120:47748) 2024-09-19T09:20:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.028+0000 7fad3f7fe640 1 -- 172.21.15.120:0/3960576454 learned_addr learned my addr 172.21.15.120:0/3960576454 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.028+0000 7fad4922f740 10 client.0 _ll_register_callbacks cb 0x561c4e69b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:20:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.029+0000 7fad3ffff640 1 --2- 172.21.15.120:0/3960576454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4e7826f0 0x561c4e78c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.029+0000 7fad3f7fe640 1 --2- 172.21.15.120:0/3960576454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c4e787780 0x561c4e7934f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.029+0000 7fad3f7fe640 1 -- 172.21.15.120:0/3960576454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4e7826f0 msgr2=0x561c4e78c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.029+0000 7fad3f7fe640 1 --2- 172.21.15.120:0/3960576454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4e7826f0 0x561c4e78c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.029+0000 7fad3f7fe640 1 -- 172.21.15.120:0/3960576454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c4e787780 msgr2=0x561c4e7934f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.029+0000 7fad3f7fe640 1 --2- 172.21.15.120:0/3960576454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c4e787780 0x561c4e7934f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.029+0000 7fad3f7fe640 1 -- 172.21.15.120:0/3960576454 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c4e3f4210 con 0x561c4e781db0 2024-09-19T09:20:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.029+0000 7fad3f7fe640 1 --2- 172.21.15.120:0/3960576454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c4e781db0 0x561c4e78bf70 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fad34003640 tx=0x7fad34004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d948c8a40db95ac6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.029+0000 7fad1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:20:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.029+0000 7fad1f7fe640 1 -- 172.21.15.120:0/3960576454 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad340128b0 con 0x561c4e781db0 2024-09-19T09:20:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.029+0000 7fad1f7fe640 1 -- 172.21.15.120:0/3960576454 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fad34012a50 con 0x561c4e781db0 2024-09-19T09:20:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.029+0000 7fad4922f740 1 -- 172.21.15.120:0/3960576454 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad300043c0 con 0x561c4e781db0 2024-09-19T09:20:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.029+0000 7fad1f7fe640 1 -- 172.21.15.120:0/3960576454 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fad34012c50 con 0x561c4e781db0 2024-09-19T09:20:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.029+0000 7fad4922f740 10 client.21252 fetch_fsmap learned FSMap version 39 2024-09-19T09:20:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.029+0000 7fad4922f740 1 -- 172.21.15.120:0/3960576454 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561c4e4c0620 con 0x561c4e781db0 2024-09-19T09:20:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.029+0000 7fad1f7fe640 1 -- 172.21.15.120:0/3960576454 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad300043c0 con 0x561c4e781db0 2024-09-19T09:20:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.030+0000 7fad1f7fe640 1 -- 172.21.15.120:0/3960576454 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fad34004b10 con 0x561c4e781db0 2024-09-19T09:20:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.030+0000 7fad4922f740 10 client.21252 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:20:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.030+0000 7fad4922f740 10 client.21252 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:20:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.030+0000 7fad4922f740 20 client.21252 populate_metadata read hostname 'smithi120' 2024-09-19T09:20:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.030+0000 7fad4922f740 1 --2- 172.21.15.120:0/3960576454 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561c4e797c00 0x561c4e79a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.030+0000 7fad4922f740 4 client.21252 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:20:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.030+0000 7fad4922f740 1 -- 172.21.15.120:0/3960576454 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c4e6eea20 con 0x561c4e797c00 2024-09-19T09:20:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.030+0000 7fad3ffff640 1 --2- 172.21.15.120:0/3960576454 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561c4e797c00 0x561c4e79a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.030+0000 7fad3ffff640 1 --2- 172.21.15.120:0/3960576454 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561c4e797c00 0x561c4e79a090 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fa58e39bda4e802 server_cookie=7062d4d38f7b7b29 in_seq=0 out_seq=0 2024-09-19T09:20:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.030+0000 7fad1f7fe640 10 client.21252 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.033+0000 7fad1f7fe640 1 -- 172.21.15.120:0/3960576454 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c4e6eea20 con 0x561c4e797c00 2024-09-19T09:20:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.033+0000 7fad1f7fe640 10 client.21252 handle_command_reply: tid=0 2024-09-19T09:20:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.033+0000 7fad4922f740 1 client.21252 shutdown 2024-09-19T09:20:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.033+0000 7fad4922f740 1 -- 172.21.15.120:0/3960576454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c4e781db0 msgr2=0x561c4e78bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.033+0000 7fad4922f740 1 --2- 172.21.15.120:0/3960576454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c4e781db0 0x561c4e78bf70 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fad34003640 tx=0x7fad34004730 comp rx=0 tx=0).stop 2024-09-19T09:20:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.033+0000 7fad4922f740 1 -- 172.21.15.120:0/3960576454 shutdown_connections 2024-09-19T09:20:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.033+0000 7fad4922f740 1 --2- 172.21.15.120:0/3960576454 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561c4e797c00 0x561c4e79a090 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.033+0000 7fad4922f740 1 --2- 172.21.15.120:0/3960576454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c4e787780 0x561c4e7934f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.033+0000 7fad4922f740 1 --2- 172.21.15.120:0/3960576454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4e7826f0 0x561c4e78c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.033+0000 7fad4922f740 1 --2- 172.21.15.120:0/3960576454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c4e781db0 0x561c4e78bf70 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.033+0000 7fad1f7fe640 0 client.21252 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.033+0000 7fad4922f740 1 -- 172.21.15.120:0/3960576454 >> 172.21.15.120:0/3960576454 conn(0x561c4e779010 msgr2=0x561c4e77a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.033+0000 7fad4922f740 1 -- 172.21.15.120:0/3960576454 shutdown_connections 2024-09-19T09:20:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.033+0000 7fad4922f740 1 -- 172.21.15.120:0/3960576454 wait complete. 2024-09-19T09:20:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.034+0000 7fad4922f740 20 client.21252 trim_cache size 0 max 16384 2024-09-19T09:20:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.049+0000 7fad4922f740 1 Processor -- start 2024-09-19T09:20:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.049+0000 7fad4922f740 1 -- start start 2024-09-19T09:20:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.049+0000 7fad4922f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c4e781db0 0x561c4e779ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.049+0000 7fad4922f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c4e7826f0 0x561c4e7934b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.049+0000 7fad4922f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4e787780 0x561c4e77a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.049+0000 7fad4922f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fad401a6e00 con 0x561c4e7826f0 2024-09-19T09:20:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.049+0000 7fad4922f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fad401a8940 con 0x561c4e781db0 2024-09-19T09:20:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.049+0000 7fad4922f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fad401a74d0 con 0x561c4e787780 2024-09-19T09:20:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.050+0000 7fad3ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c4e7826f0 0x561c4e7934b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.050+0000 7fad3ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c4e7826f0 0x561c4e7934b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47754/0 (socket says 172.21.15.120:47754) 2024-09-19T09:20:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.050+0000 7fad3ffff640 1 -- 172.21.15.120:0/1198058841 learned_addr learned my addr 172.21.15.120:0/1198058841 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.050+0000 7fad3f7fe640 1 --2- 172.21.15.120:0/1198058841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c4e781db0 0x561c4e779ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.050+0000 7fad3f7fe640 1 --2- 172.21.15.120:0/1198058841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4e787780 0x561c4e77a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.050+0000 7fad3ffff640 1 -- 172.21.15.120:0/1198058841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4e787780 msgr2=0x561c4e77a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.050+0000 7fad3ffff640 1 --2- 172.21.15.120:0/1198058841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4e787780 0x561c4e77a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.050+0000 7fad3ffff640 1 -- 172.21.15.120:0/1198058841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c4e781db0 msgr2=0x561c4e779ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.050+0000 7fad3ffff640 1 --2- 172.21.15.120:0/1198058841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c4e781db0 0x561c4e779ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.050+0000 7fad3ffff640 1 -- 172.21.15.120:0/1198058841 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c4e78bab0 con 0x561c4e7826f0 2024-09-19T09:20:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.050+0000 7fad3f7fe640 1 --2- 172.21.15.120:0/1198058841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4e787780 0x561c4e77a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:20:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.050+0000 7fad3ffff640 1 --2- 172.21.15.120:0/1198058841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c4e7826f0 0x561c4e7934b0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fad30011d10 tx=0x7fad30005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1558fa0db3adc999 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.050+0000 7fad25ffb640 1 -- 172.21.15.120:0/1198058841 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad300035f0 con 0x561c4e7826f0 2024-09-19T09:20:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.050+0000 7fad25ffb640 1 -- 172.21.15.120:0/1198058841 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad30005040 con 0x561c4e7826f0 2024-09-19T09:20:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.050+0000 7fad25ffb640 1 -- 172.21.15.120:0/1198058841 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3002c340 con 0x561c4e7826f0 2024-09-19T09:20:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.051+0000 7fad4922f740 1 -- 172.21.15.120:0/1198058841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c4e7826f0 msgr2=0x561c4e7934b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.051+0000 7fad4922f740 1 --2- 172.21.15.120:0/1198058841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c4e7826f0 0x561c4e7934b0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fad30011d10 tx=0x7fad30005f90 comp rx=0 tx=0).stop 2024-09-19T09:20:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.051+0000 7fad4922f740 1 -- 172.21.15.120:0/1198058841 shutdown_connections 2024-09-19T09:20:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.051+0000 7fad4922f740 1 --2- 172.21.15.120:0/1198058841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4e787780 0x561c4e77a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.051+0000 7fad4922f740 1 --2- 172.21.15.120:0/1198058841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c4e7826f0 0x561c4e7934b0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.051+0000 7fad4922f740 1 --2- 172.21.15.120:0/1198058841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c4e781db0 0x561c4e779ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.051+0000 7fad4922f740 1 -- 172.21.15.120:0/1198058841 >> 172.21.15.120:0/1198058841 conn(0x561c4e779010 msgr2=0x561c4e78bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.051+0000 7fad4922f740 1 -- 172.21.15.120:0/1198058841 shutdown_connections 2024-09-19T09:20:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.051+0000 7fad4922f740 1 -- 172.21.15.120:0/1198058841 wait complete. 2024-09-19T09:20:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.051+0000 7fad4922f740 1 -- start start 2024-09-19T09:20:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.051+0000 7fad4922f740 1 Processor -- start 2024-09-19T09:20:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.052+0000 7fad4922f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c4e781db0 0x561c4e790510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.052+0000 7fad4922f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c4e7826f0 0x561c4e790a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.052+0000 7fad4922f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4e787780 0x561c4e791a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.052+0000 7fad4922f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fad401a8270 con 0x561c4e781db0 2024-09-19T09:20:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.052+0000 7fad4922f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fad401a7ba0 con 0x561c4e7826f0 2024-09-19T09:20:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.052+0000 7fad4922f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fad401a4dc0 con 0x561c4e787780 2024-09-19T09:20:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.052+0000 7fad4922f740 10 client.0 _ll_register_callbacks cb 0x7fad340042e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:20:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.052+0000 7fad3f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c4e781db0 0x561c4e790510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.052+0000 7fad3f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c4e781db0 0x561c4e790510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47756/0 (socket says 172.21.15.120:47756) 2024-09-19T09:20:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.052+0000 7fad3f7fe640 1 -- 172.21.15.120:0/2161141349 learned_addr learned my addr 172.21.15.120:0/2161141349 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.052+0000 7fad3ffff640 1 --2- 172.21.15.120:0/2161141349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c4e7826f0 0x561c4e790a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.052+0000 7fad3f7fe640 1 --2- 172.21.15.120:0/2161141349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4e787780 0x561c4e791a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.052+0000 7fad3f7fe640 1 -- 172.21.15.120:0/2161141349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4e787780 msgr2=0x561c4e791a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.052+0000 7fad3f7fe640 1 --2- 172.21.15.120:0/2161141349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4e787780 0x561c4e791a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.052+0000 7fad3f7fe640 1 -- 172.21.15.120:0/2161141349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c4e7826f0 msgr2=0x561c4e790a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.052+0000 7fad3f7fe640 1 --2- 172.21.15.120:0/2161141349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c4e7826f0 0x561c4e790a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.052+0000 7fad3f7fe640 1 -- 172.21.15.120:0/2161141349 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c4e792070 con 0x561c4e781db0 2024-09-19T09:20:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.052+0000 7fad3f7fe640 1 --2- 172.21.15.120:0/2161141349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c4e781db0 0x561c4e790510 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fad340043f0 tx=0x7fad3400bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a535b4af3cfe7e3d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.052+0000 7fad1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:20:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.052+0000 7fad1f7fe640 1 -- 172.21.15.120:0/2161141349 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34016070 con 0x561c4e781db0 2024-09-19T09:20:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.052+0000 7fad1f7fe640 1 -- 172.21.15.120:0/2161141349 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fad34021940 con 0x561c4e781db0 2024-09-19T09:20:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.052+0000 7fad4922f740 1 -- 172.21.15.120:0/2161141349 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad30005040 con 0x561c4e781db0 2024-09-19T09:20:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.052+0000 7fad1f7fe640 1 -- 172.21.15.120:0/2161141349 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fad34040050 con 0x561c4e781db0 2024-09-19T09:20:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.052+0000 7fad4922f740 10 client.21258 fetch_fsmap learned FSMap version 39 2024-09-19T09:20:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.052+0000 7fad4922f740 1 -- 172.21.15.120:0/2161141349 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561c4e793360 con 0x561c4e781db0 2024-09-19T09:20:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.053+0000 7fad1f7fe640 1 -- 172.21.15.120:0/2161141349 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad30005040 con 0x561c4e781db0 2024-09-19T09:20:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.053+0000 7fad1f7fe640 1 -- 172.21.15.120:0/2161141349 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fad34004f90 con 0x561c4e781db0 2024-09-19T09:20:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.053+0000 7fad4922f740 10 client.21258 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:20:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.053+0000 7fad4922f740 10 client.21258 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:20:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.053+0000 7fad4922f740 20 client.21258 populate_metadata read hostname 'smithi120' 2024-09-19T09:20:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.053+0000 7fad4922f740 1 --2- 172.21.15.120:0/2161141349 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561c4e802c20 0x561c4e805010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.053+0000 7fad4922f740 4 client.21258 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:20:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.053+0000 7fad4922f740 1 -- 172.21.15.120:0/2161141349 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561c4e6e8d70 con 0x561c4e802c20 2024-09-19T09:20:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.053+0000 7fad3ffff640 1 --2- 172.21.15.120:0/2161141349 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561c4e802c20 0x561c4e805010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.053+0000 7fad1f7fe640 10 client.21258 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.053+0000 7fad3ffff640 1 --2- 172.21.15.120:0/2161141349 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561c4e802c20 0x561c4e805010 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b8917bbbcc7853f server_cookie=24ab9615e7670ba0 in_seq=0 out_seq=0 2024-09-19T09:20:07.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:06 smithi120 ceph-mon[31150]: pgmap v862: 97 pgs: 97 active+clean; 361 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.1 MiB/s wr, 754 op/s 2024-09-19T09:20:07.365 INFO:tasks.workunit.client.0.smithi120.stdout: 10 97684 18.50 MB/sec execute 122 sec latency 206.120 ms 2024-09-19T09:20:07.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:06 smithi196 ceph-mon[35150]: pgmap v862: 97 pgs: 97 active+clean; 361 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.1 MiB/s wr, 754 op/s 2024-09-19T09:20:07.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:06 smithi153 ceph-mon[35071]: pgmap v862: 97 pgs: 97 active+clean; 361 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.1 MiB/s wr, 754 op/s 2024-09-19T09:20:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.595+0000 7fad1f7fe640 1 -- 172.21.15.120:0/2161141349 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561c4e6e8d70 con 0x561c4e802c20 2024-09-19T09:20:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.595+0000 7fad1f7fe640 10 client.21258 handle_command_reply: tid=0 2024-09-19T09:20:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.595+0000 7fad4922f740 1 client.21258 shutdown 2024-09-19T09:20:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.595+0000 7fad4922f740 1 -- 172.21.15.120:0/2161141349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c4e781db0 msgr2=0x561c4e790510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.595+0000 7fad4922f740 1 --2- 172.21.15.120:0/2161141349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c4e781db0 0x561c4e790510 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fad340043f0 tx=0x7fad3400bb50 comp rx=0 tx=0).stop 2024-09-19T09:20:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.595+0000 7fad4922f740 1 -- 172.21.15.120:0/2161141349 shutdown_connections 2024-09-19T09:20:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.595+0000 7fad4922f740 1 --2- 172.21.15.120:0/2161141349 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561c4e802c20 0x561c4e805010 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.595+0000 7fad4922f740 1 --2- 172.21.15.120:0/2161141349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4e787780 0x561c4e791a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.595+0000 7fad4922f740 1 --2- 172.21.15.120:0/2161141349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c4e7826f0 0x561c4e790a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.595+0000 7fad4922f740 1 --2- 172.21.15.120:0/2161141349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c4e781db0 0x561c4e790510 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.595+0000 7fad1f7fe640 0 client.21258 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.595+0000 7fad4922f740 1 -- 172.21.15.120:0/2161141349 >> 172.21.15.120:0/2161141349 conn(0x561c4e779010 msgr2=0x561c4e7779a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.595+0000 7fad4922f740 1 -- 172.21.15.120:0/2161141349 shutdown_connections 2024-09-19T09:20:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.595+0000 7fad4922f740 1 -- 172.21.15.120:0/2161141349 wait complete. 2024-09-19T09:20:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.595+0000 7fad4922f740 20 client.21258 trim_cache size 0 max 16384 2024-09-19T09:20:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:20:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:20:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:20:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:20:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.595+0000 7fad463c0640 1 -- 172.21.15.120:0/4195308438 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fad280a32f0 msgr2=0x7fad280a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.596+0000 7fad463c0640 1 --2- 172.21.15.120:0/4195308438 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fad280a32f0 0x7fad280a57b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fad380b9b20 tx=0x7fad380e7f20 comp rx=0 tx=0).stop 2024-09-19T09:20:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.596+0000 7fad463c0640 1 -- 172.21.15.120:0/4195308438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad401025f0 msgr2=0x7fad4019f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.596+0000 7fad463c0640 1 --2- 172.21.15.120:0/4195308438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad401025f0 0x7fad4019f0d0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fad38031ae0 tx=0x7fad38031ee0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.596+0000 7fad463c0640 1 -- 172.21.15.120:0/4195308438 shutdown_connections 2024-09-19T09:20:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.596+0000 7fad463c0640 1 --2- 172.21.15.120:0/4195308438 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fad280a32f0 0x7fad280a57b0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.596+0000 7fad463c0640 1 --2- 172.21.15.120:0/4195308438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad401025f0 0x7fad4019f0d0 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.596+0000 7fad463c0640 1 --2- 172.21.15.120:0/4195308438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad40101c30 0x7fad4019cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.596+0000 7fad463c0640 1 --2- 172.21.15.120:0/4195308438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad40100a40 0x7fad4006a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:07.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.596+0000 7fad463c0640 1 -- 172.21.15.120:0/4195308438 >> 172.21.15.120:0/4195308438 conn(0x7fad400fc1e0 msgr2=0x7fad40104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:07.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.596+0000 7fad463c0640 1 -- 172.21.15.120:0/4195308438 shutdown_connections 2024-09-19T09:20:07.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:07.596+0000 7fad463c0640 1 -- 172.21.15.120:0/4195308438 wait complete. 2024-09-19T09:20:08.366 INFO:tasks.workunit.client.0.smithi120.stdout: 10 98072 18.45 MB/sec execute 123 sec latency 308.237 ms 2024-09-19T09:20:09.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:08 smithi120 ceph-mon[31150]: pgmap v863: 97 pgs: 97 active+clean; 366 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 9.0 MiB/s wr, 808 op/s 2024-09-19T09:20:09.365 INFO:tasks.workunit.client.0.smithi120.stdout: 10 98656 18.59 MB/sec execute 124 sec latency 73.240 ms 2024-09-19T09:20:09.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:08 smithi196 ceph-mon[35150]: pgmap v863: 97 pgs: 97 active+clean; 366 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 9.0 MiB/s wr, 808 op/s 2024-09-19T09:20:09.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:08 smithi153 ceph-mon[35071]: pgmap v863: 97 pgs: 97 active+clean; 366 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 9.0 MiB/s wr, 808 op/s 2024-09-19T09:20:10.366 INFO:tasks.workunit.client.0.smithi120.stdout: 10 99388 18.71 MB/sec execute 125 sec latency 98.638 ms 2024-09-19T09:20:11.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:10 smithi120 ceph-mon[31150]: pgmap v864: 97 pgs: 97 active+clean; 350 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.6 MiB/s wr, 809 op/s 2024-09-19T09:20:11.365 INFO:tasks.workunit.client.0.smithi120.stdout: 10 100067 18.84 MB/sec execute 126 sec latency 68.513 ms 2024-09-19T09:20:11.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:10 smithi196 ceph-mon[35150]: pgmap v864: 97 pgs: 97 active+clean; 350 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.6 MiB/s wr, 809 op/s 2024-09-19T09:20:11.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:10 smithi153 ceph-mon[35071]: pgmap v864: 97 pgs: 97 active+clean; 350 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.6 MiB/s wr, 809 op/s 2024-09-19T09:20:12.366 INFO:tasks.workunit.client.0.smithi120.stdout: 10 100848 19.09 MB/sec execute 127 sec latency 254.815 ms 2024-09-19T09:20:13.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:12 smithi120 ceph-mon[31150]: pgmap v865: 97 pgs: 97 active+clean; 357 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.7 MiB/s wr, 811 op/s 2024-09-19T09:20:13.366 INFO:tasks.workunit.client.0.smithi120.stdout: 10 101301 19.19 MB/sec execute 128 sec latency 217.688 ms 2024-09-19T09:20:13.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:12 smithi196 ceph-mon[35150]: pgmap v865: 97 pgs: 97 active+clean; 357 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.7 MiB/s wr, 811 op/s 2024-09-19T09:20:13.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:12 smithi153 ceph-mon[35071]: pgmap v865: 97 pgs: 97 active+clean; 357 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.7 MiB/s wr, 811 op/s 2024-09-19T09:20:14.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:14 smithi120 ceph-mon[31150]: pgmap v866: 97 pgs: 97 active+clean; 367 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.7 MiB/s wr, 870 op/s 2024-09-19T09:20:14.366 INFO:tasks.workunit.client.0.smithi120.stdout: 10 101841 19.24 MB/sec execute 129 sec latency 96.841 ms 2024-09-19T09:20:14.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:14 smithi196 ceph-mon[35150]: pgmap v866: 97 pgs: 97 active+clean; 367 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.7 MiB/s wr, 870 op/s 2024-09-19T09:20:14.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:14 smithi153 ceph-mon[35071]: pgmap v866: 97 pgs: 97 active+clean; 367 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.7 MiB/s wr, 870 op/s 2024-09-19T09:20:15.366 INFO:tasks.workunit.client.0.smithi120.stdout: 10 102242 19.25 MB/sec execute 130 sec latency 106.307 ms 2024-09-19T09:20:15.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:15 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:20:15.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:15 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:20:15.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:15 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:20:16.366 INFO:tasks.workunit.client.0.smithi120.stdout: 10 102817 19.28 MB/sec execute 131 sec latency 115.443 ms 2024-09-19T09:20:16.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:16 smithi120 ceph-mon[31150]: pgmap v867: 97 pgs: 97 active+clean; 367 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.9 MiB/s wr, 726 op/s 2024-09-19T09:20:16.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.827 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:16 smithi120 ceph-mon[31150]: Adjusting osd_memory_target on smithi153 to 176.0M 2024-09-19T09:20:16.827 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:16 smithi120 ceph-mon[31150]: Adjusting osd_memory_target on smithi120 to 176.0M 2024-09-19T09:20:16.827 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:16 smithi120 ceph-mon[31150]: Adjusting osd_memory_target on smithi196 to 176.0M 2024-09-19T09:20:16.827 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:16 smithi120 ceph-mon[31150]: Unable to set osd_memory_target on smithi153 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-19T09:20:16.827 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:16 smithi120 ceph-mon[31150]: Unable to set osd_memory_target on smithi120 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-19T09:20:16.827 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:16 smithi120 ceph-mon[31150]: Unable to set osd_memory_target on smithi196 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-19T09:20:16.827 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:16.827 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:20:16.827 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:16 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:20:16.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:16 smithi196 ceph-mon[35150]: pgmap v867: 97 pgs: 97 active+clean; 367 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.9 MiB/s wr, 726 op/s 2024-09-19T09:20:16.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.938 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.938 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.938 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.938 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.938 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:16 smithi196 ceph-mon[35150]: Adjusting osd_memory_target on smithi153 to 176.0M 2024-09-19T09:20:16.938 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:16 smithi196 ceph-mon[35150]: Adjusting osd_memory_target on smithi120 to 176.0M 2024-09-19T09:20:16.938 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:16 smithi196 ceph-mon[35150]: Adjusting osd_memory_target on smithi196 to 176.0M 2024-09-19T09:20:16.938 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:16 smithi196 ceph-mon[35150]: Unable to set osd_memory_target on smithi153 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-19T09:20:16.938 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:16 smithi196 ceph-mon[35150]: Unable to set osd_memory_target on smithi120 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-19T09:20:16.938 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:16 smithi196 ceph-mon[35150]: Unable to set osd_memory_target on smithi196 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-19T09:20:16.938 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:16.938 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:20:16.938 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:16 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:20:16.939 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:16 smithi153 ceph-mon[35071]: pgmap v867: 97 pgs: 97 active+clean; 367 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.9 MiB/s wr, 726 op/s 2024-09-19T09:20:16.939 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.939 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.939 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.939 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.939 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.939 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.939 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.939 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.939 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.940 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.940 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.940 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:16.940 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:16 smithi153 ceph-mon[35071]: Adjusting osd_memory_target on smithi153 to 176.0M 2024-09-19T09:20:16.940 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:16 smithi153 ceph-mon[35071]: Adjusting osd_memory_target on smithi120 to 176.0M 2024-09-19T09:20:16.940 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:16 smithi153 ceph-mon[35071]: Adjusting osd_memory_target on smithi196 to 176.0M 2024-09-19T09:20:16.940 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:16 smithi153 ceph-mon[35071]: Unable to set osd_memory_target on smithi153 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-19T09:20:16.940 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:16 smithi153 ceph-mon[35071]: Unable to set osd_memory_target on smithi120 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-19T09:20:16.940 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:16 smithi153 ceph-mon[35071]: Unable to set osd_memory_target on smithi196 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-19T09:20:16.940 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:16.940 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:20:16.940 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:16 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:20:17.366 INFO:tasks.workunit.client.0.smithi120.stdout: 10 103423 19.42 MB/sec execute 132 sec latency 60.728 ms 2024-09-19T09:20:18.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:18 smithi120 ceph-mon[31150]: pgmap v868: 97 pgs: 97 active+clean; 368 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.7 MiB/s wr, 775 op/s 2024-09-19T09:20:18.366 INFO:tasks.workunit.client.0.smithi120.stdout: 10 103938 19.45 MB/sec execute 133 sec latency 232.518 ms 2024-09-19T09:20:18.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:18 smithi196 ceph-mon[35150]: pgmap v868: 97 pgs: 97 active+clean; 368 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.7 MiB/s wr, 775 op/s 2024-09-19T09:20:18.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:18 smithi153 ceph-mon[35071]: pgmap v868: 97 pgs: 97 active+clean; 368 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.7 MiB/s wr, 775 op/s 2024-09-19T09:20:19.366 INFO:tasks.workunit.client.0.smithi120.stdout: 10 104664 19.56 MB/sec execute 134 sec latency 47.909 ms 2024-09-19T09:20:20.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:20 smithi120 ceph-mon[31150]: pgmap v869: 97 pgs: 97 active+clean; 372 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.9 MiB/s wr, 756 op/s 2024-09-19T09:20:20.366 INFO:tasks.workunit.client.0.smithi120.stdout: 10 105427 19.79 MB/sec execute 135 sec latency 78.171 ms 2024-09-19T09:20:20.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:20 smithi196 ceph-mon[35150]: pgmap v869: 97 pgs: 97 active+clean; 372 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.9 MiB/s wr, 756 op/s 2024-09-19T09:20:20.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:20 smithi153 ceph-mon[35071]: pgmap v869: 97 pgs: 97 active+clean; 372 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.9 MiB/s wr, 756 op/s 2024-09-19T09:20:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.701+0000 7fb16bae4640 1 -- 172.21.15.120:0/3474594807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb164101c30 msgr2=0x7fb1641020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.701+0000 7fb16bae4640 1 --2- 172.21.15.120:0/3474594807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb164101c30 0x7fb1641020b0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fb154009900 tx=0x7fb15402a410 comp rx=0 tx=0).stop 2024-09-19T09:20:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.701+0000 7fb16bae4640 1 -- 172.21.15.120:0/3474594807 shutdown_connections 2024-09-19T09:20:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.701+0000 7fb16bae4640 1 --2- 172.21.15.120:0/3474594807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1641025f0 0x7fb164108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.701+0000 7fb16bae4640 1 --2- 172.21.15.120:0/3474594807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb164101c30 0x7fb1641020b0 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.701+0000 7fb16bae4640 1 --2- 172.21.15.120:0/3474594807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb164100a40 0x7fb164100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.701+0000 7fb16bae4640 1 -- 172.21.15.120:0/3474594807 >> 172.21.15.120:0/3474594807 conn(0x7fb1640fc1e0 msgr2=0x7fb1640fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.701+0000 7fb16bae4640 1 -- 172.21.15.120:0/3474594807 shutdown_connections 2024-09-19T09:20:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.701+0000 7fb16bae4640 1 -- 172.21.15.120:0/3474594807 wait complete. 2024-09-19T09:20:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.701+0000 7fb16bae4640 1 Processor -- start 2024-09-19T09:20:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.701+0000 7fb16bae4640 1 -- start start 2024-09-19T09:20:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.702+0000 7fb16bae4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb164100a40 0x7fb16406a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.702+0000 7fb16bae4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb164101c30 0x7fb16419cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.702+0000 7fb16bae4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1641025f0 0x7fb16419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.702+0000 7fb16bae4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb16410c6a0 con 0x7fb164100a40 2024-09-19T09:20:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.702+0000 7fb16bae4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb16410c520 con 0x7fb164101c30 2024-09-19T09:20:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.702+0000 7fb16bae4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb16410c820 con 0x7fb1641025f0 2024-09-19T09:20:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.702+0000 7fb169880640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb164100a40 0x7fb16406a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.702+0000 7fb169880640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb164100a40 0x7fb16406a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55346/0 (socket says 172.21.15.120:55346) 2024-09-19T09:20:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.702+0000 7fb169880640 1 -- 172.21.15.120:0/802317784 learned_addr learned my addr 172.21.15.120:0/802317784 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.702+0000 7fb16907f640 1 --2- 172.21.15.120:0/802317784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb164101c30 0x7fb16419cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.702+0000 7fb16a081640 1 --2- 172.21.15.120:0/802317784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1641025f0 0x7fb16419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.702+0000 7fb169880640 1 -- 172.21.15.120:0/802317784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1641025f0 msgr2=0x7fb16419f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.702+0000 7fb169880640 1 --2- 172.21.15.120:0/802317784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1641025f0 0x7fb16419f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.702+0000 7fb169880640 1 -- 172.21.15.120:0/802317784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb164101c30 msgr2=0x7fb16419cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.702+0000 7fb169880640 1 --2- 172.21.15.120:0/802317784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb164101c30 0x7fb16419cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.702+0000 7fb169880640 1 -- 172.21.15.120:0/802317784 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16419fa40 con 0x7fb164100a40 2024-09-19T09:20:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.702+0000 7fb169880640 1 --2- 172.21.15.120:0/802317784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb164100a40 0x7fb16406a410 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fb16000ebc0 tx=0x7fb16000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.702+0000 7fb15affd640 1 -- 172.21.15.120:0/802317784 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160003b70 con 0x7fb164100a40 2024-09-19T09:20:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.702+0000 7fb16bae4640 1 -- 172.21.15.120:0/802317784 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb16419fcd0 con 0x7fb164100a40 2024-09-19T09:20:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.702+0000 7fb15affd640 1 -- 172.21.15.120:0/802317784 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb160003d10 con 0x7fb164100a40 2024-09-19T09:20:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.702+0000 7fb15affd640 1 -- 172.21.15.120:0/802317784 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160019980 con 0x7fb164100a40 2024-09-19T09:20:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.702+0000 7fb16bae4640 1 -- 172.21.15.120:0/802317784 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1641a01a0 con 0x7fb164100a40 2024-09-19T09:20:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.706+0000 7fb16e886740 1 Processor -- start 2024-09-19T09:20:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.706+0000 7fb16e886740 1 -- start start 2024-09-19T09:20:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.706+0000 7fb16e886740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae38f34db0 0x55ae38f351b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.706+0000 7fb16e886740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae38f356f0 0x55ae38f3a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.706+0000 7fb16e886740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae38f3a780 0x55ae38f3cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.706+0000 7fb16e886740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ae38e942f0 con 0x55ae38f3a780 2024-09-19T09:20:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.706+0000 7fb16e886740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ae38e3c270 con 0x55ae38f356f0 2024-09-19T09:20:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.706+0000 7fb16e886740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ae38e902b0 con 0x55ae38f34db0 2024-09-19T09:20:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.706+0000 7fb15affd640 1 -- 172.21.15.120:0/802317784 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fb160019b20 con 0x7fb164100a40 2024-09-19T09:20:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.706+0000 7fb16a081640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae38f34db0 0x55ae38f351b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.706+0000 7fb16907f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae38f3a780 0x55ae38f3cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.706+0000 7fb16a081640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae38f34db0 0x55ae38f351b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:39896/0 (socket says 172.21.15.120:39896) 2024-09-19T09:20:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.706+0000 7fb16907f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae38f3a780 0x55ae38f3cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55354/0 (socket says 172.21.15.120:55354) 2024-09-19T09:20:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.706+0000 7fb16a081640 1 -- 172.21.15.120:0/2369745214 learned_addr learned my addr 172.21.15.120:0/2369745214 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.706+0000 7fb169880640 1 --2- 172.21.15.120:0/2369745214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae38f356f0 0x55ae38f3a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.707+0000 7fb16907f640 1 -- 172.21.15.120:0/2369745214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae38f34db0 msgr2=0x55ae38f351b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.707+0000 7fb16907f640 1 --2- 172.21.15.120:0/2369745214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae38f34db0 0x55ae38f351b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.707+0000 7fb16907f640 1 -- 172.21.15.120:0/2369745214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae38f356f0 msgr2=0x55ae38f3a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.707+0000 7fb16907f640 1 --2- 172.21.15.120:0/2369745214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae38f356f0 0x55ae38f3a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.707+0000 7fb16907f640 1 -- 172.21.15.120:0/2369745214 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae38e3b340 con 0x55ae38f3a780 2024-09-19T09:20:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.707+0000 7fb16a081640 1 --2- 172.21.15.120:0/2369745214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae38f34db0 0x55ae38f351b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:20:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.707+0000 7fb169880640 1 --2- 172.21.15.120:0/2369745214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae38f356f0 0x55ae38f3a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:20:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.707+0000 7fb16907f640 1 --2- 172.21.15.120:0/2369745214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae38f3a780 0x55ae38f3cc40 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fb154031440 tx=0x7fb154032c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2dfd401de336550 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.707+0000 7fb14ffff640 1 -- 172.21.15.120:0/2369745214 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154040070 con 0x55ae38f3a780 2024-09-19T09:20:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.707+0000 7fb15affd640 1 --2- 172.21.15.120:0/802317784 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb1500a3370 0x7fb1500a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.707+0000 7fb14ffff640 1 -- 172.21.15.120:0/2369745214 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb154032e20 con 0x55ae38f3a780 2024-09-19T09:20:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.707+0000 7fb14ffff640 1 -- 172.21.15.120:0/2369745214 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154035040 con 0x55ae38f3a780 2024-09-19T09:20:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.708+0000 7fb16a081640 1 --2- 172.21.15.120:0/802317784 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb1500a3370 0x7fb1500a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.708+0000 7fb15affd640 1 -- 172.21.15.120:0/802317784 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fb1600d0160 con 0x7fb164100a40 2024-09-19T09:20:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.708+0000 7fb16e886740 1 -- 172.21.15.120:0/2369745214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae38f3a780 msgr2=0x55ae38f3cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.708+0000 7fb16e886740 1 --2- 172.21.15.120:0/2369745214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae38f3a780 0x55ae38f3cc40 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fb154031440 tx=0x7fb154032c30 comp rx=0 tx=0).stop 2024-09-19T09:20:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.708+0000 7fb16a081640 1 --2- 172.21.15.120:0/802317784 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb1500a3370 0x7fb1500a5830 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00b9f0 tx=0x7fb15c005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.708+0000 7fb16e886740 1 -- 172.21.15.120:0/2369745214 shutdown_connections 2024-09-19T09:20:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.708+0000 7fb16e886740 1 --2- 172.21.15.120:0/2369745214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae38f3a780 0x55ae38f3cc40 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.708+0000 7fb16e886740 1 --2- 172.21.15.120:0/2369745214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae38f356f0 0x55ae38f3a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.708+0000 7fb16e886740 1 --2- 172.21.15.120:0/2369745214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae38f34db0 0x55ae38f351b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.708+0000 7fb16e886740 1 -- 172.21.15.120:0/2369745214 >> 172.21.15.120:0/2369745214 conn(0x55ae38f30420 msgr2=0x55ae38f30820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.709+0000 7fb16e886740 1 -- 172.21.15.120:0/2369745214 shutdown_connections 2024-09-19T09:20:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.709+0000 7fb16e886740 1 -- 172.21.15.120:0/2369745214 wait complete. 2024-09-19T09:20:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.709+0000 7fb16e886740 1 -- start start 2024-09-19T09:20:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.709+0000 7fb16e886740 1 Processor -- start 2024-09-19T09:20:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.710+0000 7fb16e886740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae38f34db0 0x55ae38f37d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.710+0000 7fb16e886740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae38f356f0 0x55ae38f382b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.710+0000 7fb16e886740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae38f3a780 0x55ae38f39250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.710+0000 7fb16e886740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ae38f2bbb0 con 0x55ae38f356f0 2024-09-19T09:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.710+0000 7fb169880640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae38f34db0 0x55ae38f37d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.710+0000 7fb16e886740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ae38e1d290 con 0x55ae38f34db0 2024-09-19T09:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.710+0000 7fb16e886740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ae38f2bd30 con 0x55ae38f3a780 2024-09-19T09:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.710+0000 7fb169880640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae38f34db0 0x55ae38f37d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:57688/0 (socket says 172.21.15.120:57688) 2024-09-19T09:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.710+0000 7fb169880640 1 -- 172.21.15.120:0/1797199713 learned_addr learned my addr 172.21.15.120:0/1797199713 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.710+0000 7fb16e886740 10 client.0 _ll_register_callbacks cb 0x55ae38e4e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.710+0000 7fb16a081640 1 --2- 172.21.15.120:0/1797199713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae38f3a780 0x55ae38f39250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.710+0000 7fb16907f640 1 --2- 172.21.15.120:0/1797199713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae38f356f0 0x55ae38f382b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.710+0000 7fb169880640 1 -- 172.21.15.120:0/1797199713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae38f3a780 msgr2=0x55ae38f39250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.710+0000 7fb169880640 1 --2- 172.21.15.120:0/1797199713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae38f3a780 0x55ae38f39250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.710+0000 7fb169880640 1 -- 172.21.15.120:0/1797199713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae38f356f0 msgr2=0x55ae38f382b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.710+0000 7fb169880640 1 --2- 172.21.15.120:0/1797199713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae38f356f0 0x55ae38f382b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.710+0000 7fb169880640 1 -- 172.21.15.120:0/1797199713 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae38ba7210 con 0x55ae38f34db0 2024-09-19T09:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.710+0000 7fb16907f640 1 --2- 172.21.15.120:0/1797199713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae38f356f0 0x55ae38f382b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.710+0000 7fb16a081640 1 --2- 172.21.15.120:0/1797199713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae38f3a780 0x55ae38f39250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.711+0000 7fb169880640 1 --2- 172.21.15.120:0/1797199713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae38f34db0 0x55ae38f37d70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb160009270 tx=0x7fb160004a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec5251056f4500a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.711+0000 7fb14d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.711+0000 7fb14d7fa640 1 -- 172.21.15.120:0/1797199713 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16009c510 con 0x55ae38f34db0 2024-09-19T09:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.711+0000 7fb14d7fa640 1 -- 172.21.15.120:0/1797199713 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fb1600a4f20 con 0x55ae38f34db0 2024-09-19T09:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.711+0000 7fb16e886740 1 -- 172.21.15.120:0/1797199713 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb154032e20 con 0x55ae38f34db0 2024-09-19T09:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.711+0000 7fb14d7fa640 1 -- 172.21.15.120:0/1797199713 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb16009c0a0 con 0x55ae38f34db0 2024-09-19T09:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.711+0000 7fb16e886740 10 client.30134 fetch_fsmap learned FSMap version 39 2024-09-19T09:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.711+0000 7fb16e886740 1 -- 172.21.15.120:0/1797199713 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ae38c73620 con 0x55ae38f34db0 2024-09-19T09:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.711+0000 7fb14d7fa640 1 -- 172.21.15.120:0/1797199713 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb154032e20 con 0x55ae38f34db0 2024-09-19T09:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.712+0000 7fb14d7fa640 1 -- 172.21.15.120:0/1797199713 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fb160004b60 con 0x55ae38f34db0 2024-09-19T09:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.712+0000 7fb16e886740 10 client.30134 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:20:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.712+0000 7fb16e886740 10 client.30134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:20:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.712+0000 7fb16e886740 20 client.30134 populate_metadata read hostname 'smithi120' 2024-09-19T09:20:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.712+0000 7fb16e886740 1 --2- 172.21.15.120:0/1797199713 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ae38f4f020 0x55ae38f51410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.712+0000 7fb16e886740 4 client.30134 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:20:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.712+0000 7fb16e886740 1 -- 172.21.15.120:0/1797199713 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae38ea1a20 con 0x55ae38f4f020 2024-09-19T09:20:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.712+0000 7fb16907f640 1 --2- 172.21.15.120:0/1797199713 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ae38f4f020 0x55ae38f51410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.712+0000 7fb14d7fa640 10 client.30134 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.712+0000 7fb16907f640 1 --2- 172.21.15.120:0/1797199713 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ae38f4f020 0x55ae38f51410 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e61419a9caf4e5a2 server_cookie=341426775d82fdc3 in_seq=0 out_seq=0 2024-09-19T09:20:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.716+0000 7fb14d7fa640 1 -- 172.21.15.120:0/1797199713 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae38ea1a20 con 0x55ae38f4f020 2024-09-19T09:20:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.716+0000 7fb14d7fa640 10 client.30134 handle_command_reply: tid=0 2024-09-19T09:20:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.716+0000 7fb16e886740 1 client.30134 shutdown 2024-09-19T09:20:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.717+0000 7fb16e886740 1 -- 172.21.15.120:0/1797199713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae38f34db0 msgr2=0x55ae38f37d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.717+0000 7fb16e886740 1 --2- 172.21.15.120:0/1797199713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae38f34db0 0x55ae38f37d70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb160009270 tx=0x7fb160004a90 comp rx=0 tx=0).stop 2024-09-19T09:20:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.717+0000 7fb16e886740 1 -- 172.21.15.120:0/1797199713 shutdown_connections 2024-09-19T09:20:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.717+0000 7fb16e886740 1 --2- 172.21.15.120:0/1797199713 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ae38f4f020 0x55ae38f51410 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.717+0000 7fb16e886740 1 --2- 172.21.15.120:0/1797199713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae38f3a780 0x55ae38f39250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.717+0000 7fb16e886740 1 --2- 172.21.15.120:0/1797199713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae38f356f0 0x55ae38f382b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.717+0000 7fb16e886740 1 --2- 172.21.15.120:0/1797199713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae38f34db0 0x55ae38f37d70 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.717+0000 7fb14d7fa640 0 client.30134 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.717+0000 7fb16e886740 1 -- 172.21.15.120:0/1797199713 >> 172.21.15.120:0/1797199713 conn(0x55ae38f30420 msgr2=0x55ae38f2b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.717+0000 7fb16e886740 1 -- 172.21.15.120:0/1797199713 shutdown_connections 2024-09-19T09:20:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.717+0000 7fb16e886740 1 -- 172.21.15.120:0/1797199713 wait complete. 2024-09-19T09:20:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.717+0000 7fb16e886740 20 client.30134 trim_cache size 0 max 16384 2024-09-19T09:20:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.736+0000 7fb16e886740 1 Processor -- start 2024-09-19T09:20:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.736+0000 7fb16e886740 1 -- start start 2024-09-19T09:20:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.736+0000 7fb16e886740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae38f34db0 0x55ae38f38340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.736+0000 7fb16e886740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae38f356f0 0x55ae38f2b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.736+0000 7fb16e886740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae38f3a780 0x55ae38f55cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.737+0000 7fb169880640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae38f34db0 0x55ae38f38340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.737+0000 7fb16e886740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb1641a6e00 con 0x55ae38f34db0 2024-09-19T09:20:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.737+0000 7fb169880640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae38f34db0 0x55ae38f38340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55372/0 (socket says 172.21.15.120:55372) 2024-09-19T09:20:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.737+0000 7fb169880640 1 -- 172.21.15.120:0/1458089007 learned_addr learned my addr 172.21.15.120:0/1458089007 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.737+0000 7fb16e886740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb1641a8940 con 0x55ae38f3a780 2024-09-19T09:20:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.737+0000 7fb16e886740 1 -- 172.21.15.120:0/1458089007 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb1641a74d0 con 0x55ae38f356f0 2024-09-19T09:20:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.737+0000 7fb16a081640 1 --2- 172.21.15.120:0/1458089007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae38f3a780 0x55ae38f55cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.737+0000 7fb16907f640 1 --2- 172.21.15.120:0/1458089007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae38f356f0 0x55ae38f2b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.737+0000 7fb16a081640 1 -- 172.21.15.120:0/1458089007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae38f356f0 msgr2=0x55ae38f2b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.737+0000 7fb16a081640 1 --2- 172.21.15.120:0/1458089007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae38f356f0 0x55ae38f2b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.737+0000 7fb16a081640 1 -- 172.21.15.120:0/1458089007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae38f34db0 msgr2=0x55ae38f38340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.737+0000 7fb16a081640 1 --2- 172.21.15.120:0/1458089007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae38f34db0 0x55ae38f38340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.737+0000 7fb16a081640 1 -- 172.21.15.120:0/1458089007 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae38f51780 con 0x55ae38f3a780 2024-09-19T09:20:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.737+0000 7fb169880640 1 --2- 172.21.15.120:0/1458089007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae38f34db0 0x55ae38f38340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:20:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.737+0000 7fb16a081640 1 --2- 172.21.15.120:0/1458089007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae38f3a780 0x55ae38f55cf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb15c0039c0 tx=0x7fb15c016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fda2a3c70ad64479 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.738+0000 7fb14f7fe640 1 -- 172.21.15.120:0/1458089007 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c003cf0 con 0x55ae38f3a780 2024-09-19T09:20:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.738+0000 7fb14f7fe640 1 -- 172.21.15.120:0/1458089007 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb15c01fe50 con 0x55ae38f3a780 2024-09-19T09:20:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.738+0000 7fb14f7fe640 1 -- 172.21.15.120:0/1458089007 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c004d90 con 0x55ae38f3a780 2024-09-19T09:20:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.738+0000 7fb16e886740 1 -- 172.21.15.120:0/1458089007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae38f3a780 msgr2=0x55ae38f55cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.738+0000 7fb16e886740 1 --2- 172.21.15.120:0/1458089007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae38f3a780 0x55ae38f55cf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb15c0039c0 tx=0x7fb15c016000 comp rx=0 tx=0).stop 2024-09-19T09:20:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.738+0000 7fb16e886740 1 -- 172.21.15.120:0/1458089007 shutdown_connections 2024-09-19T09:20:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.738+0000 7fb16e886740 1 --2- 172.21.15.120:0/1458089007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae38f3a780 0x55ae38f55cf0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.738+0000 7fb16e886740 1 --2- 172.21.15.120:0/1458089007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae38f356f0 0x55ae38f2b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.738+0000 7fb16e886740 1 --2- 172.21.15.120:0/1458089007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae38f34db0 0x55ae38f38340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.738+0000 7fb16e886740 1 -- 172.21.15.120:0/1458089007 >> 172.21.15.120:0/1458089007 conn(0x55ae38f30420 msgr2=0x55ae38f2a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.739+0000 7fb16e886740 1 -- 172.21.15.120:0/1458089007 shutdown_connections 2024-09-19T09:20:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.739+0000 7fb16e886740 1 -- 172.21.15.120:0/1458089007 wait complete. 2024-09-19T09:20:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.739+0000 7fb16e886740 1 -- start start 2024-09-19T09:20:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.739+0000 7fb16e886740 1 Processor -- start 2024-09-19T09:20:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.739+0000 7fb16e886740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae38f34db0 0x55ae38f491e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.739+0000 7fb16e886740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae38f356f0 0x55ae38f49720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.739+0000 7fb16e886740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae38f3a780 0x55ae38f4a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.740+0000 7fb16e886740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb1641a8270 con 0x55ae38f356f0 2024-09-19T09:20:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.740+0000 7fb16e886740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb1641a7ba0 con 0x55ae38f3a780 2024-09-19T09:20:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.740+0000 7fb16e886740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb1641a4dc0 con 0x55ae38f34db0 2024-09-19T09:20:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.740+0000 7fb16e886740 10 client.0 _ll_register_callbacks cb 0x7fb16000eb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:20:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.740+0000 7fb16a081640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae38f3a780 0x55ae38f4a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.740+0000 7fb16907f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae38f356f0 0x55ae38f49720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.740+0000 7fb169880640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae38f34db0 0x55ae38f491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.740+0000 7fb16a081640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae38f3a780 0x55ae38f4a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:57702/0 (socket says 172.21.15.120:57702) 2024-09-19T09:20:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.740+0000 7fb16a081640 1 -- 172.21.15.120:0/836743657 learned_addr learned my addr 172.21.15.120:0/836743657 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.740+0000 7fb16a081640 1 -- 172.21.15.120:0/836743657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae38f34db0 msgr2=0x55ae38f491e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.740+0000 7fb16a081640 1 --2- 172.21.15.120:0/836743657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae38f34db0 0x55ae38f491e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.740+0000 7fb16a081640 1 -- 172.21.15.120:0/836743657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae38f356f0 msgr2=0x55ae38f49720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.740+0000 7fb16a081640 1 --2- 172.21.15.120:0/836743657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae38f356f0 0x55ae38f49720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.740+0000 7fb16a081640 1 -- 172.21.15.120:0/836743657 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae38f4ad40 con 0x55ae38f3a780 2024-09-19T09:20:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.740+0000 7fb16907f640 1 --2- 172.21.15.120:0/836743657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae38f356f0 0x55ae38f49720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:20:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.740+0000 7fb169880640 1 --2- 172.21.15.120:0/836743657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae38f34db0 0x55ae38f491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:20:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.740+0000 7fb16a081640 1 --2- 172.21.15.120:0/836743657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae38f3a780 0x55ae38f4a730 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb15c018040 tx=0x7fb15c0041d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fb0d9394847eb57 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.740+0000 7fb14d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:20:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.740+0000 7fb14d7fa640 1 -- 172.21.15.120:0/836743657 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c036020 con 0x55ae38f3a780 2024-09-19T09:20:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.741+0000 7fb14d7fa640 1 -- 172.21.15.120:0/836743657 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fb15c004c90 con 0x55ae38f3a780 2024-09-19T09:20:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.741+0000 7fb16e886740 1 -- 172.21.15.120:0/836743657 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb15c01fe50 con 0x55ae38f3a780 2024-09-19T09:20:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.741+0000 7fb14d7fa640 1 -- 172.21.15.120:0/836743657 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb15c004e90 con 0x55ae38f3a780 2024-09-19T09:20:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.741+0000 7fb16e886740 10 client.30140 fetch_fsmap learned FSMap version 39 2024-09-19T09:20:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.741+0000 7fb16e886740 1 -- 172.21.15.120:0/836743657 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ae38fb9fd0 con 0x55ae38f3a780 2024-09-19T09:20:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.741+0000 7fb14d7fa640 1 -- 172.21.15.120:0/836743657 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb15c01fe50 con 0x55ae38f3a780 2024-09-19T09:20:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.741+0000 7fb14d7fa640 1 -- 172.21.15.120:0/836743657 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fb15c0289d0 con 0x55ae38f3a780 2024-09-19T09:20:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.741+0000 7fb16e886740 10 client.30140 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:20:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.741+0000 7fb16e886740 10 client.30140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:20:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.741+0000 7fb16e886740 20 client.30140 populate_metadata read hostname 'smithi120' 2024-09-19T09:20:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.741+0000 7fb16e886740 1 --2- 172.21.15.120:0/836743657 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ae38fba9d0 0x55ae38fbcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.741+0000 7fb16e886740 4 client.30140 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:20:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.741+0000 7fb16e886740 1 -- 172.21.15.120:0/836743657 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ae38e9bd70 con 0x55ae38fba9d0 2024-09-19T09:20:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.741+0000 7fb169880640 1 --2- 172.21.15.120:0/836743657 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ae38fba9d0 0x55ae38fbcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.742+0000 7fb14d7fa640 10 client.30140 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:20.742+0000 7fb169880640 1 --2- 172.21.15.120:0/836743657 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ae38fba9d0 0x55ae38fbcdf0 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f66584d5ac55b6bd server_cookie=466aea1f10b6d8ca in_seq=0 out_seq=0 2024-09-19T09:20:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:21.247+0000 7fb14d7fa640 1 -- 172.21.15.120:0/836743657 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ae38e9bd70 con 0x55ae38fba9d0 2024-09-19T09:20:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:21.247+0000 7fb14d7fa640 10 client.30140 handle_command_reply: tid=0 2024-09-19T09:20:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:21.247+0000 7fb16e886740 1 client.30140 shutdown 2024-09-19T09:20:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:21.247+0000 7fb16e886740 1 -- 172.21.15.120:0/836743657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae38f3a780 msgr2=0x55ae38f4a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:21.247+0000 7fb16e886740 1 --2- 172.21.15.120:0/836743657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae38f3a780 0x55ae38f4a730 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb15c018040 tx=0x7fb15c0041d0 comp rx=0 tx=0).stop 2024-09-19T09:20:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:21.247+0000 7fb16e886740 1 -- 172.21.15.120:0/836743657 shutdown_connections 2024-09-19T09:20:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:21.247+0000 7fb16e886740 1 --2- 172.21.15.120:0/836743657 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ae38fba9d0 0x55ae38fbcdf0 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:21.247+0000 7fb14d7fa640 0 client.30140 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:21.247+0000 7fb16e886740 1 --2- 172.21.15.120:0/836743657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae38f3a780 0x55ae38f4a730 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:21.247+0000 7fb16e886740 1 --2- 172.21.15.120:0/836743657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae38f356f0 0x55ae38f49720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:21.247+0000 7fb16e886740 1 --2- 172.21.15.120:0/836743657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae38f34db0 0x55ae38f491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:21.247+0000 7fb16e886740 1 -- 172.21.15.120:0/836743657 >> 172.21.15.120:0/836743657 conn(0x55ae38f30420 msgr2=0x55ae38f2b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:21.247+0000 7fb16e886740 1 -- 172.21.15.120:0/836743657 shutdown_connections 2024-09-19T09:20:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:21.247+0000 7fb16e886740 1 -- 172.21.15.120:0/836743657 wait complete. 2024-09-19T09:20:21.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:21.248+0000 7fb16e886740 20 client.30140 trim_cache size 0 max 16384 2024-09-19T09:20:21.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:20:21.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:20:21.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:20:21.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:20:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:21.248+0000 7fb16bae4640 1 -- 172.21.15.120:0/802317784 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb1500a3370 msgr2=0x7fb1500a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:21.248+0000 7fb16bae4640 1 --2- 172.21.15.120:0/802317784 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb1500a3370 0x7fb1500a5830 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00b9f0 tx=0x7fb15c005cc0 comp rx=0 tx=0).stop 2024-09-19T09:20:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:21.248+0000 7fb16bae4640 1 -- 172.21.15.120:0/802317784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb164100a40 msgr2=0x7fb16406a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:21.248+0000 7fb16bae4640 1 --2- 172.21.15.120:0/802317784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb164100a40 0x7fb16406a410 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fb16000ebc0 tx=0x7fb16000c6a0 comp rx=0 tx=0).stop 2024-09-19T09:20:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:21.248+0000 7fb16bae4640 1 -- 172.21.15.120:0/802317784 shutdown_connections 2024-09-19T09:20:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:21.248+0000 7fb16bae4640 1 --2- 172.21.15.120:0/802317784 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb1500a3370 0x7fb1500a5830 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:21.248+0000 7fb16bae4640 1 --2- 172.21.15.120:0/802317784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1641025f0 0x7fb16419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:21.248+0000 7fb16bae4640 1 --2- 172.21.15.120:0/802317784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb164101c30 0x7fb16419cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:21.248+0000 7fb16bae4640 1 --2- 172.21.15.120:0/802317784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb164100a40 0x7fb16406a410 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:21.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:21.248+0000 7fb16bae4640 1 -- 172.21.15.120:0/802317784 >> 172.21.15.120:0/802317784 conn(0x7fb1640fc1e0 msgr2=0x7fb164105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:21.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:21.248+0000 7fb16bae4640 1 -- 172.21.15.120:0/802317784 shutdown_connections 2024-09-19T09:20:21.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:21.248+0000 7fb16bae4640 1 -- 172.21.15.120:0/802317784 wait complete. 2024-09-19T09:20:21.366 INFO:tasks.workunit.client.0.smithi120.stdout: 10 105806 19.83 MB/sec execute 136 sec latency 342.003 ms 2024-09-19T09:20:22.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:22 smithi120 ceph-mon[31150]: pgmap v870: 97 pgs: 97 active+clean; 366 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.6 MiB/s wr, 734 op/s 2024-09-19T09:20:22.366 INFO:tasks.workunit.client.0.smithi120.stdout: 10 106431 19.93 MB/sec execute 137 sec latency 50.631 ms 2024-09-19T09:20:22.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:22 smithi196 ceph-mon[35150]: pgmap v870: 97 pgs: 97 active+clean; 366 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.6 MiB/s wr, 734 op/s 2024-09-19T09:20:22.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:22 smithi153 ceph-mon[35071]: pgmap v870: 97 pgs: 97 active+clean; 366 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.6 MiB/s wr, 734 op/s 2024-09-19T09:20:23.366 INFO:tasks.workunit.client.0.smithi120.stdout: 10 106819 19.89 MB/sec execute 138 sec latency 203.852 ms 2024-09-19T09:20:24.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:24 smithi120 ceph-mon[31150]: pgmap v871: 97 pgs: 97 active+clean; 364 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.1 MiB/s wr, 742 op/s 2024-09-19T09:20:24.366 INFO:tasks.workunit.client.0.smithi120.stdout: 10 107361 19.96 MB/sec execute 139 sec latency 197.549 ms 2024-09-19T09:20:24.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:24 smithi196 ceph-mon[35150]: pgmap v871: 97 pgs: 97 active+clean; 364 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.1 MiB/s wr, 742 op/s 2024-09-19T09:20:24.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:24 smithi153 ceph-mon[35071]: pgmap v871: 97 pgs: 97 active+clean; 364 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.1 MiB/s wr, 742 op/s 2024-09-19T09:20:25.366 INFO:tasks.workunit.client.0.smithi120.stdout: 10 108020 20.08 MB/sec execute 140 sec latency 53.319 ms 2024-09-19T09:20:26.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:26 smithi120 ceph-mon[31150]: pgmap v872: 97 pgs: 97 active+clean; 364 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.2 MiB/s wr, 724 op/s 2024-09-19T09:20:26.366 INFO:tasks.workunit.client.0.smithi120.stdout: 10 108780 20.19 MB/sec execute 141 sec latency 54.007 ms 2024-09-19T09:20:26.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:26 smithi196 ceph-mon[35150]: pgmap v872: 97 pgs: 97 active+clean; 364 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.2 MiB/s wr, 724 op/s 2024-09-19T09:20:26.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:26 smithi153 ceph-mon[35071]: pgmap v872: 97 pgs: 97 active+clean; 364 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.2 MiB/s wr, 724 op/s 2024-09-19T09:20:27.367 INFO:tasks.workunit.client.0.smithi120.stdout: 10 109530 20.33 MB/sec execute 142 sec latency 54.194 ms 2024-09-19T09:20:28.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:28 smithi120 ceph-mon[31150]: pgmap v873: 97 pgs: 97 active+clean; 365 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.7 MiB/s wr, 756 op/s 2024-09-19T09:20:28.366 INFO:tasks.workunit.client.0.smithi120.stdout: 10 110028 20.45 MB/sec execute 143 sec latency 171.104 ms 2024-09-19T09:20:28.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:28 smithi196 ceph-mon[35150]: pgmap v873: 97 pgs: 97 active+clean; 365 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.7 MiB/s wr, 756 op/s 2024-09-19T09:20:28.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:28 smithi153 ceph-mon[35071]: pgmap v873: 97 pgs: 97 active+clean; 365 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.7 MiB/s wr, 756 op/s 2024-09-19T09:20:29.366 INFO:tasks.workunit.client.0.smithi120.stdout: 10 110583 20.50 MB/sec execute 144 sec latency 177.849 ms 2024-09-19T09:20:29.627 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:20:29.942 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:20:30.290 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:30 smithi120 ceph-mon[31150]: pgmap v874: 97 pgs: 97 active+clean; 372 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.2 MiB/s wr, 792 op/s 2024-09-19T09:20:30.366 INFO:tasks.workunit.client.0.smithi120.stdout: 10 111084 20.55 MB/sec execute 145 sec latency 131.224 ms 2024-09-19T09:20:30.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:30 smithi196 ceph-mon[35150]: pgmap v874: 97 pgs: 97 active+clean; 372 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.2 MiB/s wr, 792 op/s 2024-09-19T09:20:30.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:30 smithi153 ceph-mon[35071]: pgmap v874: 97 pgs: 97 active+clean; 372 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.2 MiB/s wr, 792 op/s 2024-09-19T09:20:30.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.537+0000 7f460ce45640 1 -- 172.21.15.120:0/3078384804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4608102ef0 msgr2=0x7f460810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:30.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.537+0000 7f460ce45640 1 --2- 172.21.15.120:0/3078384804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4608102ef0 0x7f460810da40 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f4600009ca0 tx=0x7f460002a950 comp rx=0 tx=0).stop 2024-09-19T09:20:30.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.537+0000 7f460ce45640 1 -- 172.21.15.120:0/3078384804 shutdown_connections 2024-09-19T09:20:30.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.537+0000 7f460ce45640 1 --2- 172.21.15.120:0/3078384804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4608102ef0 0x7f460810da40 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.537+0000 7f460ce45640 1 --2- 172.21.15.120:0/3078384804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4608102570 0x7f46081029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.537+0000 7f460ce45640 1 --2- 172.21.15.120:0/3078384804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4608100e20 0x7f4608101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.537+0000 7f460ce45640 1 -- 172.21.15.120:0/3078384804 >> 172.21.15.120:0/3078384804 conn(0x7f46080fc7d0 msgr2=0x7f46080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:30.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.537+0000 7f460ce45640 1 -- 172.21.15.120:0/3078384804 shutdown_connections 2024-09-19T09:20:30.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.537+0000 7f460ce45640 1 -- 172.21.15.120:0/3078384804 wait complete. 2024-09-19T09:20:30.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.538+0000 7f460ce45640 1 Processor -- start 2024-09-19T09:20:30.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.538+0000 7f460ce45640 1 -- start start 2024-09-19T09:20:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.538+0000 7f460ce45640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4608100e20 0x7f4608198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.538+0000 7f460ce45640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4608102570 0x7f4608198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.538+0000 7f460ce45640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4608102ef0 0x7f460819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.538+0000 7f460ce45640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4608110f80 con 0x7f4608102570 2024-09-19T09:20:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.538+0000 7f460ce45640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4608110e00 con 0x7f4608102ef0 2024-09-19T09:20:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.538+0000 7f460ce45640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4608111100 con 0x7f4608100e20 2024-09-19T09:20:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.538+0000 7f4605d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4608102570 0x7f4608198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.538+0000 7f4605d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4608102570 0x7f4608198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39584/0 (socket says 172.21.15.120:39584) 2024-09-19T09:20:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.538+0000 7f4605d9b640 1 -- 172.21.15.120:0/2433275665 learned_addr learned my addr 172.21.15.120:0/2433275665 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.538+0000 7f460659c640 1 --2- 172.21.15.120:0/2433275665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4608100e20 0x7f4608198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.538+0000 7f4606d9d640 1 --2- 172.21.15.120:0/2433275665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4608102ef0 0x7f460819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.539+0000 7f4605d9b640 1 -- 172.21.15.120:0/2433275665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4608100e20 msgr2=0x7f4608198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.539+0000 7f4605d9b640 1 --2- 172.21.15.120:0/2433275665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4608100e20 0x7f4608198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.539+0000 7f4605d9b640 1 -- 172.21.15.120:0/2433275665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4608102ef0 msgr2=0x7f460819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.539+0000 7f4605d9b640 1 --2- 172.21.15.120:0/2433275665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4608102ef0 0x7f460819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.539+0000 7f4605d9b640 1 -- 172.21.15.120:0/2433275665 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f460819d400 con 0x7f4608102570 2024-09-19T09:20:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.539+0000 7f4605d9b640 1 --2- 172.21.15.120:0/2433275665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4608102570 0x7f4608198990 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f45fc00d950 tx=0x7f45fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.539+0000 7f45f77fe640 1 -- 172.21.15.120:0/2433275665 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45fc017070 con 0x7f4608102570 2024-09-19T09:20:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.539+0000 7f460ce45640 1 -- 172.21.15.120:0/2433275665 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f460819d6f0 con 0x7f4608102570 2024-09-19T09:20:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.539+0000 7f460ce45640 1 -- 172.21.15.120:0/2433275665 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46081a4fd0 con 0x7f4608102570 2024-09-19T09:20:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.540+0000 7f45f77fe640 1 -- 172.21.15.120:0/2433275665 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45fc009c30 con 0x7f4608102570 2024-09-19T09:20:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.540+0000 7f45f77fe640 1 -- 172.21.15.120:0/2433275665 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45fc020060 con 0x7f4608102570 2024-09-19T09:20:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.542+0000 7f45f77fe640 1 -- 172.21.15.120:0/2433275665 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f45fc0109d0 con 0x7f4608102570 2024-09-19T09:20:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.543+0000 7f460f21e740 1 Processor -- start 2024-09-19T09:20:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.543+0000 7f460f21e740 1 -- start start 2024-09-19T09:20:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.543+0000 7f460f21e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604103614a0 0x560410361880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.543+0000 7f460f21e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560410361dc0 0x56041036c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.543+0000 7f460f21e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56041036c680 0x56041036ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.543+0000 7f460f21e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5604102c72e0 con 0x56041036c680 2024-09-19T09:20:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.543+0000 7f460f21e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56041026ece0 con 0x5604103614a0 2024-09-19T09:20:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.543+0000 7f460f21e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5604102c32a0 con 0x560410361dc0 2024-09-19T09:20:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.543+0000 7f4605d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56041036c680 0x56041036ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.543+0000 7f460659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560410361dc0 0x56041036c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.543+0000 7f4606d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604103614a0 0x560410361880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.543+0000 7f45f77fe640 1 --2- 172.21.15.120:0/2433275665 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f45f00a3240 0x7f45f00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.543+0000 7f4605d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56041036c680 0x56041036ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39600/0 (socket says 172.21.15.120:39600) 2024-09-19T09:20:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.543+0000 7f4605d9b640 1 -- 172.21.15.120:0/3628543210 learned_addr learned my addr 172.21.15.120:0/3628543210 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.544+0000 7f4606d9d640 1 --2- 172.21.15.120:0/2433275665 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f45f00a3240 0x7f45f00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.544+0000 7f45f77fe640 1 -- 172.21.15.120:0/2433275665 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f45fc0cf550 con 0x7f4608102570 2024-09-19T09:20:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.544+0000 7f460659c640 1 -- 172.21.15.120:0/3628543210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604103614a0 msgr2=0x560410361880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:30.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.544+0000 7f460659c640 1 --2- 172.21.15.120:0/3628543210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604103614a0 0x560410361880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.544+0000 7f460659c640 1 -- 172.21.15.120:0/3628543210 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56041036c680 msgr2=0x56041036ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:30.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.544+0000 7f460659c640 1 --2- 172.21.15.120:0/3628543210 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56041036c680 0x56041036ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.544+0000 7f460659c640 1 -- 172.21.15.120:0/3628543210 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560410253440 con 0x560410361dc0 2024-09-19T09:20:30.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.544+0000 7f460659c640 1 --2- 172.21.15.120:0/3628543210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560410361dc0 0x56041036c140 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f45f800bbf0 tx=0x7f45f80103d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c46a4c552bcce988 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:30.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.544+0000 7f4606d9d640 1 --2- 172.21.15.120:0/2433275665 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f45f00a3240 0x7f45f00a5700 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4600004ad0 tx=0x7f460002a950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:30.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.544+0000 7f45f4ff9640 1 -- 172.21.15.120:0/3628543210 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8004030 con 0x560410361dc0 2024-09-19T09:20:30.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.544+0000 7f45f4ff9640 1 -- 172.21.15.120:0/3628543210 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45f80041d0 con 0x560410361dc0 2024-09-19T09:20:30.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.544+0000 7f45f4ff9640 1 -- 172.21.15.120:0/3628543210 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f80116a0 con 0x560410361dc0 2024-09-19T09:20:30.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.545+0000 7f460f21e740 1 -- 172.21.15.120:0/3628543210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560410361dc0 msgr2=0x56041036c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:30.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.545+0000 7f460f21e740 1 --2- 172.21.15.120:0/3628543210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560410361dc0 0x56041036c140 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f45f800bbf0 tx=0x7f45f80103d0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.545+0000 7f460f21e740 1 -- 172.21.15.120:0/3628543210 shutdown_connections 2024-09-19T09:20:30.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.545+0000 7f460f21e740 1 --2- 172.21.15.120:0/3628543210 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56041036c680 0x56041036ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.545+0000 7f460f21e740 1 --2- 172.21.15.120:0/3628543210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560410361dc0 0x56041036c140 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.545+0000 7f460f21e740 1 --2- 172.21.15.120:0/3628543210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5604103614a0 0x560410361880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.545+0000 7f460f21e740 1 -- 172.21.15.120:0/3628543210 >> 172.21.15.120:0/3628543210 conn(0x56041035da30 msgr2=0x56041035de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:30.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.545+0000 7f460f21e740 1 -- 172.21.15.120:0/3628543210 shutdown_connections 2024-09-19T09:20:30.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.545+0000 7f460f21e740 1 -- 172.21.15.120:0/3628543210 wait complete. 2024-09-19T09:20:30.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.546+0000 7f460f21e740 1 -- start start 2024-09-19T09:20:30.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.546+0000 7f460f21e740 1 Processor -- start 2024-09-19T09:20:30.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.546+0000 7f460f21e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604103614a0 0x56041036c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:30.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.546+0000 7f460f21e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560410361dc0 0x5604103692d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:30.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.546+0000 7f460f21e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56041036c680 0x560410369870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:30.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.546+0000 7f460f21e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56041036f250 con 0x5604103614a0 2024-09-19T09:20:30.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.546+0000 7f460f21e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560410282050 con 0x560410361dc0 2024-09-19T09:20:30.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.546+0000 7f460f21e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56041036f3d0 con 0x56041036c680 2024-09-19T09:20:30.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.546+0000 7f460659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604103614a0 0x56041036c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:30.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.546+0000 7f460f21e740 10 client.0 _ll_register_callbacks cb 0x5604102c2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:20:30.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.546+0000 7f460659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604103614a0 0x56041036c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39604/0 (socket says 172.21.15.120:39604) 2024-09-19T09:20:30.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.546+0000 7f460659c640 1 -- 172.21.15.120:0/2198667099 learned_addr learned my addr 172.21.15.120:0/2198667099 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:30.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.546+0000 7f4605d9b640 1 --2- 172.21.15.120:0/2198667099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560410361dc0 0x5604103692d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:30.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.547+0000 7f4606d9d640 1 --2- 172.21.15.120:0/2198667099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56041036c680 0x560410369870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:30.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.547+0000 7f460659c640 1 -- 172.21.15.120:0/2198667099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56041036c680 msgr2=0x560410369870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:30.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.547+0000 7f460659c640 1 --2- 172.21.15.120:0/2198667099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56041036c680 0x560410369870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.547+0000 7f460659c640 1 -- 172.21.15.120:0/2198667099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560410361dc0 msgr2=0x5604103692d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:30.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.547+0000 7f460659c640 1 --2- 172.21.15.120:0/2198667099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560410361dc0 0x5604103692d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.547+0000 7f460659c640 1 -- 172.21.15.120:0/2198667099 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604100383c0 con 0x5604103614a0 2024-09-19T09:20:30.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.547+0000 7f4605d9b640 1 --2- 172.21.15.120:0/2198667099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560410361dc0 0x5604103692d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:20:30.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.547+0000 7f460659c640 1 --2- 172.21.15.120:0/2198667099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604103614a0 0x56041036c020 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f45f8005cd0 tx=0x7f45f8003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52e0fb1115b76e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:30.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.547+0000 7f45edffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:20:30.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.547+0000 7f45edffb640 1 -- 172.21.15.120:0/2198667099 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8003a40 con 0x5604103614a0 2024-09-19T09:20:30.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.547+0000 7f45edffb640 1 -- 172.21.15.120:0/2198667099 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f45f8010cc0 con 0x5604103614a0 2024-09-19T09:20:30.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.547+0000 7f460f21e740 1 -- 172.21.15.120:0/2198667099 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45f80041d0 con 0x5604103614a0 2024-09-19T09:20:30.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.547+0000 7f45edffb640 1 -- 172.21.15.120:0/2198667099 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f45f8005040 con 0x5604103614a0 2024-09-19T09:20:30.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.547+0000 7f460f21e740 10 client.21288 fetch_fsmap learned FSMap version 39 2024-09-19T09:20:30.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.547+0000 7f460f21e740 1 -- 172.21.15.120:0/2198667099 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56041037ac60 con 0x5604103614a0 2024-09-19T09:20:30.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.548+0000 7f45edffb640 1 -- 172.21.15.120:0/2198667099 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45f80046c0 con 0x5604103614a0 2024-09-19T09:20:30.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.548+0000 7f45edffb640 1 -- 172.21.15.120:0/2198667099 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f45f8011440 con 0x5604103614a0 2024-09-19T09:20:30.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.548+0000 7f460f21e740 10 client.21288 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:20:30.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.548+0000 7f460f21e740 10 client.21288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:20:30.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.548+0000 7f460f21e740 20 client.21288 populate_metadata read hostname 'smithi120' 2024-09-19T09:20:30.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.548+0000 7f460f21e740 1 --2- 172.21.15.120:0/2198667099 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56041037c450 0x56041037e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:30.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.548+0000 7f460f21e740 4 client.21288 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:20:30.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.548+0000 7f460f21e740 1 -- 172.21.15.120:0/2198667099 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f45f80041d0 con 0x56041037c450 2024-09-19T09:20:30.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.549+0000 7f4605d9b640 1 --2- 172.21.15.120:0/2198667099 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56041037c450 0x56041037e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:30.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.549+0000 7f4605d9b640 1 --2- 172.21.15.120:0/2198667099 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56041037c450 0x56041037e8a0 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a588721aabe34fa server_cookie=78badd80855e8cba in_seq=0 out_seq=0 2024-09-19T09:20:30.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.549+0000 7f45edffb640 10 client.21288 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:30.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.552+0000 7f45edffb640 1 -- 172.21.15.120:0/2198667099 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f45f80041d0 con 0x56041037c450 2024-09-19T09:20:30.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.552+0000 7f45edffb640 10 client.21288 handle_command_reply: tid=0 2024-09-19T09:20:30.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.552+0000 7f460f21e740 1 client.21288 shutdown 2024-09-19T09:20:30.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.552+0000 7f460f21e740 1 -- 172.21.15.120:0/2198667099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604103614a0 msgr2=0x56041036c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:30.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.552+0000 7f460f21e740 1 --2- 172.21.15.120:0/2198667099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604103614a0 0x56041036c020 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f45f8005cd0 tx=0x7f45f8003ef0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.552+0000 7f460f21e740 1 -- 172.21.15.120:0/2198667099 shutdown_connections 2024-09-19T09:20:30.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.552+0000 7f460f21e740 1 --2- 172.21.15.120:0/2198667099 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56041037c450 0x56041037e8a0 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.552+0000 7f460f21e740 1 --2- 172.21.15.120:0/2198667099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56041036c680 0x560410369870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.552+0000 7f460f21e740 1 --2- 172.21.15.120:0/2198667099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560410361dc0 0x5604103692d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.552+0000 7f460f21e740 1 --2- 172.21.15.120:0/2198667099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604103614a0 0x56041036c020 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.552+0000 7f460f21e740 1 -- 172.21.15.120:0/2198667099 >> 172.21.15.120:0/2198667099 conn(0x56041035da30 msgr2=0x560410363710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:30.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.552+0000 7f45edffb640 0 client.21288 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:30.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.552+0000 7f460f21e740 1 -- 172.21.15.120:0/2198667099 shutdown_connections 2024-09-19T09:20:30.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.552+0000 7f460f21e740 1 -- 172.21.15.120:0/2198667099 wait complete. 2024-09-19T09:20:30.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.553+0000 7f460f21e740 20 client.21288 trim_cache size 0 max 16384 2024-09-19T09:20:30.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.573+0000 7f460f21e740 1 Processor -- start 2024-09-19T09:20:30.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.573+0000 7f460f21e740 1 -- start start 2024-09-19T09:20:30.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.573+0000 7f460f21e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604103614a0 0x5604103784b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:30.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.573+0000 7f460f21e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560410361dc0 0x560410367ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:30.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.573+0000 7f460f21e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56041036c680 0x560410363600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:30.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.573+0000 7f460f21e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f46081ad5b0 con 0x5604103614a0 2024-09-19T09:20:30.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.573+0000 7f460f21e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f46081af0f0 con 0x56041036c680 2024-09-19T09:20:30.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.573+0000 7f460f21e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f46081adc80 con 0x560410361dc0 2024-09-19T09:20:30.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.573+0000 7f460659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604103614a0 0x5604103784b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:30.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.573+0000 7f460659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604103614a0 0x5604103784b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39616/0 (socket says 172.21.15.120:39616) 2024-09-19T09:20:30.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.573+0000 7f460659c640 1 -- 172.21.15.120:0/1731029484 learned_addr learned my addr 172.21.15.120:0/1731029484 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:30.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.573+0000 7f4605d9b640 1 --2- 172.21.15.120:0/1731029484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560410361dc0 0x560410367ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:30.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.573+0000 7f4606d9d640 1 --2- 172.21.15.120:0/1731029484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56041036c680 0x560410363600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:30.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.574+0000 7f460659c640 1 -- 172.21.15.120:0/1731029484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560410361dc0 msgr2=0x560410367ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:30.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.574+0000 7f460659c640 1 --2- 172.21.15.120:0/1731029484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560410361dc0 0x560410367ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.574+0000 7f460659c640 1 -- 172.21.15.120:0/1731029484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56041036c680 msgr2=0x560410363600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:30.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.574+0000 7f460659c640 1 --2- 172.21.15.120:0/1731029484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56041036c680 0x560410363600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.574+0000 7f460659c640 1 -- 172.21.15.120:0/1731029484 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560410368280 con 0x5604103614a0 2024-09-19T09:20:30.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.574+0000 7f460659c640 1 --2- 172.21.15.120:0/1731029484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604103614a0 0x5604103784b0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f45f8005d80 tx=0x7f45f801e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76cde4cbb39a4914 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:30.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.574+0000 7f45effff640 1 -- 172.21.15.120:0/1731029484 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8004600 con 0x5604103614a0 2024-09-19T09:20:30.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.574+0000 7f45effff640 1 -- 172.21.15.120:0/1731029484 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45f8004170 con 0x5604103614a0 2024-09-19T09:20:30.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.574+0000 7f45effff640 1 -- 172.21.15.120:0/1731029484 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8011ce0 con 0x5604103614a0 2024-09-19T09:20:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.575+0000 7f460f21e740 1 -- 172.21.15.120:0/1731029484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604103614a0 msgr2=0x5604103784b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.575+0000 7f460f21e740 1 --2- 172.21.15.120:0/1731029484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604103614a0 0x5604103784b0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f45f8005d80 tx=0x7f45f801e070 comp rx=0 tx=0).stop 2024-09-19T09:20:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.575+0000 7f460f21e740 1 -- 172.21.15.120:0/1731029484 shutdown_connections 2024-09-19T09:20:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.575+0000 7f460f21e740 1 --2- 172.21.15.120:0/1731029484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56041036c680 0x560410363600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.575+0000 7f460f21e740 1 --2- 172.21.15.120:0/1731029484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560410361dc0 0x560410367ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.575+0000 7f460f21e740 1 --2- 172.21.15.120:0/1731029484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5604103614a0 0x5604103784b0 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.575+0000 7f460f21e740 1 -- 172.21.15.120:0/1731029484 >> 172.21.15.120:0/1731029484 conn(0x56041035da30 msgr2=0x560410362ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:30.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.575+0000 7f460f21e740 1 -- 172.21.15.120:0/1731029484 shutdown_connections 2024-09-19T09:20:30.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.575+0000 7f460f21e740 1 -- 172.21.15.120:0/1731029484 wait complete. 2024-09-19T09:20:30.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.575+0000 7f460f21e740 1 -- start start 2024-09-19T09:20:30.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.575+0000 7f460f21e740 1 Processor -- start 2024-09-19T09:20:30.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.575+0000 7f460f21e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604103614a0 0x56041036fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:30.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.575+0000 7f460f21e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560410361dc0 0x5604103700c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:30.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.575+0000 7f460f21e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56041036c680 0x5604103e5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:30.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.575+0000 7f460f21e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f46081aea20 con 0x560410361dc0 2024-09-19T09:20:30.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.575+0000 7f460f21e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f46081ae350 con 0x56041036c680 2024-09-19T09:20:30.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.575+0000 7f460f21e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f46081ab570 con 0x5604103614a0 2024-09-19T09:20:30.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.576+0000 7f460f21e740 10 client.0 _ll_register_callbacks cb 0x7f4600009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:20:30.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.576+0000 7f4605d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560410361dc0 0x5604103700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:30.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.576+0000 7f4605d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560410361dc0 0x5604103700c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39620/0 (socket says 172.21.15.120:39620) 2024-09-19T09:20:30.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.576+0000 7f460659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604103614a0 0x56041036fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:30.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.576+0000 7f4605d9b640 1 -- 172.21.15.120:0/3169066260 learned_addr learned my addr 172.21.15.120:0/3169066260 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:30.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.576+0000 7f4606d9d640 1 --2- 172.21.15.120:0/3169066260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56041036c680 0x5604103e5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:30.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.576+0000 7f4605d9b640 1 -- 172.21.15.120:0/3169066260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604103614a0 msgr2=0x56041036fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:30.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.576+0000 7f4605d9b640 1 --2- 172.21.15.120:0/3169066260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604103614a0 0x56041036fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.576+0000 7f4605d9b640 1 -- 172.21.15.120:0/3169066260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56041036c680 msgr2=0x5604103e5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:30.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.576+0000 7f4605d9b640 1 --2- 172.21.15.120:0/3169066260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56041036c680 0x5604103e5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.576+0000 7f4605d9b640 1 -- 172.21.15.120:0/3169066260 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56041036c390 con 0x560410361dc0 2024-09-19T09:20:30.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.576+0000 7f4606d9d640 1 --2- 172.21.15.120:0/3169066260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56041036c680 0x5604103e5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:20:30.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.576+0000 7f460659c640 1 --2- 172.21.15.120:0/3169066260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604103614a0 0x56041036fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:20:30.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.576+0000 7f4605d9b640 1 --2- 172.21.15.120:0/3169066260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560410361dc0 0x5604103700c0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f45fc098040 tx=0x7f45fc086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d61f8f220c06db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:30.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.576+0000 7f45edffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:20:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.576+0000 7f45edffb640 1 -- 172.21.15.120:0/3169066260 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45fc09ce60 con 0x560410361dc0 2024-09-19T09:20:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.576+0000 7f45edffb640 1 -- 172.21.15.120:0/3169066260 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f45fc098250 con 0x560410361dc0 2024-09-19T09:20:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.576+0000 7f460f21e740 1 -- 172.21.15.120:0/3169066260 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45f8004170 con 0x560410361dc0 2024-09-19T09:20:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.577+0000 7f45edffb640 1 -- 172.21.15.120:0/3169066260 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f45fc09bbe0 con 0x560410361dc0 2024-09-19T09:20:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.577+0000 7f460f21e740 10 client.21294 fetch_fsmap learned FSMap version 39 2024-09-19T09:20:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.577+0000 7f460f21e740 1 -- 172.21.15.120:0/3169066260 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5604103e6760 con 0x560410361dc0 2024-09-19T09:20:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.577+0000 7f45edffb640 1 -- 172.21.15.120:0/3169066260 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45f8004170 con 0x560410361dc0 2024-09-19T09:20:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.577+0000 7f45edffb640 1 -- 172.21.15.120:0/3169066260 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f45fc0860b0 con 0x560410361dc0 2024-09-19T09:20:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.577+0000 7f460f21e740 10 client.21294 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:20:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.577+0000 7f460f21e740 10 client.21294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:20:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.577+0000 7f460f21e740 20 client.21294 populate_metadata read hostname 'smithi120' 2024-09-19T09:20:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.577+0000 7f460f21e740 1 --2- 172.21.15.120:0/3169066260 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5604103e6e10 0x5604103e9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.577+0000 7f460f21e740 4 client.21294 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:20:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.577+0000 7f460f21e740 1 -- 172.21.15.120:0/3169066260 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5604102d4850 con 0x5604103e6e10 2024-09-19T09:20:30.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.577+0000 7f460659c640 1 --2- 172.21.15.120:0/3169066260 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5604103e6e10 0x5604103e9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:30.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.577+0000 7f45edffb640 10 client.21294 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:30.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.577+0000 7f460659c640 1 --2- 172.21.15.120:0/3169066260 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5604103e6e10 0x5604103e9230 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39b3b2be2269e50e server_cookie=f7182c56cbb125be in_seq=0 out_seq=0 2024-09-19T09:20:30.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.581+0000 7f45edffb640 1 -- 172.21.15.120:0/3169066260 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5604102d4850 con 0x5604103e6e10 2024-09-19T09:20:30.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.581+0000 7f45edffb640 10 client.21294 handle_command_reply: tid=0 2024-09-19T09:20:30.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.581+0000 7f460f21e740 1 client.21294 shutdown 2024-09-19T09:20:30.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.581+0000 7f460f21e740 1 -- 172.21.15.120:0/3169066260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560410361dc0 msgr2=0x5604103700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:30.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.581+0000 7f460f21e740 1 --2- 172.21.15.120:0/3169066260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560410361dc0 0x5604103700c0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f45fc098040 tx=0x7f45fc086040 comp rx=0 tx=0).stop 2024-09-19T09:20:30.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.581+0000 7f460f21e740 1 -- 172.21.15.120:0/3169066260 shutdown_connections 2024-09-19T09:20:30.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.581+0000 7f460f21e740 1 --2- 172.21.15.120:0/3169066260 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5604103e6e10 0x5604103e9230 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.581+0000 7f460f21e740 1 --2- 172.21.15.120:0/3169066260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56041036c680 0x5604103e5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.581+0000 7f460f21e740 1 --2- 172.21.15.120:0/3169066260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560410361dc0 0x5604103700c0 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.581+0000 7f460f21e740 1 --2- 172.21.15.120:0/3169066260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604103614a0 0x56041036fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.581+0000 7f460f21e740 1 -- 172.21.15.120:0/3169066260 >> 172.21.15.120:0/3169066260 conn(0x56041035da30 msgr2=0x56041035c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:30.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.581+0000 7f45edffb640 0 client.21294 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:30.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.581+0000 7f460f21e740 1 -- 172.21.15.120:0/3169066260 shutdown_connections 2024-09-19T09:20:30.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.581+0000 7f460f21e740 1 -- 172.21.15.120:0/3169066260 wait complete. 2024-09-19T09:20:30.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.581+0000 7f460f21e740 20 client.21294 trim_cache size 0 max 16384 2024-09-19T09:20:30.585 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:20:30.586 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:20:30.586 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:20:30.586 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:20:30.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.582+0000 7f460ce45640 1 -- 172.21.15.120:0/2433275665 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f45f00a3240 msgr2=0x7f45f00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:30.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.582+0000 7f460ce45640 1 --2- 172.21.15.120:0/2433275665 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f45f00a3240 0x7f45f00a5700 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4600004ad0 tx=0x7f460002a950 comp rx=0 tx=0).stop 2024-09-19T09:20:30.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.582+0000 7f460ce45640 1 -- 172.21.15.120:0/2433275665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4608102570 msgr2=0x7f4608198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:30.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.582+0000 7f460ce45640 1 --2- 172.21.15.120:0/2433275665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4608102570 0x7f4608198990 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f45fc00d950 tx=0x7f45fc00de10 comp rx=0 tx=0).stop 2024-09-19T09:20:30.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.582+0000 7f460ce45640 1 -- 172.21.15.120:0/2433275665 shutdown_connections 2024-09-19T09:20:30.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.582+0000 7f460ce45640 1 --2- 172.21.15.120:0/2433275665 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f45f00a3240 0x7f45f00a5700 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.582+0000 7f460ce45640 1 --2- 172.21.15.120:0/2433275665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4608102ef0 0x7f460819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.582+0000 7f460ce45640 1 --2- 172.21.15.120:0/2433275665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4608102570 0x7f4608198990 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.582+0000 7f460ce45640 1 --2- 172.21.15.120:0/2433275665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4608100e20 0x7f4608198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.582+0000 7f460ce45640 1 -- 172.21.15.120:0/2433275665 >> 172.21.15.120:0/2433275665 conn(0x7f46080fc7d0 msgr2=0x7f460806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:30.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.582+0000 7f460ce45640 1 -- 172.21.15.120:0/2433275665 shutdown_connections 2024-09-19T09:20:30.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:30.582+0000 7f460ce45640 1 -- 172.21.15.120:0/2433275665 wait complete. 2024-09-19T09:20:31.367 INFO:tasks.workunit.client.0.smithi120.stdout: 10 111635 20.57 MB/sec execute 146 sec latency 50.552 ms 2024-09-19T09:20:31.407 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:20:31.407 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:20:31.735 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:20:32.137 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:32 smithi120 ceph-mon[31150]: pgmap v875: 97 pgs: 97 active+clean; 381 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.3 MiB/s wr, 805 op/s 2024-09-19T09:20:32.367 INFO:tasks.workunit.client.0.smithi120.stdout: 10 112217 20.63 MB/sec execute 147 sec latency 65.176 ms 2024-09-19T09:20:32.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:32 smithi196 ceph-mon[35150]: pgmap v875: 97 pgs: 97 active+clean; 381 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.3 MiB/s wr, 805 op/s 2024-09-19T09:20:32.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:32 smithi153 ceph-mon[35071]: pgmap v875: 97 pgs: 97 active+clean; 381 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.3 MiB/s wr, 805 op/s 2024-09-19T09:20:32.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.222+0000 7fb0bef1e640 1 -- 172.21.15.120:0/702173699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0b8100e20 msgr2=0x7fb0b8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:32.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.222+0000 7fb0bef1e640 1 --2- 172.21.15.120:0/702173699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0b8100e20 0x7fb0b8101200 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac009900 tx=0x7fb0ac02a480 comp rx=0 tx=0).stop 2024-09-19T09:20:32.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.222+0000 7fb0bef1e640 1 -- 172.21.15.120:0/702173699 shutdown_connections 2024-09-19T09:20:32.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.222+0000 7fb0bef1e640 1 --2- 172.21.15.120:0/702173699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0b8102ef0 0x7fb0b810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:32.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.222+0000 7fb0bef1e640 1 --2- 172.21.15.120:0/702173699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0b8102570 0x7fb0b81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:32.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.222+0000 7fb0bef1e640 1 --2- 172.21.15.120:0/702173699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0b8100e20 0x7fb0b8101200 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:32.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.222+0000 7fb0bef1e640 1 -- 172.21.15.120:0/702173699 >> 172.21.15.120:0/702173699 conn(0x7fb0b80fc7d0 msgr2=0x7fb0b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:32.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.222+0000 7fb0bef1e640 1 -- 172.21.15.120:0/702173699 shutdown_connections 2024-09-19T09:20:32.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.222+0000 7fb0bef1e640 1 -- 172.21.15.120:0/702173699 wait complete. 2024-09-19T09:20:32.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.223+0000 7fb0bef1e640 1 Processor -- start 2024-09-19T09:20:32.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.223+0000 7fb0bef1e640 1 -- start start 2024-09-19T09:20:32.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.223+0000 7fb0bef1e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0b8100e20 0x7fb0b8197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:32.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.223+0000 7fb0bef1e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0b8102570 0x7fb0b8197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:32.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.223+0000 7fb0bef1e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0b8102ef0 0x7fb0b819bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:32.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.223+0000 7fb0bef1e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b8110f50 con 0x7fb0b8102ef0 2024-09-19T09:20:32.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.223+0000 7fb0bef1e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b8110dd0 con 0x7fb0b8102570 2024-09-19T09:20:32.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.223+0000 7fb0bef1e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b81110d0 con 0x7fb0b8100e20 2024-09-19T09:20:32.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.223+0000 7fb0bccba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0b8100e20 0x7fb0b8197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:32.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.223+0000 7fb0bccba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0b8100e20 0x7fb0b8197450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:56916/0 (socket says 172.21.15.120:56916) 2024-09-19T09:20:32.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.223+0000 7fb0bccba640 1 -- 172.21.15.120:0/1628039012 learned_addr learned my addr 172.21.15.120:0/1628039012 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:32.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.223+0000 7fb0bd4bb640 1 --2- 172.21.15.120:0/1628039012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0b8102ef0 0x7fb0b819bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:32.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.223+0000 7fb0b7fff640 1 --2- 172.21.15.120:0/1628039012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0b8102570 0x7fb0b8197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:32.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.224+0000 7fb0bd4bb640 1 -- 172.21.15.120:0/1628039012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0b8100e20 msgr2=0x7fb0b8197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:32.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.224+0000 7fb0bd4bb640 1 --2- 172.21.15.120:0/1628039012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0b8100e20 0x7fb0b8197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:32.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.224+0000 7fb0bd4bb640 1 -- 172.21.15.120:0/1628039012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0b8102570 msgr2=0x7fb0b8197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:32.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.224+0000 7fb0bd4bb640 1 --2- 172.21.15.120:0/1628039012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0b8102570 0x7fb0b8197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:32.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.224+0000 7fb0bd4bb640 1 -- 172.21.15.120:0/1628039012 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b819c400 con 0x7fb0b8102ef0 2024-09-19T09:20:32.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.224+0000 7fb0bd4bb640 1 --2- 172.21.15.120:0/1628039012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0b8102ef0 0x7fb0b819bd20 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0009fd0 tx=0x7fb0b000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:32.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.224+0000 7fb0b5ffb640 1 -- 172.21.15.120:0/1628039012 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b000ce20 con 0x7fb0b8102ef0 2024-09-19T09:20:32.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.224+0000 7fb0b5ffb640 1 -- 172.21.15.120:0/1628039012 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0b0005020 con 0x7fb0b8102ef0 2024-09-19T09:20:32.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.224+0000 7fb0bef1e640 1 -- 172.21.15.120:0/1628039012 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b819c6f0 con 0x7fb0b8102ef0 2024-09-19T09:20:32.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.224+0000 7fb0b5ffb640 1 -- 172.21.15.120:0/1628039012 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0010720 con 0x7fb0b8102ef0 2024-09-19T09:20:32.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.224+0000 7fb0bef1e640 1 -- 172.21.15.120:0/1628039012 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b8069f70 con 0x7fb0b8102ef0 2024-09-19T09:20:32.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.225+0000 7fb0bef1e640 1 -- 172.21.15.120:0/1628039012 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0b81021b0 con 0x7fb0b8102ef0 2024-09-19T09:20:32.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.226+0000 7fb0b5ffb640 1 -- 172.21.15.120:0/1628039012 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fb0b0005540 con 0x7fb0b8102ef0 2024-09-19T09:20:32.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.227+0000 7fb0b5ffb640 1 --2- 172.21.15.120:0/1628039012 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb0a80a3240 0x7fb0a80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:32.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.227+0000 7fb0bccba640 1 --2- 172.21.15.120:0/1628039012 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb0a80a3240 0x7fb0a80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:32.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.227+0000 7fb0b5ffb640 1 -- 172.21.15.120:0/1628039012 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fb0b00cff10 con 0x7fb0b8102ef0 2024-09-19T09:20:32.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.228+0000 7fb0bccba640 1 --2- 172.21.15.120:0/1628039012 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb0a80a3240 0x7fb0a80a5700 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac005770 tx=0x7fb0ac005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:32.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.231+0000 7fb0b5ffb640 1 -- 172.21.15.120:0/1628039012 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb0b009cc30 con 0x7fb0b8102ef0 2024-09-19T09:20:32.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.497+0000 7fb0bef1e640 1 -- 172.21.15.120:0/1628039012 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0b8101200 con 0x7fb0b8102ef0 2024-09-19T09:20:32.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.498+0000 7fb0b5ffb640 1 -- 172.21.15.120:0/1628039012 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11988 (secure 0 0 0) 0x7fb0b8101200 con 0x7fb0b8102ef0 2024-09-19T09:20:32.499 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:20:32.499 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":39,"btime":"2024-09-19T09:18:53:120858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:18:53.070639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:20:32.499 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 39 2024-09-19T09:20:32.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.500+0000 7fb0bef1e640 1 -- 172.21.15.120:0/1628039012 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb0a80a3240 msgr2=0x7fb0a80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:32.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.500+0000 7fb0bef1e640 1 --2- 172.21.15.120:0/1628039012 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb0a80a3240 0x7fb0a80a5700 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac005770 tx=0x7fb0ac005a20 comp rx=0 tx=0).stop 2024-09-19T09:20:32.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.500+0000 7fb0bef1e640 1 -- 172.21.15.120:0/1628039012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0b8102ef0 msgr2=0x7fb0b819bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:32.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.500+0000 7fb0bef1e640 1 --2- 172.21.15.120:0/1628039012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0b8102ef0 0x7fb0b819bd20 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0009fd0 tx=0x7fb0b000ef10 comp rx=0 tx=0).stop 2024-09-19T09:20:32.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.500+0000 7fb0bef1e640 1 -- 172.21.15.120:0/1628039012 shutdown_connections 2024-09-19T09:20:32.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.500+0000 7fb0bef1e640 1 --2- 172.21.15.120:0/1628039012 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb0a80a3240 0x7fb0a80a5700 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:32.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.500+0000 7fb0bef1e640 1 --2- 172.21.15.120:0/1628039012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0b8102ef0 0x7fb0b819bd20 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:32.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.500+0000 7fb0bef1e640 1 --2- 172.21.15.120:0/1628039012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0b8102570 0x7fb0b8197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:32.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.500+0000 7fb0bef1e640 1 --2- 172.21.15.120:0/1628039012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0b8100e20 0x7fb0b8197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:32.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.500+0000 7fb0bef1e640 1 -- 172.21.15.120:0/1628039012 >> 172.21.15.120:0/1628039012 conn(0x7fb0b80fc7d0 msgr2=0x7fb0b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:32.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.500+0000 7fb0bef1e640 1 -- 172.21.15.120:0/1628039012 shutdown_connections 2024-09-19T09:20:32.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:32.500+0000 7fb0bef1e640 1 -- 172.21.15.120:0/1628039012 wait complete. 2024-09-19T09:20:33.176 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:33 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1628039012' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:20:33.210 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:20:33.367 INFO:tasks.workunit.client.0.smithi120.stdout: 10 112925 20.77 MB/sec execute 148 sec latency 110.581 ms 2024-09-19T09:20:33.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:33 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1628039012' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:20:33.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:33 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1628039012' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:20:33.563 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:20:34.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.088+0000 7fcb8b098640 1 -- 172.21.15.120:0/2065189774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb84100a80 msgr2=0x7fcb84100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:34.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.088+0000 7fcb8b098640 1 --2- 172.21.15.120:0/2065189774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb84100a80 0x7fcb84100ee0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fcb70009930 tx=0x7fcb7002a420 comp rx=0 tx=0).stop 2024-09-19T09:20:34.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.088+0000 7fcb8b098640 1 -- 172.21.15.120:0/2065189774 shutdown_connections 2024-09-19T09:20:34.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.088+0000 7fcb8b098640 1 --2- 172.21.15.120:0/2065189774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb84101420 0x7fcb8410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.088+0000 7fcb8b098640 1 --2- 172.21.15.120:0/2065189774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb84100a80 0x7fcb84100ee0 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.088+0000 7fcb8b098640 1 --2- 172.21.15.120:0/2065189774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb84106a80 0x7fcb84106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.088+0000 7fcb8b098640 1 -- 172.21.15.120:0/2065189774 >> 172.21.15.120:0/2065189774 conn(0x7fcb840fc7c0 msgr2=0x7fcb840febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:34.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.088+0000 7fcb8b098640 1 -- 172.21.15.120:0/2065189774 shutdown_connections 2024-09-19T09:20:34.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.088+0000 7fcb8b098640 1 -- 172.21.15.120:0/2065189774 wait complete. 2024-09-19T09:20:34.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.089+0000 7fcb8b098640 1 Processor -- start 2024-09-19T09:20:34.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.089+0000 7fcb8b098640 1 -- start start 2024-09-19T09:20:34.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.089+0000 7fcb8b098640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb84100a80 0x7fcb84198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.089+0000 7fcb8b098640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb84101420 0x7fcb84198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.089+0000 7fcb8b098640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb84106a80 0x7fcb8419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.089+0000 7fcb8b098640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb84110ed0 con 0x7fcb84106a80 2024-09-19T09:20:34.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.089+0000 7fcb8b098640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcb84110d50 con 0x7fcb84100a80 2024-09-19T09:20:34.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.089+0000 7fcb8b098640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcb84111050 con 0x7fcb84101420 2024-09-19T09:20:34.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.090+0000 7fcb88e34640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb84100a80 0x7fcb84198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.090+0000 7fcb83fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb84101420 0x7fcb84198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.090+0000 7fcb89635640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb84106a80 0x7fcb8419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.090+0000 7fcb88e34640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb84100a80 0x7fcb84198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:60376/0 (socket says 172.21.15.120:60376) 2024-09-19T09:20:34.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.090+0000 7fcb88e34640 1 -- 172.21.15.120:0/3887848557 learned_addr learned my addr 172.21.15.120:0/3887848557 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:34.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.090+0000 7fcb89635640 1 -- 172.21.15.120:0/3887848557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb84101420 msgr2=0x7fcb84198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:34.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.090+0000 7fcb89635640 1 --2- 172.21.15.120:0/3887848557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb84101420 0x7fcb84198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.090+0000 7fcb89635640 1 -- 172.21.15.120:0/3887848557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb84100a80 msgr2=0x7fcb84198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:34.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.090+0000 7fcb89635640 1 --2- 172.21.15.120:0/3887848557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb84100a80 0x7fcb84198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.090+0000 7fcb89635640 1 -- 172.21.15.120:0/3887848557 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8419d3d0 con 0x7fcb84106a80 2024-09-19T09:20:34.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.090+0000 7fcb89635640 1 --2- 172.21.15.120:0/3887848557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb84106a80 0x7fcb8419ccf0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c00e940 tx=0x7fcb7c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:34.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.090+0000 7fcb81ffb640 1 -- 172.21.15.120:0/3887848557 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c004d20 con 0x7fcb84106a80 2024-09-19T09:20:34.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.090+0000 7fcb8b098640 1 -- 172.21.15.120:0/3887848557 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb8419d6c0 con 0x7fcb84106a80 2024-09-19T09:20:34.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.090+0000 7fcb81ffb640 1 -- 172.21.15.120:0/3887848557 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb7c005020 con 0x7fcb84106a80 2024-09-19T09:20:34.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.090+0000 7fcb81ffb640 1 -- 172.21.15.120:0/3887848557 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c0116d0 con 0x7fcb84106a80 2024-09-19T09:20:34.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.091+0000 7fcb8b098640 1 -- 172.21.15.120:0/3887848557 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb840fcd90 con 0x7fcb84106a80 2024-09-19T09:20:34.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.094+0000 7fcb8d471740 1 Processor -- start 2024-09-19T09:20:34.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.095+0000 7fcb81ffb640 1 -- 172.21.15.120:0/3887848557 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fcb7c0054d0 con 0x7fcb84106a80 2024-09-19T09:20:34.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.095+0000 7fcb8d471740 1 -- start start 2024-09-19T09:20:34.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.095+0000 7fcb8d471740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561af87dfb40 0x561af87de090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.095+0000 7fcb8d471740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af87de5d0 0x561af87dea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.095+0000 7fcb88e34640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af87de5d0 0x561af87dea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.095+0000 7fcb88e34640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af87de5d0 0x561af87dea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40062/0 (socket says 172.21.15.120:40062) 2024-09-19T09:20:34.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.095+0000 7fcb83fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561af87dfb40 0x561af87de090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.095+0000 7fcb83fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561af87dfb40 0x561af87de090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:56938/0 (socket says 172.21.15.120:56938) 2024-09-19T09:20:34.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.096+0000 7fcb8d471740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561af87e9370 0x561af87eb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.096+0000 7fcb8d471740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561af87442f0 con 0x561af87de5d0 2024-09-19T09:20:34.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.096+0000 7fcb8d471740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561af86ebcf0 con 0x561af87e9370 2024-09-19T09:20:34.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.096+0000 7fcb8d471740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561af87402b0 con 0x561af87dfb40 2024-09-19T09:20:34.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.096+0000 7fcb88e34640 1 -- 172.21.15.120:0/3996116432 learned_addr learned my addr 172.21.15.120:0/3996116432 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:34.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.096+0000 7fcb83fff640 1 --2- 172.21.15.120:0/3996116432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561af87e9370 0x561af87eb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.096+0000 7fcb81ffb640 1 --2- 172.21.15.120:0/3887848557 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcb740a3240 0x7fcb740a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.097+0000 7fcb81ffb640 1 -- 172.21.15.120:0/3887848557 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fcb7c0cfcd0 con 0x7fcb84106a80 2024-09-19T09:20:34.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.097+0000 7fcb89635640 1 --2- 172.21.15.120:0/3887848557 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcb740a3240 0x7fcb740a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.097+0000 7fcb88e34640 1 -- 172.21.15.120:0/3996116432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561af87dfb40 msgr2=0x561af87de090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:34.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.097+0000 7fcb88e34640 1 --2- 172.21.15.120:0/3996116432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561af87dfb40 0x561af87de090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.097+0000 7fcb88e34640 1 -- 172.21.15.120:0/3996116432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561af87e9370 msgr2=0x561af87eb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:34.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.097+0000 7fcb88e34640 1 --2- 172.21.15.120:0/3996116432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561af87e9370 0x561af87eb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.097+0000 7fcb88e34640 1 -- 172.21.15.120:0/3996116432 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561af86d03a0 con 0x561af87de5d0 2024-09-19T09:20:34.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.097+0000 7fcb83fff640 1 --2- 172.21.15.120:0/3996116432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561af87e9370 0x561af87eb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:20:34.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.097+0000 7fcb88e34640 1 --2- 172.21.15.120:0/3996116432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af87de5d0 0x561af87dea10 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fcb78010e80 tx=0x7fcb78005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201eeafd1b300861 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:34.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.098+0000 7fcb6effd640 1 -- 172.21.15.120:0/3996116432 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7801e070 con 0x561af87de5d0 2024-09-19T09:20:34.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.098+0000 7fcb6effd640 1 -- 172.21.15.120:0/3996116432 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb78005500 con 0x561af87de5d0 2024-09-19T09:20:34.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.098+0000 7fcb6effd640 1 -- 172.21.15.120:0/3996116432 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb780035f0 con 0x561af87de5d0 2024-09-19T09:20:34.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.099+0000 7fcb8d471740 1 -- 172.21.15.120:0/3996116432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af87de5d0 msgr2=0x561af87dea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:34.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.099+0000 7fcb8d471740 1 --2- 172.21.15.120:0/3996116432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af87de5d0 0x561af87dea10 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fcb78010e80 tx=0x7fcb78005f90 comp rx=0 tx=0).stop 2024-09-19T09:20:34.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.099+0000 7fcb89635640 1 --2- 172.21.15.120:0/3887848557 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcb740a3240 0x7fcb740a5700 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c005fd0 tx=0x7fcb7c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:34.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.099+0000 7fcb8d471740 1 -- 172.21.15.120:0/3996116432 shutdown_connections 2024-09-19T09:20:34.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.099+0000 7fcb8d471740 1 --2- 172.21.15.120:0/3996116432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561af87e9370 0x561af87eb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.099+0000 7fcb8d471740 1 --2- 172.21.15.120:0/3996116432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af87de5d0 0x561af87dea10 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.099+0000 7fcb8d471740 1 --2- 172.21.15.120:0/3996116432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561af87dfb40 0x561af87de090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.099+0000 7fcb8d471740 1 -- 172.21.15.120:0/3996116432 >> 172.21.15.120:0/3996116432 conn(0x561af87a7310 msgr2=0x561af87dac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:34.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.099+0000 7fcb8d471740 1 -- 172.21.15.120:0/3996116432 shutdown_connections 2024-09-19T09:20:34.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.099+0000 7fcb8d471740 1 -- 172.21.15.120:0/3996116432 wait complete. 2024-09-19T09:20:34.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.099+0000 7fcb8d471740 1 -- start start 2024-09-19T09:20:34.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.100+0000 7fcb8d471740 1 Processor -- start 2024-09-19T09:20:34.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.100+0000 7fcb8d471740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561af87de5d0 0x561af87e8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.100+0000 7fcb8d471740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561af87dfb40 0x561af87e5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.100+0000 7fcb8d471740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af87e9370 0x561af87e64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.100+0000 7fcb8d471740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561af87ec0c0 con 0x561af87e9370 2024-09-19T09:20:34.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.100+0000 7fcb8d471740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561af86ff060 con 0x561af87de5d0 2024-09-19T09:20:34.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.100+0000 7fcb8d471740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561af87ec240 con 0x561af87dfb40 2024-09-19T09:20:34.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.100+0000 7fcb8d471740 10 client.0 _ll_register_callbacks cb 0x561af84528c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:20:34.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.101+0000 7fcb88e34640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561af87dfb40 0x561af87e5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.101+0000 7fcb88e34640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561af87dfb40 0x561af87e5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:56948/0 (socket says 172.21.15.120:56948) 2024-09-19T09:20:34.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.101+0000 7fcb88e34640 1 -- 172.21.15.120:0/1411779706 learned_addr learned my addr 172.21.15.120:0/1411779706 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:34.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.101+0000 7fcb83fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561af87de5d0 0x561af87e8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.101+0000 7fcb83fff640 1 --2- 172.21.15.120:0/1411779706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af87e9370 0x561af87e64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.101+0000 7fcb83fff640 1 -- 172.21.15.120:0/1411779706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561af87dfb40 msgr2=0x561af87e5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:34.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.101+0000 7fcb83fff640 1 --2- 172.21.15.120:0/1411779706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561af87dfb40 0x561af87e5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.101+0000 7fcb83fff640 1 -- 172.21.15.120:0/1411779706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af87e9370 msgr2=0x561af87e64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:34.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.101+0000 7fcb83fff640 1 --2- 172.21.15.120:0/1411779706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af87e9370 0x561af87e64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.101+0000 7fcb83fff640 1 -- 172.21.15.120:0/1411779706 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561af84b53c0 con 0x561af87de5d0 2024-09-19T09:20:34.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.102+0000 7fcb83fff640 1 --2- 172.21.15.120:0/1411779706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561af87de5d0 0x561af87e8cc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fcb7002a940 tx=0x7fcb70005e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb720a6e0a501b42 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:34.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.102+0000 7fcb63fff640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:20:34.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.102+0000 7fcb63fff640 1 -- 172.21.15.120:0/1411779706 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb70035040 con 0x561af87de5d0 2024-09-19T09:20:34.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.102+0000 7fcb8d471740 1 -- 172.21.15.120:0/1411779706 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb78005500 con 0x561af87de5d0 2024-09-19T09:20:34.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.102+0000 7fcb63fff640 1 -- 172.21.15.120:0/1411779706 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fcb700046c0 con 0x561af87de5d0 2024-09-19T09:20:34.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.102+0000 7fcb63fff640 1 -- 172.21.15.120:0/1411779706 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcb70035040 con 0x561af87de5d0 2024-09-19T09:20:34.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.103+0000 7fcb8d471740 10 client.30173 fetch_fsmap learned FSMap version 39 2024-09-19T09:20:34.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.103+0000 7fcb8d471740 1 -- 172.21.15.120:0/1411779706 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561af87f7ae0 con 0x561af87de5d0 2024-09-19T09:20:34.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.103+0000 7fcb63fff640 1 -- 172.21.15.120:0/1411779706 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb78005500 con 0x561af87de5d0 2024-09-19T09:20:34.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.103+0000 7fcb63fff640 1 -- 172.21.15.120:0/1411779706 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fcb70003960 con 0x561af87de5d0 2024-09-19T09:20:34.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.103+0000 7fcb8d471740 10 client.30173 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:20:34.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.103+0000 7fcb8d471740 10 client.30173 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:20:34.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.103+0000 7fcb8d471740 20 client.30173 populate_metadata read hostname 'smithi120' 2024-09-19T09:20:34.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.103+0000 7fcb8d471740 1 --2- 172.21.15.120:0/1411779706 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x561af87f92d0 0x561af87fb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.103+0000 7fcb8d471740 4 client.30173 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:20:34.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.103+0000 7fcb8d471740 1 -- 172.21.15.120:0/1411779706 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561af8751860 con 0x561af87f92d0 2024-09-19T09:20:34.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.104+0000 7fcb88e34640 1 --2- 172.21.15.120:0/1411779706 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x561af87f92d0 0x561af87fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.104+0000 7fcb63fff640 10 client.30173 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:20:34.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.104+0000 7fcb88e34640 1 --2- 172.21.15.120:0/1411779706 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x561af87f92d0 0x561af87fb720 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fd405e29a4ce8c0d server_cookie=775ab6186b68ef41 in_seq=0 out_seq=0 2024-09-19T09:20:34.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.105+0000 7fcb63fff640 1 -- 172.21.15.120:0/1411779706 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561af8751860 con 0x561af87f92d0 2024-09-19T09:20:34.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.105+0000 7fcb63fff640 10 client.30173 handle_command_reply: tid=0 2024-09-19T09:20:34.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.105+0000 7fcb8d471740 1 client.30173 shutdown 2024-09-19T09:20:34.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.105+0000 7fcb8d471740 1 -- 172.21.15.120:0/1411779706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561af87de5d0 msgr2=0x561af87e8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:34.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.105+0000 7fcb8d471740 1 --2- 172.21.15.120:0/1411779706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561af87de5d0 0x561af87e8cc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fcb7002a940 tx=0x7fcb70005e30 comp rx=0 tx=0).stop 2024-09-19T09:20:34.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.105+0000 7fcb8d471740 1 -- 172.21.15.120:0/1411779706 shutdown_connections 2024-09-19T09:20:34.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.105+0000 7fcb8d471740 1 --2- 172.21.15.120:0/1411779706 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x561af87f92d0 0x561af87fb720 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.105+0000 7fcb8d471740 1 --2- 172.21.15.120:0/1411779706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af87e9370 0x561af87e64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.105+0000 7fcb8d471740 1 --2- 172.21.15.120:0/1411779706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561af87dfb40 0x561af87e5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.105+0000 7fcb8d471740 1 --2- 172.21.15.120:0/1411779706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561af87de5d0 0x561af87e8cc0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.105+0000 7fcb63fff640 0 client.30173 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:20:34.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.105+0000 7fcb8d471740 1 -- 172.21.15.120:0/1411779706 >> 172.21.15.120:0/1411779706 conn(0x561af87a7310 msgr2=0x561af87e0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:34.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.105+0000 7fcb8d471740 1 -- 172.21.15.120:0/1411779706 shutdown_connections 2024-09-19T09:20:34.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.105+0000 7fcb8d471740 1 -- 172.21.15.120:0/1411779706 wait complete. 2024-09-19T09:20:34.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.106+0000 7fcb8d471740 20 client.30173 trim_cache size 0 max 16384 2024-09-19T09:20:34.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.124+0000 7fcb8d471740 1 Processor -- start 2024-09-19T09:20:34.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.124+0000 7fcb8d471740 1 -- start start 2024-09-19T09:20:34.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.124+0000 7fcb8d471740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561af87de5d0 0x561af87f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.124+0000 7fcb8d471740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561af87dfb40 0x561af87ea760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.124+0000 7fcb8d471740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af87e9370 0x561af87eaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.124+0000 7fcb8d471740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb841ab700 con 0x561af87e9370 2024-09-19T09:20:34.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.124+0000 7fcb8d471740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcb841ad240 con 0x561af87de5d0 2024-09-19T09:20:34.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.125+0000 7fcb8d471740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcb841abdd0 con 0x561af87dfb40 2024-09-19T09:20:34.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.125+0000 7fcb88e34640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561af87dfb40 0x561af87ea760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.125+0000 7fcb88e34640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561af87dfb40 0x561af87ea760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:56950/0 (socket says 172.21.15.120:56950) 2024-09-19T09:20:34.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.125+0000 7fcb83fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561af87de5d0 0x561af87f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.125+0000 7fcb88e34640 1 -- 172.21.15.120:0/2382265119 learned_addr learned my addr 172.21.15.120:0/2382265119 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:34.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.125+0000 7fcb83fff640 1 --2- 172.21.15.120:0/2382265119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af87e9370 0x561af87eaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.125+0000 7fcb83fff640 1 -- 172.21.15.120:0/2382265119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561af87dfb40 msgr2=0x561af87ea760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:34.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.125+0000 7fcb83fff640 1 --2- 172.21.15.120:0/2382265119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561af87dfb40 0x561af87ea760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.125+0000 7fcb83fff640 1 -- 172.21.15.120:0/2382265119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af87e9370 msgr2=0x561af87eaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:34.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.125+0000 7fcb83fff640 1 --2- 172.21.15.120:0/2382265119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af87e9370 0x561af87eaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.125+0000 7fcb83fff640 1 -- 172.21.15.120:0/2382265119 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561af87db960 con 0x561af87de5d0 2024-09-19T09:20:34.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.126+0000 7fcb83fff640 1 --2- 172.21.15.120:0/2382265119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561af87de5d0 0x561af87f53b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fcb70048450 tx=0x7fcb70005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b44e688bbe2ca29c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:34.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.126+0000 7fcb6e7fc640 1 -- 172.21.15.120:0/2382265119 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb70035040 con 0x561af87de5d0 2024-09-19T09:20:34.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.126+0000 7fcb6e7fc640 1 -- 172.21.15.120:0/2382265119 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb70005610 con 0x561af87de5d0 2024-09-19T09:20:34.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.126+0000 7fcb6e7fc640 1 -- 172.21.15.120:0/2382265119 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb70003c80 con 0x561af87de5d0 2024-09-19T09:20:34.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.127+0000 7fcb8d471740 1 -- 172.21.15.120:0/2382265119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561af87de5d0 msgr2=0x561af87f53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:34.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.127+0000 7fcb8d471740 1 --2- 172.21.15.120:0/2382265119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561af87de5d0 0x561af87f53b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fcb70048450 tx=0x7fcb70005fd0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.127+0000 7fcb8d471740 1 -- 172.21.15.120:0/2382265119 shutdown_connections 2024-09-19T09:20:34.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.127+0000 7fcb8d471740 1 --2- 172.21.15.120:0/2382265119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af87e9370 0x561af87eaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.127+0000 7fcb8d471740 1 --2- 172.21.15.120:0/2382265119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561af87dfb40 0x561af87ea760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.127+0000 7fcb8d471740 1 --2- 172.21.15.120:0/2382265119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561af87de5d0 0x561af87f53b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.127+0000 7fcb8d471740 1 -- 172.21.15.120:0/2382265119 >> 172.21.15.120:0/2382265119 conn(0x561af87a7310 msgr2=0x561af87e54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:34.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.127+0000 7fcb8d471740 1 -- 172.21.15.120:0/2382265119 shutdown_connections 2024-09-19T09:20:34.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.127+0000 7fcb8d471740 1 -- 172.21.15.120:0/2382265119 wait complete. 2024-09-19T09:20:34.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.128+0000 7fcb8d471740 1 -- start start 2024-09-19T09:20:34.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.128+0000 7fcb8d471740 1 Processor -- start 2024-09-19T09:20:34.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.128+0000 7fcb8d471740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561af87de5d0 0x561af88017d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.128+0000 7fcb8d471740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561af87dfb40 0x561af87f1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.128+0000 7fcb8d471740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af87e9370 0x561af87f1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.128+0000 7fcb8d471740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb841acb70 con 0x561af87e9370 2024-09-19T09:20:34.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.128+0000 7fcb8d471740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcb841ac4a0 con 0x561af87de5d0 2024-09-19T09:20:34.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.128+0000 7fcb83fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561af87de5d0 0x561af88017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.128+0000 7fcb8d471740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcb841a9680 con 0x561af87dfb40 2024-09-19T09:20:34.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.128+0000 7fcb83fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561af87de5d0 0x561af88017d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:60418/0 (socket says 172.21.15.120:60418) 2024-09-19T09:20:34.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.129+0000 7fcb83fff640 1 -- 172.21.15.120:0/2710880804 learned_addr learned my addr 172.21.15.120:0/2710880804 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:34.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.129+0000 7fcb88e34640 1 --2- 172.21.15.120:0/2710880804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561af87dfb40 0x561af87f1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.129+0000 7fcb8d471740 10 client.0 _ll_register_callbacks cb 0x7fcb700044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:20:34.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.129+0000 7fcb83fff640 1 --2- 172.21.15.120:0/2710880804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af87e9370 0x561af87f1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.129+0000 7fcb88e34640 1 -- 172.21.15.120:0/2710880804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561af87de5d0 msgr2=0x561af88017d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:34.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.129+0000 7fcb88e34640 1 --2- 172.21.15.120:0/2710880804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561af87de5d0 0x561af88017d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.129+0000 7fcb88e34640 1 -- 172.21.15.120:0/2710880804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af87e9370 msgr2=0x561af87f1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:34.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.129+0000 7fcb88e34640 1 --2- 172.21.15.120:0/2710880804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af87e9370 0x561af87f1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.129+0000 7fcb88e34640 1 -- 172.21.15.120:0/2710880804 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561af87e3210 con 0x561af87dfb40 2024-09-19T09:20:34.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.129+0000 7fcb83fff640 1 --2- 172.21.15.120:0/2710880804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af87e9370 0x561af87f1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:20:34.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.129+0000 7fcb88e34640 1 --2- 172.21.15.120:0/2710880804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561af87dfb40 0x561af87f1520 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcb78012010 tx=0x7fcb78005f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e14243e585e9ca09 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:34.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.129+0000 7fcb63fff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:20:34.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.129+0000 7fcb63fff640 1 -- 172.21.15.120:0/2710880804 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7801e030 con 0x561af87dfb40 2024-09-19T09:20:34.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.130+0000 7fcb63fff640 1 -- 172.21.15.120:0/2710880804 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fcb78004240 con 0x561af87dfb40 2024-09-19T09:20:34.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.130+0000 7fcb8d471740 1 -- 172.21.15.120:0/2710880804 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb70005610 con 0x561af87dfb40 2024-09-19T09:20:34.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.130+0000 7fcb63fff640 1 -- 172.21.15.120:0/2710880804 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcb78004440 con 0x561af87dfb40 2024-09-19T09:20:34.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.130+0000 7fcb8d471740 10 client.29995 fetch_fsmap learned FSMap version 39 2024-09-19T09:20:34.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.130+0000 7fcb8d471740 1 -- 172.21.15.120:0/2710880804 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561af8800250 con 0x561af87dfb40 2024-09-19T09:20:34.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.130+0000 7fcb63fff640 1 -- 172.21.15.120:0/2710880804 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb70005610 con 0x561af87dfb40 2024-09-19T09:20:34.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.130+0000 7fcb63fff640 1 -- 172.21.15.120:0/2710880804 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fcb78035050 con 0x561af87dfb40 2024-09-19T09:20:34.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.130+0000 7fcb8d471740 10 client.29995 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:20:34.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.130+0000 7fcb8d471740 10 client.29995 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:20:34.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.130+0000 7fcb8d471740 20 client.29995 populate_metadata read hostname 'smithi120' 2024-09-19T09:20:34.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.130+0000 7fcb8d471740 1 --2- 172.21.15.120:0/2710880804 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x561af8800810 0x561af886e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.130+0000 7fcb8d471740 4 client.29995 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:20:34.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.130+0000 7fcb8d471740 1 -- 172.21.15.120:0/2710880804 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561af874bd70 con 0x561af8800810 2024-09-19T09:20:34.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.131+0000 7fcb83fff640 1 --2- 172.21.15.120:0/2710880804 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x561af8800810 0x561af886e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.132+0000 7fcb83fff640 1 --2- 172.21.15.120:0/2710880804 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x561af8800810 0x561af886e820 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3d271f3a324ab870 server_cookie=85bace503f94e7d7 in_seq=0 out_seq=0 2024-09-19T09:20:34.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.132+0000 7fcb63fff640 10 client.29995 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:20:34.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.132+0000 7fcb63fff640 1 -- 172.21.15.120:0/2710880804 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561af874bd70 con 0x561af8800810 2024-09-19T09:20:34.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.132+0000 7fcb63fff640 10 client.29995 handle_command_reply: tid=0 2024-09-19T09:20:34.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.132+0000 7fcb8d471740 1 client.29995 shutdown 2024-09-19T09:20:34.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.133+0000 7fcb8d471740 1 -- 172.21.15.120:0/2710880804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561af87dfb40 msgr2=0x561af87f1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:34.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.133+0000 7fcb8d471740 1 --2- 172.21.15.120:0/2710880804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561af87dfb40 0x561af87f1520 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcb78012010 tx=0x7fcb78005f50 comp rx=0 tx=0).stop 2024-09-19T09:20:34.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.133+0000 7fcb8d471740 1 -- 172.21.15.120:0/2710880804 shutdown_connections 2024-09-19T09:20:34.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.133+0000 7fcb8d471740 1 --2- 172.21.15.120:0/2710880804 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x561af8800810 0x561af886e820 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.133+0000 7fcb8d471740 1 --2- 172.21.15.120:0/2710880804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af87e9370 0x561af87f1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.133+0000 7fcb8d471740 1 --2- 172.21.15.120:0/2710880804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561af87dfb40 0x561af87f1520 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.133+0000 7fcb8d471740 1 --2- 172.21.15.120:0/2710880804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561af87de5d0 0x561af88017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.133+0000 7fcb8d471740 1 -- 172.21.15.120:0/2710880804 >> 172.21.15.120:0/2710880804 conn(0x561af87a7310 msgr2=0x561af87e47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:34.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.133+0000 7fcb63fff640 0 client.29995 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:20:34.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.134+0000 7fcb8d471740 1 -- 172.21.15.120:0/2710880804 shutdown_connections 2024-09-19T09:20:34.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.134+0000 7fcb8d471740 1 -- 172.21.15.120:0/2710880804 wait complete. 2024-09-19T09:20:34.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.134+0000 7fcb8d471740 20 client.29995 trim_cache size 0 max 16384 2024-09-19T09:20:34.139 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:20:34.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.136+0000 7fcb8b098640 1 -- 172.21.15.120:0/3887848557 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcb740a3240 msgr2=0x7fcb740a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:34.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.136+0000 7fcb8b098640 1 --2- 172.21.15.120:0/3887848557 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcb740a3240 0x7fcb740a5700 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c005fd0 tx=0x7fcb7c0058d0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.136+0000 7fcb8b098640 1 -- 172.21.15.120:0/3887848557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb84106a80 msgr2=0x7fcb8419ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:34.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.136+0000 7fcb8b098640 1 --2- 172.21.15.120:0/3887848557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb84106a80 0x7fcb8419ccf0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c00e940 tx=0x7fcb7c00ee00 comp rx=0 tx=0).stop 2024-09-19T09:20:34.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.137+0000 7fcb8b098640 1 -- 172.21.15.120:0/3887848557 shutdown_connections 2024-09-19T09:20:34.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.137+0000 7fcb8b098640 1 --2- 172.21.15.120:0/3887848557 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcb740a3240 0x7fcb740a5700 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.137+0000 7fcb8b098640 1 --2- 172.21.15.120:0/3887848557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb84106a80 0x7fcb8419ccf0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.137+0000 7fcb8b098640 1 --2- 172.21.15.120:0/3887848557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb84101420 0x7fcb84198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.137+0000 7fcb8b098640 1 --2- 172.21.15.120:0/3887848557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb84100a80 0x7fcb84198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.137+0000 7fcb8b098640 1 -- 172.21.15.120:0/3887848557 >> 172.21.15.120:0/3887848557 conn(0x7fcb840fc7c0 msgr2=0x7fcb8406a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:34.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.137+0000 7fcb8b098640 1 -- 172.21.15.120:0/3887848557 shutdown_connections 2024-09-19T09:20:34.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:34.137+0000 7fcb8b098640 1 -- 172.21.15.120:0/3887848557 wait complete. 2024-09-19T09:20:34.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:34 smithi120 ceph-mon[31150]: pgmap v876: 97 pgs: 97 active+clean; 378 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.4 MiB/s wr, 713 op/s 2024-09-19T09:20:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.343+0000 7fef55c0e640 1 -- 172.21.15.120:0/1545447900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef50101c30 msgr2=0x7fef501020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.343+0000 7fef55c0e640 1 --2- 172.21.15.120:0/1545447900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef50101c30 0x7fef501020b0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fef44009900 tx=0x7fef4402a410 comp rx=0 tx=0).stop 2024-09-19T09:20:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.343+0000 7fef55c0e640 1 -- 172.21.15.120:0/1545447900 shutdown_connections 2024-09-19T09:20:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.343+0000 7fef55c0e640 1 --2- 172.21.15.120:0/1545447900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef501025f0 0x7fef50108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.343+0000 7fef55c0e640 1 --2- 172.21.15.120:0/1545447900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef50101c30 0x7fef501020b0 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.343+0000 7fef55c0e640 1 --2- 172.21.15.120:0/1545447900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef50100a40 0x7fef50100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.343+0000 7fef55c0e640 1 -- 172.21.15.120:0/1545447900 >> 172.21.15.120:0/1545447900 conn(0x7fef500fc1e0 msgr2=0x7fef500fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.343+0000 7fef55c0e640 1 -- 172.21.15.120:0/1545447900 shutdown_connections 2024-09-19T09:20:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.343+0000 7fef55c0e640 1 -- 172.21.15.120:0/1545447900 wait complete. 2024-09-19T09:20:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.344+0000 7fef55c0e640 1 Processor -- start 2024-09-19T09:20:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.344+0000 7fef55c0e640 1 -- start start 2024-09-19T09:20:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.344+0000 7fef55c0e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef50100a40 0x7fef50197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.344+0000 7fef55c0e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef50101c30 0x7fef50198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.344+0000 7fef55c0e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef501025f0 0x7fef5019f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.344+0000 7fef55c0e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fef5010c6c0 con 0x7fef50101c30 2024-09-19T09:20:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.344+0000 7fef55c0e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fef5010c540 con 0x7fef501025f0 2024-09-19T09:20:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.344+0000 7fef55c0e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fef5010c840 con 0x7fef50100a40 2024-09-19T09:20:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.344+0000 7fef4effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef50101c30 0x7fef50198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.344+0000 7fef4f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef50100a40 0x7fef50197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.344+0000 7fef4effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef50101c30 0x7fef50198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40108/0 (socket says 172.21.15.120:40108) 2024-09-19T09:20:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.344+0000 7fef4effd640 1 -- 172.21.15.120:0/109167187 learned_addr learned my addr 172.21.15.120:0/109167187 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.344+0000 7fef4ffff640 1 --2- 172.21.15.120:0/109167187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef501025f0 0x7fef5019f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.345+0000 7fef4effd640 1 -- 172.21.15.120:0/109167187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef50100a40 msgr2=0x7fef50197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.345+0000 7fef4effd640 1 --2- 172.21.15.120:0/109167187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef50100a40 0x7fef50197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.345+0000 7fef4effd640 1 -- 172.21.15.120:0/109167187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef501025f0 msgr2=0x7fef5019f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.345+0000 7fef4effd640 1 --2- 172.21.15.120:0/109167187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef501025f0 0x7fef5019f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.345+0000 7fef4effd640 1 -- 172.21.15.120:0/109167187 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef5019fa10 con 0x7fef50101c30 2024-09-19T09:20:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.345+0000 7fef4effd640 1 --2- 172.21.15.120:0/109167187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef50101c30 0x7fef50198330 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fef44009230 tx=0x7fef440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.345+0000 7fef4cff9640 1 -- 172.21.15.120:0/109167187 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4403f030 con 0x7fef50101c30 2024-09-19T09:20:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.345+0000 7fef55c0e640 1 -- 172.21.15.120:0/109167187 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef5019fca0 con 0x7fef50101c30 2024-09-19T09:20:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.345+0000 7fef4cff9640 1 -- 172.21.15.120:0/109167187 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef44003ce0 con 0x7fef50101c30 2024-09-19T09:20:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.345+0000 7fef4cff9640 1 -- 172.21.15.120:0/109167187 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef44035070 con 0x7fef50101c30 2024-09-19T09:20:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.345+0000 7fef55c0e640 1 -- 172.21.15.120:0/109167187 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef501a0190 con 0x7fef50101c30 2024-09-19T09:20:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.348+0000 7fef4cff9640 1 -- 172.21.15.120:0/109167187 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fef44042030 con 0x7fef50101c30 2024-09-19T09:20:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.348+0000 7fef4cff9640 1 --2- 172.21.15.120:0/109167187 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fef380a3370 0x7fef380a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.349+0000 7fef4cff9640 1 -- 172.21.15.120:0/109167187 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fef440efce0 con 0x7fef50101c30 2024-09-19T09:20:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.349+0000 7fef4f7fe640 1 --2- 172.21.15.120:0/109167187 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fef380a3370 0x7fef380a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.349+0000 7fef58a83740 1 Processor -- start 2024-09-19T09:20:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.349+0000 7fef58a83740 1 -- start start 2024-09-19T09:20:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.349+0000 7fef58a83740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4b40cdb0 0x55bb4b40d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.349+0000 7fef58a83740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb4b40d6f0 0x55bb4b4121b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.349+0000 7fef58a83740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b412780 0x55bb4b414c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.349+0000 7fef58a83740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bb4b36c2f0 con 0x55bb4b40d6f0 2024-09-19T09:20:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.349+0000 7fef58a83740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bb4b314270 con 0x55bb4b40cdb0 2024-09-19T09:20:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.349+0000 7fef58a83740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bb4b3682b0 con 0x55bb4b412780 2024-09-19T09:20:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.349+0000 7fef4effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb4b40d6f0 0x55bb4b4121b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.349+0000 7fef4effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb4b40d6f0 0x55bb4b4121b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40110/0 (socket says 172.21.15.120:40110) 2024-09-19T09:20:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.349+0000 7fef4effd640 1 -- 172.21.15.120:0/3136998583 learned_addr learned my addr 172.21.15.120:0/3136998583 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.349+0000 7fef4f7fe640 1 --2- 172.21.15.120:0/109167187 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fef380a3370 0x7fef380a5830 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fef50101a90 tx=0x7fef40005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.349+0000 7fef4ffff640 1 --2- 172.21.15.120:0/3136998583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b412780 0x55bb4b414c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.349+0000 7fef4f7fe640 1 --2- 172.21.15.120:0/3136998583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4b40cdb0 0x55bb4b40d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.350+0000 7fef4effd640 1 -- 172.21.15.120:0/3136998583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b412780 msgr2=0x55bb4b414c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.350+0000 7fef4effd640 1 --2- 172.21.15.120:0/3136998583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b412780 0x55bb4b414c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.350+0000 7fef4effd640 1 -- 172.21.15.120:0/3136998583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4b40cdb0 msgr2=0x55bb4b40d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.350+0000 7fef4effd640 1 --2- 172.21.15.120:0/3136998583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4b40cdb0 0x55bb4b40d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.350+0000 7fef4effd640 1 -- 172.21.15.120:0/3136998583 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb4b313340 con 0x55bb4b40d6f0 2024-09-19T09:20:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.350+0000 7fef4f7fe640 1 --2- 172.21.15.120:0/3136998583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4b40cdb0 0x55bb4b40d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:20:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.350+0000 7fef4effd640 1 --2- 172.21.15.120:0/3136998583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb4b40d6f0 0x55bb4b4121b0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fef440bc340 tx=0x7fef440bc250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d74847783b3c24 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.350+0000 7fef35ffb640 1 -- 172.21.15.120:0/3136998583 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef44049040 con 0x55bb4b40d6f0 2024-09-19T09:20:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.350+0000 7fef35ffb640 1 -- 172.21.15.120:0/3136998583 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef440bc3b0 con 0x55bb4b40d6f0 2024-09-19T09:20:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.350+0000 7fef35ffb640 1 -- 172.21.15.120:0/3136998583 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4403bc10 con 0x55bb4b40d6f0 2024-09-19T09:20:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.351+0000 7fef58a83740 1 -- 172.21.15.120:0/3136998583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb4b40d6f0 msgr2=0x55bb4b4121b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.351+0000 7fef58a83740 1 --2- 172.21.15.120:0/3136998583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb4b40d6f0 0x55bb4b4121b0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fef440bc340 tx=0x7fef440bc250 comp rx=0 tx=0).stop 2024-09-19T09:20:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.351+0000 7fef58a83740 1 -- 172.21.15.120:0/3136998583 shutdown_connections 2024-09-19T09:20:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.351+0000 7fef58a83740 1 --2- 172.21.15.120:0/3136998583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b412780 0x55bb4b414c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.351+0000 7fef58a83740 1 --2- 172.21.15.120:0/3136998583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb4b40d6f0 0x55bb4b4121b0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.351+0000 7fef58a83740 1 --2- 172.21.15.120:0/3136998583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4b40cdb0 0x55bb4b40d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.351+0000 7fef58a83740 1 -- 172.21.15.120:0/3136998583 >> 172.21.15.120:0/3136998583 conn(0x55bb4b408420 msgr2=0x55bb4b408820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.351+0000 7fef58a83740 1 -- 172.21.15.120:0/3136998583 shutdown_connections 2024-09-19T09:20:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.351+0000 7fef58a83740 1 -- 172.21.15.120:0/3136998583 wait complete. 2024-09-19T09:20:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.351+0000 7fef58a83740 1 -- start start 2024-09-19T09:20:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.351+0000 7fef58a83740 1 Processor -- start 2024-09-19T09:20:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.352+0000 7fef58a83740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4b40cdb0 0x55bb4b416de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.352+0000 7fef58a83740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b40d6f0 0x55bb4b417320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.352+0000 7fef58a83740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb4b412780 0x55bb4b41e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.352+0000 7fef58a83740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bb4b415510 con 0x55bb4b412780 2024-09-19T09:20:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.352+0000 7fef58a83740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bb4b2f5290 con 0x55bb4b40cdb0 2024-09-19T09:20:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.352+0000 7fef58a83740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bb4b415690 con 0x55bb4b40d6f0 2024-09-19T09:20:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.352+0000 7fef58a83740 10 client.0 _ll_register_callbacks cb 0x55bb4b326140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:20:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.352+0000 7fef4effd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b40d6f0 0x55bb4b417320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.352+0000 7fef4f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4b40cdb0 0x55bb4b416de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.352+0000 7fef4ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb4b412780 0x55bb4b41e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.352+0000 7fef4effd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b40d6f0 0x55bb4b417320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:57004/0 (socket says 172.21.15.120:57004) 2024-09-19T09:20:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.352+0000 7fef4effd640 1 -- 172.21.15.120:0/1597385157 learned_addr learned my addr 172.21.15.120:0/1597385157 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.352+0000 7fef4effd640 1 -- 172.21.15.120:0/1597385157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4b40cdb0 msgr2=0x55bb4b416de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.352+0000 7fef4effd640 1 --2- 172.21.15.120:0/1597385157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4b40cdb0 0x55bb4b416de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.352+0000 7fef4effd640 1 -- 172.21.15.120:0/1597385157 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb4b412780 msgr2=0x55bb4b41e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.352+0000 7fef4effd640 1 --2- 172.21.15.120:0/1597385157 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb4b412780 0x55bb4b41e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.352+0000 7fef4effd640 1 -- 172.21.15.120:0/1597385157 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb4b07f210 con 0x55bb4b40d6f0 2024-09-19T09:20:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.352+0000 7fef4f7fe640 1 --2- 172.21.15.120:0/1597385157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4b40cdb0 0x55bb4b416de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:20:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.352+0000 7fef4effd640 1 --2- 172.21.15.120:0/1597385157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b40d6f0 0x55bb4b417320 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fef44001eb0 tx=0x7fef440bbe10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc8b9e04923b5d1a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.352+0000 7fef2effd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:20:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.352+0000 7fef2effd640 1 -- 172.21.15.120:0/1597385157 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef440d4070 con 0x55bb4b40d6f0 2024-09-19T09:20:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.352+0000 7fef58a83740 1 -- 172.21.15.120:0/1597385157 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef440bc3b0 con 0x55bb4b40d6f0 2024-09-19T09:20:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.353+0000 7fef2effd640 1 -- 172.21.15.120:0/1597385157 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fef440d5260 con 0x55bb4b40d6f0 2024-09-19T09:20:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.353+0000 7fef2effd640 1 -- 172.21.15.120:0/1597385157 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fef44049050 con 0x55bb4b40d6f0 2024-09-19T09:20:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.353+0000 7fef58a83740 10 client.30007 fetch_fsmap learned FSMap version 39 2024-09-19T09:20:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.353+0000 7fef58a83740 1 -- 172.21.15.120:0/1597385157 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55bb4b14b620 con 0x55bb4b40d6f0 2024-09-19T09:20:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.353+0000 7fef2effd640 1 -- 172.21.15.120:0/1597385157 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef440bc3b0 con 0x55bb4b40d6f0 2024-09-19T09:20:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.353+0000 7fef2effd640 1 -- 172.21.15.120:0/1597385157 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fef440c4ae0 con 0x55bb4b40d6f0 2024-09-19T09:20:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.353+0000 7fef58a83740 10 client.30007 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:20:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.353+0000 7fef58a83740 10 client.30007 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:20:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.353+0000 7fef58a83740 20 client.30007 populate_metadata read hostname 'smithi120' 2024-09-19T09:20:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.353+0000 7fef58a83740 1 --2- 172.21.15.120:0/1597385157 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bb4b422720 0x55bb4b424b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.353+0000 7fef58a83740 4 client.30007 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:20:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.353+0000 7fef58a83740 1 -- 172.21.15.120:0/1597385157 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb4b379a20 con 0x55bb4b422720 2024-09-19T09:20:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.353+0000 7fef4f7fe640 1 --2- 172.21.15.120:0/1597385157 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bb4b422720 0x55bb4b424b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.354+0000 7fef4f7fe640 1 --2- 172.21.15.120:0/1597385157 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bb4b422720 0x55bb4b424b10 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=894308547aa5fe44 server_cookie=521ff3e7970d5241 in_seq=0 out_seq=0 2024-09-19T09:20:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.354+0000 7fef2effd640 10 client.30007 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.357+0000 7fef2effd640 1 -- 172.21.15.120:0/1597385157 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bb4b379a20 con 0x55bb4b422720 2024-09-19T09:20:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.357+0000 7fef2effd640 10 client.30007 handle_command_reply: tid=0 2024-09-19T09:20:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.357+0000 7fef58a83740 1 client.30007 shutdown 2024-09-19T09:20:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.357+0000 7fef58a83740 1 -- 172.21.15.120:0/1597385157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b40d6f0 msgr2=0x55bb4b417320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.357+0000 7fef58a83740 1 --2- 172.21.15.120:0/1597385157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b40d6f0 0x55bb4b417320 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fef44001eb0 tx=0x7fef440bbe10 comp rx=0 tx=0).stop 2024-09-19T09:20:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.357+0000 7fef58a83740 1 -- 172.21.15.120:0/1597385157 shutdown_connections 2024-09-19T09:20:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.357+0000 7fef58a83740 1 --2- 172.21.15.120:0/1597385157 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bb4b422720 0x55bb4b424b10 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.357+0000 7fef58a83740 1 --2- 172.21.15.120:0/1597385157 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb4b412780 0x55bb4b41e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.357+0000 7fef58a83740 1 --2- 172.21.15.120:0/1597385157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b40d6f0 0x55bb4b417320 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.357+0000 7fef2effd640 0 client.30007 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.357+0000 7fef58a83740 1 --2- 172.21.15.120:0/1597385157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4b40cdb0 0x55bb4b416de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.357+0000 7fef58a83740 1 -- 172.21.15.120:0/1597385157 >> 172.21.15.120:0/1597385157 conn(0x55bb4b408420 msgr2=0x55bb4b4031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.357+0000 7fef58a83740 1 -- 172.21.15.120:0/1597385157 shutdown_connections 2024-09-19T09:20:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.357+0000 7fef58a83740 1 -- 172.21.15.120:0/1597385157 wait complete. 2024-09-19T09:20:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.357+0000 7fef58a83740 20 client.30007 trim_cache size 0 max 16384 2024-09-19T09:20:34.368 INFO:tasks.workunit.client.0.smithi120.stdout: 10 113521 20.81 MB/sec execute 149 sec latency 169.962 ms 2024-09-19T09:20:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.374+0000 7fef58a83740 1 Processor -- start 2024-09-19T09:20:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.374+0000 7fef58a83740 1 -- start start 2024-09-19T09:20:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.374+0000 7fef58a83740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb4b40cdb0 0x55bb4b4030b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.374+0000 7fef58a83740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b40d6f0 0x55bb4b41e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.374+0000 7fef58a83740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4b412780 0x55bb4b4293a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.374+0000 7fef58a83740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fef501a6e40 con 0x55bb4b40cdb0 2024-09-19T09:20:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.374+0000 7fef58a83740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fef501a8980 con 0x55bb4b412780 2024-09-19T09:20:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.374+0000 7fef58a83740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fef501a7510 con 0x55bb4b40d6f0 2024-09-19T09:20:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.375+0000 7fef4f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb4b40cdb0 0x55bb4b4030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.375+0000 7fef4f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb4b40cdb0 0x55bb4b4030b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40118/0 (socket says 172.21.15.120:40118) 2024-09-19T09:20:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.375+0000 7fef4f7fe640 1 -- 172.21.15.120:0/2389203840 learned_addr learned my addr 172.21.15.120:0/2389203840 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.375+0000 7fef4effd640 1 --2- 172.21.15.120:0/2389203840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b40d6f0 0x55bb4b41e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.375+0000 7fef4ffff640 1 --2- 172.21.15.120:0/2389203840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4b412780 0x55bb4b4293a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.375+0000 7fef4f7fe640 1 -- 172.21.15.120:0/2389203840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b40d6f0 msgr2=0x55bb4b41e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.375+0000 7fef4f7fe640 1 --2- 172.21.15.120:0/2389203840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b40d6f0 0x55bb4b41e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.375+0000 7fef4f7fe640 1 -- 172.21.15.120:0/2389203840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4b412780 msgr2=0x55bb4b4293a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.375+0000 7fef4f7fe640 1 --2- 172.21.15.120:0/2389203840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4b412780 0x55bb4b4293a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.375+0000 7fef4f7fe640 1 -- 172.21.15.120:0/2389203840 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb4b408240 con 0x55bb4b40cdb0 2024-09-19T09:20:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.375+0000 7fef4f7fe640 1 --2- 172.21.15.120:0/2389203840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb4b40cdb0 0x55bb4b4030b0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fef40013010 tx=0x7fef40017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f716b74a0dd652 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.375+0000 7fef357fa640 1 -- 172.21.15.120:0/2389203840 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40004020 con 0x55bb4b40cdb0 2024-09-19T09:20:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.375+0000 7fef357fa640 1 -- 172.21.15.120:0/2389203840 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef4001ce70 con 0x55bb4b40cdb0 2024-09-19T09:20:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.375+0000 7fef357fa640 1 -- 172.21.15.120:0/2389203840 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40005590 con 0x55bb4b40cdb0 2024-09-19T09:20:34.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.376+0000 7fef58a83740 1 -- 172.21.15.120:0/2389203840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb4b40cdb0 msgr2=0x55bb4b4030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.376+0000 7fef58a83740 1 --2- 172.21.15.120:0/2389203840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb4b40cdb0 0x55bb4b4030b0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fef40013010 tx=0x7fef40017000 comp rx=0 tx=0).stop 2024-09-19T09:20:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.376+0000 7fef58a83740 1 -- 172.21.15.120:0/2389203840 shutdown_connections 2024-09-19T09:20:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.376+0000 7fef58a83740 1 --2- 172.21.15.120:0/2389203840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4b412780 0x55bb4b4293a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.376+0000 7fef58a83740 1 --2- 172.21.15.120:0/2389203840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b40d6f0 0x55bb4b41e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.376+0000 7fef58a83740 1 --2- 172.21.15.120:0/2389203840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb4b40cdb0 0x55bb4b4030b0 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.376+0000 7fef58a83740 1 -- 172.21.15.120:0/2389203840 >> 172.21.15.120:0/2389203840 conn(0x55bb4b408420 msgr2=0x55bb4b4029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.376+0000 7fef58a83740 1 -- 172.21.15.120:0/2389203840 shutdown_connections 2024-09-19T09:20:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.376+0000 7fef58a83740 1 -- 172.21.15.120:0/2389203840 wait complete. 2024-09-19T09:20:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.377+0000 7fef58a83740 1 -- start start 2024-09-19T09:20:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.377+0000 7fef58a83740 1 Processor -- start 2024-09-19T09:20:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.377+0000 7fef58a83740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb4b40d6f0 0x55bb4b41b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.377+0000 7fef58a83740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4b412780 0x55bb4b41bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.377+0000 7fef58a83740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b422720 0x55bb4b41ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.377+0000 7fef58a83740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fef501a82b0 con 0x55bb4b40d6f0 2024-09-19T09:20:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.377+0000 7fef58a83740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fef501a7be0 con 0x55bb4b412780 2024-09-19T09:20:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.377+0000 7fef58a83740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fef501a4e00 con 0x55bb4b422720 2024-09-19T09:20:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.377+0000 7fef4f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb4b40d6f0 0x55bb4b41b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.377+0000 7fef58a83740 10 client.0 _ll_register_callbacks cb 0x7fef440c0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:20:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.377+0000 7fef4ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b422720 0x55bb4b41ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.377+0000 7fef4ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b422720 0x55bb4b41ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:57018/0 (socket says 172.21.15.120:57018) 2024-09-19T09:20:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.377+0000 7fef4f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb4b40d6f0 0x55bb4b41b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40132/0 (socket says 172.21.15.120:40132) 2024-09-19T09:20:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.377+0000 7fef4ffff640 1 -- 172.21.15.120:0/4249292009 learned_addr learned my addr 172.21.15.120:0/4249292009 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.378+0000 7fef4effd640 1 --2- 172.21.15.120:0/4249292009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4b412780 0x55bb4b41bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.378+0000 7fef4ffff640 1 -- 172.21.15.120:0/4249292009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4b412780 msgr2=0x55bb4b41bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.378+0000 7fef4ffff640 1 --2- 172.21.15.120:0/4249292009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4b412780 0x55bb4b41bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.378+0000 7fef4ffff640 1 -- 172.21.15.120:0/4249292009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb4b40d6f0 msgr2=0x55bb4b41b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.378+0000 7fef4ffff640 1 --2- 172.21.15.120:0/4249292009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb4b40d6f0 0x55bb4b41b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.378+0000 7fef4ffff640 1 -- 172.21.15.120:0/4249292009 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb4b41d2c0 con 0x55bb4b422720 2024-09-19T09:20:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.378+0000 7fef4effd640 1 --2- 172.21.15.120:0/4249292009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4b412780 0x55bb4b41bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:20:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.378+0000 7fef4f7fe640 1 --2- 172.21.15.120:0/4249292009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb4b40d6f0 0x55bb4b41b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:20:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.378+0000 7fef4ffff640 1 --2- 172.21.15.120:0/4249292009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b422720 0x55bb4b41ccb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fef48003fd0 tx=0x7fef48004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9540319afdf30c71 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.378+0000 7fef2effd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:20:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.378+0000 7fef2effd640 1 -- 172.21.15.120:0/4249292009 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48010e60 con 0x55bb4b422720 2024-09-19T09:20:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.378+0000 7fef2effd640 1 -- 172.21.15.120:0/4249292009 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fef4801c470 con 0x55bb4b422720 2024-09-19T09:20:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.378+0000 7fef58a83740 1 -- 172.21.15.120:0/4249292009 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef4001ce70 con 0x55bb4b422720 2024-09-19T09:20:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.378+0000 7fef2effd640 1 -- 172.21.15.120:0/4249292009 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fef4801c670 con 0x55bb4b422720 2024-09-19T09:20:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.378+0000 7fef58a83740 10 client.30013 fetch_fsmap learned FSMap version 39 2024-09-19T09:20:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.378+0000 7fef58a83740 1 -- 172.21.15.120:0/4249292009 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55bb4b48d380 con 0x55bb4b422720 2024-09-19T09:20:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.379+0000 7fef2effd640 1 -- 172.21.15.120:0/4249292009 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef4001ce70 con 0x55bb4b422720 2024-09-19T09:20:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.379+0000 7fef2effd640 1 -- 172.21.15.120:0/4249292009 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fef48025900 con 0x55bb4b422720 2024-09-19T09:20:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.379+0000 7fef58a83740 10 client.30013 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:20:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.379+0000 7fef58a83740 10 client.30013 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:20:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.379+0000 7fef58a83740 20 client.30013 populate_metadata read hostname 'smithi120' 2024-09-19T09:20:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.379+0000 7fef58a83740 1 --2- 172.21.15.120:0/4249292009 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bb4b48dc90 0x55bb4b4900b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.379+0000 7fef58a83740 4 client.30013 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:20:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.379+0000 7fef58a83740 1 -- 172.21.15.120:0/4249292009 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bb4b373d70 con 0x55bb4b48dc90 2024-09-19T09:20:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.380+0000 7fef4f7fe640 1 --2- 172.21.15.120:0/4249292009 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bb4b48dc90 0x55bb4b4900b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.382+0000 7fef2effd640 10 client.30013 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.382+0000 7fef4f7fe640 1 --2- 172.21.15.120:0/4249292009 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bb4b48dc90 0x55bb4b4900b0 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=778f5f656d4446b9 server_cookie=7eeef26e772269e3 in_seq=0 out_seq=0 2024-09-19T09:20:34.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:34 smithi196 ceph-mon[35150]: pgmap v876: 97 pgs: 97 active+clean; 378 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.4 MiB/s wr, 713 op/s 2024-09-19T09:20:34.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:34 smithi153 ceph-mon[35071]: pgmap v876: 97 pgs: 97 active+clean; 378 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.4 MiB/s wr, 713 op/s 2024-09-19T09:20:34.877 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:20:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.974+0000 7fef2effd640 1 -- 172.21.15.120:0/4249292009 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bb4b373d70 con 0x55bb4b48dc90 2024-09-19T09:20:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.974+0000 7fef2effd640 10 client.30013 handle_command_reply: tid=0 2024-09-19T09:20:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.974+0000 7fef58a83740 1 client.30013 shutdown 2024-09-19T09:20:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.974+0000 7fef58a83740 1 -- 172.21.15.120:0/4249292009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b422720 msgr2=0x55bb4b41ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.974+0000 7fef58a83740 1 --2- 172.21.15.120:0/4249292009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b422720 0x55bb4b41ccb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fef48003fd0 tx=0x7fef48004750 comp rx=0 tx=0).stop 2024-09-19T09:20:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.974+0000 7fef58a83740 1 -- 172.21.15.120:0/4249292009 shutdown_connections 2024-09-19T09:20:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.974+0000 7fef58a83740 1 --2- 172.21.15.120:0/4249292009 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bb4b48dc90 0x55bb4b4900b0 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.974+0000 7fef58a83740 1 --2- 172.21.15.120:0/4249292009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4b422720 0x55bb4b41ccb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.974+0000 7fef58a83740 1 --2- 172.21.15.120:0/4249292009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bb4b412780 0x55bb4b41bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.974+0000 7fef58a83740 1 --2- 172.21.15.120:0/4249292009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bb4b40d6f0 0x55bb4b41b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.974+0000 7fef58a83740 1 -- 172.21.15.120:0/4249292009 >> 172.21.15.120:0/4249292009 conn(0x55bb4b408420 msgr2=0x55bb4b4029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.974+0000 7fef2effd640 0 client.30013 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.975+0000 7fef58a83740 1 -- 172.21.15.120:0/4249292009 shutdown_connections 2024-09-19T09:20:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.975+0000 7fef58a83740 1 -- 172.21.15.120:0/4249292009 wait complete. 2024-09-19T09:20:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.975+0000 7fef58a83740 20 client.30013 trim_cache size 0 max 16384 2024-09-19T09:20:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:20:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:20:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:20:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:20:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.976+0000 7fef55c0e640 1 -- 172.21.15.120:0/109167187 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fef380a3370 msgr2=0x7fef380a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.976+0000 7fef55c0e640 1 --2- 172.21.15.120:0/109167187 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fef380a3370 0x7fef380a5830 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fef50101a90 tx=0x7fef40005f90 comp rx=0 tx=0).stop 2024-09-19T09:20:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.976+0000 7fef55c0e640 1 -- 172.21.15.120:0/109167187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef50101c30 msgr2=0x7fef50198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.976+0000 7fef55c0e640 1 --2- 172.21.15.120:0/109167187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef50101c30 0x7fef50198330 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fef44009230 tx=0x7fef440042f0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.976+0000 7fef55c0e640 1 -- 172.21.15.120:0/109167187 shutdown_connections 2024-09-19T09:20:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.976+0000 7fef55c0e640 1 --2- 172.21.15.120:0/109167187 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fef380a3370 0x7fef380a5830 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.976+0000 7fef55c0e640 1 --2- 172.21.15.120:0/109167187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef501025f0 0x7fef5019f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.976+0000 7fef55c0e640 1 --2- 172.21.15.120:0/109167187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef50101c30 0x7fef50198330 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.976+0000 7fef55c0e640 1 --2- 172.21.15.120:0/109167187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef50100a40 0x7fef50197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.976+0000 7fef55c0e640 1 -- 172.21.15.120:0/109167187 >> 172.21.15.120:0/109167187 conn(0x7fef500fc1e0 msgr2=0x7fef500698a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.977+0000 7fef55c0e640 1 -- 172.21.15.120:0/109167187 shutdown_connections 2024-09-19T09:20:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:34.977+0000 7fef55c0e640 1 -- 172.21.15.120:0/109167187 wait complete. 2024-09-19T09:20:35.212 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:20:35.367 INFO:tasks.workunit.client.0.smithi120.stdout: 10 113912 20.82 MB/sec execute 150 sec latency 293.591 ms 2024-09-19T09:20:35.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.846+0000 7fd24392c640 1 -- 172.21.15.120:0/3466744963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd23c105580 msgr2=0x7fd23c10d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:35.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.846+0000 7fd24392c640 1 --2- 172.21.15.120:0/3466744963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd23c105580 0x7fd23c10d0e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd234009ca0 tx=0x7fd23402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.846+0000 7fd24392c640 1 -- 172.21.15.120:0/3466744963 shutdown_connections 2024-09-19T09:20:35.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.846+0000 7fd24392c640 1 --2- 172.21.15.120:0/3466744963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd23c105580 0x7fd23c10d0e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.846+0000 7fd24392c640 1 --2- 172.21.15.120:0/3466744963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd23c101080 0x7fd23c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.846+0000 7fd24392c640 1 --2- 172.21.15.120:0/3466744963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd23c1006d0 0x7fd23c100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.846+0000 7fd24392c640 1 -- 172.21.15.120:0/3466744963 >> 172.21.15.120:0/3466744963 conn(0x7fd23c0fba90 msgr2=0x7fd23c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:35.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.846+0000 7fd24392c640 1 -- 172.21.15.120:0/3466744963 shutdown_connections 2024-09-19T09:20:35.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.846+0000 7fd24392c640 1 -- 172.21.15.120:0/3466744963 wait complete. 2024-09-19T09:20:35.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.847+0000 7fd24392c640 1 Processor -- start 2024-09-19T09:20:35.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.847+0000 7fd24392c640 1 -- start start 2024-09-19T09:20:35.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.847+0000 7fd24392c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd23c1006d0 0x7fd23c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:35.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.847+0000 7fd24392c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd23c101080 0x7fd23c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:35.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.847+0000 7fd24392c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd23c105580 0x7fd23c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:35.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.847+0000 7fd24392c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd23c10fd40 con 0x7fd23c101080 2024-09-19T09:20:35.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.847+0000 7fd24392c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd23c10fbc0 con 0x7fd23c105580 2024-09-19T09:20:35.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.847+0000 7fd24392c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd23c10fec0 con 0x7fd23c1006d0 2024-09-19T09:20:35.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.848+0000 7fd240ec7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd23c101080 0x7fd23c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:35.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.848+0000 7fd2416c8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd23c1006d0 0x7fd23c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:35.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.848+0000 7fd240ec7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd23c101080 0x7fd23c198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40158/0 (socket says 172.21.15.120:40158) 2024-09-19T09:20:35.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.848+0000 7fd240ec7640 1 -- 172.21.15.120:0/958607268 learned_addr learned my addr 172.21.15.120:0/958607268 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:35.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.848+0000 7fd241ec9640 1 --2- 172.21.15.120:0/958607268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd23c105580 0x7fd23c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:35.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.848+0000 7fd2416c8640 1 -- 172.21.15.120:0/958607268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd23c105580 msgr2=0x7fd23c19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:35.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.848+0000 7fd2416c8640 1 --2- 172.21.15.120:0/958607268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd23c105580 0x7fd23c19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.848+0000 7fd2416c8640 1 -- 172.21.15.120:0/958607268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd23c101080 msgr2=0x7fd23c198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:35.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.848+0000 7fd2416c8640 1 --2- 172.21.15.120:0/958607268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd23c101080 0x7fd23c198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.848+0000 7fd2416c8640 1 -- 172.21.15.120:0/958607268 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd23c19d3b0 con 0x7fd23c1006d0 2024-09-19T09:20:35.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.848+0000 7fd241ec9640 1 --2- 172.21.15.120:0/958607268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd23c105580 0x7fd23c19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:20:35.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.848+0000 7fd2416c8640 1 --2- 172.21.15.120:0/958607268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd23c1006d0 0x7fd23c198400 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd23800ea90 tx=0x7fd23800ef50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:35.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.849+0000 7fd2327fc640 1 -- 172.21.15.120:0/958607268 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23801b070 con 0x7fd23c1006d0 2024-09-19T09:20:35.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.849+0000 7fd2327fc640 1 -- 172.21.15.120:0/958607268 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd23800cc80 con 0x7fd23c1006d0 2024-09-19T09:20:35.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.849+0000 7fd24392c640 1 -- 172.21.15.120:0/958607268 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd23c19d6a0 con 0x7fd23c1006d0 2024-09-19T09:20:35.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.849+0000 7fd2327fc640 1 -- 172.21.15.120:0/958607268 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23800b760 con 0x7fd23c1006d0 2024-09-19T09:20:35.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.849+0000 7fd24392c640 1 -- 172.21.15.120:0/958607268 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd23c1a4f80 con 0x7fd23c1006d0 2024-09-19T09:20:35.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.851+0000 7fd2327fc640 1 -- 172.21.15.120:0/958607268 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd23801e070 con 0x7fd23c1006d0 2024-09-19T09:20:35.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.851+0000 7fd245d05740 1 Processor -- start 2024-09-19T09:20:35.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.852+0000 7fd245d05740 1 -- start start 2024-09-19T09:20:35.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.852+0000 7fd245d05740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e509c5b40 0x557e509c4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:35.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.852+0000 7fd245d05740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e509c45d0 0x557e509c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:35.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.852+0000 7fd245d05740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e509d3780 0x557e509d5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:35.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.852+0000 7fd245d05740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557e5092a2f0 con 0x557e509c5b40 2024-09-19T09:20:35.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.852+0000 7fd245d05740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557e508d1cf0 con 0x557e509d3780 2024-09-19T09:20:35.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.852+0000 7fd245d05740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557e509262b0 con 0x557e509c45d0 2024-09-19T09:20:35.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.852+0000 7fd2327fc640 1 --2- 172.21.15.120:0/958607268 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd2280a3240 0x7fd2280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:35.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.852+0000 7fd241ec9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e509c5b40 0x557e509c4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:35.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.852+0000 7fd2416c8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e509c45d0 0x557e509c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:35.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.852+0000 7fd2416c8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e509c45d0 0x557e509c4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:57048/0 (socket says 172.21.15.120:57048) 2024-09-19T09:20:35.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.852+0000 7fd2416c8640 1 -- 172.21.15.120:0/3870970262 learned_addr learned my addr 172.21.15.120:0/3870970262 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:35.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.852+0000 7fd240ec7640 1 --2- 172.21.15.120:0/3870970262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e509d3780 0x557e509d5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:35.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.852+0000 7fd241ec9640 1 --2- 172.21.15.120:0/958607268 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd2280a3240 0x7fd2280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:35.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.852+0000 7fd2327fc640 1 -- 172.21.15.120:0/958607268 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd2380cf430 con 0x7fd23c1006d0 2024-09-19T09:20:35.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.852+0000 7fd2416c8640 1 -- 172.21.15.120:0/3870970262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e509d3780 msgr2=0x557e509d5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:35.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.852+0000 7fd2416c8640 1 --2- 172.21.15.120:0/3870970262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e509d3780 0x557e509d5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.852+0000 7fd2416c8640 1 -- 172.21.15.120:0/3870970262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e509c5b40 msgr2=0x557e509c4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:35.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.852+0000 7fd2416c8640 1 --2- 172.21.15.120:0/3870970262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e509c5b40 0x557e509c4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.852+0000 7fd2416c8640 1 -- 172.21.15.120:0/3870970262 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e508b63a0 con 0x557e509c45d0 2024-09-19T09:20:35.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.853+0000 7fd2416c8640 1 --2- 172.21.15.120:0/3870970262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e509c45d0 0x557e509c4a10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd2380a2560 tx=0x7fd23800af90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5de51435dd90e085 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:35.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.853+0000 7fd241ec9640 1 --2- 172.21.15.120:0/958607268 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd2280a3240 0x7fd2280a5700 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd234009430 tx=0x7fd23402a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:35.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.853+0000 7fd2277fe640 1 -- 172.21.15.120:0/3870970262 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2380a46c0 con 0x557e509c45d0 2024-09-19T09:20:35.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.853+0000 7fd2277fe640 1 -- 172.21.15.120:0/3870970262 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd238084e50 con 0x557e509c45d0 2024-09-19T09:20:35.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.853+0000 7fd2277fe640 1 -- 172.21.15.120:0/3870970262 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2380860f0 con 0x557e509c45d0 2024-09-19T09:20:35.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.854+0000 7fd245d05740 1 -- 172.21.15.120:0/3870970262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e509c45d0 msgr2=0x557e509c4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:35.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.854+0000 7fd245d05740 1 --2- 172.21.15.120:0/3870970262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e509c45d0 0x557e509c4a10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd2380a2560 tx=0x7fd23800af90 comp rx=0 tx=0).stop 2024-09-19T09:20:35.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.854+0000 7fd245d05740 1 -- 172.21.15.120:0/3870970262 shutdown_connections 2024-09-19T09:20:35.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.854+0000 7fd245d05740 1 --2- 172.21.15.120:0/3870970262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e509d3780 0x557e509d5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.854+0000 7fd245d05740 1 --2- 172.21.15.120:0/3870970262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e509c45d0 0x557e509c4a10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.854+0000 7fd245d05740 1 --2- 172.21.15.120:0/3870970262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e509c5b40 0x557e509c4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.854+0000 7fd245d05740 1 -- 172.21.15.120:0/3870970262 >> 172.21.15.120:0/3870970262 conn(0x557e5098d310 msgr2=0x557e509c0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:35.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.854+0000 7fd245d05740 1 -- 172.21.15.120:0/3870970262 shutdown_connections 2024-09-19T09:20:35.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.854+0000 7fd245d05740 1 -- 172.21.15.120:0/3870970262 wait complete. 2024-09-19T09:20:35.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.854+0000 7fd245d05740 1 -- start start 2024-09-19T09:20:35.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.854+0000 7fd245d05740 1 Processor -- start 2024-09-19T09:20:35.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.855+0000 7fd245d05740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e509c45d0 0x557e509d31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:35.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.855+0000 7fd245d05740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e509c5b40 0x557e509d0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:35.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.855+0000 7fd245d05740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e509d3780 0x557e509d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:35.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.855+0000 7fd245d05740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557e509d6480 con 0x557e509d3780 2024-09-19T09:20:35.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.855+0000 7fd245d05740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557e508e5060 con 0x557e509c5b40 2024-09-19T09:20:35.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.855+0000 7fd245d05740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557e509d6600 con 0x557e509c45d0 2024-09-19T09:20:35.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.855+0000 7fd245d05740 10 client.0 _ll_register_callbacks cb 0x557e506388c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:20:35.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.855+0000 7fd2416c8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e509c45d0 0x557e509d31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:35.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.855+0000 7fd240ec7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e509c5b40 0x557e509d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:35.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.855+0000 7fd2416c8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e509c45d0 0x557e509d31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:57054/0 (socket says 172.21.15.120:57054) 2024-09-19T09:20:35.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.855+0000 7fd2416c8640 1 -- 172.21.15.120:0/3729459548 learned_addr learned my addr 172.21.15.120:0/3729459548 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:35.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.855+0000 7fd241ec9640 1 --2- 172.21.15.120:0/3729459548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e509d3780 0x557e509d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:35.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.855+0000 7fd2416c8640 1 -- 172.21.15.120:0/3729459548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e509c5b40 msgr2=0x557e509d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:35.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.855+0000 7fd2416c8640 1 --2- 172.21.15.120:0/3729459548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e509c5b40 0x557e509d0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.855+0000 7fd2416c8640 1 -- 172.21.15.120:0/3729459548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e509d3780 msgr2=0x557e509d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:35.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.855+0000 7fd2416c8640 1 --2- 172.21.15.120:0/3729459548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e509d3780 0x557e509d0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.855+0000 7fd2416c8640 1 -- 172.21.15.120:0/3729459548 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e5069b3c0 con 0x557e509c45d0 2024-09-19T09:20:35.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.855+0000 7fd241ec9640 1 --2- 172.21.15.120:0/3729459548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e509d3780 0x557e509d0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:20:35.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.856+0000 7fd2416c8640 1 --2- 172.21.15.120:0/3729459548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e509c45d0 0x557e509d31a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd238000c00 tx=0x7fd2380ccea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cd5f44fd1ddee5b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:35.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.856+0000 7fd224ff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:20:35.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.856+0000 7fd224ff9640 1 -- 172.21.15.120:0/3729459548 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2380a46c0 con 0x557e509c45d0 2024-09-19T09:20:35.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.856+0000 7fd245d05740 1 -- 172.21.15.120:0/3729459548 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd238084e50 con 0x557e509c45d0 2024-09-19T09:20:35.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.856+0000 7fd224ff9640 1 -- 172.21.15.120:0/3729459548 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd238099e00 con 0x557e509c45d0 2024-09-19T09:20:35.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.856+0000 7fd224ff9640 1 -- 172.21.15.120:0/3729459548 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd2380ccfb0 con 0x557e509c45d0 2024-09-19T09:20:35.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.856+0000 7fd245d05740 10 client.30025 fetch_fsmap learned FSMap version 39 2024-09-19T09:20:35.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.856+0000 7fd245d05740 1 -- 172.21.15.120:0/3729459548 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557e509d6480 con 0x557e509c45d0 2024-09-19T09:20:35.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.856+0000 7fd224ff9640 1 -- 172.21.15.120:0/3729459548 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd238084e50 con 0x557e509c45d0 2024-09-19T09:20:35.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.856+0000 7fd224ff9640 1 -- 172.21.15.120:0/3729459548 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fd23800b040 con 0x557e509c45d0 2024-09-19T09:20:35.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.856+0000 7fd245d05740 10 client.30025 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:20:35.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.856+0000 7fd245d05740 10 client.30025 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:20:35.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.856+0000 7fd245d05740 20 client.30025 populate_metadata read hostname 'smithi120' 2024-09-19T09:20:35.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.856+0000 7fd245d05740 1 --2- 172.21.15.120:0/3729459548 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x557e509e3100 0x557e509e54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:35.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.856+0000 7fd245d05740 4 client.30025 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:20:35.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.856+0000 7fd245d05740 1 -- 172.21.15.120:0/3729459548 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e50937860 con 0x557e509e3100 2024-09-19T09:20:35.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.857+0000 7fd240ec7640 1 --2- 172.21.15.120:0/3729459548 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x557e509e3100 0x557e509e54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:35.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.857+0000 7fd224ff9640 10 client.30025 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:20:35.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.857+0000 7fd240ec7640 1 --2- 172.21.15.120:0/3729459548 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x557e509e3100 0x557e509e54f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ca50f505e8cc45c0 server_cookie=40d07715e933b5cb in_seq=0 out_seq=0 2024-09-19T09:20:35.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.858+0000 7fd224ff9640 1 -- 172.21.15.120:0/3729459548 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557e50937860 con 0x557e509e3100 2024-09-19T09:20:35.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.858+0000 7fd224ff9640 10 client.30025 handle_command_reply: tid=0 2024-09-19T09:20:35.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.858+0000 7fd245d05740 1 client.30025 shutdown 2024-09-19T09:20:35.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.858+0000 7fd245d05740 1 -- 172.21.15.120:0/3729459548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e509c45d0 msgr2=0x557e509d31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:35.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.858+0000 7fd245d05740 1 --2- 172.21.15.120:0/3729459548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e509c45d0 0x557e509d31a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd238000c00 tx=0x7fd2380ccea0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.858+0000 7fd245d05740 1 -- 172.21.15.120:0/3729459548 shutdown_connections 2024-09-19T09:20:35.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.858+0000 7fd245d05740 1 --2- 172.21.15.120:0/3729459548 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x557e509e3100 0x557e509e54f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.858+0000 7fd245d05740 1 --2- 172.21.15.120:0/3729459548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e509d3780 0x557e509d0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.858+0000 7fd245d05740 1 --2- 172.21.15.120:0/3729459548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e509c5b40 0x557e509d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.858+0000 7fd245d05740 1 --2- 172.21.15.120:0/3729459548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e509c45d0 0x557e509d31a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.858+0000 7fd245d05740 1 -- 172.21.15.120:0/3729459548 >> 172.21.15.120:0/3729459548 conn(0x557e5098d310 msgr2=0x557e509c6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:35.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.858+0000 7fd224ff9640 0 client.30025 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:20:35.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.858+0000 7fd245d05740 1 -- 172.21.15.120:0/3729459548 shutdown_connections 2024-09-19T09:20:35.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.858+0000 7fd245d05740 1 -- 172.21.15.120:0/3729459548 wait complete. 2024-09-19T09:20:35.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.858+0000 7fd245d05740 20 client.30025 trim_cache size 0 max 16384 2024-09-19T09:20:35.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.872+0000 7fd245d05740 1 Processor -- start 2024-09-19T09:20:35.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.872+0000 7fd245d05740 1 -- start start 2024-09-19T09:20:35.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.872+0000 7fd245d05740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e509c45d0 0x557e509c0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:35.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.872+0000 7fd245d05740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e509c5b40 0x557e509df750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:35.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.872+0000 7fd245d05740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e509d3780 0x557e509eabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:35.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.872+0000 7fd245d05740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd23c1ab3e0 con 0x557e509c45d0 2024-09-19T09:20:35.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.872+0000 7fd245d05740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd23c1acf20 con 0x557e509d3780 2024-09-19T09:20:35.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.872+0000 7fd245d05740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd23c1abab0 con 0x557e509c5b40 2024-09-19T09:20:35.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.873+0000 7fd2416c8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e509c45d0 0x557e509c0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:35.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.873+0000 7fd2416c8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e509c45d0 0x557e509c0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40172/0 (socket says 172.21.15.120:40172) 2024-09-19T09:20:35.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.873+0000 7fd2416c8640 1 -- 172.21.15.120:0/2866977459 learned_addr learned my addr 172.21.15.120:0/2866977459 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:35.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.873+0000 7fd240ec7640 1 --2- 172.21.15.120:0/2866977459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e509c5b40 0x557e509df750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:35.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.873+0000 7fd241ec9640 1 --2- 172.21.15.120:0/2866977459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e509d3780 0x557e509eabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:35.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.873+0000 7fd2416c8640 1 -- 172.21.15.120:0/2866977459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e509c5b40 msgr2=0x557e509df750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:35.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.873+0000 7fd2416c8640 1 --2- 172.21.15.120:0/2866977459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e509c5b40 0x557e509df750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.873+0000 7fd2416c8640 1 -- 172.21.15.120:0/2866977459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e509d3780 msgr2=0x557e509eabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:35.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.873+0000 7fd2416c8640 1 --2- 172.21.15.120:0/2866977459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e509d3780 0x557e509eabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.873+0000 7fd2416c8640 1 -- 172.21.15.120:0/2866977459 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e509c1960 con 0x557e509c45d0 2024-09-19T09:20:35.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.873+0000 7fd2416c8640 1 --2- 172.21.15.120:0/2866977459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e509c45d0 0x557e509c0e10 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fd2380ccf30 tx=0x7fd23800af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766c4a63a56133a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:35.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.873+0000 7fd226ffd640 1 -- 172.21.15.120:0/2866977459 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd238099e00 con 0x557e509c45d0 2024-09-19T09:20:35.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.873+0000 7fd226ffd640 1 -- 172.21.15.120:0/2866977459 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd238084e50 con 0x557e509c45d0 2024-09-19T09:20:35.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.873+0000 7fd226ffd640 1 -- 172.21.15.120:0/2866977459 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd238086270 con 0x557e509c45d0 2024-09-19T09:20:35.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.874+0000 7fd245d05740 1 -- 172.21.15.120:0/2866977459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e509c45d0 msgr2=0x557e509c0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:35.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.874+0000 7fd245d05740 1 --2- 172.21.15.120:0/2866977459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e509c45d0 0x557e509c0e10 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fd2380ccf30 tx=0x7fd23800af90 comp rx=0 tx=0).stop 2024-09-19T09:20:35.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.874+0000 7fd245d05740 1 -- 172.21.15.120:0/2866977459 shutdown_connections 2024-09-19T09:20:35.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.874+0000 7fd245d05740 1 --2- 172.21.15.120:0/2866977459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e509d3780 0x557e509eabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.874+0000 7fd245d05740 1 --2- 172.21.15.120:0/2866977459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e509c5b40 0x557e509df750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.874+0000 7fd245d05740 1 --2- 172.21.15.120:0/2866977459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e509c45d0 0x557e509c0e10 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.874+0000 7fd245d05740 1 -- 172.21.15.120:0/2866977459 >> 172.21.15.120:0/2866977459 conn(0x557e5098d310 msgr2=0x557e509c0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:35.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.874+0000 7fd245d05740 1 -- 172.21.15.120:0/2866977459 shutdown_connections 2024-09-19T09:20:35.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.874+0000 7fd245d05740 1 -- 172.21.15.120:0/2866977459 wait complete. 2024-09-19T09:20:35.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.875+0000 7fd245d05740 1 -- start start 2024-09-19T09:20:35.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.875+0000 7fd245d05740 1 Processor -- start 2024-09-19T09:20:35.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.875+0000 7fd245d05740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e509c45d0 0x557e509dbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:35.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.875+0000 7fd245d05740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e509c5b40 0x557e509cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:35.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.875+0000 7fd245d05740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e509d3780 0x557e509cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:35.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.875+0000 7fd245d05740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd23c1ac850 con 0x557e509c5b40 2024-09-19T09:20:35.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.875+0000 7fd245d05740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd23c1ac180 con 0x557e509d3780 2024-09-19T09:20:35.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.875+0000 7fd245d05740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd23c1a93a0 con 0x557e509c45d0 2024-09-19T09:20:35.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.875+0000 7fd245d05740 10 client.0 _ll_register_callbacks cb 0x7fd22c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:20:35.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.875+0000 7fd2416c8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e509c45d0 0x557e509dbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:35.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.875+0000 7fd2416c8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e509c45d0 0x557e509dbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:57070/0 (socket says 172.21.15.120:57070) 2024-09-19T09:20:35.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.876+0000 7fd2416c8640 1 -- 172.21.15.120:0/2838831920 learned_addr learned my addr 172.21.15.120:0/2838831920 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:35.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.876+0000 7fd240ec7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e509c5b40 0x557e509cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:35.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.876+0000 7fd241ec9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e509d3780 0x557e509cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:35.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.876+0000 7fd2416c8640 1 -- 172.21.15.120:0/2838831920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e509d3780 msgr2=0x557e509cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:35.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.876+0000 7fd2416c8640 1 --2- 172.21.15.120:0/2838831920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e509d3780 0x557e509cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.876+0000 7fd2416c8640 1 -- 172.21.15.120:0/2838831920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e509c5b40 msgr2=0x557e509cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:35.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.876+0000 7fd2416c8640 1 --2- 172.21.15.120:0/2838831920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e509c5b40 0x557e509cd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.876+0000 7fd2416c8640 1 -- 172.21.15.120:0/2838831920 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e509d13f0 con 0x557e509c45d0 2024-09-19T09:20:35.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.876+0000 7fd2416c8640 1 --2- 172.21.15.120:0/2838831920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e509c45d0 0x557e509dbee0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd23809ac90 tx=0x7fd2380ba530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f454b8b7fb2a7bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:35.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.876+0000 7fd224ff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:20:35.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.876+0000 7fd224ff9640 1 -- 172.21.15.120:0/2838831920 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23809ad70 con 0x557e509c45d0 2024-09-19T09:20:35.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.876+0000 7fd224ff9640 1 -- 172.21.15.120:0/2838831920 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd2380860e0 con 0x557e509c45d0 2024-09-19T09:20:35.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.876+0000 7fd245d05740 1 -- 172.21.15.120:0/2838831920 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd238084e50 con 0x557e509c45d0 2024-09-19T09:20:35.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.876+0000 7fd224ff9640 1 -- 172.21.15.120:0/2838831920 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd2380862e0 con 0x557e509c45d0 2024-09-19T09:20:35.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.876+0000 7fd245d05740 10 client.30031 fetch_fsmap learned FSMap version 39 2024-09-19T09:20:35.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.876+0000 7fd245d05740 1 -- 172.21.15.120:0/2838831920 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557e509d13f0 con 0x557e509c45d0 2024-09-19T09:20:35.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.877+0000 7fd224ff9640 1 -- 172.21.15.120:0/2838831920 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2380a46c0 con 0x557e509c45d0 2024-09-19T09:20:35.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.877+0000 7fd224ff9640 1 -- 172.21.15.120:0/2838831920 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fd2380a11c0 con 0x557e509c45d0 2024-09-19T09:20:35.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.877+0000 7fd245d05740 10 client.30031 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:20:35.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.877+0000 7fd245d05740 10 client.30031 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:20:35.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.877+0000 7fd245d05740 20 client.30031 populate_metadata read hostname 'smithi120' 2024-09-19T09:20:35.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.877+0000 7fd245d05740 1 --2- 172.21.15.120:0/2838831920 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x557e50a4dbd0 0x557e50a4fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:35.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.877+0000 7fd245d05740 4 client.30031 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:20:35.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.877+0000 7fd245d05740 1 -- 172.21.15.120:0/2838831920 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd238084e50 con 0x557e50a4dbd0 2024-09-19T09:20:35.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.877+0000 7fd240ec7640 1 --2- 172.21.15.120:0/2838831920 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x557e50a4dbd0 0x557e50a4fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:35.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.878+0000 7fd224ff9640 10 client.30031 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:20:35.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.878+0000 7fd240ec7640 1 --2- 172.21.15.120:0/2838831920 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x557e50a4dbd0 0x557e50a4fff0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=250cebe9527515a7 server_cookie=637e97ca8a462e56 in_seq=0 out_seq=0 2024-09-19T09:20:35.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.878+0000 7fd224ff9640 1 -- 172.21.15.120:0/2838831920 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd238084e50 con 0x557e50a4dbd0 2024-09-19T09:20:35.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.878+0000 7fd224ff9640 10 client.30031 handle_command_reply: tid=0 2024-09-19T09:20:35.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.878+0000 7fd245d05740 1 client.30031 shutdown 2024-09-19T09:20:35.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.878+0000 7fd245d05740 1 -- 172.21.15.120:0/2838831920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e509c45d0 msgr2=0x557e509dbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:35.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.878+0000 7fd245d05740 1 --2- 172.21.15.120:0/2838831920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e509c45d0 0x557e509dbee0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd23809ac90 tx=0x7fd2380ba530 comp rx=0 tx=0).stop 2024-09-19T09:20:35.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.878+0000 7fd245d05740 1 -- 172.21.15.120:0/2838831920 shutdown_connections 2024-09-19T09:20:35.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.878+0000 7fd245d05740 1 --2- 172.21.15.120:0/2838831920 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x557e50a4dbd0 0x557e50a4fff0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.878+0000 7fd245d05740 1 --2- 172.21.15.120:0/2838831920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e509d3780 0x557e509cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.878+0000 7fd245d05740 1 --2- 172.21.15.120:0/2838831920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e509c5b40 0x557e509cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.878+0000 7fd245d05740 1 --2- 172.21.15.120:0/2838831920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e509c45d0 0x557e509dbee0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.878+0000 7fd245d05740 1 -- 172.21.15.120:0/2838831920 >> 172.21.15.120:0/2838831920 conn(0x557e5098d310 msgr2=0x557e509c6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:35.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.878+0000 7fd224ff9640 0 client.30031 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:20:35.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.878+0000 7fd245d05740 1 -- 172.21.15.120:0/2838831920 shutdown_connections 2024-09-19T09:20:35.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.878+0000 7fd245d05740 1 -- 172.21.15.120:0/2838831920 wait complete. 2024-09-19T09:20:35.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.878+0000 7fd245d05740 20 client.30031 trim_cache size 0 max 16384 2024-09-19T09:20:35.885 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:20:35.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.879+0000 7fd24392c640 1 -- 172.21.15.120:0/958607268 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd2280a3240 msgr2=0x7fd2280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:35.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.879+0000 7fd24392c640 1 --2- 172.21.15.120:0/958607268 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd2280a3240 0x7fd2280a5700 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd234009430 tx=0x7fd23402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.879+0000 7fd24392c640 1 -- 172.21.15.120:0/958607268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd23c1006d0 msgr2=0x7fd23c198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:35.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.879+0000 7fd24392c640 1 --2- 172.21.15.120:0/958607268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd23c1006d0 0x7fd23c198400 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd23800ea90 tx=0x7fd23800ef50 comp rx=0 tx=0).stop 2024-09-19T09:20:35.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.879+0000 7fd24392c640 1 -- 172.21.15.120:0/958607268 shutdown_connections 2024-09-19T09:20:35.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.879+0000 7fd24392c640 1 --2- 172.21.15.120:0/958607268 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd2280a3240 0x7fd2280a5700 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.879+0000 7fd24392c640 1 --2- 172.21.15.120:0/958607268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd23c105580 0x7fd23c19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.879+0000 7fd24392c640 1 --2- 172.21.15.120:0/958607268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd23c101080 0x7fd23c198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.879+0000 7fd24392c640 1 --2- 172.21.15.120:0/958607268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd23c1006d0 0x7fd23c198400 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:35.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.879+0000 7fd24392c640 1 -- 172.21.15.120:0/958607268 >> 172.21.15.120:0/958607268 conn(0x7fd23c0fba90 msgr2=0x7fd23c0fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:35.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.879+0000 7fd24392c640 1 -- 172.21.15.120:0/958607268 shutdown_connections 2024-09-19T09:20:35.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:35.879+0000 7fd24392c640 1 -- 172.21.15.120:0/958607268 wait complete. 2024-09-19T09:20:36.255 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:36 smithi120 ceph-mon[31150]: pgmap v877: 97 pgs: 97 active+clean; 373 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.4 MiB/s wr, 558 op/s 2024-09-19T09:20:36.367 INFO:tasks.workunit.client.0.smithi120.stdout: 10 114615 20.95 MB/sec execute 151 sec latency 78.873 ms 2024-09-19T09:20:36.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:36 smithi196 ceph-mon[35150]: pgmap v877: 97 pgs: 97 active+clean; 373 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.4 MiB/s wr, 558 op/s 2024-09-19T09:20:36.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:36 smithi153 ceph-mon[35071]: pgmap v877: 97 pgs: 97 active+clean; 373 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.4 MiB/s wr, 558 op/s 2024-09-19T09:20:36.610 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:20:36.895 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:20:37.367 INFO:tasks.workunit.client.0.smithi120.stdout: 10 115252 21.09 MB/sec execute 152 sec latency 80.662 ms 2024-09-19T09:20:37.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.857+0000 7f905486f640 1 -- 172.21.15.120:0/141629164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9050102570 msgr2=0x7f90501029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:37.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.857+0000 7f905486f640 1 --2- 172.21.15.120:0/141629164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9050102570 0x7f90501029b0 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f9044009900 tx=0x7f904402a480 comp rx=0 tx=0).stop 2024-09-19T09:20:37.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.857+0000 7f905486f640 1 -- 172.21.15.120:0/141629164 shutdown_connections 2024-09-19T09:20:37.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.857+0000 7f905486f640 1 --2- 172.21.15.120:0/141629164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9050102ef0 0x7f905010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.857+0000 7f905486f640 1 --2- 172.21.15.120:0/141629164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9050102570 0x7f90501029b0 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.857+0000 7f905486f640 1 --2- 172.21.15.120:0/141629164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9050100e20 0x7f9050101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.857+0000 7f905486f640 1 -- 172.21.15.120:0/141629164 >> 172.21.15.120:0/141629164 conn(0x7f90500fc7d0 msgr2=0x7f90500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:37.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.857+0000 7f905486f640 1 -- 172.21.15.120:0/141629164 shutdown_connections 2024-09-19T09:20:37.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.857+0000 7f905486f640 1 -- 172.21.15.120:0/141629164 wait complete. 2024-09-19T09:20:37.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.858+0000 7f905486f640 1 Processor -- start 2024-09-19T09:20:37.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.858+0000 7f905486f640 1 -- start start 2024-09-19T09:20:37.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.858+0000 7f905486f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9050100e20 0x7f90501984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:37.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.858+0000 7f905486f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9050102570 0x7f90501989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:37.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.858+0000 7f905486f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9050102ef0 0x7f905019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:37.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.858+0000 7f905486f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9050112090 con 0x7f9050102ef0 2024-09-19T09:20:37.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.858+0000 7f905486f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9050111f10 con 0x7f9050100e20 2024-09-19T09:20:37.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.858+0000 7f905486f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9050112210 con 0x7f9050102570 2024-09-19T09:20:37.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.858+0000 7f904e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9050100e20 0x7f90501984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:37.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.858+0000 7f904e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9050100e20 0x7f90501984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:60554/0 (socket says 172.21.15.120:60554) 2024-09-19T09:20:37.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.858+0000 7f904e59c640 1 -- 172.21.15.120:0/2831256595 learned_addr learned my addr 172.21.15.120:0/2831256595 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:37.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.858+0000 7f904ed9d640 1 --2- 172.21.15.120:0/2831256595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9050102ef0 0x7f905019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:37.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.858+0000 7f904dd9b640 1 --2- 172.21.15.120:0/2831256595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9050102570 0x7f90501989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:37.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.859+0000 7f904ed9d640 1 -- 172.21.15.120:0/2831256595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9050102570 msgr2=0x7f90501989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:37.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.859+0000 7f904ed9d640 1 --2- 172.21.15.120:0/2831256595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9050102570 0x7f90501989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.859+0000 7f904ed9d640 1 -- 172.21.15.120:0/2831256595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9050100e20 msgr2=0x7f90501984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:37.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.859+0000 7f904ed9d640 1 --2- 172.21.15.120:0/2831256595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9050100e20 0x7f90501984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.859+0000 7f904ed9d640 1 -- 172.21.15.120:0/2831256595 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f905019d460 con 0x7f9050102ef0 2024-09-19T09:20:37.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.859+0000 7f904e59c640 1 --2- 172.21.15.120:0/2831256595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9050100e20 0x7f90501984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:20:37.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.859+0000 7f904ed9d640 1 --2- 172.21.15.120:0/2831256595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9050102ef0 0x7f905019cd80 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f9048009fd0 tx=0x7f904800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:37.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.859+0000 7f903f7fe640 1 -- 172.21.15.120:0/2831256595 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904800ce20 con 0x7f9050102ef0 2024-09-19T09:20:37.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.859+0000 7f903f7fe640 1 -- 172.21.15.120:0/2831256595 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9048005020 con 0x7f9050102ef0 2024-09-19T09:20:37.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.859+0000 7f905486f640 1 -- 172.21.15.120:0/2831256595 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f905019d750 con 0x7f9050102ef0 2024-09-19T09:20:37.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.859+0000 7f903f7fe640 1 -- 172.21.15.120:0/2831256595 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9048010720 con 0x7f9050102ef0 2024-09-19T09:20:37.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.859+0000 7f905486f640 1 -- 172.21.15.120:0/2831256595 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f905010e280 con 0x7f9050102ef0 2024-09-19T09:20:37.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.861+0000 7f903f7fe640 1 -- 172.21.15.120:0/2831256595 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f9048005540 con 0x7f9050102ef0 2024-09-19T09:20:37.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.862+0000 7f903f7fe640 1 --2- 172.21.15.120:0/2831256595 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f90380a3240 0x7f90380a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:37.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.862+0000 7f9056c48740 1 Processor -- start 2024-09-19T09:20:37.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.862+0000 7f904e59c640 1 --2- 172.21.15.120:0/2831256595 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f90380a3240 0x7f90380a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:37.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.862+0000 7f903f7fe640 1 -- 172.21.15.120:0/2831256595 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f90480cff10 con 0x7f9050102ef0 2024-09-19T09:20:37.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.862+0000 7f9056c48740 1 -- start start 2024-09-19T09:20:37.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.863+0000 7f9056c48740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f705f4fb40 0x55f705f4e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:37.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.863+0000 7f9056c48740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f705f4e5d0 0x55f705f4ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:37.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.863+0000 7f9056c48740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f705f59370 0x55f705f5b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:37.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.863+0000 7f9056c48740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f705eb42f0 con 0x55f705f4fb40 2024-09-19T09:20:37.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.863+0000 7f9056c48740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f705e5bcf0 con 0x55f705f59370 2024-09-19T09:20:37.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.863+0000 7f9056c48740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f705eb02b0 con 0x55f705f4e5d0 2024-09-19T09:20:37.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.863+0000 7f904dd9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f705f4e5d0 0x55f705f4ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:37.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.863+0000 7f904e59c640 1 --2- 172.21.15.120:0/2831256595 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f90380a3240 0x7f90380a5700 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9040011550 tx=0x7f9040005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:37.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.863+0000 7f904e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f705f4fb40 0x55f705f4e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:37.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.863+0000 7f904dd9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f705f4e5d0 0x55f705f4ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:57104/0 (socket says 172.21.15.120:57104) 2024-09-19T09:20:37.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.863+0000 7f904dd9b640 1 -- 172.21.15.120:0/2552455090 learned_addr learned my addr 172.21.15.120:0/2552455090 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:37.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.863+0000 7f904ed9d640 1 --2- 172.21.15.120:0/2552455090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f705f59370 0x55f705f5b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:37.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.863+0000 7f904dd9b640 1 -- 172.21.15.120:0/2552455090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f705f59370 msgr2=0x55f705f5b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:37.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.863+0000 7f904dd9b640 1 --2- 172.21.15.120:0/2552455090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f705f59370 0x55f705f5b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.863+0000 7f904dd9b640 1 -- 172.21.15.120:0/2552455090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f705f4fb40 msgr2=0x55f705f4e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:37.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.863+0000 7f904dd9b640 1 --2- 172.21.15.120:0/2552455090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f705f4fb40 0x55f705f4e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.863+0000 7f904dd9b640 1 -- 172.21.15.120:0/2552455090 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f705e403a0 con 0x55f705f4e5d0 2024-09-19T09:20:37.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.863+0000 7f904e59c640 1 --2- 172.21.15.120:0/2552455090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f705f4fb40 0x55f705f4e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:20:37.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.863+0000 7f904dd9b640 1 --2- 172.21.15.120:0/2552455090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f705f4e5d0 0x55f705f4ea10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f904402a9a0 tx=0x7f9044005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3e5ef1389d1d20a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:37.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.864+0000 7f903cff9640 1 -- 172.21.15.120:0/2552455090 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044035050 con 0x55f705f4e5d0 2024-09-19T09:20:37.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.864+0000 7f903cff9640 1 -- 172.21.15.120:0/2552455090 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9044049070 con 0x55f705f4e5d0 2024-09-19T09:20:37.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.864+0000 7f903cff9640 1 -- 172.21.15.120:0/2552455090 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044003970 con 0x55f705f4e5d0 2024-09-19T09:20:37.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.864+0000 7f9056c48740 1 -- 172.21.15.120:0/2552455090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f705f4e5d0 msgr2=0x55f705f4ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:37.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.864+0000 7f9056c48740 1 --2- 172.21.15.120:0/2552455090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f705f4e5d0 0x55f705f4ea10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f904402a9a0 tx=0x7f9044005d50 comp rx=0 tx=0).stop 2024-09-19T09:20:37.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.865+0000 7f9056c48740 1 -- 172.21.15.120:0/2552455090 shutdown_connections 2024-09-19T09:20:37.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.865+0000 7f9056c48740 1 --2- 172.21.15.120:0/2552455090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f705f59370 0x55f705f5b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.865+0000 7f9056c48740 1 --2- 172.21.15.120:0/2552455090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f705f4e5d0 0x55f705f4ea10 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.865+0000 7f9056c48740 1 --2- 172.21.15.120:0/2552455090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f705f4fb40 0x55f705f4e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.865+0000 7f9056c48740 1 -- 172.21.15.120:0/2552455090 >> 172.21.15.120:0/2552455090 conn(0x55f705f17310 msgr2=0x55f705f4ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:37.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.865+0000 7f9056c48740 1 -- 172.21.15.120:0/2552455090 shutdown_connections 2024-09-19T09:20:37.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.865+0000 7f9056c48740 1 -- 172.21.15.120:0/2552455090 wait complete. 2024-09-19T09:20:37.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.865+0000 7f9056c48740 1 -- start start 2024-09-19T09:20:37.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.865+0000 7f9056c48740 1 Processor -- start 2024-09-19T09:20:37.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.866+0000 7f9056c48740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f705f4e5d0 0x55f705f58d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:37.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.866+0000 7f9056c48740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f705f4fb40 0x55f705f55fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:37.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.866+0000 7f9056c48740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f705f59370 0x55f705f56570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:37.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.866+0000 7f9056c48740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f705f5c010 con 0x55f705f59370 2024-09-19T09:20:37.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.866+0000 7f9056c48740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f705e6f060 con 0x55f705f4e5d0 2024-09-19T09:20:37.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.866+0000 7f9056c48740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f705f5c190 con 0x55f705f4fb40 2024-09-19T09:20:37.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.866+0000 7f9056c48740 10 client.0 _ll_register_callbacks cb 0x55f705bc28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:20:37.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.866+0000 7f904dd9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f705f4fb40 0x55f705f55fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:37.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.866+0000 7f904ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f705f59370 0x55f705f56570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:37.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.866+0000 7f904dd9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f705f4fb40 0x55f705f55fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:57108/0 (socket says 172.21.15.120:57108) 2024-09-19T09:20:37.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.866+0000 7f904dd9b640 1 -- 172.21.15.120:0/2339314129 learned_addr learned my addr 172.21.15.120:0/2339314129 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:37.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.866+0000 7f904e59c640 1 --2- 172.21.15.120:0/2339314129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f705f4e5d0 0x55f705f58d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:37.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.866+0000 7f904dd9b640 1 -- 172.21.15.120:0/2339314129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f705f4e5d0 msgr2=0x55f705f58d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:37.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.866+0000 7f904dd9b640 1 --2- 172.21.15.120:0/2339314129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f705f4e5d0 0x55f705f58d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.866+0000 7f904dd9b640 1 -- 172.21.15.120:0/2339314129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f705f59370 msgr2=0x55f705f56570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:37.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.866+0000 7f904dd9b640 1 --2- 172.21.15.120:0/2339314129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f705f59370 0x55f705f56570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.866+0000 7f904dd9b640 1 -- 172.21.15.120:0/2339314129 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f705c253c0 con 0x55f705f4fb40 2024-09-19T09:20:37.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.866+0000 7f904dd9b640 1 --2- 172.21.15.120:0/2339314129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f705f4fb40 0x55f705f55fd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9044009900 tx=0x7f904402a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=959e63974cc1b54f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:37.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.866+0000 7f9035ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:20:37.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.867+0000 7f9035ffb640 1 -- 172.21.15.120:0/2339314129 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044009520 con 0x55f705f4fb40 2024-09-19T09:20:37.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.867+0000 7f9056c48740 1 -- 172.21.15.120:0/2339314129 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9044049070 con 0x55f705f4fb40 2024-09-19T09:20:37.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.867+0000 7f9035ffb640 1 -- 172.21.15.120:0/2339314129 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f9044031540 con 0x55f705f4fb40 2024-09-19T09:20:37.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.867+0000 7f9035ffb640 1 -- 172.21.15.120:0/2339314129 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9044031740 con 0x55f705f4fb40 2024-09-19T09:20:37.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.867+0000 7f9056c48740 10 client.30043 fetch_fsmap learned FSMap version 39 2024-09-19T09:20:37.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.867+0000 7f9056c48740 1 -- 172.21.15.120:0/2339314129 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f705f5c010 con 0x55f705f4fb40 2024-09-19T09:20:37.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.867+0000 7f9035ffb640 1 -- 172.21.15.120:0/2339314129 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9044049070 con 0x55f705f4fb40 2024-09-19T09:20:37.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.867+0000 7f9035ffb640 1 -- 172.21.15.120:0/2339314129 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f9044055030 con 0x55f705f4fb40 2024-09-19T09:20:37.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.867+0000 7f9056c48740 10 client.30043 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:20:37.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.867+0000 7f9056c48740 10 client.30043 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:20:37.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.867+0000 7f9056c48740 20 client.30043 populate_metadata read hostname 'smithi120' 2024-09-19T09:20:37.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.867+0000 7f9056c48740 1 --2- 172.21.15.120:0/2339314129 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55f705f713f0 0x55f705f737e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:37.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.867+0000 7f9056c48740 4 client.30043 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:20:37.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.867+0000 7f9056c48740 1 -- 172.21.15.120:0/2339314129 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f705ec1860 con 0x55f705f713f0 2024-09-19T09:20:37.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.868+0000 7f904e59c640 1 --2- 172.21.15.120:0/2339314129 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55f705f713f0 0x55f705f737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:37.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.869+0000 7f904e59c640 1 --2- 172.21.15.120:0/2339314129 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55f705f713f0 0x55f705f737e0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5f41ee3245f5e60 server_cookie=f2e557ff1256b7ff in_seq=0 out_seq=0 2024-09-19T09:20:37.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.869+0000 7f9035ffb640 10 client.30043 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:20:37.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.870+0000 7f9035ffb640 1 -- 172.21.15.120:0/2339314129 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f705ec1860 con 0x55f705f713f0 2024-09-19T09:20:37.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.870+0000 7f9035ffb640 10 client.30043 handle_command_reply: tid=0 2024-09-19T09:20:37.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.870+0000 7f9056c48740 1 client.30043 shutdown 2024-09-19T09:20:37.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.870+0000 7f9056c48740 1 -- 172.21.15.120:0/2339314129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f705f4fb40 msgr2=0x55f705f55fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:37.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.870+0000 7f9056c48740 1 --2- 172.21.15.120:0/2339314129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f705f4fb40 0x55f705f55fd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9044009900 tx=0x7f904402a480 comp rx=0 tx=0).stop 2024-09-19T09:20:37.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.870+0000 7f9056c48740 1 -- 172.21.15.120:0/2339314129 shutdown_connections 2024-09-19T09:20:37.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.870+0000 7f9056c48740 1 --2- 172.21.15.120:0/2339314129 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55f705f713f0 0x55f705f737e0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.870+0000 7f9056c48740 1 --2- 172.21.15.120:0/2339314129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f705f59370 0x55f705f56570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.870+0000 7f9056c48740 1 --2- 172.21.15.120:0/2339314129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f705f4fb40 0x55f705f55fd0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.870+0000 7f9035ffb640 0 client.30043 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:20:37.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.870+0000 7f9056c48740 1 --2- 172.21.15.120:0/2339314129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f705f4e5d0 0x55f705f58d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.870+0000 7f9056c48740 1 -- 172.21.15.120:0/2339314129 >> 172.21.15.120:0/2339314129 conn(0x55f705f17310 msgr2=0x55f705f50960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:37.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.870+0000 7f9056c48740 1 -- 172.21.15.120:0/2339314129 shutdown_connections 2024-09-19T09:20:37.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.870+0000 7f9056c48740 1 -- 172.21.15.120:0/2339314129 wait complete. 2024-09-19T09:20:37.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.870+0000 7f9056c48740 20 client.30043 trim_cache size 0 max 16384 2024-09-19T09:20:37.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.884+0000 7f9056c48740 1 Processor -- start 2024-09-19T09:20:37.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.884+0000 7f9056c48740 1 -- start start 2024-09-19T09:20:37.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.884+0000 7f9056c48740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f705f4e5d0 0x55f705f652b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:37.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.884+0000 7f9056c48740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f705f4fb40 0x55f705f57f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:37.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.884+0000 7f9056c48740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f705f59370 0x55f705f5a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:37.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.884+0000 7f9056c48740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f90501ab290 con 0x55f705f4fb40 2024-09-19T09:20:37.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.884+0000 7f9056c48740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f90501acdd0 con 0x55f705f59370 2024-09-19T09:20:37.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.884+0000 7f9056c48740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f90501ab960 con 0x55f705f4e5d0 2024-09-19T09:20:37.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.885+0000 7f904dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f705f4fb40 0x55f705f57f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:37.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.885+0000 7f904dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f705f4fb40 0x55f705f57f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40220/0 (socket says 172.21.15.120:40220) 2024-09-19T09:20:37.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.885+0000 7f904dd9b640 1 -- 172.21.15.120:0/1245463237 learned_addr learned my addr 172.21.15.120:0/1245463237 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:37.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.885+0000 7f904ed9d640 1 --2- 172.21.15.120:0/1245463237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f705f59370 0x55f705f5a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:37.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.885+0000 7f904e59c640 1 --2- 172.21.15.120:0/1245463237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f705f4e5d0 0x55f705f652b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:37.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.885+0000 7f904dd9b640 1 -- 172.21.15.120:0/1245463237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f705f4e5d0 msgr2=0x55f705f652b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:37.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.885+0000 7f904dd9b640 1 --2- 172.21.15.120:0/1245463237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f705f4e5d0 0x55f705f652b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.885+0000 7f904dd9b640 1 -- 172.21.15.120:0/1245463237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f705f59370 msgr2=0x55f705f5a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:37.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.885+0000 7f904dd9b640 1 --2- 172.21.15.120:0/1245463237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f705f59370 0x55f705f5a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.885+0000 7f904dd9b640 1 -- 172.21.15.120:0/1245463237 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f705f51120 con 0x55f705f4fb40 2024-09-19T09:20:37.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.885+0000 7f904dd9b640 1 --2- 172.21.15.120:0/1245463237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f705f4fb40 0x55f705f57f40 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f90440314e0 tx=0x7f9044005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef478092c180110 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:37.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.885+0000 7f9037fff640 1 -- 172.21.15.120:0/1245463237 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044035040 con 0x55f705f4fb40 2024-09-19T09:20:37.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.885+0000 7f9037fff640 1 -- 172.21.15.120:0/1245463237 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9044049040 con 0x55f705f4fb40 2024-09-19T09:20:37.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.885+0000 7f9037fff640 1 -- 172.21.15.120:0/1245463237 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044031d70 con 0x55f705f4fb40 2024-09-19T09:20:37.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.886+0000 7f9056c48740 1 -- 172.21.15.120:0/1245463237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f705f4fb40 msgr2=0x55f705f57f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:37.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.886+0000 7f9056c48740 1 --2- 172.21.15.120:0/1245463237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f705f4fb40 0x55f705f57f40 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f90440314e0 tx=0x7f9044005850 comp rx=0 tx=0).stop 2024-09-19T09:20:37.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.886+0000 7f9056c48740 1 -- 172.21.15.120:0/1245463237 shutdown_connections 2024-09-19T09:20:37.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.886+0000 7f9056c48740 1 --2- 172.21.15.120:0/1245463237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f705f59370 0x55f705f5a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.886+0000 7f9056c48740 1 --2- 172.21.15.120:0/1245463237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f705f4fb40 0x55f705f57f40 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.886+0000 7f9056c48740 1 --2- 172.21.15.120:0/1245463237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f705f4e5d0 0x55f705f652b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.886+0000 7f9056c48740 1 -- 172.21.15.120:0/1245463237 >> 172.21.15.120:0/1245463237 conn(0x55f705f17310 msgr2=0x55f705f57830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:37.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.886+0000 7f9056c48740 1 -- 172.21.15.120:0/1245463237 shutdown_connections 2024-09-19T09:20:37.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.886+0000 7f9056c48740 1 -- 172.21.15.120:0/1245463237 wait complete. 2024-09-19T09:20:37.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.887+0000 7f9056c48740 1 -- start start 2024-09-19T09:20:37.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.887+0000 7f9056c48740 1 Processor -- start 2024-09-19T09:20:37.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.887+0000 7f9056c48740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f705f4e5d0 0x55f705f616a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:37.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.887+0000 7f9056c48740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f705f4fb40 0x55f705f61be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:37.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.887+0000 7f9056c48740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f705f59370 0x55f705f5c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:37.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.887+0000 7f9056c48740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f90501ac700 con 0x55f705f59370 2024-09-19T09:20:37.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.887+0000 7f9056c48740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f90501ac030 con 0x55f705f4fb40 2024-09-19T09:20:37.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.887+0000 7f9056c48740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f90501a9270 con 0x55f705f4e5d0 2024-09-19T09:20:37.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.887+0000 7f904ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f705f59370 0x55f705f5c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:37.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.887+0000 7f904ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f705f59370 0x55f705f5c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40234/0 (socket says 172.21.15.120:40234) 2024-09-19T09:20:37.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.887+0000 7f904ed9d640 1 -- 172.21.15.120:0/399897477 learned_addr learned my addr 172.21.15.120:0/399897477 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:37.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.887+0000 7f904e59c640 1 --2- 172.21.15.120:0/399897477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f705f4e5d0 0x55f705f616a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:37.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.887+0000 7f904dd9b640 1 --2- 172.21.15.120:0/399897477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f705f4fb40 0x55f705f61be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:37.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.887+0000 7f9056c48740 10 client.0 _ll_register_callbacks cb 0x7f9040009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:20:37.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.888+0000 7f904ed9d640 1 -- 172.21.15.120:0/399897477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f705f4e5d0 msgr2=0x55f705f616a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:37.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.888+0000 7f904ed9d640 1 --2- 172.21.15.120:0/399897477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f705f4e5d0 0x55f705f616a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.888+0000 7f904ed9d640 1 -- 172.21.15.120:0/399897477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f705f4fb40 msgr2=0x55f705f61be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:37.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.888+0000 7f904ed9d640 1 --2- 172.21.15.120:0/399897477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f705f4fb40 0x55f705f61be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.888+0000 7f904ed9d640 1 -- 172.21.15.120:0/399897477 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f705f5cc70 con 0x55f705f59370 2024-09-19T09:20:37.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.888+0000 7f904e59c640 1 --2- 172.21.15.120:0/399897477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f705f4e5d0 0x55f705f616a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:20:37.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.888+0000 7f904ed9d640 1 --2- 172.21.15.120:0/399897477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f705f59370 0x55f705f5c440 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f9048098f00 tx=0x7f90480095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b86d85f95b520a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:37.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.888+0000 7f9035ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:20:37.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.888+0000 7f9035ffb640 1 -- 172.21.15.120:0/399897477 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90480bc040 con 0x55f705f59370 2024-09-19T09:20:37.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.888+0000 7f9035ffb640 1 -- 172.21.15.120:0/399897477 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f90480ba260 con 0x55f705f59370 2024-09-19T09:20:37.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.888+0000 7f9056c48740 1 -- 172.21.15.120:0/399897477 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9044049040 con 0x55f705f59370 2024-09-19T09:20:37.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.888+0000 7f9035ffb640 1 -- 172.21.15.120:0/399897477 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f90480a51a0 con 0x55f705f59370 2024-09-19T09:20:37.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.888+0000 7f9056c48740 10 client.21366 fetch_fsmap learned FSMap version 39 2024-09-19T09:20:37.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.888+0000 7f9056c48740 1 -- 172.21.15.120:0/399897477 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f705fdcaf0 con 0x55f705f59370 2024-09-19T09:20:37.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.889+0000 7f9035ffb640 1 -- 172.21.15.120:0/399897477 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9044049040 con 0x55f705f59370 2024-09-19T09:20:37.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.889+0000 7f9035ffb640 1 -- 172.21.15.120:0/399897477 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f9048081020 con 0x55f705f59370 2024-09-19T09:20:37.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.889+0000 7f9056c48740 10 client.21366 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:20:37.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.889+0000 7f9056c48740 10 client.21366 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:20:37.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.889+0000 7f9056c48740 20 client.21366 populate_metadata read hostname 'smithi120' 2024-09-19T09:20:37.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.889+0000 7f9056c48740 1 --2- 172.21.15.120:0/399897477 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55f705fdd1c0 0x55f705fdf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:37.978 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:20:37.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.889+0000 7f9056c48740 4 client.21366 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:20:37.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.889+0000 7f9056c48740 1 -- 172.21.15.120:0/399897477 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f705ebbd70 con 0x55f705fdd1c0 2024-09-19T09:20:37.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.889+0000 7f904e59c640 1 --2- 172.21.15.120:0/399897477 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55f705fdd1c0 0x55f705fdf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:37.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.889+0000 7f9035ffb640 10 client.21366 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:20:37.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.889+0000 7f904e59c640 1 --2- 172.21.15.120:0/399897477 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55f705fdd1c0 0x55f705fdf5e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f09eebede0aec5c0 server_cookie=8e4aafb45982852d in_seq=0 out_seq=0 2024-09-19T09:20:37.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.890+0000 7f9035ffb640 1 -- 172.21.15.120:0/399897477 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f705ebbd70 con 0x55f705fdd1c0 2024-09-19T09:20:37.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.890+0000 7f9035ffb640 10 client.21366 handle_command_reply: tid=0 2024-09-19T09:20:37.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.890+0000 7f9056c48740 1 client.21366 shutdown 2024-09-19T09:20:37.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.890+0000 7f9056c48740 1 -- 172.21.15.120:0/399897477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f705f59370 msgr2=0x55f705f5c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:37.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.890+0000 7f9056c48740 1 --2- 172.21.15.120:0/399897477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f705f59370 0x55f705f5c440 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f9048098f00 tx=0x7f90480095b0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.890+0000 7f9056c48740 1 -- 172.21.15.120:0/399897477 shutdown_connections 2024-09-19T09:20:37.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.890+0000 7f9056c48740 1 --2- 172.21.15.120:0/399897477 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55f705fdd1c0 0x55f705fdf5e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.890+0000 7f9056c48740 1 --2- 172.21.15.120:0/399897477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f705f59370 0x55f705f5c440 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.890+0000 7f9056c48740 1 --2- 172.21.15.120:0/399897477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f705f4fb40 0x55f705f61be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.890+0000 7f9056c48740 1 --2- 172.21.15.120:0/399897477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f705f4e5d0 0x55f705f616a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.890+0000 7f9056c48740 1 -- 172.21.15.120:0/399897477 >> 172.21.15.120:0/399897477 conn(0x55f705f17310 msgr2=0x55f705f57830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:37.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.890+0000 7f9035ffb640 0 client.21366 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:20:37.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.890+0000 7f9056c48740 1 -- 172.21.15.120:0/399897477 shutdown_connections 2024-09-19T09:20:37.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.890+0000 7f9056c48740 1 -- 172.21.15.120:0/399897477 wait complete. 2024-09-19T09:20:37.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.890+0000 7f9056c48740 20 client.21366 trim_cache size 0 max 16384 2024-09-19T09:20:37.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.891+0000 7f905486f640 1 -- 172.21.15.120:0/2831256595 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f90380a3240 msgr2=0x7f90380a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:37.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.891+0000 7f905486f640 1 --2- 172.21.15.120:0/2831256595 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f90380a3240 0x7f90380a5700 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9040011550 tx=0x7f9040005f90 comp rx=0 tx=0).stop 2024-09-19T09:20:37.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.891+0000 7f905486f640 1 -- 172.21.15.120:0/2831256595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9050102ef0 msgr2=0x7f905019cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:37.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.891+0000 7f905486f640 1 --2- 172.21.15.120:0/2831256595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9050102ef0 0x7f905019cd80 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f9048009fd0 tx=0x7f904800ef10 comp rx=0 tx=0).stop 2024-09-19T09:20:37.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.891+0000 7f905486f640 1 -- 172.21.15.120:0/2831256595 shutdown_connections 2024-09-19T09:20:37.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.891+0000 7f905486f640 1 --2- 172.21.15.120:0/2831256595 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f90380a3240 0x7f90380a5700 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.891+0000 7f905486f640 1 --2- 172.21.15.120:0/2831256595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9050102ef0 0x7f905019cd80 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.891+0000 7f905486f640 1 --2- 172.21.15.120:0/2831256595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9050102570 0x7f90501989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.891+0000 7f905486f640 1 --2- 172.21.15.120:0/2831256595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9050100e20 0x7f90501984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:37.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.891+0000 7f905486f640 1 -- 172.21.15.120:0/2831256595 >> 172.21.15.120:0/2831256595 conn(0x7f90500fc7d0 msgr2=0x7f90500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:37.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.891+0000 7f905486f640 1 -- 172.21.15.120:0/2831256595 shutdown_connections 2024-09-19T09:20:37.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:37.891+0000 7f905486f640 1 -- 172.21.15.120:0/2831256595 wait complete. 2024-09-19T09:20:38.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:38 smithi120 ceph-mon[31150]: pgmap v878: 97 pgs: 97 active+clean; 375 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.3 MiB/s wr, 686 op/s 2024-09-19T09:20:38.367 INFO:tasks.workunit.client.0.smithi120.stdout: 10 115815 21.15 MB/sec execute 153 sec latency 102.798 ms 2024-09-19T09:20:38.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:38 smithi196 ceph-mon[35150]: pgmap v878: 97 pgs: 97 active+clean; 375 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.3 MiB/s wr, 686 op/s 2024-09-19T09:20:38.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:38 smithi153 ceph-mon[35071]: pgmap v878: 97 pgs: 97 active+clean; 375 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.3 MiB/s wr, 686 op/s 2024-09-19T09:20:39.367 INFO:tasks.workunit.client.0.smithi120.stdout: 10 116250 21.12 MB/sec execute 154 sec latency 210.304 ms 2024-09-19T09:20:39.549 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:20:39.860 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:20:40.283 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:40 smithi120 ceph-mon[31150]: pgmap v879: 97 pgs: 97 active+clean; 367 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.5 MiB/s wr, 792 op/s 2024-09-19T09:20:40.367 INFO:tasks.workunit.client.0.smithi120.stdout: 10 116754 21.14 MB/sec execute 155 sec latency 102.083 ms 2024-09-19T09:20:40.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.374+0000 7f32ce4cd640 1 -- 172.21.15.120:0/2424062226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32c8100ca0 msgr2=0x7f32c8106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:40.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.374+0000 7f32ce4cd640 1 --2- 172.21.15.120:0/2424062226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32c8100ca0 0x7f32c8106840 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009930 tx=0x7f32bc02a420 comp rx=0 tx=0).stop 2024-09-19T09:20:40.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.374+0000 7f32ce4cd640 1 -- 172.21.15.120:0/2424062226 shutdown_connections 2024-09-19T09:20:40.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.374+0000 7f32ce4cd640 1 --2- 172.21.15.120:0/2424062226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32c8106f50 0x7f32c810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.374+0000 7f32ce4cd640 1 --2- 172.21.15.120:0/2424062226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32c8100ca0 0x7f32c8106840 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.374+0000 7f32ce4cd640 1 --2- 172.21.15.120:0/2424062226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32c81002f0 0x7f32c81006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.374+0000 7f32ce4cd640 1 -- 172.21.15.120:0/2424062226 >> 172.21.15.120:0/2424062226 conn(0x7f32c8077220 msgr2=0x7f32c80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:40.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.374+0000 7f32ce4cd640 1 -- 172.21.15.120:0/2424062226 shutdown_connections 2024-09-19T09:20:40.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.374+0000 7f32ce4cd640 1 -- 172.21.15.120:0/2424062226 wait complete. 2024-09-19T09:20:40.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.375+0000 7f32ce4cd640 1 Processor -- start 2024-09-19T09:20:40.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.375+0000 7f32ce4cd640 1 -- start start 2024-09-19T09:20:40.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.375+0000 7f32ce4cd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32c81002f0 0x7f32c8198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:40.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.375+0000 7f32ce4cd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32c8100ca0 0x7f32c8198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:40.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.375+0000 7f32ce4cd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32c8106f50 0x7f32c819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:40.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.375+0000 7f32ce4cd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f32c8111fa0 con 0x7f32c8106f50 2024-09-19T09:20:40.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.375+0000 7f32ce4cd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f32c8111e20 con 0x7f32c81002f0 2024-09-19T09:20:40.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.375+0000 7f32ce4cd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f32c8112120 con 0x7f32c8100ca0 2024-09-19T09:20:40.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.376+0000 7f32c7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32c81002f0 0x7f32c8198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:40.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.376+0000 7f32cca6a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32c8106f50 0x7f32c819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:40.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.376+0000 7f32c7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32c81002f0 0x7f32c8198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:60622/0 (socket says 172.21.15.120:60622) 2024-09-19T09:20:40.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.376+0000 7f32c77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32c8100ca0 0x7f32c8198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:40.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.376+0000 7f32c7fff640 1 -- 172.21.15.120:0/3787962553 learned_addr learned my addr 172.21.15.120:0/3787962553 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:40.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.376+0000 7f32c7fff640 1 -- 172.21.15.120:0/3787962553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32c8100ca0 msgr2=0x7f32c8198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:40.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.376+0000 7f32c7fff640 1 --2- 172.21.15.120:0/3787962553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32c8100ca0 0x7f32c8198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.376+0000 7f32c7fff640 1 -- 172.21.15.120:0/3787962553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32c8106f50 msgr2=0x7f32c819cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:40.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.376+0000 7f32c7fff640 1 --2- 172.21.15.120:0/3787962553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32c8106f50 0x7f32c819cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.376+0000 7f32c7fff640 1 -- 172.21.15.120:0/3787962553 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c819d4d0 con 0x7f32c81002f0 2024-09-19T09:20:40.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.376+0000 7f32cca6a640 1 --2- 172.21.15.120:0/3787962553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32c8106f50 0x7f32c819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:20:40.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.376+0000 7f32c7fff640 1 --2- 172.21.15.120:0/3787962553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32c81002f0 0x7f32c8198520 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f32b800e9f0 tx=0x7f32b800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:40.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.376+0000 7f32c57fa640 1 -- 172.21.15.120:0/3787962553 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b800cd80 con 0x7f32c81002f0 2024-09-19T09:20:40.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.376+0000 7f32ce4cd640 1 -- 172.21.15.120:0/3787962553 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c819d7c0 con 0x7f32c81002f0 2024-09-19T09:20:40.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.376+0000 7f32c57fa640 1 -- 172.21.15.120:0/3787962553 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32b8010910 con 0x7f32c81002f0 2024-09-19T09:20:40.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.376+0000 7f32c57fa640 1 -- 172.21.15.120:0/3787962553 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b80188f0 con 0x7f32c81002f0 2024-09-19T09:20:40.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.376+0000 7f32ce4cd640 1 -- 172.21.15.120:0/3787962553 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c81a50a0 con 0x7f32c81002f0 2024-09-19T09:20:40.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.379+0000 7f32c57fa640 1 -- 172.21.15.120:0/3787962553 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f32b8010ac0 con 0x7f32c81002f0 2024-09-19T09:20:40.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.379+0000 7f32d08a6740 1 Processor -- start 2024-09-19T09:20:40.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.379+0000 7f32d08a6740 1 -- start start 2024-09-19T09:20:40.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.379+0000 7f32d08a6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560db3a3d840 0x560db3a3dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:40.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.379+0000 7f32d08a6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560db3a3c120 0x560db3a3c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:40.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.379+0000 7f32d08a6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560db3a3cb80 0x560db3a51f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:40.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.379+0000 7f32d08a6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560db39a22f0 con 0x560db3a3d840 2024-09-19T09:20:40.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.379+0000 7f32d08a6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560db3949cf0 con 0x560db3a3cb80 2024-09-19T09:20:40.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.379+0000 7f32d08a6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560db399e2b0 con 0x560db3a3c120 2024-09-19T09:20:40.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.380+0000 7f32cca6a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560db3a3d840 0x560db3a3dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:40.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.380+0000 7f32cca6a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560db3a3d840 0x560db3a3dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40256/0 (socket says 172.21.15.120:40256) 2024-09-19T09:20:40.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.380+0000 7f32cca6a640 1 -- 172.21.15.120:0/4174408911 learned_addr learned my addr 172.21.15.120:0/4174408911 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:40.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.380+0000 7f32c77fe640 1 --2- 172.21.15.120:0/4174408911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560db3a3cb80 0x560db3a51f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:40.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.380+0000 7f32c7fff640 1 --2- 172.21.15.120:0/4174408911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560db3a3c120 0x560db3a3c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:40.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.380+0000 7f32cca6a640 1 -- 172.21.15.120:0/4174408911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560db3a3c120 msgr2=0x560db3a3c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:40.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.380+0000 7f32cca6a640 1 --2- 172.21.15.120:0/4174408911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560db3a3c120 0x560db3a3c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.380+0000 7f32cca6a640 1 -- 172.21.15.120:0/4174408911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560db3a3cb80 msgr2=0x560db3a51f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:40.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.380+0000 7f32cca6a640 1 --2- 172.21.15.120:0/4174408911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560db3a3cb80 0x560db3a51f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.380+0000 7f32cca6a640 1 -- 172.21.15.120:0/4174408911 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560db392e3a0 con 0x560db3a3d840 2024-09-19T09:20:40.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.380+0000 7f32c7fff640 1 --2- 172.21.15.120:0/4174408911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560db3a3c120 0x560db3a3c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:20:40.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.380+0000 7f32c57fa640 1 --2- 172.21.15.120:0/3787962553 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f32b00a3240 0x7f32b00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:40.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.380+0000 7f32c7fff640 1 --2- 172.21.15.120:0/3787962553 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f32b00a3240 0x7f32b00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:40.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.380+0000 7f32c57fa640 1 -- 172.21.15.120:0/3787962553 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f32b80cfca0 con 0x7f32c81002f0 2024-09-19T09:20:40.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.380+0000 7f32cca6a640 1 --2- 172.21.15.120:0/4174408911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560db3a3d840 0x560db3a3dc20 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f32c0011790 tx=0x7f32c0011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ae28c26f34bc91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:40.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.381+0000 7f32ae7fc640 1 -- 172.21.15.120:0/4174408911 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c0003c30 con 0x560db3a3d840 2024-09-19T09:20:40.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.381+0000 7f32c7fff640 1 --2- 172.21.15.120:0/3787962553 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f32b00a3240 0x7f32b00a5700 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f32b809c070 tx=0x7f32b801ff90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:40.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.381+0000 7f32ae7fc640 1 -- 172.21.15.120:0/4174408911 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32c000fe80 con 0x560db3a3d840 2024-09-19T09:20:40.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.381+0000 7f32ae7fc640 1 -- 172.21.15.120:0/4174408911 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c00193e0 con 0x560db3a3d840 2024-09-19T09:20:40.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.382+0000 7f32d08a6740 1 -- 172.21.15.120:0/4174408911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560db3a3d840 msgr2=0x560db3a3dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:40.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.382+0000 7f32d08a6740 1 --2- 172.21.15.120:0/4174408911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560db3a3d840 0x560db3a3dc20 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f32c0011790 tx=0x7f32c0011c50 comp rx=0 tx=0).stop 2024-09-19T09:20:40.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.382+0000 7f32d08a6740 1 -- 172.21.15.120:0/4174408911 shutdown_connections 2024-09-19T09:20:40.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.382+0000 7f32d08a6740 1 --2- 172.21.15.120:0/4174408911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560db3a3cb80 0x560db3a51f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.382+0000 7f32d08a6740 1 --2- 172.21.15.120:0/4174408911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560db3a3c120 0x560db3a3c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.382+0000 7f32d08a6740 1 --2- 172.21.15.120:0/4174408911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560db3a3d840 0x560db3a3dc20 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.382+0000 7f32d08a6740 1 -- 172.21.15.120:0/4174408911 >> 172.21.15.120:0/4174408911 conn(0x560db3a05310 msgr2=0x560db3a38c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:40.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.382+0000 7f32d08a6740 1 -- 172.21.15.120:0/4174408911 shutdown_connections 2024-09-19T09:20:40.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.382+0000 7f32d08a6740 1 -- 172.21.15.120:0/4174408911 wait complete. 2024-09-19T09:20:40.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.382+0000 7f32d08a6740 1 -- start start 2024-09-19T09:20:40.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.383+0000 7f32d08a6740 1 Processor -- start 2024-09-19T09:20:40.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.383+0000 7f32d08a6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560db3a3c120 0x560db3a51de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:40.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.383+0000 7f32d08a6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560db3a3cb80 0x560db3a4f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:40.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.383+0000 7f32d08a6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560db3a3d840 0x560db3a4f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:40.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.383+0000 7f32d08a6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560db3a52930 con 0x560db3a3d840 2024-09-19T09:20:40.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.383+0000 7f32d08a6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560db395d060 con 0x560db3a3cb80 2024-09-19T09:20:40.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.383+0000 7f32d08a6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560db3a52ab0 con 0x560db3a3c120 2024-09-19T09:20:40.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.383+0000 7f32d08a6740 10 client.0 _ll_register_callbacks cb 0x560db36b08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:20:40.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.383+0000 7f32c77fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560db3a3cb80 0x560db3a4f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:40.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.383+0000 7f32cca6a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560db3a3d840 0x560db3a4f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:40.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.383+0000 7f32c77fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560db3a3cb80 0x560db3a4f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:60652/0 (socket says 172.21.15.120:60652) 2024-09-19T09:20:40.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.383+0000 7f32c77fe640 1 -- 172.21.15.120:0/1843215796 learned_addr learned my addr 172.21.15.120:0/1843215796 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:40.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.383+0000 7f32cca6a640 1 --2- 172.21.15.120:0/1843215796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560db3a3c120 0x560db3a51de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:40.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.384+0000 7f32c77fe640 1 -- 172.21.15.120:0/1843215796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560db3a3c120 msgr2=0x560db3a51de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:40.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.384+0000 7f32c77fe640 1 --2- 172.21.15.120:0/1843215796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560db3a3c120 0x560db3a51de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.384+0000 7f32c77fe640 1 -- 172.21.15.120:0/1843215796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560db3a3d840 msgr2=0x560db3a4f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:40.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.384+0000 7f32c77fe640 1 --2- 172.21.15.120:0/1843215796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560db3a3d840 0x560db3a4f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.384+0000 7f32c77fe640 1 -- 172.21.15.120:0/1843215796 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560db37133c0 con 0x560db3a3cb80 2024-09-19T09:20:40.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.384+0000 7f32cca6a640 1 --2- 172.21.15.120:0/1843215796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560db3a3c120 0x560db3a51de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:20:40.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.384+0000 7f32c77fe640 1 --2- 172.21.15.120:0/1843215796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560db3a3cb80 0x560db3a4f090 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009930 tx=0x7f32bc0317e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69aca3fee3d4995a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:40.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.384+0000 7f32a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:20:40.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.384+0000 7f32a77fe640 1 -- 172.21.15.120:0/1843215796 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc035040 con 0x560db3a3cb80 2024-09-19T09:20:40.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.384+0000 7f32a77fe640 1 -- 172.21.15.120:0/1843215796 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f32bc004240 con 0x560db3a3cb80 2024-09-19T09:20:40.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.384+0000 7f32d08a6740 1 -- 172.21.15.120:0/1843215796 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560db39af860 con 0x560db3a3cb80 2024-09-19T09:20:40.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.384+0000 7f32a77fe640 1 -- 172.21.15.120:0/1843215796 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f32bc004440 con 0x560db3a3cb80 2024-09-19T09:20:40.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.384+0000 7f32d08a6740 10 client.30242 fetch_fsmap learned FSMap version 39 2024-09-19T09:20:40.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.384+0000 7f32d08a6740 1 -- 172.21.15.120:0/1843215796 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560db3a5e370 con 0x560db3a3cb80 2024-09-19T09:20:40.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.384+0000 7f32a77fe640 1 -- 172.21.15.120:0/1843215796 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560db39af860 con 0x560db3a3cb80 2024-09-19T09:20:40.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.385+0000 7f32a77fe640 1 -- 172.21.15.120:0/1843215796 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f32bc004bb0 con 0x560db3a3cb80 2024-09-19T09:20:40.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.385+0000 7f32d08a6740 10 client.30242 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:20:40.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.385+0000 7f32d08a6740 10 client.30242 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:20:40.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.385+0000 7f32d08a6740 20 client.30242 populate_metadata read hostname 'smithi120' 2024-09-19T09:20:40.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.385+0000 7f32d08a6740 1 --2- 172.21.15.120:0/1843215796 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x560db3a5f830 0x560db3a61c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:40.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.385+0000 7f32d08a6740 4 client.30242 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:20:40.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.385+0000 7f32d08a6740 1 -- 172.21.15.120:0/1843215796 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560db39a9d70 con 0x560db3a5f830 2024-09-19T09:20:40.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.385+0000 7f32cca6a640 1 --2- 172.21.15.120:0/1843215796 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x560db3a5f830 0x560db3a61c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:40.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.386+0000 7f32cca6a640 1 --2- 172.21.15.120:0/1843215796 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x560db3a5f830 0x560db3a61c20 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=84ad3c33ffe456ab server_cookie=8154ebc7d7358e5f in_seq=0 out_seq=0 2024-09-19T09:20:40.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.386+0000 7f32a77fe640 10 client.30242 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:20:40.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.387+0000 7f32a77fe640 1 -- 172.21.15.120:0/1843215796 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560db39a9d70 con 0x560db3a5f830 2024-09-19T09:20:40.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.387+0000 7f32a77fe640 10 client.30242 handle_command_reply: tid=0 2024-09-19T09:20:40.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.387+0000 7f32d08a6740 1 client.30242 shutdown 2024-09-19T09:20:40.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.387+0000 7f32d08a6740 1 -- 172.21.15.120:0/1843215796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560db3a3cb80 msgr2=0x560db3a4f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:40.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.387+0000 7f32d08a6740 1 --2- 172.21.15.120:0/1843215796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560db3a3cb80 0x560db3a4f090 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009930 tx=0x7f32bc0317e0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.387+0000 7f32d08a6740 1 -- 172.21.15.120:0/1843215796 shutdown_connections 2024-09-19T09:20:40.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.387+0000 7f32d08a6740 1 --2- 172.21.15.120:0/1843215796 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x560db3a5f830 0x560db3a61c20 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.387+0000 7f32d08a6740 1 --2- 172.21.15.120:0/1843215796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560db3a3d840 0x560db3a4f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.387+0000 7f32d08a6740 1 --2- 172.21.15.120:0/1843215796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560db3a3cb80 0x560db3a4f090 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.387+0000 7f32d08a6740 1 --2- 172.21.15.120:0/1843215796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560db3a3c120 0x560db3a51de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.387+0000 7f32d08a6740 1 -- 172.21.15.120:0/1843215796 >> 172.21.15.120:0/1843215796 conn(0x560db3a05310 msgr2=0x560db3a37d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:40.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.387+0000 7f32a77fe640 0 client.30242 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:20:40.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.388+0000 7f32d08a6740 1 -- 172.21.15.120:0/1843215796 shutdown_connections 2024-09-19T09:20:40.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.388+0000 7f32d08a6740 1 -- 172.21.15.120:0/1843215796 wait complete. 2024-09-19T09:20:40.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.388+0000 7f32d08a6740 20 client.30242 trim_cache size 0 max 16384 2024-09-19T09:20:40.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.403+0000 7f32d08a6740 1 Processor -- start 2024-09-19T09:20:40.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.403+0000 7f32d08a6740 1 -- start start 2024-09-19T09:20:40.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.403+0000 7f32d08a6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560db3a3c120 0x560db3a5bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:40.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.404+0000 7f32d08a6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560db3a3cb80 0x560db3a3ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:40.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.404+0000 7f32d08a6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560db3a3d840 0x560db3a4e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:40.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.404+0000 7f32d08a6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f32c81ab2e0 con 0x560db3a3cb80 2024-09-19T09:20:40.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.404+0000 7f32d08a6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f32c81ace20 con 0x560db3a3d840 2024-09-19T09:20:40.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.404+0000 7f32d08a6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f32c81ab9b0 con 0x560db3a3c120 2024-09-19T09:20:40.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.404+0000 7f32c77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560db3a3cb80 0x560db3a3ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:40.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.404+0000 7f32c77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560db3a3cb80 0x560db3a3ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40274/0 (socket says 172.21.15.120:40274) 2024-09-19T09:20:40.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.404+0000 7f32c77fe640 1 -- 172.21.15.120:0/2458307578 learned_addr learned my addr 172.21.15.120:0/2458307578 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:40.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.404+0000 7f32cca6a640 1 --2- 172.21.15.120:0/2458307578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560db3a3c120 0x560db3a5bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:40.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.404+0000 7f32cca6a640 1 --2- 172.21.15.120:0/2458307578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560db3a3d840 0x560db3a4e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:40.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.404+0000 7f32c77fe640 1 -- 172.21.15.120:0/2458307578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560db3a3c120 msgr2=0x560db3a5bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:40.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.404+0000 7f32c77fe640 1 --2- 172.21.15.120:0/2458307578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560db3a3c120 0x560db3a5bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.404+0000 7f32c77fe640 1 -- 172.21.15.120:0/2458307578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560db3a3d840 msgr2=0x560db3a4e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:40.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.404+0000 7f32c77fe640 1 --2- 172.21.15.120:0/2458307578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560db3a3d840 0x560db3a4e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.404+0000 7f32c77fe640 1 -- 172.21.15.120:0/2458307578 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560db3a4d5e0 con 0x560db3a3cb80 2024-09-19T09:20:40.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.404+0000 7f32cca6a640 1 --2- 172.21.15.120:0/2458307578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560db3a3d840 0x560db3a4e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:20:40.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.404+0000 7f32c77fe640 1 --2- 172.21.15.120:0/2458307578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560db3a3cb80 0x560db3a3ec30 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f32bc005940 tx=0x7f32bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb5d4c02e6f763a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:40.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.404+0000 7f32adffb640 1 -- 172.21.15.120:0/2458307578 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc000e60 con 0x560db3a3cb80 2024-09-19T09:20:40.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.404+0000 7f32adffb640 1 -- 172.21.15.120:0/2458307578 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32bc055070 con 0x560db3a3cb80 2024-09-19T09:20:40.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.404+0000 7f32adffb640 1 -- 172.21.15.120:0/2458307578 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc035070 con 0x560db3a3cb80 2024-09-19T09:20:40.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.405+0000 7f32d08a6740 1 -- 172.21.15.120:0/2458307578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560db3a3cb80 msgr2=0x560db3a3ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:40.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.405+0000 7f32d08a6740 1 --2- 172.21.15.120:0/2458307578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560db3a3cb80 0x560db3a3ec30 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f32bc005940 tx=0x7f32bc0046c0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.405+0000 7f32d08a6740 1 -- 172.21.15.120:0/2458307578 shutdown_connections 2024-09-19T09:20:40.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.405+0000 7f32d08a6740 1 --2- 172.21.15.120:0/2458307578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560db3a3d840 0x560db3a4e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.405+0000 7f32d08a6740 1 --2- 172.21.15.120:0/2458307578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560db3a3cb80 0x560db3a3ec30 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.405+0000 7f32d08a6740 1 --2- 172.21.15.120:0/2458307578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560db3a3c120 0x560db3a5bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.405+0000 7f32d08a6740 1 -- 172.21.15.120:0/2458307578 >> 172.21.15.120:0/2458307578 conn(0x560db3a05310 msgr2=0x560db3a4e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:40.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.405+0000 7f32d08a6740 1 -- 172.21.15.120:0/2458307578 shutdown_connections 2024-09-19T09:20:40.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.405+0000 7f32d08a6740 1 -- 172.21.15.120:0/2458307578 wait complete. 2024-09-19T09:20:40.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.406+0000 7f32d08a6740 1 -- start start 2024-09-19T09:20:40.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.406+0000 7f32d08a6740 1 Processor -- start 2024-09-19T09:20:40.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.406+0000 7f32d08a6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560db3a3c120 0x560db3a41330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:40.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.406+0000 7f32d08a6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560db3a3d840 0x560db3a41870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:40.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.406+0000 7f32d08a6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560db3a41db0 0x560db3a564a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:40.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.406+0000 7f32d08a6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f32c81ac750 con 0x560db3a3c120 2024-09-19T09:20:40.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.406+0000 7f32d08a6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f32c81ac080 con 0x560db3a3d840 2024-09-19T09:20:40.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.406+0000 7f32d08a6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f32c81a92a0 con 0x560db3a41db0 2024-09-19T09:20:40.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.406+0000 7f32d08a6740 10 client.0 _ll_register_callbacks cb 0x7f32bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:20:40.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.406+0000 7f32cca6a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560db3a3c120 0x560db3a41330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:40.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.406+0000 7f32cca6a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560db3a3c120 0x560db3a41330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40286/0 (socket says 172.21.15.120:40286) 2024-09-19T09:20:40.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.406+0000 7f32cca6a640 1 -- 172.21.15.120:0/563426292 learned_addr learned my addr 172.21.15.120:0/563426292 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:40.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.407+0000 7f32cca6a640 1 -- 172.21.15.120:0/563426292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560db3a41db0 msgr2=0x560db3a564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.407+0000 7f32cca6a640 1 --2- 172.21.15.120:0/563426292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560db3a41db0 0x560db3a564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.407+0000 7f32cca6a640 1 -- 172.21.15.120:0/563426292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560db3a3d840 msgr2=0x560db3a41870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.407+0000 7f32cca6a640 1 --2- 172.21.15.120:0/563426292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560db3a3d840 0x560db3a41870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.407+0000 7f32cca6a640 1 -- 172.21.15.120:0/563426292 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560db3a56a90 con 0x560db3a3c120 2024-09-19T09:20:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.407+0000 7f32cca6a640 1 --2- 172.21.15.120:0/563426292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560db3a3c120 0x560db3a41330 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f32c00115a0 tx=0x7f32c0004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f33e956b56a70d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.407+0000 7f32a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:20:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.407+0000 7f32a77fe640 1 -- 172.21.15.120:0/563426292 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c0033060 con 0x560db3a3c120 2024-09-19T09:20:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.407+0000 7f32a77fe640 1 -- 172.21.15.120:0/563426292 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f32c000fc70 con 0x560db3a3c120 2024-09-19T09:20:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.407+0000 7f32d08a6740 1 -- 172.21.15.120:0/563426292 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32bc055070 con 0x560db3a3c120 2024-09-19T09:20:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.407+0000 7f32a77fe640 1 -- 172.21.15.120:0/563426292 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f32c0011020 con 0x560db3a3c120 2024-09-19T09:20:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.407+0000 7f32d08a6740 10 client.21384 fetch_fsmap learned FSMap version 39 2024-09-19T09:20:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.407+0000 7f32d08a6740 1 -- 172.21.15.120:0/563426292 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560db3acad60 con 0x560db3a3c120 2024-09-19T09:20:40.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.408+0000 7f32a77fe640 1 -- 172.21.15.120:0/563426292 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32bc055070 con 0x560db3a3c120 2024-09-19T09:20:40.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.408+0000 7f32a77fe640 1 -- 172.21.15.120:0/563426292 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f32c0018070 con 0x560db3a3c120 2024-09-19T09:20:40.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.408+0000 7f32d08a6740 10 client.21384 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:20:40.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.408+0000 7f32d08a6740 10 client.21384 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:20:40.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.408+0000 7f32d08a6740 20 client.21384 populate_metadata read hostname 'smithi120' 2024-09-19T09:20:40.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.408+0000 7f32d08a6740 1 --2- 172.21.15.120:0/563426292 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x560db3acb2e0 0x560db3acd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:40.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.408+0000 7f32d08a6740 4 client.21384 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:20:40.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.408+0000 7f32d08a6740 1 -- 172.21.15.120:0/563426292 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560db3926170 con 0x560db3acb2e0 2024-09-19T09:20:40.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.408+0000 7f32c77fe640 1 --2- 172.21.15.120:0/563426292 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x560db3acb2e0 0x560db3acd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:40.427 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:20:40.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.409+0000 7f32c77fe640 1 --2- 172.21.15.120:0/563426292 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x560db3acb2e0 0x560db3acd6d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d08075d38170afff server_cookie=bf3e89f4c8af112e in_seq=0 out_seq=0 2024-09-19T09:20:40.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.409+0000 7f32a77fe640 10 client.21384 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:20:40.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.409+0000 7f32a77fe640 1 -- 172.21.15.120:0/563426292 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560db3926170 con 0x560db3acb2e0 2024-09-19T09:20:40.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.409+0000 7f32a77fe640 10 client.21384 handle_command_reply: tid=0 2024-09-19T09:20:40.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.409+0000 7f32d08a6740 1 client.21384 shutdown 2024-09-19T09:20:40.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.409+0000 7f32d08a6740 1 -- 172.21.15.120:0/563426292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560db3a3c120 msgr2=0x560db3a41330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:40.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.409+0000 7f32d08a6740 1 --2- 172.21.15.120:0/563426292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560db3a3c120 0x560db3a41330 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f32c00115a0 tx=0x7f32c0004620 comp rx=0 tx=0).stop 2024-09-19T09:20:40.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.409+0000 7f32d08a6740 1 -- 172.21.15.120:0/563426292 shutdown_connections 2024-09-19T09:20:40.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.409+0000 7f32d08a6740 1 --2- 172.21.15.120:0/563426292 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x560db3acb2e0 0x560db3acd6d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.409+0000 7f32d08a6740 1 --2- 172.21.15.120:0/563426292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560db3a41db0 0x560db3a564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.409+0000 7f32d08a6740 1 --2- 172.21.15.120:0/563426292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560db3a3d840 0x560db3a41870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.409+0000 7f32a77fe640 0 client.21384 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:20:40.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.409+0000 7f32d08a6740 1 --2- 172.21.15.120:0/563426292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560db3a3c120 0x560db3a41330 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.409+0000 7f32d08a6740 1 -- 172.21.15.120:0/563426292 >> 172.21.15.120:0/563426292 conn(0x560db3a05310 msgr2=0x560db3a508f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:40.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.409+0000 7f32d08a6740 1 -- 172.21.15.120:0/563426292 shutdown_connections 2024-09-19T09:20:40.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.409+0000 7f32d08a6740 1 -- 172.21.15.120:0/563426292 wait complete. 2024-09-19T09:20:40.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.409+0000 7f32d08a6740 20 client.21384 trim_cache size 0 max 16384 2024-09-19T09:20:40.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.410+0000 7f32ce4cd640 1 -- 172.21.15.120:0/3787962553 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f32b00a3240 msgr2=0x7f32b00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:40.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.410+0000 7f32ce4cd640 1 --2- 172.21.15.120:0/3787962553 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f32b00a3240 0x7f32b00a5700 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f32b809c070 tx=0x7f32b801ff90 comp rx=0 tx=0).stop 2024-09-19T09:20:40.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.410+0000 7f32ce4cd640 1 -- 172.21.15.120:0/3787962553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32c81002f0 msgr2=0x7f32c8198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:40.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.410+0000 7f32ce4cd640 1 --2- 172.21.15.120:0/3787962553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32c81002f0 0x7f32c8198520 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f32b800e9f0 tx=0x7f32b800eeb0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.410+0000 7f32ce4cd640 1 -- 172.21.15.120:0/3787962553 shutdown_connections 2024-09-19T09:20:40.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.410+0000 7f32ce4cd640 1 --2- 172.21.15.120:0/3787962553 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f32b00a3240 0x7f32b00a5700 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.410+0000 7f32ce4cd640 1 --2- 172.21.15.120:0/3787962553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32c8106f50 0x7f32c819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.410+0000 7f32ce4cd640 1 --2- 172.21.15.120:0/3787962553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32c8100ca0 0x7f32c8198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.410+0000 7f32ce4cd640 1 --2- 172.21.15.120:0/3787962553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32c81002f0 0x7f32c8198520 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:40.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.410+0000 7f32ce4cd640 1 -- 172.21.15.120:0/3787962553 >> 172.21.15.120:0/3787962553 conn(0x7f32c8077220 msgr2=0x7f32c80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:40.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.411+0000 7f32ce4cd640 1 -- 172.21.15.120:0/3787962553 shutdown_connections 2024-09-19T09:20:40.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:40.411+0000 7f32ce4cd640 1 -- 172.21.15.120:0/3787962553 wait complete. 2024-09-19T09:20:40.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:40 smithi196 ceph-mon[35150]: pgmap v879: 97 pgs: 97 active+clean; 367 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.5 MiB/s wr, 792 op/s 2024-09-19T09:20:40.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:40 smithi153 ceph-mon[35071]: pgmap v879: 97 pgs: 97 active+clean; 367 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.5 MiB/s wr, 792 op/s 2024-09-19T09:20:41.367 INFO:tasks.workunit.client.0.smithi120.stdout: 10 117539 21.30 MB/sec execute 156 sec latency 74.446 ms 2024-09-19T09:20:41.446 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:20:41.731 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:20:42.107 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:42 smithi120 ceph-mon[31150]: pgmap v880: 97 pgs: 97 active+clean; 373 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.6 MiB/s wr, 801 op/s 2024-09-19T09:20:42.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.334+0000 7ff99cfd6640 1 -- 172.21.15.120:0/2238921143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff998106f20 msgr2=0x7ff99810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:42.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.334+0000 7ff99cfd6640 1 --2- 172.21.15.120:0/2238921143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff998106f20 0x7ff99810eaa0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7ff990009ca0 tx=0x7ff99002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.334+0000 7ff99cfd6640 1 -- 172.21.15.120:0/2238921143 shutdown_connections 2024-09-19T09:20:42.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.334+0000 7ff99cfd6640 1 --2- 172.21.15.120:0/2238921143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff998106f20 0x7ff99810eaa0 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.334+0000 7ff99cfd6640 1 --2- 172.21.15.120:0/2238921143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9981027a0 0x7ff9981067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.334+0000 7ff99cfd6640 1 --2- 172.21.15.120:0/2238921143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff998101df0 0x7ff9981021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.334+0000 7ff99cfd6640 1 -- 172.21.15.120:0/2238921143 >> 172.21.15.120:0/2238921143 conn(0x7ff9980fd8f0 msgr2=0x7ff9980ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:42.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.334+0000 7ff99cfd6640 1 -- 172.21.15.120:0/2238921143 shutdown_connections 2024-09-19T09:20:42.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.334+0000 7ff99cfd6640 1 -- 172.21.15.120:0/2238921143 wait complete. 2024-09-19T09:20:42.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.335+0000 7ff99cfd6640 1 Processor -- start 2024-09-19T09:20:42.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.335+0000 7ff99cfd6640 1 -- start start 2024-09-19T09:20:42.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.335+0000 7ff99cfd6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff998101df0 0x7ff9981983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:42.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.335+0000 7ff99cfd6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9981027a0 0x7ff998198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:42.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.335+0000 7ff99cfd6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff998106f20 0x7ff99819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:42.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.335+0000 7ff99cfd6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff998111f80 con 0x7ff998101df0 2024-09-19T09:20:42.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.335+0000 7ff99cfd6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff998111e00 con 0x7ff9981027a0 2024-09-19T09:20:42.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.335+0000 7ff99cfd6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff998112100 con 0x7ff998106f20 2024-09-19T09:20:42.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.335+0000 7ff99659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff998101df0 0x7ff9981983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:42.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.335+0000 7ff99659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff998101df0 0x7ff9981983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42902/0 (socket says 172.21.15.120:42902) 2024-09-19T09:20:42.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.335+0000 7ff99659c640 1 -- 172.21.15.120:0/2953774113 learned_addr learned my addr 172.21.15.120:0/2953774113 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:42.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.335+0000 7ff995d9b640 1 --2- 172.21.15.120:0/2953774113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9981027a0 0x7ff998198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:42.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.335+0000 7ff996d9d640 1 --2- 172.21.15.120:0/2953774113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff998106f20 0x7ff99819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:42.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.336+0000 7ff99659c640 1 -- 172.21.15.120:0/2953774113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff998106f20 msgr2=0x7ff99819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:42.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.336+0000 7ff99659c640 1 --2- 172.21.15.120:0/2953774113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff998106f20 0x7ff99819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.336+0000 7ff99659c640 1 -- 172.21.15.120:0/2953774113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9981027a0 msgr2=0x7ff998198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:42.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.336+0000 7ff99659c640 1 --2- 172.21.15.120:0/2953774113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9981027a0 0x7ff998198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.336+0000 7ff99659c640 1 -- 172.21.15.120:0/2953774113 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99819d390 con 0x7ff998101df0 2024-09-19T09:20:42.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.336+0000 7ff995d9b640 1 --2- 172.21.15.120:0/2953774113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9981027a0 0x7ff998198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:20:42.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.336+0000 7ff99659c640 1 --2- 172.21.15.120:0/2953774113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff998101df0 0x7ff9981983e0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7ff98800d770 tx=0x7ff98800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:42.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.336+0000 7ff9877fe640 1 -- 172.21.15.120:0/2953774113 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff988017070 con 0x7ff998101df0 2024-09-19T09:20:42.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.336+0000 7ff99cfd6640 1 -- 172.21.15.120:0/2953774113 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff99819d680 con 0x7ff998101df0 2024-09-19T09:20:42.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.336+0000 7ff9877fe640 1 -- 172.21.15.120:0/2953774113 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff988005500 con 0x7ff998101df0 2024-09-19T09:20:42.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.336+0000 7ff9877fe640 1 -- 172.21.15.120:0/2953774113 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff988004e00 con 0x7ff998101df0 2024-09-19T09:20:42.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.337+0000 7ff99cfd6640 1 -- 172.21.15.120:0/2953774113 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9981a4f60 con 0x7ff998101df0 2024-09-19T09:20:42.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.339+0000 7ff9877fe640 1 -- 172.21.15.120:0/2953774113 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7ff988004fa0 con 0x7ff998101df0 2024-09-19T09:20:42.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.340+0000 7ff99f3af740 1 Processor -- start 2024-09-19T09:20:42.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.340+0000 7ff99f3af740 1 -- start start 2024-09-19T09:20:42.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.340+0000 7ff9877fe640 1 --2- 172.21.15.120:0/2953774113 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff9800a32c0 0x7ff9800a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:42.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.340+0000 7ff99f3af740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c984451b40 0x55c984450090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:42.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.340+0000 7ff99f3af740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9844505d0 0x55c984450a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:42.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.340+0000 7ff99f3af740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c98445b370 0x55c98445d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:42.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.340+0000 7ff99f3af740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c9843b62f0 con 0x55c984451b40 2024-09-19T09:20:42.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.340+0000 7ff99f3af740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c98435dcf0 con 0x55c98445b370 2024-09-19T09:20:42.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.340+0000 7ff99f3af740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c9843b22b0 con 0x55c9844505d0 2024-09-19T09:20:42.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.340+0000 7ff9877fe640 1 -- 172.21.15.120:0/2953774113 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff9880cf230 con 0x7ff998101df0 2024-09-19T09:20:42.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.340+0000 7ff996d9d640 1 --2- 172.21.15.120:0/2953774113 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff9800a32c0 0x7ff9800a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:42.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.340+0000 7ff99659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c984451b40 0x55c984450090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:42.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.340+0000 7ff99659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c984451b40 0x55c984450090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42906/0 (socket says 172.21.15.120:42906) 2024-09-19T09:20:42.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.340+0000 7ff99659c640 1 -- 172.21.15.120:0/2686002062 learned_addr learned my addr 172.21.15.120:0/2686002062 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:42.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.340+0000 7ff995d9b640 1 --2- 172.21.15.120:0/2686002062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9844505d0 0x55c984450a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:42.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.341+0000 7ff996d9d640 1 --2- 172.21.15.120:0/2686002062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c98445b370 0x55c98445d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:42.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.341+0000 7ff99659c640 1 -- 172.21.15.120:0/2686002062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9844505d0 msgr2=0x55c984450a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:42.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.341+0000 7ff99659c640 1 --2- 172.21.15.120:0/2686002062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9844505d0 0x55c984450a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.341+0000 7ff99659c640 1 -- 172.21.15.120:0/2686002062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c98445b370 msgr2=0x55c98445d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:42.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.341+0000 7ff99659c640 1 --2- 172.21.15.120:0/2686002062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c98445b370 0x55c98445d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.341+0000 7ff99659c640 1 -- 172.21.15.120:0/2686002062 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9843423a0 con 0x55c984451b40 2024-09-19T09:20:42.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.341+0000 7ff995d9b640 1 --2- 172.21.15.120:0/2686002062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9844505d0 0x55c984450a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:20:42.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.341+0000 7ff996d9d640 1 --2- 172.21.15.120:0/2953774113 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff9800a32c0 0x7ff9800a5780 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff990004af0 tx=0x7ff990031c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:42.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.341+0000 7ff99659c640 1 --2- 172.21.15.120:0/2686002062 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c984451b40 0x55c984450090 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7ff988097d60 tx=0x7ff98809b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a1af48c5717fa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:42.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.341+0000 7ff984ff9640 1 -- 172.21.15.120:0/2686002062 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff988020070 con 0x55c984451b40 2024-09-19T09:20:42.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.341+0000 7ff984ff9640 1 -- 172.21.15.120:0/2686002062 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff98801a360 con 0x55c984451b40 2024-09-19T09:20:42.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.341+0000 7ff984ff9640 1 -- 172.21.15.120:0/2686002062 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9880a44c0 con 0x55c984451b40 2024-09-19T09:20:42.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.342+0000 7ff99f3af740 1 -- 172.21.15.120:0/2686002062 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c984451b40 msgr2=0x55c984450090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:42.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.342+0000 7ff99f3af740 1 --2- 172.21.15.120:0/2686002062 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c984451b40 0x55c984450090 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7ff988097d60 tx=0x7ff98809b390 comp rx=0 tx=0).stop 2024-09-19T09:20:42.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.342+0000 7ff99f3af740 1 -- 172.21.15.120:0/2686002062 shutdown_connections 2024-09-19T09:20:42.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.342+0000 7ff99f3af740 1 --2- 172.21.15.120:0/2686002062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c98445b370 0x55c98445d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.342+0000 7ff99f3af740 1 --2- 172.21.15.120:0/2686002062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9844505d0 0x55c984450a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.342+0000 7ff99f3af740 1 --2- 172.21.15.120:0/2686002062 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c984451b40 0x55c984450090 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.342+0000 7ff99f3af740 1 -- 172.21.15.120:0/2686002062 >> 172.21.15.120:0/2686002062 conn(0x55c984419310 msgr2=0x55c98444cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:42.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.342+0000 7ff99f3af740 1 -- 172.21.15.120:0/2686002062 shutdown_connections 2024-09-19T09:20:42.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.342+0000 7ff99f3af740 1 -- 172.21.15.120:0/2686002062 wait complete. 2024-09-19T09:20:42.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.342+0000 7ff99f3af740 1 -- start start 2024-09-19T09:20:42.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.342+0000 7ff99f3af740 1 Processor -- start 2024-09-19T09:20:42.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.343+0000 7ff99f3af740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9844505d0 0x55c98445ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:42.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.343+0000 7ff99f3af740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c984451b40 0x55c984458020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:42.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.343+0000 7ff99f3af740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c98445b370 0x55c984458610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:42.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.343+0000 7ff99f3af740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c98445e070 con 0x55c9844505d0 2024-09-19T09:20:42.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.343+0000 7ff99f3af740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c984371060 con 0x55c98445b370 2024-09-19T09:20:42.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.343+0000 7ff99f3af740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c98445e1f0 con 0x55c984451b40 2024-09-19T09:20:42.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.343+0000 7ff99f3af740 10 client.0 _ll_register_callbacks cb 0x55c9840c48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:20:42.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.343+0000 7ff99659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9844505d0 0x55c98445ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:42.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.343+0000 7ff99659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9844505d0 0x55c98445ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42908/0 (socket says 172.21.15.120:42908) 2024-09-19T09:20:42.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.343+0000 7ff99659c640 1 -- 172.21.15.120:0/4166980895 learned_addr learned my addr 172.21.15.120:0/4166980895 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:42.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.343+0000 7ff995d9b640 1 --2- 172.21.15.120:0/4166980895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c984451b40 0x55c984458020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:42.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.343+0000 7ff996d9d640 1 --2- 172.21.15.120:0/4166980895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c98445b370 0x55c984458610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:42.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.343+0000 7ff99659c640 1 -- 172.21.15.120:0/4166980895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c984451b40 msgr2=0x55c984458020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:42.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.343+0000 7ff99659c640 1 --2- 172.21.15.120:0/4166980895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c984451b40 0x55c984458020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.343+0000 7ff99659c640 1 -- 172.21.15.120:0/4166980895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c98445b370 msgr2=0x55c984458610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:42.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.343+0000 7ff99659c640 1 --2- 172.21.15.120:0/4166980895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c98445b370 0x55c984458610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.343+0000 7ff99659c640 1 -- 172.21.15.120:0/4166980895 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9841273c0 con 0x55c9844505d0 2024-09-19T09:20:42.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.343+0000 7ff996d9d640 1 --2- 172.21.15.120:0/4166980895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c98445b370 0x55c984458610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:20:42.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.343+0000 7ff995d9b640 1 --2- 172.21.15.120:0/4166980895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c984451b40 0x55c984458020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:20:42.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.343+0000 7ff99659c640 1 --2- 172.21.15.120:0/4166980895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9844505d0 0x55c98445ad70 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7ff988009610 tx=0x7ff9880048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876e99d43eb249be server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:42.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.343+0000 7ff97dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:20:42.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.343+0000 7ff97dffb640 1 -- 172.21.15.120:0/4166980895 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9880b3070 con 0x55c9844505d0 2024-09-19T09:20:42.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.343+0000 7ff97dffb640 1 -- 172.21.15.120:0/4166980895 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff9880ba1f0 con 0x55c9844505d0 2024-09-19T09:20:42.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.344+0000 7ff99f3af740 1 -- 172.21.15.120:0/4166980895 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff98801a360 con 0x55c9844505d0 2024-09-19T09:20:42.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.344+0000 7ff97dffb640 1 -- 172.21.15.120:0/4166980895 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff988020050 con 0x55c9844505d0 2024-09-19T09:20:42.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.344+0000 7ff99f3af740 10 client.21396 fetch_fsmap learned FSMap version 39 2024-09-19T09:20:42.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.344+0000 7ff99f3af740 1 -- 172.21.15.120:0/4166980895 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c984469a80 con 0x55c9844505d0 2024-09-19T09:20:42.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.344+0000 7ff97dffb640 1 -- 172.21.15.120:0/4166980895 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff98801a360 con 0x55c9844505d0 2024-09-19T09:20:42.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.344+0000 7ff97dffb640 1 -- 172.21.15.120:0/4166980895 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7ff988097f70 con 0x55c9844505d0 2024-09-19T09:20:42.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.344+0000 7ff99f3af740 10 client.21396 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:20:42.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.344+0000 7ff99f3af740 10 client.21396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:20:42.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.344+0000 7ff99f3af740 20 client.21396 populate_metadata read hostname 'smithi120' 2024-09-19T09:20:42.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.344+0000 7ff99f3af740 1 --2- 172.21.15.120:0/4166980895 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c98446add0 0x55c98446d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:42.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.344+0000 7ff99f3af740 4 client.21396 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:20:42.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.344+0000 7ff99f3af740 1 -- 172.21.15.120:0/4166980895 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9843c3860 con 0x55c98446add0 2024-09-19T09:20:42.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.344+0000 7ff995d9b640 1 --2- 172.21.15.120:0/4166980895 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c98446add0 0x55c98446d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:42.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.345+0000 7ff97dffb640 10 client.21396 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:42.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.345+0000 7ff995d9b640 1 --2- 172.21.15.120:0/4166980895 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c98446add0 0x55c98446d1c0 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a05e0b48faf80e24 server_cookie=f21fc89dc0dca22b in_seq=0 out_seq=0 2024-09-19T09:20:42.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.347+0000 7ff97dffb640 1 -- 172.21.15.120:0/4166980895 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c9843c3860 con 0x55c98446add0 2024-09-19T09:20:42.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.347+0000 7ff97dffb640 10 client.21396 handle_command_reply: tid=0 2024-09-19T09:20:42.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.347+0000 7ff99f3af740 1 client.21396 shutdown 2024-09-19T09:20:42.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.347+0000 7ff99f3af740 1 -- 172.21.15.120:0/4166980895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9844505d0 msgr2=0x55c98445ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:42.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.347+0000 7ff99f3af740 1 --2- 172.21.15.120:0/4166980895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9844505d0 0x55c98445ad70 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7ff988009610 tx=0x7ff9880048a0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.347+0000 7ff99f3af740 1 -- 172.21.15.120:0/4166980895 shutdown_connections 2024-09-19T09:20:42.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.347+0000 7ff99f3af740 1 --2- 172.21.15.120:0/4166980895 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c98446add0 0x55c98446d1c0 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.347+0000 7ff99f3af740 1 --2- 172.21.15.120:0/4166980895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c98445b370 0x55c984458610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.347+0000 7ff99f3af740 1 --2- 172.21.15.120:0/4166980895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c984451b40 0x55c984458020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.347+0000 7ff97dffb640 0 client.21396 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:42.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.347+0000 7ff99f3af740 1 --2- 172.21.15.120:0/4166980895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9844505d0 0x55c98445ad70 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.347+0000 7ff99f3af740 1 -- 172.21.15.120:0/4166980895 >> 172.21.15.120:0/4166980895 conn(0x55c984419310 msgr2=0x55c984452960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:42.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.347+0000 7ff99f3af740 1 -- 172.21.15.120:0/4166980895 shutdown_connections 2024-09-19T09:20:42.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.347+0000 7ff99f3af740 1 -- 172.21.15.120:0/4166980895 wait complete. 2024-09-19T09:20:42.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.347+0000 7ff99f3af740 20 client.21396 trim_cache size 0 max 16384 2024-09-19T09:20:42.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.361+0000 7ff99f3af740 1 Processor -- start 2024-09-19T09:20:42.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.361+0000 7ff99f3af740 1 -- start start 2024-09-19T09:20:42.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.362+0000 7ff99f3af740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9844505d0 0x55c984453120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:42.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.362+0000 7ff99f3af740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c984451b40 0x55c984453660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:42.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.362+0000 7ff99f3af740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c98445b370 0x55c9844701e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:42.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.362+0000 7ff99f3af740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff9981ad5b0 con 0x55c98445b370 2024-09-19T09:20:42.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.362+0000 7ff99f3af740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff9981af0f0 con 0x55c984451b40 2024-09-19T09:20:42.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.362+0000 7ff99f3af740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff9981adc80 con 0x55c9844505d0 2024-09-19T09:20:42.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.362+0000 7ff996d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c98445b370 0x55c9844701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:42.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.362+0000 7ff995d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c984451b40 0x55c984453660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:42.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.362+0000 7ff996d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c98445b370 0x55c9844701e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42922/0 (socket says 172.21.15.120:42922) 2024-09-19T09:20:42.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.362+0000 7ff996d9d640 1 -- 172.21.15.120:0/1411622118 learned_addr learned my addr 172.21.15.120:0/1411622118 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:42.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.362+0000 7ff99659c640 1 --2- 172.21.15.120:0/1411622118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9844505d0 0x55c984453120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:42.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.362+0000 7ff996d9d640 1 -- 172.21.15.120:0/1411622118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9844505d0 msgr2=0x55c984453120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:42.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.362+0000 7ff996d9d640 1 --2- 172.21.15.120:0/1411622118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9844505d0 0x55c984453120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.362+0000 7ff996d9d640 1 -- 172.21.15.120:0/1411622118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c984451b40 msgr2=0x55c984453660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:42.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.362+0000 7ff996d9d640 1 --2- 172.21.15.120:0/1411622118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c984451b40 0x55c984453660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.362+0000 7ff996d9d640 1 -- 172.21.15.120:0/1411622118 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c98445b0e0 con 0x55c98445b370 2024-09-19T09:20:42.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.362+0000 7ff996d9d640 1 --2- 172.21.15.120:0/1411622118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c98445b370 0x55c9844701e0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7ff990009ca0 tx=0x7ff990049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8cdd801610c9366 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:42.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.362+0000 7ff97ffff640 1 -- 172.21.15.120:0/1411622118 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990045040 con 0x55c98445b370 2024-09-19T09:20:42.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.363+0000 7ff97ffff640 1 -- 172.21.15.120:0/1411622118 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff990050070 con 0x55c98445b370 2024-09-19T09:20:42.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.363+0000 7ff97ffff640 1 -- 172.21.15.120:0/1411622118 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99002fed0 con 0x55c98445b370 2024-09-19T09:20:42.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.363+0000 7ff99f3af740 1 -- 172.21.15.120:0/1411622118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c98445b370 msgr2=0x55c9844701e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:42.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.363+0000 7ff99f3af740 1 --2- 172.21.15.120:0/1411622118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c98445b370 0x55c9844701e0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7ff990009ca0 tx=0x7ff990049000 comp rx=0 tx=0).stop 2024-09-19T09:20:42.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.363+0000 7ff99f3af740 1 -- 172.21.15.120:0/1411622118 shutdown_connections 2024-09-19T09:20:42.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.363+0000 7ff99f3af740 1 --2- 172.21.15.120:0/1411622118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c98445b370 0x55c9844701e0 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.363+0000 7ff99f3af740 1 --2- 172.21.15.120:0/1411622118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c984451b40 0x55c984453660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.363+0000 7ff99f3af740 1 --2- 172.21.15.120:0/1411622118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9844505d0 0x55c984453120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.363+0000 7ff99f3af740 1 -- 172.21.15.120:0/1411622118 >> 172.21.15.120:0/1411622118 conn(0x55c984419310 msgr2=0x55c984457110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:42.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.363+0000 7ff99f3af740 1 -- 172.21.15.120:0/1411622118 shutdown_connections 2024-09-19T09:20:42.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.363+0000 7ff99f3af740 1 -- 172.21.15.120:0/1411622118 wait complete. 2024-09-19T09:20:42.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.364+0000 7ff99f3af740 1 -- start start 2024-09-19T09:20:42.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.364+0000 7ff99f3af740 1 Processor -- start 2024-09-19T09:20:42.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.364+0000 7ff99f3af740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9844505d0 0x55c9844ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:42.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.364+0000 7ff99f3af740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c984451b40 0x55c9844ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:42.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.364+0000 7ff99f3af740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c98445b370 0x55c98446d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:42.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.364+0000 7ff99f3af740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff9981aea20 con 0x55c984451b40 2024-09-19T09:20:42.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.364+0000 7ff99f3af740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff9981ae350 con 0x55c98445b370 2024-09-19T09:20:42.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.364+0000 7ff99f3af740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff9981ab570 con 0x55c9844505d0 2024-09-19T09:20:42.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.364+0000 7ff99f3af740 10 client.0 _ll_register_callbacks cb 0x7ff98809b260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:20:42.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.364+0000 7ff996d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c98445b370 0x55c98446d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:42.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.364+0000 7ff996d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c98445b370 0x55c98446d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:45724/0 (socket says 172.21.15.120:45724) 2024-09-19T09:20:42.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.364+0000 7ff996d9d640 1 -- 172.21.15.120:0/1525896021 learned_addr learned my addr 172.21.15.120:0/1525896021 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:42.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.364+0000 7ff995d9b640 1 --2- 172.21.15.120:0/1525896021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c984451b40 0x55c9844ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:42.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.364+0000 7ff99659c640 1 --2- 172.21.15.120:0/1525896021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9844505d0 0x55c9844ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:42.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.365+0000 7ff996d9d640 1 -- 172.21.15.120:0/1525896021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9844505d0 msgr2=0x55c9844ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:42.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.365+0000 7ff996d9d640 1 --2- 172.21.15.120:0/1525896021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9844505d0 0x55c9844ccaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.365+0000 7ff996d9d640 1 -- 172.21.15.120:0/1525896021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c984451b40 msgr2=0x55c9844ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:42.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.365+0000 7ff996d9d640 1 --2- 172.21.15.120:0/1525896021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c984451b40 0x55c9844ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.365+0000 7ff996d9d640 1 -- 172.21.15.120:0/1525896021 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c98446dfd0 con 0x55c98445b370 2024-09-19T09:20:42.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.365+0000 7ff995d9b640 1 --2- 172.21.15.120:0/1525896021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c984451b40 0x55c9844ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:20:42.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.365+0000 7ff996d9d640 1 --2- 172.21.15.120:0/1525896021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c98445b370 0x55c98446d7a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff99002ade0 tx=0x7ff99002a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ce5162f149524b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:42.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.365+0000 7ff97dffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:20:42.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.365+0000 7ff97dffb640 1 -- 172.21.15.120:0/1525896021 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990045040 con 0x55c98445b370 2024-09-19T09:20:42.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.365+0000 7ff97dffb640 1 -- 172.21.15.120:0/1525896021 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff990004450 con 0x55c98445b370 2024-09-19T09:20:42.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.365+0000 7ff99f3af740 1 -- 172.21.15.120:0/1525896021 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff990050070 con 0x55c98445b370 2024-09-19T09:20:42.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.365+0000 7ff97dffb640 1 -- 172.21.15.120:0/1525896021 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff99002f610 con 0x55c98445b370 2024-09-19T09:20:42.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.365+0000 7ff99f3af740 10 client.30263 fetch_fsmap learned FSMap version 39 2024-09-19T09:20:42.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.365+0000 7ff99f3af740 1 -- 172.21.15.120:0/1525896021 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c9844d5240 con 0x55c98445b370 2024-09-19T09:20:42.373 INFO:tasks.workunit.client.0.smithi120.stdout: 10 118238 21.41 MB/sec execute 157 sec latency 54.905 ms 2024-09-19T09:20:42.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.365+0000 7ff97dffb640 1 -- 172.21.15.120:0/1525896021 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff990050070 con 0x55c98445b370 2024-09-19T09:20:42.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.366+0000 7ff97dffb640 1 -- 172.21.15.120:0/1525896021 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7ff990030bc0 con 0x55c98445b370 2024-09-19T09:20:42.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.366+0000 7ff99f3af740 10 client.30263 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:20:42.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.366+0000 7ff99f3af740 10 client.30263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:20:42.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.366+0000 7ff99f3af740 20 client.30263 populate_metadata read hostname 'smithi120' 2024-09-19T09:20:42.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.366+0000 7ff99f3af740 1 --2- 172.21.15.120:0/1525896021 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c98446add0 0x55c9844d7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:42.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.366+0000 7ff99f3af740 4 client.30263 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:20:42.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.366+0000 7ff99f3af740 1 -- 172.21.15.120:0/1525896021 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c9843bdd70 con 0x55c98446add0 2024-09-19T09:20:42.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.366+0000 7ff99659c640 1 --2- 172.21.15.120:0/1525896021 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c98446add0 0x55c9844d7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:42.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.366+0000 7ff97dffb640 10 client.30263 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:42.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.366+0000 7ff99659c640 1 --2- 172.21.15.120:0/1525896021 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c98446add0 0x55c9844d7910 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=481e25ac45ff384a server_cookie=2db31687da5064a in_seq=0 out_seq=0 2024-09-19T09:20:42.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.372+0000 7ff97dffb640 1 -- 172.21.15.120:0/1525896021 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c9843bdd70 con 0x55c98446add0 2024-09-19T09:20:42.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.372+0000 7ff97dffb640 10 client.30263 handle_command_reply: tid=0 2024-09-19T09:20:42.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.372+0000 7ff99f3af740 1 client.30263 shutdown 2024-09-19T09:20:42.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.372+0000 7ff99f3af740 1 -- 172.21.15.120:0/1525896021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c98445b370 msgr2=0x55c98446d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:42.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.372+0000 7ff99f3af740 1 --2- 172.21.15.120:0/1525896021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c98445b370 0x55c98446d7a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff99002ade0 tx=0x7ff99002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.372+0000 7ff99f3af740 1 -- 172.21.15.120:0/1525896021 shutdown_connections 2024-09-19T09:20:42.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.372+0000 7ff99f3af740 1 --2- 172.21.15.120:0/1525896021 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c98446add0 0x55c9844d7910 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.372+0000 7ff99f3af740 1 --2- 172.21.15.120:0/1525896021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c98445b370 0x55c98446d7a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.372+0000 7ff99f3af740 1 --2- 172.21.15.120:0/1525896021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c984451b40 0x55c9844ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.372+0000 7ff99f3af740 1 --2- 172.21.15.120:0/1525896021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c9844505d0 0x55c9844ccaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.372+0000 7ff97dffb640 0 client.30263 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:42.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.372+0000 7ff99f3af740 1 -- 172.21.15.120:0/1525896021 >> 172.21.15.120:0/1525896021 conn(0x55c984419310 msgr2=0x55c98444b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:42.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.372+0000 7ff99f3af740 1 -- 172.21.15.120:0/1525896021 shutdown_connections 2024-09-19T09:20:42.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.372+0000 7ff99f3af740 1 -- 172.21.15.120:0/1525896021 wait complete. 2024-09-19T09:20:42.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.372+0000 7ff99f3af740 20 client.30263 trim_cache size 0 max 16384 2024-09-19T09:20:42.376 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:20:42.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.373+0000 7ff99cfd6640 1 -- 172.21.15.120:0/2953774113 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff9800a32c0 msgr2=0x7ff9800a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:42.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.373+0000 7ff99cfd6640 1 --2- 172.21.15.120:0/2953774113 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff9800a32c0 0x7ff9800a5780 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff990004af0 tx=0x7ff990031c90 comp rx=0 tx=0).stop 2024-09-19T09:20:42.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.373+0000 7ff99cfd6640 1 -- 172.21.15.120:0/2953774113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff998101df0 msgr2=0x7ff9981983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:42.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.373+0000 7ff99cfd6640 1 --2- 172.21.15.120:0/2953774113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff998101df0 0x7ff9981983e0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7ff98800d770 tx=0x7ff98800dc30 comp rx=0 tx=0).stop 2024-09-19T09:20:42.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.373+0000 7ff99cfd6640 1 -- 172.21.15.120:0/2953774113 shutdown_connections 2024-09-19T09:20:42.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.373+0000 7ff99cfd6640 1 --2- 172.21.15.120:0/2953774113 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff9800a32c0 0x7ff9800a5780 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.373+0000 7ff99cfd6640 1 --2- 172.21.15.120:0/2953774113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff998106f20 0x7ff99819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.373+0000 7ff99cfd6640 1 --2- 172.21.15.120:0/2953774113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9981027a0 0x7ff998198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.373+0000 7ff99cfd6640 1 --2- 172.21.15.120:0/2953774113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff998101df0 0x7ff9981983e0 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:42.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.373+0000 7ff99cfd6640 1 -- 172.21.15.120:0/2953774113 >> 172.21.15.120:0/2953774113 conn(0x7ff9980fd8f0 msgr2=0x7ff9980fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:42.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.373+0000 7ff99cfd6640 1 -- 172.21.15.120:0/2953774113 shutdown_connections 2024-09-19T09:20:42.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:42.373+0000 7ff99cfd6640 1 -- 172.21.15.120:0/2953774113 wait complete. 2024-09-19T09:20:42.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:42 smithi196 ceph-mon[35150]: pgmap v880: 97 pgs: 97 active+clean; 373 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.6 MiB/s wr, 801 op/s 2024-09-19T09:20:42.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:42 smithi153 ceph-mon[35071]: pgmap v880: 97 pgs: 97 active+clean; 373 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.6 MiB/s wr, 801 op/s 2024-09-19T09:20:43.367 INFO:tasks.workunit.client.0.smithi120.stdout: 10 118919 21.54 MB/sec execute 158 sec latency 86.728 ms 2024-09-19T09:20:44.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:44 smithi120 ceph-mon[31150]: pgmap v881: 97 pgs: 97 active+clean; 373 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.8 MiB/s wr, 711 op/s 2024-09-19T09:20:44.367 INFO:tasks.workunit.client.0.smithi120.stdout: 10 119336 21.51 MB/sec execute 159 sec latency 276.729 ms 2024-09-19T09:20:44.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:44 smithi196 ceph-mon[35150]: pgmap v881: 97 pgs: 97 active+clean; 373 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.8 MiB/s wr, 711 op/s 2024-09-19T09:20:44.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:44 smithi153 ceph-mon[35071]: pgmap v881: 97 pgs: 97 active+clean; 373 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.8 MiB/s wr, 711 op/s 2024-09-19T09:20:44.681 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:20:44.682 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:20:45.022 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:20:45.367 INFO:tasks.workunit.client.0.smithi120.stdout: 10 119907 21.62 MB/sec execute 160 sec latency 63.167 ms 2024-09-19T09:20:45.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.519+0000 7fd81e69c640 1 -- 172.21.15.120:0/4247097690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd818100e20 msgr2=0x7fd818101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:45.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.519+0000 7fd81e69c640 1 --2- 172.21.15.120:0/4247097690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd818100e20 0x7fd818101200 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd808009930 tx=0x7fd80802a4d0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.519+0000 7fd81e69c640 1 -- 172.21.15.120:0/4247097690 shutdown_connections 2024-09-19T09:20:45.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.519+0000 7fd81e69c640 1 --2- 172.21.15.120:0/4247097690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd818102ef0 0x7fd81810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.519+0000 7fd81e69c640 1 --2- 172.21.15.120:0/4247097690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd818102570 0x7fd8181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.519+0000 7fd81e69c640 1 --2- 172.21.15.120:0/4247097690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd818100e20 0x7fd818101200 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.519+0000 7fd81e69c640 1 -- 172.21.15.120:0/4247097690 >> 172.21.15.120:0/4247097690 conn(0x7fd8180fc7d0 msgr2=0x7fd8180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:45.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.519+0000 7fd81e69c640 1 -- 172.21.15.120:0/4247097690 shutdown_connections 2024-09-19T09:20:45.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.520+0000 7fd81e69c640 1 -- 172.21.15.120:0/4247097690 wait complete. 2024-09-19T09:20:45.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.520+0000 7fd81e69c640 1 Processor -- start 2024-09-19T09:20:45.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.520+0000 7fd81e69c640 1 -- start start 2024-09-19T09:20:45.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.521+0000 7fd81e69c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd818100e20 0x7fd818197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:45.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.521+0000 7fd81e69c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd818102570 0x7fd818197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:45.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.521+0000 7fd81e69c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd818102ef0 0x7fd81819bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:45.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.521+0000 7fd81e69c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd818110fa0 con 0x7fd818100e20 2024-09-19T09:20:45.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.521+0000 7fd81e69c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd818110e20 con 0x7fd818102ef0 2024-09-19T09:20:45.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.521+0000 7fd81e69c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd818111120 con 0x7fd818102570 2024-09-19T09:20:45.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.521+0000 7fd817fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd818100e20 0x7fd818197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:45.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.521+0000 7fd817fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd818100e20 0x7fd818197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42940/0 (socket says 172.21.15.120:42940) 2024-09-19T09:20:45.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.521+0000 7fd817fff640 1 -- 172.21.15.120:0/2961310129 learned_addr learned my addr 172.21.15.120:0/2961310129 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:45.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.521+0000 7fd8177fe640 1 --2- 172.21.15.120:0/2961310129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd818102570 0x7fd818197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:45.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.521+0000 7fd81cc39640 1 --2- 172.21.15.120:0/2961310129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd818102ef0 0x7fd81819bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:45.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.522+0000 7fd81cc39640 1 -- 172.21.15.120:0/2961310129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd818102570 msgr2=0x7fd818197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:45.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.522+0000 7fd81cc39640 1 --2- 172.21.15.120:0/2961310129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd818102570 0x7fd818197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.522+0000 7fd81cc39640 1 -- 172.21.15.120:0/2961310129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd818100e20 msgr2=0x7fd818197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:45.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.522+0000 7fd81cc39640 1 --2- 172.21.15.120:0/2961310129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd818100e20 0x7fd818197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.522+0000 7fd81cc39640 1 -- 172.21.15.120:0/2961310129 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81819c3e0 con 0x7fd818102ef0 2024-09-19T09:20:45.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.522+0000 7fd81cc39640 1 --2- 172.21.15.120:0/2961310129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd818102ef0 0x7fd81819bd00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd810009fd0 tx=0x7fd81000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:45.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.522+0000 7fd8157fa640 1 -- 172.21.15.120:0/2961310129 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd810004b20 con 0x7fd818102ef0 2024-09-19T09:20:45.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.522+0000 7fd8157fa640 1 -- 172.21.15.120:0/2961310129 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd810004cc0 con 0x7fd818102ef0 2024-09-19T09:20:45.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.522+0000 7fd8157fa640 1 -- 172.21.15.120:0/2961310129 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8100052a0 con 0x7fd818102ef0 2024-09-19T09:20:45.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.522+0000 7fd81e69c640 1 -- 172.21.15.120:0/2961310129 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd81819c6d0 con 0x7fd818102ef0 2024-09-19T09:20:45.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.522+0000 7fd81e69c640 1 -- 172.21.15.120:0/2961310129 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd818069f70 con 0x7fd818102ef0 2024-09-19T09:20:45.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.525+0000 7fd8157fa640 1 -- 172.21.15.120:0/2961310129 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd8100159e0 con 0x7fd818102ef0 2024-09-19T09:20:45.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.526+0000 7fd820a75740 1 Processor -- start 2024-09-19T09:20:45.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.526+0000 7fd820a75740 1 -- start start 2024-09-19T09:20:45.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.526+0000 7fd8157fa640 1 --2- 172.21.15.120:0/2961310129 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd8000a32c0 0x7fd8000a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:45.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.526+0000 7fd8157fa640 1 -- 172.21.15.120:0/2961310129 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd8100d3d60 con 0x7fd818102ef0 2024-09-19T09:20:45.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.526+0000 7fd820a75740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d674b60 0x56048d674f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:45.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.526+0000 7fd820a75740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d675480 0x56048d67f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:45.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.526+0000 7fd820a75740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56048d67fd40 0x56048d682130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:45.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.526+0000 7fd820a75740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56048d5da280 con 0x56048d675480 2024-09-19T09:20:45.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.526+0000 7fd820a75740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56048d581b10 con 0x56048d674b60 2024-09-19T09:20:45.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.526+0000 7fd820a75740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56048d5d6240 con 0x56048d67fd40 2024-09-19T09:20:45.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.526+0000 7fd8177fe640 1 --2- 172.21.15.120:0/2961310129 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd8000a32c0 0x7fd8000a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:45.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.527+0000 7fd8177fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d675480 0x56048d67f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:45.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.527+0000 7fd8177fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d675480 0x56048d67f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42954/0 (socket says 172.21.15.120:42954) 2024-09-19T09:20:45.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.527+0000 7fd8177fe640 1 -- 172.21.15.120:0/4008552196 learned_addr learned my addr 172.21.15.120:0/4008552196 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:45.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.527+0000 7fd817fff640 1 --2- 172.21.15.120:0/4008552196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d674b60 0x56048d674f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:45.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.527+0000 7fd817fff640 1 -- 172.21.15.120:0/4008552196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56048d67fd40 msgr2=0x56048d682130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:20:45.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.527+0000 7fd817fff640 1 --2- 172.21.15.120:0/4008552196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56048d67fd40 0x56048d682130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.527+0000 7fd817fff640 1 -- 172.21.15.120:0/4008552196 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d675480 msgr2=0x56048d67f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:45.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.527+0000 7fd817fff640 1 --2- 172.21.15.120:0/4008552196 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d675480 0x56048d67f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.527+0000 7fd817fff640 1 -- 172.21.15.120:0/4008552196 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56048d58e1f0 con 0x56048d674b60 2024-09-19T09:20:45.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.527+0000 7fd8177fe640 1 --2- 172.21.15.120:0/4008552196 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d675480 0x56048d67f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:20:45.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.527+0000 7fd8177fe640 1 --2- 172.21.15.120:0/2961310129 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd8000a32c0 0x7fd8000a5780 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd80c013570 tx=0x7fd80c013970 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:45.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.527+0000 7fd817fff640 1 --2- 172.21.15.120:0/4008552196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d674b60 0x56048d674f40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd808000c00 tx=0x7fd808005e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb70138728671b30 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:45.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.528+0000 7fd7fe7fc640 1 -- 172.21.15.120:0/4008552196 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd808040070 con 0x56048d674b60 2024-09-19T09:20:45.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.528+0000 7fd7fe7fc640 1 -- 172.21.15.120:0/4008552196 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd808004480 con 0x56048d674b60 2024-09-19T09:20:45.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.528+0000 7fd7fe7fc640 1 -- 172.21.15.120:0/4008552196 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd808035040 con 0x56048d674b60 2024-09-19T09:20:45.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.529+0000 7fd820a75740 1 -- 172.21.15.120:0/4008552196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d674b60 msgr2=0x56048d674f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:45.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.529+0000 7fd820a75740 1 --2- 172.21.15.120:0/4008552196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d674b60 0x56048d674f40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd808000c00 tx=0x7fd808005e30 comp rx=0 tx=0).stop 2024-09-19T09:20:45.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.529+0000 7fd820a75740 1 -- 172.21.15.120:0/4008552196 shutdown_connections 2024-09-19T09:20:45.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.529+0000 7fd820a75740 1 --2- 172.21.15.120:0/4008552196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56048d67fd40 0x56048d682130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.529+0000 7fd820a75740 1 --2- 172.21.15.120:0/4008552196 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d675480 0x56048d67f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.529+0000 7fd820a75740 1 --2- 172.21.15.120:0/4008552196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d674b60 0x56048d674f40 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.529+0000 7fd820a75740 1 -- 172.21.15.120:0/4008552196 >> 172.21.15.120:0/4008552196 conn(0x56048d6710f0 msgr2=0x56048d6714d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:45.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.529+0000 7fd820a75740 1 -- 172.21.15.120:0/4008552196 shutdown_connections 2024-09-19T09:20:45.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.529+0000 7fd820a75740 1 -- 172.21.15.120:0/4008552196 wait complete. 2024-09-19T09:20:45.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.529+0000 7fd820a75740 1 -- start start 2024-09-19T09:20:45.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.530+0000 7fd820a75740 1 Processor -- start 2024-09-19T09:20:45.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.531+0000 7fd820a75740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56048d674b60 0x56048d67cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:45.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.531+0000 7fd820a75740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d675480 0x56048d67d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:45.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.531+0000 7fd820a75740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d67fd40 0x56048d67d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:45.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.531+0000 7fd817fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56048d674b60 0x56048d67cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:45.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.531+0000 7fd8177fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d675480 0x56048d67d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:45.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.531+0000 7fd8177fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d675480 0x56048d67d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:45756/0 (socket says 172.21.15.120:45756) 2024-09-19T09:20:45.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.531+0000 7fd817fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56048d674b60 0x56048d67cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36380/0 (socket says 172.21.15.120:36380) 2024-09-19T09:20:45.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.531+0000 7fd8177fe640 1 -- 172.21.15.120:0/606727364 learned_addr learned my addr 172.21.15.120:0/606727364 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:45.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.532+0000 7fd820a75740 1 -- 172.21.15.120:0/606727364 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56048d682960 con 0x56048d67fd40 2024-09-19T09:20:45.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.532+0000 7fd820a75740 1 -- 172.21.15.120:0/606727364 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56048d594f10 con 0x56048d675480 2024-09-19T09:20:45.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.532+0000 7fd820a75740 1 -- 172.21.15.120:0/606727364 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56048d682ae0 con 0x56048d674b60 2024-09-19T09:20:45.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.532+0000 7fd820a75740 10 client.0 _ll_register_callbacks cb 0x56048d55da40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:20:45.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.532+0000 7fd8177fe640 1 -- 172.21.15.120:0/606727364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56048d674b60 msgr2=0x56048d67cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:45.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.532+0000 7fd8177fe640 1 --2- 172.21.15.120:0/606727364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56048d674b60 0x56048d67cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.532+0000 7fd8177fe640 1 -- 172.21.15.120:0/606727364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d67fd40 msgr2=0x56048d67d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:45.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.532+0000 7fd8177fe640 1 --2- 172.21.15.120:0/606727364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d67fd40 0x56048d67d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.532+0000 7fd8177fe640 1 -- 172.21.15.120:0/606727364 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56048d34b6a0 con 0x56048d675480 2024-09-19T09:20:45.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.533+0000 7fd8177fe640 1 --2- 172.21.15.120:0/606727364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d675480 0x56048d67d250 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd80c009cf0 tx=0x7fd80c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c5083236262780d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:45.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.533+0000 7fd7f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:20:45.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.533+0000 7fd7f77fe640 1 -- 172.21.15.120:0/606727364 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c024070 con 0x56048d675480 2024-09-19T09:20:45.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.533+0000 7fd7f77fe640 1 -- 172.21.15.120:0/606727364 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd80c003cf0 con 0x56048d675480 2024-09-19T09:20:45.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.533+0000 7fd820a75740 1 -- 172.21.15.120:0/606727364 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd808004480 con 0x56048d675480 2024-09-19T09:20:45.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.533+0000 7fd7f77fe640 1 -- 172.21.15.120:0/606727364 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd80c02f050 con 0x56048d675480 2024-09-19T09:20:45.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.533+0000 7fd820a75740 10 client.30275 fetch_fsmap learned FSMap version 39 2024-09-19T09:20:45.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.533+0000 7fd820a75740 1 -- 172.21.15.120:0/606727364 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56048d68e660 con 0x56048d675480 2024-09-19T09:20:45.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.534+0000 7fd7f77fe640 1 -- 172.21.15.120:0/606727364 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd808004480 con 0x56048d675480 2024-09-19T09:20:45.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.534+0000 7fd7f77fe640 1 -- 172.21.15.120:0/606727364 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fd80c01f070 con 0x56048d675480 2024-09-19T09:20:45.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.534+0000 7fd820a75740 10 client.30275 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:20:45.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.534+0000 7fd820a75740 10 client.30275 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:20:45.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.534+0000 7fd820a75740 20 client.30275 populate_metadata read hostname 'smithi120' 2024-09-19T09:20:45.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.534+0000 7fd820a75740 1 --2- 172.21.15.120:0/606727364 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56048d68fb30 0x56048d691fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:45.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.534+0000 7fd820a75740 4 client.30275 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:20:45.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.534+0000 7fd820a75740 1 -- 172.21.15.120:0/606727364 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56048d5e77f0 con 0x56048d68fb30 2024-09-19T09:20:45.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.535+0000 7fd817fff640 1 --2- 172.21.15.120:0/606727364 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56048d68fb30 0x56048d691fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:45.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.536+0000 7fd817fff640 1 --2- 172.21.15.120:0/606727364 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56048d68fb30 0x56048d691fd0 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1207e69fc14cde66 server_cookie=2b8954a75c04dee0 in_seq=0 out_seq=0 2024-09-19T09:20:45.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.536+0000 7fd7f77fe640 10 client.30275 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:45.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.540+0000 7fd7f77fe640 1 -- 172.21.15.120:0/606727364 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56048d5e77f0 con 0x56048d68fb30 2024-09-19T09:20:45.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.540+0000 7fd7f77fe640 10 client.30275 handle_command_reply: tid=0 2024-09-19T09:20:45.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.540+0000 7fd820a75740 1 client.30275 shutdown 2024-09-19T09:20:45.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.541+0000 7fd820a75740 1 -- 172.21.15.120:0/606727364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d675480 msgr2=0x56048d67d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:45.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.541+0000 7fd820a75740 1 --2- 172.21.15.120:0/606727364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d675480 0x56048d67d250 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd80c009cf0 tx=0x7fd80c004730 comp rx=0 tx=0).stop 2024-09-19T09:20:45.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.541+0000 7fd820a75740 1 -- 172.21.15.120:0/606727364 shutdown_connections 2024-09-19T09:20:45.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.541+0000 7fd820a75740 1 --2- 172.21.15.120:0/606727364 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56048d68fb30 0x56048d691fd0 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.541+0000 7fd7f77fe640 0 client.30275 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:45.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.541+0000 7fd820a75740 1 --2- 172.21.15.120:0/606727364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d67fd40 0x56048d67d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.541+0000 7fd820a75740 1 --2- 172.21.15.120:0/606727364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d675480 0x56048d67d250 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.541+0000 7fd820a75740 1 --2- 172.21.15.120:0/606727364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56048d674b60 0x56048d67cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.541+0000 7fd820a75740 1 -- 172.21.15.120:0/606727364 >> 172.21.15.120:0/606727364 conn(0x56048d6710f0 msgr2=0x56048d676e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:45.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.541+0000 7fd820a75740 1 -- 172.21.15.120:0/606727364 shutdown_connections 2024-09-19T09:20:45.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.542+0000 7fd820a75740 1 -- 172.21.15.120:0/606727364 wait complete. 2024-09-19T09:20:45.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.542+0000 7fd820a75740 20 client.30275 trim_cache size 0 max 16384 2024-09-19T09:20:45.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.562+0000 7fd820a75740 1 Processor -- start 2024-09-19T09:20:45.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.562+0000 7fd820a75740 1 -- start start 2024-09-19T09:20:45.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.562+0000 7fd820a75740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56048d674b60 0x56048d6978a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:45.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.562+0000 7fd820a75740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d675480 0x56048d697de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:45.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.562+0000 7fd820a75740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d67fd40 0x56048d67e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:45.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.562+0000 7fd820a75740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd8181ab310 con 0x56048d675480 2024-09-19T09:20:45.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.562+0000 7fd820a75740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd8181ace50 con 0x56048d67fd40 2024-09-19T09:20:45.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.562+0000 7fd820a75740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd8181ab9e0 con 0x56048d674b60 2024-09-19T09:20:45.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.562+0000 7fd8177fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d675480 0x56048d697de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:45.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.562+0000 7fd8177fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d675480 0x56048d697de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42974/0 (socket says 172.21.15.120:42974) 2024-09-19T09:20:45.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.562+0000 7fd8177fe640 1 -- 172.21.15.120:0/2409275416 learned_addr learned my addr 172.21.15.120:0/2409275416 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:45.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.562+0000 7fd817fff640 1 --2- 172.21.15.120:0/2409275416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56048d674b60 0x56048d6978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:45.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.562+0000 7fd8177fe640 1 -- 172.21.15.120:0/2409275416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56048d674b60 msgr2=0x56048d6978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:45.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.562+0000 7fd8177fe640 1 --2- 172.21.15.120:0/2409275416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56048d674b60 0x56048d6978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.562+0000 7fd81cc39640 1 --2- 172.21.15.120:0/2409275416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d67fd40 0x56048d67e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:45.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.562+0000 7fd8177fe640 1 -- 172.21.15.120:0/2409275416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d67fd40 msgr2=0x56048d67e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:45.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.562+0000 7fd8177fe640 1 --2- 172.21.15.120:0/2409275416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d67fd40 0x56048d67e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.562+0000 7fd8177fe640 1 -- 172.21.15.120:0/2409275416 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56048d6752b0 con 0x56048d675480 2024-09-19T09:20:45.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.562+0000 7fd81cc39640 1 --2- 172.21.15.120:0/2409275416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d67fd40 0x56048d67e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:20:45.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.563+0000 7fd8177fe640 1 --2- 172.21.15.120:0/2409275416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d675480 0x56048d697de0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fd80c013f10 tx=0x7fd80c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efabf28444d1afc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:45.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.563+0000 7fd7fdffb640 1 -- 172.21.15.120:0/2409275416 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c024070 con 0x56048d675480 2024-09-19T09:20:45.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.563+0000 7fd7fdffb640 1 -- 172.21.15.120:0/2409275416 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd80c004550 con 0x56048d675480 2024-09-19T09:20:45.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.563+0000 7fd7fdffb640 1 -- 172.21.15.120:0/2409275416 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c005020 con 0x56048d675480 2024-09-19T09:20:45.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.564+0000 7fd820a75740 1 -- 172.21.15.120:0/2409275416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d675480 msgr2=0x56048d697de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:45.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.564+0000 7fd820a75740 1 --2- 172.21.15.120:0/2409275416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d675480 0x56048d697de0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fd80c013f10 tx=0x7fd80c0099e0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.564+0000 7fd820a75740 1 -- 172.21.15.120:0/2409275416 shutdown_connections 2024-09-19T09:20:45.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.564+0000 7fd820a75740 1 --2- 172.21.15.120:0/2409275416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d67fd40 0x56048d67e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.564+0000 7fd820a75740 1 --2- 172.21.15.120:0/2409275416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d675480 0x56048d697de0 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.564+0000 7fd820a75740 1 --2- 172.21.15.120:0/2409275416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56048d674b60 0x56048d6978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.564+0000 7fd820a75740 1 -- 172.21.15.120:0/2409275416 >> 172.21.15.120:0/2409275416 conn(0x56048d6710f0 msgr2=0x56048d67be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:45.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.564+0000 7fd820a75740 1 -- 172.21.15.120:0/2409275416 shutdown_connections 2024-09-19T09:20:45.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.564+0000 7fd820a75740 1 -- 172.21.15.120:0/2409275416 wait complete. 2024-09-19T09:20:45.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.564+0000 7fd820a75740 1 -- start start 2024-09-19T09:20:45.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.564+0000 7fd820a75740 1 Processor -- start 2024-09-19T09:20:45.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.564+0000 7fd820a75740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56048d674b60 0x56048d683ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:45.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.564+0000 7fd820a75740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d675480 0x56048d684020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:45.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.565+0000 7fd820a75740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d67fd40 0x56048d684560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:45.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.565+0000 7fd817fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56048d674b60 0x56048d683ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:45.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.565+0000 7fd81cc39640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d67fd40 0x56048d684560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:45.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.565+0000 7fd817fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56048d674b60 0x56048d683ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36400/0 (socket says 172.21.15.120:36400) 2024-09-19T09:20:45.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.565+0000 7fd81cc39640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d67fd40 0x56048d684560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42982/0 (socket says 172.21.15.120:42982) 2024-09-19T09:20:45.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.565+0000 7fd817fff640 1 -- 172.21.15.120:0/3594920833 learned_addr learned my addr 172.21.15.120:0/3594920833 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:45.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.565+0000 7fd820a75740 1 -- 172.21.15.120:0/3594920833 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd8181ac780 con 0x56048d67fd40 2024-09-19T09:20:45.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.565+0000 7fd820a75740 1 -- 172.21.15.120:0/3594920833 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd8181ac0b0 con 0x56048d675480 2024-09-19T09:20:45.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.565+0000 7fd820a75740 1 -- 172.21.15.120:0/3594920833 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd8181a92d0 con 0x56048d674b60 2024-09-19T09:20:45.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.565+0000 7fd820a75740 10 client.0 _ll_register_callbacks cb 0x7fd80c00f5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:20:45.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.565+0000 7fd817fff640 1 -- 172.21.15.120:0/3594920833 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d675480 msgr2=0x56048d684020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:20:45.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.565+0000 7fd817fff640 1 --2- 172.21.15.120:0/3594920833 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d675480 0x56048d684020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.565+0000 7fd817fff640 1 -- 172.21.15.120:0/3594920833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d67fd40 msgr2=0x56048d684560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:45.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.565+0000 7fd817fff640 1 --2- 172.21.15.120:0/3594920833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d67fd40 0x56048d684560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.565+0000 7fd817fff640 1 -- 172.21.15.120:0/3594920833 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56048d6f9870 con 0x56048d674b60 2024-09-19T09:20:45.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.565+0000 7fd81cc39640 1 --2- 172.21.15.120:0/3594920833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d67fd40 0x56048d684560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:20:45.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.565+0000 7fd820a75740 1 -- 172.21.15.120:0/3594920833 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd80c004550 con 0x56048d674b60 2024-09-19T09:20:45.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.565+0000 7fd817fff640 1 --2- 172.21.15.120:0/3594920833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56048d674b60 0x56048d683ae0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd808009850 tx=0x7fd808004bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd01e4173a47166c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:45.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.565+0000 7fd7f77fe640 10 client.30094 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:20:45.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.566+0000 7fd7f77fe640 1 -- 172.21.15.120:0/3594920833 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd808058060 con 0x56048d674b60 2024-09-19T09:20:45.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.566+0000 7fd7f77fe640 1 -- 172.21.15.120:0/3594920833 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd8080315b0 con 0x56048d674b60 2024-09-19T09:20:45.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.566+0000 7fd7f77fe640 1 -- 172.21.15.120:0/3594920833 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd808040040 con 0x56048d674b60 2024-09-19T09:20:45.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.566+0000 7fd820a75740 10 client.30094 fetch_fsmap learned FSMap version 39 2024-09-19T09:20:45.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.566+0000 7fd820a75740 1 -- 172.21.15.120:0/3594920833 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({fsmap=39}) -- 0x56048d6faa00 con 0x56048d674b60 2024-09-19T09:20:45.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.566+0000 7fd7f77fe640 1 -- 172.21.15.120:0/3594920833 <== mon.1 v2:172.21.15.196:3300/0 4 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fd8080317b0 con 0x56048d674b60 2024-09-19T09:20:45.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.566+0000 7fd820a75740 10 client.30094 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:20:45.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.566+0000 7fd820a75740 10 client.30094 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:20:45.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.566+0000 7fd820a75740 20 client.30094 populate_metadata read hostname 'smithi120' 2024-09-19T09:20:45.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.566+0000 7fd820a75740 1 --2- 172.21.15.120:0/3594920833 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56048d6facd0 0x56048d6fd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:45.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.566+0000 7fd820a75740 4 client.30094 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:20:45.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.566+0000 7fd820a75740 1 -- 172.21.15.120:0/3594920833 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56048d5e1d00 con 0x56048d6facd0 2024-09-19T09:20:45.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.581+0000 7fd8177fe640 1 --2- 172.21.15.120:0/3594920833 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56048d6facd0 0x56048d6fd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:45.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.582+0000 7fd7f77fe640 10 client.30094 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:45.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.582+0000 7fd8177fe640 1 --2- 172.21.15.120:0/3594920833 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56048d6facd0 0x56048d6fd0e0 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4574305a88559780 server_cookie=c1a2f16e5aa86ae8 in_seq=0 out_seq=0 2024-09-19T09:20:45.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.609+0000 7fd7f77fe640 1 -- 172.21.15.120:0/3594920833 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56048d5e1d00 con 0x56048d6facd0 2024-09-19T09:20:45.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.609+0000 7fd7f77fe640 10 client.30094 handle_command_reply: tid=0 2024-09-19T09:20:45.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.609+0000 7fd820a75740 1 client.30094 shutdown 2024-09-19T09:20:45.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.609+0000 7fd820a75740 1 -- 172.21.15.120:0/3594920833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56048d674b60 msgr2=0x56048d683ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:45.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.609+0000 7fd820a75740 1 --2- 172.21.15.120:0/3594920833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56048d674b60 0x56048d683ae0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd808009850 tx=0x7fd808004bc0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.609+0000 7fd820a75740 1 -- 172.21.15.120:0/3594920833 shutdown_connections 2024-09-19T09:20:45.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.609+0000 7fd820a75740 1 --2- 172.21.15.120:0/3594920833 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56048d6facd0 0x56048d6fd0e0 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.609+0000 7fd820a75740 1 --2- 172.21.15.120:0/3594920833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d67fd40 0x56048d684560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.609+0000 7fd820a75740 1 --2- 172.21.15.120:0/3594920833 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d675480 0x56048d684020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.609+0000 7fd7f77fe640 0 client.30094 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:45.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.609+0000 7fd820a75740 1 --2- 172.21.15.120:0/3594920833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56048d674b60 0x56048d683ae0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.609+0000 7fd820a75740 1 -- 172.21.15.120:0/3594920833 >> 172.21.15.120:0/3594920833 conn(0x56048d6710f0 msgr2=0x56048d6765b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:45.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.609+0000 7fd820a75740 1 -- 172.21.15.120:0/3594920833 shutdown_connections 2024-09-19T09:20:45.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.609+0000 7fd820a75740 1 -- 172.21.15.120:0/3594920833 wait complete. 2024-09-19T09:20:45.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.609+0000 7fd820a75740 20 client.30094 trim_cache size 0 max 16384 2024-09-19T09:20:45.612 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:20:45.612 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:20:45.612 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "5d666f85-c4ef-4ff9-bd5f-bab7f7c7b5cd", 2024-09-19T09:20:45.612 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:20:45.612 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:20:45.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.610+0000 7fd81e69c640 1 -- 172.21.15.120:0/2961310129 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd8000a32c0 msgr2=0x7fd8000a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:45.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.610+0000 7fd81e69c640 1 --2- 172.21.15.120:0/2961310129 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd8000a32c0 0x7fd8000a5780 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd80c013570 tx=0x7fd80c013970 comp rx=0 tx=0).stop 2024-09-19T09:20:45.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.610+0000 7fd81e69c640 1 -- 172.21.15.120:0/2961310129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd818102ef0 msgr2=0x7fd81819bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:45.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.610+0000 7fd81e69c640 1 --2- 172.21.15.120:0/2961310129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd818102ef0 0x7fd81819bd00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd810009fd0 tx=0x7fd81000ef10 comp rx=0 tx=0).stop 2024-09-19T09:20:45.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.610+0000 7fd81e69c640 1 -- 172.21.15.120:0/2961310129 shutdown_connections 2024-09-19T09:20:45.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.610+0000 7fd81e69c640 1 --2- 172.21.15.120:0/2961310129 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd8000a32c0 0x7fd8000a5780 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.610+0000 7fd81e69c640 1 --2- 172.21.15.120:0/2961310129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd818102ef0 0x7fd81819bd00 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.610+0000 7fd81e69c640 1 --2- 172.21.15.120:0/2961310129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd818102570 0x7fd818197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.610+0000 7fd81e69c640 1 --2- 172.21.15.120:0/2961310129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd818100e20 0x7fd818197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:45.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.610+0000 7fd81e69c640 1 -- 172.21.15.120:0/2961310129 >> 172.21.15.120:0/2961310129 conn(0x7fd8180fc7d0 msgr2=0x7fd818104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:45.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.610+0000 7fd81e69c640 1 -- 172.21.15.120:0/2961310129 shutdown_connections 2024-09-19T09:20:45.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:45.610+0000 7fd81e69c640 1 -- 172.21.15.120:0/2961310129 wait complete. 2024-09-19T09:20:46.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:46 smithi120 ceph-mon[31150]: pgmap v882: 97 pgs: 97 active+clean; 378 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.8 MiB/s wr, 703 op/s 2024-09-19T09:20:46.367 INFO:tasks.workunit.client.0.smithi120.stdout: 10 120414 21.61 MB/sec execute 161 sec latency 115.243 ms 2024-09-19T09:20:46.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:46 smithi196 ceph-mon[35150]: pgmap v882: 97 pgs: 97 active+clean; 378 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.8 MiB/s wr, 703 op/s 2024-09-19T09:20:46.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:46 smithi153 ceph-mon[35071]: pgmap v882: 97 pgs: 97 active+clean; 378 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.8 MiB/s wr, 703 op/s 2024-09-19T09:20:46.529 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:20:46.805 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:20:47.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:47 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:20:47.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:47 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:20:47.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:47 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:20:47.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:47 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:20:47.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:47 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:20:47.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:47 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:20:47.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:47 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:20:47.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:47 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:20:47.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:47 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:20:47.367 INFO:tasks.workunit.client.0.smithi120.stdout: 10 120947 21.65 MB/sec execute 162 sec latency 85.390 ms 2024-09-19T09:20:47.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:47 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:20:47.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:47 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:20:47.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:47 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:20:47.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:47 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:20:47.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:47 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:20:47.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:47 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:20:47.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:47 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:20:47.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:47 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:20:47.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:47 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:20:47.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:47 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:20:47.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:47 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:20:47.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:47 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:20:47.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:47 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:20:47.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:47 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:20:47.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:47 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:20:47.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:47 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:20:47.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:47 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:20:47.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:47 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:20:47.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.466+0000 7ffaa1b35640 1 -- 172.21.15.120:0/3628112288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa9c1008d0 msgr2=0x7ffa9c100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:47.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.466+0000 7ffaa1b35640 1 --2- 172.21.15.120:0/3628112288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa9c1008d0 0x7ffa9c100d30 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7ffa88009930 tx=0x7ffa8802a420 comp rx=0 tx=0).stop 2024-09-19T09:20:47.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.466+0000 7ffaa1b35640 1 -- 172.21.15.120:0/3628112288 shutdown_connections 2024-09-19T09:20:47.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.466+0000 7ffaa1b35640 1 --2- 172.21.15.120:0/3628112288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa9c101270 0x7ffa9c10d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.466+0000 7ffaa1b35640 1 --2- 172.21.15.120:0/3628112288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa9c1008d0 0x7ffa9c100d30 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.466+0000 7ffaa1b35640 1 --2- 172.21.15.120:0/3628112288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa9c1068d0 0x7ffa9c106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.466+0000 7ffaa1b35640 1 -- 172.21.15.120:0/3628112288 >> 172.21.15.120:0/3628112288 conn(0x7ffa9c0fc610 msgr2=0x7ffa9c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:47.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.466+0000 7ffaa1b35640 1 -- 172.21.15.120:0/3628112288 shutdown_connections 2024-09-19T09:20:47.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.466+0000 7ffaa1b35640 1 -- 172.21.15.120:0/3628112288 wait complete. 2024-09-19T09:20:47.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.467+0000 7ffaa1b35640 1 Processor -- start 2024-09-19T09:20:47.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.467+0000 7ffaa1b35640 1 -- start start 2024-09-19T09:20:47.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.467+0000 7ffaa1b35640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa9c1008d0 0x7ffa9c1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:47.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.467+0000 7ffaa1b35640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa9c101270 0x7ffa9c1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:47.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.467+0000 7ffaa1b35640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa9c1068d0 0x7ffa9c19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:47.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.467+0000 7ffaa1b35640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c110d20 con 0x7ffa9c1008d0 2024-09-19T09:20:47.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.467+0000 7ffaa1b35640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c110ba0 con 0x7ffa9c101270 2024-09-19T09:20:47.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.467+0000 7ffaa1b35640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c110ea0 con 0x7ffa9c1068d0 2024-09-19T09:20:47.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.467+0000 7ffaa0b33640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa9c1008d0 0x7ffa9c1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:47.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.467+0000 7ffaa0b33640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa9c1008d0 0x7ffa9c1982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43002/0 (socket says 172.21.15.120:43002) 2024-09-19T09:20:47.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.467+0000 7ffaa0b33640 1 -- 172.21.15.120:0/1766924948 learned_addr learned my addr 172.21.15.120:0/1766924948 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:47.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.467+0000 7ffaa1334640 1 --2- 172.21.15.120:0/1766924948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa9c1068d0 0x7ffa9c19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:47.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.467+0000 7ffa9bfff640 1 --2- 172.21.15.120:0/1766924948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa9c101270 0x7ffa9c1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:47.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.467+0000 7ffaa0b33640 1 -- 172.21.15.120:0/1766924948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa9c1068d0 msgr2=0x7ffa9c19cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:47.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.467+0000 7ffaa0b33640 1 --2- 172.21.15.120:0/1766924948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa9c1068d0 0x7ffa9c19cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.467+0000 7ffaa0b33640 1 -- 172.21.15.120:0/1766924948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa9c101270 msgr2=0x7ffa9c1987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:47.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.467+0000 7ffaa0b33640 1 --2- 172.21.15.120:0/1766924948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa9c101270 0x7ffa9c1987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.467+0000 7ffaa0b33640 1 -- 172.21.15.120:0/1766924948 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa9c19d260 con 0x7ffa9c1008d0 2024-09-19T09:20:47.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.467+0000 7ffa9bfff640 1 --2- 172.21.15.120:0/1766924948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa9c101270 0x7ffa9c1987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:20:47.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.468+0000 7ffaa0b33640 1 --2- 172.21.15.120:0/1766924948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa9c1008d0 0x7ffa9c1982b0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7ffa90009e00 tx=0x7ffa9000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:47.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.468+0000 7ffa99ffb640 1 -- 172.21.15.120:0/1766924948 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa90005220 con 0x7ffa9c1008d0 2024-09-19T09:20:47.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.468+0000 7ffa99ffb640 1 -- 172.21.15.120:0/1766924948 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa90012530 con 0x7ffa9c1008d0 2024-09-19T09:20:47.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.468+0000 7ffa99ffb640 1 -- 172.21.15.120:0/1766924948 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa90004db0 con 0x7ffa9c1008d0 2024-09-19T09:20:47.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.468+0000 7ffaa1b35640 1 -- 172.21.15.120:0/1766924948 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa9c19d550 con 0x7ffa9c1008d0 2024-09-19T09:20:47.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.468+0000 7ffaa1b35640 1 -- 172.21.15.120:0/1766924948 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa9c0fcb70 con 0x7ffa9c1008d0 2024-09-19T09:20:47.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.471+0000 7ffa99ffb640 1 -- 172.21.15.120:0/1766924948 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7ffa90012050 con 0x7ffa9c1008d0 2024-09-19T09:20:47.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.471+0000 7ffaa5170740 1 Processor -- start 2024-09-19T09:20:47.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.471+0000 7ffaa5170740 1 -- start start 2024-09-19T09:20:47.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.472+0000 7ffaa5170740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615d5cf0590 0x5615d5cf0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:47.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.472+0000 7ffaa5170740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615d5cf0eb0 0x5615d5d03b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:47.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.472+0000 7ffaa5170740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615d5d04060 0x5615d5d06450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:47.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.472+0000 7ffaa5170740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5615d5c562e0 con 0x5615d5d04060 2024-09-19T09:20:47.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.472+0000 7ffaa5170740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5615d5bfdce0 con 0x5615d5cf0eb0 2024-09-19T09:20:47.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.472+0000 7ffaa5170740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5615d5c522a0 con 0x5615d5cf0590 2024-09-19T09:20:47.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.472+0000 7ffaa1334640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615d5cf0590 0x5615d5cf0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:47.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.472+0000 7ffa9bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615d5d04060 0x5615d5d06450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:47.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.472+0000 7ffaa1334640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615d5cf0590 0x5615d5cf0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36426/0 (socket says 172.21.15.120:36426) 2024-09-19T09:20:47.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.472+0000 7ffaa1334640 1 -- 172.21.15.120:0/3756930131 learned_addr learned my addr 172.21.15.120:0/3756930131 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:47.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.472+0000 7ffaa0b33640 1 --2- 172.21.15.120:0/3756930131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615d5cf0eb0 0x5615d5d03b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:47.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.472+0000 7ffa9bfff640 1 -- 172.21.15.120:0/3756930131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615d5cf0590 msgr2=0x5615d5cf0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:47.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.472+0000 7ffa9bfff640 1 --2- 172.21.15.120:0/3756930131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615d5cf0590 0x5615d5cf0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.472+0000 7ffa9bfff640 1 -- 172.21.15.120:0/3756930131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615d5cf0eb0 msgr2=0x5615d5d03b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:47.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.472+0000 7ffa9bfff640 1 --2- 172.21.15.120:0/3756930131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615d5cf0eb0 0x5615d5d03b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.472+0000 7ffa9bfff640 1 -- 172.21.15.120:0/3756930131 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615d5be2440 con 0x5615d5d04060 2024-09-19T09:20:47.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.473+0000 7ffa9bfff640 1 --2- 172.21.15.120:0/3756930131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615d5d04060 0x5615d5d06450 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7ffa8802a940 tx=0x7ffa88004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd98558a61d1fcea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:47.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.473+0000 7ffa86ffd640 1 -- 172.21.15.120:0/3756930131 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa88035030 con 0x5615d5d04060 2024-09-19T09:20:47.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.473+0000 7ffa99ffb640 1 --2- 172.21.15.120:0/1766924948 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ffa8c0a3240 0x7ffa8c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:47.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.473+0000 7ffa86ffd640 1 -- 172.21.15.120:0/3756930131 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa88005520 con 0x5615d5d04060 2024-09-19T09:20:47.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.473+0000 7ffa86ffd640 1 -- 172.21.15.120:0/3756930131 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa880035f0 con 0x5615d5d04060 2024-09-19T09:20:47.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.473+0000 7ffa99ffb640 1 -- 172.21.15.120:0/1766924948 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ffa900d3910 con 0x7ffa9c1008d0 2024-09-19T09:20:47.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.473+0000 7ffaa1334640 1 --2- 172.21.15.120:0/1766924948 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ffa8c0a3240 0x7ffa8c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:47.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.474+0000 7ffaa1334640 1 --2- 172.21.15.120:0/1766924948 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ffa8c0a3240 0x7ffa8c0a5700 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ffa940116b0 tx=0x7ffa94009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:47.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.474+0000 7ffaa5170740 1 -- 172.21.15.120:0/3756930131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615d5d04060 msgr2=0x5615d5d06450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:47.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.474+0000 7ffaa5170740 1 --2- 172.21.15.120:0/3756930131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615d5d04060 0x5615d5d06450 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7ffa8802a940 tx=0x7ffa88004040 comp rx=0 tx=0).stop 2024-09-19T09:20:47.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.474+0000 7ffaa5170740 1 -- 172.21.15.120:0/3756930131 shutdown_connections 2024-09-19T09:20:47.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.474+0000 7ffaa5170740 1 --2- 172.21.15.120:0/3756930131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615d5d04060 0x5615d5d06450 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.474+0000 7ffaa5170740 1 --2- 172.21.15.120:0/3756930131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615d5cf0eb0 0x5615d5d03b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.474+0000 7ffaa5170740 1 --2- 172.21.15.120:0/3756930131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615d5cf0590 0x5615d5cf0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.474+0000 7ffaa5170740 1 -- 172.21.15.120:0/3756930131 >> 172.21.15.120:0/3756930131 conn(0x5615d5cecc00 msgr2=0x5615d5cecfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:47.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.474+0000 7ffaa5170740 1 -- 172.21.15.120:0/3756930131 shutdown_connections 2024-09-19T09:20:47.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.474+0000 7ffaa5170740 1 -- 172.21.15.120:0/3756930131 wait complete. 2024-09-19T09:20:47.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.475+0000 7ffaa5170740 1 -- start start 2024-09-19T09:20:47.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.475+0000 7ffaa5170740 1 Processor -- start 2024-09-19T09:20:47.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.476+0000 7ffaa5170740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615d5cf0590 0x5615d5d03a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:47.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.476+0000 7ffaa5170740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615d5cf0eb0 0x5615d5d00d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:47.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.476+0000 7ffaa5170740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615d5d04060 0x5615d5d01340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:47.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.476+0000 7ffaa5170740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5615d5d06b70 con 0x5615d5d04060 2024-09-19T09:20:47.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.476+0000 7ffaa5170740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5615d5c11050 con 0x5615d5cf0eb0 2024-09-19T09:20:47.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.476+0000 7ffaa5170740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5615d5d06cf0 con 0x5615d5cf0590 2024-09-19T09:20:47.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.476+0000 7ffaa5170740 10 client.0 _ll_register_callbacks cb 0x5615d5c51160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:20:47.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.476+0000 7ffaa0b33640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615d5cf0590 0x5615d5d03a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:47.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.476+0000 7ffaa0b33640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615d5cf0590 0x5615d5d03a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36442/0 (socket says 172.21.15.120:36442) 2024-09-19T09:20:47.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.476+0000 7ffaa0b33640 1 -- 172.21.15.120:0/1689693491 learned_addr learned my addr 172.21.15.120:0/1689693491 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:47.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.476+0000 7ffa9bfff640 1 --2- 172.21.15.120:0/1689693491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615d5cf0eb0 0x5615d5d00d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:47.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.476+0000 7ffaa1334640 1 --2- 172.21.15.120:0/1689693491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615d5d04060 0x5615d5d01340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:47.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.476+0000 7ffa9bfff640 1 -- 172.21.15.120:0/1689693491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615d5cf0590 msgr2=0x5615d5d03a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:47.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.476+0000 7ffa9bfff640 1 --2- 172.21.15.120:0/1689693491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615d5cf0590 0x5615d5d03a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.476+0000 7ffa9bfff640 1 -- 172.21.15.120:0/1689693491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615d5d04060 msgr2=0x5615d5d01340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:47.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.476+0000 7ffa9bfff640 1 --2- 172.21.15.120:0/1689693491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615d5d04060 0x5615d5d01340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.477+0000 7ffa9bfff640 1 -- 172.21.15.120:0/1689693491 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615d59c73c0 con 0x5615d5cf0eb0 2024-09-19T09:20:47.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.477+0000 7ffa9bfff640 1 --2- 172.21.15.120:0/1689693491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615d5cf0eb0 0x5615d5d00d30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ffa88009930 tx=0x7ffa880048d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2216abf3748bb381 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:47.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.477+0000 7ffa7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:20:47.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.478+0000 7ffa7bfff640 1 -- 172.21.15.120:0/1689693491 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa88052060 con 0x5615d5cf0eb0 2024-09-19T09:20:47.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.478+0000 7ffa7bfff640 1 -- 172.21.15.120:0/1689693491 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ffa88004f40 con 0x5615d5cf0eb0 2024-09-19T09:20:47.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.478+0000 7ffaa5170740 1 -- 172.21.15.120:0/1689693491 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa88005520 con 0x5615d5cf0eb0 2024-09-19T09:20:47.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.478+0000 7ffa7bfff640 1 -- 172.21.15.120:0/1689693491 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ffa88035050 con 0x5615d5cf0eb0 2024-09-19T09:20:47.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.478+0000 7ffaa5170740 10 client.30287 fetch_fsmap learned FSMap version 39 2024-09-19T09:20:47.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.478+0000 7ffaa5170740 1 -- 172.21.15.120:0/1689693491 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5615d5d12640 con 0x5615d5cf0eb0 2024-09-19T09:20:47.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.478+0000 7ffa7bfff640 1 -- 172.21.15.120:0/1689693491 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa88005140 con 0x5615d5cf0eb0 2024-09-19T09:20:47.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.478+0000 7ffa7bfff640 1 -- 172.21.15.120:0/1689693491 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7ffa880036e0 con 0x5615d5cf0eb0 2024-09-19T09:20:47.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.479+0000 7ffaa5170740 10 client.30287 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:20:47.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.479+0000 7ffaa5170740 10 client.30287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:20:47.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.479+0000 7ffaa5170740 20 client.30287 populate_metadata read hostname 'smithi120' 2024-09-19T09:20:47.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.479+0000 7ffaa5170740 1 --2- 172.21.15.120:0/1689693491 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5615d5d13b90 0x5615d5d15f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:47.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.479+0000 7ffaa5170740 4 client.30287 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:20:47.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.479+0000 7ffaa5170740 1 -- 172.21.15.120:0/1689693491 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ffa88005520 con 0x5615d5d13b90 2024-09-19T09:20:47.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.479+0000 7ffaa0b33640 1 --2- 172.21.15.120:0/1689693491 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5615d5d13b90 0x5615d5d15f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:47.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.480+0000 7ffaa0b33640 1 --2- 172.21.15.120:0/1689693491 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5615d5d13b90 0x5615d5d15f80 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35c8ccbb4ccfdb2b server_cookie=68dcffff66e3722a in_seq=0 out_seq=0 2024-09-19T09:20:47.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.480+0000 7ffa7bfff640 10 client.30287 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:47.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.483+0000 7ffa7bfff640 1 -- 172.21.15.120:0/1689693491 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ffa88005520 con 0x5615d5d13b90 2024-09-19T09:20:47.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.483+0000 7ffa7bfff640 10 client.30287 handle_command_reply: tid=0 2024-09-19T09:20:47.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.483+0000 7ffaa5170740 1 client.30287 shutdown 2024-09-19T09:20:47.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.484+0000 7ffaa5170740 1 -- 172.21.15.120:0/1689693491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615d5cf0eb0 msgr2=0x5615d5d00d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:47.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.484+0000 7ffaa5170740 1 --2- 172.21.15.120:0/1689693491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615d5cf0eb0 0x5615d5d00d30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ffa88009930 tx=0x7ffa880048d0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.484+0000 7ffaa5170740 1 -- 172.21.15.120:0/1689693491 shutdown_connections 2024-09-19T09:20:47.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.484+0000 7ffaa5170740 1 --2- 172.21.15.120:0/1689693491 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5615d5d13b90 0x5615d5d15f80 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.484+0000 7ffaa5170740 1 --2- 172.21.15.120:0/1689693491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615d5d04060 0x5615d5d01340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.484+0000 7ffa7bfff640 0 client.30287 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:47.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.484+0000 7ffaa5170740 1 --2- 172.21.15.120:0/1689693491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615d5cf0eb0 0x5615d5d00d30 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.484+0000 7ffaa5170740 1 --2- 172.21.15.120:0/1689693491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615d5cf0590 0x5615d5d03a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.484+0000 7ffaa5170740 1 -- 172.21.15.120:0/1689693491 >> 172.21.15.120:0/1689693491 conn(0x5615d5cecc00 msgr2=0x5615d5cf2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:47.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.484+0000 7ffaa5170740 1 -- 172.21.15.120:0/1689693491 shutdown_connections 2024-09-19T09:20:47.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.484+0000 7ffaa5170740 1 -- 172.21.15.120:0/1689693491 wait complete. 2024-09-19T09:20:47.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.484+0000 7ffaa5170740 20 client.30287 trim_cache size 0 max 16384 2024-09-19T09:20:47.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.510+0000 7ffaa5170740 1 Processor -- start 2024-09-19T09:20:47.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.510+0000 7ffaa5170740 1 -- start start 2024-09-19T09:20:47.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.510+0000 7ffaa5170740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615d5cf0590 0x5615d5d0fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:47.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.510+0000 7ffaa5170740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615d5cf0eb0 0x5615d5cfc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:47.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.510+0000 7ffaa5170740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615d5d04060 0x5615d5d1b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:47.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.510+0000 7ffaa5170740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c1ab320 con 0x5615d5cf0eb0 2024-09-19T09:20:47.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.510+0000 7ffaa5170740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c1ace60 con 0x5615d5d04060 2024-09-19T09:20:47.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.510+0000 7ffaa5170740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c1ab9f0 con 0x5615d5cf0590 2024-09-19T09:20:47.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.510+0000 7ffa9bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615d5cf0eb0 0x5615d5cfc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:47.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.510+0000 7ffa9bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615d5cf0eb0 0x5615d5cfc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43030/0 (socket says 172.21.15.120:43030) 2024-09-19T09:20:47.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.510+0000 7ffa9bfff640 1 -- 172.21.15.120:0/2505458076 learned_addr learned my addr 172.21.15.120:0/2505458076 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:47.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.510+0000 7ffaa1334640 1 --2- 172.21.15.120:0/2505458076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615d5d04060 0x5615d5d1b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:47.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.511+0000 7ffaa0b33640 1 --2- 172.21.15.120:0/2505458076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615d5cf0590 0x5615d5d0fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:47.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.511+0000 7ffa9bfff640 1 -- 172.21.15.120:0/2505458076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615d5cf0590 msgr2=0x5615d5d0fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:47.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.511+0000 7ffa9bfff640 1 --2- 172.21.15.120:0/2505458076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615d5cf0590 0x5615d5d0fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.511+0000 7ffa9bfff640 1 -- 172.21.15.120:0/2505458076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615d5d04060 msgr2=0x5615d5d1b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:47.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.511+0000 7ffa9bfff640 1 --2- 172.21.15.120:0/2505458076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615d5d04060 0x5615d5d1b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.511+0000 7ffa9bfff640 1 -- 172.21.15.120:0/2505458076 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615d5d016b0 con 0x5615d5cf0eb0 2024-09-19T09:20:47.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.511+0000 7ffaa0b33640 1 --2- 172.21.15.120:0/2505458076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615d5cf0590 0x5615d5d0fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:20:47.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.511+0000 7ffa9bfff640 1 --2- 172.21.15.120:0/2505458076 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615d5cf0eb0 0x5615d5cfc1d0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7ffa8802aa10 tx=0x7ffa88005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef26ad13ebfecb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:47.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.511+0000 7ffa867fc640 1 -- 172.21.15.120:0/2505458076 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8805c070 con 0x5615d5cf0eb0 2024-09-19T09:20:47.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.511+0000 7ffa867fc640 1 -- 172.21.15.120:0/2505458076 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa88005420 con 0x5615d5cf0eb0 2024-09-19T09:20:47.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.511+0000 7ffa867fc640 1 -- 172.21.15.120:0/2505458076 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa88052040 con 0x5615d5cf0eb0 2024-09-19T09:20:47.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.513+0000 7ffaa5170740 1 -- 172.21.15.120:0/2505458076 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615d5cf0eb0 msgr2=0x5615d5cfc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:47.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.513+0000 7ffaa5170740 1 --2- 172.21.15.120:0/2505458076 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615d5cf0eb0 0x5615d5cfc1d0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7ffa8802aa10 tx=0x7ffa88005990 comp rx=0 tx=0).stop 2024-09-19T09:20:47.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.513+0000 7ffaa5170740 1 -- 172.21.15.120:0/2505458076 shutdown_connections 2024-09-19T09:20:47.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.513+0000 7ffaa5170740 1 --2- 172.21.15.120:0/2505458076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615d5d04060 0x5615d5d1b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.513+0000 7ffaa5170740 1 --2- 172.21.15.120:0/2505458076 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615d5cf0eb0 0x5615d5cfc1d0 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.513+0000 7ffaa5170740 1 --2- 172.21.15.120:0/2505458076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615d5cf0590 0x5615d5d0fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.513+0000 7ffaa5170740 1 -- 172.21.15.120:0/2505458076 >> 172.21.15.120:0/2505458076 conn(0x5615d5cecc00 msgr2=0x5615d5cfbac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:47.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.513+0000 7ffaa5170740 1 -- 172.21.15.120:0/2505458076 shutdown_connections 2024-09-19T09:20:47.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.513+0000 7ffaa5170740 1 -- 172.21.15.120:0/2505458076 wait complete. 2024-09-19T09:20:47.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.513+0000 7ffaa5170740 1 -- start start 2024-09-19T09:20:47.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.514+0000 7ffaa5170740 1 Processor -- start 2024-09-19T09:20:47.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.514+0000 7ffaa5170740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615d5cf0590 0x5615d5d0bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:47.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.514+0000 7ffaa5170740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615d5cf0eb0 0x5615d5d0c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:47.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.514+0000 7ffaa5170740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615d5d04060 0x5615d5d08f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:47.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.514+0000 7ffaa5170740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c1ac790 con 0x5615d5cf0eb0 2024-09-19T09:20:47.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.514+0000 7ffaa5170740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c1ac0c0 con 0x5615d5d04060 2024-09-19T09:20:47.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.514+0000 7ffaa0b33640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615d5cf0590 0x5615d5d0bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:47.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.514+0000 7ffaa0b33640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615d5cf0590 0x5615d5d0bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36456/0 (socket says 172.21.15.120:36456) 2024-09-19T09:20:47.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.514+0000 7ffaa0b33640 1 -- 172.21.15.120:0/9077428 learned_addr learned my addr 172.21.15.120:0/9077428 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:47.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.515+0000 7ffaa1334640 1 --2- 172.21.15.120:0/9077428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615d5d04060 0x5615d5d08f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:47.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.515+0000 7ffaa5170740 1 -- 172.21.15.120:0/9077428 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c1a92e0 con 0x5615d5cf0590 2024-09-19T09:20:47.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.515+0000 7ffaa5170740 10 client.0 _ll_register_callbacks cb 0x7ffa900113e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:20:47.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.515+0000 7ffa9bfff640 1 --2- 172.21.15.120:0/9077428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615d5cf0eb0 0x5615d5d0c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:47.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.515+0000 7ffaa0b33640 1 -- 172.21.15.120:0/9077428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615d5d04060 msgr2=0x5615d5d08f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:47.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.515+0000 7ffaa0b33640 1 --2- 172.21.15.120:0/9077428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615d5d04060 0x5615d5d08f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.515+0000 7ffaa0b33640 1 -- 172.21.15.120:0/9077428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615d5cf0eb0 msgr2=0x5615d5d0c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:47.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.515+0000 7ffaa0b33640 1 --2- 172.21.15.120:0/9077428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615d5cf0eb0 0x5615d5d0c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.515+0000 7ffaa0b33640 1 -- 172.21.15.120:0/9077428 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615d5cfec40 con 0x5615d5cf0590 2024-09-19T09:20:47.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.515+0000 7ffa9bfff640 1 --2- 172.21.15.120:0/9077428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615d5cf0eb0 0x5615d5d0c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:20:47.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.515+0000 7ffaa0b33640 1 --2- 172.21.15.120:0/9077428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615d5cf0590 0x5615d5d0bf80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ffa9009c390 tx=0x7ffa90011510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c88f6a80dcf7e8db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:47.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.515+0000 7ffa7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:20:47.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.516+0000 7ffa7bfff640 1 -- 172.21.15.120:0/9077428 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9009f890 con 0x5615d5cf0590 2024-09-19T09:20:47.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.516+0000 7ffaa5170740 1 -- 172.21.15.120:0/9077428 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa88005420 con 0x5615d5cf0590 2024-09-19T09:20:47.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.516+0000 7ffa7bfff640 1 -- 172.21.15.120:0/9077428 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ffa900054c0 con 0x5615d5cf0590 2024-09-19T09:20:47.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.516+0000 7ffa7bfff640 1 -- 172.21.15.120:0/9077428 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ffa900bc070 con 0x5615d5cf0590 2024-09-19T09:20:47.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.516+0000 7ffaa5170740 10 client.30109 fetch_fsmap learned FSMap version 39 2024-09-19T09:20:47.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.516+0000 7ffaa5170740 1 -- 172.21.15.120:0/9077428 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5615d5d7ed40 con 0x5615d5cf0590 2024-09-19T09:20:47.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.516+0000 7ffa7bfff640 1 -- 172.21.15.120:0/9077428 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa88005420 con 0x5615d5cf0590 2024-09-19T09:20:47.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.516+0000 7ffa7bfff640 1 -- 172.21.15.120:0/9077428 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7ffa9009c5a0 con 0x5615d5cf0590 2024-09-19T09:20:47.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.516+0000 7ffaa5170740 10 client.30109 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:20:47.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.516+0000 7ffaa5170740 10 client.30109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:20:47.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.516+0000 7ffaa5170740 20 client.30109 populate_metadata read hostname 'smithi120' 2024-09-19T09:20:47.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.516+0000 7ffaa5170740 1 --2- 172.21.15.120:0/9077428 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5615d5d13b90 0x5615d5d81520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:47.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.517+0000 7ffaa5170740 4 client.30109 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:20:47.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.517+0000 7ffaa5170740 1 -- 172.21.15.120:0/9077428 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5615d5c63850 con 0x5615d5d13b90 2024-09-19T09:20:47.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.517+0000 7ffa9bfff640 1 --2- 172.21.15.120:0/9077428 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5615d5d13b90 0x5615d5d81520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:47.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.517+0000 7ffa9bfff640 1 --2- 172.21.15.120:0/9077428 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5615d5d13b90 0x5615d5d81520 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b99ab38dd0b7081 server_cookie=e172075104b849cd in_seq=0 out_seq=0 2024-09-19T09:20:47.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.517+0000 7ffa7bfff640 10 client.30109 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:47.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.518+0000 7ffa7bfff640 1 -- 172.21.15.120:0/9077428 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5615d5c63850 con 0x5615d5d13b90 2024-09-19T09:20:47.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.519+0000 7ffa7bfff640 10 client.30109 handle_command_reply: tid=0 2024-09-19T09:20:47.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.519+0000 7ffaa5170740 1 client.30109 shutdown 2024-09-19T09:20:47.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.519+0000 7ffaa5170740 1 -- 172.21.15.120:0/9077428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615d5cf0590 msgr2=0x5615d5d0bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:47.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.519+0000 7ffaa5170740 1 --2- 172.21.15.120:0/9077428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615d5cf0590 0x5615d5d0bf80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ffa9009c390 tx=0x7ffa90011510 comp rx=0 tx=0).stop 2024-09-19T09:20:47.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.519+0000 7ffaa5170740 1 -- 172.21.15.120:0/9077428 shutdown_connections 2024-09-19T09:20:47.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.519+0000 7ffaa5170740 1 --2- 172.21.15.120:0/9077428 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5615d5d13b90 0x5615d5d81520 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.519+0000 7ffa7bfff640 0 client.30109 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:47.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.519+0000 7ffaa5170740 1 --2- 172.21.15.120:0/9077428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615d5d04060 0x5615d5d08f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.519+0000 7ffaa5170740 1 --2- 172.21.15.120:0/9077428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615d5cf0eb0 0x5615d5d0c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.519+0000 7ffaa5170740 1 --2- 172.21.15.120:0/9077428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615d5cf0590 0x5615d5d0bf80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.519+0000 7ffaa5170740 1 -- 172.21.15.120:0/9077428 >> 172.21.15.120:0/9077428 conn(0x5615d5cecc00 msgr2=0x5615d5cfc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:47.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.519+0000 7ffaa5170740 1 -- 172.21.15.120:0/9077428 shutdown_connections 2024-09-19T09:20:47.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.519+0000 7ffaa5170740 1 -- 172.21.15.120:0/9077428 wait complete. 2024-09-19T09:20:47.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.519+0000 7ffaa5170740 20 client.30109 trim_cache size 0 max 16384 2024-09-19T09:20:47.523 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:20:47.523 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:20:47.523 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-19T09:20:47.523 INFO:teuthology.orchestra.run.smithi120.stdout: "5d666f85-c4ef-4ff9-bd5f-bab7f7c7b5cd": { 2024-09-19T09:20:47.523 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-19T09:20:47.523 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "5d666f85-c4ef-4ff9-bd5f-bab7f7c7b5cd", 2024-09-19T09:20:47.523 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-19T09:20:47.523 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:20:47.524 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:20:47.524 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:20:47.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.520+0000 7ffaa1b35640 1 -- 172.21.15.120:0/1766924948 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ffa8c0a3240 msgr2=0x7ffa8c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:47.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.520+0000 7ffaa1b35640 1 --2- 172.21.15.120:0/1766924948 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ffa8c0a3240 0x7ffa8c0a5700 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ffa940116b0 tx=0x7ffa94009500 comp rx=0 tx=0).stop 2024-09-19T09:20:47.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.520+0000 7ffaa1b35640 1 -- 172.21.15.120:0/1766924948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa9c1008d0 msgr2=0x7ffa9c1982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:47.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.520+0000 7ffaa1b35640 1 --2- 172.21.15.120:0/1766924948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa9c1008d0 0x7ffa9c1982b0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7ffa90009e00 tx=0x7ffa9000c6a0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.520+0000 7ffaa1b35640 1 -- 172.21.15.120:0/1766924948 shutdown_connections 2024-09-19T09:20:47.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.520+0000 7ffaa1b35640 1 --2- 172.21.15.120:0/1766924948 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ffa8c0a3240 0x7ffa8c0a5700 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.520+0000 7ffaa1b35640 1 --2- 172.21.15.120:0/1766924948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa9c1068d0 0x7ffa9c19cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.520+0000 7ffaa1b35640 1 --2- 172.21.15.120:0/1766924948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa9c101270 0x7ffa9c1987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.520+0000 7ffaa1b35640 1 --2- 172.21.15.120:0/1766924948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa9c1008d0 0x7ffa9c1982b0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:47.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.520+0000 7ffaa1b35640 1 -- 172.21.15.120:0/1766924948 >> 172.21.15.120:0/1766924948 conn(0x7ffa9c0fc610 msgr2=0x7ffa9c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:47.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.521+0000 7ffaa1b35640 1 -- 172.21.15.120:0/1766924948 shutdown_connections 2024-09-19T09:20:47.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:47.521+0000 7ffaa1b35640 1 -- 172.21.15.120:0/1766924948 wait complete. 2024-09-19T09:20:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.081+0000 7fe5dffff640 1 -- 172.21.15.120:0/1684749484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5e0101a30 msgr2=0x7fe5e0101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.081+0000 7fe5dffff640 1 --2- 172.21.15.120:0/1684749484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5e0101a30 0x7fe5e0101eb0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009900 tx=0x7fe5d402a410 comp rx=0 tx=0).stop 2024-09-19T09:20:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.081+0000 7fe5dffff640 1 -- 172.21.15.120:0/1684749484 shutdown_connections 2024-09-19T09:20:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.081+0000 7fe5dffff640 1 --2- 172.21.15.120:0/1684749484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5e01023f0 0x7fe5e0108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.081+0000 7fe5dffff640 1 --2- 172.21.15.120:0/1684749484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5e0101a30 0x7fe5e0101eb0 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.081+0000 7fe5dffff640 1 --2- 172.21.15.120:0/1684749484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5e0100840 0x7fe5e0100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.081+0000 7fe5dffff640 1 -- 172.21.15.120:0/1684749484 >> 172.21.15.120:0/1684749484 conn(0x7fe5e00fbfa0 msgr2=0x7fe5e00fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.082+0000 7fe5dffff640 1 -- 172.21.15.120:0/1684749484 shutdown_connections 2024-09-19T09:20:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.082+0000 7fe5dffff640 1 -- 172.21.15.120:0/1684749484 wait complete. 2024-09-19T09:20:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.082+0000 7fe5dffff640 1 Processor -- start 2024-09-19T09:20:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.083+0000 7fe5dffff640 1 -- start start 2024-09-19T09:20:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.083+0000 7fe5dffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5e0100840 0x7fe5e01047d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.083+0000 7fe5dffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5e0101a30 0x7fe5e0102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.083+0000 7fe5dffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5e01023f0 0x7fe5e0103350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.083+0000 7fe5dffff640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e010c570 con 0x7fe5e01023f0 2024-09-19T09:20:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.083+0000 7fe5dffff640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e010c3f0 con 0x7fe5e0100840 2024-09-19T09:20:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.083+0000 7fe5dffff640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e010c6f0 con 0x7fe5e0101a30 2024-09-19T09:20:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.083+0000 7fe5de7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5e0101a30 0x7fe5e0102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.083+0000 7fe5df7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5e01023f0 0x7fe5e0103350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.083+0000 7fe5de7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5e0101a30 0x7fe5e0102e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36476/0 (socket says 172.21.15.120:36476) 2024-09-19T09:20:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.083+0000 7fe5de7fc640 1 -- 172.21.15.120:0/3025632692 learned_addr learned my addr 172.21.15.120:0/3025632692 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.084+0000 7fe5deffd640 1 --2- 172.21.15.120:0/3025632692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5e0100840 0x7fe5e01047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.084+0000 7fe5df7fe640 1 -- 172.21.15.120:0/3025632692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5e0101a30 msgr2=0x7fe5e0102e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.084+0000 7fe5df7fe640 1 --2- 172.21.15.120:0/3025632692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5e0101a30 0x7fe5e0102e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.084+0000 7fe5df7fe640 1 -- 172.21.15.120:0/3025632692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5e0100840 msgr2=0x7fe5e01047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.084+0000 7fe5df7fe640 1 --2- 172.21.15.120:0/3025632692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5e0100840 0x7fe5e01047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.084+0000 7fe5df7fe640 1 -- 172.21.15.120:0/3025632692 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e0103890 con 0x7fe5e01023f0 2024-09-19T09:20:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.084+0000 7fe5de7fc640 1 --2- 172.21.15.120:0/3025632692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5e0101a30 0x7fe5e0102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:20:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.084+0000 7fe5df7fe640 1 --2- 172.21.15.120:0/3025632692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5e01023f0 0x7fe5e0103350 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800be20 tx=0x7fe5d800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.084+0000 7fe5c7fff640 1 -- 172.21.15.120:0/3025632692 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8019070 con 0x7fe5e01023f0 2024-09-19T09:20:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.084+0000 7fe5c7fff640 1 -- 172.21.15.120:0/3025632692 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d8014920 con 0x7fe5e01023f0 2024-09-19T09:20:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.084+0000 7fe5dffff640 1 -- 172.21.15.120:0/3025632692 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e0103b80 con 0x7fe5e01023f0 2024-09-19T09:20:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.084+0000 7fe5c7fff640 1 -- 172.21.15.120:0/3025632692 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8012920 con 0x7fe5e01023f0 2024-09-19T09:20:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.085+0000 7fe5dffff640 1 -- 172.21.15.120:0/3025632692 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e01a6800 con 0x7fe5e01023f0 2024-09-19T09:20:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.087+0000 7fe5c7fff640 1 -- 172.21.15.120:0/3025632692 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe5d801c070 con 0x7fe5e01023f0 2024-09-19T09:20:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.088+0000 7fe5c7fff640 1 --2- 172.21.15.120:0/3025632692 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5c80a3240 0x7fe5c80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.088+0000 7fe5c7fff640 1 -- 172.21.15.120:0/3025632692 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe5d80cf180 con 0x7fe5e01023f0 2024-09-19T09:20:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.088+0000 7fe5deffd640 1 --2- 172.21.15.120:0/3025632692 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5c80a3240 0x7fe5c80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.088+0000 7fe5e8275740 1 Processor -- start 2024-09-19T09:20:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.088+0000 7fe5e8275740 1 -- start start 2024-09-19T09:20:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.088+0000 7fe5e8275740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cde230b70 0x561cde230f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.088+0000 7fe5e8275740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cde2314b0 0x561cde235f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.088+0000 7fe5e8275740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561cde236540 0x561cde238a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.088+0000 7fe5e8275740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561cde1942f0 con 0x561cde230b70 2024-09-19T09:20:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.088+0000 7fe5e8275740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561cde13c270 con 0x561cde2314b0 2024-09-19T09:20:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.088+0000 7fe5e8275740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561cde1902b0 con 0x561cde236540 2024-09-19T09:20:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.089+0000 7fe5deffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cde230b70 0x561cde230f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.089+0000 7fe5deffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cde230b70 0x561cde230f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43066/0 (socket says 172.21.15.120:43066) 2024-09-19T09:20:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.089+0000 7fe5deffd640 1 -- 172.21.15.120:0/486898583 learned_addr learned my addr 172.21.15.120:0/486898583 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.089+0000 7fe5df7fe640 1 --2- 172.21.15.120:0/486898583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561cde236540 0x561cde238a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.089+0000 7fe5deffd640 1 --2- 172.21.15.120:0/3025632692 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5c80a3240 0x7fe5c80a5700 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0101890 tx=0x7fe5d0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.089+0000 7fe5de7fc640 1 --2- 172.21.15.120:0/486898583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cde2314b0 0x561cde235f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.089+0000 7fe5deffd640 1 -- 172.21.15.120:0/486898583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561cde236540 msgr2=0x561cde238a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.089+0000 7fe5deffd640 1 --2- 172.21.15.120:0/486898583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561cde236540 0x561cde238a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.089+0000 7fe5deffd640 1 -- 172.21.15.120:0/486898583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cde2314b0 msgr2=0x561cde235f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.089+0000 7fe5deffd640 1 --2- 172.21.15.120:0/486898583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cde2314b0 0x561cde235f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.089+0000 7fe5deffd640 1 -- 172.21.15.120:0/486898583 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cde13b340 con 0x561cde230b70 2024-09-19T09:20:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.089+0000 7fe5de7fc640 1 --2- 172.21.15.120:0/486898583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cde2314b0 0x561cde235f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:20:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.089+0000 7fe5deffd640 1 --2- 172.21.15.120:0/486898583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cde230b70 0x561cde230f70 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000e620 tx=0x7fe5d0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852697a73cc49f65 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.089+0000 7fe5c57fa640 1 -- 172.21.15.120:0/486898583 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0004020 con 0x561cde230b70 2024-09-19T09:20:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.090+0000 7fe5c57fa640 1 -- 172.21.15.120:0/486898583 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d00041c0 con 0x561cde230b70 2024-09-19T09:20:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.090+0000 7fe5c57fa640 1 -- 172.21.15.120:0/486898583 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0004020 con 0x561cde230b70 2024-09-19T09:20:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.090+0000 7fe5e8275740 1 -- 172.21.15.120:0/486898583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cde230b70 msgr2=0x561cde230f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.090+0000 7fe5e8275740 1 --2- 172.21.15.120:0/486898583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cde230b70 0x561cde230f70 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000e620 tx=0x7fe5d0014000 comp rx=0 tx=0).stop 2024-09-19T09:20:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.091+0000 7fe5e8275740 1 -- 172.21.15.120:0/486898583 shutdown_connections 2024-09-19T09:20:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.091+0000 7fe5e8275740 1 --2- 172.21.15.120:0/486898583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561cde236540 0x561cde238a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.091+0000 7fe5e8275740 1 --2- 172.21.15.120:0/486898583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cde2314b0 0x561cde235f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.091+0000 7fe5e8275740 1 --2- 172.21.15.120:0/486898583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cde230b70 0x561cde230f70 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.091+0000 7fe5e8275740 1 -- 172.21.15.120:0/486898583 >> 172.21.15.120:0/486898583 conn(0x561cde22c1e0 msgr2=0x561cde22c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.091+0000 7fe5e8275740 1 -- 172.21.15.120:0/486898583 shutdown_connections 2024-09-19T09:20:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.091+0000 7fe5e8275740 1 -- 172.21.15.120:0/486898583 wait complete. 2024-09-19T09:20:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.091+0000 7fe5e8275740 1 -- start start 2024-09-19T09:20:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.091+0000 7fe5e8275740 1 Processor -- start 2024-09-19T09:20:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.091+0000 7fe5e8275740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cde230b70 0x561cde23ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.091+0000 7fe5e8275740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cde2314b0 0x561cde23b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.091+0000 7fe5e8275740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561cde236540 0x561cde2422a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.091+0000 7fe5e8275740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561cde239510 con 0x561cde2314b0 2024-09-19T09:20:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.091+0000 7fe5e8275740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561cde11d290 con 0x561cde230b70 2024-09-19T09:20:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.091+0000 7fe5e8275740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561cde239690 con 0x561cde236540 2024-09-19T09:20:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.092+0000 7fe5e8275740 10 client.0 _ll_register_callbacks cb 0x561cde14e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:20:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.092+0000 7fe5de7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cde2314b0 0x561cde23b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.092+0000 7fe5de7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cde2314b0 0x561cde23b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43076/0 (socket says 172.21.15.120:43076) 2024-09-19T09:20:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.092+0000 7fe5de7fc640 1 -- 172.21.15.120:0/2371616893 learned_addr learned my addr 172.21.15.120:0/2371616893 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.092+0000 7fe5deffd640 1 --2- 172.21.15.120:0/2371616893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cde230b70 0x561cde23ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.092+0000 7fe5df7fe640 1 --2- 172.21.15.120:0/2371616893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561cde236540 0x561cde2422a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.092+0000 7fe5de7fc640 1 -- 172.21.15.120:0/2371616893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561cde236540 msgr2=0x561cde2422a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.092+0000 7fe5de7fc640 1 --2- 172.21.15.120:0/2371616893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561cde236540 0x561cde2422a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.092+0000 7fe5de7fc640 1 -- 172.21.15.120:0/2371616893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cde230b70 msgr2=0x561cde23ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.092+0000 7fe5de7fc640 1 --2- 172.21.15.120:0/2371616893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cde230b70 0x561cde23ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.092+0000 7fe5de7fc640 1 -- 172.21.15.120:0/2371616893 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cddea7210 con 0x561cde2314b0 2024-09-19T09:20:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.092+0000 7fe5de7fc640 1 --2- 172.21.15.120:0/2371616893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cde2314b0 0x561cde23b2d0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fe5d402aa00 tx=0x7fe5d4031960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b227f331f3996 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.092+0000 7fe5c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:20:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.092+0000 7fe5c2ff5640 1 -- 172.21.15.120:0/2371616893 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4035040 con 0x561cde2314b0 2024-09-19T09:20:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.092+0000 7fe5c2ff5640 1 -- 172.21.15.120:0/2371616893 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe5d4031d50 con 0x561cde2314b0 2024-09-19T09:20:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.093+0000 7fe5e8275740 1 -- 172.21.15.120:0/2371616893 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5d00041c0 con 0x561cde2314b0 2024-09-19T09:20:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.093+0000 7fe5c2ff5640 1 -- 172.21.15.120:0/2371616893 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe5d4005020 con 0x561cde2314b0 2024-09-19T09:20:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.093+0000 7fe5e8275740 10 client.21447 fetch_fsmap learned FSMap version 39 2024-09-19T09:20:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.093+0000 7fe5e8275740 1 -- 172.21.15.120:0/2371616893 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561cddf73620 con 0x561cde2314b0 2024-09-19T09:20:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.093+0000 7fe5c2ff5640 1 -- 172.21.15.120:0/2371616893 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d00041c0 con 0x561cde2314b0 2024-09-19T09:20:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.093+0000 7fe5c2ff5640 1 -- 172.21.15.120:0/2371616893 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fe5d40035f0 con 0x561cde2314b0 2024-09-19T09:20:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.093+0000 7fe5e8275740 10 client.21447 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:20:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.093+0000 7fe5e8275740 10 client.21447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:20:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.093+0000 7fe5e8275740 20 client.21447 populate_metadata read hostname 'smithi120' 2024-09-19T09:20:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.093+0000 7fe5e8275740 1 --2- 172.21.15.120:0/2371616893 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561cde2469e0 0x561cde248e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.093+0000 7fe5e8275740 4 client.21447 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:20:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.093+0000 7fe5e8275740 1 -- 172.21.15.120:0/2371616893 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cde1a1a20 con 0x561cde2469e0 2024-09-19T09:20:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.093+0000 7fe5deffd640 1 --2- 172.21.15.120:0/2371616893 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561cde2469e0 0x561cde248e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.094+0000 7fe5deffd640 1 --2- 172.21.15.120:0/2371616893 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561cde2469e0 0x561cde248e10 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=347440c526c0410c server_cookie=18d1342fba2e00fc in_seq=0 out_seq=0 2024-09-19T09:20:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.094+0000 7fe5c2ff5640 10 client.21447 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.100+0000 7fe5c2ff5640 1 -- 172.21.15.120:0/2371616893 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561cde1a1a20 con 0x561cde2469e0 2024-09-19T09:20:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.100+0000 7fe5c2ff5640 10 client.21447 handle_command_reply: tid=0 2024-09-19T09:20:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.100+0000 7fe5e8275740 1 client.21447 shutdown 2024-09-19T09:20:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.100+0000 7fe5e8275740 1 -- 172.21.15.120:0/2371616893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cde2314b0 msgr2=0x561cde23b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.100+0000 7fe5e8275740 1 --2- 172.21.15.120:0/2371616893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cde2314b0 0x561cde23b2d0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fe5d402aa00 tx=0x7fe5d4031960 comp rx=0 tx=0).stop 2024-09-19T09:20:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.100+0000 7fe5e8275740 1 -- 172.21.15.120:0/2371616893 shutdown_connections 2024-09-19T09:20:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.100+0000 7fe5e8275740 1 --2- 172.21.15.120:0/2371616893 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561cde2469e0 0x561cde248e10 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.101+0000 7fe5c2ff5640 0 client.21447 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.101+0000 7fe5e8275740 1 --2- 172.21.15.120:0/2371616893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561cde236540 0x561cde2422a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.101+0000 7fe5e8275740 1 --2- 172.21.15.120:0/2371616893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cde2314b0 0x561cde23b2d0 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.101+0000 7fe5e8275740 1 --2- 172.21.15.120:0/2371616893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cde230b70 0x561cde23ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.101+0000 7fe5e8275740 1 -- 172.21.15.120:0/2371616893 >> 172.21.15.120:0/2371616893 conn(0x561cde22c1e0 msgr2=0x561cde22d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.101+0000 7fe5e8275740 1 -- 172.21.15.120:0/2371616893 shutdown_connections 2024-09-19T09:20:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.101+0000 7fe5e8275740 1 -- 172.21.15.120:0/2371616893 wait complete. 2024-09-19T09:20:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.101+0000 7fe5e8275740 20 client.21447 trim_cache size 0 max 16384 2024-09-19T09:20:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.116+0000 7fe5e8275740 1 Processor -- start 2024-09-19T09:20:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.116+0000 7fe5e8275740 1 -- start start 2024-09-19T09:20:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.117+0000 7fe5e8275740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cde230b70 0x561cde22d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.117+0000 7fe5e8275740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cde2314b0 0x561cde242260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.117+0000 7fe5e8275740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561cde236540 0x561cde22db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.117+0000 7fe5e8275740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e01af5d0 con 0x561cde230b70 2024-09-19T09:20:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.117+0000 7fe5e8275740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e01b1110 con 0x561cde2314b0 2024-09-19T09:20:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.117+0000 7fe5e8275740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e01afca0 con 0x561cde236540 2024-09-19T09:20:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.117+0000 7fe5df7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561cde236540 0x561cde22db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.117+0000 7fe5df7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561cde236540 0x561cde22db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36502/0 (socket says 172.21.15.120:36502) 2024-09-19T09:20:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.117+0000 7fe5df7fe640 1 -- 172.21.15.120:0/2629312007 learned_addr learned my addr 172.21.15.120:0/2629312007 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.117+0000 7fe5df7fe640 1 -- 172.21.15.120:0/2629312007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cde2314b0 msgr2=0x561cde242260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.117+0000 7fe5df7fe640 1 --2- 172.21.15.120:0/2629312007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cde2314b0 0x561cde242260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.117+0000 7fe5df7fe640 1 -- 172.21.15.120:0/2629312007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cde230b70 msgr2=0x561cde22d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.117+0000 7fe5df7fe640 1 --2- 172.21.15.120:0/2629312007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cde230b70 0x561cde22d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.117+0000 7fe5df7fe640 1 -- 172.21.15.120:0/2629312007 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cde22c000 con 0x561cde236540 2024-09-19T09:20:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.118+0000 7fe5df7fe640 1 --2- 172.21.15.120:0/2629312007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561cde236540 0x561cde22db10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80ccc80 tx=0x7fe5d8086040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d8dcf0cbbf1f123 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.118+0000 7fe5c4ff9640 1 -- 172.21.15.120:0/2629312007 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d80a4410 con 0x561cde236540 2024-09-19T09:20:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.118+0000 7fe5c4ff9640 1 -- 172.21.15.120:0/2629312007 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d80b4070 con 0x561cde236540 2024-09-19T09:20:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.118+0000 7fe5c4ff9640 1 -- 172.21.15.120:0/2629312007 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8084e00 con 0x561cde236540 2024-09-19T09:20:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.119+0000 7fe5e8275740 1 -- 172.21.15.120:0/2629312007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561cde236540 msgr2=0x561cde22db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.119+0000 7fe5e8275740 1 --2- 172.21.15.120:0/2629312007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561cde236540 0x561cde22db10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80ccc80 tx=0x7fe5d8086040 comp rx=0 tx=0).stop 2024-09-19T09:20:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.119+0000 7fe5e8275740 1 -- 172.21.15.120:0/2629312007 shutdown_connections 2024-09-19T09:20:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.119+0000 7fe5e8275740 1 --2- 172.21.15.120:0/2629312007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561cde236540 0x561cde22db10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.119+0000 7fe5e8275740 1 --2- 172.21.15.120:0/2629312007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cde2314b0 0x561cde242260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.119+0000 7fe5e8275740 1 --2- 172.21.15.120:0/2629312007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cde230b70 0x561cde22d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.119+0000 7fe5e8275740 1 -- 172.21.15.120:0/2629312007 >> 172.21.15.120:0/2629312007 conn(0x561cde22c1e0 msgr2=0x561cde22b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.119+0000 7fe5e8275740 1 -- 172.21.15.120:0/2629312007 shutdown_connections 2024-09-19T09:20:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.119+0000 7fe5e8275740 1 -- 172.21.15.120:0/2629312007 wait complete. 2024-09-19T09:20:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.119+0000 7fe5e8275740 1 -- start start 2024-09-19T09:20:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.119+0000 7fe5e8275740 1 Processor -- start 2024-09-19T09:20:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.119+0000 7fe5e8275740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cde230b70 0x561cde240390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.120+0000 7fe5e8275740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561cde2314b0 0x561cde2408d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.120+0000 7fe5e8275740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cde236540 0x561cde241770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.120+0000 7fe5e8275740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e01b0a40 con 0x561cde236540 2024-09-19T09:20:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.120+0000 7fe5e8275740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e01b0370 con 0x561cde230b70 2024-09-19T09:20:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.120+0000 7fe5e8275740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e01ad590 con 0x561cde2314b0 2024-09-19T09:20:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.120+0000 7fe5de7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561cde2314b0 0x561cde2408d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.120+0000 7fe5de7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561cde2314b0 0x561cde2408d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36504/0 (socket says 172.21.15.120:36504) 2024-09-19T09:20:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.120+0000 7fe5de7fc640 1 -- 172.21.15.120:0/866716954 learned_addr learned my addr 172.21.15.120:0/866716954 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.120+0000 7fe5e8275740 10 client.0 _ll_register_callbacks cb 0x7fe5d4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:20:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.120+0000 7fe5df7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cde236540 0x561cde241770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.120+0000 7fe5de7fc640 1 -- 172.21.15.120:0/866716954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cde230b70 msgr2=0x561cde240390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.120+0000 7fe5de7fc640 1 --2- 172.21.15.120:0/866716954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cde230b70 0x561cde240390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.120+0000 7fe5de7fc640 1 -- 172.21.15.120:0/866716954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cde236540 msgr2=0x561cde241770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.120+0000 7fe5de7fc640 1 --2- 172.21.15.120:0/866716954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cde236540 0x561cde241770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.120+0000 7fe5de7fc640 1 -- 172.21.15.120:0/866716954 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cde241e30 con 0x561cde2314b0 2024-09-19T09:20:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.120+0000 7fe5de7fc640 1 --2- 172.21.15.120:0/866716954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561cde2314b0 0x561cde2408d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe5d402a930 tx=0x7fe5d4005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee8a13db1c16951f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.120+0000 7fe5c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:20:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.121+0000 7fe5c2ff5640 1 -- 172.21.15.120:0/866716954 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4035050 con 0x561cde2314b0 2024-09-19T09:20:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.121+0000 7fe5c2ff5640 1 -- 172.21.15.120:0/866716954 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe5d40055d0 con 0x561cde2314b0 2024-09-19T09:20:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.121+0000 7fe5e8275740 1 -- 172.21.15.120:0/866716954 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5d80b4070 con 0x561cde2314b0 2024-09-19T09:20:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.121+0000 7fe5c2ff5640 1 -- 172.21.15.120:0/866716954 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe5d40043f0 con 0x561cde2314b0 2024-09-19T09:20:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.121+0000 7fe5e8275740 10 client.30127 fetch_fsmap learned FSMap version 39 2024-09-19T09:20:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.121+0000 7fe5e8275740 1 -- 172.21.15.120:0/866716954 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561cde2b20c0 con 0x561cde2314b0 2024-09-19T09:20:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.121+0000 7fe5c2ff5640 1 -- 172.21.15.120:0/866716954 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d80b4070 con 0x561cde2314b0 2024-09-19T09:20:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.121+0000 7fe5c2ff5640 1 -- 172.21.15.120:0/866716954 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fe5d4004be0 con 0x561cde2314b0 2024-09-19T09:20:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.121+0000 7fe5e8275740 10 client.30127 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:20:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.121+0000 7fe5e8275740 10 client.30127 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:20:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.121+0000 7fe5e8275740 20 client.30127 populate_metadata read hostname 'smithi120' 2024-09-19T09:20:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.121+0000 7fe5e8275740 1 --2- 172.21.15.120:0/866716954 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561cde2469e0 0x561cde2b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.121+0000 7fe5e8275740 4 client.30127 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:20:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.121+0000 7fe5e8275740 1 -- 172.21.15.120:0/866716954 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561cde238e80 con 0x561cde2469e0 2024-09-19T09:20:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.125+0000 7fe5deffd640 1 --2- 172.21.15.120:0/866716954 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561cde2469e0 0x561cde2b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.127+0000 7fe5c2ff5640 10 client.30127 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.127+0000 7fe5deffd640 1 --2- 172.21.15.120:0/866716954 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561cde2469e0 0x561cde2b4a50 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73c6573ff5ff89b server_cookie=e9d1d6d5779b1c31 in_seq=0 out_seq=0 2024-09-19T09:20:48.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:48 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:20:48.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:48 smithi120 ceph-mon[31150]: pgmap v883: 97 pgs: 97 active+clean; 386 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.0 MiB/s wr, 797 op/s 2024-09-19T09:20:48.367 INFO:tasks.workunit.client.0.smithi120.stdout: 10 121419 21.66 MB/sec execute 163 sec latency 193.554 ms 2024-09-19T09:20:48.396 INFO:tasks.cephfs.filesystem:scrub status for tag:5d666f85-c4ef-4ff9-bd5f-bab7f7c7b5cd - {'path': '/', 'tag': '5d666f85-c4ef-4ff9-bd5f-bab7f7c7b5cd', 'options': 'recursive,force'} 2024-09-19T09:20:48.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:48 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:20:48.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:48 smithi196 ceph-mon[35150]: pgmap v883: 97 pgs: 97 active+clean; 386 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.0 MiB/s wr, 797 op/s 2024-09-19T09:20:48.441 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:48 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:20:48.441 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:48 smithi153 ceph-mon[35071]: pgmap v883: 97 pgs: 97 active+clean; 386 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.0 MiB/s wr, 797 op/s 2024-09-19T09:20:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.652+0000 7fe5c2ff5640 1 -- 172.21.15.120:0/866716954 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561cde238e80 con 0x561cde2469e0 2024-09-19T09:20:48.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.653+0000 7fe5c2ff5640 10 client.30127 handle_command_reply: tid=0 2024-09-19T09:20:48.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.653+0000 7fe5e8275740 1 client.30127 shutdown 2024-09-19T09:20:48.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.653+0000 7fe5e8275740 1 -- 172.21.15.120:0/866716954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561cde2314b0 msgr2=0x561cde2408d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:48.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.653+0000 7fe5e8275740 1 --2- 172.21.15.120:0/866716954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561cde2314b0 0x561cde2408d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe5d402a930 tx=0x7fe5d4005fd0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.653+0000 7fe5e8275740 1 -- 172.21.15.120:0/866716954 shutdown_connections 2024-09-19T09:20:48.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.653+0000 7fe5e8275740 1 --2- 172.21.15.120:0/866716954 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561cde2469e0 0x561cde2b4a50 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.653+0000 7fe5e8275740 1 --2- 172.21.15.120:0/866716954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cde236540 0x561cde241770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.653+0000 7fe5e8275740 1 --2- 172.21.15.120:0/866716954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561cde2314b0 0x561cde2408d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.653+0000 7fe5e8275740 1 --2- 172.21.15.120:0/866716954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561cde230b70 0x561cde240390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.653+0000 7fe5e8275740 1 -- 172.21.15.120:0/866716954 >> 172.21.15.120:0/866716954 conn(0x561cde22c1e0 msgr2=0x561cde22c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:48.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.653+0000 7fe5c2ff5640 0 client.30127 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:48.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.653+0000 7fe5e8275740 1 -- 172.21.15.120:0/866716954 shutdown_connections 2024-09-19T09:20:48.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.653+0000 7fe5e8275740 1 -- 172.21.15.120:0/866716954 wait complete. 2024-09-19T09:20:48.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.654+0000 7fe5e8275740 20 client.30127 trim_cache size 0 max 16384 2024-09-19T09:20:48.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:20:48.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:20:48.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:20:48.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:20:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.654+0000 7fe5dffff640 1 -- 172.21.15.120:0/3025632692 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5c80a3240 msgr2=0x7fe5c80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.654+0000 7fe5dffff640 1 --2- 172.21.15.120:0/3025632692 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5c80a3240 0x7fe5c80a5700 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0101890 tx=0x7fe5d0005f90 comp rx=0 tx=0).stop 2024-09-19T09:20:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.655+0000 7fe5dffff640 1 -- 172.21.15.120:0/3025632692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5e01023f0 msgr2=0x7fe5e0103350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.655+0000 7fe5dffff640 1 --2- 172.21.15.120:0/3025632692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5e01023f0 0x7fe5e0103350 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800be20 tx=0x7fe5d800bf10 comp rx=0 tx=0).stop 2024-09-19T09:20:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.655+0000 7fe5dffff640 1 -- 172.21.15.120:0/3025632692 shutdown_connections 2024-09-19T09:20:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.655+0000 7fe5dffff640 1 --2- 172.21.15.120:0/3025632692 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5c80a3240 0x7fe5c80a5700 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.655+0000 7fe5dffff640 1 --2- 172.21.15.120:0/3025632692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5e01023f0 0x7fe5e0103350 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.655+0000 7fe5dffff640 1 --2- 172.21.15.120:0/3025632692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5e0101a30 0x7fe5e0102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.655+0000 7fe5dffff640 1 --2- 172.21.15.120:0/3025632692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5e0100840 0x7fe5e01047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.656+0000 7fe5dffff640 1 -- 172.21.15.120:0/3025632692 >> 172.21.15.120:0/3025632692 conn(0x7fe5e00fbfa0 msgr2=0x7fe5e00698d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.657+0000 7fe5dffff640 1 -- 172.21.15.120:0/3025632692 shutdown_connections 2024-09-19T09:20:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:20:48.657+0000 7fe5dffff640 1 -- 172.21.15.120:0/3025632692 wait complete. 2024-09-19T09:20:49.367 INFO:tasks.workunit.client.0.smithi120.stdout: 10 121836 21.67 MB/sec execute 164 sec latency 349.183 ms 2024-09-19T09:20:50.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:50 smithi120 ceph-mon[31150]: pgmap v884: 97 pgs: 97 active+clean; 382 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.9 MiB/s wr, 736 op/s 2024-09-19T09:20:50.367 INFO:tasks.workunit.client.0.smithi120.stdout: 10 122455 21.74 MB/sec execute 165 sec latency 60.638 ms 2024-09-19T09:20:50.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:50 smithi196 ceph-mon[35150]: pgmap v884: 97 pgs: 97 active+clean; 382 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.9 MiB/s wr, 736 op/s 2024-09-19T09:20:50.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:50 smithi153 ceph-mon[35071]: pgmap v884: 97 pgs: 97 active+clean; 382 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.9 MiB/s wr, 736 op/s 2024-09-19T09:20:51.367 INFO:tasks.workunit.client.0.smithi120.stdout: 10 123232 21.87 MB/sec execute 166 sec latency 52.763 ms 2024-09-19T09:20:51.509 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.1 2024-09-19T09:20:51.509 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:20:51.824 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:20:52.155 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:52 smithi120 ceph-mon[31150]: pgmap v885: 97 pgs: 97 active+clean; 377 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.9 MiB/s wr, 752 op/s 2024-09-19T09:20:52.367 INFO:tasks.workunit.client.0.smithi120.stdout: 10 123859 21.94 MB/sec execute 167 sec latency 65.941 ms 2024-09-19T09:20:52.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:52 smithi196 ceph-mon[35150]: pgmap v885: 97 pgs: 97 active+clean; 377 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.9 MiB/s wr, 752 op/s 2024-09-19T09:20:52.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:52 smithi153 ceph-mon[35071]: pgmap v885: 97 pgs: 97 active+clean; 377 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.9 MiB/s wr, 752 op/s 2024-09-19T09:20:52.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.498+0000 7ff0c2d99640 1 -- 172.21.15.120:0/3567152779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0bc100e20 msgr2=0x7ff0bc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:52.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.498+0000 7ff0c2d99640 1 --2- 172.21.15.120:0/3567152779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0bc100e20 0x7ff0bc101200 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0009900 tx=0x7ff0b002a480 comp rx=0 tx=0).stop 2024-09-19T09:20:52.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.498+0000 7ff0c2d99640 1 -- 172.21.15.120:0/3567152779 shutdown_connections 2024-09-19T09:20:52.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.498+0000 7ff0c2d99640 1 --2- 172.21.15.120:0/3567152779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0bc102ef0 0x7ff0bc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:52.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.498+0000 7ff0c2d99640 1 --2- 172.21.15.120:0/3567152779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0bc102570 0x7ff0bc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:52.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.498+0000 7ff0c2d99640 1 --2- 172.21.15.120:0/3567152779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0bc100e20 0x7ff0bc101200 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:52.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.498+0000 7ff0c2d99640 1 -- 172.21.15.120:0/3567152779 >> 172.21.15.120:0/3567152779 conn(0x7ff0bc0fc7d0 msgr2=0x7ff0bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:52.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.498+0000 7ff0c2d99640 1 -- 172.21.15.120:0/3567152779 shutdown_connections 2024-09-19T09:20:52.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.498+0000 7ff0c2d99640 1 -- 172.21.15.120:0/3567152779 wait complete. 2024-09-19T09:20:52.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.499+0000 7ff0c2d99640 1 Processor -- start 2024-09-19T09:20:52.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.499+0000 7ff0c2d99640 1 -- start start 2024-09-19T09:20:52.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.499+0000 7ff0c2d99640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0bc100e20 0x7ff0bc199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:52.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.499+0000 7ff0c2d99640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0bc102570 0x7ff0bc199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:52.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.499+0000 7ff0c2d99640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0bc102ef0 0x7ff0bc1a4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:52.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.499+0000 7ff0c2d99640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc110fd0 con 0x7ff0bc102570 2024-09-19T09:20:52.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.500+0000 7ff0c2d99640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc110e50 con 0x7ff0bc100e20 2024-09-19T09:20:52.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.500+0000 7ff0c2d99640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc111150 con 0x7ff0bc102ef0 2024-09-19T09:20:52.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.500+0000 7ff0bbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0bc102570 0x7ff0bc199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:52.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.500+0000 7ff0bbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0bc102570 0x7ff0bc199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41438/0 (socket says 172.21.15.120:41438) 2024-09-19T09:20:52.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.500+0000 7ff0bbfff640 1 -- 172.21.15.120:0/3448785522 learned_addr learned my addr 172.21.15.120:0/3448785522 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:52.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.500+0000 7ff0c1336640 1 --2- 172.21.15.120:0/3448785522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0bc102ef0 0x7ff0bc1a4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:52.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.500+0000 7ff0c0b35640 1 --2- 172.21.15.120:0/3448785522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0bc100e20 0x7ff0bc199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:52.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.500+0000 7ff0bbfff640 1 -- 172.21.15.120:0/3448785522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0bc102ef0 msgr2=0x7ff0bc1a4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:52.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.500+0000 7ff0bbfff640 1 --2- 172.21.15.120:0/3448785522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0bc102ef0 0x7ff0bc1a4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:52.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.500+0000 7ff0bbfff640 1 -- 172.21.15.120:0/3448785522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0bc100e20 msgr2=0x7ff0bc199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:52.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.500+0000 7ff0bbfff640 1 --2- 172.21.15.120:0/3448785522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0bc100e20 0x7ff0bc199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:52.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.500+0000 7ff0bbfff640 1 -- 172.21.15.120:0/3448785522 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0bc1a49c0 con 0x7ff0bc102570 2024-09-19T09:20:52.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.500+0000 7ff0bbfff640 1 --2- 172.21.15.120:0/3448785522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0bc102570 0x7ff0bc199940 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7ff0a800d950 tx=0x7ff0a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:52.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.500+0000 7ff0b9ffb640 1 -- 172.21.15.120:0/3448785522 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8017070 con 0x7ff0bc102570 2024-09-19T09:20:52.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.500+0000 7ff0c2d99640 1 -- 172.21.15.120:0/3448785522 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0bc1a4b90 con 0x7ff0bc102570 2024-09-19T09:20:52.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.500+0000 7ff0b9ffb640 1 -- 172.21.15.120:0/3448785522 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a8009c30 con 0x7ff0bc102570 2024-09-19T09:20:52.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.500+0000 7ff0b9ffb640 1 -- 172.21.15.120:0/3448785522 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a800f6b0 con 0x7ff0bc102570 2024-09-19T09:20:52.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.501+0000 7ff0c2d99640 1 -- 172.21.15.120:0/3448785522 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0bc1a50d0 con 0x7ff0bc102570 2024-09-19T09:20:52.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.502+0000 7ff0c2d99640 1 -- 172.21.15.120:0/3448785522 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0bc101200 con 0x7ff0bc102570 2024-09-19T09:20:52.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.503+0000 7ff0b9ffb640 1 -- 172.21.15.120:0/3448785522 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7ff0a80109d0 con 0x7ff0bc102570 2024-09-19T09:20:52.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.504+0000 7ff0b9ffb640 1 --2- 172.21.15.120:0/3448785522 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff0ac0a3240 0x7ff0ac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:52.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.504+0000 7ff0c0b35640 1 --2- 172.21.15.120:0/3448785522 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff0ac0a3240 0x7ff0ac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:52.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.504+0000 7ff0b9ffb640 1 -- 172.21.15.120:0/3448785522 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff0a80cf470 con 0x7ff0bc102570 2024-09-19T09:20:52.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.505+0000 7ff0c0b35640 1 --2- 172.21.15.120:0/3448785522 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff0ac0a3240 0x7ff0ac0a5700 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0000c00 tx=0x7ff0b0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:52.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.509+0000 7ff0b9ffb640 1 -- 172.21.15.120:0/3448785522 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff0a8009de0 con 0x7ff0bc102570 2024-09-19T09:20:52.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.788+0000 7ff0c2d99640 1 -- 172.21.15.120:0/3448785522 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0bc109090 con 0x7ff0bc102570 2024-09-19T09:20:52.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.789+0000 7ff0b9ffb640 1 -- 172.21.15.120:0/3448785522 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11988 (secure 0 0 0) 0x7ff0a809bb40 con 0x7ff0bc102570 2024-09-19T09:20:52.790 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:20:52.790 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":39,"btime":"2024-09-19T09:18:53:120858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:18:53.070639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:20:52.791 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 39 2024-09-19T09:20:52.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.791+0000 7ff0c2d99640 1 -- 172.21.15.120:0/3448785522 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff0ac0a3240 msgr2=0x7ff0ac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:52.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.791+0000 7ff0c2d99640 1 --2- 172.21.15.120:0/3448785522 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff0ac0a3240 0x7ff0ac0a5700 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0000c00 tx=0x7ff0b0005b50 comp rx=0 tx=0).stop 2024-09-19T09:20:52.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.791+0000 7ff0c2d99640 1 -- 172.21.15.120:0/3448785522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0bc102570 msgr2=0x7ff0bc199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:52.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.791+0000 7ff0c2d99640 1 --2- 172.21.15.120:0/3448785522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0bc102570 0x7ff0bc199940 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7ff0a800d950 tx=0x7ff0a800de10 comp rx=0 tx=0).stop 2024-09-19T09:20:52.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.792+0000 7ff0c2d99640 1 -- 172.21.15.120:0/3448785522 shutdown_connections 2024-09-19T09:20:52.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.792+0000 7ff0c2d99640 1 --2- 172.21.15.120:0/3448785522 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff0ac0a3240 0x7ff0ac0a5700 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:52.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.792+0000 7ff0c2d99640 1 --2- 172.21.15.120:0/3448785522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0bc102ef0 0x7ff0bc1a4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:52.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.792+0000 7ff0c2d99640 1 --2- 172.21.15.120:0/3448785522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0bc102570 0x7ff0bc199940 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:52.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.792+0000 7ff0c2d99640 1 --2- 172.21.15.120:0/3448785522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0bc100e20 0x7ff0bc199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:52.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.792+0000 7ff0c2d99640 1 -- 172.21.15.120:0/3448785522 >> 172.21.15.120:0/3448785522 conn(0x7ff0bc0fc7d0 msgr2=0x7ff0bc0fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:52.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.792+0000 7ff0c2d99640 1 -- 172.21.15.120:0/3448785522 shutdown_connections 2024-09-19T09:20:52.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:52.792+0000 7ff0c2d99640 1 -- 172.21.15.120:0/3448785522 wait complete. 2024-09-19T09:20:53.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:53 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3448785522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:20:53.368 INFO:tasks.workunit.client.0.smithi120.stdout: 10 124478 22.04 MB/sec execute 168 sec latency 71.081 ms 2024-09-19T09:20:53.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:53 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3448785522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:20:53.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:53 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3448785522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:20:53.639 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db / --timeout 90 --expiration 137.1 --await-for 45.0' 2024-09-19T09:20:53.639 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db / --timeout 90 --expiration 137.1 --await-for 45.0 2024-09-19T09:20:53.994 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:20:54.294 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:54 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:20:54.295 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:54 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:20:54.295 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:54 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:20:54.295 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:54 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:20:54.295 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:54 smithi120 ceph-mon[31150]: pgmap v886: 97 pgs: 97 active+clean; 380 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.7 MiB/s wr, 823 op/s 2024-09-19T09:20:54.369 INFO:tasks.workunit.client.0.smithi120.stdout: 10 124900 22.04 MB/sec execute 169 sec latency 223.961 ms 2024-09-19T09:20:54.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:54 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:20:54.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:54 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:20:54.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:54 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:20:54.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:54 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:20:54.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:54 smithi196 ceph-mon[35150]: pgmap v886: 97 pgs: 97 active+clean; 380 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.7 MiB/s wr, 823 op/s 2024-09-19T09:20:54.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:54 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:20:54.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:54 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:20:54.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:54 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:20:54.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:54 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:20:54.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:54 smithi153 ceph-mon[35071]: pgmap v886: 97 pgs: 97 active+clean; 380 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.7 MiB/s wr, 823 op/s 2024-09-19T09:20:54.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.512+0000 7f6282f5f640 1 -- 172.21.15.120:0/1579973524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f627c100100 msgr2=0x7f627c100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:54.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.512+0000 7f6282f5f640 1 --2- 172.21.15.120:0/1579973524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f627c100100 0x7f627c100560 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f6268009900 tx=0x7f626802a410 comp rx=0 tx=0).stop 2024-09-19T09:20:54.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.512+0000 7f6282f5f640 1 -- 172.21.15.120:0/1579973524 shutdown_connections 2024-09-19T09:20:54.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.512+0000 7f6282f5f640 1 --2- 172.21.15.120:0/1579973524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f627c100aa0 0x7f627c10ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:54.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.512+0000 7f6282f5f640 1 --2- 172.21.15.120:0/1579973524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f627c100100 0x7f627c100560 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:54.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.512+0000 7f6282f5f640 1 --2- 172.21.15.120:0/1579973524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f627c106b80 0x7f627c106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:54.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.512+0000 7f6282f5f640 1 -- 172.21.15.120:0/1579973524 >> 172.21.15.120:0/1579973524 conn(0x7f627c077230 msgr2=0x7f627c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:54.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.512+0000 7f6282f5f640 1 -- 172.21.15.120:0/1579973524 shutdown_connections 2024-09-19T09:20:54.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.512+0000 7f6282f5f640 1 -- 172.21.15.120:0/1579973524 wait complete. 2024-09-19T09:20:54.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.513+0000 7f6282f5f640 1 Processor -- start 2024-09-19T09:20:54.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.513+0000 7f6282f5f640 1 -- start start 2024-09-19T09:20:54.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.513+0000 7f6282f5f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f627c100100 0x7f627c198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:54.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.513+0000 7f6282f5f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f627c100aa0 0x7f627c198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:54.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.513+0000 7f6282f5f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f627c106b80 0x7f627c19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:54.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.513+0000 7f6282f5f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f627c112070 con 0x7f627c100aa0 2024-09-19T09:20:54.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.513+0000 7f6282f5f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f627c111ef0 con 0x7f627c106b80 2024-09-19T09:20:54.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.513+0000 7f6282f5f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f627c1121f0 con 0x7f627c100100 2024-09-19T09:20:54.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.513+0000 7f627bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f627c100aa0 0x7f627c198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:54.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.514+0000 7f627bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f627c100aa0 0x7f627c198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41458/0 (socket says 172.21.15.120:41458) 2024-09-19T09:20:54.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.514+0000 7f627bfff640 1 -- 172.21.15.120:0/226750324 learned_addr learned my addr 172.21.15.120:0/226750324 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:54.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.514+0000 7f6280cfb640 1 --2- 172.21.15.120:0/226750324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f627c100100 0x7f627c198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:54.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.514+0000 7f62814fc640 1 --2- 172.21.15.120:0/226750324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f627c106b80 0x7f627c19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:54.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.514+0000 7f627bfff640 1 -- 172.21.15.120:0/226750324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f627c100100 msgr2=0x7f627c198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:54.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.514+0000 7f627bfff640 1 --2- 172.21.15.120:0/226750324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f627c100100 0x7f627c198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:54.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.514+0000 7f627bfff640 1 -- 172.21.15.120:0/226750324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f627c106b80 msgr2=0x7f627c19ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:54.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.514+0000 7f627bfff640 1 --2- 172.21.15.120:0/226750324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f627c106b80 0x7f627c19ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:54.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.514+0000 7f627bfff640 1 -- 172.21.15.120:0/226750324 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f627c19d500 con 0x7f627c100aa0 2024-09-19T09:20:54.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.514+0000 7f6280cfb640 1 --2- 172.21.15.120:0/226750324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f627c100100 0x7f627c198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:20:54.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.514+0000 7f627bfff640 1 --2- 172.21.15.120:0/226750324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f627c100aa0 0x7f627c198a90 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f6268006010 tx=0x7f62680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:54.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.514+0000 7f6279ffb640 1 -- 172.21.15.120:0/226750324 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626803f030 con 0x7f627c100aa0 2024-09-19T09:20:54.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.514+0000 7f6279ffb640 1 -- 172.21.15.120:0/226750324 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6268003ce0 con 0x7f627c100aa0 2024-09-19T09:20:54.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.514+0000 7f6282f5f640 1 -- 172.21.15.120:0/226750324 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f627c19d790 con 0x7f627c100aa0 2024-09-19T09:20:54.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.514+0000 7f6279ffb640 1 -- 172.21.15.120:0/226750324 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6268035070 con 0x7f627c100aa0 2024-09-19T09:20:54.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.514+0000 7f6282f5f640 1 -- 172.21.15.120:0/226750324 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f627c1a50d0 con 0x7f627c100aa0 2024-09-19T09:20:54.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.517+0000 7f6279ffb640 1 -- 172.21.15.120:0/226750324 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f6268042030 con 0x7f627c100aa0 2024-09-19T09:20:54.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.517+0000 7f6285338740 1 Processor -- start 2024-09-19T09:20:54.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.517+0000 7f6285338740 1 -- start start 2024-09-19T09:20:54.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.517+0000 7f6285338740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b1905e620 0x559b1905ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:54.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.517+0000 7f6285338740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b1905ef40 0x559b1906d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:54.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.517+0000 7f6285338740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1906dd50 0x559b19070140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:54.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.517+0000 7f6285338740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559b18fc42a0 con 0x559b1905ef40 2024-09-19T09:20:54.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.517+0000 7f6285338740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559b18f6bb10 con 0x559b1905e620 2024-09-19T09:20:54.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.517+0000 7f6285338740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559b18fc0260 con 0x559b1906dd50 2024-09-19T09:20:54.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.517+0000 7f6280cfb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b1905ef40 0x559b1906d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:54.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.517+0000 7f62814fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b1905e620 0x559b1905ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:54.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.517+0000 7f6280cfb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b1905ef40 0x559b1906d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41464/0 (socket says 172.21.15.120:41464) 2024-09-19T09:20:54.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.517+0000 7f6280cfb640 1 -- 172.21.15.120:0/924349269 learned_addr learned my addr 172.21.15.120:0/924349269 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:54.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.517+0000 7f627bfff640 1 --2- 172.21.15.120:0/924349269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1906dd50 0x559b19070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:54.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.518+0000 7f6279ffb640 1 --2- 172.21.15.120:0/226750324 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f626c0a3240 0x7f626c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:54.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.518+0000 7f6280cfb640 1 -- 172.21.15.120:0/924349269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1906dd50 msgr2=0x559b19070140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:54.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.518+0000 7f6280cfb640 1 --2- 172.21.15.120:0/924349269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1906dd50 0x559b19070140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:54.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.518+0000 7f6280cfb640 1 -- 172.21.15.120:0/924349269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b1905e620 msgr2=0x559b1905ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:54.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.518+0000 7f6280cfb640 1 --2- 172.21.15.120:0/924349269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b1905e620 0x559b1905ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:54.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.518+0000 7f6280cfb640 1 -- 172.21.15.120:0/924349269 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b18d35b90 con 0x559b1905ef40 2024-09-19T09:20:54.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.518+0000 7f62814fc640 1 --2- 172.21.15.120:0/226750324 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f626c0a3240 0x7f626c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:54.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.518+0000 7f6279ffb640 1 -- 172.21.15.120:0/226750324 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f62680efef0 con 0x7f627c100aa0 2024-09-19T09:20:54.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.518+0000 7f6280cfb640 1 --2- 172.21.15.120:0/924349269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b1905ef40 0x559b1906d810 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f6270007850 tx=0x7f6270007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27789a921ff60ffd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:54.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.518+0000 7f6266ffd640 1 -- 172.21.15.120:0/924349269 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627000d950 con 0x559b1905ef40 2024-09-19T09:20:54.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.518+0000 7f6266ffd640 1 -- 172.21.15.120:0/924349269 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6270018070 con 0x559b1905ef40 2024-09-19T09:20:54.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.518+0000 7f6266ffd640 1 -- 172.21.15.120:0/924349269 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62700123e0 con 0x559b1905ef40 2024-09-19T09:20:54.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.519+0000 7f6285338740 1 -- 172.21.15.120:0/924349269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b1905ef40 msgr2=0x559b1906d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:54.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.519+0000 7f6285338740 1 --2- 172.21.15.120:0/924349269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b1905ef40 0x559b1906d810 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f6270007850 tx=0x7f6270007d10 comp rx=0 tx=0).stop 2024-09-19T09:20:54.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.519+0000 7f62814fc640 1 --2- 172.21.15.120:0/226750324 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f626c0a3240 0x7f626c0a5700 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f627c1077c0 tx=0x7f6274004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:54.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.519+0000 7f6285338740 1 -- 172.21.15.120:0/924349269 shutdown_connections 2024-09-19T09:20:54.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.519+0000 7f6285338740 1 --2- 172.21.15.120:0/924349269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1906dd50 0x559b19070140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:54.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.519+0000 7f6285338740 1 --2- 172.21.15.120:0/924349269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b1905ef40 0x559b1906d810 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:54.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.519+0000 7f6285338740 1 --2- 172.21.15.120:0/924349269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b1905e620 0x559b1905ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:54.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.519+0000 7f6285338740 1 -- 172.21.15.120:0/924349269 >> 172.21.15.120:0/924349269 conn(0x559b1905acf0 msgr2=0x559b1905b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:54.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.519+0000 7f6285338740 1 -- 172.21.15.120:0/924349269 shutdown_connections 2024-09-19T09:20:54.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.519+0000 7f6285338740 1 -- 172.21.15.120:0/924349269 wait complete. 2024-09-19T09:20:54.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.520+0000 7f6285338740 1 -- start start 2024-09-19T09:20:54.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.520+0000 7f6285338740 1 Processor -- start 2024-09-19T09:20:54.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.520+0000 7f6285338740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b1905e620 0x559b1906aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:54.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.520+0000 7f6285338740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1906dd50 0x559b1906b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:54.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.520+0000 7f6285338740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b1906b930 0x559b19079dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:54.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.520+0000 7f6285338740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559b19070920 con 0x559b1906b930 2024-09-19T09:20:54.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.520+0000 7f6285338740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559b18f4c6e0 con 0x559b1905e620 2024-09-19T09:20:54.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.520+0000 7f6285338740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559b19070aa0 con 0x559b1906dd50 2024-09-19T09:20:54.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.520+0000 7f6285338740 10 client.0 _ll_register_callbacks cb 0x559b18f70a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:20:54.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.520+0000 7f62814fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b1906b930 0x559b19079dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:54.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.520+0000 7f627bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1906dd50 0x559b1906b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:54.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.520+0000 7f62814fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b1906b930 0x559b19079dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41478/0 (socket says 172.21.15.120:41478) 2024-09-19T09:20:54.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.520+0000 7f62814fc640 1 -- 172.21.15.120:0/3532640526 learned_addr learned my addr 172.21.15.120:0/3532640526 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:54.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.521+0000 7f6280cfb640 1 --2- 172.21.15.120:0/3532640526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b1905e620 0x559b1906aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:54.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.521+0000 7f62814fc640 1 -- 172.21.15.120:0/3532640526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1906dd50 msgr2=0x559b1906b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:54.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.521+0000 7f62814fc640 1 --2- 172.21.15.120:0/3532640526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1906dd50 0x559b1906b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:54.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.521+0000 7f62814fc640 1 -- 172.21.15.120:0/3532640526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b1905e620 msgr2=0x559b1906aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:54.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.521+0000 7f62814fc640 1 --2- 172.21.15.120:0/3532640526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b1905e620 0x559b1906aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:54.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.521+0000 7f62814fc640 1 -- 172.21.15.120:0/3532640526 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b1907a490 con 0x559b1906b930 2024-09-19T09:20:54.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.521+0000 7f62814fc640 1 --2- 172.21.15.120:0/3532640526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b1906b930 0x559b19079dd0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f6274003ae0 tx=0x7f6274004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a22966e4110f50 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:54.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.521+0000 7f625bfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:20:54.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.521+0000 7f625bfff640 1 -- 172.21.15.120:0/3532640526 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627401a040 con 0x559b1906b930 2024-09-19T09:20:54.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.521+0000 7f625bfff640 1 -- 172.21.15.120:0/3532640526 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f6274004440 con 0x559b1906b930 2024-09-19T09:20:54.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.521+0000 7f6285338740 1 -- 172.21.15.120:0/3532640526 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6270018070 con 0x559b1906b930 2024-09-19T09:20:54.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.521+0000 7f625bfff640 1 -- 172.21.15.120:0/3532640526 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6274004640 con 0x559b1906b930 2024-09-19T09:20:54.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.522+0000 7f6285338740 10 client.21468 fetch_fsmap learned FSMap version 39 2024-09-19T09:20:54.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.522+0000 7f6285338740 1 -- 172.21.15.120:0/3532640526 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559b1907cea0 con 0x559b1906b930 2024-09-19T09:20:54.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.522+0000 7f625bfff640 1 -- 172.21.15.120:0/3532640526 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6270018070 con 0x559b1906b930 2024-09-19T09:20:54.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.522+0000 7f625bfff640 1 -- 172.21.15.120:0/3532640526 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f6274029400 con 0x559b1906b930 2024-09-19T09:20:54.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.522+0000 7f6285338740 10 client.21468 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:20:54.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.522+0000 7f6285338740 10 client.21468 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:20:54.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.522+0000 7f6285338740 20 client.21468 populate_metadata read hostname 'smithi120' 2024-09-19T09:20:54.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.522+0000 7f6285338740 1 --2- 172.21.15.120:0/3532640526 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559b1907e360 0x559b19080790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:54.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.522+0000 7f6285338740 4 client.21468 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:20:54.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.522+0000 7f6285338740 1 -- 172.21.15.120:0/3532640526 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b18fd19d0 con 0x559b1907e360 2024-09-19T09:20:54.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.522+0000 7f6280cfb640 1 --2- 172.21.15.120:0/3532640526 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559b1907e360 0x559b19080790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:54.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.523+0000 7f6280cfb640 1 --2- 172.21.15.120:0/3532640526 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559b1907e360 0x559b19080790 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9fde23bae2bebe9 server_cookie=56742a856f57fa5e in_seq=0 out_seq=0 2024-09-19T09:20:54.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.523+0000 7f625bfff640 10 client.21468 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:54.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.526+0000 7f625bfff640 1 -- 172.21.15.120:0/3532640526 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559b18fd19d0 con 0x559b1907e360 2024-09-19T09:20:54.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.526+0000 7f625bfff640 10 client.21468 handle_command_reply: tid=0 2024-09-19T09:20:54.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.526+0000 7f6285338740 1 client.21468 shutdown 2024-09-19T09:20:54.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.526+0000 7f6285338740 1 -- 172.21.15.120:0/3532640526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b1906b930 msgr2=0x559b19079dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:54.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.526+0000 7f6285338740 1 --2- 172.21.15.120:0/3532640526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b1906b930 0x559b19079dd0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f6274003ae0 tx=0x7f6274004a40 comp rx=0 tx=0).stop 2024-09-19T09:20:54.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.527+0000 7f6285338740 1 -- 172.21.15.120:0/3532640526 shutdown_connections 2024-09-19T09:20:54.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.527+0000 7f6285338740 1 --2- 172.21.15.120:0/3532640526 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559b1907e360 0x559b19080790 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:54.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.527+0000 7f6285338740 1 --2- 172.21.15.120:0/3532640526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b1906b930 0x559b19079dd0 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:54.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.527+0000 7f625bfff640 0 client.21468 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:54.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.527+0000 7f6285338740 1 --2- 172.21.15.120:0/3532640526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1906dd50 0x559b1906b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:54.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.527+0000 7f6285338740 1 --2- 172.21.15.120:0/3532640526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b1905e620 0x559b1906aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:54.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.527+0000 7f6285338740 1 -- 172.21.15.120:0/3532640526 >> 172.21.15.120:0/3532640526 conn(0x559b1905acf0 msgr2=0x559b19060a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:54.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.527+0000 7f6285338740 1 -- 172.21.15.120:0/3532640526 shutdown_connections 2024-09-19T09:20:54.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.527+0000 7f6285338740 1 -- 172.21.15.120:0/3532640526 wait complete. 2024-09-19T09:20:54.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.527+0000 7f6285338740 20 client.21468 trim_cache size 0 max 16384 2024-09-19T09:20:54.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.545+0000 7f6285338740 1 Processor -- start 2024-09-19T09:20:54.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.545+0000 7f6285338740 1 -- start start 2024-09-19T09:20:54.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.545+0000 7f6285338740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1905e620 0x559b19079bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:54.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.545+0000 7f6285338740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b1906b930 0x559b1906ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:54.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.545+0000 7f6285338740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b1906dd50 0x559b19086010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:54.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.545+0000 7f6285338740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f627c1ab3f0 con 0x559b1906dd50 2024-09-19T09:20:54.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.545+0000 7f6285338740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f627c1acf30 con 0x559b1906b930 2024-09-19T09:20:54.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.545+0000 7f6285338740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f627c1abac0 con 0x559b1905e620 2024-09-19T09:20:54.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.546+0000 7f62814fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b1906dd50 0x559b19086010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:54.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.546+0000 7f627bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b1906b930 0x559b1906ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:54.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.546+0000 7f62814fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b1906dd50 0x559b19086010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41490/0 (socket says 172.21.15.120:41490) 2024-09-19T09:20:54.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.546+0000 7f62814fc640 1 -- 172.21.15.120:0/1627312651 learned_addr learned my addr 172.21.15.120:0/1627312651 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:54.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.546+0000 7f6280cfb640 1 --2- 172.21.15.120:0/1627312651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1905e620 0x559b19079bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:54.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.546+0000 7f62814fc640 1 -- 172.21.15.120:0/1627312651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1905e620 msgr2=0x559b19079bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:54.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.546+0000 7f62814fc640 1 --2- 172.21.15.120:0/1627312651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1905e620 0x559b19079bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:54.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.546+0000 7f62814fc640 1 -- 172.21.15.120:0/1627312651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b1906b930 msgr2=0x559b1906ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:54.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.546+0000 7f62814fc640 1 --2- 172.21.15.120:0/1627312651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b1906b930 0x559b1906ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:54.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.546+0000 7f62814fc640 1 -- 172.21.15.120:0/1627312651 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b19056260 con 0x559b1906dd50 2024-09-19T09:20:54.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.546+0000 7f62814fc640 1 --2- 172.21.15.120:0/1627312651 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b1906dd50 0x559b19086010 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f6274029820 tx=0x7f627400fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e60a09de21f73245 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:54.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.546+0000 7f62667fc640 1 -- 172.21.15.120:0/1627312651 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627401a070 con 0x559b1906dd50 2024-09-19T09:20:54.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.546+0000 7f62667fc640 1 -- 172.21.15.120:0/1627312651 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6274015040 con 0x559b1906dd50 2024-09-19T09:20:54.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.546+0000 7f62667fc640 1 -- 172.21.15.120:0/1627312651 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6274004240 con 0x559b1906dd50 2024-09-19T09:20:54.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.547+0000 7f6285338740 1 -- 172.21.15.120:0/1627312651 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b1906dd50 msgr2=0x559b19086010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:54.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.547+0000 7f6285338740 1 --2- 172.21.15.120:0/1627312651 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b1906dd50 0x559b19086010 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f6274029820 tx=0x7f627400fc80 comp rx=0 tx=0).stop 2024-09-19T09:20:54.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.547+0000 7f6285338740 1 -- 172.21.15.120:0/1627312651 shutdown_connections 2024-09-19T09:20:54.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.547+0000 7f6285338740 1 --2- 172.21.15.120:0/1627312651 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b1906dd50 0x559b19086010 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:54.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.547+0000 7f6285338740 1 --2- 172.21.15.120:0/1627312651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b1906b930 0x559b1906ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:54.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.547+0000 7f6285338740 1 --2- 172.21.15.120:0/1627312651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1905e620 0x559b19079bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:54.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.547+0000 7f6285338740 1 -- 172.21.15.120:0/1627312651 >> 172.21.15.120:0/1627312651 conn(0x559b1905acf0 msgr2=0x559b1906c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:54.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.547+0000 7f6285338740 1 -- 172.21.15.120:0/1627312651 shutdown_connections 2024-09-19T09:20:54.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.547+0000 7f6285338740 1 -- 172.21.15.120:0/1627312651 wait complete. 2024-09-19T09:20:54.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.547+0000 7f6285338740 1 -- start start 2024-09-19T09:20:54.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.547+0000 7f6285338740 1 Processor -- start 2024-09-19T09:20:54.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.548+0000 7f6285338740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b1905e620 0x559b19071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:54.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.548+0000 7f6285338740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1906b930 0x559b19072360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:54.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.548+0000 7f6285338740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b1906dd50 0x559b190728c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:54.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.548+0000 7f6285338740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f627c1ac860 con 0x559b1906dd50 2024-09-19T09:20:54.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.548+0000 7f6285338740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f627c1ac190 con 0x559b1905e620 2024-09-19T09:20:54.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.548+0000 7f6285338740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f627c1a93b0 con 0x559b1906b930 2024-09-19T09:20:54.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.548+0000 7f6285338740 10 client.0 _ll_register_callbacks cb 0x7f6274003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:20:54.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.548+0000 7f62814fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b1906dd50 0x559b190728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:54.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.548+0000 7f6280cfb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b1905e620 0x559b19071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:54.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.548+0000 7f6280cfb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b1905e620 0x559b19071e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:53686/0 (socket says 172.21.15.120:53686) 2024-09-19T09:20:54.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.548+0000 7f6280cfb640 1 -- 172.21.15.120:0/1328656441 learned_addr learned my addr 172.21.15.120:0/1328656441 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:20:54.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.548+0000 7f627bfff640 1 --2- 172.21.15.120:0/1328656441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1906b930 0x559b19072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:54.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.548+0000 7f62814fc640 1 -- 172.21.15.120:0/1328656441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1906b930 msgr2=0x559b19072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:54.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.548+0000 7f62814fc640 1 --2- 172.21.15.120:0/1328656441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1906b930 0x559b19072360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:54.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.548+0000 7f62814fc640 1 -- 172.21.15.120:0/1328656441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b1905e620 msgr2=0x559b19071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:20:54.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.548+0000 7f62814fc640 1 --2- 172.21.15.120:0/1328656441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b1905e620 0x559b19071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:54.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.548+0000 7f62814fc640 1 -- 172.21.15.120:0/1328656441 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b190e82b0 con 0x559b1906dd50 2024-09-19T09:20:54.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.548+0000 7f6280cfb640 1 --2- 172.21.15.120:0/1328656441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b1905e620 0x559b19071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:20:54.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.549+0000 7f62814fc640 1 --2- 172.21.15.120:0/1328656441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b1906dd50 0x559b190728c0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f6274000f80 tx=0x7f6274029c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f6f88ea6951ad2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:54.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.549+0000 7f625bfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:20:54.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.549+0000 7f625bfff640 1 -- 172.21.15.120:0/1328656441 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627401a040 con 0x559b1906dd50 2024-09-19T09:20:54.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.549+0000 7f625bfff640 1 -- 172.21.15.120:0/1328656441 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f6274004240 con 0x559b1906dd50 2024-09-19T09:20:54.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.549+0000 7f6285338740 1 -- 172.21.15.120:0/1328656441 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6274015040 con 0x559b1906dd50 2024-09-19T09:20:54.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.549+0000 7f625bfff640 1 -- 172.21.15.120:0/1328656441 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6274029600 con 0x559b1906dd50 2024-09-19T09:20:54.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.549+0000 7f6285338740 10 client.21474 fetch_fsmap learned FSMap version 39 2024-09-19T09:20:54.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.549+0000 7f6285338740 1 -- 172.21.15.120:0/1328656441 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559b190e9590 con 0x559b1906dd50 2024-09-19T09:20:54.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.549+0000 7f625bfff640 1 -- 172.21.15.120:0/1328656441 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6274015040 con 0x559b1906dd50 2024-09-19T09:20:54.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.549+0000 7f625bfff640 1 -- 172.21.15.120:0/1328656441 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f6274005590 con 0x559b1906dd50 2024-09-19T09:20:54.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.549+0000 7f6285338740 10 client.21474 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:20:54.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.549+0000 7f6285338740 10 client.21474 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:20:54.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.549+0000 7f6285338740 20 client.21474 populate_metadata read hostname 'smithi120' 2024-09-19T09:20:54.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.549+0000 7f6285338740 1 --2- 172.21.15.120:0/1328656441 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559b190e9c60 0x559b190ec050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:54.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.549+0000 7f6285338740 4 client.21474 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.1, "await_for": 45.0}] 2024-09-19T09:20:54.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.549+0000 7f6285338740 1 -- 172.21.15.120:0/1328656441 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.1, "await_for": 45.0}) -- 0x559b18fcbd20 con 0x559b190e9c60 2024-09-19T09:20:54.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.549+0000 7f6280cfb640 1 --2- 172.21.15.120:0/1328656441 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559b190e9c60 0x559b190ec050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:54.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.550+0000 7f625bfff640 10 client.21474 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:54.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:54.550+0000 7f6280cfb640 1 --2- 172.21.15.120:0/1328656441 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559b190e9c60 0x559b190ec050 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=628323cd953c2f4 server_cookie=d3ac594661869a6 in_seq=0 out_seq=0 2024-09-19T09:20:55.368 INFO:tasks.workunit.client.0.smithi120.stdout: 10 125017 21.92 MB/sec execute 170 sec latency 814.674 ms 2024-09-19T09:20:55.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:55.372+0000 7f625bfff640 1 -- 172.21.15.120:0/1328656441 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559b18fcbd20 con 0x559b190e9c60 2024-09-19T09:20:55.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:55.372+0000 7f625bfff640 10 client.21474 handle_command_reply: tid=0 2024-09-19T09:20:55.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:55.372+0000 7f6285338740 1 client.21474 shutdown 2024-09-19T09:20:55.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:55.372+0000 7f6285338740 1 -- 172.21.15.120:0/1328656441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b1906dd50 msgr2=0x559b190728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:55.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:55.372+0000 7f6285338740 1 --2- 172.21.15.120:0/1328656441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b1906dd50 0x559b190728c0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f6274000f80 tx=0x7f6274029c40 comp rx=0 tx=0).stop 2024-09-19T09:20:55.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:55.372+0000 7f6285338740 1 -- 172.21.15.120:0/1328656441 shutdown_connections 2024-09-19T09:20:55.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:55.372+0000 7f6285338740 1 --2- 172.21.15.120:0/1328656441 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559b190e9c60 0x559b190ec050 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:55.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:55.372+0000 7f6285338740 1 --2- 172.21.15.120:0/1328656441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b1906dd50 0x559b190728c0 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:55.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:55.372+0000 7f6285338740 1 --2- 172.21.15.120:0/1328656441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1906b930 0x559b19072360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:55.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:55.372+0000 7f6285338740 1 --2- 172.21.15.120:0/1328656441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b1905e620 0x559b19071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:55.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:55.372+0000 7f6285338740 1 -- 172.21.15.120:0/1328656441 >> 172.21.15.120:0/1328656441 conn(0x559b1905acf0 msgr2=0x559b1906cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:55.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:55.372+0000 7f625bfff640 0 client.21474 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:20:55.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:55.373+0000 7f6285338740 1 -- 172.21.15.120:0/1328656441 shutdown_connections 2024-09-19T09:20:55.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:55.373+0000 7f6285338740 1 -- 172.21.15.120:0/1328656441 wait complete. 2024-09-19T09:20:55.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:55.373+0000 7f6285338740 20 client.21474 trim_cache size 0 max 16384 2024-09-19T09:20:55.375 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:20:55.375 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 39, 2024-09-19T09:20:55.375 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:20:55.375 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 58, 2024-09-19T09:20:55.375 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:20:55.375 INFO:teuthology.orchestra.run.smithi120.stdout: "3cc5cd89": { 2024-09-19T09:20:55.375 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 58, 2024-09-19T09:20:55.376 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:20:55.376 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:20:55.376 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:20:55.376 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:20:55.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:20:55.376 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:20:55.376 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 137.1, 2024-09-19T09:20:55.376 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:20:55.376 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:20:55.376 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:20:55.376 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:20:55.376 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:20:55.376 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:20:55.376 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:20:55.377 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:20:55.377 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:20:55.377 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:20:55.377 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:20:55.377 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:20:55.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:55.373+0000 7f6282f5f640 1 -- 172.21.15.120:0/226750324 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f626c0a3240 msgr2=0x7f626c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:55.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:55.373+0000 7f6282f5f640 1 --2- 172.21.15.120:0/226750324 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f626c0a3240 0x7f626c0a5700 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f627c1077c0 tx=0x7f6274004ab0 comp rx=0 tx=0).stop 2024-09-19T09:20:55.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:55.373+0000 7f6282f5f640 1 -- 172.21.15.120:0/226750324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f627c100aa0 msgr2=0x7f627c198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:55.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:55.373+0000 7f6282f5f640 1 --2- 172.21.15.120:0/226750324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f627c100aa0 0x7f627c198a90 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f6268006010 tx=0x7f62680042f0 comp rx=0 tx=0).stop 2024-09-19T09:20:55.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:55.373+0000 7f6282f5f640 1 -- 172.21.15.120:0/226750324 shutdown_connections 2024-09-19T09:20:55.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:55.373+0000 7f6282f5f640 1 --2- 172.21.15.120:0/226750324 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f626c0a3240 0x7f626c0a5700 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:55.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:55.373+0000 7f6282f5f640 1 --2- 172.21.15.120:0/226750324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f627c106b80 0x7f627c19ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:55.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:55.373+0000 7f6282f5f640 1 --2- 172.21.15.120:0/226750324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f627c100aa0 0x7f627c198a90 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:55.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:55.373+0000 7f6282f5f640 1 --2- 172.21.15.120:0/226750324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f627c100100 0x7f627c198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:55.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:55.373+0000 7f6282f5f640 1 -- 172.21.15.120:0/226750324 >> 172.21.15.120:0/226750324 conn(0x7f627c077230 msgr2=0x7f627c104b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:55.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:55.373+0000 7f6282f5f640 1 -- 172.21.15.120:0/226750324 shutdown_connections 2024-09-19T09:20:55.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:20:55.373+0000 7f6282f5f640 1 -- 172.21.15.120:0/226750324 wait complete. 2024-09-19T09:20:56.214 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3cc5cd89', quiesce took 4.7 seconds. Will release after: 12.400000000000002 2024-09-19T09:20:56.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:56 smithi120 ceph-mon[31150]: pgmap v887: 97 pgs: 97 active+clean; 381 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 8.3 MiB/s wr, 785 op/s 2024-09-19T09:20:56.368 INFO:tasks.workunit.client.0.smithi120.stdout: 10 125021 21.80 MB/sec execute 171 sec latency 1814.755 ms 2024-09-19T09:20:56.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:56 smithi196 ceph-mon[35150]: pgmap v887: 97 pgs: 97 active+clean; 381 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 8.3 MiB/s wr, 785 op/s 2024-09-19T09:20:56.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:56 smithi153 ceph-mon[35071]: pgmap v887: 97 pgs: 97 active+clean; 381 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 8.3 MiB/s wr, 785 op/s 2024-09-19T09:20:57.368 INFO:tasks.workunit.client.0.smithi120.stdout: 10 125021 21.67 MB/sec execute 172 sec latency 2814.833 ms 2024-09-19T09:20:58.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:20:58 smithi120 ceph-mon[31150]: pgmap v888: 97 pgs: 97 active+clean; 382 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 8.1 MiB/s wr, 797 op/s 2024-09-19T09:20:58.368 INFO:tasks.workunit.client.0.smithi120.stdout: 10 125021 21.54 MB/sec execute 173 sec latency 3814.956 ms 2024-09-19T09:20:58.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:20:58 smithi196 ceph-mon[35150]: pgmap v888: 97 pgs: 97 active+clean; 382 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 8.1 MiB/s wr, 797 op/s 2024-09-19T09:20:58.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:20:58 smithi153 ceph-mon[35071]: pgmap v888: 97 pgs: 97 active+clean; 382 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 8.1 MiB/s wr, 797 op/s 2024-09-19T09:20:59.368 INFO:tasks.workunit.client.0.smithi120.stdout: 10 125021 21.42 MB/sec execute 174 sec latency 4815.056 ms 2024-09-19T09:21:00.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:00 smithi196 ceph-mon[35150]: pgmap v889: 97 pgs: 97 active+clean; 385 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.8 MiB/s wr, 728 op/s 2024-09-19T09:21:00.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:00 smithi120 ceph-mon[31150]: pgmap v889: 97 pgs: 97 active+clean; 385 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.8 MiB/s wr, 728 op/s 2024-09-19T09:21:00.368 INFO:tasks.workunit.client.0.smithi120.stdout: 10 125021 21.30 MB/sec execute 175 sec latency 5815.179 ms 2024-09-19T09:21:00.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:00 smithi153 ceph-mon[35071]: pgmap v889: 97 pgs: 97 active+clean; 385 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.8 MiB/s wr, 728 op/s 2024-09-19T09:21:01.368 INFO:tasks.workunit.client.0.smithi120.stdout: 10 125021 21.18 MB/sec execute 176 sec latency 6815.228 ms 2024-09-19T09:21:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.768+0000 7efe5ee2e640 1 -- 172.21.15.120:0/2031589992 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe58100a50 msgr2=0x7efe58100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.768+0000 7efe5ee2e640 1 --2- 172.21.15.120:0/2031589992 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe58100a50 0x7efe58100e50 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7efe4c009900 tx=0x7efe4c02a410 comp rx=0 tx=0).stop 2024-09-19T09:21:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.768+0000 7efe5ee2e640 1 -- 172.21.15.120:0/2031589992 shutdown_connections 2024-09-19T09:21:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.768+0000 7efe5ee2e640 1 --2- 172.21.15.120:0/2031589992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe58102600 0x7efe58108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.768+0000 7efe5ee2e640 1 --2- 172.21.15.120:0/2031589992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe58101c40 0x7efe581020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.768+0000 7efe5ee2e640 1 --2- 172.21.15.120:0/2031589992 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe58100a50 0x7efe58100e50 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.768+0000 7efe5ee2e640 1 -- 172.21.15.120:0/2031589992 >> 172.21.15.120:0/2031589992 conn(0x7efe580fc1b0 msgr2=0x7efe580fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.768+0000 7efe5ee2e640 1 -- 172.21.15.120:0/2031589992 shutdown_connections 2024-09-19T09:21:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.768+0000 7efe5ee2e640 1 -- 172.21.15.120:0/2031589992 wait complete. 2024-09-19T09:21:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.768+0000 7efe5ee2e640 1 Processor -- start 2024-09-19T09:21:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.768+0000 7efe5ee2e640 1 -- start start 2024-09-19T09:21:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.768+0000 7efe5ee2e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe58100a50 0x7efe58197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.769+0000 7efe5ee2e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe58101c40 0x7efe581982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.769+0000 7efe5ee2e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe58102600 0x7efe5819f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.769+0000 7efe5ee2e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efe5810d7e0 con 0x7efe58100a50 2024-09-19T09:21:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.769+0000 7efe5ee2e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efe5810d660 con 0x7efe58102600 2024-09-19T09:21:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.769+0000 7efe5ee2e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efe5810d960 con 0x7efe58101c40 2024-09-19T09:21:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.769+0000 7efe5cbca640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe58100a50 0x7efe58197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.769+0000 7efe5cbca640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe58100a50 0x7efe58197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55716/0 (socket says 172.21.15.120:55716) 2024-09-19T09:21:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.769+0000 7efe5cbca640 1 -- 172.21.15.120:0/395411279 learned_addr learned my addr 172.21.15.120:0/395411279 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.769+0000 7efe57fff640 1 --2- 172.21.15.120:0/395411279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe58101c40 0x7efe581982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.769+0000 7efe5d3cb640 1 --2- 172.21.15.120:0/395411279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe58102600 0x7efe5819f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.769+0000 7efe5cbca640 1 -- 172.21.15.120:0/395411279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe58101c40 msgr2=0x7efe581982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.769+0000 7efe5cbca640 1 --2- 172.21.15.120:0/395411279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe58101c40 0x7efe581982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.769+0000 7efe5cbca640 1 -- 172.21.15.120:0/395411279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe58102600 msgr2=0x7efe5819f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.769+0000 7efe5cbca640 1 --2- 172.21.15.120:0/395411279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe58102600 0x7efe5819f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.769+0000 7efe5cbca640 1 -- 172.21.15.120:0/395411279 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5819f9c0 con 0x7efe58100a50 2024-09-19T09:21:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.769+0000 7efe5d3cb640 1 --2- 172.21.15.120:0/395411279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe58102600 0x7efe5819f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:21:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.769+0000 7efe5cbca640 1 --2- 172.21.15.120:0/395411279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe58100a50 0x7efe58197db0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7efe4c031610 tx=0x7efe4c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.769+0000 7efe55ffb640 1 -- 172.21.15.120:0/395411279 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c03f030 con 0x7efe58100a50 2024-09-19T09:21:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.769+0000 7efe55ffb640 1 -- 172.21.15.120:0/395411279 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe4c003ce0 con 0x7efe58100a50 2024-09-19T09:21:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.769+0000 7efe5ee2e640 1 -- 172.21.15.120:0/395411279 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe5819fc50 con 0x7efe58100a50 2024-09-19T09:21:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.769+0000 7efe55ffb640 1 -- 172.21.15.120:0/395411279 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c035070 con 0x7efe58100a50 2024-09-19T09:21:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.770+0000 7efe5ee2e640 1 -- 172.21.15.120:0/395411279 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe581a0140 con 0x7efe58100a50 2024-09-19T09:21:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.772+0000 7efe55ffb640 1 -- 172.21.15.120:0/395411279 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7efe4c042030 con 0x7efe58100a50 2024-09-19T09:21:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.773+0000 7efe61bcd740 1 Processor -- start 2024-09-19T09:21:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.773+0000 7efe61bcd740 1 -- start start 2024-09-19T09:21:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.773+0000 7efe61bcd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6e4ecadb0 0x55b6e4ecb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.773+0000 7efe61bcd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6e4ecb6f0 0x55b6e4ed01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.773+0000 7efe61bcd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6e4ed0780 0x55b6e4ed2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.773+0000 7efe61bcd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b6e4e2a2f0 con 0x55b6e4ecadb0 2024-09-19T09:21:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.773+0000 7efe61bcd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b6e4dd2270 con 0x55b6e4ed0780 2024-09-19T09:21:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.773+0000 7efe61bcd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b6e4e262b0 con 0x55b6e4ecb6f0 2024-09-19T09:21:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.773+0000 7efe5d3cb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6e4ecadb0 0x55b6e4ecb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.773+0000 7efe5d3cb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6e4ecadb0 0x55b6e4ecb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55720/0 (socket says 172.21.15.120:55720) 2024-09-19T09:21:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.773+0000 7efe5d3cb640 1 -- 172.21.15.120:0/2176259672 learned_addr learned my addr 172.21.15.120:0/2176259672 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.773+0000 7efe5cbca640 1 --2- 172.21.15.120:0/2176259672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6e4ecb6f0 0x55b6e4ed01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.773+0000 7efe55ffb640 1 --2- 172.21.15.120:0/395411279 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efe480a32f0 0x7efe480a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.773+0000 7efe57fff640 1 --2- 172.21.15.120:0/2176259672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6e4ed0780 0x55b6e4ed2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.773+0000 7efe5d3cb640 1 -- 172.21.15.120:0/2176259672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6e4ecb6f0 msgr2=0x55b6e4ed01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.773+0000 7efe5d3cb640 1 --2- 172.21.15.120:0/2176259672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6e4ecb6f0 0x55b6e4ed01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.773+0000 7efe5d3cb640 1 -- 172.21.15.120:0/2176259672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6e4ed0780 msgr2=0x55b6e4ed2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.773+0000 7efe5d3cb640 1 --2- 172.21.15.120:0/2176259672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6e4ed0780 0x55b6e4ed2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.773+0000 7efe5d3cb640 1 -- 172.21.15.120:0/2176259672 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6e4dd1340 con 0x55b6e4ecadb0 2024-09-19T09:21:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.773+0000 7efe57fff640 1 --2- 172.21.15.120:0/2176259672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6e4ed0780 0x55b6e4ed2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:21:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.773+0000 7efe55ffb640 1 -- 172.21.15.120:0/395411279 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7efe4c0eff40 con 0x7efe58100a50 2024-09-19T09:21:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.773+0000 7efe5d3cb640 1 --2- 172.21.15.120:0/395411279 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efe480a32f0 0x7efe480a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.774+0000 7efe5d3cb640 1 --2- 172.21.15.120:0/2176259672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6e4ecadb0 0x55b6e4ecb1b0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7efe5000d340 tx=0x7efe5000d800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a39ca3be731e85e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.774+0000 7efe42ffd640 1 -- 172.21.15.120:0/2176259672 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5001a920 con 0x55b6e4ecadb0 2024-09-19T09:21:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.774+0000 7efe42ffd640 1 -- 172.21.15.120:0/2176259672 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe50019410 con 0x55b6e4ecadb0 2024-09-19T09:21:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.774+0000 7efe42ffd640 1 -- 172.21.15.120:0/2176259672 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe50018600 con 0x55b6e4ecadb0 2024-09-19T09:21:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.774+0000 7efe61bcd740 1 -- 172.21.15.120:0/2176259672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6e4ecadb0 msgr2=0x55b6e4ecb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.774+0000 7efe5d3cb640 1 --2- 172.21.15.120:0/395411279 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efe480a32f0 0x7efe480a57b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7efe5001e000 tx=0x7efe5001f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.774+0000 7efe61bcd740 1 --2- 172.21.15.120:0/2176259672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6e4ecadb0 0x55b6e4ecb1b0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7efe5000d340 tx=0x7efe5000d800 comp rx=0 tx=0).stop 2024-09-19T09:21:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.775+0000 7efe61bcd740 1 -- 172.21.15.120:0/2176259672 shutdown_connections 2024-09-19T09:21:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.775+0000 7efe61bcd740 1 --2- 172.21.15.120:0/2176259672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6e4ed0780 0x55b6e4ed2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.775+0000 7efe61bcd740 1 --2- 172.21.15.120:0/2176259672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6e4ecb6f0 0x55b6e4ed01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.775+0000 7efe61bcd740 1 --2- 172.21.15.120:0/2176259672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6e4ecadb0 0x55b6e4ecb1b0 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.775+0000 7efe61bcd740 1 -- 172.21.15.120:0/2176259672 >> 172.21.15.120:0/2176259672 conn(0x55b6e4ec2010 msgr2=0x55b6e4ec2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.775+0000 7efe61bcd740 1 -- 172.21.15.120:0/2176259672 shutdown_connections 2024-09-19T09:21:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.775+0000 7efe61bcd740 1 -- 172.21.15.120:0/2176259672 wait complete. 2024-09-19T09:21:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.775+0000 7efe61bcd740 1 -- start start 2024-09-19T09:21:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.775+0000 7efe61bcd740 1 Processor -- start 2024-09-19T09:21:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.775+0000 7efe61bcd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6e4ecadb0 0x55b6e4ed4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.775+0000 7efe61bcd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6e4ecb6f0 0x55b6e4ed5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.775+0000 7efe61bcd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6e4ed0780 0x55b6e4edc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.775+0000 7efe61bcd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b6e4ed3750 con 0x55b6e4ed0780 2024-09-19T09:21:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.775+0000 7efe61bcd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b6e4db3290 con 0x55b6e4ecb6f0 2024-09-19T09:21:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.775+0000 7efe61bcd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b6e4ed38d0 con 0x55b6e4ecadb0 2024-09-19T09:21:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.775+0000 7efe61bcd740 10 client.0 _ll_register_callbacks cb 0x55b6e4de4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.775+0000 7efe5d3cb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6e4ed0780 0x55b6e4edc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.775+0000 7efe5d3cb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6e4ed0780 0x55b6e4edc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55722/0 (socket says 172.21.15.120:55722) 2024-09-19T09:21:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.775+0000 7efe5d3cb640 1 -- 172.21.15.120:0/2807915409 learned_addr learned my addr 172.21.15.120:0/2807915409 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.776+0000 7efe57fff640 1 --2- 172.21.15.120:0/2807915409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6e4ecb6f0 0x55b6e4ed5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.776+0000 7efe5cbca640 1 --2- 172.21.15.120:0/2807915409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6e4ecadb0 0x55b6e4ed4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.776+0000 7efe5d3cb640 1 -- 172.21.15.120:0/2807915409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6e4ecadb0 msgr2=0x55b6e4ed4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.776+0000 7efe5d3cb640 1 --2- 172.21.15.120:0/2807915409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6e4ecadb0 0x55b6e4ed4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.776+0000 7efe5d3cb640 1 -- 172.21.15.120:0/2807915409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6e4ecb6f0 msgr2=0x55b6e4ed5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.776+0000 7efe5d3cb640 1 --2- 172.21.15.120:0/2807915409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6e4ecb6f0 0x55b6e4ed5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.776+0000 7efe5d3cb640 1 -- 172.21.15.120:0/2807915409 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6e4b3d210 con 0x55b6e4ed0780 2024-09-19T09:21:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.776+0000 7efe5d3cb640 1 --2- 172.21.15.120:0/2807915409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6e4ed0780 0x55b6e4edc550 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7efe5000dd00 tx=0x7efe50008460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e98bb418ff8e9bec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.776+0000 7efe37fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:21:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.776+0000 7efe37fff640 1 -- 172.21.15.120:0/2807915409 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe50011560 con 0x55b6e4ed0780 2024-09-19T09:21:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.776+0000 7efe37fff640 1 -- 172.21.15.120:0/2807915409 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7efe50020040 con 0x55b6e4ed0780 2024-09-19T09:21:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.776+0000 7efe61bcd740 1 -- 172.21.15.120:0/2807915409 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe50019410 con 0x55b6e4ed0780 2024-09-19T09:21:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.776+0000 7efe37fff640 1 -- 172.21.15.120:0/2807915409 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7efe50008ae0 con 0x55b6e4ed0780 2024-09-19T09:21:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.776+0000 7efe61bcd740 10 client.21486 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.776+0000 7efe61bcd740 1 -- 172.21.15.120:0/2807915409 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b6e4c09620 con 0x55b6e4ed0780 2024-09-19T09:21:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.776+0000 7efe37fff640 1 -- 172.21.15.120:0/2807915409 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe50019410 con 0x55b6e4ed0780 2024-09-19T09:21:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.777+0000 7efe37fff640 1 -- 172.21.15.120:0/2807915409 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7efe500186e0 con 0x55b6e4ed0780 2024-09-19T09:21:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.777+0000 7efe61bcd740 10 client.21486 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.777+0000 7efe61bcd740 10 client.21486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:21:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.777+0000 7efe61bcd740 20 client.21486 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.777+0000 7efe61bcd740 1 --2- 172.21.15.120:0/2807915409 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b6e4ee09a0 0x55b6e4ee2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.777+0000 7efe61bcd740 4 client.21486 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:21:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.777+0000 7efe61bcd740 1 -- 172.21.15.120:0/2807915409 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6e4e37a20 con 0x55b6e4ee09a0 2024-09-19T09:21:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.777+0000 7efe5cbca640 1 --2- 172.21.15.120:0/2807915409 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b6e4ee09a0 0x55b6e4ee2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.777+0000 7efe37fff640 10 client.21486 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.777+0000 7efe5cbca640 1 --2- 172.21.15.120:0/2807915409 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b6e4ee09a0 0x55b6e4ee2e00 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6228cb35bfc8cae5 server_cookie=1b13a75fe254df2d in_seq=0 out_seq=0 2024-09-19T09:21:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.778+0000 7efe37fff640 1 -- 172.21.15.120:0/2807915409 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b6e4e37a20 con 0x55b6e4ee09a0 2024-09-19T09:21:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.778+0000 7efe37fff640 10 client.21486 handle_command_reply: tid=0 2024-09-19T09:21:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.778+0000 7efe61bcd740 1 client.21486 shutdown 2024-09-19T09:21:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.778+0000 7efe61bcd740 1 -- 172.21.15.120:0/2807915409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6e4ed0780 msgr2=0x55b6e4edc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.778+0000 7efe61bcd740 1 --2- 172.21.15.120:0/2807915409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6e4ed0780 0x55b6e4edc550 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7efe5000dd00 tx=0x7efe50008460 comp rx=0 tx=0).stop 2024-09-19T09:21:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.778+0000 7efe61bcd740 1 -- 172.21.15.120:0/2807915409 shutdown_connections 2024-09-19T09:21:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.778+0000 7efe61bcd740 1 --2- 172.21.15.120:0/2807915409 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b6e4ee09a0 0x55b6e4ee2e00 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.778+0000 7efe61bcd740 1 --2- 172.21.15.120:0/2807915409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6e4ed0780 0x55b6e4edc550 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.778+0000 7efe37fff640 0 client.21486 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.778+0000 7efe61bcd740 1 --2- 172.21.15.120:0/2807915409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6e4ecb6f0 0x55b6e4ed5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.778+0000 7efe61bcd740 1 --2- 172.21.15.120:0/2807915409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6e4ecadb0 0x55b6e4ed4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.778+0000 7efe61bcd740 1 -- 172.21.15.120:0/2807915409 >> 172.21.15.120:0/2807915409 conn(0x55b6e4ec2010 msgr2=0x55b6e4ec3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.778+0000 7efe61bcd740 1 -- 172.21.15.120:0/2807915409 shutdown_connections 2024-09-19T09:21:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.779+0000 7efe61bcd740 1 -- 172.21.15.120:0/2807915409 wait complete. 2024-09-19T09:21:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.779+0000 7efe61bcd740 20 client.21486 trim_cache size 0 max 16384 2024-09-19T09:21:01.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.793+0000 7efe61bcd740 1 Processor -- start 2024-09-19T09:21:01.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.793+0000 7efe61bcd740 1 -- start start 2024-09-19T09:21:01.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.793+0000 7efe61bcd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6e4ecadb0 0x55b6e4ec10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.793+0000 7efe61bcd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6e4ecb6f0 0x55b6e4edc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.793+0000 7efe61bcd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6e4ed0780 0x55b6e4ee76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.793+0000 7efe61bcd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efe581a6da0 con 0x55b6e4ed0780 2024-09-19T09:21:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.793+0000 7efe61bcd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efe581a88e0 con 0x55b6e4ecadb0 2024-09-19T09:21:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.793+0000 7efe61bcd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efe581a7470 con 0x55b6e4ecb6f0 2024-09-19T09:21:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.793+0000 7efe5d3cb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6e4ed0780 0x55b6e4ee76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.793+0000 7efe5d3cb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6e4ed0780 0x55b6e4ee76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55732/0 (socket says 172.21.15.120:55732) 2024-09-19T09:21:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.793+0000 7efe5d3cb640 1 -- 172.21.15.120:0/1579192893 learned_addr learned my addr 172.21.15.120:0/1579192893 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.793+0000 7efe57fff640 1 --2- 172.21.15.120:0/1579192893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6e4ecb6f0 0x55b6e4edc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.793+0000 7efe5cbca640 1 --2- 172.21.15.120:0/1579192893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6e4ecadb0 0x55b6e4ec10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.793+0000 7efe5d3cb640 1 -- 172.21.15.120:0/1579192893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6e4ecb6f0 msgr2=0x55b6e4edc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.793+0000 7efe5d3cb640 1 --2- 172.21.15.120:0/1579192893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6e4ecb6f0 0x55b6e4edc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:01.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.793+0000 7efe5d3cb640 1 -- 172.21.15.120:0/1579192893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6e4ecadb0 msgr2=0x55b6e4ec10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:01.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.793+0000 7efe5d3cb640 1 --2- 172.21.15.120:0/1579192893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6e4ecadb0 0x55b6e4ec10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:01.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.793+0000 7efe5d3cb640 1 -- 172.21.15.120:0/1579192893 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6e4ed3a70 con 0x55b6e4ed0780 2024-09-19T09:21:01.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.793+0000 7efe5cbca640 1 --2- 172.21.15.120:0/1579192893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6e4ecadb0 0x55b6e4ec10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:21:01.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.793+0000 7efe5d3cb640 1 --2- 172.21.15.120:0/1579192893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6e4ed0780 0x55b6e4ee76c0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7efe50018ad0 tx=0x7efe500088e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e60c1f4eb3922b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:01.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.794+0000 7efe427fc640 1 -- 172.21.15.120:0/1579192893 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe50019410 con 0x55b6e4ed0780 2024-09-19T09:21:01.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.794+0000 7efe427fc640 1 -- 172.21.15.120:0/1579192893 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe5001ae00 con 0x55b6e4ed0780 2024-09-19T09:21:01.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.794+0000 7efe427fc640 1 -- 172.21.15.120:0/1579192893 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe500185c0 con 0x55b6e4ed0780 2024-09-19T09:21:01.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.794+0000 7efe61bcd740 1 -- 172.21.15.120:0/1579192893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6e4ed0780 msgr2=0x55b6e4ee76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:01.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.794+0000 7efe61bcd740 1 --2- 172.21.15.120:0/1579192893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6e4ed0780 0x55b6e4ee76c0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7efe50018ad0 tx=0x7efe500088e0 comp rx=0 tx=0).stop 2024-09-19T09:21:01.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.794+0000 7efe61bcd740 1 -- 172.21.15.120:0/1579192893 shutdown_connections 2024-09-19T09:21:01.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.794+0000 7efe61bcd740 1 --2- 172.21.15.120:0/1579192893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6e4ed0780 0x55b6e4ee76c0 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:01.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.794+0000 7efe61bcd740 1 --2- 172.21.15.120:0/1579192893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6e4ecb6f0 0x55b6e4edc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.794+0000 7efe61bcd740 1 --2- 172.21.15.120:0/1579192893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6e4ecadb0 0x55b6e4ec10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.794+0000 7efe61bcd740 1 -- 172.21.15.120:0/1579192893 >> 172.21.15.120:0/1579192893 conn(0x55b6e4ec2010 msgr2=0x55b6e4ec09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.794+0000 7efe61bcd740 1 -- 172.21.15.120:0/1579192893 shutdown_connections 2024-09-19T09:21:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.794+0000 7efe61bcd740 1 -- 172.21.15.120:0/1579192893 wait complete. 2024-09-19T09:21:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.794+0000 7efe61bcd740 1 -- start start 2024-09-19T09:21:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.795+0000 7efe61bcd740 1 Processor -- start 2024-09-19T09:21:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.795+0000 7efe61bcd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6e4ecadb0 0x55b6e4ed9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.795+0000 7efe61bcd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6e4ecb6f0 0x55b6e4eda150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.795+0000 7efe61bcd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6e4ed0780 0x55b6e4edb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.795+0000 7efe61bcd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efe581a8210 con 0x55b6e4ecb6f0 2024-09-19T09:21:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.795+0000 7efe61bcd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efe581a7b40 con 0x55b6e4ecadb0 2024-09-19T09:21:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.795+0000 7efe61bcd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efe581a4d60 con 0x55b6e4ed0780 2024-09-19T09:21:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.795+0000 7efe61bcd740 10 client.0 _ll_register_callbacks cb 0x7efe5001d020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.795+0000 7efe57fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6e4ecb6f0 0x55b6e4eda150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.795+0000 7efe57fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6e4ecb6f0 0x55b6e4eda150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55746/0 (socket says 172.21.15.120:55746) 2024-09-19T09:21:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.795+0000 7efe57fff640 1 -- 172.21.15.120:0/234897976 learned_addr learned my addr 172.21.15.120:0/234897976 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.795+0000 7efe5d3cb640 1 --2- 172.21.15.120:0/234897976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6e4ed0780 0x55b6e4edb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.795+0000 7efe5cbca640 1 --2- 172.21.15.120:0/234897976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6e4ecadb0 0x55b6e4ed9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.795+0000 7efe57fff640 1 -- 172.21.15.120:0/234897976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6e4ed0780 msgr2=0x55b6e4edb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.795+0000 7efe57fff640 1 --2- 172.21.15.120:0/234897976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6e4ed0780 0x55b6e4edb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.795+0000 7efe57fff640 1 -- 172.21.15.120:0/234897976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6e4ecadb0 msgr2=0x55b6e4ed9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.795+0000 7efe57fff640 1 --2- 172.21.15.120:0/234897976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6e4ecadb0 0x55b6e4ed9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.795+0000 7efe57fff640 1 -- 172.21.15.120:0/234897976 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6e4ebd2d0 con 0x55b6e4ecb6f0 2024-09-19T09:21:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.795+0000 7efe57fff640 1 --2- 172.21.15.120:0/234897976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6e4ecb6f0 0x55b6e4eda150 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7efe4400ba80 tx=0x7efe4400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792a34b11f2bd457 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.795+0000 7efe37fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:21:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.796+0000 7efe37fff640 1 -- 172.21.15.120:0/234897976 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44005220 con 0x55b6e4ecb6f0 2024-09-19T09:21:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.796+0000 7efe37fff640 1 -- 172.21.15.120:0/234897976 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7efe440053c0 con 0x55b6e4ecb6f0 2024-09-19T09:21:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.796+0000 7efe61bcd740 1 -- 172.21.15.120:0/234897976 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe5001ae00 con 0x55b6e4ecb6f0 2024-09-19T09:21:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.796+0000 7efe37fff640 1 -- 172.21.15.120:0/234897976 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7efe44003e60 con 0x55b6e4ecb6f0 2024-09-19T09:21:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.796+0000 7efe61bcd740 10 client.21492 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.796+0000 7efe61bcd740 1 -- 172.21.15.120:0/234897976 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b6e4f4b6a0 con 0x55b6e4ecb6f0 2024-09-19T09:21:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.796+0000 7efe37fff640 1 -- 172.21.15.120:0/234897976 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe5001ae00 con 0x55b6e4ecb6f0 2024-09-19T09:21:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.796+0000 7efe37fff640 1 -- 172.21.15.120:0/234897976 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7efe4401b070 con 0x55b6e4ecb6f0 2024-09-19T09:21:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.796+0000 7efe61bcd740 10 client.21492 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.796+0000 7efe61bcd740 10 client.21492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:21:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.796+0000 7efe61bcd740 20 client.21492 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.796+0000 7efe61bcd740 1 --2- 172.21.15.120:0/234897976 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b6e4f4bfe0 0x55b6e4f4e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.796+0000 7efe61bcd740 4 client.21492 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:21:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.796+0000 7efe61bcd740 1 -- 172.21.15.120:0/234897976 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b6e4e31d70 con 0x55b6e4f4bfe0 2024-09-19T09:21:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.797+0000 7efe5cbca640 1 --2- 172.21.15.120:0/234897976 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b6e4f4bfe0 0x55b6e4f4e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.797+0000 7efe37fff640 10 client.21492 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:01.797+0000 7efe5cbca640 1 --2- 172.21.15.120:0/234897976 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b6e4f4bfe0 0x55b6e4f4e400 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c5155beee653d90 server_cookie=d28ecef7f1342428 in_seq=0 out_seq=0 2024-09-19T09:21:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:02.124+0000 7efe37fff640 1 -- 172.21.15.120:0/234897976 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b6e4e31d70 con 0x55b6e4f4bfe0 2024-09-19T09:21:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:02.124+0000 7efe37fff640 10 client.21492 handle_command_reply: tid=0 2024-09-19T09:21:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:02.124+0000 7efe61bcd740 1 client.21492 shutdown 2024-09-19T09:21:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:02.124+0000 7efe61bcd740 1 -- 172.21.15.120:0/234897976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6e4ecb6f0 msgr2=0x55b6e4eda150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:02.124+0000 7efe61bcd740 1 --2- 172.21.15.120:0/234897976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6e4ecb6f0 0x55b6e4eda150 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7efe4400ba80 tx=0x7efe4400bf40 comp rx=0 tx=0).stop 2024-09-19T09:21:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:02.124+0000 7efe61bcd740 1 -- 172.21.15.120:0/234897976 shutdown_connections 2024-09-19T09:21:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:02.124+0000 7efe61bcd740 1 --2- 172.21.15.120:0/234897976 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b6e4f4bfe0 0x55b6e4f4e400 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:02.124+0000 7efe61bcd740 1 --2- 172.21.15.120:0/234897976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6e4ed0780 0x55b6e4edb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:02.124+0000 7efe37fff640 0 client.21492 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:02.124+0000 7efe61bcd740 1 --2- 172.21.15.120:0/234897976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6e4ecb6f0 0x55b6e4eda150 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:02.124+0000 7efe61bcd740 1 --2- 172.21.15.120:0/234897976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6e4ecadb0 0x55b6e4ed9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:02.124+0000 7efe61bcd740 1 -- 172.21.15.120:0/234897976 >> 172.21.15.120:0/234897976 conn(0x55b6e4ec2010 msgr2=0x55b6e4ec11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:02.125+0000 7efe61bcd740 1 -- 172.21.15.120:0/234897976 shutdown_connections 2024-09-19T09:21:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:02.125+0000 7efe61bcd740 1 -- 172.21.15.120:0/234897976 wait complete. 2024-09-19T09:21:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:02.125+0000 7efe61bcd740 20 client.21492 trim_cache size 0 max 16384 2024-09-19T09:21:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:21:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:21:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:21:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:21:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:02.125+0000 7efe5ee2e640 1 -- 172.21.15.120:0/395411279 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efe480a32f0 msgr2=0x7efe480a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:02.125+0000 7efe5ee2e640 1 --2- 172.21.15.120:0/395411279 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efe480a32f0 0x7efe480a57b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7efe5001e000 tx=0x7efe5001f000 comp rx=0 tx=0).stop 2024-09-19T09:21:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:02.125+0000 7efe5ee2e640 1 -- 172.21.15.120:0/395411279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe58100a50 msgr2=0x7efe58197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:02.125+0000 7efe5ee2e640 1 --2- 172.21.15.120:0/395411279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe58100a50 0x7efe58197db0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7efe4c031610 tx=0x7efe4c0043b0 comp rx=0 tx=0).stop 2024-09-19T09:21:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:02.125+0000 7efe5ee2e640 1 -- 172.21.15.120:0/395411279 shutdown_connections 2024-09-19T09:21:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:02.125+0000 7efe5ee2e640 1 --2- 172.21.15.120:0/395411279 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7efe480a32f0 0x7efe480a57b0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:02.125+0000 7efe5ee2e640 1 --2- 172.21.15.120:0/395411279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe58102600 0x7efe5819f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:02.125+0000 7efe5ee2e640 1 --2- 172.21.15.120:0/395411279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe58101c40 0x7efe581982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:02.125+0000 7efe5ee2e640 1 --2- 172.21.15.120:0/395411279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe58100a50 0x7efe58197db0 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:02.125+0000 7efe5ee2e640 1 -- 172.21.15.120:0/395411279 >> 172.21.15.120:0/395411279 conn(0x7efe580fc1b0 msgr2=0x7efe58104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:02.125+0000 7efe5ee2e640 1 -- 172.21.15.120:0/395411279 shutdown_connections 2024-09-19T09:21:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:02.125+0000 7efe5ee2e640 1 -- 172.21.15.120:0/395411279 wait complete. 2024-09-19T09:21:02.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:02 smithi120 ceph-mon[31150]: pgmap v890: 97 pgs: 97 active+clean; 396 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.0 MiB/s wr, 568 op/s 2024-09-19T09:21:02.368 INFO:tasks.workunit.client.0.smithi120.stdout: 10 125021 21.06 MB/sec execute 177 sec latency 7815.311 ms 2024-09-19T09:21:02.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:02 smithi196 ceph-mon[35150]: pgmap v890: 97 pgs: 97 active+clean; 396 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.0 MiB/s wr, 568 op/s 2024-09-19T09:21:02.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:02 smithi153 ceph-mon[35071]: pgmap v890: 97 pgs: 97 active+clean; 396 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.0 MiB/s wr, 568 op/s 2024-09-19T09:21:03.368 INFO:tasks.workunit.client.0.smithi120.stdout: 10 125021 20.94 MB/sec execute 178 sec latency 8815.387 ms 2024-09-19T09:21:04.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:04 smithi120 ceph-mon[31150]: pgmap v891: 97 pgs: 97 active+clean; 386 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.9 MiB/s wr, 489 op/s 2024-09-19T09:21:04.369 INFO:tasks.workunit.client.0.smithi120.stdout: 10 125021 20.82 MB/sec execute 179 sec latency 9815.494 ms 2024-09-19T09:21:04.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:04 smithi196 ceph-mon[35150]: pgmap v891: 97 pgs: 97 active+clean; 386 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.9 MiB/s wr, 489 op/s 2024-09-19T09:21:04.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:04 smithi153 ceph-mon[35071]: pgmap v891: 97 pgs: 97 active+clean; 386 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.9 MiB/s wr, 489 op/s 2024-09-19T09:21:05.369 INFO:tasks.workunit.client.0.smithi120.stdout: 10 125021 20.71 MB/sec execute 180 sec latency 10815.567 ms 2024-09-19T09:21:06.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:06 smithi120 ceph-mon[31150]: pgmap v892: 97 pgs: 97 active+clean; 382 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.4 MiB/s wr, 338 op/s 2024-09-19T09:21:06.369 INFO:tasks.workunit.client.0.smithi120.stdout: 10 125021 20.59 MB/sec execute 181 sec latency 11815.621 ms 2024-09-19T09:21:06.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:06 smithi196 ceph-mon[35150]: pgmap v892: 97 pgs: 97 active+clean; 382 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.4 MiB/s wr, 338 op/s 2024-09-19T09:21:06.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:06 smithi153 ceph-mon[35071]: pgmap v892: 97 pgs: 97 active+clean; 382 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.4 MiB/s wr, 338 op/s 2024-09-19T09:21:07.369 INFO:tasks.workunit.client.0.smithi120.stdout: 10 125021 20.48 MB/sec execute 182 sec latency 12815.701 ms 2024-09-19T09:21:08.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:08 smithi120 ceph-mon[31150]: pgmap v893: 97 pgs: 97 active+clean; 375 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.3 MiB/s wr, 387 op/s 2024-09-19T09:21:08.369 INFO:tasks.workunit.client.0.smithi120.stdout: 10 125021 20.37 MB/sec execute 183 sec latency 13815.796 ms 2024-09-19T09:21:08.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:08 smithi196 ceph-mon[35150]: pgmap v893: 97 pgs: 97 active+clean; 375 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.3 MiB/s wr, 387 op/s 2024-09-19T09:21:08.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:08 smithi153 ceph-mon[35071]: pgmap v893: 97 pgs: 97 active+clean; 375 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.3 MiB/s wr, 387 op/s 2024-09-19T09:21:08.616 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:21:08.870 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:21:09.369 INFO:tasks.workunit.client.0.smithi120.stdout: 10 125021 20.26 MB/sec execute 184 sec latency 14815.841 ms 2024-09-19T09:21:09.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.414+0000 7f05cf41f640 1 -- 172.21.15.120:0/3440215303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05c8102d40 msgr2=0x7f05c8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:09.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.414+0000 7f05cf41f640 1 --2- 172.21.15.120:0/3440215303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05c8102d40 0x7f05c8103120 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f05c4009900 tx=0x7f05c402a480 comp rx=0 tx=0).stop 2024-09-19T09:21:09.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.414+0000 7f05cf41f640 1 -- 172.21.15.120:0/3440215303 shutdown_connections 2024-09-19T09:21:09.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.414+0000 7f05cf41f640 1 --2- 172.21.15.120:0/3440215303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05c810c170 0x7f05c810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:09.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.414+0000 7f05cf41f640 1 --2- 172.21.15.120:0/3440215303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05c8103660 0x7f05c810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:09.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.414+0000 7f05cf41f640 1 --2- 172.21.15.120:0/3440215303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05c8102d40 0x7f05c8103120 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:09.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.414+0000 7f05cf41f640 1 -- 172.21.15.120:0/3440215303 >> 172.21.15.120:0/3440215303 conn(0x7f05c80fc7a0 msgr2=0x7f05c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:09.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.414+0000 7f05cf41f640 1 -- 172.21.15.120:0/3440215303 shutdown_connections 2024-09-19T09:21:09.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.414+0000 7f05cf41f640 1 -- 172.21.15.120:0/3440215303 wait complete. 2024-09-19T09:21:09.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.415+0000 7f05cf41f640 1 Processor -- start 2024-09-19T09:21:09.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.415+0000 7f05cf41f640 1 -- start start 2024-09-19T09:21:09.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.415+0000 7f05cf41f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05c8102d40 0x7f05c8198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:09.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.415+0000 7f05cf41f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05c8103660 0x7f05c81989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:09.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.415+0000 7f05cf41f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05c810c170 0x7f05c819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:09.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.415+0000 7f05cf41f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f05c8111f70 con 0x7f05c8102d40 2024-09-19T09:21:09.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.415+0000 7f05cf41f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f05c8111df0 con 0x7f05c810c170 2024-09-19T09:21:09.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.415+0000 7f05cf41f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f05c81120f0 con 0x7f05c8103660 2024-09-19T09:21:09.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.415+0000 7f05cd1bb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05c8102d40 0x7f05c8198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:09.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.415+0000 7f05cd1bb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05c8102d40 0x7f05c8198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55766/0 (socket says 172.21.15.120:55766) 2024-09-19T09:21:09.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.416+0000 7f05cd1bb640 1 -- 172.21.15.120:0/2002288458 learned_addr learned my addr 172.21.15.120:0/2002288458 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:09.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.416+0000 7f05cd9bc640 1 --2- 172.21.15.120:0/2002288458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05c810c170 0x7f05c819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:09.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.416+0000 7f05cc9ba640 1 --2- 172.21.15.120:0/2002288458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05c8103660 0x7f05c81989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:09.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.416+0000 7f05cd1bb640 1 -- 172.21.15.120:0/2002288458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05c8103660 msgr2=0x7f05c81989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:09.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.416+0000 7f05cd1bb640 1 --2- 172.21.15.120:0/2002288458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05c8103660 0x7f05c81989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:09.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.416+0000 7f05cd1bb640 1 -- 172.21.15.120:0/2002288458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05c810c170 msgr2=0x7f05c819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:09.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.416+0000 7f05cd1bb640 1 --2- 172.21.15.120:0/2002288458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05c810c170 0x7f05c819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:09.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.416+0000 7f05cd1bb640 1 -- 172.21.15.120:0/2002288458 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c819d420 con 0x7f05c8102d40 2024-09-19T09:21:09.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.416+0000 7f05cd1bb640 1 --2- 172.21.15.120:0/2002288458 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05c8102d40 0x7f05c8198470 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f05c4006010 tx=0x7f05c40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:09.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.416+0000 7f05be7fc640 1 -- 172.21.15.120:0/2002288458 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c403f030 con 0x7f05c8102d40 2024-09-19T09:21:09.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.416+0000 7f05be7fc640 1 -- 172.21.15.120:0/2002288458 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05c4003ce0 con 0x7f05c8102d40 2024-09-19T09:21:09.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.416+0000 7f05cf41f640 1 -- 172.21.15.120:0/2002288458 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c819d6b0 con 0x7f05c8102d40 2024-09-19T09:21:09.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.416+0000 7f05be7fc640 1 -- 172.21.15.120:0/2002288458 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c4035070 con 0x7f05c8102d40 2024-09-19T09:21:09.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.416+0000 7f05cf41f640 1 -- 172.21.15.120:0/2002288458 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c81a4ff0 con 0x7f05c8102d40 2024-09-19T09:21:09.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.417+0000 7f05cf41f640 1 -- 172.21.15.120:0/2002288458 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05c806b070 con 0x7f05c8102d40 2024-09-19T09:21:09.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.418+0000 7f05be7fc640 1 -- 172.21.15.120:0/2002288458 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f05c4042030 con 0x7f05c8102d40 2024-09-19T09:21:09.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.419+0000 7f05be7fc640 1 --2- 172.21.15.120:0/2002288458 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f05b40a3240 0x7f05b40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:09.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.419+0000 7f05cc9ba640 1 --2- 172.21.15.120:0/2002288458 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f05b40a3240 0x7f05b40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:09.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.419+0000 7f05be7fc640 1 -- 172.21.15.120:0/2002288458 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f05c40efe20 con 0x7f05c8102d40 2024-09-19T09:21:09.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.419+0000 7f05cc9ba640 1 --2- 172.21.15.120:0/2002288458 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f05b40a3240 0x7f05b40a5700 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f05b80058c0 tx=0x7f05b8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:09.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.423+0000 7f05be7fc640 1 -- 172.21.15.120:0/2002288458 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f05c40bcc00 con 0x7f05c8102d40 2024-09-19T09:21:09.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.624+0000 7f05cf41f640 1 -- 172.21.15.120:0/2002288458 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05c81071e0 con 0x7f05c8102d40 2024-09-19T09:21:09.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.625+0000 7f05be7fc640 1 -- 172.21.15.120:0/2002288458 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11988 (secure 0 0 0) 0x7f05c40bc500 con 0x7f05c8102d40 2024-09-19T09:21:09.626 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:21:09.626 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":39,"btime":"2024-09-19T09:18:53:120858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:18:53.070639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:21:09.626 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 39 2024-09-19T09:21:09.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.627+0000 7f05cf41f640 1 -- 172.21.15.120:0/2002288458 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f05b40a3240 msgr2=0x7f05b40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:09.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.627+0000 7f05cf41f640 1 --2- 172.21.15.120:0/2002288458 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f05b40a3240 0x7f05b40a5700 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f05b80058c0 tx=0x7f05b8005850 comp rx=0 tx=0).stop 2024-09-19T09:21:09.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.627+0000 7f05cf41f640 1 -- 172.21.15.120:0/2002288458 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05c8102d40 msgr2=0x7f05c8198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:09.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.627+0000 7f05cf41f640 1 --2- 172.21.15.120:0/2002288458 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05c8102d40 0x7f05c8198470 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f05c4006010 tx=0x7f05c40042f0 comp rx=0 tx=0).stop 2024-09-19T09:21:09.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.627+0000 7f05cf41f640 1 -- 172.21.15.120:0/2002288458 shutdown_connections 2024-09-19T09:21:09.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.627+0000 7f05cf41f640 1 --2- 172.21.15.120:0/2002288458 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f05b40a3240 0x7f05b40a5700 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:09.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.627+0000 7f05cf41f640 1 --2- 172.21.15.120:0/2002288458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05c810c170 0x7f05c819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:09.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.627+0000 7f05cf41f640 1 --2- 172.21.15.120:0/2002288458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05c8103660 0x7f05c81989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:09.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.627+0000 7f05cf41f640 1 --2- 172.21.15.120:0/2002288458 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05c8102d40 0x7f05c8198470 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:09.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.627+0000 7f05cf41f640 1 -- 172.21.15.120:0/2002288458 >> 172.21.15.120:0/2002288458 conn(0x7f05c80fc7a0 msgr2=0x7f05c80fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:09.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.627+0000 7f05cf41f640 1 -- 172.21.15.120:0/2002288458 shutdown_connections 2024-09-19T09:21:09.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:09.627+0000 7f05cf41f640 1 -- 172.21.15.120:0/2002288458 wait complete. 2024-09-19T09:21:09.912 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:09 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2002288458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:21:09.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:09 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2002288458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:21:09.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:09 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2002288458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:21:10.369 INFO:tasks.workunit.client.0.smithi120.stdout: 10 125021 20.15 MB/sec execute 185 sec latency 15815.876 ms 2024-09-19T09:21:10.513 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db --set-id 3cc5cd89 --release --await' 2024-09-19T09:21:10.513 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db --set-id 3cc5cd89 --release --await 2024-09-19T09:21:10.769 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:21:10.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:10 smithi120 ceph-mon[31150]: pgmap v894: 97 pgs: 97 active+clean; 383 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 5.6 MiB/s wr, 347 op/s 2024-09-19T09:21:10.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:10 smithi196 ceph-mon[35150]: pgmap v894: 97 pgs: 97 active+clean; 383 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 5.6 MiB/s wr, 347 op/s 2024-09-19T09:21:10.940 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:10 smithi153 ceph-mon[35071]: pgmap v894: 97 pgs: 97 active+clean; 383 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 5.6 MiB/s wr, 347 op/s 2024-09-19T09:21:11.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.336+0000 7f6dffcd4640 1 -- 172.21.15.120:0/2603143921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6df8102d40 msgr2=0x7f6df8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:11.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.336+0000 7f6dffcd4640 1 --2- 172.21.15.120:0/2603143921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6df8102d40 0x7f6df8103120 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f6df4009ca0 tx=0x7f6df402a950 comp rx=0 tx=0).stop 2024-09-19T09:21:11.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.336+0000 7f6dffcd4640 1 -- 172.21.15.120:0/2603143921 shutdown_connections 2024-09-19T09:21:11.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.336+0000 7f6dffcd4640 1 --2- 172.21.15.120:0/2603143921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6df810c170 0x7f6df810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.336+0000 7f6dffcd4640 1 --2- 172.21.15.120:0/2603143921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6df8103660 0x7f6df810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.336+0000 7f6dffcd4640 1 --2- 172.21.15.120:0/2603143921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6df8102d40 0x7f6df8103120 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.336+0000 7f6dffcd4640 1 -- 172.21.15.120:0/2603143921 >> 172.21.15.120:0/2603143921 conn(0x7f6df80fc7a0 msgr2=0x7f6df80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:11.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.336+0000 7f6dffcd4640 1 -- 172.21.15.120:0/2603143921 shutdown_connections 2024-09-19T09:21:11.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.336+0000 7f6dffcd4640 1 -- 172.21.15.120:0/2603143921 wait complete. 2024-09-19T09:21:11.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.336+0000 7f6dffcd4640 1 Processor -- start 2024-09-19T09:21:11.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.336+0000 7f6dffcd4640 1 -- start start 2024-09-19T09:21:11.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.337+0000 7f6dffcd4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6df8102d40 0x7f6df81984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:11.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.337+0000 7f6dffcd4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6df8103660 0x7f6df8198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:11.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.337+0000 7f6dffcd4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6df810c170 0x7f6df819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:11.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.337+0000 7f6dffcd4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6df8111fb0 con 0x7f6df810c170 2024-09-19T09:21:11.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.337+0000 7f6dffcd4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6df8111e30 con 0x7f6df8102d40 2024-09-19T09:21:11.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.337+0000 7f6dffcd4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6df8112130 con 0x7f6df8103660 2024-09-19T09:21:11.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.337+0000 7f6dfe271640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6df810c170 0x7f6df819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:11.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.337+0000 7f6dfe271640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6df810c170 0x7f6df819cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48680/0 (socket says 172.21.15.120:48680) 2024-09-19T09:21:11.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.337+0000 7f6dfe271640 1 -- 172.21.15.120:0/1842562168 learned_addr learned my addr 172.21.15.120:0/1842562168 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:11.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.337+0000 7f6dfda70640 1 --2- 172.21.15.120:0/1842562168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6df8102d40 0x7f6df81984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:11.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.337+0000 7f6dfd26f640 1 --2- 172.21.15.120:0/1842562168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6df8103660 0x7f6df8198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:11.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.337+0000 7f6dfe271640 1 -- 172.21.15.120:0/1842562168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6df8103660 msgr2=0x7f6df8198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:11.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.337+0000 7f6dfe271640 1 --2- 172.21.15.120:0/1842562168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6df8103660 0x7f6df8198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.337+0000 7f6dfe271640 1 -- 172.21.15.120:0/1842562168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6df8102d40 msgr2=0x7f6df81984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:11.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.337+0000 7f6dfe271640 1 --2- 172.21.15.120:0/1842562168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6df8102d40 0x7f6df81984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.337+0000 7f6dfe271640 1 -- 172.21.15.120:0/1842562168 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df819d470 con 0x7f6df810c170 2024-09-19T09:21:11.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.337+0000 7f6dfe271640 1 --2- 172.21.15.120:0/1842562168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6df810c170 0x7f6df819cd90 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f6df000c970 tx=0x7f6df000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:11.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.337+0000 7f6deeffd640 1 -- 172.21.15.120:0/1842562168 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df0007bb0 con 0x7f6df810c170 2024-09-19T09:21:11.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.337+0000 7f6deeffd640 1 -- 172.21.15.120:0/1842562168 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6df0007d50 con 0x7f6df810c170 2024-09-19T09:21:11.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.337+0000 7f6dffcd4640 1 -- 172.21.15.120:0/1842562168 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df819d760 con 0x7f6df810c170 2024-09-19T09:21:11.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.337+0000 7f6deeffd640 1 -- 172.21.15.120:0/1842562168 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df0018800 con 0x7f6df810c170 2024-09-19T09:21:11.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.337+0000 7f6dffcd4640 1 -- 172.21.15.120:0/1842562168 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df81a5040 con 0x7f6df810c170 2024-09-19T09:21:11.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.340+0000 7f6deeffd640 1 -- 172.21.15.120:0/1842562168 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f6df000fa50 con 0x7f6df810c170 2024-09-19T09:21:11.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.341+0000 7f6e020ad740 1 Processor -- start 2024-09-19T09:21:11.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.341+0000 7f6e020ad740 1 -- start start 2024-09-19T09:21:11.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.341+0000 7f6deeffd640 1 --2- 172.21.15.120:0/1842562168 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6de40a32c0 0x7f6de40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:11.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.341+0000 7f6e020ad740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5ad7eb060 0x55f5ad7eb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:11.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.341+0000 7f6e020ad740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5ad7eb980 0x55f5ad7f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:11.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.341+0000 7f6e020ad740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5ad7f6240 0x55f5ad7f8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:11.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.341+0000 7f6e020ad740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f5ad750620 con 0x55f5ad7eb980 2024-09-19T09:21:11.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.341+0000 7f6e020ad740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f5ad6f7e50 con 0x55f5ad7eb060 2024-09-19T09:21:11.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.341+0000 7f6e020ad740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f5ad74c5e0 con 0x55f5ad7f6240 2024-09-19T09:21:11.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.341+0000 7f6dfd26f640 1 --2- 172.21.15.120:0/1842562168 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6de40a32c0 0x7f6de40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:11.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.341+0000 7f6deeffd640 1 -- 172.21.15.120:0/1842562168 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f6df00cf8e0 con 0x7f6df810c170 2024-09-19T09:21:11.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.341+0000 7f6dfd26f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5ad7eb980 0x55f5ad7f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:11.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.341+0000 7f6dfd26f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5ad7eb980 0x55f5ad7f5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48694/0 (socket says 172.21.15.120:48694) 2024-09-19T09:21:11.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.341+0000 7f6dfd26f640 1 -- 172.21.15.120:0/859864079 learned_addr learned my addr 172.21.15.120:0/859864079 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:11.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.342+0000 7f6dfe271640 1 --2- 172.21.15.120:0/859864079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5ad7f6240 0x55f5ad7f8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:11.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.342+0000 7f6dfda70640 1 --2- 172.21.15.120:0/859864079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5ad7eb060 0x55f5ad7eb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:11.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.342+0000 7f6dfd26f640 1 -- 172.21.15.120:0/859864079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5ad7f6240 msgr2=0x55f5ad7f8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:11.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.342+0000 7f6dfd26f640 1 --2- 172.21.15.120:0/859864079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5ad7f6240 0x55f5ad7f8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.342+0000 7f6dfd26f640 1 -- 172.21.15.120:0/859864079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5ad7eb060 msgr2=0x55f5ad7eb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:11.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.342+0000 7f6dfd26f640 1 --2- 172.21.15.120:0/859864079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5ad7eb060 0x55f5ad7eb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.342+0000 7f6dfd26f640 1 -- 172.21.15.120:0/859864079 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5ad6e8d40 con 0x55f5ad7eb980 2024-09-19T09:21:11.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.342+0000 7f6dfd26f640 1 --2- 172.21.15.120:0/1842562168 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6de40a32c0 0x7f6de40a5780 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f6de8015410 tx=0x7f6de8009880 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:11.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.342+0000 7f6dfd26f640 1 --2- 172.21.15.120:0/859864079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5ad7eb980 0x55f5ad7f5d00 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f6de8017620 tx=0x7f6de8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebea0c2375e170e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:11.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.342+0000 7f6de3fff640 1 -- 172.21.15.120:0/859864079 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de801e070 con 0x55f5ad7eb980 2024-09-19T09:21:11.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.342+0000 7f6de3fff640 1 -- 172.21.15.120:0/859864079 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de8018b60 con 0x55f5ad7eb980 2024-09-19T09:21:11.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.342+0000 7f6de3fff640 1 -- 172.21.15.120:0/859864079 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de800f040 con 0x55f5ad7eb980 2024-09-19T09:21:11.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.343+0000 7f6e020ad740 1 -- 172.21.15.120:0/859864079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5ad7eb980 msgr2=0x55f5ad7f5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:11.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.343+0000 7f6e020ad740 1 --2- 172.21.15.120:0/859864079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5ad7eb980 0x55f5ad7f5d00 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f6de8017620 tx=0x7f6de8018000 comp rx=0 tx=0).stop 2024-09-19T09:21:11.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.343+0000 7f6e020ad740 1 -- 172.21.15.120:0/859864079 shutdown_connections 2024-09-19T09:21:11.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.343+0000 7f6e020ad740 1 --2- 172.21.15.120:0/859864079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5ad7f6240 0x55f5ad7f8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.343+0000 7f6e020ad740 1 --2- 172.21.15.120:0/859864079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5ad7eb980 0x55f5ad7f5d00 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.343+0000 7f6e020ad740 1 --2- 172.21.15.120:0/859864079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5ad7eb060 0x55f5ad7eb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.343+0000 7f6e020ad740 1 -- 172.21.15.120:0/859864079 >> 172.21.15.120:0/859864079 conn(0x55f5ad7e75f0 msgr2=0x55f5ad7e79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:11.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.343+0000 7f6e020ad740 1 -- 172.21.15.120:0/859864079 shutdown_connections 2024-09-19T09:21:11.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.343+0000 7f6e020ad740 1 -- 172.21.15.120:0/859864079 wait complete. 2024-09-19T09:21:11.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.343+0000 7f6e020ad740 1 -- start start 2024-09-19T09:21:11.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.343+0000 7f6e020ad740 1 Processor -- start 2024-09-19T09:21:11.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.344+0000 7f6e020ad740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5ad7eb060 0x55f5ad7f3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:11.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.344+0000 7f6e020ad740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5ad7eb980 0x55f5ad7f3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:11.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.344+0000 7f6e020ad740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5ad7f6240 0x55f5ad7f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:11.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.344+0000 7f6e020ad740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f5ad7f8e10 con 0x55f5ad7eb980 2024-09-19T09:21:11.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.344+0000 7f6e020ad740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f5ad6d8f10 con 0x55f5ad7eb060 2024-09-19T09:21:11.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.344+0000 7f6e020ad740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f5ad7f8f90 con 0x55f5ad7f6240 2024-09-19T09:21:11.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.344+0000 7f6e020ad740 10 client.0 _ll_register_callbacks cb 0x55f5ad6fcdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:11.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.344+0000 7f6dfd26f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5ad7eb980 0x55f5ad7f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:11.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.344+0000 7f6dfd26f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5ad7eb980 0x55f5ad7f3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48702/0 (socket says 172.21.15.120:48702) 2024-09-19T09:21:11.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.344+0000 7f6dfd26f640 1 -- 172.21.15.120:0/1934189749 learned_addr learned my addr 172.21.15.120:0/1934189749 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:11.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.344+0000 7f6dfda70640 1 --2- 172.21.15.120:0/1934189749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5ad7eb060 0x55f5ad7f3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:11.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.344+0000 7f6dfd26f640 1 -- 172.21.15.120:0/1934189749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5ad7f6240 msgr2=0x55f5ad7f3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:11.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.344+0000 7f6dfe271640 1 --2- 172.21.15.120:0/1934189749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5ad7f6240 0x55f5ad7f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:11.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.344+0000 7f6dfd26f640 1 --2- 172.21.15.120:0/1934189749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5ad7f6240 0x55f5ad7f3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.344+0000 7f6dfd26f640 1 -- 172.21.15.120:0/1934189749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5ad7eb060 msgr2=0x55f5ad7f3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:11.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.344+0000 7f6dfd26f640 1 --2- 172.21.15.120:0/1934189749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5ad7eb060 0x55f5ad7f3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.344+0000 7f6dfd26f640 1 -- 172.21.15.120:0/1934189749 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5ad45f580 con 0x55f5ad7eb980 2024-09-19T09:21:11.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.344+0000 7f6dfda70640 1 --2- 172.21.15.120:0/1934189749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5ad7eb060 0x55f5ad7f3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:21:11.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.344+0000 7f6dfe271640 1 --2- 172.21.15.120:0/1934189749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5ad7f6240 0x55f5ad7f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:21:11.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.344+0000 7f6dfd26f640 1 --2- 172.21.15.120:0/1934189749 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5ad7eb980 0x55f5ad7f3740 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f6de80235d0 tx=0x7f6de8018a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d35c2f6ac4ecdf39 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:11.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.344+0000 7f6de17fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:21:11.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.345+0000 7f6de17fa640 1 -- 172.21.15.120:0/1934189749 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de801e050 con 0x55f5ad7eb980 2024-09-19T09:21:11.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.345+0000 7f6de17fa640 1 -- 172.21.15.120:0/1934189749 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f6de8023a80 con 0x55f5ad7eb980 2024-09-19T09:21:11.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.345+0000 7f6e020ad740 1 -- 172.21.15.120:0/1934189749 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6de8018b60 con 0x55f5ad7eb980 2024-09-19T09:21:11.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.345+0000 7f6de17fa640 1 -- 172.21.15.120:0/1934189749 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6de8009a50 con 0x55f5ad7eb980 2024-09-19T09:21:11.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.345+0000 7f6e020ad740 10 client.21510 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:11.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.345+0000 7f6e020ad740 1 -- 172.21.15.120:0/1934189749 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f5ad529960 con 0x55f5ad7eb980 2024-09-19T09:21:11.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.345+0000 7f6de17fa640 1 -- 172.21.15.120:0/1934189749 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de8018b60 con 0x55f5ad7eb980 2024-09-19T09:21:11.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.345+0000 7f6de17fa640 1 -- 172.21.15.120:0/1934189749 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f6de802ea60 con 0x55f5ad7eb980 2024-09-19T09:21:11.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.345+0000 7f6e020ad740 10 client.21510 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:11.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.345+0000 7f6e020ad740 10 client.21510 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:21:11.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.345+0000 7f6e020ad740 20 client.21510 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:11.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.345+0000 7f6e020ad740 1 --2- 172.21.15.120:0/1934189749 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f5ad805d60 0x55f5ad808150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:11.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.345+0000 7f6e020ad740 4 client.21510 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:21:11.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.345+0000 7f6e020ad740 1 -- 172.21.15.120:0/1934189749 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5ad75db90 con 0x55f5ad805d60 2024-09-19T09:21:11.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.346+0000 7f6dfda70640 1 --2- 172.21.15.120:0/1934189749 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f5ad805d60 0x55f5ad808150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:11.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.346+0000 7f6de17fa640 10 client.21510 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:11.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.346+0000 7f6dfda70640 1 --2- 172.21.15.120:0/1934189749 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f5ad805d60 0x55f5ad808150 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e08f7a2ad500c843 server_cookie=2413b03380dbf1b1 in_seq=0 out_seq=0 2024-09-19T09:21:11.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.347+0000 7f6de17fa640 1 -- 172.21.15.120:0/1934189749 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f5ad75db90 con 0x55f5ad805d60 2024-09-19T09:21:11.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.347+0000 7f6de17fa640 10 client.21510 handle_command_reply: tid=0 2024-09-19T09:21:11.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.347+0000 7f6e020ad740 1 client.21510 shutdown 2024-09-19T09:21:11.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.347+0000 7f6e020ad740 1 -- 172.21.15.120:0/1934189749 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5ad7eb980 msgr2=0x55f5ad7f3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:11.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.347+0000 7f6e020ad740 1 --2- 172.21.15.120:0/1934189749 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5ad7eb980 0x55f5ad7f3740 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f6de80235d0 tx=0x7f6de8018a50 comp rx=0 tx=0).stop 2024-09-19T09:21:11.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.348+0000 7f6e020ad740 1 -- 172.21.15.120:0/1934189749 shutdown_connections 2024-09-19T09:21:11.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.348+0000 7f6e020ad740 1 --2- 172.21.15.120:0/1934189749 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f5ad805d60 0x55f5ad808150 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.348+0000 7f6e020ad740 1 --2- 172.21.15.120:0/1934189749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5ad7f6240 0x55f5ad7f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.348+0000 7f6e020ad740 1 --2- 172.21.15.120:0/1934189749 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5ad7eb980 0x55f5ad7f3740 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.348+0000 7f6de17fa640 0 client.21510 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:11.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.348+0000 7f6e020ad740 1 --2- 172.21.15.120:0/1934189749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5ad7eb060 0x55f5ad7f3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.348+0000 7f6e020ad740 1 -- 172.21.15.120:0/1934189749 >> 172.21.15.120:0/1934189749 conn(0x55f5ad7e75f0 msgr2=0x55f5ad7e6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:11.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.348+0000 7f6e020ad740 1 -- 172.21.15.120:0/1934189749 shutdown_connections 2024-09-19T09:21:11.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.348+0000 7f6e020ad740 1 -- 172.21.15.120:0/1934189749 wait complete. 2024-09-19T09:21:11.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.348+0000 7f6e020ad740 20 client.21510 trim_cache size 0 max 16384 2024-09-19T09:21:11.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.364+0000 7f6e020ad740 1 Processor -- start 2024-09-19T09:21:11.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.364+0000 7f6e020ad740 1 -- start start 2024-09-19T09:21:11.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.364+0000 7f6e020ad740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5ad7eb060 0x55f5ad7edac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:11.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.364+0000 7f6e020ad740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5ad7eb980 0x55f5ad7f20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:11.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.364+0000 7f6e020ad740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5ad7f6240 0x55f5ad80d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:11.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.364+0000 7f6e020ad740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6df81ad600 con 0x55f5ad7eb060 2024-09-19T09:21:11.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.364+0000 7f6e020ad740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6df81af140 con 0x55f5ad7f6240 2024-09-19T09:21:11.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.364+0000 7f6e020ad740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6df81adcd0 con 0x55f5ad7eb980 2024-09-19T09:21:11.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.364+0000 7f6dfda70640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5ad7eb060 0x55f5ad7edac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:11.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.364+0000 7f6dfda70640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5ad7eb060 0x55f5ad7edac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48718/0 (socket says 172.21.15.120:48718) 2024-09-19T09:21:11.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.365+0000 7f6dfda70640 1 -- 172.21.15.120:0/4082368430 learned_addr learned my addr 172.21.15.120:0/4082368430 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:11.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.365+0000 7f6dfd26f640 1 --2- 172.21.15.120:0/4082368430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5ad7eb980 0x55f5ad7f20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:11.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.365+0000 7f6dfda70640 1 -- 172.21.15.120:0/4082368430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5ad7eb980 msgr2=0x55f5ad7f20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:11.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.365+0000 7f6dfda70640 1 --2- 172.21.15.120:0/4082368430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5ad7eb980 0x55f5ad7f20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.365+0000 7f6dfda70640 1 -- 172.21.15.120:0/4082368430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5ad7f6240 msgr2=0x55f5ad80d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:11.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.365+0000 7f6dfda70640 1 --2- 172.21.15.120:0/4082368430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5ad7f6240 0x55f5ad80d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.365+0000 7f6dfda70640 1 -- 172.21.15.120:0/4082368430 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5ad7eae90 con 0x55f5ad7eb060 2024-09-19T09:21:11.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.365+0000 7f6dfda70640 1 --2- 172.21.15.120:0/4082368430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5ad7eb060 0x55f5ad7edac0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f6df4006010 tx=0x7f6df40099d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0923eca609182f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:11.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.365+0000 7f6de37fe640 1 -- 172.21.15.120:0/4082368430 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df4034040 con 0x55f5ad7eb060 2024-09-19T09:21:11.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.365+0000 7f6de37fe640 1 -- 172.21.15.120:0/4082368430 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6df4003e50 con 0x55f5ad7eb060 2024-09-19T09:21:11.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.365+0000 7f6de37fe640 1 -- 172.21.15.120:0/4082368430 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df40045d0 con 0x55f5ad7eb060 2024-09-19T09:21:11.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.366+0000 7f6e020ad740 1 -- 172.21.15.120:0/4082368430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5ad7eb060 msgr2=0x55f5ad7edac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:11.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.366+0000 7f6e020ad740 1 --2- 172.21.15.120:0/4082368430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5ad7eb060 0x55f5ad7edac0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f6df4006010 tx=0x7f6df40099d0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.366+0000 7f6e020ad740 1 -- 172.21.15.120:0/4082368430 shutdown_connections 2024-09-19T09:21:11.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.366+0000 7f6e020ad740 1 --2- 172.21.15.120:0/4082368430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5ad7f6240 0x55f5ad80d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.366+0000 7f6e020ad740 1 --2- 172.21.15.120:0/4082368430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5ad7eb980 0x55f5ad7f20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.366+0000 7f6e020ad740 1 --2- 172.21.15.120:0/4082368430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5ad7eb060 0x55f5ad7edac0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.366+0000 7f6e020ad740 1 -- 172.21.15.120:0/4082368430 >> 172.21.15.120:0/4082368430 conn(0x55f5ad7e75f0 msgr2=0x55f5ad7f19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:11.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.366+0000 7f6e020ad740 1 -- 172.21.15.120:0/4082368430 shutdown_connections 2024-09-19T09:21:11.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.366+0000 7f6e020ad740 1 -- 172.21.15.120:0/4082368430 wait complete. 2024-09-19T09:21:11.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.366+0000 7f6e020ad740 1 -- start start 2024-09-19T09:21:11.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.366+0000 7f6e020ad740 1 Processor -- start 2024-09-19T09:21:11.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.367+0000 7f6e020ad740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5ad7eb060 0x55f5ad7fe590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:11.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.367+0000 7f6e020ad740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5ad7eb980 0x55f5ad7fead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:11.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.367+0000 7f6e020ad740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5ad7f6240 0x55f5ad7fb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:11.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.367+0000 7f6e020ad740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6df81aea70 con 0x55f5ad7eb060 2024-09-19T09:21:11.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.367+0000 7f6e020ad740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6df81ae3a0 con 0x55f5ad7eb980 2024-09-19T09:21:11.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.367+0000 7f6e020ad740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6df81ab5c0 con 0x55f5ad7f6240 2024-09-19T09:21:11.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.367+0000 7f6e020ad740 10 client.0 _ll_register_callbacks cb 0x7f6de80188a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:11.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.367+0000 7f6dfda70640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5ad7eb060 0x55f5ad7fe590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:11.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.367+0000 7f6dfd26f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5ad7eb980 0x55f5ad7fead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:11.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.367+0000 7f6dfda70640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5ad7eb060 0x55f5ad7fe590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48732/0 (socket says 172.21.15.120:48732) 2024-09-19T09:21:11.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.367+0000 7f6dfda70640 1 -- 172.21.15.120:0/1244992263 learned_addr learned my addr 172.21.15.120:0/1244992263 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:11.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.367+0000 7f6dfe271640 1 --2- 172.21.15.120:0/1244992263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5ad7f6240 0x55f5ad7fb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:11.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.367+0000 7f6dfda70640 1 -- 172.21.15.120:0/1244992263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5ad7f6240 msgr2=0x55f5ad7fb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:11.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.367+0000 7f6dfda70640 1 --2- 172.21.15.120:0/1244992263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5ad7f6240 0x55f5ad7fb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.367+0000 7f6dfda70640 1 -- 172.21.15.120:0/1244992263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5ad7eb980 msgr2=0x55f5ad7fead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:11.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.367+0000 7f6dfda70640 1 --2- 172.21.15.120:0/1244992263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5ad7eb980 0x55f5ad7fead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.367+0000 7f6dfda70640 1 -- 172.21.15.120:0/1244992263 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5ad7fbbe0 con 0x55f5ad7eb060 2024-09-19T09:21:11.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.367+0000 7f6dfda70640 1 --2- 172.21.15.120:0/1244992263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5ad7eb060 0x55f5ad7fe590 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f6df402a990 tx=0x7f6df4004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903464fdfed58ae2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:11.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.367+0000 7f6de17fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:21:11.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.367+0000 7f6de17fa640 1 -- 172.21.15.120:0/1244992263 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df4034030 con 0x55f5ad7eb060 2024-09-19T09:21:11.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.367+0000 7f6de17fa640 1 -- 172.21.15.120:0/1244992263 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f6df402f3a0 con 0x55f5ad7eb060 2024-09-19T09:21:11.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.367+0000 7f6e020ad740 1 -- 172.21.15.120:0/1244992263 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6df4003e50 con 0x55f5ad7eb060 2024-09-19T09:21:11.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.368+0000 7f6de17fa640 1 -- 172.21.15.120:0/1244992263 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6df402f5a0 con 0x55f5ad7eb060 2024-09-19T09:21:11.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.368+0000 7f6e020ad740 10 client.21516 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:11.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.368+0000 7f6e020ad740 1 -- 172.21.15.120:0/1244992263 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f5ad871460 con 0x55f5ad7eb060 2024-09-19T09:21:11.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.368+0000 7f6de17fa640 1 -- 172.21.15.120:0/1244992263 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6df4003e50 con 0x55f5ad7eb060 2024-09-19T09:21:11.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.368+0000 7f6de17fa640 1 -- 172.21.15.120:0/1244992263 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f6df4046800 con 0x55f5ad7eb060 2024-09-19T09:21:11.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.368+0000 7f6e020ad740 10 client.21516 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:11.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.368+0000 7f6e020ad740 10 client.21516 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:21:11.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.368+0000 7f6e020ad740 20 client.21516 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:11.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.368+0000 7f6e020ad740 1 --2- 172.21.15.120:0/1244992263 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f5ad871be0 0x55f5ad874000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:11.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.368+0000 7f6e020ad740 4 client.21516 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3cc5cd89", "release": true, "await": true}] 2024-09-19T09:21:11.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.368+0000 7f6e020ad740 1 -- 172.21.15.120:0/1244992263 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3cc5cd89", "release": true, "await": true}) -- 0x55f5ad7580a0 con 0x55f5ad871be0 2024-09-19T09:21:11.374 INFO:tasks.workunit.client.0.smithi120.stdout: 10 125021 20.04 MB/sec execute 186 sec latency 16816.234 ms 2024-09-19T09:21:11.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.368+0000 7f6dfd26f640 1 --2- 172.21.15.120:0/1244992263 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f5ad871be0 0x55f5ad874000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:11.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.369+0000 7f6de17fa640 10 client.21516 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:11.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.369+0000 7f6dfd26f640 1 --2- 172.21.15.120:0/1244992263 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f5ad871be0 0x55f5ad874000 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51ce246aa23a60cb server_cookie=fddab8504d8f55f in_seq=0 out_seq=0 2024-09-19T09:21:11.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.467+0000 7f6de17fa640 1 -- 172.21.15.120:0/1244992263 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f5ad7580a0 con 0x55f5ad871be0 2024-09-19T09:21:11.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.467+0000 7f6de17fa640 10 client.21516 handle_command_reply: tid=0 2024-09-19T09:21:11.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.467+0000 7f6e020ad740 1 client.21516 shutdown 2024-09-19T09:21:11.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.467+0000 7f6e020ad740 1 -- 172.21.15.120:0/1244992263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5ad7eb060 msgr2=0x55f5ad7fe590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:11.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.467+0000 7f6e020ad740 1 --2- 172.21.15.120:0/1244992263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5ad7eb060 0x55f5ad7fe590 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f6df402a990 tx=0x7f6df4004250 comp rx=0 tx=0).stop 2024-09-19T09:21:11.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.467+0000 7f6e020ad740 1 -- 172.21.15.120:0/1244992263 shutdown_connections 2024-09-19T09:21:11.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.467+0000 7f6e020ad740 1 --2- 172.21.15.120:0/1244992263 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f5ad871be0 0x55f5ad874000 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.467+0000 7f6e020ad740 1 --2- 172.21.15.120:0/1244992263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5ad7f6240 0x55f5ad7fb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.467+0000 7f6de17fa640 0 client.21516 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:11.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.467+0000 7f6e020ad740 1 --2- 172.21.15.120:0/1244992263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5ad7eb980 0x55f5ad7fead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.467+0000 7f6e020ad740 1 --2- 172.21.15.120:0/1244992263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5ad7eb060 0x55f5ad7fe590 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.467+0000 7f6e020ad740 1 -- 172.21.15.120:0/1244992263 >> 172.21.15.120:0/1244992263 conn(0x55f5ad7e75f0 msgr2=0x55f5ad7f21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:11.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.467+0000 7f6e020ad740 1 -- 172.21.15.120:0/1244992263 shutdown_connections 2024-09-19T09:21:11.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.467+0000 7f6e020ad740 1 -- 172.21.15.120:0/1244992263 wait complete. 2024-09-19T09:21:11.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.468+0000 7f6e020ad740 20 client.21516 trim_cache size 0 max 16384 2024-09-19T09:21:11.470 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:21:11.470 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 39, 2024-09-19T09:21:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:21:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 60, 2024-09-19T09:21:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:21:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "3cc5cd89": { 2024-09-19T09:21:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 60, 2024-09-19T09:21:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:21:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:21:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:21:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:21:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:21:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:21:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 137.1, 2024-09-19T09:21:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:21:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:21:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:21:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:21:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:21:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:21:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:21:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:21:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:21:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:21:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:21:11.472 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:21:11.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.468+0000 7f6dffcd4640 1 -- 172.21.15.120:0/1842562168 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6de40a32c0 msgr2=0x7f6de40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:11.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.468+0000 7f6dffcd4640 1 --2- 172.21.15.120:0/1842562168 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6de40a32c0 0x7f6de40a5780 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f6de8015410 tx=0x7f6de8009880 comp rx=0 tx=0).stop 2024-09-19T09:21:11.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.468+0000 7f6dffcd4640 1 -- 172.21.15.120:0/1842562168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6df810c170 msgr2=0x7f6df819cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:11.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.468+0000 7f6dffcd4640 1 --2- 172.21.15.120:0/1842562168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6df810c170 0x7f6df819cd90 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f6df000c970 tx=0x7f6df000ce30 comp rx=0 tx=0).stop 2024-09-19T09:21:11.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.468+0000 7f6dffcd4640 1 -- 172.21.15.120:0/1842562168 shutdown_connections 2024-09-19T09:21:11.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.468+0000 7f6dffcd4640 1 --2- 172.21.15.120:0/1842562168 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6de40a32c0 0x7f6de40a5780 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.468+0000 7f6dffcd4640 1 --2- 172.21.15.120:0/1842562168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6df810c170 0x7f6df819cd90 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.468+0000 7f6dffcd4640 1 --2- 172.21.15.120:0/1842562168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6df8103660 0x7f6df8198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.468+0000 7f6dffcd4640 1 --2- 172.21.15.120:0/1842562168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6df8102d40 0x7f6df81984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:11.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.468+0000 7f6dffcd4640 1 -- 172.21.15.120:0/1842562168 >> 172.21.15.120:0/1842562168 conn(0x7f6df80fc7a0 msgr2=0x7f6df80fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:11.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.468+0000 7f6dffcd4640 1 -- 172.21.15.120:0/1842562168 shutdown_connections 2024-09-19T09:21:11.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:11.469+0000 7f6dffcd4640 1 -- 172.21.15.120:0/1842562168 wait complete. 2024-09-19T09:21:12.297 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3cc5cd89', total seconds elapsed: 20.8 2024-09-19T09:21:12.372 INFO:tasks.workunit.client.0.smithi120.stdout: 10 125540 20.08 MB/sec execute 187 sec latency 16926.106 ms 2024-09-19T09:21:12.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:12 smithi196 ceph-mon[35150]: pgmap v895: 97 pgs: 97 active+clean; 385 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 4.7 MiB/s wr, 289 op/s 2024-09-19T09:21:12.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:12 smithi153 ceph-mon[35071]: pgmap v895: 97 pgs: 97 active+clean; 385 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 4.7 MiB/s wr, 289 op/s 2024-09-19T09:21:13.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:12 smithi120 ceph-mon[31150]: pgmap v895: 97 pgs: 97 active+clean; 385 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 4.7 MiB/s wr, 289 op/s 2024-09-19T09:21:13.369 INFO:tasks.workunit.client.0.smithi120.stdout: 10 126123 20.13 MB/sec execute 188 sec latency 66.892 ms 2024-09-19T09:21:14.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:14 smithi120 ceph-mon[31150]: pgmap v896: 97 pgs: 97 active+clean; 376 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 5.4 MiB/s wr, 368 op/s 2024-09-19T09:21:14.369 INFO:tasks.workunit.client.0.smithi120.stdout: 10 126726 20.17 MB/sec execute 189 sec latency 146.319 ms 2024-09-19T09:21:14.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:14 smithi196 ceph-mon[35150]: pgmap v896: 97 pgs: 97 active+clean; 376 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 5.4 MiB/s wr, 368 op/s 2024-09-19T09:21:14.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:14 smithi153 ceph-mon[35071]: pgmap v896: 97 pgs: 97 active+clean; 376 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 5.4 MiB/s wr, 368 op/s 2024-09-19T09:21:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.244+0000 7f2dd6896640 1 -- 172.21.15.120:0/3892674853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dd0101700 msgr2=0x7f2dd0101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.244+0000 7f2dd6896640 1 --2- 172.21.15.120:0/3892674853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dd0101700 0x7f2dd0101bb0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8009ca0 tx=0x7f2dc802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.244+0000 7f2dd6896640 1 -- 172.21.15.120:0/3892674853 shutdown_connections 2024-09-19T09:21:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.244+0000 7f2dd6896640 1 --2- 172.21.15.120:0/3892674853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dd0101700 0x7f2dd0101bb0 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.244+0000 7f2dd6896640 1 --2- 172.21.15.120:0/3892674853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dd0100d60 0x7f2dd01011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.244+0000 7f2dd6896640 1 --2- 172.21.15.120:0/3892674853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2dd0100420 0x7f2dd0100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.244+0000 7f2dd6896640 1 -- 172.21.15.120:0/3892674853 >> 172.21.15.120:0/3892674853 conn(0x7f2dd00fc1b0 msgr2=0x7f2dd00fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.245+0000 7f2dd6896640 1 -- 172.21.15.120:0/3892674853 shutdown_connections 2024-09-19T09:21:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.245+0000 7f2dd6896640 1 -- 172.21.15.120:0/3892674853 wait complete. 2024-09-19T09:21:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.245+0000 7f2dd6896640 1 Processor -- start 2024-09-19T09:21:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.245+0000 7f2dd6896640 1 -- start start 2024-09-19T09:21:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.246+0000 7f2dd6896640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dd0100420 0x7f2dd0199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.246+0000 7f2dd6896640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2dd0100d60 0x7f2dd019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.246+0000 7f2dd6896640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dd0101700 0x7f2dd01a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.246+0000 7f2dd6896640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd010f9b0 con 0x7f2dd0101700 2024-09-19T09:21:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.246+0000 7f2dd6896640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd010f830 con 0x7f2dd0100420 2024-09-19T09:21:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.246+0000 7f2dd6896640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd010fb30 con 0x7f2dd0100d60 2024-09-19T09:21:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.246+0000 7f2dd4e33640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dd0101700 0x7f2dd01a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.246+0000 7f2dd4e33640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dd0101700 0x7f2dd01a1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48742/0 (socket says 172.21.15.120:48742) 2024-09-19T09:21:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.246+0000 7f2dd4e33640 1 -- 172.21.15.120:0/3080145987 learned_addr learned my addr 172.21.15.120:0/3080145987 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.246+0000 7f2dcf7fe640 1 --2- 172.21.15.120:0/3080145987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2dd0100d60 0x7f2dd019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.246+0000 7f2dcffff640 1 --2- 172.21.15.120:0/3080145987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dd0100420 0x7f2dd0199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.246+0000 7f2dd4e33640 1 -- 172.21.15.120:0/3080145987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2dd0100d60 msgr2=0x7f2dd019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.246+0000 7f2dd4e33640 1 --2- 172.21.15.120:0/3080145987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2dd0100d60 0x7f2dd019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.246+0000 7f2dd4e33640 1 -- 172.21.15.120:0/3080145987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dd0100420 msgr2=0x7f2dd0199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.246+0000 7f2dd4e33640 1 --2- 172.21.15.120:0/3080145987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dd0100420 0x7f2dd0199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.246+0000 7f2dd4e33640 1 -- 172.21.15.120:0/3080145987 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd01a1ba0 con 0x7f2dd0101700 2024-09-19T09:21:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.246+0000 7f2dd4e33640 1 --2- 172.21.15.120:0/3080145987 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dd0101700 0x7f2dd01a1530 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f2dc802ade0 tx=0x7f2dc80319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.246+0000 7f2dcd7fa640 1 -- 172.21.15.120:0/3080145987 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8003b70 con 0x7f2dd0101700 2024-09-19T09:21:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.247+0000 7f2dd6896640 1 -- 172.21.15.120:0/3080145987 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dd01a1e30 con 0x7f2dd0101700 2024-09-19T09:21:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.247+0000 7f2dcd7fa640 1 -- 172.21.15.120:0/3080145987 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dc8003d10 con 0x7f2dd0101700 2024-09-19T09:21:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.247+0000 7f2dcd7fa640 1 -- 172.21.15.120:0/3080145987 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc802fc70 con 0x7f2dd0101700 2024-09-19T09:21:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.247+0000 7f2dd6896640 1 -- 172.21.15.120:0/3080145987 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dd01a2340 con 0x7f2dd0101700 2024-09-19T09:21:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.250+0000 7f2dcd7fa640 1 -- 172.21.15.120:0/3080145987 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f2dc8004040 con 0x7f2dd0101700 2024-09-19T09:21:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.250+0000 7f2dd962e740 1 Processor -- start 2024-09-19T09:21:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.250+0000 7f2dd962e740 1 -- start start 2024-09-19T09:21:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.250+0000 7f2dd962e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638969e09a0 0x5638969e0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.250+0000 7f2dd962e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638969e12e0 0x5638969e5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.250+0000 7f2dd962e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638969e6370 0x5638969e8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.250+0000 7f2dd962e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5638969442f0 con 0x5638969e6370 2024-09-19T09:21:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.250+0000 7f2dd962e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5638968ec270 con 0x5638969e09a0 2024-09-19T09:21:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.250+0000 7f2dd962e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5638969402b0 con 0x5638969e12e0 2024-09-19T09:21:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.250+0000 7f2dcd7fa640 1 --2- 172.21.15.120:0/3080145987 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2db80a3370 0x7f2db80a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.251+0000 7f2dcf7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638969e6370 0x5638969e8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.251+0000 7f2dcd7fa640 1 -- 172.21.15.120:0/3080145987 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f2dc80f0d80 con 0x7f2dd0101700 2024-09-19T09:21:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.251+0000 7f2dcf7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638969e6370 0x5638969e8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48750/0 (socket says 172.21.15.120:48750) 2024-09-19T09:21:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.251+0000 7f2dcf7fe640 1 -- 172.21.15.120:0/4191062233 learned_addr learned my addr 172.21.15.120:0/4191062233 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.251+0000 7f2dd4e33640 1 --2- 172.21.15.120:0/3080145987 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2db80a3370 0x7f2db80a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.251+0000 7f2dcf7fe640 1 --2- 172.21.15.120:0/4191062233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638969e09a0 0x5638969e0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.251+0000 7f2dcffff640 1 --2- 172.21.15.120:0/4191062233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638969e12e0 0x5638969e5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.251+0000 7f2dcf7fe640 1 -- 172.21.15.120:0/4191062233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638969e12e0 msgr2=0x5638969e5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.251+0000 7f2dcf7fe640 1 --2- 172.21.15.120:0/4191062233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638969e12e0 0x5638969e5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.251+0000 7f2dcf7fe640 1 -- 172.21.15.120:0/4191062233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638969e6370 msgr2=0x5638969e8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.251+0000 7f2dcf7fe640 1 --2- 172.21.15.120:0/4191062233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638969e6370 0x5638969e8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.251+0000 7f2dcf7fe640 1 -- 172.21.15.120:0/4191062233 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638968eb340 con 0x5638969e09a0 2024-09-19T09:21:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.251+0000 7f2dd4e33640 1 --2- 172.21.15.120:0/3080145987 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2db80a3370 0x7f2db80a5830 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2dc803cfd0 tx=0x7f2dc802a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.251+0000 7f2dcf7fe640 1 --2- 172.21.15.120:0/4191062233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638969e09a0 0x5638969e0da0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2dc40058c0 tx=0x7f2dc4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b72bd22adf398a82 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.251+0000 7f2db67fc640 1 -- 172.21.15.120:0/4191062233 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4011d70 con 0x5638969e09a0 2024-09-19T09:21:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.251+0000 7f2db67fc640 1 -- 172.21.15.120:0/4191062233 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dc4005500 con 0x5638969e09a0 2024-09-19T09:21:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.251+0000 7f2db67fc640 1 -- 172.21.15.120:0/4191062233 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4004a00 con 0x5638969e09a0 2024-09-19T09:21:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.252+0000 7f2dd962e740 1 -- 172.21.15.120:0/4191062233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638969e09a0 msgr2=0x5638969e0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.252+0000 7f2dd962e740 1 --2- 172.21.15.120:0/4191062233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638969e09a0 0x5638969e0da0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2dc40058c0 tx=0x7f2dc4005f90 comp rx=0 tx=0).stop 2024-09-19T09:21:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.252+0000 7f2dd962e740 1 -- 172.21.15.120:0/4191062233 shutdown_connections 2024-09-19T09:21:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.252+0000 7f2dd962e740 1 --2- 172.21.15.120:0/4191062233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638969e6370 0x5638969e8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.252+0000 7f2dd962e740 1 --2- 172.21.15.120:0/4191062233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638969e12e0 0x5638969e5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.252+0000 7f2dd962e740 1 --2- 172.21.15.120:0/4191062233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638969e09a0 0x5638969e0da0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.252+0000 7f2dd962e740 1 -- 172.21.15.120:0/4191062233 >> 172.21.15.120:0/4191062233 conn(0x5638969dc010 msgr2=0x5638969dc410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.252+0000 7f2dd962e740 1 -- 172.21.15.120:0/4191062233 shutdown_connections 2024-09-19T09:21:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.253+0000 7f2dd962e740 1 -- 172.21.15.120:0/4191062233 wait complete. 2024-09-19T09:21:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.253+0000 7f2dd962e740 1 -- start start 2024-09-19T09:21:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.253+0000 7f2dd962e740 1 Processor -- start 2024-09-19T09:21:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.253+0000 7f2dd962e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638969e09a0 0x5638969eabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.253+0000 7f2dd962e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638969e12e0 0x5638969eb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.253+0000 7f2dd962e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638969e6370 0x5638969f2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.253+0000 7f2dd962e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5638969e9340 con 0x5638969e6370 2024-09-19T09:21:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.253+0000 7f2dd962e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5638968cd290 con 0x5638969e09a0 2024-09-19T09:21:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.253+0000 7f2dd962e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5638969e94c0 con 0x5638969e12e0 2024-09-19T09:21:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.253+0000 7f2dcf7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638969e6370 0x5638969f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.253+0000 7f2dd962e740 10 client.0 _ll_register_callbacks cb 0x5638968fe140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.253+0000 7f2dcf7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638969e6370 0x5638969f2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48762/0 (socket says 172.21.15.120:48762) 2024-09-19T09:21:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.253+0000 7f2dcffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638969e12e0 0x5638969eb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.253+0000 7f2dcf7fe640 1 -- 172.21.15.120:0/1798404747 learned_addr learned my addr 172.21.15.120:0/1798404747 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.254+0000 7f2dcf7fe640 1 --2- 172.21.15.120:0/1798404747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638969e09a0 0x5638969eabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.254+0000 7f2dcf7fe640 1 -- 172.21.15.120:0/1798404747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638969e12e0 msgr2=0x5638969eb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.254+0000 7f2dcf7fe640 1 --2- 172.21.15.120:0/1798404747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638969e12e0 0x5638969eb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.254+0000 7f2dcf7fe640 1 -- 172.21.15.120:0/1798404747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638969e09a0 msgr2=0x5638969eabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.254+0000 7f2dcf7fe640 1 --2- 172.21.15.120:0/1798404747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638969e09a0 0x5638969eabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.254+0000 7f2dcf7fe640 1 -- 172.21.15.120:0/1798404747 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563896657210 con 0x5638969e6370 2024-09-19T09:21:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.254+0000 7f2dcffff640 1 --2- 172.21.15.120:0/1798404747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638969e12e0 0x5638969eb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:21:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.254+0000 7f2dcf7fe640 1 --2- 172.21.15.120:0/1798404747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638969e6370 0x5638969f2140 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x5638969ea0f0 tx=0x7f2dc40114c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451915564f2c7a55 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.254+0000 7f2daf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:21:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.254+0000 7f2daf7fe640 1 -- 172.21.15.120:0/1798404747 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc400fe20 con 0x5638969e6370 2024-09-19T09:21:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.254+0000 7f2daf7fe640 1 -- 172.21.15.120:0/1798404747 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f2dc4011d70 con 0x5638969e6370 2024-09-19T09:21:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.254+0000 7f2dd962e740 1 -- 172.21.15.120:0/1798404747 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dc4005500 con 0x5638969e6370 2024-09-19T09:21:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.254+0000 7f2daf7fe640 1 -- 172.21.15.120:0/1798404747 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2dc400f790 con 0x5638969e6370 2024-09-19T09:21:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.254+0000 7f2dd962e740 10 client.21528 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.254+0000 7f2dd962e740 1 -- 172.21.15.120:0/1798404747 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563896723620 con 0x5638969e6370 2024-09-19T09:21:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.255+0000 7f2daf7fe640 1 -- 172.21.15.120:0/1798404747 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dc4005500 con 0x5638969e6370 2024-09-19T09:21:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.255+0000 7f2daf7fe640 1 -- 172.21.15.120:0/1798404747 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f2dc4004cf0 con 0x5638969e6370 2024-09-19T09:21:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.255+0000 7f2dd962e740 10 client.21528 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.255+0000 7f2dd962e740 10 client.21528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:21:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.255+0000 7f2dd962e740 20 client.21528 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.255+0000 7f2dd962e740 1 --2- 172.21.15.120:0/1798404747 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5638969f6630 0x5638969f8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.255+0000 7f2dd962e740 4 client.21528 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:21:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.255+0000 7f2dd962e740 1 -- 172.21.15.120:0/1798404747 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563896951a20 con 0x5638969f6630 2024-09-19T09:21:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.255+0000 7f2dcffff640 1 --2- 172.21.15.120:0/1798404747 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5638969f6630 0x5638969f8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.256+0000 7f2daf7fe640 10 client.21528 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.256+0000 7f2dcffff640 1 --2- 172.21.15.120:0/1798404747 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5638969f6630 0x5638969f8a60 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=555824f846673779 server_cookie=9a8f9b20a9e724a0 in_seq=0 out_seq=0 2024-09-19T09:21:15.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.259+0000 7f2daf7fe640 1 -- 172.21.15.120:0/1798404747 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563896951a20 con 0x5638969f6630 2024-09-19T09:21:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.259+0000 7f2daf7fe640 10 client.21528 handle_command_reply: tid=0 2024-09-19T09:21:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.259+0000 7f2dd962e740 1 client.21528 shutdown 2024-09-19T09:21:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.259+0000 7f2dd962e740 1 -- 172.21.15.120:0/1798404747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638969e6370 msgr2=0x5638969f2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.259+0000 7f2dd962e740 1 --2- 172.21.15.120:0/1798404747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638969e6370 0x5638969f2140 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x5638969ea0f0 tx=0x7f2dc40114c0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.259+0000 7f2dd962e740 1 -- 172.21.15.120:0/1798404747 shutdown_connections 2024-09-19T09:21:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.259+0000 7f2dd962e740 1 --2- 172.21.15.120:0/1798404747 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5638969f6630 0x5638969f8a60 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.259+0000 7f2dd962e740 1 --2- 172.21.15.120:0/1798404747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638969e6370 0x5638969f2140 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.259+0000 7f2dd962e740 1 --2- 172.21.15.120:0/1798404747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638969e12e0 0x5638969eb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.259+0000 7f2daf7fe640 0 client.21528 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.259+0000 7f2dd962e740 1 --2- 172.21.15.120:0/1798404747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638969e09a0 0x5638969eabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.259+0000 7f2dd962e740 1 -- 172.21.15.120:0/1798404747 >> 172.21.15.120:0/1798404747 conn(0x5638969dc010 msgr2=0x5638969dd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.259+0000 7f2dd962e740 1 -- 172.21.15.120:0/1798404747 shutdown_connections 2024-09-19T09:21:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.259+0000 7f2dd962e740 1 -- 172.21.15.120:0/1798404747 wait complete. 2024-09-19T09:21:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.259+0000 7f2dd962e740 20 client.21528 trim_cache size 0 max 16384 2024-09-19T09:21:15.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.278+0000 7f2dd962e740 1 Processor -- start 2024-09-19T09:21:15.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.278+0000 7f2dd962e740 1 -- start start 2024-09-19T09:21:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.278+0000 7f2dd962e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638969e09a0 0x5638969dcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.278+0000 7f2dd962e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638969e12e0 0x5638969f2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.278+0000 7f2dd962e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638969e6370 0x5638969dd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.278+0000 7f2dd962e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd01ab280 con 0x5638969e12e0 2024-09-19T09:21:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.278+0000 7f2dd962e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd01acdc0 con 0x5638969e6370 2024-09-19T09:21:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.278+0000 7f2dd962e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd01ab950 con 0x5638969e09a0 2024-09-19T09:21:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.278+0000 7f2dcffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638969e12e0 0x5638969f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.278+0000 7f2dcf7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638969e09a0 0x5638969dcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.278+0000 7f2dcffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638969e12e0 0x5638969f2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48770/0 (socket says 172.21.15.120:48770) 2024-09-19T09:21:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.278+0000 7f2dcffff640 1 -- 172.21.15.120:0/1536605533 learned_addr learned my addr 172.21.15.120:0/1536605533 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.278+0000 7f2dcf7fe640 1 --2- 172.21.15.120:0/1536605533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638969e6370 0x5638969dd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.279+0000 7f2dcffff640 1 -- 172.21.15.120:0/1536605533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638969e09a0 msgr2=0x5638969dcec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.279+0000 7f2dcffff640 1 --2- 172.21.15.120:0/1536605533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638969e09a0 0x5638969dcec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.279+0000 7f2dcffff640 1 -- 172.21.15.120:0/1536605533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638969e6370 msgr2=0x5638969dd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.279+0000 7f2dcffff640 1 --2- 172.21.15.120:0/1536605533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638969e6370 0x5638969dd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.279+0000 7f2dcffff640 1 -- 172.21.15.120:0/1536605533 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638969f90e0 con 0x5638969e12e0 2024-09-19T09:21:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.279+0000 7f2dcffff640 1 --2- 172.21.15.120:0/1536605533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638969e12e0 0x5638969f2100 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x5638969dce90 tx=0x7f2dc0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e90ec739f0be68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.279+0000 7f2db5ffb640 1 -- 172.21.15.120:0/1536605533 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0004050 con 0x5638969e12e0 2024-09-19T09:21:15.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.279+0000 7f2db5ffb640 1 -- 172.21.15.120:0/1536605533 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dc000fdd0 con 0x5638969e12e0 2024-09-19T09:21:15.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.279+0000 7f2db5ffb640 1 -- 172.21.15.120:0/1536605533 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc00055e0 con 0x5638969e12e0 2024-09-19T09:21:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.280+0000 7f2dd962e740 1 -- 172.21.15.120:0/1536605533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638969e12e0 msgr2=0x5638969f2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.280+0000 7f2dd962e740 1 --2- 172.21.15.120:0/1536605533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638969e12e0 0x5638969f2100 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x5638969dce90 tx=0x7f2dc0005f90 comp rx=0 tx=0).stop 2024-09-19T09:21:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.280+0000 7f2dd962e740 1 -- 172.21.15.120:0/1536605533 shutdown_connections 2024-09-19T09:21:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.280+0000 7f2dd962e740 1 --2- 172.21.15.120:0/1536605533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638969e6370 0x5638969dd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.280+0000 7f2dd962e740 1 --2- 172.21.15.120:0/1536605533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638969e12e0 0x5638969f2100 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.280+0000 7f2dd962e740 1 --2- 172.21.15.120:0/1536605533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638969e09a0 0x5638969dcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.280+0000 7f2dd962e740 1 -- 172.21.15.120:0/1536605533 >> 172.21.15.120:0/1536605533 conn(0x5638969dc010 msgr2=0x5638969eabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.280+0000 7f2dd962e740 1 -- 172.21.15.120:0/1536605533 shutdown_connections 2024-09-19T09:21:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.280+0000 7f2dd962e740 1 -- 172.21.15.120:0/1536605533 wait complete. 2024-09-19T09:21:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.281+0000 7f2dd962e740 1 -- start start 2024-09-19T09:21:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.281+0000 7f2dd962e740 1 Processor -- start 2024-09-19T09:21:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.281+0000 7f2dd962e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638969e09a0 0x5638969ef110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.281+0000 7f2dd962e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638969e12e0 0x5638969ef650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.281+0000 7f2dcf7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638969e09a0 0x5638969ef110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.281+0000 7f2dcf7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638969e09a0 0x5638969ef110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:49088/0 (socket says 172.21.15.120:49088) 2024-09-19T09:21:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.281+0000 7f2dcf7fe640 1 -- 172.21.15.120:0/4043045432 learned_addr learned my addr 172.21.15.120:0/4043045432 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.281+0000 7f2dd962e740 1 --2- 172.21.15.120:0/4043045432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638969e6370 0x563896a5f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.281+0000 7f2dd962e740 1 -- 172.21.15.120:0/4043045432 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd01ac6f0 con 0x5638969e12e0 2024-09-19T09:21:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.281+0000 7f2dd962e740 1 -- 172.21.15.120:0/4043045432 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd01ac020 con 0x5638969e6370 2024-09-19T09:21:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.281+0000 7f2dd962e740 1 -- 172.21.15.120:0/4043045432 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd01a9240 con 0x5638969e09a0 2024-09-19T09:21:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.282+0000 7f2dcffff640 1 --2- 172.21.15.120:0/4043045432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638969e12e0 0x5638969ef650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.282+0000 7f2dcf7fe640 1 --2- 172.21.15.120:0/4043045432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638969e6370 0x563896a5f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.282+0000 7f2dd962e740 10 client.0 _ll_register_callbacks cb 0x7f2dc4009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.282+0000 7f2dcf7fe640 1 -- 172.21.15.120:0/4043045432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638969e6370 msgr2=0x563896a5f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.282+0000 7f2dcf7fe640 1 --2- 172.21.15.120:0/4043045432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638969e6370 0x563896a5f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.282+0000 7f2dcf7fe640 1 -- 172.21.15.120:0/4043045432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638969e12e0 msgr2=0x5638969ef650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.282+0000 7f2dcf7fe640 1 --2- 172.21.15.120:0/4043045432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638969e12e0 0x5638969ef650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.282+0000 7f2dcf7fe640 1 -- 172.21.15.120:0/4043045432 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563896a5f8a0 con 0x5638969e09a0 2024-09-19T09:21:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.282+0000 7f2dd962e740 1 -- 172.21.15.120:0/4043045432 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dc000fdd0 con 0x5638969e09a0 2024-09-19T09:21:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.282+0000 7f2dcf7fe640 1 --2- 172.21.15.120:0/4043045432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638969e09a0 0x5638969ef110 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4000c00 tx=0x7f2dc40114c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=519cbc94cbd01c62 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.282+0000 7f2daf7fe640 10 client.30193 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:21:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.283+0000 7f2daf7fe640 1 -- 172.21.15.120:0/4043045432 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4036080 con 0x5638969e09a0 2024-09-19T09:21:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.283+0000 7f2daf7fe640 1 -- 172.21.15.120:0/4043045432 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f2dc4005330 con 0x5638969e09a0 2024-09-19T09:21:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.283+0000 7f2daf7fe640 1 -- 172.21.15.120:0/4043045432 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2dc4005570 con 0x5638969e09a0 2024-09-19T09:21:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.283+0000 7f2dd962e740 10 client.30193 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.283+0000 7f2dd962e740 1 -- 172.21.15.120:0/4043045432 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({fsmap=39}) -- 0x563896a60a90 con 0x5638969e09a0 2024-09-19T09:21:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.283+0000 7f2daf7fe640 1 -- 172.21.15.120:0/4043045432 <== mon.1 v2:172.21.15.196:3300/0 4 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f2dc400f930 con 0x5638969e09a0 2024-09-19T09:21:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.283+0000 7f2dd962e740 10 client.30193 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.283+0000 7f2dd962e740 10 client.30193 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:21:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.283+0000 7f2dd962e740 20 client.30193 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.283+0000 7f2dd962e740 1 --2- 172.21.15.120:0/4043045432 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x563896a612c0 0x563896a636e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.283+0000 7f2dd962e740 4 client.30193 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:21:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.283+0000 7f2dd962e740 1 -- 172.21.15.120:0/4043045432 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56389694bd70 con 0x563896a612c0 2024-09-19T09:21:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.283+0000 7f2dcffff640 1 --2- 172.21.15.120:0/4043045432 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x563896a612c0 0x563896a636e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.284+0000 7f2daf7fe640 10 client.30193 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.284+0000 7f2dcffff640 1 --2- 172.21.15.120:0/4043045432 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x563896a612c0 0x563896a636e0 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4b18a56151ec565 server_cookie=4bd80650f7ae9acf in_seq=0 out_seq=0 2024-09-19T09:21:15.371 INFO:tasks.workunit.client.0.smithi120.stdout: 10 127338 20.26 MB/sec execute 190 sec latency 161.307 ms 2024-09-19T09:21:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.623+0000 7f2daf7fe640 1 -- 172.21.15.120:0/4043045432 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56389694bd70 con 0x563896a612c0 2024-09-19T09:21:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.623+0000 7f2daf7fe640 10 client.30193 handle_command_reply: tid=0 2024-09-19T09:21:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.623+0000 7f2dd962e740 1 client.30193 shutdown 2024-09-19T09:21:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.623+0000 7f2dd962e740 1 -- 172.21.15.120:0/4043045432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638969e09a0 msgr2=0x5638969ef110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.623+0000 7f2dd962e740 1 --2- 172.21.15.120:0/4043045432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638969e09a0 0x5638969ef110 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4000c00 tx=0x7f2dc40114c0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.623+0000 7f2dd962e740 1 -- 172.21.15.120:0/4043045432 shutdown_connections 2024-09-19T09:21:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.623+0000 7f2dd962e740 1 --2- 172.21.15.120:0/4043045432 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x563896a612c0 0x563896a636e0 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.623+0000 7f2dd962e740 1 --2- 172.21.15.120:0/4043045432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638969e6370 0x563896a5f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.623+0000 7f2daf7fe640 0 client.30193 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.623+0000 7f2dd962e740 1 --2- 172.21.15.120:0/4043045432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638969e12e0 0x5638969ef650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.623+0000 7f2dd962e740 1 --2- 172.21.15.120:0/4043045432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638969e09a0 0x5638969ef110 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.623+0000 7f2dd962e740 1 -- 172.21.15.120:0/4043045432 >> 172.21.15.120:0/4043045432 conn(0x5638969dc010 msgr2=0x5638969da9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.624+0000 7f2dd962e740 1 -- 172.21.15.120:0/4043045432 shutdown_connections 2024-09-19T09:21:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.624+0000 7f2dd962e740 1 -- 172.21.15.120:0/4043045432 wait complete. 2024-09-19T09:21:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.624+0000 7f2dd962e740 20 client.30193 trim_cache size 0 max 16384 2024-09-19T09:21:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:21:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:21:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:21:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:21:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.624+0000 7f2dd6896640 1 -- 172.21.15.120:0/3080145987 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2db80a3370 msgr2=0x7f2db80a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.624+0000 7f2dd6896640 1 --2- 172.21.15.120:0/3080145987 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2db80a3370 0x7f2db80a5830 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2dc803cfd0 tx=0x7f2dc802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.624+0000 7f2dd6896640 1 -- 172.21.15.120:0/3080145987 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dd0101700 msgr2=0x7f2dd01a1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.624+0000 7f2dd6896640 1 --2- 172.21.15.120:0/3080145987 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dd0101700 0x7f2dd01a1530 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f2dc802ade0 tx=0x7f2dc80319f0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.624+0000 7f2dd6896640 1 -- 172.21.15.120:0/3080145987 shutdown_connections 2024-09-19T09:21:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.624+0000 7f2dd6896640 1 --2- 172.21.15.120:0/3080145987 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2db80a3370 0x7f2db80a5830 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.624+0000 7f2dd6896640 1 --2- 172.21.15.120:0/3080145987 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2dd0101700 0x7f2dd01a1530 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.624+0000 7f2dd6896640 1 --2- 172.21.15.120:0/3080145987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2dd0100d60 0x7f2dd019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.624+0000 7f2dd6896640 1 --2- 172.21.15.120:0/3080145987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dd0100420 0x7f2dd0199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.624+0000 7f2dd6896640 1 -- 172.21.15.120:0/3080145987 >> 172.21.15.120:0/3080145987 conn(0x7f2dd00fc1b0 msgr2=0x7f2dd00fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.624+0000 7f2dd6896640 1 -- 172.21.15.120:0/3080145987 shutdown_connections 2024-09-19T09:21:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:15.624+0000 7f2dd6896640 1 -- 172.21.15.120:0/3080145987 wait complete. 2024-09-19T09:21:16.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:16 smithi120 ceph-mon[31150]: pgmap v897: 97 pgs: 97 active+clean; 379 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.1 MiB/s wr, 318 op/s 2024-09-19T09:21:16.369 INFO:tasks.workunit.client.0.smithi120.stdout: 10 127844 20.31 MB/sec execute 191 sec latency 191.644 ms 2024-09-19T09:21:16.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:16 smithi196 ceph-mon[35150]: pgmap v897: 97 pgs: 97 active+clean; 379 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.1 MiB/s wr, 318 op/s 2024-09-19T09:21:16.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:16 smithi153 ceph-mon[35071]: pgmap v897: 97 pgs: 97 active+clean; 379 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.1 MiB/s wr, 318 op/s 2024-09-19T09:21:17.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:17 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:21:17.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:17 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:17.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:17 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:21:17.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:17 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:21:17.369 INFO:tasks.workunit.client.0.smithi120.stdout: 10 128536 20.41 MB/sec execute 192 sec latency 54.407 ms 2024-09-19T09:21:17.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:17 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:21:17.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:17 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:17.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:17 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:21:17.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:17 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:21:17.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:17 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:21:17.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:17 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:17.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:17 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:21:17.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:17 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:21:18.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:18 smithi120 ceph-mon[31150]: pgmap v898: 97 pgs: 97 active+clean; 371 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.3 MiB/s wr, 389 op/s 2024-09-19T09:21:18.369 INFO:tasks.workunit.client.0.smithi120.stdout: 10 129080 20.45 MB/sec execute 193 sec latency 99.119 ms 2024-09-19T09:21:18.397 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:21:18.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:18 smithi196 ceph-mon[35150]: pgmap v898: 97 pgs: 97 active+clean; 371 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.3 MiB/s wr, 389 op/s 2024-09-19T09:21:18.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:18 smithi153 ceph-mon[35071]: pgmap v898: 97 pgs: 97 active+clean; 371 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.3 MiB/s wr, 389 op/s 2024-09-19T09:21:18.711 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:21:19.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.243+0000 7f446fee6640 1 -- 172.21.15.120:0/1854069890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4468102d30 msgr2=0x7f4468103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:19.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.243+0000 7f446fee6640 1 --2- 172.21.15.120:0/1854069890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4468102d30 0x7f4468103110 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f4464009930 tx=0x7f446402a420 comp rx=0 tx=0).stop 2024-09-19T09:21:19.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.243+0000 7f446fee6640 1 -- 172.21.15.120:0/1854069890 shutdown_connections 2024-09-19T09:21:19.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.243+0000 7f446fee6640 1 --2- 172.21.15.120:0/1854069890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44681082a0 0x7f446810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.243+0000 7f446fee6640 1 --2- 172.21.15.120:0/1854069890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44681036e0 0x7f4468107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.243+0000 7f446fee6640 1 --2- 172.21.15.120:0/1854069890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4468102d30 0x7f4468103110 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.243+0000 7f446fee6640 1 -- 172.21.15.120:0/1854069890 >> 172.21.15.120:0/1854069890 conn(0x7f44680fc7d0 msgr2=0x7f44680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:19.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.243+0000 7f446fee6640 1 -- 172.21.15.120:0/1854069890 shutdown_connections 2024-09-19T09:21:19.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.243+0000 7f446fee6640 1 -- 172.21.15.120:0/1854069890 wait complete. 2024-09-19T09:21:19.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.244+0000 7f446fee6640 1 Processor -- start 2024-09-19T09:21:19.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.244+0000 7f446fee6640 1 -- start start 2024-09-19T09:21:19.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.244+0000 7f446fee6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4468102d30 0x7f4468100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:19.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.244+0000 7f446fee6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44681036e0 0x7f4468100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:19.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.244+0000 7f446fee6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44681082a0 0x7f4468101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:19.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.244+0000 7f446fee6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f44681120a0 con 0x7f44681082a0 2024-09-19T09:21:19.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.244+0000 7f446fee6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4468111f20 con 0x7f4468102d30 2024-09-19T09:21:19.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.244+0000 7f446fee6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4468112220 con 0x7f44681036e0 2024-09-19T09:21:19.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.244+0000 7f446e483640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44681082a0 0x7f4468101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:19.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.244+0000 7f446dc82640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4468102d30 0x7f4468100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:19.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.244+0000 7f446e483640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44681082a0 0x7f4468101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48796/0 (socket says 172.21.15.120:48796) 2024-09-19T09:21:19.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.244+0000 7f446e483640 1 -- 172.21.15.120:0/2301947598 learned_addr learned my addr 172.21.15.120:0/2301947598 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:19.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.244+0000 7f446d481640 1 --2- 172.21.15.120:0/2301947598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44681036e0 0x7f4468100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:19.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.244+0000 7f446dc82640 1 -- 172.21.15.120:0/2301947598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44681036e0 msgr2=0x7f4468100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:19.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.245+0000 7f446dc82640 1 --2- 172.21.15.120:0/2301947598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44681036e0 0x7f4468100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.245+0000 7f446dc82640 1 -- 172.21.15.120:0/2301947598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44681082a0 msgr2=0x7f4468101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:19.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.245+0000 7f446dc82640 1 --2- 172.21.15.120:0/2301947598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44681082a0 0x7f4468101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.245+0000 7f446dc82640 1 -- 172.21.15.120:0/2301947598 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44681a6a60 con 0x7f4468102d30 2024-09-19T09:21:19.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.245+0000 7f446d481640 1 --2- 172.21.15.120:0/2301947598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44681036e0 0x7f4468100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:21:19.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.245+0000 7f446e483640 1 --2- 172.21.15.120:0/2301947598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44681082a0 0x7f4468101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:21:19.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.245+0000 7f446dc82640 1 --2- 172.21.15.120:0/2301947598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4468102d30 0x7f4468100990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4464005940 tx=0x7f4464031b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:19.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.245+0000 7f445effd640 1 -- 172.21.15.120:0/2301947598 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446403f070 con 0x7f4468102d30 2024-09-19T09:21:19.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.245+0000 7f446fee6640 1 -- 172.21.15.120:0/2301947598 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44681a6cf0 con 0x7f4468102d30 2024-09-19T09:21:19.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.245+0000 7f445effd640 1 -- 172.21.15.120:0/2301947598 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4464003ad0 con 0x7f4468102d30 2024-09-19T09:21:19.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.245+0000 7f445effd640 1 -- 172.21.15.120:0/2301947598 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4464035070 con 0x7f4468102d30 2024-09-19T09:21:19.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.245+0000 7f446fee6640 1 -- 172.21.15.120:0/2301947598 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44681a7170 con 0x7f4468102d30 2024-09-19T09:21:19.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.248+0000 7f44722bf740 1 Processor -- start 2024-09-19T09:21:19.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.248+0000 7f44722bf740 1 -- start start 2024-09-19T09:21:19.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.248+0000 7f445effd640 1 -- 172.21.15.120:0/2301947598 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f44640035f0 con 0x7f4468102d30 2024-09-19T09:21:19.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.248+0000 7f44722bf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55913c7114a0 0x55913c711880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:19.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.248+0000 7f44722bf740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55913c711dc0 0x55913c71c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:19.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.248+0000 7f44722bf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55913c71c680 0x55913c71ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:19.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.248+0000 7f44722bf740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55913c6772e0 con 0x55913c711dc0 2024-09-19T09:21:19.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.248+0000 7f44722bf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55913c61ece0 con 0x55913c7114a0 2024-09-19T09:21:19.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.248+0000 7f44722bf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55913c6732a0 con 0x55913c71c680 2024-09-19T09:21:19.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.249+0000 7f446dc82640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55913c711dc0 0x55913c71c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:19.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.249+0000 7f446e483640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55913c7114a0 0x55913c711880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:19.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.249+0000 7f446e483640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55913c7114a0 0x55913c711880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:44308/0 (socket says 172.21.15.120:44308) 2024-09-19T09:21:19.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.249+0000 7f446e483640 1 -- 172.21.15.120:0/136804055 learned_addr learned my addr 172.21.15.120:0/136804055 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:19.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.249+0000 7f446d481640 1 --2- 172.21.15.120:0/136804055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55913c71c680 0x55913c71ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:19.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.249+0000 7f446e483640 1 -- 172.21.15.120:0/136804055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55913c71c680 msgr2=0x55913c71ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:19.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.249+0000 7f446e483640 1 --2- 172.21.15.120:0/136804055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55913c71c680 0x55913c71ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.249+0000 7f446e483640 1 -- 172.21.15.120:0/136804055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55913c711dc0 msgr2=0x55913c71c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:19.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.249+0000 7f446e483640 1 --2- 172.21.15.120:0/136804055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55913c711dc0 0x55913c71c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.249+0000 7f446e483640 1 -- 172.21.15.120:0/136804055 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55913c603440 con 0x55913c7114a0 2024-09-19T09:21:19.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.249+0000 7f446dc82640 1 --2- 172.21.15.120:0/136804055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55913c711dc0 0x55913c71c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:21:19.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.249+0000 7f445effd640 1 --2- 172.21.15.120:0/2301947598 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f44540a3240 0x7f44540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:19.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.249+0000 7f446e483640 1 --2- 172.21.15.120:0/136804055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55913c7114a0 0x55913c711880 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4460011850 tx=0x7f4460011d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72854ecf51deb432 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:19.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.249+0000 7f446dc82640 1 --2- 172.21.15.120:0/2301947598 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f44540a3240 0x7f44540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:19.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.249+0000 7f4453fff640 1 -- 172.21.15.120:0/136804055 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4460011330 con 0x55913c7114a0 2024-09-19T09:21:19.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.249+0000 7f4453fff640 1 -- 172.21.15.120:0/136804055 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f446000fe80 con 0x55913c7114a0 2024-09-19T09:21:19.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.249+0000 7f4453fff640 1 -- 172.21.15.120:0/136804055 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44600193e0 con 0x55913c7114a0 2024-09-19T09:21:19.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.249+0000 7f445effd640 1 -- 172.21.15.120:0/2301947598 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f44640efd50 con 0x7f4468102d30 2024-09-19T09:21:19.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.250+0000 7f44722bf740 1 -- 172.21.15.120:0/136804055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55913c7114a0 msgr2=0x55913c711880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:19.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.250+0000 7f446dc82640 1 --2- 172.21.15.120:0/2301947598 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f44540a3240 0x7f44540a5700 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f44640b8df0 tx=0x7f44640f37d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:19.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.250+0000 7f44722bf740 1 --2- 172.21.15.120:0/136804055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55913c7114a0 0x55913c711880 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4460011850 tx=0x7f4460011d10 comp rx=0 tx=0).stop 2024-09-19T09:21:19.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.250+0000 7f44722bf740 1 -- 172.21.15.120:0/136804055 shutdown_connections 2024-09-19T09:21:19.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.250+0000 7f44722bf740 1 --2- 172.21.15.120:0/136804055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55913c71c680 0x55913c71ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.250+0000 7f44722bf740 1 --2- 172.21.15.120:0/136804055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55913c711dc0 0x55913c71c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.250+0000 7f44722bf740 1 --2- 172.21.15.120:0/136804055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55913c7114a0 0x55913c711880 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.250+0000 7f44722bf740 1 -- 172.21.15.120:0/136804055 >> 172.21.15.120:0/136804055 conn(0x55913c70da30 msgr2=0x55913c70de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:19.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.250+0000 7f44722bf740 1 -- 172.21.15.120:0/136804055 shutdown_connections 2024-09-19T09:21:19.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.250+0000 7f44722bf740 1 -- 172.21.15.120:0/136804055 wait complete. 2024-09-19T09:21:19.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.250+0000 7f44722bf740 1 -- start start 2024-09-19T09:21:19.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.251+0000 7f44722bf740 1 Processor -- start 2024-09-19T09:21:19.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.251+0000 7f44722bf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55913c7114a0 0x55913c71bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:19.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.251+0000 7f44722bf740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55913c711dc0 0x55913c719220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:19.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.251+0000 7f44722bf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55913c71c680 0x55913c719760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:19.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.251+0000 7f44722bf740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55913c71f250 con 0x55913c711dc0 2024-09-19T09:21:19.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.251+0000 7f44722bf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55913c632050 con 0x55913c7114a0 2024-09-19T09:21:19.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.251+0000 7f44722bf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55913c71f3d0 con 0x55913c71c680 2024-09-19T09:21:19.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.251+0000 7f44722bf740 10 client.0 _ll_register_callbacks cb 0x55913c672160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:19.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.251+0000 7f446e483640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55913c7114a0 0x55913c71bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:19.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.251+0000 7f446d481640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55913c711dc0 0x55913c719220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:19.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.251+0000 7f446e483640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55913c7114a0 0x55913c71bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:44310/0 (socket says 172.21.15.120:44310) 2024-09-19T09:21:19.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.251+0000 7f446e483640 1 -- 172.21.15.120:0/3317113632 learned_addr learned my addr 172.21.15.120:0/3317113632 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:19.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.251+0000 7f446d481640 1 --2- 172.21.15.120:0/3317113632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55913c711dc0 0x55913c719220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48814/0 (socket says 172.21.15.120:48814) 2024-09-19T09:21:19.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.251+0000 7f446e483640 1 --2- 172.21.15.120:0/3317113632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55913c71c680 0x55913c719760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:19.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.251+0000 7f446d481640 1 -- 172.21.15.120:0/3317113632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55913c71c680 msgr2=0x55913c719760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:19.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.251+0000 7f446d481640 1 --2- 172.21.15.120:0/3317113632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55913c71c680 0x55913c719760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.251+0000 7f446d481640 1 -- 172.21.15.120:0/3317113632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55913c7114a0 msgr2=0x55913c71bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:19.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.251+0000 7f446d481640 1 --2- 172.21.15.120:0/3317113632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55913c7114a0 0x55913c71bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.252+0000 7f446d481640 1 -- 172.21.15.120:0/3317113632 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55913c3e83c0 con 0x55913c711dc0 2024-09-19T09:21:19.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.252+0000 7f446d481640 1 --2- 172.21.15.120:0/3317113632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55913c711dc0 0x55913c719220 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f4468102420 tx=0x7f4458012e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0df070e812a94d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:19.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.252+0000 7f44517fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:21:19.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.252+0000 7f44517fa640 1 -- 172.21.15.120:0/3317113632 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4458004050 con 0x55913c711dc0 2024-09-19T09:21:19.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.252+0000 7f44722bf740 1 -- 172.21.15.120:0/3317113632 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55913c684850 con 0x55913c711dc0 2024-09-19T09:21:19.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.252+0000 7f44517fa640 1 -- 172.21.15.120:0/3317113632 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f44580041f0 con 0x55913c711dc0 2024-09-19T09:21:19.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.252+0000 7f44517fa640 1 -- 172.21.15.120:0/3317113632 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f44580043f0 con 0x55913c711dc0 2024-09-19T09:21:19.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.252+0000 7f44722bf740 10 client.21546 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:19.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.252+0000 7f44722bf740 1 -- 172.21.15.120:0/3317113632 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55913c733480 con 0x55913c711dc0 2024-09-19T09:21:19.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.252+0000 7f44517fa640 1 -- 172.21.15.120:0/3317113632 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55913c684850 con 0x55913c711dc0 2024-09-19T09:21:19.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.253+0000 7f44517fa640 1 -- 172.21.15.120:0/3317113632 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f4458003cb0 con 0x55913c711dc0 2024-09-19T09:21:19.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.253+0000 7f44722bf740 10 client.21546 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:19.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.253+0000 7f44722bf740 10 client.21546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:21:19.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.253+0000 7f44722bf740 20 client.21546 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:19.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.253+0000 7f44722bf740 1 --2- 172.21.15.120:0/3317113632 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55913c734890 0x55913c736c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:19.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.253+0000 7f44722bf740 4 client.21546 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:21:19.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.253+0000 7f44722bf740 1 -- 172.21.15.120:0/3317113632 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55913c67ed60 con 0x55913c734890 2024-09-19T09:21:19.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.253+0000 7f446e483640 1 --2- 172.21.15.120:0/3317113632 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55913c734890 0x55913c736c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:19.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.253+0000 7f44517fa640 10 client.21546 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:19.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.253+0000 7f446e483640 1 --2- 172.21.15.120:0/3317113632 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55913c734890 0x55913c736c80 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd7d534127d8091e server_cookie=8d00fa59426644e7 in_seq=0 out_seq=0 2024-09-19T09:21:19.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.255+0000 7f44517fa640 1 -- 172.21.15.120:0/3317113632 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55913c67ed60 con 0x55913c734890 2024-09-19T09:21:19.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.255+0000 7f44517fa640 10 client.21546 handle_command_reply: tid=0 2024-09-19T09:21:19.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.255+0000 7f44722bf740 1 client.21546 shutdown 2024-09-19T09:21:19.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.255+0000 7f44722bf740 1 -- 172.21.15.120:0/3317113632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55913c711dc0 msgr2=0x55913c719220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:19.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.255+0000 7f44722bf740 1 --2- 172.21.15.120:0/3317113632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55913c711dc0 0x55913c719220 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f4468102420 tx=0x7f4458012e70 comp rx=0 tx=0).stop 2024-09-19T09:21:19.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.255+0000 7f44722bf740 1 -- 172.21.15.120:0/3317113632 shutdown_connections 2024-09-19T09:21:19.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.255+0000 7f44722bf740 1 --2- 172.21.15.120:0/3317113632 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55913c734890 0x55913c736c80 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.255+0000 7f44722bf740 1 --2- 172.21.15.120:0/3317113632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55913c71c680 0x55913c719760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.255+0000 7f44722bf740 1 --2- 172.21.15.120:0/3317113632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55913c711dc0 0x55913c719220 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.255+0000 7f44722bf740 1 --2- 172.21.15.120:0/3317113632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55913c7114a0 0x55913c71bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.255+0000 7f44517fa640 0 client.21546 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:19.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.255+0000 7f44722bf740 1 -- 172.21.15.120:0/3317113632 >> 172.21.15.120:0/3317113632 conn(0x55913c70da30 msgr2=0x55913c70cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:19.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.256+0000 7f44722bf740 1 -- 172.21.15.120:0/3317113632 shutdown_connections 2024-09-19T09:21:19.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.256+0000 7f44722bf740 1 -- 172.21.15.120:0/3317113632 wait complete. 2024-09-19T09:21:19.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.256+0000 7f44722bf740 20 client.21546 trim_cache size 0 max 16384 2024-09-19T09:21:19.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.277+0000 7f44722bf740 1 Processor -- start 2024-09-19T09:21:19.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.277+0000 7f44722bf740 1 -- start start 2024-09-19T09:21:19.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.277+0000 7f44722bf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55913c7114a0 0x55913c73c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:19.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.277+0000 7f44722bf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55913c711dc0 0x55913c73ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:19.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.277+0000 7f44722bf740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55913c71c680 0x55913c716480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:19.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.277+0000 7f44722bf740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f44681ab480 con 0x55913c71c680 2024-09-19T09:21:19.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.277+0000 7f44722bf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f44681acfc0 con 0x55913c7114a0 2024-09-19T09:21:19.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.277+0000 7f44722bf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f44681abb50 con 0x55913c711dc0 2024-09-19T09:21:19.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.277+0000 7f446e483640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55913c71c680 0x55913c716480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:19.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.277+0000 7f446e483640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55913c71c680 0x55913c716480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48818/0 (socket says 172.21.15.120:48818) 2024-09-19T09:21:19.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.277+0000 7f446e483640 1 -- 172.21.15.120:0/281889342 learned_addr learned my addr 172.21.15.120:0/281889342 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:19.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.277+0000 7f446d481640 1 --2- 172.21.15.120:0/281889342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55913c711dc0 0x55913c73ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:19.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.277+0000 7f446e483640 1 --2- 172.21.15.120:0/281889342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55913c7114a0 0x55913c73c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:19.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.277+0000 7f446e483640 1 -- 172.21.15.120:0/281889342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55913c711dc0 msgr2=0x55913c73ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:19.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.277+0000 7f446e483640 1 --2- 172.21.15.120:0/281889342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55913c711dc0 0x55913c73ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.278+0000 7f446e483640 1 -- 172.21.15.120:0/281889342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55913c7114a0 msgr2=0x55913c73c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:19.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.278+0000 7f446e483640 1 --2- 172.21.15.120:0/281889342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55913c7114a0 0x55913c73c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.278+0000 7f446e483640 1 -- 172.21.15.120:0/281889342 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55913c711bf0 con 0x55913c71c680 2024-09-19T09:21:19.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.278+0000 7f446d481640 1 --2- 172.21.15.120:0/281889342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55913c711dc0 0x55913c73ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:21:19.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.278+0000 7f446e483640 1 --2- 172.21.15.120:0/281889342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55913c71c680 0x55913c716480 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x55913c7024e0 tx=0x7f4460009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb773e955581f8e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:19.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.278+0000 7f44537fe640 1 -- 172.21.15.120:0/281889342 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44600112d0 con 0x55913c71c680 2024-09-19T09:21:19.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.278+0000 7f44537fe640 1 -- 172.21.15.120:0/281889342 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4460010430 con 0x55913c71c680 2024-09-19T09:21:19.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.278+0000 7f44537fe640 1 -- 172.21.15.120:0/281889342 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446001a5c0 con 0x55913c71c680 2024-09-19T09:21:19.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.279+0000 7f44722bf740 1 -- 172.21.15.120:0/281889342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55913c71c680 msgr2=0x55913c716480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:19.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.279+0000 7f44722bf740 1 --2- 172.21.15.120:0/281889342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55913c71c680 0x55913c716480 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x55913c7024e0 tx=0x7f4460009500 comp rx=0 tx=0).stop 2024-09-19T09:21:19.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.279+0000 7f44722bf740 1 -- 172.21.15.120:0/281889342 shutdown_connections 2024-09-19T09:21:19.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.279+0000 7f44722bf740 1 --2- 172.21.15.120:0/281889342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55913c71c680 0x55913c716480 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.279+0000 7f44722bf740 1 --2- 172.21.15.120:0/281889342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55913c711dc0 0x55913c73ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.279+0000 7f44722bf740 1 --2- 172.21.15.120:0/281889342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55913c7114a0 0x55913c73c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.279+0000 7f44722bf740 1 -- 172.21.15.120:0/281889342 >> 172.21.15.120:0/281889342 conn(0x55913c70da30 msgr2=0x55913c71b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:19.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.279+0000 7f44722bf740 1 -- 172.21.15.120:0/281889342 shutdown_connections 2024-09-19T09:21:19.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.279+0000 7f44722bf740 1 -- 172.21.15.120:0/281889342 wait complete. 2024-09-19T09:21:19.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.279+0000 7f44722bf740 1 -- start start 2024-09-19T09:21:19.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.279+0000 7f44722bf740 1 Processor -- start 2024-09-19T09:21:19.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.280+0000 7f44722bf740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55913c7114a0 0x55913c728b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:19.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.280+0000 7f44722bf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55913c711dc0 0x55913c729090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:19.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.280+0000 7f44722bf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55913c71c680 0x55913c7295d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:19.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.280+0000 7f44722bf740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f44681ac8f0 con 0x55913c7114a0 2024-09-19T09:21:19.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.280+0000 7f44722bf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f44681ac220 con 0x55913c711dc0 2024-09-19T09:21:19.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.280+0000 7f44722bf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f44681a9440 con 0x55913c71c680 2024-09-19T09:21:19.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.280+0000 7f44722bf740 10 client.0 _ll_register_callbacks cb 0x7f4458012620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:19.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.280+0000 7f446e483640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55913c7114a0 0x55913c728b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:19.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.280+0000 7f446d481640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55913c711dc0 0x55913c729090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:19.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.280+0000 7f446e483640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55913c7114a0 0x55913c728b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48824/0 (socket says 172.21.15.120:48824) 2024-09-19T09:21:19.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.280+0000 7f446d481640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55913c711dc0 0x55913c729090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:44330/0 (socket says 172.21.15.120:44330) 2024-09-19T09:21:19.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.280+0000 7f446e483640 1 -- 172.21.15.120:0/3235788423 learned_addr learned my addr 172.21.15.120:0/3235788423 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:19.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.280+0000 7f446e483640 1 --2- 172.21.15.120:0/3235788423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55913c71c680 0x55913c7295d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:19.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.280+0000 7f446d481640 1 -- 172.21.15.120:0/3235788423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55913c71c680 msgr2=0x55913c7295d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:19.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.280+0000 7f446d481640 1 --2- 172.21.15.120:0/3235788423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55913c71c680 0x55913c7295d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.280+0000 7f446d481640 1 -- 172.21.15.120:0/3235788423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55913c7114a0 msgr2=0x55913c728b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:19.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.280+0000 7f446d481640 1 --2- 172.21.15.120:0/3235788423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55913c7114a0 0x55913c728b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.280+0000 7f446d481640 1 -- 172.21.15.120:0/3235788423 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55913c79e520 con 0x55913c711dc0 2024-09-19T09:21:19.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.280+0000 7f446e483640 1 --2- 172.21.15.120:0/3235788423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55913c7114a0 0x55913c728b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:21:19.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.280+0000 7f446d481640 1 --2- 172.21.15.120:0/3235788423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55913c711dc0 0x55913c729090 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4458010670 tx=0x7f445800fbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d4bcd8c221a6af2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:19.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.280+0000 7f44517fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:21:19.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.281+0000 7f44517fa640 1 -- 172.21.15.120:0/3235788423 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445802c040 con 0x55913c711dc0 2024-09-19T09:21:19.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.281+0000 7f44722bf740 1 -- 172.21.15.120:0/3235788423 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4460010430 con 0x55913c711dc0 2024-09-19T09:21:19.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.281+0000 7f44517fa640 1 -- 172.21.15.120:0/3235788423 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f445800fd50 con 0x55913c711dc0 2024-09-19T09:21:19.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.281+0000 7f44517fa640 1 -- 172.21.15.120:0/3235788423 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4458005040 con 0x55913c711dc0 2024-09-19T09:21:19.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.281+0000 7f44722bf740 10 client.30377 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:19.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.281+0000 7f44722bf740 1 -- 172.21.15.120:0/3235788423 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55913c79f7a0 con 0x55913c711dc0 2024-09-19T09:21:19.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.281+0000 7f44517fa640 1 -- 172.21.15.120:0/3235788423 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4460010430 con 0x55913c711dc0 2024-09-19T09:21:19.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.281+0000 7f44517fa640 1 -- 172.21.15.120:0/3235788423 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f4458039050 con 0x55913c711dc0 2024-09-19T09:21:19.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.281+0000 7f44722bf740 10 client.30377 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:19.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.281+0000 7f44722bf740 10 client.30377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:21:19.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.281+0000 7f44722bf740 20 client.30377 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:19.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.281+0000 7f44722bf740 1 --2- 172.21.15.120:0/3235788423 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55913c79fef0 0x55913c7a2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:19.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.281+0000 7f44722bf740 4 client.30377 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:21:19.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.281+0000 7f44722bf740 1 -- 172.21.15.120:0/3235788423 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55913c5fb210 con 0x55913c79fef0 2024-09-19T09:21:19.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.281+0000 7f446e483640 1 --2- 172.21.15.120:0/3235788423 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55913c79fef0 0x55913c7a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:19.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.282+0000 7f44517fa640 10 client.30377 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:19.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.282+0000 7f446e483640 1 --2- 172.21.15.120:0/3235788423 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55913c79fef0 0x55913c7a2310 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=291ed816ab7ee93 server_cookie=80b4eff8cb1bd6e5 in_seq=0 out_seq=0 2024-09-19T09:21:19.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.284+0000 7f44517fa640 1 -- 172.21.15.120:0/3235788423 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55913c5fb210 con 0x55913c79fef0 2024-09-19T09:21:19.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.284+0000 7f44517fa640 10 client.30377 handle_command_reply: tid=0 2024-09-19T09:21:19.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.284+0000 7f44722bf740 1 client.30377 shutdown 2024-09-19T09:21:19.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.284+0000 7f44722bf740 1 -- 172.21.15.120:0/3235788423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55913c711dc0 msgr2=0x55913c729090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:19.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.284+0000 7f44722bf740 1 --2- 172.21.15.120:0/3235788423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55913c711dc0 0x55913c729090 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4458010670 tx=0x7f445800fbe0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.284+0000 7f44722bf740 1 -- 172.21.15.120:0/3235788423 shutdown_connections 2024-09-19T09:21:19.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.284+0000 7f44722bf740 1 --2- 172.21.15.120:0/3235788423 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55913c79fef0 0x55913c7a2310 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.284+0000 7f44722bf740 1 --2- 172.21.15.120:0/3235788423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55913c71c680 0x55913c7295d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.284+0000 7f44722bf740 1 --2- 172.21.15.120:0/3235788423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55913c711dc0 0x55913c729090 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.284+0000 7f44722bf740 1 --2- 172.21.15.120:0/3235788423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55913c7114a0 0x55913c728b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.284+0000 7f44517fa640 0 client.30377 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:19.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.284+0000 7f44722bf740 1 -- 172.21.15.120:0/3235788423 >> 172.21.15.120:0/3235788423 conn(0x55913c70da30 msgr2=0x55913c712ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:19.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.284+0000 7f44722bf740 1 -- 172.21.15.120:0/3235788423 shutdown_connections 2024-09-19T09:21:19.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.284+0000 7f44722bf740 1 -- 172.21.15.120:0/3235788423 wait complete. 2024-09-19T09:21:19.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.284+0000 7f44722bf740 20 client.30377 trim_cache size 0 max 16384 2024-09-19T09:21:19.290 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:21:19.290 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:21:19.291 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:21:19.291 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:21:19.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.285+0000 7f446fee6640 1 -- 172.21.15.120:0/2301947598 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f44540a3240 msgr2=0x7f44540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:19.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.285+0000 7f446fee6640 1 --2- 172.21.15.120:0/2301947598 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f44540a3240 0x7f44540a5700 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f44640b8df0 tx=0x7f44640f37d0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.285+0000 7f446fee6640 1 -- 172.21.15.120:0/2301947598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4468102d30 msgr2=0x7f4468100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:19.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.285+0000 7f446fee6640 1 --2- 172.21.15.120:0/2301947598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4468102d30 0x7f4468100990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4464005940 tx=0x7f4464031b50 comp rx=0 tx=0).stop 2024-09-19T09:21:19.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.285+0000 7f446fee6640 1 -- 172.21.15.120:0/2301947598 shutdown_connections 2024-09-19T09:21:19.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.285+0000 7f446fee6640 1 --2- 172.21.15.120:0/2301947598 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f44540a3240 0x7f44540a5700 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.285+0000 7f446fee6640 1 --2- 172.21.15.120:0/2301947598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44681082a0 0x7f4468101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.285+0000 7f446fee6640 1 --2- 172.21.15.120:0/2301947598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44681036e0 0x7f4468100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.285+0000 7f446fee6640 1 --2- 172.21.15.120:0/2301947598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4468102d30 0x7f4468100990 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:19.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.285+0000 7f446fee6640 1 -- 172.21.15.120:0/2301947598 >> 172.21.15.120:0/2301947598 conn(0x7f44680fc7d0 msgr2=0x7f4468105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:19.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.285+0000 7f446fee6640 1 -- 172.21.15.120:0/2301947598 shutdown_connections 2024-09-19T09:21:19.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:19.285+0000 7f446fee6640 1 -- 172.21.15.120:0/2301947598 wait complete. 2024-09-19T09:21:19.370 INFO:tasks.workunit.client.0.smithi120.stdout: 10 129649 20.53 MB/sec execute 194 sec latency 99.327 ms 2024-09-19T09:21:20.090 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:21:20.091 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:21:20.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:20 smithi120 ceph-mon[31150]: pgmap v899: 97 pgs: 97 active+clean; 369 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 7.5 MiB/s wr, 501 op/s 2024-09-19T09:21:20.370 INFO:tasks.workunit.client.0.smithi120.stdout: 10 130031 20.49 MB/sec execute 195 sec latency 240.873 ms 2024-09-19T09:21:20.409 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:21:20.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:20 smithi196 ceph-mon[35150]: pgmap v899: 97 pgs: 97 active+clean; 369 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 7.5 MiB/s wr, 501 op/s 2024-09-19T09:21:20.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:20 smithi153 ceph-mon[35071]: pgmap v899: 97 pgs: 97 active+clean; 369 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 7.5 MiB/s wr, 501 op/s 2024-09-19T09:21:21.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.188+0000 7f822e148640 1 -- 172.21.15.120:0/466284761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82281027a0 msgr2=0x7f82281067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:21.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.188+0000 7f822e148640 1 --2- 172.21.15.120:0/466284761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82281027a0 0x7f82281067f0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f821c009900 tx=0x7f821c02a410 comp rx=0 tx=0).stop 2024-09-19T09:21:21.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.188+0000 7f822e148640 1 -- 172.21.15.120:0/466284761 shutdown_connections 2024-09-19T09:21:21.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.188+0000 7f822e148640 1 --2- 172.21.15.120:0/466284761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8228106f20 0x7f822810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:21.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.188+0000 7f822e148640 1 --2- 172.21.15.120:0/466284761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82281027a0 0x7f82281067f0 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:21.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.188+0000 7f822e148640 1 --2- 172.21.15.120:0/466284761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8228101df0 0x7f82281021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:21.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.188+0000 7f822e148640 1 -- 172.21.15.120:0/466284761 >> 172.21.15.120:0/466284761 conn(0x7f82280fd8f0 msgr2=0x7f82280ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:21.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.188+0000 7f822e148640 1 -- 172.21.15.120:0/466284761 shutdown_connections 2024-09-19T09:21:21.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.188+0000 7f822e148640 1 -- 172.21.15.120:0/466284761 wait complete. 2024-09-19T09:21:21.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.188+0000 7f822e148640 1 Processor -- start 2024-09-19T09:21:21.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.189+0000 7f822e148640 1 -- start start 2024-09-19T09:21:21.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.189+0000 7f822e148640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8228101df0 0x7f82281984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:21.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.189+0000 7f822e148640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82281027a0 0x7f8228198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:21.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.189+0000 7f822e148640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8228106f20 0x7f822819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:21.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.189+0000 7f822e148640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8228111f80 con 0x7f82281027a0 2024-09-19T09:21:21.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.189+0000 7f822e148640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8228111e00 con 0x7f8228106f20 2024-09-19T09:21:21.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.189+0000 7f822e148640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8228112100 con 0x7f8228101df0 2024-09-19T09:21:21.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.189+0000 7f8226ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82281027a0 0x7f8228198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:21.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.189+0000 7f8226ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82281027a0 0x7f8228198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48836/0 (socket says 172.21.15.120:48836) 2024-09-19T09:21:21.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.189+0000 7f8226ffd640 1 -- 172.21.15.120:0/1151925556 learned_addr learned my addr 172.21.15.120:0/1151925556 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:21.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.189+0000 7f8227fff640 1 --2- 172.21.15.120:0/1151925556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8228106f20 0x7f822819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:21.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.189+0000 7f82277fe640 1 --2- 172.21.15.120:0/1151925556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8228101df0 0x7f82281984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:21.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.189+0000 7f8226ffd640 1 -- 172.21.15.120:0/1151925556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8228101df0 msgr2=0x7f82281984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:21.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.189+0000 7f8226ffd640 1 --2- 172.21.15.120:0/1151925556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8228101df0 0x7f82281984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:21.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.189+0000 7f8226ffd640 1 -- 172.21.15.120:0/1151925556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8228106f20 msgr2=0x7f822819cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:21.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.189+0000 7f8226ffd640 1 --2- 172.21.15.120:0/1151925556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8228106f20 0x7f822819cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:21.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.189+0000 7f8226ffd640 1 -- 172.21.15.120:0/1151925556 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f822819d470 con 0x7f82281027a0 2024-09-19T09:21:21.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.189+0000 7f8226ffd640 1 --2- 172.21.15.120:0/1151925556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82281027a0 0x7f8228198a00 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f821c006010 tx=0x7f821c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:21.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.190+0000 7f8224ff9640 1 -- 172.21.15.120:0/1151925556 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c03f030 con 0x7f82281027a0 2024-09-19T09:21:21.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.190+0000 7f822e148640 1 -- 172.21.15.120:0/1151925556 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f822819d700 con 0x7f82281027a0 2024-09-19T09:21:21.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.190+0000 7f8224ff9640 1 -- 172.21.15.120:0/1151925556 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f821c003ce0 con 0x7f82281027a0 2024-09-19T09:21:21.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.190+0000 7f8224ff9640 1 -- 172.21.15.120:0/1151925556 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c035070 con 0x7f82281027a0 2024-09-19T09:21:21.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.190+0000 7f822e148640 1 -- 172.21.15.120:0/1151925556 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82281a5040 con 0x7f82281027a0 2024-09-19T09:21:21.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.191+0000 7f822e148640 1 -- 172.21.15.120:0/1151925556 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82281021d0 con 0x7f82281027a0 2024-09-19T09:21:21.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.192+0000 7f8224ff9640 1 -- 172.21.15.120:0/1151925556 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f821c042030 con 0x7f82281027a0 2024-09-19T09:21:21.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.192+0000 7f8224ff9640 1 --2- 172.21.15.120:0/1151925556 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f82100a3240 0x7f82100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:21.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.193+0000 7f8224ff9640 1 -- 172.21.15.120:0/1151925556 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f821c0efdc0 con 0x7f82281027a0 2024-09-19T09:21:21.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.193+0000 7f82277fe640 1 --2- 172.21.15.120:0/1151925556 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f82100a3240 0x7f82100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:21.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.193+0000 7f82277fe640 1 --2- 172.21.15.120:0/1151925556 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f82100a3240 0x7f82100a5700 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8218009bd0 tx=0x7f8218005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:21.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.197+0000 7f8224ff9640 1 -- 172.21.15.120:0/1151925556 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f821c0bc550 con 0x7f82281027a0 2024-09-19T09:21:21.370 INFO:tasks.workunit.client.0.smithi120.stdout: 10 130686 20.57 MB/sec execute 196 sec latency 70.164 ms 2024-09-19T09:21:21.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.421+0000 7f822e148640 1 -- 172.21.15.120:0/1151925556 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f822810a160 con 0x7f82281027a0 2024-09-19T09:21:21.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.422+0000 7f8224ff9640 1 -- 172.21.15.120:0/1151925556 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11988 (secure 0 0 0) 0x7f821c0c0850 con 0x7f82281027a0 2024-09-19T09:21:21.425 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:21:21.425 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":39,"btime":"2024-09-19T09:18:53:120858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:18:53.070639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:21:21.425 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 39 2024-09-19T09:21:21.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.425+0000 7f822e148640 1 -- 172.21.15.120:0/1151925556 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f82100a3240 msgr2=0x7f82100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:21.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.425+0000 7f822e148640 1 --2- 172.21.15.120:0/1151925556 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f82100a3240 0x7f82100a5700 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8218009bd0 tx=0x7f8218005f90 comp rx=0 tx=0).stop 2024-09-19T09:21:21.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.425+0000 7f822e148640 1 -- 172.21.15.120:0/1151925556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82281027a0 msgr2=0x7f8228198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:21.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.425+0000 7f822e148640 1 --2- 172.21.15.120:0/1151925556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82281027a0 0x7f8228198a00 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f821c006010 tx=0x7f821c0042f0 comp rx=0 tx=0).stop 2024-09-19T09:21:21.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.425+0000 7f822e148640 1 -- 172.21.15.120:0/1151925556 shutdown_connections 2024-09-19T09:21:21.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.425+0000 7f822e148640 1 --2- 172.21.15.120:0/1151925556 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f82100a3240 0x7f82100a5700 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:21.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.425+0000 7f822e148640 1 --2- 172.21.15.120:0/1151925556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8228106f20 0x7f822819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:21.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.425+0000 7f822e148640 1 --2- 172.21.15.120:0/1151925556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82281027a0 0x7f8228198a00 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:21.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.425+0000 7f822e148640 1 --2- 172.21.15.120:0/1151925556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8228101df0 0x7f82281984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:21.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.425+0000 7f822e148640 1 -- 172.21.15.120:0/1151925556 >> 172.21.15.120:0/1151925556 conn(0x7f82280fd8f0 msgr2=0x7f82280fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:21.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.425+0000 7f822e148640 1 -- 172.21.15.120:0/1151925556 shutdown_connections 2024-09-19T09:21:21.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:21.425+0000 7f822e148640 1 -- 172.21.15.120:0/1151925556 wait complete. 2024-09-19T09:21:21.743 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:21 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1151925556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:21:21.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:21 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1151925556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:21:21.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:21 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1151925556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:21:22.356 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:21:22.370 INFO:tasks.workunit.client.0.smithi120.stdout: 10 131342 20.62 MB/sec execute 197 sec latency 44.564 ms 2024-09-19T09:21:22.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:22 smithi120 ceph-mon[31150]: pgmap v900: 97 pgs: 97 active+clean; 373 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 7.9 MiB/s wr, 586 op/s 2024-09-19T09:21:22.683 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:21:22.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:22 smithi196 ceph-mon[35150]: pgmap v900: 97 pgs: 97 active+clean; 373 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 7.9 MiB/s wr, 586 op/s 2024-09-19T09:21:22.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:22 smithi153 ceph-mon[35071]: pgmap v900: 97 pgs: 97 active+clean; 373 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 7.9 MiB/s wr, 586 op/s 2024-09-19T09:21:23.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.206+0000 7fbaa9780640 1 -- 172.21.15.120:0/4152880571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaa410c160 msgr2=0x7fbaa410e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:23.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.206+0000 7fbaa9780640 1 --2- 172.21.15.120:0/4152880571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaa410c160 0x7fbaa410e600 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fba9c009ca0 tx=0x7fba9c02a950 comp rx=0 tx=0).stop 2024-09-19T09:21:23.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.206+0000 7fbaa9780640 1 -- 172.21.15.120:0/4152880571 shutdown_connections 2024-09-19T09:21:23.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.206+0000 7fbaa9780640 1 --2- 172.21.15.120:0/4152880571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaa410c160 0x7fbaa410e600 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.206+0000 7fbaa9780640 1 --2- 172.21.15.120:0/4152880571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaa4103650 0x7fbaa410bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.206+0000 7fbaa9780640 1 --2- 172.21.15.120:0/4152880571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaa4102d30 0x7fbaa4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.206+0000 7fbaa9780640 1 -- 172.21.15.120:0/4152880571 >> 172.21.15.120:0/4152880571 conn(0x7fbaa40fc7d0 msgr2=0x7fbaa40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:23.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.206+0000 7fbaa9780640 1 -- 172.21.15.120:0/4152880571 shutdown_connections 2024-09-19T09:21:23.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.206+0000 7fbaa9780640 1 -- 172.21.15.120:0/4152880571 wait complete. 2024-09-19T09:21:23.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.207+0000 7fbaa9780640 1 Processor -- start 2024-09-19T09:21:23.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.207+0000 7fbaa9780640 1 -- start start 2024-09-19T09:21:23.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.207+0000 7fbaa9780640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaa4102d30 0x7fbaa4198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:23.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.207+0000 7fbaa9780640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaa4103650 0x7fbaa4198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:23.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.207+0000 7fbaa9780640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaa410c160 0x7fbaa419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:23.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.207+0000 7fbaa9780640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa4111f40 con 0x7fbaa4102d30 2024-09-19T09:21:23.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.207+0000 7fbaa9780640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa4111dc0 con 0x7fbaa4103650 2024-09-19T09:21:23.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.207+0000 7fbaa9780640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa41120c0 con 0x7fbaa410c160 2024-09-19T09:21:23.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.207+0000 7fbaa2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaa4102d30 0x7fbaa4198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:23.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.207+0000 7fba9bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaa4103650 0x7fbaa4198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:23.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.207+0000 7fba9bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaa4103650 0x7fbaa4198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:39278/0 (socket says 172.21.15.120:39278) 2024-09-19T09:21:23.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.207+0000 7fba9bfff640 1 -- 172.21.15.120:0/2022117330 learned_addr learned my addr 172.21.15.120:0/2022117330 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:23.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.207+0000 7fbaa2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaa4102d30 0x7fbaa4198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39826/0 (socket says 172.21.15.120:39826) 2024-09-19T09:21:23.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.208+0000 7fbaa37fe640 1 --2- 172.21.15.120:0/2022117330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaa410c160 0x7fbaa419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:23.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.208+0000 7fba9bfff640 1 -- 172.21.15.120:0/2022117330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaa410c160 msgr2=0x7fbaa419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:23.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.208+0000 7fba9bfff640 1 --2- 172.21.15.120:0/2022117330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaa410c160 0x7fbaa419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.208+0000 7fba9bfff640 1 -- 172.21.15.120:0/2022117330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaa4102d30 msgr2=0x7fbaa4198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:23.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.208+0000 7fba9bfff640 1 --2- 172.21.15.120:0/2022117330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaa4102d30 0x7fbaa4198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.208+0000 7fba9bfff640 1 -- 172.21.15.120:0/2022117330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa419d3b0 con 0x7fbaa4103650 2024-09-19T09:21:23.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.208+0000 7fbaa2ffd640 1 --2- 172.21.15.120:0/2022117330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaa4102d30 0x7fbaa4198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:21:23.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.208+0000 7fba9bfff640 1 --2- 172.21.15.120:0/2022117330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaa4103650 0x7fbaa4198940 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00e9f0 tx=0x7fba8c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:23.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.208+0000 7fbaa0ff9640 1 -- 172.21.15.120:0/2022117330 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c00ccc0 con 0x7fbaa4103650 2024-09-19T09:21:23.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.208+0000 7fbaa0ff9640 1 -- 172.21.15.120:0/2022117330 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba8c00ce60 con 0x7fbaa4103650 2024-09-19T09:21:23.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.209+0000 7fbaa0ff9640 1 -- 172.21.15.120:0/2022117330 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c010690 con 0x7fbaa4103650 2024-09-19T09:21:23.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.209+0000 7fbaa9780640 1 -- 172.21.15.120:0/2022117330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa419d6a0 con 0x7fbaa4103650 2024-09-19T09:21:23.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.209+0000 7fbaa9780640 1 -- 172.21.15.120:0/2022117330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa41a4f80 con 0x7fbaa4103650 2024-09-19T09:21:23.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.211+0000 7fbaa0ff9640 1 -- 172.21.15.120:0/2022117330 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fba8c0046c0 con 0x7fbaa4103650 2024-09-19T09:21:23.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.212+0000 7fbaabb59740 1 Processor -- start 2024-09-19T09:21:23.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.212+0000 7fbaabb59740 1 -- start start 2024-09-19T09:21:23.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.212+0000 7fbaa0ff9640 1 --2- 172.21.15.120:0/2022117330 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fba900a3240 0x7fba900a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:23.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.212+0000 7fbaa37fe640 1 --2- 172.21.15.120:0/2022117330 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fba900a3240 0x7fba900a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:23.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.212+0000 7fbaa0ff9640 1 -- 172.21.15.120:0/2022117330 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fba8c0cfc20 con 0x7fbaa4103650 2024-09-19T09:21:23.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.213+0000 7fbaabb59740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55798d134f50 0x55798d1334a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:23.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.213+0000 7fbaabb59740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d1339e0 0x55798d133e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:23.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.213+0000 7fbaabb59740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55798d13e780 0x55798d140b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:23.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.213+0000 7fbaabb59740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55798d0952f0 con 0x55798d13e780 2024-09-19T09:21:23.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.213+0000 7fbaabb59740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55798d03ccf0 con 0x55798d1339e0 2024-09-19T09:21:23.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.213+0000 7fbaabb59740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55798d0912b0 con 0x55798d134f50 2024-09-19T09:21:23.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.213+0000 7fba9bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d1339e0 0x55798d133e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:23.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.213+0000 7fba9bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d1339e0 0x55798d133e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:39292/0 (socket says 172.21.15.120:39292) 2024-09-19T09:21:23.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.213+0000 7fbaa2ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55798d134f50 0x55798d1334a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:23.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.213+0000 7fba9bfff640 1 -- 172.21.15.120:0/581275805 learned_addr learned my addr 172.21.15.120:0/581275805 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:23.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.213+0000 7fbaa37fe640 1 --2- 172.21.15.120:0/581275805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55798d13e780 0x55798d140b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:23.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.213+0000 7fbaa37fe640 1 --2- 172.21.15.120:0/2022117330 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fba900a3240 0x7fba900a5700 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fba9c005e30 tx=0x7fba9c02ff90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:23.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.213+0000 7fba9bfff640 1 -- 172.21.15.120:0/581275805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55798d134f50 msgr2=0x55798d1334a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:23.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.213+0000 7fba9bfff640 1 --2- 172.21.15.120:0/581275805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55798d134f50 0x55798d1334a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.213+0000 7fba9bfff640 1 -- 172.21.15.120:0/581275805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55798d13e780 msgr2=0x55798d140b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:23.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.213+0000 7fba9bfff640 1 --2- 172.21.15.120:0/581275805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55798d13e780 0x55798d140b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.213+0000 7fba9bfff640 1 -- 172.21.15.120:0/581275805 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55798d0213a0 con 0x55798d1339e0 2024-09-19T09:21:23.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.213+0000 7fba9bfff640 1 --2- 172.21.15.120:0/581275805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d1339e0 0x55798d133e20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0986a0 tx=0x7fba8c09bc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f38d0d6494a41882 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:23.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.213+0000 7fba997fa640 1 -- 172.21.15.120:0/581275805 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c0b5070 con 0x55798d1339e0 2024-09-19T09:21:23.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.214+0000 7fba997fa640 1 -- 172.21.15.120:0/581275805 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba8c0b5210 con 0x55798d1339e0 2024-09-19T09:21:23.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.214+0000 7fba997fa640 1 -- 172.21.15.120:0/581275805 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c0a03d0 con 0x55798d1339e0 2024-09-19T09:21:23.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.214+0000 7fbaabb59740 1 -- 172.21.15.120:0/581275805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d1339e0 msgr2=0x55798d133e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:23.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.214+0000 7fbaabb59740 1 --2- 172.21.15.120:0/581275805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d1339e0 0x55798d133e20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0986a0 tx=0x7fba8c09bc60 comp rx=0 tx=0).stop 2024-09-19T09:21:23.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.214+0000 7fbaabb59740 1 -- 172.21.15.120:0/581275805 shutdown_connections 2024-09-19T09:21:23.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.214+0000 7fbaabb59740 1 --2- 172.21.15.120:0/581275805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55798d13e780 0x55798d140b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.214+0000 7fbaabb59740 1 --2- 172.21.15.120:0/581275805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d1339e0 0x55798d133e20 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.214+0000 7fbaabb59740 1 --2- 172.21.15.120:0/581275805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55798d134f50 0x55798d1334a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.214+0000 7fbaabb59740 1 -- 172.21.15.120:0/581275805 >> 172.21.15.120:0/581275805 conn(0x55798d0f8310 msgr2=0x55798d130070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:23.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.214+0000 7fbaabb59740 1 -- 172.21.15.120:0/581275805 shutdown_connections 2024-09-19T09:21:23.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.214+0000 7fbaabb59740 1 -- 172.21.15.120:0/581275805 wait complete. 2024-09-19T09:21:23.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.215+0000 7fbaabb59740 1 -- start start 2024-09-19T09:21:23.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.215+0000 7fbaabb59740 1 Processor -- start 2024-09-19T09:21:23.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.215+0000 7fbaabb59740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d1339e0 0x55798d13e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:23.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.215+0000 7fbaabb59740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55798d134f50 0x55798d13b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:23.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.215+0000 7fbaabb59740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55798d13e780 0x55798d13bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:23.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.215+0000 7fbaabb59740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55798d141480 con 0x55798d13e780 2024-09-19T09:21:23.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.215+0000 7fbaabb59740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55798d050060 con 0x55798d1339e0 2024-09-19T09:21:23.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.215+0000 7fbaabb59740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55798d141600 con 0x55798d134f50 2024-09-19T09:21:23.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.215+0000 7fbaabb59740 10 client.0 _ll_register_callbacks cb 0x55798cda38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:23.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.215+0000 7fbaa2ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d1339e0 0x55798d13e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:23.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.215+0000 7fbaa2ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d1339e0 0x55798d13e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:39302/0 (socket says 172.21.15.120:39302) 2024-09-19T09:21:23.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.215+0000 7fbaa2ffd640 1 -- 172.21.15.120:0/2454951121 learned_addr learned my addr 172.21.15.120:0/2454951121 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:23.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.215+0000 7fba9bfff640 1 --2- 172.21.15.120:0/2454951121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55798d134f50 0x55798d13b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:23.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.216+0000 7fbaa37fe640 1 --2- 172.21.15.120:0/2454951121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55798d13e780 0x55798d13bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:23.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.216+0000 7fbaa2ffd640 1 -- 172.21.15.120:0/2454951121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55798d134f50 msgr2=0x55798d13b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:23.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.216+0000 7fbaa2ffd640 1 --2- 172.21.15.120:0/2454951121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55798d134f50 0x55798d13b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.216+0000 7fbaa2ffd640 1 -- 172.21.15.120:0/2454951121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55798d13e780 msgr2=0x55798d13bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:23.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.216+0000 7fbaa2ffd640 1 --2- 172.21.15.120:0/2454951121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55798d13e780 0x55798d13bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.216+0000 7fbaa2ffd640 1 -- 172.21.15.120:0/2454951121 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55798ce063c0 con 0x55798d1339e0 2024-09-19T09:21:23.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.216+0000 7fba9bfff640 1 --2- 172.21.15.120:0/2454951121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55798d134f50 0x55798d13b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:21:23.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.216+0000 7fbaa2ffd640 1 --2- 172.21.15.120:0/2454951121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d1339e0 0x55798d13e1a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fba94011830 tx=0x7fba94011cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14cd58fa97e43ee6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:23.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.216+0000 7fba8a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:21:23.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.216+0000 7fba8a7fc640 1 -- 172.21.15.120:0/2454951121 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba94010ba0 con 0x55798d1339e0 2024-09-19T09:21:23.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.216+0000 7fba8a7fc640 1 -- 172.21.15.120:0/2454951121 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fba94010d40 con 0x55798d1339e0 2024-09-19T09:21:23.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.216+0000 7fbaabb59740 1 -- 172.21.15.120:0/2454951121 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba8c0b5210 con 0x55798d1339e0 2024-09-19T09:21:23.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.216+0000 7fba8a7fc640 1 -- 172.21.15.120:0/2454951121 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fba94004020 con 0x55798d1339e0 2024-09-19T09:21:23.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.216+0000 7fbaabb59740 10 client.30395 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:23.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.216+0000 7fbaabb59740 1 -- 172.21.15.120:0/2454951121 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55798d14cf90 con 0x55798d1339e0 2024-09-19T09:21:23.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.217+0000 7fba8a7fc640 1 -- 172.21.15.120:0/2454951121 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba8c0b5210 con 0x55798d1339e0 2024-09-19T09:21:23.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.217+0000 7fba8a7fc640 1 -- 172.21.15.120:0/2454951121 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fba9401a410 con 0x55798d1339e0 2024-09-19T09:21:23.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.217+0000 7fbaabb59740 10 client.30395 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:23.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.217+0000 7fbaabb59740 10 client.30395 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:21:23.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.217+0000 7fbaabb59740 20 client.30395 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:23.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.217+0000 7fbaabb59740 1 --2- 172.21.15.120:0/2454951121 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55798d14e450 0x55798d1508d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:23.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.217+0000 7fbaabb59740 4 client.30395 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:21:23.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.217+0000 7fbaabb59740 1 -- 172.21.15.120:0/2454951121 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55798d0a2860 con 0x55798d14e450 2024-09-19T09:21:23.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.217+0000 7fba9bfff640 1 --2- 172.21.15.120:0/2454951121 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55798d14e450 0x55798d1508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:23.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.218+0000 7fba9bfff640 1 --2- 172.21.15.120:0/2454951121 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55798d14e450 0x55798d1508d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3d431e1d8dd0d0d0 server_cookie=b6fc77a3963608a6 in_seq=0 out_seq=0 2024-09-19T09:21:23.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.218+0000 7fba8a7fc640 10 client.30395 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:21:23.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.218+0000 7fba8a7fc640 1 -- 172.21.15.120:0/2454951121 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55798d0a2860 con 0x55798d14e450 2024-09-19T09:21:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.218+0000 7fba8a7fc640 10 client.30395 handle_command_reply: tid=0 2024-09-19T09:21:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.218+0000 7fbaabb59740 1 client.30395 shutdown 2024-09-19T09:21:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.218+0000 7fbaabb59740 1 -- 172.21.15.120:0/2454951121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d1339e0 msgr2=0x55798d13e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.218+0000 7fbaabb59740 1 --2- 172.21.15.120:0/2454951121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d1339e0 0x55798d13e1a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fba94011830 tx=0x7fba94011cf0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.218+0000 7fbaabb59740 1 -- 172.21.15.120:0/2454951121 shutdown_connections 2024-09-19T09:21:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.218+0000 7fbaabb59740 1 --2- 172.21.15.120:0/2454951121 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55798d14e450 0x55798d1508d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.218+0000 7fbaabb59740 1 --2- 172.21.15.120:0/2454951121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55798d13e780 0x55798d13bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.219+0000 7fbaabb59740 1 --2- 172.21.15.120:0/2454951121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55798d134f50 0x55798d13b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.219+0000 7fba8a7fc640 0 client.30395 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:21:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.219+0000 7fbaabb59740 1 --2- 172.21.15.120:0/2454951121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d1339e0 0x55798d13e1a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.219+0000 7fbaabb59740 1 -- 172.21.15.120:0/2454951121 >> 172.21.15.120:0/2454951121 conn(0x55798d0f8310 msgr2=0x55798d135d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.219+0000 7fbaabb59740 1 -- 172.21.15.120:0/2454951121 shutdown_connections 2024-09-19T09:21:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.219+0000 7fbaabb59740 1 -- 172.21.15.120:0/2454951121 wait complete. 2024-09-19T09:21:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.219+0000 7fbaabb59740 20 client.30395 trim_cache size 0 max 16384 2024-09-19T09:21:23.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.232+0000 7fbaabb59740 1 Processor -- start 2024-09-19T09:21:23.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.232+0000 7fbaabb59740 1 -- start start 2024-09-19T09:21:23.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.233+0000 7fbaabb59740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d1339e0 0x55798d14a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:23.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.233+0000 7fbaabb59740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55798d134f50 0x55798d136530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:23.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.233+0000 7fbaabb59740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55798d13e780 0x55798d136a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:23.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.233+0000 7fbaabb59740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa41ab400 con 0x55798d13e780 2024-09-19T09:21:23.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.233+0000 7fbaabb59740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa41acf40 con 0x55798d1339e0 2024-09-19T09:21:23.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.233+0000 7fbaabb59740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa41abad0 con 0x55798d134f50 2024-09-19T09:21:23.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.233+0000 7fbaa37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55798d13e780 0x55798d136a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:23.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.233+0000 7fbaa2ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d1339e0 0x55798d14a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:23.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.233+0000 7fbaa2ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d1339e0 0x55798d14a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:39304/0 (socket says 172.21.15.120:39304) 2024-09-19T09:21:23.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.233+0000 7fbaa2ffd640 1 -- 172.21.15.120:0/2577430877 learned_addr learned my addr 172.21.15.120:0/2577430877 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:23.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.233+0000 7fba9bfff640 1 --2- 172.21.15.120:0/2577430877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55798d134f50 0x55798d136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:23.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.233+0000 7fbaa37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55798d13e780 0x55798d136a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39846/0 (socket says 172.21.15.120:39846) 2024-09-19T09:21:23.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.233+0000 7fbaa2ffd640 1 -- 172.21.15.120:0/2577430877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55798d134f50 msgr2=0x55798d136530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:23.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.233+0000 7fbaa2ffd640 1 --2- 172.21.15.120:0/2577430877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55798d134f50 0x55798d136530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.233+0000 7fbaa2ffd640 1 -- 172.21.15.120:0/2577430877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55798d13e780 msgr2=0x55798d136a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:23.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.233+0000 7fbaa2ffd640 1 --2- 172.21.15.120:0/2577430877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55798d13e780 0x55798d136a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.233+0000 7fbaa2ffd640 1 -- 172.21.15.120:0/2577430877 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55798d13e510 con 0x55798d1339e0 2024-09-19T09:21:23.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.233+0000 7fbaa2ffd640 1 --2- 172.21.15.120:0/2577430877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d1339e0 0x55798d14a720 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fba94000c00 tx=0x7fba94005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aca34ab7f6727f76 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:23.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.234+0000 7fba98ff9640 1 -- 172.21.15.120:0/2577430877 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba94029040 con 0x55798d1339e0 2024-09-19T09:21:23.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.234+0000 7fba98ff9640 1 -- 172.21.15.120:0/2577430877 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba94005520 con 0x55798d1339e0 2024-09-19T09:21:23.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.234+0000 7fba98ff9640 1 -- 172.21.15.120:0/2577430877 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba940041c0 con 0x55798d1339e0 2024-09-19T09:21:23.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.234+0000 7fbaabb59740 1 -- 172.21.15.120:0/2577430877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d1339e0 msgr2=0x55798d14a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:23.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.234+0000 7fbaabb59740 1 --2- 172.21.15.120:0/2577430877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d1339e0 0x55798d14a720 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fba94000c00 tx=0x7fba94005bd0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.235+0000 7fbaabb59740 1 -- 172.21.15.120:0/2577430877 shutdown_connections 2024-09-19T09:21:23.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.235+0000 7fbaabb59740 1 --2- 172.21.15.120:0/2577430877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55798d13e780 0x55798d136a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.235+0000 7fbaabb59740 1 --2- 172.21.15.120:0/2577430877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55798d134f50 0x55798d136530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.235+0000 7fbaabb59740 1 --2- 172.21.15.120:0/2577430877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d1339e0 0x55798d14a720 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.235+0000 7fbaabb59740 1 -- 172.21.15.120:0/2577430877 >> 172.21.15.120:0/2577430877 conn(0x55798d0f8310 msgr2=0x55798d13a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:23.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.235+0000 7fbaabb59740 1 -- 172.21.15.120:0/2577430877 shutdown_connections 2024-09-19T09:21:23.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.235+0000 7fbaabb59740 1 -- 172.21.15.120:0/2577430877 wait complete. 2024-09-19T09:21:23.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.235+0000 7fbaabb59740 1 -- start start 2024-09-19T09:21:23.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.235+0000 7fbaabb59740 1 Processor -- start 2024-09-19T09:21:23.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.235+0000 7fbaabb59740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d1339e0 0x55798d146d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:23.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.235+0000 7fbaabb59740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55798d134f50 0x55798d142060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:23.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.235+0000 7fbaabb59740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55798d13e780 0x55798d1425a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:23.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.235+0000 7fbaabb59740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa41ac870 con 0x55798d134f50 2024-09-19T09:21:23.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.235+0000 7fbaabb59740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa41ac1a0 con 0x55798d1339e0 2024-09-19T09:21:23.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.235+0000 7fbaabb59740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa41a93c0 con 0x55798d13e780 2024-09-19T09:21:23.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.235+0000 7fbaabb59740 10 client.0 _ll_register_callbacks cb 0x7fba9400bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:23.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.235+0000 7fba9bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55798d134f50 0x55798d142060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:23.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.235+0000 7fbaa2ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d1339e0 0x55798d146d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:23.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.235+0000 7fba9bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55798d134f50 0x55798d142060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39860/0 (socket says 172.21.15.120:39860) 2024-09-19T09:21:23.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.235+0000 7fba9bfff640 1 -- 172.21.15.120:0/1588711443 learned_addr learned my addr 172.21.15.120:0/1588711443 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:23.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.236+0000 7fbaa37fe640 1 --2- 172.21.15.120:0/1588711443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55798d13e780 0x55798d1425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:23.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.236+0000 7fba9bfff640 1 -- 172.21.15.120:0/1588711443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55798d13e780 msgr2=0x55798d1425a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:23.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.236+0000 7fba9bfff640 1 --2- 172.21.15.120:0/1588711443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55798d13e780 0x55798d1425a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.236+0000 7fba9bfff640 1 -- 172.21.15.120:0/1588711443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d1339e0 msgr2=0x55798d146d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:23.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.236+0000 7fba9bfff640 1 --2- 172.21.15.120:0/1588711443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d1339e0 0x55798d146d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.236+0000 7fba9bfff640 1 -- 172.21.15.120:0/1588711443 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55798d142ae0 con 0x55798d134f50 2024-09-19T09:21:23.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.236+0000 7fba9bfff640 1 --2- 172.21.15.120:0/1588711443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55798d134f50 0x55798d142060 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fba8c09c1d0 tx=0x7fba8c098610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32fdc7891a778034 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:23.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.236+0000 7fba8a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:21:23.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.236+0000 7fba8a7fc640 1 -- 172.21.15.120:0/1588711443 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c0b5040 con 0x55798d134f50 2024-09-19T09:21:23.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.236+0000 7fba8a7fc640 1 -- 172.21.15.120:0/1588711443 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fba8c0cdbe0 con 0x55798d134f50 2024-09-19T09:21:23.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.236+0000 7fbaabb59740 1 -- 172.21.15.120:0/1588711443 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba94005520 con 0x55798d134f50 2024-09-19T09:21:23.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.236+0000 7fba8a7fc640 1 -- 172.21.15.120:0/1588711443 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fba8c09a360 con 0x55798d134f50 2024-09-19T09:21:23.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.236+0000 7fbaabb59740 10 client.21576 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:23.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.236+0000 7fbaabb59740 1 -- 172.21.15.120:0/1588711443 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55798d1b8ec0 con 0x55798d134f50 2024-09-19T09:21:23.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.237+0000 7fba8a7fc640 1 -- 172.21.15.120:0/1588711443 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba94005520 con 0x55798d134f50 2024-09-19T09:21:23.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.237+0000 7fba8a7fc640 1 -- 172.21.15.120:0/1588711443 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fba8c0a4a20 con 0x55798d134f50 2024-09-19T09:21:23.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.237+0000 7fbaabb59740 10 client.21576 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:23.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.237+0000 7fbaabb59740 10 client.21576 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:21:23.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.237+0000 7fbaabb59740 20 client.21576 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:23.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.237+0000 7fbaabb59740 1 --2- 172.21.15.120:0/1588711443 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55798d1b91c0 0x55798d1bb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:23.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.237+0000 7fbaabb59740 4 client.21576 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:21:23.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.237+0000 7fbaabb59740 1 -- 172.21.15.120:0/1588711443 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55798d09cd70 con 0x55798d1b91c0 2024-09-19T09:21:23.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.237+0000 7fbaa2ffd640 1 --2- 172.21.15.120:0/1588711443 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55798d1b91c0 0x55798d1bb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:23.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.237+0000 7fbaa2ffd640 1 --2- 172.21.15.120:0/1588711443 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55798d1b91c0 0x55798d1bb5e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bf036257725004ea server_cookie=5f6664c3f92ad51c in_seq=0 out_seq=0 2024-09-19T09:21:23.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.237+0000 7fba8a7fc640 10 client.21576 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:21:23.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.238+0000 7fba8a7fc640 1 -- 172.21.15.120:0/1588711443 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55798d09cd70 con 0x55798d1b91c0 2024-09-19T09:21:23.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.238+0000 7fba8a7fc640 10 client.21576 handle_command_reply: tid=0 2024-09-19T09:21:23.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.238+0000 7fbaabb59740 1 client.21576 shutdown 2024-09-19T09:21:23.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.238+0000 7fbaabb59740 1 -- 172.21.15.120:0/1588711443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55798d134f50 msgr2=0x55798d142060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:23.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.238+0000 7fbaabb59740 1 --2- 172.21.15.120:0/1588711443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55798d134f50 0x55798d142060 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fba8c09c1d0 tx=0x7fba8c098610 comp rx=0 tx=0).stop 2024-09-19T09:21:23.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.238+0000 7fbaabb59740 1 -- 172.21.15.120:0/1588711443 shutdown_connections 2024-09-19T09:21:23.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.238+0000 7fbaabb59740 1 --2- 172.21.15.120:0/1588711443 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55798d1b91c0 0x55798d1bb5e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.238+0000 7fba8a7fc640 0 client.21576 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:21:23.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.238+0000 7fbaabb59740 1 --2- 172.21.15.120:0/1588711443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55798d13e780 0x55798d1425a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.238+0000 7fbaabb59740 1 --2- 172.21.15.120:0/1588711443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55798d134f50 0x55798d142060 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.238+0000 7fbaabb59740 1 --2- 172.21.15.120:0/1588711443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d1339e0 0x55798d146d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.238+0000 7fbaabb59740 1 -- 172.21.15.120:0/1588711443 >> 172.21.15.120:0/1588711443 conn(0x55798d0f8310 msgr2=0x55798d12a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:23.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.239+0000 7fbaabb59740 1 -- 172.21.15.120:0/1588711443 shutdown_connections 2024-09-19T09:21:23.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.239+0000 7fbaabb59740 1 -- 172.21.15.120:0/1588711443 wait complete. 2024-09-19T09:21:23.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.239+0000 7fbaabb59740 20 client.21576 trim_cache size 0 max 16384 2024-09-19T09:21:23.254 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:21:23.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.240+0000 7fbaa9780640 1 -- 172.21.15.120:0/2022117330 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fba900a3240 msgr2=0x7fba900a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:23.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.240+0000 7fbaa9780640 1 --2- 172.21.15.120:0/2022117330 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fba900a3240 0x7fba900a5700 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fba9c005e30 tx=0x7fba9c02ff90 comp rx=0 tx=0).stop 2024-09-19T09:21:23.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.240+0000 7fbaa9780640 1 -- 172.21.15.120:0/2022117330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaa4103650 msgr2=0x7fbaa4198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:23.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.240+0000 7fbaa9780640 1 --2- 172.21.15.120:0/2022117330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaa4103650 0x7fbaa4198940 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00e9f0 tx=0x7fba8c00eeb0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.240+0000 7fbaa9780640 1 -- 172.21.15.120:0/2022117330 shutdown_connections 2024-09-19T09:21:23.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.240+0000 7fbaa9780640 1 --2- 172.21.15.120:0/2022117330 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fba900a3240 0x7fba900a5700 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.240+0000 7fbaa9780640 1 --2- 172.21.15.120:0/2022117330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaa410c160 0x7fbaa419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.240+0000 7fbaa9780640 1 --2- 172.21.15.120:0/2022117330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaa4103650 0x7fbaa4198940 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.240+0000 7fbaa9780640 1 --2- 172.21.15.120:0/2022117330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaa4102d30 0x7fbaa4198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:23.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.240+0000 7fbaa9780640 1 -- 172.21.15.120:0/2022117330 >> 172.21.15.120:0/2022117330 conn(0x7fbaa40fc7d0 msgr2=0x7fbaa40fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:23.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.240+0000 7fbaa9780640 1 -- 172.21.15.120:0/2022117330 shutdown_connections 2024-09-19T09:21:23.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:23.241+0000 7fbaa9780640 1 -- 172.21.15.120:0/2022117330 wait complete. 2024-09-19T09:21:23.370 INFO:tasks.workunit.client.0.smithi120.stdout: 10 132075 20.74 MB/sec execute 198 sec latency 62.638 ms 2024-09-19T09:21:24.080 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:21:24.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:24 smithi120 ceph-mon[31150]: pgmap v901: 97 pgs: 97 active+clean; 381 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.9 MiB/s wr, 676 op/s 2024-09-19T09:21:24.370 INFO:tasks.workunit.client.0.smithi120.stdout: 10 132795 20.84 MB/sec execute 199 sec latency 48.674 ms 2024-09-19T09:21:24.421 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:21:24.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:24 smithi196 ceph-mon[35150]: pgmap v901: 97 pgs: 97 active+clean; 381 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.9 MiB/s wr, 676 op/s 2024-09-19T09:21:24.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:24 smithi153 ceph-mon[35071]: pgmap v901: 97 pgs: 97 active+clean; 381 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.9 MiB/s wr, 676 op/s 2024-09-19T09:21:25.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.022+0000 7f8908b36640 1 -- 172.21.15.120:0/2327780850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8904102ef0 msgr2=0x7f890410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:25.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.022+0000 7f8908b36640 1 --2- 172.21.15.120:0/2327780850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8904102ef0 0x7f890410da40 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f88fc009ca0 tx=0x7f88fc02a950 comp rx=0 tx=0).stop 2024-09-19T09:21:25.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.023+0000 7f8908b36640 1 -- 172.21.15.120:0/2327780850 shutdown_connections 2024-09-19T09:21:25.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.023+0000 7f8908b36640 1 --2- 172.21.15.120:0/2327780850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8904102ef0 0x7f890410da40 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.023+0000 7f8908b36640 1 --2- 172.21.15.120:0/2327780850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8904102570 0x7f89041029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.023+0000 7f8908b36640 1 --2- 172.21.15.120:0/2327780850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8904100e20 0x7f8904101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.023+0000 7f8908b36640 1 -- 172.21.15.120:0/2327780850 >> 172.21.15.120:0/2327780850 conn(0x7f89040fc7d0 msgr2=0x7f89040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:25.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.023+0000 7f8908b36640 1 -- 172.21.15.120:0/2327780850 shutdown_connections 2024-09-19T09:21:25.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.023+0000 7f8908b36640 1 -- 172.21.15.120:0/2327780850 wait complete. 2024-09-19T09:21:25.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.023+0000 7f8908b36640 1 Processor -- start 2024-09-19T09:21:25.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.023+0000 7f8908b36640 1 -- start start 2024-09-19T09:21:25.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.024+0000 7f8908b36640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8904100e20 0x7f89041992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:25.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.024+0000 7f8908b36640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8904102570 0x7f8904199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:25.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.024+0000 7f8908b36640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8904102ef0 0x7f89041a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:25.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.024+0000 7f8908b36640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8904110f80 con 0x7f8904100e20 2024-09-19T09:21:25.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.024+0000 7f8908b36640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8904110e00 con 0x7f8904102570 2024-09-19T09:21:25.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.024+0000 7f8908b36640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8904111100 con 0x7f8904102ef0 2024-09-19T09:21:25.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.024+0000 7f890259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8904100e20 0x7f89041992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:25.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.024+0000 7f890259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8904100e20 0x7f89041992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39878/0 (socket says 172.21.15.120:39878) 2024-09-19T09:21:25.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.024+0000 7f890259c640 1 -- 172.21.15.120:0/1256715607 learned_addr learned my addr 172.21.15.120:0/1256715607 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:25.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.024+0000 7f8902d9d640 1 --2- 172.21.15.120:0/1256715607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8904102ef0 0x7f89041a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:25.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.024+0000 7f88fbfff640 1 --2- 172.21.15.120:0/1256715607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8904102570 0x7f8904199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:25.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.024+0000 7f8902d9d640 1 -- 172.21.15.120:0/1256715607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8904102570 msgr2=0x7f8904199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:25.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.024+0000 7f8902d9d640 1 --2- 172.21.15.120:0/1256715607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8904102570 0x7f8904199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.024+0000 7f8902d9d640 1 -- 172.21.15.120:0/1256715607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8904100e20 msgr2=0x7f89041992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:25.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.024+0000 7f8902d9d640 1 --2- 172.21.15.120:0/1256715607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8904100e20 0x7f89041992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.024+0000 7f8902d9d640 1 -- 172.21.15.120:0/1256715607 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89041a48b0 con 0x7f8904102ef0 2024-09-19T09:21:25.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.024+0000 7f8902d9d640 1 --2- 172.21.15.120:0/1256715607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8904102ef0 0x7f89041a4370 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f88fc005cc0 tx=0x7f88fc02f3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:25.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.025+0000 7f88fb7fe640 1 -- 172.21.15.120:0/1256715607 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc034040 con 0x7f8904102ef0 2024-09-19T09:21:25.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.025+0000 7f88fb7fe640 1 -- 172.21.15.120:0/1256715607 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88fc030920 con 0x7f8904102ef0 2024-09-19T09:21:25.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.025+0000 7f88fb7fe640 1 -- 172.21.15.120:0/1256715607 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc0045b0 con 0x7f8904102ef0 2024-09-19T09:21:25.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.025+0000 7f8908b36640 1 -- 172.21.15.120:0/1256715607 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89041970e0 con 0x7f8904102ef0 2024-09-19T09:21:25.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.025+0000 7f8908b36640 1 -- 172.21.15.120:0/1256715607 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8904197610 con 0x7f8904102ef0 2024-09-19T09:21:25.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.027+0000 7f88fb7fe640 1 -- 172.21.15.120:0/1256715607 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f88fc037020 con 0x7f8904102ef0 2024-09-19T09:21:25.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.027+0000 7f890af0f740 1 Processor -- start 2024-09-19T09:21:25.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.028+0000 7f890af0f740 1 -- start start 2024-09-19T09:21:25.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.028+0000 7f890af0f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56282279bb40 0x56282279a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:25.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.028+0000 7f890af0f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56282279a5d0 0x56282279aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:25.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.028+0000 7f890af0f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628227a5370 0x5628227a7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:25.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.028+0000 7f890af0f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5628227002f0 con 0x56282279bb40 2024-09-19T09:21:25.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.028+0000 7f890af0f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5628226a7cf0 con 0x56282279a5d0 2024-09-19T09:21:25.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.028+0000 7f890af0f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5628226fc2b0 con 0x5628227a5370 2024-09-19T09:21:25.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.029+0000 7f88fbfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628227a5370 0x5628227a7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:25.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.029+0000 7f88fbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56282279bb40 0x56282279a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:25.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.029+0000 7f88fbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56282279bb40 0x56282279a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39892/0 (socket says 172.21.15.120:39892) 2024-09-19T09:21:25.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.029+0000 7f88fbfff640 1 -- 172.21.15.120:0/3712729725 learned_addr learned my addr 172.21.15.120:0/3712729725 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:25.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.029+0000 7f890259c640 1 --2- 172.21.15.120:0/3712729725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56282279a5d0 0x56282279aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:25.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.029+0000 7f88fbfff640 1 -- 172.21.15.120:0/3712729725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56282279a5d0 msgr2=0x56282279aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:25.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.029+0000 7f88fbfff640 1 --2- 172.21.15.120:0/3712729725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56282279a5d0 0x56282279aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.029+0000 7f88fbfff640 1 -- 172.21.15.120:0/3712729725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56282279bb40 msgr2=0x56282279a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:25.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.029+0000 7f88fbfff640 1 --2- 172.21.15.120:0/3712729725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56282279bb40 0x56282279a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.029+0000 7f88fbfff640 1 -- 172.21.15.120:0/3712729725 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56282268c3a0 con 0x5628227a5370 2024-09-19T09:21:25.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.029+0000 7f88fb7fe640 1 --2- 172.21.15.120:0/1256715607 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f88f00a32c0 0x7f88f00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:25.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.030+0000 7f88fbfff640 1 --2- 172.21.15.120:0/3712729725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628227a5370 0x5628227a7760 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f88ec010620 tx=0x7f88ec0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bd383a0677fde26 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:25.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.030+0000 7f88fb7fe640 1 -- 172.21.15.120:0/1256715607 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f88fc0f05e0 con 0x7f8904102ef0 2024-09-19T09:21:25.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.030+0000 7f890259c640 1 --2- 172.21.15.120:0/1256715607 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f88f00a32c0 0x7f88f00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:25.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.030+0000 7f88f8ff9640 1 -- 172.21.15.120:0/3712729725 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec00d8e0 con 0x5628227a5370 2024-09-19T09:21:25.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.031+0000 7f88f8ff9640 1 -- 172.21.15.120:0/3712729725 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88ec01e420 con 0x5628227a5370 2024-09-19T09:21:25.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.031+0000 7f88f8ff9640 1 -- 172.21.15.120:0/3712729725 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec01d630 con 0x5628227a5370 2024-09-19T09:21:25.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.032+0000 7f890af0f740 1 -- 172.21.15.120:0/3712729725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628227a5370 msgr2=0x5628227a7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:25.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.032+0000 7f890259c640 1 --2- 172.21.15.120:0/1256715607 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f88f00a32c0 0x7f88f00a5780 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x562822797ac0 tx=0x7f88f4019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:25.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.032+0000 7f890af0f740 1 --2- 172.21.15.120:0/3712729725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628227a5370 0x5628227a7760 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f88ec010620 tx=0x7f88ec0046c0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.032+0000 7f890af0f740 1 -- 172.21.15.120:0/3712729725 shutdown_connections 2024-09-19T09:21:25.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.032+0000 7f890af0f740 1 --2- 172.21.15.120:0/3712729725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628227a5370 0x5628227a7760 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.032+0000 7f890af0f740 1 --2- 172.21.15.120:0/3712729725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56282279a5d0 0x56282279aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.032+0000 7f890af0f740 1 --2- 172.21.15.120:0/3712729725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56282279bb40 0x56282279a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.032+0000 7f890af0f740 1 -- 172.21.15.120:0/3712729725 >> 172.21.15.120:0/3712729725 conn(0x562822763310 msgr2=0x562822796c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:25.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.032+0000 7f890af0f740 1 -- 172.21.15.120:0/3712729725 shutdown_connections 2024-09-19T09:21:25.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.032+0000 7f890af0f740 1 -- 172.21.15.120:0/3712729725 wait complete. 2024-09-19T09:21:25.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.032+0000 7f890af0f740 1 -- start start 2024-09-19T09:21:25.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.032+0000 7f890af0f740 1 Processor -- start 2024-09-19T09:21:25.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.032+0000 7f890af0f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56282279a5d0 0x5628227a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:25.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.032+0000 7f890af0f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56282279bb40 0x5628227a2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:25.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.032+0000 7f890af0f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628227a5370 0x5628227a2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:25.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.032+0000 7f890af0f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5628227a8060 con 0x5628227a5370 2024-09-19T09:21:25.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.033+0000 7f890af0f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5628226bb060 con 0x56282279bb40 2024-09-19T09:21:25.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.033+0000 7f890af0f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5628227a81e0 con 0x56282279a5d0 2024-09-19T09:21:25.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.033+0000 7f890af0f740 10 client.0 _ll_register_callbacks cb 0x56282240e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:25.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.033+0000 7f8902d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628227a5370 0x5628227a2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:25.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.033+0000 7f88fbfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56282279bb40 0x5628227a2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:25.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.033+0000 7f8902d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628227a5370 0x5628227a2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39906/0 (socket says 172.21.15.120:39906) 2024-09-19T09:21:25.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.033+0000 7f88fbfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56282279bb40 0x5628227a2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:39330/0 (socket says 172.21.15.120:39330) 2024-09-19T09:21:25.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.033+0000 7f8902d9d640 1 -- 172.21.15.120:0/3683415053 learned_addr learned my addr 172.21.15.120:0/3683415053 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:25.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.033+0000 7f890259c640 1 --2- 172.21.15.120:0/3683415053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56282279a5d0 0x5628227a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:25.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.033+0000 7f8902d9d640 1 -- 172.21.15.120:0/3683415053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56282279a5d0 msgr2=0x5628227a4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:25.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.033+0000 7f8902d9d640 1 --2- 172.21.15.120:0/3683415053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56282279a5d0 0x5628227a4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.033+0000 7f8902d9d640 1 -- 172.21.15.120:0/3683415053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56282279bb40 msgr2=0x5628227a2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:25.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.033+0000 7f8902d9d640 1 --2- 172.21.15.120:0/3683415053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56282279bb40 0x5628227a2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.033+0000 7f8902d9d640 1 -- 172.21.15.120:0/3683415053 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628224713c0 con 0x5628227a5370 2024-09-19T09:21:25.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.033+0000 7f88fbfff640 1 --2- 172.21.15.120:0/3683415053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56282279bb40 0x5628227a2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:21:25.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.033+0000 7f8902d9d640 1 --2- 172.21.15.120:0/3683415053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628227a5370 0x5628227a2800 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f88fc02ae70 tx=0x7f88fc0098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d47610fdd8a12f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:25.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.033+0000 7f88e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:21:25.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.033+0000 7f88e9ffb640 1 -- 172.21.15.120:0/3683415053 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc02c040 con 0x5628227a5370 2024-09-19T09:21:25.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.033+0000 7f88e9ffb640 1 -- 172.21.15.120:0/3683415053 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f88fc0c53a0 con 0x5628227a5370 2024-09-19T09:21:25.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.033+0000 7f890af0f740 1 -- 172.21.15.120:0/3683415053 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88ec01e420 con 0x5628227a5370 2024-09-19T09:21:25.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.034+0000 7f88e9ffb640 1 -- 172.21.15.120:0/3683415053 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f88fc0c0f70 con 0x5628227a5370 2024-09-19T09:21:25.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.034+0000 7f890af0f740 10 client.21588 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:25.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.034+0000 7f890af0f740 1 -- 172.21.15.120:0/3683415053 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5628227b3b70 con 0x5628227a5370 2024-09-19T09:21:25.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.034+0000 7f88e9ffb640 1 -- 172.21.15.120:0/3683415053 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88ec01e420 con 0x5628227a5370 2024-09-19T09:21:25.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.034+0000 7f88e9ffb640 1 -- 172.21.15.120:0/3683415053 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f88fc04d030 con 0x5628227a5370 2024-09-19T09:21:25.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.034+0000 7f890af0f740 10 client.21588 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:25.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.034+0000 7f890af0f740 10 client.21588 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:21:25.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.034+0000 7f890af0f740 20 client.21588 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:25.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.034+0000 7f890af0f740 1 --2- 172.21.15.120:0/3683415053 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5628227b5070 0x5628227b7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:25.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.034+0000 7f890af0f740 4 client.21588 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:21:25.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.034+0000 7f890af0f740 1 -- 172.21.15.120:0/3683415053 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56282270d860 con 0x5628227b5070 2024-09-19T09:21:25.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.034+0000 7f890259c640 1 --2- 172.21.15.120:0/3683415053 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5628227b5070 0x5628227b7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:25.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.035+0000 7f88e9ffb640 10 client.21588 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:21:25.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.035+0000 7f890259c640 1 --2- 172.21.15.120:0/3683415053 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5628227b5070 0x5628227b7480 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a61f530c8c901eb4 server_cookie=93051ff29ed6b5f2 in_seq=0 out_seq=0 2024-09-19T09:21:25.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.036+0000 7f88e9ffb640 1 -- 172.21.15.120:0/3683415053 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56282270d860 con 0x5628227b5070 2024-09-19T09:21:25.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.036+0000 7f88e9ffb640 10 client.21588 handle_command_reply: tid=0 2024-09-19T09:21:25.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.036+0000 7f890af0f740 1 client.21588 shutdown 2024-09-19T09:21:25.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.036+0000 7f890af0f740 1 -- 172.21.15.120:0/3683415053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628227a5370 msgr2=0x5628227a2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:25.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.036+0000 7f890af0f740 1 --2- 172.21.15.120:0/3683415053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628227a5370 0x5628227a2800 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f88fc02ae70 tx=0x7f88fc0098d0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.036+0000 7f890af0f740 1 -- 172.21.15.120:0/3683415053 shutdown_connections 2024-09-19T09:21:25.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.036+0000 7f890af0f740 1 --2- 172.21.15.120:0/3683415053 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5628227b5070 0x5628227b7480 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.037+0000 7f890af0f740 1 --2- 172.21.15.120:0/3683415053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628227a5370 0x5628227a2800 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.037+0000 7f890af0f740 1 --2- 172.21.15.120:0/3683415053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56282279bb40 0x5628227a2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.037+0000 7f890af0f740 1 --2- 172.21.15.120:0/3683415053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56282279a5d0 0x5628227a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.037+0000 7f890af0f740 1 -- 172.21.15.120:0/3683415053 >> 172.21.15.120:0/3683415053 conn(0x562822763310 msgr2=0x56282279c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:25.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.037+0000 7f88e9ffb640 0 client.21588 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:21:25.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.037+0000 7f890af0f740 1 -- 172.21.15.120:0/3683415053 shutdown_connections 2024-09-19T09:21:25.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.037+0000 7f890af0f740 1 -- 172.21.15.120:0/3683415053 wait complete. 2024-09-19T09:21:25.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.037+0000 7f890af0f740 20 client.21588 trim_cache size 0 max 16384 2024-09-19T09:21:25.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.052+0000 7f890af0f740 1 Processor -- start 2024-09-19T09:21:25.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.052+0000 7f890af0f740 1 -- start start 2024-09-19T09:21:25.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.052+0000 7f890af0f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56282279a5d0 0x562822796e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:25.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.052+0000 7f890af0f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56282279bb40 0x56282279d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:25.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.052+0000 7f890af0f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628227a5370 0x56282279d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:25.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.052+0000 7f890af0f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f89041ab2a0 con 0x56282279bb40 2024-09-19T09:21:25.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.052+0000 7f890af0f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f89041acde0 con 0x5628227a5370 2024-09-19T09:21:25.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.052+0000 7f890af0f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f89041ab970 con 0x56282279a5d0 2024-09-19T09:21:25.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.053+0000 7f88fbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56282279bb40 0x56282279d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:25.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.053+0000 7f890259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56282279a5d0 0x562822796e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:25.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.053+0000 7f88fbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56282279bb40 0x56282279d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39912/0 (socket says 172.21.15.120:39912) 2024-09-19T09:21:25.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.053+0000 7f88fbfff640 1 -- 172.21.15.120:0/1647272373 learned_addr learned my addr 172.21.15.120:0/1647272373 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:25.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.053+0000 7f8902d9d640 1 --2- 172.21.15.120:0/1647272373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628227a5370 0x56282279d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:25.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.053+0000 7f88fbfff640 1 -- 172.21.15.120:0/1647272373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56282279a5d0 msgr2=0x562822796e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:25.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.053+0000 7f88fbfff640 1 --2- 172.21.15.120:0/1647272373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56282279a5d0 0x562822796e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.053+0000 7f88fbfff640 1 -- 172.21.15.120:0/1647272373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628227a5370 msgr2=0x56282279d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:25.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.053+0000 7f88fbfff640 1 --2- 172.21.15.120:0/1647272373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628227a5370 0x56282279d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.053+0000 7f88fbfff640 1 -- 172.21.15.120:0/1647272373 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628227a05c0 con 0x56282279bb40 2024-09-19T09:21:25.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.053+0000 7f88fbfff640 1 --2- 172.21.15.120:0/1647272373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56282279bb40 0x56282279d120 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x56282279b070 tx=0x7f88ec01ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e97f9ee9cd3ed1f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:25.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.053+0000 7f88ebfff640 1 -- 172.21.15.120:0/1647272373 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec02e070 con 0x56282279bb40 2024-09-19T09:21:25.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.054+0000 7f88ebfff640 1 -- 172.21.15.120:0/1647272373 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88ec00dd90 con 0x56282279bb40 2024-09-19T09:21:25.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.054+0000 7f88ebfff640 1 -- 172.21.15.120:0/1647272373 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec01eb10 con 0x56282279bb40 2024-09-19T09:21:25.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.054+0000 7f890af0f740 1 -- 172.21.15.120:0/1647272373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56282279bb40 msgr2=0x56282279d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:25.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.054+0000 7f890af0f740 1 --2- 172.21.15.120:0/1647272373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56282279bb40 0x56282279d120 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x56282279b070 tx=0x7f88ec01ef10 comp rx=0 tx=0).stop 2024-09-19T09:21:25.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.054+0000 7f890af0f740 1 -- 172.21.15.120:0/1647272373 shutdown_connections 2024-09-19T09:21:25.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.054+0000 7f890af0f740 1 --2- 172.21.15.120:0/1647272373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5628227a5370 0x56282279d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.054+0000 7f890af0f740 1 --2- 172.21.15.120:0/1647272373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56282279bb40 0x56282279d120 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.054+0000 7f890af0f740 1 --2- 172.21.15.120:0/1647272373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56282279a5d0 0x562822796e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.054+0000 7f890af0f740 1 -- 172.21.15.120:0/1647272373 >> 172.21.15.120:0/1647272373 conn(0x562822763310 msgr2=0x562822796700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:25.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.054+0000 7f890af0f740 1 -- 172.21.15.120:0/1647272373 shutdown_connections 2024-09-19T09:21:25.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.054+0000 7f890af0f740 1 -- 172.21.15.120:0/1647272373 wait complete. 2024-09-19T09:21:25.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.055+0000 7f890af0f740 1 -- start start 2024-09-19T09:21:25.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.055+0000 7f890af0f740 1 Processor -- start 2024-09-19T09:21:25.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.055+0000 7f890af0f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56282279a5d0 0x5628227ad650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:25.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.055+0000 7f890af0f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56282279bb40 0x5628227adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:25.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.055+0000 7f890af0f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628227a5370 0x56282279e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:25.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.055+0000 7f890af0f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f89041ac710 con 0x5628227a5370 2024-09-19T09:21:25.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.055+0000 7f890af0f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f89041ac040 con 0x56282279bb40 2024-09-19T09:21:25.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.055+0000 7f890af0f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f89041a9260 con 0x56282279a5d0 2024-09-19T09:21:25.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.055+0000 7f8902d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628227a5370 0x56282279e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:25.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.056+0000 7f890259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56282279a5d0 0x5628227ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:25.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.056+0000 7f8902d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628227a5370 0x56282279e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39916/0 (socket says 172.21.15.120:39916) 2024-09-19T09:21:25.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.056+0000 7f890af0f740 10 client.0 _ll_register_callbacks cb 0x7f88fc0095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:25.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.056+0000 7f8902d9d640 1 -- 172.21.15.120:0/3505619543 learned_addr learned my addr 172.21.15.120:0/3505619543 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:25.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.056+0000 7f88fbfff640 1 --2- 172.21.15.120:0/3505619543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56282279bb40 0x5628227adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:25.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.056+0000 7f8902d9d640 1 -- 172.21.15.120:0/3505619543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56282279a5d0 msgr2=0x5628227ad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:25.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.056+0000 7f8902d9d640 1 --2- 172.21.15.120:0/3505619543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56282279a5d0 0x5628227ad650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.056+0000 7f8902d9d640 1 -- 172.21.15.120:0/3505619543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56282279bb40 msgr2=0x5628227adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:25.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.056+0000 7f8902d9d640 1 --2- 172.21.15.120:0/3505619543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56282279bb40 0x5628227adb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.056+0000 7f8902d9d640 1 -- 172.21.15.120:0/3505619543 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56282279f0b0 con 0x5628227a5370 2024-09-19T09:21:25.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.056+0000 7f890259c640 1 --2- 172.21.15.120:0/3505619543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56282279a5d0 0x5628227ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:21:25.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.056+0000 7f8902d9d640 1 --2- 172.21.15.120:0/3505619543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628227a5370 0x56282279e880 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f88fc0b9020 tx=0x7f88fc0f0540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571ca0eed4a2320a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:25.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.056+0000 7f88e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:21:25.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.056+0000 7f88e9ffb640 1 -- 172.21.15.120:0/3505619543 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc02c040 con 0x5628227a5370 2024-09-19T09:21:25.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.056+0000 7f890af0f740 1 -- 172.21.15.120:0/3505619543 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88ec00dd90 con 0x5628227a5370 2024-09-19T09:21:25.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.056+0000 7f88e9ffb640 1 -- 172.21.15.120:0/3505619543 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f88fc0c55a0 con 0x5628227a5370 2024-09-19T09:21:25.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.057+0000 7f88e9ffb640 1 -- 172.21.15.120:0/3505619543 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f88fc0bc700 con 0x5628227a5370 2024-09-19T09:21:25.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.057+0000 7f890af0f740 10 client.21594 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:25.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.057+0000 7f890af0f740 1 -- 172.21.15.120:0/3505619543 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56282279f0b0 con 0x5628227a5370 2024-09-19T09:21:25.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.057+0000 7f88e9ffb640 1 -- 172.21.15.120:0/3505619543 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88ec00dd90 con 0x5628227a5370 2024-09-19T09:21:25.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.057+0000 7f88e9ffb640 1 -- 172.21.15.120:0/3505619543 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f88fc0a3020 con 0x5628227a5370 2024-09-19T09:21:25.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.057+0000 7f890af0f740 10 client.21594 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:25.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.057+0000 7f890af0f740 10 client.21594 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:21:25.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.057+0000 7f890af0f740 20 client.21594 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:25.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.057+0000 7f890af0f740 1 --2- 172.21.15.120:0/3505619543 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56282281fc10 0x562822822030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:25.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.057+0000 7f890af0f740 4 client.21594 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:21:25.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.057+0000 7f890af0f740 1 -- 172.21.15.120:0/3505619543 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562822707d70 con 0x56282281fc10 2024-09-19T09:21:25.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.057+0000 7f890259c640 1 --2- 172.21.15.120:0/3505619543 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56282281fc10 0x562822822030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:25.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.058+0000 7f890259c640 1 --2- 172.21.15.120:0/3505619543 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56282281fc10 0x562822822030 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c78b6eff081defe2 server_cookie=ec60b541d0b9316 in_seq=0 out_seq=0 2024-09-19T09:21:25.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.058+0000 7f88e9ffb640 10 client.21594 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:21:25.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.059+0000 7f88e9ffb640 1 -- 172.21.15.120:0/3505619543 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562822707d70 con 0x56282281fc10 2024-09-19T09:21:25.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.059+0000 7f88e9ffb640 10 client.21594 handle_command_reply: tid=0 2024-09-19T09:21:25.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.059+0000 7f890af0f740 1 client.21594 shutdown 2024-09-19T09:21:25.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.059+0000 7f890af0f740 1 -- 172.21.15.120:0/3505619543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628227a5370 msgr2=0x56282279e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:25.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.059+0000 7f890af0f740 1 --2- 172.21.15.120:0/3505619543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628227a5370 0x56282279e880 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f88fc0b9020 tx=0x7f88fc0f0540 comp rx=0 tx=0).stop 2024-09-19T09:21:25.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.059+0000 7f890af0f740 1 -- 172.21.15.120:0/3505619543 shutdown_connections 2024-09-19T09:21:25.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.059+0000 7f890af0f740 1 --2- 172.21.15.120:0/3505619543 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56282281fc10 0x562822822030 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.059+0000 7f890af0f740 1 --2- 172.21.15.120:0/3505619543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5628227a5370 0x56282279e880 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.059+0000 7f890af0f740 1 --2- 172.21.15.120:0/3505619543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56282279bb40 0x5628227adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.059+0000 7f88e9ffb640 0 client.21594 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:21:25.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.059+0000 7f890af0f740 1 --2- 172.21.15.120:0/3505619543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56282279a5d0 0x5628227ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.059+0000 7f890af0f740 1 -- 172.21.15.120:0/3505619543 >> 172.21.15.120:0/3505619543 conn(0x562822763310 msgr2=0x5628227954e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:25.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.059+0000 7f890af0f740 1 -- 172.21.15.120:0/3505619543 shutdown_connections 2024-09-19T09:21:25.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.059+0000 7f890af0f740 1 -- 172.21.15.120:0/3505619543 wait complete. 2024-09-19T09:21:25.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.059+0000 7f890af0f740 20 client.21594 trim_cache size 0 max 16384 2024-09-19T09:21:25.068 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:21:25.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.060+0000 7f8908b36640 1 -- 172.21.15.120:0/1256715607 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f88f00a32c0 msgr2=0x7f88f00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:25.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.060+0000 7f8908b36640 1 --2- 172.21.15.120:0/1256715607 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f88f00a32c0 0x7f88f00a5780 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x562822797ac0 tx=0x7f88f4019040 comp rx=0 tx=0).stop 2024-09-19T09:21:25.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.060+0000 7f8908b36640 1 -- 172.21.15.120:0/1256715607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8904102ef0 msgr2=0x7f89041a4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:25.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.060+0000 7f8908b36640 1 --2- 172.21.15.120:0/1256715607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8904102ef0 0x7f89041a4370 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f88fc005cc0 tx=0x7f88fc02f3a0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.060+0000 7f8908b36640 1 -- 172.21.15.120:0/1256715607 shutdown_connections 2024-09-19T09:21:25.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.060+0000 7f8908b36640 1 --2- 172.21.15.120:0/1256715607 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f88f00a32c0 0x7f88f00a5780 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.060+0000 7f8908b36640 1 --2- 172.21.15.120:0/1256715607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8904102ef0 0x7f89041a4370 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.060+0000 7f8908b36640 1 --2- 172.21.15.120:0/1256715607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8904102570 0x7f8904199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.060+0000 7f8908b36640 1 --2- 172.21.15.120:0/1256715607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8904100e20 0x7f89041992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:25.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.060+0000 7f8908b36640 1 -- 172.21.15.120:0/1256715607 >> 172.21.15.120:0/1256715607 conn(0x7f89040fc7d0 msgr2=0x7f8904106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:25.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.060+0000 7f8908b36640 1 -- 172.21.15.120:0/1256715607 shutdown_connections 2024-09-19T09:21:25.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:25.060+0000 7f8908b36640 1 -- 172.21.15.120:0/1256715607 wait complete. 2024-09-19T09:21:25.370 INFO:tasks.workunit.client.0.smithi120.stdout: 10 133215 20.87 MB/sec execute 200 sec latency 270.536 ms 2024-09-19T09:21:26.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:26 smithi120 ceph-mon[31150]: pgmap v902: 97 pgs: 97 active+clean; 385 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.0 MiB/s wr, 602 op/s 2024-09-19T09:21:26.359 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:21:26.370 INFO:tasks.workunit.client.0.smithi120.stdout: 10 133707 20.87 MB/sec execute 201 sec latency 104.316 ms 2024-09-19T09:21:26.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:26 smithi196 ceph-mon[35150]: pgmap v902: 97 pgs: 97 active+clean; 385 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.0 MiB/s wr, 602 op/s 2024-09-19T09:21:26.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:26 smithi153 ceph-mon[35071]: pgmap v902: 97 pgs: 97 active+clean; 385 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.0 MiB/s wr, 602 op/s 2024-09-19T09:21:26.639 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:21:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.182+0000 7fec5e73c640 1 -- 172.21.15.120:0/514057053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec58106a60 msgr2=0x7fec58106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.182+0000 7fec5e73c640 1 --2- 172.21.15.120:0/514057053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec58106a60 0x7fec58106e40 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fec48009930 tx=0x7fec4802a420 comp rx=0 tx=0).stop 2024-09-19T09:21:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.182+0000 7fec5e73c640 1 -- 172.21.15.120:0/514057053 shutdown_connections 2024-09-19T09:21:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.182+0000 7fec5e73c640 1 --2- 172.21.15.120:0/514057053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec58101400 0x7fec5810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.182+0000 7fec5e73c640 1 --2- 172.21.15.120:0/514057053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec58100a60 0x7fec58100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.182+0000 7fec5e73c640 1 --2- 172.21.15.120:0/514057053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec58106a60 0x7fec58106e40 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.182+0000 7fec5e73c640 1 -- 172.21.15.120:0/514057053 >> 172.21.15.120:0/514057053 conn(0x7fec580fc780 msgr2=0x7fec580feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.182+0000 7fec5e73c640 1 -- 172.21.15.120:0/514057053 shutdown_connections 2024-09-19T09:21:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.182+0000 7fec5e73c640 1 -- 172.21.15.120:0/514057053 wait complete. 2024-09-19T09:21:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.183+0000 7fec5e73c640 1 Processor -- start 2024-09-19T09:21:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.183+0000 7fec5e73c640 1 -- start start 2024-09-19T09:21:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.183+0000 7fec5e73c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec58100a60 0x7fec58198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.183+0000 7fec5e73c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec58101400 0x7fec58198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.183+0000 7fec5e73c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec58106a60 0x7fec5819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.183+0000 7fec5e73c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fec58110e60 con 0x7fec58100a60 2024-09-19T09:21:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.183+0000 7fec5e73c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fec58110ce0 con 0x7fec58106a60 2024-09-19T09:21:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.183+0000 7fec5e73c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fec58110fe0 con 0x7fec58101400 2024-09-19T09:21:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.184+0000 7fec57fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec58100a60 0x7fec58198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.184+0000 7fec5ccd9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec58106a60 0x7fec5819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.184+0000 7fec577fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec58101400 0x7fec58198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.184+0000 7fec57fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec58100a60 0x7fec58198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39928/0 (socket says 172.21.15.120:39928) 2024-09-19T09:21:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.184+0000 7fec57fff640 1 -- 172.21.15.120:0/817477591 learned_addr learned my addr 172.21.15.120:0/817477591 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.184+0000 7fec57fff640 1 -- 172.21.15.120:0/817477591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec58101400 msgr2=0x7fec58198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.184+0000 7fec57fff640 1 --2- 172.21.15.120:0/817477591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec58101400 0x7fec58198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.184+0000 7fec57fff640 1 -- 172.21.15.120:0/817477591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec58106a60 msgr2=0x7fec5819cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.184+0000 7fec57fff640 1 --2- 172.21.15.120:0/817477591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec58106a60 0x7fec5819cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.184+0000 7fec57fff640 1 -- 172.21.15.120:0/817477591 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec5819d4c0 con 0x7fec58100a60 2024-09-19T09:21:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.184+0000 7fec5ccd9640 1 --2- 172.21.15.120:0/817477591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec58106a60 0x7fec5819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:21:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.184+0000 7fec57fff640 1 --2- 172.21.15.120:0/817477591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec58100a60 0x7fec58198510 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fec48009230 tx=0x7fec48004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.184+0000 7fec557fa640 1 -- 172.21.15.120:0/817477591 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4803f030 con 0x7fec58100a60 2024-09-19T09:21:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.184+0000 7fec5e73c640 1 -- 172.21.15.120:0/817477591 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec5819d750 con 0x7fec58100a60 2024-09-19T09:21:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.184+0000 7fec557fa640 1 -- 172.21.15.120:0/817477591 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec48004440 con 0x7fec58100a60 2024-09-19T09:21:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.184+0000 7fec557fa640 1 -- 172.21.15.120:0/817477591 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec48035070 con 0x7fec58100a60 2024-09-19T09:21:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.184+0000 7fec5e73c640 1 -- 172.21.15.120:0/817477591 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec580fcda0 con 0x7fec58100a60 2024-09-19T09:21:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.186+0000 7fec557fa640 1 -- 172.21.15.120:0/817477591 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fec48042030 con 0x7fec58100a60 2024-09-19T09:21:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.187+0000 7fec60b15740 1 Processor -- start 2024-09-19T09:21:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.187+0000 7fec60b15740 1 -- start start 2024-09-19T09:21:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.187+0000 7fec557fa640 1 --2- 172.21.15.120:0/817477591 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fec400a3240 0x7fec400a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.187+0000 7fec60b15740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1c891b40 0x557a1c890090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.187+0000 7fec60b15740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1c8905d0 0x557a1c890a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.187+0000 7fec60b15740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a1c89b370 0x557a1c89d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.187+0000 7fec60b15740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557a1c7f62f0 con 0x557a1c891b40 2024-09-19T09:21:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.187+0000 7fec60b15740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557a1c79dcf0 con 0x557a1c8905d0 2024-09-19T09:21:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.187+0000 7fec60b15740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557a1c7f22b0 con 0x557a1c89b370 2024-09-19T09:21:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.187+0000 7fec557fa640 1 -- 172.21.15.120:0/817477591 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fec480efc70 con 0x7fec58100a60 2024-09-19T09:21:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.187+0000 7fec5ccd9640 1 --2- 172.21.15.120:0/817477591 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fec400a3240 0x7fec400a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.187+0000 7fec57fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1c891b40 0x557a1c890090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.187+0000 7fec57fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1c891b40 0x557a1c890090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39932/0 (socket says 172.21.15.120:39932) 2024-09-19T09:21:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.187+0000 7fec57fff640 1 -- 172.21.15.120:0/3097476631 learned_addr learned my addr 172.21.15.120:0/3097476631 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:27.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.187+0000 7fec577fe640 1 --2- 172.21.15.120:0/3097476631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1c8905d0 0x557a1c890a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:27.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.187+0000 7fec5ccd9640 1 --2- 172.21.15.120:0/3097476631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a1c89b370 0x557a1c89d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:27.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.188+0000 7fec57fff640 1 -- 172.21.15.120:0/3097476631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a1c89b370 msgr2=0x557a1c89d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:27.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.188+0000 7fec57fff640 1 --2- 172.21.15.120:0/3097476631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a1c89b370 0x557a1c89d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.188+0000 7fec57fff640 1 -- 172.21.15.120:0/3097476631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1c8905d0 msgr2=0x557a1c890a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:27.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.188+0000 7fec57fff640 1 --2- 172.21.15.120:0/3097476631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1c8905d0 0x557a1c890a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.188+0000 7fec57fff640 1 -- 172.21.15.120:0/3097476631 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a1c7823a0 con 0x557a1c891b40 2024-09-19T09:21:27.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.188+0000 7fec577fe640 1 --2- 172.21.15.120:0/3097476631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1c8905d0 0x557a1c890a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:21:27.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.188+0000 7fec5ccd9640 1 --2- 172.21.15.120:0/817477591 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fec400a3240 0x7fec400a5700 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fec50009fd0 tx=0x7fec500116b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:27.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.188+0000 7fec57fff640 1 --2- 172.21.15.120:0/3097476631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1c891b40 0x557a1c890090 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fec480bc290 tx=0x7fec480bbf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddcd1a1391a539c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:27.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.188+0000 7fec3e7fc640 1 -- 172.21.15.120:0/3097476631 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec48049040 con 0x557a1c891b40 2024-09-19T09:21:27.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.188+0000 7fec3e7fc640 1 -- 172.21.15.120:0/3097476631 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec4804a9b0 con 0x557a1c891b40 2024-09-19T09:21:27.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.188+0000 7fec3e7fc640 1 -- 172.21.15.120:0/3097476631 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec480c4ec0 con 0x557a1c891b40 2024-09-19T09:21:27.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.189+0000 7fec60b15740 1 -- 172.21.15.120:0/3097476631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1c891b40 msgr2=0x557a1c890090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.189+0000 7fec60b15740 1 --2- 172.21.15.120:0/3097476631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1c891b40 0x557a1c890090 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fec480bc290 tx=0x7fec480bbf00 comp rx=0 tx=0).stop 2024-09-19T09:21:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.189+0000 7fec60b15740 1 -- 172.21.15.120:0/3097476631 shutdown_connections 2024-09-19T09:21:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.189+0000 7fec60b15740 1 --2- 172.21.15.120:0/3097476631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a1c89b370 0x557a1c89d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.189+0000 7fec60b15740 1 --2- 172.21.15.120:0/3097476631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1c8905d0 0x557a1c890a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.189+0000 7fec60b15740 1 --2- 172.21.15.120:0/3097476631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1c891b40 0x557a1c890090 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.189+0000 7fec60b15740 1 -- 172.21.15.120:0/3097476631 >> 172.21.15.120:0/3097476631 conn(0x557a1c859310 msgr2=0x557a1c88cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.189+0000 7fec60b15740 1 -- 172.21.15.120:0/3097476631 shutdown_connections 2024-09-19T09:21:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.189+0000 7fec60b15740 1 -- 172.21.15.120:0/3097476631 wait complete. 2024-09-19T09:21:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.189+0000 7fec60b15740 1 -- start start 2024-09-19T09:21:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.190+0000 7fec60b15740 1 Processor -- start 2024-09-19T09:21:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.190+0000 7fec60b15740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1c8905d0 0x557a1c89adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.190+0000 7fec60b15740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1c891b40 0x557a1c898060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.190+0000 7fec60b15740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a1c89b370 0x557a1c8987f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.190+0000 7fec60b15740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557a1c89de30 con 0x557a1c8905d0 2024-09-19T09:21:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.190+0000 7fec60b15740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557a1c7b1060 con 0x557a1c891b40 2024-09-19T09:21:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.190+0000 7fec60b15740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557a1c89dfb0 con 0x557a1c89b370 2024-09-19T09:21:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.190+0000 7fec60b15740 10 client.0 _ll_register_callbacks cb 0x557a1c5048c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.190+0000 7fec57fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1c8905d0 0x557a1c89adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.190+0000 7fec577fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1c891b40 0x557a1c898060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.190+0000 7fec5ccd9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a1c89b370 0x557a1c8987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.190+0000 7fec577fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1c891b40 0x557a1c898060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:39386/0 (socket says 172.21.15.120:39386) 2024-09-19T09:21:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.190+0000 7fec5ccd9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a1c89b370 0x557a1c8987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:46190/0 (socket says 172.21.15.120:46190) 2024-09-19T09:21:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.190+0000 7fec577fe640 1 -- 172.21.15.120:0/1055542084 learned_addr learned my addr 172.21.15.120:0/1055542084 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.190+0000 7fec577fe640 1 -- 172.21.15.120:0/1055542084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a1c89b370 msgr2=0x557a1c8987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.190+0000 7fec577fe640 1 --2- 172.21.15.120:0/1055542084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a1c89b370 0x557a1c8987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.190+0000 7fec577fe640 1 -- 172.21.15.120:0/1055542084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1c8905d0 msgr2=0x557a1c89adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.190+0000 7fec577fe640 1 --2- 172.21.15.120:0/1055542084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1c8905d0 0x557a1c89adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.190+0000 7fec577fe640 1 -- 172.21.15.120:0/1055542084 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a1c5673c0 con 0x557a1c891b40 2024-09-19T09:21:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.191+0000 7fec577fe640 1 --2- 172.21.15.120:0/1055542084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1c891b40 0x557a1c898060 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fec4c012720 tx=0x7fec4c012be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e957b6ec55f965b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.191+0000 7fec377fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:21:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.191+0000 7fec377fe640 1 -- 172.21.15.120:0/1055542084 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c004050 con 0x557a1c891b40 2024-09-19T09:21:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.191+0000 7fec60b15740 1 -- 172.21.15.120:0/1055542084 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec4804a9b0 con 0x557a1c891b40 2024-09-19T09:21:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.191+0000 7fec377fe640 1 -- 172.21.15.120:0/1055542084 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fec4c0041f0 con 0x557a1c891b40 2024-09-19T09:21:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.191+0000 7fec377fe640 1 -- 172.21.15.120:0/1055542084 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fec4c0043f0 con 0x557a1c891b40 2024-09-19T09:21:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.191+0000 7fec60b15740 10 client.30431 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.191+0000 7fec60b15740 1 -- 172.21.15.120:0/1055542084 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557a1c8a9910 con 0x557a1c891b40 2024-09-19T09:21:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.191+0000 7fec377fe640 1 -- 172.21.15.120:0/1055542084 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec4804a9b0 con 0x557a1c891b40 2024-09-19T09:21:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.191+0000 7fec377fe640 1 -- 172.21.15.120:0/1055542084 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fec4c003c40 con 0x557a1c891b40 2024-09-19T09:21:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.192+0000 7fec60b15740 10 client.30431 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.192+0000 7fec60b15740 10 client.30431 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:21:27.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.192+0000 7fec60b15740 20 client.30431 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:27.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.192+0000 7fec60b15740 1 --2- 172.21.15.120:0/1055542084 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557a1c8aad20 0x557a1c8ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:27.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.192+0000 7fec60b15740 4 client.30431 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:21:27.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.192+0000 7fec60b15740 1 -- 172.21.15.120:0/1055542084 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a1c803860 con 0x557a1c8aad20 2024-09-19T09:21:27.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.192+0000 7fec57fff640 1 --2- 172.21.15.120:0/1055542084 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557a1c8aad20 0x557a1c8ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:27.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.192+0000 7fec57fff640 1 --2- 172.21.15.120:0/1055542084 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557a1c8aad20 0x557a1c8ad150 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=799c6bf3327693e0 server_cookie=536ac5acc006072c in_seq=0 out_seq=0 2024-09-19T09:21:27.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.192+0000 7fec377fe640 10 client.30431 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:21:27.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.193+0000 7fec377fe640 1 -- 172.21.15.120:0/1055542084 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a1c803860 con 0x557a1c8aad20 2024-09-19T09:21:27.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.193+0000 7fec377fe640 10 client.30431 handle_command_reply: tid=0 2024-09-19T09:21:27.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.193+0000 7fec60b15740 1 client.30431 shutdown 2024-09-19T09:21:27.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.193+0000 7fec60b15740 1 -- 172.21.15.120:0/1055542084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1c891b40 msgr2=0x557a1c898060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:27.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.193+0000 7fec60b15740 1 --2- 172.21.15.120:0/1055542084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1c891b40 0x557a1c898060 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fec4c012720 tx=0x7fec4c012be0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.193+0000 7fec60b15740 1 -- 172.21.15.120:0/1055542084 shutdown_connections 2024-09-19T09:21:27.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.193+0000 7fec60b15740 1 --2- 172.21.15.120:0/1055542084 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557a1c8aad20 0x557a1c8ad150 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.193+0000 7fec60b15740 1 --2- 172.21.15.120:0/1055542084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a1c89b370 0x557a1c8987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.193+0000 7fec60b15740 1 --2- 172.21.15.120:0/1055542084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1c891b40 0x557a1c898060 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.193+0000 7fec60b15740 1 --2- 172.21.15.120:0/1055542084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1c8905d0 0x557a1c89adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.193+0000 7fec377fe640 0 client.30431 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:21:27.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.193+0000 7fec60b15740 1 -- 172.21.15.120:0/1055542084 >> 172.21.15.120:0/1055542084 conn(0x557a1c859310 msgr2=0x557a1c892990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:27.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.194+0000 7fec60b15740 1 -- 172.21.15.120:0/1055542084 shutdown_connections 2024-09-19T09:21:27.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.194+0000 7fec60b15740 1 -- 172.21.15.120:0/1055542084 wait complete. 2024-09-19T09:21:27.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.194+0000 7fec60b15740 20 client.30431 trim_cache size 0 max 16384 2024-09-19T09:21:27.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.207+0000 7fec60b15740 1 Processor -- start 2024-09-19T09:21:27.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.207+0000 7fec60b15740 1 -- start start 2024-09-19T09:21:27.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.207+0000 7fec60b15740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1c8905d0 0x557a1c8b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:27.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.207+0000 7fec60b15740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a1c891b40 0x557a1c8b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:27.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.207+0000 7fec60b15740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1c89b370 0x557a1c8952e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:27.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.207+0000 7fec60b15740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fec581ab410 con 0x557a1c89b370 2024-09-19T09:21:27.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.207+0000 7fec60b15740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fec581acf50 con 0x557a1c8905d0 2024-09-19T09:21:27.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.207+0000 7fec60b15740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fec581abae0 con 0x557a1c891b40 2024-09-19T09:21:27.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.207+0000 7fec57fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1c8905d0 0x557a1c8b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:27.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.207+0000 7fec577fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a1c891b40 0x557a1c8b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:27.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.207+0000 7fec57fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1c8905d0 0x557a1c8b2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:39400/0 (socket says 172.21.15.120:39400) 2024-09-19T09:21:27.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.207+0000 7fec577fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a1c891b40 0x557a1c8b2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:46194/0 (socket says 172.21.15.120:46194) 2024-09-19T09:21:27.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.207+0000 7fec57fff640 1 -- 172.21.15.120:0/2442026064 learned_addr learned my addr 172.21.15.120:0/2442026064 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:27.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.207+0000 7fec5ccd9640 1 --2- 172.21.15.120:0/2442026064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1c89b370 0x557a1c8952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:27.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.208+0000 7fec57fff640 1 -- 172.21.15.120:0/2442026064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a1c891b40 msgr2=0x557a1c8b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:27.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.208+0000 7fec57fff640 1 --2- 172.21.15.120:0/2442026064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a1c891b40 0x557a1c8b2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.208+0000 7fec57fff640 1 -- 172.21.15.120:0/2442026064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1c89b370 msgr2=0x557a1c8952e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.208+0000 7fec57fff640 1 --2- 172.21.15.120:0/2442026064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1c89b370 0x557a1c8952e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.208+0000 7fec57fff640 1 -- 172.21.15.120:0/2442026064 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a1c893b30 con 0x557a1c8905d0 2024-09-19T09:21:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.208+0000 7fec577fe640 1 --2- 172.21.15.120:0/2442026064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a1c891b40 0x557a1c8b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:21:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.208+0000 7fec57fff640 1 --2- 172.21.15.120:0/2442026064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1c8905d0 0x557a1c8b2a60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fec480ba5b0 tx=0x7fec4802a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40888efd08a614ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.208+0000 7fec3dffb640 1 -- 172.21.15.120:0/2442026064 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec48049040 con 0x557a1c8905d0 2024-09-19T09:21:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.208+0000 7fec3dffb640 1 -- 172.21.15.120:0/2442026064 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec480c4ec0 con 0x557a1c8905d0 2024-09-19T09:21:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.208+0000 7fec3dffb640 1 -- 172.21.15.120:0/2442026064 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec480b8970 con 0x557a1c8905d0 2024-09-19T09:21:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.209+0000 7fec60b15740 1 -- 172.21.15.120:0/2442026064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1c8905d0 msgr2=0x557a1c8b2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.209+0000 7fec60b15740 1 --2- 172.21.15.120:0/2442026064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1c8905d0 0x557a1c8b2a60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fec480ba5b0 tx=0x7fec4802a420 comp rx=0 tx=0).stop 2024-09-19T09:21:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.209+0000 7fec60b15740 1 -- 172.21.15.120:0/2442026064 shutdown_connections 2024-09-19T09:21:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.209+0000 7fec60b15740 1 --2- 172.21.15.120:0/2442026064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1c89b370 0x557a1c8952e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.209+0000 7fec60b15740 1 --2- 172.21.15.120:0/2442026064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a1c891b40 0x557a1c8b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.209+0000 7fec60b15740 1 --2- 172.21.15.120:0/2442026064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1c8905d0 0x557a1c8b2a60 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.209+0000 7fec60b15740 1 -- 172.21.15.120:0/2442026064 >> 172.21.15.120:0/2442026064 conn(0x557a1c859310 msgr2=0x557a1c896930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.209+0000 7fec60b15740 1 -- 172.21.15.120:0/2442026064 shutdown_connections 2024-09-19T09:21:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.209+0000 7fec60b15740 1 -- 172.21.15.120:0/2442026064 wait complete. 2024-09-19T09:21:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.209+0000 7fec60b15740 1 -- start start 2024-09-19T09:21:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.210+0000 7fec60b15740 1 Processor -- start 2024-09-19T09:21:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.210+0000 7fec60b15740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a1c8905d0 0x557a1c8951f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.210+0000 7fec60b15740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1c891b40 0x557a1c895730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.210+0000 7fec60b15740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1c89b370 0x557a1c895c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.210+0000 7fec60b15740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fec581ac880 con 0x557a1c89b370 2024-09-19T09:21:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.210+0000 7fec60b15740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fec581ac1b0 con 0x557a1c891b40 2024-09-19T09:21:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.210+0000 7fec60b15740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fec581a93d0 con 0x557a1c8905d0 2024-09-19T09:21:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.210+0000 7fec60b15740 10 client.0 _ll_register_callbacks cb 0x7fec4c009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.210+0000 7fec5ccd9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1c89b370 0x557a1c895c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.210+0000 7fec5ccd9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1c89b370 0x557a1c895c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39948/0 (socket says 172.21.15.120:39948) 2024-09-19T09:21:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.210+0000 7fec577fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1c891b40 0x557a1c895730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.210+0000 7fec5ccd9640 1 -- 172.21.15.120:0/3357180513 learned_addr learned my addr 172.21.15.120:0/3357180513 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.210+0000 7fec57fff640 1 --2- 172.21.15.120:0/3357180513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a1c8905d0 0x557a1c8951f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.210+0000 7fec5ccd9640 1 -- 172.21.15.120:0/3357180513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a1c8905d0 msgr2=0x557a1c8951f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.210+0000 7fec5ccd9640 1 --2- 172.21.15.120:0/3357180513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a1c8905d0 0x557a1c8951f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.210+0000 7fec5ccd9640 1 -- 172.21.15.120:0/3357180513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1c891b40 msgr2=0x557a1c895730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.210+0000 7fec5ccd9640 1 --2- 172.21.15.120:0/3357180513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1c891b40 0x557a1c895730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.210+0000 7fec5ccd9640 1 -- 172.21.15.120:0/3357180513 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a1c914b30 con 0x557a1c89b370 2024-09-19T09:21:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.210+0000 7fec5ccd9640 1 --2- 172.21.15.120:0/3357180513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1c89b370 0x557a1c895c70 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7fec500099e0 tx=0x7fec50009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404dbc87ddf31e39 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.210+0000 7fec377fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:21:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.211+0000 7fec377fe640 1 -- 172.21.15.120:0/3357180513 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5002e030 con 0x557a1c89b370 2024-09-19T09:21:27.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.211+0000 7fec377fe640 1 -- 172.21.15.120:0/3357180513 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fec5001a070 con 0x557a1c89b370 2024-09-19T09:21:27.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.211+0000 7fec60b15740 1 -- 172.21.15.120:0/3357180513 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec480c4ec0 con 0x557a1c89b370 2024-09-19T09:21:27.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.211+0000 7fec377fe640 1 -- 172.21.15.120:0/3357180513 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fec50026040 con 0x557a1c89b370 2024-09-19T09:21:27.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.211+0000 7fec60b15740 10 client.21612 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:27.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.211+0000 7fec60b15740 1 -- 172.21.15.120:0/3357180513 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557a1c915e10 con 0x557a1c89b370 2024-09-19T09:21:27.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.211+0000 7fec377fe640 1 -- 172.21.15.120:0/3357180513 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec480c4ec0 con 0x557a1c89b370 2024-09-19T09:21:27.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.211+0000 7fec377fe640 1 -- 172.21.15.120:0/3357180513 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fec50029030 con 0x557a1c89b370 2024-09-19T09:21:27.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.211+0000 7fec60b15740 10 client.21612 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:27.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.211+0000 7fec60b15740 10 client.21612 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:21:27.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.211+0000 7fec60b15740 20 client.21612 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:27.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.211+0000 7fec60b15740 1 --2- 172.21.15.120:0/3357180513 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557a1c916580 0x557a1c9189a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:27.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.211+0000 7fec60b15740 4 client.21612 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:21:27.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.211+0000 7fec60b15740 1 -- 172.21.15.120:0/3357180513 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a1c7fdd70 con 0x557a1c916580 2024-09-19T09:21:27.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.212+0000 7fec57fff640 1 --2- 172.21.15.120:0/3357180513 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557a1c916580 0x557a1c9189a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:27.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.212+0000 7fec377fe640 10 client.21612 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:21:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.212+0000 7fec57fff640 1 --2- 172.21.15.120:0/3357180513 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557a1c916580 0x557a1c9189a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8090312ff2dbd5a5 server_cookie=2e789b59265c25b1 in_seq=0 out_seq=0 2024-09-19T09:21:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.212+0000 7fec377fe640 1 -- 172.21.15.120:0/3357180513 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a1c7fdd70 con 0x557a1c916580 2024-09-19T09:21:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.212+0000 7fec377fe640 10 client.21612 handle_command_reply: tid=0 2024-09-19T09:21:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.212+0000 7fec60b15740 1 client.21612 shutdown 2024-09-19T09:21:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.212+0000 7fec60b15740 1 -- 172.21.15.120:0/3357180513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1c89b370 msgr2=0x557a1c895c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.212+0000 7fec60b15740 1 --2- 172.21.15.120:0/3357180513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1c89b370 0x557a1c895c70 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7fec500099e0 tx=0x7fec50009500 comp rx=0 tx=0).stop 2024-09-19T09:21:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.212+0000 7fec60b15740 1 -- 172.21.15.120:0/3357180513 shutdown_connections 2024-09-19T09:21:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.212+0000 7fec60b15740 1 --2- 172.21.15.120:0/3357180513 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557a1c916580 0x557a1c9189a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.212+0000 7fec60b15740 1 --2- 172.21.15.120:0/3357180513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1c89b370 0x557a1c895c70 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.212+0000 7fec60b15740 1 --2- 172.21.15.120:0/3357180513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1c891b40 0x557a1c895730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.212+0000 7fec60b15740 1 --2- 172.21.15.120:0/3357180513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a1c8905d0 0x557a1c8951f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.212+0000 7fec60b15740 1 -- 172.21.15.120:0/3357180513 >> 172.21.15.120:0/3357180513 conn(0x557a1c859310 msgr2=0x557a1c8998c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.212+0000 7fec377fe640 0 client.21612 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:21:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.212+0000 7fec60b15740 1 -- 172.21.15.120:0/3357180513 shutdown_connections 2024-09-19T09:21:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.212+0000 7fec60b15740 1 -- 172.21.15.120:0/3357180513 wait complete. 2024-09-19T09:21:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.213+0000 7fec60b15740 20 client.21612 trim_cache size 0 max 16384 2024-09-19T09:21:27.219 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:21:27.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.213+0000 7fec5e73c640 1 -- 172.21.15.120:0/817477591 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fec400a3240 msgr2=0x7fec400a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:27.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.213+0000 7fec5e73c640 1 --2- 172.21.15.120:0/817477591 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fec400a3240 0x7fec400a5700 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fec50009fd0 tx=0x7fec500116b0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.213+0000 7fec5e73c640 1 -- 172.21.15.120:0/817477591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec58100a60 msgr2=0x7fec58198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:27.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.213+0000 7fec5e73c640 1 --2- 172.21.15.120:0/817477591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec58100a60 0x7fec58198510 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fec48009230 tx=0x7fec48004040 comp rx=0 tx=0).stop 2024-09-19T09:21:27.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.213+0000 7fec5e73c640 1 -- 172.21.15.120:0/817477591 shutdown_connections 2024-09-19T09:21:27.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.213+0000 7fec5e73c640 1 --2- 172.21.15.120:0/817477591 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fec400a3240 0x7fec400a5700 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.213+0000 7fec5e73c640 1 --2- 172.21.15.120:0/817477591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec58106a60 0x7fec5819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.213+0000 7fec5e73c640 1 --2- 172.21.15.120:0/817477591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec58101400 0x7fec58198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.213+0000 7fec5e73c640 1 --2- 172.21.15.120:0/817477591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec58100a60 0x7fec58198510 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:27.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.213+0000 7fec5e73c640 1 -- 172.21.15.120:0/817477591 >> 172.21.15.120:0/817477591 conn(0x7fec580fc780 msgr2=0x7fec5806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:27.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.213+0000 7fec5e73c640 1 -- 172.21.15.120:0/817477591 shutdown_connections 2024-09-19T09:21:27.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:27.213+0000 7fec5e73c640 1 -- 172.21.15.120:0/817477591 wait complete. 2024-09-19T09:21:27.370 INFO:tasks.workunit.client.0.smithi120.stdout: 10 134271 20.94 MB/sec execute 202 sec latency 66.895 ms 2024-09-19T09:21:28.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:28 smithi120 ceph-mon[31150]: pgmap v903: 97 pgs: 97 active+clean; 389 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.3 MiB/s wr, 654 op/s 2024-09-19T09:21:28.285 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:21:28.370 INFO:tasks.workunit.client.0.smithi120.stdout: 10 134896 20.97 MB/sec execute 203 sec latency 64.472 ms 2024-09-19T09:21:28.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:28 smithi196 ceph-mon[35150]: pgmap v903: 97 pgs: 97 active+clean; 389 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.3 MiB/s wr, 654 op/s 2024-09-19T09:21:28.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:28 smithi153 ceph-mon[35071]: pgmap v903: 97 pgs: 97 active+clean; 389 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.3 MiB/s wr, 654 op/s 2024-09-19T09:21:28.602 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:21:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.733+0000 7f635927b640 1 -- 172.21.15.120:0/1993264000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6354100a50 msgr2=0x7f6354100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.733+0000 7f635927b640 1 --2- 172.21.15.120:0/1993264000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6354100a50 0x7f6354100e50 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f6344009900 tx=0x7f634402a410 comp rx=0 tx=0).stop 2024-09-19T09:21:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.733+0000 7f635927b640 1 -- 172.21.15.120:0/1993264000 shutdown_connections 2024-09-19T09:21:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.733+0000 7f635927b640 1 --2- 172.21.15.120:0/1993264000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6354102600 0x7f6354108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.733+0000 7f635927b640 1 --2- 172.21.15.120:0/1993264000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6354101c40 0x7f63541020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.733+0000 7f635927b640 1 --2- 172.21.15.120:0/1993264000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6354100a50 0x7f6354100e50 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.733+0000 7f635927b640 1 -- 172.21.15.120:0/1993264000 >> 172.21.15.120:0/1993264000 conn(0x7f63540fc1b0 msgr2=0x7f63540fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.733+0000 7f635927b640 1 -- 172.21.15.120:0/1993264000 shutdown_connections 2024-09-19T09:21:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.733+0000 7f635927b640 1 -- 172.21.15.120:0/1993264000 wait complete. 2024-09-19T09:21:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.733+0000 7f635927b640 1 Processor -- start 2024-09-19T09:21:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.733+0000 7f635927b640 1 -- start start 2024-09-19T09:21:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.733+0000 7f635927b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6354100a50 0x7f6354197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.733+0000 7f635927b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6354101c40 0x7f6354198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.733+0000 7f635927b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6354102600 0x7f635419f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.733+0000 7f635927b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f635410c6d0 con 0x7f6354100a50 2024-09-19T09:21:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.733+0000 7f635927b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f635410c550 con 0x7f6354101c40 2024-09-19T09:21:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.733+0000 7f635927b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f635410c850 con 0x7f6354102600 2024-09-19T09:21:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.734+0000 7f6352ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6354100a50 0x7f6354197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.734+0000 7f6352ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6354100a50 0x7f6354197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39966/0 (socket says 172.21.15.120:39966) 2024-09-19T09:21:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.734+0000 7f63537fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6354102600 0x7f635419f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.734+0000 7f6352ffd640 1 -- 172.21.15.120:0/2653273709 learned_addr learned my addr 172.21.15.120:0/2653273709 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.734+0000 7f634bfff640 1 --2- 172.21.15.120:0/2653273709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6354101c40 0x7f6354198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.734+0000 7f6352ffd640 1 -- 172.21.15.120:0/2653273709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6354102600 msgr2=0x7f635419f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.734+0000 7f6352ffd640 1 --2- 172.21.15.120:0/2653273709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6354102600 0x7f635419f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.734+0000 7f6352ffd640 1 -- 172.21.15.120:0/2653273709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6354101c40 msgr2=0x7f6354198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.734+0000 7f6352ffd640 1 --2- 172.21.15.120:0/2653273709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6354101c40 0x7f6354198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.734+0000 7f6352ffd640 1 -- 172.21.15.120:0/2653273709 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f635419fa70 con 0x7f6354100a50 2024-09-19T09:21:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.734+0000 7f6352ffd640 1 --2- 172.21.15.120:0/2653273709 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6354100a50 0x7f6354197e50 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f6344031610 tx=0x7f6344004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.734+0000 7f6350ff9640 1 -- 172.21.15.120:0/2653273709 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634403f030 con 0x7f6354100a50 2024-09-19T09:21:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.734+0000 7f6350ff9640 1 -- 172.21.15.120:0/2653273709 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6344003ad0 con 0x7f6354100a50 2024-09-19T09:21:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.734+0000 7f6350ff9640 1 -- 172.21.15.120:0/2653273709 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6344035070 con 0x7f6354100a50 2024-09-19T09:21:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.734+0000 7f635927b640 1 -- 172.21.15.120:0/2653273709 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f635419fd00 con 0x7f6354100a50 2024-09-19T09:21:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.734+0000 7f635927b640 1 -- 172.21.15.120:0/2653273709 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63541a0180 con 0x7f6354100a50 2024-09-19T09:21:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.737+0000 7f6350ff9640 1 -- 172.21.15.120:0/2653273709 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f6344042030 con 0x7f6354100a50 2024-09-19T09:21:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.737+0000 7f635c01c740 1 Processor -- start 2024-09-19T09:21:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.737+0000 7f635c01c740 1 -- start start 2024-09-19T09:21:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.738+0000 7f635c01c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e48ab0b9a0 0x55e48ab0bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.738+0000 7f635c01c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e48ab0c2e0 0x55e48ab10da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.738+0000 7f635c01c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e48ab11370 0x55e48ab13830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.738+0000 7f635c01c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e48aa6f2f0 con 0x55e48ab11370 2024-09-19T09:21:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.738+0000 7f635c01c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e48aa17270 con 0x55e48ab0b9a0 2024-09-19T09:21:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.738+0000 7f635c01c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e48aa6b2b0 con 0x55e48ab0c2e0 2024-09-19T09:21:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.738+0000 7f6350ff9640 1 --2- 172.21.15.120:0/2653273709 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f63400a3370 0x7f63400a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.738+0000 7f634bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e48ab11370 0x55e48ab13830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.738+0000 7f63537fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e48ab0b9a0 0x55e48ab0bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.738+0000 7f634bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e48ab11370 0x55e48ab13830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39980/0 (socket says 172.21.15.120:39980) 2024-09-19T09:21:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.738+0000 7f634bfff640 1 -- 172.21.15.120:0/1011031469 learned_addr learned my addr 172.21.15.120:0/1011031469 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.738+0000 7f6352ffd640 1 --2- 172.21.15.120:0/1011031469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e48ab0c2e0 0x55e48ab10da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.738+0000 7f63537fe640 1 --2- 172.21.15.120:0/2653273709 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f63400a3370 0x7f63400a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.738+0000 7f6352ffd640 1 -- 172.21.15.120:0/1011031469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e48ab0b9a0 msgr2=0x55e48ab0bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.738+0000 7f6352ffd640 1 --2- 172.21.15.120:0/1011031469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e48ab0b9a0 0x55e48ab0bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.738+0000 7f6352ffd640 1 -- 172.21.15.120:0/1011031469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e48ab11370 msgr2=0x55e48ab13830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.738+0000 7f6352ffd640 1 --2- 172.21.15.120:0/1011031469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e48ab11370 0x55e48ab13830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.739+0000 7f6352ffd640 1 -- 172.21.15.120:0/1011031469 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e48aa16340 con 0x55e48ab0c2e0 2024-09-19T09:21:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.739+0000 7f634bfff640 1 --2- 172.21.15.120:0/1011031469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e48ab11370 0x55e48ab13830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:21:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.739+0000 7f63537fe640 1 --2- 172.21.15.120:0/1011031469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e48ab0b9a0 0x55e48ab0bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:21:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.739+0000 7f6352ffd640 1 --2- 172.21.15.120:0/1011031469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e48ab0c2e0 0x55e48ab10da0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f63440ed8e0 tx=0x7f63440e7230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=220a39d8128ee6a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.739+0000 7f6350ff9640 1 -- 172.21.15.120:0/2653273709 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f63440efe80 con 0x7f6354100a50 2024-09-19T09:21:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.739+0000 7f63537fe640 1 --2- 172.21.15.120:0/2653273709 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f63400a3370 0x7f63400a5830 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f634c005fd0 tx=0x7f634c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.739+0000 7f63497fa640 1 -- 172.21.15.120:0/1011031469 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63440bc2e0 con 0x55e48ab0c2e0 2024-09-19T09:21:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.739+0000 7f63497fa640 1 -- 172.21.15.120:0/1011031469 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f634403be40 con 0x55e48ab0c2e0 2024-09-19T09:21:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.739+0000 7f63497fa640 1 -- 172.21.15.120:0/1011031469 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63440baad0 con 0x55e48ab0c2e0 2024-09-19T09:21:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.740+0000 7f635c01c740 1 -- 172.21.15.120:0/1011031469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e48ab0c2e0 msgr2=0x55e48ab10da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.740+0000 7f635c01c740 1 --2- 172.21.15.120:0/1011031469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e48ab0c2e0 0x55e48ab10da0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f63440ed8e0 tx=0x7f63440e7230 comp rx=0 tx=0).stop 2024-09-19T09:21:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.740+0000 7f635c01c740 1 -- 172.21.15.120:0/1011031469 shutdown_connections 2024-09-19T09:21:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.740+0000 7f635c01c740 1 --2- 172.21.15.120:0/1011031469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e48ab11370 0x55e48ab13830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.740+0000 7f635c01c740 1 --2- 172.21.15.120:0/1011031469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e48ab0c2e0 0x55e48ab10da0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.740+0000 7f635c01c740 1 --2- 172.21.15.120:0/1011031469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e48ab0b9a0 0x55e48ab0bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.740+0000 7f635c01c740 1 -- 172.21.15.120:0/1011031469 >> 172.21.15.120:0/1011031469 conn(0x55e48ab07010 msgr2=0x55e48ab07410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.740+0000 7f635c01c740 1 -- 172.21.15.120:0/1011031469 shutdown_connections 2024-09-19T09:21:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.740+0000 7f635c01c740 1 -- 172.21.15.120:0/1011031469 wait complete. 2024-09-19T09:21:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.740+0000 7f635c01c740 1 -- start start 2024-09-19T09:21:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.740+0000 7f635c01c740 1 Processor -- start 2024-09-19T09:21:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.741+0000 7f635c01c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e48ab0b9a0 0x55e48ab157d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.741+0000 7f635c01c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e48ab0c2e0 0x55e48ab15d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.741+0000 7f635c01c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e48ab11370 0x55e48ab1cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.741+0000 7f635c01c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e48aa6b2b0 con 0x55e48ab0b9a0 2024-09-19T09:21:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.741+0000 7f635c01c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e48a9f8290 con 0x55e48ab0c2e0 2024-09-19T09:21:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.741+0000 7f635c01c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e48ab14100 con 0x55e48ab11370 2024-09-19T09:21:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.741+0000 7f635c01c740 10 client.0 _ll_register_callbacks cb 0x55e48aa29140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.741+0000 7f6352ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e48ab0b9a0 0x55e48ab157d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.741+0000 7f634bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e48ab0c2e0 0x55e48ab15d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.741+0000 7f6352ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e48ab0b9a0 0x55e48ab157d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39982/0 (socket says 172.21.15.120:39982) 2024-09-19T09:21:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.741+0000 7f6352ffd640 1 -- 172.21.15.120:0/798546318 learned_addr learned my addr 172.21.15.120:0/798546318 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.741+0000 7f63537fe640 1 --2- 172.21.15.120:0/798546318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e48ab11370 0x55e48ab1cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.741+0000 7f6352ffd640 1 -- 172.21.15.120:0/798546318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e48ab11370 msgr2=0x55e48ab1cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.741+0000 7f6352ffd640 1 --2- 172.21.15.120:0/798546318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e48ab11370 0x55e48ab1cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.741+0000 7f6352ffd640 1 -- 172.21.15.120:0/798546318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e48ab0c2e0 msgr2=0x55e48ab15d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.741+0000 7f6352ffd640 1 --2- 172.21.15.120:0/798546318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e48ab0c2e0 0x55e48ab15d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.741+0000 7f6352ffd640 1 -- 172.21.15.120:0/798546318 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e48a782210 con 0x55e48ab0b9a0 2024-09-19T09:21:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.741+0000 7f6352ffd640 1 --2- 172.21.15.120:0/798546318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e48ab0b9a0 0x55e48ab157d0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f63440ed910 tx=0x7f63440baa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abca0ad38bf4190a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.741+0000 7f633a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:21:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.741+0000 7f633a7fc640 1 -- 172.21.15.120:0/798546318 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6344049050 con 0x55e48ab0b9a0 2024-09-19T09:21:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.741+0000 7f633a7fc640 1 -- 172.21.15.120:0/798546318 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f63440b83e0 con 0x55e48ab0b9a0 2024-09-19T09:21:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.741+0000 7f635c01c740 1 -- 172.21.15.120:0/798546318 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e48aa7ca20 con 0x55e48ab0b9a0 2024-09-19T09:21:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.741+0000 7f633a7fc640 1 -- 172.21.15.120:0/798546318 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f63440bacc0 con 0x55e48ab0b9a0 2024-09-19T09:21:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.741+0000 7f635c01c740 10 client.21624 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.741+0000 7f635c01c740 1 -- 172.21.15.120:0/798546318 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e48a84e620 con 0x55e48ab0b9a0 2024-09-19T09:21:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.742+0000 7f633a7fc640 1 -- 172.21.15.120:0/798546318 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e48aa7ca20 con 0x55e48ab0b9a0 2024-09-19T09:21:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.742+0000 7f633a7fc640 1 -- 172.21.15.120:0/798546318 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f63440035f0 con 0x55e48ab0b9a0 2024-09-19T09:21:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.742+0000 7f635c01c740 10 client.21624 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.742+0000 7f635c01c740 10 client.21624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:21:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.742+0000 7f635c01c740 20 client.21624 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.742+0000 7f635c01c740 1 --2- 172.21.15.120:0/798546318 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e48ab212b0 0x55e48ab236a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.742+0000 7f635c01c740 4 client.21624 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:21:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.742+0000 7f635c01c740 1 -- 172.21.15.120:0/798546318 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e48aa76d70 con 0x55e48ab212b0 2024-09-19T09:21:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.742+0000 7f634bfff640 1 --2- 172.21.15.120:0/798546318 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e48ab212b0 0x55e48ab236a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.742+0000 7f633a7fc640 10 client.21624 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.742+0000 7f634bfff640 1 --2- 172.21.15.120:0/798546318 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e48ab212b0 0x55e48ab236a0 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=792bf7d00686d6a9 server_cookie=590121134e41ec08 in_seq=0 out_seq=0 2024-09-19T09:21:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.744+0000 7f633a7fc640 1 -- 172.21.15.120:0/798546318 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e48aa76d70 con 0x55e48ab212b0 2024-09-19T09:21:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.744+0000 7f633a7fc640 10 client.21624 handle_command_reply: tid=0 2024-09-19T09:21:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.744+0000 7f635c01c740 1 client.21624 shutdown 2024-09-19T09:21:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.744+0000 7f635c01c740 1 -- 172.21.15.120:0/798546318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e48ab0b9a0 msgr2=0x55e48ab157d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.744+0000 7f635c01c740 1 --2- 172.21.15.120:0/798546318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e48ab0b9a0 0x55e48ab157d0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f63440ed910 tx=0x7f63440baa20 comp rx=0 tx=0).stop 2024-09-19T09:21:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.744+0000 7f635c01c740 1 -- 172.21.15.120:0/798546318 shutdown_connections 2024-09-19T09:21:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.744+0000 7f635c01c740 1 --2- 172.21.15.120:0/798546318 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e48ab212b0 0x55e48ab236a0 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.745+0000 7f633a7fc640 0 client.21624 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.745+0000 7f635c01c740 1 --2- 172.21.15.120:0/798546318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e48ab11370 0x55e48ab1cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.745+0000 7f635c01c740 1 --2- 172.21.15.120:0/798546318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e48ab0c2e0 0x55e48ab15d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.745+0000 7f635c01c740 1 --2- 172.21.15.120:0/798546318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e48ab0b9a0 0x55e48ab157d0 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.745+0000 7f635c01c740 1 -- 172.21.15.120:0/798546318 >> 172.21.15.120:0/798546318 conn(0x55e48ab07010 msgr2=0x55e48ab08820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.745+0000 7f635c01c740 1 -- 172.21.15.120:0/798546318 shutdown_connections 2024-09-19T09:21:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.745+0000 7f635c01c740 1 -- 172.21.15.120:0/798546318 wait complete. 2024-09-19T09:21:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.745+0000 7f635c01c740 20 client.21624 trim_cache size 0 max 16384 2024-09-19T09:21:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.760+0000 7f635c01c740 1 Processor -- start 2024-09-19T09:21:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.760+0000 7f635c01c740 1 -- start start 2024-09-19T09:21:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.760+0000 7f635c01c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e48ab0b9a0 0x55e48ab15da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.760+0000 7f635c01c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e48ab0c2e0 0x55e48ab1cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.760+0000 7f635c01c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e48ab11370 0x55e48ab060b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.760+0000 7f635c01c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f63541a9010 con 0x55e48ab11370 2024-09-19T09:21:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.760+0000 7f635c01c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f63541aab50 con 0x55e48ab0b9a0 2024-09-19T09:21:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.760+0000 7f635c01c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f63541a96e0 con 0x55e48ab0c2e0 2024-09-19T09:21:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.760+0000 7f63537fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e48ab11370 0x55e48ab060b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.760+0000 7f63537fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e48ab11370 0x55e48ab060b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39998/0 (socket says 172.21.15.120:39998) 2024-09-19T09:21:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.760+0000 7f63537fe640 1 -- 172.21.15.120:0/786871358 learned_addr learned my addr 172.21.15.120:0/786871358 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.760+0000 7f634bfff640 1 --2- 172.21.15.120:0/786871358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e48ab0c2e0 0x55e48ab1cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.760+0000 7f6352ffd640 1 --2- 172.21.15.120:0/786871358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e48ab0b9a0 0x55e48ab15da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.760+0000 7f63537fe640 1 -- 172.21.15.120:0/786871358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e48ab0c2e0 msgr2=0x55e48ab1cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.760+0000 7f63537fe640 1 --2- 172.21.15.120:0/786871358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e48ab0c2e0 0x55e48ab1cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.760+0000 7f63537fe640 1 -- 172.21.15.120:0/786871358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e48ab0b9a0 msgr2=0x55e48ab15da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.760+0000 7f63537fe640 1 --2- 172.21.15.120:0/786871358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e48ab0b9a0 0x55e48ab15da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.760+0000 7f63537fe640 1 -- 172.21.15.120:0/786871358 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e48ab06e30 con 0x55e48ab11370 2024-09-19T09:21:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.761+0000 7f63537fe640 1 --2- 172.21.15.120:0/786871358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e48ab11370 0x55e48ab060b0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f634c00f940 tx=0x7f634c008000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19e55233d583816 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.761+0000 7f6348ff9640 1 -- 172.21.15.120:0/786871358 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634c004b20 con 0x55e48ab11370 2024-09-19T09:21:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.761+0000 7f6348ff9640 1 -- 172.21.15.120:0/786871358 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f634c004cc0 con 0x55e48ab11370 2024-09-19T09:21:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.761+0000 7f6348ff9640 1 -- 172.21.15.120:0/786871358 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634c005920 con 0x55e48ab11370 2024-09-19T09:21:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.762+0000 7f635c01c740 1 -- 172.21.15.120:0/786871358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e48ab11370 msgr2=0x55e48ab060b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.762+0000 7f635c01c740 1 --2- 172.21.15.120:0/786871358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e48ab11370 0x55e48ab060b0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f634c00f940 tx=0x7f634c008000 comp rx=0 tx=0).stop 2024-09-19T09:21:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.762+0000 7f635c01c740 1 -- 172.21.15.120:0/786871358 shutdown_connections 2024-09-19T09:21:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.762+0000 7f635c01c740 1 --2- 172.21.15.120:0/786871358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e48ab11370 0x55e48ab060b0 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.762+0000 7f635c01c740 1 --2- 172.21.15.120:0/786871358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e48ab0c2e0 0x55e48ab1cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.762+0000 7f635c01c740 1 --2- 172.21.15.120:0/786871358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e48ab0b9a0 0x55e48ab15da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.762+0000 7f635c01c740 1 -- 172.21.15.120:0/786871358 >> 172.21.15.120:0/786871358 conn(0x55e48ab07010 msgr2=0x55e48ab059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.762+0000 7f635c01c740 1 -- 172.21.15.120:0/786871358 shutdown_connections 2024-09-19T09:21:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.762+0000 7f635c01c740 1 -- 172.21.15.120:0/786871358 wait complete. 2024-09-19T09:21:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.762+0000 7f635c01c740 1 -- start start 2024-09-19T09:21:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.762+0000 7f635c01c740 1 Processor -- start 2024-09-19T09:21:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.762+0000 7f635c01c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e48ab0b9a0 0x55e48ab1a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.762+0000 7f635c01c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e48ab0c2e0 0x55e48ab1ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.762+0000 7f635c01c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e48ab11370 0x55e48ab1bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.762+0000 7f635c01c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f63541aa480 con 0x55e48ab0b9a0 2024-09-19T09:21:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.762+0000 7f635c01c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f63541a9db0 con 0x55e48ab11370 2024-09-19T09:21:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.762+0000 7f635c01c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f63541a6fd0 con 0x55e48ab0c2e0 2024-09-19T09:21:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.762+0000 7f635c01c740 10 client.0 _ll_register_callbacks cb 0x7f63440c0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.762+0000 7f6352ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e48ab0b9a0 0x55e48ab1a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.762+0000 7f6352ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e48ab0b9a0 0x55e48ab1a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40014/0 (socket says 172.21.15.120:40014) 2024-09-19T09:21:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.762+0000 7f6352ffd640 1 -- 172.21.15.120:0/3695427172 learned_addr learned my addr 172.21.15.120:0/3695427172 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.763+0000 7f634bfff640 1 --2- 172.21.15.120:0/3695427172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e48ab0c2e0 0x55e48ab1ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.763+0000 7f63537fe640 1 --2- 172.21.15.120:0/3695427172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e48ab11370 0x55e48ab1bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.763+0000 7f6352ffd640 1 -- 172.21.15.120:0/3695427172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e48ab0c2e0 msgr2=0x55e48ab1ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.763+0000 7f6352ffd640 1 --2- 172.21.15.120:0/3695427172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e48ab0c2e0 0x55e48ab1ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.763+0000 7f6352ffd640 1 -- 172.21.15.120:0/3695427172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e48ab11370 msgr2=0x55e48ab1bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.763+0000 7f6352ffd640 1 --2- 172.21.15.120:0/3695427172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e48ab11370 0x55e48ab1bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.763+0000 7f6352ffd640 1 -- 172.21.15.120:0/3695427172 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e48ab1c150 con 0x55e48ab0b9a0 2024-09-19T09:21:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.763+0000 7f6352ffd640 1 --2- 172.21.15.120:0/3695427172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e48ab0b9a0 0x55e48ab1a5f0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f63440bc4a0 tx=0x7f63440f38c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721f072b5da51261 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.763+0000 7f633a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:21:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.763+0000 7f633a7fc640 1 -- 172.21.15.120:0/3695427172 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6344049040 con 0x55e48ab0b9a0 2024-09-19T09:21:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.763+0000 7f633a7fc640 1 -- 172.21.15.120:0/3695427172 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f63440b82e0 con 0x55e48ab0b9a0 2024-09-19T09:21:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.763+0000 7f635c01c740 1 -- 172.21.15.120:0/3695427172 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f634c004cc0 con 0x55e48ab0b9a0 2024-09-19T09:21:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.763+0000 7f633a7fc640 1 -- 172.21.15.120:0/3695427172 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f63440bbe70 con 0x55e48ab0b9a0 2024-09-19T09:21:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.763+0000 7f635c01c740 10 client.21630 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.763+0000 7f635c01c740 1 -- 172.21.15.120:0/3695427172 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e48ab8c090 con 0x55e48ab0b9a0 2024-09-19T09:21:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.763+0000 7f633a7fc640 1 -- 172.21.15.120:0/3695427172 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f634c004cc0 con 0x55e48ab0b9a0 2024-09-19T09:21:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.764+0000 7f633a7fc640 1 -- 172.21.15.120:0/3695427172 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f63440035f0 con 0x55e48ab0b9a0 2024-09-19T09:21:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.764+0000 7f635c01c740 10 client.21630 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.764+0000 7f635c01c740 10 client.21630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:21:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.764+0000 7f635c01c740 20 client.21630 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.764+0000 7f635c01c740 1 --2- 172.21.15.120:0/3695427172 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e48ab212b0 0x55e48ab8e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.764+0000 7f635c01c740 4 client.21630 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:21:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.764+0000 7f635c01c740 1 -- 172.21.15.120:0/3695427172 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e48a9f3900 con 0x55e48ab212b0 2024-09-19T09:21:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.764+0000 7f634bfff640 1 --2- 172.21.15.120:0/3695427172 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e48ab212b0 0x55e48ab8e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.764+0000 7f633a7fc640 10 client.21630 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:28.764+0000 7f634bfff640 1 --2- 172.21.15.120:0/3695427172 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e48ab212b0 0x55e48ab8e720 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d24f59d6e5ec655d server_cookie=6dce8dd4704a75ce in_seq=0 out_seq=0 2024-09-19T09:21:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:29.218+0000 7f633a7fc640 1 -- 172.21.15.120:0/3695427172 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e48a9f3900 con 0x55e48ab212b0 2024-09-19T09:21:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:29.218+0000 7f633a7fc640 10 client.21630 handle_command_reply: tid=0 2024-09-19T09:21:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:29.218+0000 7f635c01c740 1 client.21630 shutdown 2024-09-19T09:21:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:29.218+0000 7f635c01c740 1 -- 172.21.15.120:0/3695427172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e48ab0b9a0 msgr2=0x55e48ab1a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:29.218+0000 7f635c01c740 1 --2- 172.21.15.120:0/3695427172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e48ab0b9a0 0x55e48ab1a5f0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f63440bc4a0 tx=0x7f63440f38c0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:29.218+0000 7f635c01c740 1 -- 172.21.15.120:0/3695427172 shutdown_connections 2024-09-19T09:21:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:29.218+0000 7f635c01c740 1 --2- 172.21.15.120:0/3695427172 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55e48ab212b0 0x55e48ab8e720 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:29.218+0000 7f633a7fc640 0 client.21630 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:29.218+0000 7f635c01c740 1 --2- 172.21.15.120:0/3695427172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e48ab11370 0x55e48ab1bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:29.218+0000 7f635c01c740 1 --2- 172.21.15.120:0/3695427172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e48ab0c2e0 0x55e48ab1ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:29.218+0000 7f635c01c740 1 --2- 172.21.15.120:0/3695427172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e48ab0b9a0 0x55e48ab1a5f0 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:29.218+0000 7f635c01c740 1 -- 172.21.15.120:0/3695427172 >> 172.21.15.120:0/3695427172 conn(0x55e48ab07010 msgr2=0x55e48ab086e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:29.219+0000 7f635c01c740 1 -- 172.21.15.120:0/3695427172 shutdown_connections 2024-09-19T09:21:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:29.219+0000 7f635c01c740 1 -- 172.21.15.120:0/3695427172 wait complete. 2024-09-19T09:21:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:29.219+0000 7f635c01c740 20 client.21630 trim_cache size 0 max 16384 2024-09-19T09:21:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:21:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:21:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:21:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:21:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:29.220+0000 7f635927b640 1 -- 172.21.15.120:0/2653273709 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f63400a3370 msgr2=0x7f63400a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:29.220+0000 7f635927b640 1 --2- 172.21.15.120:0/2653273709 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f63400a3370 0x7f63400a5830 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f634c005fd0 tx=0x7f634c009500 comp rx=0 tx=0).stop 2024-09-19T09:21:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:29.220+0000 7f635927b640 1 -- 172.21.15.120:0/2653273709 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6354100a50 msgr2=0x7f6354197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:29.220+0000 7f635927b640 1 --2- 172.21.15.120:0/2653273709 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6354100a50 0x7f6354197e50 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f6344031610 tx=0x7f6344004270 comp rx=0 tx=0).stop 2024-09-19T09:21:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:29.220+0000 7f635927b640 1 -- 172.21.15.120:0/2653273709 shutdown_connections 2024-09-19T09:21:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:29.220+0000 7f635927b640 1 --2- 172.21.15.120:0/2653273709 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f63400a3370 0x7f63400a5830 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:29.220+0000 7f635927b640 1 --2- 172.21.15.120:0/2653273709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6354102600 0x7f635419f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:29.220+0000 7f635927b640 1 --2- 172.21.15.120:0/2653273709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6354101c40 0x7f6354198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:29.220+0000 7f635927b640 1 --2- 172.21.15.120:0/2653273709 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6354100a50 0x7f6354197e50 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:29.220+0000 7f635927b640 1 -- 172.21.15.120:0/2653273709 >> 172.21.15.120:0/2653273709 conn(0x7f63540fc1b0 msgr2=0x7f63540fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:29.220+0000 7f635927b640 1 -- 172.21.15.120:0/2653273709 shutdown_connections 2024-09-19T09:21:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:29.220+0000 7f635927b640 1 -- 172.21.15.120:0/2653273709 wait complete. 2024-09-19T09:21:29.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.285+0000 7f3cc1ee7640 1 -- 172.21.15.120:0/3214616302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cbc101460 msgr2=0x7f3cbc105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:29.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.285+0000 7f3cc1ee7640 1 --2- 172.21.15.120:0/3214616302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cbc101460 0x7f3cbc105930 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0009900 tx=0x7f3cb002a470 comp rx=0 tx=0).stop 2024-09-19T09:21:29.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.286+0000 7f3cc1ee7640 1 -- 172.21.15.120:0/3214616302 shutdown_connections 2024-09-19T09:21:29.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.286+0000 7f3cc1ee7640 1 --2- 172.21.15.120:0/3214616302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cbc105f50 0x7f3cbc1063e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.286+0000 7f3cc1ee7640 1 --2- 172.21.15.120:0/3214616302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cbc101460 0x7f3cbc105930 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.286+0000 7f3cc1ee7640 1 --2- 172.21.15.120:0/3214616302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cbc100b40 0x7f3cbc100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.286+0000 7f3cc1ee7640 1 -- 172.21.15.120:0/3214616302 >> 172.21.15.120:0/3214616302 conn(0x7f3cbc0fc7d0 msgr2=0x7f3cbc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:29.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.286+0000 7f3cc1ee7640 1 -- 172.21.15.120:0/3214616302 shutdown_connections 2024-09-19T09:21:29.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.286+0000 7f3cc1ee7640 1 -- 172.21.15.120:0/3214616302 wait complete. 2024-09-19T09:21:29.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.286+0000 7f3cc1ee7640 1 Processor -- start 2024-09-19T09:21:29.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.287+0000 7f3cc1ee7640 1 -- start start 2024-09-19T09:21:29.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.287+0000 7f3cc1ee7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cbc100b40 0x7f3cbc1a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:29.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.287+0000 7f3cc1ee7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cbc101460 0x7f3cbc1a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:29.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.287+0000 7f3cc1ee7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cbc105f50 0x7f3cbc19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:29.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.287+0000 7f3cc1ee7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc118550 con 0x7f3cbc101460 2024-09-19T09:21:29.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.287+0000 7f3cc1ee7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc1183d0 con 0x7f3cbc100b40 2024-09-19T09:21:29.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.287+0000 7f3cc1ee7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc1186d0 con 0x7f3cbc105f50 2024-09-19T09:21:29.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.287+0000 7f3cbb7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cbc100b40 0x7f3cbc1a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:29.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.287+0000 7f3cbaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cbc101460 0x7f3cbc1a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:29.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.287+0000 7f3cbb7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cbc100b40 0x7f3cbc1a46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:39498/0 (socket says 172.21.15.120:39498) 2024-09-19T09:21:29.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.287+0000 7f3cbb7fe640 1 -- 172.21.15.120:0/773223068 learned_addr learned my addr 172.21.15.120:0/773223068 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:29.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.287+0000 7f3cbaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cbc101460 0x7f3cbc1a4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40030/0 (socket says 172.21.15.120:40030) 2024-09-19T09:21:29.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.287+0000 7f3cbbfff640 1 --2- 172.21.15.120:0/773223068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cbc105f50 0x7f3cbc19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:29.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.287+0000 7f3cbaffd640 1 -- 172.21.15.120:0/773223068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cbc105f50 msgr2=0x7f3cbc19e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:29.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.287+0000 7f3cbaffd640 1 --2- 172.21.15.120:0/773223068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cbc105f50 0x7f3cbc19e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.288+0000 7f3cbaffd640 1 -- 172.21.15.120:0/773223068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cbc100b40 msgr2=0x7f3cbc1a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:29.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.288+0000 7f3cbaffd640 1 --2- 172.21.15.120:0/773223068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cbc100b40 0x7f3cbc1a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.288+0000 7f3cbaffd640 1 -- 172.21.15.120:0/773223068 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cbc19efd0 con 0x7f3cbc101460 2024-09-19T09:21:29.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.288+0000 7f3cbaffd640 1 --2- 172.21.15.120:0/773223068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cbc101460 0x7f3cbc1a4c20 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0005940 tx=0x7f3cb0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:29.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.288+0000 7f3cb8ff9640 1 -- 172.21.15.120:0/773223068 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb003f030 con 0x7f3cbc101460 2024-09-19T09:21:29.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.288+0000 7f3cc1ee7640 1 -- 172.21.15.120:0/773223068 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cbc19f260 con 0x7f3cbc101460 2024-09-19T09:21:29.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.288+0000 7f3cb8ff9640 1 -- 172.21.15.120:0/773223068 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cb0004440 con 0x7f3cbc101460 2024-09-19T09:21:29.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.288+0000 7f3cb8ff9640 1 -- 172.21.15.120:0/773223068 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0035070 con 0x7f3cbc101460 2024-09-19T09:21:29.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.288+0000 7f3cc1ee7640 1 -- 172.21.15.120:0/773223068 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cbc1ab5b0 con 0x7f3cbc101460 2024-09-19T09:21:29.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.291+0000 7f3cb8ff9640 1 -- 172.21.15.120:0/773223068 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f3cb0042030 con 0x7f3cbc101460 2024-09-19T09:21:29.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.291+0000 7f3cb8ff9640 1 --2- 172.21.15.120:0/773223068 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3ca40a3240 0x7f3ca40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:29.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.291+0000 7f3cc42c0740 1 Processor -- start 2024-09-19T09:21:29.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.291+0000 7f3cc42c0740 1 -- start start 2024-09-19T09:21:29.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.291+0000 7f3cb8ff9640 1 -- 172.21.15.120:0/773223068 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f3cb00efb20 con 0x7f3cbc101460 2024-09-19T09:21:29.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.291+0000 7f3cbb7fe640 1 --2- 172.21.15.120:0/773223068 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3ca40a3240 0x7f3ca40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:29.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.292+0000 7f3cc42c0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9a6289b40 0x55b9a6288090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:29.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.292+0000 7f3cc42c0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9a62885d0 0x55b9a6288a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:29.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.292+0000 7f3cc42c0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9a6293370 0x55b9a6295760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:29.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.292+0000 7f3cc42c0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b9a61ee2f0 con 0x55b9a6293370 2024-09-19T09:21:29.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.292+0000 7f3cc42c0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b9a6195cf0 con 0x55b9a62885d0 2024-09-19T09:21:29.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.292+0000 7f3cc42c0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b9a61ea2b0 con 0x55b9a6289b40 2024-09-19T09:21:29.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.292+0000 7f3cbbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9a6293370 0x55b9a6295760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:29.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.292+0000 7f3cbb7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9a6289b40 0x55b9a6288090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:29.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.292+0000 7f3cbb7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9a6289b40 0x55b9a6288090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:46304/0 (socket says 172.21.15.120:46304) 2024-09-19T09:21:29.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.292+0000 7f3cbb7fe640 1 -- 172.21.15.120:0/800856000 learned_addr learned my addr 172.21.15.120:0/800856000 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:29.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.292+0000 7f3cbbfff640 1 --2- 172.21.15.120:0/800856000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9a6293370 0x55b9a6295760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40046/0 (socket says 172.21.15.120:40046) 2024-09-19T09:21:29.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.292+0000 7f3cbaffd640 1 --2- 172.21.15.120:0/800856000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9a62885d0 0x55b9a6288a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:29.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.292+0000 7f3cbb7fe640 1 -- 172.21.15.120:0/800856000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9a62885d0 msgr2=0x55b9a6288a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:29.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.292+0000 7f3cbb7fe640 1 --2- 172.21.15.120:0/800856000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9a62885d0 0x55b9a6288a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.292+0000 7f3cbb7fe640 1 -- 172.21.15.120:0/800856000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9a6293370 msgr2=0x55b9a6295760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:29.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.292+0000 7f3cbb7fe640 1 --2- 172.21.15.120:0/800856000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9a6293370 0x55b9a6295760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.292+0000 7f3cbb7fe640 1 -- 172.21.15.120:0/800856000 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9a617a3a0 con 0x55b9a6289b40 2024-09-19T09:21:29.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.292+0000 7f3cbaffd640 1 --2- 172.21.15.120:0/800856000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9a62885d0 0x55b9a6288a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:21:29.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.292+0000 7f3cbbfff640 1 --2- 172.21.15.120:0/800856000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9a6293370 0x55b9a6295760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:21:29.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.292+0000 7f3cbb7fe640 1 --2- 172.21.15.120:0/800856000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9a6289b40 0x55b9a6288090 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3cac011cc0 tx=0x7f3cac00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0a6e58457b39622 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:29.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.292+0000 7f3cbb7fe640 1 --2- 172.21.15.120:0/773223068 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3ca40a3240 0x7f3ca40a5700 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3cac00fc90 tx=0x7f3cac013000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:29.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.292+0000 7f3ca1ffb640 1 -- 172.21.15.120:0/800856000 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cac01a050 con 0x55b9a6289b40 2024-09-19T09:21:29.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.293+0000 7f3ca1ffb640 1 -- 172.21.15.120:0/800856000 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cac01b040 con 0x55b9a6289b40 2024-09-19T09:21:29.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.293+0000 7f3ca1ffb640 1 -- 172.21.15.120:0/800856000 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cac004c00 con 0x55b9a6289b40 2024-09-19T09:21:29.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.293+0000 7f3cc42c0740 1 -- 172.21.15.120:0/800856000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9a6289b40 msgr2=0x55b9a6288090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:29.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.293+0000 7f3cc42c0740 1 --2- 172.21.15.120:0/800856000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9a6289b40 0x55b9a6288090 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3cac011cc0 tx=0x7f3cac00f790 comp rx=0 tx=0).stop 2024-09-19T09:21:29.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.293+0000 7f3cc42c0740 1 -- 172.21.15.120:0/800856000 shutdown_connections 2024-09-19T09:21:29.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.293+0000 7f3cc42c0740 1 --2- 172.21.15.120:0/800856000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9a6293370 0x55b9a6295760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.293+0000 7f3cc42c0740 1 --2- 172.21.15.120:0/800856000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9a62885d0 0x55b9a6288a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.293+0000 7f3cc42c0740 1 --2- 172.21.15.120:0/800856000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9a6289b40 0x55b9a6288090 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.293+0000 7f3cc42c0740 1 -- 172.21.15.120:0/800856000 >> 172.21.15.120:0/800856000 conn(0x55b9a6251310 msgr2=0x55b9a6284c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:29.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.293+0000 7f3cc42c0740 1 -- 172.21.15.120:0/800856000 shutdown_connections 2024-09-19T09:21:29.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.293+0000 7f3cc42c0740 1 -- 172.21.15.120:0/800856000 wait complete. 2024-09-19T09:21:29.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.294+0000 7f3cc42c0740 1 -- start start 2024-09-19T09:21:29.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.294+0000 7f3cc42c0740 1 Processor -- start 2024-09-19T09:21:29.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.294+0000 7f3cc42c0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9a62885d0 0x55b9a6292d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:29.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.294+0000 7f3cc42c0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9a6289b40 0x55b9a6290010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:29.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.294+0000 7f3cc42c0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9a6293370 0x55b9a6290600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:29.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.294+0000 7f3cc42c0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b9a6296010 con 0x55b9a6289b40 2024-09-19T09:21:29.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.294+0000 7f3cc42c0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b9a61a9060 con 0x55b9a62885d0 2024-09-19T09:21:29.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.294+0000 7f3cc42c0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b9a6296190 con 0x55b9a6293370 2024-09-19T09:21:29.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.294+0000 7f3cc42c0740 10 client.0 _ll_register_callbacks cb 0x55b9a5efc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:29.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.294+0000 7f3cbaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9a6289b40 0x55b9a6290010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:29.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.294+0000 7f3cbaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9a6289b40 0x55b9a6290010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40056/0 (socket says 172.21.15.120:40056) 2024-09-19T09:21:29.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.294+0000 7f3cbaffd640 1 -- 172.21.15.120:0/715865864 learned_addr learned my addr 172.21.15.120:0/715865864 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:29.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.294+0000 7f3cbb7fe640 1 --2- 172.21.15.120:0/715865864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9a62885d0 0x55b9a6292d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:29.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.294+0000 7f3cbbfff640 1 --2- 172.21.15.120:0/715865864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9a6293370 0x55b9a6290600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:29.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.295+0000 7f3cbaffd640 1 -- 172.21.15.120:0/715865864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9a6293370 msgr2=0x55b9a6290600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:29.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.295+0000 7f3cbaffd640 1 --2- 172.21.15.120:0/715865864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9a6293370 0x55b9a6290600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.295+0000 7f3cbaffd640 1 -- 172.21.15.120:0/715865864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9a62885d0 msgr2=0x55b9a6292d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:29.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.295+0000 7f3cbaffd640 1 --2- 172.21.15.120:0/715865864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9a62885d0 0x55b9a6292d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.295+0000 7f3cbaffd640 1 -- 172.21.15.120:0/715865864 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9a5f5f3c0 con 0x55b9a6289b40 2024-09-19T09:21:29.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.295+0000 7f3cbaffd640 1 --2- 172.21.15.120:0/715865864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9a6289b40 0x55b9a6290010 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0000c00 tx=0x7f3cb00c05b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97042ed6cb6a2bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:29.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.295+0000 7f3c9affd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:21:29.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.295+0000 7f3c9affd640 1 -- 172.21.15.120:0/715865864 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb00d5060 con 0x55b9a6289b40 2024-09-19T09:21:29.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.295+0000 7f3cc42c0740 1 -- 172.21.15.120:0/715865864 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cac01b040 con 0x55b9a6289b40 2024-09-19T09:21:29.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.295+0000 7f3c9affd640 1 -- 172.21.15.120:0/715865864 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f3cb00c0170 con 0x55b9a6289b40 2024-09-19T09:21:29.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.295+0000 7f3c9affd640 1 -- 172.21.15.120:0/715865864 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3cb0049040 con 0x55b9a6289b40 2024-09-19T09:21:29.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.295+0000 7f3cc42c0740 10 client.21642 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:29.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.295+0000 7f3cc42c0740 1 -- 172.21.15.120:0/715865864 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b9a62a1ab0 con 0x55b9a6289b40 2024-09-19T09:21:29.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.295+0000 7f3c9affd640 1 -- 172.21.15.120:0/715865864 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cac01b040 con 0x55b9a6289b40 2024-09-19T09:21:29.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.295+0000 7f3c9affd640 1 -- 172.21.15.120:0/715865864 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f3cb00c4920 con 0x55b9a6289b40 2024-09-19T09:21:29.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.296+0000 7f3cc42c0740 10 client.21642 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:29.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.296+0000 7f3cc42c0740 10 client.21642 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:21:29.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.296+0000 7f3cc42c0740 20 client.21642 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:29.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.296+0000 7f3cc42c0740 1 --2- 172.21.15.120:0/715865864 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55b9a62a2f60 0x55b9a62a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:29.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.296+0000 7f3cc42c0740 4 client.21642 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:21:29.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.296+0000 7f3cc42c0740 1 -- 172.21.15.120:0/715865864 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9a61fb860 con 0x55b9a62a2f60 2024-09-19T09:21:29.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.296+0000 7f3cbb7fe640 1 --2- 172.21.15.120:0/715865864 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55b9a62a2f60 0x55b9a62a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:29.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.296+0000 7f3c9affd640 10 client.21642 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:21:29.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.296+0000 7f3cbb7fe640 1 --2- 172.21.15.120:0/715865864 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55b9a62a2f60 0x55b9a62a53a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6363edf8ddad321 server_cookie=84ef90d904613da1 in_seq=0 out_seq=0 2024-09-19T09:21:29.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.297+0000 7f3c9affd640 1 -- 172.21.15.120:0/715865864 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9a61fb860 con 0x55b9a62a2f60 2024-09-19T09:21:29.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.297+0000 7f3c9affd640 10 client.21642 handle_command_reply: tid=0 2024-09-19T09:21:29.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.297+0000 7f3cc42c0740 1 client.21642 shutdown 2024-09-19T09:21:29.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.297+0000 7f3cc42c0740 1 -- 172.21.15.120:0/715865864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9a6289b40 msgr2=0x55b9a6290010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:29.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.297+0000 7f3cc42c0740 1 --2- 172.21.15.120:0/715865864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9a6289b40 0x55b9a6290010 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0000c00 tx=0x7f3cb00c05b0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.297+0000 7f3cc42c0740 1 -- 172.21.15.120:0/715865864 shutdown_connections 2024-09-19T09:21:29.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.297+0000 7f3cc42c0740 1 --2- 172.21.15.120:0/715865864 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55b9a62a2f60 0x55b9a62a53a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.298+0000 7f3cc42c0740 1 --2- 172.21.15.120:0/715865864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9a6293370 0x55b9a6290600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.298+0000 7f3cc42c0740 1 --2- 172.21.15.120:0/715865864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9a6289b40 0x55b9a6290010 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.298+0000 7f3cc42c0740 1 --2- 172.21.15.120:0/715865864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9a62885d0 0x55b9a6292d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.298+0000 7f3cc42c0740 1 -- 172.21.15.120:0/715865864 >> 172.21.15.120:0/715865864 conn(0x55b9a6251310 msgr2=0x55b9a628a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:29.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.298+0000 7f3c9affd640 0 client.21642 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:21:29.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.298+0000 7f3cc42c0740 1 -- 172.21.15.120:0/715865864 shutdown_connections 2024-09-19T09:21:29.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.298+0000 7f3cc42c0740 1 -- 172.21.15.120:0/715865864 wait complete. 2024-09-19T09:21:29.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.298+0000 7f3cc42c0740 20 client.21642 trim_cache size 0 max 16384 2024-09-19T09:21:29.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.311+0000 7f3cc42c0740 1 Processor -- start 2024-09-19T09:21:29.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.311+0000 7f3cc42c0740 1 -- start start 2024-09-19T09:21:29.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.311+0000 7f3cc42c0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9a62885d0 0x55b9a629f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:29.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.311+0000 7f3cc42c0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9a6289b40 0x55b9a628f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:29.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.311+0000 7f3cc42c0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9a6293370 0x55b9a62aac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:29.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.311+0000 7f3cc42c0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc1b3b50 con 0x55b9a6289b40 2024-09-19T09:21:29.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.311+0000 7f3cc42c0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc1b5690 con 0x55b9a6293370 2024-09-19T09:21:29.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.311+0000 7f3cc42c0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc1b4220 con 0x55b9a62885d0 2024-09-19T09:21:29.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.311+0000 7f3cbaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9a6289b40 0x55b9a628f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:29.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.311+0000 7f3cbaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9a6289b40 0x55b9a628f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40066/0 (socket says 172.21.15.120:40066) 2024-09-19T09:21:29.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.311+0000 7f3cbaffd640 1 -- 172.21.15.120:0/130009781 learned_addr learned my addr 172.21.15.120:0/130009781 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:29.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.311+0000 7f3cbbfff640 1 --2- 172.21.15.120:0/130009781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9a6293370 0x55b9a62aac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:29.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.311+0000 7f3cbb7fe640 1 --2- 172.21.15.120:0/130009781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9a62885d0 0x55b9a629f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:29.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.312+0000 7f3cbaffd640 1 -- 172.21.15.120:0/130009781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9a62885d0 msgr2=0x55b9a629f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:29.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.312+0000 7f3cbaffd640 1 --2- 172.21.15.120:0/130009781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9a62885d0 0x55b9a629f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.312+0000 7f3cbaffd640 1 -- 172.21.15.120:0/130009781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9a6293370 msgr2=0x55b9a62aac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:29.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.312+0000 7f3cbaffd640 1 --2- 172.21.15.120:0/130009781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9a6293370 0x55b9a62aac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.312+0000 7f3cbaffd640 1 -- 172.21.15.120:0/130009781 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9a6285960 con 0x55b9a6289b40 2024-09-19T09:21:29.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.312+0000 7f3cbaffd640 1 --2- 172.21.15.120:0/130009781 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9a6289b40 0x55b9a628f8d0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f3cb00c4880 tx=0x7f3cb00ba700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33951a25b57bb932 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:29.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.312+0000 7f3ca17fa640 1 -- 172.21.15.120:0/130009781 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0003d40 con 0x55b9a6289b40 2024-09-19T09:21:29.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.312+0000 7f3ca17fa640 1 -- 172.21.15.120:0/130009781 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cb00c0170 con 0x55b9a6289b40 2024-09-19T09:21:29.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.312+0000 7f3ca17fa640 1 -- 172.21.15.120:0/130009781 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb00c4920 con 0x55b9a6289b40 2024-09-19T09:21:29.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.313+0000 7f3cc42c0740 1 -- 172.21.15.120:0/130009781 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9a6289b40 msgr2=0x55b9a628f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:29.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.313+0000 7f3cc42c0740 1 --2- 172.21.15.120:0/130009781 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9a6289b40 0x55b9a628f8d0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f3cb00c4880 tx=0x7f3cb00ba700 comp rx=0 tx=0).stop 2024-09-19T09:21:29.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.313+0000 7f3cc42c0740 1 -- 172.21.15.120:0/130009781 shutdown_connections 2024-09-19T09:21:29.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.313+0000 7f3cc42c0740 1 --2- 172.21.15.120:0/130009781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9a6293370 0x55b9a62aac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.313+0000 7f3cc42c0740 1 --2- 172.21.15.120:0/130009781 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9a6289b40 0x55b9a628f8d0 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.313+0000 7f3cc42c0740 1 --2- 172.21.15.120:0/130009781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9a62885d0 0x55b9a629f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.313+0000 7f3cc42c0740 1 -- 172.21.15.120:0/130009781 >> 172.21.15.120:0/130009781 conn(0x55b9a6251310 msgr2=0x55b9a628e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:29.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.313+0000 7f3cc42c0740 1 -- 172.21.15.120:0/130009781 shutdown_connections 2024-09-19T09:21:29.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.313+0000 7f3cc42c0740 1 -- 172.21.15.120:0/130009781 wait complete. 2024-09-19T09:21:29.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.313+0000 7f3cc42c0740 1 -- start start 2024-09-19T09:21:29.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.314+0000 7f3cc42c0740 1 Processor -- start 2024-09-19T09:21:29.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.314+0000 7f3cc42c0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9a62885d0 0x55b9a6305f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:29.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.314+0000 7f3cc42c0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9a6289b40 0x55b9a6306470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:29.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.314+0000 7f3cc42c0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9a6293370 0x55b9a62969a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:29.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.314+0000 7f3cc42c0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc1b4fc0 con 0x55b9a6289b40 2024-09-19T09:21:29.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.314+0000 7f3cc42c0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc1b48f0 con 0x55b9a62885d0 2024-09-19T09:21:29.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.314+0000 7f3cc42c0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc1b1b10 con 0x55b9a6293370 2024-09-19T09:21:29.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.314+0000 7f3cc42c0740 10 client.0 _ll_register_callbacks cb 0x7f3cb00095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:29.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.314+0000 7f3cbaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9a6289b40 0x55b9a6306470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:29.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.314+0000 7f3cbb7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9a62885d0 0x55b9a6305f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:29.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.314+0000 7f3cbaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9a6289b40 0x55b9a6306470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40078/0 (socket says 172.21.15.120:40078) 2024-09-19T09:21:29.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.314+0000 7f3cbaffd640 1 -- 172.21.15.120:0/1895232069 learned_addr learned my addr 172.21.15.120:0/1895232069 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:29.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.314+0000 7f3cbbfff640 1 --2- 172.21.15.120:0/1895232069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9a6293370 0x55b9a62969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:29.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.314+0000 7f3cbaffd640 1 -- 172.21.15.120:0/1895232069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9a6293370 msgr2=0x55b9a62969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:29.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.314+0000 7f3cbaffd640 1 --2- 172.21.15.120:0/1895232069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9a6293370 0x55b9a62969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.314+0000 7f3cbaffd640 1 -- 172.21.15.120:0/1895232069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9a62885d0 msgr2=0x55b9a6305f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:29.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.314+0000 7f3cbaffd640 1 --2- 172.21.15.120:0/1895232069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9a62885d0 0x55b9a6305f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.314+0000 7f3cbaffd640 1 -- 172.21.15.120:0/1895232069 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9a6297090 con 0x55b9a6289b40 2024-09-19T09:21:29.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.314+0000 7f3cbaffd640 1 --2- 172.21.15.120:0/1895232069 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9a6289b40 0x55b9a6306470 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f3cb002a990 tx=0x7f3cb00bc320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f705570f23c5ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:29.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.314+0000 7f3c9affd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:21:29.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.314+0000 7f3c9affd640 1 -- 172.21.15.120:0/1895232069 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb00d5060 con 0x55b9a6289b40 2024-09-19T09:21:29.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.315+0000 7f3cc42c0740 1 -- 172.21.15.120:0/1895232069 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cb00c0170 con 0x55b9a6289b40 2024-09-19T09:21:29.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.315+0000 7f3c9affd640 1 -- 172.21.15.120:0/1895232069 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f3cb00a50d0 con 0x55b9a6289b40 2024-09-19T09:21:29.372 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:21:29.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.315+0000 7f3c9affd640 1 -- 172.21.15.120:0/1895232069 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3cb00a52f0 con 0x55b9a6289b40 2024-09-19T09:21:29.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.315+0000 7f3cc42c0740 10 client.21648 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:29.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.315+0000 7f3cc42c0740 1 -- 172.21.15.120:0/1895232069 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b9a6297090 con 0x55b9a6289b40 2024-09-19T09:21:29.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.315+0000 7f3c9affd640 1 -- 172.21.15.120:0/1895232069 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cb00c0170 con 0x55b9a6289b40 2024-09-19T09:21:29.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.315+0000 7f3c9affd640 1 -- 172.21.15.120:0/1895232069 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f3cb00bb7d0 con 0x55b9a6289b40 2024-09-19T09:21:29.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.315+0000 7f3cc42c0740 10 client.21648 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:29.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.315+0000 7f3cc42c0740 10 client.21648 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:21:29.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.315+0000 7f3cc42c0740 20 client.21648 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:29.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.315+0000 7f3cc42c0740 1 --2- 172.21.15.120:0/1895232069 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55b9a630d900 0x55b9a630fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:29.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.315+0000 7f3cc42c0740 4 client.21648 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:21:29.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.315+0000 7f3cc42c0740 1 -- 172.21.15.120:0/1895232069 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b9a61f5d70 con 0x55b9a630d900 2024-09-19T09:21:29.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.315+0000 7f3cbb7fe640 1 --2- 172.21.15.120:0/1895232069 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55b9a630d900 0x55b9a630fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:29.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.316+0000 7f3c9affd640 10 client.21648 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:21:29.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.316+0000 7f3cbb7fe640 1 --2- 172.21.15.120:0/1895232069 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55b9a630d900 0x55b9a630fcf0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=af2c31c74041f965 server_cookie=8677df8eb60ee02a in_seq=0 out_seq=0 2024-09-19T09:21:29.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.316+0000 7f3c9affd640 1 -- 172.21.15.120:0/1895232069 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b9a61f5d70 con 0x55b9a630d900 2024-09-19T09:21:29.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.316+0000 7f3c9affd640 10 client.21648 handle_command_reply: tid=0 2024-09-19T09:21:29.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.316+0000 7f3cc42c0740 1 client.21648 shutdown 2024-09-19T09:21:29.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.316+0000 7f3cc42c0740 1 -- 172.21.15.120:0/1895232069 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9a6289b40 msgr2=0x55b9a6306470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:29.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.316+0000 7f3cc42c0740 1 --2- 172.21.15.120:0/1895232069 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9a6289b40 0x55b9a6306470 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f3cb002a990 tx=0x7f3cb00bc320 comp rx=0 tx=0).stop 2024-09-19T09:21:29.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.316+0000 7f3cc42c0740 1 -- 172.21.15.120:0/1895232069 shutdown_connections 2024-09-19T09:21:29.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.316+0000 7f3cc42c0740 1 --2- 172.21.15.120:0/1895232069 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55b9a630d900 0x55b9a630fcf0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.316+0000 7f3cc42c0740 1 --2- 172.21.15.120:0/1895232069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9a6293370 0x55b9a62969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.316+0000 7f3c9affd640 0 client.21648 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:21:29.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.316+0000 7f3cc42c0740 1 --2- 172.21.15.120:0/1895232069 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9a6289b40 0x55b9a6306470 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.316+0000 7f3cc42c0740 1 --2- 172.21.15.120:0/1895232069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9a62885d0 0x55b9a6305f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.316+0000 7f3cc42c0740 1 -- 172.21.15.120:0/1895232069 >> 172.21.15.120:0/1895232069 conn(0x55b9a6251310 msgr2=0x55b9a628e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:29.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.316+0000 7f3cc42c0740 1 -- 172.21.15.120:0/1895232069 shutdown_connections 2024-09-19T09:21:29.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.316+0000 7f3cc42c0740 1 -- 172.21.15.120:0/1895232069 wait complete. 2024-09-19T09:21:29.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.316+0000 7f3cc42c0740 20 client.21648 trim_cache size 0 max 16384 2024-09-19T09:21:29.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.317+0000 7f3cc1ee7640 1 -- 172.21.15.120:0/773223068 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3ca40a3240 msgr2=0x7f3ca40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:29.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.317+0000 7f3cc1ee7640 1 --2- 172.21.15.120:0/773223068 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3ca40a3240 0x7f3ca40a5700 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3cac00fc90 tx=0x7f3cac013000 comp rx=0 tx=0).stop 2024-09-19T09:21:29.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.317+0000 7f3cc1ee7640 1 -- 172.21.15.120:0/773223068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cbc101460 msgr2=0x7f3cbc1a4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:29.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.317+0000 7f3cc1ee7640 1 --2- 172.21.15.120:0/773223068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cbc101460 0x7f3cbc1a4c20 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0005940 tx=0x7f3cb0004040 comp rx=0 tx=0).stop 2024-09-19T09:21:29.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.317+0000 7f3cc1ee7640 1 -- 172.21.15.120:0/773223068 shutdown_connections 2024-09-19T09:21:29.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.317+0000 7f3cc1ee7640 1 --2- 172.21.15.120:0/773223068 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3ca40a3240 0x7f3ca40a5700 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.317+0000 7f3cc1ee7640 1 --2- 172.21.15.120:0/773223068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cbc105f50 0x7f3cbc19e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.317+0000 7f3cc1ee7640 1 --2- 172.21.15.120:0/773223068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cbc101460 0x7f3cbc1a4c20 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.317+0000 7f3cc1ee7640 1 --2- 172.21.15.120:0/773223068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cbc100b40 0x7f3cbc1a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:29.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.317+0000 7f3cc1ee7640 1 -- 172.21.15.120:0/773223068 >> 172.21.15.120:0/773223068 conn(0x7f3cbc0fc7d0 msgr2=0x7f3cbc0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:29.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.317+0000 7f3cc1ee7640 1 -- 172.21.15.120:0/773223068 shutdown_connections 2024-09-19T09:21:29.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:29.317+0000 7f3cc1ee7640 1 -- 172.21.15.120:0/773223068 wait complete. 2024-09-19T09:21:29.376 INFO:tasks.workunit.client.0.smithi120.stdout: 10 135321 21.00 MB/sec execute 204 sec latency 291.260 ms 2024-09-19T09:21:30.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:30 smithi120 ceph-mon[31150]: pgmap v904: 97 pgs: 97 active+clean; 394 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.0 MiB/s wr, 709 op/s 2024-09-19T09:21:30.370 INFO:tasks.workunit.client.0.smithi120.stdout: 10 135985 21.03 MB/sec execute 205 sec latency 69.794 ms 2024-09-19T09:21:30.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:30 smithi196 ceph-mon[35150]: pgmap v904: 97 pgs: 97 active+clean; 394 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.0 MiB/s wr, 709 op/s 2024-09-19T09:21:30.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:30 smithi153 ceph-mon[35071]: pgmap v904: 97 pgs: 97 active+clean; 394 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.0 MiB/s wr, 709 op/s 2024-09-19T09:21:30.542 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:21:30.823 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:21:31.370 INFO:tasks.workunit.client.0.smithi120.stdout: 10 136658 21.14 MB/sec execute 206 sec latency 84.300 ms 2024-09-19T09:21:32.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:31.997+0000 7f3e46af1640 1 -- 172.21.15.120:0/2132757383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e40102ef0 msgr2=0x7f3e4010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:32.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:31.997+0000 7f3e46af1640 1 --2- 172.21.15.120:0/2132757383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e40102ef0 0x7f3e4010da40 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f3e38009ca0 tx=0x7f3e3802a950 comp rx=0 tx=0).stop 2024-09-19T09:21:32.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:31.998+0000 7f3e46af1640 1 -- 172.21.15.120:0/2132757383 shutdown_connections 2024-09-19T09:21:32.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:31.998+0000 7f3e46af1640 1 --2- 172.21.15.120:0/2132757383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e40102ef0 0x7f3e4010da40 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:31.998+0000 7f3e46af1640 1 --2- 172.21.15.120:0/2132757383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e40102570 0x7f3e401029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:31.998+0000 7f3e46af1640 1 --2- 172.21.15.120:0/2132757383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e40100e20 0x7f3e40101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:31.998+0000 7f3e46af1640 1 -- 172.21.15.120:0/2132757383 >> 172.21.15.120:0/2132757383 conn(0x7f3e400fc7d0 msgr2=0x7f3e400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:32.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:31.998+0000 7f3e46af1640 1 -- 172.21.15.120:0/2132757383 shutdown_connections 2024-09-19T09:21:32.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:31.998+0000 7f3e46af1640 1 -- 172.21.15.120:0/2132757383 wait complete. 2024-09-19T09:21:32.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:31.999+0000 7f3e46af1640 1 Processor -- start 2024-09-19T09:21:32.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:31.999+0000 7f3e46af1640 1 -- start start 2024-09-19T09:21:32.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:31.999+0000 7f3e46af1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e40100e20 0x7f3e401962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:32.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:31.999+0000 7f3e46af1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e40102570 0x7f3e40196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:32.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:31.999+0000 7f3e46af1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e40102ef0 0x7f3e4019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:32.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:31.999+0000 7f3e46af1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4010fec0 con 0x7f3e40100e20 2024-09-19T09:21:32.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:31.999+0000 7f3e46af1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4010fd40 con 0x7f3e40102570 2024-09-19T09:21:32.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:31.999+0000 7f3e46af1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3e40110040 con 0x7f3e40102ef0 2024-09-19T09:21:32.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:31.999+0000 7f3e4488d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e40100e20 0x7f3e401962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:32.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:31.999+0000 7f3e4488d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e40100e20 0x7f3e401962c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44248/0 (socket says 172.21.15.120:44248) 2024-09-19T09:21:32.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:31.999+0000 7f3e4488d640 1 -- 172.21.15.120:0/4224264349 learned_addr learned my addr 172.21.15.120:0/4224264349 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:32.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:31.999+0000 7f3e3ffff640 1 --2- 172.21.15.120:0/4224264349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e40102570 0x7f3e40196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:32.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:31.999+0000 7f3e4508e640 1 --2- 172.21.15.120:0/4224264349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e40102ef0 0x7f3e4019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:32.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.000+0000 7f3e4488d640 1 -- 172.21.15.120:0/4224264349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e40102ef0 msgr2=0x7f3e4019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:32.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.000+0000 7f3e4488d640 1 --2- 172.21.15.120:0/4224264349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e40102ef0 0x7f3e4019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.000+0000 7f3e4488d640 1 -- 172.21.15.120:0/4224264349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e40102570 msgr2=0x7f3e40196800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:32.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.000+0000 7f3e4488d640 1 --2- 172.21.15.120:0/4224264349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e40102570 0x7f3e40196800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.000+0000 7f3e4488d640 1 -- 172.21.15.120:0/4224264349 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e4019b290 con 0x7f3e40100e20 2024-09-19T09:21:32.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.000+0000 7f3e4508e640 1 --2- 172.21.15.120:0/4224264349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e40102ef0 0x7f3e4019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:21:32.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.000+0000 7f3e4488d640 1 --2- 172.21.15.120:0/4224264349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e40100e20 0x7f3e401962c0 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400ea60 tx=0x7f3e3400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:32.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.000+0000 7f3e3dffb640 1 -- 172.21.15.120:0/4224264349 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3400cde0 con 0x7f3e40100e20 2024-09-19T09:21:32.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.000+0000 7f3e46af1640 1 -- 172.21.15.120:0/4224264349 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e4019b580 con 0x7f3e40100e20 2024-09-19T09:21:32.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.000+0000 7f3e3dffb640 1 -- 172.21.15.120:0/4224264349 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e340035f0 con 0x7f3e40100e20 2024-09-19T09:21:32.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.001+0000 7f3e3dffb640 1 -- 172.21.15.120:0/4224264349 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34010670 con 0x7f3e40100e20 2024-09-19T09:21:32.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.001+0000 7f3e46af1640 1 -- 172.21.15.120:0/4224264349 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e401a2e40 con 0x7f3e40100e20 2024-09-19T09:21:32.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.004+0000 7f3e3dffb640 1 -- 172.21.15.120:0/4224264349 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f3e340037a0 con 0x7f3e40100e20 2024-09-19T09:21:32.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.005+0000 7f3e3dffb640 1 --2- 172.21.15.120:0/4224264349 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3e300a3390 0x7f3e300a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:32.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.005+0000 7f3e3ffff640 1 --2- 172.21.15.120:0/4224264349 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3e300a3390 0x7f3e300a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:32.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.005+0000 7f3e3dffb640 1 -- 172.21.15.120:0/4224264349 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f3e340cfb30 con 0x7f3e40100e20 2024-09-19T09:21:32.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.005+0000 7f3e3ffff640 1 --2- 172.21.15.120:0/4224264349 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3e300a3390 0x7f3e300a5850 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00b800 tx=0x7f3e2c00d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:32.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.006+0000 7f3e48eca740 1 Processor -- start 2024-09-19T09:21:32.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.006+0000 7f3e48eca740 1 -- start start 2024-09-19T09:21:32.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.006+0000 7f3e48eca740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648e772cb40 0x5648e772b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:32.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.007+0000 7f3e48eca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648e772b5d0 0x5648e772ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:32.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.007+0000 7f3e48eca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648e7736370 0x5648e7738760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:32.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.007+0000 7f3e48eca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5648e76912f0 con 0x5648e7736370 2024-09-19T09:21:32.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.007+0000 7f3e48eca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5648e7638cf0 con 0x5648e772b5d0 2024-09-19T09:21:32.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.007+0000 7f3e48eca740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5648e768d2b0 con 0x5648e772cb40 2024-09-19T09:21:32.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.007+0000 7f3e4488d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648e772cb40 0x5648e772b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:32.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.007+0000 7f3e4488d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648e772cb40 0x5648e772b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:55476/0 (socket says 172.21.15.120:55476) 2024-09-19T09:21:32.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.007+0000 7f3e4488d640 1 -- 172.21.15.120:0/105288076 learned_addr learned my addr 172.21.15.120:0/105288076 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:32.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.007+0000 7f3e3ffff640 1 --2- 172.21.15.120:0/105288076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648e772b5d0 0x5648e772ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:32.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.007+0000 7f3e4508e640 1 --2- 172.21.15.120:0/105288076 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648e7736370 0x5648e7738760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:32.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.007+0000 7f3e4488d640 1 -- 172.21.15.120:0/105288076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648e772b5d0 msgr2=0x5648e772ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:32.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.007+0000 7f3e4488d640 1 --2- 172.21.15.120:0/105288076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648e772b5d0 0x5648e772ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.007+0000 7f3e4488d640 1 -- 172.21.15.120:0/105288076 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648e7736370 msgr2=0x5648e7738760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:32.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.007+0000 7f3e4488d640 1 --2- 172.21.15.120:0/105288076 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648e7736370 0x5648e7738760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.007+0000 7f3e4488d640 1 -- 172.21.15.120:0/105288076 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648e761d3a0 con 0x5648e772cb40 2024-09-19T09:21:32.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.007+0000 7f3e4508e640 1 --2- 172.21.15.120:0/105288076 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648e7736370 0x5648e7738760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:21:32.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.007+0000 7f3e4488d640 1 --2- 172.21.15.120:0/105288076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648e772cb40 0x5648e772b090 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400f010 tx=0x7f3e3409bc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb528181be227773 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:32.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.007+0000 7f3e2affd640 1 -- 172.21.15.120:0/105288076 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3409c150 con 0x5648e772cb40 2024-09-19T09:21:32.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.008+0000 7f3e2affd640 1 -- 172.21.15.120:0/105288076 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e340a0500 con 0x5648e772cb40 2024-09-19T09:21:32.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.008+0000 7f3e2affd640 1 -- 172.21.15.120:0/105288076 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e340a02d0 con 0x5648e772cb40 2024-09-19T09:21:32.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.008+0000 7f3e48eca740 1 -- 172.21.15.120:0/105288076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648e772cb40 msgr2=0x5648e772b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:32.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.008+0000 7f3e48eca740 1 --2- 172.21.15.120:0/105288076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648e772cb40 0x5648e772b090 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400f010 tx=0x7f3e3409bc90 comp rx=0 tx=0).stop 2024-09-19T09:21:32.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.008+0000 7f3e48eca740 1 -- 172.21.15.120:0/105288076 shutdown_connections 2024-09-19T09:21:32.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.008+0000 7f3e48eca740 1 --2- 172.21.15.120:0/105288076 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648e7736370 0x5648e7738760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.008+0000 7f3e48eca740 1 --2- 172.21.15.120:0/105288076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648e772b5d0 0x5648e772ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.008+0000 7f3e48eca740 1 --2- 172.21.15.120:0/105288076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648e772cb40 0x5648e772b090 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.008+0000 7f3e48eca740 1 -- 172.21.15.120:0/105288076 >> 172.21.15.120:0/105288076 conn(0x5648e76f4310 msgr2=0x5648e7727c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:32.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.008+0000 7f3e48eca740 1 -- 172.21.15.120:0/105288076 shutdown_connections 2024-09-19T09:21:32.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.009+0000 7f3e48eca740 1 -- 172.21.15.120:0/105288076 wait complete. 2024-09-19T09:21:32.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.009+0000 7f3e48eca740 1 -- start start 2024-09-19T09:21:32.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.009+0000 7f3e48eca740 1 Processor -- start 2024-09-19T09:21:32.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.009+0000 7f3e48eca740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648e772b5d0 0x5648e7735d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:32.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.009+0000 7f3e48eca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648e772cb40 0x5648e7732fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:32.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.009+0000 7f3e48eca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648e7736370 0x5648e7733570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:32.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.010+0000 7f3e48eca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5648e7739010 con 0x5648e772cb40 2024-09-19T09:21:32.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.010+0000 7f3e48eca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5648e764c060 con 0x5648e7736370 2024-09-19T09:21:32.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.010+0000 7f3e48eca740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5648e7739190 con 0x5648e772b5d0 2024-09-19T09:21:32.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.010+0000 7f3e3ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648e772cb40 0x5648e7732fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:32.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.010+0000 7f3e4488d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648e772b5d0 0x5648e7735d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:32.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.010+0000 7f3e3ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648e772cb40 0x5648e7732fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44274/0 (socket says 172.21.15.120:44274) 2024-09-19T09:21:32.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.010+0000 7f3e3ffff640 1 -- 172.21.15.120:0/1742142972 learned_addr learned my addr 172.21.15.120:0/1742142972 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:32.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.010+0000 7f3e4488d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648e772b5d0 0x5648e7735d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:55486/0 (socket says 172.21.15.120:55486) 2024-09-19T09:21:32.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.010+0000 7f3e4508e640 1 --2- 172.21.15.120:0/1742142972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648e7736370 0x5648e7733570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:32.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.010+0000 7f3e48eca740 10 client.0 _ll_register_callbacks cb 0x5648e739f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:32.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.010+0000 7f3e4488d640 1 -- 172.21.15.120:0/1742142972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648e7736370 msgr2=0x5648e7733570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:32.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.010+0000 7f3e4488d640 1 --2- 172.21.15.120:0/1742142972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648e7736370 0x5648e7733570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.010+0000 7f3e4488d640 1 -- 172.21.15.120:0/1742142972 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648e772cb40 msgr2=0x5648e7732fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:32.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.010+0000 7f3e4488d640 1 --2- 172.21.15.120:0/1742142972 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648e772cb40 0x5648e7732fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.010+0000 7f3e4488d640 1 -- 172.21.15.120:0/1742142972 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648e74023c0 con 0x5648e772b5d0 2024-09-19T09:21:32.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.010+0000 7f3e4488d640 1 --2- 172.21.15.120:0/1742142972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648e772b5d0 0x5648e7735d20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3e340985b0 tx=0x7f3e34005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b72cddf2e8b4c1a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:32.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.010+0000 7f3e1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:21:32.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.010+0000 7f3e1ffff640 1 -- 172.21.15.120:0/1742142972 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e340a02b0 con 0x5648e772b5d0 2024-09-19T09:21:32.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.010+0000 7f3e1ffff640 1 -- 172.21.15.120:0/1742142972 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f3e340bc0c0 con 0x5648e772b5d0 2024-09-19T09:21:32.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.011+0000 7f3e48eca740 1 -- 172.21.15.120:0/1742142972 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e340a0500 con 0x5648e772b5d0 2024-09-19T09:21:32.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.011+0000 7f3e1ffff640 1 -- 172.21.15.120:0/1742142972 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3e340ac050 con 0x5648e772b5d0 2024-09-19T09:21:32.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.011+0000 7f3e48eca740 10 client.30319 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:32.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.011+0000 7f3e48eca740 1 -- 172.21.15.120:0/1742142972 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5648e7739010 con 0x5648e772b5d0 2024-09-19T09:21:32.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.011+0000 7f3e1ffff640 1 -- 172.21.15.120:0/1742142972 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e340a0500 con 0x5648e772b5d0 2024-09-19T09:21:32.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.011+0000 7f3e1ffff640 1 -- 172.21.15.120:0/1742142972 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f3e3409b5a0 con 0x5648e772b5d0 2024-09-19T09:21:32.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.011+0000 7f3e48eca740 10 client.30319 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:32.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.011+0000 7f3e48eca740 10 client.30319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:21:32.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.011+0000 7f3e48eca740 20 client.30319 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:32.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.011+0000 7f3e48eca740 1 --2- 172.21.15.120:0/1742142972 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5648e7745b10 0x5648e7747f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:32.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.011+0000 7f3e48eca740 4 client.30319 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:21:32.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.011+0000 7f3e48eca740 1 -- 172.21.15.120:0/1742142972 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648e769e860 con 0x5648e7745b10 2024-09-19T09:21:32.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.011+0000 7f3e3ffff640 1 --2- 172.21.15.120:0/1742142972 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5648e7745b10 0x5648e7747f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:32.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.012+0000 7f3e1ffff640 10 client.30319 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:32.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.012+0000 7f3e3ffff640 1 --2- 172.21.15.120:0/1742142972 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5648e7745b10 0x5648e7747f00 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67be43c659af9be4 server_cookie=a6f01a029eb984a6 in_seq=0 out_seq=0 2024-09-19T09:21:32.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.016+0000 7f3e1ffff640 1 -- 172.21.15.120:0/1742142972 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5648e769e860 con 0x5648e7745b10 2024-09-19T09:21:32.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.016+0000 7f3e1ffff640 10 client.30319 handle_command_reply: tid=0 2024-09-19T09:21:32.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.016+0000 7f3e48eca740 1 client.30319 shutdown 2024-09-19T09:21:32.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.016+0000 7f3e48eca740 1 -- 172.21.15.120:0/1742142972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648e772b5d0 msgr2=0x5648e7735d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:32.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.016+0000 7f3e48eca740 1 --2- 172.21.15.120:0/1742142972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648e772b5d0 0x5648e7735d20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3e340985b0 tx=0x7f3e34005780 comp rx=0 tx=0).stop 2024-09-19T09:21:32.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.016+0000 7f3e48eca740 1 -- 172.21.15.120:0/1742142972 shutdown_connections 2024-09-19T09:21:32.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.016+0000 7f3e48eca740 1 --2- 172.21.15.120:0/1742142972 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5648e7745b10 0x5648e7747f00 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.016+0000 7f3e1ffff640 0 client.30319 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:32.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.016+0000 7f3e48eca740 1 --2- 172.21.15.120:0/1742142972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648e7736370 0x5648e7733570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.016+0000 7f3e48eca740 1 --2- 172.21.15.120:0/1742142972 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648e772cb40 0x5648e7732fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.016+0000 7f3e48eca740 1 --2- 172.21.15.120:0/1742142972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648e772b5d0 0x5648e7735d20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.016+0000 7f3e48eca740 1 -- 172.21.15.120:0/1742142972 >> 172.21.15.120:0/1742142972 conn(0x5648e76f4310 msgr2=0x5648e772d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:32.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.017+0000 7f3e48eca740 1 -- 172.21.15.120:0/1742142972 shutdown_connections 2024-09-19T09:21:32.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.017+0000 7f3e48eca740 1 -- 172.21.15.120:0/1742142972 wait complete. 2024-09-19T09:21:32.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.017+0000 7f3e48eca740 20 client.30319 trim_cache size 0 max 16384 2024-09-19T09:21:32.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.031+0000 7f3e48eca740 1 Processor -- start 2024-09-19T09:21:32.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.031+0000 7f3e48eca740 1 -- start start 2024-09-19T09:21:32.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.031+0000 7f3e48eca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648e772b5d0 0x5648e77422b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:32.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.031+0000 7f3e48eca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648e772cb40 0x5648e7732890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:32.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.031+0000 7f3e48eca740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648e7736370 0x5648e772e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:32.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.031+0000 7f3e48eca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401b3d60 con 0x5648e772cb40 2024-09-19T09:21:32.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.031+0000 7f3e48eca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401b58a0 con 0x5648e772b5d0 2024-09-19T09:21:32.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.031+0000 7f3e48eca740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401b4430 con 0x5648e7736370 2024-09-19T09:21:32.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.031+0000 7f3e4508e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648e7736370 0x5648e772e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:32.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.031+0000 7f3e3ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648e772cb40 0x5648e7732890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:32.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.031+0000 7f3e4508e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648e7736370 0x5648e772e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:55498/0 (socket says 172.21.15.120:55498) 2024-09-19T09:21:32.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.031+0000 7f3e4508e640 1 -- 172.21.15.120:0/3240609660 learned_addr learned my addr 172.21.15.120:0/3240609660 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:32.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.032+0000 7f3e4488d640 1 --2- 172.21.15.120:0/3240609660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648e772b5d0 0x5648e77422b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:32.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.032+0000 7f3e4508e640 1 -- 172.21.15.120:0/3240609660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648e772b5d0 msgr2=0x5648e77422b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:32.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.032+0000 7f3e4508e640 1 --2- 172.21.15.120:0/3240609660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648e772b5d0 0x5648e77422b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.032+0000 7f3e4508e640 1 -- 172.21.15.120:0/3240609660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648e772cb40 msgr2=0x5648e7732890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:32.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.032+0000 7f3e4508e640 1 --2- 172.21.15.120:0/3240609660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648e772cb40 0x5648e7732890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.032+0000 7f3e4508e640 1 -- 172.21.15.120:0/3240609660 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648e7736090 con 0x5648e7736370 2024-09-19T09:21:32.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.032+0000 7f3e4508e640 1 --2- 172.21.15.120:0/3240609660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648e7736370 0x5648e772e090 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3e3802ae70 tx=0x7f3e38005d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ccdff391d284d73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:32.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.035+0000 7f3e2a7fc640 1 -- 172.21.15.120:0/3240609660 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38034040 con 0x5648e7736370 2024-09-19T09:21:32.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.035+0000 7f3e2a7fc640 1 -- 172.21.15.120:0/3240609660 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e3804a070 con 0x5648e7736370 2024-09-19T09:21:32.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.036+0000 7f3e2a7fc640 1 -- 172.21.15.120:0/3240609660 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e380046a0 con 0x5648e7736370 2024-09-19T09:21:32.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.036+0000 7f3e48eca740 1 -- 172.21.15.120:0/3240609660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648e7736370 msgr2=0x5648e772e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:32.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.036+0000 7f3e48eca740 1 --2- 172.21.15.120:0/3240609660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648e7736370 0x5648e772e090 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3e3802ae70 tx=0x7f3e38005d80 comp rx=0 tx=0).stop 2024-09-19T09:21:32.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.036+0000 7f3e48eca740 1 -- 172.21.15.120:0/3240609660 shutdown_connections 2024-09-19T09:21:32.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.036+0000 7f3e48eca740 1 --2- 172.21.15.120:0/3240609660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648e7736370 0x5648e772e090 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.036+0000 7f3e48eca740 1 --2- 172.21.15.120:0/3240609660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648e772cb40 0x5648e7732890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.036+0000 7f3e48eca740 1 --2- 172.21.15.120:0/3240609660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648e772b5d0 0x5648e77422b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.036+0000 7f3e48eca740 1 -- 172.21.15.120:0/3240609660 >> 172.21.15.120:0/3240609660 conn(0x5648e76f4310 msgr2=0x5648e7726d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:32.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.036+0000 7f3e48eca740 1 -- 172.21.15.120:0/3240609660 shutdown_connections 2024-09-19T09:21:32.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.036+0000 7f3e48eca740 1 -- 172.21.15.120:0/3240609660 wait complete. 2024-09-19T09:21:32.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.037+0000 7f3e48eca740 1 -- start start 2024-09-19T09:21:32.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.037+0000 7f3e48eca740 1 Processor -- start 2024-09-19T09:21:32.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.037+0000 7f3e48eca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648e772b5d0 0x5648e773e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:32.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.037+0000 7f3e48eca740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648e772cb40 0x5648e773ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:32.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.037+0000 7f3e48eca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648e7736370 0x5648e7739b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:32.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.037+0000 7f3e48eca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401b51d0 con 0x5648e7736370 2024-09-19T09:21:32.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.037+0000 7f3e48eca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401b4b00 con 0x5648e772b5d0 2024-09-19T09:21:32.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.037+0000 7f3e48eca740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401b1d60 con 0x5648e772cb40 2024-09-19T09:21:32.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.037+0000 7f3e48eca740 10 client.0 _ll_register_callbacks cb 0x7f3e340984c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:32.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.038+0000 7f3e3ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648e772cb40 0x5648e773ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:32.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.038+0000 7f3e3ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648e772cb40 0x5648e773ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:55512/0 (socket says 172.21.15.120:55512) 2024-09-19T09:21:32.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.038+0000 7f3e3ffff640 1 -- 172.21.15.120:0/2068885376 learned_addr learned my addr 172.21.15.120:0/2068885376 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:32.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.038+0000 7f3e4488d640 1 --2- 172.21.15.120:0/2068885376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648e772b5d0 0x5648e773e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:32.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.038+0000 7f3e4508e640 1 --2- 172.21.15.120:0/2068885376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648e7736370 0x5648e7739b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:32.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.038+0000 7f3e3ffff640 1 -- 172.21.15.120:0/2068885376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648e772b5d0 msgr2=0x5648e773e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:32.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.038+0000 7f3e3ffff640 1 --2- 172.21.15.120:0/2068885376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648e772b5d0 0x5648e773e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.038+0000 7f3e3ffff640 1 -- 172.21.15.120:0/2068885376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648e7736370 msgr2=0x5648e7739b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:32.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.038+0000 7f3e3ffff640 1 --2- 172.21.15.120:0/2068885376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648e7736370 0x5648e7739b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.038+0000 7f3e3ffff640 1 -- 172.21.15.120:0/2068885376 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648e773a3c0 con 0x5648e772cb40 2024-09-19T09:21:32.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.038+0000 7f3e3ffff640 1 --2- 172.21.15.120:0/2068885376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648e772cb40 0x5648e773ec70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x5648e7734610 tx=0x7f3e2c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e07fc032446bf19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:32.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.038+0000 7f3e1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:21:32.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.039+0000 7f3e1ffff640 1 -- 172.21.15.120:0/2068885376 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c034060 con 0x5648e772cb40 2024-09-19T09:21:32.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.039+0000 7f3e1ffff640 1 -- 172.21.15.120:0/2068885376 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f3e2c009050 con 0x5648e772cb40 2024-09-19T09:21:32.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.039+0000 7f3e48eca740 1 -- 172.21.15.120:0/2068885376 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e3804a070 con 0x5648e772cb40 2024-09-19T09:21:32.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.039+0000 7f3e1ffff640 1 -- 172.21.15.120:0/2068885376 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3e2c018750 con 0x5648e772cb40 2024-09-19T09:21:32.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.039+0000 7f3e48eca740 10 client.30325 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:32.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.039+0000 7f3e48eca740 1 -- 172.21.15.120:0/2068885376 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5648e77b0250 con 0x5648e772cb40 2024-09-19T09:21:32.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.039+0000 7f3e1ffff640 1 -- 172.21.15.120:0/2068885376 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e3804a070 con 0x5648e772cb40 2024-09-19T09:21:32.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.039+0000 7f3e1ffff640 1 -- 172.21.15.120:0/2068885376 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f3e2c039070 con 0x5648e772cb40 2024-09-19T09:21:32.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.039+0000 7f3e48eca740 10 client.30325 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:32.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.039+0000 7f3e48eca740 10 client.30325 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:21:32.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.039+0000 7f3e48eca740 20 client.30325 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:32.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.039+0000 7f3e48eca740 1 --2- 172.21.15.120:0/2068885376 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5648e77b09b0 0x5648e77b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:32.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.039+0000 7f3e48eca740 4 client.30325 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:21:32.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.039+0000 7f3e48eca740 1 -- 172.21.15.120:0/2068885376 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5648e7698d70 con 0x5648e77b09b0 2024-09-19T09:21:32.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.040+0000 7f3e4488d640 1 --2- 172.21.15.120:0/2068885376 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5648e77b09b0 0x5648e77b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:32.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.040+0000 7f3e1ffff640 10 client.30325 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:32.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.040+0000 7f3e4488d640 1 --2- 172.21.15.120:0/2068885376 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5648e77b09b0 0x5648e77b2dd0 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=887764fd8b33414d server_cookie=5bcb8db8325dc597 in_seq=0 out_seq=0 2024-09-19T09:21:32.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.042+0000 7f3e1ffff640 1 -- 172.21.15.120:0/2068885376 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5648e7698d70 con 0x5648e77b09b0 2024-09-19T09:21:32.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.042+0000 7f3e1ffff640 10 client.30325 handle_command_reply: tid=0 2024-09-19T09:21:32.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.042+0000 7f3e48eca740 1 client.30325 shutdown 2024-09-19T09:21:32.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.042+0000 7f3e48eca740 1 -- 172.21.15.120:0/2068885376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648e772cb40 msgr2=0x5648e773ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:32.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.042+0000 7f3e48eca740 1 --2- 172.21.15.120:0/2068885376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648e772cb40 0x5648e773ec70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x5648e7734610 tx=0x7f3e2c0046c0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.042+0000 7f3e48eca740 1 -- 172.21.15.120:0/2068885376 shutdown_connections 2024-09-19T09:21:32.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.042+0000 7f3e48eca740 1 --2- 172.21.15.120:0/2068885376 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5648e77b09b0 0x5648e77b2dd0 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.042+0000 7f3e48eca740 1 --2- 172.21.15.120:0/2068885376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648e7736370 0x5648e7739b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.042+0000 7f3e1ffff640 0 client.30325 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:32.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.042+0000 7f3e48eca740 1 --2- 172.21.15.120:0/2068885376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648e772cb40 0x5648e773ec70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.042+0000 7f3e48eca740 1 --2- 172.21.15.120:0/2068885376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5648e772b5d0 0x5648e773e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.042+0000 7f3e48eca740 1 -- 172.21.15.120:0/2068885376 >> 172.21.15.120:0/2068885376 conn(0x5648e76f4310 msgr2=0x5648e7731770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:32.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.042+0000 7f3e48eca740 1 -- 172.21.15.120:0/2068885376 shutdown_connections 2024-09-19T09:21:32.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.042+0000 7f3e48eca740 1 -- 172.21.15.120:0/2068885376 wait complete. 2024-09-19T09:21:32.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.042+0000 7f3e48eca740 20 client.30325 trim_cache size 0 max 16384 2024-09-19T09:21:32.045 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:21:32.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.043+0000 7f3e46af1640 1 -- 172.21.15.120:0/4224264349 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3e300a3390 msgr2=0x7f3e300a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:32.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.043+0000 7f3e46af1640 1 --2- 172.21.15.120:0/4224264349 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3e300a3390 0x7f3e300a5850 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00b800 tx=0x7f3e2c00d040 comp rx=0 tx=0).stop 2024-09-19T09:21:32.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.043+0000 7f3e46af1640 1 -- 172.21.15.120:0/4224264349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e40100e20 msgr2=0x7f3e401962c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:32.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.043+0000 7f3e46af1640 1 --2- 172.21.15.120:0/4224264349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e40100e20 0x7f3e401962c0 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400ea60 tx=0x7f3e3400ef20 comp rx=0 tx=0).stop 2024-09-19T09:21:32.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.043+0000 7f3e46af1640 1 -- 172.21.15.120:0/4224264349 shutdown_connections 2024-09-19T09:21:32.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.043+0000 7f3e46af1640 1 --2- 172.21.15.120:0/4224264349 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f3e300a3390 0x7f3e300a5850 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.043+0000 7f3e46af1640 1 --2- 172.21.15.120:0/4224264349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e40102ef0 0x7f3e4019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.043+0000 7f3e46af1640 1 --2- 172.21.15.120:0/4224264349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e40102570 0x7f3e40196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.043+0000 7f3e46af1640 1 --2- 172.21.15.120:0/4224264349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e40100e20 0x7f3e401962c0 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:32.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.043+0000 7f3e46af1640 1 -- 172.21.15.120:0/4224264349 >> 172.21.15.120:0/4224264349 conn(0x7f3e400fc7d0 msgr2=0x7f3e40106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:32.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.043+0000 7f3e46af1640 1 -- 172.21.15.120:0/4224264349 shutdown_connections 2024-09-19T09:21:32.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:32.043+0000 7f3e46af1640 1 -- 172.21.15.120:0/4224264349 wait complete. 2024-09-19T09:21:32.292 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:32 smithi120 ceph-mon[31150]: pgmap v905: 97 pgs: 97 active+clean; 390 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.7 MiB/s wr, 678 op/s 2024-09-19T09:21:32.370 INFO:tasks.workunit.client.0.smithi120.stdout: 10 137334 21.23 MB/sec execute 207 sec latency 64.740 ms 2024-09-19T09:21:32.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:32 smithi196 ceph-mon[35150]: pgmap v905: 97 pgs: 97 active+clean; 390 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.7 MiB/s wr, 678 op/s 2024-09-19T09:21:32.441 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:32 smithi153 ceph-mon[35071]: pgmap v905: 97 pgs: 97 active+clean; 390 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.7 MiB/s wr, 678 op/s 2024-09-19T09:21:33.370 INFO:tasks.workunit.client.0.smithi120.stdout: 10 137915 21.30 MB/sec execute 208 sec latency 53.412 ms 2024-09-19T09:21:33.951 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:21:33.951 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:21:34.254 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:21:34.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:34 smithi120 ceph-mon[31150]: pgmap v906: 97 pgs: 97 active+clean; 377 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 6.6 MiB/s wr, 686 op/s 2024-09-19T09:21:34.370 INFO:tasks.workunit.client.0.smithi120.stdout: 10 138428 21.30 MB/sec execute 209 sec latency 69.527 ms 2024-09-19T09:21:34.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:34 smithi196 ceph-mon[35150]: pgmap v906: 97 pgs: 97 active+clean; 377 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 6.6 MiB/s wr, 686 op/s 2024-09-19T09:21:34.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:34 smithi153 ceph-mon[35071]: pgmap v906: 97 pgs: 97 active+clean; 377 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 6.6 MiB/s wr, 686 op/s 2024-09-19T09:21:34.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.839+0000 7f2fa058e640 1 -- 172.21.15.120:0/2218389498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f981036e0 msgr2=0x7f2f98107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:34.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.839+0000 7f2fa058e640 1 --2- 172.21.15.120:0/2218389498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f981036e0 0x7f2f98107b90 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009930 tx=0x7f2f8802a420 comp rx=0 tx=0).stop 2024-09-19T09:21:34.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.839+0000 7f2fa058e640 1 -- 172.21.15.120:0/2218389498 shutdown_connections 2024-09-19T09:21:34.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.839+0000 7f2fa058e640 1 --2- 172.21.15.120:0/2218389498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2f981082a0 0x7f2f9810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.839+0000 7f2fa058e640 1 --2- 172.21.15.120:0/2218389498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f981036e0 0x7f2f98107b90 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.839+0000 7f2fa058e640 1 --2- 172.21.15.120:0/2218389498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f98102d30 0x7f2f98103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.839+0000 7f2fa058e640 1 -- 172.21.15.120:0/2218389498 >> 172.21.15.120:0/2218389498 conn(0x7f2f980fc7d0 msgr2=0x7f2f980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:34.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.839+0000 7f2fa058e640 1 -- 172.21.15.120:0/2218389498 shutdown_connections 2024-09-19T09:21:34.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.839+0000 7f2fa058e640 1 -- 172.21.15.120:0/2218389498 wait complete. 2024-09-19T09:21:34.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.839+0000 7f2fa058e640 1 Processor -- start 2024-09-19T09:21:34.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.839+0000 7f2fa058e640 1 -- start start 2024-09-19T09:21:34.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.840+0000 7f2fa058e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f98102d30 0x7f2f98100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:34.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.840+0000 7f2fa058e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2f981036e0 0x7f2f98100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:34.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.840+0000 7f2fa058e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f981082a0 0x7f2f98101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:34.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.840+0000 7f2fa058e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2f981120a0 con 0x7f2f98102d30 2024-09-19T09:21:34.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.840+0000 7f2fa058e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2f98111f20 con 0x7f2f981036e0 2024-09-19T09:21:34.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.840+0000 7f2fa058e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2f98112220 con 0x7f2f981082a0 2024-09-19T09:21:34.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.840+0000 7f2f9e32a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f98102d30 0x7f2f98100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:34.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.840+0000 7f2f9eb2b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f981082a0 0x7f2f98101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:34.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.840+0000 7f2f9e32a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f98102d30 0x7f2f98100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44308/0 (socket says 172.21.15.120:44308) 2024-09-19T09:21:34.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.840+0000 7f2f9e32a640 1 -- 172.21.15.120:0/2364660310 learned_addr learned my addr 172.21.15.120:0/2364660310 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:34.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.840+0000 7f2f9db29640 1 --2- 172.21.15.120:0/2364660310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2f981036e0 0x7f2f98100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:34.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.840+0000 7f2f9e32a640 1 -- 172.21.15.120:0/2364660310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f981082a0 msgr2=0x7f2f98101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:34.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.840+0000 7f2f9e32a640 1 --2- 172.21.15.120:0/2364660310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f981082a0 0x7f2f98101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.840+0000 7f2f9e32a640 1 -- 172.21.15.120:0/2364660310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2f981036e0 msgr2=0x7f2f98100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:34.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.840+0000 7f2f9e32a640 1 --2- 172.21.15.120:0/2364660310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2f981036e0 0x7f2f98100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.840+0000 7f2f9e32a640 1 -- 172.21.15.120:0/2364660310 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f981aaf40 con 0x7f2f98102d30 2024-09-19T09:21:34.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.841+0000 7f2f9e32a640 1 --2- 172.21.15.120:0/2364660310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f98102d30 0x7f2f98100990 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400e9f0 tx=0x7f2f9400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:34.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.841+0000 7f2f8f7fe640 1 -- 172.21.15.120:0/2364660310 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9400cd80 con 0x7f2f98102d30 2024-09-19T09:21:34.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.841+0000 7f2fa058e640 1 -- 172.21.15.120:0/2364660310 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f981ab170 con 0x7f2f98102d30 2024-09-19T09:21:34.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.841+0000 7f2f8f7fe640 1 -- 172.21.15.120:0/2364660310 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f94010910 con 0x7f2f98102d30 2024-09-19T09:21:34.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.841+0000 7f2f8f7fe640 1 -- 172.21.15.120:0/2364660310 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f94018950 con 0x7f2f98102d30 2024-09-19T09:21:34.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.841+0000 7f2fa058e640 1 -- 172.21.15.120:0/2364660310 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f981ab6d0 con 0x7f2f98102d30 2024-09-19T09:21:34.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.843+0000 7f2f8f7fe640 1 -- 172.21.15.120:0/2364660310 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f2f94010ac0 con 0x7f2f98102d30 2024-09-19T09:21:34.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.844+0000 7f2f8f7fe640 1 --2- 172.21.15.120:0/2364660310 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2f840a3240 0x7f2f840a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:34.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.844+0000 7f2f9db29640 1 --2- 172.21.15.120:0/2364660310 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2f840a3240 0x7f2f840a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:34.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.844+0000 7f2f8f7fe640 1 -- 172.21.15.120:0/2364660310 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f2f940cfcc0 con 0x7f2f98102d30 2024-09-19T09:21:34.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.844+0000 7f2fa2967740 1 Processor -- start 2024-09-19T09:21:34.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.844+0000 7f2fa2967740 1 -- start start 2024-09-19T09:21:34.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.844+0000 7f2f9db29640 1 --2- 172.21.15.120:0/2364660310 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2f840a3240 0x7f2f840a5700 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2f88005940 tx=0x7f2f880058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:34.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.845+0000 7f2fa2967740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55837d826b60 0x55837d826f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:34.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.845+0000 7f2fa2967740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837d827480 0x55837d831800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:34.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.845+0000 7f2fa2967740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55837d831d40 0x55837d834130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:34.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.845+0000 7f2fa2967740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55837d78c280 con 0x55837d831d40 2024-09-19T09:21:34.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.845+0000 7f2fa2967740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55837d733b10 con 0x55837d826b60 2024-09-19T09:21:34.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.845+0000 7f2fa2967740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55837d788240 con 0x55837d827480 2024-09-19T09:21:34.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.845+0000 7f2f9db29640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837d827480 0x55837d831800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:34.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.845+0000 7f2f9e32a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55837d826b60 0x55837d826f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:34.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.845+0000 7f2f9db29640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837d827480 0x55837d831800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:55550/0 (socket says 172.21.15.120:55550) 2024-09-19T09:21:34.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.845+0000 7f2f9db29640 1 -- 172.21.15.120:0/706740071 learned_addr learned my addr 172.21.15.120:0/706740071 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:34.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.845+0000 7f2f9eb2b640 1 --2- 172.21.15.120:0/706740071 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55837d831d40 0x55837d834130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:34.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.845+0000 7f2f9e32a640 1 -- 172.21.15.120:0/706740071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837d827480 msgr2=0x55837d831800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:34.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.845+0000 7f2f9e32a640 1 --2- 172.21.15.120:0/706740071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837d827480 0x55837d831800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.845+0000 7f2f9e32a640 1 -- 172.21.15.120:0/706740071 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55837d831d40 msgr2=0x55837d834130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:34.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.845+0000 7f2f9e32a640 1 --2- 172.21.15.120:0/706740071 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55837d831d40 0x55837d834130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.845+0000 7f2f9e32a640 1 -- 172.21.15.120:0/706740071 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55837d7401f0 con 0x55837d826b60 2024-09-19T09:21:34.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.845+0000 7f2f9db29640 1 --2- 172.21.15.120:0/706740071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837d827480 0x55837d831800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:21:34.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.845+0000 7f2f9e32a640 1 --2- 172.21.15.120:0/706740071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55837d826b60 0x55837d826f40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400e9c0 tx=0x7f2f9409c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b46923517379b04e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:34.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.845+0000 7f2f8cff9640 1 -- 172.21.15.120:0/706740071 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9409a690 con 0x55837d826b60 2024-09-19T09:21:34.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.846+0000 7f2f8cff9640 1 -- 172.21.15.120:0/706740071 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f94098a00 con 0x55837d826b60 2024-09-19T09:21:34.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.846+0000 7f2f8cff9640 1 -- 172.21.15.120:0/706740071 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f940bc3d0 con 0x55837d826b60 2024-09-19T09:21:34.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.846+0000 7f2fa2967740 1 -- 172.21.15.120:0/706740071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55837d826b60 msgr2=0x55837d826f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:34.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.846+0000 7f2fa2967740 1 --2- 172.21.15.120:0/706740071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55837d826b60 0x55837d826f40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400e9c0 tx=0x7f2f9409c450 comp rx=0 tx=0).stop 2024-09-19T09:21:34.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.846+0000 7f2fa2967740 1 -- 172.21.15.120:0/706740071 shutdown_connections 2024-09-19T09:21:34.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.846+0000 7f2fa2967740 1 --2- 172.21.15.120:0/706740071 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55837d831d40 0x55837d834130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.846+0000 7f2fa2967740 1 --2- 172.21.15.120:0/706740071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837d827480 0x55837d831800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.846+0000 7f2fa2967740 1 --2- 172.21.15.120:0/706740071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55837d826b60 0x55837d826f40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.846+0000 7f2fa2967740 1 -- 172.21.15.120:0/706740071 >> 172.21.15.120:0/706740071 conn(0x55837d8230f0 msgr2=0x55837d8234d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:34.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.847+0000 7f2fa2967740 1 -- 172.21.15.120:0/706740071 shutdown_connections 2024-09-19T09:21:34.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.847+0000 7f2fa2967740 1 -- 172.21.15.120:0/706740071 wait complete. 2024-09-19T09:21:34.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.847+0000 7f2fa2967740 1 -- start start 2024-09-19T09:21:34.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.847+0000 7f2fa2967740 1 Processor -- start 2024-09-19T09:21:34.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.847+0000 7f2fa2967740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55837d827480 0x55837d82ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:34.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.847+0000 7f2fa2967740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837d831d40 0x55837d82f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:34.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.847+0000 7f2fa2967740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55837d82f8e0 0x55837d83dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:34.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.847+0000 7f2fa2967740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55837d834910 con 0x55837d82f8e0 2024-09-19T09:21:34.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.847+0000 7f2fa2967740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55837d746f10 con 0x55837d827480 2024-09-19T09:21:34.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.847+0000 7f2fa2967740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55837d834a90 con 0x55837d831d40 2024-09-19T09:21:34.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.847+0000 7f2f9eb2b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55837d82f8e0 0x55837d83dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:34.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.848+0000 7f2f9db29640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837d831d40 0x55837d82f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:34.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.848+0000 7f2f9eb2b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55837d82f8e0 0x55837d83dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44324/0 (socket says 172.21.15.120:44324) 2024-09-19T09:21:34.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.848+0000 7f2f9eb2b640 1 -- 172.21.15.120:0/4010378389 learned_addr learned my addr 172.21.15.120:0/4010378389 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:34.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.848+0000 7f2f9db29640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837d831d40 0x55837d82f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:55554/0 (socket says 172.21.15.120:55554) 2024-09-19T09:21:34.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.848+0000 7f2fa2967740 10 client.0 _ll_register_callbacks cb 0x55837d70fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:34.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.848+0000 7f2f9e32a640 1 --2- 172.21.15.120:0/4010378389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55837d827480 0x55837d82ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:34.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.848+0000 7f2f9eb2b640 1 -- 172.21.15.120:0/4010378389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837d831d40 msgr2=0x55837d82f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:34.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.848+0000 7f2f9eb2b640 1 --2- 172.21.15.120:0/4010378389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837d831d40 0x55837d82f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.848+0000 7f2f9eb2b640 1 -- 172.21.15.120:0/4010378389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55837d827480 msgr2=0x55837d82ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:34.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.848+0000 7f2f9eb2b640 1 --2- 172.21.15.120:0/4010378389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55837d827480 0x55837d82ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.848+0000 7f2f9eb2b640 1 -- 172.21.15.120:0/4010378389 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55837d4fd6a0 con 0x55837d82f8e0 2024-09-19T09:21:34.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.848+0000 7f2f9db29640 1 --2- 172.21.15.120:0/4010378389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837d831d40 0x55837d82f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:21:34.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.848+0000 7f2f9eb2b640 1 --2- 172.21.15.120:0/4010378389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55837d82f8e0 0x55837d83dd80 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f2f90011d10 tx=0x7f2f9000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36309ce60a57c75e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:34.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.848+0000 7f2f81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:21:34.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.848+0000 7f2f81ffb640 1 -- 172.21.15.120:0/4010378389 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9000fe60 con 0x55837d82f8e0 2024-09-19T09:21:34.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.848+0000 7f2fa2967740 1 -- 172.21.15.120:0/4010378389 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f94098a00 con 0x55837d82f8e0 2024-09-19T09:21:34.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.848+0000 7f2f81ffb640 1 -- 172.21.15.120:0/4010378389 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f2f90004b20 con 0x55837d82f8e0 2024-09-19T09:21:34.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.848+0000 7f2f81ffb640 1 -- 172.21.15.120:0/4010378389 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2f90004d20 con 0x55837d82f8e0 2024-09-19T09:21:34.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.848+0000 7f2fa2967740 10 client.21675 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:34.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.848+0000 7f2fa2967740 1 -- 172.21.15.120:0/4010378389 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55837d840c50 con 0x55837d82f8e0 2024-09-19T09:21:34.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.848+0000 7f2f81ffb640 1 -- 172.21.15.120:0/4010378389 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f94098a00 con 0x55837d82f8e0 2024-09-19T09:21:34.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.849+0000 7f2f81ffb640 1 -- 172.21.15.120:0/4010378389 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f2f90005020 con 0x55837d82f8e0 2024-09-19T09:21:34.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.849+0000 7f2fa2967740 10 client.21675 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:34.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.849+0000 7f2fa2967740 10 client.21675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:21:34.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.849+0000 7f2fa2967740 20 client.21675 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:34.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.849+0000 7f2fa2967740 1 --2- 172.21.15.120:0/4010378389 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55837d842060 0x55837d844450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:34.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.849+0000 7f2fa2967740 4 client.21675 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:21:34.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.849+0000 7f2fa2967740 1 -- 172.21.15.120:0/4010378389 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55837d7997f0 con 0x55837d842060 2024-09-19T09:21:34.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.849+0000 7f2f9e32a640 1 --2- 172.21.15.120:0/4010378389 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55837d842060 0x55837d844450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:34.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.849+0000 7f2f81ffb640 10 client.21675 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:34.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.849+0000 7f2f9e32a640 1 --2- 172.21.15.120:0/4010378389 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55837d842060 0x55837d844450 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fbc12ccc9feb53d server_cookie=84530f60c0e326ab in_seq=0 out_seq=0 2024-09-19T09:21:34.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.851+0000 7f2f81ffb640 1 -- 172.21.15.120:0/4010378389 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55837d7997f0 con 0x55837d842060 2024-09-19T09:21:34.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.851+0000 7f2f81ffb640 10 client.21675 handle_command_reply: tid=0 2024-09-19T09:21:34.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.851+0000 7f2fa2967740 1 client.21675 shutdown 2024-09-19T09:21:34.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.851+0000 7f2fa2967740 1 -- 172.21.15.120:0/4010378389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55837d82f8e0 msgr2=0x55837d83dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:34.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.851+0000 7f2fa2967740 1 --2- 172.21.15.120:0/4010378389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55837d82f8e0 0x55837d83dd80 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f2f90011d10 tx=0x7f2f9000f790 comp rx=0 tx=0).stop 2024-09-19T09:21:34.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.851+0000 7f2fa2967740 1 -- 172.21.15.120:0/4010378389 shutdown_connections 2024-09-19T09:21:34.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.851+0000 7f2fa2967740 1 --2- 172.21.15.120:0/4010378389 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55837d842060 0x55837d844450 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.851+0000 7f2fa2967740 1 --2- 172.21.15.120:0/4010378389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55837d82f8e0 0x55837d83dd80 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.851+0000 7f2fa2967740 1 --2- 172.21.15.120:0/4010378389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837d831d40 0x55837d82f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.851+0000 7f2fa2967740 1 --2- 172.21.15.120:0/4010378389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55837d827480 0x55837d82ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.851+0000 7f2fa2967740 1 -- 172.21.15.120:0/4010378389 >> 172.21.15.120:0/4010378389 conn(0x55837d8230f0 msgr2=0x55837d822270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:34.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.851+0000 7f2f81ffb640 0 client.21675 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:34.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.851+0000 7f2fa2967740 1 -- 172.21.15.120:0/4010378389 shutdown_connections 2024-09-19T09:21:34.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.851+0000 7f2fa2967740 1 -- 172.21.15.120:0/4010378389 wait complete. 2024-09-19T09:21:34.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.851+0000 7f2fa2967740 20 client.21675 trim_cache size 0 max 16384 2024-09-19T09:21:34.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.871+0000 7f2fa2967740 1 Processor -- start 2024-09-19T09:21:34.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.871+0000 7f2fa2967740 1 -- start start 2024-09-19T09:21:34.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.871+0000 7f2fa2967740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55837d827480 0x55837d829610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:34.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.871+0000 7f2fa2967740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837d82f8e0 0x55837d83db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:34.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.871+0000 7f2fa2967740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55837d831d40 0x55837d849d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:34.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.871+0000 7f2fa2967740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2f981b3c80 con 0x55837d831d40 2024-09-19T09:21:34.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.871+0000 7f2fa2967740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2f981b57c0 con 0x55837d827480 2024-09-19T09:21:34.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.871+0000 7f2fa2967740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2f981b4350 con 0x55837d82f8e0 2024-09-19T09:21:34.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.871+0000 7f2f9eb2b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55837d831d40 0x55837d849d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:34.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.871+0000 7f2f9eb2b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55837d831d40 0x55837d849d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44336/0 (socket says 172.21.15.120:44336) 2024-09-19T09:21:34.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.871+0000 7f2f9eb2b640 1 -- 172.21.15.120:0/881034082 learned_addr learned my addr 172.21.15.120:0/881034082 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:34.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.871+0000 7f2f9e32a640 1 --2- 172.21.15.120:0/881034082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55837d827480 0x55837d829610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:34.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.871+0000 7f2f9db29640 1 --2- 172.21.15.120:0/881034082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837d82f8e0 0x55837d83db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:34.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.871+0000 7f2f9eb2b640 1 -- 172.21.15.120:0/881034082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837d82f8e0 msgr2=0x55837d83db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:34.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.871+0000 7f2f9eb2b640 1 --2- 172.21.15.120:0/881034082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837d82f8e0 0x55837d83db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.871+0000 7f2f9eb2b640 1 -- 172.21.15.120:0/881034082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55837d827480 msgr2=0x55837d829610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:34.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.871+0000 7f2f9eb2b640 1 --2- 172.21.15.120:0/881034082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55837d827480 0x55837d829610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.871+0000 7f2f9eb2b640 1 -- 172.21.15.120:0/881034082 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55837d81d5f0 con 0x55837d831d40 2024-09-19T09:21:34.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.871+0000 7f2f9db29640 1 --2- 172.21.15.120:0/881034082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837d82f8e0 0x55837d83db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:21:34.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.872+0000 7f2f9eb2b640 1 --2- 172.21.15.120:0/881034082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55837d831d40 0x55837d849d20 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f2f90009600 tx=0x7f2f90011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ded3aff6bcfe82 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:34.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.872+0000 7f2f83fff640 1 -- 172.21.15.120:0/881034082 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f90004b00 con 0x55837d831d40 2024-09-19T09:21:34.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.872+0000 7f2f83fff640 1 -- 172.21.15.120:0/881034082 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f900056b0 con 0x55837d831d40 2024-09-19T09:21:34.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.872+0000 7f2f83fff640 1 -- 172.21.15.120:0/881034082 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f900269e0 con 0x55837d831d40 2024-09-19T09:21:34.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.872+0000 7f2fa2967740 1 -- 172.21.15.120:0/881034082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55837d831d40 msgr2=0x55837d849d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:34.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.872+0000 7f2fa2967740 1 --2- 172.21.15.120:0/881034082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55837d831d40 0x55837d849d20 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f2f90009600 tx=0x7f2f90011900 comp rx=0 tx=0).stop 2024-09-19T09:21:34.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.873+0000 7f2fa2967740 1 -- 172.21.15.120:0/881034082 shutdown_connections 2024-09-19T09:21:34.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.873+0000 7f2fa2967740 1 --2- 172.21.15.120:0/881034082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55837d831d40 0x55837d849d20 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.873+0000 7f2fa2967740 1 --2- 172.21.15.120:0/881034082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837d82f8e0 0x55837d83db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.873+0000 7f2fa2967740 1 --2- 172.21.15.120:0/881034082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55837d827480 0x55837d829610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.873+0000 7f2fa2967740 1 -- 172.21.15.120:0/881034082 >> 172.21.15.120:0/881034082 conn(0x55837d8230f0 msgr2=0x55837d82d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:34.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.873+0000 7f2fa2967740 1 -- 172.21.15.120:0/881034082 shutdown_connections 2024-09-19T09:21:34.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.873+0000 7f2fa2967740 1 -- 172.21.15.120:0/881034082 wait complete. 2024-09-19T09:21:34.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.873+0000 7f2fa2967740 1 -- start start 2024-09-19T09:21:34.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.873+0000 7f2fa2967740 1 Processor -- start 2024-09-19T09:21:34.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.873+0000 7f2fa2967740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55837d827480 0x55837d83a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:34.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.873+0000 7f2fa2967740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837d82f8e0 0x55837d835420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:34.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.873+0000 7f2fa2967740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55837d831d40 0x55837d835960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:34.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.873+0000 7f2fa2967740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2f981b50f0 con 0x55837d827480 2024-09-19T09:21:34.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.873+0000 7f2fa2967740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2f981b4a20 con 0x55837d831d40 2024-09-19T09:21:34.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.874+0000 7f2fa2967740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2f981b1c40 con 0x55837d82f8e0 2024-09-19T09:21:34.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.874+0000 7f2fa2967740 10 client.0 _ll_register_callbacks cb 0x7f2f90009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:34.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.874+0000 7f2f9e32a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55837d827480 0x55837d83a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:34.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.874+0000 7f2f9e32a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55837d827480 0x55837d83a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44340/0 (socket says 172.21.15.120:44340) 2024-09-19T09:21:34.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.874+0000 7f2f9e32a640 1 -- 172.21.15.120:0/3519317696 learned_addr learned my addr 172.21.15.120:0/3519317696 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:34.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.874+0000 7f2f9db29640 1 --2- 172.21.15.120:0/3519317696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837d82f8e0 0x55837d835420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:34.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.874+0000 7f2f9eb2b640 1 --2- 172.21.15.120:0/3519317696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55837d831d40 0x55837d835960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:34.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.874+0000 7f2f9e32a640 1 -- 172.21.15.120:0/3519317696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837d82f8e0 msgr2=0x55837d835420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:34.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.874+0000 7f2f9e32a640 1 --2- 172.21.15.120:0/3519317696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837d82f8e0 0x55837d835420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.874+0000 7f2f9e32a640 1 -- 172.21.15.120:0/3519317696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55837d831d40 msgr2=0x55837d835960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:34.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.874+0000 7f2f9e32a640 1 --2- 172.21.15.120:0/3519317696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55837d831d40 0x55837d835960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.874+0000 7f2f9e32a640 1 -- 172.21.15.120:0/3519317696 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55837d835ea0 con 0x55837d827480 2024-09-19T09:21:34.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.874+0000 7f2f9e32a640 1 --2- 172.21.15.120:0/3519317696 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55837d827480 0x55837d83a0f0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400c6a0 tx=0x7f2f940a50d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c06156a76b7dd718 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:34.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.874+0000 7f2f81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:21:34.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.874+0000 7f2f81ffb640 1 -- 172.21.15.120:0/3519317696 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f940bc2b0 con 0x55837d827480 2024-09-19T09:21:34.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.874+0000 7f2f81ffb640 1 -- 172.21.15.120:0/3519317696 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f2f9409b7e0 con 0x55837d827480 2024-09-19T09:21:34.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.874+0000 7f2fa2967740 1 -- 172.21.15.120:0/3519317696 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f900056b0 con 0x55837d827480 2024-09-19T09:21:34.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.874+0000 7f2f81ffb640 1 -- 172.21.15.120:0/3519317696 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2f94098a00 con 0x55837d827480 2024-09-19T09:21:34.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.874+0000 7f2fa2967740 10 client.21681 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:34.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.874+0000 7f2fa2967740 1 -- 172.21.15.120:0/3519317696 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55837d8aca00 con 0x55837d827480 2024-09-19T09:21:34.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.875+0000 7f2f81ffb640 1 -- 172.21.15.120:0/3519317696 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f900056b0 con 0x55837d827480 2024-09-19T09:21:34.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.875+0000 7f2f81ffb640 1 -- 172.21.15.120:0/3519317696 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f2f94018af0 con 0x55837d827480 2024-09-19T09:21:34.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.875+0000 7f2fa2967740 10 client.21681 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:34.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.875+0000 7f2fa2967740 10 client.21681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:21:34.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.875+0000 7f2fa2967740 20 client.21681 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:34.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.875+0000 7f2fa2967740 1 --2- 172.21.15.120:0/3519317696 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55837d8accf0 0x55837d8af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:34.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.875+0000 7f2fa2967740 4 client.21681 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:21:34.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.875+0000 7f2fa2967740 1 -- 172.21.15.120:0/3519317696 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55837d793d00 con 0x55837d8accf0 2024-09-19T09:21:34.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.875+0000 7f2f9db29640 1 --2- 172.21.15.120:0/3519317696 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55837d8accf0 0x55837d8af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:34.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.875+0000 7f2f81ffb640 10 client.21681 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:34.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.875+0000 7f2f9db29640 1 --2- 172.21.15.120:0/3519317696 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55837d8accf0 0x55837d8af0e0 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b3ee122ad6371b9 server_cookie=5d3768bccc9e63d9 in_seq=0 out_seq=0 2024-09-19T09:21:34.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.884+0000 7f2f81ffb640 1 -- 172.21.15.120:0/3519317696 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55837d793d00 con 0x55837d8accf0 2024-09-19T09:21:34.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.884+0000 7f2f81ffb640 10 client.21681 handle_command_reply: tid=0 2024-09-19T09:21:34.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.884+0000 7f2fa2967740 1 client.21681 shutdown 2024-09-19T09:21:34.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.884+0000 7f2fa2967740 1 -- 172.21.15.120:0/3519317696 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55837d827480 msgr2=0x55837d83a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:34.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.884+0000 7f2fa2967740 1 --2- 172.21.15.120:0/3519317696 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55837d827480 0x55837d83a0f0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400c6a0 tx=0x7f2f940a50d0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.884+0000 7f2fa2967740 1 -- 172.21.15.120:0/3519317696 shutdown_connections 2024-09-19T09:21:34.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.884+0000 7f2fa2967740 1 --2- 172.21.15.120:0/3519317696 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55837d8accf0 0x55837d8af0e0 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.884+0000 7f2fa2967740 1 --2- 172.21.15.120:0/3519317696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55837d831d40 0x55837d835960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.884+0000 7f2f81ffb640 0 client.21681 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:34.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.884+0000 7f2fa2967740 1 --2- 172.21.15.120:0/3519317696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837d82f8e0 0x55837d835420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.884+0000 7f2fa2967740 1 --2- 172.21.15.120:0/3519317696 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55837d827480 0x55837d83a0f0 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.884+0000 7f2fa2967740 1 -- 172.21.15.120:0/3519317696 >> 172.21.15.120:0/3519317696 conn(0x55837d8230f0 msgr2=0x55837d8306c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:34.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.884+0000 7f2fa2967740 1 -- 172.21.15.120:0/3519317696 shutdown_connections 2024-09-19T09:21:34.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.884+0000 7f2fa2967740 1 -- 172.21.15.120:0/3519317696 wait complete. 2024-09-19T09:21:34.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.884+0000 7f2fa2967740 20 client.21681 trim_cache size 0 max 16384 2024-09-19T09:21:34.900 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:21:34.900 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:21:34.900 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "2665d7f2-1d07-4be8-9920-0eabe0090614", 2024-09-19T09:21:34.900 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:21:34.900 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:21:34.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.885+0000 7f2fa058e640 1 -- 172.21.15.120:0/2364660310 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2f840a3240 msgr2=0x7f2f840a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:34.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.885+0000 7f2fa058e640 1 --2- 172.21.15.120:0/2364660310 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2f840a3240 0x7f2f840a5700 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2f88005940 tx=0x7f2f880058d0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.885+0000 7f2fa058e640 1 -- 172.21.15.120:0/2364660310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f98102d30 msgr2=0x7f2f98100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:34.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.885+0000 7f2fa058e640 1 --2- 172.21.15.120:0/2364660310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f98102d30 0x7f2f98100990 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400e9f0 tx=0x7f2f9400eeb0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.885+0000 7f2fa058e640 1 -- 172.21.15.120:0/2364660310 shutdown_connections 2024-09-19T09:21:34.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.885+0000 7f2fa058e640 1 --2- 172.21.15.120:0/2364660310 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f2f840a3240 0x7f2f840a5700 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.885+0000 7f2fa058e640 1 --2- 172.21.15.120:0/2364660310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f981082a0 0x7f2f98101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.885+0000 7f2fa058e640 1 --2- 172.21.15.120:0/2364660310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2f981036e0 0x7f2f98100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.885+0000 7f2fa058e640 1 --2- 172.21.15.120:0/2364660310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f98102d30 0x7f2f98100990 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:34.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.885+0000 7f2fa058e640 1 -- 172.21.15.120:0/2364660310 >> 172.21.15.120:0/2364660310 conn(0x7f2f980fc7d0 msgr2=0x7f2f98105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:34.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.885+0000 7f2fa058e640 1 -- 172.21.15.120:0/2364660310 shutdown_connections 2024-09-19T09:21:34.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:34.885+0000 7f2fa058e640 1 -- 172.21.15.120:0/2364660310 wait complete. 2024-09-19T09:21:35.370 INFO:tasks.workunit.client.0.smithi120.stdout: 10 138919 21.35 MB/sec execute 210 sec latency 63.133 ms 2024-09-19T09:21:35.412 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:35 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:21:35.412 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:35 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:21:35.412 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:35 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:21:35.412 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:35 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:21:35.412 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:35 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:21:35.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:35 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:21:35.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:35 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:21:35.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:35 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:21:35.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:35 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:21:35.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:35 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:21:35.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:35 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:21:35.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:35 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:21:35.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:35 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:21:35.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:35 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:21:35.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:35 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:21:35.685 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:21:35.985 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:21:36.370 INFO:tasks.workunit.client.0.smithi120.stdout: 10 139449 21.34 MB/sec execute 211 sec latency 104.551 ms 2024-09-19T09:21:36.401 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:36 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:21:36.401 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:36 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:21:36.401 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:36 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:21:36.401 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:36 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:21:36.402 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:36 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:21:36.402 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:36 smithi120 ceph-mon[31150]: pgmap v907: 97 pgs: 97 active+clean; 381 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 5.2 MiB/s wr, 602 op/s 2024-09-19T09:21:36.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:36 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:21:36.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:36 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:21:36.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:36 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:21:36.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:36 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:21:36.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:36 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:21:36.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:36 smithi196 ceph-mon[35150]: pgmap v907: 97 pgs: 97 active+clean; 381 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 5.2 MiB/s wr, 602 op/s 2024-09-19T09:21:36.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:36 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:21:36.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:36 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:21:36.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:36 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:21:36.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:36 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:21:36.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:36 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:21:36.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:36 smithi153 ceph-mon[35071]: pgmap v907: 97 pgs: 97 active+clean; 381 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 5.2 MiB/s wr, 602 op/s 2024-09-19T09:21:36.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.492+0000 7fc1de692640 1 -- 172.21.15.120:0/742109050 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1d8102780 msgr2=0x7fc1d81067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:36.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.492+0000 7fc1de692640 1 --2- 172.21.15.120:0/742109050 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1d8102780 0x7fc1d81067d0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc009900 tx=0x7fc1cc02a410 comp rx=0 tx=0).stop 2024-09-19T09:21:36.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.492+0000 7fc1de692640 1 -- 172.21.15.120:0/742109050 shutdown_connections 2024-09-19T09:21:36.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.492+0000 7fc1de692640 1 --2- 172.21.15.120:0/742109050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1d8106f00 0x7fc1d810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.492+0000 7fc1de692640 1 --2- 172.21.15.120:0/742109050 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1d8102780 0x7fc1d81067d0 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.492+0000 7fc1de692640 1 --2- 172.21.15.120:0/742109050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1d8101dd0 0x7fc1d81021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.492+0000 7fc1de692640 1 -- 172.21.15.120:0/742109050 >> 172.21.15.120:0/742109050 conn(0x7fc1d80fd8b0 msgr2=0x7fc1d80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:36.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.492+0000 7fc1de692640 1 -- 172.21.15.120:0/742109050 shutdown_connections 2024-09-19T09:21:36.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.492+0000 7fc1de692640 1 -- 172.21.15.120:0/742109050 wait complete. 2024-09-19T09:21:36.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.492+0000 7fc1de692640 1 Processor -- start 2024-09-19T09:21:36.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.492+0000 7fc1de692640 1 -- start start 2024-09-19T09:21:36.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.493+0000 7fc1de692640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1d8101dd0 0x7fc1d819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:36.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.493+0000 7fc1de692640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1d8102780 0x7fc1d819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:36.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.493+0000 7fc1de692640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1d8106f00 0x7fc1d819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:36.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.493+0000 7fc1de692640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d8114150 con 0x7fc1d8101dd0 2024-09-19T09:21:36.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.493+0000 7fc1de692640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d8113fd0 con 0x7fc1d8106f00 2024-09-19T09:21:36.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.493+0000 7fc1de692640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d81142d0 con 0x7fc1d8102780 2024-09-19T09:21:36.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.493+0000 7fc1d7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1d8101dd0 0x7fc1d819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:36.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.493+0000 7fc1d7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1d8101dd0 0x7fc1d819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44352/0 (socket says 172.21.15.120:44352) 2024-09-19T09:21:36.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.493+0000 7fc1d7fff640 1 -- 172.21.15.120:0/2209013624 learned_addr learned my addr 172.21.15.120:0/2209013624 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:36.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.493+0000 7fc1d77fe640 1 --2- 172.21.15.120:0/2209013624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1d8102780 0x7fc1d819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:36.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.493+0000 7fc1dcc2f640 1 --2- 172.21.15.120:0/2209013624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1d8106f00 0x7fc1d819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:36.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.494+0000 7fc1d7fff640 1 -- 172.21.15.120:0/2209013624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1d8102780 msgr2=0x7fc1d819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:36.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.494+0000 7fc1d7fff640 1 --2- 172.21.15.120:0/2209013624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1d8102780 0x7fc1d819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.494+0000 7fc1d7fff640 1 -- 172.21.15.120:0/2209013624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1d8106f00 msgr2=0x7fc1d819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:36.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.494+0000 7fc1d7fff640 1 --2- 172.21.15.120:0/2209013624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1d8106f00 0x7fc1d819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.494+0000 7fc1d7fff640 1 -- 172.21.15.120:0/2209013624 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d819f5f0 con 0x7fc1d8101dd0 2024-09-19T09:21:36.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.494+0000 7fc1dcc2f640 1 --2- 172.21.15.120:0/2209013624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1d8106f00 0x7fc1d819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:21:36.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.494+0000 7fc1d77fe640 1 --2- 172.21.15.120:0/2209013624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1d8102780 0x7fc1d819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:21:36.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.494+0000 7fc1d7fff640 1 --2- 172.21.15.120:0/2209013624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1d8101dd0 0x7fc1d819a640 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fc1c800e9f0 tx=0x7fc1c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:36.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.494+0000 7fc1d57fa640 1 -- 172.21.15.120:0/2209013624 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c800cd80 con 0x7fc1d8101dd0 2024-09-19T09:21:36.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.494+0000 7fc1d57fa640 1 -- 172.21.15.120:0/2209013624 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1c8010910 con 0x7fc1d8101dd0 2024-09-19T09:21:36.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.494+0000 7fc1d57fa640 1 -- 172.21.15.120:0/2209013624 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8018990 con 0x7fc1d8101dd0 2024-09-19T09:21:36.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.494+0000 7fc1de692640 1 -- 172.21.15.120:0/2209013624 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d819f8e0 con 0x7fc1d8101dd0 2024-09-19T09:21:36.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.494+0000 7fc1de692640 1 -- 172.21.15.120:0/2209013624 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d81a71c0 con 0x7fc1d8101dd0 2024-09-19T09:21:36.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.497+0000 7fc1e0a6b740 1 Processor -- start 2024-09-19T09:21:36.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.497+0000 7fc1e0a6b740 1 -- start start 2024-09-19T09:21:36.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.497+0000 7fc1e0a6b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8e3cb74a0 0x55a8e3cb7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:36.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.497+0000 7fc1d57fa640 1 -- 172.21.15.120:0/2209013624 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fc1c8018b30 con 0x7fc1d8101dd0 2024-09-19T09:21:36.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.497+0000 7fc1e0a6b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8e3cb7dc0 0x55a8e3cc2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:36.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.497+0000 7fc1e0a6b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e3cc2680 0x55a8e3cc4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:36.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.497+0000 7fc1e0a6b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a8e3c1d2e0 con 0x55a8e3cc2680 2024-09-19T09:21:36.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.497+0000 7fc1e0a6b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a8e3bc4ce0 con 0x55a8e3cb74a0 2024-09-19T09:21:36.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.497+0000 7fc1e0a6b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a8e3c192a0 con 0x55a8e3cb7dc0 2024-09-19T09:21:36.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.497+0000 7fc1dcc2f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8e3cb74a0 0x55a8e3cb7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:36.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.497+0000 7fc1d7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8e3cb7dc0 0x55a8e3cc2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:36.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.497+0000 7fc1dcc2f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8e3cb74a0 0x55a8e3cb7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:53708/0 (socket says 172.21.15.120:53708) 2024-09-19T09:21:36.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.497+0000 7fc1dcc2f640 1 -- 172.21.15.120:0/1773937485 learned_addr learned my addr 172.21.15.120:0/1773937485 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:36.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.497+0000 7fc1d77fe640 1 --2- 172.21.15.120:0/1773937485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e3cc2680 0x55a8e3cc4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:36.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.498+0000 7fc1d7fff640 1 -- 172.21.15.120:0/1773937485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8e3cb74a0 msgr2=0x55a8e3cb7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:36.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.498+0000 7fc1d7fff640 1 --2- 172.21.15.120:0/1773937485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8e3cb74a0 0x55a8e3cb7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.498+0000 7fc1d7fff640 1 -- 172.21.15.120:0/1773937485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e3cc2680 msgr2=0x55a8e3cc4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:36.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.498+0000 7fc1d7fff640 1 --2- 172.21.15.120:0/1773937485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e3cc2680 0x55a8e3cc4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.498+0000 7fc1d7fff640 1 -- 172.21.15.120:0/1773937485 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8e3ba9440 con 0x55a8e3cb7dc0 2024-09-19T09:21:36.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.498+0000 7fc1d77fe640 1 --2- 172.21.15.120:0/1773937485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e3cc2680 0x55a8e3cc4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:21:36.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.498+0000 7fc1d7fff640 1 --2- 172.21.15.120:0/1773937485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8e3cb7dc0 0x55a8e3cc2140 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc1c80cd7b0 tx=0x7fc1c80d87a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8171da042d7b617a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:36.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.498+0000 7fc1d57fa640 1 --2- 172.21.15.120:0/2209013624 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc1c00a3240 0x7fc1c00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:36.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.498+0000 7fc1dcc2f640 1 --2- 172.21.15.120:0/2209013624 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc1c00a3240 0x7fc1c00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:36.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.498+0000 7fc1be7fc640 1 -- 172.21.15.120:0/1773937485 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c80985d0 con 0x55a8e3cb7dc0 2024-09-19T09:21:36.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.498+0000 7fc1d57fa640 1 -- 172.21.15.120:0/2209013624 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc1c80cfd10 con 0x7fc1d8101dd0 2024-09-19T09:21:36.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.498+0000 7fc1be7fc640 1 -- 172.21.15.120:0/1773937485 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1c80dd070 con 0x55a8e3cb7dc0 2024-09-19T09:21:36.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.498+0000 7fc1be7fc640 1 -- 172.21.15.120:0/1773937485 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c80a4d50 con 0x55a8e3cb7dc0 2024-09-19T09:21:36.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.499+0000 7fc1e0a6b740 1 -- 172.21.15.120:0/1773937485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8e3cb7dc0 msgr2=0x55a8e3cc2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:36.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.499+0000 7fc1dcc2f640 1 --2- 172.21.15.120:0/2209013624 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc1c00a3240 0x7fc1c00a5700 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0009fd0 tx=0x7fc1d0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:36.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.499+0000 7fc1e0a6b740 1 --2- 172.21.15.120:0/1773937485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8e3cb7dc0 0x55a8e3cc2140 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc1c80cd7b0 tx=0x7fc1c80d87a0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.499+0000 7fc1e0a6b740 1 -- 172.21.15.120:0/1773937485 shutdown_connections 2024-09-19T09:21:36.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.499+0000 7fc1e0a6b740 1 --2- 172.21.15.120:0/1773937485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e3cc2680 0x55a8e3cc4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.499+0000 7fc1e0a6b740 1 --2- 172.21.15.120:0/1773937485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8e3cb7dc0 0x55a8e3cc2140 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.499+0000 7fc1e0a6b740 1 --2- 172.21.15.120:0/1773937485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8e3cb74a0 0x55a8e3cb7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.500+0000 7fc1e0a6b740 1 -- 172.21.15.120:0/1773937485 >> 172.21.15.120:0/1773937485 conn(0x55a8e3cb3a30 msgr2=0x55a8e3cb3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:36.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.500+0000 7fc1e0a6b740 1 -- 172.21.15.120:0/1773937485 shutdown_connections 2024-09-19T09:21:36.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.500+0000 7fc1e0a6b740 1 -- 172.21.15.120:0/1773937485 wait complete. 2024-09-19T09:21:36.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.500+0000 7fc1e0a6b740 1 -- start start 2024-09-19T09:21:36.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.500+0000 7fc1e0a6b740 1 Processor -- start 2024-09-19T09:21:36.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.501+0000 7fc1e0a6b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8e3cb74a0 0x55a8e3cc1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:36.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.501+0000 7fc1e0a6b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e3cb7dc0 0x55a8e3cbf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:36.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.501+0000 7fc1e0a6b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8e3cc2680 0x55a8e3cbf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:36.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.501+0000 7fc1e0a6b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a8e3cc5250 con 0x55a8e3cb7dc0 2024-09-19T09:21:36.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.501+0000 7fc1e0a6b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a8e3bd8050 con 0x55a8e3cb74a0 2024-09-19T09:21:36.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.501+0000 7fc1e0a6b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a8e3cc53d0 con 0x55a8e3cc2680 2024-09-19T09:21:36.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.501+0000 7fc1e0a6b740 10 client.0 _ll_register_callbacks cb 0x55a8e3c18160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:36.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.501+0000 7fc1d7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8e3cb74a0 0x55a8e3cc1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:36.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.501+0000 7fc1d7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8e3cb74a0 0x55a8e3cc1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:53712/0 (socket says 172.21.15.120:53712) 2024-09-19T09:21:36.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.501+0000 7fc1d7fff640 1 -- 172.21.15.120:0/2998018777 learned_addr learned my addr 172.21.15.120:0/2998018777 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:36.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.501+0000 7fc1dcc2f640 1 --2- 172.21.15.120:0/2998018777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8e3cc2680 0x55a8e3cbf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:36.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.501+0000 7fc1d77fe640 1 --2- 172.21.15.120:0/2998018777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e3cb7dc0 0x55a8e3cbf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:36.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.501+0000 7fc1d7fff640 1 -- 172.21.15.120:0/2998018777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8e3cc2680 msgr2=0x55a8e3cbf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:36.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.501+0000 7fc1d7fff640 1 --2- 172.21.15.120:0/2998018777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8e3cc2680 0x55a8e3cbf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.501+0000 7fc1d7fff640 1 -- 172.21.15.120:0/2998018777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e3cb7dc0 msgr2=0x55a8e3cbf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:36.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.501+0000 7fc1d7fff640 1 --2- 172.21.15.120:0/2998018777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e3cb7dc0 0x55a8e3cbf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.501+0000 7fc1d7fff640 1 -- 172.21.15.120:0/2998018777 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8e398e3c0 con 0x55a8e3cb74a0 2024-09-19T09:21:36.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.501+0000 7fc1d7fff640 1 --2- 172.21.15.120:0/2998018777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8e3cb74a0 0x55a8e3cc1fc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc1c80a4a70 tx=0x7fc1c80cff90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f218f1f2bf2ec61 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:36.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.501+0000 7fc1b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:21:36.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.501+0000 7fc1b77fe640 1 -- 172.21.15.120:0/2998018777 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c80a4fa0 con 0x55a8e3cb74a0 2024-09-19T09:21:36.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.502+0000 7fc1b77fe640 1 -- 172.21.15.120:0/2998018777 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc1c80d3990 con 0x55a8e3cb74a0 2024-09-19T09:21:36.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.502+0000 7fc1e0a6b740 1 -- 172.21.15.120:0/2998018777 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1c80dd070 con 0x55a8e3cb74a0 2024-09-19T09:21:36.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.502+0000 7fc1b77fe640 1 -- 172.21.15.120:0/2998018777 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc1c8099980 con 0x55a8e3cb74a0 2024-09-19T09:21:36.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.502+0000 7fc1e0a6b740 10 client.30521 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:36.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.502+0000 7fc1e0a6b740 1 -- 172.21.15.120:0/2998018777 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a8e3cd0c00 con 0x55a8e3cb74a0 2024-09-19T09:21:36.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.502+0000 7fc1b77fe640 1 -- 172.21.15.120:0/2998018777 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1c80dd070 con 0x55a8e3cb74a0 2024-09-19T09:21:36.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.502+0000 7fc1b77fe640 1 -- 172.21.15.120:0/2998018777 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fc1c80cd890 con 0x55a8e3cb74a0 2024-09-19T09:21:36.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.503+0000 7fc1e0a6b740 10 client.30521 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:36.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.503+0000 7fc1e0a6b740 10 client.30521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:21:36.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.503+0000 7fc1e0a6b740 20 client.30521 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:36.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.503+0000 7fc1e0a6b740 1 --2- 172.21.15.120:0/2998018777 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a8e3cd1d80 0x55a8e3cd4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:36.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.503+0000 7fc1e0a6b740 4 client.30521 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:21:36.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.503+0000 7fc1e0a6b740 1 -- 172.21.15.120:0/2998018777 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8e3c2a850 con 0x55a8e3cd1d80 2024-09-19T09:21:36.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.503+0000 7fc1d77fe640 1 --2- 172.21.15.120:0/2998018777 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a8e3cd1d80 0x55a8e3cd4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:36.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.503+0000 7fc1d77fe640 1 --2- 172.21.15.120:0/2998018777 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a8e3cd1d80 0x55a8e3cd4170 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9995f798230f04ab server_cookie=85506f8ab25671d3 in_seq=0 out_seq=0 2024-09-19T09:21:36.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.503+0000 7fc1b77fe640 10 client.30521 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:36.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.508+0000 7fc1b77fe640 1 -- 172.21.15.120:0/2998018777 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a8e3c2a850 con 0x55a8e3cd1d80 2024-09-19T09:21:36.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.508+0000 7fc1b77fe640 10 client.30521 handle_command_reply: tid=0 2024-09-19T09:21:36.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.508+0000 7fc1e0a6b740 1 client.30521 shutdown 2024-09-19T09:21:36.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.508+0000 7fc1e0a6b740 1 -- 172.21.15.120:0/2998018777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8e3cb74a0 msgr2=0x55a8e3cc1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:36.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.508+0000 7fc1e0a6b740 1 --2- 172.21.15.120:0/2998018777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8e3cb74a0 0x55a8e3cc1fc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc1c80a4a70 tx=0x7fc1c80cff90 comp rx=0 tx=0).stop 2024-09-19T09:21:36.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.508+0000 7fc1e0a6b740 1 -- 172.21.15.120:0/2998018777 shutdown_connections 2024-09-19T09:21:36.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.508+0000 7fc1e0a6b740 1 --2- 172.21.15.120:0/2998018777 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a8e3cd1d80 0x55a8e3cd4170 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.508+0000 7fc1e0a6b740 1 --2- 172.21.15.120:0/2998018777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8e3cc2680 0x55a8e3cbf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.508+0000 7fc1e0a6b740 1 --2- 172.21.15.120:0/2998018777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e3cb7dc0 0x55a8e3cbf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.508+0000 7fc1b77fe640 0 client.30521 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:36.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.508+0000 7fc1e0a6b740 1 --2- 172.21.15.120:0/2998018777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8e3cb74a0 0x55a8e3cc1fc0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.508+0000 7fc1e0a6b740 1 -- 172.21.15.120:0/2998018777 >> 172.21.15.120:0/2998018777 conn(0x55a8e3cb3a30 msgr2=0x55a8e3cb9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:36.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.508+0000 7fc1e0a6b740 1 -- 172.21.15.120:0/2998018777 shutdown_connections 2024-09-19T09:21:36.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.508+0000 7fc1e0a6b740 1 -- 172.21.15.120:0/2998018777 wait complete. 2024-09-19T09:21:36.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.509+0000 7fc1e0a6b740 20 client.30521 trim_cache size 0 max 16384 2024-09-19T09:21:36.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.528+0000 7fc1e0a6b740 1 Processor -- start 2024-09-19T09:21:36.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.528+0000 7fc1e0a6b740 1 -- start start 2024-09-19T09:21:36.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.529+0000 7fc1e0a6b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e3cb74a0 0x55a8e3cbc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:36.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.529+0000 7fc1e0a6b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8e3cb7dc0 0x55a8e3cbca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:36.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.529+0000 7fc1e0a6b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8e3cc2680 0x55a8e3cba030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:36.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.529+0000 7fc1e0a6b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d81af8c0 con 0x55a8e3cb74a0 2024-09-19T09:21:36.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.529+0000 7fc1e0a6b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d81b1400 con 0x55a8e3cb7dc0 2024-09-19T09:21:36.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.529+0000 7fc1e0a6b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d81aff90 con 0x55a8e3cc2680 2024-09-19T09:21:36.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.529+0000 7fc1d77fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8e3cb7dc0 0x55a8e3cbca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:36.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.529+0000 7fc1d77fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8e3cb7dc0 0x55a8e3cbca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:53722/0 (socket says 172.21.15.120:53722) 2024-09-19T09:21:36.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.529+0000 7fc1dcc2f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8e3cc2680 0x55a8e3cba030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:36.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.529+0000 7fc1d77fe640 1 -- 172.21.15.120:0/1878147549 learned_addr learned my addr 172.21.15.120:0/1878147549 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:36.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.529+0000 7fc1d7fff640 1 --2- 172.21.15.120:0/1878147549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e3cb74a0 0x55a8e3cbc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:36.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.529+0000 7fc1d77fe640 1 -- 172.21.15.120:0/1878147549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8e3cc2680 msgr2=0x55a8e3cba030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:36.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.529+0000 7fc1d77fe640 1 --2- 172.21.15.120:0/1878147549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8e3cc2680 0x55a8e3cba030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.529+0000 7fc1d77fe640 1 -- 172.21.15.120:0/1878147549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e3cb74a0 msgr2=0x55a8e3cbc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:36.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.529+0000 7fc1d77fe640 1 --2- 172.21.15.120:0/1878147549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e3cb74a0 0x55a8e3cbc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.530+0000 7fc1d77fe640 1 -- 172.21.15.120:0/1878147549 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8e3cc4dd0 con 0x55a8e3cb7dc0 2024-09-19T09:21:36.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.530+0000 7fc1d77fe640 1 --2- 172.21.15.120:0/1878147549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8e3cb7dc0 0x55a8e3cbca10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc009900 tx=0x7fc1cc031510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=714d5191d11906ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:36.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.530+0000 7fc1bdffb640 1 -- 172.21.15.120:0/1878147549 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc04a020 con 0x55a8e3cb7dc0 2024-09-19T09:21:36.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.530+0000 7fc1bdffb640 1 -- 172.21.15.120:0/1878147549 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1cc0055b0 con 0x55a8e3cb7dc0 2024-09-19T09:21:36.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.530+0000 7fc1bdffb640 1 -- 172.21.15.120:0/1878147549 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc035040 con 0x55a8e3cb7dc0 2024-09-19T09:21:36.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.531+0000 7fc1e0a6b740 1 -- 172.21.15.120:0/1878147549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8e3cb7dc0 msgr2=0x55a8e3cbca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:36.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.531+0000 7fc1e0a6b740 1 --2- 172.21.15.120:0/1878147549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8e3cb7dc0 0x55a8e3cbca10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc009900 tx=0x7fc1cc031510 comp rx=0 tx=0).stop 2024-09-19T09:21:36.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.531+0000 7fc1e0a6b740 1 -- 172.21.15.120:0/1878147549 shutdown_connections 2024-09-19T09:21:36.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.531+0000 7fc1e0a6b740 1 --2- 172.21.15.120:0/1878147549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8e3cc2680 0x55a8e3cba030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.531+0000 7fc1e0a6b740 1 --2- 172.21.15.120:0/1878147549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8e3cb7dc0 0x55a8e3cbca10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.531+0000 7fc1e0a6b740 1 --2- 172.21.15.120:0/1878147549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e3cb74a0 0x55a8e3cbc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.531+0000 7fc1e0a6b740 1 -- 172.21.15.120:0/1878147549 >> 172.21.15.120:0/1878147549 conn(0x55a8e3cb3a30 msgr2=0x55a8e3cbe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:36.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.531+0000 7fc1e0a6b740 1 -- 172.21.15.120:0/1878147549 shutdown_connections 2024-09-19T09:21:36.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.531+0000 7fc1e0a6b740 1 -- 172.21.15.120:0/1878147549 wait complete. 2024-09-19T09:21:36.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.532+0000 7fc1e0a6b740 1 -- start start 2024-09-19T09:21:36.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.532+0000 7fc1e0a6b740 1 Processor -- start 2024-09-19T09:21:36.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.532+0000 7fc1e0a6b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8e3cb74a0 0x55a8e3cd72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:36.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.532+0000 7fc1e0a6b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e3cb7dc0 0x55a8e3cd7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:36.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.532+0000 7fc1e0a6b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8e3cc2680 0x55a8e3cd4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:36.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.532+0000 7fc1e0a6b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d81b0d30 con 0x55a8e3cb7dc0 2024-09-19T09:21:36.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.532+0000 7fc1e0a6b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d81b0660 con 0x55a8e3cb74a0 2024-09-19T09:21:36.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.532+0000 7fc1e0a6b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d81ad880 con 0x55a8e3cc2680 2024-09-19T09:21:36.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.532+0000 7fc1e0a6b740 10 client.0 _ll_register_callbacks cb 0x7fc1d0009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:36.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.532+0000 7fc1d77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e3cb7dc0 0x55a8e3cd7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:36.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.532+0000 7fc1d77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e3cb7dc0 0x55a8e3cd7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44372/0 (socket says 172.21.15.120:44372) 2024-09-19T09:21:36.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.532+0000 7fc1d77fe640 1 -- 172.21.15.120:0/2965130127 learned_addr learned my addr 172.21.15.120:0/2965130127 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:36.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.532+0000 7fc1dcc2f640 1 --2- 172.21.15.120:0/2965130127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8e3cc2680 0x55a8e3cd4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:36.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.533+0000 7fc1d7fff640 1 --2- 172.21.15.120:0/2965130127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8e3cb74a0 0x55a8e3cd72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:36.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.533+0000 7fc1d77fe640 1 -- 172.21.15.120:0/2965130127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8e3cc2680 msgr2=0x55a8e3cd4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:36.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.533+0000 7fc1d77fe640 1 --2- 172.21.15.120:0/2965130127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8e3cc2680 0x55a8e3cd4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.533+0000 7fc1d77fe640 1 -- 172.21.15.120:0/2965130127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8e3cb74a0 msgr2=0x55a8e3cd72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:36.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.533+0000 7fc1d77fe640 1 --2- 172.21.15.120:0/2965130127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8e3cb74a0 0x55a8e3cd72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.533+0000 7fc1d77fe640 1 -- 172.21.15.120:0/2965130127 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8e3cd4f50 con 0x55a8e3cb7dc0 2024-09-19T09:21:36.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.533+0000 7fc1dcc2f640 1 --2- 172.21.15.120:0/2965130127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8e3cc2680 0x55a8e3cd4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:21:36.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.533+0000 7fc1d77fe640 1 --2- 172.21.15.120:0/2965130127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e3cb7dc0 0x55a8e3cd7810 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc006010 tx=0x7fc1cc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592baadb1f10605 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:36.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.533+0000 7fc1b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:21:36.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.533+0000 7fc1b77fe640 1 -- 172.21.15.120:0/2965130127 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc035040 con 0x55a8e3cb7dc0 2024-09-19T09:21:36.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.533+0000 7fc1b77fe640 1 -- 172.21.15.120:0/2965130127 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc1cc031cb0 con 0x55a8e3cb7dc0 2024-09-19T09:21:36.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.533+0000 7fc1e0a6b740 1 -- 172.21.15.120:0/2965130127 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1cc0055b0 con 0x55a8e3cb7dc0 2024-09-19T09:21:36.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.533+0000 7fc1b77fe640 1 -- 172.21.15.120:0/2965130127 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc1cc005850 con 0x55a8e3cb7dc0 2024-09-19T09:21:36.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.533+0000 7fc1e0a6b740 10 client.21699 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:36.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.534+0000 7fc1e0a6b740 1 -- 172.21.15.120:0/2965130127 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a8e3d3c250 con 0x55a8e3cb7dc0 2024-09-19T09:21:36.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.534+0000 7fc1b77fe640 1 -- 172.21.15.120:0/2965130127 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1cc0055b0 con 0x55a8e3cb7dc0 2024-09-19T09:21:36.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.534+0000 7fc1b77fe640 1 -- 172.21.15.120:0/2965130127 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fc1cc003710 con 0x55a8e3cb7dc0 2024-09-19T09:21:36.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.534+0000 7fc1e0a6b740 10 client.21699 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.534+0000 7fc1e0a6b740 10 client.21699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:21:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.534+0000 7fc1e0a6b740 20 client.21699 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.534+0000 7fc1e0a6b740 1 --2- 172.21.15.120:0/2965130127 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a8e3d3c8f0 0x55a8e3d3ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.534+0000 7fc1e0a6b740 4 client.21699 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:21:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.534+0000 7fc1e0a6b740 1 -- 172.21.15.120:0/2965130127 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a8e3c24d60 con 0x55a8e3d3c8f0 2024-09-19T09:21:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.534+0000 7fc1d7fff640 1 --2- 172.21.15.120:0/2965130127 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a8e3d3c8f0 0x55a8e3d3ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.535+0000 7fc1b77fe640 10 client.21699 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.535+0000 7fc1d7fff640 1 --2- 172.21.15.120:0/2965130127 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a8e3d3c8f0 0x55a8e3d3ed10 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77c57e34aead5d6c server_cookie=ef5f98079c719057 in_seq=0 out_seq=0 2024-09-19T09:21:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.536+0000 7fc1b77fe640 1 -- 172.21.15.120:0/2965130127 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a8e3c24d60 con 0x55a8e3d3c8f0 2024-09-19T09:21:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.536+0000 7fc1b77fe640 10 client.21699 handle_command_reply: tid=0 2024-09-19T09:21:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.536+0000 7fc1e0a6b740 1 client.21699 shutdown 2024-09-19T09:21:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.537+0000 7fc1e0a6b740 1 -- 172.21.15.120:0/2965130127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e3cb7dc0 msgr2=0x55a8e3cd7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.537+0000 7fc1e0a6b740 1 --2- 172.21.15.120:0/2965130127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e3cb7dc0 0x55a8e3cd7810 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc006010 tx=0x7fc1cc02a410 comp rx=0 tx=0).stop 2024-09-19T09:21:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.537+0000 7fc1e0a6b740 1 -- 172.21.15.120:0/2965130127 shutdown_connections 2024-09-19T09:21:36.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.537+0000 7fc1e0a6b740 1 --2- 172.21.15.120:0/2965130127 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a8e3d3c8f0 0x55a8e3d3ed10 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.537+0000 7fc1e0a6b740 1 --2- 172.21.15.120:0/2965130127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a8e3cc2680 0x55a8e3cd4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.537+0000 7fc1e0a6b740 1 --2- 172.21.15.120:0/2965130127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8e3cb7dc0 0x55a8e3cd7810 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.537+0000 7fc1b77fe640 0 client.21699 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:36.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.537+0000 7fc1e0a6b740 1 --2- 172.21.15.120:0/2965130127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8e3cb74a0 0x55a8e3cd72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.537+0000 7fc1e0a6b740 1 -- 172.21.15.120:0/2965130127 >> 172.21.15.120:0/2965130127 conn(0x55a8e3cb3a30 msgr2=0x55a8e3cb9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:36.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.537+0000 7fc1e0a6b740 1 -- 172.21.15.120:0/2965130127 shutdown_connections 2024-09-19T09:21:36.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.537+0000 7fc1e0a6b740 1 -- 172.21.15.120:0/2965130127 wait complete. 2024-09-19T09:21:36.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.537+0000 7fc1e0a6b740 20 client.21699 trim_cache size 0 max 16384 2024-09-19T09:21:36.540 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:21:36.541 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:21:36.541 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-19T09:21:36.541 INFO:teuthology.orchestra.run.smithi120.stdout: "2665d7f2-1d07-4be8-9920-0eabe0090614": { 2024-09-19T09:21:36.541 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-19T09:21:36.541 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "2665d7f2-1d07-4be8-9920-0eabe0090614", 2024-09-19T09:21:36.541 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-19T09:21:36.541 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:21:36.541 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:21:36.541 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:21:36.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.538+0000 7fc1de692640 1 -- 172.21.15.120:0/2209013624 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc1c00a3240 msgr2=0x7fc1c00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:36.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.538+0000 7fc1de692640 1 --2- 172.21.15.120:0/2209013624 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc1c00a3240 0x7fc1c00a5700 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0009fd0 tx=0x7fc1d0009500 comp rx=0 tx=0).stop 2024-09-19T09:21:36.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.538+0000 7fc1de692640 1 -- 172.21.15.120:0/2209013624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1d8101dd0 msgr2=0x7fc1d819a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:36.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.538+0000 7fc1de692640 1 --2- 172.21.15.120:0/2209013624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1d8101dd0 0x7fc1d819a640 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fc1c800e9f0 tx=0x7fc1c800eeb0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.538+0000 7fc1de692640 1 -- 172.21.15.120:0/2209013624 shutdown_connections 2024-09-19T09:21:36.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.538+0000 7fc1de692640 1 --2- 172.21.15.120:0/2209013624 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc1c00a3240 0x7fc1c00a5700 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.538+0000 7fc1de692640 1 --2- 172.21.15.120:0/2209013624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1d8106f00 0x7fc1d819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.538+0000 7fc1de692640 1 --2- 172.21.15.120:0/2209013624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1d8102780 0x7fc1d819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.538+0000 7fc1de692640 1 --2- 172.21.15.120:0/2209013624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1d8101dd0 0x7fc1d819a640 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:36.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.538+0000 7fc1de692640 1 -- 172.21.15.120:0/2209013624 >> 172.21.15.120:0/2209013624 conn(0x7fc1d80fd8b0 msgr2=0x7fc1d80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:36.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.538+0000 7fc1de692640 1 -- 172.21.15.120:0/2209013624 shutdown_connections 2024-09-19T09:21:36.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:36.538+0000 7fc1de692640 1 -- 172.21.15.120:0/2209013624 wait complete. 2024-09-19T09:21:37.371 INFO:tasks.workunit.client.0.smithi120.stdout: 10 140143 21.42 MB/sec execute 212 sec latency 44.207 ms 2024-09-19T09:21:38.094 INFO:tasks.cephfs.filesystem:scrub status for tag:2665d7f2-1d07-4be8-9920-0eabe0090614 - {'path': '/', 'tag': '2665d7f2-1d07-4be8-9920-0eabe0090614', 'options': 'recursive,force'} 2024-09-19T09:21:38.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:38 smithi120 ceph-mon[31150]: pgmap v908: 97 pgs: 97 active+clean; 387 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.1 MiB/s wr, 683 op/s 2024-09-19T09:21:38.371 INFO:tasks.workunit.client.0.smithi120.stdout: 10 140859 21.47 MB/sec execute 213 sec latency 59.228 ms 2024-09-19T09:21:38.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:38 smithi196 ceph-mon[35150]: pgmap v908: 97 pgs: 97 active+clean; 387 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.1 MiB/s wr, 683 op/s 2024-09-19T09:21:38.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:38 smithi153 ceph-mon[35071]: pgmap v908: 97 pgs: 97 active+clean; 387 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.1 MiB/s wr, 683 op/s 2024-09-19T09:21:39.370 INFO:tasks.workunit.client.0.smithi120.stdout: 10 141597 21.61 MB/sec execute 214 sec latency 39.968 ms 2024-09-19T09:21:40.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:40 smithi120 ceph-mon[31150]: pgmap v909: 97 pgs: 97 active+clean; 393 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.6 MiB/s wr, 681 op/s 2024-09-19T09:21:40.370 INFO:tasks.workunit.client.0.smithi120.stdout: 10 142308 21.70 MB/sec execute 215 sec latency 60.276 ms 2024-09-19T09:21:40.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:40 smithi196 ceph-mon[35150]: pgmap v909: 97 pgs: 97 active+clean; 393 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.6 MiB/s wr, 681 op/s 2024-09-19T09:21:40.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:40 smithi153 ceph-mon[35071]: pgmap v909: 97 pgs: 97 active+clean; 393 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.6 MiB/s wr, 681 op/s 2024-09-19T09:21:41.371 INFO:tasks.workunit.client.0.smithi120.stdout: 10 142808 21.71 MB/sec execute 216 sec latency 82.164 ms 2024-09-19T09:21:42.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:42 smithi120 ceph-mon[31150]: pgmap v910: 97 pgs: 97 active+clean; 403 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.5 MiB/s wr, 614 op/s 2024-09-19T09:21:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.328+0000 7f60d46f1640 1 -- 172.21.15.120:0/1223658070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60cc101c40 msgr2=0x7f60cc1020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.328+0000 7f60d46f1640 1 --2- 172.21.15.120:0/1223658070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60cc101c40 0x7f60cc1020c0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f60bc009900 tx=0x7f60bc02a410 comp rx=0 tx=0).stop 2024-09-19T09:21:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.328+0000 7f60d46f1640 1 -- 172.21.15.120:0/1223658070 shutdown_connections 2024-09-19T09:21:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.328+0000 7f60d46f1640 1 --2- 172.21.15.120:0/1223658070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60cc102600 0x7f60cc108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.328+0000 7f60d46f1640 1 --2- 172.21.15.120:0/1223658070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60cc101c40 0x7f60cc1020c0 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.328+0000 7f60d46f1640 1 --2- 172.21.15.120:0/1223658070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60cc100a50 0x7f60cc100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.328+0000 7f60d46f1640 1 -- 172.21.15.120:0/1223658070 >> 172.21.15.120:0/1223658070 conn(0x7f60cc0fc1b0 msgr2=0x7f60cc0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.328+0000 7f60d46f1640 1 -- 172.21.15.120:0/1223658070 shutdown_connections 2024-09-19T09:21:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.328+0000 7f60d46f1640 1 -- 172.21.15.120:0/1223658070 wait complete. 2024-09-19T09:21:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.329+0000 7f60d46f1640 1 Processor -- start 2024-09-19T09:21:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.329+0000 7f60d46f1640 1 -- start start 2024-09-19T09:21:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.329+0000 7f60d46f1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60cc100a50 0x7f60cc06aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.329+0000 7f60d46f1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60cc101c40 0x7f60cc19caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.329+0000 7f60d46f1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60cc102600 0x7f60cc19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.329+0000 7f60d46f1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc10c6d0 con 0x7f60cc100a50 2024-09-19T09:21:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.329+0000 7f60d46f1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc10c550 con 0x7f60cc102600 2024-09-19T09:21:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.329+0000 7f60d46f1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc10c850 con 0x7f60cc101c40 2024-09-19T09:21:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.329+0000 7f60d248d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60cc100a50 0x7f60cc06aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.329+0000 7f60d248d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60cc100a50 0x7f60cc06aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41674/0 (socket says 172.21.15.120:41674) 2024-09-19T09:21:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.329+0000 7f60d248d640 1 -- 172.21.15.120:0/1740455266 learned_addr learned my addr 172.21.15.120:0/1740455266 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.329+0000 7f60d1c8c640 1 --2- 172.21.15.120:0/1740455266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60cc101c40 0x7f60cc19caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.329+0000 7f60d2c8e640 1 --2- 172.21.15.120:0/1740455266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60cc102600 0x7f60cc19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.329+0000 7f60d248d640 1 -- 172.21.15.120:0/1740455266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60cc101c40 msgr2=0x7f60cc19caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.329+0000 7f60d248d640 1 --2- 172.21.15.120:0/1740455266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60cc101c40 0x7f60cc19caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.329+0000 7f60d248d640 1 -- 172.21.15.120:0/1740455266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60cc102600 msgr2=0x7f60cc19f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.329+0000 7f60d248d640 1 --2- 172.21.15.120:0/1740455266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60cc102600 0x7f60cc19f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.329+0000 7f60d248d640 1 -- 172.21.15.120:0/1740455266 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60cc19fa20 con 0x7f60cc100a50 2024-09-19T09:21:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.329+0000 7f60d248d640 1 --2- 172.21.15.120:0/1740455266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60cc100a50 0x7f60cc06aeb0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f60c800ebc0 tx=0x7f60c800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.330+0000 7f60c37fe640 1 -- 172.21.15.120:0/1740455266 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8003b70 con 0x7f60cc100a50 2024-09-19T09:21:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.330+0000 7f60c37fe640 1 -- 172.21.15.120:0/1740455266 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60c8003d10 con 0x7f60cc100a50 2024-09-19T09:21:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.330+0000 7f60d46f1640 1 -- 172.21.15.120:0/1740455266 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60cc19fcb0 con 0x7f60cc100a50 2024-09-19T09:21:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.330+0000 7f60c37fe640 1 -- 172.21.15.120:0/1740455266 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8019980 con 0x7f60cc100a50 2024-09-19T09:21:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.330+0000 7f60d46f1640 1 -- 172.21.15.120:0/1740455266 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60cc1a01d0 con 0x7f60cc100a50 2024-09-19T09:21:42.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.333+0000 7f60d748c740 1 Processor -- start 2024-09-19T09:21:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.333+0000 7f60d748c740 1 -- start start 2024-09-19T09:21:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.333+0000 7f60c37fe640 1 -- 172.21.15.120:0/1740455266 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f60c8019b20 con 0x7f60cc100a50 2024-09-19T09:21:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.333+0000 7f60d748c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556afa2f9920 0x556afa2f9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.333+0000 7f60d748c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556afa2fa260 0x556afa2fcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.333+0000 7f60d748c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556afa2fd330 0x556afa30a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.333+0000 7f60d748c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556afa25d2f0 con 0x556afa2f9920 2024-09-19T09:21:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.333+0000 7f60d748c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556afa205270 con 0x556afa2fa260 2024-09-19T09:21:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.334+0000 7f60d748c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556afa2592b0 con 0x556afa2fd330 2024-09-19T09:21:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.334+0000 7f60d2c8e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556afa2f9920 0x556afa2f9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.334+0000 7f60d2c8e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556afa2f9920 0x556afa2f9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41688/0 (socket says 172.21.15.120:41688) 2024-09-19T09:21:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.334+0000 7f60d2c8e640 1 -- 172.21.15.120:0/1585754454 learned_addr learned my addr 172.21.15.120:0/1585754454 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.334+0000 7f60d1c8c640 1 --2- 172.21.15.120:0/1585754454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556afa2fd330 0x556afa30a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.334+0000 7f60d2c8e640 1 -- 172.21.15.120:0/1585754454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556afa2fd330 msgr2=0x556afa30a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.334+0000 7f60d2c8e640 1 --2- 172.21.15.120:0/1585754454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556afa2fd330 0x556afa30a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.334+0000 7f60d2c8e640 1 -- 172.21.15.120:0/1585754454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556afa2fa260 msgr2=0x556afa2fcd10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:21:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.334+0000 7f60d2c8e640 1 --2- 172.21.15.120:0/1585754454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556afa2fa260 0x556afa2fcd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.334+0000 7f60d2c8e640 1 -- 172.21.15.120:0/1585754454 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556afa204340 con 0x556afa2f9920 2024-09-19T09:21:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.335+0000 7f60c37fe640 1 --2- 172.21.15.120:0/1740455266 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f60b80a3370 0x7f60b80a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.335+0000 7f60d2c8e640 1 --2- 172.21.15.120:0/1585754454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556afa2f9920 0x556afa2f9d20 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f60c4009cd0 tx=0x7f60c4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76cadef126bbc868 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.335+0000 7f60d248d640 1 --2- 172.21.15.120:0/1740455266 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f60b80a3370 0x7f60b80a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.335+0000 7f60c0ff9640 1 -- 172.21.15.120:0/1585754454 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c401e070 con 0x556afa2f9920 2024-09-19T09:21:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.335+0000 7f60c0ff9640 1 -- 172.21.15.120:0/1585754454 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60c4019b10 con 0x556afa2f9920 2024-09-19T09:21:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.335+0000 7f60c0ff9640 1 -- 172.21.15.120:0/1585754454 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c400eb50 con 0x556afa2f9920 2024-09-19T09:21:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.335+0000 7f60c37fe640 1 -- 172.21.15.120:0/1740455266 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f60c80cffc0 con 0x7f60cc100a50 2024-09-19T09:21:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.336+0000 7f60d748c740 1 -- 172.21.15.120:0/1585754454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556afa2f9920 msgr2=0x556afa2f9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.336+0000 7f60d248d640 1 --2- 172.21.15.120:0/1740455266 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f60b80a3370 0x7f60b80a5830 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f60c80049d0 tx=0x7f60c8021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.336+0000 7f60d748c740 1 --2- 172.21.15.120:0/1585754454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556afa2f9920 0x556afa2f9d20 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f60c4009cd0 tx=0x7f60c4011810 comp rx=0 tx=0).stop 2024-09-19T09:21:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.336+0000 7f60d748c740 1 -- 172.21.15.120:0/1585754454 shutdown_connections 2024-09-19T09:21:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.336+0000 7f60d748c740 1 --2- 172.21.15.120:0/1585754454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556afa2fd330 0x556afa30a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.336+0000 7f60d748c740 1 --2- 172.21.15.120:0/1585754454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556afa2fa260 0x556afa2fcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.336+0000 7f60d748c740 1 --2- 172.21.15.120:0/1585754454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556afa2f9920 0x556afa2f9d20 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.336+0000 7f60d748c740 1 -- 172.21.15.120:0/1585754454 >> 172.21.15.120:0/1585754454 conn(0x556afa2f5010 msgr2=0x556afa2f5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.336+0000 7f60d748c740 1 -- 172.21.15.120:0/1585754454 shutdown_connections 2024-09-19T09:21:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.336+0000 7f60d748c740 1 -- 172.21.15.120:0/1585754454 wait complete. 2024-09-19T09:21:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.336+0000 7f60d748c740 1 -- start start 2024-09-19T09:21:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.336+0000 7f60d748c740 1 Processor -- start 2024-09-19T09:21:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.337+0000 7f60d748c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556afa2f9920 0x556afa30c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.337+0000 7f60d748c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556afa2fa260 0x556afa30c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.337+0000 7f60d748c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556afa2fd330 0x556afa313970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.337+0000 7f60d748c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556afa30ab70 con 0x556afa2f9920 2024-09-19T09:21:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.337+0000 7f60d748c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556afa1e6290 con 0x556afa2fa260 2024-09-19T09:21:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.337+0000 7f60d748c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556afa30acf0 con 0x556afa2fd330 2024-09-19T09:21:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.337+0000 7f60d748c740 10 client.0 _ll_register_callbacks cb 0x556afa217140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.337+0000 7f60d2c8e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556afa2f9920 0x556afa30c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.337+0000 7f60d2c8e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556afa2f9920 0x556afa30c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41700/0 (socket says 172.21.15.120:41700) 2024-09-19T09:21:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.337+0000 7f60d2c8e640 1 -- 172.21.15.120:0/3442946858 learned_addr learned my addr 172.21.15.120:0/3442946858 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.337+0000 7f60d1c8c640 1 --2- 172.21.15.120:0/3442946858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556afa2fa260 0x556afa30c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.337+0000 7f60d2c8e640 1 --2- 172.21.15.120:0/3442946858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556afa2fd330 0x556afa313970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.337+0000 7f60d1c8c640 1 -- 172.21.15.120:0/3442946858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556afa2fd330 msgr2=0x556afa313970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.337+0000 7f60d1c8c640 1 --2- 172.21.15.120:0/3442946858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556afa2fd330 0x556afa313970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.337+0000 7f60d1c8c640 1 -- 172.21.15.120:0/3442946858 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556afa2f9920 msgr2=0x556afa30c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.337+0000 7f60d1c8c640 1 --2- 172.21.15.120:0/3442946858 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556afa2f9920 0x556afa30c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.337+0000 7f60d1c8c640 1 -- 172.21.15.120:0/3442946858 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556af9f70210 con 0x556afa2fa260 2024-09-19T09:21:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.337+0000 7f60d2c8e640 1 --2- 172.21.15.120:0/3442946858 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556afa2f9920 0x556afa30c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:21:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.337+0000 7f60d1c8c640 1 --2- 172.21.15.120:0/3442946858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556afa2fa260 0x556afa30c930 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f60bc031440 tx=0x7f60bc005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70fbf09614f2097f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.337+0000 7f60b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:21:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.338+0000 7f60b5ffb640 1 -- 172.21.15.120:0/3442946858 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc035040 con 0x556afa2fa260 2024-09-19T09:21:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.338+0000 7f60d748c740 1 -- 172.21.15.120:0/3442946858 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60c4019b10 con 0x556afa2fa260 2024-09-19T09:21:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.338+0000 7f60b5ffb640 1 -- 172.21.15.120:0/3442946858 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f60bc0319b0 con 0x556afa2fa260 2024-09-19T09:21:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.338+0000 7f60b5ffb640 1 -- 172.21.15.120:0/3442946858 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f60bc004040 con 0x556afa2fa260 2024-09-19T09:21:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.338+0000 7f60d748c740 10 client.30536 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.338+0000 7f60d748c740 1 -- 172.21.15.120:0/3442946858 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556afa03c620 con 0x556afa2fa260 2024-09-19T09:21:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.338+0000 7f60b5ffb640 1 -- 172.21.15.120:0/3442946858 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60c4019b10 con 0x556afa2fa260 2024-09-19T09:21:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.338+0000 7f60b5ffb640 1 -- 172.21.15.120:0/3442946858 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f60bc004920 con 0x556afa2fa260 2024-09-19T09:21:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.338+0000 7f60d748c740 10 client.30536 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.338+0000 7f60d748c740 10 client.30536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:21:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.338+0000 7f60d748c740 20 client.30536 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.338+0000 7f60d748c740 1 --2- 172.21.15.120:0/3442946858 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556afa318010 0x556afa31a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.338+0000 7f60d748c740 4 client.30536 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:21:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.338+0000 7f60d748c740 1 -- 172.21.15.120:0/3442946858 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556afa26aa20 con 0x556afa318010 2024-09-19T09:21:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.338+0000 7f60d2c8e640 1 --2- 172.21.15.120:0/3442946858 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556afa318010 0x556afa31a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.339+0000 7f60b5ffb640 10 client.30536 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.339+0000 7f60d2c8e640 1 --2- 172.21.15.120:0/3442946858 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556afa318010 0x556afa31a420 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=475be2b3a592b22e server_cookie=285acef8a229aa06 in_seq=0 out_seq=0 2024-09-19T09:21:42.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.341+0000 7f60b5ffb640 1 -- 172.21.15.120:0/3442946858 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556afa26aa20 con 0x556afa318010 2024-09-19T09:21:42.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.341+0000 7f60b5ffb640 10 client.30536 handle_command_reply: tid=0 2024-09-19T09:21:42.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.341+0000 7f60d748c740 1 client.30536 shutdown 2024-09-19T09:21:42.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.341+0000 7f60d748c740 1 -- 172.21.15.120:0/3442946858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556afa2fa260 msgr2=0x556afa30c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.341+0000 7f60d748c740 1 --2- 172.21.15.120:0/3442946858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556afa2fa260 0x556afa30c930 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f60bc031440 tx=0x7f60bc005e50 comp rx=0 tx=0).stop 2024-09-19T09:21:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.341+0000 7f60d748c740 1 -- 172.21.15.120:0/3442946858 shutdown_connections 2024-09-19T09:21:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.341+0000 7f60d748c740 1 --2- 172.21.15.120:0/3442946858 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556afa318010 0x556afa31a420 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.341+0000 7f60d748c740 1 --2- 172.21.15.120:0/3442946858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556afa2fd330 0x556afa313970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.341+0000 7f60d748c740 1 --2- 172.21.15.120:0/3442946858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556afa2fa260 0x556afa30c930 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.341+0000 7f60d748c740 1 --2- 172.21.15.120:0/3442946858 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556afa2f9920 0x556afa30c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.341+0000 7f60d748c740 1 -- 172.21.15.120:0/3442946858 >> 172.21.15.120:0/3442946858 conn(0x556afa2f5010 msgr2=0x556afa2f6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.341+0000 7f60b5ffb640 0 client.30536 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.341+0000 7f60d748c740 1 -- 172.21.15.120:0/3442946858 shutdown_connections 2024-09-19T09:21:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.341+0000 7f60d748c740 1 -- 172.21.15.120:0/3442946858 wait complete. 2024-09-19T09:21:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.341+0000 7f60d748c740 20 client.30536 trim_cache size 0 max 16384 2024-09-19T09:21:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.360+0000 7f60d748c740 1 Processor -- start 2024-09-19T09:21:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.360+0000 7f60d748c740 1 -- start start 2024-09-19T09:21:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.361+0000 7f60d748c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556afa2f9920 0x556afa31a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.361+0000 7f60d748c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556afa2fa260 0x556afa313930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.361+0000 7f60d748c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556afa2fd330 0x556afa2f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.361+0000 7f60d748c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc1a6d90 con 0x556afa2f9920 2024-09-19T09:21:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.361+0000 7f60d748c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc1a88d0 con 0x556afa2fa260 2024-09-19T09:21:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.361+0000 7f60d748c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc1a7460 con 0x556afa2fd330 2024-09-19T09:21:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.361+0000 7f60d2c8e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556afa2f9920 0x556afa31a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.361+0000 7f60d2c8e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556afa2f9920 0x556afa31a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41712/0 (socket says 172.21.15.120:41712) 2024-09-19T09:21:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.361+0000 7f60d2c8e640 1 -- 172.21.15.120:0/2929591626 learned_addr learned my addr 172.21.15.120:0/2929591626 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.361+0000 7f60d2c8e640 1 --2- 172.21.15.120:0/2929591626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556afa2fd330 0x556afa2f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.361+0000 7f60d1c8c640 1 --2- 172.21.15.120:0/2929591626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556afa2fa260 0x556afa313930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.361+0000 7f60d2c8e640 1 -- 172.21.15.120:0/2929591626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556afa2fd330 msgr2=0x556afa2f5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.361+0000 7f60d2c8e640 1 --2- 172.21.15.120:0/2929591626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556afa2fd330 0x556afa2f5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.361+0000 7f60d2c8e640 1 -- 172.21.15.120:0/2929591626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556afa2fa260 msgr2=0x556afa313930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.361+0000 7f60d2c8e640 1 --2- 172.21.15.120:0/2929591626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556afa2fa260 0x556afa313930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.361+0000 7f60d2c8e640 1 -- 172.21.15.120:0/2929591626 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556afa3064f0 con 0x556afa2f9920 2024-09-19T09:21:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.362+0000 7f60d2c8e640 1 --2- 172.21.15.120:0/2929591626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556afa2f9920 0x556afa31a400 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f60c4006010 tx=0x7f60c4009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1256a6eba51f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.362+0000 7f60b7fff640 1 -- 172.21.15.120:0/2929591626 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c401e040 con 0x556afa2f9920 2024-09-19T09:21:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.362+0000 7f60b7fff640 1 -- 172.21.15.120:0/2929591626 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60c4020040 con 0x556afa2f9920 2024-09-19T09:21:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.362+0000 7f60b7fff640 1 -- 172.21.15.120:0/2929591626 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c400ee50 con 0x556afa2f9920 2024-09-19T09:21:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.363+0000 7f60d748c740 1 -- 172.21.15.120:0/2929591626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556afa2f9920 msgr2=0x556afa31a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.363+0000 7f60d748c740 1 --2- 172.21.15.120:0/2929591626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556afa2f9920 0x556afa31a400 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f60c4006010 tx=0x7f60c4009e80 comp rx=0 tx=0).stop 2024-09-19T09:21:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.363+0000 7f60d748c740 1 -- 172.21.15.120:0/2929591626 shutdown_connections 2024-09-19T09:21:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.363+0000 7f60d748c740 1 --2- 172.21.15.120:0/2929591626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556afa2fd330 0x556afa2f5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.363+0000 7f60d748c740 1 --2- 172.21.15.120:0/2929591626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556afa2fa260 0x556afa313930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.363+0000 7f60d748c740 1 --2- 172.21.15.120:0/2929591626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556afa2f9920 0x556afa31a400 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.363+0000 7f60d748c740 1 -- 172.21.15.120:0/2929591626 >> 172.21.15.120:0/2929591626 conn(0x556afa2f5010 msgr2=0x556afa30a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.363+0000 7f60d748c740 1 -- 172.21.15.120:0/2929591626 shutdown_connections 2024-09-19T09:21:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.363+0000 7f60d748c740 1 -- 172.21.15.120:0/2929591626 wait complete. 2024-09-19T09:21:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.364+0000 7f60d748c740 1 -- start start 2024-09-19T09:21:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.364+0000 7f60d748c740 1 Processor -- start 2024-09-19T09:21:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.364+0000 7f60d748c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556afa2f9920 0x556afa310a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.364+0000 7f60d748c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556afa2fa260 0x556afa310f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.364+0000 7f60d748c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556afa2fd330 0x556afa311f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.364+0000 7f60d748c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc1a8200 con 0x556afa2fa260 2024-09-19T09:21:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.364+0000 7f60d748c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc1a7b30 con 0x556afa2f9920 2024-09-19T09:21:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.364+0000 7f60d748c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc1a4d50 con 0x556afa2fd330 2024-09-19T09:21:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.364+0000 7f60d748c740 10 client.0 _ll_register_callbacks cb 0x7f60bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.364+0000 7f60d2c8e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556afa2f9920 0x556afa310a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.364+0000 7f60d1c8c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556afa2fa260 0x556afa310f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.364+0000 7f60d2c8e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556afa2f9920 0x556afa310a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35024/0 (socket says 172.21.15.120:35024) 2024-09-19T09:21:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.364+0000 7f60d2c8e640 1 -- 172.21.15.120:0/422593259 learned_addr learned my addr 172.21.15.120:0/422593259 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.364+0000 7f60d2c8e640 1 --2- 172.21.15.120:0/422593259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556afa2fd330 0x556afa311f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.364+0000 7f60d1c8c640 1 -- 172.21.15.120:0/422593259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556afa2fd330 msgr2=0x556afa311f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.364+0000 7f60d1c8c640 1 --2- 172.21.15.120:0/422593259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556afa2fd330 0x556afa311f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.364+0000 7f60d1c8c640 1 -- 172.21.15.120:0/422593259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556afa2f9920 msgr2=0x556afa310a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.365+0000 7f60d1c8c640 1 --2- 172.21.15.120:0/422593259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556afa2f9920 0x556afa310a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.365+0000 7f60d1c8c640 1 -- 172.21.15.120:0/422593259 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556afa3125a0 con 0x556afa2fa260 2024-09-19T09:21:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.365+0000 7f60d1c8c640 1 --2- 172.21.15.120:0/422593259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556afa2fa260 0x556afa310f80 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f60bc02a930 tx=0x7f60bc004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a05fdb0d68da6fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.365+0000 7f60b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:21:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.365+0000 7f60b5ffb640 1 -- 172.21.15.120:0/422593259 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc035030 con 0x556afa2fa260 2024-09-19T09:21:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.365+0000 7f60b5ffb640 1 -- 172.21.15.120:0/422593259 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f60bc003970 con 0x556afa2fa260 2024-09-19T09:21:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.365+0000 7f60d748c740 1 -- 172.21.15.120:0/422593259 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60c4020040 con 0x556afa2fa260 2024-09-19T09:21:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.365+0000 7f60b5ffb640 1 -- 172.21.15.120:0/422593259 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f60bc056070 con 0x556afa2fa260 2024-09-19T09:21:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.365+0000 7f60d748c740 10 client.21717 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.365+0000 7f60d748c740 1 -- 172.21.15.120:0/422593259 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556afa313920 con 0x556afa2fa260 2024-09-19T09:21:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.366+0000 7f60b5ffb640 1 -- 172.21.15.120:0/422593259 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60c4020040 con 0x556afa2fa260 2024-09-19T09:21:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.366+0000 7f60b5ffb640 1 -- 172.21.15.120:0/422593259 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f60bc0316b0 con 0x556afa2fa260 2024-09-19T09:21:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.366+0000 7f60d748c740 10 client.21717 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.366+0000 7f60d748c740 10 client.21717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:21:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.366+0000 7f60d748c740 20 client.21717 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.366+0000 7f60d748c740 1 --2- 172.21.15.120:0/422593259 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556afa383190 0x556afa3855b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.366+0000 7f60d748c740 4 client.21717 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:21:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.366+0000 7f60d748c740 1 -- 172.21.15.120:0/422593259 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556afa264d70 con 0x556afa383190 2024-09-19T09:21:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.366+0000 7f60d2c8e640 1 --2- 172.21.15.120:0/422593259 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556afa383190 0x556afa3855b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.366+0000 7f60b5ffb640 10 client.21717 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.366+0000 7f60d2c8e640 1 --2- 172.21.15.120:0/422593259 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556afa383190 0x556afa3855b0 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aacd1586d14ec8c4 server_cookie=dae19704358065cf in_seq=0 out_seq=0 2024-09-19T09:21:42.371 INFO:tasks.workunit.client.0.smithi120.stdout: 10 143370 21.75 MB/sec execute 217 sec latency 101.400 ms 2024-09-19T09:21:42.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:42 smithi196 ceph-mon[35150]: pgmap v910: 97 pgs: 97 active+clean; 403 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.5 MiB/s wr, 614 op/s 2024-09-19T09:21:42.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:42 smithi153 ceph-mon[35071]: pgmap v910: 97 pgs: 97 active+clean; 403 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.5 MiB/s wr, 614 op/s 2024-09-19T09:21:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.928+0000 7f60b5ffb640 1 -- 172.21.15.120:0/422593259 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556afa264d70 con 0x556afa383190 2024-09-19T09:21:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.928+0000 7f60b5ffb640 10 client.21717 handle_command_reply: tid=0 2024-09-19T09:21:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.928+0000 7f60d748c740 1 client.21717 shutdown 2024-09-19T09:21:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.928+0000 7f60d748c740 1 -- 172.21.15.120:0/422593259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556afa2fa260 msgr2=0x556afa310f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.928+0000 7f60d748c740 1 --2- 172.21.15.120:0/422593259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556afa2fa260 0x556afa310f80 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f60bc02a930 tx=0x7f60bc004410 comp rx=0 tx=0).stop 2024-09-19T09:21:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.928+0000 7f60d748c740 1 -- 172.21.15.120:0/422593259 shutdown_connections 2024-09-19T09:21:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.928+0000 7f60d748c740 1 --2- 172.21.15.120:0/422593259 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556afa383190 0x556afa3855b0 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.928+0000 7f60d748c740 1 --2- 172.21.15.120:0/422593259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556afa2fd330 0x556afa311f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.928+0000 7f60b5ffb640 0 client.21717 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.928+0000 7f60d748c740 1 --2- 172.21.15.120:0/422593259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556afa2fa260 0x556afa310f80 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.928+0000 7f60d748c740 1 --2- 172.21.15.120:0/422593259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556afa2f9920 0x556afa310a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.928+0000 7f60d748c740 1 -- 172.21.15.120:0/422593259 >> 172.21.15.120:0/422593259 conn(0x556afa2f5010 msgr2=0x556afa2f5f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.928+0000 7f60d748c740 1 -- 172.21.15.120:0/422593259 shutdown_connections 2024-09-19T09:21:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.928+0000 7f60d748c740 1 -- 172.21.15.120:0/422593259 wait complete. 2024-09-19T09:21:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.928+0000 7f60d748c740 20 client.21717 trim_cache size 0 max 16384 2024-09-19T09:21:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:21:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:21:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:21:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:21:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.929+0000 7f60d46f1640 1 -- 172.21.15.120:0/1740455266 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f60b80a3370 msgr2=0x7f60b80a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.929+0000 7f60d46f1640 1 --2- 172.21.15.120:0/1740455266 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f60b80a3370 0x7f60b80a5830 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f60c80049d0 tx=0x7f60c8021000 comp rx=0 tx=0).stop 2024-09-19T09:21:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.929+0000 7f60d46f1640 1 -- 172.21.15.120:0/1740455266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60cc100a50 msgr2=0x7f60cc06aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.929+0000 7f60d46f1640 1 --2- 172.21.15.120:0/1740455266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60cc100a50 0x7f60cc06aeb0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f60c800ebc0 tx=0x7f60c800c6a0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.929+0000 7f60d46f1640 1 -- 172.21.15.120:0/1740455266 shutdown_connections 2024-09-19T09:21:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.929+0000 7f60d46f1640 1 --2- 172.21.15.120:0/1740455266 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f60b80a3370 0x7f60b80a5830 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.929+0000 7f60d46f1640 1 --2- 172.21.15.120:0/1740455266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60cc102600 0x7f60cc19f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.929+0000 7f60d46f1640 1 --2- 172.21.15.120:0/1740455266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60cc101c40 0x7f60cc19caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.929+0000 7f60d46f1640 1 --2- 172.21.15.120:0/1740455266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60cc100a50 0x7f60cc06aeb0 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.929+0000 7f60d46f1640 1 -- 172.21.15.120:0/1740455266 >> 172.21.15.120:0/1740455266 conn(0x7f60cc0fc1b0 msgr2=0x7f60cc104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.929+0000 7f60d46f1640 1 -- 172.21.15.120:0/1740455266 shutdown_connections 2024-09-19T09:21:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:42.929+0000 7f60d46f1640 1 -- 172.21.15.120:0/1740455266 wait complete. 2024-09-19T09:21:43.371 INFO:tasks.workunit.client.0.smithi120.stdout: 10 143739 21.71 MB/sec execute 218 sec latency 305.741 ms 2024-09-19T09:21:43.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:43 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:21:43.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:43 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:21:43.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:43 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:21:43.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:43 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:21:43.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:43 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:21:43.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:43 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:21:44.055 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-09-19T09:21:44.056 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:21:44.330 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:21:44.371 INFO:tasks.workunit.client.0.smithi120.stdout: 10 144514 21.82 MB/sec execute 219 sec latency 49.600 ms 2024-09-19T09:21:44.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:44 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:21:44.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:44 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:21:44.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:44 smithi196 ceph-mon[35150]: pgmap v911: 97 pgs: 97 active+clean; 406 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 3.8 MiB/s wr, 654 op/s 2024-09-19T09:21:44.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:44 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:21:44.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:44 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:21:44.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:44 smithi153 ceph-mon[35071]: pgmap v911: 97 pgs: 97 active+clean; 406 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 3.8 MiB/s wr, 654 op/s 2024-09-19T09:21:44.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:44 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:21:44.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:44 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:21:44.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:44 smithi120 ceph-mon[31150]: pgmap v911: 97 pgs: 97 active+clean; 406 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 3.8 MiB/s wr, 654 op/s 2024-09-19T09:21:44.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.857+0000 7f9236061640 1 -- 172.21.15.120:0/3248476765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9230102570 msgr2=0x7f92301029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:44.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.857+0000 7f9236061640 1 --2- 172.21.15.120:0/3248476765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9230102570 0x7f92301029b0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f9224009900 tx=0x7f922402a480 comp rx=0 tx=0).stop 2024-09-19T09:21:44.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.857+0000 7f9236061640 1 -- 172.21.15.120:0/3248476765 shutdown_connections 2024-09-19T09:21:44.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.857+0000 7f9236061640 1 --2- 172.21.15.120:0/3248476765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9230102ef0 0x7f923010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:44.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.857+0000 7f9236061640 1 --2- 172.21.15.120:0/3248476765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9230102570 0x7f92301029b0 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:44.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.857+0000 7f9236061640 1 --2- 172.21.15.120:0/3248476765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9230100e20 0x7f9230101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:44.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.858+0000 7f9236061640 1 -- 172.21.15.120:0/3248476765 >> 172.21.15.120:0/3248476765 conn(0x7f92300fc7d0 msgr2=0x7f92300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:44.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.858+0000 7f9236061640 1 -- 172.21.15.120:0/3248476765 shutdown_connections 2024-09-19T09:21:44.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.858+0000 7f9236061640 1 -- 172.21.15.120:0/3248476765 wait complete. 2024-09-19T09:21:44.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.858+0000 7f9236061640 1 Processor -- start 2024-09-19T09:21:44.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.858+0000 7f9236061640 1 -- start start 2024-09-19T09:21:44.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.859+0000 7f9236061640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9230100e20 0x7f923019e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:44.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.859+0000 7f9236061640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9230102570 0x7f923019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:44.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.859+0000 7f9236061640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9230102ef0 0x7f92301964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:44.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.859+0000 7f9236061640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9230110f80 con 0x7f9230102570 2024-09-19T09:21:44.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.859+0000 7f9236061640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9230110e00 con 0x7f9230100e20 2024-09-19T09:21:44.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.859+0000 7f9236061640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9230111100 con 0x7f9230102ef0 2024-09-19T09:21:44.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.859+0000 7f922f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9230100e20 0x7f923019e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:44.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.859+0000 7f922effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9230102570 0x7f923019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:44.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.859+0000 7f922ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9230102ef0 0x7f92301964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:44.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.859+0000 7f922f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9230100e20 0x7f923019e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35032/0 (socket says 172.21.15.120:35032) 2024-09-19T09:21:44.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.859+0000 7f922f7fe640 1 -- 172.21.15.120:0/701850078 learned_addr learned my addr 172.21.15.120:0/701850078 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:44.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.859+0000 7f922f7fe640 1 -- 172.21.15.120:0/701850078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9230102ef0 msgr2=0x7f92301964a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:44.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.859+0000 7f922f7fe640 1 --2- 172.21.15.120:0/701850078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9230102ef0 0x7f92301964a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:44.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.859+0000 7f922f7fe640 1 -- 172.21.15.120:0/701850078 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9230102570 msgr2=0x7f923019e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:44.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.859+0000 7f922f7fe640 1 --2- 172.21.15.120:0/701850078 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9230102570 0x7f923019e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:44.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.859+0000 7f922f7fe640 1 -- 172.21.15.120:0/701850078 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9230196d60 con 0x7f9230100e20 2024-09-19T09:21:44.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.859+0000 7f922effd640 1 --2- 172.21.15.120:0/701850078 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9230102570 0x7f923019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:21:44.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.860+0000 7f922f7fe640 1 --2- 172.21.15.120:0/701850078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9230100e20 0x7f923019e190 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f922000e9f0 tx=0x7f922000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:44.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.860+0000 7f922cff9640 1 -- 172.21.15.120:0/701850078 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922000cd80 con 0x7f9230100e20 2024-09-19T09:21:44.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.860+0000 7f9236061640 1 -- 172.21.15.120:0/701850078 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9230197050 con 0x7f9230100e20 2024-09-19T09:21:44.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.860+0000 7f922cff9640 1 -- 172.21.15.120:0/701850078 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9220005500 con 0x7f9230100e20 2024-09-19T09:21:44.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.860+0000 7f922cff9640 1 -- 172.21.15.120:0/701850078 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9220010690 con 0x7f9230100e20 2024-09-19T09:21:44.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.860+0000 7f9236061640 1 -- 172.21.15.120:0/701850078 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92301b3e20 con 0x7f9230100e20 2024-09-19T09:21:44.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.861+0000 7f9236061640 1 -- 172.21.15.120:0/701850078 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92301021b0 con 0x7f9230100e20 2024-09-19T09:21:44.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.862+0000 7f922cff9640 1 -- 172.21.15.120:0/701850078 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f92200046c0 con 0x7f9230100e20 2024-09-19T09:21:44.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.863+0000 7f922cff9640 1 --2- 172.21.15.120:0/701850078 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92180a3240 0x7f92180a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:44.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.863+0000 7f922effd640 1 --2- 172.21.15.120:0/701850078 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92180a3240 0x7f92180a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:44.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.863+0000 7f922cff9640 1 -- 172.21.15.120:0/701850078 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f92200cffe0 con 0x7f9230100e20 2024-09-19T09:21:44.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.863+0000 7f922effd640 1 --2- 172.21.15.120:0/701850078 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92180a3240 0x7f92180a5700 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f92301978b0 tx=0x7f92240058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:44.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:44.870+0000 7f922cff9640 1 -- 172.21.15.120:0/701850078 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f922009cd00 con 0x7f9230100e20 2024-09-19T09:21:45.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:45.234+0000 7f9236061640 1 -- 172.21.15.120:0/701850078 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9230101200 con 0x7f9230100e20 2024-09-19T09:21:45.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:45.237+0000 7f922cff9640 1 -- 172.21.15.120:0/701850078 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11988 (secure 0 0 0) 0x7f9230101200 con 0x7f9230100e20 2024-09-19T09:21:45.238 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:21:45.239 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":39,"btime":"2024-09-19T09:18:53:120858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:18:53.070639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:21:45.239 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 39 2024-09-19T09:21:45.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:45.243+0000 7f9236061640 1 -- 172.21.15.120:0/701850078 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92180a3240 msgr2=0x7f92180a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:45.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:45.243+0000 7f9236061640 1 --2- 172.21.15.120:0/701850078 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92180a3240 0x7f92180a5700 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f92301978b0 tx=0x7f92240058d0 comp rx=0 tx=0).stop 2024-09-19T09:21:45.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:45.243+0000 7f9236061640 1 -- 172.21.15.120:0/701850078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9230100e20 msgr2=0x7f923019e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:45.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:45.243+0000 7f9236061640 1 --2- 172.21.15.120:0/701850078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9230100e20 0x7f923019e190 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f922000e9f0 tx=0x7f922000edf0 comp rx=0 tx=0).stop 2024-09-19T09:21:45.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:45.243+0000 7f9236061640 1 -- 172.21.15.120:0/701850078 shutdown_connections 2024-09-19T09:21:45.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:45.243+0000 7f9236061640 1 --2- 172.21.15.120:0/701850078 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92180a3240 0x7f92180a5700 secure :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f92301978b0 tx=0x7f92240058d0 comp rx=0 tx=0).stop 2024-09-19T09:21:45.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:45.243+0000 7f9236061640 1 --2- 172.21.15.120:0/701850078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9230102ef0 0x7f92301964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:45.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:45.243+0000 7f9236061640 1 --2- 172.21.15.120:0/701850078 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9230102570 0x7f923019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:45.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:45.243+0000 7f9236061640 1 --2- 172.21.15.120:0/701850078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9230100e20 0x7f923019e190 secure :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f922000e9f0 tx=0x7f922000edf0 comp rx=0 tx=0).stop 2024-09-19T09:21:45.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:45.243+0000 7f9236061640 1 -- 172.21.15.120:0/701850078 >> 172.21.15.120:0/701850078 conn(0x7f92300fc7d0 msgr2=0x7f923006a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:45.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:45.245+0000 7f9236061640 1 -- 172.21.15.120:0/701850078 shutdown_connections 2024-09-19T09:21:45.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:45.246+0000 7f9236061640 1 -- 172.21.15.120:0/701850078 wait complete. 2024-09-19T09:21:45.373 INFO:tasks.workunit.client.0.smithi120.stdout: 10 144889 21.79 MB/sec execute 220 sec latency 399.278 ms 2024-09-19T09:21:45.577 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:45 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/701850078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:21:45.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:45 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/701850078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:21:45.689 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:45 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/701850078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:21:46.371 INFO:tasks.workunit.client.0.smithi120.stdout: 10 144913 21.72 MB/sec execute 221 sec latency 792.197 ms 2024-09-19T09:21:46.420 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-09-19T09:21:46.420 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-09-19T09:21:46.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:46 smithi120 ceph-mon[31150]: pgmap v912: 97 pgs: 97 active+clean; 407 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.0 MiB/s wr, 599 op/s 2024-09-19T09:21:46.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:46 smithi196 ceph-mon[35150]: pgmap v912: 97 pgs: 97 active+clean; 407 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.0 MiB/s wr, 599 op/s 2024-09-19T09:21:46.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:46 smithi153 ceph-mon[35071]: pgmap v912: 97 pgs: 97 active+clean; 407 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.0 MiB/s wr, 599 op/s 2024-09-19T09:21:46.751 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:21:47.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.283+0000 7fc62d4af640 1 -- 172.21.15.120:0/1925739381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc628105de0 msgr2=0x7fc62810d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:47.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.283+0000 7fc62d4af640 1 --2- 172.21.15.120:0/1925739381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc628105de0 0x7fc62810d960 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fc620009900 tx=0x7fc62002a410 comp rx=0 tx=0).stop 2024-09-19T09:21:47.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.283+0000 7fc62d4af640 1 -- 172.21.15.120:0/1925739381 shutdown_connections 2024-09-19T09:21:47.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.283+0000 7fc62d4af640 1 --2- 172.21.15.120:0/1925739381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc628105de0 0x7fc62810d960 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:47.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.283+0000 7fc62d4af640 1 --2- 172.21.15.120:0/1925739381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc628101e20 0x7fc6281056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:47.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.283+0000 7fc62d4af640 1 --2- 172.21.15.120:0/1925739381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc628101470 0x7fc628101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:47.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.283+0000 7fc62d4af640 1 -- 172.21.15.120:0/1925739381 >> 172.21.15.120:0/1925739381 conn(0x7fc628077230 msgr2=0x7fc6280fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:47.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.283+0000 7fc62d4af640 1 -- 172.21.15.120:0/1925739381 shutdown_connections 2024-09-19T09:21:47.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.283+0000 7fc62d4af640 1 -- 172.21.15.120:0/1925739381 wait complete. 2024-09-19T09:21:47.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.284+0000 7fc62d4af640 1 Processor -- start 2024-09-19T09:21:47.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.284+0000 7fc62d4af640 1 -- start start 2024-09-19T09:21:47.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.284+0000 7fc62d4af640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc628101470 0x7fc6281983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:47.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.285+0000 7fc62d4af640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc628101e20 0x7fc6281988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:47.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.285+0000 7fc62d4af640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc628105de0 0x7fc62819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:47.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.285+0000 7fc62d4af640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc628110e40 con 0x7fc628101470 2024-09-19T09:21:47.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.285+0000 7fc626ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc628101470 0x7fc6281983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:47.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.285+0000 7fc62d4af640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc628110cc0 con 0x7fc628105de0 2024-09-19T09:21:47.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.285+0000 7fc62d4af640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc628110fc0 con 0x7fc628101e20 2024-09-19T09:21:47.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.285+0000 7fc626ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc628101470 0x7fc6281983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41762/0 (socket says 172.21.15.120:41762) 2024-09-19T09:21:47.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.285+0000 7fc626ffd640 1 -- 172.21.15.120:0/4054697439 learned_addr learned my addr 172.21.15.120:0/4054697439 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:47.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.285+0000 7fc6267fc640 1 --2- 172.21.15.120:0/4054697439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc628101e20 0x7fc6281988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:47.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.285+0000 7fc6277fe640 1 --2- 172.21.15.120:0/4054697439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc628105de0 0x7fc62819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:47.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.285+0000 7fc626ffd640 1 -- 172.21.15.120:0/4054697439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc628101e20 msgr2=0x7fc6281988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:47.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.285+0000 7fc626ffd640 1 --2- 172.21.15.120:0/4054697439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc628101e20 0x7fc6281988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:47.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.285+0000 7fc626ffd640 1 -- 172.21.15.120:0/4054697439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc628105de0 msgr2=0x7fc62819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:47.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.285+0000 7fc626ffd640 1 --2- 172.21.15.120:0/4054697439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc628105de0 0x7fc62819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:47.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.285+0000 7fc626ffd640 1 -- 172.21.15.120:0/4054697439 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62819d360 con 0x7fc628101470 2024-09-19T09:21:47.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.285+0000 7fc6267fc640 1 --2- 172.21.15.120:0/4054697439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc628101e20 0x7fc6281988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:21:47.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.285+0000 7fc626ffd640 1 --2- 172.21.15.120:0/4054697439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc628101470 0x7fc6281983b0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fc61800db80 tx=0x7fc61800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:47.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.285+0000 7fc60ffff640 1 -- 172.21.15.120:0/4054697439 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618017070 con 0x7fc628101470 2024-09-19T09:21:47.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.285+0000 7fc60ffff640 1 -- 172.21.15.120:0/4054697439 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc618003b70 con 0x7fc628101470 2024-09-19T09:21:47.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.285+0000 7fc60ffff640 1 -- 172.21.15.120:0/4054697439 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6180106c0 con 0x7fc628101470 2024-09-19T09:21:47.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.285+0000 7fc62d4af640 1 -- 172.21.15.120:0/4054697439 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc62819d650 con 0x7fc628101470 2024-09-19T09:21:47.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.285+0000 7fc62d4af640 1 -- 172.21.15.120:0/4054697439 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6281a4f30 con 0x7fc628101470 2024-09-19T09:21:47.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.289+0000 7fc62f888740 1 Processor -- start 2024-09-19T09:21:47.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.289+0000 7fc62f888740 1 -- start start 2024-09-19T09:21:47.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.289+0000 7fc60ffff640 1 -- 172.21.15.120:0/4054697439 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fc618004020 con 0x7fc628101470 2024-09-19T09:21:47.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.289+0000 7fc62f888740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d0b8fa760 0x564d0b8fab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:47.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.289+0000 7fc62f888740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d0b8fb080 0x564d0b905400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:47.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.289+0000 7fc62f888740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d0b905940 0x564d0b907d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:47.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.289+0000 7fc62f888740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564d0b8602a0 con 0x564d0b8fa760 2024-09-19T09:21:47.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.289+0000 7fc62f888740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564d0b807b10 con 0x564d0b905940 2024-09-19T09:21:47.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.289+0000 7fc62f888740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564d0b85c260 con 0x564d0b8fb080 2024-09-19T09:21:47.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.289+0000 7fc6277fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d0b8fa760 0x564d0b8fab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:47.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.289+0000 7fc626ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d0b8fb080 0x564d0b905400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:47.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.289+0000 7fc6277fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d0b8fa760 0x564d0b8fab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41764/0 (socket says 172.21.15.120:41764) 2024-09-19T09:21:47.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.289+0000 7fc6277fe640 1 -- 172.21.15.120:0/4169921215 learned_addr learned my addr 172.21.15.120:0/4169921215 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:47.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.289+0000 7fc6267fc640 1 --2- 172.21.15.120:0/4169921215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d0b905940 0x564d0b907d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:47.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.290+0000 7fc6277fe640 1 -- 172.21.15.120:0/4169921215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d0b8fb080 msgr2=0x564d0b905400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:47.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.290+0000 7fc6277fe640 1 --2- 172.21.15.120:0/4169921215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d0b8fb080 0x564d0b905400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:47.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.290+0000 7fc6277fe640 1 -- 172.21.15.120:0/4169921215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d0b905940 msgr2=0x564d0b907d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:47.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.290+0000 7fc6277fe640 1 --2- 172.21.15.120:0/4169921215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d0b905940 0x564d0b907d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:47.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.290+0000 7fc6277fe640 1 -- 172.21.15.120:0/4169921215 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d0b5d1b90 con 0x564d0b8fa760 2024-09-19T09:21:47.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.290+0000 7fc60ffff640 1 --2- 172.21.15.120:0/4054697439 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc6100a3240 0x7fc6100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:47.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.290+0000 7fc6277fe640 1 --2- 172.21.15.120:0/4169921215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d0b8fa760 0x564d0b8fab40 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fc620033000 tx=0x7fc620005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a77140e15b16008 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:47.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.290+0000 7fc626ffd640 1 --2- 172.21.15.120:0/4054697439 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc6100a3240 0x7fc6100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:47.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.290+0000 7fc60d7fa640 1 -- 172.21.15.120:0/4169921215 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc620040070 con 0x564d0b8fa760 2024-09-19T09:21:47.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.290+0000 7fc60ffff640 1 -- 172.21.15.120:0/4054697439 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc6180cf680 con 0x7fc628101470 2024-09-19T09:21:47.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.290+0000 7fc60d7fa640 1 -- 172.21.15.120:0/4169921215 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6200318b0 con 0x564d0b8fa760 2024-09-19T09:21:47.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.290+0000 7fc60d7fa640 1 -- 172.21.15.120:0/4169921215 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc620035040 con 0x564d0b8fa760 2024-09-19T09:21:47.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.291+0000 7fc62f888740 1 -- 172.21.15.120:0/4169921215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d0b8fa760 msgr2=0x564d0b8fab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:47.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.291+0000 7fc626ffd640 1 --2- 172.21.15.120:0/4054697439 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc6100a3240 0x7fc6100a5700 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fc61800db50 tx=0x7fc6180d9940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:47.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.291+0000 7fc62f888740 1 --2- 172.21.15.120:0/4169921215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d0b8fa760 0x564d0b8fab40 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fc620033000 tx=0x7fc620005e00 comp rx=0 tx=0).stop 2024-09-19T09:21:47.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.291+0000 7fc62f888740 1 -- 172.21.15.120:0/4169921215 shutdown_connections 2024-09-19T09:21:47.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.291+0000 7fc62f888740 1 --2- 172.21.15.120:0/4169921215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d0b905940 0x564d0b907d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:47.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.291+0000 7fc62f888740 1 --2- 172.21.15.120:0/4169921215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d0b8fb080 0x564d0b905400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:47.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.291+0000 7fc62f888740 1 --2- 172.21.15.120:0/4169921215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d0b8fa760 0x564d0b8fab40 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:47.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.291+0000 7fc62f888740 1 -- 172.21.15.120:0/4169921215 >> 172.21.15.120:0/4169921215 conn(0x564d0b8f6cf0 msgr2=0x564d0b8f70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:47.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.291+0000 7fc62f888740 1 -- 172.21.15.120:0/4169921215 shutdown_connections 2024-09-19T09:21:47.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.291+0000 7fc62f888740 1 -- 172.21.15.120:0/4169921215 wait complete. 2024-09-19T09:21:47.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.292+0000 7fc62f888740 1 -- start start 2024-09-19T09:21:47.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.292+0000 7fc62f888740 1 Processor -- start 2024-09-19T09:21:47.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.292+0000 7fc62f888740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d0b8fa760 0x564d0b9027b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:47.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.292+0000 7fc62f888740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d0b8fb080 0x564d0b902cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:47.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.292+0000 7fc62f888740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d0b905940 0x564d0b903230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:47.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.292+0000 7fc62f888740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564d0b908510 con 0x564d0b905940 2024-09-19T09:21:47.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.292+0000 7fc62f888740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564d0b7e86e0 con 0x564d0b8fb080 2024-09-19T09:21:47.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.292+0000 7fc62f888740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564d0b908690 con 0x564d0b8fa760 2024-09-19T09:21:47.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.292+0000 7fc62f888740 10 client.0 _ll_register_callbacks cb 0x564d0b80ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:47.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.292+0000 7fc6277fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d0b905940 0x564d0b903230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:47.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.292+0000 7fc6277fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d0b905940 0x564d0b903230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41776/0 (socket says 172.21.15.120:41776) 2024-09-19T09:21:47.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.292+0000 7fc6277fe640 1 -- 172.21.15.120:0/259234686 learned_addr learned my addr 172.21.15.120:0/259234686 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:47.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.292+0000 7fc6267fc640 1 --2- 172.21.15.120:0/259234686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d0b8fb080 0x564d0b902cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:47.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.292+0000 7fc6277fe640 1 --2- 172.21.15.120:0/259234686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d0b8fa760 0x564d0b9027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:47.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.292+0000 7fc6277fe640 1 -- 172.21.15.120:0/259234686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d0b8fa760 msgr2=0x564d0b9027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:47.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.292+0000 7fc6277fe640 1 --2- 172.21.15.120:0/259234686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d0b8fa760 0x564d0b9027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:47.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.292+0000 7fc6277fe640 1 -- 172.21.15.120:0/259234686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d0b8fb080 msgr2=0x564d0b902cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:47.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.292+0000 7fc6277fe640 1 --2- 172.21.15.120:0/259234686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d0b8fb080 0x564d0b902cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:47.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.292+0000 7fc6277fe640 1 -- 172.21.15.120:0/259234686 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d0b911980 con 0x564d0b905940 2024-09-19T09:21:47.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.293+0000 7fc6277fe640 1 --2- 172.21.15.120:0/259234686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d0b905940 0x564d0b903230 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fc6200096c0 tx=0x7fc6200055e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0636fa2bee224c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:47.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.293+0000 7fc60aff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:21:47.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.293+0000 7fc60aff5640 1 -- 172.21.15.120:0/259234686 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc620035030 con 0x564d0b905940 2024-09-19T09:21:47.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.293+0000 7fc60aff5640 1 -- 172.21.15.120:0/259234686 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc6200316b0 con 0x564d0b905940 2024-09-19T09:21:47.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.293+0000 7fc62f888740 1 -- 172.21.15.120:0/259234686 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6200318b0 con 0x564d0b905940 2024-09-19T09:21:47.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.293+0000 7fc60aff5640 1 -- 172.21.15.120:0/259234686 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc620040040 con 0x564d0b905940 2024-09-19T09:21:47.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.293+0000 7fc62f888740 10 client.21735 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:47.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.293+0000 7fc62f888740 1 -- 172.21.15.120:0/259234686 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564d0b9142f0 con 0x564d0b905940 2024-09-19T09:21:47.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.293+0000 7fc60aff5640 1 -- 172.21.15.120:0/259234686 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6200318b0 con 0x564d0b905940 2024-09-19T09:21:47.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.293+0000 7fc60aff5640 1 -- 172.21.15.120:0/259234686 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fc62003c9a0 con 0x564d0b905940 2024-09-19T09:21:47.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.293+0000 7fc62f888740 10 client.21735 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:47.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.293+0000 7fc62f888740 10 client.21735 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:21:47.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.293+0000 7fc62f888740 20 client.21735 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:47.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.293+0000 7fc62f888740 1 --2- 172.21.15.120:0/259234686 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564d0b915640 0x564d0b917a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:47.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.293+0000 7fc62f888740 4 client.21735 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:21:47.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.293+0000 7fc62f888740 1 -- 172.21.15.120:0/259234686 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d0b86d9d0 con 0x564d0b915640 2024-09-19T09:21:47.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.294+0000 7fc6267fc640 1 --2- 172.21.15.120:0/259234686 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564d0b915640 0x564d0b917a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:47.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.294+0000 7fc6267fc640 1 --2- 172.21.15.120:0/259234686 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564d0b915640 0x564d0b917a30 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ed3a14d54b7fc3c server_cookie=87e1306982e051eb in_seq=0 out_seq=0 2024-09-19T09:21:47.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.294+0000 7fc60aff5640 10 client.21735 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:47.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.297+0000 7fc60aff5640 1 -- 172.21.15.120:0/259234686 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564d0b86d9d0 con 0x564d0b915640 2024-09-19T09:21:47.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.297+0000 7fc60aff5640 10 client.21735 handle_command_reply: tid=0 2024-09-19T09:21:47.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.298+0000 7fc62f888740 1 client.21735 shutdown 2024-09-19T09:21:47.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.298+0000 7fc62f888740 1 -- 172.21.15.120:0/259234686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d0b905940 msgr2=0x564d0b903230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:47.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.298+0000 7fc62f888740 1 --2- 172.21.15.120:0/259234686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d0b905940 0x564d0b903230 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fc6200096c0 tx=0x7fc6200055e0 comp rx=0 tx=0).stop 2024-09-19T09:21:47.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.298+0000 7fc62f888740 1 -- 172.21.15.120:0/259234686 shutdown_connections 2024-09-19T09:21:47.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.298+0000 7fc62f888740 1 --2- 172.21.15.120:0/259234686 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564d0b915640 0x564d0b917a30 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:47.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.298+0000 7fc62f888740 1 --2- 172.21.15.120:0/259234686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d0b905940 0x564d0b903230 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:47.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.298+0000 7fc62f888740 1 --2- 172.21.15.120:0/259234686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d0b8fb080 0x564d0b902cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:47.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.298+0000 7fc62f888740 1 --2- 172.21.15.120:0/259234686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d0b8fa760 0x564d0b9027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:47.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.298+0000 7fc62f888740 1 -- 172.21.15.120:0/259234686 >> 172.21.15.120:0/259234686 conn(0x564d0b8f6cf0 msgr2=0x564d0b8f5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:47.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.298+0000 7fc60aff5640 0 client.21735 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:47.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.298+0000 7fc62f888740 1 -- 172.21.15.120:0/259234686 shutdown_connections 2024-09-19T09:21:47.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.298+0000 7fc62f888740 1 -- 172.21.15.120:0/259234686 wait complete. 2024-09-19T09:21:47.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.298+0000 7fc62f888740 20 client.21735 trim_cache size 0 max 16384 2024-09-19T09:21:47.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.315+0000 7fc62f888740 1 Processor -- start 2024-09-19T09:21:47.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.315+0000 7fc62f888740 1 -- start start 2024-09-19T09:21:47.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.315+0000 7fc62f888740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d0b8fa760 0x564d0b8fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:47.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.315+0000 7fc62f888740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d0b8fb080 0x564d0b91a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:47.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.315+0000 7fc62f888740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d0b905940 0x564d0b91abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:47.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.315+0000 7fc62f888740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc6281ab3f0 con 0x564d0b8fa760 2024-09-19T09:21:47.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.315+0000 7fc62f888740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc6281acf30 con 0x564d0b905940 2024-09-19T09:21:47.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.315+0000 7fc62f888740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc6281abac0 con 0x564d0b8fb080 2024-09-19T09:21:47.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.315+0000 7fc6277fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d0b8fa760 0x564d0b8fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:47.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.315+0000 7fc6277fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d0b8fa760 0x564d0b8fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41784/0 (socket says 172.21.15.120:41784) 2024-09-19T09:21:47.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.315+0000 7fc6277fe640 1 -- 172.21.15.120:0/3567253078 learned_addr learned my addr 172.21.15.120:0/3567253078 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:47.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.315+0000 7fc6267fc640 1 --2- 172.21.15.120:0/3567253078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d0b8fb080 0x564d0b91a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:47.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.315+0000 7fc6277fe640 1 --2- 172.21.15.120:0/3567253078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d0b905940 0x564d0b91abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:47.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.315+0000 7fc6277fe640 1 -- 172.21.15.120:0/3567253078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d0b8fb080 msgr2=0x564d0b91a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:47.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.315+0000 7fc6277fe640 1 --2- 172.21.15.120:0/3567253078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d0b8fb080 0x564d0b91a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:47.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.315+0000 7fc6277fe640 1 -- 172.21.15.120:0/3567253078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d0b905940 msgr2=0x564d0b91abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:47.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.315+0000 7fc6277fe640 1 --2- 172.21.15.120:0/3567253078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d0b905940 0x564d0b91abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:47.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.315+0000 7fc6277fe640 1 -- 172.21.15.120:0/3567253078 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d0b8faeb0 con 0x564d0b8fa760 2024-09-19T09:21:47.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.315+0000 7fc6267fc640 1 --2- 172.21.15.120:0/3567253078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d0b8fb080 0x564d0b91a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:21:47.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.316+0000 7fc6277fe640 1 --2- 172.21.15.120:0/3567253078 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d0b8fa760 0x564d0b8fd1c0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fc620005c40 tx=0x7fc62002ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b32031b540e52c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:47.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.316+0000 7fc60cff9640 1 -- 172.21.15.120:0/3567253078 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc620035040 con 0x564d0b8fa760 2024-09-19T09:21:47.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.316+0000 7fc60cff9640 1 -- 172.21.15.120:0/3567253078 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc620005650 con 0x564d0b8fa760 2024-09-19T09:21:47.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.316+0000 7fc60cff9640 1 -- 172.21.15.120:0/3567253078 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62003c440 con 0x564d0b8fa760 2024-09-19T09:21:47.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.317+0000 7fc62f888740 1 -- 172.21.15.120:0/3567253078 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d0b8fa760 msgr2=0x564d0b8fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:47.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.317+0000 7fc62f888740 1 --2- 172.21.15.120:0/3567253078 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d0b8fa760 0x564d0b8fd1c0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fc620005c40 tx=0x7fc62002ab90 comp rx=0 tx=0).stop 2024-09-19T09:21:47.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.317+0000 7fc62f888740 1 -- 172.21.15.120:0/3567253078 shutdown_connections 2024-09-19T09:21:47.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.317+0000 7fc62f888740 1 --2- 172.21.15.120:0/3567253078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d0b905940 0x564d0b91abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:47.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.317+0000 7fc62f888740 1 --2- 172.21.15.120:0/3567253078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d0b8fb080 0x564d0b91a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:47.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.317+0000 7fc62f888740 1 --2- 172.21.15.120:0/3567253078 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d0b8fa760 0x564d0b8fd1c0 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:47.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.317+0000 7fc62f888740 1 -- 172.21.15.120:0/3567253078 >> 172.21.15.120:0/3567253078 conn(0x564d0b8f6cf0 msgr2=0x564d0b911100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:47.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.317+0000 7fc62f888740 1 -- 172.21.15.120:0/3567253078 shutdown_connections 2024-09-19T09:21:47.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.317+0000 7fc62f888740 1 -- 172.21.15.120:0/3567253078 wait complete. 2024-09-19T09:21:47.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.317+0000 7fc62f888740 1 -- start start 2024-09-19T09:21:47.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.317+0000 7fc62f888740 1 Processor -- start 2024-09-19T09:21:47.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.317+0000 7fc62f888740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d0b8fa760 0x564d0b918800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:47.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.317+0000 7fc62f888740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d0b8fb080 0x564d0b918d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:47.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.317+0000 7fc62f888740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d0b905940 0x564d0b97ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:47.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.317+0000 7fc62f888740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc6281ac860 con 0x564d0b8fa760 2024-09-19T09:21:47.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.317+0000 7fc62f888740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc6281ac190 con 0x564d0b8fb080 2024-09-19T09:21:47.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.317+0000 7fc62f888740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc6281a93b0 con 0x564d0b905940 2024-09-19T09:21:47.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.317+0000 7fc62f888740 10 client.0 _ll_register_callbacks cb 0x7fc620009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:47.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.317+0000 7fc6277fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d0b8fa760 0x564d0b918800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:47.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.317+0000 7fc6277fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d0b8fa760 0x564d0b918800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41794/0 (socket says 172.21.15.120:41794) 2024-09-19T09:21:47.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.318+0000 7fc6277fe640 1 -- 172.21.15.120:0/1760468575 learned_addr learned my addr 172.21.15.120:0/1760468575 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:47.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.318+0000 7fc6267fc640 1 --2- 172.21.15.120:0/1760468575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d0b8fb080 0x564d0b918d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:47.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.318+0000 7fc6277fe640 1 --2- 172.21.15.120:0/1760468575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d0b905940 0x564d0b97ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:47.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.318+0000 7fc6277fe640 1 -- 172.21.15.120:0/1760468575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d0b905940 msgr2=0x564d0b97ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:47.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.318+0000 7fc6277fe640 1 --2- 172.21.15.120:0/1760468575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d0b905940 0x564d0b97ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:47.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.318+0000 7fc6277fe640 1 -- 172.21.15.120:0/1760468575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d0b8fb080 msgr2=0x564d0b918d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:47.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.318+0000 7fc6277fe640 1 --2- 172.21.15.120:0/1760468575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d0b8fb080 0x564d0b918d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:47.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.318+0000 7fc6277fe640 1 -- 172.21.15.120:0/1760468575 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d0b97f4c0 con 0x564d0b8fa760 2024-09-19T09:21:47.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.318+0000 7fc6277fe640 1 --2- 172.21.15.120:0/1760468575 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d0b8fa760 0x564d0b918800 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fc620004dd0 tx=0x7fc620004cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4097fa2170ee7964 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:47.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.318+0000 7fc60aff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:21:47.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.318+0000 7fc60aff5640 1 -- 172.21.15.120:0/1760468575 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc620035040 con 0x564d0b8fa760 2024-09-19T09:21:47.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.318+0000 7fc60aff5640 1 -- 172.21.15.120:0/1760468575 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc620004480 con 0x564d0b8fa760 2024-09-19T09:21:47.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.318+0000 7fc62f888740 1 -- 172.21.15.120:0/1760468575 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc620005650 con 0x564d0b8fa760 2024-09-19T09:21:47.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.318+0000 7fc60aff5640 1 -- 172.21.15.120:0/1760468575 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc6200315b0 con 0x564d0b8fa760 2024-09-19T09:21:47.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.318+0000 7fc62f888740 10 client.21741 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:47.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.318+0000 7fc62f888740 1 -- 172.21.15.120:0/1760468575 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564d0b9805f0 con 0x564d0b8fa760 2024-09-19T09:21:47.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.319+0000 7fc60aff5640 1 -- 172.21.15.120:0/1760468575 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc620005650 con 0x564d0b8fa760 2024-09-19T09:21:47.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.319+0000 7fc60aff5640 1 -- 172.21.15.120:0/1760468575 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fc6200035f0 con 0x564d0b8fa760 2024-09-19T09:21:47.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.319+0000 7fc62f888740 10 client.21741 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:47.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.319+0000 7fc62f888740 10 client.21741 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:21:47.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.319+0000 7fc62f888740 20 client.21741 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:47.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.319+0000 7fc62f888740 1 --2- 172.21.15.120:0/1760468575 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564d0b980bf0 0x564d0b982fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:47.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.319+0000 7fc62f888740 4 client.21741 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-09-19T09:21:47.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.319+0000 7fc62f888740 1 -- 172.21.15.120:0/1760468575 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x564d0b867d20 con 0x564d0b980bf0 2024-09-19T09:21:47.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.319+0000 7fc6267fc640 1 --2- 172.21.15.120:0/1760468575 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564d0b980bf0 0x564d0b982fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:47.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.319+0000 7fc6267fc640 1 --2- 172.21.15.120:0/1760468575 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564d0b980bf0 0x564d0b982fe0 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f85348736629dcbe server_cookie=5f022824912bf1c0 in_seq=0 out_seq=0 2024-09-19T09:21:47.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:47.319+0000 7fc60aff5640 10 client.21741 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:47.371 INFO:tasks.workunit.client.0.smithi120.stdout: 10 145249 21.74 MB/sec execute 222 sec latency 663.799 ms 2024-09-19T09:21:48.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:48.064+0000 7fc60aff5640 1 -- 172.21.15.120:0/1760468575 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564d0b867d20 con 0x564d0b980bf0 2024-09-19T09:21:48.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:48.064+0000 7fc60aff5640 10 client.21741 handle_command_reply: tid=0 2024-09-19T09:21:48.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:48.064+0000 7fc62f888740 1 client.21741 shutdown 2024-09-19T09:21:48.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:48.064+0000 7fc62f888740 1 -- 172.21.15.120:0/1760468575 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d0b8fa760 msgr2=0x564d0b918800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:48.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:48.064+0000 7fc62f888740 1 --2- 172.21.15.120:0/1760468575 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d0b8fa760 0x564d0b918800 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fc620004dd0 tx=0x7fc620004cd0 comp rx=0 tx=0).stop 2024-09-19T09:21:48.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:48.064+0000 7fc62f888740 1 -- 172.21.15.120:0/1760468575 shutdown_connections 2024-09-19T09:21:48.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:48.064+0000 7fc62f888740 1 --2- 172.21.15.120:0/1760468575 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564d0b980bf0 0x564d0b982fe0 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:48.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:48.065+0000 7fc62f888740 1 --2- 172.21.15.120:0/1760468575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d0b905940 0x564d0b97ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:48.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:48.065+0000 7fc62f888740 1 --2- 172.21.15.120:0/1760468575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564d0b8fb080 0x564d0b918d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:48.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:48.065+0000 7fc60aff5640 0 client.21741 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:48.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:48.065+0000 7fc62f888740 1 --2- 172.21.15.120:0/1760468575 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564d0b8fa760 0x564d0b918800 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:48.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:48.065+0000 7fc62f888740 1 -- 172.21.15.120:0/1760468575 >> 172.21.15.120:0/1760468575 conn(0x564d0b8f6cf0 msgr2=0x564d0b900f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:48.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:48.065+0000 7fc62f888740 1 -- 172.21.15.120:0/1760468575 shutdown_connections 2024-09-19T09:21:48.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:48.065+0000 7fc62f888740 1 -- 172.21.15.120:0/1760468575 wait complete. 2024-09-19T09:21:48.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:48.065+0000 7fc62f888740 20 client.21741 trim_cache size 0 max 16384 2024-09-19T09:21:48.068 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:21:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 39, 2024-09-19T09:21:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:21:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 62, 2024-09-19T09:21:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:21:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: "c69603ab": { 2024-09-19T09:21:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 62, 2024-09-19T09:21:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:21:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:21:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:21:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:21:48.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:21:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:21:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 130.0, 2024-09-19T09:21:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:21:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:21:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:21:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:21:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:21:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:21:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:21:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:21:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:21:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:21:48.069 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:21:48.069 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:21:48.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:48.065+0000 7fc62d4af640 1 -- 172.21.15.120:0/4054697439 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc6100a3240 msgr2=0x7fc6100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:48.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:48.065+0000 7fc62d4af640 1 --2- 172.21.15.120:0/4054697439 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc6100a3240 0x7fc6100a5700 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fc61800db50 tx=0x7fc6180d9940 comp rx=0 tx=0).stop 2024-09-19T09:21:48.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:48.065+0000 7fc62d4af640 1 -- 172.21.15.120:0/4054697439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc628101470 msgr2=0x7fc6281983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:48.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:48.065+0000 7fc62d4af640 1 --2- 172.21.15.120:0/4054697439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc628101470 0x7fc6281983b0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fc61800db80 tx=0x7fc61800b630 comp rx=0 tx=0).stop 2024-09-19T09:21:48.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:48.065+0000 7fc62d4af640 1 -- 172.21.15.120:0/4054697439 shutdown_connections 2024-09-19T09:21:48.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:48.065+0000 7fc62d4af640 1 --2- 172.21.15.120:0/4054697439 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc6100a3240 0x7fc6100a5700 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:48.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:48.065+0000 7fc62d4af640 1 --2- 172.21.15.120:0/4054697439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc628105de0 0x7fc62819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:48.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:48.065+0000 7fc62d4af640 1 --2- 172.21.15.120:0/4054697439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc628101e20 0x7fc6281988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:48.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:48.065+0000 7fc62d4af640 1 --2- 172.21.15.120:0/4054697439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc628101470 0x7fc6281983b0 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:48.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:48.065+0000 7fc62d4af640 1 -- 172.21.15.120:0/4054697439 >> 172.21.15.120:0/4054697439 conn(0x7fc628077230 msgr2=0x7fc628104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:48.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:48.066+0000 7fc62d4af640 1 -- 172.21.15.120:0/4054697439 shutdown_connections 2024-09-19T09:21:48.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:48.066+0000 7fc62d4af640 1 -- 172.21.15.120:0/4054697439 wait complete. 2024-09-19T09:21:48.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:48 smithi120 ceph-mon[31150]: pgmap v913: 97 pgs: 97 active+clean; 422 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 7.8 MiB/s wr, 757 op/s 2024-09-19T09:21:48.371 INFO:tasks.workunit.client.0.smithi120.stdout: 10 145261 21.65 MB/sec execute 223 sec latency 1030.046 ms 2024-09-19T09:21:48.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:48 smithi196 ceph-mon[35150]: pgmap v913: 97 pgs: 97 active+clean; 422 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 7.8 MiB/s wr, 757 op/s 2024-09-19T09:21:48.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:48 smithi153 ceph-mon[35071]: pgmap v913: 97 pgs: 97 active+clean; 422 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 7.8 MiB/s wr, 757 op/s 2024-09-19T09:21:48.956 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c69603ab', quiesce took 4.9 seconds. Will release after: 5.1 2024-09-19T09:21:49.371 INFO:tasks.workunit.client.0.smithi120.stdout: 10 145261 21.55 MB/sec execute 224 sec latency 2030.077 ms 2024-09-19T09:21:50.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:50 smithi120 ceph-mon[31150]: pgmap v914: 97 pgs: 97 active+clean; 432 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 14 MiB/s wr, 891 op/s 2024-09-19T09:21:50.371 INFO:tasks.workunit.client.0.smithi120.stdout: 10 145261 21.45 MB/sec execute 225 sec latency 3030.104 ms 2024-09-19T09:21:50.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:50 smithi196 ceph-mon[35150]: pgmap v914: 97 pgs: 97 active+clean; 432 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 14 MiB/s wr, 891 op/s 2024-09-19T09:21:50.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:50 smithi153 ceph-mon[35071]: pgmap v914: 97 pgs: 97 active+clean; 432 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 14 MiB/s wr, 891 op/s 2024-09-19T09:21:51.371 INFO:tasks.workunit.client.0.smithi120.stdout: 10 145261 21.36 MB/sec execute 226 sec latency 4030.186 ms 2024-09-19T09:21:52.329 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:52 smithi120 ceph-mon[31150]: pgmap v915: 97 pgs: 97 active+clean; 410 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 15 MiB/s wr, 902 op/s 2024-09-19T09:21:52.371 INFO:tasks.workunit.client.0.smithi120.stdout: 10 145261 21.26 MB/sec execute 227 sec latency 5030.284 ms 2024-09-19T09:21:52.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:52 smithi196 ceph-mon[35150]: pgmap v915: 97 pgs: 97 active+clean; 410 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 15 MiB/s wr, 902 op/s 2024-09-19T09:21:52.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:52 smithi153 ceph-mon[35071]: pgmap v915: 97 pgs: 97 active+clean; 410 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 15 MiB/s wr, 902 op/s 2024-09-19T09:21:53.371 INFO:tasks.workunit.client.0.smithi120.stdout: 10 145261 21.17 MB/sec execute 228 sec latency 6030.397 ms 2024-09-19T09:21:54.057 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:21:54.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:54 smithi120 ceph-mon[31150]: pgmap v916: 97 pgs: 97 active+clean; 389 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 16 MiB/s wr, 882 op/s 2024-09-19T09:21:54.345 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:21:54.371 INFO:tasks.workunit.client.0.smithi120.stdout: 10 145261 21.08 MB/sec execute 229 sec latency 7030.467 ms 2024-09-19T09:21:54.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:54 smithi196 ceph-mon[35150]: pgmap v916: 97 pgs: 97 active+clean; 389 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 16 MiB/s wr, 882 op/s 2024-09-19T09:21:54.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:54 smithi153 ceph-mon[35071]: pgmap v916: 97 pgs: 97 active+clean; 389 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 16 MiB/s wr, 882 op/s 2024-09-19T09:21:54.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.845+0000 7fbbff09d640 1 -- 172.21.15.120:0/144636801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbf8100e20 msgr2=0x7fbbf8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:54.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.845+0000 7fbbff09d640 1 --2- 172.21.15.120:0/144636801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbf8100e20 0x7fbbf8101200 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009900 tx=0x7fbbec02a480 comp rx=0 tx=0).stop 2024-09-19T09:21:54.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.845+0000 7fbbff09d640 1 -- 172.21.15.120:0/144636801 shutdown_connections 2024-09-19T09:21:54.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.845+0000 7fbbff09d640 1 --2- 172.21.15.120:0/144636801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbf8102ef0 0x7fbbf810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:54.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.845+0000 7fbbff09d640 1 --2- 172.21.15.120:0/144636801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbf8102570 0x7fbbf81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:54.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.845+0000 7fbbff09d640 1 --2- 172.21.15.120:0/144636801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbf8100e20 0x7fbbf8101200 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:54.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.845+0000 7fbbff09d640 1 -- 172.21.15.120:0/144636801 >> 172.21.15.120:0/144636801 conn(0x7fbbf80fc7d0 msgr2=0x7fbbf80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:54.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.845+0000 7fbbff09d640 1 -- 172.21.15.120:0/144636801 shutdown_connections 2024-09-19T09:21:54.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.845+0000 7fbbff09d640 1 -- 172.21.15.120:0/144636801 wait complete. 2024-09-19T09:21:54.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.846+0000 7fbbff09d640 1 Processor -- start 2024-09-19T09:21:54.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.846+0000 7fbbff09d640 1 -- start start 2024-09-19T09:21:54.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.846+0000 7fbbff09d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbf8100e20 0x7fbbf80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:54.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.846+0000 7fbbff09d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbf8102570 0x7fbbf80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:54.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.846+0000 7fbbff09d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbf8102ef0 0x7fbbf80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:54.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.846+0000 7fbbff09d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf8110f80 con 0x7fbbf8102ef0 2024-09-19T09:21:54.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.846+0000 7fbbff09d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf8110e00 con 0x7fbbf8102570 2024-09-19T09:21:54.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.846+0000 7fbbff09d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf8111100 con 0x7fbbf8100e20 2024-09-19T09:21:54.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.846+0000 7fbbfd63a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbf8102ef0 0x7fbbf80fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:54.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.846+0000 7fbbfd63a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbf8102ef0 0x7fbbf80fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42426/0 (socket says 172.21.15.120:42426) 2024-09-19T09:21:54.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.846+0000 7fbbfd63a640 1 -- 172.21.15.120:0/2366523841 learned_addr learned my addr 172.21.15.120:0/2366523841 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:54.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.846+0000 7fbbfce39640 1 --2- 172.21.15.120:0/2366523841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbf8100e20 0x7fbbf80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:54.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.846+0000 7fbbf7fff640 1 --2- 172.21.15.120:0/2366523841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbf8102570 0x7fbbf80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:54.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.847+0000 7fbbfd63a640 1 -- 172.21.15.120:0/2366523841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbf8100e20 msgr2=0x7fbbf80fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:54.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.847+0000 7fbbfd63a640 1 --2- 172.21.15.120:0/2366523841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbf8100e20 0x7fbbf80fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:54.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.847+0000 7fbbfd63a640 1 -- 172.21.15.120:0/2366523841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbf8102570 msgr2=0x7fbbf80fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:54.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.847+0000 7fbbfd63a640 1 --2- 172.21.15.120:0/2366523841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbf8102570 0x7fbbf80fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:54.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.847+0000 7fbbfd63a640 1 -- 172.21.15.120:0/2366523841 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf81a6a60 con 0x7fbbf8102ef0 2024-09-19T09:21:54.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.847+0000 7fbbfd63a640 1 --2- 172.21.15.120:0/2366523841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbf8102ef0 0x7fbbf80fd700 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000eb10 tx=0x7fbbf000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:54.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.847+0000 7fbbf5ffb640 1 -- 172.21.15.120:0/2366523841 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0004b20 con 0x7fbbf8102ef0 2024-09-19T09:21:54.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.847+0000 7fbbff09d640 1 -- 172.21.15.120:0/2366523841 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf81a6d50 con 0x7fbbf8102ef0 2024-09-19T09:21:54.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.847+0000 7fbbf5ffb640 1 -- 172.21.15.120:0/2366523841 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbf0004cc0 con 0x7fbbf8102ef0 2024-09-19T09:21:54.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.847+0000 7fbbf5ffb640 1 -- 172.21.15.120:0/2366523841 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0010760 con 0x7fbbf8102ef0 2024-09-19T09:21:54.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.847+0000 7fbbff09d640 1 -- 172.21.15.120:0/2366523841 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf81a7260 con 0x7fbbf8102ef0 2024-09-19T09:21:54.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.848+0000 7fbbff09d640 1 -- 172.21.15.120:0/2366523841 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbf8101200 con 0x7fbbf8102ef0 2024-09-19T09:21:54.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.849+0000 7fbbf5ffb640 1 -- 172.21.15.120:0/2366523841 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fbbf0005020 con 0x7fbbf8102ef0 2024-09-19T09:21:54.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.850+0000 7fbbf5ffb640 1 --2- 172.21.15.120:0/2366523841 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbbe80a3240 0x7fbbe80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:54.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.850+0000 7fbbfce39640 1 --2- 172.21.15.120:0/2366523841 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbbe80a3240 0x7fbbe80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:54.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.850+0000 7fbbf5ffb640 1 -- 172.21.15.120:0/2366523841 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fbbf00cff40 con 0x7fbbf8102ef0 2024-09-19T09:21:54.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.850+0000 7fbbfce39640 1 --2- 172.21.15.120:0/2366523841 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbbe80a3240 0x7fbbe80a5700 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009230 tx=0x7fbbec0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:54.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:54.854+0000 7fbbf5ffb640 1 -- 172.21.15.120:0/2366523841 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbbf009cc60 con 0x7fbbf8102ef0 2024-09-19T09:21:55.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:55.055+0000 7fbbff09d640 1 -- 172.21.15.120:0/2366523841 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbbf81a74f0 con 0x7fbbf8102ef0 2024-09-19T09:21:55.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:55.056+0000 7fbbf5ffb640 1 -- 172.21.15.120:0/2366523841 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11988 (secure 0 0 0) 0x7fbbf009c560 con 0x7fbbf8102ef0 2024-09-19T09:21:55.057 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:21:55.057 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":39,"btime":"2024-09-19T09:18:53:120858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:18:53.070639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:21:55.058 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 39 2024-09-19T09:21:55.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:55.058+0000 7fbbff09d640 1 -- 172.21.15.120:0/2366523841 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbbe80a3240 msgr2=0x7fbbe80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:55.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:55.058+0000 7fbbff09d640 1 --2- 172.21.15.120:0/2366523841 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbbe80a3240 0x7fbbe80a5700 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009230 tx=0x7fbbec0058d0 comp rx=0 tx=0).stop 2024-09-19T09:21:55.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:55.058+0000 7fbbff09d640 1 -- 172.21.15.120:0/2366523841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbf8102ef0 msgr2=0x7fbbf80fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:55.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:55.058+0000 7fbbff09d640 1 --2- 172.21.15.120:0/2366523841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbf8102ef0 0x7fbbf80fd700 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000eb10 tx=0x7fbbf000c2d0 comp rx=0 tx=0).stop 2024-09-19T09:21:55.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:55.058+0000 7fbbff09d640 1 -- 172.21.15.120:0/2366523841 shutdown_connections 2024-09-19T09:21:55.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:55.058+0000 7fbbff09d640 1 --2- 172.21.15.120:0/2366523841 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbbe80a3240 0x7fbbe80a5700 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:55.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:55.058+0000 7fbbff09d640 1 --2- 172.21.15.120:0/2366523841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbf8102ef0 0x7fbbf80fd700 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:55.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:55.058+0000 7fbbff09d640 1 --2- 172.21.15.120:0/2366523841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbf8102570 0x7fbbf80fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:55.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:55.058+0000 7fbbff09d640 1 --2- 172.21.15.120:0/2366523841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbf8100e20 0x7fbbf80fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:55.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:55.058+0000 7fbbff09d640 1 -- 172.21.15.120:0/2366523841 >> 172.21.15.120:0/2366523841 conn(0x7fbbf80fc7d0 msgr2=0x7fbbf8069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:55.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:55.058+0000 7fbbff09d640 1 -- 172.21.15.120:0/2366523841 shutdown_connections 2024-09-19T09:21:55.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:55.058+0000 7fbbff09d640 1 -- 172.21.15.120:0/2366523841 wait complete. 2024-09-19T09:21:55.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:55 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2366523841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:21:55.371 INFO:tasks.workunit.client.0.smithi120.stdout: 10 145261 20.99 MB/sec execute 230 sec latency 8030.571 ms 2024-09-19T09:21:55.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:55 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2366523841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:21:55.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:55 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2366523841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:21:55.762 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db --set-id c69603ab --release --await' 2024-09-19T09:21:55.763 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db --set-id c69603ab --release --await 2024-09-19T09:21:56.021 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:21:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.034+0000 7f37dfdd6640 1 -- 172.21.15.120:0/3766619281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37d8104f40 msgr2=0x7f37d8107330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.034+0000 7f37dfdd6640 1 --2- 172.21.15.120:0/3766619281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37d8104f40 0x7f37d8107330 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f37c8009900 tx=0x7f37c802a410 comp rx=0 tx=0).stop 2024-09-19T09:21:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.034+0000 7f37dfdd6640 1 -- 172.21.15.120:0/3766619281 shutdown_connections 2024-09-19T09:21:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.034+0000 7f37dfdd6640 1 --2- 172.21.15.120:0/3766619281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37d8107870 0x7f37d8109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.034+0000 7f37dfdd6640 1 --2- 172.21.15.120:0/3766619281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37d8104f40 0x7f37d8107330 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.034+0000 7f37dfdd6640 1 --2- 172.21.15.120:0/3766619281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37d8102610 0x7f37d8104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.034+0000 7f37dfdd6640 1 -- 172.21.15.120:0/3766619281 >> 172.21.15.120:0/3766619281 conn(0x7f37d80fc1f0 msgr2=0x7f37d80fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.034+0000 7f37dfdd6640 1 -- 172.21.15.120:0/3766619281 shutdown_connections 2024-09-19T09:21:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.034+0000 7f37dfdd6640 1 -- 172.21.15.120:0/3766619281 wait complete. 2024-09-19T09:21:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.034+0000 7f37dfdd6640 1 Processor -- start 2024-09-19T09:21:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.035+0000 7f37dfdd6640 1 -- start start 2024-09-19T09:21:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.035+0000 7f37dfdd6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37d8102610 0x7f37d8197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.035+0000 7f37dfdd6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37d8104f40 0x7f37d81983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.035+0000 7f37dfdd6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37d8107870 0x7f37d819f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.035+0000 7f37dfdd6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f37d810d7f0 con 0x7f37d8104f40 2024-09-19T09:21:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.035+0000 7f37dfdd6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f37d810d670 con 0x7f37d8107870 2024-09-19T09:21:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.035+0000 7f37dfdd6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37d810d970 con 0x7f37d8102610 2024-09-19T09:21:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.035+0000 7f37dd371640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37d8104f40 0x7f37d81983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.035+0000 7f37dd371640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37d8104f40 0x7f37d81983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42452/0 (socket says 172.21.15.120:42452) 2024-09-19T09:21:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.035+0000 7f37dd371640 1 -- 172.21.15.120:0/3924644241 learned_addr learned my addr 172.21.15.120:0/3924644241 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.035+0000 7f37de373640 1 --2- 172.21.15.120:0/3924644241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37d8107870 0x7f37d819f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.035+0000 7f37ddb72640 1 --2- 172.21.15.120:0/3924644241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37d8102610 0x7f37d8197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.035+0000 7f37dd371640 1 -- 172.21.15.120:0/3924644241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37d8102610 msgr2=0x7f37d8197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.035+0000 7f37dd371640 1 --2- 172.21.15.120:0/3924644241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37d8102610 0x7f37d8197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.035+0000 7f37dd371640 1 -- 172.21.15.120:0/3924644241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37d8107870 msgr2=0x7f37d819f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.035+0000 7f37dd371640 1 --2- 172.21.15.120:0/3924644241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37d8107870 0x7f37d819f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.035+0000 7f37dd371640 1 -- 172.21.15.120:0/3924644241 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d819fa90 con 0x7f37d8104f40 2024-09-19T09:21:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.035+0000 7f37dd371640 1 --2- 172.21.15.120:0/3924644241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37d8104f40 0x7f37d81983b0 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f37c8006010 tx=0x7f37c8031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.035+0000 7f37ceffd640 1 -- 172.21.15.120:0/3924644241 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c803f030 con 0x7f37d8104f40 2024-09-19T09:21:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.035+0000 7f37ceffd640 1 -- 172.21.15.120:0/3924644241 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37c80044a0 con 0x7f37d8104f40 2024-09-19T09:21:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.035+0000 7f37dfdd6640 1 -- 172.21.15.120:0/3924644241 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37d819fd20 con 0x7f37d8104f40 2024-09-19T09:21:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.035+0000 7f37ceffd640 1 -- 172.21.15.120:0/3924644241 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c8035070 con 0x7f37d8104f40 2024-09-19T09:21:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.036+0000 7f37dfdd6640 1 -- 172.21.15.120:0/3924644241 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37d81a01c0 con 0x7f37d8104f40 2024-09-19T09:21:56.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.038+0000 7f37ceffd640 1 -- 172.21.15.120:0/3924644241 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f37c8042030 con 0x7f37d8104f40 2024-09-19T09:21:56.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.038+0000 7f37e2c52740 1 Processor -- start 2024-09-19T09:21:56.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.039+0000 7f37e2c52740 1 -- start start 2024-09-19T09:21:56.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.039+0000 7f37e2c52740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644f72dd9a0 0x5644f72ddda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.039+0000 7f37e2c52740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644f72de2e0 0x5644f72e2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.039+0000 7f37e2c52740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644f72e3370 0x5644f72e5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.039+0000 7f37e2c52740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5644f72412f0 con 0x5644f72de2e0 2024-09-19T09:21:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.039+0000 7f37e2c52740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5644f71e9270 con 0x5644f72dd9a0 2024-09-19T09:21:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.039+0000 7f37e2c52740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5644f723d2b0 con 0x5644f72e3370 2024-09-19T09:21:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.039+0000 7f37ddb72640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644f72de2e0 0x5644f72e2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.039+0000 7f37dd371640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644f72e3370 0x5644f72e5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.039+0000 7f37ddb72640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644f72de2e0 0x5644f72e2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42456/0 (socket says 172.21.15.120:42456) 2024-09-19T09:21:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.039+0000 7f37ddb72640 1 -- 172.21.15.120:0/2108438742 learned_addr learned my addr 172.21.15.120:0/2108438742 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.039+0000 7f37de373640 1 --2- 172.21.15.120:0/2108438742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644f72dd9a0 0x5644f72ddda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.040+0000 7f37ddb72640 1 -- 172.21.15.120:0/2108438742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644f72e3370 msgr2=0x5644f72e5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.040+0000 7f37ddb72640 1 --2- 172.21.15.120:0/2108438742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644f72e3370 0x5644f72e5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.040+0000 7f37ddb72640 1 -- 172.21.15.120:0/2108438742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644f72dd9a0 msgr2=0x5644f72ddda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.040+0000 7f37ddb72640 1 --2- 172.21.15.120:0/2108438742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644f72dd9a0 0x5644f72ddda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.040+0000 7f37ceffd640 1 --2- 172.21.15.120:0/3924644241 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f37c40a32f0 0x7f37c40a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.040+0000 7f37ddb72640 1 -- 172.21.15.120:0/2108438742 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644f71e8340 con 0x5644f72de2e0 2024-09-19T09:21:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.040+0000 7f37dd371640 1 --2- 172.21.15.120:0/2108438742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644f72e3370 0x5644f72e5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:21:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.040+0000 7f37ddb72640 1 --2- 172.21.15.120:0/2108438742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644f72de2e0 0x5644f72e2da0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f37d400bcb0 tx=0x7f37d400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4bb397a0cc235f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.040+0000 7f37dd371640 1 --2- 172.21.15.120:0/3924644241 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f37c40a32f0 0x7f37c40a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.040+0000 7f37ceffd640 1 -- 172.21.15.120:0/3924644241 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f37c80efed0 con 0x7f37d8104f40 2024-09-19T09:21:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.040+0000 7f37c3fff640 1 -- 172.21.15.120:0/2108438742 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d4015070 con 0x5644f72de2e0 2024-09-19T09:21:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.040+0000 7f37c3fff640 1 -- 172.21.15.120:0/2108438742 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37d401a940 con 0x5644f72de2e0 2024-09-19T09:21:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.040+0000 7f37c3fff640 1 -- 172.21.15.120:0/2108438742 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d4011aa0 con 0x5644f72de2e0 2024-09-19T09:21:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.041+0000 7f37e2c52740 1 -- 172.21.15.120:0/2108438742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644f72de2e0 msgr2=0x5644f72e2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.041+0000 7f37e2c52740 1 --2- 172.21.15.120:0/2108438742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644f72de2e0 0x5644f72e2da0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f37d400bcb0 tx=0x7f37d400f730 comp rx=0 tx=0).stop 2024-09-19T09:21:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.041+0000 7f37dd371640 1 --2- 172.21.15.120:0/3924644241 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f37c40a32f0 0x7f37c40a57b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f37c80b8e80 tx=0x7f37c80f3910 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.041+0000 7f37e2c52740 1 -- 172.21.15.120:0/2108438742 shutdown_connections 2024-09-19T09:21:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.041+0000 7f37e2c52740 1 --2- 172.21.15.120:0/2108438742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644f72e3370 0x5644f72e5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.041+0000 7f37e2c52740 1 --2- 172.21.15.120:0/2108438742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644f72de2e0 0x5644f72e2da0 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.041+0000 7f37e2c52740 1 --2- 172.21.15.120:0/2108438742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644f72dd9a0 0x5644f72ddda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.041+0000 7f37e2c52740 1 -- 172.21.15.120:0/2108438742 >> 172.21.15.120:0/2108438742 conn(0x5644f72d9010 msgr2=0x5644f72d9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.041+0000 7f37e2c52740 1 -- 172.21.15.120:0/2108438742 shutdown_connections 2024-09-19T09:21:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.041+0000 7f37e2c52740 1 -- 172.21.15.120:0/2108438742 wait complete. 2024-09-19T09:21:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.041+0000 7f37e2c52740 1 -- start start 2024-09-19T09:21:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.042+0000 7f37e2c52740 1 Processor -- start 2024-09-19T09:21:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.042+0000 7f37e2c52740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644f72dd9a0 0x5644f72f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.042+0000 7f37e2c52740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644f72de2e0 0x5644f72f9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.042+0000 7f37e2c52740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644f72e3370 0x5644f73001b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.042+0000 7f37e2c52740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5644f72f7350 con 0x5644f72dd9a0 2024-09-19T09:21:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.042+0000 7f37e2c52740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5644f71ca290 con 0x5644f72de2e0 2024-09-19T09:21:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.042+0000 7f37e2c52740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5644f72f74d0 con 0x5644f72e3370 2024-09-19T09:21:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.042+0000 7f37e2c52740 10 client.0 _ll_register_callbacks cb 0x5644f71fb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.042+0000 7f37de373640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644f72dd9a0 0x5644f72f8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.042+0000 7f37de373640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644f72dd9a0 0x5644f72f8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42466/0 (socket says 172.21.15.120:42466) 2024-09-19T09:21:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.042+0000 7f37de373640 1 -- 172.21.15.120:0/2940738519 learned_addr learned my addr 172.21.15.120:0/2940738519 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.042+0000 7f37ddb72640 1 --2- 172.21.15.120:0/2940738519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644f72de2e0 0x5644f72f9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.042+0000 7f37de373640 1 -- 172.21.15.120:0/2940738519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644f72e3370 msgr2=0x5644f73001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.042+0000 7f37de373640 1 --2- 172.21.15.120:0/2940738519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644f72e3370 0x5644f73001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.042+0000 7f37de373640 1 -- 172.21.15.120:0/2940738519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644f72de2e0 msgr2=0x5644f72f9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.042+0000 7f37de373640 1 --2- 172.21.15.120:0/2940738519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644f72de2e0 0x5644f72f9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.042+0000 7f37de373640 1 -- 172.21.15.120:0/2940738519 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644f6f54210 con 0x5644f72dd9a0 2024-09-19T09:21:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.043+0000 7f37de373640 1 --2- 172.21.15.120:0/2940738519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644f72dd9a0 0x5644f72f8c30 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f37d00127e0 tx=0x7f37d0012ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1c846d6e04bacd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.043+0000 7f37c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:21:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.043+0000 7f37c17fa640 1 -- 172.21.15.120:0/2940738519 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0010d60 con 0x5644f72dd9a0 2024-09-19T09:21:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.043+0000 7f37c17fa640 1 -- 172.21.15.120:0/2940738519 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f37d0004b20 con 0x5644f72dd9a0 2024-09-19T09:21:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.043+0000 7f37e2c52740 1 -- 172.21.15.120:0/2940738519 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37d401a940 con 0x5644f72dd9a0 2024-09-19T09:21:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.043+0000 7f37c17fa640 1 -- 172.21.15.120:0/2940738519 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f37d0004d20 con 0x5644f72dd9a0 2024-09-19T09:21:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.043+0000 7f37e2c52740 10 client.21759 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.043+0000 7f37e2c52740 1 -- 172.21.15.120:0/2940738519 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5644f7020620 con 0x5644f72dd9a0 2024-09-19T09:21:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.043+0000 7f37c17fa640 1 -- 172.21.15.120:0/2940738519 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37d401a940 con 0x5644f72dd9a0 2024-09-19T09:21:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.043+0000 7f37c17fa640 1 -- 172.21.15.120:0/2940738519 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f37d000f040 con 0x5644f72dd9a0 2024-09-19T09:21:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.043+0000 7f37e2c52740 10 client.21759 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.043+0000 7f37e2c52740 10 client.21759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:21:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.043+0000 7f37e2c52740 20 client.21759 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.043+0000 7f37e2c52740 1 --2- 172.21.15.120:0/2940738519 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5644f7304900 0x5644f7306d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.043+0000 7f37e2c52740 4 client.21759 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:21:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.043+0000 7f37e2c52740 1 -- 172.21.15.120:0/2940738519 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644f724ea20 con 0x5644f7304900 2024-09-19T09:21:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.044+0000 7f37ddb72640 1 --2- 172.21.15.120:0/2940738519 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5644f7304900 0x5644f7306d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.044+0000 7f37c17fa640 10 client.21759 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.044+0000 7f37ddb72640 1 --2- 172.21.15.120:0/2940738519 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5644f7304900 0x5644f7306d10 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eac731214bbbd97f server_cookie=47e1512993ab02d3 in_seq=0 out_seq=0 2024-09-19T09:21:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.045+0000 7f37c17fa640 1 -- 172.21.15.120:0/2940738519 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5644f724ea20 con 0x5644f7304900 2024-09-19T09:21:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.045+0000 7f37c17fa640 10 client.21759 handle_command_reply: tid=0 2024-09-19T09:21:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.045+0000 7f37e2c52740 1 client.21759 shutdown 2024-09-19T09:21:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.045+0000 7f37e2c52740 1 -- 172.21.15.120:0/2940738519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644f72dd9a0 msgr2=0x5644f72f8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.045+0000 7f37e2c52740 1 --2- 172.21.15.120:0/2940738519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644f72dd9a0 0x5644f72f8c30 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f37d00127e0 tx=0x7f37d0012ca0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.045+0000 7f37e2c52740 1 -- 172.21.15.120:0/2940738519 shutdown_connections 2024-09-19T09:21:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.045+0000 7f37e2c52740 1 --2- 172.21.15.120:0/2940738519 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5644f7304900 0x5644f7306d10 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.045+0000 7f37e2c52740 1 --2- 172.21.15.120:0/2940738519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644f72e3370 0x5644f73001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.045+0000 7f37c17fa640 0 client.21759 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.045+0000 7f37e2c52740 1 --2- 172.21.15.120:0/2940738519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644f72de2e0 0x5644f72f9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.045+0000 7f37e2c52740 1 --2- 172.21.15.120:0/2940738519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644f72dd9a0 0x5644f72f8c30 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.045+0000 7f37e2c52740 1 -- 172.21.15.120:0/2940738519 >> 172.21.15.120:0/2940738519 conn(0x5644f72d9010 msgr2=0x5644f72da820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.045+0000 7f37e2c52740 1 -- 172.21.15.120:0/2940738519 shutdown_connections 2024-09-19T09:21:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.045+0000 7f37e2c52740 1 -- 172.21.15.120:0/2940738519 wait complete. 2024-09-19T09:21:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.045+0000 7f37e2c52740 20 client.21759 trim_cache size 0 max 16384 2024-09-19T09:21:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.060+0000 7f37e2c52740 1 Processor -- start 2024-09-19T09:21:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.060+0000 7f37e2c52740 1 -- start start 2024-09-19T09:21:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.060+0000 7f37e2c52740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644f72dd9a0 0x5644f7306cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.060+0000 7f37e2c52740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644f72de2e0 0x5644f7300140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.060+0000 7f37e2c52740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644f72e3370 0x5644f72d9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.060+0000 7f37e2c52740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f37d81a0850 con 0x5644f72de2e0 2024-09-19T09:21:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.060+0000 7f37e2c52740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f37d81a2390 con 0x5644f72dd9a0 2024-09-19T09:21:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.060+0000 7f37e2c52740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37d81a0f20 con 0x5644f72e3370 2024-09-19T09:21:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.060+0000 7f37ddb72640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644f72de2e0 0x5644f7300140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.060+0000 7f37ddb72640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644f72de2e0 0x5644f7300140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42480/0 (socket says 172.21.15.120:42480) 2024-09-19T09:21:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.060+0000 7f37ddb72640 1 -- 172.21.15.120:0/2400711818 learned_addr learned my addr 172.21.15.120:0/2400711818 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.060+0000 7f37de373640 1 --2- 172.21.15.120:0/2400711818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644f72dd9a0 0x5644f7306cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.060+0000 7f37ddb72640 1 -- 172.21.15.120:0/2400711818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644f72e3370 msgr2=0x5644f72d9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.060+0000 7f37ddb72640 1 --2- 172.21.15.120:0/2400711818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644f72e3370 0x5644f72d9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.060+0000 7f37ddb72640 1 -- 172.21.15.120:0/2400711818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644f72dd9a0 msgr2=0x5644f7306cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.060+0000 7f37ddb72640 1 --2- 172.21.15.120:0/2400711818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644f72dd9a0 0x5644f7306cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.060+0000 7f37ddb72640 1 -- 172.21.15.120:0/2400711818 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644f72d8e30 con 0x5644f72de2e0 2024-09-19T09:21:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.061+0000 7f37ddb72640 1 --2- 172.21.15.120:0/2400711818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644f72de2e0 0x5644f7300140 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f37d400fc30 tx=0x7f37d400b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9940ecbd85ebd005 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.061+0000 7f37c37fe640 1 -- 172.21.15.120:0/2400711818 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d4015040 con 0x5644f72de2e0 2024-09-19T09:21:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.061+0000 7f37c37fe640 1 -- 172.21.15.120:0/2400711818 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37d4005520 con 0x5644f72de2e0 2024-09-19T09:21:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.061+0000 7f37c37fe640 1 -- 172.21.15.120:0/2400711818 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d4004680 con 0x5644f72de2e0 2024-09-19T09:21:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.062+0000 7f37e2c52740 1 -- 172.21.15.120:0/2400711818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644f72de2e0 msgr2=0x5644f7300140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.062+0000 7f37e2c52740 1 --2- 172.21.15.120:0/2400711818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644f72de2e0 0x5644f7300140 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f37d400fc30 tx=0x7f37d400b5a0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.062+0000 7f37e2c52740 1 -- 172.21.15.120:0/2400711818 shutdown_connections 2024-09-19T09:21:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.062+0000 7f37e2c52740 1 --2- 172.21.15.120:0/2400711818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644f72e3370 0x5644f72d9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.062+0000 7f37e2c52740 1 --2- 172.21.15.120:0/2400711818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644f72de2e0 0x5644f7300140 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.062+0000 7f37e2c52740 1 --2- 172.21.15.120:0/2400711818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644f72dd9a0 0x5644f7306cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.062+0000 7f37e2c52740 1 -- 172.21.15.120:0/2400711818 >> 172.21.15.120:0/2400711818 conn(0x5644f72d9010 msgr2=0x5644f72f89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.062+0000 7f37e2c52740 1 -- 172.21.15.120:0/2400711818 shutdown_connections 2024-09-19T09:21:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.062+0000 7f37e2c52740 1 -- 172.21.15.120:0/2400711818 wait complete. 2024-09-19T09:21:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.062+0000 7f37e2c52740 1 -- start start 2024-09-19T09:21:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.062+0000 7f37e2c52740 1 Processor -- start 2024-09-19T09:21:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.062+0000 7f37e2c52740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644f72dd9a0 0x5644f72fd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.062+0000 7f37e2c52740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644f72de2e0 0x5644f72fd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.062+0000 7f37e2c52740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644f72e3370 0x5644f72fe6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.062+0000 7f37e2c52740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f37d81a1cc0 con 0x5644f72e3370 2024-09-19T09:21:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.062+0000 7f37e2c52740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f37d81a15f0 con 0x5644f72de2e0 2024-09-19T09:21:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.062+0000 7f37e2c52740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37d8072840 con 0x5644f72dd9a0 2024-09-19T09:21:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.062+0000 7f37e2c52740 10 client.0 _ll_register_callbacks cb 0x7f37d0009a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.062+0000 7f37de373640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644f72e3370 0x5644f72fe6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.063+0000 7f37ddb72640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644f72de2e0 0x5644f72fd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.063+0000 7f37de373640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644f72e3370 0x5644f72fe6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42482/0 (socket says 172.21.15.120:42482) 2024-09-19T09:21:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.063+0000 7f37de373640 1 -- 172.21.15.120:0/962237418 learned_addr learned my addr 172.21.15.120:0/962237418 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.063+0000 7f37de373640 1 --2- 172.21.15.120:0/962237418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644f72dd9a0 0x5644f72fd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.063+0000 7f37ddb72640 1 -- 172.21.15.120:0/962237418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644f72dd9a0 msgr2=0x5644f72fd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.063+0000 7f37ddb72640 1 --2- 172.21.15.120:0/962237418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644f72dd9a0 0x5644f72fd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.063+0000 7f37ddb72640 1 -- 172.21.15.120:0/962237418 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644f72e3370 msgr2=0x5644f72fe6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.063+0000 7f37ddb72640 1 --2- 172.21.15.120:0/962237418 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644f72e3370 0x5644f72fe6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.063+0000 7f37ddb72640 1 -- 172.21.15.120:0/962237418 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644f72fecf0 con 0x5644f72de2e0 2024-09-19T09:21:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.063+0000 7f37de373640 1 --2- 172.21.15.120:0/962237418 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644f72e3370 0x5644f72fe6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:21:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.063+0000 7f37ddb72640 1 --2- 172.21.15.120:0/962237418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644f72de2e0 0x5644f72fd6d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x5644f72f9930 tx=0x7f37d400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e69b98b448420cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.063+0000 7f37c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:21:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.063+0000 7f37c17fa640 1 -- 172.21.15.120:0/962237418 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d4015070 con 0x5644f72de2e0 2024-09-19T09:21:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.063+0000 7f37e2c52740 1 -- 172.21.15.120:0/962237418 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37d4005520 con 0x5644f72de2e0 2024-09-19T09:21:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.063+0000 7f37c17fa640 1 -- 172.21.15.120:0/962237418 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f37d4011680 con 0x5644f72de2e0 2024-09-19T09:21:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.063+0000 7f37c17fa640 1 -- 172.21.15.120:0/962237418 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f37d403f050 con 0x5644f72de2e0 2024-09-19T09:21:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.064+0000 7f37e2c52740 10 client.30590 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.064+0000 7f37e2c52740 1 -- 172.21.15.120:0/962237418 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5644f7300000 con 0x5644f72de2e0 2024-09-19T09:21:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.064+0000 7f37c17fa640 1 -- 172.21.15.120:0/962237418 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37d4005520 con 0x5644f72de2e0 2024-09-19T09:21:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.064+0000 7f37c17fa640 1 -- 172.21.15.120:0/962237418 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f37d4003bf0 con 0x5644f72de2e0 2024-09-19T09:21:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.064+0000 7f37e2c52740 10 client.30590 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.064+0000 7f37e2c52740 10 client.30590 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:21:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.064+0000 7f37e2c52740 20 client.30590 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.064+0000 7f37e2c52740 1 --2- 172.21.15.120:0/962237418 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5644f736f6b0 0x5644f7371aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.064+0000 7f37e2c52740 4 client.30590 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:21:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.064+0000 7f37e2c52740 1 -- 172.21.15.120:0/962237418 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5644f7248d70 con 0x5644f736f6b0 2024-09-19T09:21:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.064+0000 7f37de373640 1 --2- 172.21.15.120:0/962237418 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5644f736f6b0 0x5644f7371aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.065+0000 7f37c17fa640 10 client.30590 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.065+0000 7f37de373640 1 --2- 172.21.15.120:0/962237418 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5644f736f6b0 0x5644f7371aa0 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c4079bfa8f4ea9 server_cookie=5cefa9f0813e96f2 in_seq=0 out_seq=0 2024-09-19T09:21:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.223+0000 7f37c17fa640 1 -- 172.21.15.120:0/962237418 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5644f7248d70 con 0x5644f736f6b0 2024-09-19T09:21:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.223+0000 7f37c17fa640 10 client.30590 handle_command_reply: tid=0 2024-09-19T09:21:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.223+0000 7f37e2c52740 1 client.30590 shutdown 2024-09-19T09:21:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.223+0000 7f37e2c52740 1 -- 172.21.15.120:0/962237418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644f72de2e0 msgr2=0x5644f72fd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.223+0000 7f37e2c52740 1 --2- 172.21.15.120:0/962237418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644f72de2e0 0x5644f72fd6d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x5644f72f9930 tx=0x7f37d400f730 comp rx=0 tx=0).stop 2024-09-19T09:21:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.223+0000 7f37e2c52740 1 -- 172.21.15.120:0/962237418 shutdown_connections 2024-09-19T09:21:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.223+0000 7f37e2c52740 1 --2- 172.21.15.120:0/962237418 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5644f736f6b0 0x5644f7371aa0 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.223+0000 7f37e2c52740 1 --2- 172.21.15.120:0/962237418 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644f72e3370 0x5644f72fe6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.223+0000 7f37c17fa640 0 client.30590 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.223+0000 7f37e2c52740 1 --2- 172.21.15.120:0/962237418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644f72de2e0 0x5644f72fd6d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.223+0000 7f37e2c52740 1 --2- 172.21.15.120:0/962237418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644f72dd9a0 0x5644f72fd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.223+0000 7f37e2c52740 1 -- 172.21.15.120:0/962237418 >> 172.21.15.120:0/962237418 conn(0x5644f72d9010 msgr2=0x5644f72d79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.223+0000 7f37e2c52740 1 -- 172.21.15.120:0/962237418 shutdown_connections 2024-09-19T09:21:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.223+0000 7f37e2c52740 1 -- 172.21.15.120:0/962237418 wait complete. 2024-09-19T09:21:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.223+0000 7f37e2c52740 20 client.30590 trim_cache size 0 max 16384 2024-09-19T09:21:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:21:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:21:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:21:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:21:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.224+0000 7f37dfdd6640 1 -- 172.21.15.120:0/3924644241 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f37c40a32f0 msgr2=0x7f37c40a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.224+0000 7f37dfdd6640 1 --2- 172.21.15.120:0/3924644241 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f37c40a32f0 0x7f37c40a57b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f37c80b8e80 tx=0x7f37c80f3910 comp rx=0 tx=0).stop 2024-09-19T09:21:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.224+0000 7f37dfdd6640 1 -- 172.21.15.120:0/3924644241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37d8104f40 msgr2=0x7f37d81983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.224+0000 7f37dfdd6640 1 --2- 172.21.15.120:0/3924644241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37d8104f40 0x7f37d81983b0 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f37c8006010 tx=0x7f37c8031b60 comp rx=0 tx=0).stop 2024-09-19T09:21:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.224+0000 7f37dfdd6640 1 -- 172.21.15.120:0/3924644241 shutdown_connections 2024-09-19T09:21:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.224+0000 7f37dfdd6640 1 --2- 172.21.15.120:0/3924644241 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f37c40a32f0 0x7f37c40a57b0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.224+0000 7f37dfdd6640 1 --2- 172.21.15.120:0/3924644241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37d8107870 0x7f37d819f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.224+0000 7f37dfdd6640 1 --2- 172.21.15.120:0/3924644241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37d8104f40 0x7f37d81983b0 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.224+0000 7f37dfdd6640 1 --2- 172.21.15.120:0/3924644241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37d8102610 0x7f37d8197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.224+0000 7f37dfdd6640 1 -- 172.21.15.120:0/3924644241 >> 172.21.15.120:0/3924644241 conn(0x7f37d80fc1f0 msgr2=0x7f37d8103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.225+0000 7f37dfdd6640 1 -- 172.21.15.120:0/3924644241 shutdown_connections 2024-09-19T09:21:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:21:56.225+0000 7f37dfdd6640 1 -- 172.21.15.120:0/3924644241 wait complete. 2024-09-19T09:21:56.372 INFO:tasks.workunit.client.0.smithi120.stdout: 10 145261 20.90 MB/sec execute 231 sec latency 9030.617 ms 2024-09-19T09:21:56.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:56 smithi196 ceph-mon[35150]: pgmap v917: 97 pgs: 97 active+clean; 388 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 15 MiB/s wr, 706 op/s 2024-09-19T09:21:56.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:56 smithi153 ceph-mon[35071]: pgmap v917: 97 pgs: 97 active+clean; 388 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 15 MiB/s wr, 706 op/s 2024-09-19T09:21:56.454 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:56 smithi120 ceph-mon[31150]: pgmap v917: 97 pgs: 97 active+clean; 388 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 15 MiB/s wr, 706 op/s 2024-09-19T09:21:56.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.541+0000 7f82abff3640 1 -- 172.21.15.120:0/4269913953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82a4101450 msgr2=0x7f82a4105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:56.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.541+0000 7f82abff3640 1 --2- 172.21.15.120:0/4269913953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82a4101450 0x7f82a4105920 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f8294009900 tx=0x7f829402a470 comp rx=0 tx=0).stop 2024-09-19T09:21:56.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.541+0000 7f82abff3640 1 -- 172.21.15.120:0/4269913953 shutdown_connections 2024-09-19T09:21:56.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.541+0000 7f82abff3640 1 --2- 172.21.15.120:0/4269913953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82a4105f40 0x7f82a41063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.541+0000 7f82abff3640 1 --2- 172.21.15.120:0/4269913953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82a4101450 0x7f82a4105920 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.541+0000 7f82abff3640 1 --2- 172.21.15.120:0/4269913953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82a4100b30 0x7f82a4100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.541+0000 7f82abff3640 1 -- 172.21.15.120:0/4269913953 >> 172.21.15.120:0/4269913953 conn(0x7f82a40fc7a0 msgr2=0x7f82a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:56.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.541+0000 7f82abff3640 1 -- 172.21.15.120:0/4269913953 shutdown_connections 2024-09-19T09:21:56.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.541+0000 7f82abff3640 1 -- 172.21.15.120:0/4269913953 wait complete. 2024-09-19T09:21:56.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.542+0000 7f82abff3640 1 Processor -- start 2024-09-19T09:21:56.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.542+0000 7f82abff3640 1 -- start start 2024-09-19T09:21:56.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.542+0000 7f82abff3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82a4100b30 0x7f82a41a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.542+0000 7f82abff3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82a4101450 0x7f82a41a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.542+0000 7f82abff3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82a4105f40 0x7f82a419e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.542+0000 7f82abff3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f82a4118560 con 0x7f82a4105f40 2024-09-19T09:21:56.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.542+0000 7f82abff3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41183e0 con 0x7f82a4100b30 2024-09-19T09:21:56.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.542+0000 7f82abff3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41186e0 con 0x7f82a4101450 2024-09-19T09:21:56.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.542+0000 7f82aa590640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82a4105f40 0x7f82a419e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.542+0000 7f82aa590640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82a4105f40 0x7f82a419e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42496/0 (socket says 172.21.15.120:42496) 2024-09-19T09:21:56.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.542+0000 7f82aa590640 1 -- 172.21.15.120:0/2436039098 learned_addr learned my addr 172.21.15.120:0/2436039098 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:56.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.542+0000 7f82a958e640 1 --2- 172.21.15.120:0/2436039098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82a4101450 0x7f82a41a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.542+0000 7f82a9d8f640 1 --2- 172.21.15.120:0/2436039098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82a4100b30 0x7f82a41a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.542+0000 7f82aa590640 1 -- 172.21.15.120:0/2436039098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82a4101450 msgr2=0x7f82a41a4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:56.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.542+0000 7f82aa590640 1 --2- 172.21.15.120:0/2436039098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82a4101450 0x7f82a41a4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.542+0000 7f82aa590640 1 -- 172.21.15.120:0/2436039098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82a4100b30 msgr2=0x7f82a41a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:56.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.542+0000 7f82aa590640 1 --2- 172.21.15.120:0/2436039098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82a4100b30 0x7f82a41a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.542+0000 7f82aa590640 1 -- 172.21.15.120:0/2436039098 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a419f040 con 0x7f82a4105f40 2024-09-19T09:21:56.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.543+0000 7f82aa590640 1 --2- 172.21.15.120:0/2436039098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82a4105f40 0x7f82a419e910 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f829c00eaa0 tx=0x7f829c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:56.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.543+0000 7f829affd640 1 -- 172.21.15.120:0/2436039098 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c00cdb0 con 0x7f82a4105f40 2024-09-19T09:21:56.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.543+0000 7f82abff3640 1 -- 172.21.15.120:0/2436039098 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a419f330 con 0x7f82a4105f40 2024-09-19T09:21:56.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.543+0000 7f829affd640 1 -- 172.21.15.120:0/2436039098 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f829c005020 con 0x7f82a4105f40 2024-09-19T09:21:56.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.543+0000 7f829affd640 1 -- 172.21.15.120:0/2436039098 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c0116b0 con 0x7f82a4105f40 2024-09-19T09:21:56.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.543+0000 7f82abff3640 1 -- 172.21.15.120:0/2436039098 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a41ab620 con 0x7f82a4105f40 2024-09-19T09:21:56.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.545+0000 7f829affd640 1 -- 172.21.15.120:0/2436039098 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f829c005560 con 0x7f82a4105f40 2024-09-19T09:21:56.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.545+0000 7f82ae3cc740 1 Processor -- start 2024-09-19T09:21:56.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.545+0000 7f82ae3cc740 1 -- start start 2024-09-19T09:21:56.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.546+0000 7f82ae3cc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dc1af060 0x5653dc1af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.546+0000 7f82ae3cc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5653dc1af980 0x5653dc1b9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.546+0000 7f82ae3cc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653dc1ba240 0x5653dc1bc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.546+0000 7f82ae3cc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5653dc114620 con 0x5653dc1af060 2024-09-19T09:21:56.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.546+0000 7f82ae3cc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5653dc0bbe50 con 0x5653dc1af980 2024-09-19T09:21:56.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.546+0000 7f82ae3cc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5653dc1105e0 con 0x5653dc1ba240 2024-09-19T09:21:56.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.546+0000 7f829affd640 1 --2- 172.21.15.120:0/2436039098 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f82900a3240 0x7f82900a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.546+0000 7f82a958e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dc1af060 0x5653dc1af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.546+0000 7f82a958e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dc1af060 0x5653dc1af440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42500/0 (socket says 172.21.15.120:42500) 2024-09-19T09:21:56.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.546+0000 7f82a958e640 1 -- 172.21.15.120:0/3034584877 learned_addr learned my addr 172.21.15.120:0/3034584877 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:56.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.546+0000 7f82aa590640 1 --2- 172.21.15.120:0/2436039098 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f82900a3240 0x7f82900a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.546+0000 7f82a9d8f640 1 --2- 172.21.15.120:0/3034584877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5653dc1af980 0x5653dc1b9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.546+0000 7f82a958e640 1 --2- 172.21.15.120:0/3034584877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653dc1ba240 0x5653dc1bc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.546+0000 7f829affd640 1 -- 172.21.15.120:0/2436039098 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f829c0cfa50 con 0x7f82a4105f40 2024-09-19T09:21:56.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.546+0000 7f82a958e640 1 -- 172.21.15.120:0/3034584877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653dc1ba240 msgr2=0x5653dc1bc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:56.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.546+0000 7f82a958e640 1 --2- 172.21.15.120:0/3034584877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653dc1ba240 0x5653dc1bc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.546+0000 7f82a958e640 1 -- 172.21.15.120:0/3034584877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5653dc1af980 msgr2=0x5653dc1b9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:56.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.546+0000 7f82a958e640 1 --2- 172.21.15.120:0/3034584877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5653dc1af980 0x5653dc1b9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.546+0000 7f82a958e640 1 -- 172.21.15.120:0/3034584877 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653dc0acd40 con 0x5653dc1af060 2024-09-19T09:21:56.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.546+0000 7f82a958e640 1 --2- 172.21.15.120:0/3034584877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dc1af060 0x5653dc1af440 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x5653dc1ac970 tx=0x7f8294000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa0b04b3eabb58c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:56.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.546+0000 7f82aa590640 1 --2- 172.21.15.120:0/2436039098 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f82900a3240 0x7f82900a5700 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f829c089660 tx=0x7f829c06efa0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:56.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.547+0000 7f828ffff640 1 -- 172.21.15.120:0/3034584877 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294044030 con 0x5653dc1af060 2024-09-19T09:21:56.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.547+0000 7f828ffff640 1 -- 172.21.15.120:0/3034584877 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8294005020 con 0x5653dc1af060 2024-09-19T09:21:56.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.547+0000 7f828ffff640 1 -- 172.21.15.120:0/3034584877 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294035040 con 0x5653dc1af060 2024-09-19T09:21:56.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.547+0000 7f82ae3cc740 1 -- 172.21.15.120:0/3034584877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dc1af060 msgr2=0x5653dc1af440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:56.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.547+0000 7f82ae3cc740 1 --2- 172.21.15.120:0/3034584877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dc1af060 0x5653dc1af440 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x5653dc1ac970 tx=0x7f8294000bc0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.547+0000 7f82ae3cc740 1 -- 172.21.15.120:0/3034584877 shutdown_connections 2024-09-19T09:21:56.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.547+0000 7f82ae3cc740 1 --2- 172.21.15.120:0/3034584877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653dc1ba240 0x5653dc1bc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.547+0000 7f82ae3cc740 1 --2- 172.21.15.120:0/3034584877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5653dc1af980 0x5653dc1b9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.547+0000 7f82ae3cc740 1 --2- 172.21.15.120:0/3034584877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dc1af060 0x5653dc1af440 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.547+0000 7f82ae3cc740 1 -- 172.21.15.120:0/3034584877 >> 172.21.15.120:0/3034584877 conn(0x5653dc1ab5f0 msgr2=0x5653dc1ab9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:56.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.547+0000 7f82ae3cc740 1 -- 172.21.15.120:0/3034584877 shutdown_connections 2024-09-19T09:21:56.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.547+0000 7f82ae3cc740 1 -- 172.21.15.120:0/3034584877 wait complete. 2024-09-19T09:21:56.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.548+0000 7f82ae3cc740 1 -- start start 2024-09-19T09:21:56.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.548+0000 7f82ae3cc740 1 Processor -- start 2024-09-19T09:21:56.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.548+0000 7f82ae3cc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dc1af060 0x5653dc1b7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.548+0000 7f82ae3cc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653dc1af980 0x5653dc1b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.548+0000 7f82ae3cc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5653dc1ba240 0x5653dc1b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.548+0000 7f82ae3cc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5653dc1bce10 con 0x5653dc1af060 2024-09-19T09:21:56.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.548+0000 7f82ae3cc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5653dc09cf10 con 0x5653dc1ba240 2024-09-19T09:21:56.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.548+0000 7f82ae3cc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5653dc1bcf90 con 0x5653dc1af980 2024-09-19T09:21:56.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.548+0000 7f82ae3cc740 10 client.0 _ll_register_callbacks cb 0x5653dc0c0db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:56.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.548+0000 7f82a958e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dc1af060 0x5653dc1b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.548+0000 7f82a958e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dc1af060 0x5653dc1b7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42516/0 (socket says 172.21.15.120:42516) 2024-09-19T09:21:56.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.548+0000 7f82a958e640 1 -- 172.21.15.120:0/2582455678 learned_addr learned my addr 172.21.15.120:0/2582455678 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:56.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.549+0000 7f82a9d8f640 1 --2- 172.21.15.120:0/2582455678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653dc1af980 0x5653dc1b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.549+0000 7f82a958e640 1 --2- 172.21.15.120:0/2582455678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5653dc1ba240 0x5653dc1b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.549+0000 7f82a958e640 1 -- 172.21.15.120:0/2582455678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653dc1af980 msgr2=0x5653dc1b76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:56.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.549+0000 7f82a958e640 1 --2- 172.21.15.120:0/2582455678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653dc1af980 0x5653dc1b76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.549+0000 7f82a958e640 1 -- 172.21.15.120:0/2582455678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5653dc1ba240 msgr2=0x5653dc1b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:56.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.549+0000 7f82a958e640 1 --2- 172.21.15.120:0/2582455678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5653dc1ba240 0x5653dc1b7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.549+0000 7f82a958e640 1 -- 172.21.15.120:0/2582455678 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653dbe23580 con 0x5653dc1af060 2024-09-19T09:21:56.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.549+0000 7f82a958e640 1 --2- 172.21.15.120:0/2582455678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dc1af060 0x5653dc1b7180 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f8294031ba0 tx=0x7f8294031a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5b27856b952b11 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:56.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.549+0000 7f828d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:21:56.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.549+0000 7f828d7fa640 1 -- 172.21.15.120:0/2582455678 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294052060 con 0x5653dc1af060 2024-09-19T09:21:56.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.549+0000 7f828d7fa640 1 -- 172.21.15.120:0/2582455678 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f82940046c0 con 0x5653dc1af060 2024-09-19T09:21:56.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.549+0000 7f82ae3cc740 1 -- 172.21.15.120:0/2582455678 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8294005020 con 0x5653dc1af060 2024-09-19T09:21:56.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.549+0000 7f828d7fa640 1 -- 172.21.15.120:0/2582455678 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8294035070 con 0x5653dc1af060 2024-09-19T09:21:56.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.549+0000 7f82ae3cc740 10 client.21777 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:56.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.549+0000 7f82ae3cc740 1 -- 172.21.15.120:0/2582455678 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5653dbeed960 con 0x5653dc1af060 2024-09-19T09:21:56.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.549+0000 7f828d7fa640 1 -- 172.21.15.120:0/2582455678 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8294005020 con 0x5653dc1af060 2024-09-19T09:21:56.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.550+0000 7f828d7fa640 1 -- 172.21.15.120:0/2582455678 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f82940588d0 con 0x5653dc1af060 2024-09-19T09:21:56.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.550+0000 7f82ae3cc740 10 client.21777 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:56.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.550+0000 7f82ae3cc740 10 client.21777 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:21:56.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.550+0000 7f82ae3cc740 20 client.21777 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:56.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.550+0000 7f82ae3cc740 1 --2- 172.21.15.120:0/2582455678 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5653dc1c9cc0 0x5653dc1cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.550+0000 7f82ae3cc740 4 client.21777 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:21:56.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.550+0000 7f82ae3cc740 1 -- 172.21.15.120:0/2582455678 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653dc121b90 con 0x5653dc1c9cc0 2024-09-19T09:21:56.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.550+0000 7f82a9d8f640 1 --2- 172.21.15.120:0/2582455678 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5653dc1c9cc0 0x5653dc1cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.550+0000 7f828d7fa640 10 client.21777 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:56.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.550+0000 7f82a9d8f640 1 --2- 172.21.15.120:0/2582455678 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5653dc1c9cc0 0x5653dc1cc0b0 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4c9d6ed0b5734bc server_cookie=7fe89825ff19386e in_seq=0 out_seq=0 2024-09-19T09:21:56.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.551+0000 7f828d7fa640 1 -- 172.21.15.120:0/2582455678 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5653dc121b90 con 0x5653dc1c9cc0 2024-09-19T09:21:56.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.551+0000 7f828d7fa640 10 client.21777 handle_command_reply: tid=0 2024-09-19T09:21:56.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.551+0000 7f82ae3cc740 1 client.21777 shutdown 2024-09-19T09:21:56.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.551+0000 7f82ae3cc740 1 -- 172.21.15.120:0/2582455678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dc1af060 msgr2=0x5653dc1b7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:56.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.551+0000 7f82ae3cc740 1 --2- 172.21.15.120:0/2582455678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dc1af060 0x5653dc1b7180 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f8294031ba0 tx=0x7f8294031a80 comp rx=0 tx=0).stop 2024-09-19T09:21:56.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.551+0000 7f82ae3cc740 1 -- 172.21.15.120:0/2582455678 shutdown_connections 2024-09-19T09:21:56.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.551+0000 7f82ae3cc740 1 --2- 172.21.15.120:0/2582455678 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5653dc1c9cc0 0x5653dc1cc0b0 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.551+0000 7f82ae3cc740 1 --2- 172.21.15.120:0/2582455678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5653dc1ba240 0x5653dc1b7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.551+0000 7f828d7fa640 0 client.21777 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:56.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.551+0000 7f82ae3cc740 1 --2- 172.21.15.120:0/2582455678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653dc1af980 0x5653dc1b76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.551+0000 7f82ae3cc740 1 --2- 172.21.15.120:0/2582455678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dc1af060 0x5653dc1b7180 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.551+0000 7f82ae3cc740 1 -- 172.21.15.120:0/2582455678 >> 172.21.15.120:0/2582455678 conn(0x5653dc1ab5f0 msgr2=0x5653dc1b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:56.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.551+0000 7f82ae3cc740 1 -- 172.21.15.120:0/2582455678 shutdown_connections 2024-09-19T09:21:56.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.551+0000 7f82ae3cc740 1 -- 172.21.15.120:0/2582455678 wait complete. 2024-09-19T09:21:56.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.551+0000 7f82ae3cc740 20 client.21777 trim_cache size 0 max 16384 2024-09-19T09:21:56.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.569+0000 7f82ae3cc740 1 Processor -- start 2024-09-19T09:21:56.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.569+0000 7f82ae3cc740 1 -- start start 2024-09-19T09:21:56.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.569+0000 7f82ae3cc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5653dc1af060 0x5653dc1aa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.569+0000 7f82ae3cc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653dc1af980 0x5653dc1b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.569+0000 7f82ae3cc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dc1ba240 0x5653dc1b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.569+0000 7f82ae3cc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41b3bd0 con 0x5653dc1ba240 2024-09-19T09:21:56.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.569+0000 7f82ae3cc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41b5710 con 0x5653dc1af060 2024-09-19T09:21:56.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.569+0000 7f82ae3cc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41b42a0 con 0x5653dc1af980 2024-09-19T09:21:56.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.569+0000 7f82a958e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dc1ba240 0x5653dc1b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.569+0000 7f82a958e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dc1ba240 0x5653dc1b4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42524/0 (socket says 172.21.15.120:42524) 2024-09-19T09:21:56.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.569+0000 7f82a958e640 1 -- 172.21.15.120:0/2305387616 learned_addr learned my addr 172.21.15.120:0/2305387616 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:56.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.569+0000 7f82a9d8f640 1 --2- 172.21.15.120:0/2305387616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653dc1af980 0x5653dc1b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.569+0000 7f82a958e640 1 --2- 172.21.15.120:0/2305387616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5653dc1af060 0x5653dc1aa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.569+0000 7f82a958e640 1 -- 172.21.15.120:0/2305387616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653dc1af980 msgr2=0x5653dc1b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:56.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.569+0000 7f82a958e640 1 --2- 172.21.15.120:0/2305387616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653dc1af980 0x5653dc1b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.569+0000 7f82a958e640 1 -- 172.21.15.120:0/2305387616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5653dc1af060 msgr2=0x5653dc1aa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:56.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.570+0000 7f82a958e640 1 --2- 172.21.15.120:0/2305387616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5653dc1af060 0x5653dc1aa640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.570+0000 7f82a958e640 1 -- 172.21.15.120:0/2305387616 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653dc1ac280 con 0x5653dc1ba240 2024-09-19T09:21:56.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.570+0000 7f82a958e640 1 --2- 172.21.15.120:0/2305387616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dc1ba240 0x5653dc1b4920 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f82940096c0 tx=0x7f8294000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c3f899d4df1d04 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:56.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.570+0000 7f828f7fe640 1 -- 172.21.15.120:0/2305387616 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294052040 con 0x5653dc1ba240 2024-09-19T09:21:56.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.570+0000 7f828f7fe640 1 -- 172.21.15.120:0/2305387616 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82940317f0 con 0x5653dc1ba240 2024-09-19T09:21:56.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.570+0000 7f828f7fe640 1 -- 172.21.15.120:0/2305387616 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294044040 con 0x5653dc1ba240 2024-09-19T09:21:56.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.570+0000 7f82ae3cc740 1 -- 172.21.15.120:0/2305387616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dc1ba240 msgr2=0x5653dc1b4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:56.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.570+0000 7f82ae3cc740 1 --2- 172.21.15.120:0/2305387616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dc1ba240 0x5653dc1b4920 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f82940096c0 tx=0x7f8294000bc0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.571+0000 7f82ae3cc740 1 -- 172.21.15.120:0/2305387616 shutdown_connections 2024-09-19T09:21:56.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.571+0000 7f82ae3cc740 1 --2- 172.21.15.120:0/2305387616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dc1ba240 0x5653dc1b4920 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.571+0000 7f82ae3cc740 1 --2- 172.21.15.120:0/2305387616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653dc1af980 0x5653dc1b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.571+0000 7f82ae3cc740 1 --2- 172.21.15.120:0/2305387616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5653dc1af060 0x5653dc1aa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.571+0000 7f82ae3cc740 1 -- 172.21.15.120:0/2305387616 >> 172.21.15.120:0/2305387616 conn(0x5653dc1ab5f0 msgr2=0x5653dc1a9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:56.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.571+0000 7f82ae3cc740 1 -- 172.21.15.120:0/2305387616 shutdown_connections 2024-09-19T09:21:56.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.571+0000 7f82ae3cc740 1 -- 172.21.15.120:0/2305387616 wait complete. 2024-09-19T09:21:56.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.571+0000 7f82ae3cc740 1 -- start start 2024-09-19T09:21:56.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.571+0000 7f82ae3cc740 1 Processor -- start 2024-09-19T09:21:56.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.571+0000 7f82ae3cc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dc1af060 0x5653dc1cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.571+0000 7f82ae3cc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653dc1af980 0x5653dc1cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.571+0000 7f82ae3cc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5653dc1ba240 0x5653dc1cd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.571+0000 7f82ae3cc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41b5040 con 0x5653dc1af060 2024-09-19T09:21:56.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.571+0000 7f82ae3cc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41b4970 con 0x5653dc1ba240 2024-09-19T09:21:56.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.571+0000 7f82ae3cc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41b1b90 con 0x5653dc1af980 2024-09-19T09:21:56.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.571+0000 7f82ae3cc740 10 client.0 _ll_register_callbacks cb 0x7f82940315b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:21:56.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.572+0000 7f82a958e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dc1af060 0x5653dc1cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.572+0000 7f82a958e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dc1af060 0x5653dc1cf0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42526/0 (socket says 172.21.15.120:42526) 2024-09-19T09:21:56.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.572+0000 7f82a9d8f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653dc1af980 0x5653dc1cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.572+0000 7f82a958e640 1 -- 172.21.15.120:0/3851823368 learned_addr learned my addr 172.21.15.120:0/3851823368 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:21:56.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.572+0000 7f82a958e640 1 --2- 172.21.15.120:0/3851823368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5653dc1ba240 0x5653dc1cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.572+0000 7f82a9d8f640 1 -- 172.21.15.120:0/3851823368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5653dc1ba240 msgr2=0x5653dc1cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:56.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.572+0000 7f82a9d8f640 1 --2- 172.21.15.120:0/3851823368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5653dc1ba240 0x5653dc1cd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.572+0000 7f82a9d8f640 1 -- 172.21.15.120:0/3851823368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dc1af060 msgr2=0x5653dc1cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:21:56.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.572+0000 7f82a9d8f640 1 --2- 172.21.15.120:0/3851823368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dc1af060 0x5653dc1cf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.572+0000 7f82a9d8f640 1 -- 172.21.15.120:0/3851823368 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653dc1cdda0 con 0x5653dc1af980 2024-09-19T09:21:56.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.572+0000 7f82a958e640 1 --2- 172.21.15.120:0/3851823368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dc1af060 0x5653dc1cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:21:56.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.572+0000 7f82a9d8f640 1 --2- 172.21.15.120:0/3851823368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653dc1af980 0x5653dc1cf640 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x5653dc1b87c0 tx=0x7f82a00039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84c1f4efa069af73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:56.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.572+0000 7f828d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:21:56.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.572+0000 7f828d7fa640 1 -- 172.21.15.120:0/3851823368 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a0005220 con 0x5653dc1af980 2024-09-19T09:21:56.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.572+0000 7f82ae3cc740 1 -- 172.21.15.120:0/3851823368 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82940317f0 con 0x5653dc1af980 2024-09-19T09:21:56.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.572+0000 7f828d7fa640 1 -- 172.21.15.120:0/3851823368 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f82a00053c0 con 0x5653dc1af980 2024-09-19T09:21:56.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.572+0000 7f828d7fa640 1 -- 172.21.15.120:0/3851823368 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f82a00055c0 con 0x5653dc1af980 2024-09-19T09:21:56.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.572+0000 7f82ae3cc740 10 client.30439 fetch_fsmap learned FSMap version 39 2024-09-19T09:21:56.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.572+0000 7f82ae3cc740 1 -- 172.21.15.120:0/3851823368 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5653dc235190 con 0x5653dc1af980 2024-09-19T09:21:56.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.573+0000 7f828d7fa640 1 -- 172.21.15.120:0/3851823368 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82940317f0 con 0x5653dc1af980 2024-09-19T09:21:56.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.573+0000 7f828d7fa640 1 -- 172.21.15.120:0/3851823368 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f82a002f050 con 0x5653dc1af980 2024-09-19T09:21:56.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.573+0000 7f82ae3cc740 10 client.30439 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:21:56.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.573+0000 7f82ae3cc740 10 client.30439 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:21:56.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.573+0000 7f82ae3cc740 20 client.30439 populate_metadata read hostname 'smithi120' 2024-09-19T09:21:56.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.573+0000 7f82ae3cc740 1 --2- 172.21.15.120:0/3851823368 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5653dc235910 0x5653dc237d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:56.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.573+0000 7f82ae3cc740 4 client.30439 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c69603ab", "release": true, "await": true}] 2024-09-19T09:21:56.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.573+0000 7f82ae3cc740 1 -- 172.21.15.120:0/3851823368 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c69603ab", "release": true, "await": true}) -- 0x5653dc11c0a0 con 0x5653dc235910 2024-09-19T09:21:56.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.573+0000 7f82a958e640 1 --2- 172.21.15.120:0/3851823368 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5653dc235910 0x5653dc237d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:56.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.573+0000 7f82a958e640 1 --2- 172.21.15.120:0/3851823368 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5653dc235910 0x5653dc237d30 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c24efd0c9660668 server_cookie=c7fdadb01f38a281 in_seq=0 out_seq=0 2024-09-19T09:21:56.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.573+0000 7f828d7fa640 10 client.30439 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:56.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.640+0000 7f828d7fa640 1 -- 172.21.15.120:0/3851823368 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5653dc11c0a0 con 0x5653dc235910 2024-09-19T09:21:56.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.640+0000 7f828d7fa640 10 client.30439 handle_command_reply: tid=0 2024-09-19T09:21:56.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.640+0000 7f82ae3cc740 1 client.30439 shutdown 2024-09-19T09:21:56.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.641+0000 7f82ae3cc740 1 -- 172.21.15.120:0/3851823368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653dc1af980 msgr2=0x5653dc1cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:56.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.641+0000 7f82ae3cc740 1 --2- 172.21.15.120:0/3851823368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653dc1af980 0x5653dc1cf640 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x5653dc1b87c0 tx=0x7f82a00039c0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.641+0000 7f82ae3cc740 1 -- 172.21.15.120:0/3851823368 shutdown_connections 2024-09-19T09:21:56.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.641+0000 7f82ae3cc740 1 --2- 172.21.15.120:0/3851823368 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5653dc235910 0x5653dc237d30 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.641+0000 7f82ae3cc740 1 --2- 172.21.15.120:0/3851823368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5653dc1ba240 0x5653dc1cd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.641+0000 7f82ae3cc740 1 --2- 172.21.15.120:0/3851823368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653dc1af980 0x5653dc1cf640 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.641+0000 7f828d7fa640 0 client.30439 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:21:56.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.641+0000 7f82ae3cc740 1 --2- 172.21.15.120:0/3851823368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dc1af060 0x5653dc1cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.641+0000 7f82ae3cc740 1 -- 172.21.15.120:0/3851823368 >> 172.21.15.120:0/3851823368 conn(0x5653dc1ab5f0 msgr2=0x5653dc1c5a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:56.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.641+0000 7f82ae3cc740 1 -- 172.21.15.120:0/3851823368 shutdown_connections 2024-09-19T09:21:56.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.641+0000 7f82ae3cc740 1 -- 172.21.15.120:0/3851823368 wait complete. 2024-09-19T09:21:56.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.641+0000 7f82ae3cc740 20 client.30439 trim_cache size 0 max 16384 2024-09-19T09:21:56.643 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:21:56.643 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 39, 2024-09-19T09:21:56.643 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:21:56.643 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 64, 2024-09-19T09:21:56.644 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:21:56.644 INFO:teuthology.orchestra.run.smithi120.stdout: "c69603ab": { 2024-09-19T09:21:56.644 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 64, 2024-09-19T09:21:56.644 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:21:56.644 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:21:56.644 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:21:56.644 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:21:56.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:21:56.644 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:21:56.644 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 130.0, 2024-09-19T09:21:56.644 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:21:56.644 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:21:56.644 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:21:56.645 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:21:56.645 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:21:56.645 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:21:56.645 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:21:56.645 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:21:56.645 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:21:56.645 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:21:56.645 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:21:56.645 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:21:56.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.641+0000 7f82abff3640 1 -- 172.21.15.120:0/2436039098 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f82900a3240 msgr2=0x7f82900a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:56.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.641+0000 7f82abff3640 1 --2- 172.21.15.120:0/2436039098 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f82900a3240 0x7f82900a5700 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f829c089660 tx=0x7f829c06efa0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.641+0000 7f82abff3640 1 -- 172.21.15.120:0/2436039098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82a4105f40 msgr2=0x7f82a419e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:56.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.641+0000 7f82abff3640 1 --2- 172.21.15.120:0/2436039098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82a4105f40 0x7f82a419e910 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f829c00eaa0 tx=0x7f829c00ef60 comp rx=0 tx=0).stop 2024-09-19T09:21:56.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.642+0000 7f82abff3640 1 -- 172.21.15.120:0/2436039098 shutdown_connections 2024-09-19T09:21:56.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.642+0000 7f82abff3640 1 --2- 172.21.15.120:0/2436039098 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f82900a3240 0x7f82900a5700 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.642+0000 7f82abff3640 1 --2- 172.21.15.120:0/2436039098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82a4105f40 0x7f82a419e910 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.642+0000 7f82abff3640 1 --2- 172.21.15.120:0/2436039098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82a4101450 0x7f82a41a4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.642+0000 7f82abff3640 1 --2- 172.21.15.120:0/2436039098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82a4100b30 0x7f82a41a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:56.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.642+0000 7f82abff3640 1 -- 172.21.15.120:0/2436039098 >> 172.21.15.120:0/2436039098 conn(0x7f82a40fc7a0 msgr2=0x7f82a40fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:56.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.642+0000 7f82abff3640 1 -- 172.21.15.120:0/2436039098 shutdown_connections 2024-09-19T09:21:56.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:21:56.642+0000 7f82abff3640 1 -- 172.21.15.120:0/2436039098 wait complete. 2024-09-19T09:21:57.371 INFO:tasks.workunit.client.0.smithi120.stdout: 10 145801 20.97 MB/sec execute 232 sec latency 9311.575 ms 2024-09-19T09:21:57.384 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c69603ab', total seconds elapsed: 13.3 2024-09-19T09:21:58.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:21:58 smithi120 ceph-mon[31150]: pgmap v918: 97 pgs: 97 active+clean; 374 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 15 MiB/s wr, 676 op/s 2024-09-19T09:21:58.372 INFO:tasks.workunit.client.0.smithi120.stdout: 10 146442 21.05 MB/sec execute 233 sec latency 57.979 ms 2024-09-19T09:21:58.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:21:58 smithi196 ceph-mon[35150]: pgmap v918: 97 pgs: 97 active+clean; 374 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 15 MiB/s wr, 676 op/s 2024-09-19T09:21:58.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:21:58 smithi153 ceph-mon[35071]: pgmap v918: 97 pgs: 97 active+clean; 374 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 15 MiB/s wr, 676 op/s 2024-09-19T09:21:59.372 INFO:tasks.workunit.client.0.smithi120.stdout: 10 147010 21.10 MB/sec execute 234 sec latency 56.016 ms 2024-09-19T09:22:00.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:00 smithi120 ceph-mon[31150]: pgmap v919: 97 pgs: 97 active+clean; 363 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 11 MiB/s wr, 547 op/s 2024-09-19T09:22:00.372 INFO:tasks.workunit.client.0.smithi120.stdout: 10 147542 21.10 MB/sec execute 235 sec latency 62.940 ms 2024-09-19T09:22:00.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:00 smithi196 ceph-mon[35150]: pgmap v919: 97 pgs: 97 active+clean; 363 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 11 MiB/s wr, 547 op/s 2024-09-19T09:22:00.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:00 smithi153 ceph-mon[35071]: pgmap v919: 97 pgs: 97 active+clean; 363 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 11 MiB/s wr, 547 op/s 2024-09-19T09:22:01.372 INFO:tasks.workunit.client.0.smithi120.stdout: 10 148054 21.12 MB/sec execute 236 sec latency 139.350 ms 2024-09-19T09:22:02.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:02 smithi120 ceph-mon[31150]: pgmap v920: 97 pgs: 97 active+clean; 368 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.1 MiB/s wr, 418 op/s 2024-09-19T09:22:02.372 INFO:tasks.workunit.client.0.smithi120.stdout: 10 148701 21.19 MB/sec execute 237 sec latency 54.075 ms 2024-09-19T09:22:02.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:02 smithi196 ceph-mon[35150]: pgmap v920: 97 pgs: 97 active+clean; 368 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.1 MiB/s wr, 418 op/s 2024-09-19T09:22:02.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:02 smithi153 ceph-mon[35071]: pgmap v920: 97 pgs: 97 active+clean; 368 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.1 MiB/s wr, 418 op/s 2024-09-19T09:22:03.372 INFO:tasks.workunit.client.0.smithi120.stdout: 10 149361 21.23 MB/sec execute 238 sec latency 60.543 ms 2024-09-19T09:22:04.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:04 smithi120 ceph-mon[31150]: pgmap v921: 97 pgs: 97 active+clean; 369 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 345 op/s 2024-09-19T09:22:04.372 INFO:tasks.workunit.client.0.smithi120.stdout: 10 150056 21.28 MB/sec execute 239 sec latency 56.375 ms 2024-09-19T09:22:04.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:04 smithi196 ceph-mon[35150]: pgmap v921: 97 pgs: 97 active+clean; 369 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 345 op/s 2024-09-19T09:22:04.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:04 smithi153 ceph-mon[35071]: pgmap v921: 97 pgs: 97 active+clean; 369 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 345 op/s 2024-09-19T09:22:05.372 INFO:tasks.workunit.client.0.smithi120.stdout: 10 150733 21.38 MB/sec execute 240 sec latency 63.829 ms 2024-09-19T09:22:06.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:06 smithi120 ceph-mon[31150]: pgmap v922: 97 pgs: 97 active+clean; 376 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.5 MiB/s wr, 342 op/s 2024-09-19T09:22:06.372 INFO:tasks.workunit.client.0.smithi120.stdout: 10 151477 21.49 MB/sec execute 241 sec latency 59.020 ms 2024-09-19T09:22:06.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:06 smithi196 ceph-mon[35150]: pgmap v922: 97 pgs: 97 active+clean; 376 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.5 MiB/s wr, 342 op/s 2024-09-19T09:22:06.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:06 smithi153 ceph-mon[35071]: pgmap v922: 97 pgs: 97 active+clean; 376 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.5 MiB/s wr, 342 op/s 2024-09-19T09:22:07.372 INFO:tasks.workunit.client.0.smithi120.stdout: 10 152063 21.51 MB/sec execute 242 sec latency 49.566 ms 2024-09-19T09:22:08.095 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:22:08.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:08 smithi120 ceph-mon[31150]: pgmap v923: 97 pgs: 97 active+clean; 380 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.7 MiB/s wr, 402 op/s 2024-09-19T09:22:08.372 INFO:tasks.workunit.client.0.smithi120.stdout: 10 152532 21.51 MB/sec execute 243 sec latency 103.093 ms 2024-09-19T09:22:08.415 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:22:08.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:08 smithi196 ceph-mon[35150]: pgmap v923: 97 pgs: 97 active+clean; 380 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.7 MiB/s wr, 402 op/s 2024-09-19T09:22:08.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:08 smithi153 ceph-mon[35071]: pgmap v923: 97 pgs: 97 active+clean; 380 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.7 MiB/s wr, 402 op/s 2024-09-19T09:22:09.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.989+0000 7faedfdb2640 1 -- 172.21.15.120:0/1259181758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faed8103670 msgr2=0x7faed810bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:09.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.989+0000 7faedfdb2640 1 --2- 172.21.15.120:0/1259181758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faed8103670 0x7faed810bb60 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7faec8009900 tx=0x7faec802a480 comp rx=0 tx=0).stop 2024-09-19T09:22:09.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.989+0000 7faedfdb2640 1 -- 172.21.15.120:0/1259181758 shutdown_connections 2024-09-19T09:22:09.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.989+0000 7faedfdb2640 1 --2- 172.21.15.120:0/1259181758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faed810c180 0x7faed810e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.989+0000 7faedfdb2640 1 --2- 172.21.15.120:0/1259181758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faed8103670 0x7faed810bb60 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.989+0000 7faedfdb2640 1 --2- 172.21.15.120:0/1259181758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faed8102d50 0x7faed8103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.989+0000 7faedfdb2640 1 -- 172.21.15.120:0/1259181758 >> 172.21.15.120:0/1259181758 conn(0x7faed80fc7d0 msgr2=0x7faed80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:09.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.990+0000 7faedfdb2640 1 -- 172.21.15.120:0/1259181758 shutdown_connections 2024-09-19T09:22:09.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.990+0000 7faedfdb2640 1 -- 172.21.15.120:0/1259181758 wait complete. 2024-09-19T09:22:09.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.990+0000 7faedfdb2640 1 Processor -- start 2024-09-19T09:22:09.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.990+0000 7faedfdb2640 1 -- start start 2024-09-19T09:22:09.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.991+0000 7faedfdb2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faed8102d50 0x7faed81984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.991+0000 7faedfdb2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faed8103670 0x7faed8198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.991+0000 7faedfdb2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faed810c180 0x7faed819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.991+0000 7faedfdb2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faed8111fd0 con 0x7faed810c180 2024-09-19T09:22:09.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.991+0000 7faedfdb2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faed8111e50 con 0x7faed8102d50 2024-09-19T09:22:09.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.991+0000 7faedfdb2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faed8112150 con 0x7faed8103670 2024-09-19T09:22:09.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.991+0000 7faede34f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faed810c180 0x7faed819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.991+0000 7faeddb4e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faed8102d50 0x7faed81984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.991+0000 7faede34f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faed810c180 0x7faed819cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39566/0 (socket says 172.21.15.120:39566) 2024-09-19T09:22:09.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.991+0000 7faede34f640 1 -- 172.21.15.120:0/2957565696 learned_addr learned my addr 172.21.15.120:0/2957565696 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:09.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.991+0000 7faedd34d640 1 --2- 172.21.15.120:0/2957565696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faed8103670 0x7faed8198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.991+0000 7faede34f640 1 -- 172.21.15.120:0/2957565696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faed8103670 msgr2=0x7faed8198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:09.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.991+0000 7faede34f640 1 --2- 172.21.15.120:0/2957565696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faed8103670 0x7faed8198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.991+0000 7faede34f640 1 -- 172.21.15.120:0/2957565696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faed8102d50 msgr2=0x7faed81984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:09.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.991+0000 7faede34f640 1 --2- 172.21.15.120:0/2957565696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faed8102d50 0x7faed81984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.991+0000 7faede34f640 1 -- 172.21.15.120:0/2957565696 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed819d480 con 0x7faed810c180 2024-09-19T09:22:09.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.991+0000 7faede34f640 1 --2- 172.21.15.120:0/2957565696 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faed810c180 0x7faed819cda0 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7faed0009fd0 tx=0x7faed000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:09.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.991+0000 7faeceffd640 1 -- 172.21.15.120:0/2957565696 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed0017070 con 0x7faed810c180 2024-09-19T09:22:09.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.991+0000 7faeceffd640 1 -- 172.21.15.120:0/2957565696 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faed0003e50 con 0x7faed810c180 2024-09-19T09:22:09.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.991+0000 7faeceffd640 1 -- 172.21.15.120:0/2957565696 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed00106c0 con 0x7faed810c180 2024-09-19T09:22:09.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.992+0000 7faedfdb2640 1 -- 172.21.15.120:0/2957565696 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed819d770 con 0x7faed810c180 2024-09-19T09:22:09.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.992+0000 7faedfdb2640 1 -- 172.21.15.120:0/2957565696 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed81a5050 con 0x7faed810c180 2024-09-19T09:22:09.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.994+0000 7faee218b740 1 Processor -- start 2024-09-19T09:22:09.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.995+0000 7faeceffd640 1 -- 172.21.15.120:0/2957565696 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7faed0003970 con 0x7faed810c180 2024-09-19T09:22:09.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.995+0000 7faee218b740 1 -- start start 2024-09-19T09:22:09.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.995+0000 7faee218b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f68fc04a0 0x564f68fc0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.995+0000 7faee218b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f68fc0dc0 0x564f68fcb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.995+0000 7faee218b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f68fcb680 0x564f68fcda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.995+0000 7faee218b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564f68f262e0 con 0x564f68fcb680 2024-09-19T09:22:09.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.995+0000 7faee218b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564f68ecdce0 con 0x564f68fc04a0 2024-09-19T09:22:09.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.995+0000 7faee218b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564f68f222a0 con 0x564f68fc0dc0 2024-09-19T09:22:09.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.995+0000 7faedd34d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f68fc04a0 0x564f68fc0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.995+0000 7faedd34d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f68fc04a0 0x564f68fc0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:39078/0 (socket says 172.21.15.120:39078) 2024-09-19T09:22:09.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.995+0000 7faedd34d640 1 -- 172.21.15.120:0/2834775314 learned_addr learned my addr 172.21.15.120:0/2834775314 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:09.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.995+0000 7faedd34d640 1 --2- 172.21.15.120:0/2834775314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f68fcb680 0x564f68fcda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.996+0000 7faeddb4e640 1 --2- 172.21.15.120:0/2834775314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f68fc0dc0 0x564f68fcb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.996+0000 7faedd34d640 1 -- 172.21.15.120:0/2834775314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f68fc0dc0 msgr2=0x564f68fcb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:09.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.996+0000 7faedd34d640 1 --2- 172.21.15.120:0/2834775314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f68fc0dc0 0x564f68fcb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.996+0000 7faedd34d640 1 -- 172.21.15.120:0/2834775314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f68fcb680 msgr2=0x564f68fcda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:09.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.996+0000 7faedd34d640 1 --2- 172.21.15.120:0/2834775314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f68fcb680 0x564f68fcda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.996+0000 7faedd34d640 1 -- 172.21.15.120:0/2834775314 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f68eb2440 con 0x564f68fc04a0 2024-09-19T09:22:09.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.996+0000 7faeceffd640 1 --2- 172.21.15.120:0/2957565696 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faec40a32c0 0x7faec40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.996+0000 7faedd34d640 1 --2- 172.21.15.120:0/2834775314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f68fc04a0 0x564f68fc0880 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7faec8005ec0 tx=0x7faec80056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46b43e87ac1ab8ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:09.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.996+0000 7faeddb4e640 1 --2- 172.21.15.120:0/2957565696 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faec40a32c0 0x7faec40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.996+0000 7faec3fff640 1 -- 172.21.15.120:0/2834775314 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec8035040 con 0x564f68fc04a0 2024-09-19T09:22:09.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.996+0000 7faeceffd640 1 -- 172.21.15.120:0/2957565696 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7faed00cf5b0 con 0x7faed810c180 2024-09-19T09:22:09.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.996+0000 7faec3fff640 1 -- 172.21.15.120:0/2834775314 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faec8003cd0 con 0x564f68fc04a0 2024-09-19T09:22:09.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.996+0000 7faec3fff640 1 -- 172.21.15.120:0/2834775314 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec803d990 con 0x564f68fc04a0 2024-09-19T09:22:09.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.997+0000 7faee218b740 1 -- 172.21.15.120:0/2834775314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f68fc04a0 msgr2=0x564f68fc0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:09.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.997+0000 7faeddb4e640 1 --2- 172.21.15.120:0/2957565696 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faec40a32c0 0x7faec40a5780 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x564f68fbd890 tx=0x7faed4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:09.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.997+0000 7faee218b740 1 --2- 172.21.15.120:0/2834775314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f68fc04a0 0x564f68fc0880 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7faec8005ec0 tx=0x7faec80056d0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.997+0000 7faee218b740 1 -- 172.21.15.120:0/2834775314 shutdown_connections 2024-09-19T09:22:09.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.997+0000 7faee218b740 1 --2- 172.21.15.120:0/2834775314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f68fcb680 0x564f68fcda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.997+0000 7faee218b740 1 --2- 172.21.15.120:0/2834775314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f68fc0dc0 0x564f68fcb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.997+0000 7faee218b740 1 --2- 172.21.15.120:0/2834775314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f68fc04a0 0x564f68fc0880 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.997+0000 7faee218b740 1 -- 172.21.15.120:0/2834775314 >> 172.21.15.120:0/2834775314 conn(0x564f68fbca30 msgr2=0x564f68fbce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:09.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.997+0000 7faee218b740 1 -- 172.21.15.120:0/2834775314 shutdown_connections 2024-09-19T09:22:09.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.997+0000 7faee218b740 1 -- 172.21.15.120:0/2834775314 wait complete. 2024-09-19T09:22:09.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.998+0000 7faee218b740 1 -- start start 2024-09-19T09:22:09.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.998+0000 7faee218b740 1 Processor -- start 2024-09-19T09:22:09.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.998+0000 7faee218b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f68fc04a0 0x564f68fcafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.998+0000 7faee218b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f68fc0dc0 0x564f68fc8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.998+0000 7faee218b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f68fcb680 0x564f68fc87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.998+0000 7faee218b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564f68fce250 con 0x564f68fcb680 2024-09-19T09:22:09.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.998+0000 7faee218b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564f68ee1050 con 0x564f68fc0dc0 2024-09-19T09:22:09.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.998+0000 7faee218b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564f68fce3d0 con 0x564f68fc04a0 2024-09-19T09:22:09.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.998+0000 7faee218b740 10 client.0 _ll_register_callbacks cb 0x564f68f21160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:22:09.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.998+0000 7faede34f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f68fcb680 0x564f68fc87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.998+0000 7faedd34d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f68fc0dc0 0x564f68fc8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.998+0000 7faeddb4e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f68fc04a0 0x564f68fcafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.998+0000 7faede34f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f68fcb680 0x564f68fc87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39576/0 (socket says 172.21.15.120:39576) 2024-09-19T09:22:09.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.998+0000 7faede34f640 1 -- 172.21.15.120:0/4143447873 learned_addr learned my addr 172.21.15.120:0/4143447873 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:09.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.998+0000 7faedd34d640 1 --2- 172.21.15.120:0/4143447873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f68fc0dc0 0x564f68fc8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:39090/0 (socket says 172.21.15.120:39090) 2024-09-19T09:22:09.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.999+0000 7faeddb4e640 1 -- 172.21.15.120:0/4143447873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f68fc0dc0 msgr2=0x564f68fc8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:09.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.999+0000 7faeddb4e640 1 --2- 172.21.15.120:0/4143447873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f68fc0dc0 0x564f68fc8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.999+0000 7faeddb4e640 1 -- 172.21.15.120:0/4143447873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f68fcb680 msgr2=0x564f68fc87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:09.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.999+0000 7faeddb4e640 1 --2- 172.21.15.120:0/4143447873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f68fcb680 0x564f68fc87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.999+0000 7faeddb4e640 1 -- 172.21.15.120:0/4143447873 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f68c973c0 con 0x564f68fc04a0 2024-09-19T09:22:09.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.999+0000 7faede34f640 1 --2- 172.21.15.120:0/4143447873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f68fcb680 0x564f68fc87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:22:09.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.999+0000 7faedd34d640 1 --2- 172.21.15.120:0/4143447873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f68fc0dc0 0x564f68fc8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:22:09.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.999+0000 7faeddb4e640 1 --2- 172.21.15.120:0/4143447873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f68fc04a0 0x564f68fcafc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7faed40045e0 tx=0x7faed4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87719afcc92858f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:09.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.999+0000 7faec17fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:22:09.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.999+0000 7faec17fa640 1 -- 172.21.15.120:0/4143447873 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed401e680 con 0x564f68fc04a0 2024-09-19T09:22:09.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.999+0000 7faec17fa640 1 -- 172.21.15.120:0/4143447873 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7faed401e820 con 0x564f68fc04a0 2024-09-19T09:22:09.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.999+0000 7faee218b740 1 -- 172.21.15.120:0/4143447873 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faec8003cd0 con 0x564f68fc04a0 2024-09-19T09:22:09.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:08.999+0000 7faec17fa640 1 -- 172.21.15.120:0/4143447873 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7faed401ea20 con 0x564f68fc04a0 2024-09-19T09:22:09.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.000+0000 7faee218b740 10 client.30451 fetch_fsmap learned FSMap version 39 2024-09-19T09:22:09.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.000+0000 7faee218b740 1 -- 172.21.15.120:0/4143447873 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564f68fd9cb0 con 0x564f68fc04a0 2024-09-19T09:22:09.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.000+0000 7faec17fa640 1 -- 172.21.15.120:0/4143447873 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faec8003cd0 con 0x564f68fc04a0 2024-09-19T09:22:09.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.000+0000 7faec17fa640 1 -- 172.21.15.120:0/4143447873 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7faed4027070 con 0x564f68fc04a0 2024-09-19T09:22:09.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.000+0000 7faee218b740 10 client.30451 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:22:09.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.000+0000 7faee218b740 10 client.30451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:22:09.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.000+0000 7faee218b740 20 client.30451 populate_metadata read hostname 'smithi120' 2024-09-19T09:22:09.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.000+0000 7faee218b740 1 --2- 172.21.15.120:0/4143447873 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564f68fdb4f0 0x564f68fdd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.000+0000 7faee218b740 4 client.30451 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:22:09.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.000+0000 7faee218b740 1 -- 172.21.15.120:0/4143447873 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f68f33850 con 0x564f68fdb4f0 2024-09-19T09:22:09.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.000+0000 7faedd34d640 1 --2- 172.21.15.120:0/4143447873 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564f68fdb4f0 0x564f68fdd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.001+0000 7faec17fa640 10 client.30451 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:09.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.001+0000 7faedd34d640 1 --2- 172.21.15.120:0/4143447873 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564f68fdb4f0 0x564f68fdd940 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ac6296204777638 server_cookie=9a3e176f5bf33e7d in_seq=0 out_seq=0 2024-09-19T09:22:09.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.003+0000 7faec17fa640 1 -- 172.21.15.120:0/4143447873 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564f68f33850 con 0x564f68fdb4f0 2024-09-19T09:22:09.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.003+0000 7faec17fa640 10 client.30451 handle_command_reply: tid=0 2024-09-19T09:22:09.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.003+0000 7faee218b740 1 client.30451 shutdown 2024-09-19T09:22:09.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.003+0000 7faee218b740 1 -- 172.21.15.120:0/4143447873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f68fc04a0 msgr2=0x564f68fcafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:09.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.003+0000 7faee218b740 1 --2- 172.21.15.120:0/4143447873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f68fc04a0 0x564f68fcafc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7faed40045e0 tx=0x7faed4004730 comp rx=0 tx=0).stop 2024-09-19T09:22:09.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.003+0000 7faee218b740 1 -- 172.21.15.120:0/4143447873 shutdown_connections 2024-09-19T09:22:09.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.003+0000 7faee218b740 1 --2- 172.21.15.120:0/4143447873 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564f68fdb4f0 0x564f68fdd940 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.003+0000 7faee218b740 1 --2- 172.21.15.120:0/4143447873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f68fcb680 0x564f68fc87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.003+0000 7faee218b740 1 --2- 172.21.15.120:0/4143447873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f68fc0dc0 0x564f68fc8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.003+0000 7faee218b740 1 --2- 172.21.15.120:0/4143447873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f68fc04a0 0x564f68fcafc0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.003+0000 7faee218b740 1 -- 172.21.15.120:0/4143447873 >> 172.21.15.120:0/4143447873 conn(0x564f68fbca30 msgr2=0x564f68fbbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:09.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.003+0000 7faec17fa640 0 client.30451 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:09.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.003+0000 7faee218b740 1 -- 172.21.15.120:0/4143447873 shutdown_connections 2024-09-19T09:22:09.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.003+0000 7faee218b740 1 -- 172.21.15.120:0/4143447873 wait complete. 2024-09-19T09:22:09.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.003+0000 7faee218b740 20 client.30451 trim_cache size 0 max 16384 2024-09-19T09:22:09.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.022+0000 7faee218b740 1 Processor -- start 2024-09-19T09:22:09.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.022+0000 7faee218b740 1 -- start start 2024-09-19T09:22:09.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.022+0000 7faee218b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f68fc04a0 0x564f68fd74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.022+0000 7faee218b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f68fc0dc0 0x564f68fc7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.022+0000 7faee218b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f68fcb680 0x564f68fc2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.022+0000 7faee218b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faed81af740 con 0x564f68fcb680 2024-09-19T09:22:09.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.022+0000 7faee218b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faed81b1280 con 0x564f68fc0dc0 2024-09-19T09:22:09.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.022+0000 7faee218b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faed81afe10 con 0x564f68fc04a0 2024-09-19T09:22:09.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.022+0000 7faede34f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f68fcb680 0x564f68fc2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.022+0000 7faeddb4e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f68fc04a0 0x564f68fd74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.023+0000 7faeddb4e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f68fc04a0 0x564f68fd74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:44638/0 (socket says 172.21.15.120:44638) 2024-09-19T09:22:09.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.023+0000 7faeddb4e640 1 -- 172.21.15.120:0/290474619 learned_addr learned my addr 172.21.15.120:0/290474619 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:09.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.023+0000 7faedd34d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f68fc0dc0 0x564f68fc7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.023+0000 7faedd34d640 1 -- 172.21.15.120:0/290474619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f68fc04a0 msgr2=0x564f68fd74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:09.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.023+0000 7faedd34d640 1 --2- 172.21.15.120:0/290474619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f68fc04a0 0x564f68fd74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.023+0000 7faedd34d640 1 -- 172.21.15.120:0/290474619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f68fcb680 msgr2=0x564f68fc2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:09.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.023+0000 7faedd34d640 1 --2- 172.21.15.120:0/290474619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f68fcb680 0x564f68fc2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.023+0000 7faedd34d640 1 -- 172.21.15.120:0/290474619 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f68fc7220 con 0x564f68fc0dc0 2024-09-19T09:22:09.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.023+0000 7faeddb4e640 1 --2- 172.21.15.120:0/290474619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f68fc04a0 0x564f68fd74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:22:09.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.023+0000 7faede34f640 1 --2- 172.21.15.120:0/290474619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f68fcb680 0x564f68fc2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:22:09.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.023+0000 7faedd34d640 1 --2- 172.21.15.120:0/290474619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f68fc0dc0 0x564f68fc7b30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7faec8006010 tx=0x7faec8009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ad48dd60605df08 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:09.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.023+0000 7faec37fe640 1 -- 172.21.15.120:0/290474619 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec8035070 con 0x564f68fc0dc0 2024-09-19T09:22:09.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.023+0000 7faec37fe640 1 -- 172.21.15.120:0/290474619 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faec8005530 con 0x564f68fc0dc0 2024-09-19T09:22:09.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.023+0000 7faec37fe640 1 -- 172.21.15.120:0/290474619 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec8004bc0 con 0x564f68fc0dc0 2024-09-19T09:22:09.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.025+0000 7faee218b740 1 -- 172.21.15.120:0/290474619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f68fc0dc0 msgr2=0x564f68fc7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:09.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.025+0000 7faee218b740 1 --2- 172.21.15.120:0/290474619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f68fc0dc0 0x564f68fc7b30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7faec8006010 tx=0x7faec8009680 comp rx=0 tx=0).stop 2024-09-19T09:22:09.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.025+0000 7faee218b740 1 -- 172.21.15.120:0/290474619 shutdown_connections 2024-09-19T09:22:09.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.025+0000 7faee218b740 1 --2- 172.21.15.120:0/290474619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f68fcb680 0x564f68fc2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.025+0000 7faee218b740 1 --2- 172.21.15.120:0/290474619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f68fc0dc0 0x564f68fc7b30 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.025+0000 7faee218b740 1 --2- 172.21.15.120:0/290474619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f68fc04a0 0x564f68fd74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.025+0000 7faee218b740 1 -- 172.21.15.120:0/290474619 >> 172.21.15.120:0/290474619 conn(0x564f68fbca30 msgr2=0x564f68fc1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:09.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.025+0000 7faee218b740 1 -- 172.21.15.120:0/290474619 shutdown_connections 2024-09-19T09:22:09.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.025+0000 7faee218b740 1 -- 172.21.15.120:0/290474619 wait complete. 2024-09-19T09:22:09.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.025+0000 7faee218b740 1 -- start start 2024-09-19T09:22:09.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.025+0000 7faee218b740 1 Processor -- start 2024-09-19T09:22:09.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.026+0000 7faee218b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f68fc04a0 0x564f68fce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.026+0000 7faee218b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f68fc0dc0 0x564f68fceb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.026+0000 7faee218b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f68fcb680 0x564f68fcf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.026+0000 7faee218b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faed81b0bb0 con 0x564f68fcb680 2024-09-19T09:22:09.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.026+0000 7faee218b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faed81b04e0 con 0x564f68fc04a0 2024-09-19T09:22:09.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.026+0000 7faee218b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faed81ad700 con 0x564f68fc0dc0 2024-09-19T09:22:09.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.026+0000 7faee218b740 10 client.0 _ll_register_callbacks cb 0x7faed4009a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:22:09.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.026+0000 7faede34f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f68fcb680 0x564f68fcf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.026+0000 7faede34f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f68fcb680 0x564f68fcf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39602/0 (socket says 172.21.15.120:39602) 2024-09-19T09:22:09.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.026+0000 7faede34f640 1 -- 172.21.15.120:0/2793764376 learned_addr learned my addr 172.21.15.120:0/2793764376 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:09.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.026+0000 7faedd34d640 1 --2- 172.21.15.120:0/2793764376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f68fc0dc0 0x564f68fceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.026+0000 7faeddb4e640 1 --2- 172.21.15.120:0/2793764376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f68fc04a0 0x564f68fce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.026+0000 7faede34f640 1 -- 172.21.15.120:0/2793764376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f68fc0dc0 msgr2=0x564f68fceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:09.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.026+0000 7faede34f640 1 --2- 172.21.15.120:0/2793764376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f68fc0dc0 0x564f68fceb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.026+0000 7faede34f640 1 -- 172.21.15.120:0/2793764376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f68fc04a0 msgr2=0x564f68fce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:09.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.026+0000 7faede34f640 1 --2- 172.21.15.120:0/2793764376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f68fc04a0 0x564f68fce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.026+0000 7faede34f640 1 -- 172.21.15.120:0/2793764376 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f69044340 con 0x564f68fcb680 2024-09-19T09:22:09.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.026+0000 7faede34f640 1 --2- 172.21.15.120:0/2793764376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f68fcb680 0x564f68fcf060 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7faed009b8d0 tx=0x7faed0097f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca105ff1c9d7b010 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:09.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.026+0000 7faec17fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:22:09.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.027+0000 7faec17fa640 1 -- 172.21.15.120:0/2793764376 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed0020040 con 0x564f68fcb680 2024-09-19T09:22:09.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.027+0000 7faec17fa640 1 -- 172.21.15.120:0/2793764376 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7faed00cf110 con 0x564f68fcb680 2024-09-19T09:22:09.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.027+0000 7faee218b740 1 -- 172.21.15.120:0/2793764376 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faec8005530 con 0x564f68fcb680 2024-09-19T09:22:09.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.027+0000 7faec17fa640 1 -- 172.21.15.120:0/2793764376 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7faed009ff80 con 0x564f68fcb680 2024-09-19T09:22:09.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.027+0000 7faee218b740 10 client.21801 fetch_fsmap learned FSMap version 39 2024-09-19T09:22:09.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.027+0000 7faee218b740 1 -- 172.21.15.120:0/2793764376 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564f69045650 con 0x564f68fcb680 2024-09-19T09:22:09.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.027+0000 7faec17fa640 1 -- 172.21.15.120:0/2793764376 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faec8005530 con 0x564f68fcb680 2024-09-19T09:22:09.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.027+0000 7faec17fa640 1 -- 172.21.15.120:0/2793764376 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7faed00861d0 con 0x564f68fcb680 2024-09-19T09:22:09.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.027+0000 7faee218b740 10 client.21801 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:22:09.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.027+0000 7faee218b740 10 client.21801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:22:09.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.027+0000 7faee218b740 20 client.21801 populate_metadata read hostname 'smithi120' 2024-09-19T09:22:09.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.027+0000 7faee218b740 1 --2- 172.21.15.120:0/2793764376 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564f69045da0 0x564f690481c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.027+0000 7faee218b740 4 client.21801 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:22:09.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.027+0000 7faee218b740 1 -- 172.21.15.120:0/2793764376 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564f68f2dd60 con 0x564f69045da0 2024-09-19T09:22:09.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.027+0000 7faeddb4e640 1 --2- 172.21.15.120:0/2793764376 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564f69045da0 0x564f690481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.028+0000 7faec17fa640 10 client.21801 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:09.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.028+0000 7faeddb4e640 1 --2- 172.21.15.120:0/2793764376 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564f69045da0 0x564f690481c0 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b3238ceef98b329 server_cookie=8e240b1c081ce770 in_seq=0 out_seq=0 2024-09-19T09:22:09.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.029+0000 7faec17fa640 1 -- 172.21.15.120:0/2793764376 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564f68f2dd60 con 0x564f69045da0 2024-09-19T09:22:09.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.029+0000 7faec17fa640 10 client.21801 handle_command_reply: tid=0 2024-09-19T09:22:09.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.029+0000 7faee218b740 1 client.21801 shutdown 2024-09-19T09:22:09.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.029+0000 7faee218b740 1 -- 172.21.15.120:0/2793764376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f68fcb680 msgr2=0x564f68fcf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:09.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.029+0000 7faee218b740 1 --2- 172.21.15.120:0/2793764376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f68fcb680 0x564f68fcf060 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7faed009b8d0 tx=0x7faed0097f80 comp rx=0 tx=0).stop 2024-09-19T09:22:09.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.029+0000 7faee218b740 1 -- 172.21.15.120:0/2793764376 shutdown_connections 2024-09-19T09:22:09.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.029+0000 7faee218b740 1 --2- 172.21.15.120:0/2793764376 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x564f69045da0 0x564f690481c0 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.029+0000 7faee218b740 1 --2- 172.21.15.120:0/2793764376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f68fcb680 0x564f68fcf060 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.029+0000 7faee218b740 1 --2- 172.21.15.120:0/2793764376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f68fc0dc0 0x564f68fceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.029+0000 7faee218b740 1 --2- 172.21.15.120:0/2793764376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f68fc04a0 0x564f68fce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.029+0000 7faec17fa640 0 client.21801 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:09.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.029+0000 7faee218b740 1 -- 172.21.15.120:0/2793764376 >> 172.21.15.120:0/2793764376 conn(0x564f68fbca30 msgr2=0x564f68fbbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:09.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.029+0000 7faee218b740 1 -- 172.21.15.120:0/2793764376 shutdown_connections 2024-09-19T09:22:09.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.029+0000 7faee218b740 1 -- 172.21.15.120:0/2793764376 wait complete. 2024-09-19T09:22:09.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.029+0000 7faee218b740 20 client.21801 trim_cache size 0 max 16384 2024-09-19T09:22:09.035 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:22:09.035 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:22:09.035 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:22:09.035 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:22:09.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.030+0000 7faedfdb2640 1 -- 172.21.15.120:0/2957565696 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faec40a32c0 msgr2=0x7faec40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:09.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.030+0000 7faedfdb2640 1 --2- 172.21.15.120:0/2957565696 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faec40a32c0 0x7faec40a5780 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x564f68fbd890 tx=0x7faed4005f90 comp rx=0 tx=0).stop 2024-09-19T09:22:09.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.030+0000 7faedfdb2640 1 -- 172.21.15.120:0/2957565696 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faed810c180 msgr2=0x7faed819cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:09.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.030+0000 7faedfdb2640 1 --2- 172.21.15.120:0/2957565696 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faed810c180 0x7faed819cda0 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7faed0009fd0 tx=0x7faed000b260 comp rx=0 tx=0).stop 2024-09-19T09:22:09.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.030+0000 7faedfdb2640 1 -- 172.21.15.120:0/2957565696 shutdown_connections 2024-09-19T09:22:09.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.030+0000 7faedfdb2640 1 --2- 172.21.15.120:0/2957565696 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faec40a32c0 0x7faec40a5780 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.030+0000 7faedfdb2640 1 --2- 172.21.15.120:0/2957565696 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faed810c180 0x7faed819cda0 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.030+0000 7faedfdb2640 1 --2- 172.21.15.120:0/2957565696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faed8103670 0x7faed8198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.030+0000 7faedfdb2640 1 --2- 172.21.15.120:0/2957565696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faed8102d50 0x7faed81984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.030+0000 7faedfdb2640 1 -- 172.21.15.120:0/2957565696 >> 172.21.15.120:0/2957565696 conn(0x7faed80fc7d0 msgr2=0x7faed80fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:09.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.030+0000 7faedfdb2640 1 -- 172.21.15.120:0/2957565696 shutdown_connections 2024-09-19T09:22:09.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:09.030+0000 7faedfdb2640 1 -- 172.21.15.120:0/2957565696 wait complete. 2024-09-19T09:22:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.324+0000 7f79d9cd9640 1 -- 172.21.15.120:0/2224208150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79d4101b60 msgr2=0x7f79d4101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.324+0000 7f79d9cd9640 1 --2- 172.21.15.120:0/2224208150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79d4101b60 0x7f79d4101f60 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f79c4009900 tx=0x7f79c402a410 comp rx=0 tx=0).stop 2024-09-19T09:22:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.324+0000 7f79d9cd9640 1 -- 172.21.15.120:0/2224208150 shutdown_connections 2024-09-19T09:22:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.324+0000 7f79d9cd9640 1 --2- 172.21.15.120:0/2224208150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79d4103710 0x7f79d4109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.324+0000 7f79d9cd9640 1 --2- 172.21.15.120:0/2224208150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79d4102d50 0x7f79d41031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.324+0000 7f79d9cd9640 1 --2- 172.21.15.120:0/2224208150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79d4101b60 0x7f79d4101f60 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.324+0000 7f79d9cd9640 1 -- 172.21.15.120:0/2224208150 >> 172.21.15.120:0/2224208150 conn(0x7f79d40fd2e0 msgr2=0x7f79d40ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.324+0000 7f79d9cd9640 1 -- 172.21.15.120:0/2224208150 shutdown_connections 2024-09-19T09:22:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.324+0000 7f79d9cd9640 1 -- 172.21.15.120:0/2224208150 wait complete. 2024-09-19T09:22:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.325+0000 7f79d9cd9640 1 Processor -- start 2024-09-19T09:22:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.325+0000 7f79d9cd9640 1 -- start start 2024-09-19T09:22:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.325+0000 7f79d9cd9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79d4101b60 0x7f79d4197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.325+0000 7f79d9cd9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79d4102d50 0x7f79d4198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.325+0000 7f79d9cd9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79d4103710 0x7f79d419f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.325+0000 7f79d9cd9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f79d410d830 con 0x7f79d4102d50 2024-09-19T09:22:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.325+0000 7f79d9cd9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f79d410d6b0 con 0x7f79d4101b60 2024-09-19T09:22:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.325+0000 7f79d9cd9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f79d410d9b0 con 0x7f79d4103710 2024-09-19T09:22:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.326+0000 7f79d2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79d4102d50 0x7f79d4198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.326+0000 7f79d37fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79d4101b60 0x7f79d4197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.326+0000 7f79d2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79d4102d50 0x7f79d4198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39624/0 (socket says 172.21.15.120:39624) 2024-09-19T09:22:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.326+0000 7f79d2ffd640 1 -- 172.21.15.120:0/1384465697 learned_addr learned my addr 172.21.15.120:0/1384465697 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.326+0000 7f79d3fff640 1 --2- 172.21.15.120:0/1384465697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79d4103710 0x7f79d419f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.326+0000 7f79d2ffd640 1 -- 172.21.15.120:0/1384465697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79d4103710 msgr2=0x7f79d419f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.326+0000 7f79d2ffd640 1 --2- 172.21.15.120:0/1384465697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79d4103710 0x7f79d419f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.326+0000 7f79d2ffd640 1 -- 172.21.15.120:0/1384465697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79d4101b60 msgr2=0x7f79d4197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.326+0000 7f79d2ffd640 1 --2- 172.21.15.120:0/1384465697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79d4101b60 0x7f79d4197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.326+0000 7f79d2ffd640 1 -- 172.21.15.120:0/1384465697 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d419fa10 con 0x7f79d4102d50 2024-09-19T09:22:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.326+0000 7f79d2ffd640 1 --2- 172.21.15.120:0/1384465697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79d4102d50 0x7f79d4198330 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f79c800e9f0 tx=0x7f79c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.326+0000 7f79d0ff9640 1 -- 172.21.15.120:0/1384465697 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c800cd80 con 0x7f79d4102d50 2024-09-19T09:22:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.326+0000 7f79d9cd9640 1 -- 172.21.15.120:0/1384465697 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79d419fd00 con 0x7f79d4102d50 2024-09-19T09:22:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.326+0000 7f79d0ff9640 1 -- 172.21.15.120:0/1384465697 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79c8010910 con 0x7f79d4102d50 2024-09-19T09:22:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.326+0000 7f79d0ff9640 1 -- 172.21.15.120:0/1384465697 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c8018990 con 0x7f79d4102d50 2024-09-19T09:22:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.326+0000 7f79d9cd9640 1 -- 172.21.15.120:0/1384465697 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79d41a01f0 con 0x7f79d4102d50 2024-09-19T09:22:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.329+0000 7f79d0ff9640 1 -- 172.21.15.120:0/1384465697 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f79c8010ac0 con 0x7f79d4102d50 2024-09-19T09:22:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.329+0000 7f79dca6f740 1 Processor -- start 2024-09-19T09:22:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.329+0000 7f79dca6f740 1 -- start start 2024-09-19T09:22:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.329+0000 7f79dca6f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55957de509a0 0x55957de50da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.329+0000 7f79dca6f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55957de512e0 0x55957de55da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.329+0000 7f79dca6f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55957de56370 0x55957de58830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.329+0000 7f79dca6f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55957ddb42f0 con 0x55957de512e0 2024-09-19T09:22:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.329+0000 7f79dca6f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55957dd5c270 con 0x55957de509a0 2024-09-19T09:22:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.329+0000 7f79dca6f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55957ddb02b0 con 0x55957de56370 2024-09-19T09:22:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.329+0000 7f79d37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55957de512e0 0x55957de55da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.329+0000 7f79d37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55957de512e0 0x55957de55da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39630/0 (socket says 172.21.15.120:39630) 2024-09-19T09:22:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.329+0000 7f79d37fe640 1 -- 172.21.15.120:0/785666215 learned_addr learned my addr 172.21.15.120:0/785666215 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.329+0000 7f79d3fff640 1 --2- 172.21.15.120:0/785666215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55957de509a0 0x55957de50da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.330+0000 7f79d2ffd640 1 --2- 172.21.15.120:0/785666215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55957de56370 0x55957de58830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.330+0000 7f79d37fe640 1 -- 172.21.15.120:0/785666215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55957de56370 msgr2=0x55957de58830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.330+0000 7f79d37fe640 1 --2- 172.21.15.120:0/785666215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55957de56370 0x55957de58830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.330+0000 7f79d37fe640 1 -- 172.21.15.120:0/785666215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55957de509a0 msgr2=0x55957de50da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.330+0000 7f79d37fe640 1 --2- 172.21.15.120:0/785666215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55957de509a0 0x55957de50da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.330+0000 7f79d37fe640 1 -- 172.21.15.120:0/785666215 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55957dd5b340 con 0x55957de512e0 2024-09-19T09:22:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.330+0000 7f79d3fff640 1 --2- 172.21.15.120:0/785666215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55957de509a0 0x55957de50da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:22:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.330+0000 7f79d0ff9640 1 --2- 172.21.15.120:0/1384465697 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f79bc0a32f0 0x7f79bc0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.330+0000 7f79d37fe640 1 --2- 172.21.15.120:0/785666215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55957de512e0 0x55957de55da0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f79c4006010 tx=0x7f79c4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d64d380d796812 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.330+0000 7f79d3fff640 1 --2- 172.21.15.120:0/1384465697 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f79bc0a32f0 0x7f79bc0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.330+0000 7f79b9ffb640 1 -- 172.21.15.120:0/785666215 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c4035040 con 0x55957de512e0 2024-09-19T09:22:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.330+0000 7f79d0ff9640 1 -- 172.21.15.120:0/1384465697 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f79c80cfc00 con 0x7f79d4102d50 2024-09-19T09:22:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.330+0000 7f79b9ffb640 1 -- 172.21.15.120:0/785666215 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79c4004430 con 0x55957de512e0 2024-09-19T09:22:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.330+0000 7f79b9ffb640 1 -- 172.21.15.120:0/785666215 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c40316b0 con 0x55957de512e0 2024-09-19T09:22:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.331+0000 7f79dca6f740 1 -- 172.21.15.120:0/785666215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55957de512e0 msgr2=0x55957de55da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.331+0000 7f79dca6f740 1 --2- 172.21.15.120:0/785666215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55957de512e0 0x55957de55da0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f79c4006010 tx=0x7f79c4004040 comp rx=0 tx=0).stop 2024-09-19T09:22:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.331+0000 7f79d3fff640 1 --2- 172.21.15.120:0/1384465697 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f79bc0a32f0 0x7f79bc0a57b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f79cc009fd0 tx=0x7f79cc009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.331+0000 7f79dca6f740 1 -- 172.21.15.120:0/785666215 shutdown_connections 2024-09-19T09:22:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.331+0000 7f79dca6f740 1 --2- 172.21.15.120:0/785666215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55957de56370 0x55957de58830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.331+0000 7f79dca6f740 1 --2- 172.21.15.120:0/785666215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55957de512e0 0x55957de55da0 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.331+0000 7f79dca6f740 1 --2- 172.21.15.120:0/785666215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55957de509a0 0x55957de50da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.331+0000 7f79dca6f740 1 -- 172.21.15.120:0/785666215 >> 172.21.15.120:0/785666215 conn(0x55957de4c010 msgr2=0x55957de4c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.331+0000 7f79dca6f740 1 -- 172.21.15.120:0/785666215 shutdown_connections 2024-09-19T09:22:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.331+0000 7f79dca6f740 1 -- 172.21.15.120:0/785666215 wait complete. 2024-09-19T09:22:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.331+0000 7f79dca6f740 1 -- start start 2024-09-19T09:22:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.331+0000 7f79dca6f740 1 Processor -- start 2024-09-19T09:22:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.332+0000 7f79dca6f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55957de509a0 0x55957de5ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.332+0000 7f79dca6f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55957de512e0 0x55957de5b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.332+0000 7f79dca6f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55957de56370 0x55957de620f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.332+0000 7f79dca6f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55957de592e0 con 0x55957de56370 2024-09-19T09:22:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.332+0000 7f79dca6f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55957dd3d290 con 0x55957de512e0 2024-09-19T09:22:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.332+0000 7f79dca6f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55957de59460 con 0x55957de509a0 2024-09-19T09:22:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.332+0000 7f79dca6f740 10 client.0 _ll_register_callbacks cb 0x55957dd6e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:22:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.332+0000 7f79d3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55957de56370 0x55957de620f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.332+0000 7f79d2ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55957de512e0 0x55957de5b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.332+0000 7f79d37fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55957de509a0 0x55957de5ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.332+0000 7f79d3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55957de56370 0x55957de620f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39634/0 (socket says 172.21.15.120:39634) 2024-09-19T09:22:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.332+0000 7f79d3fff640 1 -- 172.21.15.120:0/1647596540 learned_addr learned my addr 172.21.15.120:0/1647596540 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.332+0000 7f79d3fff640 1 -- 172.21.15.120:0/1647596540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55957de509a0 msgr2=0x55957de5ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.332+0000 7f79d3fff640 1 --2- 172.21.15.120:0/1647596540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55957de509a0 0x55957de5ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.332+0000 7f79d3fff640 1 -- 172.21.15.120:0/1647596540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55957de512e0 msgr2=0x55957de5b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.332+0000 7f79d3fff640 1 --2- 172.21.15.120:0/1647596540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55957de512e0 0x55957de5b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.332+0000 7f79d3fff640 1 -- 172.21.15.120:0/1647596540 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55957dac7210 con 0x55957de56370 2024-09-19T09:22:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.332+0000 7f79d3fff640 1 --2- 172.21.15.120:0/1647596540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55957de56370 0x55957de620f0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f79cc0115a0 tx=0x7f79cc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adc6dcb17e6d5f40 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.332+0000 7f79b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:22:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.332+0000 7f79b2ffd640 1 -- 172.21.15.120:0/1647596540 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79cc01f9b0 con 0x55957de56370 2024-09-19T09:22:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.332+0000 7f79b2ffd640 1 -- 172.21.15.120:0/1647596540 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f79cc01fb50 con 0x55957de56370 2024-09-19T09:22:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.332+0000 7f79dca6f740 1 -- 172.21.15.120:0/1647596540 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79c4004430 con 0x55957de56370 2024-09-19T09:22:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.333+0000 7f79b2ffd640 1 -- 172.21.15.120:0/1647596540 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f79cc01fd50 con 0x55957de56370 2024-09-19T09:22:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.333+0000 7f79dca6f740 10 client.21813 fetch_fsmap learned FSMap version 39 2024-09-19T09:22:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.333+0000 7f79dca6f740 1 -- 172.21.15.120:0/1647596540 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55957db93620 con 0x55957de56370 2024-09-19T09:22:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.333+0000 7f79b2ffd640 1 -- 172.21.15.120:0/1647596540 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79c4004430 con 0x55957de56370 2024-09-19T09:22:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.333+0000 7f79b2ffd640 1 -- 172.21.15.120:0/1647596540 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f79cc026070 con 0x55957de56370 2024-09-19T09:22:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.333+0000 7f79dca6f740 10 client.21813 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:22:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.333+0000 7f79dca6f740 10 client.21813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:22:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.333+0000 7f79dca6f740 20 client.21813 populate_metadata read hostname 'smithi120' 2024-09-19T09:22:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.333+0000 7f79dca6f740 1 --2- 172.21.15.120:0/1647596540 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55957de668b0 0x55957de68ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.333+0000 7f79dca6f740 4 client.21813 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:22:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.333+0000 7f79dca6f740 1 -- 172.21.15.120:0/1647596540 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55957ddc1a20 con 0x55957de668b0 2024-09-19T09:22:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.333+0000 7f79d37fe640 1 --2- 172.21.15.120:0/1647596540 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55957de668b0 0x55957de68ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.334+0000 7f79b2ffd640 10 client.21813 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.334+0000 7f79d37fe640 1 --2- 172.21.15.120:0/1647596540 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55957de668b0 0x55957de68ce0 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37e9ee6a5ea026f0 server_cookie=62f21781fc03ff51 in_seq=0 out_seq=0 2024-09-19T09:22:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.338+0000 7f79b2ffd640 1 -- 172.21.15.120:0/1647596540 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55957ddc1a20 con 0x55957de668b0 2024-09-19T09:22:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.338+0000 7f79b2ffd640 10 client.21813 handle_command_reply: tid=0 2024-09-19T09:22:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.338+0000 7f79dca6f740 1 client.21813 shutdown 2024-09-19T09:22:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.338+0000 7f79dca6f740 1 -- 172.21.15.120:0/1647596540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55957de56370 msgr2=0x55957de620f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.338+0000 7f79dca6f740 1 --2- 172.21.15.120:0/1647596540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55957de56370 0x55957de620f0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f79cc0115a0 tx=0x7f79cc004ab0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.338+0000 7f79dca6f740 1 -- 172.21.15.120:0/1647596540 shutdown_connections 2024-09-19T09:22:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.338+0000 7f79dca6f740 1 --2- 172.21.15.120:0/1647596540 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55957de668b0 0x55957de68ce0 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.338+0000 7f79b2ffd640 0 client.21813 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.338+0000 7f79dca6f740 1 --2- 172.21.15.120:0/1647596540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55957de56370 0x55957de620f0 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.338+0000 7f79dca6f740 1 --2- 172.21.15.120:0/1647596540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55957de512e0 0x55957de5b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.338+0000 7f79dca6f740 1 --2- 172.21.15.120:0/1647596540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55957de509a0 0x55957de5ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.338+0000 7f79dca6f740 1 -- 172.21.15.120:0/1647596540 >> 172.21.15.120:0/1647596540 conn(0x55957de4c010 msgr2=0x55957de4d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.338+0000 7f79dca6f740 1 -- 172.21.15.120:0/1647596540 shutdown_connections 2024-09-19T09:22:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.338+0000 7f79dca6f740 1 -- 172.21.15.120:0/1647596540 wait complete. 2024-09-19T09:22:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.338+0000 7f79dca6f740 20 client.21813 trim_cache size 0 max 16384 2024-09-19T09:22:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.354+0000 7f79dca6f740 1 Processor -- start 2024-09-19T09:22:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.354+0000 7f79dca6f740 1 -- start start 2024-09-19T09:22:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.354+0000 7f79dca6f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55957de509a0 0x55957de55ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.354+0000 7f79dca6f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55957de512e0 0x55957de620b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.354+0000 7f79dca6f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55957de56370 0x55957de4a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.354+0000 7f79dca6f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f79d41a6da0 con 0x55957de509a0 2024-09-19T09:22:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.354+0000 7f79dca6f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f79d41a88e0 con 0x55957de512e0 2024-09-19T09:22:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.354+0000 7f79dca6f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f79d41a7470 con 0x55957de56370 2024-09-19T09:22:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.354+0000 7f79d37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55957de509a0 0x55957de55ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.354+0000 7f79d37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55957de509a0 0x55957de55ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39648/0 (socket says 172.21.15.120:39648) 2024-09-19T09:22:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.354+0000 7f79d37fe640 1 -- 172.21.15.120:0/2024886625 learned_addr learned my addr 172.21.15.120:0/2024886625 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.354+0000 7f79d3fff640 1 --2- 172.21.15.120:0/2024886625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55957de56370 0x55957de4a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.354+0000 7f79d2ffd640 1 --2- 172.21.15.120:0/2024886625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55957de512e0 0x55957de620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.354+0000 7f79d37fe640 1 -- 172.21.15.120:0/2024886625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55957de56370 msgr2=0x55957de4a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.354+0000 7f79d37fe640 1 --2- 172.21.15.120:0/2024886625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55957de56370 0x55957de4a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.354+0000 7f79d37fe640 1 -- 172.21.15.120:0/2024886625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55957de512e0 msgr2=0x55957de620b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.355+0000 7f79d37fe640 1 --2- 172.21.15.120:0/2024886625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55957de512e0 0x55957de620b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.355+0000 7f79d37fe640 1 -- 172.21.15.120:0/2024886625 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55957de517c0 con 0x55957de509a0 2024-09-19T09:22:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.355+0000 7f79d37fe640 1 --2- 172.21.15.120:0/2024886625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55957de509a0 0x55957de55ae0 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f79c4009900 tx=0x7f79c40315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88cf9a61c837a30 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.355+0000 7f79b97fa640 1 -- 172.21.15.120:0/2024886625 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c4035040 con 0x55957de509a0 2024-09-19T09:22:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.355+0000 7f79b97fa640 1 -- 172.21.15.120:0/2024886625 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79c4004970 con 0x55957de509a0 2024-09-19T09:22:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.355+0000 7f79b97fa640 1 -- 172.21.15.120:0/2024886625 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c4031d60 con 0x55957de509a0 2024-09-19T09:22:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.356+0000 7f79dca6f740 1 -- 172.21.15.120:0/2024886625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55957de509a0 msgr2=0x55957de55ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.356+0000 7f79dca6f740 1 --2- 172.21.15.120:0/2024886625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55957de509a0 0x55957de55ae0 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f79c4009900 tx=0x7f79c40315b0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.356+0000 7f79dca6f740 1 -- 172.21.15.120:0/2024886625 shutdown_connections 2024-09-19T09:22:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.356+0000 7f79dca6f740 1 --2- 172.21.15.120:0/2024886625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55957de56370 0x55957de4a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.356+0000 7f79dca6f740 1 --2- 172.21.15.120:0/2024886625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55957de512e0 0x55957de620b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.356+0000 7f79dca6f740 1 --2- 172.21.15.120:0/2024886625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55957de509a0 0x55957de55ae0 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.356+0000 7f79dca6f740 1 -- 172.21.15.120:0/2024886625 >> 172.21.15.120:0/2024886625 conn(0x55957de4c010 msgr2=0x55957de5ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.356+0000 7f79dca6f740 1 -- 172.21.15.120:0/2024886625 shutdown_connections 2024-09-19T09:22:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.356+0000 7f79dca6f740 1 -- 172.21.15.120:0/2024886625 wait complete. 2024-09-19T09:22:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.356+0000 7f79dca6f740 1 -- start start 2024-09-19T09:22:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.356+0000 7f79dca6f740 1 Processor -- start 2024-09-19T09:22:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.356+0000 7f79dca6f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55957de509a0 0x55957de5f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.356+0000 7f79dca6f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55957de512e0 0x55957de5fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.356+0000 7f79dca6f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55957de56370 0x55957de60d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.356+0000 7f79dca6f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f79d41a8210 con 0x55957de512e0 2024-09-19T09:22:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.356+0000 7f79dca6f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f79d41a7b40 con 0x55957de56370 2024-09-19T09:22:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.356+0000 7f79dca6f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f79d41a4d60 con 0x55957de509a0 2024-09-19T09:22:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.356+0000 7f79dca6f740 10 client.0 _ll_register_callbacks cb 0x7f79cc009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:22:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.357+0000 7f79d2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55957de512e0 0x55957de5fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.357+0000 7f79d2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55957de512e0 0x55957de5fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39654/0 (socket says 172.21.15.120:39654) 2024-09-19T09:22:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.357+0000 7f79d2ffd640 1 -- 172.21.15.120:0/3103762087 learned_addr learned my addr 172.21.15.120:0/3103762087 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.357+0000 7f79d37fe640 1 --2- 172.21.15.120:0/3103762087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55957de509a0 0x55957de5f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.357+0000 7f79d3fff640 1 --2- 172.21.15.120:0/3103762087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55957de56370 0x55957de60d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.357+0000 7f79d2ffd640 1 -- 172.21.15.120:0/3103762087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55957de509a0 msgr2=0x55957de5f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.357+0000 7f79d2ffd640 1 --2- 172.21.15.120:0/3103762087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55957de509a0 0x55957de5f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.357+0000 7f79d2ffd640 1 -- 172.21.15.120:0/3103762087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55957de56370 msgr2=0x55957de60d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.357+0000 7f79d2ffd640 1 --2- 172.21.15.120:0/3103762087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55957de56370 0x55957de60d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.357+0000 7f79d2ffd640 1 -- 172.21.15.120:0/3103762087 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55957de61320 con 0x55957de512e0 2024-09-19T09:22:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.357+0000 7f79d2ffd640 1 --2- 172.21.15.120:0/3103762087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55957de512e0 0x55957de5fd00 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f79c809bfd0 tx=0x7f79c809c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f821b89bfb3b01c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.357+0000 7f79b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:22:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.357+0000 7f79b2ffd640 1 -- 172.21.15.120:0/3103762087 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c80bae60 con 0x55957de512e0 2024-09-19T09:22:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.357+0000 7f79b2ffd640 1 -- 172.21.15.120:0/3103762087 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f79c80a4c00 con 0x55957de512e0 2024-09-19T09:22:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.357+0000 7f79dca6f740 1 -- 172.21.15.120:0/3103762087 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79c4004970 con 0x55957de512e0 2024-09-19T09:22:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.357+0000 7f79b2ffd640 1 -- 172.21.15.120:0/3103762087 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f79c80a4e90 con 0x55957de512e0 2024-09-19T09:22:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.357+0000 7f79dca6f740 10 client.21819 fetch_fsmap learned FSMap version 39 2024-09-19T09:22:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.357+0000 7f79dca6f740 1 -- 172.21.15.120:0/3103762087 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55957ded15c0 con 0x55957de512e0 2024-09-19T09:22:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.357+0000 7f79b2ffd640 1 -- 172.21.15.120:0/3103762087 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79c4004970 con 0x55957de512e0 2024-09-19T09:22:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.358+0000 7f79b2ffd640 1 -- 172.21.15.120:0/3103762087 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f79c80a0240 con 0x55957de512e0 2024-09-19T09:22:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.358+0000 7f79dca6f740 10 client.21819 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:22:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.358+0000 7f79dca6f740 10 client.21819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:22:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.358+0000 7f79dca6f740 20 client.21819 populate_metadata read hostname 'smithi120' 2024-09-19T09:22:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.358+0000 7f79dca6f740 1 --2- 172.21.15.120:0/3103762087 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55957de668b0 0x55957ded4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.358+0000 7f79dca6f740 4 client.21819 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:22:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.358+0000 7f79dca6f740 1 -- 172.21.15.120:0/3103762087 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55957ddbbd70 con 0x55957de668b0 2024-09-19T09:22:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.358+0000 7f79d37fe640 1 --2- 172.21.15.120:0/3103762087 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55957de668b0 0x55957ded4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.358+0000 7f79d37fe640 1 --2- 172.21.15.120:0/3103762087 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55957de668b0 0x55957ded4000 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40c34e462983bb43 server_cookie=f4d333c7d8901d14 in_seq=0 out_seq=0 2024-09-19T09:22:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.358+0000 7f79b2ffd640 10 client.21819 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:09.372 INFO:tasks.workunit.client.0.smithi120.stdout: 10 153118 21.53 MB/sec execute 244 sec latency 93.821 ms 2024-09-19T09:22:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.792+0000 7f79b2ffd640 1 -- 172.21.15.120:0/3103762087 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55957ddbbd70 con 0x55957de668b0 2024-09-19T09:22:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.792+0000 7f79b2ffd640 10 client.21819 handle_command_reply: tid=0 2024-09-19T09:22:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.792+0000 7f79dca6f740 1 client.21819 shutdown 2024-09-19T09:22:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.793+0000 7f79dca6f740 1 -- 172.21.15.120:0/3103762087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55957de512e0 msgr2=0x55957de5fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.793+0000 7f79dca6f740 1 --2- 172.21.15.120:0/3103762087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55957de512e0 0x55957de5fd00 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f79c809bfd0 tx=0x7f79c809c1d0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.793+0000 7f79dca6f740 1 -- 172.21.15.120:0/3103762087 shutdown_connections 2024-09-19T09:22:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.793+0000 7f79dca6f740 1 --2- 172.21.15.120:0/3103762087 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55957de668b0 0x55957ded4000 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.793+0000 7f79dca6f740 1 --2- 172.21.15.120:0/3103762087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55957de56370 0x55957de60d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.793+0000 7f79dca6f740 1 --2- 172.21.15.120:0/3103762087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55957de512e0 0x55957de5fd00 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.793+0000 7f79b2ffd640 0 client.21819 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.793+0000 7f79dca6f740 1 --2- 172.21.15.120:0/3103762087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55957de509a0 0x55957de5f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.793+0000 7f79dca6f740 1 -- 172.21.15.120:0/3103762087 >> 172.21.15.120:0/3103762087 conn(0x55957de4c010 msgr2=0x55957de5ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.793+0000 7f79dca6f740 1 -- 172.21.15.120:0/3103762087 shutdown_connections 2024-09-19T09:22:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.793+0000 7f79dca6f740 1 -- 172.21.15.120:0/3103762087 wait complete. 2024-09-19T09:22:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.793+0000 7f79dca6f740 20 client.21819 trim_cache size 0 max 16384 2024-09-19T09:22:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:22:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:22:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:22:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:22:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.793+0000 7f79d9cd9640 1 -- 172.21.15.120:0/1384465697 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f79bc0a32f0 msgr2=0x7f79bc0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.793+0000 7f79d9cd9640 1 --2- 172.21.15.120:0/1384465697 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f79bc0a32f0 0x7f79bc0a57b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f79cc009fd0 tx=0x7f79cc009500 comp rx=0 tx=0).stop 2024-09-19T09:22:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.793+0000 7f79d9cd9640 1 -- 172.21.15.120:0/1384465697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79d4102d50 msgr2=0x7f79d4198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.793+0000 7f79d9cd9640 1 --2- 172.21.15.120:0/1384465697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79d4102d50 0x7f79d4198330 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f79c800e9f0 tx=0x7f79c800eeb0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.793+0000 7f79d9cd9640 1 -- 172.21.15.120:0/1384465697 shutdown_connections 2024-09-19T09:22:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.793+0000 7f79d9cd9640 1 --2- 172.21.15.120:0/1384465697 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f79bc0a32f0 0x7f79bc0a57b0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.793+0000 7f79d9cd9640 1 --2- 172.21.15.120:0/1384465697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79d4103710 0x7f79d419f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.793+0000 7f79d9cd9640 1 --2- 172.21.15.120:0/1384465697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79d4102d50 0x7f79d4198330 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.793+0000 7f79d9cd9640 1 --2- 172.21.15.120:0/1384465697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79d4101b60 0x7f79d4197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.793+0000 7f79d9cd9640 1 -- 172.21.15.120:0/1384465697 >> 172.21.15.120:0/1384465697 conn(0x7f79d40fd2e0 msgr2=0x7f79d4106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.794+0000 7f79d9cd9640 1 -- 172.21.15.120:0/1384465697 shutdown_connections 2024-09-19T09:22:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:09.794+0000 7f79d9cd9640 1 -- 172.21.15.120:0/1384465697 wait complete. 2024-09-19T09:22:10.075 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:10 smithi120 ceph-mon[31150]: pgmap v924: 97 pgs: 97 active+clean; 380 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.1 MiB/s wr, 421 op/s 2024-09-19T09:22:10.155 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:22:10.156 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:22:10.372 INFO:tasks.workunit.client.0.smithi120.stdout: 10 153601 21.57 MB/sec execute 245 sec latency 304.880 ms 2024-09-19T09:22:10.431 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:22:10.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:10 smithi196 ceph-mon[35150]: pgmap v924: 97 pgs: 97 active+clean; 380 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.1 MiB/s wr, 421 op/s 2024-09-19T09:22:10.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:10 smithi153 ceph-mon[35071]: pgmap v924: 97 pgs: 97 active+clean; 380 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.1 MiB/s wr, 421 op/s 2024-09-19T09:22:10.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.965+0000 7f28adeb1640 1 -- 172.21.15.120:0/1764750152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28a8102570 msgr2=0x7f28a81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:10.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.965+0000 7f28adeb1640 1 --2- 172.21.15.120:0/1764750152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28a8102570 0x7f28a81029b0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f289c009900 tx=0x7f289c02a480 comp rx=0 tx=0).stop 2024-09-19T09:22:10.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.965+0000 7f28adeb1640 1 -- 172.21.15.120:0/1764750152 shutdown_connections 2024-09-19T09:22:10.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.965+0000 7f28adeb1640 1 --2- 172.21.15.120:0/1764750152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28a8102ef0 0x7f28a810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:10.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.965+0000 7f28adeb1640 1 --2- 172.21.15.120:0/1764750152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28a8102570 0x7f28a81029b0 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:10.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.965+0000 7f28adeb1640 1 --2- 172.21.15.120:0/1764750152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28a8100e20 0x7f28a8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:10.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.965+0000 7f28adeb1640 1 -- 172.21.15.120:0/1764750152 >> 172.21.15.120:0/1764750152 conn(0x7f28a80fc7d0 msgr2=0x7f28a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:10.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.965+0000 7f28adeb1640 1 -- 172.21.15.120:0/1764750152 shutdown_connections 2024-09-19T09:22:10.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.965+0000 7f28adeb1640 1 -- 172.21.15.120:0/1764750152 wait complete. 2024-09-19T09:22:10.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.966+0000 7f28adeb1640 1 Processor -- start 2024-09-19T09:22:10.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.966+0000 7f28adeb1640 1 -- start start 2024-09-19T09:22:10.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.967+0000 7f28adeb1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28a8100e20 0x7f28a8199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:10.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.967+0000 7f28adeb1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28a8102570 0x7f28a81998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:10.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.967+0000 7f28adeb1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28a8102ef0 0x7f28a81a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:10.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.967+0000 7f28adeb1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f28a8110f80 con 0x7f28a8102570 2024-09-19T09:22:10.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.967+0000 7f28adeb1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f28a8110e00 con 0x7f28a8100e20 2024-09-19T09:22:10.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.967+0000 7f28adeb1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f28a8111100 con 0x7f28a8102ef0 2024-09-19T09:22:10.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.967+0000 7f28a6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28a8102570 0x7f28a81998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:10.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.967+0000 7f28a6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28a8102570 0x7f28a81998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39664/0 (socket says 172.21.15.120:39664) 2024-09-19T09:22:10.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.967+0000 7f28a6ffd640 1 -- 172.21.15.120:0/3472952121 learned_addr learned my addr 172.21.15.120:0/3472952121 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:10.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.967+0000 7f28a77fe640 1 --2- 172.21.15.120:0/3472952121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28a8100e20 0x7f28a8199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:10.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.967+0000 7f28a7fff640 1 --2- 172.21.15.120:0/3472952121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28a8102ef0 0x7f28a81a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:10.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.967+0000 7f28a6ffd640 1 -- 172.21.15.120:0/3472952121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28a8102ef0 msgr2=0x7f28a81a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:10.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.967+0000 7f28a6ffd640 1 --2- 172.21.15.120:0/3472952121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28a8102ef0 0x7f28a81a43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:10.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.967+0000 7f28a6ffd640 1 -- 172.21.15.120:0/3472952121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28a8100e20 msgr2=0x7f28a8199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:10.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.967+0000 7f28a6ffd640 1 --2- 172.21.15.120:0/3472952121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28a8100e20 0x7f28a8199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:10.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.967+0000 7f28a6ffd640 1 -- 172.21.15.120:0/3472952121 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28a81a4930 con 0x7f28a8102570 2024-09-19T09:22:10.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.967+0000 7f28a77fe640 1 --2- 172.21.15.120:0/3472952121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28a8100e20 0x7f28a8199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:22:10.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.968+0000 7f28a6ffd640 1 --2- 172.21.15.120:0/3472952121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28a8102570 0x7f28a81998b0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f289c006010 tx=0x7f289c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:10.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.968+0000 7f28a4ff9640 1 -- 172.21.15.120:0/3472952121 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c03f070 con 0x7f28a8102570 2024-09-19T09:22:10.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.968+0000 7f28a4ff9640 1 -- 172.21.15.120:0/3472952121 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f289c031c30 con 0x7f28a8102570 2024-09-19T09:22:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.968+0000 7f28adeb1640 1 -- 172.21.15.120:0/3472952121 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28a8197160 con 0x7f28a8102570 2024-09-19T09:22:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.968+0000 7f28a4ff9640 1 -- 172.21.15.120:0/3472952121 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c035030 con 0x7f28a8102570 2024-09-19T09:22:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.968+0000 7f28adeb1640 1 -- 172.21.15.120:0/3472952121 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28a8197640 con 0x7f28a8102570 2024-09-19T09:22:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.969+0000 7f28adeb1640 1 -- 172.21.15.120:0/3472952121 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28a8101200 con 0x7f28a8102570 2024-09-19T09:22:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.970+0000 7f28a4ff9640 1 -- 172.21.15.120:0/3472952121 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f289c042070 con 0x7f28a8102570 2024-09-19T09:22:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.971+0000 7f28a4ff9640 1 --2- 172.21.15.120:0/3472952121 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f28900a3240 0x7f28900a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.971+0000 7f28a77fe640 1 --2- 172.21.15.120:0/3472952121 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f28900a3240 0x7f28900a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.971+0000 7f28a4ff9640 1 -- 172.21.15.120:0/3472952121 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f289c0f04f0 con 0x7f28a8102570 2024-09-19T09:22:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.971+0000 7f28a77fe640 1 --2- 172.21.15.120:0/3472952121 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f28900a3240 0x7f28900a5700 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f28980096a0 tx=0x7f2898012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:10.974+0000 7f28a4ff9640 1 -- 172.21.15.120:0/3472952121 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f289c031de0 con 0x7f28a8102570 2024-09-19T09:22:11.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:11.210+0000 7f28adeb1640 1 -- 172.21.15.120:0/3472952121 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28a8109090 con 0x7f28a8102570 2024-09-19T09:22:11.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:11.211+0000 7f28a4ff9640 1 -- 172.21.15.120:0/3472952121 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11988 (secure 0 0 0) 0x7f289c0bcbc0 con 0x7f28a8102570 2024-09-19T09:22:11.212 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:22:11.212 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":39,"btime":"2024-09-19T09:18:53:120858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:18:53.070639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:22:11.213 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 39 2024-09-19T09:22:11.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:11.213+0000 7f28adeb1640 1 -- 172.21.15.120:0/3472952121 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f28900a3240 msgr2=0x7f28900a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:11.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:11.213+0000 7f28adeb1640 1 --2- 172.21.15.120:0/3472952121 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f28900a3240 0x7f28900a5700 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f28980096a0 tx=0x7f2898012040 comp rx=0 tx=0).stop 2024-09-19T09:22:11.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:11.213+0000 7f28adeb1640 1 -- 172.21.15.120:0/3472952121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28a8102570 msgr2=0x7f28a81998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:11.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:11.213+0000 7f28adeb1640 1 --2- 172.21.15.120:0/3472952121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28a8102570 0x7f28a81998b0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f289c006010 tx=0x7f289c003c10 comp rx=0 tx=0).stop 2024-09-19T09:22:11.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:11.213+0000 7f28adeb1640 1 -- 172.21.15.120:0/3472952121 shutdown_connections 2024-09-19T09:22:11.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:11.213+0000 7f28adeb1640 1 --2- 172.21.15.120:0/3472952121 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f28900a3240 0x7f28900a5700 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:11.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:11.213+0000 7f28adeb1640 1 --2- 172.21.15.120:0/3472952121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28a8102ef0 0x7f28a81a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:11.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:11.213+0000 7f28adeb1640 1 --2- 172.21.15.120:0/3472952121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28a8102570 0x7f28a81998b0 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:11.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:11.213+0000 7f28adeb1640 1 --2- 172.21.15.120:0/3472952121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28a8100e20 0x7f28a8199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:11.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:11.213+0000 7f28adeb1640 1 -- 172.21.15.120:0/3472952121 >> 172.21.15.120:0/3472952121 conn(0x7f28a80fc7d0 msgr2=0x7f28a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:11.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:11.213+0000 7f28adeb1640 1 -- 172.21.15.120:0/3472952121 shutdown_connections 2024-09-19T09:22:11.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:11.213+0000 7f28adeb1640 1 -- 172.21.15.120:0/3472952121 wait complete. 2024-09-19T09:22:11.372 INFO:tasks.workunit.client.0.smithi120.stdout: 10 154352 21.61 MB/sec execute 246 sec latency 49.556 ms 2024-09-19T09:22:11.536 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:11 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3472952121' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:22:11.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:11 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3472952121' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:22:11.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:11 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3472952121' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:22:12.012 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:22:12.318 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:22:12.372 INFO:tasks.workunit.client.0.smithi120.stdout: 10 155030 21.69 MB/sec execute 247 sec latency 51.375 ms 2024-09-19T09:22:12.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:12 smithi120 ceph-mon[31150]: pgmap v925: 97 pgs: 97 active+clean; 381 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.7 MiB/s wr, 522 op/s 2024-09-19T09:22:12.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:12 smithi196 ceph-mon[35150]: pgmap v925: 97 pgs: 97 active+clean; 381 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.7 MiB/s wr, 522 op/s 2024-09-19T09:22:12.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:12 smithi153 ceph-mon[35071]: pgmap v925: 97 pgs: 97 active+clean; 381 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.7 MiB/s wr, 522 op/s 2024-09-19T09:22:12.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.883+0000 7f8e3b32b640 1 -- 172.21.15.120:0/3145328744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e34100a80 msgr2=0x7f8e34100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:12.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.883+0000 7f8e3b32b640 1 --2- 172.21.15.120:0/3145328744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e34100a80 0x7f8e34100ee0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f8e24009930 tx=0x7f8e2402a420 comp rx=0 tx=0).stop 2024-09-19T09:22:12.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.884+0000 7f8e3b32b640 1 -- 172.21.15.120:0/3145328744 shutdown_connections 2024-09-19T09:22:12.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.884+0000 7f8e3b32b640 1 --2- 172.21.15.120:0/3145328744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e34101420 0x7f8e3410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.884+0000 7f8e3b32b640 1 --2- 172.21.15.120:0/3145328744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e34100a80 0x7f8e34100ee0 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.884+0000 7f8e3b32b640 1 --2- 172.21.15.120:0/3145328744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e34106a80 0x7f8e34106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.884+0000 7f8e3b32b640 1 -- 172.21.15.120:0/3145328744 >> 172.21.15.120:0/3145328744 conn(0x7f8e340fc7a0 msgr2=0x7f8e340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:12.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.884+0000 7f8e3b32b640 1 -- 172.21.15.120:0/3145328744 shutdown_connections 2024-09-19T09:22:12.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.884+0000 7f8e3b32b640 1 -- 172.21.15.120:0/3145328744 wait complete. 2024-09-19T09:22:12.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.885+0000 7f8e3b32b640 1 Processor -- start 2024-09-19T09:22:12.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.885+0000 7f8e3b32b640 1 -- start start 2024-09-19T09:22:12.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.885+0000 7f8e3b32b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e34100a80 0x7f8e34198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:12.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.885+0000 7f8e3b32b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e34101420 0x7f8e341989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:12.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.885+0000 7f8e3b32b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e34106a80 0x7f8e3419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:12.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.885+0000 7f8e3b32b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8e34110e40 con 0x7f8e34101420 2024-09-19T09:22:12.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.885+0000 7f8e3b32b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8e34110cc0 con 0x7f8e34106a80 2024-09-19T09:22:12.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.885+0000 7f8e3b32b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e34110fc0 con 0x7f8e34100a80 2024-09-19T09:22:12.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.885+0000 7f8e388c6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e34101420 0x7f8e341989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:12.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.885+0000 7f8e388c6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e34101420 0x7f8e341989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43102/0 (socket says 172.21.15.120:43102) 2024-09-19T09:22:12.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.885+0000 7f8e388c6640 1 -- 172.21.15.120:0/1508520226 learned_addr learned my addr 172.21.15.120:0/1508520226 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:12.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.885+0000 7f8e398c8640 1 --2- 172.21.15.120:0/1508520226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e34106a80 0x7f8e3419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:12.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.885+0000 7f8e390c7640 1 --2- 172.21.15.120:0/1508520226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e34100a80 0x7f8e34198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:12.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.885+0000 7f8e388c6640 1 -- 172.21.15.120:0/1508520226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e34100a80 msgr2=0x7f8e34198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:12.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.885+0000 7f8e388c6640 1 --2- 172.21.15.120:0/1508520226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e34100a80 0x7f8e34198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.885+0000 7f8e388c6640 1 -- 172.21.15.120:0/1508520226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e34106a80 msgr2=0x7f8e3419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:12.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.885+0000 7f8e388c6640 1 --2- 172.21.15.120:0/1508520226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e34106a80 0x7f8e3419cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.885+0000 7f8e388c6640 1 -- 172.21.15.120:0/1508520226 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3419d410 con 0x7f8e34101420 2024-09-19T09:22:12.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.885+0000 7f8e398c8640 1 --2- 172.21.15.120:0/1508520226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e34106a80 0x7f8e3419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:22:12.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.886+0000 7f8e388c6640 1 --2- 172.21.15.120:0/1508520226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e34101420 0x7f8e341989a0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f8e24009230 tx=0x7f8e24004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:12.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.886+0000 7f8e2a7fc640 1 -- 172.21.15.120:0/1508520226 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2403f030 con 0x7f8e34101420 2024-09-19T09:22:12.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.886+0000 7f8e2a7fc640 1 -- 172.21.15.120:0/1508520226 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e24003ad0 con 0x7f8e34101420 2024-09-19T09:22:12.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.886+0000 7f8e3b32b640 1 -- 172.21.15.120:0/1508520226 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e3419d6a0 con 0x7f8e34101420 2024-09-19T09:22:12.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.886+0000 7f8e2a7fc640 1 -- 172.21.15.120:0/1508520226 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24035070 con 0x7f8e34101420 2024-09-19T09:22:12.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.886+0000 7f8e3b32b640 1 -- 172.21.15.120:0/1508520226 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e341a4fe0 con 0x7f8e34101420 2024-09-19T09:22:12.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.889+0000 7f8e2a7fc640 1 -- 172.21.15.120:0/1508520226 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f8e24042030 con 0x7f8e34101420 2024-09-19T09:22:12.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.890+0000 7f8e3d704740 1 Processor -- start 2024-09-19T09:22:12.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.890+0000 7f8e3d704740 1 -- start start 2024-09-19T09:22:12.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.891+0000 7f8e3d704740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564769c5a440 0x564769c5a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:12.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.891+0000 7f8e3d704740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564769c5adf0 0x564769c5f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:12.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.891+0000 7f8e3d704740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564769c5fa40 0x564769c5fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:12.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.891+0000 7f8e3d704740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564769bb62f0 con 0x564769c5a440 2024-09-19T09:22:12.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.891+0000 7f8e3d704740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564769b5dcf0 con 0x564769c5adf0 2024-09-19T09:22:12.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.891+0000 7f8e3d704740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564769bb22b0 con 0x564769c5fa40 2024-09-19T09:22:12.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.891+0000 7f8e388c6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564769c5fa40 0x564769c5fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:12.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.891+0000 7f8e388c6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564769c5fa40 0x564769c5fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:38404/0 (socket says 172.21.15.120:38404) 2024-09-19T09:22:12.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.891+0000 7f8e388c6640 1 -- 172.21.15.120:0/146529806 learned_addr learned my addr 172.21.15.120:0/146529806 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:12.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.891+0000 7f8e388c6640 1 -- 172.21.15.120:0/146529806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564769c5adf0 msgr2=0x564769c5f470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:22:12.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.891+0000 7f8e388c6640 1 --2- 172.21.15.120:0/146529806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564769c5adf0 0x564769c5f470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.891+0000 7f8e388c6640 1 -- 172.21.15.120:0/146529806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564769c5a440 msgr2=0x564769c5a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:12.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.891+0000 7f8e388c6640 1 --2- 172.21.15.120:0/146529806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564769c5a440 0x564769c5a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.891+0000 7f8e388c6640 1 -- 172.21.15.120:0/146529806 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564769b423a0 con 0x564769c5fa40 2024-09-19T09:22:12.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.892+0000 7f8e388c6640 1 --2- 172.21.15.120:0/146529806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564769c5fa40 0x564769c5fef0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8e240f3f40 tx=0x7f8e240f8520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7c1ee9b7b2be8d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:12.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.892+0000 7f8e2a7fc640 1 --2- 172.21.15.120:0/1508520226 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8e200a32c0 0x7f8e200a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:12.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.892+0000 7f8e1f7fe640 1 -- 172.21.15.120:0/146529806 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e240ff070 con 0x564769c5fa40 2024-09-19T09:22:12.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.892+0000 7f8e1f7fe640 1 -- 172.21.15.120:0/146529806 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e24007380 con 0x564769c5fa40 2024-09-19T09:22:12.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.892+0000 7f8e1f7fe640 1 -- 172.21.15.120:0/146529806 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e240bbd20 con 0x564769c5fa40 2024-09-19T09:22:12.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.892+0000 7f8e2a7fc640 1 -- 172.21.15.120:0/1508520226 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f8e240efde0 con 0x7f8e34101420 2024-09-19T09:22:12.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.893+0000 7f8e398c8640 1 --2- 172.21.15.120:0/1508520226 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8e200a32c0 0x7f8e200a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:12.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.894+0000 7f8e3d704740 1 -- 172.21.15.120:0/146529806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564769c5fa40 msgr2=0x564769c5fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:12.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.894+0000 7f8e3d704740 1 --2- 172.21.15.120:0/146529806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564769c5fa40 0x564769c5fef0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8e240f3f40 tx=0x7f8e240f8520 comp rx=0 tx=0).stop 2024-09-19T09:22:12.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.895+0000 7f8e398c8640 1 --2- 172.21.15.120:0/1508520226 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8e200a32c0 0x7f8e200a5780 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c005fd0 tx=0x7f8e2c009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:12.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.896+0000 7f8e3d704740 1 -- 172.21.15.120:0/146529806 shutdown_connections 2024-09-19T09:22:12.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.896+0000 7f8e3d704740 1 --2- 172.21.15.120:0/146529806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564769c5fa40 0x564769c5fef0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.896+0000 7f8e3d704740 1 --2- 172.21.15.120:0/146529806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564769c5adf0 0x564769c5f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.896+0000 7f8e3d704740 1 --2- 172.21.15.120:0/146529806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564769c5a440 0x564769c5a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.896+0000 7f8e3d704740 1 -- 172.21.15.120:0/146529806 >> 172.21.15.120:0/146529806 conn(0x564769c19310 msgr2=0x564769c4c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:12.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.897+0000 7f8e3d704740 1 -- 172.21.15.120:0/146529806 shutdown_connections 2024-09-19T09:22:12.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.897+0000 7f8e3d704740 1 -- 172.21.15.120:0/146529806 wait complete. 2024-09-19T09:22:12.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.898+0000 7f8e3d704740 1 -- start start 2024-09-19T09:22:12.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.898+0000 7f8e3d704740 1 Processor -- start 2024-09-19T09:22:12.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.898+0000 7f8e3d704740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564769c5a440 0x564769c65dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:12.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.898+0000 7f8e3d704740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564769c5adf0 0x564769c63070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:12.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.898+0000 7f8e3d704740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564769c5fa40 0x564769c635b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:12.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.899+0000 7f8e3d704740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564769c66690 con 0x564769c5fa40 2024-09-19T09:22:12.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.899+0000 7f8e3d704740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564769b71060 con 0x564769c5a440 2024-09-19T09:22:12.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.899+0000 7f8e3d704740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564769c66810 con 0x564769c5adf0 2024-09-19T09:22:12.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.899+0000 7f8e3d704740 10 client.0 _ll_register_callbacks cb 0x5647698c48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:22:12.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.899+0000 7f8e398c8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564769c5fa40 0x564769c635b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:12.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.899+0000 7f8e398c8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564769c5fa40 0x564769c635b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43106/0 (socket says 172.21.15.120:43106) 2024-09-19T09:22:12.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.899+0000 7f8e398c8640 1 -- 172.21.15.120:0/1637660441 learned_addr learned my addr 172.21.15.120:0/1637660441 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:12.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.899+0000 7f8e390c7640 1 --2- 172.21.15.120:0/1637660441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564769c5a440 0x564769c65dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:12.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.899+0000 7f8e388c6640 1 --2- 172.21.15.120:0/1637660441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564769c5adf0 0x564769c63070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:12.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.899+0000 7f8e398c8640 1 -- 172.21.15.120:0/1637660441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564769c5adf0 msgr2=0x564769c63070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:12.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.899+0000 7f8e398c8640 1 --2- 172.21.15.120:0/1637660441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564769c5adf0 0x564769c63070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.899+0000 7f8e398c8640 1 -- 172.21.15.120:0/1637660441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564769c5a440 msgr2=0x564769c65dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:12.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.899+0000 7f8e398c8640 1 --2- 172.21.15.120:0/1637660441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564769c5a440 0x564769c65dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.899+0000 7f8e398c8640 1 -- 172.21.15.120:0/1637660441 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647699273c0 con 0x564769c5fa40 2024-09-19T09:22:12.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.899+0000 7f8e398c8640 1 --2- 172.21.15.120:0/1637660441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564769c5fa40 0x564769c635b0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c011790 tx=0x7f8e2c011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58730fb7f24f1483 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:12.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.899+0000 7f8e1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:22:12.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.900+0000 7f8e1cff9640 1 -- 172.21.15.120:0/1637660441 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c022070 con 0x564769c5fa40 2024-09-19T09:22:12.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.900+0000 7f8e1cff9640 1 -- 172.21.15.120:0/1637660441 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f8e2c010b50 con 0x564769c5fa40 2024-09-19T09:22:12.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.900+0000 7f8e3d704740 1 -- 172.21.15.120:0/1637660441 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e24007380 con 0x564769c5fa40 2024-09-19T09:22:12.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.900+0000 7f8e1cff9640 1 -- 172.21.15.120:0/1637660441 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8e2c02c050 con 0x564769c5fa40 2024-09-19T09:22:12.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.900+0000 7f8e3d704740 10 client.21834 fetch_fsmap learned FSMap version 39 2024-09-19T09:22:12.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.900+0000 7f8e3d704740 1 -- 172.21.15.120:0/1637660441 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564769c71f90 con 0x564769c5fa40 2024-09-19T09:22:12.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.901+0000 7f8e1cff9640 1 -- 172.21.15.120:0/1637660441 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e2c010d50 con 0x564769c5fa40 2024-09-19T09:22:12.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.901+0000 7f8e1cff9640 1 -- 172.21.15.120:0/1637660441 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f8e2c027680 con 0x564769c5fa40 2024-09-19T09:22:12.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.901+0000 7f8e3d704740 10 client.21834 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:22:12.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.901+0000 7f8e3d704740 10 client.21834 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:22:12.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.901+0000 7f8e3d704740 20 client.21834 populate_metadata read hostname 'smithi120' 2024-09-19T09:22:12.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.901+0000 7f8e3d704740 1 --2- 172.21.15.120:0/1637660441 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x564769c73450 0x564769c75840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:12.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.901+0000 7f8e3d704740 4 client.21834 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:22:12.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.901+0000 7f8e3d704740 1 -- 172.21.15.120:0/1637660441 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8e24007380 con 0x564769c73450 2024-09-19T09:22:12.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.901+0000 7f8e390c7640 1 --2- 172.21.15.120:0/1637660441 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x564769c73450 0x564769c75840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:12.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.902+0000 7f8e1cff9640 10 client.21834 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:22:12.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.902+0000 7f8e390c7640 1 --2- 172.21.15.120:0/1637660441 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x564769c73450 0x564769c75840 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b28cd64fae6365d1 server_cookie=fb20249b0a09b988 in_seq=0 out_seq=0 2024-09-19T09:22:12.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.902+0000 7f8e1cff9640 1 -- 172.21.15.120:0/1637660441 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f8e24007380 con 0x564769c73450 2024-09-19T09:22:12.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.903+0000 7f8e1cff9640 10 client.21834 handle_command_reply: tid=0 2024-09-19T09:22:12.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.903+0000 7f8e3d704740 1 client.21834 shutdown 2024-09-19T09:22:12.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.903+0000 7f8e3d704740 1 -- 172.21.15.120:0/1637660441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564769c5fa40 msgr2=0x564769c635b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:12.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.903+0000 7f8e3d704740 1 --2- 172.21.15.120:0/1637660441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564769c5fa40 0x564769c635b0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c011790 tx=0x7f8e2c011b90 comp rx=0 tx=0).stop 2024-09-19T09:22:12.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.904+0000 7f8e3d704740 1 -- 172.21.15.120:0/1637660441 shutdown_connections 2024-09-19T09:22:12.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.904+0000 7f8e3d704740 1 --2- 172.21.15.120:0/1637660441 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x564769c73450 0x564769c75840 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.904+0000 7f8e1cff9640 0 client.21834 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:22:12.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.904+0000 7f8e3d704740 1 --2- 172.21.15.120:0/1637660441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564769c5fa40 0x564769c635b0 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.904+0000 7f8e3d704740 1 --2- 172.21.15.120:0/1637660441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564769c5adf0 0x564769c63070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.904+0000 7f8e3d704740 1 --2- 172.21.15.120:0/1637660441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564769c5a440 0x564769c65dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.904+0000 7f8e3d704740 1 -- 172.21.15.120:0/1637660441 >> 172.21.15.120:0/1637660441 conn(0x564769c19310 msgr2=0x564769c59a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:12.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.905+0000 7f8e3d704740 1 -- 172.21.15.120:0/1637660441 shutdown_connections 2024-09-19T09:22:12.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.905+0000 7f8e3d704740 1 -- 172.21.15.120:0/1637660441 wait complete. 2024-09-19T09:22:12.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.905+0000 7f8e3d704740 20 client.21834 trim_cache size 0 max 16384 2024-09-19T09:22:12.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.924+0000 7f8e3d704740 1 Processor -- start 2024-09-19T09:22:12.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.924+0000 7f8e3d704740 1 -- start start 2024-09-19T09:22:12.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.925+0000 7f8e3d704740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564769c5a440 0x564769c7b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:12.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.925+0000 7f8e3d704740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564769c5adf0 0x564769c7b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:12.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.925+0000 7f8e3d704740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564769c5fa40 0x564769c648d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:12.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.925+0000 7f8e3d704740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8e341ab530 con 0x564769c5adf0 2024-09-19T09:22:12.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.925+0000 7f8e3d704740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8e341ad070 con 0x564769c5a440 2024-09-19T09:22:12.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.925+0000 7f8e3d704740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e341abc00 con 0x564769c5fa40 2024-09-19T09:22:12.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.925+0000 7f8e390c7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564769c5a440 0x564769c7b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:12.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.925+0000 7f8e390c7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564769c5a440 0x564769c7b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:57886/0 (socket says 172.21.15.120:57886) 2024-09-19T09:22:12.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.925+0000 7f8e390c7640 1 -- 172.21.15.120:0/1410154940 learned_addr learned my addr 172.21.15.120:0/1410154940 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:12.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.925+0000 7f8e388c6640 1 --2- 172.21.15.120:0/1410154940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564769c5adf0 0x564769c7b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:12.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.925+0000 7f8e398c8640 1 --2- 172.21.15.120:0/1410154940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564769c5fa40 0x564769c648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:12.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.925+0000 7f8e390c7640 1 -- 172.21.15.120:0/1410154940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564769c5fa40 msgr2=0x564769c648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:12.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.925+0000 7f8e390c7640 1 --2- 172.21.15.120:0/1410154940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564769c5fa40 0x564769c648d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.925+0000 7f8e390c7640 1 -- 172.21.15.120:0/1410154940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564769c5adf0 msgr2=0x564769c7b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:12.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.925+0000 7f8e390c7640 1 --2- 172.21.15.120:0/1410154940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564769c5adf0 0x564769c7b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.925+0000 7f8e390c7640 1 -- 172.21.15.120:0/1410154940 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564769c4cfd0 con 0x564769c5a440 2024-09-19T09:22:12.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.925+0000 7f8e388c6640 1 --2- 172.21.15.120:0/1410154940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564769c5adf0 0x564769c7b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:22:12.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.926+0000 7f8e390c7640 1 --2- 172.21.15.120:0/1410154940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564769c5a440 0x564769c7b0c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000bfd0 tx=0x7f8e30005410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da2d343ab2ceba57 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:12.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.926+0000 7f8e1effd640 1 -- 172.21.15.120:0/1410154940 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e30004be0 con 0x564769c5a440 2024-09-19T09:22:12.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.926+0000 7f8e1effd640 1 -- 172.21.15.120:0/1410154940 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e30004d80 con 0x564769c5a440 2024-09-19T09:22:12.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.926+0000 7f8e1effd640 1 -- 172.21.15.120:0/1410154940 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e30011440 con 0x564769c5a440 2024-09-19T09:22:12.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.927+0000 7f8e3d704740 1 -- 172.21.15.120:0/1410154940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564769c5a440 msgr2=0x564769c7b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:12.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.927+0000 7f8e3d704740 1 --2- 172.21.15.120:0/1410154940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564769c5a440 0x564769c7b0c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000bfd0 tx=0x7f8e30005410 comp rx=0 tx=0).stop 2024-09-19T09:22:12.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.927+0000 7f8e3d704740 1 -- 172.21.15.120:0/1410154940 shutdown_connections 2024-09-19T09:22:12.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.927+0000 7f8e3d704740 1 --2- 172.21.15.120:0/1410154940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564769c5fa40 0x564769c648d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.927+0000 7f8e3d704740 1 --2- 172.21.15.120:0/1410154940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564769c5adf0 0x564769c7b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.927+0000 7f8e3d704740 1 --2- 172.21.15.120:0/1410154940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564769c5a440 0x564769c7b0c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.928+0000 7f8e3d704740 1 -- 172.21.15.120:0/1410154940 >> 172.21.15.120:0/1410154940 conn(0x564769c19310 msgr2=0x564769c59a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:12.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.928+0000 7f8e3d704740 1 -- 172.21.15.120:0/1410154940 shutdown_connections 2024-09-19T09:22:12.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.928+0000 7f8e3d704740 1 -- 172.21.15.120:0/1410154940 wait complete. 2024-09-19T09:22:12.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.928+0000 7f8e3d704740 1 -- start start 2024-09-19T09:22:12.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.928+0000 7f8e3d704740 1 Processor -- start 2024-09-19T09:22:12.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.929+0000 7f8e3d704740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564769c5a440 0x564769c6ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:12.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.929+0000 7f8e3d704740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564769c5adf0 0x564769c6bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:12.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.929+0000 7f8e3d704740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564769c5fa40 0x564769c68a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:12.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.930+0000 7f8e390c7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564769c5a440 0x564769c6ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:12.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.930+0000 7f8e390c7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564769c5a440 0x564769c6ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:38438/0 (socket says 172.21.15.120:38438) 2024-09-19T09:22:12.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.930+0000 7f8e388c6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564769c5adf0 0x564769c6bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:12.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.930+0000 7f8e388c6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564769c5adf0 0x564769c6bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43116/0 (socket says 172.21.15.120:43116) 2024-09-19T09:22:12.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.930+0000 7f8e390c7640 1 -- 172.21.15.120:0/440541762 learned_addr learned my addr 172.21.15.120:0/440541762 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:12.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.930+0000 7f8e3d704740 1 -- 172.21.15.120:0/440541762 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8e341ac9a0 con 0x564769c5adf0 2024-09-19T09:22:12.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.930+0000 7f8e3d704740 1 -- 172.21.15.120:0/440541762 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8e341ac2d0 con 0x564769c5fa40 2024-09-19T09:22:12.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.930+0000 7f8e3d704740 1 -- 172.21.15.120:0/440541762 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e341a94f0 con 0x564769c5a440 2024-09-19T09:22:12.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.930+0000 7f8e398c8640 1 --2- 172.21.15.120:0/440541762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564769c5fa40 0x564769c68a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:12.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.930+0000 7f8e3d704740 10 client.0 _ll_register_callbacks cb 0x7f8e2c015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:22:12.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.930+0000 7f8e388c6640 1 -- 172.21.15.120:0/440541762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564769c5a440 msgr2=0x564769c6ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:12.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.930+0000 7f8e388c6640 1 --2- 172.21.15.120:0/440541762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564769c5a440 0x564769c6ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.930+0000 7f8e388c6640 1 -- 172.21.15.120:0/440541762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564769c5fa40 msgr2=0x564769c68a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:12.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.930+0000 7f8e388c6640 1 --2- 172.21.15.120:0/440541762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564769c5fa40 0x564769c68a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.930+0000 7f8e388c6640 1 -- 172.21.15.120:0/440541762 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564769c68f70 con 0x564769c5adf0 2024-09-19T09:22:12.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.930+0000 7f8e388c6640 1 --2- 172.21.15.120:0/440541762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564769c5adf0 0x564769c6bfa0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f8e240f3e60 tx=0x7f8e2402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d79c3bbc45be27a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:12.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.931+0000 7f8e1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:22:12.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.931+0000 7f8e1cff9640 1 -- 172.21.15.120:0/440541762 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24049070 con 0x564769c5adf0 2024-09-19T09:22:12.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.931+0000 7f8e3d704740 1 -- 172.21.15.120:0/440541762 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e30004d80 con 0x564769c5adf0 2024-09-19T09:22:12.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.931+0000 7f8e1cff9640 1 -- 172.21.15.120:0/440541762 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f8e24003c80 con 0x564769c5adf0 2024-09-19T09:22:12.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.931+0000 7f8e1cff9640 1 -- 172.21.15.120:0/440541762 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8e240ff040 con 0x564769c5adf0 2024-09-19T09:22:12.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.931+0000 7f8e3d704740 10 client.21840 fetch_fsmap learned FSMap version 39 2024-09-19T09:22:12.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.931+0000 7f8e3d704740 1 -- 172.21.15.120:0/440541762 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564769cde6b0 con 0x564769c5adf0 2024-09-19T09:22:12.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.931+0000 7f8e1cff9640 1 -- 172.21.15.120:0/440541762 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e240f3350 con 0x564769c5adf0 2024-09-19T09:22:12.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.931+0000 7f8e1cff9640 1 -- 172.21.15.120:0/440541762 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f8e240b8a30 con 0x564769c5adf0 2024-09-19T09:22:12.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.931+0000 7f8e3d704740 10 client.21840 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:22:12.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.931+0000 7f8e3d704740 10 client.21840 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:22:12.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.931+0000 7f8e3d704740 20 client.21840 populate_metadata read hostname 'smithi120' 2024-09-19T09:22:12.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.932+0000 7f8e3d704740 1 --2- 172.21.15.120:0/440541762 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x564769c73450 0x564769ce0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:12.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.932+0000 7f8e3d704740 4 client.21840 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:22:12.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.932+0000 7f8e3d704740 1 -- 172.21.15.120:0/440541762 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f8e30004d80 con 0x564769c73450 2024-09-19T09:22:12.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.932+0000 7f8e390c7640 1 --2- 172.21.15.120:0/440541762 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x564769c73450 0x564769ce0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:12.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.932+0000 7f8e390c7640 1 --2- 172.21.15.120:0/440541762 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x564769c73450 0x564769ce0c90 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5b14e47404f2c98e server_cookie=eb0472c53607bf26 in_seq=0 out_seq=0 2024-09-19T09:22:12.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.932+0000 7f8e1cff9640 10 client.21840 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:22:12.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.932+0000 7f8e1cff9640 1 -- 172.21.15.120:0/440541762 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8e30004d80 con 0x564769c73450 2024-09-19T09:22:12.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.932+0000 7f8e1cff9640 10 client.21840 handle_command_reply: tid=0 2024-09-19T09:22:12.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.932+0000 7f8e3d704740 1 client.21840 shutdown 2024-09-19T09:22:12.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.933+0000 7f8e3d704740 1 -- 172.21.15.120:0/440541762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564769c5adf0 msgr2=0x564769c6bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:12.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.933+0000 7f8e3d704740 1 --2- 172.21.15.120:0/440541762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564769c5adf0 0x564769c6bfa0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f8e240f3e60 tx=0x7f8e2402a420 comp rx=0 tx=0).stop 2024-09-19T09:22:12.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.933+0000 7f8e3d704740 1 -- 172.21.15.120:0/440541762 shutdown_connections 2024-09-19T09:22:12.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.933+0000 7f8e3d704740 1 --2- 172.21.15.120:0/440541762 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x564769c73450 0x564769ce0c90 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.933+0000 7f8e3d704740 1 --2- 172.21.15.120:0/440541762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564769c5fa40 0x564769c68a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.933+0000 7f8e3d704740 1 --2- 172.21.15.120:0/440541762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564769c5adf0 0x564769c6bfa0 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.933+0000 7f8e3d704740 1 --2- 172.21.15.120:0/440541762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564769c5a440 0x564769c6ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.938 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:22:12.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.933+0000 7f8e3d704740 1 -- 172.21.15.120:0/440541762 >> 172.21.15.120:0/440541762 conn(0x564769c19310 msgr2=0x564769c4b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:12.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.933+0000 7f8e1cff9640 0 client.21840 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:22:12.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.933+0000 7f8e3d704740 1 -- 172.21.15.120:0/440541762 shutdown_connections 2024-09-19T09:22:12.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.933+0000 7f8e3d704740 1 -- 172.21.15.120:0/440541762 wait complete. 2024-09-19T09:22:12.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.933+0000 7f8e3d704740 20 client.21840 trim_cache size 0 max 16384 2024-09-19T09:22:12.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.934+0000 7f8e3b32b640 1 -- 172.21.15.120:0/1508520226 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8e200a32c0 msgr2=0x7f8e200a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:12.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.934+0000 7f8e3b32b640 1 --2- 172.21.15.120:0/1508520226 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8e200a32c0 0x7f8e200a5780 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c005fd0 tx=0x7f8e2c009560 comp rx=0 tx=0).stop 2024-09-19T09:22:12.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.934+0000 7f8e3b32b640 1 -- 172.21.15.120:0/1508520226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e34101420 msgr2=0x7f8e341989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:12.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.934+0000 7f8e3b32b640 1 --2- 172.21.15.120:0/1508520226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e34101420 0x7f8e341989a0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f8e24009230 tx=0x7f8e24004270 comp rx=0 tx=0).stop 2024-09-19T09:22:12.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.935+0000 7f8e3b32b640 1 -- 172.21.15.120:0/1508520226 shutdown_connections 2024-09-19T09:22:12.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.935+0000 7f8e3b32b640 1 --2- 172.21.15.120:0/1508520226 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8e200a32c0 0x7f8e200a5780 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.935+0000 7f8e3b32b640 1 --2- 172.21.15.120:0/1508520226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e34106a80 0x7f8e3419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.935+0000 7f8e3b32b640 1 --2- 172.21.15.120:0/1508520226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e34101420 0x7f8e341989a0 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.935+0000 7f8e3b32b640 1 --2- 172.21.15.120:0/1508520226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e34100a80 0x7f8e34198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:12.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.935+0000 7f8e3b32b640 1 -- 172.21.15.120:0/1508520226 >> 172.21.15.120:0/1508520226 conn(0x7f8e340fc7a0 msgr2=0x7f8e34104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:12.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.935+0000 7f8e3b32b640 1 -- 172.21.15.120:0/1508520226 shutdown_connections 2024-09-19T09:22:12.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:12.935+0000 7f8e3b32b640 1 -- 172.21.15.120:0/1508520226 wait complete. 2024-09-19T09:22:13.372 INFO:tasks.workunit.client.0.smithi120.stdout: 10 155761 21.79 MB/sec execute 248 sec latency 52.199 ms 2024-09-19T09:22:13.880 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:22:14.196 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:22:14.320 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:14 smithi120 ceph-mon[31150]: pgmap v926: 97 pgs: 97 active+clean; 382 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.6 MiB/s wr, 553 op/s 2024-09-19T09:22:14.372 INFO:tasks.workunit.client.0.smithi120.stdout: 10 156256 21.82 MB/sec execute 249 sec latency 49.661 ms 2024-09-19T09:22:14.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:14 smithi196 ceph-mon[35150]: pgmap v926: 97 pgs: 97 active+clean; 382 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.6 MiB/s wr, 553 op/s 2024-09-19T09:22:14.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:14 smithi153 ceph-mon[35071]: pgmap v926: 97 pgs: 97 active+clean; 382 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.6 MiB/s wr, 553 op/s 2024-09-19T09:22:15.372 INFO:tasks.workunit.client.0.smithi120.stdout: 10 156794 21.84 MB/sec execute 250 sec latency 128.299 ms 2024-09-19T09:22:15.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.854+0000 7fb9fbe4e640 1 -- 172.21.15.120:0/2493624894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9f4109450 msgr2=0x7fb9f4110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:15.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.854+0000 7fb9fbe4e640 1 --2- 172.21.15.120:0/2493624894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9f4109450 0x7fb9f4110250 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec009ca0 tx=0x7fb9ec02a8f0 comp rx=0 tx=0).stop 2024-09-19T09:22:15.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.861+0000 7fb9fbe4e640 1 -- 172.21.15.120:0/2493624894 shutdown_connections 2024-09-19T09:22:15.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.861+0000 7fb9fbe4e640 1 --2- 172.21.15.120:0/2493624894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9f411b140 0x7fb9f411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:15.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.861+0000 7fb9fbe4e640 1 --2- 172.21.15.120:0/2493624894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9f4109830 0x7fb9f4109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:15.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.861+0000 7fb9fbe4e640 1 --2- 172.21.15.120:0/2493624894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9f4109450 0x7fb9f4110250 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:15.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.861+0000 7fb9fbe4e640 1 -- 172.21.15.120:0/2493624894 >> 172.21.15.120:0/2493624894 conn(0x7fb9f406e140 msgr2=0x7fb9f406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:15.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.863+0000 7fb9fbe4e640 1 -- 172.21.15.120:0/2493624894 shutdown_connections 2024-09-19T09:22:15.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.865+0000 7fb9fbe4e640 1 -- 172.21.15.120:0/2493624894 wait complete. 2024-09-19T09:22:15.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.866+0000 7fb9fbe4e640 1 Processor -- start 2024-09-19T09:22:15.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.866+0000 7fb9fbe4e640 1 -- start start 2024-09-19T09:22:15.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.867+0000 7fb9fbe4e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9f4109450 0x7fb9f411a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:15.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.867+0000 7fb9fbe4e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9f4109830 0x7fb9f41149d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:15.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.867+0000 7fb9fbe4e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9f411b140 0x7fb9f4114f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:15.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.867+0000 7fb9fbe4e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f4111370 con 0x7fb9f411b140 2024-09-19T09:22:15.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.867+0000 7fb9fbe4e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f41111f0 con 0x7fb9f4109450 2024-09-19T09:22:15.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.867+0000 7fb9fbe4e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f41114f0 con 0x7fb9f4109830 2024-09-19T09:22:15.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.869+0000 7fb9f9bea640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9f4109450 0x7fb9f411a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:15.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.869+0000 7fb9f9bea640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9f4109450 0x7fb9f411a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:57904/0 (socket says 172.21.15.120:57904) 2024-09-19T09:22:15.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.869+0000 7fb9f9bea640 1 -- 172.21.15.120:0/1990899334 learned_addr learned my addr 172.21.15.120:0/1990899334 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:15.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.869+0000 7fb9fa3eb640 1 --2- 172.21.15.120:0/1990899334 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9f411b140 0x7fb9f4114f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:15.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.872+0000 7fb9f9bea640 1 -- 172.21.15.120:0/1990899334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9f4109830 msgr2=0x7fb9f41149d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T09:22:15.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.872+0000 7fb9f9bea640 1 --2- 172.21.15.120:0/1990899334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9f4109830 0x7fb9f41149d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:15.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.872+0000 7fb9f9bea640 1 -- 172.21.15.120:0/1990899334 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9f411b140 msgr2=0x7fb9f4114f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:15.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.872+0000 7fb9f9bea640 1 --2- 172.21.15.120:0/1990899334 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9f411b140 0x7fb9f4114f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:15.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.872+0000 7fb9f9bea640 1 -- 172.21.15.120:0/1990899334 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f41157d0 con 0x7fb9f4109450 2024-09-19T09:22:15.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.875+0000 7fb9f9bea640 1 --2- 172.21.15.120:0/1990899334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9f4109450 0x7fb9f411a950 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec006010 tx=0x7fb9ec0316b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:15.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.880+0000 7fb9eaffd640 1 -- 172.21.15.120:0/1990899334 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec004040 con 0x7fb9f4109450 2024-09-19T09:22:15.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.880+0000 7fb9eaffd640 1 -- 172.21.15.120:0/1990899334 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9ec02d070 con 0x7fb9f4109450 2024-09-19T09:22:15.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.880+0000 7fb9eaffd640 1 -- 172.21.15.120:0/1990899334 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec02fc40 con 0x7fb9f4109450 2024-09-19T09:22:15.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.880+0000 7fb9fbe4e640 1 -- 172.21.15.120:0/1990899334 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9f41bc210 con 0x7fb9f4109450 2024-09-19T09:22:15.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.880+0000 7fb9fbe4e640 1 -- 172.21.15.120:0/1990899334 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9f41bc6d0 con 0x7fb9f4109450 2024-09-19T09:22:15.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.887+0000 7fb9fe227740 1 Processor -- start 2024-09-19T09:22:15.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.888+0000 7fb9eaffd640 1 -- 172.21.15.120:0/1990899334 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fb9ec0041e0 con 0x7fb9f4109450 2024-09-19T09:22:15.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.889+0000 7fb9eaffd640 1 --2- 172.21.15.120:0/1990899334 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb9e00a3300 0x7fb9e00a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:15.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.890+0000 7fb9fe227740 1 -- start start 2024-09-19T09:22:15.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.890+0000 7fb9fe227740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578f5a5cb40 0x5578f5a5b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:15.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.890+0000 7fb9fe227740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578f5a5b5d0 0x5578f5a5ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:15.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.890+0000 7fb9fe227740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578f5a66370 0x5578f5a68760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:15.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.890+0000 7fb9fe227740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5578f59c12f0 con 0x5578f5a5cb40 2024-09-19T09:22:15.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.890+0000 7fb9fe227740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5578f5968cf0 con 0x5578f5a66370 2024-09-19T09:22:15.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.890+0000 7fb9fe227740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5578f59bd2b0 con 0x5578f5a5b5d0 2024-09-19T09:22:15.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.891+0000 7fb9f93e9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578f5a5b5d0 0x5578f5a5ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:15.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.891+0000 7fb9f93e9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578f5a5b5d0 0x5578f5a5ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:38452/0 (socket says 172.21.15.120:38452) 2024-09-19T09:22:15.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.891+0000 7fb9f93e9640 1 -- 172.21.15.120:0/3515129841 learned_addr learned my addr 172.21.15.120:0/3515129841 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:15.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.891+0000 7fb9fa3eb640 1 --2- 172.21.15.120:0/3515129841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578f5a66370 0x5578f5a68760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:15.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.892+0000 7fb9fa3eb640 1 --2- 172.21.15.120:0/1990899334 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb9e00a3300 0x7fb9e00a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:15.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.892+0000 7fb9fa3eb640 1 -- 172.21.15.120:0/3515129841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578f5a5b5d0 msgr2=0x5578f5a5ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:15.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.892+0000 7fb9fa3eb640 1 --2- 172.21.15.120:0/3515129841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578f5a5b5d0 0x5578f5a5ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:15.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.892+0000 7fb9fa3eb640 1 -- 172.21.15.120:0/3515129841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578f5a5cb40 msgr2=0x5578f5a5b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:15.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.892+0000 7fb9fa3eb640 1 --2- 172.21.15.120:0/3515129841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578f5a5cb40 0x5578f5a5b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:15.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.893+0000 7fb9fa3eb640 1 -- 172.21.15.120:0/3515129841 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578f594d3a0 con 0x5578f5a66370 2024-09-19T09:22:15.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.893+0000 7fb9fa3eb640 1 --2- 172.21.15.120:0/3515129841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578f5a66370 0x5578f5a68760 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4015410 tx=0x7fb9e4009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de0d4a1dfcc83aee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:15.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.894+0000 7fb9eaffd640 1 -- 172.21.15.120:0/1990899334 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fb9ec0f11d0 con 0x7fb9f4109450 2024-09-19T09:22:15.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.897+0000 7fb9dffff640 1 -- 172.21.15.120:0/3515129841 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4017e20 con 0x5578f5a66370 2024-09-19T09:22:15.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.897+0000 7fb9dffff640 1 -- 172.21.15.120:0/3515129841 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9e401b070 con 0x5578f5a66370 2024-09-19T09:22:15.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.897+0000 7fb9dffff640 1 -- 172.21.15.120:0/3515129841 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e40168b0 con 0x5578f5a66370 2024-09-19T09:22:15.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.899+0000 7fb9fe227740 1 -- 172.21.15.120:0/3515129841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578f5a66370 msgr2=0x5578f5a68760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:15.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.899+0000 7fb9fe227740 1 --2- 172.21.15.120:0/3515129841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578f5a66370 0x5578f5a68760 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4015410 tx=0x7fb9e4009f90 comp rx=0 tx=0).stop 2024-09-19T09:22:15.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.902+0000 7fb9fa3eb640 1 --2- 172.21.15.120:0/1990899334 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb9e00a3300 0x7fb9e00a57c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4020000 tx=0x7fb9e4019000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:15.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.904+0000 7fb9fe227740 1 -- 172.21.15.120:0/3515129841 shutdown_connections 2024-09-19T09:22:15.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.904+0000 7fb9fe227740 1 --2- 172.21.15.120:0/3515129841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578f5a66370 0x5578f5a68760 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:15.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.904+0000 7fb9fe227740 1 --2- 172.21.15.120:0/3515129841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578f5a5b5d0 0x5578f5a5ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:15.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.904+0000 7fb9fe227740 1 --2- 172.21.15.120:0/3515129841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578f5a5cb40 0x5578f5a5b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:15.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.904+0000 7fb9fe227740 1 -- 172.21.15.120:0/3515129841 >> 172.21.15.120:0/3515129841 conn(0x5578f5a24310 msgr2=0x5578f5a57c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:15.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.910+0000 7fb9fe227740 1 -- 172.21.15.120:0/3515129841 shutdown_connections 2024-09-19T09:22:15.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.912+0000 7fb9fe227740 1 -- 172.21.15.120:0/3515129841 wait complete. 2024-09-19T09:22:15.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.913+0000 7fb9fe227740 1 -- start start 2024-09-19T09:22:15.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.913+0000 7fb9fe227740 1 Processor -- start 2024-09-19T09:22:15.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.921+0000 7fb9fe227740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578f5a5b5d0 0x5578f5a65e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:15.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.921+0000 7fb9fe227740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578f5a5cb40 0x5578f5a630d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:15.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.922+0000 7fb9fe227740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578f5a66370 0x5578f5a63860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:15.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.922+0000 7fb9fe227740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5578f5a69070 con 0x5578f5a5b5d0 2024-09-19T09:22:15.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.922+0000 7fb9fe227740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5578f597c060 con 0x5578f5a5cb40 2024-09-19T09:22:15.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.922+0000 7fb9fe227740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5578f5a691f0 con 0x5578f5a66370 2024-09-19T09:22:15.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.922+0000 7fb9fe227740 10 client.0 _ll_register_callbacks cb 0x5578f56cf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:22:15.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.922+0000 7fb9fa3eb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578f5a66370 0x5578f5a63860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:15.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.922+0000 7fb9fa3eb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578f5a66370 0x5578f5a63860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:38466/0 (socket says 172.21.15.120:38466) 2024-09-19T09:22:15.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.922+0000 7fb9fa3eb640 1 -- 172.21.15.120:0/3194825189 learned_addr learned my addr 172.21.15.120:0/3194825189 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:15.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.922+0000 7fb9f93e9640 1 --2- 172.21.15.120:0/3194825189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578f5a5cb40 0x5578f5a630d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:15.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.923+0000 7fb9fa3eb640 1 -- 172.21.15.120:0/3194825189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578f5a5cb40 msgr2=0x5578f5a630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:15.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.923+0000 7fb9fa3eb640 1 --2- 172.21.15.120:0/3194825189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578f5a5cb40 0x5578f5a630d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:15.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.923+0000 7fb9fa3eb640 1 -- 172.21.15.120:0/3194825189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578f5a5b5d0 msgr2=0x5578f5a65e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:22:15.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.923+0000 7fb9fa3eb640 1 --2- 172.21.15.120:0/3194825189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578f5a5b5d0 0x5578f5a65e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:15.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.923+0000 7fb9fa3eb640 1 -- 172.21.15.120:0/3194825189 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578f57323c0 con 0x5578f5a66370 2024-09-19T09:22:15.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.925+0000 7fb9fa3eb640 1 --2- 172.21.15.120:0/3194825189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578f5a66370 0x5578f5a63860 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb9e40090c0 tx=0x7fb9e4016d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32d80ca540ed0fcb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:15.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.925+0000 7fb9dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:22:15.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.928+0000 7fb9dd7fa640 1 -- 172.21.15.120:0/3194825189 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e402f070 con 0x5578f5a66370 2024-09-19T09:22:15.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.928+0000 7fb9fe227740 1 -- 172.21.15.120:0/3194825189 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9e401b070 con 0x5578f5a66370 2024-09-19T09:22:15.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.928+0000 7fb9dd7fa640 1 -- 172.21.15.120:0/3194825189 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fb9e40217c0 con 0x5578f5a66370 2024-09-19T09:22:15.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.931+0000 7fb9dd7fa640 1 -- 172.21.15.120:0/3194825189 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb9e4039050 con 0x5578f5a66370 2024-09-19T09:22:15.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.931+0000 7fb9fe227740 10 client.30505 fetch_fsmap learned FSMap version 39 2024-09-19T09:22:15.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.931+0000 7fb9fe227740 1 -- 172.21.15.120:0/3194825189 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5578f5a74c30 con 0x5578f5a66370 2024-09-19T09:22:15.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.935+0000 7fb9dd7fa640 1 -- 172.21.15.120:0/3194825189 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9e401b070 con 0x5578f5a66370 2024-09-19T09:22:15.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.935+0000 7fb9dd7fa640 1 -- 172.21.15.120:0/3194825189 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fb9e402a070 con 0x5578f5a66370 2024-09-19T09:22:15.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.937+0000 7fb9fe227740 10 client.30505 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:22:15.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.937+0000 7fb9fe227740 10 client.30505 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:22:15.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.937+0000 7fb9fe227740 20 client.30505 populate_metadata read hostname 'smithi120' 2024-09-19T09:22:15.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.937+0000 7fb9fe227740 1 --2- 172.21.15.120:0/3194825189 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5578f5a76040 0x5578f5a78430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:15.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.937+0000 7fb9fe227740 4 client.30505 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:22:15.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.937+0000 7fb9fe227740 1 -- 172.21.15.120:0/3194825189 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578f59ce860 con 0x5578f5a76040 2024-09-19T09:22:15.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.945+0000 7fb9f9bea640 1 --2- 172.21.15.120:0/3194825189 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5578f5a76040 0x5578f5a78430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:15.956 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 34.0 2024-09-19T09:22:15.957 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:22:15.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.958+0000 7fb9f9bea640 1 --2- 172.21.15.120:0/3194825189 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5578f5a76040 0x5578f5a78430 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a933b0c85c6a6e35 server_cookie=2a0d9c361edf780b in_seq=0 out_seq=0 2024-09-19T09:22:15.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.959+0000 7fb9dd7fa640 10 client.30505 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:22:15.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.959+0000 7fb9dd7fa640 1 -- 172.21.15.120:0/3194825189 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5578f59ce860 con 0x5578f5a76040 2024-09-19T09:22:15.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.959+0000 7fb9dd7fa640 10 client.30505 handle_command_reply: tid=0 2024-09-19T09:22:15.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.962+0000 7fb9fe227740 1 client.30505 shutdown 2024-09-19T09:22:15.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.967+0000 7fb9fe227740 1 -- 172.21.15.120:0/3194825189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578f5a66370 msgr2=0x5578f5a63860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:15.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.967+0000 7fb9fe227740 1 --2- 172.21.15.120:0/3194825189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578f5a66370 0x5578f5a63860 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb9e40090c0 tx=0x7fb9e4016d40 comp rx=0 tx=0).stop 2024-09-19T09:22:15.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.967+0000 7fb9fe227740 1 -- 172.21.15.120:0/3194825189 shutdown_connections 2024-09-19T09:22:15.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.967+0000 7fb9fe227740 1 --2- 172.21.15.120:0/3194825189 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5578f5a76040 0x5578f5a78430 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:15.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.967+0000 7fb9fe227740 1 --2- 172.21.15.120:0/3194825189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578f5a66370 0x5578f5a63860 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:15.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.967+0000 7fb9fe227740 1 --2- 172.21.15.120:0/3194825189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578f5a5cb40 0x5578f5a630d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:15.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.967+0000 7fb9fe227740 1 --2- 172.21.15.120:0/3194825189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578f5a5b5d0 0x5578f5a65e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:15.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.967+0000 7fb9fe227740 1 -- 172.21.15.120:0/3194825189 >> 172.21.15.120:0/3194825189 conn(0x5578f5a24310 msgr2=0x5578f5a5d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:15.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.967+0000 7fb9dd7fa640 0 client.30505 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:22:15.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.968+0000 7fb9fe227740 1 -- 172.21.15.120:0/3194825189 shutdown_connections 2024-09-19T09:22:15.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.970+0000 7fb9fe227740 1 -- 172.21.15.120:0/3194825189 wait complete. 2024-09-19T09:22:15.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.970+0000 7fb9fe227740 20 client.30505 trim_cache size 0 max 16384 2024-09-19T09:22:16.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.998+0000 7fb9fe227740 1 Processor -- start 2024-09-19T09:22:16.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.998+0000 7fb9fe227740 1 -- start start 2024-09-19T09:22:16.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.999+0000 7fb9fe227740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578f5a5b5d0 0x5578f5a57d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:16.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.999+0000 7fb9fe227740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578f5a5cb40 0x5578f5a620b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:16.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.999+0000 7fb9fe227740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578f5a66370 0x5578f5a7dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:16.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.999+0000 7fb9f9bea640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578f5a5b5d0 0x5578f5a57d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:16.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:15.999+0000 7fb9f9bea640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578f5a5b5d0 0x5578f5a57d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:38482/0 (socket says 172.21.15.120:38482) 2024-09-19T09:22:16.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.000+0000 7fb9f9bea640 1 -- 172.21.15.120:0/3774247599 learned_addr learned my addr 172.21.15.120:0/3774247599 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:16.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.000+0000 7fb9fe227740 1 -- 172.21.15.120:0/3774247599 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f41c4e20 con 0x5578f5a66370 2024-09-19T09:22:16.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.000+0000 7fb9fe227740 1 -- 172.21.15.120:0/3774247599 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f41c6960 con 0x5578f5a5cb40 2024-09-19T09:22:16.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.000+0000 7fb9fe227740 1 -- 172.21.15.120:0/3774247599 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f41c54f0 con 0x5578f5a5b5d0 2024-09-19T09:22:16.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.000+0000 7fb9fa3eb640 1 --2- 172.21.15.120:0/3774247599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578f5a66370 0x5578f5a7dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:16.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.000+0000 7fb9f9bea640 1 -- 172.21.15.120:0/3774247599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578f5a5cb40 msgr2=0x5578f5a620b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:22:16.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.000+0000 7fb9f9bea640 1 --2- 172.21.15.120:0/3774247599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578f5a5cb40 0x5578f5a620b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:16.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.000+0000 7fb9f9bea640 1 -- 172.21.15.120:0/3774247599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578f5a66370 msgr2=0x5578f5a7dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:16.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.000+0000 7fb9f9bea640 1 --2- 172.21.15.120:0/3774247599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578f5a66370 0x5578f5a7dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:16.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.000+0000 7fb9f9bea640 1 -- 172.21.15.120:0/3774247599 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578f5a5e120 con 0x5578f5a5b5d0 2024-09-19T09:22:16.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.001+0000 7fb9f9bea640 1 --2- 172.21.15.120:0/3774247599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578f5a5b5d0 0x5578f5a57d70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec0eecd0 tx=0x7fb9ec02a8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cc7a30cfa27351e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:16.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.002+0000 7fb9df7fe640 1 -- 172.21.15.120:0/3774247599 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec0d5030 con 0x5578f5a5b5d0 2024-09-19T09:22:16.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.002+0000 7fb9df7fe640 1 -- 172.21.15.120:0/3774247599 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9ec036030 con 0x5578f5a5b5d0 2024-09-19T09:22:16.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.003+0000 7fb9fe227740 1 -- 172.21.15.120:0/3774247599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578f5a5b5d0 msgr2=0x5578f5a57d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:16.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.003+0000 7fb9fe227740 1 --2- 172.21.15.120:0/3774247599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578f5a5b5d0 0x5578f5a57d70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec0eecd0 tx=0x7fb9ec02a8f0 comp rx=0 tx=0).stop 2024-09-19T09:22:16.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.004+0000 7fb9fe227740 1 -- 172.21.15.120:0/3774247599 shutdown_connections 2024-09-19T09:22:16.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.004+0000 7fb9fe227740 1 --2- 172.21.15.120:0/3774247599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578f5a66370 0x5578f5a7dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:16.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.004+0000 7fb9fe227740 1 --2- 172.21.15.120:0/3774247599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578f5a5cb40 0x5578f5a620b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:16.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.004+0000 7fb9fe227740 1 --2- 172.21.15.120:0/3774247599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578f5a5b5d0 0x5578f5a57d70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:16.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.004+0000 7fb9fe227740 1 -- 172.21.15.120:0/3774247599 >> 172.21.15.120:0/3774247599 conn(0x5578f5a24310 msgr2=0x5578f5a619a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:16.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.008+0000 7fb9fe227740 1 -- 172.21.15.120:0/3774247599 shutdown_connections 2024-09-19T09:22:16.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.008+0000 7fb9fe227740 1 -- 172.21.15.120:0/3774247599 wait complete. 2024-09-19T09:22:16.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.009+0000 7fb9fe227740 1 -- start start 2024-09-19T09:22:16.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.009+0000 7fb9fe227740 1 Processor -- start 2024-09-19T09:22:16.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.010+0000 7fb9fe227740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578f5a5b5d0 0x5578f5a6ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:16.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.010+0000 7fb9fe227740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578f5a5cb40 0x5578f5a5fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:16.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.010+0000 7fb9fe227740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578f5a66370 0x5578f5a60340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:16.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.010+0000 7fb9fe227740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f41c6290 con 0x5578f5a5b5d0 2024-09-19T09:22:16.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.010+0000 7fb9fe227740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f41c5bc0 con 0x5578f5a5cb40 2024-09-19T09:22:16.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.010+0000 7fb9fe227740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f41c2de0 con 0x5578f5a66370 2024-09-19T09:22:16.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.010+0000 7fb9fe227740 10 client.0 _ll_register_callbacks cb 0x7fb9f0006f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:22:16.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.010+0000 7fb9f93e9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578f5a5cb40 0x5578f5a5fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:16.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.011+0000 7fb9fa3eb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578f5a66370 0x5578f5a60340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:16.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.011+0000 7fb9fa3eb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578f5a66370 0x5578f5a60340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:38492/0 (socket says 172.21.15.120:38492) 2024-09-19T09:22:16.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.011+0000 7fb9fa3eb640 1 -- 172.21.15.120:0/3286020096 learned_addr learned my addr 172.21.15.120:0/3286020096 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:16.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.011+0000 7fb9f9bea640 1 --2- 172.21.15.120:0/3286020096 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578f5a5b5d0 0x5578f5a6ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:16.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.011+0000 7fb9fa3eb640 1 -- 172.21.15.120:0/3286020096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578f5a5cb40 msgr2=0x5578f5a5fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:16.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.011+0000 7fb9fa3eb640 1 --2- 172.21.15.120:0/3286020096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578f5a5cb40 0x5578f5a5fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:16.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.011+0000 7fb9fa3eb640 1 -- 172.21.15.120:0/3286020096 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578f5a5b5d0 msgr2=0x5578f5a6ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:16.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.011+0000 7fb9fa3eb640 1 --2- 172.21.15.120:0/3286020096 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578f5a5b5d0 0x5578f5a6ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:16.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.011+0000 7fb9fa3eb640 1 -- 172.21.15.120:0/3286020096 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578f5a60b70 con 0x5578f5a66370 2024-09-19T09:22:16.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.019+0000 7fb9fa3eb640 1 --2- 172.21.15.120:0/3286020096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578f5a66370 0x5578f5a60340 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4006d80 tx=0x7fb9e40165e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98a6e049851f03 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:16.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.019+0000 7fb9dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:22:16.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.019+0000 7fb9dd7fa640 1 -- 172.21.15.120:0/3286020096 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4042020 con 0x5578f5a66370 2024-09-19T09:22:16.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.019+0000 7fb9dd7fa640 1 -- 172.21.15.120:0/3286020096 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fb9e40219c0 con 0x5578f5a66370 2024-09-19T09:22:16.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.019+0000 7fb9fe227740 1 -- 172.21.15.120:0/3286020096 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9ec036030 con 0x5578f5a66370 2024-09-19T09:22:16.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.024+0000 7fb9dd7fa640 1 -- 172.21.15.120:0/3286020096 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb9e402f040 con 0x5578f5a66370 2024-09-19T09:22:16.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.025+0000 7fb9fe227740 10 client.30511 fetch_fsmap learned FSMap version 39 2024-09-19T09:22:16.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.025+0000 7fb9fe227740 1 -- 172.21.15.120:0/3286020096 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5578f5ae0a80 con 0x5578f5a66370 2024-09-19T09:22:16.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.028+0000 7fb9dd7fa640 1 -- 172.21.15.120:0/3286020096 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9ec036030 con 0x5578f5a66370 2024-09-19T09:22:16.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.028+0000 7fb9dd7fa640 1 -- 172.21.15.120:0/3286020096 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fb9e402a040 con 0x5578f5a66370 2024-09-19T09:22:16.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.028+0000 7fb9fe227740 10 client.30511 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:22:16.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.028+0000 7fb9fe227740 10 client.30511 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:22:16.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.028+0000 7fb9fe227740 20 client.30511 populate_metadata read hostname 'smithi120' 2024-09-19T09:22:16.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.028+0000 7fb9fe227740 1 --2- 172.21.15.120:0/3286020096 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5578f5ae10c0 0x5578f5ae34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:16.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.028+0000 7fb9fe227740 4 client.30511 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:22:16.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.028+0000 7fb9fe227740 1 -- 172.21.15.120:0/3286020096 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5578f59c8d70 con 0x5578f5ae10c0 2024-09-19T09:22:16.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.030+0000 7fb9f9bea640 1 --2- 172.21.15.120:0/3286020096 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5578f5ae10c0 0x5578f5ae34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:16.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.032+0000 7fb9f9bea640 1 --2- 172.21.15.120:0/3286020096 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5578f5ae10c0 0x5578f5ae34e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bea10dae0490cf06 server_cookie=2e83356a164361a6 in_seq=0 out_seq=0 2024-09-19T09:22:16.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.032+0000 7fb9dd7fa640 10 client.30511 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:22:16.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.033+0000 7fb9dd7fa640 1 -- 172.21.15.120:0/3286020096 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5578f59c8d70 con 0x5578f5ae10c0 2024-09-19T09:22:16.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.033+0000 7fb9dd7fa640 10 client.30511 handle_command_reply: tid=0 2024-09-19T09:22:16.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.033+0000 7fb9fe227740 1 client.30511 shutdown 2024-09-19T09:22:16.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.033+0000 7fb9fe227740 1 -- 172.21.15.120:0/3286020096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578f5a66370 msgr2=0x5578f5a60340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:16.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.033+0000 7fb9fe227740 1 --2- 172.21.15.120:0/3286020096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578f5a66370 0x5578f5a60340 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4006d80 tx=0x7fb9e40165e0 comp rx=0 tx=0).stop 2024-09-19T09:22:16.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.034+0000 7fb9fe227740 1 -- 172.21.15.120:0/3286020096 shutdown_connections 2024-09-19T09:22:16.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.034+0000 7fb9fe227740 1 --2- 172.21.15.120:0/3286020096 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5578f5ae10c0 0x5578f5ae34e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:16.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.034+0000 7fb9fe227740 1 --2- 172.21.15.120:0/3286020096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578f5a66370 0x5578f5a60340 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:16.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.034+0000 7fb9fe227740 1 --2- 172.21.15.120:0/3286020096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578f5a5cb40 0x5578f5a5fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:16.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.034+0000 7fb9fe227740 1 --2- 172.21.15.120:0/3286020096 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578f5a5b5d0 0x5578f5a6ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:16.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.034+0000 7fb9fe227740 1 -- 172.21.15.120:0/3286020096 >> 172.21.15.120:0/3286020096 conn(0x5578f5a24310 msgr2=0x5578f5a621c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:16.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.034+0000 7fb9dd7fa640 0 client.30511 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:22:16.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.037+0000 7fb9fe227740 1 -- 172.21.15.120:0/3286020096 shutdown_connections 2024-09-19T09:22:16.039 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:22:16.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.037+0000 7fb9fe227740 1 -- 172.21.15.120:0/3286020096 wait complete. 2024-09-19T09:22:16.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.037+0000 7fb9fe227740 20 client.30511 trim_cache size 0 max 16384 2024-09-19T09:22:16.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.038+0000 7fb9fbe4e640 1 -- 172.21.15.120:0/1990899334 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb9e00a3300 msgr2=0x7fb9e00a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:16.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.038+0000 7fb9fbe4e640 1 --2- 172.21.15.120:0/1990899334 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb9e00a3300 0x7fb9e00a57c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4020000 tx=0x7fb9e4019000 comp rx=0 tx=0).stop 2024-09-19T09:22:16.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.038+0000 7fb9fbe4e640 1 -- 172.21.15.120:0/1990899334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9f4109450 msgr2=0x7fb9f411a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:16.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.038+0000 7fb9fbe4e640 1 --2- 172.21.15.120:0/1990899334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9f4109450 0x7fb9f411a950 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec006010 tx=0x7fb9ec0316b0 comp rx=0 tx=0).stop 2024-09-19T09:22:16.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.043+0000 7fb9fbe4e640 1 -- 172.21.15.120:0/1990899334 shutdown_connections 2024-09-19T09:22:16.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.043+0000 7fb9fbe4e640 1 --2- 172.21.15.120:0/1990899334 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb9e00a3300 0x7fb9e00a57c0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:16.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.043+0000 7fb9fbe4e640 1 --2- 172.21.15.120:0/1990899334 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9f411b140 0x7fb9f4114f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:16.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.043+0000 7fb9fbe4e640 1 --2- 172.21.15.120:0/1990899334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9f4109830 0x7fb9f41149d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:16.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.043+0000 7fb9fbe4e640 1 --2- 172.21.15.120:0/1990899334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9f4109450 0x7fb9f411a950 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:16.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.043+0000 7fb9fbe4e640 1 -- 172.21.15.120:0/1990899334 >> 172.21.15.120:0/1990899334 conn(0x7fb9f406e140 msgr2=0x7fb9f4070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:16.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.043+0000 7fb9fbe4e640 1 -- 172.21.15.120:0/1990899334 shutdown_connections 2024-09-19T09:22:16.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:16.043+0000 7fb9fbe4e640 1 -- 172.21.15.120:0/1990899334 wait complete. 2024-09-19T09:22:16.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:16 smithi120 ceph-mon[31150]: pgmap v927: 97 pgs: 97 active+clean; 383 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.3 MiB/s wr, 526 op/s 2024-09-19T09:22:16.373 INFO:tasks.workunit.client.0.smithi120.stdout: 10 156949 21.79 MB/sec execute 251 sec latency 708.946 ms 2024-09-19T09:22:16.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:16 smithi196 ceph-mon[35150]: pgmap v927: 97 pgs: 97 active+clean; 383 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.3 MiB/s wr, 526 op/s 2024-09-19T09:22:16.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:16 smithi153 ceph-mon[35071]: pgmap v927: 97 pgs: 97 active+clean; 383 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.3 MiB/s wr, 526 op/s 2024-09-19T09:22:16.874 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:22:16.916 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:22:17.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:17 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:22:17.373 INFO:tasks.workunit.client.0.smithi120.stdout: 10 157073 21.72 MB/sec execute 252 sec latency 1238.913 ms 2024-09-19T09:22:17.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:17 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:22:17.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:17 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:22:17.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.489+0000 7f59c1188640 1 -- 172.21.15.120:0/1173337668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59bc102ef0 msgr2=0x7f59bc069f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:17.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.489+0000 7f59c1188640 1 --2- 172.21.15.120:0/1173337668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59bc102ef0 0x7f59bc069f70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009ca0 tx=0x7f59b402a950 comp rx=0 tx=0).stop 2024-09-19T09:22:17.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.490+0000 7f59c1188640 1 -- 172.21.15.120:0/1173337668 shutdown_connections 2024-09-19T09:22:17.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.490+0000 7f59c1188640 1 --2- 172.21.15.120:0/1173337668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59bc102ef0 0x7f59bc069f70 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:17.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.490+0000 7f59c1188640 1 --2- 172.21.15.120:0/1173337668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59bc102570 0x7f59bc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:17.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.490+0000 7f59c1188640 1 --2- 172.21.15.120:0/1173337668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59bc100e20 0x7f59bc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:17.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.490+0000 7f59c1188640 1 -- 172.21.15.120:0/1173337668 >> 172.21.15.120:0/1173337668 conn(0x7f59bc0fc7d0 msgr2=0x7f59bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:17.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.490+0000 7f59c1188640 1 -- 172.21.15.120:0/1173337668 shutdown_connections 2024-09-19T09:22:17.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.490+0000 7f59c1188640 1 -- 172.21.15.120:0/1173337668 wait complete. 2024-09-19T09:22:17.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.491+0000 7f59c1188640 1 Processor -- start 2024-09-19T09:22:17.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.491+0000 7f59c1188640 1 -- start start 2024-09-19T09:22:17.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.492+0000 7f59c1188640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59bc100e20 0x7f59bc111530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:17.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.492+0000 7f59c1188640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59bc102570 0x7f59bc111a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:17.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.492+0000 7f59c1188640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59bc102ef0 0x7f59bc111fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:17.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.492+0000 7f59c1188640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc10b6c0 con 0x7f59bc102570 2024-09-19T09:22:17.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.492+0000 7f59c1188640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc10b540 con 0x7f59bc102ef0 2024-09-19T09:22:17.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.492+0000 7f59c1188640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc10b840 con 0x7f59bc100e20 2024-09-19T09:22:17.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.492+0000 7f59ba59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59bc102570 0x7f59bc111a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:17.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.492+0000 7f59bb59e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59bc102ef0 0x7f59bc111fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:17.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.492+0000 7f59bb59e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59bc102ef0 0x7f59bc111fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:57948/0 (socket says 172.21.15.120:57948) 2024-09-19T09:22:17.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.492+0000 7f59ba59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59bc102570 0x7f59bc111a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43194/0 (socket says 172.21.15.120:43194) 2024-09-19T09:22:17.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.492+0000 7f59bb59e640 1 -- 172.21.15.120:0/202366880 learned_addr learned my addr 172.21.15.120:0/202366880 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:17.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.492+0000 7f59bad9d640 1 --2- 172.21.15.120:0/202366880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59bc100e20 0x7f59bc111530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:17.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.493+0000 7f59ba59c640 1 -- 172.21.15.120:0/202366880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59bc100e20 msgr2=0x7f59bc111530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:17.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.493+0000 7f59ba59c640 1 --2- 172.21.15.120:0/202366880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59bc100e20 0x7f59bc111530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:17.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.493+0000 7f59ba59c640 1 -- 172.21.15.120:0/202366880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59bc102ef0 msgr2=0x7f59bc111fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:17.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.493+0000 7f59ba59c640 1 --2- 172.21.15.120:0/202366880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59bc102ef0 0x7f59bc111fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:17.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.493+0000 7f59ba59c640 1 -- 172.21.15.120:0/202366880 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59bc072e50 con 0x7f59bc102570 2024-09-19T09:22:17.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.493+0000 7f59ba59c640 1 --2- 172.21.15.120:0/202366880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59bc102570 0x7f59bc111a70 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f59b000b3f0 tx=0x7f59b000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:17.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.494+0000 7f59a3fff640 1 -- 172.21.15.120:0/202366880 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0015020 con 0x7f59bc102570 2024-09-19T09:22:17.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.494+0000 7f59c1188640 1 -- 172.21.15.120:0/202366880 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59bc073140 con 0x7f59bc102570 2024-09-19T09:22:17.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.494+0000 7f59c1188640 1 -- 172.21.15.120:0/202366880 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59bc073680 con 0x7f59bc102570 2024-09-19T09:22:17.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.494+0000 7f59a3fff640 1 -- 172.21.15.120:0/202366880 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59b0011d10 con 0x7f59bc102570 2024-09-19T09:22:17.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.494+0000 7f59a3fff640 1 -- 172.21.15.120:0/202366880 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b000f2e0 con 0x7f59bc102570 2024-09-19T09:22:17.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.495+0000 7f59c1188640 1 -- 172.21.15.120:0/202366880 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59a80052c0 con 0x7f59bc102570 2024-09-19T09:22:17.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.498+0000 7f59a3fff640 1 -- 172.21.15.120:0/202366880 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f59b0018020 con 0x7f59bc102570 2024-09-19T09:22:17.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.499+0000 7f59a3fff640 1 --2- 172.21.15.120:0/202366880 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f59a40a3390 0x7f59a40a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:17.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.499+0000 7f59bad9d640 1 --2- 172.21.15.120:0/202366880 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f59a40a3390 0x7f59a40a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:17.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.499+0000 7f59a3fff640 1 -- 172.21.15.120:0/202366880 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f59b00cf420 con 0x7f59bc102570 2024-09-19T09:22:17.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.510+0000 7f59bad9d640 1 --2- 172.21.15.120:0/202366880 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f59a40a3390 0x7f59a40a5850 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f59ac005d00 tx=0x7f59ac00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:17.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.511+0000 7f59a3fff640 1 -- 172.21.15.120:0/202366880 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f59b001f020 con 0x7f59bc102570 2024-09-19T09:22:17.515 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:22:17.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.891+0000 7f59c1188640 1 -- 172.21.15.120:0/202366880 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59a80055b0 con 0x7f59bc102570 2024-09-19T09:22:17.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.892+0000 7f59a3fff640 1 -- 172.21.15.120:0/202366880 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11988 (secure 0 0 0) 0x7f59b009c1c0 con 0x7f59bc102570 2024-09-19T09:22:17.893 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:22:17.894 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":39,"btime":"2024-09-19T09:18:53:120858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:18:53.070639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:22:17.894 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 39 2024-09-19T09:22:17.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.894+0000 7f59c1188640 1 -- 172.21.15.120:0/202366880 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f59a40a3390 msgr2=0x7f59a40a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:17.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.894+0000 7f59c1188640 1 --2- 172.21.15.120:0/202366880 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f59a40a3390 0x7f59a40a5850 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f59ac005d00 tx=0x7f59ac00b410 comp rx=0 tx=0).stop 2024-09-19T09:22:17.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.894+0000 7f59c1188640 1 -- 172.21.15.120:0/202366880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59bc102570 msgr2=0x7f59bc111a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:17.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.894+0000 7f59c1188640 1 --2- 172.21.15.120:0/202366880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59bc102570 0x7f59bc111a70 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f59b000b3f0 tx=0x7f59b000b8b0 comp rx=0 tx=0).stop 2024-09-19T09:22:17.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.894+0000 7f59c1188640 1 -- 172.21.15.120:0/202366880 shutdown_connections 2024-09-19T09:22:17.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.894+0000 7f59c1188640 1 --2- 172.21.15.120:0/202366880 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f59a40a3390 0x7f59a40a5850 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:17.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.894+0000 7f59c1188640 1 --2- 172.21.15.120:0/202366880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59bc102ef0 0x7f59bc111fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:17.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.894+0000 7f59c1188640 1 --2- 172.21.15.120:0/202366880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59bc102570 0x7f59bc111a70 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:17.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.894+0000 7f59c1188640 1 --2- 172.21.15.120:0/202366880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59bc100e20 0x7f59bc111530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:17.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.894+0000 7f59c1188640 1 -- 172.21.15.120:0/202366880 >> 172.21.15.120:0/202366880 conn(0x7f59bc0fc7d0 msgr2=0x7f59bc106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:17.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.895+0000 7f59c1188640 1 -- 172.21.15.120:0/202366880 shutdown_connections 2024-09-19T09:22:17.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:17.895+0000 7f59c1188640 1 -- 172.21.15.120:0/202366880 wait complete. 2024-09-19T09:22:18.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:18 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:18.112 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:18 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:22:18.112 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:18 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:22:18.112 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:18 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/202366880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:22:18.112 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:18 smithi120 ceph-mon[31150]: pgmap v928: 97 pgs: 97 active+clean; 423 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.0 MiB/s wr, 572 op/s 2024-09-19T09:22:18.373 INFO:tasks.workunit.client.0.smithi120.stdout: 10 157487 21.71 MB/sec execute 253 sec latency 86.024 ms 2024-09-19T09:22:18.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:18 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:18.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:18 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:22:18.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:18 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:22:18.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:18 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/202366880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:22:18.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:18 smithi196 ceph-mon[35150]: pgmap v928: 97 pgs: 97 active+clean; 423 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.0 MiB/s wr, 572 op/s 2024-09-19T09:22:18.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:18 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:18.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:18 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:22:18.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:18 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:22:18.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:18 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/202366880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:22:18.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:18 smithi153 ceph-mon[35071]: pgmap v928: 97 pgs: 97 active+clean; 423 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.0 MiB/s wr, 572 op/s 2024-09-19T09:22:18.978 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db / --timeout 90 --expiration 154.0 --await-for 45.0' 2024-09-19T09:22:18.978 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db / --timeout 90 --expiration 154.0 --await-for 45.0 2024-09-19T09:22:19.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.084+0000 7f4564072640 1 -- 172.21.15.120:0/3590033630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f455c102d50 msgr2=0x7f455c103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:19.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.084+0000 7f4564072640 1 --2- 172.21.15.120:0/3590033630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f455c102d50 0x7f455c103130 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f4558009900 tx=0x7f455802a480 comp rx=0 tx=0).stop 2024-09-19T09:22:19.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.084+0000 7f4564072640 1 -- 172.21.15.120:0/3590033630 shutdown_connections 2024-09-19T09:22:19.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.084+0000 7f4564072640 1 --2- 172.21.15.120:0/3590033630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f455c10c180 0x7f455c10e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.084+0000 7f4564072640 1 --2- 172.21.15.120:0/3590033630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f455c103670 0x7f455c10bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.084+0000 7f4564072640 1 --2- 172.21.15.120:0/3590033630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f455c102d50 0x7f455c103130 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.084+0000 7f4564072640 1 -- 172.21.15.120:0/3590033630 >> 172.21.15.120:0/3590033630 conn(0x7f455c0fc7d0 msgr2=0x7f455c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:19.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.084+0000 7f4564072640 1 -- 172.21.15.120:0/3590033630 shutdown_connections 2024-09-19T09:22:19.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.085+0000 7f4564072640 1 -- 172.21.15.120:0/3590033630 wait complete. 2024-09-19T09:22:19.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.086+0000 7f4564072640 1 Processor -- start 2024-09-19T09:22:19.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.086+0000 7f4564072640 1 -- start start 2024-09-19T09:22:19.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.086+0000 7f4564072640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f455c102d50 0x7f455c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:19.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.087+0000 7f4564072640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f455c103670 0x7f455c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:19.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.087+0000 7f4561e0e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f455c102d50 0x7f455c1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:19.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.087+0000 7f4561e0e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f455c102d50 0x7f455c1981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43212/0 (socket says 172.21.15.120:43212) 2024-09-19T09:22:19.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.087+0000 7f4561e0e640 1 -- 172.21.15.120:0/902670572 learned_addr learned my addr 172.21.15.120:0/902670572 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:19.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.087+0000 7f4564072640 1 --2- 172.21.15.120:0/902670572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f455c10c180 0x7f455c19ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:19.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.087+0000 7f456160d640 1 --2- 172.21.15.120:0/902670572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f455c103670 0x7f455c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:19.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.087+0000 7f4564072640 1 -- 172.21.15.120:0/902670572 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f455c111fd0 con 0x7f455c102d50 2024-09-19T09:22:19.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.087+0000 7f4564072640 1 -- 172.21.15.120:0/902670572 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f455c111e50 con 0x7f455c10c180 2024-09-19T09:22:19.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.087+0000 7f4564072640 1 -- 172.21.15.120:0/902670572 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f455c112150 con 0x7f455c103670 2024-09-19T09:22:19.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.087+0000 7f456260f640 1 --2- 172.21.15.120:0/902670572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f455c10c180 0x7f455c19ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:19.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.087+0000 7f456160d640 1 -- 172.21.15.120:0/902670572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f455c10c180 msgr2=0x7f455c19ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:19.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.087+0000 7f456160d640 1 --2- 172.21.15.120:0/902670572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f455c10c180 0x7f455c19ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.087+0000 7f456160d640 1 -- 172.21.15.120:0/902670572 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f455c102d50 msgr2=0x7f455c1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:19.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.087+0000 7f456160d640 1 --2- 172.21.15.120:0/902670572 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f455c102d50 0x7f455c1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.087+0000 7f456160d640 1 -- 172.21.15.120:0/902670572 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455c19d150 con 0x7f455c103670 2024-09-19T09:22:19.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.087+0000 7f456260f640 1 --2- 172.21.15.120:0/902670572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f455c10c180 0x7f455c19ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:22:19.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.088+0000 7f456160d640 1 --2- 172.21.15.120:0/902670572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f455c103670 0x7f455c1986e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f454c00c970 tx=0x7f454c00ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:19.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.088+0000 7f4552ffd640 1 -- 172.21.15.120:0/902670572 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c007bb0 con 0x7f455c103670 2024-09-19T09:22:19.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.088+0000 7f4552ffd640 1 -- 172.21.15.120:0/902670572 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f454c007d50 con 0x7f455c103670 2024-09-19T09:22:19.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.088+0000 7f4564072640 1 -- 172.21.15.120:0/902670572 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f455c19d440 con 0x7f455c103670 2024-09-19T09:22:19.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.088+0000 7f4564072640 1 -- 172.21.15.120:0/902670572 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f455c1a4d20 con 0x7f455c103670 2024-09-19T09:22:19.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.088+0000 7f4552ffd640 1 -- 172.21.15.120:0/902670572 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c018780 con 0x7f455c103670 2024-09-19T09:22:19.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.091+0000 7f4552ffd640 1 -- 172.21.15.120:0/902670572 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f454c018920 con 0x7f455c103670 2024-09-19T09:22:19.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.092+0000 7f456644b740 1 Processor -- start 2024-09-19T09:22:19.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.092+0000 7f456644b740 1 -- start start 2024-09-19T09:22:19.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.092+0000 7f456644b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ff4550440 0x558ff4550820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:19.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.092+0000 7f456644b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ff4550df0 0x558ff4555470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:19.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.092+0000 7f456644b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ff4555a40 0x558ff4555ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:19.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.093+0000 7f456644b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558ff44ac2f0 con 0x558ff4550df0 2024-09-19T09:22:19.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.093+0000 7f456644b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558ff4453cf0 con 0x558ff4555a40 2024-09-19T09:22:19.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.093+0000 7f456644b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558ff44a82b0 con 0x558ff4550440 2024-09-19T09:22:19.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.093+0000 7f456260f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ff4550440 0x558ff4550820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:19.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.093+0000 7f456260f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ff4550440 0x558ff4550820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:38518/0 (socket says 172.21.15.120:38518) 2024-09-19T09:22:19.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.093+0000 7f456160d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ff4555a40 0x558ff4555ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:19.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.093+0000 7f456260f640 1 -- 172.21.15.120:0/2282010047 learned_addr learned my addr 172.21.15.120:0/2282010047 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:19.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.093+0000 7f4561e0e640 1 --2- 172.21.15.120:0/2282010047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ff4550df0 0x558ff4555470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:19.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.093+0000 7f456260f640 1 -- 172.21.15.120:0/2282010047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ff4555a40 msgr2=0x558ff4555ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:19.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.093+0000 7f456260f640 1 --2- 172.21.15.120:0/2282010047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ff4555a40 0x558ff4555ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.093+0000 7f456260f640 1 -- 172.21.15.120:0/2282010047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ff4550df0 msgr2=0x558ff4555470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:19.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.093+0000 7f456260f640 1 --2- 172.21.15.120:0/2282010047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ff4550df0 0x558ff4555470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.093+0000 7f456260f640 1 -- 172.21.15.120:0/2282010047 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ff44383a0 con 0x558ff4550440 2024-09-19T09:22:19.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.093+0000 7f4552ffd640 1 --2- 172.21.15.120:0/902670572 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f45480a32c0 0x7f45480a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:19.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.093+0000 7f4561e0e640 1 --2- 172.21.15.120:0/902670572 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f45480a32c0 0x7f45480a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:19.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.093+0000 7f4552ffd640 1 -- 172.21.15.120:0/902670572 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f454c0cf980 con 0x7f455c103670 2024-09-19T09:22:19.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.094+0000 7f456260f640 1 --2- 172.21.15.120:0/2282010047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ff4550440 0x558ff4550820 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4554009f40 tx=0x7f455400f7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5ca82efcd93aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:19.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.094+0000 7f4547fff640 1 -- 172.21.15.120:0/2282010047 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554003b70 con 0x558ff4550440 2024-09-19T09:22:19.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.094+0000 7f4547fff640 1 -- 172.21.15.120:0/2282010047 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4554003d10 con 0x558ff4550440 2024-09-19T09:22:19.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.095+0000 7f4547fff640 1 -- 172.21.15.120:0/2282010047 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554011ab0 con 0x558ff4550440 2024-09-19T09:22:19.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.095+0000 7f4561e0e640 1 --2- 172.21.15.120:0/902670572 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f45480a32c0 0x7f45480a5780 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f45580096c0 tx=0x7f455800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:19.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.095+0000 7f456644b740 1 -- 172.21.15.120:0/2282010047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ff4550440 msgr2=0x558ff4550820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:19.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.095+0000 7f456644b740 1 --2- 172.21.15.120:0/2282010047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ff4550440 0x558ff4550820 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4554009f40 tx=0x7f455400f7f0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.095+0000 7f456644b740 1 -- 172.21.15.120:0/2282010047 shutdown_connections 2024-09-19T09:22:19.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.095+0000 7f456644b740 1 --2- 172.21.15.120:0/2282010047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ff4555a40 0x558ff4555ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.095+0000 7f456644b740 1 --2- 172.21.15.120:0/2282010047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ff4550df0 0x558ff4555470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.095+0000 7f456644b740 1 --2- 172.21.15.120:0/2282010047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ff4550440 0x558ff4550820 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.095+0000 7f456644b740 1 -- 172.21.15.120:0/2282010047 >> 172.21.15.120:0/2282010047 conn(0x558ff450f310 msgr2=0x558ff45426e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:19.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.095+0000 7f456644b740 1 -- 172.21.15.120:0/2282010047 shutdown_connections 2024-09-19T09:22:19.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.095+0000 7f456644b740 1 -- 172.21.15.120:0/2282010047 wait complete. 2024-09-19T09:22:19.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.096+0000 7f456644b740 1 -- start start 2024-09-19T09:22:19.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.096+0000 7f456644b740 1 Processor -- start 2024-09-19T09:22:19.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.096+0000 7f456644b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ff4550440 0x558ff455bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:19.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.096+0000 7f456644b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ff4550df0 0x558ff4558f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:19.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.097+0000 7f456644b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ff4555a40 0x558ff45594c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:19.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.097+0000 7f456644b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558ff455c920 con 0x558ff4550440 2024-09-19T09:22:19.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.097+0000 7f4561e0e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ff4550440 0x558ff455bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:19.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.097+0000 7f456644b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558ff4467060 con 0x558ff4550df0 2024-09-19T09:22:19.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.097+0000 7f456644b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558ff455caa0 con 0x558ff4555a40 2024-09-19T09:22:19.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.097+0000 7f4561e0e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ff4550440 0x558ff455bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43236/0 (socket says 172.21.15.120:43236) 2024-09-19T09:22:19.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.097+0000 7f4561e0e640 1 -- 172.21.15.120:0/1178800924 learned_addr learned my addr 172.21.15.120:0/1178800924 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:19.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.097+0000 7f456644b740 10 client.0 _ll_register_callbacks cb 0x558ff41ba8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:22:19.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.097+0000 7f456160d640 1 --2- 172.21.15.120:0/1178800924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ff4550df0 0x558ff4558f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:19.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.097+0000 7f456260f640 1 --2- 172.21.15.120:0/1178800924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ff4555a40 0x558ff45594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:19.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.097+0000 7f4561e0e640 1 -- 172.21.15.120:0/1178800924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ff4555a40 msgr2=0x558ff45594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:19.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.097+0000 7f4561e0e640 1 --2- 172.21.15.120:0/1178800924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ff4555a40 0x558ff45594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.097+0000 7f4561e0e640 1 -- 172.21.15.120:0/1178800924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ff4550df0 msgr2=0x558ff4558f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:19.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.097+0000 7f4561e0e640 1 --2- 172.21.15.120:0/1178800924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ff4550df0 0x558ff4558f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.097+0000 7f4561e0e640 1 -- 172.21.15.120:0/1178800924 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ff421d3c0 con 0x558ff4550440 2024-09-19T09:22:19.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.097+0000 7f4561e0e640 1 --2- 172.21.15.120:0/1178800924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ff4550440 0x558ff455bcd0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f4558031d60 tx=0x7f455802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39fbef8d2e69662e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:19.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.097+0000 7f45457fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:22:19.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.097+0000 7f45457fa640 1 -- 172.21.15.120:0/1178800924 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4558035040 con 0x558ff4550440 2024-09-19T09:22:19.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.098+0000 7f456644b740 1 -- 172.21.15.120:0/1178800924 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4554003d10 con 0x558ff4550440 2024-09-19T09:22:19.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.098+0000 7f45457fa640 1 -- 172.21.15.120:0/1178800924 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f4558004440 con 0x558ff4550440 2024-09-19T09:22:19.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.098+0000 7f45457fa640 1 -- 172.21.15.120:0/1178800924 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f455803d950 con 0x558ff4550440 2024-09-19T09:22:19.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.098+0000 7f456644b740 10 client.21870 fetch_fsmap learned FSMap version 39 2024-09-19T09:22:19.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.098+0000 7f456644b740 1 -- 172.21.15.120:0/1178800924 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558ff4568250 con 0x558ff4550440 2024-09-19T09:22:19.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.098+0000 7f45457fa640 1 -- 172.21.15.120:0/1178800924 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4554003d10 con 0x558ff4550440 2024-09-19T09:22:19.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.098+0000 7f45457fa640 1 -- 172.21.15.120:0/1178800924 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f4558044070 con 0x558ff4550440 2024-09-19T09:22:19.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.098+0000 7f456644b740 10 client.21870 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:22:19.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.098+0000 7f456644b740 10 client.21870 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:22:19.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.098+0000 7f456644b740 20 client.21870 populate_metadata read hostname 'smithi120' 2024-09-19T09:22:19.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.098+0000 7f456644b740 1 --2- 172.21.15.120:0/1178800924 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x558ff4569a90 0x558ff456bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:19.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.098+0000 7f456644b740 4 client.21870 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:22:19.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.098+0000 7f456644b740 1 -- 172.21.15.120:0/1178800924 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ff44b9860 con 0x558ff4569a90 2024-09-19T09:22:19.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.099+0000 7f456160d640 1 --2- 172.21.15.120:0/1178800924 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x558ff4569a90 0x558ff456bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:19.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.099+0000 7f456160d640 1 --2- 172.21.15.120:0/1178800924 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x558ff4569a90 0x558ff456bf50 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e02e1585e259412d server_cookie=5d0f4af9b9cb0052 in_seq=0 out_seq=0 2024-09-19T09:22:19.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.099+0000 7f45457fa640 10 client.21870 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:22:19.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.100+0000 7f45457fa640 1 -- 172.21.15.120:0/1178800924 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558ff44b9860 con 0x558ff4569a90 2024-09-19T09:22:19.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.100+0000 7f45457fa640 10 client.21870 handle_command_reply: tid=0 2024-09-19T09:22:19.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.100+0000 7f456644b740 1 client.21870 shutdown 2024-09-19T09:22:19.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.100+0000 7f456644b740 1 -- 172.21.15.120:0/1178800924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ff4550440 msgr2=0x558ff455bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:19.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.100+0000 7f456644b740 1 --2- 172.21.15.120:0/1178800924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ff4550440 0x558ff455bcd0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f4558031d60 tx=0x7f455802a480 comp rx=0 tx=0).stop 2024-09-19T09:22:19.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.100+0000 7f456644b740 1 -- 172.21.15.120:0/1178800924 shutdown_connections 2024-09-19T09:22:19.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.100+0000 7f456644b740 1 --2- 172.21.15.120:0/1178800924 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x558ff4569a90 0x558ff456bf50 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.100+0000 7f456644b740 1 --2- 172.21.15.120:0/1178800924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ff4555a40 0x558ff45594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.100+0000 7f456644b740 1 --2- 172.21.15.120:0/1178800924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ff4550df0 0x558ff4558f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.100+0000 7f456644b740 1 --2- 172.21.15.120:0/1178800924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ff4550440 0x558ff455bcd0 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.100+0000 7f456644b740 1 -- 172.21.15.120:0/1178800924 >> 172.21.15.120:0/1178800924 conn(0x558ff450f310 msgr2=0x558ff454fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:19.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.100+0000 7f45457fa640 0 client.21870 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:22:19.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.100+0000 7f456644b740 1 -- 172.21.15.120:0/1178800924 shutdown_connections 2024-09-19T09:22:19.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.100+0000 7f456644b740 1 -- 172.21.15.120:0/1178800924 wait complete. 2024-09-19T09:22:19.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.101+0000 7f456644b740 20 client.21870 trim_cache size 0 max 16384 2024-09-19T09:22:19.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.117+0000 7f456644b740 1 Processor -- start 2024-09-19T09:22:19.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.117+0000 7f456644b740 1 -- start start 2024-09-19T09:22:19.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.117+0000 7f456644b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ff4550440 0x558ff455a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:19.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.117+0000 7f456644b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ff4550df0 0x558ff455ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:19.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.117+0000 7f456644b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ff4555a40 0x558ff455b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:19.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.117+0000 7f456644b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1ab350 con 0x558ff4555a40 2024-09-19T09:22:19.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.117+0000 7f456644b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1ace90 con 0x558ff4550440 2024-09-19T09:22:19.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.117+0000 7f456644b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1aba20 con 0x558ff4550df0 2024-09-19T09:22:19.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.117+0000 7f456260f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ff4555a40 0x558ff455b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:19.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.117+0000 7f456160d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ff4550df0 0x558ff455ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:19.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.117+0000 7f456160d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ff4550df0 0x558ff455ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:38532/0 (socket says 172.21.15.120:38532) 2024-09-19T09:22:19.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.117+0000 7f456260f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ff4555a40 0x558ff455b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43238/0 (socket says 172.21.15.120:43238) 2024-09-19T09:22:19.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.117+0000 7f456160d640 1 -- 172.21.15.120:0/1272202525 learned_addr learned my addr 172.21.15.120:0/1272202525 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:19.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.118+0000 7f4561e0e640 1 --2- 172.21.15.120:0/1272202525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ff4550440 0x558ff455a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:19.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.118+0000 7f456160d640 1 -- 172.21.15.120:0/1272202525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ff4550440 msgr2=0x558ff455a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:19.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.118+0000 7f456160d640 1 --2- 172.21.15.120:0/1272202525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ff4550440 0x558ff455a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.118+0000 7f456160d640 1 -- 172.21.15.120:0/1272202525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ff4555a40 msgr2=0x558ff455b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:19.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.118+0000 7f456160d640 1 --2- 172.21.15.120:0/1272202525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ff4555a40 0x558ff455b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.118+0000 7f456160d640 1 -- 172.21.15.120:0/1272202525 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ff4546580 con 0x558ff4550df0 2024-09-19T09:22:19.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.118+0000 7f4561e0e640 1 --2- 172.21.15.120:0/1272202525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ff4550440 0x558ff455a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:22:19.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.118+0000 7f456260f640 1 --2- 172.21.15.120:0/1272202525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ff4555a40 0x558ff455b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:22:19.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.118+0000 7f456160d640 1 --2- 172.21.15.120:0/1272202525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ff4550df0 0x558ff455ad20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f454c099430 tx=0x7f454c0cd310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b148a43b6b35f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:19.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.118+0000 7f45477fe640 1 -- 172.21.15.120:0/1272202525 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c09b070 con 0x558ff4550df0 2024-09-19T09:22:19.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.118+0000 7f45477fe640 1 -- 172.21.15.120:0/1272202525 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f454c09b2d0 con 0x558ff4550df0 2024-09-19T09:22:19.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.118+0000 7f45477fe640 1 -- 172.21.15.120:0/1272202525 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c0a49a0 con 0x558ff4550df0 2024-09-19T09:22:19.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.119+0000 7f456644b740 1 -- 172.21.15.120:0/1272202525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ff4550df0 msgr2=0x558ff455ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:19.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.119+0000 7f456644b740 1 --2- 172.21.15.120:0/1272202525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ff4550df0 0x558ff455ad20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f454c099430 tx=0x7f454c0cd310 comp rx=0 tx=0).stop 2024-09-19T09:22:19.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.119+0000 7f456644b740 1 -- 172.21.15.120:0/1272202525 shutdown_connections 2024-09-19T09:22:19.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.119+0000 7f456644b740 1 --2- 172.21.15.120:0/1272202525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ff4555a40 0x558ff455b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.119+0000 7f456644b740 1 --2- 172.21.15.120:0/1272202525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ff4550df0 0x558ff455ad20 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.119+0000 7f456644b740 1 --2- 172.21.15.120:0/1272202525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ff4550440 0x558ff455a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.119+0000 7f456644b740 1 -- 172.21.15.120:0/1272202525 >> 172.21.15.120:0/1272202525 conn(0x558ff450f310 msgr2=0x558ff4558070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:19.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.120+0000 7f456644b740 1 -- 172.21.15.120:0/1272202525 shutdown_connections 2024-09-19T09:22:19.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.120+0000 7f456644b740 1 -- 172.21.15.120:0/1272202525 wait complete. 2024-09-19T09:22:19.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.121+0000 7f456644b740 1 -- start start 2024-09-19T09:22:19.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.121+0000 7f456644b740 1 Processor -- start 2024-09-19T09:22:19.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.121+0000 7f456644b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ff4550440 0x558ff45713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:19.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.121+0000 7f456644b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ff4550df0 0x558ff4571900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:19.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.121+0000 7f456644b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ff4555a40 0x558ff4571e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:19.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.121+0000 7f456644b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1ac7c0 con 0x558ff4555a40 2024-09-19T09:22:19.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.121+0000 7f456644b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1ac0f0 con 0x558ff4550df0 2024-09-19T09:22:19.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.121+0000 7f456644b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1a9330 con 0x558ff4550440 2024-09-19T09:22:19.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.121+0000 7f456644b740 10 client.0 _ll_register_callbacks cb 0x7f4558005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:22:19.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.122+0000 7f456260f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ff4555a40 0x558ff4571e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:19.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.122+0000 7f456160d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ff4550df0 0x558ff4571900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:19.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.122+0000 7f456260f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ff4555a40 0x558ff4571e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43240/0 (socket says 172.21.15.120:43240) 2024-09-19T09:22:19.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.122+0000 7f456160d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ff4550df0 0x558ff4571900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:58008/0 (socket says 172.21.15.120:58008) 2024-09-19T09:22:19.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.122+0000 7f456260f640 1 -- 172.21.15.120:0/95250555 learned_addr learned my addr 172.21.15.120:0/95250555 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:19.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.122+0000 7f4561e0e640 1 --2- 172.21.15.120:0/95250555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ff4550440 0x558ff45713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:19.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.122+0000 7f456160d640 1 -- 172.21.15.120:0/95250555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ff4550440 msgr2=0x558ff45713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:19.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.122+0000 7f456160d640 1 --2- 172.21.15.120:0/95250555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ff4550440 0x558ff45713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.122+0000 7f456160d640 1 -- 172.21.15.120:0/95250555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ff4555a40 msgr2=0x558ff4571e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:19.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.122+0000 7f456160d640 1 --2- 172.21.15.120:0/95250555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ff4555a40 0x558ff4571e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.122+0000 7f456160d640 1 -- 172.21.15.120:0/95250555 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ff45d37f0 con 0x558ff4550df0 2024-09-19T09:22:19.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.122+0000 7f456260f640 1 --2- 172.21.15.120:0/95250555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ff4555a40 0x558ff4571e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:22:19.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.122+0000 7f456160d640 1 --2- 172.21.15.120:0/95250555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ff4550df0 0x558ff4571900 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x558ff444ee40 tx=0x7f454c00fa80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=361f14918c83215f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:19.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.122+0000 7f45457fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:22:19.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.122+0000 7f45457fa640 1 -- 172.21.15.120:0/95250555 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c00f450 con 0x558ff4550df0 2024-09-19T09:22:19.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.122+0000 7f45457fa640 1 -- 172.21.15.120:0/95250555 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f454c0a4740 con 0x558ff4550df0 2024-09-19T09:22:19.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.122+0000 7f456644b740 1 -- 172.21.15.120:0/95250555 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f454c09b2d0 con 0x558ff4550df0 2024-09-19T09:22:19.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.123+0000 7f45457fa640 1 -- 172.21.15.120:0/95250555 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f454c00f5f0 con 0x558ff4550df0 2024-09-19T09:22:19.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.123+0000 7f456644b740 10 client.30683 fetch_fsmap learned FSMap version 39 2024-09-19T09:22:19.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.123+0000 7f456644b740 1 -- 172.21.15.120:0/95250555 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558ff45d4a00 con 0x558ff4550df0 2024-09-19T09:22:19.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.123+0000 7f45457fa640 1 -- 172.21.15.120:0/95250555 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f454c09b2d0 con 0x558ff4550df0 2024-09-19T09:22:19.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.123+0000 7f45457fa640 1 -- 172.21.15.120:0/95250555 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f454c0a14a0 con 0x558ff4550df0 2024-09-19T09:22:19.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.123+0000 7f456644b740 10 client.30683 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:22:19.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.123+0000 7f456644b740 10 client.30683 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:22:19.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.123+0000 7f456644b740 20 client.30683 populate_metadata read hostname 'smithi120' 2024-09-19T09:22:19.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.123+0000 7f456644b740 1 --2- 172.21.15.120:0/95250555 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x558ff45d50a0 0x558ff45d74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:19.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.123+0000 7f456644b740 4 client.30683 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:22:19.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.123+0000 7f456644b740 1 -- 172.21.15.120:0/95250555 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ff44b3d70 con 0x558ff45d50a0 2024-09-19T09:22:19.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.123+0000 7f4561e0e640 1 --2- 172.21.15.120:0/95250555 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x558ff45d50a0 0x558ff45d74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:19.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.124+0000 7f4561e0e640 1 --2- 172.21.15.120:0/95250555 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x558ff45d50a0 0x558ff45d74c0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=862f75ee931377a0 server_cookie=48cce15e628dd217 in_seq=0 out_seq=0 2024-09-19T09:22:19.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.124+0000 7f45457fa640 10 client.30683 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:22:19.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.124+0000 7f45457fa640 1 -- 172.21.15.120:0/95250555 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ff44b3d70 con 0x558ff45d50a0 2024-09-19T09:22:19.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.124+0000 7f45457fa640 10 client.30683 handle_command_reply: tid=0 2024-09-19T09:22:19.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.124+0000 7f456644b740 1 client.30683 shutdown 2024-09-19T09:22:19.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.124+0000 7f456644b740 1 -- 172.21.15.120:0/95250555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ff4550df0 msgr2=0x558ff4571900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:19.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.124+0000 7f456644b740 1 --2- 172.21.15.120:0/95250555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ff4550df0 0x558ff4571900 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x558ff444ee40 tx=0x7f454c00fa80 comp rx=0 tx=0).stop 2024-09-19T09:22:19.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.124+0000 7f456644b740 1 -- 172.21.15.120:0/95250555 shutdown_connections 2024-09-19T09:22:19.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.124+0000 7f456644b740 1 --2- 172.21.15.120:0/95250555 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x558ff45d50a0 0x558ff45d74c0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.124+0000 7f45457fa640 0 client.30683 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:22:19.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.124+0000 7f456644b740 1 --2- 172.21.15.120:0/95250555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558ff4555a40 0x558ff4571e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.124+0000 7f456644b740 1 --2- 172.21.15.120:0/95250555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ff4550df0 0x558ff4571900 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.124+0000 7f456644b740 1 --2- 172.21.15.120:0/95250555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ff4550440 0x558ff45713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.124+0000 7f456644b740 1 -- 172.21.15.120:0/95250555 >> 172.21.15.120:0/95250555 conn(0x558ff450f310 msgr2=0x558ff455a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:19.132 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:22:19.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.125+0000 7f456644b740 1 -- 172.21.15.120:0/95250555 shutdown_connections 2024-09-19T09:22:19.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.125+0000 7f456644b740 1 -- 172.21.15.120:0/95250555 wait complete. 2024-09-19T09:22:19.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.125+0000 7f456644b740 20 client.30683 trim_cache size 0 max 16384 2024-09-19T09:22:19.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.126+0000 7f4564072640 1 -- 172.21.15.120:0/902670572 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f45480a32c0 msgr2=0x7f45480a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:19.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.126+0000 7f4564072640 1 --2- 172.21.15.120:0/902670572 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f45480a32c0 0x7f45480a5780 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f45580096c0 tx=0x7f455800b040 comp rx=0 tx=0).stop 2024-09-19T09:22:19.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.126+0000 7f4564072640 1 -- 172.21.15.120:0/902670572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f455c103670 msgr2=0x7f455c1986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:19.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.126+0000 7f4564072640 1 --2- 172.21.15.120:0/902670572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f455c103670 0x7f455c1986e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f454c00c970 tx=0x7f454c00ce30 comp rx=0 tx=0).stop 2024-09-19T09:22:19.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.126+0000 7f4564072640 1 -- 172.21.15.120:0/902670572 shutdown_connections 2024-09-19T09:22:19.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.126+0000 7f4564072640 1 --2- 172.21.15.120:0/902670572 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f45480a32c0 0x7f45480a5780 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.126+0000 7f4564072640 1 --2- 172.21.15.120:0/902670572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f455c10c180 0x7f455c19ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.126+0000 7f4564072640 1 --2- 172.21.15.120:0/902670572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f455c103670 0x7f455c1986e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.126+0000 7f4564072640 1 --2- 172.21.15.120:0/902670572 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f455c102d50 0x7f455c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:19.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.126+0000 7f4564072640 1 -- 172.21.15.120:0/902670572 >> 172.21.15.120:0/902670572 conn(0x7f455c0fc7d0 msgr2=0x7f455c10dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:19.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.127+0000 7f4564072640 1 -- 172.21.15.120:0/902670572 shutdown_connections 2024-09-19T09:22:19.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:18.127+0000 7f4564072640 1 -- 172.21.15.120:0/902670572 wait complete. 2024-09-19T09:22:19.373 INFO:tasks.workunit.client.0.smithi120.stdout: 10 158140 21.78 MB/sec execute 254 sec latency 45.757 ms 2024-09-19T09:22:20.041 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:22:20.128 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:22:20.259 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:20 smithi120 ceph-mon[31150]: pgmap v929: 97 pgs: 97 active+clean; 452 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 13 MiB/s wr, 698 op/s 2024-09-19T09:22:20.373 INFO:tasks.workunit.client.0.smithi120.stdout: 10 158872 21.81 MB/sec execute 255 sec latency 55.087 ms 2024-09-19T09:22:20.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:20 smithi196 ceph-mon[35150]: pgmap v929: 97 pgs: 97 active+clean; 452 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 13 MiB/s wr, 698 op/s 2024-09-19T09:22:20.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:20 smithi153 ceph-mon[35071]: pgmap v929: 97 pgs: 97 active+clean; 452 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 13 MiB/s wr, 698 op/s 2024-09-19T09:22:20.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.682+0000 7f5e8c917640 1 -- 172.21.15.120:0/801036037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e881082a0 msgr2=0x7f5e8810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:20.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.682+0000 7f5e8c917640 1 --2- 172.21.15.120:0/801036037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e881082a0 0x7f5e8810ec40 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f5e80009c10 tx=0x7f5e8002a6e0 comp rx=0 tx=0).stop 2024-09-19T09:22:20.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.682+0000 7f5e8c917640 1 -- 172.21.15.120:0/801036037 shutdown_connections 2024-09-19T09:22:20.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.682+0000 7f5e8c917640 1 --2- 172.21.15.120:0/801036037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e881082a0 0x7f5e8810ec40 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:20.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.682+0000 7f5e8c917640 1 --2- 172.21.15.120:0/801036037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e881036e0 0x7f5e88107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:20.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.682+0000 7f5e8c917640 1 --2- 172.21.15.120:0/801036037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e88102d30 0x7f5e88103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:20.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.682+0000 7f5e8c917640 1 -- 172.21.15.120:0/801036037 >> 172.21.15.120:0/801036037 conn(0x7f5e880fc7d0 msgr2=0x7f5e880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:20.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.682+0000 7f5e8c917640 1 -- 172.21.15.120:0/801036037 shutdown_connections 2024-09-19T09:22:20.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.682+0000 7f5e8c917640 1 -- 172.21.15.120:0/801036037 wait complete. 2024-09-19T09:22:20.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.683+0000 7f5e8c917640 1 Processor -- start 2024-09-19T09:22:20.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.683+0000 7f5e8c917640 1 -- start start 2024-09-19T09:22:20.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.683+0000 7f5e8c917640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e88102d30 0x7f5e88198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:20.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.683+0000 7f5e8c917640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e881036e0 0x7f5e88198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:20.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.683+0000 7f5e8c917640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e881082a0 0x7f5e8819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:20.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.683+0000 7f5e8c917640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5e88112050 con 0x7f5e88102d30 2024-09-19T09:22:20.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.683+0000 7f5e8c917640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5e88111ed0 con 0x7f5e881082a0 2024-09-19T09:22:20.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.683+0000 7f5e8c917640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5e881121d0 con 0x7f5e881036e0 2024-09-19T09:22:20.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.683+0000 7f5e8659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e88102d30 0x7f5e88198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:20.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.683+0000 7f5e8659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e88102d30 0x7f5e88198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43260/0 (socket says 172.21.15.120:43260) 2024-09-19T09:22:20.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.683+0000 7f5e8659c640 1 -- 172.21.15.120:0/693761554 learned_addr learned my addr 172.21.15.120:0/693761554 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:20.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.683+0000 7f5e85d9b640 1 --2- 172.21.15.120:0/693761554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e881036e0 0x7f5e88198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:20.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.683+0000 7f5e86d9d640 1 --2- 172.21.15.120:0/693761554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e881082a0 0x7f5e8819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:20.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.683+0000 7f5e8659c640 1 -- 172.21.15.120:0/693761554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e881036e0 msgr2=0x7f5e88198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:20.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.683+0000 7f5e8659c640 1 --2- 172.21.15.120:0/693761554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e881036e0 0x7f5e88198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:20.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.683+0000 7f5e8659c640 1 -- 172.21.15.120:0/693761554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e881082a0 msgr2=0x7f5e8819cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:20.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.683+0000 7f5e8659c640 1 --2- 172.21.15.120:0/693761554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e881082a0 0x7f5e8819cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:20.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.683+0000 7f5e8659c640 1 -- 172.21.15.120:0/693761554 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8819d4d0 con 0x7f5e88102d30 2024-09-19T09:22:20.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.684+0000 7f5e8659c640 1 --2- 172.21.15.120:0/693761554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e88102d30 0x7f5e88198520 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f5e7800e9f0 tx=0x7f5e7800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:20.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.684+0000 7f5e777fe640 1 -- 172.21.15.120:0/693761554 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7800ccc0 con 0x7f5e88102d30 2024-09-19T09:22:20.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.684+0000 7f5e8c917640 1 -- 172.21.15.120:0/693761554 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e8819d7c0 con 0x7f5e88102d30 2024-09-19T09:22:20.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.684+0000 7f5e777fe640 1 -- 172.21.15.120:0/693761554 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e78005500 con 0x7f5e88102d30 2024-09-19T09:22:20.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.684+0000 7f5e777fe640 1 -- 172.21.15.120:0/693761554 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78010690 con 0x7f5e88102d30 2024-09-19T09:22:20.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.684+0000 7f5e8c917640 1 -- 172.21.15.120:0/693761554 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e881a50a0 con 0x7f5e88102d30 2024-09-19T09:22:20.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.686+0000 7f5e777fe640 1 -- 172.21.15.120:0/693761554 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5e780046c0 con 0x7f5e88102d30 2024-09-19T09:22:20.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.687+0000 7f5e8ecf0740 1 Processor -- start 2024-09-19T09:22:20.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.687+0000 7f5e8ecf0740 1 -- start start 2024-09-19T09:22:20.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.687+0000 7f5e777fe640 1 --2- 172.21.15.120:0/693761554 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5e700a3240 0x7f5e700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:20.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.688+0000 7f5e8ecf0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560c3f43760 0x5560c3f43b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:20.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.688+0000 7f5e8ecf0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560c3f44080 0x5560c3f4e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:20.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.688+0000 7f5e86d9d640 1 --2- 172.21.15.120:0/693761554 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5e700a3240 0x7f5e700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:20.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.688+0000 7f5e8ecf0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560c3f4e940 0x5560c3f50d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:20.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.688+0000 7f5e777fe640 1 -- 172.21.15.120:0/693761554 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f5e780cfbd0 con 0x7f5e88102d30 2024-09-19T09:22:20.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.688+0000 7f5e8ecf0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5560c3ea92a0 con 0x5560c3f44080 2024-09-19T09:22:20.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.688+0000 7f5e8ecf0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5560c3e50b10 con 0x5560c3f43760 2024-09-19T09:22:20.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.688+0000 7f5e8ecf0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5560c3ea5260 con 0x5560c3f4e940 2024-09-19T09:22:20.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.688+0000 7f5e85d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560c3f44080 0x5560c3f4e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:20.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.688+0000 7f5e8659c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560c3f43760 0x5560c3f43b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:20.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.688+0000 7f5e85d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560c3f44080 0x5560c3f4e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43264/0 (socket says 172.21.15.120:43264) 2024-09-19T09:22:20.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.688+0000 7f5e85d9b640 1 -- 172.21.15.120:0/801701875 learned_addr learned my addr 172.21.15.120:0/801701875 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:20.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.688+0000 7f5e86d9d640 1 --2- 172.21.15.120:0/801701875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560c3f4e940 0x5560c3f50d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:20.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.688+0000 7f5e85d9b640 1 -- 172.21.15.120:0/801701875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560c3f4e940 msgr2=0x5560c3f50d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:20.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.688+0000 7f5e85d9b640 1 --2- 172.21.15.120:0/801701875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560c3f4e940 0x5560c3f50d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:20.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.688+0000 7f5e85d9b640 1 -- 172.21.15.120:0/801701875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560c3f43760 msgr2=0x5560c3f43b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:20.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.688+0000 7f5e85d9b640 1 --2- 172.21.15.120:0/801701875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560c3f43760 0x5560c3f43b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:20.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.688+0000 7f5e85d9b640 1 -- 172.21.15.120:0/801701875 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560c3c1ab90 con 0x5560c3f44080 2024-09-19T09:22:20.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.688+0000 7f5e86d9d640 1 --2- 172.21.15.120:0/801701875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560c3f4e940 0x5560c3f50d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:22:20.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.689+0000 7f5e85d9b640 1 --2- 172.21.15.120:0/801701875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560c3f44080 0x5560c3f4e400 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c009ba0 tx=0x7f5e7c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51603a65c3eacb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:20.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.689+0000 7f5e86d9d640 1 --2- 172.21.15.120:0/693761554 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5e700a3240 0x7f5e700a5700 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5e80004af0 tx=0x7f5e80005bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:20.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.689+0000 7f5e74ff9640 1 -- 172.21.15.120:0/801701875 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c004230 con 0x5560c3f44080 2024-09-19T09:22:20.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.689+0000 7f5e74ff9640 1 -- 172.21.15.120:0/801701875 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e7c009e10 con 0x5560c3f44080 2024-09-19T09:22:20.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.689+0000 7f5e74ff9640 1 -- 172.21.15.120:0/801701875 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c004dd0 con 0x5560c3f44080 2024-09-19T09:22:20.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.690+0000 7f5e8ecf0740 1 -- 172.21.15.120:0/801701875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560c3f44080 msgr2=0x5560c3f4e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:20.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.690+0000 7f5e8ecf0740 1 --2- 172.21.15.120:0/801701875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560c3f44080 0x5560c3f4e400 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c009ba0 tx=0x7f5e7c011810 comp rx=0 tx=0).stop 2024-09-19T09:22:20.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.690+0000 7f5e8ecf0740 1 -- 172.21.15.120:0/801701875 shutdown_connections 2024-09-19T09:22:20.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.690+0000 7f5e8ecf0740 1 --2- 172.21.15.120:0/801701875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560c3f4e940 0x5560c3f50d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:20.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.690+0000 7f5e8ecf0740 1 --2- 172.21.15.120:0/801701875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560c3f44080 0x5560c3f4e400 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:20.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.690+0000 7f5e8ecf0740 1 --2- 172.21.15.120:0/801701875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560c3f43760 0x5560c3f43b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:20.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.690+0000 7f5e8ecf0740 1 -- 172.21.15.120:0/801701875 >> 172.21.15.120:0/801701875 conn(0x5560c3f3fcf0 msgr2=0x5560c3f400d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:20.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.691+0000 7f5e8ecf0740 1 -- 172.21.15.120:0/801701875 shutdown_connections 2024-09-19T09:22:20.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.691+0000 7f5e8ecf0740 1 -- 172.21.15.120:0/801701875 wait complete. 2024-09-19T09:22:20.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.691+0000 7f5e8ecf0740 1 -- start start 2024-09-19T09:22:20.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.691+0000 7f5e8ecf0740 1 Processor -- start 2024-09-19T09:22:20.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.691+0000 7f5e8ecf0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560c3f43760 0x5560c3f4b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:20.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.691+0000 7f5e8ecf0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560c3f44080 0x5560c3f4bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:20.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.691+0000 7f5e8ecf0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560c3f4e940 0x5560c3f4c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:20.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.691+0000 7f5e8ecf0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5560c3f55920 con 0x5560c3f4e940 2024-09-19T09:22:20.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.691+0000 7f5e8ecf0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5560c3e316e0 con 0x5560c3f44080 2024-09-19T09:22:20.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.691+0000 7f5e8ecf0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5560c3f55aa0 con 0x5560c3f43760 2024-09-19T09:22:20.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.692+0000 7f5e86d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560c3f4e940 0x5560c3f4c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:20.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.692+0000 7f5e85d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560c3f44080 0x5560c3f4bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:20.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.692+0000 7f5e86d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560c3f4e940 0x5560c3f4c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43270/0 (socket says 172.21.15.120:43270) 2024-09-19T09:22:20.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.692+0000 7f5e8659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560c3f43760 0x5560c3f4b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:20.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.692+0000 7f5e85d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560c3f44080 0x5560c3f4bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:58034/0 (socket says 172.21.15.120:58034) 2024-09-19T09:22:20.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.692+0000 7f5e8ecf0740 10 client.0 _ll_register_callbacks cb 0x5560c3e55a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:22:20.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.692+0000 7f5e86d9d640 1 -- 172.21.15.120:0/112800041 learned_addr learned my addr 172.21.15.120:0/112800041 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:20.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.692+0000 7f5e8659c640 1 -- 172.21.15.120:0/112800041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560c3f44080 msgr2=0x5560c3f4bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:20.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.692+0000 7f5e8659c640 1 --2- 172.21.15.120:0/112800041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560c3f44080 0x5560c3f4bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:20.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.692+0000 7f5e8659c640 1 -- 172.21.15.120:0/112800041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560c3f4e940 msgr2=0x5560c3f4c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:20.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.692+0000 7f5e8659c640 1 --2- 172.21.15.120:0/112800041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560c3f4e940 0x5560c3f4c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:20.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.692+0000 7f5e8659c640 1 -- 172.21.15.120:0/112800041 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560c3f5ede0 con 0x5560c3f43760 2024-09-19T09:22:20.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.692+0000 7f5e8659c640 1 --2- 172.21.15.120:0/112800041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560c3f43760 0x5560c3f4b800 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5e7809c180 tx=0x7f5e7809c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7156233d56f064c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:20.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.692+0000 7f5e6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:22:20.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.692+0000 7f5e6dffb640 1 -- 172.21.15.120:0/112800041 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e780b5060 con 0x5560c3f43760 2024-09-19T09:22:20.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.692+0000 7f5e6dffb640 1 -- 172.21.15.120:0/112800041 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f5e780a0150 con 0x5560c3f43760 2024-09-19T09:22:20.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.692+0000 7f5e8ecf0740 1 -- 172.21.15.120:0/112800041 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e7c009e10 con 0x5560c3f43760 2024-09-19T09:22:20.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.693+0000 7f5e6dffb640 1 -- 172.21.15.120:0/112800041 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5e780bc2c0 con 0x5560c3f43760 2024-09-19T09:22:20.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.693+0000 7f5e8ecf0740 10 client.30547 fetch_fsmap learned FSMap version 39 2024-09-19T09:22:20.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.693+0000 7f5e8ecf0740 1 -- 172.21.15.120:0/112800041 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5560c3f51270 con 0x5560c3f43760 2024-09-19T09:22:20.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.693+0000 7f5e6dffb640 1 -- 172.21.15.120:0/112800041 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e780a8070 con 0x5560c3f43760 2024-09-19T09:22:20.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.693+0000 7f5e6dffb640 1 -- 172.21.15.120:0/112800041 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f5e7809b640 con 0x5560c3f43760 2024-09-19T09:22:20.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.693+0000 7f5e8ecf0740 10 client.30547 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:22:20.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.693+0000 7f5e8ecf0740 10 client.30547 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:22:20.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.693+0000 7f5e8ecf0740 20 client.30547 populate_metadata read hostname 'smithi120' 2024-09-19T09:22:20.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.693+0000 7f5e8ecf0740 1 --2- 172.21.15.120:0/112800041 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5560c3f52680 0x5560c3f54ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:20.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.693+0000 7f5e8ecf0740 4 client.30547 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:22:20.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.693+0000 7f5e8ecf0740 1 -- 172.21.15.120:0/112800041 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560c3eb69d0 con 0x5560c3f52680 2024-09-19T09:22:20.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.693+0000 7f5e85d9b640 1 --2- 172.21.15.120:0/112800041 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5560c3f52680 0x5560c3f54ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:20.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.694+0000 7f5e6dffb640 10 client.30547 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:20.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.694+0000 7f5e85d9b640 1 --2- 172.21.15.120:0/112800041 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5560c3f52680 0x5560c3f54ad0 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a80a402151ca2618 server_cookie=beb93296666e73d4 in_seq=0 out_seq=0 2024-09-19T09:22:20.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.697+0000 7f5e6dffb640 1 -- 172.21.15.120:0/112800041 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5560c3eb69d0 con 0x5560c3f52680 2024-09-19T09:22:20.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.697+0000 7f5e6dffb640 10 client.30547 handle_command_reply: tid=0 2024-09-19T09:22:20.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.698+0000 7f5e8ecf0740 1 client.30547 shutdown 2024-09-19T09:22:20.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.698+0000 7f5e8ecf0740 1 -- 172.21.15.120:0/112800041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560c3f43760 msgr2=0x5560c3f4b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:20.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.698+0000 7f5e8ecf0740 1 --2- 172.21.15.120:0/112800041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560c3f43760 0x5560c3f4b800 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5e7809c180 tx=0x7f5e7809c8f0 comp rx=0 tx=0).stop 2024-09-19T09:22:20.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.698+0000 7f5e8ecf0740 1 -- 172.21.15.120:0/112800041 shutdown_connections 2024-09-19T09:22:20.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.698+0000 7f5e8ecf0740 1 --2- 172.21.15.120:0/112800041 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5560c3f52680 0x5560c3f54ad0 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:20.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.698+0000 7f5e6dffb640 0 client.30547 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:20.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.698+0000 7f5e8ecf0740 1 --2- 172.21.15.120:0/112800041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560c3f4e940 0x5560c3f4c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:20.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.698+0000 7f5e8ecf0740 1 --2- 172.21.15.120:0/112800041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560c3f44080 0x5560c3f4bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:20.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.698+0000 7f5e8ecf0740 1 --2- 172.21.15.120:0/112800041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560c3f43760 0x5560c3f4b800 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:20.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.698+0000 7f5e8ecf0740 1 -- 172.21.15.120:0/112800041 >> 172.21.15.120:0/112800041 conn(0x5560c3f3fcf0 msgr2=0x5560c3f45a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:20.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.698+0000 7f5e8ecf0740 1 -- 172.21.15.120:0/112800041 shutdown_connections 2024-09-19T09:22:20.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.698+0000 7f5e8ecf0740 1 -- 172.21.15.120:0/112800041 wait complete. 2024-09-19T09:22:20.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.698+0000 7f5e8ecf0740 20 client.30547 trim_cache size 0 max 16384 2024-09-19T09:22:20.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.725+0000 7f5e8ecf0740 1 Processor -- start 2024-09-19T09:22:20.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.725+0000 7f5e8ecf0740 1 -- start start 2024-09-19T09:22:20.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.725+0000 7f5e8ecf0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560c3f43760 0x5560c3f666e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:20.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.726+0000 7f5e8ecf0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560c3f44080 0x5560c3f66c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:20.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.726+0000 7f5e8ecf0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560c3f4e940 0x5560c3f48a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:20.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.726+0000 7f5e8ecf0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5e881ab3e0 con 0x5560c3f43760 2024-09-19T09:22:20.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.726+0000 7f5e8ecf0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5e881acf20 con 0x5560c3f44080 2024-09-19T09:22:20.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.726+0000 7f5e8ecf0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5e881abab0 con 0x5560c3f4e940 2024-09-19T09:22:20.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.726+0000 7f5e8659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560c3f43760 0x5560c3f666e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:20.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.726+0000 7f5e8659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560c3f43760 0x5560c3f666e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43276/0 (socket says 172.21.15.120:43276) 2024-09-19T09:22:20.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.726+0000 7f5e8659c640 1 -- 172.21.15.120:0/3843536235 learned_addr learned my addr 172.21.15.120:0/3843536235 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:20.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.726+0000 7f5e85d9b640 1 --2- 172.21.15.120:0/3843536235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560c3f44080 0x5560c3f66c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:20.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.726+0000 7f5e86d9d640 1 --2- 172.21.15.120:0/3843536235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560c3f4e940 0x5560c3f48a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:20.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.726+0000 7f5e8659c640 1 -- 172.21.15.120:0/3843536235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560c3f4e940 msgr2=0x5560c3f48a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:20.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.726+0000 7f5e8659c640 1 --2- 172.21.15.120:0/3843536235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560c3f4e940 0x5560c3f48a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:20.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.726+0000 7f5e8659c640 1 -- 172.21.15.120:0/3843536235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560c3f44080 msgr2=0x5560c3f66c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:20.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.726+0000 7f5e8659c640 1 --2- 172.21.15.120:0/3843536235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560c3f44080 0x5560c3f66c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:20.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.726+0000 7f5e8659c640 1 -- 172.21.15.120:0/3843536235 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560c3f3b260 con 0x5560c3f43760 2024-09-19T09:22:20.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.726+0000 7f5e85d9b640 1 --2- 172.21.15.120:0/3843536235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560c3f44080 0x5560c3f66c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:22:20.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.726+0000 7f5e8659c640 1 --2- 172.21.15.120:0/3843536235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560c3f43760 0x5560c3f666e0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f5e78004b50 tx=0x7f5e780049b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe4e61f39eec11a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:20.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.727+0000 7f5e6ffff640 1 -- 172.21.15.120:0/3843536235 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e780b5040 con 0x5560c3f43760 2024-09-19T09:22:20.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.727+0000 7f5e6ffff640 1 -- 172.21.15.120:0/3843536235 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e780a8070 con 0x5560c3f43760 2024-09-19T09:22:20.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.727+0000 7f5e6ffff640 1 -- 172.21.15.120:0/3843536235 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e780a49d0 con 0x5560c3f43760 2024-09-19T09:22:20.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.728+0000 7f5e8ecf0740 1 -- 172.21.15.120:0/3843536235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560c3f43760 msgr2=0x5560c3f666e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:20.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.728+0000 7f5e8ecf0740 1 --2- 172.21.15.120:0/3843536235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560c3f43760 0x5560c3f666e0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f5e78004b50 tx=0x7f5e780049b0 comp rx=0 tx=0).stop 2024-09-19T09:22:20.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.728+0000 7f5e8ecf0740 1 -- 172.21.15.120:0/3843536235 shutdown_connections 2024-09-19T09:22:20.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.728+0000 7f5e8ecf0740 1 --2- 172.21.15.120:0/3843536235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560c3f4e940 0x5560c3f48a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:20.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.728+0000 7f5e8ecf0740 1 --2- 172.21.15.120:0/3843536235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560c3f44080 0x5560c3f66c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:20.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.728+0000 7f5e8ecf0740 1 --2- 172.21.15.120:0/3843536235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560c3f43760 0x5560c3f666e0 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:20.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.728+0000 7f5e8ecf0740 1 -- 172.21.15.120:0/3843536235 >> 172.21.15.120:0/3843536235 conn(0x5560c3f3fcf0 msgr2=0x5560c3f4d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:20.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.728+0000 7f5e8ecf0740 1 -- 172.21.15.120:0/3843536235 shutdown_connections 2024-09-19T09:22:20.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.728+0000 7f5e8ecf0740 1 -- 172.21.15.120:0/3843536235 wait complete. 2024-09-19T09:22:20.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.728+0000 7f5e8ecf0740 1 -- start start 2024-09-19T09:22:20.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.728+0000 7f5e8ecf0740 1 Processor -- start 2024-09-19T09:22:20.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.729+0000 7f5e8ecf0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560c3f43760 0x5560c3f64c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:20.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.729+0000 7f5e8ecf0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560c3f44080 0x5560c3f65140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:20.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.729+0000 7f5e8659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560c3f43760 0x5560c3f64c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:20.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.729+0000 7f5e8659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560c3f43760 0x5560c3f64c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:38580/0 (socket says 172.21.15.120:38580) 2024-09-19T09:22:20.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.729+0000 7f5e8659c640 1 -- 172.21.15.120:0/1892749393 learned_addr learned my addr 172.21.15.120:0/1892749393 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:20.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.729+0000 7f5e85d9b640 1 --2- 172.21.15.120:0/1892749393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560c3f44080 0x5560c3f65140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:20.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.729+0000 7f5e8ecf0740 1 --2- 172.21.15.120:0/1892749393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560c3f4e940 0x5560c3fc8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:20.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.729+0000 7f5e8ecf0740 1 -- 172.21.15.120:0/1892749393 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5e881ac850 con 0x5560c3f4e940 2024-09-19T09:22:20.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.729+0000 7f5e8ecf0740 1 -- 172.21.15.120:0/1892749393 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5e881ac180 con 0x5560c3f44080 2024-09-19T09:22:20.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.730+0000 7f5e8ecf0740 1 -- 172.21.15.120:0/1892749393 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5e881a93a0 con 0x5560c3f43760 2024-09-19T09:22:20.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.730+0000 7f5e86d9d640 1 --2- 172.21.15.120:0/1892749393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560c3f4e940 0x5560c3fc8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:20.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.730+0000 7f5e8659c640 1 -- 172.21.15.120:0/1892749393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560c3f44080 msgr2=0x5560c3f65140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:20.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.730+0000 7f5e8659c640 1 --2- 172.21.15.120:0/1892749393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560c3f44080 0x5560c3f65140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:20.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.730+0000 7f5e8659c640 1 -- 172.21.15.120:0/1892749393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560c3f4e940 msgr2=0x5560c3fc8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:20.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.730+0000 7f5e8ecf0740 10 client.0 _ll_register_callbacks cb 0x7f5e7809bb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:22:20.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.730+0000 7f5e8659c640 1 --2- 172.21.15.120:0/1892749393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560c3f4e940 0x5560c3fc8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:20.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.730+0000 7f5e8659c640 1 -- 172.21.15.120:0/1892749393 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560c3f40980 con 0x5560c3f43760 2024-09-19T09:22:20.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.730+0000 7f5e8ecf0740 1 -- 172.21.15.120:0/1892749393 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e780a8070 con 0x5560c3f43760 2024-09-19T09:22:20.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.730+0000 7f5e86d9d640 1 --2- 172.21.15.120:0/1892749393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560c3f4e940 0x5560c3fc8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:22:20.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.730+0000 7f5e8659c640 1 --2- 172.21.15.120:0/1892749393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560c3f43760 0x5560c3f64c00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5e7809bfe0 tx=0x7f5e780cd770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=998af26d7c3a8531 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:20.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.731+0000 7f5e6dffb640 10 client.30553 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:22:20.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.731+0000 7f5e6dffb640 1 -- 172.21.15.120:0/1892749393 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e780b9020 con 0x5560c3f43760 2024-09-19T09:22:20.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.731+0000 7f5e6dffb640 1 -- 172.21.15.120:0/1892749393 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f5e780a4bd0 con 0x5560c3f43760 2024-09-19T09:22:20.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.731+0000 7f5e6dffb640 1 -- 172.21.15.120:0/1892749393 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5e780b5040 con 0x5560c3f43760 2024-09-19T09:22:20.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.731+0000 7f5e8ecf0740 10 client.30553 fetch_fsmap learned FSMap version 39 2024-09-19T09:22:20.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.731+0000 7f5e8ecf0740 1 -- 172.21.15.120:0/1892749393 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({fsmap=39}) -- 0x5560c3f40980 con 0x5560c3f43760 2024-09-19T09:22:20.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.731+0000 7f5e6dffb640 1 -- 172.21.15.120:0/1892749393 <== mon.1 v2:172.21.15.196:3300/0 4 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f5e780cdb90 con 0x5560c3f43760 2024-09-19T09:22:20.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.731+0000 7f5e8ecf0740 10 client.30553 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:22:20.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.731+0000 7f5e8ecf0740 10 client.30553 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:22:20.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.732+0000 7f5e8ecf0740 20 client.30553 populate_metadata read hostname 'smithi120' 2024-09-19T09:22:20.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.732+0000 7f5e8ecf0740 1 --2- 172.21.15.120:0/1892749393 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5560c3f52680 0x5560c3fcbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:20.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.732+0000 7f5e8ecf0740 4 client.30553 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.0, "await_for": 45.0}] 2024-09-19T09:22:20.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.732+0000 7f5e8ecf0740 1 -- 172.21.15.120:0/1892749393 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.0, "await_for": 45.0}) -- 0x5560c3eb0d20 con 0x5560c3f52680 2024-09-19T09:22:20.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.732+0000 7f5e85d9b640 1 --2- 172.21.15.120:0/1892749393 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5560c3f52680 0x5560c3fcbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:20.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.732+0000 7f5e6dffb640 10 client.30553 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:20.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:20.732+0000 7f5e85d9b640 1 --2- 172.21.15.120:0/1892749393 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5560c3f52680 0x5560c3fcbe60 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fbab94738ca8df6 server_cookie=4589c73b9ac857a0 in_seq=0 out_seq=0 2024-09-19T09:22:20.738 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:22:21.373 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159146 21.77 MB/sec execute 256 sec latency 625.767 ms 2024-09-19T09:22:21.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.612+0000 7fac1eba0640 1 -- 172.21.15.120:0/1187670879 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac18106df0 msgr2=0x7fac1810e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:21.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.612+0000 7fac1eba0640 1 --2- 172.21.15.120:0/1187670879 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac18106df0 0x7fac1810e920 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7fac10009930 tx=0x7fac1002a420 comp rx=0 tx=0).stop 2024-09-19T09:22:21.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.612+0000 7fac1eba0640 1 -- 172.21.15.120:0/1187670879 shutdown_connections 2024-09-19T09:22:21.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.612+0000 7fac1eba0640 1 --2- 172.21.15.120:0/1187670879 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac18106df0 0x7fac1810e920 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.612+0000 7fac1eba0640 1 --2- 172.21.15.120:0/1187670879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac18100b50 0x7fac18106690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.612+0000 7fac1eba0640 1 --2- 172.21.15.120:0/1187670879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac181001a0 0x7fac18100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.612+0000 7fac1eba0640 1 -- 172.21.15.120:0/1187670879 >> 172.21.15.120:0/1187670879 conn(0x7fac18077070 msgr2=0x7fac180fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:21.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.613+0000 7fac1eba0640 1 -- 172.21.15.120:0/1187670879 shutdown_connections 2024-09-19T09:22:21.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.613+0000 7fac1eba0640 1 -- 172.21.15.120:0/1187670879 wait complete. 2024-09-19T09:22:21.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.613+0000 7fac1eba0640 1 Processor -- start 2024-09-19T09:22:21.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.613+0000 7fac1eba0640 1 -- start start 2024-09-19T09:22:21.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.613+0000 7fac1eba0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac181001a0 0x7fac18198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:21.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.613+0000 7fac1eba0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac18100b50 0x7fac18198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:21.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.613+0000 7fac1eba0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac18106df0 0x7fac1819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:21.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.613+0000 7fac1eba0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fac18111e20 con 0x7fac181001a0 2024-09-19T09:22:21.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.613+0000 7fac1eba0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fac18111ca0 con 0x7fac18100b50 2024-09-19T09:22:21.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.613+0000 7fac1eba0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fac18111fa0 con 0x7fac18106df0 2024-09-19T09:22:21.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.613+0000 7fac1db9e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac181001a0 0x7fac18198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:21.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.613+0000 7fac1db9e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac181001a0 0x7fac18198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52516/0 (socket says 172.21.15.120:52516) 2024-09-19T09:22:21.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.613+0000 7fac1db9e640 1 -- 172.21.15.120:0/1979201165 learned_addr learned my addr 172.21.15.120:0/1979201165 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:21.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.614+0000 7fac1d39d640 1 --2- 172.21.15.120:0/1979201165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac18100b50 0x7fac18198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:21.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.614+0000 7fac1e39f640 1 --2- 172.21.15.120:0/1979201165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac18106df0 0x7fac1819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:21.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.614+0000 7fac1db9e640 1 -- 172.21.15.120:0/1979201165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac18106df0 msgr2=0x7fac1819cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:21.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.614+0000 7fac1db9e640 1 --2- 172.21.15.120:0/1979201165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac18106df0 0x7fac1819cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.614+0000 7fac1db9e640 1 -- 172.21.15.120:0/1979201165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac18100b50 msgr2=0x7fac18198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:21.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.614+0000 7fac1db9e640 1 --2- 172.21.15.120:0/1979201165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac18100b50 0x7fac18198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.614+0000 7fac1db9e640 1 -- 172.21.15.120:0/1979201165 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1819d200 con 0x7fac181001a0 2024-09-19T09:22:21.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.614+0000 7fac1db9e640 1 --2- 172.21.15.120:0/1979201165 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac181001a0 0x7fac18198250 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7fac1400eb90 tx=0x7fac1400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:21.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.614+0000 7fac0effd640 1 -- 172.21.15.120:0/1979201165 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac14004d20 con 0x7fac181001a0 2024-09-19T09:22:21.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.614+0000 7fac1eba0640 1 -- 172.21.15.120:0/1979201165 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac1819d4f0 con 0x7fac181001a0 2024-09-19T09:22:21.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.614+0000 7fac0effd640 1 -- 172.21.15.120:0/1979201165 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac1400ee90 con 0x7fac181001a0 2024-09-19T09:22:21.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.614+0000 7fac0effd640 1 -- 172.21.15.120:0/1979201165 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac14010830 con 0x7fac181001a0 2024-09-19T09:22:21.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.614+0000 7fac1eba0640 1 -- 172.21.15.120:0/1979201165 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac181a4dd0 con 0x7fac181001a0 2024-09-19T09:22:21.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.617+0000 7fac0effd640 1 -- 172.21.15.120:0/1979201165 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fac14005020 con 0x7fac181001a0 2024-09-19T09:22:21.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.617+0000 7fac221db740 1 Processor -- start 2024-09-19T09:22:21.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.618+0000 7fac221db740 1 -- start start 2024-09-19T09:22:21.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.618+0000 7fac221db740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55560ff4dd20 0x55560ff4c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:21.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.618+0000 7fac221db740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55560ff4c7b0 0x55560ff4cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:21.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.618+0000 7fac221db740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55560ff57550 0x55560ff59940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:21.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.618+0000 7fac221db740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55560feb22f0 con 0x55560ff4c7b0 2024-09-19T09:22:21.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.618+0000 7fac221db740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55560fe59cf0 con 0x55560ff57550 2024-09-19T09:22:21.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.618+0000 7fac221db740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55560feae2b0 con 0x55560ff4dd20 2024-09-19T09:22:21.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.618+0000 7fac0effd640 1 --2- 172.21.15.120:0/1979201165 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fac040a3240 0x7fac040a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:21.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.618+0000 7fac1db9e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55560ff4c7b0 0x55560ff4cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:21.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.618+0000 7fac1db9e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55560ff4c7b0 0x55560ff4cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52528/0 (socket says 172.21.15.120:52528) 2024-09-19T09:22:21.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.618+0000 7fac1e39f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55560ff4dd20 0x55560ff4c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:21.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.618+0000 7fac1db9e640 1 -- 172.21.15.120:0/3224436149 learned_addr learned my addr 172.21.15.120:0/3224436149 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:21.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.618+0000 7fac1d39d640 1 --2- 172.21.15.120:0/3224436149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55560ff57550 0x55560ff59940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:21.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.618+0000 7fac1e39f640 1 --2- 172.21.15.120:0/1979201165 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fac040a3240 0x7fac040a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:21.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.618+0000 7fac0effd640 1 -- 172.21.15.120:0/1979201165 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fac140cff50 con 0x7fac181001a0 2024-09-19T09:22:21.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.618+0000 7fac1d39d640 1 -- 172.21.15.120:0/3224436149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55560ff4dd20 msgr2=0x55560ff4c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:21.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.618+0000 7fac1d39d640 1 --2- 172.21.15.120:0/3224436149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55560ff4dd20 0x55560ff4c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.618+0000 7fac1d39d640 1 -- 172.21.15.120:0/3224436149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55560ff4c7b0 msgr2=0x55560ff4cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:21.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.618+0000 7fac1d39d640 1 --2- 172.21.15.120:0/3224436149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55560ff4c7b0 0x55560ff4cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.618+0000 7fac1d39d640 1 -- 172.21.15.120:0/3224436149 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55560fe3e3a0 con 0x55560ff57550 2024-09-19T09:22:21.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.618+0000 7fac1db9e640 1 --2- 172.21.15.120:0/3224436149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55560ff4c7b0 0x55560ff4cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:22:21.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.618+0000 7fac1e39f640 1 --2- 172.21.15.120:0/3224436149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55560ff4dd20 0x55560ff4c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:22:21.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.619+0000 7fac1d39d640 1 --2- 172.21.15.120:0/3224436149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55560ff57550 0x55560ff59940 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fac1806a080 tx=0x7fac08011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cd941cc43796c54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:21.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.619+0000 7fac1e39f640 1 --2- 172.21.15.120:0/1979201165 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fac040a3240 0x7fac040a5700 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fac10004770 tx=0x7fac100046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:21.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.619+0000 7fac03fff640 1 -- 172.21.15.120:0/3224436149 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac08009d70 con 0x55560ff57550 2024-09-19T09:22:21.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.619+0000 7fac03fff640 1 -- 172.21.15.120:0/3224436149 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac08005020 con 0x55560ff57550 2024-09-19T09:22:21.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.619+0000 7fac03fff640 1 -- 172.21.15.120:0/3224436149 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac08004b90 con 0x55560ff57550 2024-09-19T09:22:21.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.619+0000 7fac221db740 1 -- 172.21.15.120:0/3224436149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55560ff57550 msgr2=0x55560ff59940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:21.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.619+0000 7fac221db740 1 --2- 172.21.15.120:0/3224436149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55560ff57550 0x55560ff59940 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fac1806a080 tx=0x7fac08011810 comp rx=0 tx=0).stop 2024-09-19T09:22:21.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.619+0000 7fac221db740 1 -- 172.21.15.120:0/3224436149 shutdown_connections 2024-09-19T09:22:21.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.619+0000 7fac221db740 1 --2- 172.21.15.120:0/3224436149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55560ff57550 0x55560ff59940 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.619+0000 7fac221db740 1 --2- 172.21.15.120:0/3224436149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55560ff4c7b0 0x55560ff4cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.619+0000 7fac221db740 1 --2- 172.21.15.120:0/3224436149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55560ff4dd20 0x55560ff4c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.619+0000 7fac221db740 1 -- 172.21.15.120:0/3224436149 >> 172.21.15.120:0/3224436149 conn(0x55560ff15310 msgr2=0x55560ff48e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:21.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.620+0000 7fac221db740 1 -- 172.21.15.120:0/3224436149 shutdown_connections 2024-09-19T09:22:21.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.620+0000 7fac221db740 1 -- 172.21.15.120:0/3224436149 wait complete. 2024-09-19T09:22:21.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.620+0000 7fac221db740 1 -- start start 2024-09-19T09:22:21.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.620+0000 7fac221db740 1 Processor -- start 2024-09-19T09:22:21.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.620+0000 7fac221db740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55560ff4c7b0 0x55560ff56ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:21.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.620+0000 7fac221db740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55560ff4dd20 0x55560ff54150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:21.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.620+0000 7fac221db740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55560ff57550 0x55560ff546b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:21.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.620+0000 7fac221db740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55560ff5a250 con 0x55560ff57550 2024-09-19T09:22:21.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.620+0000 7fac221db740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55560fe6d060 con 0x55560ff4dd20 2024-09-19T09:22:21.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.620+0000 7fac221db740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55560ff5a3d0 con 0x55560ff4c7b0 2024-09-19T09:22:21.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.620+0000 7fac221db740 10 client.0 _ll_register_callbacks cb 0x55560fbc08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:22:21.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.620+0000 7fac1e39f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55560ff57550 0x55560ff546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:21.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.621+0000 7fac1e39f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55560ff57550 0x55560ff546b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52540/0 (socket says 172.21.15.120:52540) 2024-09-19T09:22:21.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.621+0000 7fac1e39f640 1 -- 172.21.15.120:0/1468841576 learned_addr learned my addr 172.21.15.120:0/1468841576 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:21.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.621+0000 7fac1db9e640 1 --2- 172.21.15.120:0/1468841576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55560ff4c7b0 0x55560ff56ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:21.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.621+0000 7fac1d39d640 1 --2- 172.21.15.120:0/1468841576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55560ff4dd20 0x55560ff54150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:21.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.621+0000 7fac1e39f640 1 -- 172.21.15.120:0/1468841576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55560ff4c7b0 msgr2=0x55560ff56ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:21.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.621+0000 7fac1e39f640 1 --2- 172.21.15.120:0/1468841576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55560ff4c7b0 0x55560ff56ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.621+0000 7fac1e39f640 1 -- 172.21.15.120:0/1468841576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55560ff4dd20 msgr2=0x55560ff54150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:21.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.621+0000 7fac1e39f640 1 --2- 172.21.15.120:0/1468841576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55560ff4dd20 0x55560ff54150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.621+0000 7fac1e39f640 1 -- 172.21.15.120:0/1468841576 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55560fc233c0 con 0x55560ff57550 2024-09-19T09:22:21.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.621+0000 7fac1e39f640 1 --2- 172.21.15.120:0/1468841576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55560ff57550 0x55560ff546b0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7fac10005c40 tx=0x7fac10009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f54f40aa7784fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:21.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.621+0000 7fac017fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:22:21.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.621+0000 7fac017fa640 1 -- 172.21.15.120:0/1468841576 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10035070 con 0x55560ff57550 2024-09-19T09:22:21.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.621+0000 7fac017fa640 1 -- 172.21.15.120:0/1468841576 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fac10005020 con 0x55560ff57550 2024-09-19T09:22:21.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.621+0000 7fac221db740 1 -- 172.21.15.120:0/1468841576 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac08005020 con 0x55560ff57550 2024-09-19T09:22:21.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.621+0000 7fac017fa640 1 -- 172.21.15.120:0/1468841576 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fac10005220 con 0x55560ff57550 2024-09-19T09:22:21.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.621+0000 7fac221db740 10 client.21906 fetch_fsmap learned FSMap version 39 2024-09-19T09:22:21.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.621+0000 7fac221db740 1 -- 172.21.15.120:0/1468841576 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55560ff65c60 con 0x55560ff57550 2024-09-19T09:22:21.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.621+0000 7fac017fa640 1 -- 172.21.15.120:0/1468841576 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac08005020 con 0x55560ff57550 2024-09-19T09:22:21.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.622+0000 7fac017fa640 1 -- 172.21.15.120:0/1468841576 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fac10054030 con 0x55560ff57550 2024-09-19T09:22:21.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.622+0000 7fac221db740 10 client.21906 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:22:21.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.622+0000 7fac221db740 10 client.21906 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:22:21.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.622+0000 7fac221db740 20 client.21906 populate_metadata read hostname 'smithi120' 2024-09-19T09:22:21.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.622+0000 7fac221db740 1 --2- 172.21.15.120:0/1468841576 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55560ff67120 0x55560ff69590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:21.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.622+0000 7fac221db740 4 client.21906 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:22:21.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.622+0000 7fac221db740 1 -- 172.21.15.120:0/1468841576 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55560febf860 con 0x55560ff67120 2024-09-19T09:22:21.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.622+0000 7fac1db9e640 1 --2- 172.21.15.120:0/1468841576 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55560ff67120 0x55560ff69590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:21.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.622+0000 7fac017fa640 10 client.21906 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:22:21.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.622+0000 7fac1db9e640 1 --2- 172.21.15.120:0/1468841576 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55560ff67120 0x55560ff69590 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f37ecbea4503bd30 server_cookie=7ccb776330ea3584 in_seq=0 out_seq=0 2024-09-19T09:22:21.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.623+0000 7fac017fa640 1 -- 172.21.15.120:0/1468841576 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55560febf860 con 0x55560ff67120 2024-09-19T09:22:21.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.623+0000 7fac017fa640 10 client.21906 handle_command_reply: tid=0 2024-09-19T09:22:21.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.623+0000 7fac221db740 1 client.21906 shutdown 2024-09-19T09:22:21.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.623+0000 7fac221db740 1 -- 172.21.15.120:0/1468841576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55560ff57550 msgr2=0x55560ff546b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:21.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.623+0000 7fac221db740 1 --2- 172.21.15.120:0/1468841576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55560ff57550 0x55560ff546b0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7fac10005c40 tx=0x7fac10009260 comp rx=0 tx=0).stop 2024-09-19T09:22:21.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.623+0000 7fac221db740 1 -- 172.21.15.120:0/1468841576 shutdown_connections 2024-09-19T09:22:21.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.623+0000 7fac221db740 1 --2- 172.21.15.120:0/1468841576 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55560ff67120 0x55560ff69590 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.623+0000 7fac221db740 1 --2- 172.21.15.120:0/1468841576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55560ff57550 0x55560ff546b0 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.623+0000 7fac221db740 1 --2- 172.21.15.120:0/1468841576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55560ff4dd20 0x55560ff54150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.623+0000 7fac017fa640 0 client.21906 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:22:21.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.623+0000 7fac221db740 1 --2- 172.21.15.120:0/1468841576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55560ff4c7b0 0x55560ff56ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.623+0000 7fac221db740 1 -- 172.21.15.120:0/1468841576 >> 172.21.15.120:0/1468841576 conn(0x55560ff15310 msgr2=0x55560ff4eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:21.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.623+0000 7fac221db740 1 -- 172.21.15.120:0/1468841576 shutdown_connections 2024-09-19T09:22:21.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.623+0000 7fac221db740 1 -- 172.21.15.120:0/1468841576 wait complete. 2024-09-19T09:22:21.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.623+0000 7fac221db740 20 client.21906 trim_cache size 0 max 16384 2024-09-19T09:22:21.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.636+0000 7fac221db740 1 Processor -- start 2024-09-19T09:22:21.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.636+0000 7fac221db740 1 -- start start 2024-09-19T09:22:21.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.636+0000 7fac221db740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55560ff4c7b0 0x55560ff560c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:21.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.636+0000 7fac221db740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55560ff4dd20 0x55560ff4f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:21.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.636+0000 7fac221db740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55560ff57550 0x55560ff4f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:21.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.636+0000 7fac221db740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fac181ad4a0 con 0x55560ff4dd20 2024-09-19T09:22:21.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.636+0000 7fac221db740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fac181aefe0 con 0x55560ff57550 2024-09-19T09:22:21.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.636+0000 7fac221db740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fac181adb70 con 0x55560ff4c7b0 2024-09-19T09:22:21.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.636+0000 7fac1d39d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55560ff4dd20 0x55560ff4f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:21.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.636+0000 7fac1db9e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55560ff4c7b0 0x55560ff560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:21.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.636+0000 7fac1d39d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55560ff4dd20 0x55560ff4f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52550/0 (socket says 172.21.15.120:52550) 2024-09-19T09:22:21.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.636+0000 7fac1d39d640 1 -- 172.21.15.120:0/1044734662 learned_addr learned my addr 172.21.15.120:0/1044734662 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:21.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.636+0000 7fac1e39f640 1 --2- 172.21.15.120:0/1044734662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55560ff57550 0x55560ff4f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:21.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.636+0000 7fac1d39d640 1 -- 172.21.15.120:0/1044734662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55560ff4c7b0 msgr2=0x55560ff560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:21.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.636+0000 7fac1d39d640 1 --2- 172.21.15.120:0/1044734662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55560ff4c7b0 0x55560ff560c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.636+0000 7fac1d39d640 1 -- 172.21.15.120:0/1044734662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55560ff57550 msgr2=0x55560ff4f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:21.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.636+0000 7fac1d39d640 1 --2- 172.21.15.120:0/1044734662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55560ff57550 0x55560ff4f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.636+0000 7fac1d39d640 1 -- 172.21.15.120:0/1044734662 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55560ff4fd80 con 0x55560ff4dd20 2024-09-19T09:22:21.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.636+0000 7fac1db9e640 1 --2- 172.21.15.120:0/1044734662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55560ff4c7b0 0x55560ff560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:22:21.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.637+0000 7fac1d39d640 1 --2- 172.21.15.120:0/1044734662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55560ff4dd20 0x55560ff4f300 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7fac08014040 tx=0x7fac08009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78570665d16fcf2b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:21.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.637+0000 7fac037fe640 1 -- 172.21.15.120:0/1044734662 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac08026020 con 0x55560ff4dd20 2024-09-19T09:22:21.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.637+0000 7fac037fe640 1 -- 172.21.15.120:0/1044734662 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac08005590 con 0x55560ff4dd20 2024-09-19T09:22:21.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.637+0000 7fac037fe640 1 -- 172.21.15.120:0/1044734662 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac08019410 con 0x55560ff4dd20 2024-09-19T09:22:21.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.637+0000 7fac221db740 1 -- 172.21.15.120:0/1044734662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55560ff4dd20 msgr2=0x55560ff4f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:21.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.637+0000 7fac221db740 1 --2- 172.21.15.120:0/1044734662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55560ff4dd20 0x55560ff4f300 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7fac08014040 tx=0x7fac08009d70 comp rx=0 tx=0).stop 2024-09-19T09:22:21.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.637+0000 7fac221db740 1 -- 172.21.15.120:0/1044734662 shutdown_connections 2024-09-19T09:22:21.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.637+0000 7fac221db740 1 --2- 172.21.15.120:0/1044734662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55560ff57550 0x55560ff4f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.637+0000 7fac221db740 1 --2- 172.21.15.120:0/1044734662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55560ff4dd20 0x55560ff4f300 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.637+0000 7fac221db740 1 --2- 172.21.15.120:0/1044734662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55560ff4c7b0 0x55560ff560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.637+0000 7fac221db740 1 -- 172.21.15.120:0/1044734662 >> 172.21.15.120:0/1044734662 conn(0x55560ff15310 msgr2=0x55560ff559b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:21.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.637+0000 7fac221db740 1 -- 172.21.15.120:0/1044734662 shutdown_connections 2024-09-19T09:22:21.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.637+0000 7fac221db740 1 -- 172.21.15.120:0/1044734662 wait complete. 2024-09-19T09:22:21.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.638+0000 7fac221db740 1 -- start start 2024-09-19T09:22:21.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.638+0000 7fac221db740 1 Processor -- start 2024-09-19T09:22:21.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.638+0000 7fac221db740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55560ff4c7b0 0x55560ff5f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:21.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.638+0000 7fac221db740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55560ff4dd20 0x55560ff5fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:21.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.638+0000 7fac221db740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55560ff57550 0x55560ff5c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:21.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.638+0000 7fac221db740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fac181ae910 con 0x55560ff4dd20 2024-09-19T09:22:21.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.638+0000 7fac221db740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fac181ae240 con 0x55560ff4c7b0 2024-09-19T09:22:21.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.638+0000 7fac221db740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fac181ab460 con 0x55560ff57550 2024-09-19T09:22:21.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.638+0000 7fac221db740 10 client.0 _ll_register_callbacks cb 0x7fac10009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:22:21.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.638+0000 7fac1d39d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55560ff4dd20 0x55560ff5fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:21.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.638+0000 7fac1d39d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55560ff4dd20 0x55560ff5fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52564/0 (socket says 172.21.15.120:52564) 2024-09-19T09:22:21.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.638+0000 7fac1db9e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55560ff4c7b0 0x55560ff5f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:21.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.638+0000 7fac1d39d640 1 -- 172.21.15.120:0/2910263848 learned_addr learned my addr 172.21.15.120:0/2910263848 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:21.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.639+0000 7fac1e39f640 1 --2- 172.21.15.120:0/2910263848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55560ff57550 0x55560ff5c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:21.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.639+0000 7fac1d39d640 1 -- 172.21.15.120:0/2910263848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55560ff57550 msgr2=0x55560ff5c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:21.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.639+0000 7fac1d39d640 1 --2- 172.21.15.120:0/2910263848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55560ff57550 0x55560ff5c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.639+0000 7fac1d39d640 1 -- 172.21.15.120:0/2910263848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55560ff4c7b0 msgr2=0x55560ff5f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:21.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.639+0000 7fac1d39d640 1 --2- 172.21.15.120:0/2910263848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55560ff4c7b0 0x55560ff5f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.639+0000 7fac1d39d640 1 -- 172.21.15.120:0/2910263848 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55560ff5cb70 con 0x55560ff4dd20 2024-09-19T09:22:21.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.639+0000 7fac1d39d640 1 --2- 172.21.15.120:0/2910263848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55560ff4dd20 0x55560ff5fba0 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7fac08009730 tx=0x7fac080052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4854b02c3ecac2b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:21.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.639+0000 7fac017fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:22:21.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.639+0000 7fac017fa640 1 -- 172.21.15.120:0/2910263848 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac08003800 con 0x55560ff4dd20 2024-09-19T09:22:21.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.639+0000 7fac017fa640 1 -- 172.21.15.120:0/2910263848 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fac080039a0 con 0x55560ff4dd20 2024-09-19T09:22:21.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.639+0000 7fac221db740 1 -- 172.21.15.120:0/2910263848 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac08005590 con 0x55560ff4dd20 2024-09-19T09:22:21.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.639+0000 7fac017fa640 1 -- 172.21.15.120:0/2910263848 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fac0802b420 con 0x55560ff4dd20 2024-09-19T09:22:21.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.639+0000 7fac221db740 10 client.21912 fetch_fsmap learned FSMap version 39 2024-09-19T09:22:21.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.639+0000 7fac221db740 1 -- 172.21.15.120:0/2910263848 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55560ffd2610 con 0x55560ff4dd20 2024-09-19T09:22:21.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.639+0000 7fac017fa640 1 -- 172.21.15.120:0/2910263848 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac08005590 con 0x55560ff4dd20 2024-09-19T09:22:21.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.640+0000 7fac017fa640 1 -- 172.21.15.120:0/2910263848 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fac0803c050 con 0x55560ff4dd20 2024-09-19T09:22:21.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.640+0000 7fac221db740 10 client.21912 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:22:21.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.640+0000 7fac221db740 10 client.21912 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:22:21.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.640+0000 7fac221db740 20 client.21912 populate_metadata read hostname 'smithi120' 2024-09-19T09:22:21.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.640+0000 7fac221db740 1 --2- 172.21.15.120:0/2910263848 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55560ffd2b30 0x55560ffd4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:21.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.640+0000 7fac221db740 4 client.21912 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:22:21.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.640+0000 7fac221db740 1 -- 172.21.15.120:0/2910263848 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55560feb9d70 con 0x55560ffd2b30 2024-09-19T09:22:21.664 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:22:21.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.640+0000 7fac1db9e640 1 --2- 172.21.15.120:0/2910263848 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55560ffd2b30 0x55560ffd4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:21.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.640+0000 7fac1db9e640 1 --2- 172.21.15.120:0/2910263848 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55560ffd2b30 0x55560ffd4f20 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=70611ca22dbae858 server_cookie=74e66a6933107f27 in_seq=0 out_seq=0 2024-09-19T09:22:21.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.640+0000 7fac017fa640 10 client.21912 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:22:21.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.640+0000 7fac017fa640 1 -- 172.21.15.120:0/2910263848 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55560feb9d70 con 0x55560ffd2b30 2024-09-19T09:22:21.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.640+0000 7fac017fa640 10 client.21912 handle_command_reply: tid=0 2024-09-19T09:22:21.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.640+0000 7fac221db740 1 client.21912 shutdown 2024-09-19T09:22:21.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.640+0000 7fac221db740 1 -- 172.21.15.120:0/2910263848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55560ff4dd20 msgr2=0x55560ff5fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:21.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.640+0000 7fac221db740 1 --2- 172.21.15.120:0/2910263848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55560ff4dd20 0x55560ff5fba0 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7fac08009730 tx=0x7fac080052f0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.641+0000 7fac221db740 1 -- 172.21.15.120:0/2910263848 shutdown_connections 2024-09-19T09:22:21.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.641+0000 7fac221db740 1 --2- 172.21.15.120:0/2910263848 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55560ffd2b30 0x55560ffd4f20 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.641+0000 7fac221db740 1 --2- 172.21.15.120:0/2910263848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55560ff57550 0x55560ff5c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.641+0000 7fac221db740 1 --2- 172.21.15.120:0/2910263848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55560ff4dd20 0x55560ff5fba0 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.641+0000 7fac017fa640 0 client.21912 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:22:21.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.641+0000 7fac221db740 1 --2- 172.21.15.120:0/2910263848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55560ff4c7b0 0x55560ff5f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.641+0000 7fac221db740 1 -- 172.21.15.120:0/2910263848 >> 172.21.15.120:0/2910263848 conn(0x55560ff15310 msgr2=0x55560ff47fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:21.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.641+0000 7fac221db740 1 -- 172.21.15.120:0/2910263848 shutdown_connections 2024-09-19T09:22:21.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.641+0000 7fac221db740 1 -- 172.21.15.120:0/2910263848 wait complete. 2024-09-19T09:22:21.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.641+0000 7fac221db740 20 client.21912 trim_cache size 0 max 16384 2024-09-19T09:22:21.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.641+0000 7fac1eba0640 1 -- 172.21.15.120:0/1979201165 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fac040a3240 msgr2=0x7fac040a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:21.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.641+0000 7fac1eba0640 1 --2- 172.21.15.120:0/1979201165 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fac040a3240 0x7fac040a5700 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fac10004770 tx=0x7fac100046c0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.641+0000 7fac1eba0640 1 -- 172.21.15.120:0/1979201165 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac181001a0 msgr2=0x7fac18198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:21.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.641+0000 7fac1eba0640 1 --2- 172.21.15.120:0/1979201165 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac181001a0 0x7fac18198250 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7fac1400eb90 tx=0x7fac1400c6a0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.641+0000 7fac1eba0640 1 -- 172.21.15.120:0/1979201165 shutdown_connections 2024-09-19T09:22:21.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.641+0000 7fac1eba0640 1 --2- 172.21.15.120:0/1979201165 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fac040a3240 0x7fac040a5700 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.641+0000 7fac1eba0640 1 --2- 172.21.15.120:0/1979201165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac18106df0 0x7fac1819cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.641+0000 7fac1eba0640 1 --2- 172.21.15.120:0/1979201165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac18100b50 0x7fac18198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.641+0000 7fac1eba0640 1 --2- 172.21.15.120:0/1979201165 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac181001a0 0x7fac18198250 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.641+0000 7fac1eba0640 1 -- 172.21.15.120:0/1979201165 >> 172.21.15.120:0/1979201165 conn(0x7fac18077070 msgr2=0x7fac18105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:21.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.641+0000 7fac1eba0640 1 -- 172.21.15.120:0/1979201165 shutdown_connections 2024-09-19T09:22:21.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.641+0000 7fac1eba0640 1 -- 172.21.15.120:0/1979201165 wait complete. 2024-09-19T09:22:21.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.776+0000 7f5e6dffb640 1 -- 172.21.15.120:0/1892749393 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5560c3eb0d20 con 0x5560c3f52680 2024-09-19T09:22:21.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.776+0000 7f5e6dffb640 10 client.30553 handle_command_reply: tid=0 2024-09-19T09:22:21.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.776+0000 7f5e8ecf0740 1 client.30553 shutdown 2024-09-19T09:22:21.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.776+0000 7f5e8ecf0740 1 -- 172.21.15.120:0/1892749393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560c3f43760 msgr2=0x5560c3f64c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:21.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.776+0000 7f5e8ecf0740 1 --2- 172.21.15.120:0/1892749393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560c3f43760 0x5560c3f64c00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5e7809bfe0 tx=0x7f5e780cd770 comp rx=0 tx=0).stop 2024-09-19T09:22:21.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.776+0000 7f5e8ecf0740 1 -- 172.21.15.120:0/1892749393 shutdown_connections 2024-09-19T09:22:21.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.776+0000 7f5e8ecf0740 1 --2- 172.21.15.120:0/1892749393 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5560c3f52680 0x5560c3fcbe60 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.776+0000 7f5e8ecf0740 1 --2- 172.21.15.120:0/1892749393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5560c3f4e940 0x5560c3fc8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.776+0000 7f5e6dffb640 0 client.30553 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:21.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.776+0000 7f5e8ecf0740 1 --2- 172.21.15.120:0/1892749393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5560c3f44080 0x5560c3f65140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.776+0000 7f5e8ecf0740 1 --2- 172.21.15.120:0/1892749393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5560c3f43760 0x5560c3f64c00 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.776+0000 7f5e8ecf0740 1 -- 172.21.15.120:0/1892749393 >> 172.21.15.120:0/1892749393 conn(0x5560c3f3fcf0 msgr2=0x5560c3f5dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:21.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.776+0000 7f5e8ecf0740 1 -- 172.21.15.120:0/1892749393 shutdown_connections 2024-09-19T09:22:21.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.776+0000 7f5e8ecf0740 1 -- 172.21.15.120:0/1892749393 wait complete. 2024-09-19T09:22:21.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.776+0000 7f5e8ecf0740 20 client.30553 trim_cache size 0 max 16384 2024-09-19T09:22:21.779 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:22:21.780 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 39, 2024-09-19T09:22:21.780 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:22:21.780 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 66, 2024-09-19T09:22:21.780 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:22:21.780 INFO:teuthology.orchestra.run.smithi120.stdout: "bb3ccada": { 2024-09-19T09:22:21.781 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 66, 2024-09-19T09:22:21.781 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:22:21.781 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:22:21.781 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:22:21.781 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:22:21.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:22:21.782 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:22:21.782 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 154.0, 2024-09-19T09:22:21.782 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:22:21.782 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:22:21.783 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:22:21.783 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:22:21.783 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:22:21.783 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:22:21.783 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:22:21.783 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:22:21.783 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:22:21.783 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:22:21.784 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:22:21.784 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:22:21.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.777+0000 7f5e8c917640 1 -- 172.21.15.120:0/693761554 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5e700a3240 msgr2=0x7f5e700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:21.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.777+0000 7f5e8c917640 1 --2- 172.21.15.120:0/693761554 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5e700a3240 0x7f5e700a5700 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5e80004af0 tx=0x7f5e80005bf0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.777+0000 7f5e8c917640 1 -- 172.21.15.120:0/693761554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e88102d30 msgr2=0x7f5e88198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:21.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.777+0000 7f5e8c917640 1 --2- 172.21.15.120:0/693761554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e88102d30 0x7f5e88198520 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f5e7800e9f0 tx=0x7f5e7800eeb0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.777+0000 7f5e8c917640 1 -- 172.21.15.120:0/693761554 shutdown_connections 2024-09-19T09:22:21.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.777+0000 7f5e8c917640 1 --2- 172.21.15.120:0/693761554 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5e700a3240 0x7f5e700a5700 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.777+0000 7f5e8c917640 1 --2- 172.21.15.120:0/693761554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e881082a0 0x7f5e8819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.777+0000 7f5e8c917640 1 --2- 172.21.15.120:0/693761554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e881036e0 0x7f5e88198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.777+0000 7f5e8c917640 1 --2- 172.21.15.120:0/693761554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e88102d30 0x7f5e88198520 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:21.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.777+0000 7f5e8c917640 1 -- 172.21.15.120:0/693761554 >> 172.21.15.120:0/693761554 conn(0x7f5e880fc7d0 msgr2=0x7f5e88105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:21.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.777+0000 7f5e8c917640 1 -- 172.21.15.120:0/693761554 shutdown_connections 2024-09-19T09:22:21.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:21.777+0000 7f5e8c917640 1 -- 172.21.15.120:0/693761554 wait complete. 2024-09-19T09:22:22.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:22 smithi120 ceph-mon[31150]: pgmap v930: 97 pgs: 97 active+clean; 426 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 14 MiB/s wr, 775 op/s 2024-09-19T09:22:22.373 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 21.68 MB/sec execute 257 sec latency 1602.579 ms 2024-09-19T09:22:22.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:22 smithi196 ceph-mon[35150]: pgmap v930: 97 pgs: 97 active+clean; 426 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 14 MiB/s wr, 775 op/s 2024-09-19T09:22:22.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:22 smithi153 ceph-mon[35071]: pgmap v930: 97 pgs: 97 active+clean; 426 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 14 MiB/s wr, 775 op/s 2024-09-19T09:22:22.591 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:22:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.881+0000 7f7bf8796640 1 -- 172.21.15.120:0/1466492391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bf0102620 msgr2=0x7f7bf0104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.881+0000 7f7bf8796640 1 --2- 172.21.15.120:0/1466492391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bf0102620 0x7f7bf0104a10 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f7bec009ca0 tx=0x7f7bec02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:22:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.881+0000 7f7bf8796640 1 -- 172.21.15.120:0/1466492391 shutdown_connections 2024-09-19T09:22:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.881+0000 7f7bf8796640 1 --2- 172.21.15.120:0/1466492391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7bf0107880 0x7f7bf0109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.881+0000 7f7bf8796640 1 --2- 172.21.15.120:0/1466492391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bf0104f50 0x7f7bf0107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.881+0000 7f7bf8796640 1 --2- 172.21.15.120:0/1466492391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bf0102620 0x7f7bf0104a10 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.881+0000 7f7bf8796640 1 -- 172.21.15.120:0/1466492391 >> 172.21.15.120:0/1466492391 conn(0x7f7bf00fc1e0 msgr2=0x7f7bf00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.881+0000 7f7bf8796640 1 -- 172.21.15.120:0/1466492391 shutdown_connections 2024-09-19T09:22:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.881+0000 7f7bf8796640 1 -- 172.21.15.120:0/1466492391 wait complete. 2024-09-19T09:22:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.882+0000 7f7bf8796640 1 Processor -- start 2024-09-19T09:22:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.882+0000 7f7bf8796640 1 -- start start 2024-09-19T09:22:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.882+0000 7f7bf8796640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7bf0102620 0x7f7bf0197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.882+0000 7f7bf8796640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bf0104f50 0x7f7bf0198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.882+0000 7f7bf8796640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bf0107880 0x7f7bf019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.882+0000 7f7bf8796640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf010d810 con 0x7f7bf0107880 2024-09-19T09:22:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.882+0000 7f7bf8796640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf010d690 con 0x7f7bf0102620 2024-09-19T09:22:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.882+0000 7f7bf8796640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf010d990 con 0x7f7bf0104f50 2024-09-19T09:22:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.882+0000 7f7bf6d33640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bf0107880 0x7f7bf019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.882+0000 7f7bf6d33640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bf0107880 0x7f7bf019f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52572/0 (socket says 172.21.15.120:52572) 2024-09-19T09:22:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.882+0000 7f7bf6d33640 1 -- 172.21.15.120:0/615170540 learned_addr learned my addr 172.21.15.120:0/615170540 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.883+0000 7f7bf5d31640 1 --2- 172.21.15.120:0/615170540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bf0104f50 0x7f7bf0198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.883+0000 7f7bf6d33640 1 -- 172.21.15.120:0/615170540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bf0104f50 msgr2=0x7f7bf0198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.883+0000 7f7bf6d33640 1 --2- 172.21.15.120:0/615170540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bf0104f50 0x7f7bf0198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.883+0000 7f7bf6d33640 1 -- 172.21.15.120:0/615170540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7bf0102620 msgr2=0x7f7bf0197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.883+0000 7f7bf6d33640 1 --2- 172.21.15.120:0/615170540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7bf0102620 0x7f7bf0197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.883+0000 7f7bf6d33640 1 -- 172.21.15.120:0/615170540 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf019f9f0 con 0x7f7bf0107880 2024-09-19T09:22:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.883+0000 7f7bf6d33640 1 --2- 172.21.15.120:0/615170540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bf0107880 0x7f7bf019f380 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f7be800c970 tx=0x7f7be800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.883+0000 7f7be77fe640 1 -- 172.21.15.120:0/615170540 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8007bb0 con 0x7f7bf0107880 2024-09-19T09:22:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.883+0000 7f7be77fe640 1 -- 172.21.15.120:0/615170540 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7be8007d50 con 0x7f7bf0107880 2024-09-19T09:22:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.883+0000 7f7bf8796640 1 -- 172.21.15.120:0/615170540 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf019fce0 con 0x7f7bf0107880 2024-09-19T09:22:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.883+0000 7f7be77fe640 1 -- 172.21.15.120:0/615170540 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8018800 con 0x7f7bf0107880 2024-09-19T09:22:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.883+0000 7f7bf8796640 1 -- 172.21.15.120:0/615170540 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf01a01d0 con 0x7f7bf0107880 2024-09-19T09:22:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.885+0000 7f7be77fe640 1 -- 172.21.15.120:0/615170540 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f7be800f450 con 0x7f7bf0107880 2024-09-19T09:22:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.886+0000 7f7bfb608740 1 Processor -- start 2024-09-19T09:22:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.886+0000 7f7bfb608740 1 -- start start 2024-09-19T09:22:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.886+0000 7f7bfb608740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c0536b9a0 0x561c0536bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.886+0000 7f7bfb608740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c0536c2e0 0x561c05370da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.886+0000 7f7bfb608740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c05371370 0x561c05373830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.886+0000 7f7bfb608740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561c052cf2f0 con 0x561c0536c2e0 2024-09-19T09:22:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.886+0000 7f7bfb608740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561c05277270 con 0x561c0536b9a0 2024-09-19T09:22:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.886+0000 7f7bfb608740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561c052cb2b0 con 0x561c05371370 2024-09-19T09:22:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.886+0000 7f7bf6532640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c0536c2e0 0x561c05370da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.886+0000 7f7bf6532640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c0536c2e0 0x561c05370da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52574/0 (socket says 172.21.15.120:52574) 2024-09-19T09:22:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.886+0000 7f7bf6532640 1 -- 172.21.15.120:0/3000015736 learned_addr learned my addr 172.21.15.120:0/3000015736 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.886+0000 7f7bf5d31640 1 --2- 172.21.15.120:0/3000015736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c0536b9a0 0x561c0536bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.886+0000 7f7be77fe640 1 --2- 172.21.15.120:0/615170540 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7bdc0a3370 0x7f7bdc0a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.886+0000 7f7bf5d31640 1 --2- 172.21.15.120:0/3000015736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c05371370 0x561c05373830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.887+0000 7f7bf6532640 1 -- 172.21.15.120:0/3000015736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c05371370 msgr2=0x561c05373830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.887+0000 7f7bf6532640 1 --2- 172.21.15.120:0/3000015736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c05371370 0x561c05373830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.887+0000 7f7bf6d33640 1 --2- 172.21.15.120:0/615170540 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7bdc0a3370 0x7f7bdc0a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.887+0000 7f7bf6532640 1 -- 172.21.15.120:0/3000015736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c0536b9a0 msgr2=0x561c0536bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.887+0000 7f7bf6532640 1 --2- 172.21.15.120:0/3000015736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c0536b9a0 0x561c0536bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.887+0000 7f7bf6532640 1 -- 172.21.15.120:0/3000015736 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c05276340 con 0x561c0536c2e0 2024-09-19T09:22:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.887+0000 7f7be77fe640 1 -- 172.21.15.120:0/615170540 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f7be80cfa00 con 0x7f7bf0107880 2024-09-19T09:22:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.887+0000 7f7bf6532640 1 --2- 172.21.15.120:0/3000015736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c0536c2e0 0x561c05370da0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f7bec005fd0 tx=0x7f7bec009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b994607492424ab9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.887+0000 7f7be4ff9640 1 -- 172.21.15.120:0/3000015736 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec03f070 con 0x561c0536c2e0 2024-09-19T09:22:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.887+0000 7f7bf6d33640 1 --2- 172.21.15.120:0/615170540 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7bdc0a3370 0x7f7bdc0a5830 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7be8098480 tx=0x7f7be8091500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.887+0000 7f7be4ff9640 1 -- 172.21.15.120:0/3000015736 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bec036040 con 0x561c0536c2e0 2024-09-19T09:22:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.887+0000 7f7be4ff9640 1 -- 172.21.15.120:0/3000015736 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec004320 con 0x561c0536c2e0 2024-09-19T09:22:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.888+0000 7f7bfb608740 1 -- 172.21.15.120:0/3000015736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c0536c2e0 msgr2=0x561c05370da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.888+0000 7f7bfb608740 1 --2- 172.21.15.120:0/3000015736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c0536c2e0 0x561c05370da0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f7bec005fd0 tx=0x7f7bec009800 comp rx=0 tx=0).stop 2024-09-19T09:22:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.888+0000 7f7bfb608740 1 -- 172.21.15.120:0/3000015736 shutdown_connections 2024-09-19T09:22:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.888+0000 7f7bfb608740 1 --2- 172.21.15.120:0/3000015736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c05371370 0x561c05373830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.888+0000 7f7bfb608740 1 --2- 172.21.15.120:0/3000015736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c0536c2e0 0x561c05370da0 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.888+0000 7f7bfb608740 1 --2- 172.21.15.120:0/3000015736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c0536b9a0 0x561c0536bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.888+0000 7f7bfb608740 1 -- 172.21.15.120:0/3000015736 >> 172.21.15.120:0/3000015736 conn(0x561c05367010 msgr2=0x561c05367410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.888+0000 7f7bfb608740 1 -- 172.21.15.120:0/3000015736 shutdown_connections 2024-09-19T09:22:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.888+0000 7f7bfb608740 1 -- 172.21.15.120:0/3000015736 wait complete. 2024-09-19T09:22:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.888+0000 7f7bfb608740 1 -- start start 2024-09-19T09:22:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.888+0000 7f7bfb608740 1 Processor -- start 2024-09-19T09:22:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.888+0000 7f7bfb608740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c0536b9a0 0x561c05375b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.888+0000 7f7bfb608740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c0536c2e0 0x561c053760d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.888+0000 7f7bfb608740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c05371370 0x561c0537d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.888+0000 7f7bfb608740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561c053742e0 con 0x561c05371370 2024-09-19T09:22:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.888+0000 7f7bfb608740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561c05258290 con 0x561c0536b9a0 2024-09-19T09:22:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.888+0000 7f7bfb608740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561c05374460 con 0x561c0536c2e0 2024-09-19T09:22:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.888+0000 7f7bfb608740 10 client.0 _ll_register_callbacks cb 0x561c05289140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:22:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.889+0000 7f7bf5d31640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c05371370 0x561c0537d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.889+0000 7f7bf5d31640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c05371370 0x561c0537d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52588/0 (socket says 172.21.15.120:52588) 2024-09-19T09:22:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.889+0000 7f7bf5d31640 1 -- 172.21.15.120:0/2650846356 learned_addr learned my addr 172.21.15.120:0/2650846356 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.889+0000 7f7bf5d31640 1 --2- 172.21.15.120:0/2650846356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c0536b9a0 0x561c05375b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.889+0000 7f7bf6532640 1 --2- 172.21.15.120:0/2650846356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c0536c2e0 0x561c053760d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.889+0000 7f7bf5d31640 1 -- 172.21.15.120:0/2650846356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c0536c2e0 msgr2=0x561c053760d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.889+0000 7f7bf5d31640 1 --2- 172.21.15.120:0/2650846356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c0536c2e0 0x561c053760d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.889+0000 7f7bf5d31640 1 -- 172.21.15.120:0/2650846356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c0536b9a0 msgr2=0x561c05375b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.889+0000 7f7bf5d31640 1 --2- 172.21.15.120:0/2650846356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c0536b9a0 0x561c05375b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.889+0000 7f7bf5d31640 1 -- 172.21.15.120:0/2650846356 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c04fe2210 con 0x561c05371370 2024-09-19T09:22:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.889+0000 7f7bf5d31640 1 --2- 172.21.15.120:0/2650846356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c05371370 0x561c0537d110 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f7bf006a3a0 tx=0x7f7be0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0276376f3806c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.889+0000 7f7bd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:22:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.889+0000 7f7bd9ffb640 1 -- 172.21.15.120:0/2650846356 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be00132d0 con 0x561c05371370 2024-09-19T09:22:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.889+0000 7f7bd9ffb640 1 -- 172.21.15.120:0/2650846356 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f7be0004020 con 0x561c05371370 2024-09-19T09:22:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.889+0000 7f7bfb608740 1 -- 172.21.15.120:0/2650846356 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bec036040 con 0x561c05371370 2024-09-19T09:22:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.889+0000 7f7bd9ffb640 1 -- 172.21.15.120:0/2650846356 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7be0004220 con 0x561c05371370 2024-09-19T09:22:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.889+0000 7f7bfb608740 10 client.21924 fetch_fsmap learned FSMap version 39 2024-09-19T09:22:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.889+0000 7f7bfb608740 1 -- 172.21.15.120:0/2650846356 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561c050ae620 con 0x561c05371370 2024-09-19T09:22:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.889+0000 7f7bd9ffb640 1 -- 172.21.15.120:0/2650846356 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bec036040 con 0x561c05371370 2024-09-19T09:22:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.890+0000 7f7bd9ffb640 1 -- 172.21.15.120:0/2650846356 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f7be0004a70 con 0x561c05371370 2024-09-19T09:22:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.890+0000 7f7bfb608740 10 client.21924 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:22:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.890+0000 7f7bfb608740 10 client.21924 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:22:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.890+0000 7f7bfb608740 20 client.21924 populate_metadata read hostname 'smithi120' 2024-09-19T09:22:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.890+0000 7f7bfb608740 1 --2- 172.21.15.120:0/2650846356 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561c05381810 0x561c05383cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.890+0000 7f7bfb608740 4 client.21924 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:22:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.890+0000 7f7bfb608740 1 -- 172.21.15.120:0/2650846356 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c052dca20 con 0x561c05381810 2024-09-19T09:22:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.890+0000 7f7bf6532640 1 --2- 172.21.15.120:0/2650846356 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561c05381810 0x561c05383cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.890+0000 7f7bd9ffb640 10 client.21924 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.890+0000 7f7bf6532640 1 --2- 172.21.15.120:0/2650846356 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561c05381810 0x561c05383cd0 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef87605e8375fea6 server_cookie=b528d36c420069e1 in_seq=0 out_seq=0 2024-09-19T09:22:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.891+0000 7f7bd9ffb640 1 -- 172.21.15.120:0/2650846356 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c052dca20 con 0x561c05381810 2024-09-19T09:22:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.891+0000 7f7bd9ffb640 10 client.21924 handle_command_reply: tid=0 2024-09-19T09:22:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.891+0000 7f7bfb608740 1 client.21924 shutdown 2024-09-19T09:22:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.891+0000 7f7bfb608740 1 -- 172.21.15.120:0/2650846356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c05371370 msgr2=0x561c0537d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.891+0000 7f7bfb608740 1 --2- 172.21.15.120:0/2650846356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c05371370 0x561c0537d110 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f7bf006a3a0 tx=0x7f7be0004730 comp rx=0 tx=0).stop 2024-09-19T09:22:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.891+0000 7f7bfb608740 1 -- 172.21.15.120:0/2650846356 shutdown_connections 2024-09-19T09:22:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.891+0000 7f7bfb608740 1 --2- 172.21.15.120:0/2650846356 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561c05381810 0x561c05383cd0 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.891+0000 7f7bfb608740 1 --2- 172.21.15.120:0/2650846356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c05371370 0x561c0537d110 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.891+0000 7f7bfb608740 1 --2- 172.21.15.120:0/2650846356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c0536c2e0 0x561c053760d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.891+0000 7f7bd9ffb640 0 client.21924 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.891+0000 7f7bfb608740 1 --2- 172.21.15.120:0/2650846356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c0536b9a0 0x561c05375b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.891+0000 7f7bfb608740 1 -- 172.21.15.120:0/2650846356 >> 172.21.15.120:0/2650846356 conn(0x561c05367010 msgr2=0x561c05368820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.891+0000 7f7bfb608740 1 -- 172.21.15.120:0/2650846356 shutdown_connections 2024-09-19T09:22:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.891+0000 7f7bfb608740 1 -- 172.21.15.120:0/2650846356 wait complete. 2024-09-19T09:22:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.891+0000 7f7bfb608740 20 client.21924 trim_cache size 0 max 16384 2024-09-19T09:22:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.905+0000 7f7bfb608740 1 Processor -- start 2024-09-19T09:22:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.905+0000 7f7bfb608740 1 -- start start 2024-09-19T09:22:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.905+0000 7f7bfb608740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c0536b9a0 0x561c053661c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.905+0000 7f7bfb608740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c0536c2e0 0x561c0537d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.905+0000 7f7bfb608740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c05371370 0x561c05388590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.905+0000 7f7bfb608740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01a8fc0 con 0x561c0536b9a0 2024-09-19T09:22:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.905+0000 7f7bfb608740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01aab00 con 0x561c05371370 2024-09-19T09:22:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.905+0000 7f7bfb608740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01a9690 con 0x561c0536c2e0 2024-09-19T09:22:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.906+0000 7f7bf5d31640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c0536b9a0 0x561c053661c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.906+0000 7f7bf5d31640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c0536b9a0 0x561c053661c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52598/0 (socket says 172.21.15.120:52598) 2024-09-19T09:22:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.906+0000 7f7bf5d31640 1 -- 172.21.15.120:0/926627008 learned_addr learned my addr 172.21.15.120:0/926627008 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.906+0000 7f7bf6532640 1 --2- 172.21.15.120:0/926627008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c0536c2e0 0x561c0537d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.906+0000 7f7bf5d31640 1 -- 172.21.15.120:0/926627008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c0536c2e0 msgr2=0x561c0537d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.906+0000 7f7bf5d31640 1 --2- 172.21.15.120:0/926627008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c0536c2e0 0x561c0537d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.906+0000 7f7bf5d31640 1 -- 172.21.15.120:0/926627008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c05371370 msgr2=0x561c05388590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.906+0000 7f7bf5d31640 1 --2- 172.21.15.120:0/926627008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c05371370 0x561c05388590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.906+0000 7f7bf5d31640 1 -- 172.21.15.120:0/926627008 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c0536c7c0 con 0x561c0536b9a0 2024-09-19T09:22:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.906+0000 7f7bf5d31640 1 --2- 172.21.15.120:0/926627008 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c0536b9a0 0x561c053661c0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f7be0013d60 tx=0x7f7be0005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea351af6e05e017 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.906+0000 7f7bdbfff640 1 -- 172.21.15.120:0/926627008 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be00130d0 con 0x561c0536b9a0 2024-09-19T09:22:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.906+0000 7f7bdbfff640 1 -- 172.21.15.120:0/926627008 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7be0013270 con 0x561c0536b9a0 2024-09-19T09:22:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.906+0000 7f7bdbfff640 1 -- 172.21.15.120:0/926627008 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be0004fb0 con 0x561c0536b9a0 2024-09-19T09:22:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.907+0000 7f7bfb608740 1 -- 172.21.15.120:0/926627008 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c0536b9a0 msgr2=0x561c053661c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.907+0000 7f7bfb608740 1 --2- 172.21.15.120:0/926627008 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c0536b9a0 0x561c053661c0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f7be0013d60 tx=0x7f7be0005ce0 comp rx=0 tx=0).stop 2024-09-19T09:22:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.907+0000 7f7bfb608740 1 -- 172.21.15.120:0/926627008 shutdown_connections 2024-09-19T09:22:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.907+0000 7f7bfb608740 1 --2- 172.21.15.120:0/926627008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c05371370 0x561c05388590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.907+0000 7f7bfb608740 1 --2- 172.21.15.120:0/926627008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c0536c2e0 0x561c0537d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.907+0000 7f7bfb608740 1 --2- 172.21.15.120:0/926627008 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c0536b9a0 0x561c053661c0 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.907+0000 7f7bfb608740 1 -- 172.21.15.120:0/926627008 >> 172.21.15.120:0/926627008 conn(0x561c05367010 msgr2=0x561c05374ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.907+0000 7f7bfb608740 1 -- 172.21.15.120:0/926627008 shutdown_connections 2024-09-19T09:22:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.907+0000 7f7bfb608740 1 -- 172.21.15.120:0/926627008 wait complete. 2024-09-19T09:22:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.907+0000 7f7bfb608740 1 -- start start 2024-09-19T09:22:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.907+0000 7f7bfb608740 1 Processor -- start 2024-09-19T09:22:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.907+0000 7f7bfb608740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c0536b9a0 0x561c0537a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.907+0000 7f7bfb608740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c0536c2e0 0x561c0537adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.907+0000 7f7bfb608740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c05371370 0x561c0537be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.907+0000 7f7bfb608740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01aa430 con 0x561c0536b9a0 2024-09-19T09:22:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.907+0000 7f7bfb608740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01a9d60 con 0x561c05371370 2024-09-19T09:22:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.907+0000 7f7bfb608740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01a6f80 con 0x561c0536c2e0 2024-09-19T09:22:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.907+0000 7f7bfb608740 10 client.0 _ll_register_callbacks cb 0x7f7be00103d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:22:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.907+0000 7f7bf5d31640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c0536b9a0 0x561c0537a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.907+0000 7f7bf5d31640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c0536b9a0 0x561c0537a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52612/0 (socket says 172.21.15.120:52612) 2024-09-19T09:22:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.907+0000 7f7bf5d31640 1 -- 172.21.15.120:0/908920293 learned_addr learned my addr 172.21.15.120:0/908920293 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.907+0000 7f7bf6532640 1 --2- 172.21.15.120:0/908920293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c0536c2e0 0x561c0537adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.908+0000 7f7bf5d31640 1 -- 172.21.15.120:0/908920293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c0536c2e0 msgr2=0x561c0537adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.908+0000 7f7bf5d31640 1 --2- 172.21.15.120:0/908920293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c0536c2e0 0x561c0537adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.908+0000 7f7bf5d31640 1 -- 172.21.15.120:0/908920293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c05371370 msgr2=0x561c0537be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.908+0000 7f7bf5d31640 1 --2- 172.21.15.120:0/908920293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c05371370 0x561c0537be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.908+0000 7f7bf5d31640 1 -- 172.21.15.120:0/908920293 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c0537c410 con 0x561c0536b9a0 2024-09-19T09:22:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.908+0000 7f7bf5d31640 1 --2- 172.21.15.120:0/908920293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c0536b9a0 0x561c0537a8b0 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f7be0004ee0 tx=0x7f7be0013480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c263010ada503a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.908+0000 7f7bd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:22:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.908+0000 7f7bd9ffb640 1 -- 172.21.15.120:0/908920293 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be00130d0 con 0x561c0536b9a0 2024-09-19T09:22:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.908+0000 7f7bd9ffb640 1 -- 172.21.15.120:0/908920293 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f7be0004fb0 con 0x561c0536b9a0 2024-09-19T09:22:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.908+0000 7f7bfb608740 1 -- 172.21.15.120:0/908920293 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7be0013270 con 0x561c0536b9a0 2024-09-19T09:22:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.908+0000 7f7bd9ffb640 1 -- 172.21.15.120:0/908920293 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7be00046e0 con 0x561c0536b9a0 2024-09-19T09:22:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.908+0000 7f7bfb608740 10 client.21930 fetch_fsmap learned FSMap version 39 2024-09-19T09:22:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.908+0000 7f7bfb608740 1 -- 172.21.15.120:0/908920293 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561c053ec700 con 0x561c0536b9a0 2024-09-19T09:22:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.908+0000 7f7bd9ffb640 1 -- 172.21.15.120:0/908920293 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7be0013270 con 0x561c0536b9a0 2024-09-19T09:22:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.908+0000 7f7bd9ffb640 1 -- 172.21.15.120:0/908920293 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f7be0032020 con 0x561c0536b9a0 2024-09-19T09:22:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.908+0000 7f7bfb608740 10 client.21930 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:22:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.908+0000 7f7bfb608740 10 client.21930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:22:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.908+0000 7f7bfb608740 20 client.21930 populate_metadata read hostname 'smithi120' 2024-09-19T09:22:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.908+0000 7f7bfb608740 1 --2- 172.21.15.120:0/908920293 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561c053ed040 0x561c053ef460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.908+0000 7f7bfb608740 4 client.21930 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:22:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.908+0000 7f7bfb608740 1 -- 172.21.15.120:0/908920293 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561c052d6d70 con 0x561c053ed040 2024-09-19T09:22:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.909+0000 7f7bf6532640 1 --2- 172.21.15.120:0/908920293 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561c053ed040 0x561c053ef460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.909+0000 7f7bd9ffb640 10 client.21930 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:22.909+0000 7f7bf6532640 1 --2- 172.21.15.120:0/908920293 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561c053ed040 0x561c053ef460 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84c29e5df74e34e5 server_cookie=dbf8e3eb7296d6b3 in_seq=0 out_seq=0 2024-09-19T09:22:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:23.165+0000 7f7bd9ffb640 1 -- 172.21.15.120:0/908920293 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561c052d6d70 con 0x561c053ed040 2024-09-19T09:22:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:23.165+0000 7f7bd9ffb640 10 client.21930 handle_command_reply: tid=0 2024-09-19T09:22:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:23.165+0000 7f7bfb608740 1 client.21930 shutdown 2024-09-19T09:22:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:23.165+0000 7f7bfb608740 1 -- 172.21.15.120:0/908920293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c0536b9a0 msgr2=0x561c0537a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:23.165+0000 7f7bfb608740 1 --2- 172.21.15.120:0/908920293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c0536b9a0 0x561c0537a8b0 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f7be0004ee0 tx=0x7f7be0013480 comp rx=0 tx=0).stop 2024-09-19T09:22:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:23.165+0000 7f7bfb608740 1 -- 172.21.15.120:0/908920293 shutdown_connections 2024-09-19T09:22:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:23.165+0000 7f7bfb608740 1 --2- 172.21.15.120:0/908920293 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561c053ed040 0x561c053ef460 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:23.165+0000 7f7bfb608740 1 --2- 172.21.15.120:0/908920293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561c05371370 0x561c0537be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:23.165+0000 7f7bd9ffb640 0 client.21930 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:23.165+0000 7f7bfb608740 1 --2- 172.21.15.120:0/908920293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c0536c2e0 0x561c0537adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:23.165+0000 7f7bfb608740 1 --2- 172.21.15.120:0/908920293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561c0536b9a0 0x561c0537a8b0 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:23.165+0000 7f7bfb608740 1 -- 172.21.15.120:0/908920293 >> 172.21.15.120:0/908920293 conn(0x561c05367010 msgr2=0x561c053661c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:23.165+0000 7f7bfb608740 1 -- 172.21.15.120:0/908920293 shutdown_connections 2024-09-19T09:22:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:23.165+0000 7f7bfb608740 1 -- 172.21.15.120:0/908920293 wait complete. 2024-09-19T09:22:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:23.165+0000 7f7bfb608740 20 client.21930 trim_cache size 0 max 16384 2024-09-19T09:22:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:22:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:22:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:22:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:22:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:23.166+0000 7f7bf8796640 1 -- 172.21.15.120:0/615170540 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7bdc0a3370 msgr2=0x7f7bdc0a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:23.166+0000 7f7bf8796640 1 --2- 172.21.15.120:0/615170540 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7bdc0a3370 0x7f7bdc0a5830 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7be8098480 tx=0x7f7be8091500 comp rx=0 tx=0).stop 2024-09-19T09:22:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:23.166+0000 7f7bf8796640 1 -- 172.21.15.120:0/615170540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bf0107880 msgr2=0x7f7bf019f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:23.166+0000 7f7bf8796640 1 --2- 172.21.15.120:0/615170540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bf0107880 0x7f7bf019f380 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f7be800c970 tx=0x7f7be800ce30 comp rx=0 tx=0).stop 2024-09-19T09:22:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:23.166+0000 7f7bf8796640 1 -- 172.21.15.120:0/615170540 shutdown_connections 2024-09-19T09:22:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:23.166+0000 7f7bf8796640 1 --2- 172.21.15.120:0/615170540 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7bdc0a3370 0x7f7bdc0a5830 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:23.166+0000 7f7bf8796640 1 --2- 172.21.15.120:0/615170540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bf0107880 0x7f7bf019f380 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:23.166+0000 7f7bf8796640 1 --2- 172.21.15.120:0/615170540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bf0104f50 0x7f7bf0198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:23.166+0000 7f7bf8796640 1 --2- 172.21.15.120:0/615170540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7bf0102620 0x7f7bf0197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:23.166+0000 7f7bf8796640 1 -- 172.21.15.120:0/615170540 >> 172.21.15.120:0/615170540 conn(0x7f7bf00fc1e0 msgr2=0x7f7bf0102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:23.166+0000 7f7bf8796640 1 -- 172.21.15.120:0/615170540 shutdown_connections 2024-09-19T09:22:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:23.166+0000 7f7bf8796640 1 -- 172.21.15.120:0/615170540 wait complete. 2024-09-19T09:22:23.373 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 21.60 MB/sec execute 258 sec latency 2602.606 ms 2024-09-19T09:22:23.497 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'bb3ccada', quiesce took 7.5 seconds. Will release after: 26.5 2024-09-19T09:22:23.571 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:22:24.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.169+0000 7fdbf03c7640 1 -- 172.21.15.120:0/2462918106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbe8100a80 msgr2=0x7fdbe8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:24.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.169+0000 7fdbf03c7640 1 --2- 172.21.15.120:0/2462918106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbe8100a80 0x7fdbe8100ee0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc009930 tx=0x7fdbdc02a420 comp rx=0 tx=0).stop 2024-09-19T09:22:24.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.169+0000 7fdbf03c7640 1 -- 172.21.15.120:0/2462918106 shutdown_connections 2024-09-19T09:22:24.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.169+0000 7fdbf03c7640 1 --2- 172.21.15.120:0/2462918106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbe8101420 0x7fdbe810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.169+0000 7fdbf03c7640 1 --2- 172.21.15.120:0/2462918106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbe8100a80 0x7fdbe8100ee0 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.169+0000 7fdbf03c7640 1 --2- 172.21.15.120:0/2462918106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbe8106a80 0x7fdbe8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.169+0000 7fdbf03c7640 1 -- 172.21.15.120:0/2462918106 >> 172.21.15.120:0/2462918106 conn(0x7fdbe80fc7a0 msgr2=0x7fdbe80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:24.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.169+0000 7fdbf03c7640 1 -- 172.21.15.120:0/2462918106 shutdown_connections 2024-09-19T09:22:24.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.169+0000 7fdbf03c7640 1 -- 172.21.15.120:0/2462918106 wait complete. 2024-09-19T09:22:24.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.170+0000 7fdbf03c7640 1 Processor -- start 2024-09-19T09:22:24.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.170+0000 7fdbf03c7640 1 -- start start 2024-09-19T09:22:24.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.170+0000 7fdbf03c7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbe8100a80 0x7fdbe81983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:24.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.170+0000 7fdbf03c7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbe8101420 0x7fdbe8198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:24.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.170+0000 7fdbf03c7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbe8106a80 0x7fdbe819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:24.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.170+0000 7fdbf03c7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe8110e80 con 0x7fdbe8106a80 2024-09-19T09:22:24.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.170+0000 7fdbf03c7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe8110d00 con 0x7fdbe8101420 2024-09-19T09:22:24.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.170+0000 7fdbf03c7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe8111000 con 0x7fdbe8100a80 2024-09-19T09:22:24.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.170+0000 7fdbee964640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbe8106a80 0x7fdbe819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:24.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.170+0000 7fdbee964640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbe8106a80 0x7fdbe819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52634/0 (socket says 172.21.15.120:52634) 2024-09-19T09:22:24.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.170+0000 7fdbee964640 1 -- 172.21.15.120:0/3625214681 learned_addr learned my addr 172.21.15.120:0/3625214681 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:24.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.170+0000 7fdbee163640 1 --2- 172.21.15.120:0/3625214681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbe8100a80 0x7fdbe81983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:24.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.170+0000 7fdbee964640 1 -- 172.21.15.120:0/3625214681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbe8100a80 msgr2=0x7fdbe81983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:24.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.170+0000 7fdbee964640 1 --2- 172.21.15.120:0/3625214681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbe8100a80 0x7fdbe81983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.170+0000 7fdbee964640 1 -- 172.21.15.120:0/3625214681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbe8101420 msgr2=0x7fdbe8198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:24.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.170+0000 7fdbed962640 1 --2- 172.21.15.120:0/3625214681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbe8101420 0x7fdbe8198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:24.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.170+0000 7fdbee964640 1 --2- 172.21.15.120:0/3625214681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbe8101420 0x7fdbe8198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.170+0000 7fdbee964640 1 -- 172.21.15.120:0/3625214681 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe819d380 con 0x7fdbe8106a80 2024-09-19T09:22:24.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.170+0000 7fdbee163640 1 --2- 172.21.15.120:0/3625214681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbe8100a80 0x7fdbe81983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:22:24.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.171+0000 7fdbee964640 1 --2- 172.21.15.120:0/3625214681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbe8106a80 0x7fdbe819cca0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000c970 tx=0x7fdbe000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:24.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.171+0000 7fdbe77fe640 1 -- 172.21.15.120:0/3625214681 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0007bb0 con 0x7fdbe8106a80 2024-09-19T09:22:24.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.171+0000 7fdbf03c7640 1 -- 172.21.15.120:0/3625214681 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbe819d670 con 0x7fdbe8106a80 2024-09-19T09:22:24.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.171+0000 7fdbe77fe640 1 -- 172.21.15.120:0/3625214681 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbe0007d50 con 0x7fdbe8106a80 2024-09-19T09:22:24.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.171+0000 7fdbe77fe640 1 -- 172.21.15.120:0/3625214681 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0018800 con 0x7fdbe8106a80 2024-09-19T09:22:24.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.171+0000 7fdbf03c7640 1 -- 172.21.15.120:0/3625214681 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbe81a4f50 con 0x7fdbe8106a80 2024-09-19T09:22:24.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.173+0000 7fdbe77fe640 1 -- 172.21.15.120:0/3625214681 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fdbe000fa50 con 0x7fdbe8106a80 2024-09-19T09:22:24.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.173+0000 7fdbf27a0740 1 Processor -- start 2024-09-19T09:22:24.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.174+0000 7fdbf27a0740 1 -- start start 2024-09-19T09:22:24.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.174+0000 7fdbf27a0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555eabefab40 0x555eabef9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:24.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.174+0000 7fdbf27a0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555eabef95d0 0x555eabef9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:24.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.174+0000 7fdbf27a0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555eabf04370 0x555eabf06760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:24.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.174+0000 7fdbf27a0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555eabe5f2f0 con 0x555eabefab40 2024-09-19T09:22:24.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.174+0000 7fdbf27a0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555eabe06cf0 con 0x555eabf04370 2024-09-19T09:22:24.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.174+0000 7fdbf27a0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555eabe5b2b0 con 0x555eabef95d0 2024-09-19T09:22:24.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.174+0000 7fdbe77fe640 1 --2- 172.21.15.120:0/3625214681 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdbd00a3240 0x7fdbd00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:24.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.174+0000 7fdbed962640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555eabefab40 0x555eabef9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:24.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.174+0000 7fdbed962640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555eabefab40 0x555eabef9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52648/0 (socket says 172.21.15.120:52648) 2024-09-19T09:22:24.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.174+0000 7fdbed962640 1 -- 172.21.15.120:0/3892825546 learned_addr learned my addr 172.21.15.120:0/3892825546 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:24.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.174+0000 7fdbee964640 1 --2- 172.21.15.120:0/3625214681 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdbd00a3240 0x7fdbd00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:24.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.174+0000 7fdbee163640 1 --2- 172.21.15.120:0/3892825546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555eabef95d0 0x555eabef9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:24.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.174+0000 7fdbed962640 1 --2- 172.21.15.120:0/3892825546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555eabf04370 0x555eabf06760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:24.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.174+0000 7fdbe77fe640 1 -- 172.21.15.120:0/3625214681 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fdbe00cf9f0 con 0x7fdbe8106a80 2024-09-19T09:22:24.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.174+0000 7fdbed962640 1 -- 172.21.15.120:0/3892825546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555eabef95d0 msgr2=0x555eabef9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:24.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.174+0000 7fdbed962640 1 --2- 172.21.15.120:0/3892825546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555eabef95d0 0x555eabef9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.174+0000 7fdbed962640 1 -- 172.21.15.120:0/3892825546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555eabf04370 msgr2=0x555eabf06760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:24.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.174+0000 7fdbed962640 1 --2- 172.21.15.120:0/3892825546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555eabf04370 0x555eabf06760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.174+0000 7fdbed962640 1 -- 172.21.15.120:0/3892825546 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eabdeb3a0 con 0x555eabefab40 2024-09-19T09:22:24.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.175+0000 7fdbed962640 1 --2- 172.21.15.120:0/3892825546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555eabefab40 0x555eabef9090 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc031550 tx=0x7fdbdc005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5316b488a32d9b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:24.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.175+0000 7fdbee964640 1 --2- 172.21.15.120:0/3625214681 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdbd00a3240 0x7fdbd00a5700 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fdbe00a4750 tx=0x7fdbe006dde0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:24.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.175+0000 7fdbe4dea640 1 -- 172.21.15.120:0/3892825546 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc035070 con 0x555eabefab40 2024-09-19T09:22:24.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.175+0000 7fdbe4dea640 1 -- 172.21.15.120:0/3892825546 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbdc005040 con 0x555eabefab40 2024-09-19T09:22:24.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.175+0000 7fdbe4dea640 1 -- 172.21.15.120:0/3892825546 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc004920 con 0x555eabefab40 2024-09-19T09:22:24.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.175+0000 7fdbf27a0740 1 -- 172.21.15.120:0/3892825546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555eabefab40 msgr2=0x555eabef9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:24.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.176+0000 7fdbf27a0740 1 --2- 172.21.15.120:0/3892825546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555eabefab40 0x555eabef9090 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc031550 tx=0x7fdbdc005a20 comp rx=0 tx=0).stop 2024-09-19T09:22:24.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.176+0000 7fdbf27a0740 1 -- 172.21.15.120:0/3892825546 shutdown_connections 2024-09-19T09:22:24.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.176+0000 7fdbf27a0740 1 --2- 172.21.15.120:0/3892825546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555eabf04370 0x555eabf06760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.176+0000 7fdbf27a0740 1 --2- 172.21.15.120:0/3892825546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555eabef95d0 0x555eabef9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.176+0000 7fdbf27a0740 1 --2- 172.21.15.120:0/3892825546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555eabefab40 0x555eabef9090 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.176+0000 7fdbf27a0740 1 -- 172.21.15.120:0/3892825546 >> 172.21.15.120:0/3892825546 conn(0x555eabec2310 msgr2=0x555eabef5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:24.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.176+0000 7fdbf27a0740 1 -- 172.21.15.120:0/3892825546 shutdown_connections 2024-09-19T09:22:24.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.176+0000 7fdbf27a0740 1 -- 172.21.15.120:0/3892825546 wait complete. 2024-09-19T09:22:24.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.176+0000 7fdbf27a0740 1 -- start start 2024-09-19T09:22:24.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.176+0000 7fdbf27a0740 1 Processor -- start 2024-09-19T09:22:24.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.176+0000 7fdbf27a0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555eabef95d0 0x555eabf03d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:24.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.176+0000 7fdbf27a0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555eabefab40 0x555eabf01020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:24.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.176+0000 7fdbf27a0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555eabf04370 0x555eabf01610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:24.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.176+0000 7fdbf27a0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555eabf07010 con 0x555eabf04370 2024-09-19T09:22:24.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.176+0000 7fdbf27a0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555eabe1a060 con 0x555eabef95d0 2024-09-19T09:22:24.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.176+0000 7fdbf27a0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555eabf07190 con 0x555eabefab40 2024-09-19T09:22:24.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.176+0000 7fdbf27a0740 10 client.0 _ll_register_callbacks cb 0x555eabb6d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:22:24.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.176+0000 7fdbed962640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555eabf04370 0x555eabf01610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:24.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.176+0000 7fdbed962640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555eabf04370 0x555eabf01610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52658/0 (socket says 172.21.15.120:52658) 2024-09-19T09:22:24.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.177+0000 7fdbed962640 1 -- 172.21.15.120:0/1958429610 learned_addr learned my addr 172.21.15.120:0/1958429610 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:24.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.177+0000 7fdbee163640 1 --2- 172.21.15.120:0/1958429610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555eabefab40 0x555eabf01020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:24.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.177+0000 7fdbed962640 1 --2- 172.21.15.120:0/1958429610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555eabef95d0 0x555eabf03d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:24.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.177+0000 7fdbed962640 1 -- 172.21.15.120:0/1958429610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555eabefab40 msgr2=0x555eabf01020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:24.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.177+0000 7fdbed962640 1 --2- 172.21.15.120:0/1958429610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555eabefab40 0x555eabf01020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.177+0000 7fdbed962640 1 -- 172.21.15.120:0/1958429610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555eabef95d0 msgr2=0x555eabf03d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:24.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.177+0000 7fdbed962640 1 --2- 172.21.15.120:0/1958429610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555eabef95d0 0x555eabf03d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.177+0000 7fdbed962640 1 -- 172.21.15.120:0/1958429610 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eabbd03c0 con 0x555eabf04370 2024-09-19T09:22:24.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.177+0000 7fdbed962640 1 --2- 172.21.15.120:0/1958429610 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555eabf04370 0x555eabf01610 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc02a460 tx=0x7fdbdc02aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ff56717d0ecb43 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:24.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.177+0000 7fdbcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:22:24.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.177+0000 7fdbcdffb640 1 -- 172.21.15.120:0/1958429610 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc035030 con 0x555eabf04370 2024-09-19T09:22:24.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.177+0000 7fdbcdffb640 1 -- 172.21.15.120:0/1958429610 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fdbdc0046c0 con 0x555eabf04370 2024-09-19T09:22:24.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.177+0000 7fdbf27a0740 1 -- 172.21.15.120:0/1958429610 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbdc005040 con 0x555eabf04370 2024-09-19T09:22:24.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.177+0000 7fdbcdffb640 1 -- 172.21.15.120:0/1958429610 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdbdc0048c0 con 0x555eabf04370 2024-09-19T09:22:24.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.177+0000 7fdbf27a0740 10 client.21942 fetch_fsmap learned FSMap version 39 2024-09-19T09:22:24.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.177+0000 7fdbf27a0740 1 -- 172.21.15.120:0/1958429610 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555eabf12ab0 con 0x555eabf04370 2024-09-19T09:22:24.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.178+0000 7fdbcdffb640 1 -- 172.21.15.120:0/1958429610 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbdc005040 con 0x555eabf04370 2024-09-19T09:22:24.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.178+0000 7fdbcdffb640 1 -- 172.21.15.120:0/1958429610 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fdbdc039410 con 0x555eabf04370 2024-09-19T09:22:24.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.178+0000 7fdbf27a0740 10 client.21942 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:22:24.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.178+0000 7fdbf27a0740 10 client.21942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:22:24.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.178+0000 7fdbf27a0740 20 client.21942 populate_metadata read hostname 'smithi120' 2024-09-19T09:22:24.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.178+0000 7fdbf27a0740 1 --2- 172.21.15.120:0/1958429610 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555eabf13e00 0x555eabf161f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:24.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.178+0000 7fdbf27a0740 4 client.21942 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:22:24.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.178+0000 7fdbf27a0740 1 -- 172.21.15.120:0/1958429610 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555eabe6c860 con 0x555eabf13e00 2024-09-19T09:22:24.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.178+0000 7fdbee163640 1 --2- 172.21.15.120:0/1958429610 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555eabf13e00 0x555eabf161f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:24.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.179+0000 7fdbcdffb640 10 client.21942 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:24.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.179+0000 7fdbee163640 1 --2- 172.21.15.120:0/1958429610 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555eabf13e00 0x555eabf161f0 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa1f7fe66b93cc1 server_cookie=f6409eabbfe3b073 in_seq=0 out_seq=0 2024-09-19T09:22:24.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.179+0000 7fdbcdffb640 1 -- 172.21.15.120:0/1958429610 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555eabe6c860 con 0x555eabf13e00 2024-09-19T09:22:24.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.179+0000 7fdbcdffb640 10 client.21942 handle_command_reply: tid=0 2024-09-19T09:22:24.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.179+0000 7fdbf27a0740 1 client.21942 shutdown 2024-09-19T09:22:24.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.179+0000 7fdbf27a0740 1 -- 172.21.15.120:0/1958429610 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555eabf04370 msgr2=0x555eabf01610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:24.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.179+0000 7fdbf27a0740 1 --2- 172.21.15.120:0/1958429610 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555eabf04370 0x555eabf01610 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc02a460 tx=0x7fdbdc02aaa0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.179+0000 7fdbf27a0740 1 -- 172.21.15.120:0/1958429610 shutdown_connections 2024-09-19T09:22:24.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.179+0000 7fdbf27a0740 1 --2- 172.21.15.120:0/1958429610 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555eabf13e00 0x555eabf161f0 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.179+0000 7fdbf27a0740 1 --2- 172.21.15.120:0/1958429610 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555eabf04370 0x555eabf01610 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.179+0000 7fdbcdffb640 0 client.21942 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:24.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.179+0000 7fdbf27a0740 1 --2- 172.21.15.120:0/1958429610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555eabefab40 0x555eabf01020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.179+0000 7fdbf27a0740 1 --2- 172.21.15.120:0/1958429610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555eabef95d0 0x555eabf03d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.179+0000 7fdbf27a0740 1 -- 172.21.15.120:0/1958429610 >> 172.21.15.120:0/1958429610 conn(0x555eabec2310 msgr2=0x555eabefb990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:24.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.179+0000 7fdbf27a0740 1 -- 172.21.15.120:0/1958429610 shutdown_connections 2024-09-19T09:22:24.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.180+0000 7fdbf27a0740 1 -- 172.21.15.120:0/1958429610 wait complete. 2024-09-19T09:22:24.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.180+0000 7fdbf27a0740 20 client.21942 trim_cache size 0 max 16384 2024-09-19T09:22:24.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.192+0000 7fdbf27a0740 1 Processor -- start 2024-09-19T09:22:24.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.192+0000 7fdbf27a0740 1 -- start start 2024-09-19T09:22:24.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.193+0000 7fdbf27a0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555eabef95d0 0x555eabf19210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:24.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.193+0000 7fdbf27a0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555eabefab40 0x555eabf19750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:24.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.193+0000 7fdbf27a0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555eabf04370 0x555eabefe280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:24.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.193+0000 7fdbf27a0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe81ad490 con 0x555eabefab40 2024-09-19T09:22:24.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.193+0000 7fdbf27a0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe81aefd0 con 0x555eabf04370 2024-09-19T09:22:24.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.193+0000 7fdbf27a0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe81adb60 con 0x555eabef95d0 2024-09-19T09:22:24.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.193+0000 7fdbee163640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555eabefab40 0x555eabf19750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:24.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.193+0000 7fdbee163640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555eabefab40 0x555eabf19750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52664/0 (socket says 172.21.15.120:52664) 2024-09-19T09:22:24.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.193+0000 7fdbee163640 1 -- 172.21.15.120:0/2563080362 learned_addr learned my addr 172.21.15.120:0/2563080362 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:24.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.193+0000 7fdbed962640 1 --2- 172.21.15.120:0/2563080362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555eabf04370 0x555eabefe280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:24.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.193+0000 7fdbee163640 1 -- 172.21.15.120:0/2563080362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555eabef95d0 msgr2=0x555eabf19210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:24.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.193+0000 7fdbee163640 1 --2- 172.21.15.120:0/2563080362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555eabef95d0 0x555eabf19210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.193+0000 7fdbee163640 1 -- 172.21.15.120:0/2563080362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555eabf04370 msgr2=0x555eabefe280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:24.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.193+0000 7fdbee163640 1 --2- 172.21.15.120:0/2563080362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555eabf04370 0x555eabefe280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.193+0000 7fdbee163640 1 -- 172.21.15.120:0/2563080362 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eabefc120 con 0x555eabefab40 2024-09-19T09:22:24.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.193+0000 7fdbee163640 1 --2- 172.21.15.120:0/2563080362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555eabefab40 0x555eabf19750 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x555eabef6ac0 tx=0x7fdbd80118f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8023d19b5e9f36c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:24.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.193+0000 7fdbcffff640 1 -- 172.21.15.120:0/2563080362 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8021070 con 0x555eabefab40 2024-09-19T09:22:24.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.193+0000 7fdbcffff640 1 -- 172.21.15.120:0/2563080362 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbd801dbe0 con 0x555eabefab40 2024-09-19T09:22:24.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.193+0000 7fdbcffff640 1 -- 172.21.15.120:0/2563080362 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8026e40 con 0x555eabefab40 2024-09-19T09:22:24.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.194+0000 7fdbf27a0740 1 -- 172.21.15.120:0/2563080362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555eabefab40 msgr2=0x555eabf19750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:24.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.194+0000 7fdbf27a0740 1 --2- 172.21.15.120:0/2563080362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555eabefab40 0x555eabf19750 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x555eabef6ac0 tx=0x7fdbd80118f0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.194+0000 7fdbf27a0740 1 -- 172.21.15.120:0/2563080362 shutdown_connections 2024-09-19T09:22:24.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.194+0000 7fdbf27a0740 1 --2- 172.21.15.120:0/2563080362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555eabf04370 0x555eabefe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.194+0000 7fdbf27a0740 1 --2- 172.21.15.120:0/2563080362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555eabefab40 0x555eabf19750 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.194+0000 7fdbf27a0740 1 --2- 172.21.15.120:0/2563080362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555eabef95d0 0x555eabf19210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.194+0000 7fdbf27a0740 1 -- 172.21.15.120:0/2563080362 >> 172.21.15.120:0/2563080362 conn(0x555eabec2310 msgr2=0x555eabeff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:24.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.194+0000 7fdbf27a0740 1 -- 172.21.15.120:0/2563080362 shutdown_connections 2024-09-19T09:22:24.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.194+0000 7fdbf27a0740 1 -- 172.21.15.120:0/2563080362 wait complete. 2024-09-19T09:22:24.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.194+0000 7fdbf27a0740 1 -- start start 2024-09-19T09:22:24.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.194+0000 7fdbf27a0740 1 Processor -- start 2024-09-19T09:22:24.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.195+0000 7fdbf27a0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555eabef95d0 0x555eabf19770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:24.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.195+0000 7fdbf27a0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555eabefab40 0x555eabf16ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:24.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.195+0000 7fdbf27a0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555eabf04370 0x555eabf17000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:24.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.195+0000 7fdbf27a0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe81ae900 con 0x555eabf04370 2024-09-19T09:22:24.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.195+0000 7fdbf27a0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe81ae230 con 0x555eabef95d0 2024-09-19T09:22:24.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.195+0000 7fdbf27a0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe81ab450 con 0x555eabefab40 2024-09-19T09:22:24.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.195+0000 7fdbf27a0740 10 client.0 _ll_register_callbacks cb 0x7fdbd8008510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:22:24.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.195+0000 7fdbed962640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555eabf04370 0x555eabf17000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:24.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.195+0000 7fdbed962640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555eabf04370 0x555eabf17000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52672/0 (socket says 172.21.15.120:52672) 2024-09-19T09:22:24.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.195+0000 7fdbed962640 1 -- 172.21.15.120:0/2640463763 learned_addr learned my addr 172.21.15.120:0/2640463763 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:24.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.195+0000 7fdbed962640 1 --2- 172.21.15.120:0/2640463763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555eabef95d0 0x555eabf19770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:24.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.195+0000 7fdbee163640 1 --2- 172.21.15.120:0/2640463763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555eabefab40 0x555eabf16ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:24.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.195+0000 7fdbed962640 1 -- 172.21.15.120:0/2640463763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555eabefab40 msgr2=0x555eabf16ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:24.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.195+0000 7fdbed962640 1 --2- 172.21.15.120:0/2640463763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555eabefab40 0x555eabf16ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.195+0000 7fdbed962640 1 -- 172.21.15.120:0/2640463763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555eabef95d0 msgr2=0x555eabf19770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:24.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.195+0000 7fdbed962640 1 --2- 172.21.15.120:0/2640463763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555eabef95d0 0x555eabf19770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.195+0000 7fdbed962640 1 -- 172.21.15.120:0/2640463763 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eabf175f0 con 0x555eabf04370 2024-09-19T09:22:24.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.195+0000 7fdbed962640 1 --2- 172.21.15.120:0/2640463763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555eabf04370 0x555eabf17000 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc031bf0 tx=0x7fdbdc02aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2d66694a8a142d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:24.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.195+0000 7fdbcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:22:24.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.195+0000 7fdbcdffb640 1 -- 172.21.15.120:0/2640463763 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc035040 con 0x555eabf04370 2024-09-19T09:22:24.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.195+0000 7fdbcdffb640 1 -- 172.21.15.120:0/2640463763 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fdbdc039a00 con 0x555eabf04370 2024-09-19T09:22:24.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.195+0000 7fdbf27a0740 1 -- 172.21.15.120:0/2640463763 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbd801dbe0 con 0x555eabf04370 2024-09-19T09:22:24.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.196+0000 7fdbcdffb640 1 -- 172.21.15.120:0/2640463763 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdbdc0395f0 con 0x555eabf04370 2024-09-19T09:22:24.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.196+0000 7fdbf27a0740 10 client.21948 fetch_fsmap learned FSMap version 39 2024-09-19T09:22:24.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.196+0000 7fdbf27a0740 1 -- 172.21.15.120:0/2640463763 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555eabf7e540 con 0x555eabf04370 2024-09-19T09:22:24.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.196+0000 7fdbcdffb640 1 -- 172.21.15.120:0/2640463763 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbd801dbe0 con 0x555eabf04370 2024-09-19T09:22:24.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.196+0000 7fdbcdffb640 1 -- 172.21.15.120:0/2640463763 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fdbdc0035f0 con 0x555eabf04370 2024-09-19T09:22:24.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.196+0000 7fdbf27a0740 10 client.21948 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:22:24.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.196+0000 7fdbf27a0740 10 client.21948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:22:24.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.196+0000 7fdbf27a0740 20 client.21948 populate_metadata read hostname 'smithi120' 2024-09-19T09:22:24.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.196+0000 7fdbf27a0740 1 --2- 172.21.15.120:0/2640463763 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555eabf7ec00 0x555eabf81020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:24.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.196+0000 7fdbf27a0740 4 client.21948 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:22:24.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.196+0000 7fdbf27a0740 1 -- 172.21.15.120:0/2640463763 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555eabe66d70 con 0x555eabf7ec00 2024-09-19T09:22:24.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.196+0000 7fdbee163640 1 --2- 172.21.15.120:0/2640463763 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555eabf7ec00 0x555eabf81020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:24.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.197+0000 7fdbee163640 1 --2- 172.21.15.120:0/2640463763 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555eabf7ec00 0x555eabf81020 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8426f5a42335073 server_cookie=625a1a64098e7a09 in_seq=0 out_seq=0 2024-09-19T09:22:24.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.197+0000 7fdbcdffb640 10 client.21948 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:24.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.197+0000 7fdbcdffb640 1 -- 172.21.15.120:0/2640463763 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555eabe66d70 con 0x555eabf7ec00 2024-09-19T09:22:24.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.197+0000 7fdbcdffb640 10 client.21948 handle_command_reply: tid=0 2024-09-19T09:22:24.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.197+0000 7fdbf27a0740 1 client.21948 shutdown 2024-09-19T09:22:24.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.197+0000 7fdbf27a0740 1 -- 172.21.15.120:0/2640463763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555eabf04370 msgr2=0x555eabf17000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:24.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.197+0000 7fdbf27a0740 1 --2- 172.21.15.120:0/2640463763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555eabf04370 0x555eabf17000 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc031bf0 tx=0x7fdbdc02aaa0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.197+0000 7fdbf27a0740 1 -- 172.21.15.120:0/2640463763 shutdown_connections 2024-09-19T09:22:24.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.197+0000 7fdbf27a0740 1 --2- 172.21.15.120:0/2640463763 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555eabf7ec00 0x555eabf81020 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.197+0000 7fdbf27a0740 1 --2- 172.21.15.120:0/2640463763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555eabf04370 0x555eabf17000 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.197+0000 7fdbf27a0740 1 --2- 172.21.15.120:0/2640463763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555eabefab40 0x555eabf16ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.197+0000 7fdbf27a0740 1 --2- 172.21.15.120:0/2640463763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555eabef95d0 0x555eabf19770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.197+0000 7fdbcdffb640 0 client.21948 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:24.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.197+0000 7fdbf27a0740 1 -- 172.21.15.120:0/2640463763 >> 172.21.15.120:0/2640463763 conn(0x555eabec2310 msgr2=0x555eabefb110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:24.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.197+0000 7fdbf27a0740 1 -- 172.21.15.120:0/2640463763 shutdown_connections 2024-09-19T09:22:24.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.197+0000 7fdbf27a0740 1 -- 172.21.15.120:0/2640463763 wait complete. 2024-09-19T09:22:24.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.197+0000 7fdbf27a0740 20 client.21948 trim_cache size 0 max 16384 2024-09-19T09:22:24.215 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:22:24.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.198+0000 7fdbf03c7640 1 -- 172.21.15.120:0/3625214681 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdbd00a3240 msgr2=0x7fdbd00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:24.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.198+0000 7fdbf03c7640 1 --2- 172.21.15.120:0/3625214681 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdbd00a3240 0x7fdbd00a5700 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fdbe00a4750 tx=0x7fdbe006dde0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.198+0000 7fdbf03c7640 1 -- 172.21.15.120:0/3625214681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbe8106a80 msgr2=0x7fdbe819cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:24.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.198+0000 7fdbf03c7640 1 --2- 172.21.15.120:0/3625214681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbe8106a80 0x7fdbe819cca0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000c970 tx=0x7fdbe000ce30 comp rx=0 tx=0).stop 2024-09-19T09:22:24.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.198+0000 7fdbf03c7640 1 -- 172.21.15.120:0/3625214681 shutdown_connections 2024-09-19T09:22:24.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.198+0000 7fdbf03c7640 1 --2- 172.21.15.120:0/3625214681 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdbd00a3240 0x7fdbd00a5700 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.198+0000 7fdbf03c7640 1 --2- 172.21.15.120:0/3625214681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbe8106a80 0x7fdbe819cca0 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.198+0000 7fdbf03c7640 1 --2- 172.21.15.120:0/3625214681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbe8101420 0x7fdbe8198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.198+0000 7fdbf03c7640 1 --2- 172.21.15.120:0/3625214681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbe8100a80 0x7fdbe81983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:24.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.198+0000 7fdbf03c7640 1 -- 172.21.15.120:0/3625214681 >> 172.21.15.120:0/3625214681 conn(0x7fdbe80fc7a0 msgr2=0x7fdbe806aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:24.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.198+0000 7fdbf03c7640 1 -- 172.21.15.120:0/3625214681 shutdown_connections 2024-09-19T09:22:24.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:24.198+0000 7fdbf03c7640 1 -- 172.21.15.120:0/3625214681 wait complete. 2024-09-19T09:22:24.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:24 smithi120 ceph-mon[31150]: pgmap v931: 97 pgs: 97 active+clean; 399 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 13 MiB/s wr, 741 op/s 2024-09-19T09:22:24.373 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 21.51 MB/sec execute 259 sec latency 3602.685 ms 2024-09-19T09:22:24.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:24 smithi196 ceph-mon[35150]: pgmap v931: 97 pgs: 97 active+clean; 399 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 13 MiB/s wr, 741 op/s 2024-09-19T09:22:24.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:24 smithi153 ceph-mon[35071]: pgmap v931: 97 pgs: 97 active+clean; 399 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 13 MiB/s wr, 741 op/s 2024-09-19T09:22:25.373 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 21.43 MB/sec execute 260 sec latency 4602.763 ms 2024-09-19T09:22:25.980 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:22:25.980 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:22:26.240 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:22:26.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:26 smithi120 ceph-mon[31150]: pgmap v932: 97 pgs: 97 active+clean; 399 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 13 MiB/s wr, 625 op/s 2024-09-19T09:22:26.373 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 21.35 MB/sec execute 261 sec latency 5602.853 ms 2024-09-19T09:22:26.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:26 smithi196 ceph-mon[35150]: pgmap v932: 97 pgs: 97 active+clean; 399 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 13 MiB/s wr, 625 op/s 2024-09-19T09:22:26.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:26 smithi153 ceph-mon[35071]: pgmap v932: 97 pgs: 97 active+clean; 399 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 13 MiB/s wr, 625 op/s 2024-09-19T09:22:26.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.739+0000 7f8338e98640 1 -- 172.21.15.120:0/2162088159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83341055b0 msgr2=0x7f833410d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:26.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.739+0000 7f8338e98640 1 --2- 172.21.15.120:0/2162088159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83341055b0 0x7f833410d130 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f832c009ca0 tx=0x7f832c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.739+0000 7f8338e98640 1 -- 172.21.15.120:0/2162088159 shutdown_connections 2024-09-19T09:22:26.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.739+0000 7f8338e98640 1 --2- 172.21.15.120:0/2162088159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83341055b0 0x7f833410d130 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.739+0000 7f8338e98640 1 --2- 172.21.15.120:0/2162088159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8334100e30 0x7f8334104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.739+0000 7f8338e98640 1 --2- 172.21.15.120:0/2162088159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8334100480 0x7f8334100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.739+0000 7f8338e98640 1 -- 172.21.15.120:0/2162088159 >> 172.21.15.120:0/2162088159 conn(0x7f8334077210 msgr2=0x7f83340fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:26.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.739+0000 7f8338e98640 1 -- 172.21.15.120:0/2162088159 shutdown_connections 2024-09-19T09:22:26.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.739+0000 7f8338e98640 1 -- 172.21.15.120:0/2162088159 wait complete. 2024-09-19T09:22:26.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.739+0000 7f8338e98640 1 Processor -- start 2024-09-19T09:22:26.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.739+0000 7f8338e98640 1 -- start start 2024-09-19T09:22:26.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.740+0000 7f8338e98640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8334100480 0x7f83341961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:26.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.740+0000 7f8338e98640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8334100e30 0x7f83341966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:26.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.740+0000 7f8338e98640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83341055b0 0x7f833419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:26.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.740+0000 7f8338e98640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f833410fd90 con 0x7f8334100e30 2024-09-19T09:22:26.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.740+0000 7f8338e98640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f833410fc10 con 0x7f83341055b0 2024-09-19T09:22:26.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.740+0000 7f8338e98640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f833410ff10 con 0x7f8334100480 2024-09-19T09:22:26.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.740+0000 7f8331d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8334100e30 0x7f83341966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:26.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.740+0000 7f8331d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8334100e30 0x7f83341966e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52692/0 (socket says 172.21.15.120:52692) 2024-09-19T09:22:26.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.740+0000 7f8331d9b640 1 -- 172.21.15.120:0/3663658595 learned_addr learned my addr 172.21.15.120:0/3663658595 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:26.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.740+0000 7f833259c640 1 --2- 172.21.15.120:0/3663658595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8334100480 0x7f83341961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:26.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.740+0000 7f8332d9d640 1 --2- 172.21.15.120:0/3663658595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83341055b0 0x7f833419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:26.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.740+0000 7f8331d9b640 1 -- 172.21.15.120:0/3663658595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8334100480 msgr2=0x7f83341961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:26.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.740+0000 7f8331d9b640 1 --2- 172.21.15.120:0/3663658595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8334100480 0x7f83341961a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.740+0000 7f8331d9b640 1 -- 172.21.15.120:0/3663658595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83341055b0 msgr2=0x7f833419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:26.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.740+0000 7f8331d9b640 1 --2- 172.21.15.120:0/3663658595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83341055b0 0x7f833419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.740+0000 7f8331d9b640 1 -- 172.21.15.120:0/3663658595 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f833419b150 con 0x7f8334100e30 2024-09-19T09:22:26.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.740+0000 7f8331d9b640 1 --2- 172.21.15.120:0/3663658595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8334100e30 0x7f83341966e0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f832400edd0 tx=0x7f832400a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:26.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.741+0000 7f832b7fe640 1 -- 172.21.15.120:0/3663658595 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8324018620 con 0x7f8334100e30 2024-09-19T09:22:26.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.741+0000 7f832b7fe640 1 -- 172.21.15.120:0/3663658595 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f832400ce50 con 0x7f8334100e30 2024-09-19T09:22:26.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.741+0000 7f832b7fe640 1 -- 172.21.15.120:0/3663658595 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832400bbb0 con 0x7f8334100e30 2024-09-19T09:22:26.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.741+0000 7f8338e98640 1 -- 172.21.15.120:0/3663658595 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f833419b440 con 0x7f8334100e30 2024-09-19T09:22:26.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.741+0000 7f8338e98640 1 -- 172.21.15.120:0/3663658595 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83341a2d20 con 0x7f8334100e30 2024-09-19T09:22:26.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.743+0000 7f832b7fe640 1 -- 172.21.15.120:0/3663658595 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f83240187c0 con 0x7f8334100e30 2024-09-19T09:22:26.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.743+0000 7f833b271740 1 Processor -- start 2024-09-19T09:22:26.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.744+0000 7f833b271740 1 -- start start 2024-09-19T09:22:26.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.744+0000 7f833b271740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b33a56eb60 0x55b33a56ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:26.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.744+0000 7f833b271740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b33a56f480 0x55b33a579800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:26.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.744+0000 7f833b271740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b33a579d40 0x55b33a57c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:26.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.744+0000 7f833b271740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b33a4d4280 con 0x55b33a56eb60 2024-09-19T09:22:26.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.744+0000 7f833b271740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b33a47bb10 con 0x55b33a579d40 2024-09-19T09:22:26.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.744+0000 7f833b271740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b33a4d0240 con 0x55b33a56f480 2024-09-19T09:22:26.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.744+0000 7f8332d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b33a56eb60 0x55b33a56ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:26.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.744+0000 7f8332d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b33a56eb60 0x55b33a56ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52702/0 (socket says 172.21.15.120:52702) 2024-09-19T09:22:26.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.744+0000 7f8332d9d640 1 -- 172.21.15.120:0/1619378480 learned_addr learned my addr 172.21.15.120:0/1619378480 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:26.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.744+0000 7f832b7fe640 1 --2- 172.21.15.120:0/3663658595 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83200a32c0 0x7f83200a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:26.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.744+0000 7f8331d9b640 1 --2- 172.21.15.120:0/1619378480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b33a579d40 0x55b33a57c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:26.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.744+0000 7f833259c640 1 --2- 172.21.15.120:0/1619378480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b33a56f480 0x55b33a579800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:26.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.744+0000 7f8332d9d640 1 --2- 172.21.15.120:0/3663658595 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83200a32c0 0x7f83200a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:26.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.744+0000 7f832b7fe640 1 -- 172.21.15.120:0/3663658595 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f83240d0800 con 0x7f8334100e30 2024-09-19T09:22:26.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.744+0000 7f8332d9d640 1 -- 172.21.15.120:0/1619378480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b33a56f480 msgr2=0x55b33a579800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:26.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.744+0000 7f8332d9d640 1 --2- 172.21.15.120:0/1619378480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b33a56f480 0x55b33a579800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.744+0000 7f8332d9d640 1 -- 172.21.15.120:0/1619378480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b33a579d40 msgr2=0x55b33a57c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:26.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.744+0000 7f8332d9d640 1 --2- 172.21.15.120:0/1619378480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b33a579d40 0x55b33a57c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.744+0000 7f8332d9d640 1 -- 172.21.15.120:0/1619378480 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b33a4881f0 con 0x55b33a56eb60 2024-09-19T09:22:26.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.745+0000 7f8332d9d640 1 --2- 172.21.15.120:0/1619378480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b33a56eb60 0x55b33a56ef40 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f832c005fe0 tx=0x7f832c009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff7872ca7b9deec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:26.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.745+0000 7f8328ff9640 1 -- 172.21.15.120:0/1619378480 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c003e10 con 0x55b33a56eb60 2024-09-19T09:22:26.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.745+0000 7f8328ff9640 1 -- 172.21.15.120:0/1619378480 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f832c04e070 con 0x55b33a56eb60 2024-09-19T09:22:26.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.745+0000 7f8328ff9640 1 -- 172.21.15.120:0/1619378480 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c02fa10 con 0x55b33a56eb60 2024-09-19T09:22:26.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.746+0000 7f833b271740 1 -- 172.21.15.120:0/1619378480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b33a56eb60 msgr2=0x55b33a56ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:26.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.746+0000 7f833b271740 1 --2- 172.21.15.120:0/1619378480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b33a56eb60 0x55b33a56ef40 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f832c005fe0 tx=0x7f832c009a00 comp rx=0 tx=0).stop 2024-09-19T09:22:26.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.746+0000 7f8332d9d640 1 --2- 172.21.15.120:0/3663658595 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83200a32c0 0x7f83200a5780 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f832c02f540 tx=0x7f832c04b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:26.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.746+0000 7f833b271740 1 -- 172.21.15.120:0/1619378480 shutdown_connections 2024-09-19T09:22:26.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.746+0000 7f833b271740 1 --2- 172.21.15.120:0/1619378480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b33a579d40 0x55b33a57c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.746+0000 7f833b271740 1 --2- 172.21.15.120:0/1619378480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b33a56f480 0x55b33a579800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.746+0000 7f833b271740 1 --2- 172.21.15.120:0/1619378480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b33a56eb60 0x55b33a56ef40 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.746+0000 7f833b271740 1 -- 172.21.15.120:0/1619378480 >> 172.21.15.120:0/1619378480 conn(0x55b33a56b0f0 msgr2=0x55b33a56b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:26.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.746+0000 7f833b271740 1 -- 172.21.15.120:0/1619378480 shutdown_connections 2024-09-19T09:22:26.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.746+0000 7f833b271740 1 -- 172.21.15.120:0/1619378480 wait complete. 2024-09-19T09:22:26.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.746+0000 7f833b271740 1 -- start start 2024-09-19T09:22:26.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.746+0000 7f833b271740 1 Processor -- start 2024-09-19T09:22:26.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.746+0000 7f833b271740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b33a56eb60 0x55b33a57cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:26.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.746+0000 7f833b271740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b33a56f480 0x55b33a57d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:26.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.746+0000 7f833b271740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b33a579d40 0x55b33a57d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:26.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.746+0000 7f833b271740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b33a57c790 con 0x55b33a56eb60 2024-09-19T09:22:26.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.746+0000 7f833b271740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b33a48ef10 con 0x55b33a579d40 2024-09-19T09:22:26.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.746+0000 7f833b271740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b33a57c910 con 0x55b33a56f480 2024-09-19T09:22:26.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.746+0000 7f833b271740 10 client.0 _ll_register_callbacks cb 0x55b33a457a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:22:26.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.747+0000 7f833259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b33a56eb60 0x55b33a57cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:26.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.747+0000 7f833259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b33a56eb60 0x55b33a57cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52712/0 (socket says 172.21.15.120:52712) 2024-09-19T09:22:26.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.747+0000 7f833259c640 1 -- 172.21.15.120:0/4118260106 learned_addr learned my addr 172.21.15.120:0/4118260106 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:26.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.747+0000 7f8331d9b640 1 --2- 172.21.15.120:0/4118260106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b33a56f480 0x55b33a57d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:26.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.747+0000 7f8332d9d640 1 --2- 172.21.15.120:0/4118260106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b33a579d40 0x55b33a57d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:26.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.747+0000 7f833259c640 1 -- 172.21.15.120:0/4118260106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b33a56f480 msgr2=0x55b33a57d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:26.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.747+0000 7f833259c640 1 --2- 172.21.15.120:0/4118260106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b33a56f480 0x55b33a57d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.747+0000 7f833259c640 1 -- 172.21.15.120:0/4118260106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b33a579d40 msgr2=0x55b33a57d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:26.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.747+0000 7f833259c640 1 --2- 172.21.15.120:0/4118260106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b33a579d40 0x55b33a57d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.747+0000 7f833259c640 1 -- 172.21.15.120:0/4118260106 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b33a2456a0 con 0x55b33a56eb60 2024-09-19T09:22:26.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.747+0000 7f8332d9d640 1 --2- 172.21.15.120:0/4118260106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b33a579d40 0x55b33a57d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:22:26.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.747+0000 7f833259c640 1 --2- 172.21.15.120:0/4118260106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b33a56eb60 0x55b33a57cba0 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f831c00bef0 tx=0x7f831c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd91b144f9fad72 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:26.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.747+0000 7f8319ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:22:26.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.747+0000 7f8319ffb640 1 -- 172.21.15.120:0/4118260106 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c005220 con 0x55b33a56eb60 2024-09-19T09:22:26.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.747+0000 7f8319ffb640 1 -- 172.21.15.120:0/4118260106 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f831c0053c0 con 0x55b33a56eb60 2024-09-19T09:22:26.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.747+0000 7f833b271740 1 -- 172.21.15.120:0/4118260106 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f832c04e070 con 0x55b33a56eb60 2024-09-19T09:22:26.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.747+0000 7f8319ffb640 1 -- 172.21.15.120:0/4118260106 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f831c0055c0 con 0x55b33a56eb60 2024-09-19T09:22:26.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.747+0000 7f833b271740 10 client.21960 fetch_fsmap learned FSMap version 39 2024-09-19T09:22:26.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.747+0000 7f833b271740 1 -- 172.21.15.120:0/4118260106 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b33a58a660 con 0x55b33a56eb60 2024-09-19T09:22:26.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.748+0000 7f8319ffb640 1 -- 172.21.15.120:0/4118260106 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f832c04e070 con 0x55b33a56eb60 2024-09-19T09:22:26.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.748+0000 7f8319ffb640 1 -- 172.21.15.120:0/4118260106 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f831c010cb0 con 0x55b33a56eb60 2024-09-19T09:22:26.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.748+0000 7f833b271740 10 client.21960 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:22:26.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.748+0000 7f833b271740 10 client.21960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:22:26.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.748+0000 7f833b271740 20 client.21960 populate_metadata read hostname 'smithi120' 2024-09-19T09:22:26.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.748+0000 7f833b271740 1 --2- 172.21.15.120:0/4118260106 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b33a58bb20 0x55b33a58df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:26.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.748+0000 7f833b271740 4 client.21960 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:22:26.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.748+0000 7f833b271740 1 -- 172.21.15.120:0/4118260106 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b33a4e17f0 con 0x55b33a58bb20 2024-09-19T09:22:26.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.748+0000 7f8331d9b640 1 --2- 172.21.15.120:0/4118260106 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b33a58bb20 0x55b33a58df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:26.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.749+0000 7f8331d9b640 1 --2- 172.21.15.120:0/4118260106 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b33a58bb20 0x55b33a58df90 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c5af9509fe469e server_cookie=1b37451526555cf in_seq=0 out_seq=0 2024-09-19T09:22:26.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.749+0000 7f8319ffb640 10 client.21960 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:26.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.749+0000 7f8319ffb640 1 -- 172.21.15.120:0/4118260106 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b33a4e17f0 con 0x55b33a58bb20 2024-09-19T09:22:26.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.749+0000 7f8319ffb640 10 client.21960 handle_command_reply: tid=0 2024-09-19T09:22:26.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.749+0000 7f833b271740 1 client.21960 shutdown 2024-09-19T09:22:26.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.749+0000 7f833b271740 1 -- 172.21.15.120:0/4118260106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b33a56eb60 msgr2=0x55b33a57cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:26.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.749+0000 7f833b271740 1 --2- 172.21.15.120:0/4118260106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b33a56eb60 0x55b33a57cba0 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f831c00bef0 tx=0x7f831c012820 comp rx=0 tx=0).stop 2024-09-19T09:22:26.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.750+0000 7f833b271740 1 -- 172.21.15.120:0/4118260106 shutdown_connections 2024-09-19T09:22:26.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.750+0000 7f833b271740 1 --2- 172.21.15.120:0/4118260106 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b33a58bb20 0x55b33a58df90 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.750+0000 7f833b271740 1 --2- 172.21.15.120:0/4118260106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b33a579d40 0x55b33a57d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.750+0000 7f833b271740 1 --2- 172.21.15.120:0/4118260106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b33a56f480 0x55b33a57d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.750+0000 7f833b271740 1 --2- 172.21.15.120:0/4118260106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b33a56eb60 0x55b33a57cba0 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.750+0000 7f833b271740 1 -- 172.21.15.120:0/4118260106 >> 172.21.15.120:0/4118260106 conn(0x55b33a56b0f0 msgr2=0x55b33a56a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:26.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.750+0000 7f8319ffb640 0 client.21960 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:26.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.750+0000 7f833b271740 1 -- 172.21.15.120:0/4118260106 shutdown_connections 2024-09-19T09:22:26.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.750+0000 7f833b271740 1 -- 172.21.15.120:0/4118260106 wait complete. 2024-09-19T09:22:26.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.750+0000 7f833b271740 20 client.21960 trim_cache size 0 max 16384 2024-09-19T09:22:26.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.767+0000 7f833b271740 1 Processor -- start 2024-09-19T09:22:26.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.767+0000 7f833b271740 1 -- start start 2024-09-19T09:22:26.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.767+0000 7f833b271740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b33a56eb60 0x55b33a593860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:26.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.767+0000 7f833b271740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b33a56f480 0x55b33a593da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:26.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.767+0000 7f833b271740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b33a579d40 0x55b33a573e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:26.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.767+0000 7f833b271740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f83341ab300 con 0x55b33a56eb60 2024-09-19T09:22:26.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.767+0000 7f833b271740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f83341ace40 con 0x55b33a579d40 2024-09-19T09:22:26.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.767+0000 7f833b271740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f83341ab9d0 con 0x55b33a56f480 2024-09-19T09:22:26.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.767+0000 7f833259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b33a56eb60 0x55b33a593860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:26.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.767+0000 7f833259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b33a56eb60 0x55b33a593860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52718/0 (socket says 172.21.15.120:52718) 2024-09-19T09:22:26.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.767+0000 7f833259c640 1 -- 172.21.15.120:0/2694362710 learned_addr learned my addr 172.21.15.120:0/2694362710 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:26.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.768+0000 7f8331d9b640 1 --2- 172.21.15.120:0/2694362710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b33a56f480 0x55b33a593da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:26.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.768+0000 7f833259c640 1 -- 172.21.15.120:0/2694362710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b33a56f480 msgr2=0x55b33a593da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:26.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.768+0000 7f833259c640 1 --2- 172.21.15.120:0/2694362710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b33a56f480 0x55b33a593da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.768+0000 7f833259c640 1 -- 172.21.15.120:0/2694362710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b33a579d40 msgr2=0x55b33a573e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:26.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.768+0000 7f833259c640 1 --2- 172.21.15.120:0/2694362710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b33a579d40 0x55b33a573e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.768+0000 7f833259c640 1 -- 172.21.15.120:0/2694362710 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b33a56ab90 con 0x55b33a56eb60 2024-09-19T09:22:26.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.768+0000 7f833259c640 1 --2- 172.21.15.120:0/2694362710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b33a56eb60 0x55b33a593860 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f831c006010 tx=0x7f831c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c16a39614426001 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:26.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.768+0000 7f831bfff640 1 -- 172.21.15.120:0/2694362710 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c01c940 con 0x55b33a56eb60 2024-09-19T09:22:26.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.768+0000 7f831bfff640 1 -- 172.21.15.120:0/2694362710 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f831c02b070 con 0x55b33a56eb60 2024-09-19T09:22:26.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.768+0000 7f831bfff640 1 -- 172.21.15.120:0/2694362710 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c010b20 con 0x55b33a56eb60 2024-09-19T09:22:26.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.769+0000 7f833b271740 1 -- 172.21.15.120:0/2694362710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b33a56eb60 msgr2=0x55b33a593860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:26.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.769+0000 7f833b271740 1 --2- 172.21.15.120:0/2694362710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b33a56eb60 0x55b33a593860 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f831c006010 tx=0x7f831c00b890 comp rx=0 tx=0).stop 2024-09-19T09:22:26.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.769+0000 7f833b271740 1 -- 172.21.15.120:0/2694362710 shutdown_connections 2024-09-19T09:22:26.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.769+0000 7f833b271740 1 --2- 172.21.15.120:0/2694362710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b33a579d40 0x55b33a573e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.769+0000 7f833b271740 1 --2- 172.21.15.120:0/2694362710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b33a56f480 0x55b33a593da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.769+0000 7f833b271740 1 --2- 172.21.15.120:0/2694362710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b33a56eb60 0x55b33a593860 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.769+0000 7f833b271740 1 -- 172.21.15.120:0/2694362710 >> 172.21.15.120:0/2694362710 conn(0x55b33a56b0f0 msgr2=0x55b33a56a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:26.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.769+0000 7f833b271740 1 -- 172.21.15.120:0/2694362710 shutdown_connections 2024-09-19T09:22:26.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.769+0000 7f833b271740 1 -- 172.21.15.120:0/2694362710 wait complete. 2024-09-19T09:22:26.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.769+0000 7f833b271740 1 -- start start 2024-09-19T09:22:26.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.769+0000 7f833b271740 1 Processor -- start 2024-09-19T09:22:26.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.769+0000 7f833b271740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b33a56eb60 0x55b33a57ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:26.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.769+0000 7f833b271740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b33a56f480 0x55b33a5804c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:26.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.769+0000 7f833b271740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b33a579d40 0x55b33a5f5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:26.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.769+0000 7f833b271740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f83341ac770 con 0x55b33a579d40 2024-09-19T09:22:26.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.769+0000 7f833b271740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f83341ac0a0 con 0x55b33a56f480 2024-09-19T09:22:26.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.769+0000 7f833b271740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f83341a92c0 con 0x55b33a56eb60 2024-09-19T09:22:26.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.770+0000 7f833b271740 10 client.0 _ll_register_callbacks cb 0x7f831c00bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:22:26.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.770+0000 7f8332d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b33a579d40 0x55b33a5f5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:26.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.770+0000 7f833259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b33a56eb60 0x55b33a57ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:26.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.770+0000 7f8332d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b33a579d40 0x55b33a5f5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52720/0 (socket says 172.21.15.120:52720) 2024-09-19T09:22:26.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.770+0000 7f8332d9d640 1 -- 172.21.15.120:0/2193350390 learned_addr learned my addr 172.21.15.120:0/2193350390 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:26.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.770+0000 7f8331d9b640 1 --2- 172.21.15.120:0/2193350390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b33a56f480 0x55b33a5804c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:26.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.770+0000 7f8332d9d640 1 -- 172.21.15.120:0/2193350390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b33a56eb60 msgr2=0x55b33a57ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:26.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.770+0000 7f8332d9d640 1 --2- 172.21.15.120:0/2193350390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b33a56eb60 0x55b33a57ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.770+0000 7f8332d9d640 1 -- 172.21.15.120:0/2193350390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b33a56f480 msgr2=0x55b33a5804c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:26.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.770+0000 7f8332d9d640 1 --2- 172.21.15.120:0/2193350390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b33a56f480 0x55b33a5804c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.770+0000 7f8332d9d640 1 -- 172.21.15.120:0/2193350390 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b33a5f5d80 con 0x55b33a579d40 2024-09-19T09:22:26.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.770+0000 7f8332d9d640 1 --2- 172.21.15.120:0/2193350390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b33a579d40 0x55b33a5f5840 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f832c02f500 tx=0x7f832c005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d277ca24256089 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:26.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.770+0000 7f8319ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:22:26.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.770+0000 7f8319ffb640 1 -- 172.21.15.120:0/2193350390 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c0319a0 con 0x55b33a579d40 2024-09-19T09:22:26.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.770+0000 7f8319ffb640 1 -- 172.21.15.120:0/2193350390 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f832c004410 con 0x55b33a579d40 2024-09-19T09:22:26.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.770+0000 7f833b271740 1 -- 172.21.15.120:0/2193350390 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f831c02b070 con 0x55b33a579d40 2024-09-19T09:22:26.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.770+0000 7f8319ffb640 1 -- 172.21.15.120:0/2193350390 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f832c004610 con 0x55b33a579d40 2024-09-19T09:22:26.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.770+0000 7f833b271740 10 client.21966 fetch_fsmap learned FSMap version 39 2024-09-19T09:22:26.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.770+0000 7f833b271740 1 -- 172.21.15.120:0/2193350390 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b33a5f7020 con 0x55b33a579d40 2024-09-19T09:22:26.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.771+0000 7f8319ffb640 1 -- 172.21.15.120:0/2193350390 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f831c02b070 con 0x55b33a579d40 2024-09-19T09:22:26.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.771+0000 7f8319ffb640 1 -- 172.21.15.120:0/2193350390 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f832c030aa0 con 0x55b33a579d40 2024-09-19T09:22:26.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.771+0000 7f833b271740 10 client.21966 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:22:26.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.771+0000 7f833b271740 10 client.21966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:22:26.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.771+0000 7f833b271740 20 client.21966 populate_metadata read hostname 'smithi120' 2024-09-19T09:22:26.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.771+0000 7f833b271740 1 --2- 172.21.15.120:0/2193350390 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b33a5f74b0 0x55b33a5f98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:26.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.771+0000 7f833b271740 4 client.21966 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:22:26.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.771+0000 7f833b271740 1 -- 172.21.15.120:0/2193350390 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b33a4dbd00 con 0x55b33a5f74b0 2024-09-19T09:22:26.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.771+0000 7f833259c640 1 --2- 172.21.15.120:0/2193350390 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b33a5f74b0 0x55b33a5f98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:26.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.772+0000 7f8319ffb640 10 client.21966 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:26.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.772+0000 7f833259c640 1 --2- 172.21.15.120:0/2193350390 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b33a5f74b0 0x55b33a5f98a0 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff38459e91e854e4 server_cookie=7acfe32b6158b34b in_seq=0 out_seq=0 2024-09-19T09:22:26.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.773+0000 7f8319ffb640 1 -- 172.21.15.120:0/2193350390 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b33a4dbd00 con 0x55b33a5f74b0 2024-09-19T09:22:26.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.773+0000 7f8319ffb640 10 client.21966 handle_command_reply: tid=0 2024-09-19T09:22:26.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.773+0000 7f833b271740 1 client.21966 shutdown 2024-09-19T09:22:26.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.773+0000 7f833b271740 1 -- 172.21.15.120:0/2193350390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b33a579d40 msgr2=0x55b33a5f5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:26.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.773+0000 7f833b271740 1 --2- 172.21.15.120:0/2193350390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b33a579d40 0x55b33a5f5840 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f832c02f500 tx=0x7f832c005d80 comp rx=0 tx=0).stop 2024-09-19T09:22:26.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.773+0000 7f833b271740 1 -- 172.21.15.120:0/2193350390 shutdown_connections 2024-09-19T09:22:26.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.773+0000 7f833b271740 1 --2- 172.21.15.120:0/2193350390 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b33a5f74b0 0x55b33a5f98a0 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.773+0000 7f833b271740 1 --2- 172.21.15.120:0/2193350390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b33a579d40 0x55b33a5f5840 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.773+0000 7f833b271740 1 --2- 172.21.15.120:0/2193350390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b33a56f480 0x55b33a5804c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.773+0000 7f833b271740 1 --2- 172.21.15.120:0/2193350390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b33a56eb60 0x55b33a57ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.773+0000 7f833b271740 1 -- 172.21.15.120:0/2193350390 >> 172.21.15.120:0/2193350390 conn(0x55b33a56b0f0 msgr2=0x55b33a5869f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:26.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.773+0000 7f8319ffb640 0 client.21966 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:26.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.773+0000 7f833b271740 1 -- 172.21.15.120:0/2193350390 shutdown_connections 2024-09-19T09:22:26.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.773+0000 7f833b271740 1 -- 172.21.15.120:0/2193350390 wait complete. 2024-09-19T09:22:26.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.773+0000 7f833b271740 20 client.21966 trim_cache size 0 max 16384 2024-09-19T09:22:26.779 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:22:26.779 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:22:26.779 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "cf5e1f94-e903-4bae-9ad1-9e831bc75236", 2024-09-19T09:22:26.779 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:22:26.779 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:22:26.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.773+0000 7f8338e98640 1 -- 172.21.15.120:0/3663658595 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83200a32c0 msgr2=0x7f83200a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:26.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.774+0000 7f8338e98640 1 --2- 172.21.15.120:0/3663658595 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83200a32c0 0x7f83200a5780 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f832c02f540 tx=0x7f832c04b000 comp rx=0 tx=0).stop 2024-09-19T09:22:26.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.774+0000 7f8338e98640 1 -- 172.21.15.120:0/3663658595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8334100e30 msgr2=0x7f83341966e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:26.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.774+0000 7f8338e98640 1 --2- 172.21.15.120:0/3663658595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8334100e30 0x7f83341966e0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f832400edd0 tx=0x7f832400a720 comp rx=0 tx=0).stop 2024-09-19T09:22:26.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.774+0000 7f8338e98640 1 -- 172.21.15.120:0/3663658595 shutdown_connections 2024-09-19T09:22:26.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.774+0000 7f8338e98640 1 --2- 172.21.15.120:0/3663658595 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83200a32c0 0x7f83200a5780 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.774+0000 7f8338e98640 1 --2- 172.21.15.120:0/3663658595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83341055b0 0x7f833419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.774+0000 7f8338e98640 1 --2- 172.21.15.120:0/3663658595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8334100e30 0x7f83341966e0 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.774+0000 7f8338e98640 1 --2- 172.21.15.120:0/3663658595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8334100480 0x7f83341961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:26.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.774+0000 7f8338e98640 1 -- 172.21.15.120:0/3663658595 >> 172.21.15.120:0/3663658595 conn(0x7f8334077210 msgr2=0x7f8334103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:26.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.774+0000 7f8338e98640 1 -- 172.21.15.120:0/3663658595 shutdown_connections 2024-09-19T09:22:26.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:26.774+0000 7f8338e98640 1 -- 172.21.15.120:0/3663658595 wait complete. 2024-09-19T09:22:27.373 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 21.27 MB/sec execute 262 sec latency 6602.902 ms 2024-09-19T09:22:27.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:27 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:22:27.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:27 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:22:27.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:27 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:22:27.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:27 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:22:27.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:27 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:22:27.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:27 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:22:27.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:27 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:22:27.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:27 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:22:27.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:27 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:22:27.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:27 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:22:27.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:27 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:22:27.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:27 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:22:27.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:27 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:22:27.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:27 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:22:27.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:27 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:22:27.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:27 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:22:27.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:27 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:22:27.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:27 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:22:27.636 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:22:27.883 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:22:28.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:28 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:22:28.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:28 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:22:28.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:28 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:22:28.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:28 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:22:28.260 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:28 smithi120 ceph-mon[31150]: pgmap v933: 97 pgs: 97 active+clean; 391 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 13 MiB/s wr, 660 op/s 2024-09-19T09:22:28.373 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 21.19 MB/sec execute 263 sec latency 7602.947 ms 2024-09-19T09:22:28.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.339+0000 7fabaac37640 1 -- 172.21.15.120:0/363237974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faba4106f10 msgr2=0x7faba410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:28.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.340+0000 7fabaac37640 1 --2- 172.21.15.120:0/363237974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faba4106f10 0x7faba410ea90 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009900 tx=0x7fab9c02a410 comp rx=0 tx=0).stop 2024-09-19T09:22:28.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.340+0000 7fabaac37640 1 -- 172.21.15.120:0/363237974 shutdown_connections 2024-09-19T09:22:28.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.340+0000 7fabaac37640 1 --2- 172.21.15.120:0/363237974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faba4106f10 0x7faba410ea90 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.340+0000 7fabaac37640 1 --2- 172.21.15.120:0/363237974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faba4102790 0x7faba41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.340+0000 7fabaac37640 1 --2- 172.21.15.120:0/363237974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faba4101de0 0x7faba41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.340+0000 7fabaac37640 1 -- 172.21.15.120:0/363237974 >> 172.21.15.120:0/363237974 conn(0x7faba40fd8c0 msgr2=0x7faba40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:28.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.340+0000 7fabaac37640 1 -- 172.21.15.120:0/363237974 shutdown_connections 2024-09-19T09:22:28.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.340+0000 7fabaac37640 1 -- 172.21.15.120:0/363237974 wait complete. 2024-09-19T09:22:28.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.340+0000 7fabaac37640 1 Processor -- start 2024-09-19T09:22:28.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.340+0000 7fabaac37640 1 -- start start 2024-09-19T09:22:28.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.340+0000 7fabaac37640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faba4101de0 0x7faba41983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:28.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.340+0000 7fabaac37640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faba4102790 0x7faba41988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:28.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.340+0000 7fabaac37640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faba4106f10 0x7faba419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:28.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.340+0000 7fabaac37640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faba4111f70 con 0x7faba4102790 2024-09-19T09:22:28.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.340+0000 7fabaac37640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faba4111df0 con 0x7faba4101de0 2024-09-19T09:22:28.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.340+0000 7fabaac37640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faba41120f0 con 0x7faba4106f10 2024-09-19T09:22:28.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.340+0000 7faba3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faba4102790 0x7faba41988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:28.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.341+0000 7faba3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faba4102790 0x7faba41988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52738/0 (socket says 172.21.15.120:52738) 2024-09-19T09:22:28.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.341+0000 7faba3fff640 1 -- 172.21.15.120:0/3912044940 learned_addr learned my addr 172.21.15.120:0/3912044940 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:28.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.341+0000 7faba91d4640 1 --2- 172.21.15.120:0/3912044940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faba4106f10 0x7faba419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:28.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.341+0000 7faba89d3640 1 --2- 172.21.15.120:0/3912044940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faba4101de0 0x7faba41983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:28.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.341+0000 7faba3fff640 1 -- 172.21.15.120:0/3912044940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faba4106f10 msgr2=0x7faba419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:28.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.341+0000 7faba3fff640 1 --2- 172.21.15.120:0/3912044940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faba4106f10 0x7faba419cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.341+0000 7faba3fff640 1 -- 172.21.15.120:0/3912044940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faba4101de0 msgr2=0x7faba41983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:28.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.341+0000 7faba3fff640 1 --2- 172.21.15.120:0/3912044940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faba4101de0 0x7faba41983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.341+0000 7faba3fff640 1 -- 172.21.15.120:0/3912044940 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba419d360 con 0x7faba4102790 2024-09-19T09:22:28.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.341+0000 7faba89d3640 1 --2- 172.21.15.120:0/3912044940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faba4101de0 0x7faba41983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:22:28.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.341+0000 7faba3fff640 1 --2- 172.21.15.120:0/3912044940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faba4102790 0x7faba41988f0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fab9800d950 tx=0x7fab9800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:28.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.341+0000 7faba1ffb640 1 -- 172.21.15.120:0/3912044940 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab98017070 con 0x7faba4102790 2024-09-19T09:22:28.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.341+0000 7faba1ffb640 1 -- 172.21.15.120:0/3912044940 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab98009c30 con 0x7faba4102790 2024-09-19T09:22:28.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.341+0000 7fabaac37640 1 -- 172.21.15.120:0/3912044940 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faba419d650 con 0x7faba4102790 2024-09-19T09:22:28.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.341+0000 7faba1ffb640 1 -- 172.21.15.120:0/3912044940 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9800f6b0 con 0x7faba4102790 2024-09-19T09:22:28.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.341+0000 7fabaac37640 1 -- 172.21.15.120:0/3912044940 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faba41a4f30 con 0x7faba4102790 2024-09-19T09:22:28.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.344+0000 7faba1ffb640 1 -- 172.21.15.120:0/3912044940 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fab980109d0 con 0x7faba4102790 2024-09-19T09:22:28.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.344+0000 7fabad010740 1 Processor -- start 2024-09-19T09:22:28.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.344+0000 7fabad010740 1 -- start start 2024-09-19T09:22:28.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.344+0000 7faba1ffb640 1 --2- 172.21.15.120:0/3912044940 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fab8c0a3240 0x7fab8c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:28.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.344+0000 7fabad010740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bcd7114a0 0x561bcd711880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:28.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.344+0000 7fabad010740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bcd711dc0 0x561bcd71c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:28.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.344+0000 7fabad010740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bcd71c680 0x561bcd71ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:28.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.344+0000 7fabad010740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561bcd6772e0 con 0x561bcd711dc0 2024-09-19T09:22:28.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.344+0000 7fabad010740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561bcd61ece0 con 0x561bcd7114a0 2024-09-19T09:22:28.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.344+0000 7fabad010740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561bcd6732a0 con 0x561bcd71c680 2024-09-19T09:22:28.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.344+0000 7faba91d4640 1 --2- 172.21.15.120:0/3912044940 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fab8c0a3240 0x7fab8c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:28.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.344+0000 7faba1ffb640 1 -- 172.21.15.120:0/3912044940 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fab980cf4a0 con 0x7faba4102790 2024-09-19T09:22:28.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.344+0000 7faba3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bcd711dc0 0x561bcd71c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:28.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.344+0000 7faba3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bcd711dc0 0x561bcd71c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52744/0 (socket says 172.21.15.120:52744) 2024-09-19T09:22:28.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.344+0000 7faba3fff640 1 -- 172.21.15.120:0/1674440409 learned_addr learned my addr 172.21.15.120:0/1674440409 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:28.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.344+0000 7faba89d3640 1 --2- 172.21.15.120:0/1674440409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bcd7114a0 0x561bcd711880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:28.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.345+0000 7faba91d4640 1 --2- 172.21.15.120:0/1674440409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bcd71c680 0x561bcd71ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:28.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.345+0000 7faba3fff640 1 -- 172.21.15.120:0/1674440409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bcd71c680 msgr2=0x561bcd71ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:28.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.345+0000 7faba3fff640 1 --2- 172.21.15.120:0/1674440409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bcd71c680 0x561bcd71ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.345+0000 7faba3fff640 1 -- 172.21.15.120:0/1674440409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bcd7114a0 msgr2=0x561bcd711880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:28.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.345+0000 7faba3fff640 1 --2- 172.21.15.120:0/1674440409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bcd7114a0 0x561bcd711880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.345+0000 7faba3fff640 1 -- 172.21.15.120:0/1674440409 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bcd603440 con 0x561bcd711dc0 2024-09-19T09:22:28.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.345+0000 7faba91d4640 1 --2- 172.21.15.120:0/3912044940 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fab8c0a3240 0x7fab8c0a5700 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0096c0 tx=0x7fab9c0317a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:28.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.345+0000 7faba3fff640 1 --2- 172.21.15.120:0/1674440409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bcd711dc0 0x561bcd71c140 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7fab98097f20 tx=0x7fab98000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8771b50056b43a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:28.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.345+0000 7fab8affd640 1 -- 172.21.15.120:0/1674440409 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab98020070 con 0x561bcd711dc0 2024-09-19T09:22:28.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.345+0000 7fab8affd640 1 -- 172.21.15.120:0/1674440409 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab9809fe70 con 0x561bcd711dc0 2024-09-19T09:22:28.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.345+0000 7fab8affd640 1 -- 172.21.15.120:0/1674440409 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab980860b0 con 0x561bcd711dc0 2024-09-19T09:22:28.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.346+0000 7fabad010740 1 -- 172.21.15.120:0/1674440409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bcd711dc0 msgr2=0x561bcd71c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:28.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.346+0000 7fabad010740 1 --2- 172.21.15.120:0/1674440409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bcd711dc0 0x561bcd71c140 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7fab98097f20 tx=0x7fab98000bc0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.346+0000 7fabad010740 1 -- 172.21.15.120:0/1674440409 shutdown_connections 2024-09-19T09:22:28.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.346+0000 7fabad010740 1 --2- 172.21.15.120:0/1674440409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bcd71c680 0x561bcd71ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.346+0000 7fabad010740 1 --2- 172.21.15.120:0/1674440409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bcd711dc0 0x561bcd71c140 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.346+0000 7fabad010740 1 --2- 172.21.15.120:0/1674440409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bcd7114a0 0x561bcd711880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.346+0000 7fabad010740 1 -- 172.21.15.120:0/1674440409 >> 172.21.15.120:0/1674440409 conn(0x561bcd70da30 msgr2=0x561bcd70de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:28.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.346+0000 7fabad010740 1 -- 172.21.15.120:0/1674440409 shutdown_connections 2024-09-19T09:22:28.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.346+0000 7fabad010740 1 -- 172.21.15.120:0/1674440409 wait complete. 2024-09-19T09:22:28.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.346+0000 7fabad010740 1 -- start start 2024-09-19T09:22:28.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.346+0000 7fabad010740 1 Processor -- start 2024-09-19T09:22:28.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.347+0000 7fabad010740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bcd7114a0 0x561bcd71bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:28.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.347+0000 7fabad010740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bcd711dc0 0x561bcd719240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:28.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.347+0000 7fabad010740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bcd71c680 0x561bcd719780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:28.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.347+0000 7fabad010740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561bcd71f250 con 0x561bcd71c680 2024-09-19T09:22:28.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.347+0000 7fabad010740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561bcd632050 con 0x561bcd711dc0 2024-09-19T09:22:28.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.347+0000 7fabad010740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561bcd71f3d0 con 0x561bcd7114a0 2024-09-19T09:22:28.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.347+0000 7fabad010740 10 client.0 _ll_register_callbacks cb 0x561bcd672160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:22:28.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.347+0000 7faba91d4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bcd71c680 0x561bcd719780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:28.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.347+0000 7faba91d4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bcd71c680 0x561bcd719780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52750/0 (socket says 172.21.15.120:52750) 2024-09-19T09:22:28.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.347+0000 7faba91d4640 1 -- 172.21.15.120:0/3112972884 learned_addr learned my addr 172.21.15.120:0/3112972884 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:28.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.347+0000 7faba89d3640 1 --2- 172.21.15.120:0/3112972884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bcd7114a0 0x561bcd71bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:28.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.347+0000 7faba3fff640 1 --2- 172.21.15.120:0/3112972884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bcd711dc0 0x561bcd719240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:28.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.347+0000 7faba91d4640 1 -- 172.21.15.120:0/3112972884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bcd7114a0 msgr2=0x561bcd71bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:28.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.347+0000 7faba91d4640 1 --2- 172.21.15.120:0/3112972884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bcd7114a0 0x561bcd71bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.347+0000 7faba91d4640 1 -- 172.21.15.120:0/3112972884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bcd711dc0 msgr2=0x561bcd719240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:28.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.347+0000 7faba91d4640 1 --2- 172.21.15.120:0/3112972884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bcd711dc0 0x561bcd719240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.347+0000 7faba91d4640 1 -- 172.21.15.120:0/3112972884 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bcd3e83c0 con 0x561bcd71c680 2024-09-19T09:22:28.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.347+0000 7faba89d3640 1 --2- 172.21.15.120:0/3112972884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bcd7114a0 0x561bcd71bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:22:28.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.347+0000 7faba91d4640 1 --2- 172.21.15.120:0/3112972884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bcd71c680 0x561bcd719780 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009900 tx=0x7fab9c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32831dc3b2225c1f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:28.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.347+0000 7fab83fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:22:28.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.347+0000 7fab83fff640 1 -- 172.21.15.120:0/3112972884 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c04e070 con 0x561bcd71c680 2024-09-19T09:22:28.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.347+0000 7fab83fff640 1 -- 172.21.15.120:0/3112972884 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fab9c0037f0 con 0x561bcd71c680 2024-09-19T09:22:28.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.347+0000 7fabad010740 1 -- 172.21.15.120:0/3112972884 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab9809fe70 con 0x561bcd71c680 2024-09-19T09:22:28.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.347+0000 7fab83fff640 1 -- 172.21.15.120:0/3112972884 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fab9c05a050 con 0x561bcd71c680 2024-09-19T09:22:28.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.348+0000 7fabad010740 10 client.21978 fetch_fsmap learned FSMap version 39 2024-09-19T09:22:28.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.348+0000 7fabad010740 1 -- 172.21.15.120:0/3112972884 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561bcd72abd0 con 0x561bcd71c680 2024-09-19T09:22:28.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.348+0000 7fab83fff640 1 -- 172.21.15.120:0/3112972884 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab9809fe70 con 0x561bcd71c680 2024-09-19T09:22:28.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.348+0000 7fab83fff640 1 -- 172.21.15.120:0/3112972884 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fab9c04ab80 con 0x561bcd71c680 2024-09-19T09:22:28.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.348+0000 7fabad010740 10 client.21978 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:22:28.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.348+0000 7fabad010740 10 client.21978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:22:28.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.348+0000 7fabad010740 20 client.21978 populate_metadata read hostname 'smithi120' 2024-09-19T09:22:28.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.348+0000 7fabad010740 1 --2- 172.21.15.120:0/3112972884 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561bcd72bfe0 0x561bcd72e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:28.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.348+0000 7fabad010740 4 client.21978 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:22:28.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.348+0000 7fabad010740 1 -- 172.21.15.120:0/3112972884 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bcd684850 con 0x561bcd72bfe0 2024-09-19T09:22:28.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.348+0000 7faba89d3640 1 --2- 172.21.15.120:0/3112972884 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561bcd72bfe0 0x561bcd72e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:28.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.349+0000 7faba89d3640 1 --2- 172.21.15.120:0/3112972884 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561bcd72bfe0 0x561bcd72e460 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e94f52da8857841d server_cookie=49889f6d2b142e88 in_seq=0 out_seq=0 2024-09-19T09:22:28.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.349+0000 7fab83fff640 10 client.21978 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:28.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.349+0000 7fab83fff640 1 -- 172.21.15.120:0/3112972884 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561bcd684850 con 0x561bcd72bfe0 2024-09-19T09:22:28.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.349+0000 7fab83fff640 10 client.21978 handle_command_reply: tid=0 2024-09-19T09:22:28.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.349+0000 7fabad010740 1 client.21978 shutdown 2024-09-19T09:22:28.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.349+0000 7fabad010740 1 -- 172.21.15.120:0/3112972884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bcd71c680 msgr2=0x561bcd719780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:28.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.349+0000 7fabad010740 1 --2- 172.21.15.120:0/3112972884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bcd71c680 0x561bcd719780 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009900 tx=0x7fab9c02a410 comp rx=0 tx=0).stop 2024-09-19T09:22:28.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.349+0000 7fabad010740 1 -- 172.21.15.120:0/3112972884 shutdown_connections 2024-09-19T09:22:28.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.349+0000 7fabad010740 1 --2- 172.21.15.120:0/3112972884 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561bcd72bfe0 0x561bcd72e460 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.350+0000 7fabad010740 1 --2- 172.21.15.120:0/3112972884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bcd71c680 0x561bcd719780 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.350+0000 7fabad010740 1 --2- 172.21.15.120:0/3112972884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bcd711dc0 0x561bcd719240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.350+0000 7fabad010740 1 --2- 172.21.15.120:0/3112972884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bcd7114a0 0x561bcd71bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.350+0000 7fab83fff640 0 client.21978 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:28.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.350+0000 7fabad010740 1 -- 172.21.15.120:0/3112972884 >> 172.21.15.120:0/3112972884 conn(0x561bcd70da30 msgr2=0x561bcd713760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:28.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.350+0000 7fabad010740 1 -- 172.21.15.120:0/3112972884 shutdown_connections 2024-09-19T09:22:28.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.350+0000 7fabad010740 1 -- 172.21.15.120:0/3112972884 wait complete. 2024-09-19T09:22:28.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.350+0000 7fabad010740 20 client.21978 trim_cache size 0 max 16384 2024-09-19T09:22:28.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.366+0000 7fabad010740 1 Processor -- start 2024-09-19T09:22:28.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.366+0000 7fabad010740 1 -- start start 2024-09-19T09:22:28.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.367+0000 7fabad010740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bcd7114a0 0x561bcd7284b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:28.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.367+0000 7fabad010740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bcd711dc0 0x561bcd714740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:28.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.367+0000 7fabad010740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bcd71c680 0x561bcd733d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:28.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.367+0000 7fabad010740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faba41ad660 con 0x561bcd711dc0 2024-09-19T09:22:28.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.367+0000 7fabad010740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faba41af1a0 con 0x561bcd7114a0 2024-09-19T09:22:28.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.367+0000 7fabad010740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faba41add30 con 0x561bcd71c680 2024-09-19T09:22:28.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.367+0000 7faba3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bcd711dc0 0x561bcd714740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:28.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.367+0000 7faba3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bcd711dc0 0x561bcd714740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52758/0 (socket says 172.21.15.120:52758) 2024-09-19T09:22:28.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.367+0000 7faba3fff640 1 -- 172.21.15.120:0/894650639 learned_addr learned my addr 172.21.15.120:0/894650639 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:28.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.367+0000 7faba89d3640 1 --2- 172.21.15.120:0/894650639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bcd7114a0 0x561bcd7284b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:28.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.367+0000 7faba3fff640 1 -- 172.21.15.120:0/894650639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bcd71c680 msgr2=0x561bcd733d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:28.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.367+0000 7faba3fff640 1 --2- 172.21.15.120:0/894650639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bcd71c680 0x561bcd733d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.367+0000 7faba3fff640 1 -- 172.21.15.120:0/894650639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bcd7114a0 msgr2=0x561bcd7284b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:28.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.367+0000 7faba3fff640 1 --2- 172.21.15.120:0/894650639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bcd7114a0 0x561bcd7284b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.367+0000 7faba3fff640 1 -- 172.21.15.120:0/894650639 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bcd70d860 con 0x561bcd711dc0 2024-09-19T09:22:28.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.367+0000 7faba3fff640 1 --2- 172.21.15.120:0/894650639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bcd711dc0 0x561bcd714740 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fab98009610 tx=0x7fab9800b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b761f1e622daa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:28.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.367+0000 7fab8a7fc640 1 -- 172.21.15.120:0/894650639 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab980b4070 con 0x561bcd711dc0 2024-09-19T09:22:28.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.367+0000 7fab8a7fc640 1 -- 172.21.15.120:0/894650639 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab980a4730 con 0x561bcd711dc0 2024-09-19T09:22:28.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.367+0000 7fab8a7fc640 1 -- 172.21.15.120:0/894650639 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab98020070 con 0x561bcd711dc0 2024-09-19T09:22:28.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.368+0000 7fabad010740 1 -- 172.21.15.120:0/894650639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bcd711dc0 msgr2=0x561bcd714740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:28.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.368+0000 7fabad010740 1 --2- 172.21.15.120:0/894650639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bcd711dc0 0x561bcd714740 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fab98009610 tx=0x7fab9800b530 comp rx=0 tx=0).stop 2024-09-19T09:22:28.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.368+0000 7fabad010740 1 -- 172.21.15.120:0/894650639 shutdown_connections 2024-09-19T09:22:28.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.368+0000 7fabad010740 1 --2- 172.21.15.120:0/894650639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bcd71c680 0x561bcd733d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.368+0000 7fabad010740 1 --2- 172.21.15.120:0/894650639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bcd711dc0 0x561bcd714740 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.368+0000 7fabad010740 1 --2- 172.21.15.120:0/894650639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bcd7114a0 0x561bcd7284b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.368+0000 7fabad010740 1 -- 172.21.15.120:0/894650639 >> 172.21.15.120:0/894650639 conn(0x561bcd70da30 msgr2=0x561bcd714030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:28.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.368+0000 7fabad010740 1 -- 172.21.15.120:0/894650639 shutdown_connections 2024-09-19T09:22:28.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.368+0000 7fabad010740 1 -- 172.21.15.120:0/894650639 wait complete. 2024-09-19T09:22:28.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.368+0000 7fabad010740 1 -- start start 2024-09-19T09:22:28.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.369+0000 7fabad010740 1 Processor -- start 2024-09-19T09:22:28.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.369+0000 7fabad010740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bcd7114a0 0x561bcd724aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:28.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.369+0000 7fabad010740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bcd711dc0 0x561bcd715cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:28.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.369+0000 7fabad010740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bcd71c680 0x561bcd716230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:28.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.369+0000 7fabad010740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faba41aead0 con 0x561bcd7114a0 2024-09-19T09:22:28.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.369+0000 7fabad010740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faba41ae400 con 0x561bcd711dc0 2024-09-19T09:22:28.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.369+0000 7fabad010740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faba41ab620 con 0x561bcd71c680 2024-09-19T09:22:28.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.369+0000 7fabad010740 10 client.0 _ll_register_callbacks cb 0x7fab9c031af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:22:28.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.369+0000 7faba89d3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bcd7114a0 0x561bcd724aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:28.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.369+0000 7faba89d3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bcd7114a0 0x561bcd724aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52772/0 (socket says 172.21.15.120:52772) 2024-09-19T09:22:28.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.369+0000 7faba89d3640 1 -- 172.21.15.120:0/3865374851 learned_addr learned my addr 172.21.15.120:0/3865374851 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:28.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.369+0000 7faba89d3640 1 -- 172.21.15.120:0/3865374851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bcd71c680 msgr2=0x561bcd716230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:28.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.369+0000 7faba89d3640 1 --2- 172.21.15.120:0/3865374851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bcd71c680 0x561bcd716230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.369+0000 7faba89d3640 1 -- 172.21.15.120:0/3865374851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bcd711dc0 msgr2=0x561bcd715cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:28.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.369+0000 7faba89d3640 1 --2- 172.21.15.120:0/3865374851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bcd711dc0 0x561bcd715cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.369+0000 7faba89d3640 1 -- 172.21.15.120:0/3865374851 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bcd7167a0 con 0x561bcd7114a0 2024-09-19T09:22:28.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.369+0000 7faba89d3640 1 --2- 172.21.15.120:0/3865374851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bcd7114a0 0x561bcd724aa0 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7fab94000f80 tx=0x7fab94013a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e1579f2cfaae33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:28.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.369+0000 7fab83fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:22:28.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.370+0000 7fab83fff640 1 -- 172.21.15.120:0/3865374851 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab94024020 con 0x561bcd7114a0 2024-09-19T09:22:28.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.370+0000 7fab83fff640 1 -- 172.21.15.120:0/3865374851 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fab94003d80 con 0x561bcd7114a0 2024-09-19T09:22:28.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.370+0000 7fabad010740 1 -- 172.21.15.120:0/3865374851 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab980a4730 con 0x561bcd7114a0 2024-09-19T09:22:28.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.370+0000 7fab83fff640 1 -- 172.21.15.120:0/3865374851 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fab94004030 con 0x561bcd7114a0 2024-09-19T09:22:28.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.370+0000 7fabad010740 10 client.21984 fetch_fsmap learned FSMap version 39 2024-09-19T09:22:28.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.370+0000 7fabad010740 1 -- 172.21.15.120:0/3865374851 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561bcd796b50 con 0x561bcd7114a0 2024-09-19T09:22:28.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.370+0000 7fab83fff640 1 -- 172.21.15.120:0/3865374851 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab980a4730 con 0x561bcd7114a0 2024-09-19T09:22:28.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.370+0000 7fab83fff640 1 -- 172.21.15.120:0/3865374851 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fab9400f040 con 0x561bcd7114a0 2024-09-19T09:22:28.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.370+0000 7fabad010740 10 client.21984 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:22:28.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.370+0000 7fabad010740 10 client.21984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:22:28.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.370+0000 7fabad010740 20 client.21984 populate_metadata read hostname 'smithi120' 2024-09-19T09:22:28.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.370+0000 7fabad010740 1 --2- 172.21.15.120:0/3865374851 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561bcd797330 0x561bcd799770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:28.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.370+0000 7fabad010740 4 client.21984 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:22:28.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.370+0000 7fabad010740 1 -- 172.21.15.120:0/3865374851 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561bcd67ed60 con 0x561bcd797330 2024-09-19T09:22:28.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.370+0000 7faba3fff640 1 --2- 172.21.15.120:0/3865374851 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561bcd797330 0x561bcd799770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:28.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.371+0000 7faba3fff640 1 --2- 172.21.15.120:0/3865374851 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561bcd797330 0x561bcd799770 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17eb0aa73065251b server_cookie=efd6b543b2d5b829 in_seq=0 out_seq=0 2024-09-19T09:22:28.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.371+0000 7fab83fff640 10 client.21984 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:28.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.371+0000 7fab83fff640 1 -- 172.21.15.120:0/3865374851 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561bcd67ed60 con 0x561bcd797330 2024-09-19T09:22:28.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.371+0000 7fab83fff640 10 client.21984 handle_command_reply: tid=0 2024-09-19T09:22:28.393 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:22:28.393 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:22:28.393 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-19T09:22:28.393 INFO:teuthology.orchestra.run.smithi120.stdout: "cf5e1f94-e903-4bae-9ad1-9e831bc75236": { 2024-09-19T09:22:28.393 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-19T09:22:28.393 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "cf5e1f94-e903-4bae-9ad1-9e831bc75236", 2024-09-19T09:22:28.393 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-19T09:22:28.393 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:22:28.393 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:22:28.394 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:22:28.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.371+0000 7fabad010740 1 client.21984 shutdown 2024-09-19T09:22:28.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.371+0000 7fabad010740 1 -- 172.21.15.120:0/3865374851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bcd7114a0 msgr2=0x561bcd724aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:28.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.371+0000 7fabad010740 1 --2- 172.21.15.120:0/3865374851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bcd7114a0 0x561bcd724aa0 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7fab94000f80 tx=0x7fab94013a50 comp rx=0 tx=0).stop 2024-09-19T09:22:28.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.371+0000 7fabad010740 1 -- 172.21.15.120:0/3865374851 shutdown_connections 2024-09-19T09:22:28.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.371+0000 7fabad010740 1 --2- 172.21.15.120:0/3865374851 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561bcd797330 0x561bcd799770 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.371+0000 7fabad010740 1 --2- 172.21.15.120:0/3865374851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bcd71c680 0x561bcd716230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.371+0000 7fabad010740 1 --2- 172.21.15.120:0/3865374851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bcd711dc0 0x561bcd715cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.371+0000 7fab83fff640 0 client.21984 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:28.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.371+0000 7fabad010740 1 --2- 172.21.15.120:0/3865374851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bcd7114a0 0x561bcd724aa0 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.371+0000 7fabad010740 1 -- 172.21.15.120:0/3865374851 >> 172.21.15.120:0/3865374851 conn(0x561bcd70da30 msgr2=0x561bcd71b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:28.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.371+0000 7fabad010740 1 -- 172.21.15.120:0/3865374851 shutdown_connections 2024-09-19T09:22:28.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.371+0000 7fabad010740 1 -- 172.21.15.120:0/3865374851 wait complete. 2024-09-19T09:22:28.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.371+0000 7fabad010740 20 client.21984 trim_cache size 0 max 16384 2024-09-19T09:22:28.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.372+0000 7fabaac37640 1 -- 172.21.15.120:0/3912044940 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fab8c0a3240 msgr2=0x7fab8c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:28.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.372+0000 7fabaac37640 1 --2- 172.21.15.120:0/3912044940 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fab8c0a3240 0x7fab8c0a5700 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0096c0 tx=0x7fab9c0317a0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.372+0000 7fabaac37640 1 -- 172.21.15.120:0/3912044940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faba4102790 msgr2=0x7faba41988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:28.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.372+0000 7fabaac37640 1 --2- 172.21.15.120:0/3912044940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faba4102790 0x7faba41988f0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fab9800d950 tx=0x7fab9800de10 comp rx=0 tx=0).stop 2024-09-19T09:22:28.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.372+0000 7fabaac37640 1 -- 172.21.15.120:0/3912044940 shutdown_connections 2024-09-19T09:22:28.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.372+0000 7fabaac37640 1 --2- 172.21.15.120:0/3912044940 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fab8c0a3240 0x7fab8c0a5700 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.372+0000 7fabaac37640 1 --2- 172.21.15.120:0/3912044940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faba4106f10 0x7faba419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.372+0000 7fabaac37640 1 --2- 172.21.15.120:0/3912044940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faba4102790 0x7faba41988f0 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.372+0000 7fabaac37640 1 --2- 172.21.15.120:0/3912044940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faba4101de0 0x7faba41983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:28.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.372+0000 7fabaac37640 1 -- 172.21.15.120:0/3912044940 >> 172.21.15.120:0/3912044940 conn(0x7faba40fd8c0 msgr2=0x7faba40fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:28.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.372+0000 7fabaac37640 1 -- 172.21.15.120:0/3912044940 shutdown_connections 2024-09-19T09:22:28.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:28.372+0000 7fabaac37640 1 -- 172.21.15.120:0/3912044940 wait complete. 2024-09-19T09:22:28.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:28 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:22:28.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:28 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:22:28.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:28 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:22:28.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:28 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:22:28.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:28 smithi196 ceph-mon[35150]: pgmap v933: 97 pgs: 97 active+clean; 391 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 13 MiB/s wr, 660 op/s 2024-09-19T09:22:28.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:28 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:22:28.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:28 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:22:28.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:28 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:22:28.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:28 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:22:28.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:28 smithi153 ceph-mon[35071]: pgmap v933: 97 pgs: 97 active+clean; 391 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 13 MiB/s wr, 660 op/s 2024-09-19T09:22:29.224 INFO:tasks.cephfs.filesystem:scrub status for tag:cf5e1f94-e903-4bae-9ad1-9e831bc75236 - {'path': '/', 'tag': 'cf5e1f94-e903-4bae-9ad1-9e831bc75236', 'options': 'recursive,force'} 2024-09-19T09:22:29.373 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 21.11 MB/sec execute 264 sec latency 8603.006 ms 2024-09-19T09:22:30.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:30 smithi120 ceph-mon[31150]: pgmap v934: 97 pgs: 97 active+clean; 387 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.2 MiB/s wr, 590 op/s 2024-09-19T09:22:30.373 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 21.03 MB/sec execute 265 sec latency 9603.124 ms 2024-09-19T09:22:30.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:30 smithi196 ceph-mon[35150]: pgmap v934: 97 pgs: 97 active+clean; 387 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.2 MiB/s wr, 590 op/s 2024-09-19T09:22:30.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:30 smithi153 ceph-mon[35071]: pgmap v934: 97 pgs: 97 active+clean; 387 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.2 MiB/s wr, 590 op/s 2024-09-19T09:22:31.374 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 20.95 MB/sec execute 266 sec latency 10603.235 ms 2024-09-19T09:22:32.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:32 smithi120 ceph-mon[31150]: pgmap v935: 97 pgs: 97 active+clean; 387 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 417 op/s 2024-09-19T09:22:32.374 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 20.87 MB/sec execute 267 sec latency 11603.339 ms 2024-09-19T09:22:32.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:32 smithi196 ceph-mon[35150]: pgmap v935: 97 pgs: 97 active+clean; 387 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 417 op/s 2024-09-19T09:22:32.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:32 smithi153 ceph-mon[35071]: pgmap v935: 97 pgs: 97 active+clean; 387 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 417 op/s 2024-09-19T09:22:33.374 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 20.79 MB/sec execute 268 sec latency 12603.420 ms 2024-09-19T09:22:33.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:33 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:22:33.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:33 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:22:33.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:33 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:22:33.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:33 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:22:33.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:33 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:22:33.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:33 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:22:34.374 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 20.71 MB/sec execute 269 sec latency 13603.506 ms 2024-09-19T09:22:34.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:34 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:22:34.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:34 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:22:34.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:34 smithi196 ceph-mon[35150]: pgmap v936: 97 pgs: 97 active+clean; 387 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 607 KiB/s wr, 286 op/s 2024-09-19T09:22:34.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:34 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:22:34.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:34 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:22:34.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:34 smithi153 ceph-mon[35071]: pgmap v936: 97 pgs: 97 active+clean; 387 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 607 KiB/s wr, 286 op/s 2024-09-19T09:22:34.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:34 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:22:34.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:34 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:22:34.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:34 smithi120 ceph-mon[31150]: pgmap v936: 97 pgs: 97 active+clean; 387 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 607 KiB/s wr, 286 op/s 2024-09-19T09:22:35.374 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 20.64 MB/sec execute 270 sec latency 14603.610 ms 2024-09-19T09:22:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.266+0000 7f53547e8640 1 -- 172.21.15.120:0/4174580187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f534c102d50 msgr2=0x7f534c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.266+0000 7f53547e8640 1 --2- 172.21.15.120:0/4174580187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f534c102d50 0x7f534c1031d0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f533c009900 tx=0x7f533c02a410 comp rx=0 tx=0).stop 2024-09-19T09:22:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.266+0000 7f53547e8640 1 -- 172.21.15.120:0/4174580187 shutdown_connections 2024-09-19T09:22:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.266+0000 7f53547e8640 1 --2- 172.21.15.120:0/4174580187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f534c103710 0x7f534c109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.266+0000 7f53547e8640 1 --2- 172.21.15.120:0/4174580187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f534c102d50 0x7f534c1031d0 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.266+0000 7f53547e8640 1 --2- 172.21.15.120:0/4174580187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f534c101b60 0x7f534c101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.266+0000 7f53547e8640 1 -- 172.21.15.120:0/4174580187 >> 172.21.15.120:0/4174580187 conn(0x7f534c0fd2e0 msgr2=0x7f534c0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.266+0000 7f53547e8640 1 -- 172.21.15.120:0/4174580187 shutdown_connections 2024-09-19T09:22:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.266+0000 7f53547e8640 1 -- 172.21.15.120:0/4174580187 wait complete. 2024-09-19T09:22:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.267+0000 7f53547e8640 1 Processor -- start 2024-09-19T09:22:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.267+0000 7f53547e8640 1 -- start start 2024-09-19T09:22:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.267+0000 7f53547e8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f534c101b60 0x7f534c197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.267+0000 7f53547e8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f534c102d50 0x7f534c198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.267+0000 7f53547e8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f534c103710 0x7f534c19f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.267+0000 7f53547e8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f534c10d7e0 con 0x7f534c102d50 2024-09-19T09:22:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.267+0000 7f53547e8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f534c10d660 con 0x7f534c103710 2024-09-19T09:22:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.267+0000 7f53547e8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f534c10d960 con 0x7f534c101b60 2024-09-19T09:22:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.267+0000 7f5351d83640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f534c102d50 0x7f534c198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.267+0000 7f5351d83640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f534c102d50 0x7f534c198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48438/0 (socket says 172.21.15.120:48438) 2024-09-19T09:22:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.267+0000 7f5351d83640 1 -- 172.21.15.120:0/2062273846 learned_addr learned my addr 172.21.15.120:0/2062273846 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.268+0000 7f5352584640 1 --2- 172.21.15.120:0/2062273846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f534c101b60 0x7f534c197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.268+0000 7f5352d85640 1 --2- 172.21.15.120:0/2062273846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f534c103710 0x7f534c19f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.268+0000 7f5351d83640 1 -- 172.21.15.120:0/2062273846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f534c101b60 msgr2=0x7f534c197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.268+0000 7f5351d83640 1 --2- 172.21.15.120:0/2062273846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f534c101b60 0x7f534c197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.268+0000 7f5351d83640 1 -- 172.21.15.120:0/2062273846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f534c103710 msgr2=0x7f534c19f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.268+0000 7f5351d83640 1 --2- 172.21.15.120:0/2062273846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f534c103710 0x7f534c19f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.268+0000 7f5351d83640 1 -- 172.21.15.120:0/2062273846 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f534c19fa00 con 0x7f534c102d50 2024-09-19T09:22:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.268+0000 7f5351d83640 1 --2- 172.21.15.120:0/2062273846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f534c102d50 0x7f534c198320 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f533c009230 tx=0x7f533c031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.268+0000 7f53437fe640 1 -- 172.21.15.120:0/2062273846 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c03f030 con 0x7f534c102d50 2024-09-19T09:22:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.268+0000 7f53547e8640 1 -- 172.21.15.120:0/2062273846 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f534c19fc90 con 0x7f534c102d50 2024-09-19T09:22:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.268+0000 7f53437fe640 1 -- 172.21.15.120:0/2062273846 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f533c003ad0 con 0x7f534c102d50 2024-09-19T09:22:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.268+0000 7f53437fe640 1 -- 172.21.15.120:0/2062273846 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c035070 con 0x7f534c102d50 2024-09-19T09:22:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.268+0000 7f53547e8640 1 -- 172.21.15.120:0/2062273846 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f534c1a0180 con 0x7f534c102d50 2024-09-19T09:22:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.271+0000 7f53437fe640 1 -- 172.21.15.120:0/2062273846 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f533c042030 con 0x7f534c102d50 2024-09-19T09:22:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.271+0000 7f5357662740 1 Processor -- start 2024-09-19T09:22:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.271+0000 7f5357662740 1 -- start start 2024-09-19T09:22:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.271+0000 7f5357662740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddab4e79a0 0x55ddab4e7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.271+0000 7f5357662740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddab4e82e0 0x55ddab4ecda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.271+0000 7f5357662740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddab4ed370 0x55ddab4ef830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.271+0000 7f5357662740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ddab44b2f0 con 0x55ddab4ed370 2024-09-19T09:22:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.271+0000 7f5357662740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ddab3f3270 con 0x55ddab4e79a0 2024-09-19T09:22:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.271+0000 7f5357662740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ddab4472b0 con 0x55ddab4e82e0 2024-09-19T09:22:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.271+0000 7f5351d83640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddab4ed370 0x55ddab4ef830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.271+0000 7f5351d83640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddab4ed370 0x55ddab4ef830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48446/0 (socket says 172.21.15.120:48446) 2024-09-19T09:22:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.271+0000 7f5351d83640 1 -- 172.21.15.120:0/3165759392 learned_addr learned my addr 172.21.15.120:0/3165759392 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.272+0000 7f5352d85640 1 --2- 172.21.15.120:0/3165759392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddab4e79a0 0x55ddab4e7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.272+0000 7f5352584640 1 --2- 172.21.15.120:0/3165759392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddab4e82e0 0x55ddab4ecda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.272+0000 7f53437fe640 1 --2- 172.21.15.120:0/2062273846 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f53380a32f0 0x7f53380a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.272+0000 7f5352d85640 1 --2- 172.21.15.120:0/2062273846 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f53380a32f0 0x7f53380a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.272+0000 7f5351d83640 1 -- 172.21.15.120:0/3165759392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddab4e82e0 msgr2=0x55ddab4ecda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.272+0000 7f5351d83640 1 --2- 172.21.15.120:0/3165759392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddab4e82e0 0x55ddab4ecda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.272+0000 7f5351d83640 1 -- 172.21.15.120:0/3165759392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddab4e79a0 msgr2=0x55ddab4e7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.272+0000 7f5351d83640 1 --2- 172.21.15.120:0/3165759392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddab4e79a0 0x55ddab4e7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.272+0000 7f5351d83640 1 -- 172.21.15.120:0/3165759392 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddab3f2340 con 0x55ddab4ed370 2024-09-19T09:22:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.272+0000 7f53437fe640 1 -- 172.21.15.120:0/2062273846 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f533c0efe20 con 0x7f534c102d50 2024-09-19T09:22:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.272+0000 7f5351d83640 1 --2- 172.21.15.120:0/3165759392 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddab4ed370 0x55ddab4ef830 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f533c0b8ec0 tx=0x7f533c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835c7007c60e8af4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.272+0000 7f5352d85640 1 --2- 172.21.15.120:0/2062273846 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f53380a32f0 0x7f53380a57b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5344005c40 tx=0x7f5344004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.272+0000 7f5340ff9640 1 -- 172.21.15.120:0/3165759392 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c049070 con 0x55ddab4ed370 2024-09-19T09:22:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.272+0000 7f5340ff9640 1 -- 172.21.15.120:0/3165759392 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f533c0bc4f0 con 0x55ddab4ed370 2024-09-19T09:22:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.272+0000 7f5340ff9640 1 -- 172.21.15.120:0/3165759392 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c03b460 con 0x55ddab4ed370 2024-09-19T09:22:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.273+0000 7f5357662740 1 -- 172.21.15.120:0/3165759392 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddab4ed370 msgr2=0x55ddab4ef830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.273+0000 7f5357662740 1 --2- 172.21.15.120:0/3165759392 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddab4ed370 0x55ddab4ef830 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f533c0b8ec0 tx=0x7f533c009680 comp rx=0 tx=0).stop 2024-09-19T09:22:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.273+0000 7f5357662740 1 -- 172.21.15.120:0/3165759392 shutdown_connections 2024-09-19T09:22:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.273+0000 7f5357662740 1 --2- 172.21.15.120:0/3165759392 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddab4ed370 0x55ddab4ef830 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.273+0000 7f5357662740 1 --2- 172.21.15.120:0/3165759392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddab4e82e0 0x55ddab4ecda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.273+0000 7f5357662740 1 --2- 172.21.15.120:0/3165759392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddab4e79a0 0x55ddab4e7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.273+0000 7f5357662740 1 -- 172.21.15.120:0/3165759392 >> 172.21.15.120:0/3165759392 conn(0x55ddab4e3010 msgr2=0x55ddab4e3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.273+0000 7f5357662740 1 -- 172.21.15.120:0/3165759392 shutdown_connections 2024-09-19T09:22:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.273+0000 7f5357662740 1 -- 172.21.15.120:0/3165759392 wait complete. 2024-09-19T09:22:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.273+0000 7f5357662740 1 -- start start 2024-09-19T09:22:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.274+0000 7f5357662740 1 Processor -- start 2024-09-19T09:22:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.274+0000 7f5357662740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddab4e79a0 0x55ddab4f18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.274+0000 7f5357662740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddab4e82e0 0x55ddab4f1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.274+0000 7f5357662740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddab4ed370 0x55ddab4f8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.274+0000 7f5357662740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ddab4f0100 con 0x55ddab4e82e0 2024-09-19T09:22:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.274+0000 7f5357662740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ddab3d4290 con 0x55ddab4e79a0 2024-09-19T09:22:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.274+0000 7f5357662740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ddab4f0280 con 0x55ddab4ed370 2024-09-19T09:22:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.274+0000 7f5357662740 10 client.0 _ll_register_callbacks cb 0x55ddab405140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:22:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.274+0000 7f5351d83640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddab4e82e0 0x55ddab4f1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.274+0000 7f5351d83640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddab4e82e0 0x55ddab4f1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48460/0 (socket says 172.21.15.120:48460) 2024-09-19T09:22:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.274+0000 7f5351d83640 1 -- 172.21.15.120:0/3925727397 learned_addr learned my addr 172.21.15.120:0/3925727397 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.274+0000 7f5352d85640 1 --2- 172.21.15.120:0/3925727397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddab4ed370 0x55ddab4f8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.274+0000 7f5352584640 1 --2- 172.21.15.120:0/3925727397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddab4e79a0 0x55ddab4f18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.274+0000 7f5351d83640 1 -- 172.21.15.120:0/3925727397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddab4ed370 msgr2=0x55ddab4f8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.274+0000 7f5351d83640 1 --2- 172.21.15.120:0/3925727397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddab4ed370 0x55ddab4f8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.274+0000 7f5351d83640 1 -- 172.21.15.120:0/3925727397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddab4e79a0 msgr2=0x55ddab4f18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.274+0000 7f5351d83640 1 --2- 172.21.15.120:0/3925727397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddab4e79a0 0x55ddab4f18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.274+0000 7f5351d83640 1 -- 172.21.15.120:0/3925727397 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddab15e210 con 0x55ddab4e82e0 2024-09-19T09:22:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.274+0000 7f5352d85640 1 --2- 172.21.15.120:0/3925727397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddab4ed370 0x55ddab4f8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:22:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.274+0000 7f5352584640 1 --2- 172.21.15.120:0/3925727397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddab4e79a0 0x55ddab4f18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:22:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.274+0000 7f5351d83640 1 --2- 172.21.15.120:0/3925727397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddab4e82e0 0x55ddab4f1e00 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f533c0c4b80 tx=0x7f533c0b8800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a627f0067ab9ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.274+0000 7f5335ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:22:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.275+0000 7f5335ffb640 1 -- 172.21.15.120:0/3925727397 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c049050 con 0x55ddab4e82e0 2024-09-19T09:22:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.275+0000 7f5335ffb640 1 -- 172.21.15.120:0/3925727397 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f533c003c80 con 0x55ddab4e82e0 2024-09-19T09:22:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.275+0000 7f5357662740 1 -- 172.21.15.120:0/3925727397 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f533c0bc4f0 con 0x55ddab4e82e0 2024-09-19T09:22:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.275+0000 7f5335ffb640 1 -- 172.21.15.120:0/3925727397 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f533c0c07f0 con 0x55ddab4e82e0 2024-09-19T09:22:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.275+0000 7f5357662740 10 client.21996 fetch_fsmap learned FSMap version 39 2024-09-19T09:22:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.275+0000 7f5357662740 1 -- 172.21.15.120:0/3925727397 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ddab22a620 con 0x55ddab4e82e0 2024-09-19T09:22:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.275+0000 7f5335ffb640 1 -- 172.21.15.120:0/3925727397 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f533c0bc4f0 con 0x55ddab4e82e0 2024-09-19T09:22:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.275+0000 7f5335ffb640 1 -- 172.21.15.120:0/3925727397 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f533c0f2070 con 0x55ddab4e82e0 2024-09-19T09:22:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.275+0000 7f5357662740 10 client.21996 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:22:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.275+0000 7f5357662740 10 client.21996 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:22:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.275+0000 7f5357662740 20 client.21996 populate_metadata read hostname 'smithi120' 2024-09-19T09:22:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.275+0000 7f5357662740 1 --2- 172.21.15.120:0/3925727397 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ddab4fd250 0x55ddab4ff640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.275+0000 7f5357662740 4 client.21996 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:22:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.275+0000 7f5357662740 1 -- 172.21.15.120:0/3925727397 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddab458a20 con 0x55ddab4fd250 2024-09-19T09:22:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.276+0000 7f5352584640 1 --2- 172.21.15.120:0/3925727397 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ddab4fd250 0x55ddab4ff640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.276+0000 7f5352584640 1 --2- 172.21.15.120:0/3925727397 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ddab4fd250 0x55ddab4ff640 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc0a80bfbaef4c2b server_cookie=fb34e7a7afd9c260 in_seq=0 out_seq=0 2024-09-19T09:22:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.276+0000 7f5335ffb640 10 client.21996 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.277+0000 7f5335ffb640 1 -- 172.21.15.120:0/3925727397 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ddab458a20 con 0x55ddab4fd250 2024-09-19T09:22:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.277+0000 7f5335ffb640 10 client.21996 handle_command_reply: tid=0 2024-09-19T09:22:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.277+0000 7f5357662740 1 client.21996 shutdown 2024-09-19T09:22:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.277+0000 7f5357662740 1 -- 172.21.15.120:0/3925727397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddab4e82e0 msgr2=0x55ddab4f1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.277+0000 7f5357662740 1 --2- 172.21.15.120:0/3925727397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddab4e82e0 0x55ddab4f1e00 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f533c0c4b80 tx=0x7f533c0b8800 comp rx=0 tx=0).stop 2024-09-19T09:22:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.277+0000 7f5357662740 1 -- 172.21.15.120:0/3925727397 shutdown_connections 2024-09-19T09:22:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.277+0000 7f5357662740 1 --2- 172.21.15.120:0/3925727397 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ddab4fd250 0x55ddab4ff640 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.277+0000 7f5357662740 1 --2- 172.21.15.120:0/3925727397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddab4ed370 0x55ddab4f8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.277+0000 7f5335ffb640 0 client.21996 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.277+0000 7f5357662740 1 --2- 172.21.15.120:0/3925727397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddab4e82e0 0x55ddab4f1e00 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.277+0000 7f5357662740 1 --2- 172.21.15.120:0/3925727397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddab4e79a0 0x55ddab4f18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.277+0000 7f5357662740 1 -- 172.21.15.120:0/3925727397 >> 172.21.15.120:0/3925727397 conn(0x55ddab4e3010 msgr2=0x55ddab4e4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.277+0000 7f5357662740 1 -- 172.21.15.120:0/3925727397 shutdown_connections 2024-09-19T09:22:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.277+0000 7f5357662740 1 -- 172.21.15.120:0/3925727397 wait complete. 2024-09-19T09:22:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.277+0000 7f5357662740 20 client.21996 trim_cache size 0 max 16384 2024-09-19T09:22:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.291+0000 7f5357662740 1 Processor -- start 2024-09-19T09:22:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.291+0000 7f5357662740 1 -- start start 2024-09-19T09:22:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.291+0000 7f5357662740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddab4e79a0 0x55ddab4e3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.291+0000 7f5357662740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddab4e82e0 0x55ddab4e4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.291+0000 7f5357662740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddab4ed370 0x55ddab4e4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.291+0000 7f5357662740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f534c1a9000 con 0x55ddab4e79a0 2024-09-19T09:22:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.291+0000 7f5357662740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f534c1aab40 con 0x55ddab4e82e0 2024-09-19T09:22:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.291+0000 7f5357662740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f534c1a96d0 con 0x55ddab4ed370 2024-09-19T09:22:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.291+0000 7f5352584640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddab4e79a0 0x55ddab4e3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.292+0000 7f5352584640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddab4e79a0 0x55ddab4e3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48474/0 (socket says 172.21.15.120:48474) 2024-09-19T09:22:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.292+0000 7f5352584640 1 -- 172.21.15.120:0/2991956156 learned_addr learned my addr 172.21.15.120:0/2991956156 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.292+0000 7f5351d83640 1 --2- 172.21.15.120:0/2991956156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddab4e82e0 0x55ddab4e4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.292+0000 7f5352584640 1 -- 172.21.15.120:0/2991956156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddab4ed370 msgr2=0x55ddab4e4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.292+0000 7f5352d85640 1 --2- 172.21.15.120:0/2991956156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddab4ed370 0x55ddab4e4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.292+0000 7f5352584640 1 --2- 172.21.15.120:0/2991956156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddab4ed370 0x55ddab4e4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.292+0000 7f5352584640 1 -- 172.21.15.120:0/2991956156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddab4e82e0 msgr2=0x55ddab4e4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.292+0000 7f5352584640 1 --2- 172.21.15.120:0/2991956156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddab4e82e0 0x55ddab4e4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.292+0000 7f5352584640 1 -- 172.21.15.120:0/2991956156 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddab4f13d0 con 0x55ddab4e79a0 2024-09-19T09:22:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.292+0000 7f5352584640 1 --2- 172.21.15.120:0/2991956156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddab4e79a0 0x55ddab4e3ec0 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f5348005fd0 tx=0x7f53480118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afad94bc1b4bf3bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.292+0000 7f5337fff640 1 -- 172.21.15.120:0/2991956156 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5348005220 con 0x55ddab4e79a0 2024-09-19T09:22:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.292+0000 7f5337fff640 1 -- 172.21.15.120:0/2991956156 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5348004c80 con 0x55ddab4e79a0 2024-09-19T09:22:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.292+0000 7f5337fff640 1 -- 172.21.15.120:0/2991956156 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5348010ae0 con 0x55ddab4e79a0 2024-09-19T09:22:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.293+0000 7f5357662740 1 -- 172.21.15.120:0/2991956156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddab4e79a0 msgr2=0x55ddab4e3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.293+0000 7f5357662740 1 --2- 172.21.15.120:0/2991956156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddab4e79a0 0x55ddab4e3ec0 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f5348005fd0 tx=0x7f53480118d0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.293+0000 7f5357662740 1 -- 172.21.15.120:0/2991956156 shutdown_connections 2024-09-19T09:22:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.293+0000 7f5357662740 1 --2- 172.21.15.120:0/2991956156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddab4ed370 0x55ddab4e4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.293+0000 7f5357662740 1 --2- 172.21.15.120:0/2991956156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddab4e82e0 0x55ddab4e4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.293+0000 7f5357662740 1 --2- 172.21.15.120:0/2991956156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddab4e79a0 0x55ddab4e3ec0 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.293+0000 7f5357662740 1 -- 172.21.15.120:0/2991956156 >> 172.21.15.120:0/2991956156 conn(0x55ddab4e3010 msgr2=0x55ddab4e9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.293+0000 7f5357662740 1 -- 172.21.15.120:0/2991956156 shutdown_connections 2024-09-19T09:22:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.293+0000 7f5357662740 1 -- 172.21.15.120:0/2991956156 wait complete. 2024-09-19T09:22:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.293+0000 7f5357662740 1 -- start start 2024-09-19T09:22:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.293+0000 7f5357662740 1 Processor -- start 2024-09-19T09:22:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.293+0000 7f5357662740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddab4e79a0 0x55ddab563170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.293+0000 7f5357662740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddab4e82e0 0x55ddab5636b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.293+0000 7f5357662740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddab4ed370 0x55ddab56a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.293+0000 7f5357662740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f534c1aa470 con 0x55ddab4e79a0 2024-09-19T09:22:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.293+0000 7f5357662740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f534c1a9da0 con 0x55ddab4e82e0 2024-09-19T09:22:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.293+0000 7f5357662740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f534c1a6fc0 con 0x55ddab4ed370 2024-09-19T09:22:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.293+0000 7f5357662740 10 client.0 _ll_register_callbacks cb 0x7f533c0dc050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:22:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.293+0000 7f5352584640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddab4e79a0 0x55ddab563170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.293+0000 7f5352584640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddab4e79a0 0x55ddab563170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48482/0 (socket says 172.21.15.120:48482) 2024-09-19T09:22:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.293+0000 7f5352584640 1 -- 172.21.15.120:0/4157817297 learned_addr learned my addr 172.21.15.120:0/4157817297 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.294+0000 7f5351d83640 1 --2- 172.21.15.120:0/4157817297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddab4e82e0 0x55ddab5636b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.294+0000 7f5352d85640 1 --2- 172.21.15.120:0/4157817297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddab4ed370 0x55ddab56a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.294+0000 7f5352584640 1 -- 172.21.15.120:0/4157817297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddab4ed370 msgr2=0x55ddab56a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.294+0000 7f5352584640 1 --2- 172.21.15.120:0/4157817297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddab4ed370 0x55ddab56a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.294+0000 7f5352584640 1 -- 172.21.15.120:0/4157817297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddab4e82e0 msgr2=0x55ddab5636b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.294+0000 7f5352584640 1 --2- 172.21.15.120:0/4157817297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddab4e82e0 0x55ddab5636b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.294+0000 7f5352584640 1 -- 172.21.15.120:0/4157817297 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddab56ad00 con 0x55ddab4e79a0 2024-09-19T09:22:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.294+0000 7f5352d85640 1 --2- 172.21.15.120:0/4157817297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddab4ed370 0x55ddab56a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:22:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.294+0000 7f5352584640 1 --2- 172.21.15.120:0/4157817297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddab4e79a0 0x55ddab563170 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f534800bbc0 tx=0x7f5348004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33f16ee18fb2fd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.294+0000 7f5335ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:22:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.294+0000 7f5335ffb640 1 -- 172.21.15.120:0/4157817297 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5348005220 con 0x55ddab4e79a0 2024-09-19T09:22:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.294+0000 7f5335ffb640 1 -- 172.21.15.120:0/4157817297 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f53480053c0 con 0x55ddab4e79a0 2024-09-19T09:22:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.294+0000 7f5357662740 1 -- 172.21.15.120:0/4157817297 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5348004c80 con 0x55ddab4e79a0 2024-09-19T09:22:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.294+0000 7f5335ffb640 1 -- 172.21.15.120:0/4157817297 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f53480055c0 con 0x55ddab4e79a0 2024-09-19T09:22:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.294+0000 7f5357662740 10 client.22002 fetch_fsmap learned FSMap version 39 2024-09-19T09:22:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.294+0000 7f5357662740 1 -- 172.21.15.120:0/4157817297 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ddab56c010 con 0x55ddab4e79a0 2024-09-19T09:22:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.294+0000 7f5335ffb640 1 -- 172.21.15.120:0/4157817297 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5348004c80 con 0x55ddab4e79a0 2024-09-19T09:22:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.295+0000 7f5335ffb640 1 -- 172.21.15.120:0/4157817297 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f534802c020 con 0x55ddab4e79a0 2024-09-19T09:22:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.295+0000 7f5357662740 10 client.22002 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:22:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.295+0000 7f5357662740 10 client.22002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:22:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.295+0000 7f5357662740 20 client.22002 populate_metadata read hostname 'smithi120' 2024-09-19T09:22:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.295+0000 7f5357662740 1 --2- 172.21.15.120:0/4157817297 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ddab56c9f0 0x55ddab56ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.295+0000 7f5357662740 4 client.22002 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:22:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.295+0000 7f5357662740 1 -- 172.21.15.120:0/4157817297 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ddab452d70 con 0x55ddab56c9f0 2024-09-19T09:22:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.295+0000 7f5351d83640 1 --2- 172.21.15.120:0/4157817297 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ddab56c9f0 0x55ddab56ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.295+0000 7f5351d83640 1 --2- 172.21.15.120:0/4157817297 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ddab56c9f0 0x55ddab56ee10 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6782f0e0ed1e7676 server_cookie=a6579aabe97b7189 in_seq=0 out_seq=0 2024-09-19T09:22:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.295+0000 7f5335ffb640 10 client.22002 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:36.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:36 smithi120 ceph-mon[31150]: pgmap v937: 97 pgs: 97 active+clean; 387 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 72 KiB/s wr, 151 op/s 2024-09-19T09:22:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.351+0000 7f5335ffb640 1 -- 172.21.15.120:0/4157817297 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ddab452d70 con 0x55ddab56c9f0 2024-09-19T09:22:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.351+0000 7f5335ffb640 10 client.22002 handle_command_reply: tid=0 2024-09-19T09:22:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.351+0000 7f5357662740 1 client.22002 shutdown 2024-09-19T09:22:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.351+0000 7f5357662740 1 -- 172.21.15.120:0/4157817297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddab4e79a0 msgr2=0x55ddab563170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.351+0000 7f5357662740 1 --2- 172.21.15.120:0/4157817297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddab4e79a0 0x55ddab563170 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f534800bbc0 tx=0x7f5348004780 comp rx=0 tx=0).stop 2024-09-19T09:22:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.351+0000 7f5357662740 1 -- 172.21.15.120:0/4157817297 shutdown_connections 2024-09-19T09:22:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.351+0000 7f5357662740 1 --2- 172.21.15.120:0/4157817297 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ddab56c9f0 0x55ddab56ee10 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.351+0000 7f5357662740 1 --2- 172.21.15.120:0/4157817297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ddab4ed370 0x55ddab56a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.351+0000 7f5357662740 1 --2- 172.21.15.120:0/4157817297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ddab4e82e0 0x55ddab5636b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.351+0000 7f5335ffb640 0 client.22002 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.351+0000 7f5357662740 1 --2- 172.21.15.120:0/4157817297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ddab4e79a0 0x55ddab563170 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.351+0000 7f5357662740 1 -- 172.21.15.120:0/4157817297 >> 172.21.15.120:0/4157817297 conn(0x55ddab4e3010 msgr2=0x55ddab4e19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.352+0000 7f5357662740 1 -- 172.21.15.120:0/4157817297 shutdown_connections 2024-09-19T09:22:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.352+0000 7f5357662740 1 -- 172.21.15.120:0/4157817297 wait complete. 2024-09-19T09:22:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.352+0000 7f5357662740 20 client.22002 trim_cache size 0 max 16384 2024-09-19T09:22:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:22:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:22:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:22:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:22:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.352+0000 7f53547e8640 1 -- 172.21.15.120:0/2062273846 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f53380a32f0 msgr2=0x7f53380a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.352+0000 7f53547e8640 1 --2- 172.21.15.120:0/2062273846 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f53380a32f0 0x7f53380a57b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5344005c40 tx=0x7f5344004ab0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.352+0000 7f53547e8640 1 -- 172.21.15.120:0/2062273846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f534c102d50 msgr2=0x7f534c198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.352+0000 7f53547e8640 1 --2- 172.21.15.120:0/2062273846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f534c102d50 0x7f534c198320 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f533c009230 tx=0x7f533c031b60 comp rx=0 tx=0).stop 2024-09-19T09:22:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.352+0000 7f53547e8640 1 -- 172.21.15.120:0/2062273846 shutdown_connections 2024-09-19T09:22:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.352+0000 7f53547e8640 1 --2- 172.21.15.120:0/2062273846 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f53380a32f0 0x7f53380a57b0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.352+0000 7f53547e8640 1 --2- 172.21.15.120:0/2062273846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f534c103710 0x7f534c19f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.352+0000 7f53547e8640 1 --2- 172.21.15.120:0/2062273846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f534c102d50 0x7f534c198320 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.352+0000 7f53547e8640 1 --2- 172.21.15.120:0/2062273846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f534c101b60 0x7f534c197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.352+0000 7f53547e8640 1 -- 172.21.15.120:0/2062273846 >> 172.21.15.120:0/2062273846 conn(0x7f534c0fd2e0 msgr2=0x7f534c106120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.352+0000 7f53547e8640 1 -- 172.21.15.120:0/2062273846 shutdown_connections 2024-09-19T09:22:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:36.352+0000 7f53547e8640 1 -- 172.21.15.120:0/2062273846 wait complete. 2024-09-19T09:22:36.375 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 20.56 MB/sec execute 271 sec latency 15603.689 ms 2024-09-19T09:22:36.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:36 smithi196 ceph-mon[35150]: pgmap v937: 97 pgs: 97 active+clean; 387 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 72 KiB/s wr, 151 op/s 2024-09-19T09:22:36.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:36 smithi153 ceph-mon[35071]: pgmap v937: 97 pgs: 97 active+clean; 387 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 72 KiB/s wr, 151 op/s 2024-09-19T09:22:37.374 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 20.49 MB/sec execute 272 sec latency 16603.748 ms 2024-09-19T09:22:38.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:38 smithi120 ceph-mon[31150]: pgmap v938: 97 pgs: 97 active+clean; 387 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 31 KiB/s wr, 132 op/s 2024-09-19T09:22:38.374 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 20.41 MB/sec execute 273 sec latency 17603.839 ms 2024-09-19T09:22:38.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:38 smithi196 ceph-mon[35150]: pgmap v938: 97 pgs: 97 active+clean; 387 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 31 KiB/s wr, 132 op/s 2024-09-19T09:22:38.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:38 smithi153 ceph-mon[35071]: pgmap v938: 97 pgs: 97 active+clean; 387 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 31 KiB/s wr, 132 op/s 2024-09-19T09:22:39.374 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 20.34 MB/sec execute 274 sec latency 18603.923 ms 2024-09-19T09:22:40.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:40 smithi120 ceph-mon[31150]: pgmap v939: 97 pgs: 97 active+clean; 387 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 17 KiB/s wr, 93 op/s 2024-09-19T09:22:40.374 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 20.26 MB/sec execute 275 sec latency 19604.017 ms 2024-09-19T09:22:40.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:40 smithi196 ceph-mon[35150]: pgmap v939: 97 pgs: 97 active+clean; 387 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 17 KiB/s wr, 93 op/s 2024-09-19T09:22:40.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:40 smithi153 ceph-mon[35071]: pgmap v939: 97 pgs: 97 active+clean; 387 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 17 KiB/s wr, 93 op/s 2024-09-19T09:22:41.375 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 20.19 MB/sec execute 276 sec latency 20604.115 ms 2024-09-19T09:22:42.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:42 smithi120 ceph-mon[31150]: pgmap v940: 97 pgs: 97 active+clean; 387 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 10 KiB/s wr, 45 op/s 2024-09-19T09:22:42.374 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 20.12 MB/sec execute 277 sec latency 21604.247 ms 2024-09-19T09:22:42.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:42 smithi196 ceph-mon[35150]: pgmap v940: 97 pgs: 97 active+clean; 387 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 10 KiB/s wr, 45 op/s 2024-09-19T09:22:42.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:42 smithi153 ceph-mon[35071]: pgmap v940: 97 pgs: 97 active+clean; 387 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 10 KiB/s wr, 45 op/s 2024-09-19T09:22:43.375 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 20.04 MB/sec execute 278 sec latency 22604.369 ms 2024-09-19T09:22:44.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:44 smithi120 ceph-mon[31150]: pgmap v941: 97 pgs: 97 active+clean; 387 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 20 op/s 2024-09-19T09:22:44.375 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 19.97 MB/sec execute 279 sec latency 23604.470 ms 2024-09-19T09:22:44.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:44 smithi196 ceph-mon[35150]: pgmap v941: 97 pgs: 97 active+clean; 387 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 20 op/s 2024-09-19T09:22:44.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:44 smithi153 ceph-mon[35071]: pgmap v941: 97 pgs: 97 active+clean; 387 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 20 op/s 2024-09-19T09:22:45.375 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 19.90 MB/sec execute 280 sec latency 24604.569 ms 2024-09-19T09:22:46.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:46 smithi120 ceph-mon[31150]: pgmap v942: 97 pgs: 97 active+clean; 387 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 18 op/s 2024-09-19T09:22:46.375 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 19.83 MB/sec execute 281 sec latency 25604.637 ms 2024-09-19T09:22:46.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:46 smithi196 ceph-mon[35150]: pgmap v942: 97 pgs: 97 active+clean; 387 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 18 op/s 2024-09-19T09:22:46.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:46 smithi153 ceph-mon[35071]: pgmap v942: 97 pgs: 97 active+clean; 387 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 18 op/s 2024-09-19T09:22:47.375 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 19.76 MB/sec execute 282 sec latency 26604.737 ms 2024-09-19T09:22:48.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:48 smithi120 ceph-mon[31150]: pgmap v943: 97 pgs: 97 active+clean; 391 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 639 KiB/s wr, 30 op/s 2024-09-19T09:22:48.375 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 19.69 MB/sec execute 283 sec latency 27604.813 ms 2024-09-19T09:22:48.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:48 smithi196 ceph-mon[35150]: pgmap v943: 97 pgs: 97 active+clean; 391 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 639 KiB/s wr, 30 op/s 2024-09-19T09:22:48.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:48 smithi153 ceph-mon[35071]: pgmap v943: 97 pgs: 97 active+clean; 391 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 639 KiB/s wr, 30 op/s 2024-09-19T09:22:49.375 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 19.62 MB/sec execute 284 sec latency 28604.859 ms 2024-09-19T09:22:49.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.440+0000 7fcf7279f640 1 -- 172.21.15.120:0/2518833221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf6c101a30 msgr2=0x7fcf6c101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:49.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.440+0000 7fcf7279f640 1 --2- 172.21.15.120:0/2518833221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf6c101a30 0x7fcf6c101eb0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c009900 tx=0x7fcf5c02a410 comp rx=0 tx=0).stop 2024-09-19T09:22:49.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.440+0000 7fcf7279f640 1 -- 172.21.15.120:0/2518833221 shutdown_connections 2024-09-19T09:22:49.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.440+0000 7fcf7279f640 1 --2- 172.21.15.120:0/2518833221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf6c1023f0 0x7fcf6c108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.440+0000 7fcf7279f640 1 --2- 172.21.15.120:0/2518833221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf6c101a30 0x7fcf6c101eb0 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.440+0000 7fcf7279f640 1 --2- 172.21.15.120:0/2518833221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf6c100840 0x7fcf6c100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.440+0000 7fcf7279f640 1 -- 172.21.15.120:0/2518833221 >> 172.21.15.120:0/2518833221 conn(0x7fcf6c0fbfa0 msgr2=0x7fcf6c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:49.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.441+0000 7fcf7279f640 1 -- 172.21.15.120:0/2518833221 shutdown_connections 2024-09-19T09:22:49.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.441+0000 7fcf7279f640 1 -- 172.21.15.120:0/2518833221 wait complete. 2024-09-19T09:22:49.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.441+0000 7fcf7279f640 1 Processor -- start 2024-09-19T09:22:49.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.441+0000 7fcf7279f640 1 -- start start 2024-09-19T09:22:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.441+0000 7fcf7279f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf6c100840 0x7fcf6c197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.441+0000 7fcf7279f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf6c101a30 0x7fcf6c198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.441+0000 7fcf7279f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf6c1023f0 0x7fcf6c19f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.441+0000 7fcf7279f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c10d610 con 0x7fcf6c1023f0 2024-09-19T09:22:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.441+0000 7fcf7279f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c10d490 con 0x7fcf6c101a30 2024-09-19T09:22:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.441+0000 7fcf7279f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c10d790 con 0x7fcf6c100840 2024-09-19T09:22:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.442+0000 7fcf71f9e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf6c1023f0 0x7fcf6c19f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.442+0000 7fcf71f9e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf6c1023f0 0x7fcf6c19f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38076/0 (socket says 172.21.15.120:38076) 2024-09-19T09:22:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.442+0000 7fcf71f9e640 1 -- 172.21.15.120:0/4194476041 learned_addr learned my addr 172.21.15.120:0/4194476041 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.442+0000 7fcf70f9c640 1 --2- 172.21.15.120:0/4194476041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf6c101a30 0x7fcf6c198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.442+0000 7fcf7179d640 1 --2- 172.21.15.120:0/4194476041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf6c100840 0x7fcf6c197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.442+0000 7fcf71f9e640 1 -- 172.21.15.120:0/4194476041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf6c100840 msgr2=0x7fcf6c197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.442+0000 7fcf71f9e640 1 --2- 172.21.15.120:0/4194476041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf6c100840 0x7fcf6c197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.442+0000 7fcf71f9e640 1 -- 172.21.15.120:0/4194476041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf6c101a30 msgr2=0x7fcf6c198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.442+0000 7fcf71f9e640 1 --2- 172.21.15.120:0/4194476041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf6c101a30 0x7fcf6c198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.442+0000 7fcf71f9e640 1 -- 172.21.15.120:0/4194476041 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf6c19f900 con 0x7fcf6c1023f0 2024-09-19T09:22:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.442+0000 7fcf70f9c640 1 --2- 172.21.15.120:0/4194476041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf6c101a30 0x7fcf6c198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:22:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.442+0000 7fcf71f9e640 1 --2- 172.21.15.120:0/4194476041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf6c1023f0 0x7fcf6c19f260 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400d750 tx=0x7fcf6400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.442+0000 7fcf627fc640 1 -- 172.21.15.120:0/4194476041 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf64017070 con 0x7fcf6c1023f0 2024-09-19T09:22:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.442+0000 7fcf627fc640 1 -- 172.21.15.120:0/4194476041 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf64005500 con 0x7fcf6c1023f0 2024-09-19T09:22:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.442+0000 7fcf7279f640 1 -- 172.21.15.120:0/4194476041 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf6c19fbf0 con 0x7fcf6c1023f0 2024-09-19T09:22:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.442+0000 7fcf627fc640 1 -- 172.21.15.120:0/4194476041 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf64004d90 con 0x7fcf6c1023f0 2024-09-19T09:22:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.442+0000 7fcf7279f640 1 -- 172.21.15.120:0/4194476041 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf6c1a00f0 con 0x7fcf6c1023f0 2024-09-19T09:22:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.445+0000 7fcf627fc640 1 -- 172.21.15.120:0/4194476041 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fcf64004f30 con 0x7fcf6c1023f0 2024-09-19T09:22:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.445+0000 7fcf76873740 1 Processor -- start 2024-09-19T09:22:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.445+0000 7fcf76873740 1 -- start start 2024-09-19T09:22:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.446+0000 7fcf76873740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612a6cc1b70 0x5612a6cc1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.446+0000 7fcf76873740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612a6cc24b0 0x5612a6cc6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.446+0000 7fcf76873740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612a6cc7540 0x5612a6cc9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.446+0000 7fcf76873740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5612a6c252f0 con 0x5612a6cc24b0 2024-09-19T09:22:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.446+0000 7fcf76873740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5612a6bcd270 con 0x5612a6cc1b70 2024-09-19T09:22:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.446+0000 7fcf76873740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5612a6c212b0 con 0x5612a6cc7540 2024-09-19T09:22:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.446+0000 7fcf7179d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612a6cc24b0 0x5612a6cc6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.446+0000 7fcf7179d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612a6cc24b0 0x5612a6cc6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38092/0 (socket says 172.21.15.120:38092) 2024-09-19T09:22:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.446+0000 7fcf7179d640 1 -- 172.21.15.120:0/60775052 learned_addr learned my addr 172.21.15.120:0/60775052 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.446+0000 7fcf70f9c640 1 --2- 172.21.15.120:0/60775052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612a6cc1b70 0x5612a6cc1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.446+0000 7fcf7179d640 1 -- 172.21.15.120:0/60775052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612a6cc7540 msgr2=0x5612a6cc9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.446+0000 7fcf7179d640 1 --2- 172.21.15.120:0/60775052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612a6cc7540 0x5612a6cc9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.446+0000 7fcf7179d640 1 -- 172.21.15.120:0/60775052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612a6cc1b70 msgr2=0x5612a6cc1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.446+0000 7fcf7179d640 1 --2- 172.21.15.120:0/60775052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612a6cc1b70 0x5612a6cc1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.446+0000 7fcf7179d640 1 -- 172.21.15.120:0/60775052 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612a6bcc340 con 0x5612a6cc24b0 2024-09-19T09:22:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.446+0000 7fcf70f9c640 1 --2- 172.21.15.120:0/60775052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612a6cc1b70 0x5612a6cc1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:22:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.446+0000 7fcf627fc640 1 --2- 172.21.15.120:0/4194476041 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcf580a3240 0x7fcf580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.446+0000 7fcf7179d640 1 --2- 172.21.15.120:0/60775052 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612a6cc24b0 0x5612a6cc6f70 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7fcf68009fd0 tx=0x7fcf680118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e1deb4d442df5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.446+0000 7fcf70f9c640 1 --2- 172.21.15.120:0/4194476041 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcf580a3240 0x7fcf580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.446+0000 7fcf627fc640 1 -- 172.21.15.120:0/4194476041 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fcf640cf310 con 0x7fcf6c1023f0 2024-09-19T09:22:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.446+0000 7fcf577fe640 1 -- 172.21.15.120:0/60775052 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68003970 con 0x5612a6cc24b0 2024-09-19T09:22:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.446+0000 7fcf577fe640 1 -- 172.21.15.120:0/60775052 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf68004020 con 0x5612a6cc24b0 2024-09-19T09:22:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.446+0000 7fcf577fe640 1 -- 172.21.15.120:0/60775052 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6800f910 con 0x5612a6cc24b0 2024-09-19T09:22:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.447+0000 7fcf76873740 1 -- 172.21.15.120:0/60775052 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612a6cc24b0 msgr2=0x5612a6cc6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.447+0000 7fcf76873740 1 --2- 172.21.15.120:0/60775052 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612a6cc24b0 0x5612a6cc6f70 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7fcf68009fd0 tx=0x7fcf680118d0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.447+0000 7fcf70f9c640 1 --2- 172.21.15.120:0/4194476041 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcf580a3240 0x7fcf580a5700 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c0314e0 tx=0x7fcf5c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.447+0000 7fcf76873740 1 -- 172.21.15.120:0/60775052 shutdown_connections 2024-09-19T09:22:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.447+0000 7fcf76873740 1 --2- 172.21.15.120:0/60775052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612a6cc7540 0x5612a6cc9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.447+0000 7fcf76873740 1 --2- 172.21.15.120:0/60775052 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612a6cc24b0 0x5612a6cc6f70 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.447+0000 7fcf76873740 1 --2- 172.21.15.120:0/60775052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612a6cc1b70 0x5612a6cc1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.447+0000 7fcf76873740 1 -- 172.21.15.120:0/60775052 >> 172.21.15.120:0/60775052 conn(0x5612a6cbd1e0 msgr2=0x5612a6cbd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.447+0000 7fcf76873740 1 -- 172.21.15.120:0/60775052 shutdown_connections 2024-09-19T09:22:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.447+0000 7fcf76873740 1 -- 172.21.15.120:0/60775052 wait complete. 2024-09-19T09:22:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.447+0000 7fcf76873740 1 -- start start 2024-09-19T09:22:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.448+0000 7fcf76873740 1 Processor -- start 2024-09-19T09:22:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.448+0000 7fcf76873740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612a6cc1b70 0x5612a6ccbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.448+0000 7fcf76873740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612a6cc24b0 0x5612a6ccc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.448+0000 7fcf76873740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612a6cc7540 0x5612a6cd32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.448+0000 7fcf76873740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5612a6cca510 con 0x5612a6cc7540 2024-09-19T09:22:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.448+0000 7fcf76873740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5612a6bae290 con 0x5612a6cc24b0 2024-09-19T09:22:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.448+0000 7fcf76873740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5612a6cca690 con 0x5612a6cc1b70 2024-09-19T09:22:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.448+0000 7fcf76873740 10 client.0 _ll_register_callbacks cb 0x5612a6bdf140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:22:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.448+0000 7fcf71f9e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612a6cc7540 0x5612a6cd32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.448+0000 7fcf71f9e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612a6cc7540 0x5612a6cd32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38106/0 (socket says 172.21.15.120:38106) 2024-09-19T09:22:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.448+0000 7fcf71f9e640 1 -- 172.21.15.120:0/460094003 learned_addr learned my addr 172.21.15.120:0/460094003 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.448+0000 7fcf71f9e640 1 -- 172.21.15.120:0/460094003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612a6cc1b70 msgr2=0x5612a6ccbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.448+0000 7fcf71f9e640 1 --2- 172.21.15.120:0/460094003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612a6cc1b70 0x5612a6ccbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.448+0000 7fcf71f9e640 1 -- 172.21.15.120:0/460094003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612a6cc24b0 msgr2=0x5612a6ccc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.448+0000 7fcf71f9e640 1 --2- 172.21.15.120:0/460094003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612a6cc24b0 0x5612a6ccc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.448+0000 7fcf71f9e640 1 -- 172.21.15.120:0/460094003 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612a6938210 con 0x5612a6cc7540 2024-09-19T09:22:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.448+0000 7fcf71f9e640 1 --2- 172.21.15.120:0/460094003 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612a6cc7540 0x5612a6cd32b0 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009c10 tx=0x7fcf64099be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0fc23f56f4f3e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.448+0000 7fcf54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:22:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.449+0000 7fcf54ff9640 1 -- 172.21.15.120:0/460094003 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf64020070 con 0x5612a6cc7540 2024-09-19T09:22:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.449+0000 7fcf54ff9640 1 -- 172.21.15.120:0/460094003 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fcf64005220 con 0x5612a6cc7540 2024-09-19T09:22:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.449+0000 7fcf76873740 1 -- 172.21.15.120:0/460094003 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf68004020 con 0x5612a6cc7540 2024-09-19T09:22:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.449+0000 7fcf54ff9640 1 -- 172.21.15.120:0/460094003 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcf6409fce0 con 0x5612a6cc7540 2024-09-19T09:22:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.449+0000 7fcf76873740 10 client.22014 fetch_fsmap learned FSMap version 39 2024-09-19T09:22:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.449+0000 7fcf76873740 1 -- 172.21.15.120:0/460094003 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5612a6a04620 con 0x5612a6cc7540 2024-09-19T09:22:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.449+0000 7fcf54ff9640 1 -- 172.21.15.120:0/460094003 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf68004020 con 0x5612a6cc7540 2024-09-19T09:22:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.449+0000 7fcf54ff9640 1 -- 172.21.15.120:0/460094003 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fcf6409ae30 con 0x5612a6cc7540 2024-09-19T09:22:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.449+0000 7fcf76873740 10 client.22014 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:22:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.449+0000 7fcf76873740 10 client.22014 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:22:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.449+0000 7fcf76873740 20 client.22014 populate_metadata read hostname 'smithi120' 2024-09-19T09:22:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.449+0000 7fcf76873740 1 --2- 172.21.15.120:0/460094003 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5612a6cd7a60 0x5612a6cd9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.449+0000 7fcf76873740 4 client.22014 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:22:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.449+0000 7fcf76873740 1 -- 172.21.15.120:0/460094003 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612a6c32a20 con 0x5612a6cd7a60 2024-09-19T09:22:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.450+0000 7fcf7179d640 1 --2- 172.21.15.120:0/460094003 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5612a6cd7a60 0x5612a6cd9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.450+0000 7fcf7179d640 1 --2- 172.21.15.120:0/460094003 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5612a6cd7a60 0x5612a6cd9f00 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bcd1346fa2545b1 server_cookie=6db0c510a9f22107 in_seq=0 out_seq=0 2024-09-19T09:22:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.450+0000 7fcf54ff9640 10 client.22014 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.450+0000 7fcf54ff9640 1 -- 172.21.15.120:0/460094003 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5612a6c32a20 con 0x5612a6cd7a60 2024-09-19T09:22:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.450+0000 7fcf54ff9640 10 client.22014 handle_command_reply: tid=0 2024-09-19T09:22:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.451+0000 7fcf76873740 1 client.22014 shutdown 2024-09-19T09:22:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.451+0000 7fcf76873740 1 -- 172.21.15.120:0/460094003 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612a6cc7540 msgr2=0x5612a6cd32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.451+0000 7fcf76873740 1 --2- 172.21.15.120:0/460094003 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612a6cc7540 0x5612a6cd32b0 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009c10 tx=0x7fcf64099be0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.451+0000 7fcf76873740 1 -- 172.21.15.120:0/460094003 shutdown_connections 2024-09-19T09:22:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.451+0000 7fcf76873740 1 --2- 172.21.15.120:0/460094003 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5612a6cd7a60 0x5612a6cd9f00 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.451+0000 7fcf76873740 1 --2- 172.21.15.120:0/460094003 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612a6cc7540 0x5612a6cd32b0 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.451+0000 7fcf54ff9640 0 client.22014 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.451+0000 7fcf76873740 1 --2- 172.21.15.120:0/460094003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612a6cc24b0 0x5612a6ccc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.451+0000 7fcf76873740 1 --2- 172.21.15.120:0/460094003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612a6cc1b70 0x5612a6ccbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.451+0000 7fcf76873740 1 -- 172.21.15.120:0/460094003 >> 172.21.15.120:0/460094003 conn(0x5612a6cbd1e0 msgr2=0x5612a6cbe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.451+0000 7fcf76873740 1 -- 172.21.15.120:0/460094003 shutdown_connections 2024-09-19T09:22:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.451+0000 7fcf76873740 1 -- 172.21.15.120:0/460094003 wait complete. 2024-09-19T09:22:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.451+0000 7fcf76873740 20 client.22014 trim_cache size 0 max 16384 2024-09-19T09:22:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.465+0000 7fcf76873740 1 Processor -- start 2024-09-19T09:22:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.465+0000 7fcf76873740 1 -- start start 2024-09-19T09:22:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.465+0000 7fcf76873740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612a6cc1b70 0x5612a6cbba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.465+0000 7fcf76873740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612a6cc24b0 0x5612a6cd3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.465+0000 7fcf76873740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612a6cc7540 0x5612a6cbbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.465+0000 7fcf76873740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c1a8ec0 con 0x5612a6cc24b0 2024-09-19T09:22:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.465+0000 7fcf76873740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c1aaa00 con 0x5612a6cc1b70 2024-09-19T09:22:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.465+0000 7fcf76873740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c1a9590 con 0x5612a6cc7540 2024-09-19T09:22:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.465+0000 7fcf70f9c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612a6cc24b0 0x5612a6cd3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.465+0000 7fcf70f9c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612a6cc24b0 0x5612a6cd3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38110/0 (socket says 172.21.15.120:38110) 2024-09-19T09:22:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.465+0000 7fcf70f9c640 1 -- 172.21.15.120:0/3481526824 learned_addr learned my addr 172.21.15.120:0/3481526824 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.465+0000 7fcf7179d640 1 --2- 172.21.15.120:0/3481526824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612a6cc1b70 0x5612a6cbba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.465+0000 7fcf70f9c640 1 -- 172.21.15.120:0/3481526824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612a6cc7540 msgr2=0x5612a6cbbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.465+0000 7fcf70f9c640 1 --2- 172.21.15.120:0/3481526824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612a6cc7540 0x5612a6cbbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.465+0000 7fcf70f9c640 1 -- 172.21.15.120:0/3481526824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612a6cc1b70 msgr2=0x5612a6cbba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.465+0000 7fcf70f9c640 1 --2- 172.21.15.120:0/3481526824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612a6cc1b70 0x5612a6cbba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.465+0000 7fcf70f9c640 1 -- 172.21.15.120:0/3481526824 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612a6cc72a0 con 0x5612a6cc24b0 2024-09-19T09:22:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.465+0000 7fcf7179d640 1 --2- 172.21.15.120:0/3481526824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612a6cc1b70 0x5612a6cbba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:22:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.466+0000 7fcf70f9c640 1 --2- 172.21.15.120:0/3481526824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612a6cc24b0 0x5612a6cd3270 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c031610 tx=0x7fcf5c03f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4973259d27c0692 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.466+0000 7fcf56ffd640 1 -- 172.21.15.120:0/3481526824 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c04d070 con 0x5612a6cc24b0 2024-09-19T09:22:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.466+0000 7fcf56ffd640 1 -- 172.21.15.120:0/3481526824 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf5c004760 con 0x5612a6cc24b0 2024-09-19T09:22:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.466+0000 7fcf56ffd640 1 -- 172.21.15.120:0/3481526824 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c035040 con 0x5612a6cc24b0 2024-09-19T09:22:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.466+0000 7fcf76873740 1 -- 172.21.15.120:0/3481526824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612a6cc24b0 msgr2=0x5612a6cd3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.466+0000 7fcf76873740 1 --2- 172.21.15.120:0/3481526824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612a6cc24b0 0x5612a6cd3270 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c031610 tx=0x7fcf5c03f070 comp rx=0 tx=0).stop 2024-09-19T09:22:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.466+0000 7fcf76873740 1 -- 172.21.15.120:0/3481526824 shutdown_connections 2024-09-19T09:22:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.466+0000 7fcf76873740 1 --2- 172.21.15.120:0/3481526824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612a6cc7540 0x5612a6cbbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.466+0000 7fcf76873740 1 --2- 172.21.15.120:0/3481526824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612a6cc24b0 0x5612a6cd3270 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.466+0000 7fcf76873740 1 --2- 172.21.15.120:0/3481526824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612a6cc1b70 0x5612a6cbba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.466+0000 7fcf76873740 1 -- 172.21.15.120:0/3481526824 >> 172.21.15.120:0/3481526824 conn(0x5612a6cbd1e0 msgr2=0x5612a6ccbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.466+0000 7fcf76873740 1 -- 172.21.15.120:0/3481526824 shutdown_connections 2024-09-19T09:22:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.466+0000 7fcf76873740 1 -- 172.21.15.120:0/3481526824 wait complete. 2024-09-19T09:22:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.466+0000 7fcf76873740 1 -- start start 2024-09-19T09:22:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.467+0000 7fcf76873740 1 Processor -- start 2024-09-19T09:22:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.467+0000 7fcf76873740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612a6cc1b70 0x5612a6cd08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.467+0000 7fcf76873740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612a6cc24b0 0x5612a6cd0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.467+0000 7fcf76873740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612a6cc7540 0x5612a6cd1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.467+0000 7fcf76873740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c1aa330 con 0x5612a6cc1b70 2024-09-19T09:22:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.467+0000 7fcf76873740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c1a9c60 con 0x5612a6cc7540 2024-09-19T09:22:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.467+0000 7fcf76873740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c1a6e80 con 0x5612a6cc24b0 2024-09-19T09:22:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.467+0000 7fcf76873740 10 client.0 _ll_register_callbacks cb 0x7fcf64009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:22:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.467+0000 7fcf7179d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612a6cc1b70 0x5612a6cd08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.467+0000 7fcf7179d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612a6cc1b70 0x5612a6cd08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38112/0 (socket says 172.21.15.120:38112) 2024-09-19T09:22:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.467+0000 7fcf7179d640 1 -- 172.21.15.120:0/860156918 learned_addr learned my addr 172.21.15.120:0/860156918 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.467+0000 7fcf71f9e640 1 --2- 172.21.15.120:0/860156918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612a6cc7540 0x5612a6cd1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.467+0000 7fcf7179d640 1 -- 172.21.15.120:0/860156918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612a6cc24b0 msgr2=0x5612a6cd0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.467+0000 7fcf7179d640 1 --2- 172.21.15.120:0/860156918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612a6cc24b0 0x5612a6cd0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.467+0000 7fcf7179d640 1 -- 172.21.15.120:0/860156918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612a6cc7540 msgr2=0x5612a6cd1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.467+0000 7fcf7179d640 1 --2- 172.21.15.120:0/860156918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612a6cc7540 0x5612a6cd1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.467+0000 7fcf7179d640 1 -- 172.21.15.120:0/860156918 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612a6cd2370 con 0x5612a6cc1b70 2024-09-19T09:22:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.467+0000 7fcf7179d640 1 --2- 172.21.15.120:0/860156918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612a6cc1b70 0x5612a6cd08b0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7fcf68009d60 tx=0x7fcf68009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21fbac741bffdbb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.467+0000 7fcf54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:22:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.468+0000 7fcf54ff9640 1 -- 172.21.15.120:0/860156918 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68033060 con 0x5612a6cc1b70 2024-09-19T09:22:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.468+0000 7fcf54ff9640 1 -- 172.21.15.120:0/860156918 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fcf6802a040 con 0x5612a6cc1b70 2024-09-19T09:22:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.468+0000 7fcf76873740 1 -- 172.21.15.120:0/860156918 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf5c004760 con 0x5612a6cc1b70 2024-09-19T09:22:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.468+0000 7fcf54ff9640 1 -- 172.21.15.120:0/860156918 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcf68004a00 con 0x5612a6cc1b70 2024-09-19T09:22:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.468+0000 7fcf76873740 10 client.22020 fetch_fsmap learned FSMap version 39 2024-09-19T09:22:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.468+0000 7fcf76873740 1 -- 172.21.15.120:0/860156918 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5612a6d427e0 con 0x5612a6cc1b70 2024-09-19T09:22:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.468+0000 7fcf54ff9640 1 -- 172.21.15.120:0/860156918 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf5c004760 con 0x5612a6cc1b70 2024-09-19T09:22:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.468+0000 7fcf54ff9640 1 -- 172.21.15.120:0/860156918 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fcf68038070 con 0x5612a6cc1b70 2024-09-19T09:22:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.468+0000 7fcf76873740 10 client.22020 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:22:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.468+0000 7fcf76873740 10 client.22020 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:22:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.468+0000 7fcf76873740 20 client.22020 populate_metadata read hostname 'smithi120' 2024-09-19T09:22:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.468+0000 7fcf76873740 1 --2- 172.21.15.120:0/860156918 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5612a6d42ee0 0x5612a6d45300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.468+0000 7fcf76873740 4 client.22020 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:22:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.468+0000 7fcf76873740 1 -- 172.21.15.120:0/860156918 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5612a6c2cd70 con 0x5612a6d42ee0 2024-09-19T09:22:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.468+0000 7fcf70f9c640 1 --2- 172.21.15.120:0/860156918 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5612a6d42ee0 0x5612a6d45300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.469+0000 7fcf54ff9640 10 client.22020 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.469+0000 7fcf70f9c640 1 --2- 172.21.15.120:0/860156918 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5612a6d42ee0 0x5612a6d45300 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8cfb8cc579eaa3 server_cookie=12b6cbc2731bc292 in_seq=0 out_seq=0 2024-09-19T09:22:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.567+0000 7fcf54ff9640 1 -- 172.21.15.120:0/860156918 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5612a6c2cd70 con 0x5612a6d42ee0 2024-09-19T09:22:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.567+0000 7fcf54ff9640 10 client.22020 handle_command_reply: tid=0 2024-09-19T09:22:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.567+0000 7fcf76873740 1 client.22020 shutdown 2024-09-19T09:22:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.567+0000 7fcf76873740 1 -- 172.21.15.120:0/860156918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612a6cc1b70 msgr2=0x5612a6cd08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.567+0000 7fcf76873740 1 --2- 172.21.15.120:0/860156918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612a6cc1b70 0x5612a6cd08b0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7fcf68009d60 tx=0x7fcf68009290 comp rx=0 tx=0).stop 2024-09-19T09:22:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.567+0000 7fcf76873740 1 -- 172.21.15.120:0/860156918 shutdown_connections 2024-09-19T09:22:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.567+0000 7fcf76873740 1 --2- 172.21.15.120:0/860156918 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5612a6d42ee0 0x5612a6d45300 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.567+0000 7fcf76873740 1 --2- 172.21.15.120:0/860156918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612a6cc7540 0x5612a6cd1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.567+0000 7fcf76873740 1 --2- 172.21.15.120:0/860156918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612a6cc24b0 0x5612a6cd0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.567+0000 7fcf76873740 1 --2- 172.21.15.120:0/860156918 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612a6cc1b70 0x5612a6cd08b0 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.567+0000 7fcf54ff9640 0 client.22020 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.567+0000 7fcf76873740 1 -- 172.21.15.120:0/860156918 >> 172.21.15.120:0/860156918 conn(0x5612a6cbd1e0 msgr2=0x5612a6cbe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:49.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.567+0000 7fcf76873740 1 -- 172.21.15.120:0/860156918 shutdown_connections 2024-09-19T09:22:49.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.567+0000 7fcf76873740 1 -- 172.21.15.120:0/860156918 wait complete. 2024-09-19T09:22:49.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.567+0000 7fcf76873740 20 client.22020 trim_cache size 0 max 16384 2024-09-19T09:22:49.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:22:49.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:22:49.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:22:49.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:22:49.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.568+0000 7fcf7279f640 1 -- 172.21.15.120:0/4194476041 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcf580a3240 msgr2=0x7fcf580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:49.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.568+0000 7fcf7279f640 1 --2- 172.21.15.120:0/4194476041 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcf580a3240 0x7fcf580a5700 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c0314e0 tx=0x7fcf5c000bc0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.568+0000 7fcf7279f640 1 -- 172.21.15.120:0/4194476041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf6c1023f0 msgr2=0x7fcf6c19f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:49.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.568+0000 7fcf7279f640 1 --2- 172.21.15.120:0/4194476041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf6c1023f0 0x7fcf6c19f260 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400d750 tx=0x7fcf6400dc10 comp rx=0 tx=0).stop 2024-09-19T09:22:49.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.568+0000 7fcf7279f640 1 -- 172.21.15.120:0/4194476041 shutdown_connections 2024-09-19T09:22:49.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.568+0000 7fcf7279f640 1 --2- 172.21.15.120:0/4194476041 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fcf580a3240 0x7fcf580a5700 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.568+0000 7fcf7279f640 1 --2- 172.21.15.120:0/4194476041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf6c1023f0 0x7fcf6c19f260 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.568+0000 7fcf7279f640 1 --2- 172.21.15.120:0/4194476041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf6c101a30 0x7fcf6c198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.568+0000 7fcf7279f640 1 --2- 172.21.15.120:0/4194476041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf6c100840 0x7fcf6c197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:49.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.568+0000 7fcf7279f640 1 -- 172.21.15.120:0/4194476041 >> 172.21.15.120:0/4194476041 conn(0x7fcf6c0fbfa0 msgr2=0x7fcf6c104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:49.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.568+0000 7fcf7279f640 1 -- 172.21.15.120:0/4194476041 shutdown_connections 2024-09-19T09:22:49.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:22:49.568+0000 7fcf7279f640 1 -- 172.21.15.120:0/4194476041 wait complete. 2024-09-19T09:22:49.998 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:22:50.246 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:22:50.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:50 smithi120 ceph-mon[31150]: pgmap v944: 97 pgs: 97 active+clean; 396 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 1.5 MiB/s wr, 56 op/s 2024-09-19T09:22:50.375 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 19.55 MB/sec execute 285 sec latency 29604.929 ms 2024-09-19T09:22:50.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:50 smithi196 ceph-mon[35150]: pgmap v944: 97 pgs: 97 active+clean; 396 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 1.5 MiB/s wr, 56 op/s 2024-09-19T09:22:50.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:50 smithi153 ceph-mon[35071]: pgmap v944: 97 pgs: 97 active+clean; 396 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 1.5 MiB/s wr, 56 op/s 2024-09-19T09:22:50.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.730+0000 7fbb4e735640 1 -- 172.21.15.120:0/3573524949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb48102570 msgr2=0x7fbb481029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:50.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.730+0000 7fbb4e735640 1 --2- 172.21.15.120:0/3573524949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb48102570 0x7fbb481029b0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7fbb30009900 tx=0x7fbb3002a480 comp rx=0 tx=0).stop 2024-09-19T09:22:50.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.730+0000 7fbb4e735640 1 -- 172.21.15.120:0/3573524949 shutdown_connections 2024-09-19T09:22:50.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.730+0000 7fbb4e735640 1 --2- 172.21.15.120:0/3573524949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb48102ef0 0x7fbb4810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:50.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.730+0000 7fbb4e735640 1 --2- 172.21.15.120:0/3573524949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb48102570 0x7fbb481029b0 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:50.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.730+0000 7fbb4e735640 1 --2- 172.21.15.120:0/3573524949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb48100e20 0x7fbb48101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:50.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.730+0000 7fbb4e735640 1 -- 172.21.15.120:0/3573524949 >> 172.21.15.120:0/3573524949 conn(0x7fbb480fc7d0 msgr2=0x7fbb480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:50.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.730+0000 7fbb4e735640 1 -- 172.21.15.120:0/3573524949 shutdown_connections 2024-09-19T09:22:50.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.730+0000 7fbb4e735640 1 -- 172.21.15.120:0/3573524949 wait complete. 2024-09-19T09:22:50.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.731+0000 7fbb4e735640 1 Processor -- start 2024-09-19T09:22:50.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.731+0000 7fbb4e735640 1 -- start start 2024-09-19T09:22:50.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.731+0000 7fbb4e735640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb48100e20 0x7fbb48197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:50.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.731+0000 7fbb4e735640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb48102570 0x7fbb48197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:50.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.731+0000 7fbb4e735640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb48102ef0 0x7fbb4819bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:50.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.731+0000 7fbb4e735640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbb48110fc0 con 0x7fbb48100e20 2024-09-19T09:22:50.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.731+0000 7fbb4e735640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbb48110e40 con 0x7fbb48102570 2024-09-19T09:22:50.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.731+0000 7fbb4e735640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbb48111140 con 0x7fbb48102ef0 2024-09-19T09:22:50.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.731+0000 7fbb47fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb48100e20 0x7fbb48197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:50.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.731+0000 7fbb47fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb48100e20 0x7fbb48197440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38138/0 (socket says 172.21.15.120:38138) 2024-09-19T09:22:50.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.731+0000 7fbb47fff640 1 -- 172.21.15.120:0/3041464135 learned_addr learned my addr 172.21.15.120:0/3041464135 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:50.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.732+0000 7fbb4ccd2640 1 --2- 172.21.15.120:0/3041464135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb48102ef0 0x7fbb4819bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:50.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.732+0000 7fbb3ffff640 1 --2- 172.21.15.120:0/3041464135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb48102570 0x7fbb48197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:50.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.732+0000 7fbb47fff640 1 -- 172.21.15.120:0/3041464135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb48102ef0 msgr2=0x7fbb4819bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:50.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.732+0000 7fbb47fff640 1 --2- 172.21.15.120:0/3041464135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb48102ef0 0x7fbb4819bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:50.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.732+0000 7fbb47fff640 1 -- 172.21.15.120:0/3041464135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb48102570 msgr2=0x7fbb48197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:50.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.732+0000 7fbb47fff640 1 --2- 172.21.15.120:0/3041464135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb48102570 0x7fbb48197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:50.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.732+0000 7fbb47fff640 1 -- 172.21.15.120:0/3041464135 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb4819c3f0 con 0x7fbb48100e20 2024-09-19T09:22:50.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.732+0000 7fbb3ffff640 1 --2- 172.21.15.120:0/3041464135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb48102570 0x7fbb48197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:22:50.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.732+0000 7fbb47fff640 1 --2- 172.21.15.120:0/3041464135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb48100e20 0x7fbb48197440 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800c970 tx=0x7fbb3800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:50.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.732+0000 7fbb45ffb640 1 -- 172.21.15.120:0/3041464135 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb38007bb0 con 0x7fbb48100e20 2024-09-19T09:22:50.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.732+0000 7fbb45ffb640 1 -- 172.21.15.120:0/3041464135 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb38007d50 con 0x7fbb48100e20 2024-09-19T09:22:50.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.732+0000 7fbb4e735640 1 -- 172.21.15.120:0/3041464135 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb4819c6e0 con 0x7fbb48100e20 2024-09-19T09:22:50.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.732+0000 7fbb45ffb640 1 -- 172.21.15.120:0/3041464135 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3800f450 con 0x7fbb48100e20 2024-09-19T09:22:50.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.732+0000 7fbb4e735640 1 -- 172.21.15.120:0/3041464135 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb481a3fc0 con 0x7fbb48100e20 2024-09-19T09:22:50.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.733+0000 7fbb4e735640 1 -- 172.21.15.120:0/3041464135 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb48101200 con 0x7fbb48100e20 2024-09-19T09:22:50.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.734+0000 7fbb45ffb640 1 -- 172.21.15.120:0/3041464135 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fbb38004fc0 con 0x7fbb48100e20 2024-09-19T09:22:50.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.735+0000 7fbb45ffb640 1 --2- 172.21.15.120:0/3041464135 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbb340a3240 0x7fbb340a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:50.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.736+0000 7fbb3ffff640 1 --2- 172.21.15.120:0/3041464135 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbb340a3240 0x7fbb340a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:50.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.736+0000 7fbb45ffb640 1 -- 172.21.15.120:0/3041464135 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fbb380cfb20 con 0x7fbb48100e20 2024-09-19T09:22:50.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.736+0000 7fbb3ffff640 1 --2- 172.21.15.120:0/3041464135 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbb340a3240 0x7fbb340a5700 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fbb480ffdc0 tx=0x7fbb30005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:50.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.739+0000 7fbb45ffb640 1 -- 172.21.15.120:0/3041464135 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbb38014030 con 0x7fbb48100e20 2024-09-19T09:22:50.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.938+0000 7fbb4e735640 1 -- 172.21.15.120:0/3041464135 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb481987a0 con 0x7fbb48100e20 2024-09-19T09:22:50.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.938+0000 7fbb45ffb640 1 -- 172.21.15.120:0/3041464135 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11988 (secure 0 0 0) 0x7fbb481987a0 con 0x7fbb48100e20 2024-09-19T09:22:50.940 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:22:50.940 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":39,"btime":"2024-09-19T09:18:53:120858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:18:53.070639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:22:50.940 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 39 2024-09-19T09:22:50.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.940+0000 7fbb4e735640 1 -- 172.21.15.120:0/3041464135 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbb340a3240 msgr2=0x7fbb340a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:50.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.940+0000 7fbb4e735640 1 --2- 172.21.15.120:0/3041464135 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbb340a3240 0x7fbb340a5700 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fbb480ffdc0 tx=0x7fbb30005a20 comp rx=0 tx=0).stop 2024-09-19T09:22:50.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.940+0000 7fbb4e735640 1 -- 172.21.15.120:0/3041464135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb48100e20 msgr2=0x7fbb48197440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:50.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.940+0000 7fbb4e735640 1 --2- 172.21.15.120:0/3041464135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb48100e20 0x7fbb48197440 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800c970 tx=0x7fbb3800ce30 comp rx=0 tx=0).stop 2024-09-19T09:22:50.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.940+0000 7fbb4e735640 1 -- 172.21.15.120:0/3041464135 shutdown_connections 2024-09-19T09:22:50.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.940+0000 7fbb4e735640 1 --2- 172.21.15.120:0/3041464135 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbb340a3240 0x7fbb340a5700 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:50.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.941+0000 7fbb4e735640 1 --2- 172.21.15.120:0/3041464135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb48102ef0 0x7fbb4819bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:50.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.941+0000 7fbb4e735640 1 --2- 172.21.15.120:0/3041464135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb48102570 0x7fbb48197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:50.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.941+0000 7fbb4e735640 1 --2- 172.21.15.120:0/3041464135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb48100e20 0x7fbb48197440 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:50.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.941+0000 7fbb4e735640 1 -- 172.21.15.120:0/3041464135 >> 172.21.15.120:0/3041464135 conn(0x7fbb480fc7d0 msgr2=0x7fbb480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:50.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.941+0000 7fbb4e735640 1 -- 172.21.15.120:0/3041464135 shutdown_connections 2024-09-19T09:22:50.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:50.941+0000 7fbb4e735640 1 -- 172.21.15.120:0/3041464135 wait complete. 2024-09-19T09:22:51.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:51 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3041464135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:22:51.375 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 19.48 MB/sec execute 286 sec latency 30604.999 ms 2024-09-19T09:22:51.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:51 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3041464135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:22:51.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:51 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3041464135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:22:51.881 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db --set-id bb3ccada --release --await' 2024-09-19T09:22:51.881 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db --set-id bb3ccada --release --await 2024-09-19T09:22:52.133 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:22:52.254 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:52 smithi120 ceph-mon[31150]: pgmap v945: 97 pgs: 97 active+clean; 395 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 1.9 MiB/s wr, 78 op/s 2024-09-19T09:22:52.375 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159148 19.42 MB/sec execute 287 sec latency 31605.052 ms 2024-09-19T09:22:52.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:52 smithi196 ceph-mon[35150]: pgmap v945: 97 pgs: 97 active+clean; 395 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 1.9 MiB/s wr, 78 op/s 2024-09-19T09:22:52.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:52 smithi153 ceph-mon[35071]: pgmap v945: 97 pgs: 97 active+clean; 395 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 1.9 MiB/s wr, 78 op/s 2024-09-19T09:22:52.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.634+0000 7f61c6d40640 1 -- 172.21.15.120:0/3957471306 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61c0106f10 msgr2=0x7f61c010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:52.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.634+0000 7f61c6d40640 1 --2- 172.21.15.120:0/3957471306 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61c0106f10 0x7f61c010ea90 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009ca0 tx=0x7f61b802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.634+0000 7f61c6d40640 1 -- 172.21.15.120:0/3957471306 shutdown_connections 2024-09-19T09:22:52.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.634+0000 7f61c6d40640 1 --2- 172.21.15.120:0/3957471306 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61c0106f10 0x7f61c010ea90 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.634+0000 7f61c6d40640 1 --2- 172.21.15.120:0/3957471306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61c0102790 0x7f61c01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.634+0000 7f61c6d40640 1 --2- 172.21.15.120:0/3957471306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61c0101de0 0x7f61c01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.634+0000 7f61c6d40640 1 -- 172.21.15.120:0/3957471306 >> 172.21.15.120:0/3957471306 conn(0x7f61c00fd8c0 msgr2=0x7f61c00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:52.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.634+0000 7f61c6d40640 1 -- 172.21.15.120:0/3957471306 shutdown_connections 2024-09-19T09:22:52.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.634+0000 7f61c6d40640 1 -- 172.21.15.120:0/3957471306 wait complete. 2024-09-19T09:22:52.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.635+0000 7f61c6d40640 1 Processor -- start 2024-09-19T09:22:52.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.635+0000 7f61c6d40640 1 -- start start 2024-09-19T09:22:52.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.635+0000 7f61c6d40640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61c0101de0 0x7f61c01983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:52.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.635+0000 7f61c6d40640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61c0102790 0x7f61c0198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:52.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.635+0000 7f61c6d40640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61c0106f10 0x7f61c019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:52.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.635+0000 7f61c6d40640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f61c0111f70 con 0x7f61c0102790 2024-09-19T09:22:52.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.635+0000 7f61c6d40640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f61c0111df0 con 0x7f61c0101de0 2024-09-19T09:22:52.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.635+0000 7f61c6d40640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61c01120f0 con 0x7f61c0106f10 2024-09-19T09:22:52.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.635+0000 7f61bffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61c0102790 0x7f61c0198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:52.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.635+0000 7f61bffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61c0102790 0x7f61c0198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42690/0 (socket says 172.21.15.120:42690) 2024-09-19T09:22:52.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.635+0000 7f61bffff640 1 -- 172.21.15.120:0/2106494564 learned_addr learned my addr 172.21.15.120:0/2106494564 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:52.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.636+0000 7f61c4adc640 1 --2- 172.21.15.120:0/2106494564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61c0101de0 0x7f61c01983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:52.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.636+0000 7f61bffff640 1 -- 172.21.15.120:0/2106494564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61c0106f10 msgr2=0x7f61c019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:52.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.636+0000 7f61c52dd640 1 --2- 172.21.15.120:0/2106494564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61c0106f10 0x7f61c019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:52.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.636+0000 7f61bffff640 1 --2- 172.21.15.120:0/2106494564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61c0106f10 0x7f61c019ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.636+0000 7f61bffff640 1 -- 172.21.15.120:0/2106494564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61c0101de0 msgr2=0x7f61c01983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:52.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.636+0000 7f61bffff640 1 --2- 172.21.15.120:0/2106494564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61c0101de0 0x7f61c01983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.636+0000 7f61bffff640 1 -- 172.21.15.120:0/2106494564 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c019d3a0 con 0x7f61c0102790 2024-09-19T09:22:52.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.636+0000 7f61bffff640 1 --2- 172.21.15.120:0/2106494564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61c0102790 0x7f61c0198930 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f61ac00d950 tx=0x7f61ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:52.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.636+0000 7f61bdffb640 1 -- 172.21.15.120:0/2106494564 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ac017070 con 0x7f61c0102790 2024-09-19T09:22:52.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.636+0000 7f61bdffb640 1 -- 172.21.15.120:0/2106494564 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61ac009c30 con 0x7f61c0102790 2024-09-19T09:22:52.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.636+0000 7f61c6d40640 1 -- 172.21.15.120:0/2106494564 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c019d690 con 0x7f61c0102790 2024-09-19T09:22:52.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.636+0000 7f61bdffb640 1 -- 172.21.15.120:0/2106494564 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ac00f6b0 con 0x7f61c0102790 2024-09-19T09:22:52.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.636+0000 7f61c6d40640 1 -- 172.21.15.120:0/2106494564 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c01a4f70 con 0x7f61c0102790 2024-09-19T09:22:52.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.638+0000 7f61bdffb640 1 -- 172.21.15.120:0/2106494564 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f61ac0109d0 con 0x7f61c0102790 2024-09-19T09:22:52.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.639+0000 7f61c9119740 1 Processor -- start 2024-09-19T09:22:52.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.639+0000 7f61c9119740 1 -- start start 2024-09-19T09:22:52.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.639+0000 7f61c9119740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5ea97060 0x55fb5ea97440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:52.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.639+0000 7f61bdffb640 1 --2- 172.21.15.120:0/2106494564 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f61b00a3240 0x7f61b00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:52.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.639+0000 7f61c9119740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb5ea97980 0x55fb5eaa1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:52.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.639+0000 7f61c9119740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb5eaa2240 0x55fb5eaa4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:52.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.639+0000 7f61c9119740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fb5e9fc620 con 0x55fb5eaa2240 2024-09-19T09:22:52.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.639+0000 7f61c9119740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fb5e9a3e50 con 0x55fb5ea97980 2024-09-19T09:22:52.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.639+0000 7f61c9119740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fb5e9f85e0 con 0x55fb5ea97060 2024-09-19T09:22:52.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.639+0000 7f61c4adc640 1 --2- 172.21.15.120:0/2106494564 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f61b00a3240 0x7f61b00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:52.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.639+0000 7f61c52dd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb5eaa2240 0x55fb5eaa4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:52.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.639+0000 7f61c52dd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb5eaa2240 0x55fb5eaa4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42702/0 (socket says 172.21.15.120:42702) 2024-09-19T09:22:52.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.639+0000 7f61c52dd640 1 -- 172.21.15.120:0/4111853346 learned_addr learned my addr 172.21.15.120:0/4111853346 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:52.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.639+0000 7f61c52dd640 1 --2- 172.21.15.120:0/4111853346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5ea97060 0x55fb5ea97440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:52.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.639+0000 7f61bffff640 1 --2- 172.21.15.120:0/4111853346 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb5ea97980 0x55fb5eaa1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:52.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.639+0000 7f61bdffb640 1 -- 172.21.15.120:0/2106494564 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f61ac0cf4a0 con 0x7f61c0102790 2024-09-19T09:22:52.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.640+0000 7f61c52dd640 1 -- 172.21.15.120:0/4111853346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5ea97060 msgr2=0x55fb5ea97440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:52.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.640+0000 7f61c52dd640 1 --2- 172.21.15.120:0/4111853346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5ea97060 0x55fb5ea97440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.640+0000 7f61c52dd640 1 -- 172.21.15.120:0/4111853346 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb5ea97980 msgr2=0x55fb5eaa1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:52.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.640+0000 7f61c52dd640 1 --2- 172.21.15.120:0/4111853346 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb5ea97980 0x55fb5eaa1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.640+0000 7f61c52dd640 1 -- 172.21.15.120:0/4111853346 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb5e994d40 con 0x55fb5eaa2240 2024-09-19T09:22:52.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.640+0000 7f61c4adc640 1 --2- 172.21.15.120:0/2106494564 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f61b00a3240 0x7f61b00a5700 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f61b4005fd0 tx=0x7f61b400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:52.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.640+0000 7f61c52dd640 1 --2- 172.21.15.120:0/4111853346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb5eaa2240 0x55fb5eaa4630 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f61b8004af0 tx=0x7f61b8005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33d25bb99ffa656 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:52.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.640+0000 7f61aaffd640 1 -- 172.21.15.120:0/4111853346 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8003b70 con 0x55fb5eaa2240 2024-09-19T09:22:52.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.640+0000 7f61aaffd640 1 -- 172.21.15.120:0/4111853346 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61b8036040 con 0x55fb5eaa2240 2024-09-19T09:22:52.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.640+0000 7f61aaffd640 1 -- 172.21.15.120:0/4111853346 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8004430 con 0x55fb5eaa2240 2024-09-19T09:22:52.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.641+0000 7f61c9119740 1 -- 172.21.15.120:0/4111853346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb5eaa2240 msgr2=0x55fb5eaa4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:52.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.641+0000 7f61c9119740 1 --2- 172.21.15.120:0/4111853346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb5eaa2240 0x55fb5eaa4630 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f61b8004af0 tx=0x7f61b8005d80 comp rx=0 tx=0).stop 2024-09-19T09:22:52.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.641+0000 7f61c9119740 1 -- 172.21.15.120:0/4111853346 shutdown_connections 2024-09-19T09:22:52.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.641+0000 7f61c9119740 1 --2- 172.21.15.120:0/4111853346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb5eaa2240 0x55fb5eaa4630 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.641+0000 7f61c9119740 1 --2- 172.21.15.120:0/4111853346 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb5ea97980 0x55fb5eaa1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.641+0000 7f61c9119740 1 --2- 172.21.15.120:0/4111853346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5ea97060 0x55fb5ea97440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.641+0000 7f61c9119740 1 -- 172.21.15.120:0/4111853346 >> 172.21.15.120:0/4111853346 conn(0x55fb5ea935f0 msgr2=0x55fb5ea939d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:52.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.641+0000 7f61c9119740 1 -- 172.21.15.120:0/4111853346 shutdown_connections 2024-09-19T09:22:52.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.641+0000 7f61c9119740 1 -- 172.21.15.120:0/4111853346 wait complete. 2024-09-19T09:22:52.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.641+0000 7f61c9119740 1 -- start start 2024-09-19T09:22:52.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.641+0000 7f61c9119740 1 Processor -- start 2024-09-19T09:22:52.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.641+0000 7f61c9119740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb5ea97060 0x55fb5ea9f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:52.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.641+0000 7f61c9119740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb5ea97980 0x55fb5ea9f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:52.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.641+0000 7f61c9119740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5eaa2240 0x55fb5ea9fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:52.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.641+0000 7f61c9119740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fb5eaa4e10 con 0x55fb5ea97980 2024-09-19T09:22:52.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.641+0000 7f61c9119740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fb5e984f10 con 0x55fb5ea97060 2024-09-19T09:22:52.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.641+0000 7f61c9119740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fb5eaa4f90 con 0x55fb5eaa2240 2024-09-19T09:22:52.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.642+0000 7f61c9119740 10 client.0 _ll_register_callbacks cb 0x55fb5e9a8db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:22:52.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.642+0000 7f61bffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb5ea97980 0x55fb5ea9f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:52.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.642+0000 7f61bffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb5ea97980 0x55fb5ea9f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42706/0 (socket says 172.21.15.120:42706) 2024-09-19T09:22:52.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.642+0000 7f61bffff640 1 -- 172.21.15.120:0/2696791882 learned_addr learned my addr 172.21.15.120:0/2696791882 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:52.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.642+0000 7f61c4adc640 1 --2- 172.21.15.120:0/2696791882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb5ea97060 0x55fb5ea9f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:52.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.642+0000 7f61c52dd640 1 --2- 172.21.15.120:0/2696791882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5eaa2240 0x55fb5ea9fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:52.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.642+0000 7f61bffff640 1 -- 172.21.15.120:0/2696791882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5eaa2240 msgr2=0x55fb5ea9fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:52.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.642+0000 7f61bffff640 1 --2- 172.21.15.120:0/2696791882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5eaa2240 0x55fb5ea9fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.642+0000 7f61bffff640 1 -- 172.21.15.120:0/2696791882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb5ea97060 msgr2=0x55fb5ea9f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:52.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.642+0000 7f61bffff640 1 --2- 172.21.15.120:0/2696791882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb5ea97060 0x55fb5ea9f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.642+0000 7f61bffff640 1 -- 172.21.15.120:0/2696791882 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb5e70b580 con 0x55fb5ea97980 2024-09-19T09:22:52.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.642+0000 7f61bffff640 1 --2- 172.21.15.120:0/2696791882 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb5ea97980 0x55fb5ea9f6d0 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f61ac098490 tx=0x7f61ac009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9799cffd8823409 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:52.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.642+0000 7f619ffff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:22:52.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.642+0000 7f619ffff640 1 -- 172.21.15.120:0/2696791882 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ac020070 con 0x55fb5ea97980 2024-09-19T09:22:52.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.642+0000 7f619ffff640 1 -- 172.21.15.120:0/2696791882 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f61ac0cf0f0 con 0x55fb5ea97980 2024-09-19T09:22:52.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.642+0000 7f61c9119740 1 -- 172.21.15.120:0/2696791882 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61b8036040 con 0x55fb5ea97980 2024-09-19T09:22:52.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.642+0000 7f619ffff640 1 -- 172.21.15.120:0/2696791882 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f61ac0b9050 con 0x55fb5ea97980 2024-09-19T09:22:52.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.642+0000 7f61c9119740 10 client.22038 fetch_fsmap learned FSMap version 39 2024-09-19T09:22:52.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.642+0000 7f61c9119740 1 -- 172.21.15.120:0/2696791882 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55fb5e7d5960 con 0x55fb5ea97980 2024-09-19T09:22:52.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.643+0000 7f619ffff640 1 -- 172.21.15.120:0/2696791882 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61b8036040 con 0x55fb5ea97980 2024-09-19T09:22:52.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.643+0000 7f619ffff640 1 -- 172.21.15.120:0/2696791882 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f61ac0ba240 con 0x55fb5ea97980 2024-09-19T09:22:52.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.643+0000 7f61c9119740 10 client.22038 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:22:52.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.643+0000 7f61c9119740 10 client.22038 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:22:52.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.643+0000 7f61c9119740 20 client.22038 populate_metadata read hostname 'smithi120' 2024-09-19T09:22:52.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.643+0000 7f61c9119740 1 --2- 172.21.15.120:0/2696791882 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fb5eab1d60 0x55fb5eab4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:52.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.643+0000 7f61c9119740 4 client.22038 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:22:52.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.643+0000 7f61c9119740 1 -- 172.21.15.120:0/2696791882 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb5ea09b90 con 0x55fb5eab1d60 2024-09-19T09:22:52.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.643+0000 7f61c4adc640 1 --2- 172.21.15.120:0/2696791882 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fb5eab1d60 0x55fb5eab4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:52.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.643+0000 7f61c4adc640 1 --2- 172.21.15.120:0/2696791882 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fb5eab1d60 0x55fb5eab4190 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76cbc9691efab573 server_cookie=4462afb7c472fe49 in_seq=0 out_seq=0 2024-09-19T09:22:52.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.643+0000 7f619ffff640 10 client.22038 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:52.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.644+0000 7f619ffff640 1 -- 172.21.15.120:0/2696791882 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb5ea09b90 con 0x55fb5eab1d60 2024-09-19T09:22:52.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.644+0000 7f619ffff640 10 client.22038 handle_command_reply: tid=0 2024-09-19T09:22:52.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.644+0000 7f61c9119740 1 client.22038 shutdown 2024-09-19T09:22:52.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.644+0000 7f61c9119740 1 -- 172.21.15.120:0/2696791882 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb5ea97980 msgr2=0x55fb5ea9f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:52.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.644+0000 7f61c9119740 1 --2- 172.21.15.120:0/2696791882 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb5ea97980 0x55fb5ea9f6d0 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f61ac098490 tx=0x7f61ac009f20 comp rx=0 tx=0).stop 2024-09-19T09:22:52.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.644+0000 7f61c9119740 1 -- 172.21.15.120:0/2696791882 shutdown_connections 2024-09-19T09:22:52.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.644+0000 7f61c9119740 1 --2- 172.21.15.120:0/2696791882 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fb5eab1d60 0x55fb5eab4190 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.644+0000 7f61c9119740 1 --2- 172.21.15.120:0/2696791882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5eaa2240 0x55fb5ea9fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.644+0000 7f619ffff640 0 client.22038 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:52.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.644+0000 7f61c9119740 1 --2- 172.21.15.120:0/2696791882 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb5ea97980 0x55fb5ea9f6d0 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.644+0000 7f61c9119740 1 --2- 172.21.15.120:0/2696791882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb5ea97060 0x55fb5ea9f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.644+0000 7f61c9119740 1 -- 172.21.15.120:0/2696791882 >> 172.21.15.120:0/2696791882 conn(0x55fb5ea935f0 msgr2=0x55fb5ea992d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:52.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.644+0000 7f61c9119740 1 -- 172.21.15.120:0/2696791882 shutdown_connections 2024-09-19T09:22:52.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.644+0000 7f61c9119740 1 -- 172.21.15.120:0/2696791882 wait complete. 2024-09-19T09:22:52.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.645+0000 7f61c9119740 20 client.22038 trim_cache size 0 max 16384 2024-09-19T09:22:52.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.662+0000 7f61c9119740 1 Processor -- start 2024-09-19T09:22:52.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.662+0000 7f61c9119740 1 -- start start 2024-09-19T09:22:52.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.662+0000 7f61c9119740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb5ea97060 0x55fb5eab9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:52.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.662+0000 7f61c9119740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb5ea97980 0x55fb5eab9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:52.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.662+0000 7f61c9119740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5eaa2240 0x55fb5ea9c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:52.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.662+0000 7f61c9119740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f61c01ad620 con 0x55fb5ea97980 2024-09-19T09:22:52.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.662+0000 7f61c9119740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f61c01af160 con 0x55fb5ea97060 2024-09-19T09:22:52.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.662+0000 7f61c9119740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61c01adcf0 con 0x55fb5eaa2240 2024-09-19T09:22:52.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.662+0000 7f61bffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb5ea97980 0x55fb5eab9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:52.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.662+0000 7f61bffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb5ea97980 0x55fb5eab9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42714/0 (socket says 172.21.15.120:42714) 2024-09-19T09:22:52.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.662+0000 7f61bffff640 1 -- 172.21.15.120:0/672346038 learned_addr learned my addr 172.21.15.120:0/672346038 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:52.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.662+0000 7f61bffff640 1 -- 172.21.15.120:0/672346038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5eaa2240 msgr2=0x55fb5ea9c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:52.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.662+0000 7f61bffff640 1 --2- 172.21.15.120:0/672346038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5eaa2240 0x55fb5ea9c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.662+0000 7f61c4adc640 1 --2- 172.21.15.120:0/672346038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb5ea97060 0x55fb5eab9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:52.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.662+0000 7f61bffff640 1 -- 172.21.15.120:0/672346038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb5ea97060 msgr2=0x55fb5eab9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:52.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.662+0000 7f61bffff640 1 --2- 172.21.15.120:0/672346038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb5ea97060 0x55fb5eab9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.662+0000 7f61bffff640 1 -- 172.21.15.120:0/672346038 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb5ea93090 con 0x55fb5ea97980 2024-09-19T09:22:52.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.663+0000 7f61c4adc640 1 --2- 172.21.15.120:0/672346038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb5ea97060 0x55fb5eab9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:22:52.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.663+0000 7f61bffff640 1 --2- 172.21.15.120:0/672346038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb5ea97980 0x55fb5eab9fa0 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f61ac010fd0 tx=0x7f61ac00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57a51ba8d31dd58 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:52.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.663+0000 7f61aa7fc640 1 -- 172.21.15.120:0/672346038 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ac020070 con 0x55fb5ea97980 2024-09-19T09:22:52.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.663+0000 7f61aa7fc640 1 -- 172.21.15.120:0/672346038 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61ac09fe70 con 0x55fb5ea97980 2024-09-19T09:22:52.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.663+0000 7f61aa7fc640 1 -- 172.21.15.120:0/672346038 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ac0cf0f0 con 0x55fb5ea97980 2024-09-19T09:22:52.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.664+0000 7f61c9119740 1 -- 172.21.15.120:0/672346038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb5ea97980 msgr2=0x55fb5eab9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:52.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.664+0000 7f61c9119740 1 --2- 172.21.15.120:0/672346038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb5ea97980 0x55fb5eab9fa0 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f61ac010fd0 tx=0x7f61ac00bf60 comp rx=0 tx=0).stop 2024-09-19T09:22:52.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.664+0000 7f61c9119740 1 -- 172.21.15.120:0/672346038 shutdown_connections 2024-09-19T09:22:52.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.664+0000 7f61c9119740 1 --2- 172.21.15.120:0/672346038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5eaa2240 0x55fb5ea9c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.664+0000 7f61c9119740 1 --2- 172.21.15.120:0/672346038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb5ea97980 0x55fb5eab9fa0 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.664+0000 7f61c9119740 1 --2- 172.21.15.120:0/672346038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb5ea97060 0x55fb5eab9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.664+0000 7f61c9119740 1 -- 172.21.15.120:0/672346038 >> 172.21.15.120:0/672346038 conn(0x55fb5ea935f0 msgr2=0x55fb5eaa1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:52.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.664+0000 7f61c9119740 1 -- 172.21.15.120:0/672346038 shutdown_connections 2024-09-19T09:22:52.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.664+0000 7f61c9119740 1 -- 172.21.15.120:0/672346038 wait complete. 2024-09-19T09:22:52.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.664+0000 7f61c9119740 1 -- start start 2024-09-19T09:22:52.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.664+0000 7f61c9119740 1 Processor -- start 2024-09-19T09:22:52.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.664+0000 7f61c9119740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb5ea97060 0x55fb5eaa6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:52.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.664+0000 7f61c9119740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5ea97980 0x55fb5eaa6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:52.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.664+0000 7f61c9119740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb5eaa2240 0x55fb5eb1ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:52.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.664+0000 7f61c9119740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f61c01aea90 con 0x55fb5eaa2240 2024-09-19T09:22:52.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.664+0000 7f61c9119740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f61c01ae3c0 con 0x55fb5ea97060 2024-09-19T09:22:52.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.664+0000 7f61c9119740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61c01ab5e0 con 0x55fb5ea97980 2024-09-19T09:22:52.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.664+0000 7f61c9119740 10 client.0 _ll_register_callbacks cb 0x7f61ac09b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:22:52.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.665+0000 7f61c52dd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb5eaa2240 0x55fb5eb1ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:52.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.665+0000 7f61c52dd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb5eaa2240 0x55fb5eb1ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42722/0 (socket says 172.21.15.120:42722) 2024-09-19T09:22:52.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.665+0000 7f61c52dd640 1 -- 172.21.15.120:0/2391203362 learned_addr learned my addr 172.21.15.120:0/2391203362 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:22:52.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.665+0000 7f61c4adc640 1 --2- 172.21.15.120:0/2391203362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb5ea97060 0x55fb5eaa6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:52.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.665+0000 7f61bffff640 1 --2- 172.21.15.120:0/2391203362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5ea97980 0x55fb5eaa6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:52.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.665+0000 7f61c52dd640 1 -- 172.21.15.120:0/2391203362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5ea97980 msgr2=0x55fb5eaa6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:52.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.665+0000 7f61c52dd640 1 --2- 172.21.15.120:0/2391203362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5ea97980 0x55fb5eaa6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.665+0000 7f61c52dd640 1 -- 172.21.15.120:0/2391203362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb5ea97060 msgr2=0x55fb5eaa6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:22:52.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.665+0000 7f61c52dd640 1 --2- 172.21.15.120:0/2391203362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb5ea97060 0x55fb5eaa6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.665+0000 7f61c52dd640 1 -- 172.21.15.120:0/2391203362 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb5eb1bf70 con 0x55fb5eaa2240 2024-09-19T09:22:52.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.665+0000 7f61c52dd640 1 --2- 172.21.15.120:0/2391203362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb5eaa2240 0x55fb5eb1ba30 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f61b8004af0 tx=0x7f61b8031ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1d3c9d3478e211 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:52.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.665+0000 7f619ffff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:22:52.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.665+0000 7f619ffff640 1 -- 172.21.15.120:0/2391203362 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8053060 con 0x55fb5eaa2240 2024-09-19T09:22:52.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.665+0000 7f619ffff640 1 -- 172.21.15.120:0/2391203362 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f61b8031860 con 0x55fb5eaa2240 2024-09-19T09:22:52.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.665+0000 7f61c9119740 1 -- 172.21.15.120:0/2391203362 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61ac09fe70 con 0x55fb5eaa2240 2024-09-19T09:22:52.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.665+0000 7f619ffff640 1 -- 172.21.15.120:0/2391203362 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f61b8003970 con 0x55fb5eaa2240 2024-09-19T09:22:52.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.665+0000 7f61c9119740 10 client.22044 fetch_fsmap learned FSMap version 39 2024-09-19T09:22:52.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.665+0000 7f61c9119740 1 -- 172.21.15.120:0/2391203362 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55fb5eb1d220 con 0x55fb5eaa2240 2024-09-19T09:22:52.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.666+0000 7f619ffff640 1 -- 172.21.15.120:0/2391203362 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61ac09fe70 con 0x55fb5eaa2240 2024-09-19T09:22:52.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.666+0000 7f619ffff640 1 -- 172.21.15.120:0/2391203362 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f61b8058070 con 0x55fb5eaa2240 2024-09-19T09:22:52.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.666+0000 7f61c9119740 10 client.22044 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:22:52.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.666+0000 7f61c9119740 10 client.22044 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:22:52.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.666+0000 7f61c9119740 20 client.22044 populate_metadata read hostname 'smithi120' 2024-09-19T09:22:52.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.666+0000 7f61c9119740 1 --2- 172.21.15.120:0/2391203362 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fb5eab1d60 0x55fb5eb1f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:52.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.666+0000 7f61c9119740 4 client.22044 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "bb3ccada", "release": true, "await": true}] 2024-09-19T09:22:52.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.666+0000 7f61c9119740 1 -- 172.21.15.120:0/2391203362 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "set_id": "bb3ccada", "release": true, "await": true}) -- 0x55fb5ea040a0 con 0x55fb5eab1d60 2024-09-19T09:22:52.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.666+0000 7f61c4adc640 1 --2- 172.21.15.120:0/2391203362 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fb5eab1d60 0x55fb5eb1f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:52.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.667+0000 7f619ffff640 10 client.22044 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:52.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.667+0000 7f61c4adc640 1 --2- 172.21.15.120:0/2391203362 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fb5eab1d60 0x55fb5eb1f400 crc :-1 s=READY pgs=500 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad42d7a0c9ac34ce server_cookie=26fe3b77ddec0256 in_seq=0 out_seq=0 2024-09-19T09:22:52.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.737+0000 7f619ffff640 1 -- 172.21.15.120:0/2391203362 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fb5ea040a0 con 0x55fb5eab1d60 2024-09-19T09:22:52.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.737+0000 7f619ffff640 10 client.22044 handle_command_reply: tid=0 2024-09-19T09:22:52.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.737+0000 7f61c9119740 1 client.22044 shutdown 2024-09-19T09:22:52.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.737+0000 7f61c9119740 1 -- 172.21.15.120:0/2391203362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb5eaa2240 msgr2=0x55fb5eb1ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:52.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.737+0000 7f61c9119740 1 --2- 172.21.15.120:0/2391203362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb5eaa2240 0x55fb5eb1ba30 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f61b8004af0 tx=0x7f61b8031ef0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.737+0000 7f61c9119740 1 -- 172.21.15.120:0/2391203362 shutdown_connections 2024-09-19T09:22:52.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.737+0000 7f61c9119740 1 --2- 172.21.15.120:0/2391203362 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55fb5eab1d60 0x55fb5eb1f400 crc :-1 s=READY pgs=500 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.738+0000 7f61c9119740 1 --2- 172.21.15.120:0/2391203362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb5eaa2240 0x55fb5eb1ba30 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.738+0000 7f619ffff640 0 client.22044 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:22:52.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.738+0000 7f61c9119740 1 --2- 172.21.15.120:0/2391203362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5ea97980 0x55fb5eaa6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.738+0000 7f61c9119740 1 --2- 172.21.15.120:0/2391203362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb5ea97060 0x55fb5eaa6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.738+0000 7f61c9119740 1 -- 172.21.15.120:0/2391203362 >> 172.21.15.120:0/2391203362 conn(0x55fb5ea935f0 msgr2=0x55fb5ea98ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:52.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.738+0000 7f61c9119740 1 -- 172.21.15.120:0/2391203362 shutdown_connections 2024-09-19T09:22:52.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.738+0000 7f61c9119740 1 -- 172.21.15.120:0/2391203362 wait complete. 2024-09-19T09:22:52.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.738+0000 7f61c9119740 20 client.22044 trim_cache size 0 max 16384 2024-09-19T09:22:52.740 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:22:52.740 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 39, 2024-09-19T09:22:52.740 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:22:52.741 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 68, 2024-09-19T09:22:52.741 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:22:52.741 INFO:teuthology.orchestra.run.smithi120.stdout: "bb3ccada": { 2024-09-19T09:22:52.741 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 68, 2024-09-19T09:22:52.741 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:22:52.741 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:22:52.741 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:22:52.741 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:22:52.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:22:52.741 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:22:52.741 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 154.0, 2024-09-19T09:22:52.741 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:22:52.741 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:22:52.741 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:22:52.742 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:22:52.742 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:22:52.742 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:22:52.742 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:22:52.742 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:22:52.742 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:22:52.742 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:22:52.742 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:22:52.742 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:22:52.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.738+0000 7f61c6d40640 1 -- 172.21.15.120:0/2106494564 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f61b00a3240 msgr2=0x7f61b00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:52.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.738+0000 7f61c6d40640 1 --2- 172.21.15.120:0/2106494564 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f61b00a3240 0x7f61b00a5700 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f61b4005fd0 tx=0x7f61b400b390 comp rx=0 tx=0).stop 2024-09-19T09:22:52.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.738+0000 7f61c6d40640 1 -- 172.21.15.120:0/2106494564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61c0102790 msgr2=0x7f61c0198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:52.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.738+0000 7f61c6d40640 1 --2- 172.21.15.120:0/2106494564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61c0102790 0x7f61c0198930 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f61ac00d950 tx=0x7f61ac00de10 comp rx=0 tx=0).stop 2024-09-19T09:22:52.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.739+0000 7f61c6d40640 1 -- 172.21.15.120:0/2106494564 shutdown_connections 2024-09-19T09:22:52.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.739+0000 7f61c6d40640 1 --2- 172.21.15.120:0/2106494564 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f61b00a3240 0x7f61b00a5700 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.739+0000 7f61c6d40640 1 --2- 172.21.15.120:0/2106494564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61c0106f10 0x7f61c019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.739+0000 7f61c6d40640 1 --2- 172.21.15.120:0/2106494564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61c0102790 0x7f61c0198930 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.739+0000 7f61c6d40640 1 --2- 172.21.15.120:0/2106494564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61c0101de0 0x7f61c01983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:52.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.739+0000 7f61c6d40640 1 -- 172.21.15.120:0/2106494564 >> 172.21.15.120:0/2106494564 conn(0x7f61c00fd8c0 msgr2=0x7f61c00feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:52.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.739+0000 7f61c6d40640 1 -- 172.21.15.120:0/2106494564 shutdown_connections 2024-09-19T09:22:52.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:22:52.739+0000 7f61c6d40640 1 -- 172.21.15.120:0/2106494564 wait complete. 2024-09-19T09:22:53.375 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159527 19.46 MB/sec execute 288 sec latency 31914.123 ms 2024-09-19T09:22:53.531 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'bb3ccada', total seconds elapsed: 37.6 2024-09-19T09:22:54.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:54 smithi120 ceph-mon[31150]: pgmap v946: 97 pgs: 97 active+clean; 393 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 2.0 MiB/s wr, 110 op/s 2024-09-19T09:22:54.375 INFO:tasks.workunit.client.0.smithi120.stdout: 10 160099 19.52 MB/sec execute 289 sec latency 84.161 ms 2024-09-19T09:22:54.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:54 smithi196 ceph-mon[35150]: pgmap v946: 97 pgs: 97 active+clean; 393 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 2.0 MiB/s wr, 110 op/s 2024-09-19T09:22:54.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:54 smithi153 ceph-mon[35071]: pgmap v946: 97 pgs: 97 active+clean; 393 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 2.0 MiB/s wr, 110 op/s 2024-09-19T09:22:55.375 INFO:tasks.workunit.client.0.smithi120.stdout: 10 160649 19.56 MB/sec execute 290 sec latency 62.212 ms 2024-09-19T09:22:56.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:56 smithi120 ceph-mon[31150]: pgmap v947: 97 pgs: 97 active+clean; 393 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 113 op/s 2024-09-19T09:22:56.375 INFO:tasks.workunit.client.0.smithi120.stdout: 10 161108 19.57 MB/sec execute 291 sec latency 52.804 ms 2024-09-19T09:22:56.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:56 smithi196 ceph-mon[35150]: pgmap v947: 97 pgs: 97 active+clean; 393 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 113 op/s 2024-09-19T09:22:56.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:56 smithi153 ceph-mon[35071]: pgmap v947: 97 pgs: 97 active+clean; 393 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 113 op/s 2024-09-19T09:22:57.376 INFO:tasks.workunit.client.0.smithi120.stdout: 10 161626 19.59 MB/sec execute 292 sec latency 69.638 ms 2024-09-19T09:22:58.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:22:58 smithi120 ceph-mon[31150]: pgmap v948: 97 pgs: 97 active+clean; 393 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.2 MiB/s wr, 137 op/s 2024-09-19T09:22:58.376 INFO:tasks.workunit.client.0.smithi120.stdout: 10 162183 19.60 MB/sec execute 293 sec latency 67.857 ms 2024-09-19T09:22:58.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:22:58 smithi196 ceph-mon[35150]: pgmap v948: 97 pgs: 97 active+clean; 393 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.2 MiB/s wr, 137 op/s 2024-09-19T09:22:58.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:22:58 smithi153 ceph-mon[35071]: pgmap v948: 97 pgs: 97 active+clean; 393 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.2 MiB/s wr, 137 op/s 2024-09-19T09:22:59.225 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:22:59.376 INFO:tasks.workunit.client.0.smithi120.stdout: 10 162910 19.68 MB/sec execute 294 sec latency 48.908 ms 2024-09-19T09:22:59.534 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:23:00.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.104+0000 7f280e383640 1 -- 172.21.15.120:0/1333738004 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2808102d30 msgr2=0x7f2808103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:00.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.104+0000 7f280e383640 1 --2- 172.21.15.120:0/1333738004 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2808102d30 0x7f2808103110 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f2800009930 tx=0x7f280002a420 comp rx=0 tx=0).stop 2024-09-19T09:23:00.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.104+0000 7f280e383640 1 -- 172.21.15.120:0/1333738004 shutdown_connections 2024-09-19T09:23:00.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.104+0000 7f280e383640 1 --2- 172.21.15.120:0/1333738004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28081082a0 0x7f280810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.104+0000 7f280e383640 1 --2- 172.21.15.120:0/1333738004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28081036e0 0x7f2808107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.104+0000 7f280e383640 1 --2- 172.21.15.120:0/1333738004 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2808102d30 0x7f2808103110 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.104+0000 7f280e383640 1 -- 172.21.15.120:0/1333738004 >> 172.21.15.120:0/1333738004 conn(0x7f28080fc7d0 msgr2=0x7f28080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:00.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.105+0000 7f280e383640 1 -- 172.21.15.120:0/1333738004 shutdown_connections 2024-09-19T09:23:00.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.105+0000 7f280e383640 1 -- 172.21.15.120:0/1333738004 wait complete. 2024-09-19T09:23:00.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.105+0000 7f280e383640 1 Processor -- start 2024-09-19T09:23:00.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.105+0000 7f280e383640 1 -- start start 2024-09-19T09:23:00.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.106+0000 7f280e383640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2808102d30 0x7f28081a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:00.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.106+0000 7f280e383640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28081036e0 0x7f28081a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:00.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.106+0000 7f280e383640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28081082a0 0x7f280810de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:00.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.106+0000 7f280e383640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f28081120a0 con 0x7f28081036e0 2024-09-19T09:23:00.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.106+0000 7f280e383640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2808111f20 con 0x7f2808102d30 2024-09-19T09:23:00.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.106+0000 7f280e383640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2808112220 con 0x7f28081082a0 2024-09-19T09:23:00.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.106+0000 7f280c920640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28081082a0 0x7f280810de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:00.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.107+0000 7f280c920640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28081082a0 0x7f280810de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:47048/0 (socket says 172.21.15.120:47048) 2024-09-19T09:23:00.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.107+0000 7f280c920640 1 -- 172.21.15.120:0/4132879468 learned_addr learned my addr 172.21.15.120:0/4132879468 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:00.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.107+0000 7f28077fe640 1 --2- 172.21.15.120:0/4132879468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28081036e0 0x7f28081a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:00.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.107+0000 7f2807fff640 1 --2- 172.21.15.120:0/4132879468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2808102d30 0x7f28081a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:00.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.107+0000 7f280c920640 1 -- 172.21.15.120:0/4132879468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2808102d30 msgr2=0x7f28081a4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:00.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.107+0000 7f280c920640 1 --2- 172.21.15.120:0/4132879468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2808102d30 0x7f28081a4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.107+0000 7f280c920640 1 -- 172.21.15.120:0/4132879468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28081036e0 msgr2=0x7f28081a4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:00.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.108+0000 7f280c920640 1 --2- 172.21.15.120:0/4132879468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28081036e0 0x7f28081a4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.108+0000 7f280c920640 1 -- 172.21.15.120:0/4132879468 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280810e530 con 0x7f28081082a0 2024-09-19T09:23:00.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.108+0000 7f280c920640 1 --2- 172.21.15.120:0/4132879468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28081082a0 0x7f280810de00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f27f000e970 tx=0x7f27f000ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:00.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.108+0000 7f28057fa640 1 -- 172.21.15.120:0/4132879468 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f000ccc0 con 0x7f28081082a0 2024-09-19T09:23:00.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.109+0000 7f28057fa640 1 -- 172.21.15.120:0/4132879468 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27f000ce60 con 0x7f28081082a0 2024-09-19T09:23:00.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.109+0000 7f28057fa640 1 -- 172.21.15.120:0/4132879468 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f0010490 con 0x7f28081082a0 2024-09-19T09:23:00.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.109+0000 7f280e383640 1 -- 172.21.15.120:0/4132879468 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f280810e820 con 0x7f28081082a0 2024-09-19T09:23:00.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.109+0000 7f280e383640 1 -- 172.21.15.120:0/4132879468 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28081ab6f0 con 0x7f28081082a0 2024-09-19T09:23:00.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.112+0000 7f281075c740 1 Processor -- start 2024-09-19T09:23:00.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.112+0000 7f281075c740 1 -- start start 2024-09-19T09:23:00.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.112+0000 7f28057fa640 1 -- 172.21.15.120:0/4132879468 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f27f0005020 con 0x7f28081082a0 2024-09-19T09:23:00.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.112+0000 7f281075c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d769ad4a0 0x560d769ad880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:00.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.113+0000 7f281075c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d769addc0 0x560d769b8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:00.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.113+0000 7f281075c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d769b8680 0x560d769baa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:00.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.113+0000 7f281075c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560d769132e0 con 0x560d769addc0 2024-09-19T09:23:00.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.113+0000 7f281075c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560d768bace0 con 0x560d769ad4a0 2024-09-19T09:23:00.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.113+0000 7f281075c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560d7690f2a0 con 0x560d769b8680 2024-09-19T09:23:00.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.113+0000 7f28077fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d769ad4a0 0x560d769ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:00.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.113+0000 7f2807fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d769addc0 0x560d769b8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:00.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.113+0000 7f2807fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d769addc0 0x560d769b8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42762/0 (socket says 172.21.15.120:42762) 2024-09-19T09:23:00.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.113+0000 7f2807fff640 1 -- 172.21.15.120:0/806933306 learned_addr learned my addr 172.21.15.120:0/806933306 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:00.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.113+0000 7f28077fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d769ad4a0 0x560d769ad880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:47236/0 (socket says 172.21.15.120:47236) 2024-09-19T09:23:00.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.113+0000 7f28077fe640 1 --2- 172.21.15.120:0/806933306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d769b8680 0x560d769baa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:00.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.113+0000 7f2807fff640 1 -- 172.21.15.120:0/806933306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d769b8680 msgr2=0x560d769baa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:00.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.113+0000 7f2807fff640 1 --2- 172.21.15.120:0/806933306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d769b8680 0x560d769baa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.113+0000 7f2807fff640 1 -- 172.21.15.120:0/806933306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d769ad4a0 msgr2=0x560d769ad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:00.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.113+0000 7f2807fff640 1 --2- 172.21.15.120:0/806933306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d769ad4a0 0x560d769ad880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.113+0000 7f2807fff640 1 -- 172.21.15.120:0/806933306 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d7689f440 con 0x560d769addc0 2024-09-19T09:23:00.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.113+0000 7f28077fe640 1 --2- 172.21.15.120:0/806933306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d769ad4a0 0x560d769ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:23:00.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.114+0000 7f2807fff640 1 --2- 172.21.15.120:0/806933306 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d769addc0 0x560d769b8140 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f2800000c00 tx=0x7f2800005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e22f12737bde1f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:00.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.114+0000 7f27fe7fc640 1 -- 172.21.15.120:0/806933306 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280003f070 con 0x560d769addc0 2024-09-19T09:23:00.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.114+0000 7f27fe7fc640 1 -- 172.21.15.120:0/806933306 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2800004030 con 0x560d769addc0 2024-09-19T09:23:00.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.114+0000 7f27fe7fc640 1 -- 172.21.15.120:0/806933306 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2800035070 con 0x560d769addc0 2024-09-19T09:23:00.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.114+0000 7f28057fa640 1 --2- 172.21.15.120:0/4132879468 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f27f40a3340 0x7f27f40a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:00.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.115+0000 7f28077fe640 1 --2- 172.21.15.120:0/4132879468 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f27f40a3340 0x7f27f40a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:00.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.115+0000 7f28057fa640 1 -- 172.21.15.120:0/4132879468 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f27f00cfd60 con 0x7f28081082a0 2024-09-19T09:23:00.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.115+0000 7f281075c740 1 -- 172.21.15.120:0/806933306 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d769addc0 msgr2=0x560d769b8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:00.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.115+0000 7f281075c740 1 --2- 172.21.15.120:0/806933306 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d769addc0 0x560d769b8140 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f2800000c00 tx=0x7f2800005dd0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.115+0000 7f281075c740 1 -- 172.21.15.120:0/806933306 shutdown_connections 2024-09-19T09:23:00.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.115+0000 7f281075c740 1 --2- 172.21.15.120:0/806933306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d769b8680 0x560d769baa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.115+0000 7f281075c740 1 --2- 172.21.15.120:0/806933306 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d769addc0 0x560d769b8140 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.115+0000 7f281075c740 1 --2- 172.21.15.120:0/806933306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d769ad4a0 0x560d769ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.115+0000 7f281075c740 1 -- 172.21.15.120:0/806933306 >> 172.21.15.120:0/806933306 conn(0x560d769a9a30 msgr2=0x560d769a9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:00.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.115+0000 7f28077fe640 1 --2- 172.21.15.120:0/4132879468 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f27f40a3340 0x7f27f40a5800 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009a00 tx=0x7f27f8009990 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:00.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.115+0000 7f281075c740 1 -- 172.21.15.120:0/806933306 shutdown_connections 2024-09-19T09:23:00.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.115+0000 7f281075c740 1 -- 172.21.15.120:0/806933306 wait complete. 2024-09-19T09:23:00.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.116+0000 7f281075c740 1 -- start start 2024-09-19T09:23:00.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.116+0000 7f281075c740 1 Processor -- start 2024-09-19T09:23:00.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.116+0000 7f281075c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d769ad4a0 0x560d769b7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:00.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.116+0000 7f281075c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d769addc0 0x560d769b51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:00.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.117+0000 7f281075c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d769b8680 0x560d769b5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:00.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.117+0000 7f2807fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d769ad4a0 0x560d769b7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:00.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.117+0000 7f2807fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d769ad4a0 0x560d769b7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:47066/0 (socket says 172.21.15.120:47066) 2024-09-19T09:23:00.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.117+0000 7f2807fff640 1 -- 172.21.15.120:0/2905650571 learned_addr learned my addr 172.21.15.120:0/2905650571 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:00.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.117+0000 7f281075c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560d769bb250 con 0x560d769b8680 2024-09-19T09:23:00.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.117+0000 7f281075c740 1 -- 172.21.15.120:0/2905650571 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560d768ce050 con 0x560d769addc0 2024-09-19T09:23:00.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.117+0000 7f281075c740 1 -- 172.21.15.120:0/2905650571 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560d769bb3d0 con 0x560d769ad4a0 2024-09-19T09:23:00.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.117+0000 7f28077fe640 1 --2- 172.21.15.120:0/2905650571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d769addc0 0x560d769b51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:00.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.117+0000 7f281075c740 10 client.0 _ll_register_callbacks cb 0x560d7690e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:00.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.117+0000 7f280c920640 1 --2- 172.21.15.120:0/2905650571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d769b8680 0x560d769b5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:00.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.117+0000 7f2807fff640 1 -- 172.21.15.120:0/2905650571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d769addc0 msgr2=0x560d769b51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:00.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.117+0000 7f2807fff640 1 --2- 172.21.15.120:0/2905650571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d769addc0 0x560d769b51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.117+0000 7f2807fff640 1 -- 172.21.15.120:0/2905650571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d769b8680 msgr2=0x560d769b5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:00.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.117+0000 7f2807fff640 1 --2- 172.21.15.120:0/2905650571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d769b8680 0x560d769b5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.117+0000 7f2807fff640 1 -- 172.21.15.120:0/2905650571 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d766843c0 con 0x560d769ad4a0 2024-09-19T09:23:00.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.117+0000 7f280c920640 1 --2- 172.21.15.120:0/2905650571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d769b8680 0x560d769b5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:23:00.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.117+0000 7f2807fff640 1 --2- 172.21.15.120:0/2905650571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d769ad4a0 0x560d769b7f40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2800009850 tx=0x7f2800009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc8db6c62581898b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:00.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.117+0000 7f27e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:23:00.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.118+0000 7f27e77fe640 1 -- 172.21.15.120:0/2905650571 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2800035070 con 0x560d769ad4a0 2024-09-19T09:23:00.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.118+0000 7f27e77fe640 1 -- 172.21.15.120:0/2905650571 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f28000317b0 con 0x560d769ad4a0 2024-09-19T09:23:00.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.118+0000 7f281075c740 1 -- 172.21.15.120:0/2905650571 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2800004030 con 0x560d769ad4a0 2024-09-19T09:23:00.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.118+0000 7f27e77fe640 1 -- 172.21.15.120:0/2905650571 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f280003f040 con 0x560d769ad4a0 2024-09-19T09:23:00.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.118+0000 7f281075c740 10 client.30673 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:00.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.118+0000 7f281075c740 1 -- 172.21.15.120:0/2905650571 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560d769bb250 con 0x560d769ad4a0 2024-09-19T09:23:00.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.118+0000 7f27e77fe640 1 -- 172.21.15.120:0/2905650571 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2800004030 con 0x560d769ad4a0 2024-09-19T09:23:00.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.118+0000 7f27e77fe640 1 -- 172.21.15.120:0/2905650571 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f280004b020 con 0x560d769ad4a0 2024-09-19T09:23:00.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.118+0000 7f281075c740 10 client.30673 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:00.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.118+0000 7f281075c740 10 client.30673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:23:00.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.118+0000 7f281075c740 20 client.30673 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:00.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.118+0000 7f281075c740 1 --2- 172.21.15.120:0/2905650571 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560d769c7d30 0x560d769ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:00.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.119+0000 7f281075c740 4 client.30673 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:23:00.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.119+0000 7f281075c740 1 -- 172.21.15.120:0/2905650571 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d76920850 con 0x560d769c7d30 2024-09-19T09:23:00.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.119+0000 7f28077fe640 1 --2- 172.21.15.120:0/2905650571 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560d769c7d30 0x560d769ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:00.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.120+0000 7f28077fe640 1 --2- 172.21.15.120:0/2905650571 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560d769c7d30 0x560d769ca120 crc :-1 s=READY pgs=501 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8917c4e6caa64a2 server_cookie=6e0f7fdcbeea8460 in_seq=0 out_seq=0 2024-09-19T09:23:00.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.120+0000 7f27e77fe640 10 client.30673 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:00.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.125+0000 7f27e77fe640 1 -- 172.21.15.120:0/2905650571 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d76920850 con 0x560d769c7d30 2024-09-19T09:23:00.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.125+0000 7f27e77fe640 10 client.30673 handle_command_reply: tid=0 2024-09-19T09:23:00.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.125+0000 7f281075c740 1 client.30673 shutdown 2024-09-19T09:23:00.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.126+0000 7f281075c740 1 -- 172.21.15.120:0/2905650571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d769ad4a0 msgr2=0x560d769b7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:00.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.126+0000 7f281075c740 1 --2- 172.21.15.120:0/2905650571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d769ad4a0 0x560d769b7f40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2800009850 tx=0x7f2800009520 comp rx=0 tx=0).stop 2024-09-19T09:23:00.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.126+0000 7f281075c740 1 -- 172.21.15.120:0/2905650571 shutdown_connections 2024-09-19T09:23:00.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.126+0000 7f281075c740 1 --2- 172.21.15.120:0/2905650571 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560d769c7d30 0x560d769ca120 crc :-1 s=READY pgs=501 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.126+0000 7f281075c740 1 --2- 172.21.15.120:0/2905650571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d769b8680 0x560d769b5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.126+0000 7f281075c740 1 --2- 172.21.15.120:0/2905650571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d769addc0 0x560d769b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.126+0000 7f281075c740 1 --2- 172.21.15.120:0/2905650571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d769ad4a0 0x560d769b7f40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.126+0000 7f281075c740 1 -- 172.21.15.120:0/2905650571 >> 172.21.15.120:0/2905650571 conn(0x560d769a9a30 msgr2=0x560d769a8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:00.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.126+0000 7f27e77fe640 0 client.30673 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:00.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.126+0000 7f281075c740 1 -- 172.21.15.120:0/2905650571 shutdown_connections 2024-09-19T09:23:00.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.126+0000 7f281075c740 1 -- 172.21.15.120:0/2905650571 wait complete. 2024-09-19T09:23:00.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.126+0000 7f281075c740 20 client.30673 trim_cache size 0 max 16384 2024-09-19T09:23:00.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.151+0000 7f281075c740 1 Processor -- start 2024-09-19T09:23:00.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.151+0000 7f281075c740 1 -- start start 2024-09-19T09:23:00.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.151+0000 7f281075c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d769ad4a0 0x560d769b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:00.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.151+0000 7f281075c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d769addc0 0x560d769cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:00.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.151+0000 7f281075c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d769b8680 0x560d769cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:00.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.151+0000 7f281075c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f28081b3ce0 con 0x560d769ad4a0 2024-09-19T09:23:00.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.151+0000 7f281075c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f28081b5820 con 0x560d769b8680 2024-09-19T09:23:00.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.151+0000 7f281075c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f28081b43b0 con 0x560d769addc0 2024-09-19T09:23:00.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.151+0000 7f2807fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d769ad4a0 0x560d769b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:00.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.151+0000 7f280c920640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d769b8680 0x560d769cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:00.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.151+0000 7f2807fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d769ad4a0 0x560d769b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42778/0 (socket says 172.21.15.120:42778) 2024-09-19T09:23:00.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.151+0000 7f2807fff640 1 -- 172.21.15.120:0/1807740343 learned_addr learned my addr 172.21.15.120:0/1807740343 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:00.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.151+0000 7f28077fe640 1 --2- 172.21.15.120:0/1807740343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d769addc0 0x560d769cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:00.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.152+0000 7f2807fff640 1 -- 172.21.15.120:0/1807740343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d769addc0 msgr2=0x560d769cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:00.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.152+0000 7f2807fff640 1 --2- 172.21.15.120:0/1807740343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d769addc0 0x560d769cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.152+0000 7f2807fff640 1 -- 172.21.15.120:0/1807740343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d769b8680 msgr2=0x560d769cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:00.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.152+0000 7f2807fff640 1 --2- 172.21.15.120:0/1807740343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d769b8680 0x560d769cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.152+0000 7f2807fff640 1 -- 172.21.15.120:0/1807740343 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d769ae330 con 0x560d769ad4a0 2024-09-19T09:23:00.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.152+0000 7f280c920640 1 --2- 172.21.15.120:0/1807740343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d769b8680 0x560d769cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:00.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.152+0000 7f2807fff640 1 --2- 172.21.15.120:0/1807740343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d769ad4a0 0x560d769b4460 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f2800005e80 tx=0x7f280004d730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25eba0d9096c33ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:00.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.152+0000 7f27fdffb640 1 -- 172.21.15.120:0/1807740343 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2800035070 con 0x560d769ad4a0 2024-09-19T09:23:00.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.152+0000 7f27fdffb640 1 -- 172.21.15.120:0/1807740343 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f280004dbd0 con 0x560d769ad4a0 2024-09-19T09:23:00.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.152+0000 7f27fdffb640 1 -- 172.21.15.120:0/1807740343 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280003b690 con 0x560d769ad4a0 2024-09-19T09:23:00.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.153+0000 7f281075c740 1 -- 172.21.15.120:0/1807740343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d769ad4a0 msgr2=0x560d769b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:00.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.153+0000 7f281075c740 1 --2- 172.21.15.120:0/1807740343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d769ad4a0 0x560d769b4460 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f2800005e80 tx=0x7f280004d730 comp rx=0 tx=0).stop 2024-09-19T09:23:00.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.154+0000 7f281075c740 1 -- 172.21.15.120:0/1807740343 shutdown_connections 2024-09-19T09:23:00.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.154+0000 7f281075c740 1 --2- 172.21.15.120:0/1807740343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d769b8680 0x560d769cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.154+0000 7f281075c740 1 --2- 172.21.15.120:0/1807740343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d769addc0 0x560d769cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.154+0000 7f281075c740 1 --2- 172.21.15.120:0/1807740343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d769ad4a0 0x560d769b4460 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.154+0000 7f281075c740 1 -- 172.21.15.120:0/1807740343 >> 172.21.15.120:0/1807740343 conn(0x560d769a9a30 msgr2=0x560d769a8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:00.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.154+0000 7f281075c740 1 -- 172.21.15.120:0/1807740343 shutdown_connections 2024-09-19T09:23:00.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.154+0000 7f281075c740 1 -- 172.21.15.120:0/1807740343 wait complete. 2024-09-19T09:23:00.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.154+0000 7f281075c740 1 -- start start 2024-09-19T09:23:00.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.155+0000 7f281075c740 1 Processor -- start 2024-09-19T09:23:00.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.155+0000 7f281075c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d769ad4a0 0x560d769c0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:00.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.155+0000 7f281075c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d769addc0 0x560d769c0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:00.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.155+0000 7f281075c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d769b8680 0x560d769bdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:00.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.155+0000 7f2807fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d769ad4a0 0x560d769c0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:00.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.155+0000 7f2807fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d769ad4a0 0x560d769c0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:47094/0 (socket says 172.21.15.120:47094) 2024-09-19T09:23:00.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.155+0000 7f28077fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d769addc0 0x560d769c0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:00.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.155+0000 7f2807fff640 1 -- 172.21.15.120:0/2208123492 learned_addr learned my addr 172.21.15.120:0/2208123492 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:00.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.155+0000 7f281075c740 1 -- 172.21.15.120:0/2208123492 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f28081b5150 con 0x560d769addc0 2024-09-19T09:23:00.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.155+0000 7f280c920640 1 --2- 172.21.15.120:0/2208123492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d769b8680 0x560d769bdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:00.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.155+0000 7f281075c740 1 -- 172.21.15.120:0/2208123492 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f28081b4a80 con 0x560d769b8680 2024-09-19T09:23:00.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.155+0000 7f281075c740 1 -- 172.21.15.120:0/2208123492 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f28081b1ca0 con 0x560d769ad4a0 2024-09-19T09:23:00.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.156+0000 7f281075c740 10 client.0 _ll_register_callbacks cb 0x7f27f80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:00.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.156+0000 7f280c920640 1 -- 172.21.15.120:0/2208123492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d769ad4a0 msgr2=0x560d769c0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:00.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.156+0000 7f280c920640 1 --2- 172.21.15.120:0/2208123492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d769ad4a0 0x560d769c0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.156+0000 7f280c920640 1 -- 172.21.15.120:0/2208123492 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d769addc0 msgr2=0x560d769c0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:00.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.156+0000 7f280c920640 1 --2- 172.21.15.120:0/2208123492 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d769addc0 0x560d769c0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.156+0000 7f280c920640 1 -- 172.21.15.120:0/2208123492 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d769be7d0 con 0x560d769b8680 2024-09-19T09:23:00.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.157+0000 7f280c920640 1 --2- 172.21.15.120:0/2208123492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d769b8680 0x560d769bdfa0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f27f00cd820 tx=0x7f27f009cb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad8a4a6cee1209b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:00.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.157+0000 7f27e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:23:00.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.157+0000 7f27e77fe640 1 -- 172.21.15.120:0/2208123492 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f00b4070 con 0x560d769b8680 2024-09-19T09:23:00.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.157+0000 7f27e77fe640 1 -- 172.21.15.120:0/2208123492 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f27f00cd930 con 0x560d769b8680 2024-09-19T09:23:00.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.157+0000 7f281075c740 1 -- 172.21.15.120:0/2208123492 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f280004dbd0 con 0x560d769b8680 2024-09-19T09:23:00.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.157+0000 7f27e77fe640 1 -- 172.21.15.120:0/2208123492 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f27f00a4fb0 con 0x560d769b8680 2024-09-19T09:23:00.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.157+0000 7f281075c740 10 client.30821 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:00.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.157+0000 7f281075c740 1 -- 172.21.15.120:0/2208123492 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560d76a32520 con 0x560d769b8680 2024-09-19T09:23:00.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.158+0000 7f27e77fe640 1 -- 172.21.15.120:0/2208123492 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27f0005310 con 0x560d769b8680 2024-09-19T09:23:00.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.158+0000 7f27e77fe640 1 -- 172.21.15.120:0/2208123492 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f27f00cdce0 con 0x560d769b8680 2024-09-19T09:23:00.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.158+0000 7f281075c740 10 client.30821 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:00.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.158+0000 7f281075c740 10 client.30821 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:23:00.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.158+0000 7f281075c740 20 client.30821 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:00.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.158+0000 7f281075c740 1 --2- 172.21.15.120:0/2208123492 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560d76a32c90 0x560d76a35080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:00.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.158+0000 7f281075c740 4 client.30821 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:23:00.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.158+0000 7f281075c740 1 -- 172.21.15.120:0/2208123492 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f280004dbd0 con 0x560d76a32c90 2024-09-19T09:23:00.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.158+0000 7f2807fff640 1 --2- 172.21.15.120:0/2208123492 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560d76a32c90 0x560d76a35080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:00.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.159+0000 7f27e77fe640 10 client.30821 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:00.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.159+0000 7f2807fff640 1 --2- 172.21.15.120:0/2208123492 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560d76a32c90 0x560d76a35080 crc :-1 s=READY pgs=502 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=239cd75b8d8c866 server_cookie=109fc6b690df26f5 in_seq=0 out_seq=0 2024-09-19T09:23:00.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.161+0000 7f27e77fe640 1 -- 172.21.15.120:0/2208123492 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f280004dbd0 con 0x560d76a32c90 2024-09-19T09:23:00.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.161+0000 7f27e77fe640 10 client.30821 handle_command_reply: tid=0 2024-09-19T09:23:00.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.161+0000 7f281075c740 1 client.30821 shutdown 2024-09-19T09:23:00.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.163+0000 7f281075c740 1 -- 172.21.15.120:0/2208123492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d769b8680 msgr2=0x560d769bdfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:00.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.163+0000 7f281075c740 1 --2- 172.21.15.120:0/2208123492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d769b8680 0x560d769bdfa0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f27f00cd820 tx=0x7f27f009cb00 comp rx=0 tx=0).stop 2024-09-19T09:23:00.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.163+0000 7f281075c740 1 -- 172.21.15.120:0/2208123492 shutdown_connections 2024-09-19T09:23:00.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.163+0000 7f281075c740 1 --2- 172.21.15.120:0/2208123492 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560d76a32c90 0x560d76a35080 crc :-1 s=READY pgs=502 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.163+0000 7f27e77fe640 0 client.30821 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:00.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.163+0000 7f281075c740 1 --2- 172.21.15.120:0/2208123492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d769b8680 0x560d769bdfa0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.163+0000 7f281075c740 1 --2- 172.21.15.120:0/2208123492 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560d769addc0 0x560d769c0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.163+0000 7f281075c740 1 --2- 172.21.15.120:0/2208123492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d769ad4a0 0x560d769c0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.163+0000 7f281075c740 1 -- 172.21.15.120:0/2208123492 >> 172.21.15.120:0/2208123492 conn(0x560d769a9a30 msgr2=0x560d769b6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:00.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.163+0000 7f281075c740 1 -- 172.21.15.120:0/2208123492 shutdown_connections 2024-09-19T09:23:00.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.164+0000 7f281075c740 1 -- 172.21.15.120:0/2208123492 wait complete. 2024-09-19T09:23:00.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.164+0000 7f281075c740 20 client.30821 trim_cache size 0 max 16384 2024-09-19T09:23:00.166 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:23:00.166 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:23:00.166 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:23:00.166 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:23:00.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.165+0000 7f280e383640 1 -- 172.21.15.120:0/4132879468 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f27f40a3340 msgr2=0x7f27f40a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:00.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.165+0000 7f280e383640 1 --2- 172.21.15.120:0/4132879468 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f27f40a3340 0x7f27f40a5800 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009a00 tx=0x7f27f8009990 comp rx=0 tx=0).stop 2024-09-19T09:23:00.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.165+0000 7f280e383640 1 -- 172.21.15.120:0/4132879468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28081082a0 msgr2=0x7f280810de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:00.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.165+0000 7f280e383640 1 --2- 172.21.15.120:0/4132879468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28081082a0 0x7f280810de00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f27f000e970 tx=0x7f27f000ed70 comp rx=0 tx=0).stop 2024-09-19T09:23:00.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.165+0000 7f280e383640 1 -- 172.21.15.120:0/4132879468 shutdown_connections 2024-09-19T09:23:00.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.165+0000 7f280e383640 1 --2- 172.21.15.120:0/4132879468 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f27f40a3340 0x7f27f40a5800 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.165+0000 7f280e383640 1 --2- 172.21.15.120:0/4132879468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28081082a0 0x7f280810de00 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.165+0000 7f280e383640 1 --2- 172.21.15.120:0/4132879468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28081036e0 0x7f28081a4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.165+0000 7f280e383640 1 --2- 172.21.15.120:0/4132879468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2808102d30 0x7f28081a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.165+0000 7f280e383640 1 -- 172.21.15.120:0/4132879468 >> 172.21.15.120:0/4132879468 conn(0x7f28080fc7d0 msgr2=0x7f2808100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:00.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.166+0000 7f280e383640 1 -- 172.21.15.120:0/4132879468 shutdown_connections 2024-09-19T09:23:00.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:00.166+0000 7f280e383640 1 -- 172.21.15.120:0/4132879468 wait complete. 2024-09-19T09:23:00.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:00 smithi120 ceph-mon[31150]: pgmap v949: 97 pgs: 97 active+clean; 402 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 2.8 MiB/s wr, 292 op/s 2024-09-19T09:23:00.376 INFO:tasks.workunit.client.0.smithi120.stdout: 10 163650 19.70 MB/sec execute 295 sec latency 50.160 ms 2024-09-19T09:23:00.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:00 smithi196 ceph-mon[35150]: pgmap v949: 97 pgs: 97 active+clean; 402 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 2.8 MiB/s wr, 292 op/s 2024-09-19T09:23:00.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:00 smithi153 ceph-mon[35071]: pgmap v949: 97 pgs: 97 active+clean; 402 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 2.8 MiB/s wr, 292 op/s 2024-09-19T09:23:01.105 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:23:01.106 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:23:01.376 INFO:tasks.workunit.client.0.smithi120.stdout: 10 164390 19.83 MB/sec execute 296 sec latency 54.156 ms 2024-09-19T09:23:01.434 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:23:02.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.991+0000 7f5d30b70640 1 -- 172.21.15.120:0/3983781092 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d2c102ef0 msgr2=0x7f5d2c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:02.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.991+0000 7f5d30b70640 1 --2- 172.21.15.120:0/3983781092 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d2c102ef0 0x7f5d2c10da40 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f5d24009ca0 tx=0x7f5d2402a950 comp rx=0 tx=0).stop 2024-09-19T09:23:02.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.991+0000 7f5d30b70640 1 -- 172.21.15.120:0/3983781092 shutdown_connections 2024-09-19T09:23:02.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.991+0000 7f5d30b70640 1 --2- 172.21.15.120:0/3983781092 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d2c102ef0 0x7f5d2c10da40 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.991+0000 7f5d30b70640 1 --2- 172.21.15.120:0/3983781092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d2c102570 0x7f5d2c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.991+0000 7f5d30b70640 1 --2- 172.21.15.120:0/3983781092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d2c100e20 0x7f5d2c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.991+0000 7f5d30b70640 1 -- 172.21.15.120:0/3983781092 >> 172.21.15.120:0/3983781092 conn(0x7f5d2c0fc7d0 msgr2=0x7f5d2c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:02.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.991+0000 7f5d30b70640 1 -- 172.21.15.120:0/3983781092 shutdown_connections 2024-09-19T09:23:02.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.991+0000 7f5d30b70640 1 -- 172.21.15.120:0/3983781092 wait complete. 2024-09-19T09:23:02.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.992+0000 7f5d30b70640 1 Processor -- start 2024-09-19T09:23:02.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.992+0000 7f5d30b70640 1 -- start start 2024-09-19T09:23:02.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.992+0000 7f5d30b70640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d2c100e20 0x7f5d2c197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:02.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.992+0000 7f5d30b70640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d2c102570 0x7f5d2c197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:02.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.993+0000 7f5d30b70640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d2c102ef0 0x7f5d2c19bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:02.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.993+0000 7f5d30b70640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c110f80 con 0x7f5d2c102ef0 2024-09-19T09:23:02.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.993+0000 7f5d30b70640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c110e00 con 0x7f5d2c100e20 2024-09-19T09:23:02.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.993+0000 7f5d30b70640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c111100 con 0x7f5d2c102570 2024-09-19T09:23:02.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.993+0000 7f5d2ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d2c102ef0 0x7f5d2c19bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:02.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.993+0000 7f5d2ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d2c102ef0 0x7f5d2c19bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41896/0 (socket says 172.21.15.120:41896) 2024-09-19T09:23:02.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.993+0000 7f5d2ad9d640 1 -- 172.21.15.120:0/2665159235 learned_addr learned my addr 172.21.15.120:0/2665159235 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:02.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.993+0000 7f5d2a59c640 1 --2- 172.21.15.120:0/2665159235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d2c100e20 0x7f5d2c197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:02.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.993+0000 7f5d29d9b640 1 --2- 172.21.15.120:0/2665159235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d2c102570 0x7f5d2c197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:02.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.993+0000 7f5d2ad9d640 1 -- 172.21.15.120:0/2665159235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d2c102570 msgr2=0x7f5d2c197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:02.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.993+0000 7f5d2ad9d640 1 --2- 172.21.15.120:0/2665159235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d2c102570 0x7f5d2c197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.993+0000 7f5d2ad9d640 1 -- 172.21.15.120:0/2665159235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d2c100e20 msgr2=0x7f5d2c197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:02.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.993+0000 7f5d2ad9d640 1 --2- 172.21.15.120:0/2665159235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d2c100e20 0x7f5d2c197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.993+0000 7f5d2ad9d640 1 -- 172.21.15.120:0/2665159235 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d2c19c2f0 con 0x7f5d2c102ef0 2024-09-19T09:23:02.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.993+0000 7f5d29d9b640 1 --2- 172.21.15.120:0/2665159235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d2c102570 0x7f5d2c197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:23:02.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.993+0000 7f5d2ad9d640 1 --2- 172.21.15.120:0/2665159235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d2c102ef0 0x7f5d2c19bc10 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f5d2402ae70 tx=0x7f5d2402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:02.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.993+0000 7f5d1b7fe640 1 -- 172.21.15.120:0/2665159235 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d24034040 con 0x7f5d2c102ef0 2024-09-19T09:23:02.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.993+0000 7f5d30b70640 1 -- 172.21.15.120:0/2665159235 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d2c19c580 con 0x7f5d2c102ef0 2024-09-19T09:23:02.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.993+0000 7f5d1b7fe640 1 -- 172.21.15.120:0/2665159235 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d24030920 con 0x7f5d2c102ef0 2024-09-19T09:23:02.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.993+0000 7f5d1b7fe640 1 -- 172.21.15.120:0/2665159235 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d24045480 con 0x7f5d2c102ef0 2024-09-19T09:23:02.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.993+0000 7f5d30b70640 1 -- 172.21.15.120:0/2665159235 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d2c1a3ec0 con 0x7f5d2c102ef0 2024-09-19T09:23:02.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.994+0000 7f5d30b70640 1 -- 172.21.15.120:0/2665159235 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d2c101200 con 0x7f5d2c102ef0 2024-09-19T09:23:02.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.995+0000 7f5d1b7fe640 1 -- 172.21.15.120:0/2665159235 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5d24037020 con 0x7f5d2c102ef0 2024-09-19T09:23:02.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.996+0000 7f5d1b7fe640 1 --2- 172.21.15.120:0/2665159235 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5d140a3240 0x7f5d140a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:02.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.996+0000 7f5d2a59c640 1 --2- 172.21.15.120:0/2665159235 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5d140a3240 0x7f5d140a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:02.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.996+0000 7f5d1b7fe640 1 -- 172.21.15.120:0/2665159235 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f5d240f0840 con 0x7f5d2c102ef0 2024-09-19T09:23:02.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:01.997+0000 7f5d2a59c640 1 --2- 172.21.15.120:0/2665159235 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5d140a3240 0x7f5d140a5700 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c005d00 tx=0x7f5d1c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:02.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:02.000+0000 7f5d1b7fe640 1 -- 172.21.15.120:0/2665159235 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5d240bd560 con 0x7f5d2c102ef0 2024-09-19T09:23:02.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:02.281+0000 7f5d30b70640 1 -- 172.21.15.120:0/2665159235 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d2c1a4090 con 0x7f5d2c102ef0 2024-09-19T09:23:02.283 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:02 smithi120 ceph-mon[31150]: pgmap v950: 97 pgs: 97 active+clean; 401 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.6 MiB/s wr, 350 op/s 2024-09-19T09:23:02.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:02.282+0000 7f5d1b7fe640 1 -- 172.21.15.120:0/2665159235 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11988 (secure 0 0 0) 0x7f5d240bce60 con 0x7f5d2c102ef0 2024-09-19T09:23:02.284 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:23:02.284 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":39,"btime":"2024-09-19T09:18:53:120858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:18:53.070639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:23:02.284 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 39 2024-09-19T09:23:02.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:02.284+0000 7f5d30b70640 1 -- 172.21.15.120:0/2665159235 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5d140a3240 msgr2=0x7f5d140a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:02.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:02.284+0000 7f5d30b70640 1 --2- 172.21.15.120:0/2665159235 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5d140a3240 0x7f5d140a5700 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c005d00 tx=0x7f5d1c00b410 comp rx=0 tx=0).stop 2024-09-19T09:23:02.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:02.284+0000 7f5d30b70640 1 -- 172.21.15.120:0/2665159235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d2c102ef0 msgr2=0x7f5d2c19bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:02.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:02.284+0000 7f5d30b70640 1 --2- 172.21.15.120:0/2665159235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d2c102ef0 0x7f5d2c19bc10 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f5d2402ae70 tx=0x7f5d2402f5d0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:02.284+0000 7f5d30b70640 1 -- 172.21.15.120:0/2665159235 shutdown_connections 2024-09-19T09:23:02.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:02.284+0000 7f5d30b70640 1 --2- 172.21.15.120:0/2665159235 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5d140a3240 0x7f5d140a5700 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:02.284+0000 7f5d30b70640 1 --2- 172.21.15.120:0/2665159235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d2c102ef0 0x7f5d2c19bc10 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:02.284+0000 7f5d30b70640 1 --2- 172.21.15.120:0/2665159235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d2c102570 0x7f5d2c197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:02.284+0000 7f5d30b70640 1 --2- 172.21.15.120:0/2665159235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d2c100e20 0x7f5d2c197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:02.284+0000 7f5d30b70640 1 -- 172.21.15.120:0/2665159235 >> 172.21.15.120:0/2665159235 conn(0x7f5d2c0fc7d0 msgr2=0x7f5d2c106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:02.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:02.284+0000 7f5d30b70640 1 -- 172.21.15.120:0/2665159235 shutdown_connections 2024-09-19T09:23:02.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:02.284+0000 7f5d30b70640 1 -- 172.21.15.120:0/2665159235 wait complete. 2024-09-19T09:23:02.376 INFO:tasks.workunit.client.0.smithi120.stdout: 10 165025 19.89 MB/sec execute 297 sec latency 58.753 ms 2024-09-19T09:23:02.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:02 smithi196 ceph-mon[35150]: pgmap v950: 97 pgs: 97 active+clean; 401 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.6 MiB/s wr, 350 op/s 2024-09-19T09:23:02.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:02 smithi153 ceph-mon[35071]: pgmap v950: 97 pgs: 97 active+clean; 401 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.6 MiB/s wr, 350 op/s 2024-09-19T09:23:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.665+0000 7f095c503640 1 -- 172.21.15.120:0/1077868763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0954102600 msgr2=0x7f0954108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.665+0000 7f095c503640 1 --2- 172.21.15.120:0/1077868763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0954102600 0x7f0954108e90 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f094c009ca0 tx=0x7f094c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.665+0000 7f095c503640 1 -- 172.21.15.120:0/1077868763 shutdown_connections 2024-09-19T09:23:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.665+0000 7f095c503640 1 --2- 172.21.15.120:0/1077868763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0954102600 0x7f0954108e90 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.665+0000 7f095c503640 1 --2- 172.21.15.120:0/1077868763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0954101c40 0x7f09541020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.665+0000 7f095c503640 1 --2- 172.21.15.120:0/1077868763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0954100a50 0x7f0954100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.665+0000 7f095c503640 1 -- 172.21.15.120:0/1077868763 >> 172.21.15.120:0/1077868763 conn(0x7f09540fc1b0 msgr2=0x7f09540fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.665+0000 7f095c503640 1 -- 172.21.15.120:0/1077868763 shutdown_connections 2024-09-19T09:23:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.665+0000 7f095c503640 1 -- 172.21.15.120:0/1077868763 wait complete. 2024-09-19T09:23:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.666+0000 7f095c503640 1 Processor -- start 2024-09-19T09:23:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.666+0000 7f095c503640 1 -- start start 2024-09-19T09:23:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.666+0000 7f095c503640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0954100a50 0x7f095406ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.666+0000 7f095c503640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0954101c40 0x7f095419cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.666+0000 7f095c503640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0954102600 0x7f095419f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.666+0000 7f095c503640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f095410c6d0 con 0x7f0954100a50 2024-09-19T09:23:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.666+0000 7f095c503640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f095410c550 con 0x7f0954102600 2024-09-19T09:23:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.666+0000 7f095c503640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f095410c850 con 0x7f0954101c40 2024-09-19T09:23:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.666+0000 7f095a29f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0954100a50 0x7f095406ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.666+0000 7f095a29f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0954100a50 0x7f095406ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41910/0 (socket says 172.21.15.120:41910) 2024-09-19T09:23:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.666+0000 7f095a29f640 1 -- 172.21.15.120:0/1468365445 learned_addr learned my addr 172.21.15.120:0/1468365445 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.666+0000 7f0959a9e640 1 --2- 172.21.15.120:0/1468365445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0954101c40 0x7f095419cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.666+0000 7f095a29f640 1 -- 172.21.15.120:0/1468365445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0954101c40 msgr2=0x7f095419cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.666+0000 7f095a29f640 1 --2- 172.21.15.120:0/1468365445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0954101c40 0x7f095419cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.666+0000 7f095aaa0640 1 --2- 172.21.15.120:0/1468365445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0954102600 0x7f095419f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.666+0000 7f095a29f640 1 -- 172.21.15.120:0/1468365445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0954102600 msgr2=0x7f095419f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.666+0000 7f095a29f640 1 --2- 172.21.15.120:0/1468365445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0954102600 0x7f095419f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.666+0000 7f095a29f640 1 -- 172.21.15.120:0/1468365445 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095419f9e0 con 0x7f0954100a50 2024-09-19T09:23:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.666+0000 7f095aaa0640 1 --2- 172.21.15.120:0/1468365445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0954102600 0x7f095419f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:23:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.667+0000 7f095a29f640 1 --2- 172.21.15.120:0/1468365445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0954100a50 0x7f095406ae90 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f095000d950 tx=0x7f095000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.667+0000 7f094b7fe640 1 -- 172.21.15.120:0/1468365445 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0950017070 con 0x7f0954100a50 2024-09-19T09:23:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.667+0000 7f094b7fe640 1 -- 172.21.15.120:0/1468365445 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0950009c30 con 0x7f0954100a50 2024-09-19T09:23:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.667+0000 7f094b7fe640 1 -- 172.21.15.120:0/1468365445 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095000f600 con 0x7f0954100a50 2024-09-19T09:23:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.667+0000 7f095c503640 1 -- 172.21.15.120:0/1468365445 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f095419fc70 con 0x7f0954100a50 2024-09-19T09:23:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.667+0000 7f095c503640 1 -- 172.21.15.120:0/1468365445 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09541a01c0 con 0x7f0954100a50 2024-09-19T09:23:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.669+0000 7f094b7fe640 1 -- 172.21.15.120:0/1468365445 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f095001a070 con 0x7f0954100a50 2024-09-19T09:23:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.670+0000 7f095f29b740 1 Processor -- start 2024-09-19T09:23:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.670+0000 7f095f29b740 1 -- start start 2024-09-19T09:23:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.670+0000 7f095f29b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b443cfdb0 0x557b443d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.670+0000 7f095f29b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b443d06f0 0x557b443d51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.670+0000 7f095f29b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557b443d5780 0x557b443d7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.670+0000 7f095f29b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557b4432f2f0 con 0x557b443d5780 2024-09-19T09:23:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.670+0000 7f095f29b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557b442d7270 con 0x557b443cfdb0 2024-09-19T09:23:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.670+0000 7f095f29b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557b4432b2b0 con 0x557b443d06f0 2024-09-19T09:23:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.670+0000 7f0959a9e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557b443d5780 0x557b443d7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.670+0000 7f095aaa0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b443cfdb0 0x557b443d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.670+0000 7f0959a9e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557b443d5780 0x557b443d7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41920/0 (socket says 172.21.15.120:41920) 2024-09-19T09:23:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.670+0000 7f095aaa0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b443cfdb0 0x557b443d01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:42968/0 (socket says 172.21.15.120:42968) 2024-09-19T09:23:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.670+0000 7f094b7fe640 1 --2- 172.21.15.120:0/1468365445 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f09400a3370 0x7f09400a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.670+0000 7f0959a9e640 1 -- 172.21.15.120:0/1796682136 learned_addr learned my addr 172.21.15.120:0/1796682136 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.670+0000 7f095a29f640 1 --2- 172.21.15.120:0/1796682136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b443d06f0 0x557b443d51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.670+0000 7f095aaa0640 1 --2- 172.21.15.120:0/1468365445 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f09400a3370 0x7f09400a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.670+0000 7f0959a9e640 1 -- 172.21.15.120:0/1796682136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b443d06f0 msgr2=0x557b443d51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.670+0000 7f0959a9e640 1 --2- 172.21.15.120:0/1796682136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b443d06f0 0x557b443d51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.670+0000 7f0959a9e640 1 -- 172.21.15.120:0/1796682136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b443cfdb0 msgr2=0x557b443d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.670+0000 7f0959a9e640 1 --2- 172.21.15.120:0/1796682136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b443cfdb0 0x557b443d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.670+0000 7f0959a9e640 1 -- 172.21.15.120:0/1796682136 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b442d6340 con 0x557b443d5780 2024-09-19T09:23:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.670+0000 7f095aaa0640 1 --2- 172.21.15.120:0/1796682136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b443cfdb0 0x557b443d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.670+0000 7f094b7fe640 1 -- 172.21.15.120:0/1468365445 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f09500cf330 con 0x7f0954100a50 2024-09-19T09:23:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.671+0000 7f0959a9e640 1 --2- 172.21.15.120:0/1796682136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557b443d5780 0x557b443d7c40 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f094400bcb0 tx=0x7f09440107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51734d358bd84dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.671+0000 7f0948bfe640 1 -- 172.21.15.120:0/1796682136 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0944004030 con 0x557b443d5780 2024-09-19T09:23:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.671+0000 7f095aaa0640 1 --2- 172.21.15.120:0/1468365445 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f09400a3370 0x7f09400a5830 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f094c02ade0 tx=0x7f094c02a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.671+0000 7f0948bfe640 1 -- 172.21.15.120:0/1796682136 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09440041d0 con 0x557b443d5780 2024-09-19T09:23:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.671+0000 7f0948bfe640 1 -- 172.21.15.120:0/1796682136 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0944011800 con 0x557b443d5780 2024-09-19T09:23:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.672+0000 7f095f29b740 1 -- 172.21.15.120:0/1796682136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557b443d5780 msgr2=0x557b443d7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.672+0000 7f095f29b740 1 --2- 172.21.15.120:0/1796682136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557b443d5780 0x557b443d7c40 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f094400bcb0 tx=0x7f09440107a0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.672+0000 7f095f29b740 1 -- 172.21.15.120:0/1796682136 shutdown_connections 2024-09-19T09:23:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.672+0000 7f095f29b740 1 --2- 172.21.15.120:0/1796682136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557b443d5780 0x557b443d7c40 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.672+0000 7f095f29b740 1 --2- 172.21.15.120:0/1796682136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b443d06f0 0x557b443d51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.672+0000 7f095f29b740 1 --2- 172.21.15.120:0/1796682136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b443cfdb0 0x557b443d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.672+0000 7f095f29b740 1 -- 172.21.15.120:0/1796682136 >> 172.21.15.120:0/1796682136 conn(0x557b443c7010 msgr2=0x557b443c7410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.672+0000 7f095f29b740 1 -- 172.21.15.120:0/1796682136 shutdown_connections 2024-09-19T09:23:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.672+0000 7f095f29b740 1 -- 172.21.15.120:0/1796682136 wait complete. 2024-09-19T09:23:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.672+0000 7f095f29b740 1 -- start start 2024-09-19T09:23:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.672+0000 7f095f29b740 1 Processor -- start 2024-09-19T09:23:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.672+0000 7f095f29b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557b443cfdb0 0x557b443d9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.672+0000 7f095f29b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b443d06f0 0x557b443da4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.672+0000 7f095f29b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b443d5780 0x557b443e1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.672+0000 7f095f29b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557b443d8750 con 0x557b443cfdb0 2024-09-19T09:23:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.672+0000 7f095f29b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557b442b8290 con 0x557b443d06f0 2024-09-19T09:23:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.672+0000 7f095f29b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557b443d88d0 con 0x557b443d5780 2024-09-19T09:23:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.672+0000 7f095f29b740 10 client.0 _ll_register_callbacks cb 0x557b442e9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.672+0000 7f095a29f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557b443cfdb0 0x557b443d9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.672+0000 7f095a29f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557b443cfdb0 0x557b443d9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41922/0 (socket says 172.21.15.120:41922) 2024-09-19T09:23:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.672+0000 7f095a29f640 1 -- 172.21.15.120:0/468647190 learned_addr learned my addr 172.21.15.120:0/468647190 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.673+0000 7f0959a9e640 1 --2- 172.21.15.120:0/468647190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b443d06f0 0x557b443da4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.673+0000 7f095a29f640 1 -- 172.21.15.120:0/468647190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b443d5780 msgr2=0x557b443e1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.673+0000 7f095a29f640 1 --2- 172.21.15.120:0/468647190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b443d5780 0x557b443e1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.673+0000 7f095a29f640 1 -- 172.21.15.120:0/468647190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b443d06f0 msgr2=0x557b443da4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.673+0000 7f095a29f640 1 --2- 172.21.15.120:0/468647190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b443d06f0 0x557b443da4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.673+0000 7f095a29f640 1 -- 172.21.15.120:0/468647190 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b44042210 con 0x557b443cfdb0 2024-09-19T09:23:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.673+0000 7f0959a9e640 1 --2- 172.21.15.120:0/468647190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b443d06f0 0x557b443da4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:23:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.673+0000 7f095a29f640 1 --2- 172.21.15.120:0/468647190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557b443cfdb0 0x557b443d9fa0 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f0950098320 tx=0x7f095009b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90bbb9e46666098e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.673+0000 7f093dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:23:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.673+0000 7f093dffb640 1 -- 172.21.15.120:0/468647190 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0950021030 con 0x557b443cfdb0 2024-09-19T09:23:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.673+0000 7f093dffb640 1 -- 172.21.15.120:0/468647190 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f0950097fc0 con 0x557b443cfdb0 2024-09-19T09:23:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.673+0000 7f095f29b740 1 -- 172.21.15.120:0/468647190 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09440041d0 con 0x557b443cfdb0 2024-09-19T09:23:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.673+0000 7f093dffb640 1 -- 172.21.15.120:0/468647190 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f095009fd00 con 0x557b443cfdb0 2024-09-19T09:23:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.673+0000 7f095f29b740 10 client.22077 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.673+0000 7f095f29b740 1 -- 172.21.15.120:0/468647190 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557b4410e620 con 0x557b443cfdb0 2024-09-19T09:23:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.673+0000 7f093dffb640 1 -- 172.21.15.120:0/468647190 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09440041d0 con 0x557b443cfdb0 2024-09-19T09:23:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.673+0000 7f093dffb640 1 -- 172.21.15.120:0/468647190 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f09500ce070 con 0x557b443cfdb0 2024-09-19T09:23:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.673+0000 7f095f29b740 10 client.22077 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.673+0000 7f095f29b740 10 client.22077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:23:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.673+0000 7f095f29b740 20 client.22077 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.673+0000 7f095f29b740 1 --2- 172.21.15.120:0/468647190 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557b443e5c10 0x557b443e80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.673+0000 7f095f29b740 4 client.22077 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:23:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.673+0000 7f095f29b740 1 -- 172.21.15.120:0/468647190 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b4433ca20 con 0x557b443e5c10 2024-09-19T09:23:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.674+0000 7f0959a9e640 1 --2- 172.21.15.120:0/468647190 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557b443e5c10 0x557b443e80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.674+0000 7f0959a9e640 1 --2- 172.21.15.120:0/468647190 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557b443e5c10 0x557b443e80a0 crc :-1 s=READY pgs=503 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff3316ebe9833c20 server_cookie=ea5315023fc7db12 in_seq=0 out_seq=0 2024-09-19T09:23:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.674+0000 7f093dffb640 10 client.22077 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.675+0000 7f093dffb640 1 -- 172.21.15.120:0/468647190 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557b4433ca20 con 0x557b443e5c10 2024-09-19T09:23:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.675+0000 7f093dffb640 10 client.22077 handle_command_reply: tid=0 2024-09-19T09:23:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.675+0000 7f095f29b740 1 client.22077 shutdown 2024-09-19T09:23:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.675+0000 7f095f29b740 1 -- 172.21.15.120:0/468647190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557b443cfdb0 msgr2=0x557b443d9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.675+0000 7f095f29b740 1 --2- 172.21.15.120:0/468647190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557b443cfdb0 0x557b443d9fa0 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f0950098320 tx=0x7f095009b3b0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.675+0000 7f095f29b740 1 -- 172.21.15.120:0/468647190 shutdown_connections 2024-09-19T09:23:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.675+0000 7f095f29b740 1 --2- 172.21.15.120:0/468647190 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557b443e5c10 0x557b443e80a0 crc :-1 s=READY pgs=503 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.675+0000 7f095f29b740 1 --2- 172.21.15.120:0/468647190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b443d5780 0x557b443e1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.675+0000 7f093dffb640 0 client.22077 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.675+0000 7f095f29b740 1 --2- 172.21.15.120:0/468647190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b443d06f0 0x557b443da4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.675+0000 7f095f29b740 1 --2- 172.21.15.120:0/468647190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557b443cfdb0 0x557b443d9fa0 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.675+0000 7f095f29b740 1 -- 172.21.15.120:0/468647190 >> 172.21.15.120:0/468647190 conn(0x557b443c7010 msgr2=0x557b443c8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.675+0000 7f095f29b740 1 -- 172.21.15.120:0/468647190 shutdown_connections 2024-09-19T09:23:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.675+0000 7f095f29b740 1 -- 172.21.15.120:0/468647190 wait complete. 2024-09-19T09:23:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.675+0000 7f095f29b740 20 client.22077 trim_cache size 0 max 16384 2024-09-19T09:23:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.692+0000 7f095f29b740 1 Processor -- start 2024-09-19T09:23:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.692+0000 7f095f29b740 1 -- start start 2024-09-19T09:23:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.692+0000 7f095f29b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b443cfdb0 0x557b443c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.692+0000 7f095f29b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b443d06f0 0x557b443e14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.692+0000 7f095f29b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557b443d5780 0x557b443ec9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.692+0000 7f095f29b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f09541a6ce0 con 0x557b443d5780 2024-09-19T09:23:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.692+0000 7f095f29b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f09541a8820 con 0x557b443cfdb0 2024-09-19T09:23:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.692+0000 7f095f29b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f09541a73b0 con 0x557b443d06f0 2024-09-19T09:23:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.692+0000 7f095aaa0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557b443d5780 0x557b443ec9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.692+0000 7f095a29f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b443cfdb0 0x557b443c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.692+0000 7f0959a9e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b443d06f0 0x557b443e14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.692+0000 7f095aaa0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557b443d5780 0x557b443ec9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41932/0 (socket says 172.21.15.120:41932) 2024-09-19T09:23:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.693+0000 7f095aaa0640 1 -- 172.21.15.120:0/2586662828 learned_addr learned my addr 172.21.15.120:0/2586662828 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.693+0000 7f095aaa0640 1 -- 172.21.15.120:0/2586662828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b443d06f0 msgr2=0x557b443e14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.693+0000 7f095aaa0640 1 --2- 172.21.15.120:0/2586662828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b443d06f0 0x557b443e14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.693+0000 7f095aaa0640 1 -- 172.21.15.120:0/2586662828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b443cfdb0 msgr2=0x557b443c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.693+0000 7f095aaa0640 1 --2- 172.21.15.120:0/2586662828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b443cfdb0 0x557b443c60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.693+0000 7f095aaa0640 1 -- 172.21.15.120:0/2586662828 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b443d0bd0 con 0x557b443d5780 2024-09-19T09:23:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.693+0000 7f095a29f640 1 --2- 172.21.15.120:0/2586662828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b443cfdb0 0x557b443c60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.693+0000 7f095aaa0640 1 --2- 172.21.15.120:0/2586662828 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557b443d5780 0x557b443ec9f0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f094c031e90 tx=0x7f094c005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe29fa08f329e6f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.693+0000 7f093ffff640 1 -- 172.21.15.120:0/2586662828 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c044070 con 0x557b443d5780 2024-09-19T09:23:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.693+0000 7f093ffff640 1 -- 172.21.15.120:0/2586662828 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f094c036040 con 0x557b443d5780 2024-09-19T09:23:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.693+0000 7f093ffff640 1 -- 172.21.15.120:0/2586662828 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c02f870 con 0x557b443d5780 2024-09-19T09:23:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.694+0000 7f095f29b740 1 -- 172.21.15.120:0/2586662828 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557b443d5780 msgr2=0x557b443ec9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.694+0000 7f095f29b740 1 --2- 172.21.15.120:0/2586662828 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557b443d5780 0x557b443ec9f0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f094c031e90 tx=0x7f094c005c50 comp rx=0 tx=0).stop 2024-09-19T09:23:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.694+0000 7f095f29b740 1 -- 172.21.15.120:0/2586662828 shutdown_connections 2024-09-19T09:23:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.694+0000 7f095f29b740 1 --2- 172.21.15.120:0/2586662828 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557b443d5780 0x557b443ec9f0 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.694+0000 7f095f29b740 1 --2- 172.21.15.120:0/2586662828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b443d06f0 0x557b443e14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.694+0000 7f095f29b740 1 --2- 172.21.15.120:0/2586662828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b443cfdb0 0x557b443c60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.694+0000 7f095f29b740 1 -- 172.21.15.120:0/2586662828 >> 172.21.15.120:0/2586662828 conn(0x557b443c7010 msgr2=0x557b443c59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.694+0000 7f095f29b740 1 -- 172.21.15.120:0/2586662828 shutdown_connections 2024-09-19T09:23:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.694+0000 7f095f29b740 1 -- 172.21.15.120:0/2586662828 wait complete. 2024-09-19T09:23:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.694+0000 7f095f29b740 1 -- start start 2024-09-19T09:23:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.694+0000 7f095f29b740 1 Processor -- start 2024-09-19T09:23:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.695+0000 7f095f29b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b443cfdb0 0x557b443deff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.695+0000 7f095f29b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b443d06f0 0x557b443df530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.695+0000 7f095f29b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557b443d5780 0x557b443e0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.695+0000 7f095f29b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f09541a8150 con 0x557b443d5780 2024-09-19T09:23:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.695+0000 7f095f29b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f09541a7a80 con 0x557b443d06f0 2024-09-19T09:23:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.695+0000 7f095f29b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f09541a4ca0 con 0x557b443cfdb0 2024-09-19T09:23:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.695+0000 7f095f29b740 10 client.0 _ll_register_callbacks cb 0x7f095009b490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.695+0000 7f095aaa0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557b443d5780 0x557b443e0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.695+0000 7f095aaa0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557b443d5780 0x557b443e0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41944/0 (socket says 172.21.15.120:41944) 2024-09-19T09:23:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.695+0000 7f095aaa0640 1 -- 172.21.15.120:0/2730780065 learned_addr learned my addr 172.21.15.120:0/2730780065 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.695+0000 7f0959a9e640 1 --2- 172.21.15.120:0/2730780065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b443d06f0 0x557b443df530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.695+0000 7f095a29f640 1 --2- 172.21.15.120:0/2730780065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b443cfdb0 0x557b443deff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.695+0000 7f095aaa0640 1 -- 172.21.15.120:0/2730780065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b443cfdb0 msgr2=0x557b443deff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.695+0000 7f095aaa0640 1 --2- 172.21.15.120:0/2730780065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b443cfdb0 0x557b443deff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.695+0000 7f095aaa0640 1 -- 172.21.15.120:0/2730780065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b443d06f0 msgr2=0x557b443df530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.695+0000 7f095aaa0640 1 --2- 172.21.15.120:0/2730780065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b443d06f0 0x557b443df530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.695+0000 7f095aaa0640 1 -- 172.21.15.120:0/2730780065 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b443e0b50 con 0x557b443d5780 2024-09-19T09:23:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.695+0000 7f095aaa0640 1 --2- 172.21.15.120:0/2730780065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557b443d5780 0x557b443e0540 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f094c02f540 tx=0x7f094c003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=140ecea9e009115e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.695+0000 7f093dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:23:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.695+0000 7f093dffb640 1 -- 172.21.15.120:0/2730780065 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c05a070 con 0x557b443d5780 2024-09-19T09:23:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.695+0000 7f093dffb640 1 -- 172.21.15.120:0/2730780065 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f094c02f610 con 0x557b443d5780 2024-09-19T09:23:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.695+0000 7f095f29b740 1 -- 172.21.15.120:0/2730780065 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f094c036040 con 0x557b443d5780 2024-09-19T09:23:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.696+0000 7f093dffb640 1 -- 172.21.15.120:0/2730780065 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f094c044070 con 0x557b443d5780 2024-09-19T09:23:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.696+0000 7f095f29b740 10 client.22083 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.696+0000 7f095f29b740 1 -- 172.21.15.120:0/2730780065 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557b44450d80 con 0x557b443d5780 2024-09-19T09:23:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.696+0000 7f093dffb640 1 -- 172.21.15.120:0/2730780065 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f094c036040 con 0x557b443d5780 2024-09-19T09:23:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.696+0000 7f093dffb640 1 -- 172.21.15.120:0/2730780065 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f094c004b40 con 0x557b443d5780 2024-09-19T09:23:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.696+0000 7f095f29b740 10 client.22083 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.696+0000 7f095f29b740 10 client.22083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:23:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.696+0000 7f095f29b740 20 client.22083 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.696+0000 7f095f29b740 1 --2- 172.21.15.120:0/2730780065 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557b443e5c10 0x557b44453750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.696+0000 7f095f29b740 4 client.22083 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:23:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.696+0000 7f095f29b740 1 -- 172.21.15.120:0/2730780065 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557b44336d70 con 0x557b443e5c10 2024-09-19T09:23:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.696+0000 7f095a29f640 1 --2- 172.21.15.120:0/2730780065 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557b443e5c10 0x557b44453750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.697+0000 7f093dffb640 10 client.22083 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:02.697+0000 7f095a29f640 1 --2- 172.21.15.120:0/2730780065 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557b443e5c10 0x557b44453750 crc :-1 s=READY pgs=504 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff498bb26dc29118 server_cookie=7bf8ec53b6e513b1 in_seq=0 out_seq=0 2024-09-19T09:23:03.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:03.144+0000 7f093dffb640 1 -- 172.21.15.120:0/2730780065 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557b44336d70 con 0x557b443e5c10 2024-09-19T09:23:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:03.144+0000 7f093dffb640 10 client.22083 handle_command_reply: tid=0 2024-09-19T09:23:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:03.144+0000 7f095f29b740 1 client.22083 shutdown 2024-09-19T09:23:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:03.145+0000 7f095f29b740 1 -- 172.21.15.120:0/2730780065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557b443d5780 msgr2=0x557b443e0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:03.145+0000 7f095f29b740 1 --2- 172.21.15.120:0/2730780065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557b443d5780 0x557b443e0540 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f094c02f540 tx=0x7f094c003c00 comp rx=0 tx=0).stop 2024-09-19T09:23:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:03.145+0000 7f095f29b740 1 -- 172.21.15.120:0/2730780065 shutdown_connections 2024-09-19T09:23:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:03.145+0000 7f095f29b740 1 --2- 172.21.15.120:0/2730780065 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557b443e5c10 0x557b44453750 crc :-1 s=READY pgs=504 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:03.145+0000 7f095f29b740 1 --2- 172.21.15.120:0/2730780065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557b443d5780 0x557b443e0540 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:03.145+0000 7f095f29b740 1 --2- 172.21.15.120:0/2730780065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b443d06f0 0x557b443df530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:03.145+0000 7f095f29b740 1 --2- 172.21.15.120:0/2730780065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b443cfdb0 0x557b443deff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:03.145+0000 7f095f29b740 1 -- 172.21.15.120:0/2730780065 >> 172.21.15.120:0/2730780065 conn(0x557b443c7010 msgr2=0x557b443c61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:03.145+0000 7f093dffb640 0 client.22083 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:03.145+0000 7f095f29b740 1 -- 172.21.15.120:0/2730780065 shutdown_connections 2024-09-19T09:23:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:03.145+0000 7f095f29b740 1 -- 172.21.15.120:0/2730780065 wait complete. 2024-09-19T09:23:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:03.145+0000 7f095f29b740 20 client.22083 trim_cache size 0 max 16384 2024-09-19T09:23:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:23:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:23:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:23:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:23:03.149 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:03.146+0000 7f095c503640 1 -- 172.21.15.120:0/1468365445 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f09400a3370 msgr2=0x7f09400a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:03.149 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:03.146+0000 7f095c503640 1 --2- 172.21.15.120:0/1468365445 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f09400a3370 0x7f09400a5830 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f094c02ade0 tx=0x7f094c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:23:03.149 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:03.146+0000 7f095c503640 1 -- 172.21.15.120:0/1468365445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0954100a50 msgr2=0x7f095406ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:03.149 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:03.146+0000 7f095c503640 1 --2- 172.21.15.120:0/1468365445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0954100a50 0x7f095406ae90 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f095000d950 tx=0x7f095000de10 comp rx=0 tx=0).stop 2024-09-19T09:23:03.149 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:03.146+0000 7f095c503640 1 -- 172.21.15.120:0/1468365445 shutdown_connections 2024-09-19T09:23:03.149 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:03.146+0000 7f095c503640 1 --2- 172.21.15.120:0/1468365445 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f09400a3370 0x7f09400a5830 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:03.149 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:03.146+0000 7f095c503640 1 --2- 172.21.15.120:0/1468365445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0954102600 0x7f095419f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:03.149 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:03.146+0000 7f095c503640 1 --2- 172.21.15.120:0/1468365445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0954101c40 0x7f095419cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:03.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:03.146+0000 7f095c503640 1 --2- 172.21.15.120:0/1468365445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0954100a50 0x7f095406ae90 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:03.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:03.146+0000 7f095c503640 1 -- 172.21.15.120:0/1468365445 >> 172.21.15.120:0/1468365445 conn(0x7f09540fc1b0 msgr2=0x7f0954104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:03.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:03.146+0000 7f095c503640 1 -- 172.21.15.120:0/1468365445 shutdown_connections 2024-09-19T09:23:03.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:03.146+0000 7f095c503640 1 -- 172.21.15.120:0/1468365445 wait complete. 2024-09-19T09:23:03.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:03 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2665159235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:23:03.376 INFO:tasks.workunit.client.0.smithi120.stdout: 10 165324 19.89 MB/sec execute 298 sec latency 215.597 ms 2024-09-19T09:23:03.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:03 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2665159235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:23:03.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:03 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2665159235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:23:03.626 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:23:03.957 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:23:04.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:04 smithi120 ceph-mon[31150]: pgmap v951: 97 pgs: 97 active+clean; 393 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.7 MiB/s wr, 439 op/s 2024-09-19T09:23:04.376 INFO:tasks.workunit.client.0.smithi120.stdout: 10 165787 19.88 MB/sec execute 299 sec latency 97.769 ms 2024-09-19T09:23:04.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:04 smithi196 ceph-mon[35150]: pgmap v951: 97 pgs: 97 active+clean; 393 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.7 MiB/s wr, 439 op/s 2024-09-19T09:23:04.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:04 smithi153 ceph-mon[35071]: pgmap v951: 97 pgs: 97 active+clean; 393 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.7 MiB/s wr, 439 op/s 2024-09-19T09:23:04.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.495+0000 7fbcd7054640 1 -- 172.21.15.120:0/964795683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbcd0108290 msgr2=0x7fbcd010ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:04.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.495+0000 7fbcd7054640 1 --2- 172.21.15.120:0/964795683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbcd0108290 0x7fbcd010ec30 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009c10 tx=0x7fbcc802a6e0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.495+0000 7fbcd7054640 1 -- 172.21.15.120:0/964795683 shutdown_connections 2024-09-19T09:23:04.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.495+0000 7fbcd7054640 1 --2- 172.21.15.120:0/964795683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbcd0108290 0x7fbcd010ec30 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.495+0000 7fbcd7054640 1 --2- 172.21.15.120:0/964795683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbcd0103740 0x7fbcd0107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.495+0000 7fbcd7054640 1 --2- 172.21.15.120:0/964795683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd0102d90 0x7fbcd0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.495+0000 7fbcd7054640 1 -- 172.21.15.120:0/964795683 >> 172.21.15.120:0/964795683 conn(0x7fbcd00fc7d0 msgr2=0x7fbcd00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:04.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.495+0000 7fbcd7054640 1 -- 172.21.15.120:0/964795683 shutdown_connections 2024-09-19T09:23:04.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.495+0000 7fbcd7054640 1 -- 172.21.15.120:0/964795683 wait complete. 2024-09-19T09:23:04.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.496+0000 7fbcd7054640 1 Processor -- start 2024-09-19T09:23:04.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.496+0000 7fbcd7054640 1 -- start start 2024-09-19T09:23:04.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.496+0000 7fbcd7054640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd0102d90 0x7fbcd01009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:04.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.496+0000 7fbcd7054640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbcd0103740 0x7fbcd0100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:04.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.496+0000 7fbcd7054640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbcd0108290 0x7fbcd0101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:04.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.496+0000 7fbcd7054640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd0112040 con 0x7fbcd0103740 2024-09-19T09:23:04.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.496+0000 7fbcd7054640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd0111ec0 con 0x7fbcd0108290 2024-09-19T09:23:04.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.496+0000 7fbcd7054640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd01121c0 con 0x7fbcd0102d90 2024-09-19T09:23:04.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.496+0000 7fbccffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbcd0103740 0x7fbcd0100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:04.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.496+0000 7fbcd4df0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd0102d90 0x7fbcd01009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:04.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.496+0000 7fbcd4df0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd0102d90 0x7fbcd01009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:53610/0 (socket says 172.21.15.120:53610) 2024-09-19T09:23:04.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.496+0000 7fbccffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbcd0103740 0x7fbcd0100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41954/0 (socket says 172.21.15.120:41954) 2024-09-19T09:23:04.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.496+0000 7fbcd4df0640 1 -- 172.21.15.120:0/516159790 learned_addr learned my addr 172.21.15.120:0/516159790 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:04.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.496+0000 7fbcd55f1640 1 --2- 172.21.15.120:0/516159790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbcd0108290 0x7fbcd0101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:04.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.496+0000 7fbccffff640 1 -- 172.21.15.120:0/516159790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd0102d90 msgr2=0x7fbcd01009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:04.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.496+0000 7fbccffff640 1 --2- 172.21.15.120:0/516159790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd0102d90 0x7fbcd01009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.496+0000 7fbccffff640 1 -- 172.21.15.120:0/516159790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbcd0108290 msgr2=0x7fbcd0101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:04.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.496+0000 7fbccffff640 1 --2- 172.21.15.120:0/516159790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbcd0108290 0x7fbcd0101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.496+0000 7fbccffff640 1 -- 172.21.15.120:0/516159790 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd01a69e0 con 0x7fbcd0103740 2024-09-19T09:23:04.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.496+0000 7fbcd4df0640 1 --2- 172.21.15.120:0/516159790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd0102d90 0x7fbcd01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:04.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.497+0000 7fbccffff640 1 --2- 172.21.15.120:0/516159790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbcd0103740 0x7fbcd0100f30 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00e9c0 tx=0x7fbcbc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:04.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.497+0000 7fbccdffb640 1 -- 172.21.15.120:0/516159790 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc00ccc0 con 0x7fbcd0103740 2024-09-19T09:23:04.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.497+0000 7fbccdffb640 1 -- 172.21.15.120:0/516159790 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcbc010910 con 0x7fbcd0103740 2024-09-19T09:23:04.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.497+0000 7fbcd7054640 1 -- 172.21.15.120:0/516159790 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd01a6cd0 con 0x7fbcd0103740 2024-09-19T09:23:04.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.497+0000 7fbccdffb640 1 -- 172.21.15.120:0/516159790 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc0188c0 con 0x7fbcd0103740 2024-09-19T09:23:04.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.497+0000 7fbcd7054640 1 -- 172.21.15.120:0/516159790 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd01a71c0 con 0x7fbcd0103740 2024-09-19T09:23:04.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.499+0000 7fbccdffb640 1 -- 172.21.15.120:0/516159790 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fbcbc010ac0 con 0x7fbcd0103740 2024-09-19T09:23:04.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.500+0000 7fbcd942d740 1 Processor -- start 2024-09-19T09:23:04.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.500+0000 7fbcd942d740 1 -- start start 2024-09-19T09:23:04.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.500+0000 7fbccdffb640 1 --2- 172.21.15.120:0/516159790 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbcc00a3240 0x7fbcc00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:04.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.500+0000 7fbcd942d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e999b89350 0x55e999b878a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:04.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.500+0000 7fbcd55f1640 1 --2- 172.21.15.120:0/516159790 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbcc00a3240 0x7fbcc00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:04.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.500+0000 7fbcd942d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e999b87de0 0x55e999b88220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:04.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.500+0000 7fbccdffb640 1 -- 172.21.15.120:0/516159790 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fbcbc0cfc00 con 0x7fbcd0103740 2024-09-19T09:23:04.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.500+0000 7fbcd942d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e999ba3b90 0x55e999ba5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:04.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.500+0000 7fbcd942d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e999ae52f0 con 0x55e999b87de0 2024-09-19T09:23:04.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.500+0000 7fbcd942d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e999a8ccf0 con 0x55e999ba3b90 2024-09-19T09:23:04.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.500+0000 7fbcd942d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e999ae12b0 con 0x55e999b89350 2024-09-19T09:23:04.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.500+0000 7fbcd4df0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e999b89350 0x55e999b878a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:04.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.500+0000 7fbcd4df0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e999b89350 0x55e999b878a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:53624/0 (socket says 172.21.15.120:53624) 2024-09-19T09:23:04.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.500+0000 7fbcd4df0640 1 -- 172.21.15.120:0/1216823282 learned_addr learned my addr 172.21.15.120:0/1216823282 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:04.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.501+0000 7fbccffff640 1 --2- 172.21.15.120:0/1216823282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e999b87de0 0x55e999b88220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:04.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.501+0000 7fbcd55f1640 1 --2- 172.21.15.120:0/1216823282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e999ba3b90 0x55e999ba5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:04.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.501+0000 7fbcd4df0640 1 -- 172.21.15.120:0/1216823282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e999ba3b90 msgr2=0x55e999ba5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:04.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.501+0000 7fbcd4df0640 1 --2- 172.21.15.120:0/1216823282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e999ba3b90 0x55e999ba5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.501+0000 7fbcd4df0640 1 -- 172.21.15.120:0/1216823282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e999b87de0 msgr2=0x55e999b88220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:04.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.501+0000 7fbcd4df0640 1 --2- 172.21.15.120:0/1216823282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e999b87de0 0x55e999b88220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.501+0000 7fbcd4df0640 1 -- 172.21.15.120:0/1216823282 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e999a713a0 con 0x55e999b89350 2024-09-19T09:23:04.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.501+0000 7fbcd55f1640 1 --2- 172.21.15.120:0/516159790 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbcc00a3240 0x7fbcc00a5700 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8004af0 tx=0x7fbcc8030b00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:04.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.501+0000 7fbcd4df0640 1 --2- 172.21.15.120:0/1216823282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e999b89350 0x55e999b878a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40115c0 tx=0x7fbcc4011a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f86f769559ea98af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:04.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.501+0000 7fbcbaffd640 1 -- 172.21.15.120:0/1216823282 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc400f640 con 0x55e999b89350 2024-09-19T09:23:04.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.501+0000 7fbcbaffd640 1 -- 172.21.15.120:0/1216823282 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcc400fcc0 con 0x55e999b89350 2024-09-19T09:23:04.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.501+0000 7fbcbaffd640 1 -- 172.21.15.120:0/1216823282 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4019e40 con 0x55e999b89350 2024-09-19T09:23:04.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.502+0000 7fbcd942d740 1 -- 172.21.15.120:0/1216823282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e999b89350 msgr2=0x55e999b878a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:04.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.502+0000 7fbcd942d740 1 --2- 172.21.15.120:0/1216823282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e999b89350 0x55e999b878a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40115c0 tx=0x7fbcc4011a80 comp rx=0 tx=0).stop 2024-09-19T09:23:04.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.502+0000 7fbcd942d740 1 -- 172.21.15.120:0/1216823282 shutdown_connections 2024-09-19T09:23:04.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.502+0000 7fbcd942d740 1 --2- 172.21.15.120:0/1216823282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e999ba3b90 0x55e999ba5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.502+0000 7fbcd942d740 1 --2- 172.21.15.120:0/1216823282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e999b87de0 0x55e999b88220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.502+0000 7fbcd942d740 1 --2- 172.21.15.120:0/1216823282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e999b89350 0x55e999b878a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.502+0000 7fbcd942d740 1 -- 172.21.15.120:0/1216823282 >> 172.21.15.120:0/1216823282 conn(0x55e999b48310 msgr2=0x55e999b84470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:04.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.502+0000 7fbcd942d740 1 -- 172.21.15.120:0/1216823282 shutdown_connections 2024-09-19T09:23:04.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.502+0000 7fbcd942d740 1 -- 172.21.15.120:0/1216823282 wait complete. 2024-09-19T09:23:04.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.503+0000 7fbcd942d740 1 -- start start 2024-09-19T09:23:04.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.503+0000 7fbcd942d740 1 Processor -- start 2024-09-19T09:23:04.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.503+0000 7fbcd942d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e999b87de0 0x55e999b924b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:04.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.503+0000 7fbcd942d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e999b89350 0x55e999b8f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:04.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.504+0000 7fbccffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e999b89350 0x55e999b8f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:04.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.504+0000 7fbcd4df0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e999b87de0 0x55e999b924b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:04.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.504+0000 7fbccffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e999b89350 0x55e999b8f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41970/0 (socket says 172.21.15.120:41970) 2024-09-19T09:23:04.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.504+0000 7fbcd4df0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e999b87de0 0x55e999b924b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:53640/0 (socket says 172.21.15.120:53640) 2024-09-19T09:23:04.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.504+0000 7fbccffff640 1 -- 172.21.15.120:0/2683452363 learned_addr learned my addr 172.21.15.120:0/2683452363 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:04.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.504+0000 7fbcd942d740 1 --2- 172.21.15.120:0/2683452363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e999ba3b90 0x55e999b8fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:04.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.504+0000 7fbcd942d740 1 -- 172.21.15.120:0/2683452363 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e999ba6890 con 0x55e999b89350 2024-09-19T09:23:04.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.504+0000 7fbcd942d740 1 -- 172.21.15.120:0/2683452363 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e999aa0060 con 0x55e999ba3b90 2024-09-19T09:23:04.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.504+0000 7fbcd942d740 1 -- 172.21.15.120:0/2683452363 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e999ba6a10 con 0x55e999b87de0 2024-09-19T09:23:04.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.504+0000 7fbcd942d740 10 client.0 _ll_register_callbacks cb 0x55e9997f38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:04.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.504+0000 7fbcd55f1640 1 --2- 172.21.15.120:0/2683452363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e999ba3b90 0x55e999b8fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:04.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.504+0000 7fbcd55f1640 1 -- 172.21.15.120:0/2683452363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e999b87de0 msgr2=0x55e999b924b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:04.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.504+0000 7fbcd55f1640 1 --2- 172.21.15.120:0/2683452363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e999b87de0 0x55e999b924b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.504+0000 7fbcd55f1640 1 -- 172.21.15.120:0/2683452363 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e999b89350 msgr2=0x55e999b8f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:04.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.504+0000 7fbcd55f1640 1 --2- 172.21.15.120:0/2683452363 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e999b89350 0x55e999b8f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.505+0000 7fbcd55f1640 1 -- 172.21.15.120:0/2683452363 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9998563c0 con 0x55e999ba3b90 2024-09-19T09:23:04.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.505+0000 7fbcd4df0640 1 --2- 172.21.15.120:0/2683452363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e999b87de0 0x55e999b924b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:23:04.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.505+0000 7fbccffff640 1 --2- 172.21.15.120:0/2683452363 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e999b89350 0x55e999b8f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:04.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.505+0000 7fbcd55f1640 1 --2- 172.21.15.120:0/2683452363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e999ba3b90 0x55e999b8fca0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbcc802ac00 tx=0x7fbcc802a6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdb26026bcc6e21c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:04.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.505+0000 7fbcaffff640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:23:04.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.505+0000 7fbcaffff640 1 -- 172.21.15.120:0/2683452363 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8050060 con 0x55e999ba3b90 2024-09-19T09:23:04.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.505+0000 7fbcaffff640 1 -- 172.21.15.120:0/2683452363 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fbcc8004450 con 0x55e999ba3b90 2024-09-19T09:23:04.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.505+0000 7fbcd942d740 1 -- 172.21.15.120:0/2683452363 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcc400fcc0 con 0x55e999ba3b90 2024-09-19T09:23:04.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.505+0000 7fbcaffff640 1 -- 172.21.15.120:0/2683452363 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbcc8036050 con 0x55e999ba3b90 2024-09-19T09:23:04.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.505+0000 7fbcd942d740 10 client.30848 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:04.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.505+0000 7fbcd942d740 1 -- 172.21.15.120:0/2683452363 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e999bb2280 con 0x55e999ba3b90 2024-09-19T09:23:04.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.506+0000 7fbcaffff640 1 -- 172.21.15.120:0/2683452363 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcc400fcc0 con 0x55e999ba3b90 2024-09-19T09:23:04.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.506+0000 7fbcaffff640 1 -- 172.21.15.120:0/2683452363 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fbcc8055070 con 0x55e999ba3b90 2024-09-19T09:23:04.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.506+0000 7fbcd942d740 10 client.30848 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:04.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.506+0000 7fbcd942d740 10 client.30848 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:23:04.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.506+0000 7fbcd942d740 20 client.30848 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:04.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.506+0000 7fbcd942d740 1 --2- 172.21.15.120:0/2683452363 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55e999bb3740 0x55e999bb5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:04.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.506+0000 7fbcd942d740 4 client.30848 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:23:04.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.506+0000 7fbcd942d740 1 -- 172.21.15.120:0/2683452363 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e999af2860 con 0x55e999bb3740 2024-09-19T09:23:04.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.507+0000 7fbcd4df0640 1 --2- 172.21.15.120:0/2683452363 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55e999bb3740 0x55e999bb5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:04.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.507+0000 7fbcaffff640 10 client.30848 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:23:04.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.507+0000 7fbcd4df0640 1 --2- 172.21.15.120:0/2683452363 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55e999bb3740 0x55e999bb5b70 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5eb0c52c54002c17 server_cookie=cb7017a92a85b304 in_seq=0 out_seq=0 2024-09-19T09:23:04.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.508+0000 7fbcaffff640 1 -- 172.21.15.120:0/2683452363 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e999af2860 con 0x55e999bb3740 2024-09-19T09:23:04.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.508+0000 7fbcaffff640 10 client.30848 handle_command_reply: tid=0 2024-09-19T09:23:04.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.508+0000 7fbcd942d740 1 client.30848 shutdown 2024-09-19T09:23:04.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.508+0000 7fbcd942d740 1 -- 172.21.15.120:0/2683452363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e999ba3b90 msgr2=0x55e999b8fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:04.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.508+0000 7fbcd942d740 1 --2- 172.21.15.120:0/2683452363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e999ba3b90 0x55e999b8fca0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbcc802ac00 tx=0x7fbcc802a6e0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.508+0000 7fbcd942d740 1 -- 172.21.15.120:0/2683452363 shutdown_connections 2024-09-19T09:23:04.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.508+0000 7fbcd942d740 1 --2- 172.21.15.120:0/2683452363 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55e999bb3740 0x55e999bb5b70 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.508+0000 7fbcaffff640 0 client.30848 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:23:04.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.508+0000 7fbcd942d740 1 --2- 172.21.15.120:0/2683452363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e999ba3b90 0x55e999b8fca0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.508+0000 7fbcd942d740 1 --2- 172.21.15.120:0/2683452363 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e999b89350 0x55e999b8f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.508+0000 7fbcd942d740 1 --2- 172.21.15.120:0/2683452363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e999b87de0 0x55e999b924b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.508+0000 7fbcd942d740 1 -- 172.21.15.120:0/2683452363 >> 172.21.15.120:0/2683452363 conn(0x55e999b48310 msgr2=0x55e999b8a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:04.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.508+0000 7fbcd942d740 1 -- 172.21.15.120:0/2683452363 shutdown_connections 2024-09-19T09:23:04.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.508+0000 7fbcd942d740 1 -- 172.21.15.120:0/2683452363 wait complete. 2024-09-19T09:23:04.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.509+0000 7fbcd942d740 20 client.30848 trim_cache size 0 max 16384 2024-09-19T09:23:04.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.522+0000 7fbcd942d740 1 Processor -- start 2024-09-19T09:23:04.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.522+0000 7fbcd942d740 1 -- start start 2024-09-19T09:23:04.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.522+0000 7fbcd942d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e999b87de0 0x55e999bafb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:04.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.522+0000 7fbcd942d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e999b89350 0x55e999b8a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:04.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.522+0000 7fbcd942d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e999ba3b90 0x55e999b8ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:04.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.522+0000 7fbcd942d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd006fbf0 con 0x55e999b87de0 2024-09-19T09:23:04.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.522+0000 7fbcd942d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd0071730 con 0x55e999b89350 2024-09-19T09:23:04.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.522+0000 7fbcd942d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd00702c0 con 0x55e999ba3b90 2024-09-19T09:23:04.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.522+0000 7fbcd4df0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e999b87de0 0x55e999bafb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:04.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.522+0000 7fbcd4df0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e999b87de0 0x55e999bafb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41982/0 (socket says 172.21.15.120:41982) 2024-09-19T09:23:04.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.522+0000 7fbccffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e999b89350 0x55e999b8a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:04.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.522+0000 7fbcd4df0640 1 -- 172.21.15.120:0/1634586715 learned_addr learned my addr 172.21.15.120:0/1634586715 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:04.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.522+0000 7fbcd55f1640 1 --2- 172.21.15.120:0/1634586715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e999ba3b90 0x55e999b8ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:04.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.523+0000 7fbcd4df0640 1 -- 172.21.15.120:0/1634586715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e999ba3b90 msgr2=0x55e999b8ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:04.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.523+0000 7fbcd4df0640 1 --2- 172.21.15.120:0/1634586715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e999ba3b90 0x55e999b8ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.523+0000 7fbcd4df0640 1 -- 172.21.15.120:0/1634586715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e999b89350 msgr2=0x55e999b8a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:04.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.523+0000 7fbcd4df0640 1 --2- 172.21.15.120:0/1634586715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e999b89350 0x55e999b8a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.523+0000 7fbcd4df0640 1 -- 172.21.15.120:0/1634586715 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e999b902a0 con 0x55e999b87de0 2024-09-19T09:23:04.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.523+0000 7fbcd4df0640 1 --2- 172.21.15.120:0/1634586715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e999b87de0 0x55e999bafb30 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4011510 tx=0x7fbcc4009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec32fabc5d0eb468 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:04.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.523+0000 7fbcba7fc640 1 -- 172.21.15.120:0/1634586715 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4005df0 con 0x55e999b87de0 2024-09-19T09:23:04.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.523+0000 7fbcba7fc640 1 -- 172.21.15.120:0/1634586715 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcc40196a0 con 0x55e999b87de0 2024-09-19T09:23:04.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.523+0000 7fbcba7fc640 1 -- 172.21.15.120:0/1634586715 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc400f660 con 0x55e999b87de0 2024-09-19T09:23:04.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.524+0000 7fbcd942d740 1 -- 172.21.15.120:0/1634586715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e999b87de0 msgr2=0x55e999bafb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:04.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.524+0000 7fbcd942d740 1 --2- 172.21.15.120:0/1634586715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e999b87de0 0x55e999bafb30 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4011510 tx=0x7fbcc4009640 comp rx=0 tx=0).stop 2024-09-19T09:23:04.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.524+0000 7fbcd942d740 1 -- 172.21.15.120:0/1634586715 shutdown_connections 2024-09-19T09:23:04.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.524+0000 7fbcd942d740 1 --2- 172.21.15.120:0/1634586715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e999ba3b90 0x55e999b8ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.524+0000 7fbcd942d740 1 --2- 172.21.15.120:0/1634586715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e999b89350 0x55e999b8a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.524+0000 7fbcd942d740 1 --2- 172.21.15.120:0/1634586715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e999b87de0 0x55e999bafb30 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.524+0000 7fbcd942d740 1 -- 172.21.15.120:0/1634586715 >> 172.21.15.120:0/1634586715 conn(0x55e999b48310 msgr2=0x55e999b917e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:04.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.524+0000 7fbcd942d740 1 -- 172.21.15.120:0/1634586715 shutdown_connections 2024-09-19T09:23:04.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.524+0000 7fbcd942d740 1 -- 172.21.15.120:0/1634586715 wait complete. 2024-09-19T09:23:04.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.524+0000 7fbcd942d740 1 -- start start 2024-09-19T09:23:04.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.525+0000 7fbcd942d740 1 Processor -- start 2024-09-19T09:23:04.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.525+0000 7fbcd942d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e999b87de0 0x55e999b8c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:04.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.525+0000 7fbcd942d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e999b89350 0x55e999b8cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:04.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.525+0000 7fbcd942d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e999ba3b90 0x55e999b8d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:04.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.525+0000 7fbcd942d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd0071060 con 0x55e999b87de0 2024-09-19T09:23:04.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.525+0000 7fbcd4df0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e999b87de0 0x55e999b8c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:04.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.525+0000 7fbcd4df0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e999b87de0 0x55e999b8c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41994/0 (socket says 172.21.15.120:41994) 2024-09-19T09:23:04.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.525+0000 7fbccffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e999b89350 0x55e999b8cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:04.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.525+0000 7fbcd4df0640 1 -- 172.21.15.120:0/1905339255 learned_addr learned my addr 172.21.15.120:0/1905339255 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:04.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.525+0000 7fbcd55f1640 1 --2- 172.21.15.120:0/1905339255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e999ba3b90 0x55e999b8d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:04.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.525+0000 7fbcd942d740 1 -- 172.21.15.120:0/1905339255 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd0070990 con 0x55e999ba3b90 2024-09-19T09:23:04.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.525+0000 7fbcd942d740 1 -- 172.21.15.120:0/1905339255 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd006dbd0 con 0x55e999b89350 2024-09-19T09:23:04.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.525+0000 7fbcd942d740 10 client.0 _ll_register_callbacks cb 0x7fbcc8009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:04.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.526+0000 7fbccffff640 1 -- 172.21.15.120:0/1905339255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e999ba3b90 msgr2=0x55e999b8d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:04.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.526+0000 7fbccffff640 1 --2- 172.21.15.120:0/1905339255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e999ba3b90 0x55e999b8d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.526+0000 7fbccffff640 1 -- 172.21.15.120:0/1905339255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e999b87de0 msgr2=0x55e999b8c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:04.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.526+0000 7fbccffff640 1 --2- 172.21.15.120:0/1905339255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e999b87de0 0x55e999b8c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.526+0000 7fbccffff640 1 -- 172.21.15.120:0/1905339255 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e999c1d3e0 con 0x55e999b89350 2024-09-19T09:23:04.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.526+0000 7fbccffff640 1 --2- 172.21.15.120:0/1905339255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e999b89350 0x55e999b8cbe0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc09c260 tx=0x7fbcbc009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdb6b835d71b06e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:04.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.526+0000 7fbcaffff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:23:04.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.526+0000 7fbcaffff640 1 -- 172.21.15.120:0/1905339255 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc0bae60 con 0x55e999b89350 2024-09-19T09:23:04.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.526+0000 7fbcaffff640 1 -- 172.21.15.120:0/1905339255 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fbcbc0a0370 con 0x55e999b89350 2024-09-19T09:23:04.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.526+0000 7fbcd942d740 1 -- 172.21.15.120:0/1905339255 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcc40196a0 con 0x55e999b89350 2024-09-19T09:23:04.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.526+0000 7fbcaffff640 1 -- 172.21.15.120:0/1905339255 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbcbc0cdbc0 con 0x55e999b89350 2024-09-19T09:23:04.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.526+0000 7fbcd942d740 10 client.30715 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:04.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.526+0000 7fbcd942d740 1 -- 172.21.15.120:0/1905339255 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e999c1e5c0 con 0x55e999b89350 2024-09-19T09:23:04.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.527+0000 7fbcaffff640 1 -- 172.21.15.120:0/1905339255 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcbc0cdd60 con 0x55e999b89350 2024-09-19T09:23:04.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.527+0000 7fbcaffff640 1 -- 172.21.15.120:0/1905339255 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fbcbc098020 con 0x55e999b89350 2024-09-19T09:23:04.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.527+0000 7fbcd942d740 10 client.30715 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:04.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.527+0000 7fbcd942d740 10 client.30715 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:23:04.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.527+0000 7fbcd942d740 20 client.30715 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:04.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.527+0000 7fbcd942d740 1 --2- 172.21.15.120:0/1905339255 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55e999c1ece0 0x55e999c21100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:04.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.527+0000 7fbcd942d740 4 client.30715 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:23:04.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.527+0000 7fbcd942d740 1 -- 172.21.15.120:0/1905339255 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fbcc40196a0 con 0x55e999c1ece0 2024-09-19T09:23:04.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.527+0000 7fbcd4df0640 1 --2- 172.21.15.120:0/1905339255 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55e999c1ece0 0x55e999c21100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:04.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.528+0000 7fbcaffff640 10 client.30715 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:23:04.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.528+0000 7fbcd4df0640 1 --2- 172.21.15.120:0/1905339255 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55e999c1ece0 0x55e999c21100 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fda0e87ea7800802 server_cookie=c906f934ad8d455e in_seq=0 out_seq=0 2024-09-19T09:23:04.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.528+0000 7fbcaffff640 1 -- 172.21.15.120:0/1905339255 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fbcc40196a0 con 0x55e999c1ece0 2024-09-19T09:23:04.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.528+0000 7fbcaffff640 10 client.30715 handle_command_reply: tid=0 2024-09-19T09:23:04.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.528+0000 7fbcd942d740 1 client.30715 shutdown 2024-09-19T09:23:04.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.528+0000 7fbcd942d740 1 -- 172.21.15.120:0/1905339255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e999b89350 msgr2=0x55e999b8cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:04.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.528+0000 7fbcd942d740 1 --2- 172.21.15.120:0/1905339255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e999b89350 0x55e999b8cbe0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc09c260 tx=0x7fbcbc009700 comp rx=0 tx=0).stop 2024-09-19T09:23:04.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.528+0000 7fbcd942d740 1 -- 172.21.15.120:0/1905339255 shutdown_connections 2024-09-19T09:23:04.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.528+0000 7fbcd942d740 1 --2- 172.21.15.120:0/1905339255 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55e999c1ece0 0x55e999c21100 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.528+0000 7fbcd942d740 1 --2- 172.21.15.120:0/1905339255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e999ba3b90 0x55e999b8d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.528+0000 7fbcaffff640 0 client.30715 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:23:04.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.528+0000 7fbcd942d740 1 --2- 172.21.15.120:0/1905339255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e999b89350 0x55e999b8cbe0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.528+0000 7fbcd942d740 1 --2- 172.21.15.120:0/1905339255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e999b87de0 0x55e999b8c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.528+0000 7fbcd942d740 1 -- 172.21.15.120:0/1905339255 >> 172.21.15.120:0/1905339255 conn(0x55e999b48310 msgr2=0x55e999b7a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:04.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.528+0000 7fbcd942d740 1 -- 172.21.15.120:0/1905339255 shutdown_connections 2024-09-19T09:23:04.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.528+0000 7fbcd942d740 1 -- 172.21.15.120:0/1905339255 wait complete. 2024-09-19T09:23:04.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.528+0000 7fbcd942d740 20 client.30715 trim_cache size 0 max 16384 2024-09-19T09:23:04.534 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:23:04.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.529+0000 7fbcd7054640 1 -- 172.21.15.120:0/516159790 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbcc00a3240 msgr2=0x7fbcc00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:04.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.529+0000 7fbcd7054640 1 --2- 172.21.15.120:0/516159790 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbcc00a3240 0x7fbcc00a5700 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8004af0 tx=0x7fbcc8030b00 comp rx=0 tx=0).stop 2024-09-19T09:23:04.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.529+0000 7fbcd7054640 1 -- 172.21.15.120:0/516159790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbcd0103740 msgr2=0x7fbcd0100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:04.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.529+0000 7fbcd7054640 1 --2- 172.21.15.120:0/516159790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbcd0103740 0x7fbcd0100f30 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00e9c0 tx=0x7fbcbc00ee80 comp rx=0 tx=0).stop 2024-09-19T09:23:04.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.529+0000 7fbcd7054640 1 -- 172.21.15.120:0/516159790 shutdown_connections 2024-09-19T09:23:04.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.529+0000 7fbcd7054640 1 --2- 172.21.15.120:0/516159790 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbcc00a3240 0x7fbcc00a5700 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.529+0000 7fbcd7054640 1 --2- 172.21.15.120:0/516159790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbcd0108290 0x7fbcd0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.529+0000 7fbcd7054640 1 --2- 172.21.15.120:0/516159790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbcd0103740 0x7fbcd0100f30 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.529+0000 7fbcd7054640 1 --2- 172.21.15.120:0/516159790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd0102d90 0x7fbcd01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:04.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.529+0000 7fbcd7054640 1 -- 172.21.15.120:0/516159790 >> 172.21.15.120:0/516159790 conn(0x7fbcd00fc7d0 msgr2=0x7fbcd0105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:04.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.529+0000 7fbcd7054640 1 -- 172.21.15.120:0/516159790 shutdown_connections 2024-09-19T09:23:04.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:04.529+0000 7fbcd7054640 1 -- 172.21.15.120:0/516159790 wait complete. 2024-09-19T09:23:05.376 INFO:tasks.workunit.client.0.smithi120.stdout: 10 166325 19.90 MB/sec execute 300 sec latency 80.494 ms 2024-09-19T09:23:05.483 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:23:05.801 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:23:06.230 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:06 smithi120 ceph-mon[31150]: pgmap v952: 97 pgs: 97 active+clean; 394 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.7 MiB/s wr, 447 op/s 2024-09-19T09:23:06.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.331+0000 7f14a63a3640 1 -- 172.21.15.120:0/174105954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14a0104980 msgr2=0x7f14a01089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:06.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.331+0000 7f14a63a3640 1 --2- 172.21.15.120:0/174105954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14a0104980 0x7f14a01089d0 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f1494009900 tx=0x7f149402a410 comp rx=0 tx=0).stop 2024-09-19T09:23:06.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.331+0000 7f14a63a3640 1 -- 172.21.15.120:0/174105954 shutdown_connections 2024-09-19T09:23:06.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.331+0000 7f14a63a3640 1 --2- 172.21.15.120:0/174105954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14a0109100 0x7f14a0110c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.331+0000 7f14a63a3640 1 --2- 172.21.15.120:0/174105954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14a0104980 0x7f14a01089d0 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.331+0000 7f14a63a3640 1 --2- 172.21.15.120:0/174105954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14a0103fd0 0x7f14a01043b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.331+0000 7f14a63a3640 1 -- 172.21.15.120:0/174105954 >> 172.21.15.120:0/174105954 conn(0x7f14a00ffa30 msgr2=0x7f14a0101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:06.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.331+0000 7f14a63a3640 1 -- 172.21.15.120:0/174105954 shutdown_connections 2024-09-19T09:23:06.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.331+0000 7f14a63a3640 1 -- 172.21.15.120:0/174105954 wait complete. 2024-09-19T09:23:06.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.332+0000 7f14a63a3640 1 Processor -- start 2024-09-19T09:23:06.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.332+0000 7f14a63a3640 1 -- start start 2024-09-19T09:23:06.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.332+0000 7f14a63a3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14a0103fd0 0x7f14a019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:06.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.332+0000 7f14a63a3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14a0104980 0x7f14a019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:06.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.332+0000 7f14a63a3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14a0109100 0x7f14a019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:06.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.332+0000 7f14a63a3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f14a0114120 con 0x7f14a0103fd0 2024-09-19T09:23:06.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.332+0000 7f14a63a3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f14a0113fa0 con 0x7f14a0104980 2024-09-19T09:23:06.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.332+0000 7f14a63a3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f14a01142a0 con 0x7f14a0109100 2024-09-19T09:23:06.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.333+0000 7f149ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14a0103fd0 0x7f14a019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:06.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.333+0000 7f149ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14a0103fd0 0x7f14a019a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42020/0 (socket says 172.21.15.120:42020) 2024-09-19T09:23:06.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.333+0000 7f149ffff640 1 -- 172.21.15.120:0/1434080425 learned_addr learned my addr 172.21.15.120:0/1434080425 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:06.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.333+0000 7f14a4940640 1 --2- 172.21.15.120:0/1434080425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14a0109100 0x7f14a019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:06.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.333+0000 7f149f7fe640 1 --2- 172.21.15.120:0/1434080425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14a0104980 0x7f14a019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:06.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.333+0000 7f149ffff640 1 -- 172.21.15.120:0/1434080425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14a0109100 msgr2=0x7f14a019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:06.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.333+0000 7f149ffff640 1 --2- 172.21.15.120:0/1434080425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14a0109100 0x7f14a019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.333+0000 7f149ffff640 1 -- 172.21.15.120:0/1434080425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14a0104980 msgr2=0x7f14a019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:06.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.333+0000 7f149ffff640 1 --2- 172.21.15.120:0/1434080425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14a0104980 0x7f14a019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.333+0000 7f149ffff640 1 -- 172.21.15.120:0/1434080425 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a019f5e0 con 0x7f14a0103fd0 2024-09-19T09:23:06.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.333+0000 7f149f7fe640 1 --2- 172.21.15.120:0/1434080425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14a0104980 0x7f14a019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:23:06.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.333+0000 7f149ffff640 1 --2- 172.21.15.120:0/1434080425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14a0103fd0 0x7f14a019a630 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f149000e9f0 tx=0x7f149000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:06.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.334+0000 7f149d7fa640 1 -- 172.21.15.120:0/1434080425 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149000cd80 con 0x7f14a0103fd0 2024-09-19T09:23:06.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.334+0000 7f14a63a3640 1 -- 172.21.15.120:0/1434080425 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14a019f8d0 con 0x7f14a0103fd0 2024-09-19T09:23:06.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.334+0000 7f14a63a3640 1 -- 172.21.15.120:0/1434080425 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14a01a71b0 con 0x7f14a0103fd0 2024-09-19T09:23:06.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.334+0000 7f149d7fa640 1 -- 172.21.15.120:0/1434080425 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1490005500 con 0x7f14a0103fd0 2024-09-19T09:23:06.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.334+0000 7f149d7fa640 1 -- 172.21.15.120:0/1434080425 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1490010690 con 0x7f14a0103fd0 2024-09-19T09:23:06.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.338+0000 7f149d7fa640 1 -- 172.21.15.120:0/1434080425 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f1490010830 con 0x7f14a0103fd0 2024-09-19T09:23:06.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.339+0000 7f149d7fa640 1 --2- 172.21.15.120:0/1434080425 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f14880a32c0 0x7f14880a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:06.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.339+0000 7f149f7fe640 1 --2- 172.21.15.120:0/1434080425 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f14880a32c0 0x7f14880a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:06.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.340+0000 7f14a877c740 1 Processor -- start 2024-09-19T09:23:06.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.340+0000 7f149d7fa640 1 -- 172.21.15.120:0/1434080425 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f14900cfcf0 con 0x7f14a0103fd0 2024-09-19T09:23:06.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.340+0000 7f14a877c740 1 -- start start 2024-09-19T09:23:06.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.340+0000 7f149f7fe640 1 --2- 172.21.15.120:0/1434080425 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f14880a32c0 0x7f14880a5780 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f14a00ffec0 tx=0x7f1494005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:06.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.340+0000 7f14a877c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a46057b40 0x557a46056090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:06.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.340+0000 7f14a877c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a460565d0 0x557a46056a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:06.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.340+0000 7f14a877c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a46069b80 0x557a4606bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:06.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.340+0000 7f14a877c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557a45fbc2f0 con 0x557a46069b80 2024-09-19T09:23:06.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.341+0000 7f149f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a460565d0 0x557a46056a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:06.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.341+0000 7f149ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a46057b40 0x557a46056090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:06.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.341+0000 7f149f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a460565d0 0x557a46056a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:53690/0 (socket says 172.21.15.120:53690) 2024-09-19T09:23:06.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.341+0000 7f149f7fe640 1 -- 172.21.15.120:0/1334339208 learned_addr learned my addr 172.21.15.120:0/1334339208 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:06.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.341+0000 7f14a877c740 1 -- 172.21.15.120:0/1334339208 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557a45f63cf0 con 0x557a46057b40 2024-09-19T09:23:06.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.341+0000 7f14a877c740 1 -- 172.21.15.120:0/1334339208 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557a45fb82b0 con 0x557a460565d0 2024-09-19T09:23:06.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.341+0000 7f14a4940640 1 --2- 172.21.15.120:0/1334339208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a46069b80 0x557a4606bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:06.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.342+0000 7f149f7fe640 1 -- 172.21.15.120:0/1334339208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a46057b40 msgr2=0x557a46056090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:06.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.342+0000 7f149f7fe640 1 --2- 172.21.15.120:0/1334339208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a46057b40 0x557a46056090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.342+0000 7f149f7fe640 1 -- 172.21.15.120:0/1334339208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a46069b80 msgr2=0x557a4606bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:06.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.342+0000 7f149f7fe640 1 --2- 172.21.15.120:0/1334339208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a46069b80 0x557a4606bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.342+0000 7f149f7fe640 1 -- 172.21.15.120:0/1334339208 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a45f483a0 con 0x557a460565d0 2024-09-19T09:23:06.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.342+0000 7f14a4940640 1 --2- 172.21.15.120:0/1334339208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a46069b80 0x557a4606bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:06.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.343+0000 7f149f7fe640 1 --2- 172.21.15.120:0/1334339208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a460565d0 0x557a46056a10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1494033000 tx=0x7f149403c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0e5574bddf9d24 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:06.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.343+0000 7f14867fc640 1 -- 172.21.15.120:0/1334339208 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149404b070 con 0x557a460565d0 2024-09-19T09:23:06.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.343+0000 7f14867fc640 1 -- 172.21.15.120:0/1334339208 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14940318e0 con 0x557a460565d0 2024-09-19T09:23:06.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.343+0000 7f14867fc640 1 -- 172.21.15.120:0/1334339208 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1494035050 con 0x557a460565d0 2024-09-19T09:23:06.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.344+0000 7f14a877c740 1 -- 172.21.15.120:0/1334339208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a460565d0 msgr2=0x557a46056a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:06.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.344+0000 7f14a877c740 1 --2- 172.21.15.120:0/1334339208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a460565d0 0x557a46056a10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1494033000 tx=0x7f149403c000 comp rx=0 tx=0).stop 2024-09-19T09:23:06.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.344+0000 7f14a877c740 1 -- 172.21.15.120:0/1334339208 shutdown_connections 2024-09-19T09:23:06.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.344+0000 7f14a877c740 1 --2- 172.21.15.120:0/1334339208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a46069b80 0x557a4606bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.344+0000 7f14a877c740 1 --2- 172.21.15.120:0/1334339208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a460565d0 0x557a46056a10 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.344+0000 7f14a877c740 1 --2- 172.21.15.120:0/1334339208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a46057b40 0x557a46056090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.344+0000 7f14a877c740 1 -- 172.21.15.120:0/1334339208 >> 172.21.15.120:0/1334339208 conn(0x557a4601f310 msgr2=0x557a46052c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:06.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.344+0000 7f14a877c740 1 -- 172.21.15.120:0/1334339208 shutdown_connections 2024-09-19T09:23:06.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.344+0000 7f14a877c740 1 -- 172.21.15.120:0/1334339208 wait complete. 2024-09-19T09:23:06.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.345+0000 7f14a877c740 1 -- start start 2024-09-19T09:23:06.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.345+0000 7f14a877c740 1 Processor -- start 2024-09-19T09:23:06.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.345+0000 7f14a877c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a460565d0 0x557a46069550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:06.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.345+0000 7f14a877c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a46057b40 0x557a46066850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:06.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.346+0000 7f14a877c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a46069b80 0x557a46066eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:06.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.346+0000 7f14a877c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557a4606c820 con 0x557a460565d0 2024-09-19T09:23:06.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.346+0000 7f14a877c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557a45f77060 con 0x557a46057b40 2024-09-19T09:23:06.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.346+0000 7f14a877c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557a4606c9a0 con 0x557a46069b80 2024-09-19T09:23:06.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.346+0000 7f14a877c740 10 client.0 _ll_register_callbacks cb 0x557a45cca8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:06.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.346+0000 7f14a4940640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a46069b80 0x557a46066eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:06.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.346+0000 7f14a4940640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a46069b80 0x557a46066eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:53704/0 (socket says 172.21.15.120:53704) 2024-09-19T09:23:06.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.346+0000 7f14a4940640 1 -- 172.21.15.120:0/1861110078 learned_addr learned my addr 172.21.15.120:0/1861110078 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:06.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.346+0000 7f14a4940640 1 -- 172.21.15.120:0/1861110078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a46057b40 msgr2=0x557a46066850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:06.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.346+0000 7f14a4940640 1 --2- 172.21.15.120:0/1861110078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a46057b40 0x557a46066850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.346+0000 7f14a4940640 1 -- 172.21.15.120:0/1861110078 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a460565d0 msgr2=0x557a46069550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:06.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.346+0000 7f14a4940640 1 --2- 172.21.15.120:0/1861110078 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a460565d0 0x557a46069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.346+0000 7f14a4940640 1 -- 172.21.15.120:0/1861110078 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a45d2d3c0 con 0x557a46069b80 2024-09-19T09:23:06.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.347+0000 7f14a4940640 1 --2- 172.21.15.120:0/1861110078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a46069b80 0x557a46066eb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f14980119d0 tx=0x7f1498011e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83bdcbd4b42cfb9a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:06.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.347+0000 7f147f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:23:06.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.347+0000 7f147f7fe640 1 -- 172.21.15.120:0/1861110078 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1498010be0 con 0x557a46069b80 2024-09-19T09:23:06.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.347+0000 7f14a877c740 1 -- 172.21.15.120:0/1861110078 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14940318e0 con 0x557a46069b80 2024-09-19T09:23:06.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.347+0000 7f147f7fe640 1 -- 172.21.15.120:0/1861110078 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f1498010d80 con 0x557a46069b80 2024-09-19T09:23:06.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.347+0000 7f147f7fe640 1 -- 172.21.15.120:0/1861110078 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1498004b20 con 0x557a46069b80 2024-09-19T09:23:06.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.347+0000 7f14a877c740 10 client.30727 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:06.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.348+0000 7f14a877c740 1 -- 172.21.15.120:0/1861110078 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557a46078390 con 0x557a46069b80 2024-09-19T09:23:06.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.348+0000 7f147f7fe640 1 -- 172.21.15.120:0/1861110078 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1498004cc0 con 0x557a46069b80 2024-09-19T09:23:06.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.348+0000 7f147f7fe640 1 -- 172.21.15.120:0/1861110078 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f149801a510 con 0x557a46069b80 2024-09-19T09:23:06.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.348+0000 7f14a877c740 10 client.30727 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:06.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.348+0000 7f14a877c740 10 client.30727 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:23:06.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.348+0000 7f14a877c740 20 client.30727 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:06.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.348+0000 7f14a877c740 1 --2- 172.21.15.120:0/1861110078 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x557a460798f0 0x557a4607bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:06.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.348+0000 7f14a877c740 4 client.30727 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:23:06.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.348+0000 7f14a877c740 1 -- 172.21.15.120:0/1861110078 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f14940318e0 con 0x557a460798f0 2024-09-19T09:23:06.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.349+0000 7f149ffff640 1 --2- 172.21.15.120:0/1861110078 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x557a460798f0 0x557a4607bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:06.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.349+0000 7f149ffff640 1 --2- 172.21.15.120:0/1861110078 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x557a460798f0 0x557a4607bdb0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=78717c6be37aff02 server_cookie=87be5224a10ec2d5 in_seq=0 out_seq=0 2024-09-19T09:23:06.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.349+0000 7f147f7fe640 10 client.30727 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:23:06.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.351+0000 7f147f7fe640 1 -- 172.21.15.120:0/1861110078 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f14940318e0 con 0x557a460798f0 2024-09-19T09:23:06.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.351+0000 7f147f7fe640 10 client.30727 handle_command_reply: tid=0 2024-09-19T09:23:06.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.351+0000 7f14a877c740 1 client.30727 shutdown 2024-09-19T09:23:06.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.351+0000 7f14a877c740 1 -- 172.21.15.120:0/1861110078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a46069b80 msgr2=0x557a46066eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:06.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.351+0000 7f14a877c740 1 --2- 172.21.15.120:0/1861110078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a46069b80 0x557a46066eb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f14980119d0 tx=0x7f1498011e90 comp rx=0 tx=0).stop 2024-09-19T09:23:06.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.351+0000 7f14a877c740 1 -- 172.21.15.120:0/1861110078 shutdown_connections 2024-09-19T09:23:06.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.351+0000 7f14a877c740 1 --2- 172.21.15.120:0/1861110078 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x557a460798f0 0x557a4607bdb0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.351+0000 7f14a877c740 1 --2- 172.21.15.120:0/1861110078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a46069b80 0x557a46066eb0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.351+0000 7f14a877c740 1 --2- 172.21.15.120:0/1861110078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a46057b40 0x557a46066850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.351+0000 7f14a877c740 1 --2- 172.21.15.120:0/1861110078 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a460565d0 0x557a46069550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.351+0000 7f14a877c740 1 -- 172.21.15.120:0/1861110078 >> 172.21.15.120:0/1861110078 conn(0x557a4601f310 msgr2=0x557a46058930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:06.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.351+0000 7f147f7fe640 0 client.30727 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:23:06.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.351+0000 7f14a877c740 1 -- 172.21.15.120:0/1861110078 shutdown_connections 2024-09-19T09:23:06.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.351+0000 7f14a877c740 1 -- 172.21.15.120:0/1861110078 wait complete. 2024-09-19T09:23:06.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.351+0000 7f14a877c740 20 client.30727 trim_cache size 0 max 16384 2024-09-19T09:23:06.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.373+0000 7f14a877c740 1 Processor -- start 2024-09-19T09:23:06.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.373+0000 7f14a877c740 1 -- start start 2024-09-19T09:23:06.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.374+0000 7f14a877c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a460565d0 0x557a46075b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:06.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.374+0000 7f14a877c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a46057b40 0x557a46065830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:06.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.374+0000 7f14a877c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a46069b80 0x557a46081680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:06.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.374+0000 7f14a877c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f148c00e560 con 0x557a46057b40 2024-09-19T09:23:06.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.374+0000 7f14a877c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f148c0100a0 con 0x557a460565d0 2024-09-19T09:23:06.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.374+0000 7f14a877c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f148c00ec30 con 0x557a46069b80 2024-09-19T09:23:06.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.374+0000 7f149ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a460565d0 0x557a46075b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:06.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.374+0000 7f149ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a460565d0 0x557a46075b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:43102/0 (socket says 172.21.15.120:43102) 2024-09-19T09:23:06.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.374+0000 7f149ffff640 1 -- 172.21.15.120:0/1848548459 learned_addr learned my addr 172.21.15.120:0/1848548459 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:06.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.374+0000 7f149f7fe640 1 --2- 172.21.15.120:0/1848548459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a46057b40 0x557a46065830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:06.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.374+0000 7f14a4940640 1 --2- 172.21.15.120:0/1848548459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a46069b80 0x557a46081680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:06.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.374+0000 7f149ffff640 1 -- 172.21.15.120:0/1848548459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a46069b80 msgr2=0x557a46081680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:06.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.374+0000 7f149ffff640 1 --2- 172.21.15.120:0/1848548459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a46069b80 0x557a46081680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.374+0000 7f149ffff640 1 -- 172.21.15.120:0/1848548459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a46057b40 msgr2=0x557a46065830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:06.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.374+0000 7f149ffff640 1 --2- 172.21.15.120:0/1848548459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a46057b40 0x557a46065830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.374+0000 7f149ffff640 1 -- 172.21.15.120:0/1848548459 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a460524f0 con 0x557a460565d0 2024-09-19T09:23:06.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.374+0000 7f149f7fe640 1 --2- 172.21.15.120:0/1848548459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a46057b40 0x557a46065830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:06.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.375+0000 7f149ffff640 1 --2- 172.21.15.120:0/1848548459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a460565d0 0x557a46075b50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f149009d3c0 tx=0x7f14900cd740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd959c27dc2a6b38 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:06.377 INFO:tasks.workunit.client.0.smithi120.stdout: 10 167000 19.95 MB/sec execute 301 sec latency 66.872 ms 2024-09-19T09:23:06.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.375+0000 7f1485ffb640 1 -- 172.21.15.120:0/1848548459 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149009d430 con 0x557a460565d0 2024-09-19T09:23:06.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.375+0000 7f1485ffb640 1 -- 172.21.15.120:0/1848548459 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14900a17e0 con 0x557a460565d0 2024-09-19T09:23:06.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.375+0000 7f1485ffb640 1 -- 172.21.15.120:0/1848548459 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1490099aa0 con 0x557a460565d0 2024-09-19T09:23:06.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.377+0000 7f14a877c740 1 -- 172.21.15.120:0/1848548459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a460565d0 msgr2=0x557a46075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:06.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.377+0000 7f14a877c740 1 --2- 172.21.15.120:0/1848548459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a460565d0 0x557a46075b50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f149009d3c0 tx=0x7f14900cd740 comp rx=0 tx=0).stop 2024-09-19T09:23:06.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.377+0000 7f14a877c740 1 -- 172.21.15.120:0/1848548459 shutdown_connections 2024-09-19T09:23:06.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.377+0000 7f14a877c740 1 --2- 172.21.15.120:0/1848548459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a46069b80 0x557a46081680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.377+0000 7f14a877c740 1 --2- 172.21.15.120:0/1848548459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a46057b40 0x557a46065830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.377+0000 7f14a877c740 1 --2- 172.21.15.120:0/1848548459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a460565d0 0x557a46075b50 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.377+0000 7f14a877c740 1 -- 172.21.15.120:0/1848548459 >> 172.21.15.120:0/1848548459 conn(0x557a4601f310 msgr2=0x557a46065120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:06.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.379+0000 7f14a877c740 1 -- 172.21.15.120:0/1848548459 shutdown_connections 2024-09-19T09:23:06.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.379+0000 7f14a877c740 1 -- 172.21.15.120:0/1848548459 wait complete. 2024-09-19T09:23:06.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.380+0000 7f14a877c740 1 -- start start 2024-09-19T09:23:06.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.380+0000 7f14a877c740 1 Processor -- start 2024-09-19T09:23:06.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.381+0000 7f14a877c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a460565d0 0x557a460638c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:06.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.381+0000 7f14a877c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a46057b40 0x557a46063e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:06.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.381+0000 7f14a877c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a46069b80 0x557a46064340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:06.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.381+0000 7f14a877c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f148c00f9d0 con 0x557a460565d0 2024-09-19T09:23:06.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.381+0000 7f14a877c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f148c00f300 con 0x557a46057b40 2024-09-19T09:23:06.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.381+0000 7f14a877c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f148c00c520 con 0x557a46069b80 2024-09-19T09:23:06.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.381+0000 7f14a4940640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a46069b80 0x557a46064340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:06.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.381+0000 7f149f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a46057b40 0x557a46063e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:06.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.381+0000 7f14a4940640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a46069b80 0x557a46064340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:53716/0 (socket says 172.21.15.120:53716) 2024-09-19T09:23:06.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.381+0000 7f149f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a46057b40 0x557a46063e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:43110/0 (socket says 172.21.15.120:43110) 2024-09-19T09:23:06.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.381+0000 7f14a4940640 1 -- 172.21.15.120:0/1479640305 learned_addr learned my addr 172.21.15.120:0/1479640305 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:06.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.381+0000 7f14a877c740 10 client.0 _ll_register_callbacks cb 0x7f1498009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:06.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.382+0000 7f14a4940640 1 -- 172.21.15.120:0/1479640305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a46057b40 msgr2=0x557a46063e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:06.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.382+0000 7f14a4940640 1 --2- 172.21.15.120:0/1479640305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a46057b40 0x557a46063e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.382+0000 7f14a4940640 1 -- 172.21.15.120:0/1479640305 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a460565d0 msgr2=0x557a460638c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:23:06.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.382+0000 7f14a4940640 1 --2- 172.21.15.120:0/1479640305 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a460565d0 0x557a460638c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.382+0000 7f14a4940640 1 -- 172.21.15.120:0/1479640305 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a460e36a0 con 0x557a46069b80 2024-09-19T09:23:06.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.382+0000 7f14a4940640 1 --2- 172.21.15.120:0/1479640305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a46069b80 0x557a46064340 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f149801aaa0 tx=0x7f14980116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78cf31f272754bc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:06.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.382+0000 7f147f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:23:06.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.382+0000 7f147f7fe640 1 -- 172.21.15.120:0/1479640305 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1498029070 con 0x557a46069b80 2024-09-19T09:23:06.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.382+0000 7f147f7fe640 1 -- 172.21.15.120:0/1479640305 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f149801a3e0 con 0x557a46069b80 2024-09-19T09:23:06.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.382+0000 7f14a877c740 1 -- 172.21.15.120:0/1479640305 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14900a17e0 con 0x557a46069b80 2024-09-19T09:23:06.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.382+0000 7f147f7fe640 1 -- 172.21.15.120:0/1479640305 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f149801a5e0 con 0x557a46069b80 2024-09-19T09:23:06.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.382+0000 7f14a877c740 10 client.30733 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:06.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.383+0000 7f14a877c740 1 -- 172.21.15.120:0/1479640305 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557a460e4980 con 0x557a46069b80 2024-09-19T09:23:06.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.383+0000 7f147f7fe640 1 -- 172.21.15.120:0/1479640305 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f149802a040 con 0x557a46069b80 2024-09-19T09:23:06.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.383+0000 7f147f7fe640 1 -- 172.21.15.120:0/1479640305 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f1498004750 con 0x557a46069b80 2024-09-19T09:23:06.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.383+0000 7f14a877c740 10 client.30733 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:06.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.383+0000 7f14a877c740 10 client.30733 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:23:06.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.383+0000 7f14a877c740 20 client.30733 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:06.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.383+0000 7f14a877c740 1 --2- 172.21.15.120:0/1479640305 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x557a460e5120 0x557a460e7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:06.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.383+0000 7f14a877c740 4 client.30733 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:23:06.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.383+0000 7f14a877c740 1 -- 172.21.15.120:0/1479640305 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f14900a17e0 con 0x557a460e5120 2024-09-19T09:23:06.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.384+0000 7f149ffff640 1 --2- 172.21.15.120:0/1479640305 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x557a460e5120 0x557a460e7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:06.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.384+0000 7f147f7fe640 10 client.30733 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:23:06.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.384+0000 7f149ffff640 1 --2- 172.21.15.120:0/1479640305 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x557a460e5120 0x557a460e7540 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2b733254820e79cb server_cookie=d0b00ae7687b03f8 in_seq=0 out_seq=0 2024-09-19T09:23:06.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.385+0000 7f147f7fe640 1 -- 172.21.15.120:0/1479640305 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f14900a17e0 con 0x557a460e5120 2024-09-19T09:23:06.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.385+0000 7f147f7fe640 10 client.30733 handle_command_reply: tid=0 2024-09-19T09:23:06.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.385+0000 7f14a877c740 1 client.30733 shutdown 2024-09-19T09:23:06.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.385+0000 7f14a877c740 1 -- 172.21.15.120:0/1479640305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a46069b80 msgr2=0x557a46064340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:06.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.386+0000 7f14a877c740 1 --2- 172.21.15.120:0/1479640305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a46069b80 0x557a46064340 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f149801aaa0 tx=0x7f14980116c0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.386+0000 7f14a877c740 1 -- 172.21.15.120:0/1479640305 shutdown_connections 2024-09-19T09:23:06.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.386+0000 7f14a877c740 1 --2- 172.21.15.120:0/1479640305 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x557a460e5120 0x557a460e7540 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.386+0000 7f14a877c740 1 --2- 172.21.15.120:0/1479640305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a46069b80 0x557a46064340 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.386+0000 7f14a877c740 1 --2- 172.21.15.120:0/1479640305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a46057b40 0x557a46063e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.386+0000 7f14a877c740 1 --2- 172.21.15.120:0/1479640305 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a460565d0 0x557a460638c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.386+0000 7f14a877c740 1 -- 172.21.15.120:0/1479640305 >> 172.21.15.120:0/1479640305 conn(0x557a4601f310 msgr2=0x557a46065940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:06.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.386+0000 7f147f7fe640 0 client.30733 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:23:06.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.386+0000 7f14a877c740 1 -- 172.21.15.120:0/1479640305 shutdown_connections 2024-09-19T09:23:06.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.386+0000 7f14a877c740 1 -- 172.21.15.120:0/1479640305 wait complete. 2024-09-19T09:23:06.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.386+0000 7f14a877c740 20 client.30733 trim_cache size 0 max 16384 2024-09-19T09:23:06.389 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:23:06.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.387+0000 7f14a63a3640 1 -- 172.21.15.120:0/1434080425 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f14880a32c0 msgr2=0x7f14880a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:06.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.387+0000 7f14a63a3640 1 --2- 172.21.15.120:0/1434080425 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f14880a32c0 0x7f14880a5780 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f14a00ffec0 tx=0x7f1494005a20 comp rx=0 tx=0).stop 2024-09-19T09:23:06.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.387+0000 7f14a63a3640 1 -- 172.21.15.120:0/1434080425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14a0103fd0 msgr2=0x7f14a019a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:06.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.387+0000 7f14a63a3640 1 --2- 172.21.15.120:0/1434080425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14a0103fd0 0x7f14a019a630 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f149000e9f0 tx=0x7f149000edf0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.387+0000 7f14a63a3640 1 -- 172.21.15.120:0/1434080425 shutdown_connections 2024-09-19T09:23:06.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.387+0000 7f14a63a3640 1 --2- 172.21.15.120:0/1434080425 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f14880a32c0 0x7f14880a5780 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.387+0000 7f14a63a3640 1 --2- 172.21.15.120:0/1434080425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14a0109100 0x7f14a019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.387+0000 7f14a63a3640 1 --2- 172.21.15.120:0/1434080425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14a0104980 0x7f14a019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.387+0000 7f14a63a3640 1 --2- 172.21.15.120:0/1434080425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14a0103fd0 0x7f14a019a630 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:06.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.387+0000 7f14a63a3640 1 -- 172.21.15.120:0/1434080425 >> 172.21.15.120:0/1434080425 conn(0x7f14a00ffa30 msgr2=0x7f14a0100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:06.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.387+0000 7f14a63a3640 1 -- 172.21.15.120:0/1434080425 shutdown_connections 2024-09-19T09:23:06.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:06.387+0000 7f14a63a3640 1 -- 172.21.15.120:0/1434080425 wait complete. 2024-09-19T09:23:06.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:06 smithi196 ceph-mon[35150]: pgmap v952: 97 pgs: 97 active+clean; 394 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.7 MiB/s wr, 447 op/s 2024-09-19T09:23:06.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:06 smithi153 ceph-mon[35071]: pgmap v952: 97 pgs: 97 active+clean; 394 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.7 MiB/s wr, 447 op/s 2024-09-19T09:23:07.376 INFO:tasks.workunit.client.0.smithi120.stdout: 10 167821 20.01 MB/sec execute 302 sec latency 39.532 ms 2024-09-19T09:23:07.646 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:23:07.939 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:23:08.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:08 smithi120 ceph-mon[31150]: pgmap v953: 97 pgs: 97 active+clean; 404 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.7 MiB/s wr, 615 op/s 2024-09-19T09:23:08.376 INFO:tasks.workunit.client.0.smithi120.stdout: 10 168435 20.07 MB/sec execute 303 sec latency 61.535 ms 2024-09-19T09:23:08.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:08 smithi196 ceph-mon[35150]: pgmap v953: 97 pgs: 97 active+clean; 404 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.7 MiB/s wr, 615 op/s 2024-09-19T09:23:08.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:08 smithi153 ceph-mon[35071]: pgmap v953: 97 pgs: 97 active+clean; 404 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.7 MiB/s wr, 615 op/s 2024-09-19T09:23:08.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.802+0000 7f83ee7f5640 1 -- 172.21.15.120:0/675565992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83e81049a0 msgr2=0x7f83e81089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:08.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.802+0000 7f83ee7f5640 1 --2- 172.21.15.120:0/675565992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83e81049a0 0x7f83e81089f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009860 tx=0x7f83dc02a340 comp rx=0 tx=0).stop 2024-09-19T09:23:08.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.802+0000 7f83ee7f5640 1 -- 172.21.15.120:0/675565992 shutdown_connections 2024-09-19T09:23:08.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.802+0000 7f83ee7f5640 1 --2- 172.21.15.120:0/675565992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83e8109120 0x7f83e8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.802+0000 7f83ee7f5640 1 --2- 172.21.15.120:0/675565992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83e81049a0 0x7f83e81089f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.802+0000 7f83ee7f5640 1 --2- 172.21.15.120:0/675565992 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83e8103ff0 0x7f83e81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.802+0000 7f83ee7f5640 1 -- 172.21.15.120:0/675565992 >> 172.21.15.120:0/675565992 conn(0x7f83e80ffab0 msgr2=0x7f83e8101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:08.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.802+0000 7f83ee7f5640 1 -- 172.21.15.120:0/675565992 shutdown_connections 2024-09-19T09:23:08.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.802+0000 7f83ee7f5640 1 -- 172.21.15.120:0/675565992 wait complete. 2024-09-19T09:23:08.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.803+0000 7f83ee7f5640 1 Processor -- start 2024-09-19T09:23:08.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.803+0000 7f83ee7f5640 1 -- start start 2024-09-19T09:23:08.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.804+0000 7f83ee7f5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83e8103ff0 0x7f83e819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:08.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.804+0000 7f83ee7f5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83e81049a0 0x7f83e819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:08.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.804+0000 7f83ee7f5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83e8109120 0x7f83e819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:08.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.804+0000 7f83ee7f5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f83e8114120 con 0x7f83e81049a0 2024-09-19T09:23:08.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.804+0000 7f83ee7f5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f83e8113fa0 con 0x7f83e8103ff0 2024-09-19T09:23:08.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.804+0000 7f83ee7f5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f83e81142a0 con 0x7f83e8109120 2024-09-19T09:23:08.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.804+0000 7f83e77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83e81049a0 0x7f83e819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:08.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.804+0000 7f83e7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83e8103ff0 0x7f83e819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:08.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.804+0000 7f83e77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83e81049a0 0x7f83e819ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42050/0 (socket says 172.21.15.120:42050) 2024-09-19T09:23:08.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.804+0000 7f83e7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83e8103ff0 0x7f83e819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:43122/0 (socket says 172.21.15.120:43122) 2024-09-19T09:23:08.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.804+0000 7f83e77fe640 1 -- 172.21.15.120:0/2633455014 learned_addr learned my addr 172.21.15.120:0/2633455014 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:08.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.804+0000 7f83ecd92640 1 --2- 172.21.15.120:0/2633455014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83e8109120 0x7f83e819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:08.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.805+0000 7f83e77fe640 1 -- 172.21.15.120:0/2633455014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83e8109120 msgr2=0x7f83e819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:08.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.805+0000 7f83e77fe640 1 --2- 172.21.15.120:0/2633455014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83e8109120 0x7f83e819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.805+0000 7f83e77fe640 1 -- 172.21.15.120:0/2633455014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83e8103ff0 msgr2=0x7f83e819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:08.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.805+0000 7f83e77fe640 1 --2- 172.21.15.120:0/2633455014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83e8103ff0 0x7f83e819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.805+0000 7f83e77fe640 1 -- 172.21.15.120:0/2633455014 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e819f5f0 con 0x7f83e81049a0 2024-09-19T09:23:08.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.805+0000 7f83e77fe640 1 --2- 172.21.15.120:0/2633455014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83e81049a0 0x7f83e819ab80 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f83dc035040 tx=0x7f83dc033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:08.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.805+0000 7f83e57fa640 1 -- 172.21.15.120:0/2633455014 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc037040 con 0x7f83e81049a0 2024-09-19T09:23:08.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.805+0000 7f83e57fa640 1 -- 172.21.15.120:0/2633455014 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83dc040070 con 0x7f83e81049a0 2024-09-19T09:23:08.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.805+0000 7f83e57fa640 1 -- 172.21.15.120:0/2633455014 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc003a50 con 0x7f83e81049a0 2024-09-19T09:23:08.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.805+0000 7f83ee7f5640 1 -- 172.21.15.120:0/2633455014 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83e819f880 con 0x7f83e81049a0 2024-09-19T09:23:08.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.805+0000 7f83ee7f5640 1 -- 172.21.15.120:0/2633455014 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83e81a71c0 con 0x7f83e81049a0 2024-09-19T09:23:08.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.808+0000 7f83e57fa640 1 -- 172.21.15.120:0/2633455014 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f83dc003bf0 con 0x7f83e81049a0 2024-09-19T09:23:08.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.808+0000 7f83e57fa640 1 --2- 172.21.15.120:0/2633455014 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83d00a3240 0x7f83d00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:08.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.808+0000 7f83e7fff640 1 --2- 172.21.15.120:0/2633455014 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83d00a3240 0x7f83d00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:08.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.808+0000 7f83e57fa640 1 -- 172.21.15.120:0/2633455014 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f83dc0ef710 con 0x7f83e81049a0 2024-09-19T09:23:08.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.809+0000 7f83f0bce740 1 Processor -- start 2024-09-19T09:23:08.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.809+0000 7f83f0bce740 1 -- start start 2024-09-19T09:23:08.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.809+0000 7f83f0bce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56206e1dfb40 0x56206e1de090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:08.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.809+0000 7f83f0bce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56206e1de5d0 0x56206e1dea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:08.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.809+0000 7f83e7fff640 1 --2- 172.21.15.120:0/2633455014 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83d00a3240 0x7f83d00a5700 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009530 tx=0x7f83d8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:08.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.809+0000 7f83f0bce740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56206e1e9370 0x56206e1eb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:08.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.809+0000 7f83f0bce740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56206e1442f0 con 0x56206e1e9370 2024-09-19T09:23:08.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.809+0000 7f83f0bce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56206e0ebcf0 con 0x56206e1dfb40 2024-09-19T09:23:08.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.809+0000 7f83f0bce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56206e1402b0 con 0x56206e1de5d0 2024-09-19T09:23:08.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.809+0000 7f83e77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56206e1de5d0 0x56206e1dea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:08.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.809+0000 7f83e7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56206e1dfb40 0x56206e1de090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:08.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.809+0000 7f83e77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56206e1de5d0 0x56206e1dea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:53756/0 (socket says 172.21.15.120:53756) 2024-09-19T09:23:08.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.809+0000 7f83ecd92640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56206e1e9370 0x56206e1eb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:08.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.809+0000 7f83e77fe640 1 -- 172.21.15.120:0/3951104613 learned_addr learned my addr 172.21.15.120:0/3951104613 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:08.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.810+0000 7f83ecd92640 1 -- 172.21.15.120:0/3951104613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56206e1de5d0 msgr2=0x56206e1dea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:08.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.810+0000 7f83ecd92640 1 --2- 172.21.15.120:0/3951104613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56206e1de5d0 0x56206e1dea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.810+0000 7f83ecd92640 1 -- 172.21.15.120:0/3951104613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56206e1dfb40 msgr2=0x56206e1de090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:08.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.810+0000 7f83ecd92640 1 --2- 172.21.15.120:0/3951104613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56206e1dfb40 0x56206e1de090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.810+0000 7f83ecd92640 1 -- 172.21.15.120:0/3951104613 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56206e0d03a0 con 0x56206e1e9370 2024-09-19T09:23:08.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.810+0000 7f83ecd92640 1 --2- 172.21.15.120:0/3951104613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56206e1e9370 0x56206e1eb760 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f83e0009eb0 tx=0x7f83e00217b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e365bf7ccdb417 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:08.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.811+0000 7f83ce7fc640 1 -- 172.21.15.120:0/3951104613 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e0003b70 con 0x56206e1e9370 2024-09-19T09:23:08.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.811+0000 7f83ce7fc640 1 -- 172.21.15.120:0/3951104613 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e0026070 con 0x56206e1e9370 2024-09-19T09:23:08.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.811+0000 7f83ce7fc640 1 -- 172.21.15.120:0/3951104613 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e002b970 con 0x56206e1e9370 2024-09-19T09:23:08.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.812+0000 7f83f0bce740 1 -- 172.21.15.120:0/3951104613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56206e1e9370 msgr2=0x56206e1eb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:08.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.812+0000 7f83f0bce740 1 --2- 172.21.15.120:0/3951104613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56206e1e9370 0x56206e1eb760 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f83e0009eb0 tx=0x7f83e00217b0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.813+0000 7f83f0bce740 1 -- 172.21.15.120:0/3951104613 shutdown_connections 2024-09-19T09:23:08.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.813+0000 7f83f0bce740 1 --2- 172.21.15.120:0/3951104613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56206e1e9370 0x56206e1eb760 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.813+0000 7f83f0bce740 1 --2- 172.21.15.120:0/3951104613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56206e1de5d0 0x56206e1dea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.813+0000 7f83f0bce740 1 --2- 172.21.15.120:0/3951104613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56206e1dfb40 0x56206e1de090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.813+0000 7f83f0bce740 1 -- 172.21.15.120:0/3951104613 >> 172.21.15.120:0/3951104613 conn(0x56206e1a7310 msgr2=0x56206e1dac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:08.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.813+0000 7f83f0bce740 1 -- 172.21.15.120:0/3951104613 shutdown_connections 2024-09-19T09:23:08.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.813+0000 7f83f0bce740 1 -- 172.21.15.120:0/3951104613 wait complete. 2024-09-19T09:23:08.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.814+0000 7f83f0bce740 1 -- start start 2024-09-19T09:23:08.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.814+0000 7f83f0bce740 1 Processor -- start 2024-09-19T09:23:08.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.815+0000 7f83f0bce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56206e1de5d0 0x56206e1e8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:08.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.815+0000 7f83f0bce740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56206e1dfb40 0x56206e1e6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:08.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.815+0000 7f83f0bce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56206e1e9370 0x56206e1e6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:08.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.815+0000 7f83f0bce740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56206e1ebf30 con 0x56206e1dfb40 2024-09-19T09:23:08.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.815+0000 7f83f0bce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56206e0ff060 con 0x56206e1e9370 2024-09-19T09:23:08.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.815+0000 7f83f0bce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56206e1ec0b0 con 0x56206e1de5d0 2024-09-19T09:23:08.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.815+0000 7f83f0bce740 10 client.0 _ll_register_callbacks cb 0x56206de528c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:08.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.815+0000 7f83e77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56206e1dfb40 0x56206e1e6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:08.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.815+0000 7f83ecd92640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56206e1e9370 0x56206e1e6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:08.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.815+0000 7f83e7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56206e1de5d0 0x56206e1e8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:08.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.815+0000 7f83ecd92640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56206e1e9370 0x56206e1e6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:43144/0 (socket says 172.21.15.120:43144) 2024-09-19T09:23:08.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.815+0000 7f83e77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56206e1dfb40 0x56206e1e6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42066/0 (socket says 172.21.15.120:42066) 2024-09-19T09:23:08.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.815+0000 7f83ecd92640 1 -- 172.21.15.120:0/3957618700 learned_addr learned my addr 172.21.15.120:0/3957618700 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:08.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.815+0000 7f83e7fff640 1 -- 172.21.15.120:0/3957618700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56206e1e9370 msgr2=0x56206e1e6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:08.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.815+0000 7f83e7fff640 1 --2- 172.21.15.120:0/3957618700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56206e1e9370 0x56206e1e6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.815+0000 7f83e7fff640 1 -- 172.21.15.120:0/3957618700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56206e1dfb40 msgr2=0x56206e1e6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:08.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.815+0000 7f83e7fff640 1 --2- 172.21.15.120:0/3957618700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56206e1dfb40 0x56206e1e6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.815+0000 7f83e7fff640 1 -- 172.21.15.120:0/3957618700 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56206deb53c0 con 0x56206e1de5d0 2024-09-19T09:23:08.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.815+0000 7f83e77fe640 1 --2- 172.21.15.120:0/3957618700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56206e1dfb40 0x56206e1e6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-19T09:23:08.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.815+0000 7f83e7fff640 1 --2- 172.21.15.120:0/3957618700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56206e1de5d0 0x56206e1e8dd0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f83d8005fd0 tx=0x7f83d8013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95bc1b180a3f8212 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:08.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.816+0000 7f83c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:23:08.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.816+0000 7f83c77fe640 1 -- 172.21.15.120:0/3957618700 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d8009e10 con 0x56206e1de5d0 2024-09-19T09:23:08.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.816+0000 7f83c77fe640 1 -- 172.21.15.120:0/3957618700 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f83d800eb40 con 0x56206e1de5d0 2024-09-19T09:23:08.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.816+0000 7f83f0bce740 1 -- 172.21.15.120:0/3957618700 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83e0026070 con 0x56206e1de5d0 2024-09-19T09:23:08.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.816+0000 7f83c77fe640 1 -- 172.21.15.120:0/3957618700 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f83d800ed80 con 0x56206e1de5d0 2024-09-19T09:23:08.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.816+0000 7f83f0bce740 10 client.30745 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:08.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.816+0000 7f83f0bce740 1 -- 172.21.15.120:0/3957618700 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56206e1f7ca0 con 0x56206e1de5d0 2024-09-19T09:23:08.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.816+0000 7f83c77fe640 1 -- 172.21.15.120:0/3957618700 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83d8005500 con 0x56206e1de5d0 2024-09-19T09:23:08.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.817+0000 7f83c77fe640 1 -- 172.21.15.120:0/3957618700 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f83d8026070 con 0x56206e1de5d0 2024-09-19T09:23:08.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.817+0000 7f83f0bce740 10 client.30745 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:08.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.817+0000 7f83f0bce740 10 client.30745 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:23:08.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.817+0000 7f83f0bce740 20 client.30745 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:08.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.817+0000 7f83f0bce740 1 --2- 172.21.15.120:0/3957618700 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56206e1f9100 0x56206e1fb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:08.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.817+0000 7f83f0bce740 4 client.30745 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:23:08.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.817+0000 7f83f0bce740 1 -- 172.21.15.120:0/3957618700 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f83e0026070 con 0x56206e1f9100 2024-09-19T09:23:08.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.817+0000 7f83e77fe640 1 --2- 172.21.15.120:0/3957618700 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56206e1f9100 0x56206e1fb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:08.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.818+0000 7f83e77fe640 1 --2- 172.21.15.120:0/3957618700 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56206e1f9100 0x56206e1fb4f0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d18291b4e05124b7 server_cookie=e015810747419903 in_seq=0 out_seq=0 2024-09-19T09:23:08.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.818+0000 7f83c77fe640 10 client.30745 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:23:08.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.819+0000 7f83c77fe640 1 -- 172.21.15.120:0/3957618700 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f83e0026070 con 0x56206e1f9100 2024-09-19T09:23:08.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.819+0000 7f83c77fe640 10 client.30745 handle_command_reply: tid=0 2024-09-19T09:23:08.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.819+0000 7f83f0bce740 1 client.30745 shutdown 2024-09-19T09:23:08.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.819+0000 7f83f0bce740 1 -- 172.21.15.120:0/3957618700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56206e1de5d0 msgr2=0x56206e1e8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:08.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.819+0000 7f83f0bce740 1 --2- 172.21.15.120:0/3957618700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56206e1de5d0 0x56206e1e8dd0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f83d8005fd0 tx=0x7f83d8013040 comp rx=0 tx=0).stop 2024-09-19T09:23:08.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.819+0000 7f83f0bce740 1 -- 172.21.15.120:0/3957618700 shutdown_connections 2024-09-19T09:23:08.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.819+0000 7f83f0bce740 1 --2- 172.21.15.120:0/3957618700 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56206e1f9100 0x56206e1fb4f0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.819+0000 7f83f0bce740 1 --2- 172.21.15.120:0/3957618700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56206e1e9370 0x56206e1e6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.819+0000 7f83f0bce740 1 --2- 172.21.15.120:0/3957618700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56206e1dfb40 0x56206e1e6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.819+0000 7f83f0bce740 1 --2- 172.21.15.120:0/3957618700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56206e1de5d0 0x56206e1e8dd0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.819+0000 7f83f0bce740 1 -- 172.21.15.120:0/3957618700 >> 172.21.15.120:0/3957618700 conn(0x56206e1a7310 msgr2=0x56206e1e0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:08.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.819+0000 7f83c77fe640 0 client.30745 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:23:08.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.819+0000 7f83f0bce740 1 -- 172.21.15.120:0/3957618700 shutdown_connections 2024-09-19T09:23:08.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.820+0000 7f83f0bce740 1 -- 172.21.15.120:0/3957618700 wait complete. 2024-09-19T09:23:08.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.820+0000 7f83f0bce740 20 client.30745 trim_cache size 0 max 16384 2024-09-19T09:23:08.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.836+0000 7f83f0bce740 1 Processor -- start 2024-09-19T09:23:08.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.836+0000 7f83f0bce740 1 -- start start 2024-09-19T09:23:08.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.836+0000 7f83f0bce740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56206e1de5d0 0x56206e1e5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:08.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.836+0000 7f83f0bce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56206e1dfb40 0x56206e200d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:08.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.836+0000 7f83f0bce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56206e1e9370 0x56206e201290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:08.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.836+0000 7f83f0bce740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f83e81ab3f0 con 0x56206e1de5d0 2024-09-19T09:23:08.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.836+0000 7f83f0bce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f83e81acf30 con 0x56206e1dfb40 2024-09-19T09:23:08.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.836+0000 7f83f0bce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f83e81abac0 con 0x56206e1e9370 2024-09-19T09:23:08.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.837+0000 7f83e7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56206e1de5d0 0x56206e1e5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:08.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.837+0000 7f83e77fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56206e1dfb40 0x56206e200d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:08.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.837+0000 7f83e7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56206e1de5d0 0x56206e1e5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42072/0 (socket says 172.21.15.120:42072) 2024-09-19T09:23:08.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.837+0000 7f83e7fff640 1 -- 172.21.15.120:0/3493805592 learned_addr learned my addr 172.21.15.120:0/3493805592 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:08.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.837+0000 7f83ecd92640 1 --2- 172.21.15.120:0/3493805592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56206e1e9370 0x56206e201290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:08.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.837+0000 7f83e77fe640 1 -- 172.21.15.120:0/3493805592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56206e1e9370 msgr2=0x56206e201290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:08.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.837+0000 7f83e77fe640 1 --2- 172.21.15.120:0/3493805592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56206e1e9370 0x56206e201290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.837+0000 7f83e77fe640 1 -- 172.21.15.120:0/3493805592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56206e1de5d0 msgr2=0x56206e1e5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:08.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.837+0000 7f83e77fe640 1 --2- 172.21.15.120:0/3493805592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56206e1de5d0 0x56206e1e5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.837+0000 7f83e77fe640 1 -- 172.21.15.120:0/3493805592 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56206e1fb860 con 0x56206e1dfb40 2024-09-19T09:23:08.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.837+0000 7f83e7fff640 1 --2- 172.21.15.120:0/3493805592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56206e1de5d0 0x56206e1e5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:08.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.837+0000 7f83e77fe640 1 --2- 172.21.15.120:0/3493805592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56206e1dfb40 0x56206e200d50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0bbcc0 tx=0x7f83dc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc48be62e6772410 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:08.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.838+0000 7f83cdffb640 1 -- 172.21.15.120:0/3493805592 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc0bbd30 con 0x56206e1dfb40 2024-09-19T09:23:08.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.838+0000 7f83cdffb640 1 -- 172.21.15.120:0/3493805592 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83dc0d9070 con 0x56206e1dfb40 2024-09-19T09:23:08.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.838+0000 7f83cdffb640 1 -- 172.21.15.120:0/3493805592 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc0b83a0 con 0x56206e1dfb40 2024-09-19T09:23:08.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.838+0000 7f83f0bce740 1 -- 172.21.15.120:0/3493805592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56206e1dfb40 msgr2=0x56206e200d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:08.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.838+0000 7f83f0bce740 1 --2- 172.21.15.120:0/3493805592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56206e1dfb40 0x56206e200d50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0bbcc0 tx=0x7f83dc005f90 comp rx=0 tx=0).stop 2024-09-19T09:23:08.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.838+0000 7f83f0bce740 1 -- 172.21.15.120:0/3493805592 shutdown_connections 2024-09-19T09:23:08.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.838+0000 7f83f0bce740 1 --2- 172.21.15.120:0/3493805592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56206e1e9370 0x56206e201290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.838+0000 7f83f0bce740 1 --2- 172.21.15.120:0/3493805592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56206e1dfb40 0x56206e200d50 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.838+0000 7f83f0bce740 1 --2- 172.21.15.120:0/3493805592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56206e1de5d0 0x56206e1e5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.838+0000 7f83f0bce740 1 -- 172.21.15.120:0/3493805592 >> 172.21.15.120:0/3493805592 conn(0x56206e1a7310 msgr2=0x56206e1e4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:08.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.839+0000 7f83f0bce740 1 -- 172.21.15.120:0/3493805592 shutdown_connections 2024-09-19T09:23:08.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.839+0000 7f83f0bce740 1 -- 172.21.15.120:0/3493805592 wait complete. 2024-09-19T09:23:08.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.839+0000 7f83f0bce740 1 -- start start 2024-09-19T09:23:08.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.839+0000 7f83f0bce740 1 Processor -- start 2024-09-19T09:23:08.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.839+0000 7f83f0bce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56206e1de5d0 0x56206e1f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:08.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.839+0000 7f83f0bce740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56206e1dfb40 0x56206e1ee110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:08.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.839+0000 7f83f0bce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56206e1e9370 0x56206e1ee8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:08.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.839+0000 7f83f0bce740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f83e81ac860 con 0x56206e1dfb40 2024-09-19T09:23:08.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.839+0000 7f83f0bce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f83e81ac190 con 0x56206e1e9370 2024-09-19T09:23:08.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.839+0000 7f83f0bce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f83e81a93b0 con 0x56206e1de5d0 2024-09-19T09:23:08.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.839+0000 7f83f0bce740 10 client.0 _ll_register_callbacks cb 0x7f83d8009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:08.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.840+0000 7f83e77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56206e1dfb40 0x56206e1ee110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:08.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.840+0000 7f83e7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56206e1de5d0 0x56206e1f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:08.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.840+0000 7f83e77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56206e1dfb40 0x56206e1ee110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42076/0 (socket says 172.21.15.120:42076) 2024-09-19T09:23:08.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.840+0000 7f83e77fe640 1 -- 172.21.15.120:0/3824137163 learned_addr learned my addr 172.21.15.120:0/3824137163 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:08.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.840+0000 7f83ecd92640 1 --2- 172.21.15.120:0/3824137163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56206e1e9370 0x56206e1ee8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:08.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.840+0000 7f83e77fe640 1 -- 172.21.15.120:0/3824137163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56206e1de5d0 msgr2=0x56206e1f2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:08.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.840+0000 7f83e77fe640 1 --2- 172.21.15.120:0/3824137163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56206e1de5d0 0x56206e1f2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.840+0000 7f83e77fe640 1 -- 172.21.15.120:0/3824137163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56206e1e9370 msgr2=0x56206e1ee8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:08.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.840+0000 7f83e77fe640 1 --2- 172.21.15.120:0/3824137163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56206e1e9370 0x56206e1ee8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.840+0000 7f83e77fe640 1 -- 172.21.15.120:0/3824137163 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56206e262d20 con 0x56206e1dfb40 2024-09-19T09:23:08.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.840+0000 7f83e77fe640 1 --2- 172.21.15.120:0/3824137163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56206e1dfb40 0x56206e1ee110 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0b8700 tx=0x7f83dc0c4a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a506cf8c32876b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:08.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.840+0000 7f83c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:23:08.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.840+0000 7f83c77fe640 1 -- 172.21.15.120:0/3824137163 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc09f070 con 0x56206e1dfb40 2024-09-19T09:23:08.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.840+0000 7f83c77fe640 1 -- 172.21.15.120:0/3824137163 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f83dc004250 con 0x56206e1dfb40 2024-09-19T09:23:08.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.840+0000 7f83f0bce740 1 -- 172.21.15.120:0/3824137163 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83dc0d9070 con 0x56206e1dfb40 2024-09-19T09:23:08.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.840+0000 7f83c77fe640 1 -- 172.21.15.120:0/3824137163 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f83dc0bbd30 con 0x56206e1dfb40 2024-09-19T09:23:08.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.840+0000 7f83f0bce740 10 client.22128 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:08.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.840+0000 7f83f0bce740 1 -- 172.21.15.120:0/3824137163 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56206e263ee0 con 0x56206e1dfb40 2024-09-19T09:23:08.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.841+0000 7f83c77fe640 1 -- 172.21.15.120:0/3824137163 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83dc0d9070 con 0x56206e1dfb40 2024-09-19T09:23:08.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.841+0000 7f83c77fe640 1 -- 172.21.15.120:0/3824137163 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f83dc0d2030 con 0x56206e1dfb40 2024-09-19T09:23:08.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.841+0000 7f83f0bce740 10 client.22128 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:08.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.841+0000 7f83f0bce740 10 client.22128 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:23:08.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.841+0000 7f83f0bce740 20 client.22128 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:08.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.841+0000 7f83f0bce740 1 --2- 172.21.15.120:0/3824137163 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56206e264460 0x56206e266880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:08.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.841+0000 7f83f0bce740 4 client.22128 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:23:08.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.841+0000 7f83f0bce740 1 -- 172.21.15.120:0/3824137163 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56206e151860 con 0x56206e264460 2024-09-19T09:23:08.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.841+0000 7f83e7fff640 1 --2- 172.21.15.120:0/3824137163 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56206e264460 0x56206e266880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:08.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.841+0000 7f83c77fe640 10 client.22128 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:23:08.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.841+0000 7f83e7fff640 1 --2- 172.21.15.120:0/3824137163 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56206e264460 0x56206e266880 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5706814bfc0ca3d5 server_cookie=7e2ff7b705a20994 in_seq=0 out_seq=0 2024-09-19T09:23:08.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.842+0000 7f83c77fe640 1 -- 172.21.15.120:0/3824137163 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56206e151860 con 0x56206e264460 2024-09-19T09:23:08.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.842+0000 7f83c77fe640 10 client.22128 handle_command_reply: tid=0 2024-09-19T09:23:08.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.842+0000 7f83f0bce740 1 client.22128 shutdown 2024-09-19T09:23:08.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.842+0000 7f83f0bce740 1 -- 172.21.15.120:0/3824137163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56206e1dfb40 msgr2=0x56206e1ee110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:08.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.842+0000 7f83f0bce740 1 --2- 172.21.15.120:0/3824137163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56206e1dfb40 0x56206e1ee110 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0b8700 tx=0x7f83dc0c4a90 comp rx=0 tx=0).stop 2024-09-19T09:23:08.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.844+0000 7f83f0bce740 1 -- 172.21.15.120:0/3824137163 shutdown_connections 2024-09-19T09:23:08.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.844+0000 7f83f0bce740 1 --2- 172.21.15.120:0/3824137163 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x56206e264460 0x56206e266880 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.844+0000 7f83f0bce740 1 --2- 172.21.15.120:0/3824137163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56206e1e9370 0x56206e1ee8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.844+0000 7f83c77fe640 0 client.22128 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:23:08.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.844+0000 7f83f0bce740 1 --2- 172.21.15.120:0/3824137163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56206e1dfb40 0x56206e1ee110 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.844+0000 7f83f0bce740 1 --2- 172.21.15.120:0/3824137163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56206e1de5d0 0x56206e1f2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.844+0000 7f83f0bce740 1 -- 172.21.15.120:0/3824137163 >> 172.21.15.120:0/3824137163 conn(0x56206e1a7310 msgr2=0x56206e1e78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:08.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.844+0000 7f83f0bce740 1 -- 172.21.15.120:0/3824137163 shutdown_connections 2024-09-19T09:23:08.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.844+0000 7f83f0bce740 1 -- 172.21.15.120:0/3824137163 wait complete. 2024-09-19T09:23:08.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.845+0000 7f83f0bce740 20 client.22128 trim_cache size 0 max 16384 2024-09-19T09:23:08.858 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:23:08.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.845+0000 7f83ee7f5640 1 -- 172.21.15.120:0/2633455014 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83d00a3240 msgr2=0x7f83d00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:08.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.845+0000 7f83ee7f5640 1 --2- 172.21.15.120:0/2633455014 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83d00a3240 0x7f83d00a5700 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009530 tx=0x7f83d8009260 comp rx=0 tx=0).stop 2024-09-19T09:23:08.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.845+0000 7f83ee7f5640 1 -- 172.21.15.120:0/2633455014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83e81049a0 msgr2=0x7f83e819ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:08.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.845+0000 7f83ee7f5640 1 --2- 172.21.15.120:0/2633455014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83e81049a0 0x7f83e819ab80 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f83dc035040 tx=0x7f83dc033d90 comp rx=0 tx=0).stop 2024-09-19T09:23:08.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.845+0000 7f83ee7f5640 1 -- 172.21.15.120:0/2633455014 shutdown_connections 2024-09-19T09:23:08.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.845+0000 7f83ee7f5640 1 --2- 172.21.15.120:0/2633455014 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83d00a3240 0x7f83d00a5700 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.845+0000 7f83ee7f5640 1 --2- 172.21.15.120:0/2633455014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83e8109120 0x7f83e819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.845+0000 7f83ee7f5640 1 --2- 172.21.15.120:0/2633455014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83e81049a0 0x7f83e819ab80 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.845+0000 7f83ee7f5640 1 --2- 172.21.15.120:0/2633455014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83e8103ff0 0x7f83e819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:08.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.845+0000 7f83ee7f5640 1 -- 172.21.15.120:0/2633455014 >> 172.21.15.120:0/2633455014 conn(0x7f83e80ffab0 msgr2=0x7f83e8100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:08.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.846+0000 7f83ee7f5640 1 -- 172.21.15.120:0/2633455014 shutdown_connections 2024-09-19T09:23:08.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:08.846+0000 7f83ee7f5640 1 -- 172.21.15.120:0/2633455014 wait complete. 2024-09-19T09:23:09.376 INFO:tasks.workunit.client.0.smithi120.stdout: 10 169129 20.16 MB/sec execute 304 sec latency 55.851 ms 2024-09-19T09:23:09.848 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:23:10.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:10 smithi120 ceph-mon[31150]: pgmap v954: 97 pgs: 97 active+clean; 412 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.4 MiB/s wr, 689 op/s 2024-09-19T09:23:10.163 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:23:10.376 INFO:tasks.workunit.client.0.smithi120.stdout: 10 169669 20.19 MB/sec execute 305 sec latency 81.981 ms 2024-09-19T09:23:10.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:10 smithi196 ceph-mon[35150]: pgmap v954: 97 pgs: 97 active+clean; 412 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.4 MiB/s wr, 689 op/s 2024-09-19T09:23:10.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:10 smithi153 ceph-mon[35071]: pgmap v954: 97 pgs: 97 active+clean; 412 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.4 MiB/s wr, 689 op/s 2024-09-19T09:23:10.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.715+0000 7f18998a4640 1 -- 172.21.15.120:0/2375388656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1894106f10 msgr2=0x7f189410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:10.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.716+0000 7f18998a4640 1 --2- 172.21.15.120:0/2375388656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1894106f10 0x7f189410ea90 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f188c009ca0 tx=0x7f188c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.716+0000 7f18998a4640 1 -- 172.21.15.120:0/2375388656 shutdown_connections 2024-09-19T09:23:10.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.716+0000 7f18998a4640 1 --2- 172.21.15.120:0/2375388656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1894106f10 0x7f189410ea90 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.716+0000 7f18998a4640 1 --2- 172.21.15.120:0/2375388656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1894102790 0x7f18941067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.716+0000 7f18998a4640 1 --2- 172.21.15.120:0/2375388656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1894101de0 0x7f18941021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.716+0000 7f18998a4640 1 -- 172.21.15.120:0/2375388656 >> 172.21.15.120:0/2375388656 conn(0x7f18940fd8c0 msgr2=0x7f18940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:10.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.716+0000 7f18998a4640 1 -- 172.21.15.120:0/2375388656 shutdown_connections 2024-09-19T09:23:10.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.716+0000 7f18998a4640 1 -- 172.21.15.120:0/2375388656 wait complete. 2024-09-19T09:23:10.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.716+0000 7f18998a4640 1 Processor -- start 2024-09-19T09:23:10.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.717+0000 7f18998a4640 1 -- start start 2024-09-19T09:23:10.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.717+0000 7f18998a4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1894101de0 0x7f1894198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:10.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.717+0000 7f18998a4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1894102790 0x7f18941989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:10.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.717+0000 7f18998a4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1894106f10 0x7f189419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:10.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.717+0000 7f18998a4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1894111f90 con 0x7f1894101de0 2024-09-19T09:23:10.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.717+0000 7f18998a4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1894111e10 con 0x7f1894106f10 2024-09-19T09:23:10.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.717+0000 7f18998a4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1894112110 con 0x7f1894102790 2024-09-19T09:23:10.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.717+0000 7f1892ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1894101de0 0x7f1894198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:10.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.717+0000 7f18937fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1894106f10 0x7f189419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:10.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.717+0000 7f18937fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1894106f10 0x7f189419cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:43194/0 (socket says 172.21.15.120:43194) 2024-09-19T09:23:10.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.717+0000 7f18937fe640 1 -- 172.21.15.120:0/942796541 learned_addr learned my addr 172.21.15.120:0/942796541 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:10.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.717+0000 7f188bfff640 1 --2- 172.21.15.120:0/942796541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1894102790 0x7f18941989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:10.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.717+0000 7f18937fe640 1 -- 172.21.15.120:0/942796541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1894102790 msgr2=0x7f18941989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:10.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.717+0000 7f18937fe640 1 --2- 172.21.15.120:0/942796541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1894102790 0x7f18941989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.718+0000 7f18937fe640 1 -- 172.21.15.120:0/942796541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1894101de0 msgr2=0x7f1894198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:10.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.718+0000 7f18937fe640 1 --2- 172.21.15.120:0/942796541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1894101de0 0x7f1894198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.718+0000 7f18937fe640 1 -- 172.21.15.120:0/942796541 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189419d410 con 0x7f1894106f10 2024-09-19T09:23:10.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.718+0000 7f188bfff640 1 --2- 172.21.15.120:0/942796541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1894102790 0x7f18941989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:23:10.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.718+0000 7f1892ffd640 1 --2- 172.21.15.120:0/942796541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1894101de0 0x7f1894198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:10.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.718+0000 7f18937fe640 1 --2- 172.21.15.120:0/942796541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1894106f10 0x7f189419cd30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f188c031a20 tx=0x7f188c031a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:10.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.718+0000 7f18988a2640 1 -- 172.21.15.120:0/942796541 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c003b70 con 0x7f1894106f10 2024-09-19T09:23:10.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.718+0000 7f18998a4640 1 -- 172.21.15.120:0/942796541 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f189419d6a0 con 0x7f1894106f10 2024-09-19T09:23:10.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.718+0000 7f18998a4640 1 -- 172.21.15.120:0/942796541 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18941a4fe0 con 0x7f1894106f10 2024-09-19T09:23:10.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.718+0000 7f18988a2640 1 -- 172.21.15.120:0/942796541 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f188c003d10 con 0x7f1894106f10 2024-09-19T09:23:10.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.718+0000 7f18988a2640 1 -- 172.21.15.120:0/942796541 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c02fb40 con 0x7f1894106f10 2024-09-19T09:23:10.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.721+0000 7f18988a2640 1 -- 172.21.15.120:0/942796541 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f188c004050 con 0x7f1894106f10 2024-09-19T09:23:10.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.722+0000 7f189bc7d740 1 Processor -- start 2024-09-19T09:23:10.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.722+0000 7f189bc7d740 1 -- start start 2024-09-19T09:23:10.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.722+0000 7f189bc7d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d7d120b40 0x563d7d11f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:10.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.722+0000 7f189bc7d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d7d11f5d0 0x563d7d11fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:10.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.722+0000 7f189bc7d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d7d12a370 0x563d7d12c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:10.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.722+0000 7f189bc7d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563d7d0852f0 con 0x563d7d120b40 2024-09-19T09:23:10.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.722+0000 7f189bc7d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563d7d02ccf0 con 0x563d7d12a370 2024-09-19T09:23:10.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.722+0000 7f189bc7d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563d7d0812b0 con 0x563d7d11f5d0 2024-09-19T09:23:10.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.723+0000 7f1892ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d7d11f5d0 0x563d7d11fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:10.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.723+0000 7f1892ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d7d11f5d0 0x563d7d11fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:53806/0 (socket says 172.21.15.120:53806) 2024-09-19T09:23:10.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.723+0000 7f1892ffd640 1 -- 172.21.15.120:0/3730711251 learned_addr learned my addr 172.21.15.120:0/3730711251 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:10.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.723+0000 7f1892ffd640 1 -- 172.21.15.120:0/3730711251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d7d12a370 msgr2=0x563d7d12c760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:23:10.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.723+0000 7f1892ffd640 1 --2- 172.21.15.120:0/3730711251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d7d12a370 0x563d7d12c760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.723+0000 7f188bfff640 1 --2- 172.21.15.120:0/3730711251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d7d120b40 0x563d7d11f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:10.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.723+0000 7f1892ffd640 1 -- 172.21.15.120:0/3730711251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d7d120b40 msgr2=0x563d7d11f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:10.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.724+0000 7f1892ffd640 1 --2- 172.21.15.120:0/3730711251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d7d120b40 0x563d7d11f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.724+0000 7f1892ffd640 1 -- 172.21.15.120:0/3730711251 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d7d0113a0 con 0x563d7d11f5d0 2024-09-19T09:23:10.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.724+0000 7f188bfff640 1 --2- 172.21.15.120:0/3730711251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d7d120b40 0x563d7d11f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:23:10.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.724+0000 7f18988a2640 1 --2- 172.21.15.120:0/942796541 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f18800a76d0 0x7f18800a9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:10.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.724+0000 7f18988a2640 1 -- 172.21.15.120:0/942796541 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f188c0f09b0 con 0x7f1894106f10 2024-09-19T09:23:10.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.724+0000 7f188bfff640 1 --2- 172.21.15.120:0/942796541 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f18800a76d0 0x7f18800a9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:10.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.724+0000 7f1892ffd640 1 --2- 172.21.15.120:0/3730711251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d7d11f5d0 0x563d7d11fa10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1884009e70 tx=0x7f188400e5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e0e1921e6f110e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:10.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.724+0000 7f1889ffb640 1 -- 172.21.15.120:0/3730711251 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884017070 con 0x563d7d11f5d0 2024-09-19T09:23:10.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.725+0000 7f1889ffb640 1 -- 172.21.15.120:0/3730711251 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1884013950 con 0x563d7d11f5d0 2024-09-19T09:23:10.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.725+0000 7f1889ffb640 1 -- 172.21.15.120:0/3730711251 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884017070 con 0x563d7d11f5d0 2024-09-19T09:23:10.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.725+0000 7f189bc7d740 1 -- 172.21.15.120:0/3730711251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d7d11f5d0 msgr2=0x563d7d11fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:10.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.725+0000 7f189bc7d740 1 --2- 172.21.15.120:0/3730711251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d7d11f5d0 0x563d7d11fa10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1884009e70 tx=0x7f188400e5a0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.725+0000 7f189bc7d740 1 -- 172.21.15.120:0/3730711251 shutdown_connections 2024-09-19T09:23:10.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.725+0000 7f189bc7d740 1 --2- 172.21.15.120:0/3730711251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d7d12a370 0x563d7d12c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.725+0000 7f189bc7d740 1 --2- 172.21.15.120:0/3730711251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d7d11f5d0 0x563d7d11fa10 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.725+0000 7f189bc7d740 1 --2- 172.21.15.120:0/3730711251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d7d120b40 0x563d7d11f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.725+0000 7f189bc7d740 1 -- 172.21.15.120:0/3730711251 >> 172.21.15.120:0/3730711251 conn(0x563d7d0e8310 msgr2=0x563d7d11bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:10.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.726+0000 7f188bfff640 1 --2- 172.21.15.120:0/942796541 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f18800a76d0 0x7f18800a9b90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f187c009460 tx=0x7f187c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:10.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.726+0000 7f189bc7d740 1 -- 172.21.15.120:0/3730711251 shutdown_connections 2024-09-19T09:23:10.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.726+0000 7f189bc7d740 1 -- 172.21.15.120:0/3730711251 wait complete. 2024-09-19T09:23:10.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.726+0000 7f189bc7d740 1 -- start start 2024-09-19T09:23:10.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.726+0000 7f189bc7d740 1 Processor -- start 2024-09-19T09:23:10.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.727+0000 7f189bc7d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d7d11f5d0 0x563d7d129c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:10.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.727+0000 7f189bc7d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d7d120b40 0x563d7d126f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:10.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.727+0000 7f189bc7d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d7d12a370 0x563d7d127480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:10.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.727+0000 7f189bc7d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563d7d12d0c0 con 0x563d7d120b40 2024-09-19T09:23:10.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.727+0000 7f189bc7d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563d7d040060 con 0x563d7d11f5d0 2024-09-19T09:23:10.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.727+0000 7f189bc7d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563d7d12d240 con 0x563d7d12a370 2024-09-19T09:23:10.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.727+0000 7f189bc7d740 10 client.0 _ll_register_callbacks cb 0x563d7cd938c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:10.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.727+0000 7f1892ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d7d11f5d0 0x563d7d129c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:10.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.727+0000 7f18937fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d7d12a370 0x563d7d127480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:10.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.727+0000 7f18937fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d7d12a370 0x563d7d127480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:53814/0 (socket says 172.21.15.120:53814) 2024-09-19T09:23:10.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.727+0000 7f18937fe640 1 -- 172.21.15.120:0/1109888138 learned_addr learned my addr 172.21.15.120:0/1109888138 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:10.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.727+0000 7f188bfff640 1 --2- 172.21.15.120:0/1109888138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d7d120b40 0x563d7d126f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:10.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.727+0000 7f188bfff640 1 -- 172.21.15.120:0/1109888138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d7d12a370 msgr2=0x563d7d127480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:10.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.727+0000 7f188bfff640 1 --2- 172.21.15.120:0/1109888138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d7d12a370 0x563d7d127480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.727+0000 7f188bfff640 1 -- 172.21.15.120:0/1109888138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d7d11f5d0 msgr2=0x563d7d129c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:10.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.727+0000 7f188bfff640 1 --2- 172.21.15.120:0/1109888138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d7d11f5d0 0x563d7d129c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.727+0000 7f188bfff640 1 -- 172.21.15.120:0/1109888138 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d7cdf63c0 con 0x563d7d120b40 2024-09-19T09:23:10.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.727+0000 7f18937fe640 1 --2- 172.21.15.120:0/1109888138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d7d12a370 0x563d7d127480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:10.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.728+0000 7f188bfff640 1 --2- 172.21.15.120:0/1109888138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d7d120b40 0x563d7d126f40 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x563d7d120070 tx=0x7f187c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f78f42a7e0218f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:10.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.728+0000 7f1872ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:23:10.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.728+0000 7f1872ffd640 1 -- 172.21.15.120:0/1109888138 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c00fb20 con 0x563d7d120b40 2024-09-19T09:23:10.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.728+0000 7f1872ffd640 1 -- 172.21.15.120:0/1109888138 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f187c00fcc0 con 0x563d7d120b40 2024-09-19T09:23:10.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.728+0000 7f189bc7d740 1 -- 172.21.15.120:0/1109888138 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1884013950 con 0x563d7d120b40 2024-09-19T09:23:10.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.728+0000 7f1872ffd640 1 -- 172.21.15.120:0/1109888138 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f187c01f420 con 0x563d7d120b40 2024-09-19T09:23:10.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.728+0000 7f189bc7d740 10 client.22137 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:10.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.728+0000 7f189bc7d740 1 -- 172.21.15.120:0/1109888138 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563d7d141320 con 0x563d7d120b40 2024-09-19T09:23:10.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.728+0000 7f1872ffd640 1 -- 172.21.15.120:0/1109888138 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1884013950 con 0x563d7d120b40 2024-09-19T09:23:10.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.729+0000 7f1872ffd640 1 -- 172.21.15.120:0/1109888138 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f187c0326d0 con 0x563d7d120b40 2024-09-19T09:23:10.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.729+0000 7f189bc7d740 10 client.22137 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:10.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.729+0000 7f189bc7d740 10 client.22137 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:23:10.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.729+0000 7f189bc7d740 20 client.22137 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:10.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.729+0000 7f189bc7d740 1 --2- 172.21.15.120:0/1109888138 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x563d7d1428b0 0x563d7d144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:10.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.729+0000 7f189bc7d740 4 client.22137 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:23:10.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.729+0000 7f189bc7d740 1 -- 172.21.15.120:0/1109888138 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d7d092860 con 0x563d7d1428b0 2024-09-19T09:23:10.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.729+0000 7f1892ffd640 1 --2- 172.21.15.120:0/1109888138 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x563d7d1428b0 0x563d7d144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:10.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.730+0000 7f1872ffd640 10 client.22137 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:23:10.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.730+0000 7f1892ffd640 1 --2- 172.21.15.120:0/1109888138 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x563d7d1428b0 0x563d7d144cd0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7dc6b05307e2e00 server_cookie=125954a504233b2b in_seq=0 out_seq=0 2024-09-19T09:23:10.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.731+0000 7f1872ffd640 1 -- 172.21.15.120:0/1109888138 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d7d092860 con 0x563d7d1428b0 2024-09-19T09:23:10.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.731+0000 7f1872ffd640 10 client.22137 handle_command_reply: tid=0 2024-09-19T09:23:10.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.731+0000 7f189bc7d740 1 client.22137 shutdown 2024-09-19T09:23:10.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.731+0000 7f189bc7d740 1 -- 172.21.15.120:0/1109888138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d7d120b40 msgr2=0x563d7d126f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:10.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.731+0000 7f189bc7d740 1 --2- 172.21.15.120:0/1109888138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d7d120b40 0x563d7d126f40 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x563d7d120070 tx=0x7f187c0046c0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.732+0000 7f189bc7d740 1 -- 172.21.15.120:0/1109888138 shutdown_connections 2024-09-19T09:23:10.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.732+0000 7f189bc7d740 1 --2- 172.21.15.120:0/1109888138 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x563d7d1428b0 0x563d7d144cd0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.732+0000 7f189bc7d740 1 --2- 172.21.15.120:0/1109888138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d7d12a370 0x563d7d127480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.732+0000 7f189bc7d740 1 --2- 172.21.15.120:0/1109888138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d7d120b40 0x563d7d126f40 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.732+0000 7f189bc7d740 1 --2- 172.21.15.120:0/1109888138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d7d11f5d0 0x563d7d129c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.732+0000 7f189bc7d740 1 -- 172.21.15.120:0/1109888138 >> 172.21.15.120:0/1109888138 conn(0x563d7d0e8310 msgr2=0x563d7d121930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:10.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.732+0000 7f1872ffd640 0 client.22137 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:23:10.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.732+0000 7f189bc7d740 1 -- 172.21.15.120:0/1109888138 shutdown_connections 2024-09-19T09:23:10.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.732+0000 7f189bc7d740 1 -- 172.21.15.120:0/1109888138 wait complete. 2024-09-19T09:23:10.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.732+0000 7f189bc7d740 20 client.22137 trim_cache size 0 max 16384 2024-09-19T09:23:10.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.750+0000 7f189bc7d740 1 Processor -- start 2024-09-19T09:23:10.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.750+0000 7f189bc7d740 1 -- start start 2024-09-19T09:23:10.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.750+0000 7f189bc7d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d7d11f5d0 0x563d7d13ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:10.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.750+0000 7f189bc7d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d7d120b40 0x563d7d126f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:10.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.750+0000 7f189bc7d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d7d12a370 0x563d7d127440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:10.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.750+0000 7f189bc7d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f187400e4a0 con 0x563d7d11f5d0 2024-09-19T09:23:10.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.750+0000 7f189bc7d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f187400ffe0 con 0x563d7d12a370 2024-09-19T09:23:10.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.750+0000 7f189bc7d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f187400eb70 con 0x563d7d120b40 2024-09-19T09:23:10.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.750+0000 7f18937fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d7d12a370 0x563d7d127440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:10.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.750+0000 7f18937fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d7d12a370 0x563d7d127440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:43218/0 (socket says 172.21.15.120:43218) 2024-09-19T09:23:10.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.750+0000 7f18937fe640 1 -- 172.21.15.120:0/1149893732 learned_addr learned my addr 172.21.15.120:0/1149893732 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:10.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.750+0000 7f188bfff640 1 --2- 172.21.15.120:0/1149893732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d7d120b40 0x563d7d126f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:10.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.750+0000 7f1892ffd640 1 --2- 172.21.15.120:0/1149893732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d7d11f5d0 0x563d7d13ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:10.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.750+0000 7f18937fe640 1 -- 172.21.15.120:0/1149893732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d7d120b40 msgr2=0x563d7d126f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:10.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.750+0000 7f18937fe640 1 --2- 172.21.15.120:0/1149893732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d7d120b40 0x563d7d126f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.750+0000 7f18937fe640 1 -- 172.21.15.120:0/1149893732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d7d11f5d0 msgr2=0x563d7d13ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:10.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.750+0000 7f18937fe640 1 --2- 172.21.15.120:0/1149893732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d7d11f5d0 0x563d7d13ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.751+0000 7f18937fe640 1 -- 172.21.15.120:0/1149893732 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d7d11c1a0 con 0x563d7d12a370 2024-09-19T09:23:10.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.751+0000 7f18937fe640 1 --2- 172.21.15.120:0/1149893732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d7d12a370 0x563d7d127440 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f188c005fa0 tx=0x7f188c0bcc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e72ae9c1c80955b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:10.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.751+0000 7f18897fa640 1 -- 172.21.15.120:0/1149893732 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c0bd020 con 0x563d7d12a370 2024-09-19T09:23:10.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.751+0000 7f18897fa640 1 -- 172.21.15.120:0/1149893732 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f188c036040 con 0x563d7d12a370 2024-09-19T09:23:10.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.751+0000 7f18897fa640 1 -- 172.21.15.120:0/1149893732 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c02fd60 con 0x563d7d12a370 2024-09-19T09:23:10.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.752+0000 7f189bc7d740 1 -- 172.21.15.120:0/1149893732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d7d12a370 msgr2=0x563d7d127440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:10.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.752+0000 7f189bc7d740 1 --2- 172.21.15.120:0/1149893732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d7d12a370 0x563d7d127440 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f188c005fa0 tx=0x7f188c0bcc20 comp rx=0 tx=0).stop 2024-09-19T09:23:10.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.752+0000 7f189bc7d740 1 -- 172.21.15.120:0/1149893732 shutdown_connections 2024-09-19T09:23:10.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.752+0000 7f189bc7d740 1 --2- 172.21.15.120:0/1149893732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d7d12a370 0x563d7d127440 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.752+0000 7f189bc7d740 1 --2- 172.21.15.120:0/1149893732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d7d120b40 0x563d7d126f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.752+0000 7f189bc7d740 1 --2- 172.21.15.120:0/1149893732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d7d11f5d0 0x563d7d13ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.752+0000 7f189bc7d740 1 -- 172.21.15.120:0/1149893732 >> 172.21.15.120:0/1149893732 conn(0x563d7d0e8310 msgr2=0x563d7d12c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:10.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.752+0000 7f189bc7d740 1 -- 172.21.15.120:0/1149893732 shutdown_connections 2024-09-19T09:23:10.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.752+0000 7f189bc7d740 1 -- 172.21.15.120:0/1149893732 wait complete. 2024-09-19T09:23:10.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.752+0000 7f189bc7d740 1 -- start start 2024-09-19T09:23:10.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.752+0000 7f189bc7d740 1 Processor -- start 2024-09-19T09:23:10.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.753+0000 7f189bc7d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d7d11f5d0 0x563d7d12add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:10.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.753+0000 7f189bc7d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d7d120b40 0x563d7d12b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:10.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.753+0000 7f189bc7d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d7d12a370 0x563d7d12b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:10.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.753+0000 7f189bc7d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f187400f910 con 0x563d7d11f5d0 2024-09-19T09:23:10.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.753+0000 7f189bc7d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f187400f240 con 0x563d7d120b40 2024-09-19T09:23:10.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.753+0000 7f189bc7d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f187400c460 con 0x563d7d12a370 2024-09-19T09:23:10.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.753+0000 7f189bc7d740 10 client.0 _ll_register_callbacks cb 0x7f187c011f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:10.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.753+0000 7f1892ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d7d11f5d0 0x563d7d12add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:10.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.753+0000 7f1892ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d7d11f5d0 0x563d7d12add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42118/0 (socket says 172.21.15.120:42118) 2024-09-19T09:23:10.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.753+0000 7f188bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d7d120b40 0x563d7d12b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:10.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.753+0000 7f1892ffd640 1 -- 172.21.15.120:0/1173064334 learned_addr learned my addr 172.21.15.120:0/1173064334 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:10.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.753+0000 7f18937fe640 1 --2- 172.21.15.120:0/1173064334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d7d12a370 0x563d7d12b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:10.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.753+0000 7f1892ffd640 1 -- 172.21.15.120:0/1173064334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d7d12a370 msgr2=0x563d7d12b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:10.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.753+0000 7f1892ffd640 1 --2- 172.21.15.120:0/1173064334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d7d12a370 0x563d7d12b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.753+0000 7f1892ffd640 1 -- 172.21.15.120:0/1173064334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d7d120b40 msgr2=0x563d7d12b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:10.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.753+0000 7f1892ffd640 1 --2- 172.21.15.120:0/1173064334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d7d120b40 0x563d7d12b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.753+0000 7f1892ffd640 1 -- 172.21.15.120:0/1173064334 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d7d139ef0 con 0x563d7d11f5d0 2024-09-19T09:23:10.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.753+0000 7f188bfff640 1 --2- 172.21.15.120:0/1173064334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d7d120b40 0x563d7d12b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:10.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.753+0000 7f1892ffd640 1 --2- 172.21.15.120:0/1173064334 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d7d11f5d0 0x563d7d12add0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f188400eaa0 tx=0x7f1884009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eedd57e0cff75131 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:10.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.753+0000 7f1872ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:23:10.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.753+0000 7f1872ffd640 1 -- 172.21.15.120:0/1173064334 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884017040 con 0x563d7d11f5d0 2024-09-19T09:23:10.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.754+0000 7f1872ffd640 1 -- 172.21.15.120:0/1173064334 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f1884012680 con 0x563d7d11f5d0 2024-09-19T09:23:10.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.754+0000 7f189bc7d740 1 -- 172.21.15.120:0/1173064334 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f188c036040 con 0x563d7d11f5d0 2024-09-19T09:23:10.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.754+0000 7f1872ffd640 1 -- 172.21.15.120:0/1173064334 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1884012880 con 0x563d7d11f5d0 2024-09-19T09:23:10.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.754+0000 7f189bc7d740 10 client.22143 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:10.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.754+0000 7f189bc7d740 1 -- 172.21.15.120:0/1173064334 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563d7d13b1d0 con 0x563d7d11f5d0 2024-09-19T09:23:10.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.754+0000 7f1872ffd640 1 -- 172.21.15.120:0/1173064334 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f188c036040 con 0x563d7d11f5d0 2024-09-19T09:23:10.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.754+0000 7f1872ffd640 1 -- 172.21.15.120:0/1173064334 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f18840066e0 con 0x563d7d11f5d0 2024-09-19T09:23:10.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.754+0000 7f189bc7d740 10 client.22143 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:10.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.754+0000 7f189bc7d740 10 client.22143 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:23:10.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.754+0000 7f189bc7d740 20 client.22143 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:10.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.754+0000 7f189bc7d740 1 --2- 172.21.15.120:0/1173064334 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x563d7d13b670 0x563d7d1af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:10.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.754+0000 7f189bc7d740 4 client.22143 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:23:10.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.754+0000 7f189bc7d740 1 -- 172.21.15.120:0/1173064334 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563d7d08cd70 con 0x563d7d13b670 2024-09-19T09:23:10.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.754+0000 7f188bfff640 1 --2- 172.21.15.120:0/1173064334 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x563d7d13b670 0x563d7d1af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:10.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.755+0000 7f1872ffd640 10 client.22143 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:23:10.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.755+0000 7f188bfff640 1 --2- 172.21.15.120:0/1173064334 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x563d7d13b670 0x563d7d1af5c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3416cdeab9f77fa9 server_cookie=b2c3c63e5b0c0e20 in_seq=0 out_seq=0 2024-09-19T09:23:10.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.755+0000 7f1872ffd640 1 -- 172.21.15.120:0/1173064334 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563d7d08cd70 con 0x563d7d13b670 2024-09-19T09:23:10.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.755+0000 7f1872ffd640 10 client.22143 handle_command_reply: tid=0 2024-09-19T09:23:10.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.755+0000 7f189bc7d740 1 client.22143 shutdown 2024-09-19T09:23:10.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.755+0000 7f189bc7d740 1 -- 172.21.15.120:0/1173064334 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d7d11f5d0 msgr2=0x563d7d12add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:10.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.755+0000 7f189bc7d740 1 --2- 172.21.15.120:0/1173064334 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d7d11f5d0 0x563d7d12add0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f188400eaa0 tx=0x7f1884009ac0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.755+0000 7f189bc7d740 1 -- 172.21.15.120:0/1173064334 shutdown_connections 2024-09-19T09:23:10.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.755+0000 7f189bc7d740 1 --2- 172.21.15.120:0/1173064334 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x563d7d13b670 0x563d7d1af5c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.755+0000 7f1872ffd640 0 client.22143 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:23:10.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.755+0000 7f189bc7d740 1 --2- 172.21.15.120:0/1173064334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d7d12a370 0x563d7d12b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.755+0000 7f189bc7d740 1 --2- 172.21.15.120:0/1173064334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563d7d120b40 0x563d7d12b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.755+0000 7f189bc7d740 1 --2- 172.21.15.120:0/1173064334 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d7d11f5d0 0x563d7d12add0 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.755+0000 7f189bc7d740 1 -- 172.21.15.120:0/1173064334 >> 172.21.15.120:0/1173064334 conn(0x563d7d0e8310 msgr2=0x563d7d125760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:10.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.755+0000 7f189bc7d740 1 -- 172.21.15.120:0/1173064334 shutdown_connections 2024-09-19T09:23:10.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.755+0000 7f189bc7d740 1 -- 172.21.15.120:0/1173064334 wait complete. 2024-09-19T09:23:10.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.756+0000 7f189bc7d740 20 client.22143 trim_cache size 0 max 16384 2024-09-19T09:23:10.763 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:23:10.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.756+0000 7f18998a4640 1 -- 172.21.15.120:0/942796541 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f18800a76d0 msgr2=0x7f18800a9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:10.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.756+0000 7f18998a4640 1 --2- 172.21.15.120:0/942796541 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f18800a76d0 0x7f18800a9b90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f187c009460 tx=0x7f187c005f90 comp rx=0 tx=0).stop 2024-09-19T09:23:10.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.756+0000 7f18998a4640 1 -- 172.21.15.120:0/942796541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1894106f10 msgr2=0x7f189419cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:10.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.756+0000 7f18998a4640 1 --2- 172.21.15.120:0/942796541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1894106f10 0x7f189419cd30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f188c031a20 tx=0x7f188c031a50 comp rx=0 tx=0).stop 2024-09-19T09:23:10.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.756+0000 7f18998a4640 1 -- 172.21.15.120:0/942796541 shutdown_connections 2024-09-19T09:23:10.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.756+0000 7f18998a4640 1 --2- 172.21.15.120:0/942796541 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f18800a76d0 0x7f18800a9b90 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.757+0000 7f18998a4640 1 --2- 172.21.15.120:0/942796541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1894106f10 0x7f189419cd30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.757+0000 7f18998a4640 1 --2- 172.21.15.120:0/942796541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1894102790 0x7f18941989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.757+0000 7f18998a4640 1 --2- 172.21.15.120:0/942796541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1894101de0 0x7f1894198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:10.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.757+0000 7f18998a4640 1 -- 172.21.15.120:0/942796541 >> 172.21.15.120:0/942796541 conn(0x7f18940fd8c0 msgr2=0x7f18940fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:10.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.757+0000 7f18998a4640 1 -- 172.21.15.120:0/942796541 shutdown_connections 2024-09-19T09:23:10.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:10.757+0000 7f18998a4640 1 -- 172.21.15.120:0/942796541 wait complete. 2024-09-19T09:23:11.376 INFO:tasks.workunit.client.0.smithi120.stdout: 10 170164 20.22 MB/sec execute 306 sec latency 89.645 ms 2024-09-19T09:23:11.599 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:23:11.909 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:23:12.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:12 smithi120 ceph-mon[31150]: pgmap v955: 97 pgs: 97 active+clean; 415 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.5 MiB/s wr, 549 op/s 2024-09-19T09:23:12.376 INFO:tasks.workunit.client.0.smithi120.stdout: 10 170620 20.20 MB/sec execute 307 sec latency 116.280 ms 2024-09-19T09:23:12.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:12 smithi196 ceph-mon[35150]: pgmap v955: 97 pgs: 97 active+clean; 415 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.5 MiB/s wr, 549 op/s 2024-09-19T09:23:12.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:12 smithi153 ceph-mon[35071]: pgmap v955: 97 pgs: 97 active+clean; 415 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.5 MiB/s wr, 549 op/s 2024-09-19T09:23:12.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.448+0000 7f4b37d59640 1 -- 172.21.15.120:0/295710513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b2809b600 msgr2=0x7f4b2809b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:12.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.448+0000 7f4b37d59640 1 --2- 172.21.15.120:0/295710513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b2809b600 0x7f4b2809b9e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009900 tx=0x7f4b2c02a410 comp rx=0 tx=0).stop 2024-09-19T09:23:12.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.448+0000 7f4b37d59640 1 -- 172.21.15.120:0/295710513 shutdown_connections 2024-09-19T09:23:12.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.448+0000 7f4b37d59640 1 --2- 172.21.15.120:0/295710513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b280050e0 0x7f4b280a2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.448+0000 7f4b37d59640 1 --2- 172.21.15.120:0/295710513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b28004740 0x7f4b28004ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.448+0000 7f4b37d59640 1 --2- 172.21.15.120:0/295710513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b2809b600 0x7f4b2809b9e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.448+0000 7f4b37d59640 1 -- 172.21.15.120:0/295710513 >> 172.21.15.120:0/295710513 conn(0x7f4b2800a1b0 msgr2=0x7f4b2800a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:12.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.448+0000 7f4b37d59640 1 -- 172.21.15.120:0/295710513 shutdown_connections 2024-09-19T09:23:12.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.448+0000 7f4b37d59640 1 -- 172.21.15.120:0/295710513 wait complete. 2024-09-19T09:23:12.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.449+0000 7f4b37d59640 1 Processor -- start 2024-09-19T09:23:12.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.449+0000 7f4b37d59640 1 -- start start 2024-09-19T09:23:12.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.450+0000 7f4b37d59640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b28004740 0x7f4b2812e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:12.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.450+0000 7f4b37d59640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b280050e0 0x7f4b2812e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:12.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.450+0000 7f4b37d59640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b2809b600 0x7f4b281328d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:12.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.450+0000 7f4b37d59640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4b28096620 con 0x7f4b28004740 2024-09-19T09:23:12.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.450+0000 7f4b352f4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b280050e0 0x7f4b2812e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:12.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.450+0000 7f4b352f4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b280050e0 0x7f4b2812e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:50584/0 (socket says 172.21.15.120:50584) 2024-09-19T09:23:12.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.450+0000 7f4b352f4640 1 -- 172.21.15.120:0/3051379203 learned_addr learned my addr 172.21.15.120:0/3051379203 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:12.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.450+0000 7f4b37d59640 1 -- 172.21.15.120:0/3051379203 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4b280964a0 con 0x7f4b2809b600 2024-09-19T09:23:12.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.451+0000 7f4b37d59640 1 -- 172.21.15.120:0/3051379203 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4b280967a0 con 0x7f4b280050e0 2024-09-19T09:23:12.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.451+0000 7f4b35af5640 1 --2- 172.21.15.120:0/3051379203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b28004740 0x7f4b2812e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:12.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.451+0000 7f4b362f6640 1 --2- 172.21.15.120:0/3051379203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b2809b600 0x7f4b281328d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:12.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.451+0000 7f4b352f4640 1 -- 172.21.15.120:0/3051379203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b2809b600 msgr2=0x7f4b281328d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:12.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.451+0000 7f4b352f4640 1 --2- 172.21.15.120:0/3051379203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b2809b600 0x7f4b281328d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.451+0000 7f4b352f4640 1 -- 172.21.15.120:0/3051379203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b28004740 msgr2=0x7f4b2812e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:12.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.451+0000 7f4b352f4640 1 --2- 172.21.15.120:0/3051379203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b28004740 0x7f4b2812e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.451+0000 7f4b352f4640 1 -- 172.21.15.120:0/3051379203 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b28132fb0 con 0x7f4b280050e0 2024-09-19T09:23:12.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.451+0000 7f4b362f6640 1 --2- 172.21.15.120:0/3051379203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b2809b600 0x7f4b281328d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:12.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.452+0000 7f4b35af5640 1 --2- 172.21.15.120:0/3051379203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b28004740 0x7f4b2812e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:23:12.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.452+0000 7f4b352f4640 1 --2- 172.21.15.120:0/3051379203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b280050e0 0x7f4b2812e540 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4b2000ea60 tx=0x7f4b2000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:12.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.452+0000 7f4b26ffd640 1 -- 172.21.15.120:0/3051379203 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2000cde0 con 0x7f4b280050e0 2024-09-19T09:23:12.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.452+0000 7f4b26ffd640 1 -- 172.21.15.120:0/3051379203 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b200035f0 con 0x7f4b280050e0 2024-09-19T09:23:12.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.452+0000 7f4b37d59640 1 -- 172.21.15.120:0/3051379203 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b281332a0 con 0x7f4b280050e0 2024-09-19T09:23:12.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.452+0000 7f4b37d59640 1 -- 172.21.15.120:0/3051379203 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b2813ab80 con 0x7f4b280050e0 2024-09-19T09:23:12.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.453+0000 7f4b26ffd640 1 -- 172.21.15.120:0/3051379203 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b20010670 con 0x7f4b280050e0 2024-09-19T09:23:12.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.457+0000 7f4b26ffd640 1 -- 172.21.15.120:0/3051379203 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f4b20004000 con 0x7f4b280050e0 2024-09-19T09:23:12.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.457+0000 7f4b3a132740 1 Processor -- start 2024-09-19T09:23:12.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.458+0000 7f4b3a132740 1 -- start start 2024-09-19T09:23:12.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.459+0000 7f4b3a132740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9bcef1530 0x55b9bceefa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:12.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.459+0000 7f4b3a132740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9bceeffc0 0x55b9bcef0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:12.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.459+0000 7f4b3a132740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bcefad60 0x55b9bcefd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:12.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.459+0000 7f4b3a132740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b9bce4d2f0 con 0x55b9bcef1530 2024-09-19T09:23:12.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.459+0000 7f4b3a132740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b9bcdf4cf0 con 0x55b9bcefad60 2024-09-19T09:23:12.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.460+0000 7f4b3a132740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b9bce492b0 con 0x55b9bceeffc0 2024-09-19T09:23:12.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.460+0000 7f4b26ffd640 1 --2- 172.21.15.120:0/3051379203 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4b1c0a32d0 0x7f4b1c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:12.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.460+0000 7f4b35af5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9bceeffc0 0x55b9bcef0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:12.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.460+0000 7f4b352f4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bcefad60 0x55b9bcefd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:12.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.460+0000 7f4b35af5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9bceeffc0 0x55b9bcef0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:50590/0 (socket says 172.21.15.120:50590) 2024-09-19T09:23:12.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.460+0000 7f4b35af5640 1 -- 172.21.15.120:0/897941931 learned_addr learned my addr 172.21.15.120:0/897941931 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:12.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.460+0000 7f4b362f6640 1 --2- 172.21.15.120:0/3051379203 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4b1c0a32d0 0x7f4b1c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:12.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.460+0000 7f4b362f6640 1 --2- 172.21.15.120:0/897941931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9bcef1530 0x55b9bceefa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:12.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.460+0000 7f4b352f4640 1 -- 172.21.15.120:0/897941931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9bceeffc0 msgr2=0x55b9bcef0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:12.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.460+0000 7f4b352f4640 1 --2- 172.21.15.120:0/897941931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9bceeffc0 0x55b9bcef0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.460+0000 7f4b352f4640 1 -- 172.21.15.120:0/897941931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9bcef1530 msgr2=0x55b9bceefa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:12.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.460+0000 7f4b352f4640 1 --2- 172.21.15.120:0/897941931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9bcef1530 0x55b9bceefa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.460+0000 7f4b352f4640 1 -- 172.21.15.120:0/897941931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9bcdd93a0 con 0x55b9bcefad60 2024-09-19T09:23:12.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.460+0000 7f4b352f4640 1 --2- 172.21.15.120:0/897941931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bcefad60 0x55b9bcefd150 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4b2009bde0 tx=0x7f4b2006ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a82354feaed0e9f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:12.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.461+0000 7f4b26ffd640 1 -- 172.21.15.120:0/3051379203 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f4b200cfac0 con 0x7f4b280050e0 2024-09-19T09:23:12.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.461+0000 7f4b1bfff640 1 -- 172.21.15.120:0/897941931 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2009c7e0 con 0x55b9bcefad60 2024-09-19T09:23:12.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.461+0000 7f4b1bfff640 1 -- 172.21.15.120:0/897941931 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b2009c0e0 con 0x55b9bcefad60 2024-09-19T09:23:12.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.462+0000 7f4b362f6640 1 --2- 172.21.15.120:0/3051379203 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4b1c0a32d0 0x7f4b1c0a5790 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f4b30051500 tx=0x7f4b30068970 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:12.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.462+0000 7f4b3a132740 1 -- 172.21.15.120:0/897941931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bcefad60 msgr2=0x55b9bcefd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:12.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.462+0000 7f4b3a132740 1 --2- 172.21.15.120:0/897941931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bcefad60 0x55b9bcefd150 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4b2009bde0 tx=0x7f4b2006ef60 comp rx=0 tx=0).stop 2024-09-19T09:23:12.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.462+0000 7f4b3a132740 1 -- 172.21.15.120:0/897941931 shutdown_connections 2024-09-19T09:23:12.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.462+0000 7f4b3a132740 1 --2- 172.21.15.120:0/897941931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bcefad60 0x55b9bcefd150 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.462+0000 7f4b3a132740 1 --2- 172.21.15.120:0/897941931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9bceeffc0 0x55b9bcef0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.462+0000 7f4b3a132740 1 --2- 172.21.15.120:0/897941931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9bcef1530 0x55b9bceefa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.462+0000 7f4b3a132740 1 -- 172.21.15.120:0/897941931 >> 172.21.15.120:0/897941931 conn(0x55b9bceb0310 msgr2=0x55b9bceec650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:12.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.462+0000 7f4b3a132740 1 -- 172.21.15.120:0/897941931 shutdown_connections 2024-09-19T09:23:12.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.462+0000 7f4b3a132740 1 -- 172.21.15.120:0/897941931 wait complete. 2024-09-19T09:23:12.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.463+0000 7f4b3a132740 1 -- start start 2024-09-19T09:23:12.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.463+0000 7f4b3a132740 1 Processor -- start 2024-09-19T09:23:12.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.464+0000 7f4b3a132740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9bceeffc0 0x55b9bcefcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:12.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.464+0000 7f4b3a132740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bcef1530 0x55b9bcefa190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:12.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.464+0000 7f4b3a132740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9bcefa6d0 0x55b9bcefab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:12.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.464+0000 7f4b3a132740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b9bcefda60 con 0x55b9bcefa6d0 2024-09-19T09:23:12.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.464+0000 7f4b3a132740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b9bce08060 con 0x55b9bcef1530 2024-09-19T09:23:12.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.464+0000 7f4b3a132740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b9bcefdbe0 con 0x55b9bceeffc0 2024-09-19T09:23:12.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.464+0000 7f4b3a132740 10 client.0 _ll_register_callbacks cb 0x55b9bcb5b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:12.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.464+0000 7f4b35af5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9bceeffc0 0x55b9bcefcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:12.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.464+0000 7f4b35af5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9bceeffc0 0x55b9bcefcee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:50600/0 (socket says 172.21.15.120:50600) 2024-09-19T09:23:12.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.464+0000 7f4b35af5640 1 -- 172.21.15.120:0/564393300 learned_addr learned my addr 172.21.15.120:0/564393300 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:12.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.465+0000 7f4b362f6640 1 --2- 172.21.15.120:0/564393300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9bcefa6d0 0x55b9bcefab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:12.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.465+0000 7f4b352f4640 1 --2- 172.21.15.120:0/564393300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bcef1530 0x55b9bcefa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:12.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.465+0000 7f4b35af5640 1 -- 172.21.15.120:0/564393300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bcef1530 msgr2=0x55b9bcefa190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:12.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.465+0000 7f4b35af5640 1 --2- 172.21.15.120:0/564393300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bcef1530 0x55b9bcefa190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.465+0000 7f4b35af5640 1 -- 172.21.15.120:0/564393300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9bcefa6d0 msgr2=0x55b9bcefab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:12.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.465+0000 7f4b35af5640 1 --2- 172.21.15.120:0/564393300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9bcefa6d0 0x55b9bcefab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.465+0000 7f4b35af5640 1 -- 172.21.15.120:0/564393300 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9bcbbe3c0 con 0x55b9bceeffc0 2024-09-19T09:23:12.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.465+0000 7f4b352f4640 1 --2- 172.21.15.120:0/564393300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bcef1530 0x55b9bcefa190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:23:12.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.466+0000 7f4b35af5640 1 --2- 172.21.15.120:0/564393300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9bceeffc0 0x55b9bcefcee0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c006010 tx=0x7f4b2c005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99f3af54b32b9708 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:12.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.466+0000 7f4b197fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:23:12.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.467+0000 7f4b197fa640 1 -- 172.21.15.120:0/564393300 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c035040 con 0x55b9bceeffc0 2024-09-19T09:23:12.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.467+0000 7f4b197fa640 1 -- 172.21.15.120:0/564393300 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f4b2c0048c0 con 0x55b9bceeffc0 2024-09-19T09:23:12.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.467+0000 7f4b3a132740 1 -- 172.21.15.120:0/564393300 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b9bce5a860 con 0x55b9bceeffc0 2024-09-19T09:23:12.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.467+0000 7f4b197fa640 1 -- 172.21.15.120:0/564393300 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4b2c005d50 con 0x55b9bceeffc0 2024-09-19T09:23:12.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.467+0000 7f4b3a132740 10 client.30775 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:12.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.467+0000 7f4b3a132740 1 -- 172.21.15.120:0/564393300 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b9bcf09840 con 0x55b9bceeffc0 2024-09-19T09:23:12.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.467+0000 7f4b197fa640 1 -- 172.21.15.120:0/564393300 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b9bce5a860 con 0x55b9bceeffc0 2024-09-19T09:23:12.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.468+0000 7f4b197fa640 1 -- 172.21.15.120:0/564393300 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f4b2c031540 con 0x55b9bceeffc0 2024-09-19T09:23:12.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.468+0000 7f4b3a132740 10 client.30775 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:12.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.468+0000 7f4b3a132740 10 client.30775 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:23:12.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.468+0000 7f4b3a132740 20 client.30775 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:12.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.468+0000 7f4b3a132740 1 --2- 172.21.15.120:0/564393300 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b9bcf0ada0 0x55b9bcf0d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:12.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.468+0000 7f4b3a132740 4 client.30775 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:23:12.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.468+0000 7f4b3a132740 1 -- 172.21.15.120:0/564393300 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9bce54d70 con 0x55b9bcf0ada0 2024-09-19T09:23:12.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.468+0000 7f4b352f4640 1 --2- 172.21.15.120:0/564393300 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b9bcf0ada0 0x55b9bcf0d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:12.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.471+0000 7f4b352f4640 1 --2- 172.21.15.120:0/564393300 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b9bcf0ada0 0x55b9bcf0d240 crc :-1 s=READY pgs=505 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62a69ee5c77b0c46 server_cookie=944c89da08accaad in_seq=0 out_seq=0 2024-09-19T09:23:12.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.471+0000 7f4b197fa640 10 client.30775 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:12.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.475+0000 7f4b197fa640 1 -- 172.21.15.120:0/564393300 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9bce54d70 con 0x55b9bcf0ada0 2024-09-19T09:23:12.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.475+0000 7f4b197fa640 10 client.30775 handle_command_reply: tid=0 2024-09-19T09:23:12.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.475+0000 7f4b3a132740 1 client.30775 shutdown 2024-09-19T09:23:12.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.475+0000 7f4b3a132740 1 -- 172.21.15.120:0/564393300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9bceeffc0 msgr2=0x55b9bcefcee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:12.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.475+0000 7f4b3a132740 1 --2- 172.21.15.120:0/564393300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9bceeffc0 0x55b9bcefcee0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c006010 tx=0x7f4b2c005270 comp rx=0 tx=0).stop 2024-09-19T09:23:12.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.476+0000 7f4b3a132740 1 -- 172.21.15.120:0/564393300 shutdown_connections 2024-09-19T09:23:12.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.476+0000 7f4b3a132740 1 --2- 172.21.15.120:0/564393300 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b9bcf0ada0 0x55b9bcf0d240 crc :-1 s=READY pgs=505 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.476+0000 7f4b3a132740 1 --2- 172.21.15.120:0/564393300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9bcefa6d0 0x55b9bcefab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.476+0000 7f4b3a132740 1 --2- 172.21.15.120:0/564393300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bcef1530 0x55b9bcefa190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.476+0000 7f4b3a132740 1 --2- 172.21.15.120:0/564393300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9bceeffc0 0x55b9bcefcee0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.476+0000 7f4b3a132740 1 -- 172.21.15.120:0/564393300 >> 172.21.15.120:0/564393300 conn(0x55b9bceb0310 msgr2=0x55b9bcef2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:12.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.476+0000 7f4b197fa640 0 client.30775 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:12.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.476+0000 7f4b3a132740 1 -- 172.21.15.120:0/564393300 shutdown_connections 2024-09-19T09:23:12.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.476+0000 7f4b3a132740 1 -- 172.21.15.120:0/564393300 wait complete. 2024-09-19T09:23:12.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.477+0000 7f4b3a132740 20 client.30775 trim_cache size 0 max 16384 2024-09-19T09:23:12.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.494+0000 7f4b3a132740 1 Processor -- start 2024-09-19T09:23:12.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.494+0000 7f4b3a132740 1 -- start start 2024-09-19T09:23:12.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.495+0000 7f4b3a132740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9bceeffc0 0x55b9bcef9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:12.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.495+0000 7f4b3a132740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bcef1530 0x55b9bcf12b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:12.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.495+0000 7f4b3a132740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9bcefa6d0 0x55b9bcf13050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:12.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.495+0000 7f4b3a132740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1400e4f0 con 0x55b9bcefa6d0 2024-09-19T09:23:12.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.495+0000 7f4b3a132740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4b14010030 con 0x55b9bcef1530 2024-09-19T09:23:12.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.495+0000 7f4b3a132740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1400ebc0 con 0x55b9bceeffc0 2024-09-19T09:23:12.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.495+0000 7f4b35af5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9bceeffc0 0x55b9bcef9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:12.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.495+0000 7f4b35af5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9bceeffc0 0x55b9bcef9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:50608/0 (socket says 172.21.15.120:50608) 2024-09-19T09:23:12.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.495+0000 7f4b35af5640 1 -- 172.21.15.120:0/3726238089 learned_addr learned my addr 172.21.15.120:0/3726238089 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:12.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.495+0000 7f4b362f6640 1 --2- 172.21.15.120:0/3726238089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9bcefa6d0 0x55b9bcf13050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:12.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.495+0000 7f4b35af5640 1 -- 172.21.15.120:0/3726238089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bcef1530 msgr2=0x55b9bcf12b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:12.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.495+0000 7f4b35af5640 1 --2- 172.21.15.120:0/3726238089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bcef1530 0x55b9bcf12b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.495+0000 7f4b35af5640 1 -- 172.21.15.120:0/3726238089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9bcefa6d0 msgr2=0x55b9bcf13050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:12.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.495+0000 7f4b35af5640 1 --2- 172.21.15.120:0/3726238089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9bcefa6d0 0x55b9bcf13050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.495+0000 7f4b35af5640 1 -- 172.21.15.120:0/3726238089 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9bcef2b10 con 0x55b9bceeffc0 2024-09-19T09:23:12.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.496+0000 7f4b35af5640 1 --2- 172.21.15.120:0/3726238089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9bceeffc0 0x55b9bcef9170 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c000c00 tx=0x7f4b2c0058b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=357ae70dd9c3ae34 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:12.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.496+0000 7f4b1b7fe640 1 -- 172.21.15.120:0/3726238089 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c054020 con 0x55b9bceeffc0 2024-09-19T09:23:12.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.496+0000 7f4b1b7fe640 1 -- 172.21.15.120:0/3726238089 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b2c040070 con 0x55b9bceeffc0 2024-09-19T09:23:12.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.496+0000 7f4b1b7fe640 1 -- 172.21.15.120:0/3726238089 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c035040 con 0x55b9bceeffc0 2024-09-19T09:23:12.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.497+0000 7f4b3a132740 1 -- 172.21.15.120:0/3726238089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9bceeffc0 msgr2=0x55b9bcef9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:12.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.497+0000 7f4b3a132740 1 --2- 172.21.15.120:0/3726238089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9bceeffc0 0x55b9bcef9170 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c000c00 tx=0x7f4b2c0058b0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.497+0000 7f4b3a132740 1 -- 172.21.15.120:0/3726238089 shutdown_connections 2024-09-19T09:23:12.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.497+0000 7f4b3a132740 1 --2- 172.21.15.120:0/3726238089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9bcefa6d0 0x55b9bcf13050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.497+0000 7f4b3a132740 1 --2- 172.21.15.120:0/3726238089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bcef1530 0x55b9bcf12b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.497+0000 7f4b3a132740 1 --2- 172.21.15.120:0/3726238089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9bceeffc0 0x55b9bcef9170 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.497+0000 7f4b3a132740 1 -- 172.21.15.120:0/3726238089 >> 172.21.15.120:0/3726238089 conn(0x55b9bceb0310 msgr2=0x55b9bcef8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:12.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.498+0000 7f4b3a132740 1 -- 172.21.15.120:0/3726238089 shutdown_connections 2024-09-19T09:23:12.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.498+0000 7f4b3a132740 1 -- 172.21.15.120:0/3726238089 wait complete. 2024-09-19T09:23:12.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.498+0000 7f4b3a132740 1 -- start start 2024-09-19T09:23:12.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.498+0000 7f4b3a132740 1 Processor -- start 2024-09-19T09:23:12.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.499+0000 7f4b3a132740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9bceeffc0 0x55b9bcf02ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:12.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.499+0000 7f4b3a132740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bcef1530 0x55b9bcf033e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:12.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.499+0000 7f4b3a132740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9bcefa6d0 0x55b9bcef6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:12.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.499+0000 7f4b3a132740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1400f960 con 0x55b9bcefa6d0 2024-09-19T09:23:12.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.499+0000 7f4b3a132740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1400f290 con 0x55b9bcef1530 2024-09-19T09:23:12.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.499+0000 7f4b3a132740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1400c4b0 con 0x55b9bceeffc0 2024-09-19T09:23:12.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.499+0000 7f4b3a132740 10 client.0 _ll_register_callbacks cb 0x7f4b30050520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:12.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.499+0000 7f4b352f4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bcef1530 0x55b9bcf033e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:12.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.499+0000 7f4b362f6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9bcefa6d0 0x55b9bcef6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:12.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.499+0000 7f4b35af5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9bceeffc0 0x55b9bcf02ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:12.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.500+0000 7f4b35af5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9bceeffc0 0x55b9bcf02ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:50616/0 (socket says 172.21.15.120:50616) 2024-09-19T09:23:12.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.500+0000 7f4b35af5640 1 -- 172.21.15.120:0/1175605421 learned_addr learned my addr 172.21.15.120:0/1175605421 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:12.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.499+0000 7f4b362f6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9bcefa6d0 0x55b9bcef6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46850/0 (socket says 172.21.15.120:46850) 2024-09-19T09:23:12.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.500+0000 7f4b352f4640 1 -- 172.21.15.120:0/1175605421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9bceeffc0 msgr2=0x55b9bcf02ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:12.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.501+0000 7f4b352f4640 1 --2- 172.21.15.120:0/1175605421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9bceeffc0 0x55b9bcf02ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.501+0000 7f4b352f4640 1 -- 172.21.15.120:0/1175605421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9bcefa6d0 msgr2=0x55b9bcef6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:12.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.501+0000 7f4b352f4640 1 --2- 172.21.15.120:0/1175605421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9bcefa6d0 0x55b9bcef6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.501+0000 7f4b352f4640 1 -- 172.21.15.120:0/1175605421 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9bcef8010 con 0x55b9bcef1530 2024-09-19T09:23:12.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.501+0000 7f4b352f4640 1 --2- 172.21.15.120:0/1175605421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bcef1530 0x55b9bcf033e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4b2000c760 tx=0x7f4b200984a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e48770695daff5e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:12.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.501+0000 7f4b197fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:23:12.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.501+0000 7f4b197fa640 1 -- 172.21.15.120:0/1175605421 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b200b4070 con 0x55b9bcef1530 2024-09-19T09:23:12.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.501+0000 7f4b197fa640 1 -- 172.21.15.120:0/1175605421 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f4b20098750 con 0x55b9bcef1530 2024-09-19T09:23:12.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.501+0000 7f4b3a132740 1 -- 172.21.15.120:0/1175605421 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b2c040070 con 0x55b9bcef1530 2024-09-19T09:23:12.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.501+0000 7f4b197fa640 1 -- 172.21.15.120:0/1175605421 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4b2009c7e0 con 0x55b9bcef1530 2024-09-19T09:23:12.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.501+0000 7f4b3a132740 10 client.30914 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:12.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.502+0000 7f4b3a132740 1 -- 172.21.15.120:0/1175605421 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b9bcf74fa0 con 0x55b9bcef1530 2024-09-19T09:23:12.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.502+0000 7f4b197fa640 1 -- 172.21.15.120:0/1175605421 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b2c040070 con 0x55b9bcef1530 2024-09-19T09:23:12.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.502+0000 7f4b197fa640 1 -- 172.21.15.120:0/1175605421 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f4b200a48c0 con 0x55b9bcef1530 2024-09-19T09:23:12.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.502+0000 7f4b3a132740 10 client.30914 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:12.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.502+0000 7f4b3a132740 10 client.30914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:23:12.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.502+0000 7f4b3a132740 20 client.30914 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:12.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.502+0000 7f4b3a132740 1 --2- 172.21.15.120:0/1175605421 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b9bcf75330 0x55b9bcf77720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:12.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.502+0000 7f4b3a132740 4 client.30914 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:23:12.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.502+0000 7f4b3a132740 1 -- 172.21.15.120:0/1175605421 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b9bcdd1170 con 0x55b9bcf75330 2024-09-19T09:23:12.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.502+0000 7f4b35af5640 1 --2- 172.21.15.120:0/1175605421 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b9bcf75330 0x55b9bcf77720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:12.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.503+0000 7f4b197fa640 10 client.30914 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:12.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.503+0000 7f4b35af5640 1 --2- 172.21.15.120:0/1175605421 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b9bcf75330 0x55b9bcf77720 crc :-1 s=READY pgs=506 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ad7096cabfc923a server_cookie=aee663d273b052f1 in_seq=0 out_seq=0 2024-09-19T09:23:12.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.506+0000 7f4b197fa640 1 -- 172.21.15.120:0/1175605421 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b9bcdd1170 con 0x55b9bcf75330 2024-09-19T09:23:12.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.506+0000 7f4b197fa640 10 client.30914 handle_command_reply: tid=0 2024-09-19T09:23:12.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.506+0000 7f4b3a132740 1 client.30914 shutdown 2024-09-19T09:23:12.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.506+0000 7f4b3a132740 1 -- 172.21.15.120:0/1175605421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bcef1530 msgr2=0x55b9bcf033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:12.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.506+0000 7f4b3a132740 1 --2- 172.21.15.120:0/1175605421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bcef1530 0x55b9bcf033e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4b2000c760 tx=0x7f4b200984a0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.506+0000 7f4b3a132740 1 -- 172.21.15.120:0/1175605421 shutdown_connections 2024-09-19T09:23:12.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.506+0000 7f4b3a132740 1 --2- 172.21.15.120:0/1175605421 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b9bcf75330 0x55b9bcf77720 crc :-1 s=READY pgs=506 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.506+0000 7f4b3a132740 1 --2- 172.21.15.120:0/1175605421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b9bcefa6d0 0x55b9bcef6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.506+0000 7f4b3a132740 1 --2- 172.21.15.120:0/1175605421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b9bcef1530 0x55b9bcf033e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.506+0000 7f4b3a132740 1 --2- 172.21.15.120:0/1175605421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9bceeffc0 0x55b9bcf02ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.506+0000 7f4b197fa640 0 client.30914 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:12.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.506+0000 7f4b3a132740 1 -- 172.21.15.120:0/1175605421 >> 172.21.15.120:0/1175605421 conn(0x55b9bceb0310 msgr2=0x55b9bcef9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:12.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.506+0000 7f4b3a132740 1 -- 172.21.15.120:0/1175605421 shutdown_connections 2024-09-19T09:23:12.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.506+0000 7f4b3a132740 1 -- 172.21.15.120:0/1175605421 wait complete. 2024-09-19T09:23:12.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.506+0000 7f4b3a132740 20 client.30914 trim_cache size 0 max 16384 2024-09-19T09:23:12.509 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:23:12.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.507+0000 7f4b37d59640 1 -- 172.21.15.120:0/3051379203 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4b1c0a32d0 msgr2=0x7f4b1c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:12.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.507+0000 7f4b37d59640 1 --2- 172.21.15.120:0/3051379203 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4b1c0a32d0 0x7f4b1c0a5790 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f4b30051500 tx=0x7f4b30068970 comp rx=0 tx=0).stop 2024-09-19T09:23:12.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.507+0000 7f4b37d59640 1 -- 172.21.15.120:0/3051379203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b280050e0 msgr2=0x7f4b2812e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:12.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.507+0000 7f4b37d59640 1 --2- 172.21.15.120:0/3051379203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b280050e0 0x7f4b2812e540 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4b2000ea60 tx=0x7f4b2000ef20 comp rx=0 tx=0).stop 2024-09-19T09:23:12.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.507+0000 7f4b37d59640 1 -- 172.21.15.120:0/3051379203 shutdown_connections 2024-09-19T09:23:12.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.507+0000 7f4b37d59640 1 --2- 172.21.15.120:0/3051379203 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4b1c0a32d0 0x7f4b1c0a5790 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.507+0000 7f4b37d59640 1 --2- 172.21.15.120:0/3051379203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b2809b600 0x7f4b281328d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.507+0000 7f4b37d59640 1 --2- 172.21.15.120:0/3051379203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b280050e0 0x7f4b2812e540 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.507+0000 7f4b37d59640 1 --2- 172.21.15.120:0/3051379203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b28004740 0x7f4b2812e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:12.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.507+0000 7f4b37d59640 1 -- 172.21.15.120:0/3051379203 >> 172.21.15.120:0/3051379203 conn(0x7f4b2800a1b0 msgr2=0x7f4b28093530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:12.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.507+0000 7f4b37d59640 1 -- 172.21.15.120:0/3051379203 shutdown_connections 2024-09-19T09:23:12.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:12.507+0000 7f4b37d59640 1 -- 172.21.15.120:0/3051379203 wait complete. 2024-09-19T09:23:13.376 INFO:tasks.workunit.client.0.smithi120.stdout: 10 171117 20.21 MB/sec execute 308 sec latency 266.873 ms 2024-09-19T09:23:14.327 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:14 smithi120 ceph-mon[31150]: pgmap v956: 97 pgs: 97 active+clean; 419 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.4 MiB/s wr, 534 op/s 2024-09-19T09:23:14.376 INFO:tasks.workunit.client.0.smithi120.stdout: 10 171887 20.28 MB/sec execute 309 sec latency 35.080 ms 2024-09-19T09:23:14.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:14 smithi196 ceph-mon[35150]: pgmap v956: 97 pgs: 97 active+clean; 419 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.4 MiB/s wr, 534 op/s 2024-09-19T09:23:14.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:14 smithi153 ceph-mon[35071]: pgmap v956: 97 pgs: 97 active+clean; 419 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.4 MiB/s wr, 534 op/s 2024-09-19T09:23:14.455 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:23:14.455 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:23:14.728 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:23:15.376 INFO:tasks.workunit.client.0.smithi120.stdout: 10 172632 20.32 MB/sec execute 310 sec latency 99.498 ms 2024-09-19T09:23:15.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.541+0000 7f1b796ac640 1 -- 172.21.15.120:0/3261575893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b74106a70 msgr2=0x7f1b74106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:15.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.541+0000 7f1b796ac640 1 --2- 172.21.15.120:0/3261575893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b74106a70 0x7f1b74106e50 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f1b64009930 tx=0x7f1b6402a420 comp rx=0 tx=0).stop 2024-09-19T09:23:15.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.542+0000 7f1b796ac640 1 -- 172.21.15.120:0/3261575893 shutdown_connections 2024-09-19T09:23:15.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.542+0000 7f1b796ac640 1 --2- 172.21.15.120:0/3261575893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b74101410 0x7f1b7410d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.542+0000 7f1b796ac640 1 --2- 172.21.15.120:0/3261575893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b74100a70 0x7f1b74100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.542+0000 7f1b796ac640 1 --2- 172.21.15.120:0/3261575893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b74106a70 0x7f1b74106e50 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.542+0000 7f1b796ac640 1 -- 172.21.15.120:0/3261575893 >> 172.21.15.120:0/3261575893 conn(0x7f1b740fc790 msgr2=0x7f1b740febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:15.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.542+0000 7f1b796ac640 1 -- 172.21.15.120:0/3261575893 shutdown_connections 2024-09-19T09:23:15.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.542+0000 7f1b796ac640 1 -- 172.21.15.120:0/3261575893 wait complete. 2024-09-19T09:23:15.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.542+0000 7f1b796ac640 1 Processor -- start 2024-09-19T09:23:15.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.542+0000 7f1b796ac640 1 -- start start 2024-09-19T09:23:15.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.542+0000 7f1b796ac640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b74100a70 0x7f1b74198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:15.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.542+0000 7f1b796ac640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b74101410 0x7f1b74198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:15.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.543+0000 7f1b796ac640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b74106a70 0x7f1b7419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:15.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.543+0000 7f1b796ac640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1b74110e70 con 0x7f1b74106a70 2024-09-19T09:23:15.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.543+0000 7f1b796ac640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1b74110cf0 con 0x7f1b74100a70 2024-09-19T09:23:15.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.543+0000 7f1b796ac640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1b74110ff0 con 0x7f1b74101410 2024-09-19T09:23:15.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.543+0000 7f1b737fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b74106a70 0x7f1b7419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:15.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.543+0000 7f1b737fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b74106a70 0x7f1b7419cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46862/0 (socket says 172.21.15.120:46862) 2024-09-19T09:23:15.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.543+0000 7f1b737fe640 1 -- 172.21.15.120:0/1850273925 learned_addr learned my addr 172.21.15.120:0/1850273925 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:15.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.543+0000 7f1b6a7fc640 1 --2- 172.21.15.120:0/1850273925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b74101410 0x7f1b74198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:15.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.543+0000 7f1b72ffd640 1 --2- 172.21.15.120:0/1850273925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b74100a70 0x7f1b74198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:15.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.543+0000 7f1b737fe640 1 -- 172.21.15.120:0/1850273925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b74101410 msgr2=0x7f1b74198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:15.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.543+0000 7f1b737fe640 1 --2- 172.21.15.120:0/1850273925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b74101410 0x7f1b74198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.543+0000 7f1b737fe640 1 -- 172.21.15.120:0/1850273925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b74100a70 msgr2=0x7f1b74198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:15.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.543+0000 7f1b737fe640 1 --2- 172.21.15.120:0/1850273925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b74100a70 0x7f1b74198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.543+0000 7f1b737fe640 1 -- 172.21.15.120:0/1850273925 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7419d4b0 con 0x7f1b74106a70 2024-09-19T09:23:15.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.543+0000 7f1b737fe640 1 --2- 172.21.15.120:0/1850273925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b74106a70 0x7f1b7419cdd0 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00eaa0 tx=0x7f1b6c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:15.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.543+0000 7f1b70ff9640 1 -- 172.21.15.120:0/1850273925 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c00cdb0 con 0x7f1b74106a70 2024-09-19T09:23:15.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.543+0000 7f1b796ac640 1 -- 172.21.15.120:0/1850273925 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7419d7a0 con 0x7f1b74106a70 2024-09-19T09:23:15.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.543+0000 7f1b70ff9640 1 -- 172.21.15.120:0/1850273925 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b6c005020 con 0x7f1b74106a70 2024-09-19T09:23:15.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.543+0000 7f1b70ff9640 1 -- 172.21.15.120:0/1850273925 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c0116b0 con 0x7f1b74106a70 2024-09-19T09:23:15.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.543+0000 7f1b796ac640 1 -- 172.21.15.120:0/1850273925 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b741a5080 con 0x7f1b74106a70 2024-09-19T09:23:15.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.546+0000 7f1b70ff9640 1 -- 172.21.15.120:0/1850273925 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f1b6c005560 con 0x7f1b74106a70 2024-09-19T09:23:15.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.546+0000 7f1b7ba85740 1 Processor -- start 2024-09-19T09:23:15.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.546+0000 7f1b70ff9640 1 --2- 172.21.15.120:0/1850273925 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1b600a3240 0x7f1b600a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:15.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.546+0000 7f1b7ba85740 1 -- start start 2024-09-19T09:23:15.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.546+0000 7f1b70ff9640 1 -- 172.21.15.120:0/1850273925 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f1b6c0cfb00 con 0x7f1b74106a70 2024-09-19T09:23:15.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.546+0000 7f1b6a7fc640 1 --2- 172.21.15.120:0/1850273925 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1b600a3240 0x7f1b600a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:15.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.547+0000 7f1b7ba85740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556552558b60 0x556552558f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:15.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.547+0000 7f1b7ba85740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556552559480 0x556552563800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:15.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.547+0000 7f1b7ba85740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556552563d40 0x556552566130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:15.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.547+0000 7f1b7ba85740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5565524be280 con 0x556552563d40 2024-09-19T09:23:15.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.547+0000 7f1b7ba85740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556552465b10 con 0x556552558b60 2024-09-19T09:23:15.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.547+0000 7f1b7ba85740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5565524ba240 con 0x556552559480 2024-09-19T09:23:15.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.547+0000 7f1b737fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556552563d40 0x556552566130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:15.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.547+0000 7f1b72ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556552558b60 0x556552558f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:15.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.547+0000 7f1b737fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556552563d40 0x556552566130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46864/0 (socket says 172.21.15.120:46864) 2024-09-19T09:23:15.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.547+0000 7f1b72ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556552558b60 0x556552558f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:41324/0 (socket says 172.21.15.120:41324) 2024-09-19T09:23:15.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.547+0000 7f1b737fe640 1 -- 172.21.15.120:0/1507569971 learned_addr learned my addr 172.21.15.120:0/1507569971 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:15.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.547+0000 7f1b6a7fc640 1 --2- 172.21.15.120:0/1507569971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556552559480 0x556552563800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:15.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.547+0000 7f1b737fe640 1 -- 172.21.15.120:0/1507569971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556552559480 msgr2=0x556552563800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:15.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.547+0000 7f1b737fe640 1 --2- 172.21.15.120:0/1507569971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556552559480 0x556552563800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.547+0000 7f1b737fe640 1 -- 172.21.15.120:0/1507569971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556552558b60 msgr2=0x556552558f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:15.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.547+0000 7f1b737fe640 1 --2- 172.21.15.120:0/1507569971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556552558b60 0x556552558f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.547+0000 7f1b737fe640 1 -- 172.21.15.120:0/1507569971 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565524721f0 con 0x556552563d40 2024-09-19T09:23:15.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.547+0000 7f1b6a7fc640 1 --2- 172.21.15.120:0/1850273925 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1b600a3240 0x7f1b600a5700 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f1b740ffe10 tx=0x7f1b5c011440 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:15.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.547+0000 7f1b737fe640 1 --2- 172.21.15.120:0/1507569971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556552563d40 0x556552566130 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c098580 tx=0x7f1b6c09c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f965184752a96d1f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:15.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.548+0000 7f1b697fa640 1 -- 172.21.15.120:0/1507569971 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c01f030 con 0x556552563d40 2024-09-19T09:23:15.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.548+0000 7f1b697fa640 1 -- 172.21.15.120:0/1507569971 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b6c09a420 con 0x556552563d40 2024-09-19T09:23:15.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.548+0000 7f1b697fa640 1 -- 172.21.15.120:0/1507569971 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c0a0120 con 0x556552563d40 2024-09-19T09:23:15.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.549+0000 7f1b7ba85740 1 -- 172.21.15.120:0/1507569971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556552563d40 msgr2=0x556552566130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:15.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.549+0000 7f1b7ba85740 1 --2- 172.21.15.120:0/1507569971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556552563d40 0x556552566130 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c098580 tx=0x7f1b6c09c960 comp rx=0 tx=0).stop 2024-09-19T09:23:15.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.549+0000 7f1b7ba85740 1 -- 172.21.15.120:0/1507569971 shutdown_connections 2024-09-19T09:23:15.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.549+0000 7f1b7ba85740 1 --2- 172.21.15.120:0/1507569971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556552563d40 0x556552566130 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.549+0000 7f1b7ba85740 1 --2- 172.21.15.120:0/1507569971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556552559480 0x556552563800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.549+0000 7f1b7ba85740 1 --2- 172.21.15.120:0/1507569971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556552558b60 0x556552558f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.549+0000 7f1b7ba85740 1 -- 172.21.15.120:0/1507569971 >> 172.21.15.120:0/1507569971 conn(0x5565525550f0 msgr2=0x5565525554d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:15.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.549+0000 7f1b7ba85740 1 -- 172.21.15.120:0/1507569971 shutdown_connections 2024-09-19T09:23:15.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.549+0000 7f1b7ba85740 1 -- 172.21.15.120:0/1507569971 wait complete. 2024-09-19T09:23:15.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.549+0000 7f1b7ba85740 1 -- start start 2024-09-19T09:23:15.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.549+0000 7f1b7ba85740 1 Processor -- start 2024-09-19T09:23:15.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.550+0000 7f1b7ba85740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556552558b60 0x556552560bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:15.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.550+0000 7f1b7ba85740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556552559480 0x556552561110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:15.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.550+0000 7f1b7ba85740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556552563d40 0x556552561650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:15.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.550+0000 7f1b7ba85740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556552566910 con 0x556552558b60 2024-09-19T09:23:15.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.550+0000 7f1b7ba85740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556552478f10 con 0x556552563d40 2024-09-19T09:23:15.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.550+0000 7f1b7ba85740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556552566a90 con 0x556552559480 2024-09-19T09:23:15.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.550+0000 7f1b7ba85740 10 client.0 _ll_register_callbacks cb 0x556552441a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:15.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.550+0000 7f1b737fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556552563d40 0x556552561650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:15.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.550+0000 7f1b72ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556552558b60 0x556552560bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:15.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.550+0000 7f1b737fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556552563d40 0x556552561650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:41330/0 (socket says 172.21.15.120:41330) 2024-09-19T09:23:15.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.550+0000 7f1b72ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556552558b60 0x556552560bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46878/0 (socket says 172.21.15.120:46878) 2024-09-19T09:23:15.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.550+0000 7f1b737fe640 1 -- 172.21.15.120:0/2486359268 learned_addr learned my addr 172.21.15.120:0/2486359268 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:15.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.550+0000 7f1b6a7fc640 1 --2- 172.21.15.120:0/2486359268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556552559480 0x556552561110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:15.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.550+0000 7f1b72ffd640 1 -- 172.21.15.120:0/2486359268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556552559480 msgr2=0x556552561110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:15.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.550+0000 7f1b72ffd640 1 --2- 172.21.15.120:0/2486359268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556552559480 0x556552561110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.550+0000 7f1b72ffd640 1 -- 172.21.15.120:0/2486359268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556552563d40 msgr2=0x556552561650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:15.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.550+0000 7f1b72ffd640 1 --2- 172.21.15.120:0/2486359268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556552563d40 0x556552561650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.550+0000 7f1b72ffd640 1 -- 172.21.15.120:0/2486359268 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55655222f6a0 con 0x556552558b60 2024-09-19T09:23:15.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.550+0000 7f1b737fe640 1 --2- 172.21.15.120:0/2486359268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556552563d40 0x556552561650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:15.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.550+0000 7f1b72ffd640 1 --2- 172.21.15.120:0/2486359268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556552558b60 0x556552560bd0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f1b64009900 tx=0x7f1b640317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e08539d3131d686e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:15.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.551+0000 7f1b5a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:23:15.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.551+0000 7f1b5a7fc640 1 -- 172.21.15.120:0/2486359268 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64035050 con 0x556552558b60 2024-09-19T09:23:15.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.551+0000 7f1b5a7fc640 1 -- 172.21.15.120:0/2486359268 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f1b64004240 con 0x556552558b60 2024-09-19T09:23:15.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.551+0000 7f1b7ba85740 1 -- 172.21.15.120:0/2486359268 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b6c09a420 con 0x556552558b60 2024-09-19T09:23:15.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.551+0000 7f1b5a7fc640 1 -- 172.21.15.120:0/2486359268 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1b64004440 con 0x556552558b60 2024-09-19T09:23:15.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.551+0000 7f1b7ba85740 10 client.22173 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:15.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.551+0000 7f1b7ba85740 1 -- 172.21.15.120:0/2486359268 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5565525724c0 con 0x556552558b60 2024-09-19T09:23:15.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.551+0000 7f1b5a7fc640 1 -- 172.21.15.120:0/2486359268 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b6c09a420 con 0x556552558b60 2024-09-19T09:23:15.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.551+0000 7f1b5a7fc640 1 -- 172.21.15.120:0/2486359268 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f1b6403bc50 con 0x556552558b60 2024-09-19T09:23:15.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.551+0000 7f1b7ba85740 10 client.22173 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:15.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.551+0000 7f1b7ba85740 10 client.22173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:23:15.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.551+0000 7f1b7ba85740 20 client.22173 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:15.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.552+0000 7f1b7ba85740 1 --2- 172.21.15.120:0/2486359268 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556552573df0 0x556552576250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:15.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.552+0000 7f1b7ba85740 4 client.22173 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:23:15.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.552+0000 7f1b7ba85740 1 -- 172.21.15.120:0/2486359268 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565524cb7f0 con 0x556552573df0 2024-09-19T09:23:15.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.552+0000 7f1b6a7fc640 1 --2- 172.21.15.120:0/2486359268 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556552573df0 0x556552576250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:15.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.552+0000 7f1b5a7fc640 10 client.22173 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:15.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.552+0000 7f1b6a7fc640 1 --2- 172.21.15.120:0/2486359268 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556552573df0 0x556552576250 crc :-1 s=READY pgs=507 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb854e1b6b847a95 server_cookie=5edc03428145609f in_seq=0 out_seq=0 2024-09-19T09:23:15.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.556+0000 7f1b5a7fc640 1 -- 172.21.15.120:0/2486359268 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5565524cb7f0 con 0x556552573df0 2024-09-19T09:23:15.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.556+0000 7f1b5a7fc640 10 client.22173 handle_command_reply: tid=0 2024-09-19T09:23:15.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.556+0000 7f1b7ba85740 1 client.22173 shutdown 2024-09-19T09:23:15.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.556+0000 7f1b7ba85740 1 -- 172.21.15.120:0/2486359268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556552558b60 msgr2=0x556552560bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:15.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.556+0000 7f1b7ba85740 1 --2- 172.21.15.120:0/2486359268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556552558b60 0x556552560bd0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f1b64009900 tx=0x7f1b640317e0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.556+0000 7f1b7ba85740 1 -- 172.21.15.120:0/2486359268 shutdown_connections 2024-09-19T09:23:15.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.556+0000 7f1b7ba85740 1 --2- 172.21.15.120:0/2486359268 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556552573df0 0x556552576250 crc :-1 s=READY pgs=507 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.556+0000 7f1b7ba85740 1 --2- 172.21.15.120:0/2486359268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556552563d40 0x556552561650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.556+0000 7f1b5a7fc640 0 client.22173 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:15.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.556+0000 7f1b7ba85740 1 --2- 172.21.15.120:0/2486359268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556552559480 0x556552561110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.556+0000 7f1b7ba85740 1 --2- 172.21.15.120:0/2486359268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556552558b60 0x556552560bd0 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.556+0000 7f1b7ba85740 1 -- 172.21.15.120:0/2486359268 >> 172.21.15.120:0/2486359268 conn(0x5565525550f0 msgr2=0x556552554270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:15.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.556+0000 7f1b7ba85740 1 -- 172.21.15.120:0/2486359268 shutdown_connections 2024-09-19T09:23:15.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.556+0000 7f1b7ba85740 1 -- 172.21.15.120:0/2486359268 wait complete. 2024-09-19T09:23:15.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.556+0000 7f1b7ba85740 20 client.22173 trim_cache size 0 max 16384 2024-09-19T09:23:15.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.577+0000 7f1b7ba85740 1 Processor -- start 2024-09-19T09:23:15.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.577+0000 7f1b7ba85740 1 -- start start 2024-09-19T09:23:15.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.577+0000 7f1b7ba85740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556552558b60 0x556552562a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:15.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.577+0000 7f1b7ba85740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556552559480 0x55655257bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:15.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.577+0000 7f1b7ba85740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556552563d40 0x55655257c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:15.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.578+0000 7f1b7ba85740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1b741ab440 con 0x556552563d40 2024-09-19T09:23:15.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.578+0000 7f1b7ba85740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1b741acf80 con 0x556552558b60 2024-09-19T09:23:15.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.578+0000 7f1b7ba85740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1b741abb10 con 0x556552559480 2024-09-19T09:23:15.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.578+0000 7f1b72ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556552558b60 0x556552562a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:15.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.578+0000 7f1b72ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556552558b60 0x556552562a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:41346/0 (socket says 172.21.15.120:41346) 2024-09-19T09:23:15.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.578+0000 7f1b72ffd640 1 -- 172.21.15.120:0/1599670003 learned_addr learned my addr 172.21.15.120:0/1599670003 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:15.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.578+0000 7f1b737fe640 1 --2- 172.21.15.120:0/1599670003 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556552563d40 0x55655257c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:15.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.578+0000 7f1b6a7fc640 1 --2- 172.21.15.120:0/1599670003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556552559480 0x55655257bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:15.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.578+0000 7f1b72ffd640 1 -- 172.21.15.120:0/1599670003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556552559480 msgr2=0x55655257bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:15.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.578+0000 7f1b72ffd640 1 --2- 172.21.15.120:0/1599670003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556552559480 0x55655257bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.578+0000 7f1b72ffd640 1 -- 172.21.15.120:0/1599670003 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556552563d40 msgr2=0x55655257c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:15.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.578+0000 7f1b72ffd640 1 --2- 172.21.15.120:0/1599670003 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556552563d40 0x55655257c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.578+0000 7f1b72ffd640 1 -- 172.21.15.120:0/1599670003 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565525768d0 con 0x556552558b60 2024-09-19T09:23:15.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.578+0000 7f1b737fe640 1 --2- 172.21.15.120:0/1599670003 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556552563d40 0x55655257c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:15.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.578+0000 7f1b72ffd640 1 --2- 172.21.15.120:0/1599670003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556552558b60 0x556552562a30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1b64009930 tx=0x7f1b64031b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1593b8cb5443e51c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:15.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.578+0000 7f1b68ff9640 1 -- 172.21.15.120:0/1599670003 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64035070 con 0x556552558b60 2024-09-19T09:23:15.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.579+0000 7f1b68ff9640 1 -- 172.21.15.120:0/1599670003 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b64004b00 con 0x556552558b60 2024-09-19T09:23:15.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.579+0000 7f1b68ff9640 1 -- 172.21.15.120:0/1599670003 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64005360 con 0x556552558b60 2024-09-19T09:23:15.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.579+0000 7f1b7ba85740 1 -- 172.21.15.120:0/1599670003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556552558b60 msgr2=0x556552562a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:15.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.579+0000 7f1b7ba85740 1 --2- 172.21.15.120:0/1599670003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556552558b60 0x556552562a30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1b64009930 tx=0x7f1b64031b60 comp rx=0 tx=0).stop 2024-09-19T09:23:15.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.579+0000 7f1b7ba85740 1 -- 172.21.15.120:0/1599670003 shutdown_connections 2024-09-19T09:23:15.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.579+0000 7f1b7ba85740 1 --2- 172.21.15.120:0/1599670003 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556552563d40 0x55655257c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.579+0000 7f1b7ba85740 1 --2- 172.21.15.120:0/1599670003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556552559480 0x55655257bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.579+0000 7f1b7ba85740 1 --2- 172.21.15.120:0/1599670003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556552558b60 0x556552562a30 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.579+0000 7f1b7ba85740 1 -- 172.21.15.120:0/1599670003 >> 172.21.15.120:0/1599670003 conn(0x5565525550f0 msgr2=0x556552562320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:15.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.579+0000 7f1b7ba85740 1 -- 172.21.15.120:0/1599670003 shutdown_connections 2024-09-19T09:23:15.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.579+0000 7f1b7ba85740 1 -- 172.21.15.120:0/1599670003 wait complete. 2024-09-19T09:23:15.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.580+0000 7f1b7ba85740 1 -- start start 2024-09-19T09:23:15.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.580+0000 7f1b7ba85740 1 Processor -- start 2024-09-19T09:23:15.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.580+0000 7f1b7ba85740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556552558b60 0x55655256c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:15.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.580+0000 7f1b7ba85740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556552559480 0x55655255dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:15.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.580+0000 7f1b7ba85740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556552563d40 0x55655255e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:15.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.580+0000 7f1b7ba85740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1b741ac8b0 con 0x556552563d40 2024-09-19T09:23:15.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.580+0000 7f1b7ba85740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1b741ac1e0 con 0x556552559480 2024-09-19T09:23:15.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.580+0000 7f1b7ba85740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1b741a9420 con 0x556552558b60 2024-09-19T09:23:15.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.580+0000 7f1b7ba85740 10 client.0 _ll_register_callbacks cb 0x7f1b64009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:15.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.580+0000 7f1b737fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556552563d40 0x55655255e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:15.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.580+0000 7f1b737fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556552563d40 0x55655255e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46892/0 (socket says 172.21.15.120:46892) 2024-09-19T09:23:15.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.580+0000 7f1b737fe640 1 -- 172.21.15.120:0/1049932326 learned_addr learned my addr 172.21.15.120:0/1049932326 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:15.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.580+0000 7f1b6a7fc640 1 --2- 172.21.15.120:0/1049932326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556552559480 0x55655255dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:15.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.581+0000 7f1b72ffd640 1 --2- 172.21.15.120:0/1049932326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556552558b60 0x55655256c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:15.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.581+0000 7f1b737fe640 1 -- 172.21.15.120:0/1049932326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556552558b60 msgr2=0x55655256c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:15.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.581+0000 7f1b737fe640 1 --2- 172.21.15.120:0/1049932326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556552558b60 0x55655256c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.581+0000 7f1b737fe640 1 -- 172.21.15.120:0/1049932326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556552559480 msgr2=0x55655255dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:15.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.581+0000 7f1b737fe640 1 --2- 172.21.15.120:0/1049932326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556552559480 0x55655255dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.581+0000 7f1b737fe640 1 -- 172.21.15.120:0/1049932326 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556552554f20 con 0x556552563d40 2024-09-19T09:23:15.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.581+0000 7f1b737fe640 1 --2- 172.21.15.120:0/1049932326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556552563d40 0x55655255e250 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c09c0b0 tx=0x7f1b6c09c2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61194d9304045156 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:15.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.581+0000 7f1b5a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:23:15.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.581+0000 7f1b5a7fc640 1 -- 172.21.15.120:0/1049932326 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c01f040 con 0x556552563d40 2024-09-19T09:23:15.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.581+0000 7f1b7ba85740 1 -- 172.21.15.120:0/1049932326 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b64004b00 con 0x556552563d40 2024-09-19T09:23:15.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.581+0000 7f1b5a7fc640 1 -- 172.21.15.120:0/1049932326 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f1b6c0bc040 con 0x556552563d40 2024-09-19T09:23:15.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.581+0000 7f1b5a7fc640 1 -- 172.21.15.120:0/1049932326 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1b6c09b570 con 0x556552563d40 2024-09-19T09:23:15.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.581+0000 7f1b7ba85740 10 client.22179 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:15.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.581+0000 7f1b7ba85740 1 -- 172.21.15.120:0/1049932326 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5565525deb60 con 0x556552563d40 2024-09-19T09:23:15.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.582+0000 7f1b5a7fc640 1 -- 172.21.15.120:0/1049932326 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b64004b00 con 0x556552563d40 2024-09-19T09:23:15.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.582+0000 7f1b5a7fc640 1 -- 172.21.15.120:0/1049932326 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f1b6c00f040 con 0x556552563d40 2024-09-19T09:23:15.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.582+0000 7f1b7ba85740 10 client.22179 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:15.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.582+0000 7f1b7ba85740 10 client.22179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:23:15.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.582+0000 7f1b7ba85740 20 client.22179 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:15.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.582+0000 7f1b7ba85740 1 --2- 172.21.15.120:0/1049932326 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5565525dee90 0x5565525e1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:15.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.582+0000 7f1b7ba85740 4 client.22179 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:23:15.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.582+0000 7f1b7ba85740 1 -- 172.21.15.120:0/1049932326 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5565524c5d00 con 0x5565525dee90 2024-09-19T09:23:15.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.582+0000 7f1b72ffd640 1 --2- 172.21.15.120:0/1049932326 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5565525dee90 0x5565525e1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:15.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.583+0000 7f1b72ffd640 1 --2- 172.21.15.120:0/1049932326 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5565525dee90 0x5565525e1280 crc :-1 s=READY pgs=508 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0478f93d32ba2ea server_cookie=b6b5da385f3ec670 in_seq=0 out_seq=0 2024-09-19T09:23:15.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.583+0000 7f1b5a7fc640 10 client.22179 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:15.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.591+0000 7f1b5a7fc640 1 -- 172.21.15.120:0/1049932326 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5565524c5d00 con 0x5565525dee90 2024-09-19T09:23:15.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.591+0000 7f1b5a7fc640 10 client.22179 handle_command_reply: tid=0 2024-09-19T09:23:15.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.591+0000 7f1b7ba85740 1 client.22179 shutdown 2024-09-19T09:23:15.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.591+0000 7f1b7ba85740 1 -- 172.21.15.120:0/1049932326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556552563d40 msgr2=0x55655255e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:15.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.591+0000 7f1b7ba85740 1 --2- 172.21.15.120:0/1049932326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556552563d40 0x55655255e250 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c09c0b0 tx=0x7f1b6c09c2a0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.591+0000 7f1b7ba85740 1 -- 172.21.15.120:0/1049932326 shutdown_connections 2024-09-19T09:23:15.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.591+0000 7f1b7ba85740 1 --2- 172.21.15.120:0/1049932326 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5565525dee90 0x5565525e1280 crc :-1 s=READY pgs=508 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.591+0000 7f1b7ba85740 1 --2- 172.21.15.120:0/1049932326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556552563d40 0x55655255e250 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.591+0000 7f1b7ba85740 1 --2- 172.21.15.120:0/1049932326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556552559480 0x55655255dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.591+0000 7f1b7ba85740 1 --2- 172.21.15.120:0/1049932326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556552558b60 0x55655256c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.591+0000 7f1b7ba85740 1 -- 172.21.15.120:0/1049932326 >> 172.21.15.120:0/1049932326 conn(0x5565525550f0 msgr2=0x556552554270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:15.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.591+0000 7f1b5a7fc640 0 client.22179 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:15.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.592+0000 7f1b7ba85740 1 -- 172.21.15.120:0/1049932326 shutdown_connections 2024-09-19T09:23:15.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.592+0000 7f1b7ba85740 1 -- 172.21.15.120:0/1049932326 wait complete. 2024-09-19T09:23:15.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.592+0000 7f1b7ba85740 20 client.22179 trim_cache size 0 max 16384 2024-09-19T09:23:15.594 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:23:15.594 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:23:15.594 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "d9f36069-77c6-4eaa-8716-9bafd13e95a1", 2024-09-19T09:23:15.594 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:23:15.594 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:23:15.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.592+0000 7f1b796ac640 1 -- 172.21.15.120:0/1850273925 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1b600a3240 msgr2=0x7f1b600a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:15.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.592+0000 7f1b796ac640 1 --2- 172.21.15.120:0/1850273925 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1b600a3240 0x7f1b600a5700 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f1b740ffe10 tx=0x7f1b5c011440 comp rx=0 tx=0).stop 2024-09-19T09:23:15.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.592+0000 7f1b796ac640 1 -- 172.21.15.120:0/1850273925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b74106a70 msgr2=0x7f1b7419cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:15.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.592+0000 7f1b796ac640 1 --2- 172.21.15.120:0/1850273925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b74106a70 0x7f1b7419cdd0 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00eaa0 tx=0x7f1b6c00ef60 comp rx=0 tx=0).stop 2024-09-19T09:23:15.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.592+0000 7f1b796ac640 1 -- 172.21.15.120:0/1850273925 shutdown_connections 2024-09-19T09:23:15.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.592+0000 7f1b796ac640 1 --2- 172.21.15.120:0/1850273925 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1b600a3240 0x7f1b600a5700 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.592+0000 7f1b796ac640 1 --2- 172.21.15.120:0/1850273925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b74106a70 0x7f1b7419cdd0 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.592+0000 7f1b796ac640 1 --2- 172.21.15.120:0/1850273925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b74101410 0x7f1b74198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.592+0000 7f1b796ac640 1 --2- 172.21.15.120:0/1850273925 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b74100a70 0x7f1b74198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:15.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.592+0000 7f1b796ac640 1 -- 172.21.15.120:0/1850273925 >> 172.21.15.120:0/1850273925 conn(0x7f1b740fc790 msgr2=0x7f1b7406a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:15.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.593+0000 7f1b796ac640 1 -- 172.21.15.120:0/1850273925 shutdown_connections 2024-09-19T09:23:15.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:15.593+0000 7f1b796ac640 1 -- 172.21.15.120:0/1850273925 wait complete. 2024-09-19T09:23:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.258+0000 7fa9f1265640 1 -- 172.21.15.120:0/1166113996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9ec100a40 msgr2=0x7fa9ec100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.258+0000 7fa9f1265640 1 --2- 172.21.15.120:0/1166113996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9ec100a40 0x7fa9ec100e40 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc009900 tx=0x7fa9dc02a410 comp rx=0 tx=0).stop 2024-09-19T09:23:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.258+0000 7fa9f1265640 1 -- 172.21.15.120:0/1166113996 shutdown_connections 2024-09-19T09:23:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.258+0000 7fa9f1265640 1 --2- 172.21.15.120:0/1166113996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9ec1025f0 0x7fa9ec108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.258+0000 7fa9f1265640 1 --2- 172.21.15.120:0/1166113996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9ec101c30 0x7fa9ec1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.258+0000 7fa9f1265640 1 --2- 172.21.15.120:0/1166113996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9ec100a40 0x7fa9ec100e40 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.258+0000 7fa9f1265640 1 -- 172.21.15.120:0/1166113996 >> 172.21.15.120:0/1166113996 conn(0x7fa9ec0fc1e0 msgr2=0x7fa9ec0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.258+0000 7fa9f1265640 1 -- 172.21.15.120:0/1166113996 shutdown_connections 2024-09-19T09:23:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.258+0000 7fa9f1265640 1 -- 172.21.15.120:0/1166113996 wait complete. 2024-09-19T09:23:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.259+0000 7fa9f1265640 1 Processor -- start 2024-09-19T09:23:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.259+0000 7fa9f1265640 1 -- start start 2024-09-19T09:23:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.259+0000 7fa9f1265640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9ec100a40 0x7fa9ec196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.259+0000 7fa9f1265640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9ec101c30 0x7fa9ec197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.259+0000 7fa9f1265640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9ec1025f0 0x7fa9ec19e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.259+0000 7fa9f1265640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec10c6c0 con 0x7fa9ec101c30 2024-09-19T09:23:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.259+0000 7fa9f1265640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec10c540 con 0x7fa9ec100a40 2024-09-19T09:23:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.259+0000 7fa9f1265640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec10c840 con 0x7fa9ec1025f0 2024-09-19T09:23:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.259+0000 7fa9e3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9ec101c30 0x7fa9ec197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.259+0000 7fa9e3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9ec101c30 0x7fa9ec197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46906/0 (socket says 172.21.15.120:46906) 2024-09-19T09:23:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.259+0000 7fa9e3fff640 1 -- 172.21.15.120:0/1853857759 learned_addr learned my addr 172.21.15.120:0/1853857759 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.259+0000 7fa9eaffd640 1 --2- 172.21.15.120:0/1853857759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9ec100a40 0x7fa9ec196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.259+0000 7fa9eb7fe640 1 --2- 172.21.15.120:0/1853857759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9ec1025f0 0x7fa9ec19e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.259+0000 7fa9e3fff640 1 -- 172.21.15.120:0/1853857759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9ec1025f0 msgr2=0x7fa9ec19e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.259+0000 7fa9e3fff640 1 --2- 172.21.15.120:0/1853857759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9ec1025f0 0x7fa9ec19e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.259+0000 7fa9e3fff640 1 -- 172.21.15.120:0/1853857759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9ec100a40 msgr2=0x7fa9ec196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.259+0000 7fa9e3fff640 1 --2- 172.21.15.120:0/1853857759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9ec100a40 0x7fa9ec196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.259+0000 7fa9e3fff640 1 -- 172.21.15.120:0/1853857759 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ec19e910 con 0x7fa9ec101c30 2024-09-19T09:23:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.259+0000 7fa9eb7fe640 1 --2- 172.21.15.120:0/1853857759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9ec1025f0 0x7fa9ec19e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:23:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.260+0000 7fa9e3fff640 1 --2- 172.21.15.120:0/1853857759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9ec101c30 0x7fa9ec197240 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7fa9d400e9f0 tx=0x7fa9d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.260+0000 7fa9e8ff9640 1 -- 172.21.15.120:0/1853857759 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d401b070 con 0x7fa9ec101c30 2024-09-19T09:23:16.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.260+0000 7fa9e8ff9640 1 -- 172.21.15.120:0/1853857759 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d400db90 con 0x7fa9ec101c30 2024-09-19T09:23:16.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.260+0000 7fa9f1265640 1 -- 172.21.15.120:0/1853857759 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9ec069900 con 0x7fa9ec101c30 2024-09-19T09:23:16.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.260+0000 7fa9e8ff9640 1 -- 172.21.15.120:0/1853857759 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4015610 con 0x7fa9ec101c30 2024-09-19T09:23:16.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.260+0000 7fa9f1265640 1 -- 172.21.15.120:0/1853857759 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9ec069e20 con 0x7fa9ec101c30 2024-09-19T09:23:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.262+0000 7fa9e8ff9640 1 -- 172.21.15.120:0/1853857759 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa9d401e070 con 0x7fa9ec101c30 2024-09-19T09:23:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.263+0000 7fa9f4006740 1 Processor -- start 2024-09-19T09:23:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.263+0000 7fa9f4006740 1 -- start start 2024-09-19T09:23:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.263+0000 7fa9f4006740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd0a8d69a0 0x55dd0a8d6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.263+0000 7fa9f4006740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd0a8d72e0 0x55dd0a8dbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.263+0000 7fa9f4006740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd0a8dc370 0x55dd0a8de830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.263+0000 7fa9f4006740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dd0a83a2f0 con 0x55dd0a8dc370 2024-09-19T09:23:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.263+0000 7fa9f4006740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55dd0a7e2270 con 0x55dd0a8d69a0 2024-09-19T09:23:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.263+0000 7fa9f4006740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dd0a8362b0 con 0x55dd0a8d72e0 2024-09-19T09:23:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.263+0000 7fa9eb7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd0a8d69a0 0x55dd0a8d6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.263+0000 7fa9eaffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd0a8d72e0 0x55dd0a8dbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.263+0000 7fa9eb7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd0a8d69a0 0x55dd0a8d6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:41382/0 (socket says 172.21.15.120:41382) 2024-09-19T09:23:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.263+0000 7fa9eb7fe640 1 -- 172.21.15.120:0/437562683 learned_addr learned my addr 172.21.15.120:0/437562683 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.263+0000 7fa9e3fff640 1 --2- 172.21.15.120:0/437562683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd0a8dc370 0x55dd0a8de830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.264+0000 7fa9eb7fe640 1 -- 172.21.15.120:0/437562683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd0a8d72e0 msgr2=0x55dd0a8dbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.264+0000 7fa9eb7fe640 1 --2- 172.21.15.120:0/437562683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd0a8d72e0 0x55dd0a8dbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.264+0000 7fa9eb7fe640 1 -- 172.21.15.120:0/437562683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd0a8dc370 msgr2=0x55dd0a8de830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.264+0000 7fa9eb7fe640 1 --2- 172.21.15.120:0/437562683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd0a8dc370 0x55dd0a8de830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.264+0000 7fa9eb7fe640 1 -- 172.21.15.120:0/437562683 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd0a7e1340 con 0x55dd0a8d69a0 2024-09-19T09:23:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.264+0000 7fa9eb7fe640 1 --2- 172.21.15.120:0/437562683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd0a8d69a0 0x55dd0a8d6da0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4009fd0 tx=0x7fa9e40108e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bd5cc0233731e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.264+0000 7fa9e8ff9640 1 --2- 172.21.15.120:0/1853857759 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9d80a32f0 0x7fa9d80a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.264+0000 7fa9e17fa640 1 -- 172.21.15.120:0/437562683 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4011760 con 0x55dd0a8d69a0 2024-09-19T09:23:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.264+0000 7fa9eaffd640 1 --2- 172.21.15.120:0/1853857759 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9d80a32f0 0x7fa9d80a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.264+0000 7fa9e17fa640 1 -- 172.21.15.120:0/437562683 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e4011de0 con 0x55dd0a8d69a0 2024-09-19T09:23:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.264+0000 7fa9e17fa640 1 -- 172.21.15.120:0/437562683 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e400e3c0 con 0x55dd0a8d69a0 2024-09-19T09:23:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.264+0000 7fa9e8ff9640 1 -- 172.21.15.120:0/1853857759 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa9d40cf1b0 con 0x7fa9ec101c30 2024-09-19T09:23:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.265+0000 7fa9f4006740 1 -- 172.21.15.120:0/437562683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd0a8d69a0 msgr2=0x55dd0a8d6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.265+0000 7fa9eaffd640 1 --2- 172.21.15.120:0/1853857759 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9d80a32f0 0x7fa9d80a57b0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc005ab0 tx=0x7fa9dc03c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.265+0000 7fa9f4006740 1 --2- 172.21.15.120:0/437562683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd0a8d69a0 0x55dd0a8d6da0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4009fd0 tx=0x7fa9e40108e0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.265+0000 7fa9f4006740 1 -- 172.21.15.120:0/437562683 shutdown_connections 2024-09-19T09:23:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.265+0000 7fa9f4006740 1 --2- 172.21.15.120:0/437562683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd0a8dc370 0x55dd0a8de830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.265+0000 7fa9f4006740 1 --2- 172.21.15.120:0/437562683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd0a8d72e0 0x55dd0a8dbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.265+0000 7fa9f4006740 1 --2- 172.21.15.120:0/437562683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd0a8d69a0 0x55dd0a8d6da0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.265+0000 7fa9f4006740 1 -- 172.21.15.120:0/437562683 >> 172.21.15.120:0/437562683 conn(0x55dd0a8d2010 msgr2=0x55dd0a8d2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.265+0000 7fa9f4006740 1 -- 172.21.15.120:0/437562683 shutdown_connections 2024-09-19T09:23:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.265+0000 7fa9f4006740 1 -- 172.21.15.120:0/437562683 wait complete. 2024-09-19T09:23:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.265+0000 7fa9f4006740 1 -- start start 2024-09-19T09:23:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.266+0000 7fa9f4006740 1 Processor -- start 2024-09-19T09:23:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.266+0000 7fa9f4006740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd0a8d69a0 0x55dd0a8e0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.266+0000 7fa9f4006740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd0a8d72e0 0x55dd0a8e11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.266+0000 7fa9f4006740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd0a8dc370 0x55dd0a8e81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.266+0000 7fa9f4006740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dd0a8df340 con 0x55dd0a8dc370 2024-09-19T09:23:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.266+0000 7fa9f4006740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55dd0a7c3290 con 0x55dd0a8d72e0 2024-09-19T09:23:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.266+0000 7fa9f4006740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dd0a8df4c0 con 0x55dd0a8d69a0 2024-09-19T09:23:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.266+0000 7fa9f4006740 10 client.0 _ll_register_callbacks cb 0x55dd0a7f4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.266+0000 7fa9e3fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd0a8d72e0 0x55dd0a8e11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.266+0000 7fa9eb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd0a8dc370 0x55dd0a8e81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.266+0000 7fa9e3fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd0a8d72e0 0x55dd0a8e11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:41396/0 (socket says 172.21.15.120:41396) 2024-09-19T09:23:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.266+0000 7fa9e3fff640 1 -- 172.21.15.120:0/2723632662 learned_addr learned my addr 172.21.15.120:0/2723632662 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.266+0000 7fa9eb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd0a8dc370 0x55dd0a8e81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46926/0 (socket says 172.21.15.120:46926) 2024-09-19T09:23:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.266+0000 7fa9eaffd640 1 --2- 172.21.15.120:0/2723632662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd0a8d69a0 0x55dd0a8e0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.266+0000 7fa9eb7fe640 1 -- 172.21.15.120:0/2723632662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd0a8d69a0 msgr2=0x55dd0a8e0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.266+0000 7fa9eb7fe640 1 --2- 172.21.15.120:0/2723632662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd0a8d69a0 0x55dd0a8e0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.266+0000 7fa9eb7fe640 1 -- 172.21.15.120:0/2723632662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd0a8d72e0 msgr2=0x55dd0a8e11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.266+0000 7fa9eb7fe640 1 --2- 172.21.15.120:0/2723632662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd0a8d72e0 0x55dd0a8e11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.266+0000 7fa9eb7fe640 1 -- 172.21.15.120:0/2723632662 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd0a54d210 con 0x55dd0a8dc370 2024-09-19T09:23:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.266+0000 7fa9e3fff640 1 --2- 172.21.15.120:0/2723632662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd0a8d72e0 0x55dd0a8e11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.267+0000 7fa9eb7fe640 1 --2- 172.21.15.120:0/2723632662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd0a8dc370 0x55dd0a8e81e0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4005bf0 tx=0x7fa9e401ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b54a28cb5b253e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.267+0000 7fa9d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:23:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.267+0000 7fa9d27fc640 1 -- 172.21.15.120:0/2723632662 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e403c060 con 0x55dd0a8dc370 2024-09-19T09:23:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.267+0000 7fa9d27fc640 1 -- 172.21.15.120:0/2723632662 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa9e4011560 con 0x55dd0a8dc370 2024-09-19T09:23:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.267+0000 7fa9f4006740 1 -- 172.21.15.120:0/2723632662 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9e4011de0 con 0x55dd0a8dc370 2024-09-19T09:23:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.267+0000 7fa9d27fc640 1 -- 172.21.15.120:0/2723632662 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa9e4011760 con 0x55dd0a8dc370 2024-09-19T09:23:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.267+0000 7fa9f4006740 10 client.22191 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.267+0000 7fa9f4006740 1 -- 172.21.15.120:0/2723632662 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55dd0a619620 con 0x55dd0a8dc370 2024-09-19T09:23:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.267+0000 7fa9d27fc640 1 -- 172.21.15.120:0/2723632662 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e4011de0 con 0x55dd0a8dc370 2024-09-19T09:23:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.268+0000 7fa9d27fc640 1 -- 172.21.15.120:0/2723632662 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fa9e4041070 con 0x55dd0a8dc370 2024-09-19T09:23:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.268+0000 7fa9f4006740 10 client.22191 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.268+0000 7fa9f4006740 10 client.22191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:23:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.268+0000 7fa9f4006740 20 client.22191 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.268+0000 7fa9f4006740 1 --2- 172.21.15.120:0/2723632662 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dd0a8ec5f0 0x55dd0a8ee9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.268+0000 7fa9f4006740 4 client.22191 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:23:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.268+0000 7fa9f4006740 1 -- 172.21.15.120:0/2723632662 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd0a847a20 con 0x55dd0a8ec5f0 2024-09-19T09:23:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.268+0000 7fa9eaffd640 1 --2- 172.21.15.120:0/2723632662 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dd0a8ec5f0 0x55dd0a8ee9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.268+0000 7fa9d27fc640 10 client.22191 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.268+0000 7fa9eaffd640 1 --2- 172.21.15.120:0/2723632662 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dd0a8ec5f0 0x55dd0a8ee9e0 crc :-1 s=READY pgs=509 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebb260e169272996 server_cookie=a14b5ae82f51bef in_seq=0 out_seq=0 2024-09-19T09:23:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.271+0000 7fa9d27fc640 1 -- 172.21.15.120:0/2723632662 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dd0a847a20 con 0x55dd0a8ec5f0 2024-09-19T09:23:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.271+0000 7fa9d27fc640 10 client.22191 handle_command_reply: tid=0 2024-09-19T09:23:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.271+0000 7fa9f4006740 1 client.22191 shutdown 2024-09-19T09:23:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.271+0000 7fa9f4006740 1 -- 172.21.15.120:0/2723632662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd0a8dc370 msgr2=0x55dd0a8e81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.271+0000 7fa9f4006740 1 --2- 172.21.15.120:0/2723632662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd0a8dc370 0x55dd0a8e81e0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4005bf0 tx=0x7fa9e401ba30 comp rx=0 tx=0).stop 2024-09-19T09:23:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.271+0000 7fa9f4006740 1 -- 172.21.15.120:0/2723632662 shutdown_connections 2024-09-19T09:23:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.271+0000 7fa9f4006740 1 --2- 172.21.15.120:0/2723632662 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dd0a8ec5f0 0x55dd0a8ee9e0 crc :-1 s=READY pgs=509 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.271+0000 7fa9f4006740 1 --2- 172.21.15.120:0/2723632662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd0a8dc370 0x55dd0a8e81e0 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.271+0000 7fa9d27fc640 0 client.22191 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.271+0000 7fa9f4006740 1 --2- 172.21.15.120:0/2723632662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd0a8d72e0 0x55dd0a8e11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.271+0000 7fa9f4006740 1 --2- 172.21.15.120:0/2723632662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd0a8d69a0 0x55dd0a8e0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.271+0000 7fa9f4006740 1 -- 172.21.15.120:0/2723632662 >> 172.21.15.120:0/2723632662 conn(0x55dd0a8d2010 msgr2=0x55dd0a8d3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.271+0000 7fa9f4006740 1 -- 172.21.15.120:0/2723632662 shutdown_connections 2024-09-19T09:23:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.271+0000 7fa9f4006740 1 -- 172.21.15.120:0/2723632662 wait complete. 2024-09-19T09:23:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.271+0000 7fa9f4006740 20 client.22191 trim_cache size 0 max 16384 2024-09-19T09:23:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.285+0000 7fa9f4006740 1 Processor -- start 2024-09-19T09:23:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.286+0000 7fa9f4006740 1 -- start start 2024-09-19T09:23:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.286+0000 7fa9f4006740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd0a8d69a0 0x55dd0a8d3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.286+0000 7fa9f4006740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd0a8d72e0 0x55dd0a8e81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.286+0000 7fa9f4006740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd0a8dc370 0x55dd0a8d3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.286+0000 7fa9f4006740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec1a6e00 con 0x55dd0a8dc370 2024-09-19T09:23:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.286+0000 7fa9f4006740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec1a8940 con 0x55dd0a8d72e0 2024-09-19T09:23:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.286+0000 7fa9f4006740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec1a74d0 con 0x55dd0a8d69a0 2024-09-19T09:23:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.286+0000 7fa9eb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd0a8dc370 0x55dd0a8d3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.286+0000 7fa9eb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd0a8dc370 0x55dd0a8d3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46934/0 (socket says 172.21.15.120:46934) 2024-09-19T09:23:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.286+0000 7fa9eb7fe640 1 -- 172.21.15.120:0/142986880 learned_addr learned my addr 172.21.15.120:0/142986880 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.286+0000 7fa9e3fff640 1 --2- 172.21.15.120:0/142986880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd0a8d72e0 0x55dd0a8e81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.286+0000 7fa9eaffd640 1 --2- 172.21.15.120:0/142986880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd0a8d69a0 0x55dd0a8d3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.286+0000 7fa9eb7fe640 1 -- 172.21.15.120:0/142986880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd0a8d69a0 msgr2=0x55dd0a8d3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.286+0000 7fa9eb7fe640 1 --2- 172.21.15.120:0/142986880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd0a8d69a0 0x55dd0a8d3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.286+0000 7fa9eb7fe640 1 -- 172.21.15.120:0/142986880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd0a8d72e0 msgr2=0x55dd0a8e81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.286+0000 7fa9eb7fe640 1 --2- 172.21.15.120:0/142986880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd0a8d72e0 0x55dd0a8e81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.286+0000 7fa9eb7fe640 1 -- 172.21.15.120:0/142986880 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd0a8d3e80 con 0x55dd0a8dc370 2024-09-19T09:23:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.287+0000 7fa9eb7fe640 1 --2- 172.21.15.120:0/142986880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd0a8dc370 0x55dd0a8d3940 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4009560 tx=0x7fa9e401bb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77883d51c4c4728f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.287+0000 7fa9e0ff9640 1 -- 172.21.15.120:0/142986880 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4047070 con 0x55dd0a8dc370 2024-09-19T09:23:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.287+0000 7fa9e0ff9640 1 -- 172.21.15.120:0/142986880 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e4009aa0 con 0x55dd0a8dc370 2024-09-19T09:23:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.287+0000 7fa9e0ff9640 1 -- 172.21.15.120:0/142986880 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e403c040 con 0x55dd0a8dc370 2024-09-19T09:23:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.288+0000 7fa9f4006740 1 -- 172.21.15.120:0/142986880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd0a8dc370 msgr2=0x55dd0a8d3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.288+0000 7fa9f4006740 1 --2- 172.21.15.120:0/142986880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd0a8dc370 0x55dd0a8d3940 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4009560 tx=0x7fa9e401bb20 comp rx=0 tx=0).stop 2024-09-19T09:23:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.288+0000 7fa9f4006740 1 -- 172.21.15.120:0/142986880 shutdown_connections 2024-09-19T09:23:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.288+0000 7fa9f4006740 1 --2- 172.21.15.120:0/142986880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd0a8dc370 0x55dd0a8d3940 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.288+0000 7fa9f4006740 1 --2- 172.21.15.120:0/142986880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd0a8d72e0 0x55dd0a8e81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.288+0000 7fa9f4006740 1 --2- 172.21.15.120:0/142986880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd0a8d69a0 0x55dd0a8d3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.288+0000 7fa9f4006740 1 -- 172.21.15.120:0/142986880 >> 172.21.15.120:0/142986880 conn(0x55dd0a8d2010 msgr2=0x55dd0a8d11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.288+0000 7fa9f4006740 1 -- 172.21.15.120:0/142986880 shutdown_connections 2024-09-19T09:23:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.288+0000 7fa9f4006740 1 -- 172.21.15.120:0/142986880 wait complete. 2024-09-19T09:23:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.288+0000 7fa9f4006740 1 -- start start 2024-09-19T09:23:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.288+0000 7fa9f4006740 1 Processor -- start 2024-09-19T09:23:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.288+0000 7fa9f4006740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd0a8d69a0 0x55dd0a8e7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.288+0000 7fa9f4006740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd0a8d72e0 0x55dd0a8e62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.288+0000 7fa9f4006740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd0a8dc370 0x55dd0a8e67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.288+0000 7fa9f4006740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec1a8270 con 0x55dd0a8d72e0 2024-09-19T09:23:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.288+0000 7fa9f4006740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec1a7ba0 con 0x55dd0a8d69a0 2024-09-19T09:23:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.288+0000 7fa9f4006740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec1a4dc0 con 0x55dd0a8dc370 2024-09-19T09:23:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.289+0000 7fa9f4006740 10 client.0 _ll_register_callbacks cb 0x7fa9e40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.289+0000 7fa9e3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd0a8d72e0 0x55dd0a8e62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.289+0000 7fa9e3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd0a8d72e0 0x55dd0a8e62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46938/0 (socket says 172.21.15.120:46938) 2024-09-19T09:23:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.289+0000 7fa9eaffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd0a8d69a0 0x55dd0a8e7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.289+0000 7fa9e3fff640 1 -- 172.21.15.120:0/3964817813 learned_addr learned my addr 172.21.15.120:0/3964817813 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.289+0000 7fa9eb7fe640 1 --2- 172.21.15.120:0/3964817813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd0a8dc370 0x55dd0a8e67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.289+0000 7fa9e3fff640 1 -- 172.21.15.120:0/3964817813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd0a8dc370 msgr2=0x55dd0a8e67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.289+0000 7fa9e3fff640 1 --2- 172.21.15.120:0/3964817813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd0a8dc370 0x55dd0a8e67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.289+0000 7fa9e3fff640 1 -- 172.21.15.120:0/3964817813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd0a8d69a0 msgr2=0x55dd0a8e7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.289+0000 7fa9e3fff640 1 --2- 172.21.15.120:0/3964817813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd0a8d69a0 0x55dd0a8e7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.289+0000 7fa9e3fff640 1 -- 172.21.15.120:0/3964817813 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd0a8e6d20 con 0x55dd0a8d72e0 2024-09-19T09:23:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.289+0000 7fa9eaffd640 1 --2- 172.21.15.120:0/3964817813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd0a8d69a0 0x55dd0a8e7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.289+0000 7fa9e3fff640 1 --2- 172.21.15.120:0/3964817813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd0a8d72e0 0x55dd0a8e62a0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40097b0 tx=0x7fa9d4005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84cd3a67687220c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.289+0000 7fa9d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:23:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.289+0000 7fa9d27fc640 1 -- 172.21.15.120:0/3964817813 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d40ce040 con 0x55dd0a8d72e0 2024-09-19T09:23:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.289+0000 7fa9d27fc640 1 -- 172.21.15.120:0/3964817813 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa9d4084d40 con 0x55dd0a8d72e0 2024-09-19T09:23:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.289+0000 7fa9f4006740 1 -- 172.21.15.120:0/3964817813 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9e4009aa0 con 0x55dd0a8d72e0 2024-09-19T09:23:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.289+0000 7fa9d27fc640 1 -- 172.21.15.120:0/3964817813 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa9d409b1e0 con 0x55dd0a8d72e0 2024-09-19T09:23:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.289+0000 7fa9f4006740 10 client.22197 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.289+0000 7fa9f4006740 1 -- 172.21.15.120:0/3964817813 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55dd0a957ec0 con 0x55dd0a8d72e0 2024-09-19T09:23:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.290+0000 7fa9d27fc640 1 -- 172.21.15.120:0/3964817813 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e4009aa0 con 0x55dd0a8d72e0 2024-09-19T09:23:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.290+0000 7fa9d27fc640 1 -- 172.21.15.120:0/3964817813 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fa9d409ad60 con 0x55dd0a8d72e0 2024-09-19T09:23:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.290+0000 7fa9f4006740 10 client.22197 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.290+0000 7fa9f4006740 10 client.22197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:23:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.290+0000 7fa9f4006740 20 client.22197 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.290+0000 7fa9f4006740 1 --2- 172.21.15.120:0/3964817813 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dd0a9587a0 0x55dd0a95abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.290+0000 7fa9f4006740 4 client.22197 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:23:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.290+0000 7fa9f4006740 1 -- 172.21.15.120:0/3964817813 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dd0a841d70 con 0x55dd0a9587a0 2024-09-19T09:23:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.290+0000 7fa9eaffd640 1 --2- 172.21.15.120:0/3964817813 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dd0a9587a0 0x55dd0a95abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.291+0000 7fa9eaffd640 1 --2- 172.21.15.120:0/3964817813 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dd0a9587a0 0x55dd0a95abc0 crc :-1 s=READY pgs=510 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24da6a3b6b7f8f27 server_cookie=3d5694d1892efb46 in_seq=0 out_seq=0 2024-09-19T09:23:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.291+0000 7fa9d27fc640 10 client.22197 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:16.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:16 smithi120 ceph-mon[31150]: pgmap v957: 97 pgs: 97 active+clean; 423 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 3.3 MiB/s wr, 459 op/s 2024-09-19T09:23:16.376 INFO:tasks.workunit.client.0.smithi120.stdout: 10 173187 20.39 MB/sec execute 311 sec latency 159.748 ms 2024-09-19T09:23:16.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:16 smithi196 ceph-mon[35150]: pgmap v957: 97 pgs: 97 active+clean; 423 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 3.3 MiB/s wr, 459 op/s 2024-09-19T09:23:16.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:16 smithi153 ceph-mon[35071]: pgmap v957: 97 pgs: 97 active+clean; 423 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 3.3 MiB/s wr, 459 op/s 2024-09-19T09:23:16.465 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:23:16.884 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:23:16.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.926+0000 7fa9d27fc640 1 -- 172.21.15.120:0/3964817813 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dd0a841d70 con 0x55dd0a9587a0 2024-09-19T09:23:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.926+0000 7fa9d27fc640 10 client.22197 handle_command_reply: tid=0 2024-09-19T09:23:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.926+0000 7fa9f4006740 1 client.22197 shutdown 2024-09-19T09:23:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.926+0000 7fa9f4006740 1 -- 172.21.15.120:0/3964817813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd0a8d72e0 msgr2=0x55dd0a8e62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.926+0000 7fa9f4006740 1 --2- 172.21.15.120:0/3964817813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd0a8d72e0 0x55dd0a8e62a0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40097b0 tx=0x7fa9d4005af0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.926+0000 7fa9f4006740 1 -- 172.21.15.120:0/3964817813 shutdown_connections 2024-09-19T09:23:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.926+0000 7fa9f4006740 1 --2- 172.21.15.120:0/3964817813 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dd0a9587a0 0x55dd0a95abc0 crc :-1 s=READY pgs=510 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.926+0000 7fa9f4006740 1 --2- 172.21.15.120:0/3964817813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dd0a8dc370 0x55dd0a8e67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.926+0000 7fa9f4006740 1 --2- 172.21.15.120:0/3964817813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dd0a8d72e0 0x55dd0a8e62a0 secure :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40097b0 tx=0x7fa9d4005af0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.926+0000 7fa9f4006740 1 --2- 172.21.15.120:0/3964817813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dd0a8d69a0 0x55dd0a8e7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.926+0000 7fa9f4006740 1 -- 172.21.15.120:0/3964817813 >> 172.21.15.120:0/3964817813 conn(0x55dd0a8d2010 msgr2=0x55dd0a8d23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.927+0000 7fa9d27fc640 0 client.22197 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.927+0000 7fa9f4006740 1 -- 172.21.15.120:0/3964817813 shutdown_connections 2024-09-19T09:23:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.927+0000 7fa9f4006740 1 -- 172.21.15.120:0/3964817813 wait complete. 2024-09-19T09:23:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.927+0000 7fa9f4006740 20 client.22197 trim_cache size 0 max 16384 2024-09-19T09:23:16.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:23:16.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:23:16.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:23:16.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:23:16.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.928+0000 7fa9f1265640 1 -- 172.21.15.120:0/1853857759 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9d80a32f0 msgr2=0x7fa9d80a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.928+0000 7fa9f1265640 1 --2- 172.21.15.120:0/1853857759 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9d80a32f0 0x7fa9d80a57b0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc005ab0 tx=0x7fa9dc03c040 comp rx=0 tx=0).stop 2024-09-19T09:23:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.928+0000 7fa9f1265640 1 -- 172.21.15.120:0/1853857759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9ec101c30 msgr2=0x7fa9ec197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.928+0000 7fa9f1265640 1 --2- 172.21.15.120:0/1853857759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9ec101c30 0x7fa9ec197240 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7fa9d400e9f0 tx=0x7fa9d400eeb0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.928+0000 7fa9f1265640 1 -- 172.21.15.120:0/1853857759 shutdown_connections 2024-09-19T09:23:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.928+0000 7fa9f1265640 1 --2- 172.21.15.120:0/1853857759 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9d80a32f0 0x7fa9d80a57b0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.928+0000 7fa9f1265640 1 --2- 172.21.15.120:0/1853857759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9ec1025f0 0x7fa9ec19e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.928+0000 7fa9f1265640 1 --2- 172.21.15.120:0/1853857759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa9ec101c30 0x7fa9ec197240 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.928+0000 7fa9f1265640 1 --2- 172.21.15.120:0/1853857759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9ec100a40 0x7fa9ec196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.928+0000 7fa9f1265640 1 -- 172.21.15.120:0/1853857759 >> 172.21.15.120:0/1853857759 conn(0x7fa9ec0fc1e0 msgr2=0x7fa9ec104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.928+0000 7fa9f1265640 1 -- 172.21.15.120:0/1853857759 shutdown_connections 2024-09-19T09:23:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:16.928+0000 7fa9f1265640 1 -- 172.21.15.120:0/1853857759 wait complete. 2024-09-19T09:23:17.304 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:17 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:23:17.304 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:17 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:23:17.304 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:17 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:23:17.304 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:17 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:23:17.304 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:17 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:23:17.305 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:17 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:23:17.305 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:17 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:23:17.305 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:17 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:23:17.305 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:17 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:23:17.376 INFO:tasks.workunit.client.0.smithi120.stdout: 10 173600 20.41 MB/sec execute 312 sec latency 338.873 ms 2024-09-19T09:23:17.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.400+0000 7fa342af4640 1 -- 172.21.15.120:0/1535022022 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa33c10c160 msgr2=0x7fa33c10e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:17.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.400+0000 7fa342af4640 1 --2- 172.21.15.120:0/1535022022 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa33c10c160 0x7fa33c10e600 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7fa334009ca0 tx=0x7fa33402a950 comp rx=0 tx=0).stop 2024-09-19T09:23:17.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.400+0000 7fa342af4640 1 -- 172.21.15.120:0/1535022022 shutdown_connections 2024-09-19T09:23:17.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.400+0000 7fa342af4640 1 --2- 172.21.15.120:0/1535022022 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa33c10c160 0x7fa33c10e600 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.400+0000 7fa342af4640 1 --2- 172.21.15.120:0/1535022022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa33c103650 0x7fa33c10bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.400+0000 7fa342af4640 1 --2- 172.21.15.120:0/1535022022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa33c102d30 0x7fa33c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.400+0000 7fa342af4640 1 -- 172.21.15.120:0/1535022022 >> 172.21.15.120:0/1535022022 conn(0x7fa33c0fc790 msgr2=0x7fa33c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:17.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.400+0000 7fa342af4640 1 -- 172.21.15.120:0/1535022022 shutdown_connections 2024-09-19T09:23:17.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.401+0000 7fa342af4640 1 -- 172.21.15.120:0/1535022022 wait complete. 2024-09-19T09:23:17.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.401+0000 7fa342af4640 1 Processor -- start 2024-09-19T09:23:17.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.401+0000 7fa342af4640 1 -- start start 2024-09-19T09:23:17.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.402+0000 7fa342af4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa33c102d30 0x7fa33c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:17.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.402+0000 7fa342af4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa33c103650 0x7fa33c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:17.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.402+0000 7fa342af4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa33c10c160 0x7fa33c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:17.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.402+0000 7fa342af4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c111f60 con 0x7fa33c102d30 2024-09-19T09:23:17.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.402+0000 7fa342af4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c111de0 con 0x7fa33c103650 2024-09-19T09:23:17.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.402+0000 7fa342af4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c1120e0 con 0x7fa33c10c160 2024-09-19T09:23:17.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.402+0000 7fa340890640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa33c102d30 0x7fa33c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:17.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.402+0000 7fa33bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa33c103650 0x7fa33c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:17.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.402+0000 7fa340890640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa33c102d30 0x7fa33c100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46962/0 (socket says 172.21.15.120:46962) 2024-09-19T09:23:17.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.402+0000 7fa340890640 1 -- 172.21.15.120:0/2144414341 learned_addr learned my addr 172.21.15.120:0/2144414341 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:17.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.402+0000 7fa341091640 1 --2- 172.21.15.120:0/2144414341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa33c10c160 0x7fa33c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:17.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.402+0000 7fa33bfff640 1 -- 172.21.15.120:0/2144414341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa33c10c160 msgr2=0x7fa33c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:17.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.402+0000 7fa33bfff640 1 --2- 172.21.15.120:0/2144414341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa33c10c160 0x7fa33c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.402+0000 7fa33bfff640 1 -- 172.21.15.120:0/2144414341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa33c102d30 msgr2=0x7fa33c100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:17.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.402+0000 7fa33bfff640 1 --2- 172.21.15.120:0/2144414341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa33c102d30 0x7fa33c100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.402+0000 7fa33bfff640 1 -- 172.21.15.120:0/2144414341 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33c1a6930 con 0x7fa33c103650 2024-09-19T09:23:17.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.402+0000 7fa340890640 1 --2- 172.21.15.120:0/2144414341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa33c102d30 0x7fa33c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:17.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.402+0000 7fa33bfff640 1 --2- 172.21.15.120:0/2144414341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa33c103650 0x7fa33c100ed0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa32800e7a0 tx=0x7fa32800ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:17.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.402+0000 7fa339ffb640 1 -- 172.21.15.120:0/2144414341 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32800ccc0 con 0x7fa33c103650 2024-09-19T09:23:17.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.403+0000 7fa342af4640 1 -- 172.21.15.120:0/2144414341 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa33c1a6c20 con 0x7fa33c103650 2024-09-19T09:23:17.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.403+0000 7fa339ffb640 1 -- 172.21.15.120:0/2144414341 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa328009530 con 0x7fa33c103650 2024-09-19T09:23:17.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.403+0000 7fa339ffb640 1 -- 172.21.15.120:0/2144414341 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa328018880 con 0x7fa33c103650 2024-09-19T09:23:17.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.403+0000 7fa342af4640 1 -- 172.21.15.120:0/2144414341 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa33c1a71b0 con 0x7fa33c103650 2024-09-19T09:23:17.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.405+0000 7fa339ffb640 1 -- 172.21.15.120:0/2144414341 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa328018a20 con 0x7fa33c103650 2024-09-19T09:23:17.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.406+0000 7fa344ecd740 1 Processor -- start 2024-09-19T09:23:17.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.406+0000 7fa344ecd740 1 -- start start 2024-09-19T09:23:17.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.406+0000 7fa344ecd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4d569f4a0 0x55a4d569f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:17.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.406+0000 7fa344ecd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4d569fdc0 0x55a4d56aa140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:17.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.406+0000 7fa344ecd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a4d56aa680 0x55a4d56aca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:17.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.406+0000 7fa344ecd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a4d56052e0 con 0x55a4d56aa680 2024-09-19T09:23:17.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.406+0000 7fa344ecd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a4d55acce0 con 0x55a4d569f4a0 2024-09-19T09:23:17.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.406+0000 7fa344ecd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a4d56012a0 con 0x55a4d569fdc0 2024-09-19T09:23:17.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.406+0000 7fa33bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a4d56aa680 0x55a4d56aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:17.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.406+0000 7fa341091640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4d569f4a0 0x55a4d569f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:17.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.406+0000 7fa339ffb640 1 --2- 172.21.15.120:0/2144414341 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa32c0a3240 0x7fa32c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:17.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.406+0000 7fa340890640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4d569fdc0 0x55a4d56aa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:17.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.406+0000 7fa340890640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4d569fdc0 0x55a4d56aa140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:50740/0 (socket says 172.21.15.120:50740) 2024-09-19T09:23:17.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.406+0000 7fa33bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a4d56aa680 0x55a4d56aca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46968/0 (socket says 172.21.15.120:46968) 2024-09-19T09:23:17.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.406+0000 7fa340890640 1 -- 172.21.15.120:0/949041207 learned_addr learned my addr 172.21.15.120:0/949041207 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:17.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.406+0000 7fa340890640 1 -- 172.21.15.120:0/949041207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4d569f4a0 msgr2=0x55a4d569f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:17.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.406+0000 7fa340890640 1 --2- 172.21.15.120:0/949041207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4d569f4a0 0x55a4d569f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.406+0000 7fa341091640 1 --2- 172.21.15.120:0/2144414341 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa32c0a3240 0x7fa32c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:17.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.406+0000 7fa340890640 1 -- 172.21.15.120:0/949041207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a4d56aa680 msgr2=0x55a4d56aca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:17.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.406+0000 7fa340890640 1 --2- 172.21.15.120:0/949041207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a4d56aa680 0x55a4d56aca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.407+0000 7fa340890640 1 -- 172.21.15.120:0/949041207 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4d5591440 con 0x55a4d569fdc0 2024-09-19T09:23:17.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.407+0000 7fa339ffb640 1 -- 172.21.15.120:0/2144414341 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa3280cfa60 con 0x7fa33c103650 2024-09-19T09:23:17.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.407+0000 7fa340890640 1 --2- 172.21.15.120:0/949041207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4d569fdc0 0x55a4d56aa140 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa330011630 tx=0x7fa330011af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12222887389d61ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:17.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.407+0000 7fa326ffd640 1 -- 172.21.15.120:0/949041207 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33001e070 con 0x55a4d569fdc0 2024-09-19T09:23:17.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.407+0000 7fa341091640 1 --2- 172.21.15.120:0/2144414341 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa32c0a3240 0x7fa32c0a5700 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa334004ad0 tx=0x7fa33402a950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:17.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.407+0000 7fa326ffd640 1 -- 172.21.15.120:0/949041207 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa33000fbd0 con 0x55a4d569fdc0 2024-09-19T09:23:17.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.407+0000 7fa326ffd640 1 -- 172.21.15.120:0/949041207 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330019d00 con 0x55a4d569fdc0 2024-09-19T09:23:17.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.408+0000 7fa344ecd740 1 -- 172.21.15.120:0/949041207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4d569fdc0 msgr2=0x55a4d56aa140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:17.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.408+0000 7fa344ecd740 1 --2- 172.21.15.120:0/949041207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4d569fdc0 0x55a4d56aa140 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa330011630 tx=0x7fa330011af0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.408+0000 7fa344ecd740 1 -- 172.21.15.120:0/949041207 shutdown_connections 2024-09-19T09:23:17.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.408+0000 7fa344ecd740 1 --2- 172.21.15.120:0/949041207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a4d56aa680 0x55a4d56aca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.408+0000 7fa344ecd740 1 --2- 172.21.15.120:0/949041207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4d569fdc0 0x55a4d56aa140 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.408+0000 7fa344ecd740 1 --2- 172.21.15.120:0/949041207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4d569f4a0 0x55a4d569f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.408+0000 7fa344ecd740 1 -- 172.21.15.120:0/949041207 >> 172.21.15.120:0/949041207 conn(0x55a4d569ba30 msgr2=0x55a4d569be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:17.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.408+0000 7fa344ecd740 1 -- 172.21.15.120:0/949041207 shutdown_connections 2024-09-19T09:23:17.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.408+0000 7fa344ecd740 1 -- 172.21.15.120:0/949041207 wait complete. 2024-09-19T09:23:17.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.408+0000 7fa344ecd740 1 -- start start 2024-09-19T09:23:17.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.408+0000 7fa344ecd740 1 Processor -- start 2024-09-19T09:23:17.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.409+0000 7fa344ecd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a4d569f4a0 0x55a4d56a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:17.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.409+0000 7fa344ecd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4d569fdc0 0x55a4d56a7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:17.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.409+0000 7fa344ecd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4d56aa680 0x55a4d56a7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:17.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.409+0000 7fa344ecd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a4d56ad250 con 0x55a4d569f4a0 2024-09-19T09:23:17.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.409+0000 7fa344ecd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a4d55c0050 con 0x55a4d56aa680 2024-09-19T09:23:17.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.409+0000 7fa344ecd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a4d56ad3d0 con 0x55a4d569fdc0 2024-09-19T09:23:17.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.409+0000 7fa344ecd740 10 client.0 _ll_register_callbacks cb 0x55a4d5600160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:17.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.409+0000 7fa340890640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a4d569f4a0 0x55a4d56a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:17.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.409+0000 7fa340890640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a4d569f4a0 0x55a4d56a9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46982/0 (socket says 172.21.15.120:46982) 2024-09-19T09:23:17.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.409+0000 7fa341091640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4d56aa680 0x55a4d56a7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:17.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.409+0000 7fa340890640 1 -- 172.21.15.120:0/887176419 learned_addr learned my addr 172.21.15.120:0/887176419 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:17.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.409+0000 7fa33bfff640 1 --2- 172.21.15.120:0/887176419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4d569fdc0 0x55a4d56a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:17.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.409+0000 7fa340890640 1 -- 172.21.15.120:0/887176419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4d569fdc0 msgr2=0x55a4d56a7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:17.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.409+0000 7fa340890640 1 --2- 172.21.15.120:0/887176419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4d569fdc0 0x55a4d56a7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.409+0000 7fa340890640 1 -- 172.21.15.120:0/887176419 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4d56aa680 msgr2=0x55a4d56a7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:17.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.409+0000 7fa340890640 1 --2- 172.21.15.120:0/887176419 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4d56aa680 0x55a4d56a7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.409+0000 7fa340890640 1 -- 172.21.15.120:0/887176419 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4d53763c0 con 0x55a4d569f4a0 2024-09-19T09:23:17.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.409+0000 7fa340890640 1 --2- 172.21.15.120:0/887176419 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a4d569f4a0 0x55a4d56a9f70 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7fa33001c040 tx=0x7fa33000fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42228baf4b783805 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:17.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.409+0000 7fa31bfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:23:17.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.409+0000 7fa31bfff640 1 -- 172.21.15.120:0/887176419 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330026020 con 0x55a4d569f4a0 2024-09-19T09:23:17.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.410+0000 7fa31bfff640 1 -- 172.21.15.120:0/887176419 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa330009ce0 con 0x55a4d569f4a0 2024-09-19T09:23:17.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.410+0000 7fa344ecd740 1 -- 172.21.15.120:0/887176419 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa33000fbd0 con 0x55a4d569f4a0 2024-09-19T09:23:17.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.410+0000 7fa31bfff640 1 -- 172.21.15.120:0/887176419 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa33001e040 con 0x55a4d569f4a0 2024-09-19T09:23:17.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.410+0000 7fa344ecd740 10 client.22209 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:17.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.410+0000 7fa344ecd740 1 -- 172.21.15.120:0/887176419 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a4d56b8bc0 con 0x55a4d569f4a0 2024-09-19T09:23:17.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.410+0000 7fa31bfff640 1 -- 172.21.15.120:0/887176419 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa33000fbd0 con 0x55a4d569f4a0 2024-09-19T09:23:17.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.410+0000 7fa31bfff640 1 -- 172.21.15.120:0/887176419 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fa330021050 con 0x55a4d569f4a0 2024-09-19T09:23:17.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.410+0000 7fa344ecd740 10 client.22209 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:17.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.410+0000 7fa344ecd740 10 client.22209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:23:17.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.410+0000 7fa344ecd740 20 client.22209 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:17.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.410+0000 7fa344ecd740 1 --2- 172.21.15.120:0/887176419 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a4d56b9f70 0x55a4d56bc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:17.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.410+0000 7fa344ecd740 4 client.22209 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:23:17.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.410+0000 7fa344ecd740 1 -- 172.21.15.120:0/887176419 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4d5612850 con 0x55a4d56b9f70 2024-09-19T09:23:17.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.410+0000 7fa33bfff640 1 --2- 172.21.15.120:0/887176419 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a4d56b9f70 0x55a4d56bc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:17.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.411+0000 7fa31bfff640 10 client.22209 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:17.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.411+0000 7fa33bfff640 1 --2- 172.21.15.120:0/887176419 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a4d56b9f70 0x55a4d56bc360 crc :-1 s=READY pgs=511 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbedc4961e56f707 server_cookie=91525e85a83003b2 in_seq=0 out_seq=0 2024-09-19T09:23:17.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.413+0000 7fa31bfff640 1 -- 172.21.15.120:0/887176419 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a4d5612850 con 0x55a4d56b9f70 2024-09-19T09:23:17.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.413+0000 7fa31bfff640 10 client.22209 handle_command_reply: tid=0 2024-09-19T09:23:17.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.413+0000 7fa344ecd740 1 client.22209 shutdown 2024-09-19T09:23:17.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.413+0000 7fa344ecd740 1 -- 172.21.15.120:0/887176419 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a4d569f4a0 msgr2=0x55a4d56a9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:17.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.413+0000 7fa344ecd740 1 --2- 172.21.15.120:0/887176419 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a4d569f4a0 0x55a4d56a9f70 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7fa33001c040 tx=0x7fa33000fb20 comp rx=0 tx=0).stop 2024-09-19T09:23:17.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.413+0000 7fa344ecd740 1 -- 172.21.15.120:0/887176419 shutdown_connections 2024-09-19T09:23:17.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.413+0000 7fa344ecd740 1 --2- 172.21.15.120:0/887176419 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a4d56b9f70 0x55a4d56bc360 crc :-1 s=READY pgs=511 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.413+0000 7fa31bfff640 0 client.22209 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:17.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.413+0000 7fa344ecd740 1 --2- 172.21.15.120:0/887176419 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4d56aa680 0x55a4d56a7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.413+0000 7fa344ecd740 1 --2- 172.21.15.120:0/887176419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4d569fdc0 0x55a4d56a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.413+0000 7fa344ecd740 1 --2- 172.21.15.120:0/887176419 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a4d569f4a0 0x55a4d56a9f70 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.413+0000 7fa344ecd740 1 -- 172.21.15.120:0/887176419 >> 172.21.15.120:0/887176419 conn(0x55a4d569ba30 msgr2=0x55a4d56a1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:17.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.413+0000 7fa344ecd740 1 -- 172.21.15.120:0/887176419 shutdown_connections 2024-09-19T09:23:17.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.413+0000 7fa344ecd740 1 -- 172.21.15.120:0/887176419 wait complete. 2024-09-19T09:23:17.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.413+0000 7fa344ecd740 20 client.22209 trim_cache size 0 max 16384 2024-09-19T09:23:17.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:17 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:23:17.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:17 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:23:17.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:17 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:23:17.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:17 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:23:17.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:17 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:23:17.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:17 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:23:17.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:17 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:23:17.441 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:17 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:23:17.441 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:17 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:23:17.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:17 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:23:17.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:17 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:23:17.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:17 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:23:17.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:17 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:23:17.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:17 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:23:17.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:17 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:23:17.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:17 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:23:17.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:17 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:23:17.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:17 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:23:17.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.433+0000 7fa344ecd740 1 Processor -- start 2024-09-19T09:23:17.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.433+0000 7fa344ecd740 1 -- start start 2024-09-19T09:23:17.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.433+0000 7fa344ecd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4d569f4a0 0x55a4d56c1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:17.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.433+0000 7fa344ecd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4d569fdc0 0x55a4d56c1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:17.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.433+0000 7fa344ecd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a4d56aa680 0x55a4d56a4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:17.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.433+0000 7fa344ecd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c1af840 con 0x55a4d56aa680 2024-09-19T09:23:17.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.433+0000 7fa344ecd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c1b1380 con 0x55a4d569fdc0 2024-09-19T09:23:17.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.433+0000 7fa344ecd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c1aff10 con 0x55a4d569f4a0 2024-09-19T09:23:17.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.433+0000 7fa341091640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a4d56aa680 0x55a4d56a4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:17.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.433+0000 7fa33bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4d569fdc0 0x55a4d56c1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:17.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.433+0000 7fa33bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4d569fdc0 0x55a4d56c1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:41470/0 (socket says 172.21.15.120:41470) 2024-09-19T09:23:17.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.433+0000 7fa340890640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4d569f4a0 0x55a4d56c1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:17.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.433+0000 7fa341091640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a4d56aa680 0x55a4d56a4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46988/0 (socket says 172.21.15.120:46988) 2024-09-19T09:23:17.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.433+0000 7fa33bfff640 1 -- 172.21.15.120:0/882873812 learned_addr learned my addr 172.21.15.120:0/882873812 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:17.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.434+0000 7fa341091640 1 -- 172.21.15.120:0/882873812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4d569f4a0 msgr2=0x55a4d56c1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:17.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.434+0000 7fa341091640 1 --2- 172.21.15.120:0/882873812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4d569f4a0 0x55a4d56c1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.434+0000 7fa341091640 1 -- 172.21.15.120:0/882873812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4d569fdc0 msgr2=0x55a4d56c1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:17.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.434+0000 7fa341091640 1 --2- 172.21.15.120:0/882873812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4d569fdc0 0x55a4d56c1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.434+0000 7fa341091640 1 -- 172.21.15.120:0/882873812 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4d56aa2e0 con 0x55a4d56aa680 2024-09-19T09:23:17.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.434+0000 7fa341091640 1 --2- 172.21.15.120:0/882873812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a4d56aa680 0x55a4d56a4480 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fa334005cc0 tx=0x7fa334004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7484d3f326dc5d01 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:17.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.434+0000 7fa3267fc640 1 -- 172.21.15.120:0/882873812 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa334043050 con 0x55a4d56aa680 2024-09-19T09:23:17.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.434+0000 7fa3267fc640 1 -- 172.21.15.120:0/882873812 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa334044040 con 0x55a4d56aa680 2024-09-19T09:23:17.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.434+0000 7fa3267fc640 1 -- 172.21.15.120:0/882873812 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa334034040 con 0x55a4d56aa680 2024-09-19T09:23:17.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.435+0000 7fa344ecd740 1 -- 172.21.15.120:0/882873812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a4d56aa680 msgr2=0x55a4d56a4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:17.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.435+0000 7fa344ecd740 1 --2- 172.21.15.120:0/882873812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a4d56aa680 0x55a4d56a4480 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fa334005cc0 tx=0x7fa334004a60 comp rx=0 tx=0).stop 2024-09-19T09:23:17.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.435+0000 7fa344ecd740 1 -- 172.21.15.120:0/882873812 shutdown_connections 2024-09-19T09:23:17.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.435+0000 7fa344ecd740 1 --2- 172.21.15.120:0/882873812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a4d56aa680 0x55a4d56a4480 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.435+0000 7fa344ecd740 1 --2- 172.21.15.120:0/882873812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4d569fdc0 0x55a4d56c1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.435+0000 7fa344ecd740 1 --2- 172.21.15.120:0/882873812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4d569f4a0 0x55a4d56c1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.435+0000 7fa344ecd740 1 -- 172.21.15.120:0/882873812 >> 172.21.15.120:0/882873812 conn(0x55a4d569ba30 msgr2=0x55a4d56a61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:17.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.435+0000 7fa344ecd740 1 -- 172.21.15.120:0/882873812 shutdown_connections 2024-09-19T09:23:17.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.435+0000 7fa344ecd740 1 -- 172.21.15.120:0/882873812 wait complete. 2024-09-19T09:23:17.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.435+0000 7fa344ecd740 1 -- start start 2024-09-19T09:23:17.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.436+0000 7fa344ecd740 1 Processor -- start 2024-09-19T09:23:17.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.436+0000 7fa344ecd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4d569f4a0 0x55a4d56b2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:17.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.436+0000 7fa344ecd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a4d569fdc0 0x55a4d56afef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:17.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.436+0000 7fa344ecd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4d56aa680 0x55a4d56b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:17.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.436+0000 7fa344ecd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c1b0cb0 con 0x55a4d569fdc0 2024-09-19T09:23:17.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.436+0000 7fa344ecd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c1b05e0 con 0x55a4d56aa680 2024-09-19T09:23:17.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.436+0000 7fa344ecd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c1ad800 con 0x55a4d569f4a0 2024-09-19T09:23:17.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.436+0000 7fa344ecd740 10 client.0 _ll_register_callbacks cb 0x7fa33001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:17.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.436+0000 7fa340890640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4d569f4a0 0x55a4d56b2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:17.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.436+0000 7fa33bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a4d569fdc0 0x55a4d56afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:17.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.436+0000 7fa340890640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4d569f4a0 0x55a4d56b2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:50760/0 (socket says 172.21.15.120:50760) 2024-09-19T09:23:17.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.436+0000 7fa340890640 1 -- 172.21.15.120:0/2997437456 learned_addr learned my addr 172.21.15.120:0/2997437456 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:17.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.436+0000 7fa341091640 1 --2- 172.21.15.120:0/2997437456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4d56aa680 0x55a4d56b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:17.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.437+0000 7fa340890640 1 -- 172.21.15.120:0/2997437456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4d56aa680 msgr2=0x55a4d56b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:17.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.437+0000 7fa340890640 1 --2- 172.21.15.120:0/2997437456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4d56aa680 0x55a4d56b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.437+0000 7fa340890640 1 -- 172.21.15.120:0/2997437456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a4d569fdc0 msgr2=0x55a4d56afef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:17.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.437+0000 7fa340890640 1 --2- 172.21.15.120:0/2997437456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a4d569fdc0 0x55a4d56afef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.437+0000 7fa340890640 1 -- 172.21.15.120:0/2997437456 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4d56b0a70 con 0x55a4d569f4a0 2024-09-19T09:23:17.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.437+0000 7fa340890640 1 --2- 172.21.15.120:0/2997437456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4d569f4a0 0x55a4d56b2ba0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa3300049c0 tx=0x7fa330019410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95066336d8a0374e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:17.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.437+0000 7fa31bfff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:23:17.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.437+0000 7fa31bfff640 1 -- 172.21.15.120:0/2997437456 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330015070 con 0x55a4d569f4a0 2024-09-19T09:23:17.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.437+0000 7fa31bfff640 1 -- 172.21.15.120:0/2997437456 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa330005000 con 0x55a4d569f4a0 2024-09-19T09:23:17.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.437+0000 7fa344ecd740 1 -- 172.21.15.120:0/2997437456 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa334044040 con 0x55a4d569f4a0 2024-09-19T09:23:17.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.438+0000 7fa31bfff640 1 -- 172.21.15.120:0/2997437456 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa33003d050 con 0x55a4d569f4a0 2024-09-19T09:23:17.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.438+0000 7fa344ecd740 10 client.30835 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:17.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.438+0000 7fa344ecd740 1 -- 172.21.15.120:0/2997437456 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a4d57248c0 con 0x55a4d569f4a0 2024-09-19T09:23:17.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.438+0000 7fa31bfff640 1 -- 172.21.15.120:0/2997437456 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa334044040 con 0x55a4d569f4a0 2024-09-19T09:23:17.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.438+0000 7fa31bfff640 1 -- 172.21.15.120:0/2997437456 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fa330021040 con 0x55a4d569f4a0 2024-09-19T09:23:17.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.438+0000 7fa344ecd740 10 client.30835 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:17.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.438+0000 7fa344ecd740 10 client.30835 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:23:17.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.438+0000 7fa344ecd740 20 client.30835 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:17.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.438+0000 7fa344ecd740 1 --2- 172.21.15.120:0/2997437456 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a4d56b9f70 0x55a4d5726f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:17.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.438+0000 7fa344ecd740 4 client.30835 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:23:17.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.438+0000 7fa344ecd740 1 -- 172.21.15.120:0/2997437456 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a4d56a7af0 con 0x55a4d56b9f70 2024-09-19T09:23:17.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.438+0000 7fa33bfff640 1 --2- 172.21.15.120:0/2997437456 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a4d56b9f70 0x55a4d5726f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:17.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.439+0000 7fa31bfff640 10 client.30835 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:17.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.439+0000 7fa33bfff640 1 --2- 172.21.15.120:0/2997437456 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a4d56b9f70 0x55a4d5726f30 crc :-1 s=READY pgs=512 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19e99e2a097a649 server_cookie=a8137d038e9da71b in_seq=0 out_seq=0 2024-09-19T09:23:17.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.440+0000 7fa31bfff640 1 -- 172.21.15.120:0/2997437456 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a4d56a7af0 con 0x55a4d56b9f70 2024-09-19T09:23:17.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.440+0000 7fa31bfff640 10 client.30835 handle_command_reply: tid=0 2024-09-19T09:23:17.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.440+0000 7fa344ecd740 1 client.30835 shutdown 2024-09-19T09:23:17.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.440+0000 7fa344ecd740 1 -- 172.21.15.120:0/2997437456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4d569f4a0 msgr2=0x55a4d56b2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:17.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.440+0000 7fa344ecd740 1 --2- 172.21.15.120:0/2997437456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4d569f4a0 0x55a4d56b2ba0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa3300049c0 tx=0x7fa330019410 comp rx=0 tx=0).stop 2024-09-19T09:23:17.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.440+0000 7fa344ecd740 1 -- 172.21.15.120:0/2997437456 shutdown_connections 2024-09-19T09:23:17.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.440+0000 7fa344ecd740 1 --2- 172.21.15.120:0/2997437456 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a4d56b9f70 0x55a4d5726f30 crc :-1 s=READY pgs=512 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.440+0000 7fa344ecd740 1 --2- 172.21.15.120:0/2997437456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4d56aa680 0x55a4d56b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.440+0000 7fa31bfff640 0 client.30835 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:17.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.440+0000 7fa344ecd740 1 --2- 172.21.15.120:0/2997437456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a4d569fdc0 0x55a4d56afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.440+0000 7fa344ecd740 1 --2- 172.21.15.120:0/2997437456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4d569f4a0 0x55a4d56b2ba0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.440+0000 7fa344ecd740 1 -- 172.21.15.120:0/2997437456 >> 172.21.15.120:0/2997437456 conn(0x55a4d569ba30 msgr2=0x55a4d56a8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:17.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.441+0000 7fa344ecd740 1 -- 172.21.15.120:0/2997437456 shutdown_connections 2024-09-19T09:23:17.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.441+0000 7fa344ecd740 1 -- 172.21.15.120:0/2997437456 wait complete. 2024-09-19T09:23:17.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.441+0000 7fa344ecd740 20 client.30835 trim_cache size 0 max 16384 2024-09-19T09:23:17.452 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:23:17.452 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:23:17.452 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-19T09:23:17.452 INFO:teuthology.orchestra.run.smithi120.stdout: "d9f36069-77c6-4eaa-8716-9bafd13e95a1": { 2024-09-19T09:23:17.452 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-19T09:23:17.452 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "d9f36069-77c6-4eaa-8716-9bafd13e95a1", 2024-09-19T09:23:17.452 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-19T09:23:17.452 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:23:17.453 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:23:17.453 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:23:17.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.442+0000 7fa342af4640 1 -- 172.21.15.120:0/2144414341 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa32c0a3240 msgr2=0x7fa32c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:17.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.442+0000 7fa342af4640 1 --2- 172.21.15.120:0/2144414341 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa32c0a3240 0x7fa32c0a5700 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa334004ad0 tx=0x7fa33402a950 comp rx=0 tx=0).stop 2024-09-19T09:23:17.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.442+0000 7fa342af4640 1 -- 172.21.15.120:0/2144414341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa33c103650 msgr2=0x7fa33c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:17.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.442+0000 7fa342af4640 1 --2- 172.21.15.120:0/2144414341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa33c103650 0x7fa33c100ed0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa32800e7a0 tx=0x7fa32800ec60 comp rx=0 tx=0).stop 2024-09-19T09:23:17.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.442+0000 7fa342af4640 1 -- 172.21.15.120:0/2144414341 shutdown_connections 2024-09-19T09:23:17.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.442+0000 7fa342af4640 1 --2- 172.21.15.120:0/2144414341 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa32c0a3240 0x7fa32c0a5700 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.442+0000 7fa342af4640 1 --2- 172.21.15.120:0/2144414341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa33c10c160 0x7fa33c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.442+0000 7fa342af4640 1 --2- 172.21.15.120:0/2144414341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa33c103650 0x7fa33c100ed0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.442+0000 7fa342af4640 1 --2- 172.21.15.120:0/2144414341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa33c102d30 0x7fa33c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:17.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.442+0000 7fa342af4640 1 -- 172.21.15.120:0/2144414341 >> 172.21.15.120:0/2144414341 conn(0x7fa33c0fc790 msgr2=0x7fa33c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:17.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.442+0000 7fa342af4640 1 -- 172.21.15.120:0/2144414341 shutdown_connections 2024-09-19T09:23:17.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:17.442+0000 7fa342af4640 1 -- 172.21.15.120:0/2144414341 wait complete. 2024-09-19T09:23:18.320 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:18 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:23:18.320 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:18 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:23:18.320 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:18 smithi153 ceph-mon[35071]: pgmap v958: 97 pgs: 97 active+clean; 431 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.9 MiB/s wr, 591 op/s 2024-09-19T09:23:18.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:18 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:23:18.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:18 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:23:18.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:18 smithi120 ceph-mon[31150]: pgmap v958: 97 pgs: 97 active+clean; 431 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.9 MiB/s wr, 591 op/s 2024-09-19T09:23:18.334 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:18 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:23:18.334 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:18 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:23:18.334 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:18 smithi196 ceph-mon[35150]: pgmap v958: 97 pgs: 97 active+clean; 431 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.9 MiB/s wr, 591 op/s 2024-09-19T09:23:18.376 INFO:tasks.workunit.client.0.smithi120.stdout: 10 174120 20.45 MB/sec execute 313 sec latency 54.120 ms 2024-09-19T09:23:18.648 INFO:tasks.cephfs.filesystem:scrub status for tag:d9f36069-77c6-4eaa-8716-9bafd13e95a1 - {'path': '/', 'tag': 'd9f36069-77c6-4eaa-8716-9bafd13e95a1', 'options': 'recursive,force'} 2024-09-19T09:23:19.376 INFO:tasks.workunit.client.0.smithi120.stdout: 10 174428 20.42 MB/sec execute 314 sec latency 446.361 ms 2024-09-19T09:23:20.377 INFO:tasks.workunit.client.0.smithi120.stdout: 10 174871 20.43 MB/sec execute 315 sec latency 287.663 ms 2024-09-19T09:23:20.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:20 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:23:20.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:20 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:23:20.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:20 smithi120 ceph-mon[31150]: pgmap v959: 97 pgs: 97 active+clean; 432 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.4 MiB/s wr, 599 op/s 2024-09-19T09:23:20.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:20 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:23:20.827 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:20 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:23:20.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:20 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:23:20.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:20 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:23:20.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:20 smithi196 ceph-mon[35150]: pgmap v959: 97 pgs: 97 active+clean; 432 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.4 MiB/s wr, 599 op/s 2024-09-19T09:23:20.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:20 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:23:20.938 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:20 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:23:20.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:20 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:23:20.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:20 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:23:20.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:20 smithi153 ceph-mon[35071]: pgmap v959: 97 pgs: 97 active+clean; 432 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.4 MiB/s wr, 599 op/s 2024-09-19T09:23:20.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:20 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:23:20.939 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:20 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:23:21.376 INFO:tasks.workunit.client.0.smithi120.stdout: 10 175436 20.44 MB/sec execute 316 sec latency 112.709 ms 2024-09-19T09:23:22.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:22 smithi120 ceph-mon[31150]: pgmap v960: 97 pgs: 97 active+clean; 421 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.8 MiB/s wr, 580 op/s 2024-09-19T09:23:22.376 INFO:tasks.workunit.client.0.smithi120.stdout: 10 176246 20.51 MB/sec execute 317 sec latency 67.550 ms 2024-09-19T09:23:22.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:22 smithi196 ceph-mon[35150]: pgmap v960: 97 pgs: 97 active+clean; 421 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.8 MiB/s wr, 580 op/s 2024-09-19T09:23:22.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:22 smithi153 ceph-mon[35071]: pgmap v960: 97 pgs: 97 active+clean; 421 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.8 MiB/s wr, 580 op/s 2024-09-19T09:23:23.376 INFO:tasks.workunit.client.0.smithi120.stdout: 10 176912 20.55 MB/sec execute 318 sec latency 152.337 ms 2024-09-19T09:23:24.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:24 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:23:24.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:24 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:23:24.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:24 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:23:24.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:24 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:23:24.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:24 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:23:24.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:24 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:23:24.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:24 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:24.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:24 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:23:24.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:24 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:23:24.377 INFO:tasks.workunit.client.0.smithi120.stdout: 10 177572 20.62 MB/sec execute 319 sec latency 101.158 ms 2024-09-19T09:23:24.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:24 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:23:24.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:24 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:23:24.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:24 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:23:24.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:24 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:23:24.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:24 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:23:24.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:24 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:23:24.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:24 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:24.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:24 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:23:24.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:24 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:23:24.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:24 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:23:24.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:24 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:23:24.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:24 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:23:24.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:24 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:23:24.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:24 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:23:24.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:24 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:23:24.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:24 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:24.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:24 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:23:24.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:24 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:23:25.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:25 smithi120 ceph-mon[31150]: pgmap v961: 97 pgs: 97 active+clean; 432 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.6 MiB/s wr, 676 op/s 2024-09-19T09:23:25.377 INFO:tasks.workunit.client.0.smithi120.stdout: 10 178136 20.69 MB/sec execute 320 sec latency 75.382 ms 2024-09-19T09:23:25.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:25 smithi196 ceph-mon[35150]: pgmap v961: 97 pgs: 97 active+clean; 432 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.6 MiB/s wr, 676 op/s 2024-09-19T09:23:25.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:25 smithi153 ceph-mon[35071]: pgmap v961: 97 pgs: 97 active+clean; 432 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.6 MiB/s wr, 676 op/s 2024-09-19T09:23:26.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:26 smithi120 ceph-mon[31150]: pgmap v962: 97 pgs: 97 active+clean; 434 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.7 MiB/s wr, 635 op/s 2024-09-19T09:23:26.377 INFO:tasks.workunit.client.0.smithi120.stdout: 10 178644 20.69 MB/sec execute 321 sec latency 60.509 ms 2024-09-19T09:23:26.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:26 smithi196 ceph-mon[35150]: pgmap v962: 97 pgs: 97 active+clean; 434 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.7 MiB/s wr, 635 op/s 2024-09-19T09:23:26.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:26 smithi153 ceph-mon[35071]: pgmap v962: 97 pgs: 97 active+clean; 434 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.7 MiB/s wr, 635 op/s 2024-09-19T09:23:27.377 INFO:tasks.workunit.client.0.smithi120.stdout: 10 179095 20.69 MB/sec execute 322 sec latency 70.857 ms 2024-09-19T09:23:28.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:28 smithi120 ceph-mon[31150]: pgmap v963: 97 pgs: 97 active+clean; 436 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.0 MiB/s wr, 628 op/s 2024-09-19T09:23:28.377 INFO:tasks.workunit.client.0.smithi120.stdout: 10 179661 20.70 MB/sec execute 323 sec latency 134.211 ms 2024-09-19T09:23:28.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:28 smithi196 ceph-mon[35150]: pgmap v963: 97 pgs: 97 active+clean; 436 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.0 MiB/s wr, 628 op/s 2024-09-19T09:23:28.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:28 smithi153 ceph-mon[35071]: pgmap v963: 97 pgs: 97 active+clean; 436 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.0 MiB/s wr, 628 op/s 2024-09-19T09:23:29.377 INFO:tasks.workunit.client.0.smithi120.stdout: 10 180152 20.73 MB/sec execute 324 sec latency 280.669 ms 2024-09-19T09:23:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.021+0000 7fd0a7f3b640 1 -- 172.21.15.120:0/1025859554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0a01025f0 msgr2=0x7fd0a0108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.021+0000 7fd0a7f3b640 1 --2- 172.21.15.120:0/1025859554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0a01025f0 0x7fd0a0108e80 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7fd098009ca0 tx=0x7fd09802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.021+0000 7fd0a7f3b640 1 -- 172.21.15.120:0/1025859554 shutdown_connections 2024-09-19T09:23:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.021+0000 7fd0a7f3b640 1 --2- 172.21.15.120:0/1025859554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0a01025f0 0x7fd0a0108e80 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.021+0000 7fd0a7f3b640 1 --2- 172.21.15.120:0/1025859554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0a0101c30 0x7fd0a01020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.021+0000 7fd0a7f3b640 1 --2- 172.21.15.120:0/1025859554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0a0100a40 0x7fd0a0100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.021+0000 7fd0a7f3b640 1 -- 172.21.15.120:0/1025859554 >> 172.21.15.120:0/1025859554 conn(0x7fd0a00fc1e0 msgr2=0x7fd0a00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.021+0000 7fd0a7f3b640 1 -- 172.21.15.120:0/1025859554 shutdown_connections 2024-09-19T09:23:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.021+0000 7fd0a7f3b640 1 -- 172.21.15.120:0/1025859554 wait complete. 2024-09-19T09:23:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.022+0000 7fd0a7f3b640 1 Processor -- start 2024-09-19T09:23:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.022+0000 7fd0a7f3b640 1 -- start start 2024-09-19T09:23:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.022+0000 7fd0a7f3b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0a0100a40 0x7fd0a006a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.022+0000 7fd0a7f3b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0a0101c30 0x7fd0a019cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.022+0000 7fd0a7f3b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0a01025f0 0x7fd0a019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.022+0000 7fd0a7f3b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a010c6c0 con 0x7fd0a0100a40 2024-09-19T09:23:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.022+0000 7fd0a7f3b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a010c540 con 0x7fd0a01025f0 2024-09-19T09:23:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.022+0000 7fd0a7f3b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a010c840 con 0x7fd0a0101c30 2024-09-19T09:23:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.022+0000 7fd0a5cd7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0a0100a40 0x7fd0a006a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.022+0000 7fd0a5cd7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0a0100a40 0x7fd0a006a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59722/0 (socket says 172.21.15.120:59722) 2024-09-19T09:23:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.022+0000 7fd0a5cd7640 1 -- 172.21.15.120:0/678202072 learned_addr learned my addr 172.21.15.120:0/678202072 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.022+0000 7fd0a54d6640 1 --2- 172.21.15.120:0/678202072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0a0101c30 0x7fd0a019cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.022+0000 7fd0a64d8640 1 --2- 172.21.15.120:0/678202072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0a01025f0 0x7fd0a019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.022+0000 7fd0a5cd7640 1 -- 172.21.15.120:0/678202072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0a0101c30 msgr2=0x7fd0a019cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.022+0000 7fd0a5cd7640 1 --2- 172.21.15.120:0/678202072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0a0101c30 0x7fd0a019cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.022+0000 7fd0a5cd7640 1 -- 172.21.15.120:0/678202072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0a01025f0 msgr2=0x7fd0a019f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.022+0000 7fd0a5cd7640 1 --2- 172.21.15.120:0/678202072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0a01025f0 0x7fd0a019f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.022+0000 7fd0a5cd7640 1 -- 172.21.15.120:0/678202072 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a019fa00 con 0x7fd0a0100a40 2024-09-19T09:23:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.022+0000 7fd0a5cd7640 1 --2- 172.21.15.120:0/678202072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0a0100a40 0x7fd0a006a410 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00e9c0 tx=0x7fd09c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.023+0000 7fd096ffd640 1 -- 172.21.15.120:0/678202072 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c00cd80 con 0x7fd0a0100a40 2024-09-19T09:23:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.023+0000 7fd0a7f3b640 1 -- 172.21.15.120:0/678202072 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a019fc90 con 0x7fd0a0100a40 2024-09-19T09:23:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.023+0000 7fd096ffd640 1 -- 172.21.15.120:0/678202072 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd09c0035f0 con 0x7fd0a0100a40 2024-09-19T09:23:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.023+0000 7fd096ffd640 1 -- 172.21.15.120:0/678202072 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c010690 con 0x7fd0a0100a40 2024-09-19T09:23:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.023+0000 7fd0a7f3b640 1 -- 172.21.15.120:0/678202072 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a01a0180 con 0x7fd0a0100a40 2024-09-19T09:23:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.025+0000 7fd096ffd640 1 -- 172.21.15.120:0/678202072 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd09c0037a0 con 0x7fd0a0100a40 2024-09-19T09:23:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.025+0000 7fd0aacd5740 1 Processor -- start 2024-09-19T09:23:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.026+0000 7fd0aacd5740 1 -- start start 2024-09-19T09:23:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.026+0000 7fd0aacd5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed955479a0 0x55ed95547da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.026+0000 7fd0aacd5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed955482e0 0x55ed9554cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.026+0000 7fd0aacd5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed9554d370 0x55ed9554f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.026+0000 7fd0aacd5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ed954ab2f0 con 0x55ed955482e0 2024-09-19T09:23:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.026+0000 7fd0aacd5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ed95453270 con 0x55ed9554d370 2024-09-19T09:23:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.026+0000 7fd0aacd5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ed954a72b0 con 0x55ed955479a0 2024-09-19T09:23:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.026+0000 7fd0a5cd7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed955482e0 0x55ed9554cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.026+0000 7fd0a64d8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed955479a0 0x55ed95547da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.026+0000 7fd0a54d6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed9554d370 0x55ed9554f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.026+0000 7fd0a64d8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed955479a0 0x55ed95547da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:42838/0 (socket says 172.21.15.120:42838) 2024-09-19T09:23:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.026+0000 7fd0a5cd7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed955482e0 0x55ed9554cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59726/0 (socket says 172.21.15.120:59726) 2024-09-19T09:23:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.026+0000 7fd0a64d8640 1 -- 172.21.15.120:0/2393624794 learned_addr learned my addr 172.21.15.120:0/2393624794 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.026+0000 7fd096ffd640 1 --2- 172.21.15.120:0/678202072 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd08c0a32f0 0x7fd08c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.026+0000 7fd0a54d6640 1 -- 172.21.15.120:0/2393624794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed955479a0 msgr2=0x55ed95547da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.026+0000 7fd0a54d6640 1 --2- 172.21.15.120:0/2393624794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed955479a0 0x55ed95547da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.026+0000 7fd0a54d6640 1 -- 172.21.15.120:0/2393624794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed955482e0 msgr2=0x55ed9554cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.026+0000 7fd0a54d6640 1 --2- 172.21.15.120:0/2393624794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed955482e0 0x55ed9554cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.026+0000 7fd096ffd640 1 -- 172.21.15.120:0/678202072 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd09c0cfce0 con 0x7fd0a0100a40 2024-09-19T09:23:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.026+0000 7fd0a54d6640 1 -- 172.21.15.120:0/2393624794 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed95452340 con 0x55ed9554d370 2024-09-19T09:23:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.027+0000 7fd0a64d8640 1 --2- 172.21.15.120:0/2393624794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed955479a0 0x55ed95547da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.027+0000 7fd0a64d8640 1 --2- 172.21.15.120:0/678202072 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd08c0a32f0 0x7fd08c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.027+0000 7fd0a54d6640 1 --2- 172.21.15.120:0/2393624794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed9554d370 0x55ed9554f830 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd090009bd0 tx=0x7fd090011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3770070e59ff31db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.027+0000 7fd0a64d8640 1 --2- 172.21.15.120:0/678202072 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd08c0a32f0 0x7fd08c0a57b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd098005e30 tx=0x7fd0980095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.028+0000 7fd08bfff640 1 -- 172.21.15.120:0/2393624794 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd090004230 con 0x55ed9554d370 2024-09-19T09:23:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.028+0000 7fd08bfff640 1 -- 172.21.15.120:0/2393624794 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd090009e40 con 0x55ed9554d370 2024-09-19T09:23:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.028+0000 7fd08bfff640 1 -- 172.21.15.120:0/2393624794 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd090004db0 con 0x55ed9554d370 2024-09-19T09:23:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.028+0000 7fd0aacd5740 1 -- 172.21.15.120:0/2393624794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed9554d370 msgr2=0x55ed9554f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.028+0000 7fd0aacd5740 1 --2- 172.21.15.120:0/2393624794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed9554d370 0x55ed9554f830 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd090009bd0 tx=0x7fd090011810 comp rx=0 tx=0).stop 2024-09-19T09:23:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.028+0000 7fd0aacd5740 1 -- 172.21.15.120:0/2393624794 shutdown_connections 2024-09-19T09:23:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.028+0000 7fd0aacd5740 1 --2- 172.21.15.120:0/2393624794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed9554d370 0x55ed9554f830 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.028+0000 7fd0aacd5740 1 --2- 172.21.15.120:0/2393624794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed955482e0 0x55ed9554cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.028+0000 7fd0aacd5740 1 --2- 172.21.15.120:0/2393624794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed955479a0 0x55ed95547da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.028+0000 7fd0aacd5740 1 -- 172.21.15.120:0/2393624794 >> 172.21.15.120:0/2393624794 conn(0x55ed95543010 msgr2=0x55ed95543410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.029+0000 7fd0aacd5740 1 -- 172.21.15.120:0/2393624794 shutdown_connections 2024-09-19T09:23:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.029+0000 7fd0aacd5740 1 -- 172.21.15.120:0/2393624794 wait complete. 2024-09-19T09:23:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.029+0000 7fd0aacd5740 1 -- start start 2024-09-19T09:23:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.029+0000 7fd0aacd5740 1 Processor -- start 2024-09-19T09:23:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.029+0000 7fd0aacd5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed955479a0 0x55ed95551bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.029+0000 7fd0aacd5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed955482e0 0x55ed95552100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.029+0000 7fd0aacd5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed9554d370 0x55ed95559140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.029+0000 7fd0aacd5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ed95550340 con 0x55ed9554d370 2024-09-19T09:23:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.029+0000 7fd0aacd5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ed95434290 con 0x55ed955482e0 2024-09-19T09:23:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.029+0000 7fd0aacd5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ed955504c0 con 0x55ed955479a0 2024-09-19T09:23:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.029+0000 7fd0aacd5740 10 client.0 _ll_register_callbacks cb 0x55ed95465140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.030+0000 7fd0a5cd7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed955479a0 0x55ed95551bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.030+0000 7fd0a64d8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed9554d370 0x55ed95559140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.030+0000 7fd0a5cd7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed955479a0 0x55ed95551bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:42842/0 (socket says 172.21.15.120:42842) 2024-09-19T09:23:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.030+0000 7fd0a5cd7640 1 -- 172.21.15.120:0/800434789 learned_addr learned my addr 172.21.15.120:0/800434789 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.030+0000 7fd0a54d6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed955482e0 0x55ed95552100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.030+0000 7fd0a5cd7640 1 -- 172.21.15.120:0/800434789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed955482e0 msgr2=0x55ed95552100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.030+0000 7fd0a5cd7640 1 --2- 172.21.15.120:0/800434789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed955482e0 0x55ed95552100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.030+0000 7fd0a5cd7640 1 -- 172.21.15.120:0/800434789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed9554d370 msgr2=0x55ed95559140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.030+0000 7fd0a5cd7640 1 --2- 172.21.15.120:0/800434789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed9554d370 0x55ed95559140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.030+0000 7fd0a5cd7640 1 -- 172.21.15.120:0/800434789 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed951be210 con 0x55ed955479a0 2024-09-19T09:23:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.030+0000 7fd0a64d8640 1 --2- 172.21.15.120:0/800434789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed9554d370 0x55ed95559140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.030+0000 7fd0a5cd7640 1 --2- 172.21.15.120:0/800434789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed955479a0 0x55ed95551bc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0058c0 tx=0x7fd09c09c1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1df1424f1cd3347 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.030+0000 7fd0897fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:23:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.030+0000 7fd0897fa640 1 -- 172.21.15.120:0/800434789 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c09c300 con 0x55ed955479a0 2024-09-19T09:23:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.030+0000 7fd0aacd5740 1 -- 172.21.15.120:0/800434789 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ed954b8a20 con 0x55ed955479a0 2024-09-19T09:23:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.030+0000 7fd0897fa640 1 -- 172.21.15.120:0/800434789 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd09c0bc0d0 con 0x55ed955479a0 2024-09-19T09:23:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.030+0000 7fd0897fa640 1 -- 172.21.15.120:0/800434789 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd09c0a06b0 con 0x55ed955479a0 2024-09-19T09:23:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.030+0000 7fd0aacd5740 10 client.30847 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.030+0000 7fd0aacd5740 1 -- 172.21.15.120:0/800434789 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ed9528a620 con 0x55ed955479a0 2024-09-19T09:23:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.031+0000 7fd0897fa640 1 -- 172.21.15.120:0/800434789 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ed954b8a20 con 0x55ed955479a0 2024-09-19T09:23:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.031+0000 7fd0897fa640 1 -- 172.21.15.120:0/800434789 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fd09c09b750 con 0x55ed955479a0 2024-09-19T09:23:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.031+0000 7fd0aacd5740 10 client.30847 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.031+0000 7fd0aacd5740 10 client.30847 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:23:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.031+0000 7fd0aacd5740 20 client.30847 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.031+0000 7fd0aacd5740 1 --2- 172.21.15.120:0/800434789 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ed9555d830 0x55ed9555fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.031+0000 7fd0aacd5740 4 client.30847 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:23:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.031+0000 7fd0aacd5740 1 -- 172.21.15.120:0/800434789 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed954b2d70 con 0x55ed9555d830 2024-09-19T09:23:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.031+0000 7fd0a54d6640 1 --2- 172.21.15.120:0/800434789 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ed9555d830 0x55ed9555fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.031+0000 7fd0a54d6640 1 --2- 172.21.15.120:0/800434789 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ed9555d830 0x55ed9555fc20 crc :-1 s=READY pgs=513 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dce68459505f8173 server_cookie=8e40fd236054b439 in_seq=0 out_seq=0 2024-09-19T09:23:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.032+0000 7fd0897fa640 10 client.30847 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.035+0000 7fd0897fa640 1 -- 172.21.15.120:0/800434789 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed954b2d70 con 0x55ed9555d830 2024-09-19T09:23:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.035+0000 7fd0897fa640 10 client.30847 handle_command_reply: tid=0 2024-09-19T09:23:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.035+0000 7fd0aacd5740 1 client.30847 shutdown 2024-09-19T09:23:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.035+0000 7fd0aacd5740 1 -- 172.21.15.120:0/800434789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed955479a0 msgr2=0x55ed95551bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.035+0000 7fd0aacd5740 1 --2- 172.21.15.120:0/800434789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed955479a0 0x55ed95551bc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0058c0 tx=0x7fd09c09c1f0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.035+0000 7fd0aacd5740 1 -- 172.21.15.120:0/800434789 shutdown_connections 2024-09-19T09:23:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.035+0000 7fd0aacd5740 1 --2- 172.21.15.120:0/800434789 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ed9555d830 0x55ed9555fc20 crc :-1 s=READY pgs=513 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.035+0000 7fd0aacd5740 1 --2- 172.21.15.120:0/800434789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed9554d370 0x55ed95559140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.035+0000 7fd0aacd5740 1 --2- 172.21.15.120:0/800434789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed955482e0 0x55ed95552100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.035+0000 7fd0aacd5740 1 --2- 172.21.15.120:0/800434789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed955479a0 0x55ed95551bc0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.035+0000 7fd0aacd5740 1 -- 172.21.15.120:0/800434789 >> 172.21.15.120:0/800434789 conn(0x55ed95543010 msgr2=0x55ed95544820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.035+0000 7fd0897fa640 0 client.30847 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.035+0000 7fd0aacd5740 1 -- 172.21.15.120:0/800434789 shutdown_connections 2024-09-19T09:23:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.035+0000 7fd0aacd5740 1 -- 172.21.15.120:0/800434789 wait complete. 2024-09-19T09:23:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.035+0000 7fd0aacd5740 20 client.30847 trim_cache size 0 max 16384 2024-09-19T09:23:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.054+0000 7fd0aacd5740 1 Processor -- start 2024-09-19T09:23:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.054+0000 7fd0aacd5740 1 -- start start 2024-09-19T09:23:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.054+0000 7fd0aacd5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed955479a0 0x55ed95552080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.054+0000 7fd0aacd5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed955482e0 0x55ed95559100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.054+0000 7fd0aacd5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed9554d370 0x55ed955508c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.054+0000 7fd0aacd5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a01a8f20 con 0x55ed955479a0 2024-09-19T09:23:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.054+0000 7fd0aacd5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a01aaa60 con 0x55ed9554d370 2024-09-19T09:23:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.054+0000 7fd0aacd5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a01a95f0 con 0x55ed955482e0 2024-09-19T09:23:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.054+0000 7fd0a5cd7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed955479a0 0x55ed95552080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.054+0000 7fd0a5cd7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed955479a0 0x55ed95552080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59752/0 (socket says 172.21.15.120:59752) 2024-09-19T09:23:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.054+0000 7fd0a5cd7640 1 -- 172.21.15.120:0/3995246928 learned_addr learned my addr 172.21.15.120:0/3995246928 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.055+0000 7fd0a64d8640 1 --2- 172.21.15.120:0/3995246928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed9554d370 0x55ed955508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.055+0000 7fd0a54d6640 1 --2- 172.21.15.120:0/3995246928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed955482e0 0x55ed95559100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.055+0000 7fd0a5cd7640 1 -- 172.21.15.120:0/3995246928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed955482e0 msgr2=0x55ed95559100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.055+0000 7fd0a5cd7640 1 --2- 172.21.15.120:0/3995246928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed955482e0 0x55ed95559100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.055+0000 7fd0a5cd7640 1 -- 172.21.15.120:0/3995246928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed9554d370 msgr2=0x55ed955508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.055+0000 7fd0a5cd7640 1 --2- 172.21.15.120:0/3995246928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed9554d370 0x55ed955508c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.055+0000 7fd0a5cd7640 1 -- 172.21.15.120:0/3995246928 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed955487c0 con 0x55ed955479a0 2024-09-19T09:23:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.055+0000 7fd0a64d8640 1 --2- 172.21.15.120:0/3995246928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed9554d370 0x55ed955508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.055+0000 7fd0a5cd7640 1 --2- 172.21.15.120:0/3995246928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed955479a0 0x55ed95552080 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00c6a0 tx=0x7fd09c0cfff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d6b42982441a42 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.057+0000 7fd08b7fe640 1 -- 172.21.15.120:0/3995246928 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c065070 con 0x55ed955479a0 2024-09-19T09:23:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.057+0000 7fd08b7fe640 1 -- 172.21.15.120:0/3995246928 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd09c0d2050 con 0x55ed955479a0 2024-09-19T09:23:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.057+0000 7fd08b7fe640 1 -- 172.21.15.120:0/3995246928 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c09b750 con 0x55ed955479a0 2024-09-19T09:23:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.057+0000 7fd0aacd5740 1 -- 172.21.15.120:0/3995246928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed955479a0 msgr2=0x55ed95552080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.057+0000 7fd0aacd5740 1 --2- 172.21.15.120:0/3995246928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed955479a0 0x55ed95552080 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00c6a0 tx=0x7fd09c0cfff0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.057+0000 7fd0aacd5740 1 -- 172.21.15.120:0/3995246928 shutdown_connections 2024-09-19T09:23:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.057+0000 7fd0aacd5740 1 --2- 172.21.15.120:0/3995246928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed9554d370 0x55ed955508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.057+0000 7fd0aacd5740 1 --2- 172.21.15.120:0/3995246928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed955482e0 0x55ed95559100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.057+0000 7fd0aacd5740 1 --2- 172.21.15.120:0/3995246928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed955479a0 0x55ed95552080 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.057+0000 7fd0aacd5740 1 -- 172.21.15.120:0/3995246928 >> 172.21.15.120:0/3995246928 conn(0x55ed95543010 msgr2=0x55ed95551970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.057+0000 7fd0aacd5740 1 -- 172.21.15.120:0/3995246928 shutdown_connections 2024-09-19T09:23:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.057+0000 7fd0aacd5740 1 -- 172.21.15.120:0/3995246928 wait complete. 2024-09-19T09:23:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.058+0000 7fd0aacd5740 1 -- start start 2024-09-19T09:23:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.058+0000 7fd0aacd5740 1 Processor -- start 2024-09-19T09:23:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.058+0000 7fd0aacd5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed955479a0 0x55ed95556070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.058+0000 7fd0aacd5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed955482e0 0x55ed955565b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.058+0000 7fd0aacd5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed9554d370 0x55ed955c6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.058+0000 7fd0aacd5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a01aa390 con 0x55ed9554d370 2024-09-19T09:23:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.058+0000 7fd0aacd5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a01a9cc0 con 0x55ed955482e0 2024-09-19T09:23:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.058+0000 7fd0aacd5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a01a6ee0 con 0x55ed955479a0 2024-09-19T09:23:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.058+0000 7fd0aacd5740 10 client.0 _ll_register_callbacks cb 0x7fd09c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.058+0000 7fd0a5cd7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed955479a0 0x55ed95556070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.058+0000 7fd0a64d8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed9554d370 0x55ed955c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.058+0000 7fd0a5cd7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed955479a0 0x55ed95556070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:42852/0 (socket says 172.21.15.120:42852) 2024-09-19T09:23:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.058+0000 7fd0a54d6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed955482e0 0x55ed955565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.058+0000 7fd0a5cd7640 1 -- 172.21.15.120:0/28649047 learned_addr learned my addr 172.21.15.120:0/28649047 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.059+0000 7fd0a5cd7640 1 -- 172.21.15.120:0/28649047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed955482e0 msgr2=0x55ed955565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.059+0000 7fd0a5cd7640 1 --2- 172.21.15.120:0/28649047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed955482e0 0x55ed955565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.059+0000 7fd0a5cd7640 1 -- 172.21.15.120:0/28649047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed9554d370 msgr2=0x55ed955c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.059+0000 7fd0a5cd7640 1 --2- 172.21.15.120:0/28649047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed9554d370 0x55ed955c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.059+0000 7fd0a5cd7640 1 -- 172.21.15.120:0/28649047 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed955c6ab0 con 0x55ed955479a0 2024-09-19T09:23:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.059+0000 7fd0a64d8640 1 --2- 172.21.15.120:0/28649047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed9554d370 0x55ed955c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.059+0000 7fd0a5cd7640 1 --2- 172.21.15.120:0/28649047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed955479a0 0x55ed95556070 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0d33c0 tx=0x7fd09c0b5380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=107d523ee55055c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.059+0000 7fd0897fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:23:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.059+0000 7fd0897fa640 1 -- 172.21.15.120:0/28649047 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c065070 con 0x55ed955479a0 2024-09-19T09:23:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.059+0000 7fd0aacd5740 1 -- 172.21.15.120:0/28649047 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd09c0d2050 con 0x55ed955479a0 2024-09-19T09:23:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.059+0000 7fd0897fa640 1 -- 172.21.15.120:0/28649047 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd09c0bc2d0 con 0x55ed955479a0 2024-09-19T09:23:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.059+0000 7fd0897fa640 1 -- 172.21.15.120:0/28649047 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd09c09c300 con 0x55ed955479a0 2024-09-19T09:23:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.059+0000 7fd0aacd5740 10 client.30853 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.059+0000 7fd0aacd5740 1 -- 172.21.15.120:0/28649047 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ed955c7da0 con 0x55ed955479a0 2024-09-19T09:23:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.059+0000 7fd0897fa640 1 -- 172.21.15.120:0/28649047 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd09c0d2050 con 0x55ed955479a0 2024-09-19T09:23:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.060+0000 7fd0897fa640 1 -- 172.21.15.120:0/28649047 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fd09c083050 con 0x55ed955479a0 2024-09-19T09:23:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.060+0000 7fd0aacd5740 10 client.30853 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.060+0000 7fd0aacd5740 10 client.30853 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:23:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.060+0000 7fd0aacd5740 20 client.30853 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.060+0000 7fd0aacd5740 1 --2- 172.21.15.120:0/28649047 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ed955c8480 0x55ed955ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.060+0000 7fd0aacd5740 4 client.30853 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:23:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.060+0000 7fd0aacd5740 1 -- 172.21.15.120:0/28649047 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ed9554fc40 con 0x55ed955c8480 2024-09-19T09:23:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.060+0000 7fd0a54d6640 1 --2- 172.21.15.120:0/28649047 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ed955c8480 0x55ed955ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.060+0000 7fd0a54d6640 1 --2- 172.21.15.120:0/28649047 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ed955c8480 0x55ed955ca8a0 crc :-1 s=READY pgs=514 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d640e5ab245171b server_cookie=18aa8ef66876551 in_seq=0 out_seq=0 2024-09-19T09:23:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.060+0000 7fd0897fa640 10 client.30853 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:30.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:30 smithi120 ceph-mon[31150]: pgmap v964: 97 pgs: 97 active+clean; 444 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.7 MiB/s wr, 559 op/s 2024-09-19T09:23:30.377 INFO:tasks.workunit.client.0.smithi120.stdout: 10 180630 20.72 MB/sec execute 325 sec latency 232.708 ms 2024-09-19T09:23:30.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:30 smithi196 ceph-mon[35150]: pgmap v964: 97 pgs: 97 active+clean; 444 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.7 MiB/s wr, 559 op/s 2024-09-19T09:23:30.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:30 smithi153 ceph-mon[35071]: pgmap v964: 97 pgs: 97 active+clean; 444 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.7 MiB/s wr, 559 op/s 2024-09-19T09:23:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.622+0000 7fd0897fa640 1 -- 172.21.15.120:0/28649047 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ed9554fc40 con 0x55ed955c8480 2024-09-19T09:23:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.622+0000 7fd0897fa640 10 client.30853 handle_command_reply: tid=0 2024-09-19T09:23:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.623+0000 7fd0aacd5740 1 client.30853 shutdown 2024-09-19T09:23:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.623+0000 7fd0aacd5740 1 -- 172.21.15.120:0/28649047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed955479a0 msgr2=0x55ed95556070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.623+0000 7fd0aacd5740 1 --2- 172.21.15.120:0/28649047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed955479a0 0x55ed95556070 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0d33c0 tx=0x7fd09c0b5380 comp rx=0 tx=0).stop 2024-09-19T09:23:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.623+0000 7fd0aacd5740 1 -- 172.21.15.120:0/28649047 shutdown_connections 2024-09-19T09:23:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.623+0000 7fd0aacd5740 1 --2- 172.21.15.120:0/28649047 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ed955c8480 0x55ed955ca8a0 crc :-1 s=READY pgs=514 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.623+0000 7fd0aacd5740 1 --2- 172.21.15.120:0/28649047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed9554d370 0x55ed955c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.623+0000 7fd0897fa640 0 client.30853 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.623+0000 7fd0aacd5740 1 --2- 172.21.15.120:0/28649047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed955482e0 0x55ed955565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.623+0000 7fd0aacd5740 1 --2- 172.21.15.120:0/28649047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed955479a0 0x55ed95556070 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.623+0000 7fd0aacd5740 1 -- 172.21.15.120:0/28649047 >> 172.21.15.120:0/28649047 conn(0x55ed95543010 msgr2=0x55ed955419a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.623+0000 7fd0aacd5740 1 -- 172.21.15.120:0/28649047 shutdown_connections 2024-09-19T09:23:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.623+0000 7fd0aacd5740 1 -- 172.21.15.120:0/28649047 wait complete. 2024-09-19T09:23:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.623+0000 7fd0aacd5740 20 client.30853 trim_cache size 0 max 16384 2024-09-19T09:23:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:23:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:23:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:23:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:23:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.624+0000 7fd0a7f3b640 1 -- 172.21.15.120:0/678202072 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd08c0a32f0 msgr2=0x7fd08c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.624+0000 7fd0a7f3b640 1 --2- 172.21.15.120:0/678202072 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd08c0a32f0 0x7fd08c0a57b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd098005e30 tx=0x7fd0980095c0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.624+0000 7fd0a7f3b640 1 -- 172.21.15.120:0/678202072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0a0100a40 msgr2=0x7fd0a006a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.624+0000 7fd0a7f3b640 1 --2- 172.21.15.120:0/678202072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0a0100a40 0x7fd0a006a410 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00e9c0 tx=0x7fd09c00ee80 comp rx=0 tx=0).stop 2024-09-19T09:23:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.624+0000 7fd0a7f3b640 1 -- 172.21.15.120:0/678202072 shutdown_connections 2024-09-19T09:23:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.624+0000 7fd0a7f3b640 1 --2- 172.21.15.120:0/678202072 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd08c0a32f0 0x7fd08c0a57b0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.624+0000 7fd0a7f3b640 1 --2- 172.21.15.120:0/678202072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0a01025f0 0x7fd0a019f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.625+0000 7fd0a7f3b640 1 --2- 172.21.15.120:0/678202072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0a0101c30 0x7fd0a019cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.625+0000 7fd0a7f3b640 1 --2- 172.21.15.120:0/678202072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0a0100a40 0x7fd0a006a410 unknown :-1 s=CLOSED pgs=2181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.625+0000 7fd0a7f3b640 1 -- 172.21.15.120:0/678202072 >> 172.21.15.120:0/678202072 conn(0x7fd0a00fc1e0 msgr2=0x7fd0a0104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.625+0000 7fd0a7f3b640 1 -- 172.21.15.120:0/678202072 shutdown_connections 2024-09-19T09:23:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:30.625+0000 7fd0a7f3b640 1 -- 172.21.15.120:0/678202072 wait complete. 2024-09-19T09:23:31.377 INFO:tasks.workunit.client.0.smithi120.stdout: 10 181281 20.76 MB/sec execute 326 sec latency 398.561 ms 2024-09-19T09:23:32.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:32 smithi120 ceph-mon[31150]: pgmap v965: 97 pgs: 97 active+clean; 443 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.8 MiB/s wr, 542 op/s 2024-09-19T09:23:32.377 INFO:tasks.workunit.client.0.smithi120.stdout: 10 181893 20.82 MB/sec execute 327 sec latency 164.146 ms 2024-09-19T09:23:32.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:32 smithi196 ceph-mon[35150]: pgmap v965: 97 pgs: 97 active+clean; 443 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.8 MiB/s wr, 542 op/s 2024-09-19T09:23:32.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:32 smithi153 ceph-mon[35071]: pgmap v965: 97 pgs: 97 active+clean; 443 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.8 MiB/s wr, 542 op/s 2024-09-19T09:23:33.377 INFO:tasks.workunit.client.0.smithi120.stdout: 10 182505 20.90 MB/sec execute 328 sec latency 65.728 ms 2024-09-19T09:23:34.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:34 smithi120 ceph-mon[31150]: pgmap v966: 97 pgs: 97 active+clean; 434 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.2 MiB/s wr, 691 op/s 2024-09-19T09:23:34.377 INFO:tasks.workunit.client.0.smithi120.stdout: 10 182848 20.89 MB/sec execute 329 sec latency 265.548 ms 2024-09-19T09:23:34.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:34 smithi196 ceph-mon[35150]: pgmap v966: 97 pgs: 97 active+clean; 434 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.2 MiB/s wr, 691 op/s 2024-09-19T09:23:34.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:34 smithi153 ceph-mon[35071]: pgmap v966: 97 pgs: 97 active+clean; 434 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.2 MiB/s wr, 691 op/s 2024-09-19T09:23:35.377 INFO:tasks.workunit.client.0.smithi120.stdout: 10 183270 20.87 MB/sec execute 330 sec latency 100.744 ms 2024-09-19T09:23:36.327 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:36 smithi153 ceph-mon[35071]: pgmap v967: 97 pgs: 97 active+clean; 438 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.7 MiB/s wr, 617 op/s 2024-09-19T09:23:36.341 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:36 smithi120 ceph-mon[31150]: pgmap v967: 97 pgs: 97 active+clean; 438 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.7 MiB/s wr, 617 op/s 2024-09-19T09:23:36.395 INFO:tasks.workunit.client.0.smithi120.stdout: 10 183646 20.86 MB/sec execute 331 sec latency 151.275 ms 2024-09-19T09:23:36.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:36 smithi196 ceph-mon[35150]: pgmap v967: 97 pgs: 97 active+clean; 438 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.7 MiB/s wr, 617 op/s 2024-09-19T09:23:37.377 INFO:tasks.workunit.client.0.smithi120.stdout: 10 184259 20.90 MB/sec execute 332 sec latency 180.991 ms 2024-09-19T09:23:38.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:38 smithi120 ceph-mon[31150]: pgmap v968: 97 pgs: 97 active+clean; 445 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.6 MiB/s wr, 704 op/s 2024-09-19T09:23:38.377 INFO:tasks.workunit.client.0.smithi120.stdout: 10 184967 20.92 MB/sec execute 333 sec latency 176.873 ms 2024-09-19T09:23:38.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:38 smithi196 ceph-mon[35150]: pgmap v968: 97 pgs: 97 active+clean; 445 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.6 MiB/s wr, 704 op/s 2024-09-19T09:23:38.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:38 smithi153 ceph-mon[35071]: pgmap v968: 97 pgs: 97 active+clean; 445 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.6 MiB/s wr, 704 op/s 2024-09-19T09:23:39.377 INFO:tasks.workunit.client.0.smithi120.stdout: 10 185433 20.94 MB/sec execute 334 sec latency 228.775 ms 2024-09-19T09:23:40.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:40 smithi120 ceph-mon[31150]: pgmap v969: 97 pgs: 97 active+clean; 446 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.7 MiB/s wr, 762 op/s 2024-09-19T09:23:40.377 INFO:tasks.workunit.client.0.smithi120.stdout: 10 186083 21.00 MB/sec execute 335 sec latency 327.528 ms 2024-09-19T09:23:40.426 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:40 smithi196 ceph-mon[35150]: pgmap v969: 97 pgs: 97 active+clean; 446 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.7 MiB/s wr, 762 op/s 2024-09-19T09:23:40.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:40 smithi153 ceph-mon[35071]: pgmap v969: 97 pgs: 97 active+clean; 446 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.7 MiB/s wr, 762 op/s 2024-09-19T09:23:41.377 INFO:tasks.workunit.client.0.smithi120.stdout: 10 186691 21.07 MB/sec execute 336 sec latency 65.781 ms 2024-09-19T09:23:42.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:42 smithi120 ceph-mon[31150]: pgmap v970: 97 pgs: 97 active+clean; 446 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.6 MiB/s wr, 706 op/s 2024-09-19T09:23:42.377 INFO:tasks.workunit.client.0.smithi120.stdout: 10 187166 21.08 MB/sec execute 337 sec latency 48.982 ms 2024-09-19T09:23:42.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:42 smithi196 ceph-mon[35150]: pgmap v970: 97 pgs: 97 active+clean; 446 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.6 MiB/s wr, 706 op/s 2024-09-19T09:23:42.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:42 smithi153 ceph-mon[35071]: pgmap v970: 97 pgs: 97 active+clean; 446 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.6 MiB/s wr, 706 op/s 2024-09-19T09:23:43.377 INFO:tasks.workunit.client.0.smithi120.stdout: 10 187670 21.09 MB/sec execute 338 sec latency 59.462 ms 2024-09-19T09:23:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.742+0000 7fd068d9f640 1 -- 172.21.15.120:0/3076192008 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd064101f40 msgr2=0x7fd064109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.742+0000 7fd068d9f640 1 --2- 172.21.15.120:0/3076192008 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd064101f40 0x7fd064109db0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009ca0 tx=0x7fd05c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:23:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.743+0000 7fd068d9f640 1 -- 172.21.15.120:0/3076192008 shutdown_connections 2024-09-19T09:23:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.743+0000 7fd068d9f640 1 --2- 172.21.15.120:0/3076192008 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd064101f40 0x7fd064109db0 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.743+0000 7fd068d9f640 1 --2- 172.21.15.120:0/3076192008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd064101580 0x7fd064101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.743+0000 7fd068d9f640 1 --2- 172.21.15.120:0/3076192008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd064103050 0x7fd064103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.743+0000 7fd068d9f640 1 -- 172.21.15.120:0/3076192008 >> 172.21.15.120:0/3076192008 conn(0x7fd0640fd0d0 msgr2=0x7fd0640ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.743+0000 7fd068d9f640 1 -- 172.21.15.120:0/3076192008 shutdown_connections 2024-09-19T09:23:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.743+0000 7fd068d9f640 1 -- 172.21.15.120:0/3076192008 wait complete. 2024-09-19T09:23:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.743+0000 7fd068d9f640 1 Processor -- start 2024-09-19T09:23:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.744+0000 7fd068d9f640 1 -- start start 2024-09-19T09:23:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.744+0000 7fd068d9f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd064101580 0x7fd064199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.744+0000 7fd068d9f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd064101f40 0x7fd06419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.744+0000 7fd068d9f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd064103050 0x7fd0641a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.744+0000 7fd068d9f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd064074d20 con 0x7fd064101580 2024-09-19T09:23:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.744+0000 7fd068d9f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd064074ba0 con 0x7fd064103050 2024-09-19T09:23:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.744+0000 7fd068d9f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd064074ea0 con 0x7fd064101f40 2024-09-19T09:23:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.744+0000 7fd0637fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd064101580 0x7fd064199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.744+0000 7fd0637fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd064101580 0x7fd064199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50164/0 (socket says 172.21.15.120:50164) 2024-09-19T09:23:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.744+0000 7fd0637fe640 1 -- 172.21.15.120:0/2157246968 learned_addr learned my addr 172.21.15.120:0/2157246968 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.744+0000 7fd062ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd064101f40 0x7fd06419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.744+0000 7fd063fff640 1 --2- 172.21.15.120:0/2157246968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd064103050 0x7fd0641a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.744+0000 7fd0637fe640 1 -- 172.21.15.120:0/2157246968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd064101f40 msgr2=0x7fd06419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.744+0000 7fd0637fe640 1 --2- 172.21.15.120:0/2157246968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd064101f40 0x7fd06419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.744+0000 7fd0637fe640 1 -- 172.21.15.120:0/2157246968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd064103050 msgr2=0x7fd0641a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.744+0000 7fd0637fe640 1 --2- 172.21.15.120:0/2157246968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd064103050 0x7fd0641a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.744+0000 7fd0637fe640 1 -- 172.21.15.120:0/2157246968 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0641a1b80 con 0x7fd064101580 2024-09-19T09:23:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.744+0000 7fd062ffd640 1 --2- 172.21.15.120:0/2157246968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd064101f40 0x7fd06419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.745+0000 7fd0637fe640 1 --2- 172.21.15.120:0/2157246968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd064101580 0x7fd064199f60 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7fd05400e9c0 tx=0x7fd05400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.745+0000 7fd060ff9640 1 -- 172.21.15.120:0/2157246968 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05400cd80 con 0x7fd064101580 2024-09-19T09:23:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.745+0000 7fd068d9f640 1 -- 172.21.15.120:0/2157246968 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0641a1e70 con 0x7fd064101580 2024-09-19T09:23:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.745+0000 7fd060ff9640 1 -- 172.21.15.120:0/2157246968 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0540035f0 con 0x7fd064101580 2024-09-19T09:23:43.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.745+0000 7fd060ff9640 1 -- 172.21.15.120:0/2157246968 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054010690 con 0x7fd064101580 2024-09-19T09:23:43.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.745+0000 7fd068d9f640 1 -- 172.21.15.120:0/2157246968 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0641a22d0 con 0x7fd064101580 2024-09-19T09:23:43.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.748+0000 7fd060ff9640 1 -- 172.21.15.120:0/2157246968 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd0540037a0 con 0x7fd064101580 2024-09-19T09:23:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.749+0000 7fd060ff9640 1 --2- 172.21.15.120:0/2157246968 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd04c0a3340 0x7fd04c0a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.749+0000 7fd062ffd640 1 --2- 172.21.15.120:0/2157246968 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd04c0a3340 0x7fd04c0a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.749+0000 7fd060ff9640 1 -- 172.21.15.120:0/2157246968 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd0540cfc50 con 0x7fd064101580 2024-09-19T09:23:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.749+0000 7fd06ce73740 1 Processor -- start 2024-09-19T09:23:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.750+0000 7fd06ce73740 1 -- start start 2024-09-19T09:23:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.750+0000 7fd062ffd640 1 --2- 172.21.15.120:0/2157246968 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd04c0a3340 0x7fd04c0a5800 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd06419b470 tx=0x7fd058005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.750+0000 7fd06ce73740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55782934ab70 0x55782934af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.750+0000 7fd06ce73740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55782934b4b0 0x55782934ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.750+0000 7fd06ce73740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557829350540 0x557829352a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.750+0000 7fd06ce73740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5578292ae2f0 con 0x557829350540 2024-09-19T09:23:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.750+0000 7fd06ce73740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557829256270 con 0x55782934b4b0 2024-09-19T09:23:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.750+0000 7fd06ce73740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5578292aa2b0 con 0x55782934ab70 2024-09-19T09:23:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.751+0000 7fd063fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557829350540 0x557829352a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.751+0000 7fd063fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557829350540 0x557829352a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50172/0 (socket says 172.21.15.120:50172) 2024-09-19T09:23:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.751+0000 7fd063fff640 1 -- 172.21.15.120:0/3800405277 learned_addr learned my addr 172.21.15.120:0/3800405277 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.751+0000 7fd063fff640 1 -- 172.21.15.120:0/3800405277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55782934ab70 msgr2=0x55782934af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.751+0000 7fd063fff640 1 --2- 172.21.15.120:0/3800405277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55782934ab70 0x55782934af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.751+0000 7fd063fff640 1 -- 172.21.15.120:0/3800405277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55782934b4b0 msgr2=0x55782934ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.751+0000 7fd063fff640 1 --2- 172.21.15.120:0/3800405277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55782934b4b0 0x55782934ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.751+0000 7fd063fff640 1 -- 172.21.15.120:0/3800405277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557829255340 con 0x557829350540 2024-09-19T09:23:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.751+0000 7fd063fff640 1 --2- 172.21.15.120:0/3800405277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557829350540 0x557829352a00 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009ca0 tx=0x7fd05c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b88e6ccc3af6538 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.751+0000 7fd049ffb640 1 -- 172.21.15.120:0/3800405277 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c0098c0 con 0x557829350540 2024-09-19T09:23:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.751+0000 7fd049ffb640 1 -- 172.21.15.120:0/3800405277 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd05c036040 con 0x557829350540 2024-09-19T09:23:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.751+0000 7fd049ffb640 1 -- 172.21.15.120:0/3800405277 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c004290 con 0x557829350540 2024-09-19T09:23:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.752+0000 7fd06ce73740 1 -- 172.21.15.120:0/3800405277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557829350540 msgr2=0x557829352a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.752+0000 7fd06ce73740 1 --2- 172.21.15.120:0/3800405277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557829350540 0x557829352a00 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009ca0 tx=0x7fd05c004a40 comp rx=0 tx=0).stop 2024-09-19T09:23:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.752+0000 7fd06ce73740 1 -- 172.21.15.120:0/3800405277 shutdown_connections 2024-09-19T09:23:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.752+0000 7fd06ce73740 1 --2- 172.21.15.120:0/3800405277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557829350540 0x557829352a00 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.752+0000 7fd06ce73740 1 --2- 172.21.15.120:0/3800405277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55782934b4b0 0x55782934ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.752+0000 7fd06ce73740 1 --2- 172.21.15.120:0/3800405277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55782934ab70 0x55782934af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.752+0000 7fd06ce73740 1 -- 172.21.15.120:0/3800405277 >> 172.21.15.120:0/3800405277 conn(0x5578293461e0 msgr2=0x5578293465e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.752+0000 7fd06ce73740 1 -- 172.21.15.120:0/3800405277 shutdown_connections 2024-09-19T09:23:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.753+0000 7fd06ce73740 1 -- 172.21.15.120:0/3800405277 wait complete. 2024-09-19T09:23:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.753+0000 7fd06ce73740 1 -- start start 2024-09-19T09:23:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.753+0000 7fd06ce73740 1 Processor -- start 2024-09-19T09:23:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.753+0000 7fd06ce73740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55782934ab70 0x557829354c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.753+0000 7fd06ce73740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55782934b4b0 0x557829355190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.753+0000 7fd06ce73740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557829350540 0x55782935c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.753+0000 7fd06ce73740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5578293532d0 con 0x55782934ab70 2024-09-19T09:23:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.753+0000 7fd06ce73740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557829237290 con 0x55782934b4b0 2024-09-19T09:23:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.753+0000 7fd06ce73740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557829353450 con 0x557829350540 2024-09-19T09:23:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.753+0000 7fd06ce73740 10 client.0 _ll_register_callbacks cb 0x557829268140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.753+0000 7fd0637fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55782934ab70 0x557829354c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.754+0000 7fd0637fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55782934ab70 0x557829354c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50178/0 (socket says 172.21.15.120:50178) 2024-09-19T09:23:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.754+0000 7fd0637fe640 1 -- 172.21.15.120:0/1620162430 learned_addr learned my addr 172.21.15.120:0/1620162430 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.754+0000 7fd063fff640 1 --2- 172.21.15.120:0/1620162430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557829350540 0x55782935c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.754+0000 7fd062ffd640 1 --2- 172.21.15.120:0/1620162430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55782934b4b0 0x557829355190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.754+0000 7fd0637fe640 1 -- 172.21.15.120:0/1620162430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557829350540 msgr2=0x55782935c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.754+0000 7fd0637fe640 1 --2- 172.21.15.120:0/1620162430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557829350540 0x55782935c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.754+0000 7fd0637fe640 1 -- 172.21.15.120:0/1620162430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55782934b4b0 msgr2=0x557829355190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.754+0000 7fd0637fe640 1 --2- 172.21.15.120:0/1620162430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55782934b4b0 0x557829355190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.754+0000 7fd0637fe640 1 -- 172.21.15.120:0/1620162430 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557828fc1210 con 0x55782934ab70 2024-09-19T09:23:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.754+0000 7fd062ffd640 1 --2- 172.21.15.120:0/1620162430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55782934b4b0 0x557829355190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:23:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.754+0000 7fd0637fe640 1 --2- 172.21.15.120:0/1620162430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55782934ab70 0x557829354c50 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7fd054098c00 tx=0x7fd05409c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792c371526f9b141 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.754+0000 7fd042ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:23:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.754+0000 7fd042ffd640 1 -- 172.21.15.120:0/1620162430 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0540b5210 con 0x55782934ab70 2024-09-19T09:23:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.754+0000 7fd042ffd640 1 -- 172.21.15.120:0/1620162430 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd0540a0170 con 0x55782934ab70 2024-09-19T09:23:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.754+0000 7fd06ce73740 1 -- 172.21.15.120:0/1620162430 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd05c036040 con 0x55782934ab70 2024-09-19T09:23:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.754+0000 7fd042ffd640 1 -- 172.21.15.120:0/1620162430 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd05407ae60 con 0x55782934ab70 2024-09-19T09:23:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.754+0000 7fd06ce73740 10 client.22245 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.754+0000 7fd06ce73740 1 -- 172.21.15.120:0/1620162430 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55782908d620 con 0x55782934ab70 2024-09-19T09:23:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.755+0000 7fd042ffd640 1 -- 172.21.15.120:0/1620162430 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd05c036040 con 0x55782934ab70 2024-09-19T09:23:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.755+0000 7fd042ffd640 1 -- 172.21.15.120:0/1620162430 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fd0540a4a10 con 0x55782934ab70 2024-09-19T09:23:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.755+0000 7fd06ce73740 10 client.22245 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.755+0000 7fd06ce73740 10 client.22245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:23:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.755+0000 7fd06ce73740 20 client.22245 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.755+0000 7fd06ce73740 1 --2- 172.21.15.120:0/1620162430 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5578293608a0 0x557829362c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.755+0000 7fd06ce73740 4 client.22245 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:23:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.755+0000 7fd06ce73740 1 -- 172.21.15.120:0/1620162430 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578292bba20 con 0x5578293608a0 2024-09-19T09:23:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.764+0000 7fd062ffd640 1 --2- 172.21.15.120:0/1620162430 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5578293608a0 0x557829362c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.765+0000 7fd042ffd640 10 client.22245 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.765+0000 7fd062ffd640 1 --2- 172.21.15.120:0/1620162430 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5578293608a0 0x557829362c90 crc :-1 s=READY pgs=515 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b7e7a904b47cd89 server_cookie=4c68d20739c79fff in_seq=0 out_seq=0 2024-09-19T09:23:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.768+0000 7fd042ffd640 1 -- 172.21.15.120:0/1620162430 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5578292bba20 con 0x5578293608a0 2024-09-19T09:23:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.768+0000 7fd042ffd640 10 client.22245 handle_command_reply: tid=0 2024-09-19T09:23:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.769+0000 7fd06ce73740 1 client.22245 shutdown 2024-09-19T09:23:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.769+0000 7fd06ce73740 1 -- 172.21.15.120:0/1620162430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55782934ab70 msgr2=0x557829354c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.769+0000 7fd06ce73740 1 --2- 172.21.15.120:0/1620162430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55782934ab70 0x557829354c50 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7fd054098c00 tx=0x7fd05409c930 comp rx=0 tx=0).stop 2024-09-19T09:23:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.769+0000 7fd06ce73740 1 -- 172.21.15.120:0/1620162430 shutdown_connections 2024-09-19T09:23:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.769+0000 7fd06ce73740 1 --2- 172.21.15.120:0/1620162430 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5578293608a0 0x557829362c90 crc :-1 s=READY pgs=515 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.769+0000 7fd06ce73740 1 --2- 172.21.15.120:0/1620162430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557829350540 0x55782935c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.769+0000 7fd06ce73740 1 --2- 172.21.15.120:0/1620162430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55782934b4b0 0x557829355190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.769+0000 7fd06ce73740 1 --2- 172.21.15.120:0/1620162430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55782934ab70 0x557829354c50 secure :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7fd054098c00 tx=0x7fd05409c930 comp rx=0 tx=0).stop 2024-09-19T09:23:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.769+0000 7fd06ce73740 1 -- 172.21.15.120:0/1620162430 >> 172.21.15.120:0/1620162430 conn(0x5578293461e0 msgr2=0x5578293479f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.769+0000 7fd042ffd640 0 client.22245 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.769+0000 7fd06ce73740 1 -- 172.21.15.120:0/1620162430 shutdown_connections 2024-09-19T09:23:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.769+0000 7fd06ce73740 1 -- 172.21.15.120:0/1620162430 wait complete. 2024-09-19T09:23:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.769+0000 7fd06ce73740 20 client.22245 trim_cache size 0 max 16384 2024-09-19T09:23:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.785+0000 7fd06ce73740 1 Processor -- start 2024-09-19T09:23:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.785+0000 7fd06ce73740 1 -- start start 2024-09-19T09:23:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.785+0000 7fd06ce73740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55782934b4b0 0x557829355220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.785+0000 7fd06ce73740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557829350540 0x55782935c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.785+0000 7fd06ce73740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578293477a0 0x557829347c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.785+0000 7fd06ce73740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd0641aaf60 con 0x557829350540 2024-09-19T09:23:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.785+0000 7fd06ce73740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd0641acaa0 con 0x5578293477a0 2024-09-19T09:23:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.785+0000 7fd06ce73740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd0641ab630 con 0x55782934b4b0 2024-09-19T09:23:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.786+0000 7fd062ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557829350540 0x55782935c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.786+0000 7fd062ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557829350540 0x55782935c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50188/0 (socket says 172.21.15.120:50188) 2024-09-19T09:23:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.786+0000 7fd063fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578293477a0 0x557829347c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.786+0000 7fd062ffd640 1 -- 172.21.15.120:0/1985461423 learned_addr learned my addr 172.21.15.120:0/1985461423 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.786+0000 7fd0637fe640 1 --2- 172.21.15.120:0/1985461423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55782934b4b0 0x557829355220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.786+0000 7fd062ffd640 1 -- 172.21.15.120:0/1985461423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55782934b4b0 msgr2=0x557829355220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.786+0000 7fd062ffd640 1 --2- 172.21.15.120:0/1985461423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55782934b4b0 0x557829355220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.786+0000 7fd062ffd640 1 -- 172.21.15.120:0/1985461423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578293477a0 msgr2=0x557829347c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.786+0000 7fd062ffd640 1 --2- 172.21.15.120:0/1985461423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578293477a0 0x557829347c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.786+0000 7fd062ffd640 1 -- 172.21.15.120:0/1985461423 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557829363000 con 0x557829350540 2024-09-19T09:23:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.786+0000 7fd063fff640 1 --2- 172.21.15.120:0/1985461423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578293477a0 0x557829347c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.786+0000 7fd0637fe640 1 --2- 172.21.15.120:0/1985461423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55782934b4b0 0x557829355220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.786+0000 7fd062ffd640 1 --2- 172.21.15.120:0/1985461423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557829350540 0x55782935c0f0 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x557829363500 tx=0x7fd058014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741b3a597ebc69b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.786+0000 7fd0497fa640 1 -- 172.21.15.120:0/1985461423 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058026330 con 0x557829350540 2024-09-19T09:23:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.786+0000 7fd0497fa640 1 -- 172.21.15.120:0/1985461423 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0580269b0 con 0x557829350540 2024-09-19T09:23:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.786+0000 7fd0497fa640 1 -- 172.21.15.120:0/1985461423 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05800fb80 con 0x557829350540 2024-09-19T09:23:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.787+0000 7fd06ce73740 1 -- 172.21.15.120:0/1985461423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557829350540 msgr2=0x55782935c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.787+0000 7fd06ce73740 1 --2- 172.21.15.120:0/1985461423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557829350540 0x55782935c0f0 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x557829363500 tx=0x7fd058014000 comp rx=0 tx=0).stop 2024-09-19T09:23:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.787+0000 7fd06ce73740 1 -- 172.21.15.120:0/1985461423 shutdown_connections 2024-09-19T09:23:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.787+0000 7fd06ce73740 1 --2- 172.21.15.120:0/1985461423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578293477a0 0x557829347c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.787+0000 7fd06ce73740 1 --2- 172.21.15.120:0/1985461423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557829350540 0x55782935c0f0 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.787+0000 7fd06ce73740 1 --2- 172.21.15.120:0/1985461423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55782934b4b0 0x557829355220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.787+0000 7fd06ce73740 1 -- 172.21.15.120:0/1985461423 >> 172.21.15.120:0/1985461423 conn(0x5578293461e0 msgr2=0x557829347090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.787+0000 7fd06ce73740 1 -- 172.21.15.120:0/1985461423 shutdown_connections 2024-09-19T09:23:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.787+0000 7fd06ce73740 1 -- 172.21.15.120:0/1985461423 wait complete. 2024-09-19T09:23:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.788+0000 7fd06ce73740 1 -- start start 2024-09-19T09:23:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.788+0000 7fd06ce73740 1 Processor -- start 2024-09-19T09:23:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.788+0000 7fd06ce73740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578293477a0 0x557829359e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.788+0000 7fd06ce73740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55782934b4b0 0x55782935a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.788+0000 7fd06ce73740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557829350540 0x55782935b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.788+0000 7fd06ce73740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd0641ac3d0 con 0x557829350540 2024-09-19T09:23:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.788+0000 7fd06ce73740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd0641abd00 con 0x55782934b4b0 2024-09-19T09:23:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.788+0000 7fd06ce73740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd0641a8f20 con 0x5578293477a0 2024-09-19T09:23:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.788+0000 7fd06ce73740 10 client.0 _ll_register_callbacks cb 0x7fd0540096e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.788+0000 7fd062ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55782934b4b0 0x55782935a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.788+0000 7fd0637fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578293477a0 0x557829359e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.788+0000 7fd062ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55782934b4b0 0x55782935a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:42698/0 (socket says 172.21.15.120:42698) 2024-09-19T09:23:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.788+0000 7fd0637fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578293477a0 0x557829359e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:51320/0 (socket says 172.21.15.120:51320) 2024-09-19T09:23:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.788+0000 7fd062ffd640 1 -- 172.21.15.120:0/2072124561 learned_addr learned my addr 172.21.15.120:0/2072124561 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.788+0000 7fd063fff640 1 --2- 172.21.15.120:0/2072124561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557829350540 0x55782935b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.789+0000 7fd062ffd640 1 -- 172.21.15.120:0/2072124561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578293477a0 msgr2=0x557829359e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.789+0000 7fd062ffd640 1 --2- 172.21.15.120:0/2072124561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578293477a0 0x557829359e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.789+0000 7fd062ffd640 1 -- 172.21.15.120:0/2072124561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557829350540 msgr2=0x55782935b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.789+0000 7fd062ffd640 1 --2- 172.21.15.120:0/2072124561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557829350540 0x55782935b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.789+0000 7fd062ffd640 1 -- 172.21.15.120:0/2072124561 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55782935b970 con 0x55782934b4b0 2024-09-19T09:23:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.789+0000 7fd062ffd640 1 --2- 172.21.15.120:0/2072124561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55782934b4b0 0x55782935a350 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd058009e80 tx=0x7fd058004190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c8c1c5d9304bc85 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.789+0000 7fd042ffd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:23:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.789+0000 7fd042ffd640 1 -- 172.21.15.120:0/2072124561 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058033070 con 0x55782934b4b0 2024-09-19T09:23:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.789+0000 7fd042ffd640 1 -- 172.21.15.120:0/2072124561 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd058026c20 con 0x55782934b4b0 2024-09-19T09:23:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.789+0000 7fd06ce73740 1 -- 172.21.15.120:0/2072124561 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0580269b0 con 0x55782934b4b0 2024-09-19T09:23:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.789+0000 7fd042ffd640 1 -- 172.21.15.120:0/2072124561 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd05803f050 con 0x55782934b4b0 2024-09-19T09:23:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.789+0000 7fd06ce73740 10 client.31001 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.789+0000 7fd06ce73740 1 -- 172.21.15.120:0/2072124561 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5578293cbb00 con 0x55782934b4b0 2024-09-19T09:23:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.790+0000 7fd042ffd640 1 -- 172.21.15.120:0/2072124561 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0580269b0 con 0x55782934b4b0 2024-09-19T09:23:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.790+0000 7fd042ffd640 1 -- 172.21.15.120:0/2072124561 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fd05801a040 con 0x55782934b4b0 2024-09-19T09:23:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.790+0000 7fd06ce73740 10 client.31001 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.790+0000 7fd06ce73740 10 client.31001 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:23:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.790+0000 7fd06ce73740 20 client.31001 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.790+0000 7fd06ce73740 1 --2- 172.21.15.120:0/2072124561 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5578293cc480 0x5578293ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.790+0000 7fd06ce73740 4 client.31001 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:23:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.790+0000 7fd06ce73740 1 -- 172.21.15.120:0/2072124561 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5578292b5d70 con 0x5578293cc480 2024-09-19T09:23:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.790+0000 7fd0637fe640 1 --2- 172.21.15.120:0/2072124561 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5578293cc480 0x5578293ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.791+0000 7fd042ffd640 10 client.31001 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:43.791+0000 7fd0637fe640 1 --2- 172.21.15.120:0/2072124561 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5578293cc480 0x5578293ce8a0 crc :-1 s=READY pgs=516 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=357f476418538914 server_cookie=b023582d80598784 in_seq=0 out_seq=0 2024-09-19T09:23:44.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:44 smithi120 ceph-mon[31150]: pgmap v971: 97 pgs: 97 active+clean; 454 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.0 MiB/s wr, 700 op/s 2024-09-19T09:23:44.377 INFO:tasks.workunit.client.0.smithi120.stdout: 10 187848 21.04 MB/sec execute 339 sec latency 325.159 ms 2024-09-19T09:23:44.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:44 smithi196 ceph-mon[35150]: pgmap v971: 97 pgs: 97 active+clean; 454 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.0 MiB/s wr, 700 op/s 2024-09-19T09:23:44.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:44 smithi153 ceph-mon[35071]: pgmap v971: 97 pgs: 97 active+clean; 454 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.0 MiB/s wr, 700 op/s 2024-09-19T09:23:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:44.459+0000 7fd042ffd640 1 -- 172.21.15.120:0/2072124561 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5578292b5d70 con 0x5578293cc480 2024-09-19T09:23:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:44.459+0000 7fd042ffd640 10 client.31001 handle_command_reply: tid=0 2024-09-19T09:23:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:44.459+0000 7fd06ce73740 1 client.31001 shutdown 2024-09-19T09:23:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:44.459+0000 7fd06ce73740 1 -- 172.21.15.120:0/2072124561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55782934b4b0 msgr2=0x55782935a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:44.459+0000 7fd06ce73740 1 --2- 172.21.15.120:0/2072124561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55782934b4b0 0x55782935a350 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd058009e80 tx=0x7fd058004190 comp rx=0 tx=0).stop 2024-09-19T09:23:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:44.459+0000 7fd06ce73740 1 -- 172.21.15.120:0/2072124561 shutdown_connections 2024-09-19T09:23:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:44.459+0000 7fd06ce73740 1 --2- 172.21.15.120:0/2072124561 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5578293cc480 0x5578293ce8a0 crc :-1 s=READY pgs=516 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:44.459+0000 7fd06ce73740 1 --2- 172.21.15.120:0/2072124561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557829350540 0x55782935b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:44.459+0000 7fd06ce73740 1 --2- 172.21.15.120:0/2072124561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55782934b4b0 0x55782935a350 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:44.459+0000 7fd06ce73740 1 --2- 172.21.15.120:0/2072124561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578293477a0 0x557829359e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:44.459+0000 7fd06ce73740 1 -- 172.21.15.120:0/2072124561 >> 172.21.15.120:0/2072124561 conn(0x5578293461e0 msgr2=0x557829344a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:44.459+0000 7fd042ffd640 0 client.31001 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:44.460+0000 7fd06ce73740 1 -- 172.21.15.120:0/2072124561 shutdown_connections 2024-09-19T09:23:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:44.460+0000 7fd06ce73740 1 -- 172.21.15.120:0/2072124561 wait complete. 2024-09-19T09:23:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:44.460+0000 7fd06ce73740 20 client.31001 trim_cache size 0 max 16384 2024-09-19T09:23:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:23:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:23:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:23:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:23:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:44.460+0000 7fd068d9f640 1 -- 172.21.15.120:0/2157246968 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd04c0a3340 msgr2=0x7fd04c0a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:44.460+0000 7fd068d9f640 1 --2- 172.21.15.120:0/2157246968 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd04c0a3340 0x7fd04c0a5800 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd06419b470 tx=0x7fd058005f90 comp rx=0 tx=0).stop 2024-09-19T09:23:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:44.460+0000 7fd068d9f640 1 -- 172.21.15.120:0/2157246968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd064101580 msgr2=0x7fd064199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:44.460+0000 7fd068d9f640 1 --2- 172.21.15.120:0/2157246968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd064101580 0x7fd064199f60 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7fd05400e9c0 tx=0x7fd05400ee80 comp rx=0 tx=0).stop 2024-09-19T09:23:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:44.461+0000 7fd068d9f640 1 -- 172.21.15.120:0/2157246968 shutdown_connections 2024-09-19T09:23:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:44.461+0000 7fd068d9f640 1 --2- 172.21.15.120:0/2157246968 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd04c0a3340 0x7fd04c0a5800 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:44.461+0000 7fd068d9f640 1 --2- 172.21.15.120:0/2157246968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd064103050 0x7fd0641a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:44.461+0000 7fd068d9f640 1 --2- 172.21.15.120:0/2157246968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd064101f40 0x7fd06419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:44.461+0000 7fd068d9f640 1 --2- 172.21.15.120:0/2157246968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd064101580 0x7fd064199f60 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:44.461+0000 7fd068d9f640 1 -- 172.21.15.120:0/2157246968 >> 172.21.15.120:0/2157246968 conn(0x7fd0640fd0d0 msgr2=0x7fd0640fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:44.461+0000 7fd068d9f640 1 -- 172.21.15.120:0/2157246968 shutdown_connections 2024-09-19T09:23:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:44.461+0000 7fd068d9f640 1 -- 172.21.15.120:0/2157246968 wait complete. 2024-09-19T09:23:45.377 INFO:tasks.workunit.client.0.smithi120.stdout: 10 188350 21.04 MB/sec execute 340 sec latency 172.509 ms 2024-09-19T09:23:46.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:46 smithi120 ceph-mon[31150]: pgmap v972: 97 pgs: 97 active+clean; 455 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.7 MiB/s wr, 526 op/s 2024-09-19T09:23:46.378 INFO:tasks.workunit.client.0.smithi120.stdout: 10 188915 21.09 MB/sec execute 341 sec latency 74.074 ms 2024-09-19T09:23:46.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:46 smithi196 ceph-mon[35150]: pgmap v972: 97 pgs: 97 active+clean; 455 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.7 MiB/s wr, 526 op/s 2024-09-19T09:23:46.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:46 smithi153 ceph-mon[35071]: pgmap v972: 97 pgs: 97 active+clean; 455 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.7 MiB/s wr, 526 op/s 2024-09-19T09:23:47.378 INFO:tasks.workunit.client.0.smithi120.stdout: 10 189658 21.09 MB/sec execute 342 sec latency 192.855 ms 2024-09-19T09:23:48.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:48 smithi120 ceph-mon[31150]: pgmap v973: 97 pgs: 97 active+clean; 450 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.8 MiB/s wr, 611 op/s 2024-09-19T09:23:48.377 INFO:tasks.workunit.client.0.smithi120.stdout: 10 190382 21.18 MB/sec execute 343 sec latency 44.047 ms 2024-09-19T09:23:48.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:48 smithi196 ceph-mon[35150]: pgmap v973: 97 pgs: 97 active+clean; 450 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.8 MiB/s wr, 611 op/s 2024-09-19T09:23:48.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:48 smithi153 ceph-mon[35071]: pgmap v973: 97 pgs: 97 active+clean; 450 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.8 MiB/s wr, 611 op/s 2024-09-19T09:23:48.649 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:23:48.926 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:23:49.378 INFO:tasks.workunit.client.0.smithi120.stdout: 10 191061 21.24 MB/sec execute 344 sec latency 174.788 ms 2024-09-19T09:23:50.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.574+0000 7f85e88fa640 1 -- 172.21.15.120:0/3315231255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85e4102570 msgr2=0x7f85e41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:50.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.574+0000 7f85e88fa640 1 --2- 172.21.15.120:0/3315231255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85e4102570 0x7f85e41029b0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7f85cc009900 tx=0x7f85cc02a480 comp rx=0 tx=0).stop 2024-09-19T09:23:50.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.574+0000 7f85e88fa640 1 -- 172.21.15.120:0/3315231255 shutdown_connections 2024-09-19T09:23:50.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.574+0000 7f85e88fa640 1 --2- 172.21.15.120:0/3315231255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85e4102ef0 0x7f85e410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.574+0000 7f85e88fa640 1 --2- 172.21.15.120:0/3315231255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85e4102570 0x7f85e41029b0 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.574+0000 7f85e88fa640 1 --2- 172.21.15.120:0/3315231255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85e4100e20 0x7f85e4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.574+0000 7f85e88fa640 1 -- 172.21.15.120:0/3315231255 >> 172.21.15.120:0/3315231255 conn(0x7f85e40fc7d0 msgr2=0x7f85e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:50.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.574+0000 7f85e88fa640 1 -- 172.21.15.120:0/3315231255 shutdown_connections 2024-09-19T09:23:50.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.574+0000 7f85e88fa640 1 -- 172.21.15.120:0/3315231255 wait complete. 2024-09-19T09:23:50.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.575+0000 7f85e88fa640 1 Processor -- start 2024-09-19T09:23:50.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.575+0000 7f85e88fa640 1 -- start start 2024-09-19T09:23:50.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.575+0000 7f85e88fa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85e4100e20 0x7f85e41973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:50.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.575+0000 7f85e88fa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85e4102570 0x7f85e4197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:50.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.575+0000 7f85e88fa640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85e4102ef0 0x7f85e419bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:50.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.575+0000 7f85e88fa640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f85e4110fd0 con 0x7f85e4102570 2024-09-19T09:23:50.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.575+0000 7f85e88fa640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f85e4110e50 con 0x7f85e4102ef0 2024-09-19T09:23:50.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.575+0000 7f85e88fa640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f85e4111150 con 0x7f85e4100e20 2024-09-19T09:23:50.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.575+0000 7f85dbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85e4102570 0x7f85e4197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:50.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.575+0000 7f85dbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85e4102570 0x7f85e4197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50230/0 (socket says 172.21.15.120:50230) 2024-09-19T09:23:50.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.575+0000 7f85e259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85e4100e20 0x7f85e41973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:50.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.575+0000 7f85dbfff640 1 -- 172.21.15.120:0/1064366361 learned_addr learned my addr 172.21.15.120:0/1064366361 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:50.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.575+0000 7f85e2d9d640 1 --2- 172.21.15.120:0/1064366361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85e4102ef0 0x7f85e419bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:50.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.575+0000 7f85dbfff640 1 -- 172.21.15.120:0/1064366361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85e4100e20 msgr2=0x7f85e41973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:50.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.575+0000 7f85dbfff640 1 --2- 172.21.15.120:0/1064366361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85e4100e20 0x7f85e41973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.575+0000 7f85dbfff640 1 -- 172.21.15.120:0/1064366361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85e4102ef0 msgr2=0x7f85e419bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:50.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.575+0000 7f85dbfff640 1 --2- 172.21.15.120:0/1064366361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85e4102ef0 0x7f85e419bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.575+0000 7f85dbfff640 1 -- 172.21.15.120:0/1064366361 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e419c380 con 0x7f85e4102570 2024-09-19T09:23:50.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.576+0000 7f85dbfff640 1 --2- 172.21.15.120:0/1064366361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85e4102570 0x7f85e4197910 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f85cc02aa70 tx=0x7f85cc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:50.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.576+0000 7f85db7fe640 1 -- 172.21.15.120:0/1064366361 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc03f070 con 0x7f85e4102570 2024-09-19T09:23:50.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.576+0000 7f85e88fa640 1 -- 172.21.15.120:0/1064366361 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e419c610 con 0x7f85e4102570 2024-09-19T09:23:50.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.576+0000 7f85db7fe640 1 -- 172.21.15.120:0/1064366361 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85cc031c30 con 0x7f85e4102570 2024-09-19T09:23:50.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.576+0000 7f85db7fe640 1 -- 172.21.15.120:0/1064366361 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc035030 con 0x7f85e4102570 2024-09-19T09:23:50.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.576+0000 7f85e88fa640 1 -- 172.21.15.120:0/1064366361 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e41a3f50 con 0x7f85e4102570 2024-09-19T09:23:50.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.578+0000 7f85db7fe640 1 -- 172.21.15.120:0/1064366361 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f85cc042070 con 0x7f85e4102570 2024-09-19T09:23:50.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.578+0000 7f85eacd3740 1 Processor -- start 2024-09-19T09:23:50.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.579+0000 7f85eacd3740 1 -- start start 2024-09-19T09:23:50.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.579+0000 7f85db7fe640 1 --2- 172.21.15.120:0/1064366361 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f85d00a3240 0x7f85d00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:50.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.579+0000 7f85eacd3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da61160320 0x55da61160700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:50.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.579+0000 7f85eacd3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da61160c40 0x55da6116d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:50.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.579+0000 7f85eacd3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da6116d890 0x55da6116fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:50.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.579+0000 7f85e2d9d640 1 --2- 172.21.15.120:0/1064366361 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f85d00a3240 0x7f85d00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:50.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.579+0000 7f85eacd3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55da610c62e0 con 0x55da6116d890 2024-09-19T09:23:50.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.579+0000 7f85eacd3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55da6106dce0 con 0x55da61160c40 2024-09-19T09:23:50.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.579+0000 7f85eacd3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55da610c22a0 con 0x55da61160320 2024-09-19T09:23:50.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.579+0000 7f85db7fe640 1 -- 172.21.15.120:0/1064366361 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f85cc0eff70 con 0x7f85e4102570 2024-09-19T09:23:50.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.579+0000 7f85e259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da61160320 0x55da61160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:50.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.579+0000 7f85e259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da61160320 0x55da61160700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:51348/0 (socket says 172.21.15.120:51348) 2024-09-19T09:23:50.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.579+0000 7f85e259c640 1 -- 172.21.15.120:0/2713795738 learned_addr learned my addr 172.21.15.120:0/2713795738 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:50.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.579+0000 7f85dbfff640 1 --2- 172.21.15.120:0/2713795738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da61160c40 0x55da6116d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:50.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.579+0000 7f85e2d9d640 1 --2- 172.21.15.120:0/2713795738 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da6116d890 0x55da6116fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:50.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.579+0000 7f85dbfff640 1 -- 172.21.15.120:0/2713795738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da61160320 msgr2=0x55da61160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:50.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.579+0000 7f85dbfff640 1 --2- 172.21.15.120:0/2713795738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da61160320 0x55da61160700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.579+0000 7f85dbfff640 1 -- 172.21.15.120:0/2713795738 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da6116d890 msgr2=0x55da6116fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:50.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.579+0000 7f85dbfff640 1 --2- 172.21.15.120:0/2713795738 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da6116d890 0x55da6116fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.579+0000 7f85dbfff640 1 -- 172.21.15.120:0/2713795738 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da61052440 con 0x55da61160c40 2024-09-19T09:23:50.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.579+0000 7f85e2d9d640 1 --2- 172.21.15.120:0/2713795738 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da6116d890 0x55da6116fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:23:50.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.580+0000 7f85e2d9d640 1 --2- 172.21.15.120:0/1064366361 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f85d00a3240 0x7f85d00a5700 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009fd0 tx=0x7f85dc012b70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:50.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.580+0000 7f85dbfff640 1 --2- 172.21.15.120:0/2713795738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da61160c40 0x55da6116d350 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f85cc004650 tx=0x7f85cc005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e49cbe04ca8ef76f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:50.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.580+0000 7f85d8ff9640 1 -- 172.21.15.120:0/2713795738 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc049070 con 0x55da61160c40 2024-09-19T09:23:50.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.580+0000 7f85d8ff9640 1 -- 172.21.15.120:0/2713795738 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85cc0bc640 con 0x55da61160c40 2024-09-19T09:23:50.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.580+0000 7f85d8ff9640 1 -- 172.21.15.120:0/2713795738 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc03e170 con 0x55da61160c40 2024-09-19T09:23:50.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.581+0000 7f85eacd3740 1 -- 172.21.15.120:0/2713795738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da61160c40 msgr2=0x55da6116d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:50.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.581+0000 7f85eacd3740 1 --2- 172.21.15.120:0/2713795738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da61160c40 0x55da6116d350 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f85cc004650 tx=0x7f85cc005e50 comp rx=0 tx=0).stop 2024-09-19T09:23:50.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.581+0000 7f85eacd3740 1 -- 172.21.15.120:0/2713795738 shutdown_connections 2024-09-19T09:23:50.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.581+0000 7f85eacd3740 1 --2- 172.21.15.120:0/2713795738 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da6116d890 0x55da6116fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.581+0000 7f85eacd3740 1 --2- 172.21.15.120:0/2713795738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da61160c40 0x55da6116d350 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.581+0000 7f85eacd3740 1 --2- 172.21.15.120:0/2713795738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da61160320 0x55da61160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.581+0000 7f85eacd3740 1 -- 172.21.15.120:0/2713795738 >> 172.21.15.120:0/2713795738 conn(0x55da6115ca30 msgr2=0x55da6115ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:50.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.581+0000 7f85eacd3740 1 -- 172.21.15.120:0/2713795738 shutdown_connections 2024-09-19T09:23:50.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.581+0000 7f85eacd3740 1 -- 172.21.15.120:0/2713795738 wait complete. 2024-09-19T09:23:50.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.581+0000 7f85eacd3740 1 -- start start 2024-09-19T09:23:50.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.581+0000 7f85eacd3740 1 Processor -- start 2024-09-19T09:23:50.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.582+0000 7f85eacd3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da61160320 0x55da6116d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:50.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.582+0000 7f85eacd3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da61160c40 0x55da6116a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:50.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.582+0000 7f85eacd3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da6116d890 0x55da6116ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:50.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.582+0000 7f85eacd3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55da61170280 con 0x55da61160c40 2024-09-19T09:23:50.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.582+0000 7f85eacd3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55da61081050 con 0x55da61160320 2024-09-19T09:23:50.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.582+0000 7f85eacd3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55da61170400 con 0x55da6116d890 2024-09-19T09:23:50.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.582+0000 7f85eacd3740 10 client.0 _ll_register_callbacks cb 0x55da610c1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:50.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.582+0000 7f85e259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da61160320 0x55da6116d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:50.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.582+0000 7f85e259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da61160320 0x55da6116d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:42730/0 (socket says 172.21.15.120:42730) 2024-09-19T09:23:50.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.582+0000 7f85e259c640 1 -- 172.21.15.120:0/3160639242 learned_addr learned my addr 172.21.15.120:0/3160639242 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:50.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.582+0000 7f85dbfff640 1 --2- 172.21.15.120:0/3160639242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da61160c40 0x55da6116a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:50.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.582+0000 7f85e2d9d640 1 --2- 172.21.15.120:0/3160639242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da6116d890 0x55da6116ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:50.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.582+0000 7f85e259c640 1 -- 172.21.15.120:0/3160639242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da6116d890 msgr2=0x55da6116ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:50.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.582+0000 7f85e259c640 1 --2- 172.21.15.120:0/3160639242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da6116d890 0x55da6116ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.582+0000 7f85e259c640 1 -- 172.21.15.120:0/3160639242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da61160c40 msgr2=0x55da6116a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:50.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.582+0000 7f85e259c640 1 --2- 172.21.15.120:0/3160639242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da61160c40 0x55da6116a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.582+0000 7f85e259c640 1 -- 172.21.15.120:0/3160639242 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da60e373c0 con 0x55da61160320 2024-09-19T09:23:50.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.582+0000 7f85e259c640 1 --2- 172.21.15.120:0/3160639242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da61160320 0x55da6116d230 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f85d4012840 tx=0x7f85d4012d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f90a679a5009d3c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:50.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.582+0000 7f85c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:23:50.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.582+0000 7f85c9ffb640 1 -- 172.21.15.120:0/3160639242 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4004050 con 0x55da61160320 2024-09-19T09:23:50.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.583+0000 7f85eacd3740 1 -- 172.21.15.120:0/3160639242 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85cc0bc640 con 0x55da61160320 2024-09-19T09:23:50.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.583+0000 7f85c9ffb640 1 -- 172.21.15.120:0/3160639242 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f85d40041f0 con 0x55da61160320 2024-09-19T09:23:50.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.583+0000 7f85c9ffb640 1 -- 172.21.15.120:0/3160639242 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f85d40043f0 con 0x55da61160320 2024-09-19T09:23:50.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.583+0000 7f85eacd3740 10 client.31013 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:50.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.583+0000 7f85eacd3740 1 -- 172.21.15.120:0/3160639242 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55da6117bcb0 con 0x55da61160320 2024-09-19T09:23:50.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.583+0000 7f85c9ffb640 1 -- 172.21.15.120:0/3160639242 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85cc0bc640 con 0x55da61160320 2024-09-19T09:23:50.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.583+0000 7f85c9ffb640 1 -- 172.21.15.120:0/3160639242 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f85d4003bc0 con 0x55da61160320 2024-09-19T09:23:50.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.583+0000 7f85eacd3740 10 client.31013 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:50.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.583+0000 7f85eacd3740 10 client.31013 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:23:50.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.583+0000 7f85eacd3740 20 client.31013 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:50.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.583+0000 7f85eacd3740 1 --2- 172.21.15.120:0/3160639242 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55da6117d170 0x55da6117f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:50.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.583+0000 7f85eacd3740 4 client.31013 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:23:50.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.583+0000 7f85eacd3740 1 -- 172.21.15.120:0/3160639242 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da610d3850 con 0x55da6117d170 2024-09-19T09:23:50.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.583+0000 7f85dbfff640 1 --2- 172.21.15.120:0/3160639242 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55da6117d170 0x55da6117f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:50.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.584+0000 7f85dbfff640 1 --2- 172.21.15.120:0/3160639242 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55da6117d170 0x55da6117f580 crc :-1 s=READY pgs=517 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92938376daec474b server_cookie=db4edc843fa6fd3f in_seq=0 out_seq=0 2024-09-19T09:23:50.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.584+0000 7f85c9ffb640 10 client.31013 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:50.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.586+0000 7f85c9ffb640 1 -- 172.21.15.120:0/3160639242 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55da610d3850 con 0x55da6117d170 2024-09-19T09:23:50.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.586+0000 7f85c9ffb640 10 client.31013 handle_command_reply: tid=0 2024-09-19T09:23:50.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.586+0000 7f85eacd3740 1 client.31013 shutdown 2024-09-19T09:23:50.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.586+0000 7f85eacd3740 1 -- 172.21.15.120:0/3160639242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da61160320 msgr2=0x55da6116d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:50.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.586+0000 7f85eacd3740 1 --2- 172.21.15.120:0/3160639242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da61160320 0x55da6116d230 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f85d4012840 tx=0x7f85d4012d00 comp rx=0 tx=0).stop 2024-09-19T09:23:50.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.586+0000 7f85eacd3740 1 -- 172.21.15.120:0/3160639242 shutdown_connections 2024-09-19T09:23:50.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.586+0000 7f85eacd3740 1 --2- 172.21.15.120:0/3160639242 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55da6117d170 0x55da6117f580 crc :-1 s=READY pgs=517 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.586+0000 7f85eacd3740 1 --2- 172.21.15.120:0/3160639242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da6116d890 0x55da6116ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.586+0000 7f85eacd3740 1 --2- 172.21.15.120:0/3160639242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da61160c40 0x55da6116a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.586+0000 7f85eacd3740 1 --2- 172.21.15.120:0/3160639242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da61160320 0x55da6116d230 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.586+0000 7f85c9ffb640 0 client.31013 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:50.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.586+0000 7f85eacd3740 1 -- 172.21.15.120:0/3160639242 >> 172.21.15.120:0/3160639242 conn(0x55da6115ca30 msgr2=0x55da611626f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:50.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.586+0000 7f85eacd3740 1 -- 172.21.15.120:0/3160639242 shutdown_connections 2024-09-19T09:23:50.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.586+0000 7f85eacd3740 1 -- 172.21.15.120:0/3160639242 wait complete. 2024-09-19T09:23:50.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.586+0000 7f85eacd3740 20 client.31013 trim_cache size 0 max 16384 2024-09-19T09:23:50.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.606+0000 7f85eacd3740 1 Processor -- start 2024-09-19T09:23:50.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.606+0000 7f85eacd3740 1 -- start start 2024-09-19T09:23:50.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.606+0000 7f85eacd3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da61160320 0x55da61179420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:50.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.606+0000 7f85eacd3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da61160c40 0x55da61169e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:50.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.606+0000 7f85eacd3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da6116d890 0x55da61184e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:50.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.607+0000 7f85eacd3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f85e41ab2a0 con 0x55da6116d890 2024-09-19T09:23:50.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.607+0000 7f85eacd3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f85e41acde0 con 0x55da61160320 2024-09-19T09:23:50.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.607+0000 7f85eacd3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f85e41ab970 con 0x55da61160c40 2024-09-19T09:23:50.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.607+0000 7f85e2d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da6116d890 0x55da61184e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:50.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.607+0000 7f85e259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da61160320 0x55da61179420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:50.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.607+0000 7f85e2d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da6116d890 0x55da61184e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50254/0 (socket says 172.21.15.120:50254) 2024-09-19T09:23:50.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.607+0000 7f85e2d9d640 1 -- 172.21.15.120:0/2057485291 learned_addr learned my addr 172.21.15.120:0/2057485291 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:50.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.607+0000 7f85dbfff640 1 --2- 172.21.15.120:0/2057485291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da61160c40 0x55da61169e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:50.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.607+0000 7f85e2d9d640 1 -- 172.21.15.120:0/2057485291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da61160c40 msgr2=0x55da61169e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:50.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.607+0000 7f85e2d9d640 1 --2- 172.21.15.120:0/2057485291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da61160c40 0x55da61169e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.607+0000 7f85e2d9d640 1 -- 172.21.15.120:0/2057485291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da61160320 msgr2=0x55da61179420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:50.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.607+0000 7f85e2d9d640 1 --2- 172.21.15.120:0/2057485291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da61160320 0x55da61179420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.607+0000 7f85e2d9d640 1 -- 172.21.15.120:0/2057485291 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da6116d5a0 con 0x55da6116d890 2024-09-19T09:23:50.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.607+0000 7f85e259c640 1 --2- 172.21.15.120:0/2057485291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da61160320 0x55da61179420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:50.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.607+0000 7f85dbfff640 1 --2- 172.21.15.120:0/2057485291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da61160c40 0x55da61169e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:23:50.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.607+0000 7f85e2d9d640 1 --2- 172.21.15.120:0/2057485291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da6116d890 0x55da61184e50 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f85dc010ce0 tx=0x7f85dc01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9acf0ac6325f1b38 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:50.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.608+0000 7f85cbfff640 1 -- 172.21.15.120:0/2057485291 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc004240 con 0x55da6116d890 2024-09-19T09:23:50.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.608+0000 7f85cbfff640 1 -- 172.21.15.120:0/2057485291 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85dc02b480 con 0x55da6116d890 2024-09-19T09:23:50.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.608+0000 7f85cbfff640 1 -- 172.21.15.120:0/2057485291 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc020670 con 0x55da6116d890 2024-09-19T09:23:50.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.609+0000 7f85eacd3740 1 -- 172.21.15.120:0/2057485291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da6116d890 msgr2=0x55da61184e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:50.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.609+0000 7f85eacd3740 1 --2- 172.21.15.120:0/2057485291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da6116d890 0x55da61184e50 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f85dc010ce0 tx=0x7f85dc01e000 comp rx=0 tx=0).stop 2024-09-19T09:23:50.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.609+0000 7f85eacd3740 1 -- 172.21.15.120:0/2057485291 shutdown_connections 2024-09-19T09:23:50.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.609+0000 7f85eacd3740 1 --2- 172.21.15.120:0/2057485291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da6116d890 0x55da61184e50 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.609+0000 7f85eacd3740 1 --2- 172.21.15.120:0/2057485291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da61160c40 0x55da61169e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.609+0000 7f85eacd3740 1 --2- 172.21.15.120:0/2057485291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da61160320 0x55da61179420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.609+0000 7f85eacd3740 1 -- 172.21.15.120:0/2057485291 >> 172.21.15.120:0/2057485291 conn(0x55da6115ca30 msgr2=0x55da6115bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:50.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.609+0000 7f85eacd3740 1 -- 172.21.15.120:0/2057485291 shutdown_connections 2024-09-19T09:23:50.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.609+0000 7f85eacd3740 1 -- 172.21.15.120:0/2057485291 wait complete. 2024-09-19T09:23:50.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.609+0000 7f85eacd3740 1 -- start start 2024-09-19T09:23:50.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.609+0000 7f85eacd3740 1 Processor -- start 2024-09-19T09:23:50.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.610+0000 7f85eacd3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da61160320 0x55da61163ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:50.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.610+0000 7f85eacd3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da61160c40 0x55da61163ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:50.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.610+0000 7f85eacd3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da6116d890 0x55da61174570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:50.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.610+0000 7f85eacd3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f85e41ac710 con 0x55da61160320 2024-09-19T09:23:50.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.610+0000 7f85eacd3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f85e41ac040 con 0x55da6116d890 2024-09-19T09:23:50.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.610+0000 7f85eacd3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f85e41a9260 con 0x55da61160c40 2024-09-19T09:23:50.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.610+0000 7f85eacd3740 10 client.0 _ll_register_callbacks cb 0x7f85d40098b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:50.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.610+0000 7f85e259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da61160320 0x55da61163ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:50.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.610+0000 7f85e259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da61160320 0x55da61163ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50268/0 (socket says 172.21.15.120:50268) 2024-09-19T09:23:50.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.610+0000 7f85e259c640 1 -- 172.21.15.120:0/1372504456 learned_addr learned my addr 172.21.15.120:0/1372504456 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:50.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.610+0000 7f85dbfff640 1 --2- 172.21.15.120:0/1372504456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da61160c40 0x55da61163ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:50.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.610+0000 7f85e2d9d640 1 --2- 172.21.15.120:0/1372504456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da6116d890 0x55da61174570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:50.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.610+0000 7f85e259c640 1 -- 172.21.15.120:0/1372504456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da61160c40 msgr2=0x55da61163ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:50.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.610+0000 7f85e259c640 1 --2- 172.21.15.120:0/1372504456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da61160c40 0x55da61163ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.610+0000 7f85e259c640 1 -- 172.21.15.120:0/1372504456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da6116d890 msgr2=0x55da61174570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:50.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.610+0000 7f85e259c640 1 --2- 172.21.15.120:0/1372504456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da6116d890 0x55da61174570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.610+0000 7f85e259c640 1 -- 172.21.15.120:0/1372504456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da61174ab0 con 0x55da61160320 2024-09-19T09:23:50.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.610+0000 7f85e259c640 1 --2- 172.21.15.120:0/1372504456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da61160320 0x55da61163ab0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f85d4003f30 tx=0x7f85d40041e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ccba23306de6186 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:50.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.610+0000 7f85c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:23:50.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.610+0000 7f85c9ffb640 1 -- 172.21.15.120:0/1372504456 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4010be0 con 0x55da61160320 2024-09-19T09:23:50.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.611+0000 7f85c9ffb640 1 -- 172.21.15.120:0/1372504456 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f85d4010d80 con 0x55da61160320 2024-09-19T09:23:50.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.611+0000 7f85eacd3740 1 -- 172.21.15.120:0/1372504456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85dc02b480 con 0x55da61160320 2024-09-19T09:23:50.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.611+0000 7f85c9ffb640 1 -- 172.21.15.120:0/1372504456 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f85d400f790 con 0x55da61160320 2024-09-19T09:23:50.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.611+0000 7f85eacd3740 10 client.22269 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:50.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.611+0000 7f85eacd3740 1 -- 172.21.15.120:0/1372504456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55da61175dc0 con 0x55da61160320 2024-09-19T09:23:50.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.611+0000 7f85c9ffb640 1 -- 172.21.15.120:0/1372504456 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85d400fe10 con 0x55da61160320 2024-09-19T09:23:50.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.611+0000 7f85c9ffb640 1 -- 172.21.15.120:0/1372504456 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f85d40035f0 con 0x55da61160320 2024-09-19T09:23:50.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.611+0000 7f85eacd3740 10 client.22269 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:50.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.611+0000 7f85eacd3740 10 client.22269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:23:50.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.611+0000 7f85eacd3740 20 client.22269 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:50.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.612+0000 7f85eacd3740 1 --2- 172.21.15.120:0/1372504456 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55da611eead0 0x55da611f0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:50.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.612+0000 7f85eacd3740 4 client.22269 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:23:50.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.612+0000 7f85eacd3740 1 -- 172.21.15.120:0/1372504456 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f85dc02b480 con 0x55da611eead0 2024-09-19T09:23:50.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.612+0000 7f85dbfff640 1 --2- 172.21.15.120:0/1372504456 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55da611eead0 0x55da611f0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:50.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.612+0000 7f85c9ffb640 10 client.22269 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:50.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.612+0000 7f85dbfff640 1 --2- 172.21.15.120:0/1372504456 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55da611eead0 0x55da611f0ef0 crc :-1 s=READY pgs=518 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc2c26e3fcacb81d server_cookie=be05aad554117252 in_seq=0 out_seq=0 2024-09-19T09:23:50.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.614+0000 7f85c9ffb640 1 -- 172.21.15.120:0/1372504456 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f85dc02b480 con 0x55da611eead0 2024-09-19T09:23:50.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.614+0000 7f85c9ffb640 10 client.22269 handle_command_reply: tid=0 2024-09-19T09:23:50.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.614+0000 7f85eacd3740 1 client.22269 shutdown 2024-09-19T09:23:50.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.614+0000 7f85eacd3740 1 -- 172.21.15.120:0/1372504456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da61160320 msgr2=0x55da61163ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:50.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.614+0000 7f85eacd3740 1 --2- 172.21.15.120:0/1372504456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da61160320 0x55da61163ab0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f85d4003f30 tx=0x7f85d40041e0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.614+0000 7f85eacd3740 1 -- 172.21.15.120:0/1372504456 shutdown_connections 2024-09-19T09:23:50.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.614+0000 7f85eacd3740 1 --2- 172.21.15.120:0/1372504456 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55da611eead0 0x55da611f0ef0 crc :-1 s=READY pgs=518 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.614+0000 7f85eacd3740 1 --2- 172.21.15.120:0/1372504456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da6116d890 0x55da61174570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.614+0000 7f85eacd3740 1 --2- 172.21.15.120:0/1372504456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da61160c40 0x55da61163ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.614+0000 7f85eacd3740 1 --2- 172.21.15.120:0/1372504456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da61160320 0x55da61163ab0 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.615+0000 7f85eacd3740 1 -- 172.21.15.120:0/1372504456 >> 172.21.15.120:0/1372504456 conn(0x55da6115ca30 msgr2=0x55da6116bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:50.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.615+0000 7f85c9ffb640 0 client.22269 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:50.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.615+0000 7f85eacd3740 1 -- 172.21.15.120:0/1372504456 shutdown_connections 2024-09-19T09:23:50.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.615+0000 7f85eacd3740 1 -- 172.21.15.120:0/1372504456 wait complete. 2024-09-19T09:23:50.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.615+0000 7f85eacd3740 20 client.22269 trim_cache size 0 max 16384 2024-09-19T09:23:50.182 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:23:50.182 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:23:50.182 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:23:50.182 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:23:50.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.616+0000 7f85e88fa640 1 -- 172.21.15.120:0/1064366361 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f85d00a3240 msgr2=0x7f85d00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:50.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.616+0000 7f85e88fa640 1 --2- 172.21.15.120:0/1064366361 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f85d00a3240 0x7f85d00a5700 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009fd0 tx=0x7f85dc012b70 comp rx=0 tx=0).stop 2024-09-19T09:23:50.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.616+0000 7f85e88fa640 1 -- 172.21.15.120:0/1064366361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85e4102570 msgr2=0x7f85e4197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:50.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.616+0000 7f85e88fa640 1 --2- 172.21.15.120:0/1064366361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85e4102570 0x7f85e4197910 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f85cc02aa70 tx=0x7f85cc003c10 comp rx=0 tx=0).stop 2024-09-19T09:23:50.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.616+0000 7f85e88fa640 1 -- 172.21.15.120:0/1064366361 shutdown_connections 2024-09-19T09:23:50.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.616+0000 7f85e88fa640 1 --2- 172.21.15.120:0/1064366361 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f85d00a3240 0x7f85d00a5700 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.616+0000 7f85e88fa640 1 --2- 172.21.15.120:0/1064366361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85e4102ef0 0x7f85e419bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.616+0000 7f85e88fa640 1 --2- 172.21.15.120:0/1064366361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85e4102570 0x7f85e4197910 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.616+0000 7f85e88fa640 1 --2- 172.21.15.120:0/1064366361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85e4100e20 0x7f85e41973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.616+0000 7f85e88fa640 1 -- 172.21.15.120:0/1064366361 >> 172.21.15.120:0/1064366361 conn(0x7f85e40fc7d0 msgr2=0x7f85e4105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:50.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.616+0000 7f85e88fa640 1 -- 172.21.15.120:0/1064366361 shutdown_connections 2024-09-19T09:23:50.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:49.616+0000 7f85e88fa640 1 -- 172.21.15.120:0/1064366361 wait complete. 2024-09-19T09:23:50.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:50 smithi120 ceph-mon[31150]: pgmap v974: 97 pgs: 97 active+clean; 444 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.9 MiB/s wr, 678 op/s 2024-09-19T09:23:50.378 INFO:tasks.workunit.client.0.smithi120.stdout: 10 191478 21.28 MB/sec execute 345 sec latency 163.975 ms 2024-09-19T09:23:50.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:50 smithi196 ceph-mon[35150]: pgmap v974: 97 pgs: 97 active+clean; 444 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.9 MiB/s wr, 678 op/s 2024-09-19T09:23:50.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:50 smithi153 ceph-mon[35071]: pgmap v974: 97 pgs: 97 active+clean; 444 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.9 MiB/s wr, 678 op/s 2024-09-19T09:23:51.353 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:23:51.353 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:23:51.378 INFO:tasks.workunit.client.0.smithi120.stdout: 10 191947 21.26 MB/sec execute 346 sec latency 65.111 ms 2024-09-19T09:23:51.647 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:23:52.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.138+0000 7f9e01630640 1 -- 172.21.15.120:0/3679030978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dfc103760 msgr2=0x7f9dfc107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:52.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.138+0000 7f9e01630640 1 --2- 172.21.15.120:0/3679030978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dfc103760 0x7f9dfc107ba0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009930 tx=0x7f9df002a420 comp rx=0 tx=0).stop 2024-09-19T09:23:52.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.138+0000 7f9e01630640 1 -- 172.21.15.120:0/3679030978 shutdown_connections 2024-09-19T09:23:52.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.138+0000 7f9e01630640 1 --2- 172.21.15.120:0/3679030978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9dfc1082e0 0x7f9dfc10ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:52.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.138+0000 7f9e01630640 1 --2- 172.21.15.120:0/3679030978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dfc103760 0x7f9dfc107ba0 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:52.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.138+0000 7f9e01630640 1 --2- 172.21.15.120:0/3679030978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dfc102db0 0x7f9dfc103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:52.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.138+0000 7f9e01630640 1 -- 172.21.15.120:0/3679030978 >> 172.21.15.120:0/3679030978 conn(0x7f9dfc0fc7d0 msgr2=0x7f9dfc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:52.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.138+0000 7f9e01630640 1 -- 172.21.15.120:0/3679030978 shutdown_connections 2024-09-19T09:23:52.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.138+0000 7f9e01630640 1 -- 172.21.15.120:0/3679030978 wait complete. 2024-09-19T09:23:52.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.138+0000 7f9e01630640 1 Processor -- start 2024-09-19T09:23:52.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.139+0000 7f9e01630640 1 -- start start 2024-09-19T09:23:52.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.139+0000 7f9e01630640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9dfc102db0 0x7f9dfc100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:52.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.139+0000 7f9e01630640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dfc103760 0x7f9dfc100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:52.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.139+0000 7f9e01630640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dfc1082e0 0x7f9dfc101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:52.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.139+0000 7f9e01630640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9dfc112050 con 0x7f9dfc103760 2024-09-19T09:23:52.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.139+0000 7f9e01630640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9dfc111ed0 con 0x7f9dfc1082e0 2024-09-19T09:23:52.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.139+0000 7f9e01630640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9dfc1121d0 con 0x7f9dfc102db0 2024-09-19T09:23:52.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.139+0000 7f9dfa7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dfc103760 0x7f9dfc100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:52.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.139+0000 7f9dfa7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dfc103760 0x7f9dfc100f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52012/0 (socket says 172.21.15.120:52012) 2024-09-19T09:23:52.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.139+0000 7f9dfa7fc640 1 -- 172.21.15.120:0/4089988709 learned_addr learned my addr 172.21.15.120:0/4089988709 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:52.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.139+0000 7f9dfaffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9dfc102db0 0x7f9dfc100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:52.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.139+0000 7f9dfb7fe640 1 --2- 172.21.15.120:0/4089988709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dfc1082e0 0x7f9dfc101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:52.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.139+0000 7f9dfb7fe640 1 -- 172.21.15.120:0/4089988709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9dfc102db0 msgr2=0x7f9dfc100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.139+0000 7f9dfb7fe640 1 --2- 172.21.15.120:0/4089988709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9dfc102db0 0x7f9dfc100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.139+0000 7f9dfb7fe640 1 -- 172.21.15.120:0/4089988709 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dfc103760 msgr2=0x7f9dfc100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.139+0000 7f9dfb7fe640 1 --2- 172.21.15.120:0/4089988709 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dfc103760 0x7f9dfc100f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.139+0000 7f9dfb7fe640 1 -- 172.21.15.120:0/4089988709 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dfc1a6a70 con 0x7f9dfc1082e0 2024-09-19T09:23:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.139+0000 7f9dfa7fc640 1 --2- 172.21.15.120:0/4089988709 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dfc103760 0x7f9dfc100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.139+0000 7f9dfaffd640 1 --2- 172.21.15.120:0/4089988709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9dfc102db0 0x7f9dfc100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.140+0000 7f9dfb7fe640 1 --2- 172.21.15.120:0/4089988709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dfc1082e0 0x7f9dfc101490 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9df400da10 tx=0x7f9df400ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.140+0000 7f9de3fff640 1 -- 172.21.15.120:0/4089988709 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df400bd70 con 0x7f9dfc1082e0 2024-09-19T09:23:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.140+0000 7f9e01630640 1 -- 172.21.15.120:0/4089988709 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dfc1a6d60 con 0x7f9dfc1082e0 2024-09-19T09:23:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.140+0000 7f9de3fff640 1 -- 172.21.15.120:0/4089988709 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9df4003e50 con 0x7f9dfc1082e0 2024-09-19T09:23:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.140+0000 7f9de3fff640 1 -- 172.21.15.120:0/4089988709 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df40106a0 con 0x7f9dfc1082e0 2024-09-19T09:23:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.140+0000 7f9e01630640 1 -- 172.21.15.120:0/4089988709 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dfc1a7220 con 0x7f9dfc1082e0 2024-09-19T09:23:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.141+0000 7f9e01630640 1 -- 172.21.15.120:0/4089988709 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dfc103190 con 0x7f9dfc1082e0 2024-09-19T09:23:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.142+0000 7f9de3fff640 1 -- 172.21.15.120:0/4089988709 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f9df4004b40 con 0x7f9dfc1082e0 2024-09-19T09:23:52.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.143+0000 7f9de3fff640 1 --2- 172.21.15.120:0/4089988709 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9de40a3240 0x7f9de40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:52.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.143+0000 7f9dfaffd640 1 --2- 172.21.15.120:0/4089988709 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9de40a3240 0x7f9de40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:52.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.143+0000 7f9de3fff640 1 -- 172.21.15.120:0/4089988709 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f9df40cfbd0 con 0x7f9dfc1082e0 2024-09-19T09:23:52.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.143+0000 7f9dfaffd640 1 --2- 172.21.15.120:0/4089988709 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9de40a3240 0x7f9de40a5700 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009fc0 tx=0x7f9dec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:52.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.149+0000 7f9de3fff640 1 -- 172.21.15.120:0/4089988709 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9df409c8f0 con 0x7f9dfc1082e0 2024-09-19T09:23:52.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:52 smithi120 ceph-mon[31150]: pgmap v975: 97 pgs: 97 active+clean; 453 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.7 MiB/s wr, 722 op/s 2024-09-19T09:23:52.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.369+0000 7f9e01630640 1 -- 172.21.15.120:0/4089988709 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9dfc101f40 con 0x7f9dfc1082e0 2024-09-19T09:23:52.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.369+0000 7f9de3fff640 1 -- 172.21.15.120:0/4089988709 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11988 (secure 0 0 0) 0x7f9dfc101f40 con 0x7f9dfc1082e0 2024-09-19T09:23:52.371 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:23:52.371 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":39,"btime":"2024-09-19T09:18:53:120858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:18:53.070639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:23:52.371 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 39 2024-09-19T09:23:52.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.372+0000 7f9e01630640 1 -- 172.21.15.120:0/4089988709 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9de40a3240 msgr2=0x7f9de40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:52.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.372+0000 7f9e01630640 1 --2- 172.21.15.120:0/4089988709 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9de40a3240 0x7f9de40a5700 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009fc0 tx=0x7f9dec005f90 comp rx=0 tx=0).stop 2024-09-19T09:23:52.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.372+0000 7f9e01630640 1 -- 172.21.15.120:0/4089988709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dfc1082e0 msgr2=0x7f9dfc101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:52.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.372+0000 7f9e01630640 1 --2- 172.21.15.120:0/4089988709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dfc1082e0 0x7f9dfc101490 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9df400da10 tx=0x7f9df400ded0 comp rx=0 tx=0).stop 2024-09-19T09:23:52.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.372+0000 7f9e01630640 1 -- 172.21.15.120:0/4089988709 shutdown_connections 2024-09-19T09:23:52.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.372+0000 7f9e01630640 1 --2- 172.21.15.120:0/4089988709 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9de40a3240 0x7f9de40a5700 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:52.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.372+0000 7f9e01630640 1 --2- 172.21.15.120:0/4089988709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dfc1082e0 0x7f9dfc101490 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:52.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.372+0000 7f9e01630640 1 --2- 172.21.15.120:0/4089988709 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9dfc103760 0x7f9dfc100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:52.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.372+0000 7f9e01630640 1 --2- 172.21.15.120:0/4089988709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9dfc102db0 0x7f9dfc100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:52.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.372+0000 7f9e01630640 1 -- 172.21.15.120:0/4089988709 >> 172.21.15.120:0/4089988709 conn(0x7f9dfc0fc7d0 msgr2=0x7f9dfc105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:52.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.372+0000 7f9e01630640 1 -- 172.21.15.120:0/4089988709 shutdown_connections 2024-09-19T09:23:52.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:52.372+0000 7f9e01630640 1 -- 172.21.15.120:0/4089988709 wait complete. 2024-09-19T09:23:52.378 INFO:tasks.workunit.client.0.smithi120.stdout: 10 192357 21.26 MB/sec execute 347 sec latency 71.905 ms 2024-09-19T09:23:52.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:52 smithi196 ceph-mon[35150]: pgmap v975: 97 pgs: 97 active+clean; 453 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.7 MiB/s wr, 722 op/s 2024-09-19T09:23:52.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:52 smithi153 ceph-mon[35071]: pgmap v975: 97 pgs: 97 active+clean; 453 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.7 MiB/s wr, 722 op/s 2024-09-19T09:23:53.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:53 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/4089988709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:23:53.378 INFO:tasks.workunit.client.0.smithi120.stdout: 10 192843 21.27 MB/sec execute 348 sec latency 84.540 ms 2024-09-19T09:23:53.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:53 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/4089988709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:23:53.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:53 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/4089988709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:23:53.709 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:23:54.008 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:23:54.321 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:54 smithi120 ceph-mon[31150]: pgmap v976: 97 pgs: 97 active+clean; 453 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 7.8 MiB/s wr, 726 op/s 2024-09-19T09:23:54.378 INFO:tasks.workunit.client.0.smithi120.stdout: 10 193463 21.32 MB/sec execute 349 sec latency 54.959 ms 2024-09-19T09:23:54.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:54 smithi196 ceph-mon[35150]: pgmap v976: 97 pgs: 97 active+clean; 453 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 7.8 MiB/s wr, 726 op/s 2024-09-19T09:23:54.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:54 smithi153 ceph-mon[35071]: pgmap v976: 97 pgs: 97 active+clean; 453 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 7.8 MiB/s wr, 726 op/s 2024-09-19T09:23:54.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.618+0000 7f7ca5dde640 1 -- 172.21.15.120:0/1704506281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ca01068d0 msgr2=0x7f7ca0106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:54.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.618+0000 7f7ca5dde640 1 --2- 172.21.15.120:0/1704506281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ca01068d0 0x7f7ca0106cb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009900 tx=0x7f7c9402a440 comp rx=0 tx=0).stop 2024-09-19T09:23:54.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.620+0000 7f7ca5dde640 1 -- 172.21.15.120:0/1704506281 shutdown_connections 2024-09-19T09:23:54.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.620+0000 7f7ca5dde640 1 --2- 172.21.15.120:0/1704506281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ca0101270 0x7f7ca0069f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.620+0000 7f7ca5dde640 1 --2- 172.21.15.120:0/1704506281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ca01008d0 0x7f7ca0100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.620+0000 7f7ca5dde640 1 --2- 172.21.15.120:0/1704506281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ca01068d0 0x7f7ca0106cb0 secure :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009900 tx=0x7f7c9402a440 comp rx=0 tx=0).stop 2024-09-19T09:23:54.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.620+0000 7f7ca5dde640 1 -- 172.21.15.120:0/1704506281 >> 172.21.15.120:0/1704506281 conn(0x7f7ca00fc610 msgr2=0x7f7ca00fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:54.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.624+0000 7f7ca5dde640 1 -- 172.21.15.120:0/1704506281 shutdown_connections 2024-09-19T09:23:54.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.625+0000 7f7ca5dde640 1 -- 172.21.15.120:0/1704506281 wait complete. 2024-09-19T09:23:54.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.626+0000 7f7ca5dde640 1 Processor -- start 2024-09-19T09:23:54.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.627+0000 7f7ca5dde640 1 -- start start 2024-09-19T09:23:54.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.627+0000 7f7ca5dde640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ca01008d0 0x7f7ca0073c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:54.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.627+0000 7f7ca5dde640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ca0101270 0x7f7ca006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:54.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.627+0000 7f7ca5dde640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ca006d1b0 0x7f7ca006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:54.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.627+0000 7f7ca5dde640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca010fd40 con 0x7f7ca0101270 2024-09-19T09:23:54.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.627+0000 7f7ca5dde640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca010fbc0 con 0x7f7ca01008d0 2024-09-19T09:23:54.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.627+0000 7f7ca5dde640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca010fec0 con 0x7f7ca006d1b0 2024-09-19T09:23:54.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.627+0000 7f7c9ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ca0101270 0x7f7ca006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:54.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.627+0000 7f7c9ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ca0101270 0x7f7ca006cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52022/0 (socket says 172.21.15.120:52022) 2024-09-19T09:23:54.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.627+0000 7f7c9ffff640 1 -- 172.21.15.120:0/798105599 learned_addr learned my addr 172.21.15.120:0/798105599 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:54.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.628+0000 7f7c9ffff640 1 -- 172.21.15.120:0/798105599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ca006d1b0 msgr2=0x7f7ca006d660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T09:23:54.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.628+0000 7f7c9ffff640 1 --2- 172.21.15.120:0/798105599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ca006d1b0 0x7f7ca006d660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.628+0000 7f7c9ffff640 1 -- 172.21.15.120:0/798105599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ca01008d0 msgr2=0x7f7ca0073c00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T09:23:54.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.628+0000 7f7ca4ddc640 1 --2- 172.21.15.120:0/798105599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ca01008d0 0x7f7ca0073c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:54.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.629+0000 7f7c9ffff640 1 --2- 172.21.15.120:0/798105599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ca01008d0 0x7f7ca0073c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.629+0000 7f7c9ffff640 1 -- 172.21.15.120:0/798105599 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ca0071830 con 0x7f7ca0101270 2024-09-19T09:23:54.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.629+0000 7f7ca4ddc640 1 --2- 172.21.15.120:0/798105599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ca01008d0 0x7f7ca0073c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:23:54.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.633+0000 7f7c9ffff640 1 --2- 172.21.15.120:0/798105599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ca0101270 0x7f7ca006cc70 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c00b450 tx=0x7f7c8c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:54.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.634+0000 7f7c9dffb640 1 -- 172.21.15.120:0/798105599 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c015020 con 0x7f7ca0101270 2024-09-19T09:23:54.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.634+0000 7f7ca5dde640 1 -- 172.21.15.120:0/798105599 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ca0071b20 con 0x7f7ca0101270 2024-09-19T09:23:54.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.634+0000 7f7ca5dde640 1 -- 172.21.15.120:0/798105599 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ca0072060 con 0x7f7ca0101270 2024-09-19T09:23:54.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.634+0000 7f7c9dffb640 1 -- 172.21.15.120:0/798105599 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c8c011d70 con 0x7f7ca0101270 2024-09-19T09:23:54.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.634+0000 7f7c9dffb640 1 -- 172.21.15.120:0/798105599 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c00f340 con 0x7f7ca0101270 2024-09-19T09:23:54.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.643+0000 7f7ca9419740 1 Processor -- start 2024-09-19T09:23:54.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.644+0000 7f7c9dffb640 1 -- 172.21.15.120:0/798105599 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f7c8c018020 con 0x7f7ca0101270 2024-09-19T09:23:54.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.649+0000 7f7c9dffb640 1 --2- 172.21.15.120:0/798105599 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7c900a3450 0x7f7c900a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:54.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.650+0000 7f7c9dffb640 1 -- 172.21.15.120:0/798105599 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f7c8c0cf2e0 con 0x7f7ca0101270 2024-09-19T09:23:54.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.650+0000 7f7ca55dd640 1 --2- 172.21.15.120:0/798105599 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7c900a3450 0x7f7c900a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:54.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.651+0000 7f7ca55dd640 1 --2- 172.21.15.120:0/798105599 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7c900a3450 0x7f7c900a5910 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800ab30 tx=0x7f7c98009250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:54.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.651+0000 7f7ca9419740 1 -- start start 2024-09-19T09:23:54.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.651+0000 7f7ca9419740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f7d650cf0 0x560f7d64f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:54.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.651+0000 7f7ca9419740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f7d64f780 0x560f7d64fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:54.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.651+0000 7f7ca9419740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f7d65a520 0x560f7d65c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:54.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.651+0000 7f7ca9419740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560f7d5b52f0 con 0x560f7d64f780 2024-09-19T09:23:54.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.651+0000 7f7ca9419740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560f7d55ccf0 con 0x560f7d65a520 2024-09-19T09:23:54.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.651+0000 7f7ca9419740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560f7d5b12b0 con 0x560f7d650cf0 2024-09-19T09:23:54.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.652+0000 7f7ca4ddc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f7d650cf0 0x560f7d64f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:54.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.652+0000 7f7ca4ddc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f7d650cf0 0x560f7d64f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:40834/0 (socket says 172.21.15.120:40834) 2024-09-19T09:23:54.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.652+0000 7f7ca4ddc640 1 -- 172.21.15.120:0/3537635074 learned_addr learned my addr 172.21.15.120:0/3537635074 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:54.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.652+0000 7f7ca55dd640 1 --2- 172.21.15.120:0/3537635074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f7d65a520 0x560f7d65c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:54.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.655+0000 7f7ca4ddc640 1 -- 172.21.15.120:0/3537635074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f7d65a520 msgr2=0x560f7d65c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:54.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.655+0000 7f7ca4ddc640 1 --2- 172.21.15.120:0/3537635074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f7d65a520 0x560f7d65c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.655+0000 7f7ca4ddc640 1 -- 172.21.15.120:0/3537635074 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f7d64f780 msgr2=0x560f7d64fbc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T09:23:54.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.655+0000 7f7ca4ddc640 1 --2- 172.21.15.120:0/3537635074 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f7d64f780 0x560f7d64fbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.655+0000 7f7ca4ddc640 1 -- 172.21.15.120:0/3537635074 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f7d5413a0 con 0x560f7d650cf0 2024-09-19T09:23:54.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.656+0000 7f7ca4ddc640 1 --2- 172.21.15.120:0/3537635074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f7d650cf0 0x560f7d64f240 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009850 tx=0x7f7c94031510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3dedf574ab9bf9c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:54.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.656+0000 7f7c8affd640 1 -- 172.21.15.120:0/3537635074 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c94061020 con 0x560f7d650cf0 2024-09-19T09:23:54.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.656+0000 7f7c8affd640 1 -- 172.21.15.120:0/3537635074 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c94005520 con 0x560f7d650cf0 2024-09-19T09:23:54.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.657+0000 7f7ca9419740 1 -- 172.21.15.120:0/3537635074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f7d650cf0 msgr2=0x560f7d64f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:54.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.657+0000 7f7ca9419740 1 --2- 172.21.15.120:0/3537635074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f7d650cf0 0x560f7d64f240 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009850 tx=0x7f7c94031510 comp rx=0 tx=0).stop 2024-09-19T09:23:54.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.658+0000 7f7ca9419740 1 -- 172.21.15.120:0/3537635074 shutdown_connections 2024-09-19T09:23:54.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.658+0000 7f7ca9419740 1 --2- 172.21.15.120:0/3537635074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f7d65a520 0x560f7d65c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.658+0000 7f7ca9419740 1 --2- 172.21.15.120:0/3537635074 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f7d64f780 0x560f7d64fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.658+0000 7f7ca9419740 1 --2- 172.21.15.120:0/3537635074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f7d650cf0 0x560f7d64f240 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.658+0000 7f7ca9419740 1 -- 172.21.15.120:0/3537635074 >> 172.21.15.120:0/3537635074 conn(0x560f7d618310 msgr2=0x560f7d64be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:54.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.660+0000 7f7ca9419740 1 -- 172.21.15.120:0/3537635074 shutdown_connections 2024-09-19T09:23:54.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.661+0000 7f7ca9419740 1 -- 172.21.15.120:0/3537635074 wait complete. 2024-09-19T09:23:54.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.662+0000 7f7ca9419740 1 -- start start 2024-09-19T09:23:54.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.662+0000 7f7ca9419740 1 Processor -- start 2024-09-19T09:23:54.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.662+0000 7f7ca9419740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f7d64f780 0x560f7d659df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:54.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.662+0000 7f7ca9419740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f7d650cf0 0x560f7d6570a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:54.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.662+0000 7f7ca9419740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f7d65a520 0x560f7d6575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:54.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.662+0000 7f7ca9419740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560f7d65d210 con 0x560f7d65a520 2024-09-19T09:23:54.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.662+0000 7f7ca9419740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560f7d570060 con 0x560f7d650cf0 2024-09-19T09:23:54.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.662+0000 7f7ca9419740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560f7d65d390 con 0x560f7d64f780 2024-09-19T09:23:54.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.662+0000 7f7ca9419740 10 client.0 _ll_register_callbacks cb 0x560f7d2c38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:54.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.663+0000 7f7ca4ddc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f7d64f780 0x560f7d659df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:54.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.663+0000 7f7ca4ddc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f7d64f780 0x560f7d659df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:40840/0 (socket says 172.21.15.120:40840) 2024-09-19T09:23:54.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.663+0000 7f7ca4ddc640 1 -- 172.21.15.120:0/2930431028 learned_addr learned my addr 172.21.15.120:0/2930431028 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:54.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.663+0000 7f7ca55dd640 1 --2- 172.21.15.120:0/2930431028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f7d65a520 0x560f7d6575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:54.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.664+0000 7f7ca55dd640 1 -- 172.21.15.120:0/2930431028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f7d64f780 msgr2=0x560f7d659df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:54.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.664+0000 7f7ca55dd640 1 --2- 172.21.15.120:0/2930431028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f7d64f780 0x560f7d659df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.664+0000 7f7ca55dd640 1 -- 172.21.15.120:0/2930431028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f7d650cf0 msgr2=0x560f7d6570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:54.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.664+0000 7f7ca55dd640 1 --2- 172.21.15.120:0/2930431028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f7d650cf0 0x560f7d6570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.664+0000 7f7ca55dd640 1 -- 172.21.15.120:0/2930431028 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f7d3263c0 con 0x560f7d65a520 2024-09-19T09:23:54.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.667+0000 7f7ca55dd640 1 --2- 172.21.15.120:0/2930431028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f7d65a520 0x560f7d6575e0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800c780 tx=0x7f7c98004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=659bf8eeaf79b0e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:54.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.669+0000 7f7c7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:23:54.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.669+0000 7f7c7ffff640 1 -- 172.21.15.120:0/2930431028 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c98024640 con 0x560f7d65a520 2024-09-19T09:23:54.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.669+0000 7f7ca9419740 1 -- 172.21.15.120:0/2930431028 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c94005520 con 0x560f7d65a520 2024-09-19T09:23:54.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.669+0000 7f7c7ffff640 1 -- 172.21.15.120:0/2930431028 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f7c980233e0 con 0x560f7d65a520 2024-09-19T09:23:54.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.671+0000 7f7c7ffff640 1 -- 172.21.15.120:0/2930431028 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7c9800de30 con 0x560f7d65a520 2024-09-19T09:23:54.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.671+0000 7f7ca9419740 10 client.22281 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:54.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.671+0000 7f7ca9419740 1 -- 172.21.15.120:0/2930431028 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560f7d668ce0 con 0x560f7d65a520 2024-09-19T09:23:54.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.671+0000 7f7c7ffff640 1 -- 172.21.15.120:0/2930431028 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c94005520 con 0x560f7d65a520 2024-09-19T09:23:54.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.671+0000 7f7c7ffff640 1 -- 172.21.15.120:0/2930431028 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f7c98022070 con 0x560f7d65a520 2024-09-19T09:23:54.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.671+0000 7f7ca9419740 10 client.22281 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:54.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.671+0000 7f7ca9419740 10 client.22281 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:23:54.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.671+0000 7f7ca9419740 20 client.22281 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:54.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.671+0000 7f7ca9419740 1 --2- 172.21.15.120:0/2930431028 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560f7d66a5c0 0x560f7d66ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:54.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.671+0000 7f7ca9419740 4 client.22281 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:23:54.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.671+0000 7f7ca9419740 1 -- 172.21.15.120:0/2930431028 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f7d5c2860 con 0x560f7d66a5c0 2024-09-19T09:23:54.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.672+0000 7f7ca4ddc640 1 --2- 172.21.15.120:0/2930431028 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560f7d66a5c0 0x560f7d66ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:54.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.672+0000 7f7ca4ddc640 1 --2- 172.21.15.120:0/2930431028 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560f7d66a5c0 0x560f7d66ca80 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d8e25eaf07e65a8b server_cookie=a65c161335ecd253 in_seq=0 out_seq=0 2024-09-19T09:23:54.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.672+0000 7f7c7ffff640 10 client.22281 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:23:54.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.674+0000 7f7c7ffff640 1 -- 172.21.15.120:0/2930431028 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f7d5c2860 con 0x560f7d66a5c0 2024-09-19T09:23:54.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.674+0000 7f7c7ffff640 10 client.22281 handle_command_reply: tid=0 2024-09-19T09:23:54.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.674+0000 7f7ca9419740 1 client.22281 shutdown 2024-09-19T09:23:54.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.674+0000 7f7ca9419740 1 -- 172.21.15.120:0/2930431028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f7d65a520 msgr2=0x560f7d6575e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:54.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.674+0000 7f7ca9419740 1 --2- 172.21.15.120:0/2930431028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f7d65a520 0x560f7d6575e0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800c780 tx=0x7f7c98004ad0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.674+0000 7f7ca9419740 1 -- 172.21.15.120:0/2930431028 shutdown_connections 2024-09-19T09:23:54.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.674+0000 7f7ca9419740 1 --2- 172.21.15.120:0/2930431028 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560f7d66a5c0 0x560f7d66ca80 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.674+0000 7f7ca9419740 1 --2- 172.21.15.120:0/2930431028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f7d65a520 0x560f7d6575e0 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.674+0000 7f7ca9419740 1 --2- 172.21.15.120:0/2930431028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f7d650cf0 0x560f7d6570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.674+0000 7f7ca9419740 1 --2- 172.21.15.120:0/2930431028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f7d64f780 0x560f7d659df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.674+0000 7f7ca9419740 1 -- 172.21.15.120:0/2930431028 >> 172.21.15.120:0/2930431028 conn(0x560f7d618310 msgr2=0x560f7d651ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:54.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.674+0000 7f7c7ffff640 0 client.22281 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:23:54.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.675+0000 7f7ca9419740 1 -- 172.21.15.120:0/2930431028 shutdown_connections 2024-09-19T09:23:54.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.675+0000 7f7ca9419740 1 -- 172.21.15.120:0/2930431028 wait complete. 2024-09-19T09:23:54.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.675+0000 7f7ca9419740 20 client.22281 trim_cache size 0 max 16384 2024-09-19T09:23:54.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.690+0000 7f7ca9419740 1 Processor -- start 2024-09-19T09:23:54.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.690+0000 7f7ca9419740 1 -- start start 2024-09-19T09:23:54.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.691+0000 7f7ca9419740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f7d64f780 0x560f7d666590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:54.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.691+0000 7f7ca9419740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f7d650cf0 0x560f7d672390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:54.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.691+0000 7f7ca9419740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f7d65a520 0x560f7d6728d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:54.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.691+0000 7f7ca9419740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8400e5e0 con 0x560f7d650cf0 2024-09-19T09:23:54.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.691+0000 7f7ca9419740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7c84010120 con 0x560f7d64f780 2024-09-19T09:23:54.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.691+0000 7f7ca9419740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8400ecb0 con 0x560f7d65a520 2024-09-19T09:23:54.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.691+0000 7f7ca4ddc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f7d64f780 0x560f7d666590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:54.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.691+0000 7f7ca4ddc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f7d64f780 0x560f7d666590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:54206/0 (socket says 172.21.15.120:54206) 2024-09-19T09:23:54.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.691+0000 7f7ca4ddc640 1 -- 172.21.15.120:0/32209190 learned_addr learned my addr 172.21.15.120:0/32209190 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:54.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.691+0000 7f7c9ffff640 1 --2- 172.21.15.120:0/32209190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f7d650cf0 0x560f7d672390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:54.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.692+0000 7f7ca55dd640 1 --2- 172.21.15.120:0/32209190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f7d65a520 0x560f7d6728d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:54.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.692+0000 7f7ca55dd640 1 -- 172.21.15.120:0/32209190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f7d64f780 msgr2=0x560f7d666590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:54.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.692+0000 7f7ca55dd640 1 --2- 172.21.15.120:0/32209190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f7d64f780 0x560f7d666590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.692+0000 7f7ca55dd640 1 -- 172.21.15.120:0/32209190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f7d650cf0 msgr2=0x560f7d672390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:54.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.692+0000 7f7ca55dd640 1 --2- 172.21.15.120:0/32209190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f7d650cf0 0x560f7d672390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.692+0000 7f7ca55dd640 1 -- 172.21.15.120:0/32209190 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f7d64c350 con 0x560f7d65a520 2024-09-19T09:23:54.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.693+0000 7f7ca55dd640 1 --2- 172.21.15.120:0/32209190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f7d65a520 0x560f7d6728d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800d010 tx=0x7f7c9800c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37111458a85a2883 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:54.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.693+0000 7f7c8a7fc640 1 -- 172.21.15.120:0/32209190 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c98003a10 con 0x560f7d65a520 2024-09-19T09:23:54.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.693+0000 7f7c8a7fc640 1 -- 172.21.15.120:0/32209190 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c98003bb0 con 0x560f7d65a520 2024-09-19T09:23:54.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.693+0000 7f7c8a7fc640 1 -- 172.21.15.120:0/32209190 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c980042c0 con 0x560f7d65a520 2024-09-19T09:23:54.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.694+0000 7f7ca9419740 1 -- 172.21.15.120:0/32209190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f7d65a520 msgr2=0x560f7d6728d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:54.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.694+0000 7f7ca9419740 1 --2- 172.21.15.120:0/32209190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f7d65a520 0x560f7d6728d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800d010 tx=0x7f7c9800c8d0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.694+0000 7f7ca9419740 1 -- 172.21.15.120:0/32209190 shutdown_connections 2024-09-19T09:23:54.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.694+0000 7f7ca9419740 1 --2- 172.21.15.120:0/32209190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f7d65a520 0x560f7d6728d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.694+0000 7f7ca9419740 1 --2- 172.21.15.120:0/32209190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f7d650cf0 0x560f7d672390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.694+0000 7f7ca9419740 1 --2- 172.21.15.120:0/32209190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f7d64f780 0x560f7d666590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.694+0000 7f7ca9419740 1 -- 172.21.15.120:0/32209190 >> 172.21.15.120:0/32209190 conn(0x560f7d618310 msgr2=0x560f7d655970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:54.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.694+0000 7f7ca9419740 1 -- 172.21.15.120:0/32209190 shutdown_connections 2024-09-19T09:23:54.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.694+0000 7f7ca9419740 1 -- 172.21.15.120:0/32209190 wait complete. 2024-09-19T09:23:54.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.695+0000 7f7ca9419740 1 -- start start 2024-09-19T09:23:54.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.695+0000 7f7ca9419740 1 Processor -- start 2024-09-19T09:23:54.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.696+0000 7f7ca9419740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f7d64f780 0x560f7d6627d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:54.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.696+0000 7f7ca9419740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f7d650cf0 0x560f7d662d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:54.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.696+0000 7f7ca9419740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f7d65a520 0x560f7d65dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:54.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.696+0000 7f7ca4ddc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f7d64f780 0x560f7d6627d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:54.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.696+0000 7f7ca4ddc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f7d64f780 0x560f7d6627d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:40858/0 (socket says 172.21.15.120:40858) 2024-09-19T09:23:54.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.696+0000 7f7c9ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f7d650cf0 0x560f7d662d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:54.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.696+0000 7f7ca4ddc640 1 -- 172.21.15.120:0/1833863236 learned_addr learned my addr 172.21.15.120:0/1833863236 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:54.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.696+0000 7f7ca9419740 1 -- 172.21.15.120:0/1833863236 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8400fa50 con 0x560f7d650cf0 2024-09-19T09:23:54.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.696+0000 7f7ca9419740 1 -- 172.21.15.120:0/1833863236 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8400f380 con 0x560f7d65a520 2024-09-19T09:23:54.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.696+0000 7f7ca9419740 1 -- 172.21.15.120:0/1833863236 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8400c5a0 con 0x560f7d64f780 2024-09-19T09:23:54.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.696+0000 7f7ca9419740 10 client.0 _ll_register_callbacks cb 0x7f7c9800cd10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:54.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.696+0000 7f7ca55dd640 1 --2- 172.21.15.120:0/1833863236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f7d65a520 0x560f7d65dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:54.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.697+0000 7f7ca4ddc640 1 -- 172.21.15.120:0/1833863236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f7d65a520 msgr2=0x560f7d65dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:54.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.697+0000 7f7ca4ddc640 1 --2- 172.21.15.120:0/1833863236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f7d65a520 0x560f7d65dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.697+0000 7f7ca4ddc640 1 -- 172.21.15.120:0/1833863236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f7d650cf0 msgr2=0x560f7d662d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:54.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.697+0000 7f7ca4ddc640 1 --2- 172.21.15.120:0/1833863236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f7d650cf0 0x560f7d662d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.697+0000 7f7ca4ddc640 1 -- 172.21.15.120:0/1833863236 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f7d64b6a0 con 0x560f7d64f780 2024-09-19T09:23:54.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.697+0000 7f7c9ffff640 1 --2- 172.21.15.120:0/1833863236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f7d650cf0 0x560f7d662d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:54.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.697+0000 7f7ca4ddc640 1 --2- 172.21.15.120:0/1833863236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f7d64f780 0x560f7d6627d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7c94000c00 tx=0x7f7c94004ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9480c912d4585ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:54.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.697+0000 7f7c7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:23:54.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.697+0000 7f7c7ffff640 1 -- 172.21.15.120:0/1833863236 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c94061040 con 0x560f7d64f780 2024-09-19T09:23:54.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.697+0000 7f7ca9419740 1 -- 172.21.15.120:0/1833863236 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c98003bb0 con 0x560f7d64f780 2024-09-19T09:23:54.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.697+0000 7f7c7ffff640 1 -- 172.21.15.120:0/1833863236 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f7c94004f60 con 0x560f7d64f780 2024-09-19T09:23:54.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.698+0000 7f7c7ffff640 1 -- 172.21.15.120:0/1833863236 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7c94005160 con 0x560f7d64f780 2024-09-19T09:23:54.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.698+0000 7f7ca9419740 10 client.30907 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:54.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.698+0000 7f7ca9419740 1 -- 172.21.15.120:0/1833863236 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560f7d6d4ce0 con 0x560f7d64f780 2024-09-19T09:23:54.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.698+0000 7f7c7ffff640 1 -- 172.21.15.120:0/1833863236 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c98003bb0 con 0x560f7d64f780 2024-09-19T09:23:54.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.698+0000 7f7c7ffff640 1 -- 172.21.15.120:0/1833863236 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f7c94070050 con 0x560f7d64f780 2024-09-19T09:23:54.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.698+0000 7f7ca9419740 10 client.30907 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:54.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.698+0000 7f7ca9419740 10 client.30907 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:23:54.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.698+0000 7f7ca9419740 20 client.30907 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:54.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.698+0000 7f7ca9419740 1 --2- 172.21.15.120:0/1833863236 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560f7d6d4fd0 0x560f7d6d73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:54.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.699+0000 7f7ca9419740 4 client.30907 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:23:54.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.699+0000 7f7ca9419740 1 -- 172.21.15.120:0/1833863236 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f7d5bcd70 con 0x560f7d6d4fd0 2024-09-19T09:23:54.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.699+0000 7f7c9ffff640 1 --2- 172.21.15.120:0/1833863236 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560f7d6d4fd0 0x560f7d6d73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:54.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.699+0000 7f7c7ffff640 10 client.30907 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:23:54.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.699+0000 7f7c9ffff640 1 --2- 172.21.15.120:0/1833863236 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560f7d6d4fd0 0x560f7d6d73c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1d2c54bc1e2d5391 server_cookie=7239efadaa66ce10 in_seq=0 out_seq=0 2024-09-19T09:23:54.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.700+0000 7f7c7ffff640 1 -- 172.21.15.120:0/1833863236 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f7d5bcd70 con 0x560f7d6d4fd0 2024-09-19T09:23:54.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.700+0000 7f7c7ffff640 10 client.30907 handle_command_reply: tid=0 2024-09-19T09:23:54.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.700+0000 7f7ca9419740 1 client.30907 shutdown 2024-09-19T09:23:54.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.700+0000 7f7ca9419740 1 -- 172.21.15.120:0/1833863236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f7d64f780 msgr2=0x560f7d6627d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:54.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.700+0000 7f7ca9419740 1 --2- 172.21.15.120:0/1833863236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f7d64f780 0x560f7d6627d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7c94000c00 tx=0x7f7c94004ba0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.700+0000 7f7ca9419740 1 -- 172.21.15.120:0/1833863236 shutdown_connections 2024-09-19T09:23:54.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.700+0000 7f7ca9419740 1 --2- 172.21.15.120:0/1833863236 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x560f7d6d4fd0 0x560f7d6d73c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.700+0000 7f7ca9419740 1 --2- 172.21.15.120:0/1833863236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f7d65a520 0x560f7d65dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.700+0000 7f7ca9419740 1 --2- 172.21.15.120:0/1833863236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560f7d650cf0 0x560f7d662d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.700+0000 7f7ca9419740 1 --2- 172.21.15.120:0/1833863236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f7d64f780 0x560f7d6627d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.701+0000 7f7ca9419740 1 -- 172.21.15.120:0/1833863236 >> 172.21.15.120:0/1833863236 conn(0x560f7d618310 msgr2=0x560f7d64aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:54.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.700+0000 7f7c7ffff640 0 client.30907 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:23:54.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.701+0000 7f7ca9419740 1 -- 172.21.15.120:0/1833863236 shutdown_connections 2024-09-19T09:23:54.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.701+0000 7f7ca9419740 1 -- 172.21.15.120:0/1833863236 wait complete. 2024-09-19T09:23:54.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.701+0000 7f7ca9419740 20 client.30907 trim_cache size 0 max 16384 2024-09-19T09:23:54.706 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:23:54.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.702+0000 7f7ca5dde640 1 -- 172.21.15.120:0/798105599 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7c900a3450 msgr2=0x7f7c900a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:54.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.702+0000 7f7ca5dde640 1 --2- 172.21.15.120:0/798105599 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7c900a3450 0x7f7c900a5910 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800ab30 tx=0x7f7c98009250 comp rx=0 tx=0).stop 2024-09-19T09:23:54.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.702+0000 7f7ca5dde640 1 -- 172.21.15.120:0/798105599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ca0101270 msgr2=0x7f7ca006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:54.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.702+0000 7f7ca5dde640 1 --2- 172.21.15.120:0/798105599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ca0101270 0x7f7ca006cc70 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c00b450 tx=0x7f7c8c00b910 comp rx=0 tx=0).stop 2024-09-19T09:23:54.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.702+0000 7f7ca5dde640 1 -- 172.21.15.120:0/798105599 shutdown_connections 2024-09-19T09:23:54.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.702+0000 7f7ca5dde640 1 --2- 172.21.15.120:0/798105599 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7c900a3450 0x7f7c900a5910 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.702+0000 7f7ca5dde640 1 --2- 172.21.15.120:0/798105599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ca006d1b0 0x7f7ca006d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.702+0000 7f7ca5dde640 1 --2- 172.21.15.120:0/798105599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ca0101270 0x7f7ca006cc70 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.702+0000 7f7ca5dde640 1 --2- 172.21.15.120:0/798105599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ca01008d0 0x7f7ca0073c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:54.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.702+0000 7f7ca5dde640 1 -- 172.21.15.120:0/798105599 >> 172.21.15.120:0/798105599 conn(0x7f7ca00fc610 msgr2=0x7f7ca010d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:54.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.702+0000 7f7ca5dde640 1 -- 172.21.15.120:0/798105599 shutdown_connections 2024-09-19T09:23:54.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:54.703+0000 7f7ca5dde640 1 -- 172.21.15.120:0/798105599 wait complete. 2024-09-19T09:23:55.378 INFO:tasks.workunit.client.0.smithi120.stdout: 10 193803 21.28 MB/sec execute 350 sec latency 346.078 ms 2024-09-19T09:23:55.972 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:23:56.242 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:23:56.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:56 smithi120 ceph-mon[31150]: pgmap v977: 97 pgs: 97 active+clean; 456 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.7 MiB/s wr, 612 op/s 2024-09-19T09:23:56.378 INFO:tasks.workunit.client.0.smithi120.stdout: 10 194492 21.31 MB/sec execute 351 sec latency 92.831 ms 2024-09-19T09:23:56.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:56 smithi196 ceph-mon[35150]: pgmap v977: 97 pgs: 97 active+clean; 456 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.7 MiB/s wr, 612 op/s 2024-09-19T09:23:56.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:56 smithi153 ceph-mon[35071]: pgmap v977: 97 pgs: 97 active+clean; 456 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.7 MiB/s wr, 612 op/s 2024-09-19T09:23:56.676 INFO:tasks.quiescer.fs.[cephfs]:Total duration (49.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T09:23:56.676 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-09-19T09:23:56.676 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:23:56.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.791+0000 7f1f8dd4b640 1 -- 172.21.15.120:0/3252638597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f881082a0 msgr2=0x7f1f8810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:56.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.791+0000 7f1f8dd4b640 1 --2- 172.21.15.120:0/3252638597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f881082a0 0x7f1f8810ec40 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009cd0 tx=0x7f1f8002a8e0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.792+0000 7f1f8dd4b640 1 -- 172.21.15.120:0/3252638597 shutdown_connections 2024-09-19T09:23:56.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.792+0000 7f1f8dd4b640 1 --2- 172.21.15.120:0/3252638597 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f881082a0 0x7f1f8810ec40 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.792+0000 7f1f8dd4b640 1 --2- 172.21.15.120:0/3252638597 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f881036e0 0x7f1f88107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.792+0000 7f1f8dd4b640 1 --2- 172.21.15.120:0/3252638597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f88102d30 0x7f1f88103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.792+0000 7f1f8dd4b640 1 -- 172.21.15.120:0/3252638597 >> 172.21.15.120:0/3252638597 conn(0x7f1f880fc7d0 msgr2=0x7f1f880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:56.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.793+0000 7f1f8dd4b640 1 -- 172.21.15.120:0/3252638597 shutdown_connections 2024-09-19T09:23:56.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.793+0000 7f1f8dd4b640 1 -- 172.21.15.120:0/3252638597 wait complete. 2024-09-19T09:23:56.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.793+0000 7f1f8dd4b640 1 Processor -- start 2024-09-19T09:23:56.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.793+0000 7f1f8dd4b640 1 -- start start 2024-09-19T09:23:56.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.794+0000 7f1f8dd4b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f88102d30 0x7f1f88100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:56.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.794+0000 7f1f8dd4b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f881036e0 0x7f1f88100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:56.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.794+0000 7f1f8dd4b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f881082a0 0x7f1f88101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:56.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.794+0000 7f1f8dd4b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1f881120d0 con 0x7f1f881036e0 2024-09-19T09:23:56.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.794+0000 7f1f8dd4b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1f88111f50 con 0x7f1f881082a0 2024-09-19T09:23:56.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.794+0000 7f1f8dd4b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1f88112250 con 0x7f1f88102d30 2024-09-19T09:23:56.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.794+0000 7f1f86ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f881036e0 0x7f1f88100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:56.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.794+0000 7f1f877fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f88102d30 0x7f1f88100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:56.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.794+0000 7f1f86ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f881036e0 0x7f1f88100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52064/0 (socket says 172.21.15.120:52064) 2024-09-19T09:23:56.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.794+0000 7f1f86ffd640 1 -- 172.21.15.120:0/2297944582 learned_addr learned my addr 172.21.15.120:0/2297944582 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:56.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.794+0000 7f1f87fff640 1 --2- 172.21.15.120:0/2297944582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f881082a0 0x7f1f88101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:56.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.794+0000 7f1f86ffd640 1 -- 172.21.15.120:0/2297944582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f88102d30 msgr2=0x7f1f88100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:56.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.794+0000 7f1f86ffd640 1 --2- 172.21.15.120:0/2297944582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f88102d30 0x7f1f88100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.794+0000 7f1f86ffd640 1 -- 172.21.15.120:0/2297944582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f881082a0 msgr2=0x7f1f88101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:56.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.794+0000 7f1f86ffd640 1 --2- 172.21.15.120:0/2297944582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f881082a0 0x7f1f88101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.794+0000 7f1f86ffd640 1 -- 172.21.15.120:0/2297944582 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f881a6a70 con 0x7f1f881036e0 2024-09-19T09:23:56.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.794+0000 7f1f877fe640 1 --2- 172.21.15.120:0/2297944582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f88102d30 0x7f1f88100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:56.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.794+0000 7f1f87fff640 1 --2- 172.21.15.120:0/2297944582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f881082a0 0x7f1f88101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:56.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.795+0000 7f1f86ffd640 1 --2- 172.21.15.120:0/2297944582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f881036e0 0x7f1f88100ed0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00e9f0 tx=0x7f1f7c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:56.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.795+0000 7f1f84ff9640 1 -- 172.21.15.120:0/2297944582 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c00ccc0 con 0x7f1f881036e0 2024-09-19T09:23:56.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.795+0000 7f1f8dd4b640 1 -- 172.21.15.120:0/2297944582 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f881a6d60 con 0x7f1f881036e0 2024-09-19T09:23:56.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.795+0000 7f1f84ff9640 1 -- 172.21.15.120:0/2297944582 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f7c010910 con 0x7f1f881036e0 2024-09-19T09:23:56.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.796+0000 7f1f84ff9640 1 -- 172.21.15.120:0/2297944582 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c0188c0 con 0x7f1f881036e0 2024-09-19T09:23:56.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.796+0000 7f1f8dd4b640 1 -- 172.21.15.120:0/2297944582 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f881a7260 con 0x7f1f881036e0 2024-09-19T09:23:56.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.798+0000 7f1f84ff9640 1 -- 172.21.15.120:0/2297944582 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f1f7c010ac0 con 0x7f1f881036e0 2024-09-19T09:23:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.798+0000 7f1f90124740 1 Processor -- start 2024-09-19T09:23:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.798+0000 7f1f90124740 1 -- start start 2024-09-19T09:23:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.798+0000 7f1f90124740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f21a7a9b40 0x55f21a7a8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.798+0000 7f1f90124740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f21a7a85d0 0x55f21a7a8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.798+0000 7f1f90124740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f21a7b3370 0x55f21a7b5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.798+0000 7f1f90124740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f21a70e2f0 con 0x55f21a7a85d0 2024-09-19T09:23:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.798+0000 7f1f90124740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f21a6b5cf0 con 0x55f21a7b3370 2024-09-19T09:23:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.798+0000 7f1f90124740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f21a70a2b0 con 0x55f21a7a9b40 2024-09-19T09:23:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.798+0000 7f1f877fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f21a7a85d0 0x55f21a7a8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.798+0000 7f1f877fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f21a7a85d0 0x55f21a7a8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52080/0 (socket says 172.21.15.120:52080) 2024-09-19T09:23:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.798+0000 7f1f87fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f21a7a9b40 0x55f21a7a8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.798+0000 7f1f877fe640 1 -- 172.21.15.120:0/1556229682 learned_addr learned my addr 172.21.15.120:0/1556229682 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:56.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.799+0000 7f1f84ff9640 1 --2- 172.21.15.120:0/2297944582 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1f700a3240 0x7f1f700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.799+0000 7f1f86ffd640 1 --2- 172.21.15.120:0/1556229682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f21a7b3370 0x55f21a7b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.799+0000 7f1f87fff640 1 --2- 172.21.15.120:0/2297944582 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1f700a3240 0x7f1f700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.799+0000 7f1f87fff640 1 -- 172.21.15.120:0/1556229682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f21a7b3370 msgr2=0x55f21a7b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.799+0000 7f1f87fff640 1 --2- 172.21.15.120:0/1556229682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f21a7b3370 0x55f21a7b5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.799+0000 7f1f87fff640 1 -- 172.21.15.120:0/1556229682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f21a7a85d0 msgr2=0x55f21a7a8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.799+0000 7f1f87fff640 1 --2- 172.21.15.120:0/1556229682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f21a7a85d0 0x55f21a7a8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.799+0000 7f1f87fff640 1 -- 172.21.15.120:0/1556229682 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f21a69a3a0 con 0x55f21a7a9b40 2024-09-19T09:23:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.799+0000 7f1f84ff9640 1 -- 172.21.15.120:0/2297944582 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f1f7c0cfaf0 con 0x7f1f881036e0 2024-09-19T09:23:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.799+0000 7f1f87fff640 1 --2- 172.21.15.120:0/1556229682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f21a7a9b40 0x55f21a7a8090 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009ca0 tx=0x7f1f80005fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8d2c3e2272bffb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.799+0000 7f1f6dffb640 1 -- 172.21.15.120:0/1556229682 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80003e00 con 0x55f21a7a9b40 2024-09-19T09:23:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.799+0000 7f1f6dffb640 1 -- 172.21.15.120:0/1556229682 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f8004e070 con 0x55f21a7a9b40 2024-09-19T09:23:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.799+0000 7f1f6dffb640 1 -- 172.21.15.120:0/1556229682 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80030e10 con 0x55f21a7a9b40 2024-09-19T09:23:56.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.800+0000 7f1f90124740 1 -- 172.21.15.120:0/1556229682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f21a7a9b40 msgr2=0x55f21a7a8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:56.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.800+0000 7f1f87fff640 1 --2- 172.21.15.120:0/2297944582 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1f700a3240 0x7f1f700a5700 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1f8004a000 tx=0x7f1f8004b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:56.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.800+0000 7f1f90124740 1 --2- 172.21.15.120:0/1556229682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f21a7a9b40 0x55f21a7a8090 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009ca0 tx=0x7f1f80005fc0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.800+0000 7f1f90124740 1 -- 172.21.15.120:0/1556229682 shutdown_connections 2024-09-19T09:23:56.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.800+0000 7f1f90124740 1 --2- 172.21.15.120:0/1556229682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f21a7b3370 0x55f21a7b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.800+0000 7f1f90124740 1 --2- 172.21.15.120:0/1556229682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f21a7a85d0 0x55f21a7a8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.800+0000 7f1f90124740 1 --2- 172.21.15.120:0/1556229682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f21a7a9b40 0x55f21a7a8090 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.800+0000 7f1f90124740 1 -- 172.21.15.120:0/1556229682 >> 172.21.15.120:0/1556229682 conn(0x55f21a771310 msgr2=0x55f21a7a4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:56.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.800+0000 7f1f90124740 1 -- 172.21.15.120:0/1556229682 shutdown_connections 2024-09-19T09:23:56.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.800+0000 7f1f90124740 1 -- 172.21.15.120:0/1556229682 wait complete. 2024-09-19T09:23:56.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.801+0000 7f1f90124740 1 -- start start 2024-09-19T09:23:56.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.801+0000 7f1f90124740 1 Processor -- start 2024-09-19T09:23:56.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.801+0000 7f1f90124740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f21a7a85d0 0x55f21a7b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:56.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.801+0000 7f1f90124740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f21a7a9b40 0x55f21a7b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:56.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.801+0000 7f1f90124740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f21a7b3370 0x55f21a7b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:56.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.801+0000 7f1f90124740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f21a7b6070 con 0x55f21a7b3370 2024-09-19T09:23:56.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.802+0000 7f1f90124740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f21a6c9060 con 0x55f21a7a9b40 2024-09-19T09:23:56.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.802+0000 7f1f90124740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f21a7b61f0 con 0x55f21a7a85d0 2024-09-19T09:23:56.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.802+0000 7f1f86ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f21a7a9b40 0x55f21a7b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:56.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.802+0000 7f1f90124740 10 client.0 _ll_register_callbacks cb 0x55f21a41c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:56.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.802+0000 7f1f87fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f21a7b3370 0x55f21a7b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:56.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.802+0000 7f1f86ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f21a7a9b40 0x55f21a7b00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:54250/0 (socket says 172.21.15.120:54250) 2024-09-19T09:23:56.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.802+0000 7f1f86ffd640 1 -- 172.21.15.120:0/3792184383 learned_addr learned my addr 172.21.15.120:0/3792184383 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:56.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.802+0000 7f1f877fe640 1 --2- 172.21.15.120:0/3792184383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f21a7a85d0 0x55f21a7b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:56.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.802+0000 7f1f87fff640 1 -- 172.21.15.120:0/3792184383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f21a7a85d0 msgr2=0x55f21a7b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:56.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.802+0000 7f1f87fff640 1 --2- 172.21.15.120:0/3792184383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f21a7a85d0 0x55f21a7b2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.802+0000 7f1f87fff640 1 -- 172.21.15.120:0/3792184383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f21a7a9b40 msgr2=0x55f21a7b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:56.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.802+0000 7f1f87fff640 1 --2- 172.21.15.120:0/3792184383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f21a7a9b40 0x55f21a7b00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.802+0000 7f1f87fff640 1 -- 172.21.15.120:0/3792184383 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f21a47f3c0 con 0x55f21a7b3370 2024-09-19T09:23:56.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.802+0000 7f1f877fe640 1 --2- 172.21.15.120:0/3792184383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f21a7a85d0 0x55f21a7b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:56.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.802+0000 7f1f86ffd640 1 --2- 172.21.15.120:0/3792184383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f21a7a9b40 0x55f21a7b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:56.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.802+0000 7f1f87fff640 1 --2- 172.21.15.120:0/3792184383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f21a7b3370 0x55f21a7b07e0 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f1f80005e30 tx=0x7f1f80009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d60cf82483f3de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:56.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.802+0000 7f1f66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:23:56.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.802+0000 7f1f66ffd640 1 -- 172.21.15.120:0/3792184383 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80031b80 con 0x55f21a7b3370 2024-09-19T09:23:56.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.802+0000 7f1f66ffd640 1 -- 172.21.15.120:0/3792184383 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f1f8002e030 con 0x55f21a7b3370 2024-09-19T09:23:56.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.802+0000 7f1f90124740 1 -- 172.21.15.120:0/3792184383 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f8004e070 con 0x55f21a7b3370 2024-09-19T09:23:56.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.803+0000 7f1f66ffd640 1 -- 172.21.15.120:0/3792184383 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1f80003e00 con 0x55f21a7b3370 2024-09-19T09:23:56.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.803+0000 7f1f90124740 10 client.22299 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:56.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.803+0000 7f1f90124740 1 -- 172.21.15.120:0/3792184383 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f21a7c1ba0 con 0x55f21a7b3370 2024-09-19T09:23:56.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.803+0000 7f1f66ffd640 1 -- 172.21.15.120:0/3792184383 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f8004e070 con 0x55f21a7b3370 2024-09-19T09:23:56.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.803+0000 7f1f66ffd640 1 -- 172.21.15.120:0/3792184383 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f1f80030070 con 0x55f21a7b3370 2024-09-19T09:23:56.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.803+0000 7f1f90124740 10 client.22299 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:56.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.803+0000 7f1f90124740 10 client.22299 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:23:56.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.803+0000 7f1f90124740 20 client.22299 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:56.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.803+0000 7f1f90124740 1 --2- 172.21.15.120:0/3792184383 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55f21a7c2ef0 0x55f21a7c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:56.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.803+0000 7f1f90124740 4 client.22299 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:23:56.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.803+0000 7f1f90124740 1 -- 172.21.15.120:0/3792184383 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f21a71b860 con 0x55f21a7c2ef0 2024-09-19T09:23:56.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.803+0000 7f1f877fe640 1 --2- 172.21.15.120:0/3792184383 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55f21a7c2ef0 0x55f21a7c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:56.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.804+0000 7f1f877fe640 1 --2- 172.21.15.120:0/3792184383 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55f21a7c2ef0 0x55f21a7c52e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=53f29afbe9711d8d server_cookie=894f2624b1680c70 in_seq=0 out_seq=0 2024-09-19T09:23:56.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.804+0000 7f1f66ffd640 10 client.22299 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:23:56.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.804+0000 7f1f66ffd640 1 -- 172.21.15.120:0/3792184383 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f21a71b860 con 0x55f21a7c2ef0 2024-09-19T09:23:56.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.804+0000 7f1f66ffd640 10 client.22299 handle_command_reply: tid=0 2024-09-19T09:23:56.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.804+0000 7f1f90124740 1 client.22299 shutdown 2024-09-19T09:23:56.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.805+0000 7f1f90124740 1 -- 172.21.15.120:0/3792184383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f21a7b3370 msgr2=0x55f21a7b07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:56.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.805+0000 7f1f90124740 1 --2- 172.21.15.120:0/3792184383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f21a7b3370 0x55f21a7b07e0 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f1f80005e30 tx=0x7f1f80009360 comp rx=0 tx=0).stop 2024-09-19T09:23:56.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.805+0000 7f1f90124740 1 -- 172.21.15.120:0/3792184383 shutdown_connections 2024-09-19T09:23:56.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.805+0000 7f1f90124740 1 --2- 172.21.15.120:0/3792184383 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55f21a7c2ef0 0x55f21a7c52e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.805+0000 7f1f90124740 1 --2- 172.21.15.120:0/3792184383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f21a7b3370 0x55f21a7b07e0 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.805+0000 7f1f90124740 1 --2- 172.21.15.120:0/3792184383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f21a7a9b40 0x55f21a7b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.805+0000 7f1f90124740 1 --2- 172.21.15.120:0/3792184383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f21a7a85d0 0x55f21a7b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.805+0000 7f1f66ffd640 0 client.22299 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:23:56.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.805+0000 7f1f90124740 1 -- 172.21.15.120:0/3792184383 >> 172.21.15.120:0/3792184383 conn(0x55f21a771310 msgr2=0x55f21a7aa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:56.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.805+0000 7f1f90124740 1 -- 172.21.15.120:0/3792184383 shutdown_connections 2024-09-19T09:23:56.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.805+0000 7f1f90124740 1 -- 172.21.15.120:0/3792184383 wait complete. 2024-09-19T09:23:56.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.805+0000 7f1f90124740 20 client.22299 trim_cache size 0 max 16384 2024-09-19T09:23:56.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.821+0000 7f1f90124740 1 Processor -- start 2024-09-19T09:23:56.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.821+0000 7f1f90124740 1 -- start start 2024-09-19T09:23:56.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.821+0000 7f1f90124740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f21a7a85d0 0x55f21a7b0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:56.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.821+0000 7f1f90124740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f21a7a9b40 0x55f21a7bf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:56.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.821+0000 7f1f90124740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f21a7b3370 0x55f21a7ad320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:56.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.821+0000 7f1f90124740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1f881af7d0 con 0x55f21a7a9b40 2024-09-19T09:23:56.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.821+0000 7f1f90124740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1f881b1310 con 0x55f21a7b3370 2024-09-19T09:23:56.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.821+0000 7f1f90124740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1f881afea0 con 0x55f21a7a85d0 2024-09-19T09:23:56.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.822+0000 7f1f877fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f21a7a85d0 0x55f21a7b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:56.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.822+0000 7f1f86ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f21a7a9b40 0x55f21a7bf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:56.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.822+0000 7f1f877fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f21a7a85d0 0x55f21a7b0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:40902/0 (socket says 172.21.15.120:40902) 2024-09-19T09:23:56.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.822+0000 7f1f877fe640 1 -- 172.21.15.120:0/1014523722 learned_addr learned my addr 172.21.15.120:0/1014523722 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:56.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.822+0000 7f1f87fff640 1 --2- 172.21.15.120:0/1014523722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f21a7b3370 0x55f21a7ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:56.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.822+0000 7f1f877fe640 1 -- 172.21.15.120:0/1014523722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f21a7b3370 msgr2=0x55f21a7ad320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:56.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.822+0000 7f1f877fe640 1 --2- 172.21.15.120:0/1014523722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f21a7b3370 0x55f21a7ad320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.822+0000 7f1f877fe640 1 -- 172.21.15.120:0/1014523722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f21a7a9b40 msgr2=0x55f21a7bf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:56.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.822+0000 7f1f877fe640 1 --2- 172.21.15.120:0/1014523722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f21a7a9b40 0x55f21a7bf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.822+0000 7f1f877fe640 1 -- 172.21.15.120:0/1014523722 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f21a7af600 con 0x55f21a7a85d0 2024-09-19T09:23:56.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.822+0000 7f1f87fff640 1 --2- 172.21.15.120:0/1014523722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f21a7b3370 0x55f21a7ad320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:23:56.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.822+0000 7f1f877fe640 1 --2- 172.21.15.120:0/1014523722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f21a7a85d0 0x55f21a7b0070 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55f21a70a3c0 tx=0x7f1f78005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=845becb9ccb7ad6e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:56.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.823+0000 7f1f6d7fa640 1 -- 172.21.15.120:0/1014523722 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78010c60 con 0x55f21a7a85d0 2024-09-19T09:23:56.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.823+0000 7f1f6d7fa640 1 -- 172.21.15.120:0/1014523722 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f78010e00 con 0x55f21a7a85d0 2024-09-19T09:23:56.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.823+0000 7f1f6d7fa640 1 -- 172.21.15.120:0/1014523722 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78004e50 con 0x55f21a7a85d0 2024-09-19T09:23:56.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.823+0000 7f1f90124740 1 -- 172.21.15.120:0/1014523722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f21a7a85d0 msgr2=0x55f21a7b0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:56.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.823+0000 7f1f90124740 1 --2- 172.21.15.120:0/1014523722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f21a7a85d0 0x55f21a7b0070 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55f21a70a3c0 tx=0x7f1f78005f90 comp rx=0 tx=0).stop 2024-09-19T09:23:56.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.824+0000 7f1f90124740 1 -- 172.21.15.120:0/1014523722 shutdown_connections 2024-09-19T09:23:56.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.824+0000 7f1f90124740 1 --2- 172.21.15.120:0/1014523722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f21a7b3370 0x55f21a7ad320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.824+0000 7f1f90124740 1 --2- 172.21.15.120:0/1014523722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f21a7a9b40 0x55f21a7bf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.824+0000 7f1f90124740 1 --2- 172.21.15.120:0/1014523722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f21a7a85d0 0x55f21a7b0070 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.824+0000 7f1f90124740 1 -- 172.21.15.120:0/1014523722 >> 172.21.15.120:0/1014523722 conn(0x55f21a771310 msgr2=0x55f21a7af960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:56.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.824+0000 7f1f90124740 1 -- 172.21.15.120:0/1014523722 shutdown_connections 2024-09-19T09:23:56.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.824+0000 7f1f90124740 1 -- 172.21.15.120:0/1014523722 wait complete. 2024-09-19T09:23:56.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.825+0000 7f1f90124740 1 -- start start 2024-09-19T09:23:56.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.825+0000 7f1f90124740 1 Processor -- start 2024-09-19T09:23:56.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.825+0000 7f1f90124740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f21a7a85d0 0x55f21a7c5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:56.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.826+0000 7f1f90124740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f21a7a9b40 0x55f21a7c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:56.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.826+0000 7f1f90124740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f21a7b3370 0x55f21a834030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:56.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.826+0000 7f1f86ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f21a7a9b40 0x55f21a7c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:56.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.826+0000 7f1f86ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f21a7a9b40 0x55f21a7c59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:40914/0 (socket says 172.21.15.120:40914) 2024-09-19T09:23:56.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.826+0000 7f1f86ffd640 1 -- 172.21.15.120:0/2749339781 learned_addr learned my addr 172.21.15.120:0/2749339781 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.826+0000 7f1f90124740 1 -- 172.21.15.120:0/2749339781 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1f881b0c40 con 0x55f21a7a85d0 2024-09-19T09:23:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.826+0000 7f1f87fff640 1 --2- 172.21.15.120:0/2749339781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f21a7b3370 0x55f21a834030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.826+0000 7f1f90124740 1 -- 172.21.15.120:0/2749339781 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1f881b0570 con 0x55f21a7b3370 2024-09-19T09:23:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.826+0000 7f1f90124740 1 -- 172.21.15.120:0/2749339781 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1f881ad790 con 0x55f21a7a9b40 2024-09-19T09:23:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.826+0000 7f1f877fe640 1 --2- 172.21.15.120:0/2749339781 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f21a7a85d0 0x55f21a7c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.826+0000 7f1f90124740 10 client.0 _ll_register_callbacks cb 0x7f1f78009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.827+0000 7f1f86ffd640 1 -- 172.21.15.120:0/2749339781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f21a7b3370 msgr2=0x55f21a834030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.827+0000 7f1f86ffd640 1 --2- 172.21.15.120:0/2749339781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f21a7b3370 0x55f21a834030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.827+0000 7f1f86ffd640 1 -- 172.21.15.120:0/2749339781 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f21a7a85d0 msgr2=0x55f21a7c5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.827+0000 7f1f86ffd640 1 --2- 172.21.15.120:0/2749339781 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f21a7a85d0 0x55f21a7c5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.827+0000 7f1f86ffd640 1 -- 172.21.15.120:0/2749339781 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f21a834570 con 0x55f21a7a9b40 2024-09-19T09:23:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.827+0000 7f1f86ffd640 1 --2- 172.21.15.120:0/2749339781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f21a7a9b40 0x55f21a7c59c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c098620 tx=0x7f1f7c09a6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86d975c542222fcf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.827+0000 7f1f66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:23:56.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.827+0000 7f1f66ffd640 1 -- 172.21.15.120:0/2749339781 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c0a04c0 con 0x55f21a7a9b40 2024-09-19T09:23:56.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.827+0000 7f1f66ffd640 1 -- 172.21.15.120:0/2749339781 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f1f7c0a00a0 con 0x55f21a7a9b40 2024-09-19T09:23:56.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.827+0000 7f1f90124740 1 -- 172.21.15.120:0/2749339781 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f78010e00 con 0x55f21a7a9b40 2024-09-19T09:23:56.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.828+0000 7f1f66ffd640 1 -- 172.21.15.120:0/2749339781 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1f7c0a02a0 con 0x55f21a7a9b40 2024-09-19T09:23:56.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.828+0000 7f1f877fe640 1 --2- 172.21.15.120:0/2749339781 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f21a7a85d0 0x55f21a7c5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:23:56.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.828+0000 7f1f90124740 10 client.30925 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:56.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.828+0000 7f1f90124740 1 -- 172.21.15.120:0/2749339781 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f21a8356c0 con 0x55f21a7a9b40 2024-09-19T09:23:56.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.829+0000 7f1f66ffd640 1 -- 172.21.15.120:0/2749339781 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f78010e00 con 0x55f21a7a9b40 2024-09-19T09:23:56.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.829+0000 7f1f66ffd640 1 -- 172.21.15.120:0/2749339781 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f1f7c0cdab0 con 0x55f21a7a9b40 2024-09-19T09:23:56.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.829+0000 7f1f90124740 10 client.30925 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:56.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.829+0000 7f1f90124740 10 client.30925 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:23:56.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.829+0000 7f1f90124740 20 client.30925 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:56.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.829+0000 7f1f90124740 1 --2- 172.21.15.120:0/2749339781 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55f21a835e40 0x55f21a838260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:56.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.829+0000 7f1f90124740 4 client.30925 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:23:56.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.829+0000 7f1f90124740 1 -- 172.21.15.120:0/2749339781 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f21a715d70 con 0x55f21a835e40 2024-09-19T09:23:56.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.830+0000 7f1f877fe640 1 --2- 172.21.15.120:0/2749339781 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55f21a835e40 0x55f21a838260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:56.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.830+0000 7f1f66ffd640 10 client.30925 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:23:56.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.830+0000 7f1f877fe640 1 --2- 172.21.15.120:0/2749339781 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55f21a835e40 0x55f21a838260 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3ecba3dbdbb7b54e server_cookie=c0129d140bc5f94a in_seq=0 out_seq=0 2024-09-19T09:23:56.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.831+0000 7f1f66ffd640 1 -- 172.21.15.120:0/2749339781 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f21a715d70 con 0x55f21a835e40 2024-09-19T09:23:56.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.831+0000 7f1f66ffd640 10 client.30925 handle_command_reply: tid=0 2024-09-19T09:23:56.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.831+0000 7f1f90124740 1 client.30925 shutdown 2024-09-19T09:23:56.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.831+0000 7f1f90124740 1 -- 172.21.15.120:0/2749339781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f21a7a9b40 msgr2=0x55f21a7c59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:56.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.831+0000 7f1f90124740 1 --2- 172.21.15.120:0/2749339781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f21a7a9b40 0x55f21a7c59c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c098620 tx=0x7f1f7c09a6d0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.831+0000 7f1f90124740 1 -- 172.21.15.120:0/2749339781 shutdown_connections 2024-09-19T09:23:56.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.831+0000 7f1f90124740 1 --2- 172.21.15.120:0/2749339781 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x55f21a835e40 0x55f21a838260 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.831+0000 7f1f90124740 1 --2- 172.21.15.120:0/2749339781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f21a7b3370 0x55f21a834030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.831+0000 7f1f90124740 1 --2- 172.21.15.120:0/2749339781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f21a7a9b40 0x55f21a7c59c0 secure :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c098620 tx=0x7f1f7c09a6d0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.831+0000 7f1f90124740 1 --2- 172.21.15.120:0/2749339781 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f21a7a85d0 0x55f21a7c5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.831+0000 7f1f90124740 1 -- 172.21.15.120:0/2749339781 >> 172.21.15.120:0/2749339781 conn(0x55f21a771310 msgr2=0x55f21a7b1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:56.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.831+0000 7f1f66ffd640 0 client.30925 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:23:56.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.831+0000 7f1f90124740 1 -- 172.21.15.120:0/2749339781 shutdown_connections 2024-09-19T09:23:56.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.831+0000 7f1f90124740 1 -- 172.21.15.120:0/2749339781 wait complete. 2024-09-19T09:23:56.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.831+0000 7f1f90124740 20 client.30925 trim_cache size 0 max 16384 2024-09-19T09:23:56.837 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:23:56.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.832+0000 7f1f8dd4b640 1 -- 172.21.15.120:0/2297944582 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1f700a3240 msgr2=0x7f1f700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:56.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.832+0000 7f1f8dd4b640 1 --2- 172.21.15.120:0/2297944582 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1f700a3240 0x7f1f700a5700 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1f8004a000 tx=0x7f1f8004b000 comp rx=0 tx=0).stop 2024-09-19T09:23:56.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.832+0000 7f1f8dd4b640 1 -- 172.21.15.120:0/2297944582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f881036e0 msgr2=0x7f1f88100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:56.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.832+0000 7f1f8dd4b640 1 --2- 172.21.15.120:0/2297944582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f881036e0 0x7f1f88100ed0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00e9f0 tx=0x7f1f7c00eeb0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.832+0000 7f1f8dd4b640 1 -- 172.21.15.120:0/2297944582 shutdown_connections 2024-09-19T09:23:56.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.832+0000 7f1f8dd4b640 1 --2- 172.21.15.120:0/2297944582 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1f700a3240 0x7f1f700a5700 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.832+0000 7f1f8dd4b640 1 --2- 172.21.15.120:0/2297944582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f881082a0 0x7f1f88101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.832+0000 7f1f8dd4b640 1 --2- 172.21.15.120:0/2297944582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f881036e0 0x7f1f88100ed0 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.832+0000 7f1f8dd4b640 1 --2- 172.21.15.120:0/2297944582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f88102d30 0x7f1f88100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:56.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.832+0000 7f1f8dd4b640 1 -- 172.21.15.120:0/2297944582 >> 172.21.15.120:0/2297944582 conn(0x7f1f880fc7d0 msgr2=0x7f1f88106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:56.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.832+0000 7f1f8dd4b640 1 -- 172.21.15.120:0/2297944582 shutdown_connections 2024-09-19T09:23:56.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:56.833+0000 7f1f8dd4b640 1 -- 172.21.15.120:0/2297944582 wait complete. 2024-09-19T09:23:57.378 INFO:tasks.workunit.client.0.smithi120.stdout: 10 195206 21.38 MB/sec execute 352 sec latency 69.335 ms 2024-09-19T09:23:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.601+0000 7f8a5b0d0640 1 -- 172.21.15.120:0/3003701672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a54100a50 msgr2=0x7f8a54100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.601+0000 7f8a5b0d0640 1 --2- 172.21.15.120:0/3003701672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a54100a50 0x7f8a54100e50 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009900 tx=0x7f8a4802a410 comp rx=0 tx=0).stop 2024-09-19T09:23:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.601+0000 7f8a5b0d0640 1 -- 172.21.15.120:0/3003701672 shutdown_connections 2024-09-19T09:23:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.601+0000 7f8a5b0d0640 1 --2- 172.21.15.120:0/3003701672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a54102600 0x7f8a54108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:57.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.601+0000 7f8a5b0d0640 1 --2- 172.21.15.120:0/3003701672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a54101c40 0x7f8a541020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:57.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.601+0000 7f8a5b0d0640 1 --2- 172.21.15.120:0/3003701672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a54100a50 0x7f8a54100e50 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:57.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.601+0000 7f8a5b0d0640 1 -- 172.21.15.120:0/3003701672 >> 172.21.15.120:0/3003701672 conn(0x7f8a540fc1b0 msgr2=0x7f8a540fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:57.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.601+0000 7f8a5b0d0640 1 -- 172.21.15.120:0/3003701672 shutdown_connections 2024-09-19T09:23:57.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.601+0000 7f8a5b0d0640 1 -- 172.21.15.120:0/3003701672 wait complete. 2024-09-19T09:23:57.604 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.602+0000 7f8a5b0d0640 1 Processor -- start 2024-09-19T09:23:57.604 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.602+0000 7f8a5b0d0640 1 -- start start 2024-09-19T09:23:57.604 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.602+0000 7f8a5b0d0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a54100a50 0x7f8a54193280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:57.604 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.602+0000 7f8a5b0d0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a54101c40 0x7f8a541937c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.602+0000 7f8a5b0d0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a54102600 0x7f8a5419a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.602+0000 7f8a5b0d0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5410d7e0 con 0x7f8a54101c40 2024-09-19T09:23:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.602+0000 7f8a5b0d0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5410d660 con 0x7f8a54102600 2024-09-19T09:23:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.602+0000 7f8a5b0d0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5410d960 con 0x7f8a54100a50 2024-09-19T09:23:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.602+0000 7f8a58e6c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a54100a50 0x7f8a54193280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.602+0000 7f8a5966d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a54102600 0x7f8a5419a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.602+0000 7f8a58e6c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a54100a50 0x7f8a54193280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:40918/0 (socket says 172.21.15.120:40918) 2024-09-19T09:23:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.602+0000 7f8a58e6c640 1 -- 172.21.15.120:0/3493738421 learned_addr learned my addr 172.21.15.120:0/3493738421 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.602+0000 7f8a53fff640 1 --2- 172.21.15.120:0/3493738421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a54101c40 0x7f8a541937c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.603+0000 7f8a5966d640 1 -- 172.21.15.120:0/3493738421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a54100a50 msgr2=0x7f8a54193280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.603+0000 7f8a5966d640 1 --2- 172.21.15.120:0/3493738421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a54100a50 0x7f8a54193280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.603+0000 7f8a5966d640 1 -- 172.21.15.120:0/3493738421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a54101c40 msgr2=0x7f8a541937c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.603+0000 7f8a5966d640 1 --2- 172.21.15.120:0/3493738421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a54101c40 0x7f8a541937c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.603+0000 7f8a5966d640 1 -- 172.21.15.120:0/3493738421 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5419aea0 con 0x7f8a54102600 2024-09-19T09:23:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.603+0000 7f8a5966d640 1 --2- 172.21.15.120:0/3493738421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a54102600 0x7f8a5419a830 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009fd0 tx=0x7f8a4c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.603+0000 7f8a51ffb640 1 -- 172.21.15.120:0/3493738421 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c00ce20 con 0x7f8a54102600 2024-09-19T09:23:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.603+0000 7f8a5b0d0640 1 -- 172.21.15.120:0/3493738421 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a5419b190 con 0x7f8a54102600 2024-09-19T09:23:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.603+0000 7f8a51ffb640 1 -- 172.21.15.120:0/3493738421 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a4c005020 con 0x7f8a54102600 2024-09-19T09:23:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.603+0000 7f8a51ffb640 1 -- 172.21.15.120:0/3493738421 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c010720 con 0x7f8a54102600 2024-09-19T09:23:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.603+0000 7f8a5b0d0640 1 -- 172.21.15.120:0/3493738421 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a541af1d0 con 0x7f8a54102600 2024-09-19T09:23:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.606+0000 7f8a51ffb640 1 -- 172.21.15.120:0/3493738421 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f8a4c005540 con 0x7f8a54102600 2024-09-19T09:23:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.607+0000 7f8a5de6d740 1 Processor -- start 2024-09-19T09:23:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.607+0000 7f8a5de6d740 1 -- start start 2024-09-19T09:23:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.607+0000 7f8a5de6d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae16e549a0 0x55ae16e54da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.607+0000 7f8a5de6d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae16e552e0 0x55ae16e59da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.607+0000 7f8a51ffb640 1 --2- 172.21.15.120:0/3493738421 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a440a3370 0x7f8a440a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.607+0000 7f8a5de6d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae16e5a370 0x55ae16e5c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.607+0000 7f8a5de6d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ae16db82f0 con 0x55ae16e5a370 2024-09-19T09:23:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.607+0000 7f8a5de6d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ae16d60270 con 0x55ae16e552e0 2024-09-19T09:23:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.607+0000 7f8a5de6d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ae16db42b0 con 0x55ae16e549a0 2024-09-19T09:23:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.607+0000 7f8a53fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae16e549a0 0x55ae16e54da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.607+0000 7f8a58e6c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae16e552e0 0x55ae16e59da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.607+0000 7f8a53fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae16e549a0 0x55ae16e54da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:40920/0 (socket says 172.21.15.120:40920) 2024-09-19T09:23:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.607+0000 7f8a53fff640 1 -- 172.21.15.120:0/665242342 learned_addr learned my addr 172.21.15.120:0/665242342 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.607+0000 7f8a5966d640 1 --2- 172.21.15.120:0/665242342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae16e5a370 0x55ae16e5c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.607+0000 7f8a51ffb640 1 -- 172.21.15.120:0/3493738421 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f8a4c0d0020 con 0x7f8a54102600 2024-09-19T09:23:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.607+0000 7f8a53fff640 1 --2- 172.21.15.120:0/3493738421 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a440a3370 0x7f8a440a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.607+0000 7f8a58e6c640 1 -- 172.21.15.120:0/665242342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae16e549a0 msgr2=0x55ae16e54da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.607+0000 7f8a58e6c640 1 --2- 172.21.15.120:0/665242342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae16e549a0 0x55ae16e54da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.607+0000 7f8a58e6c640 1 -- 172.21.15.120:0/665242342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae16e5a370 msgr2=0x55ae16e5c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.607+0000 7f8a58e6c640 1 --2- 172.21.15.120:0/665242342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae16e5a370 0x55ae16e5c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.608+0000 7f8a58e6c640 1 -- 172.21.15.120:0/665242342 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae16d5f340 con 0x55ae16e552e0 2024-09-19T09:23:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.608+0000 7f8a53fff640 1 --2- 172.21.15.120:0/3493738421 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a440a3370 0x7f8a440a5830 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f8a400129d0 tx=0x7f8a40005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.608+0000 7f8a58e6c640 1 --2- 172.21.15.120:0/665242342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae16e552e0 0x55ae16e59da0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8a48005ab0 tx=0x7f8a48005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af987c4749af447a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.608+0000 7f8a3effd640 1 -- 172.21.15.120:0/665242342 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a48040070 con 0x55ae16e552e0 2024-09-19T09:23:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.608+0000 7f8a3effd640 1 -- 172.21.15.120:0/665242342 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a480319b0 con 0x55ae16e552e0 2024-09-19T09:23:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.608+0000 7f8a3effd640 1 -- 172.21.15.120:0/665242342 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a48035040 con 0x55ae16e552e0 2024-09-19T09:23:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.609+0000 7f8a5de6d740 1 -- 172.21.15.120:0/665242342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae16e552e0 msgr2=0x55ae16e59da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.609+0000 7f8a5de6d740 1 --2- 172.21.15.120:0/665242342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae16e552e0 0x55ae16e59da0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8a48005ab0 tx=0x7f8a48005de0 comp rx=0 tx=0).stop 2024-09-19T09:23:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.609+0000 7f8a5de6d740 1 -- 172.21.15.120:0/665242342 shutdown_connections 2024-09-19T09:23:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.609+0000 7f8a5de6d740 1 --2- 172.21.15.120:0/665242342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae16e5a370 0x55ae16e5c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.609+0000 7f8a5de6d740 1 --2- 172.21.15.120:0/665242342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae16e552e0 0x55ae16e59da0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.609+0000 7f8a5de6d740 1 --2- 172.21.15.120:0/665242342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae16e549a0 0x55ae16e54da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.609+0000 7f8a5de6d740 1 -- 172.21.15.120:0/665242342 >> 172.21.15.120:0/665242342 conn(0x55ae16e50010 msgr2=0x55ae16e50410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.609+0000 7f8a5de6d740 1 -- 172.21.15.120:0/665242342 shutdown_connections 2024-09-19T09:23:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.609+0000 7f8a5de6d740 1 -- 172.21.15.120:0/665242342 wait complete. 2024-09-19T09:23:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.609+0000 7f8a5de6d740 1 -- start start 2024-09-19T09:23:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.609+0000 7f8a5de6d740 1 Processor -- start 2024-09-19T09:23:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.610+0000 7f8a5de6d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae16e549a0 0x55ae16e5eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.610+0000 7f8a5de6d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae16e552e0 0x55ae16e5f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.610+0000 7f8a5de6d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae16e5a370 0x55ae16e66080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.610+0000 7f8a5de6d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ae16e5d2e0 con 0x55ae16e552e0 2024-09-19T09:23:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.610+0000 7f8a5de6d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ae16d41290 con 0x55ae16e5a370 2024-09-19T09:23:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.610+0000 7f8a5de6d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ae16e5d460 con 0x55ae16e549a0 2024-09-19T09:23:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.610+0000 7f8a5de6d740 10 client.0 _ll_register_callbacks cb 0x55ae16d72140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.610+0000 7f8a53fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae16e552e0 0x55ae16e5f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.610+0000 7f8a53fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae16e552e0 0x55ae16e5f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52152/0 (socket says 172.21.15.120:52152) 2024-09-19T09:23:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.610+0000 7f8a53fff640 1 -- 172.21.15.120:0/2881535002 learned_addr learned my addr 172.21.15.120:0/2881535002 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.610+0000 7f8a58e6c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae16e549a0 0x55ae16e5eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.610+0000 7f8a5966d640 1 --2- 172.21.15.120:0/2881535002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae16e5a370 0x55ae16e66080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.610+0000 7f8a53fff640 1 -- 172.21.15.120:0/2881535002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae16e549a0 msgr2=0x55ae16e5eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.610+0000 7f8a53fff640 1 --2- 172.21.15.120:0/2881535002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae16e549a0 0x55ae16e5eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.610+0000 7f8a53fff640 1 -- 172.21.15.120:0/2881535002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae16e5a370 msgr2=0x55ae16e66080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.610+0000 7f8a53fff640 1 --2- 172.21.15.120:0/2881535002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae16e5a370 0x55ae16e66080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.610+0000 7f8a53fff640 1 -- 172.21.15.120:0/2881535002 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae16acb210 con 0x55ae16e552e0 2024-09-19T09:23:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.610+0000 7f8a53fff640 1 --2- 172.21.15.120:0/2881535002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae16e552e0 0x55ae16e5f0b0 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f8a4000da90 tx=0x7f8a40004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1048f3048971df28 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.610+0000 7f8a33fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:23:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.611+0000 7f8a33fff640 1 -- 172.21.15.120:0/2881535002 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a400044e0 con 0x55ae16e552e0 2024-09-19T09:23:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.611+0000 7f8a33fff640 1 -- 172.21.15.120:0/2881535002 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f8a40003d80 con 0x55ae16e552e0 2024-09-19T09:23:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.611+0000 7f8a5de6d740 1 -- 172.21.15.120:0/2881535002 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a480319b0 con 0x55ae16e552e0 2024-09-19T09:23:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.611+0000 7f8a33fff640 1 -- 172.21.15.120:0/2881535002 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8a400099a0 con 0x55ae16e552e0 2024-09-19T09:23:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.611+0000 7f8a5de6d740 10 client.22317 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.611+0000 7f8a5de6d740 1 -- 172.21.15.120:0/2881535002 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ae16b97620 con 0x55ae16e552e0 2024-09-19T09:23:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.611+0000 7f8a33fff640 1 -- 172.21.15.120:0/2881535002 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a4000f930 con 0x55ae16e552e0 2024-09-19T09:23:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.611+0000 7f8a33fff640 1 -- 172.21.15.120:0/2881535002 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f8a400345a0 con 0x55ae16e552e0 2024-09-19T09:23:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.611+0000 7f8a5de6d740 10 client.22317 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.611+0000 7f8a5de6d740 10 client.22317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:23:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.611+0000 7f8a5de6d740 20 client.22317 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.611+0000 7f8a5de6d740 1 --2- 172.21.15.120:0/2881535002 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ae16e6a840 0x55ae16e6cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.611+0000 7f8a5de6d740 4 client.22317 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:23:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.611+0000 7f8a5de6d740 1 -- 172.21.15.120:0/2881535002 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8a480319b0 con 0x55ae16e6a840 2024-09-19T09:23:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.612+0000 7f8a58e6c640 1 --2- 172.21.15.120:0/2881535002 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ae16e6a840 0x55ae16e6cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.612+0000 7f8a33fff640 10 client.22317 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.612+0000 7f8a58e6c640 1 --2- 172.21.15.120:0/2881535002 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ae16e6a840 0x55ae16e6cc50 crc :-1 s=READY pgs=519 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b31287003e96b1b server_cookie=e2a7612eab209d91 in_seq=0 out_seq=0 2024-09-19T09:23:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.617+0000 7f8a33fff640 1 -- 172.21.15.120:0/2881535002 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f8a480319b0 con 0x55ae16e6a840 2024-09-19T09:23:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.617+0000 7f8a33fff640 10 client.22317 handle_command_reply: tid=0 2024-09-19T09:23:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.617+0000 7f8a5de6d740 1 client.22317 shutdown 2024-09-19T09:23:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.618+0000 7f8a5de6d740 1 -- 172.21.15.120:0/2881535002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae16e552e0 msgr2=0x55ae16e5f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.618+0000 7f8a5de6d740 1 --2- 172.21.15.120:0/2881535002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae16e552e0 0x55ae16e5f0b0 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f8a4000da90 tx=0x7f8a40004730 comp rx=0 tx=0).stop 2024-09-19T09:23:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.618+0000 7f8a5de6d740 1 -- 172.21.15.120:0/2881535002 shutdown_connections 2024-09-19T09:23:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.618+0000 7f8a5de6d740 1 --2- 172.21.15.120:0/2881535002 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ae16e6a840 0x55ae16e6cc50 crc :-1 s=READY pgs=519 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.618+0000 7f8a33fff640 0 client.22317 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.618+0000 7f8a5de6d740 1 --2- 172.21.15.120:0/2881535002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae16e5a370 0x55ae16e66080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.618+0000 7f8a5de6d740 1 --2- 172.21.15.120:0/2881535002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae16e552e0 0x55ae16e5f0b0 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.618+0000 7f8a5de6d740 1 --2- 172.21.15.120:0/2881535002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae16e549a0 0x55ae16e5eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.618+0000 7f8a5de6d740 1 -- 172.21.15.120:0/2881535002 >> 172.21.15.120:0/2881535002 conn(0x55ae16e50010 msgr2=0x55ae16e51820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.618+0000 7f8a5de6d740 1 -- 172.21.15.120:0/2881535002 shutdown_connections 2024-09-19T09:23:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.618+0000 7f8a5de6d740 1 -- 172.21.15.120:0/2881535002 wait complete. 2024-09-19T09:23:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.618+0000 7f8a5de6d740 20 client.22317 trim_cache size 0 max 16384 2024-09-19T09:23:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.636+0000 7f8a5de6d740 1 Processor -- start 2024-09-19T09:23:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.636+0000 7f8a5de6d740 1 -- start start 2024-09-19T09:23:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.636+0000 7f8a5de6d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae16e549a0 0x55ae16e50ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.636+0000 7f8a5de6d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae16e552e0 0x55ae16e66040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.637+0000 7f8a5de6d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae16e5a370 0x55ae16e51400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.637+0000 7f8a5de6d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541b7d40 con 0x55ae16e549a0 2024-09-19T09:23:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.637+0000 7f8a5de6d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541b9880 con 0x55ae16e5a370 2024-09-19T09:23:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.637+0000 7f8a5de6d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541b8410 con 0x55ae16e552e0 2024-09-19T09:23:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.637+0000 7f8a58e6c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae16e549a0 0x55ae16e50ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.637+0000 7f8a5966d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae16e5a370 0x55ae16e51400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.637+0000 7f8a53fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae16e552e0 0x55ae16e66040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.637+0000 7f8a5966d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae16e5a370 0x55ae16e51400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:54332/0 (socket says 172.21.15.120:54332) 2024-09-19T09:23:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.637+0000 7f8a5966d640 1 -- 172.21.15.120:0/698804253 learned_addr learned my addr 172.21.15.120:0/698804253 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.637+0000 7f8a53fff640 1 -- 172.21.15.120:0/698804253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae16e5a370 msgr2=0x55ae16e51400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.637+0000 7f8a53fff640 1 --2- 172.21.15.120:0/698804253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae16e5a370 0x55ae16e51400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.637+0000 7f8a53fff640 1 -- 172.21.15.120:0/698804253 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae16e549a0 msgr2=0x55ae16e50ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.637+0000 7f8a53fff640 1 --2- 172.21.15.120:0/698804253 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae16e549a0 0x55ae16e50ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.637+0000 7f8a53fff640 1 -- 172.21.15.120:0/698804253 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae16e55f70 con 0x55ae16e552e0 2024-09-19T09:23:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.637+0000 7f8a5966d640 1 --2- 172.21.15.120:0/698804253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae16e5a370 0x55ae16e51400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.637+0000 7f8a53fff640 1 --2- 172.21.15.120:0/698804253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae16e552e0 0x55ae16e66040 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8a4000da60 tx=0x7f8a40009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=199ef2513df21bfe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:57.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.638+0000 7f8a3e7fc640 1 -- 172.21.15.120:0/698804253 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a40012750 con 0x55ae16e552e0 2024-09-19T09:23:57.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.638+0000 7f8a3e7fc640 1 -- 172.21.15.120:0/698804253 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a400044c0 con 0x55ae16e552e0 2024-09-19T09:23:57.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.638+0000 7f8a3e7fc640 1 -- 172.21.15.120:0/698804253 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4000f6b0 con 0x55ae16e552e0 2024-09-19T09:23:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.639+0000 7f8a5de6d740 1 -- 172.21.15.120:0/698804253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae16e552e0 msgr2=0x55ae16e66040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.639+0000 7f8a5de6d740 1 --2- 172.21.15.120:0/698804253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae16e552e0 0x55ae16e66040 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8a4000da60 tx=0x7f8a40009e30 comp rx=0 tx=0).stop 2024-09-19T09:23:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.639+0000 7f8a5de6d740 1 -- 172.21.15.120:0/698804253 shutdown_connections 2024-09-19T09:23:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.639+0000 7f8a5de6d740 1 --2- 172.21.15.120:0/698804253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae16e5a370 0x55ae16e51400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.639+0000 7f8a5de6d740 1 --2- 172.21.15.120:0/698804253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae16e552e0 0x55ae16e66040 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.639+0000 7f8a5de6d740 1 --2- 172.21.15.120:0/698804253 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae16e549a0 0x55ae16e50ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.639+0000 7f8a5de6d740 1 -- 172.21.15.120:0/698804253 >> 172.21.15.120:0/698804253 conn(0x55ae16e50010 msgr2=0x55ae16e56480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.639+0000 7f8a5de6d740 1 -- 172.21.15.120:0/698804253 shutdown_connections 2024-09-19T09:23:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.639+0000 7f8a5de6d740 1 -- 172.21.15.120:0/698804253 wait complete. 2024-09-19T09:23:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.639+0000 7f8a5de6d740 1 -- start start 2024-09-19T09:23:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.640+0000 7f8a5de6d740 1 Processor -- start 2024-09-19T09:23:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.640+0000 7f8a5de6d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae16e549a0 0x55ae16e638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.640+0000 7f8a5de6d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae16e552e0 0x55ae16e63df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.640+0000 7f8a5de6d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae16e5a370 0x55ae16e64e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.640+0000 7f8a5de6d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541b91b0 con 0x55ae16e5a370 2024-09-19T09:23:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.640+0000 7f8a5de6d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541b8ae0 con 0x55ae16e552e0 2024-09-19T09:23:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.640+0000 7f8a5de6d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541b5d00 con 0x55ae16e549a0 2024-09-19T09:23:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.640+0000 7f8a5de6d740 10 client.0 _ll_register_callbacks cb 0x7f8a40004410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:23:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.640+0000 7f8a58e6c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae16e549a0 0x55ae16e638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.640+0000 7f8a58e6c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae16e549a0 0x55ae16e638b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:40942/0 (socket says 172.21.15.120:40942) 2024-09-19T09:23:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.640+0000 7f8a53fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae16e552e0 0x55ae16e63df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.640+0000 7f8a58e6c640 1 -- 172.21.15.120:0/2504140196 learned_addr learned my addr 172.21.15.120:0/2504140196 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.640+0000 7f8a5966d640 1 --2- 172.21.15.120:0/2504140196 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae16e5a370 0x55ae16e64e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.641+0000 7f8a53fff640 1 -- 172.21.15.120:0/2504140196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae16e549a0 msgr2=0x55ae16e638b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.641+0000 7f8a53fff640 1 --2- 172.21.15.120:0/2504140196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae16e549a0 0x55ae16e638b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.641+0000 7f8a53fff640 1 -- 172.21.15.120:0/2504140196 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae16e5a370 msgr2=0x55ae16e64e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:23:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.641+0000 7f8a53fff640 1 --2- 172.21.15.120:0/2504140196 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae16e5a370 0x55ae16e64e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.641+0000 7f8a53fff640 1 -- 172.21.15.120:0/2504140196 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae16e5db60 con 0x55ae16e552e0 2024-09-19T09:23:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.641+0000 7f8a53fff640 1 --2- 172.21.15.120:0/2504140196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae16e552e0 0x55ae16e63df0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8a4000d7b0 tx=0x7f8a400096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a0f484dba6be177 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.641+0000 7f8a33fff640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:23:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.641+0000 7f8a33fff640 1 -- 172.21.15.120:0/2504140196 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4000f650 con 0x55ae16e552e0 2024-09-19T09:23:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.641+0000 7f8a33fff640 1 -- 172.21.15.120:0/2504140196 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f8a4000f7f0 con 0x55ae16e552e0 2024-09-19T09:23:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.641+0000 7f8a5de6d740 1 -- 172.21.15.120:0/2504140196 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a400044c0 con 0x55ae16e552e0 2024-09-19T09:23:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.642+0000 7f8a33fff640 1 -- 172.21.15.120:0/2504140196 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8a4004b050 con 0x55ae16e552e0 2024-09-19T09:23:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.642+0000 7f8a5de6d740 10 client.31070 fetch_fsmap learned FSMap version 39 2024-09-19T09:23:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.642+0000 7f8a5de6d740 1 -- 172.21.15.120:0/2504140196 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ae16ed5530 con 0x55ae16e552e0 2024-09-19T09:23:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.642+0000 7f8a33fff640 1 -- 172.21.15.120:0/2504140196 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a40029400 con 0x55ae16e552e0 2024-09-19T09:23:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.642+0000 7f8a33fff640 1 -- 172.21.15.120:0/2504140196 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f8a400474f0 con 0x55ae16e552e0 2024-09-19T09:23:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.642+0000 7f8a5de6d740 10 client.31070 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:23:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.642+0000 7f8a5de6d740 10 client.31070 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:23:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.642+0000 7f8a5de6d740 20 client.31070 populate_metadata read hostname 'smithi120' 2024-09-19T09:23:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.642+0000 7f8a5de6d740 1 --2- 172.21.15.120:0/2504140196 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ae16e6a840 0x55ae16ed7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.642+0000 7f8a5de6d740 4 client.31070 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:23:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.642+0000 7f8a5de6d740 1 -- 172.21.15.120:0/2504140196 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f8a400044c0 con 0x55ae16e6a840 2024-09-19T09:23:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.642+0000 7f8a58e6c640 1 --2- 172.21.15.120:0/2504140196 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ae16e6a840 0x55ae16ed7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.643+0000 7f8a33fff640 10 client.31070 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:57.643+0000 7f8a58e6c640 1 --2- 172.21.15.120:0/2504140196 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ae16e6a840 0x55ae16ed7ee0 crc :-1 s=READY pgs=520 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80592a84ba8a66ff server_cookie=fc9f1b52f0fd3441 in_seq=0 out_seq=0 2024-09-19T09:23:57.727 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:23:57.932 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:23:58.064 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:58 smithi120 ceph-mon[31150]: pgmap v978: 97 pgs: 97 active+clean; 461 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.7 MiB/s wr, 655 op/s 2024-09-19T09:23:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:58.290+0000 7f8a33fff640 1 -- 172.21.15.120:0/2504140196 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f8a400044c0 con 0x55ae16e6a840 2024-09-19T09:23:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:58.290+0000 7f8a33fff640 10 client.31070 handle_command_reply: tid=0 2024-09-19T09:23:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:58.290+0000 7f8a5de6d740 1 client.31070 shutdown 2024-09-19T09:23:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:58.290+0000 7f8a5de6d740 1 -- 172.21.15.120:0/2504140196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae16e552e0 msgr2=0x55ae16e63df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:58.290+0000 7f8a5de6d740 1 --2- 172.21.15.120:0/2504140196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae16e552e0 0x55ae16e63df0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8a4000d7b0 tx=0x7f8a400096e0 comp rx=0 tx=0).stop 2024-09-19T09:23:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:58.290+0000 7f8a5de6d740 1 -- 172.21.15.120:0/2504140196 shutdown_connections 2024-09-19T09:23:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:58.290+0000 7f8a5de6d740 1 --2- 172.21.15.120:0/2504140196 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ae16e6a840 0x55ae16ed7ee0 crc :-1 s=READY pgs=520 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:58.290+0000 7f8a5de6d740 1 --2- 172.21.15.120:0/2504140196 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ae16e5a370 0x55ae16e64e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:58.290+0000 7f8a33fff640 0 client.31070 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:23:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:58.290+0000 7f8a5de6d740 1 --2- 172.21.15.120:0/2504140196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ae16e552e0 0x55ae16e63df0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:58.290+0000 7f8a5de6d740 1 --2- 172.21.15.120:0/2504140196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae16e549a0 0x55ae16e638b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:58.290+0000 7f8a5de6d740 1 -- 172.21.15.120:0/2504140196 >> 172.21.15.120:0/2504140196 conn(0x55ae16e50010 msgr2=0x55ae16e4e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:58.290+0000 7f8a5de6d740 1 -- 172.21.15.120:0/2504140196 shutdown_connections 2024-09-19T09:23:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:58.290+0000 7f8a5de6d740 1 -- 172.21.15.120:0/2504140196 wait complete. 2024-09-19T09:23:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:58.290+0000 7f8a5de6d740 20 client.31070 trim_cache size 0 max 16384 2024-09-19T09:23:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:23:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:23:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:23:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:23:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:58.291+0000 7f8a5b0d0640 1 -- 172.21.15.120:0/3493738421 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a440a3370 msgr2=0x7f8a440a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:58.291+0000 7f8a5b0d0640 1 --2- 172.21.15.120:0/3493738421 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a440a3370 0x7f8a440a5830 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f8a400129d0 tx=0x7f8a40005f90 comp rx=0 tx=0).stop 2024-09-19T09:23:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:58.291+0000 7f8a5b0d0640 1 -- 172.21.15.120:0/3493738421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a54102600 msgr2=0x7f8a5419a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:58.291+0000 7f8a5b0d0640 1 --2- 172.21.15.120:0/3493738421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a54102600 0x7f8a5419a830 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009fd0 tx=0x7f8a4c00ef10 comp rx=0 tx=0).stop 2024-09-19T09:23:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:58.291+0000 7f8a5b0d0640 1 -- 172.21.15.120:0/3493738421 shutdown_connections 2024-09-19T09:23:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:58.291+0000 7f8a5b0d0640 1 --2- 172.21.15.120:0/3493738421 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f8a440a3370 0x7f8a440a5830 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:58.291+0000 7f8a5b0d0640 1 --2- 172.21.15.120:0/3493738421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a54102600 0x7f8a5419a830 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:58.291+0000 7f8a5b0d0640 1 --2- 172.21.15.120:0/3493738421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a54101c40 0x7f8a541937c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:58.291+0000 7f8a5b0d0640 1 --2- 172.21.15.120:0/3493738421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a54100a50 0x7f8a54193280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:58.291+0000 7f8a5b0d0640 1 -- 172.21.15.120:0/3493738421 >> 172.21.15.120:0/3493738421 conn(0x7f8a540fc1b0 msgr2=0x7f8a54102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:58.291+0000 7f8a5b0d0640 1 -- 172.21.15.120:0/3493738421 shutdown_connections 2024-09-19T09:23:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:23:58.291+0000 7f8a5b0d0640 1 -- 172.21.15.120:0/3493738421 wait complete. 2024-09-19T09:23:58.378 INFO:tasks.workunit.client.0.smithi120.stdout: 10 195653 21.40 MB/sec execute 353 sec latency 381.934 ms 2024-09-19T09:23:58.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:58 smithi196 ceph-mon[35150]: pgmap v978: 97 pgs: 97 active+clean; 461 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.7 MiB/s wr, 655 op/s 2024-09-19T09:23:58.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:58 smithi153 ceph-mon[35071]: pgmap v978: 97 pgs: 97 active+clean; 461 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.7 MiB/s wr, 655 op/s 2024-09-19T09:23:58.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.593+0000 7f8a0bcbe640 1 -- 172.21.15.120:0/3415384202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a04101420 msgr2=0x7f8a0410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:58.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.593+0000 7f8a0bcbe640 1 --2- 172.21.15.120:0/3415384202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a04101420 0x7f8a0410d950 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f89fc009930 tx=0x7f89fc02a420 comp rx=0 tx=0).stop 2024-09-19T09:23:58.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.593+0000 7f8a0bcbe640 1 -- 172.21.15.120:0/3415384202 shutdown_connections 2024-09-19T09:23:58.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.593+0000 7f8a0bcbe640 1 --2- 172.21.15.120:0/3415384202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a04101420 0x7f8a0410d950 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:58.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.593+0000 7f8a0bcbe640 1 --2- 172.21.15.120:0/3415384202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a04100a80 0x7f8a04100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:58.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.593+0000 7f8a0bcbe640 1 --2- 172.21.15.120:0/3415384202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a04106a80 0x7f8a04106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:58.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.593+0000 7f8a0bcbe640 1 -- 172.21.15.120:0/3415384202 >> 172.21.15.120:0/3415384202 conn(0x7f8a040fc7a0 msgr2=0x7f8a040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:58.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.594+0000 7f8a0bcbe640 1 -- 172.21.15.120:0/3415384202 shutdown_connections 2024-09-19T09:23:58.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.594+0000 7f8a0bcbe640 1 -- 172.21.15.120:0/3415384202 wait complete. 2024-09-19T09:23:58.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.594+0000 7f8a0bcbe640 1 Processor -- start 2024-09-19T09:23:58.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.595+0000 7f8a0bcbe640 1 -- start start 2024-09-19T09:23:58.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.595+0000 7f8a0bcbe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a04100a80 0x7f8a0419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:58.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.595+0000 7f8a0bcbe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a04101420 0x7f8a0419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:58.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.595+0000 7f8a0bcbe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a04106a80 0x7f8a0419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:58.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.595+0000 7f8a0bcbe640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8a04110e80 con 0x7f8a04101420 2024-09-19T09:23:58.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.595+0000 7f8a0bcbe640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8a04110d00 con 0x7f8a04100a80 2024-09-19T09:23:58.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.595+0000 7f8a0bcbe640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8a04111000 con 0x7f8a04106a80 2024-09-19T09:23:58.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.595+0000 7f8a09a5a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a04100a80 0x7f8a0419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:58.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.595+0000 7f8a0a25b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a04106a80 0x7f8a0419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:58.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.595+0000 7f8a09a5a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a04100a80 0x7f8a0419a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:54358/0 (socket says 172.21.15.120:54358) 2024-09-19T09:23:58.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.595+0000 7f8a09259640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a04101420 0x7f8a0419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:58.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.595+0000 7f8a09a5a640 1 -- 172.21.15.120:0/2277144213 learned_addr learned my addr 172.21.15.120:0/2277144213 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:23:58.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.595+0000 7f8a09a5a640 1 -- 172.21.15.120:0/2277144213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a04106a80 msgr2=0x7f8a0419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:58.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.595+0000 7f8a09a5a640 1 --2- 172.21.15.120:0/2277144213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a04106a80 0x7f8a0419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:58.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.595+0000 7f8a09a5a640 1 -- 172.21.15.120:0/2277144213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a04101420 msgr2=0x7f8a0419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:58.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.595+0000 7f8a09a5a640 1 --2- 172.21.15.120:0/2277144213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a04101420 0x7f8a0419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:58.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.595+0000 7f8a09a5a640 1 -- 172.21.15.120:0/2277144213 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0419f5a0 con 0x7f8a04100a80 2024-09-19T09:23:58.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.596+0000 7f8a09259640 1 --2- 172.21.15.120:0/2277144213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a04101420 0x7f8a0419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:23:58.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.596+0000 7f8a0a25b640 1 --2- 172.21.15.120:0/2277144213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a04106a80 0x7f8a0419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:58.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.596+0000 7f8a09a5a640 1 --2- 172.21.15.120:0/2277144213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a04100a80 0x7f8a0419a5f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000ebc0 tx=0x7f8a0000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:58.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.596+0000 7f89faffd640 1 -- 172.21.15.120:0/2277144213 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00010630 con 0x7f8a04100a80 2024-09-19T09:23:58.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.596+0000 7f89faffd640 1 -- 172.21.15.120:0/2277144213 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a00010cb0 con 0x7f8a04100a80 2024-09-19T09:23:58.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.596+0000 7f8a0bcbe640 1 -- 172.21.15.120:0/2277144213 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a0419f890 con 0x7f8a04100a80 2024-09-19T09:23:58.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.596+0000 7f89faffd640 1 -- 172.21.15.120:0/2277144213 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00018bd0 con 0x7f8a04100a80 2024-09-19T09:23:58.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.596+0000 7f8a0bcbe640 1 -- 172.21.15.120:0/2277144213 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a041a7170 con 0x7f8a04100a80 2024-09-19T09:23:58.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.598+0000 7f8a0bcbe640 1 -- 172.21.15.120:0/2277144213 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a04100ee0 con 0x7f8a04100a80 2024-09-19T09:23:58.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.599+0000 7f89faffd640 1 -- 172.21.15.120:0/2277144213 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f8a000173d0 con 0x7f8a04100a80 2024-09-19T09:23:58.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.600+0000 7f89faffd640 1 --2- 172.21.15.120:0/2277144213 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f89f00a32c0 0x7f89f00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:58.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.600+0000 7f8a09259640 1 --2- 172.21.15.120:0/2277144213 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f89f00a32c0 0x7f89f00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:58.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.600+0000 7f89faffd640 1 -- 172.21.15.120:0/2277144213 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f8a000cfec0 con 0x7f8a04100a80 2024-09-19T09:23:58.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.600+0000 7f8a09259640 1 --2- 172.21.15.120:0/2277144213 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f89f00a32c0 0x7f89f00a5780 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8a040fdfa0 tx=0x7f89f4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:58.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.604+0000 7f89faffd640 1 -- 172.21.15.120:0/2277144213 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8a0009cba0 con 0x7f8a04100a80 2024-09-19T09:23:58.687 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:23:58.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.863+0000 7f8a0bcbe640 1 -- 172.21.15.120:0/2277144213 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a041a7370 con 0x7f8a04100a80 2024-09-19T09:23:58.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.864+0000 7f89faffd640 1 -- 172.21.15.120:0/2277144213 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11988 (secure 0 0 0) 0x7f8a0009c4a0 con 0x7f8a04100a80 2024-09-19T09:23:58.865 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:23:58.865 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":39,"btime":"2024-09-19T09:18:53:120858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:18:53.070639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:23:58.865 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 39 2024-09-19T09:23:58.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.866+0000 7f8a0bcbe640 1 -- 172.21.15.120:0/2277144213 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f89f00a32c0 msgr2=0x7f89f00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:58.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.866+0000 7f8a0bcbe640 1 --2- 172.21.15.120:0/2277144213 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f89f00a32c0 0x7f89f00a5780 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8a040fdfa0 tx=0x7f89f4005f90 comp rx=0 tx=0).stop 2024-09-19T09:23:58.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.866+0000 7f8a0bcbe640 1 -- 172.21.15.120:0/2277144213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a04100a80 msgr2=0x7f8a0419a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:58.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.866+0000 7f8a0bcbe640 1 --2- 172.21.15.120:0/2277144213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a04100a80 0x7f8a0419a5f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000ebc0 tx=0x7f8a0000c6a0 comp rx=0 tx=0).stop 2024-09-19T09:23:58.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.866+0000 7f8a0bcbe640 1 -- 172.21.15.120:0/2277144213 shutdown_connections 2024-09-19T09:23:58.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.866+0000 7f8a0bcbe640 1 --2- 172.21.15.120:0/2277144213 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f89f00a32c0 0x7f89f00a5780 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:58.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.866+0000 7f8a0bcbe640 1 --2- 172.21.15.120:0/2277144213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a04106a80 0x7f8a0419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:58.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.866+0000 7f8a0bcbe640 1 --2- 172.21.15.120:0/2277144213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a04101420 0x7f8a0419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:58.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.866+0000 7f8a0bcbe640 1 --2- 172.21.15.120:0/2277144213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a04100a80 0x7f8a0419a5f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:58.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.866+0000 7f8a0bcbe640 1 -- 172.21.15.120:0/2277144213 >> 172.21.15.120:0/2277144213 conn(0x7f8a040fc7a0 msgr2=0x7f8a0406a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:58.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.866+0000 7f8a0bcbe640 1 -- 172.21.15.120:0/2277144213 shutdown_connections 2024-09-19T09:23:58.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:23:58.866+0000 7f8a0bcbe640 1 -- 172.21.15.120:0/2277144213 wait complete. 2024-09-19T09:23:59.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:23:59 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2277144213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:23:59.378 INFO:tasks.workunit.client.0.smithi120.stdout: 10 196169 21.44 MB/sec execute 354 sec latency 90.135 ms 2024-09-19T09:23:59.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:23:59 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2277144213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:23:59.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:23:59 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2277144213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:23:59.791 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-09-19T09:23:59.791 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-09-19T09:24:00.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.039+0000 7fd0423a9640 1 -- 172.21.15.120:0/1761597471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd03c102ef0 msgr2=0x7fd03c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:00.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.039+0000 7fd0423a9640 1 --2- 172.21.15.120:0/1761597471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd03c102ef0 0x7fd03c10da40 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7fd034009ca0 tx=0x7fd03402a950 comp rx=0 tx=0).stop 2024-09-19T09:24:00.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.039+0000 7fd0423a9640 1 -- 172.21.15.120:0/1761597471 shutdown_connections 2024-09-19T09:24:00.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.039+0000 7fd0423a9640 1 --2- 172.21.15.120:0/1761597471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd03c102ef0 0x7fd03c10da40 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.039+0000 7fd0423a9640 1 --2- 172.21.15.120:0/1761597471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd03c102570 0x7fd03c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.039+0000 7fd0423a9640 1 --2- 172.21.15.120:0/1761597471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd03c100e20 0x7fd03c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.039+0000 7fd0423a9640 1 -- 172.21.15.120:0/1761597471 >> 172.21.15.120:0/1761597471 conn(0x7fd03c0fc7d0 msgr2=0x7fd03c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:00.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.040+0000 7fd0423a9640 1 -- 172.21.15.120:0/1761597471 shutdown_connections 2024-09-19T09:24:00.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.040+0000 7fd0423a9640 1 -- 172.21.15.120:0/1761597471 wait complete. 2024-09-19T09:24:00.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.040+0000 7fd0423a9640 1 Processor -- start 2024-09-19T09:24:00.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.040+0000 7fd0423a9640 1 -- start start 2024-09-19T09:24:00.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.041+0000 7fd0423a9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd03c100e20 0x7fd03c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:00.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.041+0000 7fd0423a9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd03c102570 0x7fd03c19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:00.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.041+0000 7fd0423a9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd03c102ef0 0x7fd03c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:00.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.041+0000 7fd0423a9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c110fd0 con 0x7fd03c102570 2024-09-19T09:24:00.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.041+0000 7fd0423a9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c110e50 con 0x7fd03c100e20 2024-09-19T09:24:00.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.041+0000 7fd0423a9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c111150 con 0x7fd03c102ef0 2024-09-19T09:24:00.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.041+0000 7fd033fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd03c102570 0x7fd03c19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:00.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.041+0000 7fd033fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd03c102570 0x7fd03c19ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52210/0 (socket says 172.21.15.120:52210) 2024-09-19T09:24:00.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.041+0000 7fd033fff640 1 -- 172.21.15.120:0/3273122963 learned_addr learned my addr 172.21.15.120:0/3273122963 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:00.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.041+0000 7fd03bfff640 1 --2- 172.21.15.120:0/3273122963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd03c100e20 0x7fd03c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:00.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.041+0000 7fd040946640 1 --2- 172.21.15.120:0/3273122963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd03c102ef0 0x7fd03c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:00.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.041+0000 7fd033fff640 1 -- 172.21.15.120:0/3273122963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd03c102ef0 msgr2=0x7fd03c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:00.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.041+0000 7fd033fff640 1 --2- 172.21.15.120:0/3273122963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd03c102ef0 0x7fd03c19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.041+0000 7fd033fff640 1 -- 172.21.15.120:0/3273122963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd03c100e20 msgr2=0x7fd03c19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:00.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.041+0000 7fd033fff640 1 --2- 172.21.15.120:0/3273122963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd03c100e20 0x7fd03c19a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.041+0000 7fd033fff640 1 -- 172.21.15.120:0/3273122963 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03c19f600 con 0x7fd03c102570 2024-09-19T09:24:00.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.042+0000 7fd033fff640 1 --2- 172.21.15.120:0/3273122963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd03c102570 0x7fd03c19ab90 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7fd02400e9f0 tx=0x7fd02400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:00.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.042+0000 7fd039ffb640 1 -- 172.21.15.120:0/3273122963 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02400cd80 con 0x7fd03c102570 2024-09-19T09:24:00.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.042+0000 7fd0423a9640 1 -- 172.21.15.120:0/3273122963 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd03c19f8f0 con 0x7fd03c102570 2024-09-19T09:24:00.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.042+0000 7fd039ffb640 1 -- 172.21.15.120:0/3273122963 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd024010910 con 0x7fd03c102570 2024-09-19T09:24:00.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.042+0000 7fd039ffb640 1 -- 172.21.15.120:0/3273122963 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd024018910 con 0x7fd03c102570 2024-09-19T09:24:00.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.042+0000 7fd0423a9640 1 -- 172.21.15.120:0/3273122963 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd03c1a71d0 con 0x7fd03c102570 2024-09-19T09:24:00.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.045+0000 7fd039ffb640 1 -- 172.21.15.120:0/3273122963 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd024010ac0 con 0x7fd03c102570 2024-09-19T09:24:00.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.046+0000 7fd044782740 1 Processor -- start 2024-09-19T09:24:00.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.046+0000 7fd044782740 1 -- start start 2024-09-19T09:24:00.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.046+0000 7fd044782740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0491cf350 0x55b0491cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:00.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.047+0000 7fd044782740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0491cdde0 0x55b0491ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:00.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.047+0000 7fd044782740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0491d8b80 0x55b0491daf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:00.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.047+0000 7fd044782740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b04912b2f0 con 0x55b0491cf350 2024-09-19T09:24:00.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.047+0000 7fd044782740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b0490d2cf0 con 0x55b0491cdde0 2024-09-19T09:24:00.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.047+0000 7fd044782740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b0491272b0 con 0x55b0491d8b80 2024-09-19T09:24:00.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.047+0000 7fd040946640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0491cf350 0x55b0491cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:00.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.047+0000 7fd03bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0491cdde0 0x55b0491ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:00.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.047+0000 7fd03bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0491cdde0 0x55b0491ce220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:54380/0 (socket says 172.21.15.120:54380) 2024-09-19T09:24:00.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.047+0000 7fd040946640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0491cf350 0x55b0491cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52212/0 (socket says 172.21.15.120:52212) 2024-09-19T09:24:00.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.047+0000 7fd03bfff640 1 -- 172.21.15.120:0/2900295653 learned_addr learned my addr 172.21.15.120:0/2900295653 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:00.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.047+0000 7fd039ffb640 1 --2- 172.21.15.120:0/3273122963 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd0280a32c0 0x7fd0280a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:00.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.047+0000 7fd033fff640 1 --2- 172.21.15.120:0/2900295653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0491d8b80 0x55b0491daf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:00.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.047+0000 7fd03bfff640 1 -- 172.21.15.120:0/2900295653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0491d8b80 msgr2=0x55b0491daf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:00.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.047+0000 7fd03bfff640 1 --2- 172.21.15.120:0/2900295653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0491d8b80 0x55b0491daf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.047+0000 7fd03bfff640 1 -- 172.21.15.120:0/2900295653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0491cf350 msgr2=0x55b0491cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:00.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.047+0000 7fd03bfff640 1 --2- 172.21.15.120:0/2900295653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0491cf350 0x55b0491cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.047+0000 7fd03bfff640 1 -- 172.21.15.120:0/2900295653 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0490b73a0 con 0x55b0491cdde0 2024-09-19T09:24:00.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.048+0000 7fd039ffb640 1 -- 172.21.15.120:0/3273122963 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd0240cfcd0 con 0x7fd03c102570 2024-09-19T09:24:00.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.048+0000 7fd040946640 1 --2- 172.21.15.120:0/3273122963 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd0280a32c0 0x7fd0280a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:00.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.048+0000 7fd033fff640 1 --2- 172.21.15.120:0/2900295653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0491d8b80 0x55b0491daf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:24:00.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.048+0000 7fd03bfff640 1 --2- 172.21.15.120:0/2900295653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0491cdde0 0x55b0491ce220 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd02c009cf0 tx=0x7fd02c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe0a350d281d151f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:00.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.048+0000 7fd0327fc640 1 -- 172.21.15.120:0/2900295653 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c004230 con 0x55b0491cdde0 2024-09-19T09:24:00.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.048+0000 7fd0327fc640 1 -- 172.21.15.120:0/2900295653 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd02c0043d0 con 0x55b0491cdde0 2024-09-19T09:24:00.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.048+0000 7fd0327fc640 1 -- 172.21.15.120:0/2900295653 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c0198c0 con 0x55b0491cdde0 2024-09-19T09:24:00.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.049+0000 7fd044782740 1 -- 172.21.15.120:0/2900295653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0491cdde0 msgr2=0x55b0491ce220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:00.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.049+0000 7fd040946640 1 --2- 172.21.15.120:0/3273122963 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd0280a32c0 0x7fd0280a5780 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd034006010 tx=0x7fd03403c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:00.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.049+0000 7fd044782740 1 --2- 172.21.15.120:0/2900295653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0491cdde0 0x55b0491ce220 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd02c009cf0 tx=0x7fd02c011810 comp rx=0 tx=0).stop 2024-09-19T09:24:00.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.049+0000 7fd044782740 1 -- 172.21.15.120:0/2900295653 shutdown_connections 2024-09-19T09:24:00.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.049+0000 7fd044782740 1 --2- 172.21.15.120:0/2900295653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0491d8b80 0x55b0491daf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.049+0000 7fd044782740 1 --2- 172.21.15.120:0/2900295653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0491cdde0 0x55b0491ce220 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.049+0000 7fd044782740 1 --2- 172.21.15.120:0/2900295653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0491cf350 0x55b0491cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.049+0000 7fd044782740 1 -- 172.21.15.120:0/2900295653 >> 172.21.15.120:0/2900295653 conn(0x55b04918e310 msgr2=0x55b0491ca470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:00.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.049+0000 7fd044782740 1 -- 172.21.15.120:0/2900295653 shutdown_connections 2024-09-19T09:24:00.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.049+0000 7fd044782740 1 -- 172.21.15.120:0/2900295653 wait complete. 2024-09-19T09:24:00.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.049+0000 7fd044782740 1 -- start start 2024-09-19T09:24:00.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.050+0000 7fd044782740 1 Processor -- start 2024-09-19T09:24:00.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.050+0000 7fd044782740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0491cdde0 0x55b0491d8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:00.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.050+0000 7fd044782740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0491cf350 0x55b0491d57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:00.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.050+0000 7fd044782740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0491d8b80 0x55b0491d5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:00.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.050+0000 7fd044782740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b0491db880 con 0x55b0491cdde0 2024-09-19T09:24:00.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.050+0000 7fd044782740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b0490e6060 con 0x55b0491cf350 2024-09-19T09:24:00.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.050+0000 7fd044782740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b0491dba00 con 0x55b0491d8b80 2024-09-19T09:24:00.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.050+0000 7fd044782740 10 client.0 _ll_register_callbacks cb 0x55b048e398c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:00.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.050+0000 7fd03bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0491cdde0 0x55b0491d8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:00.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.050+0000 7fd040946640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0491d8b80 0x55b0491d5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:00.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.050+0000 7fd03bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0491cdde0 0x55b0491d8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52222/0 (socket says 172.21.15.120:52222) 2024-09-19T09:24:00.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.050+0000 7fd03bfff640 1 -- 172.21.15.120:0/1421202294 learned_addr learned my addr 172.21.15.120:0/1421202294 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:00.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.050+0000 7fd033fff640 1 --2- 172.21.15.120:0/1421202294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0491cf350 0x55b0491d57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:00.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.050+0000 7fd040946640 1 -- 172.21.15.120:0/1421202294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0491cf350 msgr2=0x55b0491d57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:00.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.050+0000 7fd040946640 1 --2- 172.21.15.120:0/1421202294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0491cf350 0x55b0491d57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.050+0000 7fd040946640 1 -- 172.21.15.120:0/1421202294 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0491cdde0 msgr2=0x55b0491d8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:00.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.050+0000 7fd040946640 1 --2- 172.21.15.120:0/1421202294 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0491cdde0 0x55b0491d8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.051+0000 7fd040946640 1 -- 172.21.15.120:0/1421202294 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b048e9c3c0 con 0x55b0491d8b80 2024-09-19T09:24:00.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.051+0000 7fd040946640 1 --2- 172.21.15.120:0/1421202294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0491d8b80 0x55b0491d5d40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd034005e30 tx=0x7fd034005d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6beafd27717a1adc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:00.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.051+0000 7fd01b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:24:00.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.051+0000 7fd01b7fe640 1 -- 172.21.15.120:0/1421202294 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd034055060 con 0x55b0491d8b80 2024-09-19T09:24:00.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.051+0000 7fd01b7fe640 1 -- 172.21.15.120:0/1421202294 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd034030850 con 0x55b0491d8b80 2024-09-19T09:24:00.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.051+0000 7fd044782740 1 -- 172.21.15.120:0/1421202294 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd02c0043d0 con 0x55b0491d8b80 2024-09-19T09:24:00.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.052+0000 7fd01b7fe640 1 -- 172.21.15.120:0/1421202294 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd034034040 con 0x55b0491d8b80 2024-09-19T09:24:00.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.052+0000 7fd044782740 10 client.30958 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:00.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.052+0000 7fd044782740 1 -- 172.21.15.120:0/1421202294 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b0491e7220 con 0x55b0491d8b80 2024-09-19T09:24:00.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.052+0000 7fd01b7fe640 1 -- 172.21.15.120:0/1421202294 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd02c0043d0 con 0x55b0491d8b80 2024-09-19T09:24:00.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.052+0000 7fd01b7fe640 1 -- 172.21.15.120:0/1421202294 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fd03403ac70 con 0x55b0491d8b80 2024-09-19T09:24:00.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.052+0000 7fd044782740 10 client.30958 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:00.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.052+0000 7fd044782740 10 client.30958 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:24:00.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.052+0000 7fd044782740 20 client.30958 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:00.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.052+0000 7fd044782740 1 --2- 172.21.15.120:0/1421202294 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55b0491e8680 0x55b0491eaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:00.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.052+0000 7fd044782740 4 client.30958 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:24:00.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.052+0000 7fd044782740 1 -- 172.21.15.120:0/1421202294 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b049138860 con 0x55b0491e8680 2024-09-19T09:24:00.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.053+0000 7fd03bfff640 1 --2- 172.21.15.120:0/1421202294 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55b0491e8680 0x55b0491eaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:00.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.053+0000 7fd03bfff640 1 --2- 172.21.15.120:0/1421202294 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55b0491e8680 0x55b0491eaad0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=333d7b23c5c9bc44 server_cookie=f60d2afc5cd17103 in_seq=0 out_seq=0 2024-09-19T09:24:00.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.053+0000 7fd01b7fe640 10 client.30958 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:24:00.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.054+0000 7fd01b7fe640 1 -- 172.21.15.120:0/1421202294 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b049138860 con 0x55b0491e8680 2024-09-19T09:24:00.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.054+0000 7fd01b7fe640 10 client.30958 handle_command_reply: tid=0 2024-09-19T09:24:00.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.054+0000 7fd044782740 1 client.30958 shutdown 2024-09-19T09:24:00.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.054+0000 7fd044782740 1 -- 172.21.15.120:0/1421202294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0491d8b80 msgr2=0x55b0491d5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:00.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.054+0000 7fd044782740 1 --2- 172.21.15.120:0/1421202294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0491d8b80 0x55b0491d5d40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd034005e30 tx=0x7fd034005d80 comp rx=0 tx=0).stop 2024-09-19T09:24:00.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.054+0000 7fd044782740 1 -- 172.21.15.120:0/1421202294 shutdown_connections 2024-09-19T09:24:00.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.054+0000 7fd044782740 1 --2- 172.21.15.120:0/1421202294 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55b0491e8680 0x55b0491eaad0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.054+0000 7fd01b7fe640 0 client.30958 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:24:00.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.054+0000 7fd044782740 1 --2- 172.21.15.120:0/1421202294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0491d8b80 0x55b0491d5d40 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.054+0000 7fd044782740 1 --2- 172.21.15.120:0/1421202294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0491cf350 0x55b0491d57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.054+0000 7fd044782740 1 --2- 172.21.15.120:0/1421202294 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0491cdde0 0x55b0491d8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.054+0000 7fd044782740 1 -- 172.21.15.120:0/1421202294 >> 172.21.15.120:0/1421202294 conn(0x55b04918e310 msgr2=0x55b0491d0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:00.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.055+0000 7fd044782740 1 -- 172.21.15.120:0/1421202294 shutdown_connections 2024-09-19T09:24:00.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.055+0000 7fd044782740 1 -- 172.21.15.120:0/1421202294 wait complete. 2024-09-19T09:24:00.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.055+0000 7fd044782740 20 client.30958 trim_cache size 0 max 16384 2024-09-19T09:24:00.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.071+0000 7fd044782740 1 Processor -- start 2024-09-19T09:24:00.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.071+0000 7fd044782740 1 -- start start 2024-09-19T09:24:00.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.071+0000 7fd044782740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0491cdde0 0x55b0491e4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:00.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.071+0000 7fd044782740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0491cf350 0x55b0491d5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:00.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.071+0000 7fd044782740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0491d8b80 0x55b0491d47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:00.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.071+0000 7fd044782740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c1ab3f0 con 0x55b0491d8b80 2024-09-19T09:24:00.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.071+0000 7fd044782740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c1acf30 con 0x55b0491cf350 2024-09-19T09:24:00.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.071+0000 7fd044782740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c1abac0 con 0x55b0491cdde0 2024-09-19T09:24:00.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.071+0000 7fd033fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0491cf350 0x55b0491d5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:00.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.071+0000 7fd033fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0491cf350 0x55b0491d5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:54388/0 (socket says 172.21.15.120:54388) 2024-09-19T09:24:00.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.071+0000 7fd03bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0491cdde0 0x55b0491e4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:00.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.071+0000 7fd033fff640 1 -- 172.21.15.120:0/2041251489 learned_addr learned my addr 172.21.15.120:0/2041251489 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:00.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.072+0000 7fd033fff640 1 -- 172.21.15.120:0/2041251489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0491cdde0 msgr2=0x55b0491e4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:00.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.072+0000 7fd033fff640 1 --2- 172.21.15.120:0/2041251489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0491cdde0 0x55b0491e4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.072+0000 7fd033fff640 1 -- 172.21.15.120:0/2041251489 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0491d8b80 msgr2=0x55b0491d47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:00.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.072+0000 7fd033fff640 1 --2- 172.21.15.120:0/2041251489 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0491d8b80 0x55b0491d47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.072+0000 7fd033fff640 1 -- 172.21.15.120:0/2041251489 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0491d3d10 con 0x55b0491cf350 2024-09-19T09:24:00.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.072+0000 7fd03bfff640 1 --2- 172.21.15.120:0/2041251489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0491cdde0 0x55b0491e4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:24:00.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.072+0000 7fd033fff640 1 --2- 172.21.15.120:0/2041251489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0491cf350 0x55b0491d5080 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd02400c6a0 tx=0x7fd02401f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b96ae9e74e9e2db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:00.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.072+0000 7fd031ffb640 1 -- 172.21.15.120:0/2041251489 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02409a660 con 0x55b0491cf350 2024-09-19T09:24:00.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.072+0000 7fd031ffb640 1 -- 172.21.15.120:0/2041251489 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0240989d0 con 0x55b0491cf350 2024-09-19T09:24:00.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.072+0000 7fd031ffb640 1 -- 172.21.15.120:0/2041251489 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd024010430 con 0x55b0491cf350 2024-09-19T09:24:00.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.073+0000 7fd044782740 1 -- 172.21.15.120:0/2041251489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0491cf350 msgr2=0x55b0491d5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:00.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.073+0000 7fd044782740 1 --2- 172.21.15.120:0/2041251489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0491cf350 0x55b0491d5080 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd02400c6a0 tx=0x7fd02401f000 comp rx=0 tx=0).stop 2024-09-19T09:24:00.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.073+0000 7fd044782740 1 -- 172.21.15.120:0/2041251489 shutdown_connections 2024-09-19T09:24:00.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.073+0000 7fd044782740 1 --2- 172.21.15.120:0/2041251489 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0491d8b80 0x55b0491d47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.073+0000 7fd044782740 1 --2- 172.21.15.120:0/2041251489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0491cf350 0x55b0491d5080 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.073+0000 7fd044782740 1 --2- 172.21.15.120:0/2041251489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0491cdde0 0x55b0491e4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.073+0000 7fd044782740 1 -- 172.21.15.120:0/2041251489 >> 172.21.15.120:0/2041251489 conn(0x55b04918e310 msgr2=0x55b0491d4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:00.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.073+0000 7fd044782740 1 -- 172.21.15.120:0/2041251489 shutdown_connections 2024-09-19T09:24:00.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.073+0000 7fd044782740 1 -- 172.21.15.120:0/2041251489 wait complete. 2024-09-19T09:24:00.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.073+0000 7fd044782740 1 -- start start 2024-09-19T09:24:00.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.074+0000 7fd044782740 1 Processor -- start 2024-09-19T09:24:00.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.074+0000 7fd044782740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0491cdde0 0x55b04925c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:00.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.074+0000 7fd044782740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0491cf350 0x55b04925cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:00.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.074+0000 7fd044782740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0491d8b80 0x55b0491dc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:00.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.074+0000 7fd03bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0491cdde0 0x55b04925c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:00.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.074+0000 7fd03bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0491cdde0 0x55b04925c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:40990/0 (socket says 172.21.15.120:40990) 2024-09-19T09:24:00.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.074+0000 7fd03bfff640 1 -- 172.21.15.120:0/3206846770 learned_addr learned my addr 172.21.15.120:0/3206846770 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:00.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.074+0000 7fd044782740 1 -- 172.21.15.120:0/3206846770 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c1ac860 con 0x55b0491d8b80 2024-09-19T09:24:00.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.074+0000 7fd044782740 1 -- 172.21.15.120:0/3206846770 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c1ac190 con 0x55b0491cf350 2024-09-19T09:24:00.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.074+0000 7fd044782740 1 -- 172.21.15.120:0/3206846770 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c1a93f0 con 0x55b0491cdde0 2024-09-19T09:24:00.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.074+0000 7fd033fff640 1 --2- 172.21.15.120:0/3206846770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0491cf350 0x55b04925cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:00.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.074+0000 7fd040946640 1 --2- 172.21.15.120:0/3206846770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0491d8b80 0x55b0491dc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:00.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.074+0000 7fd044782740 10 client.0 _ll_register_callbacks cb 0x7fd034009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:00.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.075+0000 7fd03bfff640 1 -- 172.21.15.120:0/3206846770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0491cf350 msgr2=0x55b04925cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:00.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.075+0000 7fd03bfff640 1 --2- 172.21.15.120:0/3206846770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0491cf350 0x55b04925cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.075+0000 7fd03bfff640 1 -- 172.21.15.120:0/3206846770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0491d8b80 msgr2=0x55b0491dc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:00.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.075+0000 7fd03bfff640 1 --2- 172.21.15.120:0/3206846770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0491d8b80 0x55b0491dc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.075+0000 7fd03bfff640 1 -- 172.21.15.120:0/3206846770 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0491dc920 con 0x55b0491cdde0 2024-09-19T09:24:00.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.075+0000 7fd040946640 1 --2- 172.21.15.120:0/3206846770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0491d8b80 0x55b0491dc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:24:00.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.075+0000 7fd03bfff640 1 --2- 172.21.15.120:0/3206846770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0491cdde0 0x55b04925c640 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd02c009230 tx=0x7fd02c011de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab73aabe3fd151fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:00.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.075+0000 7fd01b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:24:00.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.075+0000 7fd01b7fe640 1 -- 172.21.15.120:0/3206846770 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c005480 con 0x55b0491cdde0 2024-09-19T09:24:00.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.075+0000 7fd01b7fe640 1 -- 172.21.15.120:0/3206846770 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd02c0035f0 con 0x55b0491cdde0 2024-09-19T09:24:00.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.075+0000 7fd044782740 1 -- 172.21.15.120:0/3206846770 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0240989d0 con 0x55b0491cdde0 2024-09-19T09:24:00.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.075+0000 7fd01b7fe640 1 -- 172.21.15.120:0/3206846770 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd02c005620 con 0x55b0491cdde0 2024-09-19T09:24:00.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.075+0000 7fd044782740 10 client.30964 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:00.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.075+0000 7fd044782740 1 -- 172.21.15.120:0/3206846770 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b049263dd0 con 0x55b0491cdde0 2024-09-19T09:24:00.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.075+0000 7fd01b7fe640 1 -- 172.21.15.120:0/3206846770 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0240989d0 con 0x55b0491cdde0 2024-09-19T09:24:00.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.076+0000 7fd01b7fe640 1 -- 172.21.15.120:0/3206846770 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fd02c019b70 con 0x55b0491cdde0 2024-09-19T09:24:00.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.076+0000 7fd044782740 10 client.30964 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:00.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.076+0000 7fd044782740 10 client.30964 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:24:00.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.076+0000 7fd044782740 20 client.30964 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:00.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.076+0000 7fd044782740 1 --2- 172.21.15.120:0/3206846770 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55b0492643b0 0x55b0492667a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:00.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.076+0000 7fd044782740 4 client.30964 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:24:00.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.076+0000 7fd044782740 1 -- 172.21.15.120:0/3206846770 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b049132d70 con 0x55b0492643b0 2024-09-19T09:24:00.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.076+0000 7fd033fff640 1 --2- 172.21.15.120:0/3206846770 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55b0492643b0 0x55b0492667a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:00.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.076+0000 7fd033fff640 1 --2- 172.21.15.120:0/3206846770 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55b0492643b0 0x55b0492667a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a8bdf198f98bdb81 server_cookie=6267c0080943477f in_seq=0 out_seq=0 2024-09-19T09:24:00.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.076+0000 7fd01b7fe640 10 client.30964 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:24:00.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.076+0000 7fd01b7fe640 1 -- 172.21.15.120:0/3206846770 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b049132d70 con 0x55b0492643b0 2024-09-19T09:24:00.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.076+0000 7fd01b7fe640 10 client.30964 handle_command_reply: tid=0 2024-09-19T09:24:00.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.076+0000 7fd044782740 1 client.30964 shutdown 2024-09-19T09:24:00.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.077+0000 7fd044782740 1 -- 172.21.15.120:0/3206846770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0491cdde0 msgr2=0x55b04925c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:00.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.077+0000 7fd044782740 1 --2- 172.21.15.120:0/3206846770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0491cdde0 0x55b04925c640 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd02c009230 tx=0x7fd02c011de0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.077+0000 7fd044782740 1 -- 172.21.15.120:0/3206846770 shutdown_connections 2024-09-19T09:24:00.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.077+0000 7fd044782740 1 --2- 172.21.15.120:0/3206846770 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55b0492643b0 0x55b0492667a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.077+0000 7fd044782740 1 --2- 172.21.15.120:0/3206846770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0491d8b80 0x55b0491dc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.077+0000 7fd01b7fe640 0 client.30964 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:24:00.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.077+0000 7fd044782740 1 --2- 172.21.15.120:0/3206846770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0491cf350 0x55b04925cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.077+0000 7fd044782740 1 --2- 172.21.15.120:0/3206846770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0491cdde0 0x55b04925c640 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.077+0000 7fd044782740 1 -- 172.21.15.120:0/3206846770 >> 172.21.15.120:0/3206846770 conn(0x55b04918e310 msgr2=0x55b0491d7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:00.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.077+0000 7fd044782740 1 -- 172.21.15.120:0/3206846770 shutdown_connections 2024-09-19T09:24:00.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.077+0000 7fd044782740 1 -- 172.21.15.120:0/3206846770 wait complete. 2024-09-19T09:24:00.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.077+0000 7fd044782740 20 client.30964 trim_cache size 0 max 16384 2024-09-19T09:24:00.088 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:24:00.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.077+0000 7fd0423a9640 1 -- 172.21.15.120:0/3273122963 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd0280a32c0 msgr2=0x7fd0280a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:00.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.077+0000 7fd0423a9640 1 --2- 172.21.15.120:0/3273122963 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd0280a32c0 0x7fd0280a5780 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd034006010 tx=0x7fd03403c040 comp rx=0 tx=0).stop 2024-09-19T09:24:00.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.077+0000 7fd0423a9640 1 -- 172.21.15.120:0/3273122963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd03c102570 msgr2=0x7fd03c19ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:00.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.077+0000 7fd0423a9640 1 --2- 172.21.15.120:0/3273122963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd03c102570 0x7fd03c19ab90 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7fd02400e9f0 tx=0x7fd02400eeb0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.078+0000 7fd0423a9640 1 -- 172.21.15.120:0/3273122963 shutdown_connections 2024-09-19T09:24:00.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.078+0000 7fd0423a9640 1 --2- 172.21.15.120:0/3273122963 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd0280a32c0 0x7fd0280a5780 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.078+0000 7fd0423a9640 1 --2- 172.21.15.120:0/3273122963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd03c102ef0 0x7fd03c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.078+0000 7fd0423a9640 1 --2- 172.21.15.120:0/3273122963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd03c102570 0x7fd03c19ab90 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.078+0000 7fd0423a9640 1 --2- 172.21.15.120:0/3273122963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd03c100e20 0x7fd03c19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:00.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.078+0000 7fd0423a9640 1 -- 172.21.15.120:0/3273122963 >> 172.21.15.120:0/3273122963 conn(0x7fd03c0fc7d0 msgr2=0x7fd03c06a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:00.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.078+0000 7fd0423a9640 1 -- 172.21.15.120:0/3273122963 shutdown_connections 2024-09-19T09:24:00.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:00.078+0000 7fd0423a9640 1 -- 172.21.15.120:0/3273122963 wait complete. 2024-09-19T09:24:00.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:00 smithi120 ceph-mon[31150]: pgmap v979: 97 pgs: 97 active+clean; 462 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.3 MiB/s wr, 768 op/s 2024-09-19T09:24:00.378 INFO:tasks.workunit.client.0.smithi120.stdout: 10 196629 21.44 MB/sec execute 355 sec latency 177.572 ms 2024-09-19T09:24:00.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:00 smithi196 ceph-mon[35150]: pgmap v979: 97 pgs: 97 active+clean; 462 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.3 MiB/s wr, 768 op/s 2024-09-19T09:24:00.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:00 smithi153 ceph-mon[35071]: pgmap v979: 97 pgs: 97 active+clean; 462 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.3 MiB/s wr, 768 op/s 2024-09-19T09:24:00.728 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:01.378 INFO:tasks.workunit.client.0.smithi120.stdout: 10 197020 21.44 MB/sec execute 356 sec latency 70.659 ms 2024-09-19T09:24:01.491 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:24:02.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.051+0000 7f3382b77640 1 -- 172.21.15.120:0/1623417908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f337c108da0 msgr2=0x7f337c10cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:02.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.051+0000 7f3382b77640 1 --2- 172.21.15.120:0/1623417908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f337c108da0 0x7f337c10cdf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3368009900 tx=0x7f336802a3f0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.051+0000 7f3382b77640 1 -- 172.21.15.120:0/1623417908 shutdown_connections 2024-09-19T09:24:02.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.051+0000 7f3382b77640 1 --2- 172.21.15.120:0/1623417908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f337c10d520 0x7f337c1150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.052+0000 7f3382b77640 1 --2- 172.21.15.120:0/1623417908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f337c108da0 0x7f337c10cdf0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.052+0000 7f3382b77640 1 --2- 172.21.15.120:0/1623417908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f337c1083f0 0x7f337c1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.052+0000 7f3382b77640 1 -- 172.21.15.120:0/1623417908 >> 172.21.15.120:0/1623417908 conn(0x7f337c077af0 msgr2=0x7f337c079f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:02.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.052+0000 7f3382b77640 1 -- 172.21.15.120:0/1623417908 shutdown_connections 2024-09-19T09:24:02.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.052+0000 7f3382b77640 1 -- 172.21.15.120:0/1623417908 wait complete. 2024-09-19T09:24:02.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.053+0000 7f3382b77640 1 Processor -- start 2024-09-19T09:24:02.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.053+0000 7f3382b77640 1 -- start start 2024-09-19T09:24:02.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.053+0000 7f3382b77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f337c1083f0 0x7f337c1a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.054+0000 7f3382b77640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f337c108da0 0x7f337c1a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.054+0000 7f3382b77640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f337c10d520 0x7f337c19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.054+0000 7f3382b77640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f337c118570 con 0x7f337c1083f0 2024-09-19T09:24:02.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.054+0000 7f3382b77640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1183f0 con 0x7f337c10d520 2024-09-19T09:24:02.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.054+0000 7f3382b77640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1186f0 con 0x7f337c108da0 2024-09-19T09:24:02.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.054+0000 7f3381114640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f337c10d520 0x7f337c19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.054+0000 7f3381114640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f337c10d520 0x7f337c19e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:53480/0 (socket says 172.21.15.120:53480) 2024-09-19T09:24:02.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.054+0000 7f3381114640 1 -- 172.21.15.120:0/3032894128 learned_addr learned my addr 172.21.15.120:0/3032894128 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:02.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.054+0000 7f337bfff640 1 --2- 172.21.15.120:0/3032894128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f337c108da0 0x7f337c1a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.054+0000 7f3380913640 1 --2- 172.21.15.120:0/3032894128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f337c1083f0 0x7f337c1a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.054+0000 7f3381114640 1 -- 172.21.15.120:0/3032894128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f337c108da0 msgr2=0x7f337c1a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:02.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.054+0000 7f3381114640 1 --2- 172.21.15.120:0/3032894128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f337c108da0 0x7f337c1a4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.055+0000 7f3381114640 1 -- 172.21.15.120:0/3032894128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f337c1083f0 msgr2=0x7f337c1a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:02.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.055+0000 7f3381114640 1 --2- 172.21.15.120:0/3032894128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f337c1083f0 0x7f337c1a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.055+0000 7f3381114640 1 -- 172.21.15.120:0/3032894128 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f337c19efb0 con 0x7f337c10d520 2024-09-19T09:24:02.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.055+0000 7f3381114640 1 --2- 172.21.15.120:0/3032894128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f337c10d520 0x7f337c19e880 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f337400eb10 tx=0x7f337400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:02.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.055+0000 7f3379ffb640 1 -- 172.21.15.120:0/3032894128 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3374004b20 con 0x7f337c10d520 2024-09-19T09:24:02.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.055+0000 7f3379ffb640 1 -- 172.21.15.120:0/3032894128 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3374004cc0 con 0x7f337c10d520 2024-09-19T09:24:02.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.056+0000 7f3382b77640 1 -- 172.21.15.120:0/3032894128 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f337c19f2a0 con 0x7f337c10d520 2024-09-19T09:24:02.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.056+0000 7f3382b77640 1 -- 172.21.15.120:0/3032894128 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f337c1ab590 con 0x7f337c10d520 2024-09-19T09:24:02.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.056+0000 7f3379ffb640 1 -- 172.21.15.120:0/3032894128 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3374010760 con 0x7f337c10d520 2024-09-19T09:24:02.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.061+0000 7f3379ffb640 1 -- 172.21.15.120:0/3032894128 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f3374005020 con 0x7f337c10d520 2024-09-19T09:24:02.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.063+0000 7f3379ffb640 1 --2- 172.21.15.120:0/3032894128 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f336c0a3340 0x7f336c0a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.063+0000 7f3384f50740 1 Processor -- start 2024-09-19T09:24:02.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.064+0000 7f3379ffb640 1 -- 172.21.15.120:0/3032894128 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f33740cfde0 con 0x7f337c10d520 2024-09-19T09:24:02.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.064+0000 7f3384f50740 1 -- start start 2024-09-19T09:24:02.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.064+0000 7f3384f50740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fa9349760 0x557fa9349b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.064+0000 7f3384f50740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fa934a080 0x557fa9354400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.064+0000 7f3384f50740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa9354940 0x557fa9356d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.064+0000 7f3384f50740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557fa92af2a0 con 0x557fa9354940 2024-09-19T09:24:02.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.064+0000 7f3384f50740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557fa9256b10 con 0x557fa9349760 2024-09-19T09:24:02.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.064+0000 7f3384f50740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557fa92ab260 con 0x557fa934a080 2024-09-19T09:24:02.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.064+0000 7f3380913640 1 --2- 172.21.15.120:0/3032894128 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f336c0a3340 0x7f336c0a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.064+0000 7f3381114640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa9354940 0x557fa9356d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.065+0000 7f3380913640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fa9349760 0x557fa9349b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.065+0000 7f3381114640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa9354940 0x557fa9356d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45228/0 (socket says 172.21.15.120:45228) 2024-09-19T09:24:02.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.065+0000 7f3381114640 1 -- 172.21.15.120:0/3171157300 learned_addr learned my addr 172.21.15.120:0/3171157300 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:02.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.065+0000 7f3381114640 1 -- 172.21.15.120:0/3171157300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fa934a080 msgr2=0x557fa9354400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:24:02.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.065+0000 7f3381114640 1 --2- 172.21.15.120:0/3171157300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fa934a080 0x557fa9354400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.065+0000 7f3381114640 1 -- 172.21.15.120:0/3171157300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fa9349760 msgr2=0x557fa9349b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:02.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.065+0000 7f3381114640 1 --2- 172.21.15.120:0/3171157300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fa9349760 0x557fa9349b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.065+0000 7f3381114640 1 -- 172.21.15.120:0/3171157300 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fa9020b90 con 0x557fa9354940 2024-09-19T09:24:02.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.065+0000 7f3381114640 1 --2- 172.21.15.120:0/3171157300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa9354940 0x557fa9356d30 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f3374098820 tx=0x7f3374098720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8533d49b06c919fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:02.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.066+0000 7f3380913640 1 --2- 172.21.15.120:0/3032894128 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f336c0a3340 0x7f336c0a5800 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f3370005d00 tx=0x7f3370005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:02.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.068+0000 7f3366ffd640 1 -- 172.21.15.120:0/3171157300 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337409cac0 con 0x557fa9354940 2024-09-19T09:24:02.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.069+0000 7f3366ffd640 1 -- 172.21.15.120:0/3171157300 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f337409c3c0 con 0x557fa9354940 2024-09-19T09:24:02.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.069+0000 7f3366ffd640 1 -- 172.21.15.120:0/3171157300 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3374091700 con 0x557fa9354940 2024-09-19T09:24:02.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.070+0000 7f3384f50740 1 -- 172.21.15.120:0/3171157300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa9354940 msgr2=0x557fa9356d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:02.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.071+0000 7f3384f50740 1 --2- 172.21.15.120:0/3171157300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa9354940 0x557fa9356d30 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f3374098820 tx=0x7f3374098720 comp rx=0 tx=0).stop 2024-09-19T09:24:02.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.071+0000 7f3384f50740 1 -- 172.21.15.120:0/3171157300 shutdown_connections 2024-09-19T09:24:02.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.071+0000 7f3384f50740 1 --2- 172.21.15.120:0/3171157300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa9354940 0x557fa9356d30 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.071+0000 7f3384f50740 1 --2- 172.21.15.120:0/3171157300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fa934a080 0x557fa9354400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.071+0000 7f3384f50740 1 --2- 172.21.15.120:0/3171157300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fa9349760 0x557fa9349b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.071+0000 7f3384f50740 1 -- 172.21.15.120:0/3171157300 >> 172.21.15.120:0/3171157300 conn(0x557fa9345cf0 msgr2=0x557fa93460d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:02.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.071+0000 7f3384f50740 1 -- 172.21.15.120:0/3171157300 shutdown_connections 2024-09-19T09:24:02.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.072+0000 7f3384f50740 1 -- 172.21.15.120:0/3171157300 wait complete. 2024-09-19T09:24:02.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.072+0000 7f3384f50740 1 -- start start 2024-09-19T09:24:02.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.072+0000 7f3384f50740 1 Processor -- start 2024-09-19T09:24:02.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.073+0000 7f3384f50740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fa9349760 0x557fa9351840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.073+0000 7f3384f50740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fa934a080 0x557fa9351d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.073+0000 7f3384f50740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa9354940 0x557fa93522c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.073+0000 7f3380913640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fa9349760 0x557fa9351840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.073+0000 7f3380913640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fa9349760 0x557fa9351840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36432/0 (socket says 172.21.15.120:36432) 2024-09-19T09:24:02.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.073+0000 7f3380913640 1 -- 172.21.15.120:0/1807275908 learned_addr learned my addr 172.21.15.120:0/1807275908 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:02.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.073+0000 7f3384f50740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557fa9357510 con 0x557fa9354940 2024-09-19T09:24:02.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.073+0000 7f3384f50740 1 -- 172.21.15.120:0/1807275908 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557fa92376e0 con 0x557fa934a080 2024-09-19T09:24:02.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.073+0000 7f3384f50740 1 -- 172.21.15.120:0/1807275908 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557fa9357690 con 0x557fa9349760 2024-09-19T09:24:02.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.073+0000 7f337bfff640 1 --2- 172.21.15.120:0/1807275908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fa934a080 0x557fa9351d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.073+0000 7f3384f50740 10 client.0 _ll_register_callbacks cb 0x557fa925ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:02.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.073+0000 7f3381114640 1 --2- 172.21.15.120:0/1807275908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa9354940 0x557fa93522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.074+0000 7f3380913640 1 -- 172.21.15.120:0/1807275908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fa934a080 msgr2=0x557fa9351d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:02.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.074+0000 7f3380913640 1 --2- 172.21.15.120:0/1807275908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fa934a080 0x557fa9351d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.074+0000 7f3380913640 1 -- 172.21.15.120:0/1807275908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa9354940 msgr2=0x557fa93522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:02.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.074+0000 7f3380913640 1 --2- 172.21.15.120:0/1807275908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa9354940 0x557fa93522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.074+0000 7f3380913640 1 -- 172.21.15.120:0/1807275908 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fa9360ac0 con 0x557fa9349760 2024-09-19T09:24:02.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.074+0000 7f337bfff640 1 --2- 172.21.15.120:0/1807275908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fa934a080 0x557fa9351d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:24:02.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.074+0000 7f3380913640 1 --2- 172.21.15.120:0/1807275908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fa9349760 0x557fa9351840 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3370011440 tx=0x7f33700046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed9cc2b066f45e63 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:02.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.074+0000 7f335bfff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:24:02.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.074+0000 7f335bfff640 1 -- 172.21.15.120:0/1807275908 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370023070 con 0x557fa9349760 2024-09-19T09:24:02.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.075+0000 7f335bfff640 1 -- 172.21.15.120:0/1807275908 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f337001f860 con 0x557fa9349760 2024-09-19T09:24:02.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.075+0000 7f3384f50740 1 -- 172.21.15.120:0/1807275908 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f337409c3c0 con 0x557fa9349760 2024-09-19T09:24:02.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.075+0000 7f335bfff640 1 -- 172.21.15.120:0/1807275908 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f337002d050 con 0x557fa9349760 2024-09-19T09:24:02.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.075+0000 7f3384f50740 10 client.30970 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:02.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.075+0000 7f3384f50740 1 -- 172.21.15.120:0/1807275908 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557fa9363470 con 0x557fa9349760 2024-09-19T09:24:02.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.075+0000 7f335bfff640 1 -- 172.21.15.120:0/1807275908 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f337409c3c0 con 0x557fa9349760 2024-09-19T09:24:02.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.076+0000 7f335bfff640 1 -- 172.21.15.120:0/1807275908 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f33700123f0 con 0x557fa9349760 2024-09-19T09:24:02.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.076+0000 7f3384f50740 10 client.30970 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:02.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.076+0000 7f3384f50740 10 client.30970 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:24:02.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.076+0000 7f3384f50740 20 client.30970 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:02.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.076+0000 7f3384f50740 1 --2- 172.21.15.120:0/1807275908 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557fa9364930 0x557fa9366d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.076+0000 7f3384f50740 4 client.30970 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:24:02.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.076+0000 7f3384f50740 1 -- 172.21.15.120:0/1807275908 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fa92bc9d0 con 0x557fa9364930 2024-09-19T09:24:02.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.077+0000 7f337bfff640 1 --2- 172.21.15.120:0/1807275908 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557fa9364930 0x557fa9366d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.078+0000 7f335bfff640 10 client.30970 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:02.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.078+0000 7f337bfff640 1 --2- 172.21.15.120:0/1807275908 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557fa9364930 0x557fa9366d80 crc :-1 s=READY pgs=521 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86c4cf5fab0425e3 server_cookie=8462f91f6a19334c in_seq=0 out_seq=0 2024-09-19T09:24:02.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.081+0000 7f335bfff640 1 -- 172.21.15.120:0/1807275908 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557fa92bc9d0 con 0x557fa9364930 2024-09-19T09:24:02.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.081+0000 7f335bfff640 10 client.30970 handle_command_reply: tid=0 2024-09-19T09:24:02.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.081+0000 7f3384f50740 1 client.30970 shutdown 2024-09-19T09:24:02.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.081+0000 7f3384f50740 1 -- 172.21.15.120:0/1807275908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fa9349760 msgr2=0x557fa9351840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:02.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.081+0000 7f3384f50740 1 --2- 172.21.15.120:0/1807275908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fa9349760 0x557fa9351840 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3370011440 tx=0x7f33700046c0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.082+0000 7f3384f50740 1 -- 172.21.15.120:0/1807275908 shutdown_connections 2024-09-19T09:24:02.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.082+0000 7f3384f50740 1 --2- 172.21.15.120:0/1807275908 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557fa9364930 0x557fa9366d80 crc :-1 s=READY pgs=521 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.082+0000 7f3384f50740 1 --2- 172.21.15.120:0/1807275908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa9354940 0x557fa93522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.082+0000 7f3384f50740 1 --2- 172.21.15.120:0/1807275908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fa934a080 0x557fa9351d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.082+0000 7f3384f50740 1 --2- 172.21.15.120:0/1807275908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fa9349760 0x557fa9351840 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.082+0000 7f335bfff640 0 client.30970 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:02.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.082+0000 7f3384f50740 1 -- 172.21.15.120:0/1807275908 >> 172.21.15.120:0/1807275908 conn(0x557fa9345cf0 msgr2=0x557fa934ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:02.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.082+0000 7f3384f50740 1 -- 172.21.15.120:0/1807275908 shutdown_connections 2024-09-19T09:24:02.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.082+0000 7f3384f50740 1 -- 172.21.15.120:0/1807275908 wait complete. 2024-09-19T09:24:02.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.082+0000 7f3384f50740 20 client.30970 trim_cache size 0 max 16384 2024-09-19T09:24:02.087 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:02.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.111+0000 7f3384f50740 1 Processor -- start 2024-09-19T09:24:02.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.111+0000 7f3384f50740 1 -- start start 2024-09-19T09:24:02.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.111+0000 7f3384f50740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fa9349760 0x557fa9353710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.111+0000 7f3384f50740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fa934a080 0x557fa9350110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.111+0000 7f3384f50740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa9354940 0x557fa9352110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.111+0000 7f3380913640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fa9349760 0x557fa9353710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.111+0000 7f3380913640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fa9349760 0x557fa9353710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:53508/0 (socket says 172.21.15.120:53508) 2024-09-19T09:24:02.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.111+0000 7f3380913640 1 -- 172.21.15.120:0/4159229237 learned_addr learned my addr 172.21.15.120:0/4159229237 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:02.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.111+0000 7f3384f50740 1 -- 172.21.15.120:0/4159229237 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f336000e560 con 0x557fa9354940 2024-09-19T09:24:02.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.111+0000 7f3384f50740 1 -- 172.21.15.120:0/4159229237 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f33600100a0 con 0x557fa9349760 2024-09-19T09:24:02.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.111+0000 7f3384f50740 1 -- 172.21.15.120:0/4159229237 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f336000ec30 con 0x557fa934a080 2024-09-19T09:24:02.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.112+0000 7f3380913640 1 -- 172.21.15.120:0/4159229237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fa934a080 msgr2=0x557fa9350110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:02.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.112+0000 7f3380913640 1 --2- 172.21.15.120:0/4159229237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fa934a080 0x557fa9350110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.112+0000 7f3380913640 1 -- 172.21.15.120:0/4159229237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa9354940 msgr2=0x557fa9352110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:02.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.112+0000 7f3380913640 1 --2- 172.21.15.120:0/4159229237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa9354940 0x557fa9352110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.112+0000 7f3380913640 1 -- 172.21.15.120:0/4159229237 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fa9341260 con 0x557fa9349760 2024-09-19T09:24:02.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.112+0000 7f3380913640 1 --2- 172.21.15.120:0/4159229237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fa9349760 0x557fa9353710 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3370006010 tx=0x7f3370012c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=767e1664c5e9e9fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:02.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.113+0000 7f33667fc640 1 -- 172.21.15.120:0/4159229237 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370023070 con 0x557fa9349760 2024-09-19T09:24:02.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.113+0000 7f33667fc640 1 -- 172.21.15.120:0/4159229237 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3370012600 con 0x557fa9349760 2024-09-19T09:24:02.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.114+0000 7f3384f50740 1 -- 172.21.15.120:0/4159229237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fa9349760 msgr2=0x557fa9353710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:02.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.114+0000 7f3384f50740 1 --2- 172.21.15.120:0/4159229237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fa9349760 0x557fa9353710 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3370006010 tx=0x7f3370012c10 comp rx=0 tx=0).stop 2024-09-19T09:24:02.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.114+0000 7f3384f50740 1 -- 172.21.15.120:0/4159229237 shutdown_connections 2024-09-19T09:24:02.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.114+0000 7f3384f50740 1 --2- 172.21.15.120:0/4159229237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa9354940 0x557fa9352110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.114+0000 7f3384f50740 1 --2- 172.21.15.120:0/4159229237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fa934a080 0x557fa9350110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.114+0000 7f3384f50740 1 --2- 172.21.15.120:0/4159229237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fa9349760 0x557fa9353710 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.114+0000 7f3384f50740 1 -- 172.21.15.120:0/4159229237 >> 172.21.15.120:0/4159229237 conn(0x557fa9345cf0 msgr2=0x557fa9344e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:02.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.114+0000 7f3384f50740 1 -- 172.21.15.120:0/4159229237 shutdown_connections 2024-09-19T09:24:02.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.115+0000 7f3384f50740 1 -- 172.21.15.120:0/4159229237 wait complete. 2024-09-19T09:24:02.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.115+0000 7f3384f50740 1 -- start start 2024-09-19T09:24:02.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.115+0000 7f3384f50740 1 Processor -- start 2024-09-19T09:24:02.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.116+0000 7f3384f50740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fa9349760 0x557fa9358db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.116+0000 7f3384f50740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa934a080 0x557fa93592f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.116+0000 7f3384f50740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fa9354940 0x557fa93ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.116+0000 7f3384f50740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f336000f9d0 con 0x557fa934a080 2024-09-19T09:24:02.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.116+0000 7f3384f50740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f336000f300 con 0x557fa9349760 2024-09-19T09:24:02.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.116+0000 7f3384f50740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f336000c520 con 0x557fa9354940 2024-09-19T09:24:02.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.116+0000 7f3384f50740 10 client.0 _ll_register_callbacks cb 0x7f337000bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:02.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.116+0000 7f3381114640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fa9354940 0x557fa93ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.116+0000 7f3380913640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fa9349760 0x557fa9358db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.116+0000 7f3381114640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fa9354940 0x557fa93ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36448/0 (socket says 172.21.15.120:36448) 2024-09-19T09:24:02.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.116+0000 7f3381114640 1 -- 172.21.15.120:0/1636624631 learned_addr learned my addr 172.21.15.120:0/1636624631 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:02.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.117+0000 7f3381114640 1 -- 172.21.15.120:0/1636624631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fa9349760 msgr2=0x557fa9358db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:02.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.117+0000 7f3381114640 1 --2- 172.21.15.120:0/1636624631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fa9349760 0x557fa9358db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.117+0000 7f3381114640 1 -- 172.21.15.120:0/1636624631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa934a080 msgr2=0x557fa93592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:02.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.117+0000 7f3381114640 1 --2- 172.21.15.120:0/1636624631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa934a080 0x557fa93592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.117+0000 7f3381114640 1 -- 172.21.15.120:0/1636624631 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fa9346980 con 0x557fa9354940 2024-09-19T09:24:02.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.119+0000 7f3381114640 1 --2- 172.21.15.120:0/1636624631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fa9354940 0x557fa93ce620 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f337409c0c0 tx=0x7f3374091930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=226e8d700655d8fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:02.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.119+0000 7f335bfff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:24:02.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.119+0000 7f335bfff640 1 -- 172.21.15.120:0/1636624631 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33740b5060 con 0x557fa9354940 2024-09-19T09:24:02.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.119+0000 7f335bfff640 1 -- 172.21.15.120:0/1636624631 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f337409c830 con 0x557fa9354940 2024-09-19T09:24:02.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.119+0000 7f3384f50740 1 -- 172.21.15.120:0/1636624631 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557fa92b6d20 con 0x557fa9354940 2024-09-19T09:24:02.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.119+0000 7f335bfff640 1 -- 172.21.15.120:0/1636624631 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f33740ba050 con 0x557fa9354940 2024-09-19T09:24:02.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.119+0000 7f3384f50740 10 client.30973 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:02.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.119+0000 7f3384f50740 1 -- 172.21.15.120:0/1636624631 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557fa93cfbd0 con 0x557fa9354940 2024-09-19T09:24:02.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.120+0000 7f335bfff640 1 -- 172.21.15.120:0/1636624631 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557fa92b6d20 con 0x557fa9354940 2024-09-19T09:24:02.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.120+0000 7f335bfff640 1 -- 172.21.15.120:0/1636624631 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f33740a4ba0 con 0x557fa9354940 2024-09-19T09:24:02.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.120+0000 7f3384f50740 10 client.30973 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:02.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.120+0000 7f3384f50740 10 client.30973 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:24:02.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.120+0000 7f3384f50740 20 client.30973 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:02.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.120+0000 7f3384f50740 1 --2- 172.21.15.120:0/1636624631 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557fa93cfe00 0x557fa93d21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.120+0000 7f3384f50740 4 client.30973 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-09-19T09:24:02.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.120+0000 7f3384f50740 1 -- 172.21.15.120:0/1636624631 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x557fa9232ec0 con 0x557fa93cfe00 2024-09-19T09:24:02.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.121+0000 7f3380913640 1 --2- 172.21.15.120:0/1636624631 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557fa93cfe00 0x557fa93d21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.122+0000 7f3380913640 1 --2- 172.21.15.120:0/1636624631 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557fa93cfe00 0x557fa93d21f0 crc :-1 s=READY pgs=522 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9573a0cb5eb7022b server_cookie=22fa0f5d9225f0b4 in_seq=0 out_seq=0 2024-09-19T09:24:02.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.124+0000 7f335bfff640 10 client.30973 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:02.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:02 smithi120 ceph-mon[31150]: pgmap v980: 97 pgs: 97 active+clean; 448 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.5 MiB/s wr, 712 op/s 2024-09-19T09:24:02.378 INFO:tasks.workunit.client.0.smithi120.stdout: 10 197294 21.40 MB/sec execute 357 sec latency 250.989 ms 2024-09-19T09:24:02.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:02 smithi196 ceph-mon[35150]: pgmap v980: 97 pgs: 97 active+clean; 448 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.5 MiB/s wr, 712 op/s 2024-09-19T09:24:02.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:02 smithi153 ceph-mon[35071]: pgmap v980: 97 pgs: 97 active+clean; 448 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.5 MiB/s wr, 712 op/s 2024-09-19T09:24:02.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.657+0000 7f04c5230640 1 -- 172.21.15.120:0/1185151470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04c01012d0 msgr2=0x7f04c0107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:02.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.657+0000 7f04c5230640 1 --2- 172.21.15.120:0/1185151470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04c01012d0 0x7f04c0107a20 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009900 tx=0x7f04b402a410 comp rx=0 tx=0).stop 2024-09-19T09:24:02.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.657+0000 7f04c5230640 1 -- 172.21.15.120:0/1185151470 shutdown_connections 2024-09-19T09:24:02.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.657+0000 7f04c5230640 1 --2- 172.21.15.120:0/1185151470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04c0108150 0x7f04c010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.657+0000 7f04c5230640 1 --2- 172.21.15.120:0/1185151470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04c01012d0 0x7f04c0107a20 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.657+0000 7f04c5230640 1 --2- 172.21.15.120:0/1185151470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04c0100920 0x7f04c0100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.657+0000 7f04c5230640 1 -- 172.21.15.120:0/1185151470 >> 172.21.15.120:0/1185151470 conn(0x7f04c0075f70 msgr2=0x7f04c0076380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:02.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.657+0000 7f04c5230640 1 -- 172.21.15.120:0/1185151470 shutdown_connections 2024-09-19T09:24:02.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.657+0000 7f04c5230640 1 -- 172.21.15.120:0/1185151470 wait complete. 2024-09-19T09:24:02.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.658+0000 7f04c5230640 1 Processor -- start 2024-09-19T09:24:02.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.658+0000 7f04c5230640 1 -- start start 2024-09-19T09:24:02.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.658+0000 7f04c5230640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04c0100920 0x7f04c010be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.658+0000 7f04c5230640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04c01012d0 0x7f04c010c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.658+0000 7f04c5230640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04c0108150 0x7f04c010c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.658+0000 7f04c5230640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f04c0103660 con 0x7f04c0108150 2024-09-19T09:24:02.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.658+0000 7f04c5230640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f04c01034e0 con 0x7f04c0100920 2024-09-19T09:24:02.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.658+0000 7f04c5230640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f04c01037e0 con 0x7f04c01012d0 2024-09-19T09:24:02.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.659+0000 7f04bf59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04c0108150 0x7f04c010c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.659+0000 7f04be59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04c01012d0 0x7f04c010c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.659+0000 7f04bf59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04c0108150 0x7f04c010c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45252/0 (socket says 172.21.15.120:45252) 2024-09-19T09:24:02.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.659+0000 7f04bed9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04c0100920 0x7f04c010be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.659+0000 7f04bf59e640 1 -- 172.21.15.120:0/1205255117 learned_addr learned my addr 172.21.15.120:0/1205255117 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:02.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.659+0000 7f04bf59e640 1 -- 172.21.15.120:0/1205255117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04c01012d0 msgr2=0x7f04c010c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:02.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.659+0000 7f04bf59e640 1 --2- 172.21.15.120:0/1205255117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04c01012d0 0x7f04c010c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.659+0000 7f04bf59e640 1 -- 172.21.15.120:0/1205255117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04c0100920 msgr2=0x7f04c010be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:02.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.659+0000 7f04bf59e640 1 --2- 172.21.15.120:0/1205255117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04c0100920 0x7f04c010be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.659+0000 7f04bf59e640 1 -- 172.21.15.120:0/1205255117 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c01aacf0 con 0x7f04c0108150 2024-09-19T09:24:02.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.659+0000 7f04bed9d640 1 --2- 172.21.15.120:0/1205255117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04c0100920 0x7f04c010be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:24:02.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.659+0000 7f04bf59e640 1 --2- 172.21.15.120:0/1205255117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04c0108150 0x7f04c010c8b0 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f04b800eb10 tx=0x7f04b800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:02.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.660+0000 7f04a7fff640 1 -- 172.21.15.120:0/1205255117 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b8004b20 con 0x7f04c0108150 2024-09-19T09:24:02.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.660+0000 7f04c5230640 1 -- 172.21.15.120:0/1205255117 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c01aafe0 con 0x7f04c0108150 2024-09-19T09:24:02.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.660+0000 7f04a7fff640 1 -- 172.21.15.120:0/1205255117 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04b8004cc0 con 0x7f04c0108150 2024-09-19T09:24:02.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.660+0000 7f04a7fff640 1 -- 172.21.15.120:0/1205255117 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b8010760 con 0x7f04c0108150 2024-09-19T09:24:02.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.660+0000 7f04c5230640 1 -- 172.21.15.120:0/1205255117 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c01ab440 con 0x7f04c0108150 2024-09-19T09:24:02.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.663+0000 7f04c7609740 1 Processor -- start 2024-09-19T09:24:02.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.663+0000 7f04a7fff640 1 -- 172.21.15.120:0/1205255117 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f04b8005020 con 0x7f04c0108150 2024-09-19T09:24:02.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.664+0000 7f04c7609740 1 -- start start 2024-09-19T09:24:02.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.664+0000 7f04c7609740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556eb1e19b40 0x556eb1e18090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.664+0000 7f04c7609740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb1e185d0 0x556eb1e18a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.664+0000 7f04c7609740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556eb1e23370 0x556eb1e25760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.664+0000 7f04c7609740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556eb1d7e2f0 con 0x556eb1e185d0 2024-09-19T09:24:02.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.664+0000 7f04c7609740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556eb1d25cf0 con 0x556eb1e19b40 2024-09-19T09:24:02.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.664+0000 7f04c7609740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556eb1d7a2b0 con 0x556eb1e23370 2024-09-19T09:24:02.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.664+0000 7f04bed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb1e185d0 0x556eb1e18a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.664+0000 7f04a7fff640 1 --2- 172.21.15.120:0/1205255117 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f04a80a3340 0x7f04a80a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.664+0000 7f04bed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb1e185d0 0x556eb1e18a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45262/0 (socket says 172.21.15.120:45262) 2024-09-19T09:24:02.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.664+0000 7f04bed9d640 1 -- 172.21.15.120:0/905516894 learned_addr learned my addr 172.21.15.120:0/905516894 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:02.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.664+0000 7f04be59c640 1 --2- 172.21.15.120:0/905516894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556eb1e19b40 0x556eb1e18090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.664+0000 7f04be59c640 1 --2- 172.21.15.120:0/905516894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556eb1e23370 0x556eb1e25760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.665+0000 7f04a7fff640 1 -- 172.21.15.120:0/1205255117 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f04b80cfe90 con 0x7f04c0108150 2024-09-19T09:24:02.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.665+0000 7f04be59c640 1 -- 172.21.15.120:0/905516894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556eb1e23370 msgr2=0x556eb1e25760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:02.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.665+0000 7f04be59c640 1 --2- 172.21.15.120:0/905516894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556eb1e23370 0x556eb1e25760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.665+0000 7f04be59c640 1 -- 172.21.15.120:0/905516894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb1e185d0 msgr2=0x556eb1e18a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:02.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.665+0000 7f04be59c640 1 --2- 172.21.15.120:0/905516894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb1e185d0 0x556eb1e18a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.665+0000 7f04be59c640 1 -- 172.21.15.120:0/905516894 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eb1d0a3a0 con 0x556eb1e19b40 2024-09-19T09:24:02.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.665+0000 7f04bf59e640 1 --2- 172.21.15.120:0/1205255117 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f04a80a3340 0x7f04a80a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.665+0000 7f04be59c640 1 --2- 172.21.15.120:0/905516894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556eb1e19b40 0x556eb1e18090 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009230 tx=0x7f04b40058b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9293eb3610210181 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:02.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.665+0000 7f04a57fa640 1 -- 172.21.15.120:0/905516894 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b4035040 con 0x556eb1e19b40 2024-09-19T09:24:02.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.665+0000 7f04a57fa640 1 -- 172.21.15.120:0/905516894 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04b4031c80 con 0x556eb1e19b40 2024-09-19T09:24:02.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.665+0000 7f04a57fa640 1 -- 172.21.15.120:0/905516894 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b4003d70 con 0x556eb1e19b40 2024-09-19T09:24:02.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.665+0000 7f04bf59e640 1 --2- 172.21.15.120:0/1205255117 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f04a80a3340 0x7f04a80a5800 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f04b8098e40 tx=0x7f04b8098040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:02.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.666+0000 7f04c7609740 1 -- 172.21.15.120:0/905516894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556eb1e19b40 msgr2=0x556eb1e18090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:02.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.666+0000 7f04c7609740 1 --2- 172.21.15.120:0/905516894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556eb1e19b40 0x556eb1e18090 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009230 tx=0x7f04b40058b0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.666+0000 7f04c7609740 1 -- 172.21.15.120:0/905516894 shutdown_connections 2024-09-19T09:24:02.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.666+0000 7f04c7609740 1 --2- 172.21.15.120:0/905516894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556eb1e23370 0x556eb1e25760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.667+0000 7f04c7609740 1 --2- 172.21.15.120:0/905516894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb1e185d0 0x556eb1e18a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.667+0000 7f04c7609740 1 --2- 172.21.15.120:0/905516894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556eb1e19b40 0x556eb1e18090 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.667+0000 7f04c7609740 1 -- 172.21.15.120:0/905516894 >> 172.21.15.120:0/905516894 conn(0x556eb1de1310 msgr2=0x556eb1e14c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:02.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.667+0000 7f04c7609740 1 -- 172.21.15.120:0/905516894 shutdown_connections 2024-09-19T09:24:02.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.667+0000 7f04c7609740 1 -- 172.21.15.120:0/905516894 wait complete. 2024-09-19T09:24:02.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.667+0000 7f04c7609740 1 -- start start 2024-09-19T09:24:02.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.667+0000 7f04c7609740 1 Processor -- start 2024-09-19T09:24:02.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.668+0000 7f04c7609740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556eb1e185d0 0x556eb1e22dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.668+0000 7f04c7609740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556eb1e19b40 0x556eb1e20070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.668+0000 7f04c7609740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb1e23370 0x556eb1e20800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.668+0000 7f04c7609740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556eb1e26010 con 0x556eb1e23370 2024-09-19T09:24:02.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.668+0000 7f04c7609740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556eb1d39060 con 0x556eb1e185d0 2024-09-19T09:24:02.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.668+0000 7f04c7609740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556eb1e26190 con 0x556eb1e19b40 2024-09-19T09:24:02.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.668+0000 7f04c7609740 10 client.0 _ll_register_callbacks cb 0x556eb1a8c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:02.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.668+0000 7f04be59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb1e23370 0x556eb1e20800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.668+0000 7f04be59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb1e23370 0x556eb1e20800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45274/0 (socket says 172.21.15.120:45274) 2024-09-19T09:24:02.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.668+0000 7f04be59c640 1 -- 172.21.15.120:0/1103078254 learned_addr learned my addr 172.21.15.120:0/1103078254 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:02.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.668+0000 7f04be59c640 1 --2- 172.21.15.120:0/1103078254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556eb1e185d0 0x556eb1e22dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.668+0000 7f04bed9d640 1 --2- 172.21.15.120:0/1103078254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556eb1e19b40 0x556eb1e20070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.668+0000 7f04be59c640 1 -- 172.21.15.120:0/1103078254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556eb1e19b40 msgr2=0x556eb1e20070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:02.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.668+0000 7f04be59c640 1 --2- 172.21.15.120:0/1103078254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556eb1e19b40 0x556eb1e20070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.668+0000 7f04be59c640 1 -- 172.21.15.120:0/1103078254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556eb1e185d0 msgr2=0x556eb1e22dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:02.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.668+0000 7f04be59c640 1 --2- 172.21.15.120:0/1103078254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556eb1e185d0 0x556eb1e22dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.668+0000 7f04be59c640 1 -- 172.21.15.120:0/1103078254 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eb1aef3c0 con 0x556eb1e23370 2024-09-19T09:24:02.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.668+0000 7f04bed9d640 1 --2- 172.21.15.120:0/1103078254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556eb1e19b40 0x556eb1e20070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:24:02.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.668+0000 7f04be59c640 1 --2- 172.21.15.120:0/1103078254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb1e23370 0x556eb1e20800 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009320 tx=0x7f04b4009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c7b8730d20cc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:02.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.668+0000 7f04a2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:02.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.669+0000 7f04a2ff5640 1 -- 172.21.15.120:0/1103078254 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b4035030 con 0x556eb1e23370 2024-09-19T09:24:02.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.669+0000 7f04a2ff5640 1 -- 172.21.15.120:0/1103078254 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f04b40039f0 con 0x556eb1e23370 2024-09-19T09:24:02.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.669+0000 7f04c7609740 1 -- 172.21.15.120:0/1103078254 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04b4031c80 con 0x556eb1e23370 2024-09-19T09:24:02.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.669+0000 7f04a2ff5640 1 -- 172.21.15.120:0/1103078254 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f04b4003bf0 con 0x556eb1e23370 2024-09-19T09:24:02.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.669+0000 7f04c7609740 10 client.22368 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:02.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.669+0000 7f04c7609740 1 -- 172.21.15.120:0/1103078254 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556eb1e31b20 con 0x556eb1e23370 2024-09-19T09:24:02.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.669+0000 7f04a2ff5640 1 -- 172.21.15.120:0/1103078254 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04b4031c80 con 0x556eb1e23370 2024-09-19T09:24:02.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.669+0000 7f04a2ff5640 1 -- 172.21.15.120:0/1103078254 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f04b403a040 con 0x556eb1e23370 2024-09-19T09:24:02.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.670+0000 7f04c7609740 10 client.22368 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:02.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.670+0000 7f04c7609740 10 client.22368 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:24:02.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.670+0000 7f04c7609740 20 client.22368 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:02.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.670+0000 7f04c7609740 1 --2- 172.21.15.120:0/1103078254 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x556eb1e32e70 0x556eb1e35260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.670+0000 7f04c7609740 4 client.22368 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:24:02.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.670+0000 7f04c7609740 1 -- 172.21.15.120:0/1103078254 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556eb1d8b860 con 0x556eb1e32e70 2024-09-19T09:24:02.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.670+0000 7f04bed9d640 1 --2- 172.21.15.120:0/1103078254 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x556eb1e32e70 0x556eb1e35260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.671+0000 7f04bed9d640 1 --2- 172.21.15.120:0/1103078254 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x556eb1e32e70 0x556eb1e35260 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=60bbd8d116414ae2 server_cookie=81c496589bc50fbc in_seq=0 out_seq=0 2024-09-19T09:24:02.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.671+0000 7f04a2ff5640 10 client.22368 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:24:02.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.671+0000 7f04a2ff5640 1 -- 172.21.15.120:0/1103078254 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556eb1d8b860 con 0x556eb1e32e70 2024-09-19T09:24:02.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.672+0000 7f04a2ff5640 10 client.22368 handle_command_reply: tid=0 2024-09-19T09:24:02.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.672+0000 7f04c7609740 1 client.22368 shutdown 2024-09-19T09:24:02.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.672+0000 7f04c7609740 1 -- 172.21.15.120:0/1103078254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb1e23370 msgr2=0x556eb1e20800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:02.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.672+0000 7f04c7609740 1 --2- 172.21.15.120:0/1103078254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb1e23370 0x556eb1e20800 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009320 tx=0x7f04b4009560 comp rx=0 tx=0).stop 2024-09-19T09:24:02.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.672+0000 7f04c7609740 1 -- 172.21.15.120:0/1103078254 shutdown_connections 2024-09-19T09:24:02.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.672+0000 7f04c7609740 1 --2- 172.21.15.120:0/1103078254 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x556eb1e32e70 0x556eb1e35260 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.672+0000 7f04c7609740 1 --2- 172.21.15.120:0/1103078254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb1e23370 0x556eb1e20800 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.672+0000 7f04c7609740 1 --2- 172.21.15.120:0/1103078254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556eb1e19b40 0x556eb1e20070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.672+0000 7f04c7609740 1 --2- 172.21.15.120:0/1103078254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556eb1e185d0 0x556eb1e22dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.672+0000 7f04c7609740 1 -- 172.21.15.120:0/1103078254 >> 172.21.15.120:0/1103078254 conn(0x556eb1de1310 msgr2=0x556eb1e1a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:02.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.672+0000 7f04a2ff5640 0 client.22368 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:24:02.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.673+0000 7f04c7609740 1 -- 172.21.15.120:0/1103078254 shutdown_connections 2024-09-19T09:24:02.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.673+0000 7f04c7609740 1 -- 172.21.15.120:0/1103078254 wait complete. 2024-09-19T09:24:02.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.673+0000 7f04c7609740 20 client.22368 trim_cache size 0 max 16384 2024-09-19T09:24:02.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.690+0000 7f04c7609740 1 Processor -- start 2024-09-19T09:24:02.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.690+0000 7f04c7609740 1 -- start start 2024-09-19T09:24:02.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.690+0000 7f04c7609740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556eb1e185d0 0x556eb1e1f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.690+0000 7f04c7609740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556eb1e19b40 0x556eb1e2eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.690+0000 7f04c7609740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb1e23370 0x556eb1e37ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.690+0000 7f04c7609740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f04c01b3b90 con 0x556eb1e23370 2024-09-19T09:24:02.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.690+0000 7f04c7609740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f04c01b56d0 con 0x556eb1e19b40 2024-09-19T09:24:02.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.690+0000 7f04c7609740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f04c01b4260 con 0x556eb1e185d0 2024-09-19T09:24:02.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.690+0000 7f04bed9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556eb1e19b40 0x556eb1e2eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.690+0000 7f04bed9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556eb1e19b40 0x556eb1e2eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:53558/0 (socket says 172.21.15.120:53558) 2024-09-19T09:24:02.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.690+0000 7f04bed9d640 1 -- 172.21.15.120:0/2236626861 learned_addr learned my addr 172.21.15.120:0/2236626861 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:02.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.690+0000 7f04be59c640 1 --2- 172.21.15.120:0/2236626861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb1e23370 0x556eb1e37ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.690+0000 7f04be59c640 1 --2- 172.21.15.120:0/2236626861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556eb1e185d0 0x556eb1e1f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.690+0000 7f04bed9d640 1 -- 172.21.15.120:0/2236626861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556eb1e185d0 msgr2=0x556eb1e1f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:02.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.690+0000 7f04bed9d640 1 --2- 172.21.15.120:0/2236626861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556eb1e185d0 0x556eb1e1f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.690+0000 7f04bed9d640 1 -- 172.21.15.120:0/2236626861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb1e23370 msgr2=0x556eb1e37ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:02.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.690+0000 7f04bed9d640 1 --2- 172.21.15.120:0/2236626861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb1e23370 0x556eb1e37ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.690+0000 7f04bed9d640 1 -- 172.21.15.120:0/2236626861 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eb1e23130 con 0x556eb1e19b40 2024-09-19T09:24:02.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.691+0000 7f04bed9d640 1 --2- 172.21.15.120:0/2236626861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556eb1e19b40 0x556eb1e2eb30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x556eb1e216b0 tx=0x7f04b000b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1588c5ceb6c27294 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:02.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.691+0000 7f04a4ff9640 1 -- 172.21.15.120:0/2236626861 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b0010b60 con 0x556eb1e19b40 2024-09-19T09:24:02.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.691+0000 7f04a4ff9640 1 -- 172.21.15.120:0/2236626861 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04b0010d00 con 0x556eb1e19b40 2024-09-19T09:24:02.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.691+0000 7f04a4ff9640 1 -- 172.21.15.120:0/2236626861 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b0025760 con 0x556eb1e19b40 2024-09-19T09:24:02.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.692+0000 7f04c7609740 1 -- 172.21.15.120:0/2236626861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556eb1e19b40 msgr2=0x556eb1e2eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:02.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.692+0000 7f04c7609740 1 --2- 172.21.15.120:0/2236626861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556eb1e19b40 0x556eb1e2eb30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x556eb1e216b0 tx=0x7f04b000b390 comp rx=0 tx=0).stop 2024-09-19T09:24:02.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.692+0000 7f04c7609740 1 -- 172.21.15.120:0/2236626861 shutdown_connections 2024-09-19T09:24:02.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.692+0000 7f04c7609740 1 --2- 172.21.15.120:0/2236626861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb1e23370 0x556eb1e37ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.692+0000 7f04c7609740 1 --2- 172.21.15.120:0/2236626861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556eb1e19b40 0x556eb1e2eb30 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.692+0000 7f04c7609740 1 --2- 172.21.15.120:0/2236626861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556eb1e185d0 0x556eb1e1f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.692+0000 7f04c7609740 1 -- 172.21.15.120:0/2236626861 >> 172.21.15.120:0/2236626861 conn(0x556eb1de1310 msgr2=0x556eb1e2e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:02.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.692+0000 7f04c7609740 1 -- 172.21.15.120:0/2236626861 shutdown_connections 2024-09-19T09:24:02.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.692+0000 7f04c7609740 1 -- 172.21.15.120:0/2236626861 wait complete. 2024-09-19T09:24:02.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.693+0000 7f04c7609740 1 -- start start 2024-09-19T09:24:02.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.693+0000 7f04c7609740 1 Processor -- start 2024-09-19T09:24:02.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.693+0000 7f04c7609740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556eb1e185d0 0x556eb1e34e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.693+0000 7f04c7609740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb1e19b40 0x556eb1e353c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.693+0000 7f04c7609740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556eb1e23370 0x556eb1e35900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.693+0000 7f04c7609740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f04c01b5000 con 0x556eb1e19b40 2024-09-19T09:24:02.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.693+0000 7f04c7609740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f04c01b4930 con 0x556eb1e185d0 2024-09-19T09:24:02.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.693+0000 7f04c7609740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f04c01b1b50 con 0x556eb1e23370 2024-09-19T09:24:02.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.693+0000 7f04c7609740 10 client.0 _ll_register_callbacks cb 0x7f04b000bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:02.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.693+0000 7f04bed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb1e19b40 0x556eb1e353c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.693+0000 7f04bed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb1e19b40 0x556eb1e353c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45306/0 (socket says 172.21.15.120:45306) 2024-09-19T09:24:02.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.693+0000 7f04bed9d640 1 -- 172.21.15.120:0/3011499788 learned_addr learned my addr 172.21.15.120:0/3011499788 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:02.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.693+0000 7f04be59c640 1 --2- 172.21.15.120:0/3011499788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556eb1e185d0 0x556eb1e34e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.693+0000 7f04be59c640 1 --2- 172.21.15.120:0/3011499788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556eb1e23370 0x556eb1e35900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.694+0000 7f04bed9d640 1 -- 172.21.15.120:0/3011499788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556eb1e23370 msgr2=0x556eb1e35900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:02.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.694+0000 7f04bed9d640 1 --2- 172.21.15.120:0/3011499788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556eb1e23370 0x556eb1e35900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.694+0000 7f04bed9d640 1 -- 172.21.15.120:0/3011499788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556eb1e185d0 msgr2=0x556eb1e34e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:02.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.694+0000 7f04bed9d640 1 --2- 172.21.15.120:0/3011499788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556eb1e185d0 0x556eb1e34e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.694+0000 7f04bed9d640 1 -- 172.21.15.120:0/3011499788 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eb1e9b8d0 con 0x556eb1e19b40 2024-09-19T09:24:02.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.694+0000 7f04bed9d640 1 --2- 172.21.15.120:0/3011499788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb1e19b40 0x556eb1e353c0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f04b000baf0 tx=0x7f04b0005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e47ca781d66602 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:02.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.694+0000 7f04a2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:02.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.694+0000 7f04a2ff5640 1 -- 172.21.15.120:0/3011499788 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b0043020 con 0x556eb1e19b40 2024-09-19T09:24:02.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.694+0000 7f04c7609740 1 -- 172.21.15.120:0/3011499788 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04b0010d00 con 0x556eb1e19b40 2024-09-19T09:24:02.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.694+0000 7f04a2ff5640 1 -- 172.21.15.120:0/3011499788 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f04b0004220 con 0x556eb1e19b40 2024-09-19T09:24:02.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.694+0000 7f04a2ff5640 1 -- 172.21.15.120:0/3011499788 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f04b003b040 con 0x556eb1e19b40 2024-09-19T09:24:02.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.694+0000 7f04c7609740 10 client.22374 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:02.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.694+0000 7f04c7609740 1 -- 172.21.15.120:0/3011499788 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556eb1e9cbb0 con 0x556eb1e19b40 2024-09-19T09:24:02.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.695+0000 7f04a2ff5640 1 -- 172.21.15.120:0/3011499788 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04b0010d00 con 0x556eb1e19b40 2024-09-19T09:24:02.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.695+0000 7f04a2ff5640 1 -- 172.21.15.120:0/3011499788 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f04b0024070 con 0x556eb1e19b40 2024-09-19T09:24:02.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.695+0000 7f04c7609740 10 client.22374 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:02.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.695+0000 7f04c7609740 10 client.22374 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:24:02.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.695+0000 7f04c7609740 20 client.22374 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:02.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.695+0000 7f04c7609740 1 --2- 172.21.15.120:0/3011499788 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x556eb1e9d280 0x556eb1e9f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:02.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.695+0000 7f04c7609740 4 client.22374 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:24:02.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.695+0000 7f04c7609740 1 -- 172.21.15.120:0/3011499788 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556eb1d85d70 con 0x556eb1e9d280 2024-09-19T09:24:02.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.695+0000 7f04be59c640 1 --2- 172.21.15.120:0/3011499788 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x556eb1e9d280 0x556eb1e9f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:02.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.696+0000 7f04be59c640 1 --2- 172.21.15.120:0/3011499788 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x556eb1e9d280 0x556eb1e9f6a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1847160a64d41917 server_cookie=aa0ef5e0d027b005 in_seq=0 out_seq=0 2024-09-19T09:24:02.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.696+0000 7f04a2ff5640 10 client.22374 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:24:02.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.696+0000 7f04a2ff5640 1 -- 172.21.15.120:0/3011499788 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556eb1d85d70 con 0x556eb1e9d280 2024-09-19T09:24:02.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.696+0000 7f04a2ff5640 10 client.22374 handle_command_reply: tid=0 2024-09-19T09:24:02.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.696+0000 7f04c7609740 1 client.22374 shutdown 2024-09-19T09:24:02.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.696+0000 7f04c7609740 1 -- 172.21.15.120:0/3011499788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb1e19b40 msgr2=0x556eb1e353c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:02.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.696+0000 7f04c7609740 1 --2- 172.21.15.120:0/3011499788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb1e19b40 0x556eb1e353c0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f04b000baf0 tx=0x7f04b0005750 comp rx=0 tx=0).stop 2024-09-19T09:24:02.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.696+0000 7f04c7609740 1 -- 172.21.15.120:0/3011499788 shutdown_connections 2024-09-19T09:24:02.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.696+0000 7f04c7609740 1 --2- 172.21.15.120:0/3011499788 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x556eb1e9d280 0x556eb1e9f6a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.696+0000 7f04a2ff5640 0 client.22374 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:24:02.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.696+0000 7f04c7609740 1 --2- 172.21.15.120:0/3011499788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556eb1e23370 0x556eb1e35900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.696+0000 7f04c7609740 1 --2- 172.21.15.120:0/3011499788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb1e19b40 0x556eb1e353c0 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.696+0000 7f04c7609740 1 --2- 172.21.15.120:0/3011499788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556eb1e185d0 0x556eb1e34e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.696+0000 7f04c7609740 1 -- 172.21.15.120:0/3011499788 >> 172.21.15.120:0/3011499788 conn(0x556eb1de1310 msgr2=0x556eb1e1a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:02.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.697+0000 7f04c7609740 1 -- 172.21.15.120:0/3011499788 shutdown_connections 2024-09-19T09:24:02.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.697+0000 7f04c7609740 1 -- 172.21.15.120:0/3011499788 wait complete. 2024-09-19T09:24:02.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.697+0000 7f04c7609740 20 client.22374 trim_cache size 0 max 16384 2024-09-19T09:24:02.703 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:24:02.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.697+0000 7f04c5230640 1 -- 172.21.15.120:0/1205255117 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f04a80a3340 msgr2=0x7f04a80a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:02.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.697+0000 7f04c5230640 1 --2- 172.21.15.120:0/1205255117 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f04a80a3340 0x7f04a80a5800 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f04b8098e40 tx=0x7f04b8098040 comp rx=0 tx=0).stop 2024-09-19T09:24:02.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.698+0000 7f04c5230640 1 -- 172.21.15.120:0/1205255117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04c0108150 msgr2=0x7f04c010c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:02.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.698+0000 7f04c5230640 1 --2- 172.21.15.120:0/1205255117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04c0108150 0x7f04c010c8b0 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f04b800eb10 tx=0x7f04b800c2d0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.698+0000 7f04c5230640 1 -- 172.21.15.120:0/1205255117 shutdown_connections 2024-09-19T09:24:02.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.698+0000 7f04c5230640 1 --2- 172.21.15.120:0/1205255117 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f04a80a3340 0x7f04a80a5800 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.698+0000 7f04c5230640 1 --2- 172.21.15.120:0/1205255117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04c0108150 0x7f04c010c8b0 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.698+0000 7f04c5230640 1 --2- 172.21.15.120:0/1205255117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04c01012d0 0x7f04c010c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.698+0000 7f04c5230640 1 --2- 172.21.15.120:0/1205255117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04c0100920 0x7f04c010be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.698+0000 7f04c5230640 1 -- 172.21.15.120:0/1205255117 >> 172.21.15.120:0/1205255117 conn(0x7f04c0075f70 msgr2=0x7f04c0113d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.698+0000 7f04c5230640 1 -- 172.21.15.120:0/1205255117 shutdown_connections 2024-09-19T09:24:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:02.698+0000 7f04c5230640 1 -- 172.21.15.120:0/1205255117 wait complete. 2024-09-19T09:24:03.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:03.280+0000 7f335bfff640 1 -- 172.21.15.120:0/1636624631 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557fa9232ec0 con 0x557fa93cfe00 2024-09-19T09:24:03.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:03.280+0000 7f335bfff640 10 client.30973 handle_command_reply: tid=0 2024-09-19T09:24:03.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:03.280+0000 7f3384f50740 1 client.30973 shutdown 2024-09-19T09:24:03.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:03.280+0000 7f3384f50740 1 -- 172.21.15.120:0/1636624631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fa9354940 msgr2=0x557fa93ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:03.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:03.280+0000 7f3384f50740 1 --2- 172.21.15.120:0/1636624631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fa9354940 0x557fa93ce620 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f337409c0c0 tx=0x7f3374091930 comp rx=0 tx=0).stop 2024-09-19T09:24:03.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:03.280+0000 7f3384f50740 1 -- 172.21.15.120:0/1636624631 shutdown_connections 2024-09-19T09:24:03.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:03.280+0000 7f3384f50740 1 --2- 172.21.15.120:0/1636624631 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557fa93cfe00 0x557fa93d21f0 crc :-1 s=READY pgs=522 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:03.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:03.280+0000 7f3384f50740 1 --2- 172.21.15.120:0/1636624631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557fa9354940 0x557fa93ce620 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:03.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:03.280+0000 7f3384f50740 1 --2- 172.21.15.120:0/1636624631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557fa934a080 0x557fa93592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:03.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:03.280+0000 7f335bfff640 0 client.30973 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:03.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:03.280+0000 7f3384f50740 1 --2- 172.21.15.120:0/1636624631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557fa9349760 0x557fa9358db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:03.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:03.280+0000 7f3384f50740 1 -- 172.21.15.120:0/1636624631 >> 172.21.15.120:0/1636624631 conn(0x557fa9345cf0 msgr2=0x557fa934b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:03.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:03.280+0000 7f3384f50740 1 -- 172.21.15.120:0/1636624631 shutdown_connections 2024-09-19T09:24:03.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:03.280+0000 7f3384f50740 1 -- 172.21.15.120:0/1636624631 wait complete. 2024-09-19T09:24:03.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:03.280+0000 7f3384f50740 20 client.30973 trim_cache size 0 max 16384 2024-09-19T09:24:03.284 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:24:03.284 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 39, 2024-09-19T09:24:03.284 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:24:03.284 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 70, 2024-09-19T09:24:03.284 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:24:03.284 INFO:teuthology.orchestra.run.smithi120.stdout: "1094dda2": { 2024-09-19T09:24:03.284 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 70, 2024-09-19T09:24:03.284 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:24:03.284 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:24:03.284 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:24:03.284 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:24:03.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:24:03.285 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:24:03.285 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 144.5, 2024-09-19T09:24:03.285 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:24:03.285 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:24:03.285 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:24:03.285 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:24:03.285 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:24:03.285 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:24:03.285 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:24:03.285 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:24:03.285 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:24:03.285 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:24:03.285 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:24:03.285 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:24:03.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:03.281+0000 7f3382b77640 1 -- 172.21.15.120:0/3032894128 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f336c0a3340 msgr2=0x7f336c0a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:03.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:03.281+0000 7f3382b77640 1 --2- 172.21.15.120:0/3032894128 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f336c0a3340 0x7f336c0a5800 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f3370005d00 tx=0x7f3370005c70 comp rx=0 tx=0).stop 2024-09-19T09:24:03.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:03.281+0000 7f3382b77640 1 -- 172.21.15.120:0/3032894128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f337c10d520 msgr2=0x7f337c19e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:03.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:03.281+0000 7f3382b77640 1 --2- 172.21.15.120:0/3032894128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f337c10d520 0x7f337c19e880 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f337400eb10 tx=0x7f337400c2d0 comp rx=0 tx=0).stop 2024-09-19T09:24:03.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:03.281+0000 7f3382b77640 1 -- 172.21.15.120:0/3032894128 shutdown_connections 2024-09-19T09:24:03.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:03.281+0000 7f3382b77640 1 --2- 172.21.15.120:0/3032894128 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f336c0a3340 0x7f336c0a5800 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:03.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:03.281+0000 7f3382b77640 1 --2- 172.21.15.120:0/3032894128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f337c10d520 0x7f337c19e880 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:03.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:03.281+0000 7f3382b77640 1 --2- 172.21.15.120:0/3032894128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f337c108da0 0x7f337c1a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:03.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:03.281+0000 7f3382b77640 1 --2- 172.21.15.120:0/3032894128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f337c1083f0 0x7f337c1a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:03.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:03.281+0000 7f3382b77640 1 -- 172.21.15.120:0/3032894128 >> 172.21.15.120:0/3032894128 conn(0x7f337c077af0 msgr2=0x7f337c078dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:03.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:03.281+0000 7f3382b77640 1 -- 172.21.15.120:0/3032894128 shutdown_connections 2024-09-19T09:24:03.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:03.281+0000 7f3382b77640 1 -- 172.21.15.120:0/3032894128 wait complete. 2024-09-19T09:24:03.378 INFO:tasks.workunit.client.0.smithi120.stdout: 10 197298 21.34 MB/sec execute 358 sec latency 1251.052 ms 2024-09-19T09:24:03.725 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:24:04.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:04 smithi120 ceph-mon[31150]: pgmap v981: 97 pgs: 97 active+clean; 455 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.3 MiB/s wr, 751 op/s 2024-09-19T09:24:04.379 INFO:tasks.workunit.client.0.smithi120.stdout: 10 197298 21.28 MB/sec execute 359 sec latency 2251.121 ms 2024-09-19T09:24:04.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:04 smithi196 ceph-mon[35150]: pgmap v981: 97 pgs: 97 active+clean; 455 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.3 MiB/s wr, 751 op/s 2024-09-19T09:24:04.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:04 smithi153 ceph-mon[35071]: pgmap v981: 97 pgs: 97 active+clean; 455 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.3 MiB/s wr, 751 op/s 2024-09-19T09:24:04.515 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1094dda2', quiesce took 7.8 seconds. Will release after: 16.7 2024-09-19T09:24:04.621 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:05.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.137+0000 7fdb8ad0e640 1 -- 172.21.15.120:0/1020589619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb84101420 msgr2=0x7fdb8410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:05.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.137+0000 7fdb8ad0e640 1 --2- 172.21.15.120:0/1020589619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb84101420 0x7fdb8410d950 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c009cd0 tx=0x7fdb7c02a8e0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.137+0000 7fdb8ad0e640 1 -- 172.21.15.120:0/1020589619 shutdown_connections 2024-09-19T09:24:05.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.138+0000 7fdb8ad0e640 1 --2- 172.21.15.120:0/1020589619 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb84101420 0x7fdb8410d950 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.138+0000 7fdb8ad0e640 1 --2- 172.21.15.120:0/1020589619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb84100a80 0x7fdb84100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.138+0000 7fdb8ad0e640 1 --2- 172.21.15.120:0/1020589619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb84106a80 0x7fdb84106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.138+0000 7fdb8ad0e640 1 -- 172.21.15.120:0/1020589619 >> 172.21.15.120:0/1020589619 conn(0x7fdb840fc7a0 msgr2=0x7fdb840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:05.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.138+0000 7fdb8ad0e640 1 -- 172.21.15.120:0/1020589619 shutdown_connections 2024-09-19T09:24:05.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.138+0000 7fdb8ad0e640 1 -- 172.21.15.120:0/1020589619 wait complete. 2024-09-19T09:24:05.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.138+0000 7fdb8ad0e640 1 Processor -- start 2024-09-19T09:24:05.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.138+0000 7fdb8ad0e640 1 -- start start 2024-09-19T09:24:05.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.138+0000 7fdb8ad0e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb84100a80 0x7fdb84198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:05.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.138+0000 7fdb8ad0e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb84101420 0x7fdb84198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:05.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.138+0000 7fdb8ad0e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb84106a80 0x7fdb8419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:05.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.138+0000 7fdb8ad0e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdb84110ed0 con 0x7fdb84100a80 2024-09-19T09:24:05.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.138+0000 7fdb8ad0e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdb84110d50 con 0x7fdb84106a80 2024-09-19T09:24:05.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.138+0000 7fdb8ad0e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdb84111050 con 0x7fdb84101420 2024-09-19T09:24:05.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.138+0000 7fdb88aaa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb84100a80 0x7fdb84198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:05.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.138+0000 7fdb88aaa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb84100a80 0x7fdb84198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45326/0 (socket says 172.21.15.120:45326) 2024-09-19T09:24:05.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.138+0000 7fdb88aaa640 1 -- 172.21.15.120:0/295914416 learned_addr learned my addr 172.21.15.120:0/295914416 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:05.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.139+0000 7fdb892ab640 1 --2- 172.21.15.120:0/295914416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb84106a80 0x7fdb8419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:05.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.139+0000 7fdb83fff640 1 --2- 172.21.15.120:0/295914416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb84101420 0x7fdb84198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:05.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.139+0000 7fdb88aaa640 1 -- 172.21.15.120:0/295914416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb84101420 msgr2=0x7fdb84198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:05.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.139+0000 7fdb88aaa640 1 --2- 172.21.15.120:0/295914416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb84101420 0x7fdb84198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.139+0000 7fdb88aaa640 1 -- 172.21.15.120:0/295914416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb84106a80 msgr2=0x7fdb8419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:05.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.139+0000 7fdb88aaa640 1 --2- 172.21.15.120:0/295914416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb84106a80 0x7fdb8419ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.139+0000 7fdb88aaa640 1 -- 172.21.15.120:0/295914416 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb8419d3d0 con 0x7fdb84100a80 2024-09-19T09:24:05.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.139+0000 7fdb88aaa640 1 --2- 172.21.15.120:0/295914416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb84100a80 0x7fdb84198420 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c00e9f0 tx=0x7fdb6c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:05.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.139+0000 7fdb81ffb640 1 -- 172.21.15.120:0/295914416 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c00cd80 con 0x7fdb84100a80 2024-09-19T09:24:05.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.139+0000 7fdb8ad0e640 1 -- 172.21.15.120:0/295914416 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb8419d6c0 con 0x7fdb84100a80 2024-09-19T09:24:05.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.139+0000 7fdb81ffb640 1 -- 172.21.15.120:0/295914416 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb6c005500 con 0x7fdb84100a80 2024-09-19T09:24:05.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.139+0000 7fdb81ffb640 1 -- 172.21.15.120:0/295914416 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c010690 con 0x7fdb84100a80 2024-09-19T09:24:05.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.139+0000 7fdb8ad0e640 1 -- 172.21.15.120:0/295914416 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb841a4fa0 con 0x7fdb84100a80 2024-09-19T09:24:05.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.141+0000 7fdb81ffb640 1 -- 172.21.15.120:0/295914416 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fdb6c0046c0 con 0x7fdb84100a80 2024-09-19T09:24:05.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.142+0000 7fdb8d0e7740 1 Processor -- start 2024-09-19T09:24:05.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.142+0000 7fdb8d0e7740 1 -- start start 2024-09-19T09:24:05.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.142+0000 7fdb81ffb640 1 --2- 172.21.15.120:0/295914416 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdb700a3240 0x7fdb700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:05.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.142+0000 7fdb8d0e7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303e1d0b40 0x56303e1cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:05.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.142+0000 7fdb8d0e7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56303e1cf5d0 0x56303e1cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:05.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.142+0000 7fdb8d0e7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56303e1de780 0x56303e1e0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:05.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.142+0000 7fdb8d0e7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56303e1352f0 con 0x56303e1de780 2024-09-19T09:24:05.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.142+0000 7fdb8d0e7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56303e0dccf0 con 0x56303e1cf5d0 2024-09-19T09:24:05.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.142+0000 7fdb8d0e7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56303e1312b0 con 0x56303e1d0b40 2024-09-19T09:24:05.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.142+0000 7fdb892ab640 1 --2- 172.21.15.120:0/295914416 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdb700a3240 0x7fdb700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:05.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.142+0000 7fdb892ab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56303e1de780 0x56303e1e0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:05.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.142+0000 7fdb83fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56303e1cf5d0 0x56303e1cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:05.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.142+0000 7fdb892ab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56303e1de780 0x56303e1e0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45340/0 (socket says 172.21.15.120:45340) 2024-09-19T09:24:05.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.142+0000 7fdb892ab640 1 -- 172.21.15.120:0/1315104602 learned_addr learned my addr 172.21.15.120:0/1315104602 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:05.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.142+0000 7fdb81ffb640 1 -- 172.21.15.120:0/295914416 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fdb6c0cfcc0 con 0x7fdb84100a80 2024-09-19T09:24:05.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.142+0000 7fdb88aaa640 1 --2- 172.21.15.120:0/1315104602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303e1d0b40 0x56303e1cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:05.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.143+0000 7fdb83fff640 1 -- 172.21.15.120:0/1315104602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303e1d0b40 msgr2=0x56303e1cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:05.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.143+0000 7fdb83fff640 1 --2- 172.21.15.120:0/1315104602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303e1d0b40 0x56303e1cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.143+0000 7fdb83fff640 1 -- 172.21.15.120:0/1315104602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56303e1de780 msgr2=0x56303e1e0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:05.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.143+0000 7fdb83fff640 1 --2- 172.21.15.120:0/1315104602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56303e1de780 0x56303e1e0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.143+0000 7fdb83fff640 1 -- 172.21.15.120:0/1315104602 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56303e0c13a0 con 0x56303e1cf5d0 2024-09-19T09:24:05.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.143+0000 7fdb892ab640 1 --2- 172.21.15.120:0/1315104602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56303e1de780 0x56303e1e0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:24:05.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.143+0000 7fdb892ab640 1 --2- 172.21.15.120:0/295914416 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdb700a3240 0x7fdb700a5700 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c004ab0 tx=0x7fdb7c0318c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:05.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.143+0000 7fdb83fff640 1 --2- 172.21.15.120:0/1315104602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56303e1cf5d0 0x56303e1cfa10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdb740058c0 tx=0x7fdb7400f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b48824cb5a318bb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:05.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.143+0000 7fdb7affd640 1 -- 172.21.15.120:0/1315104602 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb74004020 con 0x56303e1cf5d0 2024-09-19T09:24:05.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.143+0000 7fdb7affd640 1 -- 172.21.15.120:0/1315104602 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb740041c0 con 0x56303e1cf5d0 2024-09-19T09:24:05.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.143+0000 7fdb7affd640 1 -- 172.21.15.120:0/1315104602 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb740186a0 con 0x56303e1cf5d0 2024-09-19T09:24:05.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.144+0000 7fdb8d0e7740 1 -- 172.21.15.120:0/1315104602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56303e1cf5d0 msgr2=0x56303e1cfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:05.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.144+0000 7fdb8d0e7740 1 --2- 172.21.15.120:0/1315104602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56303e1cf5d0 0x56303e1cfa10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdb740058c0 tx=0x7fdb7400f3c0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.144+0000 7fdb8d0e7740 1 -- 172.21.15.120:0/1315104602 shutdown_connections 2024-09-19T09:24:05.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.144+0000 7fdb8d0e7740 1 --2- 172.21.15.120:0/1315104602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56303e1de780 0x56303e1e0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.144+0000 7fdb8d0e7740 1 --2- 172.21.15.120:0/1315104602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56303e1cf5d0 0x56303e1cfa10 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.144+0000 7fdb8d0e7740 1 --2- 172.21.15.120:0/1315104602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303e1d0b40 0x56303e1cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.144+0000 7fdb8d0e7740 1 -- 172.21.15.120:0/1315104602 >> 172.21.15.120:0/1315104602 conn(0x56303e198310 msgr2=0x56303e1cbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:05.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.144+0000 7fdb8d0e7740 1 -- 172.21.15.120:0/1315104602 shutdown_connections 2024-09-19T09:24:05.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.144+0000 7fdb8d0e7740 1 -- 172.21.15.120:0/1315104602 wait complete. 2024-09-19T09:24:05.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.144+0000 7fdb8d0e7740 1 -- start start 2024-09-19T09:24:05.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.144+0000 7fdb8d0e7740 1 Processor -- start 2024-09-19T09:24:05.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.145+0000 7fdb8d0e7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56303e1cf5d0 0x56303e1de0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:05.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.145+0000 7fdb8d0e7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56303e1d0b40 0x56303e1db390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:05.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.145+0000 7fdb8d0e7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303e1de780 0x56303e1db8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:05.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.145+0000 7fdb8d0e7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56303e1e1480 con 0x56303e1cf5d0 2024-09-19T09:24:05.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.145+0000 7fdb8d0e7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56303e0f0060 con 0x56303e1d0b40 2024-09-19T09:24:05.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.145+0000 7fdb8d0e7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56303e1e1600 con 0x56303e1de780 2024-09-19T09:24:05.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.145+0000 7fdb8d0e7740 10 client.0 _ll_register_callbacks cb 0x56303de438c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:05.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.145+0000 7fdb88aaa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56303e1cf5d0 0x56303e1de0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:05.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.145+0000 7fdb88aaa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56303e1cf5d0 0x56303e1de0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45356/0 (socket says 172.21.15.120:45356) 2024-09-19T09:24:05.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.145+0000 7fdb88aaa640 1 -- 172.21.15.120:0/2699019715 learned_addr learned my addr 172.21.15.120:0/2699019715 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:05.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.145+0000 7fdb892ab640 1 --2- 172.21.15.120:0/2699019715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303e1de780 0x56303e1db8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:05.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.145+0000 7fdb83fff640 1 --2- 172.21.15.120:0/2699019715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56303e1d0b40 0x56303e1db390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:05.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.145+0000 7fdb88aaa640 1 -- 172.21.15.120:0/2699019715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303e1de780 msgr2=0x56303e1db8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:05.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.145+0000 7fdb88aaa640 1 --2- 172.21.15.120:0/2699019715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303e1de780 0x56303e1db8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.145+0000 7fdb88aaa640 1 -- 172.21.15.120:0/2699019715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56303e1d0b40 msgr2=0x56303e1db390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:05.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.145+0000 7fdb88aaa640 1 --2- 172.21.15.120:0/2699019715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56303e1d0b40 0x56303e1db390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.145+0000 7fdb88aaa640 1 -- 172.21.15.120:0/2699019715 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56303dea63c0 con 0x56303e1cf5d0 2024-09-19T09:24:05.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.145+0000 7fdb88aaa640 1 --2- 172.21.15.120:0/2699019715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56303e1cf5d0 0x56303e1de0e0 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c00cfd0 tx=0x7fdb6c0a1e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb222e57d5c569c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:05.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.145+0000 7fdb63fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:05.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.145+0000 7fdb63fff640 1 -- 172.21.15.120:0/2699019715 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c09c2e0 con 0x56303e1cf5d0 2024-09-19T09:24:05.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.145+0000 7fdb63fff640 1 -- 172.21.15.120:0/2699019715 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fdb6c0b5190 con 0x56303e1cf5d0 2024-09-19T09:24:05.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.145+0000 7fdb8d0e7740 1 -- 172.21.15.120:0/2699019715 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb740041c0 con 0x56303e1cf5d0 2024-09-19T09:24:05.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.146+0000 7fdb63fff640 1 -- 172.21.15.120:0/2699019715 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdb6c0a0690 con 0x56303e1cf5d0 2024-09-19T09:24:05.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.146+0000 7fdb8d0e7740 10 client.22386 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:05.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.146+0000 7fdb8d0e7740 1 -- 172.21.15.120:0/2699019715 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56303e1ecec0 con 0x56303e1cf5d0 2024-09-19T09:24:05.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.146+0000 7fdb63fff640 1 -- 172.21.15.120:0/2699019715 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb740041c0 con 0x56303e1cf5d0 2024-09-19T09:24:05.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.146+0000 7fdb63fff640 1 -- 172.21.15.120:0/2699019715 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fdb6c0a4ac0 con 0x56303e1cf5d0 2024-09-19T09:24:05.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.146+0000 7fdb8d0e7740 10 client.22386 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:05.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.146+0000 7fdb8d0e7740 10 client.22386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:24:05.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.146+0000 7fdb8d0e7740 20 client.22386 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:05.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.146+0000 7fdb8d0e7740 1 --2- 172.21.15.120:0/2699019715 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56303e1ee320 0x56303e1f0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:05.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.146+0000 7fdb8d0e7740 4 client.22386 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:24:05.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.146+0000 7fdb8d0e7740 1 -- 172.21.15.120:0/2699019715 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56303e142860 con 0x56303e1ee320 2024-09-19T09:24:05.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.146+0000 7fdb83fff640 1 --2- 172.21.15.120:0/2699019715 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56303e1ee320 0x56303e1f0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:05.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.147+0000 7fdb63fff640 10 client.22386 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:05.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.147+0000 7fdb83fff640 1 --2- 172.21.15.120:0/2699019715 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56303e1ee320 0x56303e1f0770 crc :-1 s=READY pgs=523 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8e047e895a37d1f server_cookie=af45c1595c1d7159 in_seq=0 out_seq=0 2024-09-19T09:24:05.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.147+0000 7fdb63fff640 1 -- 172.21.15.120:0/2699019715 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56303e142860 con 0x56303e1ee320 2024-09-19T09:24:05.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.147+0000 7fdb63fff640 10 client.22386 handle_command_reply: tid=0 2024-09-19T09:24:05.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.147+0000 7fdb8d0e7740 1 client.22386 shutdown 2024-09-19T09:24:05.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.147+0000 7fdb8d0e7740 1 -- 172.21.15.120:0/2699019715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56303e1cf5d0 msgr2=0x56303e1de0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:05.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.147+0000 7fdb8d0e7740 1 --2- 172.21.15.120:0/2699019715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56303e1cf5d0 0x56303e1de0e0 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c00cfd0 tx=0x7fdb6c0a1e20 comp rx=0 tx=0).stop 2024-09-19T09:24:05.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.148+0000 7fdb8d0e7740 1 -- 172.21.15.120:0/2699019715 shutdown_connections 2024-09-19T09:24:05.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.148+0000 7fdb8d0e7740 1 --2- 172.21.15.120:0/2699019715 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56303e1ee320 0x56303e1f0770 crc :-1 s=READY pgs=523 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.148+0000 7fdb8d0e7740 1 --2- 172.21.15.120:0/2699019715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303e1de780 0x56303e1db8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.148+0000 7fdb8d0e7740 1 --2- 172.21.15.120:0/2699019715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56303e1d0b40 0x56303e1db390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.148+0000 7fdb63fff640 0 client.22386 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:05.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.148+0000 7fdb8d0e7740 1 --2- 172.21.15.120:0/2699019715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56303e1cf5d0 0x56303e1de0e0 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.148+0000 7fdb8d0e7740 1 -- 172.21.15.120:0/2699019715 >> 172.21.15.120:0/2699019715 conn(0x56303e198310 msgr2=0x56303e1d1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:05.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.148+0000 7fdb8d0e7740 1 -- 172.21.15.120:0/2699019715 shutdown_connections 2024-09-19T09:24:05.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.148+0000 7fdb8d0e7740 1 -- 172.21.15.120:0/2699019715 wait complete. 2024-09-19T09:24:05.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.148+0000 7fdb8d0e7740 20 client.22386 trim_cache size 0 max 16384 2024-09-19T09:24:05.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.160+0000 7fdb8d0e7740 1 Processor -- start 2024-09-19T09:24:05.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.160+0000 7fdb8d0e7740 1 -- start start 2024-09-19T09:24:05.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.161+0000 7fdb8d0e7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56303e1cf5d0 0x56303e1ea750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:05.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.161+0000 7fdb8d0e7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56303e1d0b40 0x56303e1dac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:05.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.161+0000 7fdb8d0e7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303e1de780 0x56303e1dfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:05.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.161+0000 7fdb8d0e7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdb841ab350 con 0x56303e1d0b40 2024-09-19T09:24:05.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.161+0000 7fdb8d0e7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdb841ace90 con 0x56303e1cf5d0 2024-09-19T09:24:05.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.161+0000 7fdb8d0e7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdb841aba20 con 0x56303e1de780 2024-09-19T09:24:05.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.161+0000 7fdb83fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56303e1d0b40 0x56303e1dac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:05.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.161+0000 7fdb83fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56303e1d0b40 0x56303e1dac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45364/0 (socket says 172.21.15.120:45364) 2024-09-19T09:24:05.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.161+0000 7fdb83fff640 1 -- 172.21.15.120:0/3832930277 learned_addr learned my addr 172.21.15.120:0/3832930277 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:05.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.161+0000 7fdb892ab640 1 --2- 172.21.15.120:0/3832930277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303e1de780 0x56303e1dfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:05.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.161+0000 7fdb88aaa640 1 --2- 172.21.15.120:0/3832930277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56303e1cf5d0 0x56303e1ea750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:05.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.161+0000 7fdb83fff640 1 -- 172.21.15.120:0/3832930277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303e1de780 msgr2=0x56303e1dfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:05.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.161+0000 7fdb83fff640 1 --2- 172.21.15.120:0/3832930277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303e1de780 0x56303e1dfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.161+0000 7fdb83fff640 1 -- 172.21.15.120:0/3832930277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56303e1cf5d0 msgr2=0x56303e1ea750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:05.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.161+0000 7fdb83fff640 1 --2- 172.21.15.120:0/3832930277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56303e1cf5d0 0x56303e1ea750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.161+0000 7fdb83fff640 1 -- 172.21.15.120:0/3832930277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56303e1cccf0 con 0x56303e1d0b40 2024-09-19T09:24:05.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.161+0000 7fdb88aaa640 1 --2- 172.21.15.120:0/3832930277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56303e1cf5d0 0x56303e1ea750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:24:05.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.161+0000 7fdb83fff640 1 --2- 172.21.15.120:0/3832930277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56303e1d0b40 0x56303e1dac50 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7fdb74003dd0 tx=0x7fdb74003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1efcb0a84371aba5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:05.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.161+0000 7fdb7a7fc640 1 -- 172.21.15.120:0/3832930277 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb74004470 con 0x56303e1d0b40 2024-09-19T09:24:05.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.161+0000 7fdb7a7fc640 1 -- 172.21.15.120:0/3832930277 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb74030070 con 0x56303e1d0b40 2024-09-19T09:24:05.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.161+0000 7fdb7a7fc640 1 -- 172.21.15.120:0/3832930277 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7400fd70 con 0x56303e1d0b40 2024-09-19T09:24:05.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.162+0000 7fdb8d0e7740 1 -- 172.21.15.120:0/3832930277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56303e1d0b40 msgr2=0x56303e1dac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:05.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.162+0000 7fdb8d0e7740 1 --2- 172.21.15.120:0/3832930277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56303e1d0b40 0x56303e1dac50 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7fdb74003dd0 tx=0x7fdb74003d60 comp rx=0 tx=0).stop 2024-09-19T09:24:05.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.162+0000 7fdb8d0e7740 1 -- 172.21.15.120:0/3832930277 shutdown_connections 2024-09-19T09:24:05.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.162+0000 7fdb8d0e7740 1 --2- 172.21.15.120:0/3832930277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303e1de780 0x56303e1dfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.162+0000 7fdb8d0e7740 1 --2- 172.21.15.120:0/3832930277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56303e1d0b40 0x56303e1dac50 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.162+0000 7fdb8d0e7740 1 --2- 172.21.15.120:0/3832930277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56303e1cf5d0 0x56303e1ea750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.162+0000 7fdb8d0e7740 1 -- 172.21.15.120:0/3832930277 >> 172.21.15.120:0/3832930277 conn(0x56303e198310 msgr2=0x56303e1d9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:05.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.162+0000 7fdb8d0e7740 1 -- 172.21.15.120:0/3832930277 shutdown_connections 2024-09-19T09:24:05.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.162+0000 7fdb8d0e7740 1 -- 172.21.15.120:0/3832930277 wait complete. 2024-09-19T09:24:05.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.162+0000 7fdb8d0e7740 1 -- start start 2024-09-19T09:24:05.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.162+0000 7fdb8d0e7740 1 Processor -- start 2024-09-19T09:24:05.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.163+0000 7fdb8d0e7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303e1cf5d0 0x56303e1e71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:05.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.163+0000 7fdb8d0e7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56303e1d0b40 0x56303e1f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:05.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.163+0000 7fdb8d0e7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56303e1de780 0x56303e1f6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:05.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.163+0000 7fdb8d0e7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdb841ac7c0 con 0x56303e1de780 2024-09-19T09:24:05.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.163+0000 7fdb8d0e7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdb841ac0f0 con 0x56303e1d0b40 2024-09-19T09:24:05.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.163+0000 7fdb8d0e7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdb841a9310 con 0x56303e1cf5d0 2024-09-19T09:24:05.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.163+0000 7fdb8d0e7740 10 client.0 _ll_register_callbacks cb 0x7fdb6c0985c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:05.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.163+0000 7fdb892ab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56303e1de780 0x56303e1f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:05.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.163+0000 7fdb88aaa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303e1cf5d0 0x56303e1e71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:05.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.163+0000 7fdb892ab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56303e1de780 0x56303e1f6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45368/0 (socket says 172.21.15.120:45368) 2024-09-19T09:24:05.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.163+0000 7fdb892ab640 1 -- 172.21.15.120:0/3466995742 learned_addr learned my addr 172.21.15.120:0/3466995742 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:05.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.163+0000 7fdb83fff640 1 --2- 172.21.15.120:0/3466995742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56303e1d0b40 0x56303e1f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:05.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.163+0000 7fdb892ab640 1 -- 172.21.15.120:0/3466995742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303e1cf5d0 msgr2=0x56303e1e71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:05.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.163+0000 7fdb892ab640 1 --2- 172.21.15.120:0/3466995742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303e1cf5d0 0x56303e1e71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.163+0000 7fdb892ab640 1 -- 172.21.15.120:0/3466995742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56303e1d0b40 msgr2=0x56303e1f5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:05.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.163+0000 7fdb892ab640 1 --2- 172.21.15.120:0/3466995742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56303e1d0b40 0x56303e1f5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.163+0000 7fdb892ab640 1 -- 172.21.15.120:0/3466995742 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56303e1e4cc0 con 0x56303e1de780 2024-09-19T09:24:05.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.163+0000 7fdb88aaa640 1 --2- 172.21.15.120:0/3466995742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303e1cf5d0 0x56303e1e71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:24:05.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.163+0000 7fdb892ab640 1 --2- 172.21.15.120:0/3466995742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56303e1de780 0x56303e1f6290 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c02aed0 tx=0x7fdb7c032fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab359de5ed52272 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:05.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.163+0000 7fdb63fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:05.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.164+0000 7fdb63fff640 1 -- 172.21.15.120:0/3466995742 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c059060 con 0x56303e1de780 2024-09-19T09:24:05.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.164+0000 7fdb63fff640 1 -- 172.21.15.120:0/3466995742 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fdb7c045070 con 0x56303e1de780 2024-09-19T09:24:05.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.164+0000 7fdb8d0e7740 1 -- 172.21.15.120:0/3466995742 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb74030070 con 0x56303e1de780 2024-09-19T09:24:05.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.164+0000 7fdb63fff640 1 -- 172.21.15.120:0/3466995742 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdb7c0305a0 con 0x56303e1de780 2024-09-19T09:24:05.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.164+0000 7fdb8d0e7740 10 client.22392 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:05.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.164+0000 7fdb8d0e7740 1 -- 172.21.15.120:0/3466995742 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56303e259160 con 0x56303e1de780 2024-09-19T09:24:05.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.164+0000 7fdb63fff640 1 -- 172.21.15.120:0/3466995742 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb74030070 con 0x56303e1de780 2024-09-19T09:24:05.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.164+0000 7fdb63fff640 1 -- 172.21.15.120:0/3466995742 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fdb7c053070 con 0x56303e1de780 2024-09-19T09:24:05.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.164+0000 7fdb8d0e7740 10 client.22392 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:05.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.164+0000 7fdb8d0e7740 10 client.22392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:24:05.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.164+0000 7fdb8d0e7740 20 client.22392 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:05.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.164+0000 7fdb8d0e7740 1 --2- 172.21.15.120:0/3466995742 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56303e259860 0x56303e25bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:05.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.164+0000 7fdb8d0e7740 4 client.22392 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:24:05.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.164+0000 7fdb8d0e7740 1 -- 172.21.15.120:0/3466995742 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56303e13cd70 con 0x56303e259860 2024-09-19T09:24:05.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.164+0000 7fdb88aaa640 1 --2- 172.21.15.120:0/3466995742 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56303e259860 0x56303e25bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:05.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.165+0000 7fdb88aaa640 1 --2- 172.21.15.120:0/3466995742 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56303e259860 0x56303e25bc80 crc :-1 s=READY pgs=524 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29e83da374ad2a0c server_cookie=be520dac80ba09c6 in_seq=0 out_seq=0 2024-09-19T09:24:05.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.165+0000 7fdb63fff640 10 client.22392 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:05.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.166+0000 7fdb63fff640 1 -- 172.21.15.120:0/3466995742 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56303e13cd70 con 0x56303e259860 2024-09-19T09:24:05.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.166+0000 7fdb63fff640 10 client.22392 handle_command_reply: tid=0 2024-09-19T09:24:05.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.166+0000 7fdb8d0e7740 1 client.22392 shutdown 2024-09-19T09:24:05.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.166+0000 7fdb8d0e7740 1 -- 172.21.15.120:0/3466995742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56303e1de780 msgr2=0x56303e1f6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:05.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.166+0000 7fdb8d0e7740 1 --2- 172.21.15.120:0/3466995742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56303e1de780 0x56303e1f6290 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c02aed0 tx=0x7fdb7c032fd0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.166+0000 7fdb8d0e7740 1 -- 172.21.15.120:0/3466995742 shutdown_connections 2024-09-19T09:24:05.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.166+0000 7fdb8d0e7740 1 --2- 172.21.15.120:0/3466995742 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56303e259860 0x56303e25bc80 crc :-1 s=READY pgs=524 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.166+0000 7fdb8d0e7740 1 --2- 172.21.15.120:0/3466995742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56303e1de780 0x56303e1f6290 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.166+0000 7fdb8d0e7740 1 --2- 172.21.15.120:0/3466995742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56303e1d0b40 0x56303e1f5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.166+0000 7fdb63fff640 0 client.22392 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:05.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.166+0000 7fdb8d0e7740 1 --2- 172.21.15.120:0/3466995742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303e1cf5d0 0x56303e1e71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.166+0000 7fdb8d0e7740 1 -- 172.21.15.120:0/3466995742 >> 172.21.15.120:0/3466995742 conn(0x56303e198310 msgr2=0x56303e1ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:05.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.166+0000 7fdb8d0e7740 1 -- 172.21.15.120:0/3466995742 shutdown_connections 2024-09-19T09:24:05.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.166+0000 7fdb8d0e7740 1 -- 172.21.15.120:0/3466995742 wait complete. 2024-09-19T09:24:05.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.166+0000 7fdb8d0e7740 20 client.22392 trim_cache size 0 max 16384 2024-09-19T09:24:05.184 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:24:05.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.167+0000 7fdb8ad0e640 1 -- 172.21.15.120:0/295914416 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdb700a3240 msgr2=0x7fdb700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:05.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.167+0000 7fdb8ad0e640 1 --2- 172.21.15.120:0/295914416 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdb700a3240 0x7fdb700a5700 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c004ab0 tx=0x7fdb7c0318c0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.167+0000 7fdb8ad0e640 1 -- 172.21.15.120:0/295914416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb84100a80 msgr2=0x7fdb84198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:05.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.167+0000 7fdb8ad0e640 1 --2- 172.21.15.120:0/295914416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb84100a80 0x7fdb84198420 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c00e9f0 tx=0x7fdb6c00edf0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.167+0000 7fdb8ad0e640 1 -- 172.21.15.120:0/295914416 shutdown_connections 2024-09-19T09:24:05.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.167+0000 7fdb8ad0e640 1 --2- 172.21.15.120:0/295914416 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fdb700a3240 0x7fdb700a5700 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.167+0000 7fdb8ad0e640 1 --2- 172.21.15.120:0/295914416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb84106a80 0x7fdb8419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.167+0000 7fdb8ad0e640 1 --2- 172.21.15.120:0/295914416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb84101420 0x7fdb84198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.167+0000 7fdb8ad0e640 1 --2- 172.21.15.120:0/295914416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb84100a80 0x7fdb84198420 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:05.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.167+0000 7fdb8ad0e640 1 -- 172.21.15.120:0/295914416 >> 172.21.15.120:0/295914416 conn(0x7fdb840fc7a0 msgr2=0x7fdb8406a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:05.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.167+0000 7fdb8ad0e640 1 -- 172.21.15.120:0/295914416 shutdown_connections 2024-09-19T09:24:05.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:05.167+0000 7fdb8ad0e640 1 -- 172.21.15.120:0/295914416 wait complete. 2024-09-19T09:24:05.379 INFO:tasks.workunit.client.0.smithi120.stdout: 10 197298 21.22 MB/sec execute 360 sec latency 3251.189 ms 2024-09-19T09:24:06.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:06 smithi120 ceph-mon[31150]: pgmap v982: 97 pgs: 97 active+clean; 455 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.4 MiB/s wr, 713 op/s 2024-09-19T09:24:06.379 INFO:tasks.workunit.client.0.smithi120.stdout: 10 197298 21.17 MB/sec execute 361 sec latency 4251.284 ms 2024-09-19T09:24:06.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:06 smithi196 ceph-mon[35150]: pgmap v982: 97 pgs: 97 active+clean; 455 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.4 MiB/s wr, 713 op/s 2024-09-19T09:24:06.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:06 smithi153 ceph-mon[35071]: pgmap v982: 97 pgs: 97 active+clean; 455 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.4 MiB/s wr, 713 op/s 2024-09-19T09:24:06.987 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:24:06.987 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:24:07.234 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:07.379 INFO:tasks.workunit.client.0.smithi120.stdout: 10 197298 21.11 MB/sec execute 362 sec latency 5251.376 ms 2024-09-19T09:24:07.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.716+0000 7f79b8f67640 1 -- 172.21.15.120:0/1626778402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79b41002f0 msgr2=0x7f79b41006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:07.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.716+0000 7f79b8f67640 1 --2- 172.21.15.120:0/1626778402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79b41002f0 0x7f79b41006d0 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f79a4009930 tx=0x7f79a402a420 comp rx=0 tx=0).stop 2024-09-19T09:24:07.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.716+0000 7f79b8f67640 1 -- 172.21.15.120:0/1626778402 shutdown_connections 2024-09-19T09:24:07.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.716+0000 7f79b8f67640 1 --2- 172.21.15.120:0/1626778402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79b4106f50 0x7f79b410eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.716+0000 7f79b8f67640 1 --2- 172.21.15.120:0/1626778402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79b4100ca0 0x7f79b4106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.716+0000 7f79b8f67640 1 --2- 172.21.15.120:0/1626778402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79b41002f0 0x7f79b41006d0 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.716+0000 7f79b8f67640 1 -- 172.21.15.120:0/1626778402 >> 172.21.15.120:0/1626778402 conn(0x7f79b4077210 msgr2=0x7f79b40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:07.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.716+0000 7f79b8f67640 1 -- 172.21.15.120:0/1626778402 shutdown_connections 2024-09-19T09:24:07.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.716+0000 7f79b8f67640 1 -- 172.21.15.120:0/1626778402 wait complete. 2024-09-19T09:24:07.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.717+0000 7f79b8f67640 1 Processor -- start 2024-09-19T09:24:07.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.717+0000 7f79b8f67640 1 -- start start 2024-09-19T09:24:07.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.717+0000 7f79b8f67640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79b41002f0 0x7f79b419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:07.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.717+0000 7f79b8f67640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79b4100ca0 0x7f79b419ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:07.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.717+0000 7f79b8f67640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79b4106f50 0x7f79b419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:07.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.717+0000 7f79b8f67640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f79b4111fa0 con 0x7f79b4100ca0 2024-09-19T09:24:07.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.717+0000 7f79b8f67640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f79b4111e20 con 0x7f79b4106f50 2024-09-19T09:24:07.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.717+0000 7f79b8f67640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f79b4112120 con 0x7f79b41002f0 2024-09-19T09:24:07.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.717+0000 7f79b1d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79b4100ca0 0x7f79b419ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:07.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.717+0000 7f79b1d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79b4100ca0 0x7f79b419ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45384/0 (socket says 172.21.15.120:45384) 2024-09-19T09:24:07.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.717+0000 7f79b1d9b640 1 -- 172.21.15.120:0/2976310546 learned_addr learned my addr 172.21.15.120:0/2976310546 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:07.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.717+0000 7f79b259c640 1 --2- 172.21.15.120:0/2976310546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79b41002f0 0x7f79b419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:07.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.717+0000 7f79b2d9d640 1 --2- 172.21.15.120:0/2976310546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79b4106f50 0x7f79b419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:07.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.717+0000 7f79b1d9b640 1 -- 172.21.15.120:0/2976310546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79b41002f0 msgr2=0x7f79b419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:07.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.717+0000 7f79b1d9b640 1 --2- 172.21.15.120:0/2976310546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79b41002f0 0x7f79b419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.717+0000 7f79b1d9b640 1 -- 172.21.15.120:0/2976310546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79b4106f50 msgr2=0x7f79b419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:07.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.717+0000 7f79b1d9b640 1 --2- 172.21.15.120:0/2976310546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79b4106f50 0x7f79b419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.717+0000 7f79b1d9b640 1 -- 172.21.15.120:0/2976310546 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b419f6b0 con 0x7f79b4100ca0 2024-09-19T09:24:07.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.718+0000 7f79b1d9b640 1 --2- 172.21.15.120:0/2976310546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79b4100ca0 0x7f79b419ac40 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f799c00d770 tx=0x7f799c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:07.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.718+0000 7f79ab7fe640 1 -- 172.21.15.120:0/2976310546 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c017070 con 0x7f79b4100ca0 2024-09-19T09:24:07.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.718+0000 7f79b8f67640 1 -- 172.21.15.120:0/2976310546 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b419f9a0 con 0x7f79b4100ca0 2024-09-19T09:24:07.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.718+0000 7f79ab7fe640 1 -- 172.21.15.120:0/2976310546 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f799c009c30 con 0x7f79b4100ca0 2024-09-19T09:24:07.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.718+0000 7f79ab7fe640 1 -- 172.21.15.120:0/2976310546 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c00f620 con 0x7f79b4100ca0 2024-09-19T09:24:07.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.718+0000 7f79b8f67640 1 -- 172.21.15.120:0/2976310546 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b41a7280 con 0x7f79b4100ca0 2024-09-19T09:24:07.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.721+0000 7f79bb340740 1 Processor -- start 2024-09-19T09:24:07.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.721+0000 7f79bb340740 1 -- start start 2024-09-19T09:24:07.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.721+0000 7f79ab7fe640 1 -- 172.21.15.120:0/2976310546 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f799c01a070 con 0x7f79b4100ca0 2024-09-19T09:24:07.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.721+0000 7f79bb340740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650501f6b60 0x5650501f6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:07.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.721+0000 7f79bb340740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650501f7480 0x565050201800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:07.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.721+0000 7f79bb340740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565050201d40 0x565050204130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:07.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.721+0000 7f79bb340740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56505015c280 con 0x565050201d40 2024-09-19T09:24:07.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.721+0000 7f79bb340740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x565050103b10 con 0x5650501f6b60 2024-09-19T09:24:07.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.721+0000 7f79bb340740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x565050158240 con 0x5650501f7480 2024-09-19T09:24:07.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.721+0000 7f79b1d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565050201d40 0x565050204130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:07.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.721+0000 7f79b1d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565050201d40 0x565050204130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45390/0 (socket says 172.21.15.120:45390) 2024-09-19T09:24:07.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.721+0000 7f79b1d9b640 1 -- 172.21.15.120:0/2554117090 learned_addr learned my addr 172.21.15.120:0/2554117090 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:07.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.722+0000 7f79b259c640 1 --2- 172.21.15.120:0/2554117090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650501f7480 0x565050201800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:07.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.722+0000 7f79b2d9d640 1 --2- 172.21.15.120:0/2554117090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650501f6b60 0x5650501f6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:07.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.722+0000 7f79b1d9b640 1 -- 172.21.15.120:0/2554117090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650501f7480 msgr2=0x565050201800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:07.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.722+0000 7f79b1d9b640 1 --2- 172.21.15.120:0/2554117090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650501f7480 0x565050201800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.722+0000 7f79b1d9b640 1 -- 172.21.15.120:0/2554117090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650501f6b60 msgr2=0x5650501f6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:07.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.722+0000 7f79b1d9b640 1 --2- 172.21.15.120:0/2554117090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650501f6b60 0x5650501f6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.722+0000 7f79b1d9b640 1 -- 172.21.15.120:0/2554117090 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650501101f0 con 0x565050201d40 2024-09-19T09:24:07.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.722+0000 7f79b1d9b640 1 --2- 172.21.15.120:0/2554117090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565050201d40 0x565050204130 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f799c0d7560 tx=0x7f799c0d7960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5f846ca06288d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:07.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.722+0000 7f79ab7fe640 1 --2- 172.21.15.120:0/2976310546 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f79a00a3240 0x7f79a00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:07.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.722+0000 7f79a8ff9640 1 -- 172.21.15.120:0/2554117090 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c0e3070 con 0x565050201d40 2024-09-19T09:24:07.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.722+0000 7f79a8ff9640 1 -- 172.21.15.120:0/2554117090 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f799c09c140 con 0x565050201d40 2024-09-19T09:24:07.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.722+0000 7f79a8ff9640 1 -- 172.21.15.120:0/2554117090 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c0c0680 con 0x565050201d40 2024-09-19T09:24:07.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.722+0000 7f79b2d9d640 1 --2- 172.21.15.120:0/2976310546 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f79a00a3240 0x7f79a00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:07.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.722+0000 7f79ab7fe640 1 -- 172.21.15.120:0/2976310546 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f799c0cf420 con 0x7f79b4100ca0 2024-09-19T09:24:07.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.723+0000 7f79bb340740 1 -- 172.21.15.120:0/2554117090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565050201d40 msgr2=0x565050204130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:07.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.723+0000 7f79b2d9d640 1 --2- 172.21.15.120:0/2976310546 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f79a00a3240 0x7f79a00a5700 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f79ac007e50 tx=0x7f79ac007600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:07.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.723+0000 7f79bb340740 1 --2- 172.21.15.120:0/2554117090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565050201d40 0x565050204130 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f799c0d7560 tx=0x7f799c0d7960 comp rx=0 tx=0).stop 2024-09-19T09:24:07.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.723+0000 7f79bb340740 1 -- 172.21.15.120:0/2554117090 shutdown_connections 2024-09-19T09:24:07.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.723+0000 7f79bb340740 1 --2- 172.21.15.120:0/2554117090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565050201d40 0x565050204130 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.723+0000 7f79bb340740 1 --2- 172.21.15.120:0/2554117090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650501f7480 0x565050201800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.723+0000 7f79bb340740 1 --2- 172.21.15.120:0/2554117090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650501f6b60 0x5650501f6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.723+0000 7f79bb340740 1 -- 172.21.15.120:0/2554117090 >> 172.21.15.120:0/2554117090 conn(0x5650501f30f0 msgr2=0x5650501f34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:07.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.723+0000 7f79bb340740 1 -- 172.21.15.120:0/2554117090 shutdown_connections 2024-09-19T09:24:07.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.723+0000 7f79bb340740 1 -- 172.21.15.120:0/2554117090 wait complete. 2024-09-19T09:24:07.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.723+0000 7f79bb340740 1 -- start start 2024-09-19T09:24:07.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.723+0000 7f79bb340740 1 Processor -- start 2024-09-19T09:24:07.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.724+0000 7f79bb340740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650501f6b60 0x5650501feba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:07.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.724+0000 7f79bb340740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650501f7480 0x5650501ff0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:07.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.724+0000 7f79bb340740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565050201d40 0x5650501ff620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:07.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.724+0000 7f79bb340740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x565050204910 con 0x565050201d40 2024-09-19T09:24:07.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.724+0000 7f79bb340740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x565050116f10 con 0x5650501f6b60 2024-09-19T09:24:07.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.724+0000 7f79bb340740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x565050204a90 con 0x5650501f7480 2024-09-19T09:24:07.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.724+0000 7f79bb340740 10 client.0 _ll_register_callbacks cb 0x5650500dfa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:07.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.724+0000 7f79b2d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565050201d40 0x5650501ff620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:07.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.724+0000 7f79b2d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565050201d40 0x5650501ff620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45406/0 (socket says 172.21.15.120:45406) 2024-09-19T09:24:07.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.724+0000 7f79b2d9d640 1 -- 172.21.15.120:0/814349093 learned_addr learned my addr 172.21.15.120:0/814349093 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:07.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.724+0000 7f79b1d9b640 1 --2- 172.21.15.120:0/814349093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650501f7480 0x5650501ff0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:07.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.724+0000 7f79b259c640 1 --2- 172.21.15.120:0/814349093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650501f6b60 0x5650501feba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:07.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.724+0000 7f79b2d9d640 1 -- 172.21.15.120:0/814349093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650501f7480 msgr2=0x5650501ff0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:07.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.724+0000 7f79b2d9d640 1 --2- 172.21.15.120:0/814349093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650501f7480 0x5650501ff0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.724+0000 7f79b2d9d640 1 -- 172.21.15.120:0/814349093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650501f6b60 msgr2=0x5650501feba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:07.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.724+0000 7f79b2d9d640 1 --2- 172.21.15.120:0/814349093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650501f6b60 0x5650501feba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.724+0000 7f79b2d9d640 1 -- 172.21.15.120:0/814349093 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56504fecd6a0 con 0x565050201d40 2024-09-19T09:24:07.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.724+0000 7f79b2d9d640 1 --2- 172.21.15.120:0/814349093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565050201d40 0x5650501ff620 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f79ac00ed80 tx=0x7f79ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b79bdea5d60952e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:07.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.724+0000 7f7999ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:07.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.724+0000 7f7999ffb640 1 -- 172.21.15.120:0/814349093 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac01cd30 con 0x565050201d40 2024-09-19T09:24:07.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.724+0000 7f7999ffb640 1 -- 172.21.15.120:0/814349093 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f79ac027430 con 0x565050201d40 2024-09-19T09:24:07.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.725+0000 7f79bb340740 1 -- 172.21.15.120:0/814349093 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f799c09c140 con 0x565050201d40 2024-09-19T09:24:07.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.725+0000 7f7999ffb640 1 -- 172.21.15.120:0/814349093 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f79ac007aa0 con 0x565050201d40 2024-09-19T09:24:07.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.725+0000 7f79bb340740 10 client.22404 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:07.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.725+0000 7f79bb340740 1 -- 172.21.15.120:0/814349093 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x565050210550 con 0x565050201d40 2024-09-19T09:24:07.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.725+0000 7f7999ffb640 1 -- 172.21.15.120:0/814349093 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f799c09c140 con 0x565050201d40 2024-09-19T09:24:07.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.725+0000 7f7999ffb640 1 -- 172.21.15.120:0/814349093 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f79ac01d850 con 0x565050201d40 2024-09-19T09:24:07.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.725+0000 7f79bb340740 10 client.22404 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:07.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.725+0000 7f79bb340740 10 client.22404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:24:07.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.725+0000 7f79bb340740 20 client.22404 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:07.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.725+0000 7f79bb340740 1 --2- 172.21.15.120:0/814349093 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x565050211a10 0x565050213e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:07.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.725+0000 7f79bb340740 4 client.22404 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:24:07.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.725+0000 7f79bb340740 1 -- 172.21.15.120:0/814349093 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650501697f0 con 0x565050211a10 2024-09-19T09:24:07.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.725+0000 7f79b259c640 1 --2- 172.21.15.120:0/814349093 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x565050211a10 0x565050213e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:07.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.726+0000 7f7999ffb640 10 client.22404 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:07.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.726+0000 7f79b259c640 1 --2- 172.21.15.120:0/814349093 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x565050211a10 0x565050213e00 crc :-1 s=READY pgs=525 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea71553dc364dee4 server_cookie=7dba552eb17575cc in_seq=0 out_seq=0 2024-09-19T09:24:07.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.727+0000 7f7999ffb640 1 -- 172.21.15.120:0/814349093 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5650501697f0 con 0x565050211a10 2024-09-19T09:24:07.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.727+0000 7f7999ffb640 10 client.22404 handle_command_reply: tid=0 2024-09-19T09:24:07.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.727+0000 7f79bb340740 1 client.22404 shutdown 2024-09-19T09:24:07.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.727+0000 7f79bb340740 1 -- 172.21.15.120:0/814349093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565050201d40 msgr2=0x5650501ff620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:07.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.727+0000 7f79bb340740 1 --2- 172.21.15.120:0/814349093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565050201d40 0x5650501ff620 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f79ac00ed80 tx=0x7f79ac004a40 comp rx=0 tx=0).stop 2024-09-19T09:24:07.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.727+0000 7f79bb340740 1 -- 172.21.15.120:0/814349093 shutdown_connections 2024-09-19T09:24:07.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.727+0000 7f79bb340740 1 --2- 172.21.15.120:0/814349093 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x565050211a10 0x565050213e00 crc :-1 s=READY pgs=525 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.727+0000 7f79bb340740 1 --2- 172.21.15.120:0/814349093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565050201d40 0x5650501ff620 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.727+0000 7f7999ffb640 0 client.22404 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:07.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.727+0000 7f79bb340740 1 --2- 172.21.15.120:0/814349093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650501f7480 0x5650501ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.727+0000 7f79bb340740 1 --2- 172.21.15.120:0/814349093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650501f6b60 0x5650501feba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.727+0000 7f79bb340740 1 -- 172.21.15.120:0/814349093 >> 172.21.15.120:0/814349093 conn(0x5650501f30f0 msgr2=0x5650501f8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:07.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.727+0000 7f79bb340740 1 -- 172.21.15.120:0/814349093 shutdown_connections 2024-09-19T09:24:07.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.727+0000 7f79bb340740 1 -- 172.21.15.120:0/814349093 wait complete. 2024-09-19T09:24:07.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.727+0000 7f79bb340740 20 client.22404 trim_cache size 0 max 16384 2024-09-19T09:24:07.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.744+0000 7f79bb340740 1 Processor -- start 2024-09-19T09:24:07.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.744+0000 7f79bb340740 1 -- start start 2024-09-19T09:24:07.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.744+0000 7f79bb340740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650501f6b60 0x565050219680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:07.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.744+0000 7f79bb340740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650501f7480 0x565050219bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:07.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.744+0000 7f79bb340740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565050201d40 0x5650501f85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:07.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.744+0000 7f79bb340740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f79b41af900 con 0x5650501f6b60 2024-09-19T09:24:07.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.744+0000 7f79bb340740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f79b41b1440 con 0x565050201d40 2024-09-19T09:24:07.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.744+0000 7f79bb340740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f79b41affd0 con 0x5650501f7480 2024-09-19T09:24:07.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.745+0000 7f79b259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650501f6b60 0x565050219680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:07.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.745+0000 7f79b259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650501f6b60 0x565050219680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45422/0 (socket says 172.21.15.120:45422) 2024-09-19T09:24:07.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.745+0000 7f79b259c640 1 -- 172.21.15.120:0/4227136801 learned_addr learned my addr 172.21.15.120:0/4227136801 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:07.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.745+0000 7f79b2d9d640 1 --2- 172.21.15.120:0/4227136801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565050201d40 0x5650501f85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:07.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.745+0000 7f79b259c640 1 -- 172.21.15.120:0/4227136801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650501f7480 msgr2=0x565050219bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:07.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.745+0000 7f79b259c640 1 --2- 172.21.15.120:0/4227136801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650501f7480 0x565050219bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.745+0000 7f79b259c640 1 -- 172.21.15.120:0/4227136801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565050201d40 msgr2=0x5650501f85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:07.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.745+0000 7f79b259c640 1 --2- 172.21.15.120:0/4227136801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565050201d40 0x5650501f85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.745+0000 7f79b259c640 1 -- 172.21.15.120:0/4227136801 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650501f72b0 con 0x5650501f6b60 2024-09-19T09:24:07.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.745+0000 7f79b2d9d640 1 --2- 172.21.15.120:0/4227136801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565050201d40 0x5650501f85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:24:07.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.745+0000 7f79b259c640 1 --2- 172.21.15.120:0/4227136801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650501f6b60 0x565050219680 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7f79a4009930 tx=0x7f79a40058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85c832421dd66f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:07.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.745+0000 7f799bfff640 1 -- 172.21.15.120:0/4227136801 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a4049030 con 0x5650501f6b60 2024-09-19T09:24:07.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.745+0000 7f799bfff640 1 -- 172.21.15.120:0/4227136801 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79a4005020 con 0x5650501f6b60 2024-09-19T09:24:07.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.745+0000 7f799bfff640 1 -- 172.21.15.120:0/4227136801 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a4035040 con 0x5650501f6b60 2024-09-19T09:24:07.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.746+0000 7f79bb340740 1 -- 172.21.15.120:0/4227136801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650501f6b60 msgr2=0x565050219680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:07.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.746+0000 7f79bb340740 1 --2- 172.21.15.120:0/4227136801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650501f6b60 0x565050219680 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7f79a4009930 tx=0x7f79a40058d0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.746+0000 7f79bb340740 1 -- 172.21.15.120:0/4227136801 shutdown_connections 2024-09-19T09:24:07.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.746+0000 7f79bb340740 1 --2- 172.21.15.120:0/4227136801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565050201d40 0x5650501f85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.746+0000 7f79bb340740 1 --2- 172.21.15.120:0/4227136801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650501f7480 0x565050219bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.746+0000 7f79bb340740 1 --2- 172.21.15.120:0/4227136801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650501f6b60 0x565050219680 unknown :-1 s=CLOSED pgs=2217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.746+0000 7f79bb340740 1 -- 172.21.15.120:0/4227136801 >> 172.21.15.120:0/4227136801 conn(0x5650501f30f0 msgr2=0x5650501fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:07.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.746+0000 7f79bb340740 1 -- 172.21.15.120:0/4227136801 shutdown_connections 2024-09-19T09:24:07.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.746+0000 7f79bb340740 1 -- 172.21.15.120:0/4227136801 wait complete. 2024-09-19T09:24:07.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.746+0000 7f79bb340740 1 -- start start 2024-09-19T09:24:07.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.746+0000 7f79bb340740 1 Processor -- start 2024-09-19T09:24:07.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.746+0000 7f79bb340740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650501f6b60 0x5650501fbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:07.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.746+0000 7f79bb340740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650501f7480 0x5650501fc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:07.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.746+0000 7f79bb340740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565050201d40 0x5650501fc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:07.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.747+0000 7f79bb340740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f79b41b0d70 con 0x5650501f6b60 2024-09-19T09:24:07.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.747+0000 7f79bb340740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f79b41b06a0 con 0x565050201d40 2024-09-19T09:24:07.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.747+0000 7f79bb340740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f79b41ad8c0 con 0x5650501f7480 2024-09-19T09:24:07.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.747+0000 7f79bb340740 10 client.0 _ll_register_callbacks cb 0x7f79ac00ea30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:07.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.747+0000 7f79b259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650501f6b60 0x5650501fbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:07.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.747+0000 7f79b259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650501f6b60 0x5650501fbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45430/0 (socket says 172.21.15.120:45430) 2024-09-19T09:24:07.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.747+0000 7f79b259c640 1 -- 172.21.15.120:0/890826027 learned_addr learned my addr 172.21.15.120:0/890826027 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:07.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.747+0000 7f79b2d9d640 1 --2- 172.21.15.120:0/890826027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565050201d40 0x5650501fc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:07.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.747+0000 7f79b1d9b640 1 --2- 172.21.15.120:0/890826027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650501f7480 0x5650501fc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:07.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.747+0000 7f79b259c640 1 -- 172.21.15.120:0/890826027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650501f7480 msgr2=0x5650501fc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:07.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.747+0000 7f79b259c640 1 --2- 172.21.15.120:0/890826027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650501f7480 0x5650501fc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.747+0000 7f79b259c640 1 -- 172.21.15.120:0/890826027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565050201d40 msgr2=0x5650501fc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:07.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.747+0000 7f79b259c640 1 --2- 172.21.15.120:0/890826027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565050201d40 0x5650501fc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.747+0000 7f79b259c640 1 -- 172.21.15.120:0/890826027 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56505027b650 con 0x5650501f6b60 2024-09-19T09:24:07.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.747+0000 7f79b2d9d640 1 --2- 172.21.15.120:0/890826027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565050201d40 0x5650501fc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:24:07.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.747+0000 7f79b259c640 1 --2- 172.21.15.120:0/890826027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650501f6b60 0x5650501fbb00 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f79a402a940 tx=0x7f79a4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3155c658274b1357 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:07.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.747+0000 7f7999ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:07.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.747+0000 7f7999ffb640 1 -- 172.21.15.120:0/890826027 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a4053020 con 0x5650501f6b60 2024-09-19T09:24:07.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.747+0000 7f7999ffb640 1 -- 172.21.15.120:0/890826027 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f79a4004770 con 0x5650501f6b60 2024-09-19T09:24:07.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.747+0000 7f79bb340740 1 -- 172.21.15.120:0/890826027 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79a4005020 con 0x5650501f6b60 2024-09-19T09:24:07.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.747+0000 7f7999ffb640 1 -- 172.21.15.120:0/890826027 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f79a4035040 con 0x5650501f6b60 2024-09-19T09:24:07.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.747+0000 7f79bb340740 10 client.22410 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:07.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.747+0000 7f79bb340740 1 -- 172.21.15.120:0/890826027 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56505027c820 con 0x5650501f6b60 2024-09-19T09:24:07.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.748+0000 7f7999ffb640 1 -- 172.21.15.120:0/890826027 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79a4005020 con 0x5650501f6b60 2024-09-19T09:24:07.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.748+0000 7f7999ffb640 1 -- 172.21.15.120:0/890826027 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f79a403cc30 con 0x5650501f6b60 2024-09-19T09:24:07.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.748+0000 7f79bb340740 10 client.22410 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:07.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.748+0000 7f79bb340740 10 client.22410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:24:07.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.748+0000 7f79bb340740 20 client.22410 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:07.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.748+0000 7f79bb340740 1 --2- 172.21.15.120:0/890826027 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56505027cf60 0x56505027f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:07.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.748+0000 7f79bb340740 4 client.22410 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:24:07.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.748+0000 7f79bb340740 1 -- 172.21.15.120:0/890826027 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x565050163d00 con 0x56505027cf60 2024-09-19T09:24:07.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.748+0000 7f79b1d9b640 1 --2- 172.21.15.120:0/890826027 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56505027cf60 0x56505027f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:07.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.748+0000 7f7999ffb640 10 client.22410 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:07.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.748+0000 7f79b1d9b640 1 --2- 172.21.15.120:0/890826027 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56505027cf60 0x56505027f380 crc :-1 s=READY pgs=526 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b51a774ce4b5aa37 server_cookie=2faf87a49b285f9 in_seq=0 out_seq=0 2024-09-19T09:24:07.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.749+0000 7f7999ffb640 1 -- 172.21.15.120:0/890826027 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x565050163d00 con 0x56505027cf60 2024-09-19T09:24:07.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.749+0000 7f7999ffb640 10 client.22410 handle_command_reply: tid=0 2024-09-19T09:24:07.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.749+0000 7f79bb340740 1 client.22410 shutdown 2024-09-19T09:24:07.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.749+0000 7f79bb340740 1 -- 172.21.15.120:0/890826027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650501f6b60 msgr2=0x5650501fbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:07.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.749+0000 7f79bb340740 1 --2- 172.21.15.120:0/890826027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650501f6b60 0x5650501fbb00 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f79a402a940 tx=0x7f79a4000bc0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.749+0000 7f79bb340740 1 -- 172.21.15.120:0/890826027 shutdown_connections 2024-09-19T09:24:07.784 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:24:07.784 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:24:07.784 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "c47e9c84-12e8-48b8-a09d-a63f392f87a6", 2024-09-19T09:24:07.784 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:24:07.784 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:24:07.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.749+0000 7f79bb340740 1 --2- 172.21.15.120:0/890826027 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56505027cf60 0x56505027f380 crc :-1 s=READY pgs=526 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.749+0000 7f79bb340740 1 --2- 172.21.15.120:0/890826027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565050201d40 0x5650501fc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.749+0000 7f79bb340740 1 --2- 172.21.15.120:0/890826027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650501f7480 0x5650501fc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.749+0000 7f79bb340740 1 --2- 172.21.15.120:0/890826027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650501f6b60 0x5650501fbb00 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.749+0000 7f7999ffb640 0 client.22410 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:07.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.749+0000 7f79bb340740 1 -- 172.21.15.120:0/890826027 >> 172.21.15.120:0/890826027 conn(0x5650501f30f0 msgr2=0x5650501f1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:07.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.750+0000 7f79bb340740 1 -- 172.21.15.120:0/890826027 shutdown_connections 2024-09-19T09:24:07.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.750+0000 7f79bb340740 1 -- 172.21.15.120:0/890826027 wait complete. 2024-09-19T09:24:07.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.750+0000 7f79bb340740 20 client.22410 trim_cache size 0 max 16384 2024-09-19T09:24:07.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.750+0000 7f79b8f67640 1 -- 172.21.15.120:0/2976310546 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f79a00a3240 msgr2=0x7f79a00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:07.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.750+0000 7f79b8f67640 1 --2- 172.21.15.120:0/2976310546 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f79a00a3240 0x7f79a00a5700 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f79ac007e50 tx=0x7f79ac007600 comp rx=0 tx=0).stop 2024-09-19T09:24:07.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.750+0000 7f79b8f67640 1 -- 172.21.15.120:0/2976310546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79b4100ca0 msgr2=0x7f79b419ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:07.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.750+0000 7f79b8f67640 1 --2- 172.21.15.120:0/2976310546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79b4100ca0 0x7f79b419ac40 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f799c00d770 tx=0x7f799c00dc30 comp rx=0 tx=0).stop 2024-09-19T09:24:07.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.750+0000 7f79b8f67640 1 -- 172.21.15.120:0/2976310546 shutdown_connections 2024-09-19T09:24:07.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.750+0000 7f79b8f67640 1 --2- 172.21.15.120:0/2976310546 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f79a00a3240 0x7f79a00a5700 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.750+0000 7f79b8f67640 1 --2- 172.21.15.120:0/2976310546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79b4106f50 0x7f79b419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.750+0000 7f79b8f67640 1 --2- 172.21.15.120:0/2976310546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79b4100ca0 0x7f79b419ac40 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.750+0000 7f79b8f67640 1 --2- 172.21.15.120:0/2976310546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79b41002f0 0x7f79b419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:07.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.750+0000 7f79b8f67640 1 -- 172.21.15.120:0/2976310546 >> 172.21.15.120:0/2976310546 conn(0x7f79b4077210 msgr2=0x7f79b41056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:07.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.750+0000 7f79b8f67640 1 -- 172.21.15.120:0/2976310546 shutdown_connections 2024-09-19T09:24:07.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:07.750+0000 7f79b8f67640 1 -- 172.21.15.120:0/2976310546 wait complete. 2024-09-19T09:24:08.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:08 smithi120 ceph-mon[31150]: pgmap v983: 97 pgs: 97 active+clean; 457 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.3 MiB/s wr, 695 op/s 2024-09-19T09:24:08.379 INFO:tasks.workunit.client.0.smithi120.stdout: 10 197298 21.05 MB/sec execute 363 sec latency 6251.452 ms 2024-09-19T09:24:08.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:08 smithi196 ceph-mon[35150]: pgmap v983: 97 pgs: 97 active+clean; 457 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.3 MiB/s wr, 695 op/s 2024-09-19T09:24:08.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:08 smithi153 ceph-mon[35071]: pgmap v983: 97 pgs: 97 active+clean; 457 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.3 MiB/s wr, 695 op/s 2024-09-19T09:24:08.736 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:24:08.979 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:09.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:09 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:24:09.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:09 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:09.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:09 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:24:09.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:09 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:24:09.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:09 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:09.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:09 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:24:09.327 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:09 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:09.327 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:09 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:24:09.327 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:09 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:09.327 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:09 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:09.379 INFO:tasks.workunit.client.0.smithi120.stdout: 10 197298 20.99 MB/sec execute 364 sec latency 7251.533 ms 2024-09-19T09:24:09.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:09 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:24:09.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:09 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:09.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:09 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:24:09.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:09 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:24:09.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:09 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:09.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:09 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:24:09.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:09 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:09.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:09 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:24:09.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:09 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:09.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:09 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:09.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:09 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:24:09.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:09 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:09.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:09 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:24:09.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:09 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:24:09.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:09 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:09.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:09 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:24:09.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:09 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:09.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:09 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:24:09.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:09 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:09.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:09 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:09.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.501+0000 7f0cd0b1e640 1 -- 172.21.15.120:0/1183732152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ccc100a80 msgr2=0x7f0ccc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:09.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.501+0000 7f0cd0b1e640 1 --2- 172.21.15.120:0/1183732152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ccc100a80 0x7f0ccc100ee0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4009930 tx=0x7f0cb402a420 comp rx=0 tx=0).stop 2024-09-19T09:24:09.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.501+0000 7f0cd0b1e640 1 -- 172.21.15.120:0/1183732152 shutdown_connections 2024-09-19T09:24:09.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.501+0000 7f0cd0b1e640 1 --2- 172.21.15.120:0/1183732152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ccc101420 0x7f0ccc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.501+0000 7f0cd0b1e640 1 --2- 172.21.15.120:0/1183732152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ccc100a80 0x7f0ccc100ee0 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.501+0000 7f0cd0b1e640 1 --2- 172.21.15.120:0/1183732152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ccc106a80 0x7f0ccc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.501+0000 7f0cd0b1e640 1 -- 172.21.15.120:0/1183732152 >> 172.21.15.120:0/1183732152 conn(0x7f0ccc0fc7a0 msgr2=0x7f0ccc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:09.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.501+0000 7f0cd0b1e640 1 -- 172.21.15.120:0/1183732152 shutdown_connections 2024-09-19T09:24:09.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.501+0000 7f0cd0b1e640 1 -- 172.21.15.120:0/1183732152 wait complete. 2024-09-19T09:24:09.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.502+0000 7f0cd0b1e640 1 Processor -- start 2024-09-19T09:24:09.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.502+0000 7f0cd0b1e640 1 -- start start 2024-09-19T09:24:09.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.502+0000 7f0cd0b1e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ccc100a80 0x7f0ccc1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:09.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.502+0000 7f0cd0b1e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ccc101420 0x7f0ccc198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:09.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.502+0000 7f0cd0b1e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ccc106a80 0x7f0ccc19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:09.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.502+0000 7f0cd0b1e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc110ea0 con 0x7f0ccc106a80 2024-09-19T09:24:09.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.502+0000 7f0cd0b1e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc110d20 con 0x7f0ccc100a80 2024-09-19T09:24:09.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.502+0000 7f0cd0b1e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc111020 con 0x7f0ccc101420 2024-09-19T09:24:09.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.502+0000 7f0ccad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ccc106a80 0x7f0ccc19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:09.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.502+0000 7f0ccad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ccc106a80 0x7f0ccc19cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45462/0 (socket says 172.21.15.120:45462) 2024-09-19T09:24:09.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.502+0000 7f0ccad9d640 1 -- 172.21.15.120:0/1799084673 learned_addr learned my addr 172.21.15.120:0/1799084673 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:09.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.502+0000 7f0cca59c640 1 --2- 172.21.15.120:0/1799084673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ccc100a80 0x7f0ccc1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:09.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.502+0000 7f0cc3fff640 1 --2- 172.21.15.120:0/1799084673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ccc101420 0x7f0ccc198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:09.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.502+0000 7f0ccad9d640 1 -- 172.21.15.120:0/1799084673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ccc101420 msgr2=0x7f0ccc198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:09.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.502+0000 7f0ccad9d640 1 --2- 172.21.15.120:0/1799084673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ccc101420 0x7f0ccc198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.502+0000 7f0ccad9d640 1 -- 172.21.15.120:0/1799084673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ccc100a80 msgr2=0x7f0ccc1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:09.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.502+0000 7f0ccad9d640 1 --2- 172.21.15.120:0/1799084673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ccc100a80 0x7f0ccc1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.502+0000 7f0ccad9d640 1 -- 172.21.15.120:0/1799084673 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ccc19d480 con 0x7f0ccc106a80 2024-09-19T09:24:09.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.502+0000 7f0cc3fff640 1 --2- 172.21.15.120:0/1799084673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ccc101420 0x7f0ccc198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:24:09.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.503+0000 7f0ccad9d640 1 --2- 172.21.15.120:0/1799084673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ccc106a80 0x7f0ccc19cda0 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400cad0 tx=0x7f0cc400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:09.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.503+0000 7f0cc37fe640 1 -- 172.21.15.120:0/1799084673 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc400ad60 con 0x7f0ccc106a80 2024-09-19T09:24:09.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.503+0000 7f0cc37fe640 1 -- 172.21.15.120:0/1799084673 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cc4005020 con 0x7f0ccc106a80 2024-09-19T09:24:09.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.503+0000 7f0cd0b1e640 1 -- 172.21.15.120:0/1799084673 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ccc19d770 con 0x7f0ccc106a80 2024-09-19T09:24:09.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.503+0000 7f0cc37fe640 1 -- 172.21.15.120:0/1799084673 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc400f6b0 con 0x7f0ccc106a80 2024-09-19T09:24:09.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.503+0000 7f0cd0b1e640 1 -- 172.21.15.120:0/1799084673 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ccc1a5050 con 0x7f0ccc106a80 2024-09-19T09:24:09.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.505+0000 7f0cc37fe640 1 -- 172.21.15.120:0/1799084673 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f0cc4005560 con 0x7f0ccc106a80 2024-09-19T09:24:09.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.505+0000 7f0cd2ef7740 1 Processor -- start 2024-09-19T09:24:09.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.506+0000 7f0cd2ef7740 1 -- start start 2024-09-19T09:24:09.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.506+0000 7f0cd2ef7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfc25d04a0 0x55cfc25d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:09.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.506+0000 7f0cd2ef7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfc25d0dc0 0x55cfc25db140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:09.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.506+0000 7f0cd2ef7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfc25db680 0x55cfc25dda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:09.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.506+0000 7f0cd2ef7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cfc25362e0 con 0x55cfc25d0dc0 2024-09-19T09:24:09.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.506+0000 7f0cd2ef7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cfc24ddce0 con 0x55cfc25db680 2024-09-19T09:24:09.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.506+0000 7f0cd2ef7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cfc25322a0 con 0x55cfc25d04a0 2024-09-19T09:24:09.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.506+0000 7f0cc37fe640 1 --2- 172.21.15.120:0/1799084673 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0cb80a3240 0x7f0cb80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:09.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.506+0000 7f0cca59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfc25d0dc0 0x55cfc25db140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:09.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.506+0000 7f0cca59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfc25d0dc0 0x55cfc25db140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45474/0 (socket says 172.21.15.120:45474) 2024-09-19T09:24:09.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.506+0000 7f0cca59c640 1 -- 172.21.15.120:0/604186551 learned_addr learned my addr 172.21.15.120:0/604186551 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:09.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.506+0000 7f0cc37fe640 1 -- 172.21.15.120:0/1799084673 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f0cc40cf5f0 con 0x7f0ccc106a80 2024-09-19T09:24:09.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.506+0000 7f0ccad9d640 1 --2- 172.21.15.120:0/1799084673 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0cb80a3240 0x7f0cb80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:09.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.506+0000 7f0cc3fff640 1 --2- 172.21.15.120:0/604186551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfc25db680 0x55cfc25dda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:09.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.506+0000 7f0cc3fff640 1 --2- 172.21.15.120:0/604186551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfc25d04a0 0x55cfc25d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:09.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.506+0000 7f0cca59c640 1 -- 172.21.15.120:0/604186551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfc25d04a0 msgr2=0x55cfc25d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:09.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.506+0000 7f0cca59c640 1 --2- 172.21.15.120:0/604186551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfc25d04a0 0x55cfc25d0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.506+0000 7f0cca59c640 1 -- 172.21.15.120:0/604186551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfc25db680 msgr2=0x55cfc25dda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:09.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.506+0000 7f0cca59c640 1 --2- 172.21.15.120:0/604186551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfc25db680 0x55cfc25dda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.506+0000 7f0cca59c640 1 -- 172.21.15.120:0/604186551 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfc24c2440 con 0x55cfc25d0dc0 2024-09-19T09:24:09.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.507+0000 7f0cc3fff640 1 --2- 172.21.15.120:0/604186551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfc25d04a0 0x55cfc25d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:24:09.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.507+0000 7f0cca59c640 1 --2- 172.21.15.120:0/604186551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfc25d0dc0 0x55cfc25db140 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc009a70 tx=0x7f0cbc009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd10de99c8ccfc3c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:09.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.507+0000 7f0ccad9d640 1 --2- 172.21.15.120:0/1799084673 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0cb80a3240 0x7f0cb80a5700 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0cc409bba0 tx=0x7f0cc40054d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:09.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.507+0000 7f0cc0ff9640 1 -- 172.21.15.120:0/604186551 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc017070 con 0x55cfc25d0dc0 2024-09-19T09:24:09.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.507+0000 7f0cc0ff9640 1 -- 172.21.15.120:0/604186551 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cbc013470 con 0x55cfc25d0dc0 2024-09-19T09:24:09.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.507+0000 7f0cc0ff9640 1 -- 172.21.15.120:0/604186551 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc0125f0 con 0x55cfc25d0dc0 2024-09-19T09:24:09.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.508+0000 7f0cd2ef7740 1 -- 172.21.15.120:0/604186551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfc25d0dc0 msgr2=0x55cfc25db140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:09.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.508+0000 7f0cd2ef7740 1 --2- 172.21.15.120:0/604186551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfc25d0dc0 0x55cfc25db140 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc009a70 tx=0x7f0cbc009f30 comp rx=0 tx=0).stop 2024-09-19T09:24:09.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.508+0000 7f0cd2ef7740 1 -- 172.21.15.120:0/604186551 shutdown_connections 2024-09-19T09:24:09.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.508+0000 7f0cd2ef7740 1 --2- 172.21.15.120:0/604186551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfc25db680 0x55cfc25dda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.508+0000 7f0cd2ef7740 1 --2- 172.21.15.120:0/604186551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfc25d0dc0 0x55cfc25db140 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.508+0000 7f0cd2ef7740 1 --2- 172.21.15.120:0/604186551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfc25d04a0 0x55cfc25d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.508+0000 7f0cd2ef7740 1 -- 172.21.15.120:0/604186551 >> 172.21.15.120:0/604186551 conn(0x55cfc25cca30 msgr2=0x55cfc25cce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:09.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.508+0000 7f0cd2ef7740 1 -- 172.21.15.120:0/604186551 shutdown_connections 2024-09-19T09:24:09.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.508+0000 7f0cd2ef7740 1 -- 172.21.15.120:0/604186551 wait complete. 2024-09-19T09:24:09.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.508+0000 7f0cd2ef7740 1 -- start start 2024-09-19T09:24:09.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.508+0000 7f0cd2ef7740 1 Processor -- start 2024-09-19T09:24:09.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.509+0000 7f0cd2ef7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfc25d04a0 0x55cfc25daf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:09.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.509+0000 7f0cd2ef7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfc25d0dc0 0x55cfc25d81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:09.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.509+0000 7f0cd2ef7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfc25db680 0x55cfc25d8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:09.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.509+0000 7f0cd2ef7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cfc25de250 con 0x55cfc25db680 2024-09-19T09:24:09.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.509+0000 7f0cd2ef7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cfc24f1050 con 0x55cfc25d04a0 2024-09-19T09:24:09.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.509+0000 7f0cd2ef7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cfc25de3d0 con 0x55cfc25d0dc0 2024-09-19T09:24:09.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.509+0000 7f0cd2ef7740 10 client.0 _ll_register_callbacks cb 0x55cfc2531160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:09.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.509+0000 7f0cc3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfc25db680 0x55cfc25d8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:09.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.509+0000 7f0cc3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfc25db680 0x55cfc25d8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45478/0 (socket says 172.21.15.120:45478) 2024-09-19T09:24:09.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.509+0000 7f0cc3fff640 1 -- 172.21.15.120:0/1672150081 learned_addr learned my addr 172.21.15.120:0/1672150081 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:09.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.509+0000 7f0cc3fff640 1 --2- 172.21.15.120:0/1672150081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfc25d04a0 0x55cfc25daf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:09.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.509+0000 7f0cca59c640 1 --2- 172.21.15.120:0/1672150081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfc25d0dc0 0x55cfc25d81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:09.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.509+0000 7f0cc3fff640 1 -- 172.21.15.120:0/1672150081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfc25d0dc0 msgr2=0x55cfc25d81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:09.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.509+0000 7f0cc3fff640 1 --2- 172.21.15.120:0/1672150081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfc25d0dc0 0x55cfc25d81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.509+0000 7f0cc3fff640 1 -- 172.21.15.120:0/1672150081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfc25d04a0 msgr2=0x55cfc25daf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:09.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.509+0000 7f0cc3fff640 1 --2- 172.21.15.120:0/1672150081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfc25d04a0 0x55cfc25daf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.509+0000 7f0cc3fff640 1 -- 172.21.15.120:0/1672150081 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfc22a73c0 con 0x55cfc25db680 2024-09-19T09:24:09.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.509+0000 7f0cca59c640 1 --2- 172.21.15.120:0/1672150081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfc25d0dc0 0x55cfc25d81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:24:09.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.509+0000 7f0cc3fff640 1 --2- 172.21.15.120:0/1672150081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfc25db680 0x55cfc25d8710 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4009340 tx=0x7f0cb40092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2bc76047ae11eeb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:09.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.509+0000 7f0cb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:09.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.509+0000 7f0cb1ffb640 1 -- 172.21.15.120:0/1672150081 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4035040 con 0x55cfc25db680 2024-09-19T09:24:09.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.509+0000 7f0cb1ffb640 1 -- 172.21.15.120:0/1672150081 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f0cb4004480 con 0x55cfc25db680 2024-09-19T09:24:09.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.509+0000 7f0cd2ef7740 1 -- 172.21.15.120:0/1672150081 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cbc013470 con 0x55cfc25db680 2024-09-19T09:24:09.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.510+0000 7f0cb1ffb640 1 -- 172.21.15.120:0/1672150081 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0cb4005020 con 0x55cfc25db680 2024-09-19T09:24:09.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.510+0000 7f0cd2ef7740 10 client.22422 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:09.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.510+0000 7f0cd2ef7740 1 -- 172.21.15.120:0/1672150081 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55cfc25e9ae0 con 0x55cfc25db680 2024-09-19T09:24:09.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.510+0000 7f0cb1ffb640 1 -- 172.21.15.120:0/1672150081 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cbc013470 con 0x55cfc25db680 2024-09-19T09:24:09.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.510+0000 7f0cb1ffb640 1 -- 172.21.15.120:0/1672150081 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f0cb403a040 con 0x55cfc25db680 2024-09-19T09:24:09.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.510+0000 7f0cd2ef7740 10 client.22422 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:09.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.510+0000 7f0cd2ef7740 10 client.22422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:24:09.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.510+0000 7f0cd2ef7740 20 client.22422 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:09.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.510+0000 7f0cd2ef7740 1 --2- 172.21.15.120:0/1672150081 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cfc25eafa0 0x55cfc25ed3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:09.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.510+0000 7f0cd2ef7740 4 client.22422 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:24:09.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.510+0000 7f0cd2ef7740 1 -- 172.21.15.120:0/1672150081 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfc2543850 con 0x55cfc25eafa0 2024-09-19T09:24:09.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.510+0000 7f0cca59c640 1 --2- 172.21.15.120:0/1672150081 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cfc25eafa0 0x55cfc25ed3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:09.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.511+0000 7f0cb1ffb640 10 client.22422 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:09.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.511+0000 7f0cca59c640 1 --2- 172.21.15.120:0/1672150081 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cfc25eafa0 0x55cfc25ed3b0 crc :-1 s=READY pgs=527 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1c2a05e9a585cf server_cookie=3780ef068596279c in_seq=0 out_seq=0 2024-09-19T09:24:09.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.512+0000 7f0cb1ffb640 1 -- 172.21.15.120:0/1672150081 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cfc2543850 con 0x55cfc25eafa0 2024-09-19T09:24:09.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.512+0000 7f0cb1ffb640 10 client.22422 handle_command_reply: tid=0 2024-09-19T09:24:09.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.512+0000 7f0cd2ef7740 1 client.22422 shutdown 2024-09-19T09:24:09.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.512+0000 7f0cd2ef7740 1 -- 172.21.15.120:0/1672150081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfc25db680 msgr2=0x55cfc25d8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:09.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.512+0000 7f0cd2ef7740 1 --2- 172.21.15.120:0/1672150081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfc25db680 0x55cfc25d8710 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4009340 tx=0x7f0cb40092d0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.512+0000 7f0cd2ef7740 1 -- 172.21.15.120:0/1672150081 shutdown_connections 2024-09-19T09:24:09.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.512+0000 7f0cd2ef7740 1 --2- 172.21.15.120:0/1672150081 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cfc25eafa0 0x55cfc25ed3b0 crc :-1 s=READY pgs=527 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.512+0000 7f0cd2ef7740 1 --2- 172.21.15.120:0/1672150081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfc25db680 0x55cfc25d8710 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.512+0000 7f0cd2ef7740 1 --2- 172.21.15.120:0/1672150081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfc25d0dc0 0x55cfc25d81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.512+0000 7f0cd2ef7740 1 --2- 172.21.15.120:0/1672150081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfc25d04a0 0x55cfc25daf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.512+0000 7f0cb1ffb640 0 client.22422 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:09.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.512+0000 7f0cd2ef7740 1 -- 172.21.15.120:0/1672150081 >> 172.21.15.120:0/1672150081 conn(0x55cfc25cca30 msgr2=0x55cfc25d2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:09.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.512+0000 7f0cd2ef7740 1 -- 172.21.15.120:0/1672150081 shutdown_connections 2024-09-19T09:24:09.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.512+0000 7f0cd2ef7740 1 -- 172.21.15.120:0/1672150081 wait complete. 2024-09-19T09:24:09.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.512+0000 7f0cd2ef7740 20 client.22422 trim_cache size 0 max 16384 2024-09-19T09:24:09.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.529+0000 7f0cd2ef7740 1 Processor -- start 2024-09-19T09:24:09.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.529+0000 7f0cd2ef7740 1 -- start start 2024-09-19T09:24:09.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.529+0000 7f0cd2ef7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfc25cca30 0x55cfc25da140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:09.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.529+0000 7f0cd2ef7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfc25d04a0 0x55cfc25f2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:09.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.529+0000 7f0cd2ef7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfc25d0dc0 0x55cfc25f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:09.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.529+0000 7f0cd2ef7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1ad710 con 0x55cfc25cca30 2024-09-19T09:24:09.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.529+0000 7f0cd2ef7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1af250 con 0x55cfc25d0dc0 2024-09-19T09:24:09.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.529+0000 7f0cd2ef7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1adde0 con 0x55cfc25d04a0 2024-09-19T09:24:09.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.529+0000 7f0cc3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfc25cca30 0x55cfc25da140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:09.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.529+0000 7f0cc3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfc25cca30 0x55cfc25da140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45488/0 (socket says 172.21.15.120:45488) 2024-09-19T09:24:09.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.529+0000 7f0cc3fff640 1 -- 172.21.15.120:0/432495552 learned_addr learned my addr 172.21.15.120:0/432495552 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:09.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.529+0000 7f0cc3fff640 1 -- 172.21.15.120:0/432495552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfc25d04a0 msgr2=0x55cfc25f2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:09.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.529+0000 7f0cc3fff640 1 --2- 172.21.15.120:0/432495552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfc25d04a0 0x55cfc25f2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.529+0000 7f0cc3fff640 1 -- 172.21.15.120:0/432495552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfc25d0dc0 msgr2=0x55cfc25f31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:09.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.529+0000 7f0cc3fff640 1 --2- 172.21.15.120:0/432495552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfc25d0dc0 0x55cfc25f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.529+0000 7f0cc3fff640 1 -- 172.21.15.120:0/432495552 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfc25d6720 con 0x55cfc25cca30 2024-09-19T09:24:09.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.530+0000 7f0cc3fff640 1 --2- 172.21.15.120:0/432495552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfc25cca30 0x55cfc25da140 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4001eb0 tx=0x7f0cb4005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359464716c4bd791 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:09.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.530+0000 7f0cb3fff640 1 -- 172.21.15.120:0/432495552 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4035070 con 0x55cfc25cca30 2024-09-19T09:24:09.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.530+0000 7f0cb3fff640 1 -- 172.21.15.120:0/432495552 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cb4004020 con 0x55cfc25cca30 2024-09-19T09:24:09.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.530+0000 7f0cb3fff640 1 -- 172.21.15.120:0/432495552 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4004910 con 0x55cfc25cca30 2024-09-19T09:24:09.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.530+0000 7f0cd2ef7740 1 -- 172.21.15.120:0/432495552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfc25cca30 msgr2=0x55cfc25da140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:09.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.530+0000 7f0cd2ef7740 1 --2- 172.21.15.120:0/432495552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfc25cca30 0x55cfc25da140 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4001eb0 tx=0x7f0cb4005a40 comp rx=0 tx=0).stop 2024-09-19T09:24:09.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.530+0000 7f0cd2ef7740 1 -- 172.21.15.120:0/432495552 shutdown_connections 2024-09-19T09:24:09.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.530+0000 7f0cd2ef7740 1 --2- 172.21.15.120:0/432495552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfc25d0dc0 0x55cfc25f31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.530+0000 7f0cd2ef7740 1 --2- 172.21.15.120:0/432495552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfc25d04a0 0x55cfc25f2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.530+0000 7f0cd2ef7740 1 --2- 172.21.15.120:0/432495552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfc25cca30 0x55cfc25da140 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.530+0000 7f0cd2ef7740 1 -- 172.21.15.120:0/432495552 >> 172.21.15.120:0/432495552 conn(0x55cfc25d8870 msgr2=0x55cfc25d9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:09.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.530+0000 7f0cd2ef7740 1 -- 172.21.15.120:0/432495552 shutdown_connections 2024-09-19T09:24:09.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.531+0000 7f0cd2ef7740 1 -- 172.21.15.120:0/432495552 wait complete. 2024-09-19T09:24:09.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.531+0000 7f0cd2ef7740 1 -- start start 2024-09-19T09:24:09.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.531+0000 7f0cd2ef7740 1 Processor -- start 2024-09-19T09:24:09.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.531+0000 7f0cd2ef7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfc25cca30 0x55cfc25e1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:09.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.531+0000 7f0cd2ef7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfc25d04a0 0x55cfc25e1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:09.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.531+0000 7f0cd2ef7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfc25d0dc0 0x55cfc2654c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:09.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.531+0000 7f0cd2ef7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1aeb80 con 0x55cfc25cca30 2024-09-19T09:24:09.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.531+0000 7f0cd2ef7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1ae4b0 con 0x55cfc25d04a0 2024-09-19T09:24:09.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.531+0000 7f0cd2ef7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1ab6d0 con 0x55cfc25d0dc0 2024-09-19T09:24:09.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.531+0000 7f0cd2ef7740 10 client.0 _ll_register_callbacks cb 0x7f0cb4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:09.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.531+0000 7f0cc3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfc25cca30 0x55cfc25e1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:09.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.531+0000 7f0cc3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfc25cca30 0x55cfc25e1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45502/0 (socket says 172.21.15.120:45502) 2024-09-19T09:24:09.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.531+0000 7f0cc3fff640 1 -- 172.21.15.120:0/250033347 learned_addr learned my addr 172.21.15.120:0/250033347 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:09.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.531+0000 7f0cca59c640 1 --2- 172.21.15.120:0/250033347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfc25d04a0 0x55cfc25e1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:09.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.531+0000 7f0cc3fff640 1 -- 172.21.15.120:0/250033347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfc25d0dc0 msgr2=0x55cfc2654c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:09.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.532+0000 7f0cc3fff640 1 --2- 172.21.15.120:0/250033347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfc25d0dc0 0x55cfc2654c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.532+0000 7f0cc3fff640 1 -- 172.21.15.120:0/250033347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfc25d04a0 msgr2=0x55cfc25e1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:09.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.532+0000 7f0cc3fff640 1 --2- 172.21.15.120:0/250033347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfc25d04a0 0x55cfc25e1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.532+0000 7f0cc3fff640 1 -- 172.21.15.120:0/250033347 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfc25e2500 con 0x55cfc25cca30 2024-09-19T09:24:09.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.532+0000 7f0cc3fff640 1 --2- 172.21.15.120:0/250033347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfc25cca30 0x55cfc25e1700 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4004220 tx=0x7f0cb40314b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d68fc33c16798feb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:09.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.532+0000 7f0cb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:09.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.532+0000 7f0cb1ffb640 1 -- 172.21.15.120:0/250033347 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb405e060 con 0x55cfc25cca30 2024-09-19T09:24:09.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.532+0000 7f0cb1ffb640 1 -- 172.21.15.120:0/250033347 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f0cb40055d0 con 0x55cfc25cca30 2024-09-19T09:24:09.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.532+0000 7f0cd2ef7740 1 -- 172.21.15.120:0/250033347 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cfc253dd60 con 0x55cfc25cca30 2024-09-19T09:24:09.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.532+0000 7f0cb1ffb640 1 -- 172.21.15.120:0/250033347 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0cb4035070 con 0x55cfc25cca30 2024-09-19T09:24:09.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.532+0000 7f0cd2ef7740 10 client.22428 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:09.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.532+0000 7f0cd2ef7740 1 -- 172.21.15.120:0/250033347 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55cfc26562b0 con 0x55cfc25cca30 2024-09-19T09:24:09.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.532+0000 7f0cb1ffb640 1 -- 172.21.15.120:0/250033347 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55cfc253dd60 con 0x55cfc25cca30 2024-09-19T09:24:09.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.532+0000 7f0cb1ffb640 1 -- 172.21.15.120:0/250033347 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f0cb40599c0 con 0x55cfc25cca30 2024-09-19T09:24:09.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.532+0000 7f0cd2ef7740 10 client.22428 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:09.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.532+0000 7f0cd2ef7740 10 client.22428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:24:09.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.532+0000 7f0cd2ef7740 20 client.22428 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:09.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.532+0000 7f0cd2ef7740 1 --2- 172.21.15.120:0/250033347 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cfc25db680 0x55cfc2658880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:09.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.532+0000 7f0cd2ef7740 4 client.22428 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:24:09.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.532+0000 7f0cd2ef7740 1 -- 172.21.15.120:0/250033347 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cfc24ba210 con 0x55cfc25db680 2024-09-19T09:24:09.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.533+0000 7f0cca59c640 1 --2- 172.21.15.120:0/250033347 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cfc25db680 0x55cfc2658880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:09.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.533+0000 7f0cb1ffb640 10 client.22428 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:09.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.533+0000 7f0cca59c640 1 --2- 172.21.15.120:0/250033347 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cfc25db680 0x55cfc2658880 crc :-1 s=READY pgs=528 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66982d47072a1d01 server_cookie=5307fea5179fe9d in_seq=0 out_seq=0 2024-09-19T09:24:09.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.533+0000 7f0cb1ffb640 1 -- 172.21.15.120:0/250033347 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cfc24ba210 con 0x55cfc25db680 2024-09-19T09:24:09.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.533+0000 7f0cb1ffb640 10 client.22428 handle_command_reply: tid=0 2024-09-19T09:24:09.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.533+0000 7f0cd2ef7740 1 client.22428 shutdown 2024-09-19T09:24:09.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.533+0000 7f0cd2ef7740 1 -- 172.21.15.120:0/250033347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfc25cca30 msgr2=0x55cfc25e1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:09.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.533+0000 7f0cd2ef7740 1 --2- 172.21.15.120:0/250033347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfc25cca30 0x55cfc25e1700 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4004220 tx=0x7f0cb40314b0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.533+0000 7f0cd2ef7740 1 -- 172.21.15.120:0/250033347 shutdown_connections 2024-09-19T09:24:09.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.533+0000 7f0cd2ef7740 1 --2- 172.21.15.120:0/250033347 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cfc25db680 0x55cfc2658880 crc :-1 s=READY pgs=528 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.533+0000 7f0cd2ef7740 1 --2- 172.21.15.120:0/250033347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cfc25d0dc0 0x55cfc2654c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.533+0000 7f0cd2ef7740 1 --2- 172.21.15.120:0/250033347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cfc25d04a0 0x55cfc25e1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.533+0000 7f0cb1ffb640 0 client.22428 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:09.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.533+0000 7f0cd2ef7740 1 --2- 172.21.15.120:0/250033347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cfc25cca30 0x55cfc25e1700 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.533+0000 7f0cd2ef7740 1 -- 172.21.15.120:0/250033347 >> 172.21.15.120:0/250033347 conn(0x55cfc25d8870 msgr2=0x55cfc25da250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:09.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.533+0000 7f0cd2ef7740 1 -- 172.21.15.120:0/250033347 shutdown_connections 2024-09-19T09:24:09.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.533+0000 7f0cd2ef7740 1 -- 172.21.15.120:0/250033347 wait complete. 2024-09-19T09:24:09.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.534+0000 7f0cd2ef7740 20 client.22428 trim_cache size 0 max 16384 2024-09-19T09:24:09.542 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:24:09.542 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:24:09.542 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-19T09:24:09.542 INFO:teuthology.orchestra.run.smithi120.stdout: "c47e9c84-12e8-48b8-a09d-a63f392f87a6": { 2024-09-19T09:24:09.542 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-19T09:24:09.542 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "c47e9c84-12e8-48b8-a09d-a63f392f87a6", 2024-09-19T09:24:09.542 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-19T09:24:09.542 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:24:09.542 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:24:09.542 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:24:09.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.534+0000 7f0cd0b1e640 1 -- 172.21.15.120:0/1799084673 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0cb80a3240 msgr2=0x7f0cb80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:09.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.534+0000 7f0cd0b1e640 1 --2- 172.21.15.120:0/1799084673 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0cb80a3240 0x7f0cb80a5700 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0cc409bba0 tx=0x7f0cc40054d0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.534+0000 7f0cd0b1e640 1 -- 172.21.15.120:0/1799084673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ccc106a80 msgr2=0x7f0ccc19cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:09.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.534+0000 7f0cd0b1e640 1 --2- 172.21.15.120:0/1799084673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ccc106a80 0x7f0ccc19cda0 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400cad0 tx=0x7f0cc400ced0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.534+0000 7f0cd0b1e640 1 -- 172.21.15.120:0/1799084673 shutdown_connections 2024-09-19T09:24:09.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.534+0000 7f0cd0b1e640 1 --2- 172.21.15.120:0/1799084673 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0cb80a3240 0x7f0cb80a5700 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.534+0000 7f0cd0b1e640 1 --2- 172.21.15.120:0/1799084673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ccc106a80 0x7f0ccc19cda0 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.534+0000 7f0cd0b1e640 1 --2- 172.21.15.120:0/1799084673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ccc101420 0x7f0ccc198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.534+0000 7f0cd0b1e640 1 --2- 172.21.15.120:0/1799084673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ccc100a80 0x7f0ccc1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:09.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.534+0000 7f0cd0b1e640 1 -- 172.21.15.120:0/1799084673 >> 172.21.15.120:0/1799084673 conn(0x7f0ccc0fc7a0 msgr2=0x7f0ccc06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:09.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.534+0000 7f0cd0b1e640 1 -- 172.21.15.120:0/1799084673 shutdown_connections 2024-09-19T09:24:09.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:09.534+0000 7f0cd0b1e640 1 -- 172.21.15.120:0/1799084673 wait complete. 2024-09-19T09:24:10.320 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:10 smithi120 ceph-mon[31150]: pgmap v984: 97 pgs: 97 active+clean; 459 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.3 MiB/s wr, 660 op/s 2024-09-19T09:24:10.347 INFO:tasks.cephfs.filesystem:scrub status for tag:c47e9c84-12e8-48b8-a09d-a63f392f87a6 - {'path': '/', 'tag': 'c47e9c84-12e8-48b8-a09d-a63f392f87a6', 'options': 'recursive,force'} 2024-09-19T09:24:10.379 INFO:tasks.workunit.client.0.smithi120.stdout: 10 197298 20.93 MB/sec execute 365 sec latency 8251.612 ms 2024-09-19T09:24:10.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:10 smithi196 ceph-mon[35150]: pgmap v984: 97 pgs: 97 active+clean; 459 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.3 MiB/s wr, 660 op/s 2024-09-19T09:24:10.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:10 smithi153 ceph-mon[35071]: pgmap v984: 97 pgs: 97 active+clean; 459 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.3 MiB/s wr, 660 op/s 2024-09-19T09:24:11.379 INFO:tasks.workunit.client.0.smithi120.stdout: 10 197298 20.88 MB/sec execute 366 sec latency 9251.655 ms 2024-09-19T09:24:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.390+0000 7f5ff8bf6640 1 -- 172.21.15.120:0/1823853937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ff41025f0 msgr2=0x7f5ff4108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.390+0000 7f5ff8bf6640 1 --2- 172.21.15.120:0/1823853937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ff41025f0 0x7f5ff4108e80 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f5fec009ca0 tx=0x7f5fec02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:24:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.390+0000 7f5ff8bf6640 1 -- 172.21.15.120:0/1823853937 shutdown_connections 2024-09-19T09:24:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.390+0000 7f5ff8bf6640 1 --2- 172.21.15.120:0/1823853937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ff41025f0 0x7f5ff4108e80 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.390+0000 7f5ff8bf6640 1 --2- 172.21.15.120:0/1823853937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ff4101c30 0x7f5ff41020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.390+0000 7f5ff8bf6640 1 --2- 172.21.15.120:0/1823853937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ff4100a40 0x7f5ff4100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.390+0000 7f5ff8bf6640 1 -- 172.21.15.120:0/1823853937 >> 172.21.15.120:0/1823853937 conn(0x7f5ff40fc1e0 msgr2=0x7f5ff40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.390+0000 7f5ff8bf6640 1 -- 172.21.15.120:0/1823853937 shutdown_connections 2024-09-19T09:24:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.390+0000 7f5ff8bf6640 1 -- 172.21.15.120:0/1823853937 wait complete. 2024-09-19T09:24:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.391+0000 7f5ff8bf6640 1 Processor -- start 2024-09-19T09:24:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.391+0000 7f5ff8bf6640 1 -- start start 2024-09-19T09:24:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.391+0000 7f5ff8bf6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ff4100a40 0x7f5ff4196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.391+0000 7f5ff8bf6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ff4101c30 0x7f5ff4197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.391+0000 7f5ff8bf6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ff41025f0 0x7f5ff419e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.391+0000 7f5ff8bf6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff410c6a0 con 0x7f5ff41025f0 2024-09-19T09:24:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.391+0000 7f5ff8bf6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff410c520 con 0x7f5ff4100a40 2024-09-19T09:24:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.391+0000 7f5ff8bf6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff410c820 con 0x7f5ff4101c30 2024-09-19T09:24:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.391+0000 7f5ff2d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ff41025f0 0x7f5ff419e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.391+0000 7f5ff2d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ff41025f0 0x7f5ff419e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60532/0 (socket says 172.21.15.120:60532) 2024-09-19T09:24:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.391+0000 7f5ff2d9d640 1 -- 172.21.15.120:0/3037245347 learned_addr learned my addr 172.21.15.120:0/3037245347 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.391+0000 7f5ff259c640 1 --2- 172.21.15.120:0/3037245347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ff4100a40 0x7f5ff4196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.391+0000 7f5fe9d9b640 1 --2- 172.21.15.120:0/3037245347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ff4101c30 0x7f5ff4197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.392+0000 7f5ff2d9d640 1 -- 172.21.15.120:0/3037245347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ff4101c30 msgr2=0x7f5ff4197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.392+0000 7f5ff2d9d640 1 --2- 172.21.15.120:0/3037245347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ff4101c30 0x7f5ff4197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.392+0000 7f5ff2d9d640 1 -- 172.21.15.120:0/3037245347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ff4100a40 msgr2=0x7f5ff4196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.392+0000 7f5ff2d9d640 1 --2- 172.21.15.120:0/3037245347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ff4100a40 0x7f5ff4196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.392+0000 7f5ff2d9d640 1 -- 172.21.15.120:0/3037245347 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff419e860 con 0x7f5ff41025f0 2024-09-19T09:24:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.392+0000 7f5fe9d9b640 1 --2- 172.21.15.120:0/3037245347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ff4101c30 0x7f5ff4197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:24:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.392+0000 7f5ff2d9d640 1 --2- 172.21.15.120:0/3037245347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ff41025f0 0x7f5ff419e1c0 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f5fec005e30 tx=0x7f5fec031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.392+0000 7f5febfff640 1 -- 172.21.15.120:0/3037245347 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec003c30 con 0x7f5ff41025f0 2024-09-19T09:24:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.392+0000 7f5ff8bf6640 1 -- 172.21.15.120:0/3037245347 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff419eaf0 con 0x7f5ff41025f0 2024-09-19T09:24:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.392+0000 7f5febfff640 1 -- 172.21.15.120:0/3037245347 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fec003dd0 con 0x7f5ff41025f0 2024-09-19T09:24:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.392+0000 7f5febfff640 1 -- 172.21.15.120:0/3037245347 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec02fde0 con 0x7f5ff41025f0 2024-09-19T09:24:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.392+0000 7f5ff8bf6640 1 -- 172.21.15.120:0/3037245347 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff419f040 con 0x7f5ff41025f0 2024-09-19T09:24:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.394+0000 7f5febfff640 1 -- 172.21.15.120:0/3037245347 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5fec004050 con 0x7f5ff41025f0 2024-09-19T09:24:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.395+0000 7f5ffba6d740 1 Processor -- start 2024-09-19T09:24:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.395+0000 7f5ffba6d740 1 -- start start 2024-09-19T09:24:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.395+0000 7f5ffba6d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cd909dbb0 0x560cd909dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.395+0000 7f5ffba6d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd909e4f0 0x560cd90a2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.395+0000 7f5ffba6d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cd90a3580 0x560cd90a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.395+0000 7f5ffba6d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560cd8fff2f0 con 0x560cd90a3580 2024-09-19T09:24:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.395+0000 7f5ffba6d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560cd8fa7270 con 0x560cd909e4f0 2024-09-19T09:24:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.395+0000 7f5ffba6d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560cd8ffb2b0 con 0x560cd909dbb0 2024-09-19T09:24:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.395+0000 7f5febfff640 1 --2- 172.21.15.120:0/3037245347 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5fe00a32f0 0x7f5fe00a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.395+0000 7f5fe9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cd90a3580 0x560cd90a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.395+0000 7f5ff2d9d640 1 --2- 172.21.15.120:0/3037245347 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5fe00a32f0 0x7f5fe00a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.395+0000 7f5febfff640 1 -- 172.21.15.120:0/3037245347 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f5fec0f0dc0 con 0x7f5ff41025f0 2024-09-19T09:24:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.395+0000 7f5fe9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cd90a3580 0x560cd90a5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60534/0 (socket says 172.21.15.120:60534) 2024-09-19T09:24:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.395+0000 7f5fe9d9b640 1 -- 172.21.15.120:0/3180578977 learned_addr learned my addr 172.21.15.120:0/3180578977 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.395+0000 7f5ff259c640 1 --2- 172.21.15.120:0/3180578977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd909e4f0 0x560cd90a2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.396+0000 7f5fe9d9b640 1 --2- 172.21.15.120:0/3180578977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cd909dbb0 0x560cd909dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.396+0000 7f5fe9d9b640 1 -- 172.21.15.120:0/3180578977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cd909dbb0 msgr2=0x560cd909dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.396+0000 7f5fe9d9b640 1 --2- 172.21.15.120:0/3180578977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cd909dbb0 0x560cd909dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.396+0000 7f5fe9d9b640 1 -- 172.21.15.120:0/3180578977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd909e4f0 msgr2=0x560cd90a2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.396+0000 7f5fe9d9b640 1 --2- 172.21.15.120:0/3180578977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd909e4f0 0x560cd90a2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.396+0000 7f5fe9d9b640 1 -- 172.21.15.120:0/3180578977 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cd8fa6340 con 0x560cd90a3580 2024-09-19T09:24:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.396+0000 7f5ff2d9d640 1 --2- 172.21.15.120:0/3037245347 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5fe00a32f0 0x7f5fe00a57b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0bd0e0 tx=0x7f5fec0b2830 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.396+0000 7f5fe9d9b640 1 --2- 172.21.15.120:0/3180578977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cd90a3580 0x560cd90a5a40 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc013860 tx=0x7f5fdc00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f0fcfe9522c6f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.396+0000 7f5fe8d99640 1 -- 172.21.15.120:0/3180578977 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc004220 con 0x560cd90a3580 2024-09-19T09:24:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.396+0000 7f5fe8d99640 1 -- 172.21.15.120:0/3180578977 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fdc017070 con 0x560cd90a3580 2024-09-19T09:24:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.396+0000 7f5fe8d99640 1 -- 172.21.15.120:0/3180578977 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc004b20 con 0x560cd90a3580 2024-09-19T09:24:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.397+0000 7f5ffba6d740 1 -- 172.21.15.120:0/3180578977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cd90a3580 msgr2=0x560cd90a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.397+0000 7f5ffba6d740 1 --2- 172.21.15.120:0/3180578977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cd90a3580 0x560cd90a5a40 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc013860 tx=0x7f5fdc00b480 comp rx=0 tx=0).stop 2024-09-19T09:24:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.397+0000 7f5ffba6d740 1 -- 172.21.15.120:0/3180578977 shutdown_connections 2024-09-19T09:24:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.397+0000 7f5ffba6d740 1 --2- 172.21.15.120:0/3180578977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cd90a3580 0x560cd90a5a40 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.397+0000 7f5ffba6d740 1 --2- 172.21.15.120:0/3180578977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd909e4f0 0x560cd90a2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.397+0000 7f5ffba6d740 1 --2- 172.21.15.120:0/3180578977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cd909dbb0 0x560cd909dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.397+0000 7f5ffba6d740 1 -- 172.21.15.120:0/3180578977 >> 172.21.15.120:0/3180578977 conn(0x560cd9097010 msgr2=0x560cd9097410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.397+0000 7f5ffba6d740 1 -- 172.21.15.120:0/3180578977 shutdown_connections 2024-09-19T09:24:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.397+0000 7f5ffba6d740 1 -- 172.21.15.120:0/3180578977 wait complete. 2024-09-19T09:24:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.397+0000 7f5ffba6d740 1 -- start start 2024-09-19T09:24:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.398+0000 7f5ffba6d740 1 Processor -- start 2024-09-19T09:24:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.398+0000 7f5ffba6d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cd909dbb0 0x560cd90a7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.398+0000 7f5ffba6d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd909e4f0 0x560cd90a8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.398+0000 7f5ffba6d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cd90a3580 0x560cd90af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.398+0000 7f5ffba6d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560cd90a6550 con 0x560cd90a3580 2024-09-19T09:24:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.398+0000 7f5ffba6d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560cd8f88290 con 0x560cd909e4f0 2024-09-19T09:24:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.398+0000 7f5ffba6d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560cd90a66d0 con 0x560cd909dbb0 2024-09-19T09:24:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.398+0000 7f5ffba6d740 10 client.0 _ll_register_callbacks cb 0x560cd8fb9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.398+0000 7f5fe9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cd90a3580 0x560cd90af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.398+0000 7f5ff259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd909e4f0 0x560cd90a8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.398+0000 7f5fe9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cd90a3580 0x560cd90af3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60548/0 (socket says 172.21.15.120:60548) 2024-09-19T09:24:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.398+0000 7f5fe9d9b640 1 -- 172.21.15.120:0/797093609 learned_addr learned my addr 172.21.15.120:0/797093609 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.398+0000 7f5fe9d9b640 1 --2- 172.21.15.120:0/797093609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cd909dbb0 0x560cd90a7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.398+0000 7f5fe9d9b640 1 -- 172.21.15.120:0/797093609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cd909dbb0 msgr2=0x560cd90a7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.398+0000 7f5fe9d9b640 1 --2- 172.21.15.120:0/797093609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cd909dbb0 0x560cd90a7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.398+0000 7f5fe9d9b640 1 -- 172.21.15.120:0/797093609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd909e4f0 msgr2=0x560cd90a8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.398+0000 7f5fe9d9b640 1 --2- 172.21.15.120:0/797093609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd909e4f0 0x560cd90a8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.398+0000 7f5fe9d9b640 1 -- 172.21.15.120:0/797093609 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cd8d12210 con 0x560cd90a3580 2024-09-19T09:24:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.398+0000 7f5ff259c640 1 --2- 172.21.15.120:0/797093609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd909e4f0 0x560cd90a8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:24:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.399+0000 7f5fe9d9b640 1 --2- 172.21.15.120:0/797093609 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cd90a3580 0x560cd90af3b0 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x560cd90a7360 tx=0x7f5fdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2864f904589db3e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.399+0000 7f5fd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.399+0000 7f5fd9ffb640 1 -- 172.21.15.120:0/797093609 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc027040 con 0x560cd90a3580 2024-09-19T09:24:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.399+0000 7f5ffba6d740 1 -- 172.21.15.120:0/797093609 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fdc017070 con 0x560cd90a3580 2024-09-19T09:24:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.399+0000 7f5fd9ffb640 1 -- 172.21.15.120:0/797093609 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f5fdc00eb20 con 0x560cd90a3580 2024-09-19T09:24:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.399+0000 7f5fd9ffb640 1 -- 172.21.15.120:0/797093609 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5fdc034050 con 0x560cd90a3580 2024-09-19T09:24:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.399+0000 7f5ffba6d740 10 client.22440 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.399+0000 7f5ffba6d740 1 -- 172.21.15.120:0/797093609 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560cd8dde620 con 0x560cd90a3580 2024-09-19T09:24:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.399+0000 7f5fd9ffb640 1 -- 172.21.15.120:0/797093609 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fdc017070 con 0x560cd90a3580 2024-09-19T09:24:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.399+0000 7f5fd9ffb640 1 -- 172.21.15.120:0/797093609 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f5fdc00f040 con 0x560cd90a3580 2024-09-19T09:24:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.399+0000 7f5ffba6d740 10 client.22440 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.399+0000 7f5ffba6d740 10 client.22440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:24:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.399+0000 7f5ffba6d740 20 client.22440 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.399+0000 7f5ffba6d740 1 --2- 172.21.15.120:0/797093609 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560cd90b3800 0x560cd90b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.399+0000 7f5ffba6d740 4 client.22440 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:24:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.399+0000 7f5ffba6d740 1 -- 172.21.15.120:0/797093609 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cd900ca20 con 0x560cd90b3800 2024-09-19T09:24:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.400+0000 7f5ff259c640 1 --2- 172.21.15.120:0/797093609 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560cd90b3800 0x560cd90b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.400+0000 7f5fd9ffb640 10 client.22440 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.400+0000 7f5ff259c640 1 --2- 172.21.15.120:0/797093609 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560cd90b3800 0x560cd90b5bf0 crc :-1 s=READY pgs=529 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22d64ababa2fb071 server_cookie=6b3008a09050e6a in_seq=0 out_seq=0 2024-09-19T09:24:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.401+0000 7f5fd9ffb640 1 -- 172.21.15.120:0/797093609 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560cd900ca20 con 0x560cd90b3800 2024-09-19T09:24:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.401+0000 7f5fd9ffb640 10 client.22440 handle_command_reply: tid=0 2024-09-19T09:24:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.401+0000 7f5ffba6d740 1 client.22440 shutdown 2024-09-19T09:24:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.401+0000 7f5ffba6d740 1 -- 172.21.15.120:0/797093609 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cd90a3580 msgr2=0x560cd90af3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.401+0000 7f5ffba6d740 1 --2- 172.21.15.120:0/797093609 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cd90a3580 0x560cd90af3b0 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x560cd90a7360 tx=0x7f5fdc0046c0 comp rx=0 tx=0).stop 2024-09-19T09:24:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.401+0000 7f5ffba6d740 1 -- 172.21.15.120:0/797093609 shutdown_connections 2024-09-19T09:24:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.401+0000 7f5ffba6d740 1 --2- 172.21.15.120:0/797093609 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560cd90b3800 0x560cd90b5bf0 crc :-1 s=READY pgs=529 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.401+0000 7f5ffba6d740 1 --2- 172.21.15.120:0/797093609 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cd90a3580 0x560cd90af3b0 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.401+0000 7f5ffba6d740 1 --2- 172.21.15.120:0/797093609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd909e4f0 0x560cd90a8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.401+0000 7f5ffba6d740 1 --2- 172.21.15.120:0/797093609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cd909dbb0 0x560cd90a7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.401+0000 7f5fd9ffb640 0 client.22440 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.401+0000 7f5ffba6d740 1 -- 172.21.15.120:0/797093609 >> 172.21.15.120:0/797093609 conn(0x560cd9097010 msgr2=0x560cd9098820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.401+0000 7f5ffba6d740 1 -- 172.21.15.120:0/797093609 shutdown_connections 2024-09-19T09:24:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.401+0000 7f5ffba6d740 1 -- 172.21.15.120:0/797093609 wait complete. 2024-09-19T09:24:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.401+0000 7f5ffba6d740 20 client.22440 trim_cache size 0 max 16384 2024-09-19T09:24:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.415+0000 7f5ffba6d740 1 Processor -- start 2024-09-19T09:24:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.415+0000 7f5ffba6d740 1 -- start start 2024-09-19T09:24:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.415+0000 7f5ffba6d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cd909dbb0 0x560cd9097ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.415+0000 7f5ffba6d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd909e4f0 0x560cd90af370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.415+0000 7f5ffba6d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cd90a3580 0x560cd9098400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.415+0000 7f5ffba6d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff41a6d50 con 0x560cd90a3580 2024-09-19T09:24:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.415+0000 7f5ffba6d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff41a8890 con 0x560cd909e4f0 2024-09-19T09:24:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.415+0000 7f5ffba6d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff41a7420 con 0x560cd909dbb0 2024-09-19T09:24:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.416+0000 7f5fe9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cd90a3580 0x560cd9098400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.416+0000 7f5fe9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cd90a3580 0x560cd9098400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60562/0 (socket says 172.21.15.120:60562) 2024-09-19T09:24:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.416+0000 7f5fe9d9b640 1 -- 172.21.15.120:0/3108695052 learned_addr learned my addr 172.21.15.120:0/3108695052 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.416+0000 7f5fe9d9b640 1 --2- 172.21.15.120:0/3108695052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cd909dbb0 0x560cd9097ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.416+0000 7f5ff259c640 1 --2- 172.21.15.120:0/3108695052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd909e4f0 0x560cd90af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.416+0000 7f5fe9d9b640 1 -- 172.21.15.120:0/3108695052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cd909dbb0 msgr2=0x560cd9097ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.416+0000 7f5fe9d9b640 1 --2- 172.21.15.120:0/3108695052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cd909dbb0 0x560cd9097ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.416+0000 7f5fe9d9b640 1 -- 172.21.15.120:0/3108695052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd909e4f0 msgr2=0x560cd90af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.416+0000 7f5fe9d9b640 1 --2- 172.21.15.120:0/3108695052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd909e4f0 0x560cd90af370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.416+0000 7f5fe9d9b640 1 -- 172.21.15.120:0/3108695052 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cd90b5f60 con 0x560cd90a3580 2024-09-19T09:24:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.416+0000 7f5fe9d9b640 1 --2- 172.21.15.120:0/3108695052 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cd90a3580 0x560cd9098400 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00bc80 tx=0x7f5fdc015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c554ad335f1e84a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.416+0000 7f5fdbfff640 1 -- 172.21.15.120:0/3108695052 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc01e990 con 0x560cd90a3580 2024-09-19T09:24:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.416+0000 7f5fdbfff640 1 -- 172.21.15.120:0/3108695052 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fdc017070 con 0x560cd90a3580 2024-09-19T09:24:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.416+0000 7f5fdbfff640 1 -- 172.21.15.120:0/3108695052 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc01ee20 con 0x560cd90a3580 2024-09-19T09:24:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.417+0000 7f5ffba6d740 1 -- 172.21.15.120:0/3108695052 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cd90a3580 msgr2=0x560cd9098400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.417+0000 7f5ffba6d740 1 --2- 172.21.15.120:0/3108695052 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cd90a3580 0x560cd9098400 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00bc80 tx=0x7f5fdc015040 comp rx=0 tx=0).stop 2024-09-19T09:24:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.417+0000 7f5ffba6d740 1 -- 172.21.15.120:0/3108695052 shutdown_connections 2024-09-19T09:24:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.417+0000 7f5ffba6d740 1 --2- 172.21.15.120:0/3108695052 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cd90a3580 0x560cd9098400 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.417+0000 7f5ffba6d740 1 --2- 172.21.15.120:0/3108695052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd909e4f0 0x560cd90af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.417+0000 7f5ffba6d740 1 --2- 172.21.15.120:0/3108695052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cd909dbb0 0x560cd9097ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.417+0000 7f5ffba6d740 1 -- 172.21.15.120:0/3108695052 >> 172.21.15.120:0/3108695052 conn(0x560cd9097010 msgr2=0x560cd90a7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.417+0000 7f5ffba6d740 1 -- 172.21.15.120:0/3108695052 shutdown_connections 2024-09-19T09:24:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.417+0000 7f5ffba6d740 1 -- 172.21.15.120:0/3108695052 wait complete. 2024-09-19T09:24:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.417+0000 7f5ffba6d740 1 -- start start 2024-09-19T09:24:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.417+0000 7f5ffba6d740 1 Processor -- start 2024-09-19T09:24:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.417+0000 7f5ffba6d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd909dbb0 0x560cd90abfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.417+0000 7f5ffba6d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cd909e4f0 0x560cd90ac510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.417+0000 7f5ffba6d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cd90a3580 0x560cd911c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.417+0000 7f5ffba6d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff41a81c0 con 0x560cd90a3580 2024-09-19T09:24:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.417+0000 7f5ffba6d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff41a7af0 con 0x560cd909dbb0 2024-09-19T09:24:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.417+0000 7f5ffba6d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff41a4d10 con 0x560cd909e4f0 2024-09-19T09:24:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.417+0000 7f5ffba6d740 10 client.0 _ll_register_callbacks cb 0x7f5fe400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.418+0000 7f5fe9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cd90a3580 0x560cd911c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.418+0000 7f5fe9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cd90a3580 0x560cd911c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60572/0 (socket says 172.21.15.120:60572) 2024-09-19T09:24:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.418+0000 7f5ff259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cd909e4f0 0x560cd90ac510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.418+0000 7f5fe9d9b640 1 -- 172.21.15.120:0/178946212 learned_addr learned my addr 172.21.15.120:0/178946212 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.418+0000 7f5fe9d9b640 1 --2- 172.21.15.120:0/178946212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd909dbb0 0x560cd90abfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.418+0000 7f5fe9d9b640 1 -- 172.21.15.120:0/178946212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cd909e4f0 msgr2=0x560cd90ac510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.418+0000 7f5fe9d9b640 1 --2- 172.21.15.120:0/178946212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cd909e4f0 0x560cd90ac510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.418+0000 7f5fe9d9b640 1 -- 172.21.15.120:0/178946212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd909dbb0 msgr2=0x560cd90abfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.418+0000 7f5fe9d9b640 1 --2- 172.21.15.120:0/178946212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd909dbb0 0x560cd90abfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.418+0000 7f5fe9d9b640 1 -- 172.21.15.120:0/178946212 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cd90af600 con 0x560cd90a3580 2024-09-19T09:24:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.418+0000 7f5fe9d9b640 1 --2- 172.21.15.120:0/178946212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cd90a3580 0x560cd911c470 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00efb0 tx=0x7f5fdc0137e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732a69a47e53756d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.418+0000 7f5fd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.418+0000 7f5fd9ffb640 1 -- 172.21.15.120:0/178946212 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc01e990 con 0x560cd90a3580 2024-09-19T09:24:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.418+0000 7f5fd9ffb640 1 -- 172.21.15.120:0/178946212 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f5fdc01eba0 con 0x560cd90a3580 2024-09-19T09:24:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.418+0000 7f5ffba6d740 1 -- 172.21.15.120:0/178946212 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fdc017070 con 0x560cd90a3580 2024-09-19T09:24:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.418+0000 7f5fd9ffb640 1 -- 172.21.15.120:0/178946212 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5fdc01eda0 con 0x560cd90a3580 2024-09-19T09:24:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.418+0000 7f5ffba6d740 10 client.22446 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.418+0000 7f5ffba6d740 1 -- 172.21.15.120:0/178946212 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560cd911da20 con 0x560cd90a3580 2024-09-19T09:24:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.419+0000 7f5fd9ffb640 1 -- 172.21.15.120:0/178946212 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fdc017070 con 0x560cd90a3580 2024-09-19T09:24:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.419+0000 7f5fd9ffb640 1 -- 172.21.15.120:0/178946212 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f5fdc0053f0 con 0x560cd90a3580 2024-09-19T09:24:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.419+0000 7f5ffba6d740 10 client.22446 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.419+0000 7f5ffba6d740 10 client.22446 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:24:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.419+0000 7f5ffba6d740 20 client.22446 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.419+0000 7f5ffba6d740 1 --2- 172.21.15.120:0/178946212 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560cd911e350 0x560cd9120770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.419+0000 7f5ffba6d740 4 client.22446 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:24:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.419+0000 7f5ffba6d740 1 -- 172.21.15.120:0/178946212 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560cd9006d70 con 0x560cd911e350 2024-09-19T09:24:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.419+0000 7f5ff259c640 1 --2- 172.21.15.120:0/178946212 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560cd911e350 0x560cd9120770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.419+0000 7f5fd9ffb640 10 client.22446 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:11.419+0000 7f5ff259c640 1 --2- 172.21.15.120:0/178946212 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560cd911e350 0x560cd9120770 crc :-1 s=READY pgs=530 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d13cb6848d10e40 server_cookie=45d7a3f1893d957 in_seq=0 out_seq=0 2024-09-19T09:24:12.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:12 smithi120 ceph-mon[31150]: pgmap v985: 97 pgs: 97 active+clean; 457 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.9 MiB/s wr, 480 op/s 2024-09-19T09:24:12.379 INFO:tasks.workunit.client.0.smithi120.stdout: 10 197298 20.82 MB/sec execute 367 sec latency 10251.735 ms 2024-09-19T09:24:12.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:12 smithi196 ceph-mon[35150]: pgmap v985: 97 pgs: 97 active+clean; 457 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.9 MiB/s wr, 480 op/s 2024-09-19T09:24:12.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:12 smithi153 ceph-mon[35071]: pgmap v985: 97 pgs: 97 active+clean; 457 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.9 MiB/s wr, 480 op/s 2024-09-19T09:24:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:13.374+0000 7f5fd9ffb640 1 -- 172.21.15.120:0/178946212 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560cd9006d70 con 0x560cd911e350 2024-09-19T09:24:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:13.374+0000 7f5fd9ffb640 10 client.22446 handle_command_reply: tid=0 2024-09-19T09:24:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:13.374+0000 7f5ffba6d740 1 client.22446 shutdown 2024-09-19T09:24:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:13.375+0000 7f5ffba6d740 1 -- 172.21.15.120:0/178946212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cd90a3580 msgr2=0x560cd911c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:13.375+0000 7f5ffba6d740 1 --2- 172.21.15.120:0/178946212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cd90a3580 0x560cd911c470 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00efb0 tx=0x7f5fdc0137e0 comp rx=0 tx=0).stop 2024-09-19T09:24:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:13.375+0000 7f5ffba6d740 1 -- 172.21.15.120:0/178946212 shutdown_connections 2024-09-19T09:24:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:13.375+0000 7f5ffba6d740 1 --2- 172.21.15.120:0/178946212 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560cd911e350 0x560cd9120770 crc :-1 s=READY pgs=530 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:13.375+0000 7f5ffba6d740 1 --2- 172.21.15.120:0/178946212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cd90a3580 0x560cd911c470 unknown :-1 s=CLOSED pgs=2230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:13.375+0000 7f5fd9ffb640 0 client.22446 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:13.375+0000 7f5ffba6d740 1 --2- 172.21.15.120:0/178946212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cd909e4f0 0x560cd90ac510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:13.375+0000 7f5ffba6d740 1 --2- 172.21.15.120:0/178946212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560cd909dbb0 0x560cd90abfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:13.375+0000 7f5ffba6d740 1 -- 172.21.15.120:0/178946212 >> 172.21.15.120:0/178946212 conn(0x560cd9097010 msgr2=0x560cd90959a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:13.375+0000 7f5ffba6d740 1 -- 172.21.15.120:0/178946212 shutdown_connections 2024-09-19T09:24:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:13.375+0000 7f5ffba6d740 1 -- 172.21.15.120:0/178946212 wait complete. 2024-09-19T09:24:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:13.375+0000 7f5ffba6d740 20 client.22446 trim_cache size 0 max 16384 2024-09-19T09:24:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:24:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:24:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:24:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:24:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:13.376+0000 7f5ff8bf6640 1 -- 172.21.15.120:0/3037245347 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5fe00a32f0 msgr2=0x7f5fe00a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:13.376+0000 7f5ff8bf6640 1 --2- 172.21.15.120:0/3037245347 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5fe00a32f0 0x7f5fe00a57b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0bd0e0 tx=0x7f5fec0b2830 comp rx=0 tx=0).stop 2024-09-19T09:24:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:13.376+0000 7f5ff8bf6640 1 -- 172.21.15.120:0/3037245347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ff41025f0 msgr2=0x7f5ff419e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:13.376+0000 7f5ff8bf6640 1 --2- 172.21.15.120:0/3037245347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ff41025f0 0x7f5ff419e1c0 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f5fec005e30 tx=0x7f5fec031ee0 comp rx=0 tx=0).stop 2024-09-19T09:24:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:13.376+0000 7f5ff8bf6640 1 -- 172.21.15.120:0/3037245347 shutdown_connections 2024-09-19T09:24:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:13.376+0000 7f5ff8bf6640 1 --2- 172.21.15.120:0/3037245347 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f5fe00a32f0 0x7f5fe00a57b0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:13.376+0000 7f5ff8bf6640 1 --2- 172.21.15.120:0/3037245347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ff41025f0 0x7f5ff419e1c0 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:13.376+0000 7f5ff8bf6640 1 --2- 172.21.15.120:0/3037245347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ff4101c30 0x7f5ff4197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:13.376+0000 7f5ff8bf6640 1 --2- 172.21.15.120:0/3037245347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ff4100a40 0x7f5ff4196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:13.376+0000 7f5ff8bf6640 1 -- 172.21.15.120:0/3037245347 >> 172.21.15.120:0/3037245347 conn(0x7f5ff40fc1e0 msgr2=0x7f5ff4104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:13.376+0000 7f5ff8bf6640 1 -- 172.21.15.120:0/3037245347 shutdown_connections 2024-09-19T09:24:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:13.376+0000 7f5ff8bf6640 1 -- 172.21.15.120:0/3037245347 wait complete. 2024-09-19T09:24:13.380 INFO:tasks.workunit.client.0.smithi120.stdout: 10 197298 20.76 MB/sec execute 368 sec latency 11251.770 ms 2024-09-19T09:24:13.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:13 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:24:13.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:13 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:24:13.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:13 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:24:13.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:13 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:24:13.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:13 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:24:13.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:13 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:24:14.379 INFO:tasks.workunit.client.0.smithi120.stdout: 10 197298 20.71 MB/sec execute 369 sec latency 12251.884 ms 2024-09-19T09:24:14.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:14 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:24:14.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:14 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:24:14.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:14 smithi196 ceph-mon[35150]: pgmap v986: 97 pgs: 97 active+clean; 448 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.8 MiB/s wr, 403 op/s 2024-09-19T09:24:14.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:14 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:24:14.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:14 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:24:14.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:14 smithi153 ceph-mon[35071]: pgmap v986: 97 pgs: 97 active+clean; 448 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.8 MiB/s wr, 403 op/s 2024-09-19T09:24:14.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:14 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:24:14.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:14 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:24:14.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:14 smithi120 ceph-mon[31150]: pgmap v986: 97 pgs: 97 active+clean; 448 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.8 MiB/s wr, 403 op/s 2024-09-19T09:24:15.380 INFO:tasks.workunit.client.0.smithi120.stdout: 10 197298 20.65 MB/sec execute 370 sec latency 13252.008 ms 2024-09-19T09:24:16.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:16 smithi120 ceph-mon[31150]: pgmap v987: 97 pgs: 97 active+clean; 447 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 4.6 MiB/s wr, 261 op/s 2024-09-19T09:24:16.380 INFO:tasks.workunit.client.0.smithi120.stdout: 10 197298 20.60 MB/sec execute 371 sec latency 14252.084 ms 2024-09-19T09:24:16.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:16 smithi196 ceph-mon[35150]: pgmap v987: 97 pgs: 97 active+clean; 447 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 4.6 MiB/s wr, 261 op/s 2024-09-19T09:24:16.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:16 smithi153 ceph-mon[35071]: pgmap v987: 97 pgs: 97 active+clean; 447 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 4.6 MiB/s wr, 261 op/s 2024-09-19T09:24:17.380 INFO:tasks.workunit.client.0.smithi120.stdout: 10 197298 20.54 MB/sec execute 372 sec latency 15252.168 ms 2024-09-19T09:24:18.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:18 smithi120 ceph-mon[31150]: pgmap v988: 97 pgs: 97 active+clean; 447 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 4.6 MiB/s wr, 292 op/s 2024-09-19T09:24:18.380 INFO:tasks.workunit.client.0.smithi120.stdout: 10 197298 20.49 MB/sec execute 373 sec latency 16252.219 ms 2024-09-19T09:24:18.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:18 smithi196 ceph-mon[35150]: pgmap v988: 97 pgs: 97 active+clean; 447 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 4.6 MiB/s wr, 292 op/s 2024-09-19T09:24:18.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:18 smithi153 ceph-mon[35071]: pgmap v988: 97 pgs: 97 active+clean; 447 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 4.6 MiB/s wr, 292 op/s 2024-09-19T09:24:19.380 INFO:tasks.workunit.client.0.smithi120.stdout: 10 197298 20.43 MB/sec execute 374 sec latency 17252.307 ms 2024-09-19T09:24:20.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:20 smithi120 ceph-mon[31150]: pgmap v989: 97 pgs: 97 active+clean; 453 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 4.7 MiB/s wr, 313 op/s 2024-09-19T09:24:20.380 INFO:tasks.workunit.client.0.smithi120.stdout: 10 197298 20.38 MB/sec execute 375 sec latency 18252.363 ms 2024-09-19T09:24:20.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:20 smithi196 ceph-mon[35150]: pgmap v989: 97 pgs: 97 active+clean; 453 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 4.7 MiB/s wr, 313 op/s 2024-09-19T09:24:20.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:20 smithi153 ceph-mon[35071]: pgmap v989: 97 pgs: 97 active+clean; 453 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 4.7 MiB/s wr, 313 op/s 2024-09-19T09:24:21.216 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:24:21.380 INFO:tasks.workunit.client.0.smithi120.stdout: 10 197298 20.32 MB/sec execute 376 sec latency 19252.392 ms 2024-09-19T09:24:21.477 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:21.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.952+0000 7ff7fbf83640 1 -- 172.21.15.120:0/4194443080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7f4102ef0 msgr2=0x7ff7f410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:21.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.952+0000 7ff7fbf83640 1 --2- 172.21.15.120:0/4194443080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7f4102ef0 0x7ff7f410da40 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009ca0 tx=0x7ff7ec02a950 comp rx=0 tx=0).stop 2024-09-19T09:24:21.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.952+0000 7ff7fbf83640 1 -- 172.21.15.120:0/4194443080 shutdown_connections 2024-09-19T09:24:21.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.952+0000 7ff7fbf83640 1 --2- 172.21.15.120:0/4194443080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7f4102ef0 0x7ff7f410da40 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:21.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.952+0000 7ff7fbf83640 1 --2- 172.21.15.120:0/4194443080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7f4102570 0x7ff7f41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:21.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.952+0000 7ff7fbf83640 1 --2- 172.21.15.120:0/4194443080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7f4100e20 0x7ff7f4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:21.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.952+0000 7ff7fbf83640 1 -- 172.21.15.120:0/4194443080 >> 172.21.15.120:0/4194443080 conn(0x7ff7f40fc7d0 msgr2=0x7ff7f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:21.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.952+0000 7ff7fbf83640 1 -- 172.21.15.120:0/4194443080 shutdown_connections 2024-09-19T09:24:21.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.952+0000 7ff7fbf83640 1 -- 172.21.15.120:0/4194443080 wait complete. 2024-09-19T09:24:21.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.953+0000 7ff7fbf83640 1 Processor -- start 2024-09-19T09:24:21.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.953+0000 7ff7fbf83640 1 -- start start 2024-09-19T09:24:21.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.953+0000 7ff7fbf83640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7f4100e20 0x7ff7f41992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:21.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.953+0000 7ff7fbf83640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7f4102570 0x7ff7f41997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:21.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.953+0000 7ff7fbf83640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7f4102ef0 0x7ff7f41a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:21.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.953+0000 7ff7fbf83640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f4110fa0 con 0x7ff7f4100e20 2024-09-19T09:24:21.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.953+0000 7ff7fbf83640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f4110e20 con 0x7ff7f4102570 2024-09-19T09:24:21.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.953+0000 7ff7fbf83640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f4111120 con 0x7ff7f4102ef0 2024-09-19T09:24:21.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.953+0000 7ff7f9d1f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7f4100e20 0x7ff7f41992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:21.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.953+0000 7ff7f9d1f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7f4100e20 0x7ff7f41992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52546/0 (socket says 172.21.15.120:52546) 2024-09-19T09:24:21.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.953+0000 7ff7f9d1f640 1 -- 172.21.15.120:0/2325985379 learned_addr learned my addr 172.21.15.120:0/2325985379 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:21.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.953+0000 7ff7f951e640 1 --2- 172.21.15.120:0/2325985379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7f4102570 0x7ff7f41997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:21.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.953+0000 7ff7fa520640 1 --2- 172.21.15.120:0/2325985379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7f4102ef0 0x7ff7f41a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:21.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.953+0000 7ff7f9d1f640 1 -- 172.21.15.120:0/2325985379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7f4102ef0 msgr2=0x7ff7f41a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:21.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.953+0000 7ff7f9d1f640 1 --2- 172.21.15.120:0/2325985379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7f4102ef0 0x7ff7f41a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:21.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.953+0000 7ff7f9d1f640 1 -- 172.21.15.120:0/2325985379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7f4102570 msgr2=0x7ff7f41997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:21.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.953+0000 7ff7f9d1f640 1 --2- 172.21.15.120:0/2325985379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7f4102570 0x7ff7f41997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:21.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.953+0000 7ff7f9d1f640 1 -- 172.21.15.120:0/2325985379 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f41a4860 con 0x7ff7f4100e20 2024-09-19T09:24:21.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.953+0000 7ff7f951e640 1 --2- 172.21.15.120:0/2325985379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7f4102570 0x7ff7f41997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:24:21.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.953+0000 7ff7fa520640 1 --2- 172.21.15.120:0/2325985379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7f4102ef0 0x7ff7f41a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:24:21.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.954+0000 7ff7f9d1f640 1 --2- 172.21.15.120:0/2325985379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7f4100e20 0x7ff7f41992a0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7ff7f000b4a0 tx=0x7ff7f000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:21.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.954+0000 7ff7eaffd640 1 -- 172.21.15.120:0/2325985379 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0014070 con 0x7ff7f4100e20 2024-09-19T09:24:21.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.954+0000 7ff7fbf83640 1 -- 172.21.15.120:0/2325985379 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f41a4a30 con 0x7ff7f4100e20 2024-09-19T09:24:21.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.954+0000 7ff7eaffd640 1 -- 172.21.15.120:0/2325985379 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7f0010af0 con 0x7ff7f4100e20 2024-09-19T09:24:21.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.954+0000 7ff7eaffd640 1 -- 172.21.15.120:0/2325985379 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0017d10 con 0x7ff7f4100e20 2024-09-19T09:24:21.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.954+0000 7ff7fbf83640 1 -- 172.21.15.120:0/2325985379 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f41a4f70 con 0x7ff7f4100e20 2024-09-19T09:24:21.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.955+0000 7ff7fbf83640 1 -- 172.21.15.120:0/2325985379 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7f4101200 con 0x7ff7f4100e20 2024-09-19T09:24:21.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.956+0000 7ff7eaffd640 1 -- 172.21.15.120:0/2325985379 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7ff7f0017070 con 0x7ff7f4100e20 2024-09-19T09:24:21.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.957+0000 7ff7eaffd640 1 --2- 172.21.15.120:0/2325985379 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff7e00a3240 0x7ff7e00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:21.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.957+0000 7ff7f951e640 1 --2- 172.21.15.120:0/2325985379 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff7e00a3240 0x7ff7e00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:21.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.957+0000 7ff7eaffd640 1 -- 172.21.15.120:0/2325985379 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff7f00cf120 con 0x7ff7f4100e20 2024-09-19T09:24:21.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.957+0000 7ff7f951e640 1 --2- 172.21.15.120:0/2325985379 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff7e00a3240 0x7ff7e00a5700 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff7f406aac0 tx=0x7ff7e400a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:21.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:21.960+0000 7ff7eaffd640 1 -- 172.21.15.120:0/2325985379 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff7f009be40 con 0x7ff7f4100e20 2024-09-19T09:24:22.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:22.162+0000 7ff7fbf83640 1 -- 172.21.15.120:0/2325985379 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7f41a5200 con 0x7ff7f4100e20 2024-09-19T09:24:22.164 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:22 smithi120 ceph-mon[31150]: pgmap v990: 97 pgs: 97 active+clean; 452 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 4.0 MiB/s wr, 282 op/s 2024-09-19T09:24:22.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:22.163+0000 7ff7eaffd640 1 -- 172.21.15.120:0/2325985379 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11988 (secure 0 0 0) 0x7ff7f009b740 con 0x7ff7f4100e20 2024-09-19T09:24:22.165 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:24:22.165 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":39,"btime":"2024-09-19T09:18:53:120858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:18:53.070639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:24:22.165 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 39 2024-09-19T09:24:22.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:22.165+0000 7ff7fbf83640 1 -- 172.21.15.120:0/2325985379 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff7e00a3240 msgr2=0x7ff7e00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:22.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:22.165+0000 7ff7fbf83640 1 --2- 172.21.15.120:0/2325985379 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff7e00a3240 0x7ff7e00a5700 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff7f406aac0 tx=0x7ff7e400a5c0 comp rx=0 tx=0).stop 2024-09-19T09:24:22.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:22.165+0000 7ff7fbf83640 1 -- 172.21.15.120:0/2325985379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7f4100e20 msgr2=0x7ff7f41992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:22.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:22.165+0000 7ff7fbf83640 1 --2- 172.21.15.120:0/2325985379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7f4100e20 0x7ff7f41992a0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7ff7f000b4a0 tx=0x7ff7f000b960 comp rx=0 tx=0).stop 2024-09-19T09:24:22.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:22.165+0000 7ff7fbf83640 1 -- 172.21.15.120:0/2325985379 shutdown_connections 2024-09-19T09:24:22.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:22.165+0000 7ff7fbf83640 1 --2- 172.21.15.120:0/2325985379 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff7e00a3240 0x7ff7e00a5700 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:22.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:22.165+0000 7ff7fbf83640 1 --2- 172.21.15.120:0/2325985379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7f4102ef0 0x7ff7f41a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:22.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:22.165+0000 7ff7fbf83640 1 --2- 172.21.15.120:0/2325985379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7f4102570 0x7ff7f41997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:22.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:22.165+0000 7ff7fbf83640 1 --2- 172.21.15.120:0/2325985379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7f4100e20 0x7ff7f41992a0 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:22.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:22.165+0000 7ff7fbf83640 1 -- 172.21.15.120:0/2325985379 >> 172.21.15.120:0/2325985379 conn(0x7ff7f40fc7d0 msgr2=0x7ff7f4106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:22.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:22.165+0000 7ff7fbf83640 1 -- 172.21.15.120:0/2325985379 shutdown_connections 2024-09-19T09:24:22.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:22.165+0000 7ff7fbf83640 1 -- 172.21.15.120:0/2325985379 wait complete. 2024-09-19T09:24:22.380 INFO:tasks.workunit.client.0.smithi120.stdout: 10 197298 20.27 MB/sec execute 377 sec latency 20252.443 ms 2024-09-19T09:24:22.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:22 smithi196 ceph-mon[35150]: pgmap v990: 97 pgs: 97 active+clean; 452 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 4.0 MiB/s wr, 282 op/s 2024-09-19T09:24:22.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:22 smithi153 ceph-mon[35071]: pgmap v990: 97 pgs: 97 active+clean; 452 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 4.0 MiB/s wr, 282 op/s 2024-09-19T09:24:22.917 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db --set-id 1094dda2 --release --await' 2024-09-19T09:24:22.918 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db --set-id 1094dda2 --release --await 2024-09-19T09:24:23.167 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:23.286 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:23 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2325985379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:24:23.380 INFO:tasks.workunit.client.0.smithi120.stdout: 10 197298 20.21 MB/sec execute 378 sec latency 21252.544 ms 2024-09-19T09:24:23.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:23 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2325985379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:24:23.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:23 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2325985379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:24:23.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.685+0000 7f03429fd640 1 -- 172.21.15.120:0/3760132889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f033c102ef0 msgr2=0x7f033c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:23.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.685+0000 7f03429fd640 1 --2- 172.21.15.120:0/3760132889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f033c102ef0 0x7f033c10da40 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f032c009900 tx=0x7f032c02a490 comp rx=0 tx=0).stop 2024-09-19T09:24:23.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.685+0000 7f03429fd640 1 -- 172.21.15.120:0/3760132889 shutdown_connections 2024-09-19T09:24:23.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.685+0000 7f03429fd640 1 --2- 172.21.15.120:0/3760132889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f033c102ef0 0x7f033c10da40 unknown :-1 s=CLOSED pgs=2233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.685+0000 7f03429fd640 1 --2- 172.21.15.120:0/3760132889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f033c102570 0x7f033c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.685+0000 7f03429fd640 1 --2- 172.21.15.120:0/3760132889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f033c100e20 0x7f033c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.685+0000 7f03429fd640 1 -- 172.21.15.120:0/3760132889 >> 172.21.15.120:0/3760132889 conn(0x7f033c0fc7d0 msgr2=0x7f033c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:23.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.685+0000 7f03429fd640 1 -- 172.21.15.120:0/3760132889 shutdown_connections 2024-09-19T09:24:23.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.685+0000 7f03429fd640 1 -- 172.21.15.120:0/3760132889 wait complete. 2024-09-19T09:24:23.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.687+0000 7f03429fd640 1 Processor -- start 2024-09-19T09:24:23.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.687+0000 7f03429fd640 1 -- start start 2024-09-19T09:24:23.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.687+0000 7f03429fd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f033c100e20 0x7f033c1973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:23.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.688+0000 7f03429fd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f033c102570 0x7f033c197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:23.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.688+0000 7f033bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f033c100e20 0x7f033c1973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:23.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.688+0000 7f033bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f033c100e20 0x7f033c1973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52564/0 (socket says 172.21.15.120:52564) 2024-09-19T09:24:23.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.688+0000 7f033bfff640 1 -- 172.21.15.120:0/94797528 learned_addr learned my addr 172.21.15.120:0/94797528 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:23.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.688+0000 7f03429fd640 1 --2- 172.21.15.120:0/94797528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f033c102ef0 0x7f033c19bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:23.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.688+0000 7f03429fd640 1 -- 172.21.15.120:0/94797528 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f033c110f80 con 0x7f033c100e20 2024-09-19T09:24:23.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.688+0000 7f03429fd640 1 -- 172.21.15.120:0/94797528 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f033c110e00 con 0x7f033c102ef0 2024-09-19T09:24:23.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.688+0000 7f03429fd640 1 -- 172.21.15.120:0/94797528 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f033c111100 con 0x7f033c102570 2024-09-19T09:24:23.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.688+0000 7f0340f9a640 1 --2- 172.21.15.120:0/94797528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f033c102ef0 0x7f033c19bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:23.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.688+0000 7f033b7fe640 1 --2- 172.21.15.120:0/94797528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f033c102570 0x7f033c197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:23.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.688+0000 7f0340f9a640 1 -- 172.21.15.120:0/94797528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f033c102570 msgr2=0x7f033c197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:23.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.689+0000 7f0340f9a640 1 --2- 172.21.15.120:0/94797528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f033c102570 0x7f033c197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.689+0000 7f0340f9a640 1 -- 172.21.15.120:0/94797528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f033c100e20 msgr2=0x7f033c1973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:23.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.689+0000 7f0340f9a640 1 --2- 172.21.15.120:0/94797528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f033c100e20 0x7f033c1973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.689+0000 7f0340f9a640 1 -- 172.21.15.120:0/94797528 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f033c19c370 con 0x7f033c102ef0 2024-09-19T09:24:23.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.689+0000 7f0340f9a640 1 --2- 172.21.15.120:0/94797528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f033c102ef0 0x7f033c19bc90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f032c009230 tx=0x7f032c003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:23.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.690+0000 7f03397fa640 1 -- 172.21.15.120:0/94797528 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032c03f070 con 0x7f033c102ef0 2024-09-19T09:24:23.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.690+0000 7f03397fa640 1 -- 172.21.15.120:0/94797528 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f032c031c20 con 0x7f033c102ef0 2024-09-19T09:24:23.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.690+0000 7f03397fa640 1 -- 172.21.15.120:0/94797528 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032c035030 con 0x7f033c102ef0 2024-09-19T09:24:23.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.690+0000 7f03429fd640 1 -- 172.21.15.120:0/94797528 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f033c19c600 con 0x7f033c102ef0 2024-09-19T09:24:23.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.690+0000 7f03429fd640 1 -- 172.21.15.120:0/94797528 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f033c19c9d0 con 0x7f033c102ef0 2024-09-19T09:24:23.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.694+0000 7f0344dd6740 1 Processor -- start 2024-09-19T09:24:23.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.694+0000 7f0344dd6740 1 -- start start 2024-09-19T09:24:23.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.695+0000 7f0344dd6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7f1d62a0 0x555f7f1d6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:23.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.695+0000 7f0344dd6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f7f1d6bc0 0x555f7f1d0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:23.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.695+0000 7f0344dd6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f7f1d1260 0x555f7f1d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:23.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.695+0000 7f0344dd6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555f7f12c620 con 0x555f7f1d1260 2024-09-19T09:24:23.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.695+0000 7f0344dd6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555f7f0d3e50 con 0x555f7f1d6bc0 2024-09-19T09:24:23.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.695+0000 7f0344dd6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555f7f1285e0 con 0x555f7f1d62a0 2024-09-19T09:24:23.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.695+0000 7f03397fa640 1 -- 172.21.15.120:0/94797528 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f032c042070 con 0x7f033c102ef0 2024-09-19T09:24:23.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.695+0000 7f033bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f7f1d6bc0 0x555f7f1d0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:23.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.695+0000 7f033bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f7f1d6bc0 0x555f7f1d0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35588/0 (socket says 172.21.15.120:35588) 2024-09-19T09:24:23.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.695+0000 7f033bfff640 1 -- 172.21.15.120:0/3348792360 learned_addr learned my addr 172.21.15.120:0/3348792360 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:23.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.696+0000 7f033b7fe640 1 --2- 172.21.15.120:0/3348792360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7f1d62a0 0x555f7f1d6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:23.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.696+0000 7f033b7fe640 1 --2- 172.21.15.120:0/3348792360 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f7f1d1260 0x555f7f1d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:23.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.696+0000 7f03397fa640 1 --2- 172.21.15.120:0/94797528 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f03240a3410 0x7f03240a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:23.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.696+0000 7f033bfff640 1 -- 172.21.15.120:0/3348792360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7f1d62a0 msgr2=0x555f7f1d6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:23.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.696+0000 7f033bfff640 1 --2- 172.21.15.120:0/3348792360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7f1d62a0 0x555f7f1d6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.696+0000 7f033bfff640 1 -- 172.21.15.120:0/3348792360 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f7f1d1260 msgr2=0x555f7f1d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:23.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.696+0000 7f033bfff640 1 --2- 172.21.15.120:0/3348792360 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f7f1d1260 0x555f7f1d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.696+0000 7f033bfff640 1 -- 172.21.15.120:0/3348792360 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f7f0c4d40 con 0x555f7f1d6bc0 2024-09-19T09:24:23.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.696+0000 7f0340f9a640 1 --2- 172.21.15.120:0/94797528 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f03240a3410 0x7f03240a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:23.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.697+0000 7f03397fa640 1 -- 172.21.15.120:0/94797528 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f032c0eff50 con 0x7f033c102ef0 2024-09-19T09:24:23.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.696+0000 7f033b7fe640 1 --2- 172.21.15.120:0/3348792360 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f7f1d1260 0x555f7f1d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:24:23.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.697+0000 7f033bfff640 1 --2- 172.21.15.120:0/3348792360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f7f1d6bc0 0x555f7f1d0bf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f033400d370 tx=0x7f033400d830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d87f4c6bb8f06e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:23.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.697+0000 7f03227fc640 1 -- 172.21.15.120:0/3348792360 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0334004b20 con 0x555f7f1d6bc0 2024-09-19T09:24:23.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.697+0000 7f03227fc640 1 -- 172.21.15.120:0/3348792360 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0334004cc0 con 0x555f7f1d6bc0 2024-09-19T09:24:23.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.697+0000 7f03227fc640 1 -- 172.21.15.120:0/3348792360 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0334018810 con 0x555f7f1d6bc0 2024-09-19T09:24:23.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.697+0000 7f0340f9a640 1 --2- 172.21.15.120:0/94797528 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f03240a3410 0x7f03240a58d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f032c0c4c70 tx=0x7f032c036fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:23.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.698+0000 7f0344dd6740 1 -- 172.21.15.120:0/3348792360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f7f1d6bc0 msgr2=0x555f7f1d0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:23.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.698+0000 7f0344dd6740 1 --2- 172.21.15.120:0/3348792360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f7f1d6bc0 0x555f7f1d0bf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f033400d370 tx=0x7f033400d830 comp rx=0 tx=0).stop 2024-09-19T09:24:23.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.699+0000 7f0344dd6740 1 -- 172.21.15.120:0/3348792360 shutdown_connections 2024-09-19T09:24:23.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.699+0000 7f0344dd6740 1 --2- 172.21.15.120:0/3348792360 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f7f1d1260 0x555f7f1d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.699+0000 7f0344dd6740 1 --2- 172.21.15.120:0/3348792360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f7f1d6bc0 0x555f7f1d0bf0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.699+0000 7f0344dd6740 1 --2- 172.21.15.120:0/3348792360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7f1d62a0 0x555f7f1d6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.699+0000 7f0344dd6740 1 -- 172.21.15.120:0/3348792360 >> 172.21.15.120:0/3348792360 conn(0x555f7f1c30a0 msgr2=0x555f7f1c3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:23.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.699+0000 7f0344dd6740 1 -- 172.21.15.120:0/3348792360 shutdown_connections 2024-09-19T09:24:23.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.699+0000 7f0344dd6740 1 -- 172.21.15.120:0/3348792360 wait complete. 2024-09-19T09:24:23.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.699+0000 7f0344dd6740 1 -- start start 2024-09-19T09:24:23.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.700+0000 7f0344dd6740 1 Processor -- start 2024-09-19T09:24:23.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.700+0000 7f0344dd6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f7f1d1260 0x555f7f1da1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:23.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.700+0000 7f0344dd6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f7f1d62a0 0x555f7f1da700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:23.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.700+0000 7f0344dd6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7f1d6bc0 0x555f7f1dac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:23.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.700+0000 7f0344dd6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555f7f1ee5f0 con 0x555f7f1d1260 2024-09-19T09:24:23.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.700+0000 7f0344dd6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555f7f0b4f10 con 0x555f7f1d62a0 2024-09-19T09:24:23.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.700+0000 7f0344dd6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555f7f1ee770 con 0x555f7f1d6bc0 2024-09-19T09:24:23.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.700+0000 7f0344dd6740 10 client.0 _ll_register_callbacks cb 0x555f7f0d8db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:23.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.700+0000 7f033bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f7f1d62a0 0x555f7f1da700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:23.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.700+0000 7f033bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f7f1d62a0 0x555f7f1da700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35602/0 (socket says 172.21.15.120:35602) 2024-09-19T09:24:23.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.700+0000 7f033bfff640 1 -- 172.21.15.120:0/1783285793 learned_addr learned my addr 172.21.15.120:0/1783285793 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:23.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.701+0000 7f033bfff640 1 -- 172.21.15.120:0/1783285793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7f1d6bc0 msgr2=0x555f7f1dac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:23.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.701+0000 7f033bfff640 1 --2- 172.21.15.120:0/1783285793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7f1d6bc0 0x555f7f1dac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.701+0000 7f033bfff640 1 -- 172.21.15.120:0/1783285793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f7f1d1260 msgr2=0x555f7f1da1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:23.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.701+0000 7f033bfff640 1 --2- 172.21.15.120:0/1783285793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f7f1d1260 0x555f7f1da1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.701+0000 7f033bfff640 1 -- 172.21.15.120:0/1783285793 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f7ee3b580 con 0x555f7f1d62a0 2024-09-19T09:24:23.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.701+0000 7f033bfff640 1 --2- 172.21.15.120:0/1783285793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f7f1d62a0 0x555f7f1da700 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f033401a040 tx=0x7f0334018bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ede32b0802b496c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:23.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.701+0000 7f031b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:24:23.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.702+0000 7f031b7fe640 1 -- 172.21.15.120:0/1783285793 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0334009ca0 con 0x555f7f1d62a0 2024-09-19T09:24:23.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.702+0000 7f031b7fe640 1 -- 172.21.15.120:0/1783285793 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f0334018cc0 con 0x555f7f1d62a0 2024-09-19T09:24:23.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.702+0000 7f0344dd6740 1 -- 172.21.15.120:0/1783285793 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0334004cc0 con 0x555f7f1d62a0 2024-09-19T09:24:23.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.702+0000 7f031b7fe640 1 -- 172.21.15.120:0/1783285793 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0334004e70 con 0x555f7f1d62a0 2024-09-19T09:24:23.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.703+0000 7f0344dd6740 10 client.31190 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:23.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.703+0000 7f0344dd6740 1 -- 172.21.15.120:0/1783285793 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555f7ef05960 con 0x555f7f1d62a0 2024-09-19T09:24:23.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.704+0000 7f031b7fe640 1 -- 172.21.15.120:0/1783285793 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0334004cc0 con 0x555f7f1d62a0 2024-09-19T09:24:23.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.705+0000 7f031b7fe640 1 -- 172.21.15.120:0/1783285793 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f03340215a0 con 0x555f7f1d62a0 2024-09-19T09:24:23.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.705+0000 7f0344dd6740 10 client.31190 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:23.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.705+0000 7f0344dd6740 10 client.31190 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:24:23.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.705+0000 7f0344dd6740 20 client.31190 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:23.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.705+0000 7f0344dd6740 1 --2- 172.21.15.120:0/1783285793 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555f7f1fb660 0x555f7f1fda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:23.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.705+0000 7f0344dd6740 4 client.31190 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:24:23.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.705+0000 7f0344dd6740 1 -- 172.21.15.120:0/1783285793 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f7f139b90 con 0x555f7f1fb660 2024-09-19T09:24:23.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.705+0000 7f033b7fe640 1 --2- 172.21.15.120:0/1783285793 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555f7f1fb660 0x555f7f1fda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:23.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.706+0000 7f033b7fe640 1 --2- 172.21.15.120:0/1783285793 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555f7f1fb660 0x555f7f1fda50 crc :-1 s=READY pgs=531 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79baf61fdbff4a5c server_cookie=d05653b4e5cd3783 in_seq=0 out_seq=0 2024-09-19T09:24:23.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.706+0000 7f031b7fe640 10 client.31190 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:23.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.707+0000 7f031b7fe640 1 -- 172.21.15.120:0/1783285793 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f7f139b90 con 0x555f7f1fb660 2024-09-19T09:24:23.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.707+0000 7f031b7fe640 10 client.31190 handle_command_reply: tid=0 2024-09-19T09:24:23.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.707+0000 7f0344dd6740 1 client.31190 shutdown 2024-09-19T09:24:23.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.708+0000 7f0344dd6740 1 -- 172.21.15.120:0/1783285793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f7f1d62a0 msgr2=0x555f7f1da700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:23.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.708+0000 7f0344dd6740 1 --2- 172.21.15.120:0/1783285793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f7f1d62a0 0x555f7f1da700 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f033401a040 tx=0x7f0334018bd0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.709+0000 7f0344dd6740 1 -- 172.21.15.120:0/1783285793 shutdown_connections 2024-09-19T09:24:23.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.709+0000 7f0344dd6740 1 --2- 172.21.15.120:0/1783285793 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555f7f1fb660 0x555f7f1fda50 crc :-1 s=READY pgs=531 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.709+0000 7f0344dd6740 1 --2- 172.21.15.120:0/1783285793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7f1d6bc0 0x555f7f1dac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.709+0000 7f0344dd6740 1 --2- 172.21.15.120:0/1783285793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f7f1d62a0 0x555f7f1da700 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.709+0000 7f0344dd6740 1 --2- 172.21.15.120:0/1783285793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f7f1d1260 0x555f7f1da1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.709+0000 7f0344dd6740 1 -- 172.21.15.120:0/1783285793 >> 172.21.15.120:0/1783285793 conn(0x555f7f1c30a0 msgr2=0x555f7f1c2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:23.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.709+0000 7f031b7fe640 0 client.31190 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:23.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.710+0000 7f0344dd6740 1 -- 172.21.15.120:0/1783285793 shutdown_connections 2024-09-19T09:24:23.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.710+0000 7f0344dd6740 1 -- 172.21.15.120:0/1783285793 wait complete. 2024-09-19T09:24:23.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.710+0000 7f0344dd6740 20 client.31190 trim_cache size 0 max 16384 2024-09-19T09:24:23.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.735+0000 7f0344dd6740 1 Processor -- start 2024-09-19T09:24:23.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.735+0000 7f0344dd6740 1 -- start start 2024-09-19T09:24:23.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.735+0000 7f0344dd6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f7f1d1260 0x555f7f1d0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:23.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.735+0000 7f0344dd6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7f1d62a0 0x555f7f1d9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:23.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.735+0000 7f0344dd6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f7f1d6bc0 0x555f7f2030e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:23.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.735+0000 7f0344dd6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f033c073050 con 0x555f7f1d6bc0 2024-09-19T09:24:23.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.735+0000 7f0344dd6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f033c1a46b0 con 0x555f7f1d1260 2024-09-19T09:24:23.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.735+0000 7f0344dd6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f033c073720 con 0x555f7f1d62a0 2024-09-19T09:24:23.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.735+0000 7f033b7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f7f1d1260 0x555f7f1d0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:23.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.735+0000 7f033b7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f7f1d1260 0x555f7f1d0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35606/0 (socket says 172.21.15.120:35606) 2024-09-19T09:24:23.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.735+0000 7f033b7fe640 1 -- 172.21.15.120:0/4120536577 learned_addr learned my addr 172.21.15.120:0/4120536577 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:23.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.735+0000 7f033bfff640 1 --2- 172.21.15.120:0/4120536577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7f1d62a0 0x555f7f1d9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:23.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.735+0000 7f033b7fe640 1 --2- 172.21.15.120:0/4120536577 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f7f1d6bc0 0x555f7f2030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:23.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.736+0000 7f033b7fe640 1 -- 172.21.15.120:0/4120536577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7f1d62a0 msgr2=0x555f7f1d9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:23.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.736+0000 7f033b7fe640 1 --2- 172.21.15.120:0/4120536577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7f1d62a0 0x555f7f1d9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.736+0000 7f033b7fe640 1 -- 172.21.15.120:0/4120536577 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f7f1d6bc0 msgr2=0x555f7f2030e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:23.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.736+0000 7f033b7fe640 1 --2- 172.21.15.120:0/4120536577 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f7f1d6bc0 0x555f7f2030e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.736+0000 7f033b7fe640 1 -- 172.21.15.120:0/4120536577 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f7f1ee740 con 0x555f7f1d1260 2024-09-19T09:24:23.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.736+0000 7f033b7fe640 1 --2- 172.21.15.120:0/4120536577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f7f1d1260 0x555f7f1d0bf0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x555f7f1c48f0 tx=0x7f0330004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f94cafeece35f9f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:23.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.736+0000 7f0321ffb640 1 -- 172.21.15.120:0/4120536577 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330010980 con 0x555f7f1d1260 2024-09-19T09:24:23.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.736+0000 7f0321ffb640 1 -- 172.21.15.120:0/4120536577 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03300047a0 con 0x555f7f1d1260 2024-09-19T09:24:23.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.736+0000 7f0321ffb640 1 -- 172.21.15.120:0/4120536577 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330005330 con 0x555f7f1d1260 2024-09-19T09:24:23.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.738+0000 7f0344dd6740 1 -- 172.21.15.120:0/4120536577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f7f1d1260 msgr2=0x555f7f1d0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:23.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.738+0000 7f0344dd6740 1 --2- 172.21.15.120:0/4120536577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f7f1d1260 0x555f7f1d0bf0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x555f7f1c48f0 tx=0x7f0330004730 comp rx=0 tx=0).stop 2024-09-19T09:24:23.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.739+0000 7f0344dd6740 1 -- 172.21.15.120:0/4120536577 shutdown_connections 2024-09-19T09:24:23.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.739+0000 7f0344dd6740 1 --2- 172.21.15.120:0/4120536577 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f7f1d6bc0 0x555f7f2030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.739+0000 7f0344dd6740 1 --2- 172.21.15.120:0/4120536577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7f1d62a0 0x555f7f1d9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.739+0000 7f0344dd6740 1 --2- 172.21.15.120:0/4120536577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f7f1d1260 0x555f7f1d0bf0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.739+0000 7f0344dd6740 1 -- 172.21.15.120:0/4120536577 >> 172.21.15.120:0/4120536577 conn(0x555f7f1c30a0 msgr2=0x555f7f1d8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:23.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.739+0000 7f0344dd6740 1 -- 172.21.15.120:0/4120536577 shutdown_connections 2024-09-19T09:24:23.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.739+0000 7f0344dd6740 1 -- 172.21.15.120:0/4120536577 wait complete. 2024-09-19T09:24:23.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.740+0000 7f0344dd6740 1 -- start start 2024-09-19T09:24:23.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.740+0000 7f0344dd6740 1 Processor -- start 2024-09-19T09:24:23.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.740+0000 7f0344dd6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7f1d1260 0x555f7f202e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:23.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.740+0000 7f0344dd6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f7f1d62a0 0x555f7f2033b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:23.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.740+0000 7f0344dd6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f7f1d6bc0 0x555f7f2038f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:23.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.740+0000 7f0344dd6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f033c1a3fe0 con 0x555f7f1d6bc0 2024-09-19T09:24:23.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.740+0000 7f0344dd6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f033c073df0 con 0x555f7f1d62a0 2024-09-19T09:24:23.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.740+0000 7f0344dd6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f033c071010 con 0x555f7f1d1260 2024-09-19T09:24:23.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.740+0000 7f0344dd6740 10 client.0 _ll_register_callbacks cb 0x7f0334018a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:23.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.741+0000 7f033bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f7f1d62a0 0x555f7f2033b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:23.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.741+0000 7f033bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f7f1d62a0 0x555f7f2033b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35618/0 (socket says 172.21.15.120:35618) 2024-09-19T09:24:23.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.741+0000 7f033bfff640 1 -- 172.21.15.120:0/741368450 learned_addr learned my addr 172.21.15.120:0/741368450 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:23.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.741+0000 7f033bfff640 1 -- 172.21.15.120:0/741368450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7f1d1260 msgr2=0x555f7f202e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:23.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.741+0000 7f033bfff640 1 --2- 172.21.15.120:0/741368450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7f1d1260 0x555f7f202e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.741+0000 7f033bfff640 1 -- 172.21.15.120:0/741368450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f7f1d6bc0 msgr2=0x555f7f2038f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:23.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.741+0000 7f033bfff640 1 --2- 172.21.15.120:0/741368450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f7f1d6bc0 0x555f7f2038f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.741+0000 7f033bfff640 1 -- 172.21.15.120:0/741368450 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f7f265260 con 0x555f7f1d62a0 2024-09-19T09:24:23.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.741+0000 7f033bfff640 1 --2- 172.21.15.120:0/741368450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f7f1d62a0 0x555f7f2033b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x555f7f1db800 tx=0x7f0334020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd924cad498a145c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:23.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.741+0000 7f031b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:24:23.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.742+0000 7f031b7fe640 1 -- 172.21.15.120:0/741368450 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03340188a0 con 0x555f7f1d62a0 2024-09-19T09:24:23.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.742+0000 7f031b7fe640 1 -- 172.21.15.120:0/741368450 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f0334013070 con 0x555f7f1d62a0 2024-09-19T09:24:23.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.742+0000 7f0344dd6740 1 -- 172.21.15.120:0/741368450 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03300047a0 con 0x555f7f1d62a0 2024-09-19T09:24:23.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.742+0000 7f031b7fe640 1 -- 172.21.15.120:0/741368450 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0334021690 con 0x555f7f1d62a0 2024-09-19T09:24:23.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.742+0000 7f0344dd6740 10 client.31196 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:23.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.742+0000 7f0344dd6740 1 -- 172.21.15.120:0/741368450 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555f7f266560 con 0x555f7f1d62a0 2024-09-19T09:24:23.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.743+0000 7f031b7fe640 1 -- 172.21.15.120:0/741368450 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03300047a0 con 0x555f7f1d62a0 2024-09-19T09:24:23.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.743+0000 7f031b7fe640 1 -- 172.21.15.120:0/741368450 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f03340057f0 con 0x555f7f1d62a0 2024-09-19T09:24:23.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.743+0000 7f0344dd6740 10 client.31196 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:23.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.743+0000 7f0344dd6740 10 client.31196 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:24:23.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.743+0000 7f0344dd6740 20 client.31196 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:23.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.743+0000 7f0344dd6740 1 --2- 172.21.15.120:0/741368450 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555f7f266cb0 0x555f7f2690d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:23.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.743+0000 7f0344dd6740 4 client.31196 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1094dda2", "release": true, "await": true}] 2024-09-19T09:24:23.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.743+0000 7f0344dd6740 1 -- 172.21.15.120:0/741368450 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1094dda2", "release": true, "await": true}) -- 0x555f7f1340a0 con 0x555f7f266cb0 2024-09-19T09:24:23.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.744+0000 7f033b7fe640 1 --2- 172.21.15.120:0/741368450 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555f7f266cb0 0x555f7f2690d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:23.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.745+0000 7f031b7fe640 10 client.31196 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:23.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.745+0000 7f033b7fe640 1 --2- 172.21.15.120:0/741368450 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555f7f266cb0 0x555f7f2690d0 crc :-1 s=READY pgs=532 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c602c804a93b1f0 server_cookie=d0cfa4311d0defd7 in_seq=0 out_seq=0 2024-09-19T09:24:23.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.842+0000 7f031b7fe640 1 -- 172.21.15.120:0/741368450 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555f7f1340a0 con 0x555f7f266cb0 2024-09-19T09:24:23.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.842+0000 7f031b7fe640 10 client.31196 handle_command_reply: tid=0 2024-09-19T09:24:23.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.843+0000 7f0344dd6740 1 client.31196 shutdown 2024-09-19T09:24:23.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.843+0000 7f0344dd6740 1 -- 172.21.15.120:0/741368450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f7f1d62a0 msgr2=0x555f7f2033b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:23.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.843+0000 7f0344dd6740 1 --2- 172.21.15.120:0/741368450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f7f1d62a0 0x555f7f2033b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x555f7f1db800 tx=0x7f0334020000 comp rx=0 tx=0).stop 2024-09-19T09:24:23.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.843+0000 7f0344dd6740 1 -- 172.21.15.120:0/741368450 shutdown_connections 2024-09-19T09:24:23.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.843+0000 7f0344dd6740 1 --2- 172.21.15.120:0/741368450 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x555f7f266cb0 0x555f7f2690d0 crc :-1 s=READY pgs=532 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.843+0000 7f0344dd6740 1 --2- 172.21.15.120:0/741368450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f7f1d6bc0 0x555f7f2038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.843+0000 7f031b7fe640 0 client.31196 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:23.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.843+0000 7f0344dd6740 1 --2- 172.21.15.120:0/741368450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f7f1d62a0 0x555f7f2033b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.843+0000 7f0344dd6740 1 --2- 172.21.15.120:0/741368450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7f1d1260 0x555f7f202e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.843+0000 7f0344dd6740 1 -- 172.21.15.120:0/741368450 >> 172.21.15.120:0/741368450 conn(0x555f7f1c30a0 msgr2=0x555f7f1c2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:23.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.843+0000 7f0344dd6740 1 -- 172.21.15.120:0/741368450 shutdown_connections 2024-09-19T09:24:23.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.843+0000 7f0344dd6740 1 -- 172.21.15.120:0/741368450 wait complete. 2024-09-19T09:24:23.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.843+0000 7f0344dd6740 20 client.31196 trim_cache size 0 max 16384 2024-09-19T09:24:23.846 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:24:23.846 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 39, 2024-09-19T09:24:23.846 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:24:23.846 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 72, 2024-09-19T09:24:23.846 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:24:23.846 INFO:teuthology.orchestra.run.smithi120.stdout: "1094dda2": { 2024-09-19T09:24:23.846 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 72, 2024-09-19T09:24:23.846 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:24:23.846 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:24:23.847 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:24:23.847 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:24:23.847 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:24:23.847 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:24:23.847 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 144.5, 2024-09-19T09:24:23.847 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:24:23.847 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:24:23.847 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:24:23.847 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:24:23.847 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:24:23.847 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:24:23.847 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:24:23.847 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:24:23.847 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:24:23.848 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:24:23.848 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:24:23.848 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:24:23.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.844+0000 7f03429fd640 1 -- 172.21.15.120:0/94797528 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f03240a3410 msgr2=0x7f03240a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:23.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.844+0000 7f03429fd640 1 --2- 172.21.15.120:0/94797528 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f03240a3410 0x7f03240a58d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f032c0c4c70 tx=0x7f032c036fd0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.844+0000 7f03429fd640 1 -- 172.21.15.120:0/94797528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f033c102ef0 msgr2=0x7f033c19bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:23.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.844+0000 7f03429fd640 1 --2- 172.21.15.120:0/94797528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f033c102ef0 0x7f033c19bc90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f032c009230 tx=0x7f032c003b90 comp rx=0 tx=0).stop 2024-09-19T09:24:23.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.844+0000 7f03429fd640 1 -- 172.21.15.120:0/94797528 shutdown_connections 2024-09-19T09:24:23.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.844+0000 7f03429fd640 1 --2- 172.21.15.120:0/94797528 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f03240a3410 0x7f03240a58d0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.844+0000 7f03429fd640 1 --2- 172.21.15.120:0/94797528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f033c102ef0 0x7f033c19bc90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.844+0000 7f03429fd640 1 --2- 172.21.15.120:0/94797528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f033c102570 0x7f033c197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.844+0000 7f03429fd640 1 --2- 172.21.15.120:0/94797528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f033c100e20 0x7f033c1973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:23.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.844+0000 7f03429fd640 1 -- 172.21.15.120:0/94797528 >> 172.21.15.120:0/94797528 conn(0x7f033c0fc7d0 msgr2=0x7f033c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:23.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.844+0000 7f03429fd640 1 -- 172.21.15.120:0/94797528 shutdown_connections 2024-09-19T09:24:23.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:23.844+0000 7f03429fd640 1 -- 172.21.15.120:0/94797528 wait complete. 2024-09-19T09:24:24.213 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:24 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:24.213 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:24 smithi120 ceph-mon[31150]: pgmap v991: 97 pgs: 97 active+clean; 458 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 4.6 MiB/s wr, 253 op/s 2024-09-19T09:24:24.380 INFO:tasks.workunit.client.0.smithi120.stdout: 10 197605 20.20 MB/sec execute 379 sec latency 21715.085 ms 2024-09-19T09:24:24.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:24 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:24.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:24 smithi196 ceph-mon[35150]: pgmap v991: 97 pgs: 97 active+clean; 458 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 4.6 MiB/s wr, 253 op/s 2024-09-19T09:24:24.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:24 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:24.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:24 smithi153 ceph-mon[35071]: pgmap v991: 97 pgs: 97 active+clean; 458 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 4.6 MiB/s wr, 253 op/s 2024-09-19T09:24:24.593 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1094dda2', total seconds elapsed: 27.9 2024-09-19T09:24:24.593 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-09-19T09:24:24.594 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:24:24.883 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:25.311 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:25 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:24:25.311 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:25 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:24:25.311 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:25 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:24:25.311 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:25 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:24:25.312 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:25 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:24:25.312 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:25 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:24:25.312 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:25 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:24:25.312 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:25 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:24:25.312 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:25 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:25.312 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:25 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:25.312 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:25 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:24:25.380 INFO:tasks.workunit.client.0.smithi120.stdout: 10 198079 20.23 MB/sec execute 380 sec latency 217.790 ms 2024-09-19T09:24:25.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.403+0000 7fd52313d640 1 -- 172.21.15.120:0/1134482868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd51c101df0 msgr2=0x7fd51c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:25.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.403+0000 7fd52313d640 1 --2- 172.21.15.120:0/1134482868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd51c101df0 0x7fd51c1021d0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7fd510009900 tx=0x7fd51002a410 comp rx=0 tx=0).stop 2024-09-19T09:24:25.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.403+0000 7fd52313d640 1 -- 172.21.15.120:0/1134482868 shutdown_connections 2024-09-19T09:24:25.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.403+0000 7fd52313d640 1 --2- 172.21.15.120:0/1134482868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd51c106f20 0x7fd51c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:25.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.403+0000 7fd52313d640 1 --2- 172.21.15.120:0/1134482868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd51c1027a0 0x7fd51c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:25.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.403+0000 7fd52313d640 1 --2- 172.21.15.120:0/1134482868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd51c101df0 0x7fd51c1021d0 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:25.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.403+0000 7fd52313d640 1 -- 172.21.15.120:0/1134482868 >> 172.21.15.120:0/1134482868 conn(0x7fd51c0fd8f0 msgr2=0x7fd51c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:25.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.403+0000 7fd52313d640 1 -- 172.21.15.120:0/1134482868 shutdown_connections 2024-09-19T09:24:25.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.403+0000 7fd52313d640 1 -- 172.21.15.120:0/1134482868 wait complete. 2024-09-19T09:24:25.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.403+0000 7fd52313d640 1 Processor -- start 2024-09-19T09:24:25.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.403+0000 7fd52313d640 1 -- start start 2024-09-19T09:24:25.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.403+0000 7fd52313d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd51c101df0 0x7fd51c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:25.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.404+0000 7fd52313d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd51c1027a0 0x7fd51c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:25.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.404+0000 7fd52313d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd51c106f20 0x7fd51c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:25.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.404+0000 7fd52313d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c111f80 con 0x7fd51c101df0 2024-09-19T09:24:25.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.404+0000 7fd52313d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c111e00 con 0x7fd51c106f20 2024-09-19T09:24:25.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.404+0000 7fd52313d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c112100 con 0x7fd51c1027a0 2024-09-19T09:24:25.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.404+0000 7fd520ed9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd51c101df0 0x7fd51c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:25.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.404+0000 7fd520ed9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd51c101df0 0x7fd51c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52620/0 (socket says 172.21.15.120:52620) 2024-09-19T09:24:25.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.404+0000 7fd520ed9640 1 -- 172.21.15.120:0/3052456541 learned_addr learned my addr 172.21.15.120:0/3052456541 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:25.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.404+0000 7fd51bfff640 1 --2- 172.21.15.120:0/3052456541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd51c1027a0 0x7fd51c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:25.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.404+0000 7fd5216da640 1 --2- 172.21.15.120:0/3052456541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd51c106f20 0x7fd51c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:25.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.404+0000 7fd520ed9640 1 -- 172.21.15.120:0/3052456541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd51c1027a0 msgr2=0x7fd51c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:25.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.404+0000 7fd520ed9640 1 --2- 172.21.15.120:0/3052456541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd51c1027a0 0x7fd51c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:25.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.404+0000 7fd520ed9640 1 -- 172.21.15.120:0/3052456541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd51c106f20 msgr2=0x7fd51c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:25.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.404+0000 7fd520ed9640 1 --2- 172.21.15.120:0/3052456541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd51c106f20 0x7fd51c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:25.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.404+0000 7fd520ed9640 1 -- 172.21.15.120:0/3052456541 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51c1a6980 con 0x7fd51c101df0 2024-09-19T09:24:25.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.404+0000 7fd520ed9640 1 --2- 172.21.15.120:0/3052456541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd51c101df0 0x7fd51c0744f0 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7fd510006010 tx=0x7fd510004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:25.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.404+0000 7fd519ffb640 1 -- 172.21.15.120:0/3052456541 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51003f030 con 0x7fd51c101df0 2024-09-19T09:24:25.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.404+0000 7fd519ffb640 1 -- 172.21.15.120:0/3052456541 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd510004440 con 0x7fd51c101df0 2024-09-19T09:24:25.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.404+0000 7fd519ffb640 1 -- 172.21.15.120:0/3052456541 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd510035070 con 0x7fd51c101df0 2024-09-19T09:24:25.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.404+0000 7fd52313d640 1 -- 172.21.15.120:0/3052456541 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd51c1a6c10 con 0x7fd51c101df0 2024-09-19T09:24:25.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.405+0000 7fd52313d640 1 -- 172.21.15.120:0/3052456541 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd51c1a70a0 con 0x7fd51c101df0 2024-09-19T09:24:25.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.405+0000 7fd52313d640 1 -- 172.21.15.120:0/3052456541 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd51c1021d0 con 0x7fd51c101df0 2024-09-19T09:24:25.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.407+0000 7fd519ffb640 1 -- 172.21.15.120:0/3052456541 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd510042030 con 0x7fd51c101df0 2024-09-19T09:24:25.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.407+0000 7fd519ffb640 1 --2- 172.21.15.120:0/3052456541 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd50c0a3240 0x7fd50c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:25.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.407+0000 7fd51bfff640 1 --2- 172.21.15.120:0/3052456541 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd50c0a3240 0x7fd50c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:25.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.407+0000 7fd519ffb640 1 -- 172.21.15.120:0/3052456541 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd5100efd00 con 0x7fd51c101df0 2024-09-19T09:24:25.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.408+0000 7fd51bfff640 1 --2- 172.21.15.120:0/3052456541 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd50c0a3240 0x7fd50c0a5700 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd51c075f80 tx=0x7fd508005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:25.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.411+0000 7fd519ffb640 1 -- 172.21.15.120:0/3052456541 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd5100bca20 con 0x7fd51c101df0 2024-09-19T09:24:25.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:24:25.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:24:25.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:24:25.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:24:25.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:24:25.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:24:25.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:24:25.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:24:25.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:25.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:25.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:24:25.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:25 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:24:25.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:25 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:24:25.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:25 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:24:25.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:25 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:24:25.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:25 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:24:25.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:25 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:24:25.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:25 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:24:25.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:25 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:24:25.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:25 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:25.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:25 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:25.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:25 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:24:25.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.734+0000 7fd52313d640 1 -- 172.21.15.120:0/3052456541 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd51c02d7c0 con 0x7fd51c101df0 2024-09-19T09:24:25.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.735+0000 7fd519ffb640 1 -- 172.21.15.120:0/3052456541 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11988 (secure 0 0 0) 0x7fd5100bc320 con 0x7fd51c101df0 2024-09-19T09:24:25.737 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:24:25.737 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":39,"btime":"2024-09-19T09:18:53:120858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:18:53.070639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:24:25.737 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 39 2024-09-19T09:24:25.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.738+0000 7fd52313d640 1 -- 172.21.15.120:0/3052456541 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd50c0a3240 msgr2=0x7fd50c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:25.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.738+0000 7fd52313d640 1 --2- 172.21.15.120:0/3052456541 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd50c0a3240 0x7fd50c0a5700 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd51c075f80 tx=0x7fd508005f90 comp rx=0 tx=0).stop 2024-09-19T09:24:25.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.738+0000 7fd52313d640 1 -- 172.21.15.120:0/3052456541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd51c101df0 msgr2=0x7fd51c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:25.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.738+0000 7fd52313d640 1 --2- 172.21.15.120:0/3052456541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd51c101df0 0x7fd51c0744f0 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7fd510006010 tx=0x7fd510004040 comp rx=0 tx=0).stop 2024-09-19T09:24:25.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.738+0000 7fd52313d640 1 -- 172.21.15.120:0/3052456541 shutdown_connections 2024-09-19T09:24:25.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.738+0000 7fd52313d640 1 --2- 172.21.15.120:0/3052456541 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd50c0a3240 0x7fd50c0a5700 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:25.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.738+0000 7fd52313d640 1 --2- 172.21.15.120:0/3052456541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd51c106f20 0x7fd51c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:25.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.738+0000 7fd52313d640 1 --2- 172.21.15.120:0/3052456541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd51c1027a0 0x7fd51c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:25.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.738+0000 7fd52313d640 1 --2- 172.21.15.120:0/3052456541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd51c101df0 0x7fd51c0744f0 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:25.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.738+0000 7fd52313d640 1 -- 172.21.15.120:0/3052456541 >> 172.21.15.120:0/3052456541 conn(0x7fd51c0fd8f0 msgr2=0x7fd51c0feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:25.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.738+0000 7fd52313d640 1 -- 172.21.15.120:0/3052456541 shutdown_connections 2024-09-19T09:24:25.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:25.738+0000 7fd52313d640 1 -- 172.21.15.120:0/3052456541 wait complete. 2024-09-19T09:24:26.380 INFO:tasks.workunit.client.0.smithi120.stdout: 10 198704 20.22 MB/sec execute 381 sec latency 323.569 ms 2024-09-19T09:24:26.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:26 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3052456541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:24:26.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:26 smithi196 ceph-mon[35150]: pgmap v992: 97 pgs: 97 active+clean; 458 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 4.1 MiB/s wr, 216 op/s 2024-09-19T09:24:26.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:26 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3052456541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:24:26.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:26 smithi153 ceph-mon[35071]: pgmap v992: 97 pgs: 97 active+clean; 458 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 4.1 MiB/s wr, 216 op/s 2024-09-19T09:24:26.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:26 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3052456541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:24:26.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:26 smithi120 ceph-mon[31150]: pgmap v992: 97 pgs: 97 active+clean; 458 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 4.1 MiB/s wr, 216 op/s 2024-09-19T09:24:26.494 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-09-19T09:24:26.494 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-09-19T09:24:26.501 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.497+0000 7f3a1951a640 1 -- 172.21.15.120:0/3745842988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a14102640 msgr2=0x7f3a14108ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:26.501 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.497+0000 7f3a1951a640 1 --2- 172.21.15.120:0/3745842988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a14102640 0x7f3a14108ed0 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c009be0 tx=0x7f3a0c02a750 comp rx=0 tx=0).stop 2024-09-19T09:24:26.501 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.497+0000 7f3a1951a640 1 -- 172.21.15.120:0/3745842988 shutdown_connections 2024-09-19T09:24:26.501 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.497+0000 7f3a1951a640 1 --2- 172.21.15.120:0/3745842988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a14102640 0x7f3a14108ed0 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.497+0000 7f3a1951a640 1 --2- 172.21.15.120:0/3745842988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a14101c80 0x7f3a14102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.497+0000 7f3a1951a640 1 --2- 172.21.15.120:0/3745842988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a14100f70 0x7f3a14101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.497+0000 7f3a1951a640 1 -- 172.21.15.120:0/3745842988 >> 172.21.15.120:0/3745842988 conn(0x7f3a140fb930 msgr2=0x7f3a140fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.497+0000 7f3a1951a640 1 -- 172.21.15.120:0/3745842988 shutdown_connections 2024-09-19T09:24:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.497+0000 7f3a1951a640 1 -- 172.21.15.120:0/3745842988 wait complete. 2024-09-19T09:24:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.498+0000 7f3a1951a640 1 Processor -- start 2024-09-19T09:24:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.498+0000 7f3a1951a640 1 -- start start 2024-09-19T09:24:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.499+0000 7f3a1951a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a14100f70 0x7f3a14196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.499+0000 7f3a1951a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a14101c80 0x7f3a141971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.499+0000 7f3a1951a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a14102640 0x7f3a1419e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.499+0000 7f3a1951a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1410c680 con 0x7f3a14100f70 2024-09-19T09:24:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.499+0000 7f3a1951a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1410c500 con 0x7f3a14102640 2024-09-19T09:24:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.499+0000 7f3a1951a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1410c800 con 0x7f3a14101c80 2024-09-19T09:24:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.499+0000 7f3a12ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a14100f70 0x7f3a14196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.499+0000 7f3a127fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a14101c80 0x7f3a141971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.499+0000 7f3a12ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a14100f70 0x7f3a14196c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52640/0 (socket says 172.21.15.120:52640) 2024-09-19T09:24:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.499+0000 7f3a12ffd640 1 -- 172.21.15.120:0/1647409184 learned_addr learned my addr 172.21.15.120:0/1647409184 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.499+0000 7f3a137fe640 1 --2- 172.21.15.120:0/1647409184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a14102640 0x7f3a1419e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.499+0000 7f3a127fc640 1 -- 172.21.15.120:0/1647409184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a14102640 msgr2=0x7f3a1419e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.499+0000 7f3a127fc640 1 --2- 172.21.15.120:0/1647409184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a14102640 0x7f3a1419e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.499+0000 7f3a127fc640 1 -- 172.21.15.120:0/1647409184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a14100f70 msgr2=0x7f3a14196c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.499+0000 7f3a127fc640 1 --2- 172.21.15.120:0/1647409184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a14100f70 0x7f3a14196c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.499+0000 7f3a127fc640 1 -- 172.21.15.120:0/1647409184 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a1419e8c0 con 0x7f3a14101c80 2024-09-19T09:24:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.500+0000 7f3a127fc640 1 --2- 172.21.15.120:0/1647409184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a14101c80 0x7f3a141971c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3a0800e790 tx=0x7f3a0800ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.500+0000 7f39fbfff640 1 -- 172.21.15.120:0/1647409184 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0801e070 con 0x7f3a14101c80 2024-09-19T09:24:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.500+0000 7f3a1951a640 1 -- 172.21.15.120:0/1647409184 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a140ffb20 con 0x7f3a14101c80 2024-09-19T09:24:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.500+0000 7f3a1951a640 1 -- 172.21.15.120:0/1647409184 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a141000c0 con 0x7f3a14101c80 2024-09-19T09:24:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.500+0000 7f39fbfff640 1 -- 172.21.15.120:0/1647409184 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a0800ccc0 con 0x7f3a14101c80 2024-09-19T09:24:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.500+0000 7f39fbfff640 1 -- 172.21.15.120:0/1647409184 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a080197a0 con 0x7f3a14101c80 2024-09-19T09:24:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.503+0000 7f3a1c2ae740 1 Processor -- start 2024-09-19T09:24:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.503+0000 7f3a1c2ae740 1 -- start start 2024-09-19T09:24:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.503+0000 7f3a1c2ae740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d461fdd920 0x55d461fddd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.503+0000 7f3a1c2ae740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d461fde260 0x55d461fe0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.503+0000 7f3a1c2ae740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d461fe1330 0x55d461fee2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.503+0000 7f3a1c2ae740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d461f412f0 con 0x55d461fdd920 2024-09-19T09:24:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.503+0000 7f3a1c2ae740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d461ee9270 con 0x55d461fe1330 2024-09-19T09:24:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.503+0000 7f3a1c2ae740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d461f3d2b0 con 0x55d461fde260 2024-09-19T09:24:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.503+0000 7f39fbfff640 1 -- 172.21.15.120:0/1647409184 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f3a08021070 con 0x7f3a14101c80 2024-09-19T09:24:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.504+0000 7f3a137fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d461fdd920 0x55d461fddd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.504+0000 7f3a12ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d461fde260 0x55d461fe0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.504+0000 7f3a137fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d461fdd920 0x55d461fddd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52646/0 (socket says 172.21.15.120:52646) 2024-09-19T09:24:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.504+0000 7f3a137fe640 1 -- 172.21.15.120:0/2455388878 learned_addr learned my addr 172.21.15.120:0/2455388878 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.504+0000 7f3a127fc640 1 --2- 172.21.15.120:0/2455388878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d461fe1330 0x55d461fee2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.504+0000 7f3a12ffd640 1 -- 172.21.15.120:0/2455388878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d461fe1330 msgr2=0x55d461fee2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.504+0000 7f3a12ffd640 1 --2- 172.21.15.120:0/2455388878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d461fe1330 0x55d461fee2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.504+0000 7f3a12ffd640 1 -- 172.21.15.120:0/2455388878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d461fdd920 msgr2=0x55d461fddd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.504+0000 7f3a12ffd640 1 --2- 172.21.15.120:0/2455388878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d461fdd920 0x55d461fddd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.504+0000 7f3a12ffd640 1 -- 172.21.15.120:0/2455388878 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d461ee8340 con 0x55d461fde260 2024-09-19T09:24:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.504+0000 7f3a12ffd640 1 --2- 172.21.15.120:0/2455388878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d461fde260 0x55d461fe0d10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3a040058c0 tx=0x7f3a04011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bca38db038de50e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.505+0000 7f39f97fa640 1 -- 172.21.15.120:0/2455388878 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0401e070 con 0x55d461fde260 2024-09-19T09:24:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.505+0000 7f39fbfff640 1 --2- 172.21.15.120:0/1647409184 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f39fc0a33f0 0x7f39fc0a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.505+0000 7f39f97fa640 1 -- 172.21.15.120:0/2455388878 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a04009e50 con 0x55d461fde260 2024-09-19T09:24:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.505+0000 7f39f97fa640 1 -- 172.21.15.120:0/2455388878 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a04018810 con 0x55d461fde260 2024-09-19T09:24:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.505+0000 7f3a137fe640 1 --2- 172.21.15.120:0/1647409184 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f39fc0a33f0 0x7f39fc0a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.505+0000 7f39fbfff640 1 -- 172.21.15.120:0/1647409184 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f3a080cf500 con 0x7f3a14101c80 2024-09-19T09:24:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.505+0000 7f3a137fe640 1 --2- 172.21.15.120:0/1647409184 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f39fc0a33f0 0x7f39fc0a58b0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0099a0 tx=0x7f3a0c005c10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.505+0000 7f3a1c2ae740 1 -- 172.21.15.120:0/2455388878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d461fde260 msgr2=0x55d461fe0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.506+0000 7f3a1c2ae740 1 --2- 172.21.15.120:0/2455388878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d461fde260 0x55d461fe0d10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3a040058c0 tx=0x7f3a04011810 comp rx=0 tx=0).stop 2024-09-19T09:24:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.506+0000 7f3a1c2ae740 1 -- 172.21.15.120:0/2455388878 shutdown_connections 2024-09-19T09:24:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.506+0000 7f3a1c2ae740 1 --2- 172.21.15.120:0/2455388878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d461fe1330 0x55d461fee2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.506+0000 7f3a1c2ae740 1 --2- 172.21.15.120:0/2455388878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d461fde260 0x55d461fe0d10 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.506+0000 7f3a1c2ae740 1 --2- 172.21.15.120:0/2455388878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d461fdd920 0x55d461fddd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.506+0000 7f3a1c2ae740 1 -- 172.21.15.120:0/2455388878 >> 172.21.15.120:0/2455388878 conn(0x55d461fd9010 msgr2=0x55d461fd9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.506+0000 7f3a1c2ae740 1 -- 172.21.15.120:0/2455388878 shutdown_connections 2024-09-19T09:24:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.506+0000 7f3a1c2ae740 1 -- 172.21.15.120:0/2455388878 wait complete. 2024-09-19T09:24:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.506+0000 7f3a1c2ae740 1 -- start start 2024-09-19T09:24:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.506+0000 7f3a1c2ae740 1 Processor -- start 2024-09-19T09:24:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.506+0000 7f3a1c2ae740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d461fdd920 0x55d461ff0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.506+0000 7f3a1c2ae740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d461fde260 0x55d461ff0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.506+0000 7f3a1c2ae740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d461fe1330 0x55d461ff7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.506+0000 7f3a1c2ae740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d461feeb70 con 0x55d461fdd920 2024-09-19T09:24:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.507+0000 7f3a1c2ae740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d461eca290 con 0x55d461fe1330 2024-09-19T09:24:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.507+0000 7f3a1c2ae740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d461feecf0 con 0x55d461fde260 2024-09-19T09:24:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.507+0000 7f3a12ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d461fdd920 0x55d461ff0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.507+0000 7f3a1c2ae740 10 client.0 _ll_register_callbacks cb 0x55d461efb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.507+0000 7f3a12ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d461fdd920 0x55d461ff0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52656/0 (socket says 172.21.15.120:52656) 2024-09-19T09:24:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.507+0000 7f3a12ffd640 1 -- 172.21.15.120:0/4091679884 learned_addr learned my addr 172.21.15.120:0/4091679884 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.507+0000 7f3a127fc640 1 --2- 172.21.15.120:0/4091679884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d461fde260 0x55d461ff0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.507+0000 7f3a137fe640 1 --2- 172.21.15.120:0/4091679884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d461fe1330 0x55d461ff7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.507+0000 7f3a12ffd640 1 -- 172.21.15.120:0/4091679884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d461fde260 msgr2=0x55d461ff0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.507+0000 7f3a12ffd640 1 --2- 172.21.15.120:0/4091679884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d461fde260 0x55d461ff0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.507+0000 7f3a12ffd640 1 -- 172.21.15.120:0/4091679884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d461fe1330 msgr2=0x55d461ff7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.507+0000 7f3a12ffd640 1 --2- 172.21.15.120:0/4091679884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d461fe1330 0x55d461ff7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.507+0000 7f3a12ffd640 1 -- 172.21.15.120:0/4091679884 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d461c54210 con 0x55d461fdd920 2024-09-19T09:24:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.507+0000 7f3a127fc640 1 --2- 172.21.15.120:0/4091679884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d461fde260 0x55d461ff0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:24:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.507+0000 7f3a12ffd640 1 --2- 172.21.15.120:0/4091679884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d461fdd920 0x55d461ff0400 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f3a04019d50 tx=0x7f3a040056c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffafc56f5adaf923 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.507+0000 7f39f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.508+0000 7f39f6ff5640 1 -- 172.21.15.120:0/4091679884 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a04028020 con 0x55d461fdd920 2024-09-19T09:24:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.508+0000 7f3a1c2ae740 1 -- 172.21.15.120:0/4091679884 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a04009e50 con 0x55d461fdd920 2024-09-19T09:24:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.508+0000 7f39f6ff5640 1 -- 172.21.15.120:0/4091679884 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f3a04019410 con 0x55d461fdd920 2024-09-19T09:24:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.508+0000 7f39f6ff5640 1 -- 172.21.15.120:0/4091679884 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3a0401e040 con 0x55d461fdd920 2024-09-19T09:24:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.508+0000 7f3a1c2ae740 10 client.22479 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.508+0000 7f3a1c2ae740 1 -- 172.21.15.120:0/4091679884 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d461d20620 con 0x55d461fdd920 2024-09-19T09:24:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.508+0000 7f39f6ff5640 1 -- 172.21.15.120:0/4091679884 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a04009e50 con 0x55d461fdd920 2024-09-19T09:24:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.508+0000 7f39f6ff5640 1 -- 172.21.15.120:0/4091679884 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f3a04021030 con 0x55d461fdd920 2024-09-19T09:24:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.508+0000 7f3a1c2ae740 10 client.22479 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.508+0000 7f3a1c2ae740 10 client.22479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:24:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.508+0000 7f3a1c2ae740 20 client.22479 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.508+0000 7f3a1c2ae740 1 --2- 172.21.15.120:0/4091679884 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d461ffbd90 0x55d461ffe200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.508+0000 7f3a1c2ae740 4 client.22479 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:24:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.508+0000 7f3a1c2ae740 1 -- 172.21.15.120:0/4091679884 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d461f4ea20 con 0x55d461ffbd90 2024-09-19T09:24:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.509+0000 7f3a127fc640 1 --2- 172.21.15.120:0/4091679884 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d461ffbd90 0x55d461ffe200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.509+0000 7f3a127fc640 1 --2- 172.21.15.120:0/4091679884 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d461ffbd90 0x55d461ffe200 crc :-1 s=READY pgs=533 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=681dc27c53215ee2 server_cookie=82b7d7c15c8e726a in_seq=0 out_seq=0 2024-09-19T09:24:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.509+0000 7f39f6ff5640 10 client.22479 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.510+0000 7f39f6ff5640 1 -- 172.21.15.120:0/4091679884 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d461f4ea20 con 0x55d461ffbd90 2024-09-19T09:24:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.510+0000 7f39f6ff5640 10 client.22479 handle_command_reply: tid=0 2024-09-19T09:24:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.510+0000 7f3a1c2ae740 1 client.22479 shutdown 2024-09-19T09:24:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.510+0000 7f3a1c2ae740 1 -- 172.21.15.120:0/4091679884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d461fdd920 msgr2=0x55d461ff0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.510+0000 7f3a1c2ae740 1 --2- 172.21.15.120:0/4091679884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d461fdd920 0x55d461ff0400 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f3a04019d50 tx=0x7f3a040056c0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.511+0000 7f3a1c2ae740 1 -- 172.21.15.120:0/4091679884 shutdown_connections 2024-09-19T09:24:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.511+0000 7f3a1c2ae740 1 --2- 172.21.15.120:0/4091679884 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d461ffbd90 0x55d461ffe200 crc :-1 s=READY pgs=533 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.511+0000 7f39f6ff5640 0 client.22479 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.511+0000 7f3a1c2ae740 1 --2- 172.21.15.120:0/4091679884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d461fe1330 0x55d461ff7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.511+0000 7f3a1c2ae740 1 --2- 172.21.15.120:0/4091679884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d461fde260 0x55d461ff0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.511+0000 7f3a1c2ae740 1 --2- 172.21.15.120:0/4091679884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d461fdd920 0x55d461ff0400 unknown :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.511+0000 7f3a1c2ae740 1 -- 172.21.15.120:0/4091679884 >> 172.21.15.120:0/4091679884 conn(0x55d461fd9010 msgr2=0x55d461fda820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.511+0000 7f3a1c2ae740 1 -- 172.21.15.120:0/4091679884 shutdown_connections 2024-09-19T09:24:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.511+0000 7f3a1c2ae740 1 -- 172.21.15.120:0/4091679884 wait complete. 2024-09-19T09:24:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.511+0000 7f3a1c2ae740 20 client.22479 trim_cache size 0 max 16384 2024-09-19T09:24:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.529+0000 7f3a1c2ae740 1 Processor -- start 2024-09-19T09:24:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.529+0000 7f3a1c2ae740 1 -- start start 2024-09-19T09:24:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.529+0000 7f3a1c2ae740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d461fdd920 0x55d461ff0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.529+0000 7f3a1c2ae740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d461fde260 0x55d461ff7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.529+0000 7f3a1c2ae740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d461fe1330 0x55d461fda400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.529+0000 7f3a1c2ae740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0000e4d0 con 0x55d461fdd920 2024-09-19T09:24:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.529+0000 7f3a1c2ae740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3a00010010 con 0x55d461fde260 2024-09-19T09:24:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.529+0000 7f3a1c2ae740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0000eba0 con 0x55d461fe1330 2024-09-19T09:24:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.529+0000 7f3a12ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d461fdd920 0x55d461ff0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.529+0000 7f3a12ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d461fdd920 0x55d461ff0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52660/0 (socket says 172.21.15.120:52660) 2024-09-19T09:24:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.529+0000 7f3a12ffd640 1 -- 172.21.15.120:0/1625854803 learned_addr learned my addr 172.21.15.120:0/1625854803 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.529+0000 7f3a127fc640 1 --2- 172.21.15.120:0/1625854803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d461fde260 0x55d461ff7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.529+0000 7f3a137fe640 1 --2- 172.21.15.120:0/1625854803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d461fe1330 0x55d461fda400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.529+0000 7f3a12ffd640 1 -- 172.21.15.120:0/1625854803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d461fe1330 msgr2=0x55d461fda400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.529+0000 7f3a12ffd640 1 --2- 172.21.15.120:0/1625854803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d461fe1330 0x55d461fda400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.529+0000 7f3a12ffd640 1 -- 172.21.15.120:0/1625854803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d461fde260 msgr2=0x55d461ff7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.529+0000 7f3a12ffd640 1 --2- 172.21.15.120:0/1625854803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d461fde260 0x55d461ff7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.529+0000 7f3a12ffd640 1 -- 172.21.15.120:0/1625854803 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d461fd8e30 con 0x55d461fdd920 2024-09-19T09:24:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.529+0000 7f3a12ffd640 1 --2- 172.21.15.120:0/1625854803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d461fdd920 0x55d461ff0820 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f3a040058c0 tx=0x7f3a04009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943209d9718388ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.530+0000 7f39f8ff9640 1 -- 172.21.15.120:0/1625854803 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a04028040 con 0x55d461fdd920 2024-09-19T09:24:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.530+0000 7f39f8ff9640 1 -- 172.21.15.120:0/1625854803 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a04009e50 con 0x55d461fdd920 2024-09-19T09:24:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.530+0000 7f39f8ff9640 1 -- 172.21.15.120:0/1625854803 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a04018c30 con 0x55d461fdd920 2024-09-19T09:24:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.530+0000 7f3a1c2ae740 1 -- 172.21.15.120:0/1625854803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d461fdd920 msgr2=0x55d461ff0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.530+0000 7f3a1c2ae740 1 --2- 172.21.15.120:0/1625854803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d461fdd920 0x55d461ff0820 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f3a040058c0 tx=0x7f3a04009b50 comp rx=0 tx=0).stop 2024-09-19T09:24:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.531+0000 7f3a1c2ae740 1 -- 172.21.15.120:0/1625854803 shutdown_connections 2024-09-19T09:24:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.531+0000 7f3a1c2ae740 1 --2- 172.21.15.120:0/1625854803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d461fe1330 0x55d461fda400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.531+0000 7f3a1c2ae740 1 --2- 172.21.15.120:0/1625854803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d461fde260 0x55d461ff7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.531+0000 7f3a1c2ae740 1 --2- 172.21.15.120:0/1625854803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d461fdd920 0x55d461ff0820 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.531+0000 7f3a1c2ae740 1 -- 172.21.15.120:0/1625854803 >> 172.21.15.120:0/1625854803 conn(0x55d461fd9010 msgr2=0x55d461fd81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.531+0000 7f3a1c2ae740 1 -- 172.21.15.120:0/1625854803 shutdown_connections 2024-09-19T09:24:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.531+0000 7f3a1c2ae740 1 -- 172.21.15.120:0/1625854803 wait complete. 2024-09-19T09:24:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.531+0000 7f3a1c2ae740 1 -- start start 2024-09-19T09:24:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.531+0000 7f3a1c2ae740 1 Processor -- start 2024-09-19T09:24:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.531+0000 7f3a1c2ae740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d461fdd920 0x55d461ff54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.531+0000 7f3a1c2ae740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d461fde260 0x55d461ff5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.531+0000 7f3a1c2ae740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d461fe1330 0x55d461ff6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.531+0000 7f3a1c2ae740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0000f940 con 0x55d461fe1330 2024-09-19T09:24:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.531+0000 7f3a1c2ae740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0000f270 con 0x55d461fde260 2024-09-19T09:24:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.531+0000 7f3a1c2ae740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0000c490 con 0x55d461fdd920 2024-09-19T09:24:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.531+0000 7f3a1c2ae740 10 client.0 _ll_register_callbacks cb 0x7f3a0401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.532+0000 7f3a137fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d461fe1330 0x55d461ff6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.532+0000 7f3a137fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d461fe1330 0x55d461ff6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52668/0 (socket says 172.21.15.120:52668) 2024-09-19T09:24:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.532+0000 7f3a137fe640 1 -- 172.21.15.120:0/1518494406 learned_addr learned my addr 172.21.15.120:0/1518494406 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.532+0000 7f3a12ffd640 1 --2- 172.21.15.120:0/1518494406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d461fdd920 0x55d461ff54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.532+0000 7f3a127fc640 1 --2- 172.21.15.120:0/1518494406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d461fde260 0x55d461ff5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.532+0000 7f3a137fe640 1 -- 172.21.15.120:0/1518494406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d461fdd920 msgr2=0x55d461ff54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.532+0000 7f3a137fe640 1 --2- 172.21.15.120:0/1518494406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d461fdd920 0x55d461ff54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.532+0000 7f3a137fe640 1 -- 172.21.15.120:0/1518494406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d461fde260 msgr2=0x55d461ff5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.532+0000 7f3a137fe640 1 --2- 172.21.15.120:0/1518494406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d461fde260 0x55d461ff5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.532+0000 7f3a137fe640 1 -- 172.21.15.120:0/1518494406 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d461ff7030 con 0x55d461fe1330 2024-09-19T09:24:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.532+0000 7f3a137fe640 1 --2- 172.21.15.120:0/1518494406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d461fe1330 0x55d461ff6a20 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c009b30 tx=0x7f3a0c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee33128afd49230 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.532+0000 7f39f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.532+0000 7f39f6ff5640 1 -- 172.21.15.120:0/1518494406 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c036050 con 0x55d461fe1330 2024-09-19T09:24:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.532+0000 7f39f6ff5640 1 -- 172.21.15.120:0/1518494406 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f3a0c004250 con 0x55d461fe1330 2024-09-19T09:24:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.532+0000 7f3a1c2ae740 1 -- 172.21.15.120:0/1518494406 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a04009e50 con 0x55d461fe1330 2024-09-19T09:24:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.532+0000 7f39f6ff5640 1 -- 172.21.15.120:0/1518494406 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3a0c004450 con 0x55d461fe1330 2024-09-19T09:24:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.532+0000 7f3a1c2ae740 10 client.22485 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.532+0000 7f3a1c2ae740 1 -- 172.21.15.120:0/1518494406 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d462066f90 con 0x55d461fe1330 2024-09-19T09:24:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.533+0000 7f39f6ff5640 1 -- 172.21.15.120:0/1518494406 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a04009e50 con 0x55d461fe1330 2024-09-19T09:24:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.533+0000 7f39f6ff5640 1 -- 172.21.15.120:0/1518494406 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f3a0c03c040 con 0x55d461fe1330 2024-09-19T09:24:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.533+0000 7f3a1c2ae740 10 client.22485 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.533+0000 7f3a1c2ae740 10 client.22485 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:24:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.533+0000 7f3a1c2ae740 20 client.22485 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.533+0000 7f3a1c2ae740 1 --2- 172.21.15.120:0/1518494406 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d462067940 0x55d462069d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.533+0000 7f3a1c2ae740 4 client.22485 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:24:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.533+0000 7f3a1c2ae740 1 -- 172.21.15.120:0/1518494406 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d461f48d70 con 0x55d462067940 2024-09-19T09:24:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.533+0000 7f3a12ffd640 1 --2- 172.21.15.120:0/1518494406 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d462067940 0x55d462069d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.534+0000 7f39f6ff5640 10 client.22485 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.534+0000 7f3a12ffd640 1 --2- 172.21.15.120:0/1518494406 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d462067940 0x55d462069d60 crc :-1 s=READY pgs=534 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cf2b5c7512f6a94 server_cookie=e30ae0b8b22719db in_seq=0 out_seq=0 2024-09-19T09:24:26.868 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.892+0000 7f39f6ff5640 1 -- 172.21.15.120:0/1518494406 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d461f48d70 con 0x55d462067940 2024-09-19T09:24:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.892+0000 7f39f6ff5640 10 client.22485 handle_command_reply: tid=0 2024-09-19T09:24:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.892+0000 7f3a1c2ae740 1 client.22485 shutdown 2024-09-19T09:24:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.892+0000 7f3a1c2ae740 1 -- 172.21.15.120:0/1518494406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d461fe1330 msgr2=0x55d461ff6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.892+0000 7f3a1c2ae740 1 --2- 172.21.15.120:0/1518494406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d461fe1330 0x55d461ff6a20 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c009b30 tx=0x7f3a0c009500 comp rx=0 tx=0).stop 2024-09-19T09:24:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.892+0000 7f3a1c2ae740 1 -- 172.21.15.120:0/1518494406 shutdown_connections 2024-09-19T09:24:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.892+0000 7f3a1c2ae740 1 --2- 172.21.15.120:0/1518494406 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55d462067940 0x55d462069d60 crc :-1 s=READY pgs=534 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.892+0000 7f3a1c2ae740 1 --2- 172.21.15.120:0/1518494406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d461fe1330 0x55d461ff6a20 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.892+0000 7f3a1c2ae740 1 --2- 172.21.15.120:0/1518494406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d461fde260 0x55d461ff5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.892+0000 7f3a1c2ae740 1 --2- 172.21.15.120:0/1518494406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d461fdd920 0x55d461ff54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.892+0000 7f3a1c2ae740 1 -- 172.21.15.120:0/1518494406 >> 172.21.15.120:0/1518494406 conn(0x55d461fd9010 msgr2=0x55d461fd93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.892+0000 7f39f6ff5640 0 client.22485 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.892+0000 7f3a1c2ae740 1 -- 172.21.15.120:0/1518494406 shutdown_connections 2024-09-19T09:24:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.892+0000 7f3a1c2ae740 1 -- 172.21.15.120:0/1518494406 wait complete. 2024-09-19T09:24:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.892+0000 7f3a1c2ae740 20 client.22485 trim_cache size 0 max 16384 2024-09-19T09:24:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:24:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:24:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:24:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:24:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.893+0000 7f3a1951a640 1 -- 172.21.15.120:0/1647409184 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f39fc0a33f0 msgr2=0x7f39fc0a58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.893+0000 7f3a1951a640 1 --2- 172.21.15.120:0/1647409184 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f39fc0a33f0 0x7f39fc0a58b0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0099a0 tx=0x7f3a0c005c10 comp rx=0 tx=0).stop 2024-09-19T09:24:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.893+0000 7f3a1951a640 1 -- 172.21.15.120:0/1647409184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a14101c80 msgr2=0x7f3a141971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.893+0000 7f3a1951a640 1 --2- 172.21.15.120:0/1647409184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a14101c80 0x7f3a141971c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3a0800e790 tx=0x7f3a0800ec50 comp rx=0 tx=0).stop 2024-09-19T09:24:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.893+0000 7f3a1951a640 1 -- 172.21.15.120:0/1647409184 shutdown_connections 2024-09-19T09:24:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.893+0000 7f3a1951a640 1 --2- 172.21.15.120:0/1647409184 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f39fc0a33f0 0x7f39fc0a58b0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.893+0000 7f3a1951a640 1 --2- 172.21.15.120:0/1647409184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a14102640 0x7f3a1419e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.893+0000 7f3a1951a640 1 --2- 172.21.15.120:0/1647409184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a14101c80 0x7f3a141971c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.893+0000 7f3a1951a640 1 --2- 172.21.15.120:0/1647409184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a14100f70 0x7f3a14196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.893+0000 7f3a1951a640 1 -- 172.21.15.120:0/1647409184 >> 172.21.15.120:0/1647409184 conn(0x7f3a140fb930 msgr2=0x7f3a14104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.893+0000 7f3a1951a640 1 -- 172.21.15.120:0/1647409184 shutdown_connections 2024-09-19T09:24:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:26.893+0000 7f3a1951a640 1 -- 172.21.15.120:0/1647409184 wait complete. 2024-09-19T09:24:27.380 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199263 20.26 MB/sec execute 382 sec latency 175.764 ms 2024-09-19T09:24:27.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.420+0000 7f6c7ced1640 1 -- 172.21.15.120:0/315848095 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c78100a90 msgr2=0x7f6c78100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:27.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.420+0000 7f6c7ced1640 1 --2- 172.21.15.120:0/315848095 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c78100a90 0x7f6c78100ef0 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f6c60009930 tx=0x7f6c6002a420 comp rx=0 tx=0).stop 2024-09-19T09:24:27.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.420+0000 7f6c7ced1640 1 -- 172.21.15.120:0/315848095 shutdown_connections 2024-09-19T09:24:27.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.420+0000 7f6c7ced1640 1 --2- 172.21.15.120:0/315848095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c78101430 0x7f6c7810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:27.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.420+0000 7f6c7ced1640 1 --2- 172.21.15.120:0/315848095 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c78100a90 0x7f6c78100ef0 unknown :-1 s=CLOSED pgs=2240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:27.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.420+0000 7f6c7ced1640 1 --2- 172.21.15.120:0/315848095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c78106a90 0x7f6c78106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:27.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.420+0000 7f6c7ced1640 1 -- 172.21.15.120:0/315848095 >> 172.21.15.120:0/315848095 conn(0x7f6c780fc7d0 msgr2=0x7f6c780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:27.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.420+0000 7f6c7ced1640 1 -- 172.21.15.120:0/315848095 shutdown_connections 2024-09-19T09:24:27.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.420+0000 7f6c7ced1640 1 -- 172.21.15.120:0/315848095 wait complete. 2024-09-19T09:24:27.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.421+0000 7f6c7ced1640 1 Processor -- start 2024-09-19T09:24:27.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.421+0000 7f6c7ced1640 1 -- start start 2024-09-19T09:24:27.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.421+0000 7f6c7ced1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c78100a90 0x7f6c78198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:27.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.421+0000 7f6c7ced1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c78101430 0x7f6c78198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:27.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.421+0000 7f6c7ced1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c78106a90 0x7f6c7819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:27.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.421+0000 7f6c7ced1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6c78110e90 con 0x7f6c78101430 2024-09-19T09:24:27.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.421+0000 7f6c7ced1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6c78110d10 con 0x7f6c78106a90 2024-09-19T09:24:27.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.421+0000 7f6c7ced1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6c78111010 con 0x7f6c78100a90 2024-09-19T09:24:27.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.421+0000 7f6c6ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c78101430 0x7f6c78198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:27.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.421+0000 7f6c7659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c78100a90 0x7f6c78198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:27.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.421+0000 7f6c6ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c78101430 0x7f6c78198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52692/0 (socket says 172.21.15.120:52692) 2024-09-19T09:24:27.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.421+0000 7f6c6ffff640 1 -- 172.21.15.120:0/2690974459 learned_addr learned my addr 172.21.15.120:0/2690974459 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:27.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.421+0000 7f6c76d9d640 1 --2- 172.21.15.120:0/2690974459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c78106a90 0x7f6c7819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:27.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.422+0000 7f6c76d9d640 1 -- 172.21.15.120:0/2690974459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c78100a90 msgr2=0x7f6c78198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:27.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.422+0000 7f6c76d9d640 1 --2- 172.21.15.120:0/2690974459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c78100a90 0x7f6c78198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:27.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.422+0000 7f6c76d9d640 1 -- 172.21.15.120:0/2690974459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c78101430 msgr2=0x7f6c78198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:27.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.422+0000 7f6c76d9d640 1 --2- 172.21.15.120:0/2690974459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c78101430 0x7f6c78198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:27.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.422+0000 7f6c76d9d640 1 -- 172.21.15.120:0/2690974459 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7819d3f0 con 0x7f6c78106a90 2024-09-19T09:24:27.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.422+0000 7f6c7659c640 1 --2- 172.21.15.120:0/2690974459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c78100a90 0x7f6c78198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:24:27.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.422+0000 7f6c6ffff640 1 --2- 172.21.15.120:0/2690974459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c78101430 0x7f6c78198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:24:27.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.422+0000 7f6c76d9d640 1 --2- 172.21.15.120:0/2690974459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c78106a90 0x7f6c7819cd10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6c70009fd0 tx=0x7f6c7000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:27.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.422+0000 7f6c6f7fe640 1 -- 172.21.15.120:0/2690974459 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7000ce20 con 0x7f6c78106a90 2024-09-19T09:24:27.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.422+0000 7f6c7ced1640 1 -- 172.21.15.120:0/2690974459 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c7819d6e0 con 0x7f6c78106a90 2024-09-19T09:24:27.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.422+0000 7f6c6f7fe640 1 -- 172.21.15.120:0/2690974459 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c70005020 con 0x7f6c78106a90 2024-09-19T09:24:27.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.422+0000 7f6c6f7fe640 1 -- 172.21.15.120:0/2690974459 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70010720 con 0x7f6c78106a90 2024-09-19T09:24:27.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.422+0000 7f6c7ced1640 1 -- 172.21.15.120:0/2690974459 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c780fccf0 con 0x7f6c78106a90 2024-09-19T09:24:27.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.425+0000 7f6c6f7fe640 1 -- 172.21.15.120:0/2690974459 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f6c70005540 con 0x7f6c78106a90 2024-09-19T09:24:27.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.425+0000 7f6c7f2aa740 1 Processor -- start 2024-09-19T09:24:27.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.425+0000 7f6c7f2aa740 1 -- start start 2024-09-19T09:24:27.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.425+0000 7f6c7f2aa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cab4eed760 0x55cab4eedb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:27.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.425+0000 7f6c7f2aa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cab4eee080 0x55cab4ef8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:27.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.425+0000 7f6c7f2aa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cab4ef8940 0x55cab4efad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:27.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.425+0000 7f6c7f2aa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cab4e532a0 con 0x55cab4eed760 2024-09-19T09:24:27.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.425+0000 7f6c7f2aa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cab4dfab10 con 0x55cab4eee080 2024-09-19T09:24:27.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.425+0000 7f6c7f2aa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cab4e4f260 con 0x55cab4ef8940 2024-09-19T09:24:27.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.425+0000 7f6c6ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cab4eed760 0x55cab4eedb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:27.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.425+0000 7f6c7659c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cab4eee080 0x55cab4ef8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:27.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.426+0000 7f6c6ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cab4eed760 0x55cab4eedb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52694/0 (socket says 172.21.15.120:52694) 2024-09-19T09:24:27.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.426+0000 7f6c6ffff640 1 -- 172.21.15.120:0/487900415 learned_addr learned my addr 172.21.15.120:0/487900415 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:27.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.426+0000 7f6c6ffff640 1 --2- 172.21.15.120:0/487900415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cab4ef8940 0x55cab4efad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:27.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.426+0000 7f6c6ffff640 1 -- 172.21.15.120:0/487900415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cab4ef8940 msgr2=0x55cab4efad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:27.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.426+0000 7f6c6ffff640 1 --2- 172.21.15.120:0/487900415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cab4ef8940 0x55cab4efad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:27.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.426+0000 7f6c6ffff640 1 -- 172.21.15.120:0/487900415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cab4eee080 msgr2=0x55cab4ef8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:27.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.426+0000 7f6c6ffff640 1 --2- 172.21.15.120:0/487900415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cab4eee080 0x55cab4ef8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:27.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.426+0000 7f6c6ffff640 1 -- 172.21.15.120:0/487900415 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cab4bc4b90 con 0x55cab4eed760 2024-09-19T09:24:27.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.426+0000 7f6c7659c640 1 --2- 172.21.15.120:0/487900415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cab4eee080 0x55cab4ef8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:24:27.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.426+0000 7f6c6f7fe640 1 --2- 172.21.15.120:0/2690974459 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6c640a3240 0x7f6c640a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:27.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.426+0000 7f6c7659c640 1 --2- 172.21.15.120:0/2690974459 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6c640a3240 0x7f6c640a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:27.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.426+0000 7f6c6ffff640 1 --2- 172.21.15.120:0/487900415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cab4eed760 0x55cab4eedb40 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f6c600096c0 tx=0x7f6c60031470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e088528ba9a8f746 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:27.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.426+0000 7f6c6f7fe640 1 -- 172.21.15.120:0/2690974459 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f6c700cff10 con 0x7f6c78106a90 2024-09-19T09:24:27.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.426+0000 7f6c6cff9640 1 -- 172.21.15.120:0/487900415 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60045040 con 0x55cab4eed760 2024-09-19T09:24:27.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.426+0000 7f6c6cff9640 1 -- 172.21.15.120:0/487900415 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c6003c6f0 con 0x55cab4eed760 2024-09-19T09:24:27.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.426+0000 7f6c6cff9640 1 -- 172.21.15.120:0/487900415 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60035040 con 0x55cab4eed760 2024-09-19T09:24:27.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.427+0000 7f6c7f2aa740 1 -- 172.21.15.120:0/487900415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cab4eed760 msgr2=0x55cab4eedb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:27.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.427+0000 7f6c7f2aa740 1 --2- 172.21.15.120:0/487900415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cab4eed760 0x55cab4eedb40 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f6c600096c0 tx=0x7f6c60031470 comp rx=0 tx=0).stop 2024-09-19T09:24:27.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.427+0000 7f6c7659c640 1 --2- 172.21.15.120:0/2690974459 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6c640a3240 0x7f6c640a5700 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6c68009fd0 tx=0x7f6c68005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:27.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.427+0000 7f6c7f2aa740 1 -- 172.21.15.120:0/487900415 shutdown_connections 2024-09-19T09:24:27.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.427+0000 7f6c7f2aa740 1 --2- 172.21.15.120:0/487900415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cab4ef8940 0x55cab4efad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:27.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.427+0000 7f6c7f2aa740 1 --2- 172.21.15.120:0/487900415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cab4eee080 0x55cab4ef8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:27.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.427+0000 7f6c7f2aa740 1 --2- 172.21.15.120:0/487900415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cab4eed760 0x55cab4eedb40 unknown :-1 s=CLOSED pgs=2241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:27.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.427+0000 7f6c7f2aa740 1 -- 172.21.15.120:0/487900415 >> 172.21.15.120:0/487900415 conn(0x55cab4ee9cf0 msgr2=0x55cab4eea0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:27.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.427+0000 7f6c7f2aa740 1 -- 172.21.15.120:0/487900415 shutdown_connections 2024-09-19T09:24:27.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.427+0000 7f6c7f2aa740 1 -- 172.21.15.120:0/487900415 wait complete. 2024-09-19T09:24:27.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.428+0000 7f6c7f2aa740 1 -- start start 2024-09-19T09:24:27.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.428+0000 7f6c7f2aa740 1 Processor -- start 2024-09-19T09:24:27.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.428+0000 7f6c7f2aa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cab4eed760 0x55cab4ef5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:27.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.428+0000 7f6c7f2aa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cab4eee080 0x55cab4ef5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:27.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.428+0000 7f6c7f2aa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cab4ef8940 0x55cab4ef62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:27.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.428+0000 7f6c7f2aa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cab4efb510 con 0x55cab4eed760 2024-09-19T09:24:27.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.428+0000 7f6c7f2aa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cab4ddb6e0 con 0x55cab4eee080 2024-09-19T09:24:27.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.428+0000 7f6c7f2aa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cab4efb690 con 0x55cab4ef8940 2024-09-19T09:24:27.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.428+0000 7f6c7f2aa740 10 client.0 _ll_register_callbacks cb 0x55cab4dffa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:27.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.428+0000 7f6c7659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cab4eed760 0x55cab4ef5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:27.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.428+0000 7f6c7659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cab4eed760 0x55cab4ef5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52702/0 (socket says 172.21.15.120:52702) 2024-09-19T09:24:27.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.428+0000 7f6c7659c640 1 -- 172.21.15.120:0/3739914704 learned_addr learned my addr 172.21.15.120:0/3739914704 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:27.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.428+0000 7f6c76d9d640 1 --2- 172.21.15.120:0/3739914704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cab4ef8940 0x55cab4ef62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:27.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.428+0000 7f6c6ffff640 1 --2- 172.21.15.120:0/3739914704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cab4eee080 0x55cab4ef5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:27.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.429+0000 7f6c7659c640 1 -- 172.21.15.120:0/3739914704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cab4ef8940 msgr2=0x55cab4ef62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:27.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.429+0000 7f6c7659c640 1 --2- 172.21.15.120:0/3739914704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cab4ef8940 0x55cab4ef62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:27.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.429+0000 7f6c7659c640 1 -- 172.21.15.120:0/3739914704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cab4eee080 msgr2=0x55cab4ef5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:27.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.429+0000 7f6c7659c640 1 --2- 172.21.15.120:0/3739914704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cab4eee080 0x55cab4ef5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:27.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.429+0000 7f6c7659c640 1 -- 172.21.15.120:0/3739914704 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cab4f04a30 con 0x55cab4eed760 2024-09-19T09:24:27.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.429+0000 7f6c7659c640 1 --2- 172.21.15.120:0/3739914704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cab4eed760 0x55cab4ef5860 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f6c68004250 tx=0x7f6c68004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb9c6d97c74f4392 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:27.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.429+0000 7f6c5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:27.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.429+0000 7f6c5dffb640 1 -- 172.21.15.120:0/3739914704 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6800fb80 con 0x55cab4eed760 2024-09-19T09:24:27.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.429+0000 7f6c5dffb640 1 -- 172.21.15.120:0/3739914704 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f6c6800fd20 con 0x55cab4eed760 2024-09-19T09:24:27.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.429+0000 7f6c7f2aa740 1 -- 172.21.15.120:0/3739914704 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c6003c6f0 con 0x55cab4eed760 2024-09-19T09:24:27.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.429+0000 7f6c5dffb640 1 -- 172.21.15.120:0/3739914704 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6c680035f0 con 0x55cab4eed760 2024-09-19T09:24:27.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.429+0000 7f6c7f2aa740 10 client.22497 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:27.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.429+0000 7f6c7f2aa740 1 -- 172.21.15.120:0/3739914704 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55cab4f07420 con 0x55cab4eed760 2024-09-19T09:24:27.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.430+0000 7f6c5dffb640 1 -- 172.21.15.120:0/3739914704 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c6003c6f0 con 0x55cab4eed760 2024-09-19T09:24:27.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.431+0000 7f6c5dffb640 1 -- 172.21.15.120:0/3739914704 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f6c6802a830 con 0x55cab4eed760 2024-09-19T09:24:27.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.431+0000 7f6c7f2aa740 10 client.22497 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:27.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.431+0000 7f6c7f2aa740 10 client.22497 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:24:27.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.431+0000 7f6c7f2aa740 20 client.22497 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:27.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.431+0000 7f6c7f2aa740 1 --2- 172.21.15.120:0/3739914704 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cab4f08d40 0x55cab4f0b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:27.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.431+0000 7f6c7f2aa740 4 client.22497 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:24:27.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.431+0000 7f6c7f2aa740 1 -- 172.21.15.120:0/3739914704 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cab4e609d0 con 0x55cab4f08d40 2024-09-19T09:24:27.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.431+0000 7f6c6ffff640 1 --2- 172.21.15.120:0/3739914704 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cab4f08d40 0x55cab4f0b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:27.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.431+0000 7f6c5dffb640 10 client.22497 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:27.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.431+0000 7f6c6ffff640 1 --2- 172.21.15.120:0/3739914704 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cab4f08d40 0x55cab4f0b1e0 crc :-1 s=READY pgs=535 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc4b61be00d14941 server_cookie=56981dbfdf3338ec in_seq=0 out_seq=0 2024-09-19T09:24:27.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.434+0000 7f6c5dffb640 1 -- 172.21.15.120:0/3739914704 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cab4e609d0 con 0x55cab4f08d40 2024-09-19T09:24:27.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.434+0000 7f6c5dffb640 10 client.22497 handle_command_reply: tid=0 2024-09-19T09:24:27.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.434+0000 7f6c7f2aa740 1 client.22497 shutdown 2024-09-19T09:24:27.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.434+0000 7f6c7f2aa740 1 -- 172.21.15.120:0/3739914704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cab4eed760 msgr2=0x55cab4ef5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:27.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.434+0000 7f6c7f2aa740 1 --2- 172.21.15.120:0/3739914704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cab4eed760 0x55cab4ef5860 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f6c68004250 tx=0x7f6c68004730 comp rx=0 tx=0).stop 2024-09-19T09:24:27.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.434+0000 7f6c7f2aa740 1 -- 172.21.15.120:0/3739914704 shutdown_connections 2024-09-19T09:24:27.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.434+0000 7f6c7f2aa740 1 --2- 172.21.15.120:0/3739914704 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cab4f08d40 0x55cab4f0b1e0 crc :-1 s=READY pgs=535 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:27.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.434+0000 7f6c7f2aa740 1 --2- 172.21.15.120:0/3739914704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cab4ef8940 0x55cab4ef62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:27.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.434+0000 7f6c7f2aa740 1 --2- 172.21.15.120:0/3739914704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cab4eee080 0x55cab4ef5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:27.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.434+0000 7f6c5dffb640 0 client.22497 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:27.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.434+0000 7f6c7f2aa740 1 --2- 172.21.15.120:0/3739914704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cab4eed760 0x55cab4ef5860 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:27.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.434+0000 7f6c7f2aa740 1 -- 172.21.15.120:0/3739914704 >> 172.21.15.120:0/3739914704 conn(0x55cab4ee9cf0 msgr2=0x55cab4eefa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:27.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.434+0000 7f6c7f2aa740 1 -- 172.21.15.120:0/3739914704 shutdown_connections 2024-09-19T09:24:27.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.434+0000 7f6c7f2aa740 1 -- 172.21.15.120:0/3739914704 wait complete. 2024-09-19T09:24:27.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.434+0000 7f6c7f2aa740 20 client.22497 trim_cache size 0 max 16384 2024-09-19T09:24:27.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.452+0000 7f6c7f2aa740 1 Processor -- start 2024-09-19T09:24:27.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.452+0000 7f6c7f2aa740 1 -- start start 2024-09-19T09:24:27.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.453+0000 7f6c7f2aa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cab4eed760 0x55cab4ef7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:27.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.453+0000 7f6c7f2aa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cab4eee080 0x55cab4ef4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:27.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.453+0000 7f6c7f2aa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cab4ef8940 0x55cab4ef0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:27.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.453+0000 7f6c7f2aa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6c781ab340 con 0x55cab4eed760 2024-09-19T09:24:27.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.453+0000 7f6c7f2aa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6c781ace80 con 0x55cab4ef8940 2024-09-19T09:24:27.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.453+0000 7f6c7f2aa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6c781aba10 con 0x55cab4eee080 2024-09-19T09:24:27.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.453+0000 7f6c7659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cab4eed760 0x55cab4ef7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:27.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.453+0000 7f6c7659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cab4eed760 0x55cab4ef7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52708/0 (socket says 172.21.15.120:52708) 2024-09-19T09:24:27.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.453+0000 7f6c7659c640 1 -- 172.21.15.120:0/1210713677 learned_addr learned my addr 172.21.15.120:0/1210713677 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:27.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.453+0000 7f6c76d9d640 1 --2- 172.21.15.120:0/1210713677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cab4ef8940 0x55cab4ef0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:27.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.453+0000 7f6c6ffff640 1 --2- 172.21.15.120:0/1210713677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cab4eee080 0x55cab4ef4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:27.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.453+0000 7f6c7659c640 1 -- 172.21.15.120:0/1210713677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cab4eee080 msgr2=0x55cab4ef4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:27.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.453+0000 7f6c7659c640 1 --2- 172.21.15.120:0/1210713677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cab4eee080 0x55cab4ef4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:27.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.453+0000 7f6c7659c640 1 -- 172.21.15.120:0/1210713677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cab4ef8940 msgr2=0x55cab4ef0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:27.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.453+0000 7f6c7659c640 1 --2- 172.21.15.120:0/1210713677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cab4ef8940 0x55cab4ef0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:27.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.453+0000 7f6c7659c640 1 -- 172.21.15.120:0/1210713677 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cab4ee9b20 con 0x55cab4eed760 2024-09-19T09:24:27.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.453+0000 7f6c76d9d640 1 --2- 172.21.15.120:0/1210713677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cab4ef8940 0x55cab4ef0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:24:27.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.453+0000 7f6c6ffff640 1 --2- 172.21.15.120:0/1210713677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cab4eee080 0x55cab4ef4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:24:27.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.453+0000 7f6c7659c640 1 --2- 172.21.15.120:0/1210713677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cab4eed760 0x55cab4ef7730 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800f460 tx=0x7f6c68020620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d248bdd4822e051d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:27.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.454+0000 7f6c5ffff640 1 -- 172.21.15.120:0/1210713677 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6800fa50 con 0x55cab4eed760 2024-09-19T09:24:27.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.454+0000 7f6c5ffff640 1 -- 172.21.15.120:0/1210713677 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c68039020 con 0x55cab4eed760 2024-09-19T09:24:27.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.454+0000 7f6c5ffff640 1 -- 172.21.15.120:0/1210713677 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c68003910 con 0x55cab4eed760 2024-09-19T09:24:27.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.454+0000 7f6c7f2aa740 1 -- 172.21.15.120:0/1210713677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cab4eed760 msgr2=0x55cab4ef7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:27.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.454+0000 7f6c7f2aa740 1 --2- 172.21.15.120:0/1210713677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cab4eed760 0x55cab4ef7730 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800f460 tx=0x7f6c68020620 comp rx=0 tx=0).stop 2024-09-19T09:24:27.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.454+0000 7f6c7f2aa740 1 -- 172.21.15.120:0/1210713677 shutdown_connections 2024-09-19T09:24:27.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.454+0000 7f6c7f2aa740 1 --2- 172.21.15.120:0/1210713677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cab4ef8940 0x55cab4ef0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:27.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.454+0000 7f6c7f2aa740 1 --2- 172.21.15.120:0/1210713677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cab4eee080 0x55cab4ef4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:27.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.454+0000 7f6c7f2aa740 1 --2- 172.21.15.120:0/1210713677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cab4eed760 0x55cab4ef7730 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:27.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.454+0000 7f6c7f2aa740 1 -- 172.21.15.120:0/1210713677 >> 172.21.15.120:0/1210713677 conn(0x55cab4ee9cf0 msgr2=0x55cab4ee8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:27.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.455+0000 7f6c7f2aa740 1 -- 172.21.15.120:0/1210713677 shutdown_connections 2024-09-19T09:24:27.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.455+0000 7f6c7f2aa740 1 -- 172.21.15.120:0/1210713677 wait complete. 2024-09-19T09:24:27.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.455+0000 7f6c7f2aa740 1 -- start start 2024-09-19T09:24:27.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.455+0000 7f6c7f2aa740 1 Processor -- start 2024-09-19T09:24:27.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.455+0000 7f6c7f2aa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cab4eed760 0x55cab4f008f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:27.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.455+0000 7f6c7f2aa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cab4eee080 0x55cab4f00e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:27.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.455+0000 7f6c7f2aa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cab4ef8940 0x55cab4efcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:27.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.455+0000 7f6c7f2aa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6c781ac7b0 con 0x55cab4eed760 2024-09-19T09:24:27.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.455+0000 7f6c7f2aa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6c781ac0e0 con 0x55cab4eee080 2024-09-19T09:24:27.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.455+0000 7f6c7f2aa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6c781a9300 con 0x55cab4ef8940 2024-09-19T09:24:27.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.455+0000 7f6c7f2aa740 10 client.0 _ll_register_callbacks cb 0x7f6c680040e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:27.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.456+0000 7f6c76d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cab4ef8940 0x55cab4efcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:27.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.456+0000 7f6c7659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cab4eed760 0x55cab4f008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:27.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.456+0000 7f6c6ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cab4eee080 0x55cab4f00e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:27.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.456+0000 7f6c76d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cab4ef8940 0x55cab4efcda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:38320/0 (socket says 172.21.15.120:38320) 2024-09-19T09:24:27.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.456+0000 7f6c76d9d640 1 -- 172.21.15.120:0/515209390 learned_addr learned my addr 172.21.15.120:0/515209390 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:27.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.456+0000 7f6c76d9d640 1 -- 172.21.15.120:0/515209390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cab4eee080 msgr2=0x55cab4f00e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:27.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.456+0000 7f6c76d9d640 1 --2- 172.21.15.120:0/515209390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cab4eee080 0x55cab4f00e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:27.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.456+0000 7f6c76d9d640 1 -- 172.21.15.120:0/515209390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cab4eed760 msgr2=0x55cab4f008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:27.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.456+0000 7f6c76d9d640 1 --2- 172.21.15.120:0/515209390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cab4eed760 0x55cab4f008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:27.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.456+0000 7f6c76d9d640 1 -- 172.21.15.120:0/515209390 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cab4ee5260 con 0x55cab4ef8940 2024-09-19T09:24:27.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.456+0000 7f6c7659c640 1 --2- 172.21.15.120:0/515209390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cab4eed760 0x55cab4f008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:24:27.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.456+0000 7f6c76d9d640 1 --2- 172.21.15.120:0/515209390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cab4ef8940 0x55cab4efcda0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6c70098f00 tx=0x7f6c7009c5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba2de936cf9f5b10 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:27.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.456+0000 7f6c5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:24:27.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.456+0000 7f6c5dffb640 1 -- 172.21.15.120:0/515209390 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c700bc040 con 0x55cab4ef8940 2024-09-19T09:24:27.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.456+0000 7f6c5dffb640 1 -- 172.21.15.120:0/515209390 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f6c700a4d10 con 0x55cab4ef8940 2024-09-19T09:24:27.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.457+0000 7f6c7f2aa740 1 -- 172.21.15.120:0/515209390 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c68039020 con 0x55cab4ef8940 2024-09-19T09:24:27.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.457+0000 7f6c5dffb640 1 -- 172.21.15.120:0/515209390 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6c700a4f10 con 0x55cab4ef8940 2024-09-19T09:24:27.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.457+0000 7f6c7f2aa740 10 client.31096 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:27.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.457+0000 7f6c7f2aa740 1 -- 172.21.15.120:0/515209390 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55cab4f74110 con 0x55cab4ef8940 2024-09-19T09:24:27.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.457+0000 7f6c5dffb640 1 -- 172.21.15.120:0/515209390 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c68039020 con 0x55cab4ef8940 2024-09-19T09:24:27.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.457+0000 7f6c5dffb640 1 -- 172.21.15.120:0/515209390 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f6c7009b5f0 con 0x55cab4ef8940 2024-09-19T09:24:27.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.457+0000 7f6c7f2aa740 10 client.31096 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:27.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.457+0000 7f6c7f2aa740 10 client.31096 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:24:27.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.457+0000 7f6c7f2aa740 20 client.31096 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:27.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.458+0000 7f6c7f2aa740 1 --2- 172.21.15.120:0/515209390 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cab4f748f0 0x55cab4f76ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:27.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.458+0000 7f6c7f2aa740 4 client.31096 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-09-19T09:24:27.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.458+0000 7f6c7f2aa740 1 -- 172.21.15.120:0/515209390 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x55cab4e5ad20 con 0x55cab4f748f0 2024-09-19T09:24:27.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.458+0000 7f6c7659c640 1 --2- 172.21.15.120:0/515209390 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cab4f748f0 0x55cab4f76ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:27.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.458+0000 7f6c5dffb640 10 client.31096 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:27.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:27.458+0000 7f6c7659c640 1 --2- 172.21.15.120:0/515209390 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cab4f748f0 0x55cab4f76ce0 crc :-1 s=READY pgs=536 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65a3b5462716e8bc server_cookie=220bea2cf0471c95 in_seq=0 out_seq=0 2024-09-19T09:24:28.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:28.135+0000 7f6c5dffb640 1 -- 172.21.15.120:0/515209390 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55cab4e5ad20 con 0x55cab4f748f0 2024-09-19T09:24:28.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:28.135+0000 7f6c5dffb640 10 client.31096 handle_command_reply: tid=0 2024-09-19T09:24:28.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:28.135+0000 7f6c7f2aa740 1 client.31096 shutdown 2024-09-19T09:24:28.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:28.135+0000 7f6c7f2aa740 1 -- 172.21.15.120:0/515209390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cab4ef8940 msgr2=0x55cab4efcda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:28.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:28.135+0000 7f6c7f2aa740 1 --2- 172.21.15.120:0/515209390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cab4ef8940 0x55cab4efcda0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6c70098f00 tx=0x7f6c7009c5f0 comp rx=0 tx=0).stop 2024-09-19T09:24:28.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:28.135+0000 7f6c7f2aa740 1 -- 172.21.15.120:0/515209390 shutdown_connections 2024-09-19T09:24:28.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:28.135+0000 7f6c7f2aa740 1 --2- 172.21.15.120:0/515209390 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55cab4f748f0 0x55cab4f76ce0 crc :-1 s=READY pgs=536 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:28.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:28.135+0000 7f6c7f2aa740 1 --2- 172.21.15.120:0/515209390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cab4ef8940 0x55cab4efcda0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:28.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:28.135+0000 7f6c5dffb640 0 client.31096 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:28.137 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:24:28.137 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 39, 2024-09-19T09:24:28.138 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:24:28.138 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 74, 2024-09-19T09:24:28.138 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:24:28.138 INFO:teuthology.orchestra.run.smithi120.stdout: "93457192": { 2024-09-19T09:24:28.138 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 74, 2024-09-19T09:24:28.138 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:24:28.138 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:24:28.138 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:24:28.138 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:24:28.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:24:28.138 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:24:28.138 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 144.5, 2024-09-19T09:24:28.138 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:24:28.138 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:24:28.138 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:24:28.139 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:24:28.139 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:24:28.139 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:24:28.139 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:24:28.139 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:24:28.139 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:24:28.139 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:24:28.139 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:24:28.139 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:24:28.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:28.135+0000 7f6c7f2aa740 1 --2- 172.21.15.120:0/515209390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cab4eee080 0x55cab4f00e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:28.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:28.135+0000 7f6c7f2aa740 1 --2- 172.21.15.120:0/515209390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cab4eed760 0x55cab4f008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:28.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:28.135+0000 7f6c7f2aa740 1 -- 172.21.15.120:0/515209390 >> 172.21.15.120:0/515209390 conn(0x55cab4ee9cf0 msgr2=0x55cab4f03760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:28.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:28.135+0000 7f6c7f2aa740 1 -- 172.21.15.120:0/515209390 shutdown_connections 2024-09-19T09:24:28.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:28.135+0000 7f6c7f2aa740 1 -- 172.21.15.120:0/515209390 wait complete. 2024-09-19T09:24:28.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:28.136+0000 7f6c7f2aa740 20 client.31096 trim_cache size 0 max 16384 2024-09-19T09:24:28.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:28.136+0000 7f6c7ced1640 1 -- 172.21.15.120:0/2690974459 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6c640a3240 msgr2=0x7f6c640a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:28.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:28.136+0000 7f6c7ced1640 1 --2- 172.21.15.120:0/2690974459 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6c640a3240 0x7f6c640a5700 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6c68009fd0 tx=0x7f6c68005f90 comp rx=0 tx=0).stop 2024-09-19T09:24:28.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:28.136+0000 7f6c7ced1640 1 -- 172.21.15.120:0/2690974459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c78106a90 msgr2=0x7f6c7819cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:28.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:28.136+0000 7f6c7ced1640 1 --2- 172.21.15.120:0/2690974459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c78106a90 0x7f6c7819cd10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6c70009fd0 tx=0x7f6c7000ef10 comp rx=0 tx=0).stop 2024-09-19T09:24:28.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:28.136+0000 7f6c7ced1640 1 -- 172.21.15.120:0/2690974459 shutdown_connections 2024-09-19T09:24:28.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:28.136+0000 7f6c7ced1640 1 --2- 172.21.15.120:0/2690974459 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6c640a3240 0x7f6c640a5700 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:28.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:28.136+0000 7f6c7ced1640 1 --2- 172.21.15.120:0/2690974459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c78106a90 0x7f6c7819cd10 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:28.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:28.136+0000 7f6c7ced1640 1 --2- 172.21.15.120:0/2690974459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c78101430 0x7f6c78198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:28.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:28.136+0000 7f6c7ced1640 1 --2- 172.21.15.120:0/2690974459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c78100a90 0x7f6c78198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:28.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:28.137+0000 7f6c7ced1640 1 -- 172.21.15.120:0/2690974459 >> 172.21.15.120:0/2690974459 conn(0x7f6c780fc7d0 msgr2=0x7f6c7806aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:28.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:28.137+0000 7f6c7ced1640 1 -- 172.21.15.120:0/2690974459 shutdown_connections 2024-09-19T09:24:28.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:28.137+0000 7f6c7ced1640 1 -- 172.21.15.120:0/2690974459 wait complete. 2024-09-19T09:24:28.329 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:28 smithi120 ceph-mon[31150]: pgmap v993: 97 pgs: 97 active+clean; 451 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 234 op/s 2024-09-19T09:24:28.381 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199337 20.21 MB/sec execute 383 sec latency 889.970 ms 2024-09-19T09:24:28.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:28 smithi196 ceph-mon[35150]: pgmap v993: 97 pgs: 97 active+clean; 451 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 234 op/s 2024-09-19T09:24:28.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:28 smithi153 ceph-mon[35071]: pgmap v993: 97 pgs: 97 active+clean; 451 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 234 op/s 2024-09-19T09:24:28.871 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '93457192', quiesce took 4.3 seconds. Will release after: 20.2 2024-09-19T09:24:29.380 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199337 20.16 MB/sec execute 384 sec latency 1890.080 ms 2024-09-19T09:24:30.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:30 smithi120 ceph-mon[31150]: pgmap v994: 97 pgs: 97 active+clean; 439 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.6 MiB/s wr, 399 op/s 2024-09-19T09:24:30.380 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199337 20.10 MB/sec execute 385 sec latency 2890.129 ms 2024-09-19T09:24:30.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:30 smithi196 ceph-mon[35150]: pgmap v994: 97 pgs: 97 active+clean; 439 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.6 MiB/s wr, 399 op/s 2024-09-19T09:24:30.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:30 smithi153 ceph-mon[35071]: pgmap v994: 97 pgs: 97 active+clean; 439 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.6 MiB/s wr, 399 op/s 2024-09-19T09:24:31.381 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199337 20.05 MB/sec execute 386 sec latency 3890.220 ms 2024-09-19T09:24:32.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:32 smithi120 ceph-mon[31150]: pgmap v995: 97 pgs: 97 active+clean; 437 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.9 MiB/s wr, 425 op/s 2024-09-19T09:24:32.381 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199337 20.00 MB/sec execute 387 sec latency 4890.273 ms 2024-09-19T09:24:32.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:32 smithi196 ceph-mon[35150]: pgmap v995: 97 pgs: 97 active+clean; 437 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.9 MiB/s wr, 425 op/s 2024-09-19T09:24:32.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:32 smithi153 ceph-mon[35071]: pgmap v995: 97 pgs: 97 active+clean; 437 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.9 MiB/s wr, 425 op/s 2024-09-19T09:24:33.381 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199337 19.95 MB/sec execute 388 sec latency 5890.370 ms 2024-09-19T09:24:34.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:34 smithi120 ceph-mon[31150]: pgmap v996: 97 pgs: 97 active+clean; 441 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.5 MiB/s wr, 432 op/s 2024-09-19T09:24:34.381 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199337 19.90 MB/sec execute 389 sec latency 6890.431 ms 2024-09-19T09:24:34.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:34 smithi196 ceph-mon[35150]: pgmap v996: 97 pgs: 97 active+clean; 441 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.5 MiB/s wr, 432 op/s 2024-09-19T09:24:34.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:34 smithi153 ceph-mon[35071]: pgmap v996: 97 pgs: 97 active+clean; 441 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.5 MiB/s wr, 432 op/s 2024-09-19T09:24:35.381 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199337 19.85 MB/sec execute 390 sec latency 7890.550 ms 2024-09-19T09:24:36.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:36 smithi120 ceph-mon[31150]: pgmap v997: 97 pgs: 97 active+clean; 439 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.3 MiB/s wr, 406 op/s 2024-09-19T09:24:36.381 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199337 19.80 MB/sec execute 391 sec latency 8890.650 ms 2024-09-19T09:24:36.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:36 smithi196 ceph-mon[35150]: pgmap v997: 97 pgs: 97 active+clean; 439 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.3 MiB/s wr, 406 op/s 2024-09-19T09:24:36.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:36 smithi153 ceph-mon[35071]: pgmap v997: 97 pgs: 97 active+clean; 439 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.3 MiB/s wr, 406 op/s 2024-09-19T09:24:37.381 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199337 19.75 MB/sec execute 392 sec latency 9890.734 ms 2024-09-19T09:24:38.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:38 smithi120 ceph-mon[31150]: pgmap v998: 97 pgs: 97 active+clean; 436 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.1 MiB/s wr, 398 op/s 2024-09-19T09:24:38.381 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199337 19.70 MB/sec execute 393 sec latency 10890.837 ms 2024-09-19T09:24:38.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:38 smithi196 ceph-mon[35150]: pgmap v998: 97 pgs: 97 active+clean; 436 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.1 MiB/s wr, 398 op/s 2024-09-19T09:24:38.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:38 smithi153 ceph-mon[35071]: pgmap v998: 97 pgs: 97 active+clean; 436 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.1 MiB/s wr, 398 op/s 2024-09-19T09:24:39.381 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199337 19.65 MB/sec execute 394 sec latency 11890.925 ms 2024-09-19T09:24:40.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.995+0000 7f82740c9640 1 -- 172.21.15.120:0/4066208377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f826c1037b0 msgr2=0x7f826c10a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:40.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.995+0000 7f82740c9640 1 --2- 172.21.15.120:0/4066208377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f826c1037b0 0x7f826c10a040 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f8264009ca0 tx=0x7f826402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.995+0000 7f82740c9640 1 -- 172.21.15.120:0/4066208377 shutdown_connections 2024-09-19T09:24:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.995+0000 7f82740c9640 1 --2- 172.21.15.120:0/4066208377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f826c1037b0 0x7f826c10a040 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.995+0000 7f82740c9640 1 --2- 172.21.15.120:0/4066208377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f826c102df0 0x7f826c103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.995+0000 7f82740c9640 1 --2- 172.21.15.120:0/4066208377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f826c0ffe20 0x7f826c100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.995+0000 7f82740c9640 1 -- 172.21.15.120:0/4066208377 >> 172.21.15.120:0/4066208377 conn(0x7f826c0fb980 msgr2=0x7f826c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.996+0000 7f82740c9640 1 -- 172.21.15.120:0/4066208377 shutdown_connections 2024-09-19T09:24:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.996+0000 7f82740c9640 1 -- 172.21.15.120:0/4066208377 wait complete. 2024-09-19T09:24:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.996+0000 7f82740c9640 1 Processor -- start 2024-09-19T09:24:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.996+0000 7f82740c9640 1 -- start start 2024-09-19T09:24:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.996+0000 7f82740c9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f826c0ffe20 0x7f826c102760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.996+0000 7f82740c9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f826c102df0 0x7f826c100da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.996+0000 7f82740c9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f826c1037b0 0x7f826c1012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.996+0000 7f82740c9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f826c10d850 con 0x7f826c0ffe20 2024-09-19T09:24:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.996+0000 7f82740c9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f826c10d6d0 con 0x7f826c1037b0 2024-09-19T09:24:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.996+0000 7f82740c9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f826c10d9d0 con 0x7f826c102df0 2024-09-19T09:24:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.997+0000 7f8271e65640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f826c0ffe20 0x7f826c102760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.997+0000 7f8271e65640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f826c0ffe20 0x7f826c102760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39830/0 (socket says 172.21.15.120:39830) 2024-09-19T09:24:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.997+0000 7f8271e65640 1 -- 172.21.15.120:0/1177706172 learned_addr learned my addr 172.21.15.120:0/1177706172 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.997+0000 7f8272666640 1 --2- 172.21.15.120:0/1177706172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f826c1037b0 0x7f826c1012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.997+0000 7f8271664640 1 --2- 172.21.15.120:0/1177706172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f826c102df0 0x7f826c100da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.997+0000 7f8271e65640 1 -- 172.21.15.120:0/1177706172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f826c102df0 msgr2=0x7f826c100da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.997+0000 7f8271e65640 1 --2- 172.21.15.120:0/1177706172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f826c102df0 0x7f826c100da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.997+0000 7f8271e65640 1 -- 172.21.15.120:0/1177706172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f826c1037b0 msgr2=0x7f826c1012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.997+0000 7f8271e65640 1 --2- 172.21.15.120:0/1177706172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f826c1037b0 0x7f826c1012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.997+0000 7f8271e65640 1 -- 172.21.15.120:0/1177706172 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f826c101b70 con 0x7f826c0ffe20 2024-09-19T09:24:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.997+0000 7f8271664640 1 --2- 172.21.15.120:0/1177706172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f826c102df0 0x7f826c100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:24:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.997+0000 7f8271e65640 1 --2- 172.21.15.120:0/1177706172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f826c0ffe20 0x7f826c102760 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f826000d770 tx=0x7f826000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.997+0000 7f825effd640 1 -- 172.21.15.120:0/1177706172 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8260017070 con 0x7f826c0ffe20 2024-09-19T09:24:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.997+0000 7f82740c9640 1 -- 172.21.15.120:0/1177706172 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f826c1a1f00 con 0x7f826c0ffe20 2024-09-19T09:24:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.997+0000 7f825effd640 1 -- 172.21.15.120:0/1177706172 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8260009c30 con 0x7f826c0ffe20 2024-09-19T09:24:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.997+0000 7f825effd640 1 -- 172.21.15.120:0/1177706172 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8260003dd0 con 0x7f826c0ffe20 2024-09-19T09:24:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.997+0000 7f82740c9640 1 -- 172.21.15.120:0/1177706172 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f826c1a2390 con 0x7f826c0ffe20 2024-09-19T09:24:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:39.999+0000 7f825effd640 1 -- 172.21.15.120:0/1177706172 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f8260003f70 con 0x7f826c0ffe20 2024-09-19T09:24:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.000+0000 7f8276e60740 1 Processor -- start 2024-09-19T09:24:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.000+0000 7f8276e60740 1 -- start start 2024-09-19T09:24:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.000+0000 7f8276e60740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560178f769a0 0x560178f76da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.000+0000 7f8276e60740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560178f772e0 0x560178f7bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.000+0000 7f8276e60740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560178f7c370 0x560178f7e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.000+0000 7f8276e60740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560178eda2f0 con 0x560178f769a0 2024-09-19T09:24:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.000+0000 7f8276e60740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560178e82270 con 0x560178f7c370 2024-09-19T09:24:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.000+0000 7f8276e60740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560178ed62b0 con 0x560178f772e0 2024-09-19T09:24:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.000+0000 7f825effd640 1 --2- 172.21.15.120:0/1177706172 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f82580a32f0 0x7f82580a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.000+0000 7f8272666640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560178f769a0 0x560178f76da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.000+0000 7f8272666640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560178f769a0 0x560178f76da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39838/0 (socket says 172.21.15.120:39838) 2024-09-19T09:24:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.000+0000 7f8272666640 1 -- 172.21.15.120:0/2416328868 learned_addr learned my addr 172.21.15.120:0/2416328868 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.001+0000 7f8271e65640 1 --2- 172.21.15.120:0/2416328868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560178f772e0 0x560178f7bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.001+0000 7f8271664640 1 --2- 172.21.15.120:0/2416328868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560178f7c370 0x560178f7e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.001+0000 7f8272666640 1 --2- 172.21.15.120:0/1177706172 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f82580a32f0 0x7f82580a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.001+0000 7f825effd640 1 -- 172.21.15.120:0/1177706172 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f82600cf240 con 0x7f826c0ffe20 2024-09-19T09:24:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.001+0000 7f8272666640 1 -- 172.21.15.120:0/2416328868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560178f772e0 msgr2=0x560178f7bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.001+0000 7f8272666640 1 --2- 172.21.15.120:0/2416328868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560178f772e0 0x560178f7bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.001+0000 7f8272666640 1 -- 172.21.15.120:0/2416328868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560178f7c370 msgr2=0x560178f7e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.001+0000 7f8272666640 1 --2- 172.21.15.120:0/2416328868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560178f7c370 0x560178f7e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.001+0000 7f8272666640 1 -- 172.21.15.120:0/2416328868 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560178e81340 con 0x560178f769a0 2024-09-19T09:24:40.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.001+0000 7f8271e65640 1 --2- 172.21.15.120:0/2416328868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560178f772e0 0x560178f7bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:24:40.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.001+0000 7f8272666640 1 --2- 172.21.15.120:0/2416328868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560178f769a0 0x560178f76da0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f8264009ca0 tx=0x7f8264031550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c6bf2ca749b34a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:40.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.001+0000 7f8257fff640 1 -- 172.21.15.120:0/2416328868 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8264004b40 con 0x560178f769a0 2024-09-19T09:24:40.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.001+0000 7f8272666640 1 --2- 172.21.15.120:0/1177706172 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f82580a32f0 0x7f82580a57b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f826404e000 tx=0x7f826404c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:40.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.001+0000 7f8257fff640 1 -- 172.21.15.120:0/2416328868 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f826404f070 con 0x560178f769a0 2024-09-19T09:24:40.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.001+0000 7f8257fff640 1 -- 172.21.15.120:0/2416328868 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826402fe70 con 0x560178f769a0 2024-09-19T09:24:40.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.002+0000 7f8276e60740 1 -- 172.21.15.120:0/2416328868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560178f769a0 msgr2=0x560178f76da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:40.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.002+0000 7f8276e60740 1 --2- 172.21.15.120:0/2416328868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560178f769a0 0x560178f76da0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f8264009ca0 tx=0x7f8264031550 comp rx=0 tx=0).stop 2024-09-19T09:24:40.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.002+0000 7f8276e60740 1 -- 172.21.15.120:0/2416328868 shutdown_connections 2024-09-19T09:24:40.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.002+0000 7f8276e60740 1 --2- 172.21.15.120:0/2416328868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560178f7c370 0x560178f7e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.002+0000 7f8276e60740 1 --2- 172.21.15.120:0/2416328868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560178f772e0 0x560178f7bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.002+0000 7f8276e60740 1 --2- 172.21.15.120:0/2416328868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560178f769a0 0x560178f76da0 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.002+0000 7f8276e60740 1 -- 172.21.15.120:0/2416328868 >> 172.21.15.120:0/2416328868 conn(0x560178f72010 msgr2=0x560178f72410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.002+0000 7f8276e60740 1 -- 172.21.15.120:0/2416328868 shutdown_connections 2024-09-19T09:24:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.002+0000 7f8276e60740 1 -- 172.21.15.120:0/2416328868 wait complete. 2024-09-19T09:24:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.002+0000 7f8276e60740 1 -- start start 2024-09-19T09:24:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.003+0000 7f8276e60740 1 Processor -- start 2024-09-19T09:24:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.003+0000 7f8276e60740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560178f769a0 0x560178f80b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.003+0000 7f8276e60740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560178f772e0 0x560178f810b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.003+0000 7f8276e60740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560178f7c370 0x560178f88010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.003+0000 7f8276e60740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560178f7f1c0 con 0x560178f769a0 2024-09-19T09:24:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.003+0000 7f8276e60740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560178e63290 con 0x560178f7c370 2024-09-19T09:24:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.003+0000 7f8276e60740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560178f7f340 con 0x560178f772e0 2024-09-19T09:24:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.003+0000 7f8276e60740 10 client.0 _ll_register_callbacks cb 0x560178e94140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.003+0000 7f8271e65640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560178f769a0 0x560178f80b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.003+0000 7f8271e65640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560178f769a0 0x560178f80b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39852/0 (socket says 172.21.15.120:39852) 2024-09-19T09:24:40.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.003+0000 7f8271e65640 1 -- 172.21.15.120:0/482953574 learned_addr learned my addr 172.21.15.120:0/482953574 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:40.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.003+0000 7f8272666640 1 --2- 172.21.15.120:0/482953574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560178f7c370 0x560178f88010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:40.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.003+0000 7f8271664640 1 --2- 172.21.15.120:0/482953574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560178f772e0 0x560178f810b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:40.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.003+0000 7f8271e65640 1 -- 172.21.15.120:0/482953574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560178f772e0 msgr2=0x560178f810b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:40.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.003+0000 7f8271e65640 1 --2- 172.21.15.120:0/482953574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560178f772e0 0x560178f810b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.003+0000 7f8271e65640 1 -- 172.21.15.120:0/482953574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560178f7c370 msgr2=0x560178f88010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:40.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.003+0000 7f8271e65640 1 --2- 172.21.15.120:0/482953574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560178f7c370 0x560178f88010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.003+0000 7f8271e65640 1 -- 172.21.15.120:0/482953574 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560178bed210 con 0x560178f769a0 2024-09-19T09:24:40.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.004+0000 7f8271e65640 1 --2- 172.21.15.120:0/482953574 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560178f769a0 0x560178f80b70 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f8260098230 tx=0x7f8260003620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90fc4f6785934aae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:40.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.004+0000 7f82557fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:40.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.004+0000 7f82557fa640 1 -- 172.21.15.120:0/482953574 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8260020070 con 0x560178f769a0 2024-09-19T09:24:40.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.004+0000 7f82557fa640 1 -- 172.21.15.120:0/482953574 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f8260004260 con 0x560178f769a0 2024-09-19T09:24:40.022 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.004+0000 7f8276e60740 1 -- 172.21.15.120:0/482953574 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f826404f070 con 0x560178f769a0 2024-09-19T09:24:40.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.004+0000 7f82557fa640 1 -- 172.21.15.120:0/482953574 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8260086070 con 0x560178f769a0 2024-09-19T09:24:40.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.004+0000 7f8276e60740 10 client.22515 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:40.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.004+0000 7f8276e60740 1 -- 172.21.15.120:0/482953574 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560178cb9620 con 0x560178f769a0 2024-09-19T09:24:40.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.004+0000 7f82557fa640 1 -- 172.21.15.120:0/482953574 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f826404f070 con 0x560178f769a0 2024-09-19T09:24:40.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.004+0000 7f82557fa640 1 -- 172.21.15.120:0/482953574 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f8260097ed0 con 0x560178f769a0 2024-09-19T09:24:40.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.004+0000 7f8276e60740 10 client.22515 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:40.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.004+0000 7f8276e60740 10 client.22515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:24:40.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.004+0000 7f8276e60740 20 client.22515 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:40.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.004+0000 7f8276e60740 1 --2- 172.21.15.120:0/482953574 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560178f8c700 0x560178f8eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:40.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.004+0000 7f8276e60740 4 client.22515 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:24:40.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.004+0000 7f8276e60740 1 -- 172.21.15.120:0/482953574 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560178ee7a20 con 0x560178f8c700 2024-09-19T09:24:40.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.005+0000 7f8271664640 1 --2- 172.21.15.120:0/482953574 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560178f8c700 0x560178f8eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:40.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.005+0000 7f8271664640 1 --2- 172.21.15.120:0/482953574 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560178f8c700 0x560178f8eb10 crc :-1 s=READY pgs=537 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f1321b505c862f7 server_cookie=adb05d30f4adc284 in_seq=0 out_seq=0 2024-09-19T09:24:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.005+0000 7f82557fa640 10 client.22515 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.006+0000 7f82557fa640 1 -- 172.21.15.120:0/482953574 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560178ee7a20 con 0x560178f8c700 2024-09-19T09:24:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.006+0000 7f82557fa640 10 client.22515 handle_command_reply: tid=0 2024-09-19T09:24:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.006+0000 7f8276e60740 1 client.22515 shutdown 2024-09-19T09:24:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.006+0000 7f8276e60740 1 -- 172.21.15.120:0/482953574 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560178f769a0 msgr2=0x560178f80b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.006+0000 7f8276e60740 1 --2- 172.21.15.120:0/482953574 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560178f769a0 0x560178f80b70 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f8260098230 tx=0x7f8260003620 comp rx=0 tx=0).stop 2024-09-19T09:24:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.006+0000 7f8276e60740 1 -- 172.21.15.120:0/482953574 shutdown_connections 2024-09-19T09:24:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.006+0000 7f8276e60740 1 --2- 172.21.15.120:0/482953574 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560178f8c700 0x560178f8eb10 crc :-1 s=READY pgs=537 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.006+0000 7f8276e60740 1 --2- 172.21.15.120:0/482953574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560178f7c370 0x560178f88010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.006+0000 7f8276e60740 1 --2- 172.21.15.120:0/482953574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560178f772e0 0x560178f810b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.006+0000 7f8276e60740 1 --2- 172.21.15.120:0/482953574 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560178f769a0 0x560178f80b70 unknown :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.006+0000 7f82557fa640 0 client.22515 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:40.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.006+0000 7f8276e60740 1 -- 172.21.15.120:0/482953574 >> 172.21.15.120:0/482953574 conn(0x560178f72010 msgr2=0x560178f73820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:40.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.006+0000 7f8276e60740 1 -- 172.21.15.120:0/482953574 shutdown_connections 2024-09-19T09:24:40.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.006+0000 7f8276e60740 1 -- 172.21.15.120:0/482953574 wait complete. 2024-09-19T09:24:40.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.006+0000 7f8276e60740 20 client.22515 trim_cache size 0 max 16384 2024-09-19T09:24:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.020+0000 7f8276e60740 1 Processor -- start 2024-09-19T09:24:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.020+0000 7f8276e60740 1 -- start start 2024-09-19T09:24:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.020+0000 7f8276e60740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560178f769a0 0x560178f81140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.020+0000 7f8276e60740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560178f772e0 0x560178f87fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.020+0000 7f8276e60740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560178f7c370 0x560178f710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.020+0000 7f8276e60740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f826c1ab300 con 0x560178f7c370 2024-09-19T09:24:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.020+0000 7f8276e60740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f826c1ace40 con 0x560178f772e0 2024-09-19T09:24:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.020+0000 7f8276e60740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f826c1ab9d0 con 0x560178f769a0 2024-09-19T09:24:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.021+0000 7f8272666640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560178f7c370 0x560178f710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.021+0000 7f8272666640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560178f7c370 0x560178f710b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39868/0 (socket says 172.21.15.120:39868) 2024-09-19T09:24:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.021+0000 7f8272666640 1 -- 172.21.15.120:0/2859287724 learned_addr learned my addr 172.21.15.120:0/2859287724 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.021+0000 7f8271664640 1 --2- 172.21.15.120:0/2859287724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560178f772e0 0x560178f87fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.021+0000 7f8272666640 1 -- 172.21.15.120:0/2859287724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560178f769a0 msgr2=0x560178f81140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.021+0000 7f8272666640 1 --2- 172.21.15.120:0/2859287724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560178f769a0 0x560178f81140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.021+0000 7f8272666640 1 -- 172.21.15.120:0/2859287724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560178f772e0 msgr2=0x560178f87fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.021+0000 7f8272666640 1 --2- 172.21.15.120:0/2859287724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560178f772e0 0x560178f87fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.021+0000 7f8272666640 1 -- 172.21.15.120:0/2859287724 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560178f8f190 con 0x560178f7c370 2024-09-19T09:24:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.021+0000 7f8271664640 1 --2- 172.21.15.120:0/2859287724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560178f772e0 0x560178f87fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:24:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.021+0000 7f8272666640 1 --2- 172.21.15.120:0/2859287724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560178f7c370 0x560178f710b0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f8264009bf0 tx=0x7f826402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b88de877f710e28 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.021+0000 7f82577fe640 1 -- 172.21.15.120:0/2859287724 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8264060070 con 0x560178f7c370 2024-09-19T09:24:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.021+0000 7f82577fe640 1 -- 172.21.15.120:0/2859287724 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f826404f070 con 0x560178f7c370 2024-09-19T09:24:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.021+0000 7f82577fe640 1 -- 172.21.15.120:0/2859287724 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826402fce0 con 0x560178f7c370 2024-09-19T09:24:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.022+0000 7f8276e60740 1 -- 172.21.15.120:0/2859287724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560178f7c370 msgr2=0x560178f710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.022+0000 7f8276e60740 1 --2- 172.21.15.120:0/2859287724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560178f7c370 0x560178f710b0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f8264009bf0 tx=0x7f826402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.022+0000 7f8276e60740 1 -- 172.21.15.120:0/2859287724 shutdown_connections 2024-09-19T09:24:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.022+0000 7f8276e60740 1 --2- 172.21.15.120:0/2859287724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560178f7c370 0x560178f710b0 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.022+0000 7f8276e60740 1 --2- 172.21.15.120:0/2859287724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560178f772e0 0x560178f87fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.022+0000 7f8276e60740 1 --2- 172.21.15.120:0/2859287724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560178f769a0 0x560178f81140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.022+0000 7f8276e60740 1 -- 172.21.15.120:0/2859287724 >> 172.21.15.120:0/2859287724 conn(0x560178f72010 msgr2=0x560178f709a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.022+0000 7f8276e60740 1 -- 172.21.15.120:0/2859287724 shutdown_connections 2024-09-19T09:24:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.022+0000 7f8276e60740 1 -- 172.21.15.120:0/2859287724 wait complete. 2024-09-19T09:24:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.022+0000 7f8276e60740 1 -- start start 2024-09-19T09:24:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.022+0000 7f8276e60740 1 Processor -- start 2024-09-19T09:24:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.022+0000 7f8276e60740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560178f769a0 0x560178f85de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.022+0000 7f8276e60740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560178f772e0 0x560178f86320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.022+0000 7f8276e60740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560178f7c370 0x560178f872e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.022+0000 7f8276e60740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f826c1ac770 con 0x560178f769a0 2024-09-19T09:24:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.022+0000 7f8276e60740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f826c1ac0a0 con 0x560178f772e0 2024-09-19T09:24:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.022+0000 7f8276e60740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f826c1a92c0 con 0x560178f7c370 2024-09-19T09:24:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.023+0000 7f8276e60740 10 client.0 _ll_register_callbacks cb 0x7f8260003690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.023+0000 7f8271e65640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560178f769a0 0x560178f85de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.023+0000 7f8271e65640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560178f769a0 0x560178f85de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39876/0 (socket says 172.21.15.120:39876) 2024-09-19T09:24:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.023+0000 7f8271e65640 1 -- 172.21.15.120:0/3022994077 learned_addr learned my addr 172.21.15.120:0/3022994077 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.023+0000 7f8271664640 1 --2- 172.21.15.120:0/3022994077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560178f772e0 0x560178f86320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.023+0000 7f8272666640 1 --2- 172.21.15.120:0/3022994077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560178f7c370 0x560178f872e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.023+0000 7f8271e65640 1 -- 172.21.15.120:0/3022994077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560178f7c370 msgr2=0x560178f872e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.023+0000 7f8271e65640 1 --2- 172.21.15.120:0/3022994077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560178f7c370 0x560178f872e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.023+0000 7f8271e65640 1 -- 172.21.15.120:0/3022994077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560178f772e0 msgr2=0x560178f86320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.023+0000 7f8271e65640 1 --2- 172.21.15.120:0/3022994077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560178f772e0 0x560178f86320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.023+0000 7f8271e65640 1 -- 172.21.15.120:0/3022994077 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560178f878f0 con 0x560178f769a0 2024-09-19T09:24:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.023+0000 7f8271e65640 1 --2- 172.21.15.120:0/3022994077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560178f769a0 0x560178f85de0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f826009fc80 tx=0x7f826000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1001e94f9b27cf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.023+0000 7f82557fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.023+0000 7f82557fa640 1 -- 172.21.15.120:0/3022994077 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8260020070 con 0x560178f769a0 2024-09-19T09:24:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.023+0000 7f82557fa640 1 -- 172.21.15.120:0/3022994077 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f82600a4040 con 0x560178f769a0 2024-09-19T09:24:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.023+0000 7f8276e60740 1 -- 172.21.15.120:0/3022994077 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f826404f070 con 0x560178f769a0 2024-09-19T09:24:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.023+0000 7f82557fa640 1 -- 172.21.15.120:0/3022994077 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8260022300 con 0x560178f769a0 2024-09-19T09:24:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.023+0000 7f8276e60740 10 client.22521 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.023+0000 7f8276e60740 1 -- 172.21.15.120:0/3022994077 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560178ff7af0 con 0x560178f769a0 2024-09-19T09:24:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.023+0000 7f82557fa640 1 -- 172.21.15.120:0/3022994077 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f826404f070 con 0x560178f769a0 2024-09-19T09:24:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.024+0000 7f82557fa640 1 -- 172.21.15.120:0/3022994077 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f8260084bb0 con 0x560178f769a0 2024-09-19T09:24:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.024+0000 7f8276e60740 10 client.22521 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:40.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.024+0000 7f8276e60740 10 client.22521 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:24:40.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.024+0000 7f8276e60740 20 client.22521 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:40.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.024+0000 7f8276e60740 1 --2- 172.21.15.120:0/3022994077 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560178ff82b0 0x560178ffa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:40.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.024+0000 7f8276e60740 4 client.22521 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:24:40.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.024+0000 7f8276e60740 1 -- 172.21.15.120:0/3022994077 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560178ee1d70 con 0x560178ff82b0 2024-09-19T09:24:40.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.024+0000 7f8271664640 1 --2- 172.21.15.120:0/3022994077 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560178ff82b0 0x560178ffa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:40.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.024+0000 7f82557fa640 10 client.22521 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:40.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.024+0000 7f8271664640 1 --2- 172.21.15.120:0/3022994077 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560178ff82b0 0x560178ffa6a0 crc :-1 s=READY pgs=538 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fbe4fa78fc660c7 server_cookie=3d3f208501e43791 in_seq=0 out_seq=0 2024-09-19T09:24:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.168+0000 7f82557fa640 1 -- 172.21.15.120:0/3022994077 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560178ee1d70 con 0x560178ff82b0 2024-09-19T09:24:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.168+0000 7f82557fa640 10 client.22521 handle_command_reply: tid=0 2024-09-19T09:24:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.168+0000 7f8276e60740 1 client.22521 shutdown 2024-09-19T09:24:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.169+0000 7f8276e60740 1 -- 172.21.15.120:0/3022994077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560178f769a0 msgr2=0x560178f85de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.169+0000 7f8276e60740 1 --2- 172.21.15.120:0/3022994077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560178f769a0 0x560178f85de0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f826009fc80 tx=0x7f826000b480 comp rx=0 tx=0).stop 2024-09-19T09:24:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.169+0000 7f8276e60740 1 -- 172.21.15.120:0/3022994077 shutdown_connections 2024-09-19T09:24:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.169+0000 7f8276e60740 1 --2- 172.21.15.120:0/3022994077 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560178ff82b0 0x560178ffa6a0 crc :-1 s=READY pgs=538 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.169+0000 7f8276e60740 1 --2- 172.21.15.120:0/3022994077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560178f7c370 0x560178f872e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.169+0000 7f82557fa640 0 client.22521 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.169+0000 7f8276e60740 1 --2- 172.21.15.120:0/3022994077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560178f772e0 0x560178f86320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.169+0000 7f8276e60740 1 --2- 172.21.15.120:0/3022994077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560178f769a0 0x560178f85de0 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.169+0000 7f8276e60740 1 -- 172.21.15.120:0/3022994077 >> 172.21.15.120:0/3022994077 conn(0x560178f72010 msgr2=0x560178f736e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.169+0000 7f8276e60740 1 -- 172.21.15.120:0/3022994077 shutdown_connections 2024-09-19T09:24:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.169+0000 7f8276e60740 1 -- 172.21.15.120:0/3022994077 wait complete. 2024-09-19T09:24:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.169+0000 7f8276e60740 20 client.22521 trim_cache size 0 max 16384 2024-09-19T09:24:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:24:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:24:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:24:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:24:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.169+0000 7f82740c9640 1 -- 172.21.15.120:0/1177706172 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f82580a32f0 msgr2=0x7f82580a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.169+0000 7f82740c9640 1 --2- 172.21.15.120:0/1177706172 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f82580a32f0 0x7f82580a57b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f826404e000 tx=0x7f826404c000 comp rx=0 tx=0).stop 2024-09-19T09:24:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.169+0000 7f82740c9640 1 -- 172.21.15.120:0/1177706172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f826c0ffe20 msgr2=0x7f826c102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.169+0000 7f82740c9640 1 --2- 172.21.15.120:0/1177706172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f826c0ffe20 0x7f826c102760 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f826000d770 tx=0x7f826000dc30 comp rx=0 tx=0).stop 2024-09-19T09:24:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.169+0000 7f82740c9640 1 -- 172.21.15.120:0/1177706172 shutdown_connections 2024-09-19T09:24:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.169+0000 7f82740c9640 1 --2- 172.21.15.120:0/1177706172 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f82580a32f0 0x7f82580a57b0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.169+0000 7f82740c9640 1 --2- 172.21.15.120:0/1177706172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f826c1037b0 0x7f826c1012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.169+0000 7f82740c9640 1 --2- 172.21.15.120:0/1177706172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f826c102df0 0x7f826c100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.169+0000 7f82740c9640 1 --2- 172.21.15.120:0/1177706172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f826c0ffe20 0x7f826c102760 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.170+0000 7f82740c9640 1 -- 172.21.15.120:0/1177706172 >> 172.21.15.120:0/1177706172 conn(0x7f826c0fb980 msgr2=0x7f826c1060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.170+0000 7f82740c9640 1 -- 172.21.15.120:0/1177706172 shutdown_connections 2024-09-19T09:24:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:40.170+0000 7f82740c9640 1 -- 172.21.15.120:0/1177706172 wait complete. 2024-09-19T09:24:40.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:40 smithi120 ceph-mon[31150]: pgmap v999: 97 pgs: 97 active+clean; 433 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.6 MiB/s wr, 360 op/s 2024-09-19T09:24:40.348 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:24:40.381 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199337 19.60 MB/sec execute 395 sec latency 12890.973 ms 2024-09-19T09:24:40.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:40 smithi196 ceph-mon[35150]: pgmap v999: 97 pgs: 97 active+clean; 433 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.6 MiB/s wr, 360 op/s 2024-09-19T09:24:40.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:40 smithi153 ceph-mon[35071]: pgmap v999: 97 pgs: 97 active+clean; 433 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.6 MiB/s wr, 360 op/s 2024-09-19T09:24:40.591 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:41.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.078+0000 7f1b74cae640 1 -- 172.21.15.120:0/4051469863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b70106a80 msgr2=0x7f1b70106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:41.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.078+0000 7f1b74cae640 1 --2- 172.21.15.120:0/4051469863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b70106a80 0x7f1b70106e60 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f1b60009930 tx=0x7f1b6002a420 comp rx=0 tx=0).stop 2024-09-19T09:24:41.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.078+0000 7f1b74cae640 1 -- 172.21.15.120:0/4051469863 shutdown_connections 2024-09-19T09:24:41.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.078+0000 7f1b74cae640 1 --2- 172.21.15.120:0/4051469863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b70101420 0x7f1b7010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.078+0000 7f1b74cae640 1 --2- 172.21.15.120:0/4051469863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b70100a80 0x7f1b70100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.078+0000 7f1b74cae640 1 --2- 172.21.15.120:0/4051469863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b70106a80 0x7f1b70106e60 unknown :-1 s=CLOSED pgs=2250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.078+0000 7f1b74cae640 1 -- 172.21.15.120:0/4051469863 >> 172.21.15.120:0/4051469863 conn(0x7f1b700fc7a0 msgr2=0x7f1b700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:41.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.078+0000 7f1b74cae640 1 -- 172.21.15.120:0/4051469863 shutdown_connections 2024-09-19T09:24:41.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.078+0000 7f1b74cae640 1 -- 172.21.15.120:0/4051469863 wait complete. 2024-09-19T09:24:41.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.079+0000 7f1b74cae640 1 Processor -- start 2024-09-19T09:24:41.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.079+0000 7f1b74cae640 1 -- start start 2024-09-19T09:24:41.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.079+0000 7f1b74cae640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b70100a80 0x7f1b701972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:41.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.079+0000 7f1b74cae640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b70101420 0x7f1b70197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:41.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.079+0000 7f1b74cae640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b70106a80 0x7f1b7019bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:41.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.079+0000 7f1b74cae640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1b70110e80 con 0x7f1b70100a80 2024-09-19T09:24:41.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.079+0000 7f1b74cae640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1b70110d00 con 0x7f1b70106a80 2024-09-19T09:24:41.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.079+0000 7f1b74cae640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1b70111000 con 0x7f1b70101420 2024-09-19T09:24:41.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.079+0000 7f1b6e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b70100a80 0x7f1b701972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:41.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.079+0000 7f1b6e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b70100a80 0x7f1b701972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39898/0 (socket says 172.21.15.120:39898) 2024-09-19T09:24:41.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.079+0000 7f1b6e59c640 1 -- 172.21.15.120:0/2423045031 learned_addr learned my addr 172.21.15.120:0/2423045031 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:41.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.079+0000 7f1b6dd9b640 1 --2- 172.21.15.120:0/2423045031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b70101420 0x7f1b70197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:41.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.080+0000 7f1b6ed9d640 1 --2- 172.21.15.120:0/2423045031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b70106a80 0x7f1b7019bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:41.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.080+0000 7f1b6e59c640 1 -- 172.21.15.120:0/2423045031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b70101420 msgr2=0x7f1b70197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:41.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.080+0000 7f1b6e59c640 1 --2- 172.21.15.120:0/2423045031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b70101420 0x7f1b70197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.080+0000 7f1b6e59c640 1 -- 172.21.15.120:0/2423045031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b70106a80 msgr2=0x7f1b7019bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:41.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.080+0000 7f1b6e59c640 1 --2- 172.21.15.120:0/2423045031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b70106a80 0x7f1b7019bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.080+0000 7f1b6e59c640 1 -- 172.21.15.120:0/2423045031 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7019c270 con 0x7f1b70100a80 2024-09-19T09:24:41.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.080+0000 7f1b6ed9d640 1 --2- 172.21.15.120:0/2423045031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b70106a80 0x7f1b7019bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:24:41.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.080+0000 7f1b6e59c640 1 --2- 172.21.15.120:0/2423045031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b70100a80 0x7f1b701972c0 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f1b60009230 tx=0x7f1b60004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:41.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.080+0000 7f1b5f7fe640 1 -- 172.21.15.120:0/2423045031 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6003f030 con 0x7f1b70100a80 2024-09-19T09:24:41.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.080+0000 7f1b74cae640 1 -- 172.21.15.120:0/2423045031 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7019c500 con 0x7f1b70100a80 2024-09-19T09:24:41.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.080+0000 7f1b5f7fe640 1 -- 172.21.15.120:0/2423045031 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b60003ad0 con 0x7f1b70100a80 2024-09-19T09:24:41.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.080+0000 7f1b5f7fe640 1 -- 172.21.15.120:0/2423045031 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60035070 con 0x7f1b70100a80 2024-09-19T09:24:41.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.080+0000 7f1b74cae640 1 -- 172.21.15.120:0/2423045031 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b701a3e40 con 0x7f1b70100a80 2024-09-19T09:24:41.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.082+0000 7f1b5f7fe640 1 -- 172.21.15.120:0/2423045031 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f1b60042030 con 0x7f1b70100a80 2024-09-19T09:24:41.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.082+0000 7f1b77087740 1 Processor -- start 2024-09-19T09:24:41.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.083+0000 7f1b77087740 1 -- start start 2024-09-19T09:24:41.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.083+0000 7f1b77087740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556930af1360 0x556930af1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:41.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.083+0000 7f1b77087740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556930af1c80 0x556930afe350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:41.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.083+0000 7f1b77087740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556930afe890 0x556930b00c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:41.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.083+0000 7f1b77087740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556930a572e0 con 0x556930afe890 2024-09-19T09:24:41.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.083+0000 7f1b77087740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5569309fece0 con 0x556930af1c80 2024-09-19T09:24:41.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.083+0000 7f1b77087740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556930a532a0 con 0x556930af1360 2024-09-19T09:24:41.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.083+0000 7f1b6dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556930afe890 0x556930b00c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:41.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.083+0000 7f1b6dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556930afe890 0x556930b00c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39906/0 (socket says 172.21.15.120:39906) 2024-09-19T09:24:41.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.083+0000 7f1b6dd9b640 1 -- 172.21.15.120:0/2607925600 learned_addr learned my addr 172.21.15.120:0/2607925600 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:41.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.083+0000 7f1b5f7fe640 1 --2- 172.21.15.120:0/2423045031 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1b580a3240 0x7f1b580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:41.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.083+0000 7f1b6ed9d640 1 --2- 172.21.15.120:0/2607925600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556930af1360 0x556930af1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:41.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.083+0000 7f1b6e59c640 1 --2- 172.21.15.120:0/2607925600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556930af1c80 0x556930afe350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:41.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.083+0000 7f1b6ed9d640 1 --2- 172.21.15.120:0/2423045031 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1b580a3240 0x7f1b580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:41.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.083+0000 7f1b6dd9b640 1 -- 172.21.15.120:0/2607925600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556930af1360 msgr2=0x556930af1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:41.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.083+0000 7f1b6dd9b640 1 --2- 172.21.15.120:0/2607925600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556930af1360 0x556930af1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.083+0000 7f1b6dd9b640 1 -- 172.21.15.120:0/2607925600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556930af1c80 msgr2=0x556930afe350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:41.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.083+0000 7f1b6dd9b640 1 --2- 172.21.15.120:0/2607925600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556930af1c80 0x556930afe350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.083+0000 7f1b5f7fe640 1 -- 172.21.15.120:0/2423045031 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f1b600efe10 con 0x7f1b70100a80 2024-09-19T09:24:41.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.083+0000 7f1b6dd9b640 1 -- 172.21.15.120:0/2607925600 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569309e3440 con 0x556930afe890 2024-09-19T09:24:41.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.084+0000 7f1b6dd9b640 1 --2- 172.21.15.120:0/2607925600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556930afe890 0x556930b00c80 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400b740 tx=0x7f1b6400bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43793ce78b375d3a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:41.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.084+0000 7f1b5cff9640 1 -- 172.21.15.120:0/2607925600 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64003dd0 con 0x556930afe890 2024-09-19T09:24:41.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.084+0000 7f1b5cff9640 1 -- 172.21.15.120:0/2607925600 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b64015070 con 0x556930afe890 2024-09-19T09:24:41.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.084+0000 7f1b5cff9640 1 -- 172.21.15.120:0/2607925600 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64011440 con 0x556930afe890 2024-09-19T09:24:41.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.084+0000 7f1b6ed9d640 1 --2- 172.21.15.120:0/2423045031 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1b580a3240 0x7f1b580a5700 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1b68007310 tx=0x7f1b680072a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:41.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.085+0000 7f1b77087740 1 -- 172.21.15.120:0/2607925600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556930afe890 msgr2=0x556930b00c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:41.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.085+0000 7f1b77087740 1 --2- 172.21.15.120:0/2607925600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556930afe890 0x556930b00c80 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400b740 tx=0x7f1b6400bc00 comp rx=0 tx=0).stop 2024-09-19T09:24:41.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.085+0000 7f1b77087740 1 -- 172.21.15.120:0/2607925600 shutdown_connections 2024-09-19T09:24:41.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.085+0000 7f1b77087740 1 --2- 172.21.15.120:0/2607925600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556930afe890 0x556930b00c80 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.085+0000 7f1b77087740 1 --2- 172.21.15.120:0/2607925600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556930af1c80 0x556930afe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.085+0000 7f1b77087740 1 --2- 172.21.15.120:0/2607925600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556930af1360 0x556930af1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.085+0000 7f1b77087740 1 -- 172.21.15.120:0/2607925600 >> 172.21.15.120:0/2607925600 conn(0x556930aeda30 msgr2=0x556930aede10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:41.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.085+0000 7f1b77087740 1 -- 172.21.15.120:0/2607925600 shutdown_connections 2024-09-19T09:24:41.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.085+0000 7f1b77087740 1 -- 172.21.15.120:0/2607925600 wait complete. 2024-09-19T09:24:41.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.085+0000 7f1b77087740 1 -- start start 2024-09-19T09:24:41.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.085+0000 7f1b77087740 1 Processor -- start 2024-09-19T09:24:41.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.085+0000 7f1b77087740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556930af1360 0x556930afe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:41.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.085+0000 7f1b77087740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556930af1c80 0x556930afb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:41.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.085+0000 7f1b77087740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556930afe890 0x556930afbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:41.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.085+0000 7f1b77087740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556930b01460 con 0x556930af1c80 2024-09-19T09:24:41.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.085+0000 7f1b77087740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556930a12050 con 0x556930af1360 2024-09-19T09:24:41.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.085+0000 7f1b77087740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556930b015e0 con 0x556930afe890 2024-09-19T09:24:41.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.085+0000 7f1b77087740 10 client.0 _ll_register_callbacks cb 0x556930a52160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:41.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.086+0000 7f1b6dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556930af1c80 0x556930afb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:41.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.086+0000 7f1b6dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556930af1c80 0x556930afb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39922/0 (socket says 172.21.15.120:39922) 2024-09-19T09:24:41.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.086+0000 7f1b6dd9b640 1 -- 172.21.15.120:0/2633360549 learned_addr learned my addr 172.21.15.120:0/2633360549 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:41.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.086+0000 7f1b6e59c640 1 --2- 172.21.15.120:0/2633360549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556930af1360 0x556930afe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:41.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.086+0000 7f1b6ed9d640 1 --2- 172.21.15.120:0/2633360549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556930afe890 0x556930afbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:41.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.086+0000 7f1b6dd9b640 1 -- 172.21.15.120:0/2633360549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556930afe890 msgr2=0x556930afbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:41.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.086+0000 7f1b6dd9b640 1 --2- 172.21.15.120:0/2633360549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556930afe890 0x556930afbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.086+0000 7f1b6dd9b640 1 -- 172.21.15.120:0/2633360549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556930af1360 msgr2=0x556930afe2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:41.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.086+0000 7f1b6dd9b640 1 --2- 172.21.15.120:0/2633360549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556930af1360 0x556930afe2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.086+0000 7f1b6dd9b640 1 -- 172.21.15.120:0/2633360549 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569307c83c0 con 0x556930af1c80 2024-09-19T09:24:41.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.086+0000 7f1b6dd9b640 1 --2- 172.21.15.120:0/2633360549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556930af1c80 0x556930afb550 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400b710 tx=0x7f1b64004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad58fc04a4bcecb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:41.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.086+0000 7f1b55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:41.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.086+0000 7f1b55ffb640 1 -- 172.21.15.120:0/2633360549 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64028060 con 0x556930af1c80 2024-09-19T09:24:41.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.086+0000 7f1b55ffb640 1 -- 172.21.15.120:0/2633360549 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f1b64011640 con 0x556930af1c80 2024-09-19T09:24:41.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.086+0000 7f1b77087740 1 -- 172.21.15.120:0/2633360549 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b64015070 con 0x556930af1c80 2024-09-19T09:24:41.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.086+0000 7f1b55ffb640 1 -- 172.21.15.120:0/2633360549 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1b64003ea0 con 0x556930af1c80 2024-09-19T09:24:41.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.086+0000 7f1b77087740 10 client.22533 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:41.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.086+0000 7f1b77087740 1 -- 172.21.15.120:0/2633360549 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556930b0cea0 con 0x556930af1c80 2024-09-19T09:24:41.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.087+0000 7f1b55ffb640 1 -- 172.21.15.120:0/2633360549 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b64015070 con 0x556930af1c80 2024-09-19T09:24:41.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.087+0000 7f1b55ffb640 1 -- 172.21.15.120:0/2633360549 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f1b6402d070 con 0x556930af1c80 2024-09-19T09:24:41.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.087+0000 7f1b77087740 10 client.22533 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:41.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.087+0000 7f1b77087740 10 client.22533 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:24:41.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.087+0000 7f1b77087740 20 client.22533 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:41.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.087+0000 7f1b77087740 1 --2- 172.21.15.120:0/2633360549 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556930b0e1f0 0x556930b105e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:41.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.087+0000 7f1b77087740 4 client.22533 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:24:41.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.087+0000 7f1b77087740 1 -- 172.21.15.120:0/2633360549 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556930a64850 con 0x556930b0e1f0 2024-09-19T09:24:41.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.087+0000 7f1b6e59c640 1 --2- 172.21.15.120:0/2633360549 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556930b0e1f0 0x556930b105e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:41.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.087+0000 7f1b55ffb640 10 client.22533 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:41.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.087+0000 7f1b6e59c640 1 --2- 172.21.15.120:0/2633360549 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556930b0e1f0 0x556930b105e0 crc :-1 s=READY pgs=539 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2032de69d684eb52 server_cookie=426dd4ede7b65fe3 in_seq=0 out_seq=0 2024-09-19T09:24:41.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.088+0000 7f1b55ffb640 1 -- 172.21.15.120:0/2633360549 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556930a64850 con 0x556930b0e1f0 2024-09-19T09:24:41.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.088+0000 7f1b55ffb640 10 client.22533 handle_command_reply: tid=0 2024-09-19T09:24:41.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.088+0000 7f1b77087740 1 client.22533 shutdown 2024-09-19T09:24:41.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.088+0000 7f1b77087740 1 -- 172.21.15.120:0/2633360549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556930af1c80 msgr2=0x556930afb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:41.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.088+0000 7f1b77087740 1 --2- 172.21.15.120:0/2633360549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556930af1c80 0x556930afb550 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400b710 tx=0x7f1b64004410 comp rx=0 tx=0).stop 2024-09-19T09:24:41.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.088+0000 7f1b77087740 1 -- 172.21.15.120:0/2633360549 shutdown_connections 2024-09-19T09:24:41.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.088+0000 7f1b77087740 1 --2- 172.21.15.120:0/2633360549 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556930b0e1f0 0x556930b105e0 crc :-1 s=READY pgs=539 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.088+0000 7f1b77087740 1 --2- 172.21.15.120:0/2633360549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556930afe890 0x556930afbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.088+0000 7f1b77087740 1 --2- 172.21.15.120:0/2633360549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556930af1c80 0x556930afb550 unknown :-1 s=CLOSED pgs=2253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.088+0000 7f1b77087740 1 --2- 172.21.15.120:0/2633360549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556930af1360 0x556930afe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.088+0000 7f1b77087740 1 -- 172.21.15.120:0/2633360549 >> 172.21.15.120:0/2633360549 conn(0x556930aeda30 msgr2=0x556930af3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:41.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.088+0000 7f1b55ffb640 0 client.22533 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:41.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.088+0000 7f1b77087740 1 -- 172.21.15.120:0/2633360549 shutdown_connections 2024-09-19T09:24:41.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.088+0000 7f1b77087740 1 -- 172.21.15.120:0/2633360549 wait complete. 2024-09-19T09:24:41.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.089+0000 7f1b77087740 20 client.22533 trim_cache size 0 max 16384 2024-09-19T09:24:41.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.105+0000 7f1b77087740 1 Processor -- start 2024-09-19T09:24:41.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.105+0000 7f1b77087740 1 -- start start 2024-09-19T09:24:41.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.105+0000 7f1b77087740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556930af1360 0x556930af9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:41.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.105+0000 7f1b77087740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556930af1c80 0x556930aeca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:41.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.105+0000 7f1b77087740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556930afe890 0x556930b06c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:41.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.105+0000 7f1b77087740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701ab2e0 con 0x556930af1360 2024-09-19T09:24:41.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.105+0000 7f1b77087740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701ace20 con 0x556930afe890 2024-09-19T09:24:41.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.105+0000 7f1b77087740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701ab9b0 con 0x556930af1c80 2024-09-19T09:24:41.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.105+0000 7f1b6e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556930af1360 0x556930af9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:41.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.105+0000 7f1b6e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556930af1360 0x556930af9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39934/0 (socket says 172.21.15.120:39934) 2024-09-19T09:24:41.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.105+0000 7f1b6e59c640 1 -- 172.21.15.120:0/3230588660 learned_addr learned my addr 172.21.15.120:0/3230588660 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:41.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.106+0000 7f1b6dd9b640 1 --2- 172.21.15.120:0/3230588660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556930af1c80 0x556930aeca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:41.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.106+0000 7f1b6e59c640 1 -- 172.21.15.120:0/3230588660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556930af1c80 msgr2=0x556930aeca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:41.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.106+0000 7f1b6e59c640 1 --2- 172.21.15.120:0/3230588660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556930af1c80 0x556930aeca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.106+0000 7f1b6e59c640 1 -- 172.21.15.120:0/3230588660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556930afe890 msgr2=0x556930b06c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:41.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.106+0000 7f1b6e59c640 1 --2- 172.21.15.120:0/3230588660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556930afe890 0x556930b06c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.106+0000 7f1b6e59c640 1 -- 172.21.15.120:0/3230588660 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556930afa500 con 0x556930af1360 2024-09-19T09:24:41.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.106+0000 7f1b6e59c640 1 --2- 172.21.15.120:0/3230588660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556930af1360 0x556930af9e20 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f1b600b8890 tx=0x7f1b6008f2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb04e420304d158 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:41.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.106+0000 7f1b57fff640 1 -- 172.21.15.120:0/3230588660 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60049040 con 0x556930af1360 2024-09-19T09:24:41.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.106+0000 7f1b57fff640 1 -- 172.21.15.120:0/3230588660 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b600c50a0 con 0x556930af1360 2024-09-19T09:24:41.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.106+0000 7f1b57fff640 1 -- 172.21.15.120:0/3230588660 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6003b560 con 0x556930af1360 2024-09-19T09:24:41.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.107+0000 7f1b77087740 1 -- 172.21.15.120:0/3230588660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556930af1360 msgr2=0x556930af9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:41.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.107+0000 7f1b77087740 1 --2- 172.21.15.120:0/3230588660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556930af1360 0x556930af9e20 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f1b600b8890 tx=0x7f1b6008f2b0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.107+0000 7f1b77087740 1 -- 172.21.15.120:0/3230588660 shutdown_connections 2024-09-19T09:24:41.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.107+0000 7f1b77087740 1 --2- 172.21.15.120:0/3230588660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556930afe890 0x556930b06c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.107+0000 7f1b77087740 1 --2- 172.21.15.120:0/3230588660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556930af1c80 0x556930aeca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.107+0000 7f1b77087740 1 --2- 172.21.15.120:0/3230588660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556930af1360 0x556930af9e20 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.107+0000 7f1b77087740 1 -- 172.21.15.120:0/3230588660 >> 172.21.15.120:0/3230588660 conn(0x556930aeda30 msgr2=0x556930aec350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:41.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.107+0000 7f1b77087740 1 -- 172.21.15.120:0/3230588660 shutdown_connections 2024-09-19T09:24:41.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.107+0000 7f1b77087740 1 -- 172.21.15.120:0/3230588660 wait complete. 2024-09-19T09:24:41.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.107+0000 7f1b77087740 1 -- start start 2024-09-19T09:24:41.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.107+0000 7f1b77087740 1 Processor -- start 2024-09-19T09:24:41.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.107+0000 7f1b77087740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556930af1360 0x556930af6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:41.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.107+0000 7f1b77087740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556930af1c80 0x556930af3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:41.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.107+0000 7f1b77087740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556930afe890 0x556930af4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:41.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.107+0000 7f1b77087740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701ac750 con 0x556930afe890 2024-09-19T09:24:41.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.107+0000 7f1b77087740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701ac080 con 0x556930af1c80 2024-09-19T09:24:41.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.107+0000 7f1b77087740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701a92a0 con 0x556930af1360 2024-09-19T09:24:41.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.107+0000 7f1b77087740 10 client.0 _ll_register_callbacks cb 0x7f1b68007810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:41.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.108+0000 7f1b6ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556930afe890 0x556930af4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:41.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.108+0000 7f1b6ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556930afe890 0x556930af4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39936/0 (socket says 172.21.15.120:39936) 2024-09-19T09:24:41.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.108+0000 7f1b6e59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556930af1360 0x556930af6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:41.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.108+0000 7f1b6dd9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556930af1c80 0x556930af3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:41.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.108+0000 7f1b6ed9d640 1 -- 172.21.15.120:0/10694302 learned_addr learned my addr 172.21.15.120:0/10694302 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:41.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.108+0000 7f1b6ed9d640 1 -- 172.21.15.120:0/10694302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556930af1360 msgr2=0x556930af6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:41.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.108+0000 7f1b6ed9d640 1 --2- 172.21.15.120:0/10694302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556930af1360 0x556930af6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.108+0000 7f1b6ed9d640 1 -- 172.21.15.120:0/10694302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556930af1c80 msgr2=0x556930af3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:41.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.108+0000 7f1b6ed9d640 1 --2- 172.21.15.120:0/10694302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556930af1c80 0x556930af3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.108+0000 7f1b6ed9d640 1 -- 172.21.15.120:0/10694302 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556930afbed0 con 0x556930afe890 2024-09-19T09:24:41.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.108+0000 7f1b6ed9d640 1 --2- 172.21.15.120:0/10694302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556930afe890 0x556930af4200 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f1b68012fd0 tx=0x7f1b680079b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b34343153cd2ad6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:41.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.108+0000 7f1b55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:41.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.108+0000 7f1b55ffb640 1 -- 172.21.15.120:0/10694302 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68018070 con 0x556930afe890 2024-09-19T09:24:41.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.108+0000 7f1b55ffb640 1 -- 172.21.15.120:0/10694302 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f1b68011870 con 0x556930afe890 2024-09-19T09:24:41.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.108+0000 7f1b77087740 1 -- 172.21.15.120:0/10694302 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b600c50a0 con 0x556930afe890 2024-09-19T09:24:41.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.108+0000 7f1b55ffb640 1 -- 172.21.15.120:0/10694302 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1b68011a70 con 0x556930afe890 2024-09-19T09:24:41.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.108+0000 7f1b77087740 10 client.22539 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:41.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.108+0000 7f1b77087740 1 -- 172.21.15.120:0/10694302 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556930b7a780 con 0x556930afe890 2024-09-19T09:24:41.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.109+0000 7f1b55ffb640 1 -- 172.21.15.120:0/10694302 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b600c50a0 con 0x556930afe890 2024-09-19T09:24:41.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.109+0000 7f1b55ffb640 1 -- 172.21.15.120:0/10694302 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f1b6800e440 con 0x556930afe890 2024-09-19T09:24:41.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.109+0000 7f1b77087740 10 client.22539 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:41.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.109+0000 7f1b77087740 10 client.22539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:24:41.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.109+0000 7f1b77087740 20 client.22539 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:41.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.109+0000 7f1b77087740 1 --2- 172.21.15.120:0/10694302 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556930b7aed0 0x556930b7d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:41.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.109+0000 7f1b77087740 4 client.22539 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:24:41.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.109+0000 7f1b77087740 1 -- 172.21.15.120:0/10694302 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556930a5ed60 con 0x556930b7aed0 2024-09-19T09:24:41.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.109+0000 7f1b6e59c640 1 --2- 172.21.15.120:0/10694302 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556930b7aed0 0x556930b7d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:41.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.109+0000 7f1b55ffb640 10 client.22539 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:41.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.109+0000 7f1b6e59c640 1 --2- 172.21.15.120:0/10694302 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556930b7aed0 0x556930b7d2f0 crc :-1 s=READY pgs=540 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e544a7b667d2826 server_cookie=d3909b6ba7585913 in_seq=0 out_seq=0 2024-09-19T09:24:41.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.110+0000 7f1b55ffb640 1 -- 172.21.15.120:0/10694302 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556930a5ed60 con 0x556930b7aed0 2024-09-19T09:24:41.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.110+0000 7f1b55ffb640 10 client.22539 handle_command_reply: tid=0 2024-09-19T09:24:41.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.110+0000 7f1b77087740 1 client.22539 shutdown 2024-09-19T09:24:41.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.110+0000 7f1b77087740 1 -- 172.21.15.120:0/10694302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556930afe890 msgr2=0x556930af4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:41.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.110+0000 7f1b77087740 1 --2- 172.21.15.120:0/10694302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556930afe890 0x556930af4200 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f1b68012fd0 tx=0x7f1b680079b0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.110+0000 7f1b77087740 1 -- 172.21.15.120:0/10694302 shutdown_connections 2024-09-19T09:24:41.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.110+0000 7f1b77087740 1 --2- 172.21.15.120:0/10694302 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556930b7aed0 0x556930b7d2f0 crc :-1 s=READY pgs=540 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.110+0000 7f1b77087740 1 --2- 172.21.15.120:0/10694302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556930afe890 0x556930af4200 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.110+0000 7f1b55ffb640 0 client.22539 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:41.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.110+0000 7f1b77087740 1 --2- 172.21.15.120:0/10694302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556930af1c80 0x556930af3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.110+0000 7f1b77087740 1 --2- 172.21.15.120:0/10694302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556930af1360 0x556930af6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.110+0000 7f1b77087740 1 -- 172.21.15.120:0/10694302 >> 172.21.15.120:0/10694302 conn(0x556930aeda30 msgr2=0x556930afd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:41.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.110+0000 7f1b77087740 1 -- 172.21.15.120:0/10694302 shutdown_connections 2024-09-19T09:24:41.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.110+0000 7f1b77087740 1 -- 172.21.15.120:0/10694302 wait complete. 2024-09-19T09:24:41.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.110+0000 7f1b77087740 20 client.22539 trim_cache size 0 max 16384 2024-09-19T09:24:41.119 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:24:41.119 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:24:41.119 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:24:41.120 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:24:41.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.111+0000 7f1b74cae640 1 -- 172.21.15.120:0/2423045031 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1b580a3240 msgr2=0x7f1b580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:41.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.111+0000 7f1b74cae640 1 --2- 172.21.15.120:0/2423045031 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1b580a3240 0x7f1b580a5700 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1b68007310 tx=0x7f1b680072a0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.111+0000 7f1b74cae640 1 -- 172.21.15.120:0/2423045031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b70100a80 msgr2=0x7f1b701972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:41.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.111+0000 7f1b74cae640 1 --2- 172.21.15.120:0/2423045031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b70100a80 0x7f1b701972c0 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f1b60009230 tx=0x7f1b60004270 comp rx=0 tx=0).stop 2024-09-19T09:24:41.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.111+0000 7f1b74cae640 1 -- 172.21.15.120:0/2423045031 shutdown_connections 2024-09-19T09:24:41.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.111+0000 7f1b74cae640 1 --2- 172.21.15.120:0/2423045031 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1b580a3240 0x7f1b580a5700 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.111+0000 7f1b74cae640 1 --2- 172.21.15.120:0/2423045031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b70106a80 0x7f1b7019bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.111+0000 7f1b74cae640 1 --2- 172.21.15.120:0/2423045031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b70101420 0x7f1b70197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.111+0000 7f1b74cae640 1 --2- 172.21.15.120:0/2423045031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b70100a80 0x7f1b701972c0 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.111+0000 7f1b74cae640 1 -- 172.21.15.120:0/2423045031 >> 172.21.15.120:0/2423045031 conn(0x7f1b700fc7a0 msgr2=0x7f1b700fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:41.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.111+0000 7f1b74cae640 1 -- 172.21.15.120:0/2423045031 shutdown_connections 2024-09-19T09:24:41.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:41.111+0000 7f1b74cae640 1 -- 172.21.15.120:0/2423045031 wait complete. 2024-09-19T09:24:41.381 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199337 19.55 MB/sec execute 396 sec latency 13891.041 ms 2024-09-19T09:24:41.961 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:24:41.961 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:24:42.251 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:42.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:42 smithi120 ceph-mon[31150]: pgmap v1000: 97 pgs: 97 active+clean; 431 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.2 MiB/s wr, 178 op/s 2024-09-19T09:24:42.381 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199337 19.50 MB/sec execute 397 sec latency 14891.109 ms 2024-09-19T09:24:42.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:42 smithi196 ceph-mon[35150]: pgmap v1000: 97 pgs: 97 active+clean; 431 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.2 MiB/s wr, 178 op/s 2024-09-19T09:24:42.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:42 smithi153 ceph-mon[35071]: pgmap v1000: 97 pgs: 97 active+clean; 431 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.2 MiB/s wr, 178 op/s 2024-09-19T09:24:42.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.703+0000 7fb3a1546640 1 -- 172.21.15.120:0/1524259131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb39c102570 msgr2=0x7fb39c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:42.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.703+0000 7fb3a1546640 1 --2- 172.21.15.120:0/1524259131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb39c102570 0x7fb39c1029b0 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7fb384009900 tx=0x7fb38402a480 comp rx=0 tx=0).stop 2024-09-19T09:24:42.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.703+0000 7fb3a1546640 1 -- 172.21.15.120:0/1524259131 shutdown_connections 2024-09-19T09:24:42.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.703+0000 7fb3a1546640 1 --2- 172.21.15.120:0/1524259131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb39c102ef0 0x7fb39c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:42.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.703+0000 7fb3a1546640 1 --2- 172.21.15.120:0/1524259131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb39c102570 0x7fb39c1029b0 unknown :-1 s=CLOSED pgs=2256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:42.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.703+0000 7fb3a1546640 1 --2- 172.21.15.120:0/1524259131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb39c100e20 0x7fb39c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:42.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.703+0000 7fb3a1546640 1 -- 172.21.15.120:0/1524259131 >> 172.21.15.120:0/1524259131 conn(0x7fb39c0fc7d0 msgr2=0x7fb39c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:42.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.703+0000 7fb3a1546640 1 -- 172.21.15.120:0/1524259131 shutdown_connections 2024-09-19T09:24:42.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.703+0000 7fb3a1546640 1 -- 172.21.15.120:0/1524259131 wait complete. 2024-09-19T09:24:42.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.703+0000 7fb3a1546640 1 Processor -- start 2024-09-19T09:24:42.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.703+0000 7fb3a1546640 1 -- start start 2024-09-19T09:24:42.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.704+0000 7fb3a1546640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb39c100e20 0x7fb39c1970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:42.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.704+0000 7fb3a1546640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb39c102570 0x7fb39c197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:42.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.704+0000 7fb3a1546640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb39c102ef0 0x7fb39c197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:42.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.704+0000 7fb3a1546640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c110f80 con 0x7fb39c102570 2024-09-19T09:24:42.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.704+0000 7fb3a1546640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c110e00 con 0x7fb39c102ef0 2024-09-19T09:24:42.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.704+0000 7fb3a1546640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c111100 con 0x7fb39c100e20 2024-09-19T09:24:42.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.704+0000 7fb3927fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb39c102570 0x7fb39c197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:42.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.704+0000 7fb3927fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb39c102570 0x7fb39c197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49498/0 (socket says 172.21.15.120:49498) 2024-09-19T09:24:42.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.704+0000 7fb3927fc640 1 -- 172.21.15.120:0/2741833510 learned_addr learned my addr 172.21.15.120:0/2741833510 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:42.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.704+0000 7fb39b7fe640 1 --2- 172.21.15.120:0/2741833510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb39c102ef0 0x7fb39c197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:42.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.704+0000 7fb39affd640 1 --2- 172.21.15.120:0/2741833510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb39c100e20 0x7fb39c1970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:42.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.704+0000 7fb3927fc640 1 -- 172.21.15.120:0/2741833510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb39c100e20 msgr2=0x7fb39c1970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:42.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.704+0000 7fb3927fc640 1 --2- 172.21.15.120:0/2741833510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb39c100e20 0x7fb39c1970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:42.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.704+0000 7fb3927fc640 1 -- 172.21.15.120:0/2741833510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb39c102ef0 msgr2=0x7fb39c197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:42.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.704+0000 7fb3927fc640 1 --2- 172.21.15.120:0/2741833510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb39c102ef0 0x7fb39c197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:42.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.704+0000 7fb3927fc640 1 -- 172.21.15.120:0/2741833510 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb39c1a6950 con 0x7fb39c102570 2024-09-19T09:24:42.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.704+0000 7fb3927fc640 1 --2- 172.21.15.120:0/2741833510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb39c102570 0x7fb39c197620 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7fb384006010 tx=0x7fb384003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:42.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.704+0000 7fb398ff9640 1 -- 172.21.15.120:0/2741833510 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38403f070 con 0x7fb39c102570 2024-09-19T09:24:42.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.704+0000 7fb3a1546640 1 -- 172.21.15.120:0/2741833510 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb39c1a6be0 con 0x7fb39c102570 2024-09-19T09:24:42.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.704+0000 7fb398ff9640 1 -- 172.21.15.120:0/2741833510 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb384031c30 con 0x7fb39c102570 2024-09-19T09:24:42.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.704+0000 7fb398ff9640 1 -- 172.21.15.120:0/2741833510 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb384035030 con 0x7fb39c102570 2024-09-19T09:24:42.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.704+0000 7fb3a1546640 1 -- 172.21.15.120:0/2741833510 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb39c1a7110 con 0x7fb39c102570 2024-09-19T09:24:42.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.705+0000 7fb3a1546640 1 -- 172.21.15.120:0/2741833510 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb39c101200 con 0x7fb39c102570 2024-09-19T09:24:42.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.706+0000 7fb398ff9640 1 -- 172.21.15.120:0/2741833510 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fb384042070 con 0x7fb39c102570 2024-09-19T09:24:42.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.707+0000 7fb398ff9640 1 --2- 172.21.15.120:0/2741833510 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb3880a3240 0x7fb3880a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:42.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.707+0000 7fb39affd640 1 --2- 172.21.15.120:0/2741833510 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb3880a3240 0x7fb3880a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:42.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.707+0000 7fb398ff9640 1 -- 172.21.15.120:0/2741833510 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fb3840f0170 con 0x7fb39c102570 2024-09-19T09:24:42.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.707+0000 7fb39affd640 1 --2- 172.21.15.120:0/2741833510 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb3880a3240 0x7fb3880a5700 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb38c005e10 tx=0x7fb38c005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:42.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.711+0000 7fb398ff9640 1 -- 172.21.15.120:0/2741833510 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb384031de0 con 0x7fb39c102570 2024-09-19T09:24:42.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.910+0000 7fb3a1546640 1 -- 172.21.15.120:0/2741833510 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb39c1a72e0 con 0x7fb39c102570 2024-09-19T09:24:42.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.911+0000 7fb398ff9640 1 -- 172.21.15.120:0/2741833510 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11988 (secure 0 0 0) 0x7fb3840bce90 con 0x7fb39c102570 2024-09-19T09:24:42.912 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:24:42.912 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":39,"btime":"2024-09-19T09:18:53:120858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:18:53.070639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:24:42.913 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 39 2024-09-19T09:24:42.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.913+0000 7fb3a1546640 1 -- 172.21.15.120:0/2741833510 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb3880a3240 msgr2=0x7fb3880a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:42.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.913+0000 7fb3a1546640 1 --2- 172.21.15.120:0/2741833510 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb3880a3240 0x7fb3880a5700 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb38c005e10 tx=0x7fb38c005da0 comp rx=0 tx=0).stop 2024-09-19T09:24:42.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.913+0000 7fb3a1546640 1 -- 172.21.15.120:0/2741833510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb39c102570 msgr2=0x7fb39c197620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:42.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.913+0000 7fb3a1546640 1 --2- 172.21.15.120:0/2741833510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb39c102570 0x7fb39c197620 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7fb384006010 tx=0x7fb384003c10 comp rx=0 tx=0).stop 2024-09-19T09:24:42.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.913+0000 7fb3a1546640 1 -- 172.21.15.120:0/2741833510 shutdown_connections 2024-09-19T09:24:42.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.913+0000 7fb3a1546640 1 --2- 172.21.15.120:0/2741833510 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb3880a3240 0x7fb3880a5700 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:42.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.913+0000 7fb3a1546640 1 --2- 172.21.15.120:0/2741833510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb39c102ef0 0x7fb39c197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:42.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.913+0000 7fb3a1546640 1 --2- 172.21.15.120:0/2741833510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb39c102570 0x7fb39c197620 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:42.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.913+0000 7fb3a1546640 1 --2- 172.21.15.120:0/2741833510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb39c100e20 0x7fb39c1970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:42.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.913+0000 7fb3a1546640 1 -- 172.21.15.120:0/2741833510 >> 172.21.15.120:0/2741833510 conn(0x7fb39c0fc7d0 msgr2=0x7fb39c106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:42.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.913+0000 7fb3a1546640 1 -- 172.21.15.120:0/2741833510 shutdown_connections 2024-09-19T09:24:42.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:42.913+0000 7fb3a1546640 1 -- 172.21.15.120:0/2741833510 wait complete. 2024-09-19T09:24:43.287 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:43 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2741833510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:24:43.381 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199337 19.45 MB/sec execute 398 sec latency 15891.191 ms 2024-09-19T09:24:43.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:43 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2741833510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:24:43.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:43 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2741833510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:24:43.624 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:24:43.869 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:44.283 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:44 smithi120 ceph-mon[31150]: pgmap v1001: 97 pgs: 97 active+clean; 426 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 2.0 MiB/s wr, 133 op/s 2024-09-19T09:24:44.382 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199337 19.40 MB/sec execute 399 sec latency 16891.230 ms 2024-09-19T09:24:44.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.362+0000 7f78e34d2640 1 -- 172.21.15.120:0/520293827 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78dc101de0 msgr2=0x7f78dc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:44.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.362+0000 7f78e34d2640 1 --2- 172.21.15.120:0/520293827 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78dc101de0 0x7f78dc1021c0 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f78d8009900 tx=0x7f78d802a410 comp rx=0 tx=0).stop 2024-09-19T09:24:44.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.362+0000 7f78e34d2640 1 -- 172.21.15.120:0/520293827 shutdown_connections 2024-09-19T09:24:44.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.362+0000 7f78e34d2640 1 --2- 172.21.15.120:0/520293827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78dc106f10 0x7f78dc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.362+0000 7f78e34d2640 1 --2- 172.21.15.120:0/520293827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78dc102790 0x7f78dc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.362+0000 7f78e34d2640 1 --2- 172.21.15.120:0/520293827 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78dc101de0 0x7f78dc1021c0 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.362+0000 7f78e34d2640 1 -- 172.21.15.120:0/520293827 >> 172.21.15.120:0/520293827 conn(0x7f78dc0fd8c0 msgr2=0x7f78dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:44.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.362+0000 7f78e34d2640 1 -- 172.21.15.120:0/520293827 shutdown_connections 2024-09-19T09:24:44.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.362+0000 7f78e34d2640 1 -- 172.21.15.120:0/520293827 wait complete. 2024-09-19T09:24:44.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.363+0000 7f78e34d2640 1 Processor -- start 2024-09-19T09:24:44.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.363+0000 7f78e34d2640 1 -- start start 2024-09-19T09:24:44.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.363+0000 7f78e34d2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78dc101de0 0x7f78dc1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:44.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.363+0000 7f78e34d2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78dc102790 0x7f78dc198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:44.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.363+0000 7f78e34d2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78dc106f10 0x7f78dc19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:44.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.363+0000 7f78e34d2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc112010 con 0x7f78dc102790 2024-09-19T09:24:44.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.363+0000 7f78e34d2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc111e90 con 0x7f78dc106f10 2024-09-19T09:24:44.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.363+0000 7f78e34d2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc112190 con 0x7f78dc101de0 2024-09-19T09:24:44.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.363+0000 7f78e0a6d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78dc102790 0x7f78dc198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:44.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.363+0000 7f78e0a6d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78dc102790 0x7f78dc198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49516/0 (socket says 172.21.15.120:49516) 2024-09-19T09:24:44.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.363+0000 7f78e0a6d640 1 -- 172.21.15.120:0/2690345923 learned_addr learned my addr 172.21.15.120:0/2690345923 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:44.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.363+0000 7f78e126e640 1 --2- 172.21.15.120:0/2690345923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78dc101de0 0x7f78dc1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:44.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.363+0000 7f78e0a6d640 1 -- 172.21.15.120:0/2690345923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78dc101de0 msgr2=0x7f78dc1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:44.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.363+0000 7f78e1a6f640 1 --2- 172.21.15.120:0/2690345923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78dc106f10 0x7f78dc19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:44.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.363+0000 7f78e0a6d640 1 --2- 172.21.15.120:0/2690345923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78dc101de0 0x7f78dc1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.363+0000 7f78e0a6d640 1 -- 172.21.15.120:0/2690345923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78dc106f10 msgr2=0x7f78dc19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:44.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.363+0000 7f78e0a6d640 1 --2- 172.21.15.120:0/2690345923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78dc106f10 0x7f78dc19ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.363+0000 7f78e0a6d640 1 -- 172.21.15.120:0/2690345923 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78dc19d560 con 0x7f78dc102790 2024-09-19T09:24:44.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.363+0000 7f78e126e640 1 --2- 172.21.15.120:0/2690345923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78dc101de0 0x7f78dc1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:24:44.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.364+0000 7f78e0a6d640 1 --2- 172.21.15.120:0/2690345923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78dc102790 0x7f78dc198af0 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f78cc00b4a0 tx=0x7f78cc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:44.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.364+0000 7f78d27fc640 1 -- 172.21.15.120:0/2690345923 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc014070 con 0x7f78dc102790 2024-09-19T09:24:44.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.364+0000 7f78e34d2640 1 -- 172.21.15.120:0/2690345923 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78dc19d850 con 0x7f78dc102790 2024-09-19T09:24:44.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.364+0000 7f78d27fc640 1 -- 172.21.15.120:0/2690345923 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78cc010af0 con 0x7f78dc102790 2024-09-19T09:24:44.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.364+0000 7f78d27fc640 1 -- 172.21.15.120:0/2690345923 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc017d10 con 0x7f78dc102790 2024-09-19T09:24:44.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.364+0000 7f78e34d2640 1 -- 172.21.15.120:0/2690345923 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78dc1a5130 con 0x7f78dc102790 2024-09-19T09:24:44.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.366+0000 7f78d27fc640 1 -- 172.21.15.120:0/2690345923 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f78cc017070 con 0x7f78dc102790 2024-09-19T09:24:44.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.366+0000 7f78e58ab740 1 Processor -- start 2024-09-19T09:24:44.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.367+0000 7f78e58ab740 1 -- start start 2024-09-19T09:24:44.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.367+0000 7f78e58ab740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e30af7b40 0x556e30af6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:44.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.367+0000 7f78e58ab740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e30af65d0 0x556e30af6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:44.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.367+0000 7f78e58ab740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e30b05780 0x556e30b07b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:44.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.367+0000 7f78e58ab740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556e30a5c2f0 con 0x556e30af7b40 2024-09-19T09:24:44.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.367+0000 7f78e58ab740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556e30a03cf0 con 0x556e30af65d0 2024-09-19T09:24:44.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.367+0000 7f78e58ab740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556e30a582b0 con 0x556e30b05780 2024-09-19T09:24:44.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.367+0000 7f78e1a6f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e30af7b40 0x556e30af6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:44.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.367+0000 7f78e1a6f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e30af7b40 0x556e30af6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49518/0 (socket says 172.21.15.120:49518) 2024-09-19T09:24:44.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.367+0000 7f78e1a6f640 1 -- 172.21.15.120:0/3512486908 learned_addr learned my addr 172.21.15.120:0/3512486908 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:44.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.367+0000 7f78d27fc640 1 --2- 172.21.15.120:0/2690345923 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f78c80a3240 0x7f78c80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:44.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.367+0000 7f78e0a6d640 1 --2- 172.21.15.120:0/3512486908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e30b05780 0x556e30b07b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:44.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.367+0000 7f78e126e640 1 --2- 172.21.15.120:0/3512486908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e30af65d0 0x556e30af6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:44.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.367+0000 7f78e1a6f640 1 -- 172.21.15.120:0/3512486908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e30b05780 msgr2=0x556e30b07b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:44.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.367+0000 7f78e1a6f640 1 --2- 172.21.15.120:0/3512486908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e30b05780 0x556e30b07b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.367+0000 7f78e1a6f640 1 -- 172.21.15.120:0/3512486908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e30af65d0 msgr2=0x556e30af6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:44.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.367+0000 7f78e1a6f640 1 --2- 172.21.15.120:0/3512486908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e30af65d0 0x556e30af6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.367+0000 7f78e1a6f640 1 -- 172.21.15.120:0/3512486908 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e309e83a0 con 0x556e30af7b40 2024-09-19T09:24:44.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.367+0000 7f78d27fc640 1 -- 172.21.15.120:0/2690345923 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f78cc0cee00 con 0x7f78dc102790 2024-09-19T09:24:44.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.367+0000 7f78e1a6f640 1 --2- 172.21.15.120:0/2690345923 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f78c80a3240 0x7f78c80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:44.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.368+0000 7f78e1a6f640 1 --2- 172.21.15.120:0/3512486908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e30af7b40 0x556e30af6090 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f78d4007f10 tx=0x7f78d400c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef562e1c48b8fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:44.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.368+0000 7f78c77fe640 1 -- 172.21.15.120:0/3512486908 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d4017040 con 0x556e30af7b40 2024-09-19T09:24:44.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.368+0000 7f78c77fe640 1 -- 172.21.15.120:0/3512486908 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78d401ee00 con 0x556e30af7b40 2024-09-19T09:24:44.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.368+0000 7f78c77fe640 1 -- 172.21.15.120:0/3512486908 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d401dd00 con 0x556e30af7b40 2024-09-19T09:24:44.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.368+0000 7f78e58ab740 1 -- 172.21.15.120:0/3512486908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e30af7b40 msgr2=0x556e30af6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:44.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.368+0000 7f78e58ab740 1 --2- 172.21.15.120:0/3512486908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e30af7b40 0x556e30af6090 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f78d4007f10 tx=0x7f78d400c560 comp rx=0 tx=0).stop 2024-09-19T09:24:44.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.368+0000 7f78e1a6f640 1 --2- 172.21.15.120:0/2690345923 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f78c80a3240 0x7f78c80a5700 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f78d401d000 tx=0x7f78d401e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:44.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.368+0000 7f78e58ab740 1 -- 172.21.15.120:0/3512486908 shutdown_connections 2024-09-19T09:24:44.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.368+0000 7f78e58ab740 1 --2- 172.21.15.120:0/3512486908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e30b05780 0x556e30b07b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.368+0000 7f78e58ab740 1 --2- 172.21.15.120:0/3512486908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e30af65d0 0x556e30af6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.368+0000 7f78e58ab740 1 --2- 172.21.15.120:0/3512486908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e30af7b40 0x556e30af6090 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.368+0000 7f78e58ab740 1 -- 172.21.15.120:0/3512486908 >> 172.21.15.120:0/3512486908 conn(0x556e30abf310 msgr2=0x556e30af2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:44.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.369+0000 7f78e58ab740 1 -- 172.21.15.120:0/3512486908 shutdown_connections 2024-09-19T09:24:44.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.369+0000 7f78e58ab740 1 -- 172.21.15.120:0/3512486908 wait complete. 2024-09-19T09:24:44.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.369+0000 7f78e58ab740 1 -- start start 2024-09-19T09:24:44.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.369+0000 7f78e58ab740 1 Processor -- start 2024-09-19T09:24:44.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.369+0000 7f78e58ab740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e30af65d0 0x556e30b05180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:44.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.369+0000 7f78e58ab740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e30af7b40 0x556e30b02430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:44.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.369+0000 7f78e58ab740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e30b05780 0x556e30b02a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:44.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.370+0000 7f78e58ab740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556e30b08480 con 0x556e30b05780 2024-09-19T09:24:44.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.370+0000 7f78e58ab740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556e30a17060 con 0x556e30af7b40 2024-09-19T09:24:44.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.370+0000 7f78e58ab740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556e30b08600 con 0x556e30af65d0 2024-09-19T09:24:44.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.370+0000 7f78e58ab740 10 client.0 _ll_register_callbacks cb 0x556e3076a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:44.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.370+0000 7f78e1a6f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e30b05780 0x556e30b02a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:44.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.370+0000 7f78e1a6f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e30b05780 0x556e30b02a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49528/0 (socket says 172.21.15.120:49528) 2024-09-19T09:24:44.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.370+0000 7f78e1a6f640 1 -- 172.21.15.120:0/3250254065 learned_addr learned my addr 172.21.15.120:0/3250254065 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:44.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.370+0000 7f78e126e640 1 --2- 172.21.15.120:0/3250254065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e30af65d0 0x556e30b05180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:44.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.370+0000 7f78e0a6d640 1 --2- 172.21.15.120:0/3250254065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e30af7b40 0x556e30b02430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:44.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.370+0000 7f78e1a6f640 1 -- 172.21.15.120:0/3250254065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e30af65d0 msgr2=0x556e30b05180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:44.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.370+0000 7f78e1a6f640 1 --2- 172.21.15.120:0/3250254065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e30af65d0 0x556e30b05180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.370+0000 7f78e1a6f640 1 -- 172.21.15.120:0/3250254065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e30af7b40 msgr2=0x556e30b02430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:44.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.370+0000 7f78e1a6f640 1 --2- 172.21.15.120:0/3250254065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e30af7b40 0x556e30b02430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.370+0000 7f78e1a6f640 1 -- 172.21.15.120:0/3250254065 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e307cd3c0 con 0x556e30b05780 2024-09-19T09:24:44.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.370+0000 7f78e0a6d640 1 --2- 172.21.15.120:0/3250254065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e30af7b40 0x556e30b02430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:24:44.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.370+0000 7f78e1a6f640 1 --2- 172.21.15.120:0/3250254065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e30b05780 0x556e30b02a20 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f78d401d6c0 tx=0x7f78d4007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe527e183a62d397 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:44.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.370+0000 7f78c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:44.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.370+0000 7f78c4ff9640 1 -- 172.21.15.120:0/3250254065 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d402c070 con 0x556e30b05780 2024-09-19T09:24:44.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.370+0000 7f78c4ff9640 1 -- 172.21.15.120:0/3250254065 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f78d401dac0 con 0x556e30b05780 2024-09-19T09:24:44.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.370+0000 7f78e58ab740 1 -- 172.21.15.120:0/3250254065 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78d401ee00 con 0x556e30b05780 2024-09-19T09:24:44.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.370+0000 7f78c4ff9640 1 -- 172.21.15.120:0/3250254065 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f78d4038050 con 0x556e30b05780 2024-09-19T09:24:44.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.370+0000 7f78e58ab740 10 client.22557 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:44.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.370+0000 7f78e58ab740 1 -- 172.21.15.120:0/3250254065 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556e30b13f20 con 0x556e30b05780 2024-09-19T09:24:44.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.371+0000 7f78c4ff9640 1 -- 172.21.15.120:0/3250254065 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78d401ee00 con 0x556e30b05780 2024-09-19T09:24:44.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.371+0000 7f78c4ff9640 1 -- 172.21.15.120:0/3250254065 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f78d4025050 con 0x556e30b05780 2024-09-19T09:24:44.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.371+0000 7f78e58ab740 10 client.22557 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:44.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.371+0000 7f78e58ab740 10 client.22557 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:24:44.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.371+0000 7f78e58ab740 20 client.22557 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:44.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.371+0000 7f78e58ab740 1 --2- 172.21.15.120:0/3250254065 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x556e30b15270 0x556e30b17660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:44.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.371+0000 7f78e58ab740 4 client.22557 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:24:44.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.371+0000 7f78e58ab740 1 -- 172.21.15.120:0/3250254065 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e30a69860 con 0x556e30b15270 2024-09-19T09:24:44.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.371+0000 7f78e126e640 1 --2- 172.21.15.120:0/3250254065 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x556e30b15270 0x556e30b17660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:44.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.372+0000 7f78e126e640 1 --2- 172.21.15.120:0/3250254065 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x556e30b15270 0x556e30b17660 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c878f007d5cc96ca server_cookie=6d45c9054c0ba46c in_seq=0 out_seq=0 2024-09-19T09:24:44.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.372+0000 7f78c4ff9640 10 client.22557 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:24:44.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.372+0000 7f78c4ff9640 1 -- 172.21.15.120:0/3250254065 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556e30a69860 con 0x556e30b15270 2024-09-19T09:24:44.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.372+0000 7f78c4ff9640 10 client.22557 handle_command_reply: tid=0 2024-09-19T09:24:44.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.372+0000 7f78e58ab740 1 client.22557 shutdown 2024-09-19T09:24:44.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.373+0000 7f78e58ab740 1 -- 172.21.15.120:0/3250254065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e30b05780 msgr2=0x556e30b02a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:44.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.373+0000 7f78e58ab740 1 --2- 172.21.15.120:0/3250254065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e30b05780 0x556e30b02a20 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f78d401d6c0 tx=0x7f78d4007680 comp rx=0 tx=0).stop 2024-09-19T09:24:44.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.373+0000 7f78e58ab740 1 -- 172.21.15.120:0/3250254065 shutdown_connections 2024-09-19T09:24:44.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.373+0000 7f78e58ab740 1 --2- 172.21.15.120:0/3250254065 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x556e30b15270 0x556e30b17660 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.373+0000 7f78e58ab740 1 --2- 172.21.15.120:0/3250254065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e30b05780 0x556e30b02a20 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.373+0000 7f78e58ab740 1 --2- 172.21.15.120:0/3250254065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e30af7b40 0x556e30b02430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.373+0000 7f78e58ab740 1 --2- 172.21.15.120:0/3250254065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e30af65d0 0x556e30b05180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.373+0000 7f78e58ab740 1 -- 172.21.15.120:0/3250254065 >> 172.21.15.120:0/3250254065 conn(0x556e30abf310 msgr2=0x556e30af8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:44.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.373+0000 7f78c4ff9640 0 client.22557 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:24:44.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.373+0000 7f78e58ab740 1 -- 172.21.15.120:0/3250254065 shutdown_connections 2024-09-19T09:24:44.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.373+0000 7f78e58ab740 1 -- 172.21.15.120:0/3250254065 wait complete. 2024-09-19T09:24:44.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.373+0000 7f78e58ab740 20 client.22557 trim_cache size 0 max 16384 2024-09-19T09:24:44.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.386+0000 7f78e58ab740 1 Processor -- start 2024-09-19T09:24:44.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.386+0000 7f78e58ab740 1 -- start start 2024-09-19T09:24:44.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.386+0000 7f78e58ab740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e30af65d0 0x556e30b11720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:44.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.386+0000 7f78e58ab740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e30af7b40 0x556e30b01d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:44.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.386+0000 7f78e58ab740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e30b05780 0x556e30af1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:44.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.386+0000 7f78e58ab740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc1ab530 con 0x556e30af7b40 2024-09-19T09:24:44.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.386+0000 7f78e58ab740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc1ad070 con 0x556e30af65d0 2024-09-19T09:24:44.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.386+0000 7f78e58ab740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc1abc00 con 0x556e30b05780 2024-09-19T09:24:44.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.386+0000 7f78e0a6d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e30af7b40 0x556e30b01d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:44.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.386+0000 7f78e0a6d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e30af7b40 0x556e30b01d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49540/0 (socket says 172.21.15.120:49540) 2024-09-19T09:24:44.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.386+0000 7f78e0a6d640 1 -- 172.21.15.120:0/2051274011 learned_addr learned my addr 172.21.15.120:0/2051274011 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:44.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.386+0000 7f78e1a6f640 1 --2- 172.21.15.120:0/2051274011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e30b05780 0x556e30af1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:44.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.386+0000 7f78e126e640 1 --2- 172.21.15.120:0/2051274011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e30af65d0 0x556e30b11720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:44.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.386+0000 7f78e0a6d640 1 -- 172.21.15.120:0/2051274011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e30b05780 msgr2=0x556e30af1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:44.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.386+0000 7f78e0a6d640 1 --2- 172.21.15.120:0/2051274011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e30b05780 0x556e30af1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.386+0000 7f78e0a6d640 1 -- 172.21.15.120:0/2051274011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e30af65d0 msgr2=0x556e30b11720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:44.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.386+0000 7f78e0a6d640 1 --2- 172.21.15.120:0/2051274011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e30af65d0 0x556e30b11720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.386+0000 7f78e0a6d640 1 -- 172.21.15.120:0/2051274011 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e30af6d50 con 0x556e30af7b40 2024-09-19T09:24:44.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.387+0000 7f78e0a6d640 1 --2- 172.21.15.120:0/2051274011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e30af7b40 0x556e30b01d10 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f78cc00be60 tx=0x7f78cc017f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a312878ba21931 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:44.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.387+0000 7f78c6ffd640 1 -- 172.21.15.120:0/2051274011 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc01e070 con 0x556e30af7b40 2024-09-19T09:24:44.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.387+0000 7f78c6ffd640 1 -- 172.21.15.120:0/2051274011 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78cc0a4090 con 0x556e30af7b40 2024-09-19T09:24:44.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.387+0000 7f78c6ffd640 1 -- 172.21.15.120:0/2051274011 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc09a870 con 0x556e30af7b40 2024-09-19T09:24:44.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.387+0000 7f78e58ab740 1 -- 172.21.15.120:0/2051274011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e30af7b40 msgr2=0x556e30b01d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:44.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.387+0000 7f78e58ab740 1 --2- 172.21.15.120:0/2051274011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e30af7b40 0x556e30b01d10 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f78cc00be60 tx=0x7f78cc017f90 comp rx=0 tx=0).stop 2024-09-19T09:24:44.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.387+0000 7f78e58ab740 1 -- 172.21.15.120:0/2051274011 shutdown_connections 2024-09-19T09:24:44.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.387+0000 7f78e58ab740 1 --2- 172.21.15.120:0/2051274011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e30b05780 0x556e30af1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.387+0000 7f78e58ab740 1 --2- 172.21.15.120:0/2051274011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e30af7b40 0x556e30b01d10 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.387+0000 7f78e58ab740 1 --2- 172.21.15.120:0/2051274011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e30af65d0 0x556e30b11720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.387+0000 7f78e58ab740 1 -- 172.21.15.120:0/2051274011 >> 172.21.15.120:0/2051274011 conn(0x556e30abf310 msgr2=0x556e30af1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:44.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.387+0000 7f78e58ab740 1 -- 172.21.15.120:0/2051274011 shutdown_connections 2024-09-19T09:24:44.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.387+0000 7f78e58ab740 1 -- 172.21.15.120:0/2051274011 wait complete. 2024-09-19T09:24:44.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.388+0000 7f78e58ab740 1 -- start start 2024-09-19T09:24:44.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.388+0000 7f78e58ab740 1 Processor -- start 2024-09-19T09:24:44.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.388+0000 7f78e58ab740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e30af65d0 0x556e30b0de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:44.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.388+0000 7f78e58ab740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e30af7b40 0x556e30aff030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:44.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.388+0000 7f78e58ab740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e30b05780 0x556e30aff570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:44.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.388+0000 7f78e58ab740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc1ac9a0 con 0x556e30af7b40 2024-09-19T09:24:44.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.388+0000 7f78e58ab740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc1ac2d0 con 0x556e30b05780 2024-09-19T09:24:44.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.388+0000 7f78e58ab740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc1a94f0 con 0x556e30af65d0 2024-09-19T09:24:44.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.388+0000 7f78e58ab740 10 client.0 _ll_register_callbacks cb 0x7f78d4007d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:44.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.388+0000 7f78e0a6d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e30af7b40 0x556e30aff030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:44.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.388+0000 7f78e0a6d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e30af7b40 0x556e30aff030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49552/0 (socket says 172.21.15.120:49552) 2024-09-19T09:24:44.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.388+0000 7f78e0a6d640 1 -- 172.21.15.120:0/1460921517 learned_addr learned my addr 172.21.15.120:0/1460921517 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:44.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.389+0000 7f78e0a6d640 1 -- 172.21.15.120:0/1460921517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e30af65d0 msgr2=0x556e30b0de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:44.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.389+0000 7f78e126e640 1 --2- 172.21.15.120:0/1460921517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e30af65d0 0x556e30b0de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:44.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.389+0000 7f78e1a6f640 1 --2- 172.21.15.120:0/1460921517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e30b05780 0x556e30aff570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:44.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.389+0000 7f78e0a6d640 1 --2- 172.21.15.120:0/1460921517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e30af65d0 0x556e30b0de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.389+0000 7f78e0a6d640 1 -- 172.21.15.120:0/1460921517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e30b05780 msgr2=0x556e30aff570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:44.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.389+0000 7f78e0a6d640 1 --2- 172.21.15.120:0/1460921517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e30b05780 0x556e30aff570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.389+0000 7f78e0a6d640 1 -- 172.21.15.120:0/1460921517 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e30affc10 con 0x556e30af7b40 2024-09-19T09:24:44.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.389+0000 7f78e126e640 1 --2- 172.21.15.120:0/1460921517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e30af65d0 0x556e30b0de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:24:44.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.389+0000 7f78e1a6f640 1 --2- 172.21.15.120:0/1460921517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e30b05780 0x556e30aff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:24:44.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.389+0000 7f78e0a6d640 1 --2- 172.21.15.120:0/1460921517 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e30af7b40 0x556e30aff030 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f78cc097df0 tx=0x7f78cc09b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606408c60d69865a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:44.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.389+0000 7f78c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:44.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.389+0000 7f78c4ff9640 1 -- 172.21.15.120:0/1460921517 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc01e070 con 0x556e30af7b40 2024-09-19T09:24:44.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.389+0000 7f78c4ff9640 1 -- 172.21.15.120:0/1460921517 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f78cc010ca0 con 0x556e30af7b40 2024-09-19T09:24:44.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.389+0000 7f78e58ab740 1 -- 172.21.15.120:0/1460921517 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78cc0a4090 con 0x556e30af7b40 2024-09-19T09:24:44.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.389+0000 7f78c4ff9640 1 -- 172.21.15.120:0/1460921517 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f78cc066070 con 0x556e30af7b40 2024-09-19T09:24:44.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.389+0000 7f78e58ab740 10 client.22563 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:44.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.389+0000 7f78e58ab740 1 -- 172.21.15.120:0/1460921517 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556e30b7fae0 con 0x556e30af7b40 2024-09-19T09:24:44.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.389+0000 7f78c4ff9640 1 -- 172.21.15.120:0/1460921517 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78cc0a4090 con 0x556e30af7b40 2024-09-19T09:24:44.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.389+0000 7f78c4ff9640 1 -- 172.21.15.120:0/1460921517 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f78cc097170 con 0x556e30af7b40 2024-09-19T09:24:44.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.389+0000 7f78e58ab740 10 client.22563 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:44.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.389+0000 7f78e58ab740 10 client.22563 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:24:44.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.389+0000 7f78e58ab740 20 client.22563 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:44.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.389+0000 7f78e58ab740 1 --2- 172.21.15.120:0/1460921517 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x556e30b80100 0x556e30b824f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:44.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.389+0000 7f78e58ab740 4 client.22563 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:24:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.389+0000 7f78e58ab740 1 -- 172.21.15.120:0/1460921517 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e30a63d70 con 0x556e30b80100 2024-09-19T09:24:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.390+0000 7f78e126e640 1 --2- 172.21.15.120:0/1460921517 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x556e30b80100 0x556e30b824f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.390+0000 7f78c4ff9640 10 client.22563 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:24:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.390+0000 7f78e126e640 1 --2- 172.21.15.120:0/1460921517 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x556e30b80100 0x556e30b824f0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d035c03a30eaa4d3 server_cookie=a81ebe285436367 in_seq=0 out_seq=0 2024-09-19T09:24:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.390+0000 7f78c4ff9640 1 -- 172.21.15.120:0/1460921517 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e30a63d70 con 0x556e30b80100 2024-09-19T09:24:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.390+0000 7f78c4ff9640 10 client.22563 handle_command_reply: tid=0 2024-09-19T09:24:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.390+0000 7f78e58ab740 1 client.22563 shutdown 2024-09-19T09:24:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.390+0000 7f78e58ab740 1 -- 172.21.15.120:0/1460921517 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e30af7b40 msgr2=0x556e30aff030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.390+0000 7f78e58ab740 1 --2- 172.21.15.120:0/1460921517 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e30af7b40 0x556e30aff030 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f78cc097df0 tx=0x7f78cc09b5a0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.390+0000 7f78e58ab740 1 -- 172.21.15.120:0/1460921517 shutdown_connections 2024-09-19T09:24:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.390+0000 7f78e58ab740 1 --2- 172.21.15.120:0/1460921517 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x556e30b80100 0x556e30b824f0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.390+0000 7f78e58ab740 1 --2- 172.21.15.120:0/1460921517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e30b05780 0x556e30aff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.390+0000 7f78e58ab740 1 --2- 172.21.15.120:0/1460921517 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e30af7b40 0x556e30aff030 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.390+0000 7f78e58ab740 1 --2- 172.21.15.120:0/1460921517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e30af65d0 0x556e30b0de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.390+0000 7f78e58ab740 1 -- 172.21.15.120:0/1460921517 >> 172.21.15.120:0/1460921517 conn(0x556e30abf310 msgr2=0x556e30b04270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:44.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.390+0000 7f78c4ff9640 0 client.22563 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:24:44.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.391+0000 7f78e58ab740 1 -- 172.21.15.120:0/1460921517 shutdown_connections 2024-09-19T09:24:44.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.391+0000 7f78e58ab740 1 -- 172.21.15.120:0/1460921517 wait complete. 2024-09-19T09:24:44.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.391+0000 7f78e58ab740 20 client.22563 trim_cache size 0 max 16384 2024-09-19T09:24:44.413 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:24:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.391+0000 7f78e34d2640 1 -- 172.21.15.120:0/2690345923 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f78c80a3240 msgr2=0x7f78c80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.391+0000 7f78e34d2640 1 --2- 172.21.15.120:0/2690345923 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f78c80a3240 0x7f78c80a5700 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f78d401d000 tx=0x7f78d401e000 comp rx=0 tx=0).stop 2024-09-19T09:24:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.391+0000 7f78e34d2640 1 -- 172.21.15.120:0/2690345923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78dc102790 msgr2=0x7f78dc198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.391+0000 7f78e34d2640 1 --2- 172.21.15.120:0/2690345923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78dc102790 0x7f78dc198af0 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f78cc00b4a0 tx=0x7f78cc00b960 comp rx=0 tx=0).stop 2024-09-19T09:24:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.391+0000 7f78e34d2640 1 -- 172.21.15.120:0/2690345923 shutdown_connections 2024-09-19T09:24:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.391+0000 7f78e34d2640 1 --2- 172.21.15.120:0/2690345923 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f78c80a3240 0x7f78c80a5700 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.391+0000 7f78e34d2640 1 --2- 172.21.15.120:0/2690345923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78dc106f10 0x7f78dc19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.391+0000 7f78e34d2640 1 --2- 172.21.15.120:0/2690345923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78dc102790 0x7f78dc198af0 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.391+0000 7f78e34d2640 1 --2- 172.21.15.120:0/2690345923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78dc101de0 0x7f78dc1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.391+0000 7f78e34d2640 1 -- 172.21.15.120:0/2690345923 >> 172.21.15.120:0/2690345923 conn(0x7f78dc0fd8c0 msgr2=0x7f78dc0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.391+0000 7f78e34d2640 1 -- 172.21.15.120:0/2690345923 shutdown_connections 2024-09-19T09:24:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:44.391+0000 7f78e34d2640 1 -- 172.21.15.120:0/2690345923 wait complete. 2024-09-19T09:24:44.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:44 smithi196 ceph-mon[35150]: pgmap v1001: 97 pgs: 97 active+clean; 426 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 2.0 MiB/s wr, 133 op/s 2024-09-19T09:24:44.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:44 smithi153 ceph-mon[35071]: pgmap v1001: 97 pgs: 97 active+clean; 426 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 2.0 MiB/s wr, 133 op/s 2024-09-19T09:24:45.334 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:24:45.381 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199337 19.35 MB/sec execute 400 sec latency 17891.281 ms 2024-09-19T09:24:45.582 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:46.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.065+0000 7f62b5a97640 1 -- 172.21.15.120:0/2052503630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62b0103720 msgr2=0x7f62b010bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:46.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.065+0000 7f62b5a97640 1 --2- 172.21.15.120:0/2052503630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62b0103720 0x7f62b010bc10 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f62a4009900 tx=0x7f62a402a480 comp rx=0 tx=0).stop 2024-09-19T09:24:46.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.065+0000 7f62b5a97640 1 -- 172.21.15.120:0/2052503630 shutdown_connections 2024-09-19T09:24:46.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.065+0000 7f62b5a97640 1 --2- 172.21.15.120:0/2052503630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62b010c1a0 0x7f62b010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.065+0000 7f62b5a97640 1 --2- 172.21.15.120:0/2052503630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62b0103720 0x7f62b010bc10 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.065+0000 7f62b5a97640 1 --2- 172.21.15.120:0/2052503630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62b0102e00 0x7f62b01031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.065+0000 7f62b5a97640 1 -- 172.21.15.120:0/2052503630 >> 172.21.15.120:0/2052503630 conn(0x7f62b00fc7a0 msgr2=0x7f62b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:46.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.065+0000 7f62b5a97640 1 -- 172.21.15.120:0/2052503630 shutdown_connections 2024-09-19T09:24:46.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.065+0000 7f62b5a97640 1 -- 172.21.15.120:0/2052503630 wait complete. 2024-09-19T09:24:46.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.066+0000 7f62b5a97640 1 Processor -- start 2024-09-19T09:24:46.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.066+0000 7f62b5a97640 1 -- start start 2024-09-19T09:24:46.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.066+0000 7f62b5a97640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62b0102e00 0x7f62b0198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:46.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.066+0000 7f62b5a97640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62b0103720 0x7f62b0198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:46.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.066+0000 7f62b5a97640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62b010c1a0 0x7f62b019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:46.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.066+0000 7f62b5a97640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f62b0111f70 con 0x7f62b0103720 2024-09-19T09:24:46.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.066+0000 7f62b5a97640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f62b0111df0 con 0x7f62b010c1a0 2024-09-19T09:24:46.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.066+0000 7f62b5a97640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f62b01120f0 con 0x7f62b0102e00 2024-09-19T09:24:46.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.066+0000 7f62aeffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62b0103720 0x7f62b0198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:46.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.066+0000 7f62aeffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62b0103720 0x7f62b0198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49576/0 (socket says 172.21.15.120:49576) 2024-09-19T09:24:46.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.066+0000 7f62aeffd640 1 -- 172.21.15.120:0/2879704115 learned_addr learned my addr 172.21.15.120:0/2879704115 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:46.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.067+0000 7f62af7fe640 1 --2- 172.21.15.120:0/2879704115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62b0102e00 0x7f62b0198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:46.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.067+0000 7f62affff640 1 --2- 172.21.15.120:0/2879704115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62b010c1a0 0x7f62b019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:46.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.067+0000 7f62aeffd640 1 -- 172.21.15.120:0/2879704115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62b0102e00 msgr2=0x7f62b0198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:46.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.067+0000 7f62aeffd640 1 --2- 172.21.15.120:0/2879704115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62b0102e00 0x7f62b0198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.067+0000 7f62aeffd640 1 -- 172.21.15.120:0/2879704115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62b010c1a0 msgr2=0x7f62b019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:46.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.067+0000 7f62aeffd640 1 --2- 172.21.15.120:0/2879704115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62b010c1a0 0x7f62b019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.067+0000 7f62aeffd640 1 -- 172.21.15.120:0/2879704115 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b019d3c0 con 0x7f62b0103720 2024-09-19T09:24:46.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.067+0000 7f62affff640 1 --2- 172.21.15.120:0/2879704115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62b010c1a0 0x7f62b019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:24:46.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.067+0000 7f62aeffd640 1 --2- 172.21.15.120:0/2879704115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62b0103720 0x7f62b0198950 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f62a4006010 tx=0x7f62a4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:46.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.067+0000 7f62acff9640 1 -- 172.21.15.120:0/2879704115 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a403f030 con 0x7f62b0103720 2024-09-19T09:24:46.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.067+0000 7f62b5a97640 1 -- 172.21.15.120:0/2879704115 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b019d650 con 0x7f62b0103720 2024-09-19T09:24:46.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.067+0000 7f62acff9640 1 -- 172.21.15.120:0/2879704115 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62a4004440 con 0x7f62b0103720 2024-09-19T09:24:46.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.067+0000 7f62acff9640 1 -- 172.21.15.120:0/2879704115 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a4035070 con 0x7f62b0103720 2024-09-19T09:24:46.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.067+0000 7f62b5a97640 1 -- 172.21.15.120:0/2879704115 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b01a4f90 con 0x7f62b0103720 2024-09-19T09:24:46.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.070+0000 7f62b7e70740 1 Processor -- start 2024-09-19T09:24:46.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.070+0000 7f62acff9640 1 -- 172.21.15.120:0/2879704115 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f62a4042030 con 0x7f62b0103720 2024-09-19T09:24:46.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.070+0000 7f62b7e70740 1 -- start start 2024-09-19T09:24:46.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.070+0000 7f62b7e70740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b5ed74b40 0x559b5ed73090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:46.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.070+0000 7f62b7e70740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b5ed735d0 0x559b5ed73a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:46.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.070+0000 7f62b7e70740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b5ed7e370 0x559b5ed80760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:46.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.070+0000 7f62b7e70740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559b5ecd92f0 con 0x559b5ed74b40 2024-09-19T09:24:46.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.070+0000 7f62b7e70740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559b5ec80cf0 con 0x559b5ed735d0 2024-09-19T09:24:46.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.070+0000 7f62b7e70740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559b5ecd52b0 con 0x559b5ed7e370 2024-09-19T09:24:46.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.070+0000 7f62affff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b5ed74b40 0x559b5ed73090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:46.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.070+0000 7f62affff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b5ed74b40 0x559b5ed73090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49580/0 (socket says 172.21.15.120:49580) 2024-09-19T09:24:46.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.071+0000 7f62affff640 1 -- 172.21.15.120:0/3674464621 learned_addr learned my addr 172.21.15.120:0/3674464621 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:46.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.071+0000 7f62af7fe640 1 --2- 172.21.15.120:0/3674464621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b5ed735d0 0x559b5ed73a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:46.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.071+0000 7f62aeffd640 1 --2- 172.21.15.120:0/3674464621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b5ed7e370 0x559b5ed80760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:46.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.071+0000 7f62affff640 1 -- 172.21.15.120:0/3674464621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b5ed7e370 msgr2=0x559b5ed80760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:46.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.071+0000 7f62affff640 1 --2- 172.21.15.120:0/3674464621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b5ed7e370 0x559b5ed80760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.071+0000 7f62affff640 1 -- 172.21.15.120:0/3674464621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b5ed735d0 msgr2=0x559b5ed73a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:46.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.071+0000 7f62affff640 1 --2- 172.21.15.120:0/3674464621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b5ed735d0 0x559b5ed73a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.071+0000 7f62affff640 1 -- 172.21.15.120:0/3674464621 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b5ec653a0 con 0x559b5ed74b40 2024-09-19T09:24:46.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.071+0000 7f62acff9640 1 --2- 172.21.15.120:0/2879704115 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f62980a3240 0x7f62980a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:46.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.071+0000 7f62af7fe640 1 --2- 172.21.15.120:0/2879704115 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f62980a3240 0x7f62980a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:46.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.071+0000 7f62affff640 1 --2- 172.21.15.120:0/3674464621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b5ed74b40 0x559b5ed73090 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009fd0 tx=0x7f62a8010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50dac3195113cb68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:46.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.071+0000 7f62acff9640 1 -- 172.21.15.120:0/2879704115 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f62a40efd00 con 0x7f62b0103720 2024-09-19T09:24:46.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.071+0000 7f6295ffb640 1 -- 172.21.15.120:0/3674464621 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8011700 con 0x559b5ed74b40 2024-09-19T09:24:46.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.071+0000 7f6295ffb640 1 -- 172.21.15.120:0/3674464621 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62a8011d80 con 0x559b5ed74b40 2024-09-19T09:24:46.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.071+0000 7f6295ffb640 1 -- 172.21.15.120:0/3674464621 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a800e3c0 con 0x559b5ed74b40 2024-09-19T09:24:46.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.072+0000 7f62b7e70740 1 -- 172.21.15.120:0/3674464621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b5ed74b40 msgr2=0x559b5ed73090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:46.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.072+0000 7f62af7fe640 1 --2- 172.21.15.120:0/2879704115 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f62980a3240 0x7f62980a5700 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x559b5ed70ac0 tx=0x7f62a0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:46.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.072+0000 7f62b7e70740 1 --2- 172.21.15.120:0/3674464621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b5ed74b40 0x559b5ed73090 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009fd0 tx=0x7f62a8010860 comp rx=0 tx=0).stop 2024-09-19T09:24:46.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.072+0000 7f62b7e70740 1 -- 172.21.15.120:0/3674464621 shutdown_connections 2024-09-19T09:24:46.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.072+0000 7f62b7e70740 1 --2- 172.21.15.120:0/3674464621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b5ed7e370 0x559b5ed80760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.072+0000 7f62b7e70740 1 --2- 172.21.15.120:0/3674464621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b5ed735d0 0x559b5ed73a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.072+0000 7f62b7e70740 1 --2- 172.21.15.120:0/3674464621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b5ed74b40 0x559b5ed73090 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.072+0000 7f62b7e70740 1 -- 172.21.15.120:0/3674464621 >> 172.21.15.120:0/3674464621 conn(0x559b5ed3c310 msgr2=0x559b5ed6fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:46.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.072+0000 7f62b7e70740 1 -- 172.21.15.120:0/3674464621 shutdown_connections 2024-09-19T09:24:46.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.072+0000 7f62b7e70740 1 -- 172.21.15.120:0/3674464621 wait complete. 2024-09-19T09:24:46.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.072+0000 7f62b7e70740 1 -- start start 2024-09-19T09:24:46.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.073+0000 7f62b7e70740 1 Processor -- start 2024-09-19T09:24:46.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.073+0000 7f62b7e70740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b5ed735d0 0x559b5ed7dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:46.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.073+0000 7f62b7e70740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b5ed74b40 0x559b5ed7aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:46.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.073+0000 7f62b7e70740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b5ed7e370 0x559b5ed7b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:46.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.073+0000 7f62b7e70740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559b5ed81070 con 0x559b5ed74b40 2024-09-19T09:24:46.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.073+0000 7f62b7e70740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559b5ec94060 con 0x559b5ed7e370 2024-09-19T09:24:46.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.073+0000 7f62b7e70740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559b5ed811f0 con 0x559b5ed735d0 2024-09-19T09:24:46.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.073+0000 7f62b7e70740 10 client.0 _ll_register_callbacks cb 0x559b5e9e78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:46.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.073+0000 7f62aeffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b5ed74b40 0x559b5ed7aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:46.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.073+0000 7f62aeffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b5ed74b40 0x559b5ed7aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49594/0 (socket says 172.21.15.120:49594) 2024-09-19T09:24:46.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.073+0000 7f62aeffd640 1 -- 172.21.15.120:0/3389029185 learned_addr learned my addr 172.21.15.120:0/3389029185 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:46.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.073+0000 7f62aeffd640 1 -- 172.21.15.120:0/3389029185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b5ed735d0 msgr2=0x559b5ed7dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:46.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.073+0000 7f62af7fe640 1 --2- 172.21.15.120:0/3389029185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b5ed735d0 0x559b5ed7dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:46.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.073+0000 7f62affff640 1 --2- 172.21.15.120:0/3389029185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b5ed7e370 0x559b5ed7b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:46.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.073+0000 7f62aeffd640 1 --2- 172.21.15.120:0/3389029185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b5ed735d0 0x559b5ed7dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.073+0000 7f62aeffd640 1 -- 172.21.15.120:0/3389029185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b5ed7e370 msgr2=0x559b5ed7b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:46.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.073+0000 7f62aeffd640 1 --2- 172.21.15.120:0/3389029185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b5ed7e370 0x559b5ed7b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.073+0000 7f62aeffd640 1 -- 172.21.15.120:0/3389029185 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b5ea4a3c0 con 0x559b5ed74b40 2024-09-19T09:24:46.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.073+0000 7f62affff640 1 --2- 172.21.15.120:0/3389029185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b5ed7e370 0x559b5ed7b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:24:46.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.074+0000 7f62aeffd640 1 --2- 172.21.15.120:0/3389029185 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b5ed74b40 0x559b5ed7aff0 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f62a40c4a60 tx=0x7f62a4005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f94130d26b4e7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:46.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.074+0000 7f628effd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:46.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.074+0000 7f628effd640 1 -- 172.21.15.120:0/3389029185 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a4049070 con 0x559b5ed74b40 2024-09-19T09:24:46.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.074+0000 7f628effd640 1 -- 172.21.15.120:0/3389029185 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f62a40b8990 con 0x559b5ed74b40 2024-09-19T09:24:46.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.074+0000 7f62b7e70740 1 -- 172.21.15.120:0/3389029185 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62a8011d80 con 0x559b5ed74b40 2024-09-19T09:24:46.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.074+0000 7f628effd640 1 -- 172.21.15.120:0/3389029185 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f62a40bc0f0 con 0x559b5ed74b40 2024-09-19T09:24:46.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.074+0000 7f62b7e70740 10 client.22575 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:46.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.074+0000 7f62b7e70740 1 -- 172.21.15.120:0/3389029185 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559b5ed8cae0 con 0x559b5ed74b40 2024-09-19T09:24:46.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.074+0000 7f628effd640 1 -- 172.21.15.120:0/3389029185 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62a8011d80 con 0x559b5ed74b40 2024-09-19T09:24:46.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.074+0000 7f628effd640 1 -- 172.21.15.120:0/3389029185 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f62a40c4b00 con 0x559b5ed74b40 2024-09-19T09:24:46.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.074+0000 7f62b7e70740 10 client.22575 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:46.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.074+0000 7f62b7e70740 10 client.22575 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:24:46.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.074+0000 7f62b7e70740 20 client.22575 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:46.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.074+0000 7f62b7e70740 1 --2- 172.21.15.120:0/3389029185 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x559b5ed8df90 0x559b5ed903e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:46.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.074+0000 7f62b7e70740 4 client.22575 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:24:46.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.074+0000 7f62b7e70740 1 -- 172.21.15.120:0/3389029185 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b5ece6860 con 0x559b5ed8df90 2024-09-19T09:24:46.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.075+0000 7f62af7fe640 1 --2- 172.21.15.120:0/3389029185 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x559b5ed8df90 0x559b5ed903e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:46.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.075+0000 7f62af7fe640 1 --2- 172.21.15.120:0/3389029185 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x559b5ed8df90 0x559b5ed903e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d63bcf85087e77af server_cookie=f5489ffb10df3bb in_seq=0 out_seq=0 2024-09-19T09:24:46.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.075+0000 7f628effd640 10 client.22575 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:24:46.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.076+0000 7f628effd640 1 -- 172.21.15.120:0/3389029185 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559b5ece6860 con 0x559b5ed8df90 2024-09-19T09:24:46.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.076+0000 7f628effd640 10 client.22575 handle_command_reply: tid=0 2024-09-19T09:24:46.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.076+0000 7f62b7e70740 1 client.22575 shutdown 2024-09-19T09:24:46.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.076+0000 7f62b7e70740 1 -- 172.21.15.120:0/3389029185 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b5ed74b40 msgr2=0x559b5ed7aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:46.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.076+0000 7f62b7e70740 1 --2- 172.21.15.120:0/3389029185 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b5ed74b40 0x559b5ed7aff0 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f62a40c4a60 tx=0x7f62a4005b50 comp rx=0 tx=0).stop 2024-09-19T09:24:46.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.076+0000 7f62b7e70740 1 -- 172.21.15.120:0/3389029185 shutdown_connections 2024-09-19T09:24:46.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.076+0000 7f62b7e70740 1 --2- 172.21.15.120:0/3389029185 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x559b5ed8df90 0x559b5ed903e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.076+0000 7f628effd640 0 client.22575 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:24:46.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.076+0000 7f62b7e70740 1 --2- 172.21.15.120:0/3389029185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b5ed7e370 0x559b5ed7b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.076+0000 7f62b7e70740 1 --2- 172.21.15.120:0/3389029185 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b5ed74b40 0x559b5ed7aff0 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.076+0000 7f62b7e70740 1 --2- 172.21.15.120:0/3389029185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b5ed735d0 0x559b5ed7dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.076+0000 7f62b7e70740 1 -- 172.21.15.120:0/3389029185 >> 172.21.15.120:0/3389029185 conn(0x559b5ed3c310 msgr2=0x559b5ed75930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:46.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.076+0000 7f62b7e70740 1 -- 172.21.15.120:0/3389029185 shutdown_connections 2024-09-19T09:24:46.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.076+0000 7f62b7e70740 1 -- 172.21.15.120:0/3389029185 wait complete. 2024-09-19T09:24:46.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.076+0000 7f62b7e70740 20 client.22575 trim_cache size 0 max 16384 2024-09-19T09:24:46.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.089+0000 7f62b7e70740 1 Processor -- start 2024-09-19T09:24:46.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.089+0000 7f62b7e70740 1 -- start start 2024-09-19T09:24:46.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.089+0000 7f62b7e70740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b5ed735d0 0x559b5ed7a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:46.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.089+0000 7f62b7e70740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b5ed74b40 0x559b5ed7cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:46.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.089+0000 7f62b7e70740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b5ed7e370 0x559b5ed76120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:46.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.089+0000 7f62b7e70740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f62b01ad580 con 0x559b5ed7e370 2024-09-19T09:24:46.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.089+0000 7f62b7e70740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f62b01af0c0 con 0x559b5ed735d0 2024-09-19T09:24:46.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.089+0000 7f62b7e70740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f62b01adc50 con 0x559b5ed74b40 2024-09-19T09:24:46.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.089+0000 7f62affff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b5ed7e370 0x559b5ed76120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:46.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.089+0000 7f62affff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b5ed7e370 0x559b5ed76120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49598/0 (socket says 172.21.15.120:49598) 2024-09-19T09:24:46.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.089+0000 7f62affff640 1 -- 172.21.15.120:0/2037183145 learned_addr learned my addr 172.21.15.120:0/2037183145 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:46.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.089+0000 7f62af7fe640 1 --2- 172.21.15.120:0/2037183145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b5ed735d0 0x559b5ed7a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:46.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.089+0000 7f62aeffd640 1 --2- 172.21.15.120:0/2037183145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b5ed74b40 0x559b5ed7cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:46.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.089+0000 7f62affff640 1 -- 172.21.15.120:0/2037183145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b5ed74b40 msgr2=0x559b5ed7cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:46.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.089+0000 7f62affff640 1 --2- 172.21.15.120:0/2037183145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b5ed74b40 0x559b5ed7cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.089+0000 7f62affff640 1 -- 172.21.15.120:0/2037183145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b5ed735d0 msgr2=0x559b5ed7a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:46.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.089+0000 7f62affff640 1 --2- 172.21.15.120:0/2037183145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b5ed735d0 0x559b5ed7a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.089+0000 7f62affff640 1 -- 172.21.15.120:0/2037183145 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b5ed6e310 con 0x559b5ed7e370 2024-09-19T09:24:46.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.090+0000 7f62af7fe640 1 --2- 172.21.15.120:0/2037183145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b5ed735d0 0x559b5ed7a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:24:46.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.090+0000 7f62aeffd640 1 --2- 172.21.15.120:0/2037183145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b5ed74b40 0x559b5ed7cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:24:46.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.090+0000 7f62affff640 1 --2- 172.21.15.120:0/2037183145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b5ed7e370 0x559b5ed76120 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009d20 tx=0x7f62a8009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b67079d24ae29ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:46.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.090+0000 7f62957fa640 1 -- 172.21.15.120:0/2037183145 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8004b20 con 0x559b5ed7e370 2024-09-19T09:24:46.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.090+0000 7f62957fa640 1 -- 172.21.15.120:0/2037183145 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62a8004cc0 con 0x559b5ed7e370 2024-09-19T09:24:46.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.090+0000 7f62957fa640 1 -- 172.21.15.120:0/2037183145 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a80053c0 con 0x559b5ed7e370 2024-09-19T09:24:46.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.090+0000 7f62b7e70740 1 -- 172.21.15.120:0/2037183145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b5ed7e370 msgr2=0x559b5ed76120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:46.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.090+0000 7f62b7e70740 1 --2- 172.21.15.120:0/2037183145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b5ed7e370 0x559b5ed76120 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009d20 tx=0x7f62a8009d50 comp rx=0 tx=0).stop 2024-09-19T09:24:46.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.090+0000 7f62b7e70740 1 -- 172.21.15.120:0/2037183145 shutdown_connections 2024-09-19T09:24:46.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.090+0000 7f62b7e70740 1 --2- 172.21.15.120:0/2037183145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b5ed7e370 0x559b5ed76120 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.090+0000 7f62b7e70740 1 --2- 172.21.15.120:0/2037183145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b5ed74b40 0x559b5ed7cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.091+0000 7f62b7e70740 1 --2- 172.21.15.120:0/2037183145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b5ed735d0 0x559b5ed7a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.091+0000 7f62b7e70740 1 -- 172.21.15.120:0/2037183145 >> 172.21.15.120:0/2037183145 conn(0x559b5ed3c310 msgr2=0x559b5ed7c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:46.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.091+0000 7f62b7e70740 1 -- 172.21.15.120:0/2037183145 shutdown_connections 2024-09-19T09:24:46.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.091+0000 7f62b7e70740 1 -- 172.21.15.120:0/2037183145 wait complete. 2024-09-19T09:24:46.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.091+0000 7f62b7e70740 1 -- start start 2024-09-19T09:24:46.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.091+0000 7f62b7e70740 1 Processor -- start 2024-09-19T09:24:46.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.091+0000 7f62b7e70740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b5ed735d0 0x559b5ed86760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:46.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.091+0000 7f62b7e70740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b5ed74b40 0x559b5ed86ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:46.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.091+0000 7f62b7e70740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b5ed7e370 0x559b5ed77a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:46.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.091+0000 7f62b7e70740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f62b01ae9f0 con 0x559b5ed7e370 2024-09-19T09:24:46.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.091+0000 7f62b7e70740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f62b01ae320 con 0x559b5ed74b40 2024-09-19T09:24:46.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.091+0000 7f62b7e70740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f62b01ab540 con 0x559b5ed735d0 2024-09-19T09:24:46.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.091+0000 7f62b7e70740 10 client.0 _ll_register_callbacks cb 0x7f62a4032b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:46.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.092+0000 7f62affff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b5ed7e370 0x559b5ed77a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:46.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.092+0000 7f62affff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b5ed7e370 0x559b5ed77a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49608/0 (socket says 172.21.15.120:49608) 2024-09-19T09:24:46.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.092+0000 7f62affff640 1 -- 172.21.15.120:0/2966104693 learned_addr learned my addr 172.21.15.120:0/2966104693 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:46.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.092+0000 7f62af7fe640 1 --2- 172.21.15.120:0/2966104693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b5ed735d0 0x559b5ed86760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:46.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.092+0000 7f62aeffd640 1 --2- 172.21.15.120:0/2966104693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b5ed74b40 0x559b5ed86ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:46.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.092+0000 7f62affff640 1 -- 172.21.15.120:0/2966104693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b5ed735d0 msgr2=0x559b5ed86760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:46.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.092+0000 7f62affff640 1 --2- 172.21.15.120:0/2966104693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b5ed735d0 0x559b5ed86760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.092+0000 7f62affff640 1 -- 172.21.15.120:0/2966104693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b5ed74b40 msgr2=0x559b5ed86ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:46.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.092+0000 7f62affff640 1 --2- 172.21.15.120:0/2966104693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b5ed74b40 0x559b5ed86ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.092+0000 7f62affff640 1 -- 172.21.15.120:0/2966104693 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b5ed6f510 con 0x559b5ed7e370 2024-09-19T09:24:46.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.092+0000 7f62affff640 1 --2- 172.21.15.120:0/2966104693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b5ed7e370 0x559b5ed77a80 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f62a8011f60 tx=0x7f62a8011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ae80c43e56297a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:46.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.092+0000 7f628effd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:46.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.092+0000 7f628effd640 1 -- 172.21.15.120:0/2966104693 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8035060 con 0x559b5ed7e370 2024-09-19T09:24:46.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.092+0000 7f62b7e70740 1 -- 172.21.15.120:0/2966104693 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62a8004cc0 con 0x559b5ed7e370 2024-09-19T09:24:46.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.092+0000 7f628effd640 1 -- 172.21.15.120:0/2966104693 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f62a801b640 con 0x559b5ed7e370 2024-09-19T09:24:46.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.092+0000 7f628effd640 1 -- 172.21.15.120:0/2966104693 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f62a8004b20 con 0x559b5ed7e370 2024-09-19T09:24:46.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.092+0000 7f62b7e70740 10 client.22581 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:46.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.092+0000 7f62b7e70740 1 -- 172.21.15.120:0/2966104693 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559b5edf8640 con 0x559b5ed7e370 2024-09-19T09:24:46.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.093+0000 7f628effd640 1 -- 172.21.15.120:0/2966104693 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62a8004cc0 con 0x559b5ed7e370 2024-09-19T09:24:46.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.093+0000 7f628effd640 1 -- 172.21.15.120:0/2966104693 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f62a80308d0 con 0x559b5ed7e370 2024-09-19T09:24:46.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.093+0000 7f62b7e70740 10 client.22581 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:46.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.093+0000 7f62b7e70740 10 client.22581 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:24:46.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.093+0000 7f62b7e70740 20 client.22581 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:46.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.093+0000 7f62b7e70740 1 --2- 172.21.15.120:0/2966104693 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x559b5ed8df90 0x559b5edfab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:46.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.093+0000 7f62b7e70740 4 client.22581 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:24:46.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.093+0000 7f62b7e70740 1 -- 172.21.15.120:0/2966104693 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559b5ece0d70 con 0x559b5ed8df90 2024-09-19T09:24:46.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.093+0000 7f62af7fe640 1 --2- 172.21.15.120:0/2966104693 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x559b5ed8df90 0x559b5edfab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:46.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.094+0000 7f62af7fe640 1 --2- 172.21.15.120:0/2966104693 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x559b5ed8df90 0x559b5edfab80 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3d854c16fe78860d server_cookie=69bef1a446da7a1e in_seq=0 out_seq=0 2024-09-19T09:24:46.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.094+0000 7f628effd640 10 client.22581 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:24:46.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.094+0000 7f628effd640 1 -- 172.21.15.120:0/2966104693 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559b5ece0d70 con 0x559b5ed8df90 2024-09-19T09:24:46.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.094+0000 7f628effd640 10 client.22581 handle_command_reply: tid=0 2024-09-19T09:24:46.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.094+0000 7f62b7e70740 1 client.22581 shutdown 2024-09-19T09:24:46.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.094+0000 7f62b7e70740 1 -- 172.21.15.120:0/2966104693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b5ed7e370 msgr2=0x559b5ed77a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:46.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.094+0000 7f62b7e70740 1 --2- 172.21.15.120:0/2966104693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b5ed7e370 0x559b5ed77a80 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f62a8011f60 tx=0x7f62a8011ca0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.094+0000 7f62b7e70740 1 -- 172.21.15.120:0/2966104693 shutdown_connections 2024-09-19T09:24:46.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.094+0000 7f62b7e70740 1 --2- 172.21.15.120:0/2966104693 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x559b5ed8df90 0x559b5edfab80 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.094+0000 7f62b7e70740 1 --2- 172.21.15.120:0/2966104693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559b5ed7e370 0x559b5ed77a80 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.094+0000 7f62b7e70740 1 --2- 172.21.15.120:0/2966104693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559b5ed74b40 0x559b5ed86ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.094+0000 7f628effd640 0 client.22581 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:24:46.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.094+0000 7f62b7e70740 1 --2- 172.21.15.120:0/2966104693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b5ed735d0 0x559b5ed86760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.094+0000 7f62b7e70740 1 -- 172.21.15.120:0/2966104693 >> 172.21.15.120:0/2966104693 conn(0x559b5ed3c310 msgr2=0x559b5ed7d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:46.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.094+0000 7f62b7e70740 1 -- 172.21.15.120:0/2966104693 shutdown_connections 2024-09-19T09:24:46.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.094+0000 7f62b7e70740 1 -- 172.21.15.120:0/2966104693 wait complete. 2024-09-19T09:24:46.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.094+0000 7f62b7e70740 20 client.22581 trim_cache size 0 max 16384 2024-09-19T09:24:46.109 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:24:46.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.095+0000 7f62b5a97640 1 -- 172.21.15.120:0/2879704115 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f62980a3240 msgr2=0x7f62980a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:46.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.095+0000 7f62b5a97640 1 --2- 172.21.15.120:0/2879704115 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f62980a3240 0x7f62980a5700 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x559b5ed70ac0 tx=0x7f62a0005f90 comp rx=0 tx=0).stop 2024-09-19T09:24:46.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.095+0000 7f62b5a97640 1 -- 172.21.15.120:0/2879704115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62b0103720 msgr2=0x7f62b0198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:46.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.095+0000 7f62b5a97640 1 --2- 172.21.15.120:0/2879704115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62b0103720 0x7f62b0198950 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f62a4006010 tx=0x7f62a4004040 comp rx=0 tx=0).stop 2024-09-19T09:24:46.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.095+0000 7f62b5a97640 1 -- 172.21.15.120:0/2879704115 shutdown_connections 2024-09-19T09:24:46.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.095+0000 7f62b5a97640 1 --2- 172.21.15.120:0/2879704115 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f62980a3240 0x7f62980a5700 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.095+0000 7f62b5a97640 1 --2- 172.21.15.120:0/2879704115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62b010c1a0 0x7f62b019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.095+0000 7f62b5a97640 1 --2- 172.21.15.120:0/2879704115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62b0103720 0x7f62b0198950 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.095+0000 7f62b5a97640 1 --2- 172.21.15.120:0/2879704115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62b0102e00 0x7f62b0198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.095+0000 7f62b5a97640 1 -- 172.21.15.120:0/2879704115 >> 172.21.15.120:0/2879704115 conn(0x7f62b00fc7a0 msgr2=0x7f62b00fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:46.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.095+0000 7f62b5a97640 1 -- 172.21.15.120:0/2879704115 shutdown_connections 2024-09-19T09:24:46.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:46.095+0000 7f62b5a97640 1 -- 172.21.15.120:0/2879704115 wait complete. 2024-09-19T09:24:46.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:46 smithi120 ceph-mon[31150]: pgmap v1002: 97 pgs: 97 active+clean; 426 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 691 KiB/s wr, 100 op/s 2024-09-19T09:24:46.382 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199337 19.30 MB/sec execute 401 sec latency 18891.364 ms 2024-09-19T09:24:46.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:46 smithi196 ceph-mon[35150]: pgmap v1002: 97 pgs: 97 active+clean; 426 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 691 KiB/s wr, 100 op/s 2024-09-19T09:24:46.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:46 smithi153 ceph-mon[35071]: pgmap v1002: 97 pgs: 97 active+clean; 426 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 691 KiB/s wr, 100 op/s 2024-09-19T09:24:46.890 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:24:47.140 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:47.382 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199337 19.25 MB/sec execute 402 sec latency 19891.401 ms 2024-09-19T09:24:47.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.616+0000 7fe28a7b4640 1 -- 172.21.15.120:0/186442859 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe284100e20 msgr2=0x7fe284101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:47.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.617+0000 7fe28a7b4640 1 --2- 172.21.15.120:0/186442859 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe284100e20 0x7fe284101200 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7fe274009900 tx=0x7fe27402a480 comp rx=0 tx=0).stop 2024-09-19T09:24:47.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.617+0000 7fe28a7b4640 1 -- 172.21.15.120:0/186442859 shutdown_connections 2024-09-19T09:24:47.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.617+0000 7fe28a7b4640 1 --2- 172.21.15.120:0/186442859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe284102ef0 0x7fe28410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.617+0000 7fe28a7b4640 1 --2- 172.21.15.120:0/186442859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe284102570 0x7fe2841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.617+0000 7fe28a7b4640 1 --2- 172.21.15.120:0/186442859 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe284100e20 0x7fe284101200 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.617+0000 7fe28a7b4640 1 -- 172.21.15.120:0/186442859 >> 172.21.15.120:0/186442859 conn(0x7fe2840fc7d0 msgr2=0x7fe2840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:47.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.617+0000 7fe28a7b4640 1 -- 172.21.15.120:0/186442859 shutdown_connections 2024-09-19T09:24:47.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.617+0000 7fe28a7b4640 1 -- 172.21.15.120:0/186442859 wait complete. 2024-09-19T09:24:47.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.617+0000 7fe28a7b4640 1 Processor -- start 2024-09-19T09:24:47.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.617+0000 7fe28a7b4640 1 -- start start 2024-09-19T09:24:47.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.618+0000 7fe28a7b4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe284100e20 0x7fe2841994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:47.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.618+0000 7fe28a7b4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe284102570 0x7fe284199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:47.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.618+0000 7fe28a7b4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe284102ef0 0x7fe2841a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:47.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.618+0000 7fe28a7b4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe284111020 con 0x7fe284102ef0 2024-09-19T09:24:47.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.618+0000 7fe28a7b4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe284110ea0 con 0x7fe284100e20 2024-09-19T09:24:47.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.618+0000 7fe28a7b4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe2841111a0 con 0x7fe284102570 2024-09-19T09:24:47.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.618+0000 7fe288d51640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe284102ef0 0x7fe2841a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:47.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.618+0000 7fe288d51640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe284102ef0 0x7fe2841a4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49630/0 (socket says 172.21.15.120:49630) 2024-09-19T09:24:47.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.618+0000 7fe288d51640 1 -- 172.21.15.120:0/2221459118 learned_addr learned my addr 172.21.15.120:0/2221459118 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:47.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.618+0000 7fe283fff640 1 --2- 172.21.15.120:0/2221459118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe284100e20 0x7fe2841994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:47.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.618+0000 7fe288d51640 1 -- 172.21.15.120:0/2221459118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe284102570 msgr2=0x7fe284199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:47.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.618+0000 7fe288d51640 1 --2- 172.21.15.120:0/2221459118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe284102570 0x7fe284199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.618+0000 7fe288d51640 1 -- 172.21.15.120:0/2221459118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe284100e20 msgr2=0x7fe2841994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:47.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.618+0000 7fe288d51640 1 --2- 172.21.15.120:0/2221459118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe284100e20 0x7fe2841994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.618+0000 7fe288d51640 1 -- 172.21.15.120:0/2221459118 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2841a4ab0 con 0x7fe284102ef0 2024-09-19T09:24:47.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.618+0000 7fe288d51640 1 --2- 172.21.15.120:0/2221459118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe284102ef0 0x7fe2841a4570 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00b970 tx=0x7fe27c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:47.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.618+0000 7fe281ffb640 1 -- 172.21.15.120:0/2221459118 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c019070 con 0x7fe284102ef0 2024-09-19T09:24:47.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.618+0000 7fe28a7b4640 1 -- 172.21.15.120:0/2221459118 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2841a4c80 con 0x7fe284102ef0 2024-09-19T09:24:47.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.618+0000 7fe281ffb640 1 -- 172.21.15.120:0/2221459118 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe27c00cb40 con 0x7fe284102ef0 2024-09-19T09:24:47.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.618+0000 7fe281ffb640 1 -- 172.21.15.120:0/2221459118 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c007610 con 0x7fe284102ef0 2024-09-19T09:24:47.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.618+0000 7fe28a7b4640 1 -- 172.21.15.120:0/2221459118 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2841a5170 con 0x7fe284102ef0 2024-09-19T09:24:47.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.621+0000 7fe281ffb640 1 -- 172.21.15.120:0/2221459118 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe27c01c070 con 0x7fe284102ef0 2024-09-19T09:24:47.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.621+0000 7fe28cb8d740 1 Processor -- start 2024-09-19T09:24:47.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.621+0000 7fe28cb8d740 1 -- start start 2024-09-19T09:24:47.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.621+0000 7fe28cb8d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bd23e1b40 0x561bd23e0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:47.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.621+0000 7fe28cb8d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bd23e05d0 0x561bd23e0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:47.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.621+0000 7fe28cb8d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd23eb370 0x561bd23ed760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:47.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.621+0000 7fe28cb8d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561bd23462f0 con 0x561bd23e1b40 2024-09-19T09:24:47.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.621+0000 7fe28cb8d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561bd22edcf0 con 0x561bd23e05d0 2024-09-19T09:24:47.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.621+0000 7fe28cb8d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561bd23422b0 con 0x561bd23eb370 2024-09-19T09:24:47.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.621+0000 7fe27bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bd23e1b40 0x561bd23e0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:47.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.621+0000 7fe27bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bd23e1b40 0x561bd23e0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49644/0 (socket says 172.21.15.120:49644) 2024-09-19T09:24:47.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.621+0000 7fe27bfff640 1 -- 172.21.15.120:0/3076169899 learned_addr learned my addr 172.21.15.120:0/3076169899 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:47.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.621+0000 7fe281ffb640 1 --2- 172.21.15.120:0/2221459118 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe2700a3240 0x7fe2700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:47.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.621+0000 7fe283fff640 1 --2- 172.21.15.120:0/3076169899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bd23e05d0 0x561bd23e0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:47.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.622+0000 7fe288d51640 1 --2- 172.21.15.120:0/2221459118 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe2700a3240 0x7fe2700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:47.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.622+0000 7fe27bfff640 1 --2- 172.21.15.120:0/3076169899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd23eb370 0x561bd23ed760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:47.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.622+0000 7fe281ffb640 1 -- 172.21.15.120:0/2221459118 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe27c0cef30 con 0x7fe284102ef0 2024-09-19T09:24:47.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.622+0000 7fe27bfff640 1 -- 172.21.15.120:0/3076169899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd23eb370 msgr2=0x561bd23ed760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:47.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.622+0000 7fe27bfff640 1 --2- 172.21.15.120:0/3076169899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd23eb370 0x561bd23ed760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.622+0000 7fe27bfff640 1 -- 172.21.15.120:0/3076169899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bd23e05d0 msgr2=0x561bd23e0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:47.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.622+0000 7fe27bfff640 1 --2- 172.21.15.120:0/3076169899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bd23e05d0 0x561bd23e0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.622+0000 7fe27bfff640 1 -- 172.21.15.120:0/3076169899 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bd22d23a0 con 0x561bd23e1b40 2024-09-19T09:24:47.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.622+0000 7fe283fff640 1 --2- 172.21.15.120:0/3076169899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bd23e05d0 0x561bd23e0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:24:47.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.622+0000 7fe27bfff640 1 --2- 172.21.15.120:0/3076169899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bd23e1b40 0x561bd23e0090 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009d40 tx=0x7fe26c0092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9a15d091b7642e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:47.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.622+0000 7fe288d51640 1 --2- 172.21.15.120:0/2221459118 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe2700a3240 0x7fe2700a5700 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe27c09b4e0 tx=0x7fe27c090a30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:47.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.622+0000 7fe27a7fc640 1 -- 172.21.15.120:0/3076169899 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c00c050 con 0x561bd23e1b40 2024-09-19T09:24:47.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.622+0000 7fe27a7fc640 1 -- 172.21.15.120:0/3076169899 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe26c010040 con 0x561bd23e1b40 2024-09-19T09:24:47.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.622+0000 7fe27a7fc640 1 -- 172.21.15.120:0/3076169899 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c00f040 con 0x561bd23e1b40 2024-09-19T09:24:47.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.623+0000 7fe28cb8d740 1 -- 172.21.15.120:0/3076169899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bd23e1b40 msgr2=0x561bd23e0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:47.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.623+0000 7fe28cb8d740 1 --2- 172.21.15.120:0/3076169899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bd23e1b40 0x561bd23e0090 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009d40 tx=0x7fe26c0092d0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.623+0000 7fe28cb8d740 1 -- 172.21.15.120:0/3076169899 shutdown_connections 2024-09-19T09:24:47.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.623+0000 7fe28cb8d740 1 --2- 172.21.15.120:0/3076169899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd23eb370 0x561bd23ed760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.623+0000 7fe28cb8d740 1 --2- 172.21.15.120:0/3076169899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bd23e05d0 0x561bd23e0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.623+0000 7fe28cb8d740 1 --2- 172.21.15.120:0/3076169899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bd23e1b40 0x561bd23e0090 unknown :-1 s=CLOSED pgs=2272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.623+0000 7fe28cb8d740 1 -- 172.21.15.120:0/3076169899 >> 172.21.15.120:0/3076169899 conn(0x561bd23a9310 msgr2=0x561bd23dcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:47.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.623+0000 7fe28cb8d740 1 -- 172.21.15.120:0/3076169899 shutdown_connections 2024-09-19T09:24:47.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.623+0000 7fe28cb8d740 1 -- 172.21.15.120:0/3076169899 wait complete. 2024-09-19T09:24:47.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.623+0000 7fe28cb8d740 1 -- start start 2024-09-19T09:24:47.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.623+0000 7fe28cb8d740 1 Processor -- start 2024-09-19T09:24:47.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.624+0000 7fe28cb8d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bd23e05d0 0x561bd23eada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:47.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.624+0000 7fe28cb8d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd23e1b40 0x561bd23e8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:47.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.624+0000 7fe28cb8d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bd23eb370 0x561bd23e87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:47.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.624+0000 7fe28cb8d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561bd23ee070 con 0x561bd23e05d0 2024-09-19T09:24:47.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.624+0000 7fe28cb8d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561bd2301060 con 0x561bd23eb370 2024-09-19T09:24:47.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.624+0000 7fe28cb8d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561bd23ee1f0 con 0x561bd23e1b40 2024-09-19T09:24:47.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.624+0000 7fe28cb8d740 10 client.0 _ll_register_callbacks cb 0x561bd20548c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:47.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.624+0000 7fe27bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bd23e05d0 0x561bd23eada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:47.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.624+0000 7fe27bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bd23e05d0 0x561bd23eada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49646/0 (socket says 172.21.15.120:49646) 2024-09-19T09:24:47.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.624+0000 7fe27bfff640 1 -- 172.21.15.120:0/2944888505 learned_addr learned my addr 172.21.15.120:0/2944888505 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:47.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.624+0000 7fe27bfff640 1 -- 172.21.15.120:0/2944888505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd23e1b40 msgr2=0x561bd23e8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:47.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.624+0000 7fe27bfff640 1 --2- 172.21.15.120:0/2944888505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd23e1b40 0x561bd23e8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.624+0000 7fe27bfff640 1 -- 172.21.15.120:0/2944888505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bd23eb370 msgr2=0x561bd23e87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:47.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.624+0000 7fe27bfff640 1 --2- 172.21.15.120:0/2944888505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bd23eb370 0x561bd23e87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.624+0000 7fe27bfff640 1 -- 172.21.15.120:0/2944888505 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bd20b73c0 con 0x561bd23e05d0 2024-09-19T09:24:47.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.624+0000 7fe27bfff640 1 --2- 172.21.15.120:0/2944888505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bd23e05d0 0x561bd23eada0 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7fe26c003730 tx=0x7fe26c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a936a2e7df2a89d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:47.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.624+0000 7fe2637fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:47.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.624+0000 7fe2637fe640 1 -- 172.21.15.120:0/2944888505 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c024020 con 0x561bd23e05d0 2024-09-19T09:24:47.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.624+0000 7fe2637fe640 1 -- 172.21.15.120:0/2944888505 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe26c004030 con 0x561bd23e05d0 2024-09-19T09:24:47.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.624+0000 7fe28cb8d740 1 -- 172.21.15.120:0/2944888505 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe26c010040 con 0x561bd23e05d0 2024-09-19T09:24:47.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.625+0000 7fe2637fe640 1 -- 172.21.15.120:0/2944888505 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe26c00c050 con 0x561bd23e05d0 2024-09-19T09:24:47.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.625+0000 7fe28cb8d740 10 client.22593 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:47.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.625+0000 7fe28cb8d740 1 -- 172.21.15.120:0/2944888505 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561bd23f9b50 con 0x561bd23e05d0 2024-09-19T09:24:47.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.625+0000 7fe2637fe640 1 -- 172.21.15.120:0/2944888505 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe26c010040 con 0x561bd23e05d0 2024-09-19T09:24:47.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.625+0000 7fe2637fe640 1 -- 172.21.15.120:0/2944888505 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fe26c01fc80 con 0x561bd23e05d0 2024-09-19T09:24:47.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.625+0000 7fe28cb8d740 10 client.22593 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:47.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.625+0000 7fe28cb8d740 10 client.22593 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:24:47.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.625+0000 7fe28cb8d740 20 client.22593 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:47.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.625+0000 7fe28cb8d740 1 --2- 172.21.15.120:0/2944888505 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x561bd23faf00 0x561bd23fd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:47.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.625+0000 7fe28cb8d740 4 client.22593 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:24:47.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.625+0000 7fe28cb8d740 1 -- 172.21.15.120:0/2944888505 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bd2353860 con 0x561bd23faf00 2024-09-19T09:24:47.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.625+0000 7fe283fff640 1 --2- 172.21.15.120:0/2944888505 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x561bd23faf00 0x561bd23fd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:47.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.626+0000 7fe2637fe640 10 client.22593 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:24:47.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.626+0000 7fe283fff640 1 --2- 172.21.15.120:0/2944888505 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x561bd23faf00 0x561bd23fd2f0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=88d6a11b627dfc5b server_cookie=a0d55c20abd3f50f in_seq=0 out_seq=0 2024-09-19T09:24:47.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.626+0000 7fe2637fe640 1 -- 172.21.15.120:0/2944888505 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561bd2353860 con 0x561bd23faf00 2024-09-19T09:24:47.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.626+0000 7fe2637fe640 10 client.22593 handle_command_reply: tid=0 2024-09-19T09:24:47.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.626+0000 7fe28cb8d740 1 client.22593 shutdown 2024-09-19T09:24:47.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.627+0000 7fe28cb8d740 1 -- 172.21.15.120:0/2944888505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bd23e05d0 msgr2=0x561bd23eada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:47.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.627+0000 7fe28cb8d740 1 --2- 172.21.15.120:0/2944888505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bd23e05d0 0x561bd23eada0 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7fe26c003730 tx=0x7fe26c0039c0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.627+0000 7fe28cb8d740 1 -- 172.21.15.120:0/2944888505 shutdown_connections 2024-09-19T09:24:47.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.627+0000 7fe28cb8d740 1 --2- 172.21.15.120:0/2944888505 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x561bd23faf00 0x561bd23fd2f0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.627+0000 7fe28cb8d740 1 --2- 172.21.15.120:0/2944888505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bd23eb370 0x561bd23e87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.627+0000 7fe28cb8d740 1 --2- 172.21.15.120:0/2944888505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd23e1b40 0x561bd23e8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.627+0000 7fe2637fe640 0 client.22593 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:24:47.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.627+0000 7fe28cb8d740 1 --2- 172.21.15.120:0/2944888505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bd23e05d0 0x561bd23eada0 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.627+0000 7fe28cb8d740 1 -- 172.21.15.120:0/2944888505 >> 172.21.15.120:0/2944888505 conn(0x561bd23a9310 msgr2=0x561bd23e2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:47.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.627+0000 7fe28cb8d740 1 -- 172.21.15.120:0/2944888505 shutdown_connections 2024-09-19T09:24:47.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.627+0000 7fe28cb8d740 1 -- 172.21.15.120:0/2944888505 wait complete. 2024-09-19T09:24:47.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.627+0000 7fe28cb8d740 20 client.22593 trim_cache size 0 max 16384 2024-09-19T09:24:47.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.640+0000 7fe28cb8d740 1 Processor -- start 2024-09-19T09:24:47.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.640+0000 7fe28cb8d740 1 -- start start 2024-09-19T09:24:47.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.640+0000 7fe28cb8d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bd23e05d0 0x561bd23e7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:47.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.640+0000 7fe28cb8d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd23e1b40 0x561bd23e3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:47.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.640+0000 7fe28cb8d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bd23eb370 0x561bd23e3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:47.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.640+0000 7fe28cb8d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe2841ad770 con 0x561bd23e05d0 2024-09-19T09:24:47.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.640+0000 7fe28cb8d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe2841af2b0 con 0x561bd23eb370 2024-09-19T09:24:47.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.640+0000 7fe28cb8d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe2841ade40 con 0x561bd23e1b40 2024-09-19T09:24:47.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.640+0000 7fe27bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bd23e05d0 0x561bd23e7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:47.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.640+0000 7fe27bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bd23e05d0 0x561bd23e7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49652/0 (socket says 172.21.15.120:49652) 2024-09-19T09:24:47.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.640+0000 7fe27bfff640 1 -- 172.21.15.120:0/3804111141 learned_addr learned my addr 172.21.15.120:0/3804111141 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:47.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.640+0000 7fe27bfff640 1 -- 172.21.15.120:0/3804111141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd23e1b40 msgr2=0x561bd23e3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:47.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.640+0000 7fe27bfff640 1 --2- 172.21.15.120:0/3804111141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd23e1b40 0x561bd23e3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.640+0000 7fe27bfff640 1 -- 172.21.15.120:0/3804111141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bd23eb370 msgr2=0x561bd23e3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:47.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.640+0000 7fe27bfff640 1 --2- 172.21.15.120:0/3804111141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bd23eb370 0x561bd23e3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.640+0000 7fe27bfff640 1 -- 172.21.15.120:0/3804111141 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bd23e9f90 con 0x561bd23e05d0 2024-09-19T09:24:47.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.641+0000 7fe27bfff640 1 --2- 172.21.15.120:0/3804111141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bd23e05d0 0x561bd23e7910 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7fe26c003990 tx=0x7fe26c003610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47eb709244aa9092 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:47.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.641+0000 7fe279ffb640 1 -- 172.21.15.120:0/3804111141 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c038070 con 0x561bd23e05d0 2024-09-19T09:24:47.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.641+0000 7fe279ffb640 1 -- 172.21.15.120:0/3804111141 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe26c003700 con 0x561bd23e05d0 2024-09-19T09:24:47.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.641+0000 7fe279ffb640 1 -- 172.21.15.120:0/3804111141 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c024070 con 0x561bd23e05d0 2024-09-19T09:24:47.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.641+0000 7fe28cb8d740 1 -- 172.21.15.120:0/3804111141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bd23e05d0 msgr2=0x561bd23e7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:47.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.641+0000 7fe28cb8d740 1 --2- 172.21.15.120:0/3804111141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bd23e05d0 0x561bd23e7910 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7fe26c003990 tx=0x7fe26c003610 comp rx=0 tx=0).stop 2024-09-19T09:24:47.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.641+0000 7fe28cb8d740 1 -- 172.21.15.120:0/3804111141 shutdown_connections 2024-09-19T09:24:47.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.641+0000 7fe28cb8d740 1 --2- 172.21.15.120:0/3804111141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bd23eb370 0x561bd23e3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.641+0000 7fe28cb8d740 1 --2- 172.21.15.120:0/3804111141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd23e1b40 0x561bd23e3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.641+0000 7fe28cb8d740 1 --2- 172.21.15.120:0/3804111141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bd23e05d0 0x561bd23e7910 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.641+0000 7fe28cb8d740 1 -- 172.21.15.120:0/3804111141 >> 172.21.15.120:0/3804111141 conn(0x561bd23a9310 msgr2=0x561bd23f6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:47.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.642+0000 7fe28cb8d740 1 -- 172.21.15.120:0/3804111141 shutdown_connections 2024-09-19T09:24:47.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.642+0000 7fe28cb8d740 1 -- 172.21.15.120:0/3804111141 wait complete. 2024-09-19T09:24:47.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.642+0000 7fe28cb8d740 1 -- start start 2024-09-19T09:24:47.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.642+0000 7fe28cb8d740 1 Processor -- start 2024-09-19T09:24:47.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.642+0000 7fe28cb8d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bd23e05d0 0x561bd24006b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:47.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.642+0000 7fe28cb8d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd23e1b40 0x561bd2400bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:47.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.642+0000 7fe28cb8d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bd23eb370 0x561bd23fdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:47.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.642+0000 7fe28cb8d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe2841aebe0 con 0x561bd23eb370 2024-09-19T09:24:47.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.642+0000 7fe28cb8d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe2841ae510 con 0x561bd23e05d0 2024-09-19T09:24:47.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.642+0000 7fe28cb8d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe2841ab730 con 0x561bd23e1b40 2024-09-19T09:24:47.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.642+0000 7fe28cb8d740 10 client.0 _ll_register_callbacks cb 0x7fe26c009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:47.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.642+0000 7fe27bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bd23eb370 0x561bd23fdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:47.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.643+0000 7fe27bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bd23eb370 0x561bd23fdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49668/0 (socket says 172.21.15.120:49668) 2024-09-19T09:24:47.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.643+0000 7fe27bfff640 1 -- 172.21.15.120:0/630837517 learned_addr learned my addr 172.21.15.120:0/630837517 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:47.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.643+0000 7fe283fff640 1 --2- 172.21.15.120:0/630837517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd23e1b40 0x561bd2400bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:47.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.643+0000 7fe27bfff640 1 --2- 172.21.15.120:0/630837517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bd23e05d0 0x561bd24006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:47.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.643+0000 7fe27bfff640 1 -- 172.21.15.120:0/630837517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd23e1b40 msgr2=0x561bd2400bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:47.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.643+0000 7fe27bfff640 1 --2- 172.21.15.120:0/630837517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd23e1b40 0x561bd2400bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.643+0000 7fe27bfff640 1 -- 172.21.15.120:0/630837517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bd23e05d0 msgr2=0x561bd24006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:47.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.643+0000 7fe27bfff640 1 --2- 172.21.15.120:0/630837517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bd23e05d0 0x561bd24006b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.643+0000 7fe27bfff640 1 -- 172.21.15.120:0/630837517 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bd23fe4d0 con 0x561bd23eb370 2024-09-19T09:24:47.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.643+0000 7fe27bfff640 1 --2- 172.21.15.120:0/630837517 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bd23eb370 0x561bd23fdca0 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7fe26c036040 tx=0x7fe26c009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51055634a6a92261 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:47.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.643+0000 7fe2637fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:47.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.643+0000 7fe2637fe640 1 -- 172.21.15.120:0/630837517 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c024030 con 0x561bd23eb370 2024-09-19T09:24:47.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.643+0000 7fe2637fe640 1 -- 172.21.15.120:0/630837517 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe26c004450 con 0x561bd23eb370 2024-09-19T09:24:47.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.643+0000 7fe28cb8d740 1 -- 172.21.15.120:0/630837517 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe26c003700 con 0x561bd23eb370 2024-09-19T09:24:47.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.643+0000 7fe2637fe640 1 -- 172.21.15.120:0/630837517 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe26c046050 con 0x561bd23eb370 2024-09-19T09:24:47.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.643+0000 7fe28cb8d740 10 client.22599 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:47.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.643+0000 7fe28cb8d740 1 -- 172.21.15.120:0/630837517 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561bd24654a0 con 0x561bd23eb370 2024-09-19T09:24:47.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.643+0000 7fe2637fe640 1 -- 172.21.15.120:0/630837517 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe26c003700 con 0x561bd23eb370 2024-09-19T09:24:47.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.644+0000 7fe2637fe640 1 -- 172.21.15.120:0/630837517 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fe26c03e020 con 0x561bd23eb370 2024-09-19T09:24:47.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.644+0000 7fe28cb8d740 10 client.22599 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:47.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.644+0000 7fe28cb8d740 10 client.22599 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:24:47.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.644+0000 7fe28cb8d740 20 client.22599 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:47.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.644+0000 7fe28cb8d740 1 --2- 172.21.15.120:0/630837517 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x561bd2465af0 0x561bd2467ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:47.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.644+0000 7fe28cb8d740 4 client.22599 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:24:47.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.644+0000 7fe28cb8d740 1 -- 172.21.15.120:0/630837517 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561bd234dd70 con 0x561bd2465af0 2024-09-19T09:24:47.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.644+0000 7fe283fff640 1 --2- 172.21.15.120:0/630837517 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x561bd2465af0 0x561bd2467ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:47.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.645+0000 7fe2637fe640 10 client.22599 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:24:47.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.644+0000 7fe283fff640 1 --2- 172.21.15.120:0/630837517 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x561bd2465af0 0x561bd2467ee0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=35928d687aba13f5 server_cookie=b661f04056433c6a in_seq=0 out_seq=0 2024-09-19T09:24:47.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.645+0000 7fe2637fe640 1 -- 172.21.15.120:0/630837517 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561bd234dd70 con 0x561bd2465af0 2024-09-19T09:24:47.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.645+0000 7fe2637fe640 10 client.22599 handle_command_reply: tid=0 2024-09-19T09:24:47.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.645+0000 7fe28cb8d740 1 client.22599 shutdown 2024-09-19T09:24:47.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.645+0000 7fe28cb8d740 1 -- 172.21.15.120:0/630837517 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bd23eb370 msgr2=0x561bd23fdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:47.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.645+0000 7fe28cb8d740 1 --2- 172.21.15.120:0/630837517 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bd23eb370 0x561bd23fdca0 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7fe26c036040 tx=0x7fe26c009ef0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.645+0000 7fe28cb8d740 1 -- 172.21.15.120:0/630837517 shutdown_connections 2024-09-19T09:24:47.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.645+0000 7fe28cb8d740 1 --2- 172.21.15.120:0/630837517 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x561bd2465af0 0x561bd2467ee0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.645+0000 7fe28cb8d740 1 --2- 172.21.15.120:0/630837517 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561bd23eb370 0x561bd23fdca0 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.645+0000 7fe28cb8d740 1 --2- 172.21.15.120:0/630837517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bd23e1b40 0x561bd2400bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.645+0000 7fe2637fe640 0 client.22599 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:24:47.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.645+0000 7fe28cb8d740 1 --2- 172.21.15.120:0/630837517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561bd23e05d0 0x561bd24006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.645+0000 7fe28cb8d740 1 -- 172.21.15.120:0/630837517 >> 172.21.15.120:0/630837517 conn(0x561bd23a9310 msgr2=0x561bd23e67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:47.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.645+0000 7fe28cb8d740 1 -- 172.21.15.120:0/630837517 shutdown_connections 2024-09-19T09:24:47.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.645+0000 7fe28cb8d740 1 -- 172.21.15.120:0/630837517 wait complete. 2024-09-19T09:24:47.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.645+0000 7fe28cb8d740 20 client.22599 trim_cache size 0 max 16384 2024-09-19T09:24:47.663 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:24:47.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.646+0000 7fe28a7b4640 1 -- 172.21.15.120:0/2221459118 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe2700a3240 msgr2=0x7fe2700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:47.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.646+0000 7fe28a7b4640 1 --2- 172.21.15.120:0/2221459118 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe2700a3240 0x7fe2700a5700 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe27c09b4e0 tx=0x7fe27c090a30 comp rx=0 tx=0).stop 2024-09-19T09:24:47.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.646+0000 7fe28a7b4640 1 -- 172.21.15.120:0/2221459118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe284102ef0 msgr2=0x7fe2841a4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:47.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.646+0000 7fe28a7b4640 1 --2- 172.21.15.120:0/2221459118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe284102ef0 0x7fe2841a4570 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00b970 tx=0x7fe27c00be30 comp rx=0 tx=0).stop 2024-09-19T09:24:47.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.646+0000 7fe28a7b4640 1 -- 172.21.15.120:0/2221459118 shutdown_connections 2024-09-19T09:24:47.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.646+0000 7fe28a7b4640 1 --2- 172.21.15.120:0/2221459118 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe2700a3240 0x7fe2700a5700 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.646+0000 7fe28a7b4640 1 --2- 172.21.15.120:0/2221459118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe284102ef0 0x7fe2841a4570 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.646+0000 7fe28a7b4640 1 --2- 172.21.15.120:0/2221459118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe284102570 0x7fe284199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.646+0000 7fe28a7b4640 1 --2- 172.21.15.120:0/2221459118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe284100e20 0x7fe2841994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:47.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.646+0000 7fe28a7b4640 1 -- 172.21.15.120:0/2221459118 >> 172.21.15.120:0/2221459118 conn(0x7fe2840fc7d0 msgr2=0x7fe2840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:47.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.646+0000 7fe28a7b4640 1 -- 172.21.15.120:0/2221459118 shutdown_connections 2024-09-19T09:24:47.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:47.646+0000 7fe28a7b4640 1 -- 172.21.15.120:0/2221459118 wait complete. 2024-09-19T09:24:48.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:48 smithi120 ceph-mon[31150]: pgmap v1003: 97 pgs: 97 active+clean; 426 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 417 KiB/s wr, 90 op/s 2024-09-19T09:24:48.382 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199337 19.21 MB/sec execute 403 sec latency 20891.442 ms 2024-09-19T09:24:48.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:48 smithi196 ceph-mon[35150]: pgmap v1003: 97 pgs: 97 active+clean; 426 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 417 KiB/s wr, 90 op/s 2024-09-19T09:24:48.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:48 smithi153 ceph-mon[35071]: pgmap v1003: 97 pgs: 97 active+clean; 426 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 417 KiB/s wr, 90 op/s 2024-09-19T09:24:48.471 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:24:48.715 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:49.072 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:24:49.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.169+0000 7fe57a270640 1 -- 172.21.15.120:0/3542982077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5741082a0 msgr2=0x7fe57410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:49.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.169+0000 7fe57a270640 1 --2- 172.21.15.120:0/3542982077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5741082a0 0x7fe57410ec40 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7fe56c009cd0 tx=0x7fe56c02a8e0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.169+0000 7fe57a270640 1 -- 172.21.15.120:0/3542982077 shutdown_connections 2024-09-19T09:24:49.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.169+0000 7fe57a270640 1 --2- 172.21.15.120:0/3542982077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5741082a0 0x7fe57410ec40 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.169+0000 7fe57a270640 1 --2- 172.21.15.120:0/3542982077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5741036e0 0x7fe574107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.169+0000 7fe57a270640 1 --2- 172.21.15.120:0/3542982077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe574102d30 0x7fe574103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.169+0000 7fe57a270640 1 -- 172.21.15.120:0/3542982077 >> 172.21.15.120:0/3542982077 conn(0x7fe5740fc7d0 msgr2=0x7fe5740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:49.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.169+0000 7fe57a270640 1 -- 172.21.15.120:0/3542982077 shutdown_connections 2024-09-19T09:24:49.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.169+0000 7fe57a270640 1 -- 172.21.15.120:0/3542982077 wait complete. 2024-09-19T09:24:49.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.169+0000 7fe57a270640 1 Processor -- start 2024-09-19T09:24:49.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.169+0000 7fe57a270640 1 -- start start 2024-09-19T09:24:49.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.170+0000 7fe57a270640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe574102d30 0x7fe574198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:49.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.170+0000 7fe57a270640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5741036e0 0x7fe574198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:49.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.170+0000 7fe57a270640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5741082a0 0x7fe57419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:49.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.170+0000 7fe57a270640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe5741120a0 con 0x7fe574102d30 2024-09-19T09:24:49.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.170+0000 7fe57a270640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe574111f20 con 0x7fe5741036e0 2024-09-19T09:24:49.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.170+0000 7fe57a270640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe574112220 con 0x7fe5741082a0 2024-09-19T09:24:49.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.170+0000 7fe573fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe574102d30 0x7fe574198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:49.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.170+0000 7fe573fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe574102d30 0x7fe574198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49674/0 (socket says 172.21.15.120:49674) 2024-09-19T09:24:49.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.170+0000 7fe573fff640 1 -- 172.21.15.120:0/1081644140 learned_addr learned my addr 172.21.15.120:0/1081644140 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:49.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.170+0000 7fe57880d640 1 --2- 172.21.15.120:0/1081644140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5741082a0 0x7fe57419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:49.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.170+0000 7fe56bfff640 1 --2- 172.21.15.120:0/1081644140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5741036e0 0x7fe574198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:49.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.170+0000 7fe573fff640 1 -- 172.21.15.120:0/1081644140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5741082a0 msgr2=0x7fe57419ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:49.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.170+0000 7fe573fff640 1 --2- 172.21.15.120:0/1081644140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5741082a0 0x7fe57419ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.170+0000 7fe573fff640 1 -- 172.21.15.120:0/1081644140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5741036e0 msgr2=0x7fe574198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:49.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.170+0000 7fe573fff640 1 --2- 172.21.15.120:0/1081644140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5741036e0 0x7fe574198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.170+0000 7fe573fff640 1 -- 172.21.15.120:0/1081644140 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe57419d510 con 0x7fe574102d30 2024-09-19T09:24:49.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.170+0000 7fe573fff640 1 --2- 172.21.15.120:0/1081644140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe574102d30 0x7fe574198560 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7fe56400d950 tx=0x7fe56400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:49.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.170+0000 7fe571ffb640 1 -- 172.21.15.120:0/1081644140 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe564017070 con 0x7fe574102d30 2024-09-19T09:24:49.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.170+0000 7fe57a270640 1 -- 172.21.15.120:0/1081644140 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe57419d800 con 0x7fe574102d30 2024-09-19T09:24:49.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.170+0000 7fe571ffb640 1 -- 172.21.15.120:0/1081644140 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe564009c30 con 0x7fe574102d30 2024-09-19T09:24:49.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.170+0000 7fe571ffb640 1 -- 172.21.15.120:0/1081644140 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56400f6b0 con 0x7fe574102d30 2024-09-19T09:24:49.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.170+0000 7fe57a270640 1 -- 172.21.15.120:0/1081644140 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5741a50e0 con 0x7fe574102d30 2024-09-19T09:24:49.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.173+0000 7fe571ffb640 1 -- 172.21.15.120:0/1081644140 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe5640109d0 con 0x7fe574102d30 2024-09-19T09:24:49.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.173+0000 7fe571ffb640 1 --2- 172.21.15.120:0/1081644140 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5600a3240 0x7fe5600a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:49.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.173+0000 7fe56bfff640 1 --2- 172.21.15.120:0/1081644140 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5600a3240 0x7fe5600a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:49.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.173+0000 7fe571ffb640 1 -- 172.21.15.120:0/1081644140 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe5640cf630 con 0x7fe574102d30 2024-09-19T09:24:49.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.174+0000 7fe56bfff640 1 --2- 172.21.15.120:0/1081644140 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5600a3240 0x7fe5600a5700 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe5740fee00 tx=0x7fe55c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:49.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.174+0000 7fe57c649740 1 Processor -- start 2024-09-19T09:24:49.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.174+0000 7fe57c649740 1 -- start start 2024-09-19T09:24:49.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.174+0000 7fe57c649740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd556d8b40 0x55cd556d7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:49.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.174+0000 7fe57c649740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd556d75d0 0x55cd556d7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:49.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.174+0000 7fe57c649740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd556e2370 0x55cd556e4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:49.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.174+0000 7fe57c649740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cd5563d2f0 con 0x55cd556e2370 2024-09-19T09:24:49.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.174+0000 7fe57c649740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cd555e4cf0 con 0x55cd556d75d0 2024-09-19T09:24:49.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.174+0000 7fe57c649740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cd556392b0 con 0x55cd556d8b40 2024-09-19T09:24:49.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.174+0000 7fe57880d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd556e2370 0x55cd556e4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:49.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.175+0000 7fe57880d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd556e2370 0x55cd556e4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49682/0 (socket says 172.21.15.120:49682) 2024-09-19T09:24:49.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.175+0000 7fe57880d640 1 -- 172.21.15.120:0/348746602 learned_addr learned my addr 172.21.15.120:0/348746602 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:49.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.175+0000 7fe573fff640 1 --2- 172.21.15.120:0/348746602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd556d8b40 0x55cd556d7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:49.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.175+0000 7fe56bfff640 1 --2- 172.21.15.120:0/348746602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd556d75d0 0x55cd556d7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:49.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.175+0000 7fe57880d640 1 -- 172.21.15.120:0/348746602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd556d8b40 msgr2=0x55cd556d7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:49.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.175+0000 7fe57880d640 1 --2- 172.21.15.120:0/348746602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd556d8b40 0x55cd556d7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.175+0000 7fe57880d640 1 -- 172.21.15.120:0/348746602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd556d75d0 msgr2=0x55cd556d7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:49.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.175+0000 7fe57880d640 1 --2- 172.21.15.120:0/348746602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd556d75d0 0x55cd556d7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.175+0000 7fe57880d640 1 -- 172.21.15.120:0/348746602 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd555c93a0 con 0x55cd556e2370 2024-09-19T09:24:49.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.175+0000 7fe56bfff640 1 --2- 172.21.15.120:0/348746602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd556d75d0 0x55cd556d7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:24:49.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.175+0000 7fe57880d640 1 --2- 172.21.15.120:0/348746602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd556e2370 0x55cd556e4760 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7fe56c005cc0 tx=0x7fe56c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c863b1faf0f38a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:49.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.175+0000 7fe56a7fc640 1 -- 172.21.15.120:0/348746602 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c0316b0 con 0x55cd556e2370 2024-09-19T09:24:49.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.175+0000 7fe56a7fc640 1 -- 172.21.15.120:0/348746602 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe56c036040 con 0x55cd556e2370 2024-09-19T09:24:49.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.175+0000 7fe56a7fc640 1 -- 172.21.15.120:0/348746602 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c004500 con 0x55cd556e2370 2024-09-19T09:24:49.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.176+0000 7fe57c649740 1 -- 172.21.15.120:0/348746602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd556e2370 msgr2=0x55cd556e4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:49.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.176+0000 7fe57c649740 1 --2- 172.21.15.120:0/348746602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd556e2370 0x55cd556e4760 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7fe56c005cc0 tx=0x7fe56c009a20 comp rx=0 tx=0).stop 2024-09-19T09:24:49.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.176+0000 7fe57c649740 1 -- 172.21.15.120:0/348746602 shutdown_connections 2024-09-19T09:24:49.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.176+0000 7fe57c649740 1 --2- 172.21.15.120:0/348746602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd556e2370 0x55cd556e4760 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.176+0000 7fe57c649740 1 --2- 172.21.15.120:0/348746602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd556d75d0 0x55cd556d7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.176+0000 7fe57c649740 1 --2- 172.21.15.120:0/348746602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd556d8b40 0x55cd556d7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.176+0000 7fe57c649740 1 -- 172.21.15.120:0/348746602 >> 172.21.15.120:0/348746602 conn(0x55cd556a0310 msgr2=0x55cd556d3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:49.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.176+0000 7fe57c649740 1 -- 172.21.15.120:0/348746602 shutdown_connections 2024-09-19T09:24:49.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.176+0000 7fe57c649740 1 -- 172.21.15.120:0/348746602 wait complete. 2024-09-19T09:24:49.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.176+0000 7fe57c649740 1 -- start start 2024-09-19T09:24:49.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.177+0000 7fe57c649740 1 Processor -- start 2024-09-19T09:24:49.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.177+0000 7fe57c649740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd556d75d0 0x55cd556e1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:49.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.177+0000 7fe57c649740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd556d8b40 0x55cd556df090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:49.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.177+0000 7fe57c649740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd556e2370 0x55cd556df7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:49.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.177+0000 7fe57c649740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cd556e5010 con 0x55cd556e2370 2024-09-19T09:24:49.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.177+0000 7fe57c649740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cd555f8060 con 0x55cd556d8b40 2024-09-19T09:24:49.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.177+0000 7fe57c649740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cd556e5190 con 0x55cd556d75d0 2024-09-19T09:24:49.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.177+0000 7fe57c649740 10 client.0 _ll_register_callbacks cb 0x55cd5534b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:49.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.177+0000 7fe57880d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd556e2370 0x55cd556df7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:49.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.177+0000 7fe57880d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd556e2370 0x55cd556df7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49696/0 (socket says 172.21.15.120:49696) 2024-09-19T09:24:49.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.177+0000 7fe57880d640 1 -- 172.21.15.120:0/1601515637 learned_addr learned my addr 172.21.15.120:0/1601515637 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:49.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.177+0000 7fe573fff640 1 --2- 172.21.15.120:0/1601515637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd556d75d0 0x55cd556e1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:49.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.178+0000 7fe56bfff640 1 --2- 172.21.15.120:0/1601515637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd556d8b40 0x55cd556df090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:49.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.178+0000 7fe57880d640 1 -- 172.21.15.120:0/1601515637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd556d75d0 msgr2=0x55cd556e1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:49.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.178+0000 7fe57880d640 1 --2- 172.21.15.120:0/1601515637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd556d75d0 0x55cd556e1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.178+0000 7fe57880d640 1 -- 172.21.15.120:0/1601515637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd556d8b40 msgr2=0x55cd556df090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:49.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.178+0000 7fe57880d640 1 --2- 172.21.15.120:0/1601515637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd556d8b40 0x55cd556df090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.178+0000 7fe57880d640 1 -- 172.21.15.120:0/1601515637 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd553ae3c0 con 0x55cd556e2370 2024-09-19T09:24:49.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.178+0000 7fe56bfff640 1 --2- 172.21.15.120:0/1601515637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd556d8b40 0x55cd556df090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:24:49.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.178+0000 7fe57880d640 1 --2- 172.21.15.120:0/1601515637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd556e2370 0x55cd556df7d0 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7fe56c005e30 tx=0x7fe56c0316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33ee670601ca103 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:49.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.178+0000 7fe5537fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:49.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.178+0000 7fe5537fe640 1 -- 172.21.15.120:0/1601515637 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c031cd0 con 0x55cd556e2370 2024-09-19T09:24:49.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.178+0000 7fe5537fe640 1 -- 172.21.15.120:0/1601515637 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe56c004280 con 0x55cd556e2370 2024-09-19T09:24:49.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.178+0000 7fe57c649740 1 -- 172.21.15.120:0/1601515637 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe56c036040 con 0x55cd556e2370 2024-09-19T09:24:49.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.178+0000 7fe5537fe640 1 -- 172.21.15.120:0/1601515637 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe56c004480 con 0x55cd556e2370 2024-09-19T09:24:49.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.178+0000 7fe57c649740 10 client.22611 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:49.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.178+0000 7fe57c649740 1 -- 172.21.15.120:0/1601515637 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55cd556f0b40 con 0x55cd556e2370 2024-09-19T09:24:49.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.179+0000 7fe5537fe640 1 -- 172.21.15.120:0/1601515637 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe56c036040 con 0x55cd556e2370 2024-09-19T09:24:49.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.179+0000 7fe5537fe640 1 -- 172.21.15.120:0/1601515637 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fe56c030070 con 0x55cd556e2370 2024-09-19T09:24:49.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.179+0000 7fe57c649740 10 client.22611 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:49.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.179+0000 7fe57c649740 10 client.22611 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:24:49.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.179+0000 7fe57c649740 20 client.22611 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:49.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.179+0000 7fe57c649740 1 --2- 172.21.15.120:0/1601515637 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55cd556f1e90 0x55cd556f4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:49.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.179+0000 7fe57c649740 4 client.22611 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:24:49.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.179+0000 7fe57c649740 1 -- 172.21.15.120:0/1601515637 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd5564a860 con 0x55cd556f1e90 2024-09-19T09:24:49.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.179+0000 7fe573fff640 1 --2- 172.21.15.120:0/1601515637 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55cd556f1e90 0x55cd556f4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:49.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.180+0000 7fe573fff640 1 --2- 172.21.15.120:0/1601515637 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55cd556f1e90 0x55cd556f4280 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=82178b2da708d4a3 server_cookie=eabab41d6426ebf0 in_seq=0 out_seq=0 2024-09-19T09:24:49.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.180+0000 7fe5537fe640 10 client.22611 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:24:49.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.180+0000 7fe5537fe640 1 -- 172.21.15.120:0/1601515637 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cd5564a860 con 0x55cd556f1e90 2024-09-19T09:24:49.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.180+0000 7fe5537fe640 10 client.22611 handle_command_reply: tid=0 2024-09-19T09:24:49.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.180+0000 7fe57c649740 1 client.22611 shutdown 2024-09-19T09:24:49.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.181+0000 7fe57c649740 1 -- 172.21.15.120:0/1601515637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd556e2370 msgr2=0x55cd556df7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:49.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.181+0000 7fe57c649740 1 --2- 172.21.15.120:0/1601515637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd556e2370 0x55cd556df7d0 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7fe56c005e30 tx=0x7fe56c0316e0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.181+0000 7fe57c649740 1 -- 172.21.15.120:0/1601515637 shutdown_connections 2024-09-19T09:24:49.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.181+0000 7fe57c649740 1 --2- 172.21.15.120:0/1601515637 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55cd556f1e90 0x55cd556f4280 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.181+0000 7fe57c649740 1 --2- 172.21.15.120:0/1601515637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd556e2370 0x55cd556df7d0 unknown :-1 s=CLOSED pgs=2279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.181+0000 7fe57c649740 1 --2- 172.21.15.120:0/1601515637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd556d8b40 0x55cd556df090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.181+0000 7fe57c649740 1 --2- 172.21.15.120:0/1601515637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd556d75d0 0x55cd556e1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.181+0000 7fe57c649740 1 -- 172.21.15.120:0/1601515637 >> 172.21.15.120:0/1601515637 conn(0x55cd556a0310 msgr2=0x55cd556d9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:49.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.181+0000 7fe5537fe640 0 client.22611 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:24:49.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.181+0000 7fe57c649740 1 -- 172.21.15.120:0/1601515637 shutdown_connections 2024-09-19T09:24:49.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.181+0000 7fe57c649740 1 -- 172.21.15.120:0/1601515637 wait complete. 2024-09-19T09:24:49.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.181+0000 7fe57c649740 20 client.22611 trim_cache size 0 max 16384 2024-09-19T09:24:49.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.208+0000 7fe57c649740 1 Processor -- start 2024-09-19T09:24:49.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.208+0000 7fe57c649740 1 -- start start 2024-09-19T09:24:49.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.208+0000 7fe57c649740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd556d75d0 0x55cd556da8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:49.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.208+0000 7fe57c649740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd556d8b40 0x55cd556ee430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:49.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.208+0000 7fe57c649740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd556e2370 0x55cd556de070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:49.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.208+0000 7fe57c649740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe5741ad700 con 0x55cd556d8b40 2024-09-19T09:24:49.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.208+0000 7fe57c649740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe5741af240 con 0x55cd556d75d0 2024-09-19T09:24:49.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.208+0000 7fe57c649740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5741addd0 con 0x55cd556e2370 2024-09-19T09:24:49.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.208+0000 7fe56bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd556d8b40 0x55cd556ee430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:49.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.208+0000 7fe56bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd556d8b40 0x55cd556ee430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49698/0 (socket says 172.21.15.120:49698) 2024-09-19T09:24:49.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.208+0000 7fe56bfff640 1 -- 172.21.15.120:0/2925166826 learned_addr learned my addr 172.21.15.120:0/2925166826 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:49.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.208+0000 7fe573fff640 1 --2- 172.21.15.120:0/2925166826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd556d75d0 0x55cd556da8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:49.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.208+0000 7fe57880d640 1 --2- 172.21.15.120:0/2925166826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd556e2370 0x55cd556de070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:49.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.208+0000 7fe56bfff640 1 -- 172.21.15.120:0/2925166826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd556e2370 msgr2=0x55cd556de070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:49.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.208+0000 7fe56bfff640 1 --2- 172.21.15.120:0/2925166826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd556e2370 0x55cd556de070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.208+0000 7fe56bfff640 1 -- 172.21.15.120:0/2925166826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd556d75d0 msgr2=0x55cd556da8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:49.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.208+0000 7fe56bfff640 1 --2- 172.21.15.120:0/2925166826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd556d75d0 0x55cd556da8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.208+0000 7fe56bfff640 1 -- 172.21.15.120:0/2925166826 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd556d34f0 con 0x55cd556d8b40 2024-09-19T09:24:49.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.209+0000 7fe56bfff640 1 --2- 172.21.15.120:0/2925166826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd556d8b40 0x55cd556ee430 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7fe57406a370 tx=0x7fe55c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805f2afcce83f1b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:49.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.209+0000 7fe569ffb640 1 -- 172.21.15.120:0/2925166826 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c003d40 con 0x55cd556d8b40 2024-09-19T09:24:49.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.209+0000 7fe569ffb640 1 -- 172.21.15.120:0/2925166826 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe55c0044d0 con 0x55cd556d8b40 2024-09-19T09:24:49.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.209+0000 7fe569ffb640 1 -- 172.21.15.120:0/2925166826 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c029840 con 0x55cd556d8b40 2024-09-19T09:24:49.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.210+0000 7fe57c649740 1 -- 172.21.15.120:0/2925166826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd556d8b40 msgr2=0x55cd556ee430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:49.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.210+0000 7fe57c649740 1 --2- 172.21.15.120:0/2925166826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd556d8b40 0x55cd556ee430 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7fe57406a370 tx=0x7fe55c014000 comp rx=0 tx=0).stop 2024-09-19T09:24:49.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.210+0000 7fe57c649740 1 -- 172.21.15.120:0/2925166826 shutdown_connections 2024-09-19T09:24:49.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.210+0000 7fe57c649740 1 --2- 172.21.15.120:0/2925166826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd556e2370 0x55cd556de070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.210+0000 7fe57c649740 1 --2- 172.21.15.120:0/2925166826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd556d8b40 0x55cd556ee430 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.210+0000 7fe57c649740 1 --2- 172.21.15.120:0/2925166826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd556d75d0 0x55cd556da8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.210+0000 7fe57c649740 1 -- 172.21.15.120:0/2925166826 >> 172.21.15.120:0/2925166826 conn(0x55cd556a0310 msgr2=0x55cd556dd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:49.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.210+0000 7fe57c649740 1 -- 172.21.15.120:0/2925166826 shutdown_connections 2024-09-19T09:24:49.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.210+0000 7fe57c649740 1 -- 172.21.15.120:0/2925166826 wait complete. 2024-09-19T09:24:49.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.210+0000 7fe57c649740 1 -- start start 2024-09-19T09:24:49.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.210+0000 7fe57c649740 1 Processor -- start 2024-09-19T09:24:49.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.211+0000 7fe57c649740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd556d75d0 0x55cd556f72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:49.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.211+0000 7fe57c649740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd556d8b40 0x55cd556f77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:49.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.211+0000 7fe57c649740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd556e2370 0x55cd556f48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:49.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.211+0000 7fe57c649740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe5741aeb70 con 0x55cd556d75d0 2024-09-19T09:24:49.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.211+0000 7fe57c649740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe5741ae4a0 con 0x55cd556e2370 2024-09-19T09:24:49.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.211+0000 7fe57c649740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5741ab6c0 con 0x55cd556d8b40 2024-09-19T09:24:49.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.211+0000 7fe57c649740 10 client.0 _ll_register_callbacks cb 0x7fe564097850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:49.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.211+0000 7fe573fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd556d75d0 0x55cd556f72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:49.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.211+0000 7fe573fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd556d75d0 0x55cd556f72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49714/0 (socket says 172.21.15.120:49714) 2024-09-19T09:24:49.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.211+0000 7fe573fff640 1 -- 172.21.15.120:0/978131964 learned_addr learned my addr 172.21.15.120:0/978131964 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:49.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.211+0000 7fe56bfff640 1 --2- 172.21.15.120:0/978131964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd556d8b40 0x55cd556f77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:49.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.211+0000 7fe57880d640 1 --2- 172.21.15.120:0/978131964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd556e2370 0x55cd556f48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:49.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.211+0000 7fe573fff640 1 -- 172.21.15.120:0/978131964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd556d8b40 msgr2=0x55cd556f77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:49.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.211+0000 7fe573fff640 1 --2- 172.21.15.120:0/978131964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd556d8b40 0x55cd556f77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.211+0000 7fe573fff640 1 -- 172.21.15.120:0/978131964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd556e2370 msgr2=0x55cd556f48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:49.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.211+0000 7fe573fff640 1 --2- 172.21.15.120:0/978131964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd556e2370 0x55cd556f48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.211+0000 7fe573fff640 1 -- 172.21.15.120:0/978131964 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd556f50d0 con 0x55cd556d75d0 2024-09-19T09:24:49.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.211+0000 7fe56bfff640 1 --2- 172.21.15.120:0/978131964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd556d8b40 0x55cd556f77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:24:49.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.211+0000 7fe57880d640 1 --2- 172.21.15.120:0/978131964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd556e2370 0x55cd556f48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:24:49.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.211+0000 7fe573fff640 1 --2- 172.21.15.120:0/978131964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd556d75d0 0x55cd556f72b0 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7fe56409b950 tx=0x7fe564086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b9b77d08975856 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:49.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.211+0000 7fe5537fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:49.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.212+0000 7fe5537fe640 1 -- 172.21.15.120:0/978131964 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5640b8070 con 0x55cd556d75d0 2024-09-19T09:24:49.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.212+0000 7fe5537fe640 1 -- 172.21.15.120:0/978131964 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe5640982c0 con 0x55cd556d75d0 2024-09-19T09:24:49.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.212+0000 7fe57c649740 1 -- 172.21.15.120:0/978131964 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe55c0044d0 con 0x55cd556d75d0 2024-09-19T09:24:49.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.212+0000 7fe5537fe640 1 -- 172.21.15.120:0/978131964 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe5640b1070 con 0x55cd556d75d0 2024-09-19T09:24:49.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.212+0000 7fe57c649740 10 client.22617 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:49.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.212+0000 7fe57c649740 1 -- 172.21.15.120:0/978131964 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55cd5575c4e0 con 0x55cd556d75d0 2024-09-19T09:24:49.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.212+0000 7fe5537fe640 1 -- 172.21.15.120:0/978131964 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe55c0044d0 con 0x55cd556d75d0 2024-09-19T09:24:49.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.212+0000 7fe5537fe640 1 -- 172.21.15.120:0/978131964 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fe56409ab10 con 0x55cd556d75d0 2024-09-19T09:24:49.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.212+0000 7fe57c649740 10 client.22617 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:49.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.212+0000 7fe57c649740 10 client.22617 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:24:49.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.212+0000 7fe57c649740 20 client.22617 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:49.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.212+0000 7fe57c649740 1 --2- 172.21.15.120:0/978131964 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55cd5575cbb0 0x55cd5575efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:49.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.212+0000 7fe57c649740 4 client.22617 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:24:49.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.212+0000 7fe57c649740 1 -- 172.21.15.120:0/978131964 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cd55644d70 con 0x55cd5575cbb0 2024-09-19T09:24:49.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.213+0000 7fe56bfff640 1 --2- 172.21.15.120:0/978131964 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55cd5575cbb0 0x55cd5575efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:49.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.213+0000 7fe5537fe640 10 client.22617 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:24:49.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.213+0000 7fe56bfff640 1 --2- 172.21.15.120:0/978131964 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55cd5575cbb0 0x55cd5575efd0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=845af1dbcc0f8f41 server_cookie=ffac10a5744558bb in_seq=0 out_seq=0 2024-09-19T09:24:49.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.213+0000 7fe5537fe640 1 -- 172.21.15.120:0/978131964 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cd55644d70 con 0x55cd5575cbb0 2024-09-19T09:24:49.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.213+0000 7fe5537fe640 10 client.22617 handle_command_reply: tid=0 2024-09-19T09:24:49.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.213+0000 7fe57c649740 1 client.22617 shutdown 2024-09-19T09:24:49.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.213+0000 7fe57c649740 1 -- 172.21.15.120:0/978131964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd556d75d0 msgr2=0x55cd556f72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:49.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.213+0000 7fe57c649740 1 --2- 172.21.15.120:0/978131964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd556d75d0 0x55cd556f72b0 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7fe56409b950 tx=0x7fe564086040 comp rx=0 tx=0).stop 2024-09-19T09:24:49.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.214+0000 7fe57c649740 1 -- 172.21.15.120:0/978131964 shutdown_connections 2024-09-19T09:24:49.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.214+0000 7fe57c649740 1 --2- 172.21.15.120:0/978131964 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55cd5575cbb0 0x55cd5575efd0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.214+0000 7fe57c649740 1 --2- 172.21.15.120:0/978131964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cd556e2370 0x55cd556f48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.214+0000 7fe57c649740 1 --2- 172.21.15.120:0/978131964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cd556d8b40 0x55cd556f77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.214+0000 7fe57c649740 1 --2- 172.21.15.120:0/978131964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cd556d75d0 0x55cd556f72b0 unknown :-1 s=CLOSED pgs=2281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.214+0000 7fe57c649740 1 -- 172.21.15.120:0/978131964 >> 172.21.15.120:0/978131964 conn(0x55cd556a0310 msgr2=0x55cd556de180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:49.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.214+0000 7fe5537fe640 0 client.22617 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:24:49.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.214+0000 7fe57c649740 1 -- 172.21.15.120:0/978131964 shutdown_connections 2024-09-19T09:24:49.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.214+0000 7fe57c649740 1 -- 172.21.15.120:0/978131964 wait complete. 2024-09-19T09:24:49.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.214+0000 7fe57c649740 20 client.22617 trim_cache size 0 max 16384 2024-09-19T09:24:49.229 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:24:49.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.214+0000 7fe57a270640 1 -- 172.21.15.120:0/1081644140 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5600a3240 msgr2=0x7fe5600a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:49.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.214+0000 7fe57a270640 1 --2- 172.21.15.120:0/1081644140 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5600a3240 0x7fe5600a5700 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe5740fee00 tx=0x7fe55c005f90 comp rx=0 tx=0).stop 2024-09-19T09:24:49.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.214+0000 7fe57a270640 1 -- 172.21.15.120:0/1081644140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe574102d30 msgr2=0x7fe574198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:49.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.214+0000 7fe57a270640 1 --2- 172.21.15.120:0/1081644140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe574102d30 0x7fe574198560 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7fe56400d950 tx=0x7fe56400de10 comp rx=0 tx=0).stop 2024-09-19T09:24:49.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.215+0000 7fe57a270640 1 -- 172.21.15.120:0/1081644140 shutdown_connections 2024-09-19T09:24:49.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.215+0000 7fe57a270640 1 --2- 172.21.15.120:0/1081644140 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe5600a3240 0x7fe5600a5700 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.215+0000 7fe57a270640 1 --2- 172.21.15.120:0/1081644140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5741082a0 0x7fe57419ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.215+0000 7fe57a270640 1 --2- 172.21.15.120:0/1081644140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5741036e0 0x7fe574198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.215+0000 7fe57a270640 1 --2- 172.21.15.120:0/1081644140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe574102d30 0x7fe574198560 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:49.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.215+0000 7fe57a270640 1 -- 172.21.15.120:0/1081644140 >> 172.21.15.120:0/1081644140 conn(0x7fe5740fc7d0 msgr2=0x7fe574105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:49.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.215+0000 7fe57a270640 1 -- 172.21.15.120:0/1081644140 shutdown_connections 2024-09-19T09:24:49.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:49.215+0000 7fe57a270640 1 -- 172.21.15.120:0/1081644140 wait complete. 2024-09-19T09:24:49.382 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199337 19.16 MB/sec execute 404 sec latency 21891.537 ms 2024-09-19T09:24:50.084 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:24:50.158 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:50.287 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:50 smithi120 ceph-mon[31150]: pgmap v1004: 97 pgs: 97 active+clean; 426 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 327 KiB/s wr, 84 op/s 2024-09-19T09:24:50.382 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199337 19.11 MB/sec execute 405 sec latency 22891.620 ms 2024-09-19T09:24:50.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:50 smithi196 ceph-mon[35150]: pgmap v1004: 97 pgs: 97 active+clean; 426 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 327 KiB/s wr, 84 op/s 2024-09-19T09:24:50.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:50 smithi153 ceph-mon[35071]: pgmap v1004: 97 pgs: 97 active+clean; 426 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 327 KiB/s wr, 84 op/s 2024-09-19T09:24:50.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.668+0000 7faf01c27640 1 -- 172.21.15.120:0/3710025271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faefc106b80 msgr2=0x7faefc106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:50.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.668+0000 7faf01c27640 1 --2- 172.21.15.120:0/3710025271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faefc106b80 0x7faefc106f60 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7faeec009900 tx=0x7faeec02a410 comp rx=0 tx=0).stop 2024-09-19T09:24:50.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.668+0000 7faf01c27640 1 -- 172.21.15.120:0/3710025271 shutdown_connections 2024-09-19T09:24:50.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.668+0000 7faf01c27640 1 --2- 172.21.15.120:0/3710025271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faefc100aa0 0x7faefc10ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:50.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.668+0000 7faf01c27640 1 --2- 172.21.15.120:0/3710025271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faefc100100 0x7faefc100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:50.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.668+0000 7faf01c27640 1 --2- 172.21.15.120:0/3710025271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faefc106b80 0x7faefc106f60 unknown :-1 s=CLOSED pgs=2282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:50.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.668+0000 7faf01c27640 1 -- 172.21.15.120:0/3710025271 >> 172.21.15.120:0/3710025271 conn(0x7faefc077230 msgr2=0x7faefc0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:50.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.668+0000 7faf01c27640 1 -- 172.21.15.120:0/3710025271 shutdown_connections 2024-09-19T09:24:50.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.668+0000 7faf01c27640 1 -- 172.21.15.120:0/3710025271 wait complete. 2024-09-19T09:24:50.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.668+0000 7faf01c27640 1 Processor -- start 2024-09-19T09:24:50.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.668+0000 7faf01c27640 1 -- start start 2024-09-19T09:24:50.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.669+0000 7faf01c27640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faefc100100 0x7faefc1012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:50.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.669+0000 7faf01c27640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faefc100aa0 0x7faefc101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:50.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.669+0000 7faf01c27640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faefc106b80 0x7faefc101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:50.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.669+0000 7faf01c27640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faefc112090 con 0x7faefc106b80 2024-09-19T09:24:50.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.669+0000 7faf01c27640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faefc111f10 con 0x7faefc100aa0 2024-09-19T09:24:50.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.669+0000 7faf01c27640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faefc112210 con 0x7faefc100100 2024-09-19T09:24:50.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.669+0000 7faefbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faefc106b80 0x7faefc101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:50.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.669+0000 7faefb7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faefc100100 0x7faefc1012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:50.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.669+0000 7faefbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faefc106b80 0x7faefc101d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49736/0 (socket says 172.21.15.120:49736) 2024-09-19T09:24:50.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.669+0000 7faefb7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faefc100100 0x7faefc1012e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:44666/0 (socket says 172.21.15.120:44666) 2024-09-19T09:24:50.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.669+0000 7faefbfff640 1 -- 172.21.15.120:0/69432621 learned_addr learned my addr 172.21.15.120:0/69432621 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:50.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.669+0000 7faef3fff640 1 --2- 172.21.15.120:0/69432621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faefc100aa0 0x7faefc101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:50.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.669+0000 7faefbfff640 1 -- 172.21.15.120:0/69432621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faefc100100 msgr2=0x7faefc1012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:50.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.669+0000 7faefbfff640 1 --2- 172.21.15.120:0/69432621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faefc100100 0x7faefc1012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:50.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.669+0000 7faefbfff640 1 -- 172.21.15.120:0/69432621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faefc100aa0 msgr2=0x7faefc101820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:50.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.669+0000 7faefbfff640 1 --2- 172.21.15.120:0/69432621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faefc100aa0 0x7faefc101820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:50.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.669+0000 7faefbfff640 1 -- 172.21.15.120:0/69432621 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faefc1a6a50 con 0x7faefc106b80 2024-09-19T09:24:50.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.669+0000 7faefbfff640 1 --2- 172.21.15.120:0/69432621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faefc106b80 0x7faefc101d60 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7faef400b9e0 tx=0x7faef400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:50.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.669+0000 7faef97fa640 1 -- 172.21.15.120:0/69432621 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef4019070 con 0x7faefc106b80 2024-09-19T09:24:50.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.669+0000 7faf01c27640 1 -- 172.21.15.120:0/69432621 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faefc1a6d40 con 0x7faefc106b80 2024-09-19T09:24:50.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.669+0000 7faef97fa640 1 -- 172.21.15.120:0/69432621 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faef4014440 con 0x7faefc106b80 2024-09-19T09:24:50.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.670+0000 7faef97fa640 1 -- 172.21.15.120:0/69432621 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef40125c0 con 0x7faefc106b80 2024-09-19T09:24:50.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.670+0000 7faf01c27640 1 -- 172.21.15.120:0/69432621 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faefc1a7220 con 0x7faefc106b80 2024-09-19T09:24:50.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.671+0000 7faf01c27640 1 -- 172.21.15.120:0/69432621 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faefc100560 con 0x7faefc106b80 2024-09-19T09:24:50.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.672+0000 7faef97fa640 1 -- 172.21.15.120:0/69432621 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7faef401c070 con 0x7faefc106b80 2024-09-19T09:24:50.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.672+0000 7faef97fa640 1 --2- 172.21.15.120:0/69432621 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faee80a3240 0x7faee80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:50.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.672+0000 7faefb7fe640 1 --2- 172.21.15.120:0/69432621 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faee80a3240 0x7faee80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:50.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.672+0000 7faef97fa640 1 -- 172.21.15.120:0/69432621 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7faef40cee10 con 0x7faefc106b80 2024-09-19T09:24:50.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.673+0000 7faefb7fe640 1 --2- 172.21.15.120:0/69432621 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faee80a3240 0x7faee80a5700 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7faeec009230 tx=0x7faeec0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:50.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.676+0000 7faef97fa640 1 -- 172.21.15.120:0/69432621 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7faef409b4e0 con 0x7faefc106b80 2024-09-19T09:24:50.739 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:50.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.888+0000 7faf01c27640 1 -- 172.21.15.120:0/69432621 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faefc10a0f0 con 0x7faefc106b80 2024-09-19T09:24:50.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.889+0000 7faef97fa640 1 -- 172.21.15.120:0/69432621 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11988 (secure 0 0 0) 0x7faef409f7e0 con 0x7faefc106b80 2024-09-19T09:24:50.892 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:24:50.892 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":39,"btime":"2024-09-19T09:18:53:120858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:18:53.070639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:24:50.892 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 39 2024-09-19T09:24:50.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.891+0000 7faf01c27640 1 -- 172.21.15.120:0/69432621 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faee80a3240 msgr2=0x7faee80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:50.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.891+0000 7faf01c27640 1 --2- 172.21.15.120:0/69432621 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faee80a3240 0x7faee80a5700 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7faeec009230 tx=0x7faeec0058d0 comp rx=0 tx=0).stop 2024-09-19T09:24:50.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.891+0000 7faf01c27640 1 -- 172.21.15.120:0/69432621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faefc106b80 msgr2=0x7faefc101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:50.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.891+0000 7faf01c27640 1 --2- 172.21.15.120:0/69432621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faefc106b80 0x7faefc101d60 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7faef400b9e0 tx=0x7faef400bea0 comp rx=0 tx=0).stop 2024-09-19T09:24:50.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.891+0000 7faf01c27640 1 -- 172.21.15.120:0/69432621 shutdown_connections 2024-09-19T09:24:50.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.891+0000 7faf01c27640 1 --2- 172.21.15.120:0/69432621 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faee80a3240 0x7faee80a5700 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:50.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.891+0000 7faf01c27640 1 --2- 172.21.15.120:0/69432621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faefc106b80 0x7faefc101d60 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:50.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.891+0000 7faf01c27640 1 --2- 172.21.15.120:0/69432621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faefc100aa0 0x7faefc101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:50.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.891+0000 7faf01c27640 1 --2- 172.21.15.120:0/69432621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faefc100100 0x7faefc1012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:50.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.891+0000 7faf01c27640 1 -- 172.21.15.120:0/69432621 >> 172.21.15.120:0/69432621 conn(0x7faefc077230 msgr2=0x7faefc104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:50.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.892+0000 7faf01c27640 1 -- 172.21.15.120:0/69432621 shutdown_connections 2024-09-19T09:24:50.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:50.892+0000 7faf01c27640 1 -- 172.21.15.120:0/69432621 wait complete. 2024-09-19T09:24:51.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:51 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/69432621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:24:51.382 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199337 19.06 MB/sec execute 406 sec latency 23891.711 ms 2024-09-19T09:24:51.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:51 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/69432621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:24:51.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:51 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/69432621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:24:51.795 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db --set-id 93457192 --release --await' 2024-09-19T09:24:51.795 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db --set-id 93457192 --release --await 2024-09-19T09:24:52.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.025+0000 7f30ae72b640 1 -- 172.21.15.120:0/2815932599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30a8103760 msgr2=0x7f30a8107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:52.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.025+0000 7f30ae72b640 1 --2- 172.21.15.120:0/2815932599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30a8103760 0x7f30a8107ba0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f3090009930 tx=0x7f309002a420 comp rx=0 tx=0).stop 2024-09-19T09:24:52.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.025+0000 7f30ae72b640 1 -- 172.21.15.120:0/2815932599 shutdown_connections 2024-09-19T09:24:52.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.025+0000 7f30ae72b640 1 --2- 172.21.15.120:0/2815932599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30a81082e0 0x7f30a810ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.025+0000 7f30ae72b640 1 --2- 172.21.15.120:0/2815932599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30a8103760 0x7f30a8107ba0 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.025+0000 7f30ae72b640 1 --2- 172.21.15.120:0/2815932599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30a8102db0 0x7f30a8103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.025+0000 7f30ae72b640 1 -- 172.21.15.120:0/2815932599 >> 172.21.15.120:0/2815932599 conn(0x7f30a80fc7d0 msgr2=0x7f30a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:52.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.025+0000 7f30ae72b640 1 -- 172.21.15.120:0/2815932599 shutdown_connections 2024-09-19T09:24:52.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.025+0000 7f30ae72b640 1 -- 172.21.15.120:0/2815932599 wait complete. 2024-09-19T09:24:52.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.026+0000 7f30ae72b640 1 Processor -- start 2024-09-19T09:24:52.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.026+0000 7f30ae72b640 1 -- start start 2024-09-19T09:24:52.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.026+0000 7f30ae72b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30a8102db0 0x7f30a81985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:52.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.026+0000 7f30ae72b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30a8103760 0x7f30a8198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:52.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.026+0000 7f30ae72b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30a81082e0 0x7f30a819cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:52.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.026+0000 7f30ae72b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f30a81120d0 con 0x7f30a8102db0 2024-09-19T09:24:52.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.026+0000 7f30ae72b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f30a8111f50 con 0x7f30a8103760 2024-09-19T09:24:52.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.026+0000 7f30ae72b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f30a8112250 con 0x7f30a81082e0 2024-09-19T09:24:52.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.026+0000 7f30a7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30a8102db0 0x7f30a81985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:52.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.026+0000 7f30a7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30a8102db0 0x7f30a81985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57852/0 (socket says 172.21.15.120:57852) 2024-09-19T09:24:52.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.026+0000 7f30a7fff640 1 -- 172.21.15.120:0/3703273312 learned_addr learned my addr 172.21.15.120:0/3703273312 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:52.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.027+0000 7f309ffff640 1 --2- 172.21.15.120:0/3703273312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30a8103760 0x7f30a8198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:52.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.027+0000 7f30accc8640 1 --2- 172.21.15.120:0/3703273312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30a81082e0 0x7f30a819cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:52.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.027+0000 7f30a7fff640 1 -- 172.21.15.120:0/3703273312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30a81082e0 msgr2=0x7f30a819cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:52.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.027+0000 7f30a7fff640 1 --2- 172.21.15.120:0/3703273312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30a81082e0 0x7f30a819cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.027+0000 7f30a7fff640 1 -- 172.21.15.120:0/3703273312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30a8103760 msgr2=0x7f30a8198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:52.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.027+0000 7f30a7fff640 1 --2- 172.21.15.120:0/3703273312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30a8103760 0x7f30a8198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.027+0000 7f30a7fff640 1 -- 172.21.15.120:0/3703273312 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a819d5a0 con 0x7f30a8102db0 2024-09-19T09:24:52.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.027+0000 7f309ffff640 1 --2- 172.21.15.120:0/3703273312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30a8103760 0x7f30a8198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:24:52.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.027+0000 7f30accc8640 1 --2- 172.21.15.120:0/3703273312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30a81082e0 0x7f30a819cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:24:52.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.027+0000 7f30a7fff640 1 --2- 172.21.15.120:0/3703273312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30a8102db0 0x7f30a81985f0 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f309800d950 tx=0x7f309800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:52.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.027+0000 7f30a5ffb640 1 -- 172.21.15.120:0/3703273312 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3098017070 con 0x7f30a8102db0 2024-09-19T09:24:52.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.027+0000 7f30a5ffb640 1 -- 172.21.15.120:0/3703273312 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3098009c30 con 0x7f30a8102db0 2024-09-19T09:24:52.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.027+0000 7f30ae72b640 1 -- 172.21.15.120:0/3703273312 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30a819d890 con 0x7f30a8102db0 2024-09-19T09:24:52.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.027+0000 7f30a5ffb640 1 -- 172.21.15.120:0/3703273312 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309800f6b0 con 0x7f30a8102db0 2024-09-19T09:24:52.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.027+0000 7f30ae72b640 1 -- 172.21.15.120:0/3703273312 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30a81a5170 con 0x7f30a8102db0 2024-09-19T09:24:52.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.029+0000 7f30a5ffb640 1 -- 172.21.15.120:0/3703273312 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f30980109d0 con 0x7f30a8102db0 2024-09-19T09:24:52.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.030+0000 7f30b0b04740 1 Processor -- start 2024-09-19T09:24:52.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.030+0000 7f30b0b04740 1 -- start start 2024-09-19T09:24:52.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.030+0000 7f30b0b04740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5577a6b40 0x55f5577a5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:52.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.030+0000 7f30b0b04740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5577a55d0 0x55f5577a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:52.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.030+0000 7f30b0b04740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5577b0370 0x55f5577b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:52.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.030+0000 7f30b0b04740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f55770b2f0 con 0x55f5577a55d0 2024-09-19T09:24:52.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.030+0000 7f30b0b04740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f5576b2cf0 con 0x55f5577b0370 2024-09-19T09:24:52.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.030+0000 7f30b0b04740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f5577072b0 con 0x55f5577a6b40 2024-09-19T09:24:52.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.030+0000 7f30a7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5577a55d0 0x55f5577a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:52.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.030+0000 7f30a7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5577a55d0 0x55f5577a5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57868/0 (socket says 172.21.15.120:57868) 2024-09-19T09:24:52.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.030+0000 7f30a7fff640 1 -- 172.21.15.120:0/2753224687 learned_addr learned my addr 172.21.15.120:0/2753224687 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:52.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.030+0000 7f30a5ffb640 1 --2- 172.21.15.120:0/3703273312 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f30940a3240 0x7f30940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:52.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.030+0000 7f30accc8640 1 --2- 172.21.15.120:0/2753224687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5577a6b40 0x55f5577a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:52.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.030+0000 7f309ffff640 1 --2- 172.21.15.120:0/2753224687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5577b0370 0x55f5577b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:52.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.031+0000 7f30a7fff640 1 -- 172.21.15.120:0/2753224687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5577a6b40 msgr2=0x55f5577a5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:52.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.031+0000 7f30a7fff640 1 --2- 172.21.15.120:0/2753224687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5577a6b40 0x55f5577a5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.031+0000 7f30a7fff640 1 -- 172.21.15.120:0/2753224687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5577b0370 msgr2=0x55f5577b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:52.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.031+0000 7f30a7fff640 1 --2- 172.21.15.120:0/2753224687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5577b0370 0x55f5577b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.031+0000 7f30a5ffb640 1 -- 172.21.15.120:0/3703273312 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f30980cf650 con 0x7f30a8102db0 2024-09-19T09:24:52.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.031+0000 7f30a7fff640 1 -- 172.21.15.120:0/2753224687 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5576973a0 con 0x55f5577a55d0 2024-09-19T09:24:52.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.031+0000 7f30accc8640 1 --2- 172.21.15.120:0/2753224687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5577a6b40 0x55f5577a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:24:52.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.031+0000 7f309ffff640 1 --2- 172.21.15.120:0/2753224687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5577b0370 0x55f5577b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:24:52.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.031+0000 7f30accc8640 1 --2- 172.21.15.120:0/3703273312 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f30940a3240 0x7f30940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:52.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.031+0000 7f30a7fff640 1 --2- 172.21.15.120:0/2753224687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5577a55d0 0x55f5577a5a10 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f30980980d0 tx=0x7f309809bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ea78445152296f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:52.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.031+0000 7f309e7fc640 1 -- 172.21.15.120:0/2753224687 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309809c370 con 0x55f5577a55d0 2024-09-19T09:24:52.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.031+0000 7f309e7fc640 1 -- 172.21.15.120:0/2753224687 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30980a48e0 con 0x55f5577a55d0 2024-09-19T09:24:52.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.031+0000 7f309e7fc640 1 -- 172.21.15.120:0/2753224687 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30980d1b50 con 0x55f5577a55d0 2024-09-19T09:24:52.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.032+0000 7f30b0b04740 1 -- 172.21.15.120:0/2753224687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5577a55d0 msgr2=0x55f5577a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:52.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.032+0000 7f30accc8640 1 --2- 172.21.15.120:0/3703273312 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f30940a3240 0x7f30940a5700 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f30a0009f50 tx=0x7f30a0004b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:52.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.032+0000 7f30b0b04740 1 --2- 172.21.15.120:0/2753224687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5577a55d0 0x55f5577a5a10 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f30980980d0 tx=0x7f309809bb70 comp rx=0 tx=0).stop 2024-09-19T09:24:52.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.032+0000 7f30b0b04740 1 -- 172.21.15.120:0/2753224687 shutdown_connections 2024-09-19T09:24:52.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.032+0000 7f30b0b04740 1 --2- 172.21.15.120:0/2753224687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5577b0370 0x55f5577b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.032+0000 7f30b0b04740 1 --2- 172.21.15.120:0/2753224687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5577a55d0 0x55f5577a5a10 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.032+0000 7f30b0b04740 1 --2- 172.21.15.120:0/2753224687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5577a6b40 0x55f5577a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.032+0000 7f30b0b04740 1 -- 172.21.15.120:0/2753224687 >> 172.21.15.120:0/2753224687 conn(0x55f55776e310 msgr2=0x55f5577a1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:52.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.032+0000 7f30b0b04740 1 -- 172.21.15.120:0/2753224687 shutdown_connections 2024-09-19T09:24:52.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.032+0000 7f30b0b04740 1 -- 172.21.15.120:0/2753224687 wait complete. 2024-09-19T09:24:52.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.032+0000 7f30b0b04740 1 -- start start 2024-09-19T09:24:52.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.032+0000 7f30b0b04740 1 Processor -- start 2024-09-19T09:24:52.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.033+0000 7f30b0b04740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5577a55d0 0x55f5577afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:52.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.033+0000 7f30b0b04740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5577a6b40 0x55f5577ad010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:52.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.033+0000 7f30b0b04740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5577b0370 0x55f5577ad570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:52.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.033+0000 7f30b0b04740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f5577b3010 con 0x55f5577a6b40 2024-09-19T09:24:52.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.033+0000 7f30b0b04740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f5576c6060 con 0x55f5577b0370 2024-09-19T09:24:52.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.033+0000 7f30b0b04740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f5577b3190 con 0x55f5577a55d0 2024-09-19T09:24:52.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.033+0000 7f30b0b04740 10 client.0 _ll_register_callbacks cb 0x55f5574198c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:52.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.033+0000 7f309ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5577a6b40 0x55f5577ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:52.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.033+0000 7f309ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5577a6b40 0x55f5577ad010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57880/0 (socket says 172.21.15.120:57880) 2024-09-19T09:24:52.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.033+0000 7f309ffff640 1 -- 172.21.15.120:0/2247329711 learned_addr learned my addr 172.21.15.120:0/2247329711 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:52.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.033+0000 7f30a7fff640 1 --2- 172.21.15.120:0/2247329711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5577a55d0 0x55f5577afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:52.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.033+0000 7f30accc8640 1 --2- 172.21.15.120:0/2247329711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5577b0370 0x55f5577ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:52.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.033+0000 7f309ffff640 1 -- 172.21.15.120:0/2247329711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5577a55d0 msgr2=0x55f5577afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:52.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.033+0000 7f309ffff640 1 --2- 172.21.15.120:0/2247329711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5577a55d0 0x55f5577afcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.033+0000 7f309ffff640 1 -- 172.21.15.120:0/2247329711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5577b0370 msgr2=0x55f5577ad570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:52.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.033+0000 7f309ffff640 1 --2- 172.21.15.120:0/2247329711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5577b0370 0x55f5577ad570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.033+0000 7f309ffff640 1 -- 172.21.15.120:0/2247329711 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f55747c3c0 con 0x55f5577a6b40 2024-09-19T09:24:52.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.033+0000 7f30a7fff640 1 --2- 172.21.15.120:0/2247329711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5577a55d0 0x55f5577afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:24:52.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.033+0000 7f30accc8640 1 --2- 172.21.15.120:0/2247329711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5577b0370 0x55f5577ad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:24:52.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.033+0000 7f309ffff640 1 --2- 172.21.15.120:0/2247329711 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5577a6b40 0x55f5577ad010 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f3090005ab0 tx=0x7f3090004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7705c4d85bd7c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:52.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.033+0000 7f30877fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:52.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.033+0000 7f30877fe640 1 -- 172.21.15.120:0/2247329711 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309003f070 con 0x55f5577a6b40 2024-09-19T09:24:52.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.033+0000 7f30877fe640 1 -- 172.21.15.120:0/2247329711 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f3090005450 con 0x55f5577a6b40 2024-09-19T09:24:52.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.033+0000 7f30b0b04740 1 -- 172.21.15.120:0/2247329711 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30980a48e0 con 0x55f5577a6b40 2024-09-19T09:24:52.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.034+0000 7f30877fe640 1 -- 172.21.15.120:0/2247329711 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f309004b050 con 0x55f5577a6b40 2024-09-19T09:24:52.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.034+0000 7f30b0b04740 10 client.22635 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:52.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.034+0000 7f30b0b04740 1 -- 172.21.15.120:0/2247329711 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f5577be9e0 con 0x55f5577a6b40 2024-09-19T09:24:52.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.034+0000 7f30877fe640 1 -- 172.21.15.120:0/2247329711 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30980a48e0 con 0x55f5577a6b40 2024-09-19T09:24:52.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.034+0000 7f30877fe640 1 -- 172.21.15.120:0/2247329711 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f3090003950 con 0x55f5577a6b40 2024-09-19T09:24:52.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.034+0000 7f30b0b04740 10 client.22635 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:52.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.034+0000 7f30b0b04740 10 client.22635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:24:52.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.034+0000 7f30b0b04740 20 client.22635 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:52.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.034+0000 7f30b0b04740 1 --2- 172.21.15.120:0/2247329711 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f5577bfe40 0x55f5577c2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:52.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.034+0000 7f30b0b04740 4 client.22635 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:24:52.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.034+0000 7f30b0b04740 1 -- 172.21.15.120:0/2247329711 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f557718860 con 0x55f5577bfe40 2024-09-19T09:24:52.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.034+0000 7f30a7fff640 1 --2- 172.21.15.120:0/2247329711 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f5577bfe40 0x55f5577c2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:52.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.035+0000 7f30877fe640 10 client.22635 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:52.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.035+0000 7f30a7fff640 1 --2- 172.21.15.120:0/2247329711 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f5577bfe40 0x55f5577c2280 crc :-1 s=READY pgs=541 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86709b7c7c1ffcef server_cookie=2b4ae35fd6008975 in_seq=0 out_seq=0 2024-09-19T09:24:52.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.035+0000 7f30877fe640 1 -- 172.21.15.120:0/2247329711 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f557718860 con 0x55f5577bfe40 2024-09-19T09:24:52.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.035+0000 7f30877fe640 10 client.22635 handle_command_reply: tid=0 2024-09-19T09:24:52.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.036+0000 7f30b0b04740 1 client.22635 shutdown 2024-09-19T09:24:52.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.036+0000 7f30b0b04740 1 -- 172.21.15.120:0/2247329711 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5577a6b40 msgr2=0x55f5577ad010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:52.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.036+0000 7f30b0b04740 1 --2- 172.21.15.120:0/2247329711 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5577a6b40 0x55f5577ad010 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f3090005ab0 tx=0x7f3090004430 comp rx=0 tx=0).stop 2024-09-19T09:24:52.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.036+0000 7f30b0b04740 1 -- 172.21.15.120:0/2247329711 shutdown_connections 2024-09-19T09:24:52.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.036+0000 7f30b0b04740 1 --2- 172.21.15.120:0/2247329711 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f5577bfe40 0x55f5577c2280 crc :-1 s=READY pgs=541 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.036+0000 7f30b0b04740 1 --2- 172.21.15.120:0/2247329711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5577b0370 0x55f5577ad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.036+0000 7f30b0b04740 1 --2- 172.21.15.120:0/2247329711 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5577a6b40 0x55f5577ad010 unknown :-1 s=CLOSED pgs=2287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.036+0000 7f30877fe640 0 client.22635 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:52.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.036+0000 7f30b0b04740 1 --2- 172.21.15.120:0/2247329711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5577a55d0 0x55f5577afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.036+0000 7f30b0b04740 1 -- 172.21.15.120:0/2247329711 >> 172.21.15.120:0/2247329711 conn(0x55f55776e310 msgr2=0x55f5577a7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:52.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.036+0000 7f30b0b04740 1 -- 172.21.15.120:0/2247329711 shutdown_connections 2024-09-19T09:24:52.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.036+0000 7f30b0b04740 1 -- 172.21.15.120:0/2247329711 wait complete. 2024-09-19T09:24:52.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.036+0000 7f30b0b04740 20 client.22635 trim_cache size 0 max 16384 2024-09-19T09:24:52.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.049+0000 7f30b0b04740 1 Processor -- start 2024-09-19T09:24:52.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.049+0000 7f30b0b04740 1 -- start start 2024-09-19T09:24:52.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.049+0000 7f30b0b04740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5577a55d0 0x55f5577acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:52.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.049+0000 7f30b0b04740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5577a6b40 0x55f5577bc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:52.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.049+0000 7f30b0b04740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5577b0370 0x55f5577c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:52.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.049+0000 7f30b0b04740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f30a81ad670 con 0x55f5577a6b40 2024-09-19T09:24:52.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.049+0000 7f30b0b04740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f30a81af1b0 con 0x55f5577b0370 2024-09-19T09:24:52.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.049+0000 7f30b0b04740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f30a81add40 con 0x55f5577a55d0 2024-09-19T09:24:52.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.049+0000 7f309ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5577a6b40 0x55f5577bc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:52.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.049+0000 7f309ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5577a6b40 0x55f5577bc2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57884/0 (socket says 172.21.15.120:57884) 2024-09-19T09:24:52.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.049+0000 7f309ffff640 1 -- 172.21.15.120:0/3967294076 learned_addr learned my addr 172.21.15.120:0/3967294076 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:52.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.049+0000 7f30a7fff640 1 --2- 172.21.15.120:0/3967294076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5577a55d0 0x55f5577acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:52.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.049+0000 7f30accc8640 1 --2- 172.21.15.120:0/3967294076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5577b0370 0x55f5577c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:52.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.049+0000 7f309ffff640 1 -- 172.21.15.120:0/3967294076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5577a55d0 msgr2=0x55f5577acfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:52.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.049+0000 7f309ffff640 1 --2- 172.21.15.120:0/3967294076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5577a55d0 0x55f5577acfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.049+0000 7f309ffff640 1 -- 172.21.15.120:0/3967294076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5577b0370 msgr2=0x55f5577c7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:52.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.049+0000 7f309ffff640 1 --2- 172.21.15.120:0/3967294076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5577b0370 0x55f5577c7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.049+0000 7f309ffff640 1 -- 172.21.15.120:0/3967294076 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5577a0310 con 0x55f5577a6b40 2024-09-19T09:24:52.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.049+0000 7f30accc8640 1 --2- 172.21.15.120:0/3967294076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5577b0370 0x55f5577c7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:24:52.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.049+0000 7f309ffff640 1 --2- 172.21.15.120:0/3967294076 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5577a6b40 0x55f5577bc2b0 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f309002a940 tx=0x7f309002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28b330e97609625 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:52.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.050+0000 7f309dffb640 1 -- 172.21.15.120:0/3967294076 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3090054070 con 0x55f5577a6b40 2024-09-19T09:24:52.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.050+0000 7f309dffb640 1 -- 172.21.15.120:0/3967294076 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30900035f0 con 0x55f5577a6b40 2024-09-19T09:24:52.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.050+0000 7f309dffb640 1 -- 172.21.15.120:0/3967294076 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309003f070 con 0x55f5577a6b40 2024-09-19T09:24:52.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.050+0000 7f30b0b04740 1 -- 172.21.15.120:0/3967294076 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5577a6b40 msgr2=0x55f5577bc2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:52.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.050+0000 7f30b0b04740 1 --2- 172.21.15.120:0/3967294076 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5577a6b40 0x55f5577bc2b0 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f309002a940 tx=0x7f309002a420 comp rx=0 tx=0).stop 2024-09-19T09:24:52.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.050+0000 7f30b0b04740 1 -- 172.21.15.120:0/3967294076 shutdown_connections 2024-09-19T09:24:52.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.050+0000 7f30b0b04740 1 --2- 172.21.15.120:0/3967294076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5577b0370 0x55f5577c7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.050+0000 7f30b0b04740 1 --2- 172.21.15.120:0/3967294076 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5577a6b40 0x55f5577bc2b0 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.050+0000 7f30b0b04740 1 --2- 172.21.15.120:0/3967294076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5577a55d0 0x55f5577acfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.050+0000 7f30b0b04740 1 -- 172.21.15.120:0/3967294076 >> 172.21.15.120:0/3967294076 conn(0x55f55776e310 msgr2=0x55f5577ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:52.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.050+0000 7f30b0b04740 1 -- 172.21.15.120:0/3967294076 shutdown_connections 2024-09-19T09:24:52.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.050+0000 7f30b0b04740 1 -- 172.21.15.120:0/3967294076 wait complete. 2024-09-19T09:24:52.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.050+0000 7f30b0b04740 1 -- start start 2024-09-19T09:24:52.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.051+0000 7f30b0b04740 1 Processor -- start 2024-09-19T09:24:52.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.051+0000 7f30b0b04740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5577a55d0 0x55f5577b4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:52.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.051+0000 7f30b0b04740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5577a6b40 0x55f5577b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:52.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.051+0000 7f30b0b04740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5577b0370 0x55f557829b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:52.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.051+0000 7f30b0b04740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f30a81aeae0 con 0x55f5577a55d0 2024-09-19T09:24:52.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.051+0000 7f30b0b04740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f30a81ae410 con 0x55f5577b0370 2024-09-19T09:24:52.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.051+0000 7f30b0b04740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f30a81ab630 con 0x55f5577a6b40 2024-09-19T09:24:52.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.051+0000 7f30b0b04740 10 client.0 _ll_register_callbacks cb 0x7f3090009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:52.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.051+0000 7f30a7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5577a55d0 0x55f5577b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:52.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.051+0000 7f30a7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5577a55d0 0x55f5577b4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57890/0 (socket says 172.21.15.120:57890) 2024-09-19T09:24:52.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.051+0000 7f30a7fff640 1 -- 172.21.15.120:0/3207447358 learned_addr learned my addr 172.21.15.120:0/3207447358 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:52.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.051+0000 7f309ffff640 1 --2- 172.21.15.120:0/3207447358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5577a6b40 0x55f5577b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:52.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.051+0000 7f30accc8640 1 --2- 172.21.15.120:0/3207447358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5577b0370 0x55f557829b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:52.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.051+0000 7f30a7fff640 1 -- 172.21.15.120:0/3207447358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5577a6b40 msgr2=0x55f5577b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:52.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.051+0000 7f30a7fff640 1 --2- 172.21.15.120:0/3207447358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5577a6b40 0x55f5577b4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.051+0000 7f30a7fff640 1 -- 172.21.15.120:0/3207447358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5577b0370 msgr2=0x55f557829b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:52.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.051+0000 7f30a7fff640 1 --2- 172.21.15.120:0/3207447358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5577b0370 0x55f557829b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.051+0000 7f30a7fff640 1 -- 172.21.15.120:0/3207447358 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f55782a060 con 0x55f5577a55d0 2024-09-19T09:24:52.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.052+0000 7f30a7fff640 1 --2- 172.21.15.120:0/3207447358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5577a55d0 0x55f5577b4820 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f309809bc00 tx=0x7f3098097910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed451122edd5f21 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:52.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.052+0000 7f30877fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:52.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.052+0000 7f30877fe640 1 -- 172.21.15.120:0/3207447358 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3098020070 con 0x55f5577a55d0 2024-09-19T09:24:52.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.052+0000 7f30877fe640 1 -- 172.21.15.120:0/3207447358 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f30980982e0 con 0x55f5577a55d0 2024-09-19T09:24:52.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.052+0000 7f30b0b04740 1 -- 172.21.15.120:0/3207447358 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30900035f0 con 0x55f5577a55d0 2024-09-19T09:24:52.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.052+0000 7f30877fe640 1 -- 172.21.15.120:0/3207447358 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3098097f10 con 0x55f5577a55d0 2024-09-19T09:24:52.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.052+0000 7f30b0b04740 10 client.22641 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:52.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.052+0000 7f30b0b04740 1 -- 172.21.15.120:0/3207447358 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f55782b1e0 con 0x55f5577a55d0 2024-09-19T09:24:52.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.052+0000 7f30877fe640 1 -- 172.21.15.120:0/3207447358 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30900035f0 con 0x55f5577a55d0 2024-09-19T09:24:52.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.052+0000 7f30877fe640 1 -- 172.21.15.120:0/3207447358 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f30980979c0 con 0x55f5577a55d0 2024-09-19T09:24:52.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.052+0000 7f30b0b04740 10 client.22641 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:52.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.052+0000 7f30b0b04740 10 client.22641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:24:52.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.052+0000 7f30b0b04740 20 client.22641 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:52.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.052+0000 7f30b0b04740 1 --2- 172.21.15.120:0/3207447358 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f5577bfe40 0x55f55782d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:52.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.052+0000 7f30b0b04740 4 client.22641 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:24:52.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.052+0000 7f30b0b04740 1 -- 172.21.15.120:0/3207447358 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f557712d70 con 0x55f5577bfe40 2024-09-19T09:24:52.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.052+0000 7f309ffff640 1 --2- 172.21.15.120:0/3207447358 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f5577bfe40 0x55f55782d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:52.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.053+0000 7f309ffff640 1 --2- 172.21.15.120:0/3207447358 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f5577bfe40 0x55f55782d3c0 crc :-1 s=READY pgs=542 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48cd3bca90ed394f server_cookie=8e4d3344a38f3ccf in_seq=0 out_seq=0 2024-09-19T09:24:52.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.053+0000 7f30877fe640 10 client.22641 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:52.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.053+0000 7f30877fe640 1 -- 172.21.15.120:0/3207447358 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f557712d70 con 0x55f5577bfe40 2024-09-19T09:24:52.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.053+0000 7f30877fe640 10 client.22641 handle_command_reply: tid=0 2024-09-19T09:24:52.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.053+0000 7f30b0b04740 1 client.22641 shutdown 2024-09-19T09:24:52.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.054+0000 7f30b0b04740 1 -- 172.21.15.120:0/3207447358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5577a55d0 msgr2=0x55f5577b4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:52.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.054+0000 7f30b0b04740 1 --2- 172.21.15.120:0/3207447358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5577a55d0 0x55f5577b4820 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f309809bc00 tx=0x7f3098097910 comp rx=0 tx=0).stop 2024-09-19T09:24:52.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.054+0000 7f30b0b04740 1 -- 172.21.15.120:0/3207447358 shutdown_connections 2024-09-19T09:24:52.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.054+0000 7f30b0b04740 1 --2- 172.21.15.120:0/3207447358 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55f5577bfe40 0x55f55782d3c0 crc :-1 s=READY pgs=542 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.054+0000 7f30b0b04740 1 --2- 172.21.15.120:0/3207447358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5577b0370 0x55f557829b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.054+0000 7f30b0b04740 1 --2- 172.21.15.120:0/3207447358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5577a6b40 0x55f5577b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.054+0000 7f30877fe640 0 client.22641 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:52.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.054+0000 7f30b0b04740 1 --2- 172.21.15.120:0/3207447358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f5577a55d0 0x55f5577b4820 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.054+0000 7f30b0b04740 1 -- 172.21.15.120:0/3207447358 >> 172.21.15.120:0/3207447358 conn(0x55f55776e310 msgr2=0x55f5577a0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:52.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.054+0000 7f30b0b04740 1 -- 172.21.15.120:0/3207447358 shutdown_connections 2024-09-19T09:24:52.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.054+0000 7f30b0b04740 1 -- 172.21.15.120:0/3207447358 wait complete. 2024-09-19T09:24:52.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.054+0000 7f30b0b04740 20 client.22641 trim_cache size 0 max 16384 2024-09-19T09:24:52.073 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:24:52.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.055+0000 7f30ae72b640 1 -- 172.21.15.120:0/3703273312 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f30940a3240 msgr2=0x7f30940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:52.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.055+0000 7f30ae72b640 1 --2- 172.21.15.120:0/3703273312 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f30940a3240 0x7f30940a5700 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f30a0009f50 tx=0x7f30a0004b50 comp rx=0 tx=0).stop 2024-09-19T09:24:52.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.055+0000 7f30ae72b640 1 -- 172.21.15.120:0/3703273312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30a8102db0 msgr2=0x7f30a81985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:52.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.055+0000 7f30ae72b640 1 --2- 172.21.15.120:0/3703273312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30a8102db0 0x7f30a81985f0 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f309800d950 tx=0x7f309800de10 comp rx=0 tx=0).stop 2024-09-19T09:24:52.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.055+0000 7f30ae72b640 1 -- 172.21.15.120:0/3703273312 shutdown_connections 2024-09-19T09:24:52.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.055+0000 7f30ae72b640 1 --2- 172.21.15.120:0/3703273312 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f30940a3240 0x7f30940a5700 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.055+0000 7f30ae72b640 1 --2- 172.21.15.120:0/3703273312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30a81082e0 0x7f30a819cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.055+0000 7f30ae72b640 1 --2- 172.21.15.120:0/3703273312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30a8103760 0x7f30a8198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.055+0000 7f30ae72b640 1 --2- 172.21.15.120:0/3703273312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30a8102db0 0x7f30a81985f0 unknown :-1 s=CLOSED pgs=2285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.055+0000 7f30ae72b640 1 -- 172.21.15.120:0/3703273312 >> 172.21.15.120:0/3703273312 conn(0x7f30a80fc7d0 msgr2=0x7f30a8106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:52.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.055+0000 7f30ae72b640 1 -- 172.21.15.120:0/3703273312 shutdown_connections 2024-09-19T09:24:52.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:52.055+0000 7f30ae72b640 1 -- 172.21.15.120:0/3703273312 wait complete. 2024-09-19T09:24:52.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:52 smithi120 ceph-mon[31150]: pgmap v1005: 97 pgs: 97 active+clean; 426 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 232 KiB/s wr, 73 op/s 2024-09-19T09:24:52.382 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199337 19.02 MB/sec execute 407 sec latency 24891.777 ms 2024-09-19T09:24:52.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:52 smithi196 ceph-mon[35150]: pgmap v1005: 97 pgs: 97 active+clean; 426 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 232 KiB/s wr, 73 op/s 2024-09-19T09:24:52.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:52 smithi153 ceph-mon[35071]: pgmap v1005: 97 pgs: 97 active+clean; 426 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 232 KiB/s wr, 73 op/s 2024-09-19T09:24:52.987 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.263+0000 7f57a5af1640 1 -- 172.21.15.120:0/374353868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57a0100a40 msgr2=0x7f57a0100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.263+0000 7f57a5af1640 1 --2- 172.21.15.120:0/374353868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57a0100a40 0x7f57a0100e40 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f5790009860 tx=0x7f579002a340 comp rx=0 tx=0).stop 2024-09-19T09:24:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.263+0000 7f57a5af1640 1 -- 172.21.15.120:0/374353868 shutdown_connections 2024-09-19T09:24:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.263+0000 7f57a5af1640 1 --2- 172.21.15.120:0/374353868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57a01025f0 0x7f57a0108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.263+0000 7f57a5af1640 1 --2- 172.21.15.120:0/374353868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57a0101c30 0x7f57a01020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.263+0000 7f57a5af1640 1 --2- 172.21.15.120:0/374353868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57a0100a40 0x7f57a0100e40 unknown :-1 s=CLOSED pgs=2290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.263+0000 7f57a5af1640 1 -- 172.21.15.120:0/374353868 >> 172.21.15.120:0/374353868 conn(0x7f57a00fc1e0 msgr2=0x7f57a00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.263+0000 7f57a5af1640 1 -- 172.21.15.120:0/374353868 shutdown_connections 2024-09-19T09:24:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.263+0000 7f57a5af1640 1 -- 172.21.15.120:0/374353868 wait complete. 2024-09-19T09:24:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.264+0000 7f57a5af1640 1 Processor -- start 2024-09-19T09:24:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.264+0000 7f57a5af1640 1 -- start start 2024-09-19T09:24:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.264+0000 7f57a5af1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57a0100a40 0x7f57a0197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.264+0000 7f57a5af1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57a0101c30 0x7f57a0198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.264+0000 7f57a5af1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57a01025f0 0x7f57a019f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.264+0000 7f57a5af1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f57a010c6c0 con 0x7f57a01025f0 2024-09-19T09:24:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.264+0000 7f57a5af1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f57a010c540 con 0x7f57a0101c30 2024-09-19T09:24:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.264+0000 7f57a5af1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f57a010c840 con 0x7f57a0100a40 2024-09-19T09:24:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.264+0000 7f579ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57a01025f0 0x7f57a019f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.264+0000 7f579ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57a01025f0 0x7f57a019f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57904/0 (socket says 172.21.15.120:57904) 2024-09-19T09:24:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.264+0000 7f579ffff640 1 -- 172.21.15.120:0/1310348439 learned_addr learned my addr 172.21.15.120:0/1310348439 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.264+0000 7f579f7fe640 1 --2- 172.21.15.120:0/1310348439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57a0100a40 0x7f57a0197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.264+0000 7f579effd640 1 --2- 172.21.15.120:0/1310348439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57a0101c30 0x7f57a0198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.264+0000 7f579ffff640 1 -- 172.21.15.120:0/1310348439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57a0100a40 msgr2=0x7f57a0197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.264+0000 7f579ffff640 1 --2- 172.21.15.120:0/1310348439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57a0100a40 0x7f57a0197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.264+0000 7f579ffff640 1 -- 172.21.15.120:0/1310348439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57a0101c30 msgr2=0x7f57a0198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.264+0000 7f579ffff640 1 --2- 172.21.15.120:0/1310348439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57a0101c30 0x7f57a0198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.264+0000 7f579ffff640 1 -- 172.21.15.120:0/1310348439 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a019fa30 con 0x7f57a01025f0 2024-09-19T09:24:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.265+0000 7f579effd640 1 --2- 172.21.15.120:0/1310348439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57a0101c30 0x7f57a0198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:24:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.265+0000 7f579ffff640 1 --2- 172.21.15.120:0/1310348439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57a01025f0 0x7f57a019f3c0 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f579800eb10 tx=0x7f579800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.265+0000 7f579cff9640 1 -- 172.21.15.120:0/1310348439 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5798004b20 con 0x7f57a01025f0 2024-09-19T09:24:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.265+0000 7f579cff9640 1 -- 172.21.15.120:0/1310348439 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5798004cc0 con 0x7f57a01025f0 2024-09-19T09:24:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.265+0000 7f57a5af1640 1 -- 172.21.15.120:0/1310348439 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57a019fd20 con 0x7f57a01025f0 2024-09-19T09:24:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.265+0000 7f579cff9640 1 -- 172.21.15.120:0/1310348439 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5798010760 con 0x7f57a01025f0 2024-09-19T09:24:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.265+0000 7f57a5af1640 1 -- 172.21.15.120:0/1310348439 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57a01a0260 con 0x7f57a01025f0 2024-09-19T09:24:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.267+0000 7f579cff9640 1 -- 172.21.15.120:0/1310348439 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5798005020 con 0x7f57a01025f0 2024-09-19T09:24:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.267+0000 7f57a8890740 1 Processor -- start 2024-09-19T09:24:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.268+0000 7f57a8890740 1 -- start start 2024-09-19T09:24:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.268+0000 7f579cff9640 1 --2- 172.21.15.120:0/1310348439 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f57880a3370 0x7f57880a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.268+0000 7f57a8890740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c06c449a0 0x560c06c44da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.268+0000 7f57a8890740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c06c452e0 0x560c06c49da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.268+0000 7f57a8890740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560c06c4a370 0x560c06c4c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.268+0000 7f57a8890740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560c06ba82f0 con 0x560c06c4a370 2024-09-19T09:24:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.268+0000 7f57a8890740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560c06b50270 con 0x560c06c449a0 2024-09-19T09:24:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.268+0000 7f57a8890740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560c06ba42b0 con 0x560c06c452e0 2024-09-19T09:24:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.268+0000 7f579effd640 1 --2- 172.21.15.120:0/1310348439 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f57880a3370 0x7f57880a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.268+0000 7f579ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560c06c4a370 0x560c06c4c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.268+0000 7f579effd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c06c452e0 0x560c06c49da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.268+0000 7f579cff9640 1 -- 172.21.15.120:0/1310348439 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f57980d00a0 con 0x7f57a01025f0 2024-09-19T09:24:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.268+0000 7f579ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560c06c4a370 0x560c06c4c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57908/0 (socket says 172.21.15.120:57908) 2024-09-19T09:24:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.268+0000 7f579ffff640 1 -- 172.21.15.120:0/2745104216 learned_addr learned my addr 172.21.15.120:0/2745104216 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.268+0000 7f579f7fe640 1 --2- 172.21.15.120:0/2745104216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c06c449a0 0x560c06c44da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.268+0000 7f579ffff640 1 -- 172.21.15.120:0/2745104216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c06c452e0 msgr2=0x560c06c49da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.268+0000 7f579ffff640 1 --2- 172.21.15.120:0/2745104216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c06c452e0 0x560c06c49da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.268+0000 7f579ffff640 1 -- 172.21.15.120:0/2745104216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c06c449a0 msgr2=0x560c06c44da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.268+0000 7f579ffff640 1 --2- 172.21.15.120:0/2745104216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c06c449a0 0x560c06c44da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.268+0000 7f579ffff640 1 -- 172.21.15.120:0/2745104216 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c06b4f340 con 0x560c06c4a370 2024-09-19T09:24:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.268+0000 7f579effd640 1 --2- 172.21.15.120:0/1310348439 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f57880a3370 0x7f57880a5830 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f57a0199320 tx=0x7f5794014410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.268+0000 7f579ffff640 1 --2- 172.21.15.120:0/2745104216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560c06c4a370 0x560c06c4c830 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f5798005880 tx=0x7f5798004f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c807fc37e875e7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.268+0000 7f5785ffb640 1 -- 172.21.15.120:0/2745104216 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579809c020 con 0x560c06c4a370 2024-09-19T09:24:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.269+0000 7f5785ffb640 1 -- 172.21.15.120:0/2745104216 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f579809cdc0 con 0x560c06c4a370 2024-09-19T09:24:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.269+0000 7f5785ffb640 1 -- 172.21.15.120:0/2745104216 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57980801b0 con 0x560c06c4a370 2024-09-19T09:24:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.269+0000 7f57a8890740 1 -- 172.21.15.120:0/2745104216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560c06c4a370 msgr2=0x560c06c4c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.269+0000 7f57a8890740 1 --2- 172.21.15.120:0/2745104216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560c06c4a370 0x560c06c4c830 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f5798005880 tx=0x7f5798004f40 comp rx=0 tx=0).stop 2024-09-19T09:24:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.269+0000 7f57a8890740 1 -- 172.21.15.120:0/2745104216 shutdown_connections 2024-09-19T09:24:53.280 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.269+0000 7f57a8890740 1 --2- 172.21.15.120:0/2745104216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560c06c4a370 0x560c06c4c830 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.280 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.269+0000 7f57a8890740 1 --2- 172.21.15.120:0/2745104216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c06c452e0 0x560c06c49da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.269+0000 7f57a8890740 1 --2- 172.21.15.120:0/2745104216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c06c449a0 0x560c06c44da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.269+0000 7f57a8890740 1 -- 172.21.15.120:0/2745104216 >> 172.21.15.120:0/2745104216 conn(0x560c06c40010 msgr2=0x560c06c40410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.269+0000 7f57a8890740 1 -- 172.21.15.120:0/2745104216 shutdown_connections 2024-09-19T09:24:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.270+0000 7f57a8890740 1 -- 172.21.15.120:0/2745104216 wait complete. 2024-09-19T09:24:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.270+0000 7f57a8890740 1 -- start start 2024-09-19T09:24:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.270+0000 7f57a8890740 1 Processor -- start 2024-09-19T09:24:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.270+0000 7f57a8890740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c06c449a0 0x560c06c4ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.270+0000 7f57a8890740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c06c452e0 0x560c06c4f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.270+0000 7f57a8890740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560c06c4a370 0x560c06c56130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.270+0000 7f57a8890740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560c06c4d340 con 0x560c06c4a370 2024-09-19T09:24:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.270+0000 7f57a8890740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560c06b31290 con 0x560c06c452e0 2024-09-19T09:24:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.270+0000 7f57a8890740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560c06c4d4c0 con 0x560c06c449a0 2024-09-19T09:24:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.270+0000 7f57a8890740 10 client.0 _ll_register_callbacks cb 0x560c06b62140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.271+0000 7f579f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c06c449a0 0x560c06c4ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.271+0000 7f579ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560c06c4a370 0x560c06c56130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.271+0000 7f579f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c06c449a0 0x560c06c4ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:33674/0 (socket says 172.21.15.120:33674) 2024-09-19T09:24:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.271+0000 7f579f7fe640 1 -- 172.21.15.120:0/1399865559 learned_addr learned my addr 172.21.15.120:0/1399865559 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.271+0000 7f579effd640 1 --2- 172.21.15.120:0/1399865559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c06c452e0 0x560c06c4f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.271+0000 7f579f7fe640 1 -- 172.21.15.120:0/1399865559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c06c452e0 msgr2=0x560c06c4f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.271+0000 7f579f7fe640 1 --2- 172.21.15.120:0/1399865559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c06c452e0 0x560c06c4f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.271+0000 7f579f7fe640 1 -- 172.21.15.120:0/1399865559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560c06c4a370 msgr2=0x560c06c56130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.271+0000 7f579f7fe640 1 --2- 172.21.15.120:0/1399865559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560c06c4a370 0x560c06c56130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.271+0000 7f579f7fe640 1 -- 172.21.15.120:0/1399865559 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c068bb210 con 0x560c06c449a0 2024-09-19T09:24:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.271+0000 7f579ffff640 1 --2- 172.21.15.120:0/1399865559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560c06c4a370 0x560c06c56130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:24:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.271+0000 7f579f7fe640 1 --2- 172.21.15.120:0/1399865559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c06c449a0 0x560c06c4ec20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5790000c00 tx=0x7f5790030860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e09dc97d2bbaedab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.271+0000 7f577effd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:24:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.271+0000 7f577effd640 1 -- 172.21.15.120:0/1399865559 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790037040 con 0x560c06c449a0 2024-09-19T09:24:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.271+0000 7f57a8890740 1 -- 172.21.15.120:0/1399865559 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f579809cdc0 con 0x560c06c449a0 2024-09-19T09:24:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.272+0000 7f577effd640 1 -- 172.21.15.120:0/1399865559 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f5790004240 con 0x560c06c449a0 2024-09-19T09:24:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.272+0000 7f577effd640 1 -- 172.21.15.120:0/1399865559 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5790004440 con 0x560c06c449a0 2024-09-19T09:24:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.272+0000 7f57a8890740 10 client.31198 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.272+0000 7f57a8890740 1 -- 172.21.15.120:0/1399865559 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560c06987620 con 0x560c06c449a0 2024-09-19T09:24:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.272+0000 7f577effd640 1 -- 172.21.15.120:0/1399865559 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f579809cdc0 con 0x560c06c449a0 2024-09-19T09:24:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.273+0000 7f577effd640 1 -- 172.21.15.120:0/1399865559 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f5790045030 con 0x560c06c449a0 2024-09-19T09:24:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.273+0000 7f57a8890740 10 client.31198 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:53.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.273+0000 7f57a8890740 10 client.31198 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:24:53.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.273+0000 7f57a8890740 20 client.31198 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:53.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.273+0000 7f57a8890740 1 --2- 172.21.15.120:0/1399865559 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560c06c5a8e0 0x560c06c5cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.273+0000 7f57a8890740 4 client.31198 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:24:53.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.273+0000 7f57a8890740 1 -- 172.21.15.120:0/1399865559 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c06bb5a20 con 0x560c06c5a8e0 2024-09-19T09:24:53.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.273+0000 7f579effd640 1 --2- 172.21.15.120:0/1399865559 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560c06c5a8e0 0x560c06c5cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.273+0000 7f577effd640 10 client.31198 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.273+0000 7f579effd640 1 --2- 172.21.15.120:0/1399865559 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560c06c5a8e0 0x560c06c5cd60 crc :-1 s=READY pgs=543 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba79b2eeadc0d28e server_cookie=91e2f1024cf05369 in_seq=0 out_seq=0 2024-09-19T09:24:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.274+0000 7f577effd640 1 -- 172.21.15.120:0/1399865559 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560c06bb5a20 con 0x560c06c5a8e0 2024-09-19T09:24:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.274+0000 7f577effd640 10 client.31198 handle_command_reply: tid=0 2024-09-19T09:24:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.274+0000 7f57a8890740 1 client.31198 shutdown 2024-09-19T09:24:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.274+0000 7f57a8890740 1 -- 172.21.15.120:0/1399865559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c06c449a0 msgr2=0x560c06c4ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.274+0000 7f57a8890740 1 --2- 172.21.15.120:0/1399865559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c06c449a0 0x560c06c4ec20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5790000c00 tx=0x7f5790030860 comp rx=0 tx=0).stop 2024-09-19T09:24:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.274+0000 7f57a8890740 1 -- 172.21.15.120:0/1399865559 shutdown_connections 2024-09-19T09:24:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.274+0000 7f57a8890740 1 --2- 172.21.15.120:0/1399865559 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560c06c5a8e0 0x560c06c5cd60 crc :-1 s=READY pgs=543 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.274+0000 7f57a8890740 1 --2- 172.21.15.120:0/1399865559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560c06c4a370 0x560c06c56130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.274+0000 7f577effd640 0 client.31198 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.274+0000 7f57a8890740 1 --2- 172.21.15.120:0/1399865559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c06c452e0 0x560c06c4f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.274+0000 7f57a8890740 1 --2- 172.21.15.120:0/1399865559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c06c449a0 0x560c06c4ec20 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.274+0000 7f57a8890740 1 -- 172.21.15.120:0/1399865559 >> 172.21.15.120:0/1399865559 conn(0x560c06c40010 msgr2=0x560c06c41820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.274+0000 7f57a8890740 1 -- 172.21.15.120:0/1399865559 shutdown_connections 2024-09-19T09:24:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.274+0000 7f57a8890740 1 -- 172.21.15.120:0/1399865559 wait complete. 2024-09-19T09:24:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.275+0000 7f57a8890740 20 client.31198 trim_cache size 0 max 16384 2024-09-19T09:24:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.290+0000 7f57a8890740 1 Processor -- start 2024-09-19T09:24:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.290+0000 7f57a8890740 1 -- start start 2024-09-19T09:24:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.290+0000 7f57a8890740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560c06c449a0 0x560c06c44d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.290+0000 7f57a8890740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c06c452e0 0x560c06c560c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.290+0000 7f57a8890740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c06c4a370 0x560c06c40f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.290+0000 7f57a8890740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f57a01a9140 con 0x560c06c449a0 2024-09-19T09:24:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.290+0000 7f57a8890740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f57a01aac80 con 0x560c06c452e0 2024-09-19T09:24:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.290+0000 7f57a8890740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f57a01a9810 con 0x560c06c4a370 2024-09-19T09:24:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.291+0000 7f579f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560c06c449a0 0x560c06c44d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.291+0000 7f579f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560c06c449a0 0x560c06c44d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57926/0 (socket says 172.21.15.120:57926) 2024-09-19T09:24:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.291+0000 7f579f7fe640 1 -- 172.21.15.120:0/2634349668 learned_addr learned my addr 172.21.15.120:0/2634349668 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.291+0000 7f579ffff640 1 --2- 172.21.15.120:0/2634349668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c06c4a370 0x560c06c40f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.291+0000 7f579effd640 1 --2- 172.21.15.120:0/2634349668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c06c452e0 0x560c06c560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.291+0000 7f579f7fe640 1 -- 172.21.15.120:0/2634349668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c06c4a370 msgr2=0x560c06c40f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.291+0000 7f579f7fe640 1 --2- 172.21.15.120:0/2634349668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c06c4a370 0x560c06c40f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.291+0000 7f579f7fe640 1 -- 172.21.15.120:0/2634349668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c06c452e0 msgr2=0x560c06c560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.291+0000 7f579f7fe640 1 --2- 172.21.15.120:0/2634349668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c06c452e0 0x560c06c560c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.291+0000 7f579f7fe640 1 -- 172.21.15.120:0/2634349668 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c06c5d3e0 con 0x560c06c449a0 2024-09-19T09:24:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.291+0000 7f579effd640 1 --2- 172.21.15.120:0/2634349668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c06c452e0 0x560c06c560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:24:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.291+0000 7f579f7fe640 1 --2- 172.21.15.120:0/2634349668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560c06c449a0 0x560c06c44d80 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f5790004b90 tx=0x7f5790001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491c1a57839bb562 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.291+0000 7f57857fa640 1 -- 172.21.15.120:0/2634349668 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790037030 con 0x560c06c449a0 2024-09-19T09:24:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.291+0000 7f57857fa640 1 -- 172.21.15.120:0/2634349668 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5790009450 con 0x560c06c449a0 2024-09-19T09:24:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.291+0000 7f57857fa640 1 -- 172.21.15.120:0/2634349668 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579003d480 con 0x560c06c449a0 2024-09-19T09:24:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.292+0000 7f57a8890740 1 -- 172.21.15.120:0/2634349668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560c06c449a0 msgr2=0x560c06c44d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.292+0000 7f57a8890740 1 --2- 172.21.15.120:0/2634349668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560c06c449a0 0x560c06c44d80 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f5790004b90 tx=0x7f5790001e70 comp rx=0 tx=0).stop 2024-09-19T09:24:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.292+0000 7f57a8890740 1 -- 172.21.15.120:0/2634349668 shutdown_connections 2024-09-19T09:24:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.292+0000 7f57a8890740 1 --2- 172.21.15.120:0/2634349668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c06c4a370 0x560c06c40f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.292+0000 7f57a8890740 1 --2- 172.21.15.120:0/2634349668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c06c452e0 0x560c06c560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.292+0000 7f57a8890740 1 --2- 172.21.15.120:0/2634349668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560c06c449a0 0x560c06c44d80 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.292+0000 7f57a8890740 1 -- 172.21.15.120:0/2634349668 >> 172.21.15.120:0/2634349668 conn(0x560c06c40010 msgr2=0x560c06c4e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.292+0000 7f57a8890740 1 -- 172.21.15.120:0/2634349668 shutdown_connections 2024-09-19T09:24:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.292+0000 7f57a8890740 1 -- 172.21.15.120:0/2634349668 wait complete. 2024-09-19T09:24:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.293+0000 7f57a8890740 1 -- start start 2024-09-19T09:24:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.293+0000 7f57a8890740 1 Processor -- start 2024-09-19T09:24:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.293+0000 7f57a8890740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560c06c449a0 0x560c06c53660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.293+0000 7f57a8890740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c06c452e0 0x560c06c53ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.293+0000 7f57a8890740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c06c4a370 0x560c06c54bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.293+0000 7f57a8890740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f57a01aa5b0 con 0x560c06c449a0 2024-09-19T09:24:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.293+0000 7f57a8890740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f57a01a9ee0 con 0x560c06c452e0 2024-09-19T09:24:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.293+0000 7f57a8890740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f57a01a7100 con 0x560c06c4a370 2024-09-19T09:24:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.293+0000 7f57a8890740 10 client.0 _ll_register_callbacks cb 0x7f5794005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.293+0000 7f579f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560c06c449a0 0x560c06c53660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.293+0000 7f579f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560c06c449a0 0x560c06c53660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57930/0 (socket says 172.21.15.120:57930) 2024-09-19T09:24:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.293+0000 7f579f7fe640 1 -- 172.21.15.120:0/2854948476 learned_addr learned my addr 172.21.15.120:0/2854948476 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.293+0000 7f579f7fe640 1 -- 172.21.15.120:0/2854948476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c06c4a370 msgr2=0x560c06c54bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.293+0000 7f579ffff640 1 --2- 172.21.15.120:0/2854948476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c06c4a370 0x560c06c54bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.294+0000 7f579f7fe640 1 --2- 172.21.15.120:0/2854948476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c06c4a370 0x560c06c54bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.294+0000 7f579f7fe640 1 -- 172.21.15.120:0/2854948476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c06c452e0 msgr2=0x560c06c53ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:53.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.294+0000 7f579effd640 1 --2- 172.21.15.120:0/2854948476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c06c452e0 0x560c06c53ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.294+0000 7f579f7fe640 1 --2- 172.21.15.120:0/2854948476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c06c452e0 0x560c06c53ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.294+0000 7f579f7fe640 1 -- 172.21.15.120:0/2854948476 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c06c4dbf0 con 0x560c06c449a0 2024-09-19T09:24:53.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.294+0000 7f579ffff640 1 --2- 172.21.15.120:0/2854948476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c06c4a370 0x560c06c54bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:24:53.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.294+0000 7f579f7fe640 1 --2- 172.21.15.120:0/2854948476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560c06c449a0 0x560c06c53660 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f5790004a90 tx=0x7f5790005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47fdcf0bb084911 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:53.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.294+0000 7f577effd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:53.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.294+0000 7f577effd640 1 -- 172.21.15.120:0/2854948476 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790054060 con 0x560c06c449a0 2024-09-19T09:24:53.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.294+0000 7f577effd640 1 -- 172.21.15.120:0/2854948476 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f5790004f80 con 0x560c06c449a0 2024-09-19T09:24:53.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.294+0000 7f57a8890740 1 -- 172.21.15.120:0/2854948476 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5790009450 con 0x560c06c449a0 2024-09-19T09:24:53.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.294+0000 7f577effd640 1 -- 172.21.15.120:0/2854948476 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5790037040 con 0x560c06c449a0 2024-09-19T09:24:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.294+0000 7f57a8890740 10 client.22659 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.294+0000 7f57a8890740 1 -- 172.21.15.120:0/2854948476 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560c06c56270 con 0x560c06c449a0 2024-09-19T09:24:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.294+0000 7f577effd640 1 -- 172.21.15.120:0/2854948476 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5790009450 con 0x560c06c449a0 2024-09-19T09:24:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.294+0000 7f577effd640 1 -- 172.21.15.120:0/2854948476 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f5790059070 con 0x560c06c449a0 2024-09-19T09:24:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.294+0000 7f57a8890740 10 client.22659 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.294+0000 7f57a8890740 10 client.22659 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:24:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.294+0000 7f57a8890740 20 client.22659 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.294+0000 7f57a8890740 1 --2- 172.21.15.120:0/2854948476 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560c06cc5ba0 0x560c06cc7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.294+0000 7f57a8890740 4 client.22659 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:24:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.294+0000 7f57a8890740 1 -- 172.21.15.120:0/2854948476 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560c06bafd70 con 0x560c06cc5ba0 2024-09-19T09:24:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.295+0000 7f579effd640 1 --2- 172.21.15.120:0/2854948476 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560c06cc5ba0 0x560c06cc7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.295+0000 7f577effd640 10 client.22659 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.295+0000 7f579effd640 1 --2- 172.21.15.120:0/2854948476 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560c06cc5ba0 0x560c06cc7fc0 crc :-1 s=READY pgs=544 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23097eb7fc97706d server_cookie=dcd3e249420e78c0 in_seq=0 out_seq=0 2024-09-19T09:24:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.325+0000 7f577effd640 1 -- 172.21.15.120:0/2854948476 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560c06bafd70 con 0x560c06cc5ba0 2024-09-19T09:24:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.326+0000 7f577effd640 10 client.22659 handle_command_reply: tid=0 2024-09-19T09:24:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.326+0000 7f57a8890740 1 client.22659 shutdown 2024-09-19T09:24:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.326+0000 7f57a8890740 1 -- 172.21.15.120:0/2854948476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560c06c449a0 msgr2=0x560c06c53660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.326+0000 7f57a8890740 1 --2- 172.21.15.120:0/2854948476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560c06c449a0 0x560c06c53660 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f5790004a90 tx=0x7f5790005f10 comp rx=0 tx=0).stop 2024-09-19T09:24:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.326+0000 7f57a8890740 1 -- 172.21.15.120:0/2854948476 shutdown_connections 2024-09-19T09:24:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.326+0000 7f57a8890740 1 --2- 172.21.15.120:0/2854948476 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560c06cc5ba0 0x560c06cc7fc0 crc :-1 s=READY pgs=544 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.326+0000 7f57a8890740 1 --2- 172.21.15.120:0/2854948476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c06c4a370 0x560c06c54bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.326+0000 7f577effd640 0 client.22659 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.326+0000 7f57a8890740 1 --2- 172.21.15.120:0/2854948476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560c06c452e0 0x560c06c53ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.326+0000 7f57a8890740 1 --2- 172.21.15.120:0/2854948476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560c06c449a0 0x560c06c53660 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.326+0000 7f57a8890740 1 -- 172.21.15.120:0/2854948476 >> 172.21.15.120:0/2854948476 conn(0x560c06c40010 msgr2=0x560c06c40f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.326+0000 7f57a8890740 1 -- 172.21.15.120:0/2854948476 shutdown_connections 2024-09-19T09:24:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.326+0000 7f57a8890740 1 -- 172.21.15.120:0/2854948476 wait complete. 2024-09-19T09:24:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.326+0000 7f57a8890740 20 client.22659 trim_cache size 0 max 16384 2024-09-19T09:24:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:24:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:24:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:24:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:24:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.327+0000 7f57a5af1640 1 -- 172.21.15.120:0/1310348439 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f57880a3370 msgr2=0x7f57880a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.327+0000 7f57a5af1640 1 --2- 172.21.15.120:0/1310348439 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f57880a3370 0x7f57880a5830 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f57a0199320 tx=0x7f5794014410 comp rx=0 tx=0).stop 2024-09-19T09:24:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.327+0000 7f57a5af1640 1 -- 172.21.15.120:0/1310348439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57a01025f0 msgr2=0x7f57a019f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.327+0000 7f57a5af1640 1 --2- 172.21.15.120:0/1310348439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57a01025f0 0x7f57a019f3c0 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f579800eb10 tx=0x7f579800c2d0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.327+0000 7f57a5af1640 1 -- 172.21.15.120:0/1310348439 shutdown_connections 2024-09-19T09:24:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.327+0000 7f57a5af1640 1 --2- 172.21.15.120:0/1310348439 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f57880a3370 0x7f57880a5830 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.327+0000 7f57a5af1640 1 --2- 172.21.15.120:0/1310348439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57a01025f0 0x7f57a019f3c0 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.327+0000 7f57a5af1640 1 --2- 172.21.15.120:0/1310348439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57a0101c30 0x7f57a0198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.327+0000 7f57a5af1640 1 --2- 172.21.15.120:0/1310348439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57a0100a40 0x7f57a0197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.327+0000 7f57a5af1640 1 -- 172.21.15.120:0/1310348439 >> 172.21.15.120:0/1310348439 conn(0x7f57a00fc1e0 msgr2=0x7f57a00fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.327+0000 7f57a5af1640 1 -- 172.21.15.120:0/1310348439 shutdown_connections 2024-09-19T09:24:53.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:24:53.327+0000 7f57a5af1640 1 -- 172.21.15.120:0/1310348439 wait complete. 2024-09-19T09:24:53.382 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199337 18.97 MB/sec execute 408 sec latency 25891.856 ms 2024-09-19T09:24:53.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.529+0000 7fd54e2cd640 1 -- 172.21.15.120:0/796685176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd548102570 msgr2=0x7fd5481029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:53.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.529+0000 7fd54e2cd640 1 --2- 172.21.15.120:0/796685176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd548102570 0x7fd5481029b0 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7fd53c009900 tx=0x7fd53c02a480 comp rx=0 tx=0).stop 2024-09-19T09:24:53.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.529+0000 7fd54e2cd640 1 -- 172.21.15.120:0/796685176 shutdown_connections 2024-09-19T09:24:53.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.529+0000 7fd54e2cd640 1 --2- 172.21.15.120:0/796685176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd548102ef0 0x7fd54810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.529+0000 7fd54e2cd640 1 --2- 172.21.15.120:0/796685176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd548102570 0x7fd5481029b0 unknown :-1 s=CLOSED pgs=2295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.529+0000 7fd54e2cd640 1 --2- 172.21.15.120:0/796685176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd548100e20 0x7fd548101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.529+0000 7fd54e2cd640 1 -- 172.21.15.120:0/796685176 >> 172.21.15.120:0/796685176 conn(0x7fd5480fc7d0 msgr2=0x7fd5480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:53.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.529+0000 7fd54e2cd640 1 -- 172.21.15.120:0/796685176 shutdown_connections 2024-09-19T09:24:53.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.529+0000 7fd54e2cd640 1 -- 172.21.15.120:0/796685176 wait complete. 2024-09-19T09:24:53.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.530+0000 7fd54e2cd640 1 Processor -- start 2024-09-19T09:24:53.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.530+0000 7fd54e2cd640 1 -- start start 2024-09-19T09:24:53.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.530+0000 7fd54e2cd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd548100e20 0x7fd54810e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.530+0000 7fd54e2cd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd548102570 0x7fd54810ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.530+0000 7fd54e2cd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd548102ef0 0x7fd54810ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.530+0000 7fd54e2cd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd5481120a0 con 0x7fd548100e20 2024-09-19T09:24:53.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.530+0000 7fd54e2cd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd548111f20 con 0x7fd548102ef0 2024-09-19T09:24:53.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.530+0000 7fd54e2cd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd548112220 con 0x7fd548102570 2024-09-19T09:24:53.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.530+0000 7fd547fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd548100e20 0x7fd54810e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.530+0000 7fd547fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd548100e20 0x7fd54810e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57948/0 (socket says 172.21.15.120:57948) 2024-09-19T09:24:53.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.530+0000 7fd547fff640 1 -- 172.21.15.120:0/232238330 learned_addr learned my addr 172.21.15.120:0/232238330 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:53.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.530+0000 7fd54c86a640 1 --2- 172.21.15.120:0/232238330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd548102ef0 0x7fd54810ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.530+0000 7fd5477fe640 1 --2- 172.21.15.120:0/232238330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd548102570 0x7fd54810ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.531+0000 7fd547fff640 1 -- 172.21.15.120:0/232238330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd548102570 msgr2=0x7fd54810ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:53.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.531+0000 7fd547fff640 1 --2- 172.21.15.120:0/232238330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd548102570 0x7fd54810ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.531+0000 7fd547fff640 1 -- 172.21.15.120:0/232238330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd548102ef0 msgr2=0x7fd54810ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:53.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.531+0000 7fd547fff640 1 --2- 172.21.15.120:0/232238330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd548102ef0 0x7fd54810ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.531+0000 7fd547fff640 1 -- 172.21.15.120:0/232238330 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5481a69c0 con 0x7fd548100e20 2024-09-19T09:24:53.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.531+0000 7fd547fff640 1 --2- 172.21.15.120:0/232238330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd548100e20 0x7fd54810e4d0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7fd53800e9f0 tx=0x7fd53800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:53.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.531+0000 7fd5457fa640 1 -- 172.21.15.120:0/232238330 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53800cd80 con 0x7fd548100e20 2024-09-19T09:24:53.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.531+0000 7fd5457fa640 1 -- 172.21.15.120:0/232238330 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd538010910 con 0x7fd548100e20 2024-09-19T09:24:53.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.531+0000 7fd54e2cd640 1 -- 172.21.15.120:0/232238330 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5481a6cb0 con 0x7fd548100e20 2024-09-19T09:24:53.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.531+0000 7fd5457fa640 1 -- 172.21.15.120:0/232238330 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd538018990 con 0x7fd548100e20 2024-09-19T09:24:53.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.531+0000 7fd54e2cd640 1 -- 172.21.15.120:0/232238330 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5481a71a0 con 0x7fd548100e20 2024-09-19T09:24:53.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.533+0000 7fd5457fa640 1 -- 172.21.15.120:0/232238330 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd538010ac0 con 0x7fd548100e20 2024-09-19T09:24:53.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.533+0000 7fd5506a6740 1 Processor -- start 2024-09-19T09:24:53.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.533+0000 7fd5506a6740 1 -- start start 2024-09-19T09:24:53.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.534+0000 7fd5506a6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770f409060 0x55770f409440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.534+0000 7fd5506a6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770f409980 0x55770f413d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.534+0000 7fd5506a6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770f414240 0x55770f416630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.534+0000 7fd5506a6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55770f36e620 con 0x55770f414240 2024-09-19T09:24:53.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.534+0000 7fd5506a6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55770f315e50 con 0x55770f409060 2024-09-19T09:24:53.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.534+0000 7fd5506a6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55770f36a5e0 con 0x55770f409980 2024-09-19T09:24:53.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.534+0000 7fd5477fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770f414240 0x55770f416630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.534+0000 7fd5457fa640 1 --2- 172.21.15.120:0/232238330 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd5300a3240 0x7fd5300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.534+0000 7fd5477fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770f414240 0x55770f416630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57962/0 (socket says 172.21.15.120:57962) 2024-09-19T09:24:53.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.534+0000 7fd5477fe640 1 -- 172.21.15.120:0/2715019073 learned_addr learned my addr 172.21.15.120:0/2715019073 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:53.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.534+0000 7fd547fff640 1 --2- 172.21.15.120:0/2715019073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770f409980 0x55770f413d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.534+0000 7fd54c86a640 1 --2- 172.21.15.120:0/232238330 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd5300a3240 0x7fd5300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.534+0000 7fd54c86a640 1 --2- 172.21.15.120:0/2715019073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770f409060 0x55770f409440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.534+0000 7fd5457fa640 1 -- 172.21.15.120:0/232238330 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd5380cfcb0 con 0x7fd548100e20 2024-09-19T09:24:53.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.534+0000 7fd5477fe640 1 -- 172.21.15.120:0/2715019073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770f409980 msgr2=0x55770f413d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:53.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.534+0000 7fd5477fe640 1 --2- 172.21.15.120:0/2715019073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770f409980 0x55770f413d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.534+0000 7fd5477fe640 1 -- 172.21.15.120:0/2715019073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770f409060 msgr2=0x55770f409440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:53.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.534+0000 7fd5477fe640 1 --2- 172.21.15.120:0/2715019073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770f409060 0x55770f409440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.534+0000 7fd5477fe640 1 -- 172.21.15.120:0/2715019073 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55770f306d40 con 0x55770f414240 2024-09-19T09:24:53.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.534+0000 7fd54c86a640 1 --2- 172.21.15.120:0/2715019073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770f409060 0x55770f409440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:24:53.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.534+0000 7fd5477fe640 1 --2- 172.21.15.120:0/2715019073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770f414240 0x55770f416630 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7fd53c02a9a0 tx=0x7fd53c005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f833eab2501effe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:53.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.534+0000 7fd54c86a640 1 --2- 172.21.15.120:0/232238330 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd5300a3240 0x7fd5300a5700 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd540009fd0 tx=0x7fd540009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:53.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.534+0000 7fd52e7fc640 1 -- 172.21.15.120:0/2715019073 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c035030 con 0x55770f414240 2024-09-19T09:24:53.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.535+0000 7fd52e7fc640 1 -- 172.21.15.120:0/2715019073 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd53c005520 con 0x55770f414240 2024-09-19T09:24:53.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.535+0000 7fd52e7fc640 1 -- 172.21.15.120:0/2715019073 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c0035f0 con 0x55770f414240 2024-09-19T09:24:53.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.535+0000 7fd5506a6740 1 -- 172.21.15.120:0/2715019073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770f414240 msgr2=0x55770f416630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:53.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.535+0000 7fd5506a6740 1 --2- 172.21.15.120:0/2715019073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770f414240 0x55770f416630 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7fd53c02a9a0 tx=0x7fd53c005e60 comp rx=0 tx=0).stop 2024-09-19T09:24:53.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.535+0000 7fd5506a6740 1 -- 172.21.15.120:0/2715019073 shutdown_connections 2024-09-19T09:24:53.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.535+0000 7fd5506a6740 1 --2- 172.21.15.120:0/2715019073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770f414240 0x55770f416630 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.535+0000 7fd5506a6740 1 --2- 172.21.15.120:0/2715019073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770f409980 0x55770f413d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.535+0000 7fd5506a6740 1 --2- 172.21.15.120:0/2715019073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770f409060 0x55770f409440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.535+0000 7fd5506a6740 1 -- 172.21.15.120:0/2715019073 >> 172.21.15.120:0/2715019073 conn(0x55770f4055f0 msgr2=0x55770f4059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:53.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.535+0000 7fd5506a6740 1 -- 172.21.15.120:0/2715019073 shutdown_connections 2024-09-19T09:24:53.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.535+0000 7fd5506a6740 1 -- 172.21.15.120:0/2715019073 wait complete. 2024-09-19T09:24:53.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.536+0000 7fd5506a6740 1 -- start start 2024-09-19T09:24:53.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.536+0000 7fd5506a6740 1 Processor -- start 2024-09-19T09:24:53.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.536+0000 7fd5506a6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770f409060 0x55770f411170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.536+0000 7fd5506a6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770f409980 0x55770f4116b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.536+0000 7fd5506a6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770f414240 0x55770f411bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.536+0000 7fd5506a6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55770f416e10 con 0x55770f409980 2024-09-19T09:24:53.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.536+0000 7fd5506a6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55770f2f6f10 con 0x55770f409060 2024-09-19T09:24:53.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.536+0000 7fd5506a6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55770f416f90 con 0x55770f414240 2024-09-19T09:24:53.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.536+0000 7fd5506a6740 10 client.0 _ll_register_callbacks cb 0x55770f31adb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:53.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.536+0000 7fd5477fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770f409980 0x55770f4116b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.536+0000 7fd5477fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770f409980 0x55770f4116b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57974/0 (socket says 172.21.15.120:57974) 2024-09-19T09:24:53.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.536+0000 7fd5477fe640 1 -- 172.21.15.120:0/1255885539 learned_addr learned my addr 172.21.15.120:0/1255885539 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:53.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.536+0000 7fd547fff640 1 --2- 172.21.15.120:0/1255885539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770f409060 0x55770f411170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.537+0000 7fd5477fe640 1 -- 172.21.15.120:0/1255885539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770f414240 msgr2=0x55770f411bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:53.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.537+0000 7fd5477fe640 1 --2- 172.21.15.120:0/1255885539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770f414240 0x55770f411bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.537+0000 7fd5477fe640 1 -- 172.21.15.120:0/1255885539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770f409060 msgr2=0x55770f411170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:53.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.537+0000 7fd5477fe640 1 --2- 172.21.15.120:0/1255885539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770f409060 0x55770f411170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.537+0000 7fd5477fe640 1 -- 172.21.15.120:0/1255885539 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55770f07d580 con 0x55770f409980 2024-09-19T09:24:53.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.537+0000 7fd547fff640 1 --2- 172.21.15.120:0/1255885539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770f409060 0x55770f411170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:24:53.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.537+0000 7fd5477fe640 1 --2- 172.21.15.120:0/1255885539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770f409980 0x55770f4116b0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7fd53c006010 tx=0x7fd53c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85da5c2b6e77dac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:53.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.537+0000 7fd5277fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:53.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.537+0000 7fd5277fe640 1 -- 172.21.15.120:0/1255885539 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c035030 con 0x55770f409980 2024-09-19T09:24:53.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.537+0000 7fd5506a6740 1 -- 172.21.15.120:0/1255885539 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd53c005520 con 0x55770f409980 2024-09-19T09:24:53.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.537+0000 7fd5277fe640 1 -- 172.21.15.120:0/1255885539 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd53c0050a0 con 0x55770f409980 2024-09-19T09:24:53.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.537+0000 7fd5277fe640 1 -- 172.21.15.120:0/1255885539 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd53c009520 con 0x55770f409980 2024-09-19T09:24:53.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.537+0000 7fd5506a6740 10 client.22671 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:53.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.537+0000 7fd5506a6740 1 -- 172.21.15.120:0/1255885539 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55770f147960 con 0x55770f409980 2024-09-19T09:24:53.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.537+0000 7fd5277fe640 1 -- 172.21.15.120:0/1255885539 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd53c005520 con 0x55770f409980 2024-09-19T09:24:53.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.537+0000 7fd5277fe640 1 -- 172.21.15.120:0/1255885539 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fd53c031700 con 0x55770f409980 2024-09-19T09:24:53.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.537+0000 7fd5506a6740 10 client.22671 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:53.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.537+0000 7fd5506a6740 10 client.22671 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:24:53.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.537+0000 7fd5506a6740 20 client.22671 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:53.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.537+0000 7fd5506a6740 1 --2- 172.21.15.120:0/1255885539 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55770f423c90 0x55770f426080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.537+0000 7fd5506a6740 4 client.22671 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:24:53.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.537+0000 7fd5506a6740 1 -- 172.21.15.120:0/1255885539 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55770f37bb90 con 0x55770f423c90 2024-09-19T09:24:53.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.538+0000 7fd547fff640 1 --2- 172.21.15.120:0/1255885539 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55770f423c90 0x55770f426080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.538+0000 7fd547fff640 1 --2- 172.21.15.120:0/1255885539 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55770f423c90 0x55770f426080 crc :-1 s=READY pgs=545 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5aabc2a41a41d4a8 server_cookie=68a0dd640fcbde79 in_seq=0 out_seq=0 2024-09-19T09:24:53.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.538+0000 7fd5277fe640 10 client.22671 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:53.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.539+0000 7fd5277fe640 1 -- 172.21.15.120:0/1255885539 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55770f37bb90 con 0x55770f423c90 2024-09-19T09:24:53.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.539+0000 7fd5277fe640 10 client.22671 handle_command_reply: tid=0 2024-09-19T09:24:53.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.539+0000 7fd5506a6740 1 client.22671 shutdown 2024-09-19T09:24:53.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.539+0000 7fd5506a6740 1 -- 172.21.15.120:0/1255885539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770f409980 msgr2=0x55770f4116b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:53.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.539+0000 7fd5506a6740 1 --2- 172.21.15.120:0/1255885539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770f409980 0x55770f4116b0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7fd53c006010 tx=0x7fd53c004470 comp rx=0 tx=0).stop 2024-09-19T09:24:53.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.539+0000 7fd5506a6740 1 -- 172.21.15.120:0/1255885539 shutdown_connections 2024-09-19T09:24:53.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.539+0000 7fd5506a6740 1 --2- 172.21.15.120:0/1255885539 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55770f423c90 0x55770f426080 crc :-1 s=READY pgs=545 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.539+0000 7fd5506a6740 1 --2- 172.21.15.120:0/1255885539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770f414240 0x55770f411bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.539+0000 7fd5506a6740 1 --2- 172.21.15.120:0/1255885539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770f409980 0x55770f4116b0 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.539+0000 7fd5506a6740 1 --2- 172.21.15.120:0/1255885539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770f409060 0x55770f411170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.539+0000 7fd5277fe640 0 client.22671 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:53.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.539+0000 7fd5506a6740 1 -- 172.21.15.120:0/1255885539 >> 172.21.15.120:0/1255885539 conn(0x55770f4055f0 msgr2=0x55770f40b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:53.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.539+0000 7fd5506a6740 1 -- 172.21.15.120:0/1255885539 shutdown_connections 2024-09-19T09:24:53.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.539+0000 7fd5506a6740 1 -- 172.21.15.120:0/1255885539 wait complete. 2024-09-19T09:24:53.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.539+0000 7fd5506a6740 20 client.22671 trim_cache size 0 max 16384 2024-09-19T09:24:53.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.556+0000 7fd5506a6740 1 Processor -- start 2024-09-19T09:24:53.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.556+0000 7fd5506a6740 1 -- start start 2024-09-19T09:24:53.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.556+0000 7fd5506a6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770f409060 0x55770f42b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.556+0000 7fd5506a6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770f409980 0x55770f42bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.556+0000 7fd5506a6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770f414240 0x55770f40e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.556+0000 7fd5506a6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd5481af750 con 0x55770f414240 2024-09-19T09:24:53.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.556+0000 7fd5506a6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd5481b1290 con 0x55770f409980 2024-09-19T09:24:53.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.556+0000 7fd5506a6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd5481afe20 con 0x55770f409060 2024-09-19T09:24:53.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.556+0000 7fd54c86a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770f414240 0x55770f40e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.556+0000 7fd54c86a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770f414240 0x55770f40e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57980/0 (socket says 172.21.15.120:57980) 2024-09-19T09:24:53.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.556+0000 7fd54c86a640 1 -- 172.21.15.120:0/3822788796 learned_addr learned my addr 172.21.15.120:0/3822788796 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:53.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.556+0000 7fd547fff640 1 --2- 172.21.15.120:0/3822788796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770f409060 0x55770f42b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.556+0000 7fd5477fe640 1 --2- 172.21.15.120:0/3822788796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770f409980 0x55770f42bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.556+0000 7fd54c86a640 1 -- 172.21.15.120:0/3822788796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770f409060 msgr2=0x55770f42b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:53.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.556+0000 7fd54c86a640 1 --2- 172.21.15.120:0/3822788796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770f409060 0x55770f42b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.556+0000 7fd54c86a640 1 -- 172.21.15.120:0/3822788796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770f409980 msgr2=0x55770f42bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:53.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.556+0000 7fd54c86a640 1 --2- 172.21.15.120:0/3822788796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770f409980 0x55770f42bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.556+0000 7fd54c86a640 1 -- 172.21.15.120:0/3822788796 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55770f406280 con 0x55770f414240 2024-09-19T09:24:53.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.557+0000 7fd54c86a640 1 --2- 172.21.15.120:0/3822788796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770f414240 0x55770f40e3f0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7fd540012ed0 tx=0x7fd540004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4f434fe85cbb06 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:53.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.557+0000 7fd52dffb640 1 -- 172.21.15.120:0/3822788796 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd540012590 con 0x55770f414240 2024-09-19T09:24:53.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.557+0000 7fd52dffb640 1 -- 172.21.15.120:0/3822788796 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd540011dc0 con 0x55770f414240 2024-09-19T09:24:53.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.557+0000 7fd52dffb640 1 -- 172.21.15.120:0/3822788796 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54002add0 con 0x55770f414240 2024-09-19T09:24:53.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.558+0000 7fd5506a6740 1 -- 172.21.15.120:0/3822788796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770f414240 msgr2=0x55770f40e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:53.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.558+0000 7fd5506a6740 1 --2- 172.21.15.120:0/3822788796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770f414240 0x55770f40e3f0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7fd540012ed0 tx=0x7fd540004ab0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.558+0000 7fd5506a6740 1 -- 172.21.15.120:0/3822788796 shutdown_connections 2024-09-19T09:24:53.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.558+0000 7fd5506a6740 1 --2- 172.21.15.120:0/3822788796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770f414240 0x55770f40e3f0 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.558+0000 7fd5506a6740 1 --2- 172.21.15.120:0/3822788796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770f409980 0x55770f42bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.558+0000 7fd5506a6740 1 --2- 172.21.15.120:0/3822788796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770f409060 0x55770f42b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.558+0000 7fd5506a6740 1 -- 172.21.15.120:0/3822788796 >> 172.21.15.120:0/3822788796 conn(0x55770f4055f0 msgr2=0x55770f40fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:53.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.558+0000 7fd5506a6740 1 -- 172.21.15.120:0/3822788796 shutdown_connections 2024-09-19T09:24:53.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.558+0000 7fd5506a6740 1 -- 172.21.15.120:0/3822788796 wait complete. 2024-09-19T09:24:53.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.558+0000 7fd5506a6740 1 -- start start 2024-09-19T09:24:53.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.558+0000 7fd5506a6740 1 Processor -- start 2024-09-19T09:24:53.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.558+0000 7fd5506a6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770f409060 0x55770f41c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.558+0000 7fd5506a6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770f409980 0x55770f41c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.558+0000 7fd5506a6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770f414240 0x55770f4191b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.558+0000 7fd5506a6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd5481b0bc0 con 0x55770f409060 2024-09-19T09:24:53.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.558+0000 7fd5506a6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd5481b04f0 con 0x55770f409980 2024-09-19T09:24:53.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.558+0000 7fd5506a6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd5481ad710 con 0x55770f414240 2024-09-19T09:24:53.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.559+0000 7fd5506a6740 10 client.0 _ll_register_callbacks cb 0x7fd53c0056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:53.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.559+0000 7fd547fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770f409060 0x55770f41c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.559+0000 7fd547fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770f409060 0x55770f41c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57996/0 (socket says 172.21.15.120:57996) 2024-09-19T09:24:53.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.559+0000 7fd547fff640 1 -- 172.21.15.120:0/2284072145 learned_addr learned my addr 172.21.15.120:0/2284072145 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:53.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.559+0000 7fd5477fe640 1 --2- 172.21.15.120:0/2284072145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770f409980 0x55770f41c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.559+0000 7fd54c86a640 1 --2- 172.21.15.120:0/2284072145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770f414240 0x55770f4191b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.559+0000 7fd547fff640 1 -- 172.21.15.120:0/2284072145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770f414240 msgr2=0x55770f4191b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:53.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.559+0000 7fd547fff640 1 --2- 172.21.15.120:0/2284072145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770f414240 0x55770f4191b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.559+0000 7fd547fff640 1 -- 172.21.15.120:0/2284072145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770f409980 msgr2=0x55770f41c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:53.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.559+0000 7fd547fff640 1 --2- 172.21.15.120:0/2284072145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770f409980 0x55770f41c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.559+0000 7fd547fff640 1 -- 172.21.15.120:0/2284072145 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55770f4198c0 con 0x55770f409060 2024-09-19T09:24:53.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.559+0000 7fd547fff640 1 --2- 172.21.15.120:0/2284072145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770f409060 0x55770f41c3c0 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7fd538098ca0 tx=0x7fd53806b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0041e73bcc94dbf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:53.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.559+0000 7fd5277fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:53.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.559+0000 7fd5277fe640 1 -- 172.21.15.120:0/2284072145 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd538026040 con 0x55770f409060 2024-09-19T09:24:53.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.559+0000 7fd5277fe640 1 -- 172.21.15.120:0/2284072145 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd538098940 con 0x55770f409060 2024-09-19T09:24:53.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.559+0000 7fd5506a6740 1 -- 172.21.15.120:0/2284072145 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd540011dc0 con 0x55770f409060 2024-09-19T09:24:53.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.559+0000 7fd5277fe640 1 -- 172.21.15.120:0/2284072145 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd5380b50a0 con 0x55770f409060 2024-09-19T09:24:53.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.559+0000 7fd5506a6740 10 client.22677 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:53.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.559+0000 7fd5506a6740 1 -- 172.21.15.120:0/2284072145 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55770f4978d0 con 0x55770f409060 2024-09-19T09:24:53.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.560+0000 7fd5277fe640 1 -- 172.21.15.120:0/2284072145 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd540011dc0 con 0x55770f409060 2024-09-19T09:24:53.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.560+0000 7fd5277fe640 1 -- 172.21.15.120:0/2284072145 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fd5380a0170 con 0x55770f409060 2024-09-19T09:24:53.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.560+0000 7fd5506a6740 10 client.22677 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:53.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.560+0000 7fd5506a6740 10 client.22677 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:24:53.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.560+0000 7fd5506a6740 20 client.22677 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:53.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.560+0000 7fd5506a6740 1 --2- 172.21.15.120:0/2284072145 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55770f498040 0x55770f49a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:53.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.560+0000 7fd5506a6740 4 client.22677 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "93457192", "release": true, "await": true}] 2024-09-19T09:24:53.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.560+0000 7fd5506a6740 1 -- 172.21.15.120:0/2284072145 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "set_id": "93457192", "release": true, "await": true}) -- 0x55770f3760a0 con 0x55770f498040 2024-09-19T09:24:53.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.560+0000 7fd5477fe640 1 --2- 172.21.15.120:0/2284072145 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55770f498040 0x55770f49a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:53.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.560+0000 7fd5277fe640 10 client.22677 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:53.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.560+0000 7fd5477fe640 1 --2- 172.21.15.120:0/2284072145 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55770f498040 0x55770f49a430 crc :-1 s=READY pgs=546 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=175d553e8399f92f server_cookie=f2514cb9a5c6e4d0 in_seq=0 out_seq=0 2024-09-19T09:24:53.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.632+0000 7fd5277fe640 1 -- 172.21.15.120:0/2284072145 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55770f3760a0 con 0x55770f498040 2024-09-19T09:24:53.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.632+0000 7fd5277fe640 10 client.22677 handle_command_reply: tid=0 2024-09-19T09:24:53.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.632+0000 7fd5506a6740 1 client.22677 shutdown 2024-09-19T09:24:53.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.632+0000 7fd5506a6740 1 -- 172.21.15.120:0/2284072145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770f409060 msgr2=0x55770f41c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:53.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.632+0000 7fd5506a6740 1 --2- 172.21.15.120:0/2284072145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770f409060 0x55770f41c3c0 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7fd538098ca0 tx=0x7fd53806b0d0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.632+0000 7fd5506a6740 1 -- 172.21.15.120:0/2284072145 shutdown_connections 2024-09-19T09:24:53.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.632+0000 7fd5506a6740 1 --2- 172.21.15.120:0/2284072145 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55770f498040 0x55770f49a430 crc :-1 s=READY pgs=546 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.632+0000 7fd5506a6740 1 --2- 172.21.15.120:0/2284072145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770f414240 0x55770f4191b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.632+0000 7fd5506a6740 1 --2- 172.21.15.120:0/2284072145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55770f409980 0x55770f41c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.632+0000 7fd5277fe640 0 client.22677 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:53.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.632+0000 7fd5506a6740 1 --2- 172.21.15.120:0/2284072145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55770f409060 0x55770f41c3c0 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.632+0000 7fd5506a6740 1 -- 172.21.15.120:0/2284072145 >> 172.21.15.120:0/2284072145 conn(0x55770f4055f0 msgr2=0x55770f403f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:53.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.633+0000 7fd5506a6740 1 -- 172.21.15.120:0/2284072145 shutdown_connections 2024-09-19T09:24:53.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.633+0000 7fd5506a6740 1 -- 172.21.15.120:0/2284072145 wait complete. 2024-09-19T09:24:53.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.633+0000 7fd5506a6740 20 client.22677 trim_cache size 0 max 16384 2024-09-19T09:24:53.635 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:24:53.635 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 39, 2024-09-19T09:24:53.636 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:24:53.636 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 76, 2024-09-19T09:24:53.636 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:24:53.636 INFO:teuthology.orchestra.run.smithi120.stdout: "93457192": { 2024-09-19T09:24:53.636 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 76, 2024-09-19T09:24:53.636 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:24:53.636 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:24:53.636 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:24:53.636 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:24:53.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:24:53.636 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:24:53.636 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 144.5, 2024-09-19T09:24:53.636 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:24:53.637 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:24:53.637 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:24:53.637 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:24:53.637 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:24:53.637 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:24:53.637 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:24:53.637 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:24:53.637 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:24:53.637 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:24:53.637 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:24:53.637 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:24:53.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.633+0000 7fd54e2cd640 1 -- 172.21.15.120:0/232238330 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd5300a3240 msgr2=0x7fd5300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:53.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.633+0000 7fd54e2cd640 1 --2- 172.21.15.120:0/232238330 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd5300a3240 0x7fd5300a5700 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd540009fd0 tx=0x7fd540009920 comp rx=0 tx=0).stop 2024-09-19T09:24:53.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.633+0000 7fd54e2cd640 1 -- 172.21.15.120:0/232238330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd548100e20 msgr2=0x7fd54810e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:53.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.633+0000 7fd54e2cd640 1 --2- 172.21.15.120:0/232238330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd548100e20 0x7fd54810e4d0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7fd53800e9f0 tx=0x7fd53800eeb0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.634+0000 7fd54e2cd640 1 -- 172.21.15.120:0/232238330 shutdown_connections 2024-09-19T09:24:53.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.634+0000 7fd54e2cd640 1 --2- 172.21.15.120:0/232238330 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd5300a3240 0x7fd5300a5700 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.634+0000 7fd54e2cd640 1 --2- 172.21.15.120:0/232238330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd548102ef0 0x7fd54810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.634+0000 7fd54e2cd640 1 --2- 172.21.15.120:0/232238330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd548102570 0x7fd54810ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.634+0000 7fd54e2cd640 1 --2- 172.21.15.120:0/232238330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd548100e20 0x7fd54810e4d0 unknown :-1 s=CLOSED pgs=2296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:53.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.634+0000 7fd54e2cd640 1 -- 172.21.15.120:0/232238330 >> 172.21.15.120:0/232238330 conn(0x7fd5480fc7d0 msgr2=0x7fd5480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:53.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.634+0000 7fd54e2cd640 1 -- 172.21.15.120:0/232238330 shutdown_connections 2024-09-19T09:24:53.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:53.634+0000 7fd54e2cd640 1 -- 172.21.15.120:0/232238330 wait complete. 2024-09-19T09:24:53.925 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:24:53.926 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:24:54.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:54 smithi120 ceph-mon[31150]: pgmap v1006: 97 pgs: 97 active+clean; 426 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 38 KiB/s wr, 44 op/s 2024-09-19T09:24:54.382 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199943 19.02 MB/sec execute 409 sec latency 26138.429 ms 2024-09-19T09:24:54.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:54 smithi196 ceph-mon[35150]: pgmap v1006: 97 pgs: 97 active+clean; 426 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 38 KiB/s wr, 44 op/s 2024-09-19T09:24:54.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:54 smithi153 ceph-mon[35071]: pgmap v1006: 97 pgs: 97 active+clean; 426 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 38 KiB/s wr, 44 op/s 2024-09-19T09:24:54.459 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '93457192', total seconds elapsed: 29.9 2024-09-19T09:24:54.581 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:55.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.157+0000 7fac3afab640 1 -- 172.21.15.120:0/2663469202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac34102790 msgr2=0x7fac341067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:55.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.157+0000 7fac3afab640 1 --2- 172.21.15.120:0/2663469202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac34102790 0x7fac341067e0 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7fac20009900 tx=0x7fac2002a410 comp rx=0 tx=0).stop 2024-09-19T09:24:55.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.157+0000 7fac3afab640 1 -- 172.21.15.120:0/2663469202 shutdown_connections 2024-09-19T09:24:55.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.157+0000 7fac3afab640 1 --2- 172.21.15.120:0/2663469202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac34106f10 0x7fac3410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.157+0000 7fac3afab640 1 --2- 172.21.15.120:0/2663469202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac34102790 0x7fac341067e0 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.157+0000 7fac3afab640 1 --2- 172.21.15.120:0/2663469202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac34101de0 0x7fac341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.157+0000 7fac3afab640 1 -- 172.21.15.120:0/2663469202 >> 172.21.15.120:0/2663469202 conn(0x7fac340fd8e0 msgr2=0x7fac340ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:55.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.157+0000 7fac3afab640 1 -- 172.21.15.120:0/2663469202 shutdown_connections 2024-09-19T09:24:55.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.157+0000 7fac3afab640 1 -- 172.21.15.120:0/2663469202 wait complete. 2024-09-19T09:24:55.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.158+0000 7fac3afab640 1 Processor -- start 2024-09-19T09:24:55.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.158+0000 7fac3afab640 1 -- start start 2024-09-19T09:24:55.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.158+0000 7fac3afab640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac34101de0 0x7fac340744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:55.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.158+0000 7fac3afab640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac34102790 0x7fac34074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:55.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.158+0000 7fac3afab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac34106f10 0x7fac34074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:55.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.158+0000 7fac3afab640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fac34111f70 con 0x7fac34106f10 2024-09-19T09:24:55.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.158+0000 7fac3afab640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fac34111df0 con 0x7fac34102790 2024-09-19T09:24:55.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.158+0000 7fac3afab640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fac341120f0 con 0x7fac34101de0 2024-09-19T09:24:55.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.158+0000 7fac33fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac34102790 0x7fac34074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:55.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.158+0000 7fac38d47640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac34101de0 0x7fac340744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:55.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.158+0000 7fac33fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac34102790 0x7fac34074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37714/0 (socket says 172.21.15.120:37714) 2024-09-19T09:24:55.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.158+0000 7fac33fff640 1 -- 172.21.15.120:0/2576289331 learned_addr learned my addr 172.21.15.120:0/2576289331 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:55.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.158+0000 7fac39548640 1 --2- 172.21.15.120:0/2576289331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac34106f10 0x7fac34074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:55.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.159+0000 7fac33fff640 1 -- 172.21.15.120:0/2576289331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac34101de0 msgr2=0x7fac340744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:55.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.159+0000 7fac33fff640 1 --2- 172.21.15.120:0/2576289331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac34101de0 0x7fac340744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.159+0000 7fac33fff640 1 -- 172.21.15.120:0/2576289331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac34106f10 msgr2=0x7fac34074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:55.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.159+0000 7fac33fff640 1 --2- 172.21.15.120:0/2576289331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac34106f10 0x7fac34074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.159+0000 7fac33fff640 1 -- 172.21.15.120:0/2576289331 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac341a68f0 con 0x7fac34102790 2024-09-19T09:24:55.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.159+0000 7fac33fff640 1 --2- 172.21.15.120:0/2576289331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac34102790 0x7fac34074a30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fac20006010 tx=0x7fac20004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:55.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.159+0000 7fac31ffb640 1 -- 172.21.15.120:0/2576289331 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2003f030 con 0x7fac34102790 2024-09-19T09:24:55.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.159+0000 7fac31ffb640 1 -- 172.21.15.120:0/2576289331 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac20004440 con 0x7fac34102790 2024-09-19T09:24:55.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.159+0000 7fac31ffb640 1 -- 172.21.15.120:0/2576289331 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac20035070 con 0x7fac34102790 2024-09-19T09:24:55.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.159+0000 7fac3afab640 1 -- 172.21.15.120:0/2576289331 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac341a6b80 con 0x7fac34102790 2024-09-19T09:24:55.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.160+0000 7fac3afab640 1 -- 172.21.15.120:0/2576289331 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac341a7010 con 0x7fac34102790 2024-09-19T09:24:55.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.162+0000 7fac31ffb640 1 -- 172.21.15.120:0/2576289331 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fac20042030 con 0x7fac34102790 2024-09-19T09:24:55.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.163+0000 7fac3d384740 1 Processor -- start 2024-09-19T09:24:55.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.163+0000 7fac3d384740 1 -- start start 2024-09-19T09:24:55.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.164+0000 7fac31ffb640 1 --2- 172.21.15.120:0/2576289331 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fac240a3340 0x7fac240a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:55.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.164+0000 7fac3d384740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559add54db60 0x559add54df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:55.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.164+0000 7fac3d384740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559add54e480 0x559add558800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:55.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.164+0000 7fac3d384740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559add558d40 0x559add55b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:55.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.164+0000 7fac3d384740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559add4b3280 con 0x559add54e480 2024-09-19T09:24:55.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.164+0000 7fac3d384740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559add45ab10 con 0x559add54db60 2024-09-19T09:24:55.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.164+0000 7fac3d384740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559add4af240 con 0x559add558d40 2024-09-19T09:24:55.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.164+0000 7fac31ffb640 1 -- 172.21.15.120:0/2576289331 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fac200efcc0 con 0x7fac34102790 2024-09-19T09:24:55.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.164+0000 7fac38d47640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559add54db60 0x559add54df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:55.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.164+0000 7fac38d47640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559add54db60 0x559add54df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37722/0 (socket says 172.21.15.120:37722) 2024-09-19T09:24:55.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.164+0000 7fac38d47640 1 -- 172.21.15.120:0/2479518516 learned_addr learned my addr 172.21.15.120:0/2479518516 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:55.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.164+0000 7fac39548640 1 --2- 172.21.15.120:0/2576289331 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fac240a3340 0x7fac240a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:55.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.164+0000 7fac39548640 1 --2- 172.21.15.120:0/2479518516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559add558d40 0x559add55b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:55.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.164+0000 7fac38d47640 1 -- 172.21.15.120:0/2479518516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559add558d40 msgr2=0x559add55b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:55.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.164+0000 7fac38d47640 1 --2- 172.21.15.120:0/2479518516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559add558d40 0x559add55b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.164+0000 7fac38d47640 1 -- 172.21.15.120:0/2479518516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559add54e480 msgr2=0x559add558800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:55.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.164+0000 7fac38d47640 1 --2- 172.21.15.120:0/2479518516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559add54e480 0x559add558800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.164+0000 7fac38d47640 1 -- 172.21.15.120:0/2479518516 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559add4671f0 con 0x559add54db60 2024-09-19T09:24:55.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.165+0000 7fac38d47640 1 --2- 172.21.15.120:0/2479518516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559add54db60 0x559add54df40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fac28009cf0 tx=0x7fac28011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99937af45c8f1140 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:55.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.165+0000 7fac39548640 1 --2- 172.21.15.120:0/2576289331 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fac240a3340 0x7fac240a5800 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fac2c009270 tx=0x7fac2c009d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:55.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.165+0000 7fac1effd640 1 -- 172.21.15.120:0/2479518516 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac28004220 con 0x559add54db60 2024-09-19T09:24:55.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.165+0000 7fac1effd640 1 -- 172.21.15.120:0/2479518516 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac280043c0 con 0x559add54db60 2024-09-19T09:24:55.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.165+0000 7fac1effd640 1 -- 172.21.15.120:0/2479518516 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac28018910 con 0x559add54db60 2024-09-19T09:24:55.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.166+0000 7fac3d384740 1 -- 172.21.15.120:0/2479518516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559add54db60 msgr2=0x559add54df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:55.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.166+0000 7fac3d384740 1 --2- 172.21.15.120:0/2479518516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559add54db60 0x559add54df40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fac28009cf0 tx=0x7fac28011810 comp rx=0 tx=0).stop 2024-09-19T09:24:55.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.166+0000 7fac3d384740 1 -- 172.21.15.120:0/2479518516 shutdown_connections 2024-09-19T09:24:55.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.166+0000 7fac3d384740 1 --2- 172.21.15.120:0/2479518516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559add558d40 0x559add55b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.166+0000 7fac3d384740 1 --2- 172.21.15.120:0/2479518516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559add54e480 0x559add558800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.166+0000 7fac3d384740 1 --2- 172.21.15.120:0/2479518516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559add54db60 0x559add54df40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.166+0000 7fac3d384740 1 -- 172.21.15.120:0/2479518516 >> 172.21.15.120:0/2479518516 conn(0x559add54a0f0 msgr2=0x559add54a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:55.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.166+0000 7fac3d384740 1 -- 172.21.15.120:0/2479518516 shutdown_connections 2024-09-19T09:24:55.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.166+0000 7fac3d384740 1 -- 172.21.15.120:0/2479518516 wait complete. 2024-09-19T09:24:55.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.166+0000 7fac3d384740 1 -- start start 2024-09-19T09:24:55.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.167+0000 7fac3d384740 1 Processor -- start 2024-09-19T09:24:55.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.167+0000 7fac3d384740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559add54db60 0x559add555c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:55.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.167+0000 7fac3d384740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559add54e480 0x559add556180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:55.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.167+0000 7fac3d384740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559add558d40 0x559add5566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:55.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.167+0000 7fac3d384740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559add55b910 con 0x559add558d40 2024-09-19T09:24:55.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.167+0000 7fac3d384740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559add46df10 con 0x559add54e480 2024-09-19T09:24:55.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.167+0000 7fac3d384740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559add55ba90 con 0x559add54db60 2024-09-19T09:24:55.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.167+0000 7fac3d384740 10 client.0 _ll_register_callbacks cb 0x559add436a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:55.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.167+0000 7fac33fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559add54e480 0x559add556180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:55.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.167+0000 7fac33fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559add54e480 0x559add556180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37728/0 (socket says 172.21.15.120:37728) 2024-09-19T09:24:55.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.167+0000 7fac33fff640 1 -- 172.21.15.120:0/1796693963 learned_addr learned my addr 172.21.15.120:0/1796693963 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:55.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.167+0000 7fac39548640 1 --2- 172.21.15.120:0/1796693963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559add558d40 0x559add5566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:55.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.168+0000 7fac33fff640 1 -- 172.21.15.120:0/1796693963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559add54db60 msgr2=0x559add555c40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:24:55.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.168+0000 7fac33fff640 1 --2- 172.21.15.120:0/1796693963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559add54db60 0x559add555c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.168+0000 7fac33fff640 1 -- 172.21.15.120:0/1796693963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559add558d40 msgr2=0x559add5566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:55.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.168+0000 7fac33fff640 1 --2- 172.21.15.120:0/1796693963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559add558d40 0x559add5566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.168+0000 7fac33fff640 1 -- 172.21.15.120:0/1796693963 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559add2246a0 con 0x559add54e480 2024-09-19T09:24:55.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.168+0000 7fac33fff640 1 --2- 172.21.15.120:0/1796693963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559add54e480 0x559add556180 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fac200ed6f0 tx=0x7fac200bc0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9565a741b3fb83de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:55.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.168+0000 7fac13fff640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:24:55.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.168+0000 7fac13fff640 1 -- 172.21.15.120:0/1796693963 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac20049040 con 0x559add54e480 2024-09-19T09:24:55.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.168+0000 7fac13fff640 1 -- 172.21.15.120:0/1796693963 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fac200b8910 con 0x559add54e480 2024-09-19T09:24:55.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.168+0000 7fac3d384740 1 -- 172.21.15.120:0/1796693963 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac280043c0 con 0x559add54e480 2024-09-19T09:24:55.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.168+0000 7fac13fff640 1 -- 172.21.15.120:0/1796693963 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fac200bc9a0 con 0x559add54e480 2024-09-19T09:24:55.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.168+0000 7fac3d384740 10 client.31367 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:55.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.168+0000 7fac3d384740 1 -- 172.21.15.120:0/1796693963 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559add567570 con 0x559add54e480 2024-09-19T09:24:55.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.168+0000 7fac13fff640 1 -- 172.21.15.120:0/1796693963 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac200bc2a0 con 0x559add54e480 2024-09-19T09:24:55.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.169+0000 7fac13fff640 1 -- 172.21.15.120:0/1796693963 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fac2003bbf0 con 0x559add54e480 2024-09-19T09:24:55.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.169+0000 7fac3d384740 10 client.31367 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:55.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.169+0000 7fac3d384740 10 client.31367 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:24:55.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.169+0000 7fac3d384740 20 client.31367 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:55.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.169+0000 7fac3d384740 1 --2- 172.21.15.120:0/1796693963 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559add568910 0x559add56ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:55.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.169+0000 7fac3d384740 4 client.31367 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:24:55.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.169+0000 7fac3d384740 1 -- 172.21.15.120:0/1796693963 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fac280043c0 con 0x559add568910 2024-09-19T09:24:55.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.169+0000 7fac38d47640 1 --2- 172.21.15.120:0/1796693963 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559add568910 0x559add56ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:55.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.169+0000 7fac13fff640 10 client.31367 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:55.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.169+0000 7fac38d47640 1 --2- 172.21.15.120:0/1796693963 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559add568910 0x559add56ad50 crc :-1 s=READY pgs=547 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ff4d8aa07a85068 server_cookie=3a1c12e9a9cbf48c in_seq=0 out_seq=0 2024-09-19T09:24:55.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.172+0000 7fac13fff640 1 -- 172.21.15.120:0/1796693963 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fac280043c0 con 0x559add568910 2024-09-19T09:24:55.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.172+0000 7fac13fff640 10 client.31367 handle_command_reply: tid=0 2024-09-19T09:24:55.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.172+0000 7fac3d384740 1 client.31367 shutdown 2024-09-19T09:24:55.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.173+0000 7fac3d384740 1 -- 172.21.15.120:0/1796693963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559add54e480 msgr2=0x559add556180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:55.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.173+0000 7fac3d384740 1 --2- 172.21.15.120:0/1796693963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559add54e480 0x559add556180 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fac200ed6f0 tx=0x7fac200bc0f0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.173+0000 7fac3d384740 1 -- 172.21.15.120:0/1796693963 shutdown_connections 2024-09-19T09:24:55.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.173+0000 7fac3d384740 1 --2- 172.21.15.120:0/1796693963 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559add568910 0x559add56ad50 crc :-1 s=READY pgs=547 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.173+0000 7fac13fff640 0 client.31367 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:55.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.173+0000 7fac3d384740 1 --2- 172.21.15.120:0/1796693963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559add558d40 0x559add5566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.173+0000 7fac3d384740 1 --2- 172.21.15.120:0/1796693963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559add54e480 0x559add556180 secure :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fac200ed6f0 tx=0x7fac200bc0f0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.173+0000 7fac3d384740 1 --2- 172.21.15.120:0/1796693963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559add54db60 0x559add555c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.173+0000 7fac3d384740 1 -- 172.21.15.120:0/1796693963 >> 172.21.15.120:0/1796693963 conn(0x559add54a0f0 msgr2=0x559add54fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:55.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.173+0000 7fac3d384740 1 -- 172.21.15.120:0/1796693963 shutdown_connections 2024-09-19T09:24:55.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.173+0000 7fac3d384740 1 -- 172.21.15.120:0/1796693963 wait complete. 2024-09-19T09:24:55.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.173+0000 7fac3d384740 20 client.31367 trim_cache size 0 max 16384 2024-09-19T09:24:55.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.197+0000 7fac3d384740 1 Processor -- start 2024-09-19T09:24:55.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.197+0000 7fac3d384740 1 -- start start 2024-09-19T09:24:55.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.197+0000 7fac3d384740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559add54db60 0x559add556510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:55.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.197+0000 7fac3d384740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559add558d40 0x559add557aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:55.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.197+0000 7fac3d384740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559add568910 0x559add570620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:55.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.197+0000 7fac3d384740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fac341af750 con 0x559add558d40 2024-09-19T09:24:55.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.197+0000 7fac3d384740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fac341b1290 con 0x559add54db60 2024-09-19T09:24:55.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.197+0000 7fac3d384740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fac341afe20 con 0x559add568910 2024-09-19T09:24:55.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.198+0000 7fac33fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559add558d40 0x559add557aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:55.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.198+0000 7fac33fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559add558d40 0x559add557aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58030/0 (socket says 172.21.15.120:58030) 2024-09-19T09:24:55.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.198+0000 7fac33fff640 1 -- 172.21.15.120:0/3144788530 learned_addr learned my addr 172.21.15.120:0/3144788530 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:55.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.198+0000 7fac39548640 1 --2- 172.21.15.120:0/3144788530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559add568910 0x559add570620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:55.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.198+0000 7fac33fff640 1 -- 172.21.15.120:0/3144788530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559add568910 msgr2=0x559add570620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:55.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.198+0000 7fac33fff640 1 --2- 172.21.15.120:0/3144788530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559add568910 0x559add570620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.198+0000 7fac38d47640 1 --2- 172.21.15.120:0/3144788530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559add54db60 0x559add556510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:55.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.198+0000 7fac33fff640 1 -- 172.21.15.120:0/3144788530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559add54db60 msgr2=0x559add556510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:55.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.198+0000 7fac33fff640 1 --2- 172.21.15.120:0/3144788530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559add54db60 0x559add556510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.198+0000 7fac33fff640 1 -- 172.21.15.120:0/3144788530 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559add54e2b0 con 0x559add558d40 2024-09-19T09:24:55.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.198+0000 7fac33fff640 1 --2- 172.21.15.120:0/3144788530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559add558d40 0x559add557aa0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7fac200ba550 tx=0x7fac200ba7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3319fe81147859 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:55.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.198+0000 7fac1e7fc640 1 -- 172.21.15.120:0/3144788530 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac20049050 con 0x559add558d40 2024-09-19T09:24:55.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.199+0000 7fac1e7fc640 1 -- 172.21.15.120:0/3144788530 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac200ab1a0 con 0x559add558d40 2024-09-19T09:24:55.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.199+0000 7fac1e7fc640 1 -- 172.21.15.120:0/3144788530 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2003bbf0 con 0x559add558d40 2024-09-19T09:24:55.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.199+0000 7fac3d384740 1 -- 172.21.15.120:0/3144788530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559add558d40 msgr2=0x559add557aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:55.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.199+0000 7fac3d384740 1 --2- 172.21.15.120:0/3144788530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559add558d40 0x559add557aa0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7fac200ba550 tx=0x7fac200ba7b0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.200+0000 7fac3d384740 1 -- 172.21.15.120:0/3144788530 shutdown_connections 2024-09-19T09:24:55.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.200+0000 7fac3d384740 1 --2- 172.21.15.120:0/3144788530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559add568910 0x559add570620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.200+0000 7fac3d384740 1 --2- 172.21.15.120:0/3144788530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559add558d40 0x559add557aa0 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.200+0000 7fac3d384740 1 --2- 172.21.15.120:0/3144788530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559add54db60 0x559add556510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.200+0000 7fac3d384740 1 -- 172.21.15.120:0/3144788530 >> 172.21.15.120:0/3144788530 conn(0x559add54a0f0 msgr2=0x559add557390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:55.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.200+0000 7fac3d384740 1 -- 172.21.15.120:0/3144788530 shutdown_connections 2024-09-19T09:24:55.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.200+0000 7fac3d384740 1 -- 172.21.15.120:0/3144788530 wait complete. 2024-09-19T09:24:55.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.200+0000 7fac3d384740 1 -- start start 2024-09-19T09:24:55.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.200+0000 7fac3d384740 1 Processor -- start 2024-09-19T09:24:55.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.201+0000 7fac3d384740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559add54db60 0x559add5613a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:55.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.201+0000 7fac3d384740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559add558d40 0x559add55d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:55.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.201+0000 7fac3d384740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559add568910 0x559add55dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:55.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.201+0000 7fac3d384740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fac341b0bc0 con 0x559add568910 2024-09-19T09:24:55.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.201+0000 7fac3d384740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fac341b04f0 con 0x559add54db60 2024-09-19T09:24:55.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.201+0000 7fac3d384740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fac341ad710 con 0x559add558d40 2024-09-19T09:24:55.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.201+0000 7fac38d47640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559add54db60 0x559add5613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:55.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.201+0000 7fac39548640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559add568910 0x559add55dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:55.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.201+0000 7fac33fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559add558d40 0x559add55d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:55.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.201+0000 7fac38d47640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559add54db60 0x559add5613a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37750/0 (socket says 172.21.15.120:37750) 2024-09-19T09:24:55.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.201+0000 7fac38d47640 1 -- 172.21.15.120:0/2007219198 learned_addr learned my addr 172.21.15.120:0/2007219198 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:55.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.201+0000 7fac3d384740 10 client.0 _ll_register_callbacks cb 0x7fac200b8540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:55.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.201+0000 7fac38d47640 1 -- 172.21.15.120:0/2007219198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559add558d40 msgr2=0x559add55d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:55.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.201+0000 7fac38d47640 1 --2- 172.21.15.120:0/2007219198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559add558d40 0x559add55d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.201+0000 7fac38d47640 1 -- 172.21.15.120:0/2007219198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559add568910 msgr2=0x559add55dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:55.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.201+0000 7fac38d47640 1 --2- 172.21.15.120:0/2007219198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559add568910 0x559add55dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.201+0000 7fac38d47640 1 -- 172.21.15.120:0/2007219198 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559add55e400 con 0x559add54db60 2024-09-19T09:24:55.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.202+0000 7fac38d47640 1 --2- 172.21.15.120:0/2007219198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559add54db60 0x559add5613a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fac28011d10 tx=0x7fac280183d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36e79435dfc9353a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:55.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.202+0000 7fac13fff640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:24:55.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.202+0000 7fac13fff640 1 -- 172.21.15.120:0/2007219198 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac28009910 con 0x559add54db60 2024-09-19T09:24:55.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.202+0000 7fac13fff640 1 -- 172.21.15.120:0/2007219198 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fac28005030 con 0x559add54db60 2024-09-19T09:24:55.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.202+0000 7fac3d384740 1 -- 172.21.15.120:0/2007219198 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac200ab1a0 con 0x559add54db60 2024-09-19T09:24:55.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.202+0000 7fac13fff640 1 -- 172.21.15.120:0/2007219198 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fac28005230 con 0x559add54db60 2024-09-19T09:24:55.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.202+0000 7fac3d384740 10 client.31370 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:55.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.202+0000 7fac3d384740 1 -- 172.21.15.120:0/2007219198 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559add5d45b0 con 0x559add54db60 2024-09-19T09:24:55.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.203+0000 7fac13fff640 1 -- 172.21.15.120:0/2007219198 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac200ab1a0 con 0x559add54db60 2024-09-19T09:24:55.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.203+0000 7fac13fff640 1 -- 172.21.15.120:0/2007219198 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fac2802d020 con 0x559add54db60 2024-09-19T09:24:55.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.203+0000 7fac3d384740 10 client.31370 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:55.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.203+0000 7fac3d384740 10 client.31370 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:24:55.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.203+0000 7fac3d384740 20 client.31370 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:55.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.203+0000 7fac3d384740 1 --2- 172.21.15.120:0/2007219198 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559add5d4b40 0x559add5d6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:55.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.203+0000 7fac3d384740 4 client.31370 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:24:55.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.203+0000 7fac3d384740 1 -- 172.21.15.120:0/2007219198 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559add4c07f0 con 0x559add5d4b40 2024-09-19T09:24:55.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.204+0000 7fac33fff640 1 --2- 172.21.15.120:0/2007219198 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559add5d4b40 0x559add5d6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:55.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.204+0000 7fac33fff640 1 --2- 172.21.15.120:0/2007219198 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559add5d4b40 0x559add5d6f30 crc :-1 s=READY pgs=548 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86525b79789ff2f2 server_cookie=70675dfb62fa98be in_seq=0 out_seq=0 2024-09-19T09:24:55.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.204+0000 7fac13fff640 10 client.31370 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:55.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.206+0000 7fac13fff640 1 -- 172.21.15.120:0/2007219198 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559add4c07f0 con 0x559add5d4b40 2024-09-19T09:24:55.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.206+0000 7fac13fff640 10 client.31370 handle_command_reply: tid=0 2024-09-19T09:24:55.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.206+0000 7fac3d384740 1 client.31370 shutdown 2024-09-19T09:24:55.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.206+0000 7fac3d384740 1 -- 172.21.15.120:0/2007219198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559add54db60 msgr2=0x559add5613a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:55.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.206+0000 7fac3d384740 1 --2- 172.21.15.120:0/2007219198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559add54db60 0x559add5613a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fac28011d10 tx=0x7fac280183d0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.207+0000 7fac3d384740 1 -- 172.21.15.120:0/2007219198 shutdown_connections 2024-09-19T09:24:55.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.207+0000 7fac3d384740 1 --2- 172.21.15.120:0/2007219198 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559add5d4b40 0x559add5d6f30 crc :-1 s=READY pgs=548 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.207+0000 7fac3d384740 1 --2- 172.21.15.120:0/2007219198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559add568910 0x559add55dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.207+0000 7fac3d384740 1 --2- 172.21.15.120:0/2007219198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559add558d40 0x559add55d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.207+0000 7fac3d384740 1 --2- 172.21.15.120:0/2007219198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559add54db60 0x559add5613a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.207+0000 7fac3d384740 1 -- 172.21.15.120:0/2007219198 >> 172.21.15.120:0/2007219198 conn(0x559add54a0f0 msgr2=0x559add557aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:55.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.207+0000 7fac13fff640 0 client.31370 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:55.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.207+0000 7fac3d384740 1 -- 172.21.15.120:0/2007219198 shutdown_connections 2024-09-19T09:24:55.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.207+0000 7fac3d384740 1 -- 172.21.15.120:0/2007219198 wait complete. 2024-09-19T09:24:55.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.207+0000 7fac3d384740 20 client.31370 trim_cache size 0 max 16384 2024-09-19T09:24:55.210 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:24:55.210 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:24:55.210 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "8d6dc70c-4714-45fc-87d7-1dfecf9b783d", 2024-09-19T09:24:55.210 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:24:55.210 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:24:55.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.208+0000 7fac3afab640 1 -- 172.21.15.120:0/2576289331 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fac240a3340 msgr2=0x7fac240a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:55.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.208+0000 7fac3afab640 1 --2- 172.21.15.120:0/2576289331 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fac240a3340 0x7fac240a5800 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fac2c009270 tx=0x7fac2c009d50 comp rx=0 tx=0).stop 2024-09-19T09:24:55.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.208+0000 7fac3afab640 1 -- 172.21.15.120:0/2576289331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac34102790 msgr2=0x7fac34074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:55.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.208+0000 7fac3afab640 1 --2- 172.21.15.120:0/2576289331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac34102790 0x7fac34074a30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fac20006010 tx=0x7fac20004040 comp rx=0 tx=0).stop 2024-09-19T09:24:55.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.208+0000 7fac3afab640 1 -- 172.21.15.120:0/2576289331 shutdown_connections 2024-09-19T09:24:55.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.208+0000 7fac3afab640 1 --2- 172.21.15.120:0/2576289331 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fac240a3340 0x7fac240a5800 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.208+0000 7fac3afab640 1 --2- 172.21.15.120:0/2576289331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac34106f10 0x7fac34074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.208+0000 7fac3afab640 1 --2- 172.21.15.120:0/2576289331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac34102790 0x7fac34074a30 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.208+0000 7fac3afab640 1 --2- 172.21.15.120:0/2576289331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac34101de0 0x7fac340744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.208+0000 7fac3afab640 1 -- 172.21.15.120:0/2576289331 >> 172.21.15.120:0/2576289331 conn(0x7fac340fd8e0 msgr2=0x7fac340fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:55.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.208+0000 7fac3afab640 1 -- 172.21.15.120:0/2576289331 shutdown_connections 2024-09-19T09:24:55.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:55.208+0000 7fac3afab640 1 -- 172.21.15.120:0/2576289331 wait complete. 2024-09-19T09:24:55.382 INFO:tasks.workunit.client.0.smithi120.stdout: 10 200571 19.09 MB/sec execute 410 sec latency 109.796 ms 2024-09-19T09:24:55.678 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:55 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:24:55.679 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:55 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:55.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:55 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:24:55.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:55 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:55.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:55 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:24:55.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:55 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:55.688 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 19 09:24:55 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-b[64524]: 2024-09-19T09:24:55.348+0000 7f1ed1965640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000b60e [...2,head] /volumes/qa/sv_1/f2bee15c-e720-4829-ac9c-1a0401c9d37d/client.0/tmp/clients/client5/~dmtmp/PARADOX/ auth{1=1,2=1,3=1,4=1} v30065 ap=2 RANDEPHEMERALPIN f(v1066 m2024-09-19T09:24:27.038380+0000 24=24+0) n(v40 rc2024-09-19T09:24:27.038380+0000 b5230239 25=24+1) (inest sync->mix g=1,2,3,4) caps={15204=pAsLsXsFs/pAsLsXsFsx@2344} mcw={1=AsLsXsFs} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x560de9a06680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000b60e:[<0x1000000b49b/PARADOX v268>,<0x1000000b49a/~dmtmp v47>,<0x10000000007/client5 v1645>,<0x10000000006/clients v1215>,<0x10000000005/tmp v1179>,<0x100000001fb/client.0 v1176>,<0x100000001fa/f2bee15c-e720-4829-ac9c-1a0401c9d37d v1163>,<0x10000000001/sv_1 v1171>,<0x10000000000/qa v1139>,<0x1/volumes v1126>]//[]","memoryvalue":"(2)0x1000000b60e:[<0x1000000b49b/PARADOX v30065>,<0x1000000b49a/~dmtmp v2922>,<0x10000000007/client5 v9431>,<0x10000000006/clients v2573>,<0x10000000005/tmp v2088>,<0x100000001fb/client.0 v2038>,<0x100000001fa/f2bee15c-e720-4829-ac9c-1a0401c9d37d v2016>,<0x10000000001/sv_1 v2000>,<0x10000000000/qa v1968>,<0x1/volumes v1921>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T09:24:26.989381+0000 25=25+0)","ondisk_value.rstat":"n(v0 rc2024-09-19T09:24:27.038380+0000 b5230239 25=24+1)","memory_value.dirstat":"f(v1066 m2024-09-19T09:24:27.038380+0000 24=24+0)","memory_value.rstat":"n(v40 rc2024-09-19T09:24:27.038380+0000 b5230239 25=24+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-19T09:24:56.038 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:24:56.304 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:56.382 INFO:tasks.workunit.client.0.smithi120.stdout: 10 200987 19.09 MB/sec execute 411 sec latency 159.139 ms 2024-09-19T09:24:56.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:56 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:24:56.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:56 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:56.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:56 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:24:56.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:56 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:24:56.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:56 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:56.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:56 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:24:56.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:56 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:56.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:56 smithi196 ceph-mon[35150]: Scrub error on inode 0x1000000b60e (/volumes/qa/sv_1/f2bee15c-e720-4829-ac9c-1a0401c9d37d/client.0/tmp/clients/client5/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-09-19T09:24:56.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:56 smithi196 ceph-mon[35150]: pgmap v1007: 97 pgs: 97 active+clean; 426 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 47 KiB/s wr, 22 op/s 2024-09-19T09:24:56.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:56 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:24:56.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:56 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:56.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:56 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:24:56.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:56 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:24:56.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:56 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:56.689 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:56 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:24:56.689 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:56 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:56.689 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:56 smithi153 ceph-mon[35071]: Scrub error on inode 0x1000000b60e (/volumes/qa/sv_1/f2bee15c-e720-4829-ac9c-1a0401c9d37d/client.0/tmp/clients/client5/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-09-19T09:24:56.689 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:56 smithi153 ceph-mon[35071]: pgmap v1007: 97 pgs: 97 active+clean; 426 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 47 KiB/s wr, 22 op/s 2024-09-19T09:24:56.735 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:56 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:24:56.735 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:56 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:56.735 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:56 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:24:56.735 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:56 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:24:56.735 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:56 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:56.735 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:56 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:24:56.735 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:56 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:56.736 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:56 smithi120 ceph-mon[31150]: Scrub error on inode 0x1000000b60e (/volumes/qa/sv_1/f2bee15c-e720-4829-ac9c-1a0401c9d37d/client.0/tmp/clients/client5/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-09-19T09:24:56.736 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:56 smithi120 ceph-mon[31150]: pgmap v1007: 97 pgs: 97 active+clean; 426 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 47 KiB/s wr, 22 op/s 2024-09-19T09:24:56.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.832+0000 7fc07b674640 1 -- 172.21.15.120:0/3436960570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc074102570 msgr2=0x7fc0741029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:56.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.832+0000 7fc07b674640 1 --2- 172.21.15.120:0/3436960570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc074102570 0x7fc0741029b0 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7fc064009900 tx=0x7fc06402a480 comp rx=0 tx=0).stop 2024-09-19T09:24:56.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.832+0000 7fc07b674640 1 -- 172.21.15.120:0/3436960570 shutdown_connections 2024-09-19T09:24:56.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.832+0000 7fc07b674640 1 --2- 172.21.15.120:0/3436960570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc074102ef0 0x7fc07410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.832+0000 7fc07b674640 1 --2- 172.21.15.120:0/3436960570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc074102570 0x7fc0741029b0 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.832+0000 7fc07b674640 1 --2- 172.21.15.120:0/3436960570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc074100e20 0x7fc074101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.832+0000 7fc07b674640 1 -- 172.21.15.120:0/3436960570 >> 172.21.15.120:0/3436960570 conn(0x7fc0740fc7d0 msgr2=0x7fc0740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:56.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.832+0000 7fc07b674640 1 -- 172.21.15.120:0/3436960570 shutdown_connections 2024-09-19T09:24:56.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.832+0000 7fc07b674640 1 -- 172.21.15.120:0/3436960570 wait complete. 2024-09-19T09:24:56.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.833+0000 7fc07b674640 1 Processor -- start 2024-09-19T09:24:56.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.833+0000 7fc07b674640 1 -- start start 2024-09-19T09:24:56.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.833+0000 7fc07b674640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc074100e20 0x7fc0740fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:56.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.833+0000 7fc07b674640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc074102570 0x7fc0740fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:56.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.833+0000 7fc07b674640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc074102ef0 0x7fc0740fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:56.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.833+0000 7fc07b674640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc074110f80 con 0x7fc074100e20 2024-09-19T09:24:56.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.833+0000 7fc07b674640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc074110e00 con 0x7fc074102570 2024-09-19T09:24:56.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.833+0000 7fc07b674640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc074111100 con 0x7fc074102ef0 2024-09-19T09:24:56.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.833+0000 7fc079410640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc074100e20 0x7fc0740fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:56.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.833+0000 7fc079410640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc074100e20 0x7fc0740fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58046/0 (socket says 172.21.15.120:58046) 2024-09-19T09:24:56.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.833+0000 7fc079410640 1 -- 172.21.15.120:0/1812591664 learned_addr learned my addr 172.21.15.120:0/1812591664 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:56.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.833+0000 7fc079c11640 1 --2- 172.21.15.120:0/1812591664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc074102ef0 0x7fc0740fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:56.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.833+0000 7fc078c0f640 1 --2- 172.21.15.120:0/1812591664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc074102570 0x7fc0740fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:56.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.833+0000 7fc079410640 1 -- 172.21.15.120:0/1812591664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc074102ef0 msgr2=0x7fc0740fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:56.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.833+0000 7fc079410640 1 --2- 172.21.15.120:0/1812591664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc074102ef0 0x7fc0740fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.833+0000 7fc079410640 1 -- 172.21.15.120:0/1812591664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc074102570 msgr2=0x7fc0740fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:56.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.833+0000 7fc079410640 1 --2- 172.21.15.120:0/1812591664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc074102570 0x7fc0740fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.834+0000 7fc079410640 1 -- 172.21.15.120:0/1812591664 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0741a69a0 con 0x7fc074100e20 2024-09-19T09:24:56.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.834+0000 7fc079410640 1 --2- 172.21.15.120:0/1812591664 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc074100e20 0x7fc0740fcc80 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7fc07000d950 tx=0x7fc07000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:56.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.834+0000 7fc06a7fc640 1 -- 172.21.15.120:0/1812591664 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc070017070 con 0x7fc074100e20 2024-09-19T09:24:56.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.834+0000 7fc07b674640 1 -- 172.21.15.120:0/1812591664 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0741a6c90 con 0x7fc074100e20 2024-09-19T09:24:56.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.834+0000 7fc06a7fc640 1 -- 172.21.15.120:0/1812591664 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc070009c30 con 0x7fc074100e20 2024-09-19T09:24:56.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.834+0000 7fc06a7fc640 1 -- 172.21.15.120:0/1812591664 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07000f6b0 con 0x7fc074100e20 2024-09-19T09:24:56.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.834+0000 7fc07b674640 1 -- 172.21.15.120:0/1812591664 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0741a71a0 con 0x7fc074100e20 2024-09-19T09:24:56.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.836+0000 7fc06a7fc640 1 -- 172.21.15.120:0/1812591664 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fc0700109d0 con 0x7fc074100e20 2024-09-19T09:24:56.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.837+0000 7fc07da4d740 1 Processor -- start 2024-09-19T09:24:56.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.837+0000 7fc07da4d740 1 -- start start 2024-09-19T09:24:56.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.837+0000 7fc06a7fc640 1 --2- 172.21.15.120:0/1812591664 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc0600a3240 0x7fc0600a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:56.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.837+0000 7fc07da4d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559263eeb4a0 0x559263eeb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:56.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.837+0000 7fc07da4d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559263eebdc0 0x559263ef6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:56.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.837+0000 7fc07da4d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559263ef6680 0x559263ef8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:56.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.837+0000 7fc07da4d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559263e512e0 con 0x559263eebdc0 2024-09-19T09:24:56.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.837+0000 7fc07da4d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559263df8ce0 con 0x559263ef6680 2024-09-19T09:24:56.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.837+0000 7fc07da4d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559263e4d2a0 con 0x559263eeb4a0 2024-09-19T09:24:56.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.837+0000 7fc06a7fc640 1 -- 172.21.15.120:0/1812591664 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc0700cf470 con 0x7fc074100e20 2024-09-19T09:24:56.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.837+0000 7fc079c11640 1 --2- 172.21.15.120:0/1812591664 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc0600a3240 0x7fc0600a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:56.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.837+0000 7fc079410640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559263eeb4a0 0x559263eeb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:56.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.837+0000 7fc079410640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559263eeb4a0 0x559263eeb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:33820/0 (socket says 172.21.15.120:33820) 2024-09-19T09:24:56.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.837+0000 7fc079410640 1 -- 172.21.15.120:0/515725612 learned_addr learned my addr 172.21.15.120:0/515725612 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:56.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.838+0000 7fc078c0f640 1 --2- 172.21.15.120:0/515725612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559263eebdc0 0x559263ef6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:56.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.838+0000 7fc079c11640 1 --2- 172.21.15.120:0/515725612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559263ef6680 0x559263ef8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:56.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.838+0000 7fc079410640 1 -- 172.21.15.120:0/515725612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559263ef6680 msgr2=0x559263ef8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:56.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.838+0000 7fc079410640 1 --2- 172.21.15.120:0/515725612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559263ef6680 0x559263ef8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.838+0000 7fc079410640 1 -- 172.21.15.120:0/515725612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559263eebdc0 msgr2=0x559263ef6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:56.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.838+0000 7fc079410640 1 --2- 172.21.15.120:0/515725612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559263eebdc0 0x559263ef6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.838+0000 7fc079410640 1 -- 172.21.15.120:0/515725612 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559263ddd440 con 0x559263eeb4a0 2024-09-19T09:24:56.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.838+0000 7fc079c11640 1 --2- 172.21.15.120:0/1812591664 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc0600a3240 0x7fc0600a5700 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009990 tx=0x7fc06c012ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:56.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.838+0000 7fc079410640 1 --2- 172.21.15.120:0/515725612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559263eeb4a0 0x559263eeb880 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc0700a41d0 tx=0x7fc07007f630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5baaecf4f6e2234 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:56.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.838+0000 7fc05f7fe640 1 -- 172.21.15.120:0/515725612 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc070020040 con 0x559263eeb4a0 2024-09-19T09:24:56.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.838+0000 7fc05f7fe640 1 -- 172.21.15.120:0/515725612 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc07009fe40 con 0x559263eeb4a0 2024-09-19T09:24:56.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.838+0000 7fc05f7fe640 1 -- 172.21.15.120:0/515725612 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07007f810 con 0x559263eeb4a0 2024-09-19T09:24:56.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.839+0000 7fc07da4d740 1 -- 172.21.15.120:0/515725612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559263eeb4a0 msgr2=0x559263eeb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:56.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.839+0000 7fc07da4d740 1 --2- 172.21.15.120:0/515725612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559263eeb4a0 0x559263eeb880 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc0700a41d0 tx=0x7fc07007f630 comp rx=0 tx=0).stop 2024-09-19T09:24:56.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.839+0000 7fc07da4d740 1 -- 172.21.15.120:0/515725612 shutdown_connections 2024-09-19T09:24:56.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.839+0000 7fc07da4d740 1 --2- 172.21.15.120:0/515725612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559263ef6680 0x559263ef8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.839+0000 7fc07da4d740 1 --2- 172.21.15.120:0/515725612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559263eebdc0 0x559263ef6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.839+0000 7fc07da4d740 1 --2- 172.21.15.120:0/515725612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559263eeb4a0 0x559263eeb880 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.839+0000 7fc07da4d740 1 -- 172.21.15.120:0/515725612 >> 172.21.15.120:0/515725612 conn(0x559263ee7a30 msgr2=0x559263ee7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:56.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.839+0000 7fc07da4d740 1 -- 172.21.15.120:0/515725612 shutdown_connections 2024-09-19T09:24:56.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.839+0000 7fc07da4d740 1 -- 172.21.15.120:0/515725612 wait complete. 2024-09-19T09:24:56.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.840+0000 7fc07da4d740 1 -- start start 2024-09-19T09:24:56.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.840+0000 7fc07da4d740 1 Processor -- start 2024-09-19T09:24:56.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.840+0000 7fc07da4d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559263eeb4a0 0x559263ef5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:56.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.840+0000 7fc07da4d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559263eebdc0 0x559263ef32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:56.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.840+0000 7fc07da4d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559263ef6680 0x559263ef3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:56.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.840+0000 7fc07da4d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559263ef91f0 con 0x559263eeb4a0 2024-09-19T09:24:56.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.840+0000 7fc07da4d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559263e0c050 con 0x559263eebdc0 2024-09-19T09:24:56.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.840+0000 7fc07da4d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559263ef9370 con 0x559263ef6680 2024-09-19T09:24:56.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.840+0000 7fc07da4d740 10 client.0 _ll_register_callbacks cb 0x559263e4c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:56.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.840+0000 7fc079410640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559263eeb4a0 0x559263ef5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:56.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.840+0000 7fc079410640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559263eeb4a0 0x559263ef5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58056/0 (socket says 172.21.15.120:58056) 2024-09-19T09:24:56.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.840+0000 7fc078c0f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559263eebdc0 0x559263ef32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:56.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.840+0000 7fc079410640 1 -- 172.21.15.120:0/3668503487 learned_addr learned my addr 172.21.15.120:0/3668503487 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:56.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.840+0000 7fc079c11640 1 --2- 172.21.15.120:0/3668503487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559263ef6680 0x559263ef3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:56.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.841+0000 7fc079410640 1 -- 172.21.15.120:0/3668503487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559263ef6680 msgr2=0x559263ef3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:56.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.841+0000 7fc079410640 1 --2- 172.21.15.120:0/3668503487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559263ef6680 0x559263ef3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.841+0000 7fc079410640 1 -- 172.21.15.120:0/3668503487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559263eebdc0 msgr2=0x559263ef32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:56.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.841+0000 7fc079410640 1 --2- 172.21.15.120:0/3668503487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559263eebdc0 0x559263ef32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.841+0000 7fc079410640 1 -- 172.21.15.120:0/3668503487 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559263bc23c0 con 0x559263eeb4a0 2024-09-19T09:24:56.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.841+0000 7fc079410640 1 --2- 172.21.15.120:0/3668503487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559263eeb4a0 0x559263ef5ff0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7fc070097820 tx=0x7fc070097de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7880d34f41608755 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:56.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.841+0000 7fc05cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:56.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.841+0000 7fc05cff9640 1 -- 172.21.15.120:0/3668503487 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc070020050 con 0x559263eeb4a0 2024-09-19T09:24:56.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.841+0000 7fc05cff9640 1 -- 172.21.15.120:0/3668503487 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc070086110 con 0x559263eeb4a0 2024-09-19T09:24:56.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.841+0000 7fc07da4d740 1 -- 172.21.15.120:0/3668503487 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc07009fe40 con 0x559263eeb4a0 2024-09-19T09:24:56.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.841+0000 7fc05cff9640 1 -- 172.21.15.120:0/3668503487 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc0700cf2d0 con 0x559263eeb4a0 2024-09-19T09:24:56.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.841+0000 7fc07da4d740 10 client.22704 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:56.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.841+0000 7fc07da4d740 1 -- 172.21.15.120:0/3668503487 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559263f04bc0 con 0x559263eeb4a0 2024-09-19T09:24:56.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.841+0000 7fc05cff9640 1 -- 172.21.15.120:0/3668503487 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc07009fe40 con 0x559263eeb4a0 2024-09-19T09:24:56.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.841+0000 7fc05cff9640 1 -- 172.21.15.120:0/3668503487 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fc07009aa00 con 0x559263eeb4a0 2024-09-19T09:24:56.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.841+0000 7fc07da4d740 10 client.22704 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:56.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.841+0000 7fc07da4d740 10 client.22704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:24:56.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.841+0000 7fc07da4d740 20 client.22704 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:56.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.841+0000 7fc07da4d740 1 --2- 172.21.15.120:0/3668503487 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559263f05f10 0x559263f08300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:56.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.841+0000 7fc07da4d740 4 client.22704 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:24:56.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.841+0000 7fc07da4d740 1 -- 172.21.15.120:0/3668503487 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559263e5e850 con 0x559263f05f10 2024-09-19T09:24:56.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.842+0000 7fc078c0f640 1 --2- 172.21.15.120:0/3668503487 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559263f05f10 0x559263f08300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:56.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.842+0000 7fc078c0f640 1 --2- 172.21.15.120:0/3668503487 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559263f05f10 0x559263f08300 crc :-1 s=READY pgs=549 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6f35f19ec06945e server_cookie=fc99ec2a485bd907 in_seq=0 out_seq=0 2024-09-19T09:24:56.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.842+0000 7fc05cff9640 10 client.22704 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:56.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.845+0000 7fc05cff9640 1 -- 172.21.15.120:0/3668503487 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559263e5e850 con 0x559263f05f10 2024-09-19T09:24:56.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.845+0000 7fc05cff9640 10 client.22704 handle_command_reply: tid=0 2024-09-19T09:24:56.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.845+0000 7fc07da4d740 1 client.22704 shutdown 2024-09-19T09:24:56.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.845+0000 7fc07da4d740 1 -- 172.21.15.120:0/3668503487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559263eeb4a0 msgr2=0x559263ef5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:56.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.845+0000 7fc07da4d740 1 --2- 172.21.15.120:0/3668503487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559263eeb4a0 0x559263ef5ff0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7fc070097820 tx=0x7fc070097de0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.845+0000 7fc07da4d740 1 -- 172.21.15.120:0/3668503487 shutdown_connections 2024-09-19T09:24:56.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.845+0000 7fc07da4d740 1 --2- 172.21.15.120:0/3668503487 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559263f05f10 0x559263f08300 crc :-1 s=READY pgs=549 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.845+0000 7fc07da4d740 1 --2- 172.21.15.120:0/3668503487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559263ef6680 0x559263ef3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.845+0000 7fc07da4d740 1 --2- 172.21.15.120:0/3668503487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559263eebdc0 0x559263ef32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.845+0000 7fc07da4d740 1 --2- 172.21.15.120:0/3668503487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559263eeb4a0 0x559263ef5ff0 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.845+0000 7fc07da4d740 1 -- 172.21.15.120:0/3668503487 >> 172.21.15.120:0/3668503487 conn(0x559263ee7a30 msgr2=0x559263eed760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:56.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.846+0000 7fc05cff9640 0 client.22704 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:56.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.846+0000 7fc07da4d740 1 -- 172.21.15.120:0/3668503487 shutdown_connections 2024-09-19T09:24:56.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.846+0000 7fc07da4d740 1 -- 172.21.15.120:0/3668503487 wait complete. 2024-09-19T09:24:56.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.846+0000 7fc07da4d740 20 client.22704 trim_cache size 0 max 16384 2024-09-19T09:24:56.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.864+0000 7fc07da4d740 1 Processor -- start 2024-09-19T09:24:56.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.864+0000 7fc07da4d740 1 -- start start 2024-09-19T09:24:56.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.864+0000 7fc07da4d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559263eeb4a0 0x559263f02450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:56.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.864+0000 7fc07da4d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559263eebdc0 0x559263f0d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:56.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.864+0000 7fc07da4d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559263ef6680 0x559263f0dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:56.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.864+0000 7fc07da4d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc0741af780 con 0x559263eeb4a0 2024-09-19T09:24:56.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.864+0000 7fc07da4d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc0741b12c0 con 0x559263eebdc0 2024-09-19T09:24:56.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.864+0000 7fc07da4d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc0741afe50 con 0x559263ef6680 2024-09-19T09:24:56.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.864+0000 7fc079410640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559263eeb4a0 0x559263f02450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:56.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.864+0000 7fc079410640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559263eeb4a0 0x559263f02450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58064/0 (socket says 172.21.15.120:58064) 2024-09-19T09:24:56.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.864+0000 7fc079410640 1 -- 172.21.15.120:0/2006089162 learned_addr learned my addr 172.21.15.120:0/2006089162 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:56.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.864+0000 7fc078c0f640 1 --2- 172.21.15.120:0/2006089162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559263eebdc0 0x559263f0d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:56.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.864+0000 7fc079c11640 1 --2- 172.21.15.120:0/2006089162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559263ef6680 0x559263f0dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:56.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.865+0000 7fc079410640 1 -- 172.21.15.120:0/2006089162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559263ef6680 msgr2=0x559263f0dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:56.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.865+0000 7fc079410640 1 --2- 172.21.15.120:0/2006089162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559263ef6680 0x559263f0dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.865+0000 7fc079410640 1 -- 172.21.15.120:0/2006089162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559263eebdc0 msgr2=0x559263f0d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:56.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.865+0000 7fc079410640 1 --2- 172.21.15.120:0/2006089162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559263eebdc0 0x559263f0d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.865+0000 7fc079410640 1 -- 172.21.15.120:0/2006089162 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559263ef3b70 con 0x559263eeb4a0 2024-09-19T09:24:56.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.865+0000 7fc079410640 1 --2- 172.21.15.120:0/2006089162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559263eeb4a0 0x559263f02450 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7fc070000c00 tx=0x7fc07000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1370536e19569de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:56.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.865+0000 7fc05effd640 1 -- 172.21.15.120:0/2006089162 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc070020070 con 0x559263eeb4a0 2024-09-19T09:24:56.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.865+0000 7fc05effd640 1 -- 172.21.15.120:0/2006089162 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0700860f0 con 0x559263eeb4a0 2024-09-19T09:24:56.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.865+0000 7fc05effd640 1 -- 172.21.15.120:0/2006089162 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc070086350 con 0x559263eeb4a0 2024-09-19T09:24:56.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.866+0000 7fc07da4d740 1 -- 172.21.15.120:0/2006089162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559263eeb4a0 msgr2=0x559263f02450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:56.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.866+0000 7fc07da4d740 1 --2- 172.21.15.120:0/2006089162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559263eeb4a0 0x559263f02450 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7fc070000c00 tx=0x7fc07000bf60 comp rx=0 tx=0).stop 2024-09-19T09:24:56.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.866+0000 7fc07da4d740 1 -- 172.21.15.120:0/2006089162 shutdown_connections 2024-09-19T09:24:56.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.866+0000 7fc07da4d740 1 --2- 172.21.15.120:0/2006089162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559263ef6680 0x559263f0dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.866+0000 7fc07da4d740 1 --2- 172.21.15.120:0/2006089162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559263eebdc0 0x559263f0d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.866+0000 7fc07da4d740 1 --2- 172.21.15.120:0/2006089162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559263eeb4a0 0x559263f02450 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.866+0000 7fc07da4d740 1 -- 172.21.15.120:0/2006089162 >> 172.21.15.120:0/2006089162 conn(0x559263ee7a30 msgr2=0x559263ef2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:56.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.866+0000 7fc07da4d740 1 -- 172.21.15.120:0/2006089162 shutdown_connections 2024-09-19T09:24:56.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.866+0000 7fc07da4d740 1 -- 172.21.15.120:0/2006089162 wait complete. 2024-09-19T09:24:56.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.866+0000 7fc07da4d740 1 -- start start 2024-09-19T09:24:56.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.866+0000 7fc07da4d740 1 Processor -- start 2024-09-19T09:24:56.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.867+0000 7fc07da4d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559263eeb4a0 0x559263f68bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:56.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.867+0000 7fc07da4d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559263eebdc0 0x559263f69110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:56.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.867+0000 7fc07da4d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559263ef6680 0x559263ef9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:56.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.867+0000 7fc07da4d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc0741b0bf0 con 0x559263eebdc0 2024-09-19T09:24:56.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.867+0000 7fc07da4d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc0741b0520 con 0x559263ef6680 2024-09-19T09:24:56.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.867+0000 7fc07da4d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc0741ad740 con 0x559263eeb4a0 2024-09-19T09:24:56.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.867+0000 7fc07da4d740 10 client.0 _ll_register_callbacks cb 0x7fc064005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:24:56.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.867+0000 7fc078c0f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559263eebdc0 0x559263f69110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:56.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.867+0000 7fc079c11640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559263ef6680 0x559263ef9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:56.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.867+0000 7fc078c0f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559263eebdc0 0x559263f69110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58066/0 (socket says 172.21.15.120:58066) 2024-09-19T09:24:56.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.867+0000 7fc078c0f640 1 -- 172.21.15.120:0/2729224901 learned_addr learned my addr 172.21.15.120:0/2729224901 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:24:56.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.867+0000 7fc079410640 1 --2- 172.21.15.120:0/2729224901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559263eeb4a0 0x559263f68bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:56.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.867+0000 7fc078c0f640 1 -- 172.21.15.120:0/2729224901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559263eeb4a0 msgr2=0x559263f68bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:56.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.867+0000 7fc078c0f640 1 --2- 172.21.15.120:0/2729224901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559263eeb4a0 0x559263f68bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.867+0000 7fc078c0f640 1 -- 172.21.15.120:0/2729224901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559263ef6680 msgr2=0x559263ef9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:24:56.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.867+0000 7fc078c0f640 1 --2- 172.21.15.120:0/2729224901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559263ef6680 0x559263ef9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.867+0000 7fc078c0f640 1 -- 172.21.15.120:0/2729224901 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559263efa360 con 0x559263eebdc0 2024-09-19T09:24:56.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.867+0000 7fc079c11640 1 --2- 172.21.15.120:0/2729224901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559263ef6680 0x559263ef9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:24:56.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.868+0000 7fc078c0f640 1 --2- 172.21.15.120:0/2729224901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559263eebdc0 0x559263f69110 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7fc064000c00 tx=0x7fc064004400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b7c0242672576b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:56.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.868+0000 7fc05cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:24:56.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.868+0000 7fc05cff9640 1 -- 172.21.15.120:0/2729224901 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064035040 con 0x559263eebdc0 2024-09-19T09:24:56.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.868+0000 7fc05cff9640 1 -- 172.21.15.120:0/2729224901 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc064005220 con 0x559263eebdc0 2024-09-19T09:24:56.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.868+0000 7fc07da4d740 1 -- 172.21.15.120:0/2729224901 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0700860f0 con 0x559263eebdc0 2024-09-19T09:24:56.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.868+0000 7fc05cff9640 1 -- 172.21.15.120:0/2729224901 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc064056050 con 0x559263eebdc0 2024-09-19T09:24:56.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.868+0000 7fc07da4d740 10 client.22710 fetch_fsmap learned FSMap version 39 2024-09-19T09:24:56.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.868+0000 7fc07da4d740 1 -- 172.21.15.120:0/2729224901 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559263f70410 con 0x559263eebdc0 2024-09-19T09:24:56.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.868+0000 7fc05cff9640 1 -- 172.21.15.120:0/2729224901 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0700860f0 con 0x559263eebdc0 2024-09-19T09:24:56.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.868+0000 7fc05cff9640 1 -- 172.21.15.120:0/2729224901 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fc0640049a0 con 0x559263eebdc0 2024-09-19T09:24:56.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.868+0000 7fc07da4d740 10 client.22710 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:24:56.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.868+0000 7fc07da4d740 10 client.22710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:24:56.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.868+0000 7fc07da4d740 20 client.22710 populate_metadata read hostname 'smithi120' 2024-09-19T09:24:56.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.868+0000 7fc07da4d740 1 --2- 172.21.15.120:0/2729224901 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559263f70bc0 0x559263f72fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:56.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.868+0000 7fc07da4d740 4 client.22710 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:24:56.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.868+0000 7fc07da4d740 1 -- 172.21.15.120:0/2729224901 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559263e58d60 con 0x559263f70bc0 2024-09-19T09:24:56.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.869+0000 7fc079410640 1 --2- 172.21.15.120:0/2729224901 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559263f70bc0 0x559263f72fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:56.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.869+0000 7fc079410640 1 --2- 172.21.15.120:0/2729224901 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559263f70bc0 0x559263f72fe0 crc :-1 s=READY pgs=550 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=879fdab38121690c server_cookie=cbb417740a1284f8 in_seq=0 out_seq=0 2024-09-19T09:24:56.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.869+0000 7fc05cff9640 10 client.22710 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:56.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.872+0000 7fc05cff9640 1 -- 172.21.15.120:0/2729224901 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559263e58d60 con 0x559263f70bc0 2024-09-19T09:24:56.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.872+0000 7fc05cff9640 10 client.22710 handle_command_reply: tid=0 2024-09-19T09:24:56.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.872+0000 7fc07da4d740 1 client.22710 shutdown 2024-09-19T09:24:56.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.873+0000 7fc07da4d740 1 -- 172.21.15.120:0/2729224901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559263eebdc0 msgr2=0x559263f69110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:56.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.873+0000 7fc07da4d740 1 --2- 172.21.15.120:0/2729224901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559263eebdc0 0x559263f69110 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7fc064000c00 tx=0x7fc064004400 comp rx=0 tx=0).stop 2024-09-19T09:24:56.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.873+0000 7fc07da4d740 1 -- 172.21.15.120:0/2729224901 shutdown_connections 2024-09-19T09:24:56.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.873+0000 7fc07da4d740 1 --2- 172.21.15.120:0/2729224901 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x559263f70bc0 0x559263f72fe0 crc :-1 s=READY pgs=550 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.873+0000 7fc05cff9640 0 client.22710 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:24:56.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.873+0000 7fc07da4d740 1 --2- 172.21.15.120:0/2729224901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559263ef6680 0x559263ef9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.873+0000 7fc07da4d740 1 --2- 172.21.15.120:0/2729224901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559263eebdc0 0x559263f69110 unknown :-1 s=CLOSED pgs=2307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.873+0000 7fc07da4d740 1 --2- 172.21.15.120:0/2729224901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559263eeb4a0 0x559263f68bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.873+0000 7fc07da4d740 1 -- 172.21.15.120:0/2729224901 >> 172.21.15.120:0/2729224901 conn(0x559263ee7a30 msgr2=0x559263ee6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:56.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.873+0000 7fc07da4d740 1 -- 172.21.15.120:0/2729224901 shutdown_connections 2024-09-19T09:24:56.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.873+0000 7fc07da4d740 1 -- 172.21.15.120:0/2729224901 wait complete. 2024-09-19T09:24:56.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.873+0000 7fc07da4d740 20 client.22710 trim_cache size 0 max 16384 2024-09-19T09:24:56.878 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:24:56.878 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:24:56.878 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-19T09:24:56.879 INFO:teuthology.orchestra.run.smithi120.stdout: "8d6dc70c-4714-45fc-87d7-1dfecf9b783d": { 2024-09-19T09:24:56.879 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-19T09:24:56.879 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "8d6dc70c-4714-45fc-87d7-1dfecf9b783d", 2024-09-19T09:24:56.879 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-19T09:24:56.879 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:24:56.879 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:24:56.879 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:24:56.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.874+0000 7fc07b674640 1 -- 172.21.15.120:0/1812591664 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc0600a3240 msgr2=0x7fc0600a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:56.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.874+0000 7fc07b674640 1 --2- 172.21.15.120:0/1812591664 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc0600a3240 0x7fc0600a5700 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009990 tx=0x7fc06c012ab0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.874+0000 7fc07b674640 1 -- 172.21.15.120:0/1812591664 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc074100e20 msgr2=0x7fc0740fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:56.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.874+0000 7fc07b674640 1 --2- 172.21.15.120:0/1812591664 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc074100e20 0x7fc0740fcc80 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7fc07000d950 tx=0x7fc07000de10 comp rx=0 tx=0).stop 2024-09-19T09:24:56.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.874+0000 7fc07b674640 1 -- 172.21.15.120:0/1812591664 shutdown_connections 2024-09-19T09:24:56.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.874+0000 7fc07b674640 1 --2- 172.21.15.120:0/1812591664 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc0600a3240 0x7fc0600a5700 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.874+0000 7fc07b674640 1 --2- 172.21.15.120:0/1812591664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc074102ef0 0x7fc0740fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.874+0000 7fc07b674640 1 --2- 172.21.15.120:0/1812591664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc074102570 0x7fc0740fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.874+0000 7fc07b674640 1 --2- 172.21.15.120:0/1812591664 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc074100e20 0x7fc0740fcc80 unknown :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.874+0000 7fc07b674640 1 -- 172.21.15.120:0/1812591664 >> 172.21.15.120:0/1812591664 conn(0x7fc0740fc7d0 msgr2=0x7fc07406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:56.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.874+0000 7fc07b674640 1 -- 172.21.15.120:0/1812591664 shutdown_connections 2024-09-19T09:24:56.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:24:56.874+0000 7fc07b674640 1 -- 172.21.15.120:0/1812591664 wait complete. 2024-09-19T09:24:57.382 INFO:tasks.workunit.client.0.smithi120.stdout: 10 201457 19.11 MB/sec execute 412 sec latency 61.147 ms 2024-09-19T09:24:57.671 INFO:tasks.cephfs.filesystem:scrub status for tag:8d6dc70c-4714-45fc-87d7-1dfecf9b783d - {'path': '/', 'tag': '8d6dc70c-4714-45fc-87d7-1dfecf9b783d', 'options': 'recursive,force'} 2024-09-19T09:24:57.686 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:57 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:57.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:57 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:57.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:57 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:24:58.382 INFO:tasks.workunit.client.0.smithi120.stdout: 10 201849 19.10 MB/sec execute 413 sec latency 124.950 ms 2024-09-19T09:24:58.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:24:58 smithi196 ceph-mon[35150]: pgmap v1008: 97 pgs: 97 active+clean; 427 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 417 KiB/s wr, 64 op/s 2024-09-19T09:24:58.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:24:58 smithi153 ceph-mon[35071]: pgmap v1008: 97 pgs: 97 active+clean; 427 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 417 KiB/s wr, 64 op/s 2024-09-19T09:24:58.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:24:58 smithi120 ceph-mon[31150]: pgmap v1008: 97 pgs: 97 active+clean; 427 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 417 KiB/s wr, 64 op/s 2024-09-19T09:24:59.382 INFO:tasks.workunit.client.0.smithi120.stdout: 10 202457 19.14 MB/sec execute 414 sec latency 53.041 ms 2024-09-19T09:25:00.382 INFO:tasks.workunit.client.0.smithi120.stdout: 10 203121 19.14 MB/sec execute 415 sec latency 106.900 ms 2024-09-19T09:25:00.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:00 smithi196 ceph-mon[35150]: pgmap v1009: 97 pgs: 97 active+clean; 433 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-09-19T09:25:00.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:00 smithi153 ceph-mon[35071]: pgmap v1009: 97 pgs: 97 active+clean; 433 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-09-19T09:25:00.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:00 smithi120 ceph-mon[31150]: pgmap v1009: 97 pgs: 97 active+clean; 433 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-09-19T09:25:01.382 INFO:tasks.workunit.client.0.smithi120.stdout: 10 203819 19.19 MB/sec execute 416 sec latency 137.358 ms 2024-09-19T09:25:02.383 INFO:tasks.workunit.client.0.smithi120.stdout: 10 204563 19.25 MB/sec execute 417 sec latency 86.378 ms 2024-09-19T09:25:02.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:02 smithi196 ceph-mon[35150]: pgmap v1010: 97 pgs: 97 active+clean; 441 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.6 MiB/s wr, 277 op/s 2024-09-19T09:25:02.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:02 smithi153 ceph-mon[35071]: pgmap v1010: 97 pgs: 97 active+clean; 441 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.6 MiB/s wr, 277 op/s 2024-09-19T09:25:02.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:02 smithi120 ceph-mon[31150]: pgmap v1010: 97 pgs: 97 active+clean; 441 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.6 MiB/s wr, 277 op/s 2024-09-19T09:25:03.382 INFO:tasks.workunit.client.0.smithi120.stdout: 10 205216 19.33 MB/sec execute 418 sec latency 49.377 ms 2024-09-19T09:25:03.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:03 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:25:03.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:03 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:25:03.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:03 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:25:03.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:03 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:25:03.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:03 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:25:03.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:03 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:25:04.382 INFO:tasks.workunit.client.0.smithi120.stdout: 10 205693 19.33 MB/sec execute 419 sec latency 54.049 ms 2024-09-19T09:25:04.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:04 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:25:04.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:04 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:25:04.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:04 smithi196 ceph-mon[35150]: pgmap v1011: 97 pgs: 97 active+clean; 450 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.9 MiB/s wr, 415 op/s 2024-09-19T09:25:04.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:04 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:25:04.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:04 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:25:04.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:04 smithi153 ceph-mon[35071]: pgmap v1011: 97 pgs: 97 active+clean; 450 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.9 MiB/s wr, 415 op/s 2024-09-19T09:25:04.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:04 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:25:04.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:04 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:25:04.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:04 smithi120 ceph-mon[31150]: pgmap v1011: 97 pgs: 97 active+clean; 450 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.9 MiB/s wr, 415 op/s 2024-09-19T09:25:05.383 INFO:tasks.workunit.client.0.smithi120.stdout: 10 206078 19.33 MB/sec execute 420 sec latency 88.056 ms 2024-09-19T09:25:06.383 INFO:tasks.workunit.client.0.smithi120.stdout: 10 206535 19.34 MB/sec execute 421 sec latency 86.160 ms 2024-09-19T09:25:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.423+0000 7f4b826e6640 1 -- 172.21.15.120:0/3820548651 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b7c100a50 msgr2=0x7f4b7c100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.423+0000 7f4b826e6640 1 --2- 172.21.15.120:0/3820548651 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b7c100a50 0x7f4b7c100e50 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c009900 tx=0x7f4b6c02a410 comp rx=0 tx=0).stop 2024-09-19T09:25:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.423+0000 7f4b826e6640 1 -- 172.21.15.120:0/3820548651 shutdown_connections 2024-09-19T09:25:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.423+0000 7f4b826e6640 1 --2- 172.21.15.120:0/3820548651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b7c102600 0x7f4b7c108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.423+0000 7f4b826e6640 1 --2- 172.21.15.120:0/3820548651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b7c101c40 0x7f4b7c1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.423+0000 7f4b826e6640 1 --2- 172.21.15.120:0/3820548651 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b7c100a50 0x7f4b7c100e50 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.423+0000 7f4b826e6640 1 -- 172.21.15.120:0/3820548651 >> 172.21.15.120:0/3820548651 conn(0x7f4b7c0fc1b0 msgr2=0x7f4b7c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.423+0000 7f4b826e6640 1 -- 172.21.15.120:0/3820548651 shutdown_connections 2024-09-19T09:25:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.423+0000 7f4b826e6640 1 -- 172.21.15.120:0/3820548651 wait complete. 2024-09-19T09:25:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.424+0000 7f4b826e6640 1 Processor -- start 2024-09-19T09:25:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.424+0000 7f4b826e6640 1 -- start start 2024-09-19T09:25:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.424+0000 7f4b826e6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b7c100a50 0x7f4b7c06ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.424+0000 7f4b826e6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b7c101c40 0x7f4b7c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.424+0000 7f4b826e6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b7c102600 0x7f4b7c1a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.424+0000 7f4b826e6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c10c720 con 0x7f4b7c102600 2024-09-19T09:25:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.424+0000 7f4b826e6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c10c5a0 con 0x7f4b7c100a50 2024-09-19T09:25:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.424+0000 7f4b826e6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c10c8a0 con 0x7f4b7c101c40 2024-09-19T09:25:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.425+0000 7f4b80c83640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b7c102600 0x7f4b7c1a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.425+0000 7f4b7bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b7c100a50 0x7f4b7c06ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.425+0000 7f4b7b7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b7c101c40 0x7f4b7c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.425+0000 7f4b7bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b7c100a50 0x7f4b7c06ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:56386/0 (socket says 172.21.15.120:56386) 2024-09-19T09:25:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.425+0000 7f4b7bfff640 1 -- 172.21.15.120:0/2434899838 learned_addr learned my addr 172.21.15.120:0/2434899838 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.425+0000 7f4b7b7fe640 1 -- 172.21.15.120:0/2434899838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b7c100a50 msgr2=0x7f4b7c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.425+0000 7f4b7b7fe640 1 --2- 172.21.15.120:0/2434899838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b7c100a50 0x7f4b7c06ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.425+0000 7f4b7b7fe640 1 -- 172.21.15.120:0/2434899838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b7c102600 msgr2=0x7f4b7c1a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.425+0000 7f4b7b7fe640 1 --2- 172.21.15.120:0/2434899838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b7c102600 0x7f4b7c1a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.425+0000 7f4b7b7fe640 1 -- 172.21.15.120:0/2434899838 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b7c1a1c70 con 0x7f4b7c101c40 2024-09-19T09:25:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.425+0000 7f4b7b7fe640 1 --2- 172.21.15.120:0/2434899838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b7c101c40 0x7f4b7c19ef30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000e9f0 tx=0x7f4b7000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.425+0000 7f4b797fa640 1 -- 172.21.15.120:0/2434899838 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7000ccc0 con 0x7f4b7c101c40 2024-09-19T09:25:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.425+0000 7f4b797fa640 1 -- 172.21.15.120:0/2434899838 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b70005020 con 0x7f4b7c101c40 2024-09-19T09:25:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.425+0000 7f4b826e6640 1 -- 172.21.15.120:0/2434899838 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b7c1a1f00 con 0x7f4b7c101c40 2024-09-19T09:25:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.425+0000 7f4b797fa640 1 -- 172.21.15.120:0/2434899838 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b700049e0 con 0x7f4b7c101c40 2024-09-19T09:25:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.425+0000 7f4b826e6640 1 -- 172.21.15.120:0/2434899838 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b7c1a2370 con 0x7f4b7c101c40 2024-09-19T09:25:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.428+0000 7f4b797fa640 1 -- 172.21.15.120:0/2434899838 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f4b70005560 con 0x7f4b7c101c40 2024-09-19T09:25:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.428+0000 7f4b8547f740 1 Processor -- start 2024-09-19T09:25:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.428+0000 7f4b8547f740 1 -- start start 2024-09-19T09:25:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.428+0000 7f4b8547f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2774019a0 0x55c277401da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.428+0000 7f4b8547f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2774022e0 0x55c277406da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.428+0000 7f4b8547f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c277407370 0x55c277409830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.428+0000 7f4b8547f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c2773652f0 con 0x55c2774022e0 2024-09-19T09:25:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.428+0000 7f4b8547f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c27730d270 con 0x55c2774019a0 2024-09-19T09:25:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.428+0000 7f4b8547f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c2773612b0 con 0x55c277407370 2024-09-19T09:25:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.428+0000 7f4b797fa640 1 --2- 172.21.15.120:0/2434899838 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4b640a3320 0x7f4b640a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.429+0000 7f4b7bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2774022e0 0x55c277406da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.429+0000 7f4b7bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2774022e0 0x55c277406da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46462/0 (socket says 172.21.15.120:46462) 2024-09-19T09:25:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.429+0000 7f4b7bfff640 1 -- 172.21.15.120:0/2410223366 learned_addr learned my addr 172.21.15.120:0/2410223366 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.429+0000 7f4b80c83640 1 --2- 172.21.15.120:0/2410223366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2774019a0 0x55c277401da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.429+0000 7f4b7b7fe640 1 --2- 172.21.15.120:0/2410223366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c277407370 0x55c277409830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.429+0000 7f4b80c83640 1 --2- 172.21.15.120:0/2434899838 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4b640a3320 0x7f4b640a57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.429+0000 7f4b797fa640 1 -- 172.21.15.120:0/2434899838 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f4b700cf810 con 0x7f4b7c101c40 2024-09-19T09:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.429+0000 7f4b7bfff640 1 -- 172.21.15.120:0/2410223366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c277407370 msgr2=0x55c277409830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.429+0000 7f4b7bfff640 1 --2- 172.21.15.120:0/2410223366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c277407370 0x55c277409830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.429+0000 7f4b7bfff640 1 -- 172.21.15.120:0/2410223366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2774019a0 msgr2=0x55c277401da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.429+0000 7f4b7bfff640 1 --2- 172.21.15.120:0/2410223366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2774019a0 0x55c277401da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.429+0000 7f4b7bfff640 1 -- 172.21.15.120:0/2410223366 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c27730c340 con 0x55c2774022e0 2024-09-19T09:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.429+0000 7f4b80c83640 1 --2- 172.21.15.120:0/2434899838 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4b640a3320 0x7f4b640a57e0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f4b74005fd0 tx=0x7f4b74009b80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.429+0000 7f4b7bfff640 1 --2- 172.21.15.120:0/2410223366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2774022e0 0x55c277406da0 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c02a930 tx=0x7f4b6c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2810f463c1f63ba8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.429+0000 7f4b627fc640 1 -- 172.21.15.120:0/2410223366 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c040070 con 0x55c2774022e0 2024-09-19T09:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.429+0000 7f4b627fc640 1 -- 172.21.15.120:0/2410223366 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b6c005500 con 0x55c2774022e0 2024-09-19T09:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.429+0000 7f4b627fc640 1 -- 172.21.15.120:0/2410223366 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c035040 con 0x55c2774022e0 2024-09-19T09:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.430+0000 7f4b8547f740 1 -- 172.21.15.120:0/2410223366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2774022e0 msgr2=0x55c277406da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.430+0000 7f4b8547f740 1 --2- 172.21.15.120:0/2410223366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2774022e0 0x55c277406da0 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c02a930 tx=0x7f4b6c004040 comp rx=0 tx=0).stop 2024-09-19T09:25:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.430+0000 7f4b8547f740 1 -- 172.21.15.120:0/2410223366 shutdown_connections 2024-09-19T09:25:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.430+0000 7f4b8547f740 1 --2- 172.21.15.120:0/2410223366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c277407370 0x55c277409830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.430+0000 7f4b8547f740 1 --2- 172.21.15.120:0/2410223366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2774022e0 0x55c277406da0 unknown :-1 s=CLOSED pgs=2309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.430+0000 7f4b8547f740 1 --2- 172.21.15.120:0/2410223366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2774019a0 0x55c277401da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.430+0000 7f4b8547f740 1 -- 172.21.15.120:0/2410223366 >> 172.21.15.120:0/2410223366 conn(0x55c2773fd010 msgr2=0x55c2773fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.430+0000 7f4b8547f740 1 -- 172.21.15.120:0/2410223366 shutdown_connections 2024-09-19T09:25:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.430+0000 7f4b8547f740 1 -- 172.21.15.120:0/2410223366 wait complete. 2024-09-19T09:25:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.431+0000 7f4b8547f740 1 -- start start 2024-09-19T09:25:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.431+0000 7f4b8547f740 1 Processor -- start 2024-09-19T09:25:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.431+0000 7f4b8547f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2774019a0 0x55c27740bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.431+0000 7f4b8547f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2774022e0 0x55c27740c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.431+0000 7f4b8547f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c277407370 0x55c277413110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.431+0000 7f4b8547f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c27740a2e0 con 0x55c277407370 2024-09-19T09:25:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.431+0000 7f4b8547f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c2772ee290 con 0x55c2774019a0 2024-09-19T09:25:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.431+0000 7f4b8547f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c27740a460 con 0x55c2774022e0 2024-09-19T09:25:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.432+0000 7f4b80c83640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c277407370 0x55c277413110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.432+0000 7f4b7b7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2774022e0 0x55c27740c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.432+0000 7f4b7b7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2774022e0 0x55c27740c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:48664/0 (socket says 172.21.15.120:48664) 2024-09-19T09:25:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.432+0000 7f4b7b7fe640 1 -- 172.21.15.120:0/1062565387 learned_addr learned my addr 172.21.15.120:0/1062565387 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.432+0000 7f4b7bfff640 1 --2- 172.21.15.120:0/1062565387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2774019a0 0x55c27740bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.432+0000 7f4b8547f740 10 client.0 _ll_register_callbacks cb 0x55c27731f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:25:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.432+0000 7f4b7b7fe640 1 -- 172.21.15.120:0/1062565387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2774019a0 msgr2=0x55c27740bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.432+0000 7f4b7b7fe640 1 --2- 172.21.15.120:0/1062565387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2774019a0 0x55c27740bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.432+0000 7f4b7b7fe640 1 -- 172.21.15.120:0/1062565387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c277407370 msgr2=0x55c277413110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.432+0000 7f4b7b7fe640 1 --2- 172.21.15.120:0/1062565387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c277407370 0x55c277413110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.432+0000 7f4b7b7fe640 1 -- 172.21.15.120:0/1062565387 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c277078210 con 0x55c2774022e0 2024-09-19T09:25:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.432+0000 7f4b80c83640 1 --2- 172.21.15.120:0/1062565387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c277407370 0x55c277413110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:25:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.432+0000 7f4b7b7fe640 1 --2- 172.21.15.120:0/1062565387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2774022e0 0x55c27740c0d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4b700cd310 tx=0x7f4b7009b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=499303ba7f68d57a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.432+0000 7f4b5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:25:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.432+0000 7f4b5b7fe640 1 -- 172.21.15.120:0/1062565387 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7001c070 con 0x55c2774022e0 2024-09-19T09:25:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.432+0000 7f4b5b7fe640 1 -- 172.21.15.120:0/1062565387 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f4b7009f040 con 0x55c2774022e0 2024-09-19T09:25:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.432+0000 7f4b8547f740 1 -- 172.21.15.120:0/1062565387 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b6c005500 con 0x55c2774022e0 2024-09-19T09:25:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.433+0000 7f4b5b7fe640 1 -- 172.21.15.120:0/1062565387 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4b70081050 con 0x55c2774022e0 2024-09-19T09:25:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.433+0000 7f4b8547f740 10 client.31261 fetch_fsmap learned FSMap version 39 2024-09-19T09:25:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.433+0000 7f4b8547f740 1 -- 172.21.15.120:0/1062565387 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c277144620 con 0x55c2774022e0 2024-09-19T09:25:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.433+0000 7f4b5b7fe640 1 -- 172.21.15.120:0/1062565387 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b6c005500 con 0x55c2774022e0 2024-09-19T09:25:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.433+0000 7f4b5b7fe640 1 -- 172.21.15.120:0/1062565387 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f4b700984a0 con 0x55c2774022e0 2024-09-19T09:25:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.433+0000 7f4b8547f740 10 client.31261 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:25:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.433+0000 7f4b8547f740 10 client.31261 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:25:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.433+0000 7f4b8547f740 20 client.31261 populate_metadata read hostname 'smithi120' 2024-09-19T09:25:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.433+0000 7f4b8547f740 1 --2- 172.21.15.120:0/1062565387 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c277417870 0x55c277419c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.433+0000 7f4b8547f740 4 client.31261 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:25:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.433+0000 7f4b8547f740 1 -- 172.21.15.120:0/1062565387 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c277372a20 con 0x55c277417870 2024-09-19T09:25:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.433+0000 7f4b7bfff640 1 --2- 172.21.15.120:0/1062565387 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c277417870 0x55c277419c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.434+0000 7f4b5b7fe640 10 client.31261 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.434+0000 7f4b7bfff640 1 --2- 172.21.15.120:0/1062565387 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c277417870 0x55c277419c60 crc :-1 s=READY pgs=551 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e7d5e3f37788041 server_cookie=acd791b4c5e23a28 in_seq=0 out_seq=0 2024-09-19T09:25:06.442 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:06 smithi153 ceph-mon[35071]: pgmap v1012: 97 pgs: 97 active+clean; 454 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 4.5 MiB/s wr, 458 op/s 2024-09-19T09:25:06.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:06 smithi196 ceph-mon[35150]: pgmap v1012: 97 pgs: 97 active+clean; 454 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 4.5 MiB/s wr, 458 op/s 2024-09-19T09:25:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.438+0000 7f4b5b7fe640 1 -- 172.21.15.120:0/1062565387 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c277372a20 con 0x55c277417870 2024-09-19T09:25:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.438+0000 7f4b5b7fe640 10 client.31261 handle_command_reply: tid=0 2024-09-19T09:25:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.438+0000 7f4b8547f740 1 client.31261 shutdown 2024-09-19T09:25:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.438+0000 7f4b8547f740 1 -- 172.21.15.120:0/1062565387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2774022e0 msgr2=0x55c27740c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.438+0000 7f4b8547f740 1 --2- 172.21.15.120:0/1062565387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2774022e0 0x55c27740c0d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4b700cd310 tx=0x7f4b7009b810 comp rx=0 tx=0).stop 2024-09-19T09:25:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.438+0000 7f4b8547f740 1 -- 172.21.15.120:0/1062565387 shutdown_connections 2024-09-19T09:25:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.438+0000 7f4b8547f740 1 --2- 172.21.15.120:0/1062565387 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c277417870 0x55c277419c60 crc :-1 s=READY pgs=551 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.438+0000 7f4b8547f740 1 --2- 172.21.15.120:0/1062565387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c277407370 0x55c277413110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.438+0000 7f4b8547f740 1 --2- 172.21.15.120:0/1062565387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2774022e0 0x55c27740c0d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.438+0000 7f4b8547f740 1 --2- 172.21.15.120:0/1062565387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2774019a0 0x55c27740bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.438+0000 7f4b8547f740 1 -- 172.21.15.120:0/1062565387 >> 172.21.15.120:0/1062565387 conn(0x55c2773fd010 msgr2=0x55c2773fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.438+0000 7f4b5b7fe640 0 client.31261 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.438+0000 7f4b8547f740 1 -- 172.21.15.120:0/1062565387 shutdown_connections 2024-09-19T09:25:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.438+0000 7f4b8547f740 1 -- 172.21.15.120:0/1062565387 wait complete. 2024-09-19T09:25:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.438+0000 7f4b8547f740 20 client.31261 trim_cache size 0 max 16384 2024-09-19T09:25:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.453+0000 7f4b8547f740 1 Processor -- start 2024-09-19T09:25:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.453+0000 7f4b8547f740 1 -- start start 2024-09-19T09:25:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.453+0000 7f4b8547f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2774019a0 0x55c2774034c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.453+0000 7f4b8547f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2774022e0 0x55c277403a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.453+0000 7f4b8547f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c277407370 0x55c2774130d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.453+0000 7f4b8547f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c1ab0d0 con 0x55c2774019a0 2024-09-19T09:25:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.453+0000 7f4b8547f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c1acc10 con 0x55c2774022e0 2024-09-19T09:25:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.453+0000 7f4b8547f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c1ab7a0 con 0x55c277407370 2024-09-19T09:25:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.454+0000 7f4b80c83640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c277407370 0x55c2774130d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.454+0000 7f4b7bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2774019a0 0x55c2774034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.454+0000 7f4b7b7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2774022e0 0x55c277403a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.454+0000 7f4b80c83640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c277407370 0x55c2774130d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:48670/0 (socket says 172.21.15.120:48670) 2024-09-19T09:25:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.454+0000 7f4b7bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2774019a0 0x55c2774034c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46486/0 (socket says 172.21.15.120:46486) 2024-09-19T09:25:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.454+0000 7f4b80c83640 1 -- 172.21.15.120:0/4028267824 learned_addr learned my addr 172.21.15.120:0/4028267824 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.454+0000 7f4b7b7fe640 1 -- 172.21.15.120:0/4028267824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c277407370 msgr2=0x55c2774130d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.454+0000 7f4b7b7fe640 1 --2- 172.21.15.120:0/4028267824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c277407370 0x55c2774130d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.454+0000 7f4b7b7fe640 1 -- 172.21.15.120:0/4028267824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2774019a0 msgr2=0x55c2774034c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.454+0000 7f4b7b7fe640 1 --2- 172.21.15.120:0/4028267824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2774019a0 0x55c2774034c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.454+0000 7f4b7b7fe640 1 -- 172.21.15.120:0/4028267824 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2774027c0 con 0x55c2774022e0 2024-09-19T09:25:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.454+0000 7f4b7bfff640 1 --2- 172.21.15.120:0/4028267824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2774019a0 0x55c2774034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:25:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.454+0000 7f4b7b7fe640 1 --2- 172.21.15.120:0/4028267824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2774022e0 0x55c277403a00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4b7009a0e0 tx=0x7f4b700096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b09089b580427144 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.454+0000 7f4b61ffb640 1 -- 172.21.15.120:0/4028267824 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7001c070 con 0x55c2774022e0 2024-09-19T09:25:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.454+0000 7f4b61ffb640 1 -- 172.21.15.120:0/4028267824 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b700b9020 con 0x55c2774022e0 2024-09-19T09:25:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.454+0000 7f4b61ffb640 1 -- 172.21.15.120:0/4028267824 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b700a4aa0 con 0x55c2774022e0 2024-09-19T09:25:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.455+0000 7f4b8547f740 1 -- 172.21.15.120:0/4028267824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2774022e0 msgr2=0x55c277403a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.455+0000 7f4b8547f740 1 --2- 172.21.15.120:0/4028267824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2774022e0 0x55c277403a00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4b7009a0e0 tx=0x7f4b700096e0 comp rx=0 tx=0).stop 2024-09-19T09:25:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.455+0000 7f4b8547f740 1 -- 172.21.15.120:0/4028267824 shutdown_connections 2024-09-19T09:25:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.455+0000 7f4b8547f740 1 --2- 172.21.15.120:0/4028267824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c277407370 0x55c2774130d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.455+0000 7f4b8547f740 1 --2- 172.21.15.120:0/4028267824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2774022e0 0x55c277403a00 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.455+0000 7f4b8547f740 1 --2- 172.21.15.120:0/4028267824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2774019a0 0x55c2774034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.455+0000 7f4b8547f740 1 -- 172.21.15.120:0/4028267824 >> 172.21.15.120:0/4028267824 conn(0x55c2773fd010 msgr2=0x55c27740bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.455+0000 7f4b8547f740 1 -- 172.21.15.120:0/4028267824 shutdown_connections 2024-09-19T09:25:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.455+0000 7f4b8547f740 1 -- 172.21.15.120:0/4028267824 wait complete. 2024-09-19T09:25:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.455+0000 7f4b8547f740 1 -- start start 2024-09-19T09:25:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.456+0000 7f4b8547f740 1 Processor -- start 2024-09-19T09:25:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.456+0000 7f4b8547f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2774019a0 0x55c2774100e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.456+0000 7f4b8547f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2774022e0 0x55c277410620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.456+0000 7f4b8547f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c277407370 0x55c277488cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.456+0000 7f4b8547f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c1ac540 con 0x55c2774022e0 2024-09-19T09:25:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.456+0000 7f4b8547f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c1abe70 con 0x55c2774019a0 2024-09-19T09:25:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.456+0000 7f4b8547f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c1a9090 con 0x55c277407370 2024-09-19T09:25:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.456+0000 7f4b8547f740 10 client.0 _ll_register_callbacks cb 0x7f4b70004eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:25:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.456+0000 7f4b7b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2774022e0 0x55c277410620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.456+0000 7f4b7b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2774022e0 0x55c277410620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46500/0 (socket says 172.21.15.120:46500) 2024-09-19T09:25:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.456+0000 7f4b7b7fe640 1 -- 172.21.15.120:0/1663418147 learned_addr learned my addr 172.21.15.120:0/1663418147 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.456+0000 7f4b7bfff640 1 --2- 172.21.15.120:0/1663418147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2774019a0 0x55c2774100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.456+0000 7f4b80c83640 1 --2- 172.21.15.120:0/1663418147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c277407370 0x55c277488cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.456+0000 7f4b7b7fe640 1 -- 172.21.15.120:0/1663418147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c277407370 msgr2=0x55c277488cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.456+0000 7f4b7b7fe640 1 --2- 172.21.15.120:0/1663418147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c277407370 0x55c277488cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.456+0000 7f4b7b7fe640 1 -- 172.21.15.120:0/1663418147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2774019a0 msgr2=0x55c2774100e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.456+0000 7f4b7b7fe640 1 --2- 172.21.15.120:0/1663418147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2774019a0 0x55c2774100e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.456+0000 7f4b7b7fe640 1 -- 172.21.15.120:0/1663418147 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c277489270 con 0x55c2774022e0 2024-09-19T09:25:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.457+0000 7f4b7b7fe640 1 --2- 172.21.15.120:0/1663418147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2774022e0 0x55c277410620 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f4b70098290 tx=0x7f4b70098750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37fef1f4a5830c8f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.457+0000 7f4b5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:25:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.457+0000 7f4b5b7fe640 1 -- 172.21.15.120:0/1663418147 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b700a9060 con 0x55c2774022e0 2024-09-19T09:25:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.457+0000 7f4b5b7fe640 1 -- 172.21.15.120:0/1663418147 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f4b7009b3a0 con 0x55c2774022e0 2024-09-19T09:25:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.457+0000 7f4b8547f740 1 -- 172.21.15.120:0/1663418147 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b700b9020 con 0x55c2774022e0 2024-09-19T09:25:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.457+0000 7f4b5b7fe640 1 -- 172.21.15.120:0/1663418147 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4b7001c070 con 0x55c2774022e0 2024-09-19T09:25:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.457+0000 7f4b8547f740 10 client.22728 fetch_fsmap learned FSMap version 39 2024-09-19T09:25:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.457+0000 7f4b8547f740 1 -- 172.21.15.120:0/1663418147 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c27748a580 con 0x55c2774022e0 2024-09-19T09:25:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.457+0000 7f4b5b7fe640 1 -- 172.21.15.120:0/1663418147 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b700b9020 con 0x55c2774022e0 2024-09-19T09:25:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.457+0000 7f4b5b7fe640 1 -- 172.21.15.120:0/1663418147 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f4b700ae070 con 0x55c2774022e0 2024-09-19T09:25:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.457+0000 7f4b8547f740 10 client.22728 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:25:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.457+0000 7f4b8547f740 10 client.22728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:25:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.457+0000 7f4b8547f740 20 client.22728 populate_metadata read hostname 'smithi120' 2024-09-19T09:25:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.457+0000 7f4b8547f740 1 --2- 172.21.15.120:0/1663418147 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c27748ae70 0x55c27748d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.457+0000 7f4b8547f740 4 client.22728 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:25:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.457+0000 7f4b8547f740 1 -- 172.21.15.120:0/1663418147 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c27736cd70 con 0x55c27748ae70 2024-09-19T09:25:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.457+0000 7f4b7bfff640 1 --2- 172.21.15.120:0/1663418147 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c27748ae70 0x55c27748d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.458+0000 7f4b7bfff640 1 --2- 172.21.15.120:0/1663418147 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c27748ae70 0x55c27748d260 crc :-1 s=READY pgs=552 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bc9f6c581aa1c66 server_cookie=a1090ae82d34916f in_seq=0 out_seq=0 2024-09-19T09:25:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:06.458+0000 7f4b5b7fe640 10 client.22728 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:06.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:06 smithi120 ceph-mon[31150]: pgmap v1012: 97 pgs: 97 active+clean; 454 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 4.5 MiB/s wr, 458 op/s 2024-09-19T09:25:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:07.029+0000 7f4b5b7fe640 1 -- 172.21.15.120:0/1663418147 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c27736cd70 con 0x55c27748ae70 2024-09-19T09:25:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:07.029+0000 7f4b5b7fe640 10 client.22728 handle_command_reply: tid=0 2024-09-19T09:25:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:07.029+0000 7f4b8547f740 1 client.22728 shutdown 2024-09-19T09:25:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:07.029+0000 7f4b8547f740 1 -- 172.21.15.120:0/1663418147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2774022e0 msgr2=0x55c277410620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:07.029+0000 7f4b8547f740 1 --2- 172.21.15.120:0/1663418147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2774022e0 0x55c277410620 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f4b70098290 tx=0x7f4b70098750 comp rx=0 tx=0).stop 2024-09-19T09:25:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:07.029+0000 7f4b8547f740 1 -- 172.21.15.120:0/1663418147 shutdown_connections 2024-09-19T09:25:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:07.029+0000 7f4b8547f740 1 --2- 172.21.15.120:0/1663418147 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c27748ae70 0x55c27748d260 crc :-1 s=READY pgs=552 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:07.030+0000 7f4b8547f740 1 --2- 172.21.15.120:0/1663418147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c277407370 0x55c277488cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:07.030+0000 7f4b5b7fe640 0 client.22728 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:07.030+0000 7f4b8547f740 1 --2- 172.21.15.120:0/1663418147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2774022e0 0x55c277410620 unknown :-1 s=CLOSED pgs=2310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:07.030+0000 7f4b8547f740 1 --2- 172.21.15.120:0/1663418147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2774019a0 0x55c2774100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:07.030+0000 7f4b8547f740 1 -- 172.21.15.120:0/1663418147 >> 172.21.15.120:0/1663418147 conn(0x55c2773fd010 msgr2=0x55c27740bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:07.030+0000 7f4b8547f740 1 -- 172.21.15.120:0/1663418147 shutdown_connections 2024-09-19T09:25:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:07.030+0000 7f4b8547f740 1 -- 172.21.15.120:0/1663418147 wait complete. 2024-09-19T09:25:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:07.030+0000 7f4b8547f740 20 client.22728 trim_cache size 0 max 16384 2024-09-19T09:25:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:25:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:25:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:25:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:25:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:07.030+0000 7f4b826e6640 1 -- 172.21.15.120:0/2434899838 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4b640a3320 msgr2=0x7f4b640a57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:07.030+0000 7f4b826e6640 1 --2- 172.21.15.120:0/2434899838 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4b640a3320 0x7f4b640a57e0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f4b74005fd0 tx=0x7f4b74009b80 comp rx=0 tx=0).stop 2024-09-19T09:25:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:07.030+0000 7f4b826e6640 1 -- 172.21.15.120:0/2434899838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b7c101c40 msgr2=0x7f4b7c19ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:07.030+0000 7f4b826e6640 1 --2- 172.21.15.120:0/2434899838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b7c101c40 0x7f4b7c19ef30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000e9f0 tx=0x7f4b7000eeb0 comp rx=0 tx=0).stop 2024-09-19T09:25:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:07.030+0000 7f4b826e6640 1 -- 172.21.15.120:0/2434899838 shutdown_connections 2024-09-19T09:25:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:07.030+0000 7f4b826e6640 1 --2- 172.21.15.120:0/2434899838 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f4b640a3320 0x7f4b640a57e0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:07.030+0000 7f4b826e6640 1 --2- 172.21.15.120:0/2434899838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b7c102600 0x7f4b7c1a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:07.030+0000 7f4b826e6640 1 --2- 172.21.15.120:0/2434899838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b7c101c40 0x7f4b7c19ef30 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:07.030+0000 7f4b826e6640 1 --2- 172.21.15.120:0/2434899838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b7c100a50 0x7f4b7c06ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:07.030+0000 7f4b826e6640 1 -- 172.21.15.120:0/2434899838 >> 172.21.15.120:0/2434899838 conn(0x7f4b7c0fc1b0 msgr2=0x7f4b7c104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:07.030+0000 7f4b826e6640 1 -- 172.21.15.120:0/2434899838 shutdown_connections 2024-09-19T09:25:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:07.030+0000 7f4b826e6640 1 -- 172.21.15.120:0/2434899838 wait complete. 2024-09-19T09:25:07.383 INFO:tasks.workunit.client.0.smithi120.stdout: 10 206855 19.33 MB/sec execute 422 sec latency 377.877 ms 2024-09-19T09:25:08.383 INFO:tasks.workunit.client.0.smithi120.stdout: 10 207503 19.36 MB/sec execute 423 sec latency 100.192 ms 2024-09-19T09:25:08.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:08 smithi196 ceph-mon[35150]: pgmap v1013: 97 pgs: 97 active+clean; 452 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.6 MiB/s wr, 508 op/s 2024-09-19T09:25:08.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:08 smithi153 ceph-mon[35071]: pgmap v1013: 97 pgs: 97 active+clean; 452 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.6 MiB/s wr, 508 op/s 2024-09-19T09:25:08.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:08 smithi120 ceph-mon[31150]: pgmap v1013: 97 pgs: 97 active+clean; 452 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.6 MiB/s wr, 508 op/s 2024-09-19T09:25:09.383 INFO:tasks.workunit.client.0.smithi120.stdout: 10 208240 19.39 MB/sec execute 424 sec latency 50.523 ms 2024-09-19T09:25:10.383 INFO:tasks.workunit.client.0.smithi120.stdout: 10 208933 19.44 MB/sec execute 425 sec latency 165.427 ms 2024-09-19T09:25:10.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:10 smithi196 ceph-mon[35150]: pgmap v1014: 97 pgs: 97 active+clean; 451 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 5.1 MiB/s wr, 643 op/s 2024-09-19T09:25:10.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:10 smithi153 ceph-mon[35071]: pgmap v1014: 97 pgs: 97 active+clean; 451 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 5.1 MiB/s wr, 643 op/s 2024-09-19T09:25:10.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:10 smithi120 ceph-mon[31150]: pgmap v1014: 97 pgs: 97 active+clean; 451 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 5.1 MiB/s wr, 643 op/s 2024-09-19T09:25:11.383 INFO:tasks.workunit.client.0.smithi120.stdout: 10 209543 19.50 MB/sec execute 426 sec latency 50.677 ms 2024-09-19T09:25:12.383 INFO:tasks.workunit.client.0.smithi120.stdout: 10 210137 19.53 MB/sec execute 427 sec latency 62.954 ms 2024-09-19T09:25:13.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:12 smithi120 ceph-mon[31150]: pgmap v1015: 97 pgs: 97 active+clean; 450 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.8 MiB/s wr, 653 op/s 2024-09-19T09:25:13.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:12 smithi196 ceph-mon[35150]: pgmap v1015: 97 pgs: 97 active+clean; 450 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.8 MiB/s wr, 653 op/s 2024-09-19T09:25:13.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:12 smithi153 ceph-mon[35071]: pgmap v1015: 97 pgs: 97 active+clean; 450 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.8 MiB/s wr, 653 op/s 2024-09-19T09:25:13.383 INFO:tasks.workunit.client.0.smithi120.stdout: 10 210568 19.54 MB/sec execute 428 sec latency 70.815 ms 2024-09-19T09:25:14.383 INFO:tasks.workunit.client.0.smithi120.stdout: 10 211087 19.55 MB/sec execute 429 sec latency 46.416 ms 2024-09-19T09:25:14.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:14 smithi196 ceph-mon[35150]: pgmap v1016: 97 pgs: 97 active+clean; 453 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 4.0 MiB/s wr, 604 op/s 2024-09-19T09:25:14.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:14 smithi153 ceph-mon[35071]: pgmap v1016: 97 pgs: 97 active+clean; 453 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 4.0 MiB/s wr, 604 op/s 2024-09-19T09:25:14.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:14 smithi120 ceph-mon[31150]: pgmap v1016: 97 pgs: 97 active+clean; 453 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 4.0 MiB/s wr, 604 op/s 2024-09-19T09:25:15.383 INFO:tasks.workunit.client.0.smithi120.stdout: 10 211680 19.57 MB/sec execute 430 sec latency 70.580 ms 2024-09-19T09:25:16.383 INFO:tasks.workunit.client.0.smithi120.stdout: 10 212415 19.61 MB/sec execute 431 sec latency 60.287 ms 2024-09-19T09:25:16.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:16 smithi196 ceph-mon[35150]: pgmap v1017: 97 pgs: 97 active+clean; 456 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.0 MiB/s wr, 497 op/s 2024-09-19T09:25:16.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:16 smithi153 ceph-mon[35071]: pgmap v1017: 97 pgs: 97 active+clean; 456 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.0 MiB/s wr, 497 op/s 2024-09-19T09:25:16.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:16 smithi120 ceph-mon[31150]: pgmap v1017: 97 pgs: 97 active+clean; 456 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.0 MiB/s wr, 497 op/s 2024-09-19T09:25:17.383 INFO:tasks.workunit.client.0.smithi120.stdout: 10 213074 19.65 MB/sec execute 432 sec latency 129.286 ms 2024-09-19T09:25:18.383 INFO:tasks.workunit.client.0.smithi120.stdout: 10 213770 19.71 MB/sec execute 433 sec latency 71.062 ms 2024-09-19T09:25:18.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:18 smithi196 ceph-mon[35150]: pgmap v1018: 97 pgs: 97 active+clean; 465 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.3 MiB/s wr, 553 op/s 2024-09-19T09:25:18.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:18 smithi153 ceph-mon[35071]: pgmap v1018: 97 pgs: 97 active+clean; 465 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.3 MiB/s wr, 553 op/s 2024-09-19T09:25:18.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:18 smithi120 ceph-mon[31150]: pgmap v1018: 97 pgs: 97 active+clean; 465 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.3 MiB/s wr, 553 op/s 2024-09-19T09:25:19.383 INFO:tasks.workunit.client.0.smithi120.stdout: 10 214329 19.74 MB/sec execute 434 sec latency 54.810 ms 2024-09-19T09:25:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.125+0000 7fc01a5e4640 1 -- 172.21.15.120:0/3006193996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0141025f0 msgr2=0x7fc014108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.125+0000 7fc01a5e4640 1 --2- 172.21.15.120:0/3006193996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0141025f0 0x7fc014108e80 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009ca0 tx=0x7fc00c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.125+0000 7fc01a5e4640 1 -- 172.21.15.120:0/3006193996 shutdown_connections 2024-09-19T09:25:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.125+0000 7fc01a5e4640 1 --2- 172.21.15.120:0/3006193996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0141025f0 0x7fc014108e80 unknown :-1 s=CLOSED pgs=2311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.125+0000 7fc01a5e4640 1 --2- 172.21.15.120:0/3006193996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc014101c30 0x7fc0141020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.125+0000 7fc01a5e4640 1 --2- 172.21.15.120:0/3006193996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc014100a40 0x7fc014100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.125+0000 7fc01a5e4640 1 -- 172.21.15.120:0/3006193996 >> 172.21.15.120:0/3006193996 conn(0x7fc0140fc1e0 msgr2=0x7fc0140fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.125+0000 7fc01a5e4640 1 -- 172.21.15.120:0/3006193996 shutdown_connections 2024-09-19T09:25:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.126+0000 7fc01a5e4640 1 -- 172.21.15.120:0/3006193996 wait complete. 2024-09-19T09:25:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.126+0000 7fc01a5e4640 1 Processor -- start 2024-09-19T09:25:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.126+0000 7fc01a5e4640 1 -- start start 2024-09-19T09:25:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.127+0000 7fc01a5e4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc014100a40 0x7fc014197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.127+0000 7fc01a5e4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc014101c30 0x7fc014198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.127+0000 7fc01a5e4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0141025f0 0x7fc01419f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.127+0000 7fc01a5e4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc01410d810 con 0x7fc0141025f0 2024-09-19T09:25:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.127+0000 7fc01a5e4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc01410d690 con 0x7fc014100a40 2024-09-19T09:25:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.127+0000 7fc01a5e4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc01410d990 con 0x7fc014101c30 2024-09-19T09:25:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.127+0000 7fc013fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc014100a40 0x7fc014197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.127+0000 7fc013fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc014100a40 0x7fc014197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:46438/0 (socket says 172.21.15.120:46438) 2024-09-19T09:25:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.127+0000 7fc013fff640 1 -- 172.21.15.120:0/4092960254 learned_addr learned my addr 172.21.15.120:0/4092960254 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.127+0000 7fc018b81640 1 --2- 172.21.15.120:0/4092960254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0141025f0 0x7fc01419f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.127+0000 7fc0137fe640 1 --2- 172.21.15.120:0/4092960254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc014101c30 0x7fc014198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.127+0000 7fc0137fe640 1 -- 172.21.15.120:0/4092960254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc014100a40 msgr2=0x7fc014197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.127+0000 7fc0137fe640 1 --2- 172.21.15.120:0/4092960254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc014100a40 0x7fc014197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.127+0000 7fc0137fe640 1 -- 172.21.15.120:0/4092960254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0141025f0 msgr2=0x7fc01419f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.127+0000 7fc0137fe640 1 --2- 172.21.15.120:0/4092960254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0141025f0 0x7fc01419f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.127+0000 7fc0137fe640 1 -- 172.21.15.120:0/4092960254 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01419f970 con 0x7fc014101c30 2024-09-19T09:25:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.127+0000 7fc013fff640 1 --2- 172.21.15.120:0/4092960254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc014100a40 0x7fc014197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:25:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.128+0000 7fc0137fe640 1 --2- 172.21.15.120:0/4092960254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc014101c30 0x7fc014198290 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc00800e9f0 tx=0x7fc00800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.128+0000 7fc0117fa640 1 -- 172.21.15.120:0/4092960254 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00800ccc0 con 0x7fc014101c30 2024-09-19T09:25:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.128+0000 7fc0117fa640 1 -- 172.21.15.120:0/4092960254 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc008010910 con 0x7fc014101c30 2024-09-19T09:25:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.128+0000 7fc0117fa640 1 -- 172.21.15.120:0/4092960254 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc008018960 con 0x7fc014101c30 2024-09-19T09:25:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.128+0000 7fc01a5e4640 1 -- 172.21.15.120:0/4092960254 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc01419fc60 con 0x7fc014101c30 2024-09-19T09:25:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.128+0000 7fc01a5e4640 1 -- 172.21.15.120:0/4092960254 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0141a0160 con 0x7fc014101c30 2024-09-19T09:25:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.131+0000 7fc0117fa640 1 -- 172.21.15.120:0/4092960254 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fc008010ac0 con 0x7fc014101c30 2024-09-19T09:25:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.131+0000 7fc01d459740 1 Processor -- start 2024-09-19T09:25:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.131+0000 7fc01d459740 1 -- start start 2024-09-19T09:25:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.131+0000 7fc01d459740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576e10789a0 0x5576e1078da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.131+0000 7fc01d459740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e10792e0 0x5576e107dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.131+0000 7fc01d459740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576e107e370 0x5576e1080830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.131+0000 7fc01d459740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5576e0fdc2f0 con 0x5576e10792e0 2024-09-19T09:25:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.131+0000 7fc01d459740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5576e0f84270 con 0x5576e107e370 2024-09-19T09:25:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.131+0000 7fc01d459740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5576e0fd82b0 con 0x5576e10789a0 2024-09-19T09:25:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.131+0000 7fc018b81640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576e10789a0 0x5576e1078da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.131+0000 7fc013fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e10792e0 0x5576e107dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.131+0000 7fc018b81640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576e10789a0 0x5576e1078da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:45104/0 (socket says 172.21.15.120:45104) 2024-09-19T09:25:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.131+0000 7fc018b81640 1 -- 172.21.15.120:0/741690206 learned_addr learned my addr 172.21.15.120:0/741690206 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.131+0000 7fc013fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e10792e0 0x5576e107dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48200/0 (socket says 172.21.15.120:48200) 2024-09-19T09:25:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.131+0000 7fc0137fe640 1 --2- 172.21.15.120:0/741690206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576e107e370 0x5576e1080830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.131+0000 7fc018b81640 1 -- 172.21.15.120:0/741690206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576e107e370 msgr2=0x5576e1080830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.131+0000 7fc018b81640 1 --2- 172.21.15.120:0/741690206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576e107e370 0x5576e1080830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.131+0000 7fc018b81640 1 -- 172.21.15.120:0/741690206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e10792e0 msgr2=0x5576e107dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.131+0000 7fc018b81640 1 --2- 172.21.15.120:0/741690206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e10792e0 0x5576e107dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.131+0000 7fc018b81640 1 -- 172.21.15.120:0/741690206 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576e0f83340 con 0x5576e10789a0 2024-09-19T09:25:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.131+0000 7fc0137fe640 1 --2- 172.21.15.120:0/741690206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576e107e370 0x5576e1080830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:25:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.131+0000 7fc0117fa640 1 --2- 172.21.15.120:0/4092960254 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbffc0a3370 0x7fbffc0a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.132+0000 7fc013fff640 1 --2- 172.21.15.120:0/4092960254 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbffc0a3370 0x7fbffc0a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.132+0000 7fc018b81640 1 --2- 172.21.15.120:0/741690206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576e10789a0 0x5576e1078da0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009ca0 tx=0x7fc00c005fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c60038f5d939730d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.132+0000 7fc0117fa640 1 -- 172.21.15.120:0/4092960254 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc0080cfbe0 con 0x7fc014101c30 2024-09-19T09:25:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.132+0000 7fbffa7fc640 1 -- 172.21.15.120:0/741690206 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c031650 con 0x5576e10789a0 2024-09-19T09:25:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.132+0000 7fbffa7fc640 1 -- 172.21.15.120:0/741690206 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc00c036030 con 0x5576e10789a0 2024-09-19T09:25:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.132+0000 7fbffa7fc640 1 -- 172.21.15.120:0/741690206 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c004700 con 0x5576e10789a0 2024-09-19T09:25:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.133+0000 7fc01d459740 1 -- 172.21.15.120:0/741690206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576e10789a0 msgr2=0x5576e1078da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.133+0000 7fc013fff640 1 --2- 172.21.15.120:0/4092960254 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbffc0a3370 0x7fbffc0a5830 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc00400bfd0 tx=0x7fc00400b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.133+0000 7fc01d459740 1 --2- 172.21.15.120:0/741690206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576e10789a0 0x5576e1078da0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009ca0 tx=0x7fc00c005fa0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.133+0000 7fc01d459740 1 -- 172.21.15.120:0/741690206 shutdown_connections 2024-09-19T09:25:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.133+0000 7fc01d459740 1 --2- 172.21.15.120:0/741690206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576e107e370 0x5576e1080830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.133+0000 7fc01d459740 1 --2- 172.21.15.120:0/741690206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e10792e0 0x5576e107dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.133+0000 7fc01d459740 1 --2- 172.21.15.120:0/741690206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576e10789a0 0x5576e1078da0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.133+0000 7fc01d459740 1 -- 172.21.15.120:0/741690206 >> 172.21.15.120:0/741690206 conn(0x5576e1074010 msgr2=0x5576e1074410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.133+0000 7fc01d459740 1 -- 172.21.15.120:0/741690206 shutdown_connections 2024-09-19T09:25:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.133+0000 7fc01d459740 1 -- 172.21.15.120:0/741690206 wait complete. 2024-09-19T09:25:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.133+0000 7fc01d459740 1 -- start start 2024-09-19T09:25:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.134+0000 7fc01d459740 1 Processor -- start 2024-09-19T09:25:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.134+0000 7fc01d459740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576e10789a0 0x5576e10829b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.134+0000 7fc01d459740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576e10792e0 0x5576e1082ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.134+0000 7fc01d459740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e107e370 0x5576e1089f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.134+0000 7fc01d459740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5576e1081160 con 0x5576e107e370 2024-09-19T09:25:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.134+0000 7fc01d459740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5576e0f65290 con 0x5576e10789a0 2024-09-19T09:25:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.134+0000 7fc01d459740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5576e10812e0 con 0x5576e10792e0 2024-09-19T09:25:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.134+0000 7fc01d459740 10 client.0 _ll_register_callbacks cb 0x5576e0f96140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:25:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.134+0000 7fc018b81640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e107e370 0x5576e1089f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.134+0000 7fc018b81640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e107e370 0x5576e1089f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48214/0 (socket says 172.21.15.120:48214) 2024-09-19T09:25:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.134+0000 7fc018b81640 1 -- 172.21.15.120:0/634589922 learned_addr learned my addr 172.21.15.120:0/634589922 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.134+0000 7fc0137fe640 1 --2- 172.21.15.120:0/634589922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576e10792e0 0x5576e1082ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.134+0000 7fc013fff640 1 --2- 172.21.15.120:0/634589922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576e10789a0 0x5576e10829b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.134+0000 7fc0137fe640 1 -- 172.21.15.120:0/634589922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576e10789a0 msgr2=0x5576e10829b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.134+0000 7fc0137fe640 1 --2- 172.21.15.120:0/634589922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576e10789a0 0x5576e10829b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.134+0000 7fc0137fe640 1 -- 172.21.15.120:0/634589922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e107e370 msgr2=0x5576e1089f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.134+0000 7fc0137fe640 1 --2- 172.21.15.120:0/634589922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e107e370 0x5576e1089f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.134+0000 7fc0137fe640 1 -- 172.21.15.120:0/634589922 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576e0cef210 con 0x5576e10792e0 2024-09-19T09:25:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.134+0000 7fc018b81640 1 --2- 172.21.15.120:0/634589922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e107e370 0x5576e1089f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:25:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.135+0000 7fc0137fe640 1 --2- 172.21.15.120:0/634589922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576e10792e0 0x5576e1082ef0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc008000c00 tx=0x7fc00809c0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cef47f5defdd4ff2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.135+0000 7fbff37fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:25:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.135+0000 7fbff37fe640 1 -- 172.21.15.120:0/634589922 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0080a0570 con 0x5576e10792e0 2024-09-19T09:25:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.135+0000 7fbff37fe640 1 -- 172.21.15.120:0/634589922 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc0080cd7b0 con 0x5576e10792e0 2024-09-19T09:25:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.135+0000 7fc01d459740 1 -- 172.21.15.120:0/634589922 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc00c036030 con 0x5576e10792e0 2024-09-19T09:25:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.135+0000 7fbff37fe640 1 -- 172.21.15.120:0/634589922 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc0080a4e30 con 0x5576e10792e0 2024-09-19T09:25:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.135+0000 7fc01d459740 10 client.31279 fetch_fsmap learned FSMap version 39 2024-09-19T09:25:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.135+0000 7fc01d459740 1 -- 172.21.15.120:0/634589922 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5576e0dbb620 con 0x5576e10792e0 2024-09-19T09:25:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.135+0000 7fbff37fe640 1 -- 172.21.15.120:0/634589922 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc00c036030 con 0x5576e10792e0 2024-09-19T09:25:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.135+0000 7fbff37fe640 1 -- 172.21.15.120:0/634589922 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fc0080cdbf0 con 0x5576e10792e0 2024-09-19T09:25:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.135+0000 7fc01d459740 10 client.31279 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:25:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.135+0000 7fc01d459740 10 client.31279 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:25:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.135+0000 7fc01d459740 20 client.31279 populate_metadata read hostname 'smithi120' 2024-09-19T09:25:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.135+0000 7fc01d459740 1 --2- 172.21.15.120:0/634589922 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5576e108e6c0 0x5576e1090ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.135+0000 7fc01d459740 4 client.31279 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:25:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.135+0000 7fc01d459740 1 -- 172.21.15.120:0/634589922 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576e0fe9a20 con 0x5576e108e6c0 2024-09-19T09:25:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.136+0000 7fc013fff640 1 --2- 172.21.15.120:0/634589922 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5576e108e6c0 0x5576e1090ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.137+0000 7fbff37fe640 10 client.31279 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.137+0000 7fc013fff640 1 --2- 172.21.15.120:0/634589922 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5576e108e6c0 0x5576e1090ab0 crc :-1 s=READY pgs=553 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cba678410dab672c server_cookie=9c62ae9bb3e2b90d in_seq=0 out_seq=0 2024-09-19T09:25:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.141+0000 7fbff37fe640 1 -- 172.21.15.120:0/634589922 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5576e0fe9a20 con 0x5576e108e6c0 2024-09-19T09:25:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.141+0000 7fbff37fe640 10 client.31279 handle_command_reply: tid=0 2024-09-19T09:25:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.141+0000 7fc01d459740 1 client.31279 shutdown 2024-09-19T09:25:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.141+0000 7fc01d459740 1 -- 172.21.15.120:0/634589922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576e10792e0 msgr2=0x5576e1082ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.141+0000 7fc01d459740 1 --2- 172.21.15.120:0/634589922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576e10792e0 0x5576e1082ef0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc008000c00 tx=0x7fc00809c0c0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.141+0000 7fc01d459740 1 -- 172.21.15.120:0/634589922 shutdown_connections 2024-09-19T09:25:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.141+0000 7fc01d459740 1 --2- 172.21.15.120:0/634589922 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5576e108e6c0 0x5576e1090ab0 crc :-1 s=READY pgs=553 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.141+0000 7fc01d459740 1 --2- 172.21.15.120:0/634589922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e107e370 0x5576e1089f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.141+0000 7fc01d459740 1 --2- 172.21.15.120:0/634589922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576e10792e0 0x5576e1082ef0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.141+0000 7fc01d459740 1 --2- 172.21.15.120:0/634589922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576e10789a0 0x5576e10829b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.141+0000 7fbff37fe640 0 client.31279 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.141+0000 7fc01d459740 1 -- 172.21.15.120:0/634589922 >> 172.21.15.120:0/634589922 conn(0x5576e1074010 msgr2=0x5576e1075820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.141+0000 7fc01d459740 1 -- 172.21.15.120:0/634589922 shutdown_connections 2024-09-19T09:25:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.141+0000 7fc01d459740 1 -- 172.21.15.120:0/634589922 wait complete. 2024-09-19T09:25:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.141+0000 7fc01d459740 20 client.31279 trim_cache size 0 max 16384 2024-09-19T09:25:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.157+0000 7fc01d459740 1 Processor -- start 2024-09-19T09:25:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.157+0000 7fc01d459740 1 -- start start 2024-09-19T09:25:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.157+0000 7fc01d459740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e10789a0 0x5576e1081dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.157+0000 7fc01d459740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576e10792e0 0x5576e1089ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.157+0000 7fc01d459740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576e107e370 0x5576e1074ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:20.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.157+0000 7fc01d459740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141a8fa0 con 0x5576e10789a0 2024-09-19T09:25:20.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.157+0000 7fc01d459740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141aaae0 con 0x5576e10792e0 2024-09-19T09:25:20.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.157+0000 7fc01d459740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141a9670 con 0x5576e107e370 2024-09-19T09:25:20.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.157+0000 7fc013fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e10789a0 0x5576e1081dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:20.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.157+0000 7fc013fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e10789a0 0x5576e1081dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48224/0 (socket says 172.21.15.120:48224) 2024-09-19T09:25:20.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.157+0000 7fc013fff640 1 -- 172.21.15.120:0/4119899681 learned_addr learned my addr 172.21.15.120:0/4119899681 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:20.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.157+0000 7fc018b81640 1 --2- 172.21.15.120:0/4119899681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576e107e370 0x5576e1074ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:20.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.157+0000 7fc0137fe640 1 --2- 172.21.15.120:0/4119899681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576e10792e0 0x5576e1089ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:20.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.157+0000 7fc013fff640 1 -- 172.21.15.120:0/4119899681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576e107e370 msgr2=0x5576e1074ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:20.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.157+0000 7fc013fff640 1 --2- 172.21.15.120:0/4119899681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576e107e370 0x5576e1074ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.157+0000 7fc013fff640 1 -- 172.21.15.120:0/4119899681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576e10792e0 msgr2=0x5576e1089ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:20.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.157+0000 7fc013fff640 1 --2- 172.21.15.120:0/4119899681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576e10792e0 0x5576e1089ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.157+0000 7fc013fff640 1 -- 172.21.15.120:0/4119899681 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576e1091110 con 0x5576e10789a0 2024-09-19T09:25:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.157+0000 7fc013fff640 1 --2- 172.21.15.120:0/4119899681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e10789a0 0x5576e1081dc0 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x5576e1074e90 tx=0x7fc004019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7024fcfed82959fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.157+0000 7fbff9ffb640 1 -- 172.21.15.120:0/4119899681 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004004020 con 0x5576e10789a0 2024-09-19T09:25:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.158+0000 7fbff9ffb640 1 -- 172.21.15.120:0/4119899681 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc00401ed50 con 0x5576e10789a0 2024-09-19T09:25:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.158+0000 7fbff9ffb640 1 -- 172.21.15.120:0/4119899681 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004003ab0 con 0x5576e10789a0 2024-09-19T09:25:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.158+0000 7fc01d459740 1 -- 172.21.15.120:0/4119899681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e10789a0 msgr2=0x5576e1081dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.158+0000 7fc01d459740 1 --2- 172.21.15.120:0/4119899681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e10789a0 0x5576e1081dc0 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x5576e1074e90 tx=0x7fc004019000 comp rx=0 tx=0).stop 2024-09-19T09:25:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.158+0000 7fc01d459740 1 -- 172.21.15.120:0/4119899681 shutdown_connections 2024-09-19T09:25:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.158+0000 7fc01d459740 1 --2- 172.21.15.120:0/4119899681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576e107e370 0x5576e1074ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.158+0000 7fc01d459740 1 --2- 172.21.15.120:0/4119899681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576e10792e0 0x5576e1089ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.158+0000 7fc01d459740 1 --2- 172.21.15.120:0/4119899681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e10789a0 0x5576e1081dc0 unknown :-1 s=CLOSED pgs=2312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.158+0000 7fc01d459740 1 -- 172.21.15.120:0/4119899681 >> 172.21.15.120:0/4119899681 conn(0x5576e1074010 msgr2=0x5576e10816b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.158+0000 7fc01d459740 1 -- 172.21.15.120:0/4119899681 shutdown_connections 2024-09-19T09:25:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.158+0000 7fc01d459740 1 -- 172.21.15.120:0/4119899681 wait complete. 2024-09-19T09:25:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.159+0000 7fc01d459740 1 -- start start 2024-09-19T09:25:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.159+0000 7fc01d459740 1 Processor -- start 2024-09-19T09:25:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.159+0000 7fc01d459740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e10789a0 0x5576e10874a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.159+0000 7fc01d459740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576e10792e0 0x5576e10879e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.159+0000 7fc01d459740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576e107e370 0x5576e10889f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.159+0000 7fc01d459740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141aa410 con 0x5576e10789a0 2024-09-19T09:25:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.159+0000 7fc01d459740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141a9d40 con 0x5576e107e370 2024-09-19T09:25:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.159+0000 7fc01d459740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141a6f60 con 0x5576e10792e0 2024-09-19T09:25:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.159+0000 7fc01d459740 10 client.0 _ll_register_callbacks cb 0x7fc00800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:25:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.159+0000 7fc013fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e10789a0 0x5576e10874a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.159+0000 7fc0137fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576e10792e0 0x5576e10879e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.159+0000 7fc013fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e10789a0 0x5576e10874a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48234/0 (socket says 172.21.15.120:48234) 2024-09-19T09:25:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.159+0000 7fc013fff640 1 -- 172.21.15.120:0/1682627449 learned_addr learned my addr 172.21.15.120:0/1682627449 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.159+0000 7fc018b81640 1 --2- 172.21.15.120:0/1682627449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576e107e370 0x5576e10889f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.160+0000 7fc0137fe640 1 -- 172.21.15.120:0/1682627449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576e107e370 msgr2=0x5576e10889f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.160+0000 7fc0137fe640 1 --2- 172.21.15.120:0/1682627449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576e107e370 0x5576e10889f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.160+0000 7fc0137fe640 1 -- 172.21.15.120:0/1682627449 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e10789a0 msgr2=0x5576e10874a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.160+0000 7fc0137fe640 1 --2- 172.21.15.120:0/1682627449 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e10789a0 0x5576e10874a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.160+0000 7fc0137fe640 1 -- 172.21.15.120:0/1682627449 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576e1089000 con 0x5576e10792e0 2024-09-19T09:25:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.160+0000 7fc0137fe640 1 --2- 172.21.15.120:0/1682627449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576e10792e0 0x5576e10879e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc008098c40 tx=0x7fc0080d32b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf9a953e1d11435c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.160+0000 7fbff37fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:25:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.160+0000 7fbff37fe640 1 -- 172.21.15.120:0/1682627449 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0080a0570 con 0x5576e10792e0 2024-09-19T09:25:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.160+0000 7fbff37fe640 1 -- 172.21.15.120:0/1682627449 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc0080bc0b0 con 0x5576e10792e0 2024-09-19T09:25:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.160+0000 7fc01d459740 1 -- 172.21.15.120:0/1682627449 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc00401ed50 con 0x5576e10792e0 2024-09-19T09:25:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.160+0000 7fbff37fe640 1 -- 172.21.15.120:0/1682627449 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc0080bc2b0 con 0x5576e10792e0 2024-09-19T09:25:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.160+0000 7fc01d459740 10 client.31285 fetch_fsmap learned FSMap version 39 2024-09-19T09:25:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.160+0000 7fc01d459740 1 -- 172.21.15.120:0/1682627449 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5576e10f9370 con 0x5576e10792e0 2024-09-19T09:25:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.161+0000 7fbff37fe640 1 -- 172.21.15.120:0/1682627449 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc00401ed50 con 0x5576e10792e0 2024-09-19T09:25:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.161+0000 7fbff37fe640 1 -- 172.21.15.120:0/1682627449 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fc008018b90 con 0x5576e10792e0 2024-09-19T09:25:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.161+0000 7fc01d459740 10 client.31285 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:25:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.161+0000 7fc01d459740 10 client.31285 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:25:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.161+0000 7fc01d459740 20 client.31285 populate_metadata read hostname 'smithi120' 2024-09-19T09:25:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.161+0000 7fc01d459740 1 --2- 172.21.15.120:0/1682627449 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5576e10f9c90 0x5576e10fc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.161+0000 7fc01d459740 4 client.31285 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:25:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.161+0000 7fc01d459740 1 -- 172.21.15.120:0/1682627449 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5576e0fe3d70 con 0x5576e10f9c90 2024-09-19T09:25:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.161+0000 7fc013fff640 1 --2- 172.21.15.120:0/1682627449 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5576e10f9c90 0x5576e10fc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.161+0000 7fbff37fe640 10 client.31285 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.161+0000 7fc013fff640 1 --2- 172.21.15.120:0/1682627449 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5576e10f9c90 0x5576e10fc0b0 crc :-1 s=READY pgs=554 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccf008e811f4adc4 server_cookie=b7281a69f71ea9b4 in_seq=0 out_seq=0 2024-09-19T09:25:20.377 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:20 smithi120 ceph-mon[31150]: pgmap v1019: 97 pgs: 97 active+clean; 466 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.4 MiB/s wr, 526 op/s 2024-09-19T09:25:20.383 INFO:tasks.workunit.client.0.smithi120.stdout: 10 214756 19.74 MB/sec execute 435 sec latency 115.957 ms 2024-09-19T09:25:20.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:20 smithi196 ceph-mon[35150]: pgmap v1019: 97 pgs: 97 active+clean; 466 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.4 MiB/s wr, 526 op/s 2024-09-19T09:25:20.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:20 smithi153 ceph-mon[35071]: pgmap v1019: 97 pgs: 97 active+clean; 466 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.4 MiB/s wr, 526 op/s 2024-09-19T09:25:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.779+0000 7fbff37fe640 1 -- 172.21.15.120:0/1682627449 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5576e0fe3d70 con 0x5576e10f9c90 2024-09-19T09:25:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.779+0000 7fbff37fe640 10 client.31285 handle_command_reply: tid=0 2024-09-19T09:25:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.779+0000 7fc01d459740 1 client.31285 shutdown 2024-09-19T09:25:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.779+0000 7fc01d459740 1 -- 172.21.15.120:0/1682627449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576e10792e0 msgr2=0x5576e10879e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.779+0000 7fc01d459740 1 --2- 172.21.15.120:0/1682627449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576e10792e0 0x5576e10879e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc008098c40 tx=0x7fc0080d32b0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.779+0000 7fc01d459740 1 -- 172.21.15.120:0/1682627449 shutdown_connections 2024-09-19T09:25:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.779+0000 7fc01d459740 1 --2- 172.21.15.120:0/1682627449 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5576e10f9c90 0x5576e10fc0b0 crc :-1 s=READY pgs=554 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.779+0000 7fc01d459740 1 --2- 172.21.15.120:0/1682627449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576e107e370 0x5576e10889f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.779+0000 7fc01d459740 1 --2- 172.21.15.120:0/1682627449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576e10792e0 0x5576e10879e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.779+0000 7fbff37fe640 0 client.31285 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.779+0000 7fc01d459740 1 --2- 172.21.15.120:0/1682627449 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576e10789a0 0x5576e10874a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.779+0000 7fc01d459740 1 -- 172.21.15.120:0/1682627449 >> 172.21.15.120:0/1682627449 conn(0x5576e1074010 msgr2=0x5576e1081ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.779+0000 7fc01d459740 1 -- 172.21.15.120:0/1682627449 shutdown_connections 2024-09-19T09:25:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.779+0000 7fc01d459740 1 -- 172.21.15.120:0/1682627449 wait complete. 2024-09-19T09:25:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.779+0000 7fc01d459740 20 client.31285 trim_cache size 0 max 16384 2024-09-19T09:25:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:25:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:25:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:25:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:25:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.780+0000 7fc01a5e4640 1 -- 172.21.15.120:0/4092960254 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbffc0a3370 msgr2=0x7fbffc0a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.780+0000 7fc01a5e4640 1 --2- 172.21.15.120:0/4092960254 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbffc0a3370 0x7fbffc0a5830 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc00400bfd0 tx=0x7fc00400b4a0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.780+0000 7fc01a5e4640 1 -- 172.21.15.120:0/4092960254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc014101c30 msgr2=0x7fc014198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.780+0000 7fc01a5e4640 1 --2- 172.21.15.120:0/4092960254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc014101c30 0x7fc014198290 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc00800e9f0 tx=0x7fc00800eeb0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.780+0000 7fc01a5e4640 1 -- 172.21.15.120:0/4092960254 shutdown_connections 2024-09-19T09:25:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.780+0000 7fc01a5e4640 1 --2- 172.21.15.120:0/4092960254 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbffc0a3370 0x7fbffc0a5830 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.780+0000 7fc01a5e4640 1 --2- 172.21.15.120:0/4092960254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0141025f0 0x7fc01419f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.780+0000 7fc01a5e4640 1 --2- 172.21.15.120:0/4092960254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc014101c30 0x7fc014198290 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.780+0000 7fc01a5e4640 1 --2- 172.21.15.120:0/4092960254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc014100a40 0x7fc014197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.780+0000 7fc01a5e4640 1 -- 172.21.15.120:0/4092960254 >> 172.21.15.120:0/4092960254 conn(0x7fc0140fc1e0 msgr2=0x7fc014104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.781+0000 7fc01a5e4640 1 -- 172.21.15.120:0/4092960254 shutdown_connections 2024-09-19T09:25:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:20.781+0000 7fc01a5e4640 1 -- 172.21.15.120:0/4092960254 wait complete. 2024-09-19T09:25:21.383 INFO:tasks.workunit.client.0.smithi120.stdout: 10 215059 19.74 MB/sec execute 436 sec latency 316.134 ms 2024-09-19T09:25:22.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:22 smithi120 ceph-mon[31150]: pgmap v1020: 97 pgs: 97 active+clean; 471 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.4 MiB/s wr, 522 op/s 2024-09-19T09:25:22.383 INFO:tasks.workunit.client.0.smithi120.stdout: 10 215511 19.73 MB/sec execute 437 sec latency 67.880 ms 2024-09-19T09:25:22.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:22 smithi196 ceph-mon[35150]: pgmap v1020: 97 pgs: 97 active+clean; 471 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.4 MiB/s wr, 522 op/s 2024-09-19T09:25:22.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:22 smithi153 ceph-mon[35071]: pgmap v1020: 97 pgs: 97 active+clean; 471 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.4 MiB/s wr, 522 op/s 2024-09-19T09:25:23.383 INFO:tasks.workunit.client.0.smithi120.stdout: 10 216072 19.75 MB/sec execute 438 sec latency 155.473 ms 2024-09-19T09:25:24.383 INFO:tasks.workunit.client.0.smithi120.stdout: 10 216745 19.78 MB/sec execute 439 sec latency 54.317 ms 2024-09-19T09:25:24.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:24 smithi196 ceph-mon[35150]: pgmap v1021: 97 pgs: 97 active+clean; 472 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.3 MiB/s wr, 546 op/s 2024-09-19T09:25:24.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:24 smithi153 ceph-mon[35071]: pgmap v1021: 97 pgs: 97 active+clean; 472 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.3 MiB/s wr, 546 op/s 2024-09-19T09:25:24.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:24 smithi120 ceph-mon[31150]: pgmap v1021: 97 pgs: 97 active+clean; 472 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.3 MiB/s wr, 546 op/s 2024-09-19T09:25:25.383 INFO:tasks.workunit.client.0.smithi120.stdout: 10 217574 19.84 MB/sec execute 440 sec latency 63.770 ms 2024-09-19T09:25:25.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:25 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:25:25.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:25 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:25:25.577 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:25 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:25:26.383 INFO:tasks.workunit.client.0.smithi120.stdout: 10 218212 19.89 MB/sec execute 441 sec latency 55.548 ms 2024-09-19T09:25:26.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:26 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:25:26.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:26 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:25:26.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:26 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:25:26.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:26 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:25:26.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:26 smithi120 ceph-mon[31150]: pgmap v1022: 97 pgs: 97 active+clean; 466 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.1 MiB/s wr, 545 op/s 2024-09-19T09:25:26.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:26 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:26.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:26 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:25:26.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:26 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:25:26.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:26 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:25:26.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:26 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:25:26.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:26 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:25:26.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:26 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:25:26.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:26 smithi196 ceph-mon[35150]: pgmap v1022: 97 pgs: 97 active+clean; 466 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.1 MiB/s wr, 545 op/s 2024-09-19T09:25:26.938 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:26 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:26.938 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:26 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:25:26.938 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:26 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:25:26.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:26 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:25:26.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:26 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:25:26.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:26 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:25:26.939 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:26 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:25:26.939 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:26 smithi153 ceph-mon[35071]: pgmap v1022: 97 pgs: 97 active+clean; 466 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.1 MiB/s wr, 545 op/s 2024-09-19T09:25:26.939 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:26 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:26.939 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:26 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:25:26.939 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:26 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:25:27.383 INFO:tasks.workunit.client.0.smithi120.stdout: 10 218701 19.90 MB/sec execute 442 sec latency 183.899 ms 2024-09-19T09:25:27.672 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:25:27.956 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:28.383 INFO:tasks.workunit.client.0.smithi120.stdout: 10 219152 19.92 MB/sec execute 443 sec latency 116.310 ms 2024-09-19T09:25:28.428 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:28 smithi120 ceph-mon[31150]: pgmap v1023: 97 pgs: 97 active+clean; 480 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.3 MiB/s wr, 631 op/s 2024-09-19T09:25:28.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:28 smithi196 ceph-mon[35150]: pgmap v1023: 97 pgs: 97 active+clean; 480 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.3 MiB/s wr, 631 op/s 2024-09-19T09:25:28.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:28 smithi153 ceph-mon[35071]: pgmap v1023: 97 pgs: 97 active+clean; 480 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.3 MiB/s wr, 631 op/s 2024-09-19T09:25:28.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.518+0000 7f8658043640 1 -- 172.21.15.120:0/4105439769 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8650100e20 msgr2=0x7f8650101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:28.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.518+0000 7f8658043640 1 --2- 172.21.15.120:0/4105439769 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8650100e20 0x7f8650101200 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f864c009900 tx=0x7f864c02a480 comp rx=0 tx=0).stop 2024-09-19T09:25:28.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.518+0000 7f8658043640 1 -- 172.21.15.120:0/4105439769 shutdown_connections 2024-09-19T09:25:28.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.518+0000 7f8658043640 1 --2- 172.21.15.120:0/4105439769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8650102ef0 0x7f865010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.518+0000 7f8658043640 1 --2- 172.21.15.120:0/4105439769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8650102570 0x7f86501029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.518+0000 7f8658043640 1 --2- 172.21.15.120:0/4105439769 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8650100e20 0x7f8650101200 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.518+0000 7f8658043640 1 -- 172.21.15.120:0/4105439769 >> 172.21.15.120:0/4105439769 conn(0x7f86500fc7d0 msgr2=0x7f86500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:28.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.518+0000 7f8658043640 1 -- 172.21.15.120:0/4105439769 shutdown_connections 2024-09-19T09:25:28.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.518+0000 7f8658043640 1 -- 172.21.15.120:0/4105439769 wait complete. 2024-09-19T09:25:28.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.519+0000 7f8658043640 1 Processor -- start 2024-09-19T09:25:28.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.519+0000 7f8658043640 1 -- start start 2024-09-19T09:25:28.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.519+0000 7f8658043640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8650100e20 0x7f865019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:28.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.519+0000 7f8658043640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8650102570 0x7f865019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:28.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.519+0000 7f8658043640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8650102ef0 0x7f865019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:28.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.519+0000 7f8658043640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8650110f80 con 0x7f8650102570 2024-09-19T09:25:28.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.519+0000 7f8658043640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8650110e00 con 0x7f8650100e20 2024-09-19T09:25:28.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.519+0000 7f8658043640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8650111100 con 0x7f8650102ef0 2024-09-19T09:25:28.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.519+0000 7f86555de640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8650102570 0x7f865019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:28.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.519+0000 7f86555de640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8650102570 0x7f865019abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37384/0 (socket says 172.21.15.120:37384) 2024-09-19T09:25:28.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.519+0000 7f86555de640 1 -- 172.21.15.120:0/2305254888 learned_addr learned my addr 172.21.15.120:0/2305254888 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:28.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.519+0000 7f86565e0640 1 --2- 172.21.15.120:0/2305254888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8650102ef0 0x7f865019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:28.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.519+0000 7f8655ddf640 1 --2- 172.21.15.120:0/2305254888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8650100e20 0x7f865019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:28.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.519+0000 7f86555de640 1 -- 172.21.15.120:0/2305254888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8650102ef0 msgr2=0x7f865019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:28.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.519+0000 7f86555de640 1 --2- 172.21.15.120:0/2305254888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8650102ef0 0x7f865019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.519+0000 7f86555de640 1 -- 172.21.15.120:0/2305254888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8650100e20 msgr2=0x7f865019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:28.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.519+0000 7f86555de640 1 --2- 172.21.15.120:0/2305254888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8650100e20 0x7f865019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.519+0000 7f86555de640 1 -- 172.21.15.120:0/2305254888 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865019f660 con 0x7f8650102570 2024-09-19T09:25:28.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.519+0000 7f86565e0640 1 --2- 172.21.15.120:0/2305254888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8650102ef0 0x7f865019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:25:28.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.520+0000 7f86555de640 1 --2- 172.21.15.120:0/2305254888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8650102570 0x7f865019abf0 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f864000d950 tx=0x7f864000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:28.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.520+0000 7f8646ffd640 1 -- 172.21.15.120:0/2305254888 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8640017070 con 0x7f8650102570 2024-09-19T09:25:28.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.520+0000 7f8658043640 1 -- 172.21.15.120:0/2305254888 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f865019f950 con 0x7f8650102570 2024-09-19T09:25:28.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.520+0000 7f8646ffd640 1 -- 172.21.15.120:0/2305254888 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8640009c30 con 0x7f8650102570 2024-09-19T09:25:28.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.520+0000 7f8646ffd640 1 -- 172.21.15.120:0/2305254888 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864000f6b0 con 0x7f8650102570 2024-09-19T09:25:28.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.520+0000 7f8658043640 1 -- 172.21.15.120:0/2305254888 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86501a7230 con 0x7f8650102570 2024-09-19T09:25:28.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.522+0000 7f8646ffd640 1 -- 172.21.15.120:0/2305254888 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f86400109d0 con 0x7f8650102570 2024-09-19T09:25:28.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.523+0000 7f865a41c740 1 Processor -- start 2024-09-19T09:25:28.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.523+0000 7f865a41c740 1 -- start start 2024-09-19T09:25:28.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.523+0000 7f8646ffd640 1 --2- 172.21.15.120:0/2305254888 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f863c0a3240 0x7f863c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:28.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.523+0000 7f865a41c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbffa684a0 0x55dbffa68880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:28.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.523+0000 7f865a41c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbffa68dc0 0x55dbffa73140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:28.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.523+0000 7f865a41c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbffa73680 0x55dbffa75a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:28.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.523+0000 7f865a41c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dbff9ce2e0 con 0x55dbffa68dc0 2024-09-19T09:25:28.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.523+0000 7f865a41c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55dbff975ce0 con 0x55dbffa684a0 2024-09-19T09:25:28.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.523+0000 7f865a41c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dbff9ca2a0 con 0x55dbffa73680 2024-09-19T09:25:28.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.523+0000 7f86565e0640 1 --2- 172.21.15.120:0/2305254888 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f863c0a3240 0x7f863c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:28.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.523+0000 7f8646ffd640 1 -- 172.21.15.120:0/2305254888 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f86400cf4f0 con 0x7f8650102570 2024-09-19T09:25:28.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.523+0000 7f8655ddf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbffa684a0 0x55dbffa68880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:28.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.523+0000 7f8655ddf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbffa684a0 0x55dbffa68880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:46504/0 (socket says 172.21.15.120:46504) 2024-09-19T09:25:28.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.523+0000 7f8655ddf640 1 -- 172.21.15.120:0/762286255 learned_addr learned my addr 172.21.15.120:0/762286255 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:28.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.524+0000 7f86555de640 1 --2- 172.21.15.120:0/762286255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbffa68dc0 0x55dbffa73140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:28.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.524+0000 7f86565e0640 1 --2- 172.21.15.120:0/762286255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbffa73680 0x55dbffa75a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:28.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.524+0000 7f86555de640 1 -- 172.21.15.120:0/762286255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbffa73680 msgr2=0x55dbffa75a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:28.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.524+0000 7f86555de640 1 --2- 172.21.15.120:0/762286255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbffa73680 0x55dbffa75a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.524+0000 7f86555de640 1 -- 172.21.15.120:0/762286255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbffa684a0 msgr2=0x55dbffa68880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:28.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.524+0000 7f86555de640 1 --2- 172.21.15.120:0/762286255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbffa684a0 0x55dbffa68880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.524+0000 7f86555de640 1 -- 172.21.15.120:0/762286255 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbff95a440 con 0x55dbffa68dc0 2024-09-19T09:25:28.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.524+0000 7f8655ddf640 1 --2- 172.21.15.120:0/762286255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbffa684a0 0x55dbffa68880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:25:28.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.524+0000 7f86555de640 1 --2- 172.21.15.120:0/762286255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbffa68dc0 0x55dbffa73140 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f86400ccff0 tx=0x7f864000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c271b8e4669583a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:28.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.524+0000 7f863bfff640 1 -- 172.21.15.120:0/762286255 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8640020040 con 0x55dbffa68dc0 2024-09-19T09:25:28.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.524+0000 7f863bfff640 1 -- 172.21.15.120:0/762286255 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86400a4780 con 0x55dbffa68dc0 2024-09-19T09:25:28.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.524+0000 7f86565e0640 1 --2- 172.21.15.120:0/2305254888 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f863c0a3240 0x7f863c0a5700 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f86480098b0 tx=0x7f8648005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:28.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.524+0000 7f863bfff640 1 -- 172.21.15.120:0/762286255 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864009b010 con 0x55dbffa68dc0 2024-09-19T09:25:28.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.525+0000 7f865a41c740 1 -- 172.21.15.120:0/762286255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbffa68dc0 msgr2=0x55dbffa73140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:28.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.525+0000 7f865a41c740 1 --2- 172.21.15.120:0/762286255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbffa68dc0 0x55dbffa73140 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f86400ccff0 tx=0x7f864000bf60 comp rx=0 tx=0).stop 2024-09-19T09:25:28.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.525+0000 7f865a41c740 1 -- 172.21.15.120:0/762286255 shutdown_connections 2024-09-19T09:25:28.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.525+0000 7f865a41c740 1 --2- 172.21.15.120:0/762286255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbffa73680 0x55dbffa75a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.525+0000 7f865a41c740 1 --2- 172.21.15.120:0/762286255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbffa68dc0 0x55dbffa73140 unknown :-1 s=CLOSED pgs=2315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.525+0000 7f865a41c740 1 --2- 172.21.15.120:0/762286255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbffa684a0 0x55dbffa68880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.525+0000 7f865a41c740 1 -- 172.21.15.120:0/762286255 >> 172.21.15.120:0/762286255 conn(0x55dbffa64a30 msgr2=0x55dbffa64e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:28.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.525+0000 7f865a41c740 1 -- 172.21.15.120:0/762286255 shutdown_connections 2024-09-19T09:25:28.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.525+0000 7f865a41c740 1 -- 172.21.15.120:0/762286255 wait complete. 2024-09-19T09:25:28.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.526+0000 7f865a41c740 1 -- start start 2024-09-19T09:25:28.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.526+0000 7f865a41c740 1 Processor -- start 2024-09-19T09:25:28.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.526+0000 7f865a41c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbffa684a0 0x55dbffa72fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:28.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.526+0000 7f865a41c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbffa68dc0 0x55dbffa70270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:28.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.526+0000 7f865a41c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbffa73680 0x55dbffa707b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:28.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.526+0000 7f865a41c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dbffa761f0 con 0x55dbffa73680 2024-09-19T09:25:28.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.526+0000 7f865a41c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55dbff989050 con 0x55dbffa684a0 2024-09-19T09:25:28.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.526+0000 7f865a41c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dbffa76370 con 0x55dbffa68dc0 2024-09-19T09:25:28.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.527+0000 7f865a41c740 10 client.0 _ll_register_callbacks cb 0x55dbff9c9160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:25:28.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.527+0000 7f8655ddf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbffa684a0 0x55dbffa72fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:28.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.527+0000 7f8655ddf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbffa684a0 0x55dbffa72fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:46516/0 (socket says 172.21.15.120:46516) 2024-09-19T09:25:28.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.527+0000 7f8655ddf640 1 -- 172.21.15.120:0/3346885354 learned_addr learned my addr 172.21.15.120:0/3346885354 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:28.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.527+0000 7f86555de640 1 --2- 172.21.15.120:0/3346885354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbffa68dc0 0x55dbffa70270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:28.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.527+0000 7f86565e0640 1 --2- 172.21.15.120:0/3346885354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbffa73680 0x55dbffa707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:28.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.527+0000 7f8655ddf640 1 -- 172.21.15.120:0/3346885354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbffa68dc0 msgr2=0x55dbffa70270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:28.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.527+0000 7f8655ddf640 1 --2- 172.21.15.120:0/3346885354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbffa68dc0 0x55dbffa70270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.527+0000 7f8655ddf640 1 -- 172.21.15.120:0/3346885354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbffa73680 msgr2=0x55dbffa707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:28.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.527+0000 7f8655ddf640 1 --2- 172.21.15.120:0/3346885354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbffa73680 0x55dbffa707b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.527+0000 7f8655ddf640 1 -- 172.21.15.120:0/3346885354 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbff73f3c0 con 0x55dbffa684a0 2024-09-19T09:25:28.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.527+0000 7f86555de640 1 --2- 172.21.15.120:0/3346885354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbffa68dc0 0x55dbffa70270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:25:28.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.527+0000 7f8655ddf640 1 --2- 172.21.15.120:0/3346885354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbffa684a0 0x55dbffa72fc0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f864c005940 tx=0x7f864c004440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac44ee5191e355f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:28.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.527+0000 7f86397fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:25:28.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.527+0000 7f86397fa640 1 -- 172.21.15.120:0/3346885354 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c035030 con 0x55dbffa684a0 2024-09-19T09:25:28.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.527+0000 7f865a41c740 1 -- 172.21.15.120:0/3346885354 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86400a4780 con 0x55dbffa684a0 2024-09-19T09:25:28.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.527+0000 7f86397fa640 1 -- 172.21.15.120:0/3346885354 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f864c003ac0 con 0x55dbffa684a0 2024-09-19T09:25:28.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.528+0000 7f86397fa640 1 -- 172.21.15.120:0/3346885354 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f864c003cc0 con 0x55dbffa684a0 2024-09-19T09:25:28.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.528+0000 7f865a41c740 10 client.31436 fetch_fsmap learned FSMap version 39 2024-09-19T09:25:28.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.528+0000 7f865a41c740 1 -- 172.21.15.120:0/3346885354 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55dbffa81ba0 con 0x55dbffa684a0 2024-09-19T09:25:28.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.528+0000 7f86397fa640 1 -- 172.21.15.120:0/3346885354 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86400a4780 con 0x55dbffa684a0 2024-09-19T09:25:28.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.528+0000 7f86397fa640 1 -- 172.21.15.120:0/3346885354 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f864c031540 con 0x55dbffa684a0 2024-09-19T09:25:28.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.528+0000 7f865a41c740 10 client.31436 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:25:28.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.528+0000 7f865a41c740 10 client.31436 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:25:28.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.528+0000 7f865a41c740 20 client.31436 populate_metadata read hostname 'smithi120' 2024-09-19T09:25:28.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.528+0000 7f865a41c740 1 --2- 172.21.15.120:0/3346885354 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dbffa83060 0x55dbffa85450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:28.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.528+0000 7f865a41c740 4 client.31436 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:25:28.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.528+0000 7f865a41c740 1 -- 172.21.15.120:0/3346885354 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbff9db850 con 0x55dbffa83060 2024-09-19T09:25:28.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.528+0000 7f86555de640 1 --2- 172.21.15.120:0/3346885354 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dbffa83060 0x55dbffa85450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:28.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.529+0000 7f86555de640 1 --2- 172.21.15.120:0/3346885354 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dbffa83060 0x55dbffa85450 crc :-1 s=READY pgs=555 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1717ee9bec05266f server_cookie=61c735b8b4149d5f in_seq=0 out_seq=0 2024-09-19T09:25:28.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.529+0000 7f86397fa640 10 client.31436 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:28.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.530+0000 7f86397fa640 1 -- 172.21.15.120:0/3346885354 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dbff9db850 con 0x55dbffa83060 2024-09-19T09:25:28.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.530+0000 7f86397fa640 10 client.31436 handle_command_reply: tid=0 2024-09-19T09:25:28.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.530+0000 7f865a41c740 1 client.31436 shutdown 2024-09-19T09:25:28.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.530+0000 7f865a41c740 1 -- 172.21.15.120:0/3346885354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbffa684a0 msgr2=0x55dbffa72fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:28.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.530+0000 7f865a41c740 1 --2- 172.21.15.120:0/3346885354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbffa684a0 0x55dbffa72fc0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f864c005940 tx=0x7f864c004440 comp rx=0 tx=0).stop 2024-09-19T09:25:28.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.530+0000 7f865a41c740 1 -- 172.21.15.120:0/3346885354 shutdown_connections 2024-09-19T09:25:28.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.530+0000 7f865a41c740 1 --2- 172.21.15.120:0/3346885354 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dbffa83060 0x55dbffa85450 crc :-1 s=READY pgs=555 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.530+0000 7f865a41c740 1 --2- 172.21.15.120:0/3346885354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbffa73680 0x55dbffa707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.530+0000 7f86397fa640 0 client.31436 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:28.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.530+0000 7f865a41c740 1 --2- 172.21.15.120:0/3346885354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbffa68dc0 0x55dbffa70270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.530+0000 7f865a41c740 1 --2- 172.21.15.120:0/3346885354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbffa684a0 0x55dbffa72fc0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.530+0000 7f865a41c740 1 -- 172.21.15.120:0/3346885354 >> 172.21.15.120:0/3346885354 conn(0x55dbffa64a30 msgr2=0x55dbffa63b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:28.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.530+0000 7f865a41c740 1 -- 172.21.15.120:0/3346885354 shutdown_connections 2024-09-19T09:25:28.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.530+0000 7f865a41c740 1 -- 172.21.15.120:0/3346885354 wait complete. 2024-09-19T09:25:28.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.530+0000 7f865a41c740 20 client.31436 trim_cache size 0 max 16384 2024-09-19T09:25:28.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.552+0000 7f865a41c740 1 Processor -- start 2024-09-19T09:25:28.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.552+0000 7f865a41c740 1 -- start start 2024-09-19T09:25:28.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.552+0000 7f865a41c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbffa64a30 0x55dbffa63a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:28.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.552+0000 7f865a41c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbffa684a0 0x55dbffa8ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:28.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.552+0000 7f865a41c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbffa68dc0 0x55dbffa8b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:28.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.552+0000 7f865a41c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f86501af760 con 0x55dbffa68dc0 2024-09-19T09:25:28.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.552+0000 7f865a41c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f86501b12a0 con 0x55dbffa684a0 2024-09-19T09:25:28.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.552+0000 7f865a41c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f86501afe30 con 0x55dbffa64a30 2024-09-19T09:25:28.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.552+0000 7f86565e0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbffa68dc0 0x55dbffa8b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:28.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.552+0000 7f86565e0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbffa68dc0 0x55dbffa8b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37410/0 (socket says 172.21.15.120:37410) 2024-09-19T09:25:28.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.552+0000 7f86565e0640 1 -- 172.21.15.120:0/1986267775 learned_addr learned my addr 172.21.15.120:0/1986267775 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:28.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.552+0000 7f86555de640 1 --2- 172.21.15.120:0/1986267775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbffa684a0 0x55dbffa8ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:28.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.552+0000 7f8655ddf640 1 --2- 172.21.15.120:0/1986267775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbffa64a30 0x55dbffa63a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:28.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.552+0000 7f86565e0640 1 -- 172.21.15.120:0/1986267775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbffa64a30 msgr2=0x55dbffa63a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:28.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.552+0000 7f86565e0640 1 --2- 172.21.15.120:0/1986267775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbffa64a30 0x55dbffa63a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.552+0000 7f86565e0640 1 -- 172.21.15.120:0/1986267775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbffa684a0 msgr2=0x55dbffa8ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:28.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.553+0000 7f86565e0640 1 --2- 172.21.15.120:0/1986267775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbffa684a0 0x55dbffa8ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.553+0000 7f86565e0640 1 -- 172.21.15.120:0/1986267775 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbffa73330 con 0x55dbffa68dc0 2024-09-19T09:25:28.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.553+0000 7f86565e0640 1 --2- 172.21.15.120:0/1986267775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbffa68dc0 0x55dbffa8b260 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f8648010ee0 tx=0x7f8648014050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbeee36bd171734e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:28.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.553+0000 7f863b7fe640 1 -- 172.21.15.120:0/1986267775 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864801fe30 con 0x55dbffa68dc0 2024-09-19T09:25:28.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.553+0000 7f863b7fe640 1 -- 172.21.15.120:0/1986267775 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f864802a960 con 0x55dbffa68dc0 2024-09-19T09:25:28.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.553+0000 7f863b7fe640 1 -- 172.21.15.120:0/1986267775 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864801da20 con 0x55dbffa68dc0 2024-09-19T09:25:28.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.554+0000 7f865a41c740 1 -- 172.21.15.120:0/1986267775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbffa68dc0 msgr2=0x55dbffa8b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:28.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.554+0000 7f865a41c740 1 --2- 172.21.15.120:0/1986267775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbffa68dc0 0x55dbffa8b260 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f8648010ee0 tx=0x7f8648014050 comp rx=0 tx=0).stop 2024-09-19T09:25:28.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.554+0000 7f865a41c740 1 -- 172.21.15.120:0/1986267775 shutdown_connections 2024-09-19T09:25:28.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.554+0000 7f865a41c740 1 --2- 172.21.15.120:0/1986267775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbffa68dc0 0x55dbffa8b260 unknown :-1 s=CLOSED pgs=2316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.554+0000 7f865a41c740 1 --2- 172.21.15.120:0/1986267775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbffa684a0 0x55dbffa8ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.554+0000 7f865a41c740 1 --2- 172.21.15.120:0/1986267775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbffa64a30 0x55dbffa63a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.554+0000 7f865a41c740 1 -- 172.21.15.120:0/1986267775 >> 172.21.15.120:0/1986267775 conn(0x55dbffa70910 msgr2=0x55dbffa63370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:28.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.554+0000 7f865a41c740 1 -- 172.21.15.120:0/1986267775 shutdown_connections 2024-09-19T09:25:28.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.554+0000 7f865a41c740 1 -- 172.21.15.120:0/1986267775 wait complete. 2024-09-19T09:25:28.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.554+0000 7f865a41c740 1 -- start start 2024-09-19T09:25:28.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.554+0000 7f865a41c740 1 Processor -- start 2024-09-19T09:25:28.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.555+0000 7f865a41c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbffa64a30 0x55dbffa78e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:28.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.555+0000 7f865a41c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbffa684a0 0x55dbffa79340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:28.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.555+0000 7f865a41c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbffa68dc0 0x55dbffae4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:28.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.555+0000 7f865a41c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f86501b0bd0 con 0x55dbffa64a30 2024-09-19T09:25:28.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.555+0000 7f865a41c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f86501b0500 con 0x55dbffa68dc0 2024-09-19T09:25:28.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.555+0000 7f865a41c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f86501ad720 con 0x55dbffa684a0 2024-09-19T09:25:28.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.555+0000 7f865a41c740 10 client.0 _ll_register_callbacks cb 0x7f864c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:25:28.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.555+0000 7f8655ddf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbffa64a30 0x55dbffa78e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:28.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.555+0000 7f8655ddf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbffa64a30 0x55dbffa78e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37416/0 (socket says 172.21.15.120:37416) 2024-09-19T09:25:28.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.555+0000 7f8655ddf640 1 -- 172.21.15.120:0/3351910240 learned_addr learned my addr 172.21.15.120:0/3351910240 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:28.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.555+0000 7f86565e0640 1 --2- 172.21.15.120:0/3351910240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbffa68dc0 0x55dbffae4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:28.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.555+0000 7f86555de640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbffa684a0 0x55dbffa79340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:28.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.555+0000 7f8655ddf640 1 -- 172.21.15.120:0/3351910240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbffa684a0 msgr2=0x55dbffa79340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:28.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.555+0000 7f8655ddf640 1 --2- 172.21.15.120:0/3351910240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbffa684a0 0x55dbffa79340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.555+0000 7f8655ddf640 1 -- 172.21.15.120:0/3351910240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbffa68dc0 msgr2=0x55dbffae4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:28.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.555+0000 7f8655ddf640 1 --2- 172.21.15.120:0/3351910240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbffa68dc0 0x55dbffae4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.555+0000 7f8655ddf640 1 -- 172.21.15.120:0/3351910240 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbffae5350 con 0x55dbffa64a30 2024-09-19T09:25:28.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.556+0000 7f8655ddf640 1 --2- 172.21.15.120:0/3351910240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbffa64a30 0x55dbffa78e00 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f864c031a80 tx=0x7f864c031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15a13321bdf89af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:28.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.556+0000 7f86397fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:25:28.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.556+0000 7f86397fa640 1 -- 172.21.15.120:0/3351910240 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c035070 con 0x55dbffa64a30 2024-09-19T09:25:28.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.556+0000 7f865a41c740 1 -- 172.21.15.120:0/3351910240 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f864802a960 con 0x55dbffa64a30 2024-09-19T09:25:28.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.556+0000 7f86397fa640 1 -- 172.21.15.120:0/3351910240 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f864c005040 con 0x55dbffa64a30 2024-09-19T09:25:28.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.556+0000 7f86397fa640 1 -- 172.21.15.120:0/3351910240 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f864c05f050 con 0x55dbffa64a30 2024-09-19T09:25:28.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.556+0000 7f865a41c740 10 client.22764 fetch_fsmap learned FSMap version 39 2024-09-19T09:25:28.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.556+0000 7f865a41c740 1 -- 172.21.15.120:0/3351910240 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55dbffaed8d0 con 0x55dbffa64a30 2024-09-19T09:25:28.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.556+0000 7f86397fa640 1 -- 172.21.15.120:0/3351910240 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f864802a960 con 0x55dbffa64a30 2024-09-19T09:25:28.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.556+0000 7f86397fa640 1 -- 172.21.15.120:0/3351910240 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f864c004aa0 con 0x55dbffa64a30 2024-09-19T09:25:28.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.556+0000 7f865a41c740 10 client.22764 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:25:28.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.556+0000 7f865a41c740 10 client.22764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:25:28.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.556+0000 7f865a41c740 20 client.22764 populate_metadata read hostname 'smithi120' 2024-09-19T09:25:28.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.556+0000 7f865a41c740 1 --2- 172.21.15.120:0/3351910240 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dbffa73680 0x55dbffaf0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:28.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.556+0000 7f865a41c740 4 client.22764 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:25:28.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.556+0000 7f865a41c740 1 -- 172.21.15.120:0/3351910240 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dbff9d5d60 con 0x55dbffa73680 2024-09-19T09:25:28.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.557+0000 7f86555de640 1 --2- 172.21.15.120:0/3351910240 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dbffa73680 0x55dbffaf0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:28.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.557+0000 7f86397fa640 10 client.22764 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:28.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.557+0000 7f86555de640 1 --2- 172.21.15.120:0/3351910240 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dbffa73680 0x55dbffaf0080 crc :-1 s=READY pgs=556 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=514fa2b9c14b91c5 server_cookie=4c84e6a5a072b16d in_seq=0 out_seq=0 2024-09-19T09:25:28.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.564+0000 7f86397fa640 1 -- 172.21.15.120:0/3351910240 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dbff9d5d60 con 0x55dbffa73680 2024-09-19T09:25:28.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.564+0000 7f86397fa640 10 client.22764 handle_command_reply: tid=0 2024-09-19T09:25:28.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.564+0000 7f865a41c740 1 client.22764 shutdown 2024-09-19T09:25:28.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.564+0000 7f865a41c740 1 -- 172.21.15.120:0/3351910240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbffa64a30 msgr2=0x55dbffa78e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:28.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.564+0000 7f865a41c740 1 --2- 172.21.15.120:0/3351910240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbffa64a30 0x55dbffa78e00 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f864c031a80 tx=0x7f864c031850 comp rx=0 tx=0).stop 2024-09-19T09:25:28.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.564+0000 7f865a41c740 1 -- 172.21.15.120:0/3351910240 shutdown_connections 2024-09-19T09:25:28.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.564+0000 7f865a41c740 1 --2- 172.21.15.120:0/3351910240 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55dbffa73680 0x55dbffaf0080 crc :-1 s=READY pgs=556 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.564+0000 7f865a41c740 1 --2- 172.21.15.120:0/3351910240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dbffa68dc0 0x55dbffae4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.564+0000 7f865a41c740 1 --2- 172.21.15.120:0/3351910240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbffa684a0 0x55dbffa79340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.564+0000 7f865a41c740 1 --2- 172.21.15.120:0/3351910240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbffa64a30 0x55dbffa78e00 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.564+0000 7f86397fa640 0 client.22764 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:28.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.564+0000 7f865a41c740 1 -- 172.21.15.120:0/3351910240 >> 172.21.15.120:0/3351910240 conn(0x55dbffa70910 msgr2=0x55dbffa63b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:28.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.564+0000 7f865a41c740 1 -- 172.21.15.120:0/3351910240 shutdown_connections 2024-09-19T09:25:28.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.564+0000 7f865a41c740 1 -- 172.21.15.120:0/3351910240 wait complete. 2024-09-19T09:25:28.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.564+0000 7f865a41c740 20 client.22764 trim_cache size 0 max 16384 2024-09-19T09:25:28.571 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:25:28.571 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:25:28.571 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:25:28.571 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:25:28.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.565+0000 7f8658043640 1 -- 172.21.15.120:0/2305254888 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f863c0a3240 msgr2=0x7f863c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:28.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.565+0000 7f8658043640 1 --2- 172.21.15.120:0/2305254888 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f863c0a3240 0x7f863c0a5700 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f86480098b0 tx=0x7f8648005bd0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.565+0000 7f8658043640 1 -- 172.21.15.120:0/2305254888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8650102570 msgr2=0x7f865019abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:28.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.565+0000 7f8658043640 1 --2- 172.21.15.120:0/2305254888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8650102570 0x7f865019abf0 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f864000d950 tx=0x7f864000de10 comp rx=0 tx=0).stop 2024-09-19T09:25:28.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.565+0000 7f8658043640 1 -- 172.21.15.120:0/2305254888 shutdown_connections 2024-09-19T09:25:28.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.565+0000 7f8658043640 1 --2- 172.21.15.120:0/2305254888 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f863c0a3240 0x7f863c0a5700 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.565+0000 7f8658043640 1 --2- 172.21.15.120:0/2305254888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8650102ef0 0x7f865019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.565+0000 7f8658043640 1 --2- 172.21.15.120:0/2305254888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8650102570 0x7f865019abf0 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.565+0000 7f8658043640 1 --2- 172.21.15.120:0/2305254888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8650100e20 0x7f865019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.565+0000 7f8658043640 1 -- 172.21.15.120:0/2305254888 >> 172.21.15.120:0/2305254888 conn(0x7f86500fc7d0 msgr2=0x7f865006a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:28.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.565+0000 7f8658043640 1 -- 172.21.15.120:0/2305254888 shutdown_connections 2024-09-19T09:25:28.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:28.565+0000 7f8658043640 1 -- 172.21.15.120:0/2305254888 wait complete. 2024-09-19T09:25:29.383 INFO:tasks.workunit.client.0.smithi120.stdout: 10 219694 19.94 MB/sec execute 444 sec latency 59.544 ms 2024-09-19T09:25:29.468 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:25:29.469 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:25:29.768 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:30.250 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:30 smithi120 ceph-mon[31150]: pgmap v1024: 97 pgs: 97 active+clean; 483 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 4.2 MiB/s wr, 571 op/s 2024-09-19T09:25:30.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.345+0000 7f1d3b999640 1 -- 172.21.15.120:0/1671899003 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d34102d30 msgr2=0x7f1d34103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:30.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.345+0000 7f1d3b999640 1 --2- 172.21.15.120:0/1671899003 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d34102d30 0x7f1d34103110 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009930 tx=0x7f1d3002a420 comp rx=0 tx=0).stop 2024-09-19T09:25:30.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.345+0000 7f1d3b999640 1 -- 172.21.15.120:0/1671899003 shutdown_connections 2024-09-19T09:25:30.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.345+0000 7f1d3b999640 1 --2- 172.21.15.120:0/1671899003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d341082a0 0x7f1d3410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:30.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.345+0000 7f1d3b999640 1 --2- 172.21.15.120:0/1671899003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d341036e0 0x7f1d34107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:30.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.345+0000 7f1d3b999640 1 --2- 172.21.15.120:0/1671899003 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d34102d30 0x7f1d34103110 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:30.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.345+0000 7f1d3b999640 1 -- 172.21.15.120:0/1671899003 >> 172.21.15.120:0/1671899003 conn(0x7f1d340fc7d0 msgr2=0x7f1d340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:30.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.345+0000 7f1d3b999640 1 -- 172.21.15.120:0/1671899003 shutdown_connections 2024-09-19T09:25:30.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.345+0000 7f1d3b999640 1 -- 172.21.15.120:0/1671899003 wait complete. 2024-09-19T09:25:30.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.346+0000 7f1d3b999640 1 Processor -- start 2024-09-19T09:25:30.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.346+0000 7f1d3b999640 1 -- start start 2024-09-19T09:25:30.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.346+0000 7f1d3b999640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d34102d30 0x7f1d341985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:30.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.346+0000 7f1d3b999640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d341036e0 0x7f1d34198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:30.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.347+0000 7f1d3b999640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d341082a0 0x7f1d3419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:30.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.347+0000 7f1d38f34640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d341036e0 0x7f1d34198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:30.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.347+0000 7f1d38f34640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d341036e0 0x7f1d34198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37442/0 (socket says 172.21.15.120:37442) 2024-09-19T09:25:30.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.347+0000 7f1d39735640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d34102d30 0x7f1d341985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:30.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.347+0000 7f1d38f34640 1 -- 172.21.15.120:0/1085101994 learned_addr learned my addr 172.21.15.120:0/1085101994 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:30.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.347+0000 7f1d3b999640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1d341120a0 con 0x7f1d341036e0 2024-09-19T09:25:30.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.347+0000 7f1d3b999640 1 -- 172.21.15.120:0/1085101994 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1d34111f20 con 0x7f1d341082a0 2024-09-19T09:25:30.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.347+0000 7f1d3b999640 1 -- 172.21.15.120:0/1085101994 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1d34112220 con 0x7f1d34102d30 2024-09-19T09:25:30.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.347+0000 7f1d39f36640 1 --2- 172.21.15.120:0/1085101994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d341082a0 0x7f1d3419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:30.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.347+0000 7f1d38f34640 1 -- 172.21.15.120:0/1085101994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d34102d30 msgr2=0x7f1d341985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:30.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.347+0000 7f1d38f34640 1 --2- 172.21.15.120:0/1085101994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d34102d30 0x7f1d341985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:30.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.347+0000 7f1d38f34640 1 -- 172.21.15.120:0/1085101994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d341082a0 msgr2=0x7f1d3419ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:30.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.347+0000 7f1d38f34640 1 --2- 172.21.15.120:0/1085101994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d341082a0 0x7f1d3419ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:30.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.347+0000 7f1d38f34640 1 -- 172.21.15.120:0/1085101994 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3419d590 con 0x7f1d341036e0 2024-09-19T09:25:30.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.347+0000 7f1d38f34640 1 --2- 172.21.15.120:0/1085101994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d341036e0 0x7f1d34198b20 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f1d2400e9c0 tx=0x7f1d2400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:30.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.347+0000 7f1d2a7fc640 1 -- 172.21.15.120:0/1085101994 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2400ccc0 con 0x7f1d341036e0 2024-09-19T09:25:30.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.347+0000 7f1d2a7fc640 1 -- 172.21.15.120:0/1085101994 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d24005500 con 0x7f1d341036e0 2024-09-19T09:25:30.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.348+0000 7f1d2a7fc640 1 -- 172.21.15.120:0/1085101994 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d24010690 con 0x7f1d341036e0 2024-09-19T09:25:30.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.348+0000 7f1d3b999640 1 -- 172.21.15.120:0/1085101994 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d3419d880 con 0x7f1d341036e0 2024-09-19T09:25:30.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.348+0000 7f1d3b999640 1 -- 172.21.15.120:0/1085101994 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d341a5160 con 0x7f1d341036e0 2024-09-19T09:25:30.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.349+0000 7f1d3b999640 1 -- 172.21.15.120:0/1085101994 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d3406a850 con 0x7f1d341036e0 2024-09-19T09:25:30.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.350+0000 7f1d2a7fc640 1 -- 172.21.15.120:0/1085101994 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f1d24004e80 con 0x7f1d341036e0 2024-09-19T09:25:30.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.351+0000 7f1d2a7fc640 1 --2- 172.21.15.120:0/1085101994 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1d200a3240 0x7f1d200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:30.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.351+0000 7f1d39735640 1 --2- 172.21.15.120:0/1085101994 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1d200a3240 0x7f1d200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:30.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.351+0000 7f1d2a7fc640 1 -- 172.21.15.120:0/1085101994 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f1d240cfce0 con 0x7f1d341036e0 2024-09-19T09:25:30.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.352+0000 7f1d39735640 1 --2- 172.21.15.120:0/1085101994 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1d200a3240 0x7f1d200a5700 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009900 tx=0x7f1d300058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:30.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.357+0000 7f1d2a7fc640 1 -- 172.21.15.120:0/1085101994 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1d2409ca00 con 0x7f1d341036e0 2024-09-19T09:25:30.383 INFO:tasks.workunit.client.0.smithi120.stdout: 10 220202 19.95 MB/sec execute 445 sec latency 54.973 ms 2024-09-19T09:25:30.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:30 smithi196 ceph-mon[35150]: pgmap v1024: 97 pgs: 97 active+clean; 483 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 4.2 MiB/s wr, 571 op/s 2024-09-19T09:25:30.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:30 smithi153 ceph-mon[35071]: pgmap v1024: 97 pgs: 97 active+clean; 483 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 4.2 MiB/s wr, 571 op/s 2024-09-19T09:25:30.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.610+0000 7f1d3b999640 1 -- 172.21.15.120:0/1085101994 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d3410a2b0 con 0x7f1d341036e0 2024-09-19T09:25:30.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.611+0000 7f1d2a7fc640 1 -- 172.21.15.120:0/1085101994 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11988 (secure 0 0 0) 0x7f1d2409c300 con 0x7f1d341036e0 2024-09-19T09:25:30.614 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:25:30.614 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":39,"btime":"2024-09-19T09:18:53:120858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:18:53.070639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:25:30.614 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 39 2024-09-19T09:25:30.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.614+0000 7f1d3b999640 1 -- 172.21.15.120:0/1085101994 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1d200a3240 msgr2=0x7f1d200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:30.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.614+0000 7f1d3b999640 1 --2- 172.21.15.120:0/1085101994 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1d200a3240 0x7f1d200a5700 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009900 tx=0x7f1d300058d0 comp rx=0 tx=0).stop 2024-09-19T09:25:30.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.614+0000 7f1d3b999640 1 -- 172.21.15.120:0/1085101994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d341036e0 msgr2=0x7f1d34198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:30.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.614+0000 7f1d3b999640 1 --2- 172.21.15.120:0/1085101994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d341036e0 0x7f1d34198b20 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f1d2400e9c0 tx=0x7f1d2400ee80 comp rx=0 tx=0).stop 2024-09-19T09:25:30.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.614+0000 7f1d3b999640 1 -- 172.21.15.120:0/1085101994 shutdown_connections 2024-09-19T09:25:30.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.614+0000 7f1d3b999640 1 --2- 172.21.15.120:0/1085101994 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1d200a3240 0x7f1d200a5700 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:30.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.614+0000 7f1d3b999640 1 --2- 172.21.15.120:0/1085101994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d341082a0 0x7f1d3419ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:30.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.614+0000 7f1d3b999640 1 --2- 172.21.15.120:0/1085101994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d341036e0 0x7f1d34198b20 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:30.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.614+0000 7f1d3b999640 1 --2- 172.21.15.120:0/1085101994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d34102d30 0x7f1d341985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:30.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.614+0000 7f1d3b999640 1 -- 172.21.15.120:0/1085101994 >> 172.21.15.120:0/1085101994 conn(0x7f1d340fc7d0 msgr2=0x7f1d34105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:30.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.614+0000 7f1d3b999640 1 -- 172.21.15.120:0/1085101994 shutdown_connections 2024-09-19T09:25:30.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:30.614+0000 7f1d3b999640 1 -- 172.21.15.120:0/1085101994 wait complete. 2024-09-19T09:25:31.384 INFO:tasks.workunit.client.0.smithi120.stdout: 10 220819 19.97 MB/sec execute 446 sec latency 71.713 ms 2024-09-19T09:25:31.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:31 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1085101994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:25:31.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:31 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1085101994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:25:31.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:31 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1085101994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:25:31.638 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:25:32.041 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:32.354 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:32 smithi120 ceph-mon[31150]: pgmap v1025: 97 pgs: 97 active+clean; 490 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.1 MiB/s wr, 707 op/s 2024-09-19T09:25:32.383 INFO:tasks.workunit.client.0.smithi120.stdout: 10 221236 19.97 MB/sec execute 447 sec latency 261.992 ms 2024-09-19T09:25:32.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:32 smithi196 ceph-mon[35150]: pgmap v1025: 97 pgs: 97 active+clean; 490 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.1 MiB/s wr, 707 op/s 2024-09-19T09:25:32.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:32 smithi153 ceph-mon[35071]: pgmap v1025: 97 pgs: 97 active+clean; 490 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.1 MiB/s wr, 707 op/s 2024-09-19T09:25:32.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.557+0000 7f43da78a640 1 -- 172.21.15.120:0/1241102302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43d4100e20 msgr2=0x7f43d4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:32.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.557+0000 7f43da78a640 1 --2- 172.21.15.120:0/1241102302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43d4100e20 0x7f43d4101200 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f43c4009900 tx=0x7f43c402a480 comp rx=0 tx=0).stop 2024-09-19T09:25:32.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.557+0000 7f43da78a640 1 -- 172.21.15.120:0/1241102302 shutdown_connections 2024-09-19T09:25:32.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.557+0000 7f43da78a640 1 --2- 172.21.15.120:0/1241102302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43d4102ef0 0x7f43d410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.557+0000 7f43da78a640 1 --2- 172.21.15.120:0/1241102302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43d4102570 0x7f43d41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.557+0000 7f43da78a640 1 --2- 172.21.15.120:0/1241102302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43d4100e20 0x7f43d4101200 unknown :-1 s=CLOSED pgs=2320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.557+0000 7f43da78a640 1 -- 172.21.15.120:0/1241102302 >> 172.21.15.120:0/1241102302 conn(0x7f43d40fc7d0 msgr2=0x7f43d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:32.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.557+0000 7f43da78a640 1 -- 172.21.15.120:0/1241102302 shutdown_connections 2024-09-19T09:25:32.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.557+0000 7f43da78a640 1 -- 172.21.15.120:0/1241102302 wait complete. 2024-09-19T09:25:32.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.558+0000 7f43da78a640 1 Processor -- start 2024-09-19T09:25:32.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.558+0000 7f43da78a640 1 -- start start 2024-09-19T09:25:32.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.558+0000 7f43da78a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43d4100e20 0x7f43d4197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:32.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.558+0000 7f43da78a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43d4102570 0x7f43d4197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:32.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.558+0000 7f43da78a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43d4102ef0 0x7f43d419bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:32.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.558+0000 7f43da78a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f43d4110f80 con 0x7f43d4100e20 2024-09-19T09:25:32.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.558+0000 7f43da78a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f43d4110e00 con 0x7f43d4102ef0 2024-09-19T09:25:32.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.558+0000 7f43da78a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43d4111100 con 0x7f43d4102570 2024-09-19T09:25:32.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.559+0000 7f43d3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43d4100e20 0x7f43d4197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:32.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.559+0000 7f43d3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43d4100e20 0x7f43d4197410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35622/0 (socket says 172.21.15.120:35622) 2024-09-19T09:25:32.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.559+0000 7f43d3fff640 1 -- 172.21.15.120:0/1698477424 learned_addr learned my addr 172.21.15.120:0/1698477424 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:32.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.559+0000 7f43d37fe640 1 --2- 172.21.15.120:0/1698477424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43d4102570 0x7f43d4197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:32.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.559+0000 7f43d8d27640 1 --2- 172.21.15.120:0/1698477424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43d4102ef0 0x7f43d419bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:32.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.559+0000 7f43d8d27640 1 -- 172.21.15.120:0/1698477424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43d4102570 msgr2=0x7f43d4197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:32.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.559+0000 7f43d8d27640 1 --2- 172.21.15.120:0/1698477424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43d4102570 0x7f43d4197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.559+0000 7f43d8d27640 1 -- 172.21.15.120:0/1698477424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43d4100e20 msgr2=0x7f43d4197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:32.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.559+0000 7f43d8d27640 1 --2- 172.21.15.120:0/1698477424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43d4100e20 0x7f43d4197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.559+0000 7f43d8d27640 1 -- 172.21.15.120:0/1698477424 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d419c3c0 con 0x7f43d4102ef0 2024-09-19T09:25:32.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.559+0000 7f43d3fff640 1 --2- 172.21.15.120:0/1698477424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43d4100e20 0x7f43d4197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:25:32.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.559+0000 7f43d8d27640 1 --2- 172.21.15.120:0/1698477424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43d4102ef0 0x7f43d419bce0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009fd0 tx=0x7f43cc00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:32.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.559+0000 7f43d17fa640 1 -- 172.21.15.120:0/1698477424 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc00ce20 con 0x7f43d4102ef0 2024-09-19T09:25:32.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.559+0000 7f43da78a640 1 -- 172.21.15.120:0/1698477424 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d419c6b0 con 0x7f43d4102ef0 2024-09-19T09:25:32.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.559+0000 7f43d17fa640 1 -- 172.21.15.120:0/1698477424 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43cc005020 con 0x7f43d4102ef0 2024-09-19T09:25:32.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.559+0000 7f43d17fa640 1 -- 172.21.15.120:0/1698477424 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc010720 con 0x7f43d4102ef0 2024-09-19T09:25:32.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.559+0000 7f43da78a640 1 -- 172.21.15.120:0/1698477424 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d41a3f90 con 0x7f43d4102ef0 2024-09-19T09:25:32.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.562+0000 7f43dcb63740 1 Processor -- start 2024-09-19T09:25:32.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.562+0000 7f43d17fa640 1 -- 172.21.15.120:0/1698477424 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f43cc005540 con 0x7f43d4102ef0 2024-09-19T09:25:32.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.562+0000 7f43dcb63740 1 -- start start 2024-09-19T09:25:32.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.562+0000 7f43dcb63740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e409b3b40 0x556e409b2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:32.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.562+0000 7f43dcb63740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e409b25d0 0x556e409b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:32.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.562+0000 7f43dcb63740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e409bd370 0x556e409bf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:32.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.562+0000 7f43dcb63740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556e409182f0 con 0x556e409b25d0 2024-09-19T09:25:32.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.562+0000 7f43dcb63740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556e408bfcf0 con 0x556e409b3b40 2024-09-19T09:25:32.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.562+0000 7f43dcb63740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556e409142b0 con 0x556e409bd370 2024-09-19T09:25:32.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.562+0000 7f43d3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e409b25d0 0x556e409b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:32.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.562+0000 7f43d3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e409b25d0 0x556e409b2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35624/0 (socket says 172.21.15.120:35624) 2024-09-19T09:25:32.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.562+0000 7f43d3fff640 1 -- 172.21.15.120:0/561205438 learned_addr learned my addr 172.21.15.120:0/561205438 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:32.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.562+0000 7f43d37fe640 1 --2- 172.21.15.120:0/561205438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e409b3b40 0x556e409b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:32.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.563+0000 7f43d37fe640 1 --2- 172.21.15.120:0/561205438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e409bd370 0x556e409bf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:32.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.563+0000 7f43d3fff640 1 -- 172.21.15.120:0/561205438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e409bd370 msgr2=0x556e409bf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:32.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.563+0000 7f43d3fff640 1 --2- 172.21.15.120:0/561205438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e409bd370 0x556e409bf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.563+0000 7f43d3fff640 1 -- 172.21.15.120:0/561205438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e409b3b40 msgr2=0x556e409b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:32.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.563+0000 7f43d3fff640 1 --2- 172.21.15.120:0/561205438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e409b3b40 0x556e409b2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.563+0000 7f43d3fff640 1 -- 172.21.15.120:0/561205438 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e408a43a0 con 0x556e409b25d0 2024-09-19T09:25:32.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.563+0000 7f43d17fa640 1 --2- 172.21.15.120:0/1698477424 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43bc0a3240 0x7f43bc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:32.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.563+0000 7f43d3fff640 1 --2- 172.21.15.120:0/561205438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e409b25d0 0x556e409b2a10 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f43c4009a20 tx=0x7f43c4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08e32de4d2658b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:32.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.563+0000 7f43d37fe640 1 --2- 172.21.15.120:0/1698477424 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43bc0a3240 0x7f43bc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:32.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.563+0000 7f43d17fa640 1 -- 172.21.15.120:0/1698477424 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f43cc0d0020 con 0x7f43d4102ef0 2024-09-19T09:25:32.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.563+0000 7f43ba7fc640 1 -- 172.21.15.120:0/561205438 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4040070 con 0x556e409b25d0 2024-09-19T09:25:32.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.563+0000 7f43ba7fc640 1 -- 172.21.15.120:0/561205438 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43c4003ad0 con 0x556e409b25d0 2024-09-19T09:25:32.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.563+0000 7f43ba7fc640 1 -- 172.21.15.120:0/561205438 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4035040 con 0x556e409b25d0 2024-09-19T09:25:32.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.564+0000 7f43dcb63740 1 -- 172.21.15.120:0/561205438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e409b25d0 msgr2=0x556e409b2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:32.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.564+0000 7f43d37fe640 1 --2- 172.21.15.120:0/1698477424 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43bc0a3240 0x7f43bc0a5700 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x556e409b3070 tx=0x7f43c8004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:32.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.564+0000 7f43dcb63740 1 --2- 172.21.15.120:0/561205438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e409b25d0 0x556e409b2a10 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f43c4009a20 tx=0x7f43c4005e80 comp rx=0 tx=0).stop 2024-09-19T09:25:32.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.564+0000 7f43dcb63740 1 -- 172.21.15.120:0/561205438 shutdown_connections 2024-09-19T09:25:32.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.564+0000 7f43dcb63740 1 --2- 172.21.15.120:0/561205438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e409bd370 0x556e409bf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.564+0000 7f43dcb63740 1 --2- 172.21.15.120:0/561205438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e409b25d0 0x556e409b2a10 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.564+0000 7f43dcb63740 1 --2- 172.21.15.120:0/561205438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e409b3b40 0x556e409b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.564+0000 7f43dcb63740 1 -- 172.21.15.120:0/561205438 >> 172.21.15.120:0/561205438 conn(0x556e4097b310 msgr2=0x556e409aec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:32.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.564+0000 7f43dcb63740 1 -- 172.21.15.120:0/561205438 shutdown_connections 2024-09-19T09:25:32.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.564+0000 7f43dcb63740 1 -- 172.21.15.120:0/561205438 wait complete. 2024-09-19T09:25:32.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.564+0000 7f43dcb63740 1 -- start start 2024-09-19T09:25:32.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.565+0000 7f43dcb63740 1 Processor -- start 2024-09-19T09:25:32.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.565+0000 7f43dcb63740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e409b25d0 0x556e409bcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:32.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.565+0000 7f43dcb63740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e409b3b40 0x556e409ba0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:32.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.565+0000 7f43dcb63740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e409bd370 0x556e409ba7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:32.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.565+0000 7f43dcb63740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556e409c2220 con 0x556e409bd370 2024-09-19T09:25:32.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.565+0000 7f43dcb63740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556e408d3060 con 0x556e409b3b40 2024-09-19T09:25:32.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.565+0000 7f43dcb63740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556e409c23a0 con 0x556e409b25d0 2024-09-19T09:25:32.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.565+0000 7f43dcb63740 10 client.0 _ll_register_callbacks cb 0x556e406268c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:25:32.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.565+0000 7f43d8d27640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e409bd370 0x556e409ba7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:32.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.565+0000 7f43d37fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e409b3b40 0x556e409ba0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:32.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.565+0000 7f43d3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e409b25d0 0x556e409bcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:32.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.565+0000 7f43d37fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e409b3b40 0x556e409ba0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:36696/0 (socket says 172.21.15.120:36696) 2024-09-19T09:25:32.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.565+0000 7f43d8d27640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e409bd370 0x556e409ba7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35626/0 (socket says 172.21.15.120:35626) 2024-09-19T09:25:32.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.565+0000 7f43d37fe640 1 -- 172.21.15.120:0/2931755612 learned_addr learned my addr 172.21.15.120:0/2931755612 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:32.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.565+0000 7f43d37fe640 1 -- 172.21.15.120:0/2931755612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e409b25d0 msgr2=0x556e409bcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:32.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.565+0000 7f43d37fe640 1 --2- 172.21.15.120:0/2931755612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e409b25d0 0x556e409bcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.565+0000 7f43d37fe640 1 -- 172.21.15.120:0/2931755612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e409bd370 msgr2=0x556e409ba7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:32.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.565+0000 7f43d37fe640 1 --2- 172.21.15.120:0/2931755612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e409bd370 0x556e409ba7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.565+0000 7f43d37fe640 1 -- 172.21.15.120:0/2931755612 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e406893c0 con 0x556e409b3b40 2024-09-19T09:25:32.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.566+0000 7f43d8d27640 1 --2- 172.21.15.120:0/2931755612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e409bd370 0x556e409ba7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:25:32.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.566+0000 7f43d37fe640 1 --2- 172.21.15.120:0/2931755612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e409b3b40 0x556e409ba0a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f43c8011a80 tx=0x7f43c8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8349faabceaa8a49 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:32.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.566+0000 7f43b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:25:32.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.566+0000 7f43b37fe640 1 -- 172.21.15.120:0/2931755612 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c802e060 con 0x556e409b3b40 2024-09-19T09:25:32.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.566+0000 7f43dcb63740 1 -- 172.21.15.120:0/2931755612 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43c4003ad0 con 0x556e409b3b40 2024-09-19T09:25:32.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.566+0000 7f43b37fe640 1 -- 172.21.15.120:0/2931755612 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f43c8009040 con 0x556e409b3b40 2024-09-19T09:25:32.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.566+0000 7f43b37fe640 1 -- 172.21.15.120:0/2931755612 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f43c80219b0 con 0x556e409b3b40 2024-09-19T09:25:32.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.566+0000 7f43dcb63740 10 client.31460 fetch_fsmap learned FSMap version 39 2024-09-19T09:25:32.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.566+0000 7f43dcb63740 1 -- 172.21.15.120:0/2931755612 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556e409cdd20 con 0x556e409b3b40 2024-09-19T09:25:32.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.566+0000 7f43b37fe640 1 -- 172.21.15.120:0/2931755612 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43c4003ad0 con 0x556e409b3b40 2024-09-19T09:25:32.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.566+0000 7f43b37fe640 1 -- 172.21.15.120:0/2931755612 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f43c8033070 con 0x556e409b3b40 2024-09-19T09:25:32.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.567+0000 7f43dcb63740 10 client.31460 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:25:32.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.567+0000 7f43dcb63740 10 client.31460 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:25:32.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.567+0000 7f43dcb63740 20 client.31460 populate_metadata read hostname 'smithi120' 2024-09-19T09:25:32.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.567+0000 7f43dcb63740 1 --2- 172.21.15.120:0/2931755612 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x556e409cf230 0x556e409d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:32.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.567+0000 7f43dcb63740 4 client.31460 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:25:32.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.567+0000 7f43dcb63740 1 -- 172.21.15.120:0/2931755612 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e40925860 con 0x556e409cf230 2024-09-19T09:25:32.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.567+0000 7f43d3fff640 1 --2- 172.21.15.120:0/2931755612 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x556e409cf230 0x556e409d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:32.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.567+0000 7f43d3fff640 1 --2- 172.21.15.120:0/2931755612 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x556e409cf230 0x556e409d1650 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9c71e712022d6d38 server_cookie=2da5286235688472 in_seq=0 out_seq=0 2024-09-19T09:25:32.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.567+0000 7f43b37fe640 10 client.31460 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:25:32.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.568+0000 7f43b37fe640 1 -- 172.21.15.120:0/2931755612 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556e40925860 con 0x556e409cf230 2024-09-19T09:25:32.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.568+0000 7f43b37fe640 10 client.31460 handle_command_reply: tid=0 2024-09-19T09:25:32.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.568+0000 7f43dcb63740 1 client.31460 shutdown 2024-09-19T09:25:32.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.568+0000 7f43dcb63740 1 -- 172.21.15.120:0/2931755612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e409b3b40 msgr2=0x556e409ba0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:32.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.568+0000 7f43dcb63740 1 --2- 172.21.15.120:0/2931755612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e409b3b40 0x556e409ba0a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f43c8011a80 tx=0x7f43c8005f90 comp rx=0 tx=0).stop 2024-09-19T09:25:32.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.568+0000 7f43dcb63740 1 -- 172.21.15.120:0/2931755612 shutdown_connections 2024-09-19T09:25:32.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.568+0000 7f43dcb63740 1 --2- 172.21.15.120:0/2931755612 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x556e409cf230 0x556e409d1650 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.568+0000 7f43dcb63740 1 --2- 172.21.15.120:0/2931755612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e409bd370 0x556e409ba7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.568+0000 7f43dcb63740 1 --2- 172.21.15.120:0/2931755612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e409b3b40 0x556e409ba0a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.568+0000 7f43b37fe640 0 client.31460 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:25:32.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.568+0000 7f43dcb63740 1 --2- 172.21.15.120:0/2931755612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e409b25d0 0x556e409bcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.568+0000 7f43dcb63740 1 -- 172.21.15.120:0/2931755612 >> 172.21.15.120:0/2931755612 conn(0x556e4097b310 msgr2=0x556e409b4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:32.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.568+0000 7f43dcb63740 1 -- 172.21.15.120:0/2931755612 shutdown_connections 2024-09-19T09:25:32.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.568+0000 7f43dcb63740 1 -- 172.21.15.120:0/2931755612 wait complete. 2024-09-19T09:25:32.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.569+0000 7f43dcb63740 20 client.31460 trim_cache size 0 max 16384 2024-09-19T09:25:32.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.583+0000 7f43dcb63740 1 Processor -- start 2024-09-19T09:25:32.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.583+0000 7f43dcb63740 1 -- start start 2024-09-19T09:25:32.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.583+0000 7f43dcb63740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e409b25d0 0x556e409b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:32.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.583+0000 7f43dcb63740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e409b3b40 0x556e409cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:32.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.583+0000 7f43dcb63740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e409bd370 0x556e409b9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:32.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.583+0000 7f43dcb63740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f43d41ab380 con 0x556e409bd370 2024-09-19T09:25:32.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.583+0000 7f43dcb63740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f43d41acec0 con 0x556e409b25d0 2024-09-19T09:25:32.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.583+0000 7f43dcb63740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43d41aba50 con 0x556e409b3b40 2024-09-19T09:25:32.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.583+0000 7f43d8d27640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e409bd370 0x556e409b9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:32.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.583+0000 7f43d8d27640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e409bd370 0x556e409b9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35632/0 (socket says 172.21.15.120:35632) 2024-09-19T09:25:32.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.583+0000 7f43d8d27640 1 -- 172.21.15.120:0/2096502485 learned_addr learned my addr 172.21.15.120:0/2096502485 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:32.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.583+0000 7f43d3fff640 1 --2- 172.21.15.120:0/2096502485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e409b25d0 0x556e409b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:32.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.583+0000 7f43d37fe640 1 --2- 172.21.15.120:0/2096502485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e409b3b40 0x556e409cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:32.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.584+0000 7f43d8d27640 1 -- 172.21.15.120:0/2096502485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e409b3b40 msgr2=0x556e409cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:32.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.584+0000 7f43d8d27640 1 --2- 172.21.15.120:0/2096502485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e409b3b40 0x556e409cb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.584+0000 7f43d8d27640 1 -- 172.21.15.120:0/2096502485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e409b25d0 msgr2=0x556e409b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:32.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.584+0000 7f43d8d27640 1 --2- 172.21.15.120:0/2096502485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e409b25d0 0x556e409b29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.584+0000 7f43d8d27640 1 -- 172.21.15.120:0/2096502485 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e409ae4f0 con 0x556e409bd370 2024-09-19T09:25:32.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.584+0000 7f43d8d27640 1 --2- 172.21.15.120:0/2096502485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e409bd370 0x556e409b9960 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0cdb20 tx=0x7f43cc0cd9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37996a6193f4345 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:32.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.584+0000 7f43b9ffb640 1 -- 172.21.15.120:0/2096502485 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc0b4070 con 0x556e409bd370 2024-09-19T09:25:32.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.584+0000 7f43b9ffb640 1 -- 172.21.15.120:0/2096502485 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43cc0a52b0 con 0x556e409bd370 2024-09-19T09:25:32.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.584+0000 7f43b9ffb640 1 -- 172.21.15.120:0/2096502485 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc0b0dc0 con 0x556e409bd370 2024-09-19T09:25:32.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.585+0000 7f43dcb63740 1 -- 172.21.15.120:0/2096502485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e409bd370 msgr2=0x556e409b9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:32.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.585+0000 7f43dcb63740 1 --2- 172.21.15.120:0/2096502485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e409bd370 0x556e409b9960 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0cdb20 tx=0x7f43cc0cd9f0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.585+0000 7f43dcb63740 1 -- 172.21.15.120:0/2096502485 shutdown_connections 2024-09-19T09:25:32.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.585+0000 7f43dcb63740 1 --2- 172.21.15.120:0/2096502485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e409bd370 0x556e409b9960 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.585+0000 7f43dcb63740 1 --2- 172.21.15.120:0/2096502485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e409b3b40 0x556e409cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.585+0000 7f43dcb63740 1 --2- 172.21.15.120:0/2096502485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e409b25d0 0x556e409b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.585+0000 7f43dcb63740 1 -- 172.21.15.120:0/2096502485 >> 172.21.15.120:0/2096502485 conn(0x556e4097b310 msgr2=0x556e409b8970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:32.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.585+0000 7f43dcb63740 1 -- 172.21.15.120:0/2096502485 shutdown_connections 2024-09-19T09:25:32.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.585+0000 7f43dcb63740 1 -- 172.21.15.120:0/2096502485 wait complete. 2024-09-19T09:25:32.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.585+0000 7f43dcb63740 1 -- start start 2024-09-19T09:25:32.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.586+0000 7f43dcb63740 1 Processor -- start 2024-09-19T09:25:32.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.586+0000 7f43dcb63740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e409b25d0 0x556e409b6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:32.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.586+0000 7f43dcb63740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e409b3b40 0x556e409b6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:32.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.586+0000 7f43dcb63740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e409bd370 0x556e409b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:32.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.586+0000 7f43dcb63740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f43d41ac7f0 con 0x556e409b25d0 2024-09-19T09:25:32.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.586+0000 7f43dcb63740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f43d41ac120 con 0x556e409bd370 2024-09-19T09:25:32.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.586+0000 7f43dcb63740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43d41a9360 con 0x556e409b3b40 2024-09-19T09:25:32.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.586+0000 7f43dcb63740 10 client.0 _ll_register_callbacks cb 0x7f43c8003f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:25:32.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.586+0000 7f43d3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e409b25d0 0x556e409b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:32.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.586+0000 7f43d3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e409b25d0 0x556e409b6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35646/0 (socket says 172.21.15.120:35646) 2024-09-19T09:25:32.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.586+0000 7f43d3fff640 1 -- 172.21.15.120:0/2994229689 learned_addr learned my addr 172.21.15.120:0/2994229689 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:32.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.586+0000 7f43d37fe640 1 --2- 172.21.15.120:0/2994229689 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e409b3b40 0x556e409b6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:32.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.586+0000 7f43d8d27640 1 --2- 172.21.15.120:0/2994229689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e409bd370 0x556e409b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:32.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.586+0000 7f43d3fff640 1 -- 172.21.15.120:0/2994229689 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e409b3b40 msgr2=0x556e409b6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:32.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.586+0000 7f43d3fff640 1 --2- 172.21.15.120:0/2994229689 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e409b3b40 0x556e409b6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.586+0000 7f43d3fff640 1 -- 172.21.15.120:0/2994229689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e409bd370 msgr2=0x556e409b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:32.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.586+0000 7f43d3fff640 1 --2- 172.21.15.120:0/2994229689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e409bd370 0x556e409b6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.586+0000 7f43d3fff640 1 -- 172.21.15.120:0/2994229689 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e409c66d0 con 0x556e409b25d0 2024-09-19T09:25:32.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.587+0000 7f43d3fff640 1 --2- 172.21.15.120:0/2994229689 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e409b25d0 0x556e409b6350 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f43c4006010 tx=0x7f43c4031860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57eaae618113ad76 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:32.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.587+0000 7f43b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:25:32.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.587+0000 7f43b37fe640 1 -- 172.21.15.120:0/2994229689 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4035040 con 0x556e409b25d0 2024-09-19T09:25:32.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.587+0000 7f43b37fe640 1 -- 172.21.15.120:0/2994229689 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f43c4031d80 con 0x556e409b25d0 2024-09-19T09:25:32.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.587+0000 7f43dcb63740 1 -- 172.21.15.120:0/2994229689 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43cc0a52b0 con 0x556e409b25d0 2024-09-19T09:25:32.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.587+0000 7f43b37fe640 1 -- 172.21.15.120:0/2994229689 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f43c4040040 con 0x556e409b25d0 2024-09-19T09:25:32.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.587+0000 7f43dcb63740 10 client.22788 fetch_fsmap learned FSMap version 39 2024-09-19T09:25:32.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.587+0000 7f43dcb63740 1 -- 172.21.15.120:0/2994229689 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556e409c79b0 con 0x556e409b25d0 2024-09-19T09:25:32.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.587+0000 7f43b37fe640 1 -- 172.21.15.120:0/2994229689 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43cc0a52b0 con 0x556e409b25d0 2024-09-19T09:25:32.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.587+0000 7f43b37fe640 1 -- 172.21.15.120:0/2994229689 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f43c4057030 con 0x556e409b25d0 2024-09-19T09:25:32.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.587+0000 7f43dcb63740 10 client.22788 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:25:32.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.587+0000 7f43dcb63740 10 client.22788 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:25:32.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.587+0000 7f43dcb63740 20 client.22788 populate_metadata read hostname 'smithi120' 2024-09-19T09:25:32.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.587+0000 7f43dcb63740 1 --2- 172.21.15.120:0/2994229689 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x556e409cf230 0x556e409c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:32.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.587+0000 7f43dcb63740 4 client.22788 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:25:32.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.587+0000 7f43dcb63740 1 -- 172.21.15.120:0/2994229689 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e4091fd70 con 0x556e409cf230 2024-09-19T09:25:32.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.588+0000 7f43d37fe640 1 --2- 172.21.15.120:0/2994229689 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x556e409cf230 0x556e409c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:32.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.588+0000 7f43b37fe640 10 client.22788 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:25:32.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.588+0000 7f43d37fe640 1 --2- 172.21.15.120:0/2994229689 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x556e409cf230 0x556e409c7c40 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5d283f1785eca972 server_cookie=89641df3f246dd01 in_seq=0 out_seq=0 2024-09-19T09:25:32.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.588+0000 7f43b37fe640 1 -- 172.21.15.120:0/2994229689 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e4091fd70 con 0x556e409cf230 2024-09-19T09:25:32.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.588+0000 7f43b37fe640 10 client.22788 handle_command_reply: tid=0 2024-09-19T09:25:32.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.588+0000 7f43dcb63740 1 client.22788 shutdown 2024-09-19T09:25:32.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.589+0000 7f43dcb63740 1 -- 172.21.15.120:0/2994229689 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e409b25d0 msgr2=0x556e409b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:32.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.589+0000 7f43dcb63740 1 --2- 172.21.15.120:0/2994229689 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e409b25d0 0x556e409b6350 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f43c4006010 tx=0x7f43c4031860 comp rx=0 tx=0).stop 2024-09-19T09:25:32.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.589+0000 7f43dcb63740 1 -- 172.21.15.120:0/2994229689 shutdown_connections 2024-09-19T09:25:32.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.589+0000 7f43dcb63740 1 --2- 172.21.15.120:0/2994229689 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x556e409cf230 0x556e409c7c40 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.589+0000 7f43dcb63740 1 --2- 172.21.15.120:0/2994229689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e409bd370 0x556e409b6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.589+0000 7f43dcb63740 1 --2- 172.21.15.120:0/2994229689 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e409b3b40 0x556e409b6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.589+0000 7f43b37fe640 0 client.22788 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:25:32.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.589+0000 7f43dcb63740 1 --2- 172.21.15.120:0/2994229689 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e409b25d0 0x556e409b6350 unknown :-1 s=CLOSED pgs=2323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.589+0000 7f43dcb63740 1 -- 172.21.15.120:0/2994229689 >> 172.21.15.120:0/2994229689 conn(0x556e4097b310 msgr2=0x556e409b9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:32.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.589+0000 7f43dcb63740 1 -- 172.21.15.120:0/2994229689 shutdown_connections 2024-09-19T09:25:32.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.589+0000 7f43dcb63740 1 -- 172.21.15.120:0/2994229689 wait complete. 2024-09-19T09:25:32.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.589+0000 7f43dcb63740 20 client.22788 trim_cache size 0 max 16384 2024-09-19T09:25:32.603 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:25:32.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.589+0000 7f43da78a640 1 -- 172.21.15.120:0/1698477424 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43bc0a3240 msgr2=0x7f43bc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:32.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.589+0000 7f43da78a640 1 --2- 172.21.15.120:0/1698477424 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43bc0a3240 0x7f43bc0a5700 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x556e409b3070 tx=0x7f43c8004730 comp rx=0 tx=0).stop 2024-09-19T09:25:32.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.589+0000 7f43da78a640 1 -- 172.21.15.120:0/1698477424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43d4102ef0 msgr2=0x7f43d419bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:32.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.589+0000 7f43da78a640 1 --2- 172.21.15.120:0/1698477424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43d4102ef0 0x7f43d419bce0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009fd0 tx=0x7f43cc00ef10 comp rx=0 tx=0).stop 2024-09-19T09:25:32.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.590+0000 7f43da78a640 1 -- 172.21.15.120:0/1698477424 shutdown_connections 2024-09-19T09:25:32.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.590+0000 7f43da78a640 1 --2- 172.21.15.120:0/1698477424 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f43bc0a3240 0x7f43bc0a5700 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.590+0000 7f43da78a640 1 --2- 172.21.15.120:0/1698477424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43d4102ef0 0x7f43d419bce0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.590+0000 7f43da78a640 1 --2- 172.21.15.120:0/1698477424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43d4102570 0x7f43d4197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.590+0000 7f43da78a640 1 --2- 172.21.15.120:0/1698477424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43d4100e20 0x7f43d4197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:32.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.590+0000 7f43da78a640 1 -- 172.21.15.120:0/1698477424 >> 172.21.15.120:0/1698477424 conn(0x7f43d40fc7d0 msgr2=0x7f43d40fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:32.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.590+0000 7f43da78a640 1 -- 172.21.15.120:0/1698477424 shutdown_connections 2024-09-19T09:25:32.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:32.590+0000 7f43da78a640 1 -- 172.21.15.120:0/1698477424 wait complete. 2024-09-19T09:25:33.384 INFO:tasks.workunit.client.0.smithi120.stdout: 10 221778 19.99 MB/sec execute 448 sec latency 101.743 ms 2024-09-19T09:25:33.746 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:25:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.887+0000 7fe3e7dfd640 1 -- 172.21.15.120:0/4279238080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3e0100a40 msgr2=0x7fe3e0100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.887+0000 7fe3e7dfd640 1 --2- 172.21.15.120:0/4279238080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3e0100a40 0x7fe3e0100e40 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009ca0 tx=0x7fe3d002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.887+0000 7fe3e7dfd640 1 -- 172.21.15.120:0/4279238080 shutdown_connections 2024-09-19T09:25:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.887+0000 7fe3e7dfd640 1 --2- 172.21.15.120:0/4279238080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3e01025f0 0x7fe3e0108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.887+0000 7fe3e7dfd640 1 --2- 172.21.15.120:0/4279238080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3e0101c30 0x7fe3e01020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.887+0000 7fe3e7dfd640 1 --2- 172.21.15.120:0/4279238080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3e0100a40 0x7fe3e0100e40 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.887+0000 7fe3e7dfd640 1 -- 172.21.15.120:0/4279238080 >> 172.21.15.120:0/4279238080 conn(0x7fe3e00fc1e0 msgr2=0x7fe3e00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.887+0000 7fe3e7dfd640 1 -- 172.21.15.120:0/4279238080 shutdown_connections 2024-09-19T09:25:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.887+0000 7fe3e7dfd640 1 -- 172.21.15.120:0/4279238080 wait complete. 2024-09-19T09:25:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.888+0000 7fe3e7dfd640 1 Processor -- start 2024-09-19T09:25:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.888+0000 7fe3e7dfd640 1 -- start start 2024-09-19T09:25:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.888+0000 7fe3e7dfd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3e0100a40 0x7fe3e0196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.888+0000 7fe3e7dfd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3e0101c30 0x7fe3e0197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.888+0000 7fe3e7dfd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3e01025f0 0x7fe3e019e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.888+0000 7fe3e7dfd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e010c6c0 con 0x7fe3e01025f0 2024-09-19T09:25:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.888+0000 7fe3e7dfd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e010c540 con 0x7fe3e0100a40 2024-09-19T09:25:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.888+0000 7fe3e7dfd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e010c840 con 0x7fe3e0101c30 2024-09-19T09:25:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.888+0000 7fe3e639a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3e01025f0 0x7fe3e019e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.888+0000 7fe3e5b99640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3e0100a40 0x7fe3e0196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.888+0000 7fe3e5398640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3e0101c30 0x7fe3e0197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.888+0000 7fe3e639a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3e01025f0 0x7fe3e019e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35664/0 (socket says 172.21.15.120:35664) 2024-09-19T09:25:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.888+0000 7fe3e639a640 1 -- 172.21.15.120:0/354837506 learned_addr learned my addr 172.21.15.120:0/354837506 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.889+0000 7fe3e639a640 1 -- 172.21.15.120:0/354837506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3e0101c30 msgr2=0x7fe3e0197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.889+0000 7fe3e639a640 1 --2- 172.21.15.120:0/354837506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3e0101c30 0x7fe3e0197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.889+0000 7fe3e639a640 1 -- 172.21.15.120:0/354837506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3e0100a40 msgr2=0x7fe3e0196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.889+0000 7fe3e639a640 1 --2- 172.21.15.120:0/354837506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3e0100a40 0x7fe3e0196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.889+0000 7fe3e639a640 1 -- 172.21.15.120:0/354837506 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e019e910 con 0x7fe3e01025f0 2024-09-19T09:25:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.889+0000 7fe3e639a640 1 --2- 172.21.15.120:0/354837506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3e01025f0 0x7fe3e019e270 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800e970 tx=0x7fe3d800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.889+0000 7fe3deffd640 1 -- 172.21.15.120:0/354837506 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d800ccc0 con 0x7fe3e01025f0 2024-09-19T09:25:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.889+0000 7fe3deffd640 1 -- 172.21.15.120:0/354837506 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3d800ce60 con 0x7fe3e01025f0 2024-09-19T09:25:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.889+0000 7fe3deffd640 1 -- 172.21.15.120:0/354837506 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8010490 con 0x7fe3e01025f0 2024-09-19T09:25:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.889+0000 7fe3e7dfd640 1 -- 172.21.15.120:0/354837506 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3e0069900 con 0x7fe3e01025f0 2024-09-19T09:25:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.889+0000 7fe3e7dfd640 1 -- 172.21.15.120:0/354837506 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3e0069e50 con 0x7fe3e01025f0 2024-09-19T09:25:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.892+0000 7fe3deffd640 1 -- 172.21.15.120:0/354837506 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe3d8005020 con 0x7fe3e01025f0 2024-09-19T09:25:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.893+0000 7fe3eac81740 1 Processor -- start 2024-09-19T09:25:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.893+0000 7fe3eac81740 1 -- start start 2024-09-19T09:25:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.893+0000 7fe3eac81740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aa908b9a0 0x557aa908bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.893+0000 7fe3eac81740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aa908c2e0 0x557aa9090da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.893+0000 7fe3eac81740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aa9091370 0x557aa9093830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.893+0000 7fe3eac81740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557aa8fef2f0 con 0x557aa908c2e0 2024-09-19T09:25:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.893+0000 7fe3eac81740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557aa8f97270 con 0x557aa908b9a0 2024-09-19T09:25:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.893+0000 7fe3eac81740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557aa8feb2b0 con 0x557aa9091370 2024-09-19T09:25:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.894+0000 7fe3e5398640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aa908b9a0 0x557aa908bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.894+0000 7fe3deffd640 1 --2- 172.21.15.120:0/354837506 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe3c80a3370 0x7fe3c80a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.894+0000 7fe3e5398640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aa908b9a0 0x557aa908bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:36734/0 (socket says 172.21.15.120:36734) 2024-09-19T09:25:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.894+0000 7fe3e5398640 1 -- 172.21.15.120:0/1568265222 learned_addr learned my addr 172.21.15.120:0/1568265222 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.894+0000 7fe3e5398640 1 --2- 172.21.15.120:0/1568265222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aa9091370 0x557aa9093830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.894+0000 7fe3e5b99640 1 --2- 172.21.15.120:0/1568265222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aa908c2e0 0x557aa9090da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.894+0000 7fe3e639a640 1 --2- 172.21.15.120:0/354837506 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe3c80a3370 0x7fe3c80a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.894+0000 7fe3deffd640 1 -- 172.21.15.120:0/354837506 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe3d80cfd40 con 0x7fe3e01025f0 2024-09-19T09:25:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.894+0000 7fe3e5398640 1 -- 172.21.15.120:0/1568265222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aa908b9a0 msgr2=0x557aa908bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.894+0000 7fe3e5398640 1 --2- 172.21.15.120:0/1568265222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aa908b9a0 0x557aa908bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.894+0000 7fe3e5398640 1 -- 172.21.15.120:0/1568265222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aa908c2e0 msgr2=0x557aa9090da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.894+0000 7fe3e5398640 1 --2- 172.21.15.120:0/1568265222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aa908c2e0 0x557aa9090da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.894+0000 7fe3e5398640 1 -- 172.21.15.120:0/1568265222 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557aa8f96340 con 0x557aa9091370 2024-09-19T09:25:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.894+0000 7fe3e5398640 1 --2- 172.21.15.120:0/1568265222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aa9091370 0x557aa9093830 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4011850 tx=0x7fe3d4005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34fc2b00dae6aaf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.894+0000 7fe3c7fff640 1 -- 172.21.15.120:0/1568265222 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4004030 con 0x557aa9091370 2024-09-19T09:25:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.895+0000 7fe3c7fff640 1 -- 172.21.15.120:0/1568265222 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3d40114c0 con 0x557aa9091370 2024-09-19T09:25:33.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.895+0000 7fe3e639a640 1 --2- 172.21.15.120:0/354837506 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe3c80a3370 0x7fe3c80a5830 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe3d809c2f0 tx=0x7fe3d801c590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.895+0000 7fe3c7fff640 1 -- 172.21.15.120:0/1568265222 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d400fa20 con 0x557aa9091370 2024-09-19T09:25:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.895+0000 7fe3eac81740 1 -- 172.21.15.120:0/1568265222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aa9091370 msgr2=0x557aa9093830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.895+0000 7fe3eac81740 1 --2- 172.21.15.120:0/1568265222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aa9091370 0x557aa9093830 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4011850 tx=0x7fe3d4005f90 comp rx=0 tx=0).stop 2024-09-19T09:25:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.895+0000 7fe3eac81740 1 -- 172.21.15.120:0/1568265222 shutdown_connections 2024-09-19T09:25:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.895+0000 7fe3eac81740 1 --2- 172.21.15.120:0/1568265222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aa9091370 0x557aa9093830 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.895+0000 7fe3eac81740 1 --2- 172.21.15.120:0/1568265222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aa908c2e0 0x557aa9090da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.895+0000 7fe3eac81740 1 --2- 172.21.15.120:0/1568265222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aa908b9a0 0x557aa908bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.895+0000 7fe3eac81740 1 -- 172.21.15.120:0/1568265222 >> 172.21.15.120:0/1568265222 conn(0x557aa9087010 msgr2=0x557aa9087410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.896+0000 7fe3eac81740 1 -- 172.21.15.120:0/1568265222 shutdown_connections 2024-09-19T09:25:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.896+0000 7fe3eac81740 1 -- 172.21.15.120:0/1568265222 wait complete. 2024-09-19T09:25:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.896+0000 7fe3eac81740 1 -- start start 2024-09-19T09:25:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.896+0000 7fe3eac81740 1 Processor -- start 2024-09-19T09:25:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.896+0000 7fe3eac81740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aa908b9a0 0x557aa909a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.896+0000 7fe3eac81740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aa908c2e0 0x557aa909a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.896+0000 7fe3eac81740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aa909f7b0 0x557aa90a1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.896+0000 7fe3eac81740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557aa90987f0 con 0x557aa908b9a0 2024-09-19T09:25:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.896+0000 7fe3eac81740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557aa8f78290 con 0x557aa909f7b0 2024-09-19T09:25:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.896+0000 7fe3eac81740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557aa9098970 con 0x557aa908c2e0 2024-09-19T09:25:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.896+0000 7fe3eac81740 10 client.0 _ll_register_callbacks cb 0x557aa8fa9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:25:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.897+0000 7fe3e5398640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aa908b9a0 0x557aa909a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.897+0000 7fe3e5b99640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aa908c2e0 0x557aa909a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.897+0000 7fe3e5b99640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aa908c2e0 0x557aa909a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:49948/0 (socket says 172.21.15.120:49948) 2024-09-19T09:25:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.897+0000 7fe3e5b99640 1 -- 172.21.15.120:0/1719108103 learned_addr learned my addr 172.21.15.120:0/1719108103 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.897+0000 7fe3e5398640 1 --2- 172.21.15.120:0/1719108103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aa909f7b0 0x557aa90a1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.897+0000 7fe3e5b99640 1 -- 172.21.15.120:0/1719108103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aa909f7b0 msgr2=0x557aa90a1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.897+0000 7fe3e5b99640 1 --2- 172.21.15.120:0/1719108103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aa909f7b0 0x557aa90a1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.897+0000 7fe3e5b99640 1 -- 172.21.15.120:0/1719108103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aa908b9a0 msgr2=0x557aa909a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.897+0000 7fe3e5b99640 1 --2- 172.21.15.120:0/1719108103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aa908b9a0 0x557aa909a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.897+0000 7fe3e5b99640 1 -- 172.21.15.120:0/1719108103 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557aa8d02210 con 0x557aa908c2e0 2024-09-19T09:25:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.897+0000 7fe3e5398640 1 --2- 172.21.15.120:0/1719108103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aa908b9a0 0x557aa909a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:25:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.897+0000 7fe3e5b99640 1 --2- 172.21.15.120:0/1719108103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aa908c2e0 0x557aa909a780 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0101a90 tx=0x7fe3d00319e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=898fded4e29fc138 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.897+0000 7fe3c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:25:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.897+0000 7fe3c57fa640 1 -- 172.21.15.120:0/1719108103 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d002f610 con 0x557aa908c2e0 2024-09-19T09:25:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.897+0000 7fe3c57fa640 1 -- 172.21.15.120:0/1719108103 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe3d002f7b0 con 0x557aa908c2e0 2024-09-19T09:25:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.897+0000 7fe3eac81740 1 -- 172.21.15.120:0/1719108103 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3d40114c0 con 0x557aa908c2e0 2024-09-19T09:25:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.898+0000 7fe3c57fa640 1 -- 172.21.15.120:0/1719108103 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe3d002f9b0 con 0x557aa908c2e0 2024-09-19T09:25:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.898+0000 7fe3eac81740 10 client.31339 fetch_fsmap learned FSMap version 39 2024-09-19T09:25:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.898+0000 7fe3eac81740 1 -- 172.21.15.120:0/1719108103 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557aa8dce620 con 0x557aa908c2e0 2024-09-19T09:25:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.898+0000 7fe3c57fa640 1 -- 172.21.15.120:0/1719108103 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3d40114c0 con 0x557aa908c2e0 2024-09-19T09:25:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.898+0000 7fe3c57fa640 1 -- 172.21.15.120:0/1719108103 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fe3d004e030 con 0x557aa908c2e0 2024-09-19T09:25:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.898+0000 7fe3eac81740 10 client.31339 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:25:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.898+0000 7fe3eac81740 10 client.31339 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:25:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.898+0000 7fe3eac81740 20 client.31339 populate_metadata read hostname 'smithi120' 2024-09-19T09:25:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.898+0000 7fe3eac81740 1 --2- 172.21.15.120:0/1719108103 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557aa90a6290 0x557aa90a86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.898+0000 7fe3eac81740 4 client.31339 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:25:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.898+0000 7fe3eac81740 1 -- 172.21.15.120:0/1719108103 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557aa8ffca20 con 0x557aa90a6290 2024-09-19T09:25:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.898+0000 7fe3e5398640 1 --2- 172.21.15.120:0/1719108103 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557aa90a6290 0x557aa90a86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.899+0000 7fe3e5398640 1 --2- 172.21.15.120:0/1719108103 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557aa90a6290 0x557aa90a86e0 crc :-1 s=READY pgs=557 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57c09c25ed95f598 server_cookie=e932b0a5590f4008 in_seq=0 out_seq=0 2024-09-19T09:25:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.899+0000 7fe3c57fa640 10 client.31339 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.904+0000 7fe3c57fa640 1 -- 172.21.15.120:0/1719108103 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557aa8ffca20 con 0x557aa90a6290 2024-09-19T09:25:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.904+0000 7fe3c57fa640 10 client.31339 handle_command_reply: tid=0 2024-09-19T09:25:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.904+0000 7fe3eac81740 1 client.31339 shutdown 2024-09-19T09:25:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.904+0000 7fe3eac81740 1 -- 172.21.15.120:0/1719108103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aa908c2e0 msgr2=0x557aa909a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.904+0000 7fe3eac81740 1 --2- 172.21.15.120:0/1719108103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aa908c2e0 0x557aa909a780 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0101a90 tx=0x7fe3d00319e0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.904+0000 7fe3eac81740 1 -- 172.21.15.120:0/1719108103 shutdown_connections 2024-09-19T09:25:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.904+0000 7fe3eac81740 1 --2- 172.21.15.120:0/1719108103 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557aa90a6290 0x557aa90a86e0 crc :-1 s=READY pgs=557 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.904+0000 7fe3eac81740 1 --2- 172.21.15.120:0/1719108103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aa909f7b0 0x557aa90a1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.904+0000 7fe3eac81740 1 --2- 172.21.15.120:0/1719108103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aa908c2e0 0x557aa909a780 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.904+0000 7fe3c57fa640 0 client.31339 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.904+0000 7fe3eac81740 1 --2- 172.21.15.120:0/1719108103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aa908b9a0 0x557aa909a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.904+0000 7fe3eac81740 1 -- 172.21.15.120:0/1719108103 >> 172.21.15.120:0/1719108103 conn(0x557aa9087010 msgr2=0x557aa9088820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.904+0000 7fe3eac81740 1 -- 172.21.15.120:0/1719108103 shutdown_connections 2024-09-19T09:25:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.904+0000 7fe3eac81740 1 -- 172.21.15.120:0/1719108103 wait complete. 2024-09-19T09:25:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.904+0000 7fe3eac81740 20 client.31339 trim_cache size 0 max 16384 2024-09-19T09:25:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.921+0000 7fe3eac81740 1 Processor -- start 2024-09-19T09:25:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.921+0000 7fe3eac81740 1 -- start start 2024-09-19T09:25:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.921+0000 7fe3eac81740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aa908b9a0 0x557aa90860b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.921+0000 7fe3eac81740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aa908c2e0 0x557aa9088400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.921+0000 7fe3eac81740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aa909f7b0 0x557aa9088940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.921+0000 7fe3eac81740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e01a6e90 con 0x557aa909f7b0 2024-09-19T09:25:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.921+0000 7fe3eac81740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e01a89d0 con 0x557aa908c2e0 2024-09-19T09:25:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.921+0000 7fe3eac81740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e01a7560 con 0x557aa908b9a0 2024-09-19T09:25:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.922+0000 7fe3e5398640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aa909f7b0 0x557aa9088940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.922+0000 7fe3e5b99640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aa908c2e0 0x557aa9088400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.922+0000 7fe3e5b99640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aa908c2e0 0x557aa9088400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:36754/0 (socket says 172.21.15.120:36754) 2024-09-19T09:25:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.922+0000 7fe3e5398640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aa909f7b0 0x557aa9088940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35694/0 (socket says 172.21.15.120:35694) 2024-09-19T09:25:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.922+0000 7fe3e5b99640 1 -- 172.21.15.120:0/1645337801 learned_addr learned my addr 172.21.15.120:0/1645337801 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.922+0000 7fe3e5398640 1 --2- 172.21.15.120:0/1645337801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aa908b9a0 0x557aa90860b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.922+0000 7fe3e5398640 1 -- 172.21.15.120:0/1645337801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aa908b9a0 msgr2=0x557aa90860b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.922+0000 7fe3e5398640 1 --2- 172.21.15.120:0/1645337801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aa908b9a0 0x557aa90860b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.922+0000 7fe3e5398640 1 -- 172.21.15.120:0/1645337801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aa908c2e0 msgr2=0x557aa9088400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.922+0000 7fe3e5398640 1 --2- 172.21.15.120:0/1645337801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aa908c2e0 0x557aa9088400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.922+0000 7fe3e5398640 1 -- 172.21.15.120:0/1645337801 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557aa9099d80 con 0x557aa909f7b0 2024-09-19T09:25:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.922+0000 7fe3e5398640 1 --2- 172.21.15.120:0/1645337801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aa909f7b0 0x557aa9088940 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4004000 tx=0x7fe3d4011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a8576acf68d827 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.922+0000 7fe3c77fe640 1 -- 172.21.15.120:0/1645337801 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4000e60 con 0x557aa909f7b0 2024-09-19T09:25:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.922+0000 7fe3c77fe640 1 -- 172.21.15.120:0/1645337801 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3d4034070 con 0x557aa909f7b0 2024-09-19T09:25:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.922+0000 7fe3c77fe640 1 -- 172.21.15.120:0/1645337801 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d400fc30 con 0x557aa909f7b0 2024-09-19T09:25:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.923+0000 7fe3eac81740 1 -- 172.21.15.120:0/1645337801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aa909f7b0 msgr2=0x557aa9088940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.923+0000 7fe3eac81740 1 --2- 172.21.15.120:0/1645337801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aa909f7b0 0x557aa9088940 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4004000 tx=0x7fe3d4011f90 comp rx=0 tx=0).stop 2024-09-19T09:25:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.923+0000 7fe3eac81740 1 -- 172.21.15.120:0/1645337801 shutdown_connections 2024-09-19T09:25:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.923+0000 7fe3eac81740 1 --2- 172.21.15.120:0/1645337801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aa909f7b0 0x557aa9088940 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.923+0000 7fe3eac81740 1 --2- 172.21.15.120:0/1645337801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aa908c2e0 0x557aa9088400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.923+0000 7fe3eac81740 1 --2- 172.21.15.120:0/1645337801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aa908b9a0 0x557aa90860b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.923+0000 7fe3eac81740 1 -- 172.21.15.120:0/1645337801 >> 172.21.15.120:0/1645337801 conn(0x557aa9087010 msgr2=0x557aa90859a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.923+0000 7fe3eac81740 1 -- 172.21.15.120:0/1645337801 shutdown_connections 2024-09-19T09:25:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.923+0000 7fe3eac81740 1 -- 172.21.15.120:0/1645337801 wait complete. 2024-09-19T09:25:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.924+0000 7fe3eac81740 1 -- start start 2024-09-19T09:25:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.924+0000 7fe3eac81740 1 Processor -- start 2024-09-19T09:25:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.924+0000 7fe3eac81740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aa908b9a0 0x557aa90b7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.924+0000 7fe3eac81740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aa908c2e0 0x557aa90b81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.924+0000 7fe3eac81740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aa909f7b0 0x557aa90b9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.924+0000 7fe3eac81740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e01a8300 con 0x557aa908c2e0 2024-09-19T09:25:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.924+0000 7fe3eac81740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e01a7c30 con 0x557aa909f7b0 2024-09-19T09:25:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.924+0000 7fe3eac81740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e01a4e50 con 0x557aa908b9a0 2024-09-19T09:25:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.924+0000 7fe3eac81740 10 client.0 _ll_register_callbacks cb 0x7fe3d0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:25:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.924+0000 7fe3e5b99640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aa908c2e0 0x557aa90b81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.925+0000 7fe3e5b99640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aa908c2e0 0x557aa90b81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35708/0 (socket says 172.21.15.120:35708) 2024-09-19T09:25:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.925+0000 7fe3e5b99640 1 -- 172.21.15.120:0/1840554293 learned_addr learned my addr 172.21.15.120:0/1840554293 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.925+0000 7fe3e5398640 1 --2- 172.21.15.120:0/1840554293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aa908b9a0 0x557aa90b7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.925+0000 7fe3e5398640 1 --2- 172.21.15.120:0/1840554293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aa909f7b0 0x557aa90b9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.925+0000 7fe3e5b99640 1 -- 172.21.15.120:0/1840554293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aa908b9a0 msgr2=0x557aa90b7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.925+0000 7fe3e5b99640 1 --2- 172.21.15.120:0/1840554293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aa908b9a0 0x557aa90b7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.925+0000 7fe3e5b99640 1 -- 172.21.15.120:0/1840554293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aa909f7b0 msgr2=0x557aa90b9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.925+0000 7fe3e5b99640 1 --2- 172.21.15.120:0/1840554293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aa909f7b0 0x557aa90b9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.925+0000 7fe3e5b99640 1 -- 172.21.15.120:0/1840554293 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557aa90b9710 con 0x557aa908c2e0 2024-09-19T09:25:33.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.925+0000 7fe3e5b99640 1 --2- 172.21.15.120:0/1840554293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aa908c2e0 0x557aa90b81b0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0030ce0 tx=0x7fe3d0051660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25993892c63dfbb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:33.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.925+0000 7fe3c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:25:33.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.925+0000 7fe3c57fa640 1 -- 172.21.15.120:0/1840554293 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d005f060 con 0x557aa908c2e0 2024-09-19T09:25:33.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.925+0000 7fe3c57fa640 1 -- 172.21.15.120:0/1840554293 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe3d002fd50 con 0x557aa908c2e0 2024-09-19T09:25:33.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.925+0000 7fe3eac81740 1 -- 172.21.15.120:0/1840554293 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3d4034070 con 0x557aa908c2e0 2024-09-19T09:25:33.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.925+0000 7fe3c57fa640 1 -- 172.21.15.120:0/1840554293 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe3d0005bd0 con 0x557aa908c2e0 2024-09-19T09:25:33.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.925+0000 7fe3eac81740 10 client.22806 fetch_fsmap learned FSMap version 39 2024-09-19T09:25:33.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.925+0000 7fe3eac81740 1 -- 172.21.15.120:0/1840554293 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557aa9110e40 con 0x557aa908c2e0 2024-09-19T09:25:33.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.926+0000 7fe3c57fa640 1 -- 172.21.15.120:0/1840554293 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3d4034070 con 0x557aa908c2e0 2024-09-19T09:25:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.926+0000 7fe3c57fa640 1 -- 172.21.15.120:0/1840554293 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fe3d0064070 con 0x557aa908c2e0 2024-09-19T09:25:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.926+0000 7fe3eac81740 10 client.22806 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:25:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.926+0000 7fe3eac81740 10 client.22806 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:25:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.926+0000 7fe3eac81740 20 client.22806 populate_metadata read hostname 'smithi120' 2024-09-19T09:25:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.926+0000 7fe3eac81740 1 --2- 172.21.15.120:0/1840554293 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557aa9111760 0x557aa9113b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.926+0000 7fe3eac81740 4 client.22806 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:25:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.926+0000 7fe3eac81740 1 -- 172.21.15.120:0/1840554293 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557aa8ff6d70 con 0x557aa9111760 2024-09-19T09:25:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.926+0000 7fe3e5398640 1 --2- 172.21.15.120:0/1840554293 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557aa9111760 0x557aa9113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.927+0000 7fe3c57fa640 10 client.22806 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:33.927+0000 7fe3e5398640 1 --2- 172.21.15.120:0/1840554293 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557aa9111760 0x557aa9113b80 crc :-1 s=READY pgs=558 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=337d767595e31694 server_cookie=7ee799a7b2952cfb in_seq=0 out_seq=0 2024-09-19T09:25:34.028 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:34.344 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:34 smithi120 ceph-mon[31150]: pgmap v1026: 97 pgs: 97 active+clean; 493 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.4 MiB/s wr, 594 op/s 2024-09-19T09:25:34.384 INFO:tasks.workunit.client.0.smithi120.stdout: 10 222315 20.01 MB/sec execute 449 sec latency 334.435 ms 2024-09-19T09:25:34.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:34 smithi196 ceph-mon[35150]: pgmap v1026: 97 pgs: 97 active+clean; 493 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.4 MiB/s wr, 594 op/s 2024-09-19T09:25:34.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:34 smithi153 ceph-mon[35071]: pgmap v1026: 97 pgs: 97 active+clean; 493 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.4 MiB/s wr, 594 op/s 2024-09-19T09:25:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:34.534+0000 7fe3c57fa640 1 -- 172.21.15.120:0/1840554293 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557aa8ff6d70 con 0x557aa9111760 2024-09-19T09:25:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:34.534+0000 7fe3c57fa640 10 client.22806 handle_command_reply: tid=0 2024-09-19T09:25:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:34.534+0000 7fe3eac81740 1 client.22806 shutdown 2024-09-19T09:25:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:34.534+0000 7fe3eac81740 1 -- 172.21.15.120:0/1840554293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aa908c2e0 msgr2=0x557aa90b81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:34.534+0000 7fe3eac81740 1 --2- 172.21.15.120:0/1840554293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aa908c2e0 0x557aa90b81b0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0030ce0 tx=0x7fe3d0051660 comp rx=0 tx=0).stop 2024-09-19T09:25:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:34.534+0000 7fe3eac81740 1 -- 172.21.15.120:0/1840554293 shutdown_connections 2024-09-19T09:25:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:34.534+0000 7fe3eac81740 1 --2- 172.21.15.120:0/1840554293 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557aa9111760 0x557aa9113b80 crc :-1 s=READY pgs=558 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:34.534+0000 7fe3eac81740 1 --2- 172.21.15.120:0/1840554293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557aa909f7b0 0x557aa90b9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:34.534+0000 7fe3eac81740 1 --2- 172.21.15.120:0/1840554293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557aa908c2e0 0x557aa90b81b0 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:34.535+0000 7fe3eac81740 1 --2- 172.21.15.120:0/1840554293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557aa908b9a0 0x557aa90b7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:34.535+0000 7fe3eac81740 1 -- 172.21.15.120:0/1840554293 >> 172.21.15.120:0/1840554293 conn(0x557aa9087010 msgr2=0x557aa90861c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:34.535+0000 7fe3c57fa640 0 client.22806 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:34.535+0000 7fe3eac81740 1 -- 172.21.15.120:0/1840554293 shutdown_connections 2024-09-19T09:25:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:34.535+0000 7fe3eac81740 1 -- 172.21.15.120:0/1840554293 wait complete. 2024-09-19T09:25:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:34.535+0000 7fe3eac81740 20 client.22806 trim_cache size 0 max 16384 2024-09-19T09:25:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:25:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:25:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:25:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:25:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:34.535+0000 7fe3e7dfd640 1 -- 172.21.15.120:0/354837506 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe3c80a3370 msgr2=0x7fe3c80a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:34.535+0000 7fe3e7dfd640 1 --2- 172.21.15.120:0/354837506 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe3c80a3370 0x7fe3c80a5830 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe3d809c2f0 tx=0x7fe3d801c590 comp rx=0 tx=0).stop 2024-09-19T09:25:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:34.535+0000 7fe3e7dfd640 1 -- 172.21.15.120:0/354837506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3e01025f0 msgr2=0x7fe3e019e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:34.535+0000 7fe3e7dfd640 1 --2- 172.21.15.120:0/354837506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3e01025f0 0x7fe3e019e270 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800e970 tx=0x7fe3d800ed70 comp rx=0 tx=0).stop 2024-09-19T09:25:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:34.536+0000 7fe3e7dfd640 1 -- 172.21.15.120:0/354837506 shutdown_connections 2024-09-19T09:25:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:34.536+0000 7fe3e7dfd640 1 --2- 172.21.15.120:0/354837506 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe3c80a3370 0x7fe3c80a5830 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:34.536+0000 7fe3e7dfd640 1 --2- 172.21.15.120:0/354837506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3e01025f0 0x7fe3e019e270 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:34.536+0000 7fe3e7dfd640 1 --2- 172.21.15.120:0/354837506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3e0101c30 0x7fe3e0197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:34.536+0000 7fe3e7dfd640 1 --2- 172.21.15.120:0/354837506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3e0100a40 0x7fe3e0196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:34.536+0000 7fe3e7dfd640 1 -- 172.21.15.120:0/354837506 >> 172.21.15.120:0/354837506 conn(0x7fe3e00fc1e0 msgr2=0x7fe3e0104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:34.536+0000 7fe3e7dfd640 1 -- 172.21.15.120:0/354837506 shutdown_connections 2024-09-19T09:25:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:34.536+0000 7fe3e7dfd640 1 -- 172.21.15.120:0/354837506 wait complete. 2024-09-19T09:25:34.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.805+0000 7f65cb8d8640 1 -- 172.21.15.120:0/97382895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65c4102d30 msgr2=0x7f65c4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:34.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.805+0000 7f65cb8d8640 1 --2- 172.21.15.120:0/97382895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65c4102d30 0x7f65c4103110 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009930 tx=0x7f65c002a420 comp rx=0 tx=0).stop 2024-09-19T09:25:34.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.805+0000 7f65cb8d8640 1 -- 172.21.15.120:0/97382895 shutdown_connections 2024-09-19T09:25:34.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.805+0000 7f65cb8d8640 1 --2- 172.21.15.120:0/97382895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65c41082a0 0x7f65c410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.805+0000 7f65cb8d8640 1 --2- 172.21.15.120:0/97382895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65c41036e0 0x7f65c4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.805+0000 7f65cb8d8640 1 --2- 172.21.15.120:0/97382895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65c4102d30 0x7f65c4103110 unknown :-1 s=CLOSED pgs=2328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.805+0000 7f65cb8d8640 1 -- 172.21.15.120:0/97382895 >> 172.21.15.120:0/97382895 conn(0x7f65c40fc7d0 msgr2=0x7f65c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:34.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.805+0000 7f65cb8d8640 1 -- 172.21.15.120:0/97382895 shutdown_connections 2024-09-19T09:25:34.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.806+0000 7f65cb8d8640 1 -- 172.21.15.120:0/97382895 wait complete. 2024-09-19T09:25:34.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.806+0000 7f65cb8d8640 1 Processor -- start 2024-09-19T09:25:34.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.806+0000 7f65cb8d8640 1 -- start start 2024-09-19T09:25:34.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.807+0000 7f65cb8d8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65c4102d30 0x7f65c41985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:34.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.807+0000 7f65cb8d8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65c41036e0 0x7f65c4198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:34.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.807+0000 7f65cb8d8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65c41082a0 0x7f65c419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:34.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.807+0000 7f65cb8d8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41120a0 con 0x7f65c4102d30 2024-09-19T09:25:34.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.807+0000 7f65cb8d8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f65c4111f20 con 0x7f65c41036e0 2024-09-19T09:25:34.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.807+0000 7f65cb8d8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f65c4112220 con 0x7f65c41082a0 2024-09-19T09:25:34.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.807+0000 7f65c9674640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65c4102d30 0x7f65c41985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:34.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.807+0000 7f65c9674640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65c4102d30 0x7f65c41985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35740/0 (socket says 172.21.15.120:35740) 2024-09-19T09:25:34.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.807+0000 7f65c9674640 1 -- 172.21.15.120:0/2569908441 learned_addr learned my addr 172.21.15.120:0/2569908441 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:34.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.807+0000 7f65c8e73640 1 --2- 172.21.15.120:0/2569908441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65c41036e0 0x7f65c4198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:34.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.807+0000 7f65c9e75640 1 --2- 172.21.15.120:0/2569908441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65c41082a0 0x7f65c419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:34.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.807+0000 7f65c9674640 1 -- 172.21.15.120:0/2569908441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65c41082a0 msgr2=0x7f65c419ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:34.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.807+0000 7f65c9674640 1 --2- 172.21.15.120:0/2569908441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65c41082a0 0x7f65c419ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.807+0000 7f65c9674640 1 -- 172.21.15.120:0/2569908441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65c41036e0 msgr2=0x7f65c4198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:34.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.807+0000 7f65c9674640 1 --2- 172.21.15.120:0/2569908441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65c41036e0 0x7f65c4198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.807+0000 7f65c9674640 1 -- 172.21.15.120:0/2569908441 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c419d570 con 0x7f65c4102d30 2024-09-19T09:25:34.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.807+0000 7f65c9674640 1 --2- 172.21.15.120:0/2569908441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65c4102d30 0x7f65c41985c0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009230 tx=0x7f65c0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:34.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.807+0000 7f65ba7fc640 1 -- 172.21.15.120:0/2569908441 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c003f030 con 0x7f65c4102d30 2024-09-19T09:25:34.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.807+0000 7f65cb8d8640 1 -- 172.21.15.120:0/2569908441 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c419d800 con 0x7f65c4102d30 2024-09-19T09:25:34.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.807+0000 7f65ba7fc640 1 -- 172.21.15.120:0/2569908441 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65c0003ad0 con 0x7f65c4102d30 2024-09-19T09:25:34.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.807+0000 7f65ba7fc640 1 -- 172.21.15.120:0/2569908441 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0035070 con 0x7f65c4102d30 2024-09-19T09:25:34.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.807+0000 7f65cb8d8640 1 -- 172.21.15.120:0/2569908441 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c41a5140 con 0x7f65c4102d30 2024-09-19T09:25:34.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.810+0000 7f65ba7fc640 1 -- 172.21.15.120:0/2569908441 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f65c0042030 con 0x7f65c4102d30 2024-09-19T09:25:34.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.810+0000 7f65cdcb1740 1 Processor -- start 2024-09-19T09:25:34.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.810+0000 7f65cdcb1740 1 -- start start 2024-09-19T09:25:34.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.811+0000 7f65cdcb1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f8449b40 0x5572f8448090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:34.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.811+0000 7f65cdcb1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572f84485d0 0x5572f8448a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:34.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.811+0000 7f65cdcb1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572f8453370 0x5572f8455760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:34.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.811+0000 7f65cdcb1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5572f83ae2f0 con 0x5572f84485d0 2024-09-19T09:25:34.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.811+0000 7f65cdcb1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5572f8355cf0 con 0x5572f8453370 2024-09-19T09:25:34.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.811+0000 7f65cdcb1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5572f83aa2b0 con 0x5572f8449b40 2024-09-19T09:25:34.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.811+0000 7f65c9674640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572f84485d0 0x5572f8448a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:34.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.811+0000 7f65c9674640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572f84485d0 0x5572f8448a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35746/0 (socket says 172.21.15.120:35746) 2024-09-19T09:25:34.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.811+0000 7f65c9674640 1 -- 172.21.15.120:0/1325376780 learned_addr learned my addr 172.21.15.120:0/1325376780 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:34.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.811+0000 7f65c9e75640 1 --2- 172.21.15.120:0/1325376780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f8449b40 0x5572f8448090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:34.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.811+0000 7f65ba7fc640 1 --2- 172.21.15.120:0/2569908441 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f65b00a3240 0x7f65b00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:34.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.811+0000 7f65c8e73640 1 --2- 172.21.15.120:0/1325376780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572f8453370 0x5572f8455760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:34.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.811+0000 7f65c9e75640 1 --2- 172.21.15.120:0/2569908441 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f65b00a3240 0x7f65b00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:34.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.811+0000 7f65ba7fc640 1 -- 172.21.15.120:0/2569908441 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f65c00efee0 con 0x7f65c4102d30 2024-09-19T09:25:34.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.811+0000 7f65c9674640 1 -- 172.21.15.120:0/1325376780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f8449b40 msgr2=0x5572f8448090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:34.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.811+0000 7f65c9674640 1 --2- 172.21.15.120:0/1325376780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f8449b40 0x5572f8448090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.811+0000 7f65c9674640 1 -- 172.21.15.120:0/1325376780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572f8453370 msgr2=0x5572f8455760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:34.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.811+0000 7f65c9674640 1 --2- 172.21.15.120:0/1325376780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572f8453370 0x5572f8455760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.811+0000 7f65c9674640 1 -- 172.21.15.120:0/1325376780 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572f833a3a0 con 0x5572f84485d0 2024-09-19T09:25:34.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.811+0000 7f65c9e75640 1 --2- 172.21.15.120:0/1325376780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f8449b40 0x5572f8448090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:25:34.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.811+0000 7f65c9674640 1 --2- 172.21.15.120:0/1325376780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572f84485d0 0x5572f8448a10 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f65c00bc2b0 tx=0x7f65c0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7270e12e57dda093 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:34.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.811+0000 7f65af7fe640 1 -- 172.21.15.120:0/1325376780 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0049070 con 0x5572f84485d0 2024-09-19T09:25:34.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.812+0000 7f65c9e75640 1 --2- 172.21.15.120:0/2569908441 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f65b00a3240 0x7f65b00a5700 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f65bc005fd0 tx=0x7f65bc004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:34.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.812+0000 7f65af7fe640 1 -- 172.21.15.120:0/1325376780 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65c00bc5b0 con 0x5572f84485d0 2024-09-19T09:25:34.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.812+0000 7f65af7fe640 1 -- 172.21.15.120:0/1325376780 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c003bd60 con 0x5572f84485d0 2024-09-19T09:25:34.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.812+0000 7f65cdcb1740 1 -- 172.21.15.120:0/1325376780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572f84485d0 msgr2=0x5572f8448a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:34.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.812+0000 7f65cdcb1740 1 --2- 172.21.15.120:0/1325376780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572f84485d0 0x5572f8448a10 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f65c00bc2b0 tx=0x7f65c0009260 comp rx=0 tx=0).stop 2024-09-19T09:25:34.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.812+0000 7f65cdcb1740 1 -- 172.21.15.120:0/1325376780 shutdown_connections 2024-09-19T09:25:34.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.812+0000 7f65cdcb1740 1 --2- 172.21.15.120:0/1325376780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572f8453370 0x5572f8455760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.812+0000 7f65cdcb1740 1 --2- 172.21.15.120:0/1325376780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572f84485d0 0x5572f8448a10 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.812+0000 7f65cdcb1740 1 --2- 172.21.15.120:0/1325376780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f8449b40 0x5572f8448090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.812+0000 7f65cdcb1740 1 -- 172.21.15.120:0/1325376780 >> 172.21.15.120:0/1325376780 conn(0x5572f8411310 msgr2=0x5572f8444c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:34.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.812+0000 7f65cdcb1740 1 -- 172.21.15.120:0/1325376780 shutdown_connections 2024-09-19T09:25:34.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.812+0000 7f65cdcb1740 1 -- 172.21.15.120:0/1325376780 wait complete. 2024-09-19T09:25:34.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.813+0000 7f65cdcb1740 1 -- start start 2024-09-19T09:25:34.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.813+0000 7f65cdcb1740 1 Processor -- start 2024-09-19T09:25:34.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.813+0000 7f65cdcb1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572f84485d0 0x5572f8452da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:34.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.813+0000 7f65cdcb1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f8449b40 0x5572f8450050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:34.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.813+0000 7f65cdcb1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572f8453370 0x5572f84507e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:34.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.813+0000 7f65cdcb1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5572f8455e30 con 0x5572f84485d0 2024-09-19T09:25:34.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.813+0000 7f65cdcb1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5572f8369060 con 0x5572f8453370 2024-09-19T09:25:34.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.813+0000 7f65cdcb1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5572f8455fb0 con 0x5572f8449b40 2024-09-19T09:25:34.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.813+0000 7f65cdcb1740 10 client.0 _ll_register_callbacks cb 0x5572f80bc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:25:34.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.813+0000 7f65c9674640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572f84485d0 0x5572f8452da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:34.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.813+0000 7f65c9674640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572f84485d0 0x5572f8452da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35762/0 (socket says 172.21.15.120:35762) 2024-09-19T09:25:34.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.813+0000 7f65c9674640 1 -- 172.21.15.120:0/126371872 learned_addr learned my addr 172.21.15.120:0/126371872 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:34.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.813+0000 7f65c9e75640 1 --2- 172.21.15.120:0/126371872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572f8453370 0x5572f84507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:34.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.813+0000 7f65c8e73640 1 --2- 172.21.15.120:0/126371872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f8449b40 0x5572f8450050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:34.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.814+0000 7f65c9674640 1 -- 172.21.15.120:0/126371872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f8449b40 msgr2=0x5572f8450050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:34.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.814+0000 7f65c9674640 1 --2- 172.21.15.120:0/126371872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f8449b40 0x5572f8450050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.814+0000 7f65c9674640 1 -- 172.21.15.120:0/126371872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572f8453370 msgr2=0x5572f84507e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:34.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.814+0000 7f65c9674640 1 --2- 172.21.15.120:0/126371872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572f8453370 0x5572f84507e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.814+0000 7f65c9674640 1 -- 172.21.15.120:0/126371872 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572f811f3c0 con 0x5572f84485d0 2024-09-19T09:25:34.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.814+0000 7f65c9674640 1 --2- 172.21.15.120:0/126371872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572f84485d0 0x5572f8452da0 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f65c0032c00 tx=0x7f65c00b8850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4287a6a6a5afa56 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:34.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.814+0000 7f65acff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:25:34.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.814+0000 7f65acff9640 1 -- 172.21.15.120:0/126371872 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0049070 con 0x5572f84485d0 2024-09-19T09:25:34.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.814+0000 7f65acff9640 1 -- 172.21.15.120:0/126371872 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f65c0003c80 con 0x5572f84485d0 2024-09-19T09:25:34.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.814+0000 7f65cdcb1740 1 -- 172.21.15.120:0/126371872 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65c00bc5b0 con 0x5572f84485d0 2024-09-19T09:25:34.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.814+0000 7f65acff9640 1 -- 172.21.15.120:0/126371872 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f65c00c5170 con 0x5572f84485d0 2024-09-19T09:25:34.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.814+0000 7f65cdcb1740 10 client.22818 fetch_fsmap learned FSMap version 39 2024-09-19T09:25:34.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.814+0000 7f65cdcb1740 1 -- 172.21.15.120:0/126371872 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5572f8455e30 con 0x5572f84485d0 2024-09-19T09:25:34.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.814+0000 7f65acff9640 1 -- 172.21.15.120:0/126371872 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65c00bc5b0 con 0x5572f84485d0 2024-09-19T09:25:34.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.814+0000 7f65acff9640 1 -- 172.21.15.120:0/126371872 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f65c00b8300 con 0x5572f84485d0 2024-09-19T09:25:34.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.814+0000 7f65cdcb1740 10 client.22818 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:25:34.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.814+0000 7f65cdcb1740 10 client.22818 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:25:34.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.814+0000 7f65cdcb1740 20 client.22818 populate_metadata read hostname 'smithi120' 2024-09-19T09:25:34.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.815+0000 7f65cdcb1740 1 --2- 172.21.15.120:0/126371872 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5572f8462a70 0x5572f8464e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:34.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.815+0000 7f65cdcb1740 4 client.22818 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:25:34.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.815+0000 7f65cdcb1740 1 -- 172.21.15.120:0/126371872 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572f83bb860 con 0x5572f8462a70 2024-09-19T09:25:34.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.815+0000 7f65c8e73640 1 --2- 172.21.15.120:0/126371872 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5572f8462a70 0x5572f8464e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:34.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.815+0000 7f65acff9640 10 client.22818 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:25:34.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.815+0000 7f65c8e73640 1 --2- 172.21.15.120:0/126371872 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5572f8462a70 0x5572f8464e60 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e13ce3d144318696 server_cookie=2b232727f533791c in_seq=0 out_seq=0 2024-09-19T09:25:34.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.816+0000 7f65acff9640 1 -- 172.21.15.120:0/126371872 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5572f83bb860 con 0x5572f8462a70 2024-09-19T09:25:34.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.816+0000 7f65acff9640 10 client.22818 handle_command_reply: tid=0 2024-09-19T09:25:34.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.816+0000 7f65cdcb1740 1 client.22818 shutdown 2024-09-19T09:25:34.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.816+0000 7f65cdcb1740 1 -- 172.21.15.120:0/126371872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572f84485d0 msgr2=0x5572f8452da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:34.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.816+0000 7f65cdcb1740 1 --2- 172.21.15.120:0/126371872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572f84485d0 0x5572f8452da0 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f65c0032c00 tx=0x7f65c00b8850 comp rx=0 tx=0).stop 2024-09-19T09:25:34.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.816+0000 7f65cdcb1740 1 -- 172.21.15.120:0/126371872 shutdown_connections 2024-09-19T09:25:34.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.816+0000 7f65cdcb1740 1 --2- 172.21.15.120:0/126371872 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5572f8462a70 0x5572f8464e60 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.816+0000 7f65cdcb1740 1 --2- 172.21.15.120:0/126371872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572f8453370 0x5572f84507e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.816+0000 7f65cdcb1740 1 --2- 172.21.15.120:0/126371872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f8449b40 0x5572f8450050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.816+0000 7f65acff9640 0 client.22818 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:25:34.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.816+0000 7f65cdcb1740 1 --2- 172.21.15.120:0/126371872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572f84485d0 0x5572f8452da0 unknown :-1 s=CLOSED pgs=2331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.816+0000 7f65cdcb1740 1 -- 172.21.15.120:0/126371872 >> 172.21.15.120:0/126371872 conn(0x5572f8411310 msgr2=0x5572f844a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:34.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.816+0000 7f65cdcb1740 1 -- 172.21.15.120:0/126371872 shutdown_connections 2024-09-19T09:25:34.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.816+0000 7f65cdcb1740 1 -- 172.21.15.120:0/126371872 wait complete. 2024-09-19T09:25:34.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.817+0000 7f65cdcb1740 20 client.22818 trim_cache size 0 max 16384 2024-09-19T09:25:34.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.831+0000 7f65cdcb1740 1 Processor -- start 2024-09-19T09:25:34.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.831+0000 7f65cdcb1740 1 -- start start 2024-09-19T09:25:34.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.831+0000 7f65cdcb1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572f84485d0 0x5572f8444e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:34.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.831+0000 7f65cdcb1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f8449b40 0x5572f846a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:34.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.831+0000 7f65cdcb1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572f8453370 0x5572f846ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:34.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.831+0000 7f65cdcb1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41ad660 con 0x5572f84485d0 2024-09-19T09:25:34.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.831+0000 7f65cdcb1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41af1a0 con 0x5572f8453370 2024-09-19T09:25:34.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.831+0000 7f65cdcb1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41add30 con 0x5572f8449b40 2024-09-19T09:25:34.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.831+0000 7f65c9674640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572f84485d0 0x5572f8444e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:34.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.831+0000 7f65c9674640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572f84485d0 0x5572f8444e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35768/0 (socket says 172.21.15.120:35768) 2024-09-19T09:25:34.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.831+0000 7f65c9674640 1 -- 172.21.15.120:0/3179295832 learned_addr learned my addr 172.21.15.120:0/3179295832 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:34.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.831+0000 7f65c8e73640 1 --2- 172.21.15.120:0/3179295832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f8449b40 0x5572f846a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:34.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.832+0000 7f65c9e75640 1 --2- 172.21.15.120:0/3179295832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572f8453370 0x5572f846ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:34.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.832+0000 7f65c9674640 1 -- 172.21.15.120:0/3179295832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f8449b40 msgr2=0x5572f846a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:34.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.832+0000 7f65c9674640 1 --2- 172.21.15.120:0/3179295832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f8449b40 0x5572f846a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.832+0000 7f65c9674640 1 -- 172.21.15.120:0/3179295832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572f8453370 msgr2=0x5572f846ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:34.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.832+0000 7f65c9674640 1 --2- 172.21.15.120:0/3179295832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572f8453370 0x5572f846ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.832+0000 7f65c9674640 1 -- 172.21.15.120:0/3179295832 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572f8445960 con 0x5572f84485d0 2024-09-19T09:25:34.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.832+0000 7f65c9674640 1 --2- 172.21.15.120:0/3179295832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572f84485d0 0x5572f8444e10 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f65c00bab00 tx=0x7f65c00f3500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218b8baa9acf447e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:34.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.832+0000 7f65aeffd640 1 -- 172.21.15.120:0/3179295832 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0049070 con 0x5572f84485d0 2024-09-19T09:25:34.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.832+0000 7f65aeffd640 1 -- 172.21.15.120:0/3179295832 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65c00eda60 con 0x5572f84485d0 2024-09-19T09:25:34.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.832+0000 7f65aeffd640 1 -- 172.21.15.120:0/3179295832 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c00d5310 con 0x5572f84485d0 2024-09-19T09:25:34.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.833+0000 7f65cdcb1740 1 -- 172.21.15.120:0/3179295832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572f84485d0 msgr2=0x5572f8444e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:34.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.833+0000 7f65cdcb1740 1 --2- 172.21.15.120:0/3179295832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572f84485d0 0x5572f8444e10 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f65c00bab00 tx=0x7f65c00f3500 comp rx=0 tx=0).stop 2024-09-19T09:25:34.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.833+0000 7f65cdcb1740 1 -- 172.21.15.120:0/3179295832 shutdown_connections 2024-09-19T09:25:34.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.833+0000 7f65cdcb1740 1 --2- 172.21.15.120:0/3179295832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572f8453370 0x5572f846ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.833+0000 7f65cdcb1740 1 --2- 172.21.15.120:0/3179295832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f8449b40 0x5572f846a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.833+0000 7f65cdcb1740 1 --2- 172.21.15.120:0/3179295832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572f84485d0 0x5572f8444e10 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.833+0000 7f65cdcb1740 1 -- 172.21.15.120:0/3179295832 >> 172.21.15.120:0/3179295832 conn(0x5572f8411310 msgr2=0x5572f8444700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:34.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.833+0000 7f65cdcb1740 1 -- 172.21.15.120:0/3179295832 shutdown_connections 2024-09-19T09:25:34.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.833+0000 7f65cdcb1740 1 -- 172.21.15.120:0/3179295832 wait complete. 2024-09-19T09:25:34.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.834+0000 7f65cdcb1740 1 -- start start 2024-09-19T09:25:34.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.834+0000 7f65cdcb1740 1 Processor -- start 2024-09-19T09:25:34.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.834+0000 7f65cdcb1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f84485d0 0x5572f844d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:34.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.834+0000 7f65cdcb1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572f8449b40 0x5572f844d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:34.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.834+0000 7f65cdcb1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572f8453370 0x5572f844dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:34.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.834+0000 7f65cdcb1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41aead0 con 0x5572f8453370 2024-09-19T09:25:34.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.834+0000 7f65cdcb1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41ae400 con 0x5572f8449b40 2024-09-19T09:25:34.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.834+0000 7f65cdcb1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41ab620 con 0x5572f84485d0 2024-09-19T09:25:34.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.834+0000 7f65cdcb1740 10 client.0 _ll_register_callbacks cb 0x7f65b40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:25:34.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.834+0000 7f65c8e73640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572f8449b40 0x5572f844d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:34.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.835+0000 7f65c8e73640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572f8449b40 0x5572f844d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:36822/0 (socket says 172.21.15.120:36822) 2024-09-19T09:25:34.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.835+0000 7f65c8e73640 1 -- 172.21.15.120:0/3438751409 learned_addr learned my addr 172.21.15.120:0/3438751409 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:34.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.835+0000 7f65c9e75640 1 --2- 172.21.15.120:0/3438751409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572f8453370 0x5572f844dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:34.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.835+0000 7f65c9674640 1 --2- 172.21.15.120:0/3438751409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f84485d0 0x5572f844d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:34.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.835+0000 7f65c8e73640 1 -- 172.21.15.120:0/3438751409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f84485d0 msgr2=0x5572f844d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:34.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.835+0000 7f65c8e73640 1 --2- 172.21.15.120:0/3438751409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f84485d0 0x5572f844d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.835+0000 7f65c8e73640 1 -- 172.21.15.120:0/3438751409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572f8453370 msgr2=0x5572f844dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:34.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.835+0000 7f65c8e73640 1 --2- 172.21.15.120:0/3438751409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572f8453370 0x5572f844dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.835+0000 7f65c8e73640 1 -- 172.21.15.120:0/3438751409 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572f84444f0 con 0x5572f8449b40 2024-09-19T09:25:34.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.835+0000 7f65c9e75640 1 --2- 172.21.15.120:0/3438751409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572f8453370 0x5572f844dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:25:34.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.835+0000 7f65c8e73640 1 --2- 172.21.15.120:0/3438751409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572f8449b40 0x5572f844d760 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f65b4009be0 tx=0x7f65b4009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcbeb588e6090785 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:34.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.835+0000 7f65acff9640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:25:34.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.835+0000 7f65acff9640 1 -- 172.21.15.120:0/3438751409 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b4022020 con 0x5572f8449b40 2024-09-19T09:25:34.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.835+0000 7f65cdcb1740 1 -- 172.21.15.120:0/3438751409 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65c00eda60 con 0x5572f8449b40 2024-09-19T09:25:34.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.836+0000 7f65acff9640 1 -- 172.21.15.120:0/3438751409 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f65b4004460 con 0x5572f8449b40 2024-09-19T09:25:34.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.836+0000 7f65acff9640 1 -- 172.21.15.120:0/3438751409 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f65b40035f0 con 0x5572f8449b40 2024-09-19T09:25:34.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.836+0000 7f65cdcb1740 10 client.31502 fetch_fsmap learned FSMap version 39 2024-09-19T09:25:34.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.836+0000 7f65cdcb1740 1 -- 172.21.15.120:0/3438751409 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5572f84d1de0 con 0x5572f8449b40 2024-09-19T09:25:34.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.836+0000 7f65acff9640 1 -- 172.21.15.120:0/3438751409 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65c00eda60 con 0x5572f8449b40 2024-09-19T09:25:34.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.836+0000 7f65acff9640 1 -- 172.21.15.120:0/3438751409 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f65b401ea80 con 0x5572f8449b40 2024-09-19T09:25:34.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.836+0000 7f65cdcb1740 10 client.31502 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:25:34.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.836+0000 7f65cdcb1740 10 client.31502 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:25:34.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.836+0000 7f65cdcb1740 20 client.31502 populate_metadata read hostname 'smithi120' 2024-09-19T09:25:34.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.836+0000 7f65cdcb1740 1 --2- 172.21.15.120:0/3438751409 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5572f84d2540 0x5572f84d4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:34.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.836+0000 7f65cdcb1740 4 client.31502 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:25:34.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.836+0000 7f65cdcb1740 1 -- 172.21.15.120:0/3438751409 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5572f83b5d70 con 0x5572f84d2540 2024-09-19T09:25:34.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.837+0000 7f65c9674640 1 --2- 172.21.15.120:0/3438751409 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5572f84d2540 0x5572f84d4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:34.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.837+0000 7f65acff9640 10 client.31502 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:25:34.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.837+0000 7f65c9674640 1 --2- 172.21.15.120:0/3438751409 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5572f84d2540 0x5572f84d4960 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=99527b6586ff04a7 server_cookie=ae03019cd6b48e11 in_seq=0 out_seq=0 2024-09-19T09:25:34.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.837+0000 7f65acff9640 1 -- 172.21.15.120:0/3438751409 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5572f83b5d70 con 0x5572f84d2540 2024-09-19T09:25:34.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.837+0000 7f65acff9640 10 client.31502 handle_command_reply: tid=0 2024-09-19T09:25:34.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.837+0000 7f65cdcb1740 1 client.31502 shutdown 2024-09-19T09:25:34.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.838+0000 7f65cdcb1740 1 -- 172.21.15.120:0/3438751409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572f8449b40 msgr2=0x5572f844d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:34.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.838+0000 7f65cdcb1740 1 --2- 172.21.15.120:0/3438751409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572f8449b40 0x5572f844d760 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f65b4009be0 tx=0x7f65b4009f90 comp rx=0 tx=0).stop 2024-09-19T09:25:34.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.838+0000 7f65cdcb1740 1 -- 172.21.15.120:0/3438751409 shutdown_connections 2024-09-19T09:25:34.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.838+0000 7f65cdcb1740 1 --2- 172.21.15.120:0/3438751409 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5572f84d2540 0x5572f84d4960 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.838+0000 7f65cdcb1740 1 --2- 172.21.15.120:0/3438751409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5572f8453370 0x5572f844dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.838+0000 7f65cdcb1740 1 --2- 172.21.15.120:0/3438751409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572f8449b40 0x5572f844d760 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.838+0000 7f65acff9640 0 client.31502 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:25:34.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.838+0000 7f65cdcb1740 1 --2- 172.21.15.120:0/3438751409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f84485d0 0x5572f844d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.838+0000 7f65cdcb1740 1 -- 172.21.15.120:0/3438751409 >> 172.21.15.120:0/3438751409 conn(0x5572f8411310 msgr2=0x5572f84434e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:34.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.838+0000 7f65cdcb1740 1 -- 172.21.15.120:0/3438751409 shutdown_connections 2024-09-19T09:25:34.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.838+0000 7f65cdcb1740 1 -- 172.21.15.120:0/3438751409 wait complete. 2024-09-19T09:25:34.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.838+0000 7f65cdcb1740 20 client.31502 trim_cache size 0 max 16384 2024-09-19T09:25:34.858 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:25:34.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.839+0000 7f65cb8d8640 1 -- 172.21.15.120:0/2569908441 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f65b00a3240 msgr2=0x7f65b00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:34.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.839+0000 7f65cb8d8640 1 --2- 172.21.15.120:0/2569908441 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f65b00a3240 0x7f65b00a5700 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f65bc005fd0 tx=0x7f65bc004ab0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.839+0000 7f65cb8d8640 1 -- 172.21.15.120:0/2569908441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65c4102d30 msgr2=0x7f65c41985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:34.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.839+0000 7f65cb8d8640 1 --2- 172.21.15.120:0/2569908441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65c4102d30 0x7f65c41985c0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009230 tx=0x7f65c0004270 comp rx=0 tx=0).stop 2024-09-19T09:25:34.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.839+0000 7f65cb8d8640 1 -- 172.21.15.120:0/2569908441 shutdown_connections 2024-09-19T09:25:34.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.839+0000 7f65cb8d8640 1 --2- 172.21.15.120:0/2569908441 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f65b00a3240 0x7f65b00a5700 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.839+0000 7f65cb8d8640 1 --2- 172.21.15.120:0/2569908441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65c41082a0 0x7f65c419ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.839+0000 7f65cb8d8640 1 --2- 172.21.15.120:0/2569908441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65c41036e0 0x7f65c4198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.839+0000 7f65cb8d8640 1 --2- 172.21.15.120:0/2569908441 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65c4102d30 0x7f65c41985c0 unknown :-1 s=CLOSED pgs=2329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.839+0000 7f65cb8d8640 1 -- 172.21.15.120:0/2569908441 >> 172.21.15.120:0/2569908441 conn(0x7f65c40fc7d0 msgr2=0x7f65c4105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:34.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.839+0000 7f65cb8d8640 1 -- 172.21.15.120:0/2569908441 shutdown_connections 2024-09-19T09:25:34.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:34.839+0000 7f65cb8d8640 1 -- 172.21.15.120:0/2569908441 wait complete. 2024-09-19T09:25:35.384 INFO:tasks.workunit.client.0.smithi120.stdout: 10 222913 20.06 MB/sec execute 450 sec latency 356.038 ms 2024-09-19T09:25:35.691 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:25:35.999 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:36.384 INFO:tasks.workunit.client.0.smithi120.stdout: 10 223578 20.10 MB/sec execute 451 sec latency 96.018 ms 2024-09-19T09:25:36.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:36 smithi196 ceph-mon[35150]: pgmap v1027: 97 pgs: 97 active+clean; 489 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.4 MiB/s wr, 497 op/s 2024-09-19T09:25:36.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:36 smithi153 ceph-mon[35071]: pgmap v1027: 97 pgs: 97 active+clean; 489 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.4 MiB/s wr, 497 op/s 2024-09-19T09:25:36.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:36 smithi120 ceph-mon[31150]: pgmap v1027: 97 pgs: 97 active+clean; 489 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.4 MiB/s wr, 497 op/s 2024-09-19T09:25:36.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.775+0000 7f9ee76b8640 1 -- 172.21.15.120:0/1558482502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ee0106a90 msgr2=0x7f9ee0106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:36.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.775+0000 7f9ee76b8640 1 --2- 172.21.15.120:0/1558482502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ee0106a90 0x7f9ee0106e70 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0009cd0 tx=0x7f9ed002a8e0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.775+0000 7f9ee76b8640 1 -- 172.21.15.120:0/1558482502 shutdown_connections 2024-09-19T09:25:36.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.775+0000 7f9ee76b8640 1 --2- 172.21.15.120:0/1558482502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ee0101430 0x7f9ee010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.775+0000 7f9ee76b8640 1 --2- 172.21.15.120:0/1558482502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ee0100a90 0x7f9ee0100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.775+0000 7f9ee76b8640 1 --2- 172.21.15.120:0/1558482502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ee0106a90 0x7f9ee0106e70 unknown :-1 s=CLOSED pgs=2333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.775+0000 7f9ee76b8640 1 -- 172.21.15.120:0/1558482502 >> 172.21.15.120:0/1558482502 conn(0x7f9ee00fc7d0 msgr2=0x7f9ee00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:36.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.775+0000 7f9ee76b8640 1 -- 172.21.15.120:0/1558482502 shutdown_connections 2024-09-19T09:25:36.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.775+0000 7f9ee76b8640 1 -- 172.21.15.120:0/1558482502 wait complete. 2024-09-19T09:25:36.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.776+0000 7f9ee76b8640 1 Processor -- start 2024-09-19T09:25:36.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.776+0000 7f9ee76b8640 1 -- start start 2024-09-19T09:25:36.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.776+0000 7f9ee76b8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ee0100a90 0x7f9ee0197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:36.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.776+0000 7f9ee76b8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ee0101430 0x7f9ee0197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:36.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.776+0000 7f9ee76b8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ee0106a90 0x7f9ee019bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:36.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.776+0000 7f9ee76b8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee0110e90 con 0x7f9ee0100a90 2024-09-19T09:25:36.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.776+0000 7f9ee76b8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee0110d10 con 0x7f9ee0101430 2024-09-19T09:25:36.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.776+0000 7f9ee76b8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee0111010 con 0x7f9ee0106a90 2024-09-19T09:25:36.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.776+0000 7f9ee5454640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ee0100a90 0x7f9ee0197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:36.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.776+0000 7f9ee5454640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ee0100a90 0x7f9ee0197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35798/0 (socket says 172.21.15.120:35798) 2024-09-19T09:25:36.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.776+0000 7f9ee5454640 1 -- 172.21.15.120:0/3622813878 learned_addr learned my addr 172.21.15.120:0/3622813878 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:36.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.776+0000 7f9ee4c53640 1 --2- 172.21.15.120:0/3622813878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ee0101430 0x7f9ee0197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:36.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.776+0000 7f9ee5c55640 1 --2- 172.21.15.120:0/3622813878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ee0106a90 0x7f9ee019bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:36.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.776+0000 7f9ee5454640 1 -- 172.21.15.120:0/3622813878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ee0106a90 msgr2=0x7f9ee019bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:36.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.776+0000 7f9ee5454640 1 --2- 172.21.15.120:0/3622813878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ee0106a90 0x7f9ee019bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.776+0000 7f9ee5454640 1 -- 172.21.15.120:0/3622813878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ee0101430 msgr2=0x7f9ee0197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:36.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.776+0000 7f9ee5454640 1 --2- 172.21.15.120:0/3622813878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ee0101430 0x7f9ee0197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.776+0000 7f9ee5454640 1 -- 172.21.15.120:0/3622813878 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee019c300 con 0x7f9ee0100a90 2024-09-19T09:25:36.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.776+0000 7f9ee4c53640 1 --2- 172.21.15.120:0/3622813878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ee0101430 0x7f9ee0197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:25:36.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.777+0000 7f9ee5454640 1 --2- 172.21.15.120:0/3622813878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ee0100a90 0x7f9ee0197350 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7f9ed002ae00 tx=0x7f9ed0031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:36.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.777+0000 7f9ede7fc640 1 -- 172.21.15.120:0/3622813878 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0003e00 con 0x7f9ee0100a90 2024-09-19T09:25:36.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.777+0000 7f9ede7fc640 1 -- 172.21.15.120:0/3622813878 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed0004520 con 0x7f9ee0100a90 2024-09-19T09:25:36.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.777+0000 7f9ee76b8640 1 -- 172.21.15.120:0/3622813878 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee019c590 con 0x7f9ee0100a90 2024-09-19T09:25:36.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.777+0000 7f9ede7fc640 1 -- 172.21.15.120:0/3622813878 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0030de0 con 0x7f9ee0100a90 2024-09-19T09:25:36.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.777+0000 7f9ee76b8640 1 -- 172.21.15.120:0/3622813878 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee01a3ed0 con 0x7f9ee0100a90 2024-09-19T09:25:36.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.779+0000 7f9ede7fc640 1 -- 172.21.15.120:0/3622813878 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f9ed00046d0 con 0x7f9ee0100a90 2024-09-19T09:25:36.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.780+0000 7f9ee9a91740 1 Processor -- start 2024-09-19T09:25:36.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.780+0000 7f9ee9a91740 1 -- start start 2024-09-19T09:25:36.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.780+0000 7f9ee9a91740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609dc580b40 0x5609dc57f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:36.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.780+0000 7f9ee9a91740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609dc57f5d0 0x5609dc57fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:36.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.780+0000 7f9ee9a91740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5609dc58c580 0x5609dc58e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:36.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.780+0000 7f9ee9a91740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5609dc4e52f0 con 0x5609dc57f5d0 2024-09-19T09:25:36.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.780+0000 7f9ee9a91740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5609dc48ccf0 con 0x5609dc58c580 2024-09-19T09:25:36.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.780+0000 7f9ee9a91740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5609dc4e12b0 con 0x5609dc580b40 2024-09-19T09:25:36.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.780+0000 7f9ede7fc640 1 --2- 172.21.15.120:0/3622813878 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9ec80a3240 0x7f9ec80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:36.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.780+0000 7f9ee5454640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609dc57f5d0 0x5609dc57fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:36.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.780+0000 7f9ee5454640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609dc57f5d0 0x5609dc57fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35812/0 (socket says 172.21.15.120:35812) 2024-09-19T09:25:36.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.780+0000 7f9ee5454640 1 -- 172.21.15.120:0/4278851203 learned_addr learned my addr 172.21.15.120:0/4278851203 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:36.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.780+0000 7f9ee5c55640 1 --2- 172.21.15.120:0/4278851203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609dc580b40 0x5609dc57f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:36.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.780+0000 7f9ee4c53640 1 --2- 172.21.15.120:0/4278851203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5609dc58c580 0x5609dc58e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:36.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.781+0000 7f9ee5c55640 1 --2- 172.21.15.120:0/3622813878 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9ec80a3240 0x7f9ec80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:36.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.781+0000 7f9ede7fc640 1 -- 172.21.15.120:0/3622813878 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f9ed002e030 con 0x7f9ee0100a90 2024-09-19T09:25:36.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.781+0000 7f9ee5454640 1 -- 172.21.15.120:0/4278851203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609dc580b40 msgr2=0x5609dc57f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:36.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.781+0000 7f9ee5454640 1 --2- 172.21.15.120:0/4278851203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609dc580b40 0x5609dc57f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.781+0000 7f9ee5454640 1 -- 172.21.15.120:0/4278851203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5609dc58c580 msgr2=0x5609dc58e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:36.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.781+0000 7f9ee5454640 1 --2- 172.21.15.120:0/4278851203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5609dc58c580 0x5609dc58e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.781+0000 7f9ee5454640 1 -- 172.21.15.120:0/4278851203 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609dc4713a0 con 0x5609dc57f5d0 2024-09-19T09:25:36.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.781+0000 7f9ee5c55640 1 --2- 172.21.15.120:0/4278851203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609dc580b40 0x5609dc57f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:25:36.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.781+0000 7f9ee4c53640 1 --2- 172.21.15.120:0/4278851203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5609dc58c580 0x5609dc58e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:25:36.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.781+0000 7f9ee5454640 1 --2- 172.21.15.120:0/4278851203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609dc57f5d0 0x5609dc57fa10 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0005e30 tx=0x7f9ed0090370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f040970c8015832 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:36.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.781+0000 7f9ee5c55640 1 --2- 172.21.15.120:0/3622813878 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9ec80a3240 0x7f9ec80a5700 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8005f60 tx=0x7f9ed8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:36.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.781+0000 7f9ec77fe640 1 -- 172.21.15.120:0/4278851203 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed00bce50 con 0x5609dc57f5d0 2024-09-19T09:25:36.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.781+0000 7f9ec77fe640 1 -- 172.21.15.120:0/4278851203 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed0036070 con 0x5609dc57f5d0 2024-09-19T09:25:36.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.781+0000 7f9ec77fe640 1 -- 172.21.15.120:0/4278851203 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed00c63b0 con 0x5609dc57f5d0 2024-09-19T09:25:36.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.782+0000 7f9ee9a91740 1 -- 172.21.15.120:0/4278851203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609dc57f5d0 msgr2=0x5609dc57fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:36.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.782+0000 7f9ee9a91740 1 --2- 172.21.15.120:0/4278851203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609dc57f5d0 0x5609dc57fa10 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0005e30 tx=0x7f9ed0090370 comp rx=0 tx=0).stop 2024-09-19T09:25:36.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.782+0000 7f9ee9a91740 1 -- 172.21.15.120:0/4278851203 shutdown_connections 2024-09-19T09:25:36.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.782+0000 7f9ee9a91740 1 --2- 172.21.15.120:0/4278851203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5609dc58c580 0x5609dc58e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.782+0000 7f9ee9a91740 1 --2- 172.21.15.120:0/4278851203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609dc57f5d0 0x5609dc57fa10 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.782+0000 7f9ee9a91740 1 --2- 172.21.15.120:0/4278851203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609dc580b40 0x5609dc57f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.782+0000 7f9ee9a91740 1 -- 172.21.15.120:0/4278851203 >> 172.21.15.120:0/4278851203 conn(0x5609dc548310 msgr2=0x5609dc57bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:36.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.782+0000 7f9ee9a91740 1 -- 172.21.15.120:0/4278851203 shutdown_connections 2024-09-19T09:25:36.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.782+0000 7f9ee9a91740 1 -- 172.21.15.120:0/4278851203 wait complete. 2024-09-19T09:25:36.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.782+0000 7f9ee9a91740 1 -- start start 2024-09-19T09:25:36.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.783+0000 7f9ee9a91740 1 Processor -- start 2024-09-19T09:25:36.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.783+0000 7f9ee9a91740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5609dc57f5d0 0x5609dc58bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:36.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.783+0000 7f9ee9a91740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609dc580b40 0x5609dc589250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:36.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.783+0000 7f9ee9a91740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609dc58c580 0x5609dc589860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:36.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.783+0000 7f9ee9a91740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5609dc58f040 con 0x5609dc580b40 2024-09-19T09:25:36.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.783+0000 7f9ee9a91740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5609dc4a0060 con 0x5609dc57f5d0 2024-09-19T09:25:36.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.783+0000 7f9ee9a91740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5609dc58f1c0 con 0x5609dc58c580 2024-09-19T09:25:36.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.783+0000 7f9ee9a91740 10 client.0 _ll_register_callbacks cb 0x5609dc1f38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:25:36.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.783+0000 7f9ee4c53640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609dc580b40 0x5609dc589250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:36.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.783+0000 7f9ee5454640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5609dc57f5d0 0x5609dc58bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:36.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.783+0000 7f9ee4c53640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609dc580b40 0x5609dc589250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35822/0 (socket says 172.21.15.120:35822) 2024-09-19T09:25:36.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.783+0000 7f9ee4c53640 1 -- 172.21.15.120:0/3389122778 learned_addr learned my addr 172.21.15.120:0/3389122778 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:36.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.783+0000 7f9ee5c55640 1 --2- 172.21.15.120:0/3389122778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609dc58c580 0x5609dc589860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:36.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.783+0000 7f9ee4c53640 1 -- 172.21.15.120:0/3389122778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609dc58c580 msgr2=0x5609dc589860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:36.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.784+0000 7f9ee4c53640 1 --2- 172.21.15.120:0/3389122778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609dc58c580 0x5609dc589860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.784+0000 7f9ee4c53640 1 -- 172.21.15.120:0/3389122778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5609dc57f5d0 msgr2=0x5609dc58bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:36.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.784+0000 7f9ee4c53640 1 --2- 172.21.15.120:0/3389122778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5609dc57f5d0 0x5609dc58bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.784+0000 7f9ee4c53640 1 -- 172.21.15.120:0/3389122778 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609dc2563c0 con 0x5609dc580b40 2024-09-19T09:25:36.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.784+0000 7f9ee5454640 1 --2- 172.21.15.120:0/3389122778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5609dc57f5d0 0x5609dc58bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:25:36.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.784+0000 7f9ee4c53640 1 --2- 172.21.15.120:0/3389122778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609dc580b40 0x5609dc589250 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4012b10 tx=0x7f9ed400f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444a4b58c2095dba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:36.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.784+0000 7f9ec4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:25:36.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.784+0000 7f9ec4ff9640 1 -- 172.21.15.120:0/3389122778 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed400fb40 con 0x5609dc580b40 2024-09-19T09:25:36.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.784+0000 7f9ee9a91740 1 -- 172.21.15.120:0/3389122778 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ed0036070 con 0x5609dc580b40 2024-09-19T09:25:36.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.784+0000 7f9ec4ff9640 1 -- 172.21.15.120:0/3389122778 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f9ed400fce0 con 0x5609dc580b40 2024-09-19T09:25:36.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.784+0000 7f9ec4ff9640 1 -- 172.21.15.120:0/3389122778 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9ed4010430 con 0x5609dc580b40 2024-09-19T09:25:36.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.784+0000 7f9ee9a91740 10 client.22836 fetch_fsmap learned FSMap version 39 2024-09-19T09:25:36.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.784+0000 7f9ee9a91740 1 -- 172.21.15.120:0/3389122778 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5609dc59ab00 con 0x5609dc580b40 2024-09-19T09:25:36.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.784+0000 7f9ec4ff9640 1 -- 172.21.15.120:0/3389122778 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed0036070 con 0x5609dc580b40 2024-09-19T09:25:36.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.785+0000 7f9ec4ff9640 1 -- 172.21.15.120:0/3389122778 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f9ed401c680 con 0x5609dc580b40 2024-09-19T09:25:36.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.785+0000 7f9ee9a91740 10 client.22836 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:25:36.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.785+0000 7f9ee9a91740 10 client.22836 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:25:36.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.785+0000 7f9ee9a91740 20 client.22836 populate_metadata read hostname 'smithi120' 2024-09-19T09:25:36.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.785+0000 7f9ee9a91740 1 --2- 172.21.15.120:0/3389122778 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5609dc59bf10 0x5609dc59e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:36.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.785+0000 7f9ee9a91740 4 client.22836 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:25:36.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.785+0000 7f9ee9a91740 1 -- 172.21.15.120:0/3389122778 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609dc4f2860 con 0x5609dc59bf10 2024-09-19T09:25:36.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.785+0000 7f9ee5454640 1 --2- 172.21.15.120:0/3389122778 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5609dc59bf10 0x5609dc59e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:36.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.785+0000 7f9ec4ff9640 10 client.22836 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:25:36.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.785+0000 7f9ee5454640 1 --2- 172.21.15.120:0/3389122778 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5609dc59bf10 0x5609dc59e3b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a1b86c921ee0cb47 server_cookie=a1395a8e73ed899c in_seq=0 out_seq=0 2024-09-19T09:25:36.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.786+0000 7f9ec4ff9640 1 -- 172.21.15.120:0/3389122778 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5609dc4f2860 con 0x5609dc59bf10 2024-09-19T09:25:36.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.786+0000 7f9ec4ff9640 10 client.22836 handle_command_reply: tid=0 2024-09-19T09:25:36.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.786+0000 7f9ee9a91740 1 client.22836 shutdown 2024-09-19T09:25:36.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.786+0000 7f9ee9a91740 1 -- 172.21.15.120:0/3389122778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609dc580b40 msgr2=0x5609dc589250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:36.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.786+0000 7f9ee9a91740 1 --2- 172.21.15.120:0/3389122778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609dc580b40 0x5609dc589250 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4012b10 tx=0x7f9ed400f3c0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.787+0000 7f9ee9a91740 1 -- 172.21.15.120:0/3389122778 shutdown_connections 2024-09-19T09:25:36.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.787+0000 7f9ee9a91740 1 --2- 172.21.15.120:0/3389122778 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5609dc59bf10 0x5609dc59e3b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.787+0000 7f9ee9a91740 1 --2- 172.21.15.120:0/3389122778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609dc58c580 0x5609dc589860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.787+0000 7f9ee9a91740 1 --2- 172.21.15.120:0/3389122778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609dc580b40 0x5609dc589250 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.787+0000 7f9ee9a91740 1 --2- 172.21.15.120:0/3389122778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5609dc57f5d0 0x5609dc58bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.787+0000 7f9ec4ff9640 0 client.22836 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:25:36.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.787+0000 7f9ee9a91740 1 -- 172.21.15.120:0/3389122778 >> 172.21.15.120:0/3389122778 conn(0x5609dc548310 msgr2=0x5609dc581990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:36.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.787+0000 7f9ee9a91740 1 -- 172.21.15.120:0/3389122778 shutdown_connections 2024-09-19T09:25:36.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.787+0000 7f9ee9a91740 1 -- 172.21.15.120:0/3389122778 wait complete. 2024-09-19T09:25:36.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.787+0000 7f9ee9a91740 20 client.22836 trim_cache size 0 max 16384 2024-09-19T09:25:36.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.801+0000 7f9ee9a91740 1 Processor -- start 2024-09-19T09:25:36.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.801+0000 7f9ee9a91740 1 -- start start 2024-09-19T09:25:36.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.801+0000 7f9ee9a91740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5609dc57f5d0 0x5609dc57be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:36.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.801+0000 7f9ee9a91740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609dc580b40 0x5609dc5982a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:36.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.801+0000 7f9ee9a91740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609dc58c580 0x5609dc5947f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:36.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.801+0000 7f9ee9a91740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee01ab320 con 0x5609dc580b40 2024-09-19T09:25:36.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.801+0000 7f9ee9a91740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee01ace60 con 0x5609dc57f5d0 2024-09-19T09:25:36.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.801+0000 7f9ee9a91740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee01ab9f0 con 0x5609dc58c580 2024-09-19T09:25:36.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.801+0000 7f9ee4c53640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609dc580b40 0x5609dc5982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:36.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.801+0000 7f9ee4c53640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609dc580b40 0x5609dc5982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35824/0 (socket says 172.21.15.120:35824) 2024-09-19T09:25:36.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.801+0000 7f9ee4c53640 1 -- 172.21.15.120:0/633800241 learned_addr learned my addr 172.21.15.120:0/633800241 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:36.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.801+0000 7f9ee5454640 1 --2- 172.21.15.120:0/633800241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5609dc57f5d0 0x5609dc57be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:36.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.801+0000 7f9ee5c55640 1 --2- 172.21.15.120:0/633800241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609dc58c580 0x5609dc5947f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:36.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.801+0000 7f9ee4c53640 1 -- 172.21.15.120:0/633800241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609dc58c580 msgr2=0x5609dc5947f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:36.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.801+0000 7f9ee4c53640 1 --2- 172.21.15.120:0/633800241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609dc58c580 0x5609dc5947f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.801+0000 7f9ee4c53640 1 -- 172.21.15.120:0/633800241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5609dc57f5d0 msgr2=0x5609dc57be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:36.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.801+0000 7f9ee4c53640 1 --2- 172.21.15.120:0/633800241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5609dc57f5d0 0x5609dc57be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.801+0000 7f9ee4c53640 1 -- 172.21.15.120:0/633800241 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609dc5877a0 con 0x5609dc580b40 2024-09-19T09:25:36.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.802+0000 7f9ee4c53640 1 --2- 172.21.15.120:0/633800241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609dc580b40 0x5609dc5982a0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009730 tx=0x7f9ed401dfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9748d86001ca1279 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:36.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.802+0000 7f9ec6ffd640 1 -- 172.21.15.120:0/633800241 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed402b070 con 0x5609dc580b40 2024-09-19T09:25:36.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.802+0000 7f9ec6ffd640 1 -- 172.21.15.120:0/633800241 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed400fc00 con 0x5609dc580b40 2024-09-19T09:25:36.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.802+0000 7f9ec6ffd640 1 -- 172.21.15.120:0/633800241 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed40056a0 con 0x5609dc580b40 2024-09-19T09:25:36.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.803+0000 7f9ee9a91740 1 -- 172.21.15.120:0/633800241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609dc580b40 msgr2=0x5609dc5982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:36.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.803+0000 7f9ee9a91740 1 --2- 172.21.15.120:0/633800241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609dc580b40 0x5609dc5982a0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009730 tx=0x7f9ed401dfb0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.803+0000 7f9ee9a91740 1 -- 172.21.15.120:0/633800241 shutdown_connections 2024-09-19T09:25:36.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.803+0000 7f9ee9a91740 1 --2- 172.21.15.120:0/633800241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609dc58c580 0x5609dc5947f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.803+0000 7f9ee9a91740 1 --2- 172.21.15.120:0/633800241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609dc580b40 0x5609dc5982a0 unknown :-1 s=CLOSED pgs=2337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.803+0000 7f9ee9a91740 1 --2- 172.21.15.120:0/633800241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5609dc57f5d0 0x5609dc57be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.803+0000 7f9ee9a91740 1 -- 172.21.15.120:0/633800241 >> 172.21.15.120:0/633800241 conn(0x5609dc548310 msgr2=0x5609dc57b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:36.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.803+0000 7f9ee9a91740 1 -- 172.21.15.120:0/633800241 shutdown_connections 2024-09-19T09:25:36.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.803+0000 7f9ee9a91740 1 -- 172.21.15.120:0/633800241 wait complete. 2024-09-19T09:25:36.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.803+0000 7f9ee9a91740 1 -- start start 2024-09-19T09:25:36.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.803+0000 7f9ee9a91740 1 Processor -- start 2024-09-19T09:25:36.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.804+0000 7f9ee9a91740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609dc57f5d0 0x5609dc5854c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:36.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.804+0000 7f9ee9a91740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5609dc580b40 0x5609dc585a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:36.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.804+0000 7f9ee9a91740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609dc58c580 0x5609dc586060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:36.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.804+0000 7f9ee9a91740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee01ac790 con 0x5609dc57f5d0 2024-09-19T09:25:36.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.804+0000 7f9ee9a91740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee01ac0c0 con 0x5609dc580b40 2024-09-19T09:25:36.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.804+0000 7f9ee9a91740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee01a92e0 con 0x5609dc58c580 2024-09-19T09:25:36.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.804+0000 7f9ee5454640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609dc57f5d0 0x5609dc5854c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:36.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.804+0000 7f9ee9a91740 10 client.0 _ll_register_callbacks cb 0x7f9ed4009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:25:36.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.804+0000 7f9ee5c55640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609dc58c580 0x5609dc586060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:36.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.804+0000 7f9ee5c55640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609dc58c580 0x5609dc586060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:50074/0 (socket says 172.21.15.120:50074) 2024-09-19T09:25:36.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.804+0000 7f9ee5c55640 1 -- 172.21.15.120:0/3333945575 learned_addr learned my addr 172.21.15.120:0/3333945575 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:36.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.804+0000 7f9ee4c53640 1 --2- 172.21.15.120:0/3333945575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5609dc580b40 0x5609dc585a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:36.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.804+0000 7f9ee5c55640 1 -- 172.21.15.120:0/3333945575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5609dc580b40 msgr2=0x5609dc585a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:36.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.804+0000 7f9ee5c55640 1 --2- 172.21.15.120:0/3333945575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5609dc580b40 0x5609dc585a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.804+0000 7f9ee5c55640 1 -- 172.21.15.120:0/3333945575 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609dc57f5d0 msgr2=0x5609dc5854c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:36.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.804+0000 7f9ee5c55640 1 --2- 172.21.15.120:0/3333945575 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609dc57f5d0 0x5609dc5854c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.805+0000 7f9ee5c55640 1 -- 172.21.15.120:0/3333945575 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609dc586860 con 0x5609dc58c580 2024-09-19T09:25:36.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.805+0000 7f9ee5c55640 1 --2- 172.21.15.120:0/3333945575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609dc58c580 0x5609dc586060 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9ed80041c0 tx=0x7f9ed80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=800e8e3a76ff19a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:36.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.805+0000 7f9ec4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:25:36.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.805+0000 7f9ec4ff9640 1 -- 172.21.15.120:0/3333945575 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8005020 con 0x5609dc58c580 2024-09-19T09:25:36.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.805+0000 7f9ee9a91740 1 -- 172.21.15.120:0/3333945575 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ed400fc00 con 0x5609dc58c580 2024-09-19T09:25:36.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.805+0000 7f9ec4ff9640 1 -- 172.21.15.120:0/3333945575 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f9ed80051c0 con 0x5609dc58c580 2024-09-19T09:25:36.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.805+0000 7f9ec4ff9640 1 -- 172.21.15.120:0/3333945575 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9ed80053c0 con 0x5609dc58c580 2024-09-19T09:25:36.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.805+0000 7f9ee9a91740 10 client.31381 fetch_fsmap learned FSMap version 39 2024-09-19T09:25:36.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.805+0000 7f9ee9a91740 1 -- 172.21.15.120:0/3333945575 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5609dc609370 con 0x5609dc58c580 2024-09-19T09:25:36.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.806+0000 7f9ec4ff9640 1 -- 172.21.15.120:0/3333945575 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed400fc00 con 0x5609dc58c580 2024-09-19T09:25:36.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.806+0000 7f9ec4ff9640 1 -- 172.21.15.120:0/3333945575 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f9ed80038f0 con 0x5609dc58c580 2024-09-19T09:25:36.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.806+0000 7f9ee9a91740 10 client.31381 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:25:36.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.806+0000 7f9ee9a91740 10 client.31381 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:25:36.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.806+0000 7f9ee9a91740 20 client.31381 populate_metadata read hostname 'smithi120' 2024-09-19T09:25:36.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.806+0000 7f9ee9a91740 1 --2- 172.21.15.120:0/3333945575 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5609dc609b20 0x5609dc60bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:36.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.806+0000 7f9ee9a91740 4 client.31381 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:25:36.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.806+0000 7f9ee9a91740 1 -- 172.21.15.120:0/3333945575 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5609dc4ecd70 con 0x5609dc609b20 2024-09-19T09:25:36.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.806+0000 7f9ee5454640 1 --2- 172.21.15.120:0/3333945575 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5609dc609b20 0x5609dc60bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:36.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.806+0000 7f9ee5454640 1 --2- 172.21.15.120:0/3333945575 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5609dc609b20 0x5609dc60bf40 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=93711e078112e773 server_cookie=eaa83b5eeb06894c in_seq=0 out_seq=0 2024-09-19T09:25:36.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.806+0000 7f9ec4ff9640 10 client.31381 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:25:36.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.807+0000 7f9ec4ff9640 1 -- 172.21.15.120:0/3333945575 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5609dc4ecd70 con 0x5609dc609b20 2024-09-19T09:25:36.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.807+0000 7f9ec4ff9640 10 client.31381 handle_command_reply: tid=0 2024-09-19T09:25:36.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.807+0000 7f9ee9a91740 1 client.31381 shutdown 2024-09-19T09:25:36.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.807+0000 7f9ee9a91740 1 -- 172.21.15.120:0/3333945575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609dc58c580 msgr2=0x5609dc586060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:36.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.807+0000 7f9ee9a91740 1 --2- 172.21.15.120:0/3333945575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609dc58c580 0x5609dc586060 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9ed80041c0 tx=0x7f9ed80046c0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.807+0000 7f9ee9a91740 1 -- 172.21.15.120:0/3333945575 shutdown_connections 2024-09-19T09:25:36.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.807+0000 7f9ee9a91740 1 --2- 172.21.15.120:0/3333945575 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x5609dc609b20 0x5609dc60bf40 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.807+0000 7f9ee9a91740 1 --2- 172.21.15.120:0/3333945575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609dc58c580 0x5609dc586060 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.807+0000 7f9ec4ff9640 0 client.31381 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:25:36.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.807+0000 7f9ee9a91740 1 --2- 172.21.15.120:0/3333945575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5609dc580b40 0x5609dc585a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.807+0000 7f9ee9a91740 1 --2- 172.21.15.120:0/3333945575 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609dc57f5d0 0x5609dc5854c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.807+0000 7f9ee9a91740 1 -- 172.21.15.120:0/3333945575 >> 172.21.15.120:0/3333945575 conn(0x5609dc548310 msgr2=0x5609dc58aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:36.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.807+0000 7f9ee9a91740 1 -- 172.21.15.120:0/3333945575 shutdown_connections 2024-09-19T09:25:36.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.807+0000 7f9ee9a91740 1 -- 172.21.15.120:0/3333945575 wait complete. 2024-09-19T09:25:36.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.807+0000 7f9ee9a91740 20 client.31381 trim_cache size 0 max 16384 2024-09-19T09:25:36.813 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:25:36.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.808+0000 7f9ee76b8640 1 -- 172.21.15.120:0/3622813878 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9ec80a3240 msgr2=0x7f9ec80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:36.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.808+0000 7f9ee76b8640 1 --2- 172.21.15.120:0/3622813878 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9ec80a3240 0x7f9ec80a5700 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8005f60 tx=0x7f9ed8005f90 comp rx=0 tx=0).stop 2024-09-19T09:25:36.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.808+0000 7f9ee76b8640 1 -- 172.21.15.120:0/3622813878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ee0100a90 msgr2=0x7f9ee0197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:36.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.808+0000 7f9ee76b8640 1 --2- 172.21.15.120:0/3622813878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ee0100a90 0x7f9ee0197350 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7f9ed002ae00 tx=0x7f9ed0031e60 comp rx=0 tx=0).stop 2024-09-19T09:25:36.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.808+0000 7f9ee76b8640 1 -- 172.21.15.120:0/3622813878 shutdown_connections 2024-09-19T09:25:36.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.808+0000 7f9ee76b8640 1 --2- 172.21.15.120:0/3622813878 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f9ec80a3240 0x7f9ec80a5700 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.808+0000 7f9ee76b8640 1 --2- 172.21.15.120:0/3622813878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ee0106a90 0x7f9ee019bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.808+0000 7f9ee76b8640 1 --2- 172.21.15.120:0/3622813878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ee0101430 0x7f9ee0197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.808+0000 7f9ee76b8640 1 --2- 172.21.15.120:0/3622813878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ee0100a90 0x7f9ee0197350 unknown :-1 s=CLOSED pgs=2334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.808+0000 7f9ee76b8640 1 -- 172.21.15.120:0/3622813878 >> 172.21.15.120:0/3622813878 conn(0x7f9ee00fc7d0 msgr2=0x7f9ee00fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:36.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.808+0000 7f9ee76b8640 1 -- 172.21.15.120:0/3622813878 shutdown_connections 2024-09-19T09:25:36.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:36.808+0000 7f9ee76b8640 1 -- 172.21.15.120:0/3622813878 wait complete. 2024-09-19T09:25:37.384 INFO:tasks.workunit.client.0.smithi120.stdout: 10 223955 20.10 MB/sec execute 452 sec latency 195.635 ms 2024-09-19T09:25:37.678 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:25:37.973 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:38.380 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:38 smithi120 ceph-mon[31150]: pgmap v1028: 97 pgs: 97 active+clean; 487 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.9 MiB/s wr, 675 op/s 2024-09-19T09:25:38.384 INFO:tasks.workunit.client.0.smithi120.stdout: 10 224449 20.11 MB/sec execute 453 sec latency 104.264 ms 2024-09-19T09:25:38.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:38 smithi196 ceph-mon[35150]: pgmap v1028: 97 pgs: 97 active+clean; 487 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.9 MiB/s wr, 675 op/s 2024-09-19T09:25:38.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:38 smithi153 ceph-mon[35071]: pgmap v1028: 97 pgs: 97 active+clean; 487 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.9 MiB/s wr, 675 op/s 2024-09-19T09:25:38.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.467+0000 7f6b18d08640 1 -- 172.21.15.120:0/980590589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b14100e20 msgr2=0x7f6b14101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:38.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.467+0000 7f6b18d08640 1 --2- 172.21.15.120:0/980590589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b14100e20 0x7f6b14101200 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7f6b04009900 tx=0x7f6b0402a480 comp rx=0 tx=0).stop 2024-09-19T09:25:38.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.467+0000 7f6b18d08640 1 -- 172.21.15.120:0/980590589 shutdown_connections 2024-09-19T09:25:38.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.467+0000 7f6b18d08640 1 --2- 172.21.15.120:0/980590589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b14102ef0 0x7f6b1410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.467+0000 7f6b18d08640 1 --2- 172.21.15.120:0/980590589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b14102570 0x7f6b141029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.467+0000 7f6b18d08640 1 --2- 172.21.15.120:0/980590589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b14100e20 0x7f6b14101200 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.467+0000 7f6b18d08640 1 -- 172.21.15.120:0/980590589 >> 172.21.15.120:0/980590589 conn(0x7f6b140fc7d0 msgr2=0x7f6b140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:38.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.467+0000 7f6b18d08640 1 -- 172.21.15.120:0/980590589 shutdown_connections 2024-09-19T09:25:38.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.467+0000 7f6b18d08640 1 -- 172.21.15.120:0/980590589 wait complete. 2024-09-19T09:25:38.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.468+0000 7f6b18d08640 1 Processor -- start 2024-09-19T09:25:38.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.468+0000 7f6b18d08640 1 -- start start 2024-09-19T09:25:38.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.468+0000 7f6b18d08640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b14100e20 0x7f6b140fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:38.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.468+0000 7f6b18d08640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b14102570 0x7f6b140fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:38.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.468+0000 7f6b18d08640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b14102ef0 0x7f6b140fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:38.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.468+0000 7f6b18d08640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6b14110f80 con 0x7f6b14102570 2024-09-19T09:25:38.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.468+0000 7f6b18d08640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6b14110e00 con 0x7f6b14100e20 2024-09-19T09:25:38.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.468+0000 7f6b18d08640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6b14111100 con 0x7f6b14102ef0 2024-09-19T09:25:38.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.468+0000 7f6b11d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b14102570 0x7f6b140fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:38.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.468+0000 7f6b1259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b14100e20 0x7f6b140fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:38.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.468+0000 7f6b11d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b14102570 0x7f6b140fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35856/0 (socket says 172.21.15.120:35856) 2024-09-19T09:25:38.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.468+0000 7f6b11d9b640 1 -- 172.21.15.120:0/4022795047 learned_addr learned my addr 172.21.15.120:0/4022795047 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:38.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.468+0000 7f6b12d9d640 1 --2- 172.21.15.120:0/4022795047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b14102ef0 0x7f6b140fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:38.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.469+0000 7f6b11d9b640 1 -- 172.21.15.120:0/4022795047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b14102ef0 msgr2=0x7f6b140fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:38.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.469+0000 7f6b11d9b640 1 --2- 172.21.15.120:0/4022795047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b14102ef0 0x7f6b140fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.469+0000 7f6b11d9b640 1 -- 172.21.15.120:0/4022795047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b14100e20 msgr2=0x7f6b140fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:38.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.469+0000 7f6b11d9b640 1 --2- 172.21.15.120:0/4022795047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b14100e20 0x7f6b140fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.469+0000 7f6b11d9b640 1 -- 172.21.15.120:0/4022795047 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b141aaeb0 con 0x7f6b14102570 2024-09-19T09:25:38.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.469+0000 7f6b11d9b640 1 --2- 172.21.15.120:0/4022795047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b14102570 0x7f6b140fd1c0 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800e9f0 tx=0x7f6b0800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:38.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.469+0000 7f6b037fe640 1 -- 172.21.15.120:0/4022795047 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0800cd80 con 0x7f6b14102570 2024-09-19T09:25:38.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.469+0000 7f6b037fe640 1 -- 172.21.15.120:0/4022795047 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b08005500 con 0x7f6b14102570 2024-09-19T09:25:38.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.469+0000 7f6b18d08640 1 -- 172.21.15.120:0/4022795047 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b141ab0e0 con 0x7f6b14102570 2024-09-19T09:25:38.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.469+0000 7f6b037fe640 1 -- 172.21.15.120:0/4022795047 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08010690 con 0x7f6b14102570 2024-09-19T09:25:38.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.469+0000 7f6b18d08640 1 -- 172.21.15.120:0/4022795047 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b141ab5d0 con 0x7f6b14102570 2024-09-19T09:25:38.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.471+0000 7f6b037fe640 1 -- 172.21.15.120:0/4022795047 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f6b080046c0 con 0x7f6b14102570 2024-09-19T09:25:38.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.472+0000 7f6b1b0e1740 1 Processor -- start 2024-09-19T09:25:38.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.472+0000 7f6b1b0e1740 1 -- start start 2024-09-19T09:25:38.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.472+0000 7f6b1b0e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f312987b40 0x55f312986090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:38.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.472+0000 7f6b1b0e1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3129865d0 0x55f312986a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:38.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.472+0000 7f6b1b0e1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f312991370 0x55f312993760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:38.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.472+0000 7f6b1b0e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f3128ec2f0 con 0x55f312987b40 2024-09-19T09:25:38.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.472+0000 7f6b1b0e1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f312893cf0 con 0x55f312991370 2024-09-19T09:25:38.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.472+0000 7f6b1b0e1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f3128e82b0 con 0x55f3129865d0 2024-09-19T09:25:38.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.472+0000 7f6b12d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f312987b40 0x55f312986090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:38.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.472+0000 7f6b12d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f312987b40 0x55f312986090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35868/0 (socket says 172.21.15.120:35868) 2024-09-19T09:25:38.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.472+0000 7f6b1259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3129865d0 0x55f312986a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:38.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.472+0000 7f6b12d9d640 1 -- 172.21.15.120:0/2675766228 learned_addr learned my addr 172.21.15.120:0/2675766228 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:38.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.472+0000 7f6b11d9b640 1 --2- 172.21.15.120:0/2675766228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f312991370 0x55f312993760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:38.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.472+0000 7f6b037fe640 1 --2- 172.21.15.120:0/4022795047 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6afc0a3240 0x7f6afc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:38.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.473+0000 7f6b12d9d640 1 --2- 172.21.15.120:0/4022795047 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6afc0a3240 0x7f6afc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:38.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.473+0000 7f6b12d9d640 1 -- 172.21.15.120:0/2675766228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3129865d0 msgr2=0x55f312986a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:38.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.473+0000 7f6b12d9d640 1 --2- 172.21.15.120:0/2675766228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3129865d0 0x55f312986a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.473+0000 7f6b12d9d640 1 -- 172.21.15.120:0/2675766228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f312991370 msgr2=0x55f312993760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:38.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.473+0000 7f6b12d9d640 1 --2- 172.21.15.120:0/2675766228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f312991370 0x55f312993760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.473+0000 7f6b12d9d640 1 -- 172.21.15.120:0/2675766228 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3128783a0 con 0x55f312987b40 2024-09-19T09:25:38.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.473+0000 7f6b11d9b640 1 --2- 172.21.15.120:0/2675766228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f312991370 0x55f312993760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:25:38.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.473+0000 7f6b037fe640 1 -- 172.21.15.120:0/4022795047 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f6b080cfd90 con 0x7f6b14102570 2024-09-19T09:25:38.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.473+0000 7f6b12d9d640 1 --2- 172.21.15.120:0/2675766228 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f312987b40 0x55f312986090 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c005c40 tx=0x7f6b0c013f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97933036223a3a47 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:38.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.473+0000 7f6b00ff9640 1 -- 172.21.15.120:0/2675766228 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c022020 con 0x55f312987b40 2024-09-19T09:25:38.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.473+0000 7f6b00ff9640 1 -- 172.21.15.120:0/2675766228 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b0c0134e0 con 0x55f312987b40 2024-09-19T09:25:38.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.473+0000 7f6b00ff9640 1 -- 172.21.15.120:0/2675766228 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c01e9d0 con 0x55f312987b40 2024-09-19T09:25:38.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.473+0000 7f6b12d9d640 1 --2- 172.21.15.120:0/4022795047 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6afc0a3240 0x7f6afc0a5700 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c023000 tx=0x7f6b0c021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:38.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.475+0000 7f6b1b0e1740 1 -- 172.21.15.120:0/2675766228 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f312987b40 msgr2=0x55f312986090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:38.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.475+0000 7f6b1b0e1740 1 --2- 172.21.15.120:0/2675766228 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f312987b40 0x55f312986090 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c005c40 tx=0x7f6b0c013f50 comp rx=0 tx=0).stop 2024-09-19T09:25:38.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.475+0000 7f6b1b0e1740 1 -- 172.21.15.120:0/2675766228 shutdown_connections 2024-09-19T09:25:38.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.475+0000 7f6b1b0e1740 1 --2- 172.21.15.120:0/2675766228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f312991370 0x55f312993760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.475+0000 7f6b1b0e1740 1 --2- 172.21.15.120:0/2675766228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3129865d0 0x55f312986a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.475+0000 7f6b1b0e1740 1 --2- 172.21.15.120:0/2675766228 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f312987b40 0x55f312986090 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.475+0000 7f6b1b0e1740 1 -- 172.21.15.120:0/2675766228 >> 172.21.15.120:0/2675766228 conn(0x55f31294f310 msgr2=0x55f312982c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:38.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.475+0000 7f6b1b0e1740 1 -- 172.21.15.120:0/2675766228 shutdown_connections 2024-09-19T09:25:38.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.475+0000 7f6b1b0e1740 1 -- 172.21.15.120:0/2675766228 wait complete. 2024-09-19T09:25:38.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.476+0000 7f6b1b0e1740 1 -- start start 2024-09-19T09:25:38.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.476+0000 7f6b1b0e1740 1 Processor -- start 2024-09-19T09:25:38.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.476+0000 7f6b1b0e1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3129865d0 0x55f312990d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:38.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.476+0000 7f6b1b0e1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f312987b40 0x55f31298e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:38.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.476+0000 7f6b1b0e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f312991370 0x55f31298e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:38.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.476+0000 7f6b1b0e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f312994070 con 0x55f312991370 2024-09-19T09:25:38.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.476+0000 7f6b1b0e1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f3128a7060 con 0x55f3129865d0 2024-09-19T09:25:38.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.476+0000 7f6b1b0e1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f3129941f0 con 0x55f312987b40 2024-09-19T09:25:38.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.476+0000 7f6b1b0e1740 10 client.0 _ll_register_callbacks cb 0x55f3125fa8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:25:38.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.476+0000 7f6b11d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f312987b40 0x55f31298e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:38.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.476+0000 7f6b11d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f312987b40 0x55f31298e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:50102/0 (socket says 172.21.15.120:50102) 2024-09-19T09:25:38.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.476+0000 7f6b11d9b640 1 -- 172.21.15.120:0/677140452 learned_addr learned my addr 172.21.15.120:0/677140452 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:38.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.476+0000 7f6b12d9d640 1 --2- 172.21.15.120:0/677140452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f312991370 0x55f31298e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:38.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.477+0000 7f6b1259c640 1 --2- 172.21.15.120:0/677140452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3129865d0 0x55f312990d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:38.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.477+0000 7f6b12d9d640 1 -- 172.21.15.120:0/677140452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f312987b40 msgr2=0x55f31298e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:38.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.477+0000 7f6b12d9d640 1 --2- 172.21.15.120:0/677140452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f312987b40 0x55f31298e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.477+0000 7f6b12d9d640 1 -- 172.21.15.120:0/677140452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3129865d0 msgr2=0x55f312990d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:38.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.477+0000 7f6b12d9d640 1 --2- 172.21.15.120:0/677140452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3129865d0 0x55f312990d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.477+0000 7f6b12d9d640 1 -- 172.21.15.120:0/677140452 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f31265d3c0 con 0x55f312991370 2024-09-19T09:25:38.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.477+0000 7f6b11d9b640 1 --2- 172.21.15.120:0/677140452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f312987b40 0x55f31298e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:25:38.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.477+0000 7f6b1259c640 1 --2- 172.21.15.120:0/677140452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3129865d0 0x55f312990d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:25:38.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.477+0000 7f6b12d9d640 1 --2- 172.21.15.120:0/677140452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f312991370 0x55f31298e5d0 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c005c10 tx=0x7f6b0c012e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c9b61d1c7fb28e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:38.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.477+0000 7f6af9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:25:38.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.477+0000 7f6af9ffb640 1 -- 172.21.15.120:0/677140452 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c022040 con 0x55f312991370 2024-09-19T09:25:38.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.477+0000 7f6af9ffb640 1 -- 172.21.15.120:0/677140452 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f6b0c012220 con 0x55f312991370 2024-09-19T09:25:38.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.477+0000 7f6b1b0e1740 1 -- 172.21.15.120:0/677140452 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f3128f9860 con 0x55f312991370 2024-09-19T09:25:38.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.478+0000 7f6af9ffb640 1 -- 172.21.15.120:0/677140452 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6b0c004a00 con 0x55f312991370 2024-09-19T09:25:38.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.478+0000 7f6b1b0e1740 10 client.22854 fetch_fsmap learned FSMap version 39 2024-09-19T09:25:38.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.478+0000 7f6b1b0e1740 1 -- 172.21.15.120:0/677140452 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f31299faf0 con 0x55f312991370 2024-09-19T09:25:38.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.478+0000 7f6af9ffb640 1 -- 172.21.15.120:0/677140452 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f3128f9860 con 0x55f312991370 2024-09-19T09:25:38.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.478+0000 7f6af9ffb640 1 -- 172.21.15.120:0/677140452 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f6b0c005050 con 0x55f312991370 2024-09-19T09:25:38.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.478+0000 7f6b1b0e1740 10 client.22854 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:25:38.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.478+0000 7f6b1b0e1740 10 client.22854 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:25:38.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.478+0000 7f6b1b0e1740 20 client.22854 populate_metadata read hostname 'smithi120' 2024-09-19T09:25:38.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.478+0000 7f6b1b0e1740 1 --2- 172.21.15.120:0/677140452 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55f3129a1280 0x55f3129a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:38.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.478+0000 7f6b1b0e1740 4 client.22854 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:25:38.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.478+0000 7f6b1b0e1740 1 -- 172.21.15.120:0/677140452 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3128f3d70 con 0x55f3129a1280 2024-09-19T09:25:38.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.478+0000 7f6b1259c640 1 --2- 172.21.15.120:0/677140452 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55f3129a1280 0x55f3129a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:38.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.479+0000 7f6af9ffb640 10 client.22854 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:25:38.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.480+0000 7f6b1259c640 1 --2- 172.21.15.120:0/677140452 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55f3129a1280 0x55f3129a36d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c7cc7b18641c7c04 server_cookie=df79f2ae8072e7a9 in_seq=0 out_seq=0 2024-09-19T09:25:38.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.480+0000 7f6af9ffb640 1 -- 172.21.15.120:0/677140452 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f3128f3d70 con 0x55f3129a1280 2024-09-19T09:25:38.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.480+0000 7f6af9ffb640 10 client.22854 handle_command_reply: tid=0 2024-09-19T09:25:38.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.480+0000 7f6b1b0e1740 1 client.22854 shutdown 2024-09-19T09:25:38.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.481+0000 7f6b1b0e1740 1 -- 172.21.15.120:0/677140452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f312991370 msgr2=0x55f31298e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:38.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.481+0000 7f6b1b0e1740 1 --2- 172.21.15.120:0/677140452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f312991370 0x55f31298e5d0 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c005c10 tx=0x7f6b0c012e10 comp rx=0 tx=0).stop 2024-09-19T09:25:38.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.481+0000 7f6b1b0e1740 1 -- 172.21.15.120:0/677140452 shutdown_connections 2024-09-19T09:25:38.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.481+0000 7f6b1b0e1740 1 --2- 172.21.15.120:0/677140452 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55f3129a1280 0x55f3129a36d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.481+0000 7f6b1b0e1740 1 --2- 172.21.15.120:0/677140452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f312991370 0x55f31298e5d0 unknown :-1 s=CLOSED pgs=2341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.481+0000 7f6b1b0e1740 1 --2- 172.21.15.120:0/677140452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f312987b40 0x55f31298e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.481+0000 7f6b1b0e1740 1 --2- 172.21.15.120:0/677140452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3129865d0 0x55f312990d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.481+0000 7f6b1b0e1740 1 -- 172.21.15.120:0/677140452 >> 172.21.15.120:0/677140452 conn(0x55f31294f310 msgr2=0x55f312988960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:38.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.481+0000 7f6af9ffb640 0 client.22854 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:25:38.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.481+0000 7f6b1b0e1740 1 -- 172.21.15.120:0/677140452 shutdown_connections 2024-09-19T09:25:38.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.481+0000 7f6b1b0e1740 1 -- 172.21.15.120:0/677140452 wait complete. 2024-09-19T09:25:38.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.481+0000 7f6b1b0e1740 20 client.22854 trim_cache size 0 max 16384 2024-09-19T09:25:38.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.495+0000 7f6b1b0e1740 1 Processor -- start 2024-09-19T09:25:38.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.495+0000 7f6b1b0e1740 1 -- start start 2024-09-19T09:25:38.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.496+0000 7f6b1b0e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3129865d0 0x55f31299d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:38.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.496+0000 7f6b1b0e1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f312987b40 0x55f31298c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:38.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.496+0000 7f6b1b0e1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f312991370 0x55f31298ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:38.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.496+0000 7f6b1b0e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6b141b3b70 con 0x55f3129865d0 2024-09-19T09:25:38.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.496+0000 7f6b1b0e1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6b141b56b0 con 0x55f312987b40 2024-09-19T09:25:38.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.496+0000 7f6b1b0e1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6b141b4240 con 0x55f312991370 2024-09-19T09:25:38.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.496+0000 7f6b1259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3129865d0 0x55f31299d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:38.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.496+0000 7f6b1259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3129865d0 0x55f31299d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35886/0 (socket says 172.21.15.120:35886) 2024-09-19T09:25:38.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.496+0000 7f6b1259c640 1 -- 172.21.15.120:0/1394028522 learned_addr learned my addr 172.21.15.120:0/1394028522 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:38.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.496+0000 7f6b11d9b640 1 --2- 172.21.15.120:0/1394028522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f312987b40 0x55f31298c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:38.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.496+0000 7f6b12d9d640 1 --2- 172.21.15.120:0/1394028522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f312991370 0x55f31298ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:38.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.496+0000 7f6b1259c640 1 -- 172.21.15.120:0/1394028522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f312991370 msgr2=0x55f31298ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:38.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.496+0000 7f6b1259c640 1 --2- 172.21.15.120:0/1394028522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f312991370 0x55f31298ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.496+0000 7f6b1259c640 1 -- 172.21.15.120:0/1394028522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f312987b40 msgr2=0x55f31298c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:38.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.496+0000 7f6b1259c640 1 --2- 172.21.15.120:0/1394028522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f312987b40 0x55f31298c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.496+0000 7f6b1259c640 1 -- 172.21.15.120:0/1394028522 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f312982ed0 con 0x55f3129865d0 2024-09-19T09:25:38.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.496+0000 7f6b11d9b640 1 --2- 172.21.15.120:0/1394028522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f312987b40 0x55f31298c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:25:38.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.496+0000 7f6b1259c640 1 --2- 172.21.15.120:0/1394028522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3129865d0 0x55f31299d310 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f6b04009900 tx=0x7f6b04004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c107bc0ac1791106 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:38.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.497+0000 7f6afbfff640 1 -- 172.21.15.120:0/1394028522 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b04035040 con 0x55f3129865d0 2024-09-19T09:25:38.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.497+0000 7f6afbfff640 1 -- 172.21.15.120:0/1394028522 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b04005500 con 0x55f3129865d0 2024-09-19T09:25:38.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.497+0000 7f6afbfff640 1 -- 172.21.15.120:0/1394028522 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b040466e0 con 0x55f3129865d0 2024-09-19T09:25:38.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.497+0000 7f6b1b0e1740 1 -- 172.21.15.120:0/1394028522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3129865d0 msgr2=0x55f31299d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:38.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.497+0000 7f6b1b0e1740 1 --2- 172.21.15.120:0/1394028522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3129865d0 0x55f31299d310 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f6b04009900 tx=0x7f6b04004320 comp rx=0 tx=0).stop 2024-09-19T09:25:38.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.497+0000 7f6b1b0e1740 1 -- 172.21.15.120:0/1394028522 shutdown_connections 2024-09-19T09:25:38.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.497+0000 7f6b1b0e1740 1 --2- 172.21.15.120:0/1394028522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f312991370 0x55f31298ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.497+0000 7f6b1b0e1740 1 --2- 172.21.15.120:0/1394028522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f312987b40 0x55f31298c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.497+0000 7f6b1b0e1740 1 --2- 172.21.15.120:0/1394028522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3129865d0 0x55f31299d310 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.498+0000 7f6b1b0e1740 1 -- 172.21.15.120:0/1394028522 >> 172.21.15.120:0/1394028522 conn(0x55f31294f310 msgr2=0x55f31298dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:38.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.498+0000 7f6b1b0e1740 1 -- 172.21.15.120:0/1394028522 shutdown_connections 2024-09-19T09:25:38.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.498+0000 7f6b1b0e1740 1 -- 172.21.15.120:0/1394028522 wait complete. 2024-09-19T09:25:38.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.498+0000 7f6b1b0e1740 1 -- start start 2024-09-19T09:25:38.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.498+0000 7f6b1b0e1740 1 Processor -- start 2024-09-19T09:25:38.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.499+0000 7f6b1b0e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f312987b40 0x55f312999c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:38.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.499+0000 7f6b1b0e1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f312991370 0x55f312994440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:38.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.499+0000 7f6b1b0e1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3129949d0 0x55f312994e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:38.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.499+0000 7f6b1b0e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6b141b4fe0 con 0x55f312987b40 2024-09-19T09:25:38.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.499+0000 7f6b1b0e1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6b141b4910 con 0x55f312991370 2024-09-19T09:25:38.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.499+0000 7f6b1259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f312987b40 0x55f312999c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:38.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.499+0000 7f6b11d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f312991370 0x55f312994440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:38.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.499+0000 7f6b11d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f312991370 0x55f312994440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:36942/0 (socket says 172.21.15.120:36942) 2024-09-19T09:25:38.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.499+0000 7f6b11d9b640 1 -- 172.21.15.120:0/2535740395 learned_addr learned my addr 172.21.15.120:0/2535740395 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:38.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.499+0000 7f6b1b0e1740 1 -- 172.21.15.120:0/2535740395 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6b141b1b30 con 0x55f3129949d0 2024-09-19T09:25:38.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.499+0000 7f6b1b0e1740 10 client.0 _ll_register_callbacks cb 0x7f6b0809c180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:25:38.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.499+0000 7f6b12d9d640 1 --2- 172.21.15.120:0/2535740395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3129949d0 0x55f312994e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:38.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.500+0000 7f6b11d9b640 1 -- 172.21.15.120:0/2535740395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3129949d0 msgr2=0x55f312994e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:38.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.500+0000 7f6b11d9b640 1 --2- 172.21.15.120:0/2535740395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3129949d0 0x55f312994e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.500+0000 7f6b11d9b640 1 -- 172.21.15.120:0/2535740395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f312987b40 msgr2=0x55f312999c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:38.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.500+0000 7f6b11d9b640 1 --2- 172.21.15.120:0/2535740395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f312987b40 0x55f312999c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.500+0000 7f6b11d9b640 1 -- 172.21.15.120:0/2535740395 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f312a0a170 con 0x55f312991370 2024-09-19T09:25:38.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.500+0000 7f6b1259c640 1 --2- 172.21.15.120:0/2535740395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f312987b40 0x55f312999c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:25:38.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.500+0000 7f6b11d9b640 1 --2- 172.21.15.120:0/2535740395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f312991370 0x55f312994440 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6b080cd860 tx=0x7f6b08009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2098f861335213c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:38.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.500+0000 7f6af9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:25:38.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.500+0000 7f6af9ffb640 1 -- 172.21.15.120:0/2535740395 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0809bd10 con 0x55f312991370 2024-09-19T09:25:38.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.500+0000 7f6af9ffb640 1 -- 172.21.15.120:0/2535740395 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f6b0809ba00 con 0x55f312991370 2024-09-19T09:25:38.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.500+0000 7f6b1b0e1740 1 -- 172.21.15.120:0/2535740395 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b04005500 con 0x55f312991370 2024-09-19T09:25:38.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.500+0000 7f6af9ffb640 1 -- 172.21.15.120:0/2535740395 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6b0809cab0 con 0x55f312991370 2024-09-19T09:25:38.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.501+0000 7f6b1b0e1740 10 client.31535 fetch_fsmap learned FSMap version 39 2024-09-19T09:25:38.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.501+0000 7f6b1b0e1740 1 -- 172.21.15.120:0/2535740395 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f312a0b440 con 0x55f312991370 2024-09-19T09:25:38.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.501+0000 7f6af9ffb640 1 -- 172.21.15.120:0/2535740395 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b04005500 con 0x55f312991370 2024-09-19T09:25:38.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.501+0000 7f6af9ffb640 1 -- 172.21.15.120:0/2535740395 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f6b0809b270 con 0x55f312991370 2024-09-19T09:25:38.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.501+0000 7f6b1b0e1740 10 client.31535 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:25:38.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.501+0000 7f6b1b0e1740 10 client.31535 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:25:38.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.501+0000 7f6b1b0e1740 20 client.31535 populate_metadata read hostname 'smithi120' 2024-09-19T09:25:38.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.501+0000 7f6b1b0e1740 1 --2- 172.21.15.120:0/2535740395 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55f312a0bbe0 0x55f312a0e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:38.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.501+0000 7f6b1b0e1740 4 client.31535 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:25:38.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.501+0000 7f6b1b0e1740 1 -- 172.21.15.120:0/2535740395 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f312870170 con 0x55f312a0bbe0 2024-09-19T09:25:38.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.502+0000 7f6b1259c640 1 --2- 172.21.15.120:0/2535740395 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55f312a0bbe0 0x55f312a0e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:38.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.502+0000 7f6af9ffb640 10 client.31535 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:25:38.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.502+0000 7f6b1259c640 1 --2- 172.21.15.120:0/2535740395 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55f312a0bbe0 0x55f312a0e000 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=912b29f68cf5a534 server_cookie=1d6a5a95ed1a163 in_seq=0 out_seq=0 2024-09-19T09:25:38.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.502+0000 7f6af9ffb640 1 -- 172.21.15.120:0/2535740395 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f312870170 con 0x55f312a0bbe0 2024-09-19T09:25:38.659 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:25:38.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.502+0000 7f6af9ffb640 10 client.31535 handle_command_reply: tid=0 2024-09-19T09:25:38.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.502+0000 7f6b1b0e1740 1 client.31535 shutdown 2024-09-19T09:25:38.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.502+0000 7f6b1b0e1740 1 -- 172.21.15.120:0/2535740395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f312991370 msgr2=0x55f312994440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:38.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.502+0000 7f6b1b0e1740 1 --2- 172.21.15.120:0/2535740395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f312991370 0x55f312994440 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6b080cd860 tx=0x7f6b08009310 comp rx=0 tx=0).stop 2024-09-19T09:25:38.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.503+0000 7f6b1b0e1740 1 -- 172.21.15.120:0/2535740395 shutdown_connections 2024-09-19T09:25:38.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.503+0000 7f6b1b0e1740 1 --2- 172.21.15.120:0/2535740395 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55f312a0bbe0 0x55f312a0e000 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.503+0000 7f6b1b0e1740 1 --2- 172.21.15.120:0/2535740395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3129949d0 0x55f312994e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.503+0000 7f6b1b0e1740 1 --2- 172.21.15.120:0/2535740395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f312991370 0x55f312994440 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.503+0000 7f6af9ffb640 0 client.31535 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:25:38.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.503+0000 7f6b1b0e1740 1 --2- 172.21.15.120:0/2535740395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f312987b40 0x55f312999c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.503+0000 7f6b1b0e1740 1 -- 172.21.15.120:0/2535740395 >> 172.21.15.120:0/2535740395 conn(0x55f31294f310 msgr2=0x55f31298dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:38.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.503+0000 7f6b1b0e1740 1 -- 172.21.15.120:0/2535740395 shutdown_connections 2024-09-19T09:25:38.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.503+0000 7f6b1b0e1740 1 -- 172.21.15.120:0/2535740395 wait complete. 2024-09-19T09:25:38.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.503+0000 7f6b1b0e1740 20 client.31535 trim_cache size 0 max 16384 2024-09-19T09:25:38.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.503+0000 7f6b18d08640 1 -- 172.21.15.120:0/4022795047 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6afc0a3240 msgr2=0x7f6afc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:38.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.503+0000 7f6b18d08640 1 --2- 172.21.15.120:0/4022795047 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6afc0a3240 0x7f6afc0a5700 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c023000 tx=0x7f6b0c021000 comp rx=0 tx=0).stop 2024-09-19T09:25:38.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.503+0000 7f6b18d08640 1 -- 172.21.15.120:0/4022795047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b14102570 msgr2=0x7f6b140fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:38.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.503+0000 7f6b18d08640 1 --2- 172.21.15.120:0/4022795047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b14102570 0x7f6b140fd1c0 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800e9f0 tx=0x7f6b0800edf0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.503+0000 7f6b18d08640 1 -- 172.21.15.120:0/4022795047 shutdown_connections 2024-09-19T09:25:38.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.503+0000 7f6b18d08640 1 --2- 172.21.15.120:0/4022795047 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6afc0a3240 0x7f6afc0a5700 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.503+0000 7f6b18d08640 1 --2- 172.21.15.120:0/4022795047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b14102ef0 0x7f6b140fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.503+0000 7f6b18d08640 1 --2- 172.21.15.120:0/4022795047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b14102570 0x7f6b140fd1c0 unknown :-1 s=CLOSED pgs=2339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.503+0000 7f6b18d08640 1 --2- 172.21.15.120:0/4022795047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b14100e20 0x7f6b140fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.503+0000 7f6b18d08640 1 -- 172.21.15.120:0/4022795047 >> 172.21.15.120:0/4022795047 conn(0x7f6b140fc7d0 msgr2=0x7f6b1406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:38.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.504+0000 7f6b18d08640 1 -- 172.21.15.120:0/4022795047 shutdown_connections 2024-09-19T09:25:38.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:38.504+0000 7f6b18d08640 1 -- 172.21.15.120:0/4022795047 wait complete. 2024-09-19T09:25:39.384 INFO:tasks.workunit.client.0.smithi120.stdout: 10 224983 20.11 MB/sec execute 454 sec latency 75.966 ms 2024-09-19T09:25:39.502 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:25:39.786 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:40.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:40 smithi120 ceph-mon[31150]: pgmap v1029: 97 pgs: 97 active+clean; 488 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.7 MiB/s wr, 712 op/s 2024-09-19T09:25:40.384 INFO:tasks.workunit.client.0.smithi120.stdout: 10 225662 20.14 MB/sec execute 455 sec latency 102.733 ms 2024-09-19T09:25:40.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:40 smithi196 ceph-mon[35150]: pgmap v1029: 97 pgs: 97 active+clean; 488 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.7 MiB/s wr, 712 op/s 2024-09-19T09:25:40.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:40 smithi153 ceph-mon[35071]: pgmap v1029: 97 pgs: 97 active+clean; 488 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.7 MiB/s wr, 712 op/s 2024-09-19T09:25:41.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.322+0000 7f77ba161640 1 -- 172.21.15.120:0/1614313250 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77b4100e20 msgr2=0x7f77b4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:41.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.322+0000 7f77ba161640 1 --2- 172.21.15.120:0/1614313250 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77b4100e20 0x7f77b4101200 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f77a4009900 tx=0x7f77a402a480 comp rx=0 tx=0).stop 2024-09-19T09:25:41.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.322+0000 7f77ba161640 1 -- 172.21.15.120:0/1614313250 shutdown_connections 2024-09-19T09:25:41.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.322+0000 7f77ba161640 1 --2- 172.21.15.120:0/1614313250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77b4102ef0 0x7f77b410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.322+0000 7f77ba161640 1 --2- 172.21.15.120:0/1614313250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77b4102570 0x7f77b41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.322+0000 7f77ba161640 1 --2- 172.21.15.120:0/1614313250 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77b4100e20 0x7f77b4101200 unknown :-1 s=CLOSED pgs=2343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.322+0000 7f77ba161640 1 -- 172.21.15.120:0/1614313250 >> 172.21.15.120:0/1614313250 conn(0x7f77b40fc7d0 msgr2=0x7f77b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:41.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.322+0000 7f77ba161640 1 -- 172.21.15.120:0/1614313250 shutdown_connections 2024-09-19T09:25:41.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.322+0000 7f77ba161640 1 -- 172.21.15.120:0/1614313250 wait complete. 2024-09-19T09:25:41.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.323+0000 7f77ba161640 1 Processor -- start 2024-09-19T09:25:41.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.323+0000 7f77ba161640 1 -- start start 2024-09-19T09:25:41.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.323+0000 7f77ba161640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77b4100e20 0x7f77b4198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:41.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.323+0000 7f77ba161640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77b4102570 0x7f77b4198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:41.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.323+0000 7f77ba161640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77b4102ef0 0x7f77b419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:41.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.324+0000 7f77ba161640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f77b4110f80 con 0x7f77b4102ef0 2024-09-19T09:25:41.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.324+0000 7f77ba161640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f77b4110e00 con 0x7f77b4102570 2024-09-19T09:25:41.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.324+0000 7f77ba161640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f77b4111100 con 0x7f77b4100e20 2024-09-19T09:25:41.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.324+0000 7f77b3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77b4102ef0 0x7f77b419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:41.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.324+0000 7f77b37fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77b4100e20 0x7f77b4198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:41.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.324+0000 7f77b37fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77b4100e20 0x7f77b4198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:58182/0 (socket says 172.21.15.120:58182) 2024-09-19T09:25:41.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.324+0000 7f77b37fe640 1 -- 172.21.15.120:0/3890138102 learned_addr learned my addr 172.21.15.120:0/3890138102 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:41.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.324+0000 7f77b2ffd640 1 --2- 172.21.15.120:0/3890138102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77b4102570 0x7f77b4198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:41.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.324+0000 7f77b37fe640 1 -- 172.21.15.120:0/3890138102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77b4102570 msgr2=0x7f77b4198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:41.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.324+0000 7f77b37fe640 1 --2- 172.21.15.120:0/3890138102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77b4102570 0x7f77b4198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.324+0000 7f77b37fe640 1 -- 172.21.15.120:0/3890138102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77b4102ef0 msgr2=0x7f77b419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:41.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.324+0000 7f77b37fe640 1 --2- 172.21.15.120:0/3890138102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77b4102ef0 0x7f77b419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.324+0000 7f77b37fe640 1 -- 172.21.15.120:0/3890138102 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b419d4b0 con 0x7f77b4100e20 2024-09-19T09:25:41.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.324+0000 7f77b37fe640 1 --2- 172.21.15.120:0/3890138102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77b4100e20 0x7f77b4198500 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f77a4006010 tx=0x7f77a4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:41.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.325+0000 7f77b0ff9640 1 -- 172.21.15.120:0/3890138102 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a403f070 con 0x7f77b4100e20 2024-09-19T09:25:41.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.325+0000 7f77ba161640 1 -- 172.21.15.120:0/3890138102 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77b419d740 con 0x7f77b4100e20 2024-09-19T09:25:41.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.325+0000 7f77b0ff9640 1 -- 172.21.15.120:0/3890138102 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77a4031c30 con 0x7f77b4100e20 2024-09-19T09:25:41.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.325+0000 7f77b0ff9640 1 -- 172.21.15.120:0/3890138102 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4035030 con 0x7f77b4100e20 2024-09-19T09:25:41.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.325+0000 7f77ba161640 1 -- 172.21.15.120:0/3890138102 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77b41a5080 con 0x7f77b4100e20 2024-09-19T09:25:41.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.327+0000 7f77b0ff9640 1 -- 172.21.15.120:0/3890138102 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f77a4042070 con 0x7f77b4100e20 2024-09-19T09:25:41.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.328+0000 7f77bc53a740 1 Processor -- start 2024-09-19T09:25:41.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.328+0000 7f77bc53a740 1 -- start start 2024-09-19T09:25:41.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.328+0000 7f77b0ff9640 1 --2- 172.21.15.120:0/3890138102 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f779c0a3240 0x7f779c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:41.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.328+0000 7f77b3fff640 1 --2- 172.21.15.120:0/3890138102 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f779c0a3240 0x7f779c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:41.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.328+0000 7f77b0ff9640 1 -- 172.21.15.120:0/3890138102 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f77a40eff60 con 0x7f77b4100e20 2024-09-19T09:25:41.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.328+0000 7f77bc53a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56325f923b40 0x56325f922090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:41.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.328+0000 7f77bc53a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56325f9225d0 0x56325f922a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:41.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.328+0000 7f77bc53a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56325f92d370 0x56325f92f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:41.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.329+0000 7f77bc53a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56325f8882f0 con 0x56325f9225d0 2024-09-19T09:25:41.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.329+0000 7f77bc53a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56325f82fcf0 con 0x56325f92d370 2024-09-19T09:25:41.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.329+0000 7f77bc53a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56325f8842b0 con 0x56325f923b40 2024-09-19T09:25:41.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.329+0000 7f77b2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56325f9225d0 0x56325f922a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:41.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.329+0000 7f77b2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56325f9225d0 0x56325f922a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57328/0 (socket says 172.21.15.120:57328) 2024-09-19T09:25:41.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.329+0000 7f77b2ffd640 1 -- 172.21.15.120:0/9081468 learned_addr learned my addr 172.21.15.120:0/9081468 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:41.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.329+0000 7f77b37fe640 1 --2- 172.21.15.120:0/9081468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56325f923b40 0x56325f922090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:41.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.329+0000 7f77b3fff640 1 --2- 172.21.15.120:0/9081468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56325f92d370 0x56325f92f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:41.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.329+0000 7f77b3fff640 1 --2- 172.21.15.120:0/3890138102 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f779c0a3240 0x7f779c0a5700 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f77ac005fd0 tx=0x7f77ac011810 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:41.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.329+0000 7f77b2ffd640 1 -- 172.21.15.120:0/9081468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56325f923b40 msgr2=0x56325f922090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:41.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.329+0000 7f77b2ffd640 1 --2- 172.21.15.120:0/9081468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56325f923b40 0x56325f922090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.329+0000 7f77b2ffd640 1 -- 172.21.15.120:0/9081468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56325f92d370 msgr2=0x56325f92f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:41.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.329+0000 7f77b2ffd640 1 --2- 172.21.15.120:0/9081468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56325f92d370 0x56325f92f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.329+0000 7f77b2ffd640 1 -- 172.21.15.120:0/9081468 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56325f8143a0 con 0x56325f9225d0 2024-09-19T09:25:41.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.329+0000 7f77b3fff640 1 --2- 172.21.15.120:0/9081468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56325f92d370 0x56325f92f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:25:41.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.329+0000 7f77b2ffd640 1 --2- 172.21.15.120:0/9081468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56325f9225d0 0x56325f922a10 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f77a800bcb0 tx=0x7f77a80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81447b7868fc68d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:41.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.329+0000 7f7799ffb640 1 -- 172.21.15.120:0/9081468 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a8004030 con 0x56325f9225d0 2024-09-19T09:25:41.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.330+0000 7f7799ffb640 1 -- 172.21.15.120:0/9081468 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77a80041d0 con 0x56325f9225d0 2024-09-19T09:25:41.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.330+0000 7f7799ffb640 1 -- 172.21.15.120:0/9081468 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a80117b0 con 0x56325f9225d0 2024-09-19T09:25:41.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.330+0000 7f77bc53a740 1 -- 172.21.15.120:0/9081468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56325f9225d0 msgr2=0x56325f922a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:41.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.330+0000 7f77bc53a740 1 --2- 172.21.15.120:0/9081468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56325f9225d0 0x56325f922a10 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f77a800bcb0 tx=0x7f77a80107a0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.330+0000 7f77bc53a740 1 -- 172.21.15.120:0/9081468 shutdown_connections 2024-09-19T09:25:41.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.330+0000 7f77bc53a740 1 --2- 172.21.15.120:0/9081468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56325f92d370 0x56325f92f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.330+0000 7f77bc53a740 1 --2- 172.21.15.120:0/9081468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56325f9225d0 0x56325f922a10 unknown :-1 s=CLOSED pgs=2344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.330+0000 7f77bc53a740 1 --2- 172.21.15.120:0/9081468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56325f923b40 0x56325f922090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.330+0000 7f77bc53a740 1 -- 172.21.15.120:0/9081468 >> 172.21.15.120:0/9081468 conn(0x56325f8eb310 msgr2=0x56325f91ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:41.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.330+0000 7f77bc53a740 1 -- 172.21.15.120:0/9081468 shutdown_connections 2024-09-19T09:25:41.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.330+0000 7f77bc53a740 1 -- 172.21.15.120:0/9081468 wait complete. 2024-09-19T09:25:41.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.331+0000 7f77bc53a740 1 -- start start 2024-09-19T09:25:41.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.331+0000 7f77bc53a740 1 Processor -- start 2024-09-19T09:25:41.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.331+0000 7f77bc53a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56325f9225d0 0x56325f92cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:41.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.331+0000 7f77bc53a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56325f923b40 0x56325f92a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:41.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.331+0000 7f77bc53a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56325f92d370 0x56325f92a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:41.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.331+0000 7f77bc53a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56325f930070 con 0x56325f92d370 2024-09-19T09:25:41.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.331+0000 7f77bc53a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56325f843060 con 0x56325f923b40 2024-09-19T09:25:41.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.331+0000 7f77bc53a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56325f9301f0 con 0x56325f9225d0 2024-09-19T09:25:41.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.331+0000 7f77bc53a740 10 client.0 _ll_register_callbacks cb 0x56325f5968c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:25:41.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.331+0000 7f77b3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56325f92d370 0x56325f92a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:41.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.331+0000 7f77b37fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56325f9225d0 0x56325f92cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:41.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.332+0000 7f77b2ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56325f923b40 0x56325f92a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:41.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.332+0000 7f77b3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56325f92d370 0x56325f92a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57338/0 (socket says 172.21.15.120:57338) 2024-09-19T09:25:41.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.332+0000 7f77b37fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56325f9225d0 0x56325f92cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:58202/0 (socket says 172.21.15.120:58202) 2024-09-19T09:25:41.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.332+0000 7f77b3fff640 1 -- 172.21.15.120:0/1570400456 learned_addr learned my addr 172.21.15.120:0/1570400456 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:41.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.332+0000 7f77b3fff640 1 -- 172.21.15.120:0/1570400456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56325f9225d0 msgr2=0x56325f92cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:41.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.332+0000 7f77b3fff640 1 --2- 172.21.15.120:0/1570400456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56325f9225d0 0x56325f92cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.332+0000 7f77b3fff640 1 -- 172.21.15.120:0/1570400456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56325f923b40 msgr2=0x56325f92a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:41.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.332+0000 7f77b3fff640 1 --2- 172.21.15.120:0/1570400456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56325f923b40 0x56325f92a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.332+0000 7f77b3fff640 1 -- 172.21.15.120:0/1570400456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56325f5f93c0 con 0x56325f92d370 2024-09-19T09:25:41.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.332+0000 7f77b3fff640 1 --2- 172.21.15.120:0/1570400456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56325f92d370 0x56325f92a640 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00faf0 tx=0x7f77ac004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec82d51d50e70ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:41.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.332+0000 7f7792ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:25:41.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.332+0000 7f7792ffd640 1 -- 172.21.15.120:0/1570400456 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac02b060 con 0x56325f92d370 2024-09-19T09:25:41.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.332+0000 7f7792ffd640 1 -- 172.21.15.120:0/1570400456 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f77ac01eaa0 con 0x56325f92d370 2024-09-19T09:25:41.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.332+0000 7f77bc53a740 1 -- 172.21.15.120:0/1570400456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77a80041d0 con 0x56325f92d370 2024-09-19T09:25:41.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.332+0000 7f7792ffd640 1 -- 172.21.15.120:0/1570400456 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f77ac017040 con 0x56325f92d370 2024-09-19T09:25:41.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.332+0000 7f77bc53a740 10 client.22872 fetch_fsmap learned FSMap version 39 2024-09-19T09:25:41.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.332+0000 7f77bc53a740 1 -- 172.21.15.120:0/1570400456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56325f93bba0 con 0x56325f92d370 2024-09-19T09:25:41.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.333+0000 7f7792ffd640 1 -- 172.21.15.120:0/1570400456 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77a80041d0 con 0x56325f92d370 2024-09-19T09:25:41.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.333+0000 7f7792ffd640 1 -- 172.21.15.120:0/1570400456 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f77ac030070 con 0x56325f92d370 2024-09-19T09:25:41.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.333+0000 7f77bc53a740 10 client.22872 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:25:41.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.333+0000 7f77bc53a740 10 client.22872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:25:41.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.333+0000 7f77bc53a740 20 client.22872 populate_metadata read hostname 'smithi120' 2024-09-19T09:25:41.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.333+0000 7f77bc53a740 1 --2- 172.21.15.120:0/1570400456 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56325f93cfb0 0x56325f93f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:41.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.333+0000 7f77bc53a740 4 client.22872 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:25:41.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.333+0000 7f77bc53a740 1 -- 172.21.15.120:0/1570400456 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56325f895860 con 0x56325f93cfb0 2024-09-19T09:25:41.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.333+0000 7f77b37fe640 1 --2- 172.21.15.120:0/1570400456 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56325f93cfb0 0x56325f93f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:41.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.334+0000 7f7792ffd640 10 client.22872 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:41.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.334+0000 7f77b37fe640 1 --2- 172.21.15.120:0/1570400456 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56325f93cfb0 0x56325f93f3e0 crc :-1 s=READY pgs=559 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62395c4fbed61e5 server_cookie=96551d3e8293d550 in_seq=0 out_seq=0 2024-09-19T09:25:41.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.336+0000 7f7792ffd640 1 -- 172.21.15.120:0/1570400456 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56325f895860 con 0x56325f93cfb0 2024-09-19T09:25:41.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.336+0000 7f7792ffd640 10 client.22872 handle_command_reply: tid=0 2024-09-19T09:25:41.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.336+0000 7f77bc53a740 1 client.22872 shutdown 2024-09-19T09:25:41.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.336+0000 7f77bc53a740 1 -- 172.21.15.120:0/1570400456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56325f92d370 msgr2=0x56325f92a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:41.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.336+0000 7f77bc53a740 1 --2- 172.21.15.120:0/1570400456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56325f92d370 0x56325f92a640 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00faf0 tx=0x7f77ac004ab0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.336+0000 7f77bc53a740 1 -- 172.21.15.120:0/1570400456 shutdown_connections 2024-09-19T09:25:41.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.336+0000 7f77bc53a740 1 --2- 172.21.15.120:0/1570400456 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56325f93cfb0 0x56325f93f3e0 crc :-1 s=READY pgs=559 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.336+0000 7f7792ffd640 0 client.22872 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:41.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.336+0000 7f77bc53a740 1 --2- 172.21.15.120:0/1570400456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56325f92d370 0x56325f92a640 unknown :-1 s=CLOSED pgs=2345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.336+0000 7f77bc53a740 1 --2- 172.21.15.120:0/1570400456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56325f923b40 0x56325f92a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.336+0000 7f77bc53a740 1 --2- 172.21.15.120:0/1570400456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56325f9225d0 0x56325f92cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.336+0000 7f77bc53a740 1 -- 172.21.15.120:0/1570400456 >> 172.21.15.120:0/1570400456 conn(0x56325f8eb310 msgr2=0x56325f924960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:41.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.336+0000 7f77bc53a740 1 -- 172.21.15.120:0/1570400456 shutdown_connections 2024-09-19T09:25:41.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.336+0000 7f77bc53a740 1 -- 172.21.15.120:0/1570400456 wait complete. 2024-09-19T09:25:41.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.336+0000 7f77bc53a740 20 client.22872 trim_cache size 0 max 16384 2024-09-19T09:25:41.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.354+0000 7f77bc53a740 1 Processor -- start 2024-09-19T09:25:41.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.354+0000 7f77bc53a740 1 -- start start 2024-09-19T09:25:41.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.354+0000 7f77bc53a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56325f9225d0 0x56325f9298f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:41.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.354+0000 7f77bc53a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56325f923b40 0x56325f92bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:41.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.354+0000 7f77bc53a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56325f92d370 0x56325f925120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:41.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.354+0000 7f77bc53a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f77b41ab330 con 0x56325f923b40 2024-09-19T09:25:41.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.354+0000 7f77bc53a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f77b41ace70 con 0x56325f92d370 2024-09-19T09:25:41.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.354+0000 7f77bc53a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f77b41aba00 con 0x56325f9225d0 2024-09-19T09:25:41.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.354+0000 7f77b2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56325f923b40 0x56325f92bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:41.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.354+0000 7f77b2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56325f923b40 0x56325f92bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57340/0 (socket says 172.21.15.120:57340) 2024-09-19T09:25:41.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.354+0000 7f77b2ffd640 1 -- 172.21.15.120:0/4289545273 learned_addr learned my addr 172.21.15.120:0/4289545273 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:41.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.354+0000 7f77b3fff640 1 --2- 172.21.15.120:0/4289545273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56325f92d370 0x56325f925120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:41.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.354+0000 7f77b37fe640 1 --2- 172.21.15.120:0/4289545273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56325f9225d0 0x56325f9298f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:41.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.355+0000 7f77b2ffd640 1 -- 172.21.15.120:0/4289545273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56325f9225d0 msgr2=0x56325f9298f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:41.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.355+0000 7f77b2ffd640 1 --2- 172.21.15.120:0/4289545273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56325f9225d0 0x56325f9298f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.355+0000 7f77b2ffd640 1 -- 172.21.15.120:0/4289545273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56325f92d370 msgr2=0x56325f925120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:41.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.355+0000 7f77b2ffd640 1 --2- 172.21.15.120:0/4289545273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56325f92d370 0x56325f925120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.355+0000 7f77b2ffd640 1 -- 172.21.15.120:0/4289545273 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56325f929000 con 0x56325f923b40 2024-09-19T09:25:41.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.355+0000 7f77b2ffd640 1 --2- 172.21.15.120:0/4289545273 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56325f923b40 0x56325f92bfa0 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f77a8010ca0 tx=0x7f77a80058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbaaae72d69eb80e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:41.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.355+0000 7f77997fa640 1 -- 172.21.15.120:0/4289545273 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a802b070 con 0x56325f923b40 2024-09-19T09:25:41.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.355+0000 7f77997fa640 1 -- 172.21.15.120:0/4289545273 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77a80035f0 con 0x56325f923b40 2024-09-19T09:25:41.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.355+0000 7f77997fa640 1 -- 172.21.15.120:0/4289545273 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a80116a0 con 0x56325f923b40 2024-09-19T09:25:41.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.356+0000 7f77bc53a740 1 -- 172.21.15.120:0/4289545273 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56325f923b40 msgr2=0x56325f92bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:41.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.356+0000 7f77bc53a740 1 --2- 172.21.15.120:0/4289545273 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56325f923b40 0x56325f92bfa0 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f77a8010ca0 tx=0x7f77a80058d0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.356+0000 7f77bc53a740 1 -- 172.21.15.120:0/4289545273 shutdown_connections 2024-09-19T09:25:41.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.356+0000 7f77bc53a740 1 --2- 172.21.15.120:0/4289545273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56325f92d370 0x56325f925120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.356+0000 7f77bc53a740 1 --2- 172.21.15.120:0/4289545273 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56325f923b40 0x56325f92bfa0 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.356+0000 7f77bc53a740 1 --2- 172.21.15.120:0/4289545273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56325f9225d0 0x56325f9298f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.356+0000 7f77bc53a740 1 -- 172.21.15.120:0/4289545273 >> 172.21.15.120:0/4289545273 conn(0x56325f8eb310 msgr2=0x56325f92b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:41.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.356+0000 7f77bc53a740 1 -- 172.21.15.120:0/4289545273 shutdown_connections 2024-09-19T09:25:41.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.356+0000 7f77bc53a740 1 -- 172.21.15.120:0/4289545273 wait complete. 2024-09-19T09:25:41.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.356+0000 7f77bc53a740 1 -- start start 2024-09-19T09:25:41.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.357+0000 7f77bc53a740 1 Processor -- start 2024-09-19T09:25:41.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.357+0000 7f77bc53a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56325f9225d0 0x56325f935bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:41.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.357+0000 7f77bc53a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56325f923b40 0x56325f926e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:41.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.357+0000 7f77bc53a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56325f92d370 0x56325f927350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:41.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.357+0000 7f77bc53a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f77b41ac7a0 con 0x56325f923b40 2024-09-19T09:25:41.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.357+0000 7f77bc53a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f77b41ac0d0 con 0x56325f9225d0 2024-09-19T09:25:41.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.357+0000 7f77bc53a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f77b41a92f0 con 0x56325f92d370 2024-09-19T09:25:41.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.357+0000 7f77bc53a740 10 client.0 _ll_register_callbacks cb 0x7f77ac011f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:25:41.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.357+0000 7f77b2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56325f923b40 0x56325f926e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:41.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.357+0000 7f77b37fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56325f9225d0 0x56325f935bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:41.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.357+0000 7f77b2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56325f923b40 0x56325f926e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57356/0 (socket says 172.21.15.120:57356) 2024-09-19T09:25:41.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.357+0000 7f77b37fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56325f9225d0 0x56325f935bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:49286/0 (socket says 172.21.15.120:49286) 2024-09-19T09:25:41.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.357+0000 7f77b2ffd640 1 -- 172.21.15.120:0/1591858622 learned_addr learned my addr 172.21.15.120:0/1591858622 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:41.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.357+0000 7f77b3fff640 1 --2- 172.21.15.120:0/1591858622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56325f92d370 0x56325f927350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:41.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.357+0000 7f77b2ffd640 1 -- 172.21.15.120:0/1591858622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56325f92d370 msgr2=0x56325f927350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:41.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.357+0000 7f77b2ffd640 1 --2- 172.21.15.120:0/1591858622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56325f92d370 0x56325f927350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.357+0000 7f77b2ffd640 1 -- 172.21.15.120:0/1591858622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56325f9225d0 msgr2=0x56325f935bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:41.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.357+0000 7f77b2ffd640 1 --2- 172.21.15.120:0/1591858622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56325f9225d0 0x56325f935bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.358+0000 7f77b2ffd640 1 -- 172.21.15.120:0/1591858622 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56325f927b80 con 0x56325f923b40 2024-09-19T09:25:41.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.358+0000 7f77b37fe640 1 --2- 172.21.15.120:0/1591858622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56325f9225d0 0x56325f935bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:25:41.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.358+0000 7f77b2ffd640 1 --2- 172.21.15.120:0/1591858622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56325f923b40 0x56325f926e10 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7f77a8011010 tx=0x7f77a80039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd3975f9ba040f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:41.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.358+0000 7f7792ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:25:41.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.358+0000 7f7792ffd640 1 -- 172.21.15.120:0/1591858622 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a8034060 con 0x56325f923b40 2024-09-19T09:25:41.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.358+0000 7f7792ffd640 1 -- 172.21.15.120:0/1591858622 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f77a80050c0 con 0x56325f923b40 2024-09-19T09:25:41.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.358+0000 7f77bc53a740 1 -- 172.21.15.120:0/1591858622 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77a80035f0 con 0x56325f923b40 2024-09-19T09:25:41.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.358+0000 7f7792ffd640 1 -- 172.21.15.120:0/1591858622 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f77a802b040 con 0x56325f923b40 2024-09-19T09:25:41.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.358+0000 7f77bc53a740 10 client.22878 fetch_fsmap learned FSMap version 39 2024-09-19T09:25:41.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.358+0000 7f77bc53a740 1 -- 172.21.15.120:0/1591858622 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56325f9a7ce0 con 0x56325f923b40 2024-09-19T09:25:41.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.358+0000 7f7792ffd640 1 -- 172.21.15.120:0/1591858622 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77a8011440 con 0x56325f923b40 2024-09-19T09:25:41.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.359+0000 7f7792ffd640 1 -- 172.21.15.120:0/1591858622 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f77a8039070 con 0x56325f923b40 2024-09-19T09:25:41.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.359+0000 7f77bc53a740 10 client.22878 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:25:41.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.359+0000 7f77bc53a740 10 client.22878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:25:41.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.359+0000 7f77bc53a740 20 client.22878 populate_metadata read hostname 'smithi120' 2024-09-19T09:25:41.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.359+0000 7f77bc53a740 1 --2- 172.21.15.120:0/1591858622 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56325f93cfb0 0x56325f9aa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:41.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.359+0000 7f77bc53a740 4 client.22878 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:25:41.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.359+0000 7f77bc53a740 1 -- 172.21.15.120:0/1591858622 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f77a80035f0 con 0x56325f93cfb0 2024-09-19T09:25:41.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.359+0000 7f77b37fe640 1 --2- 172.21.15.120:0/1591858622 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56325f93cfb0 0x56325f9aa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:41.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.359+0000 7f77b37fe640 1 --2- 172.21.15.120:0/1591858622 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56325f93cfb0 0x56325f9aa070 crc :-1 s=READY pgs=560 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb73879cadce0c63 server_cookie=60dca528684ce129 in_seq=0 out_seq=0 2024-09-19T09:25:41.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.359+0000 7f7792ffd640 10 client.22878 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:41.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.363+0000 7f7792ffd640 1 -- 172.21.15.120:0/1591858622 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f77a80035f0 con 0x56325f93cfb0 2024-09-19T09:25:41.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.363+0000 7f7792ffd640 10 client.22878 handle_command_reply: tid=0 2024-09-19T09:25:41.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.363+0000 7f77bc53a740 1 client.22878 shutdown 2024-09-19T09:25:41.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.364+0000 7f77bc53a740 1 -- 172.21.15.120:0/1591858622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56325f923b40 msgr2=0x56325f926e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:41.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.364+0000 7f77bc53a740 1 --2- 172.21.15.120:0/1591858622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56325f923b40 0x56325f926e10 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7f77a8011010 tx=0x7f77a80039d0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.364+0000 7f77bc53a740 1 -- 172.21.15.120:0/1591858622 shutdown_connections 2024-09-19T09:25:41.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.364+0000 7f77bc53a740 1 --2- 172.21.15.120:0/1591858622 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56325f93cfb0 0x56325f9aa070 crc :-1 s=READY pgs=560 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.364+0000 7f7792ffd640 0 client.22878 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:41.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.364+0000 7f77bc53a740 1 --2- 172.21.15.120:0/1591858622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56325f92d370 0x56325f927350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.364+0000 7f77bc53a740 1 --2- 172.21.15.120:0/1591858622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56325f923b40 0x56325f926e10 unknown :-1 s=CLOSED pgs=2347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.364+0000 7f77bc53a740 1 --2- 172.21.15.120:0/1591858622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56325f9225d0 0x56325f935bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.364+0000 7f77bc53a740 1 -- 172.21.15.120:0/1591858622 >> 172.21.15.120:0/1591858622 conn(0x56325f8eb310 msgr2=0x56325f938300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:41.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.364+0000 7f77bc53a740 1 -- 172.21.15.120:0/1591858622 shutdown_connections 2024-09-19T09:25:41.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.364+0000 7f77bc53a740 1 -- 172.21.15.120:0/1591858622 wait complete. 2024-09-19T09:25:41.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.364+0000 7f77bc53a740 20 client.22878 trim_cache size 0 max 16384 2024-09-19T09:25:41.366 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:25:41.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.364+0000 7f77ba161640 1 -- 172.21.15.120:0/3890138102 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f779c0a3240 msgr2=0x7f779c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:41.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.364+0000 7f77ba161640 1 --2- 172.21.15.120:0/3890138102 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f779c0a3240 0x7f779c0a5700 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f77ac005fd0 tx=0x7f77ac011810 comp rx=0 tx=0).stop 2024-09-19T09:25:41.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.364+0000 7f77ba161640 1 -- 172.21.15.120:0/3890138102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77b4100e20 msgr2=0x7f77b4198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:41.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.364+0000 7f77ba161640 1 --2- 172.21.15.120:0/3890138102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77b4100e20 0x7f77b4198500 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f77a4006010 tx=0x7f77a4003c10 comp rx=0 tx=0).stop 2024-09-19T09:25:41.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.364+0000 7f77ba161640 1 -- 172.21.15.120:0/3890138102 shutdown_connections 2024-09-19T09:25:41.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.364+0000 7f77ba161640 1 --2- 172.21.15.120:0/3890138102 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f779c0a3240 0x7f779c0a5700 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.364+0000 7f77ba161640 1 --2- 172.21.15.120:0/3890138102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77b4102ef0 0x7f77b419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.364+0000 7f77ba161640 1 --2- 172.21.15.120:0/3890138102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77b4102570 0x7f77b4198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.364+0000 7f77ba161640 1 --2- 172.21.15.120:0/3890138102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77b4100e20 0x7f77b4198500 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:41.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.364+0000 7f77ba161640 1 -- 172.21.15.120:0/3890138102 >> 172.21.15.120:0/3890138102 conn(0x7f77b40fc7d0 msgr2=0x7f77b406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:41.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.365+0000 7f77ba161640 1 -- 172.21.15.120:0/3890138102 shutdown_connections 2024-09-19T09:25:41.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:41.365+0000 7f77ba161640 1 -- 172.21.15.120:0/3890138102 wait complete. 2024-09-19T09:25:41.384 INFO:tasks.workunit.client.0.smithi120.stdout: 10 226272 20.17 MB/sec execute 456 sec latency 68.285 ms 2024-09-19T09:25:42.384 INFO:tasks.workunit.client.0.smithi120.stdout: 10 226845 20.19 MB/sec execute 457 sec latency 239.936 ms 2024-09-19T09:25:42.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:42 smithi196 ceph-mon[35150]: pgmap v1030: 97 pgs: 97 active+clean; 491 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.4 MiB/s wr, 767 op/s 2024-09-19T09:25:42.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:42 smithi153 ceph-mon[35071]: pgmap v1030: 97 pgs: 97 active+clean; 491 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.4 MiB/s wr, 767 op/s 2024-09-19T09:25:42.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:42 smithi120 ceph-mon[31150]: pgmap v1030: 97 pgs: 97 active+clean; 491 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.4 MiB/s wr, 767 op/s 2024-09-19T09:25:43.306 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:25:43.307 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:25:43.384 INFO:tasks.workunit.client.0.smithi120.stdout: 10 227398 20.23 MB/sec execute 458 sec latency 100.841 ms 2024-09-19T09:25:43.596 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:44.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.114+0000 7fa79b674640 1 -- 172.21.15.120:0/1836816841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa794102d30 msgr2=0x7fa794103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:44.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.114+0000 7fa79b674640 1 --2- 172.21.15.120:0/1836816841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa794102d30 0x7fa794103110 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fa790009930 tx=0x7fa79002a420 comp rx=0 tx=0).stop 2024-09-19T09:25:44.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.114+0000 7fa79b674640 1 -- 172.21.15.120:0/1836816841 shutdown_connections 2024-09-19T09:25:44.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.114+0000 7fa79b674640 1 --2- 172.21.15.120:0/1836816841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7941082a0 0x7fa79410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.114+0000 7fa79b674640 1 --2- 172.21.15.120:0/1836816841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7941036e0 0x7fa794107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.114+0000 7fa79b674640 1 --2- 172.21.15.120:0/1836816841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa794102d30 0x7fa794103110 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.114+0000 7fa79b674640 1 -- 172.21.15.120:0/1836816841 >> 172.21.15.120:0/1836816841 conn(0x7fa7940fc7d0 msgr2=0x7fa7940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:44.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.115+0000 7fa79b674640 1 -- 172.21.15.120:0/1836816841 shutdown_connections 2024-09-19T09:25:44.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.115+0000 7fa79b674640 1 -- 172.21.15.120:0/1836816841 wait complete. 2024-09-19T09:25:44.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.116+0000 7fa79b674640 1 Processor -- start 2024-09-19T09:25:44.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.116+0000 7fa79b674640 1 -- start start 2024-09-19T09:25:44.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.116+0000 7fa79b674640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa794102d30 0x7fa794198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:44.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.116+0000 7fa79b674640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7941036e0 0x7fa794198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:44.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.116+0000 7fa79b674640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7941082a0 0x7fa79419cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:44.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.116+0000 7fa79b674640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa794112100 con 0x7fa794102d30 2024-09-19T09:25:44.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.116+0000 7fa79b674640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa794111f80 con 0x7fa7941082a0 2024-09-19T09:25:44.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.116+0000 7fa79b674640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa794112280 con 0x7fa7941036e0 2024-09-19T09:25:44.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.116+0000 7fa798c0f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7941036e0 0x7fa794198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:44.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.116+0000 7fa799410640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa794102d30 0x7fa794198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:44.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.116+0000 7fa798c0f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7941036e0 0x7fa794198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:58232/0 (socket says 172.21.15.120:58232) 2024-09-19T09:25:44.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.116+0000 7fa798c0f640 1 -- 172.21.15.120:0/813023594 learned_addr learned my addr 172.21.15.120:0/813023594 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:44.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.116+0000 7fa799c11640 1 --2- 172.21.15.120:0/813023594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7941082a0 0x7fa79419cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:44.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.117+0000 7fa799410640 1 -- 172.21.15.120:0/813023594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7941036e0 msgr2=0x7fa794198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:44.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.117+0000 7fa799410640 1 --2- 172.21.15.120:0/813023594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7941036e0 0x7fa794198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.117+0000 7fa799410640 1 -- 172.21.15.120:0/813023594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7941082a0 msgr2=0x7fa79419cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:44.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.117+0000 7fa799410640 1 --2- 172.21.15.120:0/813023594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7941082a0 0x7fa79419cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.117+0000 7fa799410640 1 -- 172.21.15.120:0/813023594 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa79419d5f0 con 0x7fa794102d30 2024-09-19T09:25:44.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.117+0000 7fa799410640 1 --2- 172.21.15.120:0/813023594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa794102d30 0x7fa794198640 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fa790005940 tx=0x7fa7900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:44.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.117+0000 7fa78a7fc640 1 -- 172.21.15.120:0/813023594 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79003f070 con 0x7fa794102d30 2024-09-19T09:25:44.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.117+0000 7fa79b674640 1 -- 172.21.15.120:0/813023594 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa79419d880 con 0x7fa794102d30 2024-09-19T09:25:44.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.117+0000 7fa78a7fc640 1 -- 172.21.15.120:0/813023594 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa79003b460 con 0x7fa794102d30 2024-09-19T09:25:44.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.117+0000 7fa78a7fc640 1 -- 172.21.15.120:0/813023594 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa790035070 con 0x7fa794102d30 2024-09-19T09:25:44.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.117+0000 7fa79b674640 1 -- 172.21.15.120:0/813023594 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7941a51c0 con 0x7fa794102d30 2024-09-19T09:25:44.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.119+0000 7fa78a7fc640 1 -- 172.21.15.120:0/813023594 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa79003a620 con 0x7fa794102d30 2024-09-19T09:25:44.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.120+0000 7fa79da4d740 1 Processor -- start 2024-09-19T09:25:44.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.120+0000 7fa79da4d740 1 -- start start 2024-09-19T09:25:44.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.120+0000 7fa79da4d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c90920fb60 0x55c90920ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:44.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.120+0000 7fa79da4d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c909210480 0x55c90921a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:44.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.120+0000 7fa79da4d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c90921ad40 0x55c90921d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:44.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.120+0000 7fa79da4d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c909175280 con 0x55c909210480 2024-09-19T09:25:44.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.120+0000 7fa79da4d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c90911cb10 con 0x55c90921ad40 2024-09-19T09:25:44.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.120+0000 7fa79da4d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c909171240 con 0x55c90920fb60 2024-09-19T09:25:44.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.120+0000 7fa799410640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c909210480 0x55c90921a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:44.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.121+0000 7fa798c0f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c90921ad40 0x55c90921d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:44.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.121+0000 7fa798c0f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c90921ad40 0x55c90921d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:49310/0 (socket says 172.21.15.120:49310) 2024-09-19T09:25:44.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.121+0000 7fa799410640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c909210480 0x55c90921a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57386/0 (socket says 172.21.15.120:57386) 2024-09-19T09:25:44.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.121+0000 7fa798c0f640 1 -- 172.21.15.120:0/1041327920 learned_addr learned my addr 172.21.15.120:0/1041327920 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:44.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.121+0000 7fa799c11640 1 --2- 172.21.15.120:0/1041327920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c90920fb60 0x55c90920ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:44.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.121+0000 7fa78a7fc640 1 --2- 172.21.15.120:0/813023594 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa7800a7650 0x7fa7800a9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:44.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.121+0000 7fa798c0f640 1 -- 172.21.15.120:0/1041327920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c90920fb60 msgr2=0x55c90920ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:44.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.121+0000 7fa798c0f640 1 --2- 172.21.15.120:0/1041327920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c90920fb60 0x55c90920ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.121+0000 7fa798c0f640 1 -- 172.21.15.120:0/1041327920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c909210480 msgr2=0x55c90921a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:44.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.121+0000 7fa798c0f640 1 --2- 172.21.15.120:0/1041327920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c909210480 0x55c90921a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.121+0000 7fa78a7fc640 1 -- 172.21.15.120:0/813023594 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa7900efe30 con 0x7fa794102d30 2024-09-19T09:25:44.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.121+0000 7fa798c0f640 1 -- 172.21.15.120:0/1041327920 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9091291f0 con 0x55c90921ad40 2024-09-19T09:25:44.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.121+0000 7fa799c11640 1 --2- 172.21.15.120:0/1041327920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c90920fb60 0x55c90920ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:25:44.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.121+0000 7fa799c11640 1 --2- 172.21.15.120:0/813023594 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa7800a7650 0x7fa7800a9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:44.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.121+0000 7fa798c0f640 1 --2- 172.21.15.120:0/1041327920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c90921ad40 0x55c90921d130 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa784004630 tx=0x7fa78400bc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19272ee7d254a284 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:44.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.122+0000 7fa77f7fe640 1 -- 172.21.15.120:0/1041327920 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78400be60 con 0x55c90921ad40 2024-09-19T09:25:44.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.122+0000 7fa799c11640 1 --2- 172.21.15.120:0/813023594 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa7800a7650 0x7fa7800a9b10 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00a640 tx=0x7fa78c004b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:44.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.122+0000 7fa77f7fe640 1 -- 172.21.15.120:0/1041327920 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa784015070 con 0x55c90921ad40 2024-09-19T09:25:44.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.122+0000 7fa77f7fe640 1 -- 172.21.15.120:0/1041327920 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7840116a0 con 0x55c90921ad40 2024-09-19T09:25:44.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.123+0000 7fa79da4d740 1 -- 172.21.15.120:0/1041327920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c90921ad40 msgr2=0x55c90921d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:44.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.123+0000 7fa79da4d740 1 --2- 172.21.15.120:0/1041327920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c90921ad40 0x55c90921d130 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa784004630 tx=0x7fa78400bc10 comp rx=0 tx=0).stop 2024-09-19T09:25:44.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.123+0000 7fa79da4d740 1 -- 172.21.15.120:0/1041327920 shutdown_connections 2024-09-19T09:25:44.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.123+0000 7fa79da4d740 1 --2- 172.21.15.120:0/1041327920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c90921ad40 0x55c90921d130 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.123+0000 7fa79da4d740 1 --2- 172.21.15.120:0/1041327920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c909210480 0x55c90921a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.123+0000 7fa79da4d740 1 --2- 172.21.15.120:0/1041327920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c90920fb60 0x55c90920ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.123+0000 7fa79da4d740 1 -- 172.21.15.120:0/1041327920 >> 172.21.15.120:0/1041327920 conn(0x55c90920c0f0 msgr2=0x55c90920c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:44.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.123+0000 7fa79da4d740 1 -- 172.21.15.120:0/1041327920 shutdown_connections 2024-09-19T09:25:44.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.123+0000 7fa79da4d740 1 -- 172.21.15.120:0/1041327920 wait complete. 2024-09-19T09:25:44.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.123+0000 7fa79da4d740 1 -- start start 2024-09-19T09:25:44.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.123+0000 7fa79da4d740 1 Processor -- start 2024-09-19T09:25:44.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.124+0000 7fa79da4d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c90920fb60 0x55c909217c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:44.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.124+0000 7fa79da4d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c909210480 0x55c909218150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:44.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.124+0000 7fa79da4d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c90921ad40 0x55c909218690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:44.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.124+0000 7fa79da4d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c90921d910 con 0x55c90920fb60 2024-09-19T09:25:44.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.124+0000 7fa79da4d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c90912ff10 con 0x55c909210480 2024-09-19T09:25:44.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.124+0000 7fa79da4d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c90921da90 con 0x55c90921ad40 2024-09-19T09:25:44.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.124+0000 7fa799410640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c90920fb60 0x55c909217c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:44.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.124+0000 7fa798c0f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c909210480 0x55c909218150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:44.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.124+0000 7fa799c11640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c90921ad40 0x55c909218690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:44.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.124+0000 7fa79da4d740 10 client.0 _ll_register_callbacks cb 0x55c9090f8a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:25:44.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.124+0000 7fa799c11640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c90921ad40 0x55c909218690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:58256/0 (socket says 172.21.15.120:58256) 2024-09-19T09:25:44.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.124+0000 7fa799c11640 1 -- 172.21.15.120:0/2971376971 learned_addr learned my addr 172.21.15.120:0/2971376971 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:44.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.124+0000 7fa798c0f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c909210480 0x55c909218150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:49322/0 (socket says 172.21.15.120:49322) 2024-09-19T09:25:44.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.124+0000 7fa799c11640 1 -- 172.21.15.120:0/2971376971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c909210480 msgr2=0x55c909218150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:44.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.124+0000 7fa799c11640 1 --2- 172.21.15.120:0/2971376971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c909210480 0x55c909218150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.124+0000 7fa799c11640 1 -- 172.21.15.120:0/2971376971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c90920fb60 msgr2=0x55c909217c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:44.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.124+0000 7fa799c11640 1 --2- 172.21.15.120:0/2971376971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c90920fb60 0x55c909217c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.124+0000 7fa799c11640 1 -- 172.21.15.120:0/2971376971 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c908ee66a0 con 0x55c90921ad40 2024-09-19T09:25:44.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.125+0000 7fa799c11640 1 --2- 172.21.15.120:0/2971376971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c90921ad40 0x55c909218690 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa78c007bb0 tx=0x7fa78c00d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdbf98d6e49b02f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:44.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.125+0000 7fa77cff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:25:44.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.125+0000 7fa77cff9640 1 -- 172.21.15.120:0/2971376971 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c026020 con 0x55c90921ad40 2024-09-19T09:25:44.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.125+0000 7fa79da4d740 1 -- 172.21.15.120:0/2971376971 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa784015070 con 0x55c90921ad40 2024-09-19T09:25:44.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.125+0000 7fa77cff9640 1 -- 172.21.15.120:0/2971376971 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa78c017b00 con 0x55c90921ad40 2024-09-19T09:25:44.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.125+0000 7fa77cff9640 1 -- 172.21.15.120:0/2971376971 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa78c017d00 con 0x55c90921ad40 2024-09-19T09:25:44.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.125+0000 7fa79da4d740 10 client.31429 fetch_fsmap learned FSMap version 39 2024-09-19T09:25:44.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.125+0000 7fa79da4d740 1 -- 172.21.15.120:0/2971376971 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c9092295c0 con 0x55c90921ad40 2024-09-19T09:25:44.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.125+0000 7fa77cff9640 1 -- 172.21.15.120:0/2971376971 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa784015070 con 0x55c90921ad40 2024-09-19T09:25:44.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.125+0000 7fa77cff9640 1 -- 172.21.15.120:0/2971376971 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fa78c0214a0 con 0x55c90921ad40 2024-09-19T09:25:44.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.125+0000 7fa79da4d740 10 client.31429 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:25:44.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.125+0000 7fa79da4d740 10 client.31429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:25:44.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.125+0000 7fa79da4d740 20 client.31429 populate_metadata read hostname 'smithi120' 2024-09-19T09:25:44.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.125+0000 7fa79da4d740 1 --2- 172.21.15.120:0/2971376971 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c90922adb0 0x55c90922d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:44.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.125+0000 7fa79da4d740 4 client.31429 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:25:44.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.125+0000 7fa79da4d740 1 -- 172.21.15.120:0/2971376971 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9091827f0 con 0x55c90922adb0 2024-09-19T09:25:44.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.126+0000 7fa799410640 1 --2- 172.21.15.120:0/2971376971 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c90922adb0 0x55c90922d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:44.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.126+0000 7fa77cff9640 10 client.31429 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:44.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.126+0000 7fa799410640 1 --2- 172.21.15.120:0/2971376971 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c90922adb0 0x55c90922d250 crc :-1 s=READY pgs=561 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ad427965195d22d server_cookie=44da9826c30cfbda in_seq=0 out_seq=0 2024-09-19T09:25:44.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.128+0000 7fa77cff9640 1 -- 172.21.15.120:0/2971376971 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c9091827f0 con 0x55c90922adb0 2024-09-19T09:25:44.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.128+0000 7fa77cff9640 10 client.31429 handle_command_reply: tid=0 2024-09-19T09:25:44.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.128+0000 7fa79da4d740 1 client.31429 shutdown 2024-09-19T09:25:44.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.129+0000 7fa79da4d740 1 -- 172.21.15.120:0/2971376971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c90921ad40 msgr2=0x55c909218690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:44.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.129+0000 7fa79da4d740 1 --2- 172.21.15.120:0/2971376971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c90921ad40 0x55c909218690 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa78c007bb0 tx=0x7fa78c00d040 comp rx=0 tx=0).stop 2024-09-19T09:25:44.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.129+0000 7fa79da4d740 1 -- 172.21.15.120:0/2971376971 shutdown_connections 2024-09-19T09:25:44.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.129+0000 7fa79da4d740 1 --2- 172.21.15.120:0/2971376971 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c90922adb0 0x55c90922d250 crc :-1 s=READY pgs=561 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.129+0000 7fa79da4d740 1 --2- 172.21.15.120:0/2971376971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c90921ad40 0x55c909218690 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.129+0000 7fa79da4d740 1 --2- 172.21.15.120:0/2971376971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c909210480 0x55c909218150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.129+0000 7fa77cff9640 0 client.31429 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:44.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.129+0000 7fa79da4d740 1 --2- 172.21.15.120:0/2971376971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c90920fb60 0x55c909217c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.129+0000 7fa79da4d740 1 -- 172.21.15.120:0/2971376971 >> 172.21.15.120:0/2971376971 conn(0x55c90920c0f0 msgr2=0x55c909211e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:44.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.129+0000 7fa79da4d740 1 -- 172.21.15.120:0/2971376971 shutdown_connections 2024-09-19T09:25:44.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.129+0000 7fa79da4d740 1 -- 172.21.15.120:0/2971376971 wait complete. 2024-09-19T09:25:44.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.129+0000 7fa79da4d740 20 client.31429 trim_cache size 0 max 16384 2024-09-19T09:25:44.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.147+0000 7fa79da4d740 1 Processor -- start 2024-09-19T09:25:44.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.147+0000 7fa79da4d740 1 -- start start 2024-09-19T09:25:44.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.147+0000 7fa79da4d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c90920fb60 0x55c909212610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:44.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.147+0000 7fa79da4d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c909210480 0x55c909219a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:44.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.147+0000 7fa79da4d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c90921ad40 0x55c909232b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:44.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.147+0000 7fa79da4d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa7941ab520 con 0x55c909210480 2024-09-19T09:25:44.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.147+0000 7fa79da4d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa7941ad060 con 0x55c90920fb60 2024-09-19T09:25:44.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.147+0000 7fa79da4d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa7941abbf0 con 0x55c90921ad40 2024-09-19T09:25:44.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.147+0000 7fa798c0f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c909210480 0x55c909219a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:44.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.147+0000 7fa798c0f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c909210480 0x55c909219a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57398/0 (socket says 172.21.15.120:57398) 2024-09-19T09:25:44.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.147+0000 7fa798c0f640 1 -- 172.21.15.120:0/3152705798 learned_addr learned my addr 172.21.15.120:0/3152705798 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:44.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.147+0000 7fa799c11640 1 --2- 172.21.15.120:0/3152705798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c90921ad40 0x55c909232b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:44.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.147+0000 7fa799410640 1 --2- 172.21.15.120:0/3152705798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c90920fb60 0x55c909212610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:44.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.148+0000 7fa798c0f640 1 -- 172.21.15.120:0/3152705798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c90921ad40 msgr2=0x55c909232b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:44.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.148+0000 7fa798c0f640 1 --2- 172.21.15.120:0/3152705798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c90921ad40 0x55c909232b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.148+0000 7fa798c0f640 1 -- 172.21.15.120:0/3152705798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c90920fb60 msgr2=0x55c909212610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:44.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.148+0000 7fa798c0f640 1 --2- 172.21.15.120:0/3152705798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c90920fb60 0x55c909212610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.148+0000 7fa798c0f640 1 -- 172.21.15.120:0/3152705798 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c90920d280 con 0x55c909210480 2024-09-19T09:25:44.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.148+0000 7fa799410640 1 --2- 172.21.15.120:0/3152705798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c90920fb60 0x55c909212610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:25:44.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.148+0000 7fa798c0f640 1 --2- 172.21.15.120:0/3152705798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c909210480 0x55c909219a70 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fa78400b710 tx=0x7fa78400b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853e56b56a660389 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:44.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.148+0000 7fa77effd640 1 -- 172.21.15.120:0/3152705798 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa784003dd0 con 0x55c909210480 2024-09-19T09:25:44.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.148+0000 7fa77effd640 1 -- 172.21.15.120:0/3152705798 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa784015040 con 0x55c909210480 2024-09-19T09:25:44.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.148+0000 7fa77effd640 1 -- 172.21.15.120:0/3152705798 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7840116a0 con 0x55c909210480 2024-09-19T09:25:44.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.149+0000 7fa79da4d740 1 -- 172.21.15.120:0/3152705798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c909210480 msgr2=0x55c909219a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:44.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.149+0000 7fa79da4d740 1 --2- 172.21.15.120:0/3152705798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c909210480 0x55c909219a70 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fa78400b710 tx=0x7fa78400b4d0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.149+0000 7fa79da4d740 1 -- 172.21.15.120:0/3152705798 shutdown_connections 2024-09-19T09:25:44.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.149+0000 7fa79da4d740 1 --2- 172.21.15.120:0/3152705798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c90921ad40 0x55c909232b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.149+0000 7fa79da4d740 1 --2- 172.21.15.120:0/3152705798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c909210480 0x55c909219a70 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.149+0000 7fa79da4d740 1 --2- 172.21.15.120:0/3152705798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c90920fb60 0x55c909212610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.149+0000 7fa79da4d740 1 -- 172.21.15.120:0/3152705798 >> 172.21.15.120:0/3152705798 conn(0x55c90920c0f0 msgr2=0x55c909219360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:44.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.149+0000 7fa79da4d740 1 -- 172.21.15.120:0/3152705798 shutdown_connections 2024-09-19T09:25:44.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.149+0000 7fa79da4d740 1 -- 172.21.15.120:0/3152705798 wait complete. 2024-09-19T09:25:44.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.149+0000 7fa79da4d740 1 -- start start 2024-09-19T09:25:44.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.150+0000 7fa79da4d740 1 Processor -- start 2024-09-19T09:25:44.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.150+0000 7fa79da4d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c90920fb60 0x55c909223490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:44.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.150+0000 7fa79da4d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c909210480 0x55c90921e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:44.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.150+0000 7fa79da4d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c90921ad40 0x55c90921ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:44.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.150+0000 7fa79da4d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa7941ac990 con 0x55c90921ad40 2024-09-19T09:25:44.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.150+0000 7fa79da4d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa7941ac2c0 con 0x55c90920fb60 2024-09-19T09:25:44.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.150+0000 7fa79da4d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa7941a94e0 con 0x55c909210480 2024-09-19T09:25:44.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.150+0000 7fa79da4d740 10 client.0 _ll_register_callbacks cb 0x7fa78c00a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:25:44.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.150+0000 7fa799c11640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c90921ad40 0x55c90921ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:44.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.150+0000 7fa799c11640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c90921ad40 0x55c90921ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57410/0 (socket says 172.21.15.120:57410) 2024-09-19T09:25:44.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.150+0000 7fa798c0f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c909210480 0x55c90921e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:44.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.150+0000 7fa799c11640 1 -- 172.21.15.120:0/465307047 learned_addr learned my addr 172.21.15.120:0/465307047 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:44.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.151+0000 7fa799c11640 1 -- 172.21.15.120:0/465307047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c909210480 msgr2=0x55c90921e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:44.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.151+0000 7fa799c11640 1 --2- 172.21.15.120:0/465307047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c909210480 0x55c90921e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.151+0000 7fa799c11640 1 -- 172.21.15.120:0/465307047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c90920fb60 msgr2=0x55c909223490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:44.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.151+0000 7fa799c11640 1 --2- 172.21.15.120:0/465307047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c90920fb60 0x55c909223490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.151+0000 7fa799c11640 1 -- 172.21.15.120:0/465307047 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c90921f530 con 0x55c90921ad40 2024-09-19T09:25:44.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.151+0000 7fa799c11640 1 --2- 172.21.15.120:0/465307047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c90921ad40 0x55c90921ed00 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00aed0 tx=0x7fa78c021d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40fb70cef3699071 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:44.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.151+0000 7fa77cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:25:44.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.151+0000 7fa77cff9640 1 -- 172.21.15.120:0/465307047 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c026040 con 0x55c90921ad40 2024-09-19T09:25:44.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.151+0000 7fa77cff9640 1 -- 172.21.15.120:0/465307047 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa78c017b00 con 0x55c90921ad40 2024-09-19T09:25:44.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.151+0000 7fa79da4d740 1 -- 172.21.15.120:0/465307047 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa784015040 con 0x55c90921ad40 2024-09-19T09:25:44.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.151+0000 7fa77cff9640 1 -- 172.21.15.120:0/465307047 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa78c009650 con 0x55c90921ad40 2024-09-19T09:25:44.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.151+0000 7fa79da4d740 10 client.22896 fetch_fsmap learned FSMap version 39 2024-09-19T09:25:44.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.151+0000 7fa79da4d740 1 -- 172.21.15.120:0/465307047 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c909295c30 con 0x55c90921ad40 2024-09-19T09:25:44.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.151+0000 7fa77cff9640 1 -- 172.21.15.120:0/465307047 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa784015040 con 0x55c90921ad40 2024-09-19T09:25:44.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.152+0000 7fa77cff9640 1 -- 172.21.15.120:0/465307047 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fa78c022560 con 0x55c90921ad40 2024-09-19T09:25:44.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.152+0000 7fa79da4d740 10 client.22896 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:25:44.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.152+0000 7fa79da4d740 10 client.22896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:25:44.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.152+0000 7fa79da4d740 20 client.22896 populate_metadata read hostname 'smithi120' 2024-09-19T09:25:44.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.152+0000 7fa79da4d740 1 --2- 172.21.15.120:0/465307047 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c90922adb0 0x55c909298450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:44.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.152+0000 7fa79da4d740 4 client.22896 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:25:44.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.152+0000 7fa79da4d740 1 -- 172.21.15.120:0/465307047 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c90917cd00 con 0x55c90922adb0 2024-09-19T09:25:44.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.152+0000 7fa799410640 1 --2- 172.21.15.120:0/465307047 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c90922adb0 0x55c909298450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:44.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.152+0000 7fa77cff9640 10 client.22896 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:44.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.152+0000 7fa799410640 1 --2- 172.21.15.120:0/465307047 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c90922adb0 0x55c909298450 crc :-1 s=READY pgs=562 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89a5a0370adecda2 server_cookie=a4806fee4669dab9 in_seq=0 out_seq=0 2024-09-19T09:25:44.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.155+0000 7fa77cff9640 1 -- 172.21.15.120:0/465307047 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c90917cd00 con 0x55c90922adb0 2024-09-19T09:25:44.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.155+0000 7fa77cff9640 10 client.22896 handle_command_reply: tid=0 2024-09-19T09:25:44.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.155+0000 7fa79da4d740 1 client.22896 shutdown 2024-09-19T09:25:44.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.155+0000 7fa79da4d740 1 -- 172.21.15.120:0/465307047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c90921ad40 msgr2=0x55c90921ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:44.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.155+0000 7fa79da4d740 1 --2- 172.21.15.120:0/465307047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c90921ad40 0x55c90921ed00 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00aed0 tx=0x7fa78c021d40 comp rx=0 tx=0).stop 2024-09-19T09:25:44.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.155+0000 7fa79da4d740 1 -- 172.21.15.120:0/465307047 shutdown_connections 2024-09-19T09:25:44.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.155+0000 7fa79da4d740 1 --2- 172.21.15.120:0/465307047 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c90922adb0 0x55c909298450 crc :-1 s=READY pgs=562 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.155+0000 7fa79da4d740 1 --2- 172.21.15.120:0/465307047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c90921ad40 0x55c90921ed00 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.155+0000 7fa79da4d740 1 --2- 172.21.15.120:0/465307047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c909210480 0x55c90921e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.155+0000 7fa77cff9640 0 client.22896 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:44.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.155+0000 7fa79da4d740 1 --2- 172.21.15.120:0/465307047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c90920fb60 0x55c909223490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.155+0000 7fa79da4d740 1 -- 172.21.15.120:0/465307047 >> 172.21.15.120:0/465307047 conn(0x55c90920c0f0 msgr2=0x55c9092183d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:44.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.155+0000 7fa79da4d740 1 -- 172.21.15.120:0/465307047 shutdown_connections 2024-09-19T09:25:44.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.155+0000 7fa79da4d740 1 -- 172.21.15.120:0/465307047 wait complete. 2024-09-19T09:25:44.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.155+0000 7fa79da4d740 20 client.22896 trim_cache size 0 max 16384 2024-09-19T09:25:44.159 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:25:44.159 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:25:44.159 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "56430f73-52b4-4c6e-9837-891ab01b4c5d", 2024-09-19T09:25:44.159 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:25:44.159 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:25:44.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.156+0000 7fa79b674640 1 -- 172.21.15.120:0/813023594 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa7800a7650 msgr2=0x7fa7800a9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:44.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.156+0000 7fa79b674640 1 --2- 172.21.15.120:0/813023594 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa7800a7650 0x7fa7800a9b10 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00a640 tx=0x7fa78c004b50 comp rx=0 tx=0).stop 2024-09-19T09:25:44.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.156+0000 7fa79b674640 1 -- 172.21.15.120:0/813023594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa794102d30 msgr2=0x7fa794198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:44.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.156+0000 7fa79b674640 1 --2- 172.21.15.120:0/813023594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa794102d30 0x7fa794198640 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fa790005940 tx=0x7fa7900042f0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.156+0000 7fa79b674640 1 -- 172.21.15.120:0/813023594 shutdown_connections 2024-09-19T09:25:44.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.156+0000 7fa79b674640 1 --2- 172.21.15.120:0/813023594 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa7800a7650 0x7fa7800a9b10 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.156+0000 7fa79b674640 1 --2- 172.21.15.120:0/813023594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7941082a0 0x7fa79419cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.156+0000 7fa79b674640 1 --2- 172.21.15.120:0/813023594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7941036e0 0x7fa794198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.156+0000 7fa79b674640 1 --2- 172.21.15.120:0/813023594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa794102d30 0x7fa794198640 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.156+0000 7fa79b674640 1 -- 172.21.15.120:0/813023594 >> 172.21.15.120:0/813023594 conn(0x7fa7940fc7d0 msgr2=0x7fa794105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:44.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.156+0000 7fa79b674640 1 -- 172.21.15.120:0/813023594 shutdown_connections 2024-09-19T09:25:44.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:44.156+0000 7fa79b674640 1 -- 172.21.15.120:0/813023594 wait complete. 2024-09-19T09:25:44.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:44 smithi120 ceph-mon[31150]: pgmap v1031: 97 pgs: 97 active+clean; 493 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.7 MiB/s wr, 716 op/s 2024-09-19T09:25:44.384 INFO:tasks.workunit.client.0.smithi120.stdout: 10 227986 20.25 MB/sec execute 459 sec latency 78.344 ms 2024-09-19T09:25:44.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:44 smithi196 ceph-mon[35150]: pgmap v1031: 97 pgs: 97 active+clean; 493 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.7 MiB/s wr, 716 op/s 2024-09-19T09:25:44.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:44 smithi153 ceph-mon[35071]: pgmap v1031: 97 pgs: 97 active+clean; 493 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.7 MiB/s wr, 716 op/s 2024-09-19T09:25:45.110 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:25:45.381 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:45.384 INFO:tasks.workunit.client.0.smithi120.stdout: 10 228456 20.28 MB/sec execute 460 sec latency 163.916 ms 2024-09-19T09:25:45.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:45 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:25:45.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:45 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:25:45.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:45 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:25:45.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:45 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:25:45.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:45 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:25:45.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:45 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:25:45.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:45 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:25:45.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:45 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:25:45.502 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:45 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:25:45.502 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:45 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:25:45.502 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:45 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:25:45.502 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:45 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:25:45.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.915+0000 7f7e4aa7a640 1 -- 172.21.15.120:0/624721164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e44107440 msgr2=0x7f7e44100100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:46.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.915+0000 7f7e4aa7a640 1 --2- 172.21.15.120:0/624721164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e44107440 0x7f7e44100100 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f7e30009900 tx=0x7f7e3002a410 comp rx=0 tx=0).stop 2024-09-19T09:25:46.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.916+0000 7f7e4aa7a640 1 -- 172.21.15.120:0/624721164 shutdown_connections 2024-09-19T09:25:46.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.916+0000 7f7e4aa7a640 1 --2- 172.21.15.120:0/624721164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e44100860 0x7f7e441045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.916+0000 7f7e4aa7a640 1 --2- 172.21.15.120:0/624721164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e44107440 0x7f7e44100100 unknown :-1 s=CLOSED pgs=2352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.916+0000 7f7e4aa7a640 1 --2- 172.21.15.120:0/624721164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e44106a90 0x7f7e44106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.916+0000 7f7e4aa7a640 1 -- 172.21.15.120:0/624721164 >> 172.21.15.120:0/624721164 conn(0x7f7e44077230 msgr2=0x7f7e440fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:46.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.917+0000 7f7e4aa7a640 1 -- 172.21.15.120:0/624721164 shutdown_connections 2024-09-19T09:25:46.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.917+0000 7f7e4aa7a640 1 -- 172.21.15.120:0/624721164 wait complete. 2024-09-19T09:25:46.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.918+0000 7f7e4aa7a640 1 Processor -- start 2024-09-19T09:25:46.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.918+0000 7f7e4aa7a640 1 -- start start 2024-09-19T09:25:46.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.919+0000 7f7e4aa7a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e44100860 0x7f7e441a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:46.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.919+0000 7f7e4aa7a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e44106a90 0x7f7e441a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:46.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.919+0000 7f7e4aa7a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e4419e8d0 0x7f7e4419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:46.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.919+0000 7f7e4aa7a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7e441185e0 con 0x7f7e44106a90 2024-09-19T09:25:46.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.919+0000 7f7e4aa7a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7e44118460 con 0x7f7e44100860 2024-09-19T09:25:46.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.919+0000 7f7e4aa7a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7e44118760 con 0x7f7e4419e8d0 2024-09-19T09:25:46.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.919+0000 7f7e48816640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e44100860 0x7f7e441a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:46.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.919+0000 7f7e48816640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e44100860 0x7f7e441a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:49356/0 (socket says 172.21.15.120:49356) 2024-09-19T09:25:46.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.919+0000 7f7e48816640 1 -- 172.21.15.120:0/155040513 learned_addr learned my addr 172.21.15.120:0/155040513 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:46.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.919+0000 7f7e43fff640 1 --2- 172.21.15.120:0/155040513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e44106a90 0x7f7e441a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:46.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.919+0000 7f7e49017640 1 --2- 172.21.15.120:0/155040513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e4419e8d0 0x7f7e4419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:46.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.919+0000 7f7e48816640 1 -- 172.21.15.120:0/155040513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e4419e8d0 msgr2=0x7f7e4419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:46.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.919+0000 7f7e48816640 1 --2- 172.21.15.120:0/155040513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e4419e8d0 0x7f7e4419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.919+0000 7f7e48816640 1 -- 172.21.15.120:0/155040513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e44106a90 msgr2=0x7f7e441a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:46.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.919+0000 7f7e48816640 1 --2- 172.21.15.120:0/155040513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e44106a90 0x7f7e441a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.919+0000 7f7e48816640 1 -- 172.21.15.120:0/155040513 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e4419f3f0 con 0x7f7e44100860 2024-09-19T09:25:46.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.920+0000 7f7e48816640 1 --2- 172.21.15.120:0/155040513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e44100860 0x7f7e441a4710 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7e3800d950 tx=0x7f7e3800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:46.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.920+0000 7f7e41ffb640 1 -- 172.21.15.120:0/155040513 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e38017030 con 0x7f7e44100860 2024-09-19T09:25:46.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.920+0000 7f7e41ffb640 1 -- 172.21.15.120:0/155040513 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e38004020 con 0x7f7e44100860 2024-09-19T09:25:46.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.920+0000 7f7e4aa7a640 1 -- 172.21.15.120:0/155040513 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e4419f6e0 con 0x7f7e44100860 2024-09-19T09:25:46.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.921+0000 7f7e41ffb640 1 -- 172.21.15.120:0/155040513 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e380047a0 con 0x7f7e44100860 2024-09-19T09:25:46.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.921+0000 7f7e4aa7a640 1 -- 172.21.15.120:0/155040513 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e441ab8c0 con 0x7f7e44100860 2024-09-19T09:25:46.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.923+0000 7f7e41ffb640 1 -- 172.21.15.120:0/155040513 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f7e380041d0 con 0x7f7e44100860 2024-09-19T09:25:46.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.924+0000 7f7e4ce53740 1 Processor -- start 2024-09-19T09:25:46.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.924+0000 7f7e4ce53740 1 -- start start 2024-09-19T09:25:46.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.924+0000 7f7e4ce53740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558354934a0 0x555835493880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:46.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.924+0000 7f7e4ce53740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555835493dc0 0x55583549e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:46.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.924+0000 7f7e41ffb640 1 --2- 172.21.15.120:0/155040513 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7e340a33c0 0x7f7e340a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:46.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.924+0000 7f7e4ce53740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583549e680 0x5558354a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:46.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.924+0000 7f7e4ce53740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5558353f92e0 con 0x5558354934a0 2024-09-19T09:25:46.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.924+0000 7f7e4ce53740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5558353a0ce0 con 0x555835493dc0 2024-09-19T09:25:46.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.924+0000 7f7e4ce53740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5558353f52a0 con 0x55583549e680 2024-09-19T09:25:46.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.925+0000 7f7e48816640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555835493dc0 0x55583549e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:46.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.925+0000 7f7e49017640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558354934a0 0x555835493880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:46.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.925+0000 7f7e49017640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558354934a0 0x555835493880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57444/0 (socket says 172.21.15.120:57444) 2024-09-19T09:25:46.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.925+0000 7f7e48816640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555835493dc0 0x55583549e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:49362/0 (socket says 172.21.15.120:49362) 2024-09-19T09:25:46.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.925+0000 7f7e49017640 1 -- 172.21.15.120:0/1357117689 learned_addr learned my addr 172.21.15.120:0/1357117689 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:46.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.925+0000 7f7e41ffb640 1 -- 172.21.15.120:0/155040513 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f7e380cf110 con 0x7f7e44100860 2024-09-19T09:25:46.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.925+0000 7f7e49017640 1 --2- 172.21.15.120:0/1357117689 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583549e680 0x5558354a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:46.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.925+0000 7f7e43fff640 1 --2- 172.21.15.120:0/155040513 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7e340a33c0 0x7f7e340a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:46.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.925+0000 7f7e49017640 1 -- 172.21.15.120:0/1357117689 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583549e680 msgr2=0x5558354a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:46.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.925+0000 7f7e49017640 1 --2- 172.21.15.120:0/1357117689 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583549e680 0x5558354a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.925+0000 7f7e49017640 1 -- 172.21.15.120:0/1357117689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555835493dc0 msgr2=0x55583549e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:46.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.925+0000 7f7e49017640 1 --2- 172.21.15.120:0/1357117689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555835493dc0 0x55583549e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.925+0000 7f7e49017640 1 -- 172.21.15.120:0/1357117689 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555835385440 con 0x5558354934a0 2024-09-19T09:25:46.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.925+0000 7f7e49017640 1 --2- 172.21.15.120:0/1357117689 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558354934a0 0x555835493880 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c009fa0 tx=0x7f7e3c009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff325bd889c4712 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:46.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.925+0000 7f7e43fff640 1 --2- 172.21.15.120:0/155040513 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7e340a33c0 0x7f7e340a5880 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7e30006010 tx=0x7f7e30000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:46.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.926+0000 7f7e2effd640 1 -- 172.21.15.120:0/1357117689 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c010ac0 con 0x5558354934a0 2024-09-19T09:25:46.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.926+0000 7f7e2effd640 1 -- 172.21.15.120:0/1357117689 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e3c010c60 con 0x5558354934a0 2024-09-19T09:25:46.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.926+0000 7f7e2effd640 1 -- 172.21.15.120:0/1357117689 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c00c5f0 con 0x5558354934a0 2024-09-19T09:25:46.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.927+0000 7f7e4ce53740 1 -- 172.21.15.120:0/1357117689 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558354934a0 msgr2=0x555835493880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:46.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.927+0000 7f7e4ce53740 1 --2- 172.21.15.120:0/1357117689 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558354934a0 0x555835493880 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c009fa0 tx=0x7f7e3c009910 comp rx=0 tx=0).stop 2024-09-19T09:25:46.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.927+0000 7f7e4ce53740 1 -- 172.21.15.120:0/1357117689 shutdown_connections 2024-09-19T09:25:46.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.927+0000 7f7e4ce53740 1 --2- 172.21.15.120:0/1357117689 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583549e680 0x5558354a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.927+0000 7f7e4ce53740 1 --2- 172.21.15.120:0/1357117689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555835493dc0 0x55583549e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.927+0000 7f7e4ce53740 1 --2- 172.21.15.120:0/1357117689 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5558354934a0 0x555835493880 unknown :-1 s=CLOSED pgs=2353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.927+0000 7f7e4ce53740 1 -- 172.21.15.120:0/1357117689 >> 172.21.15.120:0/1357117689 conn(0x55583548fa30 msgr2=0x55583548fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:46.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.927+0000 7f7e4ce53740 1 -- 172.21.15.120:0/1357117689 shutdown_connections 2024-09-19T09:25:46.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.927+0000 7f7e4ce53740 1 -- 172.21.15.120:0/1357117689 wait complete. 2024-09-19T09:25:46.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.928+0000 7f7e4ce53740 1 -- start start 2024-09-19T09:25:46.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.928+0000 7f7e4ce53740 1 Processor -- start 2024-09-19T09:25:46.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.928+0000 7f7e4ce53740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558354934a0 0x55583549df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:46.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.928+0000 7f7e4ce53740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555835493dc0 0x55583549b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:46.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.928+0000 7f7e4ce53740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583549e680 0x55583549b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:46.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.928+0000 7f7e4ce53740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5558354a1250 con 0x555835493dc0 2024-09-19T09:25:46.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.928+0000 7f7e4ce53740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5558353b4050 con 0x5558354934a0 2024-09-19T09:25:46.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.928+0000 7f7e4ce53740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5558354a13d0 con 0x55583549e680 2024-09-19T09:25:46.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.928+0000 7f7e4ce53740 10 client.0 _ll_register_callbacks cb 0x5558353f4160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:25:46.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.928+0000 7f7e48816640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558354934a0 0x55583549df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:46.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.929+0000 7f7e43fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555835493dc0 0x55583549b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:46.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.929+0000 7f7e48816640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558354934a0 0x55583549df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:49370/0 (socket says 172.21.15.120:49370) 2024-09-19T09:25:46.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.929+0000 7f7e48816640 1 -- 172.21.15.120:0/305400655 learned_addr learned my addr 172.21.15.120:0/305400655 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:46.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.929+0000 7f7e49017640 1 --2- 172.21.15.120:0/305400655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583549e680 0x55583549b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:46.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.929+0000 7f7e48816640 1 -- 172.21.15.120:0/305400655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583549e680 msgr2=0x55583549b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:46.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.929+0000 7f7e48816640 1 --2- 172.21.15.120:0/305400655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583549e680 0x55583549b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.929+0000 7f7e48816640 1 -- 172.21.15.120:0/305400655 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555835493dc0 msgr2=0x55583549b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:46.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.929+0000 7f7e48816640 1 --2- 172.21.15.120:0/305400655 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555835493dc0 0x55583549b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.929+0000 7f7e48816640 1 -- 172.21.15.120:0/305400655 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55583516a3c0 con 0x5558354934a0 2024-09-19T09:25:46.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.929+0000 7f7e48816640 1 --2- 172.21.15.120:0/305400655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558354934a0 0x55583549df00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7e380ccc10 tx=0x7f7e380973a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=305e54b5c09e5386 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:46.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.929+0000 7f7e23fff640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:25:46.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.930+0000 7f7e23fff640 1 -- 172.21.15.120:0/305400655 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e380b8020 con 0x5558354934a0 2024-09-19T09:25:46.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.930+0000 7f7e4ce53740 1 -- 172.21.15.120:0/305400655 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e3c010c60 con 0x5558354934a0 2024-09-19T09:25:46.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.930+0000 7f7e23fff640 1 -- 172.21.15.120:0/305400655 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f7e380b4cf0 con 0x5558354934a0 2024-09-19T09:25:46.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.930+0000 7f7e23fff640 1 -- 172.21.15.120:0/305400655 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7e380b4930 con 0x5558354934a0 2024-09-19T09:25:46.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.930+0000 7f7e4ce53740 10 client.31577 fetch_fsmap learned FSMap version 39 2024-09-19T09:25:46.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.930+0000 7f7e4ce53740 1 -- 172.21.15.120:0/305400655 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5558354acb60 con 0x5558354934a0 2024-09-19T09:25:46.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.931+0000 7f7e23fff640 1 -- 172.21.15.120:0/305400655 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e3c010c60 con 0x5558354934a0 2024-09-19T09:25:46.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.931+0000 7f7e23fff640 1 -- 172.21.15.120:0/305400655 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f7e380b8e10 con 0x5558354934a0 2024-09-19T09:25:46.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.932+0000 7f7e4ce53740 10 client.31577 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:25:46.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.932+0000 7f7e4ce53740 10 client.31577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:25:46.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.932+0000 7f7e4ce53740 20 client.31577 populate_metadata read hostname 'smithi120' 2024-09-19T09:25:46.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.932+0000 7f7e4ce53740 1 --2- 172.21.15.120:0/305400655 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5558354adeb0 0x5558354b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:46.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.932+0000 7f7e4ce53740 4 client.31577 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:25:46.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.932+0000 7f7e4ce53740 1 -- 172.21.15.120:0/305400655 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555835406850 con 0x5558354adeb0 2024-09-19T09:25:46.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.932+0000 7f7e43fff640 1 --2- 172.21.15.120:0/305400655 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5558354adeb0 0x5558354b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:46.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.933+0000 7f7e23fff640 10 client.31577 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:46.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.933+0000 7f7e43fff640 1 --2- 172.21.15.120:0/305400655 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5558354adeb0 0x5558354b0300 crc :-1 s=READY pgs=563 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2c59a2ccb07632b server_cookie=3e5fddda163db3f1 in_seq=0 out_seq=0 2024-09-19T09:25:46.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.938+0000 7f7e23fff640 1 -- 172.21.15.120:0/305400655 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555835406850 con 0x5558354adeb0 2024-09-19T09:25:46.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.938+0000 7f7e23fff640 10 client.31577 handle_command_reply: tid=0 2024-09-19T09:25:46.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.938+0000 7f7e4ce53740 1 client.31577 shutdown 2024-09-19T09:25:46.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.938+0000 7f7e4ce53740 1 -- 172.21.15.120:0/305400655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558354934a0 msgr2=0x55583549df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:46.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.938+0000 7f7e4ce53740 1 --2- 172.21.15.120:0/305400655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558354934a0 0x55583549df00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7e380ccc10 tx=0x7f7e380973a0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.938+0000 7f7e4ce53740 1 -- 172.21.15.120:0/305400655 shutdown_connections 2024-09-19T09:25:46.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.938+0000 7f7e4ce53740 1 --2- 172.21.15.120:0/305400655 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5558354adeb0 0x5558354b0300 crc :-1 s=READY pgs=563 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.938+0000 7f7e23fff640 0 client.31577 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:46.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.938+0000 7f7e4ce53740 1 --2- 172.21.15.120:0/305400655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583549e680 0x55583549b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.938+0000 7f7e4ce53740 1 --2- 172.21.15.120:0/305400655 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555835493dc0 0x55583549b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.938+0000 7f7e4ce53740 1 --2- 172.21.15.120:0/305400655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558354934a0 0x55583549df00 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.938+0000 7f7e4ce53740 1 -- 172.21.15.120:0/305400655 >> 172.21.15.120:0/305400655 conn(0x55583548fa30 msgr2=0x55583548eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:46.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.939+0000 7f7e4ce53740 1 -- 172.21.15.120:0/305400655 shutdown_connections 2024-09-19T09:25:46.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.939+0000 7f7e4ce53740 1 -- 172.21.15.120:0/305400655 wait complete. 2024-09-19T09:25:46.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.939+0000 7f7e4ce53740 20 client.31577 trim_cache size 0 max 16384 2024-09-19T09:25:46.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.960+0000 7f7e4ce53740 1 Processor -- start 2024-09-19T09:25:46.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.960+0000 7f7e4ce53740 1 -- start start 2024-09-19T09:25:46.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.960+0000 7f7e4ce53740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55583548fa30 0x5558354aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:46.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.960+0000 7f7e4ce53740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558354934a0 0x5558354b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:46.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.960+0000 7f7e4ce53740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555835493dc0 0x5558354b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:46.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.960+0000 7f7e4ce53740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7e441b3cc0 con 0x55583548fa30 2024-09-19T09:25:46.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.960+0000 7f7e4ce53740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7e441b5800 con 0x555835493dc0 2024-09-19T09:25:46.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.960+0000 7f7e4ce53740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7e441b4390 con 0x5558354934a0 2024-09-19T09:25:46.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.960+0000 7f7e48816640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55583548fa30 0x5558354aa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:46.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.960+0000 7f7e48816640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55583548fa30 0x5558354aa4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57464/0 (socket says 172.21.15.120:57464) 2024-09-19T09:25:46.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.960+0000 7f7e48816640 1 -- 172.21.15.120:0/734961033 learned_addr learned my addr 172.21.15.120:0/734961033 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:46.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.960+0000 7f7e49017640 1 --2- 172.21.15.120:0/734961033 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555835493dc0 0x5558354b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:46.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.960+0000 7f7e43fff640 1 --2- 172.21.15.120:0/734961033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558354934a0 0x5558354b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:46.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.960+0000 7f7e48816640 1 -- 172.21.15.120:0/734961033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558354934a0 msgr2=0x5558354b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:46.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.960+0000 7f7e48816640 1 --2- 172.21.15.120:0/734961033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558354934a0 0x5558354b5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.960+0000 7f7e48816640 1 -- 172.21.15.120:0/734961033 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555835493dc0 msgr2=0x5558354b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:46.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.960+0000 7f7e48816640 1 --2- 172.21.15.120:0/734961033 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555835493dc0 0x5558354b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.960+0000 7f7e48816640 1 -- 172.21.15.120:0/734961033 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55583549e270 con 0x55583548fa30 2024-09-19T09:25:46.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.961+0000 7f7e48816640 1 --2- 172.21.15.120:0/734961033 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55583548fa30 0x5558354aa4b0 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f7e380999e0 tx=0x7f7e38000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b73e910a0009c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:46.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.961+0000 7f7e2e7fc640 1 -- 172.21.15.120:0/734961033 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e380b8070 con 0x55583548fa30 2024-09-19T09:25:46.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.961+0000 7f7e2e7fc640 1 -- 172.21.15.120:0/734961033 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e3809be30 con 0x55583548fa30 2024-09-19T09:25:46.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.961+0000 7f7e2e7fc640 1 -- 172.21.15.120:0/734961033 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3809a5f0 con 0x55583548fa30 2024-09-19T09:25:46.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.962+0000 7f7e4ce53740 1 -- 172.21.15.120:0/734961033 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55583548fa30 msgr2=0x5558354aa4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:46.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.962+0000 7f7e4ce53740 1 --2- 172.21.15.120:0/734961033 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55583548fa30 0x5558354aa4b0 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f7e380999e0 tx=0x7f7e38000bc0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.962+0000 7f7e4ce53740 1 -- 172.21.15.120:0/734961033 shutdown_connections 2024-09-19T09:25:46.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.962+0000 7f7e4ce53740 1 --2- 172.21.15.120:0/734961033 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555835493dc0 0x5558354b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.962+0000 7f7e4ce53740 1 --2- 172.21.15.120:0/734961033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558354934a0 0x5558354b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.962+0000 7f7e4ce53740 1 --2- 172.21.15.120:0/734961033 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55583548fa30 0x5558354aa4b0 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.962+0000 7f7e4ce53740 1 -- 172.21.15.120:0/734961033 >> 172.21.15.120:0/734961033 conn(0x55583549b850 msgr2=0x555835499a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:46.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.962+0000 7f7e4ce53740 1 -- 172.21.15.120:0/734961033 shutdown_connections 2024-09-19T09:25:46.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.962+0000 7f7e4ce53740 1 -- 172.21.15.120:0/734961033 wait complete. 2024-09-19T09:25:46.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.963+0000 7f7e4ce53740 1 -- start start 2024-09-19T09:25:46.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.963+0000 7f7e4ce53740 1 Processor -- start 2024-09-19T09:25:46.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.963+0000 7f7e4ce53740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583548fa30 0x5558354a6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:46.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.963+0000 7f7e4ce53740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558354934a0 0x5558354a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:46.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.963+0000 7f7e4ce53740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555835493dc0 0x5558354a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:46.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.964+0000 7f7e4ce53740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7e441b5130 con 0x555835493dc0 2024-09-19T09:25:46.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.964+0000 7f7e4ce53740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7e441b4a60 con 0x5558354934a0 2024-09-19T09:25:46.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.964+0000 7f7e4ce53740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7e441b1c80 con 0x55583548fa30 2024-09-19T09:25:46.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.964+0000 7f7e4ce53740 10 client.0 _ll_register_callbacks cb 0x7f7e3809b500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:25:46.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.964+0000 7f7e49017640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555835493dc0 0x5558354a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:46.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.964+0000 7f7e48816640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583548fa30 0x5558354a6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:46.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.964+0000 7f7e49017640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555835493dc0 0x5558354a24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57474/0 (socket says 172.21.15.120:57474) 2024-09-19T09:25:46.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.964+0000 7f7e49017640 1 -- 172.21.15.120:0/2926446498 learned_addr learned my addr 172.21.15.120:0/2926446498 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:46.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.964+0000 7f7e48816640 1 --2- 172.21.15.120:0/2926446498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583548fa30 0x5558354a6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:58336/0 (socket says 172.21.15.120:58336) 2024-09-19T09:25:46.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.964+0000 7f7e43fff640 1 --2- 172.21.15.120:0/2926446498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558354934a0 0x5558354a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:46.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.964+0000 7f7e43fff640 1 -- 172.21.15.120:0/2926446498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583548fa30 msgr2=0x5558354a6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:46.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.964+0000 7f7e43fff640 1 --2- 172.21.15.120:0/2926446498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583548fa30 0x5558354a6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.964+0000 7f7e43fff640 1 -- 172.21.15.120:0/2926446498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555835493dc0 msgr2=0x5558354a24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:46.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.964+0000 7f7e43fff640 1 --2- 172.21.15.120:0/2926446498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555835493dc0 0x5558354a24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.964+0000 7f7e43fff640 1 -- 172.21.15.120:0/2926446498 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558354a2b90 con 0x5558354934a0 2024-09-19T09:25:46.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.965+0000 7f7e43fff640 1 --2- 172.21.15.120:0/2926446498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558354934a0 0x5558354a1f60 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7e3002a930 tx=0x7f7e3003d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ffc8b0bfa29c51e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:46.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.965+0000 7f7e23fff640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:25:46.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.965+0000 7f7e23fff640 1 -- 172.21.15.120:0/2926446498 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e30035040 con 0x5558354934a0 2024-09-19T09:25:46.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.965+0000 7f7e23fff640 1 -- 172.21.15.120:0/2926446498 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f7e30005570 con 0x5558354934a0 2024-09-19T09:25:46.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.965+0000 7f7e4ce53740 1 -- 172.21.15.120:0/2926446498 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e3809be30 con 0x5558354934a0 2024-09-19T09:25:46.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.965+0000 7f7e23fff640 1 -- 172.21.15.120:0/2926446498 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7e300046c0 con 0x5558354934a0 2024-09-19T09:25:46.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.965+0000 7f7e4ce53740 10 client.31583 fetch_fsmap learned FSMap version 39 2024-09-19T09:25:46.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.965+0000 7f7e4ce53740 1 -- 172.21.15.120:0/2926446498 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555835518ac0 con 0x5558354934a0 2024-09-19T09:25:46.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.966+0000 7f7e23fff640 1 -- 172.21.15.120:0/2926446498 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e3809be30 con 0x5558354934a0 2024-09-19T09:25:46.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.966+0000 7f7e23fff640 1 -- 172.21.15.120:0/2926446498 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f7e30032b70 con 0x5558354934a0 2024-09-19T09:25:46.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.966+0000 7f7e4ce53740 10 client.31583 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:25:46.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.966+0000 7f7e4ce53740 10 client.31583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:25:46.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.966+0000 7f7e4ce53740 20 client.31583 populate_metadata read hostname 'smithi120' 2024-09-19T09:25:46.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.966+0000 7f7e4ce53740 1 --2- 172.21.15.120:0/2926446498 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55583549e680 0x55583551b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:46.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.966+0000 7f7e4ce53740 4 client.31583 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:25:46.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.966+0000 7f7e4ce53740 1 -- 172.21.15.120:0/2926446498 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555835400d60 con 0x55583549e680 2024-09-19T09:25:46.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.966+0000 7f7e48816640 1 --2- 172.21.15.120:0/2926446498 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55583549e680 0x55583551b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:46.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.967+0000 7f7e48816640 1 --2- 172.21.15.120:0/2926446498 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55583549e680 0x55583551b2b0 crc :-1 s=READY pgs=564 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baf09a9a28ac8d1b server_cookie=9f5c70b95e69f121 in_seq=0 out_seq=0 2024-09-19T09:25:46.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.967+0000 7f7e23fff640 10 client.31583 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:46.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.971+0000 7f7e23fff640 1 -- 172.21.15.120:0/2926446498 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555835400d60 con 0x55583549e680 2024-09-19T09:25:46.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.971+0000 7f7e23fff640 10 client.31583 handle_command_reply: tid=0 2024-09-19T09:25:46.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.971+0000 7f7e4ce53740 1 client.31583 shutdown 2024-09-19T09:25:46.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.972+0000 7f7e4ce53740 1 -- 172.21.15.120:0/2926446498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558354934a0 msgr2=0x5558354a1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:46.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.972+0000 7f7e4ce53740 1 --2- 172.21.15.120:0/2926446498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558354934a0 0x5558354a1f60 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7e3002a930 tx=0x7f7e3003d000 comp rx=0 tx=0).stop 2024-09-19T09:25:46.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.972+0000 7f7e4ce53740 1 -- 172.21.15.120:0/2926446498 shutdown_connections 2024-09-19T09:25:46.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.972+0000 7f7e4ce53740 1 --2- 172.21.15.120:0/2926446498 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55583549e680 0x55583551b2b0 crc :-1 s=READY pgs=564 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.972+0000 7f7e4ce53740 1 --2- 172.21.15.120:0/2926446498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555835493dc0 0x5558354a24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.972+0000 7f7e4ce53740 1 --2- 172.21.15.120:0/2926446498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5558354934a0 0x5558354a1f60 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.972+0000 7f7e4ce53740 1 --2- 172.21.15.120:0/2926446498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583548fa30 0x5558354a6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.972+0000 7f7e4ce53740 1 -- 172.21.15.120:0/2926446498 >> 172.21.15.120:0/2926446498 conn(0x55583549b850 msgr2=0x55583548e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:46.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.972+0000 7f7e23fff640 0 client.31583 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:46.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.972+0000 7f7e4ce53740 1 -- 172.21.15.120:0/2926446498 shutdown_connections 2024-09-19T09:25:46.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.972+0000 7f7e4ce53740 1 -- 172.21.15.120:0/2926446498 wait complete. 2024-09-19T09:25:46.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.972+0000 7f7e4ce53740 20 client.31583 trim_cache size 0 max 16384 2024-09-19T09:25:46.287 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:25:46.287 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:25:46.287 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-19T09:25:46.287 INFO:teuthology.orchestra.run.smithi120.stdout: "56430f73-52b4-4c6e-9837-891ab01b4c5d": { 2024-09-19T09:25:46.287 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-19T09:25:46.287 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "56430f73-52b4-4c6e-9837-891ab01b4c5d", 2024-09-19T09:25:46.287 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-19T09:25:46.287 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:25:46.287 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:25:46.287 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:25:46.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.973+0000 7f7e4aa7a640 1 -- 172.21.15.120:0/155040513 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7e340a33c0 msgr2=0x7f7e340a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:46.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.973+0000 7f7e4aa7a640 1 --2- 172.21.15.120:0/155040513 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7e340a33c0 0x7f7e340a5880 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7e30006010 tx=0x7f7e30000bc0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.973+0000 7f7e4aa7a640 1 -- 172.21.15.120:0/155040513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e44100860 msgr2=0x7f7e441a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:46.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.973+0000 7f7e4aa7a640 1 --2- 172.21.15.120:0/155040513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e44100860 0x7f7e441a4710 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7e3800d950 tx=0x7f7e3800de10 comp rx=0 tx=0).stop 2024-09-19T09:25:46.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.973+0000 7f7e4aa7a640 1 -- 172.21.15.120:0/155040513 shutdown_connections 2024-09-19T09:25:46.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.973+0000 7f7e4aa7a640 1 --2- 172.21.15.120:0/155040513 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7e340a33c0 0x7f7e340a5880 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.973+0000 7f7e4aa7a640 1 --2- 172.21.15.120:0/155040513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e4419e8d0 0x7f7e4419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.974+0000 7f7e4aa7a640 1 --2- 172.21.15.120:0/155040513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e44106a90 0x7f7e441a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.974+0000 7f7e4aa7a640 1 --2- 172.21.15.120:0/155040513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e44100860 0x7f7e441a4710 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.974+0000 7f7e4aa7a640 1 -- 172.21.15.120:0/155040513 >> 172.21.15.120:0/155040513 conn(0x7f7e44077230 msgr2=0x7f7e441026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:46.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.974+0000 7f7e4aa7a640 1 -- 172.21.15.120:0/155040513 shutdown_connections 2024-09-19T09:25:46.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:25:45.974+0000 7f7e4aa7a640 1 -- 172.21.15.120:0/155040513 wait complete. 2024-09-19T09:25:46.384 INFO:tasks.workunit.client.0.smithi120.stdout: 10 228982 20.29 MB/sec execute 461 sec latency 64.205 ms 2024-09-19T09:25:46.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:46 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:25:46.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:46 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:25:46.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:46 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:25:46.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:46 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:25:46.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:46 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:25:46.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:46 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:25:46.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:46 smithi196 ceph-mon[35150]: pgmap v1032: 97 pgs: 97 active+clean; 494 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.2 MiB/s wr, 692 op/s 2024-09-19T09:25:46.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:46 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:25:46.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:46 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:25:46.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:46 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:25:46.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:46 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:25:46.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:46 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:25:46.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:46 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:25:46.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:46 smithi153 ceph-mon[35071]: pgmap v1032: 97 pgs: 97 active+clean; 494 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.2 MiB/s wr, 692 op/s 2024-09-19T09:25:46.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:46 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:25:46.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:46 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:25:46.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:46 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:25:46.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:46 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:25:46.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:46 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:25:46.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:46 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:25:46.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:46 smithi120 ceph-mon[31150]: pgmap v1032: 97 pgs: 97 active+clean; 494 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.2 MiB/s wr, 692 op/s 2024-09-19T09:25:47.099 INFO:tasks.cephfs.filesystem:scrub status for tag:56430f73-52b4-4c6e-9837-891ab01b4c5d - {'path': '/', 'tag': '56430f73-52b4-4c6e-9837-891ab01b4c5d', 'options': 'recursive,force'} 2024-09-19T09:25:47.384 INFO:tasks.workunit.client.0.smithi120.stdout: 10 229361 20.29 MB/sec execute 462 sec latency 213.915 ms 2024-09-19T09:25:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.633+0000 7fe2bd933640 1 -- 172.21.15.120:0/2828361337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2b80ffe10 msgr2=0x7fe2b8100210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.633+0000 7fe2bd933640 1 --2- 172.21.15.120:0/2828361337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2b80ffe10 0x7fe2b8100210 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009900 tx=0x7fe2a802a410 comp rx=0 tx=0).stop 2024-09-19T09:25:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.633+0000 7fe2bd933640 1 -- 172.21.15.120:0/2828361337 shutdown_connections 2024-09-19T09:25:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.633+0000 7fe2bd933640 1 --2- 172.21.15.120:0/2828361337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2b81037a0 0x7fe2b810a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.633+0000 7fe2bd933640 1 --2- 172.21.15.120:0/2828361337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2b8102de0 0x7fe2b8103260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.633+0000 7fe2bd933640 1 --2- 172.21.15.120:0/2828361337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2b80ffe10 0x7fe2b8100210 unknown :-1 s=CLOSED pgs=2355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.633+0000 7fe2bd933640 1 -- 172.21.15.120:0/2828361337 >> 172.21.15.120:0/2828361337 conn(0x7fe2b80fb930 msgr2=0x7fe2b80fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.633+0000 7fe2bd933640 1 -- 172.21.15.120:0/2828361337 shutdown_connections 2024-09-19T09:25:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.633+0000 7fe2bd933640 1 -- 172.21.15.120:0/2828361337 wait complete. 2024-09-19T09:25:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.634+0000 7fe2bd933640 1 Processor -- start 2024-09-19T09:25:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.634+0000 7fe2bd933640 1 -- start start 2024-09-19T09:25:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.634+0000 7fe2bd933640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2b80ffe10 0x7fe2b8102750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.634+0000 7fe2bd933640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2b8102de0 0x7fe2b8100d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.634+0000 7fe2bd933640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2b81037a0 0x7fe2b81012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.634+0000 7fe2bd933640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b810d810 con 0x7fe2b8102de0 2024-09-19T09:25:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.634+0000 7fe2bd933640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b810d690 con 0x7fe2b80ffe10 2024-09-19T09:25:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.634+0000 7fe2bd933640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b810d990 con 0x7fe2b81037a0 2024-09-19T09:25:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.634+0000 7fe2b67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2b8102de0 0x7fe2b8100d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.634+0000 7fe2b6ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2b80ffe10 0x7fe2b8102750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.634+0000 7fe2b6ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2b80ffe10 0x7fe2b8102750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:49392/0 (socket says 172.21.15.120:49392) 2024-09-19T09:25:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.634+0000 7fe2b67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2b8102de0 0x7fe2b8100d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57498/0 (socket says 172.21.15.120:57498) 2024-09-19T09:25:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.634+0000 7fe2b6ffd640 1 -- 172.21.15.120:0/1705591700 learned_addr learned my addr 172.21.15.120:0/1705591700 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.634+0000 7fe2b77fe640 1 --2- 172.21.15.120:0/1705591700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2b81037a0 0x7fe2b81012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.635+0000 7fe2b6ffd640 1 -- 172.21.15.120:0/1705591700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2b81037a0 msgr2=0x7fe2b81012d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.635+0000 7fe2b6ffd640 1 --2- 172.21.15.120:0/1705591700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2b81037a0 0x7fe2b81012d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.635+0000 7fe2b6ffd640 1 -- 172.21.15.120:0/1705591700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2b8102de0 msgr2=0x7fe2b8100d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.635+0000 7fe2b6ffd640 1 --2- 172.21.15.120:0/1705591700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2b8102de0 0x7fe2b8100d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.635+0000 7fe2b6ffd640 1 -- 172.21.15.120:0/1705591700 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b8101b60 con 0x7fe2b80ffe10 2024-09-19T09:25:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.635+0000 7fe2b67fc640 1 --2- 172.21.15.120:0/1705591700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2b8102de0 0x7fe2b8100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:25:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.635+0000 7fe2b6ffd640 1 --2- 172.21.15.120:0/1705591700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2b80ffe10 0x7fe2b8102750 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8031440 tx=0x7fe2a8032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.635+0000 7fe2bc931640 1 -- 172.21.15.120:0/1705591700 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a803f030 con 0x7fe2b80ffe10 2024-09-19T09:25:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.635+0000 7fe2bd933640 1 -- 172.21.15.120:0/1705591700 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b81a1e90 con 0x7fe2b80ffe10 2024-09-19T09:25:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.635+0000 7fe2bc931640 1 -- 172.21.15.120:0/1705591700 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2a8003ce0 con 0x7fe2b80ffe10 2024-09-19T09:25:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.635+0000 7fe2bc931640 1 -- 172.21.15.120:0/1705591700 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8035070 con 0x7fe2b80ffe10 2024-09-19T09:25:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.635+0000 7fe2bd933640 1 -- 172.21.15.120:0/1705591700 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b81a22a0 con 0x7fe2b80ffe10 2024-09-19T09:25:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.638+0000 7fe2c06c8740 1 Processor -- start 2024-09-19T09:25:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.638+0000 7fe2c06c8740 1 -- start start 2024-09-19T09:25:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.638+0000 7fe2c06c8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614983fa9a0 0x5614983fada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.638+0000 7fe2c06c8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614983fb2e0 0x5614983ffda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.638+0000 7fe2c06c8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561498400370 0x561498402830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.638+0000 7fe2c06c8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56149835e2f0 con 0x5614983fa9a0 2024-09-19T09:25:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.638+0000 7fe2c06c8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561498306270 con 0x5614983fb2e0 2024-09-19T09:25:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.638+0000 7fe2c06c8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56149835a2b0 con 0x561498400370 2024-09-19T09:25:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.639+0000 7fe2bc931640 1 -- 172.21.15.120:0/1705591700 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe2a8042030 con 0x7fe2b80ffe10 2024-09-19T09:25:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.639+0000 7fe2b77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614983fa9a0 0x5614983fada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.639+0000 7fe2b67fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561498400370 0x561498402830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.639+0000 7fe2b77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614983fa9a0 0x5614983fada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57500/0 (socket says 172.21.15.120:57500) 2024-09-19T09:25:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.639+0000 7fe2b77fe640 1 -- 172.21.15.120:0/340530830 learned_addr learned my addr 172.21.15.120:0/340530830 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.639+0000 7fe2b6ffd640 1 --2- 172.21.15.120:0/340530830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614983fb2e0 0x5614983ffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.639+0000 7fe2b77fe640 1 -- 172.21.15.120:0/340530830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561498400370 msgr2=0x561498402830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.639+0000 7fe2b77fe640 1 --2- 172.21.15.120:0/340530830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561498400370 0x561498402830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.639+0000 7fe2b77fe640 1 -- 172.21.15.120:0/340530830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614983fb2e0 msgr2=0x5614983ffda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.639+0000 7fe2b77fe640 1 --2- 172.21.15.120:0/340530830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614983fb2e0 0x5614983ffda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.639+0000 7fe2b77fe640 1 -- 172.21.15.120:0/340530830 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561498305340 con 0x5614983fa9a0 2024-09-19T09:25:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.639+0000 7fe2b77fe640 1 --2- 172.21.15.120:0/340530830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614983fa9a0 0x5614983fada0 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009fd0 tx=0x7fe2b00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7933e2de9655362 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.639+0000 7fe29dffb640 1 -- 172.21.15.120:0/340530830 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0004d20 con 0x5614983fa9a0 2024-09-19T09:25:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.640+0000 7fe29dffb640 1 -- 172.21.15.120:0/340530830 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2b0004050 con 0x5614983fa9a0 2024-09-19T09:25:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.640+0000 7fe29dffb640 1 -- 172.21.15.120:0/340530830 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0005240 con 0x5614983fa9a0 2024-09-19T09:25:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.640+0000 7fe2bc931640 1 --2- 172.21.15.120:0/1705591700 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe2a00a32f0 0x7fe2a00a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.640+0000 7fe2bc931640 1 -- 172.21.15.120:0/1705591700 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe2a80efd40 con 0x7fe2b80ffe10 2024-09-19T09:25:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.640+0000 7fe2b6ffd640 1 --2- 172.21.15.120:0/1705591700 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe2a00a32f0 0x7fe2a00a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.640+0000 7fe2c06c8740 1 -- 172.21.15.120:0/340530830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614983fa9a0 msgr2=0x5614983fada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.640+0000 7fe2c06c8740 1 --2- 172.21.15.120:0/340530830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614983fa9a0 0x5614983fada0 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009fd0 tx=0x7fe2b00118d0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.641+0000 7fe2b6ffd640 1 --2- 172.21.15.120:0/1705591700 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe2a00a32f0 0x7fe2a00a57b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80bc360 tx=0x7fe2a80b8040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.641+0000 7fe2c06c8740 1 -- 172.21.15.120:0/340530830 shutdown_connections 2024-09-19T09:25:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.641+0000 7fe2c06c8740 1 --2- 172.21.15.120:0/340530830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561498400370 0x561498402830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.641+0000 7fe2c06c8740 1 --2- 172.21.15.120:0/340530830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614983fb2e0 0x5614983ffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.641+0000 7fe2c06c8740 1 --2- 172.21.15.120:0/340530830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614983fa9a0 0x5614983fada0 unknown :-1 s=CLOSED pgs=2356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.641+0000 7fe2c06c8740 1 -- 172.21.15.120:0/340530830 >> 172.21.15.120:0/340530830 conn(0x5614983f6010 msgr2=0x5614983f6410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.641+0000 7fe2c06c8740 1 -- 172.21.15.120:0/340530830 shutdown_connections 2024-09-19T09:25:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.641+0000 7fe2c06c8740 1 -- 172.21.15.120:0/340530830 wait complete. 2024-09-19T09:25:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.641+0000 7fe2c06c8740 1 -- start start 2024-09-19T09:25:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.641+0000 7fe2c06c8740 1 Processor -- start 2024-09-19T09:25:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.642+0000 7fe2c06c8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614983fa9a0 0x561498404b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.642+0000 7fe2c06c8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614983fb2e0 0x5614984050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.642+0000 7fe2c06c8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561498400370 0x56149840c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.642+0000 7fe2c06c8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561498403340 con 0x5614983fb2e0 2024-09-19T09:25:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.642+0000 7fe2c06c8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5614982e7290 con 0x5614983fa9a0 2024-09-19T09:25:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.642+0000 7fe2c06c8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5614984034c0 con 0x561498400370 2024-09-19T09:25:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.642+0000 7fe2b77fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614983fa9a0 0x561498404b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.642+0000 7fe2b77fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614983fa9a0 0x561498404b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:49414/0 (socket says 172.21.15.120:49414) 2024-09-19T09:25:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.642+0000 7fe2b77fe640 1 -- 172.21.15.120:0/3691619295 learned_addr learned my addr 172.21.15.120:0/3691619295 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.642+0000 7fe2c06c8740 10 client.0 _ll_register_callbacks cb 0x561498318140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:25:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.642+0000 7fe2b77fe640 1 -- 172.21.15.120:0/3691619295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561498400370 msgr2=0x56149840c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.642+0000 7fe2b77fe640 1 --2- 172.21.15.120:0/3691619295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561498400370 0x56149840c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.642+0000 7fe2b77fe640 1 -- 172.21.15.120:0/3691619295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614983fb2e0 msgr2=0x5614984050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.642+0000 7fe2b77fe640 1 --2- 172.21.15.120:0/3691619295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614983fb2e0 0x5614984050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.642+0000 7fe2b77fe640 1 -- 172.21.15.120:0/3691619295 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561498071210 con 0x5614983fa9a0 2024-09-19T09:25:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.643+0000 7fe2b77fe640 1 --2- 172.21.15.120:0/3691619295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614983fa9a0 0x561498404b90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0004920 tx=0x7fe2b00044f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b6aeae5231cbb36 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.643+0000 7fe296ffd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:25:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.643+0000 7fe296ffd640 1 -- 172.21.15.120:0/3691619295 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0028020 con 0x5614983fa9a0 2024-09-19T09:25:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.643+0000 7fe296ffd640 1 -- 172.21.15.120:0/3691619295 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe2b0005040 con 0x5614983fa9a0 2024-09-19T09:25:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.643+0000 7fe2c06c8740 1 -- 172.21.15.120:0/3691619295 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2b0004050 con 0x5614983fa9a0 2024-09-19T09:25:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.643+0000 7fe296ffd640 1 -- 172.21.15.120:0/3691619295 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe2b0005240 con 0x5614983fa9a0 2024-09-19T09:25:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.643+0000 7fe2c06c8740 10 client.31595 fetch_fsmap learned FSMap version 39 2024-09-19T09:25:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.643+0000 7fe2c06c8740 1 -- 172.21.15.120:0/3691619295 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56149813d620 con 0x5614983fa9a0 2024-09-19T09:25:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.644+0000 7fe296ffd640 1 -- 172.21.15.120:0/3691619295 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2b00058c0 con 0x5614983fa9a0 2024-09-19T09:25:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.644+0000 7fe296ffd640 1 -- 172.21.15.120:0/3691619295 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fe2b000fbc0 con 0x5614983fa9a0 2024-09-19T09:25:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.644+0000 7fe2c06c8740 10 client.31595 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:25:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.644+0000 7fe2c06c8740 10 client.31595 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:25:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.644+0000 7fe2c06c8740 20 client.31595 populate_metadata read hostname 'smithi120' 2024-09-19T09:25:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.644+0000 7fe2c06c8740 1 --2- 172.21.15.120:0/3691619295 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5614984105d0 0x5614984129c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.644+0000 7fe2c06c8740 4 client.31595 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:25:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.644+0000 7fe2c06c8740 1 -- 172.21.15.120:0/3691619295 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe2b0004050 con 0x5614984105d0 2024-09-19T09:25:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.645+0000 7fe2b67fc640 1 --2- 172.21.15.120:0/3691619295 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5614984105d0 0x5614984129c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.645+0000 7fe296ffd640 10 client.31595 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.645+0000 7fe2b67fc640 1 --2- 172.21.15.120:0/3691619295 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5614984105d0 0x5614984129c0 crc :-1 s=READY pgs=565 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66ef3cbdad3e391b server_cookie=a09f8e1a13d91377 in_seq=0 out_seq=0 2024-09-19T09:25:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.650+0000 7fe296ffd640 1 -- 172.21.15.120:0/3691619295 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fe2b0004050 con 0x5614984105d0 2024-09-19T09:25:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.650+0000 7fe296ffd640 10 client.31595 handle_command_reply: tid=0 2024-09-19T09:25:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.650+0000 7fe2c06c8740 1 client.31595 shutdown 2024-09-19T09:25:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.651+0000 7fe2c06c8740 1 -- 172.21.15.120:0/3691619295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614983fa9a0 msgr2=0x561498404b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.651+0000 7fe2c06c8740 1 --2- 172.21.15.120:0/3691619295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614983fa9a0 0x561498404b90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0004920 tx=0x7fe2b00044f0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.651+0000 7fe2c06c8740 1 -- 172.21.15.120:0/3691619295 shutdown_connections 2024-09-19T09:25:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.651+0000 7fe2c06c8740 1 --2- 172.21.15.120:0/3691619295 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5614984105d0 0x5614984129c0 crc :-1 s=READY pgs=565 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.651+0000 7fe2c06c8740 1 --2- 172.21.15.120:0/3691619295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561498400370 0x56149840c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.651+0000 7fe296ffd640 0 client.31595 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.651+0000 7fe2c06c8740 1 --2- 172.21.15.120:0/3691619295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614983fb2e0 0x5614984050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.651+0000 7fe2c06c8740 1 --2- 172.21.15.120:0/3691619295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614983fa9a0 0x561498404b90 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.651+0000 7fe2c06c8740 1 -- 172.21.15.120:0/3691619295 >> 172.21.15.120:0/3691619295 conn(0x5614983f6010 msgr2=0x5614983f7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.651+0000 7fe2c06c8740 1 -- 172.21.15.120:0/3691619295 shutdown_connections 2024-09-19T09:25:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.651+0000 7fe2c06c8740 1 -- 172.21.15.120:0/3691619295 wait complete. 2024-09-19T09:25:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.651+0000 7fe2c06c8740 20 client.31595 trim_cache size 0 max 16384 2024-09-19T09:25:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.671+0000 7fe2c06c8740 1 Processor -- start 2024-09-19T09:25:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.672+0000 7fe2c06c8740 1 -- start start 2024-09-19T09:25:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.672+0000 7fe2c06c8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614983fa9a0 0x5614983f59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.672+0000 7fe2c06c8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614983fb2e0 0x56149840c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.672+0000 7fe2c06c8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561498400370 0x561498404fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.672+0000 7fe2b67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614983fb2e0 0x56149840c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.672+0000 7fe2b67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614983fb2e0 0x56149840c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57522/0 (socket says 172.21.15.120:57522) 2024-09-19T09:25:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.672+0000 7fe2c06c8740 1 -- 172.21.15.120:0/510236969 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b81ab250 con 0x5614983fb2e0 2024-09-19T09:25:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.672+0000 7fe2b67fc640 1 -- 172.21.15.120:0/510236969 learned_addr learned my addr 172.21.15.120:0/510236969 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.672+0000 7fe2c06c8740 1 -- 172.21.15.120:0/510236969 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b81acd90 con 0x5614983fa9a0 2024-09-19T09:25:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.672+0000 7fe2b77fe640 1 --2- 172.21.15.120:0/510236969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614983fa9a0 0x5614983f59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.672+0000 7fe2c06c8740 1 -- 172.21.15.120:0/510236969 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b81ab920 con 0x561498400370 2024-09-19T09:25:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.672+0000 7fe2b67fc640 1 -- 172.21.15.120:0/510236969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561498400370 msgr2=0x561498404fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.672+0000 7fe2b67fc640 1 --2- 172.21.15.120:0/510236969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561498400370 0x561498404fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.672+0000 7fe2b67fc640 1 -- 172.21.15.120:0/510236969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614983fa9a0 msgr2=0x5614983f59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.672+0000 7fe2b67fc640 1 --2- 172.21.15.120:0/510236969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614983fa9a0 0x5614983f59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.672+0000 7fe2b67fc640 1 -- 172.21.15.120:0/510236969 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561498402760 con 0x5614983fb2e0 2024-09-19T09:25:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.673+0000 7fe2b67fc640 1 --2- 172.21.15.120:0/510236969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614983fb2e0 0x56149840c0d0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x561498413230 tx=0x7fe2ac0123e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7acce6be597a867 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.673+0000 7fe29d7fa640 1 -- 172.21.15.120:0/510236969 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac0042f0 con 0x5614983fb2e0 2024-09-19T09:25:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.673+0000 7fe29d7fa640 1 -- 172.21.15.120:0/510236969 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2ac004490 con 0x5614983fb2e0 2024-09-19T09:25:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.673+0000 7fe29d7fa640 1 -- 172.21.15.120:0/510236969 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac003bc0 con 0x5614983fb2e0 2024-09-19T09:25:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.674+0000 7fe2c06c8740 1 -- 172.21.15.120:0/510236969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614983fb2e0 msgr2=0x56149840c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.674+0000 7fe2c06c8740 1 --2- 172.21.15.120:0/510236969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614983fb2e0 0x56149840c0d0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x561498413230 tx=0x7fe2ac0123e0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.674+0000 7fe2c06c8740 1 -- 172.21.15.120:0/510236969 shutdown_connections 2024-09-19T09:25:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.674+0000 7fe2c06c8740 1 --2- 172.21.15.120:0/510236969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561498400370 0x561498404fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.674+0000 7fe2c06c8740 1 --2- 172.21.15.120:0/510236969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614983fb2e0 0x56149840c0d0 unknown :-1 s=CLOSED pgs=2357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.674+0000 7fe2c06c8740 1 --2- 172.21.15.120:0/510236969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614983fa9a0 0x5614983f59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.674+0000 7fe2c06c8740 1 -- 172.21.15.120:0/510236969 >> 172.21.15.120:0/510236969 conn(0x5614983f6010 msgr2=0x5614983f76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.674+0000 7fe2c06c8740 1 -- 172.21.15.120:0/510236969 shutdown_connections 2024-09-19T09:25:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.674+0000 7fe2c06c8740 1 -- 172.21.15.120:0/510236969 wait complete. 2024-09-19T09:25:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.674+0000 7fe2c06c8740 1 -- start start 2024-09-19T09:25:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.674+0000 7fe2c06c8740 1 Processor -- start 2024-09-19T09:25:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.675+0000 7fe2c06c8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614983fa9a0 0x56149840a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.675+0000 7fe2c06c8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614983fb2e0 0x56149840a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.675+0000 7fe2c06c8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561498400370 0x56149840b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.675+0000 7fe2c06c8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b81ac6c0 con 0x561498400370 2024-09-19T09:25:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.675+0000 7fe2c06c8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b81abff0 con 0x5614983fa9a0 2024-09-19T09:25:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.675+0000 7fe2c06c8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b81a9210 con 0x5614983fb2e0 2024-09-19T09:25:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.675+0000 7fe2c06c8740 10 client.0 _ll_register_callbacks cb 0x7fe2ac009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:25:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.675+0000 7fe2b77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561498400370 0x56149840b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.675+0000 7fe2b77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561498400370 0x56149840b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57528/0 (socket says 172.21.15.120:57528) 2024-09-19T09:25:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.675+0000 7fe2b77fe640 1 -- 172.21.15.120:0/3294654373 learned_addr learned my addr 172.21.15.120:0/3294654373 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:25:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.675+0000 7fe2b67fc640 1 --2- 172.21.15.120:0/3294654373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614983fb2e0 0x56149840a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.675+0000 7fe2b77fe640 1 --2- 172.21.15.120:0/3294654373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614983fa9a0 0x56149840a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.675+0000 7fe2b77fe640 1 -- 172.21.15.120:0/3294654373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614983fb2e0 msgr2=0x56149840a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.675+0000 7fe2b77fe640 1 --2- 172.21.15.120:0/3294654373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614983fb2e0 0x56149840a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.675+0000 7fe2b77fe640 1 -- 172.21.15.120:0/3294654373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614983fa9a0 msgr2=0x56149840a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:25:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.675+0000 7fe2b77fe640 1 --2- 172.21.15.120:0/3294654373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614983fa9a0 0x56149840a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.675+0000 7fe2b77fe640 1 -- 172.21.15.120:0/3294654373 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56149840bc20 con 0x561498400370 2024-09-19T09:25:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.675+0000 7fe2b77fe640 1 --2- 172.21.15.120:0/3294654373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561498400370 0x56149840b610 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0005fa0 tx=0x7fe2b0005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56cea592d6b5841 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.675+0000 7fe296ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:25:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.675+0000 7fe296ffd640 1 -- 172.21.15.120:0/3294654373 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0028040 con 0x561498400370 2024-09-19T09:25:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.675+0000 7fe296ffd640 1 -- 172.21.15.120:0/3294654373 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe2b0004ee0 con 0x561498400370 2024-09-19T09:25:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.675+0000 7fe2c06c8740 1 -- 172.21.15.120:0/3294654373 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2ac004490 con 0x561498400370 2024-09-19T09:25:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.676+0000 7fe296ffd640 1 -- 172.21.15.120:0/3294654373 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe2b002fa40 con 0x561498400370 2024-09-19T09:25:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.676+0000 7fe2c06c8740 10 client.22929 fetch_fsmap learned FSMap version 39 2024-09-19T09:25:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.676+0000 7fe2c06c8740 1 -- 172.21.15.120:0/3294654373 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56149847bb70 con 0x561498400370 2024-09-19T09:25:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.676+0000 7fe296ffd640 1 -- 172.21.15.120:0/3294654373 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2ac004490 con 0x561498400370 2024-09-19T09:25:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.677+0000 7fe296ffd640 1 -- 172.21.15.120:0/3294654373 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fe2b0039040 con 0x561498400370 2024-09-19T09:25:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.677+0000 7fe2c06c8740 10 client.22929 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:25:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.677+0000 7fe2c06c8740 10 client.22929 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:25:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.677+0000 7fe2c06c8740 20 client.22929 populate_metadata read hostname 'smithi120' 2024-09-19T09:25:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.677+0000 7fe2c06c8740 1 --2- 172.21.15.120:0/3294654373 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56149847c400 0x56149847e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.677+0000 7fe2c06c8740 4 client.22929 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:25:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.677+0000 7fe2c06c8740 1 -- 172.21.15.120:0/3294654373 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56149836ba20 con 0x56149847c400 2024-09-19T09:25:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.677+0000 7fe2b67fc640 1 --2- 172.21.15.120:0/3294654373 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56149847c400 0x56149847e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.677+0000 7fe2b67fc640 1 --2- 172.21.15.120:0/3294654373 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56149847c400 0x56149847e820 crc :-1 s=READY pgs=566 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22236f598039a2b0 server_cookie=ab8bc7041d41504f in_seq=0 out_seq=0 2024-09-19T09:25:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:47.677+0000 7fe296ffd640 10 client.22929 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:48.384 INFO:tasks.workunit.client.0.smithi120.stdout: 10 229581 20.26 MB/sec execute 463 sec latency 382.298 ms 2024-09-19T09:25:48.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:48 smithi196 ceph-mon[35150]: pgmap v1033: 97 pgs: 97 active+clean; 500 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.8 MiB/s wr, 797 op/s 2024-09-19T09:25:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:48.437+0000 7fe296ffd640 1 -- 172.21.15.120:0/3294654373 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56149836ba20 con 0x56149847c400 2024-09-19T09:25:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:48.437+0000 7fe296ffd640 10 client.22929 handle_command_reply: tid=0 2024-09-19T09:25:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:48.437+0000 7fe2c06c8740 1 client.22929 shutdown 2024-09-19T09:25:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:48.437+0000 7fe2c06c8740 1 -- 172.21.15.120:0/3294654373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561498400370 msgr2=0x56149840b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:48.437+0000 7fe2c06c8740 1 --2- 172.21.15.120:0/3294654373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561498400370 0x56149840b610 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0005fa0 tx=0x7fe2b0005f10 comp rx=0 tx=0).stop 2024-09-19T09:25:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:48.437+0000 7fe2c06c8740 1 -- 172.21.15.120:0/3294654373 shutdown_connections 2024-09-19T09:25:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:48.437+0000 7fe2c06c8740 1 --2- 172.21.15.120:0/3294654373 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56149847c400 0x56149847e820 crc :-1 s=READY pgs=566 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:48.437+0000 7fe2c06c8740 1 --2- 172.21.15.120:0/3294654373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561498400370 0x56149840b610 unknown :-1 s=CLOSED pgs=2358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:48.437+0000 7fe2c06c8740 1 --2- 172.21.15.120:0/3294654373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5614983fb2e0 0x56149840a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:48.437+0000 7fe2c06c8740 1 --2- 172.21.15.120:0/3294654373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614983fa9a0 0x56149840a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:48.437+0000 7fe296ffd640 0 client.22929 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:25:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:48.437+0000 7fe2c06c8740 1 -- 172.21.15.120:0/3294654373 >> 172.21.15.120:0/3294654373 conn(0x5614983f6010 msgr2=0x5614983f49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:48.437+0000 7fe2c06c8740 1 -- 172.21.15.120:0/3294654373 shutdown_connections 2024-09-19T09:25:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:48.437+0000 7fe2c06c8740 1 -- 172.21.15.120:0/3294654373 wait complete. 2024-09-19T09:25:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:48.437+0000 7fe2c06c8740 20 client.22929 trim_cache size 0 max 16384 2024-09-19T09:25:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:25:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:25:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:25:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:25:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:48.438+0000 7fe2bd933640 1 -- 172.21.15.120:0/1705591700 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe2a00a32f0 msgr2=0x7fe2a00a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:48.438+0000 7fe2bd933640 1 --2- 172.21.15.120:0/1705591700 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe2a00a32f0 0x7fe2a00a57b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80bc360 tx=0x7fe2a80b8040 comp rx=0 tx=0).stop 2024-09-19T09:25:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:48.438+0000 7fe2bd933640 1 -- 172.21.15.120:0/1705591700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2b80ffe10 msgr2=0x7fe2b8102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:48.438+0000 7fe2bd933640 1 --2- 172.21.15.120:0/1705591700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2b80ffe10 0x7fe2b8102750 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8031440 tx=0x7fe2a8032ce0 comp rx=0 tx=0).stop 2024-09-19T09:25:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:48.438+0000 7fe2bd933640 1 -- 172.21.15.120:0/1705591700 shutdown_connections 2024-09-19T09:25:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:48.438+0000 7fe2bd933640 1 --2- 172.21.15.120:0/1705591700 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe2a00a32f0 0x7fe2a00a57b0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:48.438+0000 7fe2bd933640 1 --2- 172.21.15.120:0/1705591700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2b81037a0 0x7fe2b81012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:48.438+0000 7fe2bd933640 1 --2- 172.21.15.120:0/1705591700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2b8102de0 0x7fe2b8100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:48.438+0000 7fe2bd933640 1 --2- 172.21.15.120:0/1705591700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2b80ffe10 0x7fe2b8102750 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:48.438+0000 7fe2bd933640 1 -- 172.21.15.120:0/1705591700 >> 172.21.15.120:0/1705591700 conn(0x7fe2b80fb930 msgr2=0x7fe2b81060e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:48.438+0000 7fe2bd933640 1 -- 172.21.15.120:0/1705591700 shutdown_connections 2024-09-19T09:25:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:25:48.438+0000 7fe2bd933640 1 -- 172.21.15.120:0/1705591700 wait complete. 2024-09-19T09:25:48.442 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:48 smithi153 ceph-mon[35071]: pgmap v1033: 97 pgs: 97 active+clean; 500 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.8 MiB/s wr, 797 op/s 2024-09-19T09:25:48.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:48 smithi120 ceph-mon[31150]: pgmap v1033: 97 pgs: 97 active+clean; 500 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.8 MiB/s wr, 797 op/s 2024-09-19T09:25:49.384 INFO:tasks.workunit.client.0.smithi120.stdout: 10 230137 20.27 MB/sec execute 464 sec latency 140.437 ms 2024-09-19T09:25:50.377 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:50 smithi120 ceph-mon[31150]: pgmap v1034: 97 pgs: 97 active+clean; 496 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.5 MiB/s wr, 691 op/s 2024-09-19T09:25:50.384 INFO:tasks.workunit.client.0.smithi120.stdout: 10 230792 20.31 MB/sec execute 465 sec latency 60.340 ms 2024-09-19T09:25:50.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:50 smithi196 ceph-mon[35150]: pgmap v1034: 97 pgs: 97 active+clean; 496 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.5 MiB/s wr, 691 op/s 2024-09-19T09:25:50.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:50 smithi153 ceph-mon[35071]: pgmap v1034: 97 pgs: 97 active+clean; 496 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.5 MiB/s wr, 691 op/s 2024-09-19T09:25:51.384 INFO:tasks.workunit.client.0.smithi120.stdout: 10 231516 20.34 MB/sec execute 466 sec latency 75.510 ms 2024-09-19T09:25:52.384 INFO:tasks.workunit.client.0.smithi120.stdout: 10 232106 20.39 MB/sec execute 467 sec latency 134.057 ms 2024-09-19T09:25:52.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:52 smithi196 ceph-mon[35150]: pgmap v1035: 97 pgs: 97 active+clean; 493 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.9 MiB/s wr, 744 op/s 2024-09-19T09:25:52.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:52 smithi153 ceph-mon[35071]: pgmap v1035: 97 pgs: 97 active+clean; 493 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.9 MiB/s wr, 744 op/s 2024-09-19T09:25:52.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:52 smithi120 ceph-mon[31150]: pgmap v1035: 97 pgs: 97 active+clean; 493 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.9 MiB/s wr, 744 op/s 2024-09-19T09:25:53.384 INFO:tasks.workunit.client.0.smithi120.stdout: 10 232571 20.39 MB/sec execute 468 sec latency 185.316 ms 2024-09-19T09:25:53.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:53 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:25:53.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:53 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:25:53.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:53 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:25:53.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:53 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:25:53.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:53 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:25:53.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:53 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:25:54.384 INFO:tasks.workunit.client.0.smithi120.stdout: 10 233158 20.44 MB/sec execute 469 sec latency 58.647 ms 2024-09-19T09:25:54.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:54 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:25:54.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:54 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:25:54.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:54 smithi120 ceph-mon[31150]: pgmap v1036: 97 pgs: 97 active+clean; 498 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.5 MiB/s wr, 813 op/s 2024-09-19T09:25:54.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:54 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:25:54.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:54 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:25:54.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:54 smithi196 ceph-mon[35150]: pgmap v1036: 97 pgs: 97 active+clean; 498 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.5 MiB/s wr, 813 op/s 2024-09-19T09:25:54.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:54 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:25:54.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:54 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:25:54.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:54 smithi153 ceph-mon[35071]: pgmap v1036: 97 pgs: 97 active+clean; 498 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.5 MiB/s wr, 813 op/s 2024-09-19T09:25:55.385 INFO:tasks.workunit.client.0.smithi120.stdout: 10 233687 20.44 MB/sec execute 470 sec latency 77.695 ms 2024-09-19T09:25:56.384 INFO:tasks.workunit.client.0.smithi120.stdout: 10 234149 20.44 MB/sec execute 471 sec latency 191.051 ms 2024-09-19T09:25:56.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:56 smithi196 ceph-mon[35150]: pgmap v1037: 97 pgs: 97 active+clean; 501 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.6 MiB/s wr, 752 op/s 2024-09-19T09:25:56.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:56 smithi153 ceph-mon[35071]: pgmap v1037: 97 pgs: 97 active+clean; 501 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.6 MiB/s wr, 752 op/s 2024-09-19T09:25:56.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:56 smithi120 ceph-mon[31150]: pgmap v1037: 97 pgs: 97 active+clean; 501 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.6 MiB/s wr, 752 op/s 2024-09-19T09:25:57.385 INFO:tasks.workunit.client.0.smithi120.stdout: 10 234676 20.46 MB/sec execute 472 sec latency 88.214 ms 2024-09-19T09:25:58.385 INFO:tasks.workunit.client.0.smithi120.stdout: 10 235143 20.46 MB/sec execute 473 sec latency 294.467 ms 2024-09-19T09:25:58.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:25:58 smithi196 ceph-mon[35150]: pgmap v1038: 97 pgs: 97 active+clean; 502 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.3 MiB/s wr, 770 op/s 2024-09-19T09:25:58.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:25:58 smithi153 ceph-mon[35071]: pgmap v1038: 97 pgs: 97 active+clean; 502 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.3 MiB/s wr, 770 op/s 2024-09-19T09:25:58.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:25:58 smithi120 ceph-mon[31150]: pgmap v1038: 97 pgs: 97 active+clean; 502 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.3 MiB/s wr, 770 op/s 2024-09-19T09:25:59.384 INFO:tasks.workunit.client.0.smithi120.stdout: 10 235763 20.50 MB/sec execute 474 sec latency 72.296 ms 2024-09-19T09:26:00.384 INFO:tasks.workunit.client.0.smithi120.stdout: 10 236477 20.53 MB/sec execute 475 sec latency 48.555 ms 2024-09-19T09:26:00.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:00 smithi196 ceph-mon[35150]: pgmap v1039: 97 pgs: 97 active+clean; 513 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.8 MiB/s wr, 719 op/s 2024-09-19T09:26:00.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:00 smithi153 ceph-mon[35071]: pgmap v1039: 97 pgs: 97 active+clean; 513 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.8 MiB/s wr, 719 op/s 2024-09-19T09:26:00.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:00 smithi120 ceph-mon[31150]: pgmap v1039: 97 pgs: 97 active+clean; 513 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.8 MiB/s wr, 719 op/s 2024-09-19T09:26:01.385 INFO:tasks.workunit.client.0.smithi120.stdout: 10 237013 20.55 MB/sec execute 476 sec latency 84.544 ms 2024-09-19T09:26:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.536+0000 7f75993f6640 1 -- 172.21.15.120:0/3917032002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f75941025f0 msgr2=0x7f7594108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.536+0000 7f75993f6640 1 --2- 172.21.15.120:0/3917032002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f75941025f0 0x7f7594108e80 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f758c009ca0 tx=0x7f758c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:26:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.536+0000 7f75993f6640 1 -- 172.21.15.120:0/3917032002 shutdown_connections 2024-09-19T09:26:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.536+0000 7f75993f6640 1 --2- 172.21.15.120:0/3917032002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f75941025f0 0x7f7594108e80 unknown :-1 s=CLOSED pgs=2359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.536+0000 7f75993f6640 1 --2- 172.21.15.120:0/3917032002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7594101c30 0x7f75941020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.536+0000 7f75993f6640 1 --2- 172.21.15.120:0/3917032002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7594100a40 0x7f7594100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.536+0000 7f75993f6640 1 -- 172.21.15.120:0/3917032002 >> 172.21.15.120:0/3917032002 conn(0x7f75940fc1e0 msgr2=0x7f75940fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.536+0000 7f75993f6640 1 -- 172.21.15.120:0/3917032002 shutdown_connections 2024-09-19T09:26:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.536+0000 7f75993f6640 1 -- 172.21.15.120:0/3917032002 wait complete. 2024-09-19T09:26:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.536+0000 7f75993f6640 1 Processor -- start 2024-09-19T09:26:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.537+0000 7f75993f6640 1 -- start start 2024-09-19T09:26:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.537+0000 7f75993f6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7594100a40 0x7f75941049d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.537+0000 7f75993f6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7594101c30 0x7f7594103010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.537+0000 7f75993f6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f75941025f0 0x7f7594103550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.537+0000 7f75993f6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f759410c6f0 con 0x7f75941025f0 2024-09-19T09:26:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.537+0000 7f75993f6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f759410c570 con 0x7f7594101c30 2024-09-19T09:26:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.537+0000 7f75993f6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f759410c870 con 0x7f7594100a40 2024-09-19T09:26:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.537+0000 7f75937fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f75941025f0 0x7f7594103550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.537+0000 7f75937fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f75941025f0 0x7f7594103550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50518/0 (socket says 172.21.15.120:50518) 2024-09-19T09:26:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.537+0000 7f75937fe640 1 -- 172.21.15.120:0/1353406007 learned_addr learned my addr 172.21.15.120:0/1353406007 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.537+0000 7f7592ffd640 1 --2- 172.21.15.120:0/1353406007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7594100a40 0x7f75941049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.537+0000 7f75927fc640 1 --2- 172.21.15.120:0/1353406007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7594101c30 0x7f7594103010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.537+0000 7f75937fe640 1 -- 172.21.15.120:0/1353406007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7594100a40 msgr2=0x7f75941049d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.537+0000 7f75937fe640 1 --2- 172.21.15.120:0/1353406007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7594100a40 0x7f75941049d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.537+0000 7f75937fe640 1 -- 172.21.15.120:0/1353406007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7594101c30 msgr2=0x7f7594103010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.537+0000 7f75937fe640 1 --2- 172.21.15.120:0/1353406007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7594101c30 0x7f7594103010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.537+0000 7f75937fe640 1 -- 172.21.15.120:0/1353406007 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7594103de0 con 0x7f75941025f0 2024-09-19T09:26:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.537+0000 7f75937fe640 1 --2- 172.21.15.120:0/1353406007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f75941025f0 0x7f7594103550 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f758c031750 tx=0x7f758c031780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.537+0000 7f757bfff640 1 -- 172.21.15.120:0/1353406007 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c003b70 con 0x7f75941025f0 2024-09-19T09:26:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.538+0000 7f757bfff640 1 -- 172.21.15.120:0/1353406007 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f758c003d10 con 0x7f75941025f0 2024-09-19T09:26:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.538+0000 7f757bfff640 1 -- 172.21.15.120:0/1353406007 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c02fc00 con 0x7f75941025f0 2024-09-19T09:26:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.538+0000 7f75993f6640 1 -- 172.21.15.120:0/1353406007 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75941a1f00 con 0x7f75941025f0 2024-09-19T09:26:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.538+0000 7f75993f6640 1 -- 172.21.15.120:0/1353406007 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75941a23c0 con 0x7f75941025f0 2024-09-19T09:26:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.540+0000 7f757bfff640 1 -- 172.21.15.120:0/1353406007 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f758c0046b0 con 0x7f75941025f0 2024-09-19T09:26:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.540+0000 7f759c26c740 1 Processor -- start 2024-09-19T09:26:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.540+0000 7f759c26c740 1 -- start start 2024-09-19T09:26:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.541+0000 7f759c26c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afe86c09a0 0x55afe86c0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.541+0000 7f759c26c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55afe86c12e0 0x55afe86c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.541+0000 7f759c26c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afe86c6370 0x55afe86c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.541+0000 7f759c26c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55afe86242f0 con 0x55afe86c6370 2024-09-19T09:26:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.541+0000 7f759c26c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55afe85cc270 con 0x55afe86c12e0 2024-09-19T09:26:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.541+0000 7f759c26c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55afe86202b0 con 0x55afe86c09a0 2024-09-19T09:26:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.541+0000 7f757bfff640 1 --2- 172.21.15.120:0/1353406007 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f757c0a3370 0x7f757c0a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.541+0000 7f75927fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afe86c6370 0x55afe86c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.541+0000 7f75927fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afe86c6370 0x55afe86c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50534/0 (socket says 172.21.15.120:50534) 2024-09-19T09:26:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.541+0000 7f75927fc640 1 -- 172.21.15.120:0/1178237669 learned_addr learned my addr 172.21.15.120:0/1178237669 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.541+0000 7f7592ffd640 1 --2- 172.21.15.120:0/1178237669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55afe86c12e0 0x55afe86c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.541+0000 7f75927fc640 1 --2- 172.21.15.120:0/1178237669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afe86c09a0 0x55afe86c0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.541+0000 7f757bfff640 1 -- 172.21.15.120:0/1353406007 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f758c0f0c50 con 0x7f75941025f0 2024-09-19T09:26:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.541+0000 7f75937fe640 1 --2- 172.21.15.120:0/1353406007 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f757c0a3370 0x7f757c0a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.541+0000 7f75927fc640 1 -- 172.21.15.120:0/1178237669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afe86c09a0 msgr2=0x55afe86c0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.541+0000 7f75927fc640 1 --2- 172.21.15.120:0/1178237669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afe86c09a0 0x55afe86c0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.541+0000 7f75927fc640 1 -- 172.21.15.120:0/1178237669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55afe86c12e0 msgr2=0x55afe86c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.541+0000 7f75927fc640 1 --2- 172.21.15.120:0/1178237669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55afe86c12e0 0x55afe86c5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.541+0000 7f75927fc640 1 -- 172.21.15.120:0/1178237669 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afe85cb340 con 0x55afe86c6370 2024-09-19T09:26:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.541+0000 7f7592ffd640 1 --2- 172.21.15.120:0/1178237669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55afe86c12e0 0x55afe86c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.541+0000 7f75937fe640 1 --2- 172.21.15.120:0/1353406007 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f757c0a3370 0x7f757c0a5830 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f758c0b9c40 tx=0x7f758c02f6a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.541+0000 7f75927fc640 1 --2- 172.21.15.120:0/1178237669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afe86c6370 0x55afe86c8830 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f75880058c0 tx=0x7f7588005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c61053d61ba21983 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.542+0000 7f75797fa640 1 -- 172.21.15.120:0/1178237669 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7588004030 con 0x55afe86c6370 2024-09-19T09:26:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.542+0000 7f75797fa640 1 -- 172.21.15.120:0/1178237669 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7588005500 con 0x55afe86c6370 2024-09-19T09:26:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.542+0000 7f75797fa640 1 -- 172.21.15.120:0/1178237669 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7588004c80 con 0x55afe86c6370 2024-09-19T09:26:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.542+0000 7f759c26c740 1 -- 172.21.15.120:0/1178237669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afe86c6370 msgr2=0x55afe86c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.542+0000 7f759c26c740 1 --2- 172.21.15.120:0/1178237669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afe86c6370 0x55afe86c8830 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f75880058c0 tx=0x7f7588005850 comp rx=0 tx=0).stop 2024-09-19T09:26:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.542+0000 7f759c26c740 1 -- 172.21.15.120:0/1178237669 shutdown_connections 2024-09-19T09:26:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.542+0000 7f759c26c740 1 --2- 172.21.15.120:0/1178237669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afe86c6370 0x55afe86c8830 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.542+0000 7f759c26c740 1 --2- 172.21.15.120:0/1178237669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55afe86c12e0 0x55afe86c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.542+0000 7f759c26c740 1 --2- 172.21.15.120:0/1178237669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afe86c09a0 0x55afe86c0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.542+0000 7f759c26c740 1 -- 172.21.15.120:0/1178237669 >> 172.21.15.120:0/1178237669 conn(0x55afe86bc010 msgr2=0x55afe86bc410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.542+0000 7f759c26c740 1 -- 172.21.15.120:0/1178237669 shutdown_connections 2024-09-19T09:26:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.542+0000 7f759c26c740 1 -- 172.21.15.120:0/1178237669 wait complete. 2024-09-19T09:26:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.543+0000 7f759c26c740 1 -- start start 2024-09-19T09:26:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.543+0000 7f759c26c740 1 Processor -- start 2024-09-19T09:26:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.543+0000 7f759c26c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55afe86c09a0 0x55afe86cac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.543+0000 7f759c26c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afe86c12e0 0x55afe86cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.543+0000 7f759c26c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afe86c6370 0x55afe86d2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.543+0000 7f759c26c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55afe86c9340 con 0x55afe86c6370 2024-09-19T09:26:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.543+0000 7f759c26c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55afe85ad290 con 0x55afe86c09a0 2024-09-19T09:26:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.543+0000 7f759c26c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55afe86c94c0 con 0x55afe86c12e0 2024-09-19T09:26:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.543+0000 7f759c26c740 10 client.0 _ll_register_callbacks cb 0x55afe85de140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.543+0000 7f75927fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55afe86c09a0 0x55afe86cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.543+0000 7f7592ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afe86c12e0 0x55afe86cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.543+0000 7f75927fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55afe86c09a0 0x55afe86cac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:57654/0 (socket says 172.21.15.120:57654) 2024-09-19T09:26:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.543+0000 7f75927fc640 1 -- 172.21.15.120:0/1469309881 learned_addr learned my addr 172.21.15.120:0/1469309881 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.543+0000 7f75927fc640 1 --2- 172.21.15.120:0/1469309881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afe86c6370 0x55afe86d2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.543+0000 7f7592ffd640 1 -- 172.21.15.120:0/1469309881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55afe86c09a0 msgr2=0x55afe86cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.543+0000 7f7592ffd640 1 --2- 172.21.15.120:0/1469309881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55afe86c09a0 0x55afe86cac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.543+0000 7f7592ffd640 1 -- 172.21.15.120:0/1469309881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afe86c6370 msgr2=0x55afe86d2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.543+0000 7f7592ffd640 1 --2- 172.21.15.120:0/1469309881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afe86c6370 0x55afe86d2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.544+0000 7f7592ffd640 1 -- 172.21.15.120:0/1469309881 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afe8337210 con 0x55afe86c12e0 2024-09-19T09:26:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.544+0000 7f75927fc640 1 --2- 172.21.15.120:0/1469309881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afe86c6370 0x55afe86d2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.544+0000 7f7592ffd640 1 --2- 172.21.15.120:0/1469309881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afe86c12e0 0x55afe86cb1d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x55afe86c1d10 tx=0x7f7584012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16aafd588d11ddfc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.544+0000 7f7576ff5640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:26:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.544+0000 7f7576ff5640 1 -- 172.21.15.120:0/1469309881 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7584004030 con 0x55afe86c12e0 2024-09-19T09:26:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.544+0000 7f759c26c740 1 -- 172.21.15.120:0/1469309881 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7588005500 con 0x55afe86c12e0 2024-09-19T09:26:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.544+0000 7f7576ff5640 1 -- 172.21.15.120:0/1469309881 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f75840041d0 con 0x55afe86c12e0 2024-09-19T09:26:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.544+0000 7f7576ff5640 1 -- 172.21.15.120:0/1469309881 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f75840043d0 con 0x55afe86c12e0 2024-09-19T09:26:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.544+0000 7f759c26c740 10 client.31477 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.544+0000 7f759c26c740 1 -- 172.21.15.120:0/1469309881 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55afe8403620 con 0x55afe86c12e0 2024-09-19T09:26:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.544+0000 7f7576ff5640 1 -- 172.21.15.120:0/1469309881 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7588005500 con 0x55afe86c12e0 2024-09-19T09:26:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.544+0000 7f7576ff5640 1 -- 172.21.15.120:0/1469309881 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f7584003a20 con 0x55afe86c12e0 2024-09-19T09:26:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.544+0000 7f759c26c740 10 client.31477 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.544+0000 7f759c26c740 10 client.31477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:26:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.545+0000 7f759c26c740 20 client.31477 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.545+0000 7f759c26c740 1 --2- 172.21.15.120:0/1469309881 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55afe86d69e0 0x55afe86d8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.545+0000 7f759c26c740 4 client.31477 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:26:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.545+0000 7f759c26c740 1 -- 172.21.15.120:0/1469309881 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afe8631a20 con 0x55afe86d69e0 2024-09-19T09:26:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.545+0000 7f75927fc640 1 --2- 172.21.15.120:0/1469309881 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55afe86d69e0 0x55afe86d8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.545+0000 7f7576ff5640 10 client.31477 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.545+0000 7f75927fc640 1 --2- 172.21.15.120:0/1469309881 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55afe86d69e0 0x55afe86d8e20 crc :-1 s=READY pgs=567 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7176bc388ed8db8 server_cookie=9b8c14872501b0c4 in_seq=0 out_seq=0 2024-09-19T09:26:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.549+0000 7f7576ff5640 1 -- 172.21.15.120:0/1469309881 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55afe8631a20 con 0x55afe86d69e0 2024-09-19T09:26:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.549+0000 7f7576ff5640 10 client.31477 handle_command_reply: tid=0 2024-09-19T09:26:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.549+0000 7f759c26c740 1 client.31477 shutdown 2024-09-19T09:26:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.549+0000 7f759c26c740 1 -- 172.21.15.120:0/1469309881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afe86c12e0 msgr2=0x55afe86cb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.549+0000 7f759c26c740 1 --2- 172.21.15.120:0/1469309881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afe86c12e0 0x55afe86cb1d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x55afe86c1d10 tx=0x7f7584012b60 comp rx=0 tx=0).stop 2024-09-19T09:26:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.549+0000 7f759c26c740 1 -- 172.21.15.120:0/1469309881 shutdown_connections 2024-09-19T09:26:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.549+0000 7f759c26c740 1 --2- 172.21.15.120:0/1469309881 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55afe86d69e0 0x55afe86d8e20 crc :-1 s=READY pgs=567 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.549+0000 7f759c26c740 1 --2- 172.21.15.120:0/1469309881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afe86c6370 0x55afe86d2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.549+0000 7f7576ff5640 0 client.31477 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.549+0000 7f759c26c740 1 --2- 172.21.15.120:0/1469309881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afe86c12e0 0x55afe86cb1d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.549+0000 7f759c26c740 1 --2- 172.21.15.120:0/1469309881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55afe86c09a0 0x55afe86cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.549+0000 7f759c26c740 1 -- 172.21.15.120:0/1469309881 >> 172.21.15.120:0/1469309881 conn(0x55afe86bc010 msgr2=0x55afe86bd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.549+0000 7f759c26c740 1 -- 172.21.15.120:0/1469309881 shutdown_connections 2024-09-19T09:26:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.549+0000 7f759c26c740 1 -- 172.21.15.120:0/1469309881 wait complete. 2024-09-19T09:26:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.549+0000 7f759c26c740 20 client.31477 trim_cache size 0 max 16384 2024-09-19T09:26:01.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.563+0000 7f759c26c740 1 Processor -- start 2024-09-19T09:26:01.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.563+0000 7f759c26c740 1 -- start start 2024-09-19T09:26:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.563+0000 7f759c26c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55afe86c09a0 0x55afe86bd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.563+0000 7f759c26c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afe86c12e0 0x55afe86d21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.563+0000 7f759c26c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afe86c6370 0x55afe86bd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.563+0000 7f759c26c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f75941ab2a0 con 0x55afe86c6370 2024-09-19T09:26:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.563+0000 7f759c26c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f75941acde0 con 0x55afe86c09a0 2024-09-19T09:26:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.563+0000 7f759c26c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f75941ab970 con 0x55afe86c12e0 2024-09-19T09:26:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.564+0000 7f75927fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afe86c6370 0x55afe86bd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.564+0000 7f75927fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afe86c6370 0x55afe86bd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50546/0 (socket says 172.21.15.120:50546) 2024-09-19T09:26:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.564+0000 7f75927fc640 1 -- 172.21.15.120:0/985429620 learned_addr learned my addr 172.21.15.120:0/985429620 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.564+0000 7f7592ffd640 1 --2- 172.21.15.120:0/985429620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afe86c12e0 0x55afe86d21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.564+0000 7f75927fc640 1 --2- 172.21.15.120:0/985429620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55afe86c09a0 0x55afe86bd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.564+0000 7f75927fc640 1 -- 172.21.15.120:0/985429620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afe86c12e0 msgr2=0x55afe86d21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.564+0000 7f75927fc640 1 --2- 172.21.15.120:0/985429620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afe86c12e0 0x55afe86d21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.564+0000 7f75927fc640 1 -- 172.21.15.120:0/985429620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55afe86c09a0 msgr2=0x55afe86bd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.564+0000 7f75927fc640 1 --2- 172.21.15.120:0/985429620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55afe86c09a0 0x55afe86bd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.564+0000 7f75927fc640 1 -- 172.21.15.120:0/985429620 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afe86bbe30 con 0x55afe86c6370 2024-09-19T09:26:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.564+0000 7f75927fc640 1 --2- 172.21.15.120:0/985429620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afe86c6370 0x55afe86bd940 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f75880117f0 tx=0x7f7588009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a9b3c1f80ca437 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.564+0000 7f7578ff9640 1 -- 172.21.15.120:0/985429620 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758800f840 con 0x55afe86c6370 2024-09-19T09:26:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.564+0000 7f7578ff9640 1 -- 172.21.15.120:0/985429620 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7588035070 con 0x55afe86c6370 2024-09-19T09:26:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.564+0000 7f7578ff9640 1 -- 172.21.15.120:0/985429620 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7588004d20 con 0x55afe86c6370 2024-09-19T09:26:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.565+0000 7f759c26c740 1 -- 172.21.15.120:0/985429620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afe86c6370 msgr2=0x55afe86bd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.565+0000 7f759c26c740 1 --2- 172.21.15.120:0/985429620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afe86c6370 0x55afe86bd940 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f75880117f0 tx=0x7f7588009b60 comp rx=0 tx=0).stop 2024-09-19T09:26:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.565+0000 7f759c26c740 1 -- 172.21.15.120:0/985429620 shutdown_connections 2024-09-19T09:26:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.565+0000 7f759c26c740 1 --2- 172.21.15.120:0/985429620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afe86c6370 0x55afe86bd940 unknown :-1 s=CLOSED pgs=2362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.565+0000 7f759c26c740 1 --2- 172.21.15.120:0/985429620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afe86c12e0 0x55afe86d21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.565+0000 7f759c26c740 1 --2- 172.21.15.120:0/985429620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55afe86c09a0 0x55afe86bd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.565+0000 7f759c26c740 1 -- 172.21.15.120:0/985429620 >> 172.21.15.120:0/985429620 conn(0x55afe86bc010 msgr2=0x55afe86bb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.565+0000 7f759c26c740 1 -- 172.21.15.120:0/985429620 shutdown_connections 2024-09-19T09:26:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.565+0000 7f759c26c740 1 -- 172.21.15.120:0/985429620 wait complete. 2024-09-19T09:26:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.565+0000 7f759c26c740 1 -- start start 2024-09-19T09:26:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.565+0000 7f759c26c740 1 Processor -- start 2024-09-19T09:26:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.566+0000 7f759c26c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afe86c09a0 0x55afe86cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.566+0000 7f759c26c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afe86c12e0 0x55afe86cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:01.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.566+0000 7f759c26c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55afe86c6370 0x55afe86d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:01.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.566+0000 7f759c26c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f75941ac710 con 0x55afe86c12e0 2024-09-19T09:26:01.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.566+0000 7f759c26c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f75941ac040 con 0x55afe86c6370 2024-09-19T09:26:01.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.566+0000 7f759c26c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f75941a9260 con 0x55afe86c09a0 2024-09-19T09:26:01.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.566+0000 7f759c26c740 10 client.0 _ll_register_callbacks cb 0x7f7584009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:01.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.566+0000 7f7592ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afe86c12e0 0x55afe86cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:01.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.566+0000 7f7592ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afe86c12e0 0x55afe86cfa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50550/0 (socket says 172.21.15.120:50550) 2024-09-19T09:26:01.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.566+0000 7f7592ffd640 1 -- 172.21.15.120:0/2023261711 learned_addr learned my addr 172.21.15.120:0/2023261711 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:01.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.566+0000 7f75927fc640 1 --2- 172.21.15.120:0/2023261711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55afe86c6370 0x55afe86d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:01.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.566+0000 7f75927fc640 1 --2- 172.21.15.120:0/2023261711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afe86c09a0 0x55afe86cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:01.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.566+0000 7f7592ffd640 1 -- 172.21.15.120:0/2023261711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afe86c09a0 msgr2=0x55afe86cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:01.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.566+0000 7f7592ffd640 1 --2- 172.21.15.120:0/2023261711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afe86c09a0 0x55afe86cf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:01.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.566+0000 7f7592ffd640 1 -- 172.21.15.120:0/2023261711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55afe86c6370 msgr2=0x55afe86d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:01.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.566+0000 7f7592ffd640 1 --2- 172.21.15.120:0/2023261711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55afe86c6370 0x55afe86d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:01.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.566+0000 7f7592ffd640 1 -- 172.21.15.120:0/2023261711 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afe86d1020 con 0x55afe86c12e0 2024-09-19T09:26:01.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.566+0000 7f75927fc640 1 --2- 172.21.15.120:0/2023261711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55afe86c6370 0x55afe86d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:01.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.566+0000 7f7592ffd640 1 --2- 172.21.15.120:0/2023261711 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afe86c12e0 0x55afe86cfa00 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f7584003e40 tx=0x7f7584004080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26055af7bb4eb1a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:01.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.566+0000 7f7576ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:26:01.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.566+0000 7f7576ff5640 1 -- 172.21.15.120:0/2023261711 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7584034060 con 0x55afe86c12e0 2024-09-19T09:26:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.566+0000 7f7576ff5640 1 -- 172.21.15.120:0/2023261711 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f7584003800 con 0x55afe86c12e0 2024-09-19T09:26:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.566+0000 7f759c26c740 1 -- 172.21.15.120:0/2023261711 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7588035070 con 0x55afe86c12e0 2024-09-19T09:26:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.566+0000 7f7576ff5640 1 -- 172.21.15.120:0/2023261711 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7584003a00 con 0x55afe86c12e0 2024-09-19T09:26:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.566+0000 7f759c26c740 10 client.22947 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.566+0000 7f759c26c740 1 -- 172.21.15.120:0/2023261711 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55afe86d23c0 con 0x55afe86c12e0 2024-09-19T09:26:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.567+0000 7f7576ff5640 1 -- 172.21.15.120:0/2023261711 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7588035070 con 0x55afe86c12e0 2024-09-19T09:26:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.567+0000 7f7576ff5640 1 -- 172.21.15.120:0/2023261711 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f7584039070 con 0x55afe86c12e0 2024-09-19T09:26:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.567+0000 7f759c26c740 10 client.22947 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.567+0000 7f759c26c740 10 client.22947 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:26:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.567+0000 7f759c26c740 20 client.22947 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.567+0000 7f759c26c740 1 --2- 172.21.15.120:0/2023261711 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55afe8741d30 0x55afe8744150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.567+0000 7f759c26c740 4 client.22947 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:26:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.567+0000 7f759c26c740 1 -- 172.21.15.120:0/2023261711 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55afe862bd70 con 0x55afe8741d30 2024-09-19T09:26:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.567+0000 7f75927fc640 1 --2- 172.21.15.120:0/2023261711 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55afe8741d30 0x55afe8744150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.567+0000 7f7576ff5640 10 client.22947 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:01.567+0000 7f75927fc640 1 --2- 172.21.15.120:0/2023261711 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55afe8741d30 0x55afe8744150 crc :-1 s=READY pgs=568 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b22aebfef72e78a8 server_cookie=bc701187d16077cb in_seq=0 out_seq=0 2024-09-19T09:26:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:02.193+0000 7f7576ff5640 1 -- 172.21.15.120:0/2023261711 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55afe862bd70 con 0x55afe8741d30 2024-09-19T09:26:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:02.193+0000 7f7576ff5640 10 client.22947 handle_command_reply: tid=0 2024-09-19T09:26:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:02.193+0000 7f759c26c740 1 client.22947 shutdown 2024-09-19T09:26:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:02.193+0000 7f759c26c740 1 -- 172.21.15.120:0/2023261711 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afe86c12e0 msgr2=0x55afe86cfa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:02.193+0000 7f759c26c740 1 --2- 172.21.15.120:0/2023261711 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afe86c12e0 0x55afe86cfa00 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f7584003e40 tx=0x7f7584004080 comp rx=0 tx=0).stop 2024-09-19T09:26:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:02.193+0000 7f759c26c740 1 -- 172.21.15.120:0/2023261711 shutdown_connections 2024-09-19T09:26:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:02.193+0000 7f759c26c740 1 --2- 172.21.15.120:0/2023261711 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55afe8741d30 0x55afe8744150 crc :-1 s=READY pgs=568 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:02.193+0000 7f759c26c740 1 --2- 172.21.15.120:0/2023261711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55afe86c6370 0x55afe86d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:02.193+0000 7f7576ff5640 0 client.22947 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:02.193+0000 7f759c26c740 1 --2- 172.21.15.120:0/2023261711 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afe86c12e0 0x55afe86cfa00 unknown :-1 s=CLOSED pgs=2363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:02.193+0000 7f759c26c740 1 --2- 172.21.15.120:0/2023261711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afe86c09a0 0x55afe86cf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:02.193+0000 7f759c26c740 1 -- 172.21.15.120:0/2023261711 >> 172.21.15.120:0/2023261711 conn(0x55afe86bc010 msgr2=0x55afe86bb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:02.193+0000 7f759c26c740 1 -- 172.21.15.120:0/2023261711 shutdown_connections 2024-09-19T09:26:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:02.193+0000 7f759c26c740 1 -- 172.21.15.120:0/2023261711 wait complete. 2024-09-19T09:26:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:02.193+0000 7f759c26c740 20 client.22947 trim_cache size 0 max 16384 2024-09-19T09:26:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:26:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:26:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:26:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:26:02.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:02.194+0000 7f75993f6640 1 -- 172.21.15.120:0/1353406007 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f757c0a3370 msgr2=0x7f757c0a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:02.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:02.194+0000 7f75993f6640 1 --2- 172.21.15.120:0/1353406007 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f757c0a3370 0x7f757c0a5830 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f758c0b9c40 tx=0x7f758c02f6a0 comp rx=0 tx=0).stop 2024-09-19T09:26:02.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:02.194+0000 7f75993f6640 1 -- 172.21.15.120:0/1353406007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f75941025f0 msgr2=0x7f7594103550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:02.194+0000 7f75993f6640 1 --2- 172.21.15.120:0/1353406007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f75941025f0 0x7f7594103550 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f758c031750 tx=0x7f758c031780 comp rx=0 tx=0).stop 2024-09-19T09:26:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:02.194+0000 7f75993f6640 1 -- 172.21.15.120:0/1353406007 shutdown_connections 2024-09-19T09:26:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:02.194+0000 7f75993f6640 1 --2- 172.21.15.120:0/1353406007 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f757c0a3370 0x7f757c0a5830 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:02.194+0000 7f75993f6640 1 --2- 172.21.15.120:0/1353406007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f75941025f0 0x7f7594103550 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:02.194+0000 7f75993f6640 1 --2- 172.21.15.120:0/1353406007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7594101c30 0x7f7594103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:02.194+0000 7f75993f6640 1 --2- 172.21.15.120:0/1353406007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7594100a40 0x7f75941049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:02.194+0000 7f75993f6640 1 -- 172.21.15.120:0/1353406007 >> 172.21.15.120:0/1353406007 conn(0x7f75940fc1e0 msgr2=0x7f75940fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:02.194+0000 7f75993f6640 1 -- 172.21.15.120:0/1353406007 shutdown_connections 2024-09-19T09:26:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:02.194+0000 7f75993f6640 1 -- 172.21.15.120:0/1353406007 wait complete. 2024-09-19T09:26:02.385 INFO:tasks.workunit.client.0.smithi120.stdout: 10 237317 20.55 MB/sec execute 477 sec latency 453.219 ms 2024-09-19T09:26:02.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:02 smithi196 ceph-mon[35150]: pgmap v1040: 97 pgs: 97 active+clean; 513 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 8.2 MiB/s wr, 659 op/s 2024-09-19T09:26:02.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:02 smithi153 ceph-mon[35071]: pgmap v1040: 97 pgs: 97 active+clean; 513 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 8.2 MiB/s wr, 659 op/s 2024-09-19T09:26:02.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:02 smithi120 ceph-mon[31150]: pgmap v1040: 97 pgs: 97 active+clean; 513 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 8.2 MiB/s wr, 659 op/s 2024-09-19T09:26:03.385 INFO:tasks.workunit.client.0.smithi120.stdout: 10 237718 20.56 MB/sec execute 478 sec latency 280.255 ms 2024-09-19T09:26:04.385 INFO:tasks.workunit.client.0.smithi120.stdout: 10 238168 20.56 MB/sec execute 479 sec latency 67.190 ms 2024-09-19T09:26:04.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:04 smithi196 ceph-mon[35150]: pgmap v1041: 97 pgs: 97 active+clean; 499 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.7 MiB/s wr, 735 op/s 2024-09-19T09:26:04.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:04 smithi153 ceph-mon[35071]: pgmap v1041: 97 pgs: 97 active+clean; 499 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.7 MiB/s wr, 735 op/s 2024-09-19T09:26:04.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:04 smithi120 ceph-mon[31150]: pgmap v1041: 97 pgs: 97 active+clean; 499 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.7 MiB/s wr, 735 op/s 2024-09-19T09:26:05.385 INFO:tasks.workunit.client.0.smithi120.stdout: 10 238716 20.57 MB/sec execute 480 sec latency 75.838 ms 2024-09-19T09:26:06.385 INFO:tasks.workunit.client.0.smithi120.stdout: 10 239258 20.58 MB/sec execute 481 sec latency 104.848 ms 2024-09-19T09:26:06.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:06 smithi196 ceph-mon[35150]: pgmap v1042: 97 pgs: 97 active+clean; 500 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.4 MiB/s wr, 689 op/s 2024-09-19T09:26:06.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:06 smithi153 ceph-mon[35071]: pgmap v1042: 97 pgs: 97 active+clean; 500 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.4 MiB/s wr, 689 op/s 2024-09-19T09:26:06.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:06 smithi120 ceph-mon[31150]: pgmap v1042: 97 pgs: 97 active+clean; 500 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.4 MiB/s wr, 689 op/s 2024-09-19T09:26:07.385 INFO:tasks.workunit.client.0.smithi120.stdout: 10 239881 20.62 MB/sec execute 482 sec latency 118.964 ms 2024-09-19T09:26:08.385 INFO:tasks.workunit.client.0.smithi120.stdout: 10 240363 20.62 MB/sec execute 483 sec latency 257.017 ms 2024-09-19T09:26:08.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:08 smithi196 ceph-mon[35150]: pgmap v1043: 97 pgs: 97 active+clean; 505 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.8 MiB/s wr, 770 op/s 2024-09-19T09:26:08.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:08 smithi153 ceph-mon[35071]: pgmap v1043: 97 pgs: 97 active+clean; 505 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.8 MiB/s wr, 770 op/s 2024-09-19T09:26:08.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:08 smithi120 ceph-mon[31150]: pgmap v1043: 97 pgs: 97 active+clean; 505 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.8 MiB/s wr, 770 op/s 2024-09-19T09:26:09.385 INFO:tasks.workunit.client.0.smithi120.stdout: 10 240945 20.64 MB/sec execute 484 sec latency 87.242 ms 2024-09-19T09:26:10.385 INFO:tasks.workunit.client.0.smithi120.stdout: 10 241569 20.68 MB/sec execute 485 sec latency 44.077 ms 2024-09-19T09:26:10.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:10 smithi196 ceph-mon[35150]: pgmap v1044: 97 pgs: 97 active+clean; 506 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.9 MiB/s wr, 740 op/s 2024-09-19T09:26:10.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:10 smithi153 ceph-mon[35071]: pgmap v1044: 97 pgs: 97 active+clean; 506 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.9 MiB/s wr, 740 op/s 2024-09-19T09:26:10.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:10 smithi120 ceph-mon[31150]: pgmap v1044: 97 pgs: 97 active+clean; 506 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.9 MiB/s wr, 740 op/s 2024-09-19T09:26:11.385 INFO:tasks.workunit.client.0.smithi120.stdout: 10 242091 20.71 MB/sec execute 486 sec latency 65.786 ms 2024-09-19T09:26:12.385 INFO:tasks.workunit.client.0.smithi120.stdout: 10 242689 20.72 MB/sec execute 487 sec latency 57.525 ms 2024-09-19T09:26:12.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:12 smithi196 ceph-mon[35150]: pgmap v1045: 97 pgs: 97 active+clean; 513 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.8 MiB/s wr, 725 op/s 2024-09-19T09:26:12.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:12 smithi153 ceph-mon[35071]: pgmap v1045: 97 pgs: 97 active+clean; 513 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.8 MiB/s wr, 725 op/s 2024-09-19T09:26:12.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:12 smithi120 ceph-mon[31150]: pgmap v1045: 97 pgs: 97 active+clean; 513 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.8 MiB/s wr, 725 op/s 2024-09-19T09:26:13.385 INFO:tasks.workunit.client.0.smithi120.stdout: 10 243022 20.71 MB/sec execute 488 sec latency 308.014 ms 2024-09-19T09:26:14.385 INFO:tasks.workunit.client.0.smithi120.stdout: 10 243530 20.72 MB/sec execute 489 sec latency 112.307 ms 2024-09-19T09:26:14.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:14 smithi196 ceph-mon[35150]: pgmap v1046: 97 pgs: 97 active+clean; 517 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.1 MiB/s wr, 796 op/s 2024-09-19T09:26:14.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:14 smithi153 ceph-mon[35071]: pgmap v1046: 97 pgs: 97 active+clean; 517 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.1 MiB/s wr, 796 op/s 2024-09-19T09:26:14.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:14 smithi120 ceph-mon[31150]: pgmap v1046: 97 pgs: 97 active+clean; 517 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 9.1 MiB/s wr, 796 op/s 2024-09-19T09:26:15.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.289+0000 7f02e3a2e640 1 -- 172.21.15.120:0/4252897595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02dc104f00 msgr2=0x7f02dc1072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.289+0000 7f02e3a2e640 1 --2- 172.21.15.120:0/4252897595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02dc104f00 0x7f02dc1072f0 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7f02cc009900 tx=0x7f02cc02a410 comp rx=0 tx=0).stop 2024-09-19T09:26:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.289+0000 7f02e3a2e640 1 -- 172.21.15.120:0/4252897595 shutdown_connections 2024-09-19T09:26:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.289+0000 7f02e3a2e640 1 --2- 172.21.15.120:0/4252897595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02dc107830 0x7f02dc109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.289+0000 7f02e3a2e640 1 --2- 172.21.15.120:0/4252897595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02dc104f00 0x7f02dc1072f0 unknown :-1 s=CLOSED pgs=2364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.289+0000 7f02e3a2e640 1 --2- 172.21.15.120:0/4252897595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02dc1025d0 0x7f02dc1049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.289+0000 7f02e3a2e640 1 -- 172.21.15.120:0/4252897595 >> 172.21.15.120:0/4252897595 conn(0x7f02dc0fc1b0 msgr2=0x7f02dc0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.289+0000 7f02e3a2e640 1 -- 172.21.15.120:0/4252897595 shutdown_connections 2024-09-19T09:26:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.290+0000 7f02e3a2e640 1 -- 172.21.15.120:0/4252897595 wait complete. 2024-09-19T09:26:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.290+0000 7f02e3a2e640 1 Processor -- start 2024-09-19T09:26:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.290+0000 7f02e3a2e640 1 -- start start 2024-09-19T09:26:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.290+0000 7f02e3a2e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02dc1025d0 0x7f02dc101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.290+0000 7f02e3a2e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02dc104f00 0x7f02dc100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.290+0000 7f02e3a2e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02dc107830 0x7f02dc100ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.290+0000 7f02e3a2e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc10d810 con 0x7f02dc1025d0 2024-09-19T09:26:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.290+0000 7f02e3a2e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc10d690 con 0x7f02dc104f00 2024-09-19T09:26:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.290+0000 7f02e3a2e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc10d990 con 0x7f02dc107830 2024-09-19T09:26:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.290+0000 7f02e17ca640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02dc1025d0 0x7f02dc101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.290+0000 7f02e17ca640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02dc1025d0 0x7f02dc101f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50972/0 (socket says 172.21.15.120:50972) 2024-09-19T09:26:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.290+0000 7f02e17ca640 1 -- 172.21.15.120:0/474160943 learned_addr learned my addr 172.21.15.120:0/474160943 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.290+0000 7f02e0fc9640 1 --2- 172.21.15.120:0/474160943 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02dc104f00 0x7f02dc100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.291+0000 7f02e1fcb640 1 --2- 172.21.15.120:0/474160943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02dc107830 0x7f02dc100ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.291+0000 7f02e17ca640 1 -- 172.21.15.120:0/474160943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02dc107830 msgr2=0x7f02dc100ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.291+0000 7f02e17ca640 1 --2- 172.21.15.120:0/474160943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02dc107830 0x7f02dc100ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.291+0000 7f02e17ca640 1 -- 172.21.15.120:0/474160943 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02dc104f00 msgr2=0x7f02dc100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.291+0000 7f02e17ca640 1 --2- 172.21.15.120:0/474160943 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02dc104f00 0x7f02dc100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.291+0000 7f02e17ca640 1 -- 172.21.15.120:0/474160943 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02dc101350 con 0x7f02dc1025d0 2024-09-19T09:26:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.291+0000 7f02e17ca640 1 --2- 172.21.15.120:0/474160943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02dc1025d0 0x7f02dc101f40 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f02d800e9f0 tx=0x7f02d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.291+0000 7f02d27fc640 1 -- 172.21.15.120:0/474160943 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d800cd80 con 0x7f02dc1025d0 2024-09-19T09:26:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.291+0000 7f02d27fc640 1 -- 172.21.15.120:0/474160943 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02d8010910 con 0x7f02dc1025d0 2024-09-19T09:26:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.291+0000 7f02e3a2e640 1 -- 172.21.15.120:0/474160943 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02dc1a1fc0 con 0x7f02dc1025d0 2024-09-19T09:26:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.291+0000 7f02d27fc640 1 -- 172.21.15.120:0/474160943 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d8018990 con 0x7f02dc1025d0 2024-09-19T09:26:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.291+0000 7f02e3a2e640 1 -- 172.21.15.120:0/474160943 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02dc1a23c0 con 0x7f02dc1025d0 2024-09-19T09:26:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.294+0000 7f02e67c9740 1 Processor -- start 2024-09-19T09:26:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.294+0000 7f02d27fc640 1 -- 172.21.15.120:0/474160943 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f02d8010ac0 con 0x7f02dc1025d0 2024-09-19T09:26:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.294+0000 7f02e67c9740 1 -- start start 2024-09-19T09:26:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.294+0000 7f02e67c9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba6e2e09a0 0x55ba6e2e0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.294+0000 7f02e67c9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba6e2e12e0 0x55ba6e2e5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.294+0000 7f02e67c9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba6e2e6370 0x55ba6e2e8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.294+0000 7f02e67c9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ba6e2442f0 con 0x55ba6e2e6370 2024-09-19T09:26:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.294+0000 7f02e67c9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ba6e1ec270 con 0x55ba6e2e09a0 2024-09-19T09:26:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.294+0000 7f02e67c9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ba6e2402b0 con 0x55ba6e2e12e0 2024-09-19T09:26:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.294+0000 7f02e0fc9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba6e2e6370 0x55ba6e2e8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.294+0000 7f02e0fc9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba6e2e6370 0x55ba6e2e8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50980/0 (socket says 172.21.15.120:50980) 2024-09-19T09:26:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.294+0000 7f02e0fc9640 1 -- 172.21.15.120:0/2698433213 learned_addr learned my addr 172.21.15.120:0/2698433213 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.294+0000 7f02e1fcb640 1 --2- 172.21.15.120:0/2698433213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba6e2e09a0 0x55ba6e2e0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.294+0000 7f02e17ca640 1 --2- 172.21.15.120:0/2698433213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba6e2e12e0 0x55ba6e2e5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.295+0000 7f02e0fc9640 1 -- 172.21.15.120:0/2698433213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba6e2e12e0 msgr2=0x55ba6e2e5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.295+0000 7f02e0fc9640 1 --2- 172.21.15.120:0/2698433213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba6e2e12e0 0x55ba6e2e5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.295+0000 7f02e0fc9640 1 -- 172.21.15.120:0/2698433213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba6e2e09a0 msgr2=0x55ba6e2e0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.295+0000 7f02e0fc9640 1 --2- 172.21.15.120:0/2698433213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba6e2e09a0 0x55ba6e2e0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.295+0000 7f02e0fc9640 1 -- 172.21.15.120:0/2698433213 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba6e1eb340 con 0x55ba6e2e6370 2024-09-19T09:26:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.295+0000 7f02e1fcb640 1 --2- 172.21.15.120:0/2698433213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba6e2e09a0 0x55ba6e2e0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.295+0000 7f02d27fc640 1 --2- 172.21.15.120:0/474160943 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f02c80a32f0 0x7f02c80a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.295+0000 7f02e0fc9640 1 --2- 172.21.15.120:0/2698433213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba6e2e6370 0x55ba6e2e8830 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f02cc009900 tx=0x7f02cc005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdfa7bfb97dc9ae8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.295+0000 7f02e1fcb640 1 --2- 172.21.15.120:0/474160943 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f02c80a32f0 0x7f02c80a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.295+0000 7f02d27fc640 1 -- 172.21.15.120:0/474160943 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f02d80cfba0 con 0x7f02dc1025d0 2024-09-19T09:26:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.295+0000 7f02c77fe640 1 -- 172.21.15.120:0/2698433213 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc040070 con 0x55ba6e2e6370 2024-09-19T09:26:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.295+0000 7f02c77fe640 1 -- 172.21.15.120:0/2698433213 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02cc031970 con 0x55ba6e2e6370 2024-09-19T09:26:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.295+0000 7f02c77fe640 1 -- 172.21.15.120:0/2698433213 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc035040 con 0x55ba6e2e6370 2024-09-19T09:26:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.296+0000 7f02e67c9740 1 -- 172.21.15.120:0/2698433213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba6e2e6370 msgr2=0x55ba6e2e8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.296+0000 7f02e1fcb640 1 --2- 172.21.15.120:0/474160943 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f02c80a32f0 0x7f02c80a57b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009fd0 tx=0x7f02d4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.296+0000 7f02e67c9740 1 --2- 172.21.15.120:0/2698433213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba6e2e6370 0x55ba6e2e8830 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f02cc009900 tx=0x7f02cc005bd0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.296+0000 7f02e67c9740 1 -- 172.21.15.120:0/2698433213 shutdown_connections 2024-09-19T09:26:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.296+0000 7f02e67c9740 1 --2- 172.21.15.120:0/2698433213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba6e2e6370 0x55ba6e2e8830 unknown :-1 s=CLOSED pgs=2366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.296+0000 7f02e67c9740 1 --2- 172.21.15.120:0/2698433213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba6e2e12e0 0x55ba6e2e5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.296+0000 7f02e67c9740 1 --2- 172.21.15.120:0/2698433213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba6e2e09a0 0x55ba6e2e0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.296+0000 7f02e67c9740 1 -- 172.21.15.120:0/2698433213 >> 172.21.15.120:0/2698433213 conn(0x55ba6e2dc010 msgr2=0x55ba6e2dc410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.296+0000 7f02e67c9740 1 -- 172.21.15.120:0/2698433213 shutdown_connections 2024-09-19T09:26:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.296+0000 7f02e67c9740 1 -- 172.21.15.120:0/2698433213 wait complete. 2024-09-19T09:26:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.297+0000 7f02e67c9740 1 -- start start 2024-09-19T09:26:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.297+0000 7f02e67c9740 1 Processor -- start 2024-09-19T09:26:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.297+0000 7f02e67c9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba6e2e09a0 0x55ba6e2eab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.297+0000 7f02e67c9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba6e2e12e0 0x55ba6e2eb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.297+0000 7f02e67c9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba6e2e6370 0x55ba6e2f2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.297+0000 7f02e67c9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ba6e2e9330 con 0x55ba6e2e12e0 2024-09-19T09:26:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.297+0000 7f02e67c9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ba6e1cd290 con 0x55ba6e2e09a0 2024-09-19T09:26:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.297+0000 7f02e67c9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ba6e2e94b0 con 0x55ba6e2e6370 2024-09-19T09:26:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.297+0000 7f02e67c9740 10 client.0 _ll_register_callbacks cb 0x55ba6e1fe140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.297+0000 7f02e0fc9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba6e2e12e0 0x55ba6e2eb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.297+0000 7f02e0fc9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba6e2e12e0 0x55ba6e2eb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50990/0 (socket says 172.21.15.120:50990) 2024-09-19T09:26:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.297+0000 7f02e0fc9640 1 -- 172.21.15.120:0/3810904098 learned_addr learned my addr 172.21.15.120:0/3810904098 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.297+0000 7f02e17ca640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba6e2e09a0 0x55ba6e2eab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.297+0000 7f02e1fcb640 1 --2- 172.21.15.120:0/3810904098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba6e2e6370 0x55ba6e2f2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.297+0000 7f02e0fc9640 1 -- 172.21.15.120:0/3810904098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba6e2e6370 msgr2=0x55ba6e2f2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.297+0000 7f02e0fc9640 1 --2- 172.21.15.120:0/3810904098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba6e2e6370 0x55ba6e2f2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.297+0000 7f02e0fc9640 1 -- 172.21.15.120:0/3810904098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba6e2e09a0 msgr2=0x55ba6e2eab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.297+0000 7f02e0fc9640 1 --2- 172.21.15.120:0/3810904098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba6e2e09a0 0x55ba6e2eab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.297+0000 7f02e0fc9640 1 -- 172.21.15.120:0/3810904098 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba6df57210 con 0x55ba6e2e12e0 2024-09-19T09:26:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.298+0000 7f02e0fc9640 1 --2- 172.21.15.120:0/3810904098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba6e2e12e0 0x55ba6e2eb0d0 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f02cc005c40 tx=0x7f02cc005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae1d78f3699dcee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.298+0000 7f02c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:26:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.298+0000 7f02c4ff9640 1 -- 172.21.15.120:0/3810904098 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc04d060 con 0x55ba6e2e12e0 2024-09-19T09:26:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.298+0000 7f02c4ff9640 1 -- 172.21.15.120:0/3810904098 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f02cc004240 con 0x55ba6e2e12e0 2024-09-19T09:26:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.298+0000 7f02e67c9740 1 -- 172.21.15.120:0/3810904098 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02cc031970 con 0x55ba6e2e12e0 2024-09-19T09:26:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.298+0000 7f02c4ff9640 1 -- 172.21.15.120:0/3810904098 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f02cc040070 con 0x55ba6e2e12e0 2024-09-19T09:26:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.298+0000 7f02e67c9740 10 client.22959 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.298+0000 7f02e67c9740 1 -- 172.21.15.120:0/3810904098 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ba6e023620 con 0x55ba6e2e12e0 2024-09-19T09:26:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.298+0000 7f02c4ff9640 1 -- 172.21.15.120:0/3810904098 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02cc031970 con 0x55ba6e2e12e0 2024-09-19T09:26:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.298+0000 7f02c4ff9640 1 -- 172.21.15.120:0/3810904098 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f02cc052070 con 0x55ba6e2e12e0 2024-09-19T09:26:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.298+0000 7f02e67c9740 10 client.22959 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.298+0000 7f02e67c9740 10 client.22959 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:26:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.298+0000 7f02e67c9740 20 client.22959 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.298+0000 7f02e67c9740 1 --2- 172.21.15.120:0/3810904098 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ba6e2f6410 0x55ba6e2f8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.298+0000 7f02e67c9740 4 client.22959 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:26:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.298+0000 7f02e67c9740 1 -- 172.21.15.120:0/3810904098 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba6e251a20 con 0x55ba6e2f6410 2024-09-19T09:26:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.299+0000 7f02e17ca640 1 --2- 172.21.15.120:0/3810904098 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ba6e2f6410 0x55ba6e2f8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.299+0000 7f02e17ca640 1 --2- 172.21.15.120:0/3810904098 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ba6e2f6410 0x55ba6e2f8800 crc :-1 s=READY pgs=569 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82fceef1cb64f836 server_cookie=603f6382423ff48c in_seq=0 out_seq=0 2024-09-19T09:26:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.299+0000 7f02c4ff9640 10 client.22959 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.301+0000 7f02c4ff9640 1 -- 172.21.15.120:0/3810904098 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ba6e251a20 con 0x55ba6e2f6410 2024-09-19T09:26:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.301+0000 7f02c4ff9640 10 client.22959 handle_command_reply: tid=0 2024-09-19T09:26:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.301+0000 7f02e67c9740 1 client.22959 shutdown 2024-09-19T09:26:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.301+0000 7f02e67c9740 1 -- 172.21.15.120:0/3810904098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba6e2e12e0 msgr2=0x55ba6e2eb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.301+0000 7f02e67c9740 1 --2- 172.21.15.120:0/3810904098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba6e2e12e0 0x55ba6e2eb0d0 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f02cc005c40 tx=0x7f02cc005250 comp rx=0 tx=0).stop 2024-09-19T09:26:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.301+0000 7f02e67c9740 1 -- 172.21.15.120:0/3810904098 shutdown_connections 2024-09-19T09:26:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.301+0000 7f02e67c9740 1 --2- 172.21.15.120:0/3810904098 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ba6e2f6410 0x55ba6e2f8800 crc :-1 s=READY pgs=569 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.301+0000 7f02e67c9740 1 --2- 172.21.15.120:0/3810904098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba6e2e6370 0x55ba6e2f2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.301+0000 7f02e67c9740 1 --2- 172.21.15.120:0/3810904098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba6e2e12e0 0x55ba6e2eb0d0 unknown :-1 s=CLOSED pgs=2367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.301+0000 7f02c4ff9640 0 client.22959 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.301+0000 7f02e67c9740 1 --2- 172.21.15.120:0/3810904098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba6e2e09a0 0x55ba6e2eab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.301+0000 7f02e67c9740 1 -- 172.21.15.120:0/3810904098 >> 172.21.15.120:0/3810904098 conn(0x55ba6e2dc010 msgr2=0x55ba6e2dd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.301+0000 7f02e67c9740 1 -- 172.21.15.120:0/3810904098 shutdown_connections 2024-09-19T09:26:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.301+0000 7f02e67c9740 1 -- 172.21.15.120:0/3810904098 wait complete. 2024-09-19T09:26:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.301+0000 7f02e67c9740 20 client.22959 trim_cache size 0 max 16384 2024-09-19T09:26:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.319+0000 7f02e67c9740 1 Processor -- start 2024-09-19T09:26:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.319+0000 7f02e67c9740 1 -- start start 2024-09-19T09:26:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.319+0000 7f02e67c9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba6e2e09a0 0x55ba6e2e0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.319+0000 7f02e67c9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba6e2e12e0 0x55ba6e2f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.319+0000 7f02e67c9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba6e2e6370 0x55ba6e2db0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.319+0000 7f02e67c9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc1ab290 con 0x55ba6e2e09a0 2024-09-19T09:26:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.319+0000 7f02e67c9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc1acdd0 con 0x55ba6e2e12e0 2024-09-19T09:26:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.319+0000 7f02e67c9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc1ab960 con 0x55ba6e2e6370 2024-09-19T09:26:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.320+0000 7f02e17ca640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba6e2e09a0 0x55ba6e2e0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.320+0000 7f02e17ca640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba6e2e09a0 0x55ba6e2e0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51004/0 (socket says 172.21.15.120:51004) 2024-09-19T09:26:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.320+0000 7f02e0fc9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba6e2e12e0 0x55ba6e2f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.320+0000 7f02e17ca640 1 -- 172.21.15.120:0/3198854381 learned_addr learned my addr 172.21.15.120:0/3198854381 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.320+0000 7f02e1fcb640 1 --2- 172.21.15.120:0/3198854381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba6e2e6370 0x55ba6e2db0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.320+0000 7f02e0fc9640 1 -- 172.21.15.120:0/3198854381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba6e2e6370 msgr2=0x55ba6e2db0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.320+0000 7f02e0fc9640 1 --2- 172.21.15.120:0/3198854381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba6e2e6370 0x55ba6e2db0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.320+0000 7f02e0fc9640 1 -- 172.21.15.120:0/3198854381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba6e2e09a0 msgr2=0x55ba6e2e0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.320+0000 7f02e0fc9640 1 --2- 172.21.15.120:0/3198854381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba6e2e09a0 0x55ba6e2e0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.320+0000 7f02e0fc9640 1 -- 172.21.15.120:0/3198854381 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba6e2e1ae0 con 0x55ba6e2e12e0 2024-09-19T09:26:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.320+0000 7f02e17ca640 1 --2- 172.21.15.120:0/3198854381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba6e2e09a0 0x55ba6e2e0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.320+0000 7f02e0fc9640 1 --2- 172.21.15.120:0/3198854381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba6e2e12e0 0x55ba6e2f20d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f02cc0038f0 tx=0x7f02cc005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5c6dd8c0ede1a3d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.320+0000 7f02c6ffd640 1 -- 172.21.15.120:0/3198854381 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc058030 con 0x55ba6e2e12e0 2024-09-19T09:26:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.320+0000 7f02c6ffd640 1 -- 172.21.15.120:0/3198854381 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02cc003e50 con 0x55ba6e2e12e0 2024-09-19T09:26:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.320+0000 7f02c6ffd640 1 -- 172.21.15.120:0/3198854381 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc04d040 con 0x55ba6e2e12e0 2024-09-19T09:26:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.321+0000 7f02e67c9740 1 -- 172.21.15.120:0/3198854381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba6e2e12e0 msgr2=0x55ba6e2f20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.321+0000 7f02e67c9740 1 --2- 172.21.15.120:0/3198854381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba6e2e12e0 0x55ba6e2f20d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f02cc0038f0 tx=0x7f02cc005bd0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.321+0000 7f02e67c9740 1 -- 172.21.15.120:0/3198854381 shutdown_connections 2024-09-19T09:26:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.321+0000 7f02e67c9740 1 --2- 172.21.15.120:0/3198854381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba6e2e6370 0x55ba6e2db0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.321+0000 7f02e67c9740 1 --2- 172.21.15.120:0/3198854381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba6e2e12e0 0x55ba6e2f20d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.321+0000 7f02e67c9740 1 --2- 172.21.15.120:0/3198854381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba6e2e09a0 0x55ba6e2e0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.321+0000 7f02e67c9740 1 -- 172.21.15.120:0/3198854381 >> 172.21.15.120:0/3198854381 conn(0x55ba6e2dc010 msgr2=0x55ba6e2da9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.321+0000 7f02e67c9740 1 -- 172.21.15.120:0/3198854381 shutdown_connections 2024-09-19T09:26:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.321+0000 7f02e67c9740 1 -- 172.21.15.120:0/3198854381 wait complete. 2024-09-19T09:26:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.322+0000 7f02e67c9740 1 -- start start 2024-09-19T09:26:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.322+0000 7f02e67c9740 1 Processor -- start 2024-09-19T09:26:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.322+0000 7f02e67c9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba6e2e09a0 0x55ba6e2ef390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.322+0000 7f02e67c9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba6e2e12e0 0x55ba6e2ef8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.322+0000 7f02e67c9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba6e2e6370 0x55ba6e2f08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.322+0000 7f02e67c9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc1ac700 con 0x55ba6e2e6370 2024-09-19T09:26:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.322+0000 7f02e67c9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc1ac030 con 0x55ba6e2e12e0 2024-09-19T09:26:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.322+0000 7f02e67c9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc1a9250 con 0x55ba6e2e09a0 2024-09-19T09:26:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.322+0000 7f02e67c9740 10 client.0 _ll_register_callbacks cb 0x7f02d4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.322+0000 7f02e1fcb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba6e2e6370 0x55ba6e2f08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.322+0000 7f02e17ca640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba6e2e09a0 0x55ba6e2ef390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.322+0000 7f02e1fcb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba6e2e6370 0x55ba6e2f08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51012/0 (socket says 172.21.15.120:51012) 2024-09-19T09:26:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.322+0000 7f02e1fcb640 1 -- 172.21.15.120:0/403892899 learned_addr learned my addr 172.21.15.120:0/403892899 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.323+0000 7f02e0fc9640 1 --2- 172.21.15.120:0/403892899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba6e2e12e0 0x55ba6e2ef8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.323+0000 7f02e1fcb640 1 -- 172.21.15.120:0/403892899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba6e2e09a0 msgr2=0x55ba6e2ef390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.323+0000 7f02e1fcb640 1 --2- 172.21.15.120:0/403892899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba6e2e09a0 0x55ba6e2ef390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.323+0000 7f02e1fcb640 1 -- 172.21.15.120:0/403892899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba6e2e12e0 msgr2=0x55ba6e2ef8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.323+0000 7f02e1fcb640 1 --2- 172.21.15.120:0/403892899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba6e2e12e0 0x55ba6e2ef8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.323+0000 7f02e1fcb640 1 -- 172.21.15.120:0/403892899 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba6e2f0ef0 con 0x55ba6e2e6370 2024-09-19T09:26:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.323+0000 7f02e17ca640 1 --2- 172.21.15.120:0/403892899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba6e2e09a0 0x55ba6e2ef390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.323+0000 7f02e0fc9640 1 --2- 172.21.15.120:0/403892899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba6e2e12e0 0x55ba6e2ef8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:15.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.323+0000 7f02e1fcb640 1 --2- 172.21.15.120:0/403892899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba6e2e6370 0x55ba6e2f08e0 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f02d4011fc0 tx=0x7f02d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4322f392cabfab4f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:15.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.323+0000 7f02c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:26:15.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.323+0000 7f02c4ff9640 1 -- 172.21.15.120:0/403892899 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d402e060 con 0x55ba6e2e6370 2024-09-19T09:26:15.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.323+0000 7f02c4ff9640 1 -- 172.21.15.120:0/403892899 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f02d40107c0 con 0x55ba6e2e6370 2024-09-19T09:26:15.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.323+0000 7f02e67c9740 1 -- 172.21.15.120:0/403892899 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02cc003e50 con 0x55ba6e2e6370 2024-09-19T09:26:15.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.324+0000 7f02c4ff9640 1 -- 172.21.15.120:0/403892899 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f02d40109c0 con 0x55ba6e2e6370 2024-09-19T09:26:15.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.324+0000 7f02e67c9740 10 client.22965 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:15.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.324+0000 7f02e67c9740 1 -- 172.21.15.120:0/403892899 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ba6e2f22d0 con 0x55ba6e2e6370 2024-09-19T09:26:15.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.324+0000 7f02c4ff9640 1 -- 172.21.15.120:0/403892899 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02cc003e50 con 0x55ba6e2e6370 2024-09-19T09:26:15.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.324+0000 7f02c4ff9640 1 -- 172.21.15.120:0/403892899 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f02d401ecd0 con 0x55ba6e2e6370 2024-09-19T09:26:15.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.324+0000 7f02e67c9740 10 client.22965 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:15.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.325+0000 7f02e67c9740 10 client.22965 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:26:15.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.325+0000 7f02e67c9740 20 client.22965 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:15.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.325+0000 7f02e67c9740 1 --2- 172.21.15.120:0/403892899 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ba6e3615b0 0x55ba6e3639d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:15.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.325+0000 7f02e67c9740 4 client.22965 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:26:15.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.325+0000 7f02e67c9740 1 -- 172.21.15.120:0/403892899 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ba6e24bd70 con 0x55ba6e3615b0 2024-09-19T09:26:15.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.325+0000 7f02e17ca640 1 --2- 172.21.15.120:0/403892899 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ba6e3615b0 0x55ba6e3639d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:15.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.325+0000 7f02c4ff9640 10 client.22965 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:15.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.325+0000 7f02e17ca640 1 --2- 172.21.15.120:0/403892899 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ba6e3615b0 0x55ba6e3639d0 crc :-1 s=READY pgs=570 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=319b5235d1f81bcb server_cookie=61f7cfe52105f3ee in_seq=0 out_seq=0 2024-09-19T09:26:15.385 INFO:tasks.workunit.client.0.smithi120.stdout: 10 244142 20.74 MB/sec execute 490 sec latency 79.751 ms 2024-09-19T09:26:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.980+0000 7f02c4ff9640 1 -- 172.21.15.120:0/403892899 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ba6e24bd70 con 0x55ba6e3615b0 2024-09-19T09:26:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.980+0000 7f02c4ff9640 10 client.22965 handle_command_reply: tid=0 2024-09-19T09:26:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.980+0000 7f02e67c9740 1 client.22965 shutdown 2024-09-19T09:26:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.980+0000 7f02e67c9740 1 -- 172.21.15.120:0/403892899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba6e2e6370 msgr2=0x55ba6e2f08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.980+0000 7f02e67c9740 1 --2- 172.21.15.120:0/403892899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba6e2e6370 0x55ba6e2f08e0 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f02d4011fc0 tx=0x7f02d4004a40 comp rx=0 tx=0).stop 2024-09-19T09:26:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.981+0000 7f02e67c9740 1 -- 172.21.15.120:0/403892899 shutdown_connections 2024-09-19T09:26:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.981+0000 7f02e67c9740 1 --2- 172.21.15.120:0/403892899 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55ba6e3615b0 0x55ba6e3639d0 crc :-1 s=READY pgs=570 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.981+0000 7f02e67c9740 1 --2- 172.21.15.120:0/403892899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba6e2e6370 0x55ba6e2f08e0 unknown :-1 s=CLOSED pgs=2368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.981+0000 7f02e67c9740 1 --2- 172.21.15.120:0/403892899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba6e2e12e0 0x55ba6e2ef8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.981+0000 7f02e67c9740 1 --2- 172.21.15.120:0/403892899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba6e2e09a0 0x55ba6e2ef390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.981+0000 7f02e67c9740 1 -- 172.21.15.120:0/403892899 >> 172.21.15.120:0/403892899 conn(0x55ba6e2dc010 msgr2=0x55ba6e2db1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.981+0000 7f02c4ff9640 0 client.22965 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.981+0000 7f02e67c9740 1 -- 172.21.15.120:0/403892899 shutdown_connections 2024-09-19T09:26:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.981+0000 7f02e67c9740 1 -- 172.21.15.120:0/403892899 wait complete. 2024-09-19T09:26:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.981+0000 7f02e67c9740 20 client.22965 trim_cache size 0 max 16384 2024-09-19T09:26:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:26:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:26:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:26:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:26:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.981+0000 7f02e3a2e640 1 -- 172.21.15.120:0/474160943 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f02c80a32f0 msgr2=0x7f02c80a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.981+0000 7f02e3a2e640 1 --2- 172.21.15.120:0/474160943 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f02c80a32f0 0x7f02c80a57b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009fd0 tx=0x7f02d4009500 comp rx=0 tx=0).stop 2024-09-19T09:26:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.981+0000 7f02e3a2e640 1 -- 172.21.15.120:0/474160943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02dc1025d0 msgr2=0x7f02dc101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.981+0000 7f02e3a2e640 1 --2- 172.21.15.120:0/474160943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02dc1025d0 0x7f02dc101f40 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f02d800e9f0 tx=0x7f02d800eeb0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.982+0000 7f02e3a2e640 1 -- 172.21.15.120:0/474160943 shutdown_connections 2024-09-19T09:26:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.982+0000 7f02e3a2e640 1 --2- 172.21.15.120:0/474160943 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f02c80a32f0 0x7f02c80a57b0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.982+0000 7f02e3a2e640 1 --2- 172.21.15.120:0/474160943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02dc107830 0x7f02dc100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.982+0000 7f02e3a2e640 1 --2- 172.21.15.120:0/474160943 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02dc104f00 0x7f02dc100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.982+0000 7f02e3a2e640 1 --2- 172.21.15.120:0/474160943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02dc1025d0 0x7f02dc101f40 unknown :-1 s=CLOSED pgs=2365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.982+0000 7f02e3a2e640 1 -- 172.21.15.120:0/474160943 >> 172.21.15.120:0/474160943 conn(0x7f02dc0fc1b0 msgr2=0x7f02dc1030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.982+0000 7f02e3a2e640 1 -- 172.21.15.120:0/474160943 shutdown_connections 2024-09-19T09:26:15.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:15.982+0000 7f02e3a2e640 1 -- 172.21.15.120:0/474160943 wait complete. 2024-09-19T09:26:16.385 INFO:tasks.workunit.client.0.smithi120.stdout: 10 244511 20.75 MB/sec execute 491 sec latency 388.713 ms 2024-09-19T09:26:16.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:16 smithi196 ceph-mon[35150]: pgmap v1047: 97 pgs: 97 active+clean; 517 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.3 MiB/s wr, 553 op/s 2024-09-19T09:26:16.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:16 smithi153 ceph-mon[35071]: pgmap v1047: 97 pgs: 97 active+clean; 517 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.3 MiB/s wr, 553 op/s 2024-09-19T09:26:16.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:16 smithi120 ceph-mon[31150]: pgmap v1047: 97 pgs: 97 active+clean; 517 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.3 MiB/s wr, 553 op/s 2024-09-19T09:26:17.101 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:26:17.386 INFO:tasks.workunit.client.0.smithi120.stdout: 10 245288 20.78 MB/sec execute 492 sec latency 52.958 ms 2024-09-19T09:26:17.386 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:18.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.975+0000 7fd22f59e640 1 -- 172.21.15.120:0/1404502491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2301008d0 msgr2=0x7fd230100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:18.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.975+0000 7fd22f59e640 1 --2- 172.21.15.120:0/1404502491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2301008d0 0x7fd230100d30 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7fd224009930 tx=0x7fd22402a420 comp rx=0 tx=0).stop 2024-09-19T09:26:18.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.975+0000 7fd22f59e640 1 -- 172.21.15.120:0/1404502491 shutdown_connections 2024-09-19T09:26:18.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.975+0000 7fd22f59e640 1 --2- 172.21.15.120:0/1404502491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd230101270 0x7fd23010d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.975+0000 7fd22f59e640 1 --2- 172.21.15.120:0/1404502491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2301008d0 0x7fd230100d30 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.975+0000 7fd22f59e640 1 --2- 172.21.15.120:0/1404502491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2301068d0 0x7fd230106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.975+0000 7fd22f59e640 1 -- 172.21.15.120:0/1404502491 >> 172.21.15.120:0/1404502491 conn(0x7fd2300fc610 msgr2=0x7fd2300fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:18.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.975+0000 7fd22f59e640 1 -- 172.21.15.120:0/1404502491 shutdown_connections 2024-09-19T09:26:18.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.975+0000 7fd22f59e640 1 -- 172.21.15.120:0/1404502491 wait complete. 2024-09-19T09:26:18.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.976+0000 7fd22f59e640 1 Processor -- start 2024-09-19T09:26:18.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.976+0000 7fd22f59e640 1 -- start start 2024-09-19T09:26:18.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.977+0000 7fd22f59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2301008d0 0x7fd230199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:18.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.977+0000 7fd22f59e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd230101270 0x7fd230199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:18.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.977+0000 7fd22f59e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2301068d0 0x7fd2301a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:18.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.977+0000 7fd22f59e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd230110d20 con 0x7fd2301008d0 2024-09-19T09:26:18.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.977+0000 7fd22f59e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd230110ba0 con 0x7fd230101270 2024-09-19T09:26:18.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.977+0000 7fd22f59e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd230110ea0 con 0x7fd2301068d0 2024-09-19T09:26:18.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.977+0000 7fd22e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2301008d0 0x7fd230199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:18.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.977+0000 7fd22e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2301008d0 0x7fd230199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51022/0 (socket says 172.21.15.120:51022) 2024-09-19T09:26:18.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.977+0000 7fd22e59c640 1 -- 172.21.15.120:0/4180172493 learned_addr learned my addr 172.21.15.120:0/4180172493 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:18.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.977+0000 7fd22dd9b640 1 --2- 172.21.15.120:0/4180172493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd230101270 0x7fd230199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:18.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.977+0000 7fd22ed9d640 1 --2- 172.21.15.120:0/4180172493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2301068d0 0x7fd2301a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:18.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.977+0000 7fd22e59c640 1 -- 172.21.15.120:0/4180172493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2301068d0 msgr2=0x7fd2301a41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:18.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.977+0000 7fd22e59c640 1 --2- 172.21.15.120:0/4180172493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2301068d0 0x7fd2301a41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.977+0000 7fd22e59c640 1 -- 172.21.15.120:0/4180172493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd230101270 msgr2=0x7fd230199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:18.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.977+0000 7fd22e59c640 1 --2- 172.21.15.120:0/4180172493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd230101270 0x7fd230199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.977+0000 7fd22e59c640 1 -- 172.21.15.120:0/4180172493 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2301a46f0 con 0x7fd2301008d0 2024-09-19T09:26:18.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.977+0000 7fd22ed9d640 1 --2- 172.21.15.120:0/4180172493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2301068d0 0x7fd2301a41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:26:18.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.978+0000 7fd22e59c640 1 --2- 172.21.15.120:0/4180172493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2301008d0 0x7fd230199130 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7fd22000e9f0 tx=0x7fd22000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:18.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.978+0000 7fd21f7fe640 1 -- 172.21.15.120:0/4180172493 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22000cd80 con 0x7fd2301008d0 2024-09-19T09:26:18.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.978+0000 7fd21f7fe640 1 -- 172.21.15.120:0/4180172493 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd220005500 con 0x7fd2301008d0 2024-09-19T09:26:18.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.978+0000 7fd22f59e640 1 -- 172.21.15.120:0/4180172493 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2301a48c0 con 0x7fd2301008d0 2024-09-19T09:26:18.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.978+0000 7fd21f7fe640 1 -- 172.21.15.120:0/4180172493 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220010690 con 0x7fd2301008d0 2024-09-19T09:26:18.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.978+0000 7fd22f59e640 1 -- 172.21.15.120:0/4180172493 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2301a4d80 con 0x7fd2301008d0 2024-09-19T09:26:18.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.980+0000 7fd21f7fe640 1 -- 172.21.15.120:0/4180172493 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd2200046c0 con 0x7fd2301008d0 2024-09-19T09:26:18.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.981+0000 7fd21f7fe640 1 --2- 172.21.15.120:0/4180172493 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd2180a3240 0x7fd2180a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:18.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.981+0000 7fd21f7fe640 1 -- 172.21.15.120:0/4180172493 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd2200cfd60 con 0x7fd2301008d0 2024-09-19T09:26:18.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.981+0000 7fd22dd9b640 1 --2- 172.21.15.120:0/4180172493 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd2180a3240 0x7fd2180a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:18.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.981+0000 7fd236de6740 1 Processor -- start 2024-09-19T09:26:18.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.982+0000 7fd236de6740 1 -- start start 2024-09-19T09:26:18.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.982+0000 7fd22dd9b640 1 --2- 172.21.15.120:0/4180172493 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd2180a3240 0x7fd2180a5700 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd224006010 tx=0x7fd224000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:18.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.982+0000 7fd236de6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9fe77670 0x558d9fe77a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:18.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.982+0000 7fd236de6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9fe77f90 0x558d9fe82310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:18.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.982+0000 7fd236de6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9fe82850 0x558d9fe84c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:18.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.982+0000 7fd236de6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558d9fddd2e0 con 0x558d9fe82850 2024-09-19T09:26:18.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.982+0000 7fd236de6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558d9fd84ce0 con 0x558d9fe77f90 2024-09-19T09:26:18.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.982+0000 7fd236de6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558d9fdd92a0 con 0x558d9fe77670 2024-09-19T09:26:18.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.982+0000 7fd22ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9fe82850 0x558d9fe84c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:18.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.982+0000 7fd22e59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9fe77670 0x558d9fe77a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:18.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.982+0000 7fd22ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9fe82850 0x558d9fe84c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51036/0 (socket says 172.21.15.120:51036) 2024-09-19T09:26:18.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.982+0000 7fd22ed9d640 1 -- 172.21.15.120:0/3548708783 learned_addr learned my addr 172.21.15.120:0/3548708783 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:18.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.982+0000 7fd22dd9b640 1 --2- 172.21.15.120:0/3548708783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9fe77f90 0x558d9fe82310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:18.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.982+0000 7fd22ed9d640 1 -- 172.21.15.120:0/3548708783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9fe77670 msgr2=0x558d9fe77a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:18.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.982+0000 7fd22ed9d640 1 --2- 172.21.15.120:0/3548708783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9fe77670 0x558d9fe77a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.982+0000 7fd22ed9d640 1 -- 172.21.15.120:0/3548708783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9fe77f90 msgr2=0x558d9fe82310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:18.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.982+0000 7fd22ed9d640 1 --2- 172.21.15.120:0/3548708783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9fe77f90 0x558d9fe82310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.983+0000 7fd22ed9d640 1 -- 172.21.15.120:0/3548708783 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d9fd69440 con 0x558d9fe82850 2024-09-19T09:26:18.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.983+0000 7fd22ed9d640 1 --2- 172.21.15.120:0/3548708783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9fe82850 0x558d9fe84c40 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7fd228009f90 tx=0x7fd22800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f523d7dbb3d5c72f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:18.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.983+0000 7fd21cff9640 1 -- 172.21.15.120:0/3548708783 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd228003b70 con 0x558d9fe82850 2024-09-19T09:26:18.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.983+0000 7fd21cff9640 1 -- 172.21.15.120:0/3548708783 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd228003d10 con 0x558d9fe82850 2024-09-19T09:26:18.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.984+0000 7fd21cff9640 1 -- 172.21.15.120:0/3548708783 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd228011a60 con 0x558d9fe82850 2024-09-19T09:26:18.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.984+0000 7fd236de6740 1 -- 172.21.15.120:0/3548708783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9fe82850 msgr2=0x558d9fe84c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:18.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.984+0000 7fd236de6740 1 --2- 172.21.15.120:0/3548708783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9fe82850 0x558d9fe84c40 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7fd228009f90 tx=0x7fd22800f7f0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.984+0000 7fd236de6740 1 -- 172.21.15.120:0/3548708783 shutdown_connections 2024-09-19T09:26:18.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.984+0000 7fd236de6740 1 --2- 172.21.15.120:0/3548708783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9fe82850 0x558d9fe84c40 unknown :-1 s=CLOSED pgs=2371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.984+0000 7fd236de6740 1 --2- 172.21.15.120:0/3548708783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9fe77f90 0x558d9fe82310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.984+0000 7fd236de6740 1 --2- 172.21.15.120:0/3548708783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9fe77670 0x558d9fe77a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.984+0000 7fd236de6740 1 -- 172.21.15.120:0/3548708783 >> 172.21.15.120:0/3548708783 conn(0x558d9fe73c00 msgr2=0x558d9fe73fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:18.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.984+0000 7fd236de6740 1 -- 172.21.15.120:0/3548708783 shutdown_connections 2024-09-19T09:26:18.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.984+0000 7fd236de6740 1 -- 172.21.15.120:0/3548708783 wait complete. 2024-09-19T09:26:18.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.984+0000 7fd236de6740 1 -- start start 2024-09-19T09:26:18.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.985+0000 7fd236de6740 1 Processor -- start 2024-09-19T09:26:18.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.985+0000 7fd236de6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9fe77670 0x558d9fe820c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:18.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.985+0000 7fd236de6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9fe77f90 0x558d9fe7f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:18.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.985+0000 7fd236de6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9fe82850 0x558d9fe7f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:18.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.985+0000 7fd236de6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558d9fe854c0 con 0x558d9fe77f90 2024-09-19T09:26:18.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.985+0000 7fd236de6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558d9fd98050 con 0x558d9fe82850 2024-09-19T09:26:18.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.985+0000 7fd236de6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558d9fe85640 con 0x558d9fe77670 2024-09-19T09:26:18.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.985+0000 7fd236de6740 10 client.0 _ll_register_callbacks cb 0x558d9fdd8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:18.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.985+0000 7fd22e59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9fe77670 0x558d9fe820c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:18.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.985+0000 7fd22e59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9fe77670 0x558d9fe820c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:33830/0 (socket says 172.21.15.120:33830) 2024-09-19T09:26:18.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.985+0000 7fd22e59c640 1 -- 172.21.15.120:0/2288835312 learned_addr learned my addr 172.21.15.120:0/2288835312 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:18.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.985+0000 7fd22dd9b640 1 --2- 172.21.15.120:0/2288835312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9fe77f90 0x558d9fe7f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:18.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.985+0000 7fd22ed9d640 1 --2- 172.21.15.120:0/2288835312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9fe82850 0x558d9fe7f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:18.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.985+0000 7fd22e59c640 1 -- 172.21.15.120:0/2288835312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9fe82850 msgr2=0x558d9fe7f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:18.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.985+0000 7fd22e59c640 1 --2- 172.21.15.120:0/2288835312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9fe82850 0x558d9fe7f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.985+0000 7fd22e59c640 1 -- 172.21.15.120:0/2288835312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9fe77f90 msgr2=0x558d9fe7f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:18.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.985+0000 7fd22e59c640 1 --2- 172.21.15.120:0/2288835312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9fe77f90 0x558d9fe7f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.985+0000 7fd22e59c640 1 -- 172.21.15.120:0/2288835312 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d9fb4e3c0 con 0x558d9fe77670 2024-09-19T09:26:18.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.986+0000 7fd22ed9d640 1 --2- 172.21.15.120:0/2288835312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9fe82850 0x558d9fe7f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:18.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.986+0000 7fd22e59c640 1 --2- 172.21.15.120:0/2288835312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9fe77670 0x558d9fe820c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd2200097d0 tx=0x7fd22009ca80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b8e5060ea458f04 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:18.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.986+0000 7fd215ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:26:18.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.986+0000 7fd215ffb640 1 -- 172.21.15.120:0/2288835312 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2200b4030 con 0x558d9fe77670 2024-09-19T09:26:18.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.986+0000 7fd236de6740 1 -- 172.21.15.120:0/2288835312 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd228003d10 con 0x558d9fe77670 2024-09-19T09:26:18.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.986+0000 7fd215ffb640 1 -- 172.21.15.120:0/2288835312 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd2200b5270 con 0x558d9fe77670 2024-09-19T09:26:18.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.986+0000 7fd215ffb640 1 -- 172.21.15.120:0/2288835312 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd2200a4ff0 con 0x558d9fe77670 2024-09-19T09:26:18.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.986+0000 7fd236de6740 10 client.31510 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:18.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.986+0000 7fd236de6740 1 -- 172.21.15.120:0/2288835312 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558d9fe90eb0 con 0x558d9fe77670 2024-09-19T09:26:18.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.986+0000 7fd215ffb640 1 -- 172.21.15.120:0/2288835312 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd228003d10 con 0x558d9fe77670 2024-09-19T09:26:18.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.987+0000 7fd215ffb640 1 -- 172.21.15.120:0/2288835312 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fd22009b240 con 0x558d9fe77670 2024-09-19T09:26:18.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.987+0000 7fd236de6740 10 client.31510 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:18.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.987+0000 7fd236de6740 10 client.31510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:26:18.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.987+0000 7fd236de6740 20 client.31510 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:18.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.987+0000 7fd236de6740 1 --2- 172.21.15.120:0/2288835312 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558d9fe922c0 0x558d9fe946b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:18.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.987+0000 7fd236de6740 4 client.31510 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:26:18.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.987+0000 7fd236de6740 1 -- 172.21.15.120:0/2288835312 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d9fdea850 con 0x558d9fe922c0 2024-09-19T09:26:18.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.991+0000 7fd22dd9b640 1 --2- 172.21.15.120:0/2288835312 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558d9fe922c0 0x558d9fe946b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:18.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.994+0000 7fd22dd9b640 1 --2- 172.21.15.120:0/2288835312 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558d9fe922c0 0x558d9fe946b0 crc :-1 s=READY pgs=571 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d616c5e789e60aa server_cookie=c341b5b990ffcce5 in_seq=0 out_seq=0 2024-09-19T09:26:18.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.994+0000 7fd215ffb640 10 client.31510 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:18.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.996+0000 7fd215ffb640 1 -- 172.21.15.120:0/2288835312 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558d9fdea850 con 0x558d9fe922c0 2024-09-19T09:26:18.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.996+0000 7fd215ffb640 10 client.31510 handle_command_reply: tid=0 2024-09-19T09:26:18.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.996+0000 7fd236de6740 1 client.31510 shutdown 2024-09-19T09:26:18.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.996+0000 7fd236de6740 1 -- 172.21.15.120:0/2288835312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9fe77670 msgr2=0x558d9fe820c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:18.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.996+0000 7fd236de6740 1 --2- 172.21.15.120:0/2288835312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9fe77670 0x558d9fe820c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd2200097d0 tx=0x7fd22009ca80 comp rx=0 tx=0).stop 2024-09-19T09:26:18.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.996+0000 7fd236de6740 1 -- 172.21.15.120:0/2288835312 shutdown_connections 2024-09-19T09:26:18.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.996+0000 7fd236de6740 1 --2- 172.21.15.120:0/2288835312 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558d9fe922c0 0x558d9fe946b0 crc :-1 s=READY pgs=571 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.996+0000 7fd236de6740 1 --2- 172.21.15.120:0/2288835312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9fe82850 0x558d9fe7f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.996+0000 7fd215ffb640 0 client.31510 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:18.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.996+0000 7fd236de6740 1 --2- 172.21.15.120:0/2288835312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9fe77f90 0x558d9fe7f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.996+0000 7fd236de6740 1 --2- 172.21.15.120:0/2288835312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9fe77670 0x558d9fe820c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.996+0000 7fd236de6740 1 -- 172.21.15.120:0/2288835312 >> 172.21.15.120:0/2288835312 conn(0x558d9fe73c00 msgr2=0x558d9fe798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:18.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.996+0000 7fd236de6740 1 -- 172.21.15.120:0/2288835312 shutdown_connections 2024-09-19T09:26:18.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.996+0000 7fd236de6740 1 -- 172.21.15.120:0/2288835312 wait complete. 2024-09-19T09:26:18.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:17.996+0000 7fd236de6740 20 client.31510 trim_cache size 0 max 16384 2024-09-19T09:26:18.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.016+0000 7fd236de6740 1 Processor -- start 2024-09-19T09:26:18.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.016+0000 7fd236de6740 1 -- start start 2024-09-19T09:26:18.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.017+0000 7fd236de6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9fe77670 0x558d9fe72c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:18.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.017+0000 7fd236de6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9fe77f90 0x558d9fe99f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:18.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.017+0000 7fd236de6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9fe82850 0x558d9fe9a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:18.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.017+0000 7fd236de6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd2301ad4d0 con 0x558d9fe82850 2024-09-19T09:26:18.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.017+0000 7fd236de6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd2301af010 con 0x558d9fe77670 2024-09-19T09:26:18.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.017+0000 7fd236de6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd2301adba0 con 0x558d9fe77f90 2024-09-19T09:26:18.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.017+0000 7fd22ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9fe82850 0x558d9fe9a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:18.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.017+0000 7fd22e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9fe77670 0x558d9fe72c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:18.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.017+0000 7fd22dd9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9fe77f90 0x558d9fe99f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:18.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.017+0000 7fd22ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9fe82850 0x558d9fe9a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51058/0 (socket says 172.21.15.120:51058) 2024-09-19T09:26:18.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.017+0000 7fd22ed9d640 1 -- 172.21.15.120:0/2380084991 learned_addr learned my addr 172.21.15.120:0/2380084991 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:18.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.017+0000 7fd22dd9b640 1 -- 172.21.15.120:0/2380084991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9fe77670 msgr2=0x558d9fe72c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:18.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.017+0000 7fd22dd9b640 1 --2- 172.21.15.120:0/2380084991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9fe77670 0x558d9fe72c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.017+0000 7fd22dd9b640 1 -- 172.21.15.120:0/2380084991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9fe82850 msgr2=0x558d9fe9a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:18.281 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:26:18.281 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:26:18.281 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:26:18.281 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:26:18.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.017+0000 7fd22dd9b640 1 --2- 172.21.15.120:0/2380084991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9fe82850 0x558d9fe9a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.017+0000 7fd22dd9b640 1 -- 172.21.15.120:0/2380084991 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d9fe74d90 con 0x558d9fe77f90 2024-09-19T09:26:18.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.018+0000 7fd22dd9b640 1 --2- 172.21.15.120:0/2380084991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9fe77f90 0x558d9fe99f50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd22402a940 tx=0x7fd224040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b092bb6da055f876 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:18.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.018+0000 7fd217fff640 1 -- 172.21.15.120:0/2380084991 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd224035040 con 0x558d9fe77f90 2024-09-19T09:26:18.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.018+0000 7fd217fff640 1 -- 172.21.15.120:0/2380084991 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2240325c0 con 0x558d9fe77f90 2024-09-19T09:26:18.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.018+0000 7fd217fff640 1 -- 172.21.15.120:0/2380084991 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd224003870 con 0x558d9fe77f90 2024-09-19T09:26:18.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.018+0000 7fd236de6740 1 -- 172.21.15.120:0/2380084991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9fe77f90 msgr2=0x558d9fe99f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:18.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.018+0000 7fd236de6740 1 --2- 172.21.15.120:0/2380084991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9fe77f90 0x558d9fe99f50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd22402a940 tx=0x7fd224040000 comp rx=0 tx=0).stop 2024-09-19T09:26:18.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.019+0000 7fd236de6740 1 -- 172.21.15.120:0/2380084991 shutdown_connections 2024-09-19T09:26:18.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.019+0000 7fd236de6740 1 --2- 172.21.15.120:0/2380084991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9fe82850 0x558d9fe9a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.019+0000 7fd236de6740 1 --2- 172.21.15.120:0/2380084991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9fe77f90 0x558d9fe99f50 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.019+0000 7fd236de6740 1 --2- 172.21.15.120:0/2380084991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9fe77670 0x558d9fe72c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.019+0000 7fd236de6740 1 -- 172.21.15.120:0/2380084991 >> 172.21.15.120:0/2380084991 conn(0x558d9fe73c00 msgr2=0x558d9fe72500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:18.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.019+0000 7fd236de6740 1 -- 172.21.15.120:0/2380084991 shutdown_connections 2024-09-19T09:26:18.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.019+0000 7fd236de6740 1 -- 172.21.15.120:0/2380084991 wait complete. 2024-09-19T09:26:18.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.019+0000 7fd236de6740 1 -- start start 2024-09-19T09:26:18.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.019+0000 7fd236de6740 1 Processor -- start 2024-09-19T09:26:18.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.019+0000 7fd236de6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9fe77670 0x558d9fe8ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:18.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.019+0000 7fd236de6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9fe77f90 0x558d9fe8b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:18.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.020+0000 7fd236de6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9fe82850 0x558d9fe87ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:18.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.020+0000 7fd236de6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd2301ae940 con 0x558d9fe77670 2024-09-19T09:26:18.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.020+0000 7fd236de6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd2301ae270 con 0x558d9fe82850 2024-09-19T09:26:18.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.020+0000 7fd236de6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd2301ab490 con 0x558d9fe77f90 2024-09-19T09:26:18.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.020+0000 7fd22dd9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9fe77f90 0x558d9fe8b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:18.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.020+0000 7fd22dd9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9fe77f90 0x558d9fe8b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:33850/0 (socket says 172.21.15.120:33850) 2024-09-19T09:26:18.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.020+0000 7fd22dd9b640 1 -- 172.21.15.120:0/4075157854 learned_addr learned my addr 172.21.15.120:0/4075157854 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:18.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.020+0000 7fd22e59c640 1 --2- 172.21.15.120:0/4075157854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9fe77670 0x558d9fe8ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:18.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.020+0000 7fd236de6740 10 client.0 _ll_register_callbacks cb 0x7fd220098660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:18.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.020+0000 7fd22dd9b640 1 -- 172.21.15.120:0/4075157854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9fe82850 msgr2=0x558d9fe87ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:18.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.020+0000 7fd22dd9b640 1 --2- 172.21.15.120:0/4075157854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9fe82850 0x558d9fe87ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.020+0000 7fd22dd9b640 1 -- 172.21.15.120:0/4075157854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9fe77670 msgr2=0x558d9fe8ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:18.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.020+0000 7fd22dd9b640 1 --2- 172.21.15.120:0/4075157854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9fe77670 0x558d9fe8ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.020+0000 7fd22dd9b640 1 -- 172.21.15.120:0/4075157854 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d9fe882e0 con 0x558d9fe77f90 2024-09-19T09:26:18.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.020+0000 7fd22e59c640 1 --2- 172.21.15.120:0/4075157854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9fe77670 0x558d9fe8ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:18.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.020+0000 7fd22dd9b640 1 --2- 172.21.15.120:0/4075157854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9fe77f90 0x558d9fe8b1d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd2240057c0 tx=0x7fd224009880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e350d3501182c06e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:18.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.020+0000 7fd215ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:26:18.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.020+0000 7fd215ffb640 1 -- 172.21.15.120:0/4075157854 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd224062060 con 0x558d9fe77f90 2024-09-19T09:26:18.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.021+0000 7fd215ffb640 1 -- 172.21.15.120:0/4075157854 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd224032970 con 0x558d9fe77f90 2024-09-19T09:26:18.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.021+0000 7fd236de6740 1 -- 172.21.15.120:0/4075157854 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2240325c0 con 0x558d9fe77f90 2024-09-19T09:26:18.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.022+0000 7fd215ffb640 1 -- 172.21.15.120:0/4075157854 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd224035070 con 0x558d9fe77f90 2024-09-19T09:26:18.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.022+0000 7fd236de6740 10 client.31516 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:18.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.022+0000 7fd236de6740 1 -- 172.21.15.120:0/4075157854 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558d9fefdc10 con 0x558d9fe77f90 2024-09-19T09:26:18.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.022+0000 7fd215ffb640 1 -- 172.21.15.120:0/4075157854 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd224004c60 con 0x558d9fe77f90 2024-09-19T09:26:18.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.023+0000 7fd215ffb640 1 -- 172.21.15.120:0/4075157854 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fd224067070 con 0x558d9fe77f90 2024-09-19T09:26:18.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.023+0000 7fd236de6740 10 client.31516 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:18.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.023+0000 7fd236de6740 10 client.31516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:26:18.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.023+0000 7fd236de6740 20 client.31516 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:18.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.023+0000 7fd236de6740 1 --2- 172.21.15.120:0/4075157854 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558d9fe922c0 0x558d9ff00370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:18.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.023+0000 7fd236de6740 4 client.31516 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:26:18.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.023+0000 7fd236de6740 1 -- 172.21.15.120:0/4075157854 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fd2240325c0 con 0x558d9fe922c0 2024-09-19T09:26:18.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.023+0000 7fd22e59c640 1 --2- 172.21.15.120:0/4075157854 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558d9fe922c0 0x558d9ff00370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:18.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.024+0000 7fd22e59c640 1 --2- 172.21.15.120:0/4075157854 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558d9fe922c0 0x558d9ff00370 crc :-1 s=READY pgs=572 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd9d65e29aa6106e server_cookie=97860462e0fe5496 in_seq=0 out_seq=0 2024-09-19T09:26:18.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.024+0000 7fd215ffb640 10 client.31516 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:18.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.024+0000 7fd215ffb640 1 -- 172.21.15.120:0/4075157854 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fd2240325c0 con 0x558d9fe922c0 2024-09-19T09:26:18.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.024+0000 7fd215ffb640 10 client.31516 handle_command_reply: tid=0 2024-09-19T09:26:18.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.024+0000 7fd236de6740 1 client.31516 shutdown 2024-09-19T09:26:18.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.024+0000 7fd236de6740 1 -- 172.21.15.120:0/4075157854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9fe77f90 msgr2=0x558d9fe8b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:18.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.024+0000 7fd236de6740 1 --2- 172.21.15.120:0/4075157854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9fe77f90 0x558d9fe8b1d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd2240057c0 tx=0x7fd224009880 comp rx=0 tx=0).stop 2024-09-19T09:26:18.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.025+0000 7fd236de6740 1 -- 172.21.15.120:0/4075157854 shutdown_connections 2024-09-19T09:26:18.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.025+0000 7fd236de6740 1 --2- 172.21.15.120:0/4075157854 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x558d9fe922c0 0x558d9ff00370 crc :-1 s=READY pgs=572 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.025+0000 7fd236de6740 1 --2- 172.21.15.120:0/4075157854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d9fe82850 0x558d9fe87ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.025+0000 7fd236de6740 1 --2- 172.21.15.120:0/4075157854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d9fe77f90 0x558d9fe8b1d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.025+0000 7fd236de6740 1 --2- 172.21.15.120:0/4075157854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558d9fe77670 0x558d9fe8ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.025+0000 7fd236de6740 1 -- 172.21.15.120:0/4075157854 >> 172.21.15.120:0/4075157854 conn(0x558d9fe73c00 msgr2=0x558d9fe798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:18.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.025+0000 7fd215ffb640 0 client.31516 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:18.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.025+0000 7fd236de6740 1 -- 172.21.15.120:0/4075157854 shutdown_connections 2024-09-19T09:26:18.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.025+0000 7fd236de6740 1 -- 172.21.15.120:0/4075157854 wait complete. 2024-09-19T09:26:18.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.025+0000 7fd236de6740 20 client.31516 trim_cache size 0 max 16384 2024-09-19T09:26:18.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.026+0000 7fd22f59e640 1 -- 172.21.15.120:0/4180172493 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd2180a3240 msgr2=0x7fd2180a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:18.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.026+0000 7fd22f59e640 1 --2- 172.21.15.120:0/4180172493 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd2180a3240 0x7fd2180a5700 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd224006010 tx=0x7fd224000bc0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.026+0000 7fd22f59e640 1 -- 172.21.15.120:0/4180172493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2301008d0 msgr2=0x7fd230199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:18.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.026+0000 7fd22f59e640 1 --2- 172.21.15.120:0/4180172493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2301008d0 0x7fd230199130 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7fd22000e9f0 tx=0x7fd22000edf0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.026+0000 7fd22f59e640 1 -- 172.21.15.120:0/4180172493 shutdown_connections 2024-09-19T09:26:18.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.026+0000 7fd22f59e640 1 --2- 172.21.15.120:0/4180172493 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd2180a3240 0x7fd2180a5700 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.026+0000 7fd22f59e640 1 --2- 172.21.15.120:0/4180172493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2301068d0 0x7fd2301a41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.026+0000 7fd22f59e640 1 --2- 172.21.15.120:0/4180172493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd230101270 0x7fd230199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.026+0000 7fd22f59e640 1 --2- 172.21.15.120:0/4180172493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2301008d0 0x7fd230199130 unknown :-1 s=CLOSED pgs=2370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.026+0000 7fd22f59e640 1 -- 172.21.15.120:0/4180172493 >> 172.21.15.120:0/4180172493 conn(0x7fd2300fc610 msgr2=0x7fd2300fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:18.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.027+0000 7fd22f59e640 1 -- 172.21.15.120:0/4180172493 shutdown_connections 2024-09-19T09:26:18.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:18.027+0000 7fd22f59e640 1 -- 172.21.15.120:0/4180172493 wait complete. 2024-09-19T09:26:18.385 INFO:tasks.workunit.client.0.smithi120.stdout: 10 245653 20.78 MB/sec execute 493 sec latency 258.024 ms 2024-09-19T09:26:18.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:18 smithi196 ceph-mon[35150]: pgmap v1048: 97 pgs: 97 active+clean; 517 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.1 MiB/s wr, 649 op/s 2024-09-19T09:26:18.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:18 smithi153 ceph-mon[35071]: pgmap v1048: 97 pgs: 97 active+clean; 517 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.1 MiB/s wr, 649 op/s 2024-09-19T09:26:18.439 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:18 smithi120 ceph-mon[31150]: pgmap v1048: 97 pgs: 97 active+clean; 517 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.1 MiB/s wr, 649 op/s 2024-09-19T09:26:19.169 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:26:19.170 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:26:19.385 INFO:tasks.workunit.client.0.smithi120.stdout: 10 246203 20.80 MB/sec execute 494 sec latency 58.092 ms 2024-09-19T09:26:19.502 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:20.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.025+0000 7fadf8d3b640 1 -- 172.21.15.120:0/3523871275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadf4102d90 msgr2=0x7fadf4103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:20.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.025+0000 7fadf8d3b640 1 --2- 172.21.15.120:0/3523871275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadf4102d90 0x7fadf4103170 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7fade4009930 tx=0x7fade402a420 comp rx=0 tx=0).stop 2024-09-19T09:26:20.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.025+0000 7fadf8d3b640 1 -- 172.21.15.120:0/3523871275 shutdown_connections 2024-09-19T09:26:20.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.025+0000 7fadf8d3b640 1 --2- 172.21.15.120:0/3523871275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadf4108290 0x7fadf410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:20.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.025+0000 7fadf8d3b640 1 --2- 172.21.15.120:0/3523871275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadf4103740 0x7fadf4107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:20.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.025+0000 7fadf8d3b640 1 --2- 172.21.15.120:0/3523871275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadf4102d90 0x7fadf4103170 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:20.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.025+0000 7fadf8d3b640 1 -- 172.21.15.120:0/3523871275 >> 172.21.15.120:0/3523871275 conn(0x7fadf40fc7d0 msgr2=0x7fadf40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:20.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.025+0000 7fadf8d3b640 1 -- 172.21.15.120:0/3523871275 shutdown_connections 2024-09-19T09:26:20.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.025+0000 7fadf8d3b640 1 -- 172.21.15.120:0/3523871275 wait complete. 2024-09-19T09:26:20.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.026+0000 7fadf8d3b640 1 Processor -- start 2024-09-19T09:26:20.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.026+0000 7fadf8d3b640 1 -- start start 2024-09-19T09:26:20.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.026+0000 7fadf8d3b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadf4102d90 0x7fadf4198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:20.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.026+0000 7fadf8d3b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadf4103740 0x7fadf4198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:20.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.026+0000 7fadf8d3b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadf4108290 0x7fadf419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:20.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.026+0000 7fadf8d3b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fadf4112090 con 0x7fadf4108290 2024-09-19T09:26:20.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.026+0000 7fadf8d3b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fadf4111f10 con 0x7fadf4102d90 2024-09-19T09:26:20.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.026+0000 7fadf8d3b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fadf4112210 con 0x7fadf4103740 2024-09-19T09:26:20.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.026+0000 7fadf2d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadf4108290 0x7fadf419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:20.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.026+0000 7fadf2d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadf4108290 0x7fadf419ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51086/0 (socket says 172.21.15.120:51086) 2024-09-19T09:26:20.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.026+0000 7fadf259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadf4102d90 0x7fadf4198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:20.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.026+0000 7fadf2d9d640 1 -- 172.21.15.120:0/3883807924 learned_addr learned my addr 172.21.15.120:0/3883807924 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:20.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.026+0000 7fadf1d9b640 1 --2- 172.21.15.120:0/3883807924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadf4103740 0x7fadf4198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:20.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.027+0000 7fadf2d9d640 1 -- 172.21.15.120:0/3883807924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadf4103740 msgr2=0x7fadf4198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:20.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.027+0000 7fadf2d9d640 1 --2- 172.21.15.120:0/3883807924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadf4103740 0x7fadf4198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:20.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.027+0000 7fadf2d9d640 1 -- 172.21.15.120:0/3883807924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadf4102d90 msgr2=0x7fadf4198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:20.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.027+0000 7fadf2d9d640 1 --2- 172.21.15.120:0/3883807924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadf4102d90 0x7fadf4198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:20.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.027+0000 7fadf2d9d640 1 -- 172.21.15.120:0/3883807924 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf419d540 con 0x7fadf4108290 2024-09-19T09:26:20.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.027+0000 7fadf259c640 1 --2- 172.21.15.120:0/3883807924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadf4102d90 0x7fadf4198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:20.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.027+0000 7fadf2d9d640 1 --2- 172.21.15.120:0/3883807924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadf4108290 0x7fadf419ce60 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7fadec00eb10 tx=0x7fadec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:20.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.027+0000 7fade37fe640 1 -- 172.21.15.120:0/3883807924 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec004b20 con 0x7fadf4108290 2024-09-19T09:26:20.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.027+0000 7fade37fe640 1 -- 172.21.15.120:0/3883807924 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadec004cc0 con 0x7fadf4108290 2024-09-19T09:26:20.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.027+0000 7fade37fe640 1 -- 172.21.15.120:0/3883807924 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec010760 con 0x7fadf4108290 2024-09-19T09:26:20.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.027+0000 7fadf8d3b640 1 -- 172.21.15.120:0/3883807924 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf419d830 con 0x7fadf4108290 2024-09-19T09:26:20.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.027+0000 7fadf8d3b640 1 -- 172.21.15.120:0/3883807924 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf41a5110 con 0x7fadf4108290 2024-09-19T09:26:20.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.029+0000 7fadf8d3b640 1 -- 172.21.15.120:0/3883807924 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadf406a7f0 con 0x7fadf4108290 2024-09-19T09:26:20.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.030+0000 7fade37fe640 1 -- 172.21.15.120:0/3883807924 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fadec005020 con 0x7fadf4108290 2024-09-19T09:26:20.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.030+0000 7fade37fe640 1 --2- 172.21.15.120:0/3883807924 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faddc0a3240 0x7faddc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:20.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.030+0000 7fadf259c640 1 --2- 172.21.15.120:0/3883807924 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faddc0a3240 0x7faddc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:20.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.030+0000 7fade37fe640 1 -- 172.21.15.120:0/3883807924 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fadec0cff50 con 0x7fadf4108290 2024-09-19T09:26:20.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.031+0000 7fadf259c640 1 --2- 172.21.15.120:0/3883807924 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faddc0a3240 0x7faddc0a5700 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fade4009900 tx=0x7fade40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:20.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.035+0000 7fade37fe640 1 -- 172.21.15.120:0/3883807924 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fadec09cc70 con 0x7fadf4108290 2024-09-19T09:26:20.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.280+0000 7fadf8d3b640 1 -- 172.21.15.120:0/3883807924 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fadf410a2a0 con 0x7fadf4108290 2024-09-19T09:26:20.283 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:20 smithi120 ceph-mon[31150]: pgmap v1049: 97 pgs: 97 active+clean; 515 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.4 MiB/s wr, 656 op/s 2024-09-19T09:26:20.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.282+0000 7fade37fe640 1 -- 172.21.15.120:0/3883807924 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11988 (secure 0 0 0) 0x7fadec0bc030 con 0x7fadf4108290 2024-09-19T09:26:20.284 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:26:20.284 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":39,"btime":"2024-09-19T09:18:53:120858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:18:53.070639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:26:20.285 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 39 2024-09-19T09:26:20.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.285+0000 7fadf8d3b640 1 -- 172.21.15.120:0/3883807924 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faddc0a3240 msgr2=0x7faddc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:20.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.285+0000 7fadf8d3b640 1 --2- 172.21.15.120:0/3883807924 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faddc0a3240 0x7faddc0a5700 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fade4009900 tx=0x7fade40058d0 comp rx=0 tx=0).stop 2024-09-19T09:26:20.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.285+0000 7fadf8d3b640 1 -- 172.21.15.120:0/3883807924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadf4108290 msgr2=0x7fadf419ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:20.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.285+0000 7fadf8d3b640 1 --2- 172.21.15.120:0/3883807924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadf4108290 0x7fadf419ce60 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7fadec00eb10 tx=0x7fadec00c2d0 comp rx=0 tx=0).stop 2024-09-19T09:26:20.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.285+0000 7fadf8d3b640 1 -- 172.21.15.120:0/3883807924 shutdown_connections 2024-09-19T09:26:20.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.285+0000 7fadf8d3b640 1 --2- 172.21.15.120:0/3883807924 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7faddc0a3240 0x7faddc0a5700 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:20.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.285+0000 7fadf8d3b640 1 --2- 172.21.15.120:0/3883807924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadf4108290 0x7fadf419ce60 unknown :-1 s=CLOSED pgs=2373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:20.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.285+0000 7fadf8d3b640 1 --2- 172.21.15.120:0/3883807924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadf4103740 0x7fadf4198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:20.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.285+0000 7fadf8d3b640 1 --2- 172.21.15.120:0/3883807924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadf4102d90 0x7fadf4198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:20.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.285+0000 7fadf8d3b640 1 -- 172.21.15.120:0/3883807924 >> 172.21.15.120:0/3883807924 conn(0x7fadf40fc7d0 msgr2=0x7fadf4105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:20.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.285+0000 7fadf8d3b640 1 -- 172.21.15.120:0/3883807924 shutdown_connections 2024-09-19T09:26:20.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:20.285+0000 7fadf8d3b640 1 -- 172.21.15.120:0/3883807924 wait complete. 2024-09-19T09:26:20.385 INFO:tasks.workunit.client.0.smithi120.stdout: 10 246581 20.82 MB/sec execute 495 sec latency 379.756 ms 2024-09-19T09:26:20.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:20 smithi196 ceph-mon[35150]: pgmap v1049: 97 pgs: 97 active+clean; 515 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.4 MiB/s wr, 656 op/s 2024-09-19T09:26:20.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:20 smithi153 ceph-mon[35071]: pgmap v1049: 97 pgs: 97 active+clean; 515 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.4 MiB/s wr, 656 op/s 2024-09-19T09:26:21.145 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:26:21.385 INFO:tasks.workunit.client.0.smithi120.stdout: 10 247092 20.83 MB/sec execute 496 sec latency 88.848 ms 2024-09-19T09:26:21.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:21 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3883807924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:26:21.438 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:21.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:21 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3883807924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:26:21.559 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:21 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3883807924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:26:22.220 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:22 smithi120 ceph-mon[31150]: pgmap v1050: 97 pgs: 97 active+clean; 503 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.9 MiB/s wr, 729 op/s 2024-09-19T09:26:22.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.315+0000 7f89c7293640 1 -- 172.21.15.120:0/3306525512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89c0106f10 msgr2=0x7f89c010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:22.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.315+0000 7f89c7293640 1 --2- 172.21.15.120:0/3306525512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89c0106f10 0x7f89c010ea90 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f89b8009ca0 tx=0x7f89b802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.315+0000 7f89c7293640 1 -- 172.21.15.120:0/3306525512 shutdown_connections 2024-09-19T09:26:22.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.315+0000 7f89c7293640 1 --2- 172.21.15.120:0/3306525512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89c0106f10 0x7f89c010ea90 unknown :-1 s=CLOSED pgs=2374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.315+0000 7f89c7293640 1 --2- 172.21.15.120:0/3306525512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89c0102790 0x7f89c01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.315+0000 7f89c7293640 1 --2- 172.21.15.120:0/3306525512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89c0101de0 0x7f89c01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.315+0000 7f89c7293640 1 -- 172.21.15.120:0/3306525512 >> 172.21.15.120:0/3306525512 conn(0x7f89c00fd880 msgr2=0x7f89c00ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:22.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.315+0000 7f89c7293640 1 -- 172.21.15.120:0/3306525512 shutdown_connections 2024-09-19T09:26:22.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.315+0000 7f89c7293640 1 -- 172.21.15.120:0/3306525512 wait complete. 2024-09-19T09:26:22.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.315+0000 7f89c7293640 1 Processor -- start 2024-09-19T09:26:22.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.316+0000 7f89c7293640 1 -- start start 2024-09-19T09:26:22.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.316+0000 7f89c7293640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89c0101de0 0x7f89c01983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:22.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.316+0000 7f89c7293640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89c0102790 0x7f89c0198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:22.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.316+0000 7f89c7293640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89c0106f10 0x7f89c019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:22.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.316+0000 7f89c7293640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f89c0111f70 con 0x7f89c0106f10 2024-09-19T09:26:22.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.316+0000 7f89c7293640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f89c0111df0 con 0x7f89c0102790 2024-09-19T09:26:22.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.316+0000 7f89c7293640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01120f0 con 0x7f89c0101de0 2024-09-19T09:26:22.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.316+0000 7f89c5830640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89c0106f10 0x7f89c019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:22.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.316+0000 7f89c502f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89c0101de0 0x7f89c01983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:22.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.316+0000 7f89c5830640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89c0106f10 0x7f89c019cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56078/0 (socket says 172.21.15.120:56078) 2024-09-19T09:26:22.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.316+0000 7f89c482e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89c0102790 0x7f89c0198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:22.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.316+0000 7f89c5830640 1 -- 172.21.15.120:0/3718856974 learned_addr learned my addr 172.21.15.120:0/3718856974 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:22.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.316+0000 7f89c5830640 1 -- 172.21.15.120:0/3718856974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89c0101de0 msgr2=0x7f89c01983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:22.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.316+0000 7f89c5830640 1 --2- 172.21.15.120:0/3718856974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89c0101de0 0x7f89c01983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.316+0000 7f89c5830640 1 -- 172.21.15.120:0/3718856974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89c0102790 msgr2=0x7f89c0198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:22.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.316+0000 7f89c5830640 1 --2- 172.21.15.120:0/3718856974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89c0102790 0x7f89c0198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.316+0000 7f89c5830640 1 -- 172.21.15.120:0/3718856974 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c019d380 con 0x7f89c0106f10 2024-09-19T09:26:22.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.316+0000 7f89c5830640 1 --2- 172.21.15.120:0/3718856974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89c0106f10 0x7f89c019cca0 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f89b8031a20 tx=0x7f89b8031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:22.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.317+0000 7f89b67fc640 1 -- 172.21.15.120:0/3718856974 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8003c30 con 0x7f89c0106f10 2024-09-19T09:26:22.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.317+0000 7f89c7293640 1 -- 172.21.15.120:0/3718856974 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89c019d610 con 0x7f89c0106f10 2024-09-19T09:26:22.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.317+0000 7f89b67fc640 1 -- 172.21.15.120:0/3718856974 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89b8003dd0 con 0x7f89c0106f10 2024-09-19T09:26:22.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.317+0000 7f89b67fc640 1 -- 172.21.15.120:0/3718856974 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b802fcf0 con 0x7f89c0106f10 2024-09-19T09:26:22.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.317+0000 7f89c7293640 1 -- 172.21.15.120:0/3718856974 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89c0074420 con 0x7f89c0106f10 2024-09-19T09:26:22.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.319+0000 7f89b67fc640 1 -- 172.21.15.120:0/3718856974 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f89b8004050 con 0x7f89c0106f10 2024-09-19T09:26:22.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.320+0000 7f89c966c740 1 Processor -- start 2024-09-19T09:26:22.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.320+0000 7f89c966c740 1 -- start start 2024-09-19T09:26:22.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.320+0000 7f89c966c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc0e5cb40 0x55abc0e5b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:22.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.320+0000 7f89c966c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc0e5b5d0 0x55abc0e5ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:22.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.320+0000 7f89c966c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc0e66370 0x55abc0e68760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:22.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.320+0000 7f89c966c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55abc0dc12f0 con 0x55abc0e66370 2024-09-19T09:26:22.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.320+0000 7f89c966c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55abc0d68cf0 con 0x55abc0e5b5d0 2024-09-19T09:26:22.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.320+0000 7f89c966c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55abc0dbd2b0 con 0x55abc0e5cb40 2024-09-19T09:26:22.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.320+0000 7f89b67fc640 1 --2- 172.21.15.120:0/3718856974 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f89ac0a32c0 0x7f89ac0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:22.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.321+0000 7f89c482e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc0e5cb40 0x55abc0e5b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:22.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.321+0000 7f89c482e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc0e5cb40 0x55abc0e5b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36698/0 (socket says 172.21.15.120:36698) 2024-09-19T09:26:22.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.321+0000 7f89c482e640 1 -- 172.21.15.120:0/45399241 learned_addr learned my addr 172.21.15.120:0/45399241 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:22.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.321+0000 7f89c502f640 1 --2- 172.21.15.120:0/45399241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc0e5b5d0 0x55abc0e5ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:22.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.321+0000 7f89c482e640 1 --2- 172.21.15.120:0/45399241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc0e66370 0x55abc0e68760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:22.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.321+0000 7f89c5830640 1 --2- 172.21.15.120:0/3718856974 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f89ac0a32c0 0x7f89ac0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:22.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.321+0000 7f89b67fc640 1 -- 172.21.15.120:0/3718856974 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f89b80f0df0 con 0x7f89c0106f10 2024-09-19T09:26:22.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.321+0000 7f89c482e640 1 -- 172.21.15.120:0/45399241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc0e5b5d0 msgr2=0x55abc0e5ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:22.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.321+0000 7f89c482e640 1 --2- 172.21.15.120:0/45399241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc0e5b5d0 0x55abc0e5ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.321+0000 7f89c482e640 1 -- 172.21.15.120:0/45399241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc0e66370 msgr2=0x55abc0e68760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:22.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.321+0000 7f89c482e640 1 --2- 172.21.15.120:0/45399241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc0e66370 0x55abc0e68760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.321+0000 7f89c482e640 1 -- 172.21.15.120:0/45399241 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abc0d4d3a0 con 0x55abc0e5cb40 2024-09-19T09:26:22.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.321+0000 7f89c482e640 1 --2- 172.21.15.120:0/45399241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc0e5cb40 0x55abc0e5b090 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f89b00126f0 tx=0x7f89b000b6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fbdf1221f4d2028 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:22.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.321+0000 7f89c5830640 1 --2- 172.21.15.120:0/3718856974 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f89ac0a32c0 0x7f89ac0a5780 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f89b80b9a20 tx=0x7f89b8090560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:22.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.321+0000 7f89ab7fe640 1 -- 172.21.15.120:0/45399241 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0012c70 con 0x55abc0e5cb40 2024-09-19T09:26:22.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.321+0000 7f89ab7fe640 1 -- 172.21.15.120:0/45399241 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89b0012e10 con 0x55abc0e5cb40 2024-09-19T09:26:22.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.321+0000 7f89ab7fe640 1 -- 172.21.15.120:0/45399241 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b00049e0 con 0x55abc0e5cb40 2024-09-19T09:26:22.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.322+0000 7f89c966c740 1 -- 172.21.15.120:0/45399241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc0e5cb40 msgr2=0x55abc0e5b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:22.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.322+0000 7f89c966c740 1 --2- 172.21.15.120:0/45399241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc0e5cb40 0x55abc0e5b090 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f89b00126f0 tx=0x7f89b000b6e0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.322+0000 7f89c966c740 1 -- 172.21.15.120:0/45399241 shutdown_connections 2024-09-19T09:26:22.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.322+0000 7f89c966c740 1 --2- 172.21.15.120:0/45399241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc0e66370 0x55abc0e68760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.322+0000 7f89c966c740 1 --2- 172.21.15.120:0/45399241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc0e5b5d0 0x55abc0e5ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.322+0000 7f89c966c740 1 --2- 172.21.15.120:0/45399241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc0e5cb40 0x55abc0e5b090 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.322+0000 7f89c966c740 1 -- 172.21.15.120:0/45399241 >> 172.21.15.120:0/45399241 conn(0x55abc0e24310 msgr2=0x55abc0e57c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:22.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.323+0000 7f89c966c740 1 -- 172.21.15.120:0/45399241 shutdown_connections 2024-09-19T09:26:22.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.323+0000 7f89c966c740 1 -- 172.21.15.120:0/45399241 wait complete. 2024-09-19T09:26:22.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.323+0000 7f89c966c740 1 -- start start 2024-09-19T09:26:22.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.323+0000 7f89c966c740 1 Processor -- start 2024-09-19T09:26:22.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.324+0000 7f89c966c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc0e5b5d0 0x55abc0e65de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:22.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.324+0000 7f89c966c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc0e5cb40 0x55abc0e63090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:22.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.324+0000 7f89c966c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc0e66370 0x55abc0e637d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:22.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.324+0000 7f89c966c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55abc0e69070 con 0x55abc0e66370 2024-09-19T09:26:22.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.324+0000 7f89c966c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55abc0d7c060 con 0x55abc0e5cb40 2024-09-19T09:26:22.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.324+0000 7f89c966c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55abc0e691f0 con 0x55abc0e5b5d0 2024-09-19T09:26:22.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.324+0000 7f89c966c740 10 client.0 _ll_register_callbacks cb 0x55abc0acf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:22.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.324+0000 7f89c502f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc0e5cb40 0x55abc0e63090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:22.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.324+0000 7f89c482e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc0e66370 0x55abc0e637d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:22.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.324+0000 7f89c502f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc0e5cb40 0x55abc0e63090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:55154/0 (socket says 172.21.15.120:55154) 2024-09-19T09:26:22.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.324+0000 7f89c502f640 1 -- 172.21.15.120:0/2936919344 learned_addr learned my addr 172.21.15.120:0/2936919344 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:22.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.324+0000 7f89c482e640 1 --2- 172.21.15.120:0/2936919344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc0e5b5d0 0x55abc0e65de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:22.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.324+0000 7f89c502f640 1 -- 172.21.15.120:0/2936919344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc0e5b5d0 msgr2=0x55abc0e65de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:22.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.324+0000 7f89c502f640 1 --2- 172.21.15.120:0/2936919344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc0e5b5d0 0x55abc0e65de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.324+0000 7f89c502f640 1 -- 172.21.15.120:0/2936919344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc0e66370 msgr2=0x55abc0e637d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:22.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.324+0000 7f89c502f640 1 --2- 172.21.15.120:0/2936919344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc0e66370 0x55abc0e637d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.324+0000 7f89c502f640 1 -- 172.21.15.120:0/2936919344 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abc0b323c0 con 0x55abc0e5cb40 2024-09-19T09:26:22.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.324+0000 7f89c482e640 1 --2- 172.21.15.120:0/2936919344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc0e5b5d0 0x55abc0e65de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:22.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.324+0000 7f89a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:26:22.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.324+0000 7f89c502f640 1 --2- 172.21.15.120:0/2936919344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc0e5cb40 0x55abc0e63090 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f89bc009b70 tx=0x7f89bc017600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b288919f7e03c9d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:22.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.325+0000 7f89a8ff9640 1 -- 172.21.15.120:0/2936919344 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc021030 con 0x55abc0e5cb40 2024-09-19T09:26:22.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.325+0000 7f89c966c740 1 -- 172.21.15.120:0/2936919344 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89b0012e10 con 0x55abc0e5cb40 2024-09-19T09:26:22.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.325+0000 7f89a8ff9640 1 -- 172.21.15.120:0/2936919344 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f89bc017c90 con 0x55abc0e5cb40 2024-09-19T09:26:22.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.325+0000 7f89a8ff9640 1 -- 172.21.15.120:0/2936919344 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f89bc017e90 con 0x55abc0e5cb40 2024-09-19T09:26:22.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.325+0000 7f89c966c740 10 client.31670 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:22.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.325+0000 7f89c966c740 1 -- 172.21.15.120:0/2936919344 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55abc0e74c00 con 0x55abc0e5cb40 2024-09-19T09:26:22.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.325+0000 7f89a8ff9640 1 -- 172.21.15.120:0/2936919344 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89b0012e10 con 0x55abc0e5cb40 2024-09-19T09:26:22.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.326+0000 7f89a8ff9640 1 -- 172.21.15.120:0/2936919344 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f89bc003c70 con 0x55abc0e5cb40 2024-09-19T09:26:22.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.326+0000 7f89c966c740 10 client.31670 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:22.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.326+0000 7f89c966c740 10 client.31670 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:26:22.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.326+0000 7f89c966c740 20 client.31670 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:22.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.326+0000 7f89c966c740 1 --2- 172.21.15.120:0/2936919344 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55abc0e76160 0x55abc0e78580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:22.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.326+0000 7f89c966c740 4 client.31670 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:26:22.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.326+0000 7f89c966c740 1 -- 172.21.15.120:0/2936919344 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abc0dce860 con 0x55abc0e76160 2024-09-19T09:26:22.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.326+0000 7f89c482e640 1 --2- 172.21.15.120:0/2936919344 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55abc0e76160 0x55abc0e78580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:22.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.326+0000 7f89a8ff9640 10 client.31670 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:26:22.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.326+0000 7f89c482e640 1 --2- 172.21.15.120:0/2936919344 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55abc0e76160 0x55abc0e78580 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a286955159525050 server_cookie=9dbc6deb84b9b5ea in_seq=0 out_seq=0 2024-09-19T09:26:22.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.327+0000 7f89a8ff9640 1 -- 172.21.15.120:0/2936919344 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55abc0dce860 con 0x55abc0e76160 2024-09-19T09:26:22.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.327+0000 7f89a8ff9640 10 client.31670 handle_command_reply: tid=0 2024-09-19T09:26:22.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.327+0000 7f89c966c740 1 client.31670 shutdown 2024-09-19T09:26:22.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.327+0000 7f89c966c740 1 -- 172.21.15.120:0/2936919344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc0e5cb40 msgr2=0x55abc0e63090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:22.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.327+0000 7f89c966c740 1 --2- 172.21.15.120:0/2936919344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc0e5cb40 0x55abc0e63090 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f89bc009b70 tx=0x7f89bc017600 comp rx=0 tx=0).stop 2024-09-19T09:26:22.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.327+0000 7f89c966c740 1 -- 172.21.15.120:0/2936919344 shutdown_connections 2024-09-19T09:26:22.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.327+0000 7f89c966c740 1 --2- 172.21.15.120:0/2936919344 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55abc0e76160 0x55abc0e78580 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.327+0000 7f89c966c740 1 --2- 172.21.15.120:0/2936919344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc0e66370 0x55abc0e637d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.327+0000 7f89c966c740 1 --2- 172.21.15.120:0/2936919344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc0e5cb40 0x55abc0e63090 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.327+0000 7f89c966c740 1 --2- 172.21.15.120:0/2936919344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc0e5b5d0 0x55abc0e65de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.327+0000 7f89a8ff9640 0 client.31670 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:26:22.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.327+0000 7f89c966c740 1 -- 172.21.15.120:0/2936919344 >> 172.21.15.120:0/2936919344 conn(0x55abc0e24310 msgr2=0x55abc0e5d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:22.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.327+0000 7f89c966c740 1 -- 172.21.15.120:0/2936919344 shutdown_connections 2024-09-19T09:26:22.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.327+0000 7f89c966c740 1 -- 172.21.15.120:0/2936919344 wait complete. 2024-09-19T09:26:22.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.327+0000 7f89c966c740 20 client.31670 trim_cache size 0 max 16384 2024-09-19T09:26:22.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.341+0000 7f89c966c740 1 Processor -- start 2024-09-19T09:26:22.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.341+0000 7f89c966c740 1 -- start start 2024-09-19T09:26:22.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.341+0000 7f89c966c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc0e5b5d0 0x55abc0e72310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:22.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.341+0000 7f89c966c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc0e5cb40 0x55abc0e61f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:22.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.341+0000 7f89c966c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc0e66370 0x55abc0e7de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:22.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.341+0000 7f89c966c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01ab2e0 con 0x55abc0e66370 2024-09-19T09:26:22.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.341+0000 7f89c966c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01ace20 con 0x55abc0e5cb40 2024-09-19T09:26:22.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.341+0000 7f89c966c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01ab9b0 con 0x55abc0e5b5d0 2024-09-19T09:26:22.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.342+0000 7f89c482e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc0e66370 0x55abc0e7de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:22.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.342+0000 7f89c502f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc0e5cb40 0x55abc0e61f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:22.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.342+0000 7f89c502f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc0e5cb40 0x55abc0e61f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:55160/0 (socket says 172.21.15.120:55160) 2024-09-19T09:26:22.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.342+0000 7f89c502f640 1 -- 172.21.15.120:0/1396538026 learned_addr learned my addr 172.21.15.120:0/1396538026 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:22.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.342+0000 7f89c482e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc0e66370 0x55abc0e7de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56096/0 (socket says 172.21.15.120:56096) 2024-09-19T09:26:22.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.342+0000 7f89c482e640 1 --2- 172.21.15.120:0/1396538026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc0e5b5d0 0x55abc0e72310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:22.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.342+0000 7f89c502f640 1 -- 172.21.15.120:0/1396538026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc0e5b5d0 msgr2=0x55abc0e72310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:22.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.342+0000 7f89c502f640 1 --2- 172.21.15.120:0/1396538026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc0e5b5d0 0x55abc0e72310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.342+0000 7f89c502f640 1 -- 172.21.15.120:0/1396538026 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc0e66370 msgr2=0x55abc0e7de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:22.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.342+0000 7f89c502f640 1 --2- 172.21.15.120:0/1396538026 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc0e66370 0x55abc0e7de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.342+0000 7f89c502f640 1 -- 172.21.15.120:0/1396538026 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abc0e58960 con 0x55abc0e5cb40 2024-09-19T09:26:22.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.342+0000 7f89c502f640 1 --2- 172.21.15.120:0/1396538026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc0e5cb40 0x55abc0e61f40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f89bc005d00 tx=0x7f89bc017b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee3da2b76818b60e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:22.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.342+0000 7f89aaffd640 1 -- 172.21.15.120:0/1396538026 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc009e70 con 0x55abc0e5cb40 2024-09-19T09:26:22.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.342+0000 7f89aaffd640 1 -- 172.21.15.120:0/1396538026 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89bc017c90 con 0x55abc0e5cb40 2024-09-19T09:26:22.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.342+0000 7f89aaffd640 1 -- 172.21.15.120:0/1396538026 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc01da00 con 0x55abc0e5cb40 2024-09-19T09:26:22.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.343+0000 7f89c966c740 1 -- 172.21.15.120:0/1396538026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc0e5cb40 msgr2=0x55abc0e61f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:22.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.343+0000 7f89c966c740 1 --2- 172.21.15.120:0/1396538026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc0e5cb40 0x55abc0e61f40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f89bc005d00 tx=0x7f89bc017b00 comp rx=0 tx=0).stop 2024-09-19T09:26:22.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.343+0000 7f89c966c740 1 -- 172.21.15.120:0/1396538026 shutdown_connections 2024-09-19T09:26:22.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.343+0000 7f89c966c740 1 --2- 172.21.15.120:0/1396538026 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc0e66370 0x55abc0e7de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.343+0000 7f89c966c740 1 --2- 172.21.15.120:0/1396538026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc0e5cb40 0x55abc0e61f40 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.343+0000 7f89c966c740 1 --2- 172.21.15.120:0/1396538026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc0e5b5d0 0x55abc0e72310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.343+0000 7f89c966c740 1 -- 172.21.15.120:0/1396538026 >> 172.21.15.120:0/1396538026 conn(0x55abc0e24310 msgr2=0x55abc0e61830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:22.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.343+0000 7f89c966c740 1 -- 172.21.15.120:0/1396538026 shutdown_connections 2024-09-19T09:26:22.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.343+0000 7f89c966c740 1 -- 172.21.15.120:0/1396538026 wait complete. 2024-09-19T09:26:22.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.344+0000 7f89c966c740 1 -- start start 2024-09-19T09:26:22.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.344+0000 7f89c966c740 1 Processor -- start 2024-09-19T09:26:22.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.344+0000 7f89c966c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc0e5b5d0 0x55abc0e6e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:22.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.344+0000 7f89c966c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc0e5cb40 0x55abc0e6ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:22.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.344+0000 7f89c966c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc0e66370 0x55abc0e69dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:22.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.344+0000 7f89c966c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01ac750 con 0x55abc0e5b5d0 2024-09-19T09:26:22.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.344+0000 7f89c966c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01ac080 con 0x55abc0e66370 2024-09-19T09:26:22.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.344+0000 7f89c966c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01a92a0 con 0x55abc0e5cb40 2024-09-19T09:26:22.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.344+0000 7f89c966c740 10 client.0 _ll_register_callbacks cb 0x7f89bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:22.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.344+0000 7f89c482e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc0e5b5d0 0x55abc0e6e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:22.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.344+0000 7f89c502f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc0e5cb40 0x55abc0e6ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:22.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.344+0000 7f89c502f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc0e5cb40 0x55abc0e6ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36718/0 (socket says 172.21.15.120:36718) 2024-09-19T09:26:22.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.344+0000 7f89c502f640 1 -- 172.21.15.120:0/1839235416 learned_addr learned my addr 172.21.15.120:0/1839235416 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:22.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.345+0000 7f89c482e640 1 --2- 172.21.15.120:0/1839235416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc0e66370 0x55abc0e69dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:22.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.345+0000 7f89c482e640 1 -- 172.21.15.120:0/1839235416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc0e5cb40 msgr2=0x55abc0e6ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:22.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.345+0000 7f89c482e640 1 --2- 172.21.15.120:0/1839235416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc0e5cb40 0x55abc0e6ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.345+0000 7f89c482e640 1 -- 172.21.15.120:0/1839235416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc0e66370 msgr2=0x55abc0e69dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:22.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.345+0000 7f89c482e640 1 --2- 172.21.15.120:0/1839235416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc0e66370 0x55abc0e69dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.345+0000 7f89c482e640 1 -- 172.21.15.120:0/1839235416 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abc0e6a5c0 con 0x55abc0e5b5d0 2024-09-19T09:26:22.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.345+0000 7f89c502f640 1 --2- 172.21.15.120:0/1839235416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc0e5cb40 0x55abc0e6ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:22.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.345+0000 7f89c482e640 1 --2- 172.21.15.120:0/1839235416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc0e5b5d0 0x55abc0e6e840 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x55abc0e78da0 tx=0x7f89b000b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc739333bdbf4997 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:22.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.345+0000 7f89a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:26:22.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.345+0000 7f89a8ff9640 1 -- 172.21.15.120:0/1839235416 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b003b020 con 0x55abc0e5b5d0 2024-09-19T09:26:22.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.345+0000 7f89a8ff9640 1 -- 172.21.15.120:0/1839235416 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f89b0035810 con 0x55abc0e5b5d0 2024-09-19T09:26:22.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.345+0000 7f89c966c740 1 -- 172.21.15.120:0/1839235416 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89bc017c90 con 0x55abc0e5b5d0 2024-09-19T09:26:22.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.345+0000 7f89a8ff9640 1 -- 172.21.15.120:0/1839235416 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f89b0035a10 con 0x55abc0e5b5d0 2024-09-19T09:26:22.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.345+0000 7f89c966c740 10 client.23007 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:22.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.345+0000 7f89c966c740 1 -- 172.21.15.120:0/1839235416 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55abc0ee0a80 con 0x55abc0e5b5d0 2024-09-19T09:26:22.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.346+0000 7f89a8ff9640 1 -- 172.21.15.120:0/1839235416 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89b0010c20 con 0x55abc0e5b5d0 2024-09-19T09:26:22.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.346+0000 7f89a8ff9640 1 -- 172.21.15.120:0/1839235416 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f89b00035f0 con 0x55abc0e5b5d0 2024-09-19T09:26:22.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.346+0000 7f89c966c740 10 client.23007 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:22.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.346+0000 7f89c966c740 10 client.23007 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:26:22.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.346+0000 7f89c966c740 20 client.23007 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:22.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.346+0000 7f89c966c740 1 --2- 172.21.15.120:0/1839235416 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55abc0ee1020 0x55abc0ee3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:22.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.346+0000 7f89c966c740 4 client.23007 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:26:22.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.346+0000 7f89c966c740 1 -- 172.21.15.120:0/1839235416 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f89bc017c90 con 0x55abc0ee1020 2024-09-19T09:26:22.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.346+0000 7f89c502f640 1 --2- 172.21.15.120:0/1839235416 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55abc0ee1020 0x55abc0ee3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:22.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.346+0000 7f89a8ff9640 10 client.23007 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:26:22.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.346+0000 7f89c502f640 1 --2- 172.21.15.120:0/1839235416 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55abc0ee1020 0x55abc0ee3410 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=beacd2b68dcb5ead server_cookie=bb9e6e6b4e4fbb37 in_seq=0 out_seq=0 2024-09-19T09:26:22.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.347+0000 7f89a8ff9640 1 -- 172.21.15.120:0/1839235416 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f89bc017c90 con 0x55abc0ee1020 2024-09-19T09:26:22.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.347+0000 7f89a8ff9640 10 client.23007 handle_command_reply: tid=0 2024-09-19T09:26:22.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.347+0000 7f89c966c740 1 client.23007 shutdown 2024-09-19T09:26:22.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.347+0000 7f89c966c740 1 -- 172.21.15.120:0/1839235416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc0e5b5d0 msgr2=0x55abc0e6e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:22.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.347+0000 7f89c966c740 1 --2- 172.21.15.120:0/1839235416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc0e5b5d0 0x55abc0e6e840 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x55abc0e78da0 tx=0x7f89b000b670 comp rx=0 tx=0).stop 2024-09-19T09:26:22.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.347+0000 7f89c966c740 1 -- 172.21.15.120:0/1839235416 shutdown_connections 2024-09-19T09:26:22.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.347+0000 7f89c966c740 1 --2- 172.21.15.120:0/1839235416 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55abc0ee1020 0x55abc0ee3410 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.347+0000 7f89c966c740 1 --2- 172.21.15.120:0/1839235416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55abc0e66370 0x55abc0e69dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.347+0000 7f89c966c740 1 --2- 172.21.15.120:0/1839235416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55abc0e5cb40 0x55abc0e6ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.347+0000 7f89a8ff9640 0 client.23007 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:26:22.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.347+0000 7f89c966c740 1 --2- 172.21.15.120:0/1839235416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abc0e5b5d0 0x55abc0e6e840 unknown :-1 s=CLOSED pgs=2376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.347+0000 7f89c966c740 1 -- 172.21.15.120:0/1839235416 >> 172.21.15.120:0/1839235416 conn(0x55abc0e24310 msgr2=0x55abc0e5d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:22.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.347+0000 7f89c966c740 1 -- 172.21.15.120:0/1839235416 shutdown_connections 2024-09-19T09:26:22.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.347+0000 7f89c966c740 1 -- 172.21.15.120:0/1839235416 wait complete. 2024-09-19T09:26:22.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.347+0000 7f89c966c740 20 client.23007 trim_cache size 0 max 16384 2024-09-19T09:26:22.355 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:26:22.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.348+0000 7f89c7293640 1 -- 172.21.15.120:0/3718856974 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f89ac0a32c0 msgr2=0x7f89ac0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:22.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.348+0000 7f89c7293640 1 --2- 172.21.15.120:0/3718856974 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f89ac0a32c0 0x7f89ac0a5780 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f89b80b9a20 tx=0x7f89b8090560 comp rx=0 tx=0).stop 2024-09-19T09:26:22.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.348+0000 7f89c7293640 1 -- 172.21.15.120:0/3718856974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89c0106f10 msgr2=0x7f89c019cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:22.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.348+0000 7f89c7293640 1 --2- 172.21.15.120:0/3718856974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89c0106f10 0x7f89c019cca0 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f89b8031a20 tx=0x7f89b8031a50 comp rx=0 tx=0).stop 2024-09-19T09:26:22.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.348+0000 7f89c7293640 1 -- 172.21.15.120:0/3718856974 shutdown_connections 2024-09-19T09:26:22.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.348+0000 7f89c7293640 1 --2- 172.21.15.120:0/3718856974 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f89ac0a32c0 0x7f89ac0a5780 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.348+0000 7f89c7293640 1 --2- 172.21.15.120:0/3718856974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89c0106f10 0x7f89c019cca0 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.348+0000 7f89c7293640 1 --2- 172.21.15.120:0/3718856974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89c0102790 0x7f89c0198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.348+0000 7f89c7293640 1 --2- 172.21.15.120:0/3718856974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89c0101de0 0x7f89c01983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.348+0000 7f89c7293640 1 -- 172.21.15.120:0/3718856974 >> 172.21.15.120:0/3718856974 conn(0x7f89c00fd880 msgr2=0x7f89c00feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:22.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.348+0000 7f89c7293640 1 -- 172.21.15.120:0/3718856974 shutdown_connections 2024-09-19T09:26:22.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:22.348+0000 7f89c7293640 1 -- 172.21.15.120:0/3718856974 wait complete. 2024-09-19T09:26:22.385 INFO:tasks.workunit.client.0.smithi120.stdout: 10 247591 20.84 MB/sec execute 497 sec latency 67.935 ms 2024-09-19T09:26:22.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:22 smithi196 ceph-mon[35150]: pgmap v1050: 97 pgs: 97 active+clean; 503 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.9 MiB/s wr, 729 op/s 2024-09-19T09:26:22.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:22 smithi153 ceph-mon[35071]: pgmap v1050: 97 pgs: 97 active+clean; 503 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.9 MiB/s wr, 729 op/s 2024-09-19T09:26:23.386 INFO:tasks.workunit.client.0.smithi120.stdout: 10 247905 20.82 MB/sec execute 498 sec latency 313.283 ms 2024-09-19T09:26:24.090 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:26:24.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:24 smithi120 ceph-mon[31150]: pgmap v1051: 97 pgs: 97 active+clean; 492 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.0 MiB/s wr, 724 op/s 2024-09-19T09:26:24.386 INFO:tasks.workunit.client.0.smithi120.stdout: 10 248488 20.83 MB/sec execute 499 sec latency 235.184 ms 2024-09-19T09:26:24.421 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:24.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:24 smithi196 ceph-mon[35150]: pgmap v1051: 97 pgs: 97 active+clean; 492 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.0 MiB/s wr, 724 op/s 2024-09-19T09:26:24.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:24 smithi153 ceph-mon[35071]: pgmap v1051: 97 pgs: 97 active+clean; 492 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.0 MiB/s wr, 724 op/s 2024-09-19T09:26:24.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.960+0000 7fef53bb2640 1 -- 172.21.15.120:0/1145556433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef4c101dd0 msgr2=0x7fef4c1021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:24.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.960+0000 7fef53bb2640 1 --2- 172.21.15.120:0/1145556433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef4c101dd0 0x7fef4c1021b0 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7fef48009900 tx=0x7fef4802a410 comp rx=0 tx=0).stop 2024-09-19T09:26:24.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.960+0000 7fef53bb2640 1 -- 172.21.15.120:0/1145556433 shutdown_connections 2024-09-19T09:26:24.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.960+0000 7fef53bb2640 1 --2- 172.21.15.120:0/1145556433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef4c106f00 0x7fef4c10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:24.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.960+0000 7fef53bb2640 1 --2- 172.21.15.120:0/1145556433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef4c102780 0x7fef4c1067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:24.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.960+0000 7fef53bb2640 1 --2- 172.21.15.120:0/1145556433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef4c101dd0 0x7fef4c1021b0 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:24.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.960+0000 7fef53bb2640 1 -- 172.21.15.120:0/1145556433 >> 172.21.15.120:0/1145556433 conn(0x7fef4c0fd8b0 msgr2=0x7fef4c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:24.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.960+0000 7fef53bb2640 1 -- 172.21.15.120:0/1145556433 shutdown_connections 2024-09-19T09:26:24.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.960+0000 7fef53bb2640 1 -- 172.21.15.120:0/1145556433 wait complete. 2024-09-19T09:26:24.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.961+0000 7fef53bb2640 1 Processor -- start 2024-09-19T09:26:24.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.961+0000 7fef53bb2640 1 -- start start 2024-09-19T09:26:24.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.961+0000 7fef53bb2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef4c101dd0 0x7fef4c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:24.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.961+0000 7fef53bb2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef4c102780 0x7fef4c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:24.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.961+0000 7fef53bb2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef4c106f00 0x7fef4c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:24.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.961+0000 7fef53bb2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c111f60 con 0x7fef4c106f00 2024-09-19T09:26:24.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.961+0000 7fef53bb2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c111de0 con 0x7fef4c102780 2024-09-19T09:26:24.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.961+0000 7fef53bb2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c1120e0 con 0x7fef4c101dd0 2024-09-19T09:26:24.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.961+0000 7fef5214f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef4c106f00 0x7fef4c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:24.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.961+0000 7fef5214f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef4c106f00 0x7fef4c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56116/0 (socket says 172.21.15.120:56116) 2024-09-19T09:26:24.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.961+0000 7fef5194e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef4c101dd0 0x7fef4c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:24.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.961+0000 7fef5214f640 1 -- 172.21.15.120:0/297098673 learned_addr learned my addr 172.21.15.120:0/297098673 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:24.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.961+0000 7fef5114d640 1 --2- 172.21.15.120:0/297098673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef4c102780 0x7fef4c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:24.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.962+0000 7fef5214f640 1 -- 172.21.15.120:0/297098673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef4c101dd0 msgr2=0x7fef4c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:24.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.962+0000 7fef5214f640 1 --2- 172.21.15.120:0/297098673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef4c101dd0 0x7fef4c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:24.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.962+0000 7fef5214f640 1 -- 172.21.15.120:0/297098673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef4c102780 msgr2=0x7fef4c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:24.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.962+0000 7fef5214f640 1 --2- 172.21.15.120:0/297098673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef4c102780 0x7fef4c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:24.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.962+0000 7fef5214f640 1 -- 172.21.15.120:0/297098673 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4c1a68e0 con 0x7fef4c106f00 2024-09-19T09:26:24.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.962+0000 7fef5214f640 1 --2- 172.21.15.120:0/297098673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef4c106f00 0x7fef4c074f70 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7fef4400eb10 tx=0x7fef4400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:24.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.962+0000 7fef42ffd640 1 -- 172.21.15.120:0/297098673 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef44004b20 con 0x7fef4c106f00 2024-09-19T09:26:24.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.962+0000 7fef42ffd640 1 -- 172.21.15.120:0/297098673 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef44004cc0 con 0x7fef4c106f00 2024-09-19T09:26:24.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.962+0000 7fef53bb2640 1 -- 172.21.15.120:0/297098673 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef4c1a6bd0 con 0x7fef4c106f00 2024-09-19T09:26:24.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.962+0000 7fef42ffd640 1 -- 172.21.15.120:0/297098673 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef44010760 con 0x7fef4c106f00 2024-09-19T09:26:24.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.962+0000 7fef53bb2640 1 -- 172.21.15.120:0/297098673 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef4c1a7110 con 0x7fef4c106f00 2024-09-19T09:26:24.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.964+0000 7fef42ffd640 1 -- 172.21.15.120:0/297098673 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fef44005020 con 0x7fef4c106f00 2024-09-19T09:26:24.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.965+0000 7fef55f8b740 1 Processor -- start 2024-09-19T09:26:24.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.965+0000 7fef55f8b740 1 -- start start 2024-09-19T09:26:24.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.965+0000 7fef55f8b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654d6deab40 0x5654d6de9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:24.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.965+0000 7fef55f8b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d6de95d0 0x5654d6de9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:24.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.965+0000 7fef55f8b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654d6df4370 0x5654d6df6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:24.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.965+0000 7fef55f8b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5654d6d4f2f0 con 0x5654d6df4370 2024-09-19T09:26:24.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.965+0000 7fef55f8b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5654d6cf6cf0 con 0x5654d6deab40 2024-09-19T09:26:24.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.965+0000 7fef55f8b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5654d6d4b2b0 con 0x5654d6de95d0 2024-09-19T09:26:24.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.965+0000 7fef42ffd640 1 --2- 172.21.15.120:0/297098673 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fef380a3240 0x7fef380a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:24.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.965+0000 7fef5194e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d6de95d0 0x5654d6de9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:24.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.966+0000 7fef5194e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d6de95d0 0x5654d6de9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36738/0 (socket says 172.21.15.120:36738) 2024-09-19T09:26:24.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.966+0000 7fef5194e640 1 -- 172.21.15.120:0/1166547966 learned_addr learned my addr 172.21.15.120:0/1166547966 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:24.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.966+0000 7fef5114d640 1 --2- 172.21.15.120:0/1166547966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654d6df4370 0x5654d6df6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:24.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.966+0000 7fef5114d640 1 --2- 172.21.15.120:0/1166547966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654d6deab40 0x5654d6de9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:24.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.966+0000 7fef5214f640 1 --2- 172.21.15.120:0/297098673 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fef380a3240 0x7fef380a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:24.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.966+0000 7fef42ffd640 1 -- 172.21.15.120:0/297098673 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fef440cff40 con 0x7fef4c106f00 2024-09-19T09:26:24.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.966+0000 7fef5194e640 1 -- 172.21.15.120:0/1166547966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654d6deab40 msgr2=0x5654d6de9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:24.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.966+0000 7fef5194e640 1 --2- 172.21.15.120:0/1166547966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654d6deab40 0x5654d6de9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:24.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.966+0000 7fef5194e640 1 -- 172.21.15.120:0/1166547966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654d6df4370 msgr2=0x5654d6df6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:24.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.966+0000 7fef5194e640 1 --2- 172.21.15.120:0/1166547966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654d6df4370 0x5654d6df6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:24.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.966+0000 7fef5194e640 1 -- 172.21.15.120:0/1166547966 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654d6cdb3a0 con 0x5654d6de95d0 2024-09-19T09:26:24.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.966+0000 7fef5114d640 1 --2- 172.21.15.120:0/1166547966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654d6df4370 0x5654d6df6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:24.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.966+0000 7fef5214f640 1 --2- 172.21.15.120:0/297098673 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fef380a3240 0x7fef380a5700 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fef44009a70 tx=0x7fef4401bb30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:24.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.966+0000 7fef5194e640 1 --2- 172.21.15.120:0/1166547966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d6de95d0 0x5654d6de9a10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fef48009900 tx=0x7fef48005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=354a0469c7a83e9a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:24.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.966+0000 7fef37fff640 1 -- 172.21.15.120:0/1166547966 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48040070 con 0x5654d6de95d0 2024-09-19T09:26:24.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.966+0000 7fef37fff640 1 -- 172.21.15.120:0/1166547966 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef48031870 con 0x5654d6de95d0 2024-09-19T09:26:24.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.966+0000 7fef37fff640 1 -- 172.21.15.120:0/1166547966 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48035040 con 0x5654d6de95d0 2024-09-19T09:26:24.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.967+0000 7fef55f8b740 1 -- 172.21.15.120:0/1166547966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d6de95d0 msgr2=0x5654d6de9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:24.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.967+0000 7fef55f8b740 1 --2- 172.21.15.120:0/1166547966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d6de95d0 0x5654d6de9a10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fef48009900 tx=0x7fef48005be0 comp rx=0 tx=0).stop 2024-09-19T09:26:24.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.967+0000 7fef55f8b740 1 -- 172.21.15.120:0/1166547966 shutdown_connections 2024-09-19T09:26:24.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.967+0000 7fef55f8b740 1 --2- 172.21.15.120:0/1166547966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654d6df4370 0x5654d6df6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:24.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.967+0000 7fef55f8b740 1 --2- 172.21.15.120:0/1166547966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d6de95d0 0x5654d6de9a10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:24.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.967+0000 7fef55f8b740 1 --2- 172.21.15.120:0/1166547966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654d6deab40 0x5654d6de9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:24.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.967+0000 7fef55f8b740 1 -- 172.21.15.120:0/1166547966 >> 172.21.15.120:0/1166547966 conn(0x5654d6db2310 msgr2=0x5654d6de5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:24.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.967+0000 7fef55f8b740 1 -- 172.21.15.120:0/1166547966 shutdown_connections 2024-09-19T09:26:24.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.967+0000 7fef55f8b740 1 -- 172.21.15.120:0/1166547966 wait complete. 2024-09-19T09:26:24.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.968+0000 7fef55f8b740 1 -- start start 2024-09-19T09:26:24.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.968+0000 7fef55f8b740 1 Processor -- start 2024-09-19T09:26:24.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.968+0000 7fef55f8b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654d6de95d0 0x5654d6df3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:24.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.968+0000 7fef55f8b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654d6deab40 0x5654d6df0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:24.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.968+0000 7fef55f8b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d6df4370 0x5654d6df14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:24.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.968+0000 7fef55f8b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5654d6df7010 con 0x5654d6deab40 2024-09-19T09:26:24.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.968+0000 7fef55f8b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5654d6d0a060 con 0x5654d6de95d0 2024-09-19T09:26:24.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.968+0000 7fef55f8b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5654d6df7190 con 0x5654d6df4370 2024-09-19T09:26:24.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.968+0000 7fef55f8b740 10 client.0 _ll_register_callbacks cb 0x5654d6a5d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:24.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.968+0000 7fef5194e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654d6deab40 0x5654d6df0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:24.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.968+0000 7fef5194e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654d6deab40 0x5654d6df0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56138/0 (socket says 172.21.15.120:56138) 2024-09-19T09:26:24.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.968+0000 7fef5194e640 1 -- 172.21.15.120:0/2564482083 learned_addr learned my addr 172.21.15.120:0/2564482083 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:24.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.968+0000 7fef5114d640 1 --2- 172.21.15.120:0/2564482083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654d6de95d0 0x5654d6df3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:24.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.969+0000 7fef5114d640 1 --2- 172.21.15.120:0/2564482083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d6df4370 0x5654d6df14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:24.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.969+0000 7fef5194e640 1 -- 172.21.15.120:0/2564482083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d6df4370 msgr2=0x5654d6df14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:24.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.969+0000 7fef5194e640 1 --2- 172.21.15.120:0/2564482083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d6df4370 0x5654d6df14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:24.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.969+0000 7fef5194e640 1 -- 172.21.15.120:0/2564482083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654d6de95d0 msgr2=0x5654d6df3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:24.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.969+0000 7fef5194e640 1 --2- 172.21.15.120:0/2564482083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654d6de95d0 0x5654d6df3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:24.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.969+0000 7fef5194e640 1 -- 172.21.15.120:0/2564482083 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654d6ac03c0 con 0x5654d6deab40 2024-09-19T09:26:24.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.969+0000 7fef5194e640 1 --2- 172.21.15.120:0/2564482083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654d6deab40 0x5654d6df0f80 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7fef4802a930 tx=0x7fef48005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c39d18aff14548 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:24.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.969+0000 7fef357fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:26:24.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.969+0000 7fef357fa640 1 -- 172.21.15.120:0/2564482083 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4804c030 con 0x5654d6deab40 2024-09-19T09:26:24.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.969+0000 7fef55f8b740 1 -- 172.21.15.120:0/2564482083 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef48031870 con 0x5654d6deab40 2024-09-19T09:26:24.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.969+0000 7fef357fa640 1 -- 172.21.15.120:0/2564482083 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fef48031670 con 0x5654d6deab40 2024-09-19T09:26:24.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.969+0000 7fef357fa640 1 -- 172.21.15.120:0/2564482083 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fef48040070 con 0x5654d6deab40 2024-09-19T09:26:24.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.969+0000 7fef55f8b740 10 client.23019 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:24.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.969+0000 7fef55f8b740 1 -- 172.21.15.120:0/2564482083 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5654d6e02a50 con 0x5654d6deab40 2024-09-19T09:26:24.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.970+0000 7fef357fa640 1 -- 172.21.15.120:0/2564482083 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef48031870 con 0x5654d6deab40 2024-09-19T09:26:24.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.970+0000 7fef357fa640 1 -- 172.21.15.120:0/2564482083 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fef48004ce0 con 0x5654d6deab40 2024-09-19T09:26:24.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.970+0000 7fef55f8b740 10 client.23019 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:24.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.970+0000 7fef55f8b740 10 client.23019 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:26:24.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.970+0000 7fef55f8b740 20 client.23019 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:24.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.970+0000 7fef55f8b740 1 --2- 172.21.15.120:0/2564482083 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5654d6e03da0 0x5654d6e06190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:24.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.970+0000 7fef55f8b740 4 client.23019 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:26:24.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.970+0000 7fef55f8b740 1 -- 172.21.15.120:0/2564482083 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654d6d5c860 con 0x5654d6e03da0 2024-09-19T09:26:24.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.970+0000 7fef5114d640 1 --2- 172.21.15.120:0/2564482083 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5654d6e03da0 0x5654d6e06190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:24.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.970+0000 7fef5114d640 1 --2- 172.21.15.120:0/2564482083 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5654d6e03da0 0x5654d6e06190 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=767d4f60dfd956d3 server_cookie=133781f9758e93fc in_seq=0 out_seq=0 2024-09-19T09:26:24.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.970+0000 7fef357fa640 10 client.23019 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:26:24.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.971+0000 7fef357fa640 1 -- 172.21.15.120:0/2564482083 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5654d6d5c860 con 0x5654d6e03da0 2024-09-19T09:26:24.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.971+0000 7fef357fa640 10 client.23019 handle_command_reply: tid=0 2024-09-19T09:26:24.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.971+0000 7fef55f8b740 1 client.23019 shutdown 2024-09-19T09:26:24.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.971+0000 7fef55f8b740 1 -- 172.21.15.120:0/2564482083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654d6deab40 msgr2=0x5654d6df0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:24.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.971+0000 7fef55f8b740 1 --2- 172.21.15.120:0/2564482083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654d6deab40 0x5654d6df0f80 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7fef4802a930 tx=0x7fef48005250 comp rx=0 tx=0).stop 2024-09-19T09:26:24.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.971+0000 7fef55f8b740 1 -- 172.21.15.120:0/2564482083 shutdown_connections 2024-09-19T09:26:24.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.971+0000 7fef55f8b740 1 --2- 172.21.15.120:0/2564482083 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5654d6e03da0 0x5654d6e06190 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:24.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.971+0000 7fef55f8b740 1 --2- 172.21.15.120:0/2564482083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d6df4370 0x5654d6df14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:24.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.971+0000 7fef55f8b740 1 --2- 172.21.15.120:0/2564482083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654d6deab40 0x5654d6df0f80 unknown :-1 s=CLOSED pgs=2379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:24.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.971+0000 7fef55f8b740 1 --2- 172.21.15.120:0/2564482083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654d6de95d0 0x5654d6df3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:24.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.971+0000 7fef357fa640 0 client.23019 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:26:24.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.971+0000 7fef55f8b740 1 -- 172.21.15.120:0/2564482083 >> 172.21.15.120:0/2564482083 conn(0x5654d6db2310 msgr2=0x5654d6deb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:24.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.971+0000 7fef55f8b740 1 -- 172.21.15.120:0/2564482083 shutdown_connections 2024-09-19T09:26:24.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.971+0000 7fef55f8b740 1 -- 172.21.15.120:0/2564482083 wait complete. 2024-09-19T09:26:24.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.971+0000 7fef55f8b740 20 client.23019 trim_cache size 0 max 16384 2024-09-19T09:26:24.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.987+0000 7fef55f8b740 1 Processor -- start 2024-09-19T09:26:24.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.987+0000 7fef55f8b740 1 -- start start 2024-09-19T09:26:24.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.988+0000 7fef55f8b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d6de95d0 0x5654d6e002e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:24.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.988+0000 7fef55f8b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654d6deab40 0x5654d6df6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:24.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.988+0000 7fef55f8b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654d6df4370 0x5654d6df5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:24.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.988+0000 7fef55f8b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c1af7b0 con 0x5654d6df4370 2024-09-19T09:26:24.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.988+0000 7fef55f8b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c1b12f0 con 0x5654d6deab40 2024-09-19T09:26:24.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.988+0000 7fef55f8b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c1afe80 con 0x5654d6de95d0 2024-09-19T09:26:24.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.988+0000 7fef5114d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654d6df4370 0x5654d6df5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:24.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.988+0000 7fef5114d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654d6df4370 0x5654d6df5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56150/0 (socket says 172.21.15.120:56150) 2024-09-19T09:26:24.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.988+0000 7fef5114d640 1 -- 172.21.15.120:0/2169238586 learned_addr learned my addr 172.21.15.120:0/2169238586 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:24.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.988+0000 7fef5114d640 1 --2- 172.21.15.120:0/2169238586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d6de95d0 0x5654d6e002e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:24.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.988+0000 7fef5194e640 1 --2- 172.21.15.120:0/2169238586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654d6deab40 0x5654d6df6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:24.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.988+0000 7fef5194e640 1 -- 172.21.15.120:0/2169238586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d6de95d0 msgr2=0x5654d6e002e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:24.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.988+0000 7fef5194e640 1 --2- 172.21.15.120:0/2169238586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d6de95d0 0x5654d6e002e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:24.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.988+0000 7fef5194e640 1 -- 172.21.15.120:0/2169238586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654d6df4370 msgr2=0x5654d6df5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:24.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.988+0000 7fef5194e640 1 --2- 172.21.15.120:0/2169238586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654d6df4370 0x5654d6df5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:24.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.988+0000 7fef5194e640 1 -- 172.21.15.120:0/2169238586 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654d6de6960 con 0x5654d6deab40 2024-09-19T09:26:24.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.988+0000 7fef5114d640 1 --2- 172.21.15.120:0/2169238586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654d6df4370 0x5654d6df5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:24.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.988+0000 7fef5194e640 1 --2- 172.21.15.120:0/2169238586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654d6deab40 0x5654d6df6330 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fef48003780 tx=0x7fef48005780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=201e6e9fdb683cd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:24.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.989+0000 7fef377fe640 1 -- 172.21.15.120:0/2169238586 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4804c040 con 0x5654d6deab40 2024-09-19T09:26:24.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.989+0000 7fef377fe640 1 -- 172.21.15.120:0/2169238586 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef48005360 con 0x5654d6deab40 2024-09-19T09:26:24.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.989+0000 7fef377fe640 1 -- 172.21.15.120:0/2169238586 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48035040 con 0x5654d6deab40 2024-09-19T09:26:24.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.990+0000 7fef55f8b740 1 -- 172.21.15.120:0/2169238586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654d6deab40 msgr2=0x5654d6df6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:24.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.990+0000 7fef55f8b740 1 --2- 172.21.15.120:0/2169238586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654d6deab40 0x5654d6df6330 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fef48003780 tx=0x7fef48005780 comp rx=0 tx=0).stop 2024-09-19T09:26:24.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.990+0000 7fef55f8b740 1 -- 172.21.15.120:0/2169238586 shutdown_connections 2024-09-19T09:26:24.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.990+0000 7fef55f8b740 1 --2- 172.21.15.120:0/2169238586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654d6df4370 0x5654d6df5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:24.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.990+0000 7fef55f8b740 1 --2- 172.21.15.120:0/2169238586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654d6deab40 0x5654d6df6330 secure :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fef48003780 tx=0x7fef48005780 comp rx=0 tx=0).stop 2024-09-19T09:26:24.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.990+0000 7fef55f8b740 1 --2- 172.21.15.120:0/2169238586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d6de95d0 0x5654d6e002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:24.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.990+0000 7fef55f8b740 1 -- 172.21.15.120:0/2169238586 >> 172.21.15.120:0/2169238586 conn(0x5654d6db2310 msgr2=0x5654d6df0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:24.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.990+0000 7fef55f8b740 1 -- 172.21.15.120:0/2169238586 shutdown_connections 2024-09-19T09:26:24.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.990+0000 7fef55f8b740 1 -- 172.21.15.120:0/2169238586 wait complete. 2024-09-19T09:26:24.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.990+0000 7fef55f8b740 1 -- start start 2024-09-19T09:26:24.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.991+0000 7fef55f8b740 1 Processor -- start 2024-09-19T09:26:24.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.991+0000 7fef55f8b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654d6de95d0 0x5654d6dfcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:24.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.991+0000 7fef55f8b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d6df4370 0x5654d6df55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:24.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.991+0000 7fef55f8b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654d6df5b90 0x5654d6df5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:24.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.991+0000 7fef55f8b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c1b0c20 con 0x5654d6df5b90 2024-09-19T09:26:24.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.991+0000 7fef55f8b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c1b0550 con 0x5654d6de95d0 2024-09-19T09:26:24.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.991+0000 7fef55f8b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c1ad770 con 0x5654d6df4370 2024-09-19T09:26:24.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.991+0000 7fef55f8b740 10 client.0 _ll_register_callbacks cb 0x7fef3c009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:24.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.991+0000 7fef5114d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654d6de95d0 0x5654d6dfcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:24.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.991+0000 7fef5194e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d6df4370 0x5654d6df55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:24.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.991+0000 7fef5114d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654d6df5b90 0x5654d6df5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:24.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.991+0000 7fef5194e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d6df4370 0x5654d6df55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36766/0 (socket says 172.21.15.120:36766) 2024-09-19T09:26:24.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.991+0000 7fef5194e640 1 -- 172.21.15.120:0/2536645589 learned_addr learned my addr 172.21.15.120:0/2536645589 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:24.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.992+0000 7fef5114d640 1 -- 172.21.15.120:0/2536645589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d6df4370 msgr2=0x5654d6df55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:24.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.992+0000 7fef5114d640 1 --2- 172.21.15.120:0/2536645589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d6df4370 0x5654d6df55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:24.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.992+0000 7fef5114d640 1 -- 172.21.15.120:0/2536645589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654d6de95d0 msgr2=0x5654d6dfcca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:24.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.992+0000 7fef5114d640 1 --2- 172.21.15.120:0/2536645589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654d6de95d0 0x5654d6dfcca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:24.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.992+0000 7fef5114d640 1 -- 172.21.15.120:0/2536645589 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654d6dfaab0 con 0x5654d6df5b90 2024-09-19T09:26:24.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.992+0000 7fef5194e640 1 --2- 172.21.15.120:0/2536645589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d6df4370 0x5654d6df55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:24.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.992+0000 7fef5114d640 1 --2- 172.21.15.120:0/2536645589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654d6df5b90 0x5654d6df5ff0 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x5654d6d94610 tx=0x7fef3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9721781d50263798 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:24.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.992+0000 7fef357fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:26:24.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.992+0000 7fef357fa640 1 -- 172.21.15.120:0/2536645589 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3c01a040 con 0x5654d6df5b90 2024-09-19T09:26:24.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.992+0000 7fef357fa640 1 -- 172.21.15.120:0/2536645589 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fef3c005020 con 0x5654d6df5b90 2024-09-19T09:26:24.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.992+0000 7fef55f8b740 1 -- 172.21.15.120:0/2536645589 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef48005360 con 0x5654d6df5b90 2024-09-19T09:26:24.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.992+0000 7fef357fa640 1 -- 172.21.15.120:0/2536645589 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fef3c012020 con 0x5654d6df5b90 2024-09-19T09:26:24.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.992+0000 7fef55f8b740 10 client.23025 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:24.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.992+0000 7fef55f8b740 1 -- 172.21.15.120:0/2536645589 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5654d6e6ecd0 con 0x5654d6df5b90 2024-09-19T09:26:24.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.993+0000 7fef357fa640 1 -- 172.21.15.120:0/2536645589 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef48005360 con 0x5654d6df5b90 2024-09-19T09:26:24.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.993+0000 7fef357fa640 1 -- 172.21.15.120:0/2536645589 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fef3c0049d0 con 0x5654d6df5b90 2024-09-19T09:26:24.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.993+0000 7fef55f8b740 10 client.23025 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:24.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.993+0000 7fef55f8b740 10 client.23025 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:26:24.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.993+0000 7fef55f8b740 20 client.23025 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:24.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.993+0000 7fef55f8b740 1 --2- 172.21.15.120:0/2536645589 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5654d6e6f3a0 0x5654d6e71790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:24.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.993+0000 7fef55f8b740 4 client.23025 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:26:24.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.993+0000 7fef55f8b740 1 -- 172.21.15.120:0/2536645589 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5654d6d56d70 con 0x5654d6e6f3a0 2024-09-19T09:26:24.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.993+0000 7fef5194e640 1 --2- 172.21.15.120:0/2536645589 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5654d6e6f3a0 0x5654d6e71790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:24.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.994+0000 7fef5194e640 1 --2- 172.21.15.120:0/2536645589 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5654d6e6f3a0 0x5654d6e71790 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=eec67ae3164575ad server_cookie=4330aacb53788b32 in_seq=0 out_seq=0 2024-09-19T09:26:24.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.994+0000 7fef357fa640 10 client.23025 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:26:25.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.995+0000 7fef357fa640 1 -- 172.21.15.120:0/2536645589 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5654d6d56d70 con 0x5654d6e6f3a0 2024-09-19T09:26:25.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.995+0000 7fef357fa640 10 client.23025 handle_command_reply: tid=0 2024-09-19T09:26:25.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.995+0000 7fef55f8b740 1 client.23025 shutdown 2024-09-19T09:26:25.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.995+0000 7fef55f8b740 1 -- 172.21.15.120:0/2536645589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654d6df5b90 msgr2=0x5654d6df5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:25.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.995+0000 7fef55f8b740 1 --2- 172.21.15.120:0/2536645589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654d6df5b90 0x5654d6df5ff0 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x5654d6d94610 tx=0x7fef3c0046c0 comp rx=0 tx=0).stop 2024-09-19T09:26:25.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.995+0000 7fef55f8b740 1 -- 172.21.15.120:0/2536645589 shutdown_connections 2024-09-19T09:26:25.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.995+0000 7fef55f8b740 1 --2- 172.21.15.120:0/2536645589 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5654d6e6f3a0 0x5654d6e71790 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:25.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.995+0000 7fef55f8b740 1 --2- 172.21.15.120:0/2536645589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654d6df5b90 0x5654d6df5ff0 unknown :-1 s=CLOSED pgs=2380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:25.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.995+0000 7fef55f8b740 1 --2- 172.21.15.120:0/2536645589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d6df4370 0x5654d6df55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:25.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.995+0000 7fef55f8b740 1 --2- 172.21.15.120:0/2536645589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654d6de95d0 0x5654d6dfcca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:25.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.995+0000 7fef55f8b740 1 -- 172.21.15.120:0/2536645589 >> 172.21.15.120:0/2536645589 conn(0x5654d6db2310 msgr2=0x5654d6df27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:25.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.995+0000 7fef357fa640 0 client.23025 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:26:25.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.995+0000 7fef55f8b740 1 -- 172.21.15.120:0/2536645589 shutdown_connections 2024-09-19T09:26:25.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.995+0000 7fef55f8b740 1 -- 172.21.15.120:0/2536645589 wait complete. 2024-09-19T09:26:25.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.996+0000 7fef55f8b740 20 client.23025 trim_cache size 0 max 16384 2024-09-19T09:26:25.002 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:26:25.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.996+0000 7fef53bb2640 1 -- 172.21.15.120:0/297098673 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fef380a3240 msgr2=0x7fef380a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:25.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.996+0000 7fef53bb2640 1 --2- 172.21.15.120:0/297098673 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fef380a3240 0x7fef380a5700 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fef44009a70 tx=0x7fef4401bb30 comp rx=0 tx=0).stop 2024-09-19T09:26:25.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.996+0000 7fef53bb2640 1 -- 172.21.15.120:0/297098673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef4c106f00 msgr2=0x7fef4c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:25.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.996+0000 7fef53bb2640 1 --2- 172.21.15.120:0/297098673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef4c106f00 0x7fef4c074f70 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7fef4400eb10 tx=0x7fef4400c2d0 comp rx=0 tx=0).stop 2024-09-19T09:26:25.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.996+0000 7fef53bb2640 1 -- 172.21.15.120:0/297098673 shutdown_connections 2024-09-19T09:26:25.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.996+0000 7fef53bb2640 1 --2- 172.21.15.120:0/297098673 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fef380a3240 0x7fef380a5700 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:25.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.996+0000 7fef53bb2640 1 --2- 172.21.15.120:0/297098673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef4c106f00 0x7fef4c074f70 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:25.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.996+0000 7fef53bb2640 1 --2- 172.21.15.120:0/297098673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef4c102780 0x7fef4c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:25.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.996+0000 7fef53bb2640 1 --2- 172.21.15.120:0/297098673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef4c101dd0 0x7fef4c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:25.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.996+0000 7fef53bb2640 1 -- 172.21.15.120:0/297098673 >> 172.21.15.120:0/297098673 conn(0x7fef4c0fd8b0 msgr2=0x7fef4c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:25.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.997+0000 7fef53bb2640 1 -- 172.21.15.120:0/297098673 shutdown_connections 2024-09-19T09:26:25.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:24.997+0000 7fef53bb2640 1 -- 172.21.15.120:0/297098673 wait complete. 2024-09-19T09:26:25.386 INFO:tasks.workunit.client.0.smithi120.stdout: 10 249014 20.87 MB/sec execute 500 sec latency 79.123 ms 2024-09-19T09:26:25.461 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.1 2024-09-19T09:26:25.461 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:26:25.828 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:26:25.865 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:26.373 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:26 smithi120 ceph-mon[31150]: pgmap v1052: 97 pgs: 97 active+clean; 491 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.3 MiB/s wr, 623 op/s 2024-09-19T09:26:26.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.371+0000 7fd5baf85640 1 -- 172.21.15.120:0/1491215981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5b4102790 msgr2=0x7fd5b41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:26.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.371+0000 7fd5baf85640 1 --2- 172.21.15.120:0/1491215981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5b4102790 0x7fd5b41067e0 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009900 tx=0x7fd5a002a410 comp rx=0 tx=0).stop 2024-09-19T09:26:26.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.372+0000 7fd5baf85640 1 -- 172.21.15.120:0/1491215981 shutdown_connections 2024-09-19T09:26:26.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.372+0000 7fd5baf85640 1 --2- 172.21.15.120:0/1491215981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5b4106f10 0x7fd5b410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:26.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.372+0000 7fd5baf85640 1 --2- 172.21.15.120:0/1491215981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5b4102790 0x7fd5b41067e0 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:26.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.372+0000 7fd5baf85640 1 --2- 172.21.15.120:0/1491215981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5b4101de0 0x7fd5b41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:26.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.372+0000 7fd5baf85640 1 -- 172.21.15.120:0/1491215981 >> 172.21.15.120:0/1491215981 conn(0x7fd5b40fd8e0 msgr2=0x7fd5b40ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:26.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.372+0000 7fd5baf85640 1 -- 172.21.15.120:0/1491215981 shutdown_connections 2024-09-19T09:26:26.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.372+0000 7fd5baf85640 1 -- 172.21.15.120:0/1491215981 wait complete. 2024-09-19T09:26:26.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.372+0000 7fd5baf85640 1 Processor -- start 2024-09-19T09:26:26.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.373+0000 7fd5baf85640 1 -- start start 2024-09-19T09:26:26.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.373+0000 7fd5baf85640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5b4101de0 0x7fd5b4198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:26.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.373+0000 7fd5baf85640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5b4102790 0x7fd5b4198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:26.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.373+0000 7fd5baf85640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5b4106f10 0x7fd5b419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:26.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.373+0000 7fd5b8d21640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5b4101de0 0x7fd5b4198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:26.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.373+0000 7fd5b8d21640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5b4101de0 0x7fd5b4198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36776/0 (socket says 172.21.15.120:36776) 2024-09-19T09:26:26.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.373+0000 7fd5b8d21640 1 -- 172.21.15.120:0/2251866976 learned_addr learned my addr 172.21.15.120:0/2251866976 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:26.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.373+0000 7fd5baf85640 1 -- 172.21.15.120:0/2251866976 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b4111f70 con 0x7fd5b4106f10 2024-09-19T09:26:26.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.373+0000 7fd5b3fff640 1 --2- 172.21.15.120:0/2251866976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5b4102790 0x7fd5b4198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:26.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.373+0000 7fd5b9522640 1 --2- 172.21.15.120:0/2251866976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5b4106f10 0x7fd5b419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:26.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.373+0000 7fd5baf85640 1 -- 172.21.15.120:0/2251866976 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b4111df0 con 0x7fd5b4102790 2024-09-19T09:26:26.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.373+0000 7fd5baf85640 1 -- 172.21.15.120:0/2251866976 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b41120f0 con 0x7fd5b4101de0 2024-09-19T09:26:26.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.374+0000 7fd5b9522640 1 -- 172.21.15.120:0/2251866976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5b4101de0 msgr2=0x7fd5b4198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:26.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.374+0000 7fd5b9522640 1 --2- 172.21.15.120:0/2251866976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5b4101de0 0x7fd5b4198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:26.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.374+0000 7fd5b9522640 1 -- 172.21.15.120:0/2251866976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5b4102790 msgr2=0x7fd5b4198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:26.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.374+0000 7fd5b9522640 1 --2- 172.21.15.120:0/2251866976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5b4102790 0x7fd5b4198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:26.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.374+0000 7fd5b9522640 1 -- 172.21.15.120:0/2251866976 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b419d400 con 0x7fd5b4106f10 2024-09-19T09:26:26.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.374+0000 7fd5b3fff640 1 --2- 172.21.15.120:0/2251866976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5b4102790 0x7fd5b4198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:26.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.374+0000 7fd5b8d21640 1 --2- 172.21.15.120:0/2251866976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5b4101de0 0x7fd5b4198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:26.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.374+0000 7fd5b9522640 1 --2- 172.21.15.120:0/2251866976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5b4106f10 0x7fd5b419cd20 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac009fd0 tx=0x7fd5ac00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:26.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.374+0000 7fd5b1ffb640 1 -- 172.21.15.120:0/2251866976 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac00ce20 con 0x7fd5b4106f10 2024-09-19T09:26:26.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.374+0000 7fd5baf85640 1 -- 172.21.15.120:0/2251866976 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b419d6f0 con 0x7fd5b4106f10 2024-09-19T09:26:26.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.374+0000 7fd5b1ffb640 1 -- 172.21.15.120:0/2251866976 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5ac005020 con 0x7fd5b4106f10 2024-09-19T09:26:26.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.374+0000 7fd5b1ffb640 1 -- 172.21.15.120:0/2251866976 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac010720 con 0x7fd5b4106f10 2024-09-19T09:26:26.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.374+0000 7fd5baf85640 1 -- 172.21.15.120:0/2251866976 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5b41a4fd0 con 0x7fd5b4106f10 2024-09-19T09:26:26.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.375+0000 7fd5baf85640 1 -- 172.21.15.120:0/2251866976 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5b41021c0 con 0x7fd5b4106f10 2024-09-19T09:26:26.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.377+0000 7fd5b1ffb640 1 -- 172.21.15.120:0/2251866976 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd5ac005540 con 0x7fd5b4106f10 2024-09-19T09:26:26.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.377+0000 7fd5b1ffb640 1 --2- 172.21.15.120:0/2251866976 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd5a40a3240 0x7fd5a40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:26.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.377+0000 7fd5b8d21640 1 --2- 172.21.15.120:0/2251866976 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd5a40a3240 0x7fd5a40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:26.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.377+0000 7fd5b1ffb640 1 -- 172.21.15.120:0/2251866976 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd5ac0d0070 con 0x7fd5b4106f10 2024-09-19T09:26:26.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.378+0000 7fd5b8d21640 1 --2- 172.21.15.120:0/2251866976 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd5a40a3240 0x7fd5a40a5700 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8011500 tx=0x7fd5a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:26.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.383+0000 7fd5b1ffb640 1 -- 172.21.15.120:0/2251866976 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd5ac09cd50 con 0x7fd5b4106f10 2024-09-19T09:26:26.386 INFO:tasks.workunit.client.0.smithi120.stdout: 10 249717 20.87 MB/sec execute 501 sec latency 65.713 ms 2024-09-19T09:26:26.428 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:26.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:26 smithi196 ceph-mon[35150]: pgmap v1052: 97 pgs: 97 active+clean; 491 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.3 MiB/s wr, 623 op/s 2024-09-19T09:26:26.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:26 smithi153 ceph-mon[35071]: pgmap v1052: 97 pgs: 97 active+clean; 491 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.3 MiB/s wr, 623 op/s 2024-09-19T09:26:26.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.619+0000 7fd5baf85640 1 -- 172.21.15.120:0/2251866976 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5b4109bc0 con 0x7fd5b4106f10 2024-09-19T09:26:26.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.620+0000 7fd5b1ffb640 1 -- 172.21.15.120:0/2251866976 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11988 (secure 0 0 0) 0x7fd5ac09c650 con 0x7fd5b4106f10 2024-09-19T09:26:26.622 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:26:26.622 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":39,"btime":"2024-09-19T09:18:53:120858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:18:53.070639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:26:26.622 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 39 2024-09-19T09:26:26.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.623+0000 7fd5baf85640 1 -- 172.21.15.120:0/2251866976 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd5a40a3240 msgr2=0x7fd5a40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:26.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.623+0000 7fd5baf85640 1 --2- 172.21.15.120:0/2251866976 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd5a40a3240 0x7fd5a40a5700 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8011500 tx=0x7fd5a8005f90 comp rx=0 tx=0).stop 2024-09-19T09:26:26.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.623+0000 7fd5baf85640 1 -- 172.21.15.120:0/2251866976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5b4106f10 msgr2=0x7fd5b419cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:26.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.623+0000 7fd5baf85640 1 --2- 172.21.15.120:0/2251866976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5b4106f10 0x7fd5b419cd20 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac009fd0 tx=0x7fd5ac00ef10 comp rx=0 tx=0).stop 2024-09-19T09:26:26.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.623+0000 7fd5baf85640 1 -- 172.21.15.120:0/2251866976 shutdown_connections 2024-09-19T09:26:26.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.623+0000 7fd5baf85640 1 --2- 172.21.15.120:0/2251866976 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd5a40a3240 0x7fd5a40a5700 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:26.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.623+0000 7fd5baf85640 1 --2- 172.21.15.120:0/2251866976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd5b4106f10 0x7fd5b419cd20 unknown :-1 s=CLOSED pgs=2382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:26.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.623+0000 7fd5baf85640 1 --2- 172.21.15.120:0/2251866976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5b4102790 0x7fd5b4198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:26.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.623+0000 7fd5baf85640 1 --2- 172.21.15.120:0/2251866976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5b4101de0 0x7fd5b4198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:26.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.623+0000 7fd5baf85640 1 -- 172.21.15.120:0/2251866976 >> 172.21.15.120:0/2251866976 conn(0x7fd5b40fd8e0 msgr2=0x7fd5b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:26.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.623+0000 7fd5baf85640 1 -- 172.21.15.120:0/2251866976 shutdown_connections 2024-09-19T09:26:26.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:26.623+0000 7fd5baf85640 1 -- 172.21.15.120:0/2251866976 wait complete. 2024-09-19T09:26:27.386 INFO:tasks.workunit.client.0.smithi120.stdout: 10 250234 20.90 MB/sec execute 502 sec latency 84.350 ms 2024-09-19T09:26:27.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:27 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:26:27.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:27 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2251866976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:26:27.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:27 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:26:27.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:27 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2251866976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:26:27.452 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:27 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:26:27.452 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:27 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2251866976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:26:27.481 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db / --timeout 90 --expiration 133.1 --await-for 45.0' 2024-09-19T09:26:27.482 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db / --timeout 90 --expiration 133.1 --await-for 45.0 2024-09-19T09:26:27.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.662+0000 7f02bad1c640 1 -- 172.21.15.120:0/2271071481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02b4106a90 msgr2=0x7f02b4106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:27.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.662+0000 7f02bad1c640 1 --2- 172.21.15.120:0/2271071481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02b4106a90 0x7f02b4106e70 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009930 tx=0x7f02a802a420 comp rx=0 tx=0).stop 2024-09-19T09:26:27.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.663+0000 7f02bad1c640 1 -- 172.21.15.120:0/2271071481 shutdown_connections 2024-09-19T09:26:27.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.663+0000 7f02bad1c640 1 --2- 172.21.15.120:0/2271071481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02b4101430 0x7f02b410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.663+0000 7f02bad1c640 1 --2- 172.21.15.120:0/2271071481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02b4100a90 0x7f02b4100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.663+0000 7f02bad1c640 1 --2- 172.21.15.120:0/2271071481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02b4106a90 0x7f02b4106e70 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.663+0000 7f02bad1c640 1 -- 172.21.15.120:0/2271071481 >> 172.21.15.120:0/2271071481 conn(0x7f02b40fc7d0 msgr2=0x7f02b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:27.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.663+0000 7f02bad1c640 1 -- 172.21.15.120:0/2271071481 shutdown_connections 2024-09-19T09:26:27.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.663+0000 7f02bad1c640 1 -- 172.21.15.120:0/2271071481 wait complete. 2024-09-19T09:26:27.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.663+0000 7f02bad1c640 1 Processor -- start 2024-09-19T09:26:27.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.664+0000 7f02bad1c640 1 -- start start 2024-09-19T09:26:27.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.664+0000 7f02bad1c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02b4100a90 0x7f02b4199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:27.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.664+0000 7f02bad1c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02b4101430 0x7f02b4199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:27.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.664+0000 7f02bad1c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02b4106a90 0x7f02b41a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:27.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.664+0000 7f02bad1c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f02b4110e90 con 0x7f02b4100a90 2024-09-19T09:26:27.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.664+0000 7f02bad1c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f02b4110d10 con 0x7f02b4106a90 2024-09-19T09:26:27.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.664+0000 7f02bad1c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f02b4111010 con 0x7f02b4101430 2024-09-19T09:26:27.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.664+0000 7f02b8ab8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02b4100a90 0x7f02b4199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:27.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.664+0000 7f02b3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02b4101430 0x7f02b4199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:27.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.664+0000 7f02b3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02b4101430 0x7f02b4199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36790/0 (socket says 172.21.15.120:36790) 2024-09-19T09:26:27.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.664+0000 7f02b3fff640 1 -- 172.21.15.120:0/2790938575 learned_addr learned my addr 172.21.15.120:0/2790938575 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:27.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.664+0000 7f02b92b9640 1 --2- 172.21.15.120:0/2790938575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02b4106a90 0x7f02b41a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:27.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.664+0000 7f02b3fff640 1 -- 172.21.15.120:0/2790938575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02b4106a90 msgr2=0x7f02b41a43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:27.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.664+0000 7f02b3fff640 1 --2- 172.21.15.120:0/2790938575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02b4106a90 0x7f02b41a43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.664+0000 7f02b3fff640 1 -- 172.21.15.120:0/2790938575 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02b4100a90 msgr2=0x7f02b4199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:27.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.664+0000 7f02b3fff640 1 --2- 172.21.15.120:0/2790938575 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02b4100a90 0x7f02b4199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.664+0000 7f02b3fff640 1 -- 172.21.15.120:0/2790938575 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b41a48e0 con 0x7f02b4101430 2024-09-19T09:26:27.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.665+0000 7f02b3fff640 1 --2- 172.21.15.120:0/2790938575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02b4101430 0x7f02b4199860 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f02a000e9f0 tx=0x7f02a000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:27.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.665+0000 7f02b1ffb640 1 -- 172.21.15.120:0/2790938575 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a000cd80 con 0x7f02b4101430 2024-09-19T09:26:27.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.665+0000 7f02bad1c640 1 -- 172.21.15.120:0/2790938575 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b41a4ab0 con 0x7f02b4101430 2024-09-19T09:26:27.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.665+0000 7f02b1ffb640 1 -- 172.21.15.120:0/2790938575 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02a0005500 con 0x7f02b4101430 2024-09-19T09:26:27.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.665+0000 7f02b1ffb640 1 -- 172.21.15.120:0/2790938575 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a0010690 con 0x7f02b4101430 2024-09-19T09:26:27.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.665+0000 7f02bad1c640 1 -- 172.21.15.120:0/2790938575 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b41a4fc0 con 0x7f02b4101430 2024-09-19T09:26:27.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.668+0000 7f02b1ffb640 1 -- 172.21.15.120:0/2790938575 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f02a0004f60 con 0x7f02b4101430 2024-09-19T09:26:27.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.668+0000 7f02bd0f5740 1 Processor -- start 2024-09-19T09:26:27.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.668+0000 7f02b1ffb640 1 --2- 172.21.15.120:0/2790938575 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f02a40a32c0 0x7f02a40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:27.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.669+0000 7f02bd0f5740 1 -- start start 2024-09-19T09:26:27.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.669+0000 7f02b92b9640 1 --2- 172.21.15.120:0/2790938575 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f02a40a32c0 0x7f02a40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:27.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.669+0000 7f02b1ffb640 1 -- 172.21.15.120:0/2790938575 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f02a00cfc70 con 0x7f02b4101430 2024-09-19T09:26:27.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.669+0000 7f02bd0f5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555965207b40 0x555965206090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:27.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.669+0000 7f02bd0f5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559652065d0 0x555965206a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:27.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.669+0000 7f02bd0f5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555965211370 0x555965213760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:27.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.669+0000 7f02bd0f5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55596516c2f0 con 0x555965207b40 2024-09-19T09:26:27.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.669+0000 7f02bd0f5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555965113cf0 con 0x5559652065d0 2024-09-19T09:26:27.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.669+0000 7f02bd0f5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5559651682b0 con 0x555965211370 2024-09-19T09:26:27.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.669+0000 7f02b8ab8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555965207b40 0x555965206090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:27.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.669+0000 7f02b3fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559652065d0 0x555965206a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:27.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.669+0000 7f02b8ab8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555965207b40 0x555965206090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56204/0 (socket says 172.21.15.120:56204) 2024-09-19T09:26:27.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.669+0000 7f02b8ab8640 1 -- 172.21.15.120:0/208141440 learned_addr learned my addr 172.21.15.120:0/208141440 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:27.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.669+0000 7f02b92b9640 1 --2- 172.21.15.120:0/208141440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555965211370 0x555965213760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:27.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.669+0000 7f02b92b9640 1 --2- 172.21.15.120:0/2790938575 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f02a40a32c0 0x7f02a40a5780 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f02ac005fd0 tx=0x7f02ac009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:27.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.669+0000 7f02b8ab8640 1 -- 172.21.15.120:0/208141440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555965211370 msgr2=0x555965213760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:27.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.669+0000 7f02b8ab8640 1 --2- 172.21.15.120:0/208141440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555965211370 0x555965213760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.669+0000 7f02b8ab8640 1 -- 172.21.15.120:0/208141440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559652065d0 msgr2=0x555965206a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:27.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.669+0000 7f02b8ab8640 1 --2- 172.21.15.120:0/208141440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559652065d0 0x555965206a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.669+0000 7f02b8ab8640 1 -- 172.21.15.120:0/208141440 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559650f83a0 con 0x555965207b40 2024-09-19T09:26:27.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.669+0000 7f02b3fff640 1 --2- 172.21.15.120:0/208141440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559652065d0 0x555965206a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:27.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.670+0000 7f02b8ab8640 1 --2- 172.21.15.120:0/208141440 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555965207b40 0x555965206090 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f02a802a940 tx=0x7f02a8005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538a3807e0a1a69a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:27.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.670+0000 7f029effd640 1 -- 172.21.15.120:0/208141440 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a8040070 con 0x555965207b40 2024-09-19T09:26:27.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.670+0000 7f029effd640 1 -- 172.21.15.120:0/208141440 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02a8031870 con 0x555965207b40 2024-09-19T09:26:27.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.670+0000 7f029effd640 1 -- 172.21.15.120:0/208141440 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a8035040 con 0x555965207b40 2024-09-19T09:26:27.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.671+0000 7f02bd0f5740 1 -- 172.21.15.120:0/208141440 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555965207b40 msgr2=0x555965206090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:27.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.671+0000 7f02bd0f5740 1 --2- 172.21.15.120:0/208141440 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555965207b40 0x555965206090 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f02a802a940 tx=0x7f02a8005bd0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.671+0000 7f02bd0f5740 1 -- 172.21.15.120:0/208141440 shutdown_connections 2024-09-19T09:26:27.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.671+0000 7f02bd0f5740 1 --2- 172.21.15.120:0/208141440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555965211370 0x555965213760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.671+0000 7f02bd0f5740 1 --2- 172.21.15.120:0/208141440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559652065d0 0x555965206a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.671+0000 7f02bd0f5740 1 --2- 172.21.15.120:0/208141440 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555965207b40 0x555965206090 unknown :-1 s=CLOSED pgs=2384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.671+0000 7f02bd0f5740 1 -- 172.21.15.120:0/208141440 >> 172.21.15.120:0/208141440 conn(0x5559651cf310 msgr2=0x555965202c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:27.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.671+0000 7f02bd0f5740 1 -- 172.21.15.120:0/208141440 shutdown_connections 2024-09-19T09:26:27.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.671+0000 7f02bd0f5740 1 -- 172.21.15.120:0/208141440 wait complete. 2024-09-19T09:26:27.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.671+0000 7f02bd0f5740 1 -- start start 2024-09-19T09:26:27.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.671+0000 7f02bd0f5740 1 Processor -- start 2024-09-19T09:26:27.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.672+0000 7f02bd0f5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559652065d0 0x55596520e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:27.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.672+0000 7f02bd0f5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555965211370 0x55596520e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:27.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.672+0000 7f02bd0f5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55596520ec20 0x55596521d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:27.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.672+0000 7f02bd0f5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555965214010 con 0x55596520ec20 2024-09-19T09:26:27.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.672+0000 7f02bd0f5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555965127060 con 0x5559652065d0 2024-09-19T09:26:27.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.672+0000 7f02bd0f5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555965214190 con 0x555965211370 2024-09-19T09:26:27.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.672+0000 7f02bd0f5740 10 client.0 _ll_register_callbacks cb 0x555964e7a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:27.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.672+0000 7f02b92b9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55596520ec20 0x55596521d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:27.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.672+0000 7f02b3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555965211370 0x55596520e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:27.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.672+0000 7f02b8ab8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559652065d0 0x55596520e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:27.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.672+0000 7f02b3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555965211370 0x55596520e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36820/0 (socket says 172.21.15.120:36820) 2024-09-19T09:26:27.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.672+0000 7f02b3fff640 1 -- 172.21.15.120:0/2169965631 learned_addr learned my addr 172.21.15.120:0/2169965631 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:27.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.672+0000 7f02b92b9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55596520ec20 0x55596521d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56220/0 (socket says 172.21.15.120:56220) 2024-09-19T09:26:27.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.672+0000 7f02b8ab8640 1 -- 172.21.15.120:0/2169965631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555965211370 msgr2=0x55596520e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:27.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.672+0000 7f02b8ab8640 1 --2- 172.21.15.120:0/2169965631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555965211370 0x55596520e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.672+0000 7f02b8ab8640 1 -- 172.21.15.120:0/2169965631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55596520ec20 msgr2=0x55596521d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:27.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.672+0000 7f02b8ab8640 1 --2- 172.21.15.120:0/2169965631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55596520ec20 0x55596521d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.672+0000 7f02b8ab8640 1 -- 172.21.15.120:0/2169965631 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555964edd3c0 con 0x5559652065d0 2024-09-19T09:26:27.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.672+0000 7f02b8ab8640 1 --2- 172.21.15.120:0/2169965631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559652065d0 0x55596520e1a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f02a803e000 tx=0x7f02a802a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23f08f00fcb78174 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:27.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.673+0000 7f0293fff640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:26:27.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.673+0000 7f0293fff640 1 -- 172.21.15.120:0/2169965631 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a804d060 con 0x5559652065d0 2024-09-19T09:26:27.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.673+0000 7f0293fff640 1 -- 172.21.15.120:0/2169965631 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f02a8004240 con 0x5559652065d0 2024-09-19T09:26:27.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.673+0000 7f02bd0f5740 1 -- 172.21.15.120:0/2169965631 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02a8031870 con 0x5559652065d0 2024-09-19T09:26:27.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.673+0000 7f0293fff640 1 -- 172.21.15.120:0/2169965631 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f02a8040070 con 0x5559652065d0 2024-09-19T09:26:27.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.673+0000 7f02bd0f5740 10 client.31712 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:27.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.673+0000 7f02bd0f5740 1 -- 172.21.15.120:0/2169965631 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555965220020 con 0x5559652065d0 2024-09-19T09:26:27.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.673+0000 7f0293fff640 1 -- 172.21.15.120:0/2169965631 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02a8031870 con 0x5559652065d0 2024-09-19T09:26:27.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.673+0000 7f0293fff640 1 -- 172.21.15.120:0/2169965631 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f02a8052070 con 0x5559652065d0 2024-09-19T09:26:27.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.673+0000 7f02bd0f5740 10 client.31712 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:27.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.673+0000 7f02bd0f5740 10 client.31712 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:26:27.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.673+0000 7f02bd0f5740 20 client.31712 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:27.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.673+0000 7f02bd0f5740 1 --2- 172.21.15.120:0/2169965631 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x555965221370 0x555965223760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:27.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.673+0000 7f02bd0f5740 4 client.31712 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:26:27.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.673+0000 7f02bd0f5740 1 -- 172.21.15.120:0/2169965631 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555965179860 con 0x555965221370 2024-09-19T09:26:27.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.674+0000 7f02b3fff640 1 --2- 172.21.15.120:0/2169965631 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x555965221370 0x555965223760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:27.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.674+0000 7f02b3fff640 1 --2- 172.21.15.120:0/2169965631 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x555965221370 0x555965223760 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8fa13e50078151ff server_cookie=46a8441f87811eae in_seq=0 out_seq=0 2024-09-19T09:26:27.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.674+0000 7f0293fff640 10 client.31712 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:26:27.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.675+0000 7f0293fff640 1 -- 172.21.15.120:0/2169965631 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555965179860 con 0x555965221370 2024-09-19T09:26:27.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.675+0000 7f0293fff640 10 client.31712 handle_command_reply: tid=0 2024-09-19T09:26:27.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.675+0000 7f02bd0f5740 1 client.31712 shutdown 2024-09-19T09:26:27.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.675+0000 7f02bd0f5740 1 -- 172.21.15.120:0/2169965631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559652065d0 msgr2=0x55596520e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:27.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.675+0000 7f02bd0f5740 1 --2- 172.21.15.120:0/2169965631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559652065d0 0x55596520e1a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f02a803e000 tx=0x7f02a802a420 comp rx=0 tx=0).stop 2024-09-19T09:26:27.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.675+0000 7f02bd0f5740 1 -- 172.21.15.120:0/2169965631 shutdown_connections 2024-09-19T09:26:27.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.675+0000 7f02bd0f5740 1 --2- 172.21.15.120:0/2169965631 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x555965221370 0x555965223760 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.675+0000 7f0293fff640 0 client.31712 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:26:27.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.675+0000 7f02bd0f5740 1 --2- 172.21.15.120:0/2169965631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55596520ec20 0x55596521d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.675+0000 7f02bd0f5740 1 --2- 172.21.15.120:0/2169965631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555965211370 0x55596520e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.675+0000 7f02bd0f5740 1 --2- 172.21.15.120:0/2169965631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559652065d0 0x55596520e1a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.675+0000 7f02bd0f5740 1 -- 172.21.15.120:0/2169965631 >> 172.21.15.120:0/2169965631 conn(0x5559651cf310 msgr2=0x555965208930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:27.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.676+0000 7f02bd0f5740 1 -- 172.21.15.120:0/2169965631 shutdown_connections 2024-09-19T09:26:27.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.676+0000 7f02bd0f5740 1 -- 172.21.15.120:0/2169965631 wait complete. 2024-09-19T09:26:27.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.676+0000 7f02bd0f5740 20 client.31712 trim_cache size 0 max 16384 2024-09-19T09:26:27.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.690+0000 7f02bd0f5740 1 Processor -- start 2024-09-19T09:26:27.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.690+0000 7f02bd0f5740 1 -- start start 2024-09-19T09:26:27.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.690+0000 7f02bd0f5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559652065d0 0x55596521d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:27.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.690+0000 7f02bd0f5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55596520ec20 0x55596521d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:27.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.690+0000 7f02bd0f5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555965211370 0x555965228d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:27.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.690+0000 7f02bd0f5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f02b41ab3c0 con 0x555965211370 2024-09-19T09:26:27.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.690+0000 7f02bd0f5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f02b41acf00 con 0x5559652065d0 2024-09-19T09:26:27.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.690+0000 7f02bd0f5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f02b41aba90 con 0x55596520ec20 2024-09-19T09:26:27.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.690+0000 7f02b92b9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555965211370 0x555965228d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:27.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.690+0000 7f02b8ab8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559652065d0 0x55596521d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:27.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.690+0000 7f02b3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55596520ec20 0x55596521d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:27.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.690+0000 7f02b8ab8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559652065d0 0x55596521d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:55280/0 (socket says 172.21.15.120:55280) 2024-09-19T09:26:27.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.690+0000 7f02b8ab8640 1 -- 172.21.15.120:0/2508016894 learned_addr learned my addr 172.21.15.120:0/2508016894 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:27.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.690+0000 7f02b92b9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555965211370 0x555965228d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56222/0 (socket says 172.21.15.120:56222) 2024-09-19T09:26:27.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.691+0000 7f02b8ab8640 1 -- 172.21.15.120:0/2508016894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55596520ec20 msgr2=0x55596521d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:27.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.691+0000 7f02b8ab8640 1 --2- 172.21.15.120:0/2508016894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55596520ec20 0x55596521d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.691+0000 7f02b8ab8640 1 -- 172.21.15.120:0/2508016894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555965211370 msgr2=0x555965228d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:27.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.691+0000 7f02b8ab8640 1 --2- 172.21.15.120:0/2508016894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555965211370 0x555965228d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.691+0000 7f02b8ab8640 1 -- 172.21.15.120:0/2508016894 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555965209120 con 0x5559652065d0 2024-09-19T09:26:27.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.691+0000 7f02b92b9640 1 --2- 172.21.15.120:0/2508016894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555965211370 0x555965228d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:27.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.691+0000 7f02b3fff640 1 --2- 172.21.15.120:0/2508016894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55596520ec20 0x55596521d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:27.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.691+0000 7f02b8ab8640 1 --2- 172.21.15.120:0/2508016894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559652065d0 0x55596521d2b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f02a8006010 tx=0x7f02a8009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7a1f986978551a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:27.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.691+0000 7f029e7fc640 1 -- 172.21.15.120:0/2508016894 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a8058070 con 0x5559652065d0 2024-09-19T09:26:27.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.691+0000 7f029e7fc640 1 -- 172.21.15.120:0/2508016894 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02a8004240 con 0x5559652065d0 2024-09-19T09:26:27.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.691+0000 7f029e7fc640 1 -- 172.21.15.120:0/2508016894 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a804d070 con 0x5559652065d0 2024-09-19T09:26:27.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.692+0000 7f02bd0f5740 1 -- 172.21.15.120:0/2508016894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559652065d0 msgr2=0x55596521d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:27.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.692+0000 7f02bd0f5740 1 --2- 172.21.15.120:0/2508016894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559652065d0 0x55596521d2b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f02a8006010 tx=0x7f02a8009620 comp rx=0 tx=0).stop 2024-09-19T09:26:27.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.692+0000 7f02bd0f5740 1 -- 172.21.15.120:0/2508016894 shutdown_connections 2024-09-19T09:26:27.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.692+0000 7f02bd0f5740 1 --2- 172.21.15.120:0/2508016894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555965211370 0x555965228d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.692+0000 7f02bd0f5740 1 --2- 172.21.15.120:0/2508016894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55596520ec20 0x55596521d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.692+0000 7f02bd0f5740 1 --2- 172.21.15.120:0/2508016894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559652065d0 0x55596521d2b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.692+0000 7f02bd0f5740 1 -- 172.21.15.120:0/2508016894 >> 172.21.15.120:0/2508016894 conn(0x5559651cf310 msgr2=0x55596520d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:27.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.692+0000 7f02bd0f5740 1 -- 172.21.15.120:0/2508016894 shutdown_connections 2024-09-19T09:26:27.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.692+0000 7f02bd0f5740 1 -- 172.21.15.120:0/2508016894 wait complete. 2024-09-19T09:26:27.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.692+0000 7f02bd0f5740 1 -- start start 2024-09-19T09:26:27.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.693+0000 7f02bd0f5740 1 Processor -- start 2024-09-19T09:26:27.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.693+0000 7f02bd0f5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559652065d0 0x5559652197c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:27.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.693+0000 7f02bd0f5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55596520ec20 0x555965219d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:27.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.693+0000 7f02bd0f5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555965211370 0x555965214cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:27.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.693+0000 7f02bd0f5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f02b41ac830 con 0x555965211370 2024-09-19T09:26:27.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.693+0000 7f02bd0f5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f02b41ac160 con 0x55596520ec20 2024-09-19T09:26:27.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.693+0000 7f02bd0f5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f02b41a9380 con 0x5559652065d0 2024-09-19T09:26:27.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.693+0000 7f02bd0f5740 10 client.0 _ll_register_callbacks cb 0x7f02a00056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:27.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.693+0000 7f02b3fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55596520ec20 0x555965219d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:27.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.693+0000 7f02b3fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55596520ec20 0x555965219d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:55296/0 (socket says 172.21.15.120:55296) 2024-09-19T09:26:27.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.693+0000 7f02b3fff640 1 -- 172.21.15.120:0/1991526222 learned_addr learned my addr 172.21.15.120:0/1991526222 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:27.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.693+0000 7f02b8ab8640 1 --2- 172.21.15.120:0/1991526222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559652065d0 0x5559652197c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:27.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.694+0000 7f02b3fff640 1 -- 172.21.15.120:0/1991526222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559652065d0 msgr2=0x5559652197c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:27.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.694+0000 7f02b3fff640 1 --2- 172.21.15.120:0/1991526222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559652065d0 0x5559652197c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.694+0000 7f02b3fff640 1 -- 172.21.15.120:0/1991526222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555965211370 msgr2=0x555965214cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:27.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.694+0000 7f02b3fff640 1 --2- 172.21.15.120:0/1991526222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555965211370 0x555965214cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.694+0000 7f02b3fff640 1 -- 172.21.15.120:0/1991526222 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555965215520 con 0x55596520ec20 2024-09-19T09:26:27.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.694+0000 7f02b3fff640 1 --2- 172.21.15.120:0/1991526222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55596520ec20 0x555965219d00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f02a0098c60 tx=0x7f02a00bc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89785b5bd38c9380 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:27.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.694+0000 7f0293fff640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:26:27.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.694+0000 7f0293fff640 1 -- 172.21.15.120:0/1991526222 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a00ba060 con 0x55596520ec20 2024-09-19T09:26:27.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.694+0000 7f02bd0f5740 1 -- 172.21.15.120:0/1991526222 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02a8004240 con 0x55596520ec20 2024-09-19T09:26:27.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.694+0000 7f0293fff640 1 -- 172.21.15.120:0/1991526222 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f02a009a590 con 0x55596520ec20 2024-09-19T09:26:27.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.694+0000 7f0293fff640 1 -- 172.21.15.120:0/1991526222 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f02a0005250 con 0x55596520ec20 2024-09-19T09:26:27.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.694+0000 7f02bd0f5740 10 client.31718 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:27.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.694+0000 7f02bd0f5740 1 -- 172.21.15.120:0/1991526222 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55596528b990 con 0x55596520ec20 2024-09-19T09:26:27.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.694+0000 7f0293fff640 1 -- 172.21.15.120:0/1991526222 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02a8004240 con 0x55596520ec20 2024-09-19T09:26:27.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.695+0000 7f0293fff640 1 -- 172.21.15.120:0/1991526222 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f02a00a4a70 con 0x55596520ec20 2024-09-19T09:26:27.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.695+0000 7f02bd0f5740 10 client.31718 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:27.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.695+0000 7f02bd0f5740 10 client.31718 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:26:27.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.695+0000 7f02bd0f5740 20 client.31718 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:27.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.695+0000 7f02bd0f5740 1 --2- 172.21.15.120:0/1991526222 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x555965221370 0x55596528e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:27.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.695+0000 7f02bd0f5740 4 client.31718 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:26:27.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.695+0000 7f02bd0f5740 1 -- 172.21.15.120:0/1991526222 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555965173d70 con 0x555965221370 2024-09-19T09:26:27.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.695+0000 7f02b8ab8640 1 --2- 172.21.15.120:0/1991526222 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x555965221370 0x55596528e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:27.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.695+0000 7f0293fff640 10 client.31718 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:26:27.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.695+0000 7f02b8ab8640 1 --2- 172.21.15.120:0/1991526222 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x555965221370 0x55596528e130 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ac06df5d3e4be587 server_cookie=8034e1671ea32bd3 in_seq=0 out_seq=0 2024-09-19T09:26:27.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.696+0000 7f0293fff640 1 -- 172.21.15.120:0/1991526222 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555965173d70 con 0x555965221370 2024-09-19T09:26:27.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.696+0000 7f0293fff640 10 client.31718 handle_command_reply: tid=0 2024-09-19T09:26:27.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.696+0000 7f02bd0f5740 1 client.31718 shutdown 2024-09-19T09:26:27.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.696+0000 7f02bd0f5740 1 -- 172.21.15.120:0/1991526222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55596520ec20 msgr2=0x555965219d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:27.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.696+0000 7f02bd0f5740 1 --2- 172.21.15.120:0/1991526222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55596520ec20 0x555965219d00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f02a0098c60 tx=0x7f02a00bc040 comp rx=0 tx=0).stop 2024-09-19T09:26:27.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.696+0000 7f02bd0f5740 1 -- 172.21.15.120:0/1991526222 shutdown_connections 2024-09-19T09:26:27.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.696+0000 7f02bd0f5740 1 --2- 172.21.15.120:0/1991526222 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x555965221370 0x55596528e130 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.696+0000 7f02bd0f5740 1 --2- 172.21.15.120:0/1991526222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555965211370 0x555965214cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.696+0000 7f0293fff640 0 client.31718 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:26:27.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.696+0000 7f02bd0f5740 1 --2- 172.21.15.120:0/1991526222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55596520ec20 0x555965219d00 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.696+0000 7f02bd0f5740 1 --2- 172.21.15.120:0/1991526222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559652065d0 0x5559652197c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.696+0000 7f02bd0f5740 1 -- 172.21.15.120:0/1991526222 >> 172.21.15.120:0/1991526222 conn(0x5559651cf310 msgr2=0x55596520c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:27.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.696+0000 7f02bd0f5740 1 -- 172.21.15.120:0/1991526222 shutdown_connections 2024-09-19T09:26:27.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.696+0000 7f02bd0f5740 1 -- 172.21.15.120:0/1991526222 wait complete. 2024-09-19T09:26:27.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.697+0000 7f02bd0f5740 20 client.31718 trim_cache size 0 max 16384 2024-09-19T09:26:27.708 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:26:27.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.697+0000 7f02bad1c640 1 -- 172.21.15.120:0/2790938575 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f02a40a32c0 msgr2=0x7f02a40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:27.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.697+0000 7f02bad1c640 1 --2- 172.21.15.120:0/2790938575 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f02a40a32c0 0x7f02a40a5780 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f02ac005fd0 tx=0x7f02ac009500 comp rx=0 tx=0).stop 2024-09-19T09:26:27.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.697+0000 7f02bad1c640 1 -- 172.21.15.120:0/2790938575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02b4101430 msgr2=0x7f02b4199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:27.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.697+0000 7f02bad1c640 1 --2- 172.21.15.120:0/2790938575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02b4101430 0x7f02b4199860 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f02a000e9f0 tx=0x7f02a000edf0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.697+0000 7f02bad1c640 1 -- 172.21.15.120:0/2790938575 shutdown_connections 2024-09-19T09:26:27.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.697+0000 7f02bad1c640 1 --2- 172.21.15.120:0/2790938575 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f02a40a32c0 0x7f02a40a5780 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.697+0000 7f02bad1c640 1 --2- 172.21.15.120:0/2790938575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02b4106a90 0x7f02b41a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.697+0000 7f02bad1c640 1 --2- 172.21.15.120:0/2790938575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02b4101430 0x7f02b4199860 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.697+0000 7f02bad1c640 1 --2- 172.21.15.120:0/2790938575 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f02b4100a90 0x7f02b4199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.697+0000 7f02bad1c640 1 -- 172.21.15.120:0/2790938575 >> 172.21.15.120:0/2790938575 conn(0x7f02b40fc7d0 msgr2=0x7f02b40fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:27.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.698+0000 7f02bad1c640 1 -- 172.21.15.120:0/2790938575 shutdown_connections 2024-09-19T09:26:27.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:27.698+0000 7f02bad1c640 1 -- 172.21.15.120:0/2790938575 wait complete. 2024-09-19T09:26:28.386 INFO:tasks.workunit.client.0.smithi120.stdout: 10 250679 20.92 MB/sec execute 503 sec latency 235.481 ms 2024-09-19T09:26:28.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:28 smithi196 ceph-mon[35150]: pgmap v1053: 97 pgs: 97 active+clean; 495 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.2 MiB/s wr, 672 op/s 2024-09-19T09:26:28.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:28 smithi153 ceph-mon[35071]: pgmap v1053: 97 pgs: 97 active+clean; 495 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.2 MiB/s wr, 672 op/s 2024-09-19T09:26:28.512 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:28 smithi120 ceph-mon[31150]: pgmap v1053: 97 pgs: 97 active+clean; 495 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.2 MiB/s wr, 672 op/s 2024-09-19T09:26:28.541 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:26:28.590 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.085+0000 7f2551fbd640 1 -- 172.21.15.120:0/227075149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f254c107830 msgr2=0x7f254c109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.086+0000 7f2551fbd640 1 --2- 172.21.15.120:0/227075149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f254c107830 0x7f254c109ca0 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f2544009ca0 tx=0x7f254402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.086+0000 7f2551fbd640 1 -- 172.21.15.120:0/227075149 shutdown_connections 2024-09-19T09:26:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.086+0000 7f2551fbd640 1 --2- 172.21.15.120:0/227075149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f254c107830 0x7f254c109ca0 unknown :-1 s=CLOSED pgs=2385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.086+0000 7f2551fbd640 1 --2- 172.21.15.120:0/227075149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f254c104f00 0x7f254c1072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.086+0000 7f2551fbd640 1 --2- 172.21.15.120:0/227075149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f254c1025d0 0x7f254c1049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.086+0000 7f2551fbd640 1 -- 172.21.15.120:0/227075149 >> 172.21.15.120:0/227075149 conn(0x7f254c0fc1b0 msgr2=0x7f254c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.086+0000 7f2551fbd640 1 -- 172.21.15.120:0/227075149 shutdown_connections 2024-09-19T09:26:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.086+0000 7f2551fbd640 1 -- 172.21.15.120:0/227075149 wait complete. 2024-09-19T09:26:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.086+0000 7f2551fbd640 1 Processor -- start 2024-09-19T09:26:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.087+0000 7f2551fbd640 1 -- start start 2024-09-19T09:26:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.087+0000 7f2551fbd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f254c1025d0 0x7f254c197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.087+0000 7f2551fbd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f254c104f00 0x7f254c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.087+0000 7f2551fbd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f254c107830 0x7f254c19f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.087+0000 7f2551fbd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f254c10d810 con 0x7f254c104f00 2024-09-19T09:26:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.087+0000 7f2551fbd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f254c10d690 con 0x7f254c1025d0 2024-09-19T09:26:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.087+0000 7f2551fbd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f254c10d990 con 0x7f254c107830 2024-09-19T09:26:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.087+0000 7f254affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f254c104f00 0x7f254c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.087+0000 7f254affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f254c104f00 0x7f254c1983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56246/0 (socket says 172.21.15.120:56246) 2024-09-19T09:26:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.087+0000 7f254affd640 1 -- 172.21.15.120:0/788179885 learned_addr learned my addr 172.21.15.120:0/788179885 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.087+0000 7f254bfff640 1 --2- 172.21.15.120:0/788179885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f254c107830 0x7f254c19f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.087+0000 7f254b7fe640 1 --2- 172.21.15.120:0/788179885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f254c1025d0 0x7f254c197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.087+0000 7f254affd640 1 -- 172.21.15.120:0/788179885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f254c107830 msgr2=0x7f254c19f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.087+0000 7f254affd640 1 --2- 172.21.15.120:0/788179885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f254c107830 0x7f254c19f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.087+0000 7f254affd640 1 -- 172.21.15.120:0/788179885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f254c1025d0 msgr2=0x7f254c197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.087+0000 7f254affd640 1 --2- 172.21.15.120:0/788179885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f254c1025d0 0x7f254c197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.087+0000 7f254affd640 1 -- 172.21.15.120:0/788179885 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f254c19fa80 con 0x7f254c104f00 2024-09-19T09:26:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.088+0000 7f254affd640 1 --2- 172.21.15.120:0/788179885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f254c104f00 0x7f254c1983a0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f254000e9c0 tx=0x7f254000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.088+0000 7f2548ff9640 1 -- 172.21.15.120:0/788179885 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254000cd80 con 0x7f254c104f00 2024-09-19T09:26:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.088+0000 7f2551fbd640 1 -- 172.21.15.120:0/788179885 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f254c19fd70 con 0x7f254c104f00 2024-09-19T09:26:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.088+0000 7f2548ff9640 1 -- 172.21.15.120:0/788179885 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2540005500 con 0x7f254c104f00 2024-09-19T09:26:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.088+0000 7f2548ff9640 1 -- 172.21.15.120:0/788179885 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2540010690 con 0x7f254c104f00 2024-09-19T09:26:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.088+0000 7f2551fbd640 1 -- 172.21.15.120:0/788179885 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f254c1a0230 con 0x7f254c104f00 2024-09-19T09:26:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.090+0000 7f2548ff9640 1 -- 172.21.15.120:0/788179885 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f25400046c0 con 0x7f254c104f00 2024-09-19T09:26:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.090+0000 7f2554e32740 1 Processor -- start 2024-09-19T09:26:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.091+0000 7f2554e32740 1 -- start start 2024-09-19T09:26:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.091+0000 7f2554e32740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56391b5c09a0 0x56391b5c0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.091+0000 7f2554e32740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56391b5c12e0 0x56391b5c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.091+0000 7f2554e32740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56391b5c6370 0x56391b5c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.091+0000 7f2554e32740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56391b5242f0 con 0x56391b5c09a0 2024-09-19T09:26:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.091+0000 7f2554e32740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56391b4cc270 con 0x56391b5c12e0 2024-09-19T09:26:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.091+0000 7f2554e32740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56391b5202b0 con 0x56391b5c6370 2024-09-19T09:26:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.091+0000 7f254bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56391b5c09a0 0x56391b5c0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.091+0000 7f254b7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56391b5c12e0 0x56391b5c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.091+0000 7f254b7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56391b5c12e0 0x56391b5c5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:55322/0 (socket says 172.21.15.120:55322) 2024-09-19T09:26:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.091+0000 7f254b7fe640 1 -- 172.21.15.120:0/2722602713 learned_addr learned my addr 172.21.15.120:0/2722602713 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.091+0000 7f254affd640 1 --2- 172.21.15.120:0/2722602713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56391b5c6370 0x56391b5c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.091+0000 7f2548ff9640 1 --2- 172.21.15.120:0/788179885 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f25340a32f0 0x7f25340a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.091+0000 7f254bfff640 1 -- 172.21.15.120:0/2722602713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56391b5c6370 msgr2=0x56391b5c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.091+0000 7f254bfff640 1 --2- 172.21.15.120:0/2722602713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56391b5c6370 0x56391b5c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.091+0000 7f254bfff640 1 -- 172.21.15.120:0/2722602713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56391b5c12e0 msgr2=0x56391b5c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.091+0000 7f254bfff640 1 --2- 172.21.15.120:0/2722602713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56391b5c12e0 0x56391b5c5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.091+0000 7f254bfff640 1 -- 172.21.15.120:0/2722602713 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56391b4cb340 con 0x56391b5c09a0 2024-09-19T09:26:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.091+0000 7f254b7fe640 1 --2- 172.21.15.120:0/2722602713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56391b5c12e0 0x56391b5c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.091+0000 7f2548ff9640 1 -- 172.21.15.120:0/788179885 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f25400cfc80 con 0x7f254c104f00 2024-09-19T09:26:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.092+0000 7f254bfff640 1 --2- 172.21.15.120:0/788179885 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f25340a32f0 0x7f25340a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.092+0000 7f254bfff640 1 --2- 172.21.15.120:0/2722602713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56391b5c09a0 0x56391b5c0da0 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7f254402ade0 tx=0x7f2544004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adc0f9d8f9a50325 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.092+0000 7f2531ffb640 1 -- 172.21.15.120:0/2722602713 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2544031d10 con 0x56391b5c09a0 2024-09-19T09:26:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.092+0000 7f2531ffb640 1 -- 172.21.15.120:0/2722602713 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2544036040 con 0x56391b5c09a0 2024-09-19T09:26:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.092+0000 7f254bfff640 1 --2- 172.21.15.120:0/788179885 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f25340a32f0 0x7f25340a57b0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f2544031ae0 tx=0x7f254404b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.092+0000 7f2531ffb640 1 -- 172.21.15.120:0/2722602713 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2544030ec0 con 0x56391b5c09a0 2024-09-19T09:26:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.093+0000 7f2554e32740 1 -- 172.21.15.120:0/2722602713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56391b5c09a0 msgr2=0x56391b5c0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.093+0000 7f2554e32740 1 --2- 172.21.15.120:0/2722602713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56391b5c09a0 0x56391b5c0da0 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7f254402ade0 tx=0x7f2544004a40 comp rx=0 tx=0).stop 2024-09-19T09:26:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.093+0000 7f2554e32740 1 -- 172.21.15.120:0/2722602713 shutdown_connections 2024-09-19T09:26:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.093+0000 7f2554e32740 1 --2- 172.21.15.120:0/2722602713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56391b5c6370 0x56391b5c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.093+0000 7f2554e32740 1 --2- 172.21.15.120:0/2722602713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56391b5c12e0 0x56391b5c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.093+0000 7f2554e32740 1 --2- 172.21.15.120:0/2722602713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56391b5c09a0 0x56391b5c0da0 unknown :-1 s=CLOSED pgs=2387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.093+0000 7f2554e32740 1 -- 172.21.15.120:0/2722602713 >> 172.21.15.120:0/2722602713 conn(0x56391b5bc010 msgr2=0x56391b5bc410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.093+0000 7f2554e32740 1 -- 172.21.15.120:0/2722602713 shutdown_connections 2024-09-19T09:26:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.093+0000 7f2554e32740 1 -- 172.21.15.120:0/2722602713 wait complete. 2024-09-19T09:26:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.094+0000 7f2554e32740 1 -- start start 2024-09-19T09:26:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.094+0000 7f2554e32740 1 Processor -- start 2024-09-19T09:26:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.094+0000 7f2554e32740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56391b5c09a0 0x56391b5ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.094+0000 7f2554e32740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56391b5c12e0 0x56391b5caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.095+0000 7f2554e32740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56391b5c6370 0x56391b5d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.095+0000 7f2554e32740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56391b5c9160 con 0x56391b5c6370 2024-09-19T09:26:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.095+0000 7f2554e32740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56391b4ad290 con 0x56391b5c09a0 2024-09-19T09:26:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.095+0000 7f2554e32740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56391b5c92e0 con 0x56391b5c12e0 2024-09-19T09:26:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.095+0000 7f2554e32740 10 client.0 _ll_register_callbacks cb 0x56391b4de140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.095+0000 7f254bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56391b5c6370 0x56391b5d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.095+0000 7f254affd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56391b5c12e0 0x56391b5caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.095+0000 7f254b7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56391b5c09a0 0x56391b5ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.095+0000 7f254bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56391b5c6370 0x56391b5d1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56260/0 (socket says 172.21.15.120:56260) 2024-09-19T09:26:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.095+0000 7f254bfff640 1 -- 172.21.15.120:0/4077256722 learned_addr learned my addr 172.21.15.120:0/4077256722 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.095+0000 7f254bfff640 1 -- 172.21.15.120:0/4077256722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56391b5c12e0 msgr2=0x56391b5caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.095+0000 7f254bfff640 1 --2- 172.21.15.120:0/4077256722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56391b5c12e0 0x56391b5caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.095+0000 7f254bfff640 1 -- 172.21.15.120:0/4077256722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56391b5c09a0 msgr2=0x56391b5ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.095+0000 7f254bfff640 1 --2- 172.21.15.120:0/4077256722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56391b5c09a0 0x56391b5ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.095+0000 7f254bfff640 1 -- 172.21.15.120:0/4077256722 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56391b237210 con 0x56391b5c6370 2024-09-19T09:26:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.095+0000 7f254bfff640 1 --2- 172.21.15.120:0/4077256722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56391b5c6370 0x56391b5d1f60 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f2544009bf0 tx=0x7f254402b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c2eb5676c651e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.095+0000 7f252affd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:26:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.095+0000 7f252affd640 1 -- 172.21.15.120:0/4077256722 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254405e060 con 0x56391b5c6370 2024-09-19T09:26:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.095+0000 7f252affd640 1 -- 172.21.15.120:0/4077256722 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f254402fa30 con 0x56391b5c6370 2024-09-19T09:26:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.095+0000 7f2554e32740 1 -- 172.21.15.120:0/4077256722 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2544036040 con 0x56391b5c6370 2024-09-19T09:26:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.096+0000 7f252affd640 1 -- 172.21.15.120:0/4077256722 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f254402fc30 con 0x56391b5c6370 2024-09-19T09:26:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.096+0000 7f2554e32740 10 client.23058 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.096+0000 7f2554e32740 1 -- 172.21.15.120:0/4077256722 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56391b303620 con 0x56391b5c6370 2024-09-19T09:26:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.096+0000 7f252affd640 1 -- 172.21.15.120:0/4077256722 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2544036040 con 0x56391b5c6370 2024-09-19T09:26:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.096+0000 7f252affd640 1 -- 172.21.15.120:0/4077256722 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f2544063070 con 0x56391b5c6370 2024-09-19T09:26:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.096+0000 7f2554e32740 10 client.23058 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.096+0000 7f2554e32740 10 client.23058 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:26:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.096+0000 7f2554e32740 20 client.23058 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.096+0000 7f2554e32740 1 --2- 172.21.15.120:0/4077256722 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56391b5d62b0 0x56391b5d8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.096+0000 7f2554e32740 4 client.23058 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:26:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.096+0000 7f2554e32740 1 -- 172.21.15.120:0/4077256722 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56391b531a20 con 0x56391b5d62b0 2024-09-19T09:26:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.096+0000 7f254b7fe640 1 --2- 172.21.15.120:0/4077256722 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56391b5d62b0 0x56391b5d8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.097+0000 7f254b7fe640 1 --2- 172.21.15.120:0/4077256722 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56391b5d62b0 0x56391b5d8710 crc :-1 s=READY pgs=573 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e46dcfae392af0a server_cookie=c3520a617109fbe0 in_seq=0 out_seq=0 2024-09-19T09:26:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.097+0000 7f252affd640 10 client.23058 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.100+0000 7f252affd640 1 -- 172.21.15.120:0/4077256722 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56391b531a20 con 0x56391b5d62b0 2024-09-19T09:26:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.100+0000 7f252affd640 10 client.23058 handle_command_reply: tid=0 2024-09-19T09:26:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.100+0000 7f2554e32740 1 client.23058 shutdown 2024-09-19T09:26:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.100+0000 7f2554e32740 1 -- 172.21.15.120:0/4077256722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56391b5c6370 msgr2=0x56391b5d1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.100+0000 7f2554e32740 1 --2- 172.21.15.120:0/4077256722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56391b5c6370 0x56391b5d1f60 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f2544009bf0 tx=0x7f254402b040 comp rx=0 tx=0).stop 2024-09-19T09:26:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.100+0000 7f2554e32740 1 -- 172.21.15.120:0/4077256722 shutdown_connections 2024-09-19T09:26:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.100+0000 7f2554e32740 1 --2- 172.21.15.120:0/4077256722 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56391b5d62b0 0x56391b5d8710 crc :-1 s=READY pgs=573 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.100+0000 7f2554e32740 1 --2- 172.21.15.120:0/4077256722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56391b5c6370 0x56391b5d1f60 unknown :-1 s=CLOSED pgs=2388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.100+0000 7f2554e32740 1 --2- 172.21.15.120:0/4077256722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56391b5c12e0 0x56391b5caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.100+0000 7f2554e32740 1 --2- 172.21.15.120:0/4077256722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56391b5c09a0 0x56391b5ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.100+0000 7f2554e32740 1 -- 172.21.15.120:0/4077256722 >> 172.21.15.120:0/4077256722 conn(0x56391b5bc010 msgr2=0x56391b5bd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.100+0000 7f252affd640 0 client.23058 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.100+0000 7f2554e32740 1 -- 172.21.15.120:0/4077256722 shutdown_connections 2024-09-19T09:26:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.100+0000 7f2554e32740 1 -- 172.21.15.120:0/4077256722 wait complete. 2024-09-19T09:26:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.100+0000 7f2554e32740 20 client.23058 trim_cache size 0 max 16384 2024-09-19T09:26:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.116+0000 7f2554e32740 1 Processor -- start 2024-09-19T09:26:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.116+0000 7f2554e32740 1 -- start start 2024-09-19T09:26:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.116+0000 7f2554e32740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56391b5c09a0 0x56391b5bcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.116+0000 7f2554e32740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56391b5c12e0 0x56391b5d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.116+0000 7f2554e32740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56391b5c6370 0x56391b5bd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.116+0000 7f2554e32740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f254c1a90a0 con 0x56391b5c6370 2024-09-19T09:26:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.116+0000 7f2554e32740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f254c1aabe0 con 0x56391b5c12e0 2024-09-19T09:26:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.116+0000 7f2554e32740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f254c1a9770 con 0x56391b5c09a0 2024-09-19T09:26:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.116+0000 7f254bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56391b5c6370 0x56391b5bd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.116+0000 7f254bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56391b5c6370 0x56391b5bd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56264/0 (socket says 172.21.15.120:56264) 2024-09-19T09:26:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.116+0000 7f254bfff640 1 -- 172.21.15.120:0/2311237304 learned_addr learned my addr 172.21.15.120:0/2311237304 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.116+0000 7f254b7fe640 1 --2- 172.21.15.120:0/2311237304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56391b5c09a0 0x56391b5bcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.116+0000 7f254affd640 1 --2- 172.21.15.120:0/2311237304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56391b5c12e0 0x56391b5d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.116+0000 7f254bfff640 1 -- 172.21.15.120:0/2311237304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56391b5c09a0 msgr2=0x56391b5bcec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.116+0000 7f254bfff640 1 --2- 172.21.15.120:0/2311237304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56391b5c09a0 0x56391b5bcec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.116+0000 7f254bfff640 1 -- 172.21.15.120:0/2311237304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56391b5c12e0 msgr2=0x56391b5d1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.116+0000 7f254bfff640 1 --2- 172.21.15.120:0/2311237304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56391b5c12e0 0x56391b5d1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.116+0000 7f254bfff640 1 -- 172.21.15.120:0/2311237304 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56391b5d3400 con 0x56391b5c6370 2024-09-19T09:26:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.116+0000 7f254bfff640 1 --2- 172.21.15.120:0/2311237304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56391b5c6370 0x56391b5bd400 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f2544002230 tx=0x7f2544000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eee80f41cf0a19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.117+0000 7f25317fa640 1 -- 172.21.15.120:0/2311237304 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254405e070 con 0x56391b5c6370 2024-09-19T09:26:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.117+0000 7f25317fa640 1 -- 172.21.15.120:0/2311237304 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2544036040 con 0x56391b5c6370 2024-09-19T09:26:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.117+0000 7f25317fa640 1 -- 172.21.15.120:0/2311237304 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25440040b0 con 0x56391b5c6370 2024-09-19T09:26:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.117+0000 7f2554e32740 1 -- 172.21.15.120:0/2311237304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56391b5c6370 msgr2=0x56391b5bd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.117+0000 7f2554e32740 1 --2- 172.21.15.120:0/2311237304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56391b5c6370 0x56391b5bd400 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f2544002230 tx=0x7f2544000f40 comp rx=0 tx=0).stop 2024-09-19T09:26:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.117+0000 7f2554e32740 1 -- 172.21.15.120:0/2311237304 shutdown_connections 2024-09-19T09:26:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.117+0000 7f2554e32740 1 --2- 172.21.15.120:0/2311237304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56391b5c6370 0x56391b5bd400 unknown :-1 s=CLOSED pgs=2389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.117+0000 7f2554e32740 1 --2- 172.21.15.120:0/2311237304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56391b5c12e0 0x56391b5d1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.117+0000 7f2554e32740 1 --2- 172.21.15.120:0/2311237304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56391b5c09a0 0x56391b5bcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.117+0000 7f2554e32740 1 -- 172.21.15.120:0/2311237304 >> 172.21.15.120:0/2311237304 conn(0x56391b5bc010 msgr2=0x56391b5ca9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.118+0000 7f2554e32740 1 -- 172.21.15.120:0/2311237304 shutdown_connections 2024-09-19T09:26:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.118+0000 7f2554e32740 1 -- 172.21.15.120:0/2311237304 wait complete. 2024-09-19T09:26:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.118+0000 7f2554e32740 1 -- start start 2024-09-19T09:26:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.118+0000 7f2554e32740 1 Processor -- start 2024-09-19T09:26:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.118+0000 7f2554e32740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56391b5c09a0 0x56391b5cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.118+0000 7f2554e32740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56391b5c12e0 0x56391b5cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.118+0000 7f2554e32740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56391b5c6370 0x56391b5d0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.118+0000 7f2554e32740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f254c1aa510 con 0x56391b5c6370 2024-09-19T09:26:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.118+0000 7f2554e32740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f254c1a9e40 con 0x56391b5c09a0 2024-09-19T09:26:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.118+0000 7f2554e32740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f254c1a7060 con 0x56391b5c12e0 2024-09-19T09:26:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.118+0000 7f2554e32740 10 client.0 _ll_register_callbacks cb 0x7f253c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.119+0000 7f254bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56391b5c6370 0x56391b5d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.119+0000 7f254b7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56391b5c09a0 0x56391b5cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.119+0000 7f254bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56391b5c6370 0x56391b5d0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56280/0 (socket says 172.21.15.120:56280) 2024-09-19T09:26:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.119+0000 7f254b7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56391b5c09a0 0x56391b5cf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:55348/0 (socket says 172.21.15.120:55348) 2024-09-19T09:26:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.119+0000 7f254bfff640 1 -- 172.21.15.120:0/3058059934 learned_addr learned my addr 172.21.15.120:0/3058059934 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.119+0000 7f254affd640 1 --2- 172.21.15.120:0/3058059934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56391b5c12e0 0x56391b5cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.119+0000 7f254bfff640 1 -- 172.21.15.120:0/3058059934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56391b5c12e0 msgr2=0x56391b5cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.119+0000 7f254bfff640 1 --2- 172.21.15.120:0/3058059934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56391b5c12e0 0x56391b5cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.119+0000 7f254bfff640 1 -- 172.21.15.120:0/3058059934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56391b5c09a0 msgr2=0x56391b5cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.119+0000 7f254bfff640 1 --2- 172.21.15.120:0/3058059934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56391b5c09a0 0x56391b5cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.119+0000 7f254bfff640 1 -- 172.21.15.120:0/3058059934 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56391b5d0b50 con 0x56391b5c6370 2024-09-19T09:26:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.119+0000 7f254b7fe640 1 --2- 172.21.15.120:0/3058059934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56391b5c09a0 0x56391b5cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.119+0000 7f254bfff640 1 --2- 172.21.15.120:0/3058059934 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56391b5c6370 0x56391b5d0540 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f254402ade0 tx=0x7f2544031bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802420e7874a4867 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.119+0000 7f252affd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:26:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.119+0000 7f252affd640 1 -- 172.21.15.120:0/3058059934 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2544069070 con 0x56391b5c6370 2024-09-19T09:26:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.119+0000 7f252affd640 1 -- 172.21.15.120:0/3058059934 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f2544003b80 con 0x56391b5c6370 2024-09-19T09:26:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.119+0000 7f2554e32740 1 -- 172.21.15.120:0/3058059934 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2544036040 con 0x56391b5c6370 2024-09-19T09:26:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.120+0000 7f252affd640 1 -- 172.21.15.120:0/3058059934 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f254405e070 con 0x56391b5c6370 2024-09-19T09:26:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.120+0000 7f2554e32740 10 client.23067 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.120+0000 7f2554e32740 1 -- 172.21.15.120:0/3058059934 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56391b5d1e60 con 0x56391b5c6370 2024-09-19T09:26:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.120+0000 7f252affd640 1 -- 172.21.15.120:0/3058059934 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2544036040 con 0x56391b5c6370 2024-09-19T09:26:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.120+0000 7f252affd640 1 -- 172.21.15.120:0/3058059934 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f2544063070 con 0x56391b5c6370 2024-09-19T09:26:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.120+0000 7f2554e32740 10 client.23067 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.120+0000 7f2554e32740 10 client.23067 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:26:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.120+0000 7f2554e32740 20 client.23067 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.120+0000 7f2554e32740 1 --2- 172.21.15.120:0/3058059934 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56391b641070 0x56391b643460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.120+0000 7f2554e32740 4 client.23067 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:26:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.120+0000 7f2554e32740 1 -- 172.21.15.120:0/3058059934 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56391b52bd70 con 0x56391b641070 2024-09-19T09:26:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.120+0000 7f254b7fe640 1 --2- 172.21.15.120:0/3058059934 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56391b641070 0x56391b643460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.121+0000 7f252affd640 10 client.23067 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.121+0000 7f254b7fe640 1 --2- 172.21.15.120:0/3058059934 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56391b641070 0x56391b643460 crc :-1 s=READY pgs=574 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3839917f36190a54 server_cookie=b207274a9f2f3ed6 in_seq=0 out_seq=0 2024-09-19T09:26:29.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.120+0000 7f935b684640 1 -- 172.21.15.120:0/511778614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9354102d40 msgr2=0x7f9354103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:29.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.120+0000 7f935b684640 1 --2- 172.21.15.120:0/511778614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9354102d40 0x7f9354103120 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f9350009900 tx=0x7f935002a480 comp rx=0 tx=0).stop 2024-09-19T09:26:29.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.121+0000 7f935b684640 1 -- 172.21.15.120:0/511778614 shutdown_connections 2024-09-19T09:26:29.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.121+0000 7f935b684640 1 --2- 172.21.15.120:0/511778614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f935410c170 0x7f935410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.121+0000 7f935b684640 1 --2- 172.21.15.120:0/511778614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9354103660 0x7f935410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.121+0000 7f935b684640 1 --2- 172.21.15.120:0/511778614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9354102d40 0x7f9354103120 unknown :-1 s=CLOSED pgs=2391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.121+0000 7f935b684640 1 -- 172.21.15.120:0/511778614 >> 172.21.15.120:0/511778614 conn(0x7f93540fc7a0 msgr2=0x7f93540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:29.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.121+0000 7f935b684640 1 -- 172.21.15.120:0/511778614 shutdown_connections 2024-09-19T09:26:29.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.121+0000 7f935b684640 1 -- 172.21.15.120:0/511778614 wait complete. 2024-09-19T09:26:29.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.121+0000 7f935b684640 1 Processor -- start 2024-09-19T09:26:29.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.121+0000 7f935b684640 1 -- start start 2024-09-19T09:26:29.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.122+0000 7f935b684640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9354102d40 0x7f9354198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.122+0000 7f935b684640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9354103660 0x7f93541989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.122+0000 7f935b684640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f935410c170 0x7f935419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.122+0000 7f935b684640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9354111f70 con 0x7f9354102d40 2024-09-19T09:26:29.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.122+0000 7f935b684640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9354111df0 con 0x7f9354103660 2024-09-19T09:26:29.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.122+0000 7f935b684640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f93541120f0 con 0x7f935410c170 2024-09-19T09:26:29.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.122+0000 7f9359420640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9354102d40 0x7f9354198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.122+0000 7f9359420640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9354102d40 0x7f9354198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56284/0 (socket says 172.21.15.120:56284) 2024-09-19T09:26:29.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.122+0000 7f9359420640 1 -- 172.21.15.120:0/3615444906 learned_addr learned my addr 172.21.15.120:0/3615444906 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:29.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.122+0000 7f9358c1f640 1 --2- 172.21.15.120:0/3615444906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9354103660 0x7f93541989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.122+0000 7f9359c21640 1 --2- 172.21.15.120:0/3615444906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f935410c170 0x7f935419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.122+0000 7f9359420640 1 -- 172.21.15.120:0/3615444906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f935410c170 msgr2=0x7f935419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:29.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.122+0000 7f9359420640 1 --2- 172.21.15.120:0/3615444906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f935410c170 0x7f935419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.122+0000 7f9359420640 1 -- 172.21.15.120:0/3615444906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9354103660 msgr2=0x7f93541989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:29.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.122+0000 7f9359420640 1 --2- 172.21.15.120:0/3615444906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9354103660 0x7f93541989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.122+0000 7f9359420640 1 -- 172.21.15.120:0/3615444906 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935419d420 con 0x7f9354102d40 2024-09-19T09:26:29.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.122+0000 7f9359420640 1 --2- 172.21.15.120:0/3615444906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9354102d40 0x7f9354198470 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f9350006010 tx=0x7f9350004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:29.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.122+0000 7f934a7fc640 1 -- 172.21.15.120:0/3615444906 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935003f030 con 0x7f9354102d40 2024-09-19T09:26:29.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.122+0000 7f935b684640 1 -- 172.21.15.120:0/3615444906 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f935419d6b0 con 0x7f9354102d40 2024-09-19T09:26:29.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.122+0000 7f934a7fc640 1 -- 172.21.15.120:0/3615444906 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9350004440 con 0x7f9354102d40 2024-09-19T09:26:29.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.122+0000 7f934a7fc640 1 -- 172.21.15.120:0/3615444906 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9350035070 con 0x7f9354102d40 2024-09-19T09:26:29.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.123+0000 7f935b684640 1 -- 172.21.15.120:0/3615444906 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93541a4ff0 con 0x7f9354102d40 2024-09-19T09:26:29.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.125+0000 7f934a7fc640 1 -- 172.21.15.120:0/3615444906 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f9350042030 con 0x7f9354102d40 2024-09-19T09:26:29.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.125+0000 7f935da5d740 1 Processor -- start 2024-09-19T09:26:29.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.125+0000 7f935da5d740 1 -- start start 2024-09-19T09:26:29.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.126+0000 7f935da5d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578371c4760 0x5578371c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.126+0000 7f935da5d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578371c5080 0x5578371cf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.126+0000 7f935da5d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578371cf940 0x5578371d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.126+0000 7f935da5d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55783712a2a0 con 0x5578371cf940 2024-09-19T09:26:29.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.126+0000 7f935da5d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5578370d1b10 con 0x5578371c4760 2024-09-19T09:26:29.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.126+0000 7f935da5d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557837126260 con 0x5578371c5080 2024-09-19T09:26:29.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.126+0000 7f934a7fc640 1 --2- 172.21.15.120:0/3615444906 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93400a3240 0x7f93400a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.126+0000 7f9359420640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578371c5080 0x5578371cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.126+0000 7f9358c1f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578371cf940 0x5578371d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.126+0000 7f9359c21640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578371c4760 0x5578371c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.126+0000 7f934a7fc640 1 -- 172.21.15.120:0/3615444906 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f93500efb90 con 0x7f9354102d40 2024-09-19T09:26:29.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.126+0000 7f9359c21640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578371c4760 0x5578371c4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:55362/0 (socket says 172.21.15.120:55362) 2024-09-19T09:26:29.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.126+0000 7f9359c21640 1 -- 172.21.15.120:0/1114599773 learned_addr learned my addr 172.21.15.120:0/1114599773 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:29.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.126+0000 7f9359c21640 1 --2- 172.21.15.120:0/3615444906 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93400a3240 0x7f93400a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.126+0000 7f9359420640 1 -- 172.21.15.120:0/1114599773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578371c4760 msgr2=0x5578371c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:29.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.126+0000 7f9359420640 1 --2- 172.21.15.120:0/1114599773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578371c4760 0x5578371c4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.126+0000 7f9359420640 1 -- 172.21.15.120:0/1114599773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578371cf940 msgr2=0x5578371d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:29.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.126+0000 7f9359420640 1 --2- 172.21.15.120:0/1114599773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578371cf940 0x5578371d1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.126+0000 7f9359420640 1 -- 172.21.15.120:0/1114599773 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557836e9bb90 con 0x5578371c5080 2024-09-19T09:26:29.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.126+0000 7f9359c21640 1 --2- 172.21.15.120:0/3615444906 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93400a3240 0x7f93400a5700 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f934c005f40 tx=0x7f934c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:29.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.126+0000 7f9359420640 1 --2- 172.21.15.120:0/1114599773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578371c5080 0x5578371cf400 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f93500bbeb0 tx=0x7f9350036450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ffb99381e23ad4c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:29.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.126+0000 7f933f7fe640 1 -- 172.21.15.120:0/1114599773 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9350049050 con 0x5578371c5080 2024-09-19T09:26:29.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.127+0000 7f933f7fe640 1 -- 172.21.15.120:0/1114599773 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93500bc8b0 con 0x5578371c5080 2024-09-19T09:26:29.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.127+0000 7f933f7fe640 1 -- 172.21.15.120:0/1114599773 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9350036260 con 0x5578371c5080 2024-09-19T09:26:29.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.127+0000 7f935da5d740 1 -- 172.21.15.120:0/1114599773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578371c5080 msgr2=0x5578371cf400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:29.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.127+0000 7f935da5d740 1 --2- 172.21.15.120:0/1114599773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578371c5080 0x5578371cf400 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f93500bbeb0 tx=0x7f9350036450 comp rx=0 tx=0).stop 2024-09-19T09:26:29.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.127+0000 7f935da5d740 1 -- 172.21.15.120:0/1114599773 shutdown_connections 2024-09-19T09:26:29.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.127+0000 7f935da5d740 1 --2- 172.21.15.120:0/1114599773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578371cf940 0x5578371d1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.127+0000 7f935da5d740 1 --2- 172.21.15.120:0/1114599773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578371c5080 0x5578371cf400 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.127+0000 7f935da5d740 1 --2- 172.21.15.120:0/1114599773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578371c4760 0x5578371c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.127+0000 7f935da5d740 1 -- 172.21.15.120:0/1114599773 >> 172.21.15.120:0/1114599773 conn(0x5578371c0cf0 msgr2=0x5578371c10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:29.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.127+0000 7f935da5d740 1 -- 172.21.15.120:0/1114599773 shutdown_connections 2024-09-19T09:26:29.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.127+0000 7f935da5d740 1 -- 172.21.15.120:0/1114599773 wait complete. 2024-09-19T09:26:29.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.128+0000 7f935da5d740 1 -- start start 2024-09-19T09:26:29.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.128+0000 7f935da5d740 1 Processor -- start 2024-09-19T09:26:29.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.128+0000 7f935da5d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578371c4760 0x5578371cc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.128+0000 7f935da5d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578371c5080 0x5578371ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.128+0000 7f935da5d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578371cf940 0x5578371cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.128+0000 7f935da5d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5578371d22d0 con 0x5578371cf940 2024-09-19T09:26:29.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.128+0000 7f935da5d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5578370b26e0 con 0x5578371c5080 2024-09-19T09:26:29.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.128+0000 7f935da5d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5578371d2450 con 0x5578371c4760 2024-09-19T09:26:29.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.128+0000 7f935da5d740 10 client.0 _ll_register_callbacks cb 0x5578370d6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:29.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.128+0000 7f9359c21640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578371cf940 0x5578371cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.128+0000 7f9359c21640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578371cf940 0x5578371cd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56310/0 (socket says 172.21.15.120:56310) 2024-09-19T09:26:29.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.128+0000 7f9359420640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578371c4760 0x5578371cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.128+0000 7f9359c21640 1 -- 172.21.15.120:0/2806629372 learned_addr learned my addr 172.21.15.120:0/2806629372 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:29.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.128+0000 7f9358c1f640 1 --2- 172.21.15.120:0/2806629372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578371c5080 0x5578371ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.128+0000 7f9359c21640 1 -- 172.21.15.120:0/2806629372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578371c4760 msgr2=0x5578371cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:29.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.129+0000 7f9359c21640 1 --2- 172.21.15.120:0/2806629372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578371c4760 0x5578371cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.129+0000 7f9359c21640 1 -- 172.21.15.120:0/2806629372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578371c5080 msgr2=0x5578371ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:29.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.129+0000 7f9359c21640 1 --2- 172.21.15.120:0/2806629372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578371c5080 0x5578371ccd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.129+0000 7f9359c21640 1 -- 172.21.15.120:0/2806629372 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578371db760 con 0x5578371cf940 2024-09-19T09:26:29.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.129+0000 7f9359420640 1 --2- 172.21.15.120:0/2806629372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578371c4760 0x5578371cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:29.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.129+0000 7f9359c21640 1 --2- 172.21.15.120:0/2806629372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578371cf940 0x5578371cd2d0 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7f934c009f80 tx=0x7f934c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8047121c72d1df37 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:29.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.129+0000 7f933cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:26:29.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.129+0000 7f933cff9640 1 -- 172.21.15.120:0/2806629372 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c02f060 con 0x5578371cf940 2024-09-19T09:26:29.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.129+0000 7f933cff9640 1 -- 172.21.15.120:0/2806629372 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f934c01a070 con 0x5578371cf940 2024-09-19T09:26:29.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.129+0000 7f935da5d740 1 -- 172.21.15.120:0/2806629372 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93500bc8b0 con 0x5578371cf940 2024-09-19T09:26:29.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.129+0000 7f933cff9640 1 -- 172.21.15.120:0/2806629372 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f934c01fda0 con 0x5578371cf940 2024-09-19T09:26:29.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.129+0000 7f935da5d740 10 client.23076 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:29.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.129+0000 7f935da5d740 1 -- 172.21.15.120:0/2806629372 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5578371de150 con 0x5578371cf940 2024-09-19T09:26:29.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.129+0000 7f933cff9640 1 -- 172.21.15.120:0/2806629372 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93500bc8b0 con 0x5578371cf940 2024-09-19T09:26:29.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.129+0000 7f933cff9640 1 -- 172.21.15.120:0/2806629372 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f934c034070 con 0x5578371cf940 2024-09-19T09:26:29.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.130+0000 7f935da5d740 10 client.23076 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:29.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.130+0000 7f935da5d740 10 client.23076 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:26:29.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.130+0000 7f935da5d740 20 client.23076 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:29.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.130+0000 7f935da5d740 1 --2- 172.21.15.120:0/2806629372 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5578371df610 0x5578371e1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.130+0000 7f935da5d740 4 client.23076 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:26:29.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.130+0000 7f935da5d740 1 -- 172.21.15.120:0/2806629372 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578371379d0 con 0x5578371df610 2024-09-19T09:26:29.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.130+0000 7f9359420640 1 --2- 172.21.15.120:0/2806629372 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5578371df610 0x5578371e1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.130+0000 7f933cff9640 10 client.23076 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:29.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.130+0000 7f9359420640 1 --2- 172.21.15.120:0/2806629372 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5578371df610 0x5578371e1a90 crc :-1 s=READY pgs=575 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0b767284ecffb9b server_cookie=a37177d24f3559 in_seq=0 out_seq=0 2024-09-19T09:26:29.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.136+0000 7f933cff9640 1 -- 172.21.15.120:0/2806629372 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5578371379d0 con 0x5578371df610 2024-09-19T09:26:29.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.136+0000 7f933cff9640 10 client.23076 handle_command_reply: tid=0 2024-09-19T09:26:29.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.136+0000 7f935da5d740 1 client.23076 shutdown 2024-09-19T09:26:29.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.136+0000 7f935da5d740 1 -- 172.21.15.120:0/2806629372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578371cf940 msgr2=0x5578371cd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:29.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.136+0000 7f935da5d740 1 --2- 172.21.15.120:0/2806629372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578371cf940 0x5578371cd2d0 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7f934c009f80 tx=0x7f934c004ab0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.136+0000 7f935da5d740 1 -- 172.21.15.120:0/2806629372 shutdown_connections 2024-09-19T09:26:29.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.136+0000 7f935da5d740 1 --2- 172.21.15.120:0/2806629372 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5578371df610 0x5578371e1a90 crc :-1 s=READY pgs=575 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.136+0000 7f935da5d740 1 --2- 172.21.15.120:0/2806629372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578371cf940 0x5578371cd2d0 unknown :-1 s=CLOSED pgs=2393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.136+0000 7f935da5d740 1 --2- 172.21.15.120:0/2806629372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578371c5080 0x5578371ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.136+0000 7f933cff9640 0 client.23076 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:29.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.136+0000 7f935da5d740 1 --2- 172.21.15.120:0/2806629372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578371c4760 0x5578371cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.136+0000 7f935da5d740 1 -- 172.21.15.120:0/2806629372 >> 172.21.15.120:0/2806629372 conn(0x5578371c0cf0 msgr2=0x5578371bfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:29.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.136+0000 7f935da5d740 1 -- 172.21.15.120:0/2806629372 shutdown_connections 2024-09-19T09:26:29.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.136+0000 7f935da5d740 1 -- 172.21.15.120:0/2806629372 wait complete. 2024-09-19T09:26:29.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.136+0000 7f935da5d740 20 client.23076 trim_cache size 0 max 16384 2024-09-19T09:26:29.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.153+0000 7f935da5d740 1 Processor -- start 2024-09-19T09:26:29.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.153+0000 7f935da5d740 1 -- start start 2024-09-19T09:26:29.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.153+0000 7f935da5d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578371c4760 0x5578371c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.153+0000 7f935da5d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578371c5080 0x5578371e7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.153+0000 7f935da5d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578371cf940 0x5578371e78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.153+0000 7f935da5d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f93541ab4b0 con 0x5578371cf940 2024-09-19T09:26:29.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.153+0000 7f935da5d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f93541acff0 con 0x5578371c4760 2024-09-19T09:26:29.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.153+0000 7f935da5d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f93541abb80 con 0x5578371c5080 2024-09-19T09:26:29.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.153+0000 7f9359c21640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578371cf940 0x5578371e78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.153+0000 7f9358c1f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578371c5080 0x5578371e7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.153+0000 7f9359c21640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578371cf940 0x5578371e78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56320/0 (socket says 172.21.15.120:56320) 2024-09-19T09:26:29.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.153+0000 7f9359c21640 1 -- 172.21.15.120:0/1814279875 learned_addr learned my addr 172.21.15.120:0/1814279875 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:29.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.154+0000 7f9359420640 1 --2- 172.21.15.120:0/1814279875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578371c4760 0x5578371c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.154+0000 7f9359c21640 1 -- 172.21.15.120:0/1814279875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578371c5080 msgr2=0x5578371e7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:29.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.154+0000 7f9359c21640 1 --2- 172.21.15.120:0/1814279875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578371c5080 0x5578371e7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.154+0000 7f9359c21640 1 -- 172.21.15.120:0/1814279875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578371c4760 msgr2=0x5578371c68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:29.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.154+0000 7f9359c21640 1 --2- 172.21.15.120:0/1814279875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578371c4760 0x5578371c68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.154+0000 7f9359c21640 1 -- 172.21.15.120:0/1814279875 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578371bc260 con 0x5578371cf940 2024-09-19T09:26:29.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.154+0000 7f9359c21640 1 --2- 172.21.15.120:0/1814279875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578371cf940 0x5578371e78a0 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f934c005c40 tx=0x7f934c0118c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c86627ac34bc15 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:29.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.154+0000 7f933effd640 1 -- 172.21.15.120:0/1814279875 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c03a070 con 0x5578371cf940 2024-09-19T09:26:29.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.154+0000 7f933effd640 1 -- 172.21.15.120:0/1814279875 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f934c011030 con 0x5578371cf940 2024-09-19T09:26:29.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.154+0000 7f933effd640 1 -- 172.21.15.120:0/1814279875 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c01da80 con 0x5578371cf940 2024-09-19T09:26:29.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.155+0000 7f935da5d740 1 -- 172.21.15.120:0/1814279875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578371cf940 msgr2=0x5578371e78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:29.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.155+0000 7f935da5d740 1 --2- 172.21.15.120:0/1814279875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578371cf940 0x5578371e78a0 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f934c005c40 tx=0x7f934c0118c0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.155+0000 7f935da5d740 1 -- 172.21.15.120:0/1814279875 shutdown_connections 2024-09-19T09:26:29.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.155+0000 7f935da5d740 1 --2- 172.21.15.120:0/1814279875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578371cf940 0x5578371e78a0 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.155+0000 7f935da5d740 1 --2- 172.21.15.120:0/1814279875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578371c5080 0x5578371e7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.155+0000 7f935da5d740 1 --2- 172.21.15.120:0/1814279875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578371c4760 0x5578371c68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.155+0000 7f935da5d740 1 -- 172.21.15.120:0/1814279875 >> 172.21.15.120:0/1814279875 conn(0x5578371c0cf0 msgr2=0x5578371c61b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:29.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.155+0000 7f935da5d740 1 -- 172.21.15.120:0/1814279875 shutdown_connections 2024-09-19T09:26:29.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.155+0000 7f935da5d740 1 -- 172.21.15.120:0/1814279875 wait complete. 2024-09-19T09:26:29.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.155+0000 7f935da5d740 1 -- start start 2024-09-19T09:26:29.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.155+0000 7f935da5d740 1 Processor -- start 2024-09-19T09:26:29.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.156+0000 7f935da5d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578371c4760 0x5578371d77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.156+0000 7f935da5d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578371c5080 0x5578371d7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.156+0000 7f935da5d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578371cf940 0x5578371d45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.156+0000 7f935da5d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f93541ac920 con 0x5578371c4760 2024-09-19T09:26:29.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.156+0000 7f935da5d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f93541ac250 con 0x5578371c5080 2024-09-19T09:26:29.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.156+0000 7f935da5d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f93541a9470 con 0x5578371cf940 2024-09-19T09:26:29.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.156+0000 7f935da5d740 10 client.0 _ll_register_callbacks cb 0x7f934c0116b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:29.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.156+0000 7f9359420640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578371c4760 0x5578371d77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.156+0000 7f9359420640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578371c4760 0x5578371d77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56326/0 (socket says 172.21.15.120:56326) 2024-09-19T09:26:29.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.156+0000 7f9359420640 1 -- 172.21.15.120:0/3931422415 learned_addr learned my addr 172.21.15.120:0/3931422415 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:29.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.156+0000 7f9358c1f640 1 --2- 172.21.15.120:0/3931422415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578371c5080 0x5578371d7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.156+0000 7f9359c21640 1 --2- 172.21.15.120:0/3931422415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578371cf940 0x5578371d45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.156+0000 7f9359420640 1 -- 172.21.15.120:0/3931422415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578371cf940 msgr2=0x5578371d45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:29.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.156+0000 7f9359420640 1 --2- 172.21.15.120:0/3931422415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578371cf940 0x5578371d45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.156+0000 7f9359420640 1 -- 172.21.15.120:0/3931422415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578371c5080 msgr2=0x5578371d7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:29.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.156+0000 7f9359420640 1 --2- 172.21.15.120:0/3931422415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578371c5080 0x5578371d7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.156+0000 7f9359420640 1 -- 172.21.15.120:0/3931422415 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578371d4e10 con 0x5578371c4760 2024-09-19T09:26:29.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.157+0000 7f9359420640 1 --2- 172.21.15.120:0/3931422415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578371c4760 0x5578371d77f0 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7f93500b8b80 tx=0x7f9350009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5991f6bb15746587 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:29.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.157+0000 7f933cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:26:29.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.157+0000 7f933cff9640 1 -- 172.21.15.120:0/3931422415 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9350049040 con 0x5578371c4760 2024-09-19T09:26:29.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.157+0000 7f933cff9640 1 -- 172.21.15.120:0/3931422415 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f93500ed710 con 0x5578371c4760 2024-09-19T09:26:29.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.157+0000 7f935da5d740 1 -- 172.21.15.120:0/3931422415 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f934c011030 con 0x5578371c4760 2024-09-19T09:26:29.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.157+0000 7f933cff9640 1 -- 172.21.15.120:0/3931422415 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f93500bbcf0 con 0x5578371c4760 2024-09-19T09:26:29.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.157+0000 7f935da5d740 10 client.23082 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:29.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.157+0000 7f935da5d740 1 -- 172.21.15.120:0/3931422415 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55783724ae00 con 0x5578371c4760 2024-09-19T09:26:29.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.157+0000 7f933cff9640 1 -- 172.21.15.120:0/3931422415 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f934c011030 con 0x5578371c4760 2024-09-19T09:26:29.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.157+0000 7f933cff9640 1 -- 172.21.15.120:0/3931422415 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f93500b8820 con 0x5578371c4760 2024-09-19T09:26:29.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.157+0000 7f935da5d740 10 client.23082 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:29.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.157+0000 7f935da5d740 10 client.23082 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:26:29.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.157+0000 7f935da5d740 20 client.23082 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:29.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.157+0000 7f935da5d740 1 --2- 172.21.15.120:0/3931422415 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55783724b220 0x55783724d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:29.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.157+0000 7f935da5d740 4 client.23082 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.1, "await_for": 45.0}] 2024-09-19T09:26:29.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.157+0000 7f935da5d740 1 -- 172.21.15.120:0/3931422415 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.1, "await_for": 45.0}) -- 0x557837131d20 con 0x55783724b220 2024-09-19T09:26:29.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.158+0000 7f9358c1f640 1 --2- 172.21.15.120:0/3931422415 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55783724b220 0x55783724d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:29.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.158+0000 7f933cff9640 10 client.23082 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:29.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:29.158+0000 7f9358c1f640 1 --2- 172.21.15.120:0/3931422415 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55783724b220 0x55783724d610 crc :-1 s=READY pgs=576 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e47372cb1d20cc26 server_cookie=c16b7aad68db53e5 in_seq=0 out_seq=0 2024-09-19T09:26:29.316 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:29.389 INFO:tasks.workunit.client.0.smithi120.stdout: 10 251194 20.94 MB/sec execute 504 sec latency 235.771 ms 2024-09-19T09:26:29.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:29 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:29.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:29 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:26:29.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:29 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:26:29.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:29 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:29.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:29 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:26:29.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:29 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:26:29.493 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:29 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:29.494 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:29 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:26:29.494 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:29 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:26:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.830+0000 7f252affd640 1 -- 172.21.15.120:0/3058059934 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56391b52bd70 con 0x56391b641070 2024-09-19T09:26:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.830+0000 7f252affd640 10 client.23067 handle_command_reply: tid=0 2024-09-19T09:26:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.830+0000 7f2554e32740 1 client.23067 shutdown 2024-09-19T09:26:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.830+0000 7f2554e32740 1 -- 172.21.15.120:0/3058059934 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56391b5c6370 msgr2=0x56391b5d0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.830+0000 7f2554e32740 1 --2- 172.21.15.120:0/3058059934 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56391b5c6370 0x56391b5d0540 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f254402ade0 tx=0x7f2544031bb0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.830+0000 7f2554e32740 1 -- 172.21.15.120:0/3058059934 shutdown_connections 2024-09-19T09:26:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.830+0000 7f2554e32740 1 --2- 172.21.15.120:0/3058059934 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56391b641070 0x56391b643460 crc :-1 s=READY pgs=574 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.830+0000 7f2554e32740 1 --2- 172.21.15.120:0/3058059934 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56391b5c6370 0x56391b5d0540 unknown :-1 s=CLOSED pgs=2390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.830+0000 7f2554e32740 1 --2- 172.21.15.120:0/3058059934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56391b5c12e0 0x56391b5cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.830+0000 7f2554e32740 1 --2- 172.21.15.120:0/3058059934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56391b5c09a0 0x56391b5cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.830+0000 7f2554e32740 1 -- 172.21.15.120:0/3058059934 >> 172.21.15.120:0/3058059934 conn(0x56391b5bc010 msgr2=0x56391b5ba9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.830+0000 7f252affd640 0 client.23067 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.830+0000 7f2554e32740 1 -- 172.21.15.120:0/3058059934 shutdown_connections 2024-09-19T09:26:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.830+0000 7f2554e32740 1 -- 172.21.15.120:0/3058059934 wait complete. 2024-09-19T09:26:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.831+0000 7f2554e32740 20 client.23067 trim_cache size 0 max 16384 2024-09-19T09:26:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:26:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:26:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:26:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:26:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.831+0000 7f2551fbd640 1 -- 172.21.15.120:0/788179885 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f25340a32f0 msgr2=0x7f25340a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.831+0000 7f2551fbd640 1 --2- 172.21.15.120:0/788179885 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f25340a32f0 0x7f25340a57b0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f2544031ae0 tx=0x7f254404b000 comp rx=0 tx=0).stop 2024-09-19T09:26:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.831+0000 7f2551fbd640 1 -- 172.21.15.120:0/788179885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f254c104f00 msgr2=0x7f254c1983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.831+0000 7f2551fbd640 1 --2- 172.21.15.120:0/788179885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f254c104f00 0x7f254c1983a0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f254000e9c0 tx=0x7f254000ee80 comp rx=0 tx=0).stop 2024-09-19T09:26:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.831+0000 7f2551fbd640 1 -- 172.21.15.120:0/788179885 shutdown_connections 2024-09-19T09:26:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.831+0000 7f2551fbd640 1 --2- 172.21.15.120:0/788179885 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f25340a32f0 0x7f25340a57b0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.831+0000 7f2551fbd640 1 --2- 172.21.15.120:0/788179885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f254c107830 0x7f254c19f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.831+0000 7f2551fbd640 1 --2- 172.21.15.120:0/788179885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f254c104f00 0x7f254c1983a0 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.831+0000 7f2551fbd640 1 --2- 172.21.15.120:0/788179885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f254c1025d0 0x7f254c197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.831+0000 7f2551fbd640 1 -- 172.21.15.120:0/788179885 >> 172.21.15.120:0/788179885 conn(0x7f254c0fc1b0 msgr2=0x7f254c102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.832+0000 7f2551fbd640 1 -- 172.21.15.120:0/788179885 shutdown_connections 2024-09-19T09:26:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:29.832+0000 7f2551fbd640 1 -- 172.21.15.120:0/788179885 wait complete. 2024-09-19T09:26:30.211 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:30 smithi120 ceph-mon[31150]: pgmap v1054: 97 pgs: 97 active+clean; 497 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.0 MiB/s wr, 587 op/s 2024-09-19T09:26:30.388 INFO:tasks.workunit.client.0.smithi120.stdout: 10 251201 20.90 MB/sec execute 505 sec latency 1072.324 ms 2024-09-19T09:26:30.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:30 smithi196 ceph-mon[35150]: pgmap v1054: 97 pgs: 97 active+clean; 497 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.0 MiB/s wr, 587 op/s 2024-09-19T09:26:30.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:30 smithi153 ceph-mon[35071]: pgmap v1054: 97 pgs: 97 active+clean; 497 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.0 MiB/s wr, 587 op/s 2024-09-19T09:26:30.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.187+0000 7fd7e550d640 1 -- 172.21.15.120:0/3823358397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd7e0101420 msgr2=0x7fd7e010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:30.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.187+0000 7fd7e550d640 1 --2- 172.21.15.120:0/3823358397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd7e0101420 0x7fd7e010d950 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8009cd0 tx=0x7fd7d802a8e0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.187+0000 7fd7e550d640 1 -- 172.21.15.120:0/3823358397 shutdown_connections 2024-09-19T09:26:30.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.187+0000 7fd7e550d640 1 --2- 172.21.15.120:0/3823358397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd7e0101420 0x7fd7e010d950 unknown :-1 s=CLOSED pgs=2396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.187+0000 7fd7e550d640 1 --2- 172.21.15.120:0/3823358397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7e0100a80 0x7fd7e0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.187+0000 7fd7e550d640 1 --2- 172.21.15.120:0/3823358397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7e0106a80 0x7fd7e0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.187+0000 7fd7e550d640 1 -- 172.21.15.120:0/3823358397 >> 172.21.15.120:0/3823358397 conn(0x7fd7e00fc7a0 msgr2=0x7fd7e00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:30.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.187+0000 7fd7e550d640 1 -- 172.21.15.120:0/3823358397 shutdown_connections 2024-09-19T09:26:30.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.187+0000 7fd7e550d640 1 -- 172.21.15.120:0/3823358397 wait complete. 2024-09-19T09:26:30.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.187+0000 7fd7e550d640 1 Processor -- start 2024-09-19T09:26:30.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.187+0000 7fd7e550d640 1 -- start start 2024-09-19T09:26:30.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.188+0000 7fd7e550d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd7e0100a80 0x7fd7e0199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:30.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.188+0000 7fd7e550d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7e0101420 0x7fd7e01997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:30.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.188+0000 7fd7e550d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7e0106a80 0x7fd7e01a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:30.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.188+0000 7fd7e550d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e0110e80 con 0x7fd7e0100a80 2024-09-19T09:26:30.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.188+0000 7fd7e550d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e0110d00 con 0x7fd7e0106a80 2024-09-19T09:26:30.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.188+0000 7fd7e550d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e0111000 con 0x7fd7e0101420 2024-09-19T09:26:30.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.188+0000 7fd7deffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd7e0100a80 0x7fd7e0199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:30.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.188+0000 7fd7deffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd7e0100a80 0x7fd7e0199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56342/0 (socket says 172.21.15.120:56342) 2024-09-19T09:26:30.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.188+0000 7fd7deffd640 1 -- 172.21.15.120:0/446693850 learned_addr learned my addr 172.21.15.120:0/446693850 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:30.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.188+0000 7fd7de7fc640 1 --2- 172.21.15.120:0/446693850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7e0101420 0x7fd7e01997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:30.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.188+0000 7fd7df7fe640 1 --2- 172.21.15.120:0/446693850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7e0106a80 0x7fd7e01a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:30.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.188+0000 7fd7deffd640 1 -- 172.21.15.120:0/446693850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7e0101420 msgr2=0x7fd7e01997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:30.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.188+0000 7fd7deffd640 1 --2- 172.21.15.120:0/446693850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7e0101420 0x7fd7e01997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.188+0000 7fd7deffd640 1 -- 172.21.15.120:0/446693850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7e0106a80 msgr2=0x7fd7e01a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:30.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.188+0000 7fd7deffd640 1 --2- 172.21.15.120:0/446693850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7e0106a80 0x7fd7e01a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.188+0000 7fd7deffd640 1 -- 172.21.15.120:0/446693850 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7e01a4850 con 0x7fd7e0100a80 2024-09-19T09:26:30.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.188+0000 7fd7de7fc640 1 --2- 172.21.15.120:0/446693850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7e0101420 0x7fd7e01997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:30.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.188+0000 7fd7deffd640 1 --2- 172.21.15.120:0/446693850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd7e0100a80 0x7fd7e0199290 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7fd7d000e9f0 tx=0x7fd7d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:30.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.188+0000 7fd7c7fff640 1 -- 172.21.15.120:0/446693850 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d000cd80 con 0x7fd7e0100a80 2024-09-19T09:26:30.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.188+0000 7fd7e550d640 1 -- 172.21.15.120:0/446693850 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7e01a4a20 con 0x7fd7e0100a80 2024-09-19T09:26:30.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.188+0000 7fd7c7fff640 1 -- 172.21.15.120:0/446693850 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7d0005500 con 0x7fd7e0100a80 2024-09-19T09:26:30.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.188+0000 7fd7c7fff640 1 -- 172.21.15.120:0/446693850 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0010690 con 0x7fd7e0100a80 2024-09-19T09:26:30.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.189+0000 7fd7e550d640 1 -- 172.21.15.120:0/446693850 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7e01a4ee0 con 0x7fd7e0100a80 2024-09-19T09:26:30.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.191+0000 7fd7c7fff640 1 -- 172.21.15.120:0/446693850 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd7d00046c0 con 0x7fd7e0100a80 2024-09-19T09:26:30.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.191+0000 7fd7e78e6740 1 Processor -- start 2024-09-19T09:26:30.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.191+0000 7fd7e78e6740 1 -- start start 2024-09-19T09:26:30.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.191+0000 7fd7e78e6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a89ca6b40 0x561a89ca5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:30.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.191+0000 7fd7e78e6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a89ca55d0 0x561a89ca5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:30.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.191+0000 7fd7e78e6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a89cb0370 0x561a89cb2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:30.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.191+0000 7fd7c7fff640 1 --2- 172.21.15.120:0/446693850 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd7c80a3240 0x7fd7c80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:30.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.191+0000 7fd7e78e6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561a89c0b2f0 con 0x561a89ca6b40 2024-09-19T09:26:30.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.191+0000 7fd7e78e6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561a89bb2cf0 con 0x561a89cb0370 2024-09-19T09:26:30.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.191+0000 7fd7e78e6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561a89c072b0 con 0x561a89ca55d0 2024-09-19T09:26:30.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.192+0000 7fd7df7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a89ca6b40 0x561a89ca5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:30.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.192+0000 7fd7deffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a89ca55d0 0x561a89ca5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:30.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.192+0000 7fd7df7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a89ca6b40 0x561a89ca5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56354/0 (socket says 172.21.15.120:56354) 2024-09-19T09:26:30.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.192+0000 7fd7df7fe640 1 -- 172.21.15.120:0/1658948556 learned_addr learned my addr 172.21.15.120:0/1658948556 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:30.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.192+0000 7fd7de7fc640 1 --2- 172.21.15.120:0/1658948556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a89cb0370 0x561a89cb2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:30.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.192+0000 7fd7df7fe640 1 --2- 172.21.15.120:0/446693850 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd7c80a3240 0x7fd7c80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:30.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.192+0000 7fd7c7fff640 1 -- 172.21.15.120:0/446693850 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd7d00cfcf0 con 0x7fd7e0100a80 2024-09-19T09:26:30.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.192+0000 7fd7df7fe640 1 -- 172.21.15.120:0/1658948556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a89ca55d0 msgr2=0x561a89ca5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:30.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.192+0000 7fd7df7fe640 1 --2- 172.21.15.120:0/1658948556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a89ca55d0 0x561a89ca5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.192+0000 7fd7df7fe640 1 -- 172.21.15.120:0/1658948556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a89cb0370 msgr2=0x561a89cb2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:30.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.192+0000 7fd7df7fe640 1 --2- 172.21.15.120:0/1658948556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a89cb0370 0x561a89cb2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.192+0000 7fd7df7fe640 1 -- 172.21.15.120:0/1658948556 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a89b973a0 con 0x561a89ca6b40 2024-09-19T09:26:30.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.192+0000 7fd7deffd640 1 --2- 172.21.15.120:0/1658948556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a89ca55d0 0x561a89ca5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:30.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.192+0000 7fd7df7fe640 1 --2- 172.21.15.120:0/1658948556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a89ca6b40 0x561a89ca5090 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8033040 tx=0x7fd7d8031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba170b619591b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:30.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.192+0000 7fd7df7fe640 1 --2- 172.21.15.120:0/446693850 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd7c80a3240 0x7fd7c80a5700 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8004ab0 tx=0x7fd7d8036000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:30.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.192+0000 7fd7c57fa640 1 -- 172.21.15.120:0/1658948556 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d804d070 con 0x561a89ca6b40 2024-09-19T09:26:30.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.192+0000 7fd7c57fa640 1 -- 172.21.15.120:0/1658948556 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7d8036070 con 0x561a89ca6b40 2024-09-19T09:26:30.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.192+0000 7fd7c57fa640 1 -- 172.21.15.120:0/1658948556 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8030b90 con 0x561a89ca6b40 2024-09-19T09:26:30.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.193+0000 7fd7e78e6740 1 -- 172.21.15.120:0/1658948556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a89ca6b40 msgr2=0x561a89ca5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:30.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.193+0000 7fd7e78e6740 1 --2- 172.21.15.120:0/1658948556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a89ca6b40 0x561a89ca5090 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8033040 tx=0x7fd7d8031af0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.193+0000 7fd7e78e6740 1 -- 172.21.15.120:0/1658948556 shutdown_connections 2024-09-19T09:26:30.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.193+0000 7fd7e78e6740 1 --2- 172.21.15.120:0/1658948556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a89cb0370 0x561a89cb2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.193+0000 7fd7e78e6740 1 --2- 172.21.15.120:0/1658948556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a89ca55d0 0x561a89ca5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.193+0000 7fd7e78e6740 1 --2- 172.21.15.120:0/1658948556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a89ca6b40 0x561a89ca5090 unknown :-1 s=CLOSED pgs=2398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.193+0000 7fd7e78e6740 1 -- 172.21.15.120:0/1658948556 >> 172.21.15.120:0/1658948556 conn(0x561a89c6e310 msgr2=0x561a89ca1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:30.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.193+0000 7fd7e78e6740 1 -- 172.21.15.120:0/1658948556 shutdown_connections 2024-09-19T09:26:30.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.193+0000 7fd7e78e6740 1 -- 172.21.15.120:0/1658948556 wait complete. 2024-09-19T09:26:30.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.193+0000 7fd7e78e6740 1 -- start start 2024-09-19T09:26:30.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.193+0000 7fd7e78e6740 1 Processor -- start 2024-09-19T09:26:30.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.194+0000 7fd7e78e6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a89ca55d0 0x561a89cafd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:30.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.194+0000 7fd7e78e6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a89ca6b40 0x561a89cad040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:30.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.194+0000 7fd7e78e6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a89cb0370 0x561a89cad650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:30.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.194+0000 7fd7e78e6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561a89cb2fb0 con 0x561a89cb0370 2024-09-19T09:26:30.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.194+0000 7fd7e78e6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561a89bc6060 con 0x561a89ca6b40 2024-09-19T09:26:30.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.194+0000 7fd7e78e6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561a89cb3130 con 0x561a89ca55d0 2024-09-19T09:26:30.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.194+0000 7fd7e78e6740 10 client.0 _ll_register_callbacks cb 0x561a899198c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:30.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.194+0000 7fd7df7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a89cb0370 0x561a89cad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:30.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.194+0000 7fd7df7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a89cb0370 0x561a89cad650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56370/0 (socket says 172.21.15.120:56370) 2024-09-19T09:26:30.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.194+0000 7fd7deffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a89ca55d0 0x561a89cafd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:30.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.194+0000 7fd7df7fe640 1 -- 172.21.15.120:0/3997229292 learned_addr learned my addr 172.21.15.120:0/3997229292 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:30.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.194+0000 7fd7de7fc640 1 --2- 172.21.15.120:0/3997229292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a89ca6b40 0x561a89cad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:30.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.194+0000 7fd7deffd640 1 -- 172.21.15.120:0/3997229292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a89ca6b40 msgr2=0x561a89cad040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:30.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.194+0000 7fd7deffd640 1 --2- 172.21.15.120:0/3997229292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a89ca6b40 0x561a89cad040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.194+0000 7fd7deffd640 1 -- 172.21.15.120:0/3997229292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a89cb0370 msgr2=0x561a89cad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:30.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.194+0000 7fd7deffd640 1 --2- 172.21.15.120:0/3997229292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a89cb0370 0x561a89cad650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.194+0000 7fd7deffd640 1 -- 172.21.15.120:0/3997229292 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a8997c3c0 con 0x561a89ca55d0 2024-09-19T09:26:30.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.194+0000 7fd7df7fe640 1 --2- 172.21.15.120:0/3997229292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a89cb0370 0x561a89cad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:30.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.194+0000 7fd7deffd640 1 --2- 172.21.15.120:0/3997229292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a89ca55d0 0x561a89cafd90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd7d009bfd0 tx=0x7fd7d006c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57bb31afa68acd77 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:30.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.194+0000 7fd7c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:26:30.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.195+0000 7fd7c2ff5640 1 -- 172.21.15.120:0/3997229292 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d00a1810 con 0x561a89ca55d0 2024-09-19T09:26:30.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.195+0000 7fd7c2ff5640 1 -- 172.21.15.120:0/3997229292 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd7d009c730 con 0x561a89ca55d0 2024-09-19T09:26:30.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.195+0000 7fd7e78e6740 1 -- 172.21.15.120:0/3997229292 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7d8036070 con 0x561a89ca55d0 2024-09-19T09:26:30.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.195+0000 7fd7c2ff5640 1 -- 172.21.15.120:0/3997229292 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd7d009c9d0 con 0x561a89ca55d0 2024-09-19T09:26:30.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.195+0000 7fd7e78e6740 10 client.31630 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:30.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.195+0000 7fd7e78e6740 1 -- 172.21.15.120:0/3997229292 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561a89cbeae0 con 0x561a89ca55d0 2024-09-19T09:26:30.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.195+0000 7fd7c2ff5640 1 -- 172.21.15.120:0/3997229292 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7d8036070 con 0x561a89ca55d0 2024-09-19T09:26:30.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.195+0000 7fd7c2ff5640 1 -- 172.21.15.120:0/3997229292 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fd7d0066e40 con 0x561a89ca55d0 2024-09-19T09:26:30.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.195+0000 7fd7e78e6740 10 client.31630 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:30.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.195+0000 7fd7e78e6740 10 client.31630 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:26:30.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.195+0000 7fd7e78e6740 20 client.31630 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:30.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.195+0000 7fd7e78e6740 1 --2- 172.21.15.120:0/3997229292 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x561a89cbfef0 0x561a89cc2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:30.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.195+0000 7fd7e78e6740 4 client.31630 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:26:30.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.195+0000 7fd7e78e6740 1 -- 172.21.15.120:0/3997229292 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a89c18860 con 0x561a89cbfef0 2024-09-19T09:26:30.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.195+0000 7fd7de7fc640 1 --2- 172.21.15.120:0/3997229292 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x561a89cbfef0 0x561a89cc2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:30.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.196+0000 7fd7c2ff5640 10 client.31630 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:26:30.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.196+0000 7fd7de7fc640 1 --2- 172.21.15.120:0/3997229292 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x561a89cbfef0 0x561a89cc2320 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c404e3f32a1296cc server_cookie=c3ce6e42bc1b400f in_seq=0 out_seq=0 2024-09-19T09:26:30.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.197+0000 7fd7c2ff5640 1 -- 172.21.15.120:0/3997229292 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561a89c18860 con 0x561a89cbfef0 2024-09-19T09:26:30.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.197+0000 7fd7c2ff5640 10 client.31630 handle_command_reply: tid=0 2024-09-19T09:26:30.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.197+0000 7fd7e78e6740 1 client.31630 shutdown 2024-09-19T09:26:30.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.197+0000 7fd7e78e6740 1 -- 172.21.15.120:0/3997229292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a89ca55d0 msgr2=0x561a89cafd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:30.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.197+0000 7fd7e78e6740 1 --2- 172.21.15.120:0/3997229292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a89ca55d0 0x561a89cafd90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd7d009bfd0 tx=0x7fd7d006c000 comp rx=0 tx=0).stop 2024-09-19T09:26:30.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.197+0000 7fd7e78e6740 1 -- 172.21.15.120:0/3997229292 shutdown_connections 2024-09-19T09:26:30.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.197+0000 7fd7e78e6740 1 --2- 172.21.15.120:0/3997229292 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x561a89cbfef0 0x561a89cc2320 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.197+0000 7fd7e78e6740 1 --2- 172.21.15.120:0/3997229292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a89cb0370 0x561a89cad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.197+0000 7fd7e78e6740 1 --2- 172.21.15.120:0/3997229292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a89ca6b40 0x561a89cad040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.197+0000 7fd7e78e6740 1 --2- 172.21.15.120:0/3997229292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a89ca55d0 0x561a89cafd90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.197+0000 7fd7c2ff5640 0 client.31630 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:26:30.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.197+0000 7fd7e78e6740 1 -- 172.21.15.120:0/3997229292 >> 172.21.15.120:0/3997229292 conn(0x561a89c6e310 msgr2=0x561a89ca7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:30.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.197+0000 7fd7e78e6740 1 -- 172.21.15.120:0/3997229292 shutdown_connections 2024-09-19T09:26:30.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.197+0000 7fd7e78e6740 1 -- 172.21.15.120:0/3997229292 wait complete. 2024-09-19T09:26:30.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.197+0000 7fd7e78e6740 20 client.31630 trim_cache size 0 max 16384 2024-09-19T09:26:30.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.210+0000 7fd7e78e6740 1 Processor -- start 2024-09-19T09:26:30.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.210+0000 7fd7e78e6740 1 -- start start 2024-09-19T09:26:30.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.210+0000 7fd7e78e6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a89ca55d0 0x561a89ca8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:30.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.210+0000 7fd7e78e6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a89ca6b40 0x561a89ca8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:30.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.210+0000 7fd7e78e6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a89cb0370 0x561a89cc7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:30.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.210+0000 7fd7e78e6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e01ab320 con 0x561a89cb0370 2024-09-19T09:26:30.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.210+0000 7fd7e78e6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e01ace60 con 0x561a89ca6b40 2024-09-19T09:26:30.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.210+0000 7fd7e78e6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e01ab9f0 con 0x561a89ca55d0 2024-09-19T09:26:30.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.210+0000 7fd7df7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a89cb0370 0x561a89cc7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:30.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.210+0000 7fd7deffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a89ca55d0 0x561a89ca8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:30.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.210+0000 7fd7df7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a89cb0370 0x561a89cc7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56378/0 (socket says 172.21.15.120:56378) 2024-09-19T09:26:30.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.210+0000 7fd7deffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a89ca55d0 0x561a89ca8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:36970/0 (socket says 172.21.15.120:36970) 2024-09-19T09:26:30.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.210+0000 7fd7df7fe640 1 -- 172.21.15.120:0/1455060385 learned_addr learned my addr 172.21.15.120:0/1455060385 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:30.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.210+0000 7fd7de7fc640 1 --2- 172.21.15.120:0/1455060385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a89ca6b40 0x561a89ca8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:30.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.210+0000 7fd7df7fe640 1 -- 172.21.15.120:0/1455060385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a89ca55d0 msgr2=0x561a89ca8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:30.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.210+0000 7fd7df7fe640 1 --2- 172.21.15.120:0/1455060385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a89ca55d0 0x561a89ca8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.210+0000 7fd7df7fe640 1 -- 172.21.15.120:0/1455060385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a89ca6b40 msgr2=0x561a89ca8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:30.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.210+0000 7fd7df7fe640 1 --2- 172.21.15.120:0/1455060385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a89ca6b40 0x561a89ca8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.210+0000 7fd7df7fe640 1 -- 172.21.15.120:0/1455060385 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a89cadfe0 con 0x561a89cb0370 2024-09-19T09:26:30.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.211+0000 7fd7de7fc640 1 --2- 172.21.15.120:0/1455060385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a89ca6b40 0x561a89ca8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:30.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.211+0000 7fd7df7fe640 1 --2- 172.21.15.120:0/1455060385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a89cb0370 0x561a89cc7bc0 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8009780 tx=0x7fd7d802a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14dbc2757c08ad83 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:30.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.211+0000 7fd7c4ff9640 1 -- 172.21.15.120:0/1455060385 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8060070 con 0x561a89cb0370 2024-09-19T09:26:30.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.211+0000 7fd7c4ff9640 1 -- 172.21.15.120:0/1455060385 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7d8030ba0 con 0x561a89cb0370 2024-09-19T09:26:30.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.211+0000 7fd7c4ff9640 1 -- 172.21.15.120:0/1455060385 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d804d040 con 0x561a89cb0370 2024-09-19T09:26:30.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.211+0000 7fd7e78e6740 1 -- 172.21.15.120:0/1455060385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a89cb0370 msgr2=0x561a89cc7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:30.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.211+0000 7fd7e78e6740 1 --2- 172.21.15.120:0/1455060385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a89cb0370 0x561a89cc7bc0 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8009780 tx=0x7fd7d802a8e0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.212+0000 7fd7e78e6740 1 -- 172.21.15.120:0/1455060385 shutdown_connections 2024-09-19T09:26:30.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.212+0000 7fd7e78e6740 1 --2- 172.21.15.120:0/1455060385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a89cb0370 0x561a89cc7bc0 unknown :-1 s=CLOSED pgs=2399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.212+0000 7fd7e78e6740 1 --2- 172.21.15.120:0/1455060385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a89ca6b40 0x561a89ca8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.212+0000 7fd7e78e6740 1 --2- 172.21.15.120:0/1455060385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a89ca55d0 0x561a89ca8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.212+0000 7fd7e78e6740 1 -- 172.21.15.120:0/1455060385 >> 172.21.15.120:0/1455060385 conn(0x561a89c6e310 msgr2=0x561a89ca0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:30.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.212+0000 7fd7e78e6740 1 -- 172.21.15.120:0/1455060385 shutdown_connections 2024-09-19T09:26:30.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.212+0000 7fd7e78e6740 1 -- 172.21.15.120:0/1455060385 wait complete. 2024-09-19T09:26:30.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.212+0000 7fd7e78e6740 1 -- start start 2024-09-19T09:26:30.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.212+0000 7fd7e78e6740 1 Processor -- start 2024-09-19T09:26:30.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.212+0000 7fd7e78e6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a89ca55d0 0x561a89cb83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:30.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.212+0000 7fd7e78e6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a89ca6b40 0x561a89cb8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:30.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.212+0000 7fd7e78e6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a89cb0370 0x561a89cb5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:30.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.212+0000 7fd7e78e6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e01ac790 con 0x561a89ca6b40 2024-09-19T09:26:30.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.212+0000 7fd7e78e6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e01ac0c0 con 0x561a89cb0370 2024-09-19T09:26:30.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.212+0000 7fd7e78e6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e01a92e0 con 0x561a89ca55d0 2024-09-19T09:26:30.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.212+0000 7fd7e78e6740 10 client.0 _ll_register_callbacks cb 0x7fd7d000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:30.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.212+0000 7fd7de7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a89ca6b40 0x561a89cb8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:30.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.212+0000 7fd7de7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a89ca6b40 0x561a89cb8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56390/0 (socket says 172.21.15.120:56390) 2024-09-19T09:26:30.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.212+0000 7fd7de7fc640 1 -- 172.21.15.120:0/3702042213 learned_addr learned my addr 172.21.15.120:0/3702042213 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:30.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.212+0000 7fd7deffd640 1 --2- 172.21.15.120:0/3702042213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a89ca55d0 0x561a89cb83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:30.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.212+0000 7fd7df7fe640 1 --2- 172.21.15.120:0/3702042213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a89cb0370 0x561a89cb5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:30.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.213+0000 7fd7de7fc640 1 -- 172.21.15.120:0/3702042213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a89ca55d0 msgr2=0x561a89cb83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:30.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.213+0000 7fd7de7fc640 1 --2- 172.21.15.120:0/3702042213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a89ca55d0 0x561a89cb83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.213+0000 7fd7de7fc640 1 -- 172.21.15.120:0/3702042213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a89cb0370 msgr2=0x561a89cb5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:30.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.213+0000 7fd7de7fc640 1 --2- 172.21.15.120:0/3702042213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a89cb0370 0x561a89cb5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.213+0000 7fd7de7fc640 1 -- 172.21.15.120:0/3702042213 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a89cb8fa0 con 0x561a89ca6b40 2024-09-19T09:26:30.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.213+0000 7fd7de7fc640 1 --2- 172.21.15.120:0/3702042213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a89ca6b40 0x561a89cb8900 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4009fd0 tx=0x7fd7d4012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac7f141ae440465 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:30.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.213+0000 7fd7c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:26:30.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.213+0000 7fd7c2ff5640 1 -- 172.21.15.120:0/3702042213 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d40106f0 con 0x561a89ca6b40 2024-09-19T09:26:30.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.213+0000 7fd7c2ff5640 1 -- 172.21.15.120:0/3702042213 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd7d4010890 con 0x561a89ca6b40 2024-09-19T09:26:30.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.213+0000 7fd7e78e6740 1 -- 172.21.15.120:0/3702042213 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7d8030ba0 con 0x561a89ca6b40 2024-09-19T09:26:30.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.213+0000 7fd7c2ff5640 1 -- 172.21.15.120:0/3702042213 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd7d4010a90 con 0x561a89ca6b40 2024-09-19T09:26:30.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.213+0000 7fd7e78e6740 10 client.23100 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:30.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.213+0000 7fd7e78e6740 1 -- 172.21.15.120:0/3702042213 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561a89d2b1d0 con 0x561a89ca6b40 2024-09-19T09:26:30.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.213+0000 7fd7c2ff5640 1 -- 172.21.15.120:0/3702042213 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7d8030ba0 con 0x561a89ca6b40 2024-09-19T09:26:30.495 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:26:30.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.214+0000 7fd7c2ff5640 1 -- 172.21.15.120:0/3702042213 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fd7d4031400 con 0x561a89ca6b40 2024-09-19T09:26:30.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.214+0000 7fd7e78e6740 10 client.23100 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:30.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.214+0000 7fd7e78e6740 10 client.23100 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:26:30.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.214+0000 7fd7e78e6740 20 client.23100 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:30.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.214+0000 7fd7e78e6740 1 --2- 172.21.15.120:0/3702042213 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x561a89d2b980 0x561a89d2dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:30.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.214+0000 7fd7e78e6740 4 client.23100 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:26:30.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.214+0000 7fd7e78e6740 1 -- 172.21.15.120:0/3702042213 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a89c12d70 con 0x561a89d2b980 2024-09-19T09:26:30.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.214+0000 7fd7deffd640 1 --2- 172.21.15.120:0/3702042213 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x561a89d2b980 0x561a89d2dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:30.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.214+0000 7fd7c2ff5640 10 client.23100 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:26:30.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.214+0000 7fd7deffd640 1 --2- 172.21.15.120:0/3702042213 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x561a89d2b980 0x561a89d2dda0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ae6e8132f81131db server_cookie=a523d0d8103b9d05 in_seq=0 out_seq=0 2024-09-19T09:26:30.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.214+0000 7fd7c2ff5640 1 -- 172.21.15.120:0/3702042213 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a89c12d70 con 0x561a89d2b980 2024-09-19T09:26:30.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.214+0000 7fd7c2ff5640 10 client.23100 handle_command_reply: tid=0 2024-09-19T09:26:30.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.214+0000 7fd7e78e6740 1 client.23100 shutdown 2024-09-19T09:26:30.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.215+0000 7fd7e78e6740 1 -- 172.21.15.120:0/3702042213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a89ca6b40 msgr2=0x561a89cb8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:30.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.215+0000 7fd7e78e6740 1 --2- 172.21.15.120:0/3702042213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a89ca6b40 0x561a89cb8900 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4009fd0 tx=0x7fd7d4012b00 comp rx=0 tx=0).stop 2024-09-19T09:26:30.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.215+0000 7fd7e78e6740 1 -- 172.21.15.120:0/3702042213 shutdown_connections 2024-09-19T09:26:30.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.215+0000 7fd7e78e6740 1 --2- 172.21.15.120:0/3702042213 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x561a89d2b980 0x561a89d2dda0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.215+0000 7fd7e78e6740 1 --2- 172.21.15.120:0/3702042213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561a89cb0370 0x561a89cb5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.215+0000 7fd7c2ff5640 0 client.23100 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:26:30.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.215+0000 7fd7e78e6740 1 --2- 172.21.15.120:0/3702042213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561a89ca6b40 0x561a89cb8900 unknown :-1 s=CLOSED pgs=2400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.215+0000 7fd7e78e6740 1 --2- 172.21.15.120:0/3702042213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a89ca55d0 0x561a89cb83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.215+0000 7fd7e78e6740 1 -- 172.21.15.120:0/3702042213 >> 172.21.15.120:0/3702042213 conn(0x561a89c6e310 msgr2=0x561a89cae8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:30.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.215+0000 7fd7e78e6740 1 -- 172.21.15.120:0/3702042213 shutdown_connections 2024-09-19T09:26:30.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.215+0000 7fd7e78e6740 1 -- 172.21.15.120:0/3702042213 wait complete. 2024-09-19T09:26:30.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.215+0000 7fd7e78e6740 20 client.23100 trim_cache size 0 max 16384 2024-09-19T09:26:30.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.215+0000 7fd7e550d640 1 -- 172.21.15.120:0/446693850 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd7c80a3240 msgr2=0x7fd7c80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:30.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.215+0000 7fd7e550d640 1 --2- 172.21.15.120:0/446693850 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd7c80a3240 0x7fd7c80a5700 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8004ab0 tx=0x7fd7d8036000 comp rx=0 tx=0).stop 2024-09-19T09:26:30.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.215+0000 7fd7e550d640 1 -- 172.21.15.120:0/446693850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd7e0100a80 msgr2=0x7fd7e0199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:30.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.215+0000 7fd7e550d640 1 --2- 172.21.15.120:0/446693850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd7e0100a80 0x7fd7e0199290 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7fd7d000e9f0 tx=0x7fd7d000edf0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.215+0000 7fd7e550d640 1 -- 172.21.15.120:0/446693850 shutdown_connections 2024-09-19T09:26:30.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.215+0000 7fd7e550d640 1 --2- 172.21.15.120:0/446693850 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd7c80a3240 0x7fd7c80a5700 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.215+0000 7fd7e550d640 1 --2- 172.21.15.120:0/446693850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7e0106a80 0x7fd7e01a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.215+0000 7fd7e550d640 1 --2- 172.21.15.120:0/446693850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7e0101420 0x7fd7e01997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.215+0000 7fd7e550d640 1 --2- 172.21.15.120:0/446693850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd7e0100a80 0x7fd7e0199290 unknown :-1 s=CLOSED pgs=2397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:30.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.215+0000 7fd7e550d640 1 -- 172.21.15.120:0/446693850 >> 172.21.15.120:0/446693850 conn(0x7fd7e00fc7a0 msgr2=0x7fd7e00fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:30.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.215+0000 7fd7e550d640 1 -- 172.21.15.120:0/446693850 shutdown_connections 2024-09-19T09:26:30.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:30.215+0000 7fd7e550d640 1 -- 172.21.15.120:0/446693850 wait complete. 2024-09-19T09:26:31.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:31.121+0000 7f933cff9640 1 -- 172.21.15.120:0/3931422415 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557837131d20 con 0x55783724b220 2024-09-19T09:26:31.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:31.121+0000 7f933cff9640 10 client.23082 handle_command_reply: tid=0 2024-09-19T09:26:31.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:31.121+0000 7f935da5d740 1 client.23082 shutdown 2024-09-19T09:26:31.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:31.121+0000 7f935da5d740 1 -- 172.21.15.120:0/3931422415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578371c4760 msgr2=0x5578371d77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:31.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:31.121+0000 7f935da5d740 1 --2- 172.21.15.120:0/3931422415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578371c4760 0x5578371d77f0 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7f93500b8b80 tx=0x7f9350009680 comp rx=0 tx=0).stop 2024-09-19T09:26:31.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:31.121+0000 7f935da5d740 1 -- 172.21.15.120:0/3931422415 shutdown_connections 2024-09-19T09:26:31.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:31.121+0000 7f935da5d740 1 --2- 172.21.15.120:0/3931422415 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55783724b220 0x55783724d610 crc :-1 s=READY pgs=576 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:31.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:31.121+0000 7f935da5d740 1 --2- 172.21.15.120:0/3931422415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578371cf940 0x5578371d45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:31.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:31.121+0000 7f935da5d740 1 --2- 172.21.15.120:0/3931422415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5578371c5080 0x5578371d7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:31.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:31.121+0000 7f933cff9640 0 client.23082 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:31.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:31.121+0000 7f935da5d740 1 --2- 172.21.15.120:0/3931422415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5578371c4760 0x5578371d77f0 unknown :-1 s=CLOSED pgs=2395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:31.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:31.121+0000 7f935da5d740 1 -- 172.21.15.120:0/3931422415 >> 172.21.15.120:0/3931422415 conn(0x5578371c0cf0 msgr2=0x5578371bfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:31.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:31.121+0000 7f935da5d740 1 -- 172.21.15.120:0/3931422415 shutdown_connections 2024-09-19T09:26:31.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:31.121+0000 7f935da5d740 1 -- 172.21.15.120:0/3931422415 wait complete. 2024-09-19T09:26:31.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:31.121+0000 7f935da5d740 20 client.23082 trim_cache size 0 max 16384 2024-09-19T09:26:31.124 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:26:31.125 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 39, 2024-09-19T09:26:31.125 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:26:31.125 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 78, 2024-09-19T09:26:31.125 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:26:31.125 INFO:teuthology.orchestra.run.smithi120.stdout: "3177af4d": { 2024-09-19T09:26:31.125 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 78, 2024-09-19T09:26:31.125 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:26:31.125 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:26:31.125 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:26:31.125 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:26:31.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:26:31.125 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:26:31.125 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 133.1, 2024-09-19T09:26:31.125 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:26:31.126 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:26:31.126 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:26:31.126 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:26:31.126 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:26:31.126 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:26:31.126 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:26:31.126 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:26:31.126 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:26:31.126 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:26:31.126 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:26:31.126 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:26:31.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:31.122+0000 7f935b684640 1 -- 172.21.15.120:0/3615444906 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93400a3240 msgr2=0x7f93400a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:31.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:31.122+0000 7f935b684640 1 --2- 172.21.15.120:0/3615444906 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93400a3240 0x7f93400a5700 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f934c005f40 tx=0x7f934c009500 comp rx=0 tx=0).stop 2024-09-19T09:26:31.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:31.122+0000 7f935b684640 1 -- 172.21.15.120:0/3615444906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9354102d40 msgr2=0x7f9354198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:31.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:31.122+0000 7f935b684640 1 --2- 172.21.15.120:0/3615444906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9354102d40 0x7f9354198470 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f9350006010 tx=0x7f9350004040 comp rx=0 tx=0).stop 2024-09-19T09:26:31.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:31.122+0000 7f935b684640 1 -- 172.21.15.120:0/3615444906 shutdown_connections 2024-09-19T09:26:31.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:31.122+0000 7f935b684640 1 --2- 172.21.15.120:0/3615444906 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f93400a3240 0x7f93400a5700 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:31.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:31.122+0000 7f935b684640 1 --2- 172.21.15.120:0/3615444906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f935410c170 0x7f935419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:31.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:31.122+0000 7f935b684640 1 --2- 172.21.15.120:0/3615444906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9354103660 0x7f93541989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:31.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:31.122+0000 7f935b684640 1 --2- 172.21.15.120:0/3615444906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9354102d40 0x7f9354198470 unknown :-1 s=CLOSED pgs=2392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:31.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:31.122+0000 7f935b684640 1 -- 172.21.15.120:0/3615444906 >> 172.21.15.120:0/3615444906 conn(0x7f93540fc7a0 msgr2=0x7f93540fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:31.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:31.122+0000 7f935b684640 1 -- 172.21.15.120:0/3615444906 shutdown_connections 2024-09-19T09:26:31.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:31.122+0000 7f935b684640 1 -- 172.21.15.120:0/3615444906 wait complete. 2024-09-19T09:26:31.343 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:26:31.388 INFO:tasks.workunit.client.0.smithi120.stdout: 10 251215 20.86 MB/sec execute 506 sec latency 2072.355 ms 2024-09-19T09:26:32.071 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3177af4d', quiesce took 6.6 seconds. Will release after: 6.5 2024-09-19T09:26:32.124 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:32.388 INFO:tasks.workunit.client.0.smithi120.stdout: 10 251215 20.82 MB/sec execute 507 sec latency 3072.388 ms 2024-09-19T09:26:32.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.690+0000 7fb85464e640 1 -- 172.21.15.120:0/1518495824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb84c1082a0 msgr2=0x7fb84c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:32.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.690+0000 7fb85464e640 1 --2- 172.21.15.120:0/1518495824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb84c1082a0 0x7fb84c10ec40 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7fb844009cd0 tx=0x7fb84402a8e0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.690+0000 7fb85464e640 1 -- 172.21.15.120:0/1518495824 shutdown_connections 2024-09-19T09:26:32.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.690+0000 7fb85464e640 1 --2- 172.21.15.120:0/1518495824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb84c1082a0 0x7fb84c10ec40 unknown :-1 s=CLOSED pgs=2401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.690+0000 7fb85464e640 1 --2- 172.21.15.120:0/1518495824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb84c1036e0 0x7fb84c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.690+0000 7fb85464e640 1 --2- 172.21.15.120:0/1518495824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb84c102d30 0x7fb84c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.690+0000 7fb85464e640 1 -- 172.21.15.120:0/1518495824 >> 172.21.15.120:0/1518495824 conn(0x7fb84c0fc7d0 msgr2=0x7fb84c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:32.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.690+0000 7fb85464e640 1 -- 172.21.15.120:0/1518495824 shutdown_connections 2024-09-19T09:26:32.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.690+0000 7fb85464e640 1 -- 172.21.15.120:0/1518495824 wait complete. 2024-09-19T09:26:32.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.691+0000 7fb85464e640 1 Processor -- start 2024-09-19T09:26:32.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.691+0000 7fb85464e640 1 -- start start 2024-09-19T09:26:32.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.691+0000 7fb85464e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb84c102d30 0x7fb84c198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:32.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.691+0000 7fb85464e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb84c1036e0 0x7fb84c198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:32.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.691+0000 7fb85464e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb84c1082a0 0x7fb84c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:32.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.691+0000 7fb85464e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb84c1120a0 con 0x7fb84c102d30 2024-09-19T09:26:32.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.691+0000 7fb85464e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb84c111f20 con 0x7fb84c1036e0 2024-09-19T09:26:32.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.691+0000 7fb85464e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb84c112220 con 0x7fb84c1082a0 2024-09-19T09:26:32.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.692+0000 7fb851be9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb84c1036e0 0x7fb84c198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:32.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.692+0000 7fb851be9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb84c1036e0 0x7fb84c198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:33516/0 (socket says 172.21.15.120:33516) 2024-09-19T09:26:32.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.692+0000 7fb851be9640 1 -- 172.21.15.120:0/168612914 learned_addr learned my addr 172.21.15.120:0/168612914 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:32.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.692+0000 7fb8523ea640 1 --2- 172.21.15.120:0/168612914 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb84c102d30 0x7fb84c198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:32.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.692+0000 7fb852beb640 1 --2- 172.21.15.120:0/168612914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb84c1082a0 0x7fb84c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:32.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.692+0000 7fb851be9640 1 -- 172.21.15.120:0/168612914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb84c1082a0 msgr2=0x7fb84c19ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:32.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.692+0000 7fb851be9640 1 --2- 172.21.15.120:0/168612914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb84c1082a0 0x7fb84c19ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.692+0000 7fb851be9640 1 -- 172.21.15.120:0/168612914 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb84c102d30 msgr2=0x7fb84c198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:32.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.692+0000 7fb851be9640 1 --2- 172.21.15.120:0/168612914 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb84c102d30 0x7fb84c198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.692+0000 7fb851be9640 1 -- 172.21.15.120:0/168612914 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84c19d370 con 0x7fb84c1036e0 2024-09-19T09:26:32.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.692+0000 7fb8523ea640 1 --2- 172.21.15.120:0/168612914 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb84c102d30 0x7fb84c198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:32.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.693+0000 7fb851be9640 1 --2- 172.21.15.120:0/168612914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb84c1036e0 0x7fb84c198aa0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb83c00d750 tx=0x7fb83c00dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:32.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.693+0000 7fb8437fe640 1 -- 172.21.15.120:0/168612914 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c017070 con 0x7fb84c1036e0 2024-09-19T09:26:32.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.693+0000 7fb85464e640 1 -- 172.21.15.120:0/168612914 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb84c19d600 con 0x7fb84c1036e0 2024-09-19T09:26:32.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.693+0000 7fb8437fe640 1 -- 172.21.15.120:0/168612914 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb83c005500 con 0x7fb84c1036e0 2024-09-19T09:26:32.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.693+0000 7fb8437fe640 1 -- 172.21.15.120:0/168612914 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c004e00 con 0x7fb84c1036e0 2024-09-19T09:26:32.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.693+0000 7fb85464e640 1 -- 172.21.15.120:0/168612914 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb84c19db40 con 0x7fb84c1036e0 2024-09-19T09:26:32.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.695+0000 7fb856a27740 1 Processor -- start 2024-09-19T09:26:32.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.695+0000 7fb856a27740 1 -- start start 2024-09-19T09:26:32.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.696+0000 7fb856a27740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b103a6b40 0x556b103a5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:32.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.696+0000 7fb8437fe640 1 -- 172.21.15.120:0/168612914 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fb83c004fa0 con 0x7fb84c1036e0 2024-09-19T09:26:32.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.696+0000 7fb856a27740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b103a55d0 0x556b103a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:32.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.696+0000 7fb856a27740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b103b0370 0x556b103b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:32.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.696+0000 7fb856a27740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556b1030b2f0 con 0x556b103b0370 2024-09-19T09:26:32.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.696+0000 7fb856a27740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556b102b2cf0 con 0x556b103a55d0 2024-09-19T09:26:32.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.696+0000 7fb856a27740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556b103072b0 con 0x556b103a6b40 2024-09-19T09:26:32.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.696+0000 7fb852beb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b103a6b40 0x556b103a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:32.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.696+0000 7fb851be9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b103b0370 0x556b103b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:32.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.696+0000 7fb852beb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b103a6b40 0x556b103a5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:51528/0 (socket says 172.21.15.120:51528) 2024-09-19T09:26:32.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.696+0000 7fb852beb640 1 -- 172.21.15.120:0/2761729172 learned_addr learned my addr 172.21.15.120:0/2761729172 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:32.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.696+0000 7fb8523ea640 1 --2- 172.21.15.120:0/2761729172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b103a55d0 0x556b103a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:32.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.696+0000 7fb852beb640 1 -- 172.21.15.120:0/2761729172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b103a55d0 msgr2=0x556b103a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:32.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.696+0000 7fb852beb640 1 --2- 172.21.15.120:0/2761729172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b103a55d0 0x556b103a5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.696+0000 7fb852beb640 1 -- 172.21.15.120:0/2761729172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b103b0370 msgr2=0x556b103b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:32.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.696+0000 7fb852beb640 1 --2- 172.21.15.120:0/2761729172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b103b0370 0x556b103b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.696+0000 7fb852beb640 1 -- 172.21.15.120:0/2761729172 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b102973a0 con 0x556b103a6b40 2024-09-19T09:26:32.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.696+0000 7fb852beb640 1 --2- 172.21.15.120:0/2761729172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b103a6b40 0x556b103a5090 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb844004ab0 tx=0x7fb844009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e79e0fc8d5c7e92c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:32.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.696+0000 7fb8437fe640 1 --2- 172.21.15.120:0/168612914 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb8380a3240 0x7fb8380a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:32.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.697+0000 7fb8523ea640 1 --2- 172.21.15.120:0/168612914 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb8380a3240 0x7fb8380a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:32.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.697+0000 7fb8437fe640 1 -- 172.21.15.120:0/168612914 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fb83c0cf1b0 con 0x7fb84c1036e0 2024-09-19T09:26:32.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.697+0000 7fb840ff9640 1 -- 172.21.15.120:0/2761729172 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb844048070 con 0x556b103a6b40 2024-09-19T09:26:32.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.697+0000 7fb840ff9640 1 -- 172.21.15.120:0/2761729172 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb844036070 con 0x556b103a6b40 2024-09-19T09:26:32.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.697+0000 7fb840ff9640 1 -- 172.21.15.120:0/2761729172 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb844004520 con 0x556b103a6b40 2024-09-19T09:26:32.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.697+0000 7fb856a27740 1 -- 172.21.15.120:0/2761729172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b103a6b40 msgr2=0x556b103a5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:32.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.697+0000 7fb856a27740 1 --2- 172.21.15.120:0/2761729172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b103a6b40 0x556b103a5090 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb844004ab0 tx=0x7fb844009a20 comp rx=0 tx=0).stop 2024-09-19T09:26:32.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.697+0000 7fb8523ea640 1 --2- 172.21.15.120:0/168612914 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb8380a3240 0x7fb8380a5700 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x556b103a2ac0 tx=0x7fb84800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:32.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.698+0000 7fb856a27740 1 -- 172.21.15.120:0/2761729172 shutdown_connections 2024-09-19T09:26:32.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.698+0000 7fb856a27740 1 --2- 172.21.15.120:0/2761729172 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b103b0370 0x556b103b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.698+0000 7fb856a27740 1 --2- 172.21.15.120:0/2761729172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b103a55d0 0x556b103a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.698+0000 7fb856a27740 1 --2- 172.21.15.120:0/2761729172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b103a6b40 0x556b103a5090 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.698+0000 7fb856a27740 1 -- 172.21.15.120:0/2761729172 >> 172.21.15.120:0/2761729172 conn(0x556b1036e310 msgr2=0x556b103a1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:32.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.698+0000 7fb856a27740 1 -- 172.21.15.120:0/2761729172 shutdown_connections 2024-09-19T09:26:32.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.698+0000 7fb856a27740 1 -- 172.21.15.120:0/2761729172 wait complete. 2024-09-19T09:26:32.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.698+0000 7fb856a27740 1 -- start start 2024-09-19T09:26:32.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.698+0000 7fb856a27740 1 Processor -- start 2024-09-19T09:26:32.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.698+0000 7fb856a27740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b103a55d0 0x556b103afd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:32.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.698+0000 7fb856a27740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b103a6b40 0x556b103ad000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:32.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.698+0000 7fb856a27740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b103b0370 0x556b103ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:32.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.698+0000 7fb856a27740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556b103b3010 con 0x556b103a6b40 2024-09-19T09:26:32.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.698+0000 7fb856a27740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556b102c6060 con 0x556b103b0370 2024-09-19T09:26:32.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.698+0000 7fb856a27740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556b103b3190 con 0x556b103a55d0 2024-09-19T09:26:32.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.698+0000 7fb856a27740 10 client.0 _ll_register_callbacks cb 0x556b100198c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:32.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.698+0000 7fb851be9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b103a6b40 0x556b103ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:32.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.698+0000 7fb851be9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b103a6b40 0x556b103ad000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41268/0 (socket says 172.21.15.120:41268) 2024-09-19T09:26:32.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.698+0000 7fb851be9640 1 -- 172.21.15.120:0/1981615722 learned_addr learned my addr 172.21.15.120:0/1981615722 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:32.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.698+0000 7fb8523ea640 1 --2- 172.21.15.120:0/1981615722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b103a55d0 0x556b103afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:32.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.699+0000 7fb852beb640 1 --2- 172.21.15.120:0/1981615722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b103b0370 0x556b103ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:32.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.699+0000 7fb851be9640 1 -- 172.21.15.120:0/1981615722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b103a55d0 msgr2=0x556b103afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:32.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.699+0000 7fb851be9640 1 --2- 172.21.15.120:0/1981615722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b103a55d0 0x556b103afd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.699+0000 7fb851be9640 1 -- 172.21.15.120:0/1981615722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b103b0370 msgr2=0x556b103ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:32.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.699+0000 7fb851be9640 1 --2- 172.21.15.120:0/1981615722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b103b0370 0x556b103ad5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.699+0000 7fb851be9640 1 -- 172.21.15.120:0/1981615722 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b1007c3c0 con 0x556b103a6b40 2024-09-19T09:26:32.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.699+0000 7fb852beb640 1 --2- 172.21.15.120:0/1981615722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b103b0370 0x556b103ad5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:32.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.699+0000 7fb851be9640 1 --2- 172.21.15.120:0/1981615722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b103a6b40 0x556b103ad000 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7fb83c097ce0 tx=0x7fb83c0d2760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ba3d34b52b4c74 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:32.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.699+0000 7fb835ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:26:32.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.699+0000 7fb835ffb640 1 -- 172.21.15.120:0/1981615722 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c0ce030 con 0x556b103a6b40 2024-09-19T09:26:32.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.699+0000 7fb835ffb640 1 -- 172.21.15.120:0/1981615722 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fb83c099b80 con 0x556b103a6b40 2024-09-19T09:26:32.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.699+0000 7fb856a27740 1 -- 172.21.15.120:0/1981615722 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb844036070 con 0x556b103a6b40 2024-09-19T09:26:32.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.699+0000 7fb835ffb640 1 -- 172.21.15.120:0/1981615722 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb83c020070 con 0x556b103a6b40 2024-09-19T09:26:32.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.699+0000 7fb856a27740 10 client.23112 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:32.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.699+0000 7fb856a27740 1 -- 172.21.15.120:0/1981615722 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556b103beb20 con 0x556b103a6b40 2024-09-19T09:26:32.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.699+0000 7fb835ffb640 1 -- 172.21.15.120:0/1981615722 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb844036070 con 0x556b103a6b40 2024-09-19T09:26:32.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.700+0000 7fb835ffb640 1 -- 172.21.15.120:0/1981615722 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fb83c097ef0 con 0x556b103a6b40 2024-09-19T09:26:32.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.700+0000 7fb856a27740 10 client.23112 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:32.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.700+0000 7fb856a27740 10 client.23112 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:26:32.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.700+0000 7fb856a27740 20 client.23112 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:32.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.700+0000 7fb856a27740 1 --2- 172.21.15.120:0/1981615722 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556b103bffe0 0x556b103c23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:32.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.700+0000 7fb856a27740 4 client.23112 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:26:32.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.700+0000 7fb856a27740 1 -- 172.21.15.120:0/1981615722 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b10318860 con 0x556b103bffe0 2024-09-19T09:26:32.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.700+0000 7fb8523ea640 1 --2- 172.21.15.120:0/1981615722 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556b103bffe0 0x556b103c23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:32.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.700+0000 7fb835ffb640 10 client.23112 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:32.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.700+0000 7fb8523ea640 1 --2- 172.21.15.120:0/1981615722 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556b103bffe0 0x556b103c23f0 crc :-1 s=READY pgs=577 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=188ce86ab3aa4ac4 server_cookie=461b63a5c7e9ccd9 in_seq=0 out_seq=0 2024-09-19T09:26:32.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.701+0000 7fb835ffb640 1 -- 172.21.15.120:0/1981615722 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556b10318860 con 0x556b103bffe0 2024-09-19T09:26:32.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.701+0000 7fb835ffb640 10 client.23112 handle_command_reply: tid=0 2024-09-19T09:26:32.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.701+0000 7fb856a27740 1 client.23112 shutdown 2024-09-19T09:26:32.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.702+0000 7fb856a27740 1 -- 172.21.15.120:0/1981615722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b103a6b40 msgr2=0x556b103ad000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:32.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.702+0000 7fb856a27740 1 --2- 172.21.15.120:0/1981615722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b103a6b40 0x556b103ad000 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7fb83c097ce0 tx=0x7fb83c0d2760 comp rx=0 tx=0).stop 2024-09-19T09:26:32.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.702+0000 7fb856a27740 1 -- 172.21.15.120:0/1981615722 shutdown_connections 2024-09-19T09:26:32.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.702+0000 7fb856a27740 1 --2- 172.21.15.120:0/1981615722 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556b103bffe0 0x556b103c23f0 crc :-1 s=READY pgs=577 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.702+0000 7fb856a27740 1 --2- 172.21.15.120:0/1981615722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b103b0370 0x556b103ad5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.702+0000 7fb835ffb640 0 client.23112 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:32.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.702+0000 7fb856a27740 1 --2- 172.21.15.120:0/1981615722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b103a6b40 0x556b103ad000 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.702+0000 7fb856a27740 1 --2- 172.21.15.120:0/1981615722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b103a55d0 0x556b103afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.702+0000 7fb856a27740 1 -- 172.21.15.120:0/1981615722 >> 172.21.15.120:0/1981615722 conn(0x556b1036e310 msgr2=0x556b103a7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:32.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.702+0000 7fb856a27740 1 -- 172.21.15.120:0/1981615722 shutdown_connections 2024-09-19T09:26:32.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.702+0000 7fb856a27740 1 -- 172.21.15.120:0/1981615722 wait complete. 2024-09-19T09:26:32.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.702+0000 7fb856a27740 20 client.23112 trim_cache size 0 max 16384 2024-09-19T09:26:32.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.714+0000 7fb856a27740 1 Processor -- start 2024-09-19T09:26:32.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.714+0000 7fb856a27740 1 -- start start 2024-09-19T09:26:32.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.715+0000 7fb856a27740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b103a55d0 0x556b103bc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:32.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.715+0000 7fb856a27740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b103a6b40 0x556b103abfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:32.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.715+0000 7fb856a27740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b103b0370 0x556b103c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:32.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.715+0000 7fb856a27740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb84c1b3b30 con 0x556b103b0370 2024-09-19T09:26:32.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.715+0000 7fb856a27740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb84c1b5670 con 0x556b103a6b40 2024-09-19T09:26:32.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.715+0000 7fb856a27740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb84c1b4200 con 0x556b103a55d0 2024-09-19T09:26:32.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.715+0000 7fb852beb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b103b0370 0x556b103c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:32.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.715+0000 7fb852beb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b103b0370 0x556b103c7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41276/0 (socket says 172.21.15.120:41276) 2024-09-19T09:26:32.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.715+0000 7fb852beb640 1 -- 172.21.15.120:0/3339168416 learned_addr learned my addr 172.21.15.120:0/3339168416 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:32.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.715+0000 7fb8523ea640 1 --2- 172.21.15.120:0/3339168416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b103a55d0 0x556b103bc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:32.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.715+0000 7fb851be9640 1 --2- 172.21.15.120:0/3339168416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b103a6b40 0x556b103abfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:32.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.715+0000 7fb852beb640 1 -- 172.21.15.120:0/3339168416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b103a55d0 msgr2=0x556b103bc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:32.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.715+0000 7fb852beb640 1 --2- 172.21.15.120:0/3339168416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b103a55d0 0x556b103bc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.715+0000 7fb852beb640 1 -- 172.21.15.120:0/3339168416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b103a6b40 msgr2=0x556b103abfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:32.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.715+0000 7fb852beb640 1 --2- 172.21.15.120:0/3339168416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b103a6b40 0x556b103abfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.715+0000 7fb852beb640 1 -- 172.21.15.120:0/3339168416 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b103ad940 con 0x556b103b0370 2024-09-19T09:26:32.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.715+0000 7fb852beb640 1 --2- 172.21.15.120:0/3339168416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b103b0370 0x556b103c7c70 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7fb844004ab0 tx=0x7fb844000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667cbfadf62723ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:32.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.715+0000 7fb837fff640 1 -- 172.21.15.120:0/3339168416 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb844048040 con 0x556b103b0370 2024-09-19T09:26:32.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.715+0000 7fb837fff640 1 -- 172.21.15.120:0/3339168416 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb844036040 con 0x556b103b0370 2024-09-19T09:26:32.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.715+0000 7fb837fff640 1 -- 172.21.15.120:0/3339168416 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb844004560 con 0x556b103b0370 2024-09-19T09:26:32.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.716+0000 7fb856a27740 1 -- 172.21.15.120:0/3339168416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b103b0370 msgr2=0x556b103c7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:32.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.716+0000 7fb856a27740 1 --2- 172.21.15.120:0/3339168416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b103b0370 0x556b103c7c70 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7fb844004ab0 tx=0x7fb844000f40 comp rx=0 tx=0).stop 2024-09-19T09:26:32.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.716+0000 7fb856a27740 1 -- 172.21.15.120:0/3339168416 shutdown_connections 2024-09-19T09:26:32.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.716+0000 7fb856a27740 1 --2- 172.21.15.120:0/3339168416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b103b0370 0x556b103c7c70 unknown :-1 s=CLOSED pgs=2403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.716+0000 7fb856a27740 1 --2- 172.21.15.120:0/3339168416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b103a6b40 0x556b103abfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.716+0000 7fb856a27740 1 --2- 172.21.15.120:0/3339168416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b103a55d0 0x556b103bc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.716+0000 7fb856a27740 1 -- 172.21.15.120:0/3339168416 >> 172.21.15.120:0/3339168416 conn(0x556b1036e310 msgr2=0x556b103ab8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:32.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.716+0000 7fb856a27740 1 -- 172.21.15.120:0/3339168416 shutdown_connections 2024-09-19T09:26:32.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.716+0000 7fb856a27740 1 -- 172.21.15.120:0/3339168416 wait complete. 2024-09-19T09:26:32.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.716+0000 7fb856a27740 1 -- start start 2024-09-19T09:26:32.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.717+0000 7fb856a27740 1 Processor -- start 2024-09-19T09:26:32.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.717+0000 7fb856a27740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b103a55d0 0x556b103b88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:32.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.717+0000 7fb856a27740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b103a6b40 0x556b103a9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:32.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.717+0000 7fb856a27740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b103b0370 0x556b103aa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:32.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.717+0000 7fb856a27740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb84c1b4fa0 con 0x556b103a6b40 2024-09-19T09:26:32.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.717+0000 7fb856a27740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb84c1b48d0 con 0x556b103b0370 2024-09-19T09:26:32.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.717+0000 7fb856a27740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb84c1b1af0 con 0x556b103a55d0 2024-09-19T09:26:32.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.717+0000 7fb856a27740 10 client.0 _ll_register_callbacks cb 0x7fb83c0d2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:32.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.717+0000 7fb851be9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b103a6b40 0x556b103a9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:32.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.717+0000 7fb851be9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b103a6b40 0x556b103a9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41278/0 (socket says 172.21.15.120:41278) 2024-09-19T09:26:32.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.717+0000 7fb851be9640 1 -- 172.21.15.120:0/3844674695 learned_addr learned my addr 172.21.15.120:0/3844674695 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:32.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.717+0000 7fb8523ea640 1 --2- 172.21.15.120:0/3844674695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b103a55d0 0x556b103b88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:32.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.717+0000 7fb851be9640 1 -- 172.21.15.120:0/3844674695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b103a55d0 msgr2=0x556b103b88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:32.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.717+0000 7fb851be9640 1 --2- 172.21.15.120:0/3844674695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b103a55d0 0x556b103b88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.717+0000 7fb852beb640 1 --2- 172.21.15.120:0/3844674695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b103b0370 0x556b103aa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:32.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.717+0000 7fb851be9640 1 -- 172.21.15.120:0/3844674695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b103b0370 msgr2=0x556b103aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:32.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.717+0000 7fb851be9640 1 --2- 172.21.15.120:0/3844674695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b103b0370 0x556b103aa050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.717+0000 7fb851be9640 1 -- 172.21.15.120:0/3844674695 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b103aa5e0 con 0x556b103a6b40 2024-09-19T09:26:32.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.717+0000 7fb851be9640 1 --2- 172.21.15.120:0/3844674695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b103a6b40 0x556b103a9b10 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7fb83c0a4600 tx=0x7fb83c009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4c4413cae40290 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:32.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.717+0000 7fb835ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:26:32.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.718+0000 7fb835ffb640 1 -- 172.21.15.120:0/3844674695 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c014070 con 0x556b103a6b40 2024-09-19T09:26:32.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.718+0000 7fb835ffb640 1 -- 172.21.15.120:0/3844674695 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fb83c0a3fb0 con 0x556b103a6b40 2024-09-19T09:26:32.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.718+0000 7fb856a27740 1 -- 172.21.15.120:0/3844674695 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb844036040 con 0x556b103a6b40 2024-09-19T09:26:32.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.718+0000 7fb835ffb640 1 -- 172.21.15.120:0/3844674695 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb83c0ce070 con 0x556b103a6b40 2024-09-19T09:26:32.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.718+0000 7fb856a27740 10 client.23118 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:32.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.718+0000 7fb856a27740 1 -- 172.21.15.120:0/3844674695 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556b1042aa00 con 0x556b103a6b40 2024-09-19T09:26:32.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.718+0000 7fb835ffb640 1 -- 172.21.15.120:0/3844674695 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb844036040 con 0x556b103a6b40 2024-09-19T09:26:32.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.718+0000 7fb835ffb640 1 -- 172.21.15.120:0/3844674695 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fb83c09acd0 con 0x556b103a6b40 2024-09-19T09:26:32.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.718+0000 7fb856a27740 10 client.23118 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:32.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.718+0000 7fb856a27740 10 client.23118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:26:32.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.718+0000 7fb856a27740 20 client.23118 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:32.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.718+0000 7fb856a27740 1 --2- 172.21.15.120:0/3844674695 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556b1042afe0 0x556b1042d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:32.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.718+0000 7fb856a27740 4 client.23118 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:26:32.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.718+0000 7fb856a27740 1 -- 172.21.15.120:0/3844674695 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b10312d70 con 0x556b1042afe0 2024-09-19T09:26:32.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.718+0000 7fb8523ea640 1 --2- 172.21.15.120:0/3844674695 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556b1042afe0 0x556b1042d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:32.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.719+0000 7fb835ffb640 10 client.23118 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:32.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.719+0000 7fb8523ea640 1 --2- 172.21.15.120:0/3844674695 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556b1042afe0 0x556b1042d400 crc :-1 s=READY pgs=578 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25e64263c888c56f server_cookie=15e1a78ccbf7eddb in_seq=0 out_seq=0 2024-09-19T09:26:32.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.719+0000 7fb835ffb640 1 -- 172.21.15.120:0/3844674695 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b10312d70 con 0x556b1042afe0 2024-09-19T09:26:32.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.719+0000 7fb835ffb640 10 client.23118 handle_command_reply: tid=0 2024-09-19T09:26:32.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.719+0000 7fb856a27740 1 client.23118 shutdown 2024-09-19T09:26:32.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.719+0000 7fb856a27740 1 -- 172.21.15.120:0/3844674695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b103a6b40 msgr2=0x556b103a9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:32.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.719+0000 7fb856a27740 1 --2- 172.21.15.120:0/3844674695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b103a6b40 0x556b103a9b10 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7fb83c0a4600 tx=0x7fb83c009e60 comp rx=0 tx=0).stop 2024-09-19T09:26:32.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.719+0000 7fb856a27740 1 -- 172.21.15.120:0/3844674695 shutdown_connections 2024-09-19T09:26:32.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.719+0000 7fb856a27740 1 --2- 172.21.15.120:0/3844674695 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x556b1042afe0 0x556b1042d400 crc :-1 s=READY pgs=578 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.719+0000 7fb856a27740 1 --2- 172.21.15.120:0/3844674695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b103b0370 0x556b103aa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.719+0000 7fb856a27740 1 --2- 172.21.15.120:0/3844674695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b103a6b40 0x556b103a9b10 unknown :-1 s=CLOSED pgs=2404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.719+0000 7fb856a27740 1 --2- 172.21.15.120:0/3844674695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b103a55d0 0x556b103b88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.719+0000 7fb856a27740 1 -- 172.21.15.120:0/3844674695 >> 172.21.15.120:0/3844674695 conn(0x556b1036e310 msgr2=0x556b103ac0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:32.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.719+0000 7fb835ffb640 0 client.23118 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:32.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.720+0000 7fb856a27740 1 -- 172.21.15.120:0/3844674695 shutdown_connections 2024-09-19T09:26:32.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.720+0000 7fb856a27740 1 -- 172.21.15.120:0/3844674695 wait complete. 2024-09-19T09:26:32.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.720+0000 7fb856a27740 20 client.23118 trim_cache size 0 max 16384 2024-09-19T09:26:32.748 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:26:32.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.720+0000 7fb85464e640 1 -- 172.21.15.120:0/168612914 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb8380a3240 msgr2=0x7fb8380a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:32.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.720+0000 7fb85464e640 1 --2- 172.21.15.120:0/168612914 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb8380a3240 0x7fb8380a5700 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x556b103a2ac0 tx=0x7fb84800b390 comp rx=0 tx=0).stop 2024-09-19T09:26:32.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.720+0000 7fb85464e640 1 -- 172.21.15.120:0/168612914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb84c1036e0 msgr2=0x7fb84c198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:32.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.720+0000 7fb85464e640 1 --2- 172.21.15.120:0/168612914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb84c1036e0 0x7fb84c198aa0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb83c00d750 tx=0x7fb83c00dc10 comp rx=0 tx=0).stop 2024-09-19T09:26:32.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.720+0000 7fb85464e640 1 -- 172.21.15.120:0/168612914 shutdown_connections 2024-09-19T09:26:32.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.720+0000 7fb85464e640 1 --2- 172.21.15.120:0/168612914 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb8380a3240 0x7fb8380a5700 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.720+0000 7fb85464e640 1 --2- 172.21.15.120:0/168612914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb84c1082a0 0x7fb84c19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.720+0000 7fb85464e640 1 --2- 172.21.15.120:0/168612914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb84c1036e0 0x7fb84c198aa0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.720+0000 7fb85464e640 1 --2- 172.21.15.120:0/168612914 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb84c102d30 0x7fb84c198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.720+0000 7fb85464e640 1 -- 172.21.15.120:0/168612914 >> 172.21.15.120:0/168612914 conn(0x7fb84c0fc7d0 msgr2=0x7fb84c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:32.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.720+0000 7fb85464e640 1 -- 172.21.15.120:0/168612914 shutdown_connections 2024-09-19T09:26:32.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:32.720+0000 7fb85464e640 1 -- 172.21.15.120:0/168612914 wait complete. 2024-09-19T09:26:33.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:32 smithi120 ceph-mon[31150]: pgmap v1055: 97 pgs: 97 active+clean; 491 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.7 MiB/s wr, 619 op/s 2024-09-19T09:26:33.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:32 smithi196 ceph-mon[35150]: pgmap v1055: 97 pgs: 97 active+clean; 491 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.7 MiB/s wr, 619 op/s 2024-09-19T09:26:33.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:32 smithi153 ceph-mon[35071]: pgmap v1055: 97 pgs: 97 active+clean; 491 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.7 MiB/s wr, 619 op/s 2024-09-19T09:26:33.388 INFO:tasks.workunit.client.0.smithi120.stdout: 10 251215 20.78 MB/sec execute 508 sec latency 4072.420 ms 2024-09-19T09:26:34.388 INFO:tasks.workunit.client.0.smithi120.stdout: 10 251215 20.74 MB/sec execute 509 sec latency 5072.464 ms 2024-09-19T09:26:34.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:34 smithi196 ceph-mon[35150]: pgmap v1056: 97 pgs: 97 active+clean; 483 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.0 MiB/s wr, 672 op/s 2024-09-19T09:26:34.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:34 smithi153 ceph-mon[35071]: pgmap v1056: 97 pgs: 97 active+clean; 483 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.0 MiB/s wr, 672 op/s 2024-09-19T09:26:34.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:34 smithi120 ceph-mon[31150]: pgmap v1056: 97 pgs: 97 active+clean; 483 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.0 MiB/s wr, 672 op/s 2024-09-19T09:26:34.578 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:26:34.579 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:26:34.823 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:35.388 INFO:tasks.workunit.client.0.smithi120.stdout: 10 251215 20.70 MB/sec execute 510 sec latency 6072.505 ms 2024-09-19T09:26:35.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:35 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:26:35.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:35 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:26:35.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.320+0000 7f104b811640 1 -- 172.21.15.120:0/861117813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1044106f10 msgr2=0x7f104410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:35.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.320+0000 7f104b811640 1 --2- 172.21.15.120:0/861117813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1044106f10 0x7f104410ea90 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f103c009ca0 tx=0x7f103c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.320+0000 7f104b811640 1 -- 172.21.15.120:0/861117813 shutdown_connections 2024-09-19T09:26:35.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.320+0000 7f104b811640 1 --2- 172.21.15.120:0/861117813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1044106f10 0x7f104410ea90 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.320+0000 7f104b811640 1 --2- 172.21.15.120:0/861117813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1044102790 0x7f10441067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.320+0000 7f104b811640 1 --2- 172.21.15.120:0/861117813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1044101de0 0x7f10441021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.320+0000 7f104b811640 1 -- 172.21.15.120:0/861117813 >> 172.21.15.120:0/861117813 conn(0x7f10440fd8c0 msgr2=0x7f10440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:35.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.320+0000 7f104b811640 1 -- 172.21.15.120:0/861117813 shutdown_connections 2024-09-19T09:26:35.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.320+0000 7f104b811640 1 -- 172.21.15.120:0/861117813 wait complete. 2024-09-19T09:26:35.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.321+0000 7f104b811640 1 Processor -- start 2024-09-19T09:26:35.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.321+0000 7f104b811640 1 -- start start 2024-09-19T09:26:35.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.321+0000 7f104b811640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1044101de0 0x7f10440744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:35.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.321+0000 7f104b811640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1044102790 0x7f1044074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:35.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.321+0000 7f104b811640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1044106f10 0x7f1044074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:35.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.321+0000 7f104b811640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1044111f70 con 0x7f1044106f10 2024-09-19T09:26:35.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.321+0000 7f104b811640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1044111df0 con 0x7f1044102790 2024-09-19T09:26:35.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.321+0000 7f104b811640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10441120f0 con 0x7f1044101de0 2024-09-19T09:26:35.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.321+0000 7f1049dae640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1044106f10 0x7f1044074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:35.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.321+0000 7f1049dae640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1044106f10 0x7f1044074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41294/0 (socket says 172.21.15.120:41294) 2024-09-19T09:26:35.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.321+0000 7f1049dae640 1 -- 172.21.15.120:0/61282157 learned_addr learned my addr 172.21.15.120:0/61282157 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:35.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.321+0000 7f10495ad640 1 --2- 172.21.15.120:0/61282157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1044101de0 0x7f10440744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:35.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.321+0000 7f1048dac640 1 --2- 172.21.15.120:0/61282157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1044102790 0x7f1044074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:35.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.322+0000 7f1049dae640 1 -- 172.21.15.120:0/61282157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1044101de0 msgr2=0x7f10440744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:35.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.322+0000 7f1049dae640 1 --2- 172.21.15.120:0/61282157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1044101de0 0x7f10440744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.322+0000 7f1049dae640 1 -- 172.21.15.120:0/61282157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1044102790 msgr2=0x7f1044074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:35.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.322+0000 7f1049dae640 1 --2- 172.21.15.120:0/61282157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1044102790 0x7f1044074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.322+0000 7f1049dae640 1 -- 172.21.15.120:0/61282157 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10441a6800 con 0x7f1044106f10 2024-09-19T09:26:35.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.322+0000 7f1049dae640 1 --2- 172.21.15.120:0/61282157 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1044106f10 0x7f1044074f70 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7f103c031a20 tx=0x7f103c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:35.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.322+0000 7f10367fc640 1 -- 172.21.15.120:0/61282157 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c003b70 con 0x7f1044106f10 2024-09-19T09:26:35.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.322+0000 7f10367fc640 1 -- 172.21.15.120:0/61282157 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f103c003d10 con 0x7f1044106f10 2024-09-19T09:26:35.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.322+0000 7f104b811640 1 -- 172.21.15.120:0/61282157 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10441a6a90 con 0x7f1044106f10 2024-09-19T09:26:35.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.322+0000 7f10367fc640 1 -- 172.21.15.120:0/61282157 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c02fc80 con 0x7f1044106f10 2024-09-19T09:26:35.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.322+0000 7f104b811640 1 -- 172.21.15.120:0/61282157 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10441a6fd0 con 0x7f1044106f10 2024-09-19T09:26:35.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.324+0000 7f10367fc640 1 -- 172.21.15.120:0/61282157 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f103c0046a0 con 0x7f1044106f10 2024-09-19T09:26:35.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.325+0000 7f104dbea740 1 Processor -- start 2024-09-19T09:26:35.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.325+0000 7f104dbea740 1 -- start start 2024-09-19T09:26:35.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.325+0000 7f104dbea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561453bbdb60 0x561453bbdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:35.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.325+0000 7f104dbea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561453bbe480 0x561453bc8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:35.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.325+0000 7f104dbea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561453bc8d40 0x561453bcb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:35.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.325+0000 7f104dbea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561453b23280 con 0x561453bc8d40 2024-09-19T09:26:35.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.325+0000 7f104dbea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561453acab10 con 0x561453bbdb60 2024-09-19T09:26:35.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.325+0000 7f104dbea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561453b1f240 con 0x561453bbe480 2024-09-19T09:26:35.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.325+0000 7f1048dac640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561453bc8d40 0x561453bcb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:35.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.325+0000 7f10367fc640 1 --2- 172.21.15.120:0/61282157 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f10300a3240 0x7f10300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:35.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.325+0000 7f1048dac640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561453bc8d40 0x561453bcb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41304/0 (socket says 172.21.15.120:41304) 2024-09-19T09:26:35.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.325+0000 7f1048dac640 1 -- 172.21.15.120:0/1078740824 learned_addr learned my addr 172.21.15.120:0/1078740824 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:35.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.326+0000 7f1048dac640 1 --2- 172.21.15.120:0/1078740824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561453bbdb60 0x561453bbdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:35.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.326+0000 7f1049dae640 1 --2- 172.21.15.120:0/61282157 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f10300a3240 0x7f10300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:35.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.326+0000 7f10495ad640 1 --2- 172.21.15.120:0/1078740824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561453bbe480 0x561453bc8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:35.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.326+0000 7f1048dac640 1 -- 172.21.15.120:0/1078740824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561453bbe480 msgr2=0x561453bc8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:35.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.326+0000 7f10367fc640 1 -- 172.21.15.120:0/61282157 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f103c0f0ce0 con 0x7f1044106f10 2024-09-19T09:26:35.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.326+0000 7f1048dac640 1 --2- 172.21.15.120:0/1078740824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561453bbe480 0x561453bc8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.326+0000 7f1048dac640 1 -- 172.21.15.120:0/1078740824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561453bbdb60 msgr2=0x561453bbdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:35.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.326+0000 7f1048dac640 1 --2- 172.21.15.120:0/1078740824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561453bbdb60 0x561453bbdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.326+0000 7f1048dac640 1 -- 172.21.15.120:0/1078740824 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561453ad71f0 con 0x561453bc8d40 2024-09-19T09:26:35.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.326+0000 7f1049dae640 1 --2- 172.21.15.120:0/61282157 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f10300a3240 0x7f10300a5700 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f103c0c5a00 tx=0x7f103c087ce0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:35.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.326+0000 7f1048dac640 1 --2- 172.21.15.120:0/1078740824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561453bc8d40 0x561453bcb130 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7f103800af90 tx=0x7f103800a790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d43d0aca96545de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:35.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.326+0000 7f102f7fe640 1 -- 172.21.15.120:0/1078740824 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103800f910 con 0x561453bc8d40 2024-09-19T09:26:35.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.326+0000 7f102f7fe640 1 -- 172.21.15.120:0/1078740824 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f103801b390 con 0x561453bc8d40 2024-09-19T09:26:35.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.326+0000 7f102f7fe640 1 -- 172.21.15.120:0/1078740824 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103800d5a0 con 0x561453bc8d40 2024-09-19T09:26:35.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.327+0000 7f104dbea740 1 -- 172.21.15.120:0/1078740824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561453bc8d40 msgr2=0x561453bcb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:35.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.327+0000 7f104dbea740 1 --2- 172.21.15.120:0/1078740824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561453bc8d40 0x561453bcb130 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7f103800af90 tx=0x7f103800a790 comp rx=0 tx=0).stop 2024-09-19T09:26:35.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.327+0000 7f104dbea740 1 -- 172.21.15.120:0/1078740824 shutdown_connections 2024-09-19T09:26:35.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.327+0000 7f104dbea740 1 --2- 172.21.15.120:0/1078740824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561453bc8d40 0x561453bcb130 unknown :-1 s=CLOSED pgs=2407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.327+0000 7f104dbea740 1 --2- 172.21.15.120:0/1078740824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561453bbe480 0x561453bc8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.327+0000 7f104dbea740 1 --2- 172.21.15.120:0/1078740824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561453bbdb60 0x561453bbdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.327+0000 7f104dbea740 1 -- 172.21.15.120:0/1078740824 >> 172.21.15.120:0/1078740824 conn(0x561453bba0f0 msgr2=0x561453bba4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:35.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.327+0000 7f104dbea740 1 -- 172.21.15.120:0/1078740824 shutdown_connections 2024-09-19T09:26:35.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.327+0000 7f104dbea740 1 -- 172.21.15.120:0/1078740824 wait complete. 2024-09-19T09:26:35.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.328+0000 7f104dbea740 1 -- start start 2024-09-19T09:26:35.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.328+0000 7f104dbea740 1 Processor -- start 2024-09-19T09:26:35.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.328+0000 7f104dbea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561453bbdb60 0x561453bc5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:35.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.328+0000 7f104dbea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561453bbe480 0x561453bc61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:35.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.328+0000 7f104dbea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561453bc8d40 0x561453bc66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:35.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.328+0000 7f104dbea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561453bcb910 con 0x561453bbe480 2024-09-19T09:26:35.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.328+0000 7f104dbea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561453addf10 con 0x561453bbdb60 2024-09-19T09:26:35.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.328+0000 7f104dbea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561453bcba90 con 0x561453bc8d40 2024-09-19T09:26:35.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.328+0000 7f104dbea740 10 client.0 _ll_register_callbacks cb 0x561453aa6a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:35.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.328+0000 7f10495ad640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561453bbe480 0x561453bc61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:35.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.328+0000 7f10495ad640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561453bbe480 0x561453bc61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41314/0 (socket says 172.21.15.120:41314) 2024-09-19T09:26:35.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.328+0000 7f10495ad640 1 -- 172.21.15.120:0/130108411 learned_addr learned my addr 172.21.15.120:0/130108411 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:35.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.328+0000 7f1048dac640 1 --2- 172.21.15.120:0/130108411 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561453bbdb60 0x561453bc5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:35.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.328+0000 7f10495ad640 1 -- 172.21.15.120:0/130108411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561453bc8d40 msgr2=0x561453bc66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:35.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.328+0000 7f10495ad640 1 --2- 172.21.15.120:0/130108411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561453bc8d40 0x561453bc66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.328+0000 7f10495ad640 1 -- 172.21.15.120:0/130108411 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561453bbdb60 msgr2=0x561453bc5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:35.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.328+0000 7f10495ad640 1 --2- 172.21.15.120:0/130108411 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561453bbdb60 0x561453bc5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.328+0000 7f10495ad640 1 -- 172.21.15.120:0/130108411 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614538946a0 con 0x561453bbe480 2024-09-19T09:26:35.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.329+0000 7f10495ad640 1 --2- 172.21.15.120:0/130108411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561453bbe480 0x561453bc61a0 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f104000ba50 tx=0x7f104000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a37c817df84ddf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:35.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.329+0000 7f102cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:26:35.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.329+0000 7f102cff9640 1 -- 172.21.15.120:0/130108411 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104001b640 con 0x561453bbe480 2024-09-19T09:26:35.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.329+0000 7f102cff9640 1 -- 172.21.15.120:0/130108411 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f104001b7e0 con 0x561453bbe480 2024-09-19T09:26:35.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.329+0000 7f104dbea740 1 -- 172.21.15.120:0/130108411 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f103801b390 con 0x561453bbe480 2024-09-19T09:26:35.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.329+0000 7f102cff9640 1 -- 172.21.15.120:0/130108411 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f104001b9e0 con 0x561453bbe480 2024-09-19T09:26:35.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.329+0000 7f104dbea740 10 client.23130 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:35.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.329+0000 7f104dbea740 1 -- 172.21.15.120:0/130108411 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561453bd75e0 con 0x561453bbe480 2024-09-19T09:26:35.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.329+0000 7f102cff9640 1 -- 172.21.15.120:0/130108411 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f103801b390 con 0x561453bbe480 2024-09-19T09:26:35.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.329+0000 7f102cff9640 1 -- 172.21.15.120:0/130108411 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f1040019020 con 0x561453bbe480 2024-09-19T09:26:35.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.329+0000 7f104dbea740 10 client.23130 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:35.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.329+0000 7f104dbea740 10 client.23130 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:26:35.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.329+0000 7f104dbea740 20 client.23130 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:35.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.329+0000 7f104dbea740 1 --2- 172.21.15.120:0/130108411 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561453bd8aa0 0x561453bdaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:35.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.329+0000 7f104dbea740 4 client.23130 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:26:35.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.329+0000 7f104dbea740 1 -- 172.21.15.120:0/130108411 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561453b307f0 con 0x561453bd8aa0 2024-09-19T09:26:35.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.330+0000 7f1048dac640 1 --2- 172.21.15.120:0/130108411 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561453bd8aa0 0x561453bdaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:35.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.330+0000 7f102cff9640 10 client.23130 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:35.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.330+0000 7f1048dac640 1 --2- 172.21.15.120:0/130108411 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561453bd8aa0 0x561453bdaeb0 crc :-1 s=READY pgs=579 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e2fc16869c82b63 server_cookie=8f893cc137c4b190 in_seq=0 out_seq=0 2024-09-19T09:26:35.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.331+0000 7f102cff9640 1 -- 172.21.15.120:0/130108411 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561453b307f0 con 0x561453bd8aa0 2024-09-19T09:26:35.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.331+0000 7f102cff9640 10 client.23130 handle_command_reply: tid=0 2024-09-19T09:26:35.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.331+0000 7f104dbea740 1 client.23130 shutdown 2024-09-19T09:26:35.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.331+0000 7f104dbea740 1 -- 172.21.15.120:0/130108411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561453bbe480 msgr2=0x561453bc61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:35.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.331+0000 7f104dbea740 1 --2- 172.21.15.120:0/130108411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561453bbe480 0x561453bc61a0 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f104000ba50 tx=0x7f104000bf10 comp rx=0 tx=0).stop 2024-09-19T09:26:35.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.331+0000 7f104dbea740 1 -- 172.21.15.120:0/130108411 shutdown_connections 2024-09-19T09:26:35.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.331+0000 7f104dbea740 1 --2- 172.21.15.120:0/130108411 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561453bd8aa0 0x561453bdaeb0 crc :-1 s=READY pgs=579 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.331+0000 7f104dbea740 1 --2- 172.21.15.120:0/130108411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561453bc8d40 0x561453bc66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.331+0000 7f102cff9640 0 client.23130 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:35.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.331+0000 7f104dbea740 1 --2- 172.21.15.120:0/130108411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561453bbe480 0x561453bc61a0 unknown :-1 s=CLOSED pgs=2408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.331+0000 7f104dbea740 1 --2- 172.21.15.120:0/130108411 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561453bbdb60 0x561453bc5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.331+0000 7f104dbea740 1 -- 172.21.15.120:0/130108411 >> 172.21.15.120:0/130108411 conn(0x561453bba0f0 msgr2=0x561453bbfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:35.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.331+0000 7f104dbea740 1 -- 172.21.15.120:0/130108411 shutdown_connections 2024-09-19T09:26:35.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.331+0000 7f104dbea740 1 -- 172.21.15.120:0/130108411 wait complete. 2024-09-19T09:26:35.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.331+0000 7f104dbea740 20 client.23130 trim_cache size 0 max 16384 2024-09-19T09:26:35.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.348+0000 7f104dbea740 1 Processor -- start 2024-09-19T09:26:35.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.348+0000 7f104dbea740 1 -- start start 2024-09-19T09:26:35.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.348+0000 7f104dbea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561453bbdb60 0x561453bc4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:35.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.348+0000 7f104dbea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561453bbe480 0x561453be07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:35.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.348+0000 7f104dbea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561453bc8d40 0x561453be0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:35.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.348+0000 7f104dbea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f10441af6d0 con 0x561453bc8d40 2024-09-19T09:26:35.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.348+0000 7f104dbea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f10441b1210 con 0x561453bbdb60 2024-09-19T09:26:35.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.348+0000 7f104dbea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10441afda0 con 0x561453bbe480 2024-09-19T09:26:35.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.348+0000 7f1048dac640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561453bc8d40 0x561453be0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:35.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.349+0000 7f1048dac640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561453bc8d40 0x561453be0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41318/0 (socket says 172.21.15.120:41318) 2024-09-19T09:26:35.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.349+0000 7f1048dac640 1 -- 172.21.15.120:0/2653139372 learned_addr learned my addr 172.21.15.120:0/2653139372 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:35.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.349+0000 7f1048dac640 1 --2- 172.21.15.120:0/2653139372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561453bbdb60 0x561453bc4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:35.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.349+0000 7f10495ad640 1 --2- 172.21.15.120:0/2653139372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561453bbe480 0x561453be07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:35.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.349+0000 7f1048dac640 1 -- 172.21.15.120:0/2653139372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561453bbe480 msgr2=0x561453be07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:35.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.349+0000 7f1048dac640 1 --2- 172.21.15.120:0/2653139372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561453bbe480 0x561453be07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.349+0000 7f1048dac640 1 -- 172.21.15.120:0/2653139372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561453bbdb60 msgr2=0x561453bc4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:35.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.349+0000 7f1048dac640 1 --2- 172.21.15.120:0/2653139372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561453bbdb60 0x561453bc4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.349+0000 7f1048dac640 1 -- 172.21.15.120:0/2653139372 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561453bb9b90 con 0x561453bc8d40 2024-09-19T09:26:35.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.349+0000 7f1048dac640 1 --2- 172.21.15.120:0/2653139372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561453bc8d40 0x561453be0d00 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7f1038011f00 tx=0x7f103801bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8276dcc2898d0d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:35.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.349+0000 7f102effd640 1 -- 172.21.15.120:0/2653139372 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103802b040 con 0x561453bc8d40 2024-09-19T09:26:35.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.349+0000 7f102effd640 1 -- 172.21.15.120:0/2653139372 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1038005510 con 0x561453bc8d40 2024-09-19T09:26:35.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.349+0000 7f102effd640 1 -- 172.21.15.120:0/2653139372 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038021040 con 0x561453bc8d40 2024-09-19T09:26:35.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.350+0000 7f104dbea740 1 -- 172.21.15.120:0/2653139372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561453bc8d40 msgr2=0x561453be0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:35.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.350+0000 7f104dbea740 1 --2- 172.21.15.120:0/2653139372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561453bc8d40 0x561453be0d00 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7f1038011f00 tx=0x7f103801bcf0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.350+0000 7f104dbea740 1 -- 172.21.15.120:0/2653139372 shutdown_connections 2024-09-19T09:26:35.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.350+0000 7f104dbea740 1 --2- 172.21.15.120:0/2653139372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561453bc8d40 0x561453be0d00 unknown :-1 s=CLOSED pgs=2409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.350+0000 7f104dbea740 1 --2- 172.21.15.120:0/2653139372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561453bbe480 0x561453be07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.350+0000 7f104dbea740 1 --2- 172.21.15.120:0/2653139372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561453bbdb60 0x561453bc4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.350+0000 7f104dbea740 1 -- 172.21.15.120:0/2653139372 >> 172.21.15.120:0/2653139372 conn(0x561453bba0f0 msgr2=0x561453bc7bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:35.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.350+0000 7f104dbea740 1 -- 172.21.15.120:0/2653139372 shutdown_connections 2024-09-19T09:26:35.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.350+0000 7f104dbea740 1 -- 172.21.15.120:0/2653139372 wait complete. 2024-09-19T09:26:35.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.350+0000 7f104dbea740 1 -- start start 2024-09-19T09:26:35.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.350+0000 7f104dbea740 1 Processor -- start 2024-09-19T09:26:35.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.351+0000 7f104dbea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561453bbdb60 0x561453bcd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:35.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.351+0000 7f104dbea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561453bbe480 0x561453bcd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:35.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.351+0000 7f104dbea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561453bc8d40 0x561453c3c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:35.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.351+0000 7f104dbea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f10441b0b40 con 0x561453bbdb60 2024-09-19T09:26:35.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.351+0000 7f104dbea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f10441b0470 con 0x561453bc8d40 2024-09-19T09:26:35.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.351+0000 7f104dbea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10441ad690 con 0x561453bbe480 2024-09-19T09:26:35.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.351+0000 7f104dbea740 10 client.0 _ll_register_callbacks cb 0x7f1040004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:35.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.351+0000 7f1048dac640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561453bbdb60 0x561453bcd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:35.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.351+0000 7f1048dac640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561453bbdb60 0x561453bcd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41320/0 (socket says 172.21.15.120:41320) 2024-09-19T09:26:35.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.351+0000 7f1048dac640 1 -- 172.21.15.120:0/1592514075 learned_addr learned my addr 172.21.15.120:0/1592514075 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:35.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.351+0000 7f10495ad640 1 --2- 172.21.15.120:0/1592514075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561453bbe480 0x561453bcd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:35.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.351+0000 7f1048dac640 1 --2- 172.21.15.120:0/1592514075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561453bc8d40 0x561453c3c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:35.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.351+0000 7f1048dac640 1 -- 172.21.15.120:0/1592514075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561453bbe480 msgr2=0x561453bcd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:35.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.351+0000 7f1048dac640 1 --2- 172.21.15.120:0/1592514075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561453bbe480 0x561453bcd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.351+0000 7f1048dac640 1 -- 172.21.15.120:0/1592514075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561453bc8d40 msgr2=0x561453c3c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:35.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.351+0000 7f1048dac640 1 --2- 172.21.15.120:0/1592514075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561453bc8d40 0x561453c3c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.351+0000 7f1048dac640 1 -- 172.21.15.120:0/1592514075 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561453c3cd90 con 0x561453bbdb60 2024-09-19T09:26:35.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.351+0000 7f10495ad640 1 --2- 172.21.15.120:0/1592514075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561453bbe480 0x561453bcd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:35.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.351+0000 7f1048dac640 1 --2- 172.21.15.120:0/1592514075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561453bbdb60 0x561453bcd330 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f103801c000 tx=0x7f103801b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8f65fd36a59b5f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:35.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.351+0000 7f102cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:26:35.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.351+0000 7f102cff9640 1 -- 172.21.15.120:0/1592514075 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038037020 con 0x561453bbdb60 2024-09-19T09:26:35.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.351+0000 7f102cff9640 1 -- 172.21.15.120:0/1592514075 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f103801b590 con 0x561453bbdb60 2024-09-19T09:26:35.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.351+0000 7f104dbea740 1 -- 172.21.15.120:0/1592514075 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1038005510 con 0x561453bbdb60 2024-09-19T09:26:35.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.352+0000 7f102cff9640 1 -- 172.21.15.120:0/1592514075 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1038021040 con 0x561453bbdb60 2024-09-19T09:26:35.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.352+0000 7f104dbea740 10 client.23136 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:35.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.352+0000 7f104dbea740 1 -- 172.21.15.120:0/1592514075 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561453c44180 con 0x561453bbdb60 2024-09-19T09:26:35.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.352+0000 7f102cff9640 1 -- 172.21.15.120:0/1592514075 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1038005510 con 0x561453bbdb60 2024-09-19T09:26:35.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.352+0000 7f102cff9640 1 -- 172.21.15.120:0/1592514075 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f103801b070 con 0x561453bbdb60 2024-09-19T09:26:35.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.352+0000 7f104dbea740 10 client.23136 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:35.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.352+0000 7f104dbea740 10 client.23136 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:26:35.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.352+0000 7f104dbea740 20 client.23136 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:35.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.352+0000 7f104dbea740 1 --2- 172.21.15.120:0/1592514075 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561453c445a0 0x561453c46990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:35.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.352+0000 7f104dbea740 4 client.23136 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:26:35.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.352+0000 7f104dbea740 1 -- 172.21.15.120:0/1592514075 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561453b2ad00 con 0x561453c445a0 2024-09-19T09:26:35.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.352+0000 7f10495ad640 1 --2- 172.21.15.120:0/1592514075 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561453c445a0 0x561453c46990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:35.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.353+0000 7f10495ad640 1 --2- 172.21.15.120:0/1592514075 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561453c445a0 0x561453c46990 crc :-1 s=READY pgs=580 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=354eef6b4d55f18e server_cookie=ad14ee283dee0e4a in_seq=0 out_seq=0 2024-09-19T09:26:35.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.353+0000 7f102cff9640 10 client.23136 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:35.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.354+0000 7f102cff9640 1 -- 172.21.15.120:0/1592514075 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561453b2ad00 con 0x561453c445a0 2024-09-19T09:26:35.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.354+0000 7f102cff9640 10 client.23136 handle_command_reply: tid=0 2024-09-19T09:26:35.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.354+0000 7f104dbea740 1 client.23136 shutdown 2024-09-19T09:26:35.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.354+0000 7f104dbea740 1 -- 172.21.15.120:0/1592514075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561453bbdb60 msgr2=0x561453bcd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:35.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.354+0000 7f104dbea740 1 --2- 172.21.15.120:0/1592514075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561453bbdb60 0x561453bcd330 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f103801c000 tx=0x7f103801b840 comp rx=0 tx=0).stop 2024-09-19T09:26:35.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.354+0000 7f104dbea740 1 -- 172.21.15.120:0/1592514075 shutdown_connections 2024-09-19T09:26:35.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.354+0000 7f104dbea740 1 --2- 172.21.15.120:0/1592514075 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x561453c445a0 0x561453c46990 crc :-1 s=READY pgs=580 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.354+0000 7f104dbea740 1 --2- 172.21.15.120:0/1592514075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561453bc8d40 0x561453c3c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.354+0000 7f104dbea740 1 --2- 172.21.15.120:0/1592514075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561453bbe480 0x561453bcd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.354+0000 7f104dbea740 1 --2- 172.21.15.120:0/1592514075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561453bbdb60 0x561453bcd330 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.354+0000 7f102cff9640 0 client.23136 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:35.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.354+0000 7f104dbea740 1 -- 172.21.15.120:0/1592514075 >> 172.21.15.120:0/1592514075 conn(0x561453bba0f0 msgr2=0x561453bc72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:35.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.354+0000 7f104dbea740 1 -- 172.21.15.120:0/1592514075 shutdown_connections 2024-09-19T09:26:35.702 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:26:35.702 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:26:35.702 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "e4d617a6-bfb5-4362-a5a0-8a58dacae253", 2024-09-19T09:26:35.702 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:26:35.702 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:26:35.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.354+0000 7f104dbea740 1 -- 172.21.15.120:0/1592514075 wait complete. 2024-09-19T09:26:35.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.354+0000 7f104dbea740 20 client.23136 trim_cache size 0 max 16384 2024-09-19T09:26:35.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.355+0000 7f104b811640 1 -- 172.21.15.120:0/61282157 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f10300a3240 msgr2=0x7f10300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:35.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.355+0000 7f104b811640 1 --2- 172.21.15.120:0/61282157 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f10300a3240 0x7f10300a5700 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f103c0c5a00 tx=0x7f103c087ce0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.355+0000 7f104b811640 1 -- 172.21.15.120:0/61282157 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1044106f10 msgr2=0x7f1044074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:35.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.355+0000 7f104b811640 1 --2- 172.21.15.120:0/61282157 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1044106f10 0x7f1044074f70 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7f103c031a20 tx=0x7f103c031a50 comp rx=0 tx=0).stop 2024-09-19T09:26:35.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.355+0000 7f104b811640 1 -- 172.21.15.120:0/61282157 shutdown_connections 2024-09-19T09:26:35.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.355+0000 7f104b811640 1 --2- 172.21.15.120:0/61282157 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f10300a3240 0x7f10300a5700 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.355+0000 7f104b811640 1 --2- 172.21.15.120:0/61282157 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1044106f10 0x7f1044074f70 unknown :-1 s=CLOSED pgs=2406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.355+0000 7f104b811640 1 --2- 172.21.15.120:0/61282157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1044102790 0x7f1044074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.355+0000 7f104b811640 1 --2- 172.21.15.120:0/61282157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1044101de0 0x7f10440744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.355+0000 7f104b811640 1 -- 172.21.15.120:0/61282157 >> 172.21.15.120:0/61282157 conn(0x7f10440fd8c0 msgr2=0x7f10440febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:35.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.355+0000 7f104b811640 1 -- 172.21.15.120:0/61282157 shutdown_connections 2024-09-19T09:26:35.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:35.355+0000 7f104b811640 1 -- 172.21.15.120:0/61282157 wait complete. 2024-09-19T09:26:35.704 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:35 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:26:35.704 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:35 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:26:35.704 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:35 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:26:35.705 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:35 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:26:36.388 INFO:tasks.workunit.client.0.smithi120.stdout: 10 251215 20.66 MB/sec execute 511 sec latency 7072.855 ms 2024-09-19T09:26:36.583 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:26:36.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:36 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:26:36.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:36 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:26:36.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:36 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:26:36.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:36 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:26:36.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:36 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:26:36.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:36 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:26:36.827 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:36 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:26:36.827 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:36 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:26:36.827 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:36 smithi120 ceph-mon[31150]: pgmap v1057: 97 pgs: 97 active+clean; 483 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.7 MiB/s wr, 587 op/s 2024-09-19T09:26:36.833 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:36.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:36 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:26:36.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:36 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:26:36.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:36 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:26:36.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:36 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:26:36.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:36 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:26:36.938 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:36 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:26:36.938 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:36 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:26:36.938 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:36 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:26:36.938 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:36 smithi196 ceph-mon[35150]: pgmap v1057: 97 pgs: 97 active+clean; 483 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.7 MiB/s wr, 587 op/s 2024-09-19T09:26:36.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:36 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:26:36.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:36 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:26:36.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:36 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:26:36.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:36 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:26:36.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:36 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:26:36.939 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:36 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:26:36.939 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:36 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:26:36.939 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:36 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:26:36.939 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:36 smithi153 ceph-mon[35071]: pgmap v1057: 97 pgs: 97 active+clean; 483 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.7 MiB/s wr, 587 op/s 2024-09-19T09:26:37.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.281+0000 7f370d38d640 1 -- 172.21.15.120:0/3203060830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3708106f10 msgr2=0x7f370810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:37.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.281+0000 7f370d38d640 1 --2- 172.21.15.120:0/3203060830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3708106f10 0x7f370810ea90 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f3700009ca0 tx=0x7f370002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.281+0000 7f370d38d640 1 -- 172.21.15.120:0/3203060830 shutdown_connections 2024-09-19T09:26:37.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.281+0000 7f370d38d640 1 --2- 172.21.15.120:0/3203060830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3708106f10 0x7f370810ea90 unknown :-1 s=CLOSED pgs=2411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.281+0000 7f370d38d640 1 --2- 172.21.15.120:0/3203060830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3708102790 0x7f37081067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.281+0000 7f370d38d640 1 --2- 172.21.15.120:0/3203060830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3708101de0 0x7f37081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.281+0000 7f370d38d640 1 -- 172.21.15.120:0/3203060830 >> 172.21.15.120:0/3203060830 conn(0x7f37080fd8c0 msgr2=0x7f37080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:37.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.281+0000 7f370d38d640 1 -- 172.21.15.120:0/3203060830 shutdown_connections 2024-09-19T09:26:37.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.281+0000 7f370d38d640 1 -- 172.21.15.120:0/3203060830 wait complete. 2024-09-19T09:26:37.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.282+0000 7f370d38d640 1 Processor -- start 2024-09-19T09:26:37.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.282+0000 7f370d38d640 1 -- start start 2024-09-19T09:26:37.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.282+0000 7f370d38d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3708101de0 0x7f3708198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:37.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.282+0000 7f370d38d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3708102790 0x7f37081989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:37.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.282+0000 7f370d38d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3708106f10 0x7f370819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:37.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.282+0000 7f370d38d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3708111fe0 con 0x7f3708101de0 2024-09-19T09:26:37.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.282+0000 7f370d38d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3708111e60 con 0x7f3708106f10 2024-09-19T09:26:37.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.282+0000 7f370d38d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3708112160 con 0x7f3708102790 2024-09-19T09:26:37.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.282+0000 7f3706ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3708101de0 0x7f3708198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:37.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.282+0000 7f3706ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3708101de0 0x7f3708198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41344/0 (socket says 172.21.15.120:41344) 2024-09-19T09:26:37.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.282+0000 7f3706ffd640 1 -- 172.21.15.120:0/323801895 learned_addr learned my addr 172.21.15.120:0/323801895 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:37.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.282+0000 7f3706ffd640 1 -- 172.21.15.120:0/323801895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3708102790 msgr2=0x7f37081989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:37.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.282+0000 7f3706ffd640 1 --2- 172.21.15.120:0/323801895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3708102790 0x7f37081989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.282+0000 7f3706ffd640 1 -- 172.21.15.120:0/323801895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3708106f10 msgr2=0x7f370819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:37.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.282+0000 7f3706ffd640 1 --2- 172.21.15.120:0/323801895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3708106f10 0x7f370819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.282+0000 7f3706ffd640 1 -- 172.21.15.120:0/323801895 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f370819d430 con 0x7f3708101de0 2024-09-19T09:26:37.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.283+0000 7f3706ffd640 1 --2- 172.21.15.120:0/323801895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3708101de0 0x7f3708198480 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7f36f800c890 tx=0x7f36f800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:37.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.283+0000 7f3704ff9640 1 -- 172.21.15.120:0/323801895 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8004220 con 0x7f3708101de0 2024-09-19T09:26:37.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.283+0000 7f3704ff9640 1 -- 172.21.15.120:0/323801895 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36f80043c0 con 0x7f3708101de0 2024-09-19T09:26:37.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.283+0000 7f370d38d640 1 -- 172.21.15.120:0/323801895 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f370819d720 con 0x7f3708101de0 2024-09-19T09:26:37.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.283+0000 7f3704ff9640 1 -- 172.21.15.120:0/323801895 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f80049e0 con 0x7f3708101de0 2024-09-19T09:26:37.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.283+0000 7f370d38d640 1 -- 172.21.15.120:0/323801895 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37081a5000 con 0x7f3708101de0 2024-09-19T09:26:37.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.285+0000 7f3704ff9640 1 -- 172.21.15.120:0/323801895 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f36f80079b0 con 0x7f3708101de0 2024-09-19T09:26:37.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.285+0000 7f370f766740 1 Processor -- start 2024-09-19T09:26:37.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.285+0000 7f370f766740 1 -- start start 2024-09-19T09:26:37.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.286+0000 7f370f766740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ddb4cd360 0x557ddb4cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:37.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.286+0000 7f370f766740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ddb4cdc80 0x557ddb4dc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:37.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.286+0000 7f370f766740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ddb4dca90 0x557ddb4dee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:37.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.286+0000 7f370f766740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557ddb4332e0 con 0x557ddb4dca90 2024-09-19T09:26:37.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.286+0000 7f370f766740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557ddb3dace0 con 0x557ddb4cdc80 2024-09-19T09:26:37.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.286+0000 7f370f766740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557ddb42f2a0 con 0x557ddb4cd360 2024-09-19T09:26:37.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.286+0000 7f36fffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ddb4dca90 0x557ddb4dee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:37.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.286+0000 7f36fffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ddb4dca90 0x557ddb4dee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41350/0 (socket says 172.21.15.120:41350) 2024-09-19T09:26:37.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.286+0000 7f36fffff640 1 -- 172.21.15.120:0/306415672 learned_addr learned my addr 172.21.15.120:0/306415672 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:37.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.286+0000 7f3704ff9640 1 --2- 172.21.15.120:0/323801895 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f36f40a32c0 0x7f36f40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:37.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.286+0000 7f3706ffd640 1 --2- 172.21.15.120:0/306415672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ddb4cdc80 0x557ddb4dc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:37.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.286+0000 7f37077fe640 1 --2- 172.21.15.120:0/306415672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ddb4cd360 0x557ddb4cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:37.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.286+0000 7f37077fe640 1 --2- 172.21.15.120:0/323801895 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f36f40a32c0 0x7f36f40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:37.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.286+0000 7f36fffff640 1 -- 172.21.15.120:0/306415672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ddb4cd360 msgr2=0x557ddb4cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:37.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.286+0000 7f36fffff640 1 --2- 172.21.15.120:0/306415672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ddb4cd360 0x557ddb4cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.286+0000 7f36fffff640 1 -- 172.21.15.120:0/306415672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ddb4cdc80 msgr2=0x557ddb4dc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:37.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.286+0000 7f36fffff640 1 --2- 172.21.15.120:0/306415672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ddb4cdc80 0x557ddb4dc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.286+0000 7f36fffff640 1 -- 172.21.15.120:0/306415672 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ddb3bf440 con 0x557ddb4dca90 2024-09-19T09:26:37.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.286+0000 7f37077fe640 1 --2- 172.21.15.120:0/306415672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ddb4cd360 0x557ddb4cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:37.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.286+0000 7f3704ff9640 1 -- 172.21.15.120:0/323801895 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f36f80cfa60 con 0x7f3708101de0 2024-09-19T09:26:37.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.286+0000 7f3706ffd640 1 --2- 172.21.15.120:0/306415672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ddb4cdc80 0x557ddb4dc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:37.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.287+0000 7f36fffff640 1 --2- 172.21.15.120:0/306415672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ddb4dca90 0x557ddb4dee80 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f36f00097f0 tx=0x7f36f0009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c646589e4fb1d38a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:37.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.287+0000 7f37077fe640 1 --2- 172.21.15.120:0/323801895 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f36f40a32c0 0x7f36f40a5780 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3700006010 tx=0x7f37000095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:37.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.287+0000 7f36fd7fa640 1 -- 172.21.15.120:0/306415672 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f0015020 con 0x557ddb4dca90 2024-09-19T09:26:37.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.287+0000 7f36fd7fa640 1 -- 172.21.15.120:0/306415672 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36f0011420 con 0x557ddb4dca90 2024-09-19T09:26:37.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.287+0000 7f36fd7fa640 1 -- 172.21.15.120:0/306415672 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f00105e0 con 0x557ddb4dca90 2024-09-19T09:26:37.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.287+0000 7f370f766740 1 -- 172.21.15.120:0/306415672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ddb4dca90 msgr2=0x557ddb4dee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:37.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.287+0000 7f370f766740 1 --2- 172.21.15.120:0/306415672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ddb4dca90 0x557ddb4dee80 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f36f00097f0 tx=0x7f36f0009cb0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.288+0000 7f370f766740 1 -- 172.21.15.120:0/306415672 shutdown_connections 2024-09-19T09:26:37.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.288+0000 7f370f766740 1 --2- 172.21.15.120:0/306415672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ddb4dca90 0x557ddb4dee80 unknown :-1 s=CLOSED pgs=2413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.288+0000 7f370f766740 1 --2- 172.21.15.120:0/306415672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ddb4cdc80 0x557ddb4dc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.288+0000 7f370f766740 1 --2- 172.21.15.120:0/306415672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ddb4cd360 0x557ddb4cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.288+0000 7f370f766740 1 -- 172.21.15.120:0/306415672 >> 172.21.15.120:0/306415672 conn(0x557ddb4c9a30 msgr2=0x557ddb4c9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:37.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.288+0000 7f370f766740 1 -- 172.21.15.120:0/306415672 shutdown_connections 2024-09-19T09:26:37.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.288+0000 7f370f766740 1 -- 172.21.15.120:0/306415672 wait complete. 2024-09-19T09:26:37.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.288+0000 7f370f766740 1 -- start start 2024-09-19T09:26:37.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.288+0000 7f370f766740 1 Processor -- start 2024-09-19T09:26:37.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.288+0000 7f370f766740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ddb4cd360 0x557ddb4dc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:37.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.288+0000 7f370f766740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ddb4cdc80 0x557ddb4d9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:37.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.288+0000 7f370f766740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ddb4dca90 0x557ddb4d9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:37.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.288+0000 7f370f766740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557ddb4df660 con 0x557ddb4cd360 2024-09-19T09:26:37.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.288+0000 7f370f766740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557ddb3ee050 con 0x557ddb4dca90 2024-09-19T09:26:37.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.288+0000 7f370f766740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557ddb4df7e0 con 0x557ddb4cdc80 2024-09-19T09:26:37.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.288+0000 7f370f766740 10 client.0 _ll_register_callbacks cb 0x557ddb42e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:37.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.288+0000 7f3706ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ddb4cd360 0x557ddb4dc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:37.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.288+0000 7f3706ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ddb4cd360 0x557ddb4dc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41358/0 (socket says 172.21.15.120:41358) 2024-09-19T09:26:37.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.288+0000 7f3706ffd640 1 -- 172.21.15.120:0/4270685949 learned_addr learned my addr 172.21.15.120:0/4270685949 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:37.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.289+0000 7f37077fe640 1 --2- 172.21.15.120:0/4270685949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ddb4dca90 0x557ddb4d9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:37.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.289+0000 7f36fffff640 1 --2- 172.21.15.120:0/4270685949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ddb4cdc80 0x557ddb4d9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:37.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.289+0000 7f3706ffd640 1 -- 172.21.15.120:0/4270685949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ddb4cdc80 msgr2=0x557ddb4d9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:37.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.289+0000 7f3706ffd640 1 --2- 172.21.15.120:0/4270685949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ddb4cdc80 0x557ddb4d9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.289+0000 7f3706ffd640 1 -- 172.21.15.120:0/4270685949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ddb4dca90 msgr2=0x557ddb4d9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:37.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.289+0000 7f3706ffd640 1 --2- 172.21.15.120:0/4270685949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ddb4dca90 0x557ddb4d9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.289+0000 7f3706ffd640 1 -- 172.21.15.120:0/4270685949 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ddb1a43c0 con 0x557ddb4cd360 2024-09-19T09:26:37.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.289+0000 7f37077fe640 1 --2- 172.21.15.120:0/4270685949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ddb4dca90 0x557ddb4d9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:37.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.289+0000 7f3706ffd640 1 --2- 172.21.15.120:0/4270685949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ddb4cd360 0x557ddb4dc3d0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f36f8098a10 tx=0x7f36f809bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2298450bedbd9e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:37.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.289+0000 7f36ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:26:37.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.289+0000 7f36ee7fc640 1 -- 172.21.15.120:0/4270685949 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f801d030 con 0x557ddb4cd360 2024-09-19T09:26:37.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.289+0000 7f36ee7fc640 1 -- 172.21.15.120:0/4270685949 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f36f809a340 con 0x557ddb4cd360 2024-09-19T09:26:37.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.289+0000 7f370f766740 1 -- 172.21.15.120:0/4270685949 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36f0011420 con 0x557ddb4cd360 2024-09-19T09:26:37.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.289+0000 7f36ee7fc640 1 -- 172.21.15.120:0/4270685949 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f36f80a03f0 con 0x557ddb4cd360 2024-09-19T09:26:37.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.289+0000 7f370f766740 10 client.23148 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:37.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.289+0000 7f370f766740 1 -- 172.21.15.120:0/4270685949 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557ddb4eb0b0 con 0x557ddb4cd360 2024-09-19T09:26:37.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.289+0000 7f36ee7fc640 1 -- 172.21.15.120:0/4270685949 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36f0011420 con 0x557ddb4cd360 2024-09-19T09:26:37.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.289+0000 7f36ee7fc640 1 -- 172.21.15.120:0/4270685949 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f36f809c2a0 con 0x557ddb4cd360 2024-09-19T09:26:37.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.289+0000 7f370f766740 10 client.23148 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:37.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.289+0000 7f370f766740 10 client.23148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:26:37.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.290+0000 7f370f766740 20 client.23148 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:37.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.290+0000 7f370f766740 1 --2- 172.21.15.120:0/4270685949 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557ddb4ec4c0 0x557ddb4ee8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:37.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.290+0000 7f370f766740 4 client.23148 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:26:37.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.290+0000 7f370f766740 1 -- 172.21.15.120:0/4270685949 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ddb440850 con 0x557ddb4ec4c0 2024-09-19T09:26:37.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.290+0000 7f36fffff640 1 --2- 172.21.15.120:0/4270685949 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557ddb4ec4c0 0x557ddb4ee8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:37.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.290+0000 7f36ee7fc640 10 client.23148 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:37.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.290+0000 7f36fffff640 1 --2- 172.21.15.120:0/4270685949 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557ddb4ec4c0 0x557ddb4ee8f0 crc :-1 s=READY pgs=581 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25eaea0c2bac75a9 server_cookie=8aff838f6e30d01a in_seq=0 out_seq=0 2024-09-19T09:26:37.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.291+0000 7f36ee7fc640 1 -- 172.21.15.120:0/4270685949 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557ddb440850 con 0x557ddb4ec4c0 2024-09-19T09:26:37.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.291+0000 7f36ee7fc640 10 client.23148 handle_command_reply: tid=0 2024-09-19T09:26:37.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.291+0000 7f370f766740 1 client.23148 shutdown 2024-09-19T09:26:37.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.291+0000 7f370f766740 1 -- 172.21.15.120:0/4270685949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ddb4cd360 msgr2=0x557ddb4dc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:37.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.291+0000 7f370f766740 1 --2- 172.21.15.120:0/4270685949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ddb4cd360 0x557ddb4dc3d0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f36f8098a10 tx=0x7f36f809bc40 comp rx=0 tx=0).stop 2024-09-19T09:26:37.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.291+0000 7f370f766740 1 -- 172.21.15.120:0/4270685949 shutdown_connections 2024-09-19T09:26:37.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.291+0000 7f370f766740 1 --2- 172.21.15.120:0/4270685949 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557ddb4ec4c0 0x557ddb4ee8f0 crc :-1 s=READY pgs=581 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.291+0000 7f370f766740 1 --2- 172.21.15.120:0/4270685949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ddb4dca90 0x557ddb4d9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.291+0000 7f370f766740 1 --2- 172.21.15.120:0/4270685949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ddb4cdc80 0x557ddb4d9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.291+0000 7f36ee7fc640 0 client.23148 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:37.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.291+0000 7f370f766740 1 --2- 172.21.15.120:0/4270685949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ddb4cd360 0x557ddb4dc3d0 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.291+0000 7f370f766740 1 -- 172.21.15.120:0/4270685949 >> 172.21.15.120:0/4270685949 conn(0x557ddb4c9a30 msgr2=0x557ddb4cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:37.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.291+0000 7f370f766740 1 -- 172.21.15.120:0/4270685949 shutdown_connections 2024-09-19T09:26:37.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.291+0000 7f370f766740 1 -- 172.21.15.120:0/4270685949 wait complete. 2024-09-19T09:26:37.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.291+0000 7f370f766740 20 client.23148 trim_cache size 0 max 16384 2024-09-19T09:26:37.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.308+0000 7f370f766740 1 Processor -- start 2024-09-19T09:26:37.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.308+0000 7f370f766740 1 -- start start 2024-09-19T09:26:37.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.308+0000 7f370f766740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ddb4cd360 0x557ddb4e8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:37.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.308+0000 7f370f766740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ddb4cdc80 0x557ddb4f41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:37.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.308+0000 7f370f766740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ddb4dca90 0x557ddb4f4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:37.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.308+0000 7f370f766740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f37081ab460 con 0x557ddb4cdc80 2024-09-19T09:26:37.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.308+0000 7f370f766740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f37081acfa0 con 0x557ddb4cd360 2024-09-19T09:26:37.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.308+0000 7f370f766740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37081abb30 con 0x557ddb4dca90 2024-09-19T09:26:37.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.309+0000 7f36fffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ddb4cdc80 0x557ddb4f41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:37.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.309+0000 7f37077fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ddb4dca90 0x557ddb4f4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:37.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.309+0000 7f37077fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ddb4dca90 0x557ddb4f4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:51646/0 (socket says 172.21.15.120:51646) 2024-09-19T09:26:37.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.309+0000 7f37077fe640 1 -- 172.21.15.120:0/330425608 learned_addr learned my addr 172.21.15.120:0/330425608 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:37.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.309+0000 7f3706ffd640 1 --2- 172.21.15.120:0/330425608 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ddb4cd360 0x557ddb4e8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:37.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.309+0000 7f37077fe640 1 -- 172.21.15.120:0/330425608 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ddb4cd360 msgr2=0x557ddb4e8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:37.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.309+0000 7f37077fe640 1 --2- 172.21.15.120:0/330425608 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ddb4cd360 0x557ddb4e8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.309+0000 7f37077fe640 1 -- 172.21.15.120:0/330425608 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ddb4cdc80 msgr2=0x557ddb4f41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:37.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.309+0000 7f37077fe640 1 --2- 172.21.15.120:0/330425608 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ddb4cdc80 0x557ddb4f41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.309+0000 7f37077fe640 1 -- 172.21.15.120:0/330425608 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ddb4dc740 con 0x557ddb4dca90 2024-09-19T09:26:37.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.309+0000 7f36fffff640 1 --2- 172.21.15.120:0/330425608 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ddb4cdc80 0x557ddb4f41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:37.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.309+0000 7f37077fe640 1 --2- 172.21.15.120:0/330425608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ddb4dca90 0x557ddb4f4700 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f370002ade0 tx=0x7f370003a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2abc2afca49bf76 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:37.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.310+0000 7f36fcff9640 1 -- 172.21.15.120:0/330425608 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370004e030 con 0x557ddb4dca90 2024-09-19T09:26:37.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.310+0000 7f36fcff9640 1 -- 172.21.15.120:0/330425608 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3700036040 con 0x557ddb4dca90 2024-09-19T09:26:37.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.310+0000 7f36fcff9640 1 -- 172.21.15.120:0/330425608 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3700042040 con 0x557ddb4dca90 2024-09-19T09:26:37.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.310+0000 7f370f766740 1 -- 172.21.15.120:0/330425608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ddb4dca90 msgr2=0x557ddb4f4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:37.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.310+0000 7f370f766740 1 --2- 172.21.15.120:0/330425608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ddb4dca90 0x557ddb4f4700 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f370002ade0 tx=0x7f370003a000 comp rx=0 tx=0).stop 2024-09-19T09:26:37.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.310+0000 7f370f766740 1 -- 172.21.15.120:0/330425608 shutdown_connections 2024-09-19T09:26:37.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.310+0000 7f370f766740 1 --2- 172.21.15.120:0/330425608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ddb4dca90 0x557ddb4f4700 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.310+0000 7f370f766740 1 --2- 172.21.15.120:0/330425608 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ddb4cdc80 0x557ddb4f41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.310+0000 7f370f766740 1 --2- 172.21.15.120:0/330425608 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ddb4cd360 0x557ddb4e8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.310+0000 7f370f766740 1 -- 172.21.15.120:0/330425608 >> 172.21.15.120:0/330425608 conn(0x557ddb4c9a30 msgr2=0x557ddb4d7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:37.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.310+0000 7f370f766740 1 -- 172.21.15.120:0/330425608 shutdown_connections 2024-09-19T09:26:37.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.310+0000 7f370f766740 1 -- 172.21.15.120:0/330425608 wait complete. 2024-09-19T09:26:37.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.311+0000 7f370f766740 1 -- start start 2024-09-19T09:26:37.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.311+0000 7f370f766740 1 Processor -- start 2024-09-19T09:26:37.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.311+0000 7f370f766740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ddb4cd360 0x557ddb4e0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:37.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.311+0000 7f370f766740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ddb4cdc80 0x557ddb4e0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:37.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.311+0000 7f370f766740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ddb4dca90 0x557ddb4e12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:37.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.311+0000 7f370f766740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f37081ac8d0 con 0x557ddb4cd360 2024-09-19T09:26:37.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.311+0000 7f370f766740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f37081ac200 con 0x557ddb4dca90 2024-09-19T09:26:37.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.311+0000 7f370f766740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37081a9420 con 0x557ddb4cdc80 2024-09-19T09:26:37.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.311+0000 7f370f766740 10 client.0 _ll_register_callbacks cb 0x7f36f80058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:37.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.311+0000 7f3706ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ddb4cd360 0x557ddb4e0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:37.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.311+0000 7f3706ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ddb4cd360 0x557ddb4e0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41374/0 (socket says 172.21.15.120:41374) 2024-09-19T09:26:37.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.311+0000 7f3706ffd640 1 -- 172.21.15.120:0/2515692410 learned_addr learned my addr 172.21.15.120:0/2515692410 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:37.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.311+0000 7f36fffff640 1 --2- 172.21.15.120:0/2515692410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ddb4cdc80 0x557ddb4e0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:37.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.312+0000 7f3706ffd640 1 -- 172.21.15.120:0/2515692410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ddb4cdc80 msgr2=0x557ddb4e0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:37.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.312+0000 7f3706ffd640 1 --2- 172.21.15.120:0/2515692410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ddb4cdc80 0x557ddb4e0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.312+0000 7f3706ffd640 1 -- 172.21.15.120:0/2515692410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ddb4dca90 msgr2=0x557ddb4e12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:37.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.312+0000 7f3706ffd640 1 --2- 172.21.15.120:0/2515692410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ddb4dca90 0x557ddb4e12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.312+0000 7f3706ffd640 1 -- 172.21.15.120:0/2515692410 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ddb556210 con 0x557ddb4cd360 2024-09-19T09:26:37.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.312+0000 7f36fffff640 1 --2- 172.21.15.120:0/2515692410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ddb4cdc80 0x557ddb4e0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:37.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.312+0000 7f3706ffd640 1 --2- 172.21.15.120:0/2515692410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ddb4cd360 0x557ddb4e0840 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f36f80989c0 tx=0x7f36f809c5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=192f183ae1038d8a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:37.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.312+0000 7f36ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:26:37.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.312+0000 7f36ee7fc640 1 -- 172.21.15.120:0/2515692410 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f801d070 con 0x557ddb4cd360 2024-09-19T09:26:37.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.312+0000 7f36ee7fc640 1 -- 172.21.15.120:0/2515692410 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f36f8096db0 con 0x557ddb4cd360 2024-09-19T09:26:37.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.312+0000 7f370f766740 1 -- 172.21.15.120:0/2515692410 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3700036040 con 0x557ddb4cd360 2024-09-19T09:26:37.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.312+0000 7f36ee7fc640 1 -- 172.21.15.120:0/2515692410 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f36f80a4cb0 con 0x557ddb4cd360 2024-09-19T09:26:37.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.312+0000 7f370f766740 10 client.23154 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:37.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.312+0000 7f370f766740 1 -- 172.21.15.120:0/2515692410 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557ddb557500 con 0x557ddb4cd360 2024-09-19T09:26:37.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.312+0000 7f36ee7fc640 1 -- 172.21.15.120:0/2515692410 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3700036040 con 0x557ddb4cd360 2024-09-19T09:26:37.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.312+0000 7f36ee7fc640 1 -- 172.21.15.120:0/2515692410 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f36f809b5e0 con 0x557ddb4cd360 2024-09-19T09:26:37.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.312+0000 7f370f766740 10 client.23154 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:37.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.312+0000 7f370f766740 10 client.23154 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:26:37.351 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:26:37.351 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:26:37.351 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-19T09:26:37.351 INFO:teuthology.orchestra.run.smithi120.stdout: "e4d617a6-bfb5-4362-a5a0-8a58dacae253": { 2024-09-19T09:26:37.351 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-19T09:26:37.351 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "e4d617a6-bfb5-4362-a5a0-8a58dacae253", 2024-09-19T09:26:37.351 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-19T09:26:37.351 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:26:37.352 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:26:37.352 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:26:37.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.312+0000 7f370f766740 20 client.23154 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:37.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.312+0000 7f370f766740 1 --2- 172.21.15.120:0/2515692410 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557ddb557c20 0x557ddb55a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:37.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.312+0000 7f370f766740 4 client.23154 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:26:37.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.312+0000 7f370f766740 1 -- 172.21.15.120:0/2515692410 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557ddb43ad60 con 0x557ddb557c20 2024-09-19T09:26:37.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.313+0000 7f36fffff640 1 --2- 172.21.15.120:0/2515692410 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557ddb557c20 0x557ddb55a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:37.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.313+0000 7f36fffff640 1 --2- 172.21.15.120:0/2515692410 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557ddb557c20 0x557ddb55a040 crc :-1 s=READY pgs=582 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b71572c6e5f562a server_cookie=81ffa09f63181bd9 in_seq=0 out_seq=0 2024-09-19T09:26:37.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.313+0000 7f36ee7fc640 10 client.23154 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:37.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.313+0000 7f36ee7fc640 1 -- 172.21.15.120:0/2515692410 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557ddb43ad60 con 0x557ddb557c20 2024-09-19T09:26:37.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.313+0000 7f36ee7fc640 10 client.23154 handle_command_reply: tid=0 2024-09-19T09:26:37.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.313+0000 7f370f766740 1 client.23154 shutdown 2024-09-19T09:26:37.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.313+0000 7f370f766740 1 -- 172.21.15.120:0/2515692410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ddb4cd360 msgr2=0x557ddb4e0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:37.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.313+0000 7f370f766740 1 --2- 172.21.15.120:0/2515692410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ddb4cd360 0x557ddb4e0840 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f36f80989c0 tx=0x7f36f809c5b0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.314+0000 7f370f766740 1 -- 172.21.15.120:0/2515692410 shutdown_connections 2024-09-19T09:26:37.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.314+0000 7f370f766740 1 --2- 172.21.15.120:0/2515692410 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x557ddb557c20 0x557ddb55a040 crc :-1 s=READY pgs=582 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.314+0000 7f370f766740 1 --2- 172.21.15.120:0/2515692410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ddb4dca90 0x557ddb4e12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.314+0000 7f370f766740 1 --2- 172.21.15.120:0/2515692410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ddb4cdc80 0x557ddb4e0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.314+0000 7f370f766740 1 --2- 172.21.15.120:0/2515692410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ddb4cd360 0x557ddb4e0840 unknown :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.314+0000 7f370f766740 1 -- 172.21.15.120:0/2515692410 >> 172.21.15.120:0/2515692410 conn(0x557ddb4c9a30 msgr2=0x557ddb4c8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:37.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.314+0000 7f36ee7fc640 0 client.23154 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:37.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.314+0000 7f370f766740 1 -- 172.21.15.120:0/2515692410 shutdown_connections 2024-09-19T09:26:37.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.314+0000 7f370f766740 1 -- 172.21.15.120:0/2515692410 wait complete. 2024-09-19T09:26:37.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.314+0000 7f370f766740 20 client.23154 trim_cache size 0 max 16384 2024-09-19T09:26:37.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.314+0000 7f370d38d640 1 -- 172.21.15.120:0/323801895 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f36f40a32c0 msgr2=0x7f36f40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:37.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.314+0000 7f370d38d640 1 --2- 172.21.15.120:0/323801895 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f36f40a32c0 0x7f36f40a5780 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3700006010 tx=0x7f37000095c0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.314+0000 7f370d38d640 1 -- 172.21.15.120:0/323801895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3708101de0 msgr2=0x7f3708198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:37.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.314+0000 7f370d38d640 1 --2- 172.21.15.120:0/323801895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3708101de0 0x7f3708198480 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7f36f800c890 tx=0x7f36f800cd50 comp rx=0 tx=0).stop 2024-09-19T09:26:37.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.314+0000 7f370d38d640 1 -- 172.21.15.120:0/323801895 shutdown_connections 2024-09-19T09:26:37.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.314+0000 7f370d38d640 1 --2- 172.21.15.120:0/323801895 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f36f40a32c0 0x7f36f40a5780 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.314+0000 7f370d38d640 1 --2- 172.21.15.120:0/323801895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3708106f10 0x7f370819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.314+0000 7f370d38d640 1 --2- 172.21.15.120:0/323801895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3708102790 0x7f37081989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.314+0000 7f370d38d640 1 --2- 172.21.15.120:0/323801895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3708101de0 0x7f3708198480 unknown :-1 s=CLOSED pgs=2412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.314+0000 7f370d38d640 1 -- 172.21.15.120:0/323801895 >> 172.21.15.120:0/323801895 conn(0x7f37080fd8c0 msgr2=0x7f37080fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:37.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.314+0000 7f370d38d640 1 -- 172.21.15.120:0/323801895 shutdown_connections 2024-09-19T09:26:37.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:37.314+0000 7f370d38d640 1 -- 172.21.15.120:0/323801895 wait complete. 2024-09-19T09:26:37.388 INFO:tasks.workunit.client.0.smithi120.stdout: 10 251215 20.62 MB/sec execute 512 sec latency 8072.910 ms 2024-09-19T09:26:38.046 INFO:tasks.cephfs.filesystem:scrub status for tag:e4d617a6-bfb5-4362-a5a0-8a58dacae253 - {'path': '/', 'tag': 'e4d617a6-bfb5-4362-a5a0-8a58dacae253', 'options': 'recursive,force'} 2024-09-19T09:26:38.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:38 smithi120 ceph-mon[31150]: pgmap v1058: 97 pgs: 97 active+clean; 482 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.3 MiB/s wr, 624 op/s 2024-09-19T09:26:38.388 INFO:tasks.workunit.client.0.smithi120.stdout: 10 251215 20.58 MB/sec execute 513 sec latency 9072.946 ms 2024-09-19T09:26:38.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:38 smithi196 ceph-mon[35150]: pgmap v1058: 97 pgs: 97 active+clean; 482 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.3 MiB/s wr, 624 op/s 2024-09-19T09:26:38.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:38 smithi153 ceph-mon[35071]: pgmap v1058: 97 pgs: 97 active+clean; 482 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.3 MiB/s wr, 624 op/s 2024-09-19T09:26:38.572 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:26:38.823 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:39.388 INFO:tasks.workunit.client.0.smithi120.stdout: 10 251215 20.54 MB/sec execute 514 sec latency 10072.982 ms 2024-09-19T09:26:39.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.475+0000 7fc7aa528640 1 -- 172.21.15.120:0/3512512824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7a4101de0 msgr2=0x7fc7a41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:39.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.475+0000 7fc7aa528640 1 --2- 172.21.15.120:0/3512512824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7a4101de0 0x7fc7a41021c0 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7fc794009900 tx=0x7fc79402a410 comp rx=0 tx=0).stop 2024-09-19T09:26:39.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.475+0000 7fc7aa528640 1 -- 172.21.15.120:0/3512512824 shutdown_connections 2024-09-19T09:26:39.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.475+0000 7fc7aa528640 1 --2- 172.21.15.120:0/3512512824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7a4106f10 0x7fc7a410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:39.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.475+0000 7fc7aa528640 1 --2- 172.21.15.120:0/3512512824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7a4102790 0x7fc7a41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:39.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.475+0000 7fc7aa528640 1 --2- 172.21.15.120:0/3512512824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7a4101de0 0x7fc7a41021c0 unknown :-1 s=CLOSED pgs=2416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:39.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.475+0000 7fc7aa528640 1 -- 172.21.15.120:0/3512512824 >> 172.21.15.120:0/3512512824 conn(0x7fc7a40fd8c0 msgr2=0x7fc7a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:39.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.475+0000 7fc7aa528640 1 -- 172.21.15.120:0/3512512824 shutdown_connections 2024-09-19T09:26:39.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.475+0000 7fc7aa528640 1 -- 172.21.15.120:0/3512512824 wait complete. 2024-09-19T09:26:39.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.476+0000 7fc7aa528640 1 Processor -- start 2024-09-19T09:26:39.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.476+0000 7fc7aa528640 1 -- start start 2024-09-19T09:26:39.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.476+0000 7fc7aa528640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7a4101de0 0x7fc7a419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:39.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.476+0000 7fc7aa528640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7a4102790 0x7fc7a419ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:39.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.476+0000 7fc7aa528640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7a4106f10 0x7fc7a419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:39.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.476+0000 7fc7aa528640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a4111fc0 con 0x7fc7a4106f10 2024-09-19T09:26:39.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.476+0000 7fc7aa528640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a4111e40 con 0x7fc7a4102790 2024-09-19T09:26:39.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.476+0000 7fc7aa528640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a4112140 con 0x7fc7a4101de0 2024-09-19T09:26:39.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.476+0000 7fc7a8ac5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7a4106f10 0x7fc7a419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:39.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.476+0000 7fc7a8ac5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7a4106f10 0x7fc7a419efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41394/0 (socket says 172.21.15.120:41394) 2024-09-19T09:26:39.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.476+0000 7fc7a8ac5640 1 -- 172.21.15.120:0/3299936808 learned_addr learned my addr 172.21.15.120:0/3299936808 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:39.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.476+0000 7fc7a3fff640 1 --2- 172.21.15.120:0/3299936808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7a4101de0 0x7fc7a419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:39.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.476+0000 7fc7a37fe640 1 --2- 172.21.15.120:0/3299936808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7a4102790 0x7fc7a419ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:39.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.477+0000 7fc7a8ac5640 1 -- 172.21.15.120:0/3299936808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7a4101de0 msgr2=0x7fc7a419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:39.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.477+0000 7fc7a8ac5640 1 --2- 172.21.15.120:0/3299936808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7a4101de0 0x7fc7a419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:39.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.477+0000 7fc7a8ac5640 1 -- 172.21.15.120:0/3299936808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7a4102790 msgr2=0x7fc7a419ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:39.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.477+0000 7fc7a8ac5640 1 --2- 172.21.15.120:0/3299936808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7a4102790 0x7fc7a419ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:39.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.477+0000 7fc7a8ac5640 1 -- 172.21.15.120:0/3299936808 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a419f680 con 0x7fc7a4106f10 2024-09-19T09:26:39.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.477+0000 7fc7a8ac5640 1 --2- 172.21.15.120:0/3299936808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7a4106f10 0x7fc7a419efa0 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009fd0 tx=0x7fc79c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:39.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.477+0000 7fc7a17fa640 1 -- 172.21.15.120:0/3299936808 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c017070 con 0x7fc7a4106f10 2024-09-19T09:26:39.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.477+0000 7fc7aa528640 1 -- 172.21.15.120:0/3299936808 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a419f970 con 0x7fc7a4106f10 2024-09-19T09:26:39.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.477+0000 7fc7a17fa640 1 -- 172.21.15.120:0/3299936808 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc79c003e50 con 0x7fc7a4106f10 2024-09-19T09:26:39.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.477+0000 7fc7a17fa640 1 -- 172.21.15.120:0/3299936808 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c0106c0 con 0x7fc7a4106f10 2024-09-19T09:26:39.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.477+0000 7fc7aa528640 1 -- 172.21.15.120:0/3299936808 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a41a7250 con 0x7fc7a4106f10 2024-09-19T09:26:39.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.478+0000 7fc7aa528640 1 -- 172.21.15.120:0/3299936808 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7a41021c0 con 0x7fc7a4106f10 2024-09-19T09:26:39.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.479+0000 7fc7a17fa640 1 -- 172.21.15.120:0/3299936808 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fc79c003970 con 0x7fc7a4106f10 2024-09-19T09:26:39.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.480+0000 7fc7a17fa640 1 --2- 172.21.15.120:0/3299936808 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc78c0a3240 0x7fc78c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:39.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.480+0000 7fc7a3fff640 1 --2- 172.21.15.120:0/3299936808 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc78c0a3240 0x7fc78c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:39.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.480+0000 7fc7a17fa640 1 -- 172.21.15.120:0/3299936808 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc79c0cf5d0 con 0x7fc7a4106f10 2024-09-19T09:26:39.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.480+0000 7fc7a3fff640 1 --2- 172.21.15.120:0/3299936808 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc78c0a3240 0x7fc78c0a5700 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fc794000c00 tx=0x7fc794005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:39.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.484+0000 7fc7a17fa640 1 -- 172.21.15.120:0/3299936808 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc79c09bca0 con 0x7fc7a4106f10 2024-09-19T09:26:39.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.688+0000 7fc7aa528640 1 -- 172.21.15.120:0/3299936808 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7a41a74f0 con 0x7fc7a4106f10 2024-09-19T09:26:39.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.688+0000 7fc7a17fa640 1 -- 172.21.15.120:0/3299936808 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11988 (secure 0 0 0) 0x7fc79c09ffa0 con 0x7fc7a4106f10 2024-09-19T09:26:39.690 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:26:39.690 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":39,"btime":"2024-09-19T09:18:53:120858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:18:53.070639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:26:39.691 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 39 2024-09-19T09:26:39.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.690+0000 7fc7aa528640 1 -- 172.21.15.120:0/3299936808 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc78c0a3240 msgr2=0x7fc78c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:39.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.690+0000 7fc7aa528640 1 --2- 172.21.15.120:0/3299936808 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc78c0a3240 0x7fc78c0a5700 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fc794000c00 tx=0x7fc794005b50 comp rx=0 tx=0).stop 2024-09-19T09:26:39.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.690+0000 7fc7aa528640 1 -- 172.21.15.120:0/3299936808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7a4106f10 msgr2=0x7fc7a419efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:39.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.690+0000 7fc7aa528640 1 --2- 172.21.15.120:0/3299936808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7a4106f10 0x7fc7a419efa0 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009fd0 tx=0x7fc79c00b260 comp rx=0 tx=0).stop 2024-09-19T09:26:39.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.690+0000 7fc7aa528640 1 -- 172.21.15.120:0/3299936808 shutdown_connections 2024-09-19T09:26:39.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.690+0000 7fc7aa528640 1 --2- 172.21.15.120:0/3299936808 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc78c0a3240 0x7fc78c0a5700 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:39.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.690+0000 7fc7aa528640 1 --2- 172.21.15.120:0/3299936808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7a4106f10 0x7fc7a419efa0 unknown :-1 s=CLOSED pgs=2417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:39.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.690+0000 7fc7aa528640 1 --2- 172.21.15.120:0/3299936808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7a4102790 0x7fc7a419ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:39.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.690+0000 7fc7aa528640 1 --2- 172.21.15.120:0/3299936808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7a4101de0 0x7fc7a419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:39.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.690+0000 7fc7aa528640 1 -- 172.21.15.120:0/3299936808 >> 172.21.15.120:0/3299936808 conn(0x7fc7a40fd8c0 msgr2=0x7fc7a40fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:39.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.690+0000 7fc7aa528640 1 -- 172.21.15.120:0/3299936808 shutdown_connections 2024-09-19T09:26:39.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:39.691+0000 7fc7aa528640 1 -- 172.21.15.120:0/3299936808 wait complete. 2024-09-19T09:26:40.060 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:39 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3299936808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:26:40.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:39 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3299936808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:26:40.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:39 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3299936808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:26:40.388 INFO:tasks.workunit.client.0.smithi120.stdout: 10 251215 20.50 MB/sec execute 515 sec latency 11073.012 ms 2024-09-19T09:26:40.417 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db --set-id 3177af4d --release --await' 2024-09-19T09:26:40.417 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db --set-id 3177af4d --release --await 2024-09-19T09:26:40.665 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:41.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:40 smithi120 ceph-mon[31150]: pgmap v1059: 97 pgs: 97 active+clean; 480 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.6 MiB/s wr, 589 op/s 2024-09-19T09:26:41.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.169+0000 7f83eef19640 1 -- 172.21.15.120:0/1569994898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83e8101de0 msgr2=0x7f83e81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:41.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.169+0000 7f83eef19640 1 --2- 172.21.15.120:0/1569994898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83e8101de0 0x7f83e81021c0 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009900 tx=0x7f83d802a410 comp rx=0 tx=0).stop 2024-09-19T09:26:41.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.169+0000 7f83eef19640 1 -- 172.21.15.120:0/1569994898 shutdown_connections 2024-09-19T09:26:41.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.169+0000 7f83eef19640 1 --2- 172.21.15.120:0/1569994898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83e8106f10 0x7f83e810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.169+0000 7f83eef19640 1 --2- 172.21.15.120:0/1569994898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83e8102790 0x7f83e81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.169+0000 7f83eef19640 1 --2- 172.21.15.120:0/1569994898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83e8101de0 0x7f83e81021c0 unknown :-1 s=CLOSED pgs=2418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.169+0000 7f83eef19640 1 -- 172.21.15.120:0/1569994898 >> 172.21.15.120:0/1569994898 conn(0x7f83e80fd8c0 msgr2=0x7f83e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:41.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.169+0000 7f83eef19640 1 -- 172.21.15.120:0/1569994898 shutdown_connections 2024-09-19T09:26:41.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.169+0000 7f83eef19640 1 -- 172.21.15.120:0/1569994898 wait complete. 2024-09-19T09:26:41.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.170+0000 7f83eef19640 1 Processor -- start 2024-09-19T09:26:41.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.170+0000 7f83eef19640 1 -- start start 2024-09-19T09:26:41.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.170+0000 7f83eef19640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83e8101de0 0x7f83e81986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:41.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.170+0000 7f83eef19640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83e8102790 0x7f83e8198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:41.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.170+0000 7f83eef19640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83e8106f10 0x7f83e819cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:41.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.170+0000 7f83eef19640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f83e8112060 con 0x7f83e8106f10 2024-09-19T09:26:41.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.170+0000 7f83eef19640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f83e8111ee0 con 0x7f83e8101de0 2024-09-19T09:26:41.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.170+0000 7f83eef19640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f83e81121e0 con 0x7f83e8102790 2024-09-19T09:26:41.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.170+0000 7f83ed4b6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83e8106f10 0x7f83e819cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:41.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.170+0000 7f83ed4b6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83e8106f10 0x7f83e819cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41406/0 (socket says 172.21.15.120:41406) 2024-09-19T09:26:41.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.170+0000 7f83ed4b6640 1 -- 172.21.15.120:0/363727517 learned_addr learned my addr 172.21.15.120:0/363727517 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:41.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.170+0000 7f83eccb5640 1 --2- 172.21.15.120:0/363727517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83e8101de0 0x7f83e81986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:41.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.170+0000 7f83e7fff640 1 --2- 172.21.15.120:0/363727517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83e8102790 0x7f83e8198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:41.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.171+0000 7f83ed4b6640 1 -- 172.21.15.120:0/363727517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83e8102790 msgr2=0x7f83e8198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:41.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.171+0000 7f83ed4b6640 1 --2- 172.21.15.120:0/363727517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83e8102790 0x7f83e8198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.171+0000 7f83ed4b6640 1 -- 172.21.15.120:0/363727517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83e8101de0 msgr2=0x7f83e81986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:41.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.171+0000 7f83ed4b6640 1 --2- 172.21.15.120:0/363727517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83e8101de0 0x7f83e81986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.171+0000 7f83ed4b6640 1 -- 172.21.15.120:0/363727517 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e819d650 con 0x7f83e8106f10 2024-09-19T09:26:41.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.171+0000 7f83ed4b6640 1 --2- 172.21.15.120:0/363727517 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83e8106f10 0x7f83e819cf70 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7f83e000b970 tx=0x7f83e000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:41.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.171+0000 7f83e5ffb640 1 -- 172.21.15.120:0/363727517 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e0019070 con 0x7f83e8106f10 2024-09-19T09:26:41.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.171+0000 7f83e5ffb640 1 -- 172.21.15.120:0/363727517 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e0014440 con 0x7f83e8106f10 2024-09-19T09:26:41.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.171+0000 7f83eef19640 1 -- 172.21.15.120:0/363727517 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83e819d940 con 0x7f83e8106f10 2024-09-19T09:26:41.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.171+0000 7f83e5ffb640 1 -- 172.21.15.120:0/363727517 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e00075d0 con 0x7f83e8106f10 2024-09-19T09:26:41.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.171+0000 7f83eef19640 1 -- 172.21.15.120:0/363727517 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83e81a5220 con 0x7f83e8106f10 2024-09-19T09:26:41.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.173+0000 7f83e5ffb640 1 -- 172.21.15.120:0/363727517 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f83e001c070 con 0x7f83e8106f10 2024-09-19T09:26:41.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.174+0000 7f83f12f2740 1 Processor -- start 2024-09-19T09:26:41.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.174+0000 7f83f12f2740 1 -- start start 2024-09-19T09:26:41.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.174+0000 7f83e5ffb640 1 --2- 172.21.15.120:0/363727517 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83d00a32c0 0x7f83d00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:41.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.174+0000 7f83f12f2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624d732af60 0x5624d732b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:41.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.174+0000 7f83f12f2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624d732b880 0x5624d733a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:41.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.174+0000 7f83f12f2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624d733a650 0x5624d733ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:41.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.174+0000 7f83f12f2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5624d7290620 con 0x5624d732b880 2024-09-19T09:26:41.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.174+0000 7f83f12f2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5624d7237e50 con 0x5624d732af60 2024-09-19T09:26:41.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.174+0000 7f83f12f2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5624d728c5e0 con 0x5624d733a650 2024-09-19T09:26:41.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.174+0000 7f83e5ffb640 1 -- 172.21.15.120:0/363727517 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f83e00cee30 con 0x7f83e8106f10 2024-09-19T09:26:41.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.174+0000 7f83e7fff640 1 --2- 172.21.15.120:0/363727517 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83d00a32c0 0x7f83d00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:41.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.174+0000 7f83e7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624d732b880 0x5624d733a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:41.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.174+0000 7f83e7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624d732b880 0x5624d733a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41416/0 (socket says 172.21.15.120:41416) 2024-09-19T09:26:41.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.174+0000 7f83e7fff640 1 -- 172.21.15.120:0/2095243207 learned_addr learned my addr 172.21.15.120:0/2095243207 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:41.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.174+0000 7f83ed4b6640 1 --2- 172.21.15.120:0/2095243207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624d733a650 0x5624d733ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:41.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.175+0000 7f83eccb5640 1 --2- 172.21.15.120:0/2095243207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624d732af60 0x5624d732b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:41.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.175+0000 7f83ed4b6640 1 -- 172.21.15.120:0/2095243207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624d732af60 msgr2=0x5624d732b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:41.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.175+0000 7f83ed4b6640 1 --2- 172.21.15.120:0/2095243207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624d732af60 0x5624d732b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.175+0000 7f83ed4b6640 1 -- 172.21.15.120:0/2095243207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624d732b880 msgr2=0x5624d733a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:41.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.175+0000 7f83ed4b6640 1 --2- 172.21.15.120:0/2095243207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624d732b880 0x5624d733a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.175+0000 7f83ed4b6640 1 -- 172.21.15.120:0/2095243207 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624d7228d40 con 0x5624d733a650 2024-09-19T09:26:41.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.175+0000 7f83e7fff640 1 --2- 172.21.15.120:0/2095243207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624d732b880 0x5624d733a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:41.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.175+0000 7f83e7fff640 1 --2- 172.21.15.120:0/363727517 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83d00a32c0 0x7f83d00a5780 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009fd0 tx=0x7f83dc0099b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:41.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.175+0000 7f83ed4b6640 1 --2- 172.21.15.120:0/2095243207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624d733a650 0x5624d733ca40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f83e000b940 tx=0x7f83e0012780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f08574625125d4d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:41.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.175+0000 7f83ceffd640 1 -- 172.21.15.120:0/2095243207 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e009f800 con 0x5624d733a650 2024-09-19T09:26:41.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.175+0000 7f83ceffd640 1 -- 172.21.15.120:0/2095243207 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e00a40c0 con 0x5624d733a650 2024-09-19T09:26:41.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.175+0000 7f83ceffd640 1 -- 172.21.15.120:0/2095243207 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e0012340 con 0x5624d733a650 2024-09-19T09:26:41.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.176+0000 7f83f12f2740 1 -- 172.21.15.120:0/2095243207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624d733a650 msgr2=0x5624d733ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:41.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.176+0000 7f83f12f2740 1 --2- 172.21.15.120:0/2095243207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624d733a650 0x5624d733ca40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f83e000b940 tx=0x7f83e0012780 comp rx=0 tx=0).stop 2024-09-19T09:26:41.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.176+0000 7f83f12f2740 1 -- 172.21.15.120:0/2095243207 shutdown_connections 2024-09-19T09:26:41.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.176+0000 7f83f12f2740 1 --2- 172.21.15.120:0/2095243207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624d733a650 0x5624d733ca40 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.176+0000 7f83f12f2740 1 --2- 172.21.15.120:0/2095243207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624d732b880 0x5624d733a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.176+0000 7f83f12f2740 1 --2- 172.21.15.120:0/2095243207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624d732af60 0x5624d732b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.176+0000 7f83f12f2740 1 -- 172.21.15.120:0/2095243207 >> 172.21.15.120:0/2095243207 conn(0x5624d73275f0 msgr2=0x5624d73279d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:41.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.176+0000 7f83f12f2740 1 -- 172.21.15.120:0/2095243207 shutdown_connections 2024-09-19T09:26:41.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.176+0000 7f83f12f2740 1 -- 172.21.15.120:0/2095243207 wait complete. 2024-09-19T09:26:41.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.176+0000 7f83f12f2740 1 -- start start 2024-09-19T09:26:41.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.176+0000 7f83f12f2740 1 Processor -- start 2024-09-19T09:26:41.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.177+0000 7f83f12f2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624d732af60 0x5624d7337670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:41.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.177+0000 7f83f12f2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624d732b880 0x5624d7337bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:41.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.177+0000 7f83f12f2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624d733a650 0x5624d73380f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:41.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.177+0000 7f83f12f2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5624d733d280 con 0x5624d733a650 2024-09-19T09:26:41.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.177+0000 7f83f12f2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5624d7218f10 con 0x5624d732b880 2024-09-19T09:26:41.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.177+0000 7f83f12f2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5624d733d400 con 0x5624d732af60 2024-09-19T09:26:41.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.177+0000 7f83f12f2740 10 client.0 _ll_register_callbacks cb 0x5624d723cdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:41.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.177+0000 7f83ed4b6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624d733a650 0x5624d73380f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:41.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.177+0000 7f83ed4b6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624d733a650 0x5624d73380f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41428/0 (socket says 172.21.15.120:41428) 2024-09-19T09:26:41.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.177+0000 7f83ed4b6640 1 -- 172.21.15.120:0/795928645 learned_addr learned my addr 172.21.15.120:0/795928645 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:41.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.177+0000 7f83e7fff640 1 --2- 172.21.15.120:0/795928645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624d732b880 0x5624d7337bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:41.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.177+0000 7f83eccb5640 1 --2- 172.21.15.120:0/795928645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624d732af60 0x5624d7337670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:41.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.177+0000 7f83ed4b6640 1 -- 172.21.15.120:0/795928645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624d732af60 msgr2=0x5624d7337670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:41.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.177+0000 7f83ed4b6640 1 --2- 172.21.15.120:0/795928645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624d732af60 0x5624d7337670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.177+0000 7f83ed4b6640 1 -- 172.21.15.120:0/795928645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624d732b880 msgr2=0x5624d7337bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:41.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.177+0000 7f83ed4b6640 1 --2- 172.21.15.120:0/795928645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624d732b880 0x5624d7337bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.177+0000 7f83ed4b6640 1 -- 172.21.15.120:0/795928645 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624d6f9f580 con 0x5624d733a650 2024-09-19T09:26:41.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.177+0000 7f83ed4b6640 1 --2- 172.21.15.120:0/795928645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624d733a650 0x5624d73380f0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f83e009b200 tx=0x7f83e00978d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7d940cff12e56a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:41.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.177+0000 7f83c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:26:41.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.177+0000 7f83c7fff640 1 -- 172.21.15.120:0/795928645 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e0097be0 con 0x5624d733a650 2024-09-19T09:26:41.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.177+0000 7f83f12f2740 1 -- 172.21.15.120:0/795928645 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83e00a40c0 con 0x5624d733a650 2024-09-19T09:26:41.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.177+0000 7f83c7fff640 1 -- 172.21.15.120:0/795928645 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f83e00cc9b0 con 0x5624d733a650 2024-09-19T09:26:41.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.178+0000 7f83c7fff640 1 -- 172.21.15.120:0/795928645 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f83e008b1e0 con 0x5624d733a650 2024-09-19T09:26:41.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.178+0000 7f83f12f2740 10 client.23172 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:41.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.178+0000 7f83f12f2740 1 -- 172.21.15.120:0/795928645 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5624d7069960 con 0x5624d733a650 2024-09-19T09:26:41.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.178+0000 7f83c7fff640 1 -- 172.21.15.120:0/795928645 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e00a40c0 con 0x5624d733a650 2024-09-19T09:26:41.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.178+0000 7f83c7fff640 1 -- 172.21.15.120:0/795928645 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f83e00a3c30 con 0x5624d733a650 2024-09-19T09:26:41.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.178+0000 7f83f12f2740 10 client.23172 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:41.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.178+0000 7f83f12f2740 10 client.23172 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:26:41.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.178+0000 7f83f12f2740 20 client.23172 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:41.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.178+0000 7f83f12f2740 1 --2- 172.21.15.120:0/795928645 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5624d734a210 0x5624d734c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:41.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.178+0000 7f83f12f2740 4 client.23172 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:26:41.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.178+0000 7f83f12f2740 1 -- 172.21.15.120:0/795928645 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624d729db90 con 0x5624d734a210 2024-09-19T09:26:41.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.178+0000 7f83eccb5640 1 --2- 172.21.15.120:0/795928645 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5624d734a210 0x5624d734c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:41.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.179+0000 7f83eccb5640 1 --2- 172.21.15.120:0/795928645 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5624d734a210 0x5624d734c600 crc :-1 s=READY pgs=583 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f73de88e25fd71c server_cookie=58001b39fdf7eda1 in_seq=0 out_seq=0 2024-09-19T09:26:41.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.179+0000 7f83c7fff640 10 client.23172 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:41.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.179+0000 7f83c7fff640 1 -- 172.21.15.120:0/795928645 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5624d729db90 con 0x5624d734a210 2024-09-19T09:26:41.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.179+0000 7f83c7fff640 10 client.23172 handle_command_reply: tid=0 2024-09-19T09:26:41.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.179+0000 7f83f12f2740 1 client.23172 shutdown 2024-09-19T09:26:41.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.179+0000 7f83f12f2740 1 -- 172.21.15.120:0/795928645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624d733a650 msgr2=0x5624d73380f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:41.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.179+0000 7f83f12f2740 1 --2- 172.21.15.120:0/795928645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624d733a650 0x5624d73380f0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f83e009b200 tx=0x7f83e00978d0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.180+0000 7f83f12f2740 1 -- 172.21.15.120:0/795928645 shutdown_connections 2024-09-19T09:26:41.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.180+0000 7f83f12f2740 1 --2- 172.21.15.120:0/795928645 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5624d734a210 0x5624d734c600 crc :-1 s=READY pgs=583 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.180+0000 7f83f12f2740 1 --2- 172.21.15.120:0/795928645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624d733a650 0x5624d73380f0 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.180+0000 7f83f12f2740 1 --2- 172.21.15.120:0/795928645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624d732b880 0x5624d7337bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.180+0000 7f83c7fff640 0 client.23172 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:41.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.180+0000 7f83f12f2740 1 --2- 172.21.15.120:0/795928645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624d732af60 0x5624d7337670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.180+0000 7f83f12f2740 1 -- 172.21.15.120:0/795928645 >> 172.21.15.120:0/795928645 conn(0x5624d73275f0 msgr2=0x5624d732d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:41.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.180+0000 7f83f12f2740 1 -- 172.21.15.120:0/795928645 shutdown_connections 2024-09-19T09:26:41.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.180+0000 7f83f12f2740 1 -- 172.21.15.120:0/795928645 wait complete. 2024-09-19T09:26:41.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.180+0000 7f83f12f2740 20 client.23172 trim_cache size 0 max 16384 2024-09-19T09:26:41.195 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:40 smithi153 ceph-mon[35071]: pgmap v1059: 97 pgs: 97 active+clean; 480 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.6 MiB/s wr, 589 op/s 2024-09-19T09:26:41.195 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:40 smithi196 ceph-mon[35150]: pgmap v1059: 97 pgs: 97 active+clean; 480 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.6 MiB/s wr, 589 op/s 2024-09-19T09:26:41.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.197+0000 7f83f12f2740 1 Processor -- start 2024-09-19T09:26:41.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.197+0000 7f83f12f2740 1 -- start start 2024-09-19T09:26:41.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.197+0000 7f83f12f2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624d732af60 0x5624d7345da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:41.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.197+0000 7f83f12f2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624d732b880 0x5624d73348d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:41.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.197+0000 7f83f12f2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624d733a650 0x5624d7334e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:41.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.197+0000 7f83f12f2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f83e81ab4a0 con 0x5624d732b880 2024-09-19T09:26:41.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.197+0000 7f83f12f2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f83e81acfe0 con 0x5624d732af60 2024-09-19T09:26:41.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.197+0000 7f83f12f2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f83e81abb70 con 0x5624d733a650 2024-09-19T09:26:41.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.197+0000 7f83e7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624d732b880 0x5624d73348d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:41.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.197+0000 7f83e7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624d732b880 0x5624d73348d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41432/0 (socket says 172.21.15.120:41432) 2024-09-19T09:26:41.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.197+0000 7f83e7fff640 1 -- 172.21.15.120:0/3732907648 learned_addr learned my addr 172.21.15.120:0/3732907648 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:41.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.197+0000 7f83eccb5640 1 --2- 172.21.15.120:0/3732907648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624d732af60 0x5624d7345da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:41.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.198+0000 7f83ed4b6640 1 --2- 172.21.15.120:0/3732907648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624d733a650 0x5624d7334e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:41.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.198+0000 7f83e7fff640 1 -- 172.21.15.120:0/3732907648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624d733a650 msgr2=0x5624d7334e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:41.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.198+0000 7f83e7fff640 1 --2- 172.21.15.120:0/3732907648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624d733a650 0x5624d7334e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.198+0000 7f83e7fff640 1 -- 172.21.15.120:0/3732907648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624d732af60 msgr2=0x5624d7345da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:41.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.198+0000 7f83e7fff640 1 --2- 172.21.15.120:0/3732907648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624d732af60 0x5624d7345da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.198+0000 7f83e7fff640 1 -- 172.21.15.120:0/3732907648 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624d732b6b0 con 0x5624d732b880 2024-09-19T09:26:41.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.198+0000 7f83ed4b6640 1 --2- 172.21.15.120:0/3732907648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624d733a650 0x5624d7334e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:26:41.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.198+0000 7f83eccb5640 1 --2- 172.21.15.120:0/3732907648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624d732af60 0x5624d7345da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:41.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.198+0000 7f83e7fff640 1 --2- 172.21.15.120:0/3732907648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624d732b880 0x5624d73348d0 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f83dc015fa0 tx=0x7f83dc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a961b25bf7feb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:41.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.198+0000 7f83ce7fc640 1 -- 172.21.15.120:0/3732907648 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc0187e0 con 0x5624d732b880 2024-09-19T09:26:41.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.198+0000 7f83ce7fc640 1 -- 172.21.15.120:0/3732907648 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83dc023400 con 0x5624d732b880 2024-09-19T09:26:41.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.198+0000 7f83ce7fc640 1 -- 172.21.15.120:0/3732907648 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc018e60 con 0x5624d732b880 2024-09-19T09:26:41.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.199+0000 7f83f12f2740 1 -- 172.21.15.120:0/3732907648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624d732b880 msgr2=0x5624d73348d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:41.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.199+0000 7f83f12f2740 1 --2- 172.21.15.120:0/3732907648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624d732b880 0x5624d73348d0 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f83dc015fa0 tx=0x7f83dc01a000 comp rx=0 tx=0).stop 2024-09-19T09:26:41.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.199+0000 7f83f12f2740 1 -- 172.21.15.120:0/3732907648 shutdown_connections 2024-09-19T09:26:41.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.199+0000 7f83f12f2740 1 --2- 172.21.15.120:0/3732907648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624d733a650 0x5624d7334e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.199+0000 7f83f12f2740 1 --2- 172.21.15.120:0/3732907648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624d732b880 0x5624d73348d0 unknown :-1 s=CLOSED pgs=2421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.199+0000 7f83f12f2740 1 --2- 172.21.15.120:0/3732907648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624d732af60 0x5624d7345da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.199+0000 7f83f12f2740 1 -- 172.21.15.120:0/3732907648 >> 172.21.15.120:0/3732907648 conn(0x5624d73275f0 msgr2=0x5624d7345690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:41.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.199+0000 7f83f12f2740 1 -- 172.21.15.120:0/3732907648 shutdown_connections 2024-09-19T09:26:41.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.199+0000 7f83f12f2740 1 -- 172.21.15.120:0/3732907648 wait complete. 2024-09-19T09:26:41.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.199+0000 7f83f12f2740 1 -- start start 2024-09-19T09:26:41.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.199+0000 7f83f12f2740 1 Processor -- start 2024-09-19T09:26:41.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.200+0000 7f83f12f2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624d732af60 0x5624d734fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:41.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.200+0000 7f83f12f2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624d732b880 0x5624d734df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:41.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.200+0000 7f83f12f2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624d733a650 0x5624d734e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:41.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.200+0000 7f83f12f2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f83e81ac910 con 0x5624d732b880 2024-09-19T09:26:41.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.200+0000 7f83f12f2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f83e81ac240 con 0x5624d732af60 2024-09-19T09:26:41.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.200+0000 7f83f12f2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f83e81a9460 con 0x5624d733a650 2024-09-19T09:26:41.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.200+0000 7f83f12f2740 10 client.0 _ll_register_callbacks cb 0x7f83e009ae60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:41.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.200+0000 7f83e7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624d732b880 0x5624d734df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:41.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.200+0000 7f83e7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624d732b880 0x5624d734df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41446/0 (socket says 172.21.15.120:41446) 2024-09-19T09:26:41.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.200+0000 7f83e7fff640 1 -- 172.21.15.120:0/99860366 learned_addr learned my addr 172.21.15.120:0/99860366 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:41.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.200+0000 7f83eccb5640 1 --2- 172.21.15.120:0/99860366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624d732af60 0x5624d734fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:41.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.200+0000 7f83ed4b6640 1 --2- 172.21.15.120:0/99860366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624d733a650 0x5624d734e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:41.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.200+0000 7f83e7fff640 1 -- 172.21.15.120:0/99860366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624d733a650 msgr2=0x5624d734e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:41.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.200+0000 7f83e7fff640 1 --2- 172.21.15.120:0/99860366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624d733a650 0x5624d734e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.200+0000 7f83e7fff640 1 -- 172.21.15.120:0/99860366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624d732af60 msgr2=0x5624d734fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:41.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.200+0000 7f83e7fff640 1 --2- 172.21.15.120:0/99860366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624d732af60 0x5624d734fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.200+0000 7f83e7fff640 1 -- 172.21.15.120:0/99860366 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624d734eb00 con 0x5624d732b880 2024-09-19T09:26:41.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.200+0000 7f83eccb5640 1 --2- 172.21.15.120:0/99860366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624d732af60 0x5624d734fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:41.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.200+0000 7f83e7fff640 1 --2- 172.21.15.120:0/99860366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624d732b880 0x5624d734df80 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009ed0 tx=0x7f83dc0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145472c797259aac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:41.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.200+0000 7f83c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:26:41.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.200+0000 7f83c7fff640 1 -- 172.21.15.120:0/99860366 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc0189f0 con 0x5624d732b880 2024-09-19T09:26:41.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.201+0000 7f83c7fff640 1 -- 172.21.15.120:0/99860366 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f83dc018b90 con 0x5624d732b880 2024-09-19T09:26:41.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.201+0000 7f83f12f2740 1 -- 172.21.15.120:0/99860366 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83dc023400 con 0x5624d732b880 2024-09-19T09:26:41.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.201+0000 7f83c7fff640 1 -- 172.21.15.120:0/99860366 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f83dc018d90 con 0x5624d732b880 2024-09-19T09:26:41.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.201+0000 7f83f12f2740 10 client.23178 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:41.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.201+0000 7f83f12f2740 1 -- 172.21.15.120:0/99860366 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5624d73b5980 con 0x5624d732b880 2024-09-19T09:26:41.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.201+0000 7f83c7fff640 1 -- 172.21.15.120:0/99860366 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83dc023400 con 0x5624d732b880 2024-09-19T09:26:41.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.201+0000 7f83c7fff640 1 -- 172.21.15.120:0/99860366 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f83dc022040 con 0x5624d732b880 2024-09-19T09:26:41.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.201+0000 7f83f12f2740 10 client.23178 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:41.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.201+0000 7f83f12f2740 10 client.23178 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:26:41.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.201+0000 7f83f12f2740 20 client.23178 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:41.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.201+0000 7f83f12f2740 1 --2- 172.21.15.120:0/99860366 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5624d73b5ff0 0x5624d73b8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:41.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.201+0000 7f83f12f2740 4 client.23178 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3177af4d", "release": true, "await": true}] 2024-09-19T09:26:41.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.201+0000 7f83f12f2740 1 -- 172.21.15.120:0/99860366 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3177af4d", "release": true, "await": true}) -- 0x5624d72980a0 con 0x5624d73b5ff0 2024-09-19T09:26:41.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.201+0000 7f83eccb5640 1 --2- 172.21.15.120:0/99860366 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5624d73b5ff0 0x5624d73b8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:41.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.202+0000 7f83c7fff640 10 client.23178 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:41.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.202+0000 7f83eccb5640 1 --2- 172.21.15.120:0/99860366 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5624d73b5ff0 0x5624d73b8410 crc :-1 s=READY pgs=584 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e30afd9e410261b server_cookie=e4c5c13ed310945d in_seq=0 out_seq=0 2024-09-19T09:26:41.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.345+0000 7f83c7fff640 1 -- 172.21.15.120:0/99860366 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5624d72980a0 con 0x5624d73b5ff0 2024-09-19T09:26:41.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.345+0000 7f83c7fff640 10 client.23178 handle_command_reply: tid=0 2024-09-19T09:26:41.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.345+0000 7f83f12f2740 1 client.23178 shutdown 2024-09-19T09:26:41.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.345+0000 7f83f12f2740 1 -- 172.21.15.120:0/99860366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624d732b880 msgr2=0x5624d734df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:41.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.345+0000 7f83f12f2740 1 --2- 172.21.15.120:0/99860366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624d732b880 0x5624d734df80 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009ed0 tx=0x7f83dc0047d0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.346+0000 7f83f12f2740 1 -- 172.21.15.120:0/99860366 shutdown_connections 2024-09-19T09:26:41.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.346+0000 7f83f12f2740 1 --2- 172.21.15.120:0/99860366 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5624d73b5ff0 0x5624d73b8410 crc :-1 s=READY pgs=584 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.346+0000 7f83f12f2740 1 --2- 172.21.15.120:0/99860366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624d733a650 0x5624d734e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.346+0000 7f83f12f2740 1 --2- 172.21.15.120:0/99860366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624d732b880 0x5624d734df80 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.346+0000 7f83f12f2740 1 --2- 172.21.15.120:0/99860366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624d732af60 0x5624d734fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.346+0000 7f83f12f2740 1 -- 172.21.15.120:0/99860366 >> 172.21.15.120:0/99860366 conn(0x5624d73275f0 msgr2=0x5624d73396f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:41.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.346+0000 7f83c7fff640 0 client.23178 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:41.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.346+0000 7f83f12f2740 1 -- 172.21.15.120:0/99860366 shutdown_connections 2024-09-19T09:26:41.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.346+0000 7f83f12f2740 1 -- 172.21.15.120:0/99860366 wait complete. 2024-09-19T09:26:41.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.346+0000 7f83f12f2740 20 client.23178 trim_cache size 0 max 16384 2024-09-19T09:26:41.349 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:26:41.349 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 39, 2024-09-19T09:26:41.349 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:26:41.349 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 80, 2024-09-19T09:26:41.349 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:26:41.349 INFO:teuthology.orchestra.run.smithi120.stdout: "3177af4d": { 2024-09-19T09:26:41.349 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 80, 2024-09-19T09:26:41.349 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:26:41.349 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:26:41.349 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:26:41.349 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:26:41.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:26:41.349 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:26:41.350 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 133.1, 2024-09-19T09:26:41.350 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:26:41.350 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:26:41.350 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:26:41.350 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:26:41.350 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:26:41.350 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:26:41.350 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:26:41.350 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:26:41.350 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:26:41.350 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:26:41.350 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:26:41.350 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:26:41.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.347+0000 7f83eef19640 1 -- 172.21.15.120:0/363727517 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83d00a32c0 msgr2=0x7f83d00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:41.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.347+0000 7f83eef19640 1 --2- 172.21.15.120:0/363727517 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83d00a32c0 0x7f83d00a5780 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009fd0 tx=0x7f83dc0099b0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.347+0000 7f83eef19640 1 -- 172.21.15.120:0/363727517 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83e8106f10 msgr2=0x7f83e819cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:41.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.347+0000 7f83eef19640 1 --2- 172.21.15.120:0/363727517 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83e8106f10 0x7f83e819cf70 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7f83e000b970 tx=0x7f83e000bd70 comp rx=0 tx=0).stop 2024-09-19T09:26:41.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.347+0000 7f83eef19640 1 -- 172.21.15.120:0/363727517 shutdown_connections 2024-09-19T09:26:41.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.347+0000 7f83eef19640 1 --2- 172.21.15.120:0/363727517 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f83d00a32c0 0x7f83d00a5780 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.347+0000 7f83eef19640 1 --2- 172.21.15.120:0/363727517 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83e8106f10 0x7f83e819cf70 unknown :-1 s=CLOSED pgs=2419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.347+0000 7f83eef19640 1 --2- 172.21.15.120:0/363727517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83e8102790 0x7f83e8198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.347+0000 7f83eef19640 1 --2- 172.21.15.120:0/363727517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83e8101de0 0x7f83e81986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.347+0000 7f83eef19640 1 -- 172.21.15.120:0/363727517 >> 172.21.15.120:0/363727517 conn(0x7f83e80fd8c0 msgr2=0x7f83e80fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:41.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.347+0000 7f83eef19640 1 -- 172.21.15.120:0/363727517 shutdown_connections 2024-09-19T09:26:41.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:26:41.347+0000 7f83eef19640 1 -- 172.21.15.120:0/363727517 wait complete. 2024-09-19T09:26:41.388 INFO:tasks.workunit.client.0.smithi120.stdout: 10 251219 20.46 MB/sec execute 516 sec latency 12073.059 ms 2024-09-19T09:26:42.192 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3177af4d', total seconds elapsed: 16.7 2024-09-19T09:26:42.389 INFO:tasks.workunit.client.0.smithi120.stdout: 10 251697 20.46 MB/sec execute 517 sec latency 12112.444 ms 2024-09-19T09:26:42.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:42 smithi196 ceph-mon[35150]: pgmap v1060: 97 pgs: 97 active+clean; 484 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 497 op/s 2024-09-19T09:26:42.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:42 smithi153 ceph-mon[35071]: pgmap v1060: 97 pgs: 97 active+clean; 484 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 497 op/s 2024-09-19T09:26:42.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:42 smithi120 ceph-mon[31150]: pgmap v1060: 97 pgs: 97 active+clean; 484 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 497 op/s 2024-09-19T09:26:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.937+0000 7faccd16d640 1 -- 172.21.15.120:0/2890624640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facc8102d30 msgr2=0x7facc81031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.937+0000 7faccd16d640 1 --2- 172.21.15.120:0/2890624640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facc8102d30 0x7facc81031b0 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7facbc009900 tx=0x7facbc02a410 comp rx=0 tx=0).stop 2024-09-19T09:26:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.937+0000 7faccd16d640 1 -- 172.21.15.120:0/2890624640 shutdown_connections 2024-09-19T09:26:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.937+0000 7faccd16d640 1 --2- 172.21.15.120:0/2890624640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facc81036f0 0x7facc8109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.937+0000 7faccd16d640 1 --2- 172.21.15.120:0/2890624640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facc8102d30 0x7facc81031b0 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.937+0000 7faccd16d640 1 --2- 172.21.15.120:0/2890624640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facc8101b40 0x7facc8101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.937+0000 7faccd16d640 1 -- 172.21.15.120:0/2890624640 >> 172.21.15.120:0/2890624640 conn(0x7facc80fd2e0 msgr2=0x7facc80ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.937+0000 7faccd16d640 1 -- 172.21.15.120:0/2890624640 shutdown_connections 2024-09-19T09:26:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.937+0000 7faccd16d640 1 -- 172.21.15.120:0/2890624640 wait complete. 2024-09-19T09:26:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.938+0000 7faccd16d640 1 Processor -- start 2024-09-19T09:26:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.938+0000 7faccd16d640 1 -- start start 2024-09-19T09:26:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.938+0000 7faccd16d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facc8101b40 0x7facc8075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.938+0000 7faccd16d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facc8102d30 0x7facc80745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.938+0000 7faccd16d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facc81036f0 0x7facc8074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.938+0000 7faccd16d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7facc810d7c0 con 0x7facc81036f0 2024-09-19T09:26:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.938+0000 7faccd16d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7facc810d640 con 0x7facc8101b40 2024-09-19T09:26:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.938+0000 7faccd16d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7facc810d940 con 0x7facc8102d30 2024-09-19T09:26:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.939+0000 7facc6d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facc8101b40 0x7facc8075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.939+0000 7facc6d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facc8101b40 0x7facc8075fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:47886/0 (socket says 172.21.15.120:47886) 2024-09-19T09:26:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.939+0000 7facc6d9d640 1 -- 172.21.15.120:0/3070215809 learned_addr learned my addr 172.21.15.120:0/3070215809 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.939+0000 7facc659c640 1 --2- 172.21.15.120:0/3070215809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facc8102d30 0x7facc80745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.939+0000 7facc759e640 1 --2- 172.21.15.120:0/3070215809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facc81036f0 0x7facc8074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.939+0000 7facc6d9d640 1 -- 172.21.15.120:0/3070215809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facc8102d30 msgr2=0x7facc80745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.939+0000 7facc6d9d640 1 --2- 172.21.15.120:0/3070215809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facc8102d30 0x7facc80745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.939+0000 7facc6d9d640 1 -- 172.21.15.120:0/3070215809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facc81036f0 msgr2=0x7facc8074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.939+0000 7facc6d9d640 1 --2- 172.21.15.120:0/3070215809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facc81036f0 0x7facc8074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.939+0000 7facc6d9d640 1 -- 172.21.15.120:0/3070215809 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc8075060 con 0x7facc8101b40 2024-09-19T09:26:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.939+0000 7facc6d9d640 1 --2- 172.21.15.120:0/3070215809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facc8101b40 0x7facc8075fa0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7facb800d950 tx=0x7facb800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.939+0000 7facaffff640 1 -- 172.21.15.120:0/3070215809 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8017020 con 0x7facc8101b40 2024-09-19T09:26:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.940+0000 7facaffff640 1 -- 172.21.15.120:0/3070215809 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facb8009c30 con 0x7facc8101b40 2024-09-19T09:26:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.940+0000 7faccd16d640 1 -- 172.21.15.120:0/3070215809 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc8075350 con 0x7facc8101b40 2024-09-19T09:26:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.940+0000 7facaffff640 1 -- 172.21.15.120:0/3070215809 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb800f740 con 0x7facc8101b40 2024-09-19T09:26:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.940+0000 7faccd16d640 1 -- 172.21.15.120:0/3070215809 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc81a6870 con 0x7facc8101b40 2024-09-19T09:26:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.942+0000 7faccffec740 1 Processor -- start 2024-09-19T09:26:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.942+0000 7faccffec740 1 -- start start 2024-09-19T09:26:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.943+0000 7facaffff640 1 -- 172.21.15.120:0/3070215809 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7facb801a020 con 0x7facc8101b40 2024-09-19T09:26:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.943+0000 7faccffec740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf341b9a0 0x55bdf341bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.943+0000 7faccffec740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf341c2e0 0x55bdf3420da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.943+0000 7faccffec740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf3421370 0x55bdf3423830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.943+0000 7faccffec740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bdf337f2f0 con 0x55bdf341c2e0 2024-09-19T09:26:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.943+0000 7faccffec740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bdf3327270 con 0x55bdf341b9a0 2024-09-19T09:26:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.943+0000 7faccffec740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bdf337b2b0 con 0x55bdf3421370 2024-09-19T09:26:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.943+0000 7facc6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf341c2e0 0x55bdf3420da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.943+0000 7facc759e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf341b9a0 0x55bdf341bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.943+0000 7facc659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf3421370 0x55bdf3423830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.943+0000 7facc6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf341c2e0 0x55bdf3420da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55944/0 (socket says 172.21.15.120:55944) 2024-09-19T09:26:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.943+0000 7facc759e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf341b9a0 0x55bdf341bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:47892/0 (socket says 172.21.15.120:47892) 2024-09-19T09:26:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.943+0000 7facc6d9d640 1 -- 172.21.15.120:0/2527185727 learned_addr learned my addr 172.21.15.120:0/2527185727 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.943+0000 7facc759e640 1 -- 172.21.15.120:0/2527185727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf3421370 msgr2=0x55bdf3423830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.943+0000 7facc759e640 1 --2- 172.21.15.120:0/2527185727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf3421370 0x55bdf3423830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.943+0000 7facc759e640 1 -- 172.21.15.120:0/2527185727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf341c2e0 msgr2=0x55bdf3420da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.943+0000 7facc759e640 1 --2- 172.21.15.120:0/2527185727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf341c2e0 0x55bdf3420da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.943+0000 7facc759e640 1 -- 172.21.15.120:0/2527185727 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdf3326340 con 0x55bdf341b9a0 2024-09-19T09:26:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.943+0000 7facc6d9d640 1 --2- 172.21.15.120:0/2527185727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf341c2e0 0x55bdf3420da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.944+0000 7facc759e640 1 --2- 172.21.15.120:0/2527185727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf341b9a0 0x55bdf341bda0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7facc0009eb0 tx=0x7facc0010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2f891d8ee80827 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.944+0000 7facad7fa640 1 -- 172.21.15.120:0/2527185727 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc0016070 con 0x55bdf341b9a0 2024-09-19T09:26:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.944+0000 7facaffff640 1 --2- 172.21.15.120:0/3070215809 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7facb00a3370 0x7facb00a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.944+0000 7facad7fa640 1 -- 172.21.15.120:0/2527185727 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facc0004d20 con 0x55bdf341b9a0 2024-09-19T09:26:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.944+0000 7facad7fa640 1 -- 172.21.15.120:0/2527185727 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc0012810 con 0x55bdf341b9a0 2024-09-19T09:26:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.944+0000 7facc6d9d640 1 --2- 172.21.15.120:0/3070215809 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7facb00a3370 0x7facb00a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.944+0000 7facaffff640 1 -- 172.21.15.120:0/3070215809 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7facb80cf210 con 0x7facc8101b40 2024-09-19T09:26:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.945+0000 7facc6d9d640 1 --2- 172.21.15.120:0/3070215809 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7facb00a3370 0x7facb00a5830 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7facb8000c00 tx=0x7facb80d2c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.945+0000 7faccffec740 1 -- 172.21.15.120:0/2527185727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf341b9a0 msgr2=0x55bdf341bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.945+0000 7faccffec740 1 --2- 172.21.15.120:0/2527185727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf341b9a0 0x55bdf341bda0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7facc0009eb0 tx=0x7facc0010740 comp rx=0 tx=0).stop 2024-09-19T09:26:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.945+0000 7faccffec740 1 -- 172.21.15.120:0/2527185727 shutdown_connections 2024-09-19T09:26:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.945+0000 7faccffec740 1 --2- 172.21.15.120:0/2527185727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf3421370 0x55bdf3423830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.945+0000 7faccffec740 1 --2- 172.21.15.120:0/2527185727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf341c2e0 0x55bdf3420da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.945+0000 7faccffec740 1 --2- 172.21.15.120:0/2527185727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf341b9a0 0x55bdf341bda0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.945+0000 7faccffec740 1 -- 172.21.15.120:0/2527185727 >> 172.21.15.120:0/2527185727 conn(0x55bdf3417010 msgr2=0x55bdf3417410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.945+0000 7faccffec740 1 -- 172.21.15.120:0/2527185727 shutdown_connections 2024-09-19T09:26:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.945+0000 7faccffec740 1 -- 172.21.15.120:0/2527185727 wait complete. 2024-09-19T09:26:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.946+0000 7faccffec740 1 -- start start 2024-09-19T09:26:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.946+0000 7faccffec740 1 Processor -- start 2024-09-19T09:26:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.946+0000 7faccffec740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf341b9a0 0x55bdf3425bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.946+0000 7faccffec740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf341c2e0 0x55bdf3426100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.946+0000 7faccffec740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf3421370 0x55bdf342d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.946+0000 7faccffec740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bdf3424340 con 0x55bdf341b9a0 2024-09-19T09:26:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.946+0000 7faccffec740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bdf3308290 con 0x55bdf341c2e0 2024-09-19T09:26:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.946+0000 7faccffec740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bdf34244c0 con 0x55bdf3421370 2024-09-19T09:26:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.946+0000 7faccffec740 10 client.0 _ll_register_callbacks cb 0x55bdf3339140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.946+0000 7facc659c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf341c2e0 0x55bdf3426100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.946+0000 7facc759e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf341b9a0 0x55bdf3425bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.946+0000 7facc659c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf341c2e0 0x55bdf3426100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:47908/0 (socket says 172.21.15.120:47908) 2024-09-19T09:26:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.946+0000 7facc659c640 1 -- 172.21.15.120:0/2115944927 learned_addr learned my addr 172.21.15.120:0/2115944927 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.946+0000 7facc759e640 1 --2- 172.21.15.120:0/2115944927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf3421370 0x55bdf342d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.947+0000 7facc759e640 1 -- 172.21.15.120:0/2115944927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf3421370 msgr2=0x55bdf342d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.947+0000 7facc759e640 1 --2- 172.21.15.120:0/2115944927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf3421370 0x55bdf342d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.947+0000 7facc759e640 1 -- 172.21.15.120:0/2115944927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf341c2e0 msgr2=0x55bdf3426100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.947+0000 7facc759e640 1 --2- 172.21.15.120:0/2115944927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf341c2e0 0x55bdf3426100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.947+0000 7facc759e640 1 -- 172.21.15.120:0/2115944927 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdf3092210 con 0x55bdf341b9a0 2024-09-19T09:26:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.947+0000 7facc659c640 1 --2- 172.21.15.120:0/2115944927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf341c2e0 0x55bdf3426100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.947+0000 7facc759e640 1 --2- 172.21.15.120:0/2115944927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf341b9a0 0x55bdf3425bc0 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7facc0009c70 tx=0x7facc0005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab09558b378242b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.947+0000 7facaaff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:26:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.947+0000 7facaaff5640 1 -- 172.21.15.120:0/2115944927 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc0024070 con 0x55bdf341b9a0 2024-09-19T09:26:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.947+0000 7facaaff5640 1 -- 172.21.15.120:0/2115944927 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7facc0003b70 con 0x55bdf341b9a0 2024-09-19T09:26:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.947+0000 7faccffec740 1 -- 172.21.15.120:0/2115944927 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facc0004d20 con 0x55bdf341b9a0 2024-09-19T09:26:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.947+0000 7facaaff5640 1 -- 172.21.15.120:0/2115944927 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7facc0032050 con 0x55bdf341b9a0 2024-09-19T09:26:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.947+0000 7faccffec740 10 client.23190 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.947+0000 7faccffec740 1 -- 172.21.15.120:0/2115944927 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55bdf315e620 con 0x55bdf341b9a0 2024-09-19T09:26:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.948+0000 7facaaff5640 1 -- 172.21.15.120:0/2115944927 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facc0004d20 con 0x55bdf341b9a0 2024-09-19T09:26:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.948+0000 7facaaff5640 1 -- 172.21.15.120:0/2115944927 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7facc000f040 con 0x55bdf341b9a0 2024-09-19T09:26:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.948+0000 7faccffec740 10 client.23190 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.948+0000 7faccffec740 10 client.23190 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:26:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.948+0000 7faccffec740 20 client.23190 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.948+0000 7faccffec740 1 --2- 172.21.15.120:0/2115944927 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bdf3431560 0x55bdf3433950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.948+0000 7faccffec740 4 client.23190 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:26:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.948+0000 7faccffec740 1 -- 172.21.15.120:0/2115944927 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdf338ca20 con 0x55bdf3431560 2024-09-19T09:26:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.948+0000 7facc659c640 1 --2- 172.21.15.120:0/2115944927 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bdf3431560 0x55bdf3433950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.949+0000 7facc659c640 1 --2- 172.21.15.120:0/2115944927 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bdf3431560 0x55bdf3433950 crc :-1 s=READY pgs=585 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb256902dd9044bc server_cookie=5df31b3d0f37bf3f in_seq=0 out_seq=0 2024-09-19T09:26:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.949+0000 7facaaff5640 10 client.23190 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.950+0000 7facaaff5640 1 -- 172.21.15.120:0/2115944927 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bdf338ca20 con 0x55bdf3431560 2024-09-19T09:26:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.950+0000 7facaaff5640 10 client.23190 handle_command_reply: tid=0 2024-09-19T09:26:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.950+0000 7faccffec740 1 client.23190 shutdown 2024-09-19T09:26:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.950+0000 7faccffec740 1 -- 172.21.15.120:0/2115944927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf341b9a0 msgr2=0x55bdf3425bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.950+0000 7faccffec740 1 --2- 172.21.15.120:0/2115944927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf341b9a0 0x55bdf3425bc0 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7facc0009c70 tx=0x7facc0005bd0 comp rx=0 tx=0).stop 2024-09-19T09:26:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.950+0000 7faccffec740 1 -- 172.21.15.120:0/2115944927 shutdown_connections 2024-09-19T09:26:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.950+0000 7faccffec740 1 --2- 172.21.15.120:0/2115944927 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bdf3431560 0x55bdf3433950 crc :-1 s=READY pgs=585 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.950+0000 7faccffec740 1 --2- 172.21.15.120:0/2115944927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf3421370 0x55bdf342d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.950+0000 7faccffec740 1 --2- 172.21.15.120:0/2115944927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf341c2e0 0x55bdf3426100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.950+0000 7faccffec740 1 --2- 172.21.15.120:0/2115944927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf341b9a0 0x55bdf3425bc0 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.950+0000 7faccffec740 1 -- 172.21.15.120:0/2115944927 >> 172.21.15.120:0/2115944927 conn(0x55bdf3417010 msgr2=0x55bdf3418820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.950+0000 7facaaff5640 0 client.23190 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.950+0000 7faccffec740 1 -- 172.21.15.120:0/2115944927 shutdown_connections 2024-09-19T09:26:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.950+0000 7faccffec740 1 -- 172.21.15.120:0/2115944927 wait complete. 2024-09-19T09:26:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.951+0000 7faccffec740 20 client.23190 trim_cache size 0 max 16384 2024-09-19T09:26:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.972+0000 7faccffec740 1 Processor -- start 2024-09-19T09:26:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.972+0000 7faccffec740 1 -- start start 2024-09-19T09:26:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.972+0000 7faccffec740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf341b9a0 0x55bdf3425fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.972+0000 7faccffec740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf341c2e0 0x55bdf342d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.972+0000 7faccffec740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf3421370 0x55bdf3418400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.972+0000 7faccffec740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7facc81af5f0 con 0x55bdf341b9a0 2024-09-19T09:26:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.972+0000 7faccffec740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7facc81b1130 con 0x55bdf341c2e0 2024-09-19T09:26:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.972+0000 7faccffec740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7facc81afcc0 con 0x55bdf3421370 2024-09-19T09:26:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.972+0000 7facc759e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf341b9a0 0x55bdf3425fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.972+0000 7facc759e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf341b9a0 0x55bdf3425fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55956/0 (socket says 172.21.15.120:55956) 2024-09-19T09:26:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.972+0000 7facc759e640 1 -- 172.21.15.120:0/2246192947 learned_addr learned my addr 172.21.15.120:0/2246192947 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.972+0000 7facc659c640 1 --2- 172.21.15.120:0/2246192947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf341c2e0 0x55bdf342d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.972+0000 7facc759e640 1 --2- 172.21.15.120:0/2246192947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf3421370 0x55bdf3418400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.972+0000 7facc759e640 1 -- 172.21.15.120:0/2246192947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf3421370 msgr2=0x55bdf3418400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.972+0000 7facc759e640 1 --2- 172.21.15.120:0/2246192947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf3421370 0x55bdf3418400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.972+0000 7facc759e640 1 -- 172.21.15.120:0/2246192947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf341c2e0 msgr2=0x55bdf342d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.972+0000 7facc759e640 1 --2- 172.21.15.120:0/2246192947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf341c2e0 0x55bdf342d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.972+0000 7facc759e640 1 -- 172.21.15.120:0/2246192947 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdf341cf70 con 0x55bdf341b9a0 2024-09-19T09:26:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.972+0000 7facc759e640 1 --2- 172.21.15.120:0/2246192947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf341b9a0 0x55bdf3425fe0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7facc0002230 tx=0x7facc00051d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c94ced1c8db58cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.973+0000 7facacff9640 1 -- 172.21.15.120:0/2246192947 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc0024040 con 0x55bdf341b9a0 2024-09-19T09:26:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.973+0000 7facacff9640 1 -- 172.21.15.120:0/2246192947 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facc0022030 con 0x55bdf341b9a0 2024-09-19T09:26:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.973+0000 7facacff9640 1 -- 172.21.15.120:0/2246192947 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc0016070 con 0x55bdf341b9a0 2024-09-19T09:26:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.973+0000 7faccffec740 1 -- 172.21.15.120:0/2246192947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf341b9a0 msgr2=0x55bdf3425fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.973+0000 7faccffec740 1 --2- 172.21.15.120:0/2246192947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf341b9a0 0x55bdf3425fe0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7facc0002230 tx=0x7facc00051d0 comp rx=0 tx=0).stop 2024-09-19T09:26:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.973+0000 7faccffec740 1 -- 172.21.15.120:0/2246192947 shutdown_connections 2024-09-19T09:26:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.973+0000 7faccffec740 1 --2- 172.21.15.120:0/2246192947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf3421370 0x55bdf3418400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.973+0000 7faccffec740 1 --2- 172.21.15.120:0/2246192947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf341c2e0 0x55bdf342d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.973+0000 7faccffec740 1 --2- 172.21.15.120:0/2246192947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf341b9a0 0x55bdf3425fe0 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.973+0000 7faccffec740 1 -- 172.21.15.120:0/2246192947 >> 172.21.15.120:0/2246192947 conn(0x55bdf3417010 msgr2=0x55bdf34161c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.973+0000 7faccffec740 1 -- 172.21.15.120:0/2246192947 shutdown_connections 2024-09-19T09:26:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.973+0000 7faccffec740 1 -- 172.21.15.120:0/2246192947 wait complete. 2024-09-19T09:26:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.974+0000 7faccffec740 1 -- start start 2024-09-19T09:26:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.974+0000 7faccffec740 1 Processor -- start 2024-09-19T09:26:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.974+0000 7faccffec740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf341b9a0 0x55bdf342a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.974+0000 7faccffec740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf341c2e0 0x55bdf342a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.974+0000 7faccffec740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3421370 0x55bdf342b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.974+0000 7faccffec740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7facc81b0a60 con 0x55bdf3421370 2024-09-19T09:26:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.974+0000 7faccffec740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7facc81b0390 con 0x55bdf341b9a0 2024-09-19T09:26:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.974+0000 7faccffec740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7facc81ad5b0 con 0x55bdf341c2e0 2024-09-19T09:26:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.974+0000 7faccffec740 10 client.0 _ll_register_callbacks cb 0x7facb80d2b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.974+0000 7facc759e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf341b9a0 0x55bdf342a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.974+0000 7facc659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf341c2e0 0x55bdf342a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.974+0000 7facc759e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf341b9a0 0x55bdf342a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:47932/0 (socket says 172.21.15.120:47932) 2024-09-19T09:26:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.974+0000 7facc759e640 1 -- 172.21.15.120:0/4024584024 learned_addr learned my addr 172.21.15.120:0/4024584024 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.974+0000 7facc759e640 1 --2- 172.21.15.120:0/4024584024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3421370 0x55bdf342b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.975+0000 7facc759e640 1 -- 172.21.15.120:0/4024584024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf341c2e0 msgr2=0x55bdf342a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.975+0000 7facc759e640 1 --2- 172.21.15.120:0/4024584024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf341c2e0 0x55bdf342a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.975+0000 7facc759e640 1 -- 172.21.15.120:0/4024584024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3421370 msgr2=0x55bdf342b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.975+0000 7facc759e640 1 --2- 172.21.15.120:0/4024584024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3421370 0x55bdf342b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.975+0000 7facc759e640 1 -- 172.21.15.120:0/4024584024 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdf342bf60 con 0x55bdf341b9a0 2024-09-19T09:26:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.975+0000 7facc759e640 1 --2- 172.21.15.120:0/4024584024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf341b9a0 0x55bdf342a400 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7facc0009620 tx=0x7facc0014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e21e8a3afd93e022 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.975+0000 7facaaff5640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:26:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.975+0000 7facaaff5640 1 -- 172.21.15.120:0/4024584024 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc0016070 con 0x55bdf341b9a0 2024-09-19T09:26:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.975+0000 7facaaff5640 1 -- 172.21.15.120:0/4024584024 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7facc0012b40 con 0x55bdf341b9a0 2024-09-19T09:26:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.975+0000 7faccffec740 1 -- 172.21.15.120:0/4024584024 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facc0022030 con 0x55bdf341b9a0 2024-09-19T09:26:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.975+0000 7facaaff5640 1 -- 172.21.15.120:0/4024584024 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7facc0010c40 con 0x55bdf341b9a0 2024-09-19T09:26:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.975+0000 7faccffec740 10 client.31853 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.975+0000 7faccffec740 1 -- 172.21.15.120:0/4024584024 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55bdf342d270 con 0x55bdf341b9a0 2024-09-19T09:26:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.976+0000 7facaaff5640 1 -- 172.21.15.120:0/4024584024 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facc0025040 con 0x55bdf341b9a0 2024-09-19T09:26:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.976+0000 7facaaff5640 1 -- 172.21.15.120:0/4024584024 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7facc0042040 con 0x55bdf341b9a0 2024-09-19T09:26:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.976+0000 7faccffec740 10 client.31853 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.976+0000 7faccffec740 10 client.31853 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:26:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.976+0000 7faccffec740 20 client.31853 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.976+0000 7faccffec740 1 --2- 172.21.15.120:0/4024584024 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bdf349c700 0x55bdf349eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.976+0000 7faccffec740 4 client.31853 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:26:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.976+0000 7faccffec740 1 -- 172.21.15.120:0/4024584024 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7facc0022030 con 0x55bdf349c700 2024-09-19T09:26:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.976+0000 7facc659c640 1 --2- 172.21.15.120:0/4024584024 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bdf349c700 0x55bdf349eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.976+0000 7facc659c640 1 --2- 172.21.15.120:0/4024584024 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bdf349c700 0x55bdf349eb20 crc :-1 s=READY pgs=586 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75aa0bae87236ce1 server_cookie=4ac267b017425c28 in_seq=0 out_seq=0 2024-09-19T09:26:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:42.976+0000 7facaaff5640 10 client.31853 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:43.270+0000 7facaaff5640 1 -- 172.21.15.120:0/4024584024 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7facc0022030 con 0x55bdf349c700 2024-09-19T09:26:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:43.270+0000 7facaaff5640 10 client.31853 handle_command_reply: tid=0 2024-09-19T09:26:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:43.270+0000 7faccffec740 1 client.31853 shutdown 2024-09-19T09:26:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:43.271+0000 7faccffec740 1 -- 172.21.15.120:0/4024584024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf341b9a0 msgr2=0x55bdf342a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:43.271+0000 7faccffec740 1 --2- 172.21.15.120:0/4024584024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf341b9a0 0x55bdf342a400 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7facc0009620 tx=0x7facc0014040 comp rx=0 tx=0).stop 2024-09-19T09:26:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:43.271+0000 7faccffec740 1 -- 172.21.15.120:0/4024584024 shutdown_connections 2024-09-19T09:26:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:43.271+0000 7faccffec740 1 --2- 172.21.15.120:0/4024584024 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bdf349c700 0x55bdf349eb20 crc :-1 s=READY pgs=586 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:43.271+0000 7faccffec740 1 --2- 172.21.15.120:0/4024584024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3421370 0x55bdf342b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:43.271+0000 7faccffec740 1 --2- 172.21.15.120:0/4024584024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf341c2e0 0x55bdf342a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:43.271+0000 7facaaff5640 0 client.31853 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:43.271+0000 7faccffec740 1 --2- 172.21.15.120:0/4024584024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf341b9a0 0x55bdf342a400 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:43.271+0000 7faccffec740 1 -- 172.21.15.120:0/4024584024 >> 172.21.15.120:0/4024584024 conn(0x55bdf3417010 msgr2=0x55bdf34173f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:43.271+0000 7faccffec740 1 -- 172.21.15.120:0/4024584024 shutdown_connections 2024-09-19T09:26:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:43.271+0000 7faccffec740 1 -- 172.21.15.120:0/4024584024 wait complete. 2024-09-19T09:26:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:43.271+0000 7faccffec740 20 client.31853 trim_cache size 0 max 16384 2024-09-19T09:26:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:26:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:26:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:26:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:26:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:43.271+0000 7faccd16d640 1 -- 172.21.15.120:0/3070215809 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7facb00a3370 msgr2=0x7facb00a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:43.271+0000 7faccd16d640 1 --2- 172.21.15.120:0/3070215809 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7facb00a3370 0x7facb00a5830 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7facb8000c00 tx=0x7facb80d2c90 comp rx=0 tx=0).stop 2024-09-19T09:26:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:43.271+0000 7faccd16d640 1 -- 172.21.15.120:0/3070215809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facc8101b40 msgr2=0x7facc8075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:43.271+0000 7faccd16d640 1 --2- 172.21.15.120:0/3070215809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facc8101b40 0x7facc8075fa0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7facb800d950 tx=0x7facb800de10 comp rx=0 tx=0).stop 2024-09-19T09:26:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:43.271+0000 7faccd16d640 1 -- 172.21.15.120:0/3070215809 shutdown_connections 2024-09-19T09:26:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:43.271+0000 7faccd16d640 1 --2- 172.21.15.120:0/3070215809 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7facb00a3370 0x7facb00a5830 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:43.271+0000 7faccd16d640 1 --2- 172.21.15.120:0/3070215809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facc81036f0 0x7facc8074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:43.271+0000 7faccd16d640 1 --2- 172.21.15.120:0/3070215809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facc8102d30 0x7facc80745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:43.271+0000 7faccd16d640 1 --2- 172.21.15.120:0/3070215809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facc8101b40 0x7facc8075fa0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:43.271+0000 7faccd16d640 1 -- 172.21.15.120:0/3070215809 >> 172.21.15.120:0/3070215809 conn(0x7facc80fd2e0 msgr2=0x7facc8106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:43.272+0000 7faccd16d640 1 -- 172.21.15.120:0/3070215809 shutdown_connections 2024-09-19T09:26:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:43.272+0000 7faccd16d640 1 -- 172.21.15.120:0/3070215809 wait complete. 2024-09-19T09:26:43.389 INFO:tasks.workunit.client.0.smithi120.stdout: 10 252153 20.47 MB/sec execute 518 sec latency 175.256 ms 2024-09-19T09:26:43.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:43 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:26:43.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:43 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:26:43.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:43 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:26:43.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:43 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:26:43.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:43 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:26:43.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:43 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:26:44.389 INFO:tasks.workunit.client.0.smithi120.stdout: 10 252511 20.46 MB/sec execute 519 sec latency 231.842 ms 2024-09-19T09:26:44.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:44 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:26:44.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:44 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:26:44.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:44 smithi120 ceph-mon[31150]: pgmap v1061: 97 pgs: 97 active+clean; 481 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.4 MiB/s wr, 419 op/s 2024-09-19T09:26:44.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:44 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:26:44.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:44 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:26:44.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:44 smithi196 ceph-mon[35150]: pgmap v1061: 97 pgs: 97 active+clean; 481 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.4 MiB/s wr, 419 op/s 2024-09-19T09:26:44.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:44 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:26:44.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:44 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:26:44.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:44 smithi153 ceph-mon[35071]: pgmap v1061: 97 pgs: 97 active+clean; 481 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.4 MiB/s wr, 419 op/s 2024-09-19T09:26:45.389 INFO:tasks.workunit.client.0.smithi120.stdout: 10 253034 20.47 MB/sec execute 520 sec latency 109.158 ms 2024-09-19T09:26:46.389 INFO:tasks.workunit.client.0.smithi120.stdout: 10 253726 20.51 MB/sec execute 521 sec latency 51.087 ms 2024-09-19T09:26:46.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:46 smithi196 ceph-mon[35150]: pgmap v1062: 97 pgs: 97 active+clean; 478 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 308 op/s 2024-09-19T09:26:46.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:46 smithi153 ceph-mon[35071]: pgmap v1062: 97 pgs: 97 active+clean; 478 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 308 op/s 2024-09-19T09:26:46.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:46 smithi120 ceph-mon[31150]: pgmap v1062: 97 pgs: 97 active+clean; 478 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 308 op/s 2024-09-19T09:26:47.389 INFO:tasks.workunit.client.0.smithi120.stdout: 10 254410 20.53 MB/sec execute 522 sec latency 77.824 ms 2024-09-19T09:26:48.389 INFO:tasks.workunit.client.0.smithi120.stdout: 10 254986 20.56 MB/sec execute 523 sec latency 66.451 ms 2024-09-19T09:26:48.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:48 smithi196 ceph-mon[35150]: pgmap v1063: 97 pgs: 97 active+clean; 476 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.9 MiB/s wr, 452 op/s 2024-09-19T09:26:48.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:48 smithi153 ceph-mon[35071]: pgmap v1063: 97 pgs: 97 active+clean; 476 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.9 MiB/s wr, 452 op/s 2024-09-19T09:26:48.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:48 smithi120 ceph-mon[31150]: pgmap v1063: 97 pgs: 97 active+clean; 476 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.9 MiB/s wr, 452 op/s 2024-09-19T09:26:49.389 INFO:tasks.workunit.client.0.smithi120.stdout: 10 255400 20.57 MB/sec execute 524 sec latency 330.338 ms 2024-09-19T09:26:50.377 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:50 smithi120 ceph-mon[31150]: pgmap v1064: 97 pgs: 97 active+clean; 481 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.3 MiB/s wr, 508 op/s 2024-09-19T09:26:50.389 INFO:tasks.workunit.client.0.smithi120.stdout: 10 255902 20.58 MB/sec execute 525 sec latency 114.013 ms 2024-09-19T09:26:50.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:50 smithi196 ceph-mon[35150]: pgmap v1064: 97 pgs: 97 active+clean; 481 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.3 MiB/s wr, 508 op/s 2024-09-19T09:26:50.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:50 smithi153 ceph-mon[35071]: pgmap v1064: 97 pgs: 97 active+clean; 481 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.3 MiB/s wr, 508 op/s 2024-09-19T09:26:51.389 INFO:tasks.workunit.client.0.smithi120.stdout: 10 256412 20.60 MB/sec execute 526 sec latency 82.021 ms 2024-09-19T09:26:52.389 INFO:tasks.workunit.client.0.smithi120.stdout: 10 256935 20.60 MB/sec execute 527 sec latency 77.328 ms 2024-09-19T09:26:52.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:52 smithi196 ceph-mon[35150]: pgmap v1065: 97 pgs: 97 active+clean; 486 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.0 MiB/s wr, 550 op/s 2024-09-19T09:26:52.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:52 smithi153 ceph-mon[35071]: pgmap v1065: 97 pgs: 97 active+clean; 486 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.0 MiB/s wr, 550 op/s 2024-09-19T09:26:52.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:52 smithi120 ceph-mon[31150]: pgmap v1065: 97 pgs: 97 active+clean; 486 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.0 MiB/s wr, 550 op/s 2024-09-19T09:26:53.389 INFO:tasks.workunit.client.0.smithi120.stdout: 10 257499 20.62 MB/sec execute 528 sec latency 76.284 ms 2024-09-19T09:26:54.389 INFO:tasks.workunit.client.0.smithi120.stdout: 10 257982 20.64 MB/sec execute 529 sec latency 159.322 ms 2024-09-19T09:26:54.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:54 smithi196 ceph-mon[35150]: pgmap v1066: 97 pgs: 97 active+clean; 487 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.5 MiB/s wr, 616 op/s 2024-09-19T09:26:54.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:54 smithi153 ceph-mon[35071]: pgmap v1066: 97 pgs: 97 active+clean; 487 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.5 MiB/s wr, 616 op/s 2024-09-19T09:26:54.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:54 smithi120 ceph-mon[31150]: pgmap v1066: 97 pgs: 97 active+clean; 487 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.5 MiB/s wr, 616 op/s 2024-09-19T09:26:55.389 INFO:tasks.workunit.client.0.smithi120.stdout: 10 258703 20.66 MB/sec execute 530 sec latency 59.145 ms 2024-09-19T09:26:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.378+0000 7fe2bc723640 1 -- 172.21.15.120:0/454997102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2b4101380 msgr2=0x7fe2b4101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.378+0000 7fe2bc723640 1 --2- 172.21.15.120:0/454997102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2b4101380 0x7fe2b4101800 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4009900 tx=0x7fe2a402a3f0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.378+0000 7fe2bc723640 1 -- 172.21.15.120:0/454997102 shutdown_connections 2024-09-19T09:26:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.378+0000 7fe2bc723640 1 --2- 172.21.15.120:0/454997102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2b4101d40 0x7fe2b41085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.378+0000 7fe2bc723640 1 --2- 172.21.15.120:0/454997102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2b4101380 0x7fe2b4101800 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.378+0000 7fe2bc723640 1 --2- 172.21.15.120:0/454997102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2b4100390 0x7fe2b4100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.378+0000 7fe2bc723640 1 -- 172.21.15.120:0/454997102 >> 172.21.15.120:0/454997102 conn(0x7fe2b40fb500 msgr2=0x7fe2b40fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.379+0000 7fe2bc723640 1 -- 172.21.15.120:0/454997102 shutdown_connections 2024-09-19T09:26:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.379+0000 7fe2bc723640 1 -- 172.21.15.120:0/454997102 wait complete. 2024-09-19T09:26:56.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.379+0000 7fe2bc723640 1 Processor -- start 2024-09-19T09:26:56.384 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.380+0000 7fe2bc723640 1 -- start start 2024-09-19T09:26:56.384 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.380+0000 7fe2bc723640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2b4100390 0x7fe2b4197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:56.384 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.380+0000 7fe2bc723640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2b4101380 0x7fe2b4198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:56.384 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.380+0000 7fe2bc723640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2b4101d40 0x7fe2b419f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:56.384 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.380+0000 7fe2bc723640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b410d840 con 0x7fe2b4100390 2024-09-19T09:26:56.384 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.380+0000 7fe2bc723640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b410d6c0 con 0x7fe2b4101380 2024-09-19T09:26:56.384 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.380+0000 7fe2bc723640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b410d9c0 con 0x7fe2b4101d40 2024-09-19T09:26:56.384 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.380+0000 7fe2ba4bf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2b4100390 0x7fe2b4197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:56.384 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.380+0000 7fe2bacc0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2b4101d40 0x7fe2b419f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:56.384 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.380+0000 7fe2bacc0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2b4101d40 0x7fe2b419f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:34688/0 (socket says 172.21.15.120:34688) 2024-09-19T09:26:56.384 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.380+0000 7fe2bacc0640 1 -- 172.21.15.120:0/2359893570 learned_addr learned my addr 172.21.15.120:0/2359893570 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.380+0000 7fe2b9cbe640 1 --2- 172.21.15.120:0/2359893570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2b4101380 0x7fe2b4198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.381+0000 7fe2bacc0640 1 -- 172.21.15.120:0/2359893570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2b4101380 msgr2=0x7fe2b4198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.381+0000 7fe2bacc0640 1 --2- 172.21.15.120:0/2359893570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2b4101380 0x7fe2b4198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.381+0000 7fe2bacc0640 1 -- 172.21.15.120:0/2359893570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2b4100390 msgr2=0x7fe2b4197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.381+0000 7fe2bacc0640 1 --2- 172.21.15.120:0/2359893570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2b4100390 0x7fe2b4197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.381+0000 7fe2bacc0640 1 -- 172.21.15.120:0/2359893570 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b419f9e0 con 0x7fe2b4101d40 2024-09-19T09:26:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.381+0000 7fe2b9cbe640 1 --2- 172.21.15.120:0/2359893570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2b4101380 0x7fe2b4198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.381+0000 7fe2ba4bf640 1 --2- 172.21.15.120:0/2359893570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2b4100390 0x7fe2b4197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.381+0000 7fe2bacc0640 1 --2- 172.21.15.120:0/2359893570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2b4101d40 0x7fe2b419f370 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac00efd0 tx=0x7fe2ac00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.381+0000 7fe2ab7fe640 1 -- 172.21.15.120:0/2359893570 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac003b70 con 0x7fe2b4101d40 2024-09-19T09:26:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.381+0000 7fe2bc723640 1 -- 172.21.15.120:0/2359893570 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b419fcd0 con 0x7fe2b4101d40 2024-09-19T09:26:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.381+0000 7fe2ab7fe640 1 -- 172.21.15.120:0/2359893570 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2ac003d10 con 0x7fe2b4101d40 2024-09-19T09:26:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.381+0000 7fe2ab7fe640 1 -- 172.21.15.120:0/2359893570 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac0109f0 con 0x7fe2b4101d40 2024-09-19T09:26:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.381+0000 7fe2bc723640 1 -- 172.21.15.120:0/2359893570 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b41a01e0 con 0x7fe2b4101d40 2024-09-19T09:26:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.384+0000 7fe2ab7fe640 1 -- 172.21.15.120:0/2359893570 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe2ac004000 con 0x7fe2b4101d40 2024-09-19T09:26:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.385+0000 7fe2bf4ba740 1 Processor -- start 2024-09-19T09:26:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.385+0000 7fe2bf4ba740 1 -- start start 2024-09-19T09:26:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.385+0000 7fe2bf4ba740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0a37db0 0x5634a0a381b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.385+0000 7fe2bf4ba740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634a0a386f0 0x5634a0a3d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.385+0000 7fe2bf4ba740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634a0a3d780 0x5634a0a3fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.385+0000 7fe2bf4ba740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5634a09972f0 con 0x5634a0a386f0 2024-09-19T09:26:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.385+0000 7fe2bf4ba740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5634a093f270 con 0x5634a0a3d780 2024-09-19T09:26:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.385+0000 7fe2bf4ba740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5634a09932b0 con 0x5634a0a37db0 2024-09-19T09:26:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.385+0000 7fe2ab7fe640 1 --2- 172.21.15.120:0/2359893570 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe2a00a33c0 0x7fe2a00a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.385+0000 7fe2ba4bf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634a0a386f0 0x5634a0a3d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.386+0000 7fe2ba4bf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634a0a386f0 0x5634a0a3d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38920/0 (socket says 172.21.15.120:38920) 2024-09-19T09:26:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.386+0000 7fe2ba4bf640 1 -- 172.21.15.120:0/1947589650 learned_addr learned my addr 172.21.15.120:0/1947589650 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.386+0000 7fe2b9cbe640 1 --2- 172.21.15.120:0/1947589650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634a0a3d780 0x5634a0a3fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.386+0000 7fe2b9cbe640 1 --2- 172.21.15.120:0/1947589650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0a37db0 0x5634a0a381b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.386+0000 7fe2bacc0640 1 --2- 172.21.15.120:0/2359893570 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe2a00a33c0 0x7fe2a00a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.386+0000 7fe2ba4bf640 1 -- 172.21.15.120:0/1947589650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0a37db0 msgr2=0x5634a0a381b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.386+0000 7fe2ba4bf640 1 --2- 172.21.15.120:0/1947589650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0a37db0 0x5634a0a381b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.386+0000 7fe2ba4bf640 1 -- 172.21.15.120:0/1947589650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634a0a3d780 msgr2=0x5634a0a3fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.386+0000 7fe2ba4bf640 1 --2- 172.21.15.120:0/1947589650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634a0a3d780 0x5634a0a3fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.386+0000 7fe2ba4bf640 1 -- 172.21.15.120:0/1947589650 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634a093e340 con 0x5634a0a386f0 2024-09-19T09:26:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.386+0000 7fe2b9cbe640 1 --2- 172.21.15.120:0/1947589650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634a0a3d780 0x5634a0a3fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.386+0000 7fe2ab7fe640 1 -- 172.21.15.120:0/2359893570 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe2ac0cfd00 con 0x7fe2b4101d40 2024-09-19T09:26:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.386+0000 7fe2ba4bf640 1 --2- 172.21.15.120:0/1947589650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634a0a386f0 0x5634a0a3d1b0 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00114f0 tx=0x7fe2b00119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab6c5463fc876ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.386+0000 7fe2a8ff9640 1 -- 172.21.15.120:0/1947589650 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b001e070 con 0x5634a0a386f0 2024-09-19T09:26:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.386+0000 7fe2a8ff9640 1 -- 172.21.15.120:0/1947589650 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2b0011250 con 0x5634a0a386f0 2024-09-19T09:26:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.386+0000 7fe2a8ff9640 1 -- 172.21.15.120:0/1947589650 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0019a10 con 0x5634a0a386f0 2024-09-19T09:26:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.386+0000 7fe2bacc0640 1 --2- 172.21.15.120:0/2359893570 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe2a00a33c0 0x7fe2a00a5880 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac098000 tx=0x7fe2ac07c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:56.391 INFO:tasks.workunit.client.0.smithi120.stdout: 10 259291 20.67 MB/sec execute 531 sec latency 57.738 ms 2024-09-19T09:26:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.387+0000 7fe2bf4ba740 1 -- 172.21.15.120:0/1947589650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634a0a386f0 msgr2=0x5634a0a3d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.387+0000 7fe2bf4ba740 1 --2- 172.21.15.120:0/1947589650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634a0a386f0 0x5634a0a3d1b0 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00114f0 tx=0x7fe2b00119b0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.387+0000 7fe2bf4ba740 1 -- 172.21.15.120:0/1947589650 shutdown_connections 2024-09-19T09:26:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.387+0000 7fe2bf4ba740 1 --2- 172.21.15.120:0/1947589650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634a0a3d780 0x5634a0a3fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.387+0000 7fe2bf4ba740 1 --2- 172.21.15.120:0/1947589650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634a0a386f0 0x5634a0a3d1b0 unknown :-1 s=CLOSED pgs=2426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.387+0000 7fe2bf4ba740 1 --2- 172.21.15.120:0/1947589650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0a37db0 0x5634a0a381b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.387+0000 7fe2bf4ba740 1 -- 172.21.15.120:0/1947589650 >> 172.21.15.120:0/1947589650 conn(0x5634a0a2f010 msgr2=0x5634a0a2f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.387+0000 7fe2bf4ba740 1 -- 172.21.15.120:0/1947589650 shutdown_connections 2024-09-19T09:26:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.387+0000 7fe2bf4ba740 1 -- 172.21.15.120:0/1947589650 wait complete. 2024-09-19T09:26:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.388+0000 7fe2bf4ba740 1 -- start start 2024-09-19T09:26:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.388+0000 7fe2bf4ba740 1 Processor -- start 2024-09-19T09:26:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.388+0000 7fe2bf4ba740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634a0a37db0 0x5634a0a42030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.388+0000 7fe2bf4ba740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634a0a386f0 0x5634a0a42570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.388+0000 7fe2bf4ba740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0a3d780 0x5634a0a495b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.388+0000 7fe2bf4ba740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5634a0a40750 con 0x5634a0a37db0 2024-09-19T09:26:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.388+0000 7fe2bf4ba740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5634a0920290 con 0x5634a0a386f0 2024-09-19T09:26:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.388+0000 7fe2bf4ba740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5634a0a408d0 con 0x5634a0a3d780 2024-09-19T09:26:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.388+0000 7fe2bf4ba740 10 client.0 _ll_register_callbacks cb 0x5634a0951140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.388+0000 7fe2b9cbe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634a0a37db0 0x5634a0a42030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.388+0000 7fe2b9cbe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634a0a37db0 0x5634a0a42030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38936/0 (socket says 172.21.15.120:38936) 2024-09-19T09:26:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.388+0000 7fe2b9cbe640 1 -- 172.21.15.120:0/1460940537 learned_addr learned my addr 172.21.15.120:0/1460940537 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.388+0000 7fe2ba4bf640 1 --2- 172.21.15.120:0/1460940537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634a0a386f0 0x5634a0a42570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.388+0000 7fe2b9cbe640 1 --2- 172.21.15.120:0/1460940537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0a3d780 0x5634a0a495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.388+0000 7fe2b9cbe640 1 -- 172.21.15.120:0/1460940537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0a3d780 msgr2=0x5634a0a495b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.388+0000 7fe2b9cbe640 1 --2- 172.21.15.120:0/1460940537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0a3d780 0x5634a0a495b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.388+0000 7fe2b9cbe640 1 -- 172.21.15.120:0/1460940537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634a0a386f0 msgr2=0x5634a0a42570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.388+0000 7fe2b9cbe640 1 --2- 172.21.15.120:0/1460940537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634a0a386f0 0x5634a0a42570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.388+0000 7fe2b9cbe640 1 -- 172.21.15.120:0/1460940537 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634a06aa210 con 0x5634a0a37db0 2024-09-19T09:26:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.388+0000 7fe2ba4bf640 1 --2- 172.21.15.120:0/1460940537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634a0a386f0 0x5634a0a42570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.389+0000 7fe2b9cbe640 1 --2- 172.21.15.120:0/1460940537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634a0a37db0 0x5634a0a42030 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7fe2a40314e0 tx=0x7fe2a4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b77966e829175e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.389+0000 7fe29dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:26:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.389+0000 7fe29dffb640 1 -- 172.21.15.120:0/1460940537 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4035040 con 0x5634a0a37db0 2024-09-19T09:26:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.389+0000 7fe29dffb640 1 -- 172.21.15.120:0/1460940537 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe2a4004990 con 0x5634a0a37db0 2024-09-19T09:26:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.389+0000 7fe2bf4ba740 1 -- 172.21.15.120:0/1460940537 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2b0011250 con 0x5634a0a37db0 2024-09-19T09:26:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.389+0000 7fe29dffb640 1 -- 172.21.15.120:0/1460940537 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe2a4004b90 con 0x5634a0a37db0 2024-09-19T09:26:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.389+0000 7fe2bf4ba740 10 client.23208 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.389+0000 7fe2bf4ba740 1 -- 172.21.15.120:0/1460940537 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5634a0776620 con 0x5634a0a37db0 2024-09-19T09:26:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.389+0000 7fe29dffb640 1 -- 172.21.15.120:0/1460940537 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2b0011250 con 0x5634a0a37db0 2024-09-19T09:26:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.389+0000 7fe29dffb640 1 -- 172.21.15.120:0/1460940537 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fe2a404a070 con 0x5634a0a37db0 2024-09-19T09:26:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.390+0000 7fe2bf4ba740 10 client.23208 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.390+0000 7fe2bf4ba740 10 client.23208 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:26:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.390+0000 7fe2bf4ba740 20 client.23208 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.390+0000 7fe2bf4ba740 1 --2- 172.21.15.120:0/1460940537 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5634a0a4dc30 0x5634a0a50080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.390+0000 7fe2bf4ba740 4 client.23208 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:26:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.390+0000 7fe2bf4ba740 1 -- 172.21.15.120:0/1460940537 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634a09a4a20 con 0x5634a0a4dc30 2024-09-19T09:26:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.390+0000 7fe2ba4bf640 1 --2- 172.21.15.120:0/1460940537 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5634a0a4dc30 0x5634a0a50080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.390+0000 7fe2ba4bf640 1 --2- 172.21.15.120:0/1460940537 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5634a0a4dc30 0x5634a0a50080 crc :-1 s=READY pgs=587 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac4bfed4b16124d9 server_cookie=1530928b7b081636 in_seq=0 out_seq=0 2024-09-19T09:26:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.390+0000 7fe29dffb640 10 client.23208 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.394+0000 7fe29dffb640 1 -- 172.21.15.120:0/1460940537 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5634a09a4a20 con 0x5634a0a4dc30 2024-09-19T09:26:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.394+0000 7fe29dffb640 10 client.23208 handle_command_reply: tid=0 2024-09-19T09:26:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.394+0000 7fe2bf4ba740 1 client.23208 shutdown 2024-09-19T09:26:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.394+0000 7fe2bf4ba740 1 -- 172.21.15.120:0/1460940537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634a0a37db0 msgr2=0x5634a0a42030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.394+0000 7fe2bf4ba740 1 --2- 172.21.15.120:0/1460940537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634a0a37db0 0x5634a0a42030 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7fe2a40314e0 tx=0x7fe2a4009260 comp rx=0 tx=0).stop 2024-09-19T09:26:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.394+0000 7fe2bf4ba740 1 -- 172.21.15.120:0/1460940537 shutdown_connections 2024-09-19T09:26:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.394+0000 7fe2bf4ba740 1 --2- 172.21.15.120:0/1460940537 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5634a0a4dc30 0x5634a0a50080 crc :-1 s=READY pgs=587 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.394+0000 7fe2bf4ba740 1 --2- 172.21.15.120:0/1460940537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0a3d780 0x5634a0a495b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.394+0000 7fe29dffb640 0 client.23208 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.394+0000 7fe2bf4ba740 1 --2- 172.21.15.120:0/1460940537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634a0a386f0 0x5634a0a42570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.394+0000 7fe2bf4ba740 1 --2- 172.21.15.120:0/1460940537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634a0a37db0 0x5634a0a42030 unknown :-1 s=CLOSED pgs=2427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.394+0000 7fe2bf4ba740 1 -- 172.21.15.120:0/1460940537 >> 172.21.15.120:0/1460940537 conn(0x5634a0a2f010 msgr2=0x5634a0a30820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.394+0000 7fe2bf4ba740 1 -- 172.21.15.120:0/1460940537 shutdown_connections 2024-09-19T09:26:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.394+0000 7fe2bf4ba740 1 -- 172.21.15.120:0/1460940537 wait complete. 2024-09-19T09:26:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.395+0000 7fe2bf4ba740 20 client.23208 trim_cache size 0 max 16384 2024-09-19T09:26:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.409+0000 7fe2bf4ba740 1 Processor -- start 2024-09-19T09:26:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.409+0000 7fe2bf4ba740 1 -- start start 2024-09-19T09:26:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.409+0000 7fe2bf4ba740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634a0a2f010 0x5634a0a42600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.409+0000 7fe2bf4ba740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0a37db0 0x5634a0a49570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.409+0000 7fe2bf4ba740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634a0a386f0 0x5634a0a305d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.409+0000 7fe2bf4ba740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b41a6dc0 con 0x5634a0a2f010 2024-09-19T09:26:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.409+0000 7fe2bf4ba740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b41a8900 con 0x5634a0a386f0 2024-09-19T09:26:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.409+0000 7fe2bf4ba740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b41a7490 con 0x5634a0a37db0 2024-09-19T09:26:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.409+0000 7fe2b9cbe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634a0a2f010 0x5634a0a42600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.409+0000 7fe2b9cbe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634a0a2f010 0x5634a0a42600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38942/0 (socket says 172.21.15.120:38942) 2024-09-19T09:26:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.409+0000 7fe2b9cbe640 1 -- 172.21.15.120:0/1409735440 learned_addr learned my addr 172.21.15.120:0/1409735440 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.410+0000 7fe2ba4bf640 1 --2- 172.21.15.120:0/1409735440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0a37db0 0x5634a0a49570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.410+0000 7fe2b9cbe640 1 --2- 172.21.15.120:0/1409735440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634a0a386f0 0x5634a0a305d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.410+0000 7fe2b9cbe640 1 -- 172.21.15.120:0/1409735440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0a37db0 msgr2=0x5634a0a49570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.410+0000 7fe2b9cbe640 1 --2- 172.21.15.120:0/1409735440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0a37db0 0x5634a0a49570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.410+0000 7fe2b9cbe640 1 -- 172.21.15.120:0/1409735440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634a0a386f0 msgr2=0x5634a0a305d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.410+0000 7fe2b9cbe640 1 --2- 172.21.15.120:0/1409735440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634a0a386f0 0x5634a0a305d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.410+0000 7fe2b9cbe640 1 -- 172.21.15.120:0/1409735440 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634a0a38bd0 con 0x5634a0a2f010 2024-09-19T09:26:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.410+0000 7fe2b9cbe640 1 --2- 172.21.15.120:0/1409735440 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634a0a2f010 0x5634a0a42600 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4006010 tx=0x7fe2a4005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdcf76a887b45cb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.410+0000 7fe29ffff640 1 -- 172.21.15.120:0/1409735440 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4035070 con 0x5634a0a2f010 2024-09-19T09:26:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.410+0000 7fe29ffff640 1 -- 172.21.15.120:0/1409735440 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2a4003880 con 0x5634a0a2f010 2024-09-19T09:26:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.410+0000 7fe29ffff640 1 -- 172.21.15.120:0/1409735440 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4003f60 con 0x5634a0a2f010 2024-09-19T09:26:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.411+0000 7fe2bf4ba740 1 -- 172.21.15.120:0/1409735440 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634a0a2f010 msgr2=0x5634a0a42600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.411+0000 7fe2bf4ba740 1 --2- 172.21.15.120:0/1409735440 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634a0a2f010 0x5634a0a42600 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4006010 tx=0x7fe2a4005970 comp rx=0 tx=0).stop 2024-09-19T09:26:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.411+0000 7fe2bf4ba740 1 -- 172.21.15.120:0/1409735440 shutdown_connections 2024-09-19T09:26:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.411+0000 7fe2bf4ba740 1 --2- 172.21.15.120:0/1409735440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634a0a386f0 0x5634a0a305d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.411+0000 7fe2bf4ba740 1 --2- 172.21.15.120:0/1409735440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0a37db0 0x5634a0a49570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.411+0000 7fe2bf4ba740 1 --2- 172.21.15.120:0/1409735440 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634a0a2f010 0x5634a0a42600 unknown :-1 s=CLOSED pgs=2428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.411+0000 7fe2bf4ba740 1 -- 172.21.15.120:0/1409735440 >> 172.21.15.120:0/1409735440 conn(0x5634a0a3d2a0 msgr2=0x5634a0a2fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.411+0000 7fe2bf4ba740 1 -- 172.21.15.120:0/1409735440 shutdown_connections 2024-09-19T09:26:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.411+0000 7fe2bf4ba740 1 -- 172.21.15.120:0/1409735440 wait complete. 2024-09-19T09:26:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.411+0000 7fe2bf4ba740 1 -- start start 2024-09-19T09:26:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.411+0000 7fe2bf4ba740 1 Processor -- start 2024-09-19T09:26:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.411+0000 7fe2bf4ba740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0a2f010 0x5634a0a470b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.411+0000 7fe2bf4ba740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634a0a37db0 0x5634a0a475f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.411+0000 7fe2bf4ba740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634a0a386f0 0x5634a0a48600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.411+0000 7fe2bf4ba740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b41a8230 con 0x5634a0a37db0 2024-09-19T09:26:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.412+0000 7fe2bf4ba740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b41a7b60 con 0x5634a0a386f0 2024-09-19T09:26:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.412+0000 7fe2bf4ba740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b41a4d80 con 0x5634a0a2f010 2024-09-19T09:26:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.412+0000 7fe2bf4ba740 10 client.0 _ll_register_callbacks cb 0x7fe2a4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.412+0000 7fe2ba4bf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634a0a37db0 0x5634a0a475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.412+0000 7fe2b9cbe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0a2f010 0x5634a0a470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.412+0000 7fe2ba4bf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634a0a37db0 0x5634a0a475f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38958/0 (socket says 172.21.15.120:38958) 2024-09-19T09:26:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.412+0000 7fe2b9cbe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0a2f010 0x5634a0a470b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:34730/0 (socket says 172.21.15.120:34730) 2024-09-19T09:26:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.412+0000 7fe2ba4bf640 1 -- 172.21.15.120:0/1755192084 learned_addr learned my addr 172.21.15.120:0/1755192084 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:26:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.412+0000 7fe2b9cbe640 1 --2- 172.21.15.120:0/1755192084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634a0a386f0 0x5634a0a48600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.412+0000 7fe2ba4bf640 1 -- 172.21.15.120:0/1755192084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0a2f010 msgr2=0x5634a0a470b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.412+0000 7fe2ba4bf640 1 --2- 172.21.15.120:0/1755192084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0a2f010 0x5634a0a470b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.412+0000 7fe2ba4bf640 1 -- 172.21.15.120:0/1755192084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634a0a386f0 msgr2=0x5634a0a48600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.412+0000 7fe2ba4bf640 1 --2- 172.21.15.120:0/1755192084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634a0a386f0 0x5634a0a48600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.412+0000 7fe2ba4bf640 1 -- 172.21.15.120:0/1755192084 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634a0a48c10 con 0x5634a0a37db0 2024-09-19T09:26:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.412+0000 7fe2ba4bf640 1 --2- 172.21.15.120:0/1755192084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634a0a37db0 0x5634a0a475f0 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00114f0 tx=0x7fe2b0009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e5e91aa2c11f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.412+0000 7fe29dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:26:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.412+0000 7fe29dffb640 1 -- 172.21.15.120:0/1755192084 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b001e040 con 0x5634a0a37db0 2024-09-19T09:26:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.412+0000 7fe29dffb640 1 -- 172.21.15.120:0/1755192084 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe2b00035f0 con 0x5634a0a37db0 2024-09-19T09:26:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.412+0000 7fe2bf4ba740 1 -- 172.21.15.120:0/1755192084 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2a4003880 con 0x5634a0a37db0 2024-09-19T09:26:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.412+0000 7fe29dffb640 1 -- 172.21.15.120:0/1755192084 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe2b00037f0 con 0x5634a0a37db0 2024-09-19T09:26:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.412+0000 7fe2bf4ba740 10 client.23214 fetch_fsmap learned FSMap version 39 2024-09-19T09:26:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.412+0000 7fe2bf4ba740 1 -- 172.21.15.120:0/1755192084 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5634a0ab8dd0 con 0x5634a0a37db0 2024-09-19T09:26:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.412+0000 7fe29dffb640 1 -- 172.21.15.120:0/1755192084 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2a4003880 con 0x5634a0a37db0 2024-09-19T09:26:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.413+0000 7fe29dffb640 1 -- 172.21.15.120:0/1755192084 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fe2b000fab0 con 0x5634a0a37db0 2024-09-19T09:26:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.413+0000 7fe2bf4ba740 10 client.23214 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:26:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.413+0000 7fe2bf4ba740 10 client.23214 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:26:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.413+0000 7fe2bf4ba740 20 client.23214 populate_metadata read hostname 'smithi120' 2024-09-19T09:26:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.413+0000 7fe2bf4ba740 1 --2- 172.21.15.120:0/1755192084 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5634a0a3d780 0x5634a0abb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.413+0000 7fe2bf4ba740 4 client.23214 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:26:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.413+0000 7fe2bf4ba740 1 -- 172.21.15.120:0/1755192084 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5634a099ed70 con 0x5634a0a3d780 2024-09-19T09:26:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.413+0000 7fe2b9cbe640 1 --2- 172.21.15.120:0/1755192084 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5634a0a3d780 0x5634a0abb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.413+0000 7fe2b9cbe640 1 --2- 172.21.15.120:0/1755192084 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5634a0a3d780 0x5634a0abb6b0 crc :-1 s=READY pgs=588 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fccbdd6a653f55ba server_cookie=838287e74f48d036 in_seq=0 out_seq=0 2024-09-19T09:26:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:56.413+0000 7fe29dffb640 10 client.23214 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:56.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:56 smithi196 ceph-mon[35150]: pgmap v1067: 97 pgs: 97 active+clean; 488 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.7 MiB/s wr, 536 op/s 2024-09-19T09:26:56.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:56 smithi153 ceph-mon[35071]: pgmap v1067: 97 pgs: 97 active+clean; 488 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.7 MiB/s wr, 536 op/s 2024-09-19T09:26:56.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:56 smithi120 ceph-mon[31150]: pgmap v1067: 97 pgs: 97 active+clean; 488 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.7 MiB/s wr, 536 op/s 2024-09-19T09:26:57.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:57.066+0000 7fe29dffb640 1 -- 172.21.15.120:0/1755192084 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5634a099ed70 con 0x5634a0a3d780 2024-09-19T09:26:57.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:57.066+0000 7fe29dffb640 10 client.23214 handle_command_reply: tid=0 2024-09-19T09:26:57.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:57.066+0000 7fe2bf4ba740 1 client.23214 shutdown 2024-09-19T09:26:57.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:57.066+0000 7fe2bf4ba740 1 -- 172.21.15.120:0/1755192084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634a0a37db0 msgr2=0x5634a0a475f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:57.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:57.066+0000 7fe2bf4ba740 1 --2- 172.21.15.120:0/1755192084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634a0a37db0 0x5634a0a475f0 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00114f0 tx=0x7fe2b0009d50 comp rx=0 tx=0).stop 2024-09-19T09:26:57.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:57.066+0000 7fe2bf4ba740 1 -- 172.21.15.120:0/1755192084 shutdown_connections 2024-09-19T09:26:57.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:57.066+0000 7fe2bf4ba740 1 --2- 172.21.15.120:0/1755192084 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5634a0a3d780 0x5634a0abb6b0 crc :-1 s=READY pgs=588 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:57.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:57.066+0000 7fe29dffb640 0 client.23214 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:26:57.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:57.066+0000 7fe2bf4ba740 1 --2- 172.21.15.120:0/1755192084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634a0a386f0 0x5634a0a48600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:57.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:57.066+0000 7fe2bf4ba740 1 --2- 172.21.15.120:0/1755192084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634a0a37db0 0x5634a0a475f0 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:57.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:57.066+0000 7fe2bf4ba740 1 --2- 172.21.15.120:0/1755192084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5634a0a2f010 0x5634a0a470b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:57.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:57.066+0000 7fe2bf4ba740 1 -- 172.21.15.120:0/1755192084 >> 172.21.15.120:0/1755192084 conn(0x5634a0a3d2a0 msgr2=0x5634a0a2fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:57.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:57.066+0000 7fe2bf4ba740 1 -- 172.21.15.120:0/1755192084 shutdown_connections 2024-09-19T09:26:57.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:57.066+0000 7fe2bf4ba740 1 -- 172.21.15.120:0/1755192084 wait complete. 2024-09-19T09:26:57.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:57.066+0000 7fe2bf4ba740 20 client.23214 trim_cache size 0 max 16384 2024-09-19T09:26:57.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:26:57.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:26:57.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:26:57.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:26:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:57.067+0000 7fe2bc723640 1 -- 172.21.15.120:0/2359893570 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe2a00a33c0 msgr2=0x7fe2a00a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:57.067+0000 7fe2bc723640 1 --2- 172.21.15.120:0/2359893570 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe2a00a33c0 0x7fe2a00a5880 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac098000 tx=0x7fe2ac07c040 comp rx=0 tx=0).stop 2024-09-19T09:26:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:57.067+0000 7fe2bc723640 1 -- 172.21.15.120:0/2359893570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2b4101d40 msgr2=0x7fe2b419f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:57.067+0000 7fe2bc723640 1 --2- 172.21.15.120:0/2359893570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2b4101d40 0x7fe2b419f370 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac00efd0 tx=0x7fe2ac00c6a0 comp rx=0 tx=0).stop 2024-09-19T09:26:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:57.067+0000 7fe2bc723640 1 -- 172.21.15.120:0/2359893570 shutdown_connections 2024-09-19T09:26:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:57.067+0000 7fe2bc723640 1 --2- 172.21.15.120:0/2359893570 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe2a00a33c0 0x7fe2a00a5880 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:57.067+0000 7fe2bc723640 1 --2- 172.21.15.120:0/2359893570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2b4101d40 0x7fe2b419f370 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:57.067+0000 7fe2bc723640 1 --2- 172.21.15.120:0/2359893570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2b4101380 0x7fe2b4198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:57.067+0000 7fe2bc723640 1 --2- 172.21.15.120:0/2359893570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2b4100390 0x7fe2b4197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:57.067+0000 7fe2bc723640 1 -- 172.21.15.120:0/2359893570 >> 172.21.15.120:0/2359893570 conn(0x7fe2b40fb500 msgr2=0x7fe2b4104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:57.067+0000 7fe2bc723640 1 -- 172.21.15.120:0/2359893570 shutdown_connections 2024-09-19T09:26:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:26:57.067+0000 7fe2bc723640 1 -- 172.21.15.120:0/2359893570 wait complete. 2024-09-19T09:26:57.389 INFO:tasks.workunit.client.0.smithi120.stdout: 10 259613 20.68 MB/sec execute 532 sec latency 438.083 ms 2024-09-19T09:26:58.389 INFO:tasks.workunit.client.0.smithi120.stdout: 10 260255 20.72 MB/sec execute 533 sec latency 74.068 ms 2024-09-19T09:26:58.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:26:58 smithi196 ceph-mon[35150]: pgmap v1068: 97 pgs: 97 active+clean; 495 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 7.7 MiB/s wr, 655 op/s 2024-09-19T09:26:58.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:26:58 smithi153 ceph-mon[35071]: pgmap v1068: 97 pgs: 97 active+clean; 495 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 7.7 MiB/s wr, 655 op/s 2024-09-19T09:26:58.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:26:58 smithi120 ceph-mon[31150]: pgmap v1068: 97 pgs: 97 active+clean; 495 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 7.7 MiB/s wr, 655 op/s 2024-09-19T09:26:59.389 INFO:tasks.workunit.client.0.smithi120.stdout: 10 260746 20.73 MB/sec execute 534 sec latency 237.220 ms 2024-09-19T09:27:00.389 INFO:tasks.workunit.client.0.smithi120.stdout: 10 261167 20.73 MB/sec execute 535 sec latency 68.613 ms 2024-09-19T09:27:00.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:00 smithi196 ceph-mon[35150]: pgmap v1069: 97 pgs: 97 active+clean; 493 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.2 MiB/s wr, 641 op/s 2024-09-19T09:27:00.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:00 smithi153 ceph-mon[35071]: pgmap v1069: 97 pgs: 97 active+clean; 493 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.2 MiB/s wr, 641 op/s 2024-09-19T09:27:00.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:00 smithi120 ceph-mon[31150]: pgmap v1069: 97 pgs: 97 active+clean; 493 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.2 MiB/s wr, 641 op/s 2024-09-19T09:27:01.389 INFO:tasks.workunit.client.0.smithi120.stdout: 10 261690 20.73 MB/sec execute 536 sec latency 100.073 ms 2024-09-19T09:27:02.389 INFO:tasks.workunit.client.0.smithi120.stdout: 10 262201 20.74 MB/sec execute 537 sec latency 116.904 ms 2024-09-19T09:27:02.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:02 smithi196 ceph-mon[35150]: pgmap v1070: 97 pgs: 97 active+clean; 486 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.5 MiB/s wr, 706 op/s 2024-09-19T09:27:02.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:02 smithi153 ceph-mon[35071]: pgmap v1070: 97 pgs: 97 active+clean; 486 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.5 MiB/s wr, 706 op/s 2024-09-19T09:27:02.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:02 smithi120 ceph-mon[31150]: pgmap v1070: 97 pgs: 97 active+clean; 486 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.5 MiB/s wr, 706 op/s 2024-09-19T09:27:03.389 INFO:tasks.workunit.client.0.smithi120.stdout: 10 262826 20.77 MB/sec execute 538 sec latency 92.498 ms 2024-09-19T09:27:04.389 INFO:tasks.workunit.client.0.smithi120.stdout: 10 263525 20.79 MB/sec execute 539 sec latency 57.448 ms 2024-09-19T09:27:04.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:04 smithi196 ceph-mon[35150]: pgmap v1071: 97 pgs: 97 active+clean; 492 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.2 MiB/s wr, 761 op/s 2024-09-19T09:27:04.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:04 smithi153 ceph-mon[35071]: pgmap v1071: 97 pgs: 97 active+clean; 492 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.2 MiB/s wr, 761 op/s 2024-09-19T09:27:04.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:04 smithi120 ceph-mon[31150]: pgmap v1071: 97 pgs: 97 active+clean; 492 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.2 MiB/s wr, 761 op/s 2024-09-19T09:27:05.389 INFO:tasks.workunit.client.0.smithi120.stdout: 10 264143 20.82 MB/sec execute 540 sec latency 49.143 ms 2024-09-19T09:27:06.389 INFO:tasks.workunit.client.0.smithi120.stdout: 10 264798 20.86 MB/sec execute 541 sec latency 42.477 ms 2024-09-19T09:27:06.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:06 smithi196 ceph-mon[35150]: pgmap v1072: 97 pgs: 97 active+clean; 495 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.1 MiB/s wr, 686 op/s 2024-09-19T09:27:06.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:06 smithi153 ceph-mon[35071]: pgmap v1072: 97 pgs: 97 active+clean; 495 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.1 MiB/s wr, 686 op/s 2024-09-19T09:27:06.522 INFO:tasks.quiescer.fs.[cephfs]:Total duration (37.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T09:27:06.522 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.5 2024-09-19T09:27:06.523 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:27:06.554 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:06 smithi120 ceph-mon[31150]: pgmap v1072: 97 pgs: 97 active+clean; 495 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.1 MiB/s wr, 686 op/s 2024-09-19T09:27:06.814 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:07.390 INFO:tasks.workunit.client.0.smithi120.stdout: 10 265297 20.87 MB/sec execute 542 sec latency 93.744 ms 2024-09-19T09:27:07.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.532+0000 7fa97ef0f640 1 -- 172.21.15.120:0/3093621093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa978100a60 msgr2=0x7fa978100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:07.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.532+0000 7fa97ef0f640 1 --2- 172.21.15.120:0/3093621093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa978100a60 0x7fa978100ec0 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7fa964009930 tx=0x7fa96402a420 comp rx=0 tx=0).stop 2024-09-19T09:27:07.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.532+0000 7fa97ef0f640 1 -- 172.21.15.120:0/3093621093 shutdown_connections 2024-09-19T09:27:07.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.532+0000 7fa97ef0f640 1 --2- 172.21.15.120:0/3093621093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa978101400 0x7fa97810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.532+0000 7fa97ef0f640 1 --2- 172.21.15.120:0/3093621093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa978100a60 0x7fa978100ec0 unknown :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.532+0000 7fa97ef0f640 1 --2- 172.21.15.120:0/3093621093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa978106a60 0x7fa978106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.532+0000 7fa97ef0f640 1 -- 172.21.15.120:0/3093621093 >> 172.21.15.120:0/3093621093 conn(0x7fa9780fc780 msgr2=0x7fa9780feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:07.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.532+0000 7fa97ef0f640 1 -- 172.21.15.120:0/3093621093 shutdown_connections 2024-09-19T09:27:07.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.532+0000 7fa97ef0f640 1 -- 172.21.15.120:0/3093621093 wait complete. 2024-09-19T09:27:07.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.533+0000 7fa97ef0f640 1 Processor -- start 2024-09-19T09:27:07.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.533+0000 7fa97ef0f640 1 -- start start 2024-09-19T09:27:07.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.533+0000 7fa97ef0f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa978100a60 0x7fa9781a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:07.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.533+0000 7fa97ef0f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa978101400 0x7fa9781a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:07.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.533+0000 7fa97ef0f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa978106a60 0x7fa97819eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:07.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.533+0000 7fa97ef0f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa978110e20 con 0x7fa978100a60 2024-09-19T09:27:07.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.533+0000 7fa97ef0f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa978110ca0 con 0x7fa978106a60 2024-09-19T09:27:07.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.533+0000 7fa97ef0f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa978110fa0 con 0x7fa978101400 2024-09-19T09:27:07.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.533+0000 7fa97ccab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa978100a60 0x7fa9781a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:07.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.533+0000 7fa97d4ac640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa978106a60 0x7fa97819eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:07.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.533+0000 7fa97ccab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa978100a60 0x7fa9781a47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55606/0 (socket says 172.21.15.120:55606) 2024-09-19T09:27:07.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.533+0000 7fa97ccab640 1 -- 172.21.15.120:0/985265260 learned_addr learned my addr 172.21.15.120:0/985265260 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:07.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.533+0000 7fa977fff640 1 --2- 172.21.15.120:0/985265260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa978101400 0x7fa9781a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:07.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.533+0000 7fa97ccab640 1 -- 172.21.15.120:0/985265260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa978101400 msgr2=0x7fa9781a4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:07.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.533+0000 7fa97ccab640 1 --2- 172.21.15.120:0/985265260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa978101400 0x7fa9781a4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.533+0000 7fa97ccab640 1 -- 172.21.15.120:0/985265260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa978106a60 msgr2=0x7fa97819eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:07.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.533+0000 7fa97ccab640 1 --2- 172.21.15.120:0/985265260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa978106a60 0x7fa97819eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.533+0000 7fa97ccab640 1 -- 172.21.15.120:0/985265260 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97819f310 con 0x7fa978100a60 2024-09-19T09:27:07.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.533+0000 7fa97d4ac640 1 --2- 172.21.15.120:0/985265260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa978106a60 0x7fa97819eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:27:07.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.534+0000 7fa97ccab640 1 --2- 172.21.15.120:0/985265260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa978100a60 0x7fa9781a47b0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7fa96c00e9f0 tx=0x7fa96c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:07.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.534+0000 7fa975ffb640 1 -- 172.21.15.120:0/985265260 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c00cd80 con 0x7fa978100a60 2024-09-19T09:27:07.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.534+0000 7fa975ffb640 1 -- 172.21.15.120:0/985265260 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa96c010910 con 0x7fa978100a60 2024-09-19T09:27:07.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.534+0000 7fa975ffb640 1 -- 172.21.15.120:0/985265260 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c018990 con 0x7fa978100a60 2024-09-19T09:27:07.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.534+0000 7fa97ef0f640 1 -- 172.21.15.120:0/985265260 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa97819f600 con 0x7fa978100a60 2024-09-19T09:27:07.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.534+0000 7fa97ef0f640 1 -- 172.21.15.120:0/985265260 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9781ab5a0 con 0x7fa978100a60 2024-09-19T09:27:07.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.535+0000 7fa97ef0f640 1 -- 172.21.15.120:0/985265260 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9781021a0 con 0x7fa978100a60 2024-09-19T09:27:07.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.536+0000 7fa975ffb640 1 -- 172.21.15.120:0/985265260 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa96c010ac0 con 0x7fa978100a60 2024-09-19T09:27:07.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.537+0000 7fa975ffb640 1 --2- 172.21.15.120:0/985265260 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9680a32c0 0x7fa9680a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:07.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.538+0000 7fa977fff640 1 --2- 172.21.15.120:0/985265260 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9680a32c0 0x7fa9680a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:07.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.538+0000 7fa975ffb640 1 -- 172.21.15.120:0/985265260 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa96c0cfae0 con 0x7fa978100a60 2024-09-19T09:27:07.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.538+0000 7fa977fff640 1 --2- 172.21.15.120:0/985265260 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9680a32c0 0x7fa9680a5780 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa964009900 tx=0x7fa9640058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:07.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.542+0000 7fa975ffb640 1 -- 172.21.15.120:0/985265260 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa96c09c1b0 con 0x7fa978100a60 2024-09-19T09:27:07.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.773+0000 7fa97ef0f640 1 -- 172.21.15.120:0/985265260 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa978100ec0 con 0x7fa978100a60 2024-09-19T09:27:07.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.773+0000 7fa975ffb640 1 -- 172.21.15.120:0/985265260 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11988 (secure 0 0 0) 0x7fa96c0a04b0 con 0x7fa978100a60 2024-09-19T09:27:07.775 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:27:07.775 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":39,"btime":"2024-09-19T09:18:53:120858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:18:53.070639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:27:07.775 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 39 2024-09-19T09:27:07.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.776+0000 7fa97ef0f640 1 -- 172.21.15.120:0/985265260 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9680a32c0 msgr2=0x7fa9680a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:07.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.776+0000 7fa97ef0f640 1 --2- 172.21.15.120:0/985265260 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9680a32c0 0x7fa9680a5780 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa964009900 tx=0x7fa9640058d0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.776+0000 7fa97ef0f640 1 -- 172.21.15.120:0/985265260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa978100a60 msgr2=0x7fa9781a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:07.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.776+0000 7fa97ef0f640 1 --2- 172.21.15.120:0/985265260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa978100a60 0x7fa9781a47b0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7fa96c00e9f0 tx=0x7fa96c00eeb0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.776+0000 7fa97ef0f640 1 -- 172.21.15.120:0/985265260 shutdown_connections 2024-09-19T09:27:07.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.776+0000 7fa97ef0f640 1 --2- 172.21.15.120:0/985265260 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa9680a32c0 0x7fa9680a5780 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.776+0000 7fa97ef0f640 1 --2- 172.21.15.120:0/985265260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa978106a60 0x7fa97819eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.776+0000 7fa97ef0f640 1 --2- 172.21.15.120:0/985265260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa978101400 0x7fa9781a4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.776+0000 7fa97ef0f640 1 --2- 172.21.15.120:0/985265260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa978100a60 0x7fa9781a47b0 unknown :-1 s=CLOSED pgs=2431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.776+0000 7fa97ef0f640 1 -- 172.21.15.120:0/985265260 >> 172.21.15.120:0/985265260 conn(0x7fa9780fc780 msgr2=0x7fa97806a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:07.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.776+0000 7fa97ef0f640 1 -- 172.21.15.120:0/985265260 shutdown_connections 2024-09-19T09:27:07.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:07.776+0000 7fa97ef0f640 1 -- 172.21.15.120:0/985265260 wait complete. 2024-09-19T09:27:08.047 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:27:08.093 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:07 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/985265260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:08.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:07 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/985265260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:08.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:07 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/985265260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:08.389 INFO:tasks.workunit.client.0.smithi120.stdout: 10 265841 20.88 MB/sec execute 543 sec latency 61.783 ms 2024-09-19T09:27:08.867 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db / --timeout 90 --expiration 138.5 --await-for 45.0' 2024-09-19T09:27:08.867 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db / --timeout 90 --expiration 138.5 --await-for 45.0 2024-09-19T09:27:08.967 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:09.095 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:08 smithi120 ceph-mon[31150]: pgmap v1073: 97 pgs: 97 active+clean; 500 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.2 MiB/s wr, 741 op/s 2024-09-19T09:27:09.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:08 smithi196 ceph-mon[35150]: pgmap v1073: 97 pgs: 97 active+clean; 500 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.2 MiB/s wr, 741 op/s 2024-09-19T09:27:09.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:08 smithi153 ceph-mon[35071]: pgmap v1073: 97 pgs: 97 active+clean; 500 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.2 MiB/s wr, 741 op/s 2024-09-19T09:27:09.390 INFO:tasks.workunit.client.0.smithi120.stdout: 10 266343 20.88 MB/sec execute 544 sec latency 64.583 ms 2024-09-19T09:27:09.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.525+0000 7fb833cbe640 1 -- 172.21.15.120:0/3543033252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb82c102d30 msgr2=0x7fb82c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:09.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.525+0000 7fb833cbe640 1 --2- 172.21.15.120:0/3543033252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb82c102d30 0x7fb82c103110 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009cd0 tx=0x7fb81c02a8e0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.525+0000 7fb833cbe640 1 -- 172.21.15.120:0/3543033252 shutdown_connections 2024-09-19T09:27:09.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.525+0000 7fb833cbe640 1 --2- 172.21.15.120:0/3543033252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb82c1082a0 0x7fb82c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.525+0000 7fb833cbe640 1 --2- 172.21.15.120:0/3543033252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb82c1036e0 0x7fb82c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.525+0000 7fb833cbe640 1 --2- 172.21.15.120:0/3543033252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb82c102d30 0x7fb82c103110 unknown :-1 s=CLOSED pgs=2432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.525+0000 7fb833cbe640 1 -- 172.21.15.120:0/3543033252 >> 172.21.15.120:0/3543033252 conn(0x7fb82c0fc7d0 msgr2=0x7fb82c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:09.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.525+0000 7fb833cbe640 1 -- 172.21.15.120:0/3543033252 shutdown_connections 2024-09-19T09:27:09.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.525+0000 7fb833cbe640 1 -- 172.21.15.120:0/3543033252 wait complete. 2024-09-19T09:27:09.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.526+0000 7fb833cbe640 1 Processor -- start 2024-09-19T09:27:09.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.526+0000 7fb833cbe640 1 -- start start 2024-09-19T09:27:09.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.526+0000 7fb833cbe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb82c102d30 0x7fb82c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.526+0000 7fb833cbe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb82c1036e0 0x7fb82c198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.526+0000 7fb833cbe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb82c1082a0 0x7fb82c19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.526+0000 7fb833cbe640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c1120a0 con 0x7fb82c102d30 2024-09-19T09:27:09.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.526+0000 7fb833cbe640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c111f20 con 0x7fb82c1082a0 2024-09-19T09:27:09.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.526+0000 7fb833cbe640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c112220 con 0x7fb82c1036e0 2024-09-19T09:27:09.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.526+0000 7fb831a5a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb82c102d30 0x7fb82c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.526+0000 7fb831a5a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb82c102d30 0x7fb82c1985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55622/0 (socket says 172.21.15.120:55622) 2024-09-19T09:27:09.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.526+0000 7fb831a5a640 1 -- 172.21.15.120:0/3561301823 learned_addr learned my addr 172.21.15.120:0/3561301823 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:09.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.526+0000 7fb831259640 1 --2- 172.21.15.120:0/3561301823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb82c1036e0 0x7fb82c198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.526+0000 7fb83225b640 1 --2- 172.21.15.120:0/3561301823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb82c1082a0 0x7fb82c19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.526+0000 7fb831a5a640 1 -- 172.21.15.120:0/3561301823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb82c1036e0 msgr2=0x7fb82c198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:09.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.526+0000 7fb831a5a640 1 --2- 172.21.15.120:0/3561301823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb82c1036e0 0x7fb82c198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.526+0000 7fb831a5a640 1 -- 172.21.15.120:0/3561301823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb82c1082a0 msgr2=0x7fb82c19ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:09.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.526+0000 7fb831a5a640 1 --2- 172.21.15.120:0/3561301823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb82c1082a0 0x7fb82c19ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.526+0000 7fb831a5a640 1 -- 172.21.15.120:0/3561301823 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb82c19d570 con 0x7fb82c102d30 2024-09-19T09:27:09.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.527+0000 7fb831a5a640 1 --2- 172.21.15.120:0/3561301823 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb82c102d30 0x7fb82c1985c0 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7fb81c02ae00 tx=0x7fb81c031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:09.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.527+0000 7fb82affd640 1 -- 172.21.15.120:0/3561301823 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c003e00 con 0x7fb82c102d30 2024-09-19T09:27:09.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.527+0000 7fb82affd640 1 -- 172.21.15.120:0/3561301823 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb81c004520 con 0x7fb82c102d30 2024-09-19T09:27:09.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.527+0000 7fb833cbe640 1 -- 172.21.15.120:0/3561301823 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb82c19d800 con 0x7fb82c102d30 2024-09-19T09:27:09.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.527+0000 7fb82affd640 1 -- 172.21.15.120:0/3561301823 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c030de0 con 0x7fb82c102d30 2024-09-19T09:27:09.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.527+0000 7fb833cbe640 1 -- 172.21.15.120:0/3561301823 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb82c1a5140 con 0x7fb82c102d30 2024-09-19T09:27:09.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.530+0000 7fb82affd640 1 -- 172.21.15.120:0/3561301823 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fb81c0046d0 con 0x7fb82c102d30 2024-09-19T09:27:09.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.530+0000 7fb836097740 1 Processor -- start 2024-09-19T09:27:09.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.530+0000 7fb836097740 1 -- start start 2024-09-19T09:27:09.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.531+0000 7fb836097740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3556cd4a0 0x55a3556cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.531+0000 7fb836097740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3556cddc0 0x55a3556d8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.531+0000 7fb836097740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3556d8680 0x55a3556daa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.531+0000 7fb82affd640 1 --2- 172.21.15.120:0/3561301823 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb8140a32c0 0x7fb8140a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.531+0000 7fb836097740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a3556332e0 con 0x55a3556d8680 2024-09-19T09:27:09.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.531+0000 7fb836097740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a3555dace0 con 0x55a3556cddc0 2024-09-19T09:27:09.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.531+0000 7fb836097740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a35562f2a0 con 0x55a3556cd4a0 2024-09-19T09:27:09.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.531+0000 7fb831259640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3556d8680 0x55a3556daa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.531+0000 7fb83225b640 1 --2- 172.21.15.120:0/3561301823 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb8140a32c0 0x7fb8140a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.531+0000 7fb831259640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3556d8680 0x55a3556daa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55624/0 (socket says 172.21.15.120:55624) 2024-09-19T09:27:09.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.531+0000 7fb831259640 1 -- 172.21.15.120:0/74926128 learned_addr learned my addr 172.21.15.120:0/74926128 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:09.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.531+0000 7fb82affd640 1 -- 172.21.15.120:0/3561301823 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fb81c02e030 con 0x7fb82c102d30 2024-09-19T09:27:09.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.531+0000 7fb831259640 1 -- 172.21.15.120:0/74926128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3556cd4a0 msgr2=0x55a3556cd880 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:27:09.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.531+0000 7fb831259640 1 --2- 172.21.15.120:0/74926128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3556cd4a0 0x55a3556cd880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.531+0000 7fb831259640 1 -- 172.21.15.120:0/74926128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3556cddc0 msgr2=0x55a3556d8140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:27:09.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.531+0000 7fb831259640 1 --2- 172.21.15.120:0/74926128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3556cddc0 0x55a3556d8140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.531+0000 7fb831259640 1 -- 172.21.15.120:0/74926128 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3555bf440 con 0x55a3556d8680 2024-09-19T09:27:09.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.531+0000 7fb83225b640 1 --2- 172.21.15.120:0/3561301823 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb8140a32c0 0x7fb8140a5780 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb824009b30 tx=0x7fb824005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:09.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.531+0000 7fb831259640 1 --2- 172.21.15.120:0/74926128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3556d8680 0x55a3556daa70 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7fb820009c20 tx=0x7fb820011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c31919d63d904c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:09.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.532+0000 7fb813fff640 1 -- 172.21.15.120:0/74926128 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb820004230 con 0x55a3556d8680 2024-09-19T09:27:09.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.532+0000 7fb813fff640 1 -- 172.21.15.120:0/74926128 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8200043d0 con 0x55a3556d8680 2024-09-19T09:27:09.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.532+0000 7fb813fff640 1 -- 172.21.15.120:0/74926128 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb820019800 con 0x55a3556d8680 2024-09-19T09:27:09.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.532+0000 7fb836097740 1 -- 172.21.15.120:0/74926128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3556d8680 msgr2=0x55a3556daa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:09.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.532+0000 7fb836097740 1 --2- 172.21.15.120:0/74926128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3556d8680 0x55a3556daa70 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7fb820009c20 tx=0x7fb820011810 comp rx=0 tx=0).stop 2024-09-19T09:27:09.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.532+0000 7fb836097740 1 -- 172.21.15.120:0/74926128 shutdown_connections 2024-09-19T09:27:09.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.532+0000 7fb836097740 1 --2- 172.21.15.120:0/74926128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3556d8680 0x55a3556daa70 unknown :-1 s=CLOSED pgs=2434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.532+0000 7fb836097740 1 --2- 172.21.15.120:0/74926128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3556cddc0 0x55a3556d8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.532+0000 7fb836097740 1 --2- 172.21.15.120:0/74926128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3556cd4a0 0x55a3556cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.532+0000 7fb836097740 1 -- 172.21.15.120:0/74926128 >> 172.21.15.120:0/74926128 conn(0x55a3556c9a30 msgr2=0x55a3556c9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:09.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.532+0000 7fb836097740 1 -- 172.21.15.120:0/74926128 shutdown_connections 2024-09-19T09:27:09.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.532+0000 7fb836097740 1 -- 172.21.15.120:0/74926128 wait complete. 2024-09-19T09:27:09.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.533+0000 7fb836097740 1 -- start start 2024-09-19T09:27:09.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.533+0000 7fb836097740 1 Processor -- start 2024-09-19T09:27:09.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.533+0000 7fb836097740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3556cd4a0 0x55a3556d8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.533+0000 7fb836097740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3556cddc0 0x55a3556d52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.533+0000 7fb836097740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3556d8680 0x55a3556d58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.533+0000 7fb836097740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a3556db250 con 0x55a3556cd4a0 2024-09-19T09:27:09.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.533+0000 7fb836097740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a3555ee050 con 0x55a3556cddc0 2024-09-19T09:27:09.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.533+0000 7fb836097740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a3556db3d0 con 0x55a3556d8680 2024-09-19T09:27:09.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.533+0000 7fb836097740 10 client.0 _ll_register_callbacks cb 0x55a35562e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:09.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.533+0000 7fb831a5a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3556cd4a0 0x55a3556d8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.533+0000 7fb831a5a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3556cd4a0 0x55a3556d8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55630/0 (socket says 172.21.15.120:55630) 2024-09-19T09:27:09.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.533+0000 7fb831a5a640 1 -- 172.21.15.120:0/618085323 learned_addr learned my addr 172.21.15.120:0/618085323 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:09.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.533+0000 7fb83225b640 1 --2- 172.21.15.120:0/618085323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3556d8680 0x55a3556d58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.534+0000 7fb831a5a640 1 -- 172.21.15.120:0/618085323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3556d8680 msgr2=0x55a3556d58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:09.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.534+0000 7fb831a5a640 1 --2- 172.21.15.120:0/618085323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3556d8680 0x55a3556d58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.534+0000 7fb831a5a640 1 -- 172.21.15.120:0/618085323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3556cddc0 msgr2=0x55a3556d52f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:27:09.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.534+0000 7fb831a5a640 1 --2- 172.21.15.120:0/618085323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3556cddc0 0x55a3556d52f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.534+0000 7fb831a5a640 1 -- 172.21.15.120:0/618085323 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3553a43c0 con 0x55a3556cd4a0 2024-09-19T09:27:09.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.534+0000 7fb831a5a640 1 --2- 172.21.15.120:0/618085323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3556cd4a0 0x55a3556d8040 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7fb81c005e30 tx=0x7fb81c0bd0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d83802592973f62 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:09.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.534+0000 7fb8117fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:27:09.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.534+0000 7fb8117fa640 1 -- 172.21.15.120:0/618085323 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c0bbb20 con 0x55a3556cd4a0 2024-09-19T09:27:09.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.534+0000 7fb8117fa640 1 -- 172.21.15.120:0/618085323 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fb81c0305a0 con 0x55a3556cd4a0 2024-09-19T09:27:09.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.534+0000 7fb836097740 1 -- 172.21.15.120:0/618085323 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8200043d0 con 0x55a3556cd4a0 2024-09-19T09:27:09.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.534+0000 7fb8117fa640 1 -- 172.21.15.120:0/618085323 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb81c0bdc60 con 0x55a3556cd4a0 2024-09-19T09:27:09.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.534+0000 7fb836097740 10 client.23232 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:09.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.534+0000 7fb836097740 1 -- 172.21.15.120:0/618085323 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a3556e6c70 con 0x55a3556cd4a0 2024-09-19T09:27:09.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.534+0000 7fb8117fa640 1 -- 172.21.15.120:0/618085323 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8200043d0 con 0x55a3556cd4a0 2024-09-19T09:27:09.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.535+0000 7fb8117fa640 1 -- 172.21.15.120:0/618085323 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fb81c04e070 con 0x55a3556cd4a0 2024-09-19T09:27:09.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.535+0000 7fb836097740 10 client.23232 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:09.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.535+0000 7fb836097740 10 client.23232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:27:09.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.535+0000 7fb836097740 20 client.23232 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:09.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.535+0000 7fb836097740 1 --2- 172.21.15.120:0/618085323 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a3556e8060 0x55a3556ea470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.535+0000 7fb836097740 4 client.23232 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:09.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.535+0000 7fb836097740 1 -- 172.21.15.120:0/618085323 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a355640850 con 0x55a3556e8060 2024-09-19T09:27:09.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.535+0000 7fb831259640 1 --2- 172.21.15.120:0/618085323 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a3556e8060 0x55a3556ea470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.535+0000 7fb831259640 1 --2- 172.21.15.120:0/618085323 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a3556e8060 0x55a3556ea470 crc :-1 s=READY pgs=589 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6729d27038fbe52 server_cookie=eb6493450272a489 in_seq=0 out_seq=0 2024-09-19T09:27:09.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.535+0000 7fb8117fa640 10 client.23232 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:09.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.538+0000 7fb8117fa640 1 -- 172.21.15.120:0/618085323 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a355640850 con 0x55a3556e8060 2024-09-19T09:27:09.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.538+0000 7fb8117fa640 10 client.23232 handle_command_reply: tid=0 2024-09-19T09:27:09.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.538+0000 7fb836097740 1 client.23232 shutdown 2024-09-19T09:27:09.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.538+0000 7fb836097740 1 -- 172.21.15.120:0/618085323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3556cd4a0 msgr2=0x55a3556d8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:09.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.538+0000 7fb836097740 1 --2- 172.21.15.120:0/618085323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3556cd4a0 0x55a3556d8040 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7fb81c005e30 tx=0x7fb81c0bd0a0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.538+0000 7fb836097740 1 -- 172.21.15.120:0/618085323 shutdown_connections 2024-09-19T09:27:09.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.538+0000 7fb836097740 1 --2- 172.21.15.120:0/618085323 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a3556e8060 0x55a3556ea470 crc :-1 s=READY pgs=589 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.539+0000 7fb836097740 1 --2- 172.21.15.120:0/618085323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3556d8680 0x55a3556d58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.539+0000 7fb836097740 1 --2- 172.21.15.120:0/618085323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3556cddc0 0x55a3556d52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.539+0000 7fb8117fa640 0 client.23232 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:09.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.539+0000 7fb836097740 1 --2- 172.21.15.120:0/618085323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3556cd4a0 0x55a3556d8040 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.539+0000 7fb836097740 1 -- 172.21.15.120:0/618085323 >> 172.21.15.120:0/618085323 conn(0x55a3556c9a30 msgr2=0x55a3556cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:09.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.539+0000 7fb836097740 1 -- 172.21.15.120:0/618085323 shutdown_connections 2024-09-19T09:27:09.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.539+0000 7fb836097740 1 -- 172.21.15.120:0/618085323 wait complete. 2024-09-19T09:27:09.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.539+0000 7fb836097740 20 client.23232 trim_cache size 0 max 16384 2024-09-19T09:27:09.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.558+0000 7fb836097740 1 Processor -- start 2024-09-19T09:27:09.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.558+0000 7fb836097740 1 -- start start 2024-09-19T09:27:09.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.559+0000 7fb836097740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3556cd4a0 0x55a3556e44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.559+0000 7fb836097740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3556cddc0 0x55a3556efd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.559+0000 7fb836097740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3556d8680 0x55a3556f0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.559+0000 7fb836097740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c1ad4f0 con 0x55a3556cd4a0 2024-09-19T09:27:09.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.559+0000 7fb836097740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c1af030 con 0x55a3556d8680 2024-09-19T09:27:09.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.559+0000 7fb836097740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c1adbc0 con 0x55a3556cddc0 2024-09-19T09:27:09.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.559+0000 7fb831259640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3556cddc0 0x55a3556efd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.559+0000 7fb831a5a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3556cd4a0 0x55a3556e44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.559+0000 7fb831259640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3556cddc0 0x55a3556efd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:37144/0 (socket says 172.21.15.120:37144) 2024-09-19T09:27:09.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.559+0000 7fb831a5a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3556cd4a0 0x55a3556e44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55632/0 (socket says 172.21.15.120:55632) 2024-09-19T09:27:09.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.559+0000 7fb831259640 1 -- 172.21.15.120:0/2256923004 learned_addr learned my addr 172.21.15.120:0/2256923004 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:09.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.559+0000 7fb83225b640 1 --2- 172.21.15.120:0/2256923004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3556d8680 0x55a3556f0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.559+0000 7fb831a5a640 1 -- 172.21.15.120:0/2256923004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3556cddc0 msgr2=0x55a3556efd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:09.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.559+0000 7fb831a5a640 1 --2- 172.21.15.120:0/2256923004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3556cddc0 0x55a3556efd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.559+0000 7fb831a5a640 1 -- 172.21.15.120:0/2256923004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3556d8680 msgr2=0x55a3556f0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:09.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.559+0000 7fb831a5a640 1 --2- 172.21.15.120:0/2256923004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3556d8680 0x55a3556f0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.559+0000 7fb831a5a640 1 -- 172.21.15.120:0/2256923004 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3556d83b0 con 0x55a3556cd4a0 2024-09-19T09:27:09.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.559+0000 7fb83225b640 1 --2- 172.21.15.120:0/2256923004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3556d8680 0x55a3556f0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:09.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.560+0000 7fb831a5a640 1 --2- 172.21.15.120:0/2256923004 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3556cd4a0 0x55a3556e44b0 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0bd4f0 tx=0x7fb81c031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3f63cd9a65a22d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:09.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.560+0000 7fb8137fe640 1 -- 172.21.15.120:0/2256923004 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c0bbb20 con 0x55a3556cd4a0 2024-09-19T09:27:09.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.560+0000 7fb8137fe640 1 -- 172.21.15.120:0/2256923004 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb81c036040 con 0x55a3556cd4a0 2024-09-19T09:27:09.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.560+0000 7fb8137fe640 1 -- 172.21.15.120:0/2256923004 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c0b9800 con 0x55a3556cd4a0 2024-09-19T09:27:09.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.561+0000 7fb836097740 1 -- 172.21.15.120:0/2256923004 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3556cd4a0 msgr2=0x55a3556e44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:09.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.561+0000 7fb836097740 1 --2- 172.21.15.120:0/2256923004 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3556cd4a0 0x55a3556e44b0 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0bd4f0 tx=0x7fb81c031000 comp rx=0 tx=0).stop 2024-09-19T09:27:09.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.561+0000 7fb836097740 1 -- 172.21.15.120:0/2256923004 shutdown_connections 2024-09-19T09:27:09.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.561+0000 7fb836097740 1 --2- 172.21.15.120:0/2256923004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3556d8680 0x55a3556f0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.561+0000 7fb836097740 1 --2- 172.21.15.120:0/2256923004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3556cddc0 0x55a3556efd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.561+0000 7fb836097740 1 --2- 172.21.15.120:0/2256923004 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3556cd4a0 0x55a3556e44b0 unknown :-1 s=CLOSED pgs=2436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.561+0000 7fb836097740 1 -- 172.21.15.120:0/2256923004 >> 172.21.15.120:0/2256923004 conn(0x55a3556c9a30 msgr2=0x55a3556c8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:09.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.561+0000 7fb836097740 1 -- 172.21.15.120:0/2256923004 shutdown_connections 2024-09-19T09:27:09.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.561+0000 7fb836097740 1 -- 172.21.15.120:0/2256923004 wait complete. 2024-09-19T09:27:09.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.561+0000 7fb836097740 1 -- start start 2024-09-19T09:27:09.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.561+0000 7fb836097740 1 Processor -- start 2024-09-19T09:27:09.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.562+0000 7fb836097740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3556cd4a0 0x55a3556dcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.562+0000 7fb836097740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3556cddc0 0x55a3556dd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.562+0000 7fb836097740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3556d8680 0x55a355756120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.562+0000 7fb836097740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c1ae960 con 0x55a3556cd4a0 2024-09-19T09:27:09.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.562+0000 7fb836097740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c1ae290 con 0x55a3556d8680 2024-09-19T09:27:09.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.562+0000 7fb836097740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c1ab4b0 con 0x55a3556cddc0 2024-09-19T09:27:09.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.562+0000 7fb836097740 10 client.0 _ll_register_callbacks cb 0x7fb81c009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:09.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.562+0000 7fb831a5a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3556cd4a0 0x55a3556dcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.562+0000 7fb831259640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3556cddc0 0x55a3556dd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.562+0000 7fb83225b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3556d8680 0x55a355756120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.562+0000 7fb831a5a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3556cd4a0 0x55a3556dcb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55644/0 (socket says 172.21.15.120:55644) 2024-09-19T09:27:09.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.562+0000 7fb831a5a640 1 -- 172.21.15.120:0/2877598671 learned_addr learned my addr 172.21.15.120:0/2877598671 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:09.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.562+0000 7fb83225b640 1 -- 172.21.15.120:0/2877598671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3556cddc0 msgr2=0x55a3556dd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:09.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.562+0000 7fb83225b640 1 --2- 172.21.15.120:0/2877598671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3556cddc0 0x55a3556dd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.562+0000 7fb83225b640 1 -- 172.21.15.120:0/2877598671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3556cd4a0 msgr2=0x55a3556dcb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:09.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.562+0000 7fb83225b640 1 --2- 172.21.15.120:0/2877598671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3556cd4a0 0x55a3556dcb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.562+0000 7fb83225b640 1 -- 172.21.15.120:0/2877598671 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a355756760 con 0x55a3556d8680 2024-09-19T09:27:09.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.563+0000 7fb83225b640 1 --2- 172.21.15.120:0/2877598671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3556d8680 0x55a355756120 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb824012010 tx=0x7fb824004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8afbe8e71504b41 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:09.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.563+0000 7fb8117fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:27:09.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.563+0000 7fb8117fa640 1 -- 172.21.15.120:0/2877598671 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824004250 con 0x55a3556d8680 2024-09-19T09:27:09.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.563+0000 7fb8117fa640 1 -- 172.21.15.120:0/2877598671 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fb8240043f0 con 0x55a3556d8680 2024-09-19T09:27:09.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.563+0000 7fb836097740 1 -- 172.21.15.120:0/2877598671 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb81c036040 con 0x55a3556d8680 2024-09-19T09:27:09.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.563+0000 7fb8117fa640 1 -- 172.21.15.120:0/2877598671 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb8240219c0 con 0x55a3556d8680 2024-09-19T09:27:09.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.563+0000 7fb836097740 10 client.31889 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:09.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.563+0000 7fb836097740 1 -- 172.21.15.120:0/2877598671 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a355757a90 con 0x55a3556d8680 2024-09-19T09:27:09.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.563+0000 7fb8117fa640 1 -- 172.21.15.120:0/2877598671 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb81c036040 con 0x55a3556d8680 2024-09-19T09:27:09.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.563+0000 7fb8117fa640 1 -- 172.21.15.120:0/2877598671 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fb824028020 con 0x55a3556d8680 2024-09-19T09:27:09.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.563+0000 7fb836097740 10 client.31889 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:09.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.564+0000 7fb836097740 10 client.31889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:27:09.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.564+0000 7fb836097740 20 client.31889 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:09.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.564+0000 7fb836097740 1 --2- 172.21.15.120:0/2877598671 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a3556e8060 0x55a35575a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.564+0000 7fb836097740 4 client.31889 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:27:09.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.564+0000 7fb836097740 1 -- 172.21.15.120:0/2877598671 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a35563ad60 con 0x55a3556e8060 2024-09-19T09:27:09.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.564+0000 7fb831a5a640 1 --2- 172.21.15.120:0/2877598671 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a3556e8060 0x55a35575a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.564+0000 7fb831a5a640 1 --2- 172.21.15.120:0/2877598671 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a3556e8060 0x55a35575a220 crc :-1 s=READY pgs=590 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff8a336216421a3b server_cookie=7559e47eebee1fd in_seq=0 out_seq=0 2024-09-19T09:27:09.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.564+0000 7fb8117fa640 10 client.31889 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:09.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.566+0000 7fb8117fa640 1 -- 172.21.15.120:0/2877598671 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a35563ad60 con 0x55a3556e8060 2024-09-19T09:27:09.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.566+0000 7fb8117fa640 10 client.31889 handle_command_reply: tid=0 2024-09-19T09:27:09.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.566+0000 7fb836097740 1 client.31889 shutdown 2024-09-19T09:27:09.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.566+0000 7fb836097740 1 -- 172.21.15.120:0/2877598671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3556d8680 msgr2=0x55a355756120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:09.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.566+0000 7fb836097740 1 --2- 172.21.15.120:0/2877598671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3556d8680 0x55a355756120 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb824012010 tx=0x7fb824004730 comp rx=0 tx=0).stop 2024-09-19T09:27:09.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.566+0000 7fb836097740 1 -- 172.21.15.120:0/2877598671 shutdown_connections 2024-09-19T09:27:09.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.566+0000 7fb836097740 1 --2- 172.21.15.120:0/2877598671 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a3556e8060 0x55a35575a220 crc :-1 s=READY pgs=590 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.566+0000 7fb836097740 1 --2- 172.21.15.120:0/2877598671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3556d8680 0x55a355756120 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.566+0000 7fb836097740 1 --2- 172.21.15.120:0/2877598671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3556cddc0 0x55a3556dd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.566+0000 7fb8117fa640 0 client.31889 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:09.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.566+0000 7fb836097740 1 --2- 172.21.15.120:0/2877598671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a3556cd4a0 0x55a3556dcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.566+0000 7fb836097740 1 -- 172.21.15.120:0/2877598671 >> 172.21.15.120:0/2877598671 conn(0x55a3556c9a30 msgr2=0x55a3556d6b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:09.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.566+0000 7fb836097740 1 -- 172.21.15.120:0/2877598671 shutdown_connections 2024-09-19T09:27:09.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.566+0000 7fb836097740 1 -- 172.21.15.120:0/2877598671 wait complete. 2024-09-19T09:27:09.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.566+0000 7fb836097740 20 client.31889 trim_cache size 0 max 16384 2024-09-19T09:27:09.582 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:27:09.582 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:27:09.582 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:27:09.583 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:27:09.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.567+0000 7fb833cbe640 1 -- 172.21.15.120:0/3561301823 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb8140a32c0 msgr2=0x7fb8140a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:09.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.567+0000 7fb833cbe640 1 --2- 172.21.15.120:0/3561301823 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb8140a32c0 0x7fb8140a5780 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb824009b30 tx=0x7fb824005f90 comp rx=0 tx=0).stop 2024-09-19T09:27:09.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.567+0000 7fb833cbe640 1 -- 172.21.15.120:0/3561301823 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb82c102d30 msgr2=0x7fb82c1985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:09.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.567+0000 7fb833cbe640 1 --2- 172.21.15.120:0/3561301823 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb82c102d30 0x7fb82c1985c0 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7fb81c02ae00 tx=0x7fb81c031e60 comp rx=0 tx=0).stop 2024-09-19T09:27:09.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.567+0000 7fb833cbe640 1 -- 172.21.15.120:0/3561301823 shutdown_connections 2024-09-19T09:27:09.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.567+0000 7fb833cbe640 1 --2- 172.21.15.120:0/3561301823 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb8140a32c0 0x7fb8140a5780 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.567+0000 7fb833cbe640 1 --2- 172.21.15.120:0/3561301823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb82c1082a0 0x7fb82c19ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.567+0000 7fb833cbe640 1 --2- 172.21.15.120:0/3561301823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb82c1036e0 0x7fb82c198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.567+0000 7fb833cbe640 1 --2- 172.21.15.120:0/3561301823 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb82c102d30 0x7fb82c1985c0 unknown :-1 s=CLOSED pgs=2433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.567+0000 7fb833cbe640 1 -- 172.21.15.120:0/3561301823 >> 172.21.15.120:0/3561301823 conn(0x7fb82c0fc7d0 msgr2=0x7fb82c105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:09.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.567+0000 7fb833cbe640 1 -- 172.21.15.120:0/3561301823 shutdown_connections 2024-09-19T09:27:09.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:09.567+0000 7fb833cbe640 1 -- 172.21.15.120:0/3561301823 wait complete. 2024-09-19T09:27:09.833 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.162+0000 7f07d245e640 1 -- 172.21.15.120:0/1860938548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07cc102d50 msgr2=0x7f07cc1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.162+0000 7f07d245e640 1 --2- 172.21.15.120:0/1860938548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07cc102d50 0x7f07cc1031d0 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009900 tx=0x7f07c002a410 comp rx=0 tx=0).stop 2024-09-19T09:27:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.162+0000 7f07d245e640 1 -- 172.21.15.120:0/1860938548 shutdown_connections 2024-09-19T09:27:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.162+0000 7f07d245e640 1 --2- 172.21.15.120:0/1860938548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07cc103710 0x7f07cc109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.162+0000 7f07d245e640 1 --2- 172.21.15.120:0/1860938548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07cc102d50 0x7f07cc1031d0 unknown :-1 s=CLOSED pgs=2437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.162+0000 7f07d245e640 1 --2- 172.21.15.120:0/1860938548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07cc101b60 0x7f07cc101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.162+0000 7f07d245e640 1 -- 172.21.15.120:0/1860938548 >> 172.21.15.120:0/1860938548 conn(0x7f07cc0fd2e0 msgr2=0x7f07cc0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.162+0000 7f07d245e640 1 -- 172.21.15.120:0/1860938548 shutdown_connections 2024-09-19T09:27:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.162+0000 7f07d245e640 1 -- 172.21.15.120:0/1860938548 wait complete. 2024-09-19T09:27:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.163+0000 7f07d245e640 1 Processor -- start 2024-09-19T09:27:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.163+0000 7f07d245e640 1 -- start start 2024-09-19T09:27:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.163+0000 7f07d245e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07cc101b60 0x7f07cc075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.163+0000 7f07d245e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07cc102d50 0x7f07cc0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.163+0000 7f07d245e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07cc103710 0x7f07cc074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.163+0000 7f07d245e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc10d7c0 con 0x7f07cc103710 2024-09-19T09:27:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.163+0000 7f07d245e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc10d640 con 0x7f07cc101b60 2024-09-19T09:27:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.163+0000 7f07d245e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc10d940 con 0x7f07cc102d50 2024-09-19T09:27:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.163+0000 7f07d09fb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07cc103710 0x7f07cc074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.163+0000 7f07d09fb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07cc103710 0x7f07cc074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55662/0 (socket says 172.21.15.120:55662) 2024-09-19T09:27:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.163+0000 7f07d09fb640 1 -- 172.21.15.120:0/3512180870 learned_addr learned my addr 172.21.15.120:0/3512180870 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.163+0000 7f07cbfff640 1 --2- 172.21.15.120:0/3512180870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07cc101b60 0x7f07cc075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.163+0000 7f07cb7fe640 1 --2- 172.21.15.120:0/3512180870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07cc102d50 0x7f07cc0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.163+0000 7f07d09fb640 1 -- 172.21.15.120:0/3512180870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07cc102d50 msgr2=0x7f07cc0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.163+0000 7f07d09fb640 1 --2- 172.21.15.120:0/3512180870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07cc102d50 0x7f07cc0745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.163+0000 7f07d09fb640 1 -- 172.21.15.120:0/3512180870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07cc101b60 msgr2=0x7f07cc075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.163+0000 7f07d09fb640 1 --2- 172.21.15.120:0/3512180870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07cc101b60 0x7f07cc075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.163+0000 7f07d09fb640 1 -- 172.21.15.120:0/3512180870 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07cc0753b0 con 0x7f07cc103710 2024-09-19T09:27:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.164+0000 7f07d09fb640 1 --2- 172.21.15.120:0/3512180870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07cc103710 0x7f07cc074b20 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f07c400eb10 tx=0x7f07c400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.164+0000 7f07c97fa640 1 -- 172.21.15.120:0/3512180870 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c4004b20 con 0x7f07cc103710 2024-09-19T09:27:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.164+0000 7f07c97fa640 1 -- 172.21.15.120:0/3512180870 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07c4004cc0 con 0x7f07cc103710 2024-09-19T09:27:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.164+0000 7f07d245e640 1 -- 172.21.15.120:0/3512180870 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07cc1a1ee0 con 0x7f07cc103710 2024-09-19T09:27:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.164+0000 7f07c97fa640 1 -- 172.21.15.120:0/3512180870 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c4010760 con 0x7f07cc103710 2024-09-19T09:27:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.164+0000 7f07d245e640 1 -- 172.21.15.120:0/3512180870 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07cc1a2340 con 0x7f07cc103710 2024-09-19T09:27:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.167+0000 7f07c97fa640 1 -- 172.21.15.120:0/3512180870 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f07c4005020 con 0x7f07cc103710 2024-09-19T09:27:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.167+0000 7f07d51fa740 1 Processor -- start 2024-09-19T09:27:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.167+0000 7f07d51fa740 1 -- start start 2024-09-19T09:27:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.167+0000 7f07d51fa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623f6a9c9a0 0x5623f6a9cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.167+0000 7f07d51fa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f6a9d2e0 0x5623f6aa1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.167+0000 7f07d51fa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623f6aa2370 0x5623f6aa4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.167+0000 7f07d51fa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5623f6a002f0 con 0x5623f6a9d2e0 2024-09-19T09:27:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.167+0000 7f07d51fa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5623f69a8270 con 0x5623f6a9c9a0 2024-09-19T09:27:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.167+0000 7f07d51fa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5623f69fc2b0 con 0x5623f6aa2370 2024-09-19T09:27:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.167+0000 7f07cbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f6a9d2e0 0x5623f6aa1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.167+0000 7f07cbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f6a9d2e0 0x5623f6aa1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55664/0 (socket says 172.21.15.120:55664) 2024-09-19T09:27:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.167+0000 7f07cbfff640 1 -- 172.21.15.120:0/2460845873 learned_addr learned my addr 172.21.15.120:0/2460845873 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.167+0000 7f07cb7fe640 1 --2- 172.21.15.120:0/2460845873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623f6a9c9a0 0x5623f6a9cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.167+0000 7f07cb7fe640 1 --2- 172.21.15.120:0/2460845873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623f6aa2370 0x5623f6aa4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.167+0000 7f07cbfff640 1 -- 172.21.15.120:0/2460845873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623f6aa2370 msgr2=0x5623f6aa4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.167+0000 7f07cbfff640 1 --2- 172.21.15.120:0/2460845873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623f6aa2370 0x5623f6aa4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.167+0000 7f07cbfff640 1 -- 172.21.15.120:0/2460845873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623f6a9c9a0 msgr2=0x5623f6a9cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.167+0000 7f07cbfff640 1 --2- 172.21.15.120:0/2460845873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623f6a9c9a0 0x5623f6a9cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.167+0000 7f07c97fa640 1 --2- 172.21.15.120:0/3512180870 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f07b40a3370 0x7f07b40a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.167+0000 7f07cbfff640 1 -- 172.21.15.120:0/2460845873 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623f69a7340 con 0x5623f6a9d2e0 2024-09-19T09:27:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.167+0000 7f07cb7fe640 1 --2- 172.21.15.120:0/2460845873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623f6a9c9a0 0x5623f6a9cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.167+0000 7f07cbfff640 1 --2- 172.21.15.120:0/2460845873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f6a9d2e0 0x5623f6aa1da0 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009ca0 tx=0x7f07bc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c247dca7eeb2d6b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.168+0000 7f07cb7fe640 1 --2- 172.21.15.120:0/3512180870 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f07b40a3370 0x7f07b40a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.168+0000 7f07c97fa640 1 -- 172.21.15.120:0/3512180870 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f07c40cfe40 con 0x7f07cc103710 2024-09-19T09:27:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.168+0000 7f07b27fc640 1 -- 172.21.15.120:0/2460845873 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc004220 con 0x5623f6a9d2e0 2024-09-19T09:27:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.168+0000 7f07b27fc640 1 -- 172.21.15.120:0/2460845873 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07bc0043c0 con 0x5623f6a9d2e0 2024-09-19T09:27:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.168+0000 7f07b27fc640 1 -- 172.21.15.120:0/2460845873 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc019930 con 0x5623f6a9d2e0 2024-09-19T09:27:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.168+0000 7f07d51fa740 1 -- 172.21.15.120:0/2460845873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f6a9d2e0 msgr2=0x5623f6aa1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.168+0000 7f07d51fa740 1 --2- 172.21.15.120:0/2460845873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f6a9d2e0 0x5623f6aa1da0 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009ca0 tx=0x7f07bc011810 comp rx=0 tx=0).stop 2024-09-19T09:27:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.168+0000 7f07cb7fe640 1 --2- 172.21.15.120:0/3512180870 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f07b40a3370 0x7f07b40a5830 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f07c00314e0 tx=0x7f07c0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.168+0000 7f07d51fa740 1 -- 172.21.15.120:0/2460845873 shutdown_connections 2024-09-19T09:27:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.168+0000 7f07d51fa740 1 --2- 172.21.15.120:0/2460845873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623f6aa2370 0x5623f6aa4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.168+0000 7f07d51fa740 1 --2- 172.21.15.120:0/2460845873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f6a9d2e0 0x5623f6aa1da0 unknown :-1 s=CLOSED pgs=2439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.168+0000 7f07d51fa740 1 --2- 172.21.15.120:0/2460845873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623f6a9c9a0 0x5623f6a9cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.168+0000 7f07d51fa740 1 -- 172.21.15.120:0/2460845873 >> 172.21.15.120:0/2460845873 conn(0x5623f6a98010 msgr2=0x5623f6a98410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.169+0000 7f07d51fa740 1 -- 172.21.15.120:0/2460845873 shutdown_connections 2024-09-19T09:27:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.169+0000 7f07d51fa740 1 -- 172.21.15.120:0/2460845873 wait complete. 2024-09-19T09:27:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.169+0000 7f07d51fa740 1 -- start start 2024-09-19T09:27:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.169+0000 7f07d51fa740 1 Processor -- start 2024-09-19T09:27:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.169+0000 7f07d51fa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623f6a9c9a0 0x5623f6aa6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.169+0000 7f07d51fa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f6a9d2e0 0x5623f6aa7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.169+0000 7f07d51fa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623f6aa2370 0x5623f6aae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.169+0000 7f07d51fa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5623f6aa5340 con 0x5623f6a9d2e0 2024-09-19T09:27:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.169+0000 7f07d51fa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5623f6989290 con 0x5623f6aa2370 2024-09-19T09:27:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.169+0000 7f07d51fa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5623f6aa54c0 con 0x5623f6a9c9a0 2024-09-19T09:27:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.169+0000 7f07d51fa740 10 client.0 _ll_register_callbacks cb 0x5623f69ba140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.169+0000 7f07cb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f6a9d2e0 0x5623f6aa7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.169+0000 7f07cb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f6a9d2e0 0x5623f6aa7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55678/0 (socket says 172.21.15.120:55678) 2024-09-19T09:27:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.169+0000 7f07cb7fe640 1 -- 172.21.15.120:0/1017191829 learned_addr learned my addr 172.21.15.120:0/1017191829 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.169+0000 7f07d09fb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623f6aa2370 0x5623f6aae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.169+0000 7f07cbfff640 1 --2- 172.21.15.120:0/1017191829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623f6a9c9a0 0x5623f6aa6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.170+0000 7f07cb7fe640 1 -- 172.21.15.120:0/1017191829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623f6a9c9a0 msgr2=0x5623f6aa6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.170+0000 7f07cb7fe640 1 --2- 172.21.15.120:0/1017191829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623f6a9c9a0 0x5623f6aa6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.170+0000 7f07cb7fe640 1 -- 172.21.15.120:0/1017191829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623f6aa2370 msgr2=0x5623f6aae0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.170+0000 7f07cb7fe640 1 --2- 172.21.15.120:0/1017191829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623f6aa2370 0x5623f6aae0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.170+0000 7f07cb7fe640 1 -- 172.21.15.120:0/1017191829 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623f6713210 con 0x5623f6a9d2e0 2024-09-19T09:27:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.170+0000 7f07cb7fe640 1 --2- 172.21.15.120:0/1017191829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f6a9d2e0 0x5623f6aa7080 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009900 tx=0x7f07c0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9236a458b9e76e40 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.170+0000 7f07ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:27:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.170+0000 7f07ab7fe640 1 -- 172.21.15.120:0/1017191829 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c0046070 con 0x5623f6a9d2e0 2024-09-19T09:27:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.170+0000 7f07ab7fe640 1 -- 172.21.15.120:0/1017191829 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f07c0005510 con 0x5623f6a9d2e0 2024-09-19T09:27:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.170+0000 7f07d51fa740 1 -- 172.21.15.120:0/1017191829 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07bc0043c0 con 0x5623f6a9d2e0 2024-09-19T09:27:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.170+0000 7f07ab7fe640 1 -- 172.21.15.120:0/1017191829 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f07c0035040 con 0x5623f6a9d2e0 2024-09-19T09:27:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.170+0000 7f07d51fa740 10 client.23250 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.170+0000 7f07d51fa740 1 -- 172.21.15.120:0/1017191829 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5623f67df620 con 0x5623f6a9d2e0 2024-09-19T09:27:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.170+0000 7f07ab7fe640 1 -- 172.21.15.120:0/1017191829 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07bc0043c0 con 0x5623f6a9d2e0 2024-09-19T09:27:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.171+0000 7f07ab7fe640 1 -- 172.21.15.120:0/1017191829 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f07c004d8e0 con 0x5623f6a9d2e0 2024-09-19T09:27:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.171+0000 7f07d51fa740 10 client.23250 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.171+0000 7f07d51fa740 10 client.23250 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:27:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.171+0000 7f07d51fa740 20 client.23250 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.171+0000 7f07d51fa740 1 --2- 172.21.15.120:0/1017191829 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5623f6ab2820 0x5623f6ab4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.171+0000 7f07d51fa740 4 client.23250 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.171+0000 7f07d51fa740 1 -- 172.21.15.120:0/1017191829 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623f6a0da20 con 0x5623f6ab2820 2024-09-19T09:27:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.171+0000 7f07cbfff640 1 --2- 172.21.15.120:0/1017191829 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5623f6ab2820 0x5623f6ab4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.171+0000 7f07ab7fe640 10 client.23250 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.171+0000 7f07cbfff640 1 --2- 172.21.15.120:0/1017191829 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5623f6ab2820 0x5623f6ab4c30 crc :-1 s=READY pgs=591 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7416fa0550ce398 server_cookie=d94a1e73bd0598e8 in_seq=0 out_seq=0 2024-09-19T09:27:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.176+0000 7f07ab7fe640 1 -- 172.21.15.120:0/1017191829 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5623f6a0da20 con 0x5623f6ab2820 2024-09-19T09:27:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.176+0000 7f07ab7fe640 10 client.23250 handle_command_reply: tid=0 2024-09-19T09:27:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.176+0000 7f07d51fa740 1 client.23250 shutdown 2024-09-19T09:27:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.176+0000 7f07d51fa740 1 -- 172.21.15.120:0/1017191829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f6a9d2e0 msgr2=0x5623f6aa7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.176+0000 7f07d51fa740 1 --2- 172.21.15.120:0/1017191829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f6a9d2e0 0x5623f6aa7080 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009900 tx=0x7f07c0000bc0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.176+0000 7f07d51fa740 1 -- 172.21.15.120:0/1017191829 shutdown_connections 2024-09-19T09:27:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.176+0000 7f07d51fa740 1 --2- 172.21.15.120:0/1017191829 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5623f6ab2820 0x5623f6ab4c30 crc :-1 s=READY pgs=591 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.176+0000 7f07d51fa740 1 --2- 172.21.15.120:0/1017191829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623f6aa2370 0x5623f6aae0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.176+0000 7f07d51fa740 1 --2- 172.21.15.120:0/1017191829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f6a9d2e0 0x5623f6aa7080 unknown :-1 s=CLOSED pgs=2440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.176+0000 7f07ab7fe640 0 client.23250 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.176+0000 7f07d51fa740 1 --2- 172.21.15.120:0/1017191829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623f6a9c9a0 0x5623f6aa6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.176+0000 7f07d51fa740 1 -- 172.21.15.120:0/1017191829 >> 172.21.15.120:0/1017191829 conn(0x5623f6a98010 msgr2=0x5623f6a99820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.176+0000 7f07d51fa740 1 -- 172.21.15.120:0/1017191829 shutdown_connections 2024-09-19T09:27:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.176+0000 7f07d51fa740 1 -- 172.21.15.120:0/1017191829 wait complete. 2024-09-19T09:27:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.177+0000 7f07d51fa740 20 client.23250 trim_cache size 0 max 16384 2024-09-19T09:27:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.192+0000 7f07d51fa740 1 Processor -- start 2024-09-19T09:27:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.192+0000 7f07d51fa740 1 -- start start 2024-09-19T09:27:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.192+0000 7f07d51fa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f6a9c9a0 0x5623f6a969a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.192+0000 7f07d51fa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623f6a9d2e0 0x5623f6aae080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.192+0000 7f07d51fa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623f6aa2370 0x5623f6a96ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.192+0000 7f07d51fa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc1ab0b0 con 0x5623f6a9c9a0 2024-09-19T09:27:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.192+0000 7f07d51fa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc1acbf0 con 0x5623f6aa2370 2024-09-19T09:27:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.192+0000 7f07d51fa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc1ab780 con 0x5623f6a9d2e0 2024-09-19T09:27:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.192+0000 7f07cbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f6a9c9a0 0x5623f6a969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.192+0000 7f07cbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f6a9c9a0 0x5623f6a969a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55684/0 (socket says 172.21.15.120:55684) 2024-09-19T09:27:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.192+0000 7f07cbfff640 1 -- 172.21.15.120:0/1705364065 learned_addr learned my addr 172.21.15.120:0/1705364065 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.192+0000 7f07cbfff640 1 -- 172.21.15.120:0/1705364065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623f6a9d2e0 msgr2=0x5623f6aae080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.192+0000 7f07cbfff640 1 --2- 172.21.15.120:0/1705364065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623f6a9d2e0 0x5623f6aae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.192+0000 7f07cbfff640 1 -- 172.21.15.120:0/1705364065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623f6aa2370 msgr2=0x5623f6a96ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.192+0000 7f07cbfff640 1 --2- 172.21.15.120:0/1705364065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623f6aa2370 0x5623f6a96ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.192+0000 7f07cbfff640 1 -- 172.21.15.120:0/1705364065 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623f6a9d7c0 con 0x5623f6a9c9a0 2024-09-19T09:27:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.193+0000 7f07cbfff640 1 --2- 172.21.15.120:0/1705364065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f6a9c9a0 0x5623f6a969a0 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f07bc011d10 tx=0x7f07bc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5db0dcfa9ccca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.193+0000 7f07b1ffb640 1 -- 172.21.15.120:0/1705364065 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc005160 con 0x5623f6a9c9a0 2024-09-19T09:27:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.193+0000 7f07b1ffb640 1 -- 172.21.15.120:0/1705364065 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07bc0035f0 con 0x5623f6a9c9a0 2024-09-19T09:27:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.193+0000 7f07b1ffb640 1 -- 172.21.15.120:0/1705364065 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc02b7f0 con 0x5623f6a9c9a0 2024-09-19T09:27:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.193+0000 7f07d51fa740 1 -- 172.21.15.120:0/1705364065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f6a9c9a0 msgr2=0x5623f6a969a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.193+0000 7f07d51fa740 1 --2- 172.21.15.120:0/1705364065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f6a9c9a0 0x5623f6a969a0 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f07bc011d10 tx=0x7f07bc011810 comp rx=0 tx=0).stop 2024-09-19T09:27:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.194+0000 7f07d51fa740 1 -- 172.21.15.120:0/1705364065 shutdown_connections 2024-09-19T09:27:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.194+0000 7f07d51fa740 1 --2- 172.21.15.120:0/1705364065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623f6aa2370 0x5623f6a96ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.194+0000 7f07d51fa740 1 --2- 172.21.15.120:0/1705364065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623f6a9d2e0 0x5623f6aae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.194+0000 7f07d51fa740 1 --2- 172.21.15.120:0/1705364065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f6a9c9a0 0x5623f6a969a0 unknown :-1 s=CLOSED pgs=2441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.194+0000 7f07d51fa740 1 -- 172.21.15.120:0/1705364065 >> 172.21.15.120:0/1705364065 conn(0x5623f6a98010 msgr2=0x5623f6a9e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.194+0000 7f07d51fa740 1 -- 172.21.15.120:0/1705364065 shutdown_connections 2024-09-19T09:27:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.194+0000 7f07d51fa740 1 -- 172.21.15.120:0/1705364065 wait complete. 2024-09-19T09:27:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.194+0000 7f07d51fa740 1 -- start start 2024-09-19T09:27:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.194+0000 7f07d51fa740 1 Processor -- start 2024-09-19T09:27:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.194+0000 7f07d51fa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f6a9c9a0 0x5623f6aabb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.194+0000 7f07d51fa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623f6a9d2e0 0x5623f6aac080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.194+0000 7f07d51fa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623f6aa2370 0x5623f6aad040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.194+0000 7f07d51fa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc1ac520 con 0x5623f6a9c9a0 2024-09-19T09:27:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.194+0000 7f07d51fa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc1abe50 con 0x5623f6a9d2e0 2024-09-19T09:27:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.194+0000 7f07d51fa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc1a9070 con 0x5623f6aa2370 2024-09-19T09:27:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.194+0000 7f07d51fa740 10 client.0 _ll_register_callbacks cb 0x7f07bc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.194+0000 7f07cbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f6a9c9a0 0x5623f6aabb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.194+0000 7f07cbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f6a9c9a0 0x5623f6aabb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55690/0 (socket says 172.21.15.120:55690) 2024-09-19T09:27:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.195+0000 7f07cbfff640 1 -- 172.21.15.120:0/1723929279 learned_addr learned my addr 172.21.15.120:0/1723929279 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.195+0000 7f07cb7fe640 1 --2- 172.21.15.120:0/1723929279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623f6a9d2e0 0x5623f6aac080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.195+0000 7f07d09fb640 1 --2- 172.21.15.120:0/1723929279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623f6aa2370 0x5623f6aad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.195+0000 7f07cbfff640 1 -- 172.21.15.120:0/1723929279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623f6aa2370 msgr2=0x5623f6aad040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.195+0000 7f07cbfff640 1 --2- 172.21.15.120:0/1723929279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623f6aa2370 0x5623f6aad040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.195+0000 7f07cbfff640 1 -- 172.21.15.120:0/1723929279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623f6a9d2e0 msgr2=0x5623f6aac080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.195+0000 7f07cbfff640 1 --2- 172.21.15.120:0/1723929279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623f6a9d2e0 0x5623f6aac080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.195+0000 7f07cbfff640 1 -- 172.21.15.120:0/1723929279 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623f6aad650 con 0x5623f6a9c9a0 2024-09-19T09:27:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.195+0000 7f07d09fb640 1 --2- 172.21.15.120:0/1723929279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623f6aa2370 0x5623f6aad040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.195+0000 7f07cbfff640 1 --2- 172.21.15.120:0/1723929279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f6a9c9a0 0x5623f6aabb40 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009bc0 tx=0x7f07bc0051f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=162d0de97724da65 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.195+0000 7f07ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:27:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.195+0000 7f07ab7fe640 1 -- 172.21.15.120:0/1723929279 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc03a070 con 0x5623f6a9c9a0 2024-09-19T09:27:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.195+0000 7f07ab7fe640 1 -- 172.21.15.120:0/1723929279 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f07bc0039a0 con 0x5623f6a9c9a0 2024-09-19T09:27:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.195+0000 7f07d51fa740 1 -- 172.21.15.120:0/1723929279 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07bc0035f0 con 0x5623f6a9c9a0 2024-09-19T09:27:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.195+0000 7f07ab7fe640 1 -- 172.21.15.120:0/1723929279 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f07bc046050 con 0x5623f6a9c9a0 2024-09-19T09:27:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.195+0000 7f07d51fa740 10 client.23256 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.195+0000 7f07d51fa740 1 -- 172.21.15.120:0/1723929279 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5623f6b1d7e0 con 0x5623f6a9c9a0 2024-09-19T09:27:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.195+0000 7f07ab7fe640 1 -- 172.21.15.120:0/1723929279 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07bc0035f0 con 0x5623f6a9c9a0 2024-09-19T09:27:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.196+0000 7f07ab7fe640 1 -- 172.21.15.120:0/1723929279 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f07bc0194a0 con 0x5623f6a9c9a0 2024-09-19T09:27:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.196+0000 7f07d51fa740 10 client.23256 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.196+0000 7f07d51fa740 10 client.23256 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:27:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.196+0000 7f07d51fa740 20 client.23256 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.196+0000 7f07d51fa740 1 --2- 172.21.15.120:0/1723929279 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5623f6b1dff0 0x5623f6b20410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.196+0000 7f07d51fa740 4 client.23256 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:27:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.196+0000 7f07d51fa740 1 -- 172.21.15.120:0/1723929279 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5623f6aa2130 con 0x5623f6b1dff0 2024-09-19T09:27:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.196+0000 7f07cb7fe640 1 --2- 172.21.15.120:0/1723929279 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5623f6b1dff0 0x5623f6b20410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.196+0000 7f07ab7fe640 10 client.23256 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.196+0000 7f07cb7fe640 1 --2- 172.21.15.120:0/1723929279 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5623f6b1dff0 0x5623f6b20410 crc :-1 s=READY pgs=592 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ad5f5acbe92f560 server_cookie=2c06bce6ac9d3e96 in_seq=0 out_seq=0 2024-09-19T09:27:10.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:10 smithi120 ceph-mon[31150]: pgmap v1074: 97 pgs: 97 active+clean; 502 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.2 MiB/s wr, 606 op/s 2024-09-19T09:27:10.390 INFO:tasks.workunit.client.0.smithi120.stdout: 10 266814 20.89 MB/sec execute 545 sec latency 150.002 ms 2024-09-19T09:27:10.437 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:27:10.437 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:27:10.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:10 smithi196 ceph-mon[35150]: pgmap v1074: 97 pgs: 97 active+clean; 502 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.2 MiB/s wr, 606 op/s 2024-09-19T09:27:10.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:10 smithi153 ceph-mon[35071]: pgmap v1074: 97 pgs: 97 active+clean; 502 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.2 MiB/s wr, 606 op/s 2024-09-19T09:27:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.851+0000 7f07ab7fe640 1 -- 172.21.15.120:0/1723929279 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5623f6aa2130 con 0x5623f6b1dff0 2024-09-19T09:27:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.851+0000 7f07ab7fe640 10 client.23256 handle_command_reply: tid=0 2024-09-19T09:27:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.851+0000 7f07d51fa740 1 client.23256 shutdown 2024-09-19T09:27:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.851+0000 7f07d51fa740 1 -- 172.21.15.120:0/1723929279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f6a9c9a0 msgr2=0x5623f6aabb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.851+0000 7f07d51fa740 1 --2- 172.21.15.120:0/1723929279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f6a9c9a0 0x5623f6aabb40 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009bc0 tx=0x7f07bc0051f0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.851+0000 7f07d51fa740 1 -- 172.21.15.120:0/1723929279 shutdown_connections 2024-09-19T09:27:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.851+0000 7f07d51fa740 1 --2- 172.21.15.120:0/1723929279 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5623f6b1dff0 0x5623f6b20410 crc :-1 s=READY pgs=592 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.851+0000 7f07d51fa740 1 --2- 172.21.15.120:0/1723929279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623f6aa2370 0x5623f6aad040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.851+0000 7f07d51fa740 1 --2- 172.21.15.120:0/1723929279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623f6a9d2e0 0x5623f6aac080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.851+0000 7f07d51fa740 1 --2- 172.21.15.120:0/1723929279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623f6a9c9a0 0x5623f6aabb40 unknown :-1 s=CLOSED pgs=2442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.851+0000 7f07d51fa740 1 -- 172.21.15.120:0/1723929279 >> 172.21.15.120:0/1723929279 conn(0x5623f6a98010 msgr2=0x5623f6a969a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.851+0000 7f07ab7fe640 0 client.23256 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.851+0000 7f07d51fa740 1 -- 172.21.15.120:0/1723929279 shutdown_connections 2024-09-19T09:27:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.851+0000 7f07d51fa740 1 -- 172.21.15.120:0/1723929279 wait complete. 2024-09-19T09:27:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.852+0000 7f07d51fa740 20 client.23256 trim_cache size 0 max 16384 2024-09-19T09:27:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:27:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:27:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:27:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:27:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.852+0000 7f07d245e640 1 -- 172.21.15.120:0/3512180870 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f07b40a3370 msgr2=0x7f07b40a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.852+0000 7f07d245e640 1 --2- 172.21.15.120:0/3512180870 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f07b40a3370 0x7f07b40a5830 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f07c00314e0 tx=0x7f07c0005fd0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.852+0000 7f07d245e640 1 -- 172.21.15.120:0/3512180870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07cc103710 msgr2=0x7f07cc074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.852+0000 7f07d245e640 1 --2- 172.21.15.120:0/3512180870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07cc103710 0x7f07cc074b20 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f07c400eb10 tx=0x7f07c400c2d0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.852+0000 7f07d245e640 1 -- 172.21.15.120:0/3512180870 shutdown_connections 2024-09-19T09:27:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.852+0000 7f07d245e640 1 --2- 172.21.15.120:0/3512180870 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f07b40a3370 0x7f07b40a5830 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.852+0000 7f07d245e640 1 --2- 172.21.15.120:0/3512180870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07cc103710 0x7f07cc074b20 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.852+0000 7f07d245e640 1 --2- 172.21.15.120:0/3512180870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07cc102d50 0x7f07cc0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.852+0000 7f07d245e640 1 --2- 172.21.15.120:0/3512180870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07cc101b60 0x7f07cc075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.852+0000 7f07d245e640 1 -- 172.21.15.120:0/3512180870 >> 172.21.15.120:0/3512180870 conn(0x7f07cc0fd2e0 msgr2=0x7f07cc106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.852+0000 7f07d245e640 1 -- 172.21.15.120:0/3512180870 shutdown_connections 2024-09-19T09:27:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:10.852+0000 7f07d245e640 1 -- 172.21.15.120:0/3512180870 wait complete. 2024-09-19T09:27:10.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.952+0000 7ff28b3d0640 1 -- 172.21.15.120:0/4021081998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff284101360 msgr2=0x7ff28410df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:10.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.953+0000 7ff28b3d0640 1 --2- 172.21.15.120:0/4021081998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff284101360 0x7ff28410df90 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7ff278009900 tx=0x7ff27802a410 comp rx=0 tx=0).stop 2024-09-19T09:27:10.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.953+0000 7ff28b3d0640 1 -- 172.21.15.120:0/4021081998 shutdown_connections 2024-09-19T09:27:10.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.953+0000 7ff28b3d0640 1 --2- 172.21.15.120:0/4021081998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff28410e4d0 0x7ff2841108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.953+0000 7ff28b3d0640 1 --2- 172.21.15.120:0/4021081998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff284101360 0x7ff28410df90 unknown :-1 s=CLOSED pgs=2443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.953+0000 7ff28b3d0640 1 --2- 172.21.15.120:0/4021081998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2841009b0 0x7ff284100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.953+0000 7ff28b3d0640 1 -- 172.21.15.120:0/4021081998 >> 172.21.15.120:0/4021081998 conn(0x7ff2840fc7d0 msgr2=0x7ff2840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:10.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.953+0000 7ff28b3d0640 1 -- 172.21.15.120:0/4021081998 shutdown_connections 2024-09-19T09:27:10.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.953+0000 7ff28b3d0640 1 -- 172.21.15.120:0/4021081998 wait complete. 2024-09-19T09:27:10.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.953+0000 7ff28b3d0640 1 Processor -- start 2024-09-19T09:27:10.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.953+0000 7ff28b3d0640 1 -- start start 2024-09-19T09:27:10.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.954+0000 7ff28b3d0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2841009b0 0x7ff2841a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.954+0000 7ff28b3d0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff284101360 0x7ff2841a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.954+0000 7ff28b3d0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff28410e4d0 0x7ff28419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.954+0000 7ff28b3d0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff284114220 con 0x7ff28410e4d0 2024-09-19T09:27:10.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.954+0000 7ff28b3d0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff2841140a0 con 0x7ff2841009b0 2024-09-19T09:27:10.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.954+0000 7ff28b3d0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff2841143a0 con 0x7ff284101360 2024-09-19T09:27:10.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.954+0000 7ff28996d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff28410e4d0 0x7ff28419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.954+0000 7ff28916c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2841009b0 0x7ff2841a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.954+0000 7ff28996d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff28410e4d0 0x7ff28419a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55706/0 (socket says 172.21.15.120:55706) 2024-09-19T09:27:10.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.954+0000 7ff28896b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff284101360 0x7ff2841a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.954+0000 7ff28996d640 1 -- 172.21.15.120:0/2491569182 learned_addr learned my addr 172.21.15.120:0/2491569182 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:10.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.954+0000 7ff28996d640 1 -- 172.21.15.120:0/2491569182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff284101360 msgr2=0x7ff2841a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:10.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.954+0000 7ff28996d640 1 --2- 172.21.15.120:0/2491569182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff284101360 0x7ff2841a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.954+0000 7ff28996d640 1 -- 172.21.15.120:0/2491569182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2841009b0 msgr2=0x7ff2841a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:10.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.954+0000 7ff28996d640 1 --2- 172.21.15.120:0/2491569182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2841009b0 0x7ff2841a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.954+0000 7ff28996d640 1 -- 172.21.15.120:0/2491569182 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28419ad00 con 0x7ff28410e4d0 2024-09-19T09:27:10.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.954+0000 7ff28996d640 1 --2- 172.21.15.120:0/2491569182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff28410e4d0 0x7ff28419a5d0 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7ff27c00e970 tx=0x7ff27c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:10.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.954+0000 7ff2827fc640 1 -- 172.21.15.120:0/2491569182 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27c00ccc0 con 0x7ff28410e4d0 2024-09-19T09:27:10.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.954+0000 7ff2827fc640 1 -- 172.21.15.120:0/2491569182 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff27c00ce60 con 0x7ff28410e4d0 2024-09-19T09:27:10.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.955+0000 7ff28b3d0640 1 -- 172.21.15.120:0/2491569182 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff28419aff0 con 0x7ff28410e4d0 2024-09-19T09:27:10.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.955+0000 7ff28b3d0640 1 -- 172.21.15.120:0/2491569182 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2841a72e0 con 0x7ff28410e4d0 2024-09-19T09:27:10.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.955+0000 7ff2827fc640 1 -- 172.21.15.120:0/2491569182 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27c010490 con 0x7ff28410e4d0 2024-09-19T09:27:10.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.957+0000 7ff28d7a9740 1 Processor -- start 2024-09-19T09:27:10.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.957+0000 7ff2827fc640 1 -- 172.21.15.120:0/2491569182 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7ff27c005020 con 0x7ff28410e4d0 2024-09-19T09:27:10.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.957+0000 7ff28d7a9740 1 -- start start 2024-09-19T09:27:10.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.957+0000 7ff28d7a9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621d7ad7760 0x5621d7ad7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.958+0000 7ff28d7a9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621d7ad8080 0x5621d7ae2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.958+0000 7ff28d7a9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621d7ae2940 0x5621d7ae4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.958+0000 7ff28d7a9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5621d7a3d2a0 con 0x5621d7ae2940 2024-09-19T09:27:10.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.958+0000 7ff28d7a9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5621d79e4b10 con 0x5621d7ad7760 2024-09-19T09:27:10.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.958+0000 7ff28d7a9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5621d7a39260 con 0x5621d7ad8080 2024-09-19T09:27:10.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.958+0000 7ff28916c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621d7ad8080 0x5621d7ae2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.958+0000 7ff28896b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621d7ad7760 0x5621d7ad7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.958+0000 7ff28916c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621d7ad8080 0x5621d7ae2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:37252/0 (socket says 172.21.15.120:37252) 2024-09-19T09:27:10.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.958+0000 7ff28916c640 1 -- 172.21.15.120:0/2015478067 learned_addr learned my addr 172.21.15.120:0/2015478067 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:10.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.958+0000 7ff28896b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621d7ad7760 0x5621d7ad7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:48304/0 (socket says 172.21.15.120:48304) 2024-09-19T09:27:10.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.958+0000 7ff28896b640 1 --2- 172.21.15.120:0/2015478067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621d7ae2940 0x5621d7ae4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.958+0000 7ff28916c640 1 -- 172.21.15.120:0/2015478067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621d7ad7760 msgr2=0x5621d7ad7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:10.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.958+0000 7ff28916c640 1 --2- 172.21.15.120:0/2015478067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621d7ad7760 0x5621d7ad7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.958+0000 7ff28916c640 1 -- 172.21.15.120:0/2015478067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621d7ae2940 msgr2=0x5621d7ae4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:10.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.958+0000 7ff28916c640 1 --2- 172.21.15.120:0/2015478067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621d7ae2940 0x5621d7ae4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.958+0000 7ff28916c640 1 -- 172.21.15.120:0/2015478067 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621d77aeb90 con 0x5621d7ad8080 2024-09-19T09:27:10.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.958+0000 7ff28896b640 1 --2- 172.21.15.120:0/2015478067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621d7ae2940 0x5621d7ae4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:10.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.958+0000 7ff2827fc640 1 --2- 172.21.15.120:0/2491569182 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff26c0a3240 0x7ff26c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.959+0000 7ff28896b640 1 --2- 172.21.15.120:0/2491569182 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff26c0a3240 0x7ff26c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.959+0000 7ff28916c640 1 --2- 172.21.15.120:0/2015478067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621d7ad8080 0x5621d7ae2400 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff274009fd0 tx=0x7ff274011950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b94d990c45d8bdd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:10.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.959+0000 7ff2827fc640 1 -- 172.21.15.120:0/2491569182 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff27c0cfce0 con 0x7ff28410e4d0 2024-09-19T09:27:10.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.959+0000 7ff26b7fe640 1 -- 172.21.15.120:0/2015478067 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff274027070 con 0x5621d7ad8080 2024-09-19T09:27:10.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.959+0000 7ff28896b640 1 --2- 172.21.15.120:0/2491569182 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff26c0a3240 0x7ff26c0a5700 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff278009230 tx=0x7ff2780058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:10.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.959+0000 7ff26b7fe640 1 -- 172.21.15.120:0/2015478067 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff27401ac30 con 0x5621d7ad8080 2024-09-19T09:27:10.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.959+0000 7ff26b7fe640 1 -- 172.21.15.120:0/2015478067 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27400f640 con 0x5621d7ad8080 2024-09-19T09:27:10.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.960+0000 7ff28d7a9740 1 -- 172.21.15.120:0/2015478067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621d7ad8080 msgr2=0x5621d7ae2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:10.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.960+0000 7ff28d7a9740 1 --2- 172.21.15.120:0/2015478067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621d7ad8080 0x5621d7ae2400 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff274009fd0 tx=0x7ff274011950 comp rx=0 tx=0).stop 2024-09-19T09:27:10.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.960+0000 7ff28d7a9740 1 -- 172.21.15.120:0/2015478067 shutdown_connections 2024-09-19T09:27:10.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.960+0000 7ff28d7a9740 1 --2- 172.21.15.120:0/2015478067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621d7ae2940 0x5621d7ae4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.960+0000 7ff28d7a9740 1 --2- 172.21.15.120:0/2015478067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621d7ad8080 0x5621d7ae2400 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.960+0000 7ff28d7a9740 1 --2- 172.21.15.120:0/2015478067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621d7ad7760 0x5621d7ad7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.960+0000 7ff28d7a9740 1 -- 172.21.15.120:0/2015478067 >> 172.21.15.120:0/2015478067 conn(0x5621d7ad3cf0 msgr2=0x5621d7ad40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:10.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.960+0000 7ff28d7a9740 1 -- 172.21.15.120:0/2015478067 shutdown_connections 2024-09-19T09:27:10.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.960+0000 7ff28d7a9740 1 -- 172.21.15.120:0/2015478067 wait complete. 2024-09-19T09:27:10.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.961+0000 7ff28d7a9740 1 -- start start 2024-09-19T09:27:10.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.961+0000 7ff28d7a9740 1 Processor -- start 2024-09-19T09:27:10.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.961+0000 7ff28d7a9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621d7ad7760 0x5621d7adf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.961+0000 7ff28d7a9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621d7ad8080 0x5621d7adfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.961+0000 7ff28d7a9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621d7ae2940 0x5621d7ae02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.961+0000 7ff28d7a9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5621d7ae5510 con 0x5621d7ad8080 2024-09-19T09:27:10.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.961+0000 7ff28d7a9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5621d79c56e0 con 0x5621d7ad7760 2024-09-19T09:27:10.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.961+0000 7ff28d7a9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5621d7ae5690 con 0x5621d7ae2940 2024-09-19T09:27:10.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.961+0000 7ff28d7a9740 10 client.0 _ll_register_callbacks cb 0x5621d79e9a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:10.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.961+0000 7ff28896b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621d7ad8080 0x5621d7adfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.961+0000 7ff28896b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621d7ad8080 0x5621d7adfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55724/0 (socket says 172.21.15.120:55724) 2024-09-19T09:27:10.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.961+0000 7ff28896b640 1 -- 172.21.15.120:0/1843567323 learned_addr learned my addr 172.21.15.120:0/1843567323 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:10.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.961+0000 7ff28996d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621d7ae2940 0x5621d7ae02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.962+0000 7ff28916c640 1 --2- 172.21.15.120:0/1843567323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621d7ad7760 0x5621d7adf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.962+0000 7ff28996d640 1 -- 172.21.15.120:0/1843567323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621d7ad7760 msgr2=0x5621d7adf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:10.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.962+0000 7ff28996d640 1 --2- 172.21.15.120:0/1843567323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621d7ad7760 0x5621d7adf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.962+0000 7ff28996d640 1 -- 172.21.15.120:0/1843567323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621d7ad8080 msgr2=0x5621d7adfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:10.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.962+0000 7ff28996d640 1 --2- 172.21.15.120:0/1843567323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621d7ad8080 0x5621d7adfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.962+0000 7ff28996d640 1 -- 172.21.15.120:0/1843567323 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621d7aeea50 con 0x5621d7ae2940 2024-09-19T09:27:10.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.962+0000 7ff28896b640 1 --2- 172.21.15.120:0/1843567323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621d7ad8080 0x5621d7adfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:27:10.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.962+0000 7ff28996d640 1 --2- 172.21.15.120:0/1843567323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621d7ae2940 0x5621d7ae02f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff27c004fd0 tx=0x7ff27c09cac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7662cbf6c724ae1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:10.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.962+0000 7ff268ff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:27:10.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.962+0000 7ff268ff9640 1 -- 172.21.15.120:0/1843567323 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27c0a4f70 con 0x5621d7ae2940 2024-09-19T09:27:10.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.962+0000 7ff268ff9640 1 -- 172.21.15.120:0/1843567323 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff27c005310 con 0x5621d7ae2940 2024-09-19T09:27:10.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.962+0000 7ff28d7a9740 1 -- 172.21.15.120:0/1843567323 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff27401ac30 con 0x5621d7ae2940 2024-09-19T09:27:10.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.962+0000 7ff268ff9640 1 -- 172.21.15.120:0/1843567323 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff27c0b5200 con 0x5621d7ae2940 2024-09-19T09:27:10.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.962+0000 7ff28d7a9740 10 client.31771 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:10.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.962+0000 7ff28d7a9740 1 -- 172.21.15.120:0/1843567323 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5621d7af1440 con 0x5621d7ae2940 2024-09-19T09:27:10.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.963+0000 7ff268ff9640 1 -- 172.21.15.120:0/1843567323 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff27401ac30 con 0x5621d7ae2940 2024-09-19T09:27:10.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.963+0000 7ff268ff9640 1 -- 172.21.15.120:0/1843567323 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7ff27c09b870 con 0x5621d7ae2940 2024-09-19T09:27:10.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.963+0000 7ff28d7a9740 10 client.31771 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:10.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.963+0000 7ff28d7a9740 10 client.31771 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:27:10.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.963+0000 7ff28d7a9740 20 client.31771 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:10.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.963+0000 7ff28d7a9740 1 --2- 172.21.15.120:0/1843567323 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5621d7af27e0 0x5621d7af4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.963+0000 7ff28d7a9740 4 client.31771 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:10.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.963+0000 7ff28d7a9740 1 -- 172.21.15.120:0/1843567323 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621d7a4a9d0 con 0x5621d7af27e0 2024-09-19T09:27:10.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.963+0000 7ff28916c640 1 --2- 172.21.15.120:0/1843567323 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5621d7af27e0 0x5621d7af4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.964+0000 7ff28916c640 1 --2- 172.21.15.120:0/1843567323 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5621d7af27e0 0x5621d7af4c80 crc :-1 s=READY pgs=593 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44dd8dba47927300 server_cookie=936264fa741fc2a2 in_seq=0 out_seq=0 2024-09-19T09:27:10.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.964+0000 7ff268ff9640 10 client.31771 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:10.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.966+0000 7ff268ff9640 1 -- 172.21.15.120:0/1843567323 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5621d7a4a9d0 con 0x5621d7af27e0 2024-09-19T09:27:10.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.966+0000 7ff268ff9640 10 client.31771 handle_command_reply: tid=0 2024-09-19T09:27:10.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.966+0000 7ff28d7a9740 1 client.31771 shutdown 2024-09-19T09:27:10.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.966+0000 7ff28d7a9740 1 -- 172.21.15.120:0/1843567323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621d7ae2940 msgr2=0x5621d7ae02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:10.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.966+0000 7ff28d7a9740 1 --2- 172.21.15.120:0/1843567323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621d7ae2940 0x5621d7ae02f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff27c004fd0 tx=0x7ff27c09cac0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.966+0000 7ff28d7a9740 1 -- 172.21.15.120:0/1843567323 shutdown_connections 2024-09-19T09:27:10.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.966+0000 7ff28d7a9740 1 --2- 172.21.15.120:0/1843567323 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5621d7af27e0 0x5621d7af4c80 crc :-1 s=READY pgs=593 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.966+0000 7ff28d7a9740 1 --2- 172.21.15.120:0/1843567323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621d7ae2940 0x5621d7ae02f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.966+0000 7ff28d7a9740 1 --2- 172.21.15.120:0/1843567323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621d7ad8080 0x5621d7adfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.966+0000 7ff28d7a9740 1 --2- 172.21.15.120:0/1843567323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621d7ad7760 0x5621d7adf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.966+0000 7ff28d7a9740 1 -- 172.21.15.120:0/1843567323 >> 172.21.15.120:0/1843567323 conn(0x5621d7ad3cf0 msgr2=0x5621d7ad99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:10.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.966+0000 7ff268ff9640 0 client.31771 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:10.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.966+0000 7ff28d7a9740 1 -- 172.21.15.120:0/1843567323 shutdown_connections 2024-09-19T09:27:10.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.966+0000 7ff28d7a9740 1 -- 172.21.15.120:0/1843567323 wait complete. 2024-09-19T09:27:10.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.967+0000 7ff28d7a9740 20 client.31771 trim_cache size 0 max 16384 2024-09-19T09:27:10.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.985+0000 7ff28d7a9740 1 Processor -- start 2024-09-19T09:27:10.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.985+0000 7ff28d7a9740 1 -- start start 2024-09-19T09:27:10.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.986+0000 7ff28d7a9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621d7ad7760 0x5621d7ad98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.986+0000 7ff28d7a9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621d7ad8080 0x5621d7afa550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.986+0000 7ff28d7a9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621d7ae2940 0x5621d7afaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.986+0000 7ff28d7a9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff2841ab410 con 0x5621d7ad8080 2024-09-19T09:27:10.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.986+0000 7ff28d7a9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff2841acf50 con 0x5621d7ae2940 2024-09-19T09:27:10.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.986+0000 7ff28d7a9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff2841abae0 con 0x5621d7ad7760 2024-09-19T09:27:10.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.986+0000 7ff28896b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621d7ad8080 0x5621d7afa550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.986+0000 7ff28896b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621d7ad8080 0x5621d7afa550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55732/0 (socket says 172.21.15.120:55732) 2024-09-19T09:27:10.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.986+0000 7ff28896b640 1 -- 172.21.15.120:0/2851953999 learned_addr learned my addr 172.21.15.120:0/2851953999 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:10.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.986+0000 7ff28916c640 1 --2- 172.21.15.120:0/2851953999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621d7ad7760 0x5621d7ad98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.986+0000 7ff28996d640 1 --2- 172.21.15.120:0/2851953999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621d7ae2940 0x5621d7afaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.986+0000 7ff28916c640 1 -- 172.21.15.120:0/2851953999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621d7ae2940 msgr2=0x5621d7afaa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:10.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.986+0000 7ff28916c640 1 --2- 172.21.15.120:0/2851953999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621d7ae2940 0x5621d7afaa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.986+0000 7ff28916c640 1 -- 172.21.15.120:0/2851953999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621d7ad8080 msgr2=0x5621d7afa550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:10.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.986+0000 7ff28916c640 1 --2- 172.21.15.120:0/2851953999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621d7ad8080 0x5621d7afa550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.986+0000 7ff28916c640 1 -- 172.21.15.120:0/2851953999 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621d7acf260 con 0x5621d7ad7760 2024-09-19T09:27:10.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.986+0000 7ff28896b640 1 --2- 172.21.15.120:0/2851953999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621d7ad8080 0x5621d7afa550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:27:10.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.986+0000 7ff28996d640 1 --2- 172.21.15.120:0/2851953999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621d7ae2940 0x5621d7afaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:27:10.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.986+0000 7ff28916c640 1 --2- 172.21.15.120:0/2851953999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621d7ad7760 0x5621d7ad98c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff274000f80 tx=0x7ff27401c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6944f9f5b9da95b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:10.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.987+0000 7ff26affd640 1 -- 172.21.15.120:0/2851953999 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff274027070 con 0x5621d7ad7760 2024-09-19T09:27:10.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.987+0000 7ff26affd640 1 -- 172.21.15.120:0/2851953999 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff274028040 con 0x5621d7ad7760 2024-09-19T09:27:10.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.987+0000 7ff26affd640 1 -- 172.21.15.120:0/2851953999 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff274022a00 con 0x5621d7ad7760 2024-09-19T09:27:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.988+0000 7ff28d7a9740 1 -- 172.21.15.120:0/2851953999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621d7ad7760 msgr2=0x5621d7ad98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.988+0000 7ff28d7a9740 1 --2- 172.21.15.120:0/2851953999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621d7ad7760 0x5621d7ad98c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff274000f80 tx=0x7ff27401c040 comp rx=0 tx=0).stop 2024-09-19T09:27:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.988+0000 7ff28d7a9740 1 -- 172.21.15.120:0/2851953999 shutdown_connections 2024-09-19T09:27:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.988+0000 7ff28d7a9740 1 --2- 172.21.15.120:0/2851953999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621d7ae2940 0x5621d7afaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.988+0000 7ff28d7a9740 1 --2- 172.21.15.120:0/2851953999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621d7ad8080 0x5621d7afa550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.988+0000 7ff28d7a9740 1 --2- 172.21.15.120:0/2851953999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621d7ad7760 0x5621d7ad98c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.988+0000 7ff28d7a9740 1 -- 172.21.15.120:0/2851953999 >> 172.21.15.120:0/2851953999 conn(0x5621d7ad3cf0 msgr2=0x5621d7ad91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:10.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.988+0000 7ff28d7a9740 1 -- 172.21.15.120:0/2851953999 shutdown_connections 2024-09-19T09:27:10.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.988+0000 7ff28d7a9740 1 -- 172.21.15.120:0/2851953999 wait complete. 2024-09-19T09:27:10.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.988+0000 7ff28d7a9740 1 -- start start 2024-09-19T09:27:10.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.988+0000 7ff28d7a9740 1 Processor -- start 2024-09-19T09:27:10.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.989+0000 7ff28d7a9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621d7ad7760 0x5621d7aeaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.989+0000 7ff28d7a9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621d7ad8080 0x5621d7aeb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.989+0000 7ff28d7a9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621d7ae2940 0x5621d7ae78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.989+0000 7ff28d7a9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff2841ac880 con 0x5621d7ae2940 2024-09-19T09:27:10.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.989+0000 7ff28d7a9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff2841ac1b0 con 0x5621d7ad7760 2024-09-19T09:27:10.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.989+0000 7ff28d7a9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff2841a93d0 con 0x5621d7ad8080 2024-09-19T09:27:10.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.989+0000 7ff28d7a9740 10 client.0 _ll_register_callbacks cb 0x7ff27c09c1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:10.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.989+0000 7ff28896b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621d7ad8080 0x5621d7aeb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.989+0000 7ff28996d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621d7ae2940 0x5621d7ae78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.989+0000 7ff28896b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621d7ad8080 0x5621d7aeb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:37282/0 (socket says 172.21.15.120:37282) 2024-09-19T09:27:10.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.989+0000 7ff28896b640 1 -- 172.21.15.120:0/3792835241 learned_addr learned my addr 172.21.15.120:0/3792835241 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:10.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.989+0000 7ff28916c640 1 --2- 172.21.15.120:0/3792835241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621d7ad7760 0x5621d7aeaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.989+0000 7ff28996d640 1 -- 172.21.15.120:0/3792835241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621d7ad8080 msgr2=0x5621d7aeb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:10.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.989+0000 7ff28996d640 1 --2- 172.21.15.120:0/3792835241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621d7ad8080 0x5621d7aeb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.989+0000 7ff28996d640 1 -- 172.21.15.120:0/3792835241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621d7ad7760 msgr2=0x5621d7aeaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:10.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.989+0000 7ff28996d640 1 --2- 172.21.15.120:0/3792835241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621d7ad7760 0x5621d7aeaac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.989+0000 7ff28996d640 1 -- 172.21.15.120:0/3792835241 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621d7ad4980 con 0x5621d7ae2940 2024-09-19T09:27:10.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.989+0000 7ff28896b640 1 --2- 172.21.15.120:0/3792835241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621d7ad8080 0x5621d7aeb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:27:10.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.989+0000 7ff28996d640 1 --2- 172.21.15.120:0/3792835241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621d7ae2940 0x5621d7ae78b0 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7ff27c098cd0 tx=0x7ff27c09a810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511a911309ec0248 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:10.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.989+0000 7ff268ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:27:10.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.990+0000 7ff268ff9640 1 -- 172.21.15.120:0/3792835241 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27c0a4f70 con 0x5621d7ae2940 2024-09-19T09:27:10.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.990+0000 7ff268ff9640 1 -- 172.21.15.120:0/3792835241 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff27c09b950 con 0x5621d7ae2940 2024-09-19T09:27:10.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.990+0000 7ff28d7a9740 1 -- 172.21.15.120:0/3792835241 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff274028040 con 0x5621d7ae2940 2024-09-19T09:27:10.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.990+0000 7ff268ff9640 1 -- 172.21.15.120:0/3792835241 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff27c0985a0 con 0x5621d7ae2940 2024-09-19T09:27:10.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.990+0000 7ff28d7a9740 10 client.23274 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:10.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.990+0000 7ff28d7a9740 1 -- 172.21.15.120:0/3792835241 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5621d7b5dd70 con 0x5621d7ae2940 2024-09-19T09:27:10.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.990+0000 7ff268ff9640 1 -- 172.21.15.120:0/3792835241 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff27c0a4cf0 con 0x5621d7ae2940 2024-09-19T09:27:10.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.990+0000 7ff268ff9640 1 -- 172.21.15.120:0/3792835241 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7ff27c0bc200 con 0x5621d7ae2940 2024-09-19T09:27:10.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.990+0000 7ff28d7a9740 10 client.23274 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:10.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.990+0000 7ff28d7a9740 10 client.23274 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:27:10.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.990+0000 7ff28d7a9740 20 client.23274 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:10.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.990+0000 7ff28d7a9740 1 --2- 172.21.15.120:0/3792835241 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5621d7b5e5b0 0x5621d7b609d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.990+0000 7ff28d7a9740 4 client.23274 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.5, "await_for": 45.0}] 2024-09-19T09:27:10.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.990+0000 7ff28d7a9740 1 -- 172.21.15.120:0/3792835241 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.5, "await_for": 45.0}) -- 0x7ff274028040 con 0x5621d7b5e5b0 2024-09-19T09:27:10.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.990+0000 7ff28916c640 1 --2- 172.21.15.120:0/3792835241 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5621d7b5e5b0 0x5621d7b609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.992+0000 7ff28916c640 1 --2- 172.21.15.120:0/3792835241 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5621d7b5e5b0 0x5621d7b609d0 crc :-1 s=READY pgs=594 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=118871a16396d0d2 server_cookie=eb318cb85758da38 in_seq=0 out_seq=0 2024-09-19T09:27:10.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:10.992+0000 7ff268ff9640 10 client.23274 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:11.032 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:11.390 INFO:tasks.workunit.client.0.smithi120.stdout: 10 266980 20.88 MB/sec execute 546 sec latency 442.732 ms 2024-09-19T09:27:11.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.544+0000 7f35cc6b6640 1 -- 172.21.15.120:0/77397911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35c4102d90 msgr2=0x7f35c4103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:11.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.544+0000 7f35cc6b6640 1 --2- 172.21.15.120:0/77397911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35c4102d90 0x7f35c4103170 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f35c0009930 tx=0x7f35c002a420 comp rx=0 tx=0).stop 2024-09-19T09:27:11.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.544+0000 7f35cc6b6640 1 -- 172.21.15.120:0/77397911 shutdown_connections 2024-09-19T09:27:11.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.544+0000 7f35cc6b6640 1 --2- 172.21.15.120:0/77397911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35c4108290 0x7f35c410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:11.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.544+0000 7f35cc6b6640 1 --2- 172.21.15.120:0/77397911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35c4103740 0x7f35c4107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:11.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.544+0000 7f35cc6b6640 1 --2- 172.21.15.120:0/77397911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35c4102d90 0x7f35c4103170 unknown :-1 s=CLOSED pgs=2446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:11.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.544+0000 7f35cc6b6640 1 -- 172.21.15.120:0/77397911 >> 172.21.15.120:0/77397911 conn(0x7f35c40fc7d0 msgr2=0x7f35c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:11.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.544+0000 7f35cc6b6640 1 -- 172.21.15.120:0/77397911 shutdown_connections 2024-09-19T09:27:11.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.544+0000 7f35cc6b6640 1 -- 172.21.15.120:0/77397911 wait complete. 2024-09-19T09:27:11.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.545+0000 7f35cc6b6640 1 Processor -- start 2024-09-19T09:27:11.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.545+0000 7f35cc6b6640 1 -- start start 2024-09-19T09:27:11.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.545+0000 7f35cc6b6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35c4102d90 0x7f35c41985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:11.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.546+0000 7f35cc6b6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35c4103740 0x7f35c4198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:11.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.546+0000 7f35cc6b6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35c4108290 0x7f35c419cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:11.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.546+0000 7f35cc6b6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f35c4112090 con 0x7f35c4108290 2024-09-19T09:27:11.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.546+0000 7f35cc6b6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f35c4111f10 con 0x7f35c4103740 2024-09-19T09:27:11.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.546+0000 7f35cc6b6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f35c4112210 con 0x7f35c4102d90 2024-09-19T09:27:11.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.546+0000 7f35c9c51640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35c4103740 0x7f35c4198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:11.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.546+0000 7f35cac53640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35c4108290 0x7f35c419cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:11.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.546+0000 7f35ca452640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35c4102d90 0x7f35c41985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:11.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.546+0000 7f35c9c51640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35c4103740 0x7f35c4198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:56342/0 (socket says 172.21.15.120:56342) 2024-09-19T09:27:11.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.546+0000 7f35cac53640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35c4108290 0x7f35c419cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34846/0 (socket says 172.21.15.120:34846) 2024-09-19T09:27:11.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.546+0000 7f35c9c51640 1 -- 172.21.15.120:0/2489810734 learned_addr learned my addr 172.21.15.120:0/2489810734 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:11.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.546+0000 7f35cac53640 1 -- 172.21.15.120:0/2489810734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35c4102d90 msgr2=0x7f35c41985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:11.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.546+0000 7f35cac53640 1 --2- 172.21.15.120:0/2489810734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35c4102d90 0x7f35c41985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:11.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.546+0000 7f35cac53640 1 -- 172.21.15.120:0/2489810734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35c4103740 msgr2=0x7f35c4198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:11.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.546+0000 7f35cac53640 1 --2- 172.21.15.120:0/2489810734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35c4103740 0x7f35c4198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:11.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.546+0000 7f35cac53640 1 -- 172.21.15.120:0/2489810734 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c419d5a0 con 0x7f35c4108290 2024-09-19T09:27:11.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.546+0000 7f35c9c51640 1 --2- 172.21.15.120:0/2489810734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35c4103740 0x7f35c4198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:27:11.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.546+0000 7f35cac53640 1 --2- 172.21.15.120:0/2489810734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35c4108290 0x7f35c419cec0 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f35bc009fd0 tx=0x7f35bc00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:11.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.546+0000 7f35bb7fe640 1 -- 172.21.15.120:0/2489810734 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc00ce20 con 0x7f35c4108290 2024-09-19T09:27:11.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.546+0000 7f35cc6b6640 1 -- 172.21.15.120:0/2489810734 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35c419d890 con 0x7f35c4108290 2024-09-19T09:27:11.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.546+0000 7f35bb7fe640 1 -- 172.21.15.120:0/2489810734 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35bc005020 con 0x7f35c4108290 2024-09-19T09:27:11.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.546+0000 7f35bb7fe640 1 -- 172.21.15.120:0/2489810734 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc010720 con 0x7f35c4108290 2024-09-19T09:27:11.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.546+0000 7f35cc6b6640 1 -- 172.21.15.120:0/2489810734 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35c41a5170 con 0x7f35c4108290 2024-09-19T09:27:11.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.547+0000 7f35cc6b6640 1 -- 172.21.15.120:0/2489810734 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35c406a7f0 con 0x7f35c4108290 2024-09-19T09:27:11.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.549+0000 7f35bb7fe640 1 -- 172.21.15.120:0/2489810734 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f35bc005540 con 0x7f35c4108290 2024-09-19T09:27:11.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.549+0000 7f35bb7fe640 1 --2- 172.21.15.120:0/2489810734 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f35b00a3240 0x7f35b00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:11.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.549+0000 7f35ca452640 1 --2- 172.21.15.120:0/2489810734 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f35b00a3240 0x7f35b00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:11.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.549+0000 7f35bb7fe640 1 -- 172.21.15.120:0/2489810734 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f35bc0cff70 con 0x7f35c4108290 2024-09-19T09:27:11.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.550+0000 7f35ca452640 1 --2- 172.21.15.120:0/2489810734 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f35b00a3240 0x7f35b00a5700 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f35c0009900 tx=0x7f35c00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:11.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.552+0000 7f35bb7fe640 1 -- 172.21.15.120:0/2489810734 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f35bc09cd20 con 0x7f35c4108290 2024-09-19T09:27:11.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.764+0000 7f35cc6b6640 1 -- 172.21.15.120:0/2489810734 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35c4103170 con 0x7f35c4108290 2024-09-19T09:27:11.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.765+0000 7f35bb7fe640 1 -- 172.21.15.120:0/2489810734 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11988 (secure 0 0 0) 0x7f35c4103170 con 0x7f35c4108290 2024-09-19T09:27:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:27:11.766 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":39,"btime":"2024-09-19T09:18:53:120858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:18:53.070639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:27:11.766 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 39 2024-09-19T09:27:11.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.767+0000 7f35cc6b6640 1 -- 172.21.15.120:0/2489810734 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f35b00a3240 msgr2=0x7f35b00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:11.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.767+0000 7f35cc6b6640 1 --2- 172.21.15.120:0/2489810734 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f35b00a3240 0x7f35b00a5700 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f35c0009900 tx=0x7f35c00058d0 comp rx=0 tx=0).stop 2024-09-19T09:27:11.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.767+0000 7f35cc6b6640 1 -- 172.21.15.120:0/2489810734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35c4108290 msgr2=0x7f35c419cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:11.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.767+0000 7f35cc6b6640 1 --2- 172.21.15.120:0/2489810734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35c4108290 0x7f35c419cec0 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f35bc009fd0 tx=0x7f35bc00ef10 comp rx=0 tx=0).stop 2024-09-19T09:27:11.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.767+0000 7f35cc6b6640 1 -- 172.21.15.120:0/2489810734 shutdown_connections 2024-09-19T09:27:11.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.767+0000 7f35cc6b6640 1 --2- 172.21.15.120:0/2489810734 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f35b00a3240 0x7f35b00a5700 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:11.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.767+0000 7f35cc6b6640 1 --2- 172.21.15.120:0/2489810734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35c4108290 0x7f35c419cec0 unknown :-1 s=CLOSED pgs=2447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:11.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.767+0000 7f35cc6b6640 1 --2- 172.21.15.120:0/2489810734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35c4103740 0x7f35c4198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:11.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.767+0000 7f35cc6b6640 1 --2- 172.21.15.120:0/2489810734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35c4102d90 0x7f35c41985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:11.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.767+0000 7f35cc6b6640 1 -- 172.21.15.120:0/2489810734 >> 172.21.15.120:0/2489810734 conn(0x7f35c40fc7d0 msgr2=0x7f35c4106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:11.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.767+0000 7f35cc6b6640 1 -- 172.21.15.120:0/2489810734 shutdown_connections 2024-09-19T09:27:11.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:11.767+0000 7f35cc6b6640 1 -- 172.21.15.120:0/2489810734 wait complete. 2024-09-19T09:27:12.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:11 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2489810734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:12.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:11 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2489810734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:12.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:11 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2489810734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:12.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:12.275+0000 7ff268ff9640 1 -- 172.21.15.120:0/3792835241 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7ff274028040 con 0x5621d7b5e5b0 2024-09-19T09:27:12.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:12.275+0000 7ff268ff9640 10 client.23274 handle_command_reply: tid=0 2024-09-19T09:27:12.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:12.275+0000 7ff28d7a9740 1 client.23274 shutdown 2024-09-19T09:27:12.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:12.275+0000 7ff28d7a9740 1 -- 172.21.15.120:0/3792835241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621d7ae2940 msgr2=0x5621d7ae78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:12.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:12.275+0000 7ff28d7a9740 1 --2- 172.21.15.120:0/3792835241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621d7ae2940 0x5621d7ae78b0 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7ff27c098cd0 tx=0x7ff27c09a810 comp rx=0 tx=0).stop 2024-09-19T09:27:12.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:12.275+0000 7ff28d7a9740 1 -- 172.21.15.120:0/3792835241 shutdown_connections 2024-09-19T09:27:12.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:12.275+0000 7ff28d7a9740 1 --2- 172.21.15.120:0/3792835241 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5621d7b5e5b0 0x5621d7b609d0 crc :-1 s=READY pgs=594 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:12.275+0000 7ff28d7a9740 1 --2- 172.21.15.120:0/3792835241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621d7ae2940 0x5621d7ae78b0 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:12.275+0000 7ff28d7a9740 1 --2- 172.21.15.120:0/3792835241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621d7ad8080 0x5621d7aeb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:12.275+0000 7ff28d7a9740 1 --2- 172.21.15.120:0/3792835241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621d7ad7760 0x5621d7aeaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:12.275+0000 7ff28d7a9740 1 -- 172.21.15.120:0/3792835241 >> 172.21.15.120:0/3792835241 conn(0x5621d7ad3cf0 msgr2=0x5621d7ad2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:12.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:12.275+0000 7ff268ff9640 0 client.23274 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:12.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:12.275+0000 7ff28d7a9740 1 -- 172.21.15.120:0/3792835241 shutdown_connections 2024-09-19T09:27:12.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:12.275+0000 7ff28d7a9740 1 -- 172.21.15.120:0/3792835241 wait complete. 2024-09-19T09:27:12.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:12.275+0000 7ff28d7a9740 20 client.23274 trim_cache size 0 max 16384 2024-09-19T09:27:12.278 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:27:12.279 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 39, 2024-09-19T09:27:12.279 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:27:12.279 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 82, 2024-09-19T09:27:12.279 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:27:12.279 INFO:teuthology.orchestra.run.smithi120.stdout: "d75d1f53": { 2024-09-19T09:27:12.279 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 82, 2024-09-19T09:27:12.279 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:27:12.279 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:27:12.279 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:27:12.279 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:27:12.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:27:12.279 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:27:12.279 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 138.5, 2024-09-19T09:27:12.280 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:27:12.280 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:27:12.280 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:27:12.280 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:27:12.280 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:27:12.280 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:27:12.280 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:27:12.280 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:27:12.280 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:27:12.280 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:27:12.280 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:27:12.280 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:27:12.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:12.276+0000 7ff28b3d0640 1 -- 172.21.15.120:0/2491569182 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff26c0a3240 msgr2=0x7ff26c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:12.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:12.276+0000 7ff28b3d0640 1 --2- 172.21.15.120:0/2491569182 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff26c0a3240 0x7ff26c0a5700 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff278009230 tx=0x7ff2780058d0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:12.276+0000 7ff28b3d0640 1 -- 172.21.15.120:0/2491569182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff28410e4d0 msgr2=0x7ff28419a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:12.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:12.276+0000 7ff28b3d0640 1 --2- 172.21.15.120:0/2491569182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff28410e4d0 0x7ff28419a5d0 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7ff27c00e970 tx=0x7ff27c00ed70 comp rx=0 tx=0).stop 2024-09-19T09:27:12.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:12.276+0000 7ff28b3d0640 1 -- 172.21.15.120:0/2491569182 shutdown_connections 2024-09-19T09:27:12.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:12.276+0000 7ff28b3d0640 1 --2- 172.21.15.120:0/2491569182 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff26c0a3240 0x7ff26c0a5700 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:12.276+0000 7ff28b3d0640 1 --2- 172.21.15.120:0/2491569182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff28410e4d0 0x7ff28419a5d0 unknown :-1 s=CLOSED pgs=2444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:12.276+0000 7ff28b3d0640 1 --2- 172.21.15.120:0/2491569182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff284101360 0x7ff2841a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:12.276+0000 7ff28b3d0640 1 --2- 172.21.15.120:0/2491569182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2841009b0 0x7ff2841a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:12.276+0000 7ff28b3d0640 1 -- 172.21.15.120:0/2491569182 >> 172.21.15.120:0/2491569182 conn(0x7ff2840fc7d0 msgr2=0x7ff2840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:12.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:12.276+0000 7ff28b3d0640 1 -- 172.21.15.120:0/2491569182 shutdown_connections 2024-09-19T09:27:12.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:12.276+0000 7ff28b3d0640 1 -- 172.21.15.120:0/2491569182 wait complete. 2024-09-19T09:27:12.390 INFO:tasks.workunit.client.0.smithi120.stdout: 10 266983 20.84 MB/sec execute 547 sec latency 1373.005 ms 2024-09-19T09:27:12.748 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:27:13.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:12 smithi120 ceph-mon[31150]: pgmap v1075: 97 pgs: 97 active+clean; 502 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.0 MiB/s wr, 619 op/s 2024-09-19T09:27:13.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:12 smithi196 ceph-mon[35150]: pgmap v1075: 97 pgs: 97 active+clean; 502 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.0 MiB/s wr, 619 op/s 2024-09-19T09:27:13.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:12 smithi153 ceph-mon[35071]: pgmap v1075: 97 pgs: 97 active+clean; 502 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.0 MiB/s wr, 619 op/s 2024-09-19T09:27:13.390 INFO:tasks.workunit.client.0.smithi120.stdout: 10 266983 20.80 MB/sec execute 548 sec latency 2373.058 ms 2024-09-19T09:27:13.469 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd75d1f53', quiesce took 6.9 seconds. Will release after: 11.6 2024-09-19T09:27:13.510 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:14.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.017+0000 7f5037f85640 1 -- 172.21.15.120:0/780424482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5030102ef0 msgr2=0x7f503010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:14.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.017+0000 7f5037f85640 1 --2- 172.21.15.120:0/780424482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5030102ef0 0x7f503010da40 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f5028009ca0 tx=0x7f502802a950 comp rx=0 tx=0).stop 2024-09-19T09:27:14.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.017+0000 7f5037f85640 1 -- 172.21.15.120:0/780424482 shutdown_connections 2024-09-19T09:27:14.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.017+0000 7f5037f85640 1 --2- 172.21.15.120:0/780424482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5030102ef0 0x7f503010da40 unknown :-1 s=CLOSED pgs=2448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.017+0000 7f5037f85640 1 --2- 172.21.15.120:0/780424482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5030102570 0x7f50301029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.017+0000 7f5037f85640 1 --2- 172.21.15.120:0/780424482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5030100e20 0x7f5030101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.017+0000 7f5037f85640 1 -- 172.21.15.120:0/780424482 >> 172.21.15.120:0/780424482 conn(0x7f50300fc7d0 msgr2=0x7f50300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:14.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.017+0000 7f5037f85640 1 -- 172.21.15.120:0/780424482 shutdown_connections 2024-09-19T09:27:14.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.017+0000 7f5037f85640 1 -- 172.21.15.120:0/780424482 wait complete. 2024-09-19T09:27:14.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.017+0000 7f5037f85640 1 Processor -- start 2024-09-19T09:27:14.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.017+0000 7f5037f85640 1 -- start start 2024-09-19T09:27:14.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.017+0000 7f5037f85640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5030100e20 0x7f5030199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.017+0000 7f5037f85640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5030102570 0x7f50301998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.017+0000 7f5037f85640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5030102ef0 0x7f50301a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.017+0000 7f5037f85640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5030110f80 con 0x7f5030100e20 2024-09-19T09:27:14.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.018+0000 7f5037f85640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5030110e00 con 0x7f5030102570 2024-09-19T09:27:14.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.018+0000 7f5037f85640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5030111100 con 0x7f5030102ef0 2024-09-19T09:27:14.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.018+0000 7f5035d21640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5030100e20 0x7f5030199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.018+0000 7f5035d21640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5030100e20 0x7f5030199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34872/0 (socket says 172.21.15.120:34872) 2024-09-19T09:27:14.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.018+0000 7f5035d21640 1 -- 172.21.15.120:0/3919634101 learned_addr learned my addr 172.21.15.120:0/3919634101 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:14.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.018+0000 7f5035520640 1 --2- 172.21.15.120:0/3919634101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5030102570 0x7f50301998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.018+0000 7f5036522640 1 --2- 172.21.15.120:0/3919634101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5030102ef0 0x7f50301a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.018+0000 7f5035d21640 1 -- 172.21.15.120:0/3919634101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5030102ef0 msgr2=0x7f50301a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:14.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.018+0000 7f5035d21640 1 --2- 172.21.15.120:0/3919634101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5030102ef0 0x7f50301a43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.018+0000 7f5035d21640 1 -- 172.21.15.120:0/3919634101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5030102570 msgr2=0x7f50301998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:14.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.018+0000 7f5035d21640 1 --2- 172.21.15.120:0/3919634101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5030102570 0x7f50301998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.018+0000 7f5035d21640 1 -- 172.21.15.120:0/3919634101 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50301a4930 con 0x7f5030100e20 2024-09-19T09:27:14.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.018+0000 7f5035520640 1 --2- 172.21.15.120:0/3919634101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5030102570 0x7f50301998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:14.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.018+0000 7f5035d21640 1 --2- 172.21.15.120:0/3919634101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5030100e20 0x7f5030199370 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f502c00d950 tx=0x7f502c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:14.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.018+0000 7f5026ffd640 1 -- 172.21.15.120:0/3919634101 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c017030 con 0x7f5030100e20 2024-09-19T09:27:14.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.018+0000 7f5026ffd640 1 -- 172.21.15.120:0/3919634101 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f502c004020 con 0x7f5030100e20 2024-09-19T09:27:14.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.018+0000 7f5026ffd640 1 -- 172.21.15.120:0/3919634101 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c0047a0 con 0x7f5030100e20 2024-09-19T09:27:14.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.018+0000 7f5037f85640 1 -- 172.21.15.120:0/3919634101 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50301a4b00 con 0x7f5030100e20 2024-09-19T09:27:14.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.018+0000 7f5037f85640 1 -- 172.21.15.120:0/3919634101 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50301a5040 con 0x7f5030100e20 2024-09-19T09:27:14.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.020+0000 7f5026ffd640 1 -- 172.21.15.120:0/3919634101 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f502c0041d0 con 0x7f5030100e20 2024-09-19T09:27:14.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.021+0000 7f503a35e740 1 Processor -- start 2024-09-19T09:27:14.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.021+0000 7f503a35e740 1 -- start start 2024-09-19T09:27:14.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.021+0000 7f503a35e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55873c457b40 0x55873c456090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.021+0000 7f503a35e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55873c4565d0 0x55873c456a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.021+0000 7f503a35e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55873c461370 0x55873c463760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.021+0000 7f503a35e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55873c3bc2f0 con 0x55873c4565d0 2024-09-19T09:27:14.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.021+0000 7f503a35e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55873c363cf0 con 0x55873c461370 2024-09-19T09:27:14.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.021+0000 7f503a35e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55873c3b82b0 con 0x55873c457b40 2024-09-19T09:27:14.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.021+0000 7f5026ffd640 1 --2- 172.21.15.120:0/3919634101 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f501c0a32c0 0x7f501c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.021+0000 7f5035d21640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55873c4565d0 0x55873c456a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.021+0000 7f5035520640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55873c461370 0x55873c463760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.021+0000 7f5035d21640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55873c4565d0 0x55873c456a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34880/0 (socket says 172.21.15.120:34880) 2024-09-19T09:27:14.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.021+0000 7f5035d21640 1 -- 172.21.15.120:0/2633429073 learned_addr learned my addr 172.21.15.120:0/2633429073 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:14.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.022+0000 7f5036522640 1 --2- 172.21.15.120:0/3919634101 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f501c0a32c0 0x7f501c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.022+0000 7f5036522640 1 --2- 172.21.15.120:0/2633429073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55873c457b40 0x55873c456090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.022+0000 7f5026ffd640 1 -- 172.21.15.120:0/3919634101 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f502c0cf330 con 0x7f5030100e20 2024-09-19T09:27:14.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.022+0000 7f5035d21640 1 -- 172.21.15.120:0/2633429073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55873c457b40 msgr2=0x55873c456090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:14.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.022+0000 7f5035d21640 1 --2- 172.21.15.120:0/2633429073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55873c457b40 0x55873c456090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.022+0000 7f5035d21640 1 -- 172.21.15.120:0/2633429073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55873c461370 msgr2=0x55873c463760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:14.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.022+0000 7f5035d21640 1 --2- 172.21.15.120:0/2633429073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55873c461370 0x55873c463760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.022+0000 7f5035d21640 1 -- 172.21.15.120:0/2633429073 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55873c3483a0 con 0x55873c4565d0 2024-09-19T09:27:14.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.022+0000 7f5035520640 1 --2- 172.21.15.120:0/2633429073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55873c461370 0x55873c463760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:27:14.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.022+0000 7f5035d21640 1 --2- 172.21.15.120:0/2633429073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55873c4565d0 0x55873c456a10 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f502c005fd0 tx=0x7f502c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70fc88fde352720 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:14.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.022+0000 7f5036522640 1 --2- 172.21.15.120:0/3919634101 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f501c0a32c0 0x7f501c0a5780 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f5028030040 tx=0x7f50280095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:14.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.022+0000 7f501bfff640 1 -- 172.21.15.120:0/2633429073 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c09b2b0 con 0x55873c4565d0 2024-09-19T09:27:14.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.022+0000 7f501bfff640 1 -- 172.21.15.120:0/2633429073 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f502c09c050 con 0x55873c4565d0 2024-09-19T09:27:14.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.022+0000 7f501bfff640 1 -- 172.21.15.120:0/2633429073 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c09ada0 con 0x55873c4565d0 2024-09-19T09:27:14.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.023+0000 7f503a35e740 1 -- 172.21.15.120:0/2633429073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55873c4565d0 msgr2=0x55873c456a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:14.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.023+0000 7f503a35e740 1 --2- 172.21.15.120:0/2633429073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55873c4565d0 0x55873c456a10 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f502c005fd0 tx=0x7f502c000bc0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.023+0000 7f503a35e740 1 -- 172.21.15.120:0/2633429073 shutdown_connections 2024-09-19T09:27:14.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.023+0000 7f503a35e740 1 --2- 172.21.15.120:0/2633429073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55873c461370 0x55873c463760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.023+0000 7f503a35e740 1 --2- 172.21.15.120:0/2633429073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55873c4565d0 0x55873c456a10 unknown :-1 s=CLOSED pgs=2450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.023+0000 7f503a35e740 1 --2- 172.21.15.120:0/2633429073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55873c457b40 0x55873c456090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.023+0000 7f503a35e740 1 -- 172.21.15.120:0/2633429073 >> 172.21.15.120:0/2633429073 conn(0x55873c41f310 msgr2=0x55873c452c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:14.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.023+0000 7f503a35e740 1 -- 172.21.15.120:0/2633429073 shutdown_connections 2024-09-19T09:27:14.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.023+0000 7f503a35e740 1 -- 172.21.15.120:0/2633429073 wait complete. 2024-09-19T09:27:14.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.023+0000 7f503a35e740 1 -- start start 2024-09-19T09:27:14.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.023+0000 7f503a35e740 1 Processor -- start 2024-09-19T09:27:14.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.024+0000 7f503a35e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55873c4565d0 0x55873c460d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.024+0000 7f503a35e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55873c457b40 0x55873c45e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.024+0000 7f503a35e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55873c461370 0x55873c45e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.024+0000 7f503a35e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55873c464010 con 0x55873c457b40 2024-09-19T09:27:14.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.024+0000 7f503a35e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55873c377060 con 0x55873c461370 2024-09-19T09:27:14.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.024+0000 7f503a35e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55873c464190 con 0x55873c4565d0 2024-09-19T09:27:14.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.024+0000 7f503a35e740 10 client.0 _ll_register_callbacks cb 0x55873c0ca8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:14.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.024+0000 7f5035520640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55873c457b40 0x55873c45e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.024+0000 7f5035520640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55873c457b40 0x55873c45e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34892/0 (socket says 172.21.15.120:34892) 2024-09-19T09:27:14.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.024+0000 7f5035520640 1 -- 172.21.15.120:0/1392215258 learned_addr learned my addr 172.21.15.120:0/1392215258 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:14.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.024+0000 7f5035d21640 1 --2- 172.21.15.120:0/1392215258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55873c4565d0 0x55873c460d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.024+0000 7f5036522640 1 --2- 172.21.15.120:0/1392215258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55873c461370 0x55873c45e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.024+0000 7f5035520640 1 -- 172.21.15.120:0/1392215258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55873c4565d0 msgr2=0x55873c460d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:14.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.024+0000 7f5035520640 1 --2- 172.21.15.120:0/1392215258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55873c4565d0 0x55873c460d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.024+0000 7f5035520640 1 -- 172.21.15.120:0/1392215258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55873c461370 msgr2=0x55873c45e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:14.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.024+0000 7f5035520640 1 --2- 172.21.15.120:0/1392215258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55873c461370 0x55873c45e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.024+0000 7f5035520640 1 -- 172.21.15.120:0/1392215258 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55873c12d3c0 con 0x55873c457b40 2024-09-19T09:27:14.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.024+0000 7f5036522640 1 --2- 172.21.15.120:0/1392215258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55873c461370 0x55873c45e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:14.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.025+0000 7f5035520640 1 --2- 172.21.15.120:0/1392215258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55873c457b40 0x55873c45e030 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f502000adf0 tx=0x7f5020009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a482ab87eb8ea19a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:14.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.025+0000 7f50197fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:27:14.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.025+0000 7f50197fa640 1 -- 172.21.15.120:0/1392215258 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5020020060 con 0x55873c457b40 2024-09-19T09:27:14.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.025+0000 7f50197fa640 1 -- 172.21.15.120:0/1392215258 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f502001b660 con 0x55873c457b40 2024-09-19T09:27:14.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.025+0000 7f503a35e740 1 -- 172.21.15.120:0/1392215258 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f502c09c050 con 0x55873c457b40 2024-09-19T09:27:14.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.025+0000 7f50197fa640 1 -- 172.21.15.120:0/1392215258 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f502001b8a0 con 0x55873c457b40 2024-09-19T09:27:14.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.025+0000 7f503a35e740 10 client.23292 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:14.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.025+0000 7f503a35e740 1 -- 172.21.15.120:0/1392215258 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55873c46fba0 con 0x55873c457b40 2024-09-19T09:27:14.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.025+0000 7f50197fa640 1 -- 172.21.15.120:0/1392215258 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f502c09c050 con 0x55873c457b40 2024-09-19T09:27:14.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.025+0000 7f50197fa640 1 -- 172.21.15.120:0/1392215258 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f5020025070 con 0x55873c457b40 2024-09-19T09:27:14.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.025+0000 7f503a35e740 10 client.23292 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:14.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.025+0000 7f503a35e740 10 client.23292 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:27:14.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.025+0000 7f503a35e740 20 client.23292 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:14.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.025+0000 7f503a35e740 1 --2- 172.21.15.120:0/1392215258 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55873c471100 0x55873c473520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.025+0000 7f503a35e740 4 client.23292 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:14.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.025+0000 7f503a35e740 1 -- 172.21.15.120:0/1392215258 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55873c3c9860 con 0x55873c471100 2024-09-19T09:27:14.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.025+0000 7f5035d21640 1 --2- 172.21.15.120:0/1392215258 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55873c471100 0x55873c473520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.026+0000 7f5035d21640 1 --2- 172.21.15.120:0/1392215258 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55873c471100 0x55873c473520 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=92058083865cd1d6 server_cookie=caf7132e9a3fb79f in_seq=0 out_seq=0 2024-09-19T09:27:14.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.026+0000 7f50197fa640 10 client.23292 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:27:14.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.027+0000 7f50197fa640 1 -- 172.21.15.120:0/1392215258 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55873c3c9860 con 0x55873c471100 2024-09-19T09:27:14.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.027+0000 7f50197fa640 10 client.23292 handle_command_reply: tid=0 2024-09-19T09:27:14.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.027+0000 7f503a35e740 1 client.23292 shutdown 2024-09-19T09:27:14.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.027+0000 7f503a35e740 1 -- 172.21.15.120:0/1392215258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55873c457b40 msgr2=0x55873c45e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:14.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.027+0000 7f503a35e740 1 --2- 172.21.15.120:0/1392215258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55873c457b40 0x55873c45e030 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f502000adf0 tx=0x7f5020009670 comp rx=0 tx=0).stop 2024-09-19T09:27:14.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.027+0000 7f503a35e740 1 -- 172.21.15.120:0/1392215258 shutdown_connections 2024-09-19T09:27:14.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.027+0000 7f503a35e740 1 --2- 172.21.15.120:0/1392215258 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55873c471100 0x55873c473520 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.027+0000 7f503a35e740 1 --2- 172.21.15.120:0/1392215258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55873c461370 0x55873c45e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.027+0000 7f50197fa640 0 client.23292 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:27:14.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.027+0000 7f503a35e740 1 --2- 172.21.15.120:0/1392215258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55873c457b40 0x55873c45e030 unknown :-1 s=CLOSED pgs=2451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.027+0000 7f503a35e740 1 --2- 172.21.15.120:0/1392215258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55873c4565d0 0x55873c460d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.027+0000 7f503a35e740 1 -- 172.21.15.120:0/1392215258 >> 172.21.15.120:0/1392215258 conn(0x55873c41f310 msgr2=0x55873c458990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:14.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.027+0000 7f503a35e740 1 -- 172.21.15.120:0/1392215258 shutdown_connections 2024-09-19T09:27:14.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.027+0000 7f503a35e740 1 -- 172.21.15.120:0/1392215258 wait complete. 2024-09-19T09:27:14.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.027+0000 7f503a35e740 20 client.23292 trim_cache size 0 max 16384 2024-09-19T09:27:14.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.040+0000 7f503a35e740 1 Processor -- start 2024-09-19T09:27:14.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.040+0000 7f503a35e740 1 -- start start 2024-09-19T09:27:14.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.040+0000 7f503a35e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55873c4565d0 0x55873c44dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.040+0000 7f503a35e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55873c457b40 0x55873c45e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.040+0000 7f503a35e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55873c461370 0x55873c46d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.040+0000 7f503a35e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f50301ab4b0 con 0x55873c457b40 2024-09-19T09:27:14.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.040+0000 7f503a35e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f50301acff0 con 0x55873c461370 2024-09-19T09:27:14.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.040+0000 7f503a35e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f50301abb80 con 0x55873c4565d0 2024-09-19T09:27:14.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.040+0000 7f5035520640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55873c457b40 0x55873c45e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.040+0000 7f5035520640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55873c457b40 0x55873c45e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34894/0 (socket says 172.21.15.120:34894) 2024-09-19T09:27:14.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.040+0000 7f5035520640 1 -- 172.21.15.120:0/1638663440 learned_addr learned my addr 172.21.15.120:0/1638663440 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:14.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.040+0000 7f5035d21640 1 --2- 172.21.15.120:0/1638663440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55873c4565d0 0x55873c44dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.041+0000 7f5035520640 1 -- 172.21.15.120:0/1638663440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55873c4565d0 msgr2=0x55873c44dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:14.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.041+0000 7f5035520640 1 --2- 172.21.15.120:0/1638663440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55873c4565d0 0x55873c44dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.041+0000 7f5036522640 1 --2- 172.21.15.120:0/1638663440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55873c461370 0x55873c46d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.041+0000 7f5035520640 1 -- 172.21.15.120:0/1638663440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55873c461370 msgr2=0x55873c46d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:14.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.041+0000 7f5035520640 1 --2- 172.21.15.120:0/1638663440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55873c461370 0x55873c46d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.041+0000 7f5035520640 1 -- 172.21.15.120:0/1638663440 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55873c45d020 con 0x55873c457b40 2024-09-19T09:27:14.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.041+0000 7f5035d21640 1 --2- 172.21.15.120:0/1638663440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55873c4565d0 0x55873c44dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:14.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.041+0000 7f5036522640 1 --2- 172.21.15.120:0/1638663440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55873c461370 0x55873c46d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:14.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.041+0000 7f5035520640 1 --2- 172.21.15.120:0/1638663440 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55873c457b40 0x55873c45e000 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f502000a730 tx=0x7f502001a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77534cb01e8f3a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:14.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.041+0000 7f501b7fe640 1 -- 172.21.15.120:0/1638663440 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502002b070 con 0x55873c457b40 2024-09-19T09:27:14.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.041+0000 7f501b7fe640 1 -- 172.21.15.120:0/1638663440 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f502001b660 con 0x55873c457b40 2024-09-19T09:27:14.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.041+0000 7f501b7fe640 1 -- 172.21.15.120:0/1638663440 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502001a760 con 0x55873c457b40 2024-09-19T09:27:14.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.042+0000 7f503a35e740 1 -- 172.21.15.120:0/1638663440 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55873c457b40 msgr2=0x55873c45e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:14.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.042+0000 7f503a35e740 1 --2- 172.21.15.120:0/1638663440 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55873c457b40 0x55873c45e000 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f502000a730 tx=0x7f502001a550 comp rx=0 tx=0).stop 2024-09-19T09:27:14.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.042+0000 7f503a35e740 1 -- 172.21.15.120:0/1638663440 shutdown_connections 2024-09-19T09:27:14.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.042+0000 7f503a35e740 1 --2- 172.21.15.120:0/1638663440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55873c461370 0x55873c46d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.042+0000 7f503a35e740 1 --2- 172.21.15.120:0/1638663440 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55873c457b40 0x55873c45e000 unknown :-1 s=CLOSED pgs=2452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.042+0000 7f503a35e740 1 --2- 172.21.15.120:0/1638663440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55873c4565d0 0x55873c44dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.042+0000 7f503a35e740 1 -- 172.21.15.120:0/1638663440 >> 172.21.15.120:0/1638663440 conn(0x55873c41f310 msgr2=0x55873c45d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:14.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.042+0000 7f503a35e740 1 -- 172.21.15.120:0/1638663440 shutdown_connections 2024-09-19T09:27:14.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.042+0000 7f503a35e740 1 -- 172.21.15.120:0/1638663440 wait complete. 2024-09-19T09:27:14.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.042+0000 7f503a35e740 1 -- start start 2024-09-19T09:27:14.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.042+0000 7f503a35e740 1 Processor -- start 2024-09-19T09:27:14.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.043+0000 7f503a35e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55873c4565d0 0x55873c45afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.043+0000 7f503a35e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55873c457b40 0x55873c45b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.043+0000 7f503a35e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55873c461370 0x55873c45ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.043+0000 7f503a35e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f50301ac920 con 0x55873c4565d0 2024-09-19T09:27:14.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.043+0000 7f503a35e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f50301ac250 con 0x55873c457b40 2024-09-19T09:27:14.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.043+0000 7f503a35e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f50301a9470 con 0x55873c461370 2024-09-19T09:27:14.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.043+0000 7f503a35e740 10 client.0 _ll_register_callbacks cb 0x7f5020010bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:14.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.043+0000 7f5035d21640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55873c4565d0 0x55873c45afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.043+0000 7f5035d21640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55873c4565d0 0x55873c45afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34896/0 (socket says 172.21.15.120:34896) 2024-09-19T09:27:14.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.043+0000 7f5035d21640 1 -- 172.21.15.120:0/512739554 learned_addr learned my addr 172.21.15.120:0/512739554 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:14.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.043+0000 7f5035520640 1 --2- 172.21.15.120:0/512739554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55873c457b40 0x55873c45b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.043+0000 7f5036522640 1 --2- 172.21.15.120:0/512739554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55873c461370 0x55873c45ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.043+0000 7f5035d21640 1 -- 172.21.15.120:0/512739554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55873c461370 msgr2=0x55873c45ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:14.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.043+0000 7f5035d21640 1 --2- 172.21.15.120:0/512739554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55873c461370 0x55873c45ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.043+0000 7f5035d21640 1 -- 172.21.15.120:0/512739554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55873c457b40 msgr2=0x55873c45b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:14.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.043+0000 7f5035d21640 1 --2- 172.21.15.120:0/512739554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55873c457b40 0x55873c45b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.043+0000 7f5035d21640 1 -- 172.21.15.120:0/512739554 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55873c4dada0 con 0x55873c4565d0 2024-09-19T09:27:14.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.043+0000 7f5035520640 1 --2- 172.21.15.120:0/512739554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55873c457b40 0x55873c45b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:14.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.043+0000 7f5036522640 1 --2- 172.21.15.120:0/512739554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55873c461370 0x55873c45ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:14.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.043+0000 7f5035d21640 1 --2- 172.21.15.120:0/512739554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55873c4565d0 0x55873c45afb0 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f502c098320 tx=0x7f502c086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae2656a3f584485 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:14.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.043+0000 7f50197fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:27:14.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.044+0000 7f50197fa640 1 -- 172.21.15.120:0/512739554 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c0b9060 con 0x55873c4565d0 2024-09-19T09:27:14.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.044+0000 7f50197fa640 1 -- 172.21.15.120:0/512739554 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f502c0cceb0 con 0x55873c4565d0 2024-09-19T09:27:14.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.044+0000 7f503a35e740 1 -- 172.21.15.120:0/512739554 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f502001b660 con 0x55873c4565d0 2024-09-19T09:27:14.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.044+0000 7f50197fa640 1 -- 172.21.15.120:0/512739554 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f502c09b0d0 con 0x55873c4565d0 2024-09-19T09:27:14.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.044+0000 7f503a35e740 10 client.23298 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:14.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.044+0000 7f503a35e740 1 -- 172.21.15.120:0/512739554 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55873c4dbf10 con 0x55873c4565d0 2024-09-19T09:27:14.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.044+0000 7f50197fa640 1 -- 172.21.15.120:0/512739554 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f502c09c050 con 0x55873c4565d0 2024-09-19T09:27:14.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.044+0000 7f50197fa640 1 -- 172.21.15.120:0/512739554 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f502c0ce030 con 0x55873c4565d0 2024-09-19T09:27:14.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.044+0000 7f503a35e740 10 client.23298 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:14.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.044+0000 7f503a35e740 10 client.23298 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:27:14.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.044+0000 7f503a35e740 20 client.23298 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:14.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.044+0000 7f503a35e740 1 --2- 172.21.15.120:0/512739554 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55873c4dc200 0x55873c4de5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.044+0000 7f503a35e740 4 client.23298 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:27:14.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.044+0000 7f503a35e740 1 -- 172.21.15.120:0/512739554 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f502001b660 con 0x55873c4dc200 2024-09-19T09:27:14.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.044+0000 7f5035520640 1 --2- 172.21.15.120:0/512739554 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55873c4dc200 0x55873c4de5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.045+0000 7f5035520640 1 --2- 172.21.15.120:0/512739554 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55873c4dc200 0x55873c4de5f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6ddc5a1183cab67e server_cookie=ef82ee59bbd47a2f in_seq=0 out_seq=0 2024-09-19T09:27:14.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.045+0000 7f50197fa640 10 client.23298 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:27:14.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.045+0000 7f50197fa640 1 -- 172.21.15.120:0/512739554 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f502001b660 con 0x55873c4dc200 2024-09-19T09:27:14.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.045+0000 7f50197fa640 10 client.23298 handle_command_reply: tid=0 2024-09-19T09:27:14.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.045+0000 7f503a35e740 1 client.23298 shutdown 2024-09-19T09:27:14.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.045+0000 7f503a35e740 1 -- 172.21.15.120:0/512739554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55873c4565d0 msgr2=0x55873c45afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:14.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.045+0000 7f503a35e740 1 --2- 172.21.15.120:0/512739554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55873c4565d0 0x55873c45afb0 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f502c098320 tx=0x7f502c086040 comp rx=0 tx=0).stop 2024-09-19T09:27:14.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.045+0000 7f503a35e740 1 -- 172.21.15.120:0/512739554 shutdown_connections 2024-09-19T09:27:14.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.045+0000 7f503a35e740 1 --2- 172.21.15.120:0/512739554 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55873c4dc200 0x55873c4de5f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.045+0000 7f503a35e740 1 --2- 172.21.15.120:0/512739554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55873c461370 0x55873c45ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.045+0000 7f503a35e740 1 --2- 172.21.15.120:0/512739554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55873c457b40 0x55873c45b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.045+0000 7f50197fa640 0 client.23298 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:27:14.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.045+0000 7f503a35e740 1 --2- 172.21.15.120:0/512739554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55873c4565d0 0x55873c45afb0 unknown :-1 s=CLOSED pgs=2453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.045+0000 7f503a35e740 1 -- 172.21.15.120:0/512739554 >> 172.21.15.120:0/512739554 conn(0x55873c41f310 msgr2=0x55873c45f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:14.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.045+0000 7f503a35e740 1 -- 172.21.15.120:0/512739554 shutdown_connections 2024-09-19T09:27:14.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.045+0000 7f503a35e740 1 -- 172.21.15.120:0/512739554 wait complete. 2024-09-19T09:27:14.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.046+0000 7f503a35e740 20 client.23298 trim_cache size 0 max 16384 2024-09-19T09:27:14.065 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:27:14.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.046+0000 7f5037f85640 1 -- 172.21.15.120:0/3919634101 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f501c0a32c0 msgr2=0x7f501c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:14.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.046+0000 7f5037f85640 1 --2- 172.21.15.120:0/3919634101 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f501c0a32c0 0x7f501c0a5780 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f5028030040 tx=0x7f50280095c0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.046+0000 7f5037f85640 1 -- 172.21.15.120:0/3919634101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5030100e20 msgr2=0x7f5030199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:14.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.046+0000 7f5037f85640 1 --2- 172.21.15.120:0/3919634101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5030100e20 0x7f5030199370 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f502c00d950 tx=0x7f502c00de10 comp rx=0 tx=0).stop 2024-09-19T09:27:14.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.046+0000 7f5037f85640 1 -- 172.21.15.120:0/3919634101 shutdown_connections 2024-09-19T09:27:14.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.046+0000 7f5037f85640 1 --2- 172.21.15.120:0/3919634101 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f501c0a32c0 0x7f501c0a5780 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.046+0000 7f5037f85640 1 --2- 172.21.15.120:0/3919634101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5030102ef0 0x7f50301a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.046+0000 7f5037f85640 1 --2- 172.21.15.120:0/3919634101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5030102570 0x7f50301998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.046+0000 7f5037f85640 1 --2- 172.21.15.120:0/3919634101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5030100e20 0x7f5030199370 unknown :-1 s=CLOSED pgs=2449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.046+0000 7f5037f85640 1 -- 172.21.15.120:0/3919634101 >> 172.21.15.120:0/3919634101 conn(0x7f50300fc7d0 msgr2=0x7f5030106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:14.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.046+0000 7f5037f85640 1 -- 172.21.15.120:0/3919634101 shutdown_connections 2024-09-19T09:27:14.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:14.046+0000 7f5037f85640 1 -- 172.21.15.120:0/3919634101 wait complete. 2024-09-19T09:27:14.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:14 smithi120 ceph-mon[31150]: pgmap v1076: 97 pgs: 97 active+clean; 501 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 4.7 MiB/s wr, 617 op/s 2024-09-19T09:27:14.390 INFO:tasks.workunit.client.0.smithi120.stdout: 10 266983 20.76 MB/sec execute 549 sec latency 3373.094 ms 2024-09-19T09:27:14.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:14 smithi196 ceph-mon[35150]: pgmap v1076: 97 pgs: 97 active+clean; 501 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 4.7 MiB/s wr, 617 op/s 2024-09-19T09:27:14.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:14 smithi153 ceph-mon[35071]: pgmap v1076: 97 pgs: 97 active+clean; 501 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 4.7 MiB/s wr, 617 op/s 2024-09-19T09:27:15.011 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:27:15.301 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:15.390 INFO:tasks.workunit.client.0.smithi120.stdout: 10 266983 20.73 MB/sec execute 550 sec latency 4373.216 ms 2024-09-19T09:27:15.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.897+0000 7fe1c0862640 1 -- 172.21.15.120:0/2278186940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe1bc100a80 msgr2=0x7fe1bc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:15.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.897+0000 7fe1c0862640 1 --2- 172.21.15.120:0/2278186940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe1bc100a80 0x7fe1bc100ee0 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0009930 tx=0x7fe1b002a420 comp rx=0 tx=0).stop 2024-09-19T09:27:15.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.897+0000 7fe1c0862640 1 -- 172.21.15.120:0/2278186940 shutdown_connections 2024-09-19T09:27:15.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.897+0000 7fe1c0862640 1 --2- 172.21.15.120:0/2278186940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1bc101420 0x7fe1bc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.897+0000 7fe1c0862640 1 --2- 172.21.15.120:0/2278186940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe1bc100a80 0x7fe1bc100ee0 unknown :-1 s=CLOSED pgs=2454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.897+0000 7fe1c0862640 1 --2- 172.21.15.120:0/2278186940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1bc106a80 0x7fe1bc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.897+0000 7fe1c0862640 1 -- 172.21.15.120:0/2278186940 >> 172.21.15.120:0/2278186940 conn(0x7fe1bc0fc7a0 msgr2=0x7fe1bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:15.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.897+0000 7fe1c0862640 1 -- 172.21.15.120:0/2278186940 shutdown_connections 2024-09-19T09:27:15.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.897+0000 7fe1c0862640 1 -- 172.21.15.120:0/2278186940 wait complete. 2024-09-19T09:27:15.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.898+0000 7fe1c0862640 1 Processor -- start 2024-09-19T09:27:15.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.898+0000 7fe1c0862640 1 -- start start 2024-09-19T09:27:15.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.898+0000 7fe1c0862640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1bc100a80 0x7fe1bc199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:15.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.898+0000 7fe1c0862640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe1bc101420 0x7fe1bc199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:15.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.898+0000 7fe1c0862640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1bc106a80 0x7fe1bc1a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:15.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.898+0000 7fe1c0862640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc110e80 con 0x7fe1bc101420 2024-09-19T09:27:15.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.898+0000 7fe1c0862640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc110d00 con 0x7fe1bc100a80 2024-09-19T09:27:15.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.898+0000 7fe1c0862640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc111000 con 0x7fe1bc106a80 2024-09-19T09:27:15.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.899+0000 7fe1b9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe1bc101420 0x7fe1bc199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:15.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.899+0000 7fe1b9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe1bc101420 0x7fe1bc199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34910/0 (socket says 172.21.15.120:34910) 2024-09-19T09:27:15.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.899+0000 7fe1b9d9b640 1 -- 172.21.15.120:0/1013276652 learned_addr learned my addr 172.21.15.120:0/1013276652 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:15.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.899+0000 7fe1bad9d640 1 --2- 172.21.15.120:0/1013276652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1bc106a80 0x7fe1bc1a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:15.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.899+0000 7fe1ba59c640 1 --2- 172.21.15.120:0/1013276652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1bc100a80 0x7fe1bc199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:15.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.899+0000 7fe1b9d9b640 1 -- 172.21.15.120:0/1013276652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1bc106a80 msgr2=0x7fe1bc1a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:15.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.899+0000 7fe1b9d9b640 1 --2- 172.21.15.120:0/1013276652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1bc106a80 0x7fe1bc1a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.899+0000 7fe1b9d9b640 1 -- 172.21.15.120:0/1013276652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1bc100a80 msgr2=0x7fe1bc199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:15.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.899+0000 7fe1b9d9b640 1 --2- 172.21.15.120:0/1013276652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1bc100a80 0x7fe1bc199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.899+0000 7fe1b9d9b640 1 -- 172.21.15.120:0/1013276652 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1bc1a48d0 con 0x7fe1bc101420 2024-09-19T09:27:15.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.899+0000 7fe1b9d9b640 1 --2- 172.21.15.120:0/1013276652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe1bc101420 0x7fe1bc199850 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0009230 tx=0x7fe1b0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:15.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.899+0000 7fe1ab7fe640 1 -- 172.21.15.120:0/1013276652 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b003f030 con 0x7fe1bc101420 2024-09-19T09:27:15.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.899+0000 7fe1c0862640 1 -- 172.21.15.120:0/1013276652 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1bc1a4aa0 con 0x7fe1bc101420 2024-09-19T09:27:15.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.899+0000 7fe1ab7fe640 1 -- 172.21.15.120:0/1013276652 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1b0003ad0 con 0x7fe1bc101420 2024-09-19T09:27:15.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.899+0000 7fe1ab7fe640 1 -- 172.21.15.120:0/1013276652 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0035070 con 0x7fe1bc101420 2024-09-19T09:27:15.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.899+0000 7fe1c0862640 1 -- 172.21.15.120:0/1013276652 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1bc1a4f50 con 0x7fe1bc101420 2024-09-19T09:27:15.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.901+0000 7fe1ab7fe640 1 -- 172.21.15.120:0/1013276652 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe1b0042030 con 0x7fe1bc101420 2024-09-19T09:27:15.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.902+0000 7fe1c2c3b740 1 Processor -- start 2024-09-19T09:27:15.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.902+0000 7fe1c2c3b740 1 -- start start 2024-09-19T09:27:15.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.902+0000 7fe1c2c3b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56358408fb40 0x56358408e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:15.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.902+0000 7fe1c2c3b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56358408e5d0 0x56358408ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:15.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.902+0000 7fe1c2c3b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563584099370 0x56358409b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:15.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.902+0000 7fe1c2c3b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563583ff42f0 con 0x56358408fb40 2024-09-19T09:27:15.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.902+0000 7fe1c2c3b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563583f9bcf0 con 0x563584099370 2024-09-19T09:27:15.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.902+0000 7fe1c2c3b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563583ff02b0 con 0x56358408e5d0 2024-09-19T09:27:15.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.903+0000 7fe1bad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56358408fb40 0x56358408e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:15.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.903+0000 7fe1bad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56358408fb40 0x56358408e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34918/0 (socket says 172.21.15.120:34918) 2024-09-19T09:27:15.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.903+0000 7fe1bad9d640 1 -- 172.21.15.120:0/3117799854 learned_addr learned my addr 172.21.15.120:0/3117799854 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:15.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.903+0000 7fe1ab7fe640 1 --2- 172.21.15.120:0/1013276652 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe1a40a3240 0x7fe1a40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:15.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.903+0000 7fe1ba59c640 1 --2- 172.21.15.120:0/3117799854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56358408e5d0 0x56358408ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:15.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.903+0000 7fe1b9d9b640 1 --2- 172.21.15.120:0/3117799854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563584099370 0x56358409b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:15.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.903+0000 7fe1bad9d640 1 --2- 172.21.15.120:0/1013276652 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe1a40a3240 0x7fe1a40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:15.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.903+0000 7fe1ab7fe640 1 -- 172.21.15.120:0/1013276652 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe1b00efef0 con 0x7fe1bc101420 2024-09-19T09:27:15.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.903+0000 7fe1ba59c640 1 -- 172.21.15.120:0/3117799854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563584099370 msgr2=0x56358409b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:15.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.903+0000 7fe1ba59c640 1 --2- 172.21.15.120:0/3117799854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563584099370 0x56358409b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.903+0000 7fe1ba59c640 1 -- 172.21.15.120:0/3117799854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56358408fb40 msgr2=0x56358408e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:15.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.903+0000 7fe1ba59c640 1 --2- 172.21.15.120:0/3117799854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56358408fb40 0x56358408e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.903+0000 7fe1ba59c640 1 -- 172.21.15.120:0/3117799854 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563583f803a0 con 0x56358408e5d0 2024-09-19T09:27:15.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.903+0000 7fe1bad9d640 1 --2- 172.21.15.120:0/3117799854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56358408fb40 0x56358408e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:27:15.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.903+0000 7fe1ba59c640 1 --2- 172.21.15.120:0/3117799854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56358408e5d0 0x56358408ea10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac00bb60 tx=0x7fe1ac00f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7170b0b6750234dc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:15.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.903+0000 7fe1bad9d640 1 --2- 172.21.15.120:0/1013276652 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe1a40a3240 0x7fe1a40a5700 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4005c40 tx=0x7fe1b4004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:15.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.904+0000 7fe1a8ff9640 1 -- 172.21.15.120:0/3117799854 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac015070 con 0x56358408e5d0 2024-09-19T09:27:15.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.904+0000 7fe1a8ff9640 1 -- 172.21.15.120:0/3117799854 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1ac00be50 con 0x56358408e5d0 2024-09-19T09:27:15.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.904+0000 7fe1a8ff9640 1 -- 172.21.15.120:0/3117799854 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac011440 con 0x56358408e5d0 2024-09-19T09:27:15.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.904+0000 7fe1c2c3b740 1 -- 172.21.15.120:0/3117799854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56358408e5d0 msgr2=0x56358408ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:15.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.904+0000 7fe1c2c3b740 1 --2- 172.21.15.120:0/3117799854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56358408e5d0 0x56358408ea10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac00bb60 tx=0x7fe1ac00f730 comp rx=0 tx=0).stop 2024-09-19T09:27:15.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.904+0000 7fe1c2c3b740 1 -- 172.21.15.120:0/3117799854 shutdown_connections 2024-09-19T09:27:15.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.904+0000 7fe1c2c3b740 1 --2- 172.21.15.120:0/3117799854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563584099370 0x56358409b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.904+0000 7fe1c2c3b740 1 --2- 172.21.15.120:0/3117799854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56358408e5d0 0x56358408ea10 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.904+0000 7fe1c2c3b740 1 --2- 172.21.15.120:0/3117799854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56358408fb40 0x56358408e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.904+0000 7fe1c2c3b740 1 -- 172.21.15.120:0/3117799854 >> 172.21.15.120:0/3117799854 conn(0x563584057310 msgr2=0x56358408ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:15.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.904+0000 7fe1c2c3b740 1 -- 172.21.15.120:0/3117799854 shutdown_connections 2024-09-19T09:27:15.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.904+0000 7fe1c2c3b740 1 -- 172.21.15.120:0/3117799854 wait complete. 2024-09-19T09:27:15.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.905+0000 7fe1c2c3b740 1 -- start start 2024-09-19T09:27:15.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.905+0000 7fe1c2c3b740 1 Processor -- start 2024-09-19T09:27:15.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.905+0000 7fe1c2c3b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56358408e5d0 0x563584098d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:15.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.905+0000 7fe1c2c3b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56358408fb40 0x563584095fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:15.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.905+0000 7fe1c2c3b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563584099370 0x5635840965d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:15.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.905+0000 7fe1c2c3b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56358409c070 con 0x56358408fb40 2024-09-19T09:27:15.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.905+0000 7fe1c2c3b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563583faf060 con 0x56358408e5d0 2024-09-19T09:27:15.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.905+0000 7fe1c2c3b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56358409c1f0 con 0x563584099370 2024-09-19T09:27:15.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.905+0000 7fe1c2c3b740 10 client.0 _ll_register_callbacks cb 0x563583d028c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:15.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.905+0000 7fe1b9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56358408fb40 0x563584095fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:15.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.905+0000 7fe1b9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56358408fb40 0x563584095fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34920/0 (socket says 172.21.15.120:34920) 2024-09-19T09:27:15.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.905+0000 7fe1b9d9b640 1 -- 172.21.15.120:0/607186359 learned_addr learned my addr 172.21.15.120:0/607186359 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:15.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.905+0000 7fe1bad9d640 1 --2- 172.21.15.120:0/607186359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563584099370 0x5635840965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:15.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.905+0000 7fe1ba59c640 1 --2- 172.21.15.120:0/607186359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56358408e5d0 0x563584098d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:15.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.905+0000 7fe1b9d9b640 1 -- 172.21.15.120:0/607186359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563584099370 msgr2=0x5635840965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:15.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.905+0000 7fe1b9d9b640 1 --2- 172.21.15.120:0/607186359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563584099370 0x5635840965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.905+0000 7fe1b9d9b640 1 -- 172.21.15.120:0/607186359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56358408e5d0 msgr2=0x563584098d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:15.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.905+0000 7fe1b9d9b640 1 --2- 172.21.15.120:0/607186359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56358408e5d0 0x563584098d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.905+0000 7fe1b9d9b640 1 -- 172.21.15.120:0/607186359 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563583d653c0 con 0x56358408fb40 2024-09-19T09:27:15.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.906+0000 7fe1b9d9b640 1 --2- 172.21.15.120:0/607186359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56358408fb40 0x563584095fe0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00bc460 tx=0x7fe1b008e2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6acb7e6100b993c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:15.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.906+0000 7fe1a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:27:15.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.906+0000 7fe1a1ffb640 1 -- 172.21.15.120:0/607186359 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0049040 con 0x56358408fb40 2024-09-19T09:27:15.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.906+0000 7fe1a1ffb640 1 -- 172.21.15.120:0/607186359 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe1b0003c80 con 0x56358408fb40 2024-09-19T09:27:15.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.906+0000 7fe1c2c3b740 1 -- 172.21.15.120:0/607186359 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1ac00be50 con 0x56358408fb40 2024-09-19T09:27:15.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.906+0000 7fe1a1ffb640 1 -- 172.21.15.120:0/607186359 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe1b00bbe30 con 0x56358408fb40 2024-09-19T09:27:15.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.906+0000 7fe1c2c3b740 10 client.23310 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:15.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.906+0000 7fe1c2c3b740 1 -- 172.21.15.120:0/607186359 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5635840a7b00 con 0x56358408fb40 2024-09-19T09:27:15.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.906+0000 7fe1a1ffb640 1 -- 172.21.15.120:0/607186359 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1ac00be50 con 0x56358408fb40 2024-09-19T09:27:15.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.906+0000 7fe1a1ffb640 1 -- 172.21.15.120:0/607186359 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fe1b00b8b40 con 0x56358408fb40 2024-09-19T09:27:15.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.906+0000 7fe1c2c3b740 10 client.23310 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:15.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.906+0000 7fe1c2c3b740 10 client.23310 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:27:15.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.906+0000 7fe1c2c3b740 20 client.23310 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:15.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.906+0000 7fe1c2c3b740 1 --2- 172.21.15.120:0/607186359 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5635840a9000 0x5635840ab450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:15.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.906+0000 7fe1c2c3b740 4 client.23310 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:15.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.906+0000 7fe1c2c3b740 1 -- 172.21.15.120:0/607186359 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563584001860 con 0x5635840a9000 2024-09-19T09:27:15.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.907+0000 7fe1ba59c640 1 --2- 172.21.15.120:0/607186359 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5635840a9000 0x5635840ab450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:15.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.907+0000 7fe1a1ffb640 10 client.23310 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:27:15.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.907+0000 7fe1ba59c640 1 --2- 172.21.15.120:0/607186359 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5635840a9000 0x5635840ab450 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e93865a1c652c68b server_cookie=d2526e27fa35b7ca in_seq=0 out_seq=0 2024-09-19T09:27:15.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.908+0000 7fe1a1ffb640 1 -- 172.21.15.120:0/607186359 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563584001860 con 0x5635840a9000 2024-09-19T09:27:15.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.908+0000 7fe1a1ffb640 10 client.23310 handle_command_reply: tid=0 2024-09-19T09:27:15.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.908+0000 7fe1c2c3b740 1 client.23310 shutdown 2024-09-19T09:27:15.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.908+0000 7fe1c2c3b740 1 -- 172.21.15.120:0/607186359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56358408fb40 msgr2=0x563584095fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:15.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.908+0000 7fe1c2c3b740 1 --2- 172.21.15.120:0/607186359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56358408fb40 0x563584095fe0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00bc460 tx=0x7fe1b008e2a0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.908+0000 7fe1c2c3b740 1 -- 172.21.15.120:0/607186359 shutdown_connections 2024-09-19T09:27:15.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.908+0000 7fe1c2c3b740 1 --2- 172.21.15.120:0/607186359 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x5635840a9000 0x5635840ab450 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.908+0000 7fe1c2c3b740 1 --2- 172.21.15.120:0/607186359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563584099370 0x5635840965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.908+0000 7fe1a1ffb640 0 client.23310 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:27:15.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.908+0000 7fe1c2c3b740 1 --2- 172.21.15.120:0/607186359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56358408fb40 0x563584095fe0 unknown :-1 s=CLOSED pgs=2456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.908+0000 7fe1c2c3b740 1 --2- 172.21.15.120:0/607186359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56358408e5d0 0x563584098d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.908+0000 7fe1c2c3b740 1 -- 172.21.15.120:0/607186359 >> 172.21.15.120:0/607186359 conn(0x563584057310 msgr2=0x563584090930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:15.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.908+0000 7fe1c2c3b740 1 -- 172.21.15.120:0/607186359 shutdown_connections 2024-09-19T09:27:15.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.908+0000 7fe1c2c3b740 1 -- 172.21.15.120:0/607186359 wait complete. 2024-09-19T09:27:15.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.908+0000 7fe1c2c3b740 20 client.23310 trim_cache size 0 max 16384 2024-09-19T09:27:15.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.921+0000 7fe1c2c3b740 1 Processor -- start 2024-09-19T09:27:15.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.921+0000 7fe1c2c3b740 1 -- start start 2024-09-19T09:27:15.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.921+0000 7fe1c2c3b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56358408e5d0 0x5635840a5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:15.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.921+0000 7fe1c2c3b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56358408fb40 0x56358408f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:15.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.921+0000 7fe1c2c3b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563584099370 0x563584091120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:15.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.921+0000 7fe1c2c3b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc1ad5c0 con 0x56358408fb40 2024-09-19T09:27:15.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.921+0000 7fe1c2c3b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc1af100 con 0x563584099370 2024-09-19T09:27:15.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.921+0000 7fe1c2c3b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc1adc90 con 0x56358408e5d0 2024-09-19T09:27:15.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.921+0000 7fe1b9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56358408fb40 0x56358408f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:15.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.921+0000 7fe1b9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56358408fb40 0x56358408f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34922/0 (socket says 172.21.15.120:34922) 2024-09-19T09:27:15.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.921+0000 7fe1b9d9b640 1 -- 172.21.15.120:0/4060874787 learned_addr learned my addr 172.21.15.120:0/4060874787 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:15.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.921+0000 7fe1ba59c640 1 --2- 172.21.15.120:0/4060874787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56358408e5d0 0x5635840a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:15.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.921+0000 7fe1b9d9b640 1 -- 172.21.15.120:0/4060874787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56358408e5d0 msgr2=0x5635840a5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:15.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.921+0000 7fe1b9d9b640 1 --2- 172.21.15.120:0/4060874787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56358408e5d0 0x5635840a5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.921+0000 7fe1b9d9b640 1 -- 172.21.15.120:0/4060874787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563584099370 msgr2=0x563584091120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:15.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.922+0000 7fe1b9d9b640 1 --2- 172.21.15.120:0/4060874787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563584099370 0x563584091120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.922+0000 7fe1b9d9b640 1 -- 172.21.15.120:0/4060874787 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563584094550 con 0x56358408fb40 2024-09-19T09:27:15.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.922+0000 7fe1b9d9b640 1 --2- 172.21.15.120:0/4060874787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56358408fb40 0x56358408f300 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00c5290 tx=0x7fe1b0005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f955d94701162a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:15.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.922+0000 7fe1a3fff640 1 -- 172.21.15.120:0/4060874787 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0049040 con 0x56358408fb40 2024-09-19T09:27:15.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.922+0000 7fe1a3fff640 1 -- 172.21.15.120:0/4060874787 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1b00bcbd0 con 0x56358408fb40 2024-09-19T09:27:15.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.922+0000 7fe1a3fff640 1 -- 172.21.15.120:0/4060874787 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0003c80 con 0x56358408fb40 2024-09-19T09:27:15.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.922+0000 7fe1c2c3b740 1 -- 172.21.15.120:0/4060874787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56358408fb40 msgr2=0x56358408f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:15.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.922+0000 7fe1c2c3b740 1 --2- 172.21.15.120:0/4060874787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56358408fb40 0x56358408f300 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00c5290 tx=0x7fe1b0005e50 comp rx=0 tx=0).stop 2024-09-19T09:27:15.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.922+0000 7fe1c2c3b740 1 -- 172.21.15.120:0/4060874787 shutdown_connections 2024-09-19T09:27:15.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.922+0000 7fe1c2c3b740 1 --2- 172.21.15.120:0/4060874787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563584099370 0x563584091120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.922+0000 7fe1c2c3b740 1 --2- 172.21.15.120:0/4060874787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56358408fb40 0x56358408f300 unknown :-1 s=CLOSED pgs=2457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.922+0000 7fe1c2c3b740 1 --2- 172.21.15.120:0/4060874787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56358408e5d0 0x5635840a5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.922+0000 7fe1c2c3b740 1 -- 172.21.15.120:0/4060874787 >> 172.21.15.120:0/4060874787 conn(0x563584057310 msgr2=0x563584085a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:15.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.922+0000 7fe1c2c3b740 1 -- 172.21.15.120:0/4060874787 shutdown_connections 2024-09-19T09:27:15.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.922+0000 7fe1c2c3b740 1 -- 172.21.15.120:0/4060874787 wait complete. 2024-09-19T09:27:15.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.923+0000 7fe1c2c3b740 1 -- start start 2024-09-19T09:27:15.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.923+0000 7fe1c2c3b740 1 Processor -- start 2024-09-19T09:27:15.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.923+0000 7fe1c2c3b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56358408e5d0 0x5635840a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:15.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.923+0000 7fe1c2c3b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56358408fb40 0x563584092b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:15.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.923+0000 7fe1c2c3b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563584099370 0x5635840930d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:15.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.923+0000 7fe1c2c3b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc1aea30 con 0x563584099370 2024-09-19T09:27:15.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.923+0000 7fe1c2c3b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc1ae360 con 0x56358408e5d0 2024-09-19T09:27:15.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.923+0000 7fe1c2c3b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc1ab580 con 0x56358408fb40 2024-09-19T09:27:15.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.923+0000 7fe1c2c3b740 10 client.0 _ll_register_callbacks cb 0x7fe1b0009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:15.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.923+0000 7fe1bad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563584099370 0x5635840930d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:15.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.923+0000 7fe1bad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563584099370 0x5635840930d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34926/0 (socket says 172.21.15.120:34926) 2024-09-19T09:27:15.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.923+0000 7fe1bad9d640 1 -- 172.21.15.120:0/3870625912 learned_addr learned my addr 172.21.15.120:0/3870625912 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:15.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.923+0000 7fe1b9d9b640 1 --2- 172.21.15.120:0/3870625912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56358408fb40 0x563584092b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:15.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.924+0000 7fe1bad9d640 1 -- 172.21.15.120:0/3870625912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56358408fb40 msgr2=0x563584092b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:15.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.924+0000 7fe1bad9d640 1 --2- 172.21.15.120:0/3870625912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56358408fb40 0x563584092b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.924+0000 7fe1bad9d640 1 -- 172.21.15.120:0/3870625912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56358408e5d0 msgr2=0x5635840a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:15.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.924+0000 7fe1ba59c640 1 --2- 172.21.15.120:0/3870625912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56358408e5d0 0x5635840a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:15.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.924+0000 7fe1bad9d640 1 --2- 172.21.15.120:0/3870625912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56358408e5d0 0x5635840a19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.924+0000 7fe1bad9d640 1 -- 172.21.15.120:0/3870625912 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563584093690 con 0x563584099370 2024-09-19T09:27:15.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.924+0000 7fe1b9d9b640 1 --2- 172.21.15.120:0/3870625912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56358408fb40 0x563584092b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:15.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.924+0000 7fe1ba59c640 1 --2- 172.21.15.120:0/3870625912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56358408e5d0 0x5635840a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:15.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.924+0000 7fe1bad9d640 1 --2- 172.21.15.120:0/3870625912 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563584099370 0x5635840930d0 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4009e30 tx=0x7fe1b4009a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaeb9e9c7a48ab40 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:15.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.924+0000 7fe1a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:27:15.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.924+0000 7fe1a1ffb640 1 -- 172.21.15.120:0/3870625912 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4027070 con 0x563584099370 2024-09-19T09:27:15.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.924+0000 7fe1a1ffb640 1 -- 172.21.15.120:0/3870625912 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe1b4010e30 con 0x563584099370 2024-09-19T09:27:15.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.924+0000 7fe1c2c3b740 1 -- 172.21.15.120:0/3870625912 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1b00bcbd0 con 0x563584099370 2024-09-19T09:27:15.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.924+0000 7fe1a1ffb640 1 -- 172.21.15.120:0/3870625912 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe1b4004b20 con 0x563584099370 2024-09-19T09:27:15.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.924+0000 7fe1c2c3b740 10 client.23316 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:15.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.924+0000 7fe1c2c3b740 1 -- 172.21.15.120:0/3870625912 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563584113ac0 con 0x563584099370 2024-09-19T09:27:15.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.924+0000 7fe1a1ffb640 1 -- 172.21.15.120:0/3870625912 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1b00bcbd0 con 0x563584099370 2024-09-19T09:27:15.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.924+0000 7fe1a1ffb640 1 -- 172.21.15.120:0/3870625912 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fe1b400f040 con 0x563584099370 2024-09-19T09:27:15.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.925+0000 7fe1c2c3b740 10 client.23316 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:15.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.925+0000 7fe1c2c3b740 10 client.23316 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:27:15.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.925+0000 7fe1c2c3b740 20 client.23316 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:15.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.925+0000 7fe1c2c3b740 1 --2- 172.21.15.120:0/3870625912 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x563584114210 0x563584116630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:15.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.925+0000 7fe1c2c3b740 4 client.23316 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:27:15.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.925+0000 7fe1c2c3b740 1 -- 172.21.15.120:0/3870625912 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563583ffbd70 con 0x563584114210 2024-09-19T09:27:15.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.925+0000 7fe1ba59c640 1 --2- 172.21.15.120:0/3870625912 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x563584114210 0x563584116630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:15.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.925+0000 7fe1a1ffb640 10 client.23316 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:27:15.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.925+0000 7fe1ba59c640 1 --2- 172.21.15.120:0/3870625912 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x563584114210 0x563584116630 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8fb45459514c8d52 server_cookie=eafc409d81ff0933 in_seq=0 out_seq=0 2024-09-19T09:27:15.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.925+0000 7fe1a1ffb640 1 -- 172.21.15.120:0/3870625912 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563583ffbd70 con 0x563584114210 2024-09-19T09:27:15.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.925+0000 7fe1a1ffb640 10 client.23316 handle_command_reply: tid=0 2024-09-19T09:27:15.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.925+0000 7fe1c2c3b740 1 client.23316 shutdown 2024-09-19T09:27:15.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.925+0000 7fe1c2c3b740 1 -- 172.21.15.120:0/3870625912 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563584099370 msgr2=0x5635840930d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:15.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.926+0000 7fe1c2c3b740 1 --2- 172.21.15.120:0/3870625912 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563584099370 0x5635840930d0 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4009e30 tx=0x7fe1b4009a60 comp rx=0 tx=0).stop 2024-09-19T09:27:15.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.926+0000 7fe1c2c3b740 1 -- 172.21.15.120:0/3870625912 shutdown_connections 2024-09-19T09:27:15.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.926+0000 7fe1c2c3b740 1 --2- 172.21.15.120:0/3870625912 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x563584114210 0x563584116630 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.957 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:27:15.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.926+0000 7fe1c2c3b740 1 --2- 172.21.15.120:0/3870625912 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563584099370 0x5635840930d0 unknown :-1 s=CLOSED pgs=2458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.926+0000 7fe1a1ffb640 0 client.23316 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:27:15.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.926+0000 7fe1c2c3b740 1 --2- 172.21.15.120:0/3870625912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56358408fb40 0x563584092b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.926+0000 7fe1c2c3b740 1 --2- 172.21.15.120:0/3870625912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56358408e5d0 0x5635840a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.926+0000 7fe1c2c3b740 1 -- 172.21.15.120:0/3870625912 >> 172.21.15.120:0/3870625912 conn(0x563584057310 msgr2=0x563584098060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:15.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.926+0000 7fe1c2c3b740 1 -- 172.21.15.120:0/3870625912 shutdown_connections 2024-09-19T09:27:15.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.926+0000 7fe1c2c3b740 1 -- 172.21.15.120:0/3870625912 wait complete. 2024-09-19T09:27:15.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.926+0000 7fe1c2c3b740 20 client.23316 trim_cache size 0 max 16384 2024-09-19T09:27:15.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.926+0000 7fe1c0862640 1 -- 172.21.15.120:0/1013276652 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe1a40a3240 msgr2=0x7fe1a40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:15.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.926+0000 7fe1c0862640 1 --2- 172.21.15.120:0/1013276652 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe1a40a3240 0x7fe1a40a5700 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4005c40 tx=0x7fe1b4004ab0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.926+0000 7fe1c0862640 1 -- 172.21.15.120:0/1013276652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe1bc101420 msgr2=0x7fe1bc199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:15.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.926+0000 7fe1c0862640 1 --2- 172.21.15.120:0/1013276652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe1bc101420 0x7fe1bc199850 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0009230 tx=0x7fe1b0004270 comp rx=0 tx=0).stop 2024-09-19T09:27:15.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.926+0000 7fe1c0862640 1 -- 172.21.15.120:0/1013276652 shutdown_connections 2024-09-19T09:27:15.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.926+0000 7fe1c0862640 1 --2- 172.21.15.120:0/1013276652 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe1a40a3240 0x7fe1a40a5700 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.926+0000 7fe1c0862640 1 --2- 172.21.15.120:0/1013276652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1bc106a80 0x7fe1bc1a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.926+0000 7fe1c0862640 1 --2- 172.21.15.120:0/1013276652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe1bc101420 0x7fe1bc199850 unknown :-1 s=CLOSED pgs=2455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.926+0000 7fe1c0862640 1 --2- 172.21.15.120:0/1013276652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1bc100a80 0x7fe1bc199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.926+0000 7fe1c0862640 1 -- 172.21.15.120:0/1013276652 >> 172.21.15.120:0/1013276652 conn(0x7fe1bc0fc7a0 msgr2=0x7fe1bc0fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:15.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.926+0000 7fe1c0862640 1 -- 172.21.15.120:0/1013276652 shutdown_connections 2024-09-19T09:27:15.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:15.926+0000 7fe1c0862640 1 -- 172.21.15.120:0/1013276652 wait complete. 2024-09-19T09:27:16.390 INFO:tasks.workunit.client.0.smithi120.stdout: 10 266983 20.69 MB/sec execute 551 sec latency 5373.305 ms 2024-09-19T09:27:16.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:16 smithi196 ceph-mon[35150]: pgmap v1077: 97 pgs: 97 active+clean; 499 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.0 MiB/s wr, 497 op/s 2024-09-19T09:27:16.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:16 smithi153 ceph-mon[35071]: pgmap v1077: 97 pgs: 97 active+clean; 499 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.0 MiB/s wr, 497 op/s 2024-09-19T09:27:16.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:16 smithi120 ceph-mon[31150]: pgmap v1077: 97 pgs: 97 active+clean; 499 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.0 MiB/s wr, 497 op/s 2024-09-19T09:27:17.032 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:27:17.280 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:17.390 INFO:tasks.workunit.client.0.smithi120.stdout: 10 266983 20.65 MB/sec execute 552 sec latency 6373.412 ms 2024-09-19T09:27:18.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.808+0000 7ff5237ea640 1 -- 172.21.15.120:0/312359354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff51c102790 msgr2=0x7ff51c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:18.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.808+0000 7ff5237ea640 1 --2- 172.21.15.120:0/312359354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff51c102790 0x7ff51c1067e0 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009900 tx=0x7ff50c02a410 comp rx=0 tx=0).stop 2024-09-19T09:27:18.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.808+0000 7ff5237ea640 1 -- 172.21.15.120:0/312359354 shutdown_connections 2024-09-19T09:27:18.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.808+0000 7ff5237ea640 1 --2- 172.21.15.120:0/312359354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff51c106f10 0x7ff51c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.808+0000 7ff5237ea640 1 --2- 172.21.15.120:0/312359354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff51c102790 0x7ff51c1067e0 unknown :-1 s=CLOSED pgs=2459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.808+0000 7ff5237ea640 1 --2- 172.21.15.120:0/312359354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff51c101de0 0x7ff51c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.808+0000 7ff5237ea640 1 -- 172.21.15.120:0/312359354 >> 172.21.15.120:0/312359354 conn(0x7ff51c0fd8c0 msgr2=0x7ff51c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:18.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.808+0000 7ff5237ea640 1 -- 172.21.15.120:0/312359354 shutdown_connections 2024-09-19T09:27:18.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.808+0000 7ff5237ea640 1 -- 172.21.15.120:0/312359354 wait complete. 2024-09-19T09:27:18.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.808+0000 7ff5237ea640 1 Processor -- start 2024-09-19T09:27:18.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.808+0000 7ff5237ea640 1 -- start start 2024-09-19T09:27:18.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.809+0000 7ff5237ea640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff51c101de0 0x7ff51c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:18.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.809+0000 7ff5237ea640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff51c102790 0x7ff51c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:18.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.809+0000 7ff5237ea640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff51c106f10 0x7ff51c19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:18.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.809+0000 7ff5237ea640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c111f70 con 0x7ff51c101de0 2024-09-19T09:27:18.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.809+0000 7ff5237ea640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c111df0 con 0x7ff51c106f10 2024-09-19T09:27:18.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.809+0000 7ff5237ea640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c1120f0 con 0x7ff51c102790 2024-09-19T09:27:18.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.809+0000 7ff521586640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff51c101de0 0x7ff51c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:18.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.809+0000 7ff521586640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff51c101de0 0x7ff51c1984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34946/0 (socket says 172.21.15.120:34946) 2024-09-19T09:27:18.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.809+0000 7ff521586640 1 -- 172.21.15.120:0/3739473268 learned_addr learned my addr 172.21.15.120:0/3739473268 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:18.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.809+0000 7ff520d85640 1 --2- 172.21.15.120:0/3739473268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff51c102790 0x7ff51c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:18.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.809+0000 7ff521d87640 1 --2- 172.21.15.120:0/3739473268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff51c106f10 0x7ff51c19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:18.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.809+0000 7ff521586640 1 -- 172.21.15.120:0/3739473268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff51c102790 msgr2=0x7ff51c1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:18.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.809+0000 7ff521586640 1 --2- 172.21.15.120:0/3739473268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff51c102790 0x7ff51c1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.809+0000 7ff521586640 1 -- 172.21.15.120:0/3739473268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff51c106f10 msgr2=0x7ff51c19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:18.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.809+0000 7ff521586640 1 --2- 172.21.15.120:0/3739473268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff51c106f10 0x7ff51c19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.809+0000 7ff521586640 1 -- 172.21.15.120:0/3739473268 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff51c19d460 con 0x7ff51c101de0 2024-09-19T09:27:18.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.809+0000 7ff521d87640 1 --2- 172.21.15.120:0/3739473268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff51c106f10 0x7ff51c19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:18.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.809+0000 7ff521586640 1 --2- 172.21.15.120:0/3739473268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff51c101de0 0x7ff51c1984b0 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7ff51800d770 tx=0x7ff51800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:18.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.809+0000 7ff5127fc640 1 -- 172.21.15.120:0/3739473268 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518017070 con 0x7ff51c101de0 2024-09-19T09:27:18.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.809+0000 7ff5127fc640 1 -- 172.21.15.120:0/3739473268 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff518005500 con 0x7ff51c101de0 2024-09-19T09:27:18.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.809+0000 7ff5237ea640 1 -- 172.21.15.120:0/3739473268 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff51c19d750 con 0x7ff51c101de0 2024-09-19T09:27:18.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.809+0000 7ff5127fc640 1 -- 172.21.15.120:0/3739473268 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518004e00 con 0x7ff51c101de0 2024-09-19T09:27:18.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.810+0000 7ff5237ea640 1 -- 172.21.15.120:0/3739473268 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff51c1a5030 con 0x7ff51c101de0 2024-09-19T09:27:18.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.812+0000 7ff5127fc640 1 -- 172.21.15.120:0/3739473268 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7ff518004fa0 con 0x7ff51c101de0 2024-09-19T09:27:18.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.812+0000 7ff525bc3740 1 Processor -- start 2024-09-19T09:27:18.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.812+0000 7ff525bc3740 1 -- start start 2024-09-19T09:27:18.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.812+0000 7ff525bc3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ac2221b40 0x557ac2220090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:18.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.812+0000 7ff525bc3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ac22205d0 0x557ac2220a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:18.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.812+0000 7ff525bc3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ac222f780 0x557ac2231b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:18.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.812+0000 7ff525bc3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557ac21862f0 con 0x557ac222f780 2024-09-19T09:27:18.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.812+0000 7ff525bc3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557ac212dcf0 con 0x557ac2221b40 2024-09-19T09:27:18.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.812+0000 7ff525bc3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557ac21822b0 con 0x557ac22205d0 2024-09-19T09:27:18.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.812+0000 7ff5127fc640 1 --2- 172.21.15.120:0/3739473268 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff5080a32c0 0x7ff5080a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:18.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.812+0000 7ff520d85640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ac222f780 0x557ac2231b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:18.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.812+0000 7ff520d85640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ac222f780 0x557ac2231b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34962/0 (socket says 172.21.15.120:34962) 2024-09-19T09:27:18.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.812+0000 7ff520d85640 1 -- 172.21.15.120:0/76098795 learned_addr learned my addr 172.21.15.120:0/76098795 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:18.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.813+0000 7ff521586640 1 --2- 172.21.15.120:0/76098795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ac22205d0 0x557ac2220a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:18.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.813+0000 7ff521d87640 1 --2- 172.21.15.120:0/3739473268 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff5080a32c0 0x7ff5080a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:18.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.813+0000 7ff521d87640 1 --2- 172.21.15.120:0/76098795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ac2221b40 0x557ac2220090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:18.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.813+0000 7ff5127fc640 1 -- 172.21.15.120:0/3739473268 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff5180cf390 con 0x7ff51c101de0 2024-09-19T09:27:18.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.813+0000 7ff520d85640 1 -- 172.21.15.120:0/76098795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ac22205d0 msgr2=0x557ac2220a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:18.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.813+0000 7ff520d85640 1 --2- 172.21.15.120:0/76098795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ac22205d0 0x557ac2220a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.813+0000 7ff520d85640 1 -- 172.21.15.120:0/76098795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ac2221b40 msgr2=0x557ac2220090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:18.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.813+0000 7ff520d85640 1 --2- 172.21.15.120:0/76098795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ac2221b40 0x557ac2220090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.813+0000 7ff520d85640 1 -- 172.21.15.120:0/76098795 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ac21123a0 con 0x557ac222f780 2024-09-19T09:27:18.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.813+0000 7ff521d87640 1 --2- 172.21.15.120:0/76098795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ac2221b40 0x557ac2220090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:18.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.813+0000 7ff520d85640 1 --2- 172.21.15.120:0/76098795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ac222f780 0x557ac2231b70 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7ff50c02a930 tx=0x7ff50c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c05f277c4d5edd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:18.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.813+0000 7ff521d87640 1 --2- 172.21.15.120:0/3739473268 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff5080a32c0 0x7ff5080a5780 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff514005c40 tx=0x7ff5140094a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:18.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.813+0000 7ff5077fe640 1 -- 172.21.15.120:0/76098795 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c03f070 con 0x557ac222f780 2024-09-19T09:27:18.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.813+0000 7ff5077fe640 1 -- 172.21.15.120:0/76098795 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff50c004030 con 0x557ac222f780 2024-09-19T09:27:18.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.813+0000 7ff5077fe640 1 -- 172.21.15.120:0/76098795 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c035070 con 0x557ac222f780 2024-09-19T09:27:18.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.814+0000 7ff525bc3740 1 -- 172.21.15.120:0/76098795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ac222f780 msgr2=0x557ac2231b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:18.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.814+0000 7ff525bc3740 1 --2- 172.21.15.120:0/76098795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ac222f780 0x557ac2231b70 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7ff50c02a930 tx=0x7ff50c005e80 comp rx=0 tx=0).stop 2024-09-19T09:27:18.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.814+0000 7ff525bc3740 1 -- 172.21.15.120:0/76098795 shutdown_connections 2024-09-19T09:27:18.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.814+0000 7ff525bc3740 1 --2- 172.21.15.120:0/76098795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ac222f780 0x557ac2231b70 unknown :-1 s=CLOSED pgs=2461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.814+0000 7ff525bc3740 1 --2- 172.21.15.120:0/76098795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ac22205d0 0x557ac2220a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.814+0000 7ff525bc3740 1 --2- 172.21.15.120:0/76098795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ac2221b40 0x557ac2220090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.814+0000 7ff525bc3740 1 -- 172.21.15.120:0/76098795 >> 172.21.15.120:0/76098795 conn(0x557ac21e9310 msgr2=0x557ac221cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:18.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.814+0000 7ff525bc3740 1 -- 172.21.15.120:0/76098795 shutdown_connections 2024-09-19T09:27:18.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.814+0000 7ff525bc3740 1 -- 172.21.15.120:0/76098795 wait complete. 2024-09-19T09:27:18.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.814+0000 7ff525bc3740 1 -- start start 2024-09-19T09:27:18.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.815+0000 7ff525bc3740 1 Processor -- start 2024-09-19T09:27:18.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.815+0000 7ff525bc3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ac22205d0 0x557ac222f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:18.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.815+0000 7ff525bc3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ac2221b40 0x557ac222c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:18.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.815+0000 7ff525bc3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ac222f780 0x557ac222c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:18.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.815+0000 7ff525bc3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557ac2232420 con 0x557ac222f780 2024-09-19T09:27:18.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.815+0000 7ff525bc3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557ac2141060 con 0x557ac2221b40 2024-09-19T09:27:18.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.815+0000 7ff525bc3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557ac22325a0 con 0x557ac22205d0 2024-09-19T09:27:18.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.815+0000 7ff525bc3740 10 client.0 _ll_register_callbacks cb 0x557ac1e948c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:18.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.815+0000 7ff521d87640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ac222f780 0x557ac222c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:18.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.815+0000 7ff521d87640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ac222f780 0x557ac222c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34970/0 (socket says 172.21.15.120:34970) 2024-09-19T09:27:18.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.815+0000 7ff521d87640 1 -- 172.21.15.120:0/4254158353 learned_addr learned my addr 172.21.15.120:0/4254158353 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:18.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.815+0000 7ff521586640 1 --2- 172.21.15.120:0/4254158353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ac22205d0 0x557ac222f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:18.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.815+0000 7ff521d87640 1 -- 172.21.15.120:0/4254158353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ac22205d0 msgr2=0x557ac222f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:18.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.815+0000 7ff521d87640 1 --2- 172.21.15.120:0/4254158353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ac22205d0 0x557ac222f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.815+0000 7ff521d87640 1 -- 172.21.15.120:0/4254158353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ac2221b40 msgr2=0x557ac222c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:18.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.815+0000 7ff521d87640 1 --2- 172.21.15.120:0/4254158353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ac2221b40 0x557ac222c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.815+0000 7ff521d87640 1 -- 172.21.15.120:0/4254158353 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ac1ef73c0 con 0x557ac222f780 2024-09-19T09:27:18.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.816+0000 7ff521d87640 1 --2- 172.21.15.120:0/4254158353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ac222f780 0x557ac222c9e0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7ff51400faa0 tx=0x7ff514004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca095edd327832f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:18.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.816+0000 7ff504ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:27:18.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.816+0000 7ff504ff9640 1 -- 172.21.15.120:0/4254158353 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff514024030 con 0x557ac222f780 2024-09-19T09:27:18.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.816+0000 7ff504ff9640 1 -- 172.21.15.120:0/4254158353 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff514014e10 con 0x557ac222f780 2024-09-19T09:27:18.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.816+0000 7ff525bc3740 1 -- 172.21.15.120:0/4254158353 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff50c004030 con 0x557ac222f780 2024-09-19T09:27:18.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.816+0000 7ff504ff9640 1 -- 172.21.15.120:0/4254158353 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff514030050 con 0x557ac222f780 2024-09-19T09:27:18.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.816+0000 7ff525bc3740 10 client.23328 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:18.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.816+0000 7ff525bc3740 1 -- 172.21.15.120:0/4254158353 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557ac223df30 con 0x557ac222f780 2024-09-19T09:27:18.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.816+0000 7ff504ff9640 1 -- 172.21.15.120:0/4254158353 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff50c004030 con 0x557ac222f780 2024-09-19T09:27:18.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.816+0000 7ff504ff9640 1 -- 172.21.15.120:0/4254158353 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7ff51401d050 con 0x557ac222f780 2024-09-19T09:27:18.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.816+0000 7ff525bc3740 10 client.23328 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:18.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.816+0000 7ff525bc3740 10 client.23328 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:27:18.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.816+0000 7ff525bc3740 20 client.23328 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:18.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.816+0000 7ff525bc3740 1 --2- 172.21.15.120:0/4254158353 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557ac223f400 0x557ac22418c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:18.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.816+0000 7ff525bc3740 4 client.23328 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:18.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.816+0000 7ff525bc3740 1 -- 172.21.15.120:0/4254158353 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ac2193860 con 0x557ac223f400 2024-09-19T09:27:18.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.817+0000 7ff521586640 1 --2- 172.21.15.120:0/4254158353 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557ac223f400 0x557ac22418c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:18.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.817+0000 7ff521586640 1 --2- 172.21.15.120:0/4254158353 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557ac223f400 0x557ac22418c0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=20ffdd6d78456da7 server_cookie=6847c30edd9080e6 in_seq=0 out_seq=0 2024-09-19T09:27:18.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.817+0000 7ff504ff9640 10 client.23328 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:27:18.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.818+0000 7ff504ff9640 1 -- 172.21.15.120:0/4254158353 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557ac2193860 con 0x557ac223f400 2024-09-19T09:27:18.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.818+0000 7ff504ff9640 10 client.23328 handle_command_reply: tid=0 2024-09-19T09:27:18.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.818+0000 7ff525bc3740 1 client.23328 shutdown 2024-09-19T09:27:18.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.818+0000 7ff525bc3740 1 -- 172.21.15.120:0/4254158353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ac222f780 msgr2=0x557ac222c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:18.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.818+0000 7ff525bc3740 1 --2- 172.21.15.120:0/4254158353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ac222f780 0x557ac222c9e0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7ff51400faa0 tx=0x7ff514004ab0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.818+0000 7ff525bc3740 1 -- 172.21.15.120:0/4254158353 shutdown_connections 2024-09-19T09:27:18.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.818+0000 7ff525bc3740 1 --2- 172.21.15.120:0/4254158353 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557ac223f400 0x557ac22418c0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.818+0000 7ff525bc3740 1 --2- 172.21.15.120:0/4254158353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ac222f780 0x557ac222c9e0 unknown :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.818+0000 7ff525bc3740 1 --2- 172.21.15.120:0/4254158353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ac2221b40 0x557ac222c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.818+0000 7ff504ff9640 0 client.23328 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:27:18.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.818+0000 7ff525bc3740 1 --2- 172.21.15.120:0/4254158353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ac22205d0 0x557ac222f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.818+0000 7ff525bc3740 1 -- 172.21.15.120:0/4254158353 >> 172.21.15.120:0/4254158353 conn(0x557ac21e9310 msgr2=0x557ac2222960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:18.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.818+0000 7ff525bc3740 1 -- 172.21.15.120:0/4254158353 shutdown_connections 2024-09-19T09:27:18.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.818+0000 7ff525bc3740 1 -- 172.21.15.120:0/4254158353 wait complete. 2024-09-19T09:27:18.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.818+0000 7ff525bc3740 20 client.23328 trim_cache size 0 max 16384 2024-09-19T09:27:18.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.831+0000 7ff525bc3740 1 Processor -- start 2024-09-19T09:27:18.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.831+0000 7ff525bc3740 1 -- start start 2024-09-19T09:27:18.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.831+0000 7ff525bc3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ac22205d0 0x557ac223b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:18.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.831+0000 7ff525bc3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ac2221b40 0x557ac2230b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:18.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.831+0000 7ff525bc3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ac222f780 0x557ac22310b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:18.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.831+0000 7ff525bc3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c1ab2d0 con 0x557ac22205d0 2024-09-19T09:27:18.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.831+0000 7ff525bc3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c1ace10 con 0x557ac222f780 2024-09-19T09:27:18.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.831+0000 7ff525bc3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c1ab9a0 con 0x557ac2221b40 2024-09-19T09:27:18.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.831+0000 7ff521586640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ac22205d0 0x557ac223b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:18.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.831+0000 7ff521586640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ac22205d0 0x557ac223b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34978/0 (socket says 172.21.15.120:34978) 2024-09-19T09:27:18.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.831+0000 7ff521586640 1 -- 172.21.15.120:0/4123153654 learned_addr learned my addr 172.21.15.120:0/4123153654 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:18.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.831+0000 7ff521586640 1 -- 172.21.15.120:0/4123153654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ac2221b40 msgr2=0x557ac2230b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:18.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.831+0000 7ff520d85640 1 --2- 172.21.15.120:0/4123153654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ac2221b40 0x557ac2230b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:18.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.831+0000 7ff521586640 1 --2- 172.21.15.120:0/4123153654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ac2221b40 0x557ac2230b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.831+0000 7ff521586640 1 -- 172.21.15.120:0/4123153654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ac222f780 msgr2=0x557ac22310b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:18.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.831+0000 7ff521586640 1 --2- 172.21.15.120:0/4123153654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ac222f780 0x557ac22310b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.831+0000 7ff521586640 1 -- 172.21.15.120:0/4123153654 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ac221ced0 con 0x557ac22205d0 2024-09-19T09:27:18.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.832+0000 7ff521586640 1 --2- 172.21.15.120:0/4123153654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ac22205d0 0x557ac223b6c0 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7ff518097e10 tx=0x7ff518016500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a4266c2b1c8f0f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:18.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.832+0000 7ff506ffd640 1 -- 172.21.15.120:0/4123153654 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518020040 con 0x557ac22205d0 2024-09-19T09:27:18.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.832+0000 7ff506ffd640 1 -- 172.21.15.120:0/4123153654 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5180b4070 con 0x557ac22205d0 2024-09-19T09:27:18.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.832+0000 7ff506ffd640 1 -- 172.21.15.120:0/4123153654 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518099d40 con 0x557ac22205d0 2024-09-19T09:27:18.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.833+0000 7ff525bc3740 1 -- 172.21.15.120:0/4123153654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ac22205d0 msgr2=0x557ac223b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:18.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.833+0000 7ff525bc3740 1 --2- 172.21.15.120:0/4123153654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ac22205d0 0x557ac223b6c0 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7ff518097e10 tx=0x7ff518016500 comp rx=0 tx=0).stop 2024-09-19T09:27:18.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.833+0000 7ff525bc3740 1 -- 172.21.15.120:0/4123153654 shutdown_connections 2024-09-19T09:27:18.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.833+0000 7ff525bc3740 1 --2- 172.21.15.120:0/4123153654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ac222f780 0x557ac22310b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.833+0000 7ff525bc3740 1 --2- 172.21.15.120:0/4123153654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ac2221b40 0x557ac2230b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.833+0000 7ff525bc3740 1 --2- 172.21.15.120:0/4123153654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ac22205d0 0x557ac223b6c0 unknown :-1 s=CLOSED pgs=2463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.833+0000 7ff525bc3740 1 -- 172.21.15.120:0/4123153654 >> 172.21.15.120:0/4123153654 conn(0x557ac21e9310 msgr2=0x557ac222acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:18.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.833+0000 7ff525bc3740 1 -- 172.21.15.120:0/4123153654 shutdown_connections 2024-09-19T09:27:18.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.833+0000 7ff525bc3740 1 -- 172.21.15.120:0/4123153654 wait complete. 2024-09-19T09:27:18.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.833+0000 7ff525bc3740 1 -- start start 2024-09-19T09:27:18.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.833+0000 7ff525bc3740 1 Processor -- start 2024-09-19T09:27:18.148 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:27:18.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.833+0000 7ff525bc3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ac22205d0 0x557ac2237f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:18.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.833+0000 7ff525bc3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ac2221b40 0x557ac2246a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:18.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.833+0000 7ff525bc3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ac222f780 0x557ac2246f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:18.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.833+0000 7ff525bc3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c1ac740 con 0x557ac2221b40 2024-09-19T09:27:18.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.833+0000 7ff525bc3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c1ac070 con 0x557ac222f780 2024-09-19T09:27:18.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.833+0000 7ff525bc3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c1a9290 con 0x557ac22205d0 2024-09-19T09:27:18.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.833+0000 7ff525bc3740 10 client.0 _ll_register_callbacks cb 0x7ff514009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:18.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.834+0000 7ff520d85640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ac2221b40 0x557ac2246a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:18.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.834+0000 7ff520d85640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ac2221b40 0x557ac2246a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34990/0 (socket says 172.21.15.120:34990) 2024-09-19T09:27:18.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.834+0000 7ff520d85640 1 -- 172.21.15.120:0/3986199350 learned_addr learned my addr 172.21.15.120:0/3986199350 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:18.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.834+0000 7ff521d87640 1 --2- 172.21.15.120:0/3986199350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ac222f780 0x557ac2246f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:18.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.834+0000 7ff521586640 1 --2- 172.21.15.120:0/3986199350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ac22205d0 0x557ac2237f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:18.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.834+0000 7ff520d85640 1 -- 172.21.15.120:0/3986199350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ac22205d0 msgr2=0x557ac2237f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:18.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.834+0000 7ff520d85640 1 --2- 172.21.15.120:0/3986199350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ac22205d0 0x557ac2237f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.834+0000 7ff520d85640 1 -- 172.21.15.120:0/3986199350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ac222f780 msgr2=0x557ac2246f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:18.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.834+0000 7ff520d85640 1 --2- 172.21.15.120:0/3986199350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ac222f780 0x557ac2246f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.834+0000 7ff520d85640 1 -- 172.21.15.120:0/3986199350 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ac2247760 con 0x557ac2221b40 2024-09-19T09:27:18.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.834+0000 7ff521586640 1 --2- 172.21.15.120:0/3986199350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ac22205d0 0x557ac2237f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:18.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.834+0000 7ff520d85640 1 --2- 172.21.15.120:0/3986199350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ac2221b40 0x557ac2246a40 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7ff50c02a930 tx=0x7ff50c02ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5008c57c9f99242e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:18.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.834+0000 7ff504ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:27:18.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.834+0000 7ff504ff9640 1 -- 172.21.15.120:0/3986199350 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c035070 con 0x557ac2221b40 2024-09-19T09:27:18.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.834+0000 7ff504ff9640 1 -- 172.21.15.120:0/3986199350 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff50c004450 con 0x557ac2221b40 2024-09-19T09:27:18.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.834+0000 7ff525bc3740 1 -- 172.21.15.120:0/3986199350 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5180b4070 con 0x557ac2221b40 2024-09-19T09:27:18.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.834+0000 7ff504ff9640 1 -- 172.21.15.120:0/3986199350 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff50c053050 con 0x557ac2221b40 2024-09-19T09:27:18.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.834+0000 7ff525bc3740 10 client.23334 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:18.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.834+0000 7ff525bc3740 1 -- 172.21.15.120:0/3986199350 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557ac22aa160 con 0x557ac2221b40 2024-09-19T09:27:18.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.835+0000 7ff504ff9640 1 -- 172.21.15.120:0/3986199350 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5180b4070 con 0x557ac2221b40 2024-09-19T09:27:18.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.835+0000 7ff504ff9640 1 -- 172.21.15.120:0/3986199350 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7ff50c004ab0 con 0x557ac2221b40 2024-09-19T09:27:18.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.835+0000 7ff525bc3740 10 client.23334 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:18.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.835+0000 7ff525bc3740 10 client.23334 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:27:18.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.835+0000 7ff525bc3740 20 client.23334 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:18.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.835+0000 7ff525bc3740 1 --2- 172.21.15.120:0/3986199350 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557ac223f400 0x557ac22ac490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:18.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.835+0000 7ff525bc3740 4 client.23334 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:27:18.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.835+0000 7ff525bc3740 1 -- 172.21.15.120:0/3986199350 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557ac218dd70 con 0x557ac223f400 2024-09-19T09:27:18.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.835+0000 7ff521586640 1 --2- 172.21.15.120:0/3986199350 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557ac223f400 0x557ac22ac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:18.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.835+0000 7ff521586640 1 --2- 172.21.15.120:0/3986199350 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557ac223f400 0x557ac22ac490 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9672c078b5abf3d8 server_cookie=bbd63b1aec2eb4ab in_seq=0 out_seq=0 2024-09-19T09:27:18.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.835+0000 7ff504ff9640 10 client.23334 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:27:18.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.836+0000 7ff504ff9640 1 -- 172.21.15.120:0/3986199350 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557ac218dd70 con 0x557ac223f400 2024-09-19T09:27:18.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.836+0000 7ff504ff9640 10 client.23334 handle_command_reply: tid=0 2024-09-19T09:27:18.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.836+0000 7ff525bc3740 1 client.23334 shutdown 2024-09-19T09:27:18.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.836+0000 7ff525bc3740 1 -- 172.21.15.120:0/3986199350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ac2221b40 msgr2=0x557ac2246a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:18.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.836+0000 7ff525bc3740 1 --2- 172.21.15.120:0/3986199350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ac2221b40 0x557ac2246a40 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7ff50c02a930 tx=0x7ff50c02ab10 comp rx=0 tx=0).stop 2024-09-19T09:27:18.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.836+0000 7ff525bc3740 1 -- 172.21.15.120:0/3986199350 shutdown_connections 2024-09-19T09:27:18.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.836+0000 7ff525bc3740 1 --2- 172.21.15.120:0/3986199350 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x557ac223f400 0x557ac22ac490 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.836+0000 7ff525bc3740 1 --2- 172.21.15.120:0/3986199350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ac222f780 0x557ac2246f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.836+0000 7ff525bc3740 1 --2- 172.21.15.120:0/3986199350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ac2221b40 0x557ac2246a40 unknown :-1 s=CLOSED pgs=2464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.836+0000 7ff525bc3740 1 --2- 172.21.15.120:0/3986199350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ac22205d0 0x557ac2237f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.836+0000 7ff525bc3740 1 -- 172.21.15.120:0/3986199350 >> 172.21.15.120:0/3986199350 conn(0x557ac21e9310 msgr2=0x557ac222dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:18.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.836+0000 7ff504ff9640 0 client.23334 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:27:18.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.836+0000 7ff525bc3740 1 -- 172.21.15.120:0/3986199350 shutdown_connections 2024-09-19T09:27:18.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.836+0000 7ff525bc3740 1 -- 172.21.15.120:0/3986199350 wait complete. 2024-09-19T09:27:18.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.836+0000 7ff525bc3740 20 client.23334 trim_cache size 0 max 16384 2024-09-19T09:27:18.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.836+0000 7ff5237ea640 1 -- 172.21.15.120:0/3739473268 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff5080a32c0 msgr2=0x7ff5080a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:18.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.837+0000 7ff5237ea640 1 --2- 172.21.15.120:0/3739473268 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff5080a32c0 0x7ff5080a5780 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff514005c40 tx=0x7ff5140094a0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.837+0000 7ff5237ea640 1 -- 172.21.15.120:0/3739473268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff51c101de0 msgr2=0x7ff51c1984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:18.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.837+0000 7ff5237ea640 1 --2- 172.21.15.120:0/3739473268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff51c101de0 0x7ff51c1984b0 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7ff51800d770 tx=0x7ff51800dc30 comp rx=0 tx=0).stop 2024-09-19T09:27:18.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.837+0000 7ff5237ea640 1 -- 172.21.15.120:0/3739473268 shutdown_connections 2024-09-19T09:27:18.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.837+0000 7ff5237ea640 1 --2- 172.21.15.120:0/3739473268 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7ff5080a32c0 0x7ff5080a5780 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.837+0000 7ff5237ea640 1 --2- 172.21.15.120:0/3739473268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff51c106f10 0x7ff51c19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.837+0000 7ff5237ea640 1 --2- 172.21.15.120:0/3739473268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff51c102790 0x7ff51c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.837+0000 7ff5237ea640 1 --2- 172.21.15.120:0/3739473268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff51c101de0 0x7ff51c1984b0 unknown :-1 s=CLOSED pgs=2460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.837+0000 7ff5237ea640 1 -- 172.21.15.120:0/3739473268 >> 172.21.15.120:0/3739473268 conn(0x7ff51c0fd8c0 msgr2=0x7ff51c0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:18.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.837+0000 7ff5237ea640 1 -- 172.21.15.120:0/3739473268 shutdown_connections 2024-09-19T09:27:18.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:17.837+0000 7ff5237ea640 1 -- 172.21.15.120:0/3739473268 wait complete. 2024-09-19T09:27:18.390 INFO:tasks.workunit.client.0.smithi120.stdout: 10 266983 20.61 MB/sec execute 553 sec latency 7373.458 ms 2024-09-19T09:27:18.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:18 smithi196 ceph-mon[35150]: pgmap v1078: 97 pgs: 97 active+clean; 502 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.2 MiB/s wr, 483 op/s 2024-09-19T09:27:18.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:18 smithi153 ceph-mon[35071]: pgmap v1078: 97 pgs: 97 active+clean; 502 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.2 MiB/s wr, 483 op/s 2024-09-19T09:27:18.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:18 smithi120 ceph-mon[31150]: pgmap v1078: 97 pgs: 97 active+clean; 502 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.2 MiB/s wr, 483 op/s 2024-09-19T09:27:19.061 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:27:19.394 INFO:tasks.workunit.client.0.smithi120.stdout: 10 266983 20.58 MB/sec execute 554 sec latency 8376.267 ms 2024-09-19T09:27:19.441 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:19.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.944+0000 7f7462689640 1 -- 172.21.15.120:0/2357114439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f745c101de0 msgr2=0x7f745c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:19.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.944+0000 7f7462689640 1 --2- 172.21.15.120:0/2357114439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f745c101de0 0x7f745c1021c0 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f744c009900 tx=0x7f744c02a410 comp rx=0 tx=0).stop 2024-09-19T09:27:19.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.944+0000 7f7462689640 1 -- 172.21.15.120:0/2357114439 shutdown_connections 2024-09-19T09:27:19.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.944+0000 7f7462689640 1 --2- 172.21.15.120:0/2357114439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f745c106f10 0x7f745c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.944+0000 7f7462689640 1 --2- 172.21.15.120:0/2357114439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f745c102790 0x7f745c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.944+0000 7f7462689640 1 --2- 172.21.15.120:0/2357114439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f745c101de0 0x7f745c1021c0 unknown :-1 s=CLOSED pgs=2465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.944+0000 7f7462689640 1 -- 172.21.15.120:0/2357114439 >> 172.21.15.120:0/2357114439 conn(0x7f745c0fd8c0 msgr2=0x7f745c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:19.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.944+0000 7f7462689640 1 -- 172.21.15.120:0/2357114439 shutdown_connections 2024-09-19T09:27:19.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.944+0000 7f7462689640 1 -- 172.21.15.120:0/2357114439 wait complete. 2024-09-19T09:27:19.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.945+0000 7f7462689640 1 Processor -- start 2024-09-19T09:27:19.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.945+0000 7f7462689640 1 -- start start 2024-09-19T09:27:19.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.945+0000 7f7462689640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f745c101de0 0x7f745c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:19.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.945+0000 7f7462689640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f745c102790 0x7f745c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:19.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.945+0000 7f7462689640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f745c106f10 0x7f745c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:19.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.945+0000 7f7462689640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f745c111f70 con 0x7f745c106f10 2024-09-19T09:27:19.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.945+0000 7f7462689640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f745c111df0 con 0x7f745c101de0 2024-09-19T09:27:19.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.945+0000 7f7462689640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1120f0 con 0x7f745c102790 2024-09-19T09:27:19.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.945+0000 7f7460c26640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f745c106f10 0x7f745c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:19.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.945+0000 7f7460c26640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f745c106f10 0x7f745c19cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34996/0 (socket says 172.21.15.120:34996) 2024-09-19T09:27:19.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.945+0000 7f7460c26640 1 -- 172.21.15.120:0/1856164674 learned_addr learned my addr 172.21.15.120:0/1856164674 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:19.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.945+0000 7f745bfff640 1 --2- 172.21.15.120:0/1856164674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f745c101de0 0x7f745c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:19.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.945+0000 7f7453fff640 1 --2- 172.21.15.120:0/1856164674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f745c102790 0x7f745c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:19.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.946+0000 7f7460c26640 1 -- 172.21.15.120:0/1856164674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f745c102790 msgr2=0x7f745c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:19.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.946+0000 7f7460c26640 1 --2- 172.21.15.120:0/1856164674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f745c102790 0x7f745c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.946+0000 7f7460c26640 1 -- 172.21.15.120:0/1856164674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f745c101de0 msgr2=0x7f745c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:19.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.946+0000 7f7460c26640 1 --2- 172.21.15.120:0/1856164674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f745c101de0 0x7f745c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.946+0000 7f7460c26640 1 -- 172.21.15.120:0/1856164674 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745c19d3e0 con 0x7f745c106f10 2024-09-19T09:27:19.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.946+0000 7f7460c26640 1 --2- 172.21.15.120:0/1856164674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f745c106f10 0x7f745c19cd00 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f7454009fd0 tx=0x7f745400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:19.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.946+0000 7f7459ffb640 1 -- 172.21.15.120:0/1856164674 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7454017070 con 0x7f745c106f10 2024-09-19T09:27:19.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.946+0000 7f7459ffb640 1 -- 172.21.15.120:0/1856164674 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7454003e50 con 0x7f745c106f10 2024-09-19T09:27:19.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.946+0000 7f7462689640 1 -- 172.21.15.120:0/1856164674 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f745c19d6d0 con 0x7f745c106f10 2024-09-19T09:27:19.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.946+0000 7f7459ffb640 1 -- 172.21.15.120:0/1856164674 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74540106c0 con 0x7f745c106f10 2024-09-19T09:27:19.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.946+0000 7f7462689640 1 -- 172.21.15.120:0/1856164674 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f745c1a4fb0 con 0x7f745c106f10 2024-09-19T09:27:19.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.949+0000 7f7459ffb640 1 -- 172.21.15.120:0/1856164674 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f7454003970 con 0x7f745c106f10 2024-09-19T09:27:19.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.949+0000 7f7464a62740 1 Processor -- start 2024-09-19T09:27:19.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.949+0000 7f7464a62740 1 -- start start 2024-09-19T09:27:19.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.949+0000 7f7464a62740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04b875b40 0x55b04b874090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:19.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.949+0000 7f7464a62740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b04b8745d0 0x55b04b874a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:19.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.949+0000 7f7464a62740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b04b883780 0x55b04b885b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:19.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.949+0000 7f7464a62740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b04b7da2f0 con 0x55b04b883780 2024-09-19T09:27:19.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.949+0000 7f7464a62740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b04b781cf0 con 0x55b04b875b40 2024-09-19T09:27:19.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.949+0000 7f7464a62740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b04b7d62b0 con 0x55b04b8745d0 2024-09-19T09:27:19.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.949+0000 7f7453fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b04b883780 0x55b04b885b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:19.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.949+0000 7f7453fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b04b883780 0x55b04b885b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35002/0 (socket says 172.21.15.120:35002) 2024-09-19T09:27:19.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.949+0000 7f7453fff640 1 -- 172.21.15.120:0/1619078935 learned_addr learned my addr 172.21.15.120:0/1619078935 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:19.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.950+0000 7f745bfff640 1 --2- 172.21.15.120:0/1619078935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b04b8745d0 0x55b04b874a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:19.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.950+0000 7f7459ffb640 1 --2- 172.21.15.120:0/1856164674 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f74480a3240 0x7f74480a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:19.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.950+0000 7f7453fff640 1 --2- 172.21.15.120:0/1619078935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04b875b40 0x55b04b874090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:19.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.950+0000 7f7453fff640 1 -- 172.21.15.120:0/1619078935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b04b8745d0 msgr2=0x55b04b874a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:19.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.950+0000 7f7460c26640 1 --2- 172.21.15.120:0/1856164674 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f74480a3240 0x7f74480a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:19.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.950+0000 7f7453fff640 1 --2- 172.21.15.120:0/1619078935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b04b8745d0 0x55b04b874a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.950+0000 7f7453fff640 1 -- 172.21.15.120:0/1619078935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04b875b40 msgr2=0x55b04b874090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:19.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.950+0000 7f7453fff640 1 --2- 172.21.15.120:0/1619078935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04b875b40 0x55b04b874090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.950+0000 7f7453fff640 1 -- 172.21.15.120:0/1619078935 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b04b7663a0 con 0x55b04b883780 2024-09-19T09:27:19.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.950+0000 7f7459ffb640 1 -- 172.21.15.120:0/1856164674 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f74540cf590 con 0x7f745c106f10 2024-09-19T09:27:19.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.950+0000 7f7453fff640 1 --2- 172.21.15.120:0/1619078935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b04b883780 0x55b04b885b70 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f744400bc10 tx=0x7f74440041f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce32b02f8ad46bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:19.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.950+0000 7f74527fc640 1 -- 172.21.15.120:0/1619078935 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7444005020 con 0x55b04b883780 2024-09-19T09:27:19.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.950+0000 7f74527fc640 1 -- 172.21.15.120:0/1619078935 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74440047a0 con 0x55b04b883780 2024-09-19T09:27:19.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.950+0000 7f7460c26640 1 --2- 172.21.15.120:0/1856164674 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f74480a3240 0x7f74480a5700 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f745409bbf0 tx=0x7f7454091140 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:19.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.950+0000 7f74527fc640 1 -- 172.21.15.120:0/1619078935 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74440106c0 con 0x55b04b883780 2024-09-19T09:27:19.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.951+0000 7f7464a62740 1 -- 172.21.15.120:0/1619078935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b04b883780 msgr2=0x55b04b885b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:19.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.951+0000 7f7464a62740 1 --2- 172.21.15.120:0/1619078935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b04b883780 0x55b04b885b70 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f744400bc10 tx=0x7f74440041f0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.951+0000 7f7464a62740 1 -- 172.21.15.120:0/1619078935 shutdown_connections 2024-09-19T09:27:19.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.951+0000 7f7464a62740 1 --2- 172.21.15.120:0/1619078935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b04b883780 0x55b04b885b70 unknown :-1 s=CLOSED pgs=2467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.951+0000 7f7464a62740 1 --2- 172.21.15.120:0/1619078935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b04b8745d0 0x55b04b874a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.951+0000 7f7464a62740 1 --2- 172.21.15.120:0/1619078935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04b875b40 0x55b04b874090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.951+0000 7f7464a62740 1 -- 172.21.15.120:0/1619078935 >> 172.21.15.120:0/1619078935 conn(0x55b04b83d310 msgr2=0x55b04b870c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:19.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.951+0000 7f7464a62740 1 -- 172.21.15.120:0/1619078935 shutdown_connections 2024-09-19T09:27:19.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.951+0000 7f7464a62740 1 -- 172.21.15.120:0/1619078935 wait complete. 2024-09-19T09:27:19.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.951+0000 7f7464a62740 1 -- start start 2024-09-19T09:27:19.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.952+0000 7f7464a62740 1 Processor -- start 2024-09-19T09:27:19.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.952+0000 7f7464a62740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04b8745d0 0x55b04b883150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:19.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.952+0000 7f7464a62740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b04b875b40 0x55b04b880400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:19.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.952+0000 7f7464a62740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b04b883780 0x55b04b8809f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:19.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.952+0000 7f7464a62740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b04b886480 con 0x55b04b883780 2024-09-19T09:27:19.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.952+0000 7f7464a62740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b04b795060 con 0x55b04b8745d0 2024-09-19T09:27:19.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.952+0000 7f7464a62740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b04b886600 con 0x55b04b875b40 2024-09-19T09:27:19.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.952+0000 7f7464a62740 10 client.0 _ll_register_callbacks cb 0x55b04b4e88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:19.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.952+0000 7f7453fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b04b883780 0x55b04b8809f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:19.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.952+0000 7f7453fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b04b883780 0x55b04b8809f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35014/0 (socket says 172.21.15.120:35014) 2024-09-19T09:27:19.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.952+0000 7f7453fff640 1 -- 172.21.15.120:0/3878858694 learned_addr learned my addr 172.21.15.120:0/3878858694 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:19.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.952+0000 7f7453fff640 1 --2- 172.21.15.120:0/3878858694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04b8745d0 0x55b04b883150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:19.995 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:27:19.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.952+0000 7f745bfff640 1 --2- 172.21.15.120:0/3878858694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b04b875b40 0x55b04b880400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:19.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.952+0000 7f7453fff640 1 -- 172.21.15.120:0/3878858694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b04b875b40 msgr2=0x55b04b880400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:19.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.952+0000 7f7453fff640 1 --2- 172.21.15.120:0/3878858694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b04b875b40 0x55b04b880400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.952+0000 7f7453fff640 1 -- 172.21.15.120:0/3878858694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04b8745d0 msgr2=0x55b04b883150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:19.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.952+0000 7f7453fff640 1 --2- 172.21.15.120:0/3878858694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04b8745d0 0x55b04b883150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.952+0000 7f7453fff640 1 -- 172.21.15.120:0/3878858694 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b04b54b3c0 con 0x55b04b883780 2024-09-19T09:27:19.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.952+0000 7f7453fff640 1 --2- 172.21.15.120:0/3878858694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b04b883780 0x55b04b8809f0 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f744400b900 tx=0x7f744400b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb29daec65241138 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:19.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.952+0000 7f743b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:27:19.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.953+0000 7f743b7fe640 1 -- 172.21.15.120:0/3878858694 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7444004e20 con 0x55b04b883780 2024-09-19T09:27:19.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.953+0000 7f743b7fe640 1 -- 172.21.15.120:0/3878858694 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f7444005210 con 0x55b04b883780 2024-09-19T09:27:19.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.953+0000 7f7464a62740 1 -- 172.21.15.120:0/3878858694 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74440047a0 con 0x55b04b883780 2024-09-19T09:27:19.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.953+0000 7f743b7fe640 1 -- 172.21.15.120:0/3878858694 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7444005410 con 0x55b04b883780 2024-09-19T09:27:19.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.953+0000 7f7464a62740 10 client.23346 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:19.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.953+0000 7f7464a62740 1 -- 172.21.15.120:0/3878858694 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b04b891ef0 con 0x55b04b883780 2024-09-19T09:27:19.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.953+0000 7f743b7fe640 1 -- 172.21.15.120:0/3878858694 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74440047a0 con 0x55b04b883780 2024-09-19T09:27:19.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.953+0000 7f743b7fe640 1 -- 172.21.15.120:0/3878858694 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f744402f030 con 0x55b04b883780 2024-09-19T09:27:19.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.953+0000 7f7464a62740 10 client.23346 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:19.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.953+0000 7f7464a62740 10 client.23346 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:27:19.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.953+0000 7f7464a62740 20 client.23346 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:19.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.953+0000 7f7464a62740 1 --2- 172.21.15.120:0/3878858694 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55b04b893240 0x55b04b895630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:19.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.953+0000 7f7464a62740 4 client.23346 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:19.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.953+0000 7f7464a62740 1 -- 172.21.15.120:0/3878858694 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b04b7e7860 con 0x55b04b893240 2024-09-19T09:27:19.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.954+0000 7f745bfff640 1 --2- 172.21.15.120:0/3878858694 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55b04b893240 0x55b04b895630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:19.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.954+0000 7f743b7fe640 10 client.23346 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:27:19.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.954+0000 7f745bfff640 1 --2- 172.21.15.120:0/3878858694 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55b04b893240 0x55b04b895630 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6f35299259a5b62a server_cookie=15bbaf55d2fa08ce in_seq=0 out_seq=0 2024-09-19T09:27:19.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.954+0000 7f743b7fe640 1 -- 172.21.15.120:0/3878858694 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b04b7e7860 con 0x55b04b893240 2024-09-19T09:27:19.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.954+0000 7f743b7fe640 10 client.23346 handle_command_reply: tid=0 2024-09-19T09:27:19.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.954+0000 7f7464a62740 1 client.23346 shutdown 2024-09-19T09:27:19.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.955+0000 7f7464a62740 1 -- 172.21.15.120:0/3878858694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b04b883780 msgr2=0x55b04b8809f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:19.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.955+0000 7f7464a62740 1 --2- 172.21.15.120:0/3878858694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b04b883780 0x55b04b8809f0 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f744400b900 tx=0x7f744400b9c0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.955+0000 7f7464a62740 1 -- 172.21.15.120:0/3878858694 shutdown_connections 2024-09-19T09:27:19.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.955+0000 7f7464a62740 1 --2- 172.21.15.120:0/3878858694 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55b04b893240 0x55b04b895630 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.955+0000 7f7464a62740 1 --2- 172.21.15.120:0/3878858694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b04b883780 0x55b04b8809f0 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.955+0000 7f7464a62740 1 --2- 172.21.15.120:0/3878858694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b04b875b40 0x55b04b880400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.955+0000 7f743b7fe640 0 client.23346 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:27:19.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.955+0000 7f7464a62740 1 --2- 172.21.15.120:0/3878858694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04b8745d0 0x55b04b883150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.955+0000 7f7464a62740 1 -- 172.21.15.120:0/3878858694 >> 172.21.15.120:0/3878858694 conn(0x55b04b83d310 msgr2=0x55b04b876960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:19.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.955+0000 7f7464a62740 1 -- 172.21.15.120:0/3878858694 shutdown_connections 2024-09-19T09:27:19.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.955+0000 7f7464a62740 1 -- 172.21.15.120:0/3878858694 wait complete. 2024-09-19T09:27:19.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.955+0000 7f7464a62740 20 client.23346 trim_cache size 0 max 16384 2024-09-19T09:27:19.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.967+0000 7f7464a62740 1 Processor -- start 2024-09-19T09:27:19.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.968+0000 7f7464a62740 1 -- start start 2024-09-19T09:27:19.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.968+0000 7f7464a62740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b04b8745d0 0x55b04b88f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:19.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.968+0000 7f7464a62740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b04b875b40 0x55b04b87f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:19.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.968+0000 7f7464a62740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04b883780 0x55b04b89adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:20.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.968+0000 7f7464a62740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1ab370 con 0x55b04b8745d0 2024-09-19T09:27:20.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.968+0000 7f7464a62740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1aceb0 con 0x55b04b883780 2024-09-19T09:27:20.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.968+0000 7f7464a62740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1aba40 con 0x55b04b875b40 2024-09-19T09:27:20.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.968+0000 7f7453fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b04b8745d0 0x55b04b88f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:20.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.968+0000 7f7453fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b04b8745d0 0x55b04b88f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35026/0 (socket says 172.21.15.120:35026) 2024-09-19T09:27:20.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.968+0000 7f7453fff640 1 -- 172.21.15.120:0/4069043358 learned_addr learned my addr 172.21.15.120:0/4069043358 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:20.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.968+0000 7f745bfff640 1 --2- 172.21.15.120:0/4069043358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b04b875b40 0x55b04b87f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:20.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.968+0000 7f7453fff640 1 -- 172.21.15.120:0/4069043358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b04b875b40 msgr2=0x55b04b87f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:20.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.968+0000 7f7453fff640 1 --2- 172.21.15.120:0/4069043358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b04b875b40 0x55b04b87f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:20.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.968+0000 7f7453fff640 1 -- 172.21.15.120:0/4069043358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04b883780 msgr2=0x55b04b89adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:20.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.968+0000 7f7453fff640 1 --2- 172.21.15.120:0/4069043358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04b883780 0x55b04b89adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:20.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.968+0000 7f7453fff640 1 -- 172.21.15.120:0/4069043358 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b04b895bc0 con 0x55b04b8745d0 2024-09-19T09:27:20.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.968+0000 7f745bfff640 1 --2- 172.21.15.120:0/4069043358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b04b875b40 0x55b04b87f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:20.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.968+0000 7f7453fff640 1 --2- 172.21.15.120:0/4069043358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b04b8745d0 0x55b04b88f720 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f744400bb60 tx=0x7f7444005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4ee0dd17dec594 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:20.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.968+0000 7f7451ffb640 1 -- 172.21.15.120:0/4069043358 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744400b4a0 con 0x55b04b8745d0 2024-09-19T09:27:20.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.968+0000 7f7451ffb640 1 -- 172.21.15.120:0/4069043358 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7444005210 con 0x55b04b8745d0 2024-09-19T09:27:20.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.968+0000 7f7451ffb640 1 -- 172.21.15.120:0/4069043358 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74440055b0 con 0x55b04b8745d0 2024-09-19T09:27:20.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.969+0000 7f7464a62740 1 -- 172.21.15.120:0/4069043358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b04b8745d0 msgr2=0x55b04b88f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:20.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.969+0000 7f7464a62740 1 --2- 172.21.15.120:0/4069043358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b04b8745d0 0x55b04b88f720 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f744400bb60 tx=0x7f7444005f30 comp rx=0 tx=0).stop 2024-09-19T09:27:20.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.969+0000 7f7464a62740 1 -- 172.21.15.120:0/4069043358 shutdown_connections 2024-09-19T09:27:20.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.969+0000 7f7464a62740 1 --2- 172.21.15.120:0/4069043358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04b883780 0x55b04b89adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:20.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.969+0000 7f7464a62740 1 --2- 172.21.15.120:0/4069043358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b04b875b40 0x55b04b87f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:20.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.969+0000 7f7464a62740 1 --2- 172.21.15.120:0/4069043358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b04b8745d0 0x55b04b88f720 unknown :-1 s=CLOSED pgs=2469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:20.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.969+0000 7f7464a62740 1 -- 172.21.15.120:0/4069043358 >> 172.21.15.120:0/4069043358 conn(0x55b04b83d310 msgr2=0x55b04b87ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:20.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.969+0000 7f7464a62740 1 -- 172.21.15.120:0/4069043358 shutdown_connections 2024-09-19T09:27:20.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.969+0000 7f7464a62740 1 -- 172.21.15.120:0/4069043358 wait complete. 2024-09-19T09:27:20.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.969+0000 7f7464a62740 1 -- start start 2024-09-19T09:27:20.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.969+0000 7f7464a62740 1 Processor -- start 2024-09-19T09:27:20.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.970+0000 7f7464a62740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b04b8745d0 0x55b04b88ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:20.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.970+0000 7f7464a62740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04b875b40 0x55b04b88bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:20.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.970+0000 7f7464a62740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b04b883780 0x55b04b87cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:20.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.970+0000 7f7464a62740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1ac7e0 con 0x55b04b883780 2024-09-19T09:27:20.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.970+0000 7f7464a62740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1ac110 con 0x55b04b875b40 2024-09-19T09:27:20.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.970+0000 7f7464a62740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1a9330 con 0x55b04b8745d0 2024-09-19T09:27:20.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.970+0000 7f7464a62740 10 client.0 _ll_register_callbacks cb 0x7f744c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:20.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.970+0000 7f7453fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b04b883780 0x55b04b87cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:20.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.970+0000 7f7453fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b04b883780 0x55b04b87cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35042/0 (socket says 172.21.15.120:35042) 2024-09-19T09:27:20.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.970+0000 7f7453fff640 1 -- 172.21.15.120:0/2653197569 learned_addr learned my addr 172.21.15.120:0/2653197569 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:20.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.970+0000 7f745bfff640 1 --2- 172.21.15.120:0/2653197569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04b875b40 0x55b04b88bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:20.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.970+0000 7f7453fff640 1 --2- 172.21.15.120:0/2653197569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b04b8745d0 0x55b04b88ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:20.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.970+0000 7f745bfff640 1 -- 172.21.15.120:0/2653197569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b04b8745d0 msgr2=0x55b04b88ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:20.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.970+0000 7f745bfff640 1 --2- 172.21.15.120:0/2653197569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b04b8745d0 0x55b04b88ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:20.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.970+0000 7f745bfff640 1 -- 172.21.15.120:0/2653197569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b04b883780 msgr2=0x55b04b87cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:20.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.970+0000 7f745bfff640 1 --2- 172.21.15.120:0/2653197569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b04b883780 0x55b04b87cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:20.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.970+0000 7f745bfff640 1 -- 172.21.15.120:0/2653197569 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b04b87d270 con 0x55b04b875b40 2024-09-19T09:27:20.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.970+0000 7f7453fff640 1 --2- 172.21.15.120:0/2653197569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b04b883780 0x55b04b87cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:27:20.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.970+0000 7f745bfff640 1 --2- 172.21.15.120:0/2653197569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04b875b40 0x55b04b88bfb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f744c005940 tx=0x7f744c031a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80fd348d5b17a79c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:20.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.970+0000 7f743b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:27:20.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.971+0000 7f743b7fe640 1 -- 172.21.15.120:0/2653197569 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c04d030 con 0x55b04b875b40 2024-09-19T09:27:20.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.971+0000 7f7464a62740 1 -- 172.21.15.120:0/2653197569 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7444005210 con 0x55b04b875b40 2024-09-19T09:27:20.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.971+0000 7f743b7fe640 1 -- 172.21.15.120:0/2653197569 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f744c004310 con 0x55b04b875b40 2024-09-19T09:27:20.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.971+0000 7f743b7fe640 1 -- 172.21.15.120:0/2653197569 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f744c035040 con 0x55b04b875b40 2024-09-19T09:27:20.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.971+0000 7f7464a62740 10 client.31973 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:20.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.971+0000 7f7464a62740 1 -- 172.21.15.120:0/2653197569 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b04b8fd6d0 con 0x55b04b875b40 2024-09-19T09:27:20.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.971+0000 7f743b7fe640 1 -- 172.21.15.120:0/2653197569 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7444005210 con 0x55b04b875b40 2024-09-19T09:27:20.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.971+0000 7f743b7fe640 1 -- 172.21.15.120:0/2653197569 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f744c004b30 con 0x55b04b875b40 2024-09-19T09:27:20.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.971+0000 7f7464a62740 10 client.31973 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:20.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.971+0000 7f7464a62740 10 client.31973 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:27:20.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.971+0000 7f7464a62740 20 client.31973 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:20.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.971+0000 7f7464a62740 1 --2- 172.21.15.120:0/2653197569 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55b04b8fde20 0x55b04b900240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:20.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.971+0000 7f7464a62740 4 client.31973 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:27:20.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.971+0000 7f7464a62740 1 -- 172.21.15.120:0/2653197569 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b04b7e1d70 con 0x55b04b8fde20 2024-09-19T09:27:20.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.971+0000 7f7453fff640 1 --2- 172.21.15.120:0/2653197569 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55b04b8fde20 0x55b04b900240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:20.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.972+0000 7f743b7fe640 10 client.31973 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:27:20.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.972+0000 7f7453fff640 1 --2- 172.21.15.120:0/2653197569 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55b04b8fde20 0x55b04b900240 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=95a6fddeb630e483 server_cookie=a65429c4a5b6581a in_seq=0 out_seq=0 2024-09-19T09:27:20.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.972+0000 7f743b7fe640 1 -- 172.21.15.120:0/2653197569 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b04b7e1d70 con 0x55b04b8fde20 2024-09-19T09:27:20.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.972+0000 7f743b7fe640 10 client.31973 handle_command_reply: tid=0 2024-09-19T09:27:20.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.972+0000 7f7464a62740 1 client.31973 shutdown 2024-09-19T09:27:20.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.972+0000 7f7464a62740 1 -- 172.21.15.120:0/2653197569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04b875b40 msgr2=0x55b04b88bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:20.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.972+0000 7f7464a62740 1 --2- 172.21.15.120:0/2653197569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04b875b40 0x55b04b88bfb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f744c005940 tx=0x7f744c031a10 comp rx=0 tx=0).stop 2024-09-19T09:27:20.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.972+0000 7f7464a62740 1 -- 172.21.15.120:0/2653197569 shutdown_connections 2024-09-19T09:27:20.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.972+0000 7f7464a62740 1 --2- 172.21.15.120:0/2653197569 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x55b04b8fde20 0x55b04b900240 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:20.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.972+0000 7f7464a62740 1 --2- 172.21.15.120:0/2653197569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b04b883780 0x55b04b87cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:20.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.972+0000 7f7464a62740 1 --2- 172.21.15.120:0/2653197569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b04b875b40 0x55b04b88bfb0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:20.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.972+0000 7f7464a62740 1 --2- 172.21.15.120:0/2653197569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b04b8745d0 0x55b04b88ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:20.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.972+0000 7f743b7fe640 0 client.31973 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:27:20.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.972+0000 7f7464a62740 1 -- 172.21.15.120:0/2653197569 >> 172.21.15.120:0/2653197569 conn(0x55b04b83d310 msgr2=0x55b04b87f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:20.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.972+0000 7f7464a62740 1 -- 172.21.15.120:0/2653197569 shutdown_connections 2024-09-19T09:27:20.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.972+0000 7f7464a62740 1 -- 172.21.15.120:0/2653197569 wait complete. 2024-09-19T09:27:20.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.972+0000 7f7464a62740 20 client.31973 trim_cache size 0 max 16384 2024-09-19T09:27:20.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.973+0000 7f7462689640 1 -- 172.21.15.120:0/1856164674 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f74480a3240 msgr2=0x7f74480a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:20.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.973+0000 7f7462689640 1 --2- 172.21.15.120:0/1856164674 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f74480a3240 0x7f74480a5700 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f745409bbf0 tx=0x7f7454091140 comp rx=0 tx=0).stop 2024-09-19T09:27:20.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.973+0000 7f7462689640 1 -- 172.21.15.120:0/1856164674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f745c106f10 msgr2=0x7f745c19cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:20.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.973+0000 7f7462689640 1 --2- 172.21.15.120:0/1856164674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f745c106f10 0x7f745c19cd00 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f7454009fd0 tx=0x7f745400b260 comp rx=0 tx=0).stop 2024-09-19T09:27:20.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.973+0000 7f7462689640 1 -- 172.21.15.120:0/1856164674 shutdown_connections 2024-09-19T09:27:20.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.973+0000 7f7462689640 1 --2- 172.21.15.120:0/1856164674 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f74480a3240 0x7f74480a5700 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:20.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.973+0000 7f7462689640 1 --2- 172.21.15.120:0/1856164674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f745c106f10 0x7f745c19cd00 unknown :-1 s=CLOSED pgs=2466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:20.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.973+0000 7f7462689640 1 --2- 172.21.15.120:0/1856164674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f745c102790 0x7f745c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:20.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.973+0000 7f7462689640 1 --2- 172.21.15.120:0/1856164674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f745c101de0 0x7f745c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:20.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.973+0000 7f7462689640 1 -- 172.21.15.120:0/1856164674 >> 172.21.15.120:0/1856164674 conn(0x7f745c0fd8c0 msgr2=0x7f745c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:20.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.973+0000 7f7462689640 1 -- 172.21.15.120:0/1856164674 shutdown_connections 2024-09-19T09:27:20.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:19.973+0000 7f7462689640 1 -- 172.21.15.120:0/1856164674 wait complete. 2024-09-19T09:27:20.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:20 smithi120 ceph-mon[31150]: pgmap v1079: 97 pgs: 97 active+clean; 504 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.1 MiB/s wr, 424 op/s 2024-09-19T09:27:20.393 INFO:tasks.workunit.client.0.smithi120.stdout: 10 266983 20.54 MB/sec execute 555 sec latency 9376.385 ms 2024-09-19T09:27:20.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:20 smithi196 ceph-mon[35150]: pgmap v1079: 97 pgs: 97 active+clean; 504 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.1 MiB/s wr, 424 op/s 2024-09-19T09:27:20.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:20 smithi153 ceph-mon[35071]: pgmap v1079: 97 pgs: 97 active+clean; 504 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.1 MiB/s wr, 424 op/s 2024-09-19T09:27:20.892 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:27:21.136 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:21.393 INFO:tasks.workunit.client.0.smithi120.stdout: 10 266983 20.50 MB/sec execute 556 sec latency 10376.473 ms 2024-09-19T09:27:21.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.688+0000 7f1b534f2640 1 -- 172.21.15.120:0/2722647349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b4c100aa0 msgr2=0x7f1b4c10ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:21.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.688+0000 7f1b534f2640 1 --2- 172.21.15.120:0/2722647349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b4c100aa0 0x7f1b4c10ea20 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f1b44009ca0 tx=0x7f1b4402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.689+0000 7f1b534f2640 1 -- 172.21.15.120:0/2722647349 shutdown_connections 2024-09-19T09:27:21.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.689+0000 7f1b534f2640 1 --2- 172.21.15.120:0/2722647349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b4c100aa0 0x7f1b4c10ea20 unknown :-1 s=CLOSED pgs=2470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.689+0000 7f1b534f2640 1 --2- 172.21.15.120:0/2722647349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b4c100100 0x7f1b4c100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.689+0000 7f1b534f2640 1 --2- 172.21.15.120:0/2722647349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b4c106b80 0x7f1b4c106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.689+0000 7f1b534f2640 1 -- 172.21.15.120:0/2722647349 >> 172.21.15.120:0/2722647349 conn(0x7f1b4c077230 msgr2=0x7f1b4c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:21.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.689+0000 7f1b534f2640 1 -- 172.21.15.120:0/2722647349 shutdown_connections 2024-09-19T09:27:21.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.689+0000 7f1b534f2640 1 -- 172.21.15.120:0/2722647349 wait complete. 2024-09-19T09:27:21.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.689+0000 7f1b534f2640 1 Processor -- start 2024-09-19T09:27:21.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.690+0000 7f1b534f2640 1 -- start start 2024-09-19T09:27:21.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.690+0000 7f1b534f2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b4c100100 0x7f1b4c10bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:21.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.690+0000 7f1b534f2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b4c100aa0 0x7f1b4c10c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:21.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.690+0000 7f1b534f2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b4c106b80 0x7f1b4c1a33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:21.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.690+0000 7f1b534f2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c112050 con 0x7f1b4c100aa0 2024-09-19T09:27:21.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.690+0000 7f1b534f2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c111ed0 con 0x7f1b4c100100 2024-09-19T09:27:21.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.690+0000 7f1b534f2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c1121d0 con 0x7f1b4c106b80 2024-09-19T09:27:21.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.690+0000 7f1b50a8d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b4c100aa0 0x7f1b4c10c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:21.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.690+0000 7f1b50a8d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b4c100aa0 0x7f1b4c10c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42388/0 (socket says 172.21.15.120:42388) 2024-09-19T09:27:21.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.690+0000 7f1b50a8d640 1 -- 172.21.15.120:0/65537436 learned_addr learned my addr 172.21.15.120:0/65537436 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:21.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.690+0000 7f1b51a8f640 1 --2- 172.21.15.120:0/65537436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b4c106b80 0x7f1b4c1a33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:21.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.690+0000 7f1b5128e640 1 --2- 172.21.15.120:0/65537436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b4c100100 0x7f1b4c10bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:21.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.690+0000 7f1b50a8d640 1 -- 172.21.15.120:0/65537436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b4c106b80 msgr2=0x7f1b4c1a33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:21.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.690+0000 7f1b50a8d640 1 --2- 172.21.15.120:0/65537436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b4c106b80 0x7f1b4c1a33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.690+0000 7f1b50a8d640 1 -- 172.21.15.120:0/65537436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b4c100100 msgr2=0x7f1b4c10bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:21.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.690+0000 7f1b50a8d640 1 --2- 172.21.15.120:0/65537436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b4c100100 0x7f1b4c10bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.691+0000 7f1b50a8d640 1 -- 172.21.15.120:0/65537436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4c1a3b00 con 0x7f1b4c100aa0 2024-09-19T09:27:21.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.691+0000 7f1b50a8d640 1 --2- 172.21.15.120:0/65537436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b4c100aa0 0x7f1b4c10c140 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c00e9f0 tx=0x7f1b3c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:21.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.691+0000 7f1b427fc640 1 -- 172.21.15.120:0/65537436 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c00cd80 con 0x7f1b4c100aa0 2024-09-19T09:27:21.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.691+0000 7f1b534f2640 1 -- 172.21.15.120:0/65537436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b4c1a3df0 con 0x7f1b4c100aa0 2024-09-19T09:27:21.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.691+0000 7f1b534f2640 1 -- 172.21.15.120:0/65537436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b4c1ab680 con 0x7f1b4c100aa0 2024-09-19T09:27:21.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.691+0000 7f1b427fc640 1 -- 172.21.15.120:0/65537436 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b3c010910 con 0x7f1b4c100aa0 2024-09-19T09:27:21.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.692+0000 7f1b427fc640 1 -- 172.21.15.120:0/65537436 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c026060 con 0x7f1b4c100aa0 2024-09-19T09:27:21.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.695+0000 7f1b558cb740 1 Processor -- start 2024-09-19T09:27:21.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.696+0000 7f1b427fc640 1 -- 172.21.15.120:0/65537436 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f1b3c010ac0 con 0x7f1b4c100aa0 2024-09-19T09:27:21.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.696+0000 7f1b558cb740 1 -- start start 2024-09-19T09:27:21.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.696+0000 7f1b558cb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc5bf92b10 0x55bc5bf91060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:21.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.696+0000 7f1b558cb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc5bf915a0 0x55bc5bf919e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:21.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.697+0000 7f1b558cb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc5bf9c340 0x55bc5bf9e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:21.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.697+0000 7f1b558cb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bc5bef72f0 con 0x55bc5bf915a0 2024-09-19T09:27:21.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.697+0000 7f1b558cb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bc5be9ecf0 con 0x55bc5bf9c340 2024-09-19T09:27:21.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.697+0000 7f1b558cb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bc5bef32b0 con 0x55bc5bf92b10 2024-09-19T09:27:21.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.697+0000 7f1b51a8f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc5bf92b10 0x55bc5bf91060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:21.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.697+0000 7f1b51a8f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc5bf92b10 0x55bc5bf91060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:58740/0 (socket says 172.21.15.120:58740) 2024-09-19T09:27:21.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.697+0000 7f1b51a8f640 1 -- 172.21.15.120:0/3944867606 learned_addr learned my addr 172.21.15.120:0/3944867606 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:21.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.697+0000 7f1b50a8d640 1 --2- 172.21.15.120:0/3944867606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc5bf9c340 0x55bc5bf9e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:21.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.697+0000 7f1b50a8d640 1 -- 172.21.15.120:0/3944867606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc5bf92b10 msgr2=0x55bc5bf91060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:21.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.697+0000 7f1b50a8d640 1 --2- 172.21.15.120:0/3944867606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc5bf92b10 0x55bc5bf91060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.697+0000 7f1b50a8d640 1 -- 172.21.15.120:0/3944867606 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc5bf915a0 msgr2=0x55bc5bf919e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:21.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.697+0000 7f1b50a8d640 1 --2- 172.21.15.120:0/3944867606 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc5bf915a0 0x55bc5bf919e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.697+0000 7f1b50a8d640 1 -- 172.21.15.120:0/3944867606 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc5be833a0 con 0x55bc5bf9c340 2024-09-19T09:27:21.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.698+0000 7f1b50a8d640 1 --2- 172.21.15.120:0/3944867606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc5bf9c340 0x55bc5bf9e730 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c0cd7c0 tx=0x7f1b3c0d87b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbb602d066115619 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:21.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.698+0000 7f1b427fc640 1 --2- 172.21.15.120:0/65537436 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1b380a3240 0x7f1b380a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:21.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.698+0000 7f1b377fe640 1 -- 172.21.15.120:0/3944867606 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c098690 con 0x55bc5bf9c340 2024-09-19T09:27:21.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.698+0000 7f1b377fe640 1 -- 172.21.15.120:0/3944867606 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b3c0dd070 con 0x55bc5bf9c340 2024-09-19T09:27:21.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.699+0000 7f1b377fe640 1 -- 172.21.15.120:0/3944867606 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c09b840 con 0x55bc5bf9c340 2024-09-19T09:27:21.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.699+0000 7f1b427fc640 1 -- 172.21.15.120:0/65537436 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f1b3c0cfd20 con 0x7f1b4c100aa0 2024-09-19T09:27:21.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.699+0000 7f1b51a8f640 1 --2- 172.21.15.120:0/65537436 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1b380a3240 0x7f1b380a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:21.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.699+0000 7f1b558cb740 1 -- 172.21.15.120:0/3944867606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc5bf9c340 msgr2=0x55bc5bf9e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:21.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.699+0000 7f1b558cb740 1 --2- 172.21.15.120:0/3944867606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc5bf9c340 0x55bc5bf9e730 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c0cd7c0 tx=0x7f1b3c0d87b0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.700+0000 7f1b558cb740 1 -- 172.21.15.120:0/3944867606 shutdown_connections 2024-09-19T09:27:21.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.700+0000 7f1b558cb740 1 --2- 172.21.15.120:0/3944867606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc5bf9c340 0x55bc5bf9e730 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.700+0000 7f1b558cb740 1 --2- 172.21.15.120:0/3944867606 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc5bf915a0 0x55bc5bf919e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.700+0000 7f1b558cb740 1 --2- 172.21.15.120:0/3944867606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc5bf92b10 0x55bc5bf91060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.700+0000 7f1b558cb740 1 -- 172.21.15.120:0/3944867606 >> 172.21.15.120:0/3944867606 conn(0x55bc5bf5a310 msgr2=0x55bc5bf8dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:21.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.700+0000 7f1b558cb740 1 -- 172.21.15.120:0/3944867606 shutdown_connections 2024-09-19T09:27:21.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.700+0000 7f1b558cb740 1 -- 172.21.15.120:0/3944867606 wait complete. 2024-09-19T09:27:21.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.700+0000 7f1b51a8f640 1 --2- 172.21.15.120:0/65537436 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1b380a3240 0x7f1b380a5700 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f1b44004af0 tx=0x7f1b44004a80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:21.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.700+0000 7f1b558cb740 1 -- start start 2024-09-19T09:27:21.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.700+0000 7f1b558cb740 1 Processor -- start 2024-09-19T09:27:21.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.701+0000 7f1b558cb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc5bf915a0 0x55bc5bf9bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:21.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.701+0000 7f1b558cb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc5bf92b10 0x55bc5bf990a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:21.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.701+0000 7f1b558cb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc5bf9c340 0x55bc5bf99830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:21.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.701+0000 7f1b5128e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc5bf915a0 0x55bc5bf9bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:21.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.701+0000 7f1b558cb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bc5bf9f040 con 0x55bc5bf915a0 2024-09-19T09:27:21.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.701+0000 7f1b5128e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc5bf915a0 0x55bc5bf9bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42404/0 (socket says 172.21.15.120:42404) 2024-09-19T09:27:21.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.701+0000 7f1b5128e640 1 -- 172.21.15.120:0/2489238786 learned_addr learned my addr 172.21.15.120:0/2489238786 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:21.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.701+0000 7f1b558cb740 1 -- 172.21.15.120:0/2489238786 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bc5beb2060 con 0x55bc5bf9c340 2024-09-19T09:27:21.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.701+0000 7f1b558cb740 1 -- 172.21.15.120:0/2489238786 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bc5bf9f1c0 con 0x55bc5bf92b10 2024-09-19T09:27:21.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.701+0000 7f1b51a8f640 1 --2- 172.21.15.120:0/2489238786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc5bf9c340 0x55bc5bf99830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:21.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.701+0000 7f1b50a8d640 1 --2- 172.21.15.120:0/2489238786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc5bf92b10 0x55bc5bf990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:21.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.702+0000 7f1b558cb740 10 client.0 _ll_register_callbacks cb 0x55bc5bc058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:21.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.702+0000 7f1b5128e640 1 -- 172.21.15.120:0/2489238786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc5bf92b10 msgr2=0x55bc5bf990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:21.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.702+0000 7f1b5128e640 1 --2- 172.21.15.120:0/2489238786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc5bf92b10 0x55bc5bf990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.702+0000 7f1b5128e640 1 -- 172.21.15.120:0/2489238786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc5bf9c340 msgr2=0x55bc5bf99830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:21.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.702+0000 7f1b5128e640 1 --2- 172.21.15.120:0/2489238786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc5bf9c340 0x55bc5bf99830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.702+0000 7f1b5128e640 1 -- 172.21.15.120:0/2489238786 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc5bc683c0 con 0x55bc5bf915a0 2024-09-19T09:27:21.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.702+0000 7f1b5128e640 1 --2- 172.21.15.120:0/2489238786 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc5bf915a0 0x55bc5bf9bdf0 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f1b480116f0 tx=0x7f1b48011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a55f387587c9bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:21.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.702+0000 7f1b34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:27:21.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.702+0000 7f1b34ff9640 1 -- 172.21.15.120:0/2489238786 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4801e060 con 0x55bc5bf915a0 2024-09-19T09:27:21.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.702+0000 7f1b558cb740 1 -- 172.21.15.120:0/2489238786 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b3c0dd070 con 0x55bc5bf915a0 2024-09-19T09:27:21.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.702+0000 7f1b34ff9640 1 -- 172.21.15.120:0/2489238786 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f1b4800fe30 con 0x55bc5bf915a0 2024-09-19T09:27:21.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.703+0000 7f1b34ff9640 1 -- 172.21.15.120:0/2489238786 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1b48019430 con 0x55bc5bf915a0 2024-09-19T09:27:21.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.703+0000 7f1b558cb740 10 client.23361 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:21.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.703+0000 7f1b558cb740 1 -- 172.21.15.120:0/2489238786 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55bc5bfaac20 con 0x55bc5bf915a0 2024-09-19T09:27:21.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.703+0000 7f1b34ff9640 1 -- 172.21.15.120:0/2489238786 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b3c0dd070 con 0x55bc5bf915a0 2024-09-19T09:27:21.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.703+0000 7f1b34ff9640 1 -- 172.21.15.120:0/2489238786 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f1b48023070 con 0x55bc5bf915a0 2024-09-19T09:27:21.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.703+0000 7f1b558cb740 10 client.23361 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:21.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.703+0000 7f1b558cb740 10 client.23361 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:27:21.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.703+0000 7f1b558cb740 20 client.23361 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:21.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.703+0000 7f1b558cb740 1 --2- 172.21.15.120:0/2489238786 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bc5bfac030 0x55bc5bfae480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:21.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.703+0000 7f1b558cb740 4 client.23361 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:21.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.703+0000 7f1b558cb740 1 -- 172.21.15.120:0/2489238786 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc5bf04860 con 0x55bc5bfac030 2024-09-19T09:27:21.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.704+0000 7f1b50a8d640 1 --2- 172.21.15.120:0/2489238786 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bc5bfac030 0x55bc5bfae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:21.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.704+0000 7f1b50a8d640 1 --2- 172.21.15.120:0/2489238786 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bc5bfac030 0x55bc5bfae480 crc :-1 s=READY pgs=595 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54bdbe90175e2325 server_cookie=35189d0cf17bc685 in_seq=0 out_seq=0 2024-09-19T09:27:21.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.704+0000 7f1b34ff9640 10 client.23361 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:21.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.705+0000 7f1b34ff9640 1 -- 172.21.15.120:0/2489238786 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc5bf04860 con 0x55bc5bfac030 2024-09-19T09:27:21.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.705+0000 7f1b34ff9640 10 client.23361 handle_command_reply: tid=0 2024-09-19T09:27:21.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.705+0000 7f1b558cb740 1 client.23361 shutdown 2024-09-19T09:27:21.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.705+0000 7f1b558cb740 1 -- 172.21.15.120:0/2489238786 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc5bf915a0 msgr2=0x55bc5bf9bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:21.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.705+0000 7f1b558cb740 1 --2- 172.21.15.120:0/2489238786 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc5bf915a0 0x55bc5bf9bdf0 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f1b480116f0 tx=0x7f1b48011bb0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.705+0000 7f1b558cb740 1 -- 172.21.15.120:0/2489238786 shutdown_connections 2024-09-19T09:27:21.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.705+0000 7f1b558cb740 1 --2- 172.21.15.120:0/2489238786 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bc5bfac030 0x55bc5bfae480 crc :-1 s=READY pgs=595 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.705+0000 7f1b558cb740 1 --2- 172.21.15.120:0/2489238786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc5bf9c340 0x55bc5bf99830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.706+0000 7f1b34ff9640 0 client.23361 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:21.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.706+0000 7f1b558cb740 1 --2- 172.21.15.120:0/2489238786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc5bf92b10 0x55bc5bf990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.706+0000 7f1b558cb740 1 --2- 172.21.15.120:0/2489238786 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc5bf915a0 0x55bc5bf9bdf0 unknown :-1 s=CLOSED pgs=2472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.706+0000 7f1b558cb740 1 -- 172.21.15.120:0/2489238786 >> 172.21.15.120:0/2489238786 conn(0x55bc5bf5a310 msgr2=0x55bc5bf93930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:21.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.706+0000 7f1b558cb740 1 -- 172.21.15.120:0/2489238786 shutdown_connections 2024-09-19T09:27:21.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.706+0000 7f1b558cb740 1 -- 172.21.15.120:0/2489238786 wait complete. 2024-09-19T09:27:21.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.706+0000 7f1b558cb740 20 client.23361 trim_cache size 0 max 16384 2024-09-19T09:27:21.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.719+0000 7f1b558cb740 1 Processor -- start 2024-09-19T09:27:21.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.719+0000 7f1b558cb740 1 -- start start 2024-09-19T09:27:21.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.719+0000 7f1b558cb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc5bf915a0 0x55bc5bf923e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:21.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.719+0000 7f1b558cb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc5bf92b10 0x55bc5bf8dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:21.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.720+0000 7f1b558cb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc5bf9c340 0x55bc5bf98960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:21.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.720+0000 7f1b558cb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c1b3c80 con 0x55bc5bf92b10 2024-09-19T09:27:21.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.720+0000 7f1b558cb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c1b57c0 con 0x55bc5bf915a0 2024-09-19T09:27:21.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.720+0000 7f1b558cb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c1b4350 con 0x55bc5bf9c340 2024-09-19T09:27:21.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.720+0000 7f1b50a8d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc5bf92b10 0x55bc5bf8dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:21.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.720+0000 7f1b50a8d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc5bf92b10 0x55bc5bf8dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42418/0 (socket says 172.21.15.120:42418) 2024-09-19T09:27:21.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.720+0000 7f1b50a8d640 1 -- 172.21.15.120:0/3603778602 learned_addr learned my addr 172.21.15.120:0/3603778602 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:21.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.720+0000 7f1b5128e640 1 --2- 172.21.15.120:0/3603778602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc5bf915a0 0x55bc5bf923e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:21.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.720+0000 7f1b51a8f640 1 --2- 172.21.15.120:0/3603778602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc5bf9c340 0x55bc5bf98960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:21.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.720+0000 7f1b50a8d640 1 -- 172.21.15.120:0/3603778602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc5bf9c340 msgr2=0x55bc5bf98960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:21.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.720+0000 7f1b50a8d640 1 --2- 172.21.15.120:0/3603778602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc5bf9c340 0x55bc5bf98960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.720+0000 7f1b50a8d640 1 -- 172.21.15.120:0/3603778602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc5bf915a0 msgr2=0x55bc5bf923e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:21.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.720+0000 7f1b50a8d640 1 --2- 172.21.15.120:0/3603778602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc5bf915a0 0x55bc5bf923e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.720+0000 7f1b50a8d640 1 -- 172.21.15.120:0/3603778602 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc5bf8e930 con 0x55bc5bf92b10 2024-09-19T09:27:21.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.720+0000 7f1b50a8d640 1 --2- 172.21.15.120:0/3603778602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc5bf92b10 0x55bc5bf8dde0 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c098850 tx=0x7f1b3c0d37a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1754f3633affa8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:21.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.720+0000 7f1b36ffd640 1 -- 172.21.15.120:0/3603778602 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c09bb70 con 0x55bc5bf92b10 2024-09-19T09:27:21.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.720+0000 7f1b36ffd640 1 -- 172.21.15.120:0/3603778602 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b3c0dd070 con 0x55bc5bf92b10 2024-09-19T09:27:21.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.720+0000 7f1b36ffd640 1 -- 172.21.15.120:0/3603778602 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c09bd50 con 0x55bc5bf92b10 2024-09-19T09:27:21.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.721+0000 7f1b558cb740 1 -- 172.21.15.120:0/3603778602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc5bf92b10 msgr2=0x55bc5bf8dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:21.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.721+0000 7f1b558cb740 1 --2- 172.21.15.120:0/3603778602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc5bf92b10 0x55bc5bf8dde0 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c098850 tx=0x7f1b3c0d37a0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.721+0000 7f1b558cb740 1 -- 172.21.15.120:0/3603778602 shutdown_connections 2024-09-19T09:27:21.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.721+0000 7f1b558cb740 1 --2- 172.21.15.120:0/3603778602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc5bf9c340 0x55bc5bf98960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.721+0000 7f1b558cb740 1 --2- 172.21.15.120:0/3603778602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc5bf92b10 0x55bc5bf8dde0 unknown :-1 s=CLOSED pgs=2473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.721+0000 7f1b558cb740 1 --2- 172.21.15.120:0/3603778602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc5bf915a0 0x55bc5bf923e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.721+0000 7f1b558cb740 1 -- 172.21.15.120:0/3603778602 >> 172.21.15.120:0/3603778602 conn(0x55bc5bf5a310 msgr2=0x55bc5bf8d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:21.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.721+0000 7f1b558cb740 1 -- 172.21.15.120:0/3603778602 shutdown_connections 2024-09-19T09:27:21.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.721+0000 7f1b558cb740 1 -- 172.21.15.120:0/3603778602 wait complete. 2024-09-19T09:27:21.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.722+0000 7f1b558cb740 1 -- start start 2024-09-19T09:27:21.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.722+0000 7f1b558cb740 1 Processor -- start 2024-09-19T09:27:21.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.722+0000 7f1b558cb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc5bf915a0 0x55bc5bfa49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:21.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.722+0000 7f1b558cb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc5bf92b10 0x55bc5bf9fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:21.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.722+0000 7f1b558cb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc5bf9c340 0x55bc5bfa0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:21.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.722+0000 7f1b558cb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c1b50f0 con 0x55bc5bf915a0 2024-09-19T09:27:21.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.722+0000 7f1b558cb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c1b4a20 con 0x55bc5bf92b10 2024-09-19T09:27:21.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.722+0000 7f1b558cb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c1b1c40 con 0x55bc5bf9c340 2024-09-19T09:27:21.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.722+0000 7f1b558cb740 10 client.0 _ll_register_callbacks cb 0x7f1b48009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:21.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.722+0000 7f1b50a8d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc5bf92b10 0x55bc5bf9fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:21.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.722+0000 7f1b5128e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc5bf915a0 0x55bc5bfa49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:21.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.722+0000 7f1b50a8d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc5bf92b10 0x55bc5bf9fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:40014/0 (socket says 172.21.15.120:40014) 2024-09-19T09:27:21.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.722+0000 7f1b50a8d640 1 -- 172.21.15.120:0/1372099977 learned_addr learned my addr 172.21.15.120:0/1372099977 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:21.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.722+0000 7f1b51a8f640 1 --2- 172.21.15.120:0/1372099977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc5bf9c340 0x55bc5bfa0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:21.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.723+0000 7f1b50a8d640 1 -- 172.21.15.120:0/1372099977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc5bf9c340 msgr2=0x55bc5bfa0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:21.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.723+0000 7f1b50a8d640 1 --2- 172.21.15.120:0/1372099977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc5bf9c340 0x55bc5bfa0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.723+0000 7f1b50a8d640 1 -- 172.21.15.120:0/1372099977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc5bf915a0 msgr2=0x55bc5bfa49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:21.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.723+0000 7f1b50a8d640 1 --2- 172.21.15.120:0/1372099977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc5bf915a0 0x55bc5bfa49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.723+0000 7f1b50a8d640 1 -- 172.21.15.120:0/1372099977 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc5bfa0880 con 0x55bc5bf92b10 2024-09-19T09:27:21.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.723+0000 7f1b50a8d640 1 --2- 172.21.15.120:0/1372099977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc5bf92b10 0x55bc5bf9fd00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c0a4a80 tx=0x7f1b3c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=265bba5386dd4148 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:21.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.723+0000 7f1b34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:27:21.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.723+0000 7f1b34ff9640 1 -- 172.21.15.120:0/1372099977 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c09bf30 con 0x55bc5bf92b10 2024-09-19T09:27:21.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.723+0000 7f1b558cb740 1 -- 172.21.15.120:0/1372099977 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b3c0dd070 con 0x55bc5bf92b10 2024-09-19T09:27:21.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.723+0000 7f1b34ff9640 1 -- 172.21.15.120:0/1372099977 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f1b3c0cd930 con 0x55bc5bf92b10 2024-09-19T09:27:21.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.723+0000 7f1b34ff9640 1 -- 172.21.15.120:0/1372099977 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1b3c09c1c0 con 0x55bc5bf92b10 2024-09-19T09:27:21.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.723+0000 7f1b558cb740 10 client.31991 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:21.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.723+0000 7f1b558cb740 1 -- 172.21.15.120:0/1372099977 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55bc5c016b50 con 0x55bc5bf92b10 2024-09-19T09:27:21.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.723+0000 7f1b34ff9640 1 -- 172.21.15.120:0/1372099977 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b3c0dd070 con 0x55bc5bf92b10 2024-09-19T09:27:21.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.724+0000 7f1b34ff9640 1 -- 172.21.15.120:0/1372099977 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f1b3c0cdce0 con 0x55bc5bf92b10 2024-09-19T09:27:21.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.724+0000 7f1b558cb740 10 client.31991 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:21.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.724+0000 7f1b558cb740 10 client.31991 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:27:21.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.724+0000 7f1b558cb740 20 client.31991 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:21.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.724+0000 7f1b558cb740 1 --2- 172.21.15.120:0/1372099977 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bc5bfac030 0x55bc5c018cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:21.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.724+0000 7f1b558cb740 4 client.31991 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:27:21.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.724+0000 7f1b558cb740 1 -- 172.21.15.120:0/1372099977 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc5befed70 con 0x55bc5bfac030 2024-09-19T09:27:21.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.724+0000 7f1b5128e640 1 --2- 172.21.15.120:0/1372099977 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bc5bfac030 0x55bc5c018cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:21.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.724+0000 7f1b5128e640 1 --2- 172.21.15.120:0/1372099977 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bc5bfac030 0x55bc5c018cf0 crc :-1 s=READY pgs=596 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5ee6b8bf7f77875 server_cookie=38832f8af7409c72 in_seq=0 out_seq=0 2024-09-19T09:27:21.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.724+0000 7f1b34ff9640 10 client.31991 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:21.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.724+0000 7f1b34ff9640 1 -- 172.21.15.120:0/1372099977 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc5befed70 con 0x55bc5bfac030 2024-09-19T09:27:21.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.724+0000 7f1b34ff9640 10 client.31991 handle_command_reply: tid=0 2024-09-19T09:27:21.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.724+0000 7f1b558cb740 1 client.31991 shutdown 2024-09-19T09:27:21.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.725+0000 7f1b558cb740 1 -- 172.21.15.120:0/1372099977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc5bf92b10 msgr2=0x55bc5bf9fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:21.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.725+0000 7f1b558cb740 1 --2- 172.21.15.120:0/1372099977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc5bf92b10 0x55bc5bf9fd00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c0a4a80 tx=0x7f1b3c000bc0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.725+0000 7f1b558cb740 1 -- 172.21.15.120:0/1372099977 shutdown_connections 2024-09-19T09:27:21.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.725+0000 7f1b558cb740 1 --2- 172.21.15.120:0/1372099977 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55bc5bfac030 0x55bc5c018cf0 crc :-1 s=READY pgs=596 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.725+0000 7f1b558cb740 1 --2- 172.21.15.120:0/1372099977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc5bf9c340 0x55bc5bfa0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.725+0000 7f1b558cb740 1 --2- 172.21.15.120:0/1372099977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc5bf92b10 0x55bc5bf9fd00 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.725+0000 7f1b558cb740 1 --2- 172.21.15.120:0/1372099977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc5bf915a0 0x55bc5bfa49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.725+0000 7f1b558cb740 1 -- 172.21.15.120:0/1372099977 >> 172.21.15.120:0/1372099977 conn(0x55bc5bf5a310 msgr2=0x55bc5bf8c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:21.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.725+0000 7f1b34ff9640 0 client.31991 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:21.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.725+0000 7f1b558cb740 1 -- 172.21.15.120:0/1372099977 shutdown_connections 2024-09-19T09:27:21.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.725+0000 7f1b558cb740 1 -- 172.21.15.120:0/1372099977 wait complete. 2024-09-19T09:27:21.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.725+0000 7f1b558cb740 20 client.31991 trim_cache size 0 max 16384 2024-09-19T09:27:21.733 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:27:21.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.725+0000 7f1b534f2640 1 -- 172.21.15.120:0/65537436 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1b380a3240 msgr2=0x7f1b380a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:21.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.725+0000 7f1b534f2640 1 --2- 172.21.15.120:0/65537436 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1b380a3240 0x7f1b380a5700 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f1b44004af0 tx=0x7f1b44004a80 comp rx=0 tx=0).stop 2024-09-19T09:27:21.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.725+0000 7f1b534f2640 1 -- 172.21.15.120:0/65537436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b4c100aa0 msgr2=0x7f1b4c10c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:21.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.725+0000 7f1b534f2640 1 --2- 172.21.15.120:0/65537436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b4c100aa0 0x7f1b4c10c140 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c00e9f0 tx=0x7f1b3c00eeb0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.725+0000 7f1b534f2640 1 -- 172.21.15.120:0/65537436 shutdown_connections 2024-09-19T09:27:21.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.725+0000 7f1b534f2640 1 --2- 172.21.15.120:0/65537436 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f1b380a3240 0x7f1b380a5700 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.725+0000 7f1b534f2640 1 --2- 172.21.15.120:0/65537436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b4c106b80 0x7f1b4c1a33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.725+0000 7f1b534f2640 1 --2- 172.21.15.120:0/65537436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b4c100aa0 0x7f1b4c10c140 unknown :-1 s=CLOSED pgs=2471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.725+0000 7f1b534f2640 1 --2- 172.21.15.120:0/65537436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b4c100100 0x7f1b4c10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.725+0000 7f1b534f2640 1 -- 172.21.15.120:0/65537436 >> 172.21.15.120:0/65537436 conn(0x7f1b4c077230 msgr2=0x7f1b4c1057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:21.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.726+0000 7f1b534f2640 1 -- 172.21.15.120:0/65537436 shutdown_connections 2024-09-19T09:27:21.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:21.726+0000 7f1b534f2640 1 -- 172.21.15.120:0/65537436 wait complete. 2024-09-19T09:27:22.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:22 smithi120 ceph-mon[31150]: pgmap v1080: 97 pgs: 97 active+clean; 510 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.5 MiB/s wr, 422 op/s 2024-09-19T09:27:22.393 INFO:tasks.workunit.client.0.smithi120.stdout: 10 266983 20.47 MB/sec execute 557 sec latency 11376.512 ms 2024-09-19T09:27:22.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:22 smithi196 ceph-mon[35150]: pgmap v1080: 97 pgs: 97 active+clean; 510 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.5 MiB/s wr, 422 op/s 2024-09-19T09:27:22.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:22 smithi153 ceph-mon[35071]: pgmap v1080: 97 pgs: 97 active+clean; 510 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.5 MiB/s wr, 422 op/s 2024-09-19T09:27:23.393 INFO:tasks.workunit.client.0.smithi120.stdout: 10 266983 20.43 MB/sec execute 558 sec latency 12376.572 ms 2024-09-19T09:27:23.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.953+0000 7f0d48237640 1 -- 172.21.15.120:0/1194391083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d40102e00 msgr2=0x7f0d40103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.953+0000 7f0d48237640 1 --2- 172.21.15.120:0/1194391083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d40102e00 0x7f0d40103280 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009900 tx=0x7f0d3002a410 comp rx=0 tx=0).stop 2024-09-19T09:27:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.953+0000 7f0d48237640 1 -- 172.21.15.120:0/1194391083 shutdown_connections 2024-09-19T09:27:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.953+0000 7f0d48237640 1 --2- 172.21.15.120:0/1194391083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d401037c0 0x7f0d4010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.953+0000 7f0d48237640 1 --2- 172.21.15.120:0/1194391083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d40102e00 0x7f0d40103280 unknown :-1 s=CLOSED pgs=2474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.953+0000 7f0d48237640 1 --2- 172.21.15.120:0/1194391083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d400ffe30 0x7f0d40100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.953+0000 7f0d48237640 1 -- 172.21.15.120:0/1194391083 >> 172.21.15.120:0/1194391083 conn(0x7f0d400fb970 msgr2=0x7f0d400fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.953+0000 7f0d48237640 1 -- 172.21.15.120:0/1194391083 shutdown_connections 2024-09-19T09:27:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.953+0000 7f0d48237640 1 -- 172.21.15.120:0/1194391083 wait complete. 2024-09-19T09:27:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.954+0000 7f0d48237640 1 Processor -- start 2024-09-19T09:27:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.954+0000 7f0d48237640 1 -- start start 2024-09-19T09:27:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.954+0000 7f0d48237640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d400ffe30 0x7f0d40197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.954+0000 7f0d48237640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d40102e00 0x7f0d40198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.954+0000 7f0d48237640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d401037c0 0x7f0d4019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.954+0000 7f0d48237640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4010d7e0 con 0x7f0d401037c0 2024-09-19T09:27:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.954+0000 7f0d48237640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4010d660 con 0x7f0d40102e00 2024-09-19T09:27:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.954+0000 7f0d48237640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4010d960 con 0x7f0d400ffe30 2024-09-19T09:27:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.954+0000 7f0d467d4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d401037c0 0x7f0d4019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.954+0000 7f0d467d4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d401037c0 0x7f0d4019f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42460/0 (socket says 172.21.15.120:42460) 2024-09-19T09:27:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.954+0000 7f0d467d4640 1 -- 172.21.15.120:0/1161866946 learned_addr learned my addr 172.21.15.120:0/1161866946 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.954+0000 7f0d457d2640 1 --2- 172.21.15.120:0/1161866946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d40102e00 0x7f0d40198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.954+0000 7f0d45fd3640 1 --2- 172.21.15.120:0/1161866946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d400ffe30 0x7f0d40197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.954+0000 7f0d467d4640 1 -- 172.21.15.120:0/1161866946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d400ffe30 msgr2=0x7f0d40197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.955+0000 7f0d467d4640 1 --2- 172.21.15.120:0/1161866946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d400ffe30 0x7f0d40197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.955+0000 7f0d467d4640 1 -- 172.21.15.120:0/1161866946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d40102e00 msgr2=0x7f0d40198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.955+0000 7f0d467d4640 1 --2- 172.21.15.120:0/1161866946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d40102e00 0x7f0d40198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.955+0000 7f0d467d4640 1 -- 172.21.15.120:0/1161866946 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4019f9f0 con 0x7f0d401037c0 2024-09-19T09:27:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.955+0000 7f0d467d4640 1 --2- 172.21.15.120:0/1161866946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d401037c0 0x7f0d4019f380 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800cce0 tx=0x7f0d3800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.955+0000 7f0d36ffd640 1 -- 172.21.15.120:0/1161866946 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3800f450 con 0x7f0d401037c0 2024-09-19T09:27:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.955+0000 7f0d36ffd640 1 -- 172.21.15.120:0/1161866946 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d3800fad0 con 0x7f0d401037c0 2024-09-19T09:27:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.955+0000 7f0d48237640 1 -- 172.21.15.120:0/1161866946 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4019fce0 con 0x7f0d401037c0 2024-09-19T09:27:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.955+0000 7f0d36ffd640 1 -- 172.21.15.120:0/1161866946 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38018c10 con 0x7f0d401037c0 2024-09-19T09:27:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.955+0000 7f0d48237640 1 -- 172.21.15.120:0/1161866946 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d401a01b0 con 0x7f0d401037c0 2024-09-19T09:27:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.957+0000 7f0d36ffd640 1 -- 172.21.15.120:0/1161866946 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f0d3800f5f0 con 0x7f0d401037c0 2024-09-19T09:27:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.957+0000 7f0d4afcf740 1 Processor -- start 2024-09-19T09:27:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.958+0000 7f0d4afcf740 1 -- start start 2024-09-19T09:27:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.958+0000 7f0d4afcf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56381b44bdb0 0x56381b44c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.958+0000 7f0d4afcf740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56381b44c6f0 0x56381b4511b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.958+0000 7f0d4afcf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56381b451780 0x56381b453c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.958+0000 7f0d4afcf740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56381b3ab2f0 con 0x56381b44c6f0 2024-09-19T09:27:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.958+0000 7f0d4afcf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56381b353270 con 0x56381b44bdb0 2024-09-19T09:27:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.958+0000 7f0d4afcf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56381b3a72b0 con 0x56381b451780 2024-09-19T09:27:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.958+0000 7f0d45fd3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56381b44c6f0 0x56381b4511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.958+0000 7f0d45fd3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56381b44c6f0 0x56381b4511b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42470/0 (socket says 172.21.15.120:42470) 2024-09-19T09:27:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.958+0000 7f0d45fd3640 1 -- 172.21.15.120:0/2196568230 learned_addr learned my addr 172.21.15.120:0/2196568230 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.958+0000 7f0d457d2640 1 --2- 172.21.15.120:0/2196568230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56381b451780 0x56381b453c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.958+0000 7f0d457d2640 1 --2- 172.21.15.120:0/2196568230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56381b44bdb0 0x56381b44c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.958+0000 7f0d36ffd640 1 --2- 172.21.15.120:0/1161866946 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0d2c0a32f0 0x7f0d2c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.958+0000 7f0d45fd3640 1 -- 172.21.15.120:0/2196568230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56381b451780 msgr2=0x56381b453c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.958+0000 7f0d45fd3640 1 --2- 172.21.15.120:0/2196568230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56381b451780 0x56381b453c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.958+0000 7f0d45fd3640 1 -- 172.21.15.120:0/2196568230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56381b44bdb0 msgr2=0x56381b44c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.958+0000 7f0d45fd3640 1 --2- 172.21.15.120:0/2196568230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56381b44bdb0 0x56381b44c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.958+0000 7f0d45fd3640 1 -- 172.21.15.120:0/2196568230 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56381b352340 con 0x56381b44c6f0 2024-09-19T09:27:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.958+0000 7f0d467d4640 1 --2- 172.21.15.120:0/1161866946 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0d2c0a32f0 0x7f0d2c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.958+0000 7f0d36ffd640 1 -- 172.21.15.120:0/1161866946 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f0d380d0000 con 0x7f0d401037c0 2024-09-19T09:27:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.958+0000 7f0d45fd3640 1 --2- 172.21.15.120:0/2196568230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56381b44c6f0 0x56381b4511b0 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00bc90 tx=0x7f0d3c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdd29f422664e738 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.959+0000 7f0d2bfff640 1 -- 172.21.15.120:0/2196568230 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c015070 con 0x56381b44c6f0 2024-09-19T09:27:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.959+0000 7f0d2bfff640 1 -- 172.21.15.120:0/2196568230 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d3c0035f0 con 0x56381b44c6f0 2024-09-19T09:27:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.959+0000 7f0d2bfff640 1 -- 172.21.15.120:0/2196568230 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c0116a0 con 0x56381b44c6f0 2024-09-19T09:27:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.959+0000 7f0d4afcf740 1 -- 172.21.15.120:0/2196568230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56381b44c6f0 msgr2=0x56381b4511b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.959+0000 7f0d467d4640 1 --2- 172.21.15.120:0/1161866946 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0d2c0a32f0 0x7f0d2c0a57b0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0d3801f000 tx=0x7f0d3800af30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.959+0000 7f0d4afcf740 1 --2- 172.21.15.120:0/2196568230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56381b44c6f0 0x56381b4511b0 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00bc90 tx=0x7f0d3c00f730 comp rx=0 tx=0).stop 2024-09-19T09:27:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.959+0000 7f0d4afcf740 1 -- 172.21.15.120:0/2196568230 shutdown_connections 2024-09-19T09:27:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.959+0000 7f0d4afcf740 1 --2- 172.21.15.120:0/2196568230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56381b451780 0x56381b453c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.959+0000 7f0d4afcf740 1 --2- 172.21.15.120:0/2196568230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56381b44c6f0 0x56381b4511b0 unknown :-1 s=CLOSED pgs=2476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.959+0000 7f0d4afcf740 1 --2- 172.21.15.120:0/2196568230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56381b44bdb0 0x56381b44c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.959+0000 7f0d4afcf740 1 -- 172.21.15.120:0/2196568230 >> 172.21.15.120:0/2196568230 conn(0x56381b443010 msgr2=0x56381b443410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.959+0000 7f0d4afcf740 1 -- 172.21.15.120:0/2196568230 shutdown_connections 2024-09-19T09:27:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.959+0000 7f0d4afcf740 1 -- 172.21.15.120:0/2196568230 wait complete. 2024-09-19T09:27:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.960+0000 7f0d4afcf740 1 -- start start 2024-09-19T09:27:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.960+0000 7f0d4afcf740 1 Processor -- start 2024-09-19T09:27:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.960+0000 7f0d4afcf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56381b44bdb0 0x56381b456010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.960+0000 7f0d4afcf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56381b44c6f0 0x56381b456550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.960+0000 7f0d4afcf740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56381b451780 0x56381b45d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.960+0000 7f0d4afcf740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56381b454750 con 0x56381b451780 2024-09-19T09:27:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.960+0000 7f0d4afcf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56381b334290 con 0x56381b44bdb0 2024-09-19T09:27:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.960+0000 7f0d4afcf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56381b4548d0 con 0x56381b44c6f0 2024-09-19T09:27:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.960+0000 7f0d4afcf740 10 client.0 _ll_register_callbacks cb 0x56381b365140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.960+0000 7f0d457d2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56381b451780 0x56381b45d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.960+0000 7f0d457d2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56381b451780 0x56381b45d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42482/0 (socket says 172.21.15.120:42482) 2024-09-19T09:27:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.960+0000 7f0d457d2640 1 -- 172.21.15.120:0/1232761585 learned_addr learned my addr 172.21.15.120:0/1232761585 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.960+0000 7f0d457d2640 1 --2- 172.21.15.120:0/1232761585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56381b44bdb0 0x56381b456010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.961+0000 7f0d45fd3640 1 --2- 172.21.15.120:0/1232761585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56381b44c6f0 0x56381b456550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.961+0000 7f0d457d2640 1 -- 172.21.15.120:0/1232761585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56381b44c6f0 msgr2=0x56381b456550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.961+0000 7f0d457d2640 1 --2- 172.21.15.120:0/1232761585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56381b44c6f0 0x56381b456550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.961+0000 7f0d457d2640 1 -- 172.21.15.120:0/1232761585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56381b44bdb0 msgr2=0x56381b456010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.961+0000 7f0d457d2640 1 --2- 172.21.15.120:0/1232761585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56381b44bdb0 0x56381b456010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.961+0000 7f0d457d2640 1 -- 172.21.15.120:0/1232761585 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56381b0be210 con 0x56381b451780 2024-09-19T09:27:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.961+0000 7f0d45fd3640 1 --2- 172.21.15.120:0/1232761585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56381b44c6f0 0x56381b456550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.961+0000 7f0d457d2640 1 --2- 172.21.15.120:0/1232761585 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56381b451780 0x56381b45d590 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f0d30005ec0 tx=0x7f0d30040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71bf4c8bc5b6cb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.961+0000 7f0d297fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:27:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.961+0000 7f0d297fa640 1 -- 172.21.15.120:0/1232761585 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30035070 con 0x56381b451780 2024-09-19T09:27:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.961+0000 7f0d297fa640 1 -- 172.21.15.120:0/1232761585 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f0d30004310 con 0x56381b451780 2024-09-19T09:27:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.961+0000 7f0d4afcf740 1 -- 172.21.15.120:0/1232761585 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d3c0035f0 con 0x56381b451780 2024-09-19T09:27:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.961+0000 7f0d297fa640 1 -- 172.21.15.120:0/1232761585 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0d30052050 con 0x56381b451780 2024-09-19T09:27:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.961+0000 7f0d4afcf740 10 client.23379 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.961+0000 7f0d4afcf740 1 -- 172.21.15.120:0/1232761585 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56381b18a620 con 0x56381b451780 2024-09-19T09:27:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.961+0000 7f0d297fa640 1 -- 172.21.15.120:0/1232761585 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d3c0035f0 con 0x56381b451780 2024-09-19T09:27:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.961+0000 7f0d297fa640 1 -- 172.21.15.120:0/1232761585 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f0d30004b70 con 0x56381b451780 2024-09-19T09:27:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.961+0000 7f0d4afcf740 10 client.23379 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.961+0000 7f0d4afcf740 10 client.23379 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:27:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.961+0000 7f0d4afcf740 20 client.23379 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.961+0000 7f0d4afcf740 1 --2- 172.21.15.120:0/1232761585 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56381b461c30 0x56381b464060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.961+0000 7f0d4afcf740 4 client.23379 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.961+0000 7f0d4afcf740 1 -- 172.21.15.120:0/1232761585 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56381b3b8a20 con 0x56381b461c30 2024-09-19T09:27:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.962+0000 7f0d45fd3640 1 --2- 172.21.15.120:0/1232761585 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56381b461c30 0x56381b464060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.962+0000 7f0d297fa640 10 client.23379 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.962+0000 7f0d45fd3640 1 --2- 172.21.15.120:0/1232761585 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56381b461c30 0x56381b464060 crc :-1 s=READY pgs=597 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6cf955a3171966f server_cookie=f2ecd92beb9c1609 in_seq=0 out_seq=0 2024-09-19T09:27:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.963+0000 7f0d297fa640 1 -- 172.21.15.120:0/1232761585 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56381b3b8a20 con 0x56381b461c30 2024-09-19T09:27:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.963+0000 7f0d297fa640 10 client.23379 handle_command_reply: tid=0 2024-09-19T09:27:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.963+0000 7f0d4afcf740 1 client.23379 shutdown 2024-09-19T09:27:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.963+0000 7f0d4afcf740 1 -- 172.21.15.120:0/1232761585 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56381b451780 msgr2=0x56381b45d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.963+0000 7f0d4afcf740 1 --2- 172.21.15.120:0/1232761585 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56381b451780 0x56381b45d590 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f0d30005ec0 tx=0x7f0d30040040 comp rx=0 tx=0).stop 2024-09-19T09:27:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.963+0000 7f0d4afcf740 1 -- 172.21.15.120:0/1232761585 shutdown_connections 2024-09-19T09:27:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.963+0000 7f0d4afcf740 1 --2- 172.21.15.120:0/1232761585 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56381b461c30 0x56381b464060 crc :-1 s=READY pgs=597 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.963+0000 7f0d4afcf740 1 --2- 172.21.15.120:0/1232761585 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56381b451780 0x56381b45d590 unknown :-1 s=CLOSED pgs=2477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.963+0000 7f0d4afcf740 1 --2- 172.21.15.120:0/1232761585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56381b44c6f0 0x56381b456550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.963+0000 7f0d297fa640 0 client.23379 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.963+0000 7f0d4afcf740 1 --2- 172.21.15.120:0/1232761585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56381b44bdb0 0x56381b456010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.963+0000 7f0d4afcf740 1 -- 172.21.15.120:0/1232761585 >> 172.21.15.120:0/1232761585 conn(0x56381b443010 msgr2=0x56381b444820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.963+0000 7f0d4afcf740 1 -- 172.21.15.120:0/1232761585 shutdown_connections 2024-09-19T09:27:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.963+0000 7f0d4afcf740 1 -- 172.21.15.120:0/1232761585 wait complete. 2024-09-19T09:27:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.963+0000 7f0d4afcf740 20 client.23379 trim_cache size 0 max 16384 2024-09-19T09:27:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.977+0000 7f0d4afcf740 1 Processor -- start 2024-09-19T09:27:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.977+0000 7f0d4afcf740 1 -- start start 2024-09-19T09:27:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.977+0000 7f0d4afcf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56381b44bdb0 0x56381b4420b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.977+0000 7f0d4afcf740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56381b44c6f0 0x56381b45d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.977+0000 7f0d4afcf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56381b451780 0x56381b468960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.977+0000 7f0d4afcf740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0d401a6de0 con 0x56381b44c6f0 2024-09-19T09:27:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.977+0000 7f0d4afcf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0d401a8920 con 0x56381b451780 2024-09-19T09:27:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.977+0000 7f0d4afcf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0d401a74b0 con 0x56381b44bdb0 2024-09-19T09:27:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.977+0000 7f0d45fd3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56381b44c6f0 0x56381b45d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.977+0000 7f0d45fd3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56381b44c6f0 0x56381b45d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42488/0 (socket says 172.21.15.120:42488) 2024-09-19T09:27:23.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.977+0000 7f0d45fd3640 1 -- 172.21.15.120:0/2769190931 learned_addr learned my addr 172.21.15.120:0/2769190931 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:23.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.977+0000 7f0d457d2640 1 --2- 172.21.15.120:0/2769190931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56381b451780 0x56381b468960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:23.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.978+0000 7f0d45fd3640 1 -- 172.21.15.120:0/2769190931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56381b44bdb0 msgr2=0x56381b4420b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:23.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.978+0000 7f0d45fd3640 1 --2- 172.21.15.120:0/2769190931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56381b44bdb0 0x56381b4420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.978+0000 7f0d45fd3640 1 -- 172.21.15.120:0/2769190931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56381b451780 msgr2=0x56381b468960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:23.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.978+0000 7f0d45fd3640 1 --2- 172.21.15.120:0/2769190931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56381b451780 0x56381b468960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.978+0000 7f0d45fd3640 1 -- 172.21.15.120:0/2769190931 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56381b43e2d0 con 0x56381b44c6f0 2024-09-19T09:27:23.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.978+0000 7f0d457d2640 1 --2- 172.21.15.120:0/2769190931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56381b451780 0x56381b468960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:23.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.978+0000 7f0d45fd3640 1 --2- 172.21.15.120:0/2769190931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56381b44c6f0 0x56381b45d550 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00bf50 tx=0x7f0d3c00b440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6721a66d07e5948a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:23.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.978+0000 7f0d2b7fe640 1 -- 172.21.15.120:0/2769190931 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c015040 con 0x56381b44c6f0 2024-09-19T09:27:23.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.978+0000 7f0d2b7fe640 1 -- 172.21.15.120:0/2769190931 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d3c005520 con 0x56381b44c6f0 2024-09-19T09:27:23.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.978+0000 7f0d2b7fe640 1 -- 172.21.15.120:0/2769190931 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c003c00 con 0x56381b44c6f0 2024-09-19T09:27:23.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.978+0000 7f0d4afcf740 1 -- 172.21.15.120:0/2769190931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56381b44c6f0 msgr2=0x56381b45d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.978+0000 7f0d4afcf740 1 --2- 172.21.15.120:0/2769190931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56381b44c6f0 0x56381b45d550 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00bf50 tx=0x7f0d3c00b440 comp rx=0 tx=0).stop 2024-09-19T09:27:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.978+0000 7f0d4afcf740 1 -- 172.21.15.120:0/2769190931 shutdown_connections 2024-09-19T09:27:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.978+0000 7f0d4afcf740 1 --2- 172.21.15.120:0/2769190931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56381b451780 0x56381b468960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.978+0000 7f0d4afcf740 1 --2- 172.21.15.120:0/2769190931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56381b44c6f0 0x56381b45d550 unknown :-1 s=CLOSED pgs=2478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.978+0000 7f0d4afcf740 1 --2- 172.21.15.120:0/2769190931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56381b44bdb0 0x56381b4420b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.978+0000 7f0d4afcf740 1 -- 172.21.15.120:0/2769190931 >> 172.21.15.120:0/2769190931 conn(0x56381b443010 msgr2=0x56381b4419a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.978+0000 7f0d4afcf740 1 -- 172.21.15.120:0/2769190931 shutdown_connections 2024-09-19T09:27:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.978+0000 7f0d4afcf740 1 -- 172.21.15.120:0/2769190931 wait complete. 2024-09-19T09:27:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.979+0000 7f0d4afcf740 1 -- start start 2024-09-19T09:27:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.979+0000 7f0d4afcf740 1 Processor -- start 2024-09-19T09:27:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.979+0000 7f0d4afcf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56381b44bdb0 0x56381b45aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.979+0000 7f0d4afcf740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56381b44c6f0 0x56381b45afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.979+0000 7f0d4afcf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56381b451780 0x56381b45bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.979+0000 7f0d4afcf740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0d401a8250 con 0x56381b44c6f0 2024-09-19T09:27:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.979+0000 7f0d4afcf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0d401a7b80 con 0x56381b44bdb0 2024-09-19T09:27:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.979+0000 7f0d4afcf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0d401a4da0 con 0x56381b451780 2024-09-19T09:27:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.979+0000 7f0d4afcf740 10 client.0 _ll_register_callbacks cb 0x7f0d300316b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.979+0000 7f0d45fd3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56381b44c6f0 0x56381b45afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.979+0000 7f0d45fd3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56381b44c6f0 0x56381b45afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42496/0 (socket says 172.21.15.120:42496) 2024-09-19T09:27:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.979+0000 7f0d45fd3640 1 -- 172.21.15.120:0/2799758599 learned_addr learned my addr 172.21.15.120:0/2799758599 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.979+0000 7f0d457d2640 1 --2- 172.21.15.120:0/2799758599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56381b451780 0x56381b45bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.979+0000 7f0d457d2640 1 --2- 172.21.15.120:0/2799758599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56381b44bdb0 0x56381b45aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.980+0000 7f0d45fd3640 1 -- 172.21.15.120:0/2799758599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56381b451780 msgr2=0x56381b45bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.980+0000 7f0d45fd3640 1 --2- 172.21.15.120:0/2799758599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56381b451780 0x56381b45bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.980+0000 7f0d45fd3640 1 -- 172.21.15.120:0/2799758599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56381b44bdb0 msgr2=0x56381b45aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.980+0000 7f0d45fd3640 1 --2- 172.21.15.120:0/2799758599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56381b44bdb0 0x56381b45aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.980+0000 7f0d45fd3640 1 -- 172.21.15.120:0/2799758599 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56381b45c5e0 con 0x56381b44c6f0 2024-09-19T09:27:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.980+0000 7f0d457d2640 1 --2- 172.21.15.120:0/2799758599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56381b451780 0x56381b45bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.980+0000 7f0d45fd3640 1 --2- 172.21.15.120:0/2799758599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56381b44c6f0 0x56381b45afc0 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00b830 tx=0x7f0d3c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a936dc597ca03922 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.980+0000 7f0d297fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:27:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.980+0000 7f0d297fa640 1 -- 172.21.15.120:0/2799758599 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c015070 con 0x56381b44c6f0 2024-09-19T09:27:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.980+0000 7f0d297fa640 1 -- 172.21.15.120:0/2799758599 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f0d3c003b00 con 0x56381b44c6f0 2024-09-19T09:27:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.980+0000 7f0d4afcf740 1 -- 172.21.15.120:0/2799758599 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d3c005520 con 0x56381b44c6f0 2024-09-19T09:27:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.980+0000 7f0d297fa640 1 -- 172.21.15.120:0/2799758599 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0d3c01d020 con 0x56381b44c6f0 2024-09-19T09:27:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.980+0000 7f0d4afcf740 10 client.23385 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.980+0000 7f0d4afcf740 1 -- 172.21.15.120:0/2799758599 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56381b45d860 con 0x56381b44c6f0 2024-09-19T09:27:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.980+0000 7f0d297fa640 1 -- 172.21.15.120:0/2799758599 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d3c005520 con 0x56381b44c6f0 2024-09-19T09:27:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.980+0000 7f0d297fa640 1 -- 172.21.15.120:0/2799758599 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f0d3c025970 con 0x56381b44c6f0 2024-09-19T09:27:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.980+0000 7f0d4afcf740 10 client.23385 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.980+0000 7f0d4afcf740 10 client.23385 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:27:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.980+0000 7f0d4afcf740 20 client.23385 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.980+0000 7f0d4afcf740 1 --2- 172.21.15.120:0/2799758599 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56381b4ccef0 0x56381b4cf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.980+0000 7f0d4afcf740 4 client.23385 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:27:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.980+0000 7f0d4afcf740 1 -- 172.21.15.120:0/2799758599 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56381b3b2d70 con 0x56381b4ccef0 2024-09-19T09:27:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.981+0000 7f0d457d2640 1 --2- 172.21.15.120:0/2799758599 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56381b4ccef0 0x56381b4cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.981+0000 7f0d297fa640 10 client.23385 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:23.981+0000 7f0d457d2640 1 --2- 172.21.15.120:0/2799758599 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56381b4ccef0 0x56381b4cf310 crc :-1 s=READY pgs=598 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a982504a2ec3ce74 server_cookie=96a2167ffc60670e in_seq=0 out_seq=0 2024-09-19T09:27:24.104 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:27:24.104 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:27:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:24.173+0000 7f0d297fa640 1 -- 172.21.15.120:0/2799758599 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56381b3b2d70 con 0x56381b4ccef0 2024-09-19T09:27:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:24.174+0000 7f0d297fa640 10 client.23385 handle_command_reply: tid=0 2024-09-19T09:27:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:24.174+0000 7f0d4afcf740 1 client.23385 shutdown 2024-09-19T09:27:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:24.174+0000 7f0d4afcf740 1 -- 172.21.15.120:0/2799758599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56381b44c6f0 msgr2=0x56381b45afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:24.174+0000 7f0d4afcf740 1 --2- 172.21.15.120:0/2799758599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56381b44c6f0 0x56381b45afc0 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00b830 tx=0x7f0d3c00b860 comp rx=0 tx=0).stop 2024-09-19T09:27:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:24.174+0000 7f0d4afcf740 1 -- 172.21.15.120:0/2799758599 shutdown_connections 2024-09-19T09:27:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:24.174+0000 7f0d4afcf740 1 --2- 172.21.15.120:0/2799758599 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56381b4ccef0 0x56381b4cf310 crc :-1 s=READY pgs=598 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:24.174+0000 7f0d4afcf740 1 --2- 172.21.15.120:0/2799758599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56381b451780 0x56381b45bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:24.174+0000 7f0d297fa640 0 client.23385 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:24.174+0000 7f0d4afcf740 1 --2- 172.21.15.120:0/2799758599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56381b44c6f0 0x56381b45afc0 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:24.174+0000 7f0d4afcf740 1 --2- 172.21.15.120:0/2799758599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56381b44bdb0 0x56381b45aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:24.174+0000 7f0d4afcf740 1 -- 172.21.15.120:0/2799758599 >> 172.21.15.120:0/2799758599 conn(0x56381b443010 msgr2=0x56381b4421c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:24.174+0000 7f0d4afcf740 1 -- 172.21.15.120:0/2799758599 shutdown_connections 2024-09-19T09:27:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:24.174+0000 7f0d4afcf740 1 -- 172.21.15.120:0/2799758599 wait complete. 2024-09-19T09:27:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:24.174+0000 7f0d4afcf740 20 client.23385 trim_cache size 0 max 16384 2024-09-19T09:27:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:27:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:27:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:27:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:27:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:24.175+0000 7f0d48237640 1 -- 172.21.15.120:0/1161866946 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0d2c0a32f0 msgr2=0x7f0d2c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:24.175+0000 7f0d48237640 1 --2- 172.21.15.120:0/1161866946 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0d2c0a32f0 0x7f0d2c0a57b0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0d3801f000 tx=0x7f0d3800af30 comp rx=0 tx=0).stop 2024-09-19T09:27:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:24.175+0000 7f0d48237640 1 -- 172.21.15.120:0/1161866946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d401037c0 msgr2=0x7f0d4019f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:24.175+0000 7f0d48237640 1 --2- 172.21.15.120:0/1161866946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d401037c0 0x7f0d4019f380 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800cce0 tx=0x7f0d3800a620 comp rx=0 tx=0).stop 2024-09-19T09:27:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:24.175+0000 7f0d48237640 1 -- 172.21.15.120:0/1161866946 shutdown_connections 2024-09-19T09:27:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:24.175+0000 7f0d48237640 1 --2- 172.21.15.120:0/1161866946 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0d2c0a32f0 0x7f0d2c0a57b0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:24.175+0000 7f0d48237640 1 --2- 172.21.15.120:0/1161866946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d401037c0 0x7f0d4019f380 unknown :-1 s=CLOSED pgs=2475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:24.179 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:24.175+0000 7f0d48237640 1 --2- 172.21.15.120:0/1161866946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d40102e00 0x7f0d40198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:24.179 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:24.175+0000 7f0d48237640 1 --2- 172.21.15.120:0/1161866946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d400ffe30 0x7f0d40197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:24.179 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:24.175+0000 7f0d48237640 1 -- 172.21.15.120:0/1161866946 >> 172.21.15.120:0/1161866946 conn(0x7f0d400fb970 msgr2=0x7f0d40106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:24.179 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:24.175+0000 7f0d48237640 1 -- 172.21.15.120:0/1161866946 shutdown_connections 2024-09-19T09:27:24.179 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:24.175+0000 7f0d48237640 1 -- 172.21.15.120:0/1161866946 wait complete. 2024-09-19T09:27:24.353 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:24.393 INFO:tasks.workunit.client.0.smithi120.stdout: 10 266983 20.39 MB/sec execute 559 sec latency 13376.602 ms 2024-09-19T09:27:24.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:24 smithi196 ceph-mon[35150]: pgmap v1081: 97 pgs: 97 active+clean; 507 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 306 op/s 2024-09-19T09:27:24.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:24 smithi153 ceph-mon[35071]: pgmap v1081: 97 pgs: 97 active+clean; 507 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 306 op/s 2024-09-19T09:27:24.474 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:24 smithi120 ceph-mon[31150]: pgmap v1081: 97 pgs: 97 active+clean; 507 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 306 op/s 2024-09-19T09:27:25.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:24.999+0000 7f057c1d0640 1 -- 172.21.15.120:0/3529839476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0574100e20 msgr2=0x7f0574101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:25.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:24.999+0000 7f057c1d0640 1 --2- 172.21.15.120:0/3529839476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0574100e20 0x7f0574101200 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f0570009900 tx=0x7f057002a480 comp rx=0 tx=0).stop 2024-09-19T09:27:25.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:24.999+0000 7f057c1d0640 1 -- 172.21.15.120:0/3529839476 shutdown_connections 2024-09-19T09:27:25.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:24.999+0000 7f057c1d0640 1 --2- 172.21.15.120:0/3529839476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0574102ef0 0x7f057410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:24.999+0000 7f057c1d0640 1 --2- 172.21.15.120:0/3529839476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0574102570 0x7f05741029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:24.999+0000 7f057c1d0640 1 --2- 172.21.15.120:0/3529839476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0574100e20 0x7f0574101200 unknown :-1 s=CLOSED pgs=2480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:24.999+0000 7f057c1d0640 1 -- 172.21.15.120:0/3529839476 >> 172.21.15.120:0/3529839476 conn(0x7f05740fc7d0 msgr2=0x7f05740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:25.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:24.999+0000 7f057c1d0640 1 -- 172.21.15.120:0/3529839476 shutdown_connections 2024-09-19T09:27:25.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.000+0000 7f057c1d0640 1 -- 172.21.15.120:0/3529839476 wait complete. 2024-09-19T09:27:25.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.000+0000 7f057c1d0640 1 Processor -- start 2024-09-19T09:27:25.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.000+0000 7f057c1d0640 1 -- start start 2024-09-19T09:27:25.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.001+0000 7f057c1d0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0574100e20 0x7f0574198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:25.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.001+0000 7f057c1d0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0574102570 0x7f0574198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:25.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.001+0000 7f057c1d0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0574102ef0 0x7f057419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:25.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.001+0000 7f057c1d0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0574110f80 con 0x7f0574102ef0 2024-09-19T09:27:25.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.001+0000 7f057c1d0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0574110e00 con 0x7f0574102570 2024-09-19T09:27:25.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.001+0000 7f057c1d0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0574111100 con 0x7f0574100e20 2024-09-19T09:27:25.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.001+0000 7f057a76d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0574102ef0 0x7f057419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:25.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.001+0000 7f057976b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0574102570 0x7f0574198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:25.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.001+0000 7f0579f6c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0574100e20 0x7f0574198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:25.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.001+0000 7f057a76d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0574102ef0 0x7f057419cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42522/0 (socket says 172.21.15.120:42522) 2024-09-19T09:27:25.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.001+0000 7f057a76d640 1 -- 172.21.15.120:0/3205040740 learned_addr learned my addr 172.21.15.120:0/3205040740 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:25.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.001+0000 7f057a76d640 1 -- 172.21.15.120:0/3205040740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0574100e20 msgr2=0x7f0574198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:25.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.001+0000 7f057a76d640 1 --2- 172.21.15.120:0/3205040740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0574100e20 0x7f0574198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.001+0000 7f057a76d640 1 -- 172.21.15.120:0/3205040740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0574102570 msgr2=0x7f0574198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:25.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.001+0000 7f057a76d640 1 --2- 172.21.15.120:0/3205040740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0574102570 0x7f0574198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.001+0000 7f057a76d640 1 -- 172.21.15.120:0/3205040740 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f057419d4d0 con 0x7f0574102ef0 2024-09-19T09:27:25.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.001+0000 7f057976b640 1 --2- 172.21.15.120:0/3205040740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0574102570 0x7f0574198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:27:25.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.001+0000 7f057a76d640 1 --2- 172.21.15.120:0/3205040740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0574102ef0 0x7f057419cdf0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f056c00eb10 tx=0x7f056c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:25.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.001+0000 7f056affd640 1 -- 172.21.15.120:0/3205040740 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c004b20 con 0x7f0574102ef0 2024-09-19T09:27:25.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.001+0000 7f056affd640 1 -- 172.21.15.120:0/3205040740 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f056c004cc0 con 0x7f0574102ef0 2024-09-19T09:27:25.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.001+0000 7f057c1d0640 1 -- 172.21.15.120:0/3205040740 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f057419d7c0 con 0x7f0574102ef0 2024-09-19T09:27:25.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.001+0000 7f056affd640 1 -- 172.21.15.120:0/3205040740 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c010760 con 0x7f0574102ef0 2024-09-19T09:27:25.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.001+0000 7f057c1d0640 1 -- 172.21.15.120:0/3205040740 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05741a50a0 con 0x7f0574102ef0 2024-09-19T09:27:25.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.004+0000 7f056affd640 1 -- 172.21.15.120:0/3205040740 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f056c005000 con 0x7f0574102ef0 2024-09-19T09:27:25.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.004+0000 7f057e5a9740 1 Processor -- start 2024-09-19T09:27:25.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.005+0000 7f057e5a9740 1 -- start start 2024-09-19T09:27:25.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.005+0000 7f057e5a9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560746dd5b60 0x560746dd5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:25.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.005+0000 7f057e5a9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560746dd6480 0x560746de0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:25.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.005+0000 7f057e5a9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560746de0d40 0x560746de3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:25.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.005+0000 7f057e5a9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560746d3b280 con 0x560746dd6480 2024-09-19T09:27:25.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.005+0000 7f057e5a9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560746ce2b10 con 0x560746dd5b60 2024-09-19T09:27:25.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.005+0000 7f057e5a9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560746d37240 con 0x560746de0d40 2024-09-19T09:27:25.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.005+0000 7f0579f6c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560746dd6480 0x560746de0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:25.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.005+0000 7f057976b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560746dd5b60 0x560746dd5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:25.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.005+0000 7f0579f6c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560746dd6480 0x560746de0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42538/0 (socket says 172.21.15.120:42538) 2024-09-19T09:27:25.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.005+0000 7f0579f6c640 1 -- 172.21.15.120:0/322485687 learned_addr learned my addr 172.21.15.120:0/322485687 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:25.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.005+0000 7f057976b640 1 --2- 172.21.15.120:0/322485687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560746de0d40 0x560746de3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:25.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.005+0000 7f057976b640 1 -- 172.21.15.120:0/322485687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560746de0d40 msgr2=0x560746de3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:25.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.005+0000 7f057976b640 1 --2- 172.21.15.120:0/322485687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560746de0d40 0x560746de3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.005+0000 7f057976b640 1 -- 172.21.15.120:0/322485687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560746dd6480 msgr2=0x560746de0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:25.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.005+0000 7f057976b640 1 --2- 172.21.15.120:0/322485687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560746dd6480 0x560746de0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.005+0000 7f057976b640 1 -- 172.21.15.120:0/322485687 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560746cef1f0 con 0x560746dd5b60 2024-09-19T09:27:25.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.005+0000 7f056affd640 1 --2- 172.21.15.120:0/3205040740 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f05600a32c0 0x7f05600a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:25.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.006+0000 7f0579f6c640 1 --2- 172.21.15.120:0/3205040740 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f05600a32c0 0x7f05600a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:25.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.006+0000 7f057976b640 1 --2- 172.21.15.120:0/322485687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560746dd5b60 0x560746dd5f40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f05640058c0 tx=0x7f0564005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87d456d8ebd6483 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:25.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.006+0000 7f056affd640 1 -- 172.21.15.120:0/3205040740 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f056c0cfe90 con 0x7f0574102ef0 2024-09-19T09:27:25.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.006+0000 7f055ffff640 1 -- 172.21.15.120:0/322485687 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0564004050 con 0x560746dd5b60 2024-09-19T09:27:25.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.006+0000 7f055ffff640 1 -- 172.21.15.120:0/322485687 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f056400fdf0 con 0x560746dd5b60 2024-09-19T09:27:25.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.006+0000 7f055ffff640 1 -- 172.21.15.120:0/322485687 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0564003be0 con 0x560746dd5b60 2024-09-19T09:27:25.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.007+0000 7f057e5a9740 1 -- 172.21.15.120:0/322485687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560746dd5b60 msgr2=0x560746dd5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:25.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.007+0000 7f0579f6c640 1 --2- 172.21.15.120:0/3205040740 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f05600a32c0 0x7f05600a5780 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f05700314e0 tx=0x7f05700058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:25.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.007+0000 7f057e5a9740 1 --2- 172.21.15.120:0/322485687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560746dd5b60 0x560746dd5f40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f05640058c0 tx=0x7f0564005850 comp rx=0 tx=0).stop 2024-09-19T09:27:25.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.007+0000 7f057e5a9740 1 -- 172.21.15.120:0/322485687 shutdown_connections 2024-09-19T09:27:25.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.007+0000 7f057e5a9740 1 --2- 172.21.15.120:0/322485687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560746de0d40 0x560746de3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.007+0000 7f057e5a9740 1 --2- 172.21.15.120:0/322485687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560746dd6480 0x560746de0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.007+0000 7f057e5a9740 1 --2- 172.21.15.120:0/322485687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560746dd5b60 0x560746dd5f40 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.007+0000 7f057e5a9740 1 -- 172.21.15.120:0/322485687 >> 172.21.15.120:0/322485687 conn(0x560746dd20f0 msgr2=0x560746dd24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:25.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.007+0000 7f057e5a9740 1 -- 172.21.15.120:0/322485687 shutdown_connections 2024-09-19T09:27:25.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.007+0000 7f057e5a9740 1 -- 172.21.15.120:0/322485687 wait complete. 2024-09-19T09:27:25.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.007+0000 7f057e5a9740 1 -- start start 2024-09-19T09:27:25.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.007+0000 7f057e5a9740 1 Processor -- start 2024-09-19T09:27:25.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.008+0000 7f057e5a9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560746dd5b60 0x560746dddc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:25.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.008+0000 7f057e5a9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560746dd6480 0x560746dde1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:25.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.008+0000 7f057e5a9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560746de0d40 0x560746dde710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:25.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.008+0000 7f057e5a9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560746de3910 con 0x560746de0d40 2024-09-19T09:27:25.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.008+0000 7f057e5a9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560746cf5f10 con 0x560746dd6480 2024-09-19T09:27:25.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.008+0000 7f057e5a9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560746de3a90 con 0x560746dd5b60 2024-09-19T09:27:25.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.008+0000 7f057e5a9740 10 client.0 _ll_register_callbacks cb 0x560746cbea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:25.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.008+0000 7f0579f6c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560746dd5b60 0x560746dddc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:25.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.008+0000 7f057a76d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560746de0d40 0x560746dde710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:25.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.008+0000 7f0579f6c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560746dd5b60 0x560746dddc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:58838/0 (socket says 172.21.15.120:58838) 2024-09-19T09:27:25.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.008+0000 7f0579f6c640 1 -- 172.21.15.120:0/1301487727 learned_addr learned my addr 172.21.15.120:0/1301487727 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:25.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.008+0000 7f057976b640 1 --2- 172.21.15.120:0/1301487727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560746dd6480 0x560746dde1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:25.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.008+0000 7f0579f6c640 1 -- 172.21.15.120:0/1301487727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560746dd6480 msgr2=0x560746dde1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:25.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.008+0000 7f0579f6c640 1 --2- 172.21.15.120:0/1301487727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560746dd6480 0x560746dde1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.008+0000 7f0579f6c640 1 -- 172.21.15.120:0/1301487727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560746de0d40 msgr2=0x560746dde710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:25.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.008+0000 7f0579f6c640 1 --2- 172.21.15.120:0/1301487727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560746de0d40 0x560746dde710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.008+0000 7f0579f6c640 1 -- 172.21.15.120:0/1301487727 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560746aac6a0 con 0x560746dd5b60 2024-09-19T09:27:25.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.008+0000 7f057a76d640 1 --2- 172.21.15.120:0/1301487727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560746de0d40 0x560746dde710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:27:25.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.008+0000 7f057976b640 1 --2- 172.21.15.120:0/1301487727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560746dd6480 0x560746dde1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:25.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.008+0000 7f0579f6c640 1 --2- 172.21.15.120:0/1301487727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560746dd5b60 0x560746dddc90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0570033000 tx=0x7f0570009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b90316f08fa47dfc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:25.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.008+0000 7f055d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:27:25.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.009+0000 7f055d7fa640 1 -- 172.21.15.120:0/1301487727 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570035040 con 0x560746dd5b60 2024-09-19T09:27:25.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.009+0000 7f057e5a9740 1 -- 172.21.15.120:0/1301487727 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f056400fdf0 con 0x560746dd5b60 2024-09-19T09:27:25.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.009+0000 7f055d7fa640 1 -- 172.21.15.120:0/1301487727 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f0570031540 con 0x560746dd5b60 2024-09-19T09:27:25.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.009+0000 7f055d7fa640 1 -- 172.21.15.120:0/1301487727 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0570031740 con 0x560746dd5b60 2024-09-19T09:27:25.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.009+0000 7f057e5a9740 10 client.31876 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:25.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.009+0000 7f057e5a9740 1 -- 172.21.15.120:0/1301487727 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560746def5e0 con 0x560746dd5b60 2024-09-19T09:27:25.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.009+0000 7f055d7fa640 1 -- 172.21.15.120:0/1301487727 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f056400fdf0 con 0x560746dd5b60 2024-09-19T09:27:25.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.009+0000 7f055d7fa640 1 -- 172.21.15.120:0/1301487727 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f0570044070 con 0x560746dd5b60 2024-09-19T09:27:25.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.009+0000 7f057e5a9740 10 client.31876 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:25.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.009+0000 7f057e5a9740 10 client.31876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:27:25.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.009+0000 7f057e5a9740 20 client.31876 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:25.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.009+0000 7f057e5a9740 1 --2- 172.21.15.120:0/1301487727 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560746df0a40 0x560746df2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:25.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.009+0000 7f057e5a9740 4 client.31876 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:25.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.009+0000 7f057e5a9740 1 -- 172.21.15.120:0/1301487727 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560746d487f0 con 0x560746df0a40 2024-09-19T09:27:25.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.010+0000 7f057976b640 1 --2- 172.21.15.120:0/1301487727 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560746df0a40 0x560746df2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:25.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.010+0000 7f057976b640 1 --2- 172.21.15.120:0/1301487727 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560746df0a40 0x560746df2e80 crc :-1 s=READY pgs=599 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e1e4e0016b0f69 server_cookie=b057a77ad40d1ca1 in_seq=0 out_seq=0 2024-09-19T09:27:25.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.010+0000 7f055d7fa640 10 client.31876 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:25.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.011+0000 7f055d7fa640 1 -- 172.21.15.120:0/1301487727 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560746d487f0 con 0x560746df0a40 2024-09-19T09:27:25.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.011+0000 7f055d7fa640 10 client.31876 handle_command_reply: tid=0 2024-09-19T09:27:25.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.011+0000 7f057e5a9740 1 client.31876 shutdown 2024-09-19T09:27:25.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.011+0000 7f057e5a9740 1 -- 172.21.15.120:0/1301487727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560746dd5b60 msgr2=0x560746dddc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:25.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.011+0000 7f057e5a9740 1 --2- 172.21.15.120:0/1301487727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560746dd5b60 0x560746dddc90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0570033000 tx=0x7f0570009260 comp rx=0 tx=0).stop 2024-09-19T09:27:25.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.011+0000 7f057e5a9740 1 -- 172.21.15.120:0/1301487727 shutdown_connections 2024-09-19T09:27:25.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.011+0000 7f057e5a9740 1 --2- 172.21.15.120:0/1301487727 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560746df0a40 0x560746df2e80 crc :-1 s=READY pgs=599 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.011+0000 7f057e5a9740 1 --2- 172.21.15.120:0/1301487727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560746de0d40 0x560746dde710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.011+0000 7f057e5a9740 1 --2- 172.21.15.120:0/1301487727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560746dd6480 0x560746dde1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.011+0000 7f055d7fa640 0 client.31876 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:25.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.011+0000 7f057e5a9740 1 --2- 172.21.15.120:0/1301487727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560746dd5b60 0x560746dddc90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.011+0000 7f057e5a9740 1 -- 172.21.15.120:0/1301487727 >> 172.21.15.120:0/1301487727 conn(0x560746dd20f0 msgr2=0x560746dd7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:25.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.011+0000 7f057e5a9740 1 -- 172.21.15.120:0/1301487727 shutdown_connections 2024-09-19T09:27:25.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.011+0000 7f057e5a9740 1 -- 172.21.15.120:0/1301487727 wait complete. 2024-09-19T09:27:25.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.011+0000 7f057e5a9740 20 client.31876 trim_cache size 0 max 16384 2024-09-19T09:27:25.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.028+0000 7f057e5a9740 1 Processor -- start 2024-09-19T09:27:25.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.028+0000 7f057e5a9740 1 -- start start 2024-09-19T09:27:25.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.028+0000 7f057e5a9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560746dd5b60 0x560746dd8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:25.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.028+0000 7f057e5a9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560746dd6480 0x560746ddfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:25.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.028+0000 7f057e5a9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560746de0d40 0x560746df8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:25.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.028+0000 7f057e5a9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f05741ad620 con 0x560746dd6480 2024-09-19T09:27:25.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.028+0000 7f057e5a9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f05741af160 con 0x560746de0d40 2024-09-19T09:27:25.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.028+0000 7f057e5a9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f05741adcf0 con 0x560746dd5b60 2024-09-19T09:27:25.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.028+0000 7f057976b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560746dd6480 0x560746ddfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:25.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.028+0000 7f057976b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560746dd6480 0x560746ddfaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42552/0 (socket says 172.21.15.120:42552) 2024-09-19T09:27:25.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.028+0000 7f057976b640 1 -- 172.21.15.120:0/3825754591 learned_addr learned my addr 172.21.15.120:0/3825754591 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:25.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.028+0000 7f0579f6c640 1 --2- 172.21.15.120:0/3825754591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560746dd5b60 0x560746dd8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:25.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.028+0000 7f057976b640 1 -- 172.21.15.120:0/3825754591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560746dd5b60 msgr2=0x560746dd8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:25.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.029+0000 7f057976b640 1 --2- 172.21.15.120:0/3825754591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560746dd5b60 0x560746dd8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.029+0000 7f057976b640 1 -- 172.21.15.120:0/3825754591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560746de0d40 msgr2=0x560746df8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:25.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.029+0000 7f057a76d640 1 --2- 172.21.15.120:0/3825754591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560746de0d40 0x560746df8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:25.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.029+0000 7f057976b640 1 --2- 172.21.15.120:0/3825754591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560746de0d40 0x560746df8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.029+0000 7f057976b640 1 -- 172.21.15.120:0/3825754591 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560746de0560 con 0x560746dd6480 2024-09-19T09:27:25.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.029+0000 7f057a76d640 1 --2- 172.21.15.120:0/3825754591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560746de0d40 0x560746df8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:25.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.029+0000 7f057976b640 1 --2- 172.21.15.120:0/3825754591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560746dd6480 0x560746ddfaf0 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f0564003790 tx=0x7f0564012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b6a95d677b3d56 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:25.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.029+0000 7f055f7fe640 1 -- 172.21.15.120:0/3825754591 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0564005250 con 0x560746dd6480 2024-09-19T09:27:25.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.029+0000 7f055f7fe640 1 -- 172.21.15.120:0/3825754591 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f056400fd70 con 0x560746dd6480 2024-09-19T09:27:25.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.029+0000 7f055f7fe640 1 -- 172.21.15.120:0/3825754591 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05640049c0 con 0x560746dd6480 2024-09-19T09:27:25.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.030+0000 7f057e5a9740 1 -- 172.21.15.120:0/3825754591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560746dd6480 msgr2=0x560746ddfaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:25.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.030+0000 7f057e5a9740 1 --2- 172.21.15.120:0/3825754591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560746dd6480 0x560746ddfaf0 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f0564003790 tx=0x7f0564012040 comp rx=0 tx=0).stop 2024-09-19T09:27:25.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.030+0000 7f057e5a9740 1 -- 172.21.15.120:0/3825754591 shutdown_connections 2024-09-19T09:27:25.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.030+0000 7f057e5a9740 1 --2- 172.21.15.120:0/3825754591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560746de0d40 0x560746df8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.030+0000 7f057e5a9740 1 --2- 172.21.15.120:0/3825754591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560746dd6480 0x560746ddfaf0 unknown :-1 s=CLOSED pgs=2482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.030+0000 7f057e5a9740 1 --2- 172.21.15.120:0/3825754591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560746dd5b60 0x560746dd8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.030+0000 7f057e5a9740 1 -- 172.21.15.120:0/3825754591 >> 172.21.15.120:0/3825754591 conn(0x560746dd20f0 msgr2=0x560746ddf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:25.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.030+0000 7f057e5a9740 1 -- 172.21.15.120:0/3825754591 shutdown_connections 2024-09-19T09:27:25.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.030+0000 7f057e5a9740 1 -- 172.21.15.120:0/3825754591 wait complete. 2024-09-19T09:27:25.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.030+0000 7f057e5a9740 1 -- start start 2024-09-19T09:27:25.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.030+0000 7f057e5a9740 1 Processor -- start 2024-09-19T09:27:25.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.031+0000 7f057e5a9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560746dd5b60 0x560746de9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:25.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.031+0000 7f057e5a9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560746dd6480 0x560746de9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:25.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.031+0000 7f057e5a9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560746de0d40 0x560746de43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:25.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.031+0000 7f057e5a9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f05741aea90 con 0x560746dd5b60 2024-09-19T09:27:25.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.031+0000 7f057e5a9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f05741ae3c0 con 0x560746dd6480 2024-09-19T09:27:25.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.031+0000 7f057e5a9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f05741ab5e0 con 0x560746de0d40 2024-09-19T09:27:25.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.031+0000 7f057e5a9740 10 client.0 _ll_register_callbacks cb 0x7f0570009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:25.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.031+0000 7f0579f6c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560746dd5b60 0x560746de9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:25.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.031+0000 7f0579f6c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560746dd5b60 0x560746de9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42554/0 (socket says 172.21.15.120:42554) 2024-09-19T09:27:25.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.031+0000 7f0579f6c640 1 -- 172.21.15.120:0/605584842 learned_addr learned my addr 172.21.15.120:0/605584842 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:25.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.031+0000 7f057a76d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560746de0d40 0x560746de43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:25.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.031+0000 7f057976b640 1 --2- 172.21.15.120:0/605584842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560746dd6480 0x560746de9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:25.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.031+0000 7f057a76d640 1 -- 172.21.15.120:0/605584842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560746dd6480 msgr2=0x560746de9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:25.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.031+0000 7f057a76d640 1 --2- 172.21.15.120:0/605584842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560746dd6480 0x560746de9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.031+0000 7f057a76d640 1 -- 172.21.15.120:0/605584842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560746dd5b60 msgr2=0x560746de9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:25.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.031+0000 7f057a76d640 1 --2- 172.21.15.120:0/605584842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560746dd5b60 0x560746de9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.031+0000 7f057a76d640 1 -- 172.21.15.120:0/605584842 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560746de4c20 con 0x560746de0d40 2024-09-19T09:27:25.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.031+0000 7f057976b640 1 --2- 172.21.15.120:0/605584842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560746dd6480 0x560746de9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:27:25.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.031+0000 7f0579f6c640 1 --2- 172.21.15.120:0/605584842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560746dd5b60 0x560746de9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:27:25.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.031+0000 7f057a76d640 1 --2- 172.21.15.120:0/605584842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560746de0d40 0x560746de43f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f056c098910 tx=0x7f056c004f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef55509dd6edef4e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:25.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.031+0000 7f055d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:27:25.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.031+0000 7f055d7fa640 1 -- 172.21.15.120:0/605584842 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c0b4070 con 0x560746de0d40 2024-09-19T09:27:25.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.031+0000 7f055d7fa640 1 -- 172.21.15.120:0/605584842 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f056c09b900 con 0x560746de0d40 2024-09-19T09:27:25.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.031+0000 7f057e5a9740 1 -- 172.21.15.120:0/605584842 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f056400fd70 con 0x560746de0d40 2024-09-19T09:27:25.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.032+0000 7f055d7fa640 1 -- 172.21.15.120:0/605584842 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f056c09c4b0 con 0x560746de0d40 2024-09-19T09:27:25.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.032+0000 7f057e5a9740 10 client.31882 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:25.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.032+0000 7f057e5a9740 1 -- 172.21.15.120:0/605584842 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560746e5b2b0 con 0x560746de0d40 2024-09-19T09:27:25.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.032+0000 7f055d7fa640 1 -- 172.21.15.120:0/605584842 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f056400fd70 con 0x560746de0d40 2024-09-19T09:27:25.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.032+0000 7f055d7fa640 1 -- 172.21.15.120:0/605584842 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f056c09b370 con 0x560746de0d40 2024-09-19T09:27:25.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.032+0000 7f057e5a9740 10 client.31882 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:25.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.032+0000 7f057e5a9740 10 client.31882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:27:25.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.032+0000 7f057e5a9740 20 client.31882 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:25.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.032+0000 7f057e5a9740 1 --2- 172.21.15.120:0/605584842 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560746df0a40 0x560746e5da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:25.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.032+0000 7f057e5a9740 4 client.31882 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:27:25.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.032+0000 7f057e5a9740 1 -- 172.21.15.120:0/605584842 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560746d42d00 con 0x560746df0a40 2024-09-19T09:27:25.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.032+0000 7f0579f6c640 1 --2- 172.21.15.120:0/605584842 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560746df0a40 0x560746e5da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:25.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.033+0000 7f055d7fa640 10 client.31882 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:25.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.033+0000 7f0579f6c640 1 --2- 172.21.15.120:0/605584842 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560746df0a40 0x560746e5da90 crc :-1 s=READY pgs=600 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c33b9641ed47052 server_cookie=ffde17b19c56c80c in_seq=0 out_seq=0 2024-09-19T09:27:25.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.033+0000 7f055d7fa640 1 -- 172.21.15.120:0/605584842 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560746d42d00 con 0x560746df0a40 2024-09-19T09:27:25.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.033+0000 7f055d7fa640 10 client.31882 handle_command_reply: tid=0 2024-09-19T09:27:25.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.033+0000 7f057e5a9740 1 client.31882 shutdown 2024-09-19T09:27:25.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.034+0000 7f057e5a9740 1 -- 172.21.15.120:0/605584842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560746de0d40 msgr2=0x560746de43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:25.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.034+0000 7f057e5a9740 1 --2- 172.21.15.120:0/605584842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560746de0d40 0x560746de43f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f056c098910 tx=0x7f056c004f20 comp rx=0 tx=0).stop 2024-09-19T09:27:25.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.034+0000 7f057e5a9740 1 -- 172.21.15.120:0/605584842 shutdown_connections 2024-09-19T09:27:25.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.034+0000 7f057e5a9740 1 --2- 172.21.15.120:0/605584842 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x560746df0a40 0x560746e5da90 crc :-1 s=READY pgs=600 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.034+0000 7f057e5a9740 1 --2- 172.21.15.120:0/605584842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560746de0d40 0x560746de43f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.034+0000 7f057e5a9740 1 --2- 172.21.15.120:0/605584842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560746dd6480 0x560746de9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.034+0000 7f055d7fa640 0 client.31882 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:25.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.034+0000 7f057e5a9740 1 --2- 172.21.15.120:0/605584842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560746dd5b60 0x560746de9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.034+0000 7f057e5a9740 1 -- 172.21.15.120:0/605584842 >> 172.21.15.120:0/605584842 conn(0x560746dd20f0 msgr2=0x560746dd1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:25.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.034+0000 7f057e5a9740 1 -- 172.21.15.120:0/605584842 shutdown_connections 2024-09-19T09:27:25.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.034+0000 7f057e5a9740 1 -- 172.21.15.120:0/605584842 wait complete. 2024-09-19T09:27:25.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.034+0000 7f057e5a9740 20 client.31882 trim_cache size 0 max 16384 2024-09-19T09:27:25.040 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:27:25.040 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-19T09:27:25.040 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "4888793d-4af3-4bb3-93b5-28cfa3d26e9c", 2024-09-19T09:27:25.041 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-19T09:27:25.041 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:27:25.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.034+0000 7f057c1d0640 1 -- 172.21.15.120:0/3205040740 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f05600a32c0 msgr2=0x7f05600a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:25.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.034+0000 7f057c1d0640 1 --2- 172.21.15.120:0/3205040740 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f05600a32c0 0x7f05600a5780 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f05700314e0 tx=0x7f05700058d0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.034+0000 7f057c1d0640 1 -- 172.21.15.120:0/3205040740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0574102ef0 msgr2=0x7f057419cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:25.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.034+0000 7f057c1d0640 1 --2- 172.21.15.120:0/3205040740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0574102ef0 0x7f057419cdf0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f056c00eb10 tx=0x7f056c00c2d0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.035+0000 7f057c1d0640 1 -- 172.21.15.120:0/3205040740 shutdown_connections 2024-09-19T09:27:25.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.035+0000 7f057c1d0640 1 --2- 172.21.15.120:0/3205040740 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f05600a32c0 0x7f05600a5780 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.035+0000 7f057c1d0640 1 --2- 172.21.15.120:0/3205040740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0574102ef0 0x7f057419cdf0 unknown :-1 s=CLOSED pgs=2481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.035+0000 7f057c1d0640 1 --2- 172.21.15.120:0/3205040740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0574102570 0x7f0574198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.035+0000 7f057c1d0640 1 --2- 172.21.15.120:0/3205040740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0574100e20 0x7f0574198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.035+0000 7f057c1d0640 1 -- 172.21.15.120:0/3205040740 >> 172.21.15.120:0/3205040740 conn(0x7f05740fc7d0 msgr2=0x7f057406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:25.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.035+0000 7f057c1d0640 1 -- 172.21.15.120:0/3205040740 shutdown_connections 2024-09-19T09:27:25.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:25.035+0000 7f057c1d0640 1 -- 172.21.15.120:0/3205040740 wait complete. 2024-09-19T09:27:25.069 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:27:25.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:25 smithi120 ceph-mon[31150]: scrub queued for path: / 2024-09-19T09:27:25.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:25 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:27:25.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:25 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:27:25.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:25 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:27:25.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:25 smithi120 ceph-mon[31150]: Scrub error on inode 0x1000000b70c (/volumes/qa/sv_1/f2bee15c-e720-4829-ac9c-1a0401c9d37d/client.0/tmp/clients/client6/~dmtmp/ACCESS) see mds.b log and `damage ls` output for details 2024-09-19T09:27:25.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:25 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:27:25.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:25 smithi120 ceph-mon[31150]: Scrub error on inode 0x1000000b60e (/volumes/qa/sv_1/f2bee15c-e720-4829-ac9c-1a0401c9d37d/client.0/tmp/clients/client5/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-09-19T09:27:25.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:25 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:27:25.394 INFO:tasks.workunit.client.0.smithi120.stdout: 10 266983 20.36 MB/sec execute 560 sec latency 14376.694 ms 2024-09-19T09:27:25.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:25 smithi153 ceph-mon[35071]: scrub queued for path: / 2024-09-19T09:27:25.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:25 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:27:25.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:25 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:27:25.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:25 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:27:25.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:25 smithi153 ceph-mon[35071]: Scrub error on inode 0x1000000b70c (/volumes/qa/sv_1/f2bee15c-e720-4829-ac9c-1a0401c9d37d/client.0/tmp/clients/client6/~dmtmp/ACCESS) see mds.b log and `damage ls` output for details 2024-09-19T09:27:25.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:25 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:27:25.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:25 smithi153 ceph-mon[35071]: Scrub error on inode 0x1000000b60e (/volumes/qa/sv_1/f2bee15c-e720-4829-ac9c-1a0401c9d37d/client.0/tmp/clients/client5/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-09-19T09:27:25.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:25 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:27:25.439 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 19 09:27:25 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-b[64524]: 2024-09-19T09:27:25.102+0000 7f1ed1965640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000b70c [...2,head] /volumes/qa/sv_1/f2bee15c-e720-4829-ac9c-1a0401c9d37d/client.0/tmp/clients/client6/~dmtmp/ACCESS/ auth{2=1} v43409 ap=3 RANDEPHEMERALPIN f(v464 m2024-09-19T09:27:09.795537+0000 1=1+0) n(v56 rc2024-09-19T09:27:09.795537+0000 b260096 2=1+1) (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:7921 nref=3)) caps={15204=pAsLsXsFs/pAsLsXsFsx@952} mcw={2=AsLsXsFs} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x560de9dbb180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000b70c:[<0x1000000b4b6/ACCESS v274>,<0x1000000b4b3/~dmtmp v47>,<0x10000000007/client6 v1657>,<0x10000000006/clients v1215>,<0x10000000005/tmp v1179>,<0x100000001fb/client.0 v1176>,<0x100000001fa/f2bee15c-e720-4829-ac9c-1a0401c9d37d v1163>,<0x10000000001/sv_1 v1171>,<0x10000000000/qa v1139>,<0x1/volumes v1126>]//[]","memoryvalue":"(2)0x1000000b70c:[<0x1000000b4b6/ACCESS v43409>,<0x1000000b4b3/~dmtmp v2066>,<0x10000000007/client6 v12094>,<0x10000000006/clients v3014>,<0x10000000005/tmp v2388>,<0x100000001fb/client.0 v2334>,<0x100000001fa/f2bee15c-e720-4829-ac9c-1a0401c9d37d v2307>,<0x10000000001/sv_1 v2288>,<0x10000000000/qa v2254>,<0x1/volumes v2199>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T09:27:06.873606+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-09-19T09:27:09.795537+0000 b260096 2=1+1)","memory_value.dirstat":"f(v464 m2024-09-19T09:27:09.795537+0000 1=1+0)","memory_value.rstat":"n(v56 rc2024-09-19T09:27:09.795537+0000 b260096 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-19T09:27:25.439 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 19 09:27:25 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-b[64524]: 2024-09-19T09:27:25.106+0000 7f1ed1965640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000b60e [...2,head] /volumes/qa/sv_1/f2bee15c-e720-4829-ac9c-1a0401c9d37d/client.0/tmp/clients/client5/~dmtmp/PARADOX/ auth{1=1,2=1,3=1,4=1} v40668 ap=3 RANDEPHEMERALPIN f(v1433 m2024-09-19T09:27:10.981509+0000 22=22+0) n(v55 rc2024-09-19T09:27:10.992509+0000 b5204459 23=22+1) (inest sync->mix g=1,2,3,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:7927 nref=3)) caps={15204=pAsLsXsFs/pAsLsXsFsx@3081} mcw={1=AsLsXsFs} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x560de9a06680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000b60e:[<0x1000000b49b/PARADOX v268>,<0x1000000b49a/~dmtmp v47>,<0x10000000007/client5 v1645>,<0x10000000006/clients v1215>,<0x10000000005/tmp v1179>,<0x100000001fb/client.0 v1176>,<0x100000001fa/f2bee15c-e720-4829-ac9c-1a0401c9d37d v1163>,<0x10000000001/sv_1 v1171>,<0x10000000000/qa v1139>,<0x1/volumes v1126>]//[]","memoryvalue":"(2)0x1000000b60e:[<0x1000000b49b/PARADOX v40668>,<0x1000000b49a/~dmtmp v3919>,<0x10000000007/client5 v12092>,<0x10000000006/clients v3014>,<0x10000000005/tmp v2388>,<0x100000001fb/client.0 v2334>,<0x100000001fa/f2bee15c-e720-4829-ac9c-1a0401c9d37d v2307>,<0x10000000001/sv_1 v2288>,<0x10000000000/qa v2254>,<0x1/volumes v2199>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T09:27:10.937510+0000 21=21+0)","ondisk_value.rstat":"n(v0 rc2024-09-19T09:27:10.992509+0000 b5204459 23=22+1)","memory_value.dirstat":"f(v1433 m2024-09-19T09:27:10.981509+0000 22=22+0)","memory_value.rstat":"n(v55 rc2024-09-19T09:27:10.992509+0000 b5204459 23=22+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-19T09:27:25.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:25 smithi196 ceph-mon[35150]: scrub queued for path: / 2024-09-19T09:27:25.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:25 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:27:25.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:25 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:27:25.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:25 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:27:25.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:25 smithi196 ceph-mon[35150]: Scrub error on inode 0x1000000b70c (/volumes/qa/sv_1/f2bee15c-e720-4829-ac9c-1a0401c9d37d/client.0/tmp/clients/client6/~dmtmp/ACCESS) see mds.b log and `damage ls` output for details 2024-09-19T09:27:25.688 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:25 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:27:25.688 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:25 smithi196 ceph-mon[35150]: Scrub error on inode 0x1000000b60e (/volumes/qa/sv_1/f2bee15c-e720-4829-ac9c-1a0401c9d37d/client.0/tmp/clients/client5/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-09-19T09:27:25.688 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:25 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:27:25.910 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:27:25.957 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:26.393 INFO:tasks.workunit.client.0.smithi120.stdout: 10 266983 20.32 MB/sec execute 561 sec latency 15376.759 ms 2024-09-19T09:27:26.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.441+0000 7f467c061640 1 -- 172.21.15.120:0/1226173936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4674105f40 msgr2=0x7f46741063d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:26.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.441+0000 7f467c061640 1 --2- 172.21.15.120:0/1226173936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4674105f40 0x7f46741063d0 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f466c009ca0 tx=0x7f466c02a940 comp rx=0 tx=0).stop 2024-09-19T09:27:26.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.441+0000 7f467c061640 1 -- 172.21.15.120:0/1226173936 shutdown_connections 2024-09-19T09:27:26.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.441+0000 7f467c061640 1 --2- 172.21.15.120:0/1226173936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4674105f40 0x7f46741063d0 unknown :-1 s=CLOSED pgs=2483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.441+0000 7f467c061640 1 --2- 172.21.15.120:0/1226173936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4674101450 0x7f4674105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.441+0000 7f467c061640 1 --2- 172.21.15.120:0/1226173936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4674100b30 0x7f4674100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.441+0000 7f467c061640 1 -- 172.21.15.120:0/1226173936 >> 172.21.15.120:0/1226173936 conn(0x7f46740fc7a0 msgr2=0x7f46740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:26.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.441+0000 7f467c061640 1 -- 172.21.15.120:0/1226173936 shutdown_connections 2024-09-19T09:27:26.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.441+0000 7f467c061640 1 -- 172.21.15.120:0/1226173936 wait complete. 2024-09-19T09:27:26.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.441+0000 7f467c061640 1 Processor -- start 2024-09-19T09:27:26.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.441+0000 7f467c061640 1 -- start start 2024-09-19T09:27:26.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.441+0000 7f467c061640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4674100b30 0x7f46741a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:26.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.441+0000 7f467c061640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4674101450 0x7f46741a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:26.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.441+0000 7f467c061640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4674105f40 0x7f467419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:26.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.442+0000 7f467c061640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4674118570 con 0x7f4674101450 2024-09-19T09:27:26.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.442+0000 7f467c061640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f46741183f0 con 0x7f4674100b30 2024-09-19T09:27:26.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.442+0000 7f467c061640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f46741186f0 con 0x7f4674105f40 2024-09-19T09:27:26.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.442+0000 7f46795fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4674101450 0x7f46741a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:26.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.442+0000 7f46795fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4674101450 0x7f46741a4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42562/0 (socket says 172.21.15.120:42562) 2024-09-19T09:27:26.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.442+0000 7f46795fc640 1 -- 172.21.15.120:0/1167268829 learned_addr learned my addr 172.21.15.120:0/1167268829 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:26.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.442+0000 7f467a5fe640 1 --2- 172.21.15.120:0/1167268829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4674105f40 0x7f467419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:26.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.442+0000 7f4679dfd640 1 --2- 172.21.15.120:0/1167268829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4674100b30 0x7f46741a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:26.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.442+0000 7f46795fc640 1 -- 172.21.15.120:0/1167268829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4674105f40 msgr2=0x7f467419e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:26.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.442+0000 7f46795fc640 1 --2- 172.21.15.120:0/1167268829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4674105f40 0x7f467419e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.442+0000 7f46795fc640 1 -- 172.21.15.120:0/1167268829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4674100b30 msgr2=0x7f46741a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:26.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.442+0000 7f46795fc640 1 --2- 172.21.15.120:0/1167268829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4674100b30 0x7f46741a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.442+0000 7f46795fc640 1 -- 172.21.15.120:0/1167268829 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f467419ef60 con 0x7f4674101450 2024-09-19T09:27:26.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.442+0000 7f4679dfd640 1 --2- 172.21.15.120:0/1167268829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4674100b30 0x7f46741a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:26.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.442+0000 7f46795fc640 1 --2- 172.21.15.120:0/1167268829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4674101450 0x7f46741a4b80 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f466400c970 tx=0x7f466400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:26.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.442+0000 7f466affd640 1 -- 172.21.15.120:0/1167268829 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4664007bb0 con 0x7f4674101450 2024-09-19T09:27:26.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.442+0000 7f466affd640 1 -- 172.21.15.120:0/1167268829 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4664007d50 con 0x7f4674101450 2024-09-19T09:27:26.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.442+0000 7f467c061640 1 -- 172.21.15.120:0/1167268829 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f467419f250 con 0x7f4674101450 2024-09-19T09:27:26.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.442+0000 7f466affd640 1 -- 172.21.15.120:0/1167268829 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4664018820 con 0x7f4674101450 2024-09-19T09:27:26.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.442+0000 7f467c061640 1 -- 172.21.15.120:0/1167268829 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46741ab4a0 con 0x7f4674101450 2024-09-19T09:27:26.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.443+0000 7f467c061640 1 -- 172.21.15.120:0/1167268829 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f467406b070 con 0x7f4674101450 2024-09-19T09:27:26.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.444+0000 7f466affd640 1 -- 172.21.15.120:0/1167268829 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f46640189c0 con 0x7f4674101450 2024-09-19T09:27:26.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.445+0000 7f466affd640 1 --2- 172.21.15.120:0/1167268829 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f46600a32c0 0x7f46600a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:26.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.445+0000 7f466affd640 1 -- 172.21.15.120:0/1167268829 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f46640cfb60 con 0x7f4674101450 2024-09-19T09:27:26.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.445+0000 7f4679dfd640 1 --2- 172.21.15.120:0/1167268829 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f46600a32c0 0x7f46600a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:26.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.446+0000 7f4679dfd640 1 --2- 172.21.15.120:0/1167268829 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f46600a32c0 0x7f46600a5780 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f4670005fd0 tx=0x7f4670005ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:26.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.448+0000 7f466affd640 1 -- 172.21.15.120:0/1167268829 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4664014030 con 0x7f4674101450 2024-09-19T09:27:26.467 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:26 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:27:26.467 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:26 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:27:26.467 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:26 smithi120 ceph-mon[31150]: scrub summary: active paths [/] 2024-09-19T09:27:26.467 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:26 smithi120 ceph-mon[31150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:27:26.467 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:26 smithi120 ceph-mon[31150]: pgmap v1082: 97 pgs: 97 active+clean; 499 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 3.8 MiB/s wr, 161 op/s 2024-09-19T09:27:26.505 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:26.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.664+0000 7f467c061640 1 -- 172.21.15.120:0/1167268829 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4674105920 con 0x7f4674101450 2024-09-19T09:27:26.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.665+0000 7f466affd640 1 -- 172.21.15.120:0/1167268829 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11988 (secure 0 0 0) 0x7f4674105920 con 0x7f4674101450 2024-09-19T09:27:26.666 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:27:26.666 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":39,"btime":"2024-09-19T09:18:53:120858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:18:53.070639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:27:26.667 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 39 2024-09-19T09:27:26.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.667+0000 7f467c061640 1 -- 172.21.15.120:0/1167268829 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f46600a32c0 msgr2=0x7f46600a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:26.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.667+0000 7f467c061640 1 --2- 172.21.15.120:0/1167268829 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f46600a32c0 0x7f46600a5780 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f4670005fd0 tx=0x7f4670005ea0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.667+0000 7f467c061640 1 -- 172.21.15.120:0/1167268829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4674101450 msgr2=0x7f46741a4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:26.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.667+0000 7f467c061640 1 --2- 172.21.15.120:0/1167268829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4674101450 0x7f46741a4b80 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f466400c970 tx=0x7f466400ce30 comp rx=0 tx=0).stop 2024-09-19T09:27:26.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.667+0000 7f467c061640 1 -- 172.21.15.120:0/1167268829 shutdown_connections 2024-09-19T09:27:26.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.667+0000 7f467c061640 1 --2- 172.21.15.120:0/1167268829 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f46600a32c0 0x7f46600a5780 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.667+0000 7f467c061640 1 --2- 172.21.15.120:0/1167268829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4674105f40 0x7f467419e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.667+0000 7f467c061640 1 --2- 172.21.15.120:0/1167268829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4674101450 0x7f46741a4b80 unknown :-1 s=CLOSED pgs=2484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.667+0000 7f467c061640 1 --2- 172.21.15.120:0/1167268829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4674100b30 0x7f46741a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.667+0000 7f467c061640 1 -- 172.21.15.120:0/1167268829 >> 172.21.15.120:0/1167268829 conn(0x7f46740fc7a0 msgr2=0x7f46741038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:26.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.667+0000 7f467c061640 1 -- 172.21.15.120:0/1167268829 shutdown_connections 2024-09-19T09:27:26.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:26.667+0000 7f467c061640 1 -- 172.21.15.120:0/1167268829 wait complete. 2024-09-19T09:27:26.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:26 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:27:26.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:26 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:27:26.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:26 smithi196 ceph-mon[35150]: scrub summary: active paths [/] 2024-09-19T09:27:26.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:26 smithi196 ceph-mon[35150]: scrub summary: idle+waiting paths [/] 2024-09-19T09:27:26.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:26 smithi196 ceph-mon[35150]: pgmap v1082: 97 pgs: 97 active+clean; 499 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 3.8 MiB/s wr, 161 op/s 2024-09-19T09:27:26.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:26 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:27:26.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:26 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:27:26.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:26 smithi153 ceph-mon[35071]: scrub summary: active paths [/] 2024-09-19T09:27:26.689 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:26 smithi153 ceph-mon[35071]: scrub summary: idle+waiting paths [/] 2024-09-19T09:27:26.689 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:26 smithi153 ceph-mon[35071]: pgmap v1082: 97 pgs: 97 active+clean; 499 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 3.8 MiB/s wr, 161 op/s 2024-09-19T09:27:27.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.120+0000 7f92891ba640 1 -- 172.21.15.120:0/1717964715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9284101240 msgr2=0x7f9284101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:27.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.120+0000 7f92891ba640 1 --2- 172.21.15.120:0/1717964715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9284101240 0x7f9284101680 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f926c009900 tx=0x7f926c02a410 comp rx=0 tx=0).stop 2024-09-19T09:27:27.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.120+0000 7f92891ba640 1 -- 172.21.15.120:0/1717964715 shutdown_connections 2024-09-19T09:27:27.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.120+0000 7f92891ba640 1 --2- 172.21.15.120:0/1717964715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9284101bc0 0x7f928410d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.120+0000 7f92891ba640 1 --2- 172.21.15.120:0/1717964715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9284101240 0x7f9284101680 unknown :-1 s=CLOSED pgs=2485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.120+0000 7f92891ba640 1 --2- 172.21.15.120:0/1717964715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9284105a90 0x7f9284105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.120+0000 7f92891ba640 1 -- 172.21.15.120:0/1717964715 >> 172.21.15.120:0/1717964715 conn(0x7f9284077210 msgr2=0x7f92840fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:27.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.120+0000 7f92891ba640 1 -- 172.21.15.120:0/1717964715 shutdown_connections 2024-09-19T09:27:27.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.120+0000 7f92891ba640 1 -- 172.21.15.120:0/1717964715 wait complete. 2024-09-19T09:27:27.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.121+0000 7f92891ba640 1 Processor -- start 2024-09-19T09:27:27.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.121+0000 7f92891ba640 1 -- start start 2024-09-19T09:27:27.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.121+0000 7f92891ba640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9284101240 0x7f92840fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:27.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.121+0000 7f92891ba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9284101bc0 0x7f92840fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:27.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.121+0000 7f92891ba640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9284105a90 0x7f92840fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:27.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.121+0000 7f92891ba640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9284110f70 con 0x7f9284101240 2024-09-19T09:27:27.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.121+0000 7f92891ba640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9284110df0 con 0x7f9284105a90 2024-09-19T09:27:27.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.121+0000 7f92891ba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f92841110f0 con 0x7f9284101bc0 2024-09-19T09:27:27.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.121+0000 7f9282d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9284101240 0x7f92840fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:27.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.121+0000 7f9282d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9284101240 0x7f92840fc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42582/0 (socket says 172.21.15.120:42582) 2024-09-19T09:27:27.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.121+0000 7f9282d9d640 1 -- 172.21.15.120:0/1424197424 learned_addr learned my addr 172.21.15.120:0/1424197424 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:27.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.121+0000 7f928359e640 1 --2- 172.21.15.120:0/1424197424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9284105a90 0x7f92840fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:27.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.121+0000 7f927bfff640 1 --2- 172.21.15.120:0/1424197424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9284101bc0 0x7f92840fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:27.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.122+0000 7f9282d9d640 1 -- 172.21.15.120:0/1424197424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9284101bc0 msgr2=0x7f92840fcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:27.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.122+0000 7f9282d9d640 1 --2- 172.21.15.120:0/1424197424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9284101bc0 0x7f92840fcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.122+0000 7f9282d9d640 1 -- 172.21.15.120:0/1424197424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9284105a90 msgr2=0x7f92840fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:27.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.122+0000 7f9282d9d640 1 --2- 172.21.15.120:0/1424197424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9284105a90 0x7f92840fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.122+0000 7f9282d9d640 1 -- 172.21.15.120:0/1424197424 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92841a6960 con 0x7f9284101240 2024-09-19T09:27:27.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.122+0000 7f928359e640 1 --2- 172.21.15.120:0/1424197424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9284105a90 0x7f92840fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:27.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.122+0000 7f927bfff640 1 --2- 172.21.15.120:0/1424197424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9284101bc0 0x7f92840fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:27.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.122+0000 7f9282d9d640 1 --2- 172.21.15.120:0/1424197424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9284101240 0x7f92840fc690 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f927400d950 tx=0x7f927400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:27.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.122+0000 7f9280d99640 1 -- 172.21.15.120:0/1424197424 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274017070 con 0x7f9284101240 2024-09-19T09:27:27.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.122+0000 7f9280d99640 1 -- 172.21.15.120:0/1424197424 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9274009c30 con 0x7f9284101240 2024-09-19T09:27:27.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.122+0000 7f92891ba640 1 -- 172.21.15.120:0/1424197424 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92841a6c50 con 0x7f9284101240 2024-09-19T09:27:27.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.122+0000 7f9280d99640 1 -- 172.21.15.120:0/1424197424 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927400f6b0 con 0x7f9284101240 2024-09-19T09:27:27.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.122+0000 7f92891ba640 1 -- 172.21.15.120:0/1424197424 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92841a7190 con 0x7f9284101240 2024-09-19T09:27:27.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.124+0000 7f9280d99640 1 -- 172.21.15.120:0/1424197424 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f92740109d0 con 0x7f9284101240 2024-09-19T09:27:27.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.124+0000 7f928b593740 1 Processor -- start 2024-09-19T09:27:27.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.125+0000 7f928b593740 1 -- start start 2024-09-19T09:27:27.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.125+0000 7f928b593740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56338c5d34a0 0x56338c5d3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:27.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.125+0000 7f928b593740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56338c5d3dc0 0x56338c5de140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:27.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.125+0000 7f928b593740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56338c5de680 0x56338c5e0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:27.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.125+0000 7f928b593740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56338c5392e0 con 0x56338c5d34a0 2024-09-19T09:27:27.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.125+0000 7f928b593740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56338c4e0ce0 con 0x56338c5d3dc0 2024-09-19T09:27:27.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.125+0000 7f928b593740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56338c5352a0 con 0x56338c5de680 2024-09-19T09:27:27.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.125+0000 7f928359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56338c5d34a0 0x56338c5d3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:27.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.125+0000 7f928359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56338c5d34a0 0x56338c5d3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42598/0 (socket says 172.21.15.120:42598) 2024-09-19T09:27:27.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.125+0000 7f928359e640 1 -- 172.21.15.120:0/1777669812 learned_addr learned my addr 172.21.15.120:0/1777669812 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:27.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.125+0000 7f927bfff640 1 --2- 172.21.15.120:0/1777669812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56338c5de680 0x56338c5e0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:27.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.125+0000 7f9280d99640 1 --2- 172.21.15.120:0/1424197424 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92700a3240 0x7f92700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:27.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.125+0000 7f9282d9d640 1 --2- 172.21.15.120:0/1777669812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56338c5d3dc0 0x56338c5de140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:27.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.125+0000 7f928359e640 1 -- 172.21.15.120:0/1777669812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56338c5de680 msgr2=0x56338c5e0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:27.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.125+0000 7f928359e640 1 --2- 172.21.15.120:0/1777669812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56338c5de680 0x56338c5e0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.125+0000 7f928359e640 1 -- 172.21.15.120:0/1777669812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56338c5d3dc0 msgr2=0x56338c5de140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:27.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.125+0000 7f928359e640 1 --2- 172.21.15.120:0/1777669812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56338c5d3dc0 0x56338c5de140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.125+0000 7f928359e640 1 -- 172.21.15.120:0/1777669812 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56338c4c5440 con 0x56338c5d34a0 2024-09-19T09:27:27.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.125+0000 7f9282d9d640 1 --2- 172.21.15.120:0/1777669812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56338c5d3dc0 0x56338c5de140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:27.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.125+0000 7f9280d99640 1 -- 172.21.15.120:0/1424197424 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f92740cf560 con 0x7f9284101240 2024-09-19T09:27:27.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.126+0000 7f928359e640 1 --2- 172.21.15.120:0/1424197424 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92700a3240 0x7f92700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:27.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.126+0000 7f928359e640 1 --2- 172.21.15.120:0/1777669812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56338c5d34a0 0x56338c5d3880 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7f927c010540 tx=0x7f927c010a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85647c857ac6a376 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:27.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.126+0000 7f92797fa640 1 -- 172.21.15.120:0/1777669812 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927c022070 con 0x56338c5d34a0 2024-09-19T09:27:27.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.126+0000 7f92797fa640 1 -- 172.21.15.120:0/1777669812 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f927c01dde0 con 0x56338c5d34a0 2024-09-19T09:27:27.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.126+0000 7f92797fa640 1 -- 172.21.15.120:0/1777669812 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927c00ec90 con 0x56338c5d34a0 2024-09-19T09:27:27.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.127+0000 7f928b593740 1 -- 172.21.15.120:0/1777669812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56338c5d34a0 msgr2=0x56338c5d3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:27.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.127+0000 7f928b593740 1 --2- 172.21.15.120:0/1777669812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56338c5d34a0 0x56338c5d3880 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7f927c010540 tx=0x7f927c010a00 comp rx=0 tx=0).stop 2024-09-19T09:27:27.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.127+0000 7f928359e640 1 --2- 172.21.15.120:0/1424197424 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92700a3240 0x7f92700a5700 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f927c01f000 tx=0x7f927c01d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:27.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.127+0000 7f928b593740 1 -- 172.21.15.120:0/1777669812 shutdown_connections 2024-09-19T09:27:27.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.127+0000 7f928b593740 1 --2- 172.21.15.120:0/1777669812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56338c5de680 0x56338c5e0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.127+0000 7f928b593740 1 --2- 172.21.15.120:0/1777669812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56338c5d3dc0 0x56338c5de140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.127+0000 7f928b593740 1 --2- 172.21.15.120:0/1777669812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56338c5d34a0 0x56338c5d3880 unknown :-1 s=CLOSED pgs=2487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.127+0000 7f928b593740 1 -- 172.21.15.120:0/1777669812 >> 172.21.15.120:0/1777669812 conn(0x56338c5cfa30 msgr2=0x56338c5cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:27.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.127+0000 7f928b593740 1 -- 172.21.15.120:0/1777669812 shutdown_connections 2024-09-19T09:27:27.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.127+0000 7f928b593740 1 -- 172.21.15.120:0/1777669812 wait complete. 2024-09-19T09:27:27.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.127+0000 7f928b593740 1 -- start start 2024-09-19T09:27:27.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.127+0000 7f928b593740 1 Processor -- start 2024-09-19T09:27:27.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.127+0000 7f928b593740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56338c5d34a0 0x56338c5ddf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:27.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.127+0000 7f928b593740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56338c5d3dc0 0x56338c5db200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:27.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.127+0000 7f928b593740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56338c5de680 0x56338c5db740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:27.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.127+0000 7f928b593740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56338c5e1250 con 0x56338c5d3dc0 2024-09-19T09:27:27.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.127+0000 7f928b593740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56338c4f4050 con 0x56338c5d34a0 2024-09-19T09:27:27.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.127+0000 7f928b593740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56338c5e13d0 con 0x56338c5de680 2024-09-19T09:27:27.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.127+0000 7f928b593740 10 client.0 _ll_register_callbacks cb 0x56338c534160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:27.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.127+0000 7f927bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56338c5d3dc0 0x56338c5db200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:27.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.127+0000 7f927bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56338c5d3dc0 0x56338c5db200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42606/0 (socket says 172.21.15.120:42606) 2024-09-19T09:27:27.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.128+0000 7f927bfff640 1 -- 172.21.15.120:0/3165755184 learned_addr learned my addr 172.21.15.120:0/3165755184 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:27.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.128+0000 7f9282d9d640 1 --2- 172.21.15.120:0/3165755184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56338c5d34a0 0x56338c5ddf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:27.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.128+0000 7f928359e640 1 --2- 172.21.15.120:0/3165755184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56338c5de680 0x56338c5db740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:27.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.128+0000 7f927bfff640 1 -- 172.21.15.120:0/3165755184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56338c5de680 msgr2=0x56338c5db740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:27.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.128+0000 7f927bfff640 1 --2- 172.21.15.120:0/3165755184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56338c5de680 0x56338c5db740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.128+0000 7f927bfff640 1 -- 172.21.15.120:0/3165755184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56338c5d34a0 msgr2=0x56338c5ddf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:27.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.128+0000 7f927bfff640 1 --2- 172.21.15.120:0/3165755184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56338c5d34a0 0x56338c5ddf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.128+0000 7f927bfff640 1 -- 172.21.15.120:0/3165755184 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56338c2aa3c0 con 0x56338c5d3dc0 2024-09-19T09:27:27.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.128+0000 7f927bfff640 1 --2- 172.21.15.120:0/3165755184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56338c5d3dc0 0x56338c5db200 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7f926c0314e0 tx=0x7f926c005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bbbcf7b7faf67e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:27.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.128+0000 7f926a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:27:27.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.128+0000 7f926a7fc640 1 -- 172.21.15.120:0/3165755184 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c035030 con 0x56338c5d3dc0 2024-09-19T09:27:27.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.128+0000 7f926a7fc640 1 -- 172.21.15.120:0/3165755184 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f926c0048c0 con 0x56338c5d3dc0 2024-09-19T09:27:27.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.128+0000 7f928b593740 1 -- 172.21.15.120:0/3165755184 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f927c01dde0 con 0x56338c5d3dc0 2024-09-19T09:27:27.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.128+0000 7f926a7fc640 1 -- 172.21.15.120:0/3165755184 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f926c004ac0 con 0x56338c5d3dc0 2024-09-19T09:27:27.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.128+0000 7f928b593740 10 client.23421 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:27.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.128+0000 7f928b593740 1 -- 172.21.15.120:0/3165755184 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56338c5ecbe0 con 0x56338c5d3dc0 2024-09-19T09:27:27.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.128+0000 7f926a7fc640 1 -- 172.21.15.120:0/3165755184 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f927c01dde0 con 0x56338c5d3dc0 2024-09-19T09:27:27.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.129+0000 7f926a7fc640 1 -- 172.21.15.120:0/3165755184 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f926c031540 con 0x56338c5d3dc0 2024-09-19T09:27:27.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.129+0000 7f928b593740 10 client.23421 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:27.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.129+0000 7f928b593740 10 client.23421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:27:27.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.129+0000 7f928b593740 20 client.23421 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:27.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.129+0000 7f928b593740 1 --2- 172.21.15.120:0/3165755184 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56338c5ee0f0 0x56338c5f0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:27.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.129+0000 7f928b593740 4 client.23421 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:27.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.129+0000 7f928b593740 1 -- 172.21.15.120:0/3165755184 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56338c546850 con 0x56338c5ee0f0 2024-09-19T09:27:27.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.129+0000 7f9282d9d640 1 --2- 172.21.15.120:0/3165755184 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56338c5ee0f0 0x56338c5f0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:27.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.130+0000 7f9282d9d640 1 --2- 172.21.15.120:0/3165755184 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56338c5ee0f0 0x56338c5f0590 crc :-1 s=READY pgs=601 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b9336416d52ea06 server_cookie=a0b39f5bde7a94fa in_seq=0 out_seq=0 2024-09-19T09:27:27.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.130+0000 7f926a7fc640 10 client.23421 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:27.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.130+0000 7f926a7fc640 1 -- 172.21.15.120:0/3165755184 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56338c546850 con 0x56338c5ee0f0 2024-09-19T09:27:27.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.130+0000 7f926a7fc640 10 client.23421 handle_command_reply: tid=0 2024-09-19T09:27:27.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.130+0000 7f928b593740 1 client.23421 shutdown 2024-09-19T09:27:27.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.130+0000 7f928b593740 1 -- 172.21.15.120:0/3165755184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56338c5d3dc0 msgr2=0x56338c5db200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:27.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.130+0000 7f928b593740 1 --2- 172.21.15.120:0/3165755184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56338c5d3dc0 0x56338c5db200 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7f926c0314e0 tx=0x7f926c005270 comp rx=0 tx=0).stop 2024-09-19T09:27:27.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.130+0000 7f928b593740 1 -- 172.21.15.120:0/3165755184 shutdown_connections 2024-09-19T09:27:27.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.130+0000 7f928b593740 1 --2- 172.21.15.120:0/3165755184 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56338c5ee0f0 0x56338c5f0590 crc :-1 s=READY pgs=601 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.130+0000 7f928b593740 1 --2- 172.21.15.120:0/3165755184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56338c5de680 0x56338c5db740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.130+0000 7f928b593740 1 --2- 172.21.15.120:0/3165755184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56338c5d3dc0 0x56338c5db200 unknown :-1 s=CLOSED pgs=2488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.130+0000 7f926a7fc640 0 client.23421 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:27.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.130+0000 7f928b593740 1 --2- 172.21.15.120:0/3165755184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56338c5d34a0 0x56338c5ddf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.130+0000 7f928b593740 1 -- 172.21.15.120:0/3165755184 >> 172.21.15.120:0/3165755184 conn(0x56338c5cfa30 msgr2=0x56338c5ceb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:27.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.131+0000 7f928b593740 1 -- 172.21.15.120:0/3165755184 shutdown_connections 2024-09-19T09:27:27.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.131+0000 7f928b593740 1 -- 172.21.15.120:0/3165755184 wait complete. 2024-09-19T09:27:27.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.131+0000 7f928b593740 20 client.23421 trim_cache size 0 max 16384 2024-09-19T09:27:27.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.147+0000 7f928b593740 1 Processor -- start 2024-09-19T09:27:27.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.147+0000 7f928b593740 1 -- start start 2024-09-19T09:27:27.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.148+0000 7f928b593740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56338c5d34a0 0x56338c5ea530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:27.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.148+0000 7f928b593740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56338c5d3dc0 0x56338c5d6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:27.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.148+0000 7f928b593740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56338c5de680 0x56338c5dd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:27.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.148+0000 7f928b593740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f92841af6f0 con 0x56338c5d34a0 2024-09-19T09:27:27.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.148+0000 7f928b593740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f92841b1230 con 0x56338c5d3dc0 2024-09-19T09:27:27.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.148+0000 7f928b593740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f92841afdc0 con 0x56338c5de680 2024-09-19T09:27:27.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.148+0000 7f9282d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56338c5d34a0 0x56338c5ea530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:27.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.148+0000 7f9282d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56338c5d34a0 0x56338c5ea530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42620/0 (socket says 172.21.15.120:42620) 2024-09-19T09:27:27.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.148+0000 7f9282d9d640 1 -- 172.21.15.120:0/1138382666 learned_addr learned my addr 172.21.15.120:0/1138382666 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:27.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.148+0000 7f927bfff640 1 --2- 172.21.15.120:0/1138382666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56338c5d3dc0 0x56338c5d6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:27.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.148+0000 7f928359e640 1 --2- 172.21.15.120:0/1138382666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56338c5de680 0x56338c5dd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:27.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.148+0000 7f9282d9d640 1 -- 172.21.15.120:0/1138382666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56338c5de680 msgr2=0x56338c5dd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:27.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.148+0000 7f9282d9d640 1 --2- 172.21.15.120:0/1138382666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56338c5de680 0x56338c5dd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.148+0000 7f9282d9d640 1 -- 172.21.15.120:0/1138382666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56338c5d3dc0 msgr2=0x56338c5d6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:27.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.148+0000 7f9282d9d640 1 --2- 172.21.15.120:0/1138382666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56338c5d3dc0 0x56338c5d6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.148+0000 7f9282d9d640 1 -- 172.21.15.120:0/1138382666 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56338c5d3bf0 con 0x56338c5d34a0 2024-09-19T09:27:27.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.148+0000 7f9282d9d640 1 --2- 172.21.15.120:0/1138382666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56338c5d34a0 0x56338c5ea530 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7f9274010fd0 tx=0x7f9274086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345e43c2dfed1862 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:27.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.148+0000 7f9278ff9640 1 -- 172.21.15.120:0/1138382666 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274020040 con 0x56338c5d34a0 2024-09-19T09:27:27.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.148+0000 7f9278ff9640 1 -- 172.21.15.120:0/1138382666 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f927409bb80 con 0x56338c5d34a0 2024-09-19T09:27:27.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.148+0000 7f9278ff9640 1 -- 172.21.15.120:0/1138382666 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92740a4360 con 0x56338c5d34a0 2024-09-19T09:27:27.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.149+0000 7f928b593740 1 -- 172.21.15.120:0/1138382666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56338c5d34a0 msgr2=0x56338c5ea530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:27.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.149+0000 7f928b593740 1 --2- 172.21.15.120:0/1138382666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56338c5d34a0 0x56338c5ea530 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7f9274010fd0 tx=0x7f9274086040 comp rx=0 tx=0).stop 2024-09-19T09:27:27.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.149+0000 7f928b593740 1 -- 172.21.15.120:0/1138382666 shutdown_connections 2024-09-19T09:27:27.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.149+0000 7f928b593740 1 --2- 172.21.15.120:0/1138382666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56338c5de680 0x56338c5dd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.149+0000 7f928b593740 1 --2- 172.21.15.120:0/1138382666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56338c5d3dc0 0x56338c5d6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.149+0000 7f928b593740 1 --2- 172.21.15.120:0/1138382666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56338c5d34a0 0x56338c5ea530 unknown :-1 s=CLOSED pgs=2489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.149+0000 7f928b593740 1 -- 172.21.15.120:0/1138382666 >> 172.21.15.120:0/1138382666 conn(0x56338c5cfa30 msgr2=0x56338c5dca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:27.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.149+0000 7f928b593740 1 -- 172.21.15.120:0/1138382666 shutdown_connections 2024-09-19T09:27:27.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.149+0000 7f928b593740 1 -- 172.21.15.120:0/1138382666 wait complete. 2024-09-19T09:27:27.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.149+0000 7f928b593740 1 -- start start 2024-09-19T09:27:27.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.150+0000 7f928b593740 1 Processor -- start 2024-09-19T09:27:27.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.150+0000 7f928b593740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56338c5d34a0 0x56338c5e6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:27.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.150+0000 7f928b593740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56338c5d3dc0 0x56338c5e22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:27.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.150+0000 7f928b593740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56338c5de680 0x56338c5e2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:27.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.150+0000 7f928b593740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f92841b0b60 con 0x56338c5d34a0 2024-09-19T09:27:27.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.150+0000 7f928b593740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f92841b0490 con 0x56338c5d3dc0 2024-09-19T09:27:27.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.150+0000 7f928b593740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f92841ad6b0 con 0x56338c5de680 2024-09-19T09:27:27.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.150+0000 7f928b593740 10 client.0 _ll_register_callbacks cb 0x7f926c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:27.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.150+0000 7f9282d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56338c5d34a0 0x56338c5e6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:27.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.150+0000 7f9282d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56338c5d34a0 0x56338c5e6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42630/0 (socket says 172.21.15.120:42630) 2024-09-19T09:27:27.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.150+0000 7f9282d9d640 1 -- 172.21.15.120:0/3319357715 learned_addr learned my addr 172.21.15.120:0/3319357715 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:27.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.150+0000 7f928359e640 1 --2- 172.21.15.120:0/3319357715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56338c5de680 0x56338c5e2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:27.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.150+0000 7f927bfff640 1 --2- 172.21.15.120:0/3319357715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56338c5d3dc0 0x56338c5e22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:27.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.150+0000 7f9282d9d640 1 -- 172.21.15.120:0/3319357715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56338c5de680 msgr2=0x56338c5e2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:27.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.150+0000 7f9282d9d640 1 --2- 172.21.15.120:0/3319357715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56338c5de680 0x56338c5e2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.150+0000 7f9282d9d640 1 -- 172.21.15.120:0/3319357715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56338c5d3dc0 msgr2=0x56338c5e22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:27.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.150+0000 7f9282d9d640 1 --2- 172.21.15.120:0/3319357715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56338c5d3dc0 0x56338c5e22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.150+0000 7f9282d9d640 1 -- 172.21.15.120:0/3319357715 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56338c657e30 con 0x56338c5d34a0 2024-09-19T09:27:27.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.150+0000 7f9282d9d640 1 --2- 172.21.15.120:0/3319357715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56338c5d34a0 0x56338c5e6fb0 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f92740058c0 tx=0x7f9274066b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816945dc473015e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:27.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.150+0000 7f926a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:27:27.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.151+0000 7f926a7fc640 1 -- 172.21.15.120:0/3319357715 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274020070 con 0x56338c5d34a0 2024-09-19T09:27:27.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.151+0000 7f926a7fc640 1 -- 172.21.15.120:0/3319357715 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f92740981f0 con 0x56338c5d34a0 2024-09-19T09:27:27.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.151+0000 7f928b593740 1 -- 172.21.15.120:0/3319357715 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f927409bb80 con 0x56338c5d34a0 2024-09-19T09:27:27.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.151+0000 7f926a7fc640 1 -- 172.21.15.120:0/3319357715 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f92740cd170 con 0x56338c5d34a0 2024-09-19T09:27:27.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.151+0000 7f928b593740 10 client.23427 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:27.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.151+0000 7f928b593740 1 -- 172.21.15.120:0/3319357715 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56338c659000 con 0x56338c5d34a0 2024-09-19T09:27:27.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.151+0000 7f926a7fc640 1 -- 172.21.15.120:0/3319357715 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f927409bb80 con 0x56338c5d34a0 2024-09-19T09:27:27.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.151+0000 7f926a7fc640 1 -- 172.21.15.120:0/3319357715 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f92740cf1a0 con 0x56338c5d34a0 2024-09-19T09:27:27.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.151+0000 7f928b593740 10 client.23427 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:27.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.151+0000 7f928b593740 10 client.23427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:27:27.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.151+0000 7f928b593740 20 client.23427 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:27.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.151+0000 7f928b593740 1 --2- 172.21.15.120:0/3319357715 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56338c6596f0 0x56338c65bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:27.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.151+0000 7f928b593740 4 client.23427 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:27:27.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.151+0000 7f928b593740 1 -- 172.21.15.120:0/3319357715 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56338c5dbad0 con 0x56338c6596f0 2024-09-19T09:27:27.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.151+0000 7f927bfff640 1 --2- 172.21.15.120:0/3319357715 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56338c6596f0 0x56338c65bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:27.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.152+0000 7f926a7fc640 10 client.23427 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:27.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.152+0000 7f927bfff640 1 --2- 172.21.15.120:0/3319357715 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56338c6596f0 0x56338c65bb10 crc :-1 s=READY pgs=602 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c35d5bc63ef6321e server_cookie=1e08134ad0355523 in_seq=0 out_seq=0 2024-09-19T09:27:27.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.152+0000 7f926a7fc640 1 -- 172.21.15.120:0/3319357715 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56338c5dbad0 con 0x56338c6596f0 2024-09-19T09:27:27.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.152+0000 7f926a7fc640 10 client.23427 handle_command_reply: tid=0 2024-09-19T09:27:27.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.152+0000 7f928b593740 1 client.23427 shutdown 2024-09-19T09:27:27.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.152+0000 7f928b593740 1 -- 172.21.15.120:0/3319357715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56338c5d34a0 msgr2=0x56338c5e6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:27.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.152+0000 7f928b593740 1 --2- 172.21.15.120:0/3319357715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56338c5d34a0 0x56338c5e6fb0 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f92740058c0 tx=0x7f9274066b60 comp rx=0 tx=0).stop 2024-09-19T09:27:27.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.152+0000 7f928b593740 1 -- 172.21.15.120:0/3319357715 shutdown_connections 2024-09-19T09:27:27.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.152+0000 7f928b593740 1 --2- 172.21.15.120:0/3319357715 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x56338c6596f0 0x56338c65bb10 crc :-1 s=READY pgs=602 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.152+0000 7f928b593740 1 --2- 172.21.15.120:0/3319357715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56338c5de680 0x56338c5e2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.152+0000 7f928b593740 1 --2- 172.21.15.120:0/3319357715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56338c5d3dc0 0x56338c5e22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.152+0000 7f928b593740 1 --2- 172.21.15.120:0/3319357715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56338c5d34a0 0x56338c5e6fb0 unknown :-1 s=CLOSED pgs=2490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.152+0000 7f928b593740 1 -- 172.21.15.120:0/3319357715 >> 172.21.15.120:0/3319357715 conn(0x56338c5cfa30 msgr2=0x56338c5d99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:27.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.152+0000 7f926a7fc640 0 client.23427 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:27.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.152+0000 7f928b593740 1 -- 172.21.15.120:0/3319357715 shutdown_connections 2024-09-19T09:27:27.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.152+0000 7f928b593740 1 -- 172.21.15.120:0/3319357715 wait complete. 2024-09-19T09:27:27.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.152+0000 7f928b593740 20 client.23427 trim_cache size 0 max 16384 2024-09-19T09:27:27.161 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:27:27.161 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:27:27.161 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-19T09:27:27.161 INFO:teuthology.orchestra.run.smithi120.stdout: "4888793d-4af3-4bb3-93b5-28cfa3d26e9c": { 2024-09-19T09:27:27.161 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-19T09:27:27.161 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "4888793d-4af3-4bb3-93b5-28cfa3d26e9c", 2024-09-19T09:27:27.161 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-19T09:27:27.161 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:27:27.161 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:27:27.161 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:27:27.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.153+0000 7f92891ba640 1 -- 172.21.15.120:0/1424197424 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92700a3240 msgr2=0x7f92700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:27.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.153+0000 7f92891ba640 1 --2- 172.21.15.120:0/1424197424 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92700a3240 0x7f92700a5700 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f927c01f000 tx=0x7f927c01d000 comp rx=0 tx=0).stop 2024-09-19T09:27:27.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.153+0000 7f92891ba640 1 -- 172.21.15.120:0/1424197424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9284101240 msgr2=0x7f92840fc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:27.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.153+0000 7f92891ba640 1 --2- 172.21.15.120:0/1424197424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9284101240 0x7f92840fc690 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f927400d950 tx=0x7f927400de10 comp rx=0 tx=0).stop 2024-09-19T09:27:27.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.153+0000 7f92891ba640 1 -- 172.21.15.120:0/1424197424 shutdown_connections 2024-09-19T09:27:27.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.153+0000 7f92891ba640 1 --2- 172.21.15.120:0/1424197424 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f92700a3240 0x7f92700a5700 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.153+0000 7f92891ba640 1 --2- 172.21.15.120:0/1424197424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9284105a90 0x7f92840fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.153+0000 7f92891ba640 1 --2- 172.21.15.120:0/1424197424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9284101bc0 0x7f92840fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.153+0000 7f92891ba640 1 --2- 172.21.15.120:0/1424197424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9284101240 0x7f92840fc690 unknown :-1 s=CLOSED pgs=2486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.153+0000 7f92891ba640 1 -- 172.21.15.120:0/1424197424 >> 172.21.15.120:0/1424197424 conn(0x7f9284077210 msgr2=0x7f92841003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:27.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.153+0000 7f92891ba640 1 -- 172.21.15.120:0/1424197424 shutdown_connections 2024-09-19T09:27:27.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:27.153+0000 7f92891ba640 1 -- 172.21.15.120:0/1424197424 wait complete. 2024-09-19T09:27:27.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:27 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1167268829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:27.394 INFO:tasks.workunit.client.0.smithi120.stdout: 10 266983 20.28 MB/sec execute 562 sec latency 16376.883 ms 2024-09-19T09:27:27.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:27 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1167268829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:27.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:27 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1167268829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:27.958 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db --set-id d75d1f53 --release --await' 2024-09-19T09:27:27.958 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db --set-id d75d1f53 --release --await 2024-09-19T09:27:28.394 INFO:tasks.workunit.client.0.smithi120.stdout: 10 266983 20.25 MB/sec execute 563 sec latency 17376.965 ms 2024-09-19T09:27:28.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:28 smithi120 ceph-mon[31150]: pgmap v1083: 97 pgs: 97 active+clean; 493 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 4.1 MiB/s wr, 218 op/s 2024-09-19T09:27:28.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:28 smithi196 ceph-mon[35150]: pgmap v1083: 97 pgs: 97 active+clean; 493 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 4.1 MiB/s wr, 218 op/s 2024-09-19T09:27:28.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:28 smithi153 ceph-mon[35071]: pgmap v1083: 97 pgs: 97 active+clean; 493 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 4.1 MiB/s wr, 218 op/s 2024-09-19T09:27:28.723 INFO:tasks.cephfs.filesystem:scrub status for tag:4888793d-4af3-4bb3-93b5-28cfa3d26e9c - {'path': '/', 'tag': '4888793d-4af3-4bb3-93b5-28cfa3d26e9c', 'options': 'recursive,force'} 2024-09-19T09:27:28.855 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:29.298 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:29 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:27:29.298 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:29 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:29.299 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:29 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:27:29.299 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:29 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:27:29.394 INFO:tasks.workunit.client.0.smithi120.stdout: 10 266983 20.21 MB/sec execute 564 sec latency 18377.008 ms 2024-09-19T09:27:29.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.385+0000 7f06c0994640 1 -- 172.21.15.120:0/1264328123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06bc102570 msgr2=0x7f06bc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:29.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.385+0000 7f06c0994640 1 --2- 172.21.15.120:0/1264328123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06bc102570 0x7f06bc1029b0 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7f06a4009900 tx=0x7f06a402a480 comp rx=0 tx=0).stop 2024-09-19T09:27:29.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.385+0000 7f06c0994640 1 -- 172.21.15.120:0/1264328123 shutdown_connections 2024-09-19T09:27:29.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.385+0000 7f06c0994640 1 --2- 172.21.15.120:0/1264328123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06bc102ef0 0x7f06bc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.385+0000 7f06c0994640 1 --2- 172.21.15.120:0/1264328123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06bc102570 0x7f06bc1029b0 unknown :-1 s=CLOSED pgs=2491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.385+0000 7f06c0994640 1 --2- 172.21.15.120:0/1264328123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06bc100e20 0x7f06bc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.385+0000 7f06c0994640 1 -- 172.21.15.120:0/1264328123 >> 172.21.15.120:0/1264328123 conn(0x7f06bc0fc7d0 msgr2=0x7f06bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:29.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.385+0000 7f06c0994640 1 -- 172.21.15.120:0/1264328123 shutdown_connections 2024-09-19T09:27:29.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.385+0000 7f06c0994640 1 -- 172.21.15.120:0/1264328123 wait complete. 2024-09-19T09:27:29.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.386+0000 7f06c0994640 1 Processor -- start 2024-09-19T09:27:29.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.386+0000 7f06c0994640 1 -- start start 2024-09-19T09:27:29.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.386+0000 7f06c0994640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06bc100e20 0x7f06bc0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.386+0000 7f06c0994640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06bc102570 0x7f06bc0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.386+0000 7f06c0994640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06bc102ef0 0x7f06bc0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.386+0000 7f06c0994640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f06bc110fc0 con 0x7f06bc100e20 2024-09-19T09:27:29.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.386+0000 7f06c0994640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f06bc110e40 con 0x7f06bc102ef0 2024-09-19T09:27:29.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.386+0000 7f06c0994640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f06bc111140 con 0x7f06bc102570 2024-09-19T09:27:29.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.386+0000 7f06ba59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06bc100e20 0x7f06bc0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.386+0000 7f06ba59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06bc100e20 0x7f06bc0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42646/0 (socket says 172.21.15.120:42646) 2024-09-19T09:27:29.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.386+0000 7f06ba59c640 1 -- 172.21.15.120:0/2589918260 learned_addr learned my addr 172.21.15.120:0/2589918260 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:29.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.386+0000 7f06bad9d640 1 --2- 172.21.15.120:0/2589918260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06bc102ef0 0x7f06bc0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.386+0000 7f06b9d9b640 1 --2- 172.21.15.120:0/2589918260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06bc102570 0x7f06bc0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.386+0000 7f06ba59c640 1 -- 172.21.15.120:0/2589918260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06bc102570 msgr2=0x7f06bc0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:29.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.386+0000 7f06ba59c640 1 --2- 172.21.15.120:0/2589918260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06bc102570 0x7f06bc0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.386+0000 7f06ba59c640 1 -- 172.21.15.120:0/2589918260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06bc102ef0 msgr2=0x7f06bc0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:29.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.386+0000 7f06ba59c640 1 --2- 172.21.15.120:0/2589918260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06bc102ef0 0x7f06bc0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.386+0000 7f06ba59c640 1 -- 172.21.15.120:0/2589918260 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06bc1a69e0 con 0x7f06bc100e20 2024-09-19T09:27:29.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.386+0000 7f06ba59c640 1 --2- 172.21.15.120:0/2589918260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06bc100e20 0x7f06bc0fcc80 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7f06b400ba50 tx=0x7f06b400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:29.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.387+0000 7f06b37fe640 1 -- 172.21.15.120:0/2589918260 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4019070 con 0x7f06bc100e20 2024-09-19T09:27:29.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.387+0000 7f06b37fe640 1 -- 172.21.15.120:0/2589918260 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06b400cbe0 con 0x7f06bc100e20 2024-09-19T09:27:29.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.387+0000 7f06c0994640 1 -- 172.21.15.120:0/2589918260 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06bc1a6cd0 con 0x7f06bc100e20 2024-09-19T09:27:29.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.387+0000 7f06b37fe640 1 -- 172.21.15.120:0/2589918260 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b40076e0 con 0x7f06bc100e20 2024-09-19T09:27:29.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.387+0000 7f06c0994640 1 -- 172.21.15.120:0/2589918260 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06bc1a71c0 con 0x7f06bc100e20 2024-09-19T09:27:29.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.389+0000 7f06b37fe640 1 -- 172.21.15.120:0/2589918260 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f06b401c070 con 0x7f06bc100e20 2024-09-19T09:27:29.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.389+0000 7f06c2d6d740 1 Processor -- start 2024-09-19T09:27:29.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.389+0000 7f06c2d6d740 1 -- start start 2024-09-19T09:27:29.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.389+0000 7f06c2d6d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649f2d3e060 0x5649f2d3e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.390+0000 7f06c2d6d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649f2d3e980 0x5649f2d48d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.390+0000 7f06c2d6d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649f2d49240 0x5649f2d4b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.390+0000 7f06c2d6d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5649f2ca3620 con 0x5649f2d3e980 2024-09-19T09:27:29.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.390+0000 7f06b37fe640 1 --2- 172.21.15.120:0/2589918260 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f06a80a3240 0x7f06a80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.390+0000 7f06c2d6d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5649f2c4ae50 con 0x5649f2d3e060 2024-09-19T09:27:29.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.390+0000 7f06c2d6d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5649f2c9f5e0 con 0x5649f2d49240 2024-09-19T09:27:29.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.390+0000 7f06ba59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649f2d3e980 0x5649f2d48d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.390+0000 7f06bad9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649f2d3e060 0x5649f2d3e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.390+0000 7f06b9d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649f2d49240 0x5649f2d4b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.390+0000 7f06bad9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649f2d3e060 0x5649f2d3e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:40232/0 (socket says 172.21.15.120:40232) 2024-09-19T09:27:29.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.390+0000 7f06bad9d640 1 -- 172.21.15.120:0/552750139 learned_addr learned my addr 172.21.15.120:0/552750139 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:29.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.390+0000 7f06ba59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649f2d3e980 0x5649f2d48d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42654/0 (socket says 172.21.15.120:42654) 2024-09-19T09:27:29.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.390+0000 7f06bad9d640 1 --2- 172.21.15.120:0/2589918260 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f06a80a3240 0x7f06a80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.390+0000 7f06b37fe640 1 -- 172.21.15.120:0/2589918260 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f06b40ceec0 con 0x7f06bc100e20 2024-09-19T09:27:29.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.390+0000 7f06ba59c640 1 -- 172.21.15.120:0/552750139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649f2d49240 msgr2=0x5649f2d4b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:29.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.390+0000 7f06ba59c640 1 --2- 172.21.15.120:0/552750139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649f2d49240 0x5649f2d4b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.390+0000 7f06ba59c640 1 -- 172.21.15.120:0/552750139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649f2d3e060 msgr2=0x5649f2d3e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:29.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.390+0000 7f06ba59c640 1 --2- 172.21.15.120:0/552750139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649f2d3e060 0x5649f2d3e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.390+0000 7f06ba59c640 1 -- 172.21.15.120:0/552750139 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649f2c3bd40 con 0x5649f2d3e980 2024-09-19T09:27:29.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.390+0000 7f06bad9d640 1 --2- 172.21.15.120:0/552750139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649f2d3e060 0x5649f2d3e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:27:29.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.390+0000 7f06ba59c640 1 --2- 172.21.15.120:0/552750139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649f2d3e980 0x5649f2d48d00 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f06b40979f0 tx=0x7f06b406e410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e7596c19a2d4bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:29.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.390+0000 7f06bad9d640 1 --2- 172.21.15.120:0/2589918260 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f06a80a3240 0x7f06a80a5700 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f06ac0129f0 tx=0x7f06ac00b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:29.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.390+0000 7f06b0ff9640 1 -- 172.21.15.120:0/552750139 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b409f890 con 0x5649f2d3e980 2024-09-19T09:27:29.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.390+0000 7f06b0ff9640 1 -- 172.21.15.120:0/552750139 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06b40a4150 con 0x5649f2d3e980 2024-09-19T09:27:29.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.390+0000 7f06b0ff9640 1 -- 172.21.15.120:0/552750139 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4097d80 con 0x5649f2d3e980 2024-09-19T09:27:29.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.391+0000 7f06c2d6d740 1 -- 172.21.15.120:0/552750139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649f2d3e980 msgr2=0x5649f2d48d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:29.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.391+0000 7f06c2d6d740 1 --2- 172.21.15.120:0/552750139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649f2d3e980 0x5649f2d48d00 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f06b40979f0 tx=0x7f06b406e410 comp rx=0 tx=0).stop 2024-09-19T09:27:29.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.391+0000 7f06c2d6d740 1 -- 172.21.15.120:0/552750139 shutdown_connections 2024-09-19T09:27:29.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.391+0000 7f06c2d6d740 1 --2- 172.21.15.120:0/552750139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649f2d49240 0x5649f2d4b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.391+0000 7f06c2d6d740 1 --2- 172.21.15.120:0/552750139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649f2d3e980 0x5649f2d48d00 unknown :-1 s=CLOSED pgs=2493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.391+0000 7f06c2d6d740 1 --2- 172.21.15.120:0/552750139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649f2d3e060 0x5649f2d3e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.391+0000 7f06c2d6d740 1 -- 172.21.15.120:0/552750139 >> 172.21.15.120:0/552750139 conn(0x5649f2d3a5f0 msgr2=0x5649f2d3a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:29.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.391+0000 7f06c2d6d740 1 -- 172.21.15.120:0/552750139 shutdown_connections 2024-09-19T09:27:29.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.391+0000 7f06c2d6d740 1 -- 172.21.15.120:0/552750139 wait complete. 2024-09-19T09:27:29.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.392+0000 7f06c2d6d740 1 -- start start 2024-09-19T09:27:29.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.392+0000 7f06c2d6d740 1 Processor -- start 2024-09-19T09:27:29.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.392+0000 7f06c2d6d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649f2d3e060 0x5649f2d461b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.392+0000 7f06c2d6d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649f2d3e980 0x5649f2d466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.392+0000 7f06c2d6d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649f2d49240 0x5649f2d46c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.392+0000 7f06c2d6d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5649f2d4be10 con 0x5649f2d3e060 2024-09-19T09:27:29.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.392+0000 7f06c2d6d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5649f2c2bf10 con 0x5649f2d3e980 2024-09-19T09:27:29.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.392+0000 7f06c2d6d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5649f2d4bf90 con 0x5649f2d49240 2024-09-19T09:27:29.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.392+0000 7f06c2d6d740 10 client.0 _ll_register_callbacks cb 0x5649f2c4fdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:29.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.392+0000 7f06ba59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649f2d3e060 0x5649f2d461b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.392+0000 7f06ba59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649f2d3e060 0x5649f2d461b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42656/0 (socket says 172.21.15.120:42656) 2024-09-19T09:27:29.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.392+0000 7f06ba59c640 1 -- 172.21.15.120:0/2253945620 learned_addr learned my addr 172.21.15.120:0/2253945620 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:29.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.392+0000 7f06ba59c640 1 -- 172.21.15.120:0/2253945620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649f2d49240 msgr2=0x5649f2d46c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:29.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.392+0000 7f06ba59c640 1 --2- 172.21.15.120:0/2253945620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649f2d49240 0x5649f2d46c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.392+0000 7f06ba59c640 1 -- 172.21.15.120:0/2253945620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649f2d3e980 msgr2=0x5649f2d466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:29.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.392+0000 7f06ba59c640 1 --2- 172.21.15.120:0/2253945620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649f2d3e980 0x5649f2d466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.392+0000 7f06ba59c640 1 -- 172.21.15.120:0/2253945620 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649f29b2580 con 0x5649f2d3e060 2024-09-19T09:27:29.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.392+0000 7f06ba59c640 1 --2- 172.21.15.120:0/2253945620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649f2d3e060 0x5649f2d461b0 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f06b4000f80 tx=0x7f06b400c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d698048ad247ddca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:29.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.392+0000 7f06a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:27:29.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.393+0000 7f06a1ffb640 1 -- 172.21.15.120:0/2253945620 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b409f890 con 0x5649f2d3e060 2024-09-19T09:27:29.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.393+0000 7f06a1ffb640 1 -- 172.21.15.120:0/2253945620 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f06b40b66f0 con 0x5649f2d3e060 2024-09-19T09:27:29.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.393+0000 7f06c2d6d740 1 -- 172.21.15.120:0/2253945620 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06b40a4150 con 0x5649f2d3e060 2024-09-19T09:27:29.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.393+0000 7f06a1ffb640 1 -- 172.21.15.120:0/2253945620 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f06b40b68f0 con 0x5649f2d3e060 2024-09-19T09:27:29.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.393+0000 7f06c2d6d740 10 client.23439 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:29.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.393+0000 7f06c2d6d740 1 -- 172.21.15.120:0/2253945620 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5649f2a7c960 con 0x5649f2d3e060 2024-09-19T09:27:29.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.393+0000 7f06a1ffb640 1 -- 172.21.15.120:0/2253945620 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06b40a4150 con 0x5649f2d3e060 2024-09-19T09:27:29.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.393+0000 7f06a1ffb640 1 -- 172.21.15.120:0/2253945620 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f06b409a9e0 con 0x5649f2d3e060 2024-09-19T09:27:29.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.393+0000 7f06c2d6d740 10 client.23439 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:29.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.393+0000 7f06c2d6d740 10 client.23439 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:27:29.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.393+0000 7f06c2d6d740 20 client.23439 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:29.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.393+0000 7f06c2d6d740 1 --2- 172.21.15.120:0/2253945620 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5649f2d58cc0 0x5649f2d5b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.393+0000 7f06c2d6d740 4 client.23439 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:29.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.393+0000 7f06c2d6d740 1 -- 172.21.15.120:0/2253945620 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649f2cb0b90 con 0x5649f2d58cc0 2024-09-19T09:27:29.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.393+0000 7f06b9d9b640 1 --2- 172.21.15.120:0/2253945620 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5649f2d58cc0 0x5649f2d5b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.394+0000 7f06b9d9b640 1 --2- 172.21.15.120:0/2253945620 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5649f2d58cc0 0x5649f2d5b0b0 crc :-1 s=READY pgs=603 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f00bd25d7038da0 server_cookie=95583937d0ca96ec in_seq=0 out_seq=0 2024-09-19T09:27:29.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.394+0000 7f06a1ffb640 10 client.23439 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:29.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.394+0000 7f06a1ffb640 1 -- 172.21.15.120:0/2253945620 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5649f2cb0b90 con 0x5649f2d58cc0 2024-09-19T09:27:29.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.394+0000 7f06a1ffb640 10 client.23439 handle_command_reply: tid=0 2024-09-19T09:27:29.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.394+0000 7f06c2d6d740 1 client.23439 shutdown 2024-09-19T09:27:29.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.394+0000 7f06c2d6d740 1 -- 172.21.15.120:0/2253945620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649f2d3e060 msgr2=0x5649f2d461b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:29.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.394+0000 7f06c2d6d740 1 --2- 172.21.15.120:0/2253945620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649f2d3e060 0x5649f2d461b0 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f06b4000f80 tx=0x7f06b400c650 comp rx=0 tx=0).stop 2024-09-19T09:27:29.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.394+0000 7f06c2d6d740 1 -- 172.21.15.120:0/2253945620 shutdown_connections 2024-09-19T09:27:29.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.394+0000 7f06c2d6d740 1 --2- 172.21.15.120:0/2253945620 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5649f2d58cc0 0x5649f2d5b0b0 crc :-1 s=READY pgs=603 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.394+0000 7f06c2d6d740 1 --2- 172.21.15.120:0/2253945620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649f2d49240 0x5649f2d46c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.394+0000 7f06a1ffb640 0 client.23439 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:29.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.394+0000 7f06c2d6d740 1 --2- 172.21.15.120:0/2253945620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649f2d3e980 0x5649f2d466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.394+0000 7f06c2d6d740 1 --2- 172.21.15.120:0/2253945620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649f2d3e060 0x5649f2d461b0 unknown :-1 s=CLOSED pgs=2494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.394+0000 7f06c2d6d740 1 -- 172.21.15.120:0/2253945620 >> 172.21.15.120:0/2253945620 conn(0x5649f2d3a5f0 msgr2=0x5649f2d40320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:29.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.395+0000 7f06c2d6d740 1 -- 172.21.15.120:0/2253945620 shutdown_connections 2024-09-19T09:27:29.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.395+0000 7f06c2d6d740 1 -- 172.21.15.120:0/2253945620 wait complete. 2024-09-19T09:27:29.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.395+0000 7f06c2d6d740 20 client.23439 trim_cache size 0 max 16384 2024-09-19T09:27:29.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.412+0000 7f06c2d6d740 1 Processor -- start 2024-09-19T09:27:29.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.412+0000 7f06c2d6d740 1 -- start start 2024-09-19T09:27:29.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.412+0000 7f06c2d6d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649f2d3e060 0x5649f2d39640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.412+0000 7f06c2d6d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649f2d3e980 0x5649f2d60650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.412+0000 7f06c2d6d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649f2d49240 0x5649f2d60b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.412+0000 7f06c2d6d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f06bc1af890 con 0x5649f2d3e060 2024-09-19T09:27:29.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.412+0000 7f06c2d6d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f06bc1b13d0 con 0x5649f2d3e980 2024-09-19T09:27:29.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.412+0000 7f06c2d6d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f06bc1aff60 con 0x5649f2d49240 2024-09-19T09:27:29.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.412+0000 7f06ba59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649f2d3e060 0x5649f2d39640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.412+0000 7f06ba59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649f2d3e060 0x5649f2d39640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42668/0 (socket says 172.21.15.120:42668) 2024-09-19T09:27:29.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.412+0000 7f06ba59c640 1 -- 172.21.15.120:0/1786082379 learned_addr learned my addr 172.21.15.120:0/1786082379 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:29.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.412+0000 7f06ba59c640 1 -- 172.21.15.120:0/1786082379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649f2d49240 msgr2=0x5649f2d60b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:29.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.412+0000 7f06bad9d640 1 --2- 172.21.15.120:0/1786082379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649f2d49240 0x5649f2d60b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.412+0000 7f06b9d9b640 1 --2- 172.21.15.120:0/1786082379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649f2d3e980 0x5649f2d60650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.412+0000 7f06ba59c640 1 --2- 172.21.15.120:0/1786082379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649f2d49240 0x5649f2d60b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.412+0000 7f06ba59c640 1 -- 172.21.15.120:0/1786082379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649f2d3e980 msgr2=0x5649f2d60650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:29.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.412+0000 7f06ba59c640 1 --2- 172.21.15.120:0/1786082379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649f2d3e980 0x5649f2d60650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.412+0000 7f06ba59c640 1 -- 172.21.15.120:0/1786082379 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649f2d5b420 con 0x5649f2d3e060 2024-09-19T09:27:29.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.412+0000 7f06bad9d640 1 --2- 172.21.15.120:0/1786082379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649f2d49240 0x5649f2d60b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:29.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.412+0000 7f06ba59c640 1 --2- 172.21.15.120:0/1786082379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649f2d3e060 0x5649f2d39640 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7f06b4014d10 tx=0x7f06b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897bb6a60c3c168d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:29.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.412+0000 7f06a3fff640 1 -- 172.21.15.120:0/1786082379 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b40a4150 con 0x5649f2d3e060 2024-09-19T09:27:29.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.412+0000 7f06a3fff640 1 -- 172.21.15.120:0/1786082379 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06b40b66a0 con 0x5649f2d3e060 2024-09-19T09:27:29.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.413+0000 7f06a3fff640 1 -- 172.21.15.120:0/1786082379 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b40b6110 con 0x5649f2d3e060 2024-09-19T09:27:29.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.413+0000 7f06c2d6d740 1 -- 172.21.15.120:0/1786082379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649f2d3e060 msgr2=0x5649f2d39640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:29.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.413+0000 7f06c2d6d740 1 --2- 172.21.15.120:0/1786082379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649f2d3e060 0x5649f2d39640 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7f06b4014d10 tx=0x7f06b4004a40 comp rx=0 tx=0).stop 2024-09-19T09:27:29.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.413+0000 7f06c2d6d740 1 -- 172.21.15.120:0/1786082379 shutdown_connections 2024-09-19T09:27:29.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.413+0000 7f06c2d6d740 1 --2- 172.21.15.120:0/1786082379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649f2d49240 0x5649f2d60b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.413+0000 7f06c2d6d740 1 --2- 172.21.15.120:0/1786082379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649f2d3e980 0x5649f2d60650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.413+0000 7f06c2d6d740 1 --2- 172.21.15.120:0/1786082379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649f2d3e060 0x5649f2d39640 unknown :-1 s=CLOSED pgs=2495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.413+0000 7f06c2d6d740 1 -- 172.21.15.120:0/1786082379 >> 172.21.15.120:0/1786082379 conn(0x5649f2d3a5f0 msgr2=0x5649f2d38f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:29.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.413+0000 7f06c2d6d740 1 -- 172.21.15.120:0/1786082379 shutdown_connections 2024-09-19T09:27:29.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.413+0000 7f06c2d6d740 1 -- 172.21.15.120:0/1786082379 wait complete. 2024-09-19T09:27:29.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.413+0000 7f06c2d6d740 1 -- start start 2024-09-19T09:27:29.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.413+0000 7f06c2d6d740 1 Processor -- start 2024-09-19T09:27:29.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.414+0000 7f06c2d6d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649f2d3e060 0x5649f2dbc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.414+0000 7f06c2d6d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649f2d3e980 0x5649f2dbce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.414+0000 7f06c2d6d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649f2d49240 0x5649f2d4d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.414+0000 7f06c2d6d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f06bc1b0d00 con 0x5649f2d3e060 2024-09-19T09:27:29.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.414+0000 7f06c2d6d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f06bc1b0630 con 0x5649f2d49240 2024-09-19T09:27:29.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.414+0000 7f06c2d6d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f06bc1ad850 con 0x5649f2d3e980 2024-09-19T09:27:29.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.414+0000 7f06c2d6d740 10 client.0 _ll_register_callbacks cb 0x7f06b4097900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:29.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.414+0000 7f06ba59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649f2d3e060 0x5649f2dbc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.414+0000 7f06ba59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649f2d3e060 0x5649f2dbc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42674/0 (socket says 172.21.15.120:42674) 2024-09-19T09:27:29.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.414+0000 7f06ba59c640 1 -- 172.21.15.120:0/3790831893 learned_addr learned my addr 172.21.15.120:0/3790831893 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:29.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.414+0000 7f06bad9d640 1 --2- 172.21.15.120:0/3790831893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649f2d49240 0x5649f2d4d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.414+0000 7f06ba59c640 1 -- 172.21.15.120:0/3790831893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649f2d3e980 msgr2=0x5649f2dbce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:29.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.414+0000 7f06b9d9b640 1 --2- 172.21.15.120:0/3790831893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649f2d3e980 0x5649f2dbce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.414+0000 7f06ba59c640 1 --2- 172.21.15.120:0/3790831893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649f2d3e980 0x5649f2dbce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.414+0000 7f06ba59c640 1 -- 172.21.15.120:0/3790831893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649f2d49240 msgr2=0x5649f2d4d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:29.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.414+0000 7f06ba59c640 1 --2- 172.21.15.120:0/3790831893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649f2d49240 0x5649f2d4d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.414+0000 7f06ba59c640 1 -- 172.21.15.120:0/3790831893 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649f2d4df80 con 0x5649f2d3e060 2024-09-19T09:27:29.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.414+0000 7f06bad9d640 1 --2- 172.21.15.120:0/3790831893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649f2d49240 0x5649f2d4d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:29.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.414+0000 7f06ba59c640 1 --2- 172.21.15.120:0/3790831893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649f2d3e060 0x5649f2dbc920 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f06b40979f0 tx=0x7f06b409ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3246782a17dcf102 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:29.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.414+0000 7f06a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:27:29.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.414+0000 7f06a1ffb640 1 -- 172.21.15.120:0/3790831893 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b40b4260 con 0x5649f2d3e060 2024-09-19T09:27:29.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.414+0000 7f06a1ffb640 1 -- 172.21.15.120:0/3790831893 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f06b40b4400 con 0x5649f2d3e060 2024-09-19T09:27:29.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.414+0000 7f06c2d6d740 1 -- 172.21.15.120:0/3790831893 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06b40b66a0 con 0x5649f2d3e060 2024-09-19T09:27:29.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.415+0000 7f06a1ffb640 1 -- 172.21.15.120:0/3790831893 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f06b40b6110 con 0x5649f2d3e060 2024-09-19T09:27:29.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.415+0000 7f06c2d6d740 10 client.23445 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:29.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.415+0000 7f06c2d6d740 1 -- 172.21.15.120:0/3790831893 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5649f2d4df80 con 0x5649f2d3e060 2024-09-19T09:27:29.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.415+0000 7f06a1ffb640 1 -- 172.21.15.120:0/3790831893 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06b40b66a0 con 0x5649f2d3e060 2024-09-19T09:27:29.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.415+0000 7f06a1ffb640 1 -- 172.21.15.120:0/3790831893 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f06b409a570 con 0x5649f2d3e060 2024-09-19T09:27:29.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.415+0000 7f06c2d6d740 10 client.23445 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:29.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.415+0000 7f06c2d6d740 10 client.23445 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:27:29.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.415+0000 7f06c2d6d740 20 client.23445 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:29.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.415+0000 7f06c2d6d740 1 --2- 172.21.15.120:0/3790831893 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5649f2dc4330 0x5649f2dc6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.415+0000 7f06c2d6d740 4 client.23445 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d75d1f53", "release": true, "await": true}] 2024-09-19T09:27:29.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.415+0000 7f06c2d6d740 1 -- 172.21.15.120:0/3790831893 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d75d1f53", "release": true, "await": true}) -- 0x5649f2cab0a0 con 0x5649f2dc4330 2024-09-19T09:27:29.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.415+0000 7f06b9d9b640 1 --2- 172.21.15.120:0/3790831893 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5649f2dc4330 0x5649f2dc6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.416+0000 7f06a1ffb640 10 client.23445 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:29.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.416+0000 7f06b9d9b640 1 --2- 172.21.15.120:0/3790831893 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5649f2dc4330 0x5649f2dc6720 crc :-1 s=READY pgs=604 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c78771cbfc65a620 server_cookie=b9aaf8fb28d776d7 in_seq=0 out_seq=0 2024-09-19T09:27:29.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.479+0000 7f06a1ffb640 1 -- 172.21.15.120:0/3790831893 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5649f2cab0a0 con 0x5649f2dc4330 2024-09-19T09:27:29.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.479+0000 7f06a1ffb640 10 client.23445 handle_command_reply: tid=0 2024-09-19T09:27:29.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.479+0000 7f06c2d6d740 1 client.23445 shutdown 2024-09-19T09:27:29.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.479+0000 7f06c2d6d740 1 -- 172.21.15.120:0/3790831893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649f2d3e060 msgr2=0x5649f2dbc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:29.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.479+0000 7f06c2d6d740 1 --2- 172.21.15.120:0/3790831893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649f2d3e060 0x5649f2dbc920 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f06b40979f0 tx=0x7f06b409ad00 comp rx=0 tx=0).stop 2024-09-19T09:27:29.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.479+0000 7f06c2d6d740 1 -- 172.21.15.120:0/3790831893 shutdown_connections 2024-09-19T09:27:29.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.479+0000 7f06c2d6d740 1 --2- 172.21.15.120:0/3790831893 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5649f2dc4330 0x5649f2dc6720 crc :-1 s=READY pgs=604 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.479+0000 7f06c2d6d740 1 --2- 172.21.15.120:0/3790831893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649f2d49240 0x5649f2d4d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.479+0000 7f06a1ffb640 0 client.23445 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:29.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.479+0000 7f06c2d6d740 1 --2- 172.21.15.120:0/3790831893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649f2d3e980 0x5649f2dbce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.479+0000 7f06c2d6d740 1 --2- 172.21.15.120:0/3790831893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649f2d3e060 0x5649f2dbc920 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.479+0000 7f06c2d6d740 1 -- 172.21.15.120:0/3790831893 >> 172.21.15.120:0/3790831893 conn(0x5649f2d3a5f0 msgr2=0x5649f2d402d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:29.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.479+0000 7f06c2d6d740 1 -- 172.21.15.120:0/3790831893 shutdown_connections 2024-09-19T09:27:29.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.479+0000 7f06c2d6d740 1 -- 172.21.15.120:0/3790831893 wait complete. 2024-09-19T09:27:29.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.480+0000 7f06c2d6d740 20 client.23445 trim_cache size 0 max 16384 2024-09-19T09:27:29.482 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:27:29.482 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 39, 2024-09-19T09:27:29.482 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:27:29.482 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 84, 2024-09-19T09:27:29.483 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:27:29.483 INFO:teuthology.orchestra.run.smithi120.stdout: "d75d1f53": { 2024-09-19T09:27:29.483 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 84, 2024-09-19T09:27:29.483 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:27:29.483 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:27:29.483 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:27:29.483 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:27:29.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:27:29.483 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:27:29.483 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 138.5, 2024-09-19T09:27:29.483 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:27:29.483 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:27:29.483 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:27:29.483 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:27:29.484 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:27:29.484 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:27:29.485 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:27:29.485 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:27:29.485 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:27:29.485 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:27:29.485 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:27:29.485 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:27:29.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.480+0000 7f06c0994640 1 -- 172.21.15.120:0/2589918260 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f06a80a3240 msgr2=0x7f06a80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:29.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.480+0000 7f06c0994640 1 --2- 172.21.15.120:0/2589918260 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f06a80a3240 0x7f06a80a5700 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f06ac0129f0 tx=0x7f06ac00b2e0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.480+0000 7f06c0994640 1 -- 172.21.15.120:0/2589918260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06bc100e20 msgr2=0x7f06bc0fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:29.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.480+0000 7f06c0994640 1 --2- 172.21.15.120:0/2589918260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06bc100e20 0x7f06bc0fcc80 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7f06b400ba50 tx=0x7f06b400bf10 comp rx=0 tx=0).stop 2024-09-19T09:27:29.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.480+0000 7f06c0994640 1 -- 172.21.15.120:0/2589918260 shutdown_connections 2024-09-19T09:27:29.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.480+0000 7f06c0994640 1 --2- 172.21.15.120:0/2589918260 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f06a80a3240 0x7f06a80a5700 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.480+0000 7f06c0994640 1 --2- 172.21.15.120:0/2589918260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06bc102ef0 0x7f06bc0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.480+0000 7f06c0994640 1 --2- 172.21.15.120:0/2589918260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06bc102570 0x7f06bc0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.480+0000 7f06c0994640 1 --2- 172.21.15.120:0/2589918260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06bc100e20 0x7f06bc0fcc80 unknown :-1 s=CLOSED pgs=2492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.480+0000 7f06c0994640 1 -- 172.21.15.120:0/2589918260 >> 172.21.15.120:0/2589918260 conn(0x7f06bc0fc7d0 msgr2=0x7f06bc06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:29.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.481+0000 7f06c0994640 1 -- 172.21.15.120:0/2589918260 shutdown_connections 2024-09-19T09:27:29.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:29.481+0000 7f06c0994640 1 -- 172.21.15.120:0/2589918260 wait complete. 2024-09-19T09:27:29.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:29 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:27:29.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:29 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:29.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:29 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:27:29.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:29 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:27:29.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:29 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:27:29.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:29 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:29.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:29 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:27:29.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:29 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:27:30.300 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:30 smithi120 ceph-mon[31150]: pgmap v1084: 97 pgs: 97 active+clean; 494 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 4.2 MiB/s wr, 268 op/s 2024-09-19T09:27:30.330 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd75d1f53', total seconds elapsed: 23.8 2024-09-19T09:27:30.330 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.5 2024-09-19T09:27:30.330 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:27:30.394 INFO:tasks.workunit.client.0.smithi120.stdout: 10 267376 20.22 MB/sec execute 565 sec latency 18488.130 ms 2024-09-19T09:27:30.631 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:30.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:30 smithi196 ceph-mon[35150]: pgmap v1084: 97 pgs: 97 active+clean; 494 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 4.2 MiB/s wr, 268 op/s 2024-09-19T09:27:30.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:30 smithi153 ceph-mon[35071]: pgmap v1084: 97 pgs: 97 active+clean; 494 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 4.2 MiB/s wr, 268 op/s 2024-09-19T09:27:31.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.175+0000 7f537c309640 1 -- 172.21.15.120:0/3061116506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5374102790 msgr2=0x7f53741067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:31.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.175+0000 7f537c309640 1 --2- 172.21.15.120:0/3061116506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5374102790 0x7f53741067e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5364009900 tx=0x7f536402a410 comp rx=0 tx=0).stop 2024-09-19T09:27:31.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.175+0000 7f537c309640 1 -- 172.21.15.120:0/3061116506 shutdown_connections 2024-09-19T09:27:31.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.175+0000 7f537c309640 1 --2- 172.21.15.120:0/3061116506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5374106f10 0x7f537410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:31.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.175+0000 7f537c309640 1 --2- 172.21.15.120:0/3061116506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5374102790 0x7f53741067e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:31.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.175+0000 7f537c309640 1 --2- 172.21.15.120:0/3061116506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5374101de0 0x7f53741021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:31.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.175+0000 7f537c309640 1 -- 172.21.15.120:0/3061116506 >> 172.21.15.120:0/3061116506 conn(0x7f53740fd8c0 msgr2=0x7f53740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:31.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.176+0000 7f537c309640 1 -- 172.21.15.120:0/3061116506 shutdown_connections 2024-09-19T09:27:31.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.176+0000 7f537c309640 1 -- 172.21.15.120:0/3061116506 wait complete. 2024-09-19T09:27:31.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.176+0000 7f537c309640 1 Processor -- start 2024-09-19T09:27:31.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.176+0000 7f537c309640 1 -- start start 2024-09-19T09:27:31.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.176+0000 7f537c309640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5374101de0 0x7f5374198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:31.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.176+0000 7f537c309640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5374102790 0x7f53741989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:31.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.176+0000 7f537c309640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5374106f10 0x7f537419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:31.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.177+0000 7f537c309640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5374111fc0 con 0x7f5374101de0 2024-09-19T09:27:31.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.177+0000 7f537c309640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5374111e40 con 0x7f5374106f10 2024-09-19T09:27:31.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.177+0000 7f537c309640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5374112140 con 0x7f5374102790 2024-09-19T09:27:31.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.177+0000 7f537a0a5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5374101de0 0x7f5374198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:31.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.177+0000 7f537a0a5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5374101de0 0x7f5374198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42696/0 (socket says 172.21.15.120:42696) 2024-09-19T09:27:31.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.177+0000 7f537a0a5640 1 -- 172.21.15.120:0/3132286442 learned_addr learned my addr 172.21.15.120:0/3132286442 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:31.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.177+0000 7f537a8a6640 1 --2- 172.21.15.120:0/3132286442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5374106f10 0x7f537419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:31.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.177+0000 7f53798a4640 1 --2- 172.21.15.120:0/3132286442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5374102790 0x7f53741989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:31.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.177+0000 7f537a0a5640 1 -- 172.21.15.120:0/3132286442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5374102790 msgr2=0x7f53741989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:31.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.177+0000 7f537a0a5640 1 --2- 172.21.15.120:0/3132286442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5374102790 0x7f53741989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:31.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.177+0000 7f537a0a5640 1 -- 172.21.15.120:0/3132286442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5374106f10 msgr2=0x7f537419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:31.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.177+0000 7f537a0a5640 1 --2- 172.21.15.120:0/3132286442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5374106f10 0x7f537419cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:31.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.177+0000 7f537a0a5640 1 -- 172.21.15.120:0/3132286442 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537419d410 con 0x7f5374101de0 2024-09-19T09:27:31.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.177+0000 7f537a0a5640 1 --2- 172.21.15.120:0/3132286442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5374101de0 0x7f5374198460 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7f537000ea60 tx=0x7f537000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:31.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.178+0000 7f536b7fe640 1 -- 172.21.15.120:0/3132286442 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537000cde0 con 0x7f5374101de0 2024-09-19T09:27:31.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.178+0000 7f537c309640 1 -- 172.21.15.120:0/3132286442 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f537419d700 con 0x7f5374101de0 2024-09-19T09:27:31.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.178+0000 7f536b7fe640 1 -- 172.21.15.120:0/3132286442 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53700035f0 con 0x7f5374101de0 2024-09-19T09:27:31.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.178+0000 7f536b7fe640 1 -- 172.21.15.120:0/3132286442 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5370010670 con 0x7f5374101de0 2024-09-19T09:27:31.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.178+0000 7f537c309640 1 -- 172.21.15.120:0/3132286442 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53741a4fe0 con 0x7f5374101de0 2024-09-19T09:27:31.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.179+0000 7f537c309640 1 -- 172.21.15.120:0/3132286442 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53741021c0 con 0x7f5374101de0 2024-09-19T09:27:31.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.181+0000 7f536b7fe640 1 -- 172.21.15.120:0/3132286442 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f53700037a0 con 0x7f5374101de0 2024-09-19T09:27:31.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.181+0000 7f536b7fe640 1 --2- 172.21.15.120:0/3132286442 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f53600a3240 0x7f53600a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:31.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.182+0000 7f53798a4640 1 --2- 172.21.15.120:0/3132286442 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f53600a3240 0x7f53600a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:31.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.182+0000 7f536b7fe640 1 -- 172.21.15.120:0/3132286442 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f53700cfc80 con 0x7f5374101de0 2024-09-19T09:27:31.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.182+0000 7f53798a4640 1 --2- 172.21.15.120:0/3132286442 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f53600a3240 0x7f53600a5700 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f5364000c00 tx=0x7f5364005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:31.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.186+0000 7f536b7fe640 1 -- 172.21.15.120:0/3132286442 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f537009ca30 con 0x7f5374101de0 2024-09-19T09:27:31.394 INFO:tasks.workunit.client.0.smithi120.stdout: 10 267918 20.22 MB/sec execute 566 sec latency 93.886 ms 2024-09-19T09:27:31.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.436+0000 7f537c309640 1 -- 172.21.15.120:0/3132286442 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f537410a0f0 con 0x7f5374101de0 2024-09-19T09:27:31.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.436+0000 7f536b7fe640 1 -- 172.21.15.120:0/3132286442 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11988 (secure 0 0 0) 0x7f537009c330 con 0x7f5374101de0 2024-09-19T09:27:31.438 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:27:31.438 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":39,"btime":"2024-09-19T09:18:53:120858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:18:53.070639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:27:31.438 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 39 2024-09-19T09:27:31.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.438+0000 7f537c309640 1 -- 172.21.15.120:0/3132286442 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f53600a3240 msgr2=0x7f53600a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:31.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.438+0000 7f537c309640 1 --2- 172.21.15.120:0/3132286442 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f53600a3240 0x7f53600a5700 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f5364000c00 tx=0x7f5364005b50 comp rx=0 tx=0).stop 2024-09-19T09:27:31.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.438+0000 7f537c309640 1 -- 172.21.15.120:0/3132286442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5374101de0 msgr2=0x7f5374198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:31.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.438+0000 7f537c309640 1 --2- 172.21.15.120:0/3132286442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5374101de0 0x7f5374198460 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7f537000ea60 tx=0x7f537000ef20 comp rx=0 tx=0).stop 2024-09-19T09:27:31.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.438+0000 7f537c309640 1 -- 172.21.15.120:0/3132286442 shutdown_connections 2024-09-19T09:27:31.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.438+0000 7f537c309640 1 --2- 172.21.15.120:0/3132286442 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f53600a3240 0x7f53600a5700 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:31.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.438+0000 7f537c309640 1 --2- 172.21.15.120:0/3132286442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5374106f10 0x7f537419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:31.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.438+0000 7f537c309640 1 --2- 172.21.15.120:0/3132286442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5374102790 0x7f53741989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:31.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.438+0000 7f537c309640 1 --2- 172.21.15.120:0/3132286442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5374101de0 0x7f5374198460 unknown :-1 s=CLOSED pgs=2497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:31.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.438+0000 7f537c309640 1 -- 172.21.15.120:0/3132286442 >> 172.21.15.120:0/3132286442 conn(0x7f53740fd8c0 msgr2=0x7f53740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:31.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.439+0000 7f537c309640 1 -- 172.21.15.120:0/3132286442 shutdown_connections 2024-09-19T09:27:31.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:31.439+0000 7f537c309640 1 -- 172.21.15.120:0/3132286442 wait complete. 2024-09-19T09:27:31.825 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:31 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3132286442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:31.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:31 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3132286442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:31.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:31 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3132286442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:32.268 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db / --timeout 90 --expiration 138.5 --await-for 45.0' 2024-09-19T09:27:32.268 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db / --timeout 90 --expiration 138.5 --await-for 45.0 2024-09-19T09:27:32.394 INFO:tasks.workunit.client.0.smithi120.stdout: 10 268512 20.25 MB/sec execute 567 sec latency 48.295 ms 2024-09-19T09:27:32.566 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:32.796 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:32 smithi120 ceph-mon[31150]: pgmap v1085: 97 pgs: 97 active+clean; 492 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.7 MiB/s wr, 314 op/s 2024-09-19T09:27:32.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:32 smithi196 ceph-mon[35150]: pgmap v1085: 97 pgs: 97 active+clean; 492 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.7 MiB/s wr, 314 op/s 2024-09-19T09:27:32.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:32 smithi153 ceph-mon[35071]: pgmap v1085: 97 pgs: 97 active+clean; 492 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.7 MiB/s wr, 314 op/s 2024-09-19T09:27:33.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.136+0000 7f496e608640 1 -- 172.21.15.120:0/1511893284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4968101dd0 msgr2=0x7f49681021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:33.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.136+0000 7f496e608640 1 --2- 172.21.15.120:0/1511893284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4968101dd0 0x7f49681021b0 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f4958009900 tx=0x7f495802a410 comp rx=0 tx=0).stop 2024-09-19T09:27:33.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.136+0000 7f496e608640 1 -- 172.21.15.120:0/1511893284 shutdown_connections 2024-09-19T09:27:33.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.136+0000 7f496e608640 1 --2- 172.21.15.120:0/1511893284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4968106f00 0x7f496810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:33.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.136+0000 7f496e608640 1 --2- 172.21.15.120:0/1511893284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4968102780 0x7f49681067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:33.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.136+0000 7f496e608640 1 --2- 172.21.15.120:0/1511893284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4968101dd0 0x7f49681021b0 unknown :-1 s=CLOSED pgs=2498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:33.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.136+0000 7f496e608640 1 -- 172.21.15.120:0/1511893284 >> 172.21.15.120:0/1511893284 conn(0x7f49680fd8b0 msgr2=0x7f49680ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:33.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.136+0000 7f496e608640 1 -- 172.21.15.120:0/1511893284 shutdown_connections 2024-09-19T09:27:33.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.136+0000 7f496e608640 1 -- 172.21.15.120:0/1511893284 wait complete. 2024-09-19T09:27:33.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.137+0000 7f496e608640 1 Processor -- start 2024-09-19T09:27:33.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.137+0000 7f496e608640 1 -- start start 2024-09-19T09:27:33.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.138+0000 7f496e608640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4968101dd0 0x7f49680744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:33.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.138+0000 7f496e608640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4968102780 0x7f4968074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:33.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.138+0000 7f496e608640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4968106f00 0x7f4968074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:33.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.138+0000 7f496e608640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4968111f60 con 0x7f4968106f00 2024-09-19T09:27:33.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.138+0000 7f496e608640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4968111de0 con 0x7f4968101dd0 2024-09-19T09:27:33.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.138+0000 7f496e608640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f49681120e0 con 0x7f4968102780 2024-09-19T09:27:33.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.138+0000 7f4967fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4968101dd0 0x7f49680744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:33.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.138+0000 7f496cba5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4968106f00 0x7f4968074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:33.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.138+0000 7f4967fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4968101dd0 0x7f49680744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:43692/0 (socket says 172.21.15.120:43692) 2024-09-19T09:27:33.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.138+0000 7f4967fff640 1 -- 172.21.15.120:0/340285295 learned_addr learned my addr 172.21.15.120:0/340285295 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:33.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.138+0000 7f496cba5640 1 -- 172.21.15.120:0/340285295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4968102780 msgr2=0x7f4968074a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T09:27:33.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.138+0000 7f496cba5640 1 --2- 172.21.15.120:0/340285295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4968102780 0x7f4968074a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:33.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.138+0000 7f496cba5640 1 -- 172.21.15.120:0/340285295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4968101dd0 msgr2=0x7f49680744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:33.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.138+0000 7f496cba5640 1 --2- 172.21.15.120:0/340285295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4968101dd0 0x7f49680744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:33.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.138+0000 7f496cba5640 1 -- 172.21.15.120:0/340285295 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49681a6920 con 0x7f4968106f00 2024-09-19T09:27:33.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.138+0000 7f496cba5640 1 --2- 172.21.15.120:0/340285295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4968106f00 0x7f4968074f70 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f4960009fd0 tx=0x7f496000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:33.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.139+0000 7f49657fa640 1 -- 172.21.15.120:0/340285295 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496000ce20 con 0x7f4968106f00 2024-09-19T09:27:33.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.139+0000 7f496e608640 1 -- 172.21.15.120:0/340285295 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49681a6c10 con 0x7f4968106f00 2024-09-19T09:27:33.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.139+0000 7f49657fa640 1 -- 172.21.15.120:0/340285295 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4960005020 con 0x7f4968106f00 2024-09-19T09:27:33.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.139+0000 7f49657fa640 1 -- 172.21.15.120:0/340285295 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4960010720 con 0x7f4968106f00 2024-09-19T09:27:33.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.139+0000 7f496e608640 1 -- 172.21.15.120:0/340285295 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49681a7100 con 0x7f4968106f00 2024-09-19T09:27:33.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.142+0000 7f49657fa640 1 -- 172.21.15.120:0/340285295 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f4960005540 con 0x7f4968106f00 2024-09-19T09:27:33.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.142+0000 7f49709e1740 1 Processor -- start 2024-09-19T09:27:33.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.142+0000 7f49709e1740 1 -- start start 2024-09-19T09:27:33.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.142+0000 7f49709e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559692321760 0x559692321b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:33.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.142+0000 7f49709e1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559692322080 0x55969232c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:33.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.142+0000 7f49709e1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55969232c940 0x55969232ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:33.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.142+0000 7f49709e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5596922872a0 con 0x559692321760 2024-09-19T09:27:33.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.142+0000 7f49709e1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55969222eb10 con 0x55969232c940 2024-09-19T09:27:33.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.142+0000 7f49709e1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559692283260 con 0x559692322080 2024-09-19T09:27:33.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.143+0000 7f4967fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559692322080 0x55969232c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:33.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.143+0000 7f49657fa640 1 --2- 172.21.15.120:0/340285295 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f49500a3240 0x7f49500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:33.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.143+0000 7f4967fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559692322080 0x55969232c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:47986/0 (socket says 172.21.15.120:47986) 2024-09-19T09:27:33.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.143+0000 7f4967fff640 1 -- 172.21.15.120:0/2701844398 learned_addr learned my addr 172.21.15.120:0/2701844398 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:33.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.143+0000 7f49677fe640 1 --2- 172.21.15.120:0/2701844398 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559692321760 0x559692321b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:33.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.143+0000 7f496cba5640 1 --2- 172.21.15.120:0/340285295 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f49500a3240 0x7f49500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:33.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.143+0000 7f49677fe640 1 --2- 172.21.15.120:0/2701844398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55969232c940 0x55969232ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:33.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.143+0000 7f4967fff640 1 -- 172.21.15.120:0/2701844398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55969232c940 msgr2=0x55969232ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:33.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.143+0000 7f4967fff640 1 --2- 172.21.15.120:0/2701844398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55969232c940 0x55969232ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:33.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.143+0000 7f4967fff640 1 -- 172.21.15.120:0/2701844398 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559692321760 msgr2=0x559692321b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:33.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.143+0000 7f4967fff640 1 --2- 172.21.15.120:0/2701844398 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559692321760 0x559692321b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:33.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.143+0000 7f49657fa640 1 -- 172.21.15.120:0/340285295 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f49600d0030 con 0x7f4968106f00 2024-09-19T09:27:33.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.143+0000 7f4967fff640 1 -- 172.21.15.120:0/2701844398 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559691ff8b90 con 0x559692322080 2024-09-19T09:27:33.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.143+0000 7f49677fe640 1 --2- 172.21.15.120:0/2701844398 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559692321760 0x559692321b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:33.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.143+0000 7f496cba5640 1 --2- 172.21.15.120:0/340285295 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f49500a3240 0x7f49500a5700 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f496009c5e0 tx=0x7f4960066fe0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:33.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.143+0000 7f4967fff640 1 --2- 172.21.15.120:0/2701844398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559692322080 0x55969232c400 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4958009a20 tx=0x7f4958005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aeb9246b3f2a0857 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:33.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.144+0000 7f494e7fc640 1 -- 172.21.15.120:0/2701844398 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958040070 con 0x559692322080 2024-09-19T09:27:33.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.144+0000 7f494e7fc640 1 -- 172.21.15.120:0/2701844398 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49580317b0 con 0x559692322080 2024-09-19T09:27:33.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.144+0000 7f494e7fc640 1 -- 172.21.15.120:0/2701844398 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958035040 con 0x559692322080 2024-09-19T09:27:33.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.144+0000 7f49709e1740 1 -- 172.21.15.120:0/2701844398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559692322080 msgr2=0x55969232c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:33.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.144+0000 7f49709e1740 1 --2- 172.21.15.120:0/2701844398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559692322080 0x55969232c400 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4958009a20 tx=0x7f4958005e80 comp rx=0 tx=0).stop 2024-09-19T09:27:33.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.145+0000 7f49709e1740 1 -- 172.21.15.120:0/2701844398 shutdown_connections 2024-09-19T09:27:33.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.145+0000 7f49709e1740 1 --2- 172.21.15.120:0/2701844398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55969232c940 0x55969232ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:33.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.145+0000 7f49709e1740 1 --2- 172.21.15.120:0/2701844398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559692322080 0x55969232c400 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:33.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.145+0000 7f49709e1740 1 --2- 172.21.15.120:0/2701844398 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559692321760 0x559692321b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:33.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.145+0000 7f49709e1740 1 -- 172.21.15.120:0/2701844398 >> 172.21.15.120:0/2701844398 conn(0x55969231dcf0 msgr2=0x55969231e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:33.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.145+0000 7f49709e1740 1 -- 172.21.15.120:0/2701844398 shutdown_connections 2024-09-19T09:27:33.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.145+0000 7f49709e1740 1 -- 172.21.15.120:0/2701844398 wait complete. 2024-09-19T09:27:33.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.145+0000 7f49709e1740 1 -- start start 2024-09-19T09:27:33.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.146+0000 7f49709e1740 1 Processor -- start 2024-09-19T09:27:33.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.146+0000 7f49709e1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559692321760 0x5596923297b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:33.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.146+0000 7f49709e1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559692322080 0x559692329cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:33.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.146+0000 7f49709e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55969232c940 0x55969232a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:33.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.146+0000 7f49677fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559692321760 0x5596923297b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:33.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.146+0000 7f49677fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559692321760 0x5596923297b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:43702/0 (socket says 172.21.15.120:43702) 2024-09-19T09:27:33.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.146+0000 7f4967fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559692322080 0x559692329cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:33.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.146+0000 7f49677fe640 1 -- 172.21.15.120:0/1202541548 learned_addr learned my addr 172.21.15.120:0/1202541548 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:33.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.146+0000 7f49709e1740 1 -- 172.21.15.120:0/1202541548 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55969232f510 con 0x55969232c940 2024-09-19T09:27:33.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.146+0000 7f49709e1740 1 -- 172.21.15.120:0/1202541548 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55969220f6e0 con 0x559692321760 2024-09-19T09:27:33.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.146+0000 7f49709e1740 1 -- 172.21.15.120:0/1202541548 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55969232f690 con 0x559692322080 2024-09-19T09:27:33.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.146+0000 7f49677fe640 1 --2- 172.21.15.120:0/1202541548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55969232c940 0x55969232a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:33.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.146+0000 7f49709e1740 10 client.0 _ll_register_callbacks cb 0x559692233a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:33.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.147+0000 7f49677fe640 1 -- 172.21.15.120:0/1202541548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559692322080 msgr2=0x559692329cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:33.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.147+0000 7f49677fe640 1 --2- 172.21.15.120:0/1202541548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559692322080 0x559692329cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:33.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.147+0000 7f49677fe640 1 -- 172.21.15.120:0/1202541548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55969232c940 msgr2=0x55969232a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:33.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.147+0000 7f49677fe640 1 --2- 172.21.15.120:0/1202541548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55969232c940 0x55969232a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:33.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.147+0000 7f49677fe640 1 -- 172.21.15.120:0/1202541548 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596923389b0 con 0x559692321760 2024-09-19T09:27:33.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.147+0000 7f49677fe640 1 --2- 172.21.15.120:0/1202541548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559692321760 0x5596923297b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f495c012bd0 tx=0x7f495c00d320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4654eac748b10143 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:33.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.147+0000 7f49477fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:27:33.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.147+0000 7f49477fe640 1 -- 172.21.15.120:0/1202541548 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c010c60 con 0x559692321760 2024-09-19T09:27:33.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.147+0000 7f49477fe640 1 -- 172.21.15.120:0/1202541548 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f495c010e00 con 0x559692321760 2024-09-19T09:27:33.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.147+0000 7f49709e1740 1 -- 172.21.15.120:0/1202541548 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49580317b0 con 0x559692321760 2024-09-19T09:27:33.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.147+0000 7f49477fe640 1 -- 172.21.15.120:0/1202541548 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f495c005020 con 0x559692321760 2024-09-19T09:27:33.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.147+0000 7f49709e1740 10 client.32063 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:33.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.147+0000 7f49709e1740 1 -- 172.21.15.120:0/1202541548 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55969233b3a0 con 0x559692321760 2024-09-19T09:27:33.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.148+0000 7f49477fe640 1 -- 172.21.15.120:0/1202541548 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49580317b0 con 0x559692321760 2024-09-19T09:27:33.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.148+0000 7f49477fe640 1 -- 172.21.15.120:0/1202541548 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f495c004c90 con 0x559692321760 2024-09-19T09:27:33.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.148+0000 7f49709e1740 10 client.32063 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:33.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.148+0000 7f49709e1740 10 client.32063 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:27:33.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.148+0000 7f49709e1740 20 client.32063 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:33.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.148+0000 7f49709e1740 1 --2- 172.21.15.120:0/1202541548 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55969233c860 0x55969233ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:33.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.148+0000 7f49709e1740 4 client.32063 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:33.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.148+0000 7f49709e1740 1 -- 172.21.15.120:0/1202541548 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596922949d0 con 0x55969233c860 2024-09-19T09:27:33.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.148+0000 7f4967fff640 1 --2- 172.21.15.120:0/1202541548 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55969233c860 0x55969233ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:33.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.149+0000 7f49477fe640 10 client.32063 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:33.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.149+0000 7f4967fff640 1 --2- 172.21.15.120:0/1202541548 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55969233c860 0x55969233ec50 crc :-1 s=READY pgs=605 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e86d7fe3383c729 server_cookie=6119641d8f93b4df in_seq=0 out_seq=0 2024-09-19T09:27:33.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.153+0000 7f49477fe640 1 -- 172.21.15.120:0/1202541548 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5596922949d0 con 0x55969233c860 2024-09-19T09:27:33.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.153+0000 7f49477fe640 10 client.32063 handle_command_reply: tid=0 2024-09-19T09:27:33.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.153+0000 7f49709e1740 1 client.32063 shutdown 2024-09-19T09:27:33.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.153+0000 7f49709e1740 1 -- 172.21.15.120:0/1202541548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559692321760 msgr2=0x5596923297b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:33.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.153+0000 7f49709e1740 1 --2- 172.21.15.120:0/1202541548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559692321760 0x5596923297b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f495c012bd0 tx=0x7f495c00d320 comp rx=0 tx=0).stop 2024-09-19T09:27:33.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.153+0000 7f49709e1740 1 -- 172.21.15.120:0/1202541548 shutdown_connections 2024-09-19T09:27:33.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.153+0000 7f49709e1740 1 --2- 172.21.15.120:0/1202541548 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55969233c860 0x55969233ec50 crc :-1 s=READY pgs=605 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:33.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.153+0000 7f49709e1740 1 --2- 172.21.15.120:0/1202541548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55969232c940 0x55969232a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:33.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.153+0000 7f49709e1740 1 --2- 172.21.15.120:0/1202541548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559692322080 0x559692329cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:33.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.153+0000 7f49477fe640 0 client.32063 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:33.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.153+0000 7f49709e1740 1 --2- 172.21.15.120:0/1202541548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559692321760 0x5596923297b0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:33.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.153+0000 7f49709e1740 1 -- 172.21.15.120:0/1202541548 >> 172.21.15.120:0/1202541548 conn(0x55969231dcf0 msgr2=0x55969231ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:33.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.153+0000 7f49709e1740 1 -- 172.21.15.120:0/1202541548 shutdown_connections 2024-09-19T09:27:33.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.153+0000 7f49709e1740 1 -- 172.21.15.120:0/1202541548 wait complete. 2024-09-19T09:27:33.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.153+0000 7f49709e1740 20 client.32063 trim_cache size 0 max 16384 2024-09-19T09:27:33.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.170+0000 7f49709e1740 1 Processor -- start 2024-09-19T09:27:33.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.171+0000 7f49709e1740 1 -- start start 2024-09-19T09:27:33.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.171+0000 7f49709e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559692321760 0x55969231e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:33.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.171+0000 7f49709e1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559692322080 0x5596923238c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:33.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.171+0000 7f49709e1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55969232c940 0x5596923444b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:33.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.171+0000 7f49709e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f49681af890 con 0x559692321760 2024-09-19T09:27:33.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.171+0000 7f49709e1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f49681b13d0 con 0x55969232c940 2024-09-19T09:27:33.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.171+0000 7f49709e1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f49681aff60 con 0x559692322080 2024-09-19T09:27:33.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.171+0000 7f49677fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559692321760 0x55969231e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:33.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.171+0000 7f49677fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559692321760 0x55969231e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40520/0 (socket says 172.21.15.120:40520) 2024-09-19T09:27:33.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.171+0000 7f49677fe640 1 -- 172.21.15.120:0/3871938313 learned_addr learned my addr 172.21.15.120:0/3871938313 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:33.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.171+0000 7f4967fff640 1 --2- 172.21.15.120:0/3871938313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559692322080 0x5596923238c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:33.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.171+0000 7f49677fe640 1 --2- 172.21.15.120:0/3871938313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55969232c940 0x5596923444b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:33.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.171+0000 7f49677fe640 1 -- 172.21.15.120:0/3871938313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559692322080 msgr2=0x5596923238c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:33.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.171+0000 7f49677fe640 1 --2- 172.21.15.120:0/3871938313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559692322080 0x5596923238c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:33.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.171+0000 7f49677fe640 1 -- 172.21.15.120:0/3871938313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55969232c940 msgr2=0x5596923444b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:33.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.171+0000 7f49677fe640 1 --2- 172.21.15.120:0/3871938313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55969232c940 0x5596923444b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:33.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.171+0000 7f49677fe640 1 -- 172.21.15.120:0/3871938313 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559692328760 con 0x559692321760 2024-09-19T09:27:33.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.171+0000 7f49677fe640 1 --2- 172.21.15.120:0/3871938313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559692321760 0x55969231e0d0 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7f495c00df30 tx=0x7f495c00dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e62ee90c8af8683 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:33.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.171+0000 7f494dffb640 1 -- 172.21.15.120:0/3871938313 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c010e30 con 0x559692321760 2024-09-19T09:27:33.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.171+0000 7f494dffb640 1 -- 172.21.15.120:0/3871938313 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f495c0056a0 con 0x559692321760 2024-09-19T09:27:33.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.171+0000 7f494dffb640 1 -- 172.21.15.120:0/3871938313 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c003970 con 0x559692321760 2024-09-19T09:27:33.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.172+0000 7f49709e1740 1 -- 172.21.15.120:0/3871938313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559692321760 msgr2=0x55969231e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:33.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.172+0000 7f49709e1740 1 --2- 172.21.15.120:0/3871938313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559692321760 0x55969231e0d0 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7f495c00df30 tx=0x7f495c00dea0 comp rx=0 tx=0).stop 2024-09-19T09:27:33.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.172+0000 7f49709e1740 1 -- 172.21.15.120:0/3871938313 shutdown_connections 2024-09-19T09:27:33.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.172+0000 7f49709e1740 1 --2- 172.21.15.120:0/3871938313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55969232c940 0x5596923444b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:33.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.172+0000 7f49709e1740 1 --2- 172.21.15.120:0/3871938313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559692322080 0x5596923238c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:33.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.172+0000 7f49709e1740 1 --2- 172.21.15.120:0/3871938313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559692321760 0x55969231e0d0 unknown :-1 s=CLOSED pgs=2500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:33.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.172+0000 7f49709e1740 1 -- 172.21.15.120:0/3871938313 >> 172.21.15.120:0/3871938313 conn(0x55969231dcf0 msgr2=0x5596923231b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:33.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.172+0000 7f49709e1740 1 -- 172.21.15.120:0/3871938313 shutdown_connections 2024-09-19T09:27:33.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.172+0000 7f49709e1740 1 -- 172.21.15.120:0/3871938313 wait complete. 2024-09-19T09:27:33.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.172+0000 7f49709e1740 1 -- start start 2024-09-19T09:27:33.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.173+0000 7f49709e1740 1 Processor -- start 2024-09-19T09:27:33.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.173+0000 7f49709e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559692321760 0x559692330f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:33.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.173+0000 7f49709e1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559692322080 0x5596923314a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:33.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.173+0000 7f49709e1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55969232c940 0x5596923a0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:33.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.173+0000 7f49709e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f49681b0d00 con 0x559692321760 2024-09-19T09:27:33.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.173+0000 7f49709e1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f49681b0630 con 0x55969232c940 2024-09-19T09:27:33.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.173+0000 7f49709e1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f49681ad850 con 0x559692322080 2024-09-19T09:27:33.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.173+0000 7f49709e1740 10 client.0 _ll_register_callbacks cb 0x7f495c0127b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:33.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.173+0000 7f49677fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559692321760 0x559692330f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:33.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.173+0000 7f49677fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559692321760 0x559692330f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40530/0 (socket says 172.21.15.120:40530) 2024-09-19T09:27:33.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.173+0000 7f49677fe640 1 -- 172.21.15.120:0/3154257998 learned_addr learned my addr 172.21.15.120:0/3154257998 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:33.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.173+0000 7f4967fff640 1 --2- 172.21.15.120:0/3154257998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559692322080 0x5596923314a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:33.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.173+0000 7f49677fe640 1 --2- 172.21.15.120:0/3154257998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55969232c940 0x5596923a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:33.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.173+0000 7f49677fe640 1 -- 172.21.15.120:0/3154257998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559692322080 msgr2=0x5596923314a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:33.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.173+0000 7f49677fe640 1 --2- 172.21.15.120:0/3154257998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559692322080 0x5596923314a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:33.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.173+0000 7f49677fe640 1 -- 172.21.15.120:0/3154257998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55969232c940 msgr2=0x5596923a0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:33.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.173+0000 7f49677fe640 1 --2- 172.21.15.120:0/3154257998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55969232c940 0x5596923a0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:33.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.173+0000 7f49677fe640 1 -- 172.21.15.120:0/3154257998 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596923a0b70 con 0x559692321760 2024-09-19T09:27:33.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.174+0000 7f49677fe640 1 --2- 172.21.15.120:0/3154257998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559692321760 0x559692330f60 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7f495c005b90 tx=0x7f495c012670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11d1d2f985d87b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:33.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.174+0000 7f49477fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:27:33.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.174+0000 7f49477fe640 1 -- 172.21.15.120:0/3154257998 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c012030 con 0x559692321760 2024-09-19T09:27:33.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.174+0000 7f49709e1740 1 -- 172.21.15.120:0/3154257998 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f495c0056a0 con 0x559692321760 2024-09-19T09:27:33.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.174+0000 7f49477fe640 1 -- 172.21.15.120:0/3154257998 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f495c0121d0 con 0x559692321760 2024-09-19T09:27:33.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.174+0000 7f49477fe640 1 -- 172.21.15.120:0/3154257998 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f495c0035f0 con 0x559692321760 2024-09-19T09:27:33.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.174+0000 7f49709e1740 10 client.23469 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:33.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.174+0000 7f49709e1740 1 -- 172.21.15.120:0/3154257998 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5596923a7ee0 con 0x559692321760 2024-09-19T09:27:33.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.174+0000 7f49477fe640 1 -- 172.21.15.120:0/3154257998 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f495c0056a0 con 0x559692321760 2024-09-19T09:27:33.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.174+0000 7f49477fe640 1 -- 172.21.15.120:0/3154257998 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f495c038020 con 0x559692321760 2024-09-19T09:27:33.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.174+0000 7f49709e1740 10 client.23469 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:33.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.174+0000 7f49709e1740 10 client.23469 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:27:33.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.174+0000 7f49709e1740 20 client.23469 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:33.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.174+0000 7f49709e1740 1 --2- 172.21.15.120:0/3154257998 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55969233c860 0x5596923aa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:33.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.174+0000 7f49709e1740 4 client.23469 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.5, "await_for": 45.0}] 2024-09-19T09:27:33.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.174+0000 7f49709e1740 1 -- 172.21.15.120:0/3154257998 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.5, "await_for": 45.0}) -- 0x55969228ed20 con 0x55969233c860 2024-09-19T09:27:33.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.175+0000 7f4967fff640 1 --2- 172.21.15.120:0/3154257998 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55969233c860 0x5596923aa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:33.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.175+0000 7f49477fe640 10 client.23469 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:33.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:33.175+0000 7f4967fff640 1 --2- 172.21.15.120:0/3154257998 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55969233c860 0x5596923aa670 crc :-1 s=READY pgs=606 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=409aeca973e42cb7 server_cookie=fa09bf9cc3e28cff in_seq=0 out_seq=0 2024-09-19T09:27:33.394 INFO:tasks.workunit.client.0.smithi120.stdout: 10 268973 20.26 MB/sec execute 568 sec latency 229.588 ms 2024-09-19T09:27:33.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:33 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:27:33.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:33 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:27:33.826 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:33 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:27:33.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:33 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:27:33.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:33 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:27:33.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:33 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:27:33.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:33 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:27:33.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:33 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:27:33.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:33 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:27:34.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:34.319+0000 7f49477fe640 1 -- 172.21.15.120:0/3154257998 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55969228ed20 con 0x55969233c860 2024-09-19T09:27:34.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:34.319+0000 7f49477fe640 10 client.23469 handle_command_reply: tid=0 2024-09-19T09:27:34.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:34.319+0000 7f49709e1740 1 client.23469 shutdown 2024-09-19T09:27:34.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:34.319+0000 7f49709e1740 1 -- 172.21.15.120:0/3154257998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559692321760 msgr2=0x559692330f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:34.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:34.319+0000 7f49709e1740 1 --2- 172.21.15.120:0/3154257998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559692321760 0x559692330f60 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7f495c005b90 tx=0x7f495c012670 comp rx=0 tx=0).stop 2024-09-19T09:27:34.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:34.319+0000 7f49709e1740 1 -- 172.21.15.120:0/3154257998 shutdown_connections 2024-09-19T09:27:34.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:34.319+0000 7f49709e1740 1 --2- 172.21.15.120:0/3154257998 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55969233c860 0x5596923aa670 crc :-1 s=READY pgs=606 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:34.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:34.319+0000 7f49477fe640 0 client.23469 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:34.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:34.319+0000 7f49709e1740 1 --2- 172.21.15.120:0/3154257998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55969232c940 0x5596923a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:34.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:34.319+0000 7f49709e1740 1 --2- 172.21.15.120:0/3154257998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559692322080 0x5596923314a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:34.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:34.319+0000 7f49709e1740 1 --2- 172.21.15.120:0/3154257998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559692321760 0x559692330f60 secure :-1 s=CLOSED pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7f495c005b90 tx=0x7f495c012670 comp rx=0 tx=0).stop 2024-09-19T09:27:34.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:34.319+0000 7f49709e1740 1 -- 172.21.15.120:0/3154257998 >> 172.21.15.120:0/3154257998 conn(0x55969231dcf0 msgr2=0x5596923239d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:34.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:34.319+0000 7f49709e1740 1 -- 172.21.15.120:0/3154257998 shutdown_connections 2024-09-19T09:27:34.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:34.319+0000 7f49709e1740 1 -- 172.21.15.120:0/3154257998 wait complete. 2024-09-19T09:27:34.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:34.320+0000 7f49709e1740 20 client.23469 trim_cache size 0 max 16384 2024-09-19T09:27:34.327 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:27:34.327 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 39, 2024-09-19T09:27:34.327 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:27:34.328 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 86, 2024-09-19T09:27:34.328 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:27:34.328 INFO:teuthology.orchestra.run.smithi120.stdout: "63770883": { 2024-09-19T09:27:34.328 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 86, 2024-09-19T09:27:34.328 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:27:34.329 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:27:34.329 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:27:34.329 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:27:34.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:27:34.330 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:27:34.330 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 138.5, 2024-09-19T09:27:34.330 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:27:34.330 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:27:34.330 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:27:34.330 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:27:34.331 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "QUIESCED", 2024-09-19T09:27:34.331 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:27:34.331 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:27:34.331 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:27:34.331 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:27:34.331 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:27:34.332 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:27:34.332 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:27:34.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:34.321+0000 7f496e608640 1 -- 172.21.15.120:0/340285295 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f49500a3240 msgr2=0x7f49500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:34.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:34.321+0000 7f496e608640 1 --2- 172.21.15.120:0/340285295 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f49500a3240 0x7f49500a5700 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f496009c5e0 tx=0x7f4960066fe0 comp rx=0 tx=0).stop 2024-09-19T09:27:34.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:34.321+0000 7f496e608640 1 -- 172.21.15.120:0/340285295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4968106f00 msgr2=0x7f4968074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:34.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:34.321+0000 7f496e608640 1 --2- 172.21.15.120:0/340285295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4968106f00 0x7f4968074f70 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f4960009fd0 tx=0x7f496000ef10 comp rx=0 tx=0).stop 2024-09-19T09:27:34.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:34.321+0000 7f496e608640 1 -- 172.21.15.120:0/340285295 shutdown_connections 2024-09-19T09:27:34.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:34.321+0000 7f496e608640 1 --2- 172.21.15.120:0/340285295 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f49500a3240 0x7f49500a5700 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:34.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:34.321+0000 7f496e608640 1 --2- 172.21.15.120:0/340285295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4968106f00 0x7f4968074f70 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:34.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:34.321+0000 7f496e608640 1 --2- 172.21.15.120:0/340285295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4968102780 0x7f4968074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:34.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:34.321+0000 7f496e608640 1 --2- 172.21.15.120:0/340285295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4968101dd0 0x7f49680744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:34.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:34.321+0000 7f496e608640 1 -- 172.21.15.120:0/340285295 >> 172.21.15.120:0/340285295 conn(0x7f49680fd8b0 msgr2=0x7f49680fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:34.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:34.321+0000 7f496e608640 1 -- 172.21.15.120:0/340285295 shutdown_connections 2024-09-19T09:27:34.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:34.321+0000 7f496e608640 1 -- 172.21.15.120:0/340285295 wait complete. 2024-09-19T09:27:34.394 INFO:tasks.workunit.client.0.smithi120.stdout: 10 268981 20.22 MB/sec execute 569 sec latency 1181.726 ms 2024-09-19T09:27:34.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:34 smithi120 ceph-mon[31150]: scrub summary: idle 2024-09-19T09:27:34.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:34 smithi120 ceph-mon[31150]: pgmap v1086: 97 pgs: 97 active+clean; 492 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.5 MiB/s wr, 411 op/s 2024-09-19T09:27:34.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:34 smithi196 ceph-mon[35150]: scrub summary: idle 2024-09-19T09:27:34.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:34 smithi196 ceph-mon[35150]: pgmap v1086: 97 pgs: 97 active+clean; 492 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.5 MiB/s wr, 411 op/s 2024-09-19T09:27:34.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:34 smithi153 ceph-mon[35071]: scrub summary: idle 2024-09-19T09:27:34.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:34 smithi153 ceph-mon[35071]: pgmap v1086: 97 pgs: 97 active+clean; 492 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.5 MiB/s wr, 411 op/s 2024-09-19T09:27:35.078 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '63770883', quiesce took 4.7 seconds. Will release after: 13.8 2024-09-19T09:27:35.395 INFO:tasks.workunit.client.0.smithi120.stdout: 10 268981 20.18 MB/sec execute 570 sec latency 2181.810 ms 2024-09-19T09:27:36.394 INFO:tasks.workunit.client.0.smithi120.stdout: 10 268981 20.15 MB/sec execute 571 sec latency 3181.840 ms 2024-09-19T09:27:36.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:36 smithi196 ceph-mon[35150]: pgmap v1087: 97 pgs: 97 active+clean; 481 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.6 MiB/s wr, 387 op/s 2024-09-19T09:27:36.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:36 smithi153 ceph-mon[35071]: pgmap v1087: 97 pgs: 97 active+clean; 481 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.6 MiB/s wr, 387 op/s 2024-09-19T09:27:36.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:36 smithi120 ceph-mon[31150]: pgmap v1087: 97 pgs: 97 active+clean; 481 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.6 MiB/s wr, 387 op/s 2024-09-19T09:27:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.270+0000 7fb5539af640 1 -- 172.21.15.120:0/4275900411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb54c101b60 msgr2=0x7fb54c101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.270+0000 7fb5539af640 1 --2- 172.21.15.120:0/4275900411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb54c101b60 0x7fb54c101f60 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7fb548009900 tx=0x7fb54802a410 comp rx=0 tx=0).stop 2024-09-19T09:27:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.270+0000 7fb5539af640 1 -- 172.21.15.120:0/4275900411 shutdown_connections 2024-09-19T09:27:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.270+0000 7fb5539af640 1 --2- 172.21.15.120:0/4275900411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb54c103710 0x7fb54c109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.270+0000 7fb5539af640 1 --2- 172.21.15.120:0/4275900411 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb54c102d50 0x7fb54c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.270+0000 7fb5539af640 1 --2- 172.21.15.120:0/4275900411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb54c101b60 0x7fb54c101f60 unknown :-1 s=CLOSED pgs=2502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.270+0000 7fb5539af640 1 -- 172.21.15.120:0/4275900411 >> 172.21.15.120:0/4275900411 conn(0x7fb54c0fd2e0 msgr2=0x7fb54c0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.270+0000 7fb5539af640 1 -- 172.21.15.120:0/4275900411 shutdown_connections 2024-09-19T09:27:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.270+0000 7fb5539af640 1 -- 172.21.15.120:0/4275900411 wait complete. 2024-09-19T09:27:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.271+0000 7fb5539af640 1 Processor -- start 2024-09-19T09:27:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.271+0000 7fb5539af640 1 -- start start 2024-09-19T09:27:37.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.271+0000 7fb5539af640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb54c101b60 0x7fb54c075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:37.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.271+0000 7fb5539af640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb54c102d50 0x7fb54c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:37.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.271+0000 7fb5539af640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb54c103710 0x7fb54c074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:37.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.271+0000 7fb5539af640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c10d7c0 con 0x7fb54c101b60 2024-09-19T09:27:37.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.271+0000 7fb5539af640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c10d640 con 0x7fb54c102d50 2024-09-19T09:27:37.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.271+0000 7fb5539af640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c10d940 con 0x7fb54c103710 2024-09-19T09:27:37.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.271+0000 7fb55174b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb54c101b60 0x7fb54c075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:37.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.271+0000 7fb55174b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb54c101b60 0x7fb54c075fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40554/0 (socket says 172.21.15.120:40554) 2024-09-19T09:27:37.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.271+0000 7fb55174b640 1 -- 172.21.15.120:0/3088710151 learned_addr learned my addr 172.21.15.120:0/3088710151 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:37.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.271+0000 7fb551f4c640 1 --2- 172.21.15.120:0/3088710151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb54c103710 0x7fb54c074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:37.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.271+0000 7fb550f4a640 1 --2- 172.21.15.120:0/3088710151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb54c102d50 0x7fb54c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:37.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.271+0000 7fb55174b640 1 -- 172.21.15.120:0/3088710151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb54c103710 msgr2=0x7fb54c074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:37.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.272+0000 7fb55174b640 1 --2- 172.21.15.120:0/3088710151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb54c103710 0x7fb54c074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.276 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.272+0000 7fb55174b640 1 -- 172.21.15.120:0/3088710151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb54c102d50 msgr2=0x7fb54c0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.272+0000 7fb55174b640 1 --2- 172.21.15.120:0/3088710151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb54c102d50 0x7fb54c0745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.272+0000 7fb55174b640 1 -- 172.21.15.120:0/3088710151 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54c0753b0 con 0x7fb54c101b60 2024-09-19T09:27:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.272+0000 7fb55174b640 1 --2- 172.21.15.120:0/3088710151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb54c101b60 0x7fb54c075fa0 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fb548031610 tx=0x7fb5480043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.272+0000 7fb5427fc640 1 -- 172.21.15.120:0/3088710151 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54803f030 con 0x7fb54c101b60 2024-09-19T09:27:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.272+0000 7fb5427fc640 1 -- 172.21.15.120:0/3088710151 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb548003ce0 con 0x7fb54c101b60 2024-09-19T09:27:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.272+0000 7fb5539af640 1 -- 172.21.15.120:0/3088710151 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb54c1a1ea0 con 0x7fb54c101b60 2024-09-19T09:27:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.272+0000 7fb5427fc640 1 -- 172.21.15.120:0/3088710151 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb548035070 con 0x7fb54c101b60 2024-09-19T09:27:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.272+0000 7fb5539af640 1 -- 172.21.15.120:0/3088710151 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb54c1a22b0 con 0x7fb54c101b60 2024-09-19T09:27:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.274+0000 7fb5427fc640 1 -- 172.21.15.120:0/3088710151 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fb548042030 con 0x7fb54c101b60 2024-09-19T09:27:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.275+0000 7fb556823740 1 Processor -- start 2024-09-19T09:27:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.275+0000 7fb556823740 1 -- start start 2024-09-19T09:27:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.275+0000 7fb556823740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622165d39a0 0x5622165d3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.275+0000 7fb556823740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622165d42e0 0x5622165d8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.275+0000 7fb556823740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622165d9370 0x5622165db830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.275+0000 7fb556823740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5622165372f0 con 0x5622165d9370 2024-09-19T09:27:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.275+0000 7fb556823740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5622164df270 con 0x5622165d39a0 2024-09-19T09:27:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.275+0000 7fb556823740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5622165332b0 con 0x5622165d42e0 2024-09-19T09:27:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.275+0000 7fb550f4a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622165d9370 0x5622165db830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.275+0000 7fb5427fc640 1 --2- 172.21.15.120:0/3088710151 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb5380a32f0 0x7fb5380a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.275+0000 7fb550f4a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622165d9370 0x5622165db830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40560/0 (socket says 172.21.15.120:40560) 2024-09-19T09:27:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.275+0000 7fb550f4a640 1 -- 172.21.15.120:0/3029033085 learned_addr learned my addr 172.21.15.120:0/3029033085 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.275+0000 7fb55174b640 1 --2- 172.21.15.120:0/3029033085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622165d42e0 0x5622165d8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.276+0000 7fb551f4c640 1 --2- 172.21.15.120:0/3088710151 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb5380a32f0 0x7fb5380a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.276+0000 7fb551f4c640 1 --2- 172.21.15.120:0/3029033085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622165d39a0 0x5622165d3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.276+0000 7fb5427fc640 1 -- 172.21.15.120:0/3088710151 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fb5480efe40 con 0x7fb54c101b60 2024-09-19T09:27:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.276+0000 7fb550f4a640 1 -- 172.21.15.120:0/3029033085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622165d42e0 msgr2=0x5622165d8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.276+0000 7fb550f4a640 1 --2- 172.21.15.120:0/3029033085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622165d42e0 0x5622165d8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.276+0000 7fb550f4a640 1 -- 172.21.15.120:0/3029033085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622165d39a0 msgr2=0x5622165d3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.276+0000 7fb550f4a640 1 --2- 172.21.15.120:0/3029033085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622165d39a0 0x5622165d3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.276+0000 7fb550f4a640 1 -- 172.21.15.120:0/3029033085 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622164de340 con 0x5622165d9370 2024-09-19T09:27:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.276+0000 7fb55174b640 1 --2- 172.21.15.120:0/3029033085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622165d42e0 0x5622165d8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.276+0000 7fb551f4c640 1 --2- 172.21.15.120:0/3029033085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622165d39a0 0x5622165d3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.276+0000 7fb550f4a640 1 --2- 172.21.15.120:0/3029033085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622165d9370 0x5622165db830 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7fb53c005bc0 tx=0x7fb53c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42872cf067131836 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.276+0000 7fb551f4c640 1 --2- 172.21.15.120:0/3088710151 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb5380a32f0 0x7fb5380a57b0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fb544009fd0 tx=0x7fb544009840 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.276+0000 7fb5377fe640 1 -- 172.21.15.120:0/3029033085 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c016070 con 0x5622165d9370 2024-09-19T09:27:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.276+0000 7fb5377fe640 1 -- 172.21.15.120:0/3029033085 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb53c005500 con 0x5622165d9370 2024-09-19T09:27:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.276+0000 7fb5377fe640 1 -- 172.21.15.120:0/3029033085 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c011470 con 0x5622165d9370 2024-09-19T09:27:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.277+0000 7fb556823740 1 -- 172.21.15.120:0/3029033085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622165d9370 msgr2=0x5622165db830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.277+0000 7fb556823740 1 --2- 172.21.15.120:0/3029033085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622165d9370 0x5622165db830 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7fb53c005bc0 tx=0x7fb53c00e6c0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.277+0000 7fb556823740 1 -- 172.21.15.120:0/3029033085 shutdown_connections 2024-09-19T09:27:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.277+0000 7fb556823740 1 --2- 172.21.15.120:0/3029033085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622165d9370 0x5622165db830 unknown :-1 s=CLOSED pgs=2504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.277+0000 7fb556823740 1 --2- 172.21.15.120:0/3029033085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622165d42e0 0x5622165d8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.277+0000 7fb556823740 1 --2- 172.21.15.120:0/3029033085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622165d39a0 0x5622165d3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.277+0000 7fb556823740 1 -- 172.21.15.120:0/3029033085 >> 172.21.15.120:0/3029033085 conn(0x5622165cf010 msgr2=0x5622165cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.277+0000 7fb556823740 1 -- 172.21.15.120:0/3029033085 shutdown_connections 2024-09-19T09:27:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.277+0000 7fb556823740 1 -- 172.21.15.120:0/3029033085 wait complete. 2024-09-19T09:27:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.277+0000 7fb556823740 1 -- start start 2024-09-19T09:27:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.277+0000 7fb556823740 1 Processor -- start 2024-09-19T09:27:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.278+0000 7fb556823740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622165d39a0 0x5622165ddb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.278+0000 7fb556823740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622165d42e0 0x5622165de050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.278+0000 7fb556823740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622165d9370 0x5622165e5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.278+0000 7fb556823740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5622165dc340 con 0x5622165d9370 2024-09-19T09:27:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.278+0000 7fb556823740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5622164c0290 con 0x5622165d39a0 2024-09-19T09:27:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.278+0000 7fb556823740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5622165dc4c0 con 0x5622165d42e0 2024-09-19T09:27:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.278+0000 7fb556823740 10 client.0 _ll_register_callbacks cb 0x5622164f1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.278+0000 7fb551f4c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622165d9370 0x5622165e5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.278+0000 7fb551f4c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622165d9370 0x5622165e5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40570/0 (socket says 172.21.15.120:40570) 2024-09-19T09:27:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.278+0000 7fb551f4c640 1 -- 172.21.15.120:0/3906663436 learned_addr learned my addr 172.21.15.120:0/3906663436 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.278+0000 7fb550f4a640 1 --2- 172.21.15.120:0/3906663436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622165d42e0 0x5622165de050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.278+0000 7fb55174b640 1 --2- 172.21.15.120:0/3906663436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622165d39a0 0x5622165ddb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.278+0000 7fb551f4c640 1 -- 172.21.15.120:0/3906663436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622165d42e0 msgr2=0x5622165de050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.278+0000 7fb551f4c640 1 --2- 172.21.15.120:0/3906663436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622165d42e0 0x5622165de050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.278+0000 7fb551f4c640 1 -- 172.21.15.120:0/3906663436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622165d39a0 msgr2=0x5622165ddb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.278+0000 7fb551f4c640 1 --2- 172.21.15.120:0/3906663436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622165d39a0 0x5622165ddb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.278+0000 7fb551f4c640 1 -- 172.21.15.120:0/3906663436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56221624a210 con 0x5622165d9370 2024-09-19T09:27:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.278+0000 7fb550f4a640 1 --2- 172.21.15.120:0/3906663436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622165d42e0 0x5622165de050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.278+0000 7fb551f4c640 1 --2- 172.21.15.120:0/3906663436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622165d9370 0x5622165e5090 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7fb544010b10 tx=0x7fb544004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da9e2a3ea159e3f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.278+0000 7fb534ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:27:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.279+0000 7fb534ff9640 1 -- 172.21.15.120:0/3906663436 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb544025070 con 0x5622165d9370 2024-09-19T09:27:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.279+0000 7fb534ff9640 1 -- 172.21.15.120:0/3906663436 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fb544004d20 con 0x5622165d9370 2024-09-19T09:27:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.279+0000 7fb556823740 1 -- 172.21.15.120:0/3906663436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb53c005500 con 0x5622165d9370 2024-09-19T09:27:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.279+0000 7fb534ff9640 1 -- 172.21.15.120:0/3906663436 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb54401eb10 con 0x5622165d9370 2024-09-19T09:27:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.279+0000 7fb556823740 10 client.23481 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.279+0000 7fb556823740 1 -- 172.21.15.120:0/3906663436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562216316620 con 0x5622165d9370 2024-09-19T09:27:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.279+0000 7fb534ff9640 1 -- 172.21.15.120:0/3906663436 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb53c005500 con 0x5622165d9370 2024-09-19T09:27:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.279+0000 7fb534ff9640 1 -- 172.21.15.120:0/3906663436 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fb54400f040 con 0x5622165d9370 2024-09-19T09:27:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.279+0000 7fb556823740 10 client.23481 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.279+0000 7fb556823740 10 client.23481 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:27:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.279+0000 7fb556823740 20 client.23481 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.279+0000 7fb556823740 1 --2- 172.21.15.120:0/3906663436 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5622165e9850 0x5622165ebc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.279+0000 7fb556823740 4 client.23481 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.279+0000 7fb556823740 1 -- 172.21.15.120:0/3906663436 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562216544a20 con 0x5622165e9850 2024-09-19T09:27:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.280+0000 7fb55174b640 1 --2- 172.21.15.120:0/3906663436 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5622165e9850 0x5622165ebc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.280+0000 7fb55174b640 1 --2- 172.21.15.120:0/3906663436 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5622165e9850 0x5622165ebc80 crc :-1 s=READY pgs=607 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4317cb809204e010 server_cookie=691c3a0bd3de5de2 in_seq=0 out_seq=0 2024-09-19T09:27:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.280+0000 7fb534ff9640 10 client.23481 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.281+0000 7fb534ff9640 1 -- 172.21.15.120:0/3906663436 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562216544a20 con 0x5622165e9850 2024-09-19T09:27:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.281+0000 7fb534ff9640 10 client.23481 handle_command_reply: tid=0 2024-09-19T09:27:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.281+0000 7fb556823740 1 client.23481 shutdown 2024-09-19T09:27:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.281+0000 7fb556823740 1 -- 172.21.15.120:0/3906663436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622165d9370 msgr2=0x5622165e5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.281+0000 7fb556823740 1 --2- 172.21.15.120:0/3906663436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622165d9370 0x5622165e5090 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7fb544010b10 tx=0x7fb544004a40 comp rx=0 tx=0).stop 2024-09-19T09:27:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.281+0000 7fb556823740 1 -- 172.21.15.120:0/3906663436 shutdown_connections 2024-09-19T09:27:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.281+0000 7fb556823740 1 --2- 172.21.15.120:0/3906663436 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5622165e9850 0x5622165ebc80 crc :-1 s=READY pgs=607 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.281+0000 7fb556823740 1 --2- 172.21.15.120:0/3906663436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622165d9370 0x5622165e5090 unknown :-1 s=CLOSED pgs=2505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.281+0000 7fb556823740 1 --2- 172.21.15.120:0/3906663436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622165d42e0 0x5622165de050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.281+0000 7fb534ff9640 0 client.23481 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.281+0000 7fb556823740 1 --2- 172.21.15.120:0/3906663436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622165d39a0 0x5622165ddb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.281+0000 7fb556823740 1 -- 172.21.15.120:0/3906663436 >> 172.21.15.120:0/3906663436 conn(0x5622165cf010 msgr2=0x5622165d0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.281+0000 7fb556823740 1 -- 172.21.15.120:0/3906663436 shutdown_connections 2024-09-19T09:27:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.281+0000 7fb556823740 1 -- 172.21.15.120:0/3906663436 wait complete. 2024-09-19T09:27:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.281+0000 7fb556823740 20 client.23481 trim_cache size 0 max 16384 2024-09-19T09:27:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.295+0000 7fb556823740 1 Processor -- start 2024-09-19T09:27:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.295+0000 7fb556823740 1 -- start start 2024-09-19T09:27:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.296+0000 7fb556823740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622165d39a0 0x5622165d3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.296+0000 7fb556823740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622165d42e0 0x5622165e5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.296+0000 7fb556823740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622165d9370 0x5622165cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.296+0000 7fb556823740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c1ab240 con 0x5622165d39a0 2024-09-19T09:27:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.296+0000 7fb556823740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c1acd80 con 0x5622165d42e0 2024-09-19T09:27:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.296+0000 7fb556823740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c1ab910 con 0x5622165d9370 2024-09-19T09:27:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.296+0000 7fb55174b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622165d39a0 0x5622165d3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.296+0000 7fb55174b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622165d39a0 0x5622165d3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40578/0 (socket says 172.21.15.120:40578) 2024-09-19T09:27:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.296+0000 7fb55174b640 1 -- 172.21.15.120:0/2503501670 learned_addr learned my addr 172.21.15.120:0/2503501670 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.296+0000 7fb551f4c640 1 --2- 172.21.15.120:0/2503501670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622165d9370 0x5622165cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.296+0000 7fb55174b640 1 -- 172.21.15.120:0/2503501670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622165d9370 msgr2=0x5622165cfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.296+0000 7fb55174b640 1 --2- 172.21.15.120:0/2503501670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622165d9370 0x5622165cfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.296+0000 7fb55174b640 1 -- 172.21.15.120:0/2503501670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622165d42e0 msgr2=0x5622165e5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.296+0000 7fb550f4a640 1 --2- 172.21.15.120:0/2503501670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622165d42e0 0x5622165e5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.296+0000 7fb55174b640 1 --2- 172.21.15.120:0/2503501670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622165d42e0 0x5622165e5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.296+0000 7fb55174b640 1 -- 172.21.15.120:0/2503501670 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622165d47c0 con 0x5622165d39a0 2024-09-19T09:27:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.296+0000 7fb55174b640 1 --2- 172.21.15.120:0/2503501670 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622165d39a0 0x5622165d3d80 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7fb5480bc3f0 tx=0x7fb54808c0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a3eb4116c593df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.296+0000 7fb536ffd640 1 -- 172.21.15.120:0/2503501670 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb548049040 con 0x5622165d39a0 2024-09-19T09:27:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.296+0000 7fb536ffd640 1 -- 172.21.15.120:0/2503501670 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5480bc460 con 0x5622165d39a0 2024-09-19T09:27:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.296+0000 7fb536ffd640 1 -- 172.21.15.120:0/2503501670 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5480eda50 con 0x5622165d39a0 2024-09-19T09:27:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.297+0000 7fb556823740 1 -- 172.21.15.120:0/2503501670 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622165d39a0 msgr2=0x5622165d3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.297+0000 7fb556823740 1 --2- 172.21.15.120:0/2503501670 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622165d39a0 0x5622165d3d80 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7fb5480bc3f0 tx=0x7fb54808c0d0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.297+0000 7fb556823740 1 -- 172.21.15.120:0/2503501670 shutdown_connections 2024-09-19T09:27:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.297+0000 7fb556823740 1 --2- 172.21.15.120:0/2503501670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622165d9370 0x5622165cfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.297+0000 7fb556823740 1 --2- 172.21.15.120:0/2503501670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622165d42e0 0x5622165e5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.297+0000 7fb556823740 1 --2- 172.21.15.120:0/2503501670 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622165d39a0 0x5622165d3d80 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.297+0000 7fb556823740 1 -- 172.21.15.120:0/2503501670 >> 172.21.15.120:0/2503501670 conn(0x5622165cf010 msgr2=0x5622165ddb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.297+0000 7fb556823740 1 -- 172.21.15.120:0/2503501670 shutdown_connections 2024-09-19T09:27:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.297+0000 7fb556823740 1 -- 172.21.15.120:0/2503501670 wait complete. 2024-09-19T09:27:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.297+0000 7fb556823740 1 -- start start 2024-09-19T09:27:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.297+0000 7fb556823740 1 Processor -- start 2024-09-19T09:27:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.298+0000 7fb556823740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622165d39a0 0x5622165e2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.298+0000 7fb556823740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622165d42e0 0x5622165e2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.298+0000 7fb556823740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622165d9370 0x5622165e3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.298+0000 7fb556823740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c1ac6b0 con 0x5622165d42e0 2024-09-19T09:27:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.298+0000 7fb556823740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c1abfe0 con 0x5622165d9370 2024-09-19T09:27:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.298+0000 7fb556823740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c1a9200 con 0x5622165d39a0 2024-09-19T09:27:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.298+0000 7fb556823740 10 client.0 _ll_register_callbacks cb 0x7fb544009a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.298+0000 7fb550f4a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622165d42e0 0x5622165e2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.298+0000 7fb550f4a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622165d42e0 0x5622165e2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40592/0 (socket says 172.21.15.120:40592) 2024-09-19T09:27:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.298+0000 7fb550f4a640 1 -- 172.21.15.120:0/1426992594 learned_addr learned my addr 172.21.15.120:0/1426992594 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.298+0000 7fb55174b640 1 --2- 172.21.15.120:0/1426992594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622165d39a0 0x5622165e2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.298+0000 7fb551f4c640 1 --2- 172.21.15.120:0/1426992594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622165d9370 0x5622165e3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.298+0000 7fb550f4a640 1 -- 172.21.15.120:0/1426992594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622165d39a0 msgr2=0x5622165e2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.298+0000 7fb550f4a640 1 --2- 172.21.15.120:0/1426992594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622165d39a0 0x5622165e2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.298+0000 7fb550f4a640 1 -- 172.21.15.120:0/1426992594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622165d9370 msgr2=0x5622165e3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.298+0000 7fb550f4a640 1 --2- 172.21.15.120:0/1426992594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622165d9370 0x5622165e3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.298+0000 7fb550f4a640 1 -- 172.21.15.120:0/1426992594 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622165e42c0 con 0x5622165d42e0 2024-09-19T09:27:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.298+0000 7fb551f4c640 1 --2- 172.21.15.120:0/1426992594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622165d9370 0x5622165e3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.298+0000 7fb550f4a640 1 --2- 172.21.15.120:0/1426992594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622165d42e0 0x5622165e2ca0 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7fb53c006010 tx=0x7fb53c009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69337fbfc36dae6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.298+0000 7fb534ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:27:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.298+0000 7fb534ff9640 1 -- 172.21.15.120:0/1426992594 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c029060 con 0x5622165d42e0 2024-09-19T09:27:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.298+0000 7fb534ff9640 1 -- 172.21.15.120:0/1426992594 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fb53c004830 con 0x5622165d42e0 2024-09-19T09:27:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.298+0000 7fb556823740 1 -- 172.21.15.120:0/1426992594 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5480bc460 con 0x5622165d42e0 2024-09-19T09:27:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.299+0000 7fb534ff9640 1 -- 172.21.15.120:0/1426992594 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb53c016040 con 0x5622165d42e0 2024-09-19T09:27:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.299+0000 7fb556823740 10 client.23487 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.299+0000 7fb556823740 1 -- 172.21.15.120:0/1426992594 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5622166545c0 con 0x5622165d42e0 2024-09-19T09:27:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.299+0000 7fb534ff9640 1 -- 172.21.15.120:0/1426992594 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5480bc460 con 0x5622165d42e0 2024-09-19T09:27:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.299+0000 7fb534ff9640 1 -- 172.21.15.120:0/1426992594 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fb53c023070 con 0x5622165d42e0 2024-09-19T09:27:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.299+0000 7fb556823740 10 client.23487 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.299+0000 7fb556823740 10 client.23487 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:27:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.299+0000 7fb556823740 20 client.23487 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.299+0000 7fb556823740 1 --2- 172.21.15.120:0/1426992594 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562216654c80 0x562216657070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.299+0000 7fb556823740 4 client.23487 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:27:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.299+0000 7fb556823740 1 -- 172.21.15.120:0/1426992594 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5622165d9130 con 0x562216654c80 2024-09-19T09:27:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.299+0000 7fb55174b640 1 --2- 172.21.15.120:0/1426992594 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562216654c80 0x562216657070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.300+0000 7fb534ff9640 10 client.23487 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.300+0000 7fb55174b640 1 --2- 172.21.15.120:0/1426992594 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562216654c80 0x562216657070 crc :-1 s=READY pgs=608 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48684574c9b0020b server_cookie=5f5c9db17578d239 in_seq=0 out_seq=0 2024-09-19T09:27:37.395 INFO:tasks.workunit.client.0.smithi120.stdout: 10 268981 20.11 MB/sec execute 572 sec latency 4182.388 ms 2024-09-19T09:27:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.633+0000 7fb534ff9640 1 -- 172.21.15.120:0/1426992594 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5622165d9130 con 0x562216654c80 2024-09-19T09:27:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.633+0000 7fb534ff9640 10 client.23487 handle_command_reply: tid=0 2024-09-19T09:27:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.633+0000 7fb556823740 1 client.23487 shutdown 2024-09-19T09:27:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.633+0000 7fb556823740 1 -- 172.21.15.120:0/1426992594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622165d42e0 msgr2=0x5622165e2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.633+0000 7fb556823740 1 --2- 172.21.15.120:0/1426992594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622165d42e0 0x5622165e2ca0 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7fb53c006010 tx=0x7fb53c009ed0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.633+0000 7fb556823740 1 -- 172.21.15.120:0/1426992594 shutdown_connections 2024-09-19T09:27:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.633+0000 7fb556823740 1 --2- 172.21.15.120:0/1426992594 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x562216654c80 0x562216657070 crc :-1 s=READY pgs=608 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.633+0000 7fb556823740 1 --2- 172.21.15.120:0/1426992594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5622165d9370 0x5622165e3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.633+0000 7fb556823740 1 --2- 172.21.15.120:0/1426992594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5622165d42e0 0x5622165e2ca0 unknown :-1 s=CLOSED pgs=2507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.633+0000 7fb556823740 1 --2- 172.21.15.120:0/1426992594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622165d39a0 0x5622165e2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.633+0000 7fb534ff9640 0 client.23487 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.633+0000 7fb556823740 1 -- 172.21.15.120:0/1426992594 >> 172.21.15.120:0/1426992594 conn(0x5622165cf010 msgr2=0x5622165cd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.633+0000 7fb556823740 1 -- 172.21.15.120:0/1426992594 shutdown_connections 2024-09-19T09:27:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.633+0000 7fb556823740 1 -- 172.21.15.120:0/1426992594 wait complete. 2024-09-19T09:27:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.633+0000 7fb556823740 20 client.23487 trim_cache size 0 max 16384 2024-09-19T09:27:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:27:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:27:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:27:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:27:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.633+0000 7fb5539af640 1 -- 172.21.15.120:0/3088710151 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb5380a32f0 msgr2=0x7fb5380a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.633+0000 7fb5539af640 1 --2- 172.21.15.120:0/3088710151 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb5380a32f0 0x7fb5380a57b0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fb544009fd0 tx=0x7fb544009840 comp rx=0 tx=0).stop 2024-09-19T09:27:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.633+0000 7fb5539af640 1 -- 172.21.15.120:0/3088710151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb54c101b60 msgr2=0x7fb54c075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.633+0000 7fb5539af640 1 --2- 172.21.15.120:0/3088710151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb54c101b60 0x7fb54c075fa0 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fb548031610 tx=0x7fb5480043b0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.633+0000 7fb5539af640 1 -- 172.21.15.120:0/3088710151 shutdown_connections 2024-09-19T09:27:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.633+0000 7fb5539af640 1 --2- 172.21.15.120:0/3088710151 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fb5380a32f0 0x7fb5380a57b0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.633+0000 7fb5539af640 1 --2- 172.21.15.120:0/3088710151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb54c103710 0x7fb54c074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.633+0000 7fb5539af640 1 --2- 172.21.15.120:0/3088710151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb54c102d50 0x7fb54c0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.633+0000 7fb5539af640 1 --2- 172.21.15.120:0/3088710151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb54c101b60 0x7fb54c075fa0 unknown :-1 s=CLOSED pgs=2503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.633+0000 7fb5539af640 1 -- 172.21.15.120:0/3088710151 >> 172.21.15.120:0/3088710151 conn(0x7fb54c0fd2e0 msgr2=0x7fb54c1060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.633+0000 7fb5539af640 1 -- 172.21.15.120:0/3088710151 shutdown_connections 2024-09-19T09:27:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:37.633+0000 7fb5539af640 1 -- 172.21.15.120:0/3088710151 wait complete. 2024-09-19T09:27:38.395 INFO:tasks.workunit.client.0.smithi120.stdout: 10 268981 20.08 MB/sec execute 573 sec latency 5182.490 ms 2024-09-19T09:27:38.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:38 smithi196 ceph-mon[35150]: pgmap v1088: 97 pgs: 97 active+clean; 477 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.3 MiB/s wr, 448 op/s 2024-09-19T09:27:38.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:38 smithi153 ceph-mon[35071]: pgmap v1088: 97 pgs: 97 active+clean; 477 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.3 MiB/s wr, 448 op/s 2024-09-19T09:27:38.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:38 smithi120 ceph-mon[31150]: pgmap v1088: 97 pgs: 97 active+clean; 477 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.3 MiB/s wr, 448 op/s 2024-09-19T09:27:39.395 INFO:tasks.workunit.client.0.smithi120.stdout: 10 268981 20.04 MB/sec execute 574 sec latency 6182.584 ms 2024-09-19T09:27:40.395 INFO:tasks.workunit.client.0.smithi120.stdout: 10 268981 20.01 MB/sec execute 575 sec latency 7182.681 ms 2024-09-19T09:27:40.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:40 smithi196 ceph-mon[35150]: pgmap v1089: 97 pgs: 97 active+clean; 465 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.6 MiB/s wr, 419 op/s 2024-09-19T09:27:40.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:40 smithi153 ceph-mon[35071]: pgmap v1089: 97 pgs: 97 active+clean; 465 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.6 MiB/s wr, 419 op/s 2024-09-19T09:27:40.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:40 smithi120 ceph-mon[31150]: pgmap v1089: 97 pgs: 97 active+clean; 465 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.6 MiB/s wr, 419 op/s 2024-09-19T09:27:41.395 INFO:tasks.workunit.client.0.smithi120.stdout: 10 268981 19.97 MB/sec execute 576 sec latency 8182.766 ms 2024-09-19T09:27:42.395 INFO:tasks.workunit.client.0.smithi120.stdout: 10 268981 19.94 MB/sec execute 577 sec latency 9182.863 ms 2024-09-19T09:27:42.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:42 smithi196 ceph-mon[35150]: pgmap v1090: 97 pgs: 97 active+clean; 466 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.6 MiB/s wr, 423 op/s 2024-09-19T09:27:42.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:42 smithi153 ceph-mon[35071]: pgmap v1090: 97 pgs: 97 active+clean; 466 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.6 MiB/s wr, 423 op/s 2024-09-19T09:27:42.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:42 smithi120 ceph-mon[31150]: pgmap v1090: 97 pgs: 97 active+clean; 466 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.6 MiB/s wr, 423 op/s 2024-09-19T09:27:43.397 INFO:tasks.workunit.client.0.smithi120.stdout: 10 268981 19.91 MB/sec execute 578 sec latency 10183.025 ms 2024-09-19T09:27:44.396 INFO:tasks.workunit.client.0.smithi120.stdout: 10 268981 19.87 MB/sec execute 579 sec latency 11183.112 ms 2024-09-19T09:27:44.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:44 smithi196 ceph-mon[35150]: pgmap v1091: 97 pgs: 97 active+clean; 464 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.9 MiB/s wr, 415 op/s 2024-09-19T09:27:44.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:44 smithi153 ceph-mon[35071]: pgmap v1091: 97 pgs: 97 active+clean; 464 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.9 MiB/s wr, 415 op/s 2024-09-19T09:27:44.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:44 smithi120 ceph-mon[31150]: pgmap v1091: 97 pgs: 97 active+clean; 464 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.9 MiB/s wr, 415 op/s 2024-09-19T09:27:45.396 INFO:tasks.workunit.client.0.smithi120.stdout: 10 268981 19.84 MB/sec execute 580 sec latency 12183.215 ms 2024-09-19T09:27:46.396 INFO:tasks.workunit.client.0.smithi120.stdout: 10 268981 19.80 MB/sec execute 581 sec latency 13183.336 ms 2024-09-19T09:27:46.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:46 smithi196 ceph-mon[35150]: pgmap v1092: 97 pgs: 97 active+clean; 461 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 297 op/s 2024-09-19T09:27:46.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:46 smithi153 ceph-mon[35071]: pgmap v1092: 97 pgs: 97 active+clean; 461 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 297 op/s 2024-09-19T09:27:46.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:46 smithi120 ceph-mon[31150]: pgmap v1092: 97 pgs: 97 active+clean; 461 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 297 op/s 2024-09-19T09:27:47.396 INFO:tasks.workunit.client.0.smithi120.stdout: 10 268981 19.77 MB/sec execute 582 sec latency 14183.437 ms 2024-09-19T09:27:48.396 INFO:tasks.workunit.client.0.smithi120.stdout: 10 268981 19.73 MB/sec execute 583 sec latency 15183.494 ms 2024-09-19T09:27:48.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:48 smithi196 ceph-mon[35150]: pgmap v1093: 97 pgs: 97 active+clean; 457 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 4.1 MiB/s wr, 296 op/s 2024-09-19T09:27:48.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:48 smithi153 ceph-mon[35071]: pgmap v1093: 97 pgs: 97 active+clean; 457 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 4.1 MiB/s wr, 296 op/s 2024-09-19T09:27:48.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:48 smithi120 ceph-mon[31150]: pgmap v1093: 97 pgs: 97 active+clean; 457 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 4.1 MiB/s wr, 296 op/s 2024-09-19T09:27:48.879 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:27:49.133 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:49.396 INFO:tasks.workunit.client.0.smithi120.stdout: 10 268981 19.70 MB/sec execute 584 sec latency 16183.539 ms 2024-09-19T09:27:49.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.660+0000 7fde934d5640 1 -- 172.21.15.120:0/3720056553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde8c103740 msgr2=0x7fde8c107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:49.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.660+0000 7fde934d5640 1 --2- 172.21.15.120:0/3720056553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde8c103740 0x7fde8c107b80 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7fde7c009930 tx=0x7fde7c02a420 comp rx=0 tx=0).stop 2024-09-19T09:27:49.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.660+0000 7fde934d5640 1 -- 172.21.15.120:0/3720056553 shutdown_connections 2024-09-19T09:27:49.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.660+0000 7fde934d5640 1 --2- 172.21.15.120:0/3720056553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde8c108290 0x7fde8c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:49.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.660+0000 7fde934d5640 1 --2- 172.21.15.120:0/3720056553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde8c103740 0x7fde8c107b80 unknown :-1 s=CLOSED pgs=2508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:49.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.660+0000 7fde934d5640 1 --2- 172.21.15.120:0/3720056553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde8c102d90 0x7fde8c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:49.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.660+0000 7fde934d5640 1 -- 172.21.15.120:0/3720056553 >> 172.21.15.120:0/3720056553 conn(0x7fde8c0fc7d0 msgr2=0x7fde8c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:49.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.660+0000 7fde934d5640 1 -- 172.21.15.120:0/3720056553 shutdown_connections 2024-09-19T09:27:49.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.660+0000 7fde934d5640 1 -- 172.21.15.120:0/3720056553 wait complete. 2024-09-19T09:27:49.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.660+0000 7fde934d5640 1 Processor -- start 2024-09-19T09:27:49.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.661+0000 7fde934d5640 1 -- start start 2024-09-19T09:27:49.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.661+0000 7fde934d5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde8c102d90 0x7fde8c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:49.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.661+0000 7fde934d5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde8c103740 0x7fde8c198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:49.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.661+0000 7fde934d5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde8c108290 0x7fde8c19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:49.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.661+0000 7fde934d5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fde8c1120b0 con 0x7fde8c108290 2024-09-19T09:27:49.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.661+0000 7fde934d5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fde8c111f30 con 0x7fde8c102d90 2024-09-19T09:27:49.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.661+0000 7fde934d5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fde8c112230 con 0x7fde8c103740 2024-09-19T09:27:49.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.661+0000 7fde91a72640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde8c108290 0x7fde8c19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:49.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.661+0000 7fde91a72640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde8c108290 0x7fde8c19ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40500/0 (socket says 172.21.15.120:40500) 2024-09-19T09:27:49.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.661+0000 7fde91a72640 1 -- 172.21.15.120:0/2627713704 learned_addr learned my addr 172.21.15.120:0/2627713704 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:49.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.661+0000 7fde91271640 1 --2- 172.21.15.120:0/2627713704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde8c102d90 0x7fde8c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:49.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.661+0000 7fde90a70640 1 --2- 172.21.15.120:0/2627713704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde8c103740 0x7fde8c198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:49.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.661+0000 7fde91a72640 1 -- 172.21.15.120:0/2627713704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde8c103740 msgr2=0x7fde8c198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:49.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.661+0000 7fde91a72640 1 --2- 172.21.15.120:0/2627713704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde8c103740 0x7fde8c198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:49.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.661+0000 7fde91a72640 1 -- 172.21.15.120:0/2627713704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde8c102d90 msgr2=0x7fde8c1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:49.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.661+0000 7fde91a72640 1 --2- 172.21.15.120:0/2627713704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde8c102d90 0x7fde8c1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:49.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.661+0000 7fde91a72640 1 -- 172.21.15.120:0/2627713704 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde8c19d570 con 0x7fde8c108290 2024-09-19T09:27:49.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.661+0000 7fde91271640 1 --2- 172.21.15.120:0/2627713704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde8c102d90 0x7fde8c1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:27:49.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.662+0000 7fde91a72640 1 --2- 172.21.15.120:0/2627713704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde8c108290 0x7fde8c19ce90 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7fde8400cce0 tx=0x7fde8400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:49.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.662+0000 7fde827fc640 1 -- 172.21.15.120:0/2627713704 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8400f450 con 0x7fde8c108290 2024-09-19T09:27:49.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.662+0000 7fde827fc640 1 -- 172.21.15.120:0/2627713704 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde8400fad0 con 0x7fde8c108290 2024-09-19T09:27:49.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.662+0000 7fde934d5640 1 -- 172.21.15.120:0/2627713704 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde8c19d860 con 0x7fde8c108290 2024-09-19T09:27:49.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.662+0000 7fde827fc640 1 -- 172.21.15.120:0/2627713704 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde84018b60 con 0x7fde8c108290 2024-09-19T09:27:49.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.662+0000 7fde934d5640 1 -- 172.21.15.120:0/2627713704 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde8c1a5140 con 0x7fde8c108290 2024-09-19T09:27:49.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.663+0000 7fde934d5640 1 -- 172.21.15.120:0/2627713704 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde8c06a7f0 con 0x7fde8c108290 2024-09-19T09:27:49.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.664+0000 7fde827fc640 1 -- 172.21.15.120:0/2627713704 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fde8400fc80 con 0x7fde8c108290 2024-09-19T09:27:49.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.665+0000 7fde827fc640 1 --2- 172.21.15.120:0/2627713704 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fde780a32c0 0x7fde780a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:49.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.665+0000 7fde91271640 1 --2- 172.21.15.120:0/2627713704 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fde780a32c0 0x7fde780a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:49.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.665+0000 7fde827fc640 1 -- 172.21.15.120:0/2627713704 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fde840cff60 con 0x7fde8c108290 2024-09-19T09:27:49.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.666+0000 7fde91271640 1 --2- 172.21.15.120:0/2627713704 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fde780a32c0 0x7fde780a5780 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fde88009d60 tx=0x7fde88009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:49.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.669+0000 7fde827fc640 1 -- 172.21.15.120:0/2627713704 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fde84014030 con 0x7fde8c108290 2024-09-19T09:27:49.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.877+0000 7fde934d5640 1 -- 172.21.15.120:0/2627713704 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde8c103170 con 0x7fde8c108290 2024-09-19T09:27:49.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.878+0000 7fde827fc640 1 -- 172.21.15.120:0/2627713704 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11988 (secure 0 0 0) 0x7fde8c103170 con 0x7fde8c108290 2024-09-19T09:27:49.880 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:27:49.880 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":39,"btime":"2024-09-19T09:18:53:120858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:18:53.070639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:27:49.880 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 39 2024-09-19T09:27:49.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.880+0000 7fde934d5640 1 -- 172.21.15.120:0/2627713704 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fde780a32c0 msgr2=0x7fde780a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:49.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.880+0000 7fde934d5640 1 --2- 172.21.15.120:0/2627713704 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fde780a32c0 0x7fde780a5780 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fde88009d60 tx=0x7fde88009110 comp rx=0 tx=0).stop 2024-09-19T09:27:49.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.880+0000 7fde934d5640 1 -- 172.21.15.120:0/2627713704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde8c108290 msgr2=0x7fde8c19ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:49.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.880+0000 7fde934d5640 1 --2- 172.21.15.120:0/2627713704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde8c108290 0x7fde8c19ce90 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7fde8400cce0 tx=0x7fde8400a620 comp rx=0 tx=0).stop 2024-09-19T09:27:49.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.880+0000 7fde934d5640 1 -- 172.21.15.120:0/2627713704 shutdown_connections 2024-09-19T09:27:49.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.880+0000 7fde934d5640 1 --2- 172.21.15.120:0/2627713704 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fde780a32c0 0x7fde780a5780 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:49.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.880+0000 7fde934d5640 1 --2- 172.21.15.120:0/2627713704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde8c108290 0x7fde8c19ce90 unknown :-1 s=CLOSED pgs=2509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:49.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.880+0000 7fde934d5640 1 --2- 172.21.15.120:0/2627713704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde8c103740 0x7fde8c198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:49.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.880+0000 7fde934d5640 1 --2- 172.21.15.120:0/2627713704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde8c102d90 0x7fde8c1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:49.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.880+0000 7fde934d5640 1 -- 172.21.15.120:0/2627713704 >> 172.21.15.120:0/2627713704 conn(0x7fde8c0fc7d0 msgr2=0x7fde8c106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:49.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.880+0000 7fde934d5640 1 -- 172.21.15.120:0/2627713704 shutdown_connections 2024-09-19T09:27:49.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:49.880+0000 7fde934d5640 1 -- 172.21.15.120:0/2627713704 wait complete. 2024-09-19T09:27:50.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:49 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2627713704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:50.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:49 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2627713704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:50.250 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:49 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2627713704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:50.396 INFO:tasks.workunit.client.0.smithi120.stdout: 10 268981 19.67 MB/sec execute 585 sec latency 17183.768 ms 2024-09-19T09:27:50.700 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db --set-id 63770883 --release --await' 2024-09-19T09:27:50.897 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db --set-id 63770883 --release --await 2024-09-19T09:27:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.727+0000 7f0a3b3b7640 1 -- 172.21.15.120:0/2471291129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a34101c30 msgr2=0x7f0a341020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.727+0000 7f0a3b3b7640 1 --2- 172.21.15.120:0/2471291129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a34101c30 0x7f0a341020b0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7f0a24009900 tx=0x7f0a2402a410 comp rx=0 tx=0).stop 2024-09-19T09:27:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.727+0000 7f0a3b3b7640 1 -- 172.21.15.120:0/2471291129 shutdown_connections 2024-09-19T09:27:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.727+0000 7f0a3b3b7640 1 --2- 172.21.15.120:0/2471291129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a341025f0 0x7f0a34108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.727+0000 7f0a3b3b7640 1 --2- 172.21.15.120:0/2471291129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a34101c30 0x7f0a341020b0 unknown :-1 s=CLOSED pgs=2510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.727+0000 7f0a3b3b7640 1 --2- 172.21.15.120:0/2471291129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a34100a40 0x7f0a34100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.727+0000 7f0a3b3b7640 1 -- 172.21.15.120:0/2471291129 >> 172.21.15.120:0/2471291129 conn(0x7f0a340fc1e0 msgr2=0x7f0a340fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.727+0000 7f0a3b3b7640 1 -- 172.21.15.120:0/2471291129 shutdown_connections 2024-09-19T09:27:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.727+0000 7f0a3b3b7640 1 -- 172.21.15.120:0/2471291129 wait complete. 2024-09-19T09:27:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.728+0000 7f0a3b3b7640 1 Processor -- start 2024-09-19T09:27:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.728+0000 7f0a3b3b7640 1 -- start start 2024-09-19T09:27:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.728+0000 7f0a3b3b7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a34100a40 0x7f0a3406a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.728+0000 7f0a3b3b7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a34101c30 0x7f0a3419cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.728+0000 7f0a3b3b7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a341025f0 0x7f0a3419f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.728+0000 7f0a3b3b7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3410c740 con 0x7f0a341025f0 2024-09-19T09:27:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.728+0000 7f0a3b3b7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3410c5c0 con 0x7f0a34101c30 2024-09-19T09:27:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.728+0000 7f0a3b3b7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3410c8c0 con 0x7f0a34100a40 2024-09-19T09:27:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.728+0000 7f0a39954640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a341025f0 0x7f0a3419f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.728+0000 7f0a39954640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a341025f0 0x7f0a3419f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40516/0 (socket says 172.21.15.120:40516) 2024-09-19T09:27:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.728+0000 7f0a39954640 1 -- 172.21.15.120:0/927052790 learned_addr learned my addr 172.21.15.120:0/927052790 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.728+0000 7f0a39153640 1 --2- 172.21.15.120:0/927052790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a34100a40 0x7f0a3406a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.729+0000 7f0a39954640 1 -- 172.21.15.120:0/927052790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a34100a40 msgr2=0x7f0a3406a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.729+0000 7f0a39954640 1 --2- 172.21.15.120:0/927052790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a34100a40 0x7f0a3406a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.729+0000 7f0a38952640 1 --2- 172.21.15.120:0/927052790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a34101c30 0x7f0a3419cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.729+0000 7f0a39954640 1 -- 172.21.15.120:0/927052790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a34101c30 msgr2=0x7f0a3419cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.729+0000 7f0a39954640 1 --2- 172.21.15.120:0/927052790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a34101c30 0x7f0a3419cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.729+0000 7f0a39954640 1 -- 172.21.15.120:0/927052790 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a3419fa50 con 0x7f0a341025f0 2024-09-19T09:27:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.729+0000 7f0a38952640 1 --2- 172.21.15.120:0/927052790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a34101c30 0x7f0a3419cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.729+0000 7f0a39954640 1 --2- 172.21.15.120:0/927052790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a341025f0 0x7f0a3419f140 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00cce0 tx=0x7f0a2c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.729+0000 7f0a2a7fc640 1 -- 172.21.15.120:0/927052790 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c00f450 con 0x7f0a341025f0 2024-09-19T09:27:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.729+0000 7f0a2a7fc640 1 -- 172.21.15.120:0/927052790 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a2c00fad0 con 0x7f0a341025f0 2024-09-19T09:27:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.729+0000 7f0a3b3b7640 1 -- 172.21.15.120:0/927052790 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a3419fce0 con 0x7f0a341025f0 2024-09-19T09:27:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.729+0000 7f0a2a7fc640 1 -- 172.21.15.120:0/927052790 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c018c10 con 0x7f0a341025f0 2024-09-19T09:27:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.729+0000 7f0a3b3b7640 1 -- 172.21.15.120:0/927052790 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a341a0200 con 0x7f0a341025f0 2024-09-19T09:27:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.731+0000 7f0a2a7fc640 1 -- 172.21.15.120:0/927052790 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f0a2c00f5f0 con 0x7f0a341025f0 2024-09-19T09:27:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.732+0000 7f0a3e22b740 1 Processor -- start 2024-09-19T09:27:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.732+0000 7f0a3e22b740 1 -- start start 2024-09-19T09:27:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.732+0000 7f0a3e22b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55864b2ae9a0 0x55864b2aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.732+0000 7f0a3e22b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864b2af2e0 0x55864b2b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.732+0000 7f0a3e22b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55864b2b4370 0x55864b2b6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.732+0000 7f0a3e22b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55864b2122f0 con 0x55864b2b4370 2024-09-19T09:27:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.732+0000 7f0a3e22b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55864b1ba270 con 0x55864b2ae9a0 2024-09-19T09:27:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.732+0000 7f0a3e22b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55864b20e2b0 con 0x55864b2af2e0 2024-09-19T09:27:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.732+0000 7f0a2a7fc640 1 --2- 172.21.15.120:0/927052790 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0a200a32f0 0x7f0a200a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.732+0000 7f0a39954640 1 --2- 172.21.15.120:0/927052790 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0a200a32f0 0x7f0a200a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.732+0000 7f0a38952640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55864b2b4370 0x55864b2b6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.732+0000 7f0a38952640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55864b2b4370 0x55864b2b6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40526/0 (socket says 172.21.15.120:40526) 2024-09-19T09:27:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.732+0000 7f0a2a7fc640 1 -- 172.21.15.120:0/927052790 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f0a2c0cffe0 con 0x7f0a341025f0 2024-09-19T09:27:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.732+0000 7f0a38952640 1 -- 172.21.15.120:0/166522191 learned_addr learned my addr 172.21.15.120:0/166522191 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.732+0000 7f0a39153640 1 --2- 172.21.15.120:0/166522191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864b2af2e0 0x55864b2b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.732+0000 7f0a38952640 1 --2- 172.21.15.120:0/166522191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55864b2ae9a0 0x55864b2aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.733+0000 7f0a38952640 1 -- 172.21.15.120:0/166522191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864b2af2e0 msgr2=0x55864b2b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.733+0000 7f0a38952640 1 --2- 172.21.15.120:0/166522191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864b2af2e0 0x55864b2b3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.733+0000 7f0a38952640 1 -- 172.21.15.120:0/166522191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55864b2ae9a0 msgr2=0x55864b2aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.733+0000 7f0a39954640 1 --2- 172.21.15.120:0/927052790 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0a200a32f0 0x7f0a200a57b0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c012040 tx=0x7f0a2c067030 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.733+0000 7f0a38952640 1 --2- 172.21.15.120:0/166522191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55864b2ae9a0 0x55864b2aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.733+0000 7f0a38952640 1 -- 172.21.15.120:0/166522191 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55864b1b9340 con 0x55864b2b4370 2024-09-19T09:27:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.733+0000 7f0a38952640 1 --2- 172.21.15.120:0/166522191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55864b2b4370 0x55864b2b6830 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7f0a240096c0 tx=0x7f0a24031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d116bd38936a02cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.733+0000 7f0a1f7fe640 1 -- 172.21.15.120:0/166522191 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a24045070 con 0x55864b2b4370 2024-09-19T09:27:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.733+0000 7f0a1f7fe640 1 -- 172.21.15.120:0/166522191 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a24004230 con 0x55864b2b4370 2024-09-19T09:27:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.733+0000 7f0a1f7fe640 1 -- 172.21.15.120:0/166522191 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a24035040 con 0x55864b2b4370 2024-09-19T09:27:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.734+0000 7f0a3e22b740 1 -- 172.21.15.120:0/166522191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55864b2b4370 msgr2=0x55864b2b6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.734+0000 7f0a3e22b740 1 --2- 172.21.15.120:0/166522191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55864b2b4370 0x55864b2b6830 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7f0a240096c0 tx=0x7f0a24031400 comp rx=0 tx=0).stop 2024-09-19T09:27:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.734+0000 7f0a3e22b740 1 -- 172.21.15.120:0/166522191 shutdown_connections 2024-09-19T09:27:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.734+0000 7f0a3e22b740 1 --2- 172.21.15.120:0/166522191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55864b2b4370 0x55864b2b6830 unknown :-1 s=CLOSED pgs=2512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.734+0000 7f0a3e22b740 1 --2- 172.21.15.120:0/166522191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864b2af2e0 0x55864b2b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.734+0000 7f0a3e22b740 1 --2- 172.21.15.120:0/166522191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55864b2ae9a0 0x55864b2aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.734+0000 7f0a3e22b740 1 -- 172.21.15.120:0/166522191 >> 172.21.15.120:0/166522191 conn(0x55864b2aa010 msgr2=0x55864b2aa410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.734+0000 7f0a3e22b740 1 -- 172.21.15.120:0/166522191 shutdown_connections 2024-09-19T09:27:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.734+0000 7f0a3e22b740 1 -- 172.21.15.120:0/166522191 wait complete. 2024-09-19T09:27:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.734+0000 7f0a3e22b740 1 -- start start 2024-09-19T09:27:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.734+0000 7f0a3e22b740 1 Processor -- start 2024-09-19T09:27:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.734+0000 7f0a3e22b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864b2ae9a0 0x55864b2b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.735+0000 7f0a3e22b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55864b2af2e0 0x55864b2b9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.735+0000 7f0a3e22b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55864b2b4370 0x55864b2c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.735+0000 7f0a3e22b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55864b2b72e0 con 0x55864b2b4370 2024-09-19T09:27:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.735+0000 7f0a3e22b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55864b19b290 con 0x55864b2af2e0 2024-09-19T09:27:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.735+0000 7f0a3e22b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55864b2b7460 con 0x55864b2ae9a0 2024-09-19T09:27:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.735+0000 7f0a3e22b740 10 client.0 _ll_register_callbacks cb 0x55864b1cc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.735+0000 7f0a38952640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55864b2b4370 0x55864b2c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.735+0000 7f0a38952640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55864b2b4370 0x55864b2c01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40538/0 (socket says 172.21.15.120:40538) 2024-09-19T09:27:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.735+0000 7f0a38952640 1 -- 172.21.15.120:0/6561430 learned_addr learned my addr 172.21.15.120:0/6561430 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.735+0000 7f0a38952640 1 --2- 172.21.15.120:0/6561430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864b2ae9a0 0x55864b2b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.735+0000 7f0a39153640 1 --2- 172.21.15.120:0/6561430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55864b2af2e0 0x55864b2b9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.735+0000 7f0a38952640 1 -- 172.21.15.120:0/6561430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864b2ae9a0 msgr2=0x55864b2b8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.735+0000 7f0a38952640 1 --2- 172.21.15.120:0/6561430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864b2ae9a0 0x55864b2b8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.735+0000 7f0a38952640 1 -- 172.21.15.120:0/6561430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55864b2af2e0 msgr2=0x55864b2b9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.735+0000 7f0a38952640 1 --2- 172.21.15.120:0/6561430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55864b2af2e0 0x55864b2b9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.735+0000 7f0a38952640 1 -- 172.21.15.120:0/6561430 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55864af25210 con 0x55864b2b4370 2024-09-19T09:27:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.735+0000 7f0a39153640 1 --2- 172.21.15.120:0/6561430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55864b2af2e0 0x55864b2b9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.735+0000 7f0a38952640 1 --2- 172.21.15.120:0/6561430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55864b2b4370 0x55864b2c01b0 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x55864b2b8160 tx=0x7f0a24005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e1ad71c2fd686be server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.735+0000 7f0a1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:27:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.735+0000 7f0a1cff9640 1 -- 172.21.15.120:0/6561430 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a24045070 con 0x55864b2b4370 2024-09-19T09:27:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.735+0000 7f0a1cff9640 1 -- 172.21.15.120:0/6561430 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f0a24049610 con 0x55864b2b4370 2024-09-19T09:27:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.735+0000 7f0a3e22b740 1 -- 172.21.15.120:0/6561430 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a24004230 con 0x55864b2b4370 2024-09-19T09:27:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.736+0000 7f0a1cff9640 1 -- 172.21.15.120:0/6561430 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0a24035040 con 0x55864b2b4370 2024-09-19T09:27:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.736+0000 7f0a3e22b740 10 client.23505 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.736+0000 7f0a3e22b740 1 -- 172.21.15.120:0/6561430 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55864aff1620 con 0x55864b2b4370 2024-09-19T09:27:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.736+0000 7f0a1cff9640 1 -- 172.21.15.120:0/6561430 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a24004230 con 0x55864b2b4370 2024-09-19T09:27:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.736+0000 7f0a1cff9640 1 -- 172.21.15.120:0/6561430 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f0a2403a040 con 0x55864b2b4370 2024-09-19T09:27:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.736+0000 7f0a3e22b740 10 client.23505 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.736+0000 7f0a3e22b740 10 client.23505 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:27:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.736+0000 7f0a3e22b740 20 client.23505 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.736+0000 7f0a3e22b740 1 --2- 172.21.15.120:0/6561430 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55864b2c44b0 0x55864b2c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.736+0000 7f0a3e22b740 4 client.23505 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.736+0000 7f0a3e22b740 1 -- 172.21.15.120:0/6561430 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55864b21fa20 con 0x55864b2c44b0 2024-09-19T09:27:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.736+0000 7f0a39153640 1 --2- 172.21.15.120:0/6561430 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55864b2c44b0 0x55864b2c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.737+0000 7f0a1cff9640 10 client.23505 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.737+0000 7f0a39153640 1 --2- 172.21.15.120:0/6561430 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55864b2c44b0 0x55864b2c68a0 crc :-1 s=READY pgs=609 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c23e51f389f6a29f server_cookie=a2d514991b24f371 in_seq=0 out_seq=0 2024-09-19T09:27:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.737+0000 7f0a1cff9640 1 -- 172.21.15.120:0/6561430 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55864b21fa20 con 0x55864b2c44b0 2024-09-19T09:27:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.737+0000 7f0a1cff9640 10 client.23505 handle_command_reply: tid=0 2024-09-19T09:27:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.738+0000 7f0a3e22b740 1 client.23505 shutdown 2024-09-19T09:27:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.738+0000 7f0a3e22b740 1 -- 172.21.15.120:0/6561430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55864b2b4370 msgr2=0x55864b2c01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.738+0000 7f0a3e22b740 1 --2- 172.21.15.120:0/6561430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55864b2b4370 0x55864b2c01b0 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x55864b2b8160 tx=0x7f0a24005e10 comp rx=0 tx=0).stop 2024-09-19T09:27:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.738+0000 7f0a3e22b740 1 -- 172.21.15.120:0/6561430 shutdown_connections 2024-09-19T09:27:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.738+0000 7f0a3e22b740 1 --2- 172.21.15.120:0/6561430 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55864b2c44b0 0x55864b2c68a0 crc :-1 s=READY pgs=609 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.738+0000 7f0a3e22b740 1 --2- 172.21.15.120:0/6561430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55864b2b4370 0x55864b2c01b0 unknown :-1 s=CLOSED pgs=2513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.738+0000 7f0a3e22b740 1 --2- 172.21.15.120:0/6561430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55864b2af2e0 0x55864b2b9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.738+0000 7f0a3e22b740 1 --2- 172.21.15.120:0/6561430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864b2ae9a0 0x55864b2b8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.738+0000 7f0a1cff9640 0 client.23505 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.738+0000 7f0a3e22b740 1 -- 172.21.15.120:0/6561430 >> 172.21.15.120:0/6561430 conn(0x55864b2aa010 msgr2=0x55864b2ab820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.738+0000 7f0a3e22b740 1 -- 172.21.15.120:0/6561430 shutdown_connections 2024-09-19T09:27:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.738+0000 7f0a3e22b740 1 -- 172.21.15.120:0/6561430 wait complete. 2024-09-19T09:27:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.738+0000 7f0a3e22b740 20 client.23505 trim_cache size 0 max 16384 2024-09-19T09:27:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.752+0000 7f0a3e22b740 1 Processor -- start 2024-09-19T09:27:50.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.752+0000 7f0a3e22b740 1 -- start start 2024-09-19T09:27:50.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.752+0000 7f0a3e22b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864b2ae9a0 0x55864b2aed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:50.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.752+0000 7f0a3e22b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55864b2af2e0 0x55864b2c0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:50.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.752+0000 7f0a3e22b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55864b2b4370 0x55864b2a90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:50.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.752+0000 7f0a3e22b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0a341a8f90 con 0x55864b2af2e0 2024-09-19T09:27:50.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.752+0000 7f0a3e22b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0a341aaad0 con 0x55864b2b4370 2024-09-19T09:27:50.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.752+0000 7f0a3e22b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0a341a9660 con 0x55864b2ae9a0 2024-09-19T09:27:50.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.752+0000 7f0a39153640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55864b2af2e0 0x55864b2c0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:50.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.752+0000 7f0a39153640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55864b2af2e0 0x55864b2c0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40546/0 (socket says 172.21.15.120:40546) 2024-09-19T09:27:50.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.752+0000 7f0a39153640 1 -- 172.21.15.120:0/3545889667 learned_addr learned my addr 172.21.15.120:0/3545889667 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.752+0000 7f0a39153640 1 -- 172.21.15.120:0/3545889667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864b2ae9a0 msgr2=0x55864b2aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.752+0000 7f0a39153640 1 --2- 172.21.15.120:0/3545889667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864b2ae9a0 0x55864b2aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.752+0000 7f0a39153640 1 -- 172.21.15.120:0/3545889667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55864b2b4370 msgr2=0x55864b2a90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.752+0000 7f0a39153640 1 --2- 172.21.15.120:0/3545889667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55864b2b4370 0x55864b2a90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.752+0000 7f0a39153640 1 -- 172.21.15.120:0/3545889667 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55864b2c6e80 con 0x55864b2af2e0 2024-09-19T09:27:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.753+0000 7f0a39153640 1 --2- 172.21.15.120:0/3545889667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55864b2af2e0 0x55864b2c0140 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f0a30000c00 tx=0x7f0a30008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d8afff7cb2f771 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.753+0000 7f0a1effd640 1 -- 172.21.15.120:0/3545889667 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a30020070 con 0x55864b2af2e0 2024-09-19T09:27:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.753+0000 7f0a1effd640 1 -- 172.21.15.120:0/3545889667 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a3001c730 con 0x55864b2af2e0 2024-09-19T09:27:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.753+0000 7f0a1effd640 1 -- 172.21.15.120:0/3545889667 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a300146f0 con 0x55864b2af2e0 2024-09-19T09:27:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.753+0000 7f0a3e22b740 1 -- 172.21.15.120:0/3545889667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55864b2af2e0 msgr2=0x55864b2c0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.753+0000 7f0a3e22b740 1 --2- 172.21.15.120:0/3545889667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55864b2af2e0 0x55864b2c0140 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f0a30000c00 tx=0x7f0a30008040 comp rx=0 tx=0).stop 2024-09-19T09:27:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.753+0000 7f0a3e22b740 1 -- 172.21.15.120:0/3545889667 shutdown_connections 2024-09-19T09:27:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.753+0000 7f0a3e22b740 1 --2- 172.21.15.120:0/3545889667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55864b2b4370 0x55864b2a90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.753+0000 7f0a3e22b740 1 --2- 172.21.15.120:0/3545889667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55864b2af2e0 0x55864b2c0140 unknown :-1 s=CLOSED pgs=2514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.753+0000 7f0a3e22b740 1 --2- 172.21.15.120:0/3545889667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864b2ae9a0 0x55864b2aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.753+0000 7f0a3e22b740 1 -- 172.21.15.120:0/3545889667 >> 172.21.15.120:0/3545889667 conn(0x55864b2aa010 msgr2=0x55864b2a89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.754+0000 7f0a3e22b740 1 -- 172.21.15.120:0/3545889667 shutdown_connections 2024-09-19T09:27:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.754+0000 7f0a3e22b740 1 -- 172.21.15.120:0/3545889667 wait complete. 2024-09-19T09:27:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.754+0000 7f0a3e22b740 1 -- start start 2024-09-19T09:27:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.754+0000 7f0a3e22b740 1 Processor -- start 2024-09-19T09:27:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.754+0000 7f0a3e22b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55864b2ae9a0 0x55864b2bdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.754+0000 7f0a3e22b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55864b2af2e0 0x55864b2be230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.754+0000 7f0a3e22b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864b2b4370 0x55864b2bf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.754+0000 7f0a3e22b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0a341aa400 con 0x55864b2af2e0 2024-09-19T09:27:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.754+0000 7f0a3e22b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0a341a9d30 con 0x55864b2ae9a0 2024-09-19T09:27:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.754+0000 7f0a3e22b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0a341a6f50 con 0x55864b2b4370 2024-09-19T09:27:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.754+0000 7f0a3e22b740 10 client.0 _ll_register_callbacks cb 0x7f0a24032e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.754+0000 7f0a39153640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55864b2af2e0 0x55864b2be230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.754+0000 7f0a39153640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55864b2af2e0 0x55864b2be230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40552/0 (socket says 172.21.15.120:40552) 2024-09-19T09:27:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.754+0000 7f0a39153640 1 -- 172.21.15.120:0/599476061 learned_addr learned my addr 172.21.15.120:0/599476061 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.754+0000 7f0a38952640 1 --2- 172.21.15.120:0/599476061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864b2b4370 0x55864b2bf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.755+0000 7f0a38952640 1 --2- 172.21.15.120:0/599476061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55864b2ae9a0 0x55864b2bdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.755+0000 7f0a39153640 1 -- 172.21.15.120:0/599476061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864b2b4370 msgr2=0x55864b2bf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.755+0000 7f0a39153640 1 --2- 172.21.15.120:0/599476061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864b2b4370 0x55864b2bf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.755+0000 7f0a39153640 1 -- 172.21.15.120:0/599476061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55864b2ae9a0 msgr2=0x55864b2bdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.755+0000 7f0a39153640 1 --2- 172.21.15.120:0/599476061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55864b2ae9a0 0x55864b2bdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.755+0000 7f0a39153640 1 -- 172.21.15.120:0/599476061 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55864b2b7390 con 0x55864b2af2e0 2024-09-19T09:27:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.755+0000 7f0a38952640 1 --2- 172.21.15.120:0/599476061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55864b2ae9a0 0x55864b2bdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.755+0000 7f0a39153640 1 --2- 172.21.15.120:0/599476061 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55864b2af2e0 0x55864b2be230 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000aa60 tx=0x7f0a3001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b117ad2251bd5b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.755+0000 7f0a1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:27:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.755+0000 7f0a1cff9640 1 -- 172.21.15.120:0/599476061 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a30020050 con 0x55864b2af2e0 2024-09-19T09:27:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.755+0000 7f0a1cff9640 1 -- 172.21.15.120:0/599476061 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f0a3001ccf0 con 0x55864b2af2e0 2024-09-19T09:27:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.755+0000 7f0a3e22b740 1 -- 172.21.15.120:0/599476061 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a3001c730 con 0x55864b2af2e0 2024-09-19T09:27:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.755+0000 7f0a1cff9640 1 -- 172.21.15.120:0/599476061 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0a300059c0 con 0x55864b2af2e0 2024-09-19T09:27:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.755+0000 7f0a3e22b740 10 client.23511 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.755+0000 7f0a3e22b740 1 -- 172.21.15.120:0/599476061 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55864b32f3c0 con 0x55864b2af2e0 2024-09-19T09:27:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.755+0000 7f0a1cff9640 1 -- 172.21.15.120:0/599476061 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a3001c730 con 0x55864b2af2e0 2024-09-19T09:27:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.755+0000 7f0a1cff9640 1 -- 172.21.15.120:0/599476061 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f0a30025450 con 0x55864b2af2e0 2024-09-19T09:27:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.755+0000 7f0a3e22b740 10 client.23511 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.755+0000 7f0a3e22b740 10 client.23511 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:27:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.755+0000 7f0a3e22b740 20 client.23511 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.756+0000 7f0a3e22b740 1 --2- 172.21.15.120:0/599476061 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55864b32fd40 0x55864b332160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.756+0000 7f0a3e22b740 4 client.23511 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:27:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.756+0000 7f0a3e22b740 1 -- 172.21.15.120:0/599476061 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55864b219d70 con 0x55864b32fd40 2024-09-19T09:27:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.756+0000 7f0a38952640 1 --2- 172.21.15.120:0/599476061 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55864b32fd40 0x55864b332160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.756+0000 7f0a38952640 1 --2- 172.21.15.120:0/599476061 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55864b32fd40 0x55864b332160 crc :-1 s=READY pgs=610 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0be2224a271f2a8 server_cookie=c7bc6d0b2847838c in_seq=0 out_seq=0 2024-09-19T09:27:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.756+0000 7f0a1cff9640 10 client.23511 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.863+0000 7f0a1cff9640 1 -- 172.21.15.120:0/599476061 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55864b219d70 con 0x55864b32fd40 2024-09-19T09:27:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.863+0000 7f0a1cff9640 10 client.23511 handle_command_reply: tid=0 2024-09-19T09:27:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.863+0000 7f0a3e22b740 1 client.23511 shutdown 2024-09-19T09:27:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.864+0000 7f0a3e22b740 1 -- 172.21.15.120:0/599476061 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55864b2af2e0 msgr2=0x55864b2be230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.864+0000 7f0a3e22b740 1 --2- 172.21.15.120:0/599476061 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55864b2af2e0 0x55864b2be230 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000aa60 tx=0x7f0a3001c500 comp rx=0 tx=0).stop 2024-09-19T09:27:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.864+0000 7f0a3e22b740 1 -- 172.21.15.120:0/599476061 shutdown_connections 2024-09-19T09:27:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.864+0000 7f0a3e22b740 1 --2- 172.21.15.120:0/599476061 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55864b32fd40 0x55864b332160 crc :-1 s=READY pgs=610 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.864+0000 7f0a3e22b740 1 --2- 172.21.15.120:0/599476061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55864b2b4370 0x55864b2bf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.864+0000 7f0a1cff9640 0 client.23511 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.864+0000 7f0a3e22b740 1 --2- 172.21.15.120:0/599476061 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55864b2af2e0 0x55864b2be230 unknown :-1 s=CLOSED pgs=2515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.864+0000 7f0a3e22b740 1 --2- 172.21.15.120:0/599476061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55864b2ae9a0 0x55864b2bdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.864+0000 7f0a3e22b740 1 -- 172.21.15.120:0/599476061 >> 172.21.15.120:0/599476061 conn(0x55864b2aa010 msgr2=0x55864b2a91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.864+0000 7f0a3e22b740 1 -- 172.21.15.120:0/599476061 shutdown_connections 2024-09-19T09:27:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.864+0000 7f0a3e22b740 1 -- 172.21.15.120:0/599476061 wait complete. 2024-09-19T09:27:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.864+0000 7f0a3e22b740 20 client.23511 trim_cache size 0 max 16384 2024-09-19T09:27:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:27:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:27:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:27:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:27:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.864+0000 7f0a3b3b7640 1 -- 172.21.15.120:0/927052790 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0a200a32f0 msgr2=0x7f0a200a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.864+0000 7f0a3b3b7640 1 --2- 172.21.15.120:0/927052790 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0a200a32f0 0x7f0a200a57b0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c012040 tx=0x7f0a2c067030 comp rx=0 tx=0).stop 2024-09-19T09:27:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.864+0000 7f0a3b3b7640 1 -- 172.21.15.120:0/927052790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a341025f0 msgr2=0x7f0a3419f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.864+0000 7f0a3b3b7640 1 --2- 172.21.15.120:0/927052790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a341025f0 0x7f0a3419f140 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00cce0 tx=0x7f0a2c00a620 comp rx=0 tx=0).stop 2024-09-19T09:27:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.864+0000 7f0a3b3b7640 1 -- 172.21.15.120:0/927052790 shutdown_connections 2024-09-19T09:27:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.864+0000 7f0a3b3b7640 1 --2- 172.21.15.120:0/927052790 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f0a200a32f0 0x7f0a200a57b0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.864+0000 7f0a3b3b7640 1 --2- 172.21.15.120:0/927052790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a341025f0 0x7f0a3419f140 unknown :-1 s=CLOSED pgs=2511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.864+0000 7f0a3b3b7640 1 --2- 172.21.15.120:0/927052790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a34101c30 0x7f0a3419cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.864+0000 7f0a3b3b7640 1 --2- 172.21.15.120:0/927052790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a34100a40 0x7f0a3406a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.864+0000 7f0a3b3b7640 1 -- 172.21.15.120:0/927052790 >> 172.21.15.120:0/927052790 conn(0x7f0a340fc1e0 msgr2=0x7f0a34104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.865+0000 7f0a3b3b7640 1 -- 172.21.15.120:0/927052790 shutdown_connections 2024-09-19T09:27:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:27:50.865+0000 7f0a3b3b7640 1 -- 172.21.15.120:0/927052790 wait complete. 2024-09-19T09:27:51.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:50 smithi196 ceph-mon[35150]: pgmap v1094: 97 pgs: 97 active+clean; 456 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 3.3 MiB/s wr, 210 op/s 2024-09-19T09:27:51.187 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:51.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:50 smithi153 ceph-mon[35071]: pgmap v1094: 97 pgs: 97 active+clean; 456 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 3.3 MiB/s wr, 210 op/s 2024-09-19T09:27:51.307 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:50 smithi120 ceph-mon[31150]: pgmap v1094: 97 pgs: 97 active+clean; 456 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 3.3 MiB/s wr, 210 op/s 2024-09-19T09:27:51.396 INFO:tasks.workunit.client.0.smithi120.stdout: 10 268981 19.63 MB/sec execute 586 sec latency 18183.808 ms 2024-09-19T09:27:51.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.702+0000 7fe9423b1640 1 -- 172.21.15.120:0/1439879315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe93c100a60 msgr2=0x7fe93c100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:51.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.702+0000 7fe9423b1640 1 --2- 172.21.15.120:0/1439879315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe93c100a60 0x7fe93c100ec0 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7fe930009930 tx=0x7fe93002a420 comp rx=0 tx=0).stop 2024-09-19T09:27:51.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.702+0000 7fe9423b1640 1 -- 172.21.15.120:0/1439879315 shutdown_connections 2024-09-19T09:27:51.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.702+0000 7fe9423b1640 1 --2- 172.21.15.120:0/1439879315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe93c101400 0x7fe93c10d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.702+0000 7fe9423b1640 1 --2- 172.21.15.120:0/1439879315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe93c100a60 0x7fe93c100ec0 unknown :-1 s=CLOSED pgs=2516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.702+0000 7fe9423b1640 1 --2- 172.21.15.120:0/1439879315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe93c106a60 0x7fe93c106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.702+0000 7fe9423b1640 1 -- 172.21.15.120:0/1439879315 >> 172.21.15.120:0/1439879315 conn(0x7fe93c0fc780 msgr2=0x7fe93c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:51.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.702+0000 7fe9423b1640 1 -- 172.21.15.120:0/1439879315 shutdown_connections 2024-09-19T09:27:51.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.702+0000 7fe9423b1640 1 -- 172.21.15.120:0/1439879315 wait complete. 2024-09-19T09:27:51.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.702+0000 7fe9423b1640 1 Processor -- start 2024-09-19T09:27:51.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.702+0000 7fe9423b1640 1 -- start start 2024-09-19T09:27:51.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.703+0000 7fe9423b1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe93c100a60 0x7fe93c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:51.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.703+0000 7fe9423b1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe93c101400 0x7fe93c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:51.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.703+0000 7fe9423b1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe93c106a60 0x7fe93c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:51.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.703+0000 7fe9423b1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c110e60 con 0x7fe93c101400 2024-09-19T09:27:51.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.703+0000 7fe9423b1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c110ce0 con 0x7fe93c106a60 2024-09-19T09:27:51.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.703+0000 7fe9423b1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c110fe0 con 0x7fe93c100a60 2024-09-19T09:27:51.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.703+0000 7fe93b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe93c101400 0x7fe93c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:51.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.703+0000 7fe93b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe93c101400 0x7fe93c19aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49850/0 (socket says 172.21.15.120:49850) 2024-09-19T09:27:51.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.703+0000 7fe93b7fe640 1 -- 172.21.15.120:0/1953500496 learned_addr learned my addr 172.21.15.120:0/1953500496 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:51.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.703+0000 7fe93bfff640 1 --2- 172.21.15.120:0/1953500496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe93c100a60 0x7fe93c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:51.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.703+0000 7fe94094e640 1 --2- 172.21.15.120:0/1953500496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe93c106a60 0x7fe93c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:51.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.703+0000 7fe93b7fe640 1 -- 172.21.15.120:0/1953500496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe93c100a60 msgr2=0x7fe93c19a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:51.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.703+0000 7fe93b7fe640 1 --2- 172.21.15.120:0/1953500496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe93c100a60 0x7fe93c19a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.703+0000 7fe93b7fe640 1 -- 172.21.15.120:0/1953500496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe93c106a60 msgr2=0x7fe93c19ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:51.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.703+0000 7fe93b7fe640 1 --2- 172.21.15.120:0/1953500496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe93c106a60 0x7fe93c19ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.703+0000 7fe93b7fe640 1 -- 172.21.15.120:0/1953500496 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93c19f560 con 0x7fe93c101400 2024-09-19T09:27:51.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.703+0000 7fe93b7fe640 1 --2- 172.21.15.120:0/1953500496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe93c101400 0x7fe93c19aaf0 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7fe930009230 tx=0x7fe930004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:51.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.703+0000 7fe9397fa640 1 -- 172.21.15.120:0/1953500496 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93003f030 con 0x7fe93c101400 2024-09-19T09:27:51.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.704+0000 7fe9423b1640 1 -- 172.21.15.120:0/1953500496 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe93c19f7f0 con 0x7fe93c101400 2024-09-19T09:27:51.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.704+0000 7fe9397fa640 1 -- 172.21.15.120:0/1953500496 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe930003ad0 con 0x7fe93c101400 2024-09-19T09:27:51.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.704+0000 7fe9397fa640 1 -- 172.21.15.120:0/1953500496 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe930035070 con 0x7fe93c101400 2024-09-19T09:27:51.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.704+0000 7fe9423b1640 1 -- 172.21.15.120:0/1953500496 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe93c1a7130 con 0x7fe93c101400 2024-09-19T09:27:51.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.706+0000 7fe9397fa640 1 -- 172.21.15.120:0/1953500496 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe930042030 con 0x7fe93c101400 2024-09-19T09:27:51.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.706+0000 7fe94478a740 1 Processor -- start 2024-09-19T09:27:51.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.706+0000 7fe94478a740 1 -- start start 2024-09-19T09:27:51.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.706+0000 7fe94478a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a947683060 0x55a947683440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:51.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.706+0000 7fe94478a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a947683980 0x55a94768dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:51.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.706+0000 7fe94478a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a94768e240 0x55a947690630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:51.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.706+0000 7fe94478a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a9475e8620 con 0x55a947683980 2024-09-19T09:27:51.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.706+0000 7fe94478a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a94758fe50 con 0x55a947683060 2024-09-19T09:27:51.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.706+0000 7fe94478a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a9475e45e0 con 0x55a94768e240 2024-09-19T09:27:51.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.706+0000 7fe93bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a947683980 0x55a94768dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:51.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.706+0000 7fe93bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a947683980 0x55a94768dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49862/0 (socket says 172.21.15.120:49862) 2024-09-19T09:27:51.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.706+0000 7fe93bfff640 1 -- 172.21.15.120:0/3303632392 learned_addr learned my addr 172.21.15.120:0/3303632392 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:51.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.706+0000 7fe94094e640 1 --2- 172.21.15.120:0/3303632392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a947683060 0x55a947683440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:51.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.707+0000 7fe9397fa640 1 --2- 172.21.15.120:0/1953500496 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9240a32c0 0x7fe9240a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:51.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.707+0000 7fe93b7fe640 1 --2- 172.21.15.120:0/3303632392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a94768e240 0x55a947690630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:51.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.707+0000 7fe94094e640 1 --2- 172.21.15.120:0/1953500496 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9240a32c0 0x7fe9240a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:51.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.707+0000 7fe93bfff640 1 -- 172.21.15.120:0/3303632392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a94768e240 msgr2=0x55a947690630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:51.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.707+0000 7fe93bfff640 1 --2- 172.21.15.120:0/3303632392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a94768e240 0x55a947690630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.707+0000 7fe93bfff640 1 -- 172.21.15.120:0/3303632392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a947683060 msgr2=0x55a947683440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:51.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.707+0000 7fe93bfff640 1 --2- 172.21.15.120:0/3303632392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a947683060 0x55a947683440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.707+0000 7fe93bfff640 1 -- 172.21.15.120:0/3303632392 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a947580d40 con 0x55a947683980 2024-09-19T09:27:51.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.707+0000 7fe93b7fe640 1 --2- 172.21.15.120:0/3303632392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a94768e240 0x55a947690630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:51.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.707+0000 7fe9397fa640 1 -- 172.21.15.120:0/1953500496 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe9300efcb0 con 0x7fe93c101400 2024-09-19T09:27:51.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.707+0000 7fe93bfff640 1 --2- 172.21.15.120:0/3303632392 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a947683980 0x55a94768dd00 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7fe92c00bc90 tx=0x7fe92c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29839967578abaa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:51.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.707+0000 7fe94094e640 1 --2- 172.21.15.120:0/1953500496 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9240a32c0 0x7fe9240a5780 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe934007310 tx=0x7fe934004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:51.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.707+0000 7fe9227fc640 1 -- 172.21.15.120:0/3303632392 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c01c480 con 0x55a947683980 2024-09-19T09:27:51.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.707+0000 7fe9227fc640 1 -- 172.21.15.120:0/3303632392 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe92c01cb00 con 0x55a947683980 2024-09-19T09:27:51.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.707+0000 7fe9227fc640 1 -- 172.21.15.120:0/3303632392 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c011bb0 con 0x55a947683980 2024-09-19T09:27:51.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.708+0000 7fe94478a740 1 -- 172.21.15.120:0/3303632392 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a947683980 msgr2=0x55a94768dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:51.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.708+0000 7fe94478a740 1 --2- 172.21.15.120:0/3303632392 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a947683980 0x55a94768dd00 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7fe92c00bc90 tx=0x7fe92c0107a0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.708+0000 7fe94478a740 1 -- 172.21.15.120:0/3303632392 shutdown_connections 2024-09-19T09:27:51.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.708+0000 7fe94478a740 1 --2- 172.21.15.120:0/3303632392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a94768e240 0x55a947690630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.708+0000 7fe94478a740 1 --2- 172.21.15.120:0/3303632392 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a947683980 0x55a94768dd00 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.708+0000 7fe94478a740 1 --2- 172.21.15.120:0/3303632392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a947683060 0x55a947683440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.708+0000 7fe94478a740 1 -- 172.21.15.120:0/3303632392 >> 172.21.15.120:0/3303632392 conn(0x55a94767f5f0 msgr2=0x55a94767f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:51.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.708+0000 7fe94478a740 1 -- 172.21.15.120:0/3303632392 shutdown_connections 2024-09-19T09:27:51.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.708+0000 7fe94478a740 1 -- 172.21.15.120:0/3303632392 wait complete. 2024-09-19T09:27:51.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.708+0000 7fe94478a740 1 -- start start 2024-09-19T09:27:51.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.708+0000 7fe94478a740 1 Processor -- start 2024-09-19T09:27:51.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.709+0000 7fe94478a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a947683060 0x55a94768b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:51.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.709+0000 7fe94478a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a947683980 0x55a94768b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:51.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.709+0000 7fe94478a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a94768e240 0x55a94768bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:51.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.709+0000 7fe94478a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a947690e10 con 0x55a94768e240 2024-09-19T09:27:51.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.709+0000 7fe94478a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a947570f10 con 0x55a947683980 2024-09-19T09:27:51.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.709+0000 7fe94478a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a947690f90 con 0x55a947683060 2024-09-19T09:27:51.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.709+0000 7fe94478a740 10 client.0 _ll_register_callbacks cb 0x55a947594db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:51.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.709+0000 7fe94094e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a94768e240 0x55a94768bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:51.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.709+0000 7fe94094e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a94768e240 0x55a94768bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49866/0 (socket says 172.21.15.120:49866) 2024-09-19T09:27:51.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.709+0000 7fe94094e640 1 -- 172.21.15.120:0/2037289877 learned_addr learned my addr 172.21.15.120:0/2037289877 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:51.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.709+0000 7fe93b7fe640 1 --2- 172.21.15.120:0/2037289877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a947683980 0x55a94768b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:51.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.709+0000 7fe93bfff640 1 --2- 172.21.15.120:0/2037289877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a947683060 0x55a94768b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:51.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.709+0000 7fe94094e640 1 -- 172.21.15.120:0/2037289877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a947683060 msgr2=0x55a94768b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:51.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.709+0000 7fe94094e640 1 --2- 172.21.15.120:0/2037289877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a947683060 0x55a94768b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.709+0000 7fe94094e640 1 -- 172.21.15.120:0/2037289877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a947683980 msgr2=0x55a94768b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:51.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.709+0000 7fe94094e640 1 --2- 172.21.15.120:0/2037289877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a947683980 0x55a94768b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.709+0000 7fe94094e640 1 -- 172.21.15.120:0/2037289877 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9472f7580 con 0x55a94768e240 2024-09-19T09:27:51.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.709+0000 7fe94094e640 1 --2- 172.21.15.120:0/2037289877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a94768e240 0x55a94768bb60 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7fe93400f440 tx=0x7fe934004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982b7b543cb04b8f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:51.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.709+0000 7fe91b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:27:51.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.709+0000 7fe91b7fe640 1 -- 172.21.15.120:0/2037289877 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe934019070 con 0x55a94768e240 2024-09-19T09:27:51.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.710+0000 7fe91b7fe640 1 -- 172.21.15.120:0/2037289877 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe934004d20 con 0x55a94768e240 2024-09-19T09:27:51.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.710+0000 7fe94478a740 1 -- 172.21.15.120:0/2037289877 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe92c01cb00 con 0x55a94768e240 2024-09-19T09:27:51.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.710+0000 7fe91b7fe640 1 -- 172.21.15.120:0/2037289877 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe934020610 con 0x55a94768e240 2024-09-19T09:27:51.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.710+0000 7fe94478a740 10 client.23523 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:51.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.710+0000 7fe94478a740 1 -- 172.21.15.120:0/2037289877 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a9473c1960 con 0x55a94768e240 2024-09-19T09:27:51.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.710+0000 7fe91b7fe640 1 -- 172.21.15.120:0/2037289877 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe92c01cb00 con 0x55a94768e240 2024-09-19T09:27:51.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.710+0000 7fe91b7fe640 1 -- 172.21.15.120:0/2037289877 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fe934028980 con 0x55a94768e240 2024-09-19T09:27:51.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.710+0000 7fe94478a740 10 client.23523 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:51.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.710+0000 7fe94478a740 10 client.23523 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:27:51.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.710+0000 7fe94478a740 20 client.23523 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:51.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.710+0000 7fe94478a740 1 --2- 172.21.15.120:0/2037289877 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a94769dc80 0x55a9476a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:51.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.710+0000 7fe94478a740 4 client.23523 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:51.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.710+0000 7fe94478a740 1 -- 172.21.15.120:0/2037289877 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9475f5b90 con 0x55a94769dc80 2024-09-19T09:27:51.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.710+0000 7fe93bfff640 1 --2- 172.21.15.120:0/2037289877 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a94769dc80 0x55a9476a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:51.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.711+0000 7fe91b7fe640 10 client.23523 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:51.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.711+0000 7fe93bfff640 1 --2- 172.21.15.120:0/2037289877 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a94769dc80 0x55a9476a00f0 crc :-1 s=READY pgs=611 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0684ee21e87d997 server_cookie=ea1f80864be37b05 in_seq=0 out_seq=0 2024-09-19T09:27:51.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.711+0000 7fe91b7fe640 1 -- 172.21.15.120:0/2037289877 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a9475f5b90 con 0x55a94769dc80 2024-09-19T09:27:51.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.711+0000 7fe91b7fe640 10 client.23523 handle_command_reply: tid=0 2024-09-19T09:27:51.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.711+0000 7fe94478a740 1 client.23523 shutdown 2024-09-19T09:27:51.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.712+0000 7fe94478a740 1 -- 172.21.15.120:0/2037289877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a94768e240 msgr2=0x55a94768bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:51.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.712+0000 7fe94478a740 1 --2- 172.21.15.120:0/2037289877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a94768e240 0x55a94768bb60 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7fe93400f440 tx=0x7fe934004a40 comp rx=0 tx=0).stop 2024-09-19T09:27:51.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.712+0000 7fe94478a740 1 -- 172.21.15.120:0/2037289877 shutdown_connections 2024-09-19T09:27:51.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.712+0000 7fe94478a740 1 --2- 172.21.15.120:0/2037289877 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a94769dc80 0x55a9476a00f0 crc :-1 s=READY pgs=611 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.712+0000 7fe94478a740 1 --2- 172.21.15.120:0/2037289877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a94768e240 0x55a94768bb60 unknown :-1 s=CLOSED pgs=2519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.712+0000 7fe94478a740 1 --2- 172.21.15.120:0/2037289877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a947683980 0x55a94768b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.712+0000 7fe94478a740 1 --2- 172.21.15.120:0/2037289877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a947683060 0x55a94768b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.712+0000 7fe91b7fe640 0 client.23523 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:51.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.712+0000 7fe94478a740 1 -- 172.21.15.120:0/2037289877 >> 172.21.15.120:0/2037289877 conn(0x55a94767f5f0 msgr2=0x55a947685320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:51.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.712+0000 7fe94478a740 1 -- 172.21.15.120:0/2037289877 shutdown_connections 2024-09-19T09:27:51.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.712+0000 7fe94478a740 1 -- 172.21.15.120:0/2037289877 wait complete. 2024-09-19T09:27:51.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.712+0000 7fe94478a740 20 client.23523 trim_cache size 0 max 16384 2024-09-19T09:27:51.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.729+0000 7fe94478a740 1 Processor -- start 2024-09-19T09:27:51.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.729+0000 7fe94478a740 1 -- start start 2024-09-19T09:27:51.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.729+0000 7fe94478a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a947683060 0x55a9476a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:51.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.729+0000 7fe94478a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a947683980 0x55a9476a5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:51.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.729+0000 7fe94478a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a94768e240 0x55a947688340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:51.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.729+0000 7fe94478a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c1af780 con 0x55a947683980 2024-09-19T09:27:51.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.729+0000 7fe94478a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c1b12c0 con 0x55a947683060 2024-09-19T09:27:51.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.729+0000 7fe94478a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c1afe50 con 0x55a94768e240 2024-09-19T09:27:51.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.729+0000 7fe93b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a947683980 0x55a9476a5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:51.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.729+0000 7fe93b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a947683980 0x55a9476a5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49872/0 (socket says 172.21.15.120:49872) 2024-09-19T09:27:51.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.729+0000 7fe93b7fe640 1 -- 172.21.15.120:0/3443717817 learned_addr learned my addr 172.21.15.120:0/3443717817 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:51.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.729+0000 7fe94094e640 1 --2- 172.21.15.120:0/3443717817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a94768e240 0x55a947688340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:51.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.729+0000 7fe93bfff640 1 --2- 172.21.15.120:0/3443717817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a947683060 0x55a9476a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:51.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.729+0000 7fe93b7fe640 1 -- 172.21.15.120:0/3443717817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a94768e240 msgr2=0x55a947688340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:51.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.729+0000 7fe93b7fe640 1 --2- 172.21.15.120:0/3443717817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a94768e240 0x55a947688340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.729+0000 7fe93b7fe640 1 -- 172.21.15.120:0/3443717817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a947683060 msgr2=0x55a9476a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:51.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.729+0000 7fe93b7fe640 1 --2- 172.21.15.120:0/3443717817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a947683060 0x55a9476a5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.729+0000 7fe93b7fe640 1 -- 172.21.15.120:0/3443717817 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a94767f420 con 0x55a947683980 2024-09-19T09:27:51.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.729+0000 7fe93b7fe640 1 --2- 172.21.15.120:0/3443717817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a947683980 0x55a9476a5f50 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7fe9300bc080 tx=0x7fe93008f200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8a348deedbaf33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:51.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.730+0000 7fe921ffb640 1 -- 172.21.15.120:0/3443717817 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe930049040 con 0x55a947683980 2024-09-19T09:27:51.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.730+0000 7fe921ffb640 1 -- 172.21.15.120:0/3443717817 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9300c0680 con 0x55a947683980 2024-09-19T09:27:51.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.730+0000 7fe921ffb640 1 -- 172.21.15.120:0/3443717817 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9300b89f0 con 0x55a947683980 2024-09-19T09:27:51.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.730+0000 7fe94478a740 1 -- 172.21.15.120:0/3443717817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a947683980 msgr2=0x55a9476a5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:51.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.730+0000 7fe94478a740 1 --2- 172.21.15.120:0/3443717817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a947683980 0x55a9476a5f50 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7fe9300bc080 tx=0x7fe93008f200 comp rx=0 tx=0).stop 2024-09-19T09:27:51.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.730+0000 7fe94478a740 1 -- 172.21.15.120:0/3443717817 shutdown_connections 2024-09-19T09:27:51.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.730+0000 7fe94478a740 1 --2- 172.21.15.120:0/3443717817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a94768e240 0x55a947688340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.730+0000 7fe94478a740 1 --2- 172.21.15.120:0/3443717817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a947683980 0x55a9476a5f50 unknown :-1 s=CLOSED pgs=2520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.730+0000 7fe94478a740 1 --2- 172.21.15.120:0/3443717817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a947683060 0x55a9476a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.730+0000 7fe94478a740 1 -- 172.21.15.120:0/3443717817 >> 172.21.15.120:0/3443717817 conn(0x55a94767f5f0 msgr2=0x55a9476899b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:51.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.730+0000 7fe94478a740 1 -- 172.21.15.120:0/3443717817 shutdown_connections 2024-09-19T09:27:51.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.730+0000 7fe94478a740 1 -- 172.21.15.120:0/3443717817 wait complete. 2024-09-19T09:27:51.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.731+0000 7fe94478a740 1 -- start start 2024-09-19T09:27:51.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.731+0000 7fe94478a740 1 Processor -- start 2024-09-19T09:27:51.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.731+0000 7fe94478a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a947683060 0x55a947696960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:51.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.731+0000 7fe94478a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a947683980 0x55a947692ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:51.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.731+0000 7fe94478a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a94768e240 0x55a9476931e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:51.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.731+0000 7fe94478a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c1b0bf0 con 0x55a947683980 2024-09-19T09:27:51.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.731+0000 7fe94478a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c1b0520 con 0x55a947683060 2024-09-19T09:27:51.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.731+0000 7fe94478a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c1ad740 con 0x55a94768e240 2024-09-19T09:27:51.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.731+0000 7fe94478a740 10 client.0 _ll_register_callbacks cb 0x7fe934011f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:51.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.731+0000 7fe93b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a947683980 0x55a947692ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:51.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.731+0000 7fe93b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a947683980 0x55a947692ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49888/0 (socket says 172.21.15.120:49888) 2024-09-19T09:27:51.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.731+0000 7fe93b7fe640 1 -- 172.21.15.120:0/130535778 learned_addr learned my addr 172.21.15.120:0/130535778 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:51.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.731+0000 7fe93bfff640 1 --2- 172.21.15.120:0/130535778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a947683060 0x55a947696960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:51.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.731+0000 7fe94094e640 1 --2- 172.21.15.120:0/130535778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a94768e240 0x55a9476931e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:51.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.732+0000 7fe93b7fe640 1 -- 172.21.15.120:0/130535778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a94768e240 msgr2=0x55a9476931e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:51.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.732+0000 7fe93b7fe640 1 --2- 172.21.15.120:0/130535778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a94768e240 0x55a9476931e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.732+0000 7fe93b7fe640 1 -- 172.21.15.120:0/130535778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a947683060 msgr2=0x55a947696960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:51.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.732+0000 7fe93b7fe640 1 --2- 172.21.15.120:0/130535778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a947683060 0x55a947696960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.732+0000 7fe93b7fe640 1 -- 172.21.15.120:0/130535778 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a947680280 con 0x55a947683980 2024-09-19T09:27:51.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.732+0000 7fe93b7fe640 1 --2- 172.21.15.120:0/130535778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a947683980 0x55a947692ca0 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7fe9300b87e0 tx=0x7fe9300c5090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947622735dc9f71 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:51.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.732+0000 7fe91b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:27:51.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.732+0000 7fe91b7fe640 1 -- 172.21.15.120:0/130535778 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93009e020 con 0x55a947683980 2024-09-19T09:27:51.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.732+0000 7fe91b7fe640 1 -- 172.21.15.120:0/130535778 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe9300b89f0 con 0x55a947683980 2024-09-19T09:27:51.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.732+0000 7fe94478a740 1 -- 172.21.15.120:0/130535778 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9300c0680 con 0x55a947683980 2024-09-19T09:27:51.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.732+0000 7fe91b7fe640 1 -- 172.21.15.120:0/130535778 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe9300ed980 con 0x55a947683980 2024-09-19T09:27:51.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.732+0000 7fe94478a740 10 client.23529 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:51.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.732+0000 7fe94478a740 1 -- 172.21.15.120:0/130535778 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a947709890 con 0x55a947683980 2024-09-19T09:27:51.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.732+0000 7fe91b7fe640 1 -- 172.21.15.120:0/130535778 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9300c0680 con 0x55a947683980 2024-09-19T09:27:51.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.732+0000 7fe91b7fe640 1 -- 172.21.15.120:0/130535778 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fe930031bd0 con 0x55a947683980 2024-09-19T09:27:51.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.732+0000 7fe94478a740 10 client.23529 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:51.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.732+0000 7fe94478a740 10 client.23529 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:27:51.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.732+0000 7fe94478a740 20 client.23529 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:51.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.732+0000 7fe94478a740 1 --2- 172.21.15.120:0/130535778 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a94770a060 0x55a94770c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:51.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.732+0000 7fe94478a740 4 client.23529 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "63770883", "release": true, "await": true}] 2024-09-19T09:27:51.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.732+0000 7fe94478a740 1 -- 172.21.15.120:0/130535778 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "set_id": "63770883", "release": true, "await": true}) -- 0x55a9475f00a0 con 0x55a94770a060 2024-09-19T09:27:51.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.733+0000 7fe93bfff640 1 --2- 172.21.15.120:0/130535778 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a94770a060 0x55a94770c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:51.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.733+0000 7fe93bfff640 1 --2- 172.21.15.120:0/130535778 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a94770a060 0x55a94770c450 crc :-1 s=READY pgs=612 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fba54d4a545f72e server_cookie=43b1c5d809a20479 in_seq=0 out_seq=0 2024-09-19T09:27:51.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.733+0000 7fe91b7fe640 10 client.23529 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:51.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.818+0000 7fe91b7fe640 1 -- 172.21.15.120:0/130535778 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a9475f00a0 con 0x55a94770a060 2024-09-19T09:27:51.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.818+0000 7fe91b7fe640 10 client.23529 handle_command_reply: tid=0 2024-09-19T09:27:51.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.818+0000 7fe94478a740 1 client.23529 shutdown 2024-09-19T09:27:51.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.819+0000 7fe94478a740 1 -- 172.21.15.120:0/130535778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a947683980 msgr2=0x55a947692ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:51.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.819+0000 7fe94478a740 1 --2- 172.21.15.120:0/130535778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a947683980 0x55a947692ca0 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7fe9300b87e0 tx=0x7fe9300c5090 comp rx=0 tx=0).stop 2024-09-19T09:27:51.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.819+0000 7fe94478a740 1 -- 172.21.15.120:0/130535778 shutdown_connections 2024-09-19T09:27:51.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.819+0000 7fe94478a740 1 --2- 172.21.15.120:0/130535778 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55a94770a060 0x55a94770c450 crc :-1 s=READY pgs=612 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.819+0000 7fe94478a740 1 --2- 172.21.15.120:0/130535778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a94768e240 0x55a9476931e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.819+0000 7fe94478a740 1 --2- 172.21.15.120:0/130535778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a947683980 0x55a947692ca0 unknown :-1 s=CLOSED pgs=2521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.819+0000 7fe94478a740 1 --2- 172.21.15.120:0/130535778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a947683060 0x55a947696960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.819+0000 7fe94478a740 1 -- 172.21.15.120:0/130535778 >> 172.21.15.120:0/130535778 conn(0x55a94767f5f0 msgr2=0x55a947684b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:51.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.819+0000 7fe91b7fe640 0 client.23529 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:51.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.819+0000 7fe94478a740 1 -- 172.21.15.120:0/130535778 shutdown_connections 2024-09-19T09:27:51.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.819+0000 7fe94478a740 1 -- 172.21.15.120:0/130535778 wait complete. 2024-09-19T09:27:51.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.819+0000 7fe94478a740 20 client.23529 trim_cache size 0 max 16384 2024-09-19T09:27:51.823 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:27:51.823 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 39, 2024-09-19T09:27:51.823 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:27:51.824 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 88, 2024-09-19T09:27:51.824 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": { 2024-09-19T09:27:51.824 INFO:teuthology.orchestra.run.smithi120.stdout: "63770883": { 2024-09-19T09:27:51.824 INFO:teuthology.orchestra.run.smithi120.stdout: "version": 88, 2024-09-19T09:27:51.824 INFO:teuthology.orchestra.run.smithi120.stdout: "age_ref": 0.0, 2024-09-19T09:27:51.824 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:27:51.824 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:27:51.824 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:27:51.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:27:51.824 INFO:teuthology.orchestra.run.smithi120.stdout: "timeout": 90.0, 2024-09-19T09:27:51.824 INFO:teuthology.orchestra.run.smithi120.stdout: "expiration": 138.5, 2024-09-19T09:27:51.824 INFO:teuthology.orchestra.run.smithi120.stdout: "members": { 2024-09-19T09:27:51.824 INFO:teuthology.orchestra.run.smithi120.stdout: "file:/": { 2024-09-19T09:27:51.824 INFO:teuthology.orchestra.run.smithi120.stdout: "excluded": false, 2024-09-19T09:27:51.825 INFO:teuthology.orchestra.run.smithi120.stdout: "state": { 2024-09-19T09:27:51.825 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RELEASED", 2024-09-19T09:27:51.825 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0 2024-09-19T09:27:51.825 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:27:51.825 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:27:51.825 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:27:51.825 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:27:51.825 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:27:51.825 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:27:51.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.819+0000 7fe9423b1640 1 -- 172.21.15.120:0/1953500496 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9240a32c0 msgr2=0x7fe9240a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:51.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.819+0000 7fe9423b1640 1 --2- 172.21.15.120:0/1953500496 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9240a32c0 0x7fe9240a5780 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe934007310 tx=0x7fe934004ab0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.819+0000 7fe9423b1640 1 -- 172.21.15.120:0/1953500496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe93c101400 msgr2=0x7fe93c19aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:51.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.819+0000 7fe9423b1640 1 --2- 172.21.15.120:0/1953500496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe93c101400 0x7fe93c19aaf0 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7fe930009230 tx=0x7fe930004270 comp rx=0 tx=0).stop 2024-09-19T09:27:51.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.820+0000 7fe9423b1640 1 -- 172.21.15.120:0/1953500496 shutdown_connections 2024-09-19T09:27:51.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.820+0000 7fe9423b1640 1 --2- 172.21.15.120:0/1953500496 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9240a32c0 0x7fe9240a5780 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.820+0000 7fe9423b1640 1 --2- 172.21.15.120:0/1953500496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe93c106a60 0x7fe93c19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.820+0000 7fe9423b1640 1 --2- 172.21.15.120:0/1953500496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe93c101400 0x7fe93c19aaf0 unknown :-1 s=CLOSED pgs=2517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.820+0000 7fe9423b1640 1 --2- 172.21.15.120:0/1953500496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe93c100a60 0x7fe93c19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:51.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.820+0000 7fe9423b1640 1 -- 172.21.15.120:0/1953500496 >> 172.21.15.120:0/1953500496 conn(0x7fe93c0fc780 msgr2=0x7fe93c06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:51.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.820+0000 7fe9423b1640 1 -- 172.21.15.120:0/1953500496 shutdown_connections 2024-09-19T09:27:51.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:51.820+0000 7fe9423b1640 1 -- 172.21.15.120:0/1953500496 wait complete. 2024-09-19T09:27:52.397 INFO:tasks.workunit.client.0.smithi120.stdout: 10 269358 19.64 MB/sec execute 587 sec latency 18629.266 ms 2024-09-19T09:27:52.693 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '63770883', total seconds elapsed: 22.4 2024-09-19T09:27:53.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:52 smithi120 ceph-mon[31150]: pgmap v1095: 97 pgs: 97 active+clean; 455 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 2.5 MiB/s wr, 163 op/s 2024-09-19T09:27:53.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:52 smithi196 ceph-mon[35150]: pgmap v1095: 97 pgs: 97 active+clean; 455 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 2.5 MiB/s wr, 163 op/s 2024-09-19T09:27:53.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:52 smithi153 ceph-mon[35071]: pgmap v1095: 97 pgs: 97 active+clean; 455 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 2.5 MiB/s wr, 163 op/s 2024-09-19T09:27:53.396 INFO:tasks.workunit.client.0.smithi120.stdout: 10 269831 19.66 MB/sec execute 588 sec latency 78.677 ms 2024-09-19T09:27:54.396 INFO:tasks.workunit.client.0.smithi120.stdout: 10 270398 19.67 MB/sec execute 589 sec latency 53.231 ms 2024-09-19T09:27:54.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:54 smithi196 ceph-mon[35150]: pgmap v1096: 97 pgs: 97 active+clean; 452 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 1.9 MiB/s wr, 116 op/s 2024-09-19T09:27:54.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:54 smithi153 ceph-mon[35071]: pgmap v1096: 97 pgs: 97 active+clean; 452 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 1.9 MiB/s wr, 116 op/s 2024-09-19T09:27:54.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:54 smithi120 ceph-mon[31150]: pgmap v1096: 97 pgs: 97 active+clean; 452 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 1.9 MiB/s wr, 116 op/s 2024-09-19T09:27:55.396 INFO:tasks.workunit.client.0.smithi120.stdout: 10 270915 19.67 MB/sec execute 590 sec latency 57.899 ms 2024-09-19T09:27:56.396 INFO:tasks.workunit.client.0.smithi120.stdout: 10 271518 19.70 MB/sec execute 591 sec latency 59.487 ms 2024-09-19T09:27:56.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:56 smithi196 ceph-mon[35150]: pgmap v1097: 97 pgs: 97 active+clean; 455 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 968 KiB/s wr, 101 op/s 2024-09-19T09:27:56.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:56 smithi153 ceph-mon[35071]: pgmap v1097: 97 pgs: 97 active+clean; 455 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 968 KiB/s wr, 101 op/s 2024-09-19T09:27:56.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:56 smithi120 ceph-mon[31150]: pgmap v1097: 97 pgs: 97 active+clean; 455 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 968 KiB/s wr, 101 op/s 2024-09-19T09:27:57.396 INFO:tasks.workunit.client.0.smithi120.stdout: 10 272167 19.73 MB/sec execute 592 sec latency 84.758 ms 2024-09-19T09:27:58.396 INFO:tasks.workunit.client.0.smithi120.stdout: 10 272821 19.73 MB/sec execute 593 sec latency 67.926 ms 2024-09-19T09:27:58.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:27:58 smithi196 ceph-mon[35150]: pgmap v1098: 97 pgs: 97 active+clean; 463 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-09-19T09:27:58.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:27:58 smithi153 ceph-mon[35071]: pgmap v1098: 97 pgs: 97 active+clean; 463 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-09-19T09:27:58.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:27:58 smithi120 ceph-mon[31150]: pgmap v1098: 97 pgs: 97 active+clean; 463 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-09-19T09:27:58.725 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:27:58.999 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:59.396 INFO:tasks.workunit.client.0.smithi120.stdout: 10 273497 19.78 MB/sec execute 594 sec latency 49.035 ms 2024-09-19T09:27:59.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.663+0000 7fc4f30ab640 1 -- 172.21.15.120:0/2662598914 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4ec1036e0 msgr2=0x7fc4ec107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:59.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.663+0000 7fc4f30ab640 1 --2- 172.21.15.120:0/2662598914 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4ec1036e0 0x7fc4ec107b90 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009930 tx=0x7fc4d802a420 comp rx=0 tx=0).stop 2024-09-19T09:27:59.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.663+0000 7fc4f30ab640 1 -- 172.21.15.120:0/2662598914 shutdown_connections 2024-09-19T09:27:59.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.663+0000 7fc4f30ab640 1 --2- 172.21.15.120:0/2662598914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4ec1082a0 0x7fc4ec10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.663+0000 7fc4f30ab640 1 --2- 172.21.15.120:0/2662598914 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4ec1036e0 0x7fc4ec107b90 unknown :-1 s=CLOSED pgs=2522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.663+0000 7fc4f30ab640 1 --2- 172.21.15.120:0/2662598914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4ec102d30 0x7fc4ec103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.663+0000 7fc4f30ab640 1 -- 172.21.15.120:0/2662598914 >> 172.21.15.120:0/2662598914 conn(0x7fc4ec0fc7d0 msgr2=0x7fc4ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:59.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.663+0000 7fc4f30ab640 1 -- 172.21.15.120:0/2662598914 shutdown_connections 2024-09-19T09:27:59.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.663+0000 7fc4f30ab640 1 -- 172.21.15.120:0/2662598914 wait complete. 2024-09-19T09:27:59.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.664+0000 7fc4f30ab640 1 Processor -- start 2024-09-19T09:27:59.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.664+0000 7fc4f30ab640 1 -- start start 2024-09-19T09:27:59.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.665+0000 7fc4f30ab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4ec102d30 0x7fc4ec1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:59.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.665+0000 7fc4f30ab640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4ec1036e0 0x7fc4ec198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:59.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.665+0000 7fc4f30ab640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4ec1082a0 0x7fc4ec19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:59.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.665+0000 7fc4f30ab640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec1120a0 con 0x7fc4ec102d30 2024-09-19T09:27:59.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.665+0000 7fc4f30ab640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec111f20 con 0x7fc4ec1036e0 2024-09-19T09:27:59.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.665+0000 7fc4f30ab640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec112220 con 0x7fc4ec1082a0 2024-09-19T09:27:59.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.665+0000 7fc4f0e47640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4ec102d30 0x7fc4ec1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:59.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.665+0000 7fc4ebfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4ec1036e0 0x7fc4ec198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:59.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.665+0000 7fc4ebfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4ec1036e0 0x7fc4ec198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:40032/0 (socket says 172.21.15.120:40032) 2024-09-19T09:27:59.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.665+0000 7fc4ebfff640 1 -- 172.21.15.120:0/2099072978 learned_addr learned my addr 172.21.15.120:0/2099072978 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:59.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.665+0000 7fc4f1648640 1 --2- 172.21.15.120:0/2099072978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4ec1082a0 0x7fc4ec19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:59.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.665+0000 7fc4ebfff640 1 -- 172.21.15.120:0/2099072978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4ec1082a0 msgr2=0x7fc4ec19ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:59.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.665+0000 7fc4ebfff640 1 --2- 172.21.15.120:0/2099072978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4ec1082a0 0x7fc4ec19ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.665+0000 7fc4ebfff640 1 -- 172.21.15.120:0/2099072978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4ec102d30 msgr2=0x7fc4ec1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:59.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.665+0000 7fc4ebfff640 1 --2- 172.21.15.120:0/2099072978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4ec102d30 0x7fc4ec1985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.665+0000 7fc4ebfff640 1 -- 172.21.15.120:0/2099072978 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ec19d590 con 0x7fc4ec1036e0 2024-09-19T09:27:59.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.665+0000 7fc4f0e47640 1 --2- 172.21.15.120:0/2099072978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4ec102d30 0x7fc4ec1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:27:59.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.666+0000 7fc4ebfff640 1 --2- 172.21.15.120:0/2099072978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4ec1036e0 0x7fc4ec198b20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8005940 tx=0x7fc4d8031b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:59.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.666+0000 7fc4e9ffb640 1 -- 172.21.15.120:0/2099072978 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d803f030 con 0x7fc4ec1036e0 2024-09-19T09:27:59.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.666+0000 7fc4f30ab640 1 -- 172.21.15.120:0/2099072978 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4ec19d820 con 0x7fc4ec1036e0 2024-09-19T09:27:59.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.666+0000 7fc4e9ffb640 1 -- 172.21.15.120:0/2099072978 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4d80043d0 con 0x7fc4ec1036e0 2024-09-19T09:27:59.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.666+0000 7fc4e9ffb640 1 -- 172.21.15.120:0/2099072978 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8035070 con 0x7fc4ec1036e0 2024-09-19T09:27:59.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.666+0000 7fc4f30ab640 1 -- 172.21.15.120:0/2099072978 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4ec1a5160 con 0x7fc4ec1036e0 2024-09-19T09:27:59.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.669+0000 7fc4e9ffb640 1 -- 172.21.15.120:0/2099072978 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fc4d8042030 con 0x7fc4ec1036e0 2024-09-19T09:27:59.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.670+0000 7fc4f5484740 1 Processor -- start 2024-09-19T09:27:59.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.670+0000 7fc4f5484740 1 -- start start 2024-09-19T09:27:59.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.670+0000 7fc4f5484740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654c552f4a0 0x5654c552f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:59.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.670+0000 7fc4f5484740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654c552fdc0 0x5654c553a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:59.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.670+0000 7fc4e9ffb640 1 --2- 172.21.15.120:0/2099072978 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc4dc0a3240 0x7fc4dc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:59.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.670+0000 7fc4f1648640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654c552f4a0 0x5654c552f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:59.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.670+0000 7fc4f1648640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654c552f4a0 0x5654c552f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:56034/0 (socket says 172.21.15.120:56034) 2024-09-19T09:27:59.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.670+0000 7fc4e9ffb640 1 -- 172.21.15.120:0/2099072978 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc4d80efbc0 con 0x7fc4ec1036e0 2024-09-19T09:27:59.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.670+0000 7fc4f5484740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654c553a680 0x5654c553ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:59.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.671+0000 7fc4f0e47640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654c552fdc0 0x5654c553a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:59.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.671+0000 7fc4f1648640 1 -- 172.21.15.120:0/3763437678 learned_addr learned my addr 172.21.15.120:0/3763437678 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:59.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.671+0000 7fc4f5484740 1 -- 172.21.15.120:0/3763437678 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5654c54952e0 con 0x5654c553a680 2024-09-19T09:27:59.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.671+0000 7fc4f5484740 1 -- 172.21.15.120:0/3763437678 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5654c543cce0 con 0x5654c552fdc0 2024-09-19T09:27:59.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.671+0000 7fc4f5484740 1 -- 172.21.15.120:0/3763437678 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5654c54912a0 con 0x5654c552f4a0 2024-09-19T09:27:59.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.671+0000 7fc4f1648640 1 --2- 172.21.15.120:0/2099072978 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc4dc0a3240 0x7fc4dc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:59.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.671+0000 7fc4ebfff640 1 --2- 172.21.15.120:0/3763437678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654c553a680 0x5654c553ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:59.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.671+0000 7fc4f1648640 1 -- 172.21.15.120:0/3763437678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654c552fdc0 msgr2=0x5654c553a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:59.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.671+0000 7fc4f1648640 1 --2- 172.21.15.120:0/3763437678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654c552fdc0 0x5654c553a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.671+0000 7fc4f1648640 1 -- 172.21.15.120:0/3763437678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654c553a680 msgr2=0x5654c553ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:59.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.671+0000 7fc4f1648640 1 --2- 172.21.15.120:0/3763437678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654c553a680 0x5654c553ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.671+0000 7fc4f1648640 1 -- 172.21.15.120:0/3763437678 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654c5421440 con 0x5654c552f4a0 2024-09-19T09:27:59.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.671+0000 7fc4ebfff640 1 --2- 172.21.15.120:0/3763437678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654c553a680 0x5654c553ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:59.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.671+0000 7fc4f1648640 1 --2- 172.21.15.120:0/3763437678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654c552f4a0 0x5654c552f880 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4005f40 tx=0x7fc4e400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a18a0075d022f55a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:59.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.671+0000 7fc4f1648640 1 --2- 172.21.15.120:0/2099072978 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc4dc0a3240 0x7fc4dc0a5700 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400fc90 tx=0x7fc4e4018000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:59.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.671+0000 7fc4d6ffd640 1 -- 172.21.15.120:0/3763437678 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e402c070 con 0x5654c552f4a0 2024-09-19T09:27:59.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.672+0000 7fc4d6ffd640 1 -- 172.21.15.120:0/3763437678 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4e4023070 con 0x5654c552f4a0 2024-09-19T09:27:59.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.672+0000 7fc4d6ffd640 1 -- 172.21.15.120:0/3763437678 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4004680 con 0x5654c552f4a0 2024-09-19T09:27:59.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.672+0000 7fc4f5484740 1 -- 172.21.15.120:0/3763437678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654c552f4a0 msgr2=0x5654c552f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:59.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.672+0000 7fc4f5484740 1 --2- 172.21.15.120:0/3763437678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654c552f4a0 0x5654c552f880 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4005f40 tx=0x7fc4e400f790 comp rx=0 tx=0).stop 2024-09-19T09:27:59.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.672+0000 7fc4f5484740 1 -- 172.21.15.120:0/3763437678 shutdown_connections 2024-09-19T09:27:59.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.672+0000 7fc4f5484740 1 --2- 172.21.15.120:0/3763437678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654c553a680 0x5654c553ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.672+0000 7fc4f5484740 1 --2- 172.21.15.120:0/3763437678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654c552fdc0 0x5654c553a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.672+0000 7fc4f5484740 1 --2- 172.21.15.120:0/3763437678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654c552f4a0 0x5654c552f880 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.672+0000 7fc4f5484740 1 -- 172.21.15.120:0/3763437678 >> 172.21.15.120:0/3763437678 conn(0x5654c552ba30 msgr2=0x5654c552be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:59.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.672+0000 7fc4f5484740 1 -- 172.21.15.120:0/3763437678 shutdown_connections 2024-09-19T09:27:59.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.673+0000 7fc4f5484740 1 -- 172.21.15.120:0/3763437678 wait complete. 2024-09-19T09:27:59.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.673+0000 7fc4f5484740 1 -- start start 2024-09-19T09:27:59.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.673+0000 7fc4f5484740 1 Processor -- start 2024-09-19T09:27:59.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.673+0000 7fc4f5484740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654c552f4a0 0x5654c5539fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:59.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.673+0000 7fc4f5484740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654c552fdc0 0x5654c5537290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:59.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.673+0000 7fc4f5484740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654c553a680 0x5654c55377f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:59.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.674+0000 7fc4f0e47640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654c552f4a0 0x5654c5539fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:59.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.674+0000 7fc4f0e47640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654c552f4a0 0x5654c5539fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:56042/0 (socket says 172.21.15.120:56042) 2024-09-19T09:27:59.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.674+0000 7fc4f1648640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654c553a680 0x5654c55377f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:59.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.674+0000 7fc4ebfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654c552fdc0 0x5654c5537290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:59.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.674+0000 7fc4f1648640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654c553a680 0x5654c55377f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49916/0 (socket says 172.21.15.120:49916) 2024-09-19T09:27:59.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.674+0000 7fc4f0e47640 1 -- 172.21.15.120:0/3725684568 learned_addr learned my addr 172.21.15.120:0/3725684568 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:59.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.674+0000 7fc4f5484740 1 -- 172.21.15.120:0/3725684568 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5654c553d2a0 con 0x5654c553a680 2024-09-19T09:27:59.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.674+0000 7fc4f5484740 1 -- 172.21.15.120:0/3725684568 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5654c5450050 con 0x5654c552fdc0 2024-09-19T09:27:59.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.674+0000 7fc4f5484740 1 -- 172.21.15.120:0/3725684568 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5654c553d420 con 0x5654c552f4a0 2024-09-19T09:27:59.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.674+0000 7fc4f5484740 10 client.0 _ll_register_callbacks cb 0x5654c5490160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:59.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.674+0000 7fc4f0e47640 1 -- 172.21.15.120:0/3725684568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654c552fdc0 msgr2=0x5654c5537290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:59.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.674+0000 7fc4f0e47640 1 --2- 172.21.15.120:0/3725684568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654c552fdc0 0x5654c5537290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.674+0000 7fc4f0e47640 1 -- 172.21.15.120:0/3725684568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654c553a680 msgr2=0x5654c55377f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:59.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.674+0000 7fc4f0e47640 1 --2- 172.21.15.120:0/3725684568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654c553a680 0x5654c55377f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.674+0000 7fc4f0e47640 1 -- 172.21.15.120:0/3725684568 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654c52063c0 con 0x5654c552f4a0 2024-09-19T09:27:59.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.675+0000 7fc4f0e47640 1 --2- 172.21.15.120:0/3725684568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654c552f4a0 0x5654c5539fe0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0012d20 tx=0x7fc4e000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22a1c2f6c9872593 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:59.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.675+0000 7fc4cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:27:59.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.675+0000 7fc4cbfff640 1 -- 172.21.15.120:0/3725684568 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0012110 con 0x5654c552f4a0 2024-09-19T09:27:59.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.675+0000 7fc4cbfff640 1 -- 172.21.15.120:0/3725684568 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc4e0004050 con 0x5654c552f4a0 2024-09-19T09:27:59.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.675+0000 7fc4f5484740 1 -- 172.21.15.120:0/3725684568 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4e4023070 con 0x5654c552f4a0 2024-09-19T09:27:59.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.675+0000 7fc4cbfff640 1 -- 172.21.15.120:0/3725684568 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc4e00122b0 con 0x5654c552f4a0 2024-09-19T09:27:59.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.675+0000 7fc4f5484740 10 client.31984 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:59.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.675+0000 7fc4f5484740 1 -- 172.21.15.120:0/3725684568 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5654c5548c20 con 0x5654c552f4a0 2024-09-19T09:27:59.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.675+0000 7fc4cbfff640 1 -- 172.21.15.120:0/3725684568 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4e4023070 con 0x5654c552f4a0 2024-09-19T09:27:59.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.676+0000 7fc4cbfff640 1 -- 172.21.15.120:0/3725684568 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fc4e0010430 con 0x5654c552f4a0 2024-09-19T09:27:59.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.676+0000 7fc4f5484740 10 client.31984 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:59.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.676+0000 7fc4f5484740 10 client.31984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:27:59.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.676+0000 7fc4f5484740 20 client.31984 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:59.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.676+0000 7fc4f5484740 1 --2- 172.21.15.120:0/3725684568 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5654c554a470 0x5654c554c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:59.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.676+0000 7fc4f5484740 4 client.31984 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:59.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.676+0000 7fc4f5484740 1 -- 172.21.15.120:0/3725684568 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654c54a2850 con 0x5654c554a470 2024-09-19T09:27:59.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.676+0000 7fc4ebfff640 1 --2- 172.21.15.120:0/3725684568 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5654c554a470 0x5654c554c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:59.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.676+0000 7fc4ebfff640 1 --2- 172.21.15.120:0/3725684568 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5654c554a470 0x5654c554c8e0 crc :-1 s=READY pgs=613 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6263156a249b1bce server_cookie=749ba5bcdacf5f7d in_seq=0 out_seq=0 2024-09-19T09:27:59.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.676+0000 7fc4cbfff640 10 client.31984 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:59.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.679+0000 7fc4cbfff640 1 -- 172.21.15.120:0/3725684568 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5654c54a2850 con 0x5654c554a470 2024-09-19T09:27:59.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.679+0000 7fc4cbfff640 10 client.31984 handle_command_reply: tid=0 2024-09-19T09:27:59.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.680+0000 7fc4f5484740 1 client.31984 shutdown 2024-09-19T09:27:59.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.680+0000 7fc4f5484740 1 -- 172.21.15.120:0/3725684568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654c552f4a0 msgr2=0x5654c5539fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:59.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.680+0000 7fc4f5484740 1 --2- 172.21.15.120:0/3725684568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654c552f4a0 0x5654c5539fe0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0012d20 tx=0x7fc4e000f790 comp rx=0 tx=0).stop 2024-09-19T09:27:59.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.680+0000 7fc4f5484740 1 -- 172.21.15.120:0/3725684568 shutdown_connections 2024-09-19T09:27:59.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.680+0000 7fc4f5484740 1 --2- 172.21.15.120:0/3725684568 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5654c554a470 0x5654c554c8e0 crc :-1 s=READY pgs=613 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.680+0000 7fc4f5484740 1 --2- 172.21.15.120:0/3725684568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654c553a680 0x5654c55377f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.680+0000 7fc4f5484740 1 --2- 172.21.15.120:0/3725684568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654c552fdc0 0x5654c5537290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.680+0000 7fc4cbfff640 0 client.31984 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:59.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.680+0000 7fc4f5484740 1 --2- 172.21.15.120:0/3725684568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654c552f4a0 0x5654c5539fe0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.680+0000 7fc4f5484740 1 -- 172.21.15.120:0/3725684568 >> 172.21.15.120:0/3725684568 conn(0x5654c552ba30 msgr2=0x5654c5531760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:59.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.680+0000 7fc4f5484740 1 -- 172.21.15.120:0/3725684568 shutdown_connections 2024-09-19T09:27:59.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.680+0000 7fc4f5484740 1 -- 172.21.15.120:0/3725684568 wait complete. 2024-09-19T09:27:59.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.680+0000 7fc4f5484740 20 client.31984 trim_cache size 0 max 16384 2024-09-19T09:27:59.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.699+0000 7fc4f5484740 1 Processor -- start 2024-09-19T09:27:59.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.699+0000 7fc4f5484740 1 -- start start 2024-09-19T09:27:59.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.700+0000 7fc4f5484740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654c552f4a0 0x5654c5546500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:59.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.700+0000 7fc4f5484740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654c552fdc0 0x5654c5536b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:59.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.700+0000 7fc4f5484740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654c553a680 0x5654c5532760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:59.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.700+0000 7fc4f5484740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec1ab400 con 0x5654c553a680 2024-09-19T09:27:59.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.700+0000 7fc4f5484740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec1acf40 con 0x5654c552f4a0 2024-09-19T09:27:59.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.700+0000 7fc4f5484740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec1abad0 con 0x5654c552fdc0 2024-09-19T09:27:59.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.700+0000 7fc4f1648640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654c553a680 0x5654c5532760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:59.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.700+0000 7fc4f0e47640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654c552f4a0 0x5654c5546500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:59.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.700+0000 7fc4f1648640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654c553a680 0x5654c5532760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49918/0 (socket says 172.21.15.120:49918) 2024-09-19T09:27:59.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.700+0000 7fc4f1648640 1 -- 172.21.15.120:0/716055485 learned_addr learned my addr 172.21.15.120:0/716055485 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:59.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.700+0000 7fc4ebfff640 1 --2- 172.21.15.120:0/716055485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654c552fdc0 0x5654c5536b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:59.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.700+0000 7fc4f1648640 1 -- 172.21.15.120:0/716055485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654c552fdc0 msgr2=0x5654c5536b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:59.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.700+0000 7fc4f1648640 1 --2- 172.21.15.120:0/716055485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654c552fdc0 0x5654c5536b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.700+0000 7fc4f1648640 1 -- 172.21.15.120:0/716055485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654c552f4a0 msgr2=0x5654c5546500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:59.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.700+0000 7fc4f1648640 1 --2- 172.21.15.120:0/716055485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654c552f4a0 0x5654c5546500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.700+0000 7fc4f1648640 1 -- 172.21.15.120:0/716055485 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654c5537b60 con 0x5654c553a680 2024-09-19T09:27:59.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.700+0000 7fc4f1648640 1 --2- 172.21.15.120:0/716055485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654c553a680 0x5654c5532760 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4004270 tx=0x7fc4e40042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f448e407e9c7175 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:59.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.701+0000 7fc4d67fc640 1 -- 172.21.15.120:0/716055485 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e402c040 con 0x5654c553a680 2024-09-19T09:27:59.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.701+0000 7fc4d67fc640 1 -- 172.21.15.120:0/716055485 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4e4023070 con 0x5654c553a680 2024-09-19T09:27:59.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.701+0000 7fc4d67fc640 1 -- 172.21.15.120:0/716055485 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e40058c0 con 0x5654c553a680 2024-09-19T09:27:59.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.701+0000 7fc4f5484740 1 -- 172.21.15.120:0/716055485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654c553a680 msgr2=0x5654c5532760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:59.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.701+0000 7fc4f5484740 1 --2- 172.21.15.120:0/716055485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654c553a680 0x5654c5532760 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4004270 tx=0x7fc4e40042a0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.702+0000 7fc4f5484740 1 -- 172.21.15.120:0/716055485 shutdown_connections 2024-09-19T09:27:59.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.702+0000 7fc4f5484740 1 --2- 172.21.15.120:0/716055485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654c553a680 0x5654c5532760 unknown :-1 s=CLOSED pgs=2523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.702+0000 7fc4f5484740 1 --2- 172.21.15.120:0/716055485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654c552fdc0 0x5654c5536b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.702+0000 7fc4f5484740 1 --2- 172.21.15.120:0/716055485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654c552f4a0 0x5654c5546500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.702+0000 7fc4f5484740 1 -- 172.21.15.120:0/716055485 >> 172.21.15.120:0/716055485 conn(0x5654c552ba30 msgr2=0x5654c5532050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:59.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.702+0000 7fc4f5484740 1 -- 172.21.15.120:0/716055485 shutdown_connections 2024-09-19T09:27:59.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.702+0000 7fc4f5484740 1 -- 172.21.15.120:0/716055485 wait complete. 2024-09-19T09:27:59.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.702+0000 7fc4f5484740 1 -- start start 2024-09-19T09:27:59.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.702+0000 7fc4f5484740 1 Processor -- start 2024-09-19T09:27:59.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.703+0000 7fc4f5484740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654c552f4a0 0x5654c5542c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:59.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.703+0000 7fc4f5484740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654c552fdc0 0x5654c553dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:59.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.703+0000 7fc4f5484740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654c553a680 0x5654c553e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:59.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.703+0000 7fc4f5484740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec1ac870 con 0x5654c552fdc0 2024-09-19T09:27:59.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.703+0000 7fc4f5484740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec1ac1a0 con 0x5654c553a680 2024-09-19T09:27:59.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.703+0000 7fc4f5484740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec1a93c0 con 0x5654c552f4a0 2024-09-19T09:27:59.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.703+0000 7fc4f5484740 10 client.0 _ll_register_callbacks cb 0x7fc4e0009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:59.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.703+0000 7fc4f0e47640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654c552f4a0 0x5654c5542c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:59.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.703+0000 7fc4ebfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654c552fdc0 0x5654c553dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:59.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.703+0000 7fc4ebfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654c552fdc0 0x5654c553dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49928/0 (socket says 172.21.15.120:49928) 2024-09-19T09:27:59.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.703+0000 7fc4ebfff640 1 -- 172.21.15.120:0/4073545761 learned_addr learned my addr 172.21.15.120:0/4073545761 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:27:59.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.703+0000 7fc4f1648640 1 --2- 172.21.15.120:0/4073545761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654c553a680 0x5654c553e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:59.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.703+0000 7fc4ebfff640 1 -- 172.21.15.120:0/4073545761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654c552f4a0 msgr2=0x5654c5542c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:59.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.703+0000 7fc4ebfff640 1 --2- 172.21.15.120:0/4073545761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654c552f4a0 0x5654c5542c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.703+0000 7fc4ebfff640 1 -- 172.21.15.120:0/4073545761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654c553a680 msgr2=0x5654c553e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:59.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.703+0000 7fc4ebfff640 1 --2- 172.21.15.120:0/4073545761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654c553a680 0x5654c553e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.703+0000 7fc4ebfff640 1 -- 172.21.15.120:0/4073545761 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654c553ebd0 con 0x5654c552fdc0 2024-09-19T09:27:59.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.703+0000 7fc4f1648640 1 --2- 172.21.15.120:0/4073545761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654c553a680 0x5654c553e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:27:59.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.703+0000 7fc4ebfff640 1 --2- 172.21.15.120:0/4073545761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654c552fdc0 0x5654c553dfa0 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80b8c60 tx=0x7fc4d80ed590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11b70ebdcba7d24 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:59.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.703+0000 7fc4cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:27:59.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.703+0000 7fc4cbfff640 1 -- 172.21.15.120:0/4073545761 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8049040 con 0x5654c552fdc0 2024-09-19T09:27:59.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.703+0000 7fc4cbfff640 1 -- 172.21.15.120:0/4073545761 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc4d80d5180 con 0x5654c552fdc0 2024-09-19T09:27:59.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.704+0000 7fc4f5484740 1 -- 172.21.15.120:0/4073545761 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4e4023070 con 0x5654c552fdc0 2024-09-19T09:27:59.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.704+0000 7fc4cbfff640 1 -- 172.21.15.120:0/4073545761 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc4d80ba590 con 0x5654c552fdc0 2024-09-19T09:27:59.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.704+0000 7fc4f5484740 10 client.23544 fetch_fsmap learned FSMap version 39 2024-09-19T09:27:59.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.704+0000 7fc4f5484740 1 -- 172.21.15.120:0/4073545761 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5654c55bd8b0 con 0x5654c552fdc0 2024-09-19T09:27:59.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.704+0000 7fc4cbfff640 1 -- 172.21.15.120:0/4073545761 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4e4023070 con 0x5654c552fdc0 2024-09-19T09:27:59.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.704+0000 7fc4cbfff640 1 -- 172.21.15.120:0/4073545761 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fc4d80bc4f0 con 0x5654c552fdc0 2024-09-19T09:27:59.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.704+0000 7fc4f5484740 10 client.23544 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:27:59.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.704+0000 7fc4f5484740 10 client.23544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:27:59.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.704+0000 7fc4f5484740 20 client.23544 populate_metadata read hostname 'smithi120' 2024-09-19T09:27:59.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.704+0000 7fc4f5484740 1 --2- 172.21.15.120:0/4073545761 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5654c55bdfe0 0x5654c55c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:59.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.704+0000 7fc4f5484740 4 client.23544 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:27:59.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.704+0000 7fc4f5484740 1 -- 172.21.15.120:0/4073545761 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5654c549cd60 con 0x5654c55bdfe0 2024-09-19T09:27:59.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.704+0000 7fc4f0e47640 1 --2- 172.21.15.120:0/4073545761 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5654c55bdfe0 0x5654c55c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:59.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.705+0000 7fc4cbfff640 10 client.23544 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:59.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.705+0000 7fc4f0e47640 1 --2- 172.21.15.120:0/4073545761 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5654c55bdfe0 0x5654c55c0400 crc :-1 s=READY pgs=614 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7946c3704478d4aa server_cookie=8f4cb99d2f2b5cea in_seq=0 out_seq=0 2024-09-19T09:27:59.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.706+0000 7fc4cbfff640 1 -- 172.21.15.120:0/4073545761 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5654c549cd60 con 0x5654c55bdfe0 2024-09-19T09:27:59.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.706+0000 7fc4cbfff640 10 client.23544 handle_command_reply: tid=0 2024-09-19T09:27:59.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.706+0000 7fc4f5484740 1 client.23544 shutdown 2024-09-19T09:27:59.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.706+0000 7fc4f5484740 1 -- 172.21.15.120:0/4073545761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654c552fdc0 msgr2=0x5654c553dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:59.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.706+0000 7fc4f5484740 1 --2- 172.21.15.120:0/4073545761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654c552fdc0 0x5654c553dfa0 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80b8c60 tx=0x7fc4d80ed590 comp rx=0 tx=0).stop 2024-09-19T09:27:59.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.706+0000 7fc4f5484740 1 -- 172.21.15.120:0/4073545761 shutdown_connections 2024-09-19T09:27:59.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.706+0000 7fc4f5484740 1 --2- 172.21.15.120:0/4073545761 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5654c55bdfe0 0x5654c55c0400 crc :-1 s=READY pgs=614 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.707+0000 7fc4cbfff640 0 client.23544 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:27:59.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.707+0000 7fc4f5484740 1 --2- 172.21.15.120:0/4073545761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5654c553a680 0x5654c553e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.707+0000 7fc4f5484740 1 --2- 172.21.15.120:0/4073545761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5654c552fdc0 0x5654c553dfa0 unknown :-1 s=CLOSED pgs=2524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.707+0000 7fc4f5484740 1 --2- 172.21.15.120:0/4073545761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654c552f4a0 0x5654c5542c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.707+0000 7fc4f5484740 1 -- 172.21.15.120:0/4073545761 >> 172.21.15.120:0/4073545761 conn(0x5654c552ba30 msgr2=0x5654c5532870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:59.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.707+0000 7fc4f5484740 1 -- 172.21.15.120:0/4073545761 shutdown_connections 2024-09-19T09:27:59.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.707+0000 7fc4f5484740 1 -- 172.21.15.120:0/4073545761 wait complete. 2024-09-19T09:27:59.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.707+0000 7fc4f5484740 20 client.23544 trim_cache size 0 max 16384 2024-09-19T09:27:59.713 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:27:59.713 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-19T09:27:59.713 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {} 2024-09-19T09:27:59.713 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:27:59.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.707+0000 7fc4f30ab640 1 -- 172.21.15.120:0/2099072978 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc4dc0a3240 msgr2=0x7fc4dc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:59.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.708+0000 7fc4f30ab640 1 --2- 172.21.15.120:0/2099072978 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc4dc0a3240 0x7fc4dc0a5700 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400fc90 tx=0x7fc4e4018000 comp rx=0 tx=0).stop 2024-09-19T09:27:59.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.708+0000 7fc4f30ab640 1 -- 172.21.15.120:0/2099072978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4ec1036e0 msgr2=0x7fc4ec198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:59.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.708+0000 7fc4f30ab640 1 --2- 172.21.15.120:0/2099072978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4ec1036e0 0x7fc4ec198b20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8005940 tx=0x7fc4d8031b50 comp rx=0 tx=0).stop 2024-09-19T09:27:59.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.708+0000 7fc4f30ab640 1 -- 172.21.15.120:0/2099072978 shutdown_connections 2024-09-19T09:27:59.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.708+0000 7fc4f30ab640 1 --2- 172.21.15.120:0/2099072978 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc4dc0a3240 0x7fc4dc0a5700 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.708+0000 7fc4f30ab640 1 --2- 172.21.15.120:0/2099072978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4ec1082a0 0x7fc4ec19ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.708+0000 7fc4f30ab640 1 --2- 172.21.15.120:0/2099072978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4ec1036e0 0x7fc4ec198b20 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.708+0000 7fc4f30ab640 1 --2- 172.21.15.120:0/2099072978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4ec102d30 0x7fc4ec1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:59.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.708+0000 7fc4f30ab640 1 -- 172.21.15.120:0/2099072978 >> 172.21.15.120:0/2099072978 conn(0x7fc4ec0fc7d0 msgr2=0x7fc4ec105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:59.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.708+0000 7fc4f30ab640 1 -- 172.21.15.120:0/2099072978 shutdown_connections 2024-09-19T09:27:59.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:27:59.708+0000 7fc4f30ab640 1 -- 172.21.15.120:0/2099072978 wait complete. 2024-09-19T09:28:00.396 INFO:tasks.workunit.client.0.smithi120.stdout: 10 274155 19.82 MB/sec execute 595 sec latency 89.039 ms 2024-09-19T09:28:00.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:00 smithi196 ceph-mon[35150]: pgmap v1099: 97 pgs: 97 active+clean; 465 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 1.6 MiB/s wr, 211 op/s 2024-09-19T09:28:00.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:00 smithi153 ceph-mon[35071]: pgmap v1099: 97 pgs: 97 active+clean; 465 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 1.6 MiB/s wr, 211 op/s 2024-09-19T09:28:00.560 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:00 smithi120 ceph-mon[31150]: pgmap v1099: 97 pgs: 97 active+clean; 465 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 1.6 MiB/s wr, 211 op/s 2024-09-19T09:28:00.592 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:28:00.593 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:28:00.947 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:01.397 INFO:tasks.workunit.client.0.smithi120.stdout: 10 274656 19.83 MB/sec execute 596 sec latency 68.631 ms 2024-09-19T09:28:01.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.459+0000 7fc3e4b79640 1 -- 172.21.15.120:0/1584831322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3e0103700 msgr2=0x7fc3e010bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:01.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.459+0000 7fc3e4b79640 1 --2- 172.21.15.120:0/1584831322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3e0103700 0x7fc3e010bbf0 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4009900 tx=0x7fc3d402a480 comp rx=0 tx=0).stop 2024-09-19T09:28:01.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.459+0000 7fc3e4b79640 1 -- 172.21.15.120:0/1584831322 shutdown_connections 2024-09-19T09:28:01.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.459+0000 7fc3e4b79640 1 --2- 172.21.15.120:0/1584831322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3e010c180 0x7fc3e010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.459+0000 7fc3e4b79640 1 --2- 172.21.15.120:0/1584831322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3e0103700 0x7fc3e010bbf0 unknown :-1 s=CLOSED pgs=2525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.459+0000 7fc3e4b79640 1 --2- 172.21.15.120:0/1584831322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3e0102de0 0x7fc3e01031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.459+0000 7fc3e4b79640 1 -- 172.21.15.120:0/1584831322 >> 172.21.15.120:0/1584831322 conn(0x7fc3e00fc7a0 msgr2=0x7fc3e00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:01.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.459+0000 7fc3e4b79640 1 -- 172.21.15.120:0/1584831322 shutdown_connections 2024-09-19T09:28:01.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.459+0000 7fc3e4b79640 1 -- 172.21.15.120:0/1584831322 wait complete. 2024-09-19T09:28:01.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.460+0000 7fc3e4b79640 1 Processor -- start 2024-09-19T09:28:01.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.460+0000 7fc3e4b79640 1 -- start start 2024-09-19T09:28:01.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.460+0000 7fc3e4b79640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3e0102de0 0x7fc3e0100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:01.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.460+0000 7fc3e4b79640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3e0103700 0x7fc3e0100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:01.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.460+0000 7fc3e4b79640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3e010c180 0x7fc3e01014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:01.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.460+0000 7fc3e4b79640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e0111f70 con 0x7fc3e0103700 2024-09-19T09:28:01.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.460+0000 7fc3e4b79640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e0111df0 con 0x7fc3e010c180 2024-09-19T09:28:01.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.460+0000 7fc3e4b79640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e01120f0 con 0x7fc3e0102de0 2024-09-19T09:28:01.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.460+0000 7fc3ddd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3e0103700 0x7fc3e0100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:01.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.460+0000 7fc3ddd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3e0103700 0x7fc3e0100f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43130/0 (socket says 172.21.15.120:43130) 2024-09-19T09:28:01.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.460+0000 7fc3ddd9b640 1 -- 172.21.15.120:0/849014534 learned_addr learned my addr 172.21.15.120:0/849014534 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:01.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.460+0000 7fc3de59c640 1 --2- 172.21.15.120:0/849014534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3e0102de0 0x7fc3e0100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:01.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.460+0000 7fc3ded9d640 1 --2- 172.21.15.120:0/849014534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3e010c180 0x7fc3e01014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:01.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.460+0000 7fc3ddd9b640 1 -- 172.21.15.120:0/849014534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3e0102de0 msgr2=0x7fc3e0100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:01.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.460+0000 7fc3ddd9b640 1 --2- 172.21.15.120:0/849014534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3e0102de0 0x7fc3e0100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.461+0000 7fc3ddd9b640 1 -- 172.21.15.120:0/849014534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3e010c180 msgr2=0x7fc3e01014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:01.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.461+0000 7fc3ddd9b640 1 --2- 172.21.15.120:0/849014534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3e010c180 0x7fc3e01014c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.461+0000 7fc3ddd9b640 1 -- 172.21.15.120:0/849014534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e01a68f0 con 0x7fc3e0103700 2024-09-19T09:28:01.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.461+0000 7fc3ded9d640 1 --2- 172.21.15.120:0/849014534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3e010c180 0x7fc3e01014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:01.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.461+0000 7fc3de59c640 1 --2- 172.21.15.120:0/849014534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3e0102de0 0x7fc3e0100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:01.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.461+0000 7fc3ddd9b640 1 --2- 172.21.15.120:0/849014534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3e0103700 0x7fc3e0100f80 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4006010 tx=0x7fc3d4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:01.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.461+0000 7fc3cf7fe640 1 -- 172.21.15.120:0/849014534 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d403f030 con 0x7fc3e0103700 2024-09-19T09:28:01.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.461+0000 7fc3e4b79640 1 -- 172.21.15.120:0/849014534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e01a6b80 con 0x7fc3e0103700 2024-09-19T09:28:01.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.461+0000 7fc3cf7fe640 1 -- 172.21.15.120:0/849014534 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3d4004440 con 0x7fc3e0103700 2024-09-19T09:28:01.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.461+0000 7fc3cf7fe640 1 -- 172.21.15.120:0/849014534 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4035070 con 0x7fc3e0103700 2024-09-19T09:28:01.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.461+0000 7fc3e4b79640 1 -- 172.21.15.120:0/849014534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e01a70b0 con 0x7fc3e0103700 2024-09-19T09:28:01.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.462+0000 7fc3e4b79640 1 -- 172.21.15.120:0/849014534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e01031c0 con 0x7fc3e0103700 2024-09-19T09:28:01.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.464+0000 7fc3cf7fe640 1 -- 172.21.15.120:0/849014534 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fc3d4042030 con 0x7fc3e0103700 2024-09-19T09:28:01.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.465+0000 7fc3cf7fe640 1 --2- 172.21.15.120:0/849014534 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc3c80a32c0 0x7fc3c80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:01.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.465+0000 7fc3de59c640 1 --2- 172.21.15.120:0/849014534 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc3c80a32c0 0x7fc3c80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:01.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.465+0000 7fc3cf7fe640 1 -- 172.21.15.120:0/849014534 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc3d40efc90 con 0x7fc3e0103700 2024-09-19T09:28:01.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.465+0000 7fc3de59c640 1 --2- 172.21.15.120:0/849014534 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc3c80a32c0 0x7fc3c80a5780 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0009fc0 tx=0x7fc3d0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:01.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.469+0000 7fc3cf7fe640 1 -- 172.21.15.120:0/849014534 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc3d40bc320 con 0x7fc3e0103700 2024-09-19T09:28:01.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.751+0000 7fc3e4b79640 1 -- 172.21.15.120:0/849014534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3e0107200 con 0x7fc3e0103700 2024-09-19T09:28:01.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.752+0000 7fc3cf7fe640 1 -- 172.21.15.120:0/849014534 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11988 (secure 0 0 0) 0x7fc3d40c0620 con 0x7fc3e0103700 2024-09-19T09:28:01.754 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:28:01.754 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":39,"btime":"2024-09-19T09:18:53:120858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:18:53.070639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24461,24446,24442,24430,24421]},"id":1}]} 2024-09-19T09:28:01.755 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 39 2024-09-19T09:28:01.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.754+0000 7fc3e4b79640 1 -- 172.21.15.120:0/849014534 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc3c80a32c0 msgr2=0x7fc3c80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:01.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.754+0000 7fc3e4b79640 1 --2- 172.21.15.120:0/849014534 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc3c80a32c0 0x7fc3c80a5780 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0009fc0 tx=0x7fc3d0005f90 comp rx=0 tx=0).stop 2024-09-19T09:28:01.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.754+0000 7fc3e4b79640 1 -- 172.21.15.120:0/849014534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3e0103700 msgr2=0x7fc3e0100f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:01.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.754+0000 7fc3e4b79640 1 --2- 172.21.15.120:0/849014534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3e0103700 0x7fc3e0100f80 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4006010 tx=0x7fc3d4004040 comp rx=0 tx=0).stop 2024-09-19T09:28:01.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.754+0000 7fc3e4b79640 1 -- 172.21.15.120:0/849014534 shutdown_connections 2024-09-19T09:28:01.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.754+0000 7fc3e4b79640 1 --2- 172.21.15.120:0/849014534 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fc3c80a32c0 0x7fc3c80a5780 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.754+0000 7fc3e4b79640 1 --2- 172.21.15.120:0/849014534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3e010c180 0x7fc3e01014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.754+0000 7fc3e4b79640 1 --2- 172.21.15.120:0/849014534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3e0103700 0x7fc3e0100f80 unknown :-1 s=CLOSED pgs=2526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.754+0000 7fc3e4b79640 1 --2- 172.21.15.120:0/849014534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3e0102de0 0x7fc3e0100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.754+0000 7fc3e4b79640 1 -- 172.21.15.120:0/849014534 >> 172.21.15.120:0/849014534 conn(0x7fc3e00fc7a0 msgr2=0x7fc3e00fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:01.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.755+0000 7fc3e4b79640 1 -- 172.21.15.120:0/849014534 shutdown_connections 2024-09-19T09:28:01.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:01.755+0000 7fc3e4b79640 1 -- 172.21.15.120:0/849014534 wait complete. 2024-09-19T09:28:02.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:01 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/849014534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:28:02.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:01 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/849014534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:28:02.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:01 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/849014534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:28:02.396 INFO:tasks.workunit.client.0.smithi120.stdout: 10 275303 19.84 MB/sec execute 597 sec latency 50.600 ms 2024-09-19T09:28:02.562 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:28:02.812 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:02 smithi120 ceph-mon[31150]: pgmap v1100: 97 pgs: 97 active+clean; 474 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 2.5 MiB/s wr, 328 op/s 2024-09-19T09:28:02.884 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:03.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:02 smithi196 ceph-mon[35150]: pgmap v1100: 97 pgs: 97 active+clean; 474 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 2.5 MiB/s wr, 328 op/s 2024-09-19T09:28:03.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:02 smithi153 ceph-mon[35071]: pgmap v1100: 97 pgs: 97 active+clean; 474 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 2.5 MiB/s wr, 328 op/s 2024-09-19T09:28:03.397 INFO:tasks.workunit.client.0.smithi120.stdout: 10 275815 19.85 MB/sec execute 598 sec latency 75.641 ms 2024-09-19T09:28:03.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.420+0000 7f653400f640 1 -- 172.21.15.120:0/2531218911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f652c103670 msgr2=0x7f652c10bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:03.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.420+0000 7f653400f640 1 --2- 172.21.15.120:0/2531218911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f652c103670 0x7f652c10bb60 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7f651c009900 tx=0x7f651c02a480 comp rx=0 tx=0).stop 2024-09-19T09:28:03.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.420+0000 7f653400f640 1 -- 172.21.15.120:0/2531218911 shutdown_connections 2024-09-19T09:28:03.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.420+0000 7f653400f640 1 --2- 172.21.15.120:0/2531218911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f652c10c180 0x7f652c10e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.420+0000 7f653400f640 1 --2- 172.21.15.120:0/2531218911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f652c103670 0x7f652c10bb60 unknown :-1 s=CLOSED pgs=2527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.420+0000 7f653400f640 1 --2- 172.21.15.120:0/2531218911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f652c102d50 0x7f652c103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.420+0000 7f653400f640 1 -- 172.21.15.120:0/2531218911 >> 172.21.15.120:0/2531218911 conn(0x7f652c0fc7d0 msgr2=0x7f652c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:03.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.420+0000 7f653400f640 1 -- 172.21.15.120:0/2531218911 shutdown_connections 2024-09-19T09:28:03.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.421+0000 7f653400f640 1 -- 172.21.15.120:0/2531218911 wait complete. 2024-09-19T09:28:03.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.421+0000 7f653400f640 1 Processor -- start 2024-09-19T09:28:03.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.422+0000 7f653400f640 1 -- start start 2024-09-19T09:28:03.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.422+0000 7f653400f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f652c102d50 0x7f652c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:03.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.422+0000 7f653400f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f652c103670 0x7f652c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:03.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.422+0000 7f653400f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f652c10c180 0x7f652c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:03.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.422+0000 7f653400f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f652c111fd0 con 0x7f652c102d50 2024-09-19T09:28:03.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.422+0000 7f653400f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f652c111e50 con 0x7f652c10c180 2024-09-19T09:28:03.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.422+0000 7f653400f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f652c112150 con 0x7f652c103670 2024-09-19T09:28:03.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.422+0000 7f6531dab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f652c102d50 0x7f652c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:03.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.422+0000 7f6531dab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f652c102d50 0x7f652c198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43146/0 (socket says 172.21.15.120:43146) 2024-09-19T09:28:03.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.422+0000 7f6531dab640 1 -- 172.21.15.120:0/3131538626 learned_addr learned my addr 172.21.15.120:0/3131538626 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:03.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.422+0000 7f65315aa640 1 --2- 172.21.15.120:0/3131538626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f652c103670 0x7f652c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:03.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.422+0000 7f6531dab640 1 -- 172.21.15.120:0/3131538626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f652c103670 msgr2=0x7f652c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:03.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.422+0000 7f6531dab640 1 --2- 172.21.15.120:0/3131538626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f652c103670 0x7f652c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.422+0000 7f6531dab640 1 -- 172.21.15.120:0/3131538626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f652c10c180 msgr2=0x7f652c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:03.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.422+0000 7f6531dab640 1 --2- 172.21.15.120:0/3131538626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f652c10c180 0x7f652c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.422+0000 7f6531dab640 1 -- 172.21.15.120:0/3131538626 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652c19d440 con 0x7f652c102d50 2024-09-19T09:28:03.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.422+0000 7f6531dab640 1 --2- 172.21.15.120:0/3131538626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f652c102d50 0x7f652c198490 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f652800e9f0 tx=0x7f652800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:03.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.423+0000 7f6522ffd640 1 -- 172.21.15.120:0/3131538626 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652800cd80 con 0x7f652c102d50 2024-09-19T09:28:03.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.423+0000 7f653400f640 1 -- 172.21.15.120:0/3131538626 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f652c19d730 con 0x7f652c102d50 2024-09-19T09:28:03.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.423+0000 7f6522ffd640 1 -- 172.21.15.120:0/3131538626 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6528010910 con 0x7f652c102d50 2024-09-19T09:28:03.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.423+0000 7f6522ffd640 1 -- 172.21.15.120:0/3131538626 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528018990 con 0x7f652c102d50 2024-09-19T09:28:03.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.423+0000 7f653400f640 1 -- 172.21.15.120:0/3131538626 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f652c1a5010 con 0x7f652c102d50 2024-09-19T09:28:03.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.425+0000 7f6522ffd640 1 -- 172.21.15.120:0/3131538626 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f6528018b30 con 0x7f652c102d50 2024-09-19T09:28:03.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.425+0000 7f65363e8740 1 Processor -- start 2024-09-19T09:28:03.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.426+0000 7f65363e8740 1 -- start start 2024-09-19T09:28:03.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.426+0000 7f65363e8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6a4708b40 0x55b6a4707090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:03.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.426+0000 7f65363e8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6a47075d0 0x55b6a4707a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:03.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.426+0000 7f65363e8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6a4716780 0x55b6a4718b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:03.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.426+0000 7f65363e8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b6a466d2f0 con 0x55b6a4716780 2024-09-19T09:28:03.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.426+0000 7f65363e8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b6a4614cf0 con 0x55b6a4708b40 2024-09-19T09:28:03.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.426+0000 7f65363e8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b6a46692b0 con 0x55b6a47075d0 2024-09-19T09:28:03.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.426+0000 7f65315aa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6a4716780 0x55b6a4718b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:03.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.426+0000 7f65325ac640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6a4708b40 0x55b6a4707090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:03.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.426+0000 7f65325ac640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6a4708b40 0x55b6a4707090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:60508/0 (socket says 172.21.15.120:60508) 2024-09-19T09:28:03.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.426+0000 7f6531dab640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6a47075d0 0x55b6a4707a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:03.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.426+0000 7f65325ac640 1 -- 172.21.15.120:0/1650923400 learned_addr learned my addr 172.21.15.120:0/1650923400 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:03.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.426+0000 7f65325ac640 1 -- 172.21.15.120:0/1650923400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6a47075d0 msgr2=0x55b6a4707a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:03.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.426+0000 7f65325ac640 1 --2- 172.21.15.120:0/1650923400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6a47075d0 0x55b6a4707a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.426+0000 7f65325ac640 1 -- 172.21.15.120:0/1650923400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6a4716780 msgr2=0x55b6a4718b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:03.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.426+0000 7f65325ac640 1 --2- 172.21.15.120:0/1650923400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6a4716780 0x55b6a4718b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.427+0000 7f65325ac640 1 -- 172.21.15.120:0/1650923400 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6a45f93a0 con 0x55b6a4708b40 2024-09-19T09:28:03.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.427+0000 7f6522ffd640 1 --2- 172.21.15.120:0/3131538626 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f65180a32c0 0x7f65180a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:03.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.427+0000 7f6531dab640 1 --2- 172.21.15.120:0/3131538626 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f65180a32c0 0x7f65180a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:03.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.427+0000 7f6522ffd640 1 -- 172.21.15.120:0/3131538626 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f65280cfc70 con 0x7f652c102d50 2024-09-19T09:28:03.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.427+0000 7f65325ac640 1 --2- 172.21.15.120:0/1650923400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6a4708b40 0x55b6a4707090 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6524009e20 tx=0x7f652400e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb72be67e69d0f87 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:03.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.427+0000 7f6517fff640 1 -- 172.21.15.120:0/1650923400 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6524004d20 con 0x55b6a4708b40 2024-09-19T09:28:03.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.427+0000 7f6517fff640 1 -- 172.21.15.120:0/1650923400 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f652400ee70 con 0x55b6a4708b40 2024-09-19T09:28:03.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.427+0000 7f6517fff640 1 -- 172.21.15.120:0/1650923400 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65240108a0 con 0x55b6a4708b40 2024-09-19T09:28:03.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.428+0000 7f65363e8740 1 -- 172.21.15.120:0/1650923400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6a4708b40 msgr2=0x55b6a4707090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:03.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.428+0000 7f65363e8740 1 --2- 172.21.15.120:0/1650923400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6a4708b40 0x55b6a4707090 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6524009e20 tx=0x7f652400e720 comp rx=0 tx=0).stop 2024-09-19T09:28:03.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.428+0000 7f6531dab640 1 --2- 172.21.15.120:0/3131538626 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f65180a32c0 0x7f65180a5780 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f652809bf90 tx=0x7f65280d36f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:03.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.428+0000 7f65363e8740 1 -- 172.21.15.120:0/1650923400 shutdown_connections 2024-09-19T09:28:03.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.428+0000 7f65363e8740 1 --2- 172.21.15.120:0/1650923400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6a4716780 0x55b6a4718b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.428+0000 7f65363e8740 1 --2- 172.21.15.120:0/1650923400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6a47075d0 0x55b6a4707a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.428+0000 7f65363e8740 1 --2- 172.21.15.120:0/1650923400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6a4708b40 0x55b6a4707090 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.428+0000 7f65363e8740 1 -- 172.21.15.120:0/1650923400 >> 172.21.15.120:0/1650923400 conn(0x55b6a46d0310 msgr2=0x55b6a4703c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:03.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.428+0000 7f65363e8740 1 -- 172.21.15.120:0/1650923400 shutdown_connections 2024-09-19T09:28:03.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.428+0000 7f65363e8740 1 -- 172.21.15.120:0/1650923400 wait complete. 2024-09-19T09:28:03.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.429+0000 7f65363e8740 1 -- start start 2024-09-19T09:28:03.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.429+0000 7f65363e8740 1 Processor -- start 2024-09-19T09:28:03.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.429+0000 7f65363e8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6a47075d0 0x55b6a47161a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:03.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.429+0000 7f65363e8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6a4708b40 0x55b6a4713450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:03.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.429+0000 7f65363e8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6a4716780 0x55b6a4713a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:03.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.429+0000 7f65363e8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b6a4719480 con 0x55b6a4708b40 2024-09-19T09:28:03.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.429+0000 7f65363e8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b6a4628060 con 0x55b6a4716780 2024-09-19T09:28:03.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.429+0000 7f65363e8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b6a4719600 con 0x55b6a47075d0 2024-09-19T09:28:03.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.429+0000 7f65363e8740 10 client.0 _ll_register_callbacks cb 0x55b6a437b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:03.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.429+0000 7f65325ac640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6a47075d0 0x55b6a47161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:03.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.429+0000 7f65325ac640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6a47075d0 0x55b6a47161a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:50324/0 (socket says 172.21.15.120:50324) 2024-09-19T09:28:03.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.429+0000 7f65325ac640 1 -- 172.21.15.120:0/2191005631 learned_addr learned my addr 172.21.15.120:0/2191005631 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:03.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.429+0000 7f65325ac640 1 --2- 172.21.15.120:0/2191005631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6a4716780 0x55b6a4713a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:03.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.430+0000 7f65315aa640 1 --2- 172.21.15.120:0/2191005631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6a4708b40 0x55b6a4713450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:03.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.430+0000 7f65325ac640 1 -- 172.21.15.120:0/2191005631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6a4716780 msgr2=0x55b6a4713a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:03.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.430+0000 7f65325ac640 1 --2- 172.21.15.120:0/2191005631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6a4716780 0x55b6a4713a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.430+0000 7f65325ac640 1 -- 172.21.15.120:0/2191005631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6a4708b40 msgr2=0x55b6a4713450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:03.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.430+0000 7f65325ac640 1 --2- 172.21.15.120:0/2191005631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6a4708b40 0x55b6a4713450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.430+0000 7f65325ac640 1 -- 172.21.15.120:0/2191005631 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6a43de3c0 con 0x55b6a47075d0 2024-09-19T09:28:03.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.430+0000 7f65315aa640 1 --2- 172.21.15.120:0/2191005631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6a4708b40 0x55b6a4713450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:03.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.430+0000 7f65325ac640 1 --2- 172.21.15.120:0/2191005631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6a47075d0 0x55b6a47161a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6524009560 tx=0x7f6524010540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e5c3431b55e7650 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:03.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.430+0000 7f65157fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:28:03.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.430+0000 7f65157fa640 1 -- 172.21.15.120:0/2191005631 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6524022040 con 0x55b6a47075d0 2024-09-19T09:28:03.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.430+0000 7f65363e8740 1 -- 172.21.15.120:0/2191005631 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f652400ee70 con 0x55b6a47075d0 2024-09-19T09:28:03.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.430+0000 7f65157fa640 1 -- 172.21.15.120:0/2191005631 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f6524010c50 con 0x55b6a47075d0 2024-09-19T09:28:03.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.431+0000 7f65157fa640 1 -- 172.21.15.120:0/2191005631 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6524005000 con 0x55b6a47075d0 2024-09-19T09:28:03.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.431+0000 7f65363e8740 10 client.32008 fetch_fsmap learned FSMap version 39 2024-09-19T09:28:03.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.431+0000 7f65363e8740 1 -- 172.21.15.120:0/2191005631 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b6a470c930 con 0x55b6a47075d0 2024-09-19T09:28:03.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.431+0000 7f65157fa640 1 -- 172.21.15.120:0/2191005631 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f652400ee70 con 0x55b6a47075d0 2024-09-19T09:28:03.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.431+0000 7f65157fa640 1 -- 172.21.15.120:0/2191005631 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f6524018870 con 0x55b6a47075d0 2024-09-19T09:28:03.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.431+0000 7f65363e8740 10 client.32008 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:28:03.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.431+0000 7f65363e8740 10 client.32008 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:28:03.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.431+0000 7f65363e8740 20 client.32008 populate_metadata read hostname 'smithi120' 2024-09-19T09:28:03.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.431+0000 7f65363e8740 1 --2- 172.21.15.120:0/2191005631 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55b6a470e080 0x55b6a472d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:03.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.432+0000 7f65363e8740 4 client.32008 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:28:03.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.432+0000 7f65363e8740 1 -- 172.21.15.120:0/2191005631 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6a467a860 con 0x55b6a470e080 2024-09-19T09:28:03.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.432+0000 7f65315aa640 1 --2- 172.21.15.120:0/2191005631 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55b6a470e080 0x55b6a472d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:03.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.432+0000 7f65157fa640 10 client.32008 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:28:03.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.432+0000 7f65315aa640 1 --2- 172.21.15.120:0/2191005631 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55b6a470e080 0x55b6a472d150 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b508ab8c444d4300 server_cookie=d4ac8b5a4533024d in_seq=0 out_seq=0 2024-09-19T09:28:03.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.433+0000 7f65157fa640 1 -- 172.21.15.120:0/2191005631 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b6a467a860 con 0x55b6a470e080 2024-09-19T09:28:03.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.433+0000 7f65157fa640 10 client.32008 handle_command_reply: tid=0 2024-09-19T09:28:03.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.433+0000 7f65363e8740 1 client.32008 shutdown 2024-09-19T09:28:03.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.433+0000 7f65363e8740 1 -- 172.21.15.120:0/2191005631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6a47075d0 msgr2=0x55b6a47161a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:03.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.433+0000 7f65363e8740 1 --2- 172.21.15.120:0/2191005631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6a47075d0 0x55b6a47161a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6524009560 tx=0x7f6524010540 comp rx=0 tx=0).stop 2024-09-19T09:28:03.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.434+0000 7f65363e8740 1 -- 172.21.15.120:0/2191005631 shutdown_connections 2024-09-19T09:28:03.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.434+0000 7f65363e8740 1 --2- 172.21.15.120:0/2191005631 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55b6a470e080 0x55b6a472d150 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.434+0000 7f65157fa640 0 client.32008 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:28:03.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.434+0000 7f65363e8740 1 --2- 172.21.15.120:0/2191005631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6a4716780 0x55b6a4713a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.434+0000 7f65363e8740 1 --2- 172.21.15.120:0/2191005631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6a4708b40 0x55b6a4713450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.434+0000 7f65363e8740 1 --2- 172.21.15.120:0/2191005631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6a47075d0 0x55b6a47161a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.434+0000 7f65363e8740 1 -- 172.21.15.120:0/2191005631 >> 172.21.15.120:0/2191005631 conn(0x55b6a46d0310 msgr2=0x55b6a4709960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:03.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.434+0000 7f65363e8740 1 -- 172.21.15.120:0/2191005631 shutdown_connections 2024-09-19T09:28:03.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.434+0000 7f65363e8740 1 -- 172.21.15.120:0/2191005631 wait complete. 2024-09-19T09:28:03.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.434+0000 7f65363e8740 20 client.32008 trim_cache size 0 max 16384 2024-09-19T09:28:03.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.449+0000 7f65363e8740 1 Processor -- start 2024-09-19T09:28:03.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.449+0000 7f65363e8740 1 -- start start 2024-09-19T09:28:03.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.449+0000 7f65363e8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6a47075d0 0x55b6a470ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:03.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.449+0000 7f65363e8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6a4708b40 0x55b6a470a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:03.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.449+0000 7f65363e8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6a470e080 0x55b6a4732780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:03.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.449+0000 7f65363e8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f652c1ab4f0 con 0x55b6a470e080 2024-09-19T09:28:03.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.449+0000 7f65363e8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f652c1ad030 con 0x55b6a4708b40 2024-09-19T09:28:03.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.449+0000 7f65363e8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f652c1abbc0 con 0x55b6a47075d0 2024-09-19T09:28:03.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.449+0000 7f65325ac640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6a470e080 0x55b6a4732780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:03.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.449+0000 7f65315aa640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6a4708b40 0x55b6a470a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:03.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.449+0000 7f65325ac640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6a470e080 0x55b6a4732780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43178/0 (socket says 172.21.15.120:43178) 2024-09-19T09:28:03.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.449+0000 7f65325ac640 1 -- 172.21.15.120:0/3701478640 learned_addr learned my addr 172.21.15.120:0/3701478640 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:03.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.449+0000 7f65325ac640 1 --2- 172.21.15.120:0/3701478640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6a47075d0 0x55b6a470ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:03.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.450+0000 7f65325ac640 1 -- 172.21.15.120:0/3701478640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6a47075d0 msgr2=0x55b6a470ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:03.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.450+0000 7f65325ac640 1 --2- 172.21.15.120:0/3701478640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6a47075d0 0x55b6a470ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.450+0000 7f65325ac640 1 -- 172.21.15.120:0/3701478640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6a4708b40 msgr2=0x55b6a470a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:03.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.450+0000 7f65325ac640 1 --2- 172.21.15.120:0/3701478640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6a4708b40 0x55b6a470a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.450+0000 7f65325ac640 1 -- 172.21.15.120:0/3701478640 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6a47143f0 con 0x55b6a470e080 2024-09-19T09:28:03.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.450+0000 7f65325ac640 1 --2- 172.21.15.120:0/3701478640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6a470e080 0x55b6a4732780 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f6524004f60 tx=0x7f652400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662d203ea09095c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:03.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.450+0000 7f65177fe640 1 -- 172.21.15.120:0/3701478640 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6524010c30 con 0x55b6a470e080 2024-09-19T09:28:03.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.450+0000 7f65177fe640 1 -- 172.21.15.120:0/3701478640 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f652401d030 con 0x55b6a470e080 2024-09-19T09:28:03.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.450+0000 7f65177fe640 1 -- 172.21.15.120:0/3701478640 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65240187f0 con 0x55b6a470e080 2024-09-19T09:28:03.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.451+0000 7f65363e8740 1 -- 172.21.15.120:0/3701478640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6a470e080 msgr2=0x55b6a4732780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:03.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.451+0000 7f65363e8740 1 --2- 172.21.15.120:0/3701478640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6a470e080 0x55b6a4732780 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f6524004f60 tx=0x7f652400e720 comp rx=0 tx=0).stop 2024-09-19T09:28:03.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.451+0000 7f65363e8740 1 -- 172.21.15.120:0/3701478640 shutdown_connections 2024-09-19T09:28:03.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.451+0000 7f65363e8740 1 --2- 172.21.15.120:0/3701478640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6a470e080 0x55b6a4732780 unknown :-1 s=CLOSED pgs=2529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.451+0000 7f65363e8740 1 --2- 172.21.15.120:0/3701478640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6a4708b40 0x55b6a470a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.451+0000 7f65363e8740 1 --2- 172.21.15.120:0/3701478640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6a47075d0 0x55b6a470ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.451+0000 7f65363e8740 1 -- 172.21.15.120:0/3701478640 >> 172.21.15.120:0/3701478640 conn(0x55b6a46d0310 msgr2=0x55b6a470e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:03.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.451+0000 7f65363e8740 1 -- 172.21.15.120:0/3701478640 shutdown_connections 2024-09-19T09:28:03.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.451+0000 7f65363e8740 1 -- 172.21.15.120:0/3701478640 wait complete. 2024-09-19T09:28:03.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.451+0000 7f65363e8740 1 -- start start 2024-09-19T09:28:03.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.452+0000 7f65363e8740 1 Processor -- start 2024-09-19T09:28:03.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.452+0000 7f65363e8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6a47075d0 0x55b6a4719850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:03.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.452+0000 7f65363e8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6a4708b40 0x55b6a4719d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:03.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.452+0000 7f65363e8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6a470e080 0x55b6a471a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:03.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.452+0000 7f65363e8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f652c1ac960 con 0x55b6a47075d0 2024-09-19T09:28:03.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.452+0000 7f65363e8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f652c1ac290 con 0x55b6a4708b40 2024-09-19T09:28:03.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.452+0000 7f65363e8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f652c1a94b0 con 0x55b6a470e080 2024-09-19T09:28:03.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.452+0000 7f65363e8740 10 client.0 _ll_register_callbacks cb 0x7f6528005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:03.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.452+0000 7f65325ac640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6a47075d0 0x55b6a4719850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:03.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.452+0000 7f65325ac640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6a47075d0 0x55b6a4719850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43192/0 (socket says 172.21.15.120:43192) 2024-09-19T09:28:03.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.452+0000 7f65325ac640 1 -- 172.21.15.120:0/1807325412 learned_addr learned my addr 172.21.15.120:0/1807325412 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:03.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.452+0000 7f65315aa640 1 --2- 172.21.15.120:0/1807325412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6a4708b40 0x55b6a4719d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:03.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.452+0000 7f65325ac640 1 --2- 172.21.15.120:0/1807325412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6a470e080 0x55b6a471a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:03.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.453+0000 7f65315aa640 1 -- 172.21.15.120:0/1807325412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6a470e080 msgr2=0x55b6a471a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:03.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.453+0000 7f65315aa640 1 --2- 172.21.15.120:0/1807325412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6a470e080 0x55b6a471a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.453+0000 7f65315aa640 1 -- 172.21.15.120:0/1807325412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6a47075d0 msgr2=0x55b6a4719850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:03.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.453+0000 7f65315aa640 1 --2- 172.21.15.120:0/1807325412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6a47075d0 0x55b6a4719850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.453+0000 7f65315aa640 1 -- 172.21.15.120:0/1807325412 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6a4793790 con 0x55b6a4708b40 2024-09-19T09:28:03.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.453+0000 7f65325ac640 1 --2- 172.21.15.120:0/1807325412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6a470e080 0x55b6a471a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:03.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.453+0000 7f65315aa640 1 --2- 172.21.15.120:0/1807325412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6a4708b40 0x55b6a4719d90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f651c02aa70 tx=0x7f651c0095d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f5bf0c5fcca433c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:03.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.453+0000 7f65157fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-19T09:28:03.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.453+0000 7f65157fa640 1 -- 172.21.15.120:0/1807325412 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c04a070 con 0x55b6a4708b40 2024-09-19T09:28:03.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.453+0000 7f65363e8740 1 -- 172.21.15.120:0/1807325412 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f652401d030 con 0x55b6a4708b40 2024-09-19T09:28:03.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.453+0000 7f65157fa640 1 -- 172.21.15.120:0/1807325412 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f651c005450 con 0x55b6a4708b40 2024-09-19T09:28:03.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.453+0000 7f65157fa640 1 -- 172.21.15.120:0/1807325412 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f651c056050 con 0x55b6a4708b40 2024-09-19T09:28:03.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.453+0000 7f65363e8740 10 client.32144 fetch_fsmap learned FSMap version 39 2024-09-19T09:28:03.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.453+0000 7f65363e8740 1 -- 172.21.15.120:0/1807325412 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b6a4794a70 con 0x55b6a4708b40 2024-09-19T09:28:03.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.454+0000 7f65157fa640 1 -- 172.21.15.120:0/1807325412 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f652401d030 con 0x55b6a4708b40 2024-09-19T09:28:03.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.454+0000 7f65157fa640 1 -- 172.21.15.120:0/1807325412 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f651c0317f0 con 0x55b6a4708b40 2024-09-19T09:28:03.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.454+0000 7f65363e8740 10 client.32144 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:28:03.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.454+0000 7f65363e8740 10 client.32144 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T09:28:03.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.454+0000 7f65363e8740 20 client.32144 populate_metadata read hostname 'smithi120' 2024-09-19T09:28:03.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.454+0000 7f65363e8740 1 --2- 172.21.15.120:0/1807325412 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55b6a4716780 0x55b6a47971e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:03.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.454+0000 7f65363e8740 4 client.32144 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:28:03.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.454+0000 7f65363e8740 1 -- 172.21.15.120:0/1807325412 --> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b6a4674d70 con 0x55b6a4716780 2024-09-19T09:28:03.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.454+0000 7f65325ac640 1 --2- 172.21.15.120:0/1807325412 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55b6a4716780 0x55b6a47971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:03.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.454+0000 7f65157fa640 10 client.32144 ms_handle_connect on v2:172.21.15.196:6834/2141168828 2024-09-19T09:28:03.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.455+0000 7f65325ac640 1 --2- 172.21.15.120:0/1807325412 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55b6a4716780 0x55b6a47971e0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f4cadb5610e43617 server_cookie=85dd64577c5a1ca6 in_seq=0 out_seq=0 2024-09-19T09:28:03.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.455+0000 7f65157fa640 1 -- 172.21.15.120:0/1807325412 <== mds.4 v2:172.21.15.196:6834/2141168828 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b6a4674d70 con 0x55b6a4716780 2024-09-19T09:28:03.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.455+0000 7f65157fa640 10 client.32144 handle_command_reply: tid=0 2024-09-19T09:28:03.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.455+0000 7f65363e8740 1 client.32144 shutdown 2024-09-19T09:28:03.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.455+0000 7f65363e8740 1 -- 172.21.15.120:0/1807325412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6a4708b40 msgr2=0x55b6a4719d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:03.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.455+0000 7f65363e8740 1 --2- 172.21.15.120:0/1807325412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6a4708b40 0x55b6a4719d90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f651c02aa70 tx=0x7f651c0095d0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.455+0000 7f65363e8740 1 -- 172.21.15.120:0/1807325412 shutdown_connections 2024-09-19T09:28:03.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.455+0000 7f65363e8740 1 --2- 172.21.15.120:0/1807325412 >> [v2:172.21.15.196:6834/2141168828,v1:172.21.15.196:6835/2141168828] conn(0x55b6a4716780 0x55b6a47971e0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.455+0000 7f65363e8740 1 --2- 172.21.15.120:0/1807325412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6a470e080 0x55b6a471a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.455+0000 7f65363e8740 1 --2- 172.21.15.120:0/1807325412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6a4708b40 0x55b6a4719d90 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.455+0000 7f65363e8740 1 --2- 172.21.15.120:0/1807325412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6a47075d0 0x55b6a4719850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.455+0000 7f65157fa640 0 client.32144 ms_handle_reset on v2:172.21.15.196:6834/2141168828 2024-09-19T09:28:03.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.455+0000 7f65363e8740 1 -- 172.21.15.120:0/1807325412 >> 172.21.15.120:0/1807325412 conn(0x55b6a46d0310 msgr2=0x55b6a47024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:03.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.455+0000 7f65363e8740 1 -- 172.21.15.120:0/1807325412 shutdown_connections 2024-09-19T09:28:03.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.455+0000 7f65363e8740 1 -- 172.21.15.120:0/1807325412 wait complete. 2024-09-19T09:28:03.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.455+0000 7f65363e8740 20 client.32144 trim_cache size 0 max 16384 2024-09-19T09:28:03.481 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:28:03.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.456+0000 7f653400f640 1 -- 172.21.15.120:0/3131538626 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f65180a32c0 msgr2=0x7f65180a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:03.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.456+0000 7f653400f640 1 --2- 172.21.15.120:0/3131538626 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f65180a32c0 0x7f65180a5780 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f652809bf90 tx=0x7f65280d36f0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.456+0000 7f653400f640 1 -- 172.21.15.120:0/3131538626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f652c102d50 msgr2=0x7f652c198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:03.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.456+0000 7f653400f640 1 --2- 172.21.15.120:0/3131538626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f652c102d50 0x7f652c198490 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f652800e9f0 tx=0x7f652800eeb0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.456+0000 7f653400f640 1 -- 172.21.15.120:0/3131538626 shutdown_connections 2024-09-19T09:28:03.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.456+0000 7f653400f640 1 --2- 172.21.15.120:0/3131538626 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f65180a32c0 0x7f65180a5780 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.456+0000 7f653400f640 1 --2- 172.21.15.120:0/3131538626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f652c10c180 0x7f652c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.456+0000 7f653400f640 1 --2- 172.21.15.120:0/3131538626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f652c103670 0x7f652c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.456+0000 7f653400f640 1 --2- 172.21.15.120:0/3131538626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f652c102d50 0x7f652c198490 unknown :-1 s=CLOSED pgs=2528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.456+0000 7f653400f640 1 -- 172.21.15.120:0/3131538626 >> 172.21.15.120:0/3131538626 conn(0x7f652c0fc7d0 msgr2=0x7f652c0fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:03.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.456+0000 7f653400f640 1 -- 172.21.15.120:0/3131538626 shutdown_connections 2024-09-19T09:28:03.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:03.456+0000 7f653400f640 1 -- 172.21.15.120:0/3131538626 wait complete. 2024-09-19T09:28:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.980+0000 7fbd2ea2e640 1 -- 172.21.15.120:0/721953532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd28100f70 msgr2=0x7fbd28101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.980+0000 7fbd2ea2e640 1 --2- 172.21.15.120:0/721953532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd28100f70 0x7fbd28101370 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7fbd18009900 tx=0x7fbd1802a480 comp rx=0 tx=0).stop 2024-09-19T09:28:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.980+0000 7fbd2ea2e640 1 -- 172.21.15.120:0/721953532 shutdown_connections 2024-09-19T09:28:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.980+0000 7fbd2ea2e640 1 --2- 172.21.15.120:0/721953532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd28102640 0x7fbd28108ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.980+0000 7fbd2ea2e640 1 --2- 172.21.15.120:0/721953532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd28101c80 0x7fbd28102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.980+0000 7fbd2ea2e640 1 --2- 172.21.15.120:0/721953532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd28100f70 0x7fbd28101370 unknown :-1 s=CLOSED pgs=2530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.980+0000 7fbd2ea2e640 1 -- 172.21.15.120:0/721953532 >> 172.21.15.120:0/721953532 conn(0x7fbd280fb970 msgr2=0x7fbd280fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.980+0000 7fbd2ea2e640 1 -- 172.21.15.120:0/721953532 shutdown_connections 2024-09-19T09:28:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.980+0000 7fbd2ea2e640 1 -- 172.21.15.120:0/721953532 wait complete. 2024-09-19T09:28:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.981+0000 7fbd2ea2e640 1 Processor -- start 2024-09-19T09:28:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.981+0000 7fbd2ea2e640 1 -- start start 2024-09-19T09:28:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.981+0000 7fbd2ea2e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd28100f70 0x7fbd28197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.981+0000 7fbd2ea2e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd28101c80 0x7fbd28198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.981+0000 7fbd2ea2e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd28102640 0x7fbd2819f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.981+0000 7fbd2ea2e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2810c6f0 con 0x7fbd28101c80 2024-09-19T09:28:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.981+0000 7fbd2ea2e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2810c570 con 0x7fbd28102640 2024-09-19T09:28:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.981+0000 7fbd2ea2e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2810c870 con 0x7fbd28100f70 2024-09-19T09:28:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.981+0000 7fbd1ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd28101c80 0x7fbd28198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.981+0000 7fbd1ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd28101c80 0x7fbd28198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43206/0 (socket says 172.21.15.120:43206) 2024-09-19T09:28:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.981+0000 7fbd1ffff640 1 -- 172.21.15.120:0/2123202003 learned_addr learned my addr 172.21.15.120:0/2123202003 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.981+0000 7fbd27fff640 1 --2- 172.21.15.120:0/2123202003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd28100f70 0x7fbd28197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.981+0000 7fbd2cfcb640 1 --2- 172.21.15.120:0/2123202003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd28102640 0x7fbd2819f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.981+0000 7fbd1ffff640 1 -- 172.21.15.120:0/2123202003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd28100f70 msgr2=0x7fbd28197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.981+0000 7fbd1ffff640 1 --2- 172.21.15.120:0/2123202003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd28100f70 0x7fbd28197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.981+0000 7fbd1ffff640 1 -- 172.21.15.120:0/2123202003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd28102640 msgr2=0x7fbd2819f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.981+0000 7fbd1ffff640 1 --2- 172.21.15.120:0/2123202003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd28102640 0x7fbd2819f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.981+0000 7fbd1ffff640 1 -- 172.21.15.120:0/2123202003 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd2819f9f0 con 0x7fbd28101c80 2024-09-19T09:28:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.982+0000 7fbd1ffff640 1 --2- 172.21.15.120:0/2123202003 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd28101c80 0x7fbd28198310 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7fbd1000e9f0 tx=0x7fbd1000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.982+0000 7fbd25ffb640 1 -- 172.21.15.120:0/2123202003 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1000cd80 con 0x7fbd28101c80 2024-09-19T09:28:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.982+0000 7fbd25ffb640 1 -- 172.21.15.120:0/2123202003 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd10005500 con 0x7fbd28101c80 2024-09-19T09:28:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.982+0000 7fbd2ea2e640 1 -- 172.21.15.120:0/2123202003 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd2819fce0 con 0x7fbd28101c80 2024-09-19T09:28:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.982+0000 7fbd25ffb640 1 -- 172.21.15.120:0/2123202003 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd10010690 con 0x7fbd28101c80 2024-09-19T09:28:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.983+0000 7fbd2ea2e640 1 -- 172.21.15.120:0/2123202003 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd281a01d0 con 0x7fbd28101c80 2024-09-19T09:28:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.985+0000 7fbd25ffb640 1 -- 172.21.15.120:0/2123202003 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fbd100046c0 con 0x7fbd28101c80 2024-09-19T09:28:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.986+0000 7fbd25ffb640 1 --2- 172.21.15.120:0/2123202003 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbd140a3370 0x7fbd140a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.986+0000 7fbd317cd740 1 Processor -- start 2024-09-19T09:28:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.986+0000 7fbd27fff640 1 --2- 172.21.15.120:0/2123202003 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbd140a3370 0x7fbd140a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.986+0000 7fbd25ffb640 1 -- 172.21.15.120:0/2123202003 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fbd100cfcf0 con 0x7fbd28101c80 2024-09-19T09:28:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.986+0000 7fbd317cd740 1 -- start start 2024-09-19T09:28:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.986+0000 7fbd317cd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2c9d649a0 0x55c2c9d64da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.986+0000 7fbd317cd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2c9d652e0 0x55c2c9d69da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.986+0000 7fbd317cd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2c9d6a370 0x55c2c9d6c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.986+0000 7fbd317cd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c2c9cc82f0 con 0x55c2c9d6a370 2024-09-19T09:28:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.986+0000 7fbd317cd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c2c9c70270 con 0x55c2c9d649a0 2024-09-19T09:28:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.986+0000 7fbd317cd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c2c9cc42b0 con 0x55c2c9d652e0 2024-09-19T09:28:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.987+0000 7fbd2cfcb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2c9d6a370 0x55c2c9d6c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.987+0000 7fbd1ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2c9d652e0 0x55c2c9d69da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.987+0000 7fbd1ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2c9d652e0 0x55c2c9d69da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:50366/0 (socket says 172.21.15.120:50366) 2024-09-19T09:28:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.987+0000 7fbd2cfcb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2c9d6a370 0x55c2c9d6c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43218/0 (socket says 172.21.15.120:43218) 2024-09-19T09:28:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.987+0000 7fbd1ffff640 1 -- 172.21.15.120:0/2391421204 learned_addr learned my addr 172.21.15.120:0/2391421204 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.987+0000 7fbd27fff640 1 --2- 172.21.15.120:0/2123202003 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbd140a3370 0x7fbd140a5830 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fbd18005ab0 tx=0x7fbd18000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.987+0000 7fbd27fff640 1 --2- 172.21.15.120:0/2391421204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2c9d649a0 0x55c2c9d64da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.987+0000 7fbd1ffff640 1 -- 172.21.15.120:0/2391421204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2c9d649a0 msgr2=0x55c2c9d64da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.987+0000 7fbd1ffff640 1 --2- 172.21.15.120:0/2391421204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2c9d649a0 0x55c2c9d64da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.987+0000 7fbd1ffff640 1 -- 172.21.15.120:0/2391421204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2c9d6a370 msgr2=0x55c2c9d6c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.987+0000 7fbd1ffff640 1 --2- 172.21.15.120:0/2391421204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2c9d6a370 0x55c2c9d6c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.987+0000 7fbd1ffff640 1 -- 172.21.15.120:0/2391421204 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2c9c6f340 con 0x55c2c9d652e0 2024-09-19T09:28:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.987+0000 7fbd1ffff640 1 --2- 172.21.15.120:0/2391421204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2c9d652e0 0x55c2c9d69da0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbd10004c00 tx=0x7fbd1009bc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=560acbbde5627dc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.987+0000 7fbd1e7fc640 1 -- 172.21.15.120:0/2391421204 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd100a06c0 con 0x55c2c9d652e0 2024-09-19T09:28:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.987+0000 7fbd1e7fc640 1 -- 172.21.15.120:0/2391421204 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd100a4f80 con 0x55c2c9d652e0 2024-09-19T09:28:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.987+0000 7fbd1e7fc640 1 -- 172.21.15.120:0/2391421204 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd10098060 con 0x55c2c9d652e0 2024-09-19T09:28:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.988+0000 7fbd317cd740 1 -- 172.21.15.120:0/2391421204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2c9d652e0 msgr2=0x55c2c9d69da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.988+0000 7fbd317cd740 1 --2- 172.21.15.120:0/2391421204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2c9d652e0 0x55c2c9d69da0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbd10004c00 tx=0x7fbd1009bc70 comp rx=0 tx=0).stop 2024-09-19T09:28:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.988+0000 7fbd317cd740 1 -- 172.21.15.120:0/2391421204 shutdown_connections 2024-09-19T09:28:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.988+0000 7fbd317cd740 1 --2- 172.21.15.120:0/2391421204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2c9d6a370 0x55c2c9d6c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.988+0000 7fbd317cd740 1 --2- 172.21.15.120:0/2391421204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2c9d652e0 0x55c2c9d69da0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.988+0000 7fbd317cd740 1 --2- 172.21.15.120:0/2391421204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2c9d649a0 0x55c2c9d64da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.988+0000 7fbd317cd740 1 -- 172.21.15.120:0/2391421204 >> 172.21.15.120:0/2391421204 conn(0x55c2c9d60010 msgr2=0x55c2c9d60410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.988+0000 7fbd317cd740 1 -- 172.21.15.120:0/2391421204 shutdown_connections 2024-09-19T09:28:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.988+0000 7fbd317cd740 1 -- 172.21.15.120:0/2391421204 wait complete. 2024-09-19T09:28:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.988+0000 7fbd317cd740 1 -- start start 2024-09-19T09:28:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.989+0000 7fbd317cd740 1 Processor -- start 2024-09-19T09:28:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.989+0000 7fbd317cd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2c9d649a0 0x55c2c9d6ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.989+0000 7fbd317cd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2c9d652e0 0x55c2c9d6f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.989+0000 7fbd317cd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2c9d6a370 0x55c2c9d76170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.989+0000 7fbd317cd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c2c9d6d340 con 0x55c2c9d652e0 2024-09-19T09:28:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.989+0000 7fbd317cd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c2c9c51290 con 0x55c2c9d6a370 2024-09-19T09:28:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.989+0000 7fbd317cd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c2c9d6d4c0 con 0x55c2c9d649a0 2024-09-19T09:28:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.989+0000 7fbd317cd740 10 client.0 _ll_register_callbacks cb 0x55c2c9c82140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.989+0000 7fbd1ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2c9d652e0 0x55c2c9d6f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.989+0000 7fbd1ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2c9d652e0 0x55c2c9d6f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43228/0 (socket says 172.21.15.120:43228) 2024-09-19T09:28:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.989+0000 7fbd27fff640 1 --2- 172.21.15.120:0/135757543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2c9d649a0 0x55c2c9d6ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.989+0000 7fbd1ffff640 1 -- 172.21.15.120:0/135757543 learned_addr learned my addr 172.21.15.120:0/135757543 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.989+0000 7fbd2cfcb640 1 --2- 172.21.15.120:0/135757543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2c9d6a370 0x55c2c9d76170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.989+0000 7fbd1ffff640 1 -- 172.21.15.120:0/135757543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2c9d649a0 msgr2=0x55c2c9d6ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.989+0000 7fbd1ffff640 1 --2- 172.21.15.120:0/135757543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2c9d649a0 0x55c2c9d6ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.990+0000 7fbd1ffff640 1 -- 172.21.15.120:0/135757543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2c9d6a370 msgr2=0x55c2c9d76170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.990+0000 7fbd1ffff640 1 --2- 172.21.15.120:0/135757543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2c9d6a370 0x55c2c9d76170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.990+0000 7fbd1ffff640 1 -- 172.21.15.120:0/135757543 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2c99db210 con 0x55c2c9d652e0 2024-09-19T09:28:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.990+0000 7fbd27fff640 1 --2- 172.21.15.120:0/135757543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2c9d649a0 0x55c2c9d6ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.990+0000 7fbd1ffff640 1 --2- 172.21.15.120:0/135757543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2c9d652e0 0x55c2c9d6f130 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7fbd100a4a50 tx=0x7fbd1009c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4347a6221cbab770 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.990+0000 7fbd077fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:28:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.990+0000 7fbd077fe640 1 -- 172.21.15.120:0/135757543 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd100b5230 con 0x55c2c9d652e0 2024-09-19T09:28:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.990+0000 7fbd077fe640 1 -- 172.21.15.120:0/135757543 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fbd1009b970 con 0x55c2c9d652e0 2024-09-19T09:28:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.990+0000 7fbd317cd740 1 -- 172.21.15.120:0/135757543 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd100a4f80 con 0x55c2c9d652e0 2024-09-19T09:28:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.990+0000 7fbd077fe640 1 -- 172.21.15.120:0/135757543 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbd100bc0c0 con 0x55c2c9d652e0 2024-09-19T09:28:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.990+0000 7fbd317cd740 10 client.23580 fetch_fsmap learned FSMap version 39 2024-09-19T09:28:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.990+0000 7fbd317cd740 1 -- 172.21.15.120:0/135757543 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c2c9aa7620 con 0x55c2c9d652e0 2024-09-19T09:28:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.990+0000 7fbd077fe640 1 -- 172.21.15.120:0/135757543 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd100a4f80 con 0x55c2c9d652e0 2024-09-19T09:28:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.990+0000 7fbd077fe640 1 -- 172.21.15.120:0/135757543 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fbd100aa020 con 0x55c2c9d652e0 2024-09-19T09:28:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.990+0000 7fbd317cd740 10 client.23580 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:28:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.990+0000 7fbd317cd740 10 client.23580 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:28:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.991+0000 7fbd317cd740 20 client.23580 populate_metadata read hostname 'smithi120' 2024-09-19T09:28:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.991+0000 7fbd317cd740 1 --2- 172.21.15.120:0/135757543 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c2c9d7a470 0x55c2c9d7c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.991+0000 7fbd317cd740 4 client.23580 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:28:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.991+0000 7fbd317cd740 1 -- 172.21.15.120:0/135757543 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2c9cd5a20 con 0x55c2c9d7a470 2024-09-19T09:28:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.991+0000 7fbd27fff640 1 --2- 172.21.15.120:0/135757543 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c2c9d7a470 0x55c2c9d7c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.991+0000 7fbd077fe640 10 client.23580 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:28:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.991+0000 7fbd27fff640 1 --2- 172.21.15.120:0/135757543 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c2c9d7a470 0x55c2c9d7c860 crc :-1 s=READY pgs=615 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48c2a2dea19c410c server_cookie=52c3db76a5ec6f3f in_seq=0 out_seq=0 2024-09-19T09:28:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.993+0000 7fbd077fe640 1 -- 172.21.15.120:0/135757543 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c2c9cd5a20 con 0x55c2c9d7a470 2024-09-19T09:28:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.993+0000 7fbd077fe640 10 client.23580 handle_command_reply: tid=0 2024-09-19T09:28:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.993+0000 7fbd317cd740 1 client.23580 shutdown 2024-09-19T09:28:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.993+0000 7fbd317cd740 1 -- 172.21.15.120:0/135757543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2c9d652e0 msgr2=0x55c2c9d6f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.993+0000 7fbd317cd740 1 --2- 172.21.15.120:0/135757543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2c9d652e0 0x55c2c9d6f130 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7fbd100a4a50 tx=0x7fbd1009c3d0 comp rx=0 tx=0).stop 2024-09-19T09:28:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.993+0000 7fbd317cd740 1 -- 172.21.15.120:0/135757543 shutdown_connections 2024-09-19T09:28:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.993+0000 7fbd317cd740 1 --2- 172.21.15.120:0/135757543 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c2c9d7a470 0x55c2c9d7c860 crc :-1 s=READY pgs=615 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.993+0000 7fbd317cd740 1 --2- 172.21.15.120:0/135757543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2c9d6a370 0x55c2c9d76170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.993+0000 7fbd077fe640 0 client.23580 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:28:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.993+0000 7fbd317cd740 1 --2- 172.21.15.120:0/135757543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2c9d652e0 0x55c2c9d6f130 unknown :-1 s=CLOSED pgs=2532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.993+0000 7fbd317cd740 1 --2- 172.21.15.120:0/135757543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2c9d649a0 0x55c2c9d6ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.993+0000 7fbd317cd740 1 -- 172.21.15.120:0/135757543 >> 172.21.15.120:0/135757543 conn(0x55c2c9d60010 msgr2=0x55c2c9d61820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.993+0000 7fbd317cd740 1 -- 172.21.15.120:0/135757543 shutdown_connections 2024-09-19T09:28:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.993+0000 7fbd317cd740 1 -- 172.21.15.120:0/135757543 wait complete. 2024-09-19T09:28:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:03.993+0000 7fbd317cd740 20 client.23580 trim_cache size 0 max 16384 2024-09-19T09:28:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.009+0000 7fbd317cd740 1 Processor -- start 2024-09-19T09:28:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.009+0000 7fbd317cd740 1 -- start start 2024-09-19T09:28:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.009+0000 7fbd317cd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2c9d649a0 0x55c2c9d64d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.009+0000 7fbd317cd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2c9d652e0 0x55c2c9d76130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.009+0000 7fbd317cd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2c9d6a370 0x55c2c9d60ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.009+0000 7fbd317cd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbd281a8fb0 con 0x55c2c9d652e0 2024-09-19T09:28:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.009+0000 7fbd317cd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbd281aaaf0 con 0x55c2c9d6a370 2024-09-19T09:28:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.009+0000 7fbd317cd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbd281a9680 con 0x55c2c9d649a0 2024-09-19T09:28:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.009+0000 7fbd1ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2c9d652e0 0x55c2c9d76130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.009+0000 7fbd27fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2c9d649a0 0x55c2c9d64d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.009+0000 7fbd1ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2c9d652e0 0x55c2c9d76130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43244/0 (socket says 172.21.15.120:43244) 2024-09-19T09:28:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.009+0000 7fbd1ffff640 1 -- 172.21.15.120:0/3896182593 learned_addr learned my addr 172.21.15.120:0/3896182593 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.009+0000 7fbd2cfcb640 1 --2- 172.21.15.120:0/3896182593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2c9d6a370 0x55c2c9d60ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.009+0000 7fbd1ffff640 1 -- 172.21.15.120:0/3896182593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2c9d649a0 msgr2=0x55c2c9d64d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.009+0000 7fbd1ffff640 1 --2- 172.21.15.120:0/3896182593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2c9d649a0 0x55c2c9d64d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.009+0000 7fbd1ffff640 1 -- 172.21.15.120:0/3896182593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2c9d6a370 msgr2=0x55c2c9d60ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.009+0000 7fbd1ffff640 1 --2- 172.21.15.120:0/3896182593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2c9d6a370 0x55c2c9d60ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.009+0000 7fbd1ffff640 1 -- 172.21.15.120:0/3896182593 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2c9d6e730 con 0x55c2c9d652e0 2024-09-19T09:28:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.009+0000 7fbd1ffff640 1 --2- 172.21.15.120:0/3896182593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2c9d652e0 0x55c2c9d76130 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7fbd1009a5c0 tx=0x7fbd1009a820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9cea2c3530790bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.010+0000 7fbd1dffb640 1 -- 172.21.15.120:0/3896182593 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd100a4f80 con 0x55c2c9d652e0 2024-09-19T09:28:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.010+0000 7fbd1dffb640 1 -- 172.21.15.120:0/3896182593 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd100ba070 con 0x55c2c9d652e0 2024-09-19T09:28:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.010+0000 7fbd1dffb640 1 -- 172.21.15.120:0/3896182593 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd10098060 con 0x55c2c9d652e0 2024-09-19T09:28:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.010+0000 7fbd317cd740 1 -- 172.21.15.120:0/3896182593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2c9d652e0 msgr2=0x55c2c9d76130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.010+0000 7fbd317cd740 1 --2- 172.21.15.120:0/3896182593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2c9d652e0 0x55c2c9d76130 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7fbd1009a5c0 tx=0x7fbd1009a820 comp rx=0 tx=0).stop 2024-09-19T09:28:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.010+0000 7fbd317cd740 1 -- 172.21.15.120:0/3896182593 shutdown_connections 2024-09-19T09:28:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.010+0000 7fbd317cd740 1 --2- 172.21.15.120:0/3896182593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2c9d6a370 0x55c2c9d60ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.010+0000 7fbd317cd740 1 --2- 172.21.15.120:0/3896182593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2c9d652e0 0x55c2c9d76130 unknown :-1 s=CLOSED pgs=2533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.010+0000 7fbd317cd740 1 --2- 172.21.15.120:0/3896182593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2c9d649a0 0x55c2c9d64d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.010+0000 7fbd317cd740 1 -- 172.21.15.120:0/3896182593 >> 172.21.15.120:0/3896182593 conn(0x55c2c9d60010 msgr2=0x55c2c9d6ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.011+0000 7fbd317cd740 1 -- 172.21.15.120:0/3896182593 shutdown_connections 2024-09-19T09:28:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.011+0000 7fbd317cd740 1 -- 172.21.15.120:0/3896182593 wait complete. 2024-09-19T09:28:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.011+0000 7fbd317cd740 1 -- start start 2024-09-19T09:28:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.011+0000 7fbd317cd740 1 Processor -- start 2024-09-19T09:28:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.011+0000 7fbd317cd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2c9d649a0 0x55c2c9d73590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.011+0000 7fbd317cd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2c9d652e0 0x55c2c9d73ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.011+0000 7fbd317cd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2c9d6a370 0x55c2c9d74ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.011+0000 7fbd317cd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbd281aa420 con 0x55c2c9d6a370 2024-09-19T09:28:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.011+0000 7fbd317cd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbd281a9d50 con 0x55c2c9d649a0 2024-09-19T09:28:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.011+0000 7fbd317cd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbd281a6f70 con 0x55c2c9d652e0 2024-09-19T09:28:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.011+0000 7fbd317cd740 10 client.0 _ll_register_callbacks cb 0x7fbd100baf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.011+0000 7fbd2cfcb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2c9d6a370 0x55c2c9d74ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.012+0000 7fbd27fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2c9d649a0 0x55c2c9d73590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.012+0000 7fbd2cfcb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2c9d6a370 0x55c2c9d74ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43256/0 (socket says 172.21.15.120:43256) 2024-09-19T09:28:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.012+0000 7fbd1ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2c9d652e0 0x55c2c9d73ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.012+0000 7fbd2cfcb640 1 -- 172.21.15.120:0/435480635 learned_addr learned my addr 172.21.15.120:0/435480635 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.012+0000 7fbd2cfcb640 1 -- 172.21.15.120:0/435480635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2c9d652e0 msgr2=0x55c2c9d73ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.012+0000 7fbd2cfcb640 1 --2- 172.21.15.120:0/435480635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2c9d652e0 0x55c2c9d73ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.012+0000 7fbd2cfcb640 1 -- 172.21.15.120:0/435480635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2c9d649a0 msgr2=0x55c2c9d73590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.012+0000 7fbd2cfcb640 1 --2- 172.21.15.120:0/435480635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2c9d649a0 0x55c2c9d73590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.012+0000 7fbd2cfcb640 1 -- 172.21.15.120:0/435480635 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2c9d750f0 con 0x55c2c9d6a370 2024-09-19T09:28:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.012+0000 7fbd1ffff640 1 --2- 172.21.15.120:0/435480635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2c9d652e0 0x55c2c9d73ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.012+0000 7fbd2cfcb640 1 --2- 172.21.15.120:0/435480635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2c9d6a370 0x55c2c9d74ae0 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7fbd200042f0 tx=0x7fbd20003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6186c33c2874c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.012+0000 7fbd077fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:28:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.012+0000 7fbd077fe640 1 -- 172.21.15.120:0/435480635 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd20022060 con 0x55c2c9d6a370 2024-09-19T09:28:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.012+0000 7fbd077fe640 1 -- 172.21.15.120:0/435480635 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fbd20012b30 con 0x55c2c9d6a370 2024-09-19T09:28:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.012+0000 7fbd317cd740 1 -- 172.21.15.120:0/435480635 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd100ba070 con 0x55c2c9d6a370 2024-09-19T09:28:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.012+0000 7fbd077fe640 1 -- 172.21.15.120:0/435480635 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbd20012d30 con 0x55c2c9d6a370 2024-09-19T09:28:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.012+0000 7fbd317cd740 10 client.23586 fetch_fsmap learned FSMap version 39 2024-09-19T09:28:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.012+0000 7fbd317cd740 1 -- 172.21.15.120:0/435480635 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c2c9de5010 con 0x55c2c9d6a370 2024-09-19T09:28:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.012+0000 7fbd077fe640 1 -- 172.21.15.120:0/435480635 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd100ba070 con 0x55c2c9d6a370 2024-09-19T09:28:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.013+0000 7fbd077fe640 1 -- 172.21.15.120:0/435480635 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fbd20027070 con 0x55c2c9d6a370 2024-09-19T09:28:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.013+0000 7fbd317cd740 10 client.23586 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:28:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.013+0000 7fbd317cd740 10 client.23586 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:28:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.013+0000 7fbd317cd740 20 client.23586 populate_metadata read hostname 'smithi120' 2024-09-19T09:28:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.013+0000 7fbd317cd740 1 --2- 172.21.15.120:0/435480635 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c2c9de5830 0x55c2c9de7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.013+0000 7fbd317cd740 4 client.23586 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:28:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.013+0000 7fbd317cd740 1 -- 172.21.15.120:0/435480635 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c2c9ccfd70 con 0x55c2c9de5830 2024-09-19T09:28:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.013+0000 7fbd27fff640 1 --2- 172.21.15.120:0/435480635 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c2c9de5830 0x55c2c9de7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.013+0000 7fbd077fe640 10 client.23586 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:28:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.013+0000 7fbd27fff640 1 --2- 172.21.15.120:0/435480635 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c2c9de5830 0x55c2c9de7c50 crc :-1 s=READY pgs=616 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e5e5a0bcf5bf556 server_cookie=51272783fe8b51ff in_seq=0 out_seq=0 2024-09-19T09:28:04.397 INFO:tasks.workunit.client.0.smithi120.stdout: 10 276323 19.87 MB/sec execute 599 sec latency 359.886 ms 2024-09-19T09:28:04.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:04 smithi196 ceph-mon[35150]: pgmap v1101: 97 pgs: 97 active+clean; 481 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.4 MiB/s wr, 398 op/s 2024-09-19T09:28:04.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:04 smithi153 ceph-mon[35071]: pgmap v1101: 97 pgs: 97 active+clean; 481 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.4 MiB/s wr, 398 op/s 2024-09-19T09:28:04.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:04 smithi120 ceph-mon[31150]: pgmap v1101: 97 pgs: 97 active+clean; 481 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.4 MiB/s wr, 398 op/s 2024-09-19T09:28:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.606+0000 7fbd077fe640 1 -- 172.21.15.120:0/435480635 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c2c9ccfd70 con 0x55c2c9de5830 2024-09-19T09:28:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.606+0000 7fbd077fe640 10 client.23586 handle_command_reply: tid=0 2024-09-19T09:28:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.606+0000 7fbd317cd740 1 client.23586 shutdown 2024-09-19T09:28:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.606+0000 7fbd317cd740 1 -- 172.21.15.120:0/435480635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2c9d6a370 msgr2=0x55c2c9d74ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.606+0000 7fbd317cd740 1 --2- 172.21.15.120:0/435480635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2c9d6a370 0x55c2c9d74ae0 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7fbd200042f0 tx=0x7fbd20003f90 comp rx=0 tx=0).stop 2024-09-19T09:28:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.606+0000 7fbd317cd740 1 -- 172.21.15.120:0/435480635 shutdown_connections 2024-09-19T09:28:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.606+0000 7fbd317cd740 1 --2- 172.21.15.120:0/435480635 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55c2c9de5830 0x55c2c9de7c50 crc :-1 s=READY pgs=616 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.606+0000 7fbd317cd740 1 --2- 172.21.15.120:0/435480635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2c9d6a370 0x55c2c9d74ae0 unknown :-1 s=CLOSED pgs=2534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.606+0000 7fbd077fe640 0 client.23586 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:28:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.606+0000 7fbd317cd740 1 --2- 172.21.15.120:0/435480635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2c9d652e0 0x55c2c9d73ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.606+0000 7fbd317cd740 1 --2- 172.21.15.120:0/435480635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2c9d649a0 0x55c2c9d73590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.606+0000 7fbd317cd740 1 -- 172.21.15.120:0/435480635 >> 172.21.15.120:0/435480635 conn(0x55c2c9d60010 msgr2=0x55c2c9d6ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.606+0000 7fbd317cd740 1 -- 172.21.15.120:0/435480635 shutdown_connections 2024-09-19T09:28:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.606+0000 7fbd317cd740 1 -- 172.21.15.120:0/435480635 wait complete. 2024-09-19T09:28:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.606+0000 7fbd317cd740 20 client.23586 trim_cache size 0 max 16384 2024-09-19T09:28:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-19T09:28:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-19T09:28:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-19T09:28:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-19T09:28:04.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.607+0000 7fbd2ea2e640 1 -- 172.21.15.120:0/2123202003 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbd140a3370 msgr2=0x7fbd140a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:04.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.607+0000 7fbd2ea2e640 1 --2- 172.21.15.120:0/2123202003 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbd140a3370 0x7fbd140a5830 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fbd18005ab0 tx=0x7fbd18000bc0 comp rx=0 tx=0).stop 2024-09-19T09:28:04.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.607+0000 7fbd2ea2e640 1 -- 172.21.15.120:0/2123202003 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd28101c80 msgr2=0x7fbd28198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:04.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.607+0000 7fbd2ea2e640 1 --2- 172.21.15.120:0/2123202003 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd28101c80 0x7fbd28198310 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7fbd1000e9f0 tx=0x7fbd1000edf0 comp rx=0 tx=0).stop 2024-09-19T09:28:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.607+0000 7fbd2ea2e640 1 -- 172.21.15.120:0/2123202003 shutdown_connections 2024-09-19T09:28:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.607+0000 7fbd2ea2e640 1 --2- 172.21.15.120:0/2123202003 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbd140a3370 0x7fbd140a5830 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.607+0000 7fbd2ea2e640 1 --2- 172.21.15.120:0/2123202003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd28102640 0x7fbd2819f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.607+0000 7fbd2ea2e640 1 --2- 172.21.15.120:0/2123202003 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd28101c80 0x7fbd28198310 unknown :-1 s=CLOSED pgs=2531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.607+0000 7fbd2ea2e640 1 --2- 172.21.15.120:0/2123202003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd28100f70 0x7fbd28197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.607+0000 7fbd2ea2e640 1 -- 172.21.15.120:0/2123202003 >> 172.21.15.120:0/2123202003 conn(0x7fbd280fb970 msgr2=0x7fbd280fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.607+0000 7fbd2ea2e640 1 -- 172.21.15.120:0/2123202003 shutdown_connections 2024-09-19T09:28:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-19T09:28:04.607+0000 7fbd2ea2e640 1 -- 172.21.15.120:0/2123202003 wait complete. 2024-09-19T09:28:04.799 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:28:05.075 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:05.397 INFO:tasks.workunit.client.0.smithi120.stdout: 10 cleanup 600 sec 2024-09-19T09:28:05.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.598+0000 7f6f64298640 1 -- 172.21.15.120:0/1340915894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f5c102ef0 msgr2=0x7f6f5c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:05.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.598+0000 7f6f64298640 1 --2- 172.21.15.120:0/1340915894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f5c102ef0 0x7f6f5c10da40 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7f6f54009ca0 tx=0x7f6f5402a950 comp rx=0 tx=0).stop 2024-09-19T09:28:05.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.598+0000 7f6f64298640 1 -- 172.21.15.120:0/1340915894 shutdown_connections 2024-09-19T09:28:05.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.598+0000 7f6f64298640 1 --2- 172.21.15.120:0/1340915894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f5c102ef0 0x7f6f5c10da40 unknown :-1 s=CLOSED pgs=2535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.598+0000 7f6f64298640 1 --2- 172.21.15.120:0/1340915894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f5c102570 0x7f6f5c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.598+0000 7f6f64298640 1 --2- 172.21.15.120:0/1340915894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f5c100e20 0x7f6f5c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.598+0000 7f6f64298640 1 -- 172.21.15.120:0/1340915894 >> 172.21.15.120:0/1340915894 conn(0x7f6f5c0fc7d0 msgr2=0x7f6f5c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:05.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.598+0000 7f6f64298640 1 -- 172.21.15.120:0/1340915894 shutdown_connections 2024-09-19T09:28:05.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.599+0000 7f6f64298640 1 -- 172.21.15.120:0/1340915894 wait complete. 2024-09-19T09:28:05.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.599+0000 7f6f64298640 1 Processor -- start 2024-09-19T09:28:05.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.599+0000 7f6f64298640 1 -- start start 2024-09-19T09:28:05.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.599+0000 7f6f64298640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f5c100e20 0x7f6f5c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.599+0000 7f6f64298640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f5c102570 0x7f6f5c19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.599+0000 7f6f64298640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f5c102ef0 0x7f6f5c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.599+0000 7f6f64298640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c110f80 con 0x7f6f5c102ef0 2024-09-19T09:28:05.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.599+0000 7f6f64298640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c110e00 con 0x7f6f5c100e20 2024-09-19T09:28:05.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.599+0000 7f6f64298640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c111100 con 0x7f6f5c102570 2024-09-19T09:28:05.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.599+0000 7f6f62835640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f5c102ef0 0x7f6f5c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.599+0000 7f6f62835640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f5c102ef0 0x7f6f5c19efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43276/0 (socket says 172.21.15.120:43276) 2024-09-19T09:28:05.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.599+0000 7f6f62835640 1 -- 172.21.15.120:0/1264178719 learned_addr learned my addr 172.21.15.120:0/1264178719 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:05.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.599+0000 7f6f62034640 1 --2- 172.21.15.120:0/1264178719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f5c100e20 0x7f6f5c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.599+0000 7f6f61833640 1 --2- 172.21.15.120:0/1264178719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f5c102570 0x7f6f5c19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.600+0000 7f6f62835640 1 -- 172.21.15.120:0/1264178719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f5c102570 msgr2=0x7f6f5c19ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:05.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.600+0000 7f6f62835640 1 --2- 172.21.15.120:0/1264178719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f5c102570 0x7f6f5c19ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.600+0000 7f6f62835640 1 -- 172.21.15.120:0/1264178719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f5c100e20 msgr2=0x7f6f5c19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:05.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.600+0000 7f6f62835640 1 --2- 172.21.15.120:0/1264178719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f5c100e20 0x7f6f5c19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.600+0000 7f6f62835640 1 -- 172.21.15.120:0/1264178719 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f5c19f6c0 con 0x7f6f5c102ef0 2024-09-19T09:28:05.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.600+0000 7f6f62034640 1 --2- 172.21.15.120:0/1264178719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f5c100e20 0x7f6f5c19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:05.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.600+0000 7f6f62835640 1 --2- 172.21.15.120:0/1264178719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f5c102ef0 0x7f6f5c19efe0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f6f54004af0 tx=0x7f6f5402f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:05.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.600+0000 7f6f537fe640 1 -- 172.21.15.120:0/1264178719 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f54034040 con 0x7f6f5c102ef0 2024-09-19T09:28:05.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.600+0000 7f6f537fe640 1 -- 172.21.15.120:0/1264178719 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f54030920 con 0x7f6f5c102ef0 2024-09-19T09:28:05.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.600+0000 7f6f64298640 1 -- 172.21.15.120:0/1264178719 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f5c19f950 con 0x7f6f5c102ef0 2024-09-19T09:28:05.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.600+0000 7f6f537fe640 1 -- 172.21.15.120:0/1264178719 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f54004630 con 0x7f6f5c102ef0 2024-09-19T09:28:05.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.600+0000 7f6f64298640 1 -- 172.21.15.120:0/1264178719 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f5c1a7290 con 0x7f6f5c102ef0 2024-09-19T09:28:05.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.603+0000 7f6f537fe640 1 -- 172.21.15.120:0/1264178719 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f6f54037020 con 0x7f6f5c102ef0 2024-09-19T09:28:05.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.603+0000 7f6f66671740 1 Processor -- start 2024-09-19T09:28:05.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.603+0000 7f6f66671740 1 -- start start 2024-09-19T09:28:05.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.603+0000 7f6f66671740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172d90db40 0x56172d90c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.603+0000 7f6f66671740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172d90c5d0 0x56172d90ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.603+0000 7f6f66671740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172d917370 0x56172d919760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.603+0000 7f6f66671740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56172d8722f0 con 0x56172d917370 2024-09-19T09:28:05.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.603+0000 7f6f66671740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56172d819cf0 con 0x56172d90db40 2024-09-19T09:28:05.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.603+0000 7f6f66671740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56172d86e2b0 con 0x56172d90c5d0 2024-09-19T09:28:05.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.604+0000 7f6f62034640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172d90c5d0 0x56172d90ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.604+0000 7f6f61833640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172d917370 0x56172d919760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.604+0000 7f6f62034640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172d90c5d0 0x56172d90ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:50402/0 (socket says 172.21.15.120:50402) 2024-09-19T09:28:05.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.604+0000 7f6f62034640 1 -- 172.21.15.120:0/1268742493 learned_addr learned my addr 172.21.15.120:0/1268742493 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:05.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.604+0000 7f6f61833640 1 --2- 172.21.15.120:0/1268742493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172d90db40 0x56172d90c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.604+0000 7f6f62034640 1 -- 172.21.15.120:0/1268742493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172d90db40 msgr2=0x56172d90c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:05.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.604+0000 7f6f62034640 1 --2- 172.21.15.120:0/1268742493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172d90db40 0x56172d90c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.604+0000 7f6f62034640 1 -- 172.21.15.120:0/1268742493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172d917370 msgr2=0x56172d919760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:05.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.604+0000 7f6f62034640 1 --2- 172.21.15.120:0/1268742493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172d917370 0x56172d919760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.604+0000 7f6f62034640 1 -- 172.21.15.120:0/1268742493 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56172d7fe3a0 con 0x56172d90c5d0 2024-09-19T09:28:05.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.604+0000 7f6f61833640 1 --2- 172.21.15.120:0/1268742493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172d917370 0x56172d919760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:05.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.604+0000 7f6f537fe640 1 --2- 172.21.15.120:0/1264178719 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6f480a3390 0x7f6f480a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.604+0000 7f6f61833640 1 --2- 172.21.15.120:0/1264178719 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6f480a3390 0x7f6f480a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.604+0000 7f6f62034640 1 --2- 172.21.15.120:0/1268742493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172d90c5d0 0x56172d90ca10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6f580115c0 tx=0x7f6f58011a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=425def0b407b04da server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:05.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.604+0000 7f6f537fe640 1 -- 172.21.15.120:0/1264178719 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f6f540f04e0 con 0x7f6f5c102ef0 2024-09-19T09:28:05.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.604+0000 7f6f50dea640 1 -- 172.21.15.120:0/1268742493 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5800f640 con 0x56172d90c5d0 2024-09-19T09:28:05.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.604+0000 7f6f50dea640 1 -- 172.21.15.120:0/1268742493 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f5800fcc0 con 0x56172d90c5d0 2024-09-19T09:28:05.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.604+0000 7f6f50dea640 1 -- 172.21.15.120:0/1268742493 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f58019d50 con 0x56172d90c5d0 2024-09-19T09:28:05.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.605+0000 7f6f66671740 1 -- 172.21.15.120:0/1268742493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172d90c5d0 msgr2=0x56172d90ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:05.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.605+0000 7f6f66671740 1 --2- 172.21.15.120:0/1268742493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172d90c5d0 0x56172d90ca10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6f580115c0 tx=0x7f6f58011a80 comp rx=0 tx=0).stop 2024-09-19T09:28:05.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.605+0000 7f6f61833640 1 --2- 172.21.15.120:0/1264178719 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6f480a3390 0x7f6f480a5850 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x56172d90d070 tx=0x7f6f4c004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:05.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.605+0000 7f6f66671740 1 -- 172.21.15.120:0/1268742493 shutdown_connections 2024-09-19T09:28:05.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.605+0000 7f6f66671740 1 --2- 172.21.15.120:0/1268742493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172d917370 0x56172d919760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.605+0000 7f6f66671740 1 --2- 172.21.15.120:0/1268742493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172d90c5d0 0x56172d90ca10 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.605+0000 7f6f66671740 1 --2- 172.21.15.120:0/1268742493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172d90db40 0x56172d90c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.605+0000 7f6f66671740 1 -- 172.21.15.120:0/1268742493 >> 172.21.15.120:0/1268742493 conn(0x56172d8d5310 msgr2=0x56172d908c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:05.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.605+0000 7f6f66671740 1 -- 172.21.15.120:0/1268742493 shutdown_connections 2024-09-19T09:28:05.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.605+0000 7f6f66671740 1 -- 172.21.15.120:0/1268742493 wait complete. 2024-09-19T09:28:05.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.606+0000 7f6f66671740 1 -- start start 2024-09-19T09:28:05.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.606+0000 7f6f66671740 1 Processor -- start 2024-09-19T09:28:05.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.606+0000 7f6f66671740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172d90c5d0 0x56172d916db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.606+0000 7f6f66671740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172d90db40 0x56172d914060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.606+0000 7f6f66671740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172d917370 0x56172d9147f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.606+0000 7f6f66671740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56172d91a070 con 0x56172d90db40 2024-09-19T09:28:05.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.606+0000 7f6f66671740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56172d82d060 con 0x56172d917370 2024-09-19T09:28:05.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.606+0000 7f6f66671740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56172d91a1f0 con 0x56172d90c5d0 2024-09-19T09:28:05.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.606+0000 7f6f66671740 10 client.0 _ll_register_callbacks cb 0x56172d5808c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:05.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.606+0000 7f6f61833640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172d90db40 0x56172d914060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.606+0000 7f6f62034640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172d90c5d0 0x56172d916db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.606+0000 7f6f62034640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172d90c5d0 0x56172d916db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:50404/0 (socket says 172.21.15.120:50404) 2024-09-19T09:28:05.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.606+0000 7f6f62034640 1 -- 172.21.15.120:0/122544788 learned_addr learned my addr 172.21.15.120:0/122544788 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:05.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.606+0000 7f6f62835640 1 --2- 172.21.15.120:0/122544788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172d917370 0x56172d9147f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.606+0000 7f6f61833640 1 -- 172.21.15.120:0/122544788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172d90c5d0 msgr2=0x56172d916db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:05.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.606+0000 7f6f61833640 1 --2- 172.21.15.120:0/122544788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172d90c5d0 0x56172d916db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.606+0000 7f6f61833640 1 -- 172.21.15.120:0/122544788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172d917370 msgr2=0x56172d9147f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:05.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.606+0000 7f6f61833640 1 --2- 172.21.15.120:0/122544788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172d917370 0x56172d9147f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.606+0000 7f6f61833640 1 -- 172.21.15.120:0/122544788 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56172d5e33c0 con 0x56172d90db40 2024-09-19T09:28:05.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.606+0000 7f6f62835640 1 --2- 172.21.15.120:0/122544788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172d917370 0x56172d9147f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:05.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.607+0000 7f6f61833640 1 --2- 172.21.15.120:0/122544788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172d90db40 0x56172d914060 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0ff0c0 tx=0x7f6f4c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f215dc152a0e915 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:05.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.607+0000 7f6f45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:28:05.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.607+0000 7f6f45ffb640 1 -- 172.21.15.120:0/122544788 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c02d060 con 0x56172d90db40 2024-09-19T09:28:05.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.607+0000 7f6f45ffb640 1 -- 172.21.15.120:0/122544788 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f6f4c01a070 con 0x56172d90db40 2024-09-19T09:28:05.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.607+0000 7f6f66671740 1 -- 172.21.15.120:0/122544788 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f5800fcc0 con 0x56172d90db40 2024-09-19T09:28:05.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.607+0000 7f6f45ffb640 1 -- 172.21.15.120:0/122544788 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6f4c01e620 con 0x56172d90db40 2024-09-19T09:28:05.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.607+0000 7f6f66671740 10 client.23598 fetch_fsmap learned FSMap version 39 2024-09-19T09:28:05.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.607+0000 7f6f66671740 1 -- 172.21.15.120:0/122544788 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56172d925c10 con 0x56172d90db40 2024-09-19T09:28:05.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.607+0000 7f6f45ffb640 1 -- 172.21.15.120:0/122544788 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f5800fcc0 con 0x56172d90db40 2024-09-19T09:28:05.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.607+0000 7f6f45ffb640 1 -- 172.21.15.120:0/122544788 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f6f4c032070 con 0x56172d90db40 2024-09-19T09:28:05.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.607+0000 7f6f66671740 10 client.23598 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:28:05.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.607+0000 7f6f66671740 10 client.23598 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:28:05.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.607+0000 7f6f66671740 20 client.23598 populate_metadata read hostname 'smithi120' 2024-09-19T09:28:05.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.607+0000 7f6f66671740 1 --2- 172.21.15.120:0/122544788 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56172d927120 0x56172d929540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.607+0000 7f6f66671740 4 client.23598 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:28:05.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.607+0000 7f6f66671740 1 -- 172.21.15.120:0/122544788 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56172d87f860 con 0x56172d927120 2024-09-19T09:28:05.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.607+0000 7f6f62034640 1 --2- 172.21.15.120:0/122544788 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56172d927120 0x56172d929540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.608+0000 7f6f62034640 1 --2- 172.21.15.120:0/122544788 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56172d927120 0x56172d929540 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ebc9ee427d4eaab8 server_cookie=ceb19f8b23ffa3af in_seq=0 out_seq=0 2024-09-19T09:28:05.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.608+0000 7f6f45ffb640 10 client.23598 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:28:05.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.609+0000 7f6f45ffb640 1 -- 172.21.15.120:0/122544788 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56172d87f860 con 0x56172d927120 2024-09-19T09:28:05.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.609+0000 7f6f45ffb640 10 client.23598 handle_command_reply: tid=0 2024-09-19T09:28:05.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.609+0000 7f6f66671740 1 client.23598 shutdown 2024-09-19T09:28:05.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.609+0000 7f6f66671740 1 -- 172.21.15.120:0/122544788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172d90db40 msgr2=0x56172d914060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:05.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.609+0000 7f6f66671740 1 --2- 172.21.15.120:0/122544788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172d90db40 0x56172d914060 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0ff0c0 tx=0x7f6f4c005f90 comp rx=0 tx=0).stop 2024-09-19T09:28:05.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.609+0000 7f6f66671740 1 -- 172.21.15.120:0/122544788 shutdown_connections 2024-09-19T09:28:05.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.609+0000 7f6f66671740 1 --2- 172.21.15.120:0/122544788 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56172d927120 0x56172d929540 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.609+0000 7f6f66671740 1 --2- 172.21.15.120:0/122544788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172d917370 0x56172d9147f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.609+0000 7f6f45ffb640 0 client.23598 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:28:05.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.609+0000 7f6f66671740 1 --2- 172.21.15.120:0/122544788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172d90db40 0x56172d914060 unknown :-1 s=CLOSED pgs=2537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.609+0000 7f6f66671740 1 --2- 172.21.15.120:0/122544788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172d90c5d0 0x56172d916db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.609+0000 7f6f66671740 1 -- 172.21.15.120:0/122544788 >> 172.21.15.120:0/122544788 conn(0x56172d8d5310 msgr2=0x56172d90e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:05.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.610+0000 7f6f66671740 1 -- 172.21.15.120:0/122544788 shutdown_connections 2024-09-19T09:28:05.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.610+0000 7f6f66671740 1 -- 172.21.15.120:0/122544788 wait complete. 2024-09-19T09:28:05.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.610+0000 7f6f66671740 20 client.23598 trim_cache size 0 max 16384 2024-09-19T09:28:05.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.622+0000 7f6f66671740 1 Processor -- start 2024-09-19T09:28:05.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.622+0000 7f6f66671740 1 -- start start 2024-09-19T09:28:05.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.623+0000 7f6f66671740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172d90c5d0 0x56172d916db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.623+0000 7f6f66671740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172d90db40 0x56172d908700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.623+0000 7f6f66671740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172d917370 0x56172d908c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.623+0000 7f6f66671740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c1af710 con 0x56172d90db40 2024-09-19T09:28:05.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.623+0000 7f6f66671740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c1b1250 con 0x56172d90c5d0 2024-09-19T09:28:05.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.623+0000 7f6f66671740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c1afde0 con 0x56172d917370 2024-09-19T09:28:05.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.623+0000 7f6f61833640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172d90db40 0x56172d908700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.623+0000 7f6f61833640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172d90db40 0x56172d908700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43304/0 (socket says 172.21.15.120:43304) 2024-09-19T09:28:05.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.623+0000 7f6f61833640 1 -- 172.21.15.120:0/1791747777 learned_addr learned my addr 172.21.15.120:0/1791747777 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:05.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.623+0000 7f6f62835640 1 --2- 172.21.15.120:0/1791747777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172d917370 0x56172d908c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.623+0000 7f6f62034640 1 --2- 172.21.15.120:0/1791747777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172d90c5d0 0x56172d916db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.623+0000 7f6f61833640 1 -- 172.21.15.120:0/1791747777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172d917370 msgr2=0x56172d908c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:05.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.623+0000 7f6f61833640 1 --2- 172.21.15.120:0/1791747777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172d917370 0x56172d908c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.623+0000 7f6f61833640 1 -- 172.21.15.120:0/1791747777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172d90c5d0 msgr2=0x56172d916db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:05.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.623+0000 7f6f61833640 1 --2- 172.21.15.120:0/1791747777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172d90c5d0 0x56172d916db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.623+0000 7f6f61833640 1 -- 172.21.15.120:0/1791747777 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56172d909dd0 con 0x56172d90db40 2024-09-19T09:28:05.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.623+0000 7f6f61833640 1 --2- 172.21.15.120:0/1791747777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172d90db40 0x56172d908700 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c0109f0 tx=0x7f6f4c020e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5844778890c3f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:05.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.623+0000 7f6f47fff640 1 -- 172.21.15.120:0/1791747777 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c038030 con 0x56172d90db40 2024-09-19T09:28:05.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.623+0000 7f6f47fff640 1 -- 172.21.15.120:0/1791747777 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f4c009a40 con 0x56172d90db40 2024-09-19T09:28:05.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.623+0000 7f6f47fff640 1 -- 172.21.15.120:0/1791747777 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c01ed70 con 0x56172d90db40 2024-09-19T09:28:05.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.624+0000 7f6f66671740 1 -- 172.21.15.120:0/1791747777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172d90db40 msgr2=0x56172d908700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:05.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.624+0000 7f6f66671740 1 --2- 172.21.15.120:0/1791747777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172d90db40 0x56172d908700 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c0109f0 tx=0x7f6f4c020e30 comp rx=0 tx=0).stop 2024-09-19T09:28:05.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.624+0000 7f6f66671740 1 -- 172.21.15.120:0/1791747777 shutdown_connections 2024-09-19T09:28:05.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.624+0000 7f6f66671740 1 --2- 172.21.15.120:0/1791747777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172d917370 0x56172d908c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.624+0000 7f6f66671740 1 --2- 172.21.15.120:0/1791747777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172d90db40 0x56172d908700 unknown :-1 s=CLOSED pgs=2538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.624+0000 7f6f66671740 1 --2- 172.21.15.120:0/1791747777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172d90c5d0 0x56172d916db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.624+0000 7f6f66671740 1 -- 172.21.15.120:0/1791747777 >> 172.21.15.120:0/1791747777 conn(0x56172d8d5310 msgr2=0x56172d903a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:05.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.624+0000 7f6f66671740 1 -- 172.21.15.120:0/1791747777 shutdown_connections 2024-09-19T09:28:05.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.624+0000 7f6f66671740 1 -- 172.21.15.120:0/1791747777 wait complete. 2024-09-19T09:28:05.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.624+0000 7f6f66671740 1 -- start start 2024-09-19T09:28:05.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.625+0000 7f6f66671740 1 Processor -- start 2024-09-19T09:28:05.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.625+0000 7f6f66671740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172d90c5d0 0x56172d91fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.625+0000 7f6f66671740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172d90db40 0x56172d91af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.625+0000 7f6f66671740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172d917370 0x56172d91b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.625+0000 7f6f66671740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c1b0b80 con 0x56172d90c5d0 2024-09-19T09:28:05.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.625+0000 7f6f66671740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c1b04b0 con 0x56172d917370 2024-09-19T09:28:05.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.625+0000 7f6f66671740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c1ad6d0 con 0x56172d90db40 2024-09-19T09:28:05.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.625+0000 7f6f66671740 10 client.0 _ll_register_callbacks cb 0x7f6f4c009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:05.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.625+0000 7f6f62034640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172d90c5d0 0x56172d91fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.625+0000 7f6f62835640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172d917370 0x56172d91b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.625+0000 7f6f62034640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172d90c5d0 0x56172d91fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43312/0 (socket says 172.21.15.120:43312) 2024-09-19T09:28:05.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.625+0000 7f6f62034640 1 -- 172.21.15.120:0/44369168 learned_addr learned my addr 172.21.15.120:0/44369168 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:05.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.625+0000 7f6f61833640 1 --2- 172.21.15.120:0/44369168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172d90db40 0x56172d91af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.625+0000 7f6f62034640 1 -- 172.21.15.120:0/44369168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172d90db40 msgr2=0x56172d91af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:05.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.625+0000 7f6f62034640 1 --2- 172.21.15.120:0/44369168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172d90db40 0x56172d91af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.625+0000 7f6f62034640 1 -- 172.21.15.120:0/44369168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172d917370 msgr2=0x56172d91b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:05.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.625+0000 7f6f62034640 1 --2- 172.21.15.120:0/44369168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172d917370 0x56172d91b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.625+0000 7f6f62034640 1 -- 172.21.15.120:0/44369168 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56172d90f120 con 0x56172d90c5d0 2024-09-19T09:28:05.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.625+0000 7f6f62034640 1 --2- 172.21.15.120:0/44369168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172d90c5d0 0x56172d91fc00 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009380 tx=0x7f6f58009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c395aff826e40 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:05.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.625+0000 7f6f45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:28:05.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.626+0000 7f6f45ffb640 1 -- 172.21.15.120:0/44369168 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f580050c0 con 0x56172d90c5d0 2024-09-19T09:28:05.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.626+0000 7f6f66671740 1 -- 172.21.15.120:0/44369168 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f4c009a40 con 0x56172d90c5d0 2024-09-19T09:28:05.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.626+0000 7f6f45ffb640 1 -- 172.21.15.120:0/44369168 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f6f58005260 con 0x56172d90c5d0 2024-09-19T09:28:05.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.626+0000 7f6f45ffb640 1 -- 172.21.15.120:0/44369168 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6f58039050 con 0x56172d90c5d0 2024-09-19T09:28:05.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.626+0000 7f6f66671740 10 client.23604 fetch_fsmap learned FSMap version 39 2024-09-19T09:28:05.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.626+0000 7f6f66671740 1 -- 172.21.15.120:0/44369168 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56172d991c40 con 0x56172d90c5d0 2024-09-19T09:28:05.663 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:28:05.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.626+0000 7f6f45ffb640 1 -- 172.21.15.120:0/44369168 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f4c009a40 con 0x56172d90c5d0 2024-09-19T09:28:05.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.626+0000 7f6f45ffb640 1 -- 172.21.15.120:0/44369168 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f6f5800f800 con 0x56172d90c5d0 2024-09-19T09:28:05.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.626+0000 7f6f66671740 10 client.23604 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:28:05.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.626+0000 7f6f66671740 10 client.23604 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T09:28:05.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.626+0000 7f6f66671740 20 client.23604 populate_metadata read hostname 'smithi120' 2024-09-19T09:28:05.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.626+0000 7f6f66671740 1 --2- 172.21.15.120:0/44369168 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56172d9921d0 0x56172d9945c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.626+0000 7f6f66671740 4 client.23604 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:28:05.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.626+0000 7f6f66671740 1 -- 172.21.15.120:0/44369168 --> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56172d879d70 con 0x56172d9921d0 2024-09-19T09:28:05.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.626+0000 7f6f61833640 1 --2- 172.21.15.120:0/44369168 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56172d9921d0 0x56172d9945c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.627+0000 7f6f45ffb640 10 client.23604 ms_handle_connect on v2:172.21.15.196:6836/776889709 2024-09-19T09:28:05.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.627+0000 7f6f61833640 1 --2- 172.21.15.120:0/44369168 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56172d9921d0 0x56172d9945c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7d06aaa88d439858 server_cookie=77e2d1c8b8ec4252 in_seq=0 out_seq=0 2024-09-19T09:28:05.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.627+0000 7f6f45ffb640 1 -- 172.21.15.120:0/44369168 <== mds.3 v2:172.21.15.196:6836/776889709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56172d879d70 con 0x56172d9921d0 2024-09-19T09:28:05.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.627+0000 7f6f45ffb640 10 client.23604 handle_command_reply: tid=0 2024-09-19T09:28:05.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.627+0000 7f6f66671740 1 client.23604 shutdown 2024-09-19T09:28:05.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.627+0000 7f6f66671740 1 -- 172.21.15.120:0/44369168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172d90c5d0 msgr2=0x56172d91fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:05.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.627+0000 7f6f66671740 1 --2- 172.21.15.120:0/44369168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172d90c5d0 0x56172d91fc00 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009380 tx=0x7f6f58009b70 comp rx=0 tx=0).stop 2024-09-19T09:28:05.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.627+0000 7f6f66671740 1 -- 172.21.15.120:0/44369168 shutdown_connections 2024-09-19T09:28:05.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.627+0000 7f6f66671740 1 --2- 172.21.15.120:0/44369168 >> [v2:172.21.15.196:6836/776889709,v1:172.21.15.196:6837/776889709] conn(0x56172d9921d0 0x56172d9945c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.627+0000 7f6f66671740 1 --2- 172.21.15.120:0/44369168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56172d917370 0x56172d91b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.627+0000 7f6f45ffb640 0 client.23604 ms_handle_reset on v2:172.21.15.196:6836/776889709 2024-09-19T09:28:05.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.627+0000 7f6f66671740 1 --2- 172.21.15.120:0/44369168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56172d90db40 0x56172d91af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.627+0000 7f6f66671740 1 --2- 172.21.15.120:0/44369168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56172d90c5d0 0x56172d91fc00 unknown :-1 s=CLOSED pgs=2539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.627+0000 7f6f66671740 1 -- 172.21.15.120:0/44369168 >> 172.21.15.120:0/44369168 conn(0x56172d8d5310 msgr2=0x56172d908700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:05.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.627+0000 7f6f66671740 1 -- 172.21.15.120:0/44369168 shutdown_connections 2024-09-19T09:28:05.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.628+0000 7f6f66671740 1 -- 172.21.15.120:0/44369168 wait complete. 2024-09-19T09:28:05.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.628+0000 7f6f66671740 20 client.23604 trim_cache size 0 max 16384 2024-09-19T09:28:05.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.628+0000 7f6f64298640 1 -- 172.21.15.120:0/1264178719 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6f480a3390 msgr2=0x7f6f480a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:05.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.628+0000 7f6f64298640 1 --2- 172.21.15.120:0/1264178719 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6f480a3390 0x7f6f480a5850 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x56172d90d070 tx=0x7f6f4c004730 comp rx=0 tx=0).stop 2024-09-19T09:28:05.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.628+0000 7f6f64298640 1 -- 172.21.15.120:0/1264178719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f5c102ef0 msgr2=0x7f6f5c19efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:05.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.628+0000 7f6f64298640 1 --2- 172.21.15.120:0/1264178719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f5c102ef0 0x7f6f5c19efe0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f6f54004af0 tx=0x7f6f5402f3a0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.628+0000 7f6f64298640 1 -- 172.21.15.120:0/1264178719 shutdown_connections 2024-09-19T09:28:05.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.628+0000 7f6f64298640 1 --2- 172.21.15.120:0/1264178719 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f6f480a3390 0x7f6f480a5850 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.628+0000 7f6f64298640 1 --2- 172.21.15.120:0/1264178719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f5c102ef0 0x7f6f5c19efe0 unknown :-1 s=CLOSED pgs=2536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.628+0000 7f6f64298640 1 --2- 172.21.15.120:0/1264178719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f5c102570 0x7f6f5c19ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.628+0000 7f6f64298640 1 --2- 172.21.15.120:0/1264178719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f5c100e20 0x7f6f5c19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.628+0000 7f6f64298640 1 -- 172.21.15.120:0/1264178719 >> 172.21.15.120:0/1264178719 conn(0x7f6f5c0fc7d0 msgr2=0x7f6f5c06a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:05.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.628+0000 7f6f64298640 1 -- 172.21.15.120:0/1264178719 shutdown_connections 2024-09-19T09:28:05.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:05.628+0000 7f6f64298640 1 -- 172.21.15.120:0/1264178719 wait complete. 2024-09-19T09:28:06.397 INFO:tasks.workunit.client.0.smithi120.stdout: 9 cleanup 601 sec 2024-09-19T09:28:06.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:06 smithi196 ceph-mon[35150]: pgmap v1102: 97 pgs: 97 active+clean; 477 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 3.5 MiB/s wr, 425 op/s 2024-09-19T09:28:06.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:06 smithi153 ceph-mon[35071]: pgmap v1102: 97 pgs: 97 active+clean; 477 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 3.5 MiB/s wr, 425 op/s 2024-09-19T09:28:06.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:06 smithi120 ceph-mon[31150]: pgmap v1102: 97 pgs: 97 active+clean; 477 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 3.5 MiB/s wr, 425 op/s 2024-09-19T09:28:06.613 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:28:06.879 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:07.108 INFO:tasks.workunit.client.0.smithi120.stdout: 0 cleanup 602 sec 2024-09-19T09:28:07.108 INFO:tasks.workunit.client.0.smithi120.stdout: 2024-09-19T09:28:07.108 INFO:tasks.workunit.client.0.smithi120.stdout: Operation Count AvgLat MaxLat 2024-09-19T09:28:07.108 INFO:tasks.workunit.client.0.smithi120.stdout: ---------------------------------------- 2024-09-19T09:28:07.108 INFO:tasks.workunit.client.0.smithi120.stdout: NTCreateX 382721 6.672 31914.120 2024-09-19T09:28:07.109 INFO:tasks.workunit.client.0.smithi120.stdout: Close 281056 0.002 0.119 2024-09-19T09:28:07.109 INFO:tasks.workunit.client.0.smithi120.stdout: Rename 16200 36.354 31196.980 2024-09-19T09:28:07.109 INFO:tasks.workunit.client.0.smithi120.stdout: Unlink 77309 12.785 30442.649 2024-09-19T09:28:07.109 INFO:tasks.workunit.client.0.smithi120.stdout: Deltree 2 87.334 105.236 2024-09-19T09:28:07.109 INFO:tasks.workunit.client.0.smithi120.stdout: Mkdir 1 0.011 0.011 2024-09-19T09:28:07.109 INFO:tasks.workunit.client.0.smithi120.stdout: Qpathinfo 347054 3.346 993.457 2024-09-19T09:28:07.109 INFO:tasks.workunit.client.0.smithi120.stdout: Qfileinfo 60549 0.006 278.826 2024-09-19T09:28:07.109 INFO:tasks.workunit.client.0.smithi120.stdout: Qfsinfo 63547 0.164 18.426 2024-09-19T09:28:07.109 INFO:tasks.workunit.client.0.smithi120.stdout: Sfileinfo 31188 9.763 31895.480 2024-09-19T09:28:07.109 INFO:tasks.workunit.client.0.smithi120.stdout: Find 134040 0.452 678.459 2024-09-19T09:28:07.109 INFO:tasks.workunit.client.0.smithi120.stdout: WriteX 189312 0.589 29828.524 2024-09-19T09:28:07.109 INFO:tasks.workunit.client.0.smithi120.stdout: ReadX 599624 0.034 876.404 2024-09-19T09:28:07.109 INFO:tasks.workunit.client.0.smithi120.stdout: LockX 1244 6.071 129.623 2024-09-19T09:28:07.110 INFO:tasks.workunit.client.0.smithi120.stdout: UnlockX 1244 5.310 131.048 2024-09-19T09:28:07.110 INFO:tasks.workunit.client.0.smithi120.stdout: Flush 26826 9.357 446.356 2024-09-19T09:28:07.110 INFO:tasks.workunit.client.0.smithi120.stdout: 2024-09-19T09:28:07.110 INFO:tasks.workunit.client.0.smithi120.stdout:Throughput 19.8695 MB/sec 10 clients 10 procs max_latency=31914.123 ms 2024-09-19T09:28:07.110 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-19T09:28:07.110 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-19T09:28:07.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.840+0000 7f32c0800640 1 -- 172.21.15.120:0/745386508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32bc103530 msgr2=0x7f32bc10ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:07.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.840+0000 7f32c0800640 1 --2- 172.21.15.120:0/745386508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32bc103530 0x7f32bc10ba20 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009900 tx=0x7f32b002a480 comp rx=0 tx=0).stop 2024-09-19T09:28:07.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.840+0000 7f32c0800640 1 -- 172.21.15.120:0/745386508 shutdown_connections 2024-09-19T09:28:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.840+0000 7f32c0800640 1 --2- 172.21.15.120:0/745386508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32bc10bff0 0x7f32bc10e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.840+0000 7f32c0800640 1 --2- 172.21.15.120:0/745386508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32bc103530 0x7f32bc10ba20 unknown :-1 s=CLOSED pgs=2540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.840+0000 7f32c0800640 1 --2- 172.21.15.120:0/745386508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32bc102c10 0x7f32bc102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.840+0000 7f32c0800640 1 -- 172.21.15.120:0/745386508 >> 172.21.15.120:0/745386508 conn(0x7f32bc0fc610 msgr2=0x7f32bc0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.840+0000 7f32c0800640 1 -- 172.21.15.120:0/745386508 shutdown_connections 2024-09-19T09:28:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.840+0000 7f32c0800640 1 -- 172.21.15.120:0/745386508 wait complete. 2024-09-19T09:28:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.841+0000 7f32c0800640 1 Processor -- start 2024-09-19T09:28:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.841+0000 7f32c0800640 1 -- start start 2024-09-19T09:28:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.842+0000 7f32c0800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32bc102c10 0x7f32bc110cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.842+0000 7f32c0800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32bc103530 0x7f32bc111230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.842+0000 7f32c0800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32bc10bff0 0x7f32bc111770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.842+0000 7f32c0800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f32bc1019a0 con 0x7f32bc10bff0 2024-09-19T09:28:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.842+0000 7f32c0800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f32bc101820 con 0x7f32bc102c10 2024-09-19T09:28:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.842+0000 7f32c0800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f32bc101b20 con 0x7f32bc103530 2024-09-19T09:28:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.842+0000 7f32ba59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32bc103530 0x7f32bc111230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.842+0000 7f32bb59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32bc10bff0 0x7f32bc111770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.842+0000 7f32bb59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32bc10bff0 0x7f32bc111770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43336/0 (socket says 172.21.15.120:43336) 2024-09-19T09:28:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.842+0000 7f32bb59e640 1 -- 172.21.15.120:0/7599483 learned_addr learned my addr 172.21.15.120:0/7599483 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.842+0000 7f32bad9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32bc102c10 0x7f32bc110cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.843+0000 7f32bb59e640 1 -- 172.21.15.120:0/7599483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32bc103530 msgr2=0x7f32bc111230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.843+0000 7f32bb59e640 1 --2- 172.21.15.120:0/7599483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32bc103530 0x7f32bc111230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.843+0000 7f32bb59e640 1 -- 172.21.15.120:0/7599483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32bc102c10 msgr2=0x7f32bc110cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.843+0000 7f32bb59e640 1 --2- 172.21.15.120:0/7599483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32bc102c10 0x7f32bc110cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.843+0000 7f32bb59e640 1 -- 172.21.15.120:0/7599483 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32bc19a360 con 0x7f32bc10bff0 2024-09-19T09:28:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.843+0000 7f32bb59e640 1 --2- 172.21.15.120:0/7599483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32bc10bff0 0x7f32bc111770 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7f32b400eb10 tx=0x7f32b400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.844+0000 7f32a3fff640 1 -- 172.21.15.120:0/7599483 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b4004b20 con 0x7f32bc10bff0 2024-09-19T09:28:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.844+0000 7f32a3fff640 1 -- 172.21.15.120:0/7599483 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32b4004cc0 con 0x7f32bc10bff0 2024-09-19T09:28:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.844+0000 7f32a3fff640 1 -- 172.21.15.120:0/7599483 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b4010720 con 0x7f32bc10bff0 2024-09-19T09:28:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.844+0000 7f32c0800640 1 -- 172.21.15.120:0/7599483 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32bc19a650 con 0x7f32bc10bff0 2024-09-19T09:28:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.845+0000 7f32c0800640 1 -- 172.21.15.120:0/7599483 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32bc19aab0 con 0x7f32bc10bff0 2024-09-19T09:28:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.848+0000 7f32c33da740 1 Processor -- start 2024-09-19T09:28:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.848+0000 7f32a3fff640 1 -- 172.21.15.120:0/7599483 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f32b4005000 con 0x7f32bc10bff0 2024-09-19T09:28:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.848+0000 7f32c33da740 1 -- start start 2024-09-19T09:28:07.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.849+0000 7f32c33da740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb3476dd20 0x55fb3476c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.849+0000 7f32c33da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb3476c7b0 0x55fb3476cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.849+0000 7f32c33da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb34777550 0x55fb34779940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.849+0000 7f32c33da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fb346d22f0 con 0x55fb34777550 2024-09-19T09:28:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.849+0000 7f32c33da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fb34679cf0 con 0x55fb3476c7b0 2024-09-19T09:28:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.849+0000 7f32c33da740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fb346ce2b0 con 0x55fb3476dd20 2024-09-19T09:28:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.849+0000 7f32bad9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb3476c7b0 0x55fb3476cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.849+0000 7f32bad9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb3476c7b0 0x55fb3476cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:60684/0 (socket says 172.21.15.120:60684) 2024-09-19T09:28:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.849+0000 7f32bad9d640 1 -- 172.21.15.120:0/979814297 learned_addr learned my addr 172.21.15.120:0/979814297 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.849+0000 7f32ba59c640 1 --2- 172.21.15.120:0/979814297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb3476dd20 0x55fb3476c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.849+0000 7f32ba59c640 1 --2- 172.21.15.120:0/979814297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb34777550 0x55fb34779940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.849+0000 7f32bad9d640 1 -- 172.21.15.120:0/979814297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb3476dd20 msgr2=0x55fb3476c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.849+0000 7f32bad9d640 1 --2- 172.21.15.120:0/979814297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb3476dd20 0x55fb3476c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.849+0000 7f32bad9d640 1 -- 172.21.15.120:0/979814297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb34777550 msgr2=0x55fb34779940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.849+0000 7f32bad9d640 1 --2- 172.21.15.120:0/979814297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb34777550 0x55fb34779940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.849+0000 7f32bad9d640 1 -- 172.21.15.120:0/979814297 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb3465e3a0 con 0x55fb3476c7b0 2024-09-19T09:28:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.849+0000 7f32ba59c640 1 --2- 172.21.15.120:0/979814297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb34777550 0x55fb34779940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:28:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.850+0000 7f32bad9d640 1 --2- 172.21.15.120:0/979814297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb3476c7b0 0x55fb3476cbf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00b820 tx=0x7f32ac00bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e0699d1cea281d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.850+0000 7f32a3fff640 1 --2- 172.21.15.120:0/7599483 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f32a40a3390 0x7f32a40a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.850+0000 7f32a17fa640 1 -- 172.21.15.120:0/979814297 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac00d950 con 0x55fb3476c7b0 2024-09-19T09:28:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.850+0000 7f32a17fa640 1 -- 172.21.15.120:0/979814297 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32ac019070 con 0x55fb3476c7b0 2024-09-19T09:28:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.850+0000 7f32a3fff640 1 -- 172.21.15.120:0/7599483 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f32b40cfef0 con 0x7f32bc10bff0 2024-09-19T09:28:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.850+0000 7f32ba59c640 1 --2- 172.21.15.120:0/7599483 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f32a40a3390 0x7f32a40a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.850+0000 7f32a17fa640 1 -- 172.21.15.120:0/979814297 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac0123e0 con 0x55fb3476c7b0 2024-09-19T09:28:07.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.851+0000 7f32c33da740 1 -- 172.21.15.120:0/979814297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb3476c7b0 msgr2=0x55fb3476cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:07.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.851+0000 7f32c33da740 1 --2- 172.21.15.120:0/979814297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb3476c7b0 0x55fb3476cbf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00b820 tx=0x7f32ac00bce0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.851+0000 7f32ba59c640 1 --2- 172.21.15.120:0/7599483 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f32a40a3390 0x7f32a40a5850 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f32bc112540 tx=0x7f32b0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:07.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.851+0000 7f32c33da740 1 -- 172.21.15.120:0/979814297 shutdown_connections 2024-09-19T09:28:07.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.851+0000 7f32c33da740 1 --2- 172.21.15.120:0/979814297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb34777550 0x55fb34779940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.851+0000 7f32c33da740 1 --2- 172.21.15.120:0/979814297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb3476c7b0 0x55fb3476cbf0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.851+0000 7f32c33da740 1 --2- 172.21.15.120:0/979814297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb3476dd20 0x55fb3476c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.851+0000 7f32c33da740 1 -- 172.21.15.120:0/979814297 >> 172.21.15.120:0/979814297 conn(0x55fb34735310 msgr2=0x55fb34768e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:07.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.851+0000 7f32c33da740 1 -- 172.21.15.120:0/979814297 shutdown_connections 2024-09-19T09:28:07.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.851+0000 7f32c33da740 1 -- 172.21.15.120:0/979814297 wait complete. 2024-09-19T09:28:07.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.852+0000 7f32c33da740 1 -- start start 2024-09-19T09:28:07.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.852+0000 7f32c33da740 1 Processor -- start 2024-09-19T09:28:07.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.852+0000 7f32c33da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb3476c7b0 0x55fb34776ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.852+0000 7f32c33da740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb3476dd20 0x55fb347741a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.852+0000 7f32c33da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb34777550 0x55fb34774740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.852+0000 7f32c33da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fb3477a2a0 con 0x55fb34777550 2024-09-19T09:28:07.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.852+0000 7f32c33da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fb3468d060 con 0x55fb3476c7b0 2024-09-19T09:28:07.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.852+0000 7f32c33da740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fb3477a420 con 0x55fb3476dd20 2024-09-19T09:28:07.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.852+0000 7f32c33da740 10 client.0 _ll_register_callbacks cb 0x55fb343e08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:07.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.852+0000 7f32ba59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb3476dd20 0x55fb347741a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.852+0000 7f32ba59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb3476dd20 0x55fb347741a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:50450/0 (socket says 172.21.15.120:50450) 2024-09-19T09:28:07.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.852+0000 7f32ba59c640 1 -- 172.21.15.120:0/2238436125 learned_addr learned my addr 172.21.15.120:0/2238436125 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:07.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.853+0000 7f32bb59e640 1 --2- 172.21.15.120:0/2238436125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb34777550 0x55fb34774740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.853+0000 7f32bad9d640 1 --2- 172.21.15.120:0/2238436125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb3476c7b0 0x55fb34776ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.853+0000 7f32ba59c640 1 -- 172.21.15.120:0/2238436125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb3476c7b0 msgr2=0x55fb34776ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:07.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.853+0000 7f32ba59c640 1 --2- 172.21.15.120:0/2238436125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb3476c7b0 0x55fb34776ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.853+0000 7f32ba59c640 1 -- 172.21.15.120:0/2238436125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb34777550 msgr2=0x55fb34774740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:07.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.853+0000 7f32ba59c640 1 --2- 172.21.15.120:0/2238436125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb34777550 0x55fb34774740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.853+0000 7f32ba59c640 1 -- 172.21.15.120:0/2238436125 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb344433c0 con 0x55fb3476dd20 2024-09-19T09:28:07.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.853+0000 7f32ba59c640 1 --2- 172.21.15.120:0/2238436125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb3476dd20 0x55fb347741a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f32b0033000 tx=0x7f32b002a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=821bca30d4819485 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:07.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.853+0000 7f329eff5640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-19T09:28:07.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.854+0000 7f329eff5640 1 -- 172.21.15.120:0/2238436125 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b0035050 con 0x55fb3476dd20 2024-09-19T09:28:07.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.854+0000 7f329eff5640 1 -- 172.21.15.120:0/2238436125 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f32b00048c0 con 0x55fb3476dd20 2024-09-19T09:28:07.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.854+0000 7f32c33da740 1 -- 172.21.15.120:0/2238436125 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32ac019070 con 0x55fb3476dd20 2024-09-19T09:28:07.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.854+0000 7f329eff5640 1 -- 172.21.15.120:0/2238436125 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f32b0004ac0 con 0x55fb3476dd20 2024-09-19T09:28:07.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.854+0000 7f32c33da740 10 client.32062 fetch_fsmap learned FSMap version 39 2024-09-19T09:28:07.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.854+0000 7f32c33da740 1 -- 172.21.15.120:0/2238436125 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55fb34785c60 con 0x55fb3476dd20 2024-09-19T09:28:07.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.854+0000 7f329eff5640 1 -- 172.21.15.120:0/2238436125 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32ac019070 con 0x55fb3476dd20 2024-09-19T09:28:07.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.854+0000 7f329eff5640 1 -- 172.21.15.120:0/2238436125 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f32b003a040 con 0x55fb3476dd20 2024-09-19T09:28:07.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.855+0000 7f32c33da740 10 client.32062 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:28:07.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.855+0000 7f32c33da740 10 client.32062 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:28:07.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.855+0000 7f32c33da740 20 client.32062 populate_metadata read hostname 'smithi120' 2024-09-19T09:28:07.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.855+0000 7f32c33da740 1 --2- 172.21.15.120:0/2238436125 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55fb347870c0 0x55fb34789520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.855+0000 7f32c33da740 4 client.32062 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:28:07.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.855+0000 7f32c33da740 1 -- 172.21.15.120:0/2238436125 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb346df860 con 0x55fb347870c0 2024-09-19T09:28:07.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.855+0000 7f32bad9d640 1 --2- 172.21.15.120:0/2238436125 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55fb347870c0 0x55fb34789520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.856+0000 7f32bad9d640 1 --2- 172.21.15.120:0/2238436125 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55fb347870c0 0x55fb34789520 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9c9bb2d07184cdf4 server_cookie=5968278b24593322 in_seq=0 out_seq=0 2024-09-19T09:28:07.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.856+0000 7f329eff5640 10 client.32062 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:28:07.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.857+0000 7f329eff5640 1 -- 172.21.15.120:0/2238436125 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb346df860 con 0x55fb347870c0 2024-09-19T09:28:07.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.857+0000 7f329eff5640 10 client.32062 handle_command_reply: tid=0 2024-09-19T09:28:07.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.857+0000 7f32c33da740 1 client.32062 shutdown 2024-09-19T09:28:07.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.857+0000 7f32c33da740 1 -- 172.21.15.120:0/2238436125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb3476dd20 msgr2=0x55fb347741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:07.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.857+0000 7f32c33da740 1 --2- 172.21.15.120:0/2238436125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb3476dd20 0x55fb347741a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f32b0033000 tx=0x7f32b002a480 comp rx=0 tx=0).stop 2024-09-19T09:28:07.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.857+0000 7f32c33da740 1 -- 172.21.15.120:0/2238436125 shutdown_connections 2024-09-19T09:28:07.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.857+0000 7f32c33da740 1 --2- 172.21.15.120:0/2238436125 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55fb347870c0 0x55fb34789520 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.857+0000 7f32c33da740 1 --2- 172.21.15.120:0/2238436125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb34777550 0x55fb34774740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.857+0000 7f32c33da740 1 --2- 172.21.15.120:0/2238436125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb3476dd20 0x55fb347741a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.858+0000 7f32c33da740 1 --2- 172.21.15.120:0/2238436125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb3476c7b0 0x55fb34776ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.858+0000 7f32c33da740 1 -- 172.21.15.120:0/2238436125 >> 172.21.15.120:0/2238436125 conn(0x55fb34735310 msgr2=0x55fb3476eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:07.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.858+0000 7f329eff5640 0 client.32062 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:28:07.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.858+0000 7f32c33da740 1 -- 172.21.15.120:0/2238436125 shutdown_connections 2024-09-19T09:28:07.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.858+0000 7f32c33da740 1 -- 172.21.15.120:0/2238436125 wait complete. 2024-09-19T09:28:07.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.858+0000 7f32c33da740 20 client.32062 trim_cache size 0 max 16384 2024-09-19T09:28:07.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.875+0000 7f32c33da740 1 Processor -- start 2024-09-19T09:28:07.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.875+0000 7f32c33da740 1 -- start start 2024-09-19T09:28:07.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.875+0000 7f32c33da740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb3476c7b0 0x55fb34774170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.875+0000 7f32c33da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb3476dd20 0x55fb34783540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.875+0000 7f32c33da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb34777550 0x55fb3476f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.875+0000 7f32c33da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f32bc1be500 con 0x55fb34777550 2024-09-19T09:28:07.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.875+0000 7f32c33da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f32bc1bc2f0 con 0x55fb3476dd20 2024-09-19T09:28:07.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.875+0000 7f32c33da740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f32bc1bde30 con 0x55fb3476c7b0 2024-09-19T09:28:07.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.875+0000 7f32bad9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb3476c7b0 0x55fb34774170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.875+0000 7f32bad9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb3476c7b0 0x55fb34774170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.120:50456/0 (socket says 172.21.15.120:50456) 2024-09-19T09:28:07.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.875+0000 7f32bad9d640 1 -- 172.21.15.120:0/1954198830 learned_addr learned my addr 172.21.15.120:0/1954198830 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:07.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.875+0000 7f32bb59e640 1 --2- 172.21.15.120:0/1954198830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb34777550 0x55fb3476f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.875+0000 7f32bb59e640 1 -- 172.21.15.120:0/1954198830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb3476c7b0 msgr2=0x55fb34774170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:07.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.876+0000 7f32bb59e640 1 --2- 172.21.15.120:0/1954198830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb3476c7b0 0x55fb34774170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.876+0000 7f32bb59e640 1 -- 172.21.15.120:0/1954198830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb3476dd20 msgr2=0x55fb34783540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:07.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.876+0000 7f32bb59e640 1 --2- 172.21.15.120:0/1954198830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb3476dd20 0x55fb34783540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.876+0000 7f32bb59e640 1 -- 172.21.15.120:0/1954198830 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb347726f0 con 0x55fb34777550 2024-09-19T09:28:07.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.876+0000 7f32bad9d640 1 --2- 172.21.15.120:0/1954198830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb3476c7b0 0x55fb34774170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:07.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.876+0000 7f32bb59e640 1 --2- 172.21.15.120:0/1954198830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb34777550 0x55fb3476f300 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7f32b409c210 tx=0x7f32b4000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256f863a63ed2476 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:07.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.876+0000 7f32a0ff9640 1 -- 172.21.15.120:0/1954198830 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b40ba070 con 0x55fb34777550 2024-09-19T09:28:07.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.877+0000 7f32a0ff9640 1 -- 172.21.15.120:0/1954198830 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32b409c510 con 0x55fb34777550 2024-09-19T09:28:07.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.877+0000 7f32a0ff9640 1 -- 172.21.15.120:0/1954198830 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b40b52b0 con 0x55fb34777550 2024-09-19T09:28:07.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.877+0000 7f32c33da740 1 -- 172.21.15.120:0/1954198830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb34777550 msgr2=0x55fb3476f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:07.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.877+0000 7f32c33da740 1 --2- 172.21.15.120:0/1954198830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb34777550 0x55fb3476f300 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7f32b409c210 tx=0x7f32b4000f40 comp rx=0 tx=0).stop 2024-09-19T09:28:07.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.878+0000 7f32c33da740 1 -- 172.21.15.120:0/1954198830 shutdown_connections 2024-09-19T09:28:07.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.878+0000 7f32c33da740 1 --2- 172.21.15.120:0/1954198830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb34777550 0x55fb3476f300 unknown :-1 s=CLOSED pgs=2542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.878+0000 7f32c33da740 1 --2- 172.21.15.120:0/1954198830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb3476dd20 0x55fb34783540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.878+0000 7f32c33da740 1 --2- 172.21.15.120:0/1954198830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb3476c7b0 0x55fb34774170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.878+0000 7f32c33da740 1 -- 172.21.15.120:0/1954198830 >> 172.21.15.120:0/1954198830 conn(0x55fb34735310 msgr2=0x55fb34773a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:07.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.878+0000 7f32c33da740 1 -- 172.21.15.120:0/1954198830 shutdown_connections 2024-09-19T09:28:07.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.878+0000 7f32c33da740 1 -- 172.21.15.120:0/1954198830 wait complete. 2024-09-19T09:28:07.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.879+0000 7f32c33da740 1 -- start start 2024-09-19T09:28:07.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.879+0000 7f32c33da740 1 Processor -- start 2024-09-19T09:28:07.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.879+0000 7f32c33da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb3476c7b0 0x55fb3477feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.879+0000 7f32c33da740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb3476dd20 0x55fb34770fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.879+0000 7f32c33da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb34777550 0x55fb34771540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.879+0000 7f32bad9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb3476c7b0 0x55fb3477feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.879+0000 7f32bad9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb3476c7b0 0x55fb3477feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:60718/0 (socket says 172.21.15.120:60718) 2024-09-19T09:28:07.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.879+0000 7f32bad9d640 1 -- 172.21.15.120:0/566180672 learned_addr learned my addr 172.21.15.120:0/566180672 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:07.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.879+0000 7f32c33da740 1 -- 172.21.15.120:0/566180672 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f32bc1bd090 con 0x55fb34777550 2024-09-19T09:28:07.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.879+0000 7f32c33da740 1 -- 172.21.15.120:0/566180672 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f32bc1bc9c0 con 0x55fb3476c7b0 2024-09-19T09:28:07.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.879+0000 7f32c33da740 1 -- 172.21.15.120:0/566180672 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f32bc1bd760 con 0x55fb3476dd20 2024-09-19T09:28:07.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.879+0000 7f32bb59e640 1 --2- 172.21.15.120:0/566180672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb34777550 0x55fb34771540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.879+0000 7f32c33da740 10 client.0 _ll_register_callbacks cb 0x7f32b40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:07.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.880+0000 7f32ba59c640 1 --2- 172.21.15.120:0/566180672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb3476dd20 0x55fb34770fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.880+0000 7f32bb59e640 1 -- 172.21.15.120:0/566180672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb3476dd20 msgr2=0x55fb34770fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:07.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.880+0000 7f32bb59e640 1 --2- 172.21.15.120:0/566180672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb3476dd20 0x55fb34770fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.880+0000 7f32bb59e640 1 -- 172.21.15.120:0/566180672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb3476c7b0 msgr2=0x55fb3477feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:07.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.880+0000 7f32bb59e640 1 --2- 172.21.15.120:0/566180672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb3476c7b0 0x55fb3477feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.880+0000 7f32bb59e640 1 -- 172.21.15.120:0/566180672 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb34771d70 con 0x55fb34777550 2024-09-19T09:28:07.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.880+0000 7f32bb59e640 1 --2- 172.21.15.120:0/566180672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb34777550 0x55fb34771540 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7f32b40b4000 tx=0x7f32b4005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90382a506e1a1c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:07.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.880+0000 7f329eff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:28:07.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.881+0000 7f329eff5640 1 -- 172.21.15.120:0/566180672 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b40aa020 con 0x55fb34777550 2024-09-19T09:28:07.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.881+0000 7f32c33da740 1 -- 172.21.15.120:0/566180672 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32b409c510 con 0x55fb34777550 2024-09-19T09:28:07.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.881+0000 7f329eff5640 1 -- 172.21.15.120:0/566180672 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f32b40b54b0 con 0x55fb34777550 2024-09-19T09:28:07.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.881+0000 7f329eff5640 1 -- 172.21.15.120:0/566180672 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f32b40ba050 con 0x55fb34777550 2024-09-19T09:28:07.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.881+0000 7f32c33da740 10 client.23619 fetch_fsmap learned FSMap version 39 2024-09-19T09:28:07.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.881+0000 7f32c33da740 1 -- 172.21.15.120:0/566180672 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55fb347f1e90 con 0x55fb34777550 2024-09-19T09:28:07.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.881+0000 7f329eff5640 1 -- 172.21.15.120:0/566180672 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32b409c510 con 0x55fb34777550 2024-09-19T09:28:07.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.882+0000 7f329eff5640 1 -- 172.21.15.120:0/566180672 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f32b409b960 con 0x55fb34777550 2024-09-19T09:28:07.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.882+0000 7f32c33da740 10 client.23619 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T09:28:07.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.882+0000 7f32c33da740 10 client.23619 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T09:28:07.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.882+0000 7f32c33da740 20 client.23619 populate_metadata read hostname 'smithi120' 2024-09-19T09:28:07.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.882+0000 7f32c33da740 1 --2- 172.21.15.120:0/566180672 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55fb347f24b0 0x55fb347f48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.882+0000 7f32c33da740 4 client.23619 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:28:07.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.882+0000 7f32c33da740 1 -- 172.21.15.120:0/566180672 --> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fb346d9d70 con 0x55fb347f24b0 2024-09-19T09:28:07.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.882+0000 7f32bad9d640 1 --2- 172.21.15.120:0/566180672 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55fb347f24b0 0x55fb347f48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.883+0000 7f32bad9d640 1 --2- 172.21.15.120:0/566180672 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55fb347f24b0 0x55fb347f48a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=58df83d7111482e8 server_cookie=10ca2015c8d8db3c in_seq=0 out_seq=0 2024-09-19T09:28:07.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.883+0000 7f329eff5640 10 client.23619 ms_handle_connect on v2:172.21.15.196:6838/862542635 2024-09-19T09:28:07.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.883+0000 7f329eff5640 1 -- 172.21.15.120:0/566180672 <== mds.2 v2:172.21.15.196:6838/862542635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fb346d9d70 con 0x55fb347f24b0 2024-09-19T09:28:07.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.883+0000 7f329eff5640 10 client.23619 handle_command_reply: tid=0 2024-09-19T09:28:07.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.883+0000 7f32c33da740 1 client.23619 shutdown 2024-09-19T09:28:07.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.884+0000 7f32c33da740 1 -- 172.21.15.120:0/566180672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb34777550 msgr2=0x55fb34771540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:07.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.884+0000 7f32c33da740 1 --2- 172.21.15.120:0/566180672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb34777550 0x55fb34771540 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7f32b40b4000 tx=0x7f32b4005ea0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.884+0000 7f32c33da740 1 -- 172.21.15.120:0/566180672 shutdown_connections 2024-09-19T09:28:07.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.884+0000 7f32c33da740 1 --2- 172.21.15.120:0/566180672 >> [v2:172.21.15.196:6838/862542635,v1:172.21.15.196:6839/862542635] conn(0x55fb347f24b0 0x55fb347f48a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.884+0000 7f32c33da740 1 --2- 172.21.15.120:0/566180672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fb34777550 0x55fb34771540 unknown :-1 s=CLOSED pgs=2543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.884+0000 7f32c33da740 1 --2- 172.21.15.120:0/566180672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb3476dd20 0x55fb34770fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.884+0000 7f32c33da740 1 --2- 172.21.15.120:0/566180672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fb3476c7b0 0x55fb3477feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.884+0000 7f32c33da740 1 -- 172.21.15.120:0/566180672 >> 172.21.15.120:0/566180672 conn(0x55fb34735310 msgr2=0x55fb34775a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:07.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.884+0000 7f329eff5640 0 client.23619 ms_handle_reset on v2:172.21.15.196:6838/862542635 2024-09-19T09:28:07.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.884+0000 7f32c33da740 1 -- 172.21.15.120:0/566180672 shutdown_connections 2024-09-19T09:28:07.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.884+0000 7f32c33da740 1 -- 172.21.15.120:0/566180672 wait complete. 2024-09-19T09:28:07.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.884+0000 7f32c33da740 20 client.23619 trim_cache size 0 max 16384 2024-09-19T09:28:07.891 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:28:07.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.885+0000 7f32c0800640 1 -- 172.21.15.120:0/7599483 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f32a40a3390 msgr2=0x7f32a40a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:07.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.885+0000 7f32c0800640 1 --2- 172.21.15.120:0/7599483 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f32a40a3390 0x7f32a40a5850 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f32bc112540 tx=0x7f32b0009260 comp rx=0 tx=0).stop 2024-09-19T09:28:07.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.885+0000 7f32c0800640 1 -- 172.21.15.120:0/7599483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32bc10bff0 msgr2=0x7f32bc111770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:07.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.885+0000 7f32c0800640 1 --2- 172.21.15.120:0/7599483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32bc10bff0 0x7f32bc111770 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7f32b400eb10 tx=0x7f32b400c2d0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.885+0000 7f32c0800640 1 -- 172.21.15.120:0/7599483 shutdown_connections 2024-09-19T09:28:07.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.885+0000 7f32c0800640 1 --2- 172.21.15.120:0/7599483 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f32a40a3390 0x7f32a40a5850 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.885+0000 7f32c0800640 1 --2- 172.21.15.120:0/7599483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32bc10bff0 0x7f32bc111770 unknown :-1 s=CLOSED pgs=2541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.885+0000 7f32c0800640 1 --2- 172.21.15.120:0/7599483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32bc103530 0x7f32bc111230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.885+0000 7f32c0800640 1 --2- 172.21.15.120:0/7599483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32bc102c10 0x7f32bc110cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.885+0000 7f32c0800640 1 -- 172.21.15.120:0/7599483 >> 172.21.15.120:0/7599483 conn(0x7f32bc0fc610 msgr2=0x7f32bc10d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:07.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.886+0000 7f32c0800640 1 -- 172.21.15.120:0/7599483 shutdown_connections 2024-09-19T09:28:07.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:07.886+0000 7f32c0800640 1 -- 172.21.15.120:0/7599483 wait complete. 2024-09-19T09:28:08.029 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-09-19T09:28:08.030 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-19T09:28:08.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:08 smithi120 ceph-mon[31150]: pgmap v1103: 97 pgs: 97 active+clean; 461 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.4 MiB/s wr, 547 op/s 2024-09-19T09:28:08.340 DEBUG:teuthology.parallel:result is None 2024-09-19T09:28:08.341 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T09:28:08.374 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T09:28:08.374 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-19T09:28:08.387 INFO:tasks.quiescer:joining Quiescers 2024-09-19T09:28:08.388 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-19T09:28:08.388 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:28:08.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:08 smithi196 ceph-mon[35150]: pgmap v1103: 97 pgs: 97 active+clean; 461 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.4 MiB/s wr, 547 op/s 2024-09-19T09:28:08.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:08 smithi153 ceph-mon[35071]: pgmap v1103: 97 pgs: 97 active+clean; 461 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.4 MiB/s wr, 547 op/s 2024-09-19T09:28:08.871 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:28:08.995 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:09.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.519+0000 7fd9c52be640 1 -- 172.21.15.120:0/3701446104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9c0100e20 msgr2=0x7fd9c0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:09.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.519+0000 7fd9c52be640 1 --2- 172.21.15.120:0/3701446104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9c0100e20 0x7fd9c0101200 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8009900 tx=0x7fd9b802a480 comp rx=0 tx=0).stop 2024-09-19T09:28:09.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.519+0000 7fd9c52be640 1 -- 172.21.15.120:0/3701446104 shutdown_connections 2024-09-19T09:28:09.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.519+0000 7fd9c52be640 1 --2- 172.21.15.120:0/3701446104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9c0102ef0 0x7fd9c010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.519+0000 7fd9c52be640 1 --2- 172.21.15.120:0/3701446104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9c0102570 0x7fd9c01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.519+0000 7fd9c52be640 1 --2- 172.21.15.120:0/3701446104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9c0100e20 0x7fd9c0101200 unknown :-1 s=CLOSED pgs=2544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.519+0000 7fd9c52be640 1 -- 172.21.15.120:0/3701446104 >> 172.21.15.120:0/3701446104 conn(0x7fd9c00fc7d0 msgr2=0x7fd9c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:09.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.519+0000 7fd9c52be640 1 -- 172.21.15.120:0/3701446104 shutdown_connections 2024-09-19T09:28:09.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.519+0000 7fd9c52be640 1 -- 172.21.15.120:0/3701446104 wait complete. 2024-09-19T09:28:09.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.519+0000 7fd9c52be640 1 Processor -- start 2024-09-19T09:28:09.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.519+0000 7fd9c52be640 1 -- start start 2024-09-19T09:28:09.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.520+0000 7fd9c52be640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9c0100e20 0x7fd9c00fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:09.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.520+0000 7fd9c52be640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9c0102570 0x7fd9c00fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:09.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.520+0000 7fd9c52be640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9c0102ef0 0x7fd9c00fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:09.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.520+0000 7fd9c52be640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c0110f80 con 0x7fd9c0100e20 2024-09-19T09:28:09.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.520+0000 7fd9c52be640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c0110e00 con 0x7fd9c0102ef0 2024-09-19T09:28:09.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.520+0000 7fd9c52be640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c0111100 con 0x7fd9c0102570 2024-09-19T09:28:09.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.520+0000 7fd9beffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9c0100e20 0x7fd9c00fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:09.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.520+0000 7fd9beffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9c0100e20 0x7fd9c00fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43382/0 (socket says 172.21.15.120:43382) 2024-09-19T09:28:09.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.520+0000 7fd9beffd640 1 -- 172.21.15.120:0/3578546640 learned_addr learned my addr 172.21.15.120:0/3578546640 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:09.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.520+0000 7fd9be7fc640 1 --2- 172.21.15.120:0/3578546640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9c0102570 0x7fd9c00fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:09.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.520+0000 7fd9bf7fe640 1 --2- 172.21.15.120:0/3578546640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9c0102ef0 0x7fd9c00fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:09.885 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:28:09.885 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":40,"btime":"2024-09-19T09:28:09:100862+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:28:08.506554+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:28:09.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.520+0000 7fd9beffd640 1 -- 172.21.15.120:0/3578546640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9c0102570 msgr2=0x7fd9c00fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:09.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.520+0000 7fd9beffd640 1 --2- 172.21.15.120:0/3578546640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9c0102570 0x7fd9c00fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.520+0000 7fd9beffd640 1 -- 172.21.15.120:0/3578546640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9c0102ef0 msgr2=0x7fd9c00fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:09.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.520+0000 7fd9beffd640 1 --2- 172.21.15.120:0/3578546640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9c0102ef0 0x7fd9c00fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.520+0000 7fd9beffd640 1 -- 172.21.15.120:0/3578546640 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c01a6a60 con 0x7fd9c0100e20 2024-09-19T09:28:09.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.520+0000 7fd9beffd640 1 --2- 172.21.15.120:0/3578546640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9c0100e20 0x7fd9c00fcc80 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8006010 tx=0x7fd9b8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:09.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.521+0000 7fd9b7fff640 1 -- 172.21.15.120:0/3578546640 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b803f070 con 0x7fd9c0100e20 2024-09-19T09:28:09.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.521+0000 7fd9b7fff640 1 -- 172.21.15.120:0/3578546640 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9b8031c30 con 0x7fd9c0100e20 2024-09-19T09:28:09.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.521+0000 7fd9c52be640 1 -- 172.21.15.120:0/3578546640 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9c01a6cf0 con 0x7fd9c0100e20 2024-09-19T09:28:09.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.521+0000 7fd9b7fff640 1 -- 172.21.15.120:0/3578546640 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8035030 con 0x7fd9c0100e20 2024-09-19T09:28:09.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.521+0000 7fd9c52be640 1 -- 172.21.15.120:0/3578546640 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9c01a71c0 con 0x7fd9c0100e20 2024-09-19T09:28:09.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.522+0000 7fd9c52be640 1 -- 172.21.15.120:0/3578546640 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9c0101200 con 0x7fd9c0100e20 2024-09-19T09:28:09.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.523+0000 7fd9b7fff640 1 -- 172.21.15.120:0/3578546640 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd9b8042070 con 0x7fd9c0100e20 2024-09-19T09:28:09.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.523+0000 7fd9b7fff640 1 --2- 172.21.15.120:0/3578546640 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd9ac0a32c0 0x7fd9ac0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:09.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.524+0000 7fd9b7fff640 1 -- 172.21.15.120:0/3578546640 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd9b80f0450 con 0x7fd9c0100e20 2024-09-19T09:28:09.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.524+0000 7fd9be7fc640 1 --2- 172.21.15.120:0/3578546640 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd9ac0a32c0 0x7fd9ac0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:09.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.524+0000 7fd9be7fc640 1 --2- 172.21.15.120:0/3578546640 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd9ac0a32c0 0x7fd9ac0a5780 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fd9c00fe440 tx=0x7fd9a800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:09.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.527+0000 7fd9b7fff640 1 -- 172.21.15.120:0/3578546640 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd9b80bd170 con 0x7fd9c0100e20 2024-09-19T09:28:09.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.753+0000 7fd9c52be640 1 -- 172.21.15.120:0/3578546640 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9c01a73f0 con 0x7fd9c0100e20 2024-09-19T09:28:09.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.754+0000 7fd9b7fff640 1 -- 172.21.15.120:0/3578546640 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11989 (secure 0 0 0) 0x7fd9b80bca70 con 0x7fd9c0100e20 2024-09-19T09:28:09.887 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 40 2024-09-19T09:28:09.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.757+0000 7fd9c52be640 1 -- 172.21.15.120:0/3578546640 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd9ac0a32c0 msgr2=0x7fd9ac0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.757+0000 7fd9c52be640 1 --2- 172.21.15.120:0/3578546640 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd9ac0a32c0 0x7fd9ac0a5780 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fd9c00fe440 tx=0x7fd9a800a2d0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.757+0000 7fd9c52be640 1 -- 172.21.15.120:0/3578546640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9c0100e20 msgr2=0x7fd9c00fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.757+0000 7fd9c52be640 1 --2- 172.21.15.120:0/3578546640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9c0100e20 0x7fd9c00fcc80 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8006010 tx=0x7fd9b8003c10 comp rx=0 tx=0).stop 2024-09-19T09:28:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.759+0000 7fd9c52be640 1 -- 172.21.15.120:0/3578546640 shutdown_connections 2024-09-19T09:28:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.759+0000 7fd9c52be640 1 --2- 172.21.15.120:0/3578546640 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fd9ac0a32c0 0x7fd9ac0a5780 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.759+0000 7fd9c52be640 1 --2- 172.21.15.120:0/3578546640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9c0102ef0 0x7fd9c00fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.759+0000 7fd9c52be640 1 --2- 172.21.15.120:0/3578546640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9c0102570 0x7fd9c00fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.759+0000 7fd9c52be640 1 --2- 172.21.15.120:0/3578546640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9c0100e20 0x7fd9c00fcc80 unknown :-1 s=CLOSED pgs=2545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.759+0000 7fd9c52be640 1 -- 172.21.15.120:0/3578546640 >> 172.21.15.120:0/3578546640 conn(0x7fd9c00fc7d0 msgr2=0x7fd9c0069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.760+0000 7fd9c52be640 1 -- 172.21.15.120:0/3578546640 shutdown_connections 2024-09-19T09:28:09.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:09.760+0000 7fd9c52be640 1 -- 172.21.15.120:0/3578546640 wait complete. 2024-09-19T09:28:10.130 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:10.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:10 smithi196 ceph-mon[35150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:28:10.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:10 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3578546640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:28:10.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:10 smithi196 ceph-mon[35150]: pgmap v1104: 97 pgs: 97 active+clean; 423 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 4.9 MiB/s wr, 577 op/s 2024-09-19T09:28:10.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:10 smithi153 ceph-mon[35071]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:28:10.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:10 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3578546640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:28:10.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:10 smithi153 ceph-mon[35071]: pgmap v1104: 97 pgs: 97 active+clean; 423 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 4.9 MiB/s wr, 577 op/s 2024-09-19T09:28:10.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:10 smithi120 ceph-mon[31150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:28:10.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:10 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3578546640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:28:10.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:10 smithi120 ceph-mon[31150]: pgmap v1104: 97 pgs: 97 active+clean; 423 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 4.9 MiB/s wr, 577 op/s 2024-09-19T09:28:10.700 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24461 quiesce db --cancel --all' 2024-09-19T09:28:10.701 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24461 quiesce db --cancel --all 2024-09-19T09:28:11.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.163+0000 7fa737cd8640 1 -- 172.21.15.120:0/3149378665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa730100e20 msgr2=0x7fa730101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:11.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.163+0000 7fa737cd8640 1 --2- 172.21.15.120:0/3149378665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa730100e20 0x7fa730101200 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7fa720009ca0 tx=0x7fa72002a950 comp rx=0 tx=0).stop 2024-09-19T09:28:11.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.163+0000 7fa737cd8640 1 -- 172.21.15.120:0/3149378665 shutdown_connections 2024-09-19T09:28:11.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.163+0000 7fa737cd8640 1 --2- 172.21.15.120:0/3149378665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa730102ef0 0x7fa73010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.163+0000 7fa737cd8640 1 --2- 172.21.15.120:0/3149378665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa730102570 0x7fa7301029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.163+0000 7fa737cd8640 1 --2- 172.21.15.120:0/3149378665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa730100e20 0x7fa730101200 unknown :-1 s=CLOSED pgs=2546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.163+0000 7fa737cd8640 1 -- 172.21.15.120:0/3149378665 >> 172.21.15.120:0/3149378665 conn(0x7fa7300fc7d0 msgr2=0x7fa7300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:11.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.164+0000 7fa737cd8640 1 -- 172.21.15.120:0/3149378665 shutdown_connections 2024-09-19T09:28:11.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.164+0000 7fa737cd8640 1 -- 172.21.15.120:0/3149378665 wait complete. 2024-09-19T09:28:11.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.164+0000 7fa737cd8640 1 Processor -- start 2024-09-19T09:28:11.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.164+0000 7fa737cd8640 1 -- start start 2024-09-19T09:28:11.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.164+0000 7fa737cd8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa730100e20 0x7fa7301971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.164+0000 7fa737cd8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa730102570 0x7fa730197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.164+0000 7fa737cd8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa730102ef0 0x7fa730197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.164+0000 7fa737cd8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa730110ff0 con 0x7fa730102ef0 2024-09-19T09:28:11.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.164+0000 7fa737cd8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa730110e70 con 0x7fa730100e20 2024-09-19T09:28:11.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.164+0000 7fa737cd8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa730111170 con 0x7fa730102570 2024-09-19T09:28:11.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.164+0000 7fa736275640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa730102ef0 0x7fa730197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.164+0000 7fa736275640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa730102ef0 0x7fa730197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43414/0 (socket says 172.21.15.120:43414) 2024-09-19T09:28:11.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.164+0000 7fa736275640 1 -- 172.21.15.120:0/3323035060 learned_addr learned my addr 172.21.15.120:0/3323035060 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:11.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.165+0000 7fa735a74640 1 --2- 172.21.15.120:0/3323035060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa730100e20 0x7fa7301971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.165+0000 7fa736275640 1 -- 172.21.15.120:0/3323035060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa730102570 msgr2=0x7fa730197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:11.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.165+0000 7fa735273640 1 --2- 172.21.15.120:0/3323035060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa730102570 0x7fa730197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.165+0000 7fa736275640 1 --2- 172.21.15.120:0/3323035060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa730102570 0x7fa730197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.165+0000 7fa736275640 1 -- 172.21.15.120:0/3323035060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa730100e20 msgr2=0x7fa7301971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:11.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.165+0000 7fa736275640 1 --2- 172.21.15.120:0/3323035060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa730100e20 0x7fa7301971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.165+0000 7fa736275640 1 -- 172.21.15.120:0/3323035060 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7301a6a60 con 0x7fa730102ef0 2024-09-19T09:28:11.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.165+0000 7fa735a74640 1 --2- 172.21.15.120:0/3323035060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa730100e20 0x7fa7301971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:11.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.165+0000 7fa735273640 1 --2- 172.21.15.120:0/3323035060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa730102570 0x7fa730197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:28:11.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.165+0000 7fa736275640 1 --2- 172.21.15.120:0/3323035060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa730102ef0 0x7fa730197c70 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7fa72800c970 tx=0x7fa72800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:11.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.165+0000 7fa72effd640 1 -- 172.21.15.120:0/3323035060 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa728007bb0 con 0x7fa730102ef0 2024-09-19T09:28:11.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.165+0000 7fa72effd640 1 -- 172.21.15.120:0/3323035060 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa728007d50 con 0x7fa730102ef0 2024-09-19T09:28:11.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.165+0000 7fa737cd8640 1 -- 172.21.15.120:0/3323035060 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7301a6d50 con 0x7fa730102ef0 2024-09-19T09:28:11.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.165+0000 7fa72effd640 1 -- 172.21.15.120:0/3323035060 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72800f450 con 0x7fa730102ef0 2024-09-19T09:28:11.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.165+0000 7fa737cd8640 1 -- 172.21.15.120:0/3323035060 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7301a72c0 con 0x7fa730102ef0 2024-09-19T09:28:11.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.167+0000 7fa72effd640 1 -- 172.21.15.120:0/3323035060 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa728004f60 con 0x7fa730102ef0 2024-09-19T09:28:11.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.168+0000 7fa73a0b1740 1 Processor -- start 2024-09-19T09:28:11.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.168+0000 7fa73a0b1740 1 -- start start 2024-09-19T09:28:11.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.168+0000 7fa73a0b1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646dabd8b40 0x5646dabd7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.168+0000 7fa73a0b1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646dabd75d0 0x5646dabd7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.168+0000 7fa73a0b1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646dabe2370 0x5646dabe4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.168+0000 7fa72effd640 1 --2- 172.21.15.120:0/3323035060 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa7180a3240 0x7fa7180a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.168+0000 7fa73a0b1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5646dab3d2f0 con 0x5646dabd75d0 2024-09-19T09:28:11.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.168+0000 7fa73a0b1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5646daae4cf0 con 0x5646dabd8b40 2024-09-19T09:28:11.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.168+0000 7fa73a0b1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5646dab392b0 con 0x5646dabe2370 2024-09-19T09:28:11.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.168+0000 7fa736275640 1 --2- 172.21.15.120:0/3323035060 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa7180a3240 0x7fa7180a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.168+0000 7fa735a74640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646dabd75d0 0x5646dabd7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.168+0000 7fa735a74640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646dabd75d0 0x5646dabd7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43424/0 (socket says 172.21.15.120:43424) 2024-09-19T09:28:11.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.168+0000 7fa735a74640 1 -- 172.21.15.120:0/303829202 learned_addr learned my addr 172.21.15.120:0/303829202 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:11.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.168+0000 7fa735273640 1 --2- 172.21.15.120:0/303829202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646dabd8b40 0x5646dabd7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.168+0000 7fa72effd640 1 -- 172.21.15.120:0/3323035060 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa7280cf730 con 0x7fa730102ef0 2024-09-19T09:28:11.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.168+0000 7fa735273640 1 --2- 172.21.15.120:0/303829202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646dabe2370 0x5646dabe4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.169+0000 7fa735a74640 1 -- 172.21.15.120:0/303829202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646dabe2370 msgr2=0x5646dabe4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:11.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.169+0000 7fa735a74640 1 --2- 172.21.15.120:0/303829202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646dabe2370 0x5646dabe4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.169+0000 7fa735a74640 1 -- 172.21.15.120:0/303829202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646dabd8b40 msgr2=0x5646dabd7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:11.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.169+0000 7fa735a74640 1 --2- 172.21.15.120:0/303829202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646dabd8b40 0x5646dabd7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.169+0000 7fa735a74640 1 -- 172.21.15.120:0/303829202 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646daac93a0 con 0x5646dabd75d0 2024-09-19T09:28:11.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.169+0000 7fa736275640 1 --2- 172.21.15.120:0/3323035060 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa7180a3240 0x7fa7180a5700 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fa72809bca0 tx=0x7fa72806eb90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:11.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.169+0000 7fa735a74640 1 --2- 172.21.15.120:0/303829202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646dabd75d0 0x5646dabd7a10 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7fa720035000 tx=0x7fa72002f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0b66b304dcbcfe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:11.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.169+0000 7fa717fff640 1 -- 172.21.15.120:0/303829202 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa720034040 con 0x5646dabd75d0 2024-09-19T09:28:11.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.169+0000 7fa717fff640 1 -- 172.21.15.120:0/303829202 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa72002fe10 con 0x5646dabd75d0 2024-09-19T09:28:11.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.169+0000 7fa717fff640 1 -- 172.21.15.120:0/303829202 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7200046a0 con 0x5646dabd75d0 2024-09-19T09:28:11.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.170+0000 7fa73a0b1740 1 -- 172.21.15.120:0/303829202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646dabd75d0 msgr2=0x5646dabd7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:11.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.170+0000 7fa73a0b1740 1 --2- 172.21.15.120:0/303829202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646dabd75d0 0x5646dabd7a10 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7fa720035000 tx=0x7fa72002f770 comp rx=0 tx=0).stop 2024-09-19T09:28:11.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.170+0000 7fa73a0b1740 1 -- 172.21.15.120:0/303829202 shutdown_connections 2024-09-19T09:28:11.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.170+0000 7fa73a0b1740 1 --2- 172.21.15.120:0/303829202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646dabe2370 0x5646dabe4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.170+0000 7fa73a0b1740 1 --2- 172.21.15.120:0/303829202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646dabd75d0 0x5646dabd7a10 unknown :-1 s=CLOSED pgs=2548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.170+0000 7fa73a0b1740 1 --2- 172.21.15.120:0/303829202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646dabd8b40 0x5646dabd7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.170+0000 7fa73a0b1740 1 -- 172.21.15.120:0/303829202 >> 172.21.15.120:0/303829202 conn(0x5646daba0310 msgr2=0x5646dabd3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:11.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.170+0000 7fa73a0b1740 1 -- 172.21.15.120:0/303829202 shutdown_connections 2024-09-19T09:28:11.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.170+0000 7fa73a0b1740 1 -- 172.21.15.120:0/303829202 wait complete. 2024-09-19T09:28:11.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.170+0000 7fa73a0b1740 1 -- start start 2024-09-19T09:28:11.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.170+0000 7fa73a0b1740 1 Processor -- start 2024-09-19T09:28:11.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.171+0000 7fa73a0b1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646dabd75d0 0x5646dabe1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.171+0000 7fa73a0b1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646dabd8b40 0x5646dabdef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.171+0000 7fa73a0b1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646dabe2370 0x5646dabdf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.171+0000 7fa73a0b1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5646dabe5010 con 0x5646dabd75d0 2024-09-19T09:28:11.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.171+0000 7fa73a0b1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5646daaf8060 con 0x5646dabe2370 2024-09-19T09:28:11.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.171+0000 7fa73a0b1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5646dabe5190 con 0x5646dabd8b40 2024-09-19T09:28:11.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.171+0000 7fa73a0b1740 10 client.0 _ll_register_callbacks cb 0x5646da84b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:11.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.171+0000 7fa735273640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646dabd75d0 0x5646dabe1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.171+0000 7fa735273640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646dabd75d0 0x5646dabe1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43428/0 (socket says 172.21.15.120:43428) 2024-09-19T09:28:11.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.171+0000 7fa735273640 1 -- 172.21.15.120:0/2725203862 learned_addr learned my addr 172.21.15.120:0/2725203862 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:11.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.171+0000 7fa735a74640 1 --2- 172.21.15.120:0/2725203862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646dabd8b40 0x5646dabdef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.171+0000 7fa735273640 1 -- 172.21.15.120:0/2725203862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646dabd8b40 msgr2=0x5646dabdef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:11.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.171+0000 7fa735273640 1 --2- 172.21.15.120:0/2725203862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646dabd8b40 0x5646dabdef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.171+0000 7fa735273640 1 -- 172.21.15.120:0/2725203862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646dabe2370 msgr2=0x5646dabdf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:11.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.171+0000 7fa735273640 1 --2- 172.21.15.120:0/2725203862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646dabe2370 0x5646dabdf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.171+0000 7fa735273640 1 -- 172.21.15.120:0/2725203862 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646da8ae3c0 con 0x5646dabd75d0 2024-09-19T09:28:11.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.171+0000 7fa735a74640 1 --2- 172.21.15.120:0/2725203862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646dabd8b40 0x5646dabdef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:28:11.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.171+0000 7fa735273640 1 --2- 172.21.15.120:0/2725203862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646dabd75d0 0x5646dabe1cc0 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7fa724013790 tx=0x7fa724013b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4706cbe502802e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:11.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.171+0000 7fa7157fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:28:11.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.171+0000 7fa7157fa640 1 -- 172.21.15.120:0/2725203862 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72401a020 con 0x5646dabd75d0 2024-09-19T09:28:11.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.172+0000 7fa7157fa640 1 -- 172.21.15.120:0/2725203862 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa7240128f0 con 0x5646dabd75d0 2024-09-19T09:28:11.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.172+0000 7fa73a0b1740 1 -- 172.21.15.120:0/2725203862 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa72002fe10 con 0x5646dabd75d0 2024-09-19T09:28:11.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.172+0000 7fa7157fa640 1 -- 172.21.15.120:0/2725203862 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa72400f040 con 0x5646dabd75d0 2024-09-19T09:28:11.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.172+0000 7fa73a0b1740 10 client.23637 fetch_fsmap learned FSMap version 41 2024-09-19T09:28:11.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.172+0000 7fa73a0b1740 1 -- 172.21.15.120:0/2725203862 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5646dabf0ab0 con 0x5646dabd75d0 2024-09-19T09:28:11.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.172+0000 7fa7157fa640 1 -- 172.21.15.120:0/2725203862 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa72002fe10 con 0x5646dabd75d0 2024-09-19T09:28:11.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.172+0000 7fa7157fa640 1 -- 172.21.15.120:0/2725203862 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7fa724015070 con 0x5646dabd75d0 2024-09-19T09:28:11.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.172+0000 7fa73a0b1740 10 client.23637 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T09:28:11.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.172+0000 7fa73a0b1740 10 client.23637 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:28:11.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.172+0000 7fa73a0b1740 20 client.23637 populate_metadata read hostname 'smithi120' 2024-09-19T09:28:11.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.172+0000 7fa73a0b1740 1 --2- 172.21.15.120:0/2725203862 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5646dabf1ec0 0x5646dabf4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.172+0000 7fa73a0b1740 4 client.23637 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:28:11.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.172+0000 7fa73a0b1740 1 -- 172.21.15.120:0/2725203862 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646dab4a860 con 0x5646dabf1ec0 2024-09-19T09:28:11.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.172+0000 7fa735a74640 1 --2- 172.21.15.120:0/2725203862 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5646dabf1ec0 0x5646dabf4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.173+0000 7fa735a74640 1 --2- 172.21.15.120:0/2725203862 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5646dabf1ec0 0x5646dabf4330 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=75fcdf248149ef52 server_cookie=d56b6d3429f2986f in_seq=0 out_seq=0 2024-09-19T09:28:11.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.173+0000 7fa7157fa640 10 client.23637 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:28:11.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.173+0000 7fa7157fa640 1 -- 172.21.15.120:0/2725203862 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5646dab4a860 con 0x5646dabf1ec0 2024-09-19T09:28:11.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.173+0000 7fa7157fa640 10 client.23637 handle_command_reply: tid=0 2024-09-19T09:28:11.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.174+0000 7fa73a0b1740 1 client.23637 shutdown 2024-09-19T09:28:11.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.174+0000 7fa73a0b1740 1 -- 172.21.15.120:0/2725203862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646dabd75d0 msgr2=0x5646dabe1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:11.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.174+0000 7fa73a0b1740 1 --2- 172.21.15.120:0/2725203862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646dabd75d0 0x5646dabe1cc0 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7fa724013790 tx=0x7fa724013b90 comp rx=0 tx=0).stop 2024-09-19T09:28:11.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.174+0000 7fa73a0b1740 1 -- 172.21.15.120:0/2725203862 shutdown_connections 2024-09-19T09:28:11.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.174+0000 7fa73a0b1740 1 --2- 172.21.15.120:0/2725203862 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5646dabf1ec0 0x5646dabf4330 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.174+0000 7fa73a0b1740 1 --2- 172.21.15.120:0/2725203862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646dabe2370 0x5646dabdf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.174+0000 7fa73a0b1740 1 --2- 172.21.15.120:0/2725203862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646dabd8b40 0x5646dabdef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.174+0000 7fa73a0b1740 1 --2- 172.21.15.120:0/2725203862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646dabd75d0 0x5646dabe1cc0 unknown :-1 s=CLOSED pgs=2549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.174+0000 7fa7157fa640 0 client.23637 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:28:11.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.174+0000 7fa73a0b1740 1 -- 172.21.15.120:0/2725203862 >> 172.21.15.120:0/2725203862 conn(0x5646daba0310 msgr2=0x5646dabd9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:11.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.174+0000 7fa73a0b1740 1 -- 172.21.15.120:0/2725203862 shutdown_connections 2024-09-19T09:28:11.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.174+0000 7fa73a0b1740 1 -- 172.21.15.120:0/2725203862 wait complete. 2024-09-19T09:28:11.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.174+0000 7fa73a0b1740 20 client.23637 trim_cache size 0 max 16384 2024-09-19T09:28:11.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.188+0000 7fa73a0b1740 1 Processor -- start 2024-09-19T09:28:11.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.188+0000 7fa73a0b1740 1 -- start start 2024-09-19T09:28:11.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.188+0000 7fa73a0b1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646dabd75d0 0x5646dabd2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.188+0000 7fa73a0b1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646dabd8b40 0x5646dabda120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.188+0000 7fa73a0b1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646dabe2370 0x5646dabda660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.188+0000 7fa73a0b1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa7301af8a0 con 0x5646dabe2370 2024-09-19T09:28:11.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.188+0000 7fa73a0b1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa7301b1410 con 0x5646dabd8b40 2024-09-19T09:28:11.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.188+0000 7fa73a0b1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa7301aff70 con 0x5646dabd75d0 2024-09-19T09:28:11.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.188+0000 7fa735273640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646dabe2370 0x5646dabda660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.188+0000 7fa735273640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646dabe2370 0x5646dabda660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43434/0 (socket says 172.21.15.120:43434) 2024-09-19T09:28:11.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.188+0000 7fa735273640 1 -- 172.21.15.120:0/1894081278 learned_addr learned my addr 172.21.15.120:0/1894081278 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:11.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.189+0000 7fa735a74640 1 --2- 172.21.15.120:0/1894081278 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646dabd8b40 0x5646dabda120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.189+0000 7fa735273640 1 --2- 172.21.15.120:0/1894081278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646dabd75d0 0x5646dabd2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.189+0000 7fa735273640 1 -- 172.21.15.120:0/1894081278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646dabd75d0 msgr2=0x5646dabd2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:11.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.189+0000 7fa735273640 1 --2- 172.21.15.120:0/1894081278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646dabd75d0 0x5646dabd2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.189+0000 7fa735273640 1 -- 172.21.15.120:0/1894081278 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646dabd8b40 msgr2=0x5646dabda120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:11.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.189+0000 7fa735273640 1 --2- 172.21.15.120:0/1894081278 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646dabd8b40 0x5646dabda120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.189+0000 7fa735273640 1 -- 172.21.15.120:0/1894081278 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646dabd2310 con 0x5646dabe2370 2024-09-19T09:28:11.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.189+0000 7fa735273640 1 --2- 172.21.15.120:0/1894081278 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646dabe2370 0x5646dabda660 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7fa724023040 tx=0x7fa724001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800823262c58e9db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:11.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.189+0000 7fa7177fe640 1 -- 172.21.15.120:0/1894081278 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72401a040 con 0x5646dabe2370 2024-09-19T09:28:11.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.189+0000 7fa7177fe640 1 -- 172.21.15.120:0/1894081278 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa724016950 con 0x5646dabe2370 2024-09-19T09:28:11.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.189+0000 7fa7177fe640 1 -- 172.21.15.120:0/1894081278 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa724015a30 con 0x5646dabe2370 2024-09-19T09:28:11.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.190+0000 7fa73a0b1740 1 -- 172.21.15.120:0/1894081278 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646dabe2370 msgr2=0x5646dabda660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:11.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.190+0000 7fa73a0b1740 1 --2- 172.21.15.120:0/1894081278 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646dabe2370 0x5646dabda660 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7fa724023040 tx=0x7fa724001e50 comp rx=0 tx=0).stop 2024-09-19T09:28:11.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.190+0000 7fa73a0b1740 1 -- 172.21.15.120:0/1894081278 shutdown_connections 2024-09-19T09:28:11.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.190+0000 7fa73a0b1740 1 --2- 172.21.15.120:0/1894081278 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646dabe2370 0x5646dabda660 unknown :-1 s=CLOSED pgs=2550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.190+0000 7fa73a0b1740 1 --2- 172.21.15.120:0/1894081278 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646dabd8b40 0x5646dabda120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.190+0000 7fa73a0b1740 1 --2- 172.21.15.120:0/1894081278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646dabd75d0 0x5646dabd2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.190+0000 7fa73a0b1740 1 -- 172.21.15.120:0/1894081278 >> 172.21.15.120:0/1894081278 conn(0x5646daba0310 msgr2=0x5646dabd2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:11.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.190+0000 7fa73a0b1740 1 -- 172.21.15.120:0/1894081278 shutdown_connections 2024-09-19T09:28:11.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.190+0000 7fa73a0b1740 1 -- 172.21.15.120:0/1894081278 wait complete. 2024-09-19T09:28:11.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.190+0000 7fa73a0b1740 1 -- start start 2024-09-19T09:28:11.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.191+0000 7fa73a0b1740 1 Processor -- start 2024-09-19T09:28:11.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.191+0000 7fa73a0b1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646dabd75d0 0x5646dabdbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.191+0000 7fa73a0b1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646dabd8b40 0x5646dabdc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.191+0000 7fa73a0b1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646dabe2370 0x5646dabdc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.191+0000 7fa73a0b1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa7301b0d40 con 0x5646dabe2370 2024-09-19T09:28:11.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.191+0000 7fa73a0b1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa7301b0670 con 0x5646dabd75d0 2024-09-19T09:28:11.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.191+0000 7fa73a0b1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa7301ad860 con 0x5646dabd8b40 2024-09-19T09:28:11.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.191+0000 7fa73a0b1740 10 client.0 _ll_register_callbacks cb 0x7fa724006f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:11.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.191+0000 7fa735273640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646dabe2370 0x5646dabdc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.191+0000 7fa735273640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646dabe2370 0x5646dabdc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43438/0 (socket says 172.21.15.120:43438) 2024-09-19T09:28:11.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.191+0000 7fa735273640 1 -- 172.21.15.120:0/3850735323 learned_addr learned my addr 172.21.15.120:0/3850735323 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:11.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.191+0000 7fa735a74640 1 --2- 172.21.15.120:0/3850735323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646dabd8b40 0x5646dabdc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.191+0000 7fa735273640 1 --2- 172.21.15.120:0/3850735323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646dabd75d0 0x5646dabdbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.191+0000 7fa735273640 1 -- 172.21.15.120:0/3850735323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646dabd8b40 msgr2=0x5646dabdc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:11.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.191+0000 7fa735273640 1 --2- 172.21.15.120:0/3850735323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646dabd8b40 0x5646dabdc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.191+0000 7fa735273640 1 -- 172.21.15.120:0/3850735323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646dabd75d0 msgr2=0x5646dabdbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:11.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.192+0000 7fa735273640 1 --2- 172.21.15.120:0/3850735323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646dabd75d0 0x5646dabdbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.192+0000 7fa735273640 1 -- 172.21.15.120:0/3850735323 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646dac5bbd0 con 0x5646dabe2370 2024-09-19T09:28:11.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.192+0000 7fa735273640 1 --2- 172.21.15.120:0/3850735323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646dabe2370 0x5646dabdc9b0 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7fa724016f20 tx=0x7fa724015510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4e724fbd2e7870 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:11.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.192+0000 7fa7157fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:28:11.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.192+0000 7fa7157fa640 1 -- 172.21.15.120:0/3850735323 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72401a040 con 0x5646dabe2370 2024-09-19T09:28:11.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.192+0000 7fa7157fa640 1 -- 172.21.15.120:0/3850735323 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa724015c10 con 0x5646dabe2370 2024-09-19T09:28:11.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.192+0000 7fa73a0b1740 1 -- 172.21.15.120:0/3850735323 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa724016950 con 0x5646dabe2370 2024-09-19T09:28:11.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.192+0000 7fa7157fa640 1 -- 172.21.15.120:0/3850735323 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa724015e10 con 0x5646dabe2370 2024-09-19T09:28:11.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.192+0000 7fa73a0b1740 10 client.23643 fetch_fsmap learned FSMap version 41 2024-09-19T09:28:11.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.192+0000 7fa73a0b1740 1 -- 172.21.15.120:0/3850735323 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5646dac5ce50 con 0x5646dabe2370 2024-09-19T09:28:11.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.192+0000 7fa7157fa640 1 -- 172.21.15.120:0/3850735323 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa724016950 con 0x5646dabe2370 2024-09-19T09:28:11.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.193+0000 7fa7157fa640 1 -- 172.21.15.120:0/3850735323 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7fa724015040 con 0x5646dabe2370 2024-09-19T09:28:11.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.193+0000 7fa73a0b1740 10 client.23643 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T09:28:11.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.193+0000 7fa73a0b1740 10 client.23643 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T09:28:11.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.193+0000 7fa73a0b1740 20 client.23643 populate_metadata read hostname 'smithi120' 2024-09-19T09:28:11.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.193+0000 7fa73a0b1740 1 --2- 172.21.15.120:0/3850735323 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5646dac5d4b0 0x5646dac5f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.193+0000 7fa73a0b1740 4 client.23643 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:28:11.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.193+0000 7fa73a0b1740 1 -- 172.21.15.120:0/3850735323 --> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5646dab44d70 con 0x5646dac5d4b0 2024-09-19T09:28:11.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.193+0000 7fa735a74640 1 --2- 172.21.15.120:0/3850735323 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5646dac5d4b0 0x5646dac5f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.193+0000 7fa7157fa640 10 client.23643 ms_handle_connect on v2:172.21.15.153:6836/3048520124 2024-09-19T09:28:11.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.193+0000 7fa735a74640 1 --2- 172.21.15.120:0/3850735323 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5646dac5d4b0 0x5646dac5f8d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6730faf064934d1d server_cookie=693e2d0e3d08e43e in_seq=0 out_seq=0 2024-09-19T09:28:11.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.194+0000 7fa7157fa640 1 -- 172.21.15.120:0/3850735323 <== mds.1 v2:172.21.15.153:6836/3048520124 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5646dab44d70 con 0x5646dac5d4b0 2024-09-19T09:28:11.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.194+0000 7fa7157fa640 10 client.23643 handle_command_reply: tid=0 2024-09-19T09:28:11.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.194+0000 7fa73a0b1740 1 client.23643 shutdown 2024-09-19T09:28:11.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.194+0000 7fa73a0b1740 1 -- 172.21.15.120:0/3850735323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646dabe2370 msgr2=0x5646dabdc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:11.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.194+0000 7fa73a0b1740 1 --2- 172.21.15.120:0/3850735323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646dabe2370 0x5646dabdc9b0 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7fa724016f20 tx=0x7fa724015510 comp rx=0 tx=0).stop 2024-09-19T09:28:11.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.194+0000 7fa73a0b1740 1 -- 172.21.15.120:0/3850735323 shutdown_connections 2024-09-19T09:28:11.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.194+0000 7fa73a0b1740 1 --2- 172.21.15.120:0/3850735323 >> [v2:172.21.15.153:6836/3048520124,v1:172.21.15.153:6837/3048520124] conn(0x5646dac5d4b0 0x5646dac5f8d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.194+0000 7fa73a0b1740 1 --2- 172.21.15.120:0/3850735323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5646dabe2370 0x5646dabdc9b0 unknown :-1 s=CLOSED pgs=2551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.194+0000 7fa73a0b1740 1 --2- 172.21.15.120:0/3850735323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646dabd8b40 0x5646dabdc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.194+0000 7fa7157fa640 0 client.23643 ms_handle_reset on v2:172.21.15.153:6836/3048520124 2024-09-19T09:28:11.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.194+0000 7fa73a0b1740 1 --2- 172.21.15.120:0/3850735323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646dabd75d0 0x5646dabdbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.194+0000 7fa73a0b1740 1 -- 172.21.15.120:0/3850735323 >> 172.21.15.120:0/3850735323 conn(0x5646daba0310 msgr2=0x5646dabd2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:11.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.194+0000 7fa73a0b1740 1 -- 172.21.15.120:0/3850735323 shutdown_connections 2024-09-19T09:28:11.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.194+0000 7fa73a0b1740 1 -- 172.21.15.120:0/3850735323 wait complete. 2024-09-19T09:28:11.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.194+0000 7fa73a0b1740 20 client.23643 trim_cache size 0 max 16384 2024-09-19T09:28:11.203 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:28:11.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.195+0000 7fa737cd8640 1 -- 172.21.15.120:0/3323035060 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa7180a3240 msgr2=0x7fa7180a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:11.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.195+0000 7fa737cd8640 1 --2- 172.21.15.120:0/3323035060 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa7180a3240 0x7fa7180a5700 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fa72809bca0 tx=0x7fa72806eb90 comp rx=0 tx=0).stop 2024-09-19T09:28:11.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.195+0000 7fa737cd8640 1 -- 172.21.15.120:0/3323035060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa730102ef0 msgr2=0x7fa730197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:11.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.195+0000 7fa737cd8640 1 --2- 172.21.15.120:0/3323035060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa730102ef0 0x7fa730197c70 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7fa72800c970 tx=0x7fa72800ce30 comp rx=0 tx=0).stop 2024-09-19T09:28:11.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.195+0000 7fa737cd8640 1 -- 172.21.15.120:0/3323035060 shutdown_connections 2024-09-19T09:28:11.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.195+0000 7fa737cd8640 1 --2- 172.21.15.120:0/3323035060 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa7180a3240 0x7fa7180a5700 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.195+0000 7fa737cd8640 1 --2- 172.21.15.120:0/3323035060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa730102ef0 0x7fa730197c70 unknown :-1 s=CLOSED pgs=2547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.195+0000 7fa737cd8640 1 --2- 172.21.15.120:0/3323035060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa730102570 0x7fa730197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.195+0000 7fa737cd8640 1 --2- 172.21.15.120:0/3323035060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa730100e20 0x7fa7301971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.195+0000 7fa737cd8640 1 -- 172.21.15.120:0/3323035060 >> 172.21.15.120:0/3323035060 conn(0x7fa7300fc7d0 msgr2=0x7fa7300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:11.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.195+0000 7fa737cd8640 1 -- 172.21.15.120:0/3323035060 shutdown_connections 2024-09-19T09:28:11.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:11.195+0000 7fa737cd8640 1 -- 172.21.15.120:0/3323035060 wait complete. 2024-09-19T09:28:11.448 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:11.967 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:28:12.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:12 smithi196 ceph-mon[35150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:28:12.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:12 smithi196 ceph-mon[35150]: pgmap v1105: 97 pgs: 97 active+clean; 349 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.0 MiB/s wr, 591 op/s 2024-09-19T09:28:12.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:12 smithi153 ceph-mon[35071]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:28:12.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:12 smithi153 ceph-mon[35071]: pgmap v1105: 97 pgs: 97 active+clean; 349 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.0 MiB/s wr, 591 op/s 2024-09-19T09:28:12.443 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:12 smithi120 ceph-mon[31150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:28:12.443 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:12 smithi120 ceph-mon[31150]: pgmap v1105: 97 pgs: 97 active+clean; 349 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.0 MiB/s wr, 591 op/s 2024-09-19T09:28:12.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.526+0000 7f7c0fd9c640 1 -- 172.21.15.120:0/3372836826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c08106a80 msgr2=0x7f7c08106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:12.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.526+0000 7f7c0fd9c640 1 --2- 172.21.15.120:0/3372836826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c08106a80 0x7f7c08106e60 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009930 tx=0x7f7c0402a420 comp rx=0 tx=0).stop 2024-09-19T09:28:12.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.526+0000 7f7c0fd9c640 1 -- 172.21.15.120:0/3372836826 shutdown_connections 2024-09-19T09:28:12.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.526+0000 7f7c0fd9c640 1 --2- 172.21.15.120:0/3372836826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c08101420 0x7f7c0810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.526+0000 7f7c0fd9c640 1 --2- 172.21.15.120:0/3372836826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c08100a80 0x7f7c08100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.526+0000 7f7c0fd9c640 1 --2- 172.21.15.120:0/3372836826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c08106a80 0x7f7c08106e60 unknown :-1 s=CLOSED pgs=2552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.526+0000 7f7c0fd9c640 1 -- 172.21.15.120:0/3372836826 >> 172.21.15.120:0/3372836826 conn(0x7f7c080fc7a0 msgr2=0x7f7c080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:12.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.526+0000 7f7c0fd9c640 1 -- 172.21.15.120:0/3372836826 shutdown_connections 2024-09-19T09:28:12.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.526+0000 7f7c0fd9c640 1 -- 172.21.15.120:0/3372836826 wait complete. 2024-09-19T09:28:12.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.527+0000 7f7c0fd9c640 1 Processor -- start 2024-09-19T09:28:12.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.527+0000 7f7c0fd9c640 1 -- start start 2024-09-19T09:28:12.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.527+0000 7f7c0fd9c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c08100a80 0x7f7c08198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:12.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.527+0000 7f7c0fd9c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c08101420 0x7f7c08198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:12.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.527+0000 7f7c0fd9c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c08106a80 0x7f7c0819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:12.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.527+0000 7f7c0fd9c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7c08110ea0 con 0x7f7c08106a80 2024-09-19T09:28:12.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.527+0000 7f7c0fd9c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7c08110d20 con 0x7f7c08100a80 2024-09-19T09:28:12.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.527+0000 7f7c0fd9c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7c08111020 con 0x7f7c08101420 2024-09-19T09:28:12.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.527+0000 7f7c0e339640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c08106a80 0x7f7c0819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:12.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.527+0000 7f7c0e339640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c08106a80 0x7f7c0819cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40732/0 (socket says 172.21.15.120:40732) 2024-09-19T09:28:12.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.527+0000 7f7c0e339640 1 -- 172.21.15.120:0/2859945649 learned_addr learned my addr 172.21.15.120:0/2859945649 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:12.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.527+0000 7f7c0db38640 1 --2- 172.21.15.120:0/2859945649 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c08100a80 0x7f7c08198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:12.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.527+0000 7f7c0d337640 1 --2- 172.21.15.120:0/2859945649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c08101420 0x7f7c08198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:12.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.528+0000 7f7c0e339640 1 -- 172.21.15.120:0/2859945649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c08101420 msgr2=0x7f7c08198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:12.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.528+0000 7f7c0e339640 1 --2- 172.21.15.120:0/2859945649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c08101420 0x7f7c08198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.528+0000 7f7c0e339640 1 -- 172.21.15.120:0/2859945649 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c08100a80 msgr2=0x7f7c08198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:12.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.528+0000 7f7c0e339640 1 --2- 172.21.15.120:0/2859945649 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c08100a80 0x7f7c08198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.528+0000 7f7c0e339640 1 -- 172.21.15.120:0/2859945649 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0819d4c0 con 0x7f7c08106a80 2024-09-19T09:28:12.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.528+0000 7f7c0d337640 1 --2- 172.21.15.120:0/2859945649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c08101420 0x7f7c08198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:28:12.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.528+0000 7f7c0e339640 1 --2- 172.21.15.120:0/2859945649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c08106a80 0x7f7c0819cde0 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7f7c0000b970 tx=0x7f7c0000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:12.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.528+0000 7f7bfeffd640 1 -- 172.21.15.120:0/2859945649 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c00019070 con 0x7f7c08106a80 2024-09-19T09:28:12.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.528+0000 7f7bfeffd640 1 -- 172.21.15.120:0/2859945649 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c0000cb40 con 0x7f7c08106a80 2024-09-19T09:28:12.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.528+0000 7f7c0fd9c640 1 -- 172.21.15.120:0/2859945649 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c0819d7b0 con 0x7f7c08106a80 2024-09-19T09:28:12.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.528+0000 7f7bfeffd640 1 -- 172.21.15.120:0/2859945649 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c00007650 con 0x7f7c08106a80 2024-09-19T09:28:12.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.528+0000 7f7c0fd9c640 1 -- 172.21.15.120:0/2859945649 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c081a5090 con 0x7f7c08106a80 2024-09-19T09:28:12.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.530+0000 7f7bfeffd640 1 -- 172.21.15.120:0/2859945649 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f7c0001c070 con 0x7f7c08106a80 2024-09-19T09:28:12.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.530+0000 7f7c12175740 1 Processor -- start 2024-09-19T09:28:12.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.531+0000 7f7c12175740 1 -- start start 2024-09-19T09:28:12.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.531+0000 7f7c12175740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642b04f7a00 0x5642b04f7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:12.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.531+0000 7f7c12175740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642b04f8320 0x5642b05026a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:12.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.531+0000 7f7c12175740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642b0502be0 0x5642b0504fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:12.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.531+0000 7f7c12175740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5642b045c200 con 0x5642b0502be0 2024-09-19T09:28:12.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.531+0000 7f7c12175740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5642b0402dc0 con 0x5642b04f8320 2024-09-19T09:28:12.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.531+0000 7f7c12175740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5642b04581c0 con 0x5642b04f7a00 2024-09-19T09:28:12.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.531+0000 7f7bfeffd640 1 --2- 172.21.15.120:0/2859945649 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7bf40a32c0 0x7f7bf40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:12.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.531+0000 7f7c0d337640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642b0502be0 0x5642b0504fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:12.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.531+0000 7f7c0d337640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642b0502be0 0x5642b0504fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40736/0 (socket says 172.21.15.120:40736) 2024-09-19T09:28:12.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.531+0000 7f7c0d337640 1 -- 172.21.15.120:0/4089393430 learned_addr learned my addr 172.21.15.120:0/4089393430 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:12.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.531+0000 7f7c0db38640 1 --2- 172.21.15.120:0/4089393430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642b04f8320 0x5642b05026a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:12.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.531+0000 7f7bfeffd640 1 -- 172.21.15.120:0/2859945649 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f7c000cee40 con 0x7f7c08106a80 2024-09-19T09:28:12.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.531+0000 7f7c0d337640 1 --2- 172.21.15.120:0/4089393430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642b04f7a00 0x5642b04f7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:12.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.531+0000 7f7c0e339640 1 --2- 172.21.15.120:0/2859945649 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7bf40a32c0 0x7f7bf40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:12.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.531+0000 7f7c0d337640 1 -- 172.21.15.120:0/4089393430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642b04f7a00 msgr2=0x5642b04f7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:12.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.531+0000 7f7c0d337640 1 --2- 172.21.15.120:0/4089393430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642b04f7a00 0x5642b04f7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.531+0000 7f7c0d337640 1 -- 172.21.15.120:0/4089393430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642b04f8320 msgr2=0x5642b05026a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:12.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.531+0000 7f7c0d337640 1 --2- 172.21.15.120:0/4089393430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642b04f8320 0x5642b05026a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.531+0000 7f7c0d337640 1 -- 172.21.15.120:0/4089393430 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642b01cd620 con 0x5642b0502be0 2024-09-19T09:28:12.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.532+0000 7f7c0d337640 1 --2- 172.21.15.120:0/4089393430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642b0502be0 0x5642b0504fd0 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8014510 tx=0x7f7bf800d5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ecfeb998daaac3f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:12.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.532+0000 7f7bf3fff640 1 -- 172.21.15.120:0/4089393430 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8012710 con 0x5642b0502be0 2024-09-19T09:28:12.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.532+0000 7f7c0e339640 1 --2- 172.21.15.120:0/2859945649 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7bf40a32c0 0x7f7bf40a5780 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f7c00097970 tx=0x7f7c0000e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:12.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.532+0000 7f7bf3fff640 1 -- 172.21.15.120:0/4089393430 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bf801a070 con 0x5642b0502be0 2024-09-19T09:28:12.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.532+0000 7f7bf3fff640 1 -- 172.21.15.120:0/4089393430 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8011bb0 con 0x5642b0502be0 2024-09-19T09:28:12.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.533+0000 7f7c12175740 1 -- 172.21.15.120:0/4089393430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642b0502be0 msgr2=0x5642b0504fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:12.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.533+0000 7f7c12175740 1 --2- 172.21.15.120:0/4089393430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642b0502be0 0x5642b0504fd0 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8014510 tx=0x7f7bf800d5a0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.533+0000 7f7c12175740 1 -- 172.21.15.120:0/4089393430 shutdown_connections 2024-09-19T09:28:12.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.533+0000 7f7c12175740 1 --2- 172.21.15.120:0/4089393430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642b0502be0 0x5642b0504fd0 unknown :-1 s=CLOSED pgs=2554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.533+0000 7f7c12175740 1 --2- 172.21.15.120:0/4089393430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642b04f8320 0x5642b05026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.533+0000 7f7c12175740 1 --2- 172.21.15.120:0/4089393430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642b04f7a00 0x5642b04f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.533+0000 7f7c12175740 1 -- 172.21.15.120:0/4089393430 >> 172.21.15.120:0/4089393430 conn(0x5642b04f3f90 msgr2=0x5642b04f4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:12.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.533+0000 7f7c12175740 1 -- 172.21.15.120:0/4089393430 shutdown_connections 2024-09-19T09:28:12.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.533+0000 7f7c12175740 1 -- 172.21.15.120:0/4089393430 wait complete. 2024-09-19T09:28:12.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.533+0000 7f7c12175740 1 -- start start 2024-09-19T09:28:12.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.533+0000 7f7c12175740 1 Processor -- start 2024-09-19T09:28:12.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.534+0000 7f7c12175740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642b04f7a00 0x5642b0502550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:12.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.534+0000 7f7c12175740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642b04f8320 0x5642b04ff800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:12.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.534+0000 7f7c12175740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642b0502be0 0x5642b04ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:12.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.534+0000 7f7c12175740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5642b05057b0 con 0x5642b0502be0 2024-09-19T09:28:12.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.534+0000 7f7c12175740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5642b03f6c50 con 0x5642b04f8320 2024-09-19T09:28:12.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.534+0000 7f7c12175740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5642b0505930 con 0x5642b04f7a00 2024-09-19T09:28:12.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.534+0000 7f7c12175740 10 client.0 _ll_register_callbacks cb 0x5642b0408ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:12.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.534+0000 7f7c0d337640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642b0502be0 0x5642b04ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:12.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.534+0000 7f7c0d337640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642b0502be0 0x5642b04ffd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40740/0 (socket says 172.21.15.120:40740) 2024-09-19T09:28:12.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.534+0000 7f7c0d337640 1 -- 172.21.15.120:0/456468021 learned_addr learned my addr 172.21.15.120:0/456468021 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:12.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.534+0000 7f7c0db38640 1 --2- 172.21.15.120:0/456468021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642b04f8320 0x5642b04ff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:12.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.534+0000 7f7c0d337640 1 --2- 172.21.15.120:0/456468021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642b04f7a00 0x5642b0502550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:12.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.534+0000 7f7c0d337640 1 -- 172.21.15.120:0/456468021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642b04f7a00 msgr2=0x5642b0502550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:12.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.534+0000 7f7c0d337640 1 --2- 172.21.15.120:0/456468021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642b04f7a00 0x5642b0502550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.534+0000 7f7c0d337640 1 -- 172.21.15.120:0/456468021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642b04f8320 msgr2=0x5642b04ff800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:12.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.534+0000 7f7c0d337640 1 --2- 172.21.15.120:0/456468021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642b04f8320 0x5642b04ff800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.534+0000 7f7c0d337640 1 -- 172.21.15.120:0/456468021 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642b0500590 con 0x5642b0502be0 2024-09-19T09:28:12.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.534+0000 7f7c0db38640 1 --2- 172.21.15.120:0/456468021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642b04f8320 0x5642b04ff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:12.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.534+0000 7f7c0d337640 1 --2- 172.21.15.120:0/456468021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642b0502be0 0x5642b04ffd60 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800d410 tx=0x7f7bf8016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e874e7b60460dd30 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:12.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.534+0000 7f7bf17fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:28:12.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.535+0000 7f7bf17fa640 1 -- 172.21.15.120:0/456468021 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8011d70 con 0x5642b0502be0 2024-09-19T09:28:12.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.535+0000 7f7bf17fa640 1 -- 172.21.15.120:0/456468021 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f7bf8012510 con 0x5642b0502be0 2024-09-19T09:28:12.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.535+0000 7f7c12175740 1 -- 172.21.15.120:0/456468021 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bf801a070 con 0x5642b0502be0 2024-09-19T09:28:12.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.535+0000 7f7bf17fa640 1 -- 172.21.15.120:0/456468021 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7bf8012710 con 0x5642b0502be0 2024-09-19T09:28:12.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.535+0000 7f7c12175740 10 client.23655 fetch_fsmap learned FSMap version 41 2024-09-19T09:28:12.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.535+0000 7f7c12175740 1 -- 172.21.15.120:0/456468021 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5642b0511360 con 0x5642b0502be0 2024-09-19T09:28:12.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.535+0000 7f7bf17fa640 1 -- 172.21.15.120:0/456468021 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bf801a070 con 0x5642b0502be0 2024-09-19T09:28:12.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.535+0000 7f7bf17fa640 1 -- 172.21.15.120:0/456468021 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f7bf802d030 con 0x5642b0502be0 2024-09-19T09:28:12.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.535+0000 7f7c12175740 10 client.23655 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T09:28:12.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.535+0000 7f7c12175740 10 client.23655 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:28:12.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.535+0000 7f7c12175740 20 client.23655 populate_metadata read hostname 'smithi120' 2024-09-19T09:28:12.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.535+0000 7f7c12175740 1 --2- 172.21.15.120:0/456468021 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5642b0512700 0x5642b0514af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:12.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.535+0000 7f7c12175740 4 client.23655 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:28:12.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.535+0000 7f7c12175740 1 -- 172.21.15.120:0/456468021 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642b0469770 con 0x5642b0512700 2024-09-19T09:28:12.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.536+0000 7f7c0db38640 1 --2- 172.21.15.120:0/456468021 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5642b0512700 0x5642b0514af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:12.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.536+0000 7f7bf17fa640 10 client.23655 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:28:12.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.536+0000 7f7c0db38640 1 --2- 172.21.15.120:0/456468021 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5642b0512700 0x5642b0514af0 crc :-1 s=READY pgs=617 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=294b7bad904604 server_cookie=47052e5eae3b1243 in_seq=0 out_seq=0 2024-09-19T09:28:12.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.537+0000 7f7bf17fa640 1 -- 172.21.15.120:0/456468021 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5642b0469770 con 0x5642b0512700 2024-09-19T09:28:12.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.537+0000 7f7bf17fa640 10 client.23655 handle_command_reply: tid=0 2024-09-19T09:28:12.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.537+0000 7f7c12175740 1 client.23655 shutdown 2024-09-19T09:28:12.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.537+0000 7f7c12175740 1 -- 172.21.15.120:0/456468021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642b0502be0 msgr2=0x5642b04ffd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:12.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.537+0000 7f7c12175740 1 --2- 172.21.15.120:0/456468021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642b0502be0 0x5642b04ffd60 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800d410 tx=0x7f7bf8016040 comp rx=0 tx=0).stop 2024-09-19T09:28:12.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.537+0000 7f7c12175740 1 -- 172.21.15.120:0/456468021 shutdown_connections 2024-09-19T09:28:12.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.537+0000 7f7c12175740 1 --2- 172.21.15.120:0/456468021 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5642b0512700 0x5642b0514af0 crc :-1 s=READY pgs=617 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.537+0000 7f7c12175740 1 --2- 172.21.15.120:0/456468021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642b0502be0 0x5642b04ffd60 unknown :-1 s=CLOSED pgs=2555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.537+0000 7f7c12175740 1 --2- 172.21.15.120:0/456468021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642b04f8320 0x5642b04ff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.537+0000 7f7bf17fa640 0 client.23655 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:28:12.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.537+0000 7f7c12175740 1 --2- 172.21.15.120:0/456468021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642b04f7a00 0x5642b0502550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.537+0000 7f7c12175740 1 -- 172.21.15.120:0/456468021 >> 172.21.15.120:0/456468021 conn(0x5642b04f3f90 msgr2=0x5642b04f31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:12.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.537+0000 7f7c12175740 1 -- 172.21.15.120:0/456468021 shutdown_connections 2024-09-19T09:28:12.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.537+0000 7f7c12175740 1 -- 172.21.15.120:0/456468021 wait complete. 2024-09-19T09:28:12.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.537+0000 7f7c12175740 20 client.23655 trim_cache size 0 max 16384 2024-09-19T09:28:12.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.554+0000 7f7c12175740 1 Processor -- start 2024-09-19T09:28:12.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.554+0000 7f7c12175740 1 -- start start 2024-09-19T09:28:12.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.554+0000 7f7c12175740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642b04f7a00 0x5642b04ff7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:12.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.554+0000 7f7c12175740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642b04f8320 0x5642b050ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:12.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.554+0000 7f7c12175740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642b0502be0 0x5642b04fe6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:12.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.554+0000 7f7c12175740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7c081ad6b0 con 0x5642b04f8320 2024-09-19T09:28:12.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.554+0000 7f7c12175740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7c081af220 con 0x5642b04f7a00 2024-09-19T09:28:12.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.554+0000 7f7c12175740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7c081add80 con 0x5642b0502be0 2024-09-19T09:28:12.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.554+0000 7f7c0db38640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642b04f8320 0x5642b050ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:12.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.554+0000 7f7c0db38640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642b04f8320 0x5642b050ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40756/0 (socket says 172.21.15.120:40756) 2024-09-19T09:28:12.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.554+0000 7f7c0db38640 1 -- 172.21.15.120:0/550969158 learned_addr learned my addr 172.21.15.120:0/550969158 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:12.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.554+0000 7f7c0d337640 1 --2- 172.21.15.120:0/550969158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642b04f7a00 0x5642b04ff7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:12.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.554+0000 7f7c0db38640 1 -- 172.21.15.120:0/550969158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642b0502be0 msgr2=0x5642b04fe6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:12.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.554+0000 7f7c0db38640 1 --2- 172.21.15.120:0/550969158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642b0502be0 0x5642b04fe6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.554+0000 7f7c0db38640 1 -- 172.21.15.120:0/550969158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642b04f7a00 msgr2=0x5642b04ff7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:12.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.554+0000 7f7c0db38640 1 --2- 172.21.15.120:0/550969158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642b04f7a00 0x5642b04ff7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.554+0000 7f7c0db38640 1 -- 172.21.15.120:0/550969158 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642b0501740 con 0x5642b04f8320 2024-09-19T09:28:12.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.554+0000 7f7c0db38640 1 --2- 172.21.15.120:0/550969158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642b04f8320 0x5642b050ea10 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7f7c0402a940 tx=0x7f7c0402ab60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa8499d3253febf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:12.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.555+0000 7f7bf37fe640 1 -- 172.21.15.120:0/550969158 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0404d070 con 0x5642b04f8320 2024-09-19T09:28:12.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.555+0000 7f7bf37fe640 1 -- 172.21.15.120:0/550969158 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c04005500 con 0x5642b04f8320 2024-09-19T09:28:12.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.555+0000 7f7bf37fe640 1 -- 172.21.15.120:0/550969158 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04035040 con 0x5642b04f8320 2024-09-19T09:28:12.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.555+0000 7f7c12175740 1 -- 172.21.15.120:0/550969158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642b04f8320 msgr2=0x5642b050ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:12.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.555+0000 7f7c12175740 1 --2- 172.21.15.120:0/550969158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642b04f8320 0x5642b050ea10 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7f7c0402a940 tx=0x7f7c0402ab60 comp rx=0 tx=0).stop 2024-09-19T09:28:12.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.555+0000 7f7c12175740 1 -- 172.21.15.120:0/550969158 shutdown_connections 2024-09-19T09:28:12.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.555+0000 7f7c12175740 1 --2- 172.21.15.120:0/550969158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642b0502be0 0x5642b04fe6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.555+0000 7f7c12175740 1 --2- 172.21.15.120:0/550969158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642b04f8320 0x5642b050ea10 unknown :-1 s=CLOSED pgs=2556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.555+0000 7f7c12175740 1 --2- 172.21.15.120:0/550969158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642b04f7a00 0x5642b04ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.555+0000 7f7c12175740 1 -- 172.21.15.120:0/550969158 >> 172.21.15.120:0/550969158 conn(0x5642b04f3f90 msgr2=0x5642b04fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:12.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.555+0000 7f7c12175740 1 -- 172.21.15.120:0/550969158 shutdown_connections 2024-09-19T09:28:12.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.555+0000 7f7c12175740 1 -- 172.21.15.120:0/550969158 wait complete. 2024-09-19T09:28:12.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.556+0000 7f7c12175740 1 -- start start 2024-09-19T09:28:12.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.556+0000 7f7c12175740 1 Processor -- start 2024-09-19T09:28:12.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.556+0000 7f7c12175740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642b04f7a00 0x5642b0575420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:12.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.556+0000 7f7c12175740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642b04f8320 0x5642b0575960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:12.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.556+0000 7f7c12175740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642b0502be0 0x5642b0506120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:12.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.556+0000 7f7c12175740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7c081aeb50 con 0x5642b0502be0 2024-09-19T09:28:12.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.556+0000 7f7c12175740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7c081ae480 con 0x5642b04f7a00 2024-09-19T09:28:12.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.556+0000 7f7c12175740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7c081ab670 con 0x5642b04f8320 2024-09-19T09:28:12.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.556+0000 7f7c12175740 10 client.0 _ll_register_callbacks cb 0x7f7c04005d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:12.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.556+0000 7f7c0d337640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642b0502be0 0x5642b0506120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:12.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.556+0000 7f7c0d337640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642b0502be0 0x5642b0506120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40764/0 (socket says 172.21.15.120:40764) 2024-09-19T09:28:12.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.556+0000 7f7c0d337640 1 -- 172.21.15.120:0/1323889825 learned_addr learned my addr 172.21.15.120:0/1323889825 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:12.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.556+0000 7f7c0d337640 1 --2- 172.21.15.120:0/1323889825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642b04f7a00 0x5642b0575420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:12.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.556+0000 7f7c0db38640 1 --2- 172.21.15.120:0/1323889825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642b04f8320 0x5642b0575960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:12.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.556+0000 7f7c0d337640 1 -- 172.21.15.120:0/1323889825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642b04f8320 msgr2=0x5642b0575960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:12.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.556+0000 7f7c0d337640 1 --2- 172.21.15.120:0/1323889825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642b04f8320 0x5642b0575960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.556+0000 7f7c0d337640 1 -- 172.21.15.120:0/1323889825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642b04f7a00 msgr2=0x5642b0575420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:12.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.556+0000 7f7c0d337640 1 --2- 172.21.15.120:0/1323889825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642b04f7a00 0x5642b0575420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.556+0000 7f7c0d337640 1 -- 172.21.15.120:0/1323889825 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642b0506920 con 0x5642b0502be0 2024-09-19T09:28:12.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.557+0000 7f7c0d337640 1 --2- 172.21.15.120:0/1323889825 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642b0502be0 0x5642b0506120 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8014510 tx=0x7f7bf800d440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f06c2056d3f54c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:12.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.557+0000 7f7bf17fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:28:12.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.557+0000 7f7bf17fa640 1 -- 172.21.15.120:0/1323889825 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8012e60 con 0x5642b0502be0 2024-09-19T09:28:12.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.557+0000 7f7bf17fa640 1 -- 172.21.15.120:0/1323889825 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f7bf8012510 con 0x5642b0502be0 2024-09-19T09:28:12.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.557+0000 7f7c12175740 1 -- 172.21.15.120:0/1323889825 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c04005500 con 0x5642b0502be0 2024-09-19T09:28:12.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.557+0000 7f7bf17fa640 1 -- 172.21.15.120:0/1323889825 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7bf8014da0 con 0x5642b0502be0 2024-09-19T09:28:12.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.557+0000 7f7c12175740 10 client.23661 fetch_fsmap learned FSMap version 41 2024-09-19T09:28:12.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.557+0000 7f7c12175740 1 -- 172.21.15.120:0/1323889825 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5642b057cba0 con 0x5642b0502be0 2024-09-19T09:28:12.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.557+0000 7f7bf17fa640 1 -- 172.21.15.120:0/1323889825 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c04005500 con 0x5642b0502be0 2024-09-19T09:28:12.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.557+0000 7f7bf17fa640 1 -- 172.21.15.120:0/1323889825 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f7bf8043050 con 0x5642b0502be0 2024-09-19T09:28:12.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.557+0000 7f7c12175740 10 client.23661 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T09:28:12.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.557+0000 7f7c12175740 10 client.23661 resolve_mds: validated gid 24461 aka daemon mds.b 2024-09-19T09:28:12.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.557+0000 7f7c12175740 20 client.23661 populate_metadata read hostname 'smithi120' 2024-09-19T09:28:12.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.557+0000 7f7c12175740 1 --2- 172.21.15.120:0/1323889825 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5642b057d1f0 0x5642b057f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:12.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.557+0000 7f7c12175740 4 client.23661 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-19T09:28:12.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.557+0000 7f7c12175740 1 -- 172.21.15.120:0/1323889825 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x5642b0463c80 con 0x5642b057d1f0 2024-09-19T09:28:12.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.558+0000 7f7c0db38640 1 --2- 172.21.15.120:0/1323889825 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5642b057d1f0 0x5642b057f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:12.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.558+0000 7f7bf17fa640 10 client.23661 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:28:12.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.558+0000 7f7c0db38640 1 --2- 172.21.15.120:0/1323889825 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5642b057d1f0 0x5642b057f610 crc :-1 s=READY pgs=618 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d60e47c8aee00729 server_cookie=bb3271df6cdd1650 in_seq=0 out_seq=0 2024-09-19T09:28:12.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.558+0000 7f7bf17fa640 1 -- 172.21.15.120:0/1323889825 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x5642b0463c80 con 0x5642b057d1f0 2024-09-19T09:28:12.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.558+0000 7f7bf17fa640 10 client.23661 handle_command_reply: tid=0 2024-09-19T09:28:12.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.558+0000 7f7c12175740 1 client.23661 shutdown 2024-09-19T09:28:12.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.558+0000 7f7c12175740 1 -- 172.21.15.120:0/1323889825 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642b0502be0 msgr2=0x5642b0506120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:12.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.558+0000 7f7c12175740 1 --2- 172.21.15.120:0/1323889825 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642b0502be0 0x5642b0506120 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8014510 tx=0x7f7bf800d440 comp rx=0 tx=0).stop 2024-09-19T09:28:12.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.559+0000 7f7c12175740 1 -- 172.21.15.120:0/1323889825 shutdown_connections 2024-09-19T09:28:12.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.559+0000 7f7c12175740 1 --2- 172.21.15.120:0/1323889825 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x5642b057d1f0 0x5642b057f610 crc :-1 s=READY pgs=618 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.559+0000 7f7c12175740 1 --2- 172.21.15.120:0/1323889825 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642b0502be0 0x5642b0506120 unknown :-1 s=CLOSED pgs=2557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.559+0000 7f7c12175740 1 --2- 172.21.15.120:0/1323889825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642b04f8320 0x5642b0575960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.559+0000 7f7c12175740 1 --2- 172.21.15.120:0/1323889825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642b04f7a00 0x5642b0575420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.559+0000 7f7bf17fa640 0 client.23661 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:28:12.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.559+0000 7f7c12175740 1 -- 172.21.15.120:0/1323889825 >> 172.21.15.120:0/1323889825 conn(0x5642b04f3f90 msgr2=0x5642b04fadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:12.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.559+0000 7f7c12175740 1 -- 172.21.15.120:0/1323889825 shutdown_connections 2024-09-19T09:28:12.606 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:28:12.607 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 41, 2024-09-19T09:28:12.607 INFO:teuthology.orchestra.run.smithi120.stdout: "leader": 24461, 2024-09-19T09:28:12.607 INFO:teuthology.orchestra.run.smithi120.stdout: "set_version": 88, 2024-09-19T09:28:12.607 INFO:teuthology.orchestra.run.smithi120.stdout: "sets": {} 2024-09-19T09:28:12.607 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:28:12.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.559+0000 7f7c12175740 1 -- 172.21.15.120:0/1323889825 wait complete. 2024-09-19T09:28:12.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.559+0000 7f7c12175740 20 client.23661 trim_cache size 0 max 16384 2024-09-19T09:28:12.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.559+0000 7f7c0fd9c640 1 -- 172.21.15.120:0/2859945649 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7bf40a32c0 msgr2=0x7f7bf40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:12.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.559+0000 7f7c0fd9c640 1 --2- 172.21.15.120:0/2859945649 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7bf40a32c0 0x7f7bf40a5780 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f7c00097970 tx=0x7f7c0000e040 comp rx=0 tx=0).stop 2024-09-19T09:28:12.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.559+0000 7f7c0fd9c640 1 -- 172.21.15.120:0/2859945649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c08106a80 msgr2=0x7f7c0819cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:12.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.559+0000 7f7c0fd9c640 1 --2- 172.21.15.120:0/2859945649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c08106a80 0x7f7c0819cde0 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7f7c0000b970 tx=0x7f7c0000be30 comp rx=0 tx=0).stop 2024-09-19T09:28:12.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.559+0000 7f7c0fd9c640 1 -- 172.21.15.120:0/2859945649 shutdown_connections 2024-09-19T09:28:12.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.559+0000 7f7c0fd9c640 1 --2- 172.21.15.120:0/2859945649 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f7bf40a32c0 0x7f7bf40a5780 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.559+0000 7f7c0fd9c640 1 --2- 172.21.15.120:0/2859945649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c08106a80 0x7f7c0819cde0 unknown :-1 s=CLOSED pgs=2553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.559+0000 7f7c0fd9c640 1 --2- 172.21.15.120:0/2859945649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c08101420 0x7f7c08198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.559+0000 7f7c0fd9c640 1 --2- 172.21.15.120:0/2859945649 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c08100a80 0x7f7c08198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:12.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.559+0000 7f7c0fd9c640 1 -- 172.21.15.120:0/2859945649 >> 172.21.15.120:0/2859945649 conn(0x7f7c080fc7a0 msgr2=0x7f7c0806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:12.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.560+0000 7f7c0fd9c640 1 -- 172.21.15.120:0/2859945649 shutdown_connections 2024-09-19T09:28:12.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:12.560+0000 7f7c0fd9c640 1 -- 172.21.15.120:0/2859945649 wait complete. 2024-09-19T09:28:13.388 INFO:tasks.quiescer:done joining Quiescers 2024-09-19T09:28:13.388 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-19T09:28:13.399 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi120.front.sepia.ceph.com 2024-09-19T09:28:13.407 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-19T09:28:13.419 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-19T09:28:13.459 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:14.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.985+0000 7fe9800b2640 1 -- 172.21.15.120:0/3163682171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe978101df0 msgr2=0x7fe978105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:14.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.985+0000 7fe9800b2640 1 --2- 172.21.15.120:0/3163682171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe978101df0 0x7fe978105920 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7fe974009900 tx=0x7fe97402a410 comp rx=0 tx=0).stop 2024-09-19T09:28:14.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.985+0000 7fe9800b2640 1 -- 172.21.15.120:0/3163682171 shutdown_connections 2024-09-19T09:28:14.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.985+0000 7fe9800b2640 1 --2- 172.21.15.120:0/3163682171 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe978106050 0x7fe97810dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.985+0000 7fe9800b2640 1 --2- 172.21.15.120:0/3163682171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe978101df0 0x7fe978105920 unknown :-1 s=CLOSED pgs=2558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.985+0000 7fe9800b2640 1 --2- 172.21.15.120:0/3163682171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe978101440 0x7fe978101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.985+0000 7fe9800b2640 1 -- 172.21.15.120:0/3163682171 >> 172.21.15.120:0/3163682171 conn(0x7fe978077210 msgr2=0x7fe9780fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:14.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.985+0000 7fe9800b2640 1 -- 172.21.15.120:0/3163682171 shutdown_connections 2024-09-19T09:28:14.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.985+0000 7fe9800b2640 1 -- 172.21.15.120:0/3163682171 wait complete. 2024-09-19T09:28:14.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.986+0000 7fe9800b2640 1 Processor -- start 2024-09-19T09:28:14.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.986+0000 7fe9800b2640 1 -- start start 2024-09-19T09:28:14.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.986+0000 7fe9800b2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe978101440 0x7fe9781986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.986+0000 7fe9800b2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe978101df0 0x7fe978198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.986+0000 7fe9800b2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe978106050 0x7fe97819cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.986+0000 7fe9800b2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe978111070 con 0x7fe978101440 2024-09-19T09:28:14.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.986+0000 7fe9800b2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe978110ef0 con 0x7fe978101df0 2024-09-19T09:28:14.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.986+0000 7fe9800b2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe9781111f0 con 0x7fe978106050 2024-09-19T09:28:14.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.986+0000 7fe97de4e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe978101440 0x7fe9781986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.986+0000 7fe97de4e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe978101440 0x7fe9781986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40772/0 (socket says 172.21.15.120:40772) 2024-09-19T09:28:14.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.986+0000 7fe97de4e640 1 -- 172.21.15.120:0/2306125446 learned_addr learned my addr 172.21.15.120:0/2306125446 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:14.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.986+0000 7fe97e64f640 1 --2- 172.21.15.120:0/2306125446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe978106050 0x7fe97819cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.986+0000 7fe97d64d640 1 --2- 172.21.15.120:0/2306125446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe978101df0 0x7fe978198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.986+0000 7fe97de4e640 1 -- 172.21.15.120:0/2306125446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe978106050 msgr2=0x7fe97819cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:14.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.986+0000 7fe97de4e640 1 --2- 172.21.15.120:0/2306125446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe978106050 0x7fe97819cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.986+0000 7fe97de4e640 1 -- 172.21.15.120:0/2306125446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe978101df0 msgr2=0x7fe978198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:14.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.986+0000 7fe97de4e640 1 --2- 172.21.15.120:0/2306125446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe978101df0 0x7fe978198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.986+0000 7fe97de4e640 1 -- 172.21.15.120:0/2306125446 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97819d660 con 0x7fe978101440 2024-09-19T09:28:14.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.987+0000 7fe97de4e640 1 --2- 172.21.15.120:0/2306125446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe978101440 0x7fe9781986b0 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00d950 tx=0x7fe96c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:14.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.987+0000 7fe96affd640 1 -- 172.21.15.120:0/2306125446 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c017070 con 0x7fe978101440 2024-09-19T09:28:14.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.987+0000 7fe96affd640 1 -- 172.21.15.120:0/2306125446 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe96c009c30 con 0x7fe978101440 2024-09-19T09:28:14.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.987+0000 7fe9800b2640 1 -- 172.21.15.120:0/2306125446 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe97819d950 con 0x7fe978101440 2024-09-19T09:28:14.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.987+0000 7fe96affd640 1 -- 172.21.15.120:0/2306125446 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c00f6b0 con 0x7fe978101440 2024-09-19T09:28:14.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.987+0000 7fe9800b2640 1 -- 172.21.15.120:0/2306125446 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9781a5230 con 0x7fe978101440 2024-09-19T09:28:14.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.989+0000 7fe96affd640 1 -- 172.21.15.120:0/2306125446 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe96c00f850 con 0x7fe978101440 2024-09-19T09:28:14.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.989+0000 7fe98248b740 1 Processor -- start 2024-09-19T09:28:14.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.990+0000 7fe98248b740 1 -- start start 2024-09-19T09:28:14.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.990+0000 7fe98248b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b78e260b40 0x55b78e25f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.990+0000 7fe98248b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b78e25f5d0 0x55b78e25fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.990+0000 7fe98248b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b78e26a370 0x55b78e26c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.990+0000 7fe98248b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b78e1c52f0 con 0x55b78e25f5d0 2024-09-19T09:28:14.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.990+0000 7fe98248b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b78e16ccf0 con 0x55b78e26a370 2024-09-19T09:28:14.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.990+0000 7fe98248b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b78e1c12b0 con 0x55b78e260b40 2024-09-19T09:28:14.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.990+0000 7fe97de4e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b78e25f5d0 0x55b78e25fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.990+0000 7fe97de4e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b78e25f5d0 0x55b78e25fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40780/0 (socket says 172.21.15.120:40780) 2024-09-19T09:28:14.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.990+0000 7fe97de4e640 1 -- 172.21.15.120:0/3963035942 learned_addr learned my addr 172.21.15.120:0/3963035942 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:14.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.990+0000 7fe97e64f640 1 --2- 172.21.15.120:0/3963035942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b78e260b40 0x55b78e25f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.990+0000 7fe96affd640 1 --2- 172.21.15.120:0/2306125446 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9640a3240 0x7fe9640a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.990+0000 7fe97d64d640 1 --2- 172.21.15.120:0/3963035942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b78e26a370 0x55b78e26c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.990+0000 7fe97de4e640 1 -- 172.21.15.120:0/3963035942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b78e260b40 msgr2=0x55b78e25f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:14.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.990+0000 7fe97de4e640 1 --2- 172.21.15.120:0/3963035942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b78e260b40 0x55b78e25f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.990+0000 7fe97de4e640 1 -- 172.21.15.120:0/3963035942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b78e26a370 msgr2=0x55b78e26c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:14.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.990+0000 7fe97de4e640 1 --2- 172.21.15.120:0/3963035942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b78e26a370 0x55b78e26c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.990+0000 7fe97e64f640 1 --2- 172.21.15.120:0/2306125446 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9640a3240 0x7fe9640a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.990+0000 7fe97de4e640 1 -- 172.21.15.120:0/3963035942 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b78e1513a0 con 0x55b78e25f5d0 2024-09-19T09:28:14.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.990+0000 7fe97d64d640 1 --2- 172.21.15.120:0/3963035942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b78e26a370 0x55b78e26c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:14.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.990+0000 7fe96affd640 1 -- 172.21.15.120:0/2306125446 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe96c0cf550 con 0x7fe978101440 2024-09-19T09:28:14.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.991+0000 7fe97de4e640 1 --2- 172.21.15.120:0/3963035942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b78e25f5d0 0x55b78e25fa10 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7fe96c009610 tx=0x7fe96c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4962dbc4d0fd8a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:14.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.991+0000 7fe97e64f640 1 --2- 172.21.15.120:0/2306125446 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9640a3240 0x7fe9640a5700 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fe970005fd0 tx=0x7fe970004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:14.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.991+0000 7fe963fff640 1 -- 172.21.15.120:0/3963035942 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c020070 con 0x55b78e25f5d0 2024-09-19T09:28:14.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.991+0000 7fe963fff640 1 -- 172.21.15.120:0/3963035942 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe96c009de0 con 0x55b78e25f5d0 2024-09-19T09:28:14.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.991+0000 7fe963fff640 1 -- 172.21.15.120:0/3963035942 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c0982c0 con 0x55b78e25f5d0 2024-09-19T09:28:14.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.991+0000 7fe98248b740 1 -- 172.21.15.120:0/3963035942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b78e25f5d0 msgr2=0x55b78e25fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:14.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.991+0000 7fe98248b740 1 --2- 172.21.15.120:0/3963035942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b78e25f5d0 0x55b78e25fa10 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7fe96c009610 tx=0x7fe96c005880 comp rx=0 tx=0).stop 2024-09-19T09:28:14.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.991+0000 7fe98248b740 1 -- 172.21.15.120:0/3963035942 shutdown_connections 2024-09-19T09:28:14.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.992+0000 7fe98248b740 1 --2- 172.21.15.120:0/3963035942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b78e26a370 0x55b78e26c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.992+0000 7fe98248b740 1 --2- 172.21.15.120:0/3963035942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b78e25f5d0 0x55b78e25fa10 unknown :-1 s=CLOSED pgs=2560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.992+0000 7fe98248b740 1 --2- 172.21.15.120:0/3963035942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b78e260b40 0x55b78e25f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.992+0000 7fe98248b740 1 -- 172.21.15.120:0/3963035942 >> 172.21.15.120:0/3963035942 conn(0x55b78e228310 msgr2=0x55b78e25bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:14.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.992+0000 7fe98248b740 1 -- 172.21.15.120:0/3963035942 shutdown_connections 2024-09-19T09:28:14.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.992+0000 7fe98248b740 1 -- 172.21.15.120:0/3963035942 wait complete. 2024-09-19T09:28:14.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.992+0000 7fe98248b740 1 -- start start 2024-09-19T09:28:14.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.992+0000 7fe98248b740 1 Processor -- start 2024-09-19T09:28:14.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.992+0000 7fe98248b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b78e25f5d0 0x55b78e269d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.992+0000 7fe98248b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b78e260b40 0x55b78e266ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.992+0000 7fe98248b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b78e26a370 0x55b78e2675e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.992+0000 7fe98248b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b78e26d010 con 0x55b78e260b40 2024-09-19T09:28:14.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.992+0000 7fe98248b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b78e180060 con 0x55b78e26a370 2024-09-19T09:28:14.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.992+0000 7fe98248b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b78e26d190 con 0x55b78e25f5d0 2024-09-19T09:28:14.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.992+0000 7fe98248b740 10 client.0 _ll_register_callbacks cb 0x55b78ded38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:14.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.992+0000 7fe97d64d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b78e260b40 0x55b78e266ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.993+0000 7fe97d64d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b78e260b40 0x55b78e266ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40788/0 (socket says 172.21.15.120:40788) 2024-09-19T09:28:14.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.993+0000 7fe97d64d640 1 -- 172.21.15.120:0/1676249375 learned_addr learned my addr 172.21.15.120:0/1676249375 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:14.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.993+0000 7fe97de4e640 1 --2- 172.21.15.120:0/1676249375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b78e25f5d0 0x55b78e269d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.993+0000 7fe97e64f640 1 --2- 172.21.15.120:0/1676249375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b78e26a370 0x55b78e2675e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.993+0000 7fe97d64d640 1 -- 172.21.15.120:0/1676249375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b78e25f5d0 msgr2=0x55b78e269d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:14.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.993+0000 7fe97d64d640 1 --2- 172.21.15.120:0/1676249375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b78e25f5d0 0x55b78e269d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.993+0000 7fe97d64d640 1 -- 172.21.15.120:0/1676249375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b78e26a370 msgr2=0x55b78e2675e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:14.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.993+0000 7fe97d64d640 1 --2- 172.21.15.120:0/1676249375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b78e26a370 0x55b78e2675e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.993+0000 7fe97d64d640 1 -- 172.21.15.120:0/1676249375 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b78df363c0 con 0x55b78e260b40 2024-09-19T09:28:14.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.993+0000 7fe97de4e640 1 --2- 172.21.15.120:0/1676249375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b78e25f5d0 0x55b78e269d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:14.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.993+0000 7fe97d64d640 1 --2- 172.21.15.120:0/1676249375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b78e260b40 0x55b78e266ff0 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7fe974000c00 tx=0x7fe974031860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7aeeb4dbafc0a37 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:14.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.993+0000 7fe9617fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:28:14.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.993+0000 7fe9617fa640 1 -- 172.21.15.120:0/1676249375 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97403f070 con 0x55b78e260b40 2024-09-19T09:28:14.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.993+0000 7fe9617fa640 1 -- 172.21.15.120:0/1676249375 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe974004450 con 0x55b78e260b40 2024-09-19T09:28:14.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.993+0000 7fe98248b740 1 -- 172.21.15.120:0/1676249375 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe96c009de0 con 0x55b78e260b40 2024-09-19T09:28:14.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.993+0000 7fe9617fa640 1 -- 172.21.15.120:0/1676249375 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fe974035070 con 0x55b78e260b40 2024-09-19T09:28:14.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.993+0000 7fe98248b740 10 client.23673 fetch_fsmap learned FSMap version 42 2024-09-19T09:28:14.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.993+0000 7fe98248b740 1 -- 172.21.15.120:0/1676249375 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b78e278a80 con 0x55b78e260b40 2024-09-19T09:28:14.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.994+0000 7fe9617fa640 1 -- 172.21.15.120:0/1676249375 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe96c009de0 con 0x55b78e260b40 2024-09-19T09:28:14.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.994+0000 7fe9617fa640 1 -- 172.21.15.120:0/1676249375 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7fe974004ac0 con 0x55b78e260b40 2024-09-19T09:28:14.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.994+0000 7fe98248b740 10 client.23673 fetch_fsmap finished waiting for FSMap version 42 2024-09-19T09:28:14.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.994+0000 7fe98248b740 10 client.23673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:28:14.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.994+0000 7fe98248b740 20 client.23673 populate_metadata read hostname 'smithi120' 2024-09-19T09:28:14.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.994+0000 7fe98248b740 1 --2- 172.21.15.120:0/1676249375 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b78e279f40 0x55b78e27c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.994+0000 7fe98248b740 4 client.23673 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:28:14.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.994+0000 7fe98248b740 1 -- 172.21.15.120:0/1676249375 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b78e1d2860 con 0x55b78e279f40 2024-09-19T09:28:14.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.994+0000 7fe97de4e640 1 --2- 172.21.15.120:0/1676249375 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b78e279f40 0x55b78e27c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.994+0000 7fe9617fa640 10 client.23673 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:28:14.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.994+0000 7fe97de4e640 1 --2- 172.21.15.120:0/1676249375 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b78e279f40 0x55b78e27c3a0 crc :-1 s=READY pgs=619 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e167369cf8e53cf server_cookie=d95a58248f3b977d in_seq=0 out_seq=0 2024-09-19T09:28:14.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.995+0000 7fe9617fa640 1 -- 172.21.15.120:0/1676249375 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b78e1d2860 con 0x55b78e279f40 2024-09-19T09:28:14.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.995+0000 7fe9617fa640 10 client.23673 handle_command_reply: tid=0 2024-09-19T09:28:14.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.995+0000 7fe98248b740 1 client.23673 shutdown 2024-09-19T09:28:14.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.995+0000 7fe98248b740 1 -- 172.21.15.120:0/1676249375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b78e260b40 msgr2=0x55b78e266ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:14.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.995+0000 7fe98248b740 1 --2- 172.21.15.120:0/1676249375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b78e260b40 0x55b78e266ff0 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7fe974000c00 tx=0x7fe974031860 comp rx=0 tx=0).stop 2024-09-19T09:28:14.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.995+0000 7fe98248b740 1 -- 172.21.15.120:0/1676249375 shutdown_connections 2024-09-19T09:28:14.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.995+0000 7fe98248b740 1 --2- 172.21.15.120:0/1676249375 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b78e279f40 0x55b78e27c3a0 crc :-1 s=READY pgs=619 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.995+0000 7fe98248b740 1 --2- 172.21.15.120:0/1676249375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b78e26a370 0x55b78e2675e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.995+0000 7fe98248b740 1 --2- 172.21.15.120:0/1676249375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b78e260b40 0x55b78e266ff0 unknown :-1 s=CLOSED pgs=2561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.995+0000 7fe9617fa640 0 client.23673 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:28:14.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.995+0000 7fe98248b740 1 --2- 172.21.15.120:0/1676249375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b78e25f5d0 0x55b78e269d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.995+0000 7fe98248b740 1 -- 172.21.15.120:0/1676249375 >> 172.21.15.120:0/1676249375 conn(0x55b78e228310 msgr2=0x55b78e261990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:14.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.995+0000 7fe98248b740 1 -- 172.21.15.120:0/1676249375 shutdown_connections 2024-09-19T09:28:14.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.995+0000 7fe98248b740 1 -- 172.21.15.120:0/1676249375 wait complete. 2024-09-19T09:28:14.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:13.996+0000 7fe98248b740 20 client.23673 trim_cache size 0 max 16384 2024-09-19T09:28:14.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.008+0000 7fe98248b740 1 Processor -- start 2024-09-19T09:28:14.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.008+0000 7fe98248b740 1 -- start start 2024-09-19T09:28:14.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.009+0000 7fe98248b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b78e25f5d0 0x55b78e2668b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.009+0000 7fe98248b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b78e260b40 0x55b78e25abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.009+0000 7fe98248b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b78e26a370 0x55b78e262180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.009+0000 7fe98248b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe9781ad6d0 con 0x55b78e260b40 2024-09-19T09:28:14.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.009+0000 7fe98248b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe9781af270 con 0x55b78e26a370 2024-09-19T09:28:14.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.009+0000 7fe98248b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe9781addd0 con 0x55b78e25f5d0 2024-09-19T09:28:14.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.009+0000 7fe97d64d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b78e260b40 0x55b78e25abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.009+0000 7fe97d64d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b78e260b40 0x55b78e25abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40794/0 (socket says 172.21.15.120:40794) 2024-09-19T09:28:14.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.009+0000 7fe97d64d640 1 -- 172.21.15.120:0/3216122129 learned_addr learned my addr 172.21.15.120:0/3216122129 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:14.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.009+0000 7fe97de4e640 1 --2- 172.21.15.120:0/3216122129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b78e25f5d0 0x55b78e2668b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.009+0000 7fe97e64f640 1 --2- 172.21.15.120:0/3216122129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b78e26a370 0x55b78e262180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.009+0000 7fe97d64d640 1 -- 172.21.15.120:0/3216122129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b78e25f5d0 msgr2=0x55b78e2668b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:14.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.009+0000 7fe97d64d640 1 --2- 172.21.15.120:0/3216122129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b78e25f5d0 0x55b78e2668b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.009+0000 7fe97d64d640 1 -- 172.21.15.120:0/3216122129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b78e26a370 msgr2=0x55b78e262180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:14.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.009+0000 7fe97d64d640 1 --2- 172.21.15.120:0/3216122129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b78e26a370 0x55b78e262180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.009+0000 7fe97d64d640 1 -- 172.21.15.120:0/3216122129 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b78e265540 con 0x55b78e260b40 2024-09-19T09:28:14.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.009+0000 7fe97d64d640 1 --2- 172.21.15.120:0/3216122129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b78e260b40 0x55b78e25abf0 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7fe97402a930 tx=0x7fe974001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab40e6bfb6f24ea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:14.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.009+0000 7fe9637fe640 1 -- 172.21.15.120:0/3216122129 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97403f040 con 0x55b78e260b40 2024-09-19T09:28:14.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.009+0000 7fe9637fe640 1 -- 172.21.15.120:0/3216122129 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe974052020 con 0x55b78e260b40 2024-09-19T09:28:14.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.009+0000 7fe9637fe640 1 -- 172.21.15.120:0/3216122129 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe974003d90 con 0x55b78e260b40 2024-09-19T09:28:14.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.010+0000 7fe98248b740 1 -- 172.21.15.120:0/3216122129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b78e260b40 msgr2=0x55b78e25abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:14.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.010+0000 7fe98248b740 1 --2- 172.21.15.120:0/3216122129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b78e260b40 0x55b78e25abf0 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7fe97402a930 tx=0x7fe974001e70 comp rx=0 tx=0).stop 2024-09-19T09:28:14.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.010+0000 7fe98248b740 1 -- 172.21.15.120:0/3216122129 shutdown_connections 2024-09-19T09:28:14.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.010+0000 7fe98248b740 1 --2- 172.21.15.120:0/3216122129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b78e26a370 0x55b78e262180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.010+0000 7fe98248b740 1 --2- 172.21.15.120:0/3216122129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b78e260b40 0x55b78e25abf0 unknown :-1 s=CLOSED pgs=2562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.010+0000 7fe98248b740 1 --2- 172.21.15.120:0/3216122129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b78e25f5d0 0x55b78e2668b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.010+0000 7fe98248b740 1 -- 172.21.15.120:0/3216122129 >> 172.21.15.120:0/3216122129 conn(0x55b78e228310 msgr2=0x55b78e25a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:14.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.010+0000 7fe98248b740 1 -- 172.21.15.120:0/3216122129 shutdown_connections 2024-09-19T09:28:14.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.010+0000 7fe98248b740 1 -- 172.21.15.120:0/3216122129 wait complete. 2024-09-19T09:28:14.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.010+0000 7fe98248b740 1 -- start start 2024-09-19T09:28:14.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.011+0000 7fe98248b740 1 Processor -- start 2024-09-19T09:28:14.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.011+0000 7fe98248b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b78e25f5d0 0x55b78e272c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.011+0000 7fe98248b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b78e260b40 0x55b78e26dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.011+0000 7fe98248b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b78e26a370 0x55b78e26e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.011+0000 7fe98248b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe9781aeba0 con 0x55b78e25f5d0 2024-09-19T09:28:14.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.011+0000 7fe98248b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe9781ae4d0 con 0x55b78e260b40 2024-09-19T09:28:14.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.011+0000 7fe98248b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe9781ab690 con 0x55b78e26a370 2024-09-19T09:28:14.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.011+0000 7fe98248b740 10 client.0 _ll_register_callbacks cb 0x7fe974009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:14.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.011+0000 7fe97de4e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b78e25f5d0 0x55b78e272c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.011+0000 7fe97de4e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b78e25f5d0 0x55b78e272c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40810/0 (socket says 172.21.15.120:40810) 2024-09-19T09:28:14.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.011+0000 7fe97de4e640 1 -- 172.21.15.120:0/2507276037 learned_addr learned my addr 172.21.15.120:0/2507276037 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:14.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.011+0000 7fe97d64d640 1 --2- 172.21.15.120:0/2507276037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b78e260b40 0x55b78e26dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.011+0000 7fe97e64f640 1 --2- 172.21.15.120:0/2507276037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b78e26a370 0x55b78e26e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.011+0000 7fe97de4e640 1 -- 172.21.15.120:0/2507276037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b78e26a370 msgr2=0x55b78e26e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:14.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.011+0000 7fe97de4e640 1 --2- 172.21.15.120:0/2507276037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b78e26a370 0x55b78e26e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.011+0000 7fe97de4e640 1 -- 172.21.15.120:0/2507276037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b78e260b40 msgr2=0x55b78e26dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:14.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.011+0000 7fe97de4e640 1 --2- 172.21.15.120:0/2507276037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b78e260b40 0x55b78e26dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.011+0000 7fe97de4e640 1 -- 172.21.15.120:0/2507276037 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b78e26ede0 con 0x55b78e25f5d0 2024-09-19T09:28:14.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.011+0000 7fe97e64f640 1 --2- 172.21.15.120:0/2507276037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b78e26a370 0x55b78e26e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:28:14.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.012+0000 7fe97de4e640 1 --2- 172.21.15.120:0/2507276037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b78e25f5d0 0x55b78e272c90 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7fe96c0980b0 tx=0x7fe96c097fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd6ddb689913bf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:14.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.012+0000 7fe9617fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-19T09:28:14.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.012+0000 7fe9617fa640 1 -- 172.21.15.120:0/2507276037 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c020040 con 0x55b78e25f5d0 2024-09-19T09:28:14.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.012+0000 7fe9617fa640 1 -- 172.21.15.120:0/2507276037 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe96c0cd160 con 0x55b78e25f5d0 2024-09-19T09:28:14.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.012+0000 7fe98248b740 1 -- 172.21.15.120:0/2507276037 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe974052020 con 0x55b78e25f5d0 2024-09-19T09:28:14.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.012+0000 7fe9617fa640 1 -- 172.21.15.120:0/2507276037 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fe96c0982c0 con 0x55b78e25f5d0 2024-09-19T09:28:14.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.012+0000 7fe98248b740 10 client.23679 fetch_fsmap learned FSMap version 42 2024-09-19T09:28:14.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.012+0000 7fe98248b740 1 -- 172.21.15.120:0/2507276037 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b78e2e9160 con 0x55b78e25f5d0 2024-09-19T09:28:14.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.012+0000 7fe9617fa640 1 -- 172.21.15.120:0/2507276037 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe974052020 con 0x55b78e25f5d0 2024-09-19T09:28:14.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.012+0000 7fe9617fa640 1 -- 172.21.15.120:0/2507276037 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7fe96c09b0a0 con 0x55b78e25f5d0 2024-09-19T09:28:14.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.012+0000 7fe98248b740 10 client.23679 fetch_fsmap finished waiting for FSMap version 42 2024-09-19T09:28:14.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.012+0000 7fe98248b740 10 client.23679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:28:14.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.012+0000 7fe98248b740 20 client.23679 populate_metadata read hostname 'smithi120' 2024-09-19T09:28:14.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.012+0000 7fe98248b740 1 --2- 172.21.15.120:0/2507276037 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b78e2e93f0 0x55b78e2eb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.012+0000 7fe98248b740 4 client.23679 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:28:14.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.012+0000 7fe98248b740 1 -- 172.21.15.120:0/2507276037 --> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b78e1ccd70 con 0x55b78e2e93f0 2024-09-19T09:28:14.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.013+0000 7fe97d64d640 1 --2- 172.21.15.120:0/2507276037 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b78e2e93f0 0x55b78e2eb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.013+0000 7fe97d64d640 1 --2- 172.21.15.120:0/2507276037 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b78e2e93f0 0x55b78e2eb7e0 crc :-1 s=READY pgs=620 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3df021ffc8d096d5 server_cookie=ea45a9ad5dfd27a in_seq=0 out_seq=0 2024-09-19T09:28:14.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.013+0000 7fe9617fa640 10 client.23679 ms_handle_connect on v2:172.21.15.153:6838/3290519661 2024-09-19T09:28:14.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.013+0000 7fe9617fa640 1 -- 172.21.15.120:0/2507276037 <== mds.0 v2:172.21.15.153:6838/3290519661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b78e1ccd70 con 0x55b78e2e93f0 2024-09-19T09:28:14.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.013+0000 7fe9617fa640 10 client.23679 handle_command_reply: tid=0 2024-09-19T09:28:14.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.013+0000 7fe98248b740 1 client.23679 shutdown 2024-09-19T09:28:14.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.013+0000 7fe98248b740 1 -- 172.21.15.120:0/2507276037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b78e25f5d0 msgr2=0x55b78e272c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:14.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.013+0000 7fe98248b740 1 --2- 172.21.15.120:0/2507276037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b78e25f5d0 0x55b78e272c90 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7fe96c0980b0 tx=0x7fe96c097fe0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.013+0000 7fe98248b740 1 -- 172.21.15.120:0/2507276037 shutdown_connections 2024-09-19T09:28:14.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.013+0000 7fe98248b740 1 --2- 172.21.15.120:0/2507276037 >> [v2:172.21.15.153:6838/3290519661,v1:172.21.15.153:6839/3290519661] conn(0x55b78e2e93f0 0x55b78e2eb7e0 crc :-1 s=READY pgs=620 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.013+0000 7fe98248b740 1 --2- 172.21.15.120:0/2507276037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b78e26a370 0x55b78e26e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.013+0000 7fe98248b740 1 --2- 172.21.15.120:0/2507276037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b78e260b40 0x55b78e26dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.013+0000 7fe98248b740 1 --2- 172.21.15.120:0/2507276037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b78e25f5d0 0x55b78e272c90 unknown :-1 s=CLOSED pgs=2563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.013+0000 7fe9617fa640 0 client.23679 ms_handle_reset on v2:172.21.15.153:6838/3290519661 2024-09-19T09:28:14.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.013+0000 7fe98248b740 1 -- 172.21.15.120:0/2507276037 >> 172.21.15.120:0/2507276037 conn(0x55b78e228310 msgr2=0x55b78e268850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:14.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.013+0000 7fe98248b740 1 -- 172.21.15.120:0/2507276037 shutdown_connections 2024-09-19T09:28:14.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.013+0000 7fe98248b740 1 -- 172.21.15.120:0/2507276037 wait complete. 2024-09-19T09:28:14.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.014+0000 7fe98248b740 20 client.23679 trim_cache size 0 max 16384 2024-09-19T09:28:14.028 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-19T09:28:14.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.014+0000 7fe9800b2640 1 -- 172.21.15.120:0/2306125446 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9640a3240 msgr2=0x7fe9640a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:14.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.014+0000 7fe9800b2640 1 --2- 172.21.15.120:0/2306125446 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9640a3240 0x7fe9640a5700 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fe970005fd0 tx=0x7fe970004ab0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.014+0000 7fe9800b2640 1 -- 172.21.15.120:0/2306125446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe978101440 msgr2=0x7fe9781986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:14.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.014+0000 7fe9800b2640 1 --2- 172.21.15.120:0/2306125446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe978101440 0x7fe9781986b0 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00d950 tx=0x7fe96c00de10 comp rx=0 tx=0).stop 2024-09-19T09:28:14.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.014+0000 7fe9800b2640 1 -- 172.21.15.120:0/2306125446 shutdown_connections 2024-09-19T09:28:14.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.014+0000 7fe9800b2640 1 --2- 172.21.15.120:0/2306125446 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe9640a3240 0x7fe9640a5700 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.014+0000 7fe9800b2640 1 --2- 172.21.15.120:0/2306125446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe978106050 0x7fe97819cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.014+0000 7fe9800b2640 1 --2- 172.21.15.120:0/2306125446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe978101df0 0x7fe978198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.014+0000 7fe9800b2640 1 --2- 172.21.15.120:0/2306125446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe978101440 0x7fe9781986b0 unknown :-1 s=CLOSED pgs=2559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.014+0000 7fe9800b2640 1 -- 172.21.15.120:0/2306125446 >> 172.21.15.120:0/2306125446 conn(0x7fe978077210 msgr2=0x7fe978100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:14.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.014+0000 7fe9800b2640 1 -- 172.21.15.120:0/2306125446 shutdown_connections 2024-09-19T09:28:14.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.014+0000 7fe9800b2640 1 -- 172.21.15.120:0/2306125446 wait complete. 2024-09-19T09:28:14.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:14 smithi196 ceph-mon[35150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:28:14.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:14 smithi196 ceph-mon[35150]: pgmap v1106: 97 pgs: 97 active+clean; 303 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.0 MiB/s wr, 486 op/s 2024-09-19T09:28:14.689 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:14 smithi153 ceph-mon[35071]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:28:14.689 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:14 smithi153 ceph-mon[35071]: pgmap v1106: 97 pgs: 97 active+clean; 303 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.0 MiB/s wr, 486 op/s 2024-09-19T09:28:14.804 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:14 smithi120 ceph-mon[31150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:28:14.804 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:14 smithi120 ceph-mon[31150]: pgmap v1106: 97 pgs: 97 active+clean; 303 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.0 MiB/s wr, 486 op/s 2024-09-19T09:28:14.835 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-19T09:28:14.836 INFO:tasks.fwd_scrub:done joining 2024-09-19T09:28:14.836 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-19T09:28:14.846 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T09:28:14.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.949+0000 7fa02be1e640 1 -- 172.21.15.120:0/2353517400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa024104f40 msgr2=0x7fa024107330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:14.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.949+0000 7fa02be1e640 1 --2- 172.21.15.120:0/2353517400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa024104f40 0x7fa024107330 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7fa014009900 tx=0x7fa01402a410 comp rx=0 tx=0).stop 2024-09-19T09:28:14.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.949+0000 7fa02be1e640 1 -- 172.21.15.120:0/2353517400 shutdown_connections 2024-09-19T09:28:14.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.949+0000 7fa02be1e640 1 --2- 172.21.15.120:0/2353517400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa024107870 0x7fa024109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.949+0000 7fa02be1e640 1 --2- 172.21.15.120:0/2353517400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa024104f40 0x7fa024107330 unknown :-1 s=CLOSED pgs=2564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.949+0000 7fa02be1e640 1 --2- 172.21.15.120:0/2353517400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa024102610 0x7fa024104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.949+0000 7fa02be1e640 1 -- 172.21.15.120:0/2353517400 >> 172.21.15.120:0/2353517400 conn(0x7fa0240fc170 msgr2=0x7fa0240fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:14.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.949+0000 7fa02be1e640 1 -- 172.21.15.120:0/2353517400 shutdown_connections 2024-09-19T09:28:14.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.949+0000 7fa02be1e640 1 -- 172.21.15.120:0/2353517400 wait complete. 2024-09-19T09:28:14.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.949+0000 7fa02be1e640 1 Processor -- start 2024-09-19T09:28:14.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.949+0000 7fa02be1e640 1 -- start start 2024-09-19T09:28:14.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.950+0000 7fa02be1e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa024102610 0x7fa0241980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.950+0000 7fa02be1e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa024104f40 0x7fa024198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.950+0000 7fa02be1e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa024107870 0x7fa02419f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.950+0000 7fa02be1e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa02410d8a0 con 0x7fa024107870 2024-09-19T09:28:14.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.950+0000 7fa02be1e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa02410d720 con 0x7fa024102610 2024-09-19T09:28:14.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.950+0000 7fa02be1e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa02410da20 con 0x7fa024104f40 2024-09-19T09:28:14.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.950+0000 7fa02a3bb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa024107870 0x7fa02419f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.950+0000 7fa02a3bb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa024107870 0x7fa02419f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40828/0 (socket says 172.21.15.120:40828) 2024-09-19T09:28:14.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.950+0000 7fa02a3bb640 1 -- 172.21.15.120:0/1589443667 learned_addr learned my addr 172.21.15.120:0/1589443667 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:14.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.950+0000 7fa0293b9640 1 --2- 172.21.15.120:0/1589443667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa024104f40 0x7fa024198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.950+0000 7fa029bba640 1 --2- 172.21.15.120:0/1589443667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa024102610 0x7fa0241980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.950+0000 7fa02a3bb640 1 -- 172.21.15.120:0/1589443667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa024104f40 msgr2=0x7fa024198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:14.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.950+0000 7fa02a3bb640 1 --2- 172.21.15.120:0/1589443667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa024104f40 0x7fa024198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.950+0000 7fa02a3bb640 1 -- 172.21.15.120:0/1589443667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa024102610 msgr2=0x7fa0241980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:14.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.950+0000 7fa02a3bb640 1 --2- 172.21.15.120:0/1589443667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa024102610 0x7fa0241980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.950+0000 7fa02a3bb640 1 -- 172.21.15.120:0/1589443667 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa02419fd00 con 0x7fa024107870 2024-09-19T09:28:14.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.950+0000 7fa0293b9640 1 --2- 172.21.15.120:0/1589443667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa024104f40 0x7fa024198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:14.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.950+0000 7fa02a3bb640 1 --2- 172.21.15.120:0/1589443667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa024107870 0x7fa02419f690 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00b9e0 tx=0x7fa01c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:14.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.950+0000 7fa01affd640 1 -- 172.21.15.120:0/1589443667 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01c019070 con 0x7fa024107870 2024-09-19T09:28:14.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.950+0000 7fa01affd640 1 -- 172.21.15.120:0/1589443667 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa01c014440 con 0x7fa024107870 2024-09-19T09:28:14.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.950+0000 7fa02be1e640 1 -- 172.21.15.120:0/1589443667 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa02419fff0 con 0x7fa024107870 2024-09-19T09:28:14.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.950+0000 7fa01affd640 1 -- 172.21.15.120:0/1589443667 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01c0125e0 con 0x7fa024107870 2024-09-19T09:28:14.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.950+0000 7fa02be1e640 1 -- 172.21.15.120:0/1589443667 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0241a0460 con 0x7fa024107870 2024-09-19T09:28:14.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.951+0000 7fa02be1e640 1 -- 172.21.15.120:0/1589443667 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0240fdc60 con 0x7fa024107870 2024-09-19T09:28:14.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.952+0000 7fa01affd640 1 -- 172.21.15.120:0/1589443667 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa01c01c070 con 0x7fa024107870 2024-09-19T09:28:14.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.953+0000 7fa01affd640 1 --2- 172.21.15.120:0/1589443667 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa0100a3250 0x7fa0100a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.953+0000 7fa029bba640 1 --2- 172.21.15.120:0/1589443667 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa0100a3250 0x7fa0100a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.953+0000 7fa01affd640 1 -- 172.21.15.120:0/1589443667 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa01c0cf5a0 con 0x7fa024107870 2024-09-19T09:28:14.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.953+0000 7fa029bba640 1 --2- 172.21.15.120:0/1589443667 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa0100a3250 0x7fa0100a5710 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fa02000a810 tx=0x7fa020008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:14.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:14.956+0000 7fa01affd640 1 -- 172.21.15.120:0/1589443667 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa01c09c2c0 con 0x7fa024107870 2024-09-19T09:28:15.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.128+0000 7fa02be1e640 1 -- 172.21.15.120:0/1589443667 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa02402d9c0 con 0x7fa024107870 2024-09-19T09:28:15.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.130+0000 7fa01affd640 1 -- 172.21.15.120:0/1589443667 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22844 (secure 0 0 0) 0x7fa01c09bbc0 con 0x7fa024107870 2024-09-19T09:28:15.131 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:28:15.131 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":85,"fsid":"507393d2-7664-11ef-bceb-c7b262605968","created":"2024-09-19T08:53:06.386526+0000","modified":"2024-09-19T09:16:31.379026+0000","last_up_change":"2024-09-19T08:59:55.692659+0000","last_in_change":"2024-09-19T08:59:32.808860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:56:07.830047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:00:43.774339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:00:45.786262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33ec3993-576c-428c-bf75-adcb17cd72a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6803","nonce":4131180019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6805","nonce":4131180019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6809","nonce":4131180019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6807","nonce":4131180019}]},"public_addr":"172.21.15.120:6803/4131180019","cluster_addr":"172.21.15.120:6805/4131180019","heartbeat_back_addr":"172.21.15.120:6809/4131180019","heartbeat_front_addr":"172.21.15.120:6807/4131180019","state":["exists","up"]},{"osd":1,"uuid":"450bbebf-2aac-47c6-8474-bbd37fd0a1a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6801","nonce":2259730294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6803","nonce":2259730294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6807","nonce":2259730294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6805","nonce":2259730294}]},"public_addr":"172.21.15.153:6801/2259730294","cluster_addr":"172.21.15.153:6803/2259730294","heartbeat_back_addr":"172.21.15.153:6807/2259730294","heartbeat_front_addr":"172.21.15.153:6805/2259730294","state":["exists","up"]},{"osd":2,"uuid":"ac8c4b83-8e8e-4803-b266-262f7873bd85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6801","nonce":2942286570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6803","nonce":2942286570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6807","nonce":2942286570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6805","nonce":2942286570}]},"public_addr":"172.21.15.196:6801/2942286570","cluster_addr":"172.21.15.196:6803/2942286570","heartbeat_back_addr":"172.21.15.196:6807/2942286570","heartbeat_front_addr":"172.21.15.196:6805/2942286570","state":["exists","up"]},{"osd":3,"uuid":"c1d9e507-8f02-403e-87da-9b4966fd8fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6811","nonce":713205668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6813","nonce":713205668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6817","nonce":713205668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6815","nonce":713205668}]},"public_addr":"172.21.15.120:6811/713205668","cluster_addr":"172.21.15.120:6813/713205668","heartbeat_back_addr":"172.21.15.120:6817/713205668","heartbeat_front_addr":"172.21.15.120:6815/713205668","state":["exists","up"]},{"osd":4,"uuid":"7f8bf3b9-0ec4-46f6-a406-d26d7971179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6809","nonce":2680865854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6811","nonce":2680865854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6815","nonce":2680865854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6813","nonce":2680865854}]},"public_addr":"172.21.15.153:6809/2680865854","cluster_addr":"172.21.15.153:6811/2680865854","heartbeat_back_addr":"172.21.15.153:6815/2680865854","heartbeat_front_addr":"172.21.15.153:6813/2680865854","state":["exists","up"]},{"osd":5,"uuid":"70b270b2-209d-4ff5-875a-04ee7b992a93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6809","nonce":3763331116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6811","nonce":3763331116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6815","nonce":3763331116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6813","nonce":3763331116}]},"public_addr":"172.21.15.196:6809/3763331116","cluster_addr":"172.21.15.196:6811/3763331116","heartbeat_back_addr":"172.21.15.196:6815/3763331116","heartbeat_front_addr":"172.21.15.196:6813/3763331116","state":["exists","up"]},{"osd":6,"uuid":"8087f69c-dc31-4522-a18d-4e6ab1b7cb27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6819","nonce":4171773765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6821","nonce":4171773765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6825","nonce":4171773765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6823","nonce":4171773765}]},"public_addr":"172.21.15.120:6819/4171773765","cluster_addr":"172.21.15.120:6821/4171773765","heartbeat_back_addr":"172.21.15.120:6825/4171773765","heartbeat_front_addr":"172.21.15.120:6823/4171773765","state":["exists","up"]},{"osd":7,"uuid":"c3b1ead3-150f-43fd-8924-f639203f4cdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6817","nonce":2535148453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6819","nonce":2535148453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6823","nonce":2535148453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6821","nonce":2535148453}]},"public_addr":"172.21.15.153:6817/2535148453","cluster_addr":"172.21.15.153:6819/2535148453","heartbeat_back_addr":"172.21.15.153:6823/2535148453","heartbeat_front_addr":"172.21.15.153:6821/2535148453","state":["exists","up"]},{"osd":8,"uuid":"a3273a2a-968e-4e93-a9cf-3879892bdeab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6817","nonce":186195906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6819","nonce":186195906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6823","nonce":186195906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6821","nonce":186195906}]},"public_addr":"172.21.15.196:6817/186195906","cluster_addr":"172.21.15.196:6819/186195906","heartbeat_back_addr":"172.21.15.196:6823/186195906","heartbeat_front_addr":"172.21.15.196:6821/186195906","state":["exists","up"]},{"osd":9,"uuid":"6d184513-dab8-45f7-bc08-fa2e8b2f10b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6827","nonce":1559320883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6829","nonce":1559320883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6833","nonce":1559320883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6831","nonce":1559320883}]},"public_addr":"172.21.15.120:6827/1559320883","cluster_addr":"172.21.15.120:6829/1559320883","heartbeat_back_addr":"172.21.15.120:6833/1559320883","heartbeat_front_addr":"172.21.15.120:6831/1559320883","state":["exists","up"]},{"osd":10,"uuid":"797db886-62f9-4316-a45b-4d50512e65d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6825","nonce":3216877560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6827","nonce":3216877560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6831","nonce":3216877560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6829","nonce":3216877560}]},"public_addr":"172.21.15.153:6825/3216877560","cluster_addr":"172.21.15.153:6827/3216877560","heartbeat_back_addr":"172.21.15.153:6831/3216877560","heartbeat_front_addr":"172.21.15.153:6829/3216877560","state":["exists","up"]},{"osd":11,"uuid":"17e0afaa-84eb-4b26-9810-bfa3cc7e0f3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6825","nonce":1857200740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6827","nonce":1857200740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6831","nonce":1857200740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6829","nonce":1857200740}]},"public_addr":"172.21.15.196:6825/1857200740","cluster_addr":"172.21.15.196:6827/1857200740","heartbeat_back_addr":"172.21.15.196:6831/1857200740","heartbeat_front_addr":"172.21.15.196:6829/1857200740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:55:15.595645+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:55:40.405487+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:04.938034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:29.718696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:55.636029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:57:21.257083+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:57:46.316552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:58:11.946863+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:58:37.098578+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:03.759023+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:28.659049+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:54.350475+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:0/3010783707":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6801/3562882246":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6800/3562882246":"2024-09-20T08:53:51.528772+0000","172.21.15.120:0/3224512035":"2024-09-20T08:53:51.528772+0000","172.21.15.120:0/3465105835":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/3317449610":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/2800697451":"2024-09-20T08:53:28.144376+0000","172.21.15.120:6801/2938446665":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/109127432":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6800/2938446665":"2024-09-20T08:53:28.144376+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:28:15.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.132+0000 7fa02be1e640 1 -- 172.21.15.120:0/1589443667 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa0100a3250 msgr2=0x7fa0100a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:15.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.132+0000 7fa02be1e640 1 --2- 172.21.15.120:0/1589443667 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa0100a3250 0x7fa0100a5710 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fa02000a810 tx=0x7fa020008040 comp rx=0 tx=0).stop 2024-09-19T09:28:15.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.132+0000 7fa02be1e640 1 -- 172.21.15.120:0/1589443667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa024107870 msgr2=0x7fa02419f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:15.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.132+0000 7fa02be1e640 1 --2- 172.21.15.120:0/1589443667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa024107870 0x7fa02419f690 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00b9e0 tx=0x7fa01c00bea0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.132+0000 7fa02be1e640 1 -- 172.21.15.120:0/1589443667 shutdown_connections 2024-09-19T09:28:15.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.132+0000 7fa02be1e640 1 --2- 172.21.15.120:0/1589443667 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa0100a3250 0x7fa0100a5710 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.132+0000 7fa02be1e640 1 --2- 172.21.15.120:0/1589443667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa024107870 0x7fa02419f690 unknown :-1 s=CLOSED pgs=2565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.132+0000 7fa02be1e640 1 --2- 172.21.15.120:0/1589443667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa024104f40 0x7fa024198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.132+0000 7fa02be1e640 1 --2- 172.21.15.120:0/1589443667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa024102610 0x7fa0241980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.132+0000 7fa02be1e640 1 -- 172.21.15.120:0/1589443667 >> 172.21.15.120:0/1589443667 conn(0x7fa0240fc170 msgr2=0x7fa0241043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:15.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.132+0000 7fa02be1e640 1 -- 172.21.15.120:0/1589443667 shutdown_connections 2024-09-19T09:28:15.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.132+0000 7fa02be1e640 1 -- 172.21.15.120:0/1589443667 wait complete. 2024-09-19T09:28:15.145 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T08:56:07.830047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T09:00:43.774339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T09:00:45.786262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T09:28:15.145 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T09:28:15.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.245+0000 7f210e152640 1 -- 172.21.15.120:0/3505683751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2108104ec0 msgr2=0x7f21081072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:15.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.245+0000 7f210e152640 1 --2- 172.21.15.120:0/3505683751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2108104ec0 0x7f21081072b0 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7f20fc009900 tx=0x7f20fc02a410 comp rx=0 tx=0).stop 2024-09-19T09:28:15.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.245+0000 7f210e152640 1 -- 172.21.15.120:0/3505683751 shutdown_connections 2024-09-19T09:28:15.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.245+0000 7f210e152640 1 --2- 172.21.15.120:0/3505683751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21081077f0 0x7f2108109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.245+0000 7f210e152640 1 --2- 172.21.15.120:0/3505683751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2108104ec0 0x7f21081072b0 unknown :-1 s=CLOSED pgs=2566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.245+0000 7f210e152640 1 --2- 172.21.15.120:0/3505683751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2108102590 0x7f2108104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.245+0000 7f210e152640 1 -- 172.21.15.120:0/3505683751 >> 172.21.15.120:0/3505683751 conn(0x7f21080fc1b0 msgr2=0x7f21080fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:15.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.245+0000 7f210e152640 1 -- 172.21.15.120:0/3505683751 shutdown_connections 2024-09-19T09:28:15.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.245+0000 7f210e152640 1 -- 172.21.15.120:0/3505683751 wait complete. 2024-09-19T09:28:15.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.246+0000 7f210e152640 1 Processor -- start 2024-09-19T09:28:15.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.246+0000 7f210e152640 1 -- start start 2024-09-19T09:28:15.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.246+0000 7f210e152640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2108102590 0x7f2108197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:15.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.246+0000 7f210e152640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2108104ec0 0x7f2108198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:15.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.246+0000 7f210e152640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21081077f0 0x7f210819f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:15.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.246+0000 7f210e152640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f210810d7a0 con 0x7f21081077f0 2024-09-19T09:28:15.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.246+0000 7f210e152640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f210810d620 con 0x7f2108102590 2024-09-19T09:28:15.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.246+0000 7f210e152640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f210810d920 con 0x7f2108104ec0 2024-09-19T09:28:15.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.246+0000 7f2107fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21081077f0 0x7f210819f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:15.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.246+0000 7f2107fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21081077f0 0x7f210819f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40846/0 (socket says 172.21.15.120:40846) 2024-09-19T09:28:15.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.246+0000 7f2107fff640 1 -- 172.21.15.120:0/3553260181 learned_addr learned my addr 172.21.15.120:0/3553260181 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:15.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.247+0000 7f2106ffd640 1 --2- 172.21.15.120:0/3553260181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2108104ec0 0x7f2108198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:15.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.247+0000 7f21077fe640 1 --2- 172.21.15.120:0/3553260181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2108102590 0x7f2108197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:15.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.247+0000 7f2107fff640 1 -- 172.21.15.120:0/3553260181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2108104ec0 msgr2=0x7f2108198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:15.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.247+0000 7f2107fff640 1 --2- 172.21.15.120:0/3553260181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2108104ec0 0x7f2108198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.247+0000 7f2107fff640 1 -- 172.21.15.120:0/3553260181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2108102590 msgr2=0x7f2108197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:15.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.247+0000 7f2107fff640 1 --2- 172.21.15.120:0/3553260181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2108102590 0x7f2108197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.247+0000 7f2107fff640 1 -- 172.21.15.120:0/3553260181 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f210819fb20 con 0x7f21081077f0 2024-09-19T09:28:15.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.247+0000 7f2106ffd640 1 --2- 172.21.15.120:0/3553260181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2108104ec0 0x7f2108198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:15.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.247+0000 7f2107fff640 1 --2- 172.21.15.120:0/3553260181 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21081077f0 0x7f210819f4b0 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7f210000dac0 tx=0x7f210000df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:15.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.247+0000 7f2104ff9640 1 -- 172.21.15.120:0/3553260181 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100017070 con 0x7f21081077f0 2024-09-19T09:28:15.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.247+0000 7f210e152640 1 -- 172.21.15.120:0/3553260181 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f210819fe10 con 0x7f21081077f0 2024-09-19T09:28:15.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.247+0000 7f2104ff9640 1 -- 172.21.15.120:0/3553260181 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2100003e50 con 0x7f21081077f0 2024-09-19T09:28:15.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.247+0000 7f2104ff9640 1 -- 172.21.15.120:0/3553260181 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21000106c0 con 0x7f21081077f0 2024-09-19T09:28:15.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.247+0000 7f210e152640 1 -- 172.21.15.120:0/3553260181 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21081a0280 con 0x7f21081077f0 2024-09-19T09:28:15.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.248+0000 7f210e152640 1 -- 172.21.15.120:0/3553260181 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21080fdca0 con 0x7f21081077f0 2024-09-19T09:28:15.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.249+0000 7f2104ff9640 1 -- 172.21.15.120:0/3553260181 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f2100003970 con 0x7f21081077f0 2024-09-19T09:28:15.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.250+0000 7f2104ff9640 1 --2- 172.21.15.120:0/3553260181 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f20f00a3250 0x7f20f00a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:15.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.250+0000 7f21077fe640 1 --2- 172.21.15.120:0/3553260181 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f20f00a3250 0x7f20f00a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:15.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.250+0000 7f2104ff9640 1 -- 172.21.15.120:0/3553260181 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f21000cf6c0 con 0x7f21081077f0 2024-09-19T09:28:15.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.250+0000 7f21077fe640 1 --2- 172.21.15.120:0/3553260181 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f20f00a3250 0x7f20f00a5710 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009b60 tx=0x7f20f8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:15.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.254+0000 7f2104ff9640 1 -- 172.21.15.120:0/3553260181 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f210009c3e0 con 0x7f21081077f0 2024-09-19T09:28:15.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.420+0000 7f210e152640 1 -- 172.21.15.120:0/3553260181 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2108062e90 con 0x7f21081077f0 2024-09-19T09:28:15.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.420+0000 7f2104ff9640 1 -- 172.21.15.120:0/3553260181 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7f210009bce0 con 0x7f21081077f0 2024-09-19T09:28:15.422 INFO:teuthology.orchestra.run.smithi120.stdout:pg_num: 1 2024-09-19T09:28:15.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.422+0000 7f210e152640 1 -- 172.21.15.120:0/3553260181 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f20f00a3250 msgr2=0x7f20f00a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:15.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.422+0000 7f210e152640 1 --2- 172.21.15.120:0/3553260181 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f20f00a3250 0x7f20f00a5710 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009b60 tx=0x7f20f8005f90 comp rx=0 tx=0).stop 2024-09-19T09:28:15.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.423+0000 7f210e152640 1 -- 172.21.15.120:0/3553260181 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21081077f0 msgr2=0x7f210819f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:15.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.423+0000 7f210e152640 1 --2- 172.21.15.120:0/3553260181 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21081077f0 0x7f210819f4b0 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7f210000dac0 tx=0x7f210000df80 comp rx=0 tx=0).stop 2024-09-19T09:28:15.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.423+0000 7f210e152640 1 -- 172.21.15.120:0/3553260181 shutdown_connections 2024-09-19T09:28:15.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.423+0000 7f210e152640 1 --2- 172.21.15.120:0/3553260181 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f20f00a3250 0x7f20f00a5710 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.423+0000 7f210e152640 1 --2- 172.21.15.120:0/3553260181 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21081077f0 0x7f210819f4b0 unknown :-1 s=CLOSED pgs=2567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.423+0000 7f210e152640 1 --2- 172.21.15.120:0/3553260181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2108104ec0 0x7f2108198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.423+0000 7f210e152640 1 --2- 172.21.15.120:0/3553260181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2108102590 0x7f2108197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.423+0000 7f210e152640 1 -- 172.21.15.120:0/3553260181 >> 172.21.15.120:0/3553260181 conn(0x7f21080fc1b0 msgr2=0x7f2108102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:15.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.423+0000 7f210e152640 1 -- 172.21.15.120:0/3553260181 shutdown_connections 2024-09-19T09:28:15.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.423+0000 7f210e152640 1 -- 172.21.15.120:0/3553260181 wait complete. 2024-09-19T09:28:15.435 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:15 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1589443667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:28:15.435 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T09:28:15.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.536+0000 7fae695b7640 1 -- 172.21.15.120:0/4006664486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae641025b0 msgr2=0x7fae64108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:15.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.536+0000 7fae695b7640 1 --2- 172.21.15.120:0/4006664486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae641025b0 0x7fae64108e40 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7fae5c009ca0 tx=0x7fae5c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.536+0000 7fae695b7640 1 -- 172.21.15.120:0/4006664486 shutdown_connections 2024-09-19T09:28:15.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.536+0000 7fae695b7640 1 --2- 172.21.15.120:0/4006664486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae641025b0 0x7fae64108e40 unknown :-1 s=CLOSED pgs=2568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.536+0000 7fae695b7640 1 --2- 172.21.15.120:0/4006664486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae64101bf0 0x7fae64102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.536+0000 7fae695b7640 1 --2- 172.21.15.120:0/4006664486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae64100a00 0x7fae64100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.536+0000 7fae695b7640 1 -- 172.21.15.120:0/4006664486 >> 172.21.15.120:0/4006664486 conn(0x7fae640fc140 msgr2=0x7fae640fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:15.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.536+0000 7fae695b7640 1 -- 172.21.15.120:0/4006664486 shutdown_connections 2024-09-19T09:28:15.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.536+0000 7fae695b7640 1 -- 172.21.15.120:0/4006664486 wait complete. 2024-09-19T09:28:15.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.537+0000 7fae695b7640 1 Processor -- start 2024-09-19T09:28:15.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.537+0000 7fae695b7640 1 -- start start 2024-09-19T09:28:15.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.537+0000 7fae695b7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae64100a00 0x7fae64197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:15.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.537+0000 7fae695b7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae64101bf0 0x7fae64198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:15.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.537+0000 7fae695b7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae641025b0 0x7fae6419f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:15.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.537+0000 7fae695b7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fae6410d790 con 0x7fae64101bf0 2024-09-19T09:28:15.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.537+0000 7fae695b7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fae6410d610 con 0x7fae64100a00 2024-09-19T09:28:15.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.537+0000 7fae695b7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae6410d910 con 0x7fae641025b0 2024-09-19T09:28:15.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.537+0000 7fae627fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae64101bf0 0x7fae64198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:15.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.537+0000 7fae627fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae64101bf0 0x7fae64198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40856/0 (socket says 172.21.15.120:40856) 2024-09-19T09:28:15.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.537+0000 7fae627fc640 1 -- 172.21.15.120:0/1153944687 learned_addr learned my addr 172.21.15.120:0/1153944687 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:15.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.538+0000 7fae62ffd640 1 --2- 172.21.15.120:0/1153944687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae64100a00 0x7fae64197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:15.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.538+0000 7fae637fe640 1 --2- 172.21.15.120:0/1153944687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae641025b0 0x7fae6419f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:15.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.538+0000 7fae627fc640 1 -- 172.21.15.120:0/1153944687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae641025b0 msgr2=0x7fae6419f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:15.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.538+0000 7fae627fc640 1 --2- 172.21.15.120:0/1153944687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae641025b0 0x7fae6419f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.538+0000 7fae627fc640 1 -- 172.21.15.120:0/1153944687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae64100a00 msgr2=0x7fae64197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:15.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.538+0000 7fae627fc640 1 --2- 172.21.15.120:0/1153944687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae64100a00 0x7fae64197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.538+0000 7fae627fc640 1 -- 172.21.15.120:0/1153944687 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6419f9d0 con 0x7fae64101bf0 2024-09-19T09:28:15.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.538+0000 7fae62ffd640 1 --2- 172.21.15.120:0/1153944687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae64100a00 0x7fae64197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:28:15.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.538+0000 7fae637fe640 1 --2- 172.21.15.120:0/1153944687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae641025b0 0x7fae6419f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:28:15.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.538+0000 7fae627fc640 1 --2- 172.21.15.120:0/1153944687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae64101bf0 0x7fae64198300 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7fae5800c890 tx=0x7fae5800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:15.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.538+0000 7fae4bfff640 1 -- 172.21.15.120:0/1153944687 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae58004220 con 0x7fae64101bf0 2024-09-19T09:28:15.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.538+0000 7fae695b7640 1 -- 172.21.15.120:0/1153944687 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae6419fcc0 con 0x7fae64101bf0 2024-09-19T09:28:15.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.538+0000 7fae4bfff640 1 -- 172.21.15.120:0/1153944687 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae580043c0 con 0x7fae64101bf0 2024-09-19T09:28:15.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.538+0000 7fae4bfff640 1 -- 172.21.15.120:0/1153944687 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae58004a00 con 0x7fae64101bf0 2024-09-19T09:28:15.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.538+0000 7fae695b7640 1 -- 172.21.15.120:0/1153944687 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae641a01b0 con 0x7fae64101bf0 2024-09-19T09:28:15.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.539+0000 7fae695b7640 1 -- 172.21.15.120:0/1153944687 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae64107dc0 con 0x7fae64101bf0 2024-09-19T09:28:15.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.540+0000 7fae4bfff640 1 -- 172.21.15.120:0/1153944687 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fae580079b0 con 0x7fae64101bf0 2024-09-19T09:28:15.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.541+0000 7fae4bfff640 1 --2- 172.21.15.120:0/1153944687 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fae4c0a3250 0x7fae4c0a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:15.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.541+0000 7fae62ffd640 1 --2- 172.21.15.120:0/1153944687 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fae4c0a3250 0x7fae4c0a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:15.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.541+0000 7fae4bfff640 1 -- 172.21.15.120:0/1153944687 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fae580cf750 con 0x7fae64101bf0 2024-09-19T09:28:15.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.541+0000 7fae62ffd640 1 --2- 172.21.15.120:0/1153944687 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fae4c0a3250 0x7fae4c0a5710 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fae64199290 tx=0x7fae54005dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:15.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.545+0000 7fae4bfff640 1 -- 172.21.15.120:0/1153944687 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fae58014030 con 0x7fae64101bf0 2024-09-19T09:28:15.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:15 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1589443667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:28:15.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:15 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1589443667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:28:15.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.710+0000 7fae695b7640 1 -- 172.21.15.120:0/1153944687 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fae64062f00 con 0x7fae64101bf0 2024-09-19T09:28:15.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.710+0000 7fae4bfff640 1 -- 172.21.15.120:0/1153944687 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) ==== 104+0+11 (secure 0 0 0) 0x7fae5809c470 con 0x7fae64101bf0 2024-09-19T09:28:15.713 INFO:teuthology.orchestra.run.smithi120.stdout:pg_num: 32 2024-09-19T09:28:15.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.712+0000 7fae695b7640 1 -- 172.21.15.120:0/1153944687 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fae4c0a3250 msgr2=0x7fae4c0a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:15.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.712+0000 7fae695b7640 1 --2- 172.21.15.120:0/1153944687 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fae4c0a3250 0x7fae4c0a5710 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fae64199290 tx=0x7fae54005dd0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.712+0000 7fae695b7640 1 -- 172.21.15.120:0/1153944687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae64101bf0 msgr2=0x7fae64198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:15.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.712+0000 7fae695b7640 1 --2- 172.21.15.120:0/1153944687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae64101bf0 0x7fae64198300 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7fae5800c890 tx=0x7fae5800cd50 comp rx=0 tx=0).stop 2024-09-19T09:28:15.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.713+0000 7fae695b7640 1 -- 172.21.15.120:0/1153944687 shutdown_connections 2024-09-19T09:28:15.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.713+0000 7fae695b7640 1 --2- 172.21.15.120:0/1153944687 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fae4c0a3250 0x7fae4c0a5710 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.713+0000 7fae695b7640 1 --2- 172.21.15.120:0/1153944687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae641025b0 0x7fae6419f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.713+0000 7fae695b7640 1 --2- 172.21.15.120:0/1153944687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae64101bf0 0x7fae64198300 unknown :-1 s=CLOSED pgs=2569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.713+0000 7fae695b7640 1 --2- 172.21.15.120:0/1153944687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae64100a00 0x7fae64197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.713+0000 7fae695b7640 1 -- 172.21.15.120:0/1153944687 >> 172.21.15.120:0/1153944687 conn(0x7fae640fc140 msgr2=0x7fae64104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:15.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.713+0000 7fae695b7640 1 -- 172.21.15.120:0/1153944687 shutdown_connections 2024-09-19T09:28:15.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.713+0000 7fae695b7640 1 -- 172.21.15.120:0/1153944687 wait complete. 2024-09-19T09:28:15.726 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T09:28:15.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.828+0000 7fa1ee986640 1 -- 172.21.15.120:0/327865314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1e81025b0 msgr2=0x7fa1e8108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:15.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.828+0000 7fa1ee986640 1 --2- 172.21.15.120:0/327865314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1e81025b0 0x7fa1e8108e40 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0009ca0 tx=0x7fa1e002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.828+0000 7fa1ee986640 1 -- 172.21.15.120:0/327865314 shutdown_connections 2024-09-19T09:28:15.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.828+0000 7fa1ee986640 1 --2- 172.21.15.120:0/327865314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1e81025b0 0x7fa1e8108e40 unknown :-1 s=CLOSED pgs=2570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.828+0000 7fa1ee986640 1 --2- 172.21.15.120:0/327865314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1e8101bf0 0x7fa1e8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.828+0000 7fa1ee986640 1 --2- 172.21.15.120:0/327865314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1e8100a00 0x7fa1e8100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.828+0000 7fa1ee986640 1 -- 172.21.15.120:0/327865314 >> 172.21.15.120:0/327865314 conn(0x7fa1e80fc140 msgr2=0x7fa1e80fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:15.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.828+0000 7fa1ee986640 1 -- 172.21.15.120:0/327865314 shutdown_connections 2024-09-19T09:28:15.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.828+0000 7fa1ee986640 1 -- 172.21.15.120:0/327865314 wait complete. 2024-09-19T09:28:15.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.828+0000 7fa1ee986640 1 Processor -- start 2024-09-19T09:28:15.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.828+0000 7fa1ee986640 1 -- start start 2024-09-19T09:28:15.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.829+0000 7fa1ee986640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1e8100a00 0x7fa1e8197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:15.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.829+0000 7fa1ee986640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1e8101bf0 0x7fa1e8198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:15.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.829+0000 7fa1ee986640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1e81025b0 0x7fa1e819f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:15.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.829+0000 7fa1ee986640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e810d790 con 0x7fa1e8101bf0 2024-09-19T09:28:15.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.829+0000 7fa1ee986640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e810d610 con 0x7fa1e81025b0 2024-09-19T09:28:15.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.829+0000 7fa1ee986640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e810d910 con 0x7fa1e8100a00 2024-09-19T09:28:15.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.829+0000 7fa1e77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1e8101bf0 0x7fa1e8198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:15.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.829+0000 7fa1e77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1e8101bf0 0x7fa1e8198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40870/0 (socket says 172.21.15.120:40870) 2024-09-19T09:28:15.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.829+0000 7fa1e77fe640 1 -- 172.21.15.120:0/3148083081 learned_addr learned my addr 172.21.15.120:0/3148083081 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:15.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.829+0000 7fa1e7fff640 1 --2- 172.21.15.120:0/3148083081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1e8100a00 0x7fa1e8197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:15.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.829+0000 7fa1ecf23640 1 --2- 172.21.15.120:0/3148083081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1e81025b0 0x7fa1e819f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:15.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.829+0000 7fa1e77fe640 1 -- 172.21.15.120:0/3148083081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1e8100a00 msgr2=0x7fa1e8197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:15.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.829+0000 7fa1e77fe640 1 --2- 172.21.15.120:0/3148083081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1e8100a00 0x7fa1e8197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.829+0000 7fa1e77fe640 1 -- 172.21.15.120:0/3148083081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1e81025b0 msgr2=0x7fa1e819f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:15.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.829+0000 7fa1e77fe640 1 --2- 172.21.15.120:0/3148083081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1e81025b0 0x7fa1e819f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:15.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.829+0000 7fa1e77fe640 1 -- 172.21.15.120:0/3148083081 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e819fa40 con 0x7fa1e8101bf0 2024-09-19T09:28:15.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.829+0000 7fa1ecf23640 1 --2- 172.21.15.120:0/3148083081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1e81025b0 0x7fa1e819f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:15.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.829+0000 7fa1e77fe640 1 --2- 172.21.15.120:0/3148083081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1e8101bf0 0x7fa1e8198360 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc00d950 tx=0x7fa1dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:15.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.829+0000 7fa1e57fa640 1 -- 172.21.15.120:0/3148083081 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc017070 con 0x7fa1e8101bf0 2024-09-19T09:28:15.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.829+0000 7fa1ee986640 1 -- 172.21.15.120:0/3148083081 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e819fd30 con 0x7fa1e8101bf0 2024-09-19T09:28:15.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.829+0000 7fa1e57fa640 1 -- 172.21.15.120:0/3148083081 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1dc009c30 con 0x7fa1e8101bf0 2024-09-19T09:28:15.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.829+0000 7fa1e57fa640 1 -- 172.21.15.120:0/3148083081 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc00f6b0 con 0x7fa1e8101bf0 2024-09-19T09:28:15.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.829+0000 7fa1ee986640 1 -- 172.21.15.120:0/3148083081 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e81a0220 con 0x7fa1e8101bf0 2024-09-19T09:28:15.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.830+0000 7fa1ee986640 1 -- 172.21.15.120:0/3148083081 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1e8107e90 con 0x7fa1e8101bf0 2024-09-19T09:28:15.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.831+0000 7fa1e57fa640 1 -- 172.21.15.120:0/3148083081 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa1dc0109d0 con 0x7fa1e8101bf0 2024-09-19T09:28:15.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.832+0000 7fa1e57fa640 1 --2- 172.21.15.120:0/3148083081 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa1d00a3250 0x7fa1d00a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:15.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.832+0000 7fa1e7fff640 1 --2- 172.21.15.120:0/3148083081 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa1d00a3250 0x7fa1d00a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:15.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.832+0000 7fa1e57fa640 1 -- 172.21.15.120:0/3148083081 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa1dc0cf4a0 con 0x7fa1e8101bf0 2024-09-19T09:28:15.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.833+0000 7fa1e7fff640 1 --2- 172.21.15.120:0/3148083081 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa1d00a3250 0x7fa1d00a5710 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8005fd0 tx=0x7fa1d800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:15.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:15.835+0000 7fa1e57fa640 1 -- 172.21.15.120:0/3148083081 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa1dc009de0 con 0x7fa1e8101bf0 2024-09-19T09:28:16.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.002+0000 7fa1ee986640 1 -- 172.21.15.120:0/3148083081 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa1e802da30 con 0x7fa1e8101bf0 2024-09-19T09:28:16.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.002+0000 7fa1e57fa640 1 -- 172.21.15.120:0/3148083081 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) ==== 100+0+11 (secure 0 0 0) 0x7fa1dc09c1c0 con 0x7fa1e8101bf0 2024-09-19T09:28:16.005 INFO:teuthology.orchestra.run.smithi120.stdout:pg_num: 64 2024-09-19T09:28:16.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.004+0000 7fa1ee986640 1 -- 172.21.15.120:0/3148083081 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa1d00a3250 msgr2=0x7fa1d00a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:16.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.004+0000 7fa1ee986640 1 --2- 172.21.15.120:0/3148083081 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa1d00a3250 0x7fa1d00a5710 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8005fd0 tx=0x7fa1d800b410 comp rx=0 tx=0).stop 2024-09-19T09:28:16.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.004+0000 7fa1ee986640 1 -- 172.21.15.120:0/3148083081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1e8101bf0 msgr2=0x7fa1e8198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:16.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.004+0000 7fa1ee986640 1 --2- 172.21.15.120:0/3148083081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1e8101bf0 0x7fa1e8198360 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc00d950 tx=0x7fa1dc00de10 comp rx=0 tx=0).stop 2024-09-19T09:28:16.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.004+0000 7fa1ee986640 1 -- 172.21.15.120:0/3148083081 shutdown_connections 2024-09-19T09:28:16.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.004+0000 7fa1ee986640 1 --2- 172.21.15.120:0/3148083081 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fa1d00a3250 0x7fa1d00a5710 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.005+0000 7fa1ee986640 1 --2- 172.21.15.120:0/3148083081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1e81025b0 0x7fa1e819f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.005+0000 7fa1ee986640 1 --2- 172.21.15.120:0/3148083081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1e8101bf0 0x7fa1e8198360 unknown :-1 s=CLOSED pgs=2571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.005+0000 7fa1ee986640 1 --2- 172.21.15.120:0/3148083081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1e8100a00 0x7fa1e8197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.005+0000 7fa1ee986640 1 -- 172.21.15.120:0/3148083081 >> 172.21.15.120:0/3148083081 conn(0x7fa1e80fc140 msgr2=0x7fa1e8105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:16.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.005+0000 7fa1ee986640 1 -- 172.21.15.120:0/3148083081 shutdown_connections 2024-09-19T09:28:16.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.005+0000 7fa1ee986640 1 -- 172.21.15.120:0/3148083081 wait complete. 2024-09-19T09:28:16.018 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-19T09:28:16.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.115+0000 7f84c2e77640 1 -- 172.21.15.120:0/2343302148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84bc102d20 msgr2=0x7f84bc1031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:16.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.115+0000 7f84c2e77640 1 --2- 172.21.15.120:0/2343302148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84bc102d20 0x7f84bc1031a0 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7f84a8009900 tx=0x7f84a802a410 comp rx=0 tx=0).stop 2024-09-19T09:28:16.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.115+0000 7f84c2e77640 1 -- 172.21.15.120:0/2343302148 shutdown_connections 2024-09-19T09:28:16.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.115+0000 7f84c2e77640 1 --2- 172.21.15.120:0/2343302148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84bc1036e0 0x7f84bc109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.115+0000 7f84c2e77640 1 --2- 172.21.15.120:0/2343302148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84bc102d20 0x7f84bc1031a0 unknown :-1 s=CLOSED pgs=2572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.115+0000 7f84c2e77640 1 --2- 172.21.15.120:0/2343302148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84bc101b30 0x7f84bc101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.115+0000 7f84c2e77640 1 -- 172.21.15.120:0/2343302148 >> 172.21.15.120:0/2343302148 conn(0x7f84bc0fd2b0 msgr2=0x7f84bc0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:16.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.115+0000 7f84c2e77640 1 -- 172.21.15.120:0/2343302148 shutdown_connections 2024-09-19T09:28:16.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.115+0000 7f84c2e77640 1 -- 172.21.15.120:0/2343302148 wait complete. 2024-09-19T09:28:16.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.116+0000 7f84c2e77640 1 Processor -- start 2024-09-19T09:28:16.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.116+0000 7f84c2e77640 1 -- start start 2024-09-19T09:28:16.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.116+0000 7f84c2e77640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84bc101b30 0x7f84bc19a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:16.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.116+0000 7f84c2e77640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84bc102d20 0x7f84bc19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:16.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.116+0000 7f84c2e77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84bc1036e0 0x7f84bc1a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:16.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.116+0000 7f84c2e77640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc10d830 con 0x7f84bc1036e0 2024-09-19T09:28:16.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.116+0000 7f84c2e77640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc10d6b0 con 0x7f84bc101b30 2024-09-19T09:28:16.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.116+0000 7f84c2e77640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc10d9b0 con 0x7f84bc102d20 2024-09-19T09:28:16.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.116+0000 7f84c1414640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84bc1036e0 0x7f84bc1a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:16.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.116+0000 7f84c1414640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84bc1036e0 0x7f84bc1a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40882/0 (socket says 172.21.15.120:40882) 2024-09-19T09:28:16.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.116+0000 7f84c1414640 1 -- 172.21.15.120:0/61509941 learned_addr learned my addr 172.21.15.120:0/61509941 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:16.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.116+0000 7f84c1414640 1 -- 172.21.15.120:0/61509941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84bc102d20 msgr2=0x7f84bc19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:16.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.116+0000 7f84c1414640 1 --2- 172.21.15.120:0/61509941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84bc102d20 0x7f84bc19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.116+0000 7f84c1414640 1 -- 172.21.15.120:0/61509941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84bc101b30 msgr2=0x7f84bc19a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:16.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.116+0000 7f84c1414640 1 --2- 172.21.15.120:0/61509941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84bc101b30 0x7f84bc19a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.116+0000 7f84c1414640 1 -- 172.21.15.120:0/61509941 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84bc1a1d00 con 0x7f84bc1036e0 2024-09-19T09:28:16.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.116+0000 7f84c1414640 1 --2- 172.21.15.120:0/61509941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84bc1036e0 0x7f84bc1a1690 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7f84b400cce0 tx=0x7f84b400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:16.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.116+0000 7f84b9ffb640 1 -- 172.21.15.120:0/61509941 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b400f450 con 0x7f84bc1036e0 2024-09-19T09:28:16.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.117+0000 7f84b9ffb640 1 -- 172.21.15.120:0/61509941 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b400fad0 con 0x7f84bc1036e0 2024-09-19T09:28:16.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.117+0000 7f84c2e77640 1 -- 172.21.15.120:0/61509941 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84bc1a1ff0 con 0x7f84bc1036e0 2024-09-19T09:28:16.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.117+0000 7f84b9ffb640 1 -- 172.21.15.120:0/61509941 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b4018c10 con 0x7f84bc1036e0 2024-09-19T09:28:16.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.117+0000 7f84c2e77640 1 -- 172.21.15.120:0/61509941 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84bc1a2590 con 0x7f84bc1036e0 2024-09-19T09:28:16.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.118+0000 7f84c2e77640 1 -- 172.21.15.120:0/61509941 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84bc1a2820 con 0x7f84bc1036e0 2024-09-19T09:28:16.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.119+0000 7f84b9ffb640 1 -- 172.21.15.120:0/61509941 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f84b400f5f0 con 0x7f84bc1036e0 2024-09-19T09:28:16.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.119+0000 7f84b9ffb640 1 --2- 172.21.15.120:0/61509941 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f84ac0a32d0 0x7f84ac0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:16.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.119+0000 7f84b9ffb640 1 -- 172.21.15.120:0/61509941 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f84b40d0000 con 0x7f84bc1036e0 2024-09-19T09:28:16.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.119+0000 7f84c0c13640 1 --2- 172.21.15.120:0/61509941 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f84ac0a32d0 0x7f84ac0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:16.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.120+0000 7f84c0c13640 1 --2- 172.21.15.120:0/61509941 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f84ac0a32d0 0x7f84ac0a5790 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f84bc102b80 tx=0x7f84b0006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:16.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.123+0000 7f84b9ffb640 1 -- 172.21.15.120:0/61509941 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f84b4014030 con 0x7f84bc1036e0 2024-09-19T09:28:16.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.337+0000 7f84c2e77640 1 -- 172.21.15.120:0/61509941 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f84bc1a2a90 con 0x7f84bc1036e0 2024-09-19T09:28:16.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.345+0000 7f84b9ffb640 1 -- 172.21.15.120:0/61509941 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+742349 (secure 0 0 0) 0x7f84b409cd20 con 0x7f84bc1036e0 2024-09-19T09:28:16.357 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:28:16.357 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:28:16.357 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 20, 2024-09-19T09:28:16.357 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.357 INFO:teuthology.orchestra.run.smithi120.stdout: "active_gid": 14150, 2024-09-19T09:28:16.357 INFO:teuthology.orchestra.run.smithi120.stdout: "active_name": "x", 2024-09-19T09:28:16.357 INFO:teuthology.orchestra.run.smithi120.stdout: "active_addrs": { 2024-09-19T09:28:16.357 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-19T09:28:16.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.357 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-19T09:28:16.357 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6800", 2024-09-19T09:28:16.358 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3841841273 2024-09-19T09:28:16.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.358 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-19T09:28:16.358 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6801", 2024-09-19T09:28:16.358 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3841841273 2024-09-19T09:28:16.358 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.358 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-19T09:28:16.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.358 INFO:teuthology.orchestra.run.smithi120.stdout: "active_addr": "172.21.15.120:6801/3841841273", 2024-09-19T09:28:16.358 INFO:teuthology.orchestra.run.smithi120.stdout: "active_change": "2024-09-19T08:53:51.528880+0000", 2024-09-19T09:28:16.358 INFO:teuthology.orchestra.run.smithi120.stdout: "active_mgr_features": 4540701547738038271, 2024-09-19T09:28:16.358 INFO:teuthology.orchestra.run.smithi120.stdout: "available": true, 2024-09-19T09:28:16.358 INFO:teuthology.orchestra.run.smithi120.stdout: "standbys": [ 2024-09-19T09:28:16.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.359 INFO:teuthology.orchestra.run.smithi120.stdout: "gid": 24106, 2024-09-19T09:28:16.359 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "z", 2024-09-19T09:28:16.359 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_features": 4540701547738038271, 2024-09-19T09:28:16.359 INFO:teuthology.orchestra.run.smithi120.stdout: "available_modules": [ 2024-09-19T09:28:16.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.359 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "alerts", 2024-09-19T09:28:16.359 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.359 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.359 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.359 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-19T09:28:16.359 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-19T09:28:16.359 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.359 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.360 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.360 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-19T09:28:16.360 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.360 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.360 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.360 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T09:28:16.360 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.360 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.360 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.360 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.360 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.360 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.361 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.361 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.361 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.361 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.361 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.361 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.361 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.361 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.361 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.361 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.361 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.361 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.361 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.361 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.362 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.362 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.362 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.362 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.362 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.362 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.362 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.362 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.362 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.362 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.362 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.362 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.362 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.363 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.363 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.363 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.363 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.363 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.363 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.363 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.363 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.363 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.363 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.363 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.363 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.363 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.364 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.364 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.364 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.364 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.364 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.364 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.364 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.364 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.364 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.364 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.364 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.364 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.365 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.365 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.365 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.365 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.365 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.365 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.365 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.365 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.365 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.365 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.365 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.365 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_destination": { 2024-09-19T09:28:16.365 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_destination", 2024-09-19T09:28:16.366 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.366 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.366 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.366 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.366 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.366 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.366 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.366 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Email address to send alerts to", 2024-09-19T09:28:16.366 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.366 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.366 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.366 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_from_name": { 2024-09-19T09:28:16.366 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_from_name", 2024-09-19T09:28:16.367 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.367 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.367 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.367 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "Ceph", 2024-09-19T09:28:16.367 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.367 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.367 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.367 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Email From: name", 2024-09-19T09:28:16.367 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.367 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.367 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.367 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_host": { 2024-09-19T09:28:16.368 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_host", 2024-09-19T09:28:16.368 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.368 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.368 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.368 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.368 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.368 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.368 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.368 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "SMTP server", 2024-09-19T09:28:16.368 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.368 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.368 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.368 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_password": { 2024-09-19T09:28:16.369 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_password", 2024-09-19T09:28:16.369 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.369 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.369 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.369 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.369 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.369 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.369 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.369 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Password to authenticate with", 2024-09-19T09:28:16.369 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.369 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.369 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.369 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_port": { 2024-09-19T09:28:16.370 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_port", 2024-09-19T09:28:16.370 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.370 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.370 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.370 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "465", 2024-09-19T09:28:16.370 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.370 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.370 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.370 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "SMTP port", 2024-09-19T09:28:16.370 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.370 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.370 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.370 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_sender": { 2024-09-19T09:28:16.371 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_sender", 2024-09-19T09:28:16.371 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.371 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.371 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.371 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.371 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.371 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.371 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.371 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "SMTP envelope sender", 2024-09-19T09:28:16.371 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.371 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.371 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.372 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_ssl": { 2024-09-19T09:28:16.372 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_ssl", 2024-09-19T09:28:16.372 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.372 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.372 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.372 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.372 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.372 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.372 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.372 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T09:28:16.372 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.372 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.372 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.373 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_user": { 2024-09-19T09:28:16.373 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_user", 2024-09-19T09:28:16.373 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.373 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.373 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.373 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.373 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.373 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.373 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.373 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "User to authenticate as", 2024-09-19T09:28:16.373 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.373 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.374 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.374 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.374 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.374 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "balancer", 2024-09-19T09:28:16.374 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.374 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.374 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.374 INFO:teuthology.orchestra.run.smithi120.stdout: "active": { 2024-09-19T09:28:16.374 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "active", 2024-09-19T09:28:16.374 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.374 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.374 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.375 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.375 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.375 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.375 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.375 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T09:28:16.375 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.375 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.375 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.375 INFO:teuthology.orchestra.run.smithi120.stdout: "begin_time": { 2024-09-19T09:28:16.375 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "begin_time", 2024-09-19T09:28:16.375 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.375 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.375 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.376 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0000", 2024-09-19T09:28:16.376 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.376 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.376 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.376 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T09:28:16.376 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T09:28:16.376 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.376 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.376 INFO:teuthology.orchestra.run.smithi120.stdout: "begin_weekday": { 2024-09-19T09:28:16.376 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "begin_weekday", 2024-09-19T09:28:16.376 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-19T09:28:16.376 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.376 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.377 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-19T09:28:16.377 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "0", 2024-09-19T09:28:16.377 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "6", 2024-09-19T09:28:16.377 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.377 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T09:28:16.377 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T09:28:16.377 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.377 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.377 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_compat_max_iterations": { 2024-09-19T09:28:16.377 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crush_compat_max_iterations", 2024-09-19T09:28:16.377 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-19T09:28:16.377 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.377 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.378 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "25", 2024-09-19T09:28:16.378 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1", 2024-09-19T09:28:16.378 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "250", 2024-09-19T09:28:16.378 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.378 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T09:28:16.378 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.378 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.378 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.378 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_compat_metrics": { 2024-09-19T09:28:16.378 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crush_compat_metrics", 2024-09-19T09:28:16.378 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.378 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.378 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.379 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T09:28:16.379 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.379 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.379 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.379 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T09:28:16.379 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T09:28:16.379 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.379 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.379 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_compat_step": { 2024-09-19T09:28:16.379 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crush_compat_step", 2024-09-19T09:28:16.379 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-19T09:28:16.379 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.379 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.380 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0.5", 2024-09-19T09:28:16.380 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "0.001", 2024-09-19T09:28:16.380 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "0.999", 2024-09-19T09:28:16.380 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.380 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "aggressiveness of optimization", 2024-09-19T09:28:16.380 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T09:28:16.380 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.380 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.380 INFO:teuthology.orchestra.run.smithi120.stdout: "end_time": { 2024-09-19T09:28:16.380 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "end_time", 2024-09-19T09:28:16.380 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.380 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.380 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.381 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "2359", 2024-09-19T09:28:16.381 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.381 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.381 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.381 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T09:28:16.381 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T09:28:16.381 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.381 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.381 INFO:teuthology.orchestra.run.smithi120.stdout: "end_weekday": { 2024-09-19T09:28:16.381 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "end_weekday", 2024-09-19T09:28:16.381 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-19T09:28:16.381 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.381 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.382 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-19T09:28:16.382 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "0", 2024-09-19T09:28:16.382 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "6", 2024-09-19T09:28:16.382 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.382 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T09:28:16.382 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T09:28:16.382 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.382 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.382 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.382 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.382 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.382 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.382 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.382 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.383 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.383 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.383 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.383 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.383 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.383 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.383 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.383 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.383 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.383 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.383 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.383 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.383 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.383 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.384 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.384 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.384 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.384 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.384 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.384 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.384 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.384 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.384 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.384 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.384 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.385 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.385 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.385 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.385 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.385 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.385 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.385 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.385 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.385 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.385 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.385 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.385 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.385 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.386 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.386 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.386 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.386 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.386 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.386 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.386 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.386 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.386 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.386 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.386 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.386 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.386 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.387 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.387 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.387 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.387 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.387 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.387 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.387 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.387 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.387 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.387 INFO:teuthology.orchestra.run.smithi120.stdout: "min_score": { 2024-09-19T09:28:16.387 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "min_score", 2024-09-19T09:28:16.387 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-19T09:28:16.387 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.387 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.388 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-19T09:28:16.388 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.388 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.388 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.388 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T09:28:16.388 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.388 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.388 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.388 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": { 2024-09-19T09:28:16.388 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mode", 2024-09-19T09:28:16.388 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.388 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.389 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.389 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "upmap", 2024-09-19T09:28:16.389 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.389 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.389 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.389 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-compat", 2024-09-19T09:28:16.389 INFO:teuthology.orchestra.run.smithi120.stdout: "none", 2024-09-19T09:28:16.389 INFO:teuthology.orchestra.run.smithi120.stdout: "read", 2024-09-19T09:28:16.389 INFO:teuthology.orchestra.run.smithi120.stdout: "upmap", 2024-09-19T09:28:16.389 INFO:teuthology.orchestra.run.smithi120.stdout: "upmap-read" 2024-09-19T09:28:16.389 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.389 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Balancer mode", 2024-09-19T09:28:16.389 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.389 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.390 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.390 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_ids": { 2024-09-19T09:28:16.390 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "pool_ids", 2024-09-19T09:28:16.390 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.390 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.390 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.390 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.390 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.390 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.390 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.390 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T09:28:16.390 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.390 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.391 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.391 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-19T09:28:16.391 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-19T09:28:16.391 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.391 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.391 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.391 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-19T09:28:16.391 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.391 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.391 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.391 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T09:28:16.391 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.391 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.391 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.392 INFO:teuthology.orchestra.run.smithi120.stdout: "upmap_max_deviation": { 2024-09-19T09:28:16.392 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "upmap_max_deviation", 2024-09-19T09:28:16.392 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.392 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.392 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.392 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-19T09:28:16.392 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1", 2024-09-19T09:28:16.392 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.392 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.392 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T09:28:16.392 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T09:28:16.392 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.392 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.393 INFO:teuthology.orchestra.run.smithi120.stdout: "upmap_max_optimizations": { 2024-09-19T09:28:16.393 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "upmap_max_optimizations", 2024-09-19T09:28:16.393 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-19T09:28:16.393 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.393 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.393 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-19T09:28:16.393 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.393 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.393 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.393 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T09:28:16.393 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.393 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.393 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.394 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.394 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.394 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cephadm", 2024-09-19T09:28:16.394 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.394 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.394 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.394 INFO:teuthology.orchestra.run.smithi120.stdout: "agent_down_multiplier": { 2024-09-19T09:28:16.394 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "agent_down_multiplier", 2024-09-19T09:28:16.394 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-19T09:28:16.394 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.394 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.394 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "3.0", 2024-09-19T09:28:16.395 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.395 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.395 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.395 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T09:28:16.395 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.395 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.395 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.395 INFO:teuthology.orchestra.run.smithi120.stdout: "agent_refresh_rate": { 2024-09-19T09:28:16.395 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "agent_refresh_rate", 2024-09-19T09:28:16.395 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.395 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.395 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.396 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "20", 2024-09-19T09:28:16.396 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.396 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.396 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.396 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T09:28:16.396 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.396 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.396 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.396 INFO:teuthology.orchestra.run.smithi120.stdout: "agent_starting_port": { 2024-09-19T09:28:16.396 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "agent_starting_port", 2024-09-19T09:28:16.396 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.396 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.396 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.397 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "4721", 2024-09-19T09:28:16.397 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.397 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.397 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.397 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T09:28:16.397 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.397 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.397 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.397 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_ptrace": { 2024-09-19T09:28:16.397 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "allow_ptrace", 2024-09-19T09:28:16.397 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.398 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.398 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.398 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.398 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.398 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.398 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.398 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T09:28:16.398 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T09:28:16.398 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.398 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.398 INFO:teuthology.orchestra.run.smithi120.stdout: "autotune_interval": { 2024-09-19T09:28:16.398 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "autotune_interval", 2024-09-19T09:28:16.398 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.399 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.399 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.399 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-19T09:28:16.399 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.399 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.399 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.399 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T09:28:16.399 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.399 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.399 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.399 INFO:teuthology.orchestra.run.smithi120.stdout: "autotune_memory_target_ratio": { 2024-09-19T09:28:16.399 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T09:28:16.399 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-19T09:28:16.400 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.400 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.400 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0.7", 2024-09-19T09:28:16.400 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.400 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.400 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.400 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T09:28:16.400 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.400 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.400 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.400 INFO:teuthology.orchestra.run.smithi120.stdout: "cephadm_log_destination": { 2024-09-19T09:28:16.400 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cephadm_log_destination", 2024-09-19T09:28:16.401 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.401 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.401 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.401 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.401 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.401 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.401 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.401 INFO:teuthology.orchestra.run.smithi120.stdout: "file", 2024-09-19T09:28:16.401 INFO:teuthology.orchestra.run.smithi120.stdout: "file,syslog", 2024-09-19T09:28:16.401 INFO:teuthology.orchestra.run.smithi120.stdout: "syslog" 2024-09-19T09:28:16.402 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.402 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T09:28:16.402 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.402 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.402 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.402 INFO:teuthology.orchestra.run.smithi120.stdout: "cgroups_split": { 2024-09-19T09:28:16.402 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cgroups_split", 2024-09-19T09:28:16.402 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.402 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.402 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.402 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.402 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.403 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.403 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.403 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T09:28:16.403 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.403 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.403 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.403 INFO:teuthology.orchestra.run.smithi120.stdout: "config_checks_enabled": { 2024-09-19T09:28:16.403 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "config_checks_enabled", 2024-09-19T09:28:16.403 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.403 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.403 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.403 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.403 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.403 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.404 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.404 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T09:28:16.404 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.404 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.404 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.404 INFO:teuthology.orchestra.run.smithi120.stdout: "config_dashboard": { 2024-09-19T09:28:16.404 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "config_dashboard", 2024-09-19T09:28:16.404 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.404 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.404 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.404 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.404 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.405 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.405 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.405 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T09:28:16.405 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.405 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.405 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.405 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_alertmanager": { 2024-09-19T09:28:16.405 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_alertmanager", 2024-09-19T09:28:16.405 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.405 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.405 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.405 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T09:28:16.405 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.406 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.406 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.406 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Prometheus container image", 2024-09-19T09:28:16.406 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.406 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.406 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.406 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_base": { 2024-09-19T09:28:16.406 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_base", 2024-09-19T09:28:16.406 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.406 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.406 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.406 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T09:28:16.406 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.406 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.407 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.407 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Container image name, without the tag", 2024-09-19T09:28:16.407 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.407 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.407 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.407 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_elasticsearch": { 2024-09-19T09:28:16.407 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_elasticsearch", 2024-09-19T09:28:16.407 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.407 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.407 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.407 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T09:28:16.407 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.407 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.408 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.408 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "elasticsearch container image", 2024-09-19T09:28:16.408 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.408 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.408 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.408 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_grafana": { 2024-09-19T09:28:16.408 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_grafana", 2024-09-19T09:28:16.408 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.408 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.408 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.408 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T09:28:16.408 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.408 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.409 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.409 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Prometheus container image", 2024-09-19T09:28:16.409 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.409 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.409 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.409 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_haproxy": { 2024-09-19T09:28:16.409 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_haproxy", 2024-09-19T09:28:16.409 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.409 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.409 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.409 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T09:28:16.409 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.410 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.410 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.410 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "HAproxy container image", 2024-09-19T09:28:16.410 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.410 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.410 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.410 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_jaeger_agent": { 2024-09-19T09:28:16.410 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_jaeger_agent", 2024-09-19T09:28:16.410 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.410 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.410 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.410 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T09:28:16.410 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.411 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.411 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.411 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Jaeger agent container image", 2024-09-19T09:28:16.411 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.411 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.411 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.411 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_jaeger_collector": { 2024-09-19T09:28:16.411 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_jaeger_collector", 2024-09-19T09:28:16.411 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.411 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.411 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.411 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T09:28:16.411 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.412 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.412 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.412 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Jaeger collector container image", 2024-09-19T09:28:16.412 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.412 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.412 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.412 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_jaeger_query": { 2024-09-19T09:28:16.412 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_jaeger_query", 2024-09-19T09:28:16.412 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.412 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.412 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.412 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T09:28:16.412 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.413 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.413 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.413 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Jaeger query container image", 2024-09-19T09:28:16.413 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.413 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.413 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.413 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_keepalived": { 2024-09-19T09:28:16.413 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_keepalived", 2024-09-19T09:28:16.413 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.413 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.413 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.413 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T09:28:16.413 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.414 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.414 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.414 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Keepalived container image", 2024-09-19T09:28:16.414 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.414 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.414 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.414 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_loki": { 2024-09-19T09:28:16.414 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_loki", 2024-09-19T09:28:16.414 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.414 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.414 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.414 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T09:28:16.414 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.415 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.415 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.415 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Loki container image", 2024-09-19T09:28:16.415 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.415 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.415 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.415 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_node_exporter": { 2024-09-19T09:28:16.415 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_node_exporter", 2024-09-19T09:28:16.415 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.415 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.415 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.415 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T09:28:16.415 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.416 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.416 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.416 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Prometheus container image", 2024-09-19T09:28:16.416 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.416 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.416 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.416 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_nvmeof": { 2024-09-19T09:28:16.416 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_nvmeof", 2024-09-19T09:28:16.416 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.416 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.416 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.416 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T09:28:16.416 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.417 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.417 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.417 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Nvme-of container image", 2024-09-19T09:28:16.417 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.417 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.417 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.417 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_prometheus": { 2024-09-19T09:28:16.417 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_prometheus", 2024-09-19T09:28:16.417 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.417 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.417 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.417 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T09:28:16.417 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.418 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.418 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.418 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Prometheus container image", 2024-09-19T09:28:16.418 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.418 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.418 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.418 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_promtail": { 2024-09-19T09:28:16.418 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_promtail", 2024-09-19T09:28:16.418 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.418 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.418 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.418 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T09:28:16.418 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.418 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.419 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.419 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Promtail container image", 2024-09-19T09:28:16.419 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.419 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.419 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.419 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_samba": { 2024-09-19T09:28:16.419 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_samba", 2024-09-19T09:28:16.419 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.419 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.419 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.419 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T09:28:16.419 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.419 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.420 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.420 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Samba/SMB container image", 2024-09-19T09:28:16.420 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.420 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.420 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.420 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_snmp_gateway": { 2024-09-19T09:28:16.420 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_snmp_gateway", 2024-09-19T09:28:16.420 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.420 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.420 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.420 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T09:28:16.420 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.420 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.421 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.421 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "SNMP Gateway container image", 2024-09-19T09:28:16.421 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.421 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.421 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.421 INFO:teuthology.orchestra.run.smithi120.stdout: "container_init": { 2024-09-19T09:28:16.421 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_init", 2024-09-19T09:28:16.421 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.421 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.421 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.421 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.421 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.422 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.422 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.422 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T09:28:16.422 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.422 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.422 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.422 INFO:teuthology.orchestra.run.smithi120.stdout: "daemon_cache_timeout": { 2024-09-19T09:28:16.422 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "daemon_cache_timeout", 2024-09-19T09:28:16.422 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.422 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.422 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.422 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-19T09:28:16.422 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.423 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.423 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.423 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T09:28:16.423 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.423 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.423 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.423 INFO:teuthology.orchestra.run.smithi120.stdout: "default_cephadm_command_timeout": { 2024-09-19T09:28:16.423 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T09:28:16.423 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.423 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.423 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.423 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "900", 2024-09-19T09:28:16.423 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.424 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.424 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.424 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T09:28:16.424 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.424 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.424 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.424 INFO:teuthology.orchestra.run.smithi120.stdout: "default_registry": { 2024-09-19T09:28:16.424 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "default_registry", 2024-09-19T09:28:16.424 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.424 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.424 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.424 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "docker.io", 2024-09-19T09:28:16.425 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.425 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.425 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.425 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T09:28:16.425 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.425 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.425 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.425 INFO:teuthology.orchestra.run.smithi120.stdout: "device_cache_timeout": { 2024-09-19T09:28:16.425 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "device_cache_timeout", 2024-09-19T09:28:16.425 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.425 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.425 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.426 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "1800", 2024-09-19T09:28:16.426 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.426 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.426 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.426 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "seconds to cache device inventory", 2024-09-19T09:28:16.426 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.426 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.426 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.426 INFO:teuthology.orchestra.run.smithi120.stdout: "device_enhanced_scan": { 2024-09-19T09:28:16.426 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "device_enhanced_scan", 2024-09-19T09:28:16.426 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.426 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.426 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.427 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.427 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.427 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.427 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.427 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T09:28:16.427 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.427 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.427 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.427 INFO:teuthology.orchestra.run.smithi120.stdout: "facts_cache_timeout": { 2024-09-19T09:28:16.427 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "facts_cache_timeout", 2024-09-19T09:28:16.427 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.428 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.428 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.428 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-19T09:28:16.428 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.428 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.428 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.428 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "seconds to cache host facts data", 2024-09-19T09:28:16.428 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.428 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.428 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.428 INFO:teuthology.orchestra.run.smithi120.stdout: "grafana_dashboards_path": { 2024-09-19T09:28:16.428 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "grafana_dashboards_path", 2024-09-19T09:28:16.429 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.429 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.429 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.429 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T09:28:16.429 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.429 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.429 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.429 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T09:28:16.429 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.429 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.429 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.429 INFO:teuthology.orchestra.run.smithi120.stdout: "host_check_interval": { 2024-09-19T09:28:16.429 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "host_check_interval", 2024-09-19T09:28:16.430 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.430 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.430 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.430 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-19T09:28:16.430 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.430 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.430 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.430 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to perform a host check", 2024-09-19T09:28:16.430 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.430 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.430 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.430 INFO:teuthology.orchestra.run.smithi120.stdout: "hw_monitoring": { 2024-09-19T09:28:16.430 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "hw_monitoring", 2024-09-19T09:28:16.430 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.431 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.431 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.431 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.431 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.431 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.431 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.431 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T09:28:16.431 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.431 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.431 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.431 INFO:teuthology.orchestra.run.smithi120.stdout: "inventory_list_all": { 2024-09-19T09:28:16.431 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "inventory_list_all", 2024-09-19T09:28:16.431 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.432 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.432 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.432 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.432 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.432 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.432 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.432 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T09:28:16.432 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.432 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.432 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.432 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.432 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.432 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.433 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.433 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.433 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.433 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.433 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.433 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.433 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.433 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.433 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.433 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.433 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.433 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.433 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.433 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.433 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.434 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.434 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.434 INFO:teuthology.orchestra.run.smithi120.stdout: "log_refresh_metadata": { 2024-09-19T09:28:16.434 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_refresh_metadata", 2024-09-19T09:28:16.434 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.434 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.434 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.434 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.434 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.434 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.434 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.435 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T09:28:16.435 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.435 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.435 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.435 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.435 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.435 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.435 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.435 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.435 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.435 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.435 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.436 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.436 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T09:28:16.436 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.436 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.436 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.436 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.436 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.437 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.437 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.437 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.437 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.437 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.437 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.438 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.438 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.438 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.438 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.438 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.438 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.438 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.438 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.438 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.438 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.438 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.438 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.439 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.439 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.439 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.439 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.439 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.439 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.439 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.439 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.439 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.439 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.439 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.439 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.439 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.440 INFO:teuthology.orchestra.run.smithi120.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T09:28:16.440 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T09:28:16.440 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.440 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.440 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.440 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.440 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.440 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.440 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.440 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T09:28:16.441 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.441 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.441 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.441 INFO:teuthology.orchestra.run.smithi120.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T09:28:16.441 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T09:28:16.441 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.441 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.441 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.441 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "*", 2024-09-19T09:28:16.441 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.441 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.441 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.441 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T09:28:16.442 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.442 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.442 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.442 INFO:teuthology.orchestra.run.smithi120.stdout: "max_count_per_host": { 2024-09-19T09:28:16.442 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_count_per_host", 2024-09-19T09:28:16.442 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.442 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.442 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.442 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-19T09:28:16.442 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.442 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.442 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.442 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "max number of daemons per service per host", 2024-09-19T09:28:16.443 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.443 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.443 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.443 INFO:teuthology.orchestra.run.smithi120.stdout: "max_osd_draining_count": { 2024-09-19T09:28:16.443 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_osd_draining_count", 2024-09-19T09:28:16.443 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.443 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.443 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.443 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-19T09:28:16.443 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.443 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.443 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.444 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T09:28:16.444 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.444 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.444 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.444 INFO:teuthology.orchestra.run.smithi120.stdout: "migration_current": { 2024-09-19T09:28:16.444 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "migration_current", 2024-09-19T09:28:16.444 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.444 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.444 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.444 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.444 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.444 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.445 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.445 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "internal - do not modify", 2024-09-19T09:28:16.445 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.445 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.445 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.445 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": { 2024-09-19T09:28:16.445 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mode", 2024-09-19T09:28:16.445 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.445 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.445 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.445 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "root", 2024-09-19T09:28:16.445 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.445 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.446 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.446 INFO:teuthology.orchestra.run.smithi120.stdout: "cephadm-package", 2024-09-19T09:28:16.446 INFO:teuthology.orchestra.run.smithi120.stdout: "root" 2024-09-19T09:28:16.446 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.446 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T09:28:16.446 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.446 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.446 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.446 INFO:teuthology.orchestra.run.smithi120.stdout: "oob_default_addr": { 2024-09-19T09:28:16.446 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "oob_default_addr", 2024-09-19T09:28:16.446 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.446 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.447 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.447 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "169.254.1.1", 2024-09-19T09:28:16.447 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.447 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.447 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.447 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T09:28:16.447 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.447 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.447 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.447 INFO:teuthology.orchestra.run.smithi120.stdout: "prometheus_alerts_path": { 2024-09-19T09:28:16.447 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "prometheus_alerts_path", 2024-09-19T09:28:16.447 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.447 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.448 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.448 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T09:28:16.448 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.448 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.448 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.448 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T09:28:16.448 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.448 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.448 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.448 INFO:teuthology.orchestra.run.smithi120.stdout: "registry_insecure": { 2024-09-19T09:28:16.448 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "registry_insecure", 2024-09-19T09:28:16.449 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.449 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.449 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.449 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.449 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.449 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.449 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.449 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T09:28:16.449 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.449 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.449 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.449 INFO:teuthology.orchestra.run.smithi120.stdout: "registry_password": { 2024-09-19T09:28:16.450 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "registry_password", 2024-09-19T09:28:16.450 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.450 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.450 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.450 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.450 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.450 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.450 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.450 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T09:28:16.450 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.450 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.450 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.451 INFO:teuthology.orchestra.run.smithi120.stdout: "registry_url": { 2024-09-19T09:28:16.451 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "registry_url", 2024-09-19T09:28:16.451 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.451 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.451 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.451 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.451 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.451 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.451 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.451 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T09:28:16.451 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.451 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.451 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.452 INFO:teuthology.orchestra.run.smithi120.stdout: "registry_username": { 2024-09-19T09:28:16.452 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "registry_username", 2024-09-19T09:28:16.452 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.452 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.452 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.452 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.452 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.452 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.452 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.452 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T09:28:16.452 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.452 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.452 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.453 INFO:teuthology.orchestra.run.smithi120.stdout: "secure_monitoring_stack": { 2024-09-19T09:28:16.453 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "secure_monitoring_stack", 2024-09-19T09:28:16.453 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.453 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.453 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.453 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.453 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.453 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.453 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.453 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T09:28:16.453 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.453 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.453 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.454 INFO:teuthology.orchestra.run.smithi120.stdout: "service_discovery_port": { 2024-09-19T09:28:16.454 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "service_discovery_port", 2024-09-19T09:28:16.454 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.454 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.454 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.454 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8765", 2024-09-19T09:28:16.454 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.454 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.454 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.454 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "cephadm service discovery port", 2024-09-19T09:28:16.454 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.454 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.455 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.455 INFO:teuthology.orchestra.run.smithi120.stdout: "ssh_config_file": { 2024-09-19T09:28:16.455 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ssh_config_file", 2024-09-19T09:28:16.455 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.455 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.455 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.455 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.455 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.455 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.455 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.455 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T09:28:16.455 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.456 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.456 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.456 INFO:teuthology.orchestra.run.smithi120.stdout: "use_agent": { 2024-09-19T09:28:16.456 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "use_agent", 2024-09-19T09:28:16.456 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.456 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.456 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.456 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.456 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.456 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.456 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.456 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T09:28:16.456 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.457 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.457 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.457 INFO:teuthology.orchestra.run.smithi120.stdout: "use_repo_digest": { 2024-09-19T09:28:16.457 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "use_repo_digest", 2024-09-19T09:28:16.457 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.457 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.457 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.457 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.457 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.457 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.457 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.457 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T09:28:16.458 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.458 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.458 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.458 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_on_failed_host_check": { 2024-09-19T09:28:16.458 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_on_failed_host_check", 2024-09-19T09:28:16.458 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.458 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.458 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.458 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.458 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.458 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.458 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.458 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T09:28:16.459 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.459 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.459 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.459 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_on_stray_daemons": { 2024-09-19T09:28:16.459 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_on_stray_daemons", 2024-09-19T09:28:16.459 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.459 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.459 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.459 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.459 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.459 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.459 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.459 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T09:28:16.460 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.460 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.460 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.460 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_on_stray_hosts": { 2024-09-19T09:28:16.460 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_on_stray_hosts", 2024-09-19T09:28:16.460 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.460 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.460 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.460 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.460 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.460 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.460 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.461 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T09:28:16.461 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.461 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.461 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.461 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.461 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.461 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crash", 2024-09-19T09:28:16.461 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.461 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.461 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.462 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.462 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.462 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.462 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.462 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.462 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.462 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.462 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.462 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.462 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.462 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.462 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.462 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.462 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.465 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.465 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.465 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.465 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.465 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.465 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.465 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.465 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.465 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.465 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.465 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.465 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.465 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.466 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.466 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.466 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.466 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.466 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.466 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.466 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.466 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.466 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.466 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.466 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.466 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.466 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.467 INFO:teuthology.orchestra.run.smithi120.stdout: "retain_interval": { 2024-09-19T09:28:16.467 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "retain_interval", 2024-09-19T09:28:16.467 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.467 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.467 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.467 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "31536000", 2024-09-19T09:28:16.467 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.467 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.467 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.467 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T09:28:16.467 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.467 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.467 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.468 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_recent_interval": { 2024-09-19T09:28:16.468 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_recent_interval", 2024-09-19T09:28:16.468 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.468 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.468 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.468 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "1209600", 2024-09-19T09:28:16.468 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.468 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.468 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.468 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T09:28:16.468 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.468 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.468 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.469 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.469 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.469 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "dashboard", 2024-09-19T09:28:16.469 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.469 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.469 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.469 INFO:teuthology.orchestra.run.smithi120.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T09:28:16.469 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T09:28:16.469 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.469 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.469 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.470 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-19T09:28:16.470 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.470 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.470 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.470 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.470 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.470 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.470 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.470 INFO:teuthology.orchestra.run.smithi120.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T09:28:16.470 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T09:28:16.470 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.470 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.470 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.471 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.471 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.471 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.471 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.471 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.471 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.471 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.471 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.471 INFO:teuthology.orchestra.run.smithi120.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T09:28:16.471 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T09:28:16.471 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.471 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.471 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.472 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.472 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.472 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.472 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.472 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.472 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.472 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.472 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.472 INFO:teuthology.orchestra.run.smithi120.stdout: "AUDIT_API_ENABLED": { 2024-09-19T09:28:16.472 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T09:28:16.472 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.472 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.472 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.473 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.473 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.473 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.473 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.473 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.473 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.473 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.473 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.473 INFO:teuthology.orchestra.run.smithi120.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T09:28:16.473 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T09:28:16.473 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.473 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.474 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.474 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.474 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.474 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.474 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.474 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.474 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.474 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.474 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.474 INFO:teuthology.orchestra.run.smithi120.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T09:28:16.474 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T09:28:16.474 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.474 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.475 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.475 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.475 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.475 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.475 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.475 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.475 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.475 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.475 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.475 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T09:28:16.475 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T09:28:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T09:28:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T09:28:16.477 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.477 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.477 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.477 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.477 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.477 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.477 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.477 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.477 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.477 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.477 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.477 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T09:28:16.478 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T09:28:16.478 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.478 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.478 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.478 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.478 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.478 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.478 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.478 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.478 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.478 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.478 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.478 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T09:28:16.479 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T09:28:16.479 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.479 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.479 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.479 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.479 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.479 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.479 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.479 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.479 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.479 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.479 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.479 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T09:28:16.480 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T09:28:16.480 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.480 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.480 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.480 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.480 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.480 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.480 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.480 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.480 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.480 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.480 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.480 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T09:28:16.481 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T09:28:16.481 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.481 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.481 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.481 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.481 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.481 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.481 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.481 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.481 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.481 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.481 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.481 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T09:28:16.482 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T09:28:16.482 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.482 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.482 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.482 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.482 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.482 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.482 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.482 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.482 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.482 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.482 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T09:28:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T09:28:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T09:28:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T09:28:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "admin", 2024-09-19T09:28:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T09:28:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T09:28:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.486 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_API_URL": { 2024-09-19T09:28:16.486 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_API_URL", 2024-09-19T09:28:16.486 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.486 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.486 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.486 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.486 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.486 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.486 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.486 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.486 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.486 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.486 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.487 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T09:28:16.487 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T09:28:16.487 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.487 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.487 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.487 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "admin", 2024-09-19T09:28:16.487 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.487 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.487 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.487 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.487 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.487 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.487 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T09:28:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T09:28:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T09:28:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T09:28:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T09:28:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T09:28:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T09:28:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T09:28:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T09:28:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T09:28:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T09:28:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T09:28:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T09:28:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T09:28:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T09:28:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T09:28:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T09:28:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T09:28:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T09:28:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T09:28:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T09:28:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T09:28:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T09:28:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T09:28:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T09:28:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T09:28:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T09:28:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T09:28:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T09:28:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T09:28:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T09:28:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T09:28:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T09:28:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-19T09:28:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T09:28:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T09:28:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8", 2024-09-19T09:28:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T09:28:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T09:28:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "45", 2024-09-19T09:28:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T09:28:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T09:28:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T09:28:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T09:28:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "admin", 2024-09-19T09:28:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T09:28:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T09:28:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T09:28:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T09:28:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T09:28:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T09:28:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-19T09:28:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T09:28:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T09:28:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-19T09:28:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T09:28:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T09:28:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-19T09:28:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "cross_origin_url": { 2024-09-19T09:28:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cross_origin_url", 2024-09-19T09:28:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: "crt_file": { 2024-09-19T09:28:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crt_file", 2024-09-19T09:28:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: "debug": { 2024-09-19T09:28:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "debug", 2024-09-19T09:28:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Enable/disable debug options", 2024-09-19T09:28:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: "jwt_token_ttl": { 2024-09-19T09:28:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "jwt_token_ttl", 2024-09-19T09:28:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "28800", 2024-09-19T09:28:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "key_file": { 2024-09-19T09:28:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "key_file", 2024-09-19T09:28:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "motd": { 2024-09-19T09:28:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "motd", 2024-09-19T09:28:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "The message of the day", 2024-09-19T09:28:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "redirect_resolve_ip_addr": { 2024-09-19T09:28:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T09:28:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "server_addr": { 2024-09-19T09:28:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_addr", 2024-09-19T09:28:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "::", 2024-09-19T09:28:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "server_port": { 2024-09-19T09:28:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_port", 2024-09-19T09:28:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8080", 2024-09-19T09:28:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: "ssl": { 2024-09-19T09:28:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ssl", 2024-09-19T09:28:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "ssl_server_port": { 2024-09-19T09:28:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ssl_server_port", 2024-09-19T09:28:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8443", 2024-09-19T09:28:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: "standby_behaviour": { 2024-09-19T09:28:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "standby_behaviour", 2024-09-19T09:28:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "redirect", 2024-09-19T09:28:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: "redirect" 2024-09-19T09:28:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "standby_error_status_code": { 2024-09-19T09:28:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "standby_error_status_code", 2024-09-19T09:28:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "500", 2024-09-19T09:28:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "400", 2024-09-19T09:28:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "599", 2024-09-19T09:28:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: "url_prefix": { 2024-09-19T09:28:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "url_prefix", 2024-09-19T09:28:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "devicehealth", 2024-09-19T09:28:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "enable_monitoring": { 2024-09-19T09:28:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "enable_monitoring", 2024-09-19T09:28:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "monitor device health metrics", 2024-09-19T09:28:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: "mark_out_threshold": { 2024-09-19T09:28:16.543 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mark_out_threshold", 2024-09-19T09:28:16.543 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.544 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.544 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.544 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "2419200", 2024-09-19T09:28:16.544 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.545 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.545 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.545 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T09:28:16.545 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.545 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.545 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.545 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": { 2024-09-19T09:28:16.545 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "pool_name", 2024-09-19T09:28:16.545 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.545 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.545 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.545 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "device_health_metrics", 2024-09-19T09:28:16.546 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.546 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.546 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.546 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T09:28:16.546 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.546 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.546 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.546 INFO:teuthology.orchestra.run.smithi120.stdout: "retention_period": { 2024-09-19T09:28:16.546 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "retention_period", 2024-09-19T09:28:16.546 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.546 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.546 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.546 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "15552000", 2024-09-19T09:28:16.547 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.547 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.547 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.547 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how long to retain device health metrics", 2024-09-19T09:28:16.547 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.547 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.547 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.547 INFO:teuthology.orchestra.run.smithi120.stdout: "scrape_frequency": { 2024-09-19T09:28:16.547 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "scrape_frequency", 2024-09-19T09:28:16.547 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.547 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.547 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.547 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "86400", 2024-09-19T09:28:16.547 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.548 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.548 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.548 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T09:28:16.548 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.548 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.548 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.548 INFO:teuthology.orchestra.run.smithi120.stdout: "self_heal": { 2024-09-19T09:28:16.548 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "self_heal", 2024-09-19T09:28:16.548 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.548 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.549 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.549 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.549 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.549 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.549 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.549 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T09:28:16.549 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.549 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.549 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.549 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-19T09:28:16.549 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-19T09:28:16.549 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.549 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.550 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.550 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-19T09:28:16.550 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.550 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.550 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.550 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T09:28:16.550 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.550 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.550 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.550 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_threshold": { 2024-09-19T09:28:16.550 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_threshold", 2024-09-19T09:28:16.550 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.550 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.551 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.551 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "7257600", 2024-09-19T09:28:16.551 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.551 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.551 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.551 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T09:28:16.551 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.551 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.551 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.551 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.551 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.551 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "diskprediction_local", 2024-09-19T09:28:16.552 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.552 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.552 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.552 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.552 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.552 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.552 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.552 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.552 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.552 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.552 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.552 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.552 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.553 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.553 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.553 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.553 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.553 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.553 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.553 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.553 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.553 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.553 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.553 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.553 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.553 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.554 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.554 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.554 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.554 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.554 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.554 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.554 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.554 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.554 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.554 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.554 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.554 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.554 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.555 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.555 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.555 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.555 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.555 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.555 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.555 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.555 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.555 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.555 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.555 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.555 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.555 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.555 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.556 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.556 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.556 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.556 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.556 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.556 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.556 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.556 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.556 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.556 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.556 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.556 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.556 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.557 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.557 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.557 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.557 INFO:teuthology.orchestra.run.smithi120.stdout: "predict_interval": { 2024-09-19T09:28:16.557 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "predict_interval", 2024-09-19T09:28:16.557 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.557 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.557 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.557 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "86400", 2024-09-19T09:28:16.557 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.557 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.557 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.557 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.558 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.558 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.558 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.558 INFO:teuthology.orchestra.run.smithi120.stdout: "predictor_model": { 2024-09-19T09:28:16.558 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "predictor_model", 2024-09-19T09:28:16.558 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.558 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.558 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.558 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "prophetstor", 2024-09-19T09:28:16.558 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.558 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.558 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.559 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.559 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.559 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.559 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.559 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-19T09:28:16.559 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-19T09:28:16.559 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.559 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.559 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.559 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-19T09:28:16.559 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.559 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.559 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.560 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.560 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.560 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.560 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.560 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.560 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.560 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "influx", 2024-09-19T09:28:16.560 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": false, 2024-09-19T09:28:16.560 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "influxdb python module not found", 2024-09-19T09:28:16.560 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.560 INFO:teuthology.orchestra.run.smithi120.stdout: "batch_size": { 2024-09-19T09:28:16.560 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "batch_size", 2024-09-19T09:28:16.561 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.561 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.561 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.561 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5000", 2024-09-19T09:28:16.561 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.561 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.561 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.561 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T09:28:16.561 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.561 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.561 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.561 INFO:teuthology.orchestra.run.smithi120.stdout: "database": { 2024-09-19T09:28:16.562 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "database", 2024-09-19T09:28:16.562 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.562 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.562 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.562 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "ceph", 2024-09-19T09:28:16.562 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.562 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.562 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.562 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T09:28:16.562 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.562 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.562 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.563 INFO:teuthology.orchestra.run.smithi120.stdout: "hostname": { 2024-09-19T09:28:16.563 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "hostname", 2024-09-19T09:28:16.563 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.563 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.563 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.563 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.563 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.563 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.563 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.563 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "InfluxDB server hostname", 2024-09-19T09:28:16.563 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.563 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.563 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.564 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-19T09:28:16.564 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-19T09:28:16.564 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.564 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.564 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.564 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "30", 2024-09-19T09:28:16.564 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "5", 2024-09-19T09:28:16.564 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.564 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.564 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T09:28:16.564 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.564 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.564 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.565 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.565 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.565 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.565 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.565 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.565 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.565 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.565 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.565 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.565 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.565 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.565 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.565 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.565 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.566 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.567 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.568 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.568 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.568 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.568 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.568 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.568 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.568 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.568 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.568 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.568 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.568 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.568 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.568 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.569 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.569 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.569 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.569 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.569 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.569 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.569 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.569 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.569 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.569 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.569 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.569 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.569 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.570 INFO:teuthology.orchestra.run.smithi120.stdout: "password": { 2024-09-19T09:28:16.570 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "password", 2024-09-19T09:28:16.570 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.570 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.570 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.570 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.570 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.570 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.570 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.570 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "password of InfluxDB server user", 2024-09-19T09:28:16.570 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.570 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.570 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.571 INFO:teuthology.orchestra.run.smithi120.stdout: "port": { 2024-09-19T09:28:16.571 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "port", 2024-09-19T09:28:16.571 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.571 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.571 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.571 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8086", 2024-09-19T09:28:16.571 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.571 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.571 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.571 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "InfluxDB server port", 2024-09-19T09:28:16.571 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.571 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.571 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.572 INFO:teuthology.orchestra.run.smithi120.stdout: "ssl": { 2024-09-19T09:28:16.572 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ssl", 2024-09-19T09:28:16.572 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.572 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.572 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.572 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "false", 2024-09-19T09:28:16.572 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.572 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.572 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.572 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T09:28:16.572 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.572 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.572 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.573 INFO:teuthology.orchestra.run.smithi120.stdout: "threads": { 2024-09-19T09:28:16.573 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "threads", 2024-09-19T09:28:16.573 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.573 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.573 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.573 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-19T09:28:16.573 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1", 2024-09-19T09:28:16.573 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "32", 2024-09-19T09:28:16.573 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.573 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T09:28:16.573 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.573 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.573 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.574 INFO:teuthology.orchestra.run.smithi120.stdout: "username": { 2024-09-19T09:28:16.574 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "username", 2024-09-19T09:28:16.574 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.574 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.574 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.574 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.574 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.574 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.574 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.574 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "username of InfluxDB server user", 2024-09-19T09:28:16.574 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.574 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.574 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.575 INFO:teuthology.orchestra.run.smithi120.stdout: "verify_ssl": { 2024-09-19T09:28:16.575 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "verify_ssl", 2024-09-19T09:28:16.575 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.575 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.575 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.575 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "true", 2024-09-19T09:28:16.575 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.575 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.575 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.575 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T09:28:16.575 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.575 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.576 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.576 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.576 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.576 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "insights", 2024-09-19T09:28:16.576 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.576 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.576 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.576 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.576 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.576 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.576 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.576 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.577 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.577 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.577 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.577 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.577 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.577 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.577 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.577 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.577 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.577 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.577 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.577 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.577 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.578 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.578 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.578 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.578 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.578 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.578 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.578 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.578 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.578 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.578 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.578 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.578 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.578 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.579 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.579 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.579 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.579 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.579 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.579 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.579 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.579 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.579 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.579 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.579 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.579 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.579 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.580 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.580 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.580 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.580 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.580 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.580 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.580 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.580 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.580 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.580 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.580 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.580 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.580 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.581 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.581 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.581 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.581 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.581 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.581 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.581 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.581 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.581 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.581 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.581 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.581 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "iostat", 2024-09-19T09:28:16.582 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.582 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.582 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.582 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.582 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.582 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.582 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.582 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.582 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.582 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.582 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.582 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.582 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.583 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.583 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.583 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.583 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.583 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.583 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.583 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.583 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.583 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.583 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.583 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.583 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.583 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.583 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.584 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.584 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.584 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.584 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.584 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.584 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.584 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.584 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.584 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.584 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.584 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.584 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.585 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.585 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.585 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.585 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.585 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.585 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.585 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.585 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.585 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.585 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.585 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.585 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.585 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.585 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.586 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.586 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.586 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.586 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.586 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.586 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.586 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.586 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.586 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.586 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.586 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.586 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.586 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.586 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.587 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.587 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.587 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.587 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.587 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "k8sevents", 2024-09-19T09:28:16.587 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.587 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.587 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.587 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_event_retention_days": { 2024-09-19T09:28:16.587 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ceph_event_retention_days", 2024-09-19T09:28:16.587 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.587 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.588 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.588 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "7", 2024-09-19T09:28:16.588 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.588 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.588 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.588 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T09:28:16.588 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.588 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.588 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.588 INFO:teuthology.orchestra.run.smithi120.stdout: "config_check_secs": { 2024-09-19T09:28:16.588 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "config_check_secs", 2024-09-19T09:28:16.588 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.588 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.589 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.589 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-19T09:28:16.589 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "10", 2024-09-19T09:28:16.589 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.589 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.589 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T09:28:16.589 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.589 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.589 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.589 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.589 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.590 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.590 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.590 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.590 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.590 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.590 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.590 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.590 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.590 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.590 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.590 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.590 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.590 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.590 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.591 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.591 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.591 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.591 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.591 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.591 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.591 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.591 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.591 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.591 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.591 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.591 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.591 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.592 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.592 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.592 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.592 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.592 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.592 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.592 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.592 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.592 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.592 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.592 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.592 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.593 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.593 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.593 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.593 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.593 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.593 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.593 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.593 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.593 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.593 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.593 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.593 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.593 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.594 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.594 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.594 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.594 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.594 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.594 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.594 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.594 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.594 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.594 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.594 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.594 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.594 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.594 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.595 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "localpool", 2024-09-19T09:28:16.595 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.595 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.595 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.595 INFO:teuthology.orchestra.run.smithi120.stdout: "failure_domain": { 2024-09-19T09:28:16.595 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "failure_domain", 2024-09-19T09:28:16.595 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.595 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.595 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.595 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "host", 2024-09-19T09:28:16.595 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.595 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.596 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.596 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "failure domain for any created local pool", 2024-09-19T09:28:16.596 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T09:28:16.596 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.596 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.596 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.596 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.596 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.596 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.596 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.596 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.596 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.597 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.597 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.597 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.597 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.597 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.597 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.597 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.597 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.597 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.597 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.597 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.597 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.597 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.598 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.598 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.598 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.598 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.598 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.598 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.598 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.598 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.598 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.598 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.598 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.598 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.598 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.599 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.599 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.599 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.599 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.599 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.599 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.599 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.599 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.599 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.599 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.599 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.599 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.599 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.599 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.599 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.600 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.600 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.600 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.600 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.600 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.600 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.600 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.600 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.600 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.600 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.600 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.600 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.600 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.601 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.601 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.601 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.601 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.601 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.601 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": { 2024-09-19T09:28:16.601 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "min_size", 2024-09-19T09:28:16.601 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.601 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.601 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.601 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.601 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.601 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.602 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.602 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "default min_size for any created local pool", 2024-09-19T09:28:16.602 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T09:28:16.602 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.602 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.602 INFO:teuthology.orchestra.run.smithi120.stdout: "num_rep": { 2024-09-19T09:28:16.602 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "num_rep", 2024-09-19T09:28:16.602 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.602 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.602 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.602 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "3", 2024-09-19T09:28:16.602 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.603 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.603 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.603 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "default replica count for any created local pool", 2024-09-19T09:28:16.603 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.603 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.603 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.604 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": { 2024-09-19T09:28:16.604 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "pg_num", 2024-09-19T09:28:16.604 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.604 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.604 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.604 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "128", 2024-09-19T09:28:16.604 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.604 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.604 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.604 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T09:28:16.604 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.604 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.605 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.605 INFO:teuthology.orchestra.run.smithi120.stdout: "prefix": { 2024-09-19T09:28:16.605 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "prefix", 2024-09-19T09:28:16.605 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.605 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.605 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.605 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.605 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.605 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.605 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.605 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "name prefix for any created local pool", 2024-09-19T09:28:16.605 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.605 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.606 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.606 INFO:teuthology.orchestra.run.smithi120.stdout: "subtree": { 2024-09-19T09:28:16.606 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "subtree", 2024-09-19T09:28:16.606 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.606 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.606 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.606 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "rack", 2024-09-19T09:28:16.606 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.606 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.606 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.606 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T09:28:16.606 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T09:28:16.606 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.607 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.607 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.607 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.607 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mds_autoscaler", 2024-09-19T09:28:16.607 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.607 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.607 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.607 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.607 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.607 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.607 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.607 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.608 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.608 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.608 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.608 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.608 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.608 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.608 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.608 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.608 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.608 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.608 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.608 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.608 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.609 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.609 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.609 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.609 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.609 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.609 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.609 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.609 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.609 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.609 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.609 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.609 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.609 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.609 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.610 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.610 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.610 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.610 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.610 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.610 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.610 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.610 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.610 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.610 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.610 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.610 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.610 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.611 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.611 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.611 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.611 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.611 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.611 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.611 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.611 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.611 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.611 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.611 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.611 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.611 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.612 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.612 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.612 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.612 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.612 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.612 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.612 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.612 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.612 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.612 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.612 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mirroring", 2024-09-19T09:28:16.612 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.613 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.613 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.613 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.613 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.613 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.613 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.613 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.613 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.613 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.613 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.613 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.613 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.613 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.613 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.614 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.614 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.614 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.614 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.614 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.614 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.614 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.614 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.614 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.614 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.614 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.614 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.614 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.615 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.615 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.615 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.615 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.615 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.615 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.615 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.615 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.615 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.615 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.615 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.615 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.615 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.616 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.616 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.616 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.616 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.616 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.616 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.616 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.616 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.616 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.616 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.616 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.616 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.616 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.616 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.617 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.617 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.617 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.617 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.617 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.617 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.617 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.617 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.617 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.617 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.617 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.617 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.618 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.618 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.618 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.618 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.618 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "nfs", 2024-09-19T09:28:16.618 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.618 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.618 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.618 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.618 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.618 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.618 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.619 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.619 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.619 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.619 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.619 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.619 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.619 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.619 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.619 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.619 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.619 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.619 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.619 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.619 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.620 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.620 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.620 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.620 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.620 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.620 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.620 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.620 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.620 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.620 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.620 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.620 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.620 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.621 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.621 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.621 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.621 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.621 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.621 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.621 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.621 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.621 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.621 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.621 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.621 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.621 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.622 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.622 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.622 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.622 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.622 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.622 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.622 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.622 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.622 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.622 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.622 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.622 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.622 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.623 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.623 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.623 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.623 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.623 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.623 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.623 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.623 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.623 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.623 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.623 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.623 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "orchestrator", 2024-09-19T09:28:16.624 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.624 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.624 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.624 INFO:teuthology.orchestra.run.smithi120.stdout: "fail_fs": { 2024-09-19T09:28:16.624 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "fail_fs", 2024-09-19T09:28:16.624 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.624 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.624 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.624 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.624 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.624 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.624 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.624 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T09:28:16.624 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.625 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.625 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.625 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.625 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.625 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.625 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.625 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.625 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.625 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.625 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.625 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.625 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.625 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.626 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.626 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.626 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.626 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.626 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.626 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.626 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.626 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.626 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.626 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.626 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.626 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.626 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.627 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.627 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.627 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.627 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.627 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.627 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.627 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.627 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.627 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.627 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.627 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.627 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.627 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.628 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.628 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.628 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.628 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.628 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.628 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.628 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.628 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.628 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.628 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.628 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.628 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.628 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.628 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.629 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.629 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.629 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.629 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.629 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.629 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.629 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.629 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.629 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.629 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.629 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.629 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.629 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.630 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.630 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.630 INFO:teuthology.orchestra.run.smithi120.stdout: "orchestrator": { 2024-09-19T09:28:16.630 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "orchestrator", 2024-09-19T09:28:16.630 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.630 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.630 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.630 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.630 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.630 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.630 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.630 INFO:teuthology.orchestra.run.smithi120.stdout: "cephadm", 2024-09-19T09:28:16.630 INFO:teuthology.orchestra.run.smithi120.stdout: "rook", 2024-09-19T09:28:16.631 INFO:teuthology.orchestra.run.smithi120.stdout: "test_orchestrator" 2024-09-19T09:28:16.631 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.631 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Orchestrator backend", 2024-09-19T09:28:16.631 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.631 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.631 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.631 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.631 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.631 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "osd_perf_query", 2024-09-19T09:28:16.631 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.631 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.632 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.632 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.632 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.632 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.632 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.632 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.632 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.632 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.632 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.632 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.632 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.632 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.632 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.633 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.633 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.633 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.633 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.633 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.633 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.633 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.633 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.633 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.633 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.633 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.633 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.633 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.634 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.634 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.634 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.634 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.634 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.634 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.634 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.634 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.634 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.634 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.634 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.634 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.634 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.635 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.635 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.635 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.635 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.635 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.635 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.635 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.635 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.635 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.635 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.635 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.635 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.635 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.635 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.636 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.636 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.636 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.636 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.636 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.636 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.636 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.636 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.636 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.636 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.636 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.636 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.637 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.637 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.637 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.637 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.637 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "osd_support", 2024-09-19T09:28:16.637 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.637 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.637 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.637 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.637 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.637 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.638 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.638 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.638 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.638 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.638 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.638 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.638 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.638 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.638 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.638 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.639 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.639 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.639 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.639 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.639 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.639 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.639 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.639 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.639 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.639 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.639 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.639 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.639 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.640 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.640 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.640 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.640 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.640 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.640 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.640 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.640 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.640 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.640 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.640 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.640 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.640 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.641 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.641 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.641 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.641 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.641 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.641 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.641 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.641 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.641 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.641 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.641 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.641 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.641 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.641 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.642 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.642 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.642 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.642 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.642 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.642 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.642 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.642 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.642 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.642 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.642 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.642 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.642 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.643 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.643 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.643 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "pg_autoscaler", 2024-09-19T09:28:16.643 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.643 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.643 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.643 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.643 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.643 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.643 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.643 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.643 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.644 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.644 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.644 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.644 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.644 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.644 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.644 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.644 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.644 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.644 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.644 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.644 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.644 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.644 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.645 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.645 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.645 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.645 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.645 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.645 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.645 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.645 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.645 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.645 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.645 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.645 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.645 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.646 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.646 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.646 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.646 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.646 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.646 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.646 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.646 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.646 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.646 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.646 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.647 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.647 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.647 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.647 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.647 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.647 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.647 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.647 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.647 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.647 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.647 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.647 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.647 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.648 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.648 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.648 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.648 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.648 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.648 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.648 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.648 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.648 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.648 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-19T09:28:16.648 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-19T09:28:16.648 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.649 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.649 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.649 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-19T09:28:16.649 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.649 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.649 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.649 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.649 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.649 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.649 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.649 INFO:teuthology.orchestra.run.smithi120.stdout: "threshold": { 2024-09-19T09:28:16.649 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "threshold", 2024-09-19T09:28:16.649 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-19T09:28:16.650 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.650 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.650 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "3.0", 2024-09-19T09:28:16.650 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1.0", 2024-09-19T09:28:16.650 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.650 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.650 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "scaling threshold", 2024-09-19T09:28:16.650 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T09:28:16.650 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.650 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.650 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.650 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.651 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "progress", 2024-09-19T09:28:16.651 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.651 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.651 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.651 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_pg_recovery_event": { 2024-09-19T09:28:16.651 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "allow_pg_recovery_event", 2024-09-19T09:28:16.651 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.651 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.651 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.651 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.651 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.651 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.651 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.651 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T09:28:16.652 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.652 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.652 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.652 INFO:teuthology.orchestra.run.smithi120.stdout: "enabled": { 2024-09-19T09:28:16.652 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "enabled", 2024-09-19T09:28:16.652 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.652 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.652 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.652 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.652 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.652 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.652 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.652 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.653 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.653 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.653 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.653 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.653 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.653 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.653 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.653 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.653 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.653 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.653 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.653 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.653 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.654 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.656 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.657 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: "max_completed_events": { 2024-09-19T09:28:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_completed_events", 2024-09-19T09:28:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "50", 2024-09-19T09:28:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.658 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "number of past completed events to remember", 2024-09-19T09:28:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-19T09:28:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-19T09:28:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-19T09:28:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.659 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how long the module is going to sleep", 2024-09-19T09:28:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "prometheus", 2024-09-19T09:28:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.660 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "cache": { 2024-09-19T09:28:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cache", 2024-09-19T09:28:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: "exclude_perf_counters": { 2024-09-19T09:28:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "exclude_perf_counters", 2024-09-19T09:28:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T09:28:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T09:28:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.663 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.664 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.665 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.666 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.667 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_stats_pools": { 2024-09-19T09:28:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rbd_stats_pools", 2024-09-19T09:28:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.668 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T09:28:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T09:28:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "300", 2024-09-19T09:28:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.669 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: "scrape_interval": { 2024-09-19T09:28:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "scrape_interval", 2024-09-19T09:28:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-19T09:28:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "15.0", 2024-09-19T09:28:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.670 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: "server_addr": { 2024-09-19T09:28:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_addr", 2024-09-19T09:28:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "::", 2024-09-19T09:28:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T09:28:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.671 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: "server_port": { 2024-09-19T09:28:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_port", 2024-09-19T09:28:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "9283", 2024-09-19T09:28:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T09:28:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.672 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "stale_cache_strategy": { 2024-09-19T09:28:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "stale_cache_strategy", 2024-09-19T09:28:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "log", 2024-09-19T09:28:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.673 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: "standby_behaviour": { 2024-09-19T09:28:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "standby_behaviour", 2024-09-19T09:28:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "default", 2024-09-19T09:28:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.674 INFO:teuthology.orchestra.run.smithi120.stdout: "default", 2024-09-19T09:28:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: "error" 2024-09-19T09:28:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: "standby_error_status_code": { 2024-09-19T09:28:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "standby_error_status_code", 2024-09-19T09:28:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "500", 2024-09-19T09:28:16.675 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "400", 2024-09-19T09:28:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "599", 2024-09-19T09:28:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rbd_support", 2024-09-19T09:28:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.676 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.678 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.679 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.680 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.681 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "max_concurrent_snap_create": { 2024-09-19T09:28:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_concurrent_snap_create", 2024-09-19T09:28:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-19T09:28:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.682 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "mirror_snapshot_schedule": { 2024-09-19T09:28:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T09:28:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.683 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "trash_purge_schedule": { 2024-09-19T09:28:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "trash_purge_schedule", 2024-09-19T09:28:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.684 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "restful", 2024-09-19T09:28:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: "enable_auth": { 2024-09-19T09:28:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "enable_auth", 2024-09-19T09:28:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.685 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: "key_file": { 2024-09-19T09:28:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "key_file", 2024-09-19T09:28:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.686 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.687 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.688 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.689 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.690 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: "server_addr": { 2024-09-19T09:28:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_addr", 2024-09-19T09:28:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "server_port": { 2024-09-19T09:28:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_port", 2024-09-19T09:28:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.694 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rgw", 2024-09-19T09:28:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.695 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.697 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.698 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.699 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T09:28:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T09:28:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-19T09:28:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.700 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T09:28:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rook", 2024-09-19T09:28:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.701 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.702 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.703 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.704 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.705 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.706 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: "storage_class": { 2024-09-19T09:28:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "storage_class", 2024-09-19T09:28:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.707 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "local", 2024-09-19T09:28:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T09:28:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "selftest", 2024-09-19T09:28:16.708 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.709 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.710 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.711 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.712 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.713 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: "roption1": { 2024-09-19T09:28:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "roption1", 2024-09-19T09:28:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.714 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "roption2": { 2024-09-19T09:28:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "roption2", 2024-09-19T09:28:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "xyz", 2024-09-19T09:28:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.715 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption1": { 2024-09-19T09:28:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption1", 2024-09-19T09:28:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.716 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption2": { 2024-09-19T09:28:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption2", 2024-09-19T09:28:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.717 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption3": { 2024-09-19T09:28:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption3", 2024-09-19T09:28:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-19T09:28:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.718 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption4": { 2024-09-19T09:28:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption4", 2024-09-19T09:28:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.719 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption5": { 2024-09-19T09:28:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption5", 2024-09-19T09:28:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.720 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption6": { 2024-09-19T09:28:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption6", 2024-09-19T09:28:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption7": { 2024-09-19T09:28:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption7", 2024-09-19T09:28:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1", 2024-09-19T09:28:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "42", 2024-09-19T09:28:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.722 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "testkey": { 2024-09-19T09:28:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "testkey", 2024-09-19T09:28:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.723 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: "testlkey": { 2024-09-19T09:28:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "testlkey", 2024-09-19T09:28:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.724 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: "testnewline": { 2024-09-19T09:28:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "testnewline", 2024-09-19T09:28:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.725 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "snap_schedule", 2024-09-19T09:28:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.726 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_m_granularity": { 2024-09-19T09:28:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "allow_m_granularity", 2024-09-19T09:28:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T09:28:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: "dump_on_update": { 2024-09-19T09:28:16.727 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "dump_on_update", 2024-09-19T09:28:16.728 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.728 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.728 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.728 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.728 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.728 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.728 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.728 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "dump database to debug log on update", 2024-09-19T09:28:16.728 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.728 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.728 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.728 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.729 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.730 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.730 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.730 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.730 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.730 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.730 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.730 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.730 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.730 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.730 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.730 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.730 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.731 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.731 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.731 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.731 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.731 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.731 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.731 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.731 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.731 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.731 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.731 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.731 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.731 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.732 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.732 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.732 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.732 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.732 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.732 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.732 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.732 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.732 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.732 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.732 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.732 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.733 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.733 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.733 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.733 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.733 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.733 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.733 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.733 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.733 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.733 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.733 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.733 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.733 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.734 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.734 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.734 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "stats", 2024-09-19T09:28:16.734 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.734 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.734 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.734 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.734 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.734 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.734 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.734 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.734 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.735 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.735 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.735 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.735 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.735 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.735 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.735 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.735 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.735 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.735 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.735 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.735 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.735 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.736 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.736 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.736 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.736 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.736 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.736 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.736 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.736 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.736 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.736 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.736 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.736 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.737 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.737 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.737 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.737 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.737 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.737 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.737 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.737 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.737 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.737 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.737 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.737 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.737 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.738 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.738 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.738 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.738 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.738 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.738 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.738 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.738 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.738 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.739 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.739 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.739 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.739 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.739 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.739 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.739 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.739 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.739 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.739 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.739 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.739 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.739 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.740 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.740 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.740 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "status", 2024-09-19T09:28:16.740 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.740 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.740 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.740 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.740 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.740 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.740 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.740 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.741 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.741 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.741 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.741 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.741 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.741 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.741 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.741 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.741 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.741 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.741 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.741 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.741 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.741 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.742 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.742 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.742 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.742 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.742 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.742 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.742 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.742 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.742 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.742 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.742 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.742 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.742 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.743 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.744 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.744 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.744 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.744 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.744 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.744 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.744 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.744 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.744 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.744 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.744 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.744 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.745 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.745 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.745 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.745 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.745 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.745 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.745 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.745 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.745 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.745 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.745 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "telegraf", 2024-09-19T09:28:16.746 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.746 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.746 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.746 INFO:teuthology.orchestra.run.smithi120.stdout: "address": { 2024-09-19T09:28:16.746 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "address", 2024-09-19T09:28:16.746 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.746 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.746 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.746 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T09:28:16.746 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.746 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.746 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.746 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.746 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.747 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.747 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.747 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-19T09:28:16.747 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-19T09:28:16.747 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.747 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.747 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.747 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "15", 2024-09-19T09:28:16.747 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.747 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.747 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.747 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.748 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.748 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.748 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.748 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.748 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.748 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.748 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.748 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.748 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.748 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.748 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.749 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.749 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.749 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.749 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.749 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.749 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.749 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.749 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.749 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.749 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.749 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.749 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.749 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.750 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.750 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.750 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.750 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.750 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.750 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.750 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.750 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.750 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.750 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.750 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.750 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.751 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.751 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.751 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.751 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.751 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.751 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.751 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.751 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.751 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.751 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.751 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.751 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.751 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.751 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.752 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.752 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.752 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.752 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.752 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.752 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.752 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.752 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.752 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.752 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.752 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.752 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.752 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.753 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.753 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.753 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.753 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.753 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.753 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.753 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.753 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.753 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "telemetry", 2024-09-19T09:28:16.753 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.753 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.754 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.754 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_basic": { 2024-09-19T09:28:16.754 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_basic", 2024-09-19T09:28:16.754 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.754 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.754 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.754 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.754 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.754 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.754 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.754 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T09:28:16.754 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.754 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.754 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.755 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_crash": { 2024-09-19T09:28:16.755 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_crash", 2024-09-19T09:28:16.755 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.755 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.755 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.755 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.755 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.755 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.755 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.755 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T09:28:16.755 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.755 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.755 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.756 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_device": { 2024-09-19T09:28:16.756 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_device", 2024-09-19T09:28:16.756 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.756 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.756 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.756 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.756 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.756 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.756 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.756 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T09:28:16.756 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.756 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.756 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.757 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_ident": { 2024-09-19T09:28:16.757 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_ident", 2024-09-19T09:28:16.757 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.757 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.757 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.757 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.757 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.757 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.757 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.757 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T09:28:16.757 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.757 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.757 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.758 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_perf": { 2024-09-19T09:28:16.758 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_perf", 2024-09-19T09:28:16.758 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.758 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.758 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.758 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.758 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.758 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.758 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.758 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T09:28:16.758 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.758 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.758 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.759 INFO:teuthology.orchestra.run.smithi120.stdout: "contact": { 2024-09-19T09:28:16.759 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "contact", 2024-09-19T09:28:16.759 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.759 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.759 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.759 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.759 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.759 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.759 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.759 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.759 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.759 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.760 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.760 INFO:teuthology.orchestra.run.smithi120.stdout: "description": { 2024-09-19T09:28:16.760 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "description", 2024-09-19T09:28:16.760 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.760 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.760 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.760 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.760 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.760 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.760 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.760 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.760 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.760 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.761 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.761 INFO:teuthology.orchestra.run.smithi120.stdout: "device_url": { 2024-09-19T09:28:16.761 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "device_url", 2024-09-19T09:28:16.761 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.761 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.761 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.761 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T09:28:16.761 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.761 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.761 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.761 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.761 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.761 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.762 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.762 INFO:teuthology.orchestra.run.smithi120.stdout: "enabled": { 2024-09-19T09:28:16.762 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "enabled", 2024-09-19T09:28:16.762 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.762 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.762 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.762 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.762 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.762 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.762 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.762 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.762 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.763 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.763 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.763 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-19T09:28:16.763 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-19T09:28:16.763 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.763 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.763 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.763 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "24", 2024-09-19T09:28:16.763 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "8", 2024-09-19T09:28:16.763 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.763 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.763 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.763 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.763 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.764 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.764 INFO:teuthology.orchestra.run.smithi120.stdout: "last_opt_revision": { 2024-09-19T09:28:16.764 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "last_opt_revision", 2024-09-19T09:28:16.764 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.764 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.764 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.764 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "1", 2024-09-19T09:28:16.764 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.764 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.764 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.764 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.764 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.765 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.765 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.765 INFO:teuthology.orchestra.run.smithi120.stdout: "leaderboard": { 2024-09-19T09:28:16.765 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "leaderboard", 2024-09-19T09:28:16.765 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.765 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.765 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.765 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.765 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.765 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.765 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.765 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.765 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.766 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.766 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.766 INFO:teuthology.orchestra.run.smithi120.stdout: "leaderboard_description": { 2024-09-19T09:28:16.766 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "leaderboard_description", 2024-09-19T09:28:16.766 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.766 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.766 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.766 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.766 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.766 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.766 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.766 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.766 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.767 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.767 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.767 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.767 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.767 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.767 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.767 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.767 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.767 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.767 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.767 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.767 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.767 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.768 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.768 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.768 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.768 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.768 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.768 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.768 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.768 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.768 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.768 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.768 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.768 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.768 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.769 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.769 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.769 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.769 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.769 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.769 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.769 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.769 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.769 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.769 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.769 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.769 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.769 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.770 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.770 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.770 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.770 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.770 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.770 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.770 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.770 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.770 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.770 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.770 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.770 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.770 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.770 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.771 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.771 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.771 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.771 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.771 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.771 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.771 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.771 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.771 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.771 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.771 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.771 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.771 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.771 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.772 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.772 INFO:teuthology.orchestra.run.smithi120.stdout: "organization": { 2024-09-19T09:28:16.772 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "organization", 2024-09-19T09:28:16.772 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.772 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.772 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.772 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.772 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.772 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.772 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.772 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.773 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.773 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.773 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.773 INFO:teuthology.orchestra.run.smithi120.stdout: "proxy": { 2024-09-19T09:28:16.773 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "proxy", 2024-09-19T09:28:16.773 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.773 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.773 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.773 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.773 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.773 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.773 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.773 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.774 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.774 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.774 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.774 INFO:teuthology.orchestra.run.smithi120.stdout: "url": { 2024-09-19T09:28:16.774 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "url", 2024-09-19T09:28:16.774 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.774 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.774 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.774 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T09:28:16.774 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.774 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.774 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.774 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.775 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.775 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.775 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.775 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.775 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.775 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "test_orchestrator", 2024-09-19T09:28:16.775 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.775 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.775 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.775 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.775 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.775 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.776 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.776 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.776 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.776 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.776 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.776 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.776 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.776 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.776 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.776 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.776 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.776 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.776 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.776 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.777 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.777 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.777 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.777 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.777 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.777 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.777 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.777 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.777 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.777 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.777 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.777 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.777 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.778 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.778 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.778 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.778 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.778 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.778 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.778 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.778 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.778 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.778 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.778 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.778 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.779 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.779 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.779 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.779 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.779 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.779 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.779 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.779 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.779 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.779 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.779 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.779 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.779 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.780 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.780 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.780 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.780 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.780 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.780 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.780 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.780 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.780 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.780 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.780 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.780 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.780 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.781 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "volumes", 2024-09-19T09:28:16.781 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.781 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.781 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.781 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.781 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.781 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.781 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.781 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.781 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.781 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.781 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.782 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.782 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.782 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.782 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.782 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.782 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.782 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.782 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.782 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.782 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.782 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.782 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.782 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.782 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.783 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.783 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.783 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.783 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.783 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.783 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.783 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.783 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.783 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.783 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.783 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.783 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.783 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.784 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.784 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.784 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.784 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.784 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.784 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.784 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.784 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.784 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.784 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.784 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.784 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.784 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.784 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.785 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.785 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.785 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.785 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.785 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.785 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.785 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.785 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.785 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.785 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.785 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.785 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.785 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.786 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.786 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.786 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.786 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.786 INFO:teuthology.orchestra.run.smithi120.stdout: "max_concurrent_clones": { 2024-09-19T09:28:16.786 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_concurrent_clones", 2024-09-19T09:28:16.786 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.786 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.786 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.786 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "4", 2024-09-19T09:28:16.786 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.786 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.786 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.787 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T09:28:16.787 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.787 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.787 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.787 INFO:teuthology.orchestra.run.smithi120.stdout: "periodic_async_work": { 2024-09-19T09:28:16.787 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "periodic_async_work", 2024-09-19T09:28:16.787 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.787 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.787 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.787 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.787 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.787 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.787 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.788 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Periodically check for async work", 2024-09-19T09:28:16.788 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.788 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.788 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.788 INFO:teuthology.orchestra.run.smithi120.stdout: "snapshot_clone_delay": { 2024-09-19T09:28:16.788 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "snapshot_clone_delay", 2024-09-19T09:28:16.788 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.788 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.788 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.788 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-19T09:28:16.788 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.788 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.789 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.789 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T09:28:16.789 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.789 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.789 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.789 INFO:teuthology.orchestra.run.smithi120.stdout: "snapshot_clone_no_wait": { 2024-09-19T09:28:16.789 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T09:28:16.789 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.789 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.789 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.789 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.789 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.789 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.790 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.790 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T09:28:16.790 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.790 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.790 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.790 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.790 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.790 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "zabbix", 2024-09-19T09:28:16.790 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.790 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.790 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.790 INFO:teuthology.orchestra.run.smithi120.stdout: "discovery_interval": { 2024-09-19T09:28:16.791 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "discovery_interval", 2024-09-19T09:28:16.791 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-19T09:28:16.791 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.791 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.791 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "100", 2024-09-19T09:28:16.791 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.791 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.791 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.791 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.791 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.791 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.791 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.791 INFO:teuthology.orchestra.run.smithi120.stdout: "identifier": { 2024-09-19T09:28:16.792 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "identifier", 2024-09-19T09:28:16.792 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.792 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.792 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.792 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.792 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.792 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.792 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.792 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.792 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.792 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.792 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.792 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-19T09:28:16.793 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-19T09:28:16.793 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.793 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.793 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.793 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-19T09:28:16.793 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.793 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.793 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.793 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.793 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.793 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.793 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.793 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.794 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.794 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.794 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.794 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.794 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.794 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.794 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.794 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.794 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.794 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.794 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.794 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.794 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.794 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.795 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.795 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.795 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.795 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.795 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.795 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.795 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.795 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.795 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.795 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.795 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.795 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.795 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.796 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.796 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.796 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.796 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.796 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.796 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.796 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.796 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.796 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.796 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.796 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.796 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.796 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.797 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.797 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.797 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.797 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.797 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.797 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.797 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.797 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.797 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.797 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.797 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.797 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.797 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.798 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.798 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.798 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.798 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.798 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.798 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.798 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.798 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.798 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.798 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.798 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.798 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.798 INFO:teuthology.orchestra.run.smithi120.stdout: "zabbix_host": { 2024-09-19T09:28:16.799 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "zabbix_host", 2024-09-19T09:28:16.799 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.799 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.799 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.799 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.799 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.799 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.799 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.799 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.799 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.799 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.799 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.799 INFO:teuthology.orchestra.run.smithi120.stdout: "zabbix_port": { 2024-09-19T09:28:16.799 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "zabbix_port", 2024-09-19T09:28:16.800 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.800 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.800 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.800 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10051", 2024-09-19T09:28:16.800 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.800 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.800 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.800 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.800 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.800 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.800 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.800 INFO:teuthology.orchestra.run.smithi120.stdout: "zabbix_sender": { 2024-09-19T09:28:16.801 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "zabbix_sender", 2024-09-19T09:28:16.801 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.801 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.801 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.801 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T09:28:16.801 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.801 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.801 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.801 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.801 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.801 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.801 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.801 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.801 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.802 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.802 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-19T09:28:16.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.802 INFO:teuthology.orchestra.run.smithi120.stdout: "gid": 24110, 2024-09-19T09:28:16.802 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "y", 2024-09-19T09:28:16.802 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_features": 4540701547738038271, 2024-09-19T09:28:16.802 INFO:teuthology.orchestra.run.smithi120.stdout: "available_modules": [ 2024-09-19T09:28:16.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.802 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "alerts", 2024-09-19T09:28:16.802 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.802 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.802 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.802 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-19T09:28:16.803 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-19T09:28:16.803 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.803 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.803 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.803 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-19T09:28:16.803 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.803 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.803 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.803 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T09:28:16.803 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.803 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.803 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.804 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.804 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.804 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.804 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.804 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.804 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.804 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.804 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.804 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.805 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.805 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.805 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.805 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.805 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.805 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.805 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.805 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.805 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.805 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.805 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.805 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.805 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.806 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.806 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.806 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.806 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.806 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.806 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.806 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.806 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.806 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.806 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.806 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.806 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.806 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.807 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.807 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.807 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.807 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.807 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.807 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.807 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.807 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.807 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.807 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.807 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.807 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.807 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.807 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.808 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.808 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.808 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.808 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.808 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.808 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.808 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.808 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.808 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.808 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.808 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.808 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.808 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.809 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.809 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.809 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.809 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.809 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_destination": { 2024-09-19T09:28:16.809 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_destination", 2024-09-19T09:28:16.809 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.809 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.809 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.809 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.809 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.809 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.810 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.810 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Email address to send alerts to", 2024-09-19T09:28:16.810 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.810 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.810 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.810 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_from_name": { 2024-09-19T09:28:16.810 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_from_name", 2024-09-19T09:28:16.810 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.810 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.810 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.810 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "Ceph", 2024-09-19T09:28:16.810 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.810 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.811 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.811 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Email From: name", 2024-09-19T09:28:16.811 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.811 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.811 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.811 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_host": { 2024-09-19T09:28:16.811 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_host", 2024-09-19T09:28:16.811 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.811 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.811 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.811 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.811 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.811 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.812 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.812 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "SMTP server", 2024-09-19T09:28:16.812 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.812 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.812 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.812 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_password": { 2024-09-19T09:28:16.812 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_password", 2024-09-19T09:28:16.812 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.812 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.812 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.812 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.812 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.812 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.813 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.813 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Password to authenticate with", 2024-09-19T09:28:16.813 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.813 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.813 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.813 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_port": { 2024-09-19T09:28:16.813 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_port", 2024-09-19T09:28:16.813 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.813 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.813 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.813 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "465", 2024-09-19T09:28:16.813 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.813 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.814 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.814 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "SMTP port", 2024-09-19T09:28:16.814 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.814 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.814 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.814 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_sender": { 2024-09-19T09:28:16.814 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_sender", 2024-09-19T09:28:16.814 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.814 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.814 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.814 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.814 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.814 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.815 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.815 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "SMTP envelope sender", 2024-09-19T09:28:16.815 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.815 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.815 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.815 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_ssl": { 2024-09-19T09:28:16.815 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_ssl", 2024-09-19T09:28:16.815 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.815 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.815 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.815 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.816 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.816 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.816 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.816 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T09:28:16.816 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.816 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.816 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.816 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_user": { 2024-09-19T09:28:16.816 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_user", 2024-09-19T09:28:16.816 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.816 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.816 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.816 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.817 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.817 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.817 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.817 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "User to authenticate as", 2024-09-19T09:28:16.817 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.817 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.817 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.817 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.817 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.817 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "balancer", 2024-09-19T09:28:16.817 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.817 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.818 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.818 INFO:teuthology.orchestra.run.smithi120.stdout: "active": { 2024-09-19T09:28:16.818 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "active", 2024-09-19T09:28:16.818 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.818 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.818 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.818 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.818 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.818 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.818 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.818 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T09:28:16.818 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.818 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.818 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.819 INFO:teuthology.orchestra.run.smithi120.stdout: "begin_time": { 2024-09-19T09:28:16.819 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "begin_time", 2024-09-19T09:28:16.819 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.819 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.819 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.819 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0000", 2024-09-19T09:28:16.819 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.819 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.819 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.819 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T09:28:16.819 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T09:28:16.819 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.820 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.820 INFO:teuthology.orchestra.run.smithi120.stdout: "begin_weekday": { 2024-09-19T09:28:16.820 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "begin_weekday", 2024-09-19T09:28:16.820 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-19T09:28:16.820 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.820 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.820 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-19T09:28:16.820 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "0", 2024-09-19T09:28:16.820 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "6", 2024-09-19T09:28:16.820 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.820 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T09:28:16.820 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T09:28:16.821 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.821 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.821 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_compat_max_iterations": { 2024-09-19T09:28:16.821 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crush_compat_max_iterations", 2024-09-19T09:28:16.821 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-19T09:28:16.821 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.821 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.821 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "25", 2024-09-19T09:28:16.821 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1", 2024-09-19T09:28:16.821 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "250", 2024-09-19T09:28:16.821 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.821 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T09:28:16.821 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.822 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.822 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.822 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_compat_metrics": { 2024-09-19T09:28:16.822 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crush_compat_metrics", 2024-09-19T09:28:16.822 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.822 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.822 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.822 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T09:28:16.822 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.822 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.822 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.822 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T09:28:16.822 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T09:28:16.823 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.823 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.823 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_compat_step": { 2024-09-19T09:28:16.823 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crush_compat_step", 2024-09-19T09:28:16.823 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-19T09:28:16.823 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.823 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.823 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0.5", 2024-09-19T09:28:16.823 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "0.001", 2024-09-19T09:28:16.823 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "0.999", 2024-09-19T09:28:16.823 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.823 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "aggressiveness of optimization", 2024-09-19T09:28:16.824 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T09:28:16.824 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.824 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.824 INFO:teuthology.orchestra.run.smithi120.stdout: "end_time": { 2024-09-19T09:28:16.824 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "end_time", 2024-09-19T09:28:16.824 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.824 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.824 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.824 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "2359", 2024-09-19T09:28:16.824 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.824 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.824 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.824 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T09:28:16.825 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T09:28:16.825 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.825 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.825 INFO:teuthology.orchestra.run.smithi120.stdout: "end_weekday": { 2024-09-19T09:28:16.825 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "end_weekday", 2024-09-19T09:28:16.825 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-19T09:28:16.825 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.825 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.825 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-19T09:28:16.825 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "0", 2024-09-19T09:28:16.825 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "6", 2024-09-19T09:28:16.825 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.825 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T09:28:16.826 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T09:28:16.826 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.826 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.826 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.826 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.826 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.826 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.826 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.826 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.826 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.826 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.826 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.826 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.827 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.827 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.827 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.827 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.827 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.827 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.827 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.827 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.827 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.827 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.827 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.827 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.828 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.828 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.828 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.828 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.828 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.828 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.828 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.828 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.828 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.828 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.828 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.828 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.829 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.829 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.829 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.829 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.829 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.829 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.829 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.829 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.829 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.829 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.829 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.829 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.829 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.830 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.830 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.830 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.830 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.830 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.830 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.830 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.830 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.830 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.830 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.830 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.830 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.830 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.831 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.831 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.831 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.831 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.831 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.831 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.831 INFO:teuthology.orchestra.run.smithi120.stdout: "min_score": { 2024-09-19T09:28:16.831 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "min_score", 2024-09-19T09:28:16.831 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-19T09:28:16.831 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.831 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.831 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-19T09:28:16.831 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.832 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.832 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.832 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T09:28:16.832 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.832 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.832 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.832 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": { 2024-09-19T09:28:16.832 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mode", 2024-09-19T09:28:16.832 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.832 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.832 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.832 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "upmap", 2024-09-19T09:28:16.833 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.833 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.833 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.833 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-compat", 2024-09-19T09:28:16.833 INFO:teuthology.orchestra.run.smithi120.stdout: "none", 2024-09-19T09:28:16.833 INFO:teuthology.orchestra.run.smithi120.stdout: "read", 2024-09-19T09:28:16.833 INFO:teuthology.orchestra.run.smithi120.stdout: "upmap", 2024-09-19T09:28:16.833 INFO:teuthology.orchestra.run.smithi120.stdout: "upmap-read" 2024-09-19T09:28:16.833 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.833 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Balancer mode", 2024-09-19T09:28:16.833 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.833 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.833 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.833 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.834 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_ids": { 2024-09-19T09:28:16.834 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "pool_ids", 2024-09-19T09:28:16.834 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.834 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.834 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.834 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.834 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.834 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.834 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.834 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T09:28:16.834 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.834 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.834 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.835 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-19T09:28:16.835 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-19T09:28:16.835 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.835 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.835 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.835 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-19T09:28:16.835 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.835 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.835 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.835 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T09:28:16.835 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.835 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.835 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.836 INFO:teuthology.orchestra.run.smithi120.stdout: "upmap_max_deviation": { 2024-09-19T09:28:16.836 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "upmap_max_deviation", 2024-09-19T09:28:16.836 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.836 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.836 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.836 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-19T09:28:16.836 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1", 2024-09-19T09:28:16.836 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.836 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.836 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T09:28:16.836 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T09:28:16.836 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.836 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.837 INFO:teuthology.orchestra.run.smithi120.stdout: "upmap_max_optimizations": { 2024-09-19T09:28:16.837 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "upmap_max_optimizations", 2024-09-19T09:28:16.837 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-19T09:28:16.837 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.837 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.837 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-19T09:28:16.837 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.837 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.837 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.837 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T09:28:16.837 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.837 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.837 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.837 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.838 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.838 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.838 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cephadm", 2024-09-19T09:28:16.838 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.838 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.838 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.838 INFO:teuthology.orchestra.run.smithi120.stdout: "agent_down_multiplier": { 2024-09-19T09:28:16.838 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "agent_down_multiplier", 2024-09-19T09:28:16.838 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-19T09:28:16.839 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.839 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.839 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "3.0", 2024-09-19T09:28:16.839 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.839 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.839 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.839 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T09:28:16.839 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.839 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.839 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.839 INFO:teuthology.orchestra.run.smithi120.stdout: "agent_refresh_rate": { 2024-09-19T09:28:16.839 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "agent_refresh_rate", 2024-09-19T09:28:16.839 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.840 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.840 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.840 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "20", 2024-09-19T09:28:16.840 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.840 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.840 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.840 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T09:28:16.840 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.840 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.840 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.840 INFO:teuthology.orchestra.run.smithi120.stdout: "agent_starting_port": { 2024-09-19T09:28:16.840 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "agent_starting_port", 2024-09-19T09:28:16.841 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.841 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.841 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.841 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "4721", 2024-09-19T09:28:16.841 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.841 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.841 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.841 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T09:28:16.841 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.841 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.841 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.841 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.841 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_ptrace": { 2024-09-19T09:28:16.842 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "allow_ptrace", 2024-09-19T09:28:16.842 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.842 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.842 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.842 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.842 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.842 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.842 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.842 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T09:28:16.842 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T09:28:16.842 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.842 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.842 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.842 INFO:teuthology.orchestra.run.smithi120.stdout: "autotune_interval": { 2024-09-19T09:28:16.843 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "autotune_interval", 2024-09-19T09:28:16.843 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.843 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.843 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.843 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-19T09:28:16.843 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.843 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.843 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.843 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T09:28:16.843 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.843 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.843 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.843 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.844 INFO:teuthology.orchestra.run.smithi120.stdout: "autotune_memory_target_ratio": { 2024-09-19T09:28:16.844 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T09:28:16.844 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-19T09:28:16.844 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.844 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.844 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0.7", 2024-09-19T09:28:16.844 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.844 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.844 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.844 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T09:28:16.844 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.844 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.844 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.844 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.845 INFO:teuthology.orchestra.run.smithi120.stdout: "cephadm_log_destination": { 2024-09-19T09:28:16.845 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cephadm_log_destination", 2024-09-19T09:28:16.845 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.845 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.845 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.845 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.845 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.845 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.845 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.845 INFO:teuthology.orchestra.run.smithi120.stdout: "file", 2024-09-19T09:28:16.845 INFO:teuthology.orchestra.run.smithi120.stdout: "file,syslog", 2024-09-19T09:28:16.845 INFO:teuthology.orchestra.run.smithi120.stdout: "syslog" 2024-09-19T09:28:16.845 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.846 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T09:28:16.846 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.846 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.846 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.846 INFO:teuthology.orchestra.run.smithi120.stdout: "cgroups_split": { 2024-09-19T09:28:16.846 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cgroups_split", 2024-09-19T09:28:16.846 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.846 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.846 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.846 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.846 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.846 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.846 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.847 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T09:28:16.847 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.847 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.847 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.847 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.847 INFO:teuthology.orchestra.run.smithi120.stdout: "config_checks_enabled": { 2024-09-19T09:28:16.847 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "config_checks_enabled", 2024-09-19T09:28:16.847 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.847 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.847 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.847 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.847 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.847 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.847 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.847 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T09:28:16.848 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.848 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.848 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.848 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.848 INFO:teuthology.orchestra.run.smithi120.stdout: "config_dashboard": { 2024-09-19T09:28:16.848 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "config_dashboard", 2024-09-19T09:28:16.848 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.848 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.848 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.848 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.848 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.848 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.849 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.849 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T09:28:16.849 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.849 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.849 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.849 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.849 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_alertmanager": { 2024-09-19T09:28:16.849 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_alertmanager", 2024-09-19T09:28:16.849 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.849 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.849 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.849 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T09:28:16.849 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.849 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.850 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.850 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Prometheus container image", 2024-09-19T09:28:16.850 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.850 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.850 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.850 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.850 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_base": { 2024-09-19T09:28:16.850 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_base", 2024-09-19T09:28:16.850 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.850 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.850 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.850 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T09:28:16.850 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.851 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.851 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.851 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Container image name, without the tag", 2024-09-19T09:28:16.851 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.851 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.851 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.851 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_elasticsearch": { 2024-09-19T09:28:16.851 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_elasticsearch", 2024-09-19T09:28:16.851 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.851 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.851 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.851 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T09:28:16.851 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.852 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.852 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.852 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "elasticsearch container image", 2024-09-19T09:28:16.852 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.852 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.852 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.852 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.852 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_grafana": { 2024-09-19T09:28:16.852 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_grafana", 2024-09-19T09:28:16.852 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.852 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.852 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.852 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T09:28:16.853 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.853 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.853 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.853 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Prometheus container image", 2024-09-19T09:28:16.853 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.853 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.853 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.853 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_haproxy": { 2024-09-19T09:28:16.853 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_haproxy", 2024-09-19T09:28:16.853 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.853 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.853 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.854 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T09:28:16.854 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.854 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.854 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.854 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "HAproxy container image", 2024-09-19T09:28:16.854 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.854 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.854 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.854 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.854 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_jaeger_agent": { 2024-09-19T09:28:16.854 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_jaeger_agent", 2024-09-19T09:28:16.854 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.854 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.854 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.855 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T09:28:16.855 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.855 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.855 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.855 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Jaeger agent container image", 2024-09-19T09:28:16.855 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.855 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.855 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.855 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.855 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_jaeger_collector": { 2024-09-19T09:28:16.855 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_jaeger_collector", 2024-09-19T09:28:16.855 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.855 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.856 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.856 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T09:28:16.856 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.856 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.856 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.856 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Jaeger collector container image", 2024-09-19T09:28:16.856 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.856 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.856 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.856 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.856 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_jaeger_query": { 2024-09-19T09:28:16.856 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_jaeger_query", 2024-09-19T09:28:16.856 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.856 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.857 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.857 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T09:28:16.857 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.857 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.857 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.857 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Jaeger query container image", 2024-09-19T09:28:16.857 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.857 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.857 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.857 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.857 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_keepalived": { 2024-09-19T09:28:16.857 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_keepalived", 2024-09-19T09:28:16.857 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.858 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.858 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.858 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T09:28:16.858 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.858 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.858 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.858 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Keepalived container image", 2024-09-19T09:28:16.858 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.858 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.858 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.858 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.858 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_loki": { 2024-09-19T09:28:16.858 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_loki", 2024-09-19T09:28:16.858 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.859 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.859 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.859 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T09:28:16.859 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.859 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.859 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.859 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Loki container image", 2024-09-19T09:28:16.859 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.859 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.859 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.859 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.859 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_node_exporter": { 2024-09-19T09:28:16.859 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_node_exporter", 2024-09-19T09:28:16.860 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.860 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.860 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.860 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T09:28:16.860 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.860 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.860 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.860 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Prometheus container image", 2024-09-19T09:28:16.860 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.860 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.860 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.860 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.860 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_nvmeof": { 2024-09-19T09:28:16.860 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_nvmeof", 2024-09-19T09:28:16.861 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.861 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.861 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.861 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T09:28:16.861 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.861 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.861 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.861 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Nvme-of container image", 2024-09-19T09:28:16.861 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.861 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.861 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.861 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.861 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_prometheus": { 2024-09-19T09:28:16.862 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_prometheus", 2024-09-19T09:28:16.862 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.862 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.862 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.862 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T09:28:16.862 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.862 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.862 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.862 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Prometheus container image", 2024-09-19T09:28:16.862 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.862 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.862 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.862 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.863 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_promtail": { 2024-09-19T09:28:16.863 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_promtail", 2024-09-19T09:28:16.863 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.863 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.863 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.863 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T09:28:16.863 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.863 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.863 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.863 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Promtail container image", 2024-09-19T09:28:16.863 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.863 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.863 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.864 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.864 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_samba": { 2024-09-19T09:28:16.864 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_samba", 2024-09-19T09:28:16.864 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.864 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.864 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.864 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T09:28:16.864 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.864 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.864 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.864 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Samba/SMB container image", 2024-09-19T09:28:16.864 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.864 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.865 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.865 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.865 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_snmp_gateway": { 2024-09-19T09:28:16.865 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_snmp_gateway", 2024-09-19T09:28:16.865 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.865 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.865 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.865 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T09:28:16.865 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.865 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.865 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.865 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "SNMP Gateway container image", 2024-09-19T09:28:16.865 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.866 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.866 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.866 INFO:teuthology.orchestra.run.smithi120.stdout: "container_init": { 2024-09-19T09:28:16.866 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_init", 2024-09-19T09:28:16.866 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.866 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.866 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.866 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.866 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.866 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.866 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.867 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T09:28:16.867 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.867 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.867 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.867 INFO:teuthology.orchestra.run.smithi120.stdout: "daemon_cache_timeout": { 2024-09-19T09:28:16.867 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "daemon_cache_timeout", 2024-09-19T09:28:16.867 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.867 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.867 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.867 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-19T09:28:16.867 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.868 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.868 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.868 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T09:28:16.868 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.868 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.868 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.868 INFO:teuthology.orchestra.run.smithi120.stdout: "default_cephadm_command_timeout": { 2024-09-19T09:28:16.868 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T09:28:16.868 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.868 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.868 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.868 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "900", 2024-09-19T09:28:16.868 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.869 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.869 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.869 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T09:28:16.869 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.869 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.869 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.869 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.869 INFO:teuthology.orchestra.run.smithi120.stdout: "default_registry": { 2024-09-19T09:28:16.869 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "default_registry", 2024-09-19T09:28:16.869 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.869 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.869 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.869 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "docker.io", 2024-09-19T09:28:16.870 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.870 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.870 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.870 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T09:28:16.870 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.870 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.870 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.870 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.870 INFO:teuthology.orchestra.run.smithi120.stdout: "device_cache_timeout": { 2024-09-19T09:28:16.870 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "device_cache_timeout", 2024-09-19T09:28:16.870 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.870 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.870 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.870 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "1800", 2024-09-19T09:28:16.871 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.871 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.871 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.871 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "seconds to cache device inventory", 2024-09-19T09:28:16.871 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.871 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.871 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.871 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.871 INFO:teuthology.orchestra.run.smithi120.stdout: "device_enhanced_scan": { 2024-09-19T09:28:16.871 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "device_enhanced_scan", 2024-09-19T09:28:16.871 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.871 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.871 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.871 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.872 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.872 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.872 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.872 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T09:28:16.872 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.872 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.872 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.872 INFO:teuthology.orchestra.run.smithi120.stdout: "facts_cache_timeout": { 2024-09-19T09:28:16.872 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "facts_cache_timeout", 2024-09-19T09:28:16.872 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.872 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.872 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.872 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-19T09:28:16.873 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.873 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.873 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.873 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "seconds to cache host facts data", 2024-09-19T09:28:16.873 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.873 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.873 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.873 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.873 INFO:teuthology.orchestra.run.smithi120.stdout: "grafana_dashboards_path": { 2024-09-19T09:28:16.873 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "grafana_dashboards_path", 2024-09-19T09:28:16.873 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.873 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.873 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.873 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T09:28:16.874 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.874 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.874 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.874 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T09:28:16.874 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.874 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.874 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.874 INFO:teuthology.orchestra.run.smithi120.stdout: "host_check_interval": { 2024-09-19T09:28:16.874 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "host_check_interval", 2024-09-19T09:28:16.874 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.874 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.874 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.874 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-19T09:28:16.875 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.875 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.875 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.875 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to perform a host check", 2024-09-19T09:28:16.875 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.875 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.875 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.875 INFO:teuthology.orchestra.run.smithi120.stdout: "hw_monitoring": { 2024-09-19T09:28:16.875 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "hw_monitoring", 2024-09-19T09:28:16.875 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.875 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.875 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.875 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.876 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.876 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.876 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.876 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T09:28:16.876 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.876 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.876 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.876 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.876 INFO:teuthology.orchestra.run.smithi120.stdout: "inventory_list_all": { 2024-09-19T09:28:16.876 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "inventory_list_all", 2024-09-19T09:28:16.876 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.876 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.876 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.876 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.877 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.877 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.877 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.877 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T09:28:16.877 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.877 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.877 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.877 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.877 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.877 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.877 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.877 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.877 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.878 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.878 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.878 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.878 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.878 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.878 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.878 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.878 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.878 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.878 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.878 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.878 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.878 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.878 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.879 INFO:teuthology.orchestra.run.smithi120.stdout: "log_refresh_metadata": { 2024-09-19T09:28:16.879 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_refresh_metadata", 2024-09-19T09:28:16.879 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.879 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.879 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.879 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.879 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.879 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.879 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.879 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T09:28:16.879 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.880 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.880 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.880 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.880 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.880 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.880 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.880 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.880 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.880 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.880 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.880 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.880 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T09:28:16.880 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.881 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.881 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.881 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.881 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.881 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.881 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.881 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.881 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.881 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.881 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.881 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.881 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.882 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.882 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.882 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.882 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.882 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.882 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.882 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.882 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.882 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.882 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.882 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.882 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.882 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.883 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.883 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.883 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.883 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.883 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.883 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.883 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.883 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.883 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.883 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.883 INFO:teuthology.orchestra.run.smithi120.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T09:28:16.883 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T09:28:16.883 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.884 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.884 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.884 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.884 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.884 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.884 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.884 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T09:28:16.884 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.884 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.884 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.884 INFO:teuthology.orchestra.run.smithi120.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T09:28:16.884 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T09:28:16.885 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.885 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.885 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.885 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "*", 2024-09-19T09:28:16.885 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.885 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.885 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.885 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T09:28:16.885 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.885 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.885 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.885 INFO:teuthology.orchestra.run.smithi120.stdout: "max_count_per_host": { 2024-09-19T09:28:16.886 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_count_per_host", 2024-09-19T09:28:16.886 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.886 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.886 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.886 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-19T09:28:16.886 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.886 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.886 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.886 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "max number of daemons per service per host", 2024-09-19T09:28:16.886 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.886 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.886 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.886 INFO:teuthology.orchestra.run.smithi120.stdout: "max_osd_draining_count": { 2024-09-19T09:28:16.887 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_osd_draining_count", 2024-09-19T09:28:16.887 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.887 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.887 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.887 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-19T09:28:16.887 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.887 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.887 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.887 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T09:28:16.887 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.887 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.887 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.887 INFO:teuthology.orchestra.run.smithi120.stdout: "migration_current": { 2024-09-19T09:28:16.888 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "migration_current", 2024-09-19T09:28:16.888 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.888 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.888 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.888 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.888 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.888 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.888 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.888 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "internal - do not modify", 2024-09-19T09:28:16.888 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.888 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.888 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.888 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": { 2024-09-19T09:28:16.889 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mode", 2024-09-19T09:28:16.889 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.889 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.889 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.889 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "root", 2024-09-19T09:28:16.889 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.889 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.889 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.889 INFO:teuthology.orchestra.run.smithi120.stdout: "cephadm-package", 2024-09-19T09:28:16.889 INFO:teuthology.orchestra.run.smithi120.stdout: "root" 2024-09-19T09:28:16.889 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.889 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T09:28:16.889 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.890 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.890 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.890 INFO:teuthology.orchestra.run.smithi120.stdout: "oob_default_addr": { 2024-09-19T09:28:16.890 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "oob_default_addr", 2024-09-19T09:28:16.890 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.890 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.890 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.890 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "169.254.1.1", 2024-09-19T09:28:16.890 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.890 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.890 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.890 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T09:28:16.890 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.890 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.891 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.891 INFO:teuthology.orchestra.run.smithi120.stdout: "prometheus_alerts_path": { 2024-09-19T09:28:16.891 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "prometheus_alerts_path", 2024-09-19T09:28:16.891 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.891 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.891 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.891 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T09:28:16.891 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.891 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.891 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.891 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T09:28:16.891 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.892 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.892 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.892 INFO:teuthology.orchestra.run.smithi120.stdout: "registry_insecure": { 2024-09-19T09:28:16.892 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "registry_insecure", 2024-09-19T09:28:16.892 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.892 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.892 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.892 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.892 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.892 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.892 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.892 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T09:28:16.893 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.893 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.893 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.893 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.893 INFO:teuthology.orchestra.run.smithi120.stdout: "registry_password": { 2024-09-19T09:28:16.893 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "registry_password", 2024-09-19T09:28:16.893 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.893 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.893 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.893 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.893 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.893 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.893 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.893 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T09:28:16.894 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.894 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.894 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.894 INFO:teuthology.orchestra.run.smithi120.stdout: "registry_url": { 2024-09-19T09:28:16.894 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "registry_url", 2024-09-19T09:28:16.894 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.894 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.894 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.894 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.894 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.894 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.894 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.894 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T09:28:16.895 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.895 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.895 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.895 INFO:teuthology.orchestra.run.smithi120.stdout: "registry_username": { 2024-09-19T09:28:16.895 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "registry_username", 2024-09-19T09:28:16.895 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.895 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.895 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.895 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.895 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.895 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.895 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.895 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T09:28:16.896 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.896 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.896 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.896 INFO:teuthology.orchestra.run.smithi120.stdout: "secure_monitoring_stack": { 2024-09-19T09:28:16.896 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "secure_monitoring_stack", 2024-09-19T09:28:16.896 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.896 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.896 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.896 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.896 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.896 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.896 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.896 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T09:28:16.897 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.897 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.897 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.897 INFO:teuthology.orchestra.run.smithi120.stdout: "service_discovery_port": { 2024-09-19T09:28:16.897 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "service_discovery_port", 2024-09-19T09:28:16.897 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.897 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.897 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.897 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8765", 2024-09-19T09:28:16.897 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.897 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.897 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.897 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "cephadm service discovery port", 2024-09-19T09:28:16.898 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.898 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.898 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.898 INFO:teuthology.orchestra.run.smithi120.stdout: "ssh_config_file": { 2024-09-19T09:28:16.898 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ssh_config_file", 2024-09-19T09:28:16.898 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.898 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.898 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.898 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.898 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.898 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.898 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.898 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T09:28:16.899 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.899 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.899 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.899 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.899 INFO:teuthology.orchestra.run.smithi120.stdout: "use_agent": { 2024-09-19T09:28:16.899 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "use_agent", 2024-09-19T09:28:16.899 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.899 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.899 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.899 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.899 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.899 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.899 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.899 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T09:28:16.900 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.900 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.900 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.900 INFO:teuthology.orchestra.run.smithi120.stdout: "use_repo_digest": { 2024-09-19T09:28:16.900 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "use_repo_digest", 2024-09-19T09:28:16.900 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.900 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.900 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.900 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.900 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.900 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.900 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.900 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T09:28:16.901 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.901 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.901 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.901 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_on_failed_host_check": { 2024-09-19T09:28:16.901 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_on_failed_host_check", 2024-09-19T09:28:16.901 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.901 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.901 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.901 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.901 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.901 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.901 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.901 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T09:28:16.902 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.902 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.902 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.902 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_on_stray_daemons": { 2024-09-19T09:28:16.902 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_on_stray_daemons", 2024-09-19T09:28:16.902 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.902 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.902 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.902 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.902 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.902 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.902 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.902 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T09:28:16.903 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.903 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.903 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.903 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_on_stray_hosts": { 2024-09-19T09:28:16.903 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_on_stray_hosts", 2024-09-19T09:28:16.903 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.903 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.903 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.903 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.903 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.903 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.903 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.903 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T09:28:16.904 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.904 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.904 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.904 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.904 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.904 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.904 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.904 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crash", 2024-09-19T09:28:16.904 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.904 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.904 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.905 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.905 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.905 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.905 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.905 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.905 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.905 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.905 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.905 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.905 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.905 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.906 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.906 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.906 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.906 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.906 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.906 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.906 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.906 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.906 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.906 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.906 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.906 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.906 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.907 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.907 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.907 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.907 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.907 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.907 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.907 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.907 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.907 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.907 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.907 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.907 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.907 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.908 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.908 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.908 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.908 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.908 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.908 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.908 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.908 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.908 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.908 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.908 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.908 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.908 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.909 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.909 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.909 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.909 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.909 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.909 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.909 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.909 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.909 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.909 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.909 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.909 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.909 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.910 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.910 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.910 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.910 INFO:teuthology.orchestra.run.smithi120.stdout: "retain_interval": { 2024-09-19T09:28:16.910 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "retain_interval", 2024-09-19T09:28:16.910 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.910 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.910 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.910 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "31536000", 2024-09-19T09:28:16.910 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.910 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.910 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.910 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T09:28:16.911 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.911 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.911 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.911 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_recent_interval": { 2024-09-19T09:28:16.911 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_recent_interval", 2024-09-19T09:28:16.911 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.911 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.911 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.911 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "1209600", 2024-09-19T09:28:16.911 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.911 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.911 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.911 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T09:28:16.912 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.912 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.912 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.912 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.912 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.912 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "dashboard", 2024-09-19T09:28:16.912 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.912 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.912 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.912 INFO:teuthology.orchestra.run.smithi120.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T09:28:16.912 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T09:28:16.912 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.913 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.913 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.913 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-19T09:28:16.913 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.913 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.913 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.913 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.913 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.913 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.913 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.913 INFO:teuthology.orchestra.run.smithi120.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T09:28:16.913 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T09:28:16.913 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.913 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.914 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.914 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.914 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.914 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.914 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.914 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.914 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.914 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.914 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.914 INFO:teuthology.orchestra.run.smithi120.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T09:28:16.914 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T09:28:16.914 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.914 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.915 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.915 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.915 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.915 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.915 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.915 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.915 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.915 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.915 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.915 INFO:teuthology.orchestra.run.smithi120.stdout: "AUDIT_API_ENABLED": { 2024-09-19T09:28:16.915 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T09:28:16.915 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.915 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.916 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.916 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.916 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.916 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.916 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.916 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.916 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.916 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.916 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.916 INFO:teuthology.orchestra.run.smithi120.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T09:28:16.916 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T09:28:16.916 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.916 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.917 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.917 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.917 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.917 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.917 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.917 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.917 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.917 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.917 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.917 INFO:teuthology.orchestra.run.smithi120.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T09:28:16.917 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T09:28:16.917 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.917 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.918 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.918 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.918 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.918 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.918 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.918 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.918 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.918 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.918 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.918 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T09:28:16.918 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T09:28:16.918 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.919 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.919 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.919 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.919 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.919 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.919 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.919 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.919 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.919 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.919 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.919 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T09:28:16.919 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T09:28:16.920 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.920 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.920 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.920 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.920 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.920 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.920 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.920 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.920 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.920 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.920 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.920 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T09:28:16.920 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T09:28:16.921 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.921 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.921 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.921 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.921 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.921 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.921 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.921 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.921 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.921 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.921 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.921 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T09:28:16.922 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T09:28:16.922 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.922 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.922 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.922 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.922 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.922 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.922 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.922 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.922 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.922 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.922 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.923 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T09:28:16.923 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T09:28:16.923 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.923 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.923 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.923 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.923 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.923 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.923 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.923 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.923 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.923 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.923 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.923 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T09:28:16.924 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T09:28:16.924 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.924 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.924 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.924 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.924 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.924 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.924 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.924 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.924 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.924 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.924 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.925 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T09:28:16.925 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T09:28:16.925 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.925 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.925 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.925 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.925 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.925 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.925 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.925 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.925 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.925 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.925 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.926 INFO:teuthology.orchestra.run.smithi120.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T09:28:16.926 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T09:28:16.926 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.926 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.926 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.926 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.926 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.926 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.926 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.926 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.926 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.926 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.926 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.927 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T09:28:16.927 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T09:28:16.927 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.927 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.927 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.927 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "admin", 2024-09-19T09:28:16.927 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.927 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.927 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.927 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.927 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.927 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.927 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.928 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T09:28:16.928 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T09:28:16.928 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.928 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.928 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.928 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.928 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.928 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.928 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.928 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.928 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.928 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.928 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.929 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_API_URL": { 2024-09-19T09:28:16.929 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_API_URL", 2024-09-19T09:28:16.929 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.929 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.929 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.929 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.929 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.929 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.929 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.929 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.929 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.929 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.929 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.929 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T09:28:16.930 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T09:28:16.930 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.930 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.930 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.930 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "admin", 2024-09-19T09:28:16.930 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.930 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.930 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.930 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.930 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.930 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.930 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.930 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T09:28:16.931 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T09:28:16.931 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.931 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.931 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.931 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.931 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.931 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.931 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.931 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.931 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.931 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.931 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.932 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T09:28:16.932 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T09:28:16.932 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.932 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.932 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.932 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.932 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.932 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.932 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.932 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.932 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.932 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.932 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.933 INFO:teuthology.orchestra.run.smithi120.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T09:28:16.933 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T09:28:16.933 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.933 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.933 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.933 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.933 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.933 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.933 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.933 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.933 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.933 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.933 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.934 INFO:teuthology.orchestra.run.smithi120.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T09:28:16.934 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T09:28:16.934 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.934 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.934 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.934 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.934 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.934 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.934 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.934 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.934 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.934 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.934 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.935 INFO:teuthology.orchestra.run.smithi120.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T09:28:16.935 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T09:28:16.935 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.935 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.935 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.935 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.935 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.935 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.935 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.935 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.935 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.935 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.935 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.936 INFO:teuthology.orchestra.run.smithi120.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T09:28:16.936 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T09:28:16.936 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.936 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.936 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.936 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.936 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.936 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.936 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.936 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.936 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.936 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.937 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.937 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T09:28:16.937 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T09:28:16.937 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.937 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.937 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.937 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.937 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.937 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.937 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.937 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.937 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.937 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.938 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.938 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T09:28:16.938 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T09:28:16.938 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.938 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.938 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.938 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.938 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.938 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.938 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.938 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.939 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.939 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.939 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.939 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T09:28:16.939 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T09:28:16.939 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.939 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.939 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.939 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.939 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.940 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.940 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.940 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.940 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.940 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.940 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.940 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T09:28:16.940 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T09:28:16.940 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.940 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.940 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.940 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.940 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.941 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.941 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.941 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.941 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.941 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.941 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.941 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T09:28:16.941 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T09:28:16.941 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.941 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.941 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.941 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.941 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.942 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.942 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.942 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.942 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.942 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.942 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.942 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T09:28:16.942 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T09:28:16.942 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.942 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.942 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.942 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.942 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.943 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.943 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.943 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.943 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.943 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.943 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.943 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T09:28:16.943 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T09:28:16.943 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.943 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.943 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.943 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.943 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.944 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.944 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.944 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.944 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.944 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.944 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.944 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T09:28:16.944 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T09:28:16.944 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.944 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.944 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.944 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.944 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.945 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.945 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.945 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.945 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.945 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.945 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.945 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T09:28:16.945 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T09:28:16.945 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.945 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.945 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.946 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T09:28:16.946 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.946 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.946 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.946 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.946 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.946 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.946 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.946 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T09:28:16.946 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T09:28:16.946 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.946 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.946 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.947 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-19T09:28:16.947 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.947 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.947 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.947 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.947 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.947 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.947 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.947 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T09:28:16.947 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T09:28:16.947 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.947 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.947 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.948 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8", 2024-09-19T09:28:16.948 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.948 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.948 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.948 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.948 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.948 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.948 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.948 INFO:teuthology.orchestra.run.smithi120.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T09:28:16.948 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T09:28:16.948 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.948 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.949 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.949 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "45", 2024-09-19T09:28:16.949 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.949 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.949 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.949 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.949 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.949 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.949 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.949 INFO:teuthology.orchestra.run.smithi120.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T09:28:16.949 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T09:28:16.949 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.949 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.950 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.950 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.950 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.950 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.950 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.950 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.950 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.950 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.950 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.950 INFO:teuthology.orchestra.run.smithi120.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T09:28:16.950 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T09:28:16.950 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.950 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.951 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.951 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "admin", 2024-09-19T09:28:16.951 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.951 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.951 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.951 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.951 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.951 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.951 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.951 INFO:teuthology.orchestra.run.smithi120.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T09:28:16.951 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T09:28:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T09:28:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T09:28:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T09:28:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T09:28:16.954 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.954 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.954 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.954 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-19T09:28:16.954 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.954 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.954 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.954 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.954 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.954 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.954 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.954 INFO:teuthology.orchestra.run.smithi120.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T09:28:16.954 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T09:28:16.955 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.955 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.955 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.955 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-19T09:28:16.955 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.955 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.955 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.955 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.955 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.955 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.955 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.955 INFO:teuthology.orchestra.run.smithi120.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T09:28:16.956 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T09:28:16.956 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.956 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.956 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.956 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-19T09:28:16.956 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.956 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.956 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.956 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.956 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.956 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.956 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.956 INFO:teuthology.orchestra.run.smithi120.stdout: "cross_origin_url": { 2024-09-19T09:28:16.957 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cross_origin_url", 2024-09-19T09:28:16.957 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.957 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.957 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.957 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.957 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.957 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.957 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.957 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.957 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.957 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.957 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.957 INFO:teuthology.orchestra.run.smithi120.stdout: "crt_file": { 2024-09-19T09:28:16.958 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crt_file", 2024-09-19T09:28:16.958 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.958 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.958 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.958 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.958 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.958 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.958 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.958 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.958 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.958 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.958 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.959 INFO:teuthology.orchestra.run.smithi120.stdout: "debug": { 2024-09-19T09:28:16.959 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "debug", 2024-09-19T09:28:16.959 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.959 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.959 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.959 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.959 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.959 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.959 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.959 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Enable/disable debug options", 2024-09-19T09:28:16.959 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.959 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.960 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.960 INFO:teuthology.orchestra.run.smithi120.stdout: "jwt_token_ttl": { 2024-09-19T09:28:16.960 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "jwt_token_ttl", 2024-09-19T09:28:16.960 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.960 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.960 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.960 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "28800", 2024-09-19T09:28:16.960 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.960 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.960 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.960 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.960 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.961 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.961 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.961 INFO:teuthology.orchestra.run.smithi120.stdout: "key_file": { 2024-09-19T09:28:16.961 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "key_file", 2024-09-19T09:28:16.961 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.961 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.961 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.961 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.961 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.961 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.961 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.962 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.962 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.962 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.962 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.962 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.962 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.962 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.962 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.962 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.962 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.962 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.962 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.962 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.963 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.963 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.963 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.963 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.963 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.963 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.963 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.963 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.963 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.963 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.963 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.963 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.963 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.964 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.964 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.964 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.964 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.964 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.964 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.964 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.964 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.964 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.964 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.964 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.964 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.965 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.965 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.965 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.965 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.965 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.965 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.965 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.965 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.965 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.965 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.965 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.965 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.965 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.965 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.966 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.966 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.966 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.966 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.966 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.966 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.966 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.966 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.966 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.966 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.966 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.966 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.967 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.967 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.967 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.967 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.967 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.967 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.967 INFO:teuthology.orchestra.run.smithi120.stdout: "motd": { 2024-09-19T09:28:16.967 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "motd", 2024-09-19T09:28:16.967 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.967 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.967 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.967 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.967 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.968 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.968 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.968 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "The message of the day", 2024-09-19T09:28:16.968 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.968 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.968 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.968 INFO:teuthology.orchestra.run.smithi120.stdout: "redirect_resolve_ip_addr": { 2024-09-19T09:28:16.968 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T09:28:16.968 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.968 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.968 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.968 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.968 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: "server_addr": { 2024-09-19T09:28:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_addr", 2024-09-19T09:28:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "::", 2024-09-19T09:28:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.970 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.970 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.970 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.970 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.970 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.970 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.970 INFO:teuthology.orchestra.run.smithi120.stdout: "server_port": { 2024-09-19T09:28:16.970 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_port", 2024-09-19T09:28:16.970 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.970 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.970 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.970 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8080", 2024-09-19T09:28:16.970 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.971 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.971 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.971 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.971 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.971 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.971 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.971 INFO:teuthology.orchestra.run.smithi120.stdout: "ssl": { 2024-09-19T09:28:16.971 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ssl", 2024-09-19T09:28:16.971 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.971 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.971 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.971 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.971 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.972 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.972 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.972 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.972 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.972 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.972 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.972 INFO:teuthology.orchestra.run.smithi120.stdout: "ssl_server_port": { 2024-09-19T09:28:16.972 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ssl_server_port", 2024-09-19T09:28:16.972 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.972 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.972 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.972 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8443", 2024-09-19T09:28:16.973 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.973 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.973 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.973 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.973 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.973 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.973 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.973 INFO:teuthology.orchestra.run.smithi120.stdout: "standby_behaviour": { 2024-09-19T09:28:16.973 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "standby_behaviour", 2024-09-19T09:28:16.973 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.973 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.973 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.974 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "redirect", 2024-09-19T09:28:16.974 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.974 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.974 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.974 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.974 INFO:teuthology.orchestra.run.smithi120.stdout: "redirect" 2024-09-19T09:28:16.974 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.974 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.974 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.974 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.974 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.974 INFO:teuthology.orchestra.run.smithi120.stdout: "standby_error_status_code": { 2024-09-19T09:28:16.974 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "standby_error_status_code", 2024-09-19T09:28:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "500", 2024-09-19T09:28:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "400", 2024-09-19T09:28:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "599", 2024-09-19T09:28:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: "url_prefix": { 2024-09-19T09:28:16.976 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "url_prefix", 2024-09-19T09:28:16.976 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.976 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.976 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.976 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.976 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.976 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.976 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.976 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.976 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.976 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.976 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.976 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.977 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.977 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "devicehealth", 2024-09-19T09:28:16.977 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.977 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.977 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.977 INFO:teuthology.orchestra.run.smithi120.stdout: "enable_monitoring": { 2024-09-19T09:28:16.977 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "enable_monitoring", 2024-09-19T09:28:16.977 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.977 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.977 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.977 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.978 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.978 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.978 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.978 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "monitor device health metrics", 2024-09-19T09:28:16.978 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.978 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.978 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.978 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.978 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.978 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.978 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.978 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.978 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.979 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.979 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.979 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.979 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.979 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.979 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.979 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.979 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.979 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.979 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.979 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.979 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.979 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.980 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.980 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.980 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.980 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.980 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.980 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.980 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.980 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.980 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.980 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.980 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.980 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.982 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.982 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.982 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.982 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.982 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.982 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.982 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.982 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.982 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.982 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.982 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.982 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.982 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.983 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.983 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.983 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.983 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.983 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.983 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.983 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.983 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.983 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.983 INFO:teuthology.orchestra.run.smithi120.stdout: "mark_out_threshold": { 2024-09-19T09:28:16.983 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mark_out_threshold", 2024-09-19T09:28:16.983 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.984 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.984 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.984 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "2419200", 2024-09-19T09:28:16.984 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.984 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.984 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.984 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T09:28:16.984 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.984 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.984 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.984 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": { 2024-09-19T09:28:16.984 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "pool_name", 2024-09-19T09:28:16.984 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "device_health_metrics", 2024-09-19T09:28:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T09:28:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: "retention_period": { 2024-09-19T09:28:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "retention_period", 2024-09-19T09:28:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.986 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.986 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.986 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "15552000", 2024-09-19T09:28:16.986 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.986 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.986 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.986 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how long to retain device health metrics", 2024-09-19T09:28:16.986 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.986 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.986 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.986 INFO:teuthology.orchestra.run.smithi120.stdout: "scrape_frequency": { 2024-09-19T09:28:16.986 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "scrape_frequency", 2024-09-19T09:28:16.986 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.987 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.987 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.987 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "86400", 2024-09-19T09:28:16.987 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.987 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.987 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.987 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T09:28:16.987 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.987 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.987 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.987 INFO:teuthology.orchestra.run.smithi120.stdout: "self_heal": { 2024-09-19T09:28:16.987 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "self_heal", 2024-09-19T09:28:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T09:28:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-19T09:28:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-19T09:28:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-19T09:28:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T09:28:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_threshold": { 2024-09-19T09:28:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_threshold", 2024-09-19T09:28:16.990 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:16.990 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.990 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.990 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "7257600", 2024-09-19T09:28:16.990 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.990 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.990 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.990 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T09:28:16.990 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.990 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.990 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.990 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.990 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "diskprediction_local", 2024-09-19T09:28:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.992 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.992 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.992 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.992 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.992 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.992 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.992 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.992 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.992 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.992 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.992 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.992 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:16.992 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:16.993 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.993 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.993 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.993 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.993 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.993 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.993 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.993 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.993 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.993 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.993 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.993 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:16.994 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:16.994 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.994 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.994 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.994 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:16.994 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.994 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.994 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:16.994 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:16.994 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:16.994 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:16.994 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:16.994 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:16.995 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:16.995 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:16.995 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.995 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.995 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.995 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.995 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:16.995 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:16.995 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:16.995 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.995 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:16.995 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: "predict_interval": { 2024-09-19T09:28:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "predict_interval", 2024-09-19T09:28:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "86400", 2024-09-19T09:28:16.997 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.997 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.997 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.997 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.997 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.997 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.997 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.997 INFO:teuthology.orchestra.run.smithi120.stdout: "predictor_model": { 2024-09-19T09:28:16.997 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "predictor_model", 2024-09-19T09:28:16.997 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.997 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.997 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.997 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "prophetstor", 2024-09-19T09:28:16.998 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.998 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.998 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.998 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.998 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.998 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.998 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.998 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-19T09:28:16.998 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-19T09:28:16.998 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:16.998 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:16.998 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:16.998 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-19T09:28:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "influx", 2024-09-19T09:28:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": false, 2024-09-19T09:28:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "influxdb python module not found", 2024-09-19T09:28:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: "batch_size": { 2024-09-19T09:28:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "batch_size", 2024-09-19T09:28:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5000", 2024-09-19T09:28:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T09:28:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.001 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.001 INFO:teuthology.orchestra.run.smithi120.stdout: "database": { 2024-09-19T09:28:17.001 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "database", 2024-09-19T09:28:17.001 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.001 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.001 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.001 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "ceph", 2024-09-19T09:28:17.001 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.001 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.001 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.001 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T09:28:17.001 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.001 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.002 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.002 INFO:teuthology.orchestra.run.smithi120.stdout: "hostname": { 2024-09-19T09:28:17.002 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "hostname", 2024-09-19T09:28:17.002 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.002 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.002 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.002 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.002 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.002 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.002 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.002 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "InfluxDB server hostname", 2024-09-19T09:28:17.002 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.002 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.003 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.003 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-19T09:28:17.003 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-19T09:28:17.003 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:17.003 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.003 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.003 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "30", 2024-09-19T09:28:17.003 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "5", 2024-09-19T09:28:17.003 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.003 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.003 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T09:28:17.003 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.003 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.004 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.004 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.004 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.004 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.004 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.004 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.004 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.004 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.004 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.004 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.004 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.004 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.004 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.005 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.005 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.005 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.005 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.005 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.005 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.005 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.005 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.006 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.006 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.006 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.006 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.006 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.006 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.006 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.006 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.006 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.006 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.006 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.006 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.006 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.007 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.007 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.007 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.007 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.007 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.007 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.007 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.007 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.007 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.007 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.007 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.007 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.007 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.007 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.008 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.008 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.008 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.008 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.008 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.008 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.008 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.008 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.008 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.008 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.008 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.008 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.008 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: "password": { 2024-09-19T09:28:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "password", 2024-09-19T09:28:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.010 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.010 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.010 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "password of InfluxDB server user", 2024-09-19T09:28:17.010 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.010 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.010 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.010 INFO:teuthology.orchestra.run.smithi120.stdout: "port": { 2024-09-19T09:28:17.010 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "port", 2024-09-19T09:28:17.010 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.010 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.010 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.010 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8086", 2024-09-19T09:28:17.010 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "InfluxDB server port", 2024-09-19T09:28:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: "ssl": { 2024-09-19T09:28:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ssl", 2024-09-19T09:28:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "false", 2024-09-19T09:28:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T09:28:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: "threads": { 2024-09-19T09:28:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "threads", 2024-09-19T09:28:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-19T09:28:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1", 2024-09-19T09:28:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "32", 2024-09-19T09:28:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T09:28:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: "username": { 2024-09-19T09:28:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "username", 2024-09-19T09:28:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.014 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.014 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "username of InfluxDB server user", 2024-09-19T09:28:17.014 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.014 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.014 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.014 INFO:teuthology.orchestra.run.smithi120.stdout: "verify_ssl": { 2024-09-19T09:28:17.014 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "verify_ssl", 2024-09-19T09:28:17.014 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.014 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.014 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.014 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "true", 2024-09-19T09:28:17.014 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.015 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.015 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.015 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T09:28:17.015 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.015 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.015 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.015 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.015 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.015 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "insights", 2024-09-19T09:28:17.015 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.015 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.015 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "iostat", 2024-09-19T09:28:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "k8sevents", 2024-09-19T09:28:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_event_retention_days": { 2024-09-19T09:28:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ceph_event_retention_days", 2024-09-19T09:28:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "7", 2024-09-19T09:28:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T09:28:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "config_check_secs": { 2024-09-19T09:28:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "config_check_secs", 2024-09-19T09:28:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-19T09:28:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "10", 2024-09-19T09:28:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T09:28:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "localpool", 2024-09-19T09:28:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "failure_domain": { 2024-09-19T09:28:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "failure_domain", 2024-09-19T09:28:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "host", 2024-09-19T09:28:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "failure domain for any created local pool", 2024-09-19T09:28:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T09:28:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.036 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.036 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.036 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.036 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.036 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.036 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.036 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.036 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.036 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.036 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.036 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.036 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.036 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.036 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.037 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.037 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.037 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.037 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.037 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.037 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.037 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.037 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.037 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.037 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.037 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.037 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.037 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.038 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.038 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.038 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.038 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.038 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.038 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.038 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.038 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.038 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.038 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.038 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.038 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.038 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.039 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.039 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.039 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.039 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.039 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.039 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.039 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.039 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.039 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.040 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.040 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.040 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.040 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.040 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.040 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.040 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.040 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.040 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.040 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.040 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.040 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.040 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": { 2024-09-19T09:28:17.041 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "min_size", 2024-09-19T09:28:17.041 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.041 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.041 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.041 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.041 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.041 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.041 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.041 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "default min_size for any created local pool", 2024-09-19T09:28:17.041 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T09:28:17.041 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.041 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.041 INFO:teuthology.orchestra.run.smithi120.stdout: "num_rep": { 2024-09-19T09:28:17.042 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "num_rep", 2024-09-19T09:28:17.042 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.042 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.042 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.042 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "3", 2024-09-19T09:28:17.042 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.042 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.042 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.042 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "default replica count for any created local pool", 2024-09-19T09:28:17.042 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.043 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.043 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.043 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": { 2024-09-19T09:28:17.043 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "pg_num", 2024-09-19T09:28:17.043 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.043 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.043 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.043 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "128", 2024-09-19T09:28:17.043 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.043 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.043 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.043 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T09:28:17.043 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.044 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.044 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.044 INFO:teuthology.orchestra.run.smithi120.stdout: "prefix": { 2024-09-19T09:28:17.044 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "prefix", 2024-09-19T09:28:17.044 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.044 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.044 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.044 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.044 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.044 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.044 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.044 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "name prefix for any created local pool", 2024-09-19T09:28:17.045 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.045 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.045 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.045 INFO:teuthology.orchestra.run.smithi120.stdout: "subtree": { 2024-09-19T09:28:17.045 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "subtree", 2024-09-19T09:28:17.045 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.045 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.045 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.045 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "rack", 2024-09-19T09:28:17.045 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.045 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.045 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.045 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T09:28:17.046 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T09:28:17.046 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.046 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.046 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.046 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.046 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mds_autoscaler", 2024-09-19T09:28:17.046 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.046 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.046 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.046 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.046 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.047 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.047 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.047 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.047 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.047 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.047 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.047 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.047 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.047 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.047 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.047 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.047 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.047 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.048 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.048 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.048 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.048 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.048 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.048 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.048 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.048 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.048 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.048 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.048 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.048 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.049 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.049 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.049 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.049 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.049 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.049 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.049 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.049 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.049 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.049 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.049 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.049 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.049 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.050 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.050 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.050 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.050 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.050 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.050 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.050 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.050 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.050 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.050 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.050 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.050 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.050 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.051 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.051 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.051 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.051 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.051 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.051 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.051 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.051 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.051 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.051 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.051 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.051 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.051 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.051 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.052 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.052 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mirroring", 2024-09-19T09:28:17.052 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.052 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.052 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.052 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.052 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.052 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.052 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.052 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.052 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.053 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.053 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.053 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.053 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.053 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.053 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.053 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.053 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.053 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.053 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.053 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.053 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.053 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.053 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.054 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.054 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.054 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.054 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.054 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.054 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.054 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.054 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.054 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.054 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.054 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.054 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.055 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.055 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.055 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.055 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.055 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.055 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.055 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.055 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.055 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.055 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.055 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.055 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.055 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.055 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.056 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.056 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.056 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.056 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.056 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.056 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.056 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.056 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.056 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.056 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.057 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.057 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.057 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.057 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.057 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.057 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.057 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.057 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.057 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.057 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.057 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.057 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.058 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "nfs", 2024-09-19T09:28:17.058 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.058 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.058 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.058 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.058 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.058 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.058 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.058 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.058 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.058 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.058 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.058 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.058 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.059 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.059 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.059 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.059 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.059 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.059 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.059 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.059 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.059 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.059 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.059 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.059 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.059 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.060 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.060 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.060 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.060 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.060 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.060 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.060 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.060 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.060 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.060 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.060 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.060 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.060 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.061 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.061 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.061 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.061 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.061 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.061 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.061 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.061 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.061 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.061 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.061 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.061 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.061 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.061 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.062 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.062 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.062 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.062 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.062 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.062 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.062 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.062 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.062 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.062 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.062 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.062 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.062 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.063 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.063 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.063 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.063 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.063 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.063 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "orchestrator", 2024-09-19T09:28:17.063 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.063 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.063 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.063 INFO:teuthology.orchestra.run.smithi120.stdout: "fail_fs": { 2024-09-19T09:28:17.063 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "fail_fs", 2024-09-19T09:28:17.063 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.063 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.064 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.064 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.064 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.064 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.064 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.064 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T09:28:17.064 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.064 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.064 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.064 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.064 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.064 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.064 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.065 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.065 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.065 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.065 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.065 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.065 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.065 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.065 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.065 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.065 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.065 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.065 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.065 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.065 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.066 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.066 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.066 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.066 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.066 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.066 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.066 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.066 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.066 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.066 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.066 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.066 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.066 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.067 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.067 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.067 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.067 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.067 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.067 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.067 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.067 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.067 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.067 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.067 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.067 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.067 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.068 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.068 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.068 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.068 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.068 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.068 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.068 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.068 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.068 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.068 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.068 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.068 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.068 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.069 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.069 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.069 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.069 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.069 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.069 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.069 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.069 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.069 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.069 INFO:teuthology.orchestra.run.smithi120.stdout: "orchestrator": { 2024-09-19T09:28:17.069 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "orchestrator", 2024-09-19T09:28:17.069 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.069 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.070 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.070 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.070 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.070 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.070 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.070 INFO:teuthology.orchestra.run.smithi120.stdout: "cephadm", 2024-09-19T09:28:17.070 INFO:teuthology.orchestra.run.smithi120.stdout: "rook", 2024-09-19T09:28:17.070 INFO:teuthology.orchestra.run.smithi120.stdout: "test_orchestrator" 2024-09-19T09:28:17.070 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.070 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Orchestrator backend", 2024-09-19T09:28:17.070 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.070 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.070 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.071 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.071 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.071 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "osd_perf_query", 2024-09-19T09:28:17.071 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.071 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.071 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.071 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.071 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.071 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.071 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.071 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.071 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.071 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.072 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.072 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.072 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.072 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.072 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.072 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.072 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.072 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.072 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.072 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.072 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.072 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.072 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.073 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.073 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.073 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.073 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.073 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.073 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.073 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.073 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.073 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.073 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.073 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.073 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.073 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.074 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.074 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.074 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.074 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.074 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.074 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.074 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.074 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.074 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.074 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.074 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.074 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.074 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.075 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.075 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.075 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.075 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.075 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.075 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.075 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.075 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.075 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.075 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.075 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.075 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.075 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.076 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.076 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.076 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.076 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.076 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.076 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.076 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.076 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.076 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.076 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "osd_support", 2024-09-19T09:28:17.076 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.076 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.077 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.077 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.077 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.077 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.077 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.077 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.077 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.077 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.077 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.077 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.077 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.077 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.077 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.077 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.078 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.078 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.078 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.078 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.078 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.078 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.078 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.078 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.078 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.078 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.078 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.078 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.078 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.079 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.079 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.079 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.079 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.079 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.079 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.079 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.079 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.079 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.079 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.079 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.079 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.079 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.080 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.080 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.080 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.080 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.080 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.080 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.080 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.080 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.080 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.080 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.080 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.080 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.080 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.080 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.081 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.081 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.081 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.081 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.081 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.081 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.081 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.081 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.081 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.081 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.081 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.081 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.081 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.081 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.082 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.082 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "pg_autoscaler", 2024-09-19T09:28:17.082 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.082 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.082 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.082 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.082 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.082 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.082 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.082 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.082 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.082 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.083 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.083 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.083 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.083 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.083 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.083 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.083 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.083 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.083 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.083 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.083 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.083 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.083 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.084 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.084 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.084 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.084 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.084 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.084 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.084 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.084 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.084 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.084 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.084 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.084 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.084 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.085 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.085 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.085 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.085 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.085 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.085 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.085 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.085 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.085 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.085 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.085 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.085 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.086 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.086 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.086 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.086 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.086 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.086 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.086 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.086 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.086 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.086 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.086 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.086 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.086 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.087 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.087 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.087 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.087 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.087 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.087 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.087 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.087 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.087 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-19T09:28:17.087 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-19T09:28:17.087 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:17.087 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.087 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.088 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-19T09:28:17.088 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.088 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.088 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.088 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.088 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.088 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.088 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.088 INFO:teuthology.orchestra.run.smithi120.stdout: "threshold": { 2024-09-19T09:28:17.088 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "threshold", 2024-09-19T09:28:17.088 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-19T09:28:17.088 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.089 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.089 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "3.0", 2024-09-19T09:28:17.089 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1.0", 2024-09-19T09:28:17.089 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.089 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.089 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "scaling threshold", 2024-09-19T09:28:17.089 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T09:28:17.089 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.089 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.089 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.089 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.089 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "progress", 2024-09-19T09:28:17.090 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.090 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.090 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.090 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_pg_recovery_event": { 2024-09-19T09:28:17.090 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "allow_pg_recovery_event", 2024-09-19T09:28:17.090 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.090 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.090 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.090 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.090 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.090 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.090 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.090 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T09:28:17.090 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.091 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.091 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.091 INFO:teuthology.orchestra.run.smithi120.stdout: "enabled": { 2024-09-19T09:28:17.091 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "enabled", 2024-09-19T09:28:17.091 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.091 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.091 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.091 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.091 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.091 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.091 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.091 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.091 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.092 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.092 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.092 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.092 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.092 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.092 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.092 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.092 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.092 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.092 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.092 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.092 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.092 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.093 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.093 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.093 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.093 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.093 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.093 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.093 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.093 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.093 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.093 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.093 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.093 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.093 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.094 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.094 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.094 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.094 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.094 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.094 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.094 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.094 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.094 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.094 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.094 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.094 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.094 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.095 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.095 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.095 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.095 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.095 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.095 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.095 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.095 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.095 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.095 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.095 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.095 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.095 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.095 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.095 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.096 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.096 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.096 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.096 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.096 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.096 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.096 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.096 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.096 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.096 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.096 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.096 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.097 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.097 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.097 INFO:teuthology.orchestra.run.smithi120.stdout: "max_completed_events": { 2024-09-19T09:28:17.097 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_completed_events", 2024-09-19T09:28:17.097 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.097 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.097 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.097 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "50", 2024-09-19T09:28:17.097 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.097 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.097 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.097 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "number of past completed events to remember", 2024-09-19T09:28:17.097 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.097 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.098 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.098 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-19T09:28:17.098 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-19T09:28:17.098 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:17.098 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.098 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.098 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-19T09:28:17.098 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.098 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.098 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.098 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how long the module is going to sleep", 2024-09-19T09:28:17.098 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.099 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.099 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.099 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.099 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.099 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "prometheus", 2024-09-19T09:28:17.099 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.099 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.099 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.099 INFO:teuthology.orchestra.run.smithi120.stdout: "cache": { 2024-09-19T09:28:17.099 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cache", 2024-09-19T09:28:17.099 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.099 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.100 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.100 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.100 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.100 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.100 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.100 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.100 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.100 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.100 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.100 INFO:teuthology.orchestra.run.smithi120.stdout: "exclude_perf_counters": { 2024-09-19T09:28:17.100 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "exclude_perf_counters", 2024-09-19T09:28:17.100 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.100 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.101 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.101 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.101 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.101 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.101 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.101 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T09:28:17.101 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T09:28:17.101 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.101 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.101 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.101 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.101 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.102 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.102 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.102 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.102 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.102 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.102 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.102 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.102 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.102 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.102 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.102 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.102 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.102 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.102 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.103 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.103 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.103 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.103 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.103 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.103 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.103 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.103 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.103 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.103 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.103 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.103 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.103 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.104 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.104 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.104 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.104 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.104 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.104 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.104 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.104 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.104 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.104 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.104 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.104 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.104 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.105 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.105 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.105 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.105 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.105 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.105 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.105 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.105 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.105 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.106 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.106 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.106 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.106 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.106 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.106 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.106 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.106 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.106 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.106 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.106 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.106 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.106 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.107 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.107 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_stats_pools": { 2024-09-19T09:28:17.107 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rbd_stats_pools", 2024-09-19T09:28:17.107 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.107 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.107 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.107 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.107 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.107 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.107 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.107 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.107 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.107 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.108 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.108 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T09:28:17.108 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T09:28:17.108 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.108 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.108 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.108 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "300", 2024-09-19T09:28:17.108 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.108 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.108 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.108 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.108 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.109 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.109 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.109 INFO:teuthology.orchestra.run.smithi120.stdout: "scrape_interval": { 2024-09-19T09:28:17.109 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "scrape_interval", 2024-09-19T09:28:17.109 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-19T09:28:17.109 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.109 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.109 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "15.0", 2024-09-19T09:28:17.109 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.109 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.109 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.109 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.109 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.110 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.110 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.110 INFO:teuthology.orchestra.run.smithi120.stdout: "server_addr": { 2024-09-19T09:28:17.110 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_addr", 2024-09-19T09:28:17.110 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.110 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.110 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.110 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "::", 2024-09-19T09:28:17.110 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.110 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.110 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.110 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T09:28:17.110 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.111 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.111 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.111 INFO:teuthology.orchestra.run.smithi120.stdout: "server_port": { 2024-09-19T09:28:17.111 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_port", 2024-09-19T09:28:17.111 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.111 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.111 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.111 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "9283", 2024-09-19T09:28:17.111 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.111 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.111 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.111 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T09:28:17.111 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.112 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.112 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.112 INFO:teuthology.orchestra.run.smithi120.stdout: "stale_cache_strategy": { 2024-09-19T09:28:17.112 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "stale_cache_strategy", 2024-09-19T09:28:17.112 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.112 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.112 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.112 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "log", 2024-09-19T09:28:17.112 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.112 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.112 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.112 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.113 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.113 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.113 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.113 INFO:teuthology.orchestra.run.smithi120.stdout: "standby_behaviour": { 2024-09-19T09:28:17.113 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "standby_behaviour", 2024-09-19T09:28:17.113 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.113 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.113 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.113 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "default", 2024-09-19T09:28:17.113 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.113 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.113 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.114 INFO:teuthology.orchestra.run.smithi120.stdout: "default", 2024-09-19T09:28:17.114 INFO:teuthology.orchestra.run.smithi120.stdout: "error" 2024-09-19T09:28:17.114 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.114 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.114 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.114 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.114 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.114 INFO:teuthology.orchestra.run.smithi120.stdout: "standby_error_status_code": { 2024-09-19T09:28:17.114 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "standby_error_status_code", 2024-09-19T09:28:17.114 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.114 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.114 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.114 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "500", 2024-09-19T09:28:17.115 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "400", 2024-09-19T09:28:17.115 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "599", 2024-09-19T09:28:17.115 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.115 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.115 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.115 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.115 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.115 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.115 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.115 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rbd_support", 2024-09-19T09:28:17.115 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.115 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.116 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.116 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.116 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.116 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.116 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.116 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.116 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.116 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.116 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.116 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.116 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.116 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.116 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.116 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.117 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.117 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.117 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.117 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.117 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.117 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.117 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.117 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.117 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.117 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.117 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.117 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.117 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.118 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.118 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.118 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.118 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.118 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.118 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.118 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.118 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.118 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.118 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.118 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.118 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.118 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.119 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.119 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.119 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.119 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.119 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.119 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.119 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.119 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.119 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.119 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.119 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.119 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.119 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.119 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.120 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.120 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.120 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.120 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.120 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.120 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.120 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.120 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.120 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.120 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.120 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.120 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.120 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.121 INFO:teuthology.orchestra.run.smithi120.stdout: "max_concurrent_snap_create": { 2024-09-19T09:28:17.121 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_concurrent_snap_create", 2024-09-19T09:28:17.121 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.121 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.121 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.121 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-19T09:28:17.121 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.121 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.121 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.121 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.121 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.121 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.121 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.122 INFO:teuthology.orchestra.run.smithi120.stdout: "mirror_snapshot_schedule": { 2024-09-19T09:28:17.122 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T09:28:17.122 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.122 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.122 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.122 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.122 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.122 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.122 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.122 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.122 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.122 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.122 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.123 INFO:teuthology.orchestra.run.smithi120.stdout: "trash_purge_schedule": { 2024-09-19T09:28:17.123 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "trash_purge_schedule", 2024-09-19T09:28:17.123 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.123 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.123 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.123 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.123 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.123 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.123 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.123 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.123 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.123 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.123 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.123 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.124 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.124 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "restful", 2024-09-19T09:28:17.124 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.124 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.124 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.124 INFO:teuthology.orchestra.run.smithi120.stdout: "enable_auth": { 2024-09-19T09:28:17.124 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "enable_auth", 2024-09-19T09:28:17.124 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.124 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.124 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.124 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.124 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.125 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.125 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.125 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.125 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.125 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.125 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.125 INFO:teuthology.orchestra.run.smithi120.stdout: "key_file": { 2024-09-19T09:28:17.125 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "key_file", 2024-09-19T09:28:17.125 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.125 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.125 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.125 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.125 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.126 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.126 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.126 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.126 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.126 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.126 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.126 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.126 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.126 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.126 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.126 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.126 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.127 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.127 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.127 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.127 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.127 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.127 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.127 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.127 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.127 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.127 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.127 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.127 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.127 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.127 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.128 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.128 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.128 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.128 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.128 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.128 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.128 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.128 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.128 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.128 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.128 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.128 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.128 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.129 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.129 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.129 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.129 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.129 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.129 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.129 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.129 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.129 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.129 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.129 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.129 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.129 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.130 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.130 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.130 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.130 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.130 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.130 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.130 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.130 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.130 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.130 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.130 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.130 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.130 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.131 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.131 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.131 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.131 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.131 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.131 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.131 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.131 INFO:teuthology.orchestra.run.smithi120.stdout: "server_addr": { 2024-09-19T09:28:17.131 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_addr", 2024-09-19T09:28:17.131 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.131 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.131 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.131 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.132 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.132 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.132 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.132 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.132 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.132 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.132 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.132 INFO:teuthology.orchestra.run.smithi120.stdout: "server_port": { 2024-09-19T09:28:17.132 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_port", 2024-09-19T09:28:17.132 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.132 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.132 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.132 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.133 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.133 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.133 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.133 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.133 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.133 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.133 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.133 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.133 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.133 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rgw", 2024-09-19T09:28:17.133 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.133 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.133 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.134 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.134 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.134 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.134 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.134 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.134 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.134 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.134 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.134 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.134 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.134 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.134 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.134 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.135 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.135 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.135 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.135 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.135 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.135 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.135 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.135 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.135 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.135 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.135 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.135 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.135 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.135 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.136 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.136 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.136 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.136 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.136 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.136 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.136 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.136 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.136 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.136 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.136 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.136 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.137 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.137 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.137 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.137 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.137 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.137 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.137 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.137 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.137 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.137 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.137 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.137 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.137 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.137 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.138 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.138 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.138 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.138 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.138 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.138 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.138 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.138 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.138 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.138 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.138 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.138 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.138 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.139 INFO:teuthology.orchestra.run.smithi120.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T09:28:17.139 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T09:28:17.139 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.139 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.139 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.139 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-19T09:28:17.139 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.139 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.139 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.139 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T09:28:17.139 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.139 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.139 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.140 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.140 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.140 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rook", 2024-09-19T09:28:17.140 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.140 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.140 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.140 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.140 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.140 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.140 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.140 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.141 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.141 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.141 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.141 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.141 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.141 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.141 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.141 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.141 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.141 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.141 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.141 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.141 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.141 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.141 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.142 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.142 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.142 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.142 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.142 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.142 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.142 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.142 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.142 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.142 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.142 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.143 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.143 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.143 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.143 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.143 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.143 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.143 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.143 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.143 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.143 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.143 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.143 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.143 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.143 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.144 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.144 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.144 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.144 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.144 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.144 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.144 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.144 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.144 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.144 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.144 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.144 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.145 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.145 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.145 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.145 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.145 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.145 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.145 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.145 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.145 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.145 INFO:teuthology.orchestra.run.smithi120.stdout: "storage_class": { 2024-09-19T09:28:17.145 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "storage_class", 2024-09-19T09:28:17.145 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.145 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.146 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.146 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "local", 2024-09-19T09:28:17.146 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.146 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.146 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.146 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T09:28:17.146 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.146 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.146 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.146 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.146 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.146 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "selftest", 2024-09-19T09:28:17.147 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.147 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.147 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.147 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.147 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.147 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.147 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.147 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.147 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.147 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.147 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.147 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.147 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.147 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.148 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.148 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.148 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.148 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.148 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.148 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.148 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.148 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.148 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.148 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.148 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.148 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.148 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.149 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.149 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.149 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.149 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.149 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.149 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.149 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.149 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.149 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.149 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.149 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.149 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.149 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.150 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.150 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.150 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.150 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.150 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.150 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.150 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.150 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.150 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.150 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.150 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.150 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.150 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.150 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.151 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.151 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.151 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.151 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.151 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.151 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.151 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.151 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.151 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.151 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.151 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.151 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.151 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.152 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.152 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.152 INFO:teuthology.orchestra.run.smithi120.stdout: "roption1": { 2024-09-19T09:28:17.152 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "roption1", 2024-09-19T09:28:17.152 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.152 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.152 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.152 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.152 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.152 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.152 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.152 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.152 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.153 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.153 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.153 INFO:teuthology.orchestra.run.smithi120.stdout: "roption2": { 2024-09-19T09:28:17.153 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "roption2", 2024-09-19T09:28:17.153 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.153 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.153 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.153 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "xyz", 2024-09-19T09:28:17.153 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.153 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.153 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.153 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.153 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.154 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.154 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.154 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption1": { 2024-09-19T09:28:17.154 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption1", 2024-09-19T09:28:17.154 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.154 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.154 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.154 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.154 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.154 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.154 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.154 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.154 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.155 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.155 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.155 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption2": { 2024-09-19T09:28:17.155 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption2", 2024-09-19T09:28:17.155 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.155 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.155 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.155 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.155 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.155 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.155 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.155 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.155 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.156 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.156 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.156 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption3": { 2024-09-19T09:28:17.156 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption3", 2024-09-19T09:28:17.156 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-19T09:28:17.156 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.156 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.156 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.156 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.156 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.156 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.156 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.157 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.157 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.157 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.157 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption4": { 2024-09-19T09:28:17.157 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption4", 2024-09-19T09:28:17.157 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.157 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.157 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.157 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.157 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.157 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.157 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.157 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.158 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.158 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.158 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.158 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption5": { 2024-09-19T09:28:17.158 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption5", 2024-09-19T09:28:17.158 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.158 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.158 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.158 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.158 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.158 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.158 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.159 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.159 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.159 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.159 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.159 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption6": { 2024-09-19T09:28:17.159 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption6", 2024-09-19T09:28:17.159 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.159 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.159 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.159 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.159 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.159 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.159 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.160 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.160 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.160 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.160 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.160 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption7": { 2024-09-19T09:28:17.160 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption7", 2024-09-19T09:28:17.160 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.160 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.160 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.160 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.160 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1", 2024-09-19T09:28:17.160 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "42", 2024-09-19T09:28:17.160 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.161 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.161 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.161 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.161 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.161 INFO:teuthology.orchestra.run.smithi120.stdout: "testkey": { 2024-09-19T09:28:17.161 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "testkey", 2024-09-19T09:28:17.161 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.161 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.161 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.161 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.161 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.161 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.161 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.162 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.162 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.162 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.162 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.162 INFO:teuthology.orchestra.run.smithi120.stdout: "testlkey": { 2024-09-19T09:28:17.162 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "testlkey", 2024-09-19T09:28:17.162 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.162 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.162 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.162 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.162 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.162 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.162 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.162 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.163 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.163 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.163 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.163 INFO:teuthology.orchestra.run.smithi120.stdout: "testnewline": { 2024-09-19T09:28:17.163 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "testnewline", 2024-09-19T09:28:17.163 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.163 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.163 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.163 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.163 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.163 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.163 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.163 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.164 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.164 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.164 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.164 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.164 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.164 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "snap_schedule", 2024-09-19T09:28:17.164 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.164 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.164 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.164 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_m_granularity": { 2024-09-19T09:28:17.164 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "allow_m_granularity", 2024-09-19T09:28:17.164 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.165 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.165 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.165 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.165 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.165 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.165 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.165 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T09:28:17.165 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.165 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.165 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.165 INFO:teuthology.orchestra.run.smithi120.stdout: "dump_on_update": { 2024-09-19T09:28:17.165 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "dump_on_update", 2024-09-19T09:28:17.165 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.166 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.166 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.166 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.166 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.166 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.166 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.166 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "dump database to debug log on update", 2024-09-19T09:28:17.166 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.166 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.166 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.166 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.166 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.166 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.167 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.167 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.167 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.167 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.167 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.167 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.167 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.167 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.167 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.167 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.167 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.167 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.167 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.167 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.168 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.168 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.168 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.168 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.168 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.168 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.168 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.168 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.168 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.168 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.168 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.168 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.169 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.169 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.169 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.169 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.169 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.169 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.169 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.169 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.169 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.169 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.169 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.169 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.169 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.170 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.170 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.170 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.170 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.170 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.170 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.170 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.170 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.170 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.170 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.170 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.170 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.170 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.171 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.171 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.171 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.171 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.171 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.171 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.171 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.171 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.171 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.171 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.171 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.171 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.171 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.172 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "stats", 2024-09-19T09:28:17.172 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.172 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.172 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.172 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.172 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.172 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.172 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.172 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.172 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.172 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.172 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.173 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.173 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.173 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.173 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.173 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.173 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.173 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.173 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.173 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.173 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.173 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.173 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.173 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.173 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.174 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.174 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.174 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.174 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.174 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.174 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.174 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.174 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.174 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.174 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.174 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.174 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.174 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.175 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.175 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.175 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.175 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.175 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.175 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.175 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.175 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.175 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.175 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.175 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.175 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.175 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.175 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.176 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.176 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.176 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.176 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.176 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.176 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.176 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.176 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.176 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.176 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.176 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.176 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.176 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.177 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.177 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.177 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.177 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.177 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.177 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.177 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "status", 2024-09-19T09:28:17.177 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.177 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.177 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.177 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.177 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.178 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.178 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.178 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.178 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.178 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.178 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.178 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.178 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.178 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.178 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.178 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.178 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.178 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.178 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.179 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.179 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.179 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.179 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.179 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.179 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.179 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.179 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.179 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.179 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.179 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.179 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.179 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.180 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.180 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.180 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.180 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.180 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.180 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.180 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.180 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.180 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.180 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.180 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.180 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.180 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.181 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.181 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.181 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.181 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.181 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.181 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.181 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.181 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.181 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.181 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.181 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.181 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.181 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.182 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.182 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.182 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.182 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.182 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.182 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.182 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.182 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.182 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.182 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.182 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.182 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.182 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.183 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "telegraf", 2024-09-19T09:28:17.183 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.183 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.183 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.183 INFO:teuthology.orchestra.run.smithi120.stdout: "address": { 2024-09-19T09:28:17.183 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "address", 2024-09-19T09:28:17.183 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.183 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.183 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.183 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T09:28:17.183 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.183 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.184 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.184 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.184 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.184 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.184 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.184 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-19T09:28:17.184 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-19T09:28:17.184 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:17.184 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.184 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.184 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "15", 2024-09-19T09:28:17.184 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.184 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.185 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.185 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.185 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.185 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.185 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.185 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.185 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.185 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.185 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.185 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.185 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.185 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.186 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.186 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.186 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.186 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.186 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.186 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.186 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.186 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.186 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.186 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.186 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.186 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.186 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.186 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.187 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.187 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.187 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.187 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.187 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.187 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.187 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.187 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.187 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.187 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.187 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.187 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.188 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.188 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.188 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.188 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.188 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.188 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.188 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.188 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.188 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.188 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.188 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.188 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.188 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.188 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.189 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.189 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.189 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.189 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.189 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.189 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.189 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.189 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.189 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.189 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.189 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.189 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.189 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.189 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.190 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.190 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.190 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.190 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.190 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.190 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.190 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.190 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "telemetry", 2024-09-19T09:28:17.190 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.190 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.190 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.190 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_basic": { 2024-09-19T09:28:17.191 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_basic", 2024-09-19T09:28:17.191 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.191 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.191 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.191 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.191 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.191 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.191 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.191 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T09:28:17.191 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.191 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.191 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.191 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_crash": { 2024-09-19T09:28:17.192 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_crash", 2024-09-19T09:28:17.192 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.192 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.192 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.192 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.192 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.192 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.192 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.192 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T09:28:17.192 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.192 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.192 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.192 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_device": { 2024-09-19T09:28:17.193 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_device", 2024-09-19T09:28:17.193 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.193 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.193 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.193 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.193 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.193 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.193 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.193 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T09:28:17.193 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.193 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.193 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.193 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_ident": { 2024-09-19T09:28:17.194 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_ident", 2024-09-19T09:28:17.194 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.194 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.194 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.194 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.194 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.194 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.194 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.194 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T09:28:17.194 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.194 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.194 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.195 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_perf": { 2024-09-19T09:28:17.195 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_perf", 2024-09-19T09:28:17.195 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.195 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.195 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.195 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.195 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.195 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.195 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.195 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T09:28:17.195 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.195 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.196 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.196 INFO:teuthology.orchestra.run.smithi120.stdout: "contact": { 2024-09-19T09:28:17.196 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "contact", 2024-09-19T09:28:17.196 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.196 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.196 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.196 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.196 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.196 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.196 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.196 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.196 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.196 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.197 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.197 INFO:teuthology.orchestra.run.smithi120.stdout: "description": { 2024-09-19T09:28:17.197 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "description", 2024-09-19T09:28:17.197 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.197 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.197 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.197 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.197 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.197 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.197 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.197 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.197 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.197 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.198 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.198 INFO:teuthology.orchestra.run.smithi120.stdout: "device_url": { 2024-09-19T09:28:17.198 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "device_url", 2024-09-19T09:28:17.198 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.198 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.198 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.198 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T09:28:17.198 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.198 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.198 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.198 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.198 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.198 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.199 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.199 INFO:teuthology.orchestra.run.smithi120.stdout: "enabled": { 2024-09-19T09:28:17.199 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "enabled", 2024-09-19T09:28:17.199 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.199 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.199 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.199 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.199 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.199 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.199 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.199 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.199 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.199 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.200 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.200 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-19T09:28:17.200 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-19T09:28:17.200 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.200 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.200 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.200 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "24", 2024-09-19T09:28:17.200 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "8", 2024-09-19T09:28:17.200 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.200 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.200 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.200 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.201 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.201 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.201 INFO:teuthology.orchestra.run.smithi120.stdout: "last_opt_revision": { 2024-09-19T09:28:17.201 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "last_opt_revision", 2024-09-19T09:28:17.201 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.201 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.201 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.201 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "1", 2024-09-19T09:28:17.201 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.201 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.201 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.201 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.201 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.202 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.202 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.202 INFO:teuthology.orchestra.run.smithi120.stdout: "leaderboard": { 2024-09-19T09:28:17.202 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "leaderboard", 2024-09-19T09:28:17.202 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.202 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.202 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.202 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.202 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.202 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.202 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.202 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.202 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.203 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.203 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.203 INFO:teuthology.orchestra.run.smithi120.stdout: "leaderboard_description": { 2024-09-19T09:28:17.203 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "leaderboard_description", 2024-09-19T09:28:17.203 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.203 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.203 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.203 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.203 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.203 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.203 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.203 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.203 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.203 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.204 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.204 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.204 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.204 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.204 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.204 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.204 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.204 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.204 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.204 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.204 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.204 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.205 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.205 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.205 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.205 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.205 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.205 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.205 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.205 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.205 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.205 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.205 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.206 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.206 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.206 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.206 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.206 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.206 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.206 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.206 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.206 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.206 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.206 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.207 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.207 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.207 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.207 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.207 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.207 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.207 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.207 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.207 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.207 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.207 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.207 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.207 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.207 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.208 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.208 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.208 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.208 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.208 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.208 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.208 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.208 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.208 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.208 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.208 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.208 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.208 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.209 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.209 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.209 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.209 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.209 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.209 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.209 INFO:teuthology.orchestra.run.smithi120.stdout: "organization": { 2024-09-19T09:28:17.209 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "organization", 2024-09-19T09:28:17.209 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.209 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.209 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.209 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.210 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.210 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.210 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.210 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.210 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.210 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.210 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.210 INFO:teuthology.orchestra.run.smithi120.stdout: "proxy": { 2024-09-19T09:28:17.210 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "proxy", 2024-09-19T09:28:17.210 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.210 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.210 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.210 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.211 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.211 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.211 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.211 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.211 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.211 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.211 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.211 INFO:teuthology.orchestra.run.smithi120.stdout: "url": { 2024-09-19T09:28:17.211 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "url", 2024-09-19T09:28:17.211 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.211 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.211 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.211 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T09:28:17.212 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.212 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.212 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.212 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.212 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.212 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.212 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.212 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.212 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.212 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "test_orchestrator", 2024-09-19T09:28:17.212 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.212 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.213 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.213 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.213 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.213 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.213 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.213 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.213 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.213 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.213 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.213 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.213 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.213 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.213 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.213 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.214 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.214 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.214 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.214 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.214 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.214 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.214 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.214 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.214 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.214 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.214 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.214 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.214 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.215 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.215 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.215 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.215 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.215 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.215 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.215 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.215 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.215 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.215 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.215 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.215 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.215 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.216 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.216 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.216 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.216 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.216 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.216 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.216 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.216 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.216 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.216 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.216 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.216 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.216 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.216 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.217 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.217 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.217 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.217 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.217 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.217 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.217 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.217 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.217 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.217 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.217 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.217 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.217 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.218 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.218 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.218 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "volumes", 2024-09-19T09:28:17.218 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.218 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.218 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.218 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.218 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.218 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.218 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.218 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.218 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.219 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.219 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.219 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.219 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.219 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.219 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.219 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.219 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.219 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.219 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.219 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.219 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.219 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.219 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.220 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.220 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.220 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.220 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.220 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.220 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.220 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.220 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.220 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.220 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.220 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.220 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.220 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.221 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.221 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.221 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.221 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.221 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.221 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.221 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.221 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.221 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.221 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.221 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.221 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.221 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.222 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.222 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.222 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.222 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.222 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.222 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.222 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.222 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.222 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.222 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.222 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.222 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.222 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.223 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.223 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.223 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.223 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.223 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.223 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.223 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.223 INFO:teuthology.orchestra.run.smithi120.stdout: "max_concurrent_clones": { 2024-09-19T09:28:17.223 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_concurrent_clones", 2024-09-19T09:28:17.223 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.223 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.223 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.223 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "4", 2024-09-19T09:28:17.224 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.224 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.224 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.224 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T09:28:17.224 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.224 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.224 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.224 INFO:teuthology.orchestra.run.smithi120.stdout: "periodic_async_work": { 2024-09-19T09:28:17.224 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "periodic_async_work", 2024-09-19T09:28:17.224 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.224 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.224 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.225 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.225 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.225 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.225 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.225 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Periodically check for async work", 2024-09-19T09:28:17.225 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.225 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.225 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.225 INFO:teuthology.orchestra.run.smithi120.stdout: "snapshot_clone_delay": { 2024-09-19T09:28:17.225 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "snapshot_clone_delay", 2024-09-19T09:28:17.225 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.225 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.225 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.226 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-19T09:28:17.226 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.226 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.226 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.226 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T09:28:17.226 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.226 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.226 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.226 INFO:teuthology.orchestra.run.smithi120.stdout: "snapshot_clone_no_wait": { 2024-09-19T09:28:17.226 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T09:28:17.226 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.226 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.227 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.227 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.227 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.227 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.227 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.227 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T09:28:17.227 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.227 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.227 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.227 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.227 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.227 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "zabbix", 2024-09-19T09:28:17.228 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.228 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.228 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.228 INFO:teuthology.orchestra.run.smithi120.stdout: "discovery_interval": { 2024-09-19T09:28:17.228 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "discovery_interval", 2024-09-19T09:28:17.228 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-19T09:28:17.228 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.228 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.228 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "100", 2024-09-19T09:28:17.228 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.228 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.228 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.228 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.229 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.229 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.229 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.229 INFO:teuthology.orchestra.run.smithi120.stdout: "identifier": { 2024-09-19T09:28:17.229 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "identifier", 2024-09-19T09:28:17.229 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.229 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.229 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.229 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.229 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.229 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.229 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.229 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.230 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.230 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.230 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.230 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-19T09:28:17.230 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-19T09:28:17.230 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:17.230 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.230 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.230 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-19T09:28:17.230 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.230 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.230 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.230 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.231 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.231 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.231 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.231 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.231 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.231 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.231 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.231 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.231 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.231 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.231 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.231 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.231 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.232 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.232 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.232 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.232 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.232 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.232 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.232 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.232 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.232 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.232 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.232 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.232 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.232 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.233 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.233 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.233 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.233 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.233 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.233 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.233 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.233 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.233 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.233 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.233 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.233 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.233 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.234 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.234 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.234 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.234 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.234 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.234 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.234 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.234 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.234 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.234 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.234 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.234 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.234 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.234 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.234 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.235 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.235 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.235 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.235 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.235 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.235 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.235 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.235 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.235 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.235 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.235 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.235 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.235 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.236 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.236 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.236 INFO:teuthology.orchestra.run.smithi120.stdout: "zabbix_host": { 2024-09-19T09:28:17.236 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "zabbix_host", 2024-09-19T09:28:17.236 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.236 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.236 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.236 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.236 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.236 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.236 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.236 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.236 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.237 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.237 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.237 INFO:teuthology.orchestra.run.smithi120.stdout: "zabbix_port": { 2024-09-19T09:28:17.237 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "zabbix_port", 2024-09-19T09:28:17.237 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.237 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.237 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.237 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10051", 2024-09-19T09:28:17.237 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.237 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.237 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.237 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.238 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.238 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.238 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.238 INFO:teuthology.orchestra.run.smithi120.stdout: "zabbix_sender": { 2024-09-19T09:28:17.238 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "zabbix_sender", 2024-09-19T09:28:17.238 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.238 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.238 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.238 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T09:28:17.238 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.238 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.238 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.238 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.239 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.239 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.239 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.239 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.239 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.239 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.239 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-19T09:28:17.239 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.239 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.239 INFO:teuthology.orchestra.run.smithi120.stdout: "modules": [ 2024-09-19T09:28:17.239 INFO:teuthology.orchestra.run.smithi120.stdout: "cephadm", 2024-09-19T09:28:17.239 INFO:teuthology.orchestra.run.smithi120.stdout: "dashboard", 2024-09-19T09:28:17.239 INFO:teuthology.orchestra.run.smithi120.stdout: "iostat", 2024-09-19T09:28:17.239 INFO:teuthology.orchestra.run.smithi120.stdout: "nfs", 2024-09-19T09:28:17.240 INFO:teuthology.orchestra.run.smithi120.stdout: "restful" 2024-09-19T09:28:17.240 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.240 INFO:teuthology.orchestra.run.smithi120.stdout: "available_modules": [ 2024-09-19T09:28:17.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.240 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "alerts", 2024-09-19T09:28:17.240 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.240 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.240 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.240 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-19T09:28:17.240 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-19T09:28:17.240 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:17.240 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.240 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.240 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-19T09:28:17.241 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.241 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.241 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.241 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T09:28:17.241 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.241 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.241 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.241 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.241 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.241 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.241 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.241 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.241 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.242 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.242 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.242 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.242 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.242 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.242 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.242 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.242 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.242 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.242 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.242 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.242 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.242 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.242 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.243 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.243 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.243 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.243 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.243 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.243 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.243 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.243 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.243 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.243 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.243 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.243 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.243 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.244 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.244 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.244 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.244 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.244 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.244 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.244 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.244 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.244 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.244 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.244 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.244 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.245 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.245 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.245 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.245 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.245 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.245 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.245 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.245 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.245 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.245 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.245 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.245 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.245 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.245 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.246 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.246 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.246 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.246 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.246 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.246 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.246 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.246 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_destination": { 2024-09-19T09:28:17.246 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_destination", 2024-09-19T09:28:17.246 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.246 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.246 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.246 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.247 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.247 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.247 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.247 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Email address to send alerts to", 2024-09-19T09:28:17.247 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.247 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.247 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.247 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_from_name": { 2024-09-19T09:28:17.247 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_from_name", 2024-09-19T09:28:17.247 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.247 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.247 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.247 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "Ceph", 2024-09-19T09:28:17.248 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.248 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.248 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.248 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Email From: name", 2024-09-19T09:28:17.248 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.248 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.248 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.248 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_host": { 2024-09-19T09:28:17.248 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_host", 2024-09-19T09:28:17.248 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.248 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.248 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.248 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.249 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.249 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.249 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.249 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "SMTP server", 2024-09-19T09:28:17.249 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.249 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.249 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.249 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_password": { 2024-09-19T09:28:17.249 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_password", 2024-09-19T09:28:17.249 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.249 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.249 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.249 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.250 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.250 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.250 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.250 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Password to authenticate with", 2024-09-19T09:28:17.250 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.250 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.250 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.250 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_port": { 2024-09-19T09:28:17.250 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_port", 2024-09-19T09:28:17.250 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.250 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.250 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.250 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "465", 2024-09-19T09:28:17.251 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.251 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.251 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.251 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "SMTP port", 2024-09-19T09:28:17.251 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.251 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.251 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.251 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_sender": { 2024-09-19T09:28:17.251 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_sender", 2024-09-19T09:28:17.251 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.251 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.251 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.251 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.252 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.252 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.252 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.252 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "SMTP envelope sender", 2024-09-19T09:28:17.252 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.252 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.252 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.252 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_ssl": { 2024-09-19T09:28:17.252 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_ssl", 2024-09-19T09:28:17.252 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.252 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.252 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.252 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.253 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.253 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.253 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.253 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T09:28:17.253 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.253 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.253 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.253 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_user": { 2024-09-19T09:28:17.253 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_user", 2024-09-19T09:28:17.253 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.253 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.253 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.253 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.254 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.254 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.254 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.254 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "User to authenticate as", 2024-09-19T09:28:17.254 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.254 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.254 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.254 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.254 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.254 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "balancer", 2024-09-19T09:28:17.254 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.255 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.255 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.255 INFO:teuthology.orchestra.run.smithi120.stdout: "active": { 2024-09-19T09:28:17.255 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "active", 2024-09-19T09:28:17.255 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.255 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.255 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.255 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.255 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.255 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.255 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.255 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T09:28:17.255 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.255 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.256 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.256 INFO:teuthology.orchestra.run.smithi120.stdout: "begin_time": { 2024-09-19T09:28:17.256 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "begin_time", 2024-09-19T09:28:17.256 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.256 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.256 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.256 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0000", 2024-09-19T09:28:17.256 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.256 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.256 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.256 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T09:28:17.256 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T09:28:17.256 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.257 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.257 INFO:teuthology.orchestra.run.smithi120.stdout: "begin_weekday": { 2024-09-19T09:28:17.257 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "begin_weekday", 2024-09-19T09:28:17.257 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-19T09:28:17.257 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.257 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.257 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-19T09:28:17.257 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "0", 2024-09-19T09:28:17.257 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "6", 2024-09-19T09:28:17.257 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.257 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T09:28:17.257 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T09:28:17.257 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.258 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.258 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_compat_max_iterations": { 2024-09-19T09:28:17.258 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crush_compat_max_iterations", 2024-09-19T09:28:17.258 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-19T09:28:17.258 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.258 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.258 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "25", 2024-09-19T09:28:17.258 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1", 2024-09-19T09:28:17.258 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "250", 2024-09-19T09:28:17.258 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.258 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T09:28:17.258 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.258 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.259 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.259 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_compat_metrics": { 2024-09-19T09:28:17.259 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crush_compat_metrics", 2024-09-19T09:28:17.259 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.259 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.259 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.259 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T09:28:17.259 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.259 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.259 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.259 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T09:28:17.259 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T09:28:17.259 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.260 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.260 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_compat_step": { 2024-09-19T09:28:17.260 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crush_compat_step", 2024-09-19T09:28:17.260 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-19T09:28:17.260 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.260 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.260 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0.5", 2024-09-19T09:28:17.260 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "0.001", 2024-09-19T09:28:17.260 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "0.999", 2024-09-19T09:28:17.260 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.260 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "aggressiveness of optimization", 2024-09-19T09:28:17.260 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T09:28:17.260 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.261 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.261 INFO:teuthology.orchestra.run.smithi120.stdout: "end_time": { 2024-09-19T09:28:17.261 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "end_time", 2024-09-19T09:28:17.261 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.261 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.261 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.261 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "2359", 2024-09-19T09:28:17.261 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.261 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.261 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.261 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T09:28:17.261 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T09:28:17.261 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.262 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.262 INFO:teuthology.orchestra.run.smithi120.stdout: "end_weekday": { 2024-09-19T09:28:17.262 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "end_weekday", 2024-09-19T09:28:17.262 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-19T09:28:17.262 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.262 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.262 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-19T09:28:17.262 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "0", 2024-09-19T09:28:17.262 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "6", 2024-09-19T09:28:17.262 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.262 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T09:28:17.262 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T09:28:17.262 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.263 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.263 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.263 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.263 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.263 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.263 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.263 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.263 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.263 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.263 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.263 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.263 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.263 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.263 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.264 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.264 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.264 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.264 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.264 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.264 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.264 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.264 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.264 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.264 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.264 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.264 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.264 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.265 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.265 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.265 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.265 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.265 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.265 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.265 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.265 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.265 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.265 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.265 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.265 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.265 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.266 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.266 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.266 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.266 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.266 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.266 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.266 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.266 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.266 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.266 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.266 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.266 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.266 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.266 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.267 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.267 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.267 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.267 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.267 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.267 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.267 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.267 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.267 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.267 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.267 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.267 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.267 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.268 INFO:teuthology.orchestra.run.smithi120.stdout: "min_score": { 2024-09-19T09:28:17.268 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "min_score", 2024-09-19T09:28:17.268 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-19T09:28:17.268 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.268 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.268 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-19T09:28:17.268 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.268 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.268 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.268 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T09:28:17.268 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.268 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.268 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.269 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": { 2024-09-19T09:28:17.269 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mode", 2024-09-19T09:28:17.269 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.269 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.269 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.269 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "upmap", 2024-09-19T09:28:17.269 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.269 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.269 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.269 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-compat", 2024-09-19T09:28:17.269 INFO:teuthology.orchestra.run.smithi120.stdout: "none", 2024-09-19T09:28:17.269 INFO:teuthology.orchestra.run.smithi120.stdout: "read", 2024-09-19T09:28:17.269 INFO:teuthology.orchestra.run.smithi120.stdout: "upmap", 2024-09-19T09:28:17.270 INFO:teuthology.orchestra.run.smithi120.stdout: "upmap-read" 2024-09-19T09:28:17.270 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.270 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Balancer mode", 2024-09-19T09:28:17.270 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.270 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.270 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.270 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_ids": { 2024-09-19T09:28:17.270 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "pool_ids", 2024-09-19T09:28:17.270 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.270 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.270 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.270 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.270 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.271 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.271 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.271 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T09:28:17.271 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.271 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.271 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.271 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-19T09:28:17.271 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-19T09:28:17.271 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:17.271 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.271 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.271 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-19T09:28:17.271 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.272 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.272 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.272 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T09:28:17.272 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.272 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.272 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.272 INFO:teuthology.orchestra.run.smithi120.stdout: "upmap_max_deviation": { 2024-09-19T09:28:17.272 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "upmap_max_deviation", 2024-09-19T09:28:17.272 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.272 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.272 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.272 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-19T09:28:17.272 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1", 2024-09-19T09:28:17.273 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.273 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.273 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T09:28:17.273 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T09:28:17.273 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.273 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.273 INFO:teuthology.orchestra.run.smithi120.stdout: "upmap_max_optimizations": { 2024-09-19T09:28:17.273 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "upmap_max_optimizations", 2024-09-19T09:28:17.273 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-19T09:28:17.273 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.273 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.273 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-19T09:28:17.273 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.274 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.274 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.274 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T09:28:17.274 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.274 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.274 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.274 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.274 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.274 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cephadm", 2024-09-19T09:28:17.274 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.274 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.274 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.275 INFO:teuthology.orchestra.run.smithi120.stdout: "agent_down_multiplier": { 2024-09-19T09:28:17.275 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "agent_down_multiplier", 2024-09-19T09:28:17.275 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-19T09:28:17.275 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.275 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.275 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "3.0", 2024-09-19T09:28:17.275 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.275 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.275 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.275 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T09:28:17.275 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.275 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.275 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.276 INFO:teuthology.orchestra.run.smithi120.stdout: "agent_refresh_rate": { 2024-09-19T09:28:17.276 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "agent_refresh_rate", 2024-09-19T09:28:17.276 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:17.276 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.276 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.276 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "20", 2024-09-19T09:28:17.276 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.276 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.276 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.276 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T09:28:17.276 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.276 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.276 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.277 INFO:teuthology.orchestra.run.smithi120.stdout: "agent_starting_port": { 2024-09-19T09:28:17.277 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "agent_starting_port", 2024-09-19T09:28:17.277 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.277 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.277 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.277 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "4721", 2024-09-19T09:28:17.277 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.277 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.277 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.277 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T09:28:17.277 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.277 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.277 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.278 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_ptrace": { 2024-09-19T09:28:17.278 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "allow_ptrace", 2024-09-19T09:28:17.278 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.278 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.278 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.278 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.278 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.278 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.278 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.278 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T09:28:17.278 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T09:28:17.278 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.278 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.279 INFO:teuthology.orchestra.run.smithi120.stdout: "autotune_interval": { 2024-09-19T09:28:17.279 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "autotune_interval", 2024-09-19T09:28:17.279 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:17.279 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.279 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.279 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-19T09:28:17.279 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.279 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.279 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.279 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T09:28:17.279 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.279 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.279 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.279 INFO:teuthology.orchestra.run.smithi120.stdout: "autotune_memory_target_ratio": { 2024-09-19T09:28:17.280 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T09:28:17.280 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-19T09:28:17.280 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.280 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.280 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0.7", 2024-09-19T09:28:17.280 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.280 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.280 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.280 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T09:28:17.280 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.280 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.280 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.280 INFO:teuthology.orchestra.run.smithi120.stdout: "cephadm_log_destination": { 2024-09-19T09:28:17.281 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cephadm_log_destination", 2024-09-19T09:28:17.281 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.281 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.281 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.281 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.281 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.281 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.281 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.281 INFO:teuthology.orchestra.run.smithi120.stdout: "file", 2024-09-19T09:28:17.281 INFO:teuthology.orchestra.run.smithi120.stdout: "file,syslog", 2024-09-19T09:28:17.281 INFO:teuthology.orchestra.run.smithi120.stdout: "syslog" 2024-09-19T09:28:17.281 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.281 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T09:28:17.281 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.282 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.282 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.282 INFO:teuthology.orchestra.run.smithi120.stdout: "cgroups_split": { 2024-09-19T09:28:17.282 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cgroups_split", 2024-09-19T09:28:17.282 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.282 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.282 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.282 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.282 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.282 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.282 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.282 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T09:28:17.283 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.283 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.283 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.283 INFO:teuthology.orchestra.run.smithi120.stdout: "config_checks_enabled": { 2024-09-19T09:28:17.283 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "config_checks_enabled", 2024-09-19T09:28:17.283 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.283 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.283 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.283 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.283 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.283 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.283 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.283 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T09:28:17.283 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.284 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.284 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.284 INFO:teuthology.orchestra.run.smithi120.stdout: "config_dashboard": { 2024-09-19T09:28:17.284 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "config_dashboard", 2024-09-19T09:28:17.284 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.284 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.284 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.284 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.284 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.284 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.284 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.285 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T09:28:17.285 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.285 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.285 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.285 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_alertmanager": { 2024-09-19T09:28:17.285 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_alertmanager", 2024-09-19T09:28:17.285 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.285 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.285 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.285 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T09:28:17.285 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.285 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.285 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.286 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Prometheus container image", 2024-09-19T09:28:17.286 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.286 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.286 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.286 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_base": { 2024-09-19T09:28:17.286 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_base", 2024-09-19T09:28:17.286 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.286 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.286 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.286 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T09:28:17.286 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.286 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.286 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.287 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Container image name, without the tag", 2024-09-19T09:28:17.287 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.287 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.287 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.287 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_elasticsearch": { 2024-09-19T09:28:17.287 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_elasticsearch", 2024-09-19T09:28:17.287 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.287 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.287 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.287 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T09:28:17.287 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.287 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.287 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.288 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "elasticsearch container image", 2024-09-19T09:28:17.288 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.288 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.288 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.288 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_grafana": { 2024-09-19T09:28:17.288 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_grafana", 2024-09-19T09:28:17.288 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.288 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.288 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.288 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T09:28:17.288 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.288 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.288 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.289 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Prometheus container image", 2024-09-19T09:28:17.289 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.289 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.289 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.289 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_haproxy": { 2024-09-19T09:28:17.289 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_haproxy", 2024-09-19T09:28:17.289 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.289 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.289 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.289 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T09:28:17.289 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.289 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.290 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.290 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "HAproxy container image", 2024-09-19T09:28:17.290 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.290 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.290 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.290 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_jaeger_agent": { 2024-09-19T09:28:17.290 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_jaeger_agent", 2024-09-19T09:28:17.290 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.290 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.290 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.290 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T09:28:17.290 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.290 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.291 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.291 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Jaeger agent container image", 2024-09-19T09:28:17.291 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.291 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.291 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.291 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_jaeger_collector": { 2024-09-19T09:28:17.291 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_jaeger_collector", 2024-09-19T09:28:17.291 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.291 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.291 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.291 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T09:28:17.291 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.291 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.292 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.292 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Jaeger collector container image", 2024-09-19T09:28:17.292 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.292 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.292 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.292 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_jaeger_query": { 2024-09-19T09:28:17.292 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_jaeger_query", 2024-09-19T09:28:17.292 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.292 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.292 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.292 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T09:28:17.292 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.292 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.292 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.293 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Jaeger query container image", 2024-09-19T09:28:17.293 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.293 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.293 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.293 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_keepalived": { 2024-09-19T09:28:17.293 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_keepalived", 2024-09-19T09:28:17.293 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.293 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.293 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.293 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T09:28:17.293 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.293 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.293 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.294 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Keepalived container image", 2024-09-19T09:28:17.294 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.294 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.294 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.294 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_loki": { 2024-09-19T09:28:17.294 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_loki", 2024-09-19T09:28:17.294 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.294 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.294 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.294 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T09:28:17.294 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.294 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.294 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.295 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Loki container image", 2024-09-19T09:28:17.295 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.295 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.295 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.295 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_node_exporter": { 2024-09-19T09:28:17.295 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_node_exporter", 2024-09-19T09:28:17.295 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.295 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.295 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.295 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T09:28:17.295 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.295 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.295 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.295 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Prometheus container image", 2024-09-19T09:28:17.296 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.296 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.296 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.296 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_nvmeof": { 2024-09-19T09:28:17.296 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_nvmeof", 2024-09-19T09:28:17.296 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.296 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.296 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.296 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T09:28:17.296 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.296 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.296 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.296 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Nvme-of container image", 2024-09-19T09:28:17.297 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.297 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.297 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.297 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_prometheus": { 2024-09-19T09:28:17.297 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_prometheus", 2024-09-19T09:28:17.297 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.297 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.297 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.297 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T09:28:17.297 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.297 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.297 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.297 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Prometheus container image", 2024-09-19T09:28:17.298 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.298 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.298 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.298 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_promtail": { 2024-09-19T09:28:17.298 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_promtail", 2024-09-19T09:28:17.298 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.298 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.298 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.298 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T09:28:17.298 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.298 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.298 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.299 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Promtail container image", 2024-09-19T09:28:17.299 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.299 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.299 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.299 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_samba": { 2024-09-19T09:28:17.299 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_samba", 2024-09-19T09:28:17.299 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.299 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.299 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.299 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T09:28:17.299 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.299 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.299 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.299 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Samba/SMB container image", 2024-09-19T09:28:17.300 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.300 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.300 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.300 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_snmp_gateway": { 2024-09-19T09:28:17.300 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_snmp_gateway", 2024-09-19T09:28:17.300 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.300 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.300 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.300 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T09:28:17.300 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.300 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.300 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.301 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "SNMP Gateway container image", 2024-09-19T09:28:17.301 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.301 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.301 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.301 INFO:teuthology.orchestra.run.smithi120.stdout: "container_init": { 2024-09-19T09:28:17.301 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_init", 2024-09-19T09:28:17.301 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.301 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.301 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.301 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.301 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.301 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.301 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.301 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T09:28:17.302 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.302 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.302 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.302 INFO:teuthology.orchestra.run.smithi120.stdout: "daemon_cache_timeout": { 2024-09-19T09:28:17.302 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "daemon_cache_timeout", 2024-09-19T09:28:17.302 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:17.302 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.302 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.302 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-19T09:28:17.302 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.302 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.302 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.302 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T09:28:17.303 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.303 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.303 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.303 INFO:teuthology.orchestra.run.smithi120.stdout: "default_cephadm_command_timeout": { 2024-09-19T09:28:17.303 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T09:28:17.303 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.303 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.303 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.303 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "900", 2024-09-19T09:28:17.303 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.303 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.303 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.303 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T09:28:17.304 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.304 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.304 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.304 INFO:teuthology.orchestra.run.smithi120.stdout: "default_registry": { 2024-09-19T09:28:17.304 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "default_registry", 2024-09-19T09:28:17.304 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.304 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.304 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.304 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "docker.io", 2024-09-19T09:28:17.304 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.304 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.304 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.305 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T09:28:17.305 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.305 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.305 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.305 INFO:teuthology.orchestra.run.smithi120.stdout: "device_cache_timeout": { 2024-09-19T09:28:17.305 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "device_cache_timeout", 2024-09-19T09:28:17.305 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:17.305 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.305 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.305 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "1800", 2024-09-19T09:28:17.305 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.305 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.305 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.305 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "seconds to cache device inventory", 2024-09-19T09:28:17.306 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.306 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.306 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.306 INFO:teuthology.orchestra.run.smithi120.stdout: "device_enhanced_scan": { 2024-09-19T09:28:17.306 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "device_enhanced_scan", 2024-09-19T09:28:17.306 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.306 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.306 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.307 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.307 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.307 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.307 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.307 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T09:28:17.307 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.307 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.307 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.307 INFO:teuthology.orchestra.run.smithi120.stdout: "facts_cache_timeout": { 2024-09-19T09:28:17.307 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "facts_cache_timeout", 2024-09-19T09:28:17.307 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:17.307 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.307 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.308 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-19T09:28:17.308 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.308 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.308 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.308 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "seconds to cache host facts data", 2024-09-19T09:28:17.308 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.308 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.308 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.308 INFO:teuthology.orchestra.run.smithi120.stdout: "grafana_dashboards_path": { 2024-09-19T09:28:17.308 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "grafana_dashboards_path", 2024-09-19T09:28:17.308 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.308 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.308 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.309 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T09:28:17.309 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.309 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.309 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.309 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T09:28:17.309 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.309 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.309 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.309 INFO:teuthology.orchestra.run.smithi120.stdout: "host_check_interval": { 2024-09-19T09:28:17.309 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "host_check_interval", 2024-09-19T09:28:17.309 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:17.309 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.310 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.310 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-19T09:28:17.310 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.310 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.310 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.310 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to perform a host check", 2024-09-19T09:28:17.310 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.310 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.310 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.310 INFO:teuthology.orchestra.run.smithi120.stdout: "hw_monitoring": { 2024-09-19T09:28:17.310 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "hw_monitoring", 2024-09-19T09:28:17.310 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.310 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.310 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.311 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.311 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.311 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.311 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.311 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T09:28:17.311 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.311 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.311 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.311 INFO:teuthology.orchestra.run.smithi120.stdout: "inventory_list_all": { 2024-09-19T09:28:17.311 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "inventory_list_all", 2024-09-19T09:28:17.311 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.311 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.311 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.312 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.312 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.312 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.312 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.312 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T09:28:17.312 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.312 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.312 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.312 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.312 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.312 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.312 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.312 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.313 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.313 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.313 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.313 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.313 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.313 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.313 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.313 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.313 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.313 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.313 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.313 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.313 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.313 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.314 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.314 INFO:teuthology.orchestra.run.smithi120.stdout: "log_refresh_metadata": { 2024-09-19T09:28:17.314 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_refresh_metadata", 2024-09-19T09:28:17.314 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.314 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.314 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.314 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.314 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.314 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.314 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.314 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T09:28:17.314 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.315 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.315 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.315 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.315 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.315 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.315 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.315 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.315 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.315 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.315 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.315 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.315 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T09:28:17.315 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.316 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.316 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.316 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.316 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.316 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.316 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.316 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.316 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.316 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.316 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.316 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.316 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.316 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.317 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.317 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.317 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.317 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.317 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.317 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.317 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.317 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.317 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.317 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.317 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.317 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.317 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.318 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.318 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.318 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.318 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.318 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.318 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.318 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.318 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.318 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.318 INFO:teuthology.orchestra.run.smithi120.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T09:28:17.318 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T09:28:17.318 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.318 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.319 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.319 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.319 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.319 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.319 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.319 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T09:28:17.319 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.319 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.319 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.319 INFO:teuthology.orchestra.run.smithi120.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T09:28:17.319 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T09:28:17.319 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.319 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.320 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.320 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "*", 2024-09-19T09:28:17.320 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.320 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.320 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.320 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T09:28:17.320 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.320 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.320 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.320 INFO:teuthology.orchestra.run.smithi120.stdout: "max_count_per_host": { 2024-09-19T09:28:17.320 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_count_per_host", 2024-09-19T09:28:17.320 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.321 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.321 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.321 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-19T09:28:17.321 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.321 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.321 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.321 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "max number of daemons per service per host", 2024-09-19T09:28:17.321 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.321 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.321 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.321 INFO:teuthology.orchestra.run.smithi120.stdout: "max_osd_draining_count": { 2024-09-19T09:28:17.321 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_osd_draining_count", 2024-09-19T09:28:17.321 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.321 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.322 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.322 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-19T09:28:17.322 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.322 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.322 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.322 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T09:28:17.322 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.322 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.322 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.322 INFO:teuthology.orchestra.run.smithi120.stdout: "migration_current": { 2024-09-19T09:28:17.322 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "migration_current", 2024-09-19T09:28:17.322 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.322 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.323 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.323 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.323 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.323 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.323 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.323 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "internal - do not modify", 2024-09-19T09:28:17.323 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.323 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.323 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.323 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": { 2024-09-19T09:28:17.323 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mode", 2024-09-19T09:28:17.323 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.323 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.324 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.324 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "root", 2024-09-19T09:28:17.324 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.324 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.324 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.324 INFO:teuthology.orchestra.run.smithi120.stdout: "cephadm-package", 2024-09-19T09:28:17.324 INFO:teuthology.orchestra.run.smithi120.stdout: "root" 2024-09-19T09:28:17.324 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.324 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T09:28:17.324 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.324 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.324 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.324 INFO:teuthology.orchestra.run.smithi120.stdout: "oob_default_addr": { 2024-09-19T09:28:17.325 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "oob_default_addr", 2024-09-19T09:28:17.325 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.325 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.325 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.325 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "169.254.1.1", 2024-09-19T09:28:17.325 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.325 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.325 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.325 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T09:28:17.325 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.325 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.325 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.325 INFO:teuthology.orchestra.run.smithi120.stdout: "prometheus_alerts_path": { 2024-09-19T09:28:17.326 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "prometheus_alerts_path", 2024-09-19T09:28:17.326 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.326 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.326 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.326 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T09:28:17.326 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.326 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.326 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.326 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T09:28:17.326 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.326 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.326 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.326 INFO:teuthology.orchestra.run.smithi120.stdout: "registry_insecure": { 2024-09-19T09:28:17.327 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "registry_insecure", 2024-09-19T09:28:17.327 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.327 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.327 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.327 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.327 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.327 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.327 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.327 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T09:28:17.327 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.327 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.327 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.327 INFO:teuthology.orchestra.run.smithi120.stdout: "registry_password": { 2024-09-19T09:28:17.327 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "registry_password", 2024-09-19T09:28:17.328 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.328 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.328 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.328 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.328 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.328 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.328 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.328 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T09:28:17.328 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.328 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.328 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.328 INFO:teuthology.orchestra.run.smithi120.stdout: "registry_url": { 2024-09-19T09:28:17.329 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "registry_url", 2024-09-19T09:28:17.329 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.329 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.329 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.329 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.329 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.329 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.329 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.329 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T09:28:17.329 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.329 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.329 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.329 INFO:teuthology.orchestra.run.smithi120.stdout: "registry_username": { 2024-09-19T09:28:17.330 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "registry_username", 2024-09-19T09:28:17.330 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.330 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.330 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.330 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.330 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.330 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.330 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.330 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T09:28:17.330 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.330 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.330 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.330 INFO:teuthology.orchestra.run.smithi120.stdout: "secure_monitoring_stack": { 2024-09-19T09:28:17.331 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "secure_monitoring_stack", 2024-09-19T09:28:17.331 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.331 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.331 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.331 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.331 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.331 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.331 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.331 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T09:28:17.331 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.331 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.331 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.331 INFO:teuthology.orchestra.run.smithi120.stdout: "service_discovery_port": { 2024-09-19T09:28:17.332 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "service_discovery_port", 2024-09-19T09:28:17.332 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.332 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.332 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.332 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8765", 2024-09-19T09:28:17.332 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.332 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.332 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.332 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "cephadm service discovery port", 2024-09-19T09:28:17.332 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.332 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.332 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.332 INFO:teuthology.orchestra.run.smithi120.stdout: "ssh_config_file": { 2024-09-19T09:28:17.333 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ssh_config_file", 2024-09-19T09:28:17.333 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.333 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.333 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.333 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.333 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.333 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.333 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.333 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T09:28:17.333 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.333 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.333 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.334 INFO:teuthology.orchestra.run.smithi120.stdout: "use_agent": { 2024-09-19T09:28:17.334 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "use_agent", 2024-09-19T09:28:17.334 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.334 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.334 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.334 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.334 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.334 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.334 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.334 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T09:28:17.334 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.334 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.334 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.335 INFO:teuthology.orchestra.run.smithi120.stdout: "use_repo_digest": { 2024-09-19T09:28:17.335 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "use_repo_digest", 2024-09-19T09:28:17.335 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.335 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.335 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.335 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.335 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.335 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.335 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.335 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T09:28:17.335 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.335 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.335 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.336 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_on_failed_host_check": { 2024-09-19T09:28:17.336 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_on_failed_host_check", 2024-09-19T09:28:17.336 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.336 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.336 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.336 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.336 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.336 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.336 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.336 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T09:28:17.336 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.336 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.336 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.337 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_on_stray_daemons": { 2024-09-19T09:28:17.337 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_on_stray_daemons", 2024-09-19T09:28:17.337 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.337 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.337 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.337 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.337 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.337 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.337 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.337 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T09:28:17.337 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.337 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.337 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.338 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_on_stray_hosts": { 2024-09-19T09:28:17.338 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_on_stray_hosts", 2024-09-19T09:28:17.338 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.338 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.338 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.338 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.338 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.338 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.338 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.338 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T09:28:17.338 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.338 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.338 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.339 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.339 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.339 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crash", 2024-09-19T09:28:17.339 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.339 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.339 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.339 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.339 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.339 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.339 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.339 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.339 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.339 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.340 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.340 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.340 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.340 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.340 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.340 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.340 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.340 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.340 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.340 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.340 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.340 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.340 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.341 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.341 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.341 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.341 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.341 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.341 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.341 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.341 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.341 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.341 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.341 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.341 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.341 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.342 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.342 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.342 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.342 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.342 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.342 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.342 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.342 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.342 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.342 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.342 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.342 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.342 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.342 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.343 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.343 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.343 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.343 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.343 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.343 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.343 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.343 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.343 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.343 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.343 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.343 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.343 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.344 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.344 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.344 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.344 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.344 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.344 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.344 INFO:teuthology.orchestra.run.smithi120.stdout: "retain_interval": { 2024-09-19T09:28:17.344 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "retain_interval", 2024-09-19T09:28:17.344 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:17.344 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.344 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.344 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "31536000", 2024-09-19T09:28:17.345 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.345 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.345 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.345 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T09:28:17.345 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.345 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.345 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.345 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_recent_interval": { 2024-09-19T09:28:17.345 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_recent_interval", 2024-09-19T09:28:17.345 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:17.345 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.345 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.345 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "1209600", 2024-09-19T09:28:17.346 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.346 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.346 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.346 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T09:28:17.346 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.346 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.346 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.346 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.346 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.346 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "dashboard", 2024-09-19T09:28:17.346 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.346 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.347 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.347 INFO:teuthology.orchestra.run.smithi120.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T09:28:17.347 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T09:28:17.347 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.347 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.347 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.347 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-19T09:28:17.347 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.347 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.347 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.347 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.347 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.347 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.347 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.348 INFO:teuthology.orchestra.run.smithi120.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T09:28:17.348 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T09:28:17.348 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.348 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.348 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.348 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.348 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.348 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.348 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.348 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.348 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.348 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.349 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.349 INFO:teuthology.orchestra.run.smithi120.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T09:28:17.349 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T09:28:17.349 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.349 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.349 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.349 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.349 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.349 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.349 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.349 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.349 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.349 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.350 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.350 INFO:teuthology.orchestra.run.smithi120.stdout: "AUDIT_API_ENABLED": { 2024-09-19T09:28:17.350 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T09:28:17.350 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.350 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.350 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.350 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.350 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.350 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.350 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.350 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.350 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.350 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.351 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.351 INFO:teuthology.orchestra.run.smithi120.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T09:28:17.351 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T09:28:17.351 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.351 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.351 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.351 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.351 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.351 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.351 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.351 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.351 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.351 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.352 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.352 INFO:teuthology.orchestra.run.smithi120.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T09:28:17.352 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T09:28:17.352 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.352 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.352 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.352 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.352 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.352 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.352 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.352 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.352 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.353 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.353 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.353 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T09:28:17.353 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T09:28:17.353 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.353 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.353 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.353 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.353 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.353 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.353 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.353 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.353 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.353 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.354 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.354 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T09:28:17.354 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T09:28:17.354 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.354 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.354 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.354 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.354 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.354 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.354 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.354 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.354 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.354 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.355 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.355 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T09:28:17.355 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T09:28:17.355 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.355 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.355 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.355 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.355 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.355 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.355 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.355 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.355 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.355 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.356 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.356 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T09:28:17.356 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T09:28:17.356 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.356 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.356 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.356 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.356 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.356 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.356 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.356 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.356 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.356 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.357 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.357 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T09:28:17.357 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T09:28:17.357 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.357 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.357 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.357 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.357 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.357 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.357 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.357 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.357 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.357 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.358 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.358 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T09:28:17.358 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T09:28:17.358 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.360 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.360 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.360 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.360 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.360 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.360 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.360 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.360 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.360 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.360 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.360 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T09:28:17.360 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T09:28:17.360 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.361 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.361 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.361 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.361 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.361 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.361 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.361 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.361 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.361 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.361 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.361 INFO:teuthology.orchestra.run.smithi120.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T09:28:17.361 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T09:28:17.361 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.362 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.362 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.362 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.362 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.362 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.362 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.362 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.362 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.362 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.362 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.362 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T09:28:17.362 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T09:28:17.362 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.363 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.363 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.363 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "admin", 2024-09-19T09:28:17.363 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.363 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.363 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.363 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.363 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.363 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.363 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.363 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T09:28:17.363 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T09:28:17.363 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.364 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.364 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.364 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.364 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.364 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.364 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.364 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.364 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.364 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.364 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.364 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_API_URL": { 2024-09-19T09:28:17.364 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_API_URL", 2024-09-19T09:28:17.364 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.365 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.365 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.365 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.365 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.365 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.365 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.365 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.365 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.365 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.365 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.365 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T09:28:17.365 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T09:28:17.365 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.366 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.366 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.366 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "admin", 2024-09-19T09:28:17.366 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.366 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.366 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.366 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.366 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.366 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.366 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.366 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T09:28:17.366 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T09:28:17.367 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.367 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.367 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.367 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.367 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.367 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.367 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.367 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.367 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.367 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.367 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.367 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T09:28:17.367 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T09:28:17.368 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.368 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.368 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.368 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.368 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.368 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.368 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.368 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.368 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.368 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.368 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.368 INFO:teuthology.orchestra.run.smithi120.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T09:28:17.368 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T09:28:17.369 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.369 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.369 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.369 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.369 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.369 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.369 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.369 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.369 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.369 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.369 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.369 INFO:teuthology.orchestra.run.smithi120.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T09:28:17.369 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T09:28:17.370 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.370 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.370 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.370 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.370 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.370 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.370 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.370 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.370 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.370 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.370 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.370 INFO:teuthology.orchestra.run.smithi120.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T09:28:17.370 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T09:28:17.371 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.371 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.371 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.371 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.371 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.371 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.371 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.371 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.371 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.371 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.371 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.371 INFO:teuthology.orchestra.run.smithi120.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T09:28:17.371 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T09:28:17.372 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.372 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.372 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.372 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.372 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.372 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.372 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.372 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.372 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.372 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.372 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.372 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T09:28:17.372 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T09:28:17.373 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.373 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.373 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.373 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.373 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.373 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.373 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.373 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.373 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.373 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.373 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.373 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T09:28:17.373 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T09:28:17.374 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.374 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.374 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.374 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.374 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.374 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.374 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.374 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.374 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.374 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.374 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.374 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T09:28:17.374 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T09:28:17.374 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.375 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.375 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.375 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.375 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.375 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.375 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.375 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.375 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.375 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.375 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.375 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T09:28:17.375 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T09:28:17.375 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.376 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.376 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.376 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.376 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.376 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.376 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.376 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.376 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.376 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.376 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.376 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T09:28:17.376 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T09:28:17.376 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.377 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.377 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.377 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.377 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.377 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.377 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.377 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.377 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.377 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.377 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.377 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T09:28:17.377 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T09:28:17.377 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.377 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.378 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.378 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.378 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.378 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.378 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.378 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.378 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.378 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.378 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.378 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T09:28:17.378 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T09:28:17.378 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.379 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.379 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.379 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.379 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.379 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.379 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.379 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.379 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.379 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.379 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.379 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T09:28:17.379 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T09:28:17.380 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.380 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.380 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.380 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.380 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.380 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.380 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.380 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.380 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.380 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.380 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.380 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T09:28:17.380 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T09:28:17.381 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.381 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.381 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.381 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T09:28:17.381 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.381 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.381 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.381 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.381 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.381 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.381 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.381 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T09:28:17.382 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T09:28:17.382 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.382 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.382 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.382 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-19T09:28:17.382 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.382 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.382 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.382 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.382 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.382 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.382 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.382 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T09:28:17.383 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T09:28:17.383 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.383 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.383 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.383 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8", 2024-09-19T09:28:17.383 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.383 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.383 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.383 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.383 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.383 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.383 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.383 INFO:teuthology.orchestra.run.smithi120.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T09:28:17.384 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T09:28:17.384 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.384 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.384 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.384 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "45", 2024-09-19T09:28:17.384 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.384 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.384 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.384 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.384 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.384 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.384 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.384 INFO:teuthology.orchestra.run.smithi120.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T09:28:17.385 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T09:28:17.385 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.385 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.385 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.385 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.385 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.385 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.385 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.385 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.385 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.385 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.385 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.385 INFO:teuthology.orchestra.run.smithi120.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T09:28:17.386 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T09:28:17.386 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.386 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.386 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.386 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "admin", 2024-09-19T09:28:17.386 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.386 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.386 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.386 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.386 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.386 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.386 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.386 INFO:teuthology.orchestra.run.smithi120.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T09:28:17.387 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T09:28:17.387 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.387 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.387 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.387 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.387 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.387 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.387 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.387 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.387 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.387 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.387 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.387 INFO:teuthology.orchestra.run.smithi120.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T09:28:17.387 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T09:28:17.388 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.388 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.388 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.388 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.388 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.388 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.388 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.388 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.388 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.388 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.388 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.388 INFO:teuthology.orchestra.run.smithi120.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T09:28:17.389 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T09:28:17.389 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.389 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.389 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.389 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-19T09:28:17.389 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.389 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.389 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.389 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.389 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.389 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.389 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.389 INFO:teuthology.orchestra.run.smithi120.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T09:28:17.389 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T09:28:17.390 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.390 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.390 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.390 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-19T09:28:17.390 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.390 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.390 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.390 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.390 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.390 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.390 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.390 INFO:teuthology.orchestra.run.smithi120.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T09:28:17.390 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T09:28:17.391 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.391 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.391 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.391 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-19T09:28:17.391 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.391 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.391 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.391 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.391 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.391 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.391 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.391 INFO:teuthology.orchestra.run.smithi120.stdout: "cross_origin_url": { 2024-09-19T09:28:17.391 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cross_origin_url", 2024-09-19T09:28:17.392 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.392 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.392 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.392 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.392 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.392 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.392 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.392 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.392 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.392 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.392 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.392 INFO:teuthology.orchestra.run.smithi120.stdout: "crt_file": { 2024-09-19T09:28:17.392 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crt_file", 2024-09-19T09:28:17.392 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.393 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.393 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.393 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.393 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.393 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.393 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.393 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.393 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.393 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.393 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.393 INFO:teuthology.orchestra.run.smithi120.stdout: "debug": { 2024-09-19T09:28:17.393 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "debug", 2024-09-19T09:28:17.393 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.394 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.394 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.394 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.394 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.394 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.394 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.394 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Enable/disable debug options", 2024-09-19T09:28:17.394 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.394 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.394 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.394 INFO:teuthology.orchestra.run.smithi120.stdout: "jwt_token_ttl": { 2024-09-19T09:28:17.394 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "jwt_token_ttl", 2024-09-19T09:28:17.395 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.395 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.395 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.395 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "28800", 2024-09-19T09:28:17.395 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.395 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.395 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.395 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.395 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.395 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.395 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.395 INFO:teuthology.orchestra.run.smithi120.stdout: "key_file": { 2024-09-19T09:28:17.395 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "key_file", 2024-09-19T09:28:17.396 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.396 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.396 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.396 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.396 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.396 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.396 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.396 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.396 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.396 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.396 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.396 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.396 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.397 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.397 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.397 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.397 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.397 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.397 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.397 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.397 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.397 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.397 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.397 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.397 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.397 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.397 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.398 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.398 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.398 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.398 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.398 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.398 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.398 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.398 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.398 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.398 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.398 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.398 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.398 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.399 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.399 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.399 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.399 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.399 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.399 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.399 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.399 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.399 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.399 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.399 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.399 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.399 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.400 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.400 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.400 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.400 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.400 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.400 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.400 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.400 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.400 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.400 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.400 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.400 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.401 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.401 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.401 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.401 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.401 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.401 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.401 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.401 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.401 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.401 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.401 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.401 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.401 INFO:teuthology.orchestra.run.smithi120.stdout: "motd": { 2024-09-19T09:28:17.402 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "motd", 2024-09-19T09:28:17.402 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.402 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.402 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.402 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.402 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.402 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.402 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.402 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "The message of the day", 2024-09-19T09:28:17.402 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.402 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.402 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.403 INFO:teuthology.orchestra.run.smithi120.stdout: "redirect_resolve_ip_addr": { 2024-09-19T09:28:17.403 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T09:28:17.403 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.403 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.403 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.403 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.403 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.403 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.403 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.403 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.403 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.403 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.403 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.404 INFO:teuthology.orchestra.run.smithi120.stdout: "server_addr": { 2024-09-19T09:28:17.404 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_addr", 2024-09-19T09:28:17.404 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.404 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.404 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.404 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "::", 2024-09-19T09:28:17.404 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.404 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.404 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.404 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.404 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.404 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.404 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.405 INFO:teuthology.orchestra.run.smithi120.stdout: "server_port": { 2024-09-19T09:28:17.405 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_port", 2024-09-19T09:28:17.405 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.405 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.405 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.405 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8080", 2024-09-19T09:28:17.405 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.405 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.405 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.405 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.405 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.405 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.405 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.406 INFO:teuthology.orchestra.run.smithi120.stdout: "ssl": { 2024-09-19T09:28:17.406 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ssl", 2024-09-19T09:28:17.406 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.406 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.406 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.406 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.406 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.406 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.406 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.406 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.407 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.407 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.407 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.407 INFO:teuthology.orchestra.run.smithi120.stdout: "ssl_server_port": { 2024-09-19T09:28:17.407 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ssl_server_port", 2024-09-19T09:28:17.407 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.407 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.407 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.407 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8443", 2024-09-19T09:28:17.407 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.407 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.407 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.407 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.408 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.408 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.408 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.408 INFO:teuthology.orchestra.run.smithi120.stdout: "standby_behaviour": { 2024-09-19T09:28:17.408 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "standby_behaviour", 2024-09-19T09:28:17.408 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.408 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.408 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.408 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "redirect", 2024-09-19T09:28:17.408 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.408 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.408 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.409 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.409 INFO:teuthology.orchestra.run.smithi120.stdout: "redirect" 2024-09-19T09:28:17.409 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.409 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.409 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.409 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.409 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.409 INFO:teuthology.orchestra.run.smithi120.stdout: "standby_error_status_code": { 2024-09-19T09:28:17.409 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "standby_error_status_code", 2024-09-19T09:28:17.409 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.409 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.409 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.409 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "500", 2024-09-19T09:28:17.410 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "400", 2024-09-19T09:28:17.410 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "599", 2024-09-19T09:28:17.410 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.410 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.410 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.410 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.410 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.410 INFO:teuthology.orchestra.run.smithi120.stdout: "url_prefix": { 2024-09-19T09:28:17.410 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "url_prefix", 2024-09-19T09:28:17.410 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.410 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.410 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.410 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.411 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.411 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.411 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.411 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.411 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.411 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.411 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.411 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.411 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.411 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "devicehealth", 2024-09-19T09:28:17.411 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.411 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.412 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.412 INFO:teuthology.orchestra.run.smithi120.stdout: "enable_monitoring": { 2024-09-19T09:28:17.412 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "enable_monitoring", 2024-09-19T09:28:17.412 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.412 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.412 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.412 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.412 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.412 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.412 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.412 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "monitor device health metrics", 2024-09-19T09:28:17.412 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.412 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.413 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.413 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.413 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.413 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.413 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.413 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.413 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.413 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.413 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.413 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.413 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.413 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.413 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.414 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.414 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.414 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.414 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.414 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.414 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.414 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.414 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.414 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.414 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.414 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.414 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.414 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.415 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.415 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.415 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.415 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.415 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.415 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.415 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.415 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.415 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.415 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.415 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.415 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.416 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.416 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.416 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.416 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.416 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.416 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.416 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.416 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.416 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.416 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.416 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.416 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.416 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.416 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.417 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.417 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.417 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.417 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.417 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.417 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.417 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.417 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.417 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.417 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.417 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.417 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.418 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.418 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.418 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.418 INFO:teuthology.orchestra.run.smithi120.stdout: "mark_out_threshold": { 2024-09-19T09:28:17.418 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mark_out_threshold", 2024-09-19T09:28:17.418 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:17.418 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.418 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.418 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "2419200", 2024-09-19T09:28:17.418 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.418 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.418 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.418 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T09:28:17.419 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.419 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.419 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.419 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": { 2024-09-19T09:28:17.419 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "pool_name", 2024-09-19T09:28:17.419 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.419 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.419 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.419 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "device_health_metrics", 2024-09-19T09:28:17.419 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.419 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.419 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.419 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T09:28:17.420 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.420 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.420 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.420 INFO:teuthology.orchestra.run.smithi120.stdout: "retention_period": { 2024-09-19T09:28:17.420 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "retention_period", 2024-09-19T09:28:17.420 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:17.420 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.420 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.420 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "15552000", 2024-09-19T09:28:17.420 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.420 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.420 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.421 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how long to retain device health metrics", 2024-09-19T09:28:17.421 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.421 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.421 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.421 INFO:teuthology.orchestra.run.smithi120.stdout: "scrape_frequency": { 2024-09-19T09:28:17.421 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "scrape_frequency", 2024-09-19T09:28:17.421 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:17.421 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.421 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.421 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "86400", 2024-09-19T09:28:17.421 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.421 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.421 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.422 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T09:28:17.422 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.422 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.422 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.422 INFO:teuthology.orchestra.run.smithi120.stdout: "self_heal": { 2024-09-19T09:28:17.422 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "self_heal", 2024-09-19T09:28:17.422 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.422 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.422 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.422 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.422 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.422 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.422 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.423 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T09:28:17.423 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.423 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.423 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.423 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-19T09:28:17.423 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-19T09:28:17.423 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:17.423 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.423 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.423 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-19T09:28:17.423 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.423 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.423 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.424 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T09:28:17.424 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.424 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.424 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.424 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_threshold": { 2024-09-19T09:28:17.424 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_threshold", 2024-09-19T09:28:17.424 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:17.424 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.424 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.424 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "7257600", 2024-09-19T09:28:17.424 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.424 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.425 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.425 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T09:28:17.425 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.425 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.425 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.425 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.425 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.425 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "diskprediction_local", 2024-09-19T09:28:17.425 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.425 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.425 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.425 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.426 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.426 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.426 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.426 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.426 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.426 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.426 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.426 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.426 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.426 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.426 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.426 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.426 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.426 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.426 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.427 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.427 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.427 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.427 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.427 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.427 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.427 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.427 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.427 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.427 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.427 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.427 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.427 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.428 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.428 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.428 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.428 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.428 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.428 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.428 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.428 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.428 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.429 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.429 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.429 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.429 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.429 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.429 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.429 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.429 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.429 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.429 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.429 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.429 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.429 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.430 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.430 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.430 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.430 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.430 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.430 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.430 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.430 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.430 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.430 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.430 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.430 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.430 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.431 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.431 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.431 INFO:teuthology.orchestra.run.smithi120.stdout: "predict_interval": { 2024-09-19T09:28:17.431 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "predict_interval", 2024-09-19T09:28:17.431 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.431 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.431 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.431 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "86400", 2024-09-19T09:28:17.431 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.431 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.431 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.431 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.431 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.432 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.432 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.432 INFO:teuthology.orchestra.run.smithi120.stdout: "predictor_model": { 2024-09-19T09:28:17.432 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "predictor_model", 2024-09-19T09:28:17.432 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.432 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.432 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.432 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "prophetstor", 2024-09-19T09:28:17.432 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.432 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.432 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.432 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.433 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.433 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.433 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.433 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-19T09:28:17.433 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-19T09:28:17.433 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.433 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.433 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.433 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-19T09:28:17.433 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.433 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.433 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.433 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.434 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.434 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.434 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.434 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.434 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.434 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "influx", 2024-09-19T09:28:17.434 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": false, 2024-09-19T09:28:17.434 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "influxdb python module not found", 2024-09-19T09:28:17.434 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.434 INFO:teuthology.orchestra.run.smithi120.stdout: "batch_size": { 2024-09-19T09:28:17.434 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "batch_size", 2024-09-19T09:28:17.434 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.435 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.435 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.435 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5000", 2024-09-19T09:28:17.435 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.435 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.435 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.435 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T09:28:17.435 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.435 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.435 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.435 INFO:teuthology.orchestra.run.smithi120.stdout: "database": { 2024-09-19T09:28:17.435 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "database", 2024-09-19T09:28:17.436 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.436 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.436 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.436 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "ceph", 2024-09-19T09:28:17.436 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.436 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.436 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.436 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T09:28:17.436 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.436 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.436 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.436 INFO:teuthology.orchestra.run.smithi120.stdout: "hostname": { 2024-09-19T09:28:17.436 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "hostname", 2024-09-19T09:28:17.437 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.437 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.437 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.437 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.437 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.437 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.437 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.437 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "InfluxDB server hostname", 2024-09-19T09:28:17.437 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.437 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.437 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.437 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-19T09:28:17.438 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-19T09:28:17.438 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:17.438 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.438 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.438 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "30", 2024-09-19T09:28:17.438 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "5", 2024-09-19T09:28:17.438 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.438 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.438 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T09:28:17.438 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.438 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.438 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.438 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.439 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.439 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.439 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.439 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.439 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.439 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.439 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.439 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.439 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.439 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.439 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.439 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.439 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.439 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.440 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.440 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.440 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.440 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.440 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.440 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.440 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.440 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.440 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.440 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.440 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.440 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.440 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.441 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.441 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.441 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.441 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.441 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.441 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.441 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.441 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.441 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.441 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.441 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.441 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.441 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.442 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.442 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.442 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.442 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.442 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.442 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.442 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.442 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.442 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.442 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.442 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.442 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.442 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.443 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.443 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.443 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.443 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.443 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.443 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.443 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.443 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.443 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.443 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.443 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.443 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.443 INFO:teuthology.orchestra.run.smithi120.stdout: "password": { 2024-09-19T09:28:17.444 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "password", 2024-09-19T09:28:17.444 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.444 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.444 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.444 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.444 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.444 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.444 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.444 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "password of InfluxDB server user", 2024-09-19T09:28:17.444 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.444 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.444 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.444 INFO:teuthology.orchestra.run.smithi120.stdout: "port": { 2024-09-19T09:28:17.445 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "port", 2024-09-19T09:28:17.445 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.445 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.445 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.445 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8086", 2024-09-19T09:28:17.445 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.445 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.445 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.445 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "InfluxDB server port", 2024-09-19T09:28:17.445 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.445 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.445 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.446 INFO:teuthology.orchestra.run.smithi120.stdout: "ssl": { 2024-09-19T09:28:17.446 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ssl", 2024-09-19T09:28:17.446 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.446 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.446 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.446 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "false", 2024-09-19T09:28:17.446 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.446 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.446 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.446 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T09:28:17.446 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.446 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.446 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.447 INFO:teuthology.orchestra.run.smithi120.stdout: "threads": { 2024-09-19T09:28:17.447 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "threads", 2024-09-19T09:28:17.447 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.447 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.447 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.447 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-19T09:28:17.447 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1", 2024-09-19T09:28:17.447 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "32", 2024-09-19T09:28:17.447 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.447 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T09:28:17.447 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.447 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.447 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.447 INFO:teuthology.orchestra.run.smithi120.stdout: "username": { 2024-09-19T09:28:17.448 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "username", 2024-09-19T09:28:17.448 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.448 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.448 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.448 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.448 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.448 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.448 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.448 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "username of InfluxDB server user", 2024-09-19T09:28:17.448 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.448 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.448 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.449 INFO:teuthology.orchestra.run.smithi120.stdout: "verify_ssl": { 2024-09-19T09:28:17.449 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "verify_ssl", 2024-09-19T09:28:17.449 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.449 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.449 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.449 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "true", 2024-09-19T09:28:17.449 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.449 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.449 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.449 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T09:28:17.449 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.449 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.449 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.449 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.450 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.450 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "insights", 2024-09-19T09:28:17.450 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.450 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.450 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.450 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.450 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.450 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.450 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.450 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.450 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.450 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.451 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.451 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.451 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.451 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.451 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.451 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.451 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.451 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.451 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.451 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.451 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.451 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.451 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.452 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.452 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.452 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.452 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.452 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.452 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.452 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.452 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.452 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.452 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.452 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.452 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.452 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.453 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.453 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.453 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.453 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.453 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.453 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.453 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.453 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.453 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.453 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.453 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.453 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.453 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.453 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.454 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.454 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.454 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.454 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.454 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.454 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.454 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.454 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.454 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.454 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.454 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.454 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.454 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.455 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.455 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.455 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.455 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.455 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.455 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.455 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.455 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.455 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "iostat", 2024-09-19T09:28:17.455 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.455 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.455 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.456 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.456 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.456 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.456 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.456 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.456 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.456 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.456 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.456 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.456 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.456 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.456 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.456 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.456 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.457 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.457 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.457 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.457 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.457 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.457 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.457 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.457 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.457 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.457 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.457 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.457 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.457 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.458 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.458 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.458 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.458 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.458 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.458 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.458 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.458 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.458 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.458 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.458 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.458 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.458 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.458 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.459 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.459 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.459 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.459 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.459 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.459 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.459 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.459 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.459 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.459 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.459 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.459 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.460 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.460 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.460 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.460 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.460 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.460 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.460 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.460 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.460 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.460 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.460 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.460 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.460 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.460 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.461 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.461 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "k8sevents", 2024-09-19T09:28:17.461 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.461 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.461 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.461 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_event_retention_days": { 2024-09-19T09:28:17.461 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ceph_event_retention_days", 2024-09-19T09:28:17.461 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.461 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.461 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.461 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "7", 2024-09-19T09:28:17.461 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.462 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.462 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.462 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T09:28:17.462 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.462 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.462 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.462 INFO:teuthology.orchestra.run.smithi120.stdout: "config_check_secs": { 2024-09-19T09:28:17.462 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "config_check_secs", 2024-09-19T09:28:17.462 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.462 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.462 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.462 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-19T09:28:17.463 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "10", 2024-09-19T09:28:17.463 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.463 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.463 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T09:28:17.463 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.463 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.463 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.463 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.463 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.463 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.463 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.463 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.463 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.464 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.464 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.464 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.464 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.464 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.464 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.464 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.464 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.464 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.464 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.464 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.464 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.464 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.464 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.465 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.465 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.465 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.465 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.465 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.465 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.465 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.465 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.465 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.465 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.465 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.465 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.466 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.466 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.466 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.466 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.466 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.466 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.466 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.466 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.466 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.466 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.466 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.466 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.466 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.467 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.467 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.467 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.467 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.467 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.467 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.467 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.467 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.467 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.467 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.467 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.467 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.467 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.468 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.468 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.468 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.468 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.468 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.468 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.468 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.468 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.468 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.468 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.468 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "localpool", 2024-09-19T09:28:17.469 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.469 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.469 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.469 INFO:teuthology.orchestra.run.smithi120.stdout: "failure_domain": { 2024-09-19T09:28:17.469 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "failure_domain", 2024-09-19T09:28:17.469 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.469 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.469 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.469 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "host", 2024-09-19T09:28:17.469 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.469 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.469 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.469 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "failure domain for any created local pool", 2024-09-19T09:28:17.469 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T09:28:17.470 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.470 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.470 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.470 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.470 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.470 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.470 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.470 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.470 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.470 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.470 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.470 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.470 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.471 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.471 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.471 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.471 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.471 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.471 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.471 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.471 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.471 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.471 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.471 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.471 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.471 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.472 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.472 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.472 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.472 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.472 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.472 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.472 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.472 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.472 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.472 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.472 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.472 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.473 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.473 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.473 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.473 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.473 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.473 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.473 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.473 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.473 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.473 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.473 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.473 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.473 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.473 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.474 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.474 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.474 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.474 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.474 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.474 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.474 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.474 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.474 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.474 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.474 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.474 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.474 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.474 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.475 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.475 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.475 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": { 2024-09-19T09:28:17.475 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "min_size", 2024-09-19T09:28:17.475 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.475 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.475 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.475 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.475 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.475 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.475 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.475 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "default min_size for any created local pool", 2024-09-19T09:28:17.475 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T09:28:17.476 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.476 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.476 INFO:teuthology.orchestra.run.smithi120.stdout: "num_rep": { 2024-09-19T09:28:17.476 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "num_rep", 2024-09-19T09:28:17.476 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.476 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.476 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.476 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "3", 2024-09-19T09:28:17.476 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.476 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.476 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.476 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "default replica count for any created local pool", 2024-09-19T09:28:17.476 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.477 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.477 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.477 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": { 2024-09-19T09:28:17.477 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "pg_num", 2024-09-19T09:28:17.477 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.477 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.477 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.477 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "128", 2024-09-19T09:28:17.477 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.477 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.477 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.477 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T09:28:17.477 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.478 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.478 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.478 INFO:teuthology.orchestra.run.smithi120.stdout: "prefix": { 2024-09-19T09:28:17.478 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "prefix", 2024-09-19T09:28:17.478 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.478 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.478 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.478 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.478 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.478 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.478 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.478 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "name prefix for any created local pool", 2024-09-19T09:28:17.479 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.479 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.479 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.479 INFO:teuthology.orchestra.run.smithi120.stdout: "subtree": { 2024-09-19T09:28:17.479 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "subtree", 2024-09-19T09:28:17.479 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.479 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.479 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.479 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "rack", 2024-09-19T09:28:17.479 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.479 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.479 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.480 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T09:28:17.480 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T09:28:17.480 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.480 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.480 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.480 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.480 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mds_autoscaler", 2024-09-19T09:28:17.480 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.480 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.480 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.480 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.481 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.481 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.481 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.481 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.481 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.481 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.481 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.481 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.481 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.481 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.481 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.481 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.481 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.481 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.482 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.482 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.482 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.482 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.482 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.482 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.482 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.482 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.482 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.482 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.482 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.482 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.482 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.483 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.483 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.483 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.483 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.483 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.483 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.483 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.483 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.483 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.483 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.483 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.483 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.483 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.484 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.484 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.484 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.484 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.484 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.484 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.484 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.484 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.484 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.484 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.484 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.484 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.485 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.485 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.485 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.485 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.485 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.485 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.485 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.485 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.485 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.485 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.485 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.485 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.486 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.486 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.486 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.486 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mirroring", 2024-09-19T09:28:17.486 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.486 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.486 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.486 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.486 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.486 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.486 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.486 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.487 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.487 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.487 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.487 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.487 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.487 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.487 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.487 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.487 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.487 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.487 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.487 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.487 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.487 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.488 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.488 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.488 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.488 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.488 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.488 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.488 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.488 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.488 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.488 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.488 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.488 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.488 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.489 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.489 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.489 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.489 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.489 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.489 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.489 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.489 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.489 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.489 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.489 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.489 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.489 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.490 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.490 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.490 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.490 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.490 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.490 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.490 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.490 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.490 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.490 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.490 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.490 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.490 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.491 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.491 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.491 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.491 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.491 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.491 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.491 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.491 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.491 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.491 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.491 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "nfs", 2024-09-19T09:28:17.491 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.491 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.492 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.492 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.492 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.492 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.492 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.492 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.492 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.492 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.492 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.492 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.492 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.492 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.492 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.492 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.493 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.493 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.493 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.493 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.493 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.493 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.493 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.493 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.493 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.493 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.493 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.493 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.494 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.494 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.494 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.494 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.494 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.494 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.494 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.494 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.494 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.494 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.494 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.494 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.494 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.494 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.495 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.495 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.495 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.495 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.495 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.495 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.495 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.495 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.495 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.495 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.495 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.495 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.495 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.495 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.496 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.496 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.496 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.496 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.496 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.496 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.496 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.496 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.496 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.496 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.496 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.497 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.497 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.497 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.497 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.497 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "orchestrator", 2024-09-19T09:28:17.497 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.497 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.497 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.497 INFO:teuthology.orchestra.run.smithi120.stdout: "fail_fs": { 2024-09-19T09:28:17.497 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "fail_fs", 2024-09-19T09:28:17.497 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.497 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.498 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.498 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.498 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.498 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.498 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.498 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T09:28:17.498 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.498 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.498 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.498 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.498 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.498 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.498 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.499 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.499 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.499 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.499 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.499 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.499 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.499 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.499 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.499 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.499 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.499 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.499 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.499 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.499 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.500 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.500 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.500 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.500 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.500 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.500 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.500 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.500 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.500 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.500 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.500 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.500 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.501 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.501 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.501 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.501 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.501 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.501 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.501 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.501 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.501 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.501 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.501 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.501 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.501 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.502 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.502 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.502 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.502 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.502 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.502 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.502 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.502 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.502 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.502 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.502 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.502 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.502 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.503 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.503 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.503 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.503 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.503 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.503 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.503 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.503 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.503 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.503 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.503 INFO:teuthology.orchestra.run.smithi120.stdout: "orchestrator": { 2024-09-19T09:28:17.503 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "orchestrator", 2024-09-19T09:28:17.503 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.504 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.504 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.504 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.504 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.504 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.504 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.504 INFO:teuthology.orchestra.run.smithi120.stdout: "cephadm", 2024-09-19T09:28:17.504 INFO:teuthology.orchestra.run.smithi120.stdout: "rook", 2024-09-19T09:28:17.504 INFO:teuthology.orchestra.run.smithi120.stdout: "test_orchestrator" 2024-09-19T09:28:17.504 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.504 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Orchestrator backend", 2024-09-19T09:28:17.504 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.504 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.504 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.505 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.505 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.505 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "osd_perf_query", 2024-09-19T09:28:17.505 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.505 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.505 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.505 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.505 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.505 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.505 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.505 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.505 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.506 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.506 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.506 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.506 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.506 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.506 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.506 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.506 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.506 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.506 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.506 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.506 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.506 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.507 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.507 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.507 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.507 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.507 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.507 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.507 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.507 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.507 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.507 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.507 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.507 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.507 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.508 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.508 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.508 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.508 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.508 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.508 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.508 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.508 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.508 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.508 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.508 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.508 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.508 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.509 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.509 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.509 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.509 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.509 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.509 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.509 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.509 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.509 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.509 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.509 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.509 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.509 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.510 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.510 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.510 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.510 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.510 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.510 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.510 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.510 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.510 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.510 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.510 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "osd_support", 2024-09-19T09:28:17.510 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.511 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.511 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.511 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.511 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.511 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.511 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.511 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.511 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.511 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.511 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.511 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.511 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.511 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.511 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.512 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.512 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.512 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.512 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.512 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.512 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.512 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.512 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.512 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.512 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.512 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.512 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.512 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.513 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.513 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.513 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.513 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.513 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.513 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.513 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.513 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.513 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.513 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.513 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.513 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.513 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.514 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.514 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.514 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.514 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.514 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.514 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.514 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.514 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.514 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.514 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.514 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.514 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.514 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.515 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.515 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.515 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.515 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.515 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.515 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.515 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.515 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.515 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.515 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.515 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.515 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.515 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.515 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.516 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.516 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.516 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.516 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "pg_autoscaler", 2024-09-19T09:28:17.516 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.516 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.516 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.516 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.516 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.516 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.516 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.517 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.517 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.517 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.517 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.517 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.517 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.517 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.517 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.517 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.517 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.517 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.517 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.517 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.517 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.518 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.518 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.518 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.518 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.518 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.518 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.518 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.518 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.518 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.518 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.518 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.518 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.518 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.518 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.519 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.519 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.519 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.519 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.519 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.519 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.519 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.519 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.519 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.519 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.519 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.519 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.519 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.520 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.520 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.520 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.520 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.520 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.520 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.520 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.520 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.520 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.520 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.520 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.520 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.520 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.521 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.521 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.521 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.521 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.521 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.521 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.521 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.521 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.521 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-19T09:28:17.521 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-19T09:28:17.521 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:17.521 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.521 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.522 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-19T09:28:17.522 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.522 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.522 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.522 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.522 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.522 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.522 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.522 INFO:teuthology.orchestra.run.smithi120.stdout: "threshold": { 2024-09-19T09:28:17.522 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "threshold", 2024-09-19T09:28:17.522 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-19T09:28:17.522 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.522 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.523 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "3.0", 2024-09-19T09:28:17.523 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1.0", 2024-09-19T09:28:17.523 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.523 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.523 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "scaling threshold", 2024-09-19T09:28:17.523 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T09:28:17.523 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.523 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.523 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.523 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.523 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "progress", 2024-09-19T09:28:17.523 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.524 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.524 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.524 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_pg_recovery_event": { 2024-09-19T09:28:17.524 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "allow_pg_recovery_event", 2024-09-19T09:28:17.524 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.524 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.524 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.524 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.524 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.524 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.524 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.524 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T09:28:17.524 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.524 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.525 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.525 INFO:teuthology.orchestra.run.smithi120.stdout: "enabled": { 2024-09-19T09:28:17.525 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "enabled", 2024-09-19T09:28:17.525 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.525 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.525 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.525 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.525 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.525 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.525 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.525 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.525 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.525 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.526 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.526 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.526 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.526 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.526 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.526 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.526 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.526 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.526 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.526 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.526 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.526 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.526 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.527 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.527 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.527 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.527 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.527 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.527 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.527 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.527 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.527 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.527 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.527 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.527 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.527 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.528 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.528 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.528 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.528 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.528 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.528 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.528 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.528 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.528 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.528 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.528 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.528 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.528 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.529 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.529 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.529 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.529 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.529 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.529 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.529 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.529 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.529 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.529 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.529 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.529 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.529 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.529 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.530 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.530 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.530 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.530 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.530 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.530 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.530 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.530 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.530 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.530 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.530 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.530 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.530 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.531 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.531 INFO:teuthology.orchestra.run.smithi120.stdout: "max_completed_events": { 2024-09-19T09:28:17.531 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_completed_events", 2024-09-19T09:28:17.531 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.531 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.531 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.531 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "50", 2024-09-19T09:28:17.531 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.531 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.531 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.531 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "number of past completed events to remember", 2024-09-19T09:28:17.531 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.531 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.532 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.532 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-19T09:28:17.532 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-19T09:28:17.532 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:17.532 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.532 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.532 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-19T09:28:17.532 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.532 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.532 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.532 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how long the module is going to sleep", 2024-09-19T09:28:17.532 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.532 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.533 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.533 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.533 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.533 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "prometheus", 2024-09-19T09:28:17.533 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.533 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.533 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.533 INFO:teuthology.orchestra.run.smithi120.stdout: "cache": { 2024-09-19T09:28:17.533 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cache", 2024-09-19T09:28:17.533 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.533 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.534 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.534 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.534 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.534 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.534 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.534 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.534 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.534 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.534 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.534 INFO:teuthology.orchestra.run.smithi120.stdout: "exclude_perf_counters": { 2024-09-19T09:28:17.534 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "exclude_perf_counters", 2024-09-19T09:28:17.534 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.534 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.535 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.535 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.535 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.535 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.535 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.535 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T09:28:17.535 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T09:28:17.535 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.535 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.535 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.535 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.535 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.535 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.536 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.536 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.536 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.536 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.536 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.536 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.536 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.536 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.536 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.536 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.536 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.536 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.536 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.537 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.537 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.537 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.537 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.537 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.537 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.537 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.537 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.537 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.537 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.537 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.537 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.538 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.538 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.538 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.538 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.538 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.538 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.538 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.538 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.538 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.538 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.538 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.538 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.538 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.539 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.539 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.539 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.539 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.539 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.539 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.539 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.539 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.539 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.539 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.539 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.539 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.539 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.540 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.540 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.540 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.540 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.540 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.540 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.540 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.540 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.540 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.540 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.540 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.540 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_stats_pools": { 2024-09-19T09:28:17.540 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rbd_stats_pools", 2024-09-19T09:28:17.541 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.541 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.541 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.541 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.541 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.541 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.541 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.541 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.541 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.541 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.541 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.541 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T09:28:17.541 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T09:28:17.542 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.542 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.542 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.542 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "300", 2024-09-19T09:28:17.542 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.542 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.542 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.542 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.542 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.542 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.542 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.542 INFO:teuthology.orchestra.run.smithi120.stdout: "scrape_interval": { 2024-09-19T09:28:17.542 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "scrape_interval", 2024-09-19T09:28:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-19T09:28:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "15.0", 2024-09-19T09:28:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: "server_addr": { 2024-09-19T09:28:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_addr", 2024-09-19T09:28:17.544 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.544 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.544 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.544 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "::", 2024-09-19T09:28:17.544 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.544 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.544 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.544 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T09:28:17.544 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.544 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.544 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.544 INFO:teuthology.orchestra.run.smithi120.stdout: "server_port": { 2024-09-19T09:28:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_port", 2024-09-19T09:28:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "9283", 2024-09-19T09:28:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T09:28:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: "stale_cache_strategy": { 2024-09-19T09:28:17.546 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "stale_cache_strategy", 2024-09-19T09:28:17.546 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.546 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.546 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.546 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "log", 2024-09-19T09:28:17.546 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.546 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.546 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.546 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.546 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.546 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.546 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.546 INFO:teuthology.orchestra.run.smithi120.stdout: "standby_behaviour": { 2024-09-19T09:28:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "standby_behaviour", 2024-09-19T09:28:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "default", 2024-09-19T09:28:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: "default", 2024-09-19T09:28:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: "error" 2024-09-19T09:28:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: "standby_error_status_code": { 2024-09-19T09:28:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "standby_error_status_code", 2024-09-19T09:28:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "500", 2024-09-19T09:28:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "400", 2024-09-19T09:28:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "599", 2024-09-19T09:28:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.549 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.549 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.549 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.549 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.549 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rbd_support", 2024-09-19T09:28:17.549 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.549 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.549 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.549 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.549 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.549 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.549 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.550 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.550 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.550 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.550 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.550 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.550 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.550 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.550 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.550 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.550 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.550 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.550 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.550 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.550 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.551 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.551 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.551 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.551 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.551 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.551 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.551 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.551 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.551 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.551 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.551 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.551 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.551 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.552 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.552 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.552 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.552 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.552 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.552 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.552 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.552 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.552 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.552 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.552 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.552 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.553 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.553 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.553 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.553 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.553 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.553 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.553 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.553 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.553 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.553 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.553 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.553 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.553 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.554 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.554 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.554 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.554 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.554 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.554 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.554 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.554 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.554 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.554 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.554 INFO:teuthology.orchestra.run.smithi120.stdout: "max_concurrent_snap_create": { 2024-09-19T09:28:17.554 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_concurrent_snap_create", 2024-09-19T09:28:17.554 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.555 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.555 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.555 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-19T09:28:17.555 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.555 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.555 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.555 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.555 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.555 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.555 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.555 INFO:teuthology.orchestra.run.smithi120.stdout: "mirror_snapshot_schedule": { 2024-09-19T09:28:17.555 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T09:28:17.555 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.556 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.556 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.556 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.556 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.556 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.556 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.556 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.556 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.556 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.556 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.556 INFO:teuthology.orchestra.run.smithi120.stdout: "trash_purge_schedule": { 2024-09-19T09:28:17.556 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "trash_purge_schedule", 2024-09-19T09:28:17.556 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.557 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.557 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.557 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.557 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.557 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.557 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.557 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.557 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.557 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.557 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.557 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.557 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.557 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "restful", 2024-09-19T09:28:17.558 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.558 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.558 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.558 INFO:teuthology.orchestra.run.smithi120.stdout: "enable_auth": { 2024-09-19T09:28:17.558 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "enable_auth", 2024-09-19T09:28:17.558 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.558 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.558 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.558 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.558 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.558 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.558 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.558 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.558 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.559 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.559 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.559 INFO:teuthology.orchestra.run.smithi120.stdout: "key_file": { 2024-09-19T09:28:17.559 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "key_file", 2024-09-19T09:28:17.559 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.559 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.559 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.559 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.559 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.559 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.559 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.559 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.559 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.560 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.560 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.560 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.560 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.560 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.560 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.560 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.560 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.560 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.560 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.560 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.560 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.561 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.561 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.561 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.561 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.561 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.564 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.564 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.564 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.564 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.564 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.564 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.564 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.564 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.564 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.564 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.565 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.565 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.565 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.565 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.565 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.565 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.565 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.565 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.565 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.565 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.565 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.565 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.566 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.566 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.566 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.566 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.566 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.566 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.566 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.566 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.566 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.566 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.566 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.566 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.566 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.566 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.567 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.567 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.567 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.567 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.567 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.567 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.567 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.567 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.567 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.567 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.567 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.567 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: "server_addr": { 2024-09-19T09:28:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_addr", 2024-09-19T09:28:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: "server_port": { 2024-09-19T09:28:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_port", 2024-09-19T09:28:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rgw", 2024-09-19T09:28:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T09:28:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T09:28:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-19T09:28:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T09:28:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rook", 2024-09-19T09:28:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: "storage_class": { 2024-09-19T09:28:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "storage_class", 2024-09-19T09:28:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "local", 2024-09-19T09:28:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T09:28:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "selftest", 2024-09-19T09:28:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "roption1": { 2024-09-19T09:28:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "roption1", 2024-09-19T09:28:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: "roption2": { 2024-09-19T09:28:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "roption2", 2024-09-19T09:28:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "xyz", 2024-09-19T09:28:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption1": { 2024-09-19T09:28:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption1", 2024-09-19T09:28:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption2": { 2024-09-19T09:28:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption2", 2024-09-19T09:28:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption3": { 2024-09-19T09:28:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption3", 2024-09-19T09:28:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-19T09:28:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption4": { 2024-09-19T09:28:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption4", 2024-09-19T09:28:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption5": { 2024-09-19T09:28:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption5", 2024-09-19T09:28:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption6": { 2024-09-19T09:28:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption6", 2024-09-19T09:28:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption7": { 2024-09-19T09:28:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption7", 2024-09-19T09:28:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1", 2024-09-19T09:28:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "42", 2024-09-19T09:28:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: "testkey": { 2024-09-19T09:28:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "testkey", 2024-09-19T09:28:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: "testlkey": { 2024-09-19T09:28:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "testlkey", 2024-09-19T09:28:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: "testnewline": { 2024-09-19T09:28:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "testnewline", 2024-09-19T09:28:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "snap_schedule", 2024-09-19T09:28:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_m_granularity": { 2024-09-19T09:28:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "allow_m_granularity", 2024-09-19T09:28:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.612 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.612 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T09:28:17.612 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.612 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.612 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.612 INFO:teuthology.orchestra.run.smithi120.stdout: "dump_on_update": { 2024-09-19T09:28:17.612 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "dump_on_update", 2024-09-19T09:28:17.612 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.612 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.612 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.612 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.612 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.613 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.613 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.613 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "dump database to debug log on update", 2024-09-19T09:28:17.613 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.613 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.613 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.613 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.613 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.613 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.613 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.614 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.614 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.614 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.614 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.614 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.615 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.615 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.615 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.615 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.615 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.615 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.616 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.616 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.616 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.616 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.616 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.617 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.617 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.617 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.617 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.617 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.618 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.618 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.618 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.618 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.618 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.618 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.619 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.619 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.619 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.619 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.619 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.619 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.619 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.619 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.619 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.619 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.619 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.619 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.620 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.620 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.620 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.620 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.620 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.620 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.620 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.620 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.620 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.621 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.621 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.621 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.621 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.622 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.622 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.622 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.622 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.622 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.622 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.623 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.623 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.623 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.623 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.623 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.623 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.623 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "stats", 2024-09-19T09:28:17.623 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.624 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.624 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.624 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.624 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.624 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.624 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.624 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.625 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.625 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.625 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.625 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.625 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.625 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.626 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.626 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.626 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.626 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.626 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.626 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.626 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.626 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.626 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.626 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.626 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.626 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.627 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.627 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.627 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.627 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.627 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.627 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.627 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.627 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.627 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.627 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.627 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.628 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.628 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.628 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.628 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.628 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.628 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.628 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.628 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.628 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.628 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.628 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.628 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.628 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.629 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.629 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.629 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.629 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.629 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.629 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.629 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.629 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.629 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.630 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.630 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.630 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.630 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.630 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.630 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.630 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.630 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.630 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.630 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.630 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.630 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.631 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "status", 2024-09-19T09:28:17.631 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.631 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.631 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.631 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.631 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.631 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.631 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.631 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.631 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.631 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.631 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.631 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.631 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.632 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.632 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.632 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.632 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.632 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.632 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.632 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.632 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.632 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.632 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.632 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.633 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.633 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.633 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.633 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.633 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.633 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.633 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.633 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.633 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.633 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.633 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.633 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.634 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.634 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.634 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.634 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.634 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.634 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.634 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.634 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.634 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.634 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.634 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.634 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.634 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.635 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.635 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.635 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.635 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.635 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.635 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.635 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.635 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.635 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.635 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.635 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.635 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.635 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.636 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.636 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.636 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.636 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.636 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.636 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.636 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.636 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.636 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.636 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "telegraf", 2024-09-19T09:28:17.636 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.637 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.637 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.637 INFO:teuthology.orchestra.run.smithi120.stdout: "address": { 2024-09-19T09:28:17.637 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "address", 2024-09-19T09:28:17.637 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.637 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.637 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.637 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T09:28:17.637 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.637 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.637 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.638 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.638 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.638 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.638 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.638 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-19T09:28:17.638 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-19T09:28:17.638 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:17.638 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.638 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.638 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "15", 2024-09-19T09:28:17.638 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.638 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.638 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.639 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.639 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.639 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.639 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.639 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.639 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.639 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.639 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.639 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.639 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.640 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.640 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.640 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.640 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.640 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.640 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.640 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.640 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.640 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.640 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.640 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.640 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.640 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.641 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.641 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.641 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.641 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.641 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.641 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.641 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.641 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.641 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.642 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.642 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.642 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.642 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.642 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.642 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.642 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.642 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.642 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.643 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.643 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.643 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.643 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.643 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.643 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.643 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.643 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.643 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.643 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.643 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.643 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.643 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.643 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.644 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.644 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.644 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.644 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.644 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.644 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.644 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.645 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.645 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.645 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.645 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.645 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.646 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.646 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.646 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.646 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.646 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.647 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "telemetry", 2024-09-19T09:28:17.647 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.647 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.647 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.648 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_basic": { 2024-09-19T09:28:17.648 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_basic", 2024-09-19T09:28:17.648 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.648 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.648 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.648 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.648 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.649 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.649 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.649 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T09:28:17.649 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.649 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.649 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.649 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_crash": { 2024-09-19T09:28:17.649 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_crash", 2024-09-19T09:28:17.649 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.649 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.650 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.650 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.650 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.650 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.650 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.650 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T09:28:17.650 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.650 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.650 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.650 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_device": { 2024-09-19T09:28:17.650 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_device", 2024-09-19T09:28:17.650 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.651 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.651 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.651 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.651 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.651 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.651 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.651 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T09:28:17.651 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.651 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.651 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.651 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_ident": { 2024-09-19T09:28:17.651 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_ident", 2024-09-19T09:28:17.652 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.652 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.652 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.652 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.652 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.652 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.652 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.652 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T09:28:17.652 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.652 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.653 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.653 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_perf": { 2024-09-19T09:28:17.653 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_perf", 2024-09-19T09:28:17.653 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.653 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.653 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.653 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.653 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.653 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.653 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.653 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T09:28:17.653 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.654 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.654 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.654 INFO:teuthology.orchestra.run.smithi120.stdout: "contact": { 2024-09-19T09:28:17.654 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "contact", 2024-09-19T09:28:17.654 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.654 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.654 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.654 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.654 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.654 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.654 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.654 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.654 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.655 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.655 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.655 INFO:teuthology.orchestra.run.smithi120.stdout: "description": { 2024-09-19T09:28:17.655 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "description", 2024-09-19T09:28:17.655 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.655 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.655 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.655 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.655 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.655 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.655 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.655 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.656 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.656 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.656 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.656 INFO:teuthology.orchestra.run.smithi120.stdout: "device_url": { 2024-09-19T09:28:17.656 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "device_url", 2024-09-19T09:28:17.656 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.656 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.656 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.656 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T09:28:17.656 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.657 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.657 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.657 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.657 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.657 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.658 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.658 INFO:teuthology.orchestra.run.smithi120.stdout: "enabled": { 2024-09-19T09:28:17.658 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "enabled", 2024-09-19T09:28:17.658 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.658 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.659 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.659 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.659 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.659 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.659 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.660 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.660 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.660 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.660 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.660 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-19T09:28:17.660 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-19T09:28:17.660 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.660 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.660 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.661 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "24", 2024-09-19T09:28:17.661 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "8", 2024-09-19T09:28:17.661 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.661 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.661 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.661 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.661 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.661 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.661 INFO:teuthology.orchestra.run.smithi120.stdout: "last_opt_revision": { 2024-09-19T09:28:17.661 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "last_opt_revision", 2024-09-19T09:28:17.661 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.661 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.662 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.662 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "1", 2024-09-19T09:28:17.662 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.662 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.662 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.662 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.662 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.662 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.662 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.662 INFO:teuthology.orchestra.run.smithi120.stdout: "leaderboard": { 2024-09-19T09:28:17.662 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "leaderboard", 2024-09-19T09:28:17.662 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.663 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.663 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.663 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.663 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.663 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.663 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.663 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.663 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.663 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.663 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.663 INFO:teuthology.orchestra.run.smithi120.stdout: "leaderboard_description": { 2024-09-19T09:28:17.664 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "leaderboard_description", 2024-09-19T09:28:17.664 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.664 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.664 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.664 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.664 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.664 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.664 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.664 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.664 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.664 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.665 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.665 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.665 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.665 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.665 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.665 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.665 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.665 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.665 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.665 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.665 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.665 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.665 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.666 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.667 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.667 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.667 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.667 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.667 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.667 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.667 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.667 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.667 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.667 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.667 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.667 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.667 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.667 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.668 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.668 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.668 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.668 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.668 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.668 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.668 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.668 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.668 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.668 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.668 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.668 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.668 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.669 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.669 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.669 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.669 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.669 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.669 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.669 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.669 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.669 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.669 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.669 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.669 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.669 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.670 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.670 INFO:teuthology.orchestra.run.smithi120.stdout: "organization": { 2024-09-19T09:28:17.670 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "organization", 2024-09-19T09:28:17.670 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.670 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.670 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.670 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.670 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.670 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.670 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.670 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.670 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.670 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.671 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.671 INFO:teuthology.orchestra.run.smithi120.stdout: "proxy": { 2024-09-19T09:28:17.671 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "proxy", 2024-09-19T09:28:17.671 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.671 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.671 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.671 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.671 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.671 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.671 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.671 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.671 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.671 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.671 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.672 INFO:teuthology.orchestra.run.smithi120.stdout: "url": { 2024-09-19T09:28:17.672 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "url", 2024-09-19T09:28:17.672 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.672 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.672 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.672 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T09:28:17.672 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.672 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.672 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.672 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.672 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.672 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.672 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.673 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.673 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.673 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.673 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "test_orchestrator", 2024-09-19T09:28:17.673 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.673 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.673 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.673 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.673 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.673 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.673 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.674 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.674 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.674 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.674 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.674 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.674 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.674 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.674 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.674 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.674 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.674 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.674 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.674 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.674 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.675 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.675 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.675 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.675 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.675 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.675 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.675 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.675 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.675 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.675 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.675 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.675 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.675 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.676 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.676 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.676 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.676 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.676 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.676 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.676 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.676 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.676 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.676 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.676 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.676 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.677 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.677 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.677 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.677 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.677 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.677 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.677 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.677 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.677 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.677 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.677 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.677 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.678 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.678 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.678 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.678 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.678 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.678 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.678 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.678 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.678 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.678 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.678 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.678 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.678 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.679 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.679 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "volumes", 2024-09-19T09:28:17.679 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.679 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.679 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.679 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.679 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.679 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.679 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.679 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.679 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.679 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.679 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.679 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.680 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.680 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.680 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.680 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.680 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.680 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.680 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.680 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.680 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.680 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.680 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.680 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.681 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.681 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.681 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.681 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.681 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.681 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.681 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.681 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.681 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.681 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.681 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.681 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.682 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.682 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.682 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.682 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.682 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.682 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.682 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.682 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.682 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.682 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.682 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.683 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.683 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.683 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.683 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.683 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.683 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.683 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.684 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.684 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.684 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.684 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.685 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.685 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.685 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.685 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.685 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.685 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.686 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.686 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.686 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.686 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.686 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.687 INFO:teuthology.orchestra.run.smithi120.stdout: "max_concurrent_clones": { 2024-09-19T09:28:17.687 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_concurrent_clones", 2024-09-19T09:28:17.687 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.687 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.687 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.688 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "4", 2024-09-19T09:28:17.688 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.688 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.688 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.688 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T09:28:17.688 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.688 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.688 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.689 INFO:teuthology.orchestra.run.smithi120.stdout: "periodic_async_work": { 2024-09-19T09:28:17.689 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "periodic_async_work", 2024-09-19T09:28:17.689 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.689 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.689 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.689 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.689 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.689 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.689 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.689 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Periodically check for async work", 2024-09-19T09:28:17.689 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.690 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.690 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.690 INFO:teuthology.orchestra.run.smithi120.stdout: "snapshot_clone_delay": { 2024-09-19T09:28:17.690 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "snapshot_clone_delay", 2024-09-19T09:28:17.690 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.690 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.690 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.690 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-19T09:28:17.690 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.690 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.690 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.690 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T09:28:17.690 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.691 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.691 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.691 INFO:teuthology.orchestra.run.smithi120.stdout: "snapshot_clone_no_wait": { 2024-09-19T09:28:17.691 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T09:28:17.691 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.691 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.691 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.691 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-19T09:28:17.692 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.692 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.692 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.692 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T09:28:17.692 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.692 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.692 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.692 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.692 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.693 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.693 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "zabbix", 2024-09-19T09:28:17.693 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-19T09:28:17.693 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-19T09:28:17.693 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-19T09:28:17.693 INFO:teuthology.orchestra.run.smithi120.stdout: "discovery_interval": { 2024-09-19T09:28:17.693 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "discovery_interval", 2024-09-19T09:28:17.693 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-19T09:28:17.693 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.693 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.693 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "100", 2024-09-19T09:28:17.693 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.694 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.694 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.694 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.694 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.694 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.694 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.694 INFO:teuthology.orchestra.run.smithi120.stdout: "identifier": { 2024-09-19T09:28:17.694 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "identifier", 2024-09-19T09:28:17.694 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.694 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.694 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.694 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.694 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.695 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.695 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.695 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.695 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.695 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.695 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.695 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-19T09:28:17.695 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-19T09:28:17.695 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-19T09:28:17.695 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.696 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.696 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-19T09:28:17.696 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.696 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.696 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.696 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.696 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.696 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.696 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.696 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-19T09:28:17.696 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-19T09:28:17.696 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.697 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.697 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.697 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.697 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.697 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.697 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.697 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.697 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.697 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.697 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.697 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.697 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.697 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.698 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.698 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.698 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.698 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.698 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-19T09:28:17.698 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-19T09:28:17.698 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.698 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.698 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.698 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.698 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.698 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.698 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.699 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.699 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.699 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.699 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.699 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-19T09:28:17.699 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-19T09:28:17.699 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.699 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.699 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.699 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-19T09:28:17.699 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.699 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.700 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-19T09:28:17.700 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-19T09:28:17.700 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-19T09:28:17.700 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-19T09:28:17.700 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-19T09:28:17.700 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-19T09:28:17.701 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-19T09:28:17.701 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.701 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.701 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.701 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.701 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.701 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-19T09:28:17.702 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-19T09:28:17.702 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-19T09:28:17.702 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.702 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-19T09:28:17.702 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-19T09:28:17.702 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.702 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.702 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.702 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.702 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.702 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.702 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.703 INFO:teuthology.orchestra.run.smithi120.stdout: "zabbix_host": { 2024-09-19T09:28:17.703 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "zabbix_host", 2024-09-19T09:28:17.703 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.703 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.703 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.703 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-19T09:28:17.703 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.703 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.703 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.703 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.703 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.703 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.703 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.703 INFO:teuthology.orchestra.run.smithi120.stdout: "zabbix_port": { 2024-09-19T09:28:17.704 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "zabbix_port", 2024-09-19T09:28:17.704 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-19T09:28:17.704 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.704 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.704 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10051", 2024-09-19T09:28:17.704 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.704 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.704 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.704 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.704 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.704 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.704 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.705 INFO:teuthology.orchestra.run.smithi120.stdout: "zabbix_sender": { 2024-09-19T09:28:17.705 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "zabbix_sender", 2024-09-19T09:28:17.705 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-19T09:28:17.705 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-19T09:28:17.705 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-19T09:28:17.705 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T09:28:17.705 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-19T09:28:17.705 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-19T09:28:17.705 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-19T09:28:17.705 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-19T09:28:17.705 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-19T09:28:17.706 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-19T09:28:17.706 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-19T09:28:17.706 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.706 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.706 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.706 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.706 INFO:teuthology.orchestra.run.smithi120.stdout: "services": { 2024-09-19T09:28:17.706 INFO:teuthology.orchestra.run.smithi120.stdout: "dashboard": "https://172.21.15.120:8443/" 2024-09-19T09:28:17.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.706 INFO:teuthology.orchestra.run.smithi120.stdout: "always_on_modules": { 2024-09-19T09:28:17.706 INFO:teuthology.orchestra.run.smithi120.stdout: "octopus": [ 2024-09-19T09:28:17.706 INFO:teuthology.orchestra.run.smithi120.stdout: "balancer", 2024-09-19T09:28:17.707 INFO:teuthology.orchestra.run.smithi120.stdout: "crash", 2024-09-19T09:28:17.707 INFO:teuthology.orchestra.run.smithi120.stdout: "devicehealth", 2024-09-19T09:28:17.707 INFO:teuthology.orchestra.run.smithi120.stdout: "orchestrator", 2024-09-19T09:28:17.707 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscaler", 2024-09-19T09:28:17.707 INFO:teuthology.orchestra.run.smithi120.stdout: "progress", 2024-09-19T09:28:17.707 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_support", 2024-09-19T09:28:17.707 INFO:teuthology.orchestra.run.smithi120.stdout: "status", 2024-09-19T09:28:17.707 INFO:teuthology.orchestra.run.smithi120.stdout: "telemetry", 2024-09-19T09:28:17.707 INFO:teuthology.orchestra.run.smithi120.stdout: "volumes" 2024-09-19T09:28:17.707 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.708 INFO:teuthology.orchestra.run.smithi120.stdout: "pacific": [ 2024-09-19T09:28:17.708 INFO:teuthology.orchestra.run.smithi120.stdout: "balancer", 2024-09-19T09:28:17.708 INFO:teuthology.orchestra.run.smithi120.stdout: "crash", 2024-09-19T09:28:17.708 INFO:teuthology.orchestra.run.smithi120.stdout: "devicehealth", 2024-09-19T09:28:17.708 INFO:teuthology.orchestra.run.smithi120.stdout: "orchestrator", 2024-09-19T09:28:17.708 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscaler", 2024-09-19T09:28:17.708 INFO:teuthology.orchestra.run.smithi120.stdout: "progress", 2024-09-19T09:28:17.709 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_support", 2024-09-19T09:28:17.709 INFO:teuthology.orchestra.run.smithi120.stdout: "status", 2024-09-19T09:28:17.709 INFO:teuthology.orchestra.run.smithi120.stdout: "telemetry", 2024-09-19T09:28:17.709 INFO:teuthology.orchestra.run.smithi120.stdout: "volumes" 2024-09-19T09:28:17.709 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.709 INFO:teuthology.orchestra.run.smithi120.stdout: "quincy": [ 2024-09-19T09:28:17.709 INFO:teuthology.orchestra.run.smithi120.stdout: "balancer", 2024-09-19T09:28:17.709 INFO:teuthology.orchestra.run.smithi120.stdout: "crash", 2024-09-19T09:28:17.709 INFO:teuthology.orchestra.run.smithi120.stdout: "devicehealth", 2024-09-19T09:28:17.709 INFO:teuthology.orchestra.run.smithi120.stdout: "orchestrator", 2024-09-19T09:28:17.709 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscaler", 2024-09-19T09:28:17.710 INFO:teuthology.orchestra.run.smithi120.stdout: "progress", 2024-09-19T09:28:17.710 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_support", 2024-09-19T09:28:17.710 INFO:teuthology.orchestra.run.smithi120.stdout: "status", 2024-09-19T09:28:17.710 INFO:teuthology.orchestra.run.smithi120.stdout: "telemetry", 2024-09-19T09:28:17.710 INFO:teuthology.orchestra.run.smithi120.stdout: "volumes" 2024-09-19T09:28:17.710 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.710 INFO:teuthology.orchestra.run.smithi120.stdout: "reef": [ 2024-09-19T09:28:17.710 INFO:teuthology.orchestra.run.smithi120.stdout: "balancer", 2024-09-19T09:28:17.710 INFO:teuthology.orchestra.run.smithi120.stdout: "crash", 2024-09-19T09:28:17.710 INFO:teuthology.orchestra.run.smithi120.stdout: "devicehealth", 2024-09-19T09:28:17.710 INFO:teuthology.orchestra.run.smithi120.stdout: "orchestrator", 2024-09-19T09:28:17.711 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscaler", 2024-09-19T09:28:17.711 INFO:teuthology.orchestra.run.smithi120.stdout: "progress", 2024-09-19T09:28:17.711 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_support", 2024-09-19T09:28:17.711 INFO:teuthology.orchestra.run.smithi120.stdout: "status", 2024-09-19T09:28:17.711 INFO:teuthology.orchestra.run.smithi120.stdout: "telemetry", 2024-09-19T09:28:17.712 INFO:teuthology.orchestra.run.smithi120.stdout: "volumes" 2024-09-19T09:28:17.712 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-19T09:28:17.712 INFO:teuthology.orchestra.run.smithi120.stdout: "squid": [ 2024-09-19T09:28:17.712 INFO:teuthology.orchestra.run.smithi120.stdout: "balancer", 2024-09-19T09:28:17.712 INFO:teuthology.orchestra.run.smithi120.stdout: "crash", 2024-09-19T09:28:17.713 INFO:teuthology.orchestra.run.smithi120.stdout: "devicehealth", 2024-09-19T09:28:17.713 INFO:teuthology.orchestra.run.smithi120.stdout: "orchestrator", 2024-09-19T09:28:17.713 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscaler", 2024-09-19T09:28:17.713 INFO:teuthology.orchestra.run.smithi120.stdout: "progress", 2024-09-19T09:28:17.713 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_support", 2024-09-19T09:28:17.714 INFO:teuthology.orchestra.run.smithi120.stdout: "status", 2024-09-19T09:28:17.714 INFO:teuthology.orchestra.run.smithi120.stdout: "telemetry", 2024-09-19T09:28:17.714 INFO:teuthology.orchestra.run.smithi120.stdout: "volumes" 2024-09-19T09:28:17.714 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-19T09:28:17.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.715 INFO:teuthology.orchestra.run.smithi120.stdout: "last_failure_osd_epoch": 3, 2024-09-19T09:28:17.715 INFO:teuthology.orchestra.run.smithi120.stdout: "active_clients": [ 2024-09-19T09:28:17.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.715 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "devicehealth", 2024-09-19T09:28:17.715 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-19T09:28:17.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.715 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-19T09:28:17.715 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:0", 2024-09-19T09:28:17.715 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3651871805 2024-09-19T09:28:17.716 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.716 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-19T09:28:17.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.716 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "libcephsqlite", 2024-09-19T09:28:17.716 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-19T09:28:17.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.716 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-19T09:28:17.716 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:0", 2024-09-19T09:28:17.716 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 3713436834 2024-09-19T09:28:17.716 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.716 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-19T09:28:17.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.717 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rbd_support", 2024-09-19T09:28:17.717 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-19T09:28:17.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.717 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-19T09:28:17.717 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:0", 2024-09-19T09:28:17.718 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2672339263 2024-09-19T09:28:17.718 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.718 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-19T09:28:17.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:17.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.718 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "volumes", 2024-09-19T09:28:17.719 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-19T09:28:17.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-19T09:28:17.719 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-19T09:28:17.719 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:0", 2024-09-19T09:28:17.719 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2066332567 2024-09-19T09:28:17.719 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.720 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-19T09:28:17.720 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:17.720 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-19T09:28:17.720 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:28:17.723 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:16 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3553260181' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T09:28:17.723 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:16 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1153944687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T09:28:17.723 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:16 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3148083081' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T09:28:17.723 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:16 smithi153 ceph-mon[35071]: pgmap v1107: 97 pgs: 97 active+clean; 302 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 404 op/s 2024-09-19T09:28:17.723 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:16 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/61509941' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T09:28:17.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:16 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3553260181' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T09:28:17.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:16 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1153944687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T09:28:17.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:16 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3148083081' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T09:28:17.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:16 smithi196 ceph-mon[35150]: pgmap v1107: 97 pgs: 97 active+clean; 302 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 404 op/s 2024-09-19T09:28:17.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:16 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/61509941' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T09:28:17.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.350+0000 7f84c2e77640 1 -- 172.21.15.120:0/61509941 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f84ac0a32d0 msgr2=0x7f84ac0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:17.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.350+0000 7f84c2e77640 1 --2- 172.21.15.120:0/61509941 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f84ac0a32d0 0x7f84ac0a5790 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f84bc102b80 tx=0x7f84b0006c90 comp rx=0 tx=0).stop 2024-09-19T09:28:17.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.350+0000 7f84c2e77640 1 -- 172.21.15.120:0/61509941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84bc1036e0 msgr2=0x7f84bc1a1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:17.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.350+0000 7f84c2e77640 1 --2- 172.21.15.120:0/61509941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84bc1036e0 0x7f84bc1a1690 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7f84b400cce0 tx=0x7f84b400a620 comp rx=0 tx=0).stop 2024-09-19T09:28:17.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.350+0000 7f84c2e77640 1 -- 172.21.15.120:0/61509941 shutdown_connections 2024-09-19T09:28:17.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.350+0000 7f84c2e77640 1 --2- 172.21.15.120:0/61509941 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f84ac0a32d0 0x7f84ac0a5790 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:17.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.350+0000 7f84c2e77640 1 --2- 172.21.15.120:0/61509941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84bc1036e0 0x7f84bc1a1690 unknown :-1 s=CLOSED pgs=2573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:17.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.350+0000 7f84c2e77640 1 --2- 172.21.15.120:0/61509941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84bc102d20 0x7f84bc19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:17.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.350+0000 7f84c2e77640 1 --2- 172.21.15.120:0/61509941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84bc101b30 0x7f84bc19a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:17.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.350+0000 7f84c2e77640 1 -- 172.21.15.120:0/61509941 >> 172.21.15.120:0/61509941 conn(0x7f84bc0fd2b0 msgr2=0x7f84bc074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:17.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.350+0000 7f84c2e77640 1 -- 172.21.15.120:0/61509941 shutdown_connections 2024-09-19T09:28:17.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:16.350+0000 7f84c2e77640 1 -- 172.21.15.120:0/61509941 wait complete. 2024-09-19T09:28:17.793 DEBUG:tasks.check_counter:Getting stats from a 2024-09-19T09:28:17.793 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-19T09:28:17.794 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:16 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3553260181' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T09:28:17.794 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:16 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1153944687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T09:28:17.794 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:16 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3148083081' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T09:28:17.794 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:16 smithi120 ceph-mon[31150]: pgmap v1107: 97 pgs: 97 active+clean; 302 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 404 op/s 2024-09-19T09:28:17.794 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:16 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/61509941' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T09:28:18.049 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:18.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:18 smithi196 ceph-mon[35150]: pgmap v1108: 97 pgs: 97 active+clean; 302 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 358 op/s 2024-09-19T09:28:18.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:18 smithi153 ceph-mon[35071]: pgmap v1108: 97 pgs: 97 active+clean; 302 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 358 op/s 2024-09-19T09:28:18.474 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:18 smithi120 ceph-mon[31150]: pgmap v1108: 97 pgs: 97 active+clean; 302 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 358 op/s 2024-09-19T09:28:18.591 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:28:18.591 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:28:18.591 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 2400, 2024-09-19T09:28:18.591 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 2378, 2024-09-19T09:28:18.591 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 51901269, 2024-09-19T09:28:18.591 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 887080, 2024-09-19T09:28:18.591 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 7, 2024-09-19T09:28:18.591 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 3, 2024-09-19T09:28:18.592 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.259329375, 2024-09-19T09:28:18.592 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.083075220, 2024-09-19T09:28:18.592 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.110293848, 2024-09-19T09:28:18.592 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.056750500, 2024-09-19T09:28:18.592 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:18.592 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 2378, 2024-09-19T09:28:18.592 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.093890713, 2024-09-19T09:28:18.592 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000039483 2024-09-19T09:28:18.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.592 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:18.592 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.592 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.592 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.593 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 942064, 2024-09-19T09:28:18.593 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 275248 2024-09-19T09:28:18.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.593 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:28:18.593 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 1638, 2024-09-19T09:28:18.593 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 1973, 2024-09-19T09:28:18.593 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 109462506, 2024-09-19T09:28:18.593 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 1197642, 2024-09-19T09:28:18.593 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 8, 2024-09-19T09:28:18.593 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 6, 2024-09-19T09:28:18.593 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.217181554, 2024-09-19T09:28:18.593 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.053175196, 2024-09-19T09:28:18.593 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.102527544, 2024-09-19T09:28:18.594 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.048506485, 2024-09-19T09:28:18.594 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:18.594 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1971, 2024-09-19T09:28:18.594 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.063418935, 2024-09-19T09:28:18.594 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000032176 2024-09-19T09:28:18.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.594 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:18.594 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.594 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.594 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.594 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T09:28:18.594 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 677008 2024-09-19T09:28:18.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.595 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:28:18.595 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 1908, 2024-09-19T09:28:18.595 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 1910, 2024-09-19T09:28:18.595 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 134600702, 2024-09-19T09:28:18.595 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 606745, 2024-09-19T09:28:18.595 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 7, 2024-09-19T09:28:18.595 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 5, 2024-09-19T09:28:18.595 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.219533721, 2024-09-19T09:28:18.595 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.042979289, 2024-09-19T09:28:18.596 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.119346147, 2024-09-19T09:28:18.596 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.044472193, 2024-09-19T09:28:18.596 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:18.596 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1908, 2024-09-19T09:28:18.596 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.054081697, 2024-09-19T09:28:18.596 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000028344 2024-09-19T09:28:18.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.596 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:18.596 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.596 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.597 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.597 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T09:28:18.597 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T09:28:18.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.597 INFO:teuthology.orchestra.run.smithi120.stdout: "cct": { 2024-09-19T09:28:18.597 INFO:teuthology.orchestra.run.smithi120.stdout: "total_workers": 1, 2024-09-19T09:28:18.597 INFO:teuthology.orchestra.run.smithi120.stdout: "unhealthy_workers": 0 2024-09-19T09:28:18.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.597 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-MDSRank": { 2024-09-19T09:28:18.597 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-09-19T09:28:18.597 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-09-19T09:28:18.597 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 8980, 2024-09-19T09:28:18.598 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.227077259, 2024-09-19T09:28:18.598 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000025286 2024-09-19T09:28:18.598 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:18.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.598 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-PurgeQueue": { 2024-09-19T09:28:18.598 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-09-19T09:28:18.598 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-09-19T09:28:18.598 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.598 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.598 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.598 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:18.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.598 INFO:teuthology.orchestra.run.smithi120.stdout: "mds": { 2024-09-19T09:28:18.599 INFO:teuthology.orchestra.run.smithi120.stdout: "request": 0, 2024-09-19T09:28:18.599 INFO:teuthology.orchestra.run.smithi120.stdout: "reply": 0, 2024-09-19T09:28:18.599 INFO:teuthology.orchestra.run.smithi120.stdout: "reply_latency": { 2024-09-19T09:28:18.599 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.599 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.599 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.599 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_reply": 0, 2024-09-19T09:28:18.599 INFO:teuthology.orchestra.run.smithi120.stdout: "forward": 0, 2024-09-19T09:28:18.599 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_complete": 0, 2024-09-19T09:28:18.599 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_keys": 0, 2024-09-19T09:28:18.599 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_commit": 0, 2024-09-19T09:28:18.599 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_split": 0, 2024-09-19T09:28:18.599 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_merge": 0, 2024-09-19T09:28:18.599 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes": 29730, 2024-09-19T09:28:18.600 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_top": 3308, 2024-09-19T09:28:18.600 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_bottom": 26422, 2024-09-19T09:28:18.600 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pin_tail": 0, 2024-09-19T09:28:18.600 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pinned": 25004, 2024-09-19T09:28:18.600 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_expired": 0, 2024-09-19T09:28:18.600 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_with_caps": 0, 2024-09-19T09:28:18.600 INFO:teuthology.orchestra.run.smithi120.stdout: "caps": 0, 2024-09-19T09:28:18.600 INFO:teuthology.orchestra.run.smithi120.stdout: "subtrees": 1, 2024-09-19T09:28:18.600 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse": 0, 2024-09-19T09:28:18.600 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_hit": 0, 2024-09-19T09:28:18.600 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_forward": 0, 2024-09-19T09:28:18.600 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_discover": 0, 2024-09-19T09:28:18.600 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_dir_fetch": 0, 2024-09-19T09:28:18.600 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_remote_ino": 0, 2024-09-19T09:28:18.601 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_lock": 0, 2024-09-19T09:28:18.601 INFO:teuthology.orchestra.run.smithi120.stdout: "load_cent": 0, 2024-09-19T09:28:18.601 INFO:teuthology.orchestra.run.smithi120.stdout: "q": 0, 2024-09-19T09:28:18.601 INFO:teuthology.orchestra.run.smithi120.stdout: "exported": 0, 2024-09-19T09:28:18.601 INFO:teuthology.orchestra.run.smithi120.stdout: "exported_inodes": 0, 2024-09-19T09:28:18.601 INFO:teuthology.orchestra.run.smithi120.stdout: "imported": 0, 2024-09-19T09:28:18.601 INFO:teuthology.orchestra.run.smithi120.stdout: "imported_inodes": 0, 2024-09-19T09:28:18.601 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_dir_fetch": 0, 2024-09-19T09:28:18.601 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_backtrace_fetch": 0, 2024-09-19T09:28:18.601 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_peer_discover": 0, 2024-09-19T09:28:18.601 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rfiles": 0, 2024-09-19T09:28:18.601 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rbytes": 0, 2024-09-19T09:28:18.601 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rsnaps": 0, 2024-09-19T09:28:18.602 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T09:28:18.602 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_set_tag": 0, 2024-09-19T09:28:18.602 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:28:18.602 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:28:18.602 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_inodes": 0, 2024-09-19T09:28:18.602 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T09:28:18.602 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T09:28:18.602 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_file_inodes": 0, 2024-09-19T09:28:18.602 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_inode_file_caps": 0, 2024-09-19T09:28:18.602 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T09:28:18.603 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_grant": 0, 2024-09-19T09:28:18.603 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T09:28:18.603 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:28:18.603 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:28:18.603 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps": 0, 2024-09-19T09:28:18.603 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps_dirty": 0, 2024-09-19T09:28:18.603 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_cap_release": 0, 2024-09-19T09:28:18.603 INFO:teuthology.orchestra.run.smithi120.stdout: "process_request_cap_release": 0 2024-09-19T09:28:18.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.603 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache": { 2024-09-19T09:28:18.603 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update": 0, 2024-09-19T09:28:18.603 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update_receipt": 0, 2024-09-19T09:28:18.603 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_try_discover": 0, 2024-09-19T09:28:18.603 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_send_discover": 0, 2024-09-19T09:28:18.603 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_handle_discover": 0, 2024-09-19T09:28:18.604 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays": 0, 2024-09-19T09:28:18.604 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_delayed": 0, 2024-09-19T09:28:18.604 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:28:18.604 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_created": 0, 2024-09-19T09:28:18.604 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_enqueued": 0, 2024-09-19T09:28:18.604 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_reintegrated": 0, 2024-09-19T09:28:18.604 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_migrated": 0, 2024-09-19T09:28:18.604 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_processing": 0, 2024-09-19T09:28:18.604 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:28:18.604 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:28:18.604 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_started": 0, 2024-09-19T09:28:18.604 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_completed": 0, 2024-09-19T09:28:18.605 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_path": 0, 2024-09-19T09:28:18.605 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_inode": 0, 2024-09-19T09:28:18.605 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T09:28:18.605 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_exportdir": 0, 2024-09-19T09:28:18.605 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_flush": 0, 2024-09-19T09:28:18.605 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragmentdir": 0, 2024-09-19T09:28:18.605 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragstats": 0, 2024-09-19T09:28:18.605 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_inodestats": 0 2024-09-19T09:28:18.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.605 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log": { 2024-09-19T09:28:18.605 INFO:teuthology.orchestra.run.smithi120.stdout: "evlrg": 0, 2024-09-19T09:28:18.606 INFO:teuthology.orchestra.run.smithi120.stdout: "evadd": 0, 2024-09-19T09:28:18.606 INFO:teuthology.orchestra.run.smithi120.stdout: "evex": 0, 2024-09-19T09:28:18.606 INFO:teuthology.orchestra.run.smithi120.stdout: "evtrm": 0, 2024-09-19T09:28:18.606 INFO:teuthology.orchestra.run.smithi120.stdout: "ev": 138855, 2024-09-19T09:28:18.606 INFO:teuthology.orchestra.run.smithi120.stdout: "evexg": 0, 2024-09-19T09:28:18.606 INFO:teuthology.orchestra.run.smithi120.stdout: "evexd": 0, 2024-09-19T09:28:18.606 INFO:teuthology.orchestra.run.smithi120.stdout: "segadd": 0, 2024-09-19T09:28:18.606 INFO:teuthology.orchestra.run.smithi120.stdout: "segex": 0, 2024-09-19T09:28:18.606 INFO:teuthology.orchestra.run.smithi120.stdout: "segtrm": 0, 2024-09-19T09:28:18.606 INFO:teuthology.orchestra.run.smithi120.stdout: "seg": 133, 2024-09-19T09:28:18.606 INFO:teuthology.orchestra.run.smithi120.stdout: "segmjr": 13, 2024-09-19T09:28:18.606 INFO:teuthology.orchestra.run.smithi120.stdout: "segexg": 0, 2024-09-19T09:28:18.607 INFO:teuthology.orchestra.run.smithi120.stdout: "segexd": 0, 2024-09-19T09:28:18.607 INFO:teuthology.orchestra.run.smithi120.stdout: "expos": 57515232, 2024-09-19T09:28:18.607 INFO:teuthology.orchestra.run.smithi120.stdout: "wrpos": 297684861, 2024-09-19T09:28:18.607 INFO:teuthology.orchestra.run.smithi120.stdout: "rdpos": 297682577, 2024-09-19T09:28:18.607 INFO:teuthology.orchestra.run.smithi120.stdout: "jlat": { 2024-09-19T09:28:18.607 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.607 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.607 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.607 INFO:teuthology.orchestra.run.smithi120.stdout: "replayed": 138855 2024-09-19T09:28:18.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.607 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_mem": { 2024-09-19T09:28:18.607 INFO:teuthology.orchestra.run.smithi120.stdout: "ino": 368, 2024-09-19T09:28:18.607 INFO:teuthology.orchestra.run.smithi120.stdout: "ino+": 29743, 2024-09-19T09:28:18.608 INFO:teuthology.orchestra.run.smithi120.stdout: "ino-": 29375, 2024-09-19T09:28:18.608 INFO:teuthology.orchestra.run.smithi120.stdout: "dir": 11, 2024-09-19T09:28:18.608 INFO:teuthology.orchestra.run.smithi120.stdout: "dir+": 149, 2024-09-19T09:28:18.608 INFO:teuthology.orchestra.run.smithi120.stdout: "dir-": 138, 2024-09-19T09:28:18.608 INFO:teuthology.orchestra.run.smithi120.stdout: "dn": 29730, 2024-09-19T09:28:18.608 INFO:teuthology.orchestra.run.smithi120.stdout: "dn+": 29887, 2024-09-19T09:28:18.608 INFO:teuthology.orchestra.run.smithi120.stdout: "dn-": 157, 2024-09-19T09:28:18.608 INFO:teuthology.orchestra.run.smithi120.stdout: "cap": 0, 2024-09-19T09:28:18.608 INFO:teuthology.orchestra.run.smithi120.stdout: "cap+": 0, 2024-09-19T09:28:18.608 INFO:teuthology.orchestra.run.smithi120.stdout: "cap-": 0, 2024-09-19T09:28:18.608 INFO:teuthology.orchestra.run.smithi120.stdout: "rss": 81132, 2024-09-19T09:28:18.608 INFO:teuthology.orchestra.run.smithi120.stdout: "heap": 215328 2024-09-19T09:28:18.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.608 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_server": { 2024-09-19T09:28:18.609 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_client_request": 0, 2024-09-19T09:28:18.609 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_server_request": 0, 2024-09-19T09:28:18.609 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_request": 0, 2024-09-19T09:28:18.609 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_session": 0, 2024-09-19T09:28:18.609 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_peer_request": 0, 2024-09-19T09:28:18.610 INFO:teuthology.orchestra.run.smithi120.stdout: "req_create_latency": { 2024-09-19T09:28:18.610 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.610 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.610 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.610 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getattr_latency": { 2024-09-19T09:28:18.610 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.610 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.610 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.610 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getfilelock_latency": { 2024-09-19T09:28:18.610 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.610 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.611 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.611 INFO:teuthology.orchestra.run.smithi120.stdout: "req_link_latency": { 2024-09-19T09:28:18.611 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.611 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.611 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.611 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookup_latency": { 2024-09-19T09:28:18.611 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.611 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.611 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.612 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookuphash_latency": { 2024-09-19T09:28:18.612 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.612 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.612 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.612 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupino_latency": { 2024-09-19T09:28:18.612 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.612 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.612 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.612 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupname_latency": { 2024-09-19T09:28:18.612 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.612 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.612 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.613 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupparent_latency": { 2024-09-19T09:28:18.613 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.613 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.613 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.613 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupsnap_latency": { 2024-09-19T09:28:18.613 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.613 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.613 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.613 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lssnap_latency": { 2024-09-19T09:28:18.613 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.613 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.613 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.614 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mkdir_latency": { 2024-09-19T09:28:18.614 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.614 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.614 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.614 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mknod_latency": { 2024-09-19T09:28:18.614 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.614 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.614 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.614 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mksnap_latency": { 2024-09-19T09:28:18.614 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.614 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.615 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.615 INFO:teuthology.orchestra.run.smithi120.stdout: "req_open_latency": { 2024-09-19T09:28:18.615 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.615 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.615 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.615 INFO:teuthology.orchestra.run.smithi120.stdout: "req_readdir_latency": { 2024-09-19T09:28:18.615 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.615 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.615 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.615 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rename_latency": { 2024-09-19T09:28:18.615 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.616 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.616 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.616 INFO:teuthology.orchestra.run.smithi120.stdout: "req_renamesnap_latency": { 2024-09-19T09:28:18.616 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.616 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.616 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.616 INFO:teuthology.orchestra.run.smithi120.stdout: "req_snapdiff_latency": { 2024-09-19T09:28:18.616 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.616 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.616 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.616 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmdir_latency": { 2024-09-19T09:28:18.617 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.617 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.617 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.617 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmsnap_latency": { 2024-09-19T09:28:18.617 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.617 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.617 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.617 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmxattr_latency": { 2024-09-19T09:28:18.617 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.617 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.617 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.618 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setattr_latency": { 2024-09-19T09:28:18.618 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.618 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.618 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.618 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setdirlayout_latency": { 2024-09-19T09:28:18.618 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.618 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.618 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.618 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setfilelock_latency": { 2024-09-19T09:28:18.618 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.618 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.619 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.619 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setlayout_latency": { 2024-09-19T09:28:18.619 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.619 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.619 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.619 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setxattr_latency": { 2024-09-19T09:28:18.619 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.619 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.619 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.619 INFO:teuthology.orchestra.run.smithi120.stdout: "req_symlink_latency": { 2024-09-19T09:28:18.619 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.619 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.619 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.620 INFO:teuthology.orchestra.run.smithi120.stdout: "req_unlink_latency": { 2024-09-19T09:28:18.620 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.620 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.620 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.620 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:28:18.620 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:28:18.620 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getvxattr_latency": { 2024-09-19T09:28:18.620 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.620 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.620 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.620 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:18.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.621 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_sessions": { 2024-09-19T09:28:18.621 INFO:teuthology.orchestra.run.smithi120.stdout: "session_count": 1, 2024-09-19T09:28:18.621 INFO:teuthology.orchestra.run.smithi120.stdout: "session_add": 2, 2024-09-19T09:28:18.621 INFO:teuthology.orchestra.run.smithi120.stdout: "session_remove": 1, 2024-09-19T09:28:18.621 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_open": 2, 2024-09-19T09:28:18.621 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_stale": 0, 2024-09-19T09:28:18.621 INFO:teuthology.orchestra.run.smithi120.stdout: "total_load": 0, 2024-09-19T09:28:18.621 INFO:teuthology.orchestra.run.smithi120.stdout: "average_load": 0, 2024-09-19T09:28:18.621 INFO:teuthology.orchestra.run.smithi120.stdout: "avg_session_uptime": 3680, 2024-09-19T09:28:18.621 INFO:teuthology.orchestra.run.smithi120.stdout: "mdthresh_evicted": 0 2024-09-19T09:28:18.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.621 INFO:teuthology.orchestra.run.smithi120.stdout: "mempool": { 2024-09-19T09:28:18.621 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:28:18.621 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_items": 0, 2024-09-19T09:28:18.622 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:28:18.622 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:28:18.622 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:28:18.622 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:28:18.622 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:28:18.622 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:28:18.622 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:28:18.622 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:28:18.622 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:28:18.622 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:28:18.622 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:28:18.622 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:28:18.622 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:28:18.623 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_items": 0, 2024-09-19T09:28:18.623 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:28:18.623 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_items": 0, 2024-09-19T09:28:18.623 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:28:18.623 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:28:18.623 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:28:18.623 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:28:18.623 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:28:18.623 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:28:18.623 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:28:18.623 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_items": 0, 2024-09-19T09:28:18.623 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:28:18.623 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:28:18.624 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:28:18.624 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_items": 0, 2024-09-19T09:28:18.624 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_bytes": 0, 2024-09-19T09:28:18.624 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_items": 0, 2024-09-19T09:28:18.624 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:28:18.624 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:28:18.624 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:28:18.624 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:28:18.624 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_bytes": 252606, 2024-09-19T09:28:18.624 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_items": 434, 2024-09-19T09:28:18.624 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:28:18.624 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_items": 0, 2024-09-19T09:28:18.624 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bytes": 0, 2024-09-19T09:28:18.624 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_items": 0, 2024-09-19T09:28:18.625 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:28:18.625 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_items": 0, 2024-09-19T09:28:18.625 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:28:18.625 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_items": 0, 2024-09-19T09:28:18.625 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_bytes": 7592, 2024-09-19T09:28:18.625 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_items": 156, 2024-09-19T09:28:18.625 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:28:18.625 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:28:18.625 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_bytes": 0, 2024-09-19T09:28:18.625 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_items": 0, 2024-09-19T09:28:18.625 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_bytes": 21844393, 2024-09-19T09:28:18.625 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_items": 165974, 2024-09-19T09:28:18.625 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_bytes": 0, 2024-09-19T09:28:18.625 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_items": 0, 2024-09-19T09:28:18.626 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_bytes": 0, 2024-09-19T09:28:18.626 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_items": 0 2024-09-19T09:28:18.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.626 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter": { 2024-09-19T09:28:18.626 INFO:teuthology.orchestra.run.smithi120.stdout: "op_active": 0, 2024-09-19T09:28:18.626 INFO:teuthology.orchestra.run.smithi120.stdout: "op_laggy": 0, 2024-09-19T09:28:18.626 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send": 5469, 2024-09-19T09:28:18.626 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send_bytes": 21, 2024-09-19T09:28:18.626 INFO:teuthology.orchestra.run.smithi120.stdout: "op_resend": 3, 2024-09-19T09:28:18.626 INFO:teuthology.orchestra.run.smithi120.stdout: "op_reply": 5469, 2024-09-19T09:28:18.626 INFO:teuthology.orchestra.run.smithi120.stdout: "op_latency": { 2024-09-19T09:28:18.626 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 5469, 2024-09-19T09:28:18.626 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 9.649769993, 2024-09-19T09:28:18.626 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.001764448 2024-09-19T09:28:18.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.627 INFO:teuthology.orchestra.run.smithi120.stdout: "op_inflight": 0, 2024-09-19T09:28:18.627 INFO:teuthology.orchestra.run.smithi120.stdout: "oplen_avg": { 2024-09-19T09:28:18.627 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 5469, 2024-09-19T09:28:18.627 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 5470 2024-09-19T09:28:18.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.627 INFO:teuthology.orchestra.run.smithi120.stdout: "op": 5469, 2024-09-19T09:28:18.627 INFO:teuthology.orchestra.run.smithi120.stdout: "op_r": 5469, 2024-09-19T09:28:18.627 INFO:teuthology.orchestra.run.smithi120.stdout: "op_w": 0, 2024-09-19T09:28:18.627 INFO:teuthology.orchestra.run.smithi120.stdout: "op_rmw": 0, 2024-09-19T09:28:18.627 INFO:teuthology.orchestra.run.smithi120.stdout: "op_pg": 0, 2024-09-19T09:28:18.627 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_stat": 3253, 2024-09-19T09:28:18.627 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_create": 0, 2024-09-19T09:28:18.627 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_read": 2214, 2024-09-19T09:28:18.627 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_write": 0, 2024-09-19T09:28:18.628 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writefull": 0, 2024-09-19T09:28:18.628 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writesame": 0, 2024-09-19T09:28:18.628 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_append": 0, 2024-09-19T09:28:18.628 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_zero": 0, 2024-09-19T09:28:18.628 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_truncate": 0, 2024-09-19T09:28:18.628 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_delete": 0, 2024-09-19T09:28:18.628 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_mapext": 0, 2024-09-19T09:28:18.628 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_sparse_read": 0, 2024-09-19T09:28:18.628 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_clonerange": 0, 2024-09-19T09:28:18.628 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_getxattr": 1, 2024-09-19T09:28:18.628 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_setxattr": 0, 2024-09-19T09:28:18.628 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:28:18.628 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_rmxattr": 0, 2024-09-19T09:28:18.628 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:28:18.629 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_call": 0, 2024-09-19T09:28:18.629 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_watch": 0, 2024-09-19T09:28:18.629 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_notify": 0, 2024-09-19T09:28:18.629 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:28:18.629 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls": 0, 2024-09-19T09:28:18.629 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:28:18.629 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_other": 0, 2024-09-19T09:28:18.629 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_active": 0, 2024-09-19T09:28:18.629 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_send": 0, 2024-09-19T09:28:18.629 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_resend": 0, 2024-09-19T09:28:18.629 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_ping": 0, 2024-09-19T09:28:18.629 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_active": 0, 2024-09-19T09:28:18.629 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_send": 0, 2024-09-19T09:28:18.629 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_resend": 0, 2024-09-19T09:28:18.630 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_active": 0, 2024-09-19T09:28:18.630 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_send": 0, 2024-09-19T09:28:18.630 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_resend": 0, 2024-09-19T09:28:18.630 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_active": 0, 2024-09-19T09:28:18.630 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_send": 0, 2024-09-19T09:28:18.630 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_resend": 0, 2024-09-19T09:28:18.630 INFO:teuthology.orchestra.run.smithi120.stdout: "command_active": 0, 2024-09-19T09:28:18.630 INFO:teuthology.orchestra.run.smithi120.stdout: "command_send": 0, 2024-09-19T09:28:18.630 INFO:teuthology.orchestra.run.smithi120.stdout: "command_resend": 0, 2024-09-19T09:28:18.630 INFO:teuthology.orchestra.run.smithi120.stdout: "map_epoch": 85, 2024-09-19T09:28:18.630 INFO:teuthology.orchestra.run.smithi120.stdout: "map_full": 0, 2024-09-19T09:28:18.630 INFO:teuthology.orchestra.run.smithi120.stdout: "map_inc": 6, 2024-09-19T09:28:18.630 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_sessions": 11, 2024-09-19T09:28:18.631 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_open": 13, 2024-09-19T09:28:18.631 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_close": 2, 2024-09-19T09:28:18.631 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_laggy": 0, 2024-09-19T09:28:18.631 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_wr": 0, 2024-09-19T09:28:18.631 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_rd": 2, 2024-09-19T09:28:18.631 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_del": 0 2024-09-19T09:28:18.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.631 INFO:teuthology.orchestra.run.smithi120.stdout: "oft": { 2024-09-19T09:28:18.631 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_objs": 0, 2024-09-19T09:28:18.631 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:28:18.631 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_updates": 0, 2024-09-19T09:28:18.631 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_removes": 0 2024-09-19T09:28:18.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.631 INFO:teuthology.orchestra.run.smithi120.stdout: "osd-slow-ops": { 2024-09-19T09:28:18.631 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_ops_count": 0 2024-09-19T09:28:18.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.632 INFO:teuthology.orchestra.run.smithi120.stdout: "purge_queue": { 2024-09-19T09:28:18.632 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops": 0, 2024-09-19T09:28:18.632 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T09:28:18.632 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing": 0, 2024-09-19T09:28:18.632 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_high_water": 0, 2024-09-19T09:28:18.632 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed_ops": 0, 2024-09-19T09:28:18.632 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed": 0, 2024-09-19T09:28:18.632 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_item_in_journal": 0 2024-09-19T09:28:18.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.632 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:28:18.632 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-19T09:28:18.632 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-09-19T09:28:18.633 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-19T09:28:18.633 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 5942, 2024-09-19T09:28:18.633 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 295402488, 2024-09-19T09:28:18.633 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:18.633 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 5942, 2024-09-19T09:28:18.633 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-19T09:28:18.633 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-19T09:28:18.633 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 5942, 2024-09-19T09:28:18.633 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 295402488, 2024-09-19T09:28:18.633 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-19T09:28:18.633 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.633 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.634 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.634 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:18.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.634 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_bytes": { 2024-09-19T09:28:18.634 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-19T09:28:18.634 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-09-19T09:28:18.634 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-19T09:28:18.634 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-09-19T09:28:18.634 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-09-19T09:28:18.634 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:18.634 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:18.634 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 5469, 2024-09-19T09:28:18.634 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 293496385, 2024-09-19T09:28:18.634 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 5469, 2024-09-19T09:28:18.635 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 293496385, 2024-09-19T09:28:18.635 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-19T09:28:18.635 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.635 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.635 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.635 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:18.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.635 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_ops": { 2024-09-19T09:28:18.635 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-19T09:28:18.635 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 1024, 2024-09-19T09:28:18.635 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-19T09:28:18.635 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-09-19T09:28:18.635 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-09-19T09:28:18.635 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:18.636 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:18.636 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 5469, 2024-09-19T09:28:18.636 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 5469, 2024-09-19T09:28:18.636 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 5469, 2024-09-19T09:28:18.636 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 5469, 2024-09-19T09:28:18.636 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-19T09:28:18.636 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.636 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.636 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.636 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:18.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.636 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:28:18.636 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-19T09:28:18.637 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-09-19T09:28:18.637 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-19T09:28:18.637 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-09-19T09:28:18.637 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-09-19T09:28:18.637 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:18.637 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:18.637 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-19T09:28:18.637 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-19T09:28:18.637 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 0, 2024-09-19T09:28:18.637 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 0, 2024-09-19T09:28:18.637 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-19T09:28:18.637 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.637 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.637 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.638 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:18.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:18.638 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle-0x562bc4f54b40": { 2024-09-19T09:28:18.638 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-19T09:28:18.638 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-09-19T09:28:18.638 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-19T09:28:18.638 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-09-19T09:28:18.638 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-09-19T09:28:18.638 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:18.638 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:18.638 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-19T09:28:18.639 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-19T09:28:18.639 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 0, 2024-09-19T09:28:18.639 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 0, 2024-09-19T09:28:18.639 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-19T09:28:18.639 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:18.639 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:18.639 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:18.639 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:18.639 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:18.639 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:28:19.519 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-19T09:28:19.519 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-19T09:28:19.519 DEBUG:tasks.check_counter:Getting stats from d 2024-09-19T09:28:19.520 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-19T09:28:19.764 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:20.202 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:20 smithi120 ceph-mon[31150]: pgmap v1109: 97 pgs: 97 active+clean; 302 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 203 op/s 2024-09-19T09:28:20.327 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:28:20.327 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:28:20.328 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 4, 2024-09-19T09:28:20.328 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 4, 2024-09-19T09:28:20.328 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 3024, 2024-09-19T09:28:20.328 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 448, 2024-09-19T09:28:20.328 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 4, 2024-09-19T09:28:20.328 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 0, 2024-09-19T09:28:20.328 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.003044920, 2024-09-19T09:28:20.328 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.000034588, 2024-09-19T09:28:20.328 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.002156848, 2024-09-19T09:28:20.328 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T09:28:20.328 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:20.328 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 3, 2024-09-19T09:28:20.329 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.003908396, 2024-09-19T09:28:20.329 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.001302798 2024-09-19T09:28:20.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:20.329 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:20.329 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:20.329 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:20.329 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:20.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:20.329 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-19T09:28:20.329 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T09:28:20.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:20.329 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:28:20.329 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 452, 2024-09-19T09:28:20.330 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 447, 2024-09-19T09:28:20.330 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 823232, 2024-09-19T09:28:20.330 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 276752, 2024-09-19T09:28:20.330 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 2, 2024-09-19T09:28:20.330 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 1, 2024-09-19T09:28:20.330 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.052438100, 2024-09-19T09:28:20.330 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.027100710, 2024-09-19T09:28:20.330 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.030244681, 2024-09-19T09:28:20.330 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T09:28:20.330 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:20.330 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 446, 2024-09-19T09:28:20.330 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.018474301, 2024-09-19T09:28:20.330 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000041422 2024-09-19T09:28:20.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:20.331 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:20.331 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:20.331 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:20.331 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:20.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:20.331 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 823232, 2024-09-19T09:28:20.331 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 276752 2024-09-19T09:28:20.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:20.331 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:28:20.331 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 2, 2024-09-19T09:28:20.331 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 341, 2024-09-19T09:28:20.331 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 320, 2024-09-19T09:28:20.331 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 217456, 2024-09-19T09:28:20.332 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 4, 2024-09-19T09:28:20.332 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 2, 2024-09-19T09:28:20.332 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.022233613, 2024-09-19T09:28:20.332 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.017993128, 2024-09-19T09:28:20.332 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.001437328, 2024-09-19T09:28:20.332 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T09:28:20.332 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:20.332 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 339, 2024-09-19T09:28:20.332 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.014172310, 2024-09-19T09:28:20.332 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000041806 2024-09-19T09:28:20.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:20.332 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:20.332 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:20.332 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:20.333 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:20.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:20.333 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T09:28:20.333 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 217456 2024-09-19T09:28:20.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:20.333 INFO:teuthology.orchestra.run.smithi120.stdout: "cct": { 2024-09-19T09:28:20.333 INFO:teuthology.orchestra.run.smithi120.stdout: "total_workers": 0, 2024-09-19T09:28:20.333 INFO:teuthology.orchestra.run.smithi120.stdout: "unhealthy_workers": 0 2024-09-19T09:28:20.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:20.333 INFO:teuthology.orchestra.run.smithi120.stdout: "mempool": { 2024-09-19T09:28:20.333 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:28:20.333 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_items": 0, 2024-09-19T09:28:20.333 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:28:20.333 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:28:20.334 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:28:20.334 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:28:20.334 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:28:20.334 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:28:20.334 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:28:20.334 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:28:20.334 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:28:20.334 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:28:20.334 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:28:20.334 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:28:20.334 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:28:20.334 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_items": 0, 2024-09-19T09:28:20.334 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:28:20.335 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_items": 0, 2024-09-19T09:28:20.335 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:28:20.335 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:28:20.335 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:28:20.335 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:28:20.335 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:28:20.335 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:28:20.335 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:28:20.335 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_items": 0, 2024-09-19T09:28:20.335 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:28:20.335 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:28:20.335 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:28:20.335 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_items": 0, 2024-09-19T09:28:20.335 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_bytes": 0, 2024-09-19T09:28:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_items": 0, 2024-09-19T09:28:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:28:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:28:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:28:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:28:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_bytes": 148784, 2024-09-19T09:28:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_items": 31, 2024-09-19T09:28:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:28:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_items": 0, 2024-09-19T09:28:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bytes": 0, 2024-09-19T09:28:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_items": 0, 2024-09-19T09:28:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:28:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_items": 0, 2024-09-19T09:28:20.336 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:28:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_items": 0, 2024-09-19T09:28:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_bytes": 0, 2024-09-19T09:28:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_items": 0, 2024-09-19T09:28:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:28:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:28:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_bytes": 0, 2024-09-19T09:28:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_items": 0, 2024-09-19T09:28:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_bytes": 928, 2024-09-19T09:28:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_items": 2, 2024-09-19T09:28:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_bytes": 0, 2024-09-19T09:28:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_items": 0, 2024-09-19T09:28:20.337 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_bytes": 0, 2024-09-19T09:28:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_items": 0 2024-09-19T09:28:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:28:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-19T09:28:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-09-19T09:28:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-19T09:28:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 454, 2024-09-19T09:28:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 759307, 2024-09-19T09:28:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 454, 2024-09-19T09:28:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-19T09:28:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-19T09:28:20.338 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 454, 2024-09-19T09:28:20.339 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 759307, 2024-09-19T09:28:20.339 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-19T09:28:20.339 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:20.339 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:20.339 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:20.339 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:20.339 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:20.339 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:28:20.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:20 smithi196 ceph-mon[35150]: pgmap v1109: 97 pgs: 97 active+clean; 302 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 203 op/s 2024-09-19T09:28:20.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:20 smithi153 ceph-mon[35071]: pgmap v1109: 97 pgs: 97 active+clean; 302 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 203 op/s 2024-09-19T09:28:21.051 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-19T09:28:21.051 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-19T09:28:21.051 DEBUG:tasks.check_counter:Getting stats from g 2024-09-19T09:28:21.052 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-19T09:28:21.308 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:21.850 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:28:21.968 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:28:21.969 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 2115, 2024-09-19T09:28:21.969 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 2092, 2024-09-19T09:28:21.969 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 1492730, 2024-09-19T09:28:21.969 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 796776, 2024-09-19T09:28:21.969 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 5, 2024-09-19T09:28:21.969 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 1, 2024-09-19T09:28:21.969 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.207882828, 2024-09-19T09:28:21.969 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.090169119, 2024-09-19T09:28:21.969 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.068428676, 2024-09-19T09:28:21.969 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.041609996, 2024-09-19T09:28:21.969 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:21.969 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 2092, 2024-09-19T09:28:21.970 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.098005832, 2024-09-19T09:28:21.970 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000046847 2024-09-19T09:28:21.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.970 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:21.970 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.970 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.970 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.970 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 942608, 2024-09-19T09:28:21.970 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 276528 2024-09-19T09:28:21.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.970 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:28:21.970 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 1639, 2024-09-19T09:28:21.970 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 1975, 2024-09-19T09:28:21.971 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 429936, 2024-09-19T09:28:21.971 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 1198125, 2024-09-19T09:28:21.971 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 5, 2024-09-19T09:28:21.971 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 4, 2024-09-19T09:28:21.971 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.124185088, 2024-09-19T09:28:21.971 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.049415656, 2024-09-19T09:28:21.971 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.029115722, 2024-09-19T09:28:21.971 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.034127557, 2024-09-19T09:28:21.971 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:21.971 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1973, 2024-09-19T09:28:21.971 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.057948367, 2024-09-19T09:28:21.971 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000029370 2024-09-19T09:28:21.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.971 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:21.972 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.972 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.972 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.972 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T09:28:21.972 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 677184 2024-09-19T09:28:21.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.972 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:28:21.972 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 4, 2024-09-19T09:28:21.972 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 6, 2024-09-19T09:28:21.972 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 1629, 2024-09-19T09:28:21.972 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 1284, 2024-09-19T09:28:21.972 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 7, 2024-09-19T09:28:21.973 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 4, 2024-09-19T09:28:21.973 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.002142420, 2024-09-19T09:28:21.973 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.000056438, 2024-09-19T09:28:21.973 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.001783393, 2024-09-19T09:28:21.973 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.000113300, 2024-09-19T09:28:21.973 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:21.973 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 4, 2024-09-19T09:28:21.973 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.002306263, 2024-09-19T09:28:21.973 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000576565 2024-09-19T09:28:21.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.973 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:21.973 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.973 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.974 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.974 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T09:28:21.974 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T09:28:21.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.974 INFO:teuthology.orchestra.run.smithi120.stdout: "cct": { 2024-09-19T09:28:21.974 INFO:teuthology.orchestra.run.smithi120.stdout: "total_workers": 1, 2024-09-19T09:28:21.974 INFO:teuthology.orchestra.run.smithi120.stdout: "unhealthy_workers": 0 2024-09-19T09:28:21.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.974 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-MDSRank": { 2024-09-19T09:28:21.974 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-09-19T09:28:21.974 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-09-19T09:28:21.974 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 6549, 2024-09-19T09:28:21.974 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.145736753, 2024-09-19T09:28:21.975 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000022253 2024-09-19T09:28:21.975 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:21.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.975 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-PurgeQueue": { 2024-09-19T09:28:21.975 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-09-19T09:28:21.975 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-09-19T09:28:21.975 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.975 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.975 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.975 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:21.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.975 INFO:teuthology.orchestra.run.smithi120.stdout: "mds": { 2024-09-19T09:28:21.975 INFO:teuthology.orchestra.run.smithi120.stdout: "request": 0, 2024-09-19T09:28:21.975 INFO:teuthology.orchestra.run.smithi120.stdout: "reply": 0, 2024-09-19T09:28:21.976 INFO:teuthology.orchestra.run.smithi120.stdout: "reply_latency": { 2024-09-19T09:28:21.976 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.976 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.976 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.976 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_reply": 0, 2024-09-19T09:28:21.976 INFO:teuthology.orchestra.run.smithi120.stdout: "forward": 0, 2024-09-19T09:28:21.976 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_complete": 0, 2024-09-19T09:28:21.976 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_keys": 0, 2024-09-19T09:28:21.976 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_commit": 0, 2024-09-19T09:28:21.976 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_split": 0, 2024-09-19T09:28:21.976 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_merge": 0, 2024-09-19T09:28:21.976 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes": 0, 2024-09-19T09:28:21.977 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_top": 0, 2024-09-19T09:28:21.977 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_bottom": 0, 2024-09-19T09:28:21.977 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pin_tail": 0, 2024-09-19T09:28:21.977 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pinned": 0, 2024-09-19T09:28:21.977 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_expired": 0, 2024-09-19T09:28:21.977 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_with_caps": 0, 2024-09-19T09:28:21.977 INFO:teuthology.orchestra.run.smithi120.stdout: "caps": 0, 2024-09-19T09:28:21.977 INFO:teuthology.orchestra.run.smithi120.stdout: "subtrees": 1, 2024-09-19T09:28:21.977 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse": 0, 2024-09-19T09:28:21.977 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_hit": 0, 2024-09-19T09:28:21.977 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_forward": 0, 2024-09-19T09:28:21.977 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_discover": 0, 2024-09-19T09:28:21.977 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_dir_fetch": 0, 2024-09-19T09:28:21.977 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_remote_ino": 0, 2024-09-19T09:28:21.978 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_lock": 0, 2024-09-19T09:28:21.978 INFO:teuthology.orchestra.run.smithi120.stdout: "load_cent": 0, 2024-09-19T09:28:21.978 INFO:teuthology.orchestra.run.smithi120.stdout: "q": 0, 2024-09-19T09:28:21.978 INFO:teuthology.orchestra.run.smithi120.stdout: "exported": 0, 2024-09-19T09:28:21.978 INFO:teuthology.orchestra.run.smithi120.stdout: "exported_inodes": 0, 2024-09-19T09:28:21.978 INFO:teuthology.orchestra.run.smithi120.stdout: "imported": 0, 2024-09-19T09:28:21.978 INFO:teuthology.orchestra.run.smithi120.stdout: "imported_inodes": 0, 2024-09-19T09:28:21.978 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_dir_fetch": 0, 2024-09-19T09:28:21.978 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_backtrace_fetch": 0, 2024-09-19T09:28:21.978 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_peer_discover": 0, 2024-09-19T09:28:21.978 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rfiles": 0, 2024-09-19T09:28:21.978 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rbytes": 0, 2024-09-19T09:28:21.978 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rsnaps": 0, 2024-09-19T09:28:21.978 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T09:28:21.979 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_set_tag": 0, 2024-09-19T09:28:21.979 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:28:21.979 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:28:21.979 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_inodes": 0, 2024-09-19T09:28:21.979 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T09:28:21.979 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T09:28:21.979 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_file_inodes": 0, 2024-09-19T09:28:21.979 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_inode_file_caps": 0, 2024-09-19T09:28:21.979 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T09:28:21.979 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_grant": 0, 2024-09-19T09:28:21.979 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T09:28:21.979 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:28:21.979 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:28:21.979 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps": 0, 2024-09-19T09:28:21.980 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps_dirty": 0, 2024-09-19T09:28:21.980 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_cap_release": 0, 2024-09-19T09:28:21.980 INFO:teuthology.orchestra.run.smithi120.stdout: "process_request_cap_release": 0 2024-09-19T09:28:21.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.980 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache": { 2024-09-19T09:28:21.980 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update": 0, 2024-09-19T09:28:21.980 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update_receipt": 0, 2024-09-19T09:28:21.980 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_try_discover": 0, 2024-09-19T09:28:21.980 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_send_discover": 0, 2024-09-19T09:28:21.980 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_handle_discover": 0, 2024-09-19T09:28:21.980 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays": 0, 2024-09-19T09:28:21.980 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_delayed": 0, 2024-09-19T09:28:21.980 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:28:21.981 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_created": 0, 2024-09-19T09:28:21.981 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_enqueued": 0, 2024-09-19T09:28:21.981 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_reintegrated": 0, 2024-09-19T09:28:21.981 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_migrated": 0, 2024-09-19T09:28:21.981 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_processing": 0, 2024-09-19T09:28:21.981 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:28:21.981 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:28:21.981 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_started": 0, 2024-09-19T09:28:21.981 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_completed": 0, 2024-09-19T09:28:21.981 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_path": 0, 2024-09-19T09:28:21.981 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_inode": 0, 2024-09-19T09:28:21.981 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T09:28:21.981 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_exportdir": 0, 2024-09-19T09:28:21.981 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_flush": 0, 2024-09-19T09:28:21.982 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragmentdir": 0, 2024-09-19T09:28:21.982 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragstats": 0, 2024-09-19T09:28:21.982 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_inodestats": 0 2024-09-19T09:28:21.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.982 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log": { 2024-09-19T09:28:21.982 INFO:teuthology.orchestra.run.smithi120.stdout: "evlrg": 0, 2024-09-19T09:28:21.982 INFO:teuthology.orchestra.run.smithi120.stdout: "evadd": 0, 2024-09-19T09:28:21.982 INFO:teuthology.orchestra.run.smithi120.stdout: "evex": 0, 2024-09-19T09:28:21.982 INFO:teuthology.orchestra.run.smithi120.stdout: "evtrm": 0, 2024-09-19T09:28:21.982 INFO:teuthology.orchestra.run.smithi120.stdout: "ev": 2, 2024-09-19T09:28:21.982 INFO:teuthology.orchestra.run.smithi120.stdout: "evexg": 0, 2024-09-19T09:28:21.982 INFO:teuthology.orchestra.run.smithi120.stdout: "evexd": 0, 2024-09-19T09:28:21.982 INFO:teuthology.orchestra.run.smithi120.stdout: "segadd": 0, 2024-09-19T09:28:21.982 INFO:teuthology.orchestra.run.smithi120.stdout: "segex": 0, 2024-09-19T09:28:21.983 INFO:teuthology.orchestra.run.smithi120.stdout: "segtrm": 0, 2024-09-19T09:28:21.983 INFO:teuthology.orchestra.run.smithi120.stdout: "seg": 2, 2024-09-19T09:28:21.983 INFO:teuthology.orchestra.run.smithi120.stdout: "segmjr": 2, 2024-09-19T09:28:21.983 INFO:teuthology.orchestra.run.smithi120.stdout: "segexg": 0, 2024-09-19T09:28:21.983 INFO:teuthology.orchestra.run.smithi120.stdout: "segexd": 0, 2024-09-19T09:28:21.983 INFO:teuthology.orchestra.run.smithi120.stdout: "expos": 4194304, 2024-09-19T09:28:21.983 INFO:teuthology.orchestra.run.smithi120.stdout: "wrpos": 4194892, 2024-09-19T09:28:21.983 INFO:teuthology.orchestra.run.smithi120.stdout: "rdpos": 4194352, 2024-09-19T09:28:21.983 INFO:teuthology.orchestra.run.smithi120.stdout: "jlat": { 2024-09-19T09:28:21.983 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.983 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.983 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.983 INFO:teuthology.orchestra.run.smithi120.stdout: "replayed": 2 2024-09-19T09:28:21.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.984 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_mem": { 2024-09-19T09:28:21.984 INFO:teuthology.orchestra.run.smithi120.stdout: "ino": 3, 2024-09-19T09:28:21.984 INFO:teuthology.orchestra.run.smithi120.stdout: "ino+": 3, 2024-09-19T09:28:21.984 INFO:teuthology.orchestra.run.smithi120.stdout: "ino-": 0, 2024-09-19T09:28:21.984 INFO:teuthology.orchestra.run.smithi120.stdout: "dir": 1, 2024-09-19T09:28:21.984 INFO:teuthology.orchestra.run.smithi120.stdout: "dir+": 1, 2024-09-19T09:28:21.984 INFO:teuthology.orchestra.run.smithi120.stdout: "dir-": 0, 2024-09-19T09:28:21.984 INFO:teuthology.orchestra.run.smithi120.stdout: "dn": 0, 2024-09-19T09:28:21.984 INFO:teuthology.orchestra.run.smithi120.stdout: "dn+": 0, 2024-09-19T09:28:21.984 INFO:teuthology.orchestra.run.smithi120.stdout: "dn-": 0, 2024-09-19T09:28:21.984 INFO:teuthology.orchestra.run.smithi120.stdout: "cap": 0, 2024-09-19T09:28:21.984 INFO:teuthology.orchestra.run.smithi120.stdout: "cap+": 0, 2024-09-19T09:28:21.985 INFO:teuthology.orchestra.run.smithi120.stdout: "cap-": 0, 2024-09-19T09:28:21.985 INFO:teuthology.orchestra.run.smithi120.stdout: "rss": 49172, 2024-09-19T09:28:21.985 INFO:teuthology.orchestra.run.smithi120.stdout: "heap": 215328 2024-09-19T09:28:21.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.985 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_server": { 2024-09-19T09:28:21.985 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_client_request": 0, 2024-09-19T09:28:21.985 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_server_request": 0, 2024-09-19T09:28:21.985 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_request": 0, 2024-09-19T09:28:21.985 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_session": 0, 2024-09-19T09:28:21.985 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_peer_request": 0, 2024-09-19T09:28:21.985 INFO:teuthology.orchestra.run.smithi120.stdout: "req_create_latency": { 2024-09-19T09:28:21.985 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.985 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.985 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.986 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getattr_latency": { 2024-09-19T09:28:21.986 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.986 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.986 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.986 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getfilelock_latency": { 2024-09-19T09:28:21.986 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.986 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.986 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.986 INFO:teuthology.orchestra.run.smithi120.stdout: "req_link_latency": { 2024-09-19T09:28:21.986 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.987 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.987 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.987 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookup_latency": { 2024-09-19T09:28:21.987 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.987 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.987 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.987 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookuphash_latency": { 2024-09-19T09:28:21.987 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.987 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.987 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.987 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupino_latency": { 2024-09-19T09:28:21.988 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.988 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.988 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.988 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupname_latency": { 2024-09-19T09:28:21.988 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.988 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.988 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.988 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupparent_latency": { 2024-09-19T09:28:21.988 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.988 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.988 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.989 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupsnap_latency": { 2024-09-19T09:28:21.989 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.989 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.989 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.989 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lssnap_latency": { 2024-09-19T09:28:21.989 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.989 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.989 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.989 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mkdir_latency": { 2024-09-19T09:28:21.989 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.990 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.990 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.990 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mknod_latency": { 2024-09-19T09:28:21.990 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.990 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.990 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.990 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mksnap_latency": { 2024-09-19T09:28:21.990 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.990 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.990 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.990 INFO:teuthology.orchestra.run.smithi120.stdout: "req_open_latency": { 2024-09-19T09:28:21.991 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.991 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.991 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.991 INFO:teuthology.orchestra.run.smithi120.stdout: "req_readdir_latency": { 2024-09-19T09:28:21.991 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.991 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.991 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.991 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rename_latency": { 2024-09-19T09:28:21.991 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.991 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.991 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.992 INFO:teuthology.orchestra.run.smithi120.stdout: "req_renamesnap_latency": { 2024-09-19T09:28:21.992 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.992 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.992 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.992 INFO:teuthology.orchestra.run.smithi120.stdout: "req_snapdiff_latency": { 2024-09-19T09:28:21.992 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.992 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.992 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.992 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmdir_latency": { 2024-09-19T09:28:21.992 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.993 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.993 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.993 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmsnap_latency": { 2024-09-19T09:28:21.993 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.993 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.993 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.993 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmxattr_latency": { 2024-09-19T09:28:21.993 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.993 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.993 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.993 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setattr_latency": { 2024-09-19T09:28:21.994 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.994 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.994 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.994 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setdirlayout_latency": { 2024-09-19T09:28:21.994 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.994 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.994 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.994 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setfilelock_latency": { 2024-09-19T09:28:21.994 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.994 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.994 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.995 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setlayout_latency": { 2024-09-19T09:28:21.995 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.995 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.995 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.995 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setxattr_latency": { 2024-09-19T09:28:21.995 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.995 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.995 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.995 INFO:teuthology.orchestra.run.smithi120.stdout: "req_symlink_latency": { 2024-09-19T09:28:21.995 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.995 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.996 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.996 INFO:teuthology.orchestra.run.smithi120.stdout: "req_unlink_latency": { 2024-09-19T09:28:21.996 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.996 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.996 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.996 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:28:21.996 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:28:21.996 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getvxattr_latency": { 2024-09-19T09:28:21.996 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:21.996 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:21.996 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:21.996 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:21.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.997 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_sessions": { 2024-09-19T09:28:21.997 INFO:teuthology.orchestra.run.smithi120.stdout: "session_count": 0, 2024-09-19T09:28:21.997 INFO:teuthology.orchestra.run.smithi120.stdout: "session_add": 0, 2024-09-19T09:28:21.997 INFO:teuthology.orchestra.run.smithi120.stdout: "session_remove": 0, 2024-09-19T09:28:21.997 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_open": 0, 2024-09-19T09:28:21.997 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_stale": 0, 2024-09-19T09:28:21.997 INFO:teuthology.orchestra.run.smithi120.stdout: "total_load": 0, 2024-09-19T09:28:21.997 INFO:teuthology.orchestra.run.smithi120.stdout: "average_load": 0, 2024-09-19T09:28:21.997 INFO:teuthology.orchestra.run.smithi120.stdout: "avg_session_uptime": 0, 2024-09-19T09:28:21.997 INFO:teuthology.orchestra.run.smithi120.stdout: "mdthresh_evicted": 0 2024-09-19T09:28:21.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:21.997 INFO:teuthology.orchestra.run.smithi120.stdout: "mempool": { 2024-09-19T09:28:21.997 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:28:21.998 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_items": 0, 2024-09-19T09:28:21.998 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:28:21.998 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:28:21.998 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:28:21.998 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:28:21.998 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:28:21.998 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:28:21.998 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:28:21.998 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:28:21.998 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:28:21.998 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:28:21.998 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:28:21.998 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:28:21.999 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:28:21.999 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_items": 0, 2024-09-19T09:28:21.999 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:28:21.999 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_items": 0, 2024-09-19T09:28:21.999 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:28:21.999 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:28:21.999 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:28:21.999 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:28:21.999 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:28:21.999 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:28:21.999 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:28:21.999 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_items": 0, 2024-09-19T09:28:21.999 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:28:21.999 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:28:22.000 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:28:22.000 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_items": 0, 2024-09-19T09:28:22.000 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_bytes": 0, 2024-09-19T09:28:22.000 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_items": 0, 2024-09-19T09:28:22.000 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:28:22.000 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:28:22.000 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:28:22.000 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:28:22.000 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_bytes": 198247, 2024-09-19T09:28:22.000 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_items": 56, 2024-09-19T09:28:22.000 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:28:22.000 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_items": 0, 2024-09-19T09:28:22.000 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bytes": 0, 2024-09-19T09:28:22.000 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_items": 0, 2024-09-19T09:28:22.001 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:28:22.001 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_items": 0, 2024-09-19T09:28:22.001 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:28:22.001 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_items": 0, 2024-09-19T09:28:22.001 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_bytes": 7592, 2024-09-19T09:28:22.001 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_items": 156, 2024-09-19T09:28:22.001 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:28:22.001 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:28:22.001 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_bytes": 0, 2024-09-19T09:28:22.001 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_items": 0, 2024-09-19T09:28:22.001 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_bytes": 8512, 2024-09-19T09:28:22.001 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_items": 13, 2024-09-19T09:28:22.001 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_bytes": 0, 2024-09-19T09:28:22.001 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_items": 0, 2024-09-19T09:28:22.002 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_bytes": 0, 2024-09-19T09:28:22.002 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_items": 0 2024-09-19T09:28:22.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:22.002 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter": { 2024-09-19T09:28:22.002 INFO:teuthology.orchestra.run.smithi120.stdout: "op_active": 0, 2024-09-19T09:28:22.002 INFO:teuthology.orchestra.run.smithi120.stdout: "op_laggy": 0, 2024-09-19T09:28:22.002 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send": 3278, 2024-09-19T09:28:22.002 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send_bytes": 21, 2024-09-19T09:28:22.002 INFO:teuthology.orchestra.run.smithi120.stdout: "op_resend": 3, 2024-09-19T09:28:22.002 INFO:teuthology.orchestra.run.smithi120.stdout: "op_reply": 3278, 2024-09-19T09:28:22.002 INFO:teuthology.orchestra.run.smithi120.stdout: "op_latency": { 2024-09-19T09:28:22.002 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 3278, 2024-09-19T09:28:22.002 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 6.443845919, 2024-09-19T09:28:22.002 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.001965785 2024-09-19T09:28:22.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:22.003 INFO:teuthology.orchestra.run.smithi120.stdout: "op_inflight": 0, 2024-09-19T09:28:22.003 INFO:teuthology.orchestra.run.smithi120.stdout: "oplen_avg": { 2024-09-19T09:28:22.003 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 3278, 2024-09-19T09:28:22.003 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 3279 2024-09-19T09:28:22.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:22.003 INFO:teuthology.orchestra.run.smithi120.stdout: "op": 3278, 2024-09-19T09:28:22.003 INFO:teuthology.orchestra.run.smithi120.stdout: "op_r": 3278, 2024-09-19T09:28:22.003 INFO:teuthology.orchestra.run.smithi120.stdout: "op_w": 0, 2024-09-19T09:28:22.003 INFO:teuthology.orchestra.run.smithi120.stdout: "op_rmw": 0, 2024-09-19T09:28:22.003 INFO:teuthology.orchestra.run.smithi120.stdout: "op_pg": 0, 2024-09-19T09:28:22.003 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_stat": 1636, 2024-09-19T09:28:22.003 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_create": 0, 2024-09-19T09:28:22.004 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_read": 1640, 2024-09-19T09:28:22.004 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_write": 0, 2024-09-19T09:28:22.004 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writefull": 0, 2024-09-19T09:28:22.004 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writesame": 0, 2024-09-19T09:28:22.004 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_append": 0, 2024-09-19T09:28:22.004 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_zero": 0, 2024-09-19T09:28:22.004 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_truncate": 0, 2024-09-19T09:28:22.004 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_delete": 0, 2024-09-19T09:28:22.004 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_mapext": 0, 2024-09-19T09:28:22.004 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_sparse_read": 0, 2024-09-19T09:28:22.004 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_clonerange": 0, 2024-09-19T09:28:22.004 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_getxattr": 1, 2024-09-19T09:28:22.004 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_setxattr": 0, 2024-09-19T09:28:22.004 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:28:22.005 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_rmxattr": 0, 2024-09-19T09:28:22.005 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:28:22.005 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_call": 0, 2024-09-19T09:28:22.005 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_watch": 0, 2024-09-19T09:28:22.005 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_notify": 0, 2024-09-19T09:28:22.005 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:28:22.005 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls": 0, 2024-09-19T09:28:22.005 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:28:22.005 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_other": 0, 2024-09-19T09:28:22.005 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_active": 0, 2024-09-19T09:28:22.005 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_send": 0, 2024-09-19T09:28:22.005 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_resend": 0, 2024-09-19T09:28:22.005 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_ping": 0, 2024-09-19T09:28:22.005 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_active": 0, 2024-09-19T09:28:22.006 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_send": 0, 2024-09-19T09:28:22.006 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_resend": 0, 2024-09-19T09:28:22.006 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_active": 0, 2024-09-19T09:28:22.006 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_send": 0, 2024-09-19T09:28:22.006 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_resend": 0, 2024-09-19T09:28:22.006 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_active": 0, 2024-09-19T09:28:22.006 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_send": 0, 2024-09-19T09:28:22.006 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_resend": 0, 2024-09-19T09:28:22.006 INFO:teuthology.orchestra.run.smithi120.stdout: "command_active": 0, 2024-09-19T09:28:22.006 INFO:teuthology.orchestra.run.smithi120.stdout: "command_send": 0, 2024-09-19T09:28:22.006 INFO:teuthology.orchestra.run.smithi120.stdout: "command_resend": 0, 2024-09-19T09:28:22.006 INFO:teuthology.orchestra.run.smithi120.stdout: "map_epoch": 85, 2024-09-19T09:28:22.006 INFO:teuthology.orchestra.run.smithi120.stdout: "map_full": 0, 2024-09-19T09:28:22.006 INFO:teuthology.orchestra.run.smithi120.stdout: "map_inc": 6, 2024-09-19T09:28:22.007 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_sessions": 6, 2024-09-19T09:28:22.007 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_open": 8, 2024-09-19T09:28:22.007 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_close": 2, 2024-09-19T09:28:22.007 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_laggy": 0, 2024-09-19T09:28:22.007 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_wr": 0, 2024-09-19T09:28:22.007 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_rd": 2, 2024-09-19T09:28:22.007 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_del": 0 2024-09-19T09:28:22.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:22.007 INFO:teuthology.orchestra.run.smithi120.stdout: "oft": { 2024-09-19T09:28:22.007 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_objs": 0, 2024-09-19T09:28:22.007 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:28:22.007 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_updates": 0, 2024-09-19T09:28:22.007 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_removes": 0 2024-09-19T09:28:22.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:22.008 INFO:teuthology.orchestra.run.smithi120.stdout: "osd-slow-ops": { 2024-09-19T09:28:22.008 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_ops_count": 0 2024-09-19T09:28:22.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:22.008 INFO:teuthology.orchestra.run.smithi120.stdout: "purge_queue": { 2024-09-19T09:28:22.008 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops": 0, 2024-09-19T09:28:22.008 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T09:28:22.008 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing": 0, 2024-09-19T09:28:22.008 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_high_water": 0, 2024-09-19T09:28:22.008 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed_ops": 0, 2024-09-19T09:28:22.008 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed": 0, 2024-09-19T09:28:22.008 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_item_in_journal": 0 2024-09-19T09:28:22.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:22.008 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:28:22.008 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-19T09:28:22.009 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-09-19T09:28:22.009 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-19T09:28:22.009 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 3754, 2024-09-19T09:28:22.009 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 1559122, 2024-09-19T09:28:22.009 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:22.009 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 3754, 2024-09-19T09:28:22.009 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-19T09:28:22.009 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-19T09:28:22.009 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 3754, 2024-09-19T09:28:22.009 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 1559122, 2024-09-19T09:28:22.009 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-19T09:28:22.009 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:22.009 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:22.009 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:22.010 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:22.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:22.010 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_bytes": { 2024-09-19T09:28:22.010 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-19T09:28:22.010 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-09-19T09:28:22.010 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-19T09:28:22.010 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-09-19T09:28:22.010 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-09-19T09:28:22.010 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:22.010 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:22.010 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 3278, 2024-09-19T09:28:22.010 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 593, 2024-09-19T09:28:22.010 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 3278, 2024-09-19T09:28:22.010 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 593, 2024-09-19T09:28:22.011 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-19T09:28:22.011 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:22.011 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:22.011 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:22.011 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:22.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:22.011 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_ops": { 2024-09-19T09:28:22.011 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-19T09:28:22.011 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 1024, 2024-09-19T09:28:22.011 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-19T09:28:22.011 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-09-19T09:28:22.011 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-09-19T09:28:22.011 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:22.011 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:22.012 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 3278, 2024-09-19T09:28:22.012 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 3278, 2024-09-19T09:28:22.012 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 3278, 2024-09-19T09:28:22.012 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 3278, 2024-09-19T09:28:22.012 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-19T09:28:22.012 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:22.012 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:22.012 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:22.012 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:22.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:22.012 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:28:22.012 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-19T09:28:22.012 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-09-19T09:28:22.012 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-19T09:28:22.013 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-09-19T09:28:22.013 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-09-19T09:28:22.013 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:22.013 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:22.013 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-19T09:28:22.013 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-19T09:28:22.013 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 0, 2024-09-19T09:28:22.013 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 0, 2024-09-19T09:28:22.013 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-19T09:28:22.013 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:22.013 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:22.013 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:22.013 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:22.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:22.014 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle-0x5567308aab40": { 2024-09-19T09:28:22.014 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-19T09:28:22.014 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-09-19T09:28:22.014 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-19T09:28:22.014 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-09-19T09:28:22.014 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-09-19T09:28:22.014 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:22.014 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:22.014 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-19T09:28:22.014 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-19T09:28:22.014 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 0, 2024-09-19T09:28:22.014 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 0, 2024-09-19T09:28:22.014 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-19T09:28:22.014 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:22.015 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:22.015 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:22.015 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:22.015 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:22.015 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:28:22.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:22 smithi196 ceph-mon[35150]: pgmap v1110: 97 pgs: 97 active+clean; 302 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 429 KiB/s wr, 68 op/s 2024-09-19T09:28:22.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:22 smithi153 ceph-mon[35071]: pgmap v1110: 97 pgs: 97 active+clean; 302 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 429 KiB/s wr, 68 op/s 2024-09-19T09:28:22.453 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:22 smithi120 ceph-mon[31150]: pgmap v1110: 97 pgs: 97 active+clean; 302 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 429 KiB/s wr, 68 op/s 2024-09-19T09:28:22.669 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-19T09:28:22.669 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-19T09:28:22.670 DEBUG:tasks.check_counter:Getting stats from j 2024-09-19T09:28:22.670 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-19T09:28:22.917 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:23.530 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-19T09:28:23.530 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:28:23.530 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 0, 2024-09-19T09:28:23.530 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 2, 2024-09-19T09:28:23.531 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 0, 2024-09-19T09:28:23.531 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 0, 2024-09-19T09:28:23.531 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 4, 2024-09-19T09:28:23.531 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 0, 2024-09-19T09:28:23.531 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.001186845, 2024-09-19T09:28:23.531 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T09:28:23.531 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.000350166, 2024-09-19T09:28:23.531 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T09:28:23.531 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:23.531 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:23.531 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:23.531 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:23.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:23.531 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:23.532 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:23.532 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:23.532 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:23.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:23.532 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T09:28:23.532 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T09:28:23.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:23.532 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:28:23.532 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 457, 2024-09-19T09:28:23.532 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 450, 2024-09-19T09:28:23.532 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 826528, 2024-09-19T09:28:23.532 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 277840, 2024-09-19T09:28:23.532 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 2, 2024-09-19T09:28:23.533 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 1, 2024-09-19T09:28:23.533 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.056627936, 2024-09-19T09:28:23.533 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.026090293, 2024-09-19T09:28:23.533 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.036570847, 2024-09-19T09:28:23.533 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T09:28:23.533 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:23.533 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 450, 2024-09-19T09:28:23.533 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.023900033, 2024-09-19T09:28:23.533 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000053111 2024-09-19T09:28:23.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:23.533 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:23.533 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:23.533 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:23.533 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:23.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:23.534 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 826528, 2024-09-19T09:28:23.534 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 277840 2024-09-19T09:28:23.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:23.534 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:28:23.534 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 2, 2024-09-19T09:28:23.534 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 343, 2024-09-19T09:28:23.534 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 320, 2024-09-19T09:28:23.534 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 217808, 2024-09-19T09:28:23.534 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 4, 2024-09-19T09:28:23.534 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 2, 2024-09-19T09:28:23.534 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.021667501, 2024-09-19T09:28:23.534 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.017314994, 2024-09-19T09:28:23.534 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.001607345, 2024-09-19T09:28:23.535 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T09:28:23.535 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:23.535 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 341, 2024-09-19T09:28:23.535 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.015437387, 2024-09-19T09:28:23.535 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000045270 2024-09-19T09:28:23.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:23.535 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:23.535 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:23.535 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:23.535 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:23.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:23.535 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T09:28:23.535 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 217808 2024-09-19T09:28:23.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:23.536 INFO:teuthology.orchestra.run.smithi120.stdout: "cct": { 2024-09-19T09:28:23.536 INFO:teuthology.orchestra.run.smithi120.stdout: "total_workers": 0, 2024-09-19T09:28:23.536 INFO:teuthology.orchestra.run.smithi120.stdout: "unhealthy_workers": 0 2024-09-19T09:28:23.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:23.536 INFO:teuthology.orchestra.run.smithi120.stdout: "mempool": { 2024-09-19T09:28:23.536 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:28:23.536 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_items": 0, 2024-09-19T09:28:23.536 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:28:23.536 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:28:23.536 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:28:23.536 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:28:23.536 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:28:23.536 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:28:23.537 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:28:23.537 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:28:23.537 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:28:23.537 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:28:23.537 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:28:23.537 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:28:23.537 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:28:23.537 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_items": 0, 2024-09-19T09:28:23.537 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:28:23.537 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_items": 0, 2024-09-19T09:28:23.537 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:28:23.537 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:28:23.537 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:28:23.537 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:28:23.538 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:28:23.538 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:28:23.538 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:28:23.538 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_items": 0, 2024-09-19T09:28:23.538 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:28:23.538 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:28:23.538 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:28:23.538 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_items": 0, 2024-09-19T09:28:23.538 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_bytes": 0, 2024-09-19T09:28:23.538 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_items": 0, 2024-09-19T09:28:23.539 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:28:23.539 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:28:23.539 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:28:23.539 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:28:23.539 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_bytes": 148721, 2024-09-19T09:28:23.539 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_items": 30, 2024-09-19T09:28:23.539 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:28:23.539 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_items": 0, 2024-09-19T09:28:23.539 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bytes": 0, 2024-09-19T09:28:23.539 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_items": 0, 2024-09-19T09:28:23.539 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:28:23.539 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_items": 0, 2024-09-19T09:28:23.539 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:28:23.540 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_items": 0, 2024-09-19T09:28:23.540 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_bytes": 0, 2024-09-19T09:28:23.540 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_items": 0, 2024-09-19T09:28:23.540 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:28:23.540 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:28:23.540 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_bytes": 0, 2024-09-19T09:28:23.540 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_items": 0, 2024-09-19T09:28:23.540 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_bytes": 928, 2024-09-19T09:28:23.540 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_items": 2, 2024-09-19T09:28:23.540 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_bytes": 0, 2024-09-19T09:28:23.540 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_items": 0, 2024-09-19T09:28:23.540 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_bytes": 0, 2024-09-19T09:28:23.540 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_items": 0 2024-09-19T09:28:23.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-19T09:28:23.541 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:28:23.541 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-09-19T09:28:23.541 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-09-19T09:28:23.541 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-09-19T09:28:23.541 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 455, 2024-09-19T09:28:23.541 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 759437, 2024-09-19T09:28:23.541 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:23.541 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 455, 2024-09-19T09:28:23.541 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-09-19T09:28:23.541 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-09-19T09:28:23.541 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 455, 2024-09-19T09:28:23.541 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 759437, 2024-09-19T09:28:23.541 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-09-19T09:28:23.541 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-09-19T09:28:23.542 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-09-19T09:28:23.542 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-09-19T09:28:23.542 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:23.542 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-19T09:28:23.542 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-19T09:28:24.237 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:24 smithi120 ceph-mon[31150]: pgmap v1111: 97 pgs: 97 active+clean; 302 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 KiB/s wr, 29 op/s 2024-09-19T09:28:24.265 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-19T09:28:24.265 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-19T09:28:24.265 DEBUG:tasks.check_counter:Getting stats from b 2024-09-19T09:28:24.266 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-19T09:28:24.420 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:24 smithi153 ceph-mon[35071]: pgmap v1111: 97 pgs: 97 active+clean; 302 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 KiB/s wr, 29 op/s 2024-09-19T09:28:24.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:24 smithi196 ceph-mon[35150]: pgmap v1111: 97 pgs: 97 active+clean; 302 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 KiB/s wr, 29 op/s 2024-09-19T09:28:24.659 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T09:28:25.277 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-09-19T09:28:25.278 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:28:25.278 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 1152857, 2024-09-19T09:28:25.278 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 1775598, 2024-09-19T09:28:25.278 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 389600985, 2024-09-19T09:28:25.278 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 1730806308, 2024-09-19T09:28:25.278 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 49, 2024-09-19T09:28:25.278 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 5, 2024-09-19T09:28:25.278 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 71.262543987, 2024-09-19T09:28:25.278 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 31.934096828, 2024-09-19T09:28:25.278 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 47.108895255, 2024-09-19T09:28:25.278 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 3.960821351, 2024-09-19T09:28:25.278 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:25.278 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1775595, 2024-09-19T09:28:25.279 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 121.939792230, 2024-09-19T09:28:25.279 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000068675 2024-09-19T09:28:25.279 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.279 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:25.279 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 2580851, 2024-09-19T09:28:25.279 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.379208557, 2024-09-19T09:28:25.279 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000146 2024-09-19T09:28:25.279 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.279 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 339115008, 2024-09-19T09:28:25.279 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 1329610016 2024-09-19T09:28:25.279 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.279 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:28:25.279 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 225082, 2024-09-19T09:28:25.279 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 306933, 2024-09-19T09:28:25.280 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 58302541, 2024-09-19T09:28:25.280 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 513915585, 2024-09-19T09:28:25.280 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 11, 2024-09-19T09:28:25.280 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 6, 2024-09-19T09:28:25.280 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 17.060824979, 2024-09-19T09:28:25.280 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 6.595182264, 2024-09-19T09:28:25.280 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 5.689313975, 2024-09-19T09:28:25.280 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 4.294470823, 2024-09-19T09:28:25.280 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:25.280 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 306934, 2024-09-19T09:28:25.280 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 96.570882337, 2024-09-19T09:28:25.280 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000314630 2024-09-19T09:28:25.280 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.281 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:25.281 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 247514, 2024-09-19T09:28:25.281 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.048501637, 2024-09-19T09:28:25.281 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000195 2024-09-19T09:28:25.281 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.281 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T09:28:25.281 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 690672 2024-09-19T09:28:25.281 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.281 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:28:25.281 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 278762, 2024-09-19T09:28:25.281 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 438464, 2024-09-19T09:28:25.281 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 66062479, 2024-09-19T09:28:25.281 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 609788313, 2024-09-19T09:28:25.281 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 566, 2024-09-19T09:28:25.282 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 8, 2024-09-19T09:28:25.282 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 23.734488959, 2024-09-19T09:28:25.282 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 11.241693335, 2024-09-19T09:28:25.282 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 8.700705112, 2024-09-19T09:28:25.282 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 3.934004443, 2024-09-19T09:28:25.282 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:25.282 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 438462, 2024-09-19T09:28:25.282 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 69.582797497, 2024-09-19T09:28:25.282 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000158697 2024-09-19T09:28:25.282 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.282 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:25.282 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 492860, 2024-09-19T09:28:25.282 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.080753638, 2024-09-19T09:28:25.282 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000163 2024-09-19T09:28:25.283 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.283 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 33680, 2024-09-19T09:28:25.283 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 1856 2024-09-19T09:28:25.283 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.283 INFO:teuthology.orchestra.run.smithi153.stdout: "cct": { 2024-09-19T09:28:25.283 INFO:teuthology.orchestra.run.smithi153.stdout: "total_workers": 1, 2024-09-19T09:28:25.283 INFO:teuthology.orchestra.run.smithi153.stdout: "unhealthy_workers": 0 2024-09-19T09:28:25.283 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.283 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-MDSRank": { 2024-09-19T09:28:25.283 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-09-19T09:28:25.284 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-09-19T09:28:25.284 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 242598, 2024-09-19T09:28:25.284 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 410.826456378, 2024-09-19T09:28:25.284 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.001693445 2024-09-19T09:28:25.284 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:25.284 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.284 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-PurgeQueue": { 2024-09-19T09:28:25.284 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-09-19T09:28:25.284 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-09-19T09:28:25.284 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 203858, 2024-09-19T09:28:25.285 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 49.598619028, 2024-09-19T09:28:25.285 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000243299 2024-09-19T09:28:25.285 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:25.285 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.285 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-09-19T09:28:25.285 INFO:teuthology.orchestra.run.smithi153.stdout: "request": 680851, 2024-09-19T09:28:25.285 INFO:teuthology.orchestra.run.smithi153.stdout: "reply": 674357, 2024-09-19T09:28:25.285 INFO:teuthology.orchestra.run.smithi153.stdout: "reply_latency": { 2024-09-19T09:28:25.285 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 674357, 2024-09-19T09:28:25.285 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 6503.123596225, 2024-09-19T09:28:25.285 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.009643443 2024-09-19T09:28:25.285 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.285 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_reply": 0, 2024-09-19T09:28:25.286 INFO:teuthology.orchestra.run.smithi153.stdout: "forward": 6494, 2024-09-19T09:28:25.286 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_complete": 0, 2024-09-19T09:28:25.286 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_keys": 0, 2024-09-19T09:28:25.286 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_commit": 11504, 2024-09-19T09:28:25.286 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_split": 57, 2024-09-19T09:28:25.286 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_merge": 74, 2024-09-19T09:28:25.286 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes": 2898, 2024-09-19T09:28:25.286 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_top": 1267, 2024-09-19T09:28:25.286 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_bottom": 1631, 2024-09-19T09:28:25.286 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pin_tail": 0, 2024-09-19T09:28:25.286 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pinned": 1087, 2024-09-19T09:28:25.286 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_expired": 206, 2024-09-19T09:28:25.286 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_with_caps": 814, 2024-09-19T09:28:25.286 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": 814, 2024-09-19T09:28:25.287 INFO:teuthology.orchestra.run.smithi153.stdout: "subtrees": 4, 2024-09-19T09:28:25.287 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse": 998533, 2024-09-19T09:28:25.287 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_hit": 577457, 2024-09-19T09:28:25.287 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_forward": 6494, 2024-09-19T09:28:25.287 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_discover": 3, 2024-09-19T09:28:25.287 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_dir_fetch": 0, 2024-09-19T09:28:25.287 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_remote_ino": 0, 2024-09-19T09:28:25.287 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_lock": 7971, 2024-09-19T09:28:25.287 INFO:teuthology.orchestra.run.smithi153.stdout: "load_cent": 0, 2024-09-19T09:28:25.287 INFO:teuthology.orchestra.run.smithi153.stdout: "q": 0, 2024-09-19T09:28:25.287 INFO:teuthology.orchestra.run.smithi153.stdout: "exported": 3, 2024-09-19T09:28:25.287 INFO:teuthology.orchestra.run.smithi153.stdout: "exported_inodes": 145, 2024-09-19T09:28:25.287 INFO:teuthology.orchestra.run.smithi153.stdout: "imported": 3, 2024-09-19T09:28:25.287 INFO:teuthology.orchestra.run.smithi153.stdout: "imported_inodes": 1, 2024-09-19T09:28:25.288 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_dir_fetch": 0, 2024-09-19T09:28:25.288 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_backtrace_fetch": 12271, 2024-09-19T09:28:25.288 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_peer_discover": 0, 2024-09-19T09:28:25.288 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rfiles": 114, 2024-09-19T09:28:25.288 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rbytes": 34500714, 2024-09-19T09:28:25.288 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rsnaps": 0, 2024-09-19T09:28:25.288 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_fetch": 12271, 2024-09-19T09:28:25.288 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_set_tag": 0, 2024-09-19T09:28:25.288 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:28:25.288 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:28:25.288 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_inodes": 1975, 2024-09-19T09:28:25.288 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_base_inodes": 36, 2024-09-19T09:28:25.288 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dirfrag_rstats": 1939, 2024-09-19T09:28:25.288 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_file_inodes": 10296, 2024-09-19T09:28:25.289 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_inode_file_caps": 8, 2024-09-19T09:28:25.289 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_revoke": 36628, 2024-09-19T09:28:25.289 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_grant": 444863, 2024-09-19T09:28:25.289 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T09:28:25.289 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:28:25.289 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:28:25.289 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps": 197656, 2024-09-19T09:28:25.289 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps_dirty": 88365, 2024-09-19T09:28:25.289 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_cap_release": 39801, 2024-09-19T09:28:25.289 INFO:teuthology.orchestra.run.smithi153.stdout: "process_request_cap_release": 420536 2024-09-19T09:28:25.289 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.289 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_cache": { 2024-09-19T09:28:25.289 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update": 48, 2024-09-19T09:28:25.290 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update_receipt": 9, 2024-09-19T09:28:25.290 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_try_discover": 6, 2024-09-19T09:28:25.290 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_send_discover": 6, 2024-09-19T09:28:25.290 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_handle_discover": 134, 2024-09-19T09:28:25.290 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays": 830, 2024-09-19T09:28:25.290 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_delayed": 0, 2024-09-19T09:28:25.290 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:28:25.290 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_created": 111725, 2024-09-19T09:28:25.290 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_enqueued": 110895, 2024-09-19T09:28:25.290 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_reintegrated": 0, 2024-09-19T09:28:25.290 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_migrated": 0, 2024-09-19T09:28:25.290 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_processing": 0, 2024-09-19T09:28:25.290 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:28:25.290 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:28:25.291 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_started": 0, 2024-09-19T09:28:25.291 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_completed": 0, 2024-09-19T09:28:25.291 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_path": 22, 2024-09-19T09:28:25.291 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_inode": 8913, 2024-09-19T09:28:25.291 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_enqueue_scrub": 36, 2024-09-19T09:28:25.291 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_exportdir": 3, 2024-09-19T09:28:25.291 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_flush": 0, 2024-09-19T09:28:25.291 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragmentdir": 131, 2024-09-19T09:28:25.291 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragstats": 0, 2024-09-19T09:28:25.291 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_inodestats": 0 2024-09-19T09:28:25.291 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.291 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_log": { 2024-09-19T09:28:25.291 INFO:teuthology.orchestra.run.smithi153.stdout: "evlrg": 0, 2024-09-19T09:28:25.291 INFO:teuthology.orchestra.run.smithi153.stdout: "evadd": 529908, 2024-09-19T09:28:25.292 INFO:teuthology.orchestra.run.smithi153.stdout: "evex": 527430, 2024-09-19T09:28:25.292 INFO:teuthology.orchestra.run.smithi153.stdout: "evtrm": 527430, 2024-09-19T09:28:25.292 INFO:teuthology.orchestra.run.smithi153.stdout: "ev": 2478, 2024-09-19T09:28:25.292 INFO:teuthology.orchestra.run.smithi153.stdout: "evexg": 0, 2024-09-19T09:28:25.292 INFO:teuthology.orchestra.run.smithi153.stdout: "evexd": 7152, 2024-09-19T09:28:25.292 INFO:teuthology.orchestra.run.smithi153.stdout: "segadd": 686, 2024-09-19T09:28:25.292 INFO:teuthology.orchestra.run.smithi153.stdout: "segex": 683, 2024-09-19T09:28:25.292 INFO:teuthology.orchestra.run.smithi153.stdout: "segtrm": 683, 2024-09-19T09:28:25.292 INFO:teuthology.orchestra.run.smithi153.stdout: "seg": 3, 2024-09-19T09:28:25.292 INFO:teuthology.orchestra.run.smithi153.stdout: "segmjr": 1, 2024-09-19T09:28:25.292 INFO:teuthology.orchestra.run.smithi153.stdout: "segexg": 0, 2024-09-19T09:28:25.292 INFO:teuthology.orchestra.run.smithi153.stdout: "segexd": 8, 2024-09-19T09:28:25.292 INFO:teuthology.orchestra.run.smithi153.stdout: "expos": 1126178290, 2024-09-19T09:28:25.292 INFO:teuthology.orchestra.run.smithi153.stdout: "wrpos": 1131312327, 2024-09-19T09:28:25.293 INFO:teuthology.orchestra.run.smithi153.stdout: "rdpos": 0, 2024-09-19T09:28:25.293 INFO:teuthology.orchestra.run.smithi153.stdout: "jlat": { 2024-09-19T09:28:25.293 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 144857, 2024-09-19T09:28:25.293 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 1847.252301785, 2024-09-19T09:28:25.293 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.012752247 2024-09-19T09:28:25.293 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.293 INFO:teuthology.orchestra.run.smithi153.stdout: "replayed": 0 2024-09-19T09:28:25.293 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.293 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_mem": { 2024-09-19T09:28:25.293 INFO:teuthology.orchestra.run.smithi153.stdout: "ino": 857, 2024-09-19T09:28:25.293 INFO:teuthology.orchestra.run.smithi153.stdout: "ino+": 111897, 2024-09-19T09:28:25.293 INFO:teuthology.orchestra.run.smithi153.stdout: "ino-": 111040, 2024-09-19T09:28:25.293 INFO:teuthology.orchestra.run.smithi153.stdout: "dir": 73, 2024-09-19T09:28:25.293 INFO:teuthology.orchestra.run.smithi153.stdout: "dir+": 668, 2024-09-19T09:28:25.294 INFO:teuthology.orchestra.run.smithi153.stdout: "dir-": 595, 2024-09-19T09:28:25.294 INFO:teuthology.orchestra.run.smithi153.stdout: "dn": 2898, 2024-09-19T09:28:25.294 INFO:teuthology.orchestra.run.smithi153.stdout: "dn+": 261904, 2024-09-19T09:28:25.294 INFO:teuthology.orchestra.run.smithi153.stdout: "dn-": 259006, 2024-09-19T09:28:25.294 INFO:teuthology.orchestra.run.smithi153.stdout: "cap": 814, 2024-09-19T09:28:25.294 INFO:teuthology.orchestra.run.smithi153.stdout: "cap+": 123064, 2024-09-19T09:28:25.294 INFO:teuthology.orchestra.run.smithi153.stdout: "cap-": 122250, 2024-09-19T09:28:25.294 INFO:teuthology.orchestra.run.smithi153.stdout: "rss": 127860, 2024-09-19T09:28:25.294 INFO:teuthology.orchestra.run.smithi153.stdout: "heap": 231712 2024-09-19T09:28:25.294 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.295 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_server": { 2024-09-19T09:28:25.295 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_client_request": 839873, 2024-09-19T09:28:25.295 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_server_request": 2932, 2024-09-19T09:28:25.295 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_request": 680851, 2024-09-19T09:28:25.295 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_session": 15929, 2024-09-19T09:28:25.295 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_peer_request": 5882, 2024-09-19T09:28:25.295 INFO:teuthology.orchestra.run.smithi153.stdout: "req_create_latency": { 2024-09-19T09:28:25.295 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 111716, 2024-09-19T09:28:25.295 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 1615.971303035, 2024-09-19T09:28:25.295 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.014464994 2024-09-19T09:28:25.295 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.295 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getattr_latency": { 2024-09-19T09:28:25.296 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 6573, 2024-09-19T09:28:25.296 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 147.108464437, 2024-09-19T09:28:25.296 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.022380718 2024-09-19T09:28:25.296 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.296 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getfilelock_latency": { 2024-09-19T09:28:25.296 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:25.296 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:25.296 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:25.296 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.296 INFO:teuthology.orchestra.run.smithi153.stdout: "req_link_latency": { 2024-09-19T09:28:25.296 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:25.297 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:25.297 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:25.297 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.297 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookup_latency": { 2024-09-19T09:28:25.297 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 245606, 2024-09-19T09:28:25.297 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 1111.772405269, 2024-09-19T09:28:25.297 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.004526650 2024-09-19T09:28:25.297 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.297 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookuphash_latency": { 2024-09-19T09:28:25.297 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:25.297 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:25.297 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:25.297 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.297 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupino_latency": { 2024-09-19T09:28:25.298 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1, 2024-09-19T09:28:25.298 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000146113, 2024-09-19T09:28:25.298 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000146113 2024-09-19T09:28:25.298 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.298 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupname_latency": { 2024-09-19T09:28:25.298 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:25.298 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:25.298 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:25.298 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.298 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupparent_latency": { 2024-09-19T09:28:25.298 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:25.298 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:25.298 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:25.298 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.299 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupsnap_latency": { 2024-09-19T09:28:25.299 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:25.299 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:25.299 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:25.299 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.299 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lssnap_latency": { 2024-09-19T09:28:25.299 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:25.300 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:25.300 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:25.300 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.300 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mkdir_latency": { 2024-09-19T09:28:25.300 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 119, 2024-09-19T09:28:25.300 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.565156464, 2024-09-19T09:28:25.301 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.004749213 2024-09-19T09:28:25.301 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.301 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mknod_latency": { 2024-09-19T09:28:25.301 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:25.301 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:25.301 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:25.301 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.301 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mksnap_latency": { 2024-09-19T09:28:25.301 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:25.301 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:25.301 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:25.301 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.301 INFO:teuthology.orchestra.run.smithi153.stdout: "req_open_latency": { 2024-09-19T09:28:25.302 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 10, 2024-09-19T09:28:25.302 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.028325728, 2024-09-19T09:28:25.302 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.002832572 2024-09-19T09:28:25.302 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.302 INFO:teuthology.orchestra.run.smithi153.stdout: "req_readdir_latency": { 2024-09-19T09:28:25.302 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 171017, 2024-09-19T09:28:25.302 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 1685.897372995, 2024-09-19T09:28:25.302 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.009858068 2024-09-19T09:28:25.302 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.302 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rename_latency": { 2024-09-19T09:28:25.302 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 23553, 2024-09-19T09:28:25.302 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 437.365970829, 2024-09-19T09:28:25.302 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.018569437 2024-09-19T09:28:25.302 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.303 INFO:teuthology.orchestra.run.smithi153.stdout: "req_renamesnap_latency": { 2024-09-19T09:28:25.303 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:25.303 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:25.303 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:25.303 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.303 INFO:teuthology.orchestra.run.smithi153.stdout: "req_snapdiff_latency": { 2024-09-19T09:28:25.303 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:25.303 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:25.303 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:25.303 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.303 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmdir_latency": { 2024-09-19T09:28:25.303 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 114, 2024-09-19T09:28:25.303 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.767091867, 2024-09-19T09:28:25.303 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.006728876 2024-09-19T09:28:25.304 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.304 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmsnap_latency": { 2024-09-19T09:28:25.304 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:25.304 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:25.304 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:25.304 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.304 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmxattr_latency": { 2024-09-19T09:28:25.304 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:25.304 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:25.304 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:25.304 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.304 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setattr_latency": { 2024-09-19T09:28:25.304 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 398, 2024-09-19T09:28:25.304 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 296.225930187, 2024-09-19T09:28:25.305 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.744286256 2024-09-19T09:28:25.305 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.305 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setdirlayout_latency": { 2024-09-19T09:28:25.305 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:25.305 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:25.305 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:25.305 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.305 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setfilelock_latency": { 2024-09-19T09:28:25.305 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 3620, 2024-09-19T09:28:25.305 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 17.069978138, 2024-09-19T09:28:25.305 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.004715463 2024-09-19T09:28:25.305 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.305 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setlayout_latency": { 2024-09-19T09:28:25.305 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:25.306 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:25.306 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:25.306 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.306 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setxattr_latency": { 2024-09-19T09:28:25.306 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 8, 2024-09-19T09:28:25.306 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.019809809, 2024-09-19T09:28:25.306 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.002476226 2024-09-19T09:28:25.306 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.306 INFO:teuthology.orchestra.run.smithi153.stdout: "req_symlink_latency": { 2024-09-19T09:28:25.306 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:25.306 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:25.306 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:25.306 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.306 INFO:teuthology.orchestra.run.smithi153.stdout: "req_unlink_latency": { 2024-09-19T09:28:25.307 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 111622, 2024-09-19T09:28:25.307 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 1190.331641354, 2024-09-19T09:28:25.307 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.010663951 2024-09-19T09:28:25.307 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.307 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:28:25.307 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:28:25.307 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getvxattr_latency": { 2024-09-19T09:28:25.307 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:25.307 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:25.307 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:25.307 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:25.307 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.307 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_sessions": { 2024-09-19T09:28:25.307 INFO:teuthology.orchestra.run.smithi153.stdout: "session_count": 1, 2024-09-19T09:28:25.307 INFO:teuthology.orchestra.run.smithi153.stdout: "session_add": 3, 2024-09-19T09:28:25.308 INFO:teuthology.orchestra.run.smithi153.stdout: "session_remove": 2, 2024-09-19T09:28:25.308 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_open": 1, 2024-09-19T09:28:25.308 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_stale": 0, 2024-09-19T09:28:25.308 INFO:teuthology.orchestra.run.smithi153.stdout: "total_load": 40680, 2024-09-19T09:28:25.308 INFO:teuthology.orchestra.run.smithi153.stdout: "average_load": 40680, 2024-09-19T09:28:25.308 INFO:teuthology.orchestra.run.smithi153.stdout: "avg_session_uptime": 1542, 2024-09-19T09:28:25.308 INFO:teuthology.orchestra.run.smithi153.stdout: "mdthresh_evicted": 0 2024-09-19T09:28:25.308 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.308 INFO:teuthology.orchestra.run.smithi153.stdout: "mempool": { 2024-09-19T09:28:25.308 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:28:25.308 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_items": 0, 2024-09-19T09:28:25.308 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:28:25.308 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:28:25.308 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:28:25.309 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:28:25.309 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:28:25.309 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:28:25.309 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:28:25.309 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:28:25.309 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:28:25.309 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:28:25.309 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:28:25.309 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:28:25.309 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:28:25.309 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_items": 0, 2024-09-19T09:28:25.309 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:28:25.309 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_items": 0, 2024-09-19T09:28:25.309 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:28:25.310 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:28:25.310 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:28:25.310 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:28:25.310 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:28:25.310 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:28:25.310 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:28:25.310 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_items": 0, 2024-09-19T09:28:25.310 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:28:25.310 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:28:25.310 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:28:25.310 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_items": 0, 2024-09-19T09:28:25.310 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_bytes": 0, 2024-09-19T09:28:25.310 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_items": 0, 2024-09-19T09:28:25.310 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:28:25.310 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:28:25.311 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:28:25.311 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:28:25.311 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_bytes": 335149, 2024-09-19T09:28:25.311 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_items": 946, 2024-09-19T09:28:25.311 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_bytes": 440, 2024-09-19T09:28:25.311 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_items": 5, 2024-09-19T09:28:25.311 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_bytes": 0, 2024-09-19T09:28:25.311 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_items": 0, 2024-09-19T09:28:25.311 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:28:25.311 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_items": 0, 2024-09-19T09:28:25.311 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:28:25.311 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_items": 0, 2024-09-19T09:28:25.311 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_bytes": 7592, 2024-09-19T09:28:25.312 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_items": 156, 2024-09-19T09:28:25.312 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:28:25.312 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:28:25.312 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_bytes": 0, 2024-09-19T09:28:25.312 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_items": 0, 2024-09-19T09:28:25.312 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_bytes": 4719257, 2024-09-19T09:28:25.312 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_items": 145615, 2024-09-19T09:28:25.312 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_bytes": 0, 2024-09-19T09:28:25.312 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_items": 0, 2024-09-19T09:28:25.312 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_bytes": 0, 2024-09-19T09:28:25.312 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_items": 0 2024-09-19T09:28:25.312 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.312 INFO:teuthology.orchestra.run.smithi153.stdout: "objecter": { 2024-09-19T09:28:25.312 INFO:teuthology.orchestra.run.smithi153.stdout: "op_active": 0, 2024-09-19T09:28:25.313 INFO:teuthology.orchestra.run.smithi153.stdout: "op_laggy": 0, 2024-09-19T09:28:25.313 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send": 342810, 2024-09-19T09:28:25.313 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send_bytes": 1222236656, 2024-09-19T09:28:25.313 INFO:teuthology.orchestra.run.smithi153.stdout: "op_resend": 32, 2024-09-19T09:28:25.313 INFO:teuthology.orchestra.run.smithi153.stdout: "op_reply": 342810, 2024-09-19T09:28:25.313 INFO:teuthology.orchestra.run.smithi153.stdout: "op_latency": { 2024-09-19T09:28:25.313 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 342810, 2024-09-19T09:28:25.313 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 11143.322490399, 2024-09-19T09:28:25.313 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.032505826 2024-09-19T09:28:25.313 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.313 INFO:teuthology.orchestra.run.smithi153.stdout: "op_inflight": 0, 2024-09-19T09:28:25.313 INFO:teuthology.orchestra.run.smithi153.stdout: "oplen_avg": { 2024-09-19T09:28:25.313 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 342810, 2024-09-19T09:28:25.313 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 480744 2024-09-19T09:28:25.314 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.314 INFO:teuthology.orchestra.run.smithi153.stdout: "op": 342810, 2024-09-19T09:28:25.314 INFO:teuthology.orchestra.run.smithi153.stdout: "op_r": 14718, 2024-09-19T09:28:25.314 INFO:teuthology.orchestra.run.smithi153.stdout: "op_w": 328092, 2024-09-19T09:28:25.314 INFO:teuthology.orchestra.run.smithi153.stdout: "op_rmw": 0, 2024-09-19T09:28:25.314 INFO:teuthology.orchestra.run.smithi153.stdout: "op_pg": 0, 2024-09-19T09:28:25.314 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_stat": 10843, 2024-09-19T09:28:25.314 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_create": 54112, 2024-09-19T09:28:25.314 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_read": 2411, 2024-09-19T09:28:25.314 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_write": 147232, 2024-09-19T09:28:25.314 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writefull": 1498, 2024-09-19T09:28:25.314 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writesame": 0, 2024-09-19T09:28:25.314 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_append": 0, 2024-09-19T09:28:25.314 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_zero": 0, 2024-09-19T09:28:25.314 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_truncate": 56, 2024-09-19T09:28:25.315 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_delete": 112387, 2024-09-19T09:28:25.315 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_mapext": 0, 2024-09-19T09:28:25.315 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_sparse_read": 0, 2024-09-19T09:28:25.315 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_clonerange": 0, 2024-09-19T09:28:25.315 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_getxattr": 12307, 2024-09-19T09:28:25.315 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_setxattr": 108224, 2024-09-19T09:28:25.315 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:28:25.315 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_rmxattr": 0, 2024-09-19T09:28:25.315 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:28:25.315 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_call": 0, 2024-09-19T09:28:25.315 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_watch": 0, 2024-09-19T09:28:25.315 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_notify": 0, 2024-09-19T09:28:25.315 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:28:25.315 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls": 0, 2024-09-19T09:28:25.316 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:28:25.316 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_other": 0, 2024-09-19T09:28:25.316 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_active": 0, 2024-09-19T09:28:25.316 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_send": 0, 2024-09-19T09:28:25.316 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_resend": 0, 2024-09-19T09:28:25.316 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_ping": 0, 2024-09-19T09:28:25.316 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_active": 0, 2024-09-19T09:28:25.316 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_send": 0, 2024-09-19T09:28:25.316 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_resend": 0, 2024-09-19T09:28:25.316 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_active": 0, 2024-09-19T09:28:25.316 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_send": 0, 2024-09-19T09:28:25.316 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_resend": 0, 2024-09-19T09:28:25.316 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_active": 0, 2024-09-19T09:28:25.316 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_send": 0, 2024-09-19T09:28:25.316 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_resend": 0, 2024-09-19T09:28:25.317 INFO:teuthology.orchestra.run.smithi153.stdout: "command_active": 0, 2024-09-19T09:28:25.317 INFO:teuthology.orchestra.run.smithi153.stdout: "command_send": 0, 2024-09-19T09:28:25.317 INFO:teuthology.orchestra.run.smithi153.stdout: "command_resend": 0, 2024-09-19T09:28:25.317 INFO:teuthology.orchestra.run.smithi153.stdout: "map_epoch": 85, 2024-09-19T09:28:25.317 INFO:teuthology.orchestra.run.smithi153.stdout: "map_full": 0, 2024-09-19T09:28:25.317 INFO:teuthology.orchestra.run.smithi153.stdout: "map_inc": 6, 2024-09-19T09:28:25.317 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_sessions": 12, 2024-09-19T09:28:25.317 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_open": 12, 2024-09-19T09:28:25.317 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_close": 0, 2024-09-19T09:28:25.317 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_laggy": 0, 2024-09-19T09:28:25.317 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_wr": 23470, 2024-09-19T09:28:25.317 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_rd": 0, 2024-09-19T09:28:25.320 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_del": 8204 2024-09-19T09:28:25.320 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.320 INFO:teuthology.orchestra.run.smithi153.stdout: "oft": { 2024-09-19T09:28:25.320 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_objs": 1, 2024-09-19T09:28:25.320 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:28:25.320 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_updates": 0, 2024-09-19T09:28:25.320 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_removes": 0 2024-09-19T09:28:25.320 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.320 INFO:teuthology.orchestra.run.smithi153.stdout: "osd-slow-ops": { 2024-09-19T09:28:25.320 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_ops_count": 8298 2024-09-19T09:28:25.320 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.320 INFO:teuthology.orchestra.run.smithi153.stdout: "purge_queue": { 2024-09-19T09:28:25.321 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops": 0, 2024-09-19T09:28:25.321 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops_high_water": 6, 2024-09-19T09:28:25.321 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing": 0, 2024-09-19T09:28:25.321 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_high_water": 6, 2024-09-19T09:28:25.321 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed_ops": 110895, 2024-09-19T09:28:25.321 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed": 110895, 2024-09-19T09:28:25.321 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_item_in_journal": 0 2024-09-19T09:28:25.321 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.321 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:28:25.321 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-19T09:28:25.321 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-09-19T09:28:25.321 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-19T09:28:25.321 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 1656697, 2024-09-19T09:28:25.321 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 322649568, 2024-09-19T09:28:25.322 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:25.322 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 1656697, 2024-09-19T09:28:25.322 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-19T09:28:25.322 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-19T09:28:25.322 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 1656697, 2024-09-19T09:28:25.322 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 322649568, 2024-09-19T09:28:25.322 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-19T09:28:25.322 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:25.322 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:25.322 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:25.322 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:25.322 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.322 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_bytes": { 2024-09-19T09:28:25.322 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-19T09:28:25.323 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-09-19T09:28:25.323 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-19T09:28:25.323 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-19T09:28:25.323 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-19T09:28:25.323 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:25.323 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:25.323 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 342810, 2024-09-19T09:28:25.323 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 1233437051, 2024-09-19T09:28:25.323 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 342810, 2024-09-19T09:28:25.323 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 1233437051, 2024-09-19T09:28:25.323 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-19T09:28:25.323 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:25.323 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:25.324 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:25.324 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:25.324 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.324 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_ops": { 2024-09-19T09:28:25.324 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-19T09:28:25.324 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 1024, 2024-09-19T09:28:25.324 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-19T09:28:25.324 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-19T09:28:25.324 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-19T09:28:25.324 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:25.324 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:25.324 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 342810, 2024-09-19T09:28:25.324 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 342810, 2024-09-19T09:28:25.324 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 342810, 2024-09-19T09:28:25.325 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 342810, 2024-09-19T09:28:25.325 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-19T09:28:25.325 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:25.325 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:25.325 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:25.325 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:25.325 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.325 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:28:25.325 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-19T09:28:25.325 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-09-19T09:28:25.325 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-19T09:28:25.325 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 110895, 2024-09-19T09:28:25.325 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 11200395, 2024-09-19T09:28:25.326 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:25.326 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 110895, 2024-09-19T09:28:25.326 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-19T09:28:25.326 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-19T09:28:25.326 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 2375, 2024-09-19T09:28:25.326 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 11200395, 2024-09-19T09:28:25.326 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-19T09:28:25.326 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:25.326 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:25.326 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:25.326 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:25.326 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:25.326 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle-0x560de8f62960": { 2024-09-19T09:28:25.326 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-19T09:28:25.326 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-09-19T09:28:25.327 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-19T09:28:25.327 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 529908, 2024-09-19T09:28:25.327 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 1127118023, 2024-09-19T09:28:25.327 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:25.327 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 529908, 2024-09-19T09:28:25.327 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-19T09:28:25.327 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-19T09:28:25.327 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 144857, 2024-09-19T09:28:25.327 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 1127118023, 2024-09-19T09:28:25.327 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-19T09:28:25.327 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:25.327 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:25.327 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:25.327 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:25.328 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:25.328 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-09-19T09:28:26.302 INFO:tasks.check_counter:Daemon mds.b mds.exported=3 2024-09-19T09:28:26.302 INFO:tasks.check_counter:Daemon mds.b mds.imported=3 2024-09-19T09:28:26.302 DEBUG:tasks.check_counter:Getting stats from e 2024-09-19T09:28:26.303 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-19T09:28:26.365 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:26 smithi153 ceph-mon[35071]: pgmap v1112: 97 pgs: 97 active+clean; 302 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.1 KiB/s wr, 11 op/s 2024-09-19T09:28:26.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:26 smithi196 ceph-mon[35150]: pgmap v1112: 97 pgs: 97 active+clean; 302 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.1 KiB/s wr, 11 op/s 2024-09-19T09:28:26.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:26 smithi120 ceph-mon[31150]: pgmap v1112: 97 pgs: 97 active+clean; 302 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.1 KiB/s wr, 11 op/s 2024-09-19T09:28:26.699 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T09:28:27.337 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-09-19T09:28:27.337 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:28:27.337 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 1627, 2024-09-19T09:28:27.338 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 1627, 2024-09-19T09:28:27.338 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 252133023, 2024-09-19T09:28:27.338 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 516572, 2024-09-19T09:28:27.338 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 8, 2024-09-19T09:28:27.338 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 4, 2024-09-19T09:28:27.338 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.270097205, 2024-09-19T09:28:27.338 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.039546370, 2024-09-19T09:28:27.338 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.165601954, 2024-09-19T09:28:27.338 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.054410855, 2024-09-19T09:28:27.338 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:27.338 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1626, 2024-09-19T09:28:27.338 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.062988701, 2024-09-19T09:28:27.338 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000038738 2024-09-19T09:28:27.338 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.339 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:27.339 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.339 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.339 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.339 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.339 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-19T09:28:27.339 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T09:28:27.339 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.339 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:28:27.339 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 683, 2024-09-19T09:28:27.339 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 829, 2024-09-19T09:28:27.339 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 184321664, 2024-09-19T09:28:27.340 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 563333, 2024-09-19T09:28:27.340 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 7, 2024-09-19T09:28:27.340 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 6, 2024-09-19T09:28:27.340 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.169504824, 2024-09-19T09:28:27.340 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.016799234, 2024-09-19T09:28:27.340 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.132161396, 2024-09-19T09:28:27.340 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.015139214, 2024-09-19T09:28:27.340 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:27.340 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 827, 2024-09-19T09:28:27.340 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.029299912, 2024-09-19T09:28:27.340 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000035429 2024-09-19T09:28:27.340 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.340 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:27.341 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.341 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.341 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.341 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.341 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T09:28:27.341 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 346400 2024-09-19T09:28:27.341 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.341 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:28:27.341 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 1233, 2024-09-19T09:28:27.341 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 1225, 2024-09-19T09:28:27.341 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 235836695, 2024-09-19T09:28:27.341 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 445556, 2024-09-19T09:28:27.341 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 5, 2024-09-19T09:28:27.341 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 4, 2024-09-19T09:28:27.342 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.235387386, 2024-09-19T09:28:27.342 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.030868362, 2024-09-19T09:28:27.342 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.180624062, 2024-09-19T09:28:27.342 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.022712203, 2024-09-19T09:28:27.342 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:27.342 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1224, 2024-09-19T09:28:27.342 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.041226779, 2024-09-19T09:28:27.342 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000033682 2024-09-19T09:28:27.342 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.342 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:27.342 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.342 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.342 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.342 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.343 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 258528, 2024-09-19T09:28:27.343 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 118960 2024-09-19T09:28:27.343 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.343 INFO:teuthology.orchestra.run.smithi153.stdout: "cct": { 2024-09-19T09:28:27.343 INFO:teuthology.orchestra.run.smithi153.stdout: "total_workers": 1, 2024-09-19T09:28:27.343 INFO:teuthology.orchestra.run.smithi153.stdout: "unhealthy_workers": 0 2024-09-19T09:28:27.343 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.343 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-MDSRank": { 2024-09-19T09:28:27.343 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-09-19T09:28:27.343 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-09-19T09:28:27.343 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 5177, 2024-09-19T09:28:27.343 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.679496485, 2024-09-19T09:28:27.343 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000131252 2024-09-19T09:28:27.344 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:27.344 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.344 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-PurgeQueue": { 2024-09-19T09:28:27.344 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-09-19T09:28:27.344 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-09-19T09:28:27.344 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.344 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.344 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.344 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:27.344 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.344 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-09-19T09:28:27.344 INFO:teuthology.orchestra.run.smithi153.stdout: "request": 0, 2024-09-19T09:28:27.344 INFO:teuthology.orchestra.run.smithi153.stdout: "reply": 0, 2024-09-19T09:28:27.345 INFO:teuthology.orchestra.run.smithi153.stdout: "reply_latency": { 2024-09-19T09:28:27.345 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.345 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.345 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.345 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.345 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_reply": 0, 2024-09-19T09:28:27.345 INFO:teuthology.orchestra.run.smithi153.stdout: "forward": 0, 2024-09-19T09:28:27.345 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_complete": 0, 2024-09-19T09:28:27.345 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_keys": 0, 2024-09-19T09:28:27.345 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_commit": 0, 2024-09-19T09:28:27.345 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_split": 0, 2024-09-19T09:28:27.345 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_merge": 0, 2024-09-19T09:28:27.345 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes": 66200, 2024-09-19T09:28:27.346 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_top": 45489, 2024-09-19T09:28:27.346 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_bottom": 20711, 2024-09-19T09:28:27.346 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pin_tail": 0, 2024-09-19T09:28:27.346 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pinned": 1215, 2024-09-19T09:28:27.346 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_expired": 0, 2024-09-19T09:28:27.346 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_with_caps": 0, 2024-09-19T09:28:27.346 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": 0, 2024-09-19T09:28:27.346 INFO:teuthology.orchestra.run.smithi153.stdout: "subtrees": 2, 2024-09-19T09:28:27.346 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse": 0, 2024-09-19T09:28:27.346 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_hit": 0, 2024-09-19T09:28:27.346 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_forward": 0, 2024-09-19T09:28:27.346 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_discover": 0, 2024-09-19T09:28:27.346 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_dir_fetch": 0, 2024-09-19T09:28:27.347 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_remote_ino": 0, 2024-09-19T09:28:27.347 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_lock": 0, 2024-09-19T09:28:27.347 INFO:teuthology.orchestra.run.smithi153.stdout: "load_cent": 0, 2024-09-19T09:28:27.347 INFO:teuthology.orchestra.run.smithi153.stdout: "q": 0, 2024-09-19T09:28:27.347 INFO:teuthology.orchestra.run.smithi153.stdout: "exported": 0, 2024-09-19T09:28:27.347 INFO:teuthology.orchestra.run.smithi153.stdout: "exported_inodes": 0, 2024-09-19T09:28:27.347 INFO:teuthology.orchestra.run.smithi153.stdout: "imported": 0, 2024-09-19T09:28:27.347 INFO:teuthology.orchestra.run.smithi153.stdout: "imported_inodes": 0, 2024-09-19T09:28:27.347 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_dir_fetch": 0, 2024-09-19T09:28:27.347 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_backtrace_fetch": 0, 2024-09-19T09:28:27.347 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_peer_discover": 0, 2024-09-19T09:28:27.347 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rfiles": 114, 2024-09-19T09:28:27.347 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rbytes": 34500714, 2024-09-19T09:28:27.347 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rsnaps": 0, 2024-09-19T09:28:27.348 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T09:28:27.348 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_set_tag": 0, 2024-09-19T09:28:27.348 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:28:27.348 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:28:27.348 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_inodes": 0, 2024-09-19T09:28:27.348 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T09:28:27.348 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T09:28:27.348 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_file_inodes": 0, 2024-09-19T09:28:27.348 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_inode_file_caps": 0, 2024-09-19T09:28:27.348 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T09:28:27.348 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_grant": 0, 2024-09-19T09:28:27.348 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T09:28:27.349 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:28:27.349 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:28:27.349 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps": 0, 2024-09-19T09:28:27.349 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps_dirty": 0, 2024-09-19T09:28:27.349 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_cap_release": 0, 2024-09-19T09:28:27.349 INFO:teuthology.orchestra.run.smithi153.stdout: "process_request_cap_release": 0 2024-09-19T09:28:27.349 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.349 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_cache": { 2024-09-19T09:28:27.349 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update": 0, 2024-09-19T09:28:27.349 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update_receipt": 0, 2024-09-19T09:28:27.349 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_try_discover": 0, 2024-09-19T09:28:27.349 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_send_discover": 0, 2024-09-19T09:28:27.349 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_handle_discover": 0, 2024-09-19T09:28:27.349 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays": 0, 2024-09-19T09:28:27.349 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_delayed": 0, 2024-09-19T09:28:27.350 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:28:27.350 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_created": 0, 2024-09-19T09:28:27.350 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_enqueued": 0, 2024-09-19T09:28:27.350 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_reintegrated": 0, 2024-09-19T09:28:27.350 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_migrated": 0, 2024-09-19T09:28:27.350 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_processing": 0, 2024-09-19T09:28:27.350 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:28:27.350 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:28:27.350 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_started": 0, 2024-09-19T09:28:27.350 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_completed": 0, 2024-09-19T09:28:27.350 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_path": 0, 2024-09-19T09:28:27.350 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_inode": 0, 2024-09-19T09:28:27.350 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T09:28:27.351 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_exportdir": 0, 2024-09-19T09:28:27.351 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_flush": 0, 2024-09-19T09:28:27.351 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragmentdir": 0, 2024-09-19T09:28:27.351 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragstats": 0, 2024-09-19T09:28:27.351 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_inodestats": 0 2024-09-19T09:28:27.351 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.351 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_log": { 2024-09-19T09:28:27.351 INFO:teuthology.orchestra.run.smithi153.stdout: "evlrg": 0, 2024-09-19T09:28:27.351 INFO:teuthology.orchestra.run.smithi153.stdout: "evadd": 0, 2024-09-19T09:28:27.351 INFO:teuthology.orchestra.run.smithi153.stdout: "evex": 0, 2024-09-19T09:28:27.351 INFO:teuthology.orchestra.run.smithi153.stdout: "evtrm": 0, 2024-09-19T09:28:27.351 INFO:teuthology.orchestra.run.smithi153.stdout: "ev": 314818, 2024-09-19T09:28:27.351 INFO:teuthology.orchestra.run.smithi153.stdout: "evexg": 0, 2024-09-19T09:28:27.351 INFO:teuthology.orchestra.run.smithi153.stdout: "evexd": 0, 2024-09-19T09:28:27.352 INFO:teuthology.orchestra.run.smithi153.stdout: "segadd": 0, 2024-09-19T09:28:27.352 INFO:teuthology.orchestra.run.smithi153.stdout: "segex": 0, 2024-09-19T09:28:27.352 INFO:teuthology.orchestra.run.smithi153.stdout: "segtrm": 0, 2024-09-19T09:28:27.352 INFO:teuthology.orchestra.run.smithi153.stdout: "seg": 3, 2024-09-19T09:28:27.352 INFO:teuthology.orchestra.run.smithi153.stdout: "segmjr": 52, 2024-09-19T09:28:27.352 INFO:teuthology.orchestra.run.smithi153.stdout: "segexg": 0, 2024-09-19T09:28:27.352 INFO:teuthology.orchestra.run.smithi153.stdout: "segexd": 0, 2024-09-19T09:28:27.352 INFO:teuthology.orchestra.run.smithi153.stdout: "expos": 1126178290, 2024-09-19T09:28:27.352 INFO:teuthology.orchestra.run.smithi153.stdout: "wrpos": 1132894359, 2024-09-19T09:28:27.352 INFO:teuthology.orchestra.run.smithi153.stdout: "rdpos": 1131778950, 2024-09-19T09:28:27.352 INFO:teuthology.orchestra.run.smithi153.stdout: "jlat": { 2024-09-19T09:28:27.352 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.352 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.353 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.353 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.353 INFO:teuthology.orchestra.run.smithi153.stdout: "replayed": 314818 2024-09-19T09:28:27.353 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.353 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_mem": { 2024-09-19T09:28:27.353 INFO:teuthology.orchestra.run.smithi153.stdout: "ino": 847, 2024-09-19T09:28:27.353 INFO:teuthology.orchestra.run.smithi153.stdout: "ino+": 66102, 2024-09-19T09:28:27.353 INFO:teuthology.orchestra.run.smithi153.stdout: "ino-": 65255, 2024-09-19T09:28:27.353 INFO:teuthology.orchestra.run.smithi153.stdout: "dir": 67, 2024-09-19T09:28:27.353 INFO:teuthology.orchestra.run.smithi153.stdout: "dir+": 424, 2024-09-19T09:28:27.353 INFO:teuthology.orchestra.run.smithi153.stdout: "dir-": 357, 2024-09-19T09:28:27.353 INFO:teuthology.orchestra.run.smithi153.stdout: "dn": 66200, 2024-09-19T09:28:27.353 INFO:teuthology.orchestra.run.smithi153.stdout: "dn+": 67476, 2024-09-19T09:28:27.354 INFO:teuthology.orchestra.run.smithi153.stdout: "dn-": 1276, 2024-09-19T09:28:27.354 INFO:teuthology.orchestra.run.smithi153.stdout: "cap": 0, 2024-09-19T09:28:27.354 INFO:teuthology.orchestra.run.smithi153.stdout: "cap+": 0, 2024-09-19T09:28:27.354 INFO:teuthology.orchestra.run.smithi153.stdout: "cap-": 0, 2024-09-19T09:28:27.354 INFO:teuthology.orchestra.run.smithi153.stdout: "rss": 110628, 2024-09-19T09:28:27.354 INFO:teuthology.orchestra.run.smithi153.stdout: "heap": 215328 2024-09-19T09:28:27.354 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.354 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_server": { 2024-09-19T09:28:27.354 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_client_request": 0, 2024-09-19T09:28:27.354 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_server_request": 0, 2024-09-19T09:28:27.354 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_request": 0, 2024-09-19T09:28:27.354 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_session": 0, 2024-09-19T09:28:27.355 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_peer_request": 0, 2024-09-19T09:28:27.355 INFO:teuthology.orchestra.run.smithi153.stdout: "req_create_latency": { 2024-09-19T09:28:27.355 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.355 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.355 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.355 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.355 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getattr_latency": { 2024-09-19T09:28:27.355 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.355 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.355 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.355 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.355 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getfilelock_latency": { 2024-09-19T09:28:27.355 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.356 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.356 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.356 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.356 INFO:teuthology.orchestra.run.smithi153.stdout: "req_link_latency": { 2024-09-19T09:28:27.356 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.356 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.356 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.356 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.356 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookup_latency": { 2024-09-19T09:28:27.356 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.356 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.356 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.356 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.357 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookuphash_latency": { 2024-09-19T09:28:27.357 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.357 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.357 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.357 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.357 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupino_latency": { 2024-09-19T09:28:27.357 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.357 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.357 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.357 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.357 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupname_latency": { 2024-09-19T09:28:27.357 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.357 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.358 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.358 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.358 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupparent_latency": { 2024-09-19T09:28:27.358 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.358 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.358 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.358 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.358 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupsnap_latency": { 2024-09-19T09:28:27.358 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.358 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.358 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.358 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.359 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lssnap_latency": { 2024-09-19T09:28:27.359 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.359 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.359 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.359 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.360 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mkdir_latency": { 2024-09-19T09:28:27.360 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.360 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.360 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.360 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.360 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mknod_latency": { 2024-09-19T09:28:27.360 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.360 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.360 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.360 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.360 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mksnap_latency": { 2024-09-19T09:28:27.360 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.361 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.361 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.361 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.361 INFO:teuthology.orchestra.run.smithi153.stdout: "req_open_latency": { 2024-09-19T09:28:27.361 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.361 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.361 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.361 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.361 INFO:teuthology.orchestra.run.smithi153.stdout: "req_readdir_latency": { 2024-09-19T09:28:27.361 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.361 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.361 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.361 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.361 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rename_latency": { 2024-09-19T09:28:27.362 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.362 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.362 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.362 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.362 INFO:teuthology.orchestra.run.smithi153.stdout: "req_renamesnap_latency": { 2024-09-19T09:28:27.362 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.362 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.362 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.362 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.362 INFO:teuthology.orchestra.run.smithi153.stdout: "req_snapdiff_latency": { 2024-09-19T09:28:27.362 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.362 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.362 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.363 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.363 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmdir_latency": { 2024-09-19T09:28:27.363 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.363 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.363 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.363 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.363 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmsnap_latency": { 2024-09-19T09:28:27.363 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.363 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.363 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.363 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.363 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmxattr_latency": { 2024-09-19T09:28:27.363 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.364 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.364 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.364 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.364 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setattr_latency": { 2024-09-19T09:28:27.364 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.364 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.364 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.364 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.364 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setdirlayout_latency": { 2024-09-19T09:28:27.364 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.364 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.364 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.364 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.364 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setfilelock_latency": { 2024-09-19T09:28:27.365 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.365 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.365 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.365 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.365 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setlayout_latency": { 2024-09-19T09:28:27.365 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.365 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.365 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.365 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.365 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setxattr_latency": { 2024-09-19T09:28:27.365 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.365 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.365 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.366 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.366 INFO:teuthology.orchestra.run.smithi153.stdout: "req_symlink_latency": { 2024-09-19T09:28:27.366 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.366 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.366 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.366 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.366 INFO:teuthology.orchestra.run.smithi153.stdout: "req_unlink_latency": { 2024-09-19T09:28:27.368 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.368 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.368 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.368 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.368 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:28:27.368 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:28:27.368 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getvxattr_latency": { 2024-09-19T09:28:27.368 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.368 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.368 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.369 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:27.369 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.369 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_sessions": { 2024-09-19T09:28:27.369 INFO:teuthology.orchestra.run.smithi153.stdout: "session_count": 1, 2024-09-19T09:28:27.369 INFO:teuthology.orchestra.run.smithi153.stdout: "session_add": 1, 2024-09-19T09:28:27.369 INFO:teuthology.orchestra.run.smithi153.stdout: "session_remove": 0, 2024-09-19T09:28:27.369 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_open": 0, 2024-09-19T09:28:27.369 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_stale": 0, 2024-09-19T09:28:27.369 INFO:teuthology.orchestra.run.smithi153.stdout: "total_load": 0, 2024-09-19T09:28:27.369 INFO:teuthology.orchestra.run.smithi153.stdout: "average_load": 0, 2024-09-19T09:28:27.369 INFO:teuthology.orchestra.run.smithi153.stdout: "avg_session_uptime": 2833, 2024-09-19T09:28:27.369 INFO:teuthology.orchestra.run.smithi153.stdout: "mdthresh_evicted": 0 2024-09-19T09:28:27.369 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.370 INFO:teuthology.orchestra.run.smithi153.stdout: "mempool": { 2024-09-19T09:28:27.370 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:28:27.370 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_items": 0, 2024-09-19T09:28:27.370 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:28:27.370 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:28:27.370 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:28:27.370 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:28:27.370 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:28:27.370 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:28:27.370 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:28:27.370 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:28:27.370 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:28:27.370 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:28:27.371 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:28:27.371 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:28:27.371 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:28:27.371 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_items": 0, 2024-09-19T09:28:27.371 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:28:27.371 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_items": 0, 2024-09-19T09:28:27.371 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:28:27.371 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:28:27.371 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:28:27.371 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:28:27.371 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:28:27.371 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:28:27.371 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:28:27.371 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_items": 0, 2024-09-19T09:28:27.371 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:28:27.372 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:28:27.372 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:28:27.372 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_items": 0, 2024-09-19T09:28:27.372 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_bytes": 0, 2024-09-19T09:28:27.372 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_items": 0, 2024-09-19T09:28:27.372 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:28:27.372 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:28:27.372 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:28:27.372 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:28:27.372 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_bytes": 269357, 2024-09-19T09:28:27.372 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_items": 906, 2024-09-19T09:28:27.372 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:28:27.372 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_items": 0, 2024-09-19T09:28:27.372 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_bytes": 0, 2024-09-19T09:28:27.373 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_items": 0, 2024-09-19T09:28:27.373 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:28:27.373 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_items": 0, 2024-09-19T09:28:27.373 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:28:27.373 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_items": 0, 2024-09-19T09:28:27.373 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_bytes": 7592, 2024-09-19T09:28:27.373 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_items": 156, 2024-09-19T09:28:27.373 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:28:27.373 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:28:27.373 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_bytes": 0, 2024-09-19T09:28:27.373 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_items": 0, 2024-09-19T09:28:27.373 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_bytes": 48770204, 2024-09-19T09:28:27.373 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_items": 374457, 2024-09-19T09:28:27.373 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_bytes": 0, 2024-09-19T09:28:27.373 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_items": 0, 2024-09-19T09:28:27.373 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_bytes": 0, 2024-09-19T09:28:27.374 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_items": 0 2024-09-19T09:28:27.374 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.374 INFO:teuthology.orchestra.run.smithi153.stdout: "objecter": { 2024-09-19T09:28:27.374 INFO:teuthology.orchestra.run.smithi153.stdout: "op_active": 0, 2024-09-19T09:28:27.374 INFO:teuthology.orchestra.run.smithi153.stdout: "op_laggy": 0, 2024-09-19T09:28:27.374 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send": 3332, 2024-09-19T09:28:27.374 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send_bytes": 26, 2024-09-19T09:28:27.374 INFO:teuthology.orchestra.run.smithi153.stdout: "op_resend": 4, 2024-09-19T09:28:27.374 INFO:teuthology.orchestra.run.smithi153.stdout: "op_reply": 3332, 2024-09-19T09:28:27.374 INFO:teuthology.orchestra.run.smithi153.stdout: "op_latency": { 2024-09-19T09:28:27.374 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 3332, 2024-09-19T09:28:27.374 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 11.129801494, 2024-09-19T09:28:27.374 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.003340276 2024-09-19T09:28:27.374 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.374 INFO:teuthology.orchestra.run.smithi153.stdout: "op_inflight": 0, 2024-09-19T09:28:27.375 INFO:teuthology.orchestra.run.smithi153.stdout: "oplen_avg": { 2024-09-19T09:28:27.375 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 3332, 2024-09-19T09:28:27.375 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 3333 2024-09-19T09:28:27.375 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.375 INFO:teuthology.orchestra.run.smithi153.stdout: "op": 3332, 2024-09-19T09:28:27.375 INFO:teuthology.orchestra.run.smithi153.stdout: "op_r": 3332, 2024-09-19T09:28:27.375 INFO:teuthology.orchestra.run.smithi153.stdout: "op_w": 0, 2024-09-19T09:28:27.375 INFO:teuthology.orchestra.run.smithi153.stdout: "op_rmw": 0, 2024-09-19T09:28:27.375 INFO:teuthology.orchestra.run.smithi153.stdout: "op_pg": 0, 2024-09-19T09:28:27.375 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_stat": 1944, 2024-09-19T09:28:27.375 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_create": 0, 2024-09-19T09:28:27.375 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_read": 1385, 2024-09-19T09:28:27.375 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_write": 0, 2024-09-19T09:28:27.376 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writefull": 0, 2024-09-19T09:28:27.376 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writesame": 0, 2024-09-19T09:28:27.376 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_append": 0, 2024-09-19T09:28:27.376 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_zero": 0, 2024-09-19T09:28:27.376 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_truncate": 0, 2024-09-19T09:28:27.376 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_delete": 0, 2024-09-19T09:28:27.376 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_mapext": 0, 2024-09-19T09:28:27.376 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_sparse_read": 0, 2024-09-19T09:28:27.376 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_clonerange": 0, 2024-09-19T09:28:27.376 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_getxattr": 2, 2024-09-19T09:28:27.376 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_setxattr": 0, 2024-09-19T09:28:27.376 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:28:27.376 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_rmxattr": 0, 2024-09-19T09:28:27.376 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:28:27.376 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_call": 0, 2024-09-19T09:28:27.377 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_watch": 0, 2024-09-19T09:28:27.377 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_notify": 0, 2024-09-19T09:28:27.377 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:28:27.377 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls": 0, 2024-09-19T09:28:27.377 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:28:27.377 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_other": 0, 2024-09-19T09:28:27.377 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_active": 0, 2024-09-19T09:28:27.377 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_send": 0, 2024-09-19T09:28:27.377 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_resend": 0, 2024-09-19T09:28:27.377 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_ping": 0, 2024-09-19T09:28:27.377 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_active": 0, 2024-09-19T09:28:27.377 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_send": 0, 2024-09-19T09:28:27.377 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_resend": 0, 2024-09-19T09:28:27.378 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_active": 0, 2024-09-19T09:28:27.378 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_send": 0, 2024-09-19T09:28:27.378 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_resend": 0, 2024-09-19T09:28:27.378 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_active": 0, 2024-09-19T09:28:27.378 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_send": 0, 2024-09-19T09:28:27.378 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_resend": 0, 2024-09-19T09:28:27.378 INFO:teuthology.orchestra.run.smithi153.stdout: "command_active": 0, 2024-09-19T09:28:27.378 INFO:teuthology.orchestra.run.smithi153.stdout: "command_send": 0, 2024-09-19T09:28:27.378 INFO:teuthology.orchestra.run.smithi153.stdout: "command_resend": 0, 2024-09-19T09:28:27.378 INFO:teuthology.orchestra.run.smithi153.stdout: "map_epoch": 0, 2024-09-19T09:28:27.378 INFO:teuthology.orchestra.run.smithi153.stdout: "map_full": 0, 2024-09-19T09:28:27.379 INFO:teuthology.orchestra.run.smithi153.stdout: "map_inc": 0, 2024-09-19T09:28:27.379 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_sessions": 11, 2024-09-19T09:28:27.379 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_open": 11, 2024-09-19T09:28:27.379 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_close": 0, 2024-09-19T09:28:27.379 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_laggy": 0, 2024-09-19T09:28:27.379 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_wr": 0, 2024-09-19T09:28:27.379 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_rd": 2, 2024-09-19T09:28:27.379 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_del": 0 2024-09-19T09:28:27.379 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.379 INFO:teuthology.orchestra.run.smithi153.stdout: "oft": { 2024-09-19T09:28:27.379 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_objs": 0, 2024-09-19T09:28:27.379 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:28:27.380 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_updates": 0, 2024-09-19T09:28:27.380 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_removes": 0 2024-09-19T09:28:27.380 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.380 INFO:teuthology.orchestra.run.smithi153.stdout: "osd-slow-ops": { 2024-09-19T09:28:27.380 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_ops_count": 0 2024-09-19T09:28:27.380 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.380 INFO:teuthology.orchestra.run.smithi153.stdout: "purge_queue": { 2024-09-19T09:28:27.380 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops": 0, 2024-09-19T09:28:27.380 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T09:28:27.380 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing": 0, 2024-09-19T09:28:27.380 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_high_water": 0, 2024-09-19T09:28:27.380 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed_ops": 0, 2024-09-19T09:28:27.381 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed": 0, 2024-09-19T09:28:27.381 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_item_in_journal": 0 2024-09-19T09:28:27.381 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.381 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:28:27.381 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-19T09:28:27.381 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-09-19T09:28:27.381 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-19T09:28:27.381 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 3539, 2024-09-19T09:28:27.381 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 671959270, 2024-09-19T09:28:27.381 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:27.381 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 3539, 2024-09-19T09:28:27.381 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-19T09:28:27.381 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-19T09:28:27.381 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 3539, 2024-09-19T09:28:27.381 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 671959270, 2024-09-19T09:28:27.382 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-19T09:28:27.382 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.382 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.382 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.382 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:27.382 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.382 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_bytes": { 2024-09-19T09:28:27.382 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-19T09:28:27.382 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-09-19T09:28:27.382 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-19T09:28:27.382 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-19T09:28:27.382 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-19T09:28:27.382 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:27.382 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:27.383 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 3332, 2024-09-19T09:28:27.383 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 671119722, 2024-09-19T09:28:27.383 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 3332, 2024-09-19T09:28:27.383 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 671119722, 2024-09-19T09:28:27.383 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-19T09:28:27.383 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.383 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.383 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.383 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:27.383 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.383 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_ops": { 2024-09-19T09:28:27.383 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-19T09:28:27.383 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 1024, 2024-09-19T09:28:27.383 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-19T09:28:27.384 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-19T09:28:27.384 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-19T09:28:27.384 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:27.384 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:27.384 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 3332, 2024-09-19T09:28:27.384 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 3332, 2024-09-19T09:28:27.384 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 3332, 2024-09-19T09:28:27.384 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 3332, 2024-09-19T09:28:27.384 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-19T09:28:27.384 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.384 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.384 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.384 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:27.385 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.385 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:28:27.385 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-19T09:28:27.385 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-09-19T09:28:27.385 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-19T09:28:27.385 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-19T09:28:27.385 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-19T09:28:27.385 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:27.385 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:27.385 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-19T09:28:27.385 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-19T09:28:27.385 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 0, 2024-09-19T09:28:27.385 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 0, 2024-09-19T09:28:27.385 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-19T09:28:27.386 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.386 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.386 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.386 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:27.386 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:27.386 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle-0x555891b986e0": { 2024-09-19T09:28:27.386 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-19T09:28:27.386 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-09-19T09:28:27.386 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-19T09:28:27.386 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-19T09:28:27.386 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-19T09:28:27.386 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:27.386 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:27.387 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-19T09:28:27.387 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-19T09:28:27.387 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 0, 2024-09-19T09:28:27.387 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 0, 2024-09-19T09:28:27.387 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-19T09:28:27.387 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:27.387 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:27.387 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:27.387 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:27.387 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:27.387 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-09-19T09:28:28.275 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-19T09:28:28.275 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-19T09:28:28.275 DEBUG:tasks.check_counter:Getting stats from h 2024-09-19T09:28:28.275 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-19T09:28:28.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:28 smithi196 ceph-mon[35150]: pgmap v1113: 97 pgs: 97 active+clean; 316 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.8 MiB/s wr, 56 op/s 2024-09-19T09:28:28.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:28 smithi153 ceph-mon[35071]: pgmap v1113: 97 pgs: 97 active+clean; 316 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.8 MiB/s wr, 56 op/s 2024-09-19T09:28:28.540 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T09:28:28.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:28 smithi120 ceph-mon[31150]: pgmap v1113: 97 pgs: 97 active+clean; 316 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.8 MiB/s wr, 56 op/s 2024-09-19T09:28:29.089 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-09-19T09:28:29.089 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:28:29.089 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 42692, 2024-09-19T09:28:29.089 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 48216, 2024-09-19T09:28:29.089 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 14008466, 2024-09-19T09:28:29.089 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 39722971, 2024-09-19T09:28:29.090 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 18, 2024-09-19T09:28:29.090 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 6, 2024-09-19T09:28:29.090 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 2.975913456, 2024-09-19T09:28:29.090 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 1.193523098, 2024-09-19T09:28:29.090 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 2.187554497, 2024-09-19T09:28:29.090 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.112092434, 2024-09-19T09:28:29.090 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:29.090 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 48213, 2024-09-19T09:28:29.090 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 1.395274951, 2024-09-19T09:28:29.090 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000028939 2024-09-19T09:28:29.090 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.090 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:29.090 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 80995, 2024-09-19T09:28:29.090 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.011894046, 2024-09-19T09:28:29.091 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000146 2024-09-19T09:28:29.091 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.091 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 13106576, 2024-09-19T09:28:29.091 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 26743232 2024-09-19T09:28:29.091 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.091 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:28:29.091 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 166368, 2024-09-19T09:28:29.091 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 88247, 2024-09-19T09:28:29.091 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 45693411, 2024-09-19T09:28:29.091 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 15586840, 2024-09-19T09:28:29.091 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 9, 2024-09-19T09:28:29.091 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 6, 2024-09-19T09:28:29.091 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 7.809482143, 2024-09-19T09:28:29.091 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 3.316185531, 2024-09-19T09:28:29.092 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 6.175149586, 2024-09-19T09:28:29.092 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.024109549, 2024-09-19T09:28:29.092 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:29.092 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 88248, 2024-09-19T09:28:29.092 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 1.544675785, 2024-09-19T09:28:29.092 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000017503 2024-09-19T09:28:29.092 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.092 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:29.092 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 251424, 2024-09-19T09:28:29.092 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.033385761, 2024-09-19T09:28:29.092 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000132 2024-09-19T09:28:29.092 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.092 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 972384, 2024-09-19T09:28:29.093 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 294464 2024-09-19T09:28:29.093 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.093 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:28:29.093 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 43794, 2024-09-19T09:28:29.093 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 22682, 2024-09-19T09:28:29.093 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 11962433, 2024-09-19T09:28:29.093 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 5676647, 2024-09-19T09:28:29.093 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 83, 2024-09-19T09:28:29.093 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 8, 2024-09-19T09:28:29.093 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 1.827599275, 2024-09-19T09:28:29.093 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.637974412, 2024-09-19T09:28:29.093 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 1.574534251, 2024-09-19T09:28:29.093 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.024666856, 2024-09-19T09:28:29.094 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:29.094 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 22680, 2024-09-19T09:28:29.094 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.447776783, 2024-09-19T09:28:29.094 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000019743 2024-09-19T09:28:29.094 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.094 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:29.094 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 64325, 2024-09-19T09:28:29.094 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.008626012, 2024-09-19T09:28:29.094 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000134 2024-09-19T09:28:29.094 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.094 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T09:28:29.094 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 771584 2024-09-19T09:28:29.094 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.094 INFO:teuthology.orchestra.run.smithi153.stdout: "cct": { 2024-09-19T09:28:29.095 INFO:teuthology.orchestra.run.smithi153.stdout: "total_workers": 1, 2024-09-19T09:28:29.095 INFO:teuthology.orchestra.run.smithi153.stdout: "unhealthy_workers": 0 2024-09-19T09:28:29.095 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.095 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-MDSRank": { 2024-09-19T09:28:29.095 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-09-19T09:28:29.095 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-09-19T09:28:29.095 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 5561, 2024-09-19T09:28:29.095 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 2.767502935, 2024-09-19T09:28:29.095 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000497662 2024-09-19T09:28:29.095 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:29.095 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.095 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-PurgeQueue": { 2024-09-19T09:28:29.095 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-09-19T09:28:29.096 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-09-19T09:28:29.096 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 2139, 2024-09-19T09:28:29.096 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.096658414, 2024-09-19T09:28:29.096 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000045188 2024-09-19T09:28:29.096 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:29.096 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.096 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-09-19T09:28:29.096 INFO:teuthology.orchestra.run.smithi153.stdout: "request": 32544, 2024-09-19T09:28:29.096 INFO:teuthology.orchestra.run.smithi153.stdout: "reply": 5581, 2024-09-19T09:28:29.096 INFO:teuthology.orchestra.run.smithi153.stdout: "reply_latency": { 2024-09-19T09:28:29.096 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 5581, 2024-09-19T09:28:29.096 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 41.882773333, 2024-09-19T09:28:29.096 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.007504528 2024-09-19T09:28:29.097 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.097 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_reply": 0, 2024-09-19T09:28:29.097 INFO:teuthology.orchestra.run.smithi153.stdout: "forward": 26963, 2024-09-19T09:28:29.097 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_complete": 0, 2024-09-19T09:28:29.097 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_keys": 0, 2024-09-19T09:28:29.097 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_commit": 11, 2024-09-19T09:28:29.097 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_split": 0, 2024-09-19T09:28:29.097 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_merge": 0, 2024-09-19T09:28:29.097 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes": 55, 2024-09-19T09:28:29.097 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_top": 1, 2024-09-19T09:28:29.097 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_bottom": 54, 2024-09-19T09:28:29.097 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pin_tail": 0, 2024-09-19T09:28:29.097 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pinned": 52, 2024-09-19T09:28:29.097 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_expired": 78, 2024-09-19T09:28:29.098 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_with_caps": 40, 2024-09-19T09:28:29.098 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": 40, 2024-09-19T09:28:29.098 INFO:teuthology.orchestra.run.smithi153.stdout: "subtrees": 4, 2024-09-19T09:28:29.098 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse": 40480, 2024-09-19T09:28:29.098 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_hit": 4420, 2024-09-19T09:28:29.098 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_forward": 26962, 2024-09-19T09:28:29.098 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_discover": 19, 2024-09-19T09:28:29.098 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_dir_fetch": 0, 2024-09-19T09:28:29.098 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_remote_ino": 0, 2024-09-19T09:28:29.098 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_lock": 58, 2024-09-19T09:28:29.098 INFO:teuthology.orchestra.run.smithi153.stdout: "load_cent": 0, 2024-09-19T09:28:29.098 INFO:teuthology.orchestra.run.smithi153.stdout: "q": 0, 2024-09-19T09:28:29.098 INFO:teuthology.orchestra.run.smithi153.stdout: "exported": 1, 2024-09-19T09:28:29.099 INFO:teuthology.orchestra.run.smithi153.stdout: "exported_inodes": 0, 2024-09-19T09:28:29.099 INFO:teuthology.orchestra.run.smithi153.stdout: "imported": 1, 2024-09-19T09:28:29.099 INFO:teuthology.orchestra.run.smithi153.stdout: "imported_inodes": 41, 2024-09-19T09:28:29.099 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_dir_fetch": 0, 2024-09-19T09:28:29.099 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_backtrace_fetch": 331, 2024-09-19T09:28:29.099 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_peer_discover": 0, 2024-09-19T09:28:29.099 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rfiles": 114, 2024-09-19T09:28:29.099 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rbytes": 34500714, 2024-09-19T09:28:29.099 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rsnaps": 0, 2024-09-19T09:28:29.099 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_fetch": 331, 2024-09-19T09:28:29.099 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_set_tag": 0, 2024-09-19T09:28:29.099 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:28:29.099 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:28:29.100 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_inodes": 0, 2024-09-19T09:28:29.100 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T09:28:29.100 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T09:28:29.100 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_file_inodes": 331, 2024-09-19T09:28:29.100 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_inode_file_caps": 0, 2024-09-19T09:28:29.100 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_revoke": 1112, 2024-09-19T09:28:29.100 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_grant": 6709, 2024-09-19T09:28:29.100 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T09:28:29.100 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:28:29.100 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:28:29.100 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps": 4833, 2024-09-19T09:28:29.100 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps_dirty": 1468, 2024-09-19T09:28:29.100 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_cap_release": 862, 2024-09-19T09:28:29.101 INFO:teuthology.orchestra.run.smithi153.stdout: "process_request_cap_release": 4854 2024-09-19T09:28:29.101 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.101 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_cache": { 2024-09-19T09:28:29.101 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update": 18, 2024-09-19T09:28:29.101 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update_receipt": 47, 2024-09-19T09:28:29.101 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_try_discover": 32, 2024-09-19T09:28:29.101 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_send_discover": 36, 2024-09-19T09:28:29.101 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_handle_discover": 18, 2024-09-19T09:28:29.101 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays": 38, 2024-09-19T09:28:29.101 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_delayed": 0, 2024-09-19T09:28:29.101 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:28:29.101 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_created": 1008, 2024-09-19T09:28:29.101 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_enqueued": 970, 2024-09-19T09:28:29.101 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_reintegrated": 0, 2024-09-19T09:28:29.102 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_migrated": 0, 2024-09-19T09:28:29.102 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_processing": 0, 2024-09-19T09:28:29.102 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:28:29.102 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:28:29.102 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_started": 0, 2024-09-19T09:28:29.102 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_completed": 0, 2024-09-19T09:28:29.102 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_path": 22, 2024-09-19T09:28:29.102 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_inode": 788, 2024-09-19T09:28:29.102 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T09:28:29.102 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_exportdir": 1, 2024-09-19T09:28:29.102 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_flush": 0, 2024-09-19T09:28:29.102 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragmentdir": 0, 2024-09-19T09:28:29.102 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragstats": 0, 2024-09-19T09:28:29.103 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_inodestats": 0 2024-09-19T09:28:29.103 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.103 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_log": { 2024-09-19T09:28:29.103 INFO:teuthology.orchestra.run.smithi153.stdout: "evlrg": 0, 2024-09-19T09:28:29.103 INFO:teuthology.orchestra.run.smithi153.stdout: "evadd": 8666, 2024-09-19T09:28:29.103 INFO:teuthology.orchestra.run.smithi153.stdout: "evex": 0, 2024-09-19T09:28:29.103 INFO:teuthology.orchestra.run.smithi153.stdout: "evtrm": 0, 2024-09-19T09:28:29.103 INFO:teuthology.orchestra.run.smithi153.stdout: "ev": 8666, 2024-09-19T09:28:29.103 INFO:teuthology.orchestra.run.smithi153.stdout: "evexg": 0, 2024-09-19T09:28:29.103 INFO:teuthology.orchestra.run.smithi153.stdout: "evexd": 0, 2024-09-19T09:28:29.103 INFO:teuthology.orchestra.run.smithi153.stdout: "segadd": 11, 2024-09-19T09:28:29.103 INFO:teuthology.orchestra.run.smithi153.stdout: "segex": 0, 2024-09-19T09:28:29.103 INFO:teuthology.orchestra.run.smithi153.stdout: "segtrm": 0, 2024-09-19T09:28:29.103 INFO:teuthology.orchestra.run.smithi153.stdout: "seg": 11, 2024-09-19T09:28:29.104 INFO:teuthology.orchestra.run.smithi153.stdout: "segmjr": 2, 2024-09-19T09:28:29.104 INFO:teuthology.orchestra.run.smithi153.stdout: "segexg": 0, 2024-09-19T09:28:29.104 INFO:teuthology.orchestra.run.smithi153.stdout: "segexd": 0, 2024-09-19T09:28:29.104 INFO:teuthology.orchestra.run.smithi153.stdout: "expos": 4194304, 2024-09-19T09:28:29.104 INFO:teuthology.orchestra.run.smithi153.stdout: "wrpos": 16077816, 2024-09-19T09:28:29.104 INFO:teuthology.orchestra.run.smithi153.stdout: "rdpos": 0, 2024-09-19T09:28:29.104 INFO:teuthology.orchestra.run.smithi153.stdout: "jlat": { 2024-09-19T09:28:29.104 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 2674, 2024-09-19T09:28:29.104 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 18.864731832, 2024-09-19T09:28:29.104 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.007054873 2024-09-19T09:28:29.104 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.104 INFO:teuthology.orchestra.run.smithi153.stdout: "replayed": 0 2024-09-19T09:28:29.104 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.104 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_mem": { 2024-09-19T09:28:29.105 INFO:teuthology.orchestra.run.smithi153.stdout: "ino": 60, 2024-09-19T09:28:29.105 INFO:teuthology.orchestra.run.smithi153.stdout: "ino+": 1069, 2024-09-19T09:28:29.105 INFO:teuthology.orchestra.run.smithi153.stdout: "ino-": 1009, 2024-09-19T09:28:29.105 INFO:teuthology.orchestra.run.smithi153.stdout: "dir": 28, 2024-09-19T09:28:29.105 INFO:teuthology.orchestra.run.smithi153.stdout: "dir+": 68, 2024-09-19T09:28:29.105 INFO:teuthology.orchestra.run.smithi153.stdout: "dir-": 40, 2024-09-19T09:28:29.105 INFO:teuthology.orchestra.run.smithi153.stdout: "dn": 55, 2024-09-19T09:28:29.105 INFO:teuthology.orchestra.run.smithi153.stdout: "dn+": 2501, 2024-09-19T09:28:29.105 INFO:teuthology.orchestra.run.smithi153.stdout: "dn-": 2446, 2024-09-19T09:28:29.105 INFO:teuthology.orchestra.run.smithi153.stdout: "cap": 2, 2024-09-19T09:28:29.105 INFO:teuthology.orchestra.run.smithi153.stdout: "cap+": 1749, 2024-09-19T09:28:29.105 INFO:teuthology.orchestra.run.smithi153.stdout: "cap-": 1747, 2024-09-19T09:28:29.105 INFO:teuthology.orchestra.run.smithi153.stdout: "rss": 59004, 2024-09-19T09:28:29.105 INFO:teuthology.orchestra.run.smithi153.stdout: "heap": 223520 2024-09-19T09:28:29.106 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.106 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_server": { 2024-09-19T09:28:29.106 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_client_request": 38420, 2024-09-19T09:28:29.106 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_server_request": 2, 2024-09-19T09:28:29.106 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_request": 32544, 2024-09-19T09:28:29.106 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_session": 166, 2024-09-19T09:28:29.106 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_peer_request": 2069, 2024-09-19T09:28:29.106 INFO:teuthology.orchestra.run.smithi153.stdout: "req_create_latency": { 2024-09-19T09:28:29.106 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 984, 2024-09-19T09:28:29.106 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 30.858820657, 2024-09-19T09:28:29.106 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.031360590 2024-09-19T09:28:29.106 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.106 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getattr_latency": { 2024-09-19T09:28:29.106 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 4, 2024-09-19T09:28:29.107 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.012696255, 2024-09-19T09:28:29.107 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.003174063 2024-09-19T09:28:29.107 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.107 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getfilelock_latency": { 2024-09-19T09:28:29.107 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:29.107 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:29.107 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:29.107 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.107 INFO:teuthology.orchestra.run.smithi153.stdout: "req_link_latency": { 2024-09-19T09:28:29.107 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:29.107 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:29.107 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:29.107 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.107 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookup_latency": { 2024-09-19T09:28:29.108 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1846, 2024-09-19T09:28:29.108 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.763861181, 2024-09-19T09:28:29.108 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000413792 2024-09-19T09:28:29.108 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.108 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookuphash_latency": { 2024-09-19T09:28:29.108 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:29.108 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:29.108 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:29.108 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.108 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupino_latency": { 2024-09-19T09:28:29.108 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1, 2024-09-19T09:28:29.108 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000143770, 2024-09-19T09:28:29.108 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000143770 2024-09-19T09:28:29.109 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.109 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupname_latency": { 2024-09-19T09:28:29.109 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:29.109 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:29.109 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:29.109 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.109 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupparent_latency": { 2024-09-19T09:28:29.109 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:29.109 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:29.109 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:29.109 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.109 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupsnap_latency": { 2024-09-19T09:28:29.109 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:29.110 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:29.110 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:29.110 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.110 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lssnap_latency": { 2024-09-19T09:28:29.110 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:29.110 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:29.111 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:29.111 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.111 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mkdir_latency": { 2024-09-19T09:28:29.111 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:29.111 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:29.111 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:29.111 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.111 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mknod_latency": { 2024-09-19T09:28:29.111 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:29.111 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:29.111 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:29.111 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.111 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mksnap_latency": { 2024-09-19T09:28:29.112 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:29.112 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:29.112 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:29.112 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.112 INFO:teuthology.orchestra.run.smithi153.stdout: "req_open_latency": { 2024-09-19T09:28:29.112 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:29.112 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:29.112 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:29.112 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.112 INFO:teuthology.orchestra.run.smithi153.stdout: "req_readdir_latency": { 2024-09-19T09:28:29.112 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1680, 2024-09-19T09:28:29.112 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 1.471333547, 2024-09-19T09:28:29.112 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000875793 2024-09-19T09:28:29.112 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.113 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rename_latency": { 2024-09-19T09:28:29.113 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 58, 2024-09-19T09:28:29.113 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.703820747, 2024-09-19T09:28:29.113 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.012134840 2024-09-19T09:28:29.113 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.113 INFO:teuthology.orchestra.run.smithi153.stdout: "req_renamesnap_latency": { 2024-09-19T09:28:29.113 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:29.113 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:29.113 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:29.113 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.113 INFO:teuthology.orchestra.run.smithi153.stdout: "req_snapdiff_latency": { 2024-09-19T09:28:29.113 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:29.113 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:29.114 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:29.114 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.114 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmdir_latency": { 2024-09-19T09:28:29.114 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:29.114 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:29.114 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:29.114 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.114 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmsnap_latency": { 2024-09-19T09:28:29.114 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:29.114 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:29.114 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:29.114 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.114 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmxattr_latency": { 2024-09-19T09:28:29.115 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:29.115 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:29.115 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:29.115 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.115 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setattr_latency": { 2024-09-19T09:28:29.115 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:29.115 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:29.115 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:29.115 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.115 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setdirlayout_latency": { 2024-09-19T09:28:29.115 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:29.115 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:29.115 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:29.116 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.116 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setfilelock_latency": { 2024-09-19T09:28:29.116 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:29.116 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:29.116 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:29.116 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.116 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setlayout_latency": { 2024-09-19T09:28:29.116 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:29.116 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:29.116 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:29.116 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.116 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setxattr_latency": { 2024-09-19T09:28:29.116 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:29.116 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:29.117 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:29.117 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.117 INFO:teuthology.orchestra.run.smithi153.stdout: "req_symlink_latency": { 2024-09-19T09:28:29.117 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:29.117 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:29.117 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:29.117 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.117 INFO:teuthology.orchestra.run.smithi153.stdout: "req_unlink_latency": { 2024-09-19T09:28:29.117 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1008, 2024-09-19T09:28:29.117 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 8.072097176, 2024-09-19T09:28:29.117 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.008008032 2024-09-19T09:28:29.117 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.117 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:28:29.117 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:28:29.118 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getvxattr_latency": { 2024-09-19T09:28:29.118 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:29.118 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:29.118 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:29.118 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:29.118 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.118 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_sessions": { 2024-09-19T09:28:29.118 INFO:teuthology.orchestra.run.smithi153.stdout: "session_count": 1, 2024-09-19T09:28:29.118 INFO:teuthology.orchestra.run.smithi153.stdout: "session_add": 1, 2024-09-19T09:28:29.118 INFO:teuthology.orchestra.run.smithi153.stdout: "session_remove": 0, 2024-09-19T09:28:29.118 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_open": 1, 2024-09-19T09:28:29.118 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_stale": 0, 2024-09-19T09:28:29.118 INFO:teuthology.orchestra.run.smithi153.stdout: "total_load": 667, 2024-09-19T09:28:29.119 INFO:teuthology.orchestra.run.smithi153.stdout: "average_load": 667, 2024-09-19T09:28:29.119 INFO:teuthology.orchestra.run.smithi153.stdout: "avg_session_uptime": 1545, 2024-09-19T09:28:29.119 INFO:teuthology.orchestra.run.smithi153.stdout: "mdthresh_evicted": 0 2024-09-19T09:28:29.119 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.119 INFO:teuthology.orchestra.run.smithi153.stdout: "mempool": { 2024-09-19T09:28:29.119 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:28:29.119 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_items": 0, 2024-09-19T09:28:29.119 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:28:29.119 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:28:29.119 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:28:29.119 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:28:29.119 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:28:29.119 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:28:29.119 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:28:29.119 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:28:29.120 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:28:29.120 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:28:29.120 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:28:29.120 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:28:29.120 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:28:29.120 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_items": 0, 2024-09-19T09:28:29.120 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:28:29.120 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_items": 0, 2024-09-19T09:28:29.120 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:28:29.120 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:28:29.120 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:28:29.120 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:28:29.120 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:28:29.120 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:28:29.121 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:28:29.121 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_items": 0, 2024-09-19T09:28:29.121 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:28:29.121 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:28:29.121 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:28:29.121 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_items": 0, 2024-09-19T09:28:29.121 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_bytes": 0, 2024-09-19T09:28:29.121 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_items": 0, 2024-09-19T09:28:29.121 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:28:29.121 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:28:29.121 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:28:29.121 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:28:29.121 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_bytes": 297013, 2024-09-19T09:28:29.121 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_items": 144, 2024-09-19T09:28:29.122 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:28:29.122 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_items": 0, 2024-09-19T09:28:29.122 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_bytes": 0, 2024-09-19T09:28:29.122 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_items": 0, 2024-09-19T09:28:29.122 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:28:29.122 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_items": 0, 2024-09-19T09:28:29.122 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:28:29.122 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_items": 0, 2024-09-19T09:28:29.122 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_bytes": 7592, 2024-09-19T09:28:29.122 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_items": 156, 2024-09-19T09:28:29.122 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:28:29.122 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:28:29.122 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_bytes": 0, 2024-09-19T09:28:29.122 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_items": 0, 2024-09-19T09:28:29.123 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_bytes": 237091, 2024-09-19T09:28:29.123 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_items": 6121, 2024-09-19T09:28:29.123 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_bytes": 0, 2024-09-19T09:28:29.123 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_items": 0, 2024-09-19T09:28:29.123 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_bytes": 0, 2024-09-19T09:28:29.123 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_items": 0 2024-09-19T09:28:29.123 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.123 INFO:teuthology.orchestra.run.smithi153.stdout: "objecter": { 2024-09-19T09:28:29.123 INFO:teuthology.orchestra.run.smithi153.stdout: "op_active": 0, 2024-09-19T09:28:29.123 INFO:teuthology.orchestra.run.smithi153.stdout: "op_laggy": 0, 2024-09-19T09:28:29.123 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send": 4767, 2024-09-19T09:28:29.123 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send_bytes": 12111423, 2024-09-19T09:28:29.123 INFO:teuthology.orchestra.run.smithi153.stdout: "op_resend": 28, 2024-09-19T09:28:29.123 INFO:teuthology.orchestra.run.smithi153.stdout: "op_reply": 4767, 2024-09-19T09:28:29.123 INFO:teuthology.orchestra.run.smithi153.stdout: "op_latency": { 2024-09-19T09:28:29.124 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 4767, 2024-09-19T09:28:29.124 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 30.525454168, 2024-09-19T09:28:29.124 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.006403493 2024-09-19T09:28:29.124 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.124 INFO:teuthology.orchestra.run.smithi153.stdout: "op_inflight": 0, 2024-09-19T09:28:29.124 INFO:teuthology.orchestra.run.smithi153.stdout: "oplen_avg": { 2024-09-19T09:28:29.124 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 4767, 2024-09-19T09:28:29.124 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 5134 2024-09-19T09:28:29.124 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.125 INFO:teuthology.orchestra.run.smithi153.stdout: "op": 4767, 2024-09-19T09:28:29.125 INFO:teuthology.orchestra.run.smithi153.stdout: "op_r": 490, 2024-09-19T09:28:29.125 INFO:teuthology.orchestra.run.smithi153.stdout: "op_w": 4277, 2024-09-19T09:28:29.125 INFO:teuthology.orchestra.run.smithi153.stdout: "op_rmw": 0, 2024-09-19T09:28:29.125 INFO:teuthology.orchestra.run.smithi153.stdout: "op_pg": 0, 2024-09-19T09:28:29.125 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_stat": 0, 2024-09-19T09:28:29.125 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_create": 10, 2024-09-19T09:28:29.125 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_read": 159, 2024-09-19T09:28:29.126 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_write": 2833, 2024-09-19T09:28:29.126 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writefull": 140, 2024-09-19T09:28:29.126 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writesame": 0, 2024-09-19T09:28:29.126 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_append": 0, 2024-09-19T09:28:29.126 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_zero": 0, 2024-09-19T09:28:29.126 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_truncate": 0, 2024-09-19T09:28:29.126 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_delete": 984, 2024-09-19T09:28:29.126 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_mapext": 0, 2024-09-19T09:28:29.127 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_sparse_read": 0, 2024-09-19T09:28:29.127 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_clonerange": 0, 2024-09-19T09:28:29.127 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_getxattr": 331, 2024-09-19T09:28:29.127 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_setxattr": 20, 2024-09-19T09:28:29.127 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:28:29.127 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_rmxattr": 0, 2024-09-19T09:28:29.127 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:28:29.127 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_call": 0, 2024-09-19T09:28:29.127 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_watch": 0, 2024-09-19T09:28:29.127 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_notify": 0, 2024-09-19T09:28:29.127 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:28:29.127 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls": 0, 2024-09-19T09:28:29.128 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:28:29.128 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_other": 0, 2024-09-19T09:28:29.128 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_active": 0, 2024-09-19T09:28:29.128 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_send": 0, 2024-09-19T09:28:29.128 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_resend": 0, 2024-09-19T09:28:29.128 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_ping": 0, 2024-09-19T09:28:29.128 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_active": 0, 2024-09-19T09:28:29.128 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_send": 0, 2024-09-19T09:28:29.128 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_resend": 0, 2024-09-19T09:28:29.128 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_active": 0, 2024-09-19T09:28:29.128 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_send": 0, 2024-09-19T09:28:29.128 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_resend": 0, 2024-09-19T09:28:29.128 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_active": 0, 2024-09-19T09:28:29.128 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_send": 0, 2024-09-19T09:28:29.129 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_resend": 0, 2024-09-19T09:28:29.129 INFO:teuthology.orchestra.run.smithi153.stdout: "command_active": 0, 2024-09-19T09:28:29.129 INFO:teuthology.orchestra.run.smithi153.stdout: "command_send": 0, 2024-09-19T09:28:29.129 INFO:teuthology.orchestra.run.smithi153.stdout: "command_resend": 0, 2024-09-19T09:28:29.129 INFO:teuthology.orchestra.run.smithi153.stdout: "map_epoch": 85, 2024-09-19T09:28:29.129 INFO:teuthology.orchestra.run.smithi153.stdout: "map_full": 0, 2024-09-19T09:28:29.129 INFO:teuthology.orchestra.run.smithi153.stdout: "map_inc": 6, 2024-09-19T09:28:29.129 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_sessions": 12, 2024-09-19T09:28:29.129 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_open": 16, 2024-09-19T09:28:29.129 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_close": 4, 2024-09-19T09:28:29.129 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_laggy": 0, 2024-09-19T09:28:29.129 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_wr": 522, 2024-09-19T09:28:29.129 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_rd": 0, 2024-09-19T09:28:29.130 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_del": 135 2024-09-19T09:28:29.130 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.130 INFO:teuthology.orchestra.run.smithi153.stdout: "oft": { 2024-09-19T09:28:29.130 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_objs": 1, 2024-09-19T09:28:29.130 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:28:29.130 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_updates": 0, 2024-09-19T09:28:29.130 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_removes": 0 2024-09-19T09:28:29.130 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.130 INFO:teuthology.orchestra.run.smithi153.stdout: "osd-slow-ops": { 2024-09-19T09:28:29.130 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_ops_count": 739 2024-09-19T09:28:29.130 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.130 INFO:teuthology.orchestra.run.smithi153.stdout: "purge_queue": { 2024-09-19T09:28:29.130 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops": 0, 2024-09-19T09:28:29.130 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops_high_water": 6, 2024-09-19T09:28:29.131 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing": 0, 2024-09-19T09:28:29.131 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_high_water": 6, 2024-09-19T09:28:29.131 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed_ops": 970, 2024-09-19T09:28:29.131 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed": 970, 2024-09-19T09:28:29.131 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_item_in_journal": 0 2024-09-19T09:28:29.131 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.131 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:28:29.131 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-19T09:28:29.131 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-09-19T09:28:29.131 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-19T09:28:29.131 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 252850, 2024-09-19T09:28:29.131 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 47186935, 2024-09-19T09:28:29.132 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:29.132 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 252850, 2024-09-19T09:28:29.132 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-19T09:28:29.132 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-19T09:28:29.132 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 252850, 2024-09-19T09:28:29.132 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 47186935, 2024-09-19T09:28:29.132 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-19T09:28:29.132 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:29.132 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:29.132 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:29.132 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:29.132 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.132 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_bytes": { 2024-09-19T09:28:29.132 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-19T09:28:29.133 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-09-19T09:28:29.133 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-19T09:28:29.133 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-19T09:28:29.133 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-19T09:28:29.133 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:29.133 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:29.133 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 4767, 2024-09-19T09:28:29.133 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 12209393, 2024-09-19T09:28:29.133 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 4767, 2024-09-19T09:28:29.133 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 12209393, 2024-09-19T09:28:29.133 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-19T09:28:29.133 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:29.134 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:29.134 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:29.134 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:29.134 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.134 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_ops": { 2024-09-19T09:28:29.134 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-19T09:28:29.134 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 1024, 2024-09-19T09:28:29.134 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-19T09:28:29.134 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-19T09:28:29.134 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-19T09:28:29.134 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:29.134 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:29.134 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 4767, 2024-09-19T09:28:29.134 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 4767, 2024-09-19T09:28:29.135 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 4767, 2024-09-19T09:28:29.135 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 4767, 2024-09-19T09:28:29.135 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-19T09:28:29.135 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:29.136 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:29.136 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:29.136 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:29.136 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.136 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:28:29.136 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-19T09:28:29.136 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-09-19T09:28:29.136 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-19T09:28:29.136 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 970, 2024-09-19T09:28:29.136 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 97970, 2024-09-19T09:28:29.136 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:29.136 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 970, 2024-09-19T09:28:29.136 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-19T09:28:29.136 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-19T09:28:29.137 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 159, 2024-09-19T09:28:29.137 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 97970, 2024-09-19T09:28:29.137 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-19T09:28:29.137 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:29.137 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:29.137 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:29.137 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:29.137 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:29.137 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle-0x55fa0a056960": { 2024-09-19T09:28:29.137 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 79467, 2024-09-19T09:28:29.137 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-09-19T09:28:29.137 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-19T09:28:29.137 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 8666, 2024-09-19T09:28:29.138 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 11883512, 2024-09-19T09:28:29.138 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:29.138 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 8666, 2024-09-19T09:28:29.138 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-19T09:28:29.138 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-19T09:28:29.138 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 2674, 2024-09-19T09:28:29.138 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 11804045, 2024-09-19T09:28:29.138 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-19T09:28:29.138 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:29.138 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:29.138 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:29.138 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:29.138 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:29.138 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-09-19T09:28:29.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:29 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:28:29.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:29 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:28:29.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:29 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:28:30.021 INFO:tasks.check_counter:Daemon mds.h mds.exported=1 2024-09-19T09:28:30.021 INFO:tasks.check_counter:Daemon mds.h mds.imported=1 2024-09-19T09:28:30.021 DEBUG:tasks.check_counter:Getting stats from k 2024-09-19T09:28:30.022 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-19T09:28:30.278 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T09:28:30.398 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:30 smithi153 ceph-mon[35071]: pgmap v1114: 97 pgs: 97 active+clean; 321 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 4.2 MiB/s wr, 117 op/s 2024-09-19T09:28:30.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:30 smithi196 ceph-mon[35150]: pgmap v1114: 97 pgs: 97 active+clean; 321 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 4.2 MiB/s wr, 117 op/s 2024-09-19T09:28:30.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:30 smithi120 ceph-mon[31150]: pgmap v1114: 97 pgs: 97 active+clean; 321 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 4.2 MiB/s wr, 117 op/s 2024-09-19T09:28:30.799 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-09-19T09:28:30.799 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:28:30.799 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 1880, 2024-09-19T09:28:30.799 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 1880, 2024-09-19T09:28:30.799 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 3901945, 2024-09-19T09:28:30.799 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 597080, 2024-09-19T09:28:30.799 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 7, 2024-09-19T09:28:30.799 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 2, 2024-09-19T09:28:30.800 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.147510455, 2024-09-19T09:28:30.800 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.054863953, 2024-09-19T09:28:30.800 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.038638662, 2024-09-19T09:28:30.800 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.040711054, 2024-09-19T09:28:30.800 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:30.800 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1879, 2024-09-19T09:28:30.800 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.070501178, 2024-09-19T09:28:30.800 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000037520 2024-09-19T09:28:30.800 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.800 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:30.800 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.800 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.800 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.801 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.801 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-19T09:28:30.801 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T09:28:30.801 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.801 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:28:30.801 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 1566, 2024-09-19T09:28:30.801 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 1909, 2024-09-19T09:28:30.801 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 385086, 2024-09-19T09:28:30.801 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 1268973, 2024-09-19T09:28:30.801 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 6, 2024-09-19T09:28:30.801 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 4, 2024-09-19T09:28:30.801 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.135571409, 2024-09-19T09:28:30.801 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.046308397, 2024-09-19T09:28:30.801 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.028208409, 2024-09-19T09:28:30.802 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.047224418, 2024-09-19T09:28:30.802 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:30.802 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1907, 2024-09-19T09:28:30.802 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.064240159, 2024-09-19T09:28:30.802 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000033686 2024-09-19T09:28:30.802 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.802 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:30.802 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.802 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.802 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.802 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.802 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T09:28:30.802 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 771616 2024-09-19T09:28:30.802 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.803 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:28:30.803 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 2127, 2024-09-19T09:28:30.803 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 2107, 2024-09-19T09:28:30.803 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 1494480, 2024-09-19T09:28:30.803 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 804310, 2024-09-19T09:28:30.803 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 3, 2024-09-19T09:28:30.803 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 2, 2024-09-19T09:28:30.803 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.201208831, 2024-09-19T09:28:30.803 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.070838047, 2024-09-19T09:28:30.803 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.075678666, 2024-09-19T09:28:30.803 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.049570353, 2024-09-19T09:28:30.803 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:30.803 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 2106, 2024-09-19T09:28:30.803 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.088411779, 2024-09-19T09:28:30.804 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000041980 2024-09-19T09:28:30.804 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.804 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:30.804 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.804 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.804 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.804 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.804 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 941760, 2024-09-19T09:28:30.804 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 281200 2024-09-19T09:28:30.804 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.804 INFO:teuthology.orchestra.run.smithi153.stdout: "cct": { 2024-09-19T09:28:30.804 INFO:teuthology.orchestra.run.smithi153.stdout: "total_workers": 1, 2024-09-19T09:28:30.804 INFO:teuthology.orchestra.run.smithi153.stdout: "unhealthy_workers": 0 2024-09-19T09:28:30.804 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.805 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-MDSRank": { 2024-09-19T09:28:30.805 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-09-19T09:28:30.805 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-09-19T09:28:30.805 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 8445, 2024-09-19T09:28:30.805 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.207527984, 2024-09-19T09:28:30.805 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000024574 2024-09-19T09:28:30.805 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:30.805 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.805 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-PurgeQueue": { 2024-09-19T09:28:30.805 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2024-09-19T09:28:30.805 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2024-09-19T09:28:30.805 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.805 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.805 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.806 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:30.806 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.806 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-09-19T09:28:30.806 INFO:teuthology.orchestra.run.smithi153.stdout: "request": 0, 2024-09-19T09:28:30.806 INFO:teuthology.orchestra.run.smithi153.stdout: "reply": 0, 2024-09-19T09:28:30.806 INFO:teuthology.orchestra.run.smithi153.stdout: "reply_latency": { 2024-09-19T09:28:30.806 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.806 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.806 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.806 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.806 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_reply": 0, 2024-09-19T09:28:30.806 INFO:teuthology.orchestra.run.smithi153.stdout: "forward": 0, 2024-09-19T09:28:30.806 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_complete": 0, 2024-09-19T09:28:30.806 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_keys": 0, 2024-09-19T09:28:30.807 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_commit": 0, 2024-09-19T09:28:30.807 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_split": 0, 2024-09-19T09:28:30.807 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_merge": 0, 2024-09-19T09:28:30.807 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes": 424, 2024-09-19T09:28:30.807 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_top": 0, 2024-09-19T09:28:30.807 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_bottom": 424, 2024-09-19T09:28:30.807 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pin_tail": 0, 2024-09-19T09:28:30.807 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pinned": 424, 2024-09-19T09:28:30.807 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_expired": 0, 2024-09-19T09:28:30.807 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_with_caps": 0, 2024-09-19T09:28:30.807 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": 0, 2024-09-19T09:28:30.807 INFO:teuthology.orchestra.run.smithi153.stdout: "subtrees": 1, 2024-09-19T09:28:30.807 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse": 0, 2024-09-19T09:28:30.807 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_hit": 0, 2024-09-19T09:28:30.808 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_forward": 0, 2024-09-19T09:28:30.808 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_discover": 0, 2024-09-19T09:28:30.808 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_dir_fetch": 0, 2024-09-19T09:28:30.808 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_remote_ino": 0, 2024-09-19T09:28:30.808 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_lock": 0, 2024-09-19T09:28:30.808 INFO:teuthology.orchestra.run.smithi153.stdout: "load_cent": 0, 2024-09-19T09:28:30.808 INFO:teuthology.orchestra.run.smithi153.stdout: "q": 0, 2024-09-19T09:28:30.808 INFO:teuthology.orchestra.run.smithi153.stdout: "exported": 0, 2024-09-19T09:28:30.808 INFO:teuthology.orchestra.run.smithi153.stdout: "exported_inodes": 0, 2024-09-19T09:28:30.808 INFO:teuthology.orchestra.run.smithi153.stdout: "imported": 0, 2024-09-19T09:28:30.808 INFO:teuthology.orchestra.run.smithi153.stdout: "imported_inodes": 0, 2024-09-19T09:28:30.808 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_dir_fetch": 0, 2024-09-19T09:28:30.808 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_backtrace_fetch": 0, 2024-09-19T09:28:30.809 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_peer_discover": 0, 2024-09-19T09:28:30.809 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rfiles": 0, 2024-09-19T09:28:30.809 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rbytes": 0, 2024-09-19T09:28:30.809 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rsnaps": 0, 2024-09-19T09:28:30.809 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T09:28:30.809 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_set_tag": 0, 2024-09-19T09:28:30.809 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:28:30.809 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:28:30.809 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_inodes": 0, 2024-09-19T09:28:30.809 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T09:28:30.809 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T09:28:30.809 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_file_inodes": 0, 2024-09-19T09:28:30.809 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_inode_file_caps": 0, 2024-09-19T09:28:30.809 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T09:28:30.810 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_grant": 0, 2024-09-19T09:28:30.810 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T09:28:30.810 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:28:30.810 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:28:30.810 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps": 0, 2024-09-19T09:28:30.810 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps_dirty": 0, 2024-09-19T09:28:30.810 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_cap_release": 0, 2024-09-19T09:28:30.810 INFO:teuthology.orchestra.run.smithi153.stdout: "process_request_cap_release": 0 2024-09-19T09:28:30.810 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.810 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_cache": { 2024-09-19T09:28:30.810 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update": 0, 2024-09-19T09:28:30.810 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update_receipt": 0, 2024-09-19T09:28:30.810 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_try_discover": 0, 2024-09-19T09:28:30.810 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_send_discover": 0, 2024-09-19T09:28:30.811 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_handle_discover": 0, 2024-09-19T09:28:30.811 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays": 0, 2024-09-19T09:28:30.811 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_delayed": 0, 2024-09-19T09:28:30.811 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:28:30.811 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_created": 0, 2024-09-19T09:28:30.811 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_enqueued": 0, 2024-09-19T09:28:30.811 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_reintegrated": 0, 2024-09-19T09:28:30.811 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_migrated": 0, 2024-09-19T09:28:30.811 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_processing": 0, 2024-09-19T09:28:30.811 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:28:30.811 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:28:30.811 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_started": 0, 2024-09-19T09:28:30.811 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_completed": 0, 2024-09-19T09:28:30.811 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_path": 0, 2024-09-19T09:28:30.812 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_quiesce_inode": 0, 2024-09-19T09:28:30.812 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T09:28:30.812 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_exportdir": 0, 2024-09-19T09:28:30.812 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_flush": 0, 2024-09-19T09:28:30.812 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragmentdir": 0, 2024-09-19T09:28:30.812 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragstats": 0, 2024-09-19T09:28:30.812 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_inodestats": 0 2024-09-19T09:28:30.812 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.812 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_log": { 2024-09-19T09:28:30.812 INFO:teuthology.orchestra.run.smithi153.stdout: "evlrg": 0, 2024-09-19T09:28:30.812 INFO:teuthology.orchestra.run.smithi153.stdout: "evadd": 0, 2024-09-19T09:28:30.812 INFO:teuthology.orchestra.run.smithi153.stdout: "evex": 0, 2024-09-19T09:28:30.812 INFO:teuthology.orchestra.run.smithi153.stdout: "evtrm": 0, 2024-09-19T09:28:30.812 INFO:teuthology.orchestra.run.smithi153.stdout: "ev": 2629, 2024-09-19T09:28:30.813 INFO:teuthology.orchestra.run.smithi153.stdout: "evexg": 0, 2024-09-19T09:28:30.813 INFO:teuthology.orchestra.run.smithi153.stdout: "evexd": 0, 2024-09-19T09:28:30.813 INFO:teuthology.orchestra.run.smithi153.stdout: "segadd": 0, 2024-09-19T09:28:30.813 INFO:teuthology.orchestra.run.smithi153.stdout: "segex": 0, 2024-09-19T09:28:30.813 INFO:teuthology.orchestra.run.smithi153.stdout: "segtrm": 0, 2024-09-19T09:28:30.813 INFO:teuthology.orchestra.run.smithi153.stdout: "seg": 4, 2024-09-19T09:28:30.813 INFO:teuthology.orchestra.run.smithi153.stdout: "segmjr": 2, 2024-09-19T09:28:30.813 INFO:teuthology.orchestra.run.smithi153.stdout: "segexg": 0, 2024-09-19T09:28:30.813 INFO:teuthology.orchestra.run.smithi153.stdout: "segexd": 0, 2024-09-19T09:28:30.813 INFO:teuthology.orchestra.run.smithi153.stdout: "expos": 4194304, 2024-09-19T09:28:30.813 INFO:teuthology.orchestra.run.smithi153.stdout: "wrpos": 7604722, 2024-09-19T09:28:30.813 INFO:teuthology.orchestra.run.smithi153.stdout: "rdpos": 7602650, 2024-09-19T09:28:30.814 INFO:teuthology.orchestra.run.smithi153.stdout: "jlat": { 2024-09-19T09:28:30.814 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.814 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.814 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.814 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.814 INFO:teuthology.orchestra.run.smithi153.stdout: "replayed": 2629 2024-09-19T09:28:30.814 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.814 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_mem": { 2024-09-19T09:28:30.814 INFO:teuthology.orchestra.run.smithi153.stdout: "ino": 15, 2024-09-19T09:28:30.814 INFO:teuthology.orchestra.run.smithi153.stdout: "ino+": 439, 2024-09-19T09:28:30.814 INFO:teuthology.orchestra.run.smithi153.stdout: "ino-": 424, 2024-09-19T09:28:30.814 INFO:teuthology.orchestra.run.smithi153.stdout: "dir": 4, 2024-09-19T09:28:30.814 INFO:teuthology.orchestra.run.smithi153.stdout: "dir+": 17, 2024-09-19T09:28:30.815 INFO:teuthology.orchestra.run.smithi153.stdout: "dir-": 13, 2024-09-19T09:28:30.815 INFO:teuthology.orchestra.run.smithi153.stdout: "dn": 424, 2024-09-19T09:28:30.815 INFO:teuthology.orchestra.run.smithi153.stdout: "dn+": 441, 2024-09-19T09:28:30.815 INFO:teuthology.orchestra.run.smithi153.stdout: "dn-": 17, 2024-09-19T09:28:30.815 INFO:teuthology.orchestra.run.smithi153.stdout: "cap": 0, 2024-09-19T09:28:30.815 INFO:teuthology.orchestra.run.smithi153.stdout: "cap+": 0, 2024-09-19T09:28:30.815 INFO:teuthology.orchestra.run.smithi153.stdout: "cap-": 0, 2024-09-19T09:28:30.815 INFO:teuthology.orchestra.run.smithi153.stdout: "rss": 50624, 2024-09-19T09:28:30.815 INFO:teuthology.orchestra.run.smithi153.stdout: "heap": 215328 2024-09-19T09:28:30.815 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.815 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_server": { 2024-09-19T09:28:30.815 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_client_request": 0, 2024-09-19T09:28:30.815 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_server_request": 0, 2024-09-19T09:28:30.815 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_request": 0, 2024-09-19T09:28:30.815 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_session": 0, 2024-09-19T09:28:30.816 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_peer_request": 0, 2024-09-19T09:28:30.816 INFO:teuthology.orchestra.run.smithi153.stdout: "req_create_latency": { 2024-09-19T09:28:30.816 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.816 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.816 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.816 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.816 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getattr_latency": { 2024-09-19T09:28:30.816 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.816 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.816 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.816 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.816 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getfilelock_latency": { 2024-09-19T09:28:30.816 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.816 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.817 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.817 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.817 INFO:teuthology.orchestra.run.smithi153.stdout: "req_link_latency": { 2024-09-19T09:28:30.817 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.817 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.817 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.817 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.817 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookup_latency": { 2024-09-19T09:28:30.817 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.817 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.817 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.817 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.817 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookuphash_latency": { 2024-09-19T09:28:30.817 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.818 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.818 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.818 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.818 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupino_latency": { 2024-09-19T09:28:30.818 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.818 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.818 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.818 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.818 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupname_latency": { 2024-09-19T09:28:30.818 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.818 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.818 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.818 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.818 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupparent_latency": { 2024-09-19T09:28:30.818 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.819 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.819 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.819 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.819 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupsnap_latency": { 2024-09-19T09:28:30.819 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.819 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.819 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.819 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.819 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lssnap_latency": { 2024-09-19T09:28:30.819 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.819 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.819 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.819 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.819 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mkdir_latency": { 2024-09-19T09:28:30.820 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.820 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.820 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.820 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.820 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mknod_latency": { 2024-09-19T09:28:30.820 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.820 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.821 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.821 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.821 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mksnap_latency": { 2024-09-19T09:28:30.821 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.821 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.821 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.821 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.821 INFO:teuthology.orchestra.run.smithi153.stdout: "req_open_latency": { 2024-09-19T09:28:30.821 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.821 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.821 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.821 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.821 INFO:teuthology.orchestra.run.smithi153.stdout: "req_readdir_latency": { 2024-09-19T09:28:30.821 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.822 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.822 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.822 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.822 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rename_latency": { 2024-09-19T09:28:30.822 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.822 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.822 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.822 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.822 INFO:teuthology.orchestra.run.smithi153.stdout: "req_renamesnap_latency": { 2024-09-19T09:28:30.822 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.822 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.822 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.822 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.822 INFO:teuthology.orchestra.run.smithi153.stdout: "req_snapdiff_latency": { 2024-09-19T09:28:30.823 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.823 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.823 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.823 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.823 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmdir_latency": { 2024-09-19T09:28:30.823 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.823 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.823 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.823 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.823 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmsnap_latency": { 2024-09-19T09:28:30.823 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.823 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.823 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.823 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.824 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmxattr_latency": { 2024-09-19T09:28:30.824 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.824 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.824 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.824 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.824 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setattr_latency": { 2024-09-19T09:28:30.824 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.824 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.824 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.824 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.824 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setdirlayout_latency": { 2024-09-19T09:28:30.824 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.824 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.824 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.825 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.825 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setfilelock_latency": { 2024-09-19T09:28:30.825 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.825 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.825 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.825 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.825 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setlayout_latency": { 2024-09-19T09:28:30.825 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.825 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.825 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.825 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.825 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setxattr_latency": { 2024-09-19T09:28:30.825 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.825 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.826 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.826 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.826 INFO:teuthology.orchestra.run.smithi153.stdout: "req_symlink_latency": { 2024-09-19T09:28:30.826 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.826 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.826 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.826 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.826 INFO:teuthology.orchestra.run.smithi153.stdout: "req_unlink_latency": { 2024-09-19T09:28:30.826 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.826 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.826 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.826 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.826 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:28:30.826 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:28:30.827 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getvxattr_latency": { 2024-09-19T09:28:30.827 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.827 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.827 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.827 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:30.827 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.827 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_sessions": { 2024-09-19T09:28:30.827 INFO:teuthology.orchestra.run.smithi153.stdout: "session_count": 1, 2024-09-19T09:28:30.827 INFO:teuthology.orchestra.run.smithi153.stdout: "session_add": 2, 2024-09-19T09:28:30.827 INFO:teuthology.orchestra.run.smithi153.stdout: "session_remove": 1, 2024-09-19T09:28:30.827 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_open": 1, 2024-09-19T09:28:30.827 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_stale": 0, 2024-09-19T09:28:30.827 INFO:teuthology.orchestra.run.smithi153.stdout: "total_load": 0, 2024-09-19T09:28:30.827 INFO:teuthology.orchestra.run.smithi153.stdout: "average_load": 0, 2024-09-19T09:28:30.827 INFO:teuthology.orchestra.run.smithi153.stdout: "avg_session_uptime": 2836, 2024-09-19T09:28:30.828 INFO:teuthology.orchestra.run.smithi153.stdout: "mdthresh_evicted": 0 2024-09-19T09:28:30.828 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.828 INFO:teuthology.orchestra.run.smithi153.stdout: "mempool": { 2024-09-19T09:28:30.828 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:28:30.828 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_items": 0, 2024-09-19T09:28:30.828 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:28:30.828 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:28:30.828 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:28:30.828 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:28:30.828 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:28:30.828 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:28:30.828 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:28:30.828 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:28:30.828 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:28:30.829 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:28:30.829 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:28:30.829 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:28:30.829 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:28:30.829 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_items": 0, 2024-09-19T09:28:30.829 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:28:30.829 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_items": 0, 2024-09-19T09:28:30.829 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:28:30.829 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:28:30.829 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:28:30.829 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:28:30.829 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:28:30.829 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:28:30.830 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:28:30.830 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_items": 0, 2024-09-19T09:28:30.830 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:28:30.830 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:28:30.830 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:28:30.830 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_items": 0, 2024-09-19T09:28:30.830 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_bytes": 0, 2024-09-19T09:28:30.830 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_items": 0, 2024-09-19T09:28:30.830 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:28:30.830 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:28:30.830 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:28:30.830 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:28:30.830 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_bytes": 190500, 2024-09-19T09:28:30.830 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_items": 61, 2024-09-19T09:28:30.830 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:28:30.831 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_items": 0, 2024-09-19T09:28:30.831 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_bytes": 0, 2024-09-19T09:28:30.831 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_items": 0, 2024-09-19T09:28:30.831 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:28:30.831 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_items": 0, 2024-09-19T09:28:30.831 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:28:30.831 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_items": 0, 2024-09-19T09:28:30.831 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_bytes": 7592, 2024-09-19T09:28:30.831 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_items": 156, 2024-09-19T09:28:30.831 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:28:30.831 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:28:30.831 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_bytes": 0, 2024-09-19T09:28:30.831 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_items": 0, 2024-09-19T09:28:30.831 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_bytes": 339059, 2024-09-19T09:28:30.832 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_items": 2924, 2024-09-19T09:28:30.832 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_bytes": 0, 2024-09-19T09:28:30.832 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_items": 0, 2024-09-19T09:28:30.832 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_bytes": 0, 2024-09-19T09:28:30.832 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_items": 0 2024-09-19T09:28:30.832 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.832 INFO:teuthology.orchestra.run.smithi153.stdout: "objecter": { 2024-09-19T09:28:30.832 INFO:teuthology.orchestra.run.smithi153.stdout: "op_active": 0, 2024-09-19T09:28:30.832 INFO:teuthology.orchestra.run.smithi153.stdout: "op_laggy": 0, 2024-09-19T09:28:30.832 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send": 5085, 2024-09-19T09:28:30.832 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send_bytes": 21, 2024-09-19T09:28:30.832 INFO:teuthology.orchestra.run.smithi153.stdout: "op_resend": 3, 2024-09-19T09:28:30.833 INFO:teuthology.orchestra.run.smithi153.stdout: "op_reply": 5085, 2024-09-19T09:28:30.833 INFO:teuthology.orchestra.run.smithi153.stdout: "op_latency": { 2024-09-19T09:28:30.833 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 5085, 2024-09-19T09:28:30.833 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 8.881840680, 2024-09-19T09:28:30.833 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.001746674 2024-09-19T09:28:30.833 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.833 INFO:teuthology.orchestra.run.smithi153.stdout: "op_inflight": 0, 2024-09-19T09:28:30.833 INFO:teuthology.orchestra.run.smithi153.stdout: "oplen_avg": { 2024-09-19T09:28:30.833 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 5085, 2024-09-19T09:28:30.833 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 5086 2024-09-19T09:28:30.833 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.833 INFO:teuthology.orchestra.run.smithi153.stdout: "op": 5085, 2024-09-19T09:28:30.833 INFO:teuthology.orchestra.run.smithi153.stdout: "op_r": 5085, 2024-09-19T09:28:30.833 INFO:teuthology.orchestra.run.smithi153.stdout: "op_w": 0, 2024-09-19T09:28:30.834 INFO:teuthology.orchestra.run.smithi153.stdout: "op_rmw": 0, 2024-09-19T09:28:30.834 INFO:teuthology.orchestra.run.smithi153.stdout: "op_pg": 0, 2024-09-19T09:28:30.834 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_stat": 3207, 2024-09-19T09:28:30.834 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_create": 0, 2024-09-19T09:28:30.834 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_read": 1876, 2024-09-19T09:28:30.834 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_write": 0, 2024-09-19T09:28:30.834 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writefull": 0, 2024-09-19T09:28:30.834 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writesame": 0, 2024-09-19T09:28:30.834 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_append": 0, 2024-09-19T09:28:30.834 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_zero": 0, 2024-09-19T09:28:30.834 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_truncate": 0, 2024-09-19T09:28:30.834 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_delete": 0, 2024-09-19T09:28:30.834 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_mapext": 0, 2024-09-19T09:28:30.834 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_sparse_read": 0, 2024-09-19T09:28:30.835 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_clonerange": 0, 2024-09-19T09:28:30.835 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_getxattr": 1, 2024-09-19T09:28:30.835 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_setxattr": 0, 2024-09-19T09:28:30.835 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:28:30.835 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_rmxattr": 0, 2024-09-19T09:28:30.835 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:28:30.835 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_call": 0, 2024-09-19T09:28:30.835 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_watch": 0, 2024-09-19T09:28:30.835 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_notify": 0, 2024-09-19T09:28:30.835 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:28:30.835 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls": 0, 2024-09-19T09:28:30.835 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:28:30.835 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_other": 0, 2024-09-19T09:28:30.835 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_active": 0, 2024-09-19T09:28:30.836 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_send": 0, 2024-09-19T09:28:30.836 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_resend": 0, 2024-09-19T09:28:30.836 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_ping": 0, 2024-09-19T09:28:30.836 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_active": 0, 2024-09-19T09:28:30.836 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_send": 0, 2024-09-19T09:28:30.836 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_resend": 0, 2024-09-19T09:28:30.836 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_active": 0, 2024-09-19T09:28:30.836 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_send": 0, 2024-09-19T09:28:30.836 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_resend": 0, 2024-09-19T09:28:30.836 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_active": 0, 2024-09-19T09:28:30.836 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_send": 0, 2024-09-19T09:28:30.836 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_resend": 0, 2024-09-19T09:28:30.836 INFO:teuthology.orchestra.run.smithi153.stdout: "command_active": 0, 2024-09-19T09:28:30.836 INFO:teuthology.orchestra.run.smithi153.stdout: "command_send": 0, 2024-09-19T09:28:30.837 INFO:teuthology.orchestra.run.smithi153.stdout: "command_resend": 0, 2024-09-19T09:28:30.837 INFO:teuthology.orchestra.run.smithi153.stdout: "map_epoch": 85, 2024-09-19T09:28:30.837 INFO:teuthology.orchestra.run.smithi153.stdout: "map_full": 0, 2024-09-19T09:28:30.837 INFO:teuthology.orchestra.run.smithi153.stdout: "map_inc": 6, 2024-09-19T09:28:30.837 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_sessions": 5, 2024-09-19T09:28:30.837 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_open": 6, 2024-09-19T09:28:30.837 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_close": 1, 2024-09-19T09:28:30.837 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_laggy": 0, 2024-09-19T09:28:30.837 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_wr": 0, 2024-09-19T09:28:30.837 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_rd": 2, 2024-09-19T09:28:30.837 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_del": 0 2024-09-19T09:28:30.837 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.837 INFO:teuthology.orchestra.run.smithi153.stdout: "oft": { 2024-09-19T09:28:30.837 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_objs": 0, 2024-09-19T09:28:30.838 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:28:30.838 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_updates": 0, 2024-09-19T09:28:30.838 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_removes": 0 2024-09-19T09:28:30.838 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.838 INFO:teuthology.orchestra.run.smithi153.stdout: "osd-slow-ops": { 2024-09-19T09:28:30.838 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_ops_count": 0 2024-09-19T09:28:30.838 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.838 INFO:teuthology.orchestra.run.smithi153.stdout: "purge_queue": { 2024-09-19T09:28:30.838 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops": 0, 2024-09-19T09:28:30.838 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T09:28:30.838 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing": 0, 2024-09-19T09:28:30.838 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_high_water": 0, 2024-09-19T09:28:30.838 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed_ops": 0, 2024-09-19T09:28:30.838 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed": 0, 2024-09-19T09:28:30.839 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_item_in_journal": 0 2024-09-19T09:28:30.839 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.839 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:28:30.839 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-19T09:28:30.839 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-09-19T09:28:30.839 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-19T09:28:30.839 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 5569, 2024-09-19T09:28:30.839 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 5252529, 2024-09-19T09:28:30.839 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:30.839 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 5569, 2024-09-19T09:28:30.839 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-19T09:28:30.839 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-19T09:28:30.839 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 5569, 2024-09-19T09:28:30.839 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 5252529, 2024-09-19T09:28:30.840 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-19T09:28:30.840 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.840 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.840 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.840 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:30.840 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.840 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_bytes": { 2024-09-19T09:28:30.840 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-19T09:28:30.840 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2024-09-19T09:28:30.840 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-19T09:28:30.840 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-19T09:28:30.840 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-19T09:28:30.840 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:30.840 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:30.841 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 5085, 2024-09-19T09:28:30.841 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 3410423, 2024-09-19T09:28:30.841 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 5085, 2024-09-19T09:28:30.841 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 3410423, 2024-09-19T09:28:30.841 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-19T09:28:30.841 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.844 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.844 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.844 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:30.844 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.844 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_ops": { 2024-09-19T09:28:30.844 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-19T09:28:30.844 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 1024, 2024-09-19T09:28:30.844 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-19T09:28:30.844 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-19T09:28:30.844 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-19T09:28:30.844 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:30.845 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:30.845 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 5085, 2024-09-19T09:28:30.845 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 5085, 2024-09-19T09:28:30.845 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 5085, 2024-09-19T09:28:30.845 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 5085, 2024-09-19T09:28:30.845 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-19T09:28:30.845 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.845 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.845 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.845 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:30.845 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.845 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:28:30.845 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-19T09:28:30.845 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-09-19T09:28:30.846 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-19T09:28:30.846 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-19T09:28:30.846 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-19T09:28:30.846 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:30.846 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:30.846 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-19T09:28:30.846 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-19T09:28:30.846 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 0, 2024-09-19T09:28:30.846 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 0, 2024-09-19T09:28:30.846 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-19T09:28:30.846 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.846 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.846 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.846 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:30.846 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-09-19T09:28:30.847 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle-0x559090a54b40": { 2024-09-19T09:28:30.847 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2024-09-19T09:28:30.847 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2024-09-19T09:28:30.847 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2024-09-19T09:28:30.847 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2024-09-19T09:28:30.847 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2024-09-19T09:28:30.847 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:30.847 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:30.847 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2024-09-19T09:28:30.847 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2024-09-19T09:28:30.847 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 0, 2024-09-19T09:28:30.847 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 0, 2024-09-19T09:28:30.847 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2024-09-19T09:28:30.847 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2024-09-19T09:28:30.848 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2024-09-19T09:28:30.848 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2024-09-19T09:28:30.848 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:30.848 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-09-19T09:28:30.848 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-09-19T09:28:31.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:31 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:31.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:31 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:28:31.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:31 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:28:31.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:31 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:31.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:31 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:28:31.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:31 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:28:31.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:31 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:31.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:31 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:28:31.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:31 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' 2024-09-19T09:28:31.729 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-19T09:28:31.729 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-19T09:28:31.729 DEBUG:tasks.check_counter:Getting stats from c 2024-09-19T09:28:31.730 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-19T09:28:32.056 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:28:32.388 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:32 smithi196 ceph-mon[35150]: pgmap v1115: 97 pgs: 97 active+clean; 298 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 5.2 MiB/s wr, 181 op/s 2024-09-19T09:28:32.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:32 smithi153 ceph-mon[35071]: pgmap v1115: 97 pgs: 97 active+clean; 298 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 5.2 MiB/s wr, 181 op/s 2024-09-19T09:28:32.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:32 smithi120 ceph-mon[31150]: pgmap v1115: 97 pgs: 97 active+clean; 298 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 5.2 MiB/s wr, 181 op/s 2024-09-19T09:28:32.782 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-19T09:28:32.782 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:28:32.782 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 61110, 2024-09-19T09:28:32.782 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 82561, 2024-09-19T09:28:32.783 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 16400664, 2024-09-19T09:28:32.783 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 190815457, 2024-09-19T09:28:32.783 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 56, 2024-09-19T09:28:32.783 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 5, 2024-09-19T09:28:32.783 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 5.615955140, 2024-09-19T09:28:32.783 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 2.536524314, 2024-09-19T09:28:32.783 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 1.970535014, 2024-09-19T09:28:32.783 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 1.056022385, 2024-09-19T09:28:32.783 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:32.783 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 82560, 2024-09-19T09:28:32.783 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 2.065482067, 2024-09-19T09:28:32.783 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000025017 2024-09-19T09:28:32.783 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.784 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:32.784 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 64162, 2024-09-19T09:28:32.784 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.014511636, 2024-09-19T09:28:32.784 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000226 2024-09-19T09:28:32.784 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.784 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 6400, 2024-09-19T09:28:32.784 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 4976 2024-09-19T09:28:32.784 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.784 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:28:32.784 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 316311, 2024-09-19T09:28:32.784 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 476647, 2024-09-19T09:28:32.784 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 106598497, 2024-09-19T09:28:32.784 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 457241067, 2024-09-19T09:28:32.785 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 10, 2024-09-19T09:28:32.785 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 7, 2024-09-19T09:28:32.785 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 21.738817540, 2024-09-19T09:28:32.785 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 9.991898931, 2024-09-19T09:28:32.785 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 13.927042818, 2024-09-19T09:28:32.785 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.684081627, 2024-09-19T09:28:32.785 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:32.785 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 476645, 2024-09-19T09:28:32.785 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 12.339686930, 2024-09-19T09:28:32.785 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000025888 2024-09-19T09:28:32.785 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.785 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:32.786 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 714558, 2024-09-19T09:28:32.786 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.105558272, 2024-09-19T09:28:32.786 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000147 2024-09-19T09:28:32.786 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.786 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 97423120, 2024-09-19T09:28:32.786 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 366662704 2024-09-19T09:28:32.786 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.786 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:28:32.786 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 225564, 2024-09-19T09:28:32.786 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 186873, 2024-09-19T09:28:32.786 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 56636348, 2024-09-19T09:28:32.786 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 106780090, 2024-09-19T09:28:32.786 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 43, 2024-09-19T09:28:32.787 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 7, 2024-09-19T09:28:32.787 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 11.676913806, 2024-09-19T09:28:32.787 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 4.352600477, 2024-09-19T09:28:32.787 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 8.731897090, 2024-09-19T09:28:32.787 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.500009048, 2024-09-19T09:28:32.787 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:32.787 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 186872, 2024-09-19T09:28:32.787 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 4.090744348, 2024-09-19T09:28:32.787 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000021890 2024-09-19T09:28:32.787 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.787 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:32.787 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 370581, 2024-09-19T09:28:32.787 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.060240039, 2024-09-19T09:28:32.788 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000162 2024-09-19T09:28:32.788 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.788 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 964288, 2024-09-19T09:28:32.788 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 295904 2024-09-19T09:28:32.788 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.788 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-09-19T09:28:32.788 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-09-19T09:28:32.788 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-09-19T09:28:32.788 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.788 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-09-19T09:28:32.788 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-19T09:28:32.788 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-19T09:28:32.788 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 94147, 2024-09-19T09:28:32.789 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 74.666196808, 2024-09-19T09:28:32.789 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000793080 2024-09-19T09:28:32.789 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:32.789 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.789 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-09-19T09:28:32.789 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-19T09:28:32.789 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-19T09:28:32.789 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 56945, 2024-09-19T09:28:32.789 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 3.899026983, 2024-09-19T09:28:32.789 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000068470 2024-09-19T09:28:32.789 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:32.789 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.789 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-09-19T09:28:32.789 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 204234, 2024-09-19T09:28:32.790 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 182604, 2024-09-19T09:28:32.790 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-09-19T09:28:32.790 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 182604, 2024-09-19T09:28:32.790 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 538.091866712, 2024-09-19T09:28:32.790 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.002946769 2024-09-19T09:28:32.790 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.790 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-09-19T09:28:32.790 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 21630, 2024-09-19T09:28:32.790 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-09-19T09:28:32.790 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-09-19T09:28:32.790 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 128, 2024-09-19T09:28:32.790 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 13, 2024-09-19T09:28:32.791 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 13, 2024-09-19T09:28:32.791 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 1765, 2024-09-19T09:28:32.791 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 968, 2024-09-19T09:28:32.791 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 430, 2024-09-19T09:28:32.791 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 367, 2024-09-19T09:28:32.791 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 380, 2024-09-19T09:28:32.791 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 72, 2024-09-19T09:28:32.791 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 0, 2024-09-19T09:28:32.791 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 0, 2024-09-19T09:28:32.791 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 3, 2024-09-19T09:28:32.791 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 296254, 2024-09-19T09:28:32.791 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 157752, 2024-09-19T09:28:32.791 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 21628, 2024-09-19T09:28:32.792 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 16, 2024-09-19T09:28:32.792 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-09-19T09:28:32.792 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-09-19T09:28:32.792 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 2206, 2024-09-19T09:28:32.792 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-09-19T09:28:32.792 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-09-19T09:28:32.792 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 1, 2024-09-19T09:28:32.792 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 1, 2024-09-19T09:28:32.792 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 1, 2024-09-19T09:28:32.792 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 103, 2024-09-19T09:28:32.792 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-09-19T09:28:32.792 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 1105, 2024-09-19T09:28:32.792 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 1, 2024-09-19T09:28:32.792 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 114, 2024-09-19T09:28:32.793 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 34500714, 2024-09-19T09:28:32.793 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 0, 2024-09-19T09:28:32.793 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 1105, 2024-09-19T09:28:32.793 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-09-19T09:28:32.793 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:28:32.793 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:28:32.793 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 140, 2024-09-19T09:28:32.793 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T09:28:32.793 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 140, 2024-09-19T09:28:32.793 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 965, 2024-09-19T09:28:32.793 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 2, 2024-09-19T09:28:32.793 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 5275, 2024-09-19T09:28:32.793 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 112572, 2024-09-19T09:28:32.794 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T09:28:32.794 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:28:32.794 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:28:32.794 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 52257, 2024-09-19T09:28:32.794 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 23083, 2024-09-19T09:28:32.794 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 11603, 2024-09-19T09:28:32.794 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 113598 2024-09-19T09:28:32.794 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.794 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-09-19T09:28:32.794 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 12, 2024-09-19T09:28:32.794 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 49, 2024-09-19T09:28:32.794 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 31, 2024-09-19T09:28:32.794 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 41, 2024-09-19T09:28:32.794 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 27, 2024-09-19T09:28:32.795 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-09-19T09:28:32.795 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-09-19T09:28:32.795 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:28:32.795 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 29720, 2024-09-19T09:28:32.795 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 29720, 2024-09-19T09:28:32.795 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-09-19T09:28:32.795 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-09-19T09:28:32.795 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-09-19T09:28:32.795 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:28:32.795 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:28:32.795 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-09-19T09:28:32.795 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-09-19T09:28:32.795 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 22, 2024-09-19T09:28:32.796 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 1292, 2024-09-19T09:28:32.796 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T09:28:32.796 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 1, 2024-09-19T09:28:32.796 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-09-19T09:28:32.796 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 26, 2024-09-19T09:28:32.796 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-09-19T09:28:32.796 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-09-19T09:28:32.796 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.796 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-09-19T09:28:32.796 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-09-19T09:28:32.796 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 139566, 2024-09-19T09:28:32.796 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 29714, 2024-09-19T09:28:32.796 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 24581, 2024-09-19T09:28:32.797 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 114985, 2024-09-19T09:28:32.797 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-09-19T09:28:32.797 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 5133, 2024-09-19T09:28:32.797 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 163, 2024-09-19T09:28:32.797 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 35, 2024-09-19T09:28:32.797 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 29, 2024-09-19T09:28:32.797 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 134, 2024-09-19T09:28:32.797 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 10, 2024-09-19T09:28:32.797 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-09-19T09:28:32.797 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 6, 2024-09-19T09:28:32.797 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 57515232, 2024-09-19T09:28:32.797 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 298634830, 2024-09-19T09:28:32.797 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 0, 2024-09-19T09:28:32.798 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-09-19T09:28:32.798 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 47954, 2024-09-19T09:28:32.798 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 226.932792270, 2024-09-19T09:28:32.798 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.004732301 2024-09-19T09:28:32.798 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.798 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 0 2024-09-19T09:28:32.798 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.798 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-09-19T09:28:32.798 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 322, 2024-09-19T09:28:32.798 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 29775, 2024-09-19T09:28:32.798 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 29453, 2024-09-19T09:28:32.798 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 26, 2024-09-19T09:28:32.798 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 194, 2024-09-19T09:28:32.799 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 168, 2024-09-19T09:28:32.799 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 1712, 2024-09-19T09:28:32.799 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 62365, 2024-09-19T09:28:32.799 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 60653, 2024-09-19T09:28:32.799 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 0, 2024-09-19T09:28:32.799 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 33528, 2024-09-19T09:28:32.799 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 33528, 2024-09-19T09:28:32.799 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 81088, 2024-09-19T09:28:32.799 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 223520 2024-09-19T09:28:32.799 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.799 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-09-19T09:28:32.799 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 257144, 2024-09-19T09:28:32.799 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 2, 2024-09-19T09:28:32.800 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 204234, 2024-09-19T09:28:32.800 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 4209, 2024-09-19T09:28:32.800 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 24, 2024-09-19T09:28:32.800 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-09-19T09:28:32.800 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 29645, 2024-09-19T09:28:32.800 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 118.081080338, 2024-09-19T09:28:32.800 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.003983170 2024-09-19T09:28:32.800 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.800 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-09-19T09:28:32.800 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 56, 2024-09-19T09:28:32.800 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.268577231, 2024-09-19T09:28:32.800 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.004796021 2024-09-19T09:28:32.800 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.800 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-09-19T09:28:32.801 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:32.801 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:32.801 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:32.801 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.801 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-09-19T09:28:32.801 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:32.801 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:32.801 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:32.801 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.801 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-09-19T09:28:32.801 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 68765, 2024-09-19T09:28:32.801 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 29.511879432, 2024-09-19T09:28:32.801 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000429170 2024-09-19T09:28:32.802 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.802 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-09-19T09:28:32.802 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:32.802 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:32.802 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:32.802 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.802 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-09-19T09:28:32.802 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:32.802 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:32.802 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:32.802 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.802 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-09-19T09:28:32.802 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:32.802 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:32.803 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:32.803 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.803 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-09-19T09:28:32.803 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:32.803 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:32.803 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:32.803 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.803 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-09-19T09:28:32.803 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:32.804 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:32.804 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:32.804 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.804 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-09-19T09:28:32.804 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:32.804 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:32.804 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:32.804 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.804 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-09-19T09:28:32.804 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:32.805 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:32.805 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:32.805 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.805 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-09-19T09:28:32.805 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:32.805 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:32.805 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:32.805 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.805 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-09-19T09:28:32.805 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:32.805 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:32.805 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:32.805 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.806 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-09-19T09:28:32.806 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:32.806 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:32.806 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:32.806 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.806 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-09-19T09:28:32.806 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 47242, 2024-09-19T09:28:32.806 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 189.295351552, 2024-09-19T09:28:32.806 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.004006929 2024-09-19T09:28:32.806 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.806 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-09-19T09:28:32.806 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 6202, 2024-09-19T09:28:32.806 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 32.114969704, 2024-09-19T09:28:32.806 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.005178163 2024-09-19T09:28:32.807 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.807 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-09-19T09:28:32.807 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:32.807 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:32.807 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:32.807 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.807 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-09-19T09:28:32.807 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:32.807 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:32.807 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:32.807 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.807 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-09-19T09:28:32.807 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 10, 2024-09-19T09:28:32.807 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.063330864, 2024-09-19T09:28:32.808 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.006333086 2024-09-19T09:28:32.808 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.808 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-09-19T09:28:32.808 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:32.808 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:32.808 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:32.808 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.808 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-09-19T09:28:32.808 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:32.808 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:32.808 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:32.808 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.809 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-09-19T09:28:32.809 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 18, 2024-09-19T09:28:32.809 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 79.060820673, 2024-09-19T09:28:32.809 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 4.392267815 2024-09-19T09:28:32.809 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.809 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-09-19T09:28:32.809 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:32.809 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:32.809 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:32.809 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.809 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-09-19T09:28:32.809 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 956, 2024-09-19T09:28:32.809 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.560872036, 2024-09-19T09:28:32.810 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000586686 2024-09-19T09:28:32.810 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.810 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-09-19T09:28:32.810 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:32.810 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:32.810 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:32.810 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.810 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-09-19T09:28:32.810 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:32.810 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:32.810 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:32.810 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.810 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-09-19T09:28:32.810 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:32.811 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:32.811 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:32.811 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.811 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-09-19T09:28:32.811 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 29710, 2024-09-19T09:28:32.811 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 89.134984882, 2024-09-19T09:28:32.811 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.003000167 2024-09-19T09:28:32.811 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.811 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:28:32.811 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:28:32.811 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-09-19T09:28:32.811 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:32.811 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:32.812 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:32.812 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:32.812 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.812 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-09-19T09:28:32.812 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 1, 2024-09-19T09:28:32.812 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 2, 2024-09-19T09:28:32.812 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 1, 2024-09-19T09:28:32.812 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 1, 2024-09-19T09:28:32.812 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-09-19T09:28:32.812 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 17750, 2024-09-19T09:28:32.812 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 17750, 2024-09-19T09:28:32.812 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 1524, 2024-09-19T09:28:32.812 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-09-19T09:28:32.813 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.813 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-09-19T09:28:32.813 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:28:32.813 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-09-19T09:28:32.813 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:28:32.813 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:28:32.813 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:28:32.813 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:28:32.813 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:28:32.813 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:28:32.813 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:28:32.813 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:28:32.813 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:28:32.813 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:28:32.814 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:28:32.814 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:28:32.814 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:28:32.814 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-09-19T09:28:32.814 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:28:32.814 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-09-19T09:28:32.814 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:28:32.814 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:28:32.814 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:28:32.814 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:28:32.814 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:28:32.814 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:28:32.814 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:28:32.815 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-09-19T09:28:32.815 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:28:32.815 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:28:32.815 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:28:32.815 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-09-19T09:28:32.815 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-09-19T09:28:32.815 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-09-19T09:28:32.815 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:28:32.815 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:28:32.815 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:28:32.815 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:28:32.815 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 4389054, 2024-09-19T09:28:32.815 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 1478, 2024-09-19T09:28:32.816 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:28:32.816 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-09-19T09:28:32.816 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-09-19T09:28:32.816 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-09-19T09:28:32.816 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:28:32.816 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-09-19T09:28:32.816 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:28:32.816 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-09-19T09:28:32.816 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 7592, 2024-09-19T09:28:32.816 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 156, 2024-09-19T09:28:32.816 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:28:32.816 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:28:32.817 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-09-19T09:28:32.817 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-09-19T09:28:32.817 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 2964468, 2024-09-19T09:28:32.817 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 103229, 2024-09-19T09:28:32.817 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-09-19T09:28:32.817 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-09-19T09:28:32.817 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-09-19T09:28:32.817 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-09-19T09:28:32.817 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.817 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-09-19T09:28:32.817 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-09-19T09:28:32.817 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-09-19T09:28:32.817 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 81990, 2024-09-19T09:28:32.818 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 301144624, 2024-09-19T09:28:32.818 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 33, 2024-09-19T09:28:32.818 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 81990, 2024-09-19T09:28:32.818 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-09-19T09:28:32.818 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 81990, 2024-09-19T09:28:32.818 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 405.876137620, 2024-09-19T09:28:32.818 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.004950312 2024-09-19T09:28:32.818 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.818 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-09-19T09:28:32.818 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-09-19T09:28:32.818 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 81990, 2024-09-19T09:28:32.818 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 83144 2024-09-19T09:28:32.818 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.818 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 81990, 2024-09-19T09:28:32.819 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 2018, 2024-09-19T09:28:32.819 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 79972, 2024-09-19T09:28:32.819 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-09-19T09:28:32.819 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-09-19T09:28:32.819 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 0, 2024-09-19T09:28:32.819 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 21, 2024-09-19T09:28:32.819 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 913, 2024-09-19T09:28:32.819 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 48867, 2024-09-19T09:28:32.819 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 480, 2024-09-19T09:28:32.819 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-09-19T09:28:32.819 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-09-19T09:28:32.819 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-09-19T09:28:32.819 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 13, 2024-09-19T09:28:32.820 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 29918, 2024-09-19T09:28:32.820 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-09-19T09:28:32.820 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-09-19T09:28:32.820 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-09-19T09:28:32.820 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 1105, 2024-09-19T09:28:32.820 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 42, 2024-09-19T09:28:32.820 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:28:32.820 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-09-19T09:28:32.820 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:28:32.820 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-09-19T09:28:32.820 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-09-19T09:28:32.820 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-09-19T09:28:32.820 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:28:32.820 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-09-19T09:28:32.821 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:28:32.821 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-09-19T09:28:32.821 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-09-19T09:28:32.821 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-09-19T09:28:32.821 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-09-19T09:28:32.821 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-09-19T09:28:32.821 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-09-19T09:28:32.821 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-09-19T09:28:32.821 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-09-19T09:28:32.821 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-09-19T09:28:32.821 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-09-19T09:28:32.821 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-09-19T09:28:32.821 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-09-19T09:28:32.822 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-09-19T09:28:32.822 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-09-19T09:28:32.822 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-09-19T09:28:32.822 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-09-19T09:28:32.822 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-09-19T09:28:32.822 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 85, 2024-09-19T09:28:32.822 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-09-19T09:28:32.822 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 6, 2024-09-19T09:28:32.822 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 12, 2024-09-19T09:28:32.822 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 16, 2024-09-19T09:28:32.822 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 4, 2024-09-19T09:28:32.822 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-09-19T09:28:32.822 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 1278, 2024-09-19T09:28:32.823 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 0, 2024-09-19T09:28:32.823 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 507 2024-09-19T09:28:32.823 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.823 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-09-19T09:28:32.823 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 1, 2024-09-19T09:28:32.823 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:28:32.823 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-09-19T09:28:32.823 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-09-19T09:28:32.823 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.823 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-09-19T09:28:32.823 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 1209 2024-09-19T09:28:32.823 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.823 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-09-19T09:28:32.823 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-09-19T09:28:32.824 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 6, 2024-09-19T09:28:32.824 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-09-19T09:28:32.824 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 6, 2024-09-19T09:28:32.824 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 29720, 2024-09-19T09:28:32.824 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 29720, 2024-09-19T09:28:32.824 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-09-19T09:28:32.824 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.824 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:28:32.824 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T09:28:32.824 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-19T09:28:32.824 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T09:28:32.824 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 602981, 2024-09-19T09:28:32.824 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 113174247, 2024-09-19T09:28:32.824 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:32.825 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 602981, 2024-09-19T09:28:32.825 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-19T09:28:32.825 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-19T09:28:32.825 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 602981, 2024-09-19T09:28:32.825 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 113174247, 2024-09-19T09:28:32.825 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T09:28:32.825 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:32.825 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:32.825 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:32.825 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:32.825 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.825 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-09-19T09:28:32.825 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T09:28:32.825 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-19T09:28:32.826 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T09:28:32.826 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-19T09:28:32.826 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-19T09:28:32.826 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:32.826 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:32.826 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 81990, 2024-09-19T09:28:32.826 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 304146344, 2024-09-19T09:28:32.826 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 81990, 2024-09-19T09:28:32.826 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 304146344, 2024-09-19T09:28:32.826 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T09:28:32.826 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:32.826 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:32.826 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:32.827 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:32.827 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.827 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-09-19T09:28:32.827 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T09:28:32.827 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-09-19T09:28:32.827 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T09:28:32.827 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-19T09:28:32.827 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-19T09:28:32.827 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:32.827 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:32.827 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 81990, 2024-09-19T09:28:32.827 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 81990, 2024-09-19T09:28:32.827 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 81990, 2024-09-19T09:28:32.827 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 81990, 2024-09-19T09:28:32.828 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T09:28:32.828 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:32.828 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:32.828 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:32.828 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:32.828 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.828 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:28:32.828 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T09:28:32.828 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-19T09:28:32.828 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T09:28:32.828 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 29720, 2024-09-19T09:28:32.828 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 3001720, 2024-09-19T09:28:32.828 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:32.828 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 29720, 2024-09-19T09:28:32.829 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-19T09:28:32.829 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-19T09:28:32.829 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 913, 2024-09-19T09:28:32.829 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 3001720, 2024-09-19T09:28:32.829 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T09:28:32.829 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:32.829 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:32.829 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:32.829 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:32.829 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:32.829 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x55ac36528960": { 2024-09-19T09:28:32.829 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 180898, 2024-09-19T09:28:32.829 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-19T09:28:32.829 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T09:28:32.830 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 139566, 2024-09-19T09:28:32.830 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 294440526, 2024-09-19T09:28:32.830 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:32.830 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 139566, 2024-09-19T09:28:32.830 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-19T09:28:32.830 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-19T09:28:32.830 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 47954, 2024-09-19T09:28:32.830 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 294259628, 2024-09-19T09:28:32.830 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T09:28:32.830 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:32.830 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:32.830 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:32.830 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:32.830 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:32.831 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-19T09:28:33.819 INFO:tasks.check_counter:Daemon mds.c mds.exported=1 2024-09-19T09:28:33.819 INFO:tasks.check_counter:Daemon mds.c mds.imported=1 2024-09-19T09:28:33.819 DEBUG:tasks.check_counter:Getting stats from f 2024-09-19T09:28:33.820 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-19T09:28:34.085 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:28:34.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:34 smithi196 ceph-mon[35150]: pgmap v1116: 97 pgs: 97 active+clean; 263 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 5.3 MiB/s wr, 220 op/s 2024-09-19T09:28:34.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:34 smithi153 ceph-mon[35071]: pgmap v1116: 97 pgs: 97 active+clean; 263 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 5.3 MiB/s wr, 220 op/s 2024-09-19T09:28:34.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:34 smithi120 ceph-mon[31150]: pgmap v1116: 97 pgs: 97 active+clean; 263 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 5.3 MiB/s wr, 220 op/s 2024-09-19T09:28:34.725 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-19T09:28:34.725 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:28:34.725 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 162990, 2024-09-19T09:28:34.725 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 81287, 2024-09-19T09:28:34.726 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 44727830, 2024-09-19T09:28:34.726 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 13860623, 2024-09-19T09:28:34.726 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 36, 2024-09-19T09:28:34.726 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 3, 2024-09-19T09:28:34.726 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 7.165996232, 2024-09-19T09:28:34.726 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 2.616159923, 2024-09-19T09:28:34.726 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 6.254483340, 2024-09-19T09:28:34.726 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.007842914, 2024-09-19T09:28:34.726 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:34.726 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 81286, 2024-09-19T09:28:34.726 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 1.462093694, 2024-09-19T09:28:34.726 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000017987 2024-09-19T09:28:34.726 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.727 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:34.727 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 242429, 2024-09-19T09:28:34.727 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.034711081, 2024-09-19T09:28:34.727 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000143 2024-09-19T09:28:34.727 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.727 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 953296, 2024-09-19T09:28:34.727 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 284720 2024-09-19T09:28:34.727 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.727 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:28:34.727 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 42895, 2024-09-19T09:28:34.727 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 21783, 2024-09-19T09:28:34.727 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 11606918, 2024-09-19T09:28:34.727 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 4823275, 2024-09-19T09:28:34.727 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 61, 2024-09-19T09:28:34.728 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 5, 2024-09-19T09:28:34.728 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 2.293708473, 2024-09-19T09:28:34.728 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.977837027, 2024-09-19T09:28:34.728 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 1.796920483, 2024-09-19T09:28:34.728 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.000329094, 2024-09-19T09:28:34.728 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:34.728 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 21781, 2024-09-19T09:28:34.728 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.452901960, 2024-09-19T09:28:34.728 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000020793 2024-09-19T09:28:34.728 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.728 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:34.729 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 64261, 2024-09-19T09:28:34.729 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.009848030, 2024-09-19T09:28:34.729 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000153 2024-09-19T09:28:34.729 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.729 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T09:28:34.729 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 771264 2024-09-19T09:28:34.729 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.729 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:28:34.729 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 35, 2024-09-19T09:28:34.729 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 35, 2024-09-19T09:28:34.729 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 25021, 2024-09-19T09:28:34.729 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 23503, 2024-09-19T09:28:34.729 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 10, 2024-09-19T09:28:34.729 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 5, 2024-09-19T09:28:34.730 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.006303088, 2024-09-19T09:28:34.730 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.000484108, 2024-09-19T09:28:34.730 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.005375917, 2024-09-19T09:28:34.730 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.000357187, 2024-09-19T09:28:34.730 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:34.730 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 34, 2024-09-19T09:28:34.730 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.019340194, 2024-09-19T09:28:34.730 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000568829 2024-09-19T09:28:34.730 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.730 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:34.730 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 26, 2024-09-19T09:28:34.730 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000003391, 2024-09-19T09:28:34.730 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000130 2024-09-19T09:28:34.730 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.731 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-19T09:28:34.731 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T09:28:34.731 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.731 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-09-19T09:28:34.731 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-09-19T09:28:34.731 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-09-19T09:28:34.731 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.731 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-09-19T09:28:34.731 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-19T09:28:34.731 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-19T09:28:34.731 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 25, 2024-09-19T09:28:34.731 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.002614031, 2024-09-19T09:28:34.731 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000104561 2024-09-19T09:28:34.732 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:34.732 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.732 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-09-19T09:28:34.732 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-19T09:28:34.732 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-19T09:28:34.732 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2, 2024-09-19T09:28:34.732 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000006335, 2024-09-19T09:28:34.732 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000003167 2024-09-19T09:28:34.732 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:34.732 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.732 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-09-19T09:28:34.732 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 0, 2024-09-19T09:28:34.732 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 0, 2024-09-19T09:28:34.732 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-09-19T09:28:34.733 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.733 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.733 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.733 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.733 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-09-19T09:28:34.736 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 0, 2024-09-19T09:28:34.736 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-09-19T09:28:34.736 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-09-19T09:28:34.736 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 11, 2024-09-19T09:28:34.736 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-09-19T09:28:34.736 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-09-19T09:28:34.736 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 17, 2024-09-19T09:28:34.737 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 1, 2024-09-19T09:28:34.737 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 16, 2024-09-19T09:28:34.737 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-09-19T09:28:34.737 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 13, 2024-09-19T09:28:34.737 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 78, 2024-09-19T09:28:34.737 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 0, 2024-09-19T09:28:34.737 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 0, 2024-09-19T09:28:34.737 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 4, 2024-09-19T09:28:34.737 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 85, 2024-09-19T09:28:34.737 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 67, 2024-09-19T09:28:34.737 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 0, 2024-09-19T09:28:34.737 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 18, 2024-09-19T09:28:34.737 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-09-19T09:28:34.738 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-09-19T09:28:34.738 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 0, 2024-09-19T09:28:34.738 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-09-19T09:28:34.738 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-09-19T09:28:34.738 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-09-19T09:28:34.738 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-09-19T09:28:34.738 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 0, 2024-09-19T09:28:34.738 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 0, 2024-09-19T09:28:34.738 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-09-19T09:28:34.738 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-09-19T09:28:34.739 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-09-19T09:28:34.739 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 114, 2024-09-19T09:28:34.739 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 34500714, 2024-09-19T09:28:34.739 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 0, 2024-09-19T09:28:34.739 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T09:28:34.739 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-09-19T09:28:34.739 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:28:34.739 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:28:34.739 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-09-19T09:28:34.739 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T09:28:34.739 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T09:28:34.739 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-09-19T09:28:34.739 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-09-19T09:28:34.740 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T09:28:34.740 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 0, 2024-09-19T09:28:34.740 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T09:28:34.740 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:28:34.740 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:28:34.740 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 0, 2024-09-19T09:28:34.740 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 0, 2024-09-19T09:28:34.740 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 0, 2024-09-19T09:28:34.740 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 0 2024-09-19T09:28:34.740 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.740 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-09-19T09:28:34.740 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-09-19T09:28:34.740 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 60, 2024-09-19T09:28:34.740 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 39, 2024-09-19T09:28:34.741 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 47, 2024-09-19T09:28:34.741 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-09-19T09:28:34.741 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-09-19T09:28:34.741 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-09-19T09:28:34.741 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:28:34.741 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 0, 2024-09-19T09:28:34.741 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 0, 2024-09-19T09:28:34.741 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-09-19T09:28:34.741 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-09-19T09:28:34.741 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-09-19T09:28:34.741 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:28:34.741 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:28:34.741 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-09-19T09:28:34.742 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-09-19T09:28:34.742 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 22, 2024-09-19T09:28:34.742 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 554, 2024-09-19T09:28:34.742 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T09:28:34.742 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-09-19T09:28:34.742 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-09-19T09:28:34.742 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-09-19T09:28:34.742 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-09-19T09:28:34.742 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-09-19T09:28:34.742 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.742 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-09-19T09:28:34.742 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-09-19T09:28:34.742 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 2, 2024-09-19T09:28:34.742 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-09-19T09:28:34.743 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-09-19T09:28:34.743 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 2, 2024-09-19T09:28:34.743 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-09-19T09:28:34.743 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-09-19T09:28:34.743 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 2, 2024-09-19T09:28:34.743 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-09-19T09:28:34.743 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-09-19T09:28:34.743 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 2, 2024-09-19T09:28:34.743 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-09-19T09:28:34.743 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-09-19T09:28:34.743 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-09-19T09:28:34.743 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-09-19T09:28:34.743 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 4194892, 2024-09-19T09:28:34.743 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 0, 2024-09-19T09:28:34.744 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-09-19T09:28:34.744 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1, 2024-09-19T09:28:34.744 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.001285216, 2024-09-19T09:28:34.744 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.001285216 2024-09-19T09:28:34.744 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.744 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 0 2024-09-19T09:28:34.744 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.744 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-09-19T09:28:34.744 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 22, 2024-09-19T09:28:34.744 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 61, 2024-09-19T09:28:34.744 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 39, 2024-09-19T09:28:34.744 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 28, 2024-09-19T09:28:34.744 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 73, 2024-09-19T09:28:34.744 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 45, 2024-09-19T09:28:34.745 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 17, 2024-09-19T09:28:34.745 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 95, 2024-09-19T09:28:34.745 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 78, 2024-09-19T09:28:34.745 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 0, 2024-09-19T09:28:34.745 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 0, 2024-09-19T09:28:34.745 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 0, 2024-09-19T09:28:34.745 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 55780, 2024-09-19T09:28:34.745 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 223520 2024-09-19T09:28:34.745 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.745 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-09-19T09:28:34.745 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 0, 2024-09-19T09:28:34.745 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 2, 2024-09-19T09:28:34.745 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 0, 2024-09-19T09:28:34.746 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 0, 2024-09-19T09:28:34.746 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 4, 2024-09-19T09:28:34.746 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-09-19T09:28:34.746 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.746 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.746 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.746 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.746 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-09-19T09:28:34.746 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.746 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.746 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.746 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.746 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-09-19T09:28:34.747 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.747 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.747 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.747 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.748 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-09-19T09:28:34.748 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.748 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.748 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.748 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.748 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-09-19T09:28:34.748 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.748 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.748 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.748 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.748 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-09-19T09:28:34.749 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.749 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.749 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.749 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.749 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-09-19T09:28:34.749 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.749 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.749 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.749 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.749 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-09-19T09:28:34.749 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.749 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.749 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.749 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.750 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-09-19T09:28:34.750 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.750 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.750 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.750 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.750 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-09-19T09:28:34.750 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.750 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.750 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.750 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.750 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-09-19T09:28:34.750 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.750 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.750 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.751 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.751 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-09-19T09:28:34.751 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.752 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.752 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.752 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.752 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-09-19T09:28:34.752 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.752 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.752 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.752 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.752 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-09-19T09:28:34.752 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.752 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.752 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.752 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.753 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-09-19T09:28:34.753 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.753 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.753 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.753 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.753 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-09-19T09:28:34.753 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.753 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.753 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.753 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.753 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-09-19T09:28:34.753 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.753 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.753 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.754 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.754 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-09-19T09:28:34.754 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.754 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.754 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.754 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.754 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-09-19T09:28:34.754 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.754 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.754 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.754 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.754 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-09-19T09:28:34.754 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.754 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.755 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.755 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.755 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-09-19T09:28:34.755 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.755 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.755 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.755 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.755 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-09-19T09:28:34.755 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.755 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.755 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.755 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.755 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-09-19T09:28:34.756 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.756 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.756 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.756 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.756 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-09-19T09:28:34.756 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.756 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.756 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.756 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.756 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-09-19T09:28:34.756 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.756 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.756 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.757 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.757 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-09-19T09:28:34.757 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.757 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.757 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.757 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.757 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-09-19T09:28:34.757 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.757 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.757 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.757 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.757 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-09-19T09:28:34.757 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.757 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.758 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.758 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.758 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-09-19T09:28:34.758 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.758 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.758 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.758 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.758 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:28:34.758 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:28:34.758 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-09-19T09:28:34.758 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.758 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.759 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.759 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:34.759 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.759 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-09-19T09:28:34.759 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 0, 2024-09-19T09:28:34.759 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 0, 2024-09-19T09:28:34.759 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 0, 2024-09-19T09:28:34.759 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 0, 2024-09-19T09:28:34.759 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-09-19T09:28:34.759 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 0, 2024-09-19T09:28:34.759 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 0, 2024-09-19T09:28:34.759 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 0, 2024-09-19T09:28:34.759 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-09-19T09:28:34.760 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.760 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-09-19T09:28:34.760 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:28:34.760 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-09-19T09:28:34.760 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:28:34.760 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:28:34.760 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:28:34.760 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:28:34.760 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:28:34.761 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:28:34.761 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:28:34.761 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:28:34.761 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:28:34.761 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:28:34.761 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:28:34.761 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:28:34.761 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:28:34.761 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-09-19T09:28:34.761 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:28:34.761 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-09-19T09:28:34.761 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:28:34.761 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:28:34.762 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:28:34.762 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:28:34.762 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:28:34.762 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:28:34.762 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:28:34.762 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-09-19T09:28:34.762 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:28:34.762 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:28:34.762 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:28:34.762 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-09-19T09:28:34.762 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-09-19T09:28:34.762 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-09-19T09:28:34.763 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:28:34.763 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:28:34.763 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:28:34.763 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:28:34.763 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 238855, 2024-09-19T09:28:34.763 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 79, 2024-09-19T09:28:34.763 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:28:34.763 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-09-19T09:28:34.763 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-09-19T09:28:34.763 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-09-19T09:28:34.763 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:28:34.764 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-09-19T09:28:34.764 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:28:34.764 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-09-19T09:28:34.764 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 7592, 2024-09-19T09:28:34.764 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 156, 2024-09-19T09:28:34.764 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:28:34.764 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:28:34.764 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-09-19T09:28:34.764 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-09-19T09:28:34.764 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 98389, 2024-09-19T09:28:34.764 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 363, 2024-09-19T09:28:34.764 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-09-19T09:28:34.765 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-09-19T09:28:34.765 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-09-19T09:28:34.765 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-09-19T09:28:34.765 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.765 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-09-19T09:28:34.765 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-09-19T09:28:34.765 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-09-19T09:28:34.765 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 35, 2024-09-19T09:28:34.765 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 10710, 2024-09-19T09:28:34.765 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 29, 2024-09-19T09:28:34.765 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 35, 2024-09-19T09:28:34.766 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-09-19T09:28:34.766 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 35, 2024-09-19T09:28:34.766 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.174996104, 2024-09-19T09:28:34.766 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.004999888 2024-09-19T09:28:34.766 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.766 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-09-19T09:28:34.766 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-09-19T09:28:34.766 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 35, 2024-09-19T09:28:34.766 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 57 2024-09-19T09:28:34.766 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.766 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 35, 2024-09-19T09:28:34.766 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 0, 2024-09-19T09:28:34.766 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 35, 2024-09-19T09:28:34.766 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-09-19T09:28:34.767 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-09-19T09:28:34.767 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 0, 2024-09-19T09:28:34.767 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 10, 2024-09-19T09:28:34.767 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 0, 2024-09-19T09:28:34.767 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 1, 2024-09-19T09:28:34.767 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 5, 2024-09-19T09:28:34.767 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-09-19T09:28:34.767 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-09-19T09:28:34.767 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-09-19T09:28:34.767 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-09-19T09:28:34.767 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 6, 2024-09-19T09:28:34.767 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-09-19T09:28:34.767 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-09-19T09:28:34.767 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-09-19T09:28:34.768 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 0, 2024-09-19T09:28:34.768 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 20, 2024-09-19T09:28:34.768 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:28:34.768 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-09-19T09:28:34.768 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:28:34.768 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-09-19T09:28:34.768 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-09-19T09:28:34.768 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-09-19T09:28:34.768 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:28:34.768 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-09-19T09:28:34.768 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:28:34.768 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-09-19T09:28:34.768 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-09-19T09:28:34.768 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-09-19T09:28:34.769 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-09-19T09:28:34.769 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-09-19T09:28:34.769 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-09-19T09:28:34.769 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-09-19T09:28:34.769 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-09-19T09:28:34.769 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-09-19T09:28:34.769 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-09-19T09:28:34.769 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-09-19T09:28:34.769 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-09-19T09:28:34.769 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-09-19T09:28:34.769 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-09-19T09:28:34.769 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-09-19T09:28:34.769 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-09-19T09:28:34.769 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-09-19T09:28:34.770 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 85, 2024-09-19T09:28:34.770 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-09-19T09:28:34.770 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 6, 2024-09-19T09:28:34.770 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 9, 2024-09-19T09:28:34.770 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 16, 2024-09-19T09:28:34.770 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 7, 2024-09-19T09:28:34.770 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-09-19T09:28:34.770 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 14, 2024-09-19T09:28:34.770 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 0, 2024-09-19T09:28:34.770 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 1 2024-09-19T09:28:34.770 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.770 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-09-19T09:28:34.770 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 1, 2024-09-19T09:28:34.771 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:28:34.771 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-09-19T09:28:34.771 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-09-19T09:28:34.771 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.771 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-09-19T09:28:34.771 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 532 2024-09-19T09:28:34.771 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.771 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-09-19T09:28:34.771 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-09-19T09:28:34.771 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T09:28:34.771 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-09-19T09:28:34.771 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 0, 2024-09-19T09:28:34.771 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 0, 2024-09-19T09:28:34.771 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 0, 2024-09-19T09:28:34.772 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-09-19T09:28:34.772 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.772 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:28:34.772 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T09:28:34.772 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-19T09:28:34.772 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T09:28:34.772 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 205916, 2024-09-19T09:28:34.772 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 37798321, 2024-09-19T09:28:34.772 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:34.772 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 205916, 2024-09-19T09:28:34.772 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-19T09:28:34.772 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-19T09:28:34.772 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 205916, 2024-09-19T09:28:34.773 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 37798321, 2024-09-19T09:28:34.773 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T09:28:34.773 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.773 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.773 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.773 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:34.773 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.773 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-09-19T09:28:34.773 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T09:28:34.773 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-19T09:28:34.773 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T09:28:34.773 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-19T09:28:34.773 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-19T09:28:34.773 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:34.773 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:34.774 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 35, 2024-09-19T09:28:34.774 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 10710, 2024-09-19T09:28:34.774 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 35, 2024-09-19T09:28:34.774 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 10710, 2024-09-19T09:28:34.774 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T09:28:34.774 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.774 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.774 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.774 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:34.775 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.775 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-09-19T09:28:34.775 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T09:28:34.775 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-09-19T09:28:34.775 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T09:28:34.775 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-19T09:28:34.775 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-19T09:28:34.775 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:34.775 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:34.775 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 35, 2024-09-19T09:28:34.775 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 35, 2024-09-19T09:28:34.776 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 35, 2024-09-19T09:28:34.776 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 35, 2024-09-19T09:28:34.776 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T09:28:34.776 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.776 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.776 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.776 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:34.776 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.777 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:28:34.777 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T09:28:34.777 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-19T09:28:34.777 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T09:28:34.777 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-19T09:28:34.777 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-19T09:28:34.777 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:34.777 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:34.777 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-19T09:28:34.777 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-19T09:28:34.777 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-09-19T09:28:34.777 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-09-19T09:28:34.777 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T09:28:34.777 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.778 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.778 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.778 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:34.778 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:34.778 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x55836d992960": { 2024-09-19T09:28:34.778 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T09:28:34.778 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-19T09:28:34.778 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T09:28:34.778 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 2, 2024-09-19T09:28:34.778 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 588, 2024-09-19T09:28:34.778 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:34.778 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 2, 2024-09-19T09:28:34.778 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-19T09:28:34.778 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-19T09:28:34.778 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 1, 2024-09-19T09:28:34.779 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 588, 2024-09-19T09:28:34.779 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T09:28:34.779 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:34.779 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:34.779 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:34.779 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:34.779 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:34.779 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-19T09:28:35.519 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-19T09:28:35.519 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-19T09:28:35.519 DEBUG:tasks.check_counter:Getting stats from i 2024-09-19T09:28:35.519 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-19T09:28:35.772 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:28:36.157 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:36 smithi196 ceph-mon[35150]: pgmap v1117: 97 pgs: 97 active+clean; 263 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 5.3 MiB/s wr, 219 op/s 2024-09-19T09:28:36.261 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-19T09:28:36.261 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:28:36.262 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 31102, 2024-09-19T09:28:36.262 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 33048, 2024-09-19T09:28:36.262 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 10036096, 2024-09-19T09:28:36.262 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 10573438, 2024-09-19T09:28:36.262 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 21, 2024-09-19T09:28:36.262 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-09-19T09:28:36.262 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 2.277828573, 2024-09-19T09:28:36.262 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.892008941, 2024-09-19T09:28:36.262 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 1.844995844, 2024-09-19T09:28:36.262 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.026141148, 2024-09-19T09:28:36.262 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:36.262 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 33046, 2024-09-19T09:28:36.262 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.635639798, 2024-09-19T09:28:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000019234 2024-09-19T09:28:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 61570, 2024-09-19T09:28:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.008196423, 2024-09-19T09:28:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000133 2024-09-19T09:28:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 9762080, 2024-09-19T09:28:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 6714864 2024-09-19T09:28:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:28:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 43447, 2024-09-19T09:28:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 22332, 2024-09-19T09:28:36.263 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 11758167, 2024-09-19T09:28:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 5330010, 2024-09-19T09:28:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 80, 2024-09-19T09:28:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 8, 2024-09-19T09:28:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 2.422484295, 2024-09-19T09:28:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 1.051594879, 2024-09-19T09:28:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 1.825504932, 2024-09-19T09:28:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.014770712, 2024-09-19T09:28:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 22331, 2024-09-19T09:28:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.474118947, 2024-09-19T09:28:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000021231 2024-09-19T09:28:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.264 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 64278, 2024-09-19T09:28:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.009379254, 2024-09-19T09:28:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000145 2024-09-19T09:28:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-19T09:28:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 773984 2024-09-19T09:28:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:28:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 166827, 2024-09-19T09:28:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 85279, 2024-09-19T09:28:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 45794435, 2024-09-19T09:28:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 14817405, 2024-09-19T09:28:36.265 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 8, 2024-09-19T09:28:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 6, 2024-09-19T09:28:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 7.633111636, 2024-09-19T09:28:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 2.807973357, 2024-09-19T09:28:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 6.599572496, 2024-09-19T09:28:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.026126775, 2024-09-19T09:28:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 85278, 2024-09-19T09:28:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 1.661473141, 2024-09-19T09:28:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000019483 2024-09-19T09:28:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 249041, 2024-09-19T09:28:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.035510585, 2024-09-19T09:28:36.266 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000142 2024-09-19T09:28:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 968848, 2024-09-19T09:28:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 295264 2024-09-19T09:28:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-09-19T09:28:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-09-19T09:28:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-09-19T09:28:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-09-19T09:28:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-19T09:28:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-19T09:28:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1901, 2024-09-19T09:28:36.267 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.738579384, 2024-09-19T09:28:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000388521 2024-09-19T09:28:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-09-19T09:28:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-19T09:28:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-19T09:28:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1688, 2024-09-19T09:28:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.053453440, 2024-09-19T09:28:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000031666 2024-09-19T09:28:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-09-19T09:28:36.268 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 28883, 2024-09-19T09:28:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 1713, 2024-09-19T09:28:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-09-19T09:28:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1713, 2024-09-19T09:28:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 18.849982158, 2024-09-19T09:28:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.011004075 2024-09-19T09:28:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-09-19T09:28:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 27170, 2024-09-19T09:28:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-09-19T09:28:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-09-19T09:28:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 11, 2024-09-19T09:28:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-09-19T09:28:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-09-19T09:28:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 26, 2024-09-19T09:28:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 0, 2024-09-19T09:28:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 26, 2024-09-19T09:28:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-09-19T09:28:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 22, 2024-09-19T09:28:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 80, 2024-09-19T09:28:36.270 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 0, 2024-09-19T09:28:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 0, 2024-09-19T09:28:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 4, 2024-09-19T09:28:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 31218, 2024-09-19T09:28:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 1405, 2024-09-19T09:28:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 27169, 2024-09-19T09:28:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 20, 2024-09-19T09:28:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-09-19T09:28:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-09-19T09:28:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 0, 2024-09-19T09:28:36.271 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-09-19T09:28:36.272 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-09-19T09:28:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 1, 2024-09-19T09:28:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-09-19T09:28:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 1, 2024-09-19T09:28:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 1, 2024-09-19T09:28:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-09-19T09:28:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 36, 2024-09-19T09:28:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-09-19T09:28:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 114, 2024-09-19T09:28:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 34500714, 2024-09-19T09:28:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 0, 2024-09-19T09:28:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 36, 2024-09-19T09:28:36.273 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-09-19T09:28:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:28:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:28:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-09-19T09:28:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T09:28:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T09:28:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 36, 2024-09-19T09:28:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-09-19T09:28:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 40, 2024-09-19T09:28:36.274 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 1620, 2024-09-19T09:28:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T09:28:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:28:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:28:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 782, 2024-09-19T09:28:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 500, 2024-09-19T09:28:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 275, 2024-09-19T09:28:36.275 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 1749 2024-09-19T09:28:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-09-19T09:28:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-09-19T09:28:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 60, 2024-09-19T09:28:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 39, 2024-09-19T09:28:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 49, 2024-09-19T09:28:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-09-19T09:28:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-09-19T09:28:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-09-19T09:28:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:28:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 421, 2024-09-19T09:28:36.276 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 421, 2024-09-19T09:28:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-09-19T09:28:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-09-19T09:28:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-09-19T09:28:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:28:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:28:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-09-19T09:28:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-09-19T09:28:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 22, 2024-09-19T09:28:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 590, 2024-09-19T09:28:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T09:28:36.277 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 1, 2024-09-19T09:28:36.278 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-09-19T09:28:36.278 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-09-19T09:28:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-09-19T09:28:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-09-19T09:28:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-09-19T09:28:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-09-19T09:28:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 2639, 2024-09-19T09:28:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-09-19T09:28:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-09-19T09:28:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 2639, 2024-09-19T09:28:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-09-19T09:28:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-09-19T09:28:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 4, 2024-09-19T09:28:36.279 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-09-19T09:28:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-09-19T09:28:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 4, 2024-09-19T09:28:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-09-19T09:28:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-09-19T09:28:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-09-19T09:28:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-09-19T09:28:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 7611251, 2024-09-19T09:28:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 0, 2024-09-19T09:28:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-09-19T09:28:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 820, 2024-09-19T09:28:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 3.727401286, 2024-09-19T09:28:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.004545611 2024-09-19T09:28:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.280 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 0 2024-09-19T09:28:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-09-19T09:28:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 22, 2024-09-19T09:28:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 483, 2024-09-19T09:28:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 461, 2024-09-19T09:28:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 28, 2024-09-19T09:28:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 74, 2024-09-19T09:28:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 46, 2024-09-19T09:28:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 17, 2024-09-19T09:28:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 539, 2024-09-19T09:28:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 522, 2024-09-19T09:28:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 0, 2024-09-19T09:28:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 429, 2024-09-19T09:28:36.281 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 429, 2024-09-19T09:28:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 57068, 2024-09-19T09:28:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 223520 2024-09-19T09:28:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-09-19T09:28:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 30753, 2024-09-19T09:28:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 3, 2024-09-19T09:28:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 28883, 2024-09-19T09:28:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 24, 2024-09-19T09:28:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 853, 2024-09-19T09:28:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-09-19T09:28:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 420, 2024-09-19T09:28:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 16.239170960, 2024-09-19T09:28:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.038664692 2024-09-19T09:28:36.282 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-09-19T09:28:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3, 2024-09-19T09:28:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.010112880, 2024-09-19T09:28:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.003370960 2024-09-19T09:28:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-09-19T09:28:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-09-19T09:28:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-09-19T09:28:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 383, 2024-09-19T09:28:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.095472463, 2024-09-19T09:28:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000249275 2024-09-19T09:28:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-09-19T09:28:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-09-19T09:28:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.284 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-09-19T09:28:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-09-19T09:28:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.285 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-09-19T09:28:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.286 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-09-19T09:28:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-09-19T09:28:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.287 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-09-19T09:28:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-09-19T09:28:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-09-19T09:28:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.288 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-09-19T09:28:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 486, 2024-09-19T09:28:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.551735970, 2024-09-19T09:28:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.001135259 2024-09-19T09:28:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-09-19T09:28:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-09-19T09:28:36.289 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-09-19T09:28:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-09-19T09:28:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.290 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-09-19T09:28:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-09-19T09:28:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-09-19T09:28:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-09-19T09:28:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.292 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-09-19T09:28:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-09-19T09:28:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-09-19T09:28:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.293 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-09-19T09:28:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-09-19T09:28:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 421, 2024-09-19T09:28:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 1.953489885, 2024-09-19T09:28:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.004640118 2024-09-19T09:28:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:28:36.294 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:28:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-09-19T09:28:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-09-19T09:28:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 1, 2024-09-19T09:28:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 2, 2024-09-19T09:28:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 1, 2024-09-19T09:28:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 1, 2024-09-19T09:28:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-09-19T09:28:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 178, 2024-09-19T09:28:36.295 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 178, 2024-09-19T09:28:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 1440, 2024-09-19T09:28:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-09-19T09:28:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-09-19T09:28:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:28:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-09-19T09:28:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:28:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:28:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:28:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:28:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:28:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:28:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:28:36.296 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:28:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:28:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:28:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:28:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:28:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:28:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-09-19T09:28:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:28:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-09-19T09:28:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:28:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:28:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:28:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:28:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:28:36.297 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:28:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:28:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-09-19T09:28:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:28:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:28:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:28:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-09-19T09:28:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-09-19T09:28:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-09-19T09:28:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:28:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:28:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:28:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:28:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 287655, 2024-09-19T09:28:36.298 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 104, 2024-09-19T09:28:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:28:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-09-19T09:28:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-09-19T09:28:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-09-19T09:28:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:28:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-09-19T09:28:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:28:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-09-19T09:28:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 7592, 2024-09-19T09:28:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 156, 2024-09-19T09:28:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:28:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:28:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-09-19T09:28:36.299 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-09-19T09:28:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 98565, 2024-09-19T09:28:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 385, 2024-09-19T09:28:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-09-19T09:28:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-09-19T09:28:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-09-19T09:28:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-09-19T09:28:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-09-19T09:28:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-09-19T09:28:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-09-19T09:28:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 2178, 2024-09-19T09:28:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 3552079, 2024-09-19T09:28:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 29, 2024-09-19T09:28:36.300 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 2178, 2024-09-19T09:28:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-09-19T09:28:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2178, 2024-09-19T09:28:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 10.580768690, 2024-09-19T09:28:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.004858020 2024-09-19T09:28:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-09-19T09:28:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-09-19T09:28:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2178, 2024-09-19T09:28:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 2513 2024-09-19T09:28:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 2178, 2024-09-19T09:28:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 264, 2024-09-19T09:28:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 1914, 2024-09-19T09:28:36.301 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-09-19T09:28:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-09-19T09:28:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 0, 2024-09-19T09:28:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 10, 2024-09-19T09:28:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 228, 2024-09-19T09:28:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 1048, 2024-09-19T09:28:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 175, 2024-09-19T09:28:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-09-19T09:28:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-09-19T09:28:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-09-19T09:28:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-09-19T09:28:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 433, 2024-09-19T09:28:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-09-19T09:28:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-09-19T09:28:36.302 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-09-19T09:28:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 36, 2024-09-19T09:28:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 20, 2024-09-19T09:28:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:28:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-09-19T09:28:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:28:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-09-19T09:28:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-09-19T09:28:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-09-19T09:28:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:28:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-09-19T09:28:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:28:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-09-19T09:28:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-09-19T09:28:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-09-19T09:28:36.303 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-09-19T09:28:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-09-19T09:28:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-09-19T09:28:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-09-19T09:28:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-09-19T09:28:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-09-19T09:28:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-09-19T09:28:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-09-19T09:28:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-09-19T09:28:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-09-19T09:28:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-09-19T09:28:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-09-19T09:28:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-09-19T09:28:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-09-19T09:28:36.304 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 85, 2024-09-19T09:28:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-09-19T09:28:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 6, 2024-09-19T09:28:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 12, 2024-09-19T09:28:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 15, 2024-09-19T09:28:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 3, 2024-09-19T09:28:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-09-19T09:28:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 437, 2024-09-19T09:28:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 0, 2024-09-19T09:28:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 126 2024-09-19T09:28:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-09-19T09:28:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 1, 2024-09-19T09:28:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:28:36.305 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-09-19T09:28:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-09-19T09:28:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-09-19T09:28:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 566 2024-09-19T09:28:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-09-19T09:28:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-09-19T09:28:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 6, 2024-09-19T09:28:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-09-19T09:28:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 6, 2024-09-19T09:28:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 421, 2024-09-19T09:28:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 421, 2024-09-19T09:28:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-09-19T09:28:36.306 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:28:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T09:28:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-19T09:28:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T09:28:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 241372, 2024-09-19T09:28:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 44572057, 2024-09-19T09:28:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 241372, 2024-09-19T09:28:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-19T09:28:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-19T09:28:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 241372, 2024-09-19T09:28:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 44572057, 2024-09-19T09:28:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T09:28:36.307 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-09-19T09:28:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T09:28:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-19T09:28:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T09:28:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-19T09:28:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-19T09:28:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 2178, 2024-09-19T09:28:36.308 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 3594600, 2024-09-19T09:28:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 2178, 2024-09-19T09:28:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 3594600, 2024-09-19T09:28:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T09:28:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-09-19T09:28:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T09:28:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-09-19T09:28:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T09:28:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-19T09:28:36.309 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-19T09:28:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 2178, 2024-09-19T09:28:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 2178, 2024-09-19T09:28:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 2178, 2024-09-19T09:28:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 2178, 2024-09-19T09:28:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T09:28:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:28:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T09:28:36.310 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-19T09:28:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T09:28:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 421, 2024-09-19T09:28:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 42521, 2024-09-19T09:28:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 421, 2024-09-19T09:28:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-19T09:28:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-19T09:28:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 228, 2024-09-19T09:28:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 42521, 2024-09-19T09:28:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T09:28:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.311 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x55d472024960": { 2024-09-19T09:28:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T09:28:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-19T09:28:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T09:28:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 2639, 2024-09-19T09:28:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 3416947, 2024-09-19T09:28:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 2639, 2024-09-19T09:28:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-19T09:28:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-19T09:28:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 820, 2024-09-19T09:28:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 3416947, 2024-09-19T09:28:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T09:28:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:36.313 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:36.313 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-19T09:28:36.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:36 smithi153 ceph-mon[35071]: pgmap v1117: 97 pgs: 97 active+clean; 263 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 5.3 MiB/s wr, 219 op/s 2024-09-19T09:28:36.577 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:36 smithi120 ceph-mon[31150]: pgmap v1117: 97 pgs: 97 active+clean; 263 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 5.3 MiB/s wr, 219 op/s 2024-09-19T09:28:37.035 INFO:tasks.check_counter:Daemon mds.i mds.exported=1 2024-09-19T09:28:37.035 INFO:tasks.check_counter:Daemon mds.i mds.imported=1 2024-09-19T09:28:37.035 DEBUG:tasks.check_counter:Getting stats from l 2024-09-19T09:28:37.036 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-19T09:28:37.284 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:28:37.826 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-19T09:28:37.826 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:28:37.826 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 4913, 2024-09-19T09:28:37.826 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 4913, 2024-09-19T09:28:37.826 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 13294237, 2024-09-19T09:28:37.827 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 1561517, 2024-09-19T09:28:37.827 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 6, 2024-09-19T09:28:37.827 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 2, 2024-09-19T09:28:37.827 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.355197154, 2024-09-19T09:28:37.827 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.111307876, 2024-09-19T09:28:37.827 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.094909734, 2024-09-19T09:28:37.827 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.123112475, 2024-09-19T09:28:37.827 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:37.827 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 4912, 2024-09-19T09:28:37.827 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.150272493, 2024-09-19T09:28:37.827 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000030592 2024-09-19T09:28:37.827 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.827 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:37.827 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.828 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.828 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.828 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.828 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-19T09:28:37.828 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T09:28:37.828 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.828 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:28:37.828 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 210, 2024-09-19T09:28:37.828 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 555, 2024-09-19T09:28:37.828 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 51552, 2024-09-19T09:28:37.828 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 839790, 2024-09-19T09:28:37.829 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 6, 2024-09-19T09:28:37.829 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-09-19T09:28:37.829 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.032035408, 2024-09-19T09:28:37.829 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.016494141, 2024-09-19T09:28:37.829 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.005152551, 2024-09-19T09:28:37.829 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.006217083, 2024-09-19T09:28:37.830 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:37.830 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 553, 2024-09-19T09:28:37.830 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.020819832, 2024-09-19T09:28:37.830 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000037648 2024-09-19T09:28:37.830 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.830 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:37.831 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.831 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.831 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.831 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.831 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T09:28:37.832 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 773584 2024-09-19T09:28:37.832 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.832 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:28:37.832 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 485, 2024-09-19T09:28:37.832 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 465, 2024-09-19T09:28:37.832 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 942572, 2024-09-19T09:28:37.832 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 282798, 2024-09-19T09:28:37.833 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 4, 2024-09-19T09:28:37.833 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 2, 2024-09-19T09:28:37.833 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.055335924, 2024-09-19T09:28:37.833 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.025055871, 2024-09-19T09:28:37.833 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.039222305, 2024-09-19T09:28:37.833 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.000027098, 2024-09-19T09:28:37.833 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:28:37.833 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 464, 2024-09-19T09:28:37.833 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.026487330, 2024-09-19T09:28:37.833 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000057084 2024-09-19T09:28:37.833 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.834 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:28:37.834 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.834 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.834 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.834 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.834 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 942304, 2024-09-19T09:28:37.834 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 282480 2024-09-19T09:28:37.834 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.834 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-09-19T09:28:37.834 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-09-19T09:28:37.834 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-09-19T09:28:37.834 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.834 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-09-19T09:28:37.835 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-19T09:28:37.835 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-19T09:28:37.835 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 8651, 2024-09-19T09:28:37.835 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.196506307, 2024-09-19T09:28:37.835 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000022714 2024-09-19T09:28:37.835 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:37.835 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.835 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-09-19T09:28:37.835 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-19T09:28:37.835 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-19T09:28:37.835 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.835 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.835 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.836 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:37.836 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.836 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-09-19T09:28:37.836 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 0, 2024-09-19T09:28:37.836 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 0, 2024-09-19T09:28:37.836 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-09-19T09:28:37.836 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.836 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.836 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.836 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.836 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-09-19T09:28:37.836 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 0, 2024-09-19T09:28:37.836 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-09-19T09:28:37.836 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-09-19T09:28:37.837 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 0, 2024-09-19T09:28:37.837 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-09-19T09:28:37.837 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-09-19T09:28:37.837 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 1017, 2024-09-19T09:28:37.837 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 0, 2024-09-19T09:28:37.837 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 1017, 2024-09-19T09:28:37.837 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-09-19T09:28:37.837 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 1017, 2024-09-19T09:28:37.837 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 0, 2024-09-19T09:28:37.837 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 0, 2024-09-19T09:28:37.837 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 0, 2024-09-19T09:28:37.837 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 1, 2024-09-19T09:28:37.837 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 0, 2024-09-19T09:28:37.838 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 0, 2024-09-19T09:28:37.838 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 0, 2024-09-19T09:28:37.838 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 0, 2024-09-19T09:28:37.838 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-09-19T09:28:37.838 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-09-19T09:28:37.838 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 0, 2024-09-19T09:28:37.838 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-09-19T09:28:37.838 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-09-19T09:28:37.838 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-09-19T09:28:37.838 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-09-19T09:28:37.838 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 0, 2024-09-19T09:28:37.838 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 0, 2024-09-19T09:28:37.838 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-09-19T09:28:37.838 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-09-19T09:28:37.839 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-09-19T09:28:37.839 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 0, 2024-09-19T09:28:37.839 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 0, 2024-09-19T09:28:37.839 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 0, 2024-09-19T09:28:37.839 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T09:28:37.839 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-09-19T09:28:37.839 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:28:37.839 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:28:37.839 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-09-19T09:28:37.839 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T09:28:37.840 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T09:28:37.840 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-09-19T09:28:37.840 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-09-19T09:28:37.840 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T09:28:37.840 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 0, 2024-09-19T09:28:37.840 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T09:28:37.840 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:28:37.840 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:28:37.840 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 0, 2024-09-19T09:28:37.840 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 0, 2024-09-19T09:28:37.840 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 0, 2024-09-19T09:28:37.840 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 0 2024-09-19T09:28:37.840 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.840 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-09-19T09:28:37.841 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-09-19T09:28:37.841 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 0, 2024-09-19T09:28:37.841 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 0, 2024-09-19T09:28:37.841 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 0, 2024-09-19T09:28:37.841 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-09-19T09:28:37.841 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-09-19T09:28:37.841 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-09-19T09:28:37.841 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:28:37.841 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 0, 2024-09-19T09:28:37.841 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 0, 2024-09-19T09:28:37.841 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-09-19T09:28:37.841 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-09-19T09:28:37.841 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-09-19T09:28:37.841 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:28:37.842 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:28:37.842 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-09-19T09:28:37.842 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-09-19T09:28:37.842 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 0, 2024-09-19T09:28:37.842 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 0, 2024-09-19T09:28:37.842 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T09:28:37.842 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-09-19T09:28:37.842 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-09-19T09:28:37.842 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-09-19T09:28:37.842 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-09-19T09:28:37.842 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-09-19T09:28:37.842 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.842 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-09-19T09:28:37.842 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-09-19T09:28:37.843 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 0, 2024-09-19T09:28:37.843 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-09-19T09:28:37.843 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-09-19T09:28:37.843 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 8705, 2024-09-19T09:28:37.843 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-09-19T09:28:37.843 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-09-19T09:28:37.843 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 0, 2024-09-19T09:28:37.843 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-09-19T09:28:37.843 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-09-19T09:28:37.843 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 11, 2024-09-19T09:28:37.843 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-09-19T09:28:37.844 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-09-19T09:28:37.844 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-09-19T09:28:37.844 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-09-19T09:28:37.844 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 16101716, 2024-09-19T09:28:37.844 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 16100578, 2024-09-19T09:28:37.844 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-09-19T09:28:37.844 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.845 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.845 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.845 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.845 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 8705 2024-09-19T09:28:37.845 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.845 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-09-19T09:28:37.845 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 12, 2024-09-19T09:28:37.845 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 1032, 2024-09-19T09:28:37.845 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 1020, 2024-09-19T09:28:37.845 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 10, 2024-09-19T09:28:37.845 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 23, 2024-09-19T09:28:37.845 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 13, 2024-09-19T09:28:37.845 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 1017, 2024-09-19T09:28:37.845 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 1070, 2024-09-19T09:28:37.846 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 53, 2024-09-19T09:28:37.846 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 0, 2024-09-19T09:28:37.846 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 0, 2024-09-19T09:28:37.846 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 0, 2024-09-19T09:28:37.846 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 52272, 2024-09-19T09:28:37.846 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 215328 2024-09-19T09:28:37.846 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.847 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-09-19T09:28:37.847 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 0, 2024-09-19T09:28:37.847 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 0, 2024-09-19T09:28:37.847 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 0, 2024-09-19T09:28:37.847 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 0, 2024-09-19T09:28:37.847 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 0, 2024-09-19T09:28:37.848 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-09-19T09:28:37.848 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.848 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.848 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.848 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.849 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-09-19T09:28:37.850 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.850 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.850 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.850 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.850 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-09-19T09:28:37.850 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.850 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.850 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.850 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.850 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-09-19T09:28:37.850 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.851 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.851 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.851 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.851 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-09-19T09:28:37.851 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.851 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.851 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.851 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.851 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-09-19T09:28:37.851 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.851 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.851 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.852 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.852 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-09-19T09:28:37.852 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.852 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.852 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.852 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.852 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-09-19T09:28:37.852 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.852 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.852 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.852 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.853 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-09-19T09:28:37.853 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.853 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.853 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.853 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.853 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-09-19T09:28:37.853 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.853 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.853 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.853 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.853 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-09-19T09:28:37.854 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.854 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.854 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.854 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.854 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-09-19T09:28:37.854 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.854 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.854 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.854 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.854 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-09-19T09:28:37.854 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.855 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.855 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.855 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.855 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-09-19T09:28:37.855 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.855 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.855 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.855 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.855 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-09-19T09:28:37.855 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.855 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.855 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.856 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.856 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-09-19T09:28:37.856 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.856 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.856 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.856 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.856 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-09-19T09:28:37.856 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.856 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.856 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.856 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.856 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-09-19T09:28:37.857 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.857 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.857 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.857 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.857 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-09-19T09:28:37.857 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.857 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.857 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.857 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.857 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-09-19T09:28:37.858 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.858 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.858 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.858 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.858 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-09-19T09:28:37.858 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.858 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.858 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.858 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.858 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-09-19T09:28:37.858 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.859 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.859 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.859 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.859 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-09-19T09:28:37.859 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.859 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.859 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.859 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.859 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-09-19T09:28:37.859 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.860 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.860 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.860 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.860 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-09-19T09:28:37.860 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.860 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.860 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.860 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.860 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-09-19T09:28:37.860 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.860 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.861 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.861 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-09-19T09:28:37.861 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.861 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.861 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.861 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.861 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-09-19T09:28:37.861 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.861 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.861 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.862 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.862 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-09-19T09:28:37.862 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.862 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.862 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.862 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.862 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:28:37.862 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:28:37.862 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-09-19T09:28:37.862 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.862 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.862 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.863 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:37.863 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.863 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-09-19T09:28:37.863 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 1, 2024-09-19T09:28:37.863 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 1, 2024-09-19T09:28:37.863 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 0, 2024-09-19T09:28:37.863 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 1, 2024-09-19T09:28:37.863 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-09-19T09:28:37.863 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 0, 2024-09-19T09:28:37.863 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 0, 2024-09-19T09:28:37.863 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 2881, 2024-09-19T09:28:37.864 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-09-19T09:28:37.864 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.864 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-09-19T09:28:37.864 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:28:37.864 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-09-19T09:28:37.864 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:28:37.864 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:28:37.864 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:28:37.865 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:28:37.865 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:28:37.865 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:28:37.865 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:28:37.865 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:28:37.865 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:28:37.865 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:28:37.865 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:28:37.865 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:28:37.865 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:28:37.865 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-09-19T09:28:37.866 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:28:37.866 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-09-19T09:28:37.866 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:28:37.866 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:28:37.866 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:28:37.866 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:28:37.866 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:28:37.866 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:28:37.866 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:28:37.866 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-09-19T09:28:37.866 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:28:37.867 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:28:37.867 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:28:37.867 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-09-19T09:28:37.867 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-09-19T09:28:37.867 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-09-19T09:28:37.867 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:28:37.867 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:28:37.867 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:28:37.867 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:28:37.867 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 191736, 2024-09-19T09:28:37.867 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 56, 2024-09-19T09:28:37.867 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:28:37.868 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-09-19T09:28:37.868 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-09-19T09:28:37.868 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-09-19T09:28:37.868 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:28:37.868 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-09-19T09:28:37.868 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:28:37.868 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-09-19T09:28:37.868 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 7592, 2024-09-19T09:28:37.868 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 156, 2024-09-19T09:28:37.869 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:28:37.869 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:28:37.869 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-09-19T09:28:37.869 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-09-19T09:28:37.869 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 756056, 2024-09-19T09:28:37.869 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 4185, 2024-09-19T09:28:37.869 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-09-19T09:28:37.869 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-09-19T09:28:37.869 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-09-19T09:28:37.869 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-09-19T09:28:37.869 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.870 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-09-19T09:28:37.870 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-09-19T09:28:37.870 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-09-19T09:28:37.870 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 5118, 2024-09-19T09:28:37.870 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 21, 2024-09-19T09:28:37.870 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 3, 2024-09-19T09:28:37.870 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 5118, 2024-09-19T09:28:37.870 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-09-19T09:28:37.870 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 5118, 2024-09-19T09:28:37.870 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 4.938891762, 2024-09-19T09:28:37.871 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000965004 2024-09-19T09:28:37.871 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.871 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-09-19T09:28:37.871 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-09-19T09:28:37.871 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 5118, 2024-09-19T09:28:37.871 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 5119 2024-09-19T09:28:37.871 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.871 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 5118, 2024-09-19T09:28:37.871 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 5118, 2024-09-19T09:28:37.871 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 0, 2024-09-19T09:28:37.871 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-09-19T09:28:37.872 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-09-19T09:28:37.872 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 3206, 2024-09-19T09:28:37.872 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 0, 2024-09-19T09:28:37.872 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 1910, 2024-09-19T09:28:37.872 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 0, 2024-09-19T09:28:37.872 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 0, 2024-09-19T09:28:37.872 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-09-19T09:28:37.872 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-09-19T09:28:37.872 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-09-19T09:28:37.872 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-09-19T09:28:37.873 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 0, 2024-09-19T09:28:37.873 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-09-19T09:28:37.873 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-09-19T09:28:37.873 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-09-19T09:28:37.873 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 1, 2024-09-19T09:28:37.873 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 0, 2024-09-19T09:28:37.873 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:28:37.873 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-09-19T09:28:37.873 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:28:37.873 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-09-19T09:28:37.873 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-09-19T09:28:37.873 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-09-19T09:28:37.873 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:28:37.874 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-09-19T09:28:37.874 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:28:37.874 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-09-19T09:28:37.874 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-09-19T09:28:37.874 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-09-19T09:28:37.874 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-09-19T09:28:37.874 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-09-19T09:28:37.874 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-09-19T09:28:37.874 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-09-19T09:28:37.874 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-09-19T09:28:37.874 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-09-19T09:28:37.874 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-09-19T09:28:37.875 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-09-19T09:28:37.875 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-09-19T09:28:37.875 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-09-19T09:28:37.875 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-09-19T09:28:37.875 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-09-19T09:28:37.875 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-09-19T09:28:37.875 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-09-19T09:28:37.875 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 85, 2024-09-19T09:28:37.875 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-09-19T09:28:37.875 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 6, 2024-09-19T09:28:37.875 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 5, 2024-09-19T09:28:37.875 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 6, 2024-09-19T09:28:37.876 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 1, 2024-09-19T09:28:37.876 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-09-19T09:28:37.876 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 0, 2024-09-19T09:28:37.876 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 2, 2024-09-19T09:28:37.876 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 0 2024-09-19T09:28:37.876 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.876 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-09-19T09:28:37.876 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 0, 2024-09-19T09:28:37.876 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:28:37.876 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-09-19T09:28:37.876 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-09-19T09:28:37.876 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.877 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-09-19T09:28:37.877 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 0 2024-09-19T09:28:37.877 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.877 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-09-19T09:28:37.877 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-09-19T09:28:37.877 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T09:28:37.877 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-09-19T09:28:37.877 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 0, 2024-09-19T09:28:37.877 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 0, 2024-09-19T09:28:37.877 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 0, 2024-09-19T09:28:37.878 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-09-19T09:28:37.878 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.878 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:28:37.878 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T09:28:37.878 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-19T09:28:37.878 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T09:28:37.878 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 5604, 2024-09-19T09:28:37.878 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 13756125, 2024-09-19T09:28:37.878 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:37.878 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 5604, 2024-09-19T09:28:37.878 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-19T09:28:37.879 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-19T09:28:37.879 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 5604, 2024-09-19T09:28:37.879 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 13756125, 2024-09-19T09:28:37.879 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T09:28:37.879 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.879 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.879 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.879 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:37.879 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.879 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-09-19T09:28:37.879 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T09:28:37.879 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-19T09:28:37.880 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T09:28:37.880 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-19T09:28:37.880 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-19T09:28:37.880 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:37.880 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:37.880 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 5118, 2024-09-19T09:28:37.880 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 11907417, 2024-09-19T09:28:37.880 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 5118, 2024-09-19T09:28:37.880 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 11907417, 2024-09-19T09:28:37.881 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T09:28:37.881 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.881 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.881 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.881 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:37.882 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.882 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-09-19T09:28:37.882 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T09:28:37.882 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-09-19T09:28:37.882 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T09:28:37.882 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-19T09:28:37.882 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-19T09:28:37.882 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:37.882 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:37.882 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 5118, 2024-09-19T09:28:37.882 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 5118, 2024-09-19T09:28:37.883 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 5118, 2024-09-19T09:28:37.883 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 5118, 2024-09-19T09:28:37.883 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T09:28:37.883 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.883 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.883 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.883 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:37.883 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.883 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:28:37.883 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T09:28:37.883 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-19T09:28:37.883 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T09:28:37.884 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-19T09:28:37.884 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-19T09:28:37.884 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:37.884 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:37.884 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-19T09:28:37.884 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-19T09:28:37.884 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-09-19T09:28:37.884 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-09-19T09:28:37.884 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T09:28:37.884 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.884 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.885 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.885 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:37.885 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-19T09:28:37.885 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x560d6d0deb40": { 2024-09-19T09:28:37.885 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-19T09:28:37.885 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-19T09:28:37.885 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-19T09:28:37.885 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-19T09:28:37.885 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-19T09:28:37.885 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-19T09:28:37.885 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-19T09:28:37.885 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-19T09:28:37.886 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-19T09:28:37.886 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-09-19T09:28:37.886 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-09-19T09:28:37.886 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-19T09:28:37.886 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-19T09:28:37.886 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-19T09:28:37.886 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-19T09:28:37.886 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:37.886 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-19T09:28:37.886 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-19T09:28:38.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:38 smithi196 ceph-mon[35150]: pgmap v1118: 97 pgs: 97 active+clean; 263 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 5.4 MiB/s wr, 222 op/s 2024-09-19T09:28:38.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:38 smithi153 ceph-mon[35071]: pgmap v1118: 97 pgs: 97 active+clean; 263 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 5.4 MiB/s wr, 222 op/s 2024-09-19T09:28:38.576 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:38 smithi120 ceph-mon[31150]: pgmap v1118: 97 pgs: 97 active+clean; 263 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 5.4 MiB/s wr, 222 op/s 2024-09-19T09:28:38.647 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-19T09:28:38.647 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-19T09:28:38.647 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-19T09:28:38.659 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-19T09:28:38.659 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-19T09:28:38.659 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T09:28:38.680 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-19T09:28:38.680 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T09:28:38.740 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T09:28:39.034 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:39.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.514+0000 7f87a4504640 1 -- 172.21.15.120:0/3375323057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f879c106f10 msgr2=0x7f879c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:39.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.514+0000 7f87a4504640 1 --2- 172.21.15.120:0/3375323057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f879c106f10 0x7f879c10ea90 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7f8794009ca0 tx=0x7f879402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:28:39.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.514+0000 7f87a4504640 1 -- 172.21.15.120:0/3375323057 shutdown_connections 2024-09-19T09:28:39.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.514+0000 7f87a4504640 1 --2- 172.21.15.120:0/3375323057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f879c106f10 0x7f879c10ea90 unknown :-1 s=CLOSED pgs=2574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:39.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.514+0000 7f87a4504640 1 --2- 172.21.15.120:0/3375323057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f879c102790 0x7f879c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:39.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.514+0000 7f87a4504640 1 --2- 172.21.15.120:0/3375323057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f879c101de0 0x7f879c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:39.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.514+0000 7f87a4504640 1 -- 172.21.15.120:0/3375323057 >> 172.21.15.120:0/3375323057 conn(0x7f879c0fd8c0 msgr2=0x7f879c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:39.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.514+0000 7f87a4504640 1 -- 172.21.15.120:0/3375323057 shutdown_connections 2024-09-19T09:28:39.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.514+0000 7f87a4504640 1 -- 172.21.15.120:0/3375323057 wait complete. 2024-09-19T09:28:39.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.514+0000 7f87a4504640 1 Processor -- start 2024-09-19T09:28:39.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.514+0000 7f87a4504640 1 -- start start 2024-09-19T09:28:39.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.515+0000 7f87a4504640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f879c101de0 0x7f879c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:39.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.515+0000 7f87a4504640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f879c102790 0x7f879c198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:39.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.515+0000 7f87a4504640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f879c106f10 0x7f879c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:39.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.515+0000 7f87a4504640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f879c111f70 con 0x7f879c102790 2024-09-19T09:28:39.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.515+0000 7f87a4504640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f879c111df0 con 0x7f879c106f10 2024-09-19T09:28:39.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.515+0000 7f87a4504640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f879c1120f0 con 0x7f879c101de0 2024-09-19T09:28:39.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.515+0000 7f87a1a9f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f879c102790 0x7f879c198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:39.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.515+0000 7f87a1a9f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f879c102790 0x7f879c198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35290/0 (socket says 172.21.15.120:35290) 2024-09-19T09:28:39.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.515+0000 7f87a1a9f640 1 -- 172.21.15.120:0/3715132835 learned_addr learned my addr 172.21.15.120:0/3715132835 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:39.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.515+0000 7f87a22a0640 1 --2- 172.21.15.120:0/3715132835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f879c101de0 0x7f879c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:39.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.515+0000 7f87a1a9f640 1 -- 172.21.15.120:0/3715132835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f879c101de0 msgr2=0x7f879c1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:39.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.515+0000 7f87a1a9f640 1 --2- 172.21.15.120:0/3715132835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f879c101de0 0x7f879c1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:39.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.515+0000 7f87a1a9f640 1 -- 172.21.15.120:0/3715132835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f879c106f10 msgr2=0x7f879c19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:39.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.515+0000 7f87a2aa1640 1 --2- 172.21.15.120:0/3715132835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f879c106f10 0x7f879c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:39.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.515+0000 7f87a1a9f640 1 --2- 172.21.15.120:0/3715132835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f879c106f10 0x7f879c19cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:39.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.515+0000 7f87a1a9f640 1 -- 172.21.15.120:0/3715132835 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879c19d480 con 0x7f879c102790 2024-09-19T09:28:39.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.515+0000 7f87a1a9f640 1 --2- 172.21.15.120:0/3715132835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f879c102790 0x7f879c198a10 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7f878c00d770 tx=0x7f878c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:39.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.515+0000 7f87937fe640 1 -- 172.21.15.120:0/3715132835 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c017070 con 0x7f879c102790 2024-09-19T09:28:39.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.515+0000 7f87937fe640 1 -- 172.21.15.120:0/3715132835 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f878c005500 con 0x7f879c102790 2024-09-19T09:28:39.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.515+0000 7f87937fe640 1 -- 172.21.15.120:0/3715132835 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c004e00 con 0x7f879c102790 2024-09-19T09:28:39.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.515+0000 7f87a4504640 1 -- 172.21.15.120:0/3715132835 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f879c19d770 con 0x7f879c102790 2024-09-19T09:28:39.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.515+0000 7f87a4504640 1 -- 172.21.15.120:0/3715132835 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f879c1a5050 con 0x7f879c102790 2024-09-19T09:28:39.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.516+0000 7f87a4504640 1 -- 172.21.15.120:0/3715132835 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f879c1021c0 con 0x7f879c102790 2024-09-19T09:28:39.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.517+0000 7f87937fe640 1 -- 172.21.15.120:0/3715132835 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f878c004fa0 con 0x7f879c102790 2024-09-19T09:28:39.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.518+0000 7f87937fe640 1 --2- 172.21.15.120:0/3715132835 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f87880a3240 0x7f87880a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:39.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.518+0000 7f87a22a0640 1 --2- 172.21.15.120:0/3715132835 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f87880a3240 0x7f87880a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:39.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.518+0000 7f87937fe640 1 -- 172.21.15.120:0/3715132835 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f878c0cf3e0 con 0x7f879c102790 2024-09-19T09:28:39.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.519+0000 7f87a22a0640 1 --2- 172.21.15.120:0/3715132835 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f87880a3240 0x7f87880a5700 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8798005fd0 tx=0x7f8798005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:39.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.522+0000 7f87937fe640 1 -- 172.21.15.120:0/3715132835 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f878c09c100 con 0x7f879c102790 2024-09-19T09:28:39.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.699+0000 7f87a4504640 1 -- 172.21.15.120:0/3715132835 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f879c10a150 con 0x7f879c102790 2024-09-19T09:28:39.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.700+0000 7f87937fe640 1 -- 172.21.15.120:0/3715132835 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v85) ==== 81+0+601 (secure 0 0 0) 0x7f878c09ba00 con 0x7f879c102790 2024-09-19T09:28:39.701 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120:0/3010783707 2024-09-20T08:53:51.528772+0000 2024-09-19T09:28:39.702 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120:6801/3562882246 2024-09-20T08:53:51.528772+0000 2024-09-19T09:28:39.702 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120:6800/3562882246 2024-09-20T08:53:51.528772+0000 2024-09-19T09:28:39.702 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120:0/3224512035 2024-09-20T08:53:51.528772+0000 2024-09-19T09:28:39.702 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120:0/3465105835 2024-09-20T08:53:28.144376+0000 2024-09-19T09:28:39.702 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120:0/3317449610 2024-09-20T08:53:28.144376+0000 2024-09-19T09:28:39.702 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120:0/2800697451 2024-09-20T08:53:28.144376+0000 2024-09-19T09:28:39.702 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120:6801/2938446665 2024-09-20T08:53:28.144376+0000 2024-09-19T09:28:39.702 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120:0/109127432 2024-09-20T08:53:51.528772+0000 2024-09-19T09:28:39.702 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120:6800/2938446665 2024-09-20T08:53:28.144376+0000 2024-09-19T09:28:39.702 INFO:teuthology.orchestra.run.smithi120.stderr:listed 10 entries 2024-09-19T09:28:39.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.702+0000 7f87a4504640 1 -- 172.21.15.120:0/3715132835 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f87880a3240 msgr2=0x7f87880a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:39.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.702+0000 7f87a4504640 1 --2- 172.21.15.120:0/3715132835 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f87880a3240 0x7f87880a5700 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8798005fd0 tx=0x7f8798005850 comp rx=0 tx=0).stop 2024-09-19T09:28:39.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.702+0000 7f87a4504640 1 -- 172.21.15.120:0/3715132835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f879c102790 msgr2=0x7f879c198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:39.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.702+0000 7f87a4504640 1 --2- 172.21.15.120:0/3715132835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f879c102790 0x7f879c198a10 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7f878c00d770 tx=0x7f878c00dc30 comp rx=0 tx=0).stop 2024-09-19T09:28:39.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.702+0000 7f87a4504640 1 -- 172.21.15.120:0/3715132835 shutdown_connections 2024-09-19T09:28:39.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.702+0000 7f87a4504640 1 --2- 172.21.15.120:0/3715132835 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f87880a3240 0x7f87880a5700 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:39.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.702+0000 7f87a4504640 1 --2- 172.21.15.120:0/3715132835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f879c106f10 0x7f879c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:39.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.702+0000 7f87a4504640 1 --2- 172.21.15.120:0/3715132835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f879c102790 0x7f879c198a10 unknown :-1 s=CLOSED pgs=2575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:39.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.702+0000 7f87a4504640 1 --2- 172.21.15.120:0/3715132835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f879c101de0 0x7f879c1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:39.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.702+0000 7f87a4504640 1 -- 172.21.15.120:0/3715132835 >> 172.21.15.120:0/3715132835 conn(0x7f879c0fd8c0 msgr2=0x7f879c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:39.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.702+0000 7f87a4504640 1 -- 172.21.15.120:0/3715132835 shutdown_connections 2024-09-19T09:28:39.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:39.702+0000 7f87a4504640 1 -- 172.21.15.120:0/3715132835 wait complete. 2024-09-19T09:28:39.937 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:39 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3715132835' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T09:28:40.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:39 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3715132835' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T09:28:40.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:39 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3715132835' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T09:28:40.367 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-19T09:28:40.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:28:40.368 DEBUG:teuthology.orchestra.run.smithi120:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-19T09:28:40.516 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi120.front.sepia.ceph.com 2024-09-19T09:28:40.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:28:40.516 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T09:28:40.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:28:40.586 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list 2024-09-19T09:28:40.636 INFO:teuthology.orchestra.run.smithi120.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T09:28:40.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:28:40.637 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T09:28:40.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:28:40.712 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link delete ceph-brx 2024-09-19T09:28:40.789 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-19T09:28:40.802 INFO:tasks.cephadm:Teardown begin 2024-09-19T09:28:40.802 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T09:28:40.802 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:28:41.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:40 smithi120 ceph-mon[31150]: pgmap v1119: 97 pgs: 97 active+clean; 263 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 3.6 MiB/s wr, 179 op/s 2024-09-19T09:28:41.187 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:40 smithi196 ceph-mon[35150]: pgmap v1119: 97 pgs: 97 active+clean; 263 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 3.6 MiB/s wr, 179 op/s 2024-09-19T09:28:41.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:40 smithi153 ceph-mon[35071]: pgmap v1119: 97 pgs: 97 active+clean; 263 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 3.6 MiB/s wr, 179 op/s 2024-09-19T09:28:41.238 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:41.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.707+0000 7fe763e09640 1 -- 172.21.15.120:0/2604786541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe75c1008d0 msgr2=0x7fe75c100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:41.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.707+0000 7fe763e09640 1 --2- 172.21.15.120:0/2604786541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe75c1008d0 0x7fe75c100cb0 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7fe754009900 tx=0x7fe75402a410 comp rx=0 tx=0).stop 2024-09-19T09:28:41.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.707+0000 7fe763e09640 1 -- 172.21.15.120:0/2604786541 shutdown_connections 2024-09-19T09:28:41.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.707+0000 7fe763e09640 1 --2- 172.21.15.120:0/2604786541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe75c10e480 0x7fe75c110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:41.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.707+0000 7fe763e09640 1 --2- 172.21.15.120:0/2604786541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe75c101280 0x7fe75c10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:41.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.707+0000 7fe763e09640 1 --2- 172.21.15.120:0/2604786541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe75c1008d0 0x7fe75c100cb0 unknown :-1 s=CLOSED pgs=2576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:41.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.707+0000 7fe763e09640 1 -- 172.21.15.120:0/2604786541 >> 172.21.15.120:0/2604786541 conn(0x7fe75c0fc7a0 msgr2=0x7fe75c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:41.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.707+0000 7fe763e09640 1 -- 172.21.15.120:0/2604786541 shutdown_connections 2024-09-19T09:28:41.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.707+0000 7fe763e09640 1 -- 172.21.15.120:0/2604786541 wait complete. 2024-09-19T09:28:41.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.708+0000 7fe763e09640 1 Processor -- start 2024-09-19T09:28:41.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.708+0000 7fe763e09640 1 -- start start 2024-09-19T09:28:41.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.708+0000 7fe763e09640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe75c1008d0 0x7fe75c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:41.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.708+0000 7fe763e09640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe75c101280 0x7fe75c19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:41.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.708+0000 7fe763e09640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe75c10e480 0x7fe75c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:41.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.708+0000 7fe763e09640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c1141b0 con 0x7fe75c1008d0 2024-09-19T09:28:41.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.708+0000 7fe763e09640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c114030 con 0x7fe75c101280 2024-09-19T09:28:41.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.708+0000 7fe763e09640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c114330 con 0x7fe75c10e480 2024-09-19T09:28:41.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.708+0000 7fe761ba5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe75c1008d0 0x7fe75c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:41.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.708+0000 7fe761ba5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe75c1008d0 0x7fe75c19a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43074/0 (socket says 172.21.15.120:43074) 2024-09-19T09:28:41.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.708+0000 7fe761ba5640 1 -- 172.21.15.120:0/2111226863 learned_addr learned my addr 172.21.15.120:0/2111226863 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:41.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.708+0000 7fe7623a6640 1 --2- 172.21.15.120:0/2111226863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe75c10e480 0x7fe75c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:41.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.708+0000 7fe7613a4640 1 --2- 172.21.15.120:0/2111226863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe75c101280 0x7fe75c19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:41.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.708+0000 7fe761ba5640 1 -- 172.21.15.120:0/2111226863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe75c10e480 msgr2=0x7fe75c19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:41.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.708+0000 7fe761ba5640 1 --2- 172.21.15.120:0/2111226863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe75c10e480 0x7fe75c19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:41.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.708+0000 7fe761ba5640 1 -- 172.21.15.120:0/2111226863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe75c101280 msgr2=0x7fe75c19abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:41.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.708+0000 7fe761ba5640 1 --2- 172.21.15.120:0/2111226863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe75c101280 0x7fe75c19abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:41.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.709+0000 7fe761ba5640 1 -- 172.21.15.120:0/2111226863 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75c19f640 con 0x7fe75c1008d0 2024-09-19T09:28:41.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.709+0000 7fe7623a6640 1 --2- 172.21.15.120:0/2111226863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe75c10e480 0x7fe75c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:41.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.709+0000 7fe761ba5640 1 --2- 172.21.15.120:0/2111226863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe75c1008d0 0x7fe75c19a690 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7fe754006010 tx=0x7fe7540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:41.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.709+0000 7fe75affd640 1 -- 172.21.15.120:0/2111226863 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75403f030 con 0x7fe75c1008d0 2024-09-19T09:28:41.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.709+0000 7fe763e09640 1 -- 172.21.15.120:0/2111226863 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe75c19f8d0 con 0x7fe75c1008d0 2024-09-19T09:28:41.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.709+0000 7fe75affd640 1 -- 172.21.15.120:0/2111226863 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe754003ce0 con 0x7fe75c1008d0 2024-09-19T09:28:41.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.709+0000 7fe75affd640 1 -- 172.21.15.120:0/2111226863 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754035070 con 0x7fe75c1008d0 2024-09-19T09:28:41.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.709+0000 7fe763e09640 1 -- 172.21.15.120:0/2111226863 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe75c1a7210 con 0x7fe75c1008d0 2024-09-19T09:28:41.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.710+0000 7fe763e09640 1 -- 172.21.15.120:0/2111226863 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe75c06b070 con 0x7fe75c1008d0 2024-09-19T09:28:41.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.711+0000 7fe75affd640 1 -- 172.21.15.120:0/2111226863 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe754042030 con 0x7fe75c1008d0 2024-09-19T09:28:41.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.711+0000 7fe75affd640 1 --2- 172.21.15.120:0/2111226863 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe7440a3240 0x7fe7440a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:41.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.712+0000 7fe7613a4640 1 --2- 172.21.15.120:0/2111226863 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe7440a3240 0x7fe7440a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:41.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.712+0000 7fe75affd640 1 -- 172.21.15.120:0/2111226863 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe7540efae0 con 0x7fe75c1008d0 2024-09-19T09:28:41.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.712+0000 7fe7613a4640 1 --2- 172.21.15.120:0/2111226863 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe7440a3240 0x7fe7440a5700 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe75c10ed80 tx=0x7fe750005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:41.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.716+0000 7fe75affd640 1 -- 172.21.15.120:0/2111226863 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe7540bc8c0 con 0x7fe75c1008d0 2024-09-19T09:28:41.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.916+0000 7fe763e09640 1 -- 172.21.15.120:0/2111226863 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe75c109610 con 0x7fe75c1008d0 2024-09-19T09:28:41.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.917+0000 7fe75affd640 1 -- 172.21.15.120:0/2111226863 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11991 (secure 0 0 0) 0x7fe7540bc1c0 con 0x7fe75c1008d0 2024-09-19T09:28:41.918 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:28:41.919 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":42,"btime":"2024-09-19T09:28:13:341277+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:28:13.291163+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:28:41.919 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 42 2024-09-19T09:28:41.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.919+0000 7fe763e09640 1 -- 172.21.15.120:0/2111226863 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe7440a3240 msgr2=0x7fe7440a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:41.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.919+0000 7fe763e09640 1 --2- 172.21.15.120:0/2111226863 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe7440a3240 0x7fe7440a5700 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe75c10ed80 tx=0x7fe750005f90 comp rx=0 tx=0).stop 2024-09-19T09:28:41.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.919+0000 7fe763e09640 1 -- 172.21.15.120:0/2111226863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe75c1008d0 msgr2=0x7fe75c19a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:41.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.919+0000 7fe763e09640 1 --2- 172.21.15.120:0/2111226863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe75c1008d0 0x7fe75c19a690 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7fe754006010 tx=0x7fe7540042f0 comp rx=0 tx=0).stop 2024-09-19T09:28:41.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.919+0000 7fe763e09640 1 -- 172.21.15.120:0/2111226863 shutdown_connections 2024-09-19T09:28:41.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.919+0000 7fe763e09640 1 --2- 172.21.15.120:0/2111226863 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe7440a3240 0x7fe7440a5700 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:41.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.919+0000 7fe763e09640 1 --2- 172.21.15.120:0/2111226863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe75c10e480 0x7fe75c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:41.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.919+0000 7fe763e09640 1 --2- 172.21.15.120:0/2111226863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe75c101280 0x7fe75c19abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:41.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.919+0000 7fe763e09640 1 --2- 172.21.15.120:0/2111226863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe75c1008d0 0x7fe75c19a690 unknown :-1 s=CLOSED pgs=2577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:41.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.919+0000 7fe763e09640 1 -- 172.21.15.120:0/2111226863 >> 172.21.15.120:0/2111226863 conn(0x7fe75c0fc7a0 msgr2=0x7fe75c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:41.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.919+0000 7fe763e09640 1 -- 172.21.15.120:0/2111226863 shutdown_connections 2024-09-19T09:28:41.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:41.919+0000 7fe763e09640 1 -- 172.21.15.120:0/2111226863 wait complete. 2024-09-19T09:28:42.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:41 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2111226863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:28:42.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:41 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2111226863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:28:42.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:41 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2111226863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:28:42.662 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:28:42.914 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:43.221 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:42 smithi120 ceph-mon[31150]: pgmap v1120: 97 pgs: 97 active+clean; 263 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 1.2 MiB/s wr, 117 op/s 2024-09-19T09:28:43.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:42 smithi196 ceph-mon[35150]: pgmap v1120: 97 pgs: 97 active+clean; 263 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 1.2 MiB/s wr, 117 op/s 2024-09-19T09:28:43.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:42 smithi153 ceph-mon[35071]: pgmap v1120: 97 pgs: 97 active+clean; 263 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 1.2 MiB/s wr, 117 op/s 2024-09-19T09:28:43.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.431+0000 7f520f48c640 1 -- 172.21.15.120:0/1013064901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5208103660 msgr2=0x7f520810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:43.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.431+0000 7f520f48c640 1 --2- 172.21.15.120:0/1013064901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5208103660 0x7f520810bb50 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7f51f8009900 tx=0x7f51f802a480 comp rx=0 tx=0).stop 2024-09-19T09:28:43.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.431+0000 7f520f48c640 1 -- 172.21.15.120:0/1013064901 shutdown_connections 2024-09-19T09:28:43.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.431+0000 7f520f48c640 1 --2- 172.21.15.120:0/1013064901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f520810c170 0x7f520810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.431+0000 7f520f48c640 1 --2- 172.21.15.120:0/1013064901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5208103660 0x7f520810bb50 unknown :-1 s=CLOSED pgs=2578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.431+0000 7f520f48c640 1 --2- 172.21.15.120:0/1013064901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5208102d40 0x7f5208103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.431+0000 7f520f48c640 1 -- 172.21.15.120:0/1013064901 >> 172.21.15.120:0/1013064901 conn(0x7f52080fc7a0 msgr2=0x7f52080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:43.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.432+0000 7f520f48c640 1 -- 172.21.15.120:0/1013064901 shutdown_connections 2024-09-19T09:28:43.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.432+0000 7f520f48c640 1 -- 172.21.15.120:0/1013064901 wait complete. 2024-09-19T09:28:43.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.432+0000 7f520f48c640 1 Processor -- start 2024-09-19T09:28:43.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.432+0000 7f520f48c640 1 -- start start 2024-09-19T09:28:43.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.432+0000 7f520f48c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5208102d40 0x7f5208198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:43.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.432+0000 7f520f48c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5208103660 0x7f5208198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:43.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.432+0000 7f520f48c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f520810c170 0x7f520819ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:43.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.432+0000 7f520f48c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5208112030 con 0x7f520810c170 2024-09-19T09:28:43.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.432+0000 7f520f48c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5208111eb0 con 0x7f5208103660 2024-09-19T09:28:43.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.432+0000 7f520f48c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f52081121b0 con 0x7f5208102d40 2024-09-19T09:28:43.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.433+0000 7f520da29640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f520810c170 0x7f520819ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:43.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.433+0000 7f520da29640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f520810c170 0x7f520819ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43094/0 (socket says 172.21.15.120:43094) 2024-09-19T09:28:43.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.433+0000 7f520da29640 1 -- 172.21.15.120:0/1364397697 learned_addr learned my addr 172.21.15.120:0/1364397697 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:43.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.433+0000 7f520ca27640 1 --2- 172.21.15.120:0/1364397697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5208103660 0x7f5208198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:43.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.433+0000 7f520d228640 1 --2- 172.21.15.120:0/1364397697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5208102d40 0x7f5208198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:43.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.433+0000 7f520da29640 1 -- 172.21.15.120:0/1364397697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5208102d40 msgr2=0x7f5208198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:43.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.433+0000 7f520da29640 1 --2- 172.21.15.120:0/1364397697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5208102d40 0x7f5208198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.433+0000 7f520da29640 1 -- 172.21.15.120:0/1364397697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5208103660 msgr2=0x7f5208198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:43.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.433+0000 7f520da29640 1 --2- 172.21.15.120:0/1364397697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5208103660 0x7f5208198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.433+0000 7f520da29640 1 -- 172.21.15.120:0/1364397697 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520819d530 con 0x7f520810c170 2024-09-19T09:28:43.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.433+0000 7f520da29640 1 --2- 172.21.15.120:0/1364397697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f520810c170 0x7f520819ce50 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7f520000b970 tx=0x7f520000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:43.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.433+0000 7f51fe7fc640 1 -- 172.21.15.120:0/1364397697 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5200019070 con 0x7f520810c170 2024-09-19T09:28:43.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.433+0000 7f51fe7fc640 1 -- 172.21.15.120:0/1364397697 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f520000cb40 con 0x7f520810c170 2024-09-19T09:28:43.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.433+0000 7f520f48c640 1 -- 172.21.15.120:0/1364397697 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520819d820 con 0x7f520810c170 2024-09-19T09:28:43.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.433+0000 7f51fe7fc640 1 -- 172.21.15.120:0/1364397697 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5200007650 con 0x7f520810c170 2024-09-19T09:28:43.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.433+0000 7f520f48c640 1 -- 172.21.15.120:0/1364397697 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52081a5100 con 0x7f520810c170 2024-09-19T09:28:43.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.434+0000 7f520f48c640 1 -- 172.21.15.120:0/1364397697 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52081031a0 con 0x7f520810c170 2024-09-19T09:28:43.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.435+0000 7f51fe7fc640 1 -- 172.21.15.120:0/1364397697 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f520001c070 con 0x7f520810c170 2024-09-19T09:28:43.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.436+0000 7f51fe7fc640 1 --2- 172.21.15.120:0/1364397697 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f51f40a3240 0x7f51f40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:43.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.436+0000 7f520d228640 1 --2- 172.21.15.120:0/1364397697 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f51f40a3240 0x7f51f40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:43.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.436+0000 7f51fe7fc640 1 -- 172.21.15.120:0/1364397697 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f52000cf3a0 con 0x7f520810c170 2024-09-19T09:28:43.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.437+0000 7f520d228640 1 --2- 172.21.15.120:0/1364397697 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f51f40a3240 0x7f51f40a5700 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5204009890 tx=0x7f5204008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:43.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.440+0000 7f51fe7fc640 1 -- 172.21.15.120:0/1364397697 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f520009ba30 con 0x7f520810c170 2024-09-19T09:28:43.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.642+0000 7f520f48c640 1 -- 172.21.15.120:0/1364397697 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5208106c50 con 0x7f520810c170 2024-09-19T09:28:43.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.642+0000 7f51fe7fc640 1 -- 172.21.15.120:0/1364397697 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11991 (secure 0 0 0) 0x7f520009fd30 con 0x7f520810c170 2024-09-19T09:28:43.644 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:28:43.644 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":42,"btime":"2024-09-19T09:28:13:341277+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2395347742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":2395347742},{"type":"v1","addr":"172.21.15.120:6835","nonce":2395347742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/217332579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":217332579},{"type":"v1","addr":"172.21.15.120:6837","nonce":217332579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:00:47.659717+0000","modified":"2024-09-19T09:28:13.291163+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24461,"mds_1":24446,"mds_2":24442,"mds_3":24430,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6839/1939272010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1939272010},{"type":"v1","addr":"172.21.15.120:6839","nonce":1939272010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6841/658447478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":658447478},{"type":"v1","addr":"172.21.15.120:6841","nonce":658447478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/1522304272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":1522304272},{"type":"v1","addr":"172.21.15.153:6833","nonce":1522304272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/1407049222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1407049222},{"type":"v1","addr":"172.21.15.196:6833","nonce":1407049222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6835/2141168828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2141168828},{"type":"v1","addr":"172.21.15.196:6835","nonce":2141168828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.196:6837/776889709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":776889709},{"type":"v1","addr":"172.21.15.196:6837","nonce":776889709}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.196:6839/862542635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":862542635},{"type":"v1","addr":"172.21.15.196:6839","nonce":862542635}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.153:6837/3048520124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":3048520124},{"type":"v1","addr":"172.21.15.153:6837","nonce":3048520124}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/3290519661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":3290519661},{"type":"v1","addr":"172.21.15.153:6839","nonce":3290519661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29120":{"gid":29120,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6835/3253383200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3253383200},{"type":"v1","addr":"172.21.15.153:6835","nonce":3253383200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24461,"qdb_cluster":[24421,24430,24442,24446,24461]},"id":1}]} 2024-09-19T09:28:43.645 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 42 2024-09-19T09:28:43.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.644+0000 7f520f48c640 1 -- 172.21.15.120:0/1364397697 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f51f40a3240 msgr2=0x7f51f40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:43.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.644+0000 7f520f48c640 1 --2- 172.21.15.120:0/1364397697 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f51f40a3240 0x7f51f40a5700 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5204009890 tx=0x7f5204008040 comp rx=0 tx=0).stop 2024-09-19T09:28:43.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.644+0000 7f520f48c640 1 -- 172.21.15.120:0/1364397697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f520810c170 msgr2=0x7f520819ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:43.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.644+0000 7f520f48c640 1 --2- 172.21.15.120:0/1364397697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f520810c170 0x7f520819ce50 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7f520000b970 tx=0x7f520000be30 comp rx=0 tx=0).stop 2024-09-19T09:28:43.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.644+0000 7f520f48c640 1 -- 172.21.15.120:0/1364397697 shutdown_connections 2024-09-19T09:28:43.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.644+0000 7f520f48c640 1 --2- 172.21.15.120:0/1364397697 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f51f40a3240 0x7f51f40a5700 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.644+0000 7f520f48c640 1 --2- 172.21.15.120:0/1364397697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f520810c170 0x7f520819ce50 unknown :-1 s=CLOSED pgs=2579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.644+0000 7f520f48c640 1 --2- 172.21.15.120:0/1364397697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5208103660 0x7f5208198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.644+0000 7f520f48c640 1 --2- 172.21.15.120:0/1364397697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5208102d40 0x7f5208198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.644+0000 7f520f48c640 1 -- 172.21.15.120:0/1364397697 >> 172.21.15.120:0/1364397697 conn(0x7f52080fc7a0 msgr2=0x7f52080fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:43.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.645+0000 7f520f48c640 1 -- 172.21.15.120:0/1364397697 shutdown_connections 2024-09-19T09:28:43.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:43.645+0000 7f520f48c640 1 -- 172.21.15.120:0/1364397697 wait complete. 2024-09-19T09:28:44.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:43 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1364397697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:28:44.363 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:28:44.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:43 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1364397697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:28:44.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:43 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1364397697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:28:44.607 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:45.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.100+0000 7f22c059c640 1 -- 172.21.15.120:0/4107111376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22b8101420 msgr2=0x7f22b810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:45.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.100+0000 7f22c059c640 1 --2- 172.21.15.120:0/4107111376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22b8101420 0x7f22b810d950 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7f22b0009930 tx=0x7f22b002a420 comp rx=0 tx=0).stop 2024-09-19T09:28:45.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.100+0000 7f22c059c640 1 -- 172.21.15.120:0/4107111376 shutdown_connections 2024-09-19T09:28:45.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.100+0000 7f22c059c640 1 --2- 172.21.15.120:0/4107111376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22b8101420 0x7f22b810d950 unknown :-1 s=CLOSED pgs=2580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.100+0000 7f22c059c640 1 --2- 172.21.15.120:0/4107111376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22b8100a80 0x7f22b8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.100+0000 7f22c059c640 1 --2- 172.21.15.120:0/4107111376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22b8106a80 0x7f22b8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.100+0000 7f22c059c640 1 -- 172.21.15.120:0/4107111376 >> 172.21.15.120:0/4107111376 conn(0x7f22b80fc7a0 msgr2=0x7f22b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:45.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.100+0000 7f22c059c640 1 -- 172.21.15.120:0/4107111376 shutdown_connections 2024-09-19T09:28:45.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.100+0000 7f22c059c640 1 -- 172.21.15.120:0/4107111376 wait complete. 2024-09-19T09:28:45.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.101+0000 7f22c059c640 1 Processor -- start 2024-09-19T09:28:45.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.101+0000 7f22c059c640 1 -- start start 2024-09-19T09:28:45.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.101+0000 7f22c059c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22b8100a80 0x7f22b819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:45.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.101+0000 7f22c059c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22b8101420 0x7f22b819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:45.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.101+0000 7f22c059c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22b8106a80 0x7f22b819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:45.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.101+0000 7f22c059c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f22b8110e80 con 0x7f22b8101420 2024-09-19T09:28:45.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.101+0000 7f22c059c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f22b8110d00 con 0x7f22b8100a80 2024-09-19T09:28:45.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.101+0000 7f22c059c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f22b8111000 con 0x7f22b8106a80 2024-09-19T09:28:45.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.101+0000 7f22bdb37640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22b8101420 0x7f22b819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:45.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.101+0000 7f22bdb37640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22b8101420 0x7f22b819ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43110/0 (socket says 172.21.15.120:43110) 2024-09-19T09:28:45.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.101+0000 7f22bdb37640 1 -- 172.21.15.120:0/2714696363 learned_addr learned my addr 172.21.15.120:0/2714696363 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:45.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.101+0000 7f22be338640 1 --2- 172.21.15.120:0/2714696363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22b8100a80 0x7f22b819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:45.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.101+0000 7f22beb39640 1 --2- 172.21.15.120:0/2714696363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22b8106a80 0x7f22b819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:45.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.101+0000 7f22bdb37640 1 -- 172.21.15.120:0/2714696363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22b8106a80 msgr2=0x7f22b819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:45.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.101+0000 7f22bdb37640 1 --2- 172.21.15.120:0/2714696363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22b8106a80 0x7f22b819eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.101+0000 7f22bdb37640 1 -- 172.21.15.120:0/2714696363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22b8100a80 msgr2=0x7f22b819a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:45.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.101+0000 7f22bdb37640 1 --2- 172.21.15.120:0/2714696363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22b8100a80 0x7f22b819a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.101+0000 7f22bdb37640 1 -- 172.21.15.120:0/2714696363 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b819f5b0 con 0x7f22b8101420 2024-09-19T09:28:45.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.101+0000 7f22be338640 1 --2- 172.21.15.120:0/2714696363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22b8100a80 0x7f22b819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:45.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.101+0000 7f22bdb37640 1 --2- 172.21.15.120:0/2714696363 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22b8101420 0x7f22b819ab40 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00d950 tx=0x7f22ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:45.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.102+0000 7f22b77fe640 1 -- 172.21.15.120:0/2714696363 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac017070 con 0x7f22b8101420 2024-09-19T09:28:45.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.102+0000 7f22c059c640 1 -- 172.21.15.120:0/2714696363 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22b819f8a0 con 0x7f22b8101420 2024-09-19T09:28:45.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.102+0000 7f22b77fe640 1 -- 172.21.15.120:0/2714696363 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22ac009c30 con 0x7f22b8101420 2024-09-19T09:28:45.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.102+0000 7f22b77fe640 1 -- 172.21.15.120:0/2714696363 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac00f6b0 con 0x7f22b8101420 2024-09-19T09:28:45.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.102+0000 7f22c059c640 1 -- 172.21.15.120:0/2714696363 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22b81a7180 con 0x7f22b8101420 2024-09-19T09:28:45.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.103+0000 7f22c059c640 1 -- 172.21.15.120:0/2714696363 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22b8100ee0 con 0x7f22b8101420 2024-09-19T09:28:45.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.104+0000 7f22b77fe640 1 -- 172.21.15.120:0/2714696363 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f22ac0109d0 con 0x7f22b8101420 2024-09-19T09:28:45.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.104+0000 7f22b77fe640 1 --2- 172.21.15.120:0/2714696363 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f22a00a3240 0x7f22a00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:45.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.105+0000 7f22b77fe640 1 -- 172.21.15.120:0/2714696363 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f22ac0cf5e0 con 0x7f22b8101420 2024-09-19T09:28:45.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.105+0000 7f22be338640 1 --2- 172.21.15.120:0/2714696363 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f22a00a3240 0x7f22a00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:45.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.105+0000 7f22be338640 1 --2- 172.21.15.120:0/2714696363 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f22a00a3240 0x7f22a00a5700 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f22a800a300 tx=0x7f22a800a290 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:45.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.108+0000 7f22b77fe640 1 -- 172.21.15.120:0/2714696363 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f22ac009de0 con 0x7f22b8101420 2024-09-19T09:28:45.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.277+0000 7f22c059c640 1 -- 172.21.15.120:0/2714696363 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22b819b960 con 0x7f22b8101420 2024-09-19T09:28:45.281 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:44 smithi120 ceph-mon[31150]: pgmap v1121: 97 pgs: 97 active+clean; 263 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 54 op/s 2024-09-19T09:28:45.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.278+0000 7f22b77fe640 1 -- 172.21.15.120:0/2714696363 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22844 (secure 0 0 0) 0x7f22b819b960 con 0x7f22b8101420 2024-09-19T09:28:45.282 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-19T09:28:45.282 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":85,"fsid":"507393d2-7664-11ef-bceb-c7b262605968","created":"2024-09-19T08:53:06.386526+0000","modified":"2024-09-19T09:16:31.379026+0000","last_up_change":"2024-09-19T08:59:55.692659+0000","last_in_change":"2024-09-19T08:59:32.808860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:56:07.830047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:00:43.774339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:00:45.786262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33ec3993-576c-428c-bf75-adcb17cd72a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6803","nonce":4131180019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6805","nonce":4131180019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6809","nonce":4131180019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4131180019},{"type":"v1","addr":"172.21.15.120:6807","nonce":4131180019}]},"public_addr":"172.21.15.120:6803/4131180019","cluster_addr":"172.21.15.120:6805/4131180019","heartbeat_back_addr":"172.21.15.120:6809/4131180019","heartbeat_front_addr":"172.21.15.120:6807/4131180019","state":["exists","up"]},{"osd":1,"uuid":"450bbebf-2aac-47c6-8474-bbd37fd0a1a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6801","nonce":2259730294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6803","nonce":2259730294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6807","nonce":2259730294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2259730294},{"type":"v1","addr":"172.21.15.153:6805","nonce":2259730294}]},"public_addr":"172.21.15.153:6801/2259730294","cluster_addr":"172.21.15.153:6803/2259730294","heartbeat_back_addr":"172.21.15.153:6807/2259730294","heartbeat_front_addr":"172.21.15.153:6805/2259730294","state":["exists","up"]},{"osd":2,"uuid":"ac8c4b83-8e8e-4803-b266-262f7873bd85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6801","nonce":2942286570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6803","nonce":2942286570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6807","nonce":2942286570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2942286570},{"type":"v1","addr":"172.21.15.196:6805","nonce":2942286570}]},"public_addr":"172.21.15.196:6801/2942286570","cluster_addr":"172.21.15.196:6803/2942286570","heartbeat_back_addr":"172.21.15.196:6807/2942286570","heartbeat_front_addr":"172.21.15.196:6805/2942286570","state":["exists","up"]},{"osd":3,"uuid":"c1d9e507-8f02-403e-87da-9b4966fd8fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6811","nonce":713205668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6813","nonce":713205668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6817","nonce":713205668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":713205668},{"type":"v1","addr":"172.21.15.120:6815","nonce":713205668}]},"public_addr":"172.21.15.120:6811/713205668","cluster_addr":"172.21.15.120:6813/713205668","heartbeat_back_addr":"172.21.15.120:6817/713205668","heartbeat_front_addr":"172.21.15.120:6815/713205668","state":["exists","up"]},{"osd":4,"uuid":"7f8bf3b9-0ec4-46f6-a406-d26d7971179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6809","nonce":2680865854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6811","nonce":2680865854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6815","nonce":2680865854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2680865854},{"type":"v1","addr":"172.21.15.153:6813","nonce":2680865854}]},"public_addr":"172.21.15.153:6809/2680865854","cluster_addr":"172.21.15.153:6811/2680865854","heartbeat_back_addr":"172.21.15.153:6815/2680865854","heartbeat_front_addr":"172.21.15.153:6813/2680865854","state":["exists","up"]},{"osd":5,"uuid":"70b270b2-209d-4ff5-875a-04ee7b992a93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6809","nonce":3763331116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6811","nonce":3763331116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6815","nonce":3763331116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":3763331116},{"type":"v1","addr":"172.21.15.196:6813","nonce":3763331116}]},"public_addr":"172.21.15.196:6809/3763331116","cluster_addr":"172.21.15.196:6811/3763331116","heartbeat_back_addr":"172.21.15.196:6815/3763331116","heartbeat_front_addr":"172.21.15.196:6813/3763331116","state":["exists","up"]},{"osd":6,"uuid":"8087f69c-dc31-4522-a18d-4e6ab1b7cb27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6819","nonce":4171773765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6821","nonce":4171773765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6825","nonce":4171773765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":4171773765},{"type":"v1","addr":"172.21.15.120:6823","nonce":4171773765}]},"public_addr":"172.21.15.120:6819/4171773765","cluster_addr":"172.21.15.120:6821/4171773765","heartbeat_back_addr":"172.21.15.120:6825/4171773765","heartbeat_front_addr":"172.21.15.120:6823/4171773765","state":["exists","up"]},{"osd":7,"uuid":"c3b1ead3-150f-43fd-8924-f639203f4cdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6817","nonce":2535148453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6819","nonce":2535148453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6823","nonce":2535148453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2535148453},{"type":"v1","addr":"172.21.15.153:6821","nonce":2535148453}]},"public_addr":"172.21.15.153:6817/2535148453","cluster_addr":"172.21.15.153:6819/2535148453","heartbeat_back_addr":"172.21.15.153:6823/2535148453","heartbeat_front_addr":"172.21.15.153:6821/2535148453","state":["exists","up"]},{"osd":8,"uuid":"a3273a2a-968e-4e93-a9cf-3879892bdeab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6817","nonce":186195906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6819","nonce":186195906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6823","nonce":186195906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":186195906},{"type":"v1","addr":"172.21.15.196:6821","nonce":186195906}]},"public_addr":"172.21.15.196:6817/186195906","cluster_addr":"172.21.15.196:6819/186195906","heartbeat_back_addr":"172.21.15.196:6823/186195906","heartbeat_front_addr":"172.21.15.196:6821/186195906","state":["exists","up"]},{"osd":9,"uuid":"6d184513-dab8-45f7-bc08-fa2e8b2f10b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6827","nonce":1559320883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6829","nonce":1559320883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6833","nonce":1559320883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1559320883},{"type":"v1","addr":"172.21.15.120:6831","nonce":1559320883}]},"public_addr":"172.21.15.120:6827/1559320883","cluster_addr":"172.21.15.120:6829/1559320883","heartbeat_back_addr":"172.21.15.120:6833/1559320883","heartbeat_front_addr":"172.21.15.120:6831/1559320883","state":["exists","up"]},{"osd":10,"uuid":"797db886-62f9-4316-a45b-4d50512e65d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6825","nonce":3216877560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6827","nonce":3216877560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6831","nonce":3216877560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3216877560},{"type":"v1","addr":"172.21.15.153:6829","nonce":3216877560}]},"public_addr":"172.21.15.153:6825/3216877560","cluster_addr":"172.21.15.153:6827/3216877560","heartbeat_back_addr":"172.21.15.153:6831/3216877560","heartbeat_front_addr":"172.21.15.153:6829/3216877560","state":["exists","up"]},{"osd":11,"uuid":"17e0afaa-84eb-4b26-9810-bfa3cc7e0f3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6825","nonce":1857200740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6827","nonce":1857200740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6831","nonce":1857200740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1857200740},{"type":"v1","addr":"172.21.15.196:6829","nonce":1857200740}]},"public_addr":"172.21.15.196:6825/1857200740","cluster_addr":"172.21.15.196:6827/1857200740","heartbeat_back_addr":"172.21.15.196:6831/1857200740","heartbeat_front_addr":"172.21.15.196:6829/1857200740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:55:15.595645+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:55:40.405487+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:04.938034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:29.718696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:56:55.636029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:57:21.257083+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:57:46.316552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:58:11.946863+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:58:37.098578+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:03.759023+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:28.659049+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:59:54.350475+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:0/3010783707":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6801/3562882246":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6800/3562882246":"2024-09-20T08:53:51.528772+0000","172.21.15.120:0/3224512035":"2024-09-20T08:53:51.528772+0000","172.21.15.120:0/3465105835":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/3317449610":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/2800697451":"2024-09-20T08:53:28.144376+0000","172.21.15.120:6801/2938446665":"2024-09-20T08:53:28.144376+0000","172.21.15.120:0/109127432":"2024-09-20T08:53:51.528772+0000","172.21.15.120:6800/2938446665":"2024-09-20T08:53:28.144376+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:28:45.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.281+0000 7f22c059c640 1 -- 172.21.15.120:0/2714696363 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f22a00a3240 msgr2=0x7f22a00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:45.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.281+0000 7f22c059c640 1 --2- 172.21.15.120:0/2714696363 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f22a00a3240 0x7f22a00a5700 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f22a800a300 tx=0x7f22a800a290 comp rx=0 tx=0).stop 2024-09-19T09:28:45.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.281+0000 7f22c059c640 1 -- 172.21.15.120:0/2714696363 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22b8101420 msgr2=0x7f22b819ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:45.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.281+0000 7f22c059c640 1 --2- 172.21.15.120:0/2714696363 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22b8101420 0x7f22b819ab40 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00d950 tx=0x7f22ac00de10 comp rx=0 tx=0).stop 2024-09-19T09:28:45.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.281+0000 7f22c059c640 1 -- 172.21.15.120:0/2714696363 shutdown_connections 2024-09-19T09:28:45.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.281+0000 7f22c059c640 1 --2- 172.21.15.120:0/2714696363 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f22a00a3240 0x7f22a00a5700 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.281+0000 7f22c059c640 1 --2- 172.21.15.120:0/2714696363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22b8106a80 0x7f22b819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.281+0000 7f22c059c640 1 --2- 172.21.15.120:0/2714696363 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22b8101420 0x7f22b819ab40 unknown :-1 s=CLOSED pgs=2581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.281+0000 7f22c059c640 1 --2- 172.21.15.120:0/2714696363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22b8100a80 0x7f22b819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.281+0000 7f22c059c640 1 -- 172.21.15.120:0/2714696363 >> 172.21.15.120:0/2714696363 conn(0x7f22b80fc7a0 msgr2=0x7f22b806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:45.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.281+0000 7f22c059c640 1 -- 172.21.15.120:0/2714696363 shutdown_connections 2024-09-19T09:28:45.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:45.281+0000 7f22c059c640 1 -- 172.21.15.120:0/2714696363 wait complete. 2024-09-19T09:28:45.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:44 smithi196 ceph-mon[35150]: pgmap v1121: 97 pgs: 97 active+clean; 263 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 54 op/s 2024-09-19T09:28:45.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:44 smithi153 ceph-mon[35071]: pgmap v1121: 97 pgs: 97 active+clean; 263 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 54 op/s 2024-09-19T09:28:46.006 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs fail cephfs 2024-09-19T09:28:46.256 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:46.292 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:45 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/2714696363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:28:46.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:45 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/2714696363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:28:46.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:45 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/2714696363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:28:46.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.787+0000 7fe39c037640 1 -- 172.21.15.120:0/2039666520 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe394100e20 msgr2=0x7fe394101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:46.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.788+0000 7fe39c037640 1 --2- 172.21.15.120:0/2039666520 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe394100e20 0x7fe394101200 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7fe390009900 tx=0x7fe39002a480 comp rx=0 tx=0).stop 2024-09-19T09:28:46.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.788+0000 7fe39c037640 1 -- 172.21.15.120:0/2039666520 shutdown_connections 2024-09-19T09:28:46.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.788+0000 7fe39c037640 1 --2- 172.21.15.120:0/2039666520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe394102ef0 0x7fe39410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:46.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.788+0000 7fe39c037640 1 --2- 172.21.15.120:0/2039666520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe394102570 0x7fe3941029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:46.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.788+0000 7fe39c037640 1 --2- 172.21.15.120:0/2039666520 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe394100e20 0x7fe394101200 unknown :-1 s=CLOSED pgs=2582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:46.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.788+0000 7fe39c037640 1 -- 172.21.15.120:0/2039666520 >> 172.21.15.120:0/2039666520 conn(0x7fe3940fc7d0 msgr2=0x7fe3940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:46.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.788+0000 7fe39c037640 1 -- 172.21.15.120:0/2039666520 shutdown_connections 2024-09-19T09:28:46.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.788+0000 7fe39c037640 1 -- 172.21.15.120:0/2039666520 wait complete. 2024-09-19T09:28:46.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.788+0000 7fe39c037640 1 Processor -- start 2024-09-19T09:28:46.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.788+0000 7fe39c037640 1 -- start start 2024-09-19T09:28:46.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.788+0000 7fe39c037640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe394100e20 0x7fe394198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:46.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.788+0000 7fe39c037640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe394102570 0x7fe394198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:46.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.788+0000 7fe39c037640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe394102ef0 0x7fe39419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:46.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.788+0000 7fe39c037640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe394110fa0 con 0x7fe394100e20 2024-09-19T09:28:46.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.788+0000 7fe39c037640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe394110e20 con 0x7fe394102570 2024-09-19T09:28:46.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.788+0000 7fe39c037640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe394111120 con 0x7fe394102ef0 2024-09-19T09:28:46.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.789+0000 7fe399dd3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe394100e20 0x7fe394198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:46.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.789+0000 7fe399dd3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe394100e20 0x7fe394198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43132/0 (socket says 172.21.15.120:43132) 2024-09-19T09:28:46.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.789+0000 7fe399dd3640 1 -- 172.21.15.120:0/1606760626 learned_addr learned my addr 172.21.15.120:0/1606760626 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:46.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.789+0000 7fe3995d2640 1 --2- 172.21.15.120:0/1606760626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe394102570 0x7fe394198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:46.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.789+0000 7fe39a5d4640 1 --2- 172.21.15.120:0/1606760626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe394102ef0 0x7fe39419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:46.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.789+0000 7fe399dd3640 1 -- 172.21.15.120:0/1606760626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe394102ef0 msgr2=0x7fe39419ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:46.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.789+0000 7fe399dd3640 1 --2- 172.21.15.120:0/1606760626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe394102ef0 0x7fe39419ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:46.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.789+0000 7fe399dd3640 1 -- 172.21.15.120:0/1606760626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe394102570 msgr2=0x7fe394198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:46.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.789+0000 7fe399dd3640 1 --2- 172.21.15.120:0/1606760626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe394102570 0x7fe394198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:46.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.789+0000 7fe399dd3640 1 -- 172.21.15.120:0/1606760626 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39419d4e0 con 0x7fe394100e20 2024-09-19T09:28:46.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.789+0000 7fe3995d2640 1 --2- 172.21.15.120:0/1606760626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe394102570 0x7fe394198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:46.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.789+0000 7fe399dd3640 1 --2- 172.21.15.120:0/1606760626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe394100e20 0x7fe394198530 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7fe390006010 tx=0x7fe390003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:46.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.789+0000 7fe38affd640 1 -- 172.21.15.120:0/1606760626 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39003f070 con 0x7fe394100e20 2024-09-19T09:28:46.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.789+0000 7fe38affd640 1 -- 172.21.15.120:0/1606760626 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe390031c30 con 0x7fe394100e20 2024-09-19T09:28:46.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.789+0000 7fe39c037640 1 -- 172.21.15.120:0/1606760626 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe39419d770 con 0x7fe394100e20 2024-09-19T09:28:46.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.789+0000 7fe38affd640 1 -- 172.21.15.120:0/1606760626 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390035030 con 0x7fe394100e20 2024-09-19T09:28:46.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.789+0000 7fe39c037640 1 -- 172.21.15.120:0/1606760626 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3941a50b0 con 0x7fe394100e20 2024-09-19T09:28:46.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.790+0000 7fe39c037640 1 -- 172.21.15.120:0/1606760626 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe394101200 con 0x7fe394100e20 2024-09-19T09:28:46.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.791+0000 7fe38affd640 1 -- 172.21.15.120:0/1606760626 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe390042070 con 0x7fe394100e20 2024-09-19T09:28:46.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.792+0000 7fe38affd640 1 --2- 172.21.15.120:0/1606760626 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe3800a3240 0x7fe3800a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:46.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.792+0000 7fe3995d2640 1 --2- 172.21.15.120:0/1606760626 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe3800a3240 0x7fe3800a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:46.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.792+0000 7fe38affd640 1 -- 172.21.15.120:0/1606760626 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe3900efec0 con 0x7fe394100e20 2024-09-19T09:28:46.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.792+0000 7fe3995d2640 1 --2- 172.21.15.120:0/1606760626 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe3800a3240 0x7fe3800a5700 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe3840058c0 tx=0x7fe384005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:46.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:46.796+0000 7fe38affd640 1 -- 172.21.15.120:0/1606760626 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe390031de0 con 0x7fe394100e20 2024-09-19T09:28:47.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:47.001+0000 7fe39c037640 1 -- 172.21.15.120:0/1606760626 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe39410a0a0 con 0x7fe394100e20 2024-09-19T09:28:47.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:47.003+0000 7fe38affd640 1 -- 172.21.15.120:0/1606760626 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v43) ==== 161+0+0 (secure 0 0 0) 0x7fe3900bcca0 con 0x7fe394100e20 2024-09-19T09:28:47.004 INFO:teuthology.orchestra.run.smithi120.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T09:28:47.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:47.006+0000 7fe39c037640 1 -- 172.21.15.120:0/1606760626 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe3800a3240 msgr2=0x7fe3800a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:47.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:47.006+0000 7fe39c037640 1 --2- 172.21.15.120:0/1606760626 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe3800a3240 0x7fe3800a5700 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe3840058c0 tx=0x7fe384005850 comp rx=0 tx=0).stop 2024-09-19T09:28:47.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:47.006+0000 7fe39c037640 1 -- 172.21.15.120:0/1606760626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe394100e20 msgr2=0x7fe394198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:47.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:47.006+0000 7fe39c037640 1 --2- 172.21.15.120:0/1606760626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe394100e20 0x7fe394198530 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7fe390006010 tx=0x7fe390003c10 comp rx=0 tx=0).stop 2024-09-19T09:28:47.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:47.007+0000 7fe39c037640 1 -- 172.21.15.120:0/1606760626 shutdown_connections 2024-09-19T09:28:47.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:47.007+0000 7fe39c037640 1 --2- 172.21.15.120:0/1606760626 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fe3800a3240 0x7fe3800a5700 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:47.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:47.007+0000 7fe39c037640 1 --2- 172.21.15.120:0/1606760626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe394102ef0 0x7fe39419ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:47.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:47.007+0000 7fe39c037640 1 --2- 172.21.15.120:0/1606760626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe394102570 0x7fe394198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:47.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:47.007+0000 7fe39c037640 1 --2- 172.21.15.120:0/1606760626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe394100e20 0x7fe394198530 unknown :-1 s=CLOSED pgs=2583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:47.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:47.007+0000 7fe39c037640 1 -- 172.21.15.120:0/1606760626 >> 172.21.15.120:0/1606760626 conn(0x7fe3940fc7d0 msgr2=0x7fe39406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:47.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:47.007+0000 7fe39c037640 1 -- 172.21.15.120:0/1606760626 shutdown_connections 2024-09-19T09:28:47.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:47.007+0000 7fe39c037640 1 -- 172.21.15.120:0/1606760626 wait complete. 2024-09-19T09:28:47.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:46 smithi120 ceph-mon[31150]: pgmap v1122: 97 pgs: 97 active+clean; 263 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-09-19T09:28:47.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:47 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-a[31146]: 2024-09-19T09:28:47.002+0000 7f8eba030640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T09:28:47.326 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 19 09:28:47 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-g[71844]: ignoring --setuser ceph since I am not root 2024-09-19T09:28:47.326 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 19 09:28:47 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-g[71844]: ignoring --setgroup ceph since I am not root 2024-09-19T09:28:47.327 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 19 09:28:47 smithi120 ceph-mds[71848]: -- 172.21.15.120:0/1031051674 <== mon.0 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c994fd5c20 con 0x55c994fb2800 2024-09-19T09:28:47.327 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 19 09:28:47 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-g[71844]: starting mds.g at 2024-09-19T09:28:47.327 INFO:journalctl@ceph.mds.a.smithi120.stdout:Sep 19 09:28:47 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-a[72252]: starting mds.a at 2024-09-19T09:28:47.437 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 09:28:47 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-f[64721]: ignoring --setuser ceph since I am not root 2024-09-19T09:28:47.437 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 09:28:47 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-f[64721]: ignoring --setgroup ceph since I am not root 2024-09-19T09:28:47.437 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 09:28:47 smithi196 ceph-mds[64725]: -- 172.21.15.196:0/2652195861 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561775a7d860 con 0x561775a5b000 2024-09-19T09:28:47.437 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 09:28:47 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-f[64721]: starting mds.f at 2024-09-19T09:28:47.438 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 09:28:47 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-c[65022]: ignoring --setuser ceph since I am not root 2024-09-19T09:28:47.438 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 09:28:47 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-c[65022]: ignoring --setgroup ceph since I am not root 2024-09-19T09:28:47.438 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 09:28:47 smithi196 ceph-mds[65026]: -- 172.21.15.196:0/1617556883 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e53023c680 con 0x55e530211000 2024-09-19T09:28:47.438 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 09:28:47 smithi196 ceph-mds[65026]: -- 172.21.15.196:0/1617556883 <== mon.2 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e530233860 con 0x55e530211000 2024-09-19T09:28:47.438 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 09:28:47 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-c[65022]: starting mds.c at 2024-09-19T09:28:47.438 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 09:28:47 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-l[64418]: ignoring --setuser ceph since I am not root 2024-09-19T09:28:47.438 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 09:28:47 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-l[64418]: ignoring --setgroup ceph since I am not root 2024-09-19T09:28:47.438 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 09:28:47 smithi196 ceph-mds[64422]: -- 172.21.15.196:0/3103315594 <== mon.2 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562ac45ff860 con 0x562ac45dd400 2024-09-19T09:28:47.439 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 09:28:47 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-l[64418]: starting mds.l at 2024-09-19T09:28:47.439 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 09:28:47 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-i[65325]: ignoring --setuser ceph since I am not root 2024-09-19T09:28:47.439 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 09:28:47 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-i[65325]: ignoring --setgroup ceph since I am not root 2024-09-19T09:28:47.439 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 09:28:47 smithi196 ceph-mds[65329]: -- 172.21.15.196:0/2783207288 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55fee3be8680 con 0x55fee3bbd000 2024-09-19T09:28:47.439 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 09:28:47 smithi196 ceph-mds[65329]: -- 172.21.15.196:0/2783207288 <== mon.0 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fee3bdf860 con 0x55fee3bbd000 2024-09-19T09:28:47.439 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 09:28:47 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-i[65325]: starting mds.i at 2024-09-19T09:28:47.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:46 smithi196 ceph-mon[35150]: pgmap v1122: 97 pgs: 97 active+clean; 263 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-09-19T09:28:47.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:46 smithi153 ceph-mon[35071]: pgmap v1122: 97 pgs: 97 active+clean; 263 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-09-19T09:28:47.440 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:28:47 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-e[63911]: ignoring --setuser ceph since I am not root 2024-09-19T09:28:47.440 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:28:47 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-e[63911]: ignoring --setgroup ceph since I am not root 2024-09-19T09:28:47.440 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:28:47 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-e[63911]: starting mds.e at 2024-09-19T09:28:47.440 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 19 09:28:47 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-b[64524]: ignoring --setuser ceph since I am not root 2024-09-19T09:28:47.441 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 19 09:28:47 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-b[64524]: ignoring --setgroup ceph since I am not root 2024-09-19T09:28:47.441 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 19 09:28:47 smithi153 ceph-mds[64528]: -- 172.21.15.153:0/2585332401 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c15247f860 con 0x55c15245d000 2024-09-19T09:28:47.441 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 19 09:28:47 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-b[64524]: starting mds.b at 2024-09-19T09:28:47.441 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 19 09:28:47 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-k[63597]: ignoring --setuser ceph since I am not root 2024-09-19T09:28:47.441 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 19 09:28:47 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-k[63597]: ignoring --setgroup ceph since I am not root 2024-09-19T09:28:47.441 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 19 09:28:47 smithi153 ceph-mds[63601]: -- 172.21.15.153:0/98337591 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5634ffb1e680 con 0x5634ffaf2800 2024-09-19T09:28:47.441 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 19 09:28:47 smithi153 ceph-mds[63601]: -- 172.21.15.153:0/98337591 <== mon.2 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5634ffb15860 con 0x5634ffaf2800 2024-09-19T09:28:47.442 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 19 09:28:47 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-k[63597]: starting mds.k at 2024-09-19T09:28:47.442 INFO:journalctl@ceph.mds.h.smithi153.stdout:Sep 19 09:28:47 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-h[64207]: ignoring --setuser ceph since I am not root 2024-09-19T09:28:47.442 INFO:journalctl@ceph.mds.h.smithi153.stdout:Sep 19 09:28:47 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-h[64207]: ignoring --setgroup ceph since I am not root 2024-09-19T09:28:47.442 INFO:journalctl@ceph.mds.h.smithi153.stdout:Sep 19 09:28:47 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-h[64207]: starting mds.h at 2024-09-19T09:28:47.732 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T09:28:47.980 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:48.256 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:47 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1606760626' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T09:28:48.256 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:47 smithi120 ceph-mon[31150]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T09:28:48.256 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:47 smithi120 ceph-mon[31150]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T09:28:48.257 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:47 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1606760626' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T09:28:48.257 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:47 smithi120 ceph-mon[31150]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T09:28:48.257 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:47 smithi120 ceph-mon[31150]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T09:28:48.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:47 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1606760626' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T09:28:48.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:47 smithi196 ceph-mon[35150]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T09:28:48.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:47 smithi196 ceph-mon[35150]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T09:28:48.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:47 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1606760626' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T09:28:48.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:47 smithi196 ceph-mon[35150]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T09:28:48.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:47 smithi196 ceph-mon[35150]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T09:28:48.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:47 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1606760626' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T09:28:48.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:47 smithi153 ceph-mon[35071]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T09:28:48.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:47 smithi153 ceph-mon[35071]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T09:28:48.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:47 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1606760626' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T09:28:48.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:47 smithi153 ceph-mon[35071]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T09:28:48.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:47 smithi153 ceph-mon[35071]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T09:28:48.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.502+0000 7fbaff5f7640 1 -- 172.21.15.120:0/3860097343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaf8101de0 msgr2=0x7fbaf81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:48.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.502+0000 7fbaff5f7640 1 --2- 172.21.15.120:0/3860097343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaf8101de0 0x7fbaf81021c0 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009900 tx=0x7fbaf402a410 comp rx=0 tx=0).stop 2024-09-19T09:28:48.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.502+0000 7fbaff5f7640 1 -- 172.21.15.120:0/3860097343 shutdown_connections 2024-09-19T09:28:48.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.502+0000 7fbaff5f7640 1 --2- 172.21.15.120:0/3860097343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaf8106f10 0x7fbaf810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:48.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.502+0000 7fbaff5f7640 1 --2- 172.21.15.120:0/3860097343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaf8102790 0x7fbaf81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:48.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.502+0000 7fbaff5f7640 1 --2- 172.21.15.120:0/3860097343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaf8101de0 0x7fbaf81021c0 unknown :-1 s=CLOSED pgs=2593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:48.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.502+0000 7fbaff5f7640 1 -- 172.21.15.120:0/3860097343 >> 172.21.15.120:0/3860097343 conn(0x7fbaf80fd8c0 msgr2=0x7fbaf80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:48.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.503+0000 7fbaff5f7640 1 -- 172.21.15.120:0/3860097343 shutdown_connections 2024-09-19T09:28:48.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.503+0000 7fbaff5f7640 1 -- 172.21.15.120:0/3860097343 wait complete. 2024-09-19T09:28:48.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.503+0000 7fbaff5f7640 1 Processor -- start 2024-09-19T09:28:48.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.503+0000 7fbaff5f7640 1 -- start start 2024-09-19T09:28:48.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.504+0000 7fbaff5f7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaf8101de0 0x7fbaf80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:48.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.504+0000 7fbaff5f7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaf8102790 0x7fbaf8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:48.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.504+0000 7fbaff5f7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaf8106f10 0x7fbaf8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:48.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.504+0000 7fbaff5f7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf8111f30 con 0x7fbaf8101de0 2024-09-19T09:28:48.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.504+0000 7fbaff5f7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf8111db0 con 0x7fbaf8102790 2024-09-19T09:28:48.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.504+0000 7fbaff5f7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf81120b0 con 0x7fbaf8106f10 2024-09-19T09:28:48.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.504+0000 7fbafd393640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaf8101de0 0x7fbaf80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:48.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.504+0000 7fbafd393640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaf8101de0 0x7fbaf80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43180/0 (socket says 172.21.15.120:43180) 2024-09-19T09:28:48.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.504+0000 7fbafd393640 1 -- 172.21.15.120:0/3257440883 learned_addr learned my addr 172.21.15.120:0/3257440883 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:48.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.504+0000 7fbafcb92640 1 --2- 172.21.15.120:0/3257440883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaf8102790 0x7fbaf8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:48.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.504+0000 7fbafdb94640 1 --2- 172.21.15.120:0/3257440883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaf8106f10 0x7fbaf8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:48.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.504+0000 7fbafd393640 1 -- 172.21.15.120:0/3257440883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaf8106f10 msgr2=0x7fbaf8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:48.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.504+0000 7fbafd393640 1 --2- 172.21.15.120:0/3257440883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaf8106f10 0x7fbaf8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:48.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.504+0000 7fbafd393640 1 -- 172.21.15.120:0/3257440883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaf8102790 msgr2=0x7fbaf8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:48.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.504+0000 7fbafd393640 1 --2- 172.21.15.120:0/3257440883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaf8102790 0x7fbaf8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:48.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.504+0000 7fbafd393640 1 -- 172.21.15.120:0/3257440883 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf81a6910 con 0x7fbaf8101de0 2024-09-19T09:28:48.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.504+0000 7fbafd393640 1 --2- 172.21.15.120:0/3257440883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaf8101de0 0x7fbaf80744f0 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4006010 tx=0x7fbaf40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:48.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.504+0000 7fbaee7fc640 1 -- 172.21.15.120:0/3257440883 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf403f030 con 0x7fbaf8101de0 2024-09-19T09:28:48.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.504+0000 7fbaff5f7640 1 -- 172.21.15.120:0/3257440883 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf81a6ba0 con 0x7fbaf8101de0 2024-09-19T09:28:48.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.504+0000 7fbaee7fc640 1 -- 172.21.15.120:0/3257440883 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaf4003ce0 con 0x7fbaf8101de0 2024-09-19T09:28:48.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.505+0000 7fbaee7fc640 1 -- 172.21.15.120:0/3257440883 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4035070 con 0x7fbaf8101de0 2024-09-19T09:28:48.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.505+0000 7fbaff5f7640 1 -- 172.21.15.120:0/3257440883 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf81a70d0 con 0x7fbaf8101de0 2024-09-19T09:28:48.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.505+0000 7fbaff5f7640 1 -- 172.21.15.120:0/3257440883 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaf81021c0 con 0x7fbaf8101de0 2024-09-19T09:28:48.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.507+0000 7fbaee7fc640 1 -- 172.21.15.120:0/3257440883 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fbaf4042030 con 0x7fbaf8101de0 2024-09-19T09:28:48.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.508+0000 7fbaee7fc640 1 --2- 172.21.15.120:0/3257440883 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbae40a3240 0x7fbae40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:48.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.508+0000 7fbafcb92640 1 --2- 172.21.15.120:0/3257440883 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbae40a3240 0x7fbae40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:48.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.508+0000 7fbaee7fc640 1 -- 172.21.15.120:0/3257440883 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8670+0+0 (secure 0 0 0) 0x7fbaf40efcd0 con 0x7fbaf8101de0 2024-09-19T09:28:48.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.508+0000 7fbafcb92640 1 --2- 172.21.15.120:0/3257440883 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbae40a3240 0x7fbae40a5700 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8075f80 tx=0x7fbae800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:48.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.512+0000 7fbaee7fc640 1 -- 172.21.15.120:0/3257440883 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbaf40bc840 con 0x7fbaf8101de0 2024-09-19T09:28:48.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:48.712+0000 7fbaff5f7640 1 -- 172.21.15.120:0/3257440883 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fbaf810a0f0 con 0x7fbaf8101de0 2024-09-19T09:28:49.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:49.012+0000 7fbaee7fc640 1 -- 172.21.15.120:0/3257440883 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v45) ==== 104+0+0 (secure 0 0 0) 0x7fbaf40bc140 con 0x7fbaf8101de0 2024-09-19T09:28:49.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:49.014+0000 7fbaff5f7640 1 -- 172.21.15.120:0/3257440883 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbae40a3240 msgr2=0x7fbae40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:49.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:49.014+0000 7fbaff5f7640 1 --2- 172.21.15.120:0/3257440883 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbae40a3240 0x7fbae40a5700 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8075f80 tx=0x7fbae800b390 comp rx=0 tx=0).stop 2024-09-19T09:28:49.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:49.014+0000 7fbaff5f7640 1 -- 172.21.15.120:0/3257440883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaf8101de0 msgr2=0x7fbaf80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:49.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:49.014+0000 7fbaff5f7640 1 --2- 172.21.15.120:0/3257440883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaf8101de0 0x7fbaf80744f0 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4006010 tx=0x7fbaf40042f0 comp rx=0 tx=0).stop 2024-09-19T09:28:49.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:49.014+0000 7fbaff5f7640 1 -- 172.21.15.120:0/3257440883 shutdown_connections 2024-09-19T09:28:49.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:49.014+0000 7fbaff5f7640 1 --2- 172.21.15.120:0/3257440883 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7fbae40a3240 0x7fbae40a5700 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:49.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:49.014+0000 7fbaff5f7640 1 --2- 172.21.15.120:0/3257440883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaf8106f10 0x7fbaf8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:49.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:49.014+0000 7fbaff5f7640 1 --2- 172.21.15.120:0/3257440883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaf8102790 0x7fbaf8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:49.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:49.014+0000 7fbaff5f7640 1 --2- 172.21.15.120:0/3257440883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaf8101de0 0x7fbaf80744f0 unknown :-1 s=CLOSED pgs=2594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:49.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:49.014+0000 7fbaff5f7640 1 -- 172.21.15.120:0/3257440883 >> 172.21.15.120:0/3257440883 conn(0x7fbaf80fd8c0 msgr2=0x7fbaf80fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:49.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:49.014+0000 7fbaff5f7640 1 -- 172.21.15.120:0/3257440883 shutdown_connections 2024-09-19T09:28:49.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:49.014+0000 7fbaff5f7640 1 -- 172.21.15.120:0/3257440883 wait complete. 2024-09-19T09:28:49.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:49 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.120:6840/3811354001,v1:172.21.15.120:6841/3811354001] up:boot 2024-09-19T09:28:49.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:49 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.120:6838/3338202675,v1:172.21.15.120:6839/3338202675] up:boot 2024-09-19T09:28:49.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:49 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.153:6833/3231499057,v1:172.21.15.153:6837/3231499057] up:boot 2024-09-19T09:28:49.327 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:49 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.153:6832/900864161,v1:172.21.15.153:6836/900864161] up:boot 2024-09-19T09:28:49.327 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:49 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.153:6834/4256727751,v1:172.21.15.153:6838/4256727751] up:boot 2024-09-19T09:28:49.327 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:49 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.153:6835/3383113918,v1:172.21.15.153:6839/3383113918] up:boot 2024-09-19T09:28:49.327 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:49 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.196:6832/2823184488,v1:172.21.15.196:6836/2823184488] up:boot 2024-09-19T09:28:49.327 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:49 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.196:6835/3262425677,v1:172.21.15.196:6837/3262425677] up:boot 2024-09-19T09:28:49.327 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:49 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.196:6833/586576773,v1:172.21.15.196:6838/586576773] up:boot 2024-09-19T09:28:49.327 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:49 smithi120 ceph-mon[31150]: mds.? [v2:172.21.15.196:6834/3796644744,v1:172.21.15.196:6839/3796644744] up:boot 2024-09-19T09:28:49.327 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:49 smithi120 ceph-mon[31150]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T09:28:49.327 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:49 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T09:28:49.327 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:49 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T09:28:49.327 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:49 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:28:49.327 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:49 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T09:28:49.327 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:49 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T09:28:49.327 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:49 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T09:28:49.328 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:49 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T09:28:49.328 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:49 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T09:28:49.328 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:49 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T09:28:49.328 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:49 smithi120 ceph-mon[31150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T09:28:49.328 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:49 smithi120 ceph-mon[31150]: pgmap v1124: 97 pgs: 97 active+clean; 263 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.4 KiB/s wr, 14 op/s 2024-09-19T09:28:49.328 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:49 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3257440883' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T09:28:49.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:49 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.120:6840/3811354001,v1:172.21.15.120:6841/3811354001] up:boot 2024-09-19T09:28:49.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:49 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.120:6838/3338202675,v1:172.21.15.120:6839/3338202675] up:boot 2024-09-19T09:28:49.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:49 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.153:6833/3231499057,v1:172.21.15.153:6837/3231499057] up:boot 2024-09-19T09:28:49.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:49 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.153:6832/900864161,v1:172.21.15.153:6836/900864161] up:boot 2024-09-19T09:28:49.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:49 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.153:6834/4256727751,v1:172.21.15.153:6838/4256727751] up:boot 2024-09-19T09:28:49.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:49 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.153:6835/3383113918,v1:172.21.15.153:6839/3383113918] up:boot 2024-09-19T09:28:49.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:49 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.196:6832/2823184488,v1:172.21.15.196:6836/2823184488] up:boot 2024-09-19T09:28:49.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:49 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.196:6835/3262425677,v1:172.21.15.196:6837/3262425677] up:boot 2024-09-19T09:28:49.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:49 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.196:6833/586576773,v1:172.21.15.196:6838/586576773] up:boot 2024-09-19T09:28:49.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:49 smithi196 ceph-mon[35150]: mds.? [v2:172.21.15.196:6834/3796644744,v1:172.21.15.196:6839/3796644744] up:boot 2024-09-19T09:28:49.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:49 smithi196 ceph-mon[35150]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T09:28:49.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:49 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T09:28:49.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:49 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T09:28:49.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:49 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:28:49.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:49 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T09:28:49.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:49 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T09:28:49.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:49 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T09:28:49.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:49 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T09:28:49.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:49 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T09:28:49.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:49 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T09:28:49.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:49 smithi196 ceph-mon[35150]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T09:28:49.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:49 smithi196 ceph-mon[35150]: pgmap v1124: 97 pgs: 97 active+clean; 263 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.4 KiB/s wr, 14 op/s 2024-09-19T09:28:49.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:49 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3257440883' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T09:28:49.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:49 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.120:6840/3811354001,v1:172.21.15.120:6841/3811354001] up:boot 2024-09-19T09:28:49.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:49 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.120:6838/3338202675,v1:172.21.15.120:6839/3338202675] up:boot 2024-09-19T09:28:49.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:49 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.153:6833/3231499057,v1:172.21.15.153:6837/3231499057] up:boot 2024-09-19T09:28:49.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:49 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.153:6832/900864161,v1:172.21.15.153:6836/900864161] up:boot 2024-09-19T09:28:49.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:49 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.153:6834/4256727751,v1:172.21.15.153:6838/4256727751] up:boot 2024-09-19T09:28:49.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:49 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.153:6835/3383113918,v1:172.21.15.153:6839/3383113918] up:boot 2024-09-19T09:28:49.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:49 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.196:6832/2823184488,v1:172.21.15.196:6836/2823184488] up:boot 2024-09-19T09:28:49.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:49 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.196:6835/3262425677,v1:172.21.15.196:6837/3262425677] up:boot 2024-09-19T09:28:49.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:49 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.196:6833/586576773,v1:172.21.15.196:6838/586576773] up:boot 2024-09-19T09:28:49.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:49 smithi153 ceph-mon[35071]: mds.? [v2:172.21.15.196:6834/3796644744,v1:172.21.15.196:6839/3796644744] up:boot 2024-09-19T09:28:49.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:49 smithi153 ceph-mon[35071]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T09:28:49.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:49 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T09:28:49.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:49 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T09:28:49.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:49 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:28:49.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:49 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T09:28:49.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:49 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T09:28:49.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:49 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T09:28:49.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:49 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T09:28:49.440 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:49 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T09:28:49.441 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:49 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T09:28:49.441 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:49 smithi153 ceph-mon[35071]: from='mgr.14150 172.21.15.120:0/1161684560' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T09:28:49.441 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:49 smithi153 ceph-mon[35071]: pgmap v1124: 97 pgs: 97 active+clean; 263 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.4 KiB/s wr, 14 op/s 2024-09-19T09:28:49.441 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:49 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3257440883' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T09:28:49.712 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T09:28:49.962 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:50.274 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:50 smithi120 ceph-mon[31150]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T09:28:50.274 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:50 smithi120 ceph-mon[31150]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T09:28:50.274 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:50 smithi120 ceph-mon[31150]: Cluster is now healthy 2024-09-19T09:28:50.274 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:50 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/3257440883' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T09:28:50.274 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:50 smithi120 ceph-mon[31150]: fsmap 12 up:standby 2024-09-19T09:28:50.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:50 smithi196 ceph-mon[35150]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T09:28:50.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:50 smithi196 ceph-mon[35150]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T09:28:50.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:50 smithi196 ceph-mon[35150]: Cluster is now healthy 2024-09-19T09:28:50.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:50 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/3257440883' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T09:28:50.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:50 smithi196 ceph-mon[35150]: fsmap 12 up:standby 2024-09-19T09:28:50.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:50 smithi153 ceph-mon[35071]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T09:28:50.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:50 smithi153 ceph-mon[35071]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T09:28:50.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:50 smithi153 ceph-mon[35071]: Cluster is now healthy 2024-09-19T09:28:50.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:50 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/3257440883' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T09:28:50.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:50 smithi153 ceph-mon[35071]: fsmap 12 up:standby 2024-09-19T09:28:50.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.497+0000 7f81b5dae640 1 -- 172.21.15.120:0/3985166356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81b0108160 msgr2=0x7f81b010eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:50.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.497+0000 7f81b5dae640 1 --2- 172.21.15.120:0/3985166356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81b0108160 0x7f81b010eb00 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009cd0 tx=0x7f81a802a8e0 comp rx=0 tx=0).stop 2024-09-19T09:28:50.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.497+0000 7f81b5dae640 1 -- 172.21.15.120:0/3985166356 shutdown_connections 2024-09-19T09:28:50.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.497+0000 7f81b5dae640 1 --2- 172.21.15.120:0/3985166356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81b0108160 0x7f81b010eb00 unknown :-1 s=CLOSED pgs=2595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:50.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.497+0000 7f81b5dae640 1 --2- 172.21.15.120:0/3985166356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f81b0101290 0x7f81b0107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:50.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.497+0000 7f81b5dae640 1 --2- 172.21.15.120:0/3985166356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81b01008e0 0x7f81b0100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:50.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.497+0000 7f81b5dae640 1 -- 172.21.15.120:0/3985166356 >> 172.21.15.120:0/3985166356 conn(0x7f81b0077210 msgr2=0x7f81b0077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:50.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.498+0000 7f81b5dae640 1 -- 172.21.15.120:0/3985166356 shutdown_connections 2024-09-19T09:28:50.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.498+0000 7f81b5dae640 1 -- 172.21.15.120:0/3985166356 wait complete. 2024-09-19T09:28:50.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.498+0000 7f81b5dae640 1 Processor -- start 2024-09-19T09:28:50.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.498+0000 7f81b5dae640 1 -- start start 2024-09-19T09:28:50.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.498+0000 7f81b5dae640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81b01008e0 0x7f81b0198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:50.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.498+0000 7f81b5dae640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f81b0101290 0x7f81b0198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:50.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.498+0000 7f81b5dae640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81b0108160 0x7f81b019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:50.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.498+0000 7f81b5dae640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f81b0111fd0 con 0x7f81b01008e0 2024-09-19T09:28:50.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.498+0000 7f81b5dae640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f81b0111e50 con 0x7f81b0101290 2024-09-19T09:28:50.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.498+0000 7f81b5dae640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f81b0112150 con 0x7f81b0108160 2024-09-19T09:28:50.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.499+0000 7f81af7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81b01008e0 0x7f81b0198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:50.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.499+0000 7f81af7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81b01008e0 0x7f81b0198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43186/0 (socket says 172.21.15.120:43186) 2024-09-19T09:28:50.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.499+0000 7f81af7fe640 1 -- 172.21.15.120:0/342245592 learned_addr learned my addr 172.21.15.120:0/342245592 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:50.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.499+0000 7f81a7fff640 1 --2- 172.21.15.120:0/342245592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f81b0101290 0x7f81b0198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:50.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.499+0000 7f81affff640 1 --2- 172.21.15.120:0/342245592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81b0108160 0x7f81b019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:50.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.499+0000 7f81af7fe640 1 -- 172.21.15.120:0/342245592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81b0108160 msgr2=0x7f81b019cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:50.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.499+0000 7f81af7fe640 1 --2- 172.21.15.120:0/342245592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81b0108160 0x7f81b019cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:50.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.499+0000 7f81af7fe640 1 -- 172.21.15.120:0/342245592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f81b0101290 msgr2=0x7f81b0198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:50.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.499+0000 7f81af7fe640 1 --2- 172.21.15.120:0/342245592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f81b0101290 0x7f81b0198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:50.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.499+0000 7f81af7fe640 1 -- 172.21.15.120:0/342245592 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b019d2d0 con 0x7f81b01008e0 2024-09-19T09:28:50.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.499+0000 7f81af7fe640 1 --2- 172.21.15.120:0/342245592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81b01008e0 0x7f81b0198320 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7f81a000c920 tx=0x7f81a000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:50.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.499+0000 7f81ad7fa640 1 -- 172.21.15.120:0/342245592 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0007bb0 con 0x7f81b01008e0 2024-09-19T09:28:50.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.499+0000 7f81ad7fa640 1 -- 172.21.15.120:0/342245592 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81a0007d50 con 0x7f81b01008e0 2024-09-19T09:28:50.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.499+0000 7f81b5dae640 1 -- 172.21.15.120:0/342245592 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b019d5c0 con 0x7f81b01008e0 2024-09-19T09:28:50.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.499+0000 7f81ad7fa640 1 -- 172.21.15.120:0/342245592 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a00187d0 con 0x7f81b01008e0 2024-09-19T09:28:50.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.499+0000 7f81b5dae640 1 -- 172.21.15.120:0/342245592 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b01a4ea0 con 0x7f81b01008e0 2024-09-19T09:28:50.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.500+0000 7f81b5dae640 1 -- 172.21.15.120:0/342245592 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81b0100cc0 con 0x7f81b01008e0 2024-09-19T09:28:50.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.501+0000 7f81ad7fa640 1 -- 172.21.15.120:0/342245592 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f81a000f450 con 0x7f81b01008e0 2024-09-19T09:28:50.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.502+0000 7f81ad7fa640 1 --2- 172.21.15.120:0/342245592 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f819c0a3240 0x7f819c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:50.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.502+0000 7f81a7fff640 1 --2- 172.21.15.120:0/342245592 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f819c0a3240 0x7f819c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:50.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.502+0000 7f81ad7fa640 1 -- 172.21.15.120:0/342245592 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8670+0+0 (secure 0 0 0) 0x7f81a00cfc00 con 0x7f81b01008e0 2024-09-19T09:28:50.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.503+0000 7f81a7fff640 1 --2- 172.21.15.120:0/342245592 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f819c0a3240 0x7f819c0a5700 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f81b0107930 tx=0x7f819800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:50.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.506+0000 7f81ad7fa640 1 -- 172.21.15.120:0/342245592 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f81a00d6080 con 0x7f81b01008e0 2024-09-19T09:28:50.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:50.676+0000 7f81b5dae640 1 -- 172.21.15.120:0/342245592 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f81b01a5100 con 0x7f81b01008e0 2024-09-19T09:28:51.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:51.019+0000 7f81ad7fa640 1 -- 172.21.15.120:0/342245592 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v87) ==== 181+0+0 (secure 0 0 0) 0x7f81a0014030 con 0x7f81b01008e0 2024-09-19T09:28:51.020 INFO:teuthology.orchestra.run.smithi120.stderr:pool 'cephfs_metadata' removed 2024-09-19T09:28:51.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:51.022+0000 7f81b5dae640 1 -- 172.21.15.120:0/342245592 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f819c0a3240 msgr2=0x7f819c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:51.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:51.022+0000 7f81b5dae640 1 --2- 172.21.15.120:0/342245592 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f819c0a3240 0x7f819c0a5700 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f81b0107930 tx=0x7f819800a2d0 comp rx=0 tx=0).stop 2024-09-19T09:28:51.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:51.022+0000 7f81b5dae640 1 -- 172.21.15.120:0/342245592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81b01008e0 msgr2=0x7f81b0198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:51.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:51.022+0000 7f81b5dae640 1 --2- 172.21.15.120:0/342245592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81b01008e0 0x7f81b0198320 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7f81a000c920 tx=0x7f81a000cd20 comp rx=0 tx=0).stop 2024-09-19T09:28:51.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:51.024+0000 7f81b5dae640 1 -- 172.21.15.120:0/342245592 shutdown_connections 2024-09-19T09:28:51.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:51.024+0000 7f81b5dae640 1 --2- 172.21.15.120:0/342245592 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f819c0a3240 0x7f819c0a5700 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:51.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:51.024+0000 7f81b5dae640 1 --2- 172.21.15.120:0/342245592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81b0108160 0x7f81b019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:51.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:51.024+0000 7f81b5dae640 1 --2- 172.21.15.120:0/342245592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f81b0101290 0x7f81b0198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:51.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:51.024+0000 7f81b5dae640 1 --2- 172.21.15.120:0/342245592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81b01008e0 0x7f81b0198320 unknown :-1 s=CLOSED pgs=2596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:51.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:51.024+0000 7f81b5dae640 1 -- 172.21.15.120:0/342245592 >> 172.21.15.120:0/342245592 conn(0x7f81b0077210 msgr2=0x7f81b00ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:51.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:51.024+0000 7f81b5dae640 1 -- 172.21.15.120:0/342245592 shutdown_connections 2024-09-19T09:28:51.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:51.024+0000 7f81b5dae640 1 -- 172.21.15.120:0/342245592 wait complete. 2024-09-19T09:28:51.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:51 smithi120 ceph-mon[31150]: pgmap v1125: 97 pgs: 97 active+clean; 263 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-09-19T09:28:51.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:51 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/342245592' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T09:28:51.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:51 smithi196 ceph-mon[35150]: pgmap v1125: 97 pgs: 97 active+clean; 263 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-09-19T09:28:51.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:51 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/342245592' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T09:28:51.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:51 smithi153 ceph-mon[35071]: pgmap v1125: 97 pgs: 97 active+clean; 263 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-09-19T09:28:51.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:51 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/342245592' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T09:28:51.901 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 507393d2-7664-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T09:28:52.154 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:52.275 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:52 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/342245592' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T09:28:52.276 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:52 smithi120 ceph-mon[31150]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T09:28:52.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:52 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/342245592' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T09:28:52.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:52 smithi196 ceph-mon[35150]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T09:28:52.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:52 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/342245592' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T09:28:52.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:52 smithi153 ceph-mon[35071]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T09:28:52.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.703+0000 7f9988cad640 1 -- 172.21.15.120:0/3743736628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9984106050 msgr2=0x7f998410dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:52.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.703+0000 7f9988cad640 1 --2- 172.21.15.120:0/3743736628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9984106050 0x7f998410dbd0 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7f997c009ca0 tx=0x7f997c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:28:52.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.704+0000 7f9988cad640 1 -- 172.21.15.120:0/3743736628 shutdown_connections 2024-09-19T09:28:52.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.704+0000 7f9988cad640 1 --2- 172.21.15.120:0/3743736628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9984106050 0x7f998410dbd0 unknown :-1 s=CLOSED pgs=2597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:52.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.704+0000 7f9988cad640 1 --2- 172.21.15.120:0/3743736628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9984101df0 0x7f9984105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:52.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.704+0000 7f9988cad640 1 --2- 172.21.15.120:0/3743736628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9984101440 0x7f9984101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:52.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.704+0000 7f9988cad640 1 -- 172.21.15.120:0/3743736628 >> 172.21.15.120:0/3743736628 conn(0x7f9984077210 msgr2=0x7f99840fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:52.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.704+0000 7f9988cad640 1 -- 172.21.15.120:0/3743736628 shutdown_connections 2024-09-19T09:28:52.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.704+0000 7f9988cad640 1 -- 172.21.15.120:0/3743736628 wait complete. 2024-09-19T09:28:52.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.704+0000 7f9988cad640 1 Processor -- start 2024-09-19T09:28:52.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.704+0000 7f9988cad640 1 -- start start 2024-09-19T09:28:52.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.705+0000 7f9988cad640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9984101440 0x7f9984198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:52.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.705+0000 7f9988cad640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9984101df0 0x7f9984198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:52.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.705+0000 7f9988cad640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9984106050 0x7f998419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:52.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.705+0000 7f9988cad640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9984111050 con 0x7f9984106050 2024-09-19T09:28:52.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.705+0000 7f9988cad640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9984110ed0 con 0x7f9984101440 2024-09-19T09:28:52.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.705+0000 7f9988cad640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f99841111d0 con 0x7f9984101df0 2024-09-19T09:28:52.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.705+0000 7f9982d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9984106050 0x7f998419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:52.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.705+0000 7f9982d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9984106050 0x7f998419ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48668/0 (socket says 172.21.15.120:48668) 2024-09-19T09:28:52.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.705+0000 7f998259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9984101440 0x7f9984198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:52.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.705+0000 7f9982d9d640 1 -- 172.21.15.120:0/1506736178 learned_addr learned my addr 172.21.15.120:0/1506736178 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-19T09:28:52.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.705+0000 7f997bfff640 1 --2- 172.21.15.120:0/1506736178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9984101df0 0x7f9984198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:52.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.705+0000 7f9982d9d640 1 -- 172.21.15.120:0/1506736178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9984101df0 msgr2=0x7f9984198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:52.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.705+0000 7f9982d9d640 1 --2- 172.21.15.120:0/1506736178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9984101df0 0x7f9984198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:52.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.705+0000 7f9982d9d640 1 -- 172.21.15.120:0/1506736178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9984101440 msgr2=0x7f9984198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:52.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.705+0000 7f9982d9d640 1 --2- 172.21.15.120:0/1506736178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9984101440 0x7f9984198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:52.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.705+0000 7f9982d9d640 1 -- 172.21.15.120:0/1506736178 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f998419d530 con 0x7f9984106050 2024-09-19T09:28:52.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.705+0000 7f9982d9d640 1 --2- 172.21.15.120:0/1506736178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9984106050 0x7f998419ce50 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7f997c031a20 tx=0x7f997c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:52.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.705+0000 7f997b7fe640 1 -- 172.21.15.120:0/1506736178 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c003b70 con 0x7f9984106050 2024-09-19T09:28:52.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.706+0000 7f997b7fe640 1 -- 172.21.15.120:0/1506736178 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f997c003d10 con 0x7f9984106050 2024-09-19T09:28:52.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.706+0000 7f9988cad640 1 -- 172.21.15.120:0/1506736178 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f998419d7c0 con 0x7f9984106050 2024-09-19T09:28:52.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.706+0000 7f997b7fe640 1 -- 172.21.15.120:0/1506736178 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c02fc80 con 0x7f9984106050 2024-09-19T09:28:52.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.706+0000 7f9988cad640 1 -- 172.21.15.120:0/1506736178 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99841a5100 con 0x7f9984106050 2024-09-19T09:28:52.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.707+0000 7f9988cad640 1 -- 172.21.15.120:0/1506736178 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9984101820 con 0x7f9984106050 2024-09-19T09:28:52.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.708+0000 7f997b7fe640 1 -- 172.21.15.120:0/1506736178 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f997c0046a0 con 0x7f9984106050 2024-09-19T09:28:52.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.708+0000 7f997b7fe640 1 --2- 172.21.15.120:0/1506736178 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f99700a3240 0x7f99700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:52.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.708+0000 7f998259c640 1 --2- 172.21.15.120:0/1506736178 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f99700a3240 0x7f99700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:52.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.708+0000 7f997b7fe640 1 -- 172.21.15.120:0/1506736178 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8116+0+0 (secure 0 0 0) 0x7f997c0f0a20 con 0x7f9984106050 2024-09-19T09:28:52.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.709+0000 7f998259c640 1 --2- 172.21.15.120:0/1506736178 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f99700a3240 0x7f99700a5700 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f997400abe0 tx=0x7f997400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:52.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.713+0000 7f997b7fe640 1 -- 172.21.15.120:0/1506736178 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f997c0bd890 con 0x7f9984106050 2024-09-19T09:28:52.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:52.879+0000 7f9988cad640 1 -- 172.21.15.120:0/1506736178 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f998406a5a0 con 0x7f9984106050 2024-09-19T09:28:53.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:53.025+0000 7f997b7fe640 1 -- 172.21.15.120:0/1506736178 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v88) ==== 169+0+0 (secure 0 0 0) 0x7f997c0bd190 con 0x7f9984106050 2024-09-19T09:28:53.027 INFO:teuthology.orchestra.run.smithi120.stderr:pool 'cephfs_data' removed 2024-09-19T09:28:53.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:53.028+0000 7f9988cad640 1 -- 172.21.15.120:0/1506736178 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f99700a3240 msgr2=0x7f99700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:53.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:53.028+0000 7f9988cad640 1 --2- 172.21.15.120:0/1506736178 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f99700a3240 0x7f99700a5700 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f997400abe0 tx=0x7f997400a2d0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:53.028+0000 7f9988cad640 1 -- 172.21.15.120:0/1506736178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9984106050 msgr2=0x7f998419ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:53.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:53.028+0000 7f9988cad640 1 --2- 172.21.15.120:0/1506736178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9984106050 0x7f998419ce50 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7f997c031a20 tx=0x7f997c031a50 comp rx=0 tx=0).stop 2024-09-19T09:28:53.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:53.028+0000 7f9988cad640 1 -- 172.21.15.120:0/1506736178 shutdown_connections 2024-09-19T09:28:53.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:53.028+0000 7f9988cad640 1 --2- 172.21.15.120:0/1506736178 >> [v2:172.21.15.120:6800/3841841273,v1:172.21.15.120:6801/3841841273] conn(0x7f99700a3240 0x7f99700a5700 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:53.028+0000 7f9988cad640 1 --2- 172.21.15.120:0/1506736178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9984106050 0x7f998419ce50 unknown :-1 s=CLOSED pgs=2598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:53.028+0000 7f9988cad640 1 --2- 172.21.15.120:0/1506736178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9984101df0 0x7f9984198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:53.028+0000 7f9988cad640 1 --2- 172.21.15.120:0/1506736178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9984101440 0x7f9984198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:53.028+0000 7f9988cad640 1 -- 172.21.15.120:0/1506736178 >> 172.21.15.120:0/1506736178 conn(0x7f9984077210 msgr2=0x7f9984104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:53.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:53.028+0000 7f9988cad640 1 -- 172.21.15.120:0/1506736178 shutdown_connections 2024-09-19T09:28:53.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-19T09:28:53.028+0000 7f9988cad640 1 -- 172.21.15.120:0/1506736178 wait complete. 2024-09-19T09:28:53.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:53 smithi120 ceph-mon[31150]: pgmap v1127: 65 pgs: 65 active+clean; 577 KiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:28:53.376 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:53 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1506736178' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T09:28:53.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:53 smithi196 ceph-mon[35150]: pgmap v1127: 65 pgs: 65 active+clean; 577 KiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:28:53.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:53 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1506736178' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T09:28:53.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:53 smithi153 ceph-mon[35071]: pgmap v1127: 65 pgs: 65 active+clean; 577 KiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:28:53.438 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:53 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1506736178' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T09:28:53.877 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T09:28:53.904 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T09:28:53.947 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T09:28:53.981 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-19T09:28:53.981 DEBUG:teuthology.orchestra.run.smithi120:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T09:28:53.996 DEBUG:teuthology.orchestra.run.smithi153:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T09:28:54.019 DEBUG:teuthology.orchestra.run.smithi196:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T09:28:54.038 INFO:tasks.cephadm:Stopping all daemons... 2024-09-19T09:28:54.038 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-19T09:28:54.038 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@mon.a 2024-09-19T09:28:54.188 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:54 smithi153 ceph-mon[35071]: from='client.? 172.21.15.120:0/1506736178' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T09:28:54.189 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:54 smithi153 ceph-mon[35071]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T09:28:54.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:54 smithi120 ceph-mon[31150]: from='client.? 172.21.15.120:0/1506736178' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T09:28:54.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:54 smithi120 ceph-mon[31150]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T09:28:54.326 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:54 smithi120 systemd[1]: Stopping Ceph mon.a for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:28:54.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:54 smithi196 ceph-mon[35150]: from='client.? 172.21.15.120:0/1506736178' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T09:28:54.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:54 smithi196 ceph-mon[35150]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T09:28:54.758 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:54 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-a[31146]: 2024-09-19T09:28:54.355+0000 7f8ebf83b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:28:54.759 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:54 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-a[31146]: 2024-09-19T09:28:54.355+0000 7f8ebf83b640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-19T09:28:54.759 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:54 smithi120 podman[152206]: 2024-09-19 09:28:54.417957628 +0000 UTC m=+0.191043361 container died 12359571ca9cfae382f0f597a6d85c8061cdf3be1bb0b9eb77f3f9aa369d9ea6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-19T09:28:55.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:54 smithi120 podman[152206]: 2024-09-19 09:28:54.941084065 +0000 UTC m=+0.714169795 container cleanup 12359571ca9cfae382f0f597a6d85c8061cdf3be1bb0b9eb77f3f9aa369d9ea6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-a, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-19T09:28:55.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:54 smithi120 bash[152206]: ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-a 2024-09-19T09:28:55.408 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mon.a.service' 2024-09-19T09:28:55.430 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:55 smithi120 podman[152219]: 2024-09-19 09:28:55.168441396 +0000 UTC m=+0.748883597 container remove 12359571ca9cfae382f0f597a6d85c8061cdf3be1bb0b9eb77f3f9aa369d9ea6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-19T09:28:55.430 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:55 smithi120 systemd[1]: ceph-507393d2-7664-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-19T09:28:55.430 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:55 smithi120 systemd[1]: Stopped Ceph mon.a for 507393d2-7664-11ef-bceb-c7b262605968. 2024-09-19T09:28:55.431 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 19 09:28:55 smithi120 systemd[1]: ceph-507393d2-7664-11ef-bceb-c7b262605968@mon.a.service: Consumed 46.512s CPU time. 2024-09-19T09:28:55.454 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:28:55.454 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-19T09:28:55.454 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-19T09:28:55.455 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@mon.b 2024-09-19T09:28:55.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:55 smithi153 systemd[1]: Stopping Ceph mon.b for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:28:56.418 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:55 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-b[35067]: 2024-09-19T09:28:55.948+0000 7fdb1d8e5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:28:56.418 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:55 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-b[35067]: 2024-09-19T09:28:55.948+0000 7fdb1d8e5640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-19T09:28:56.418 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:56 smithi153 podman[72615]: 2024-09-19 09:28:56.122314195 +0000 UTC m=+0.336845346 container died d4890304681e4004cb92a269dee58ba7b76315d9772489f4757d87ddfd1c1cf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0) 2024-09-19T09:28:56.688 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:56 smithi153 podman[72615]: 2024-09-19 09:28:56.516084344 +0000 UTC m=+0.730615490 container cleanup d4890304681e4004cb92a269dee58ba7b76315d9772489f4757d87ddfd1c1cf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-b, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-19T09:28:56.689 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 19 09:28:56 smithi153 bash[72615]: ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-b 2024-09-19T09:28:56.987 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mon.b.service' 2024-09-19T09:28:57.042 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:28:57.042 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-19T09:28:57.042 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-19T09:28:57.043 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@mon.c 2024-09-19T09:28:57.403 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:57 smithi196 systemd[1]: Stopping Ceph mon.c for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:28:57.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:57 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-c[35142]: 2024-09-19T09:28:57.401+0000 7f0aafc54640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:28:57.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:57 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-c[35142]: 2024-09-19T09:28:57.402+0000 7f0aafc54640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-19T09:28:57.687 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:57 smithi196 podman[72343]: 2024-09-19 09:28:57.433234321 +0000 UTC m=+0.141067150 container died 636bb695af2c65422d97e50340ec937b8d1939efa2b2dd72a7a85113d03798d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-c, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-2f599a4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:28:58.055 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:57 smithi196 podman[72343]: 2024-09-19 09:28:57.844668036 +0000 UTC m=+0.552500865 container cleanup 636bb695af2c65422d97e50340ec937b8d1939efa2b2dd72a7a85113d03798d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-c, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:28:58.055 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 19 09:28:57 smithi196 bash[72343]: ceph-507393d2-7664-11ef-bceb-c7b262605968-mon-c 2024-09-19T09:28:58.269 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mon.c.service' 2024-09-19T09:28:58.319 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:28:58.320 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-19T09:28:58.320 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-19T09:28:58.320 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@mgr.x 2024-09-19T09:28:58.806 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 09:28:58 smithi120 systemd[1]: Stopping Ceph mgr.x for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:28:59.077 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 09:28:58 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x[31375]: 2024-09-19T09:28:58.804+0000 7f69d5c04640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:28:59.077 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 09:28:58 smithi120 podman[152327]: 2024-09-19 09:28:58.836318192 +0000 UTC m=+0.325751002 container died fc4ae395f6423bd09ab30e0978bea95c9e11cbbc651a318fb2e891431581d598 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4) 2024-09-19T09:28:59.576 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 09:28:59 smithi120 podman[152327]: 2024-09-19 09:28:59.507303847 +0000 UTC m=+0.996736641 container cleanup fc4ae395f6423bd09ab30e0978bea95c9e11cbbc651a318fb2e891431581d598 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x, ceph=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-19T09:28:59.576 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 19 09:28:59 smithi120 bash[152327]: ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-x 2024-09-19T09:28:59.936 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-19T09:28:59.970 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:28:59.971 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-19T09:28:59.972 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-19T09:28:59.972 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@mgr.y 2024-09-19T09:29:00.293 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 09:29:00 smithi153 systemd[1]: Stopping Ceph mgr.y for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:29:00.598 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 09:29:00 smithi153 podman[72742]: 2024-09-19 09:29:00.292182208 +0000 UTC m=+0.125241664 container died ae5d6044121a1fd1314d1d25c9d43dc09ba889969a08365181e9a55d18e1a6d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-19T09:29:00.932 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 09:29:00 smithi153 podman[72742]: 2024-09-19 09:29:00.722208786 +0000 UTC m=+0.555268283 container cleanup ae5d6044121a1fd1314d1d25c9d43dc09ba889969a08365181e9a55d18e1a6d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-19T09:29:00.932 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 19 09:29:00 smithi153 bash[72742]: ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-y 2024-09-19T09:29:01.148 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-19T09:29:01.184 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:29:01.184 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-19T09:29:01.184 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-19T09:29:01.184 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@mgr.z 2024-09-19T09:29:01.521 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 09:29:01 smithi196 systemd[1]: Stopping Ceph mgr.z for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:29:01.809 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 09:29:01 smithi196 podman[72467]: 2024-09-19 09:29:01.520198743 +0000 UTC m=+0.140298197 container died 8eab4dfbe448eac4b33d93a7b2ac5ef5d6db3b49f420396293994b119e132e50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-19T09:29:02.187 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 09:29:01 smithi196 podman[72467]: 2024-09-19 09:29:01.908311995 +0000 UTC m=+0.528411446 container cleanup 8eab4dfbe448eac4b33d93a7b2ac5ef5d6db3b49f420396293994b119e132e50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-19T09:29:02.187 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 19 09:29:01 smithi196 bash[72467]: ceph-507393d2-7664-11ef-bceb-c7b262605968-mgr-z 2024-09-19T09:29:02.380 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-19T09:29:02.415 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:29:02.415 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-19T09:29:02.415 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-19T09:29:02.415 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.0 2024-09-19T09:29:02.738 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 19 09:29:02 smithi120 systemd[1]: Stopping Ceph osd.0 for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:29:03.076 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 19 09:29:02 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-0[45204]: 2024-09-19T09:29:02.736+0000 7fa468b85640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:29:03.076 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 19 09:29:02 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-0[45204]: 2024-09-19T09:29:02.736+0000 7fa468b85640 -1 osd.0 88 *** Got signal Terminated *** 2024-09-19T09:29:03.076 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 19 09:29:02 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-0[45204]: 2024-09-19T09:29:02.736+0000 7fa468b85640 -1 osd.0 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:29:08.456 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 19 09:29:08 smithi120 podman[152451]: 2024-09-19 09:29:08.164595347 +0000 UTC m=+5.553929039 container died 6aa2a34f6e58704cf10c05af278d8a62244e299d750956d1b43ba9d442be092d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-19T09:29:08.826 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 19 09:29:08 smithi120 podman[152451]: 2024-09-19 09:29:08.630497372 +0000 UTC m=+6.019831112 container cleanup 6aa2a34f6e58704cf10c05af278d8a62244e299d750956d1b43ba9d442be092d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:29:08.826 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 19 09:29:08 smithi120 bash[152451]: ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-0 2024-09-19T09:29:09.239 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 19 09:29:08 smithi120 podman[152469]: 2024-09-19 09:29:08.915629197 +0000 UTC m=+0.753666665 container remove 6aa2a34f6e58704cf10c05af278d8a62244e299d750956d1b43ba9d442be092d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:29:09.554 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 19 09:29:09 smithi120 podman[152556]: 2024-09-19 09:29:09.140717843 +0000 UTC m=+0.015440671 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:29:09.554 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 19 09:29:09 smithi120 podman[152556]: 2024-09-19 09:29:09.26700725 +0000 UTC m=+0.141730114 container create c722a01dc0e5c041c04dfdf169f2fff6bfd19a71c9163adf648eeaa8f06f3fe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-19T09:29:09.826 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 19 09:29:09 smithi120 podman[152556]: 2024-09-19 09:29:09.553441625 +0000 UTC m=+0.428164453 container init c722a01dc0e5c041c04dfdf169f2fff6bfd19a71c9163adf648eeaa8f06f3fe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-0-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-19T09:29:09.827 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 19 09:29:09 smithi120 podman[152556]: 2024-09-19 09:29:09.557057878 +0000 UTC m=+0.431780696 container start c722a01dc0e5c041c04dfdf169f2fff6bfd19a71c9163adf648eeaa8f06f3fe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-19T09:29:09.827 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 19 09:29:09 smithi120 podman[152556]: 2024-09-19 09:29:09.636894166 +0000 UTC m=+0.511616997 container attach c722a01dc0e5c041c04dfdf169f2fff6bfd19a71c9163adf648eeaa8f06f3fe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:29:09.827 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 19 09:29:09 smithi120 podman[152556]: 2024-09-19 09:29:09.749002511 +0000 UTC m=+0.623725332 container died c722a01dc0e5c041c04dfdf169f2fff6bfd19a71c9163adf648eeaa8f06f3fe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:29:10.524 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.0.service' 2024-09-19T09:29:10.563 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:29:10.564 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-19T09:29:10.564 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-19T09:29:10.564 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.1 2024-09-19T09:29:10.895 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 19 09:29:10 smithi153 systemd[1]: Stopping Ceph osd.1 for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:29:11.188 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 19 09:29:10 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-1[39526]: 2024-09-19T09:29:10.892+0000 7f1a9738b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:29:11.188 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 19 09:29:10 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-1[39526]: 2024-09-19T09:29:10.892+0000 7f1a9738b640 -1 osd.1 88 *** Got signal Terminated *** 2024-09-19T09:29:11.188 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 19 09:29:10 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-1[39526]: 2024-09-19T09:29:10.892+0000 7f1a9738b640 -1 osd.1 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:29:16.662 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 19 09:29:16 smithi153 podman[72876]: 2024-09-19 09:29:16.350542685 +0000 UTC m=+5.578804328 container died c7e3e511f17d0a3ba28493c97ea5debc06c964500d0a83b29f6fcacc78248525 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-1, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:29:16.938 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 19 09:29:16 smithi153 podman[72876]: 2024-09-19 09:29:16.760342848 +0000 UTC m=+5.988604492 container cleanup c7e3e511f17d0a3ba28493c97ea5debc06c964500d0a83b29f6fcacc78248525 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-19T09:29:16.938 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 19 09:29:16 smithi153 bash[72876]: ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-1 2024-09-19T09:29:17.337 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 19 09:29:17 smithi153 podman[72892]: 2024-09-19 09:29:17.012078235 +0000 UTC m=+0.657597579 container remove c7e3e511f17d0a3ba28493c97ea5debc06c964500d0a83b29f6fcacc78248525 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-1, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4) 2024-09-19T09:29:17.634 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 19 09:29:17 smithi153 podman[72980]: 2024-09-19 09:29:17.239405145 +0000 UTC m=+0.015778692 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:29:17.634 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 19 09:29:17 smithi153 podman[72980]: 2024-09-19 09:29:17.37149416 +0000 UTC m=+0.147867769 container create f7215526782b7e520d70e28049f5a3baee6abf34ba7b2fa715defebb7935bd05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:29:17.634 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 19 09:29:17 smithi153 podman[72980]: 2024-09-19 09:29:17.633188681 +0000 UTC m=+0.409562231 container init f7215526782b7e520d70e28049f5a3baee6abf34ba7b2fa715defebb7935bd05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4) 2024-09-19T09:29:17.938 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 19 09:29:17 smithi153 podman[72980]: 2024-09-19 09:29:17.636894745 +0000 UTC m=+0.413268291 container start f7215526782b7e520d70e28049f5a3baee6abf34ba7b2fa715defebb7935bd05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD) 2024-09-19T09:29:17.938 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 19 09:29:17 smithi153 podman[72980]: 2024-09-19 09:29:17.708015255 +0000 UTC m=+0.484388811 container attach f7215526782b7e520d70e28049f5a3baee6abf34ba7b2fa715defebb7935bd05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-19T09:29:18.379 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 19 09:29:18 smithi153 podman[72980]: 2024-09-19 09:29:18.015466184 +0000 UTC m=+0.791839725 container died f7215526782b7e520d70e28049f5a3baee6abf34ba7b2fa715defebb7935bd05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:29:18.705 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.1.service' 2024-09-19T09:29:18.746 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:29:18.746 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-19T09:29:18.746 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-19T09:29:18.746 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.2 2024-09-19T09:29:19.071 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 09:29:18 smithi196 systemd[1]: Stopping Ceph osd.2 for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:29:19.437 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 09:29:19 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-2[40117]: 2024-09-19T09:29:19.070+0000 7f96959ac640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:29:19.437 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 09:29:19 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-2[40117]: 2024-09-19T09:29:19.070+0000 7f96959ac640 -1 osd.2 88 *** Got signal Terminated *** 2024-09-19T09:29:19.437 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 09:29:19 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-2[40117]: 2024-09-19T09:29:19.070+0000 7f96959ac640 -1 osd.2 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:29:24.922 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 09:29:24 smithi196 podman[72594]: 2024-09-19 09:29:24.614261437 +0000 UTC m=+5.668733783 container died c6102df5df55883199884ca79c144ff8aa604c4fb8b17ba166c607308ec31c52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-19T09:29:25.187 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 09:29:25 smithi196 podman[72594]: 2024-09-19 09:29:25.003666558 +0000 UTC m=+6.058138902 container cleanup c6102df5df55883199884ca79c144ff8aa604c4fb8b17ba166c607308ec31c52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:29:25.187 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 09:29:25 smithi196 bash[72594]: ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-2 2024-09-19T09:29:25.606 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 09:29:25 smithi196 podman[72608]: 2024-09-19 09:29:25.297223704 +0000 UTC m=+0.678450987 container remove c6102df5df55883199884ca79c144ff8aa604c4fb8b17ba166c607308ec31c52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-19T09:29:25.893 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 09:29:25 smithi196 podman[72698]: 2024-09-19 09:29:25.508470124 +0000 UTC m=+0.016056405 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:29:25.893 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 09:29:25 smithi196 podman[72698]: 2024-09-19 09:29:25.623110913 +0000 UTC m=+0.130697226 container create aa571df0080631bd9827e456c833016b4fab85eb18e73d18384d4bba7bf3ef78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:29:25.893 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 09:29:25 smithi196 podman[72698]: 2024-09-19 09:29:25.892638862 +0000 UTC m=+0.400225133 container init aa571df0080631bd9827e456c833016b4fab85eb18e73d18384d4bba7bf3ef78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-2-deactivate, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-19T09:29:26.187 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 09:29:25 smithi196 podman[72698]: 2024-09-19 09:29:25.896106542 +0000 UTC m=+0.403692831 container start aa571df0080631bd9827e456c833016b4fab85eb18e73d18384d4bba7bf3ef78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:29:26.187 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 09:29:26 smithi196 podman[72698]: 2024-09-19 09:29:26.001113814 +0000 UTC m=+0.508700103 container attach aa571df0080631bd9827e456c833016b4fab85eb18e73d18384d4bba7bf3ef78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-19T09:29:26.497 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 19 09:29:26 smithi196 podman[72698]: 2024-09-19 09:29:26.194189289 +0000 UTC m=+0.701775571 container died aa571df0080631bd9827e456c833016b4fab85eb18e73d18384d4bba7bf3ef78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-19T09:29:26.834 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.2.service' 2024-09-19T09:29:26.874 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:29:26.874 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-19T09:29:26.875 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-19T09:29:26.875 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.3 2024-09-19T09:29:27.187 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 19 09:29:26 smithi120 systemd[1]: Stopping Ceph osd.3 for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:29:27.515 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 19 09:29:27 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-3[51657]: 2024-09-19T09:29:27.186+0000 7fec70a66640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:29:27.515 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 19 09:29:27 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-3[51657]: 2024-09-19T09:29:27.186+0000 7fec70a66640 -1 osd.3 88 *** Got signal Terminated *** 2024-09-19T09:29:27.515 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 19 09:29:27 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-3[51657]: 2024-09-19T09:29:27.186+0000 7fec70a66640 -1 osd.3 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:29:29.437 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:29 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:29.061+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.519869+0000 front 2024-09-19T09:29:07.519673+0000 (oldest deadline 2024-09-19T09:29:28.619790+0000) 2024-09-19T09:29:30.437 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:30 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:30.038+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.519869+0000 front 2024-09-19T09:29:07.519673+0000 (oldest deadline 2024-09-19T09:29:28.619790+0000) 2024-09-19T09:29:30.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:30 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:30.198+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:31.437 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:31 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:31.013+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.519869+0000 front 2024-09-19T09:29:07.519673+0000 (oldest deadline 2024-09-19T09:29:28.619790+0000) 2024-09-19T09:29:31.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:31 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:31.148+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:31.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:31 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:31.354+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:29:32.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:32 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:32.120+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:32.326 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:31 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:31.910+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.107758+0000 front 2024-09-19T09:29:07.107785+0000 (oldest deadline 2024-09-19T09:29:31.207911+0000) 2024-09-19T09:29:32.437 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:31 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:31.983+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.519869+0000 front 2024-09-19T09:29:07.519673+0000 (oldest deadline 2024-09-19T09:29:28.619790+0000) 2024-09-19T09:29:32.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:32 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:32.371+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:29:32.888 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 19 09:29:32 smithi120 podman[152659]: 2024-09-19 09:29:32.61732381 +0000 UTC m=+5.553953587 container died 06cfca9b5a2e96887b739ac2b23f028d37316575b8539741ceff7af591dc45a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-3, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, ceph=True) 2024-09-19T09:29:32.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:32 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:32.458+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:29:33.138 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:32 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:32.887+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.107758+0000 front 2024-09-19T09:29:07.107785+0000 (oldest deadline 2024-09-19T09:29:31.207911+0000) 2024-09-19T09:29:33.139 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:33 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:33.094+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:33.139 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 19 09:29:33 smithi120 podman[152659]: 2024-09-19 09:29:33.138014071 +0000 UTC m=+6.074643883 container cleanup 06cfca9b5a2e96887b739ac2b23f028d37316575b8539741ceff7af591dc45a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-3, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-19T09:29:33.437 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:33 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:33.004+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.519869+0000 front 2024-09-19T09:29:07.519673+0000 (oldest deadline 2024-09-19T09:29:28.619790+0000) 2024-09-19T09:29:33.576 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 19 09:29:33 smithi120 bash[152659]: ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-3 2024-09-19T09:29:33.576 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 19 09:29:33 smithi120 podman[152687]: 2024-09-19 09:29:33.365058448 +0000 UTC m=+0.742820926 container remove 06cfca9b5a2e96887b739ac2b23f028d37316575b8539741ceff7af591dc45a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-19T09:29:33.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:33 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:33.345+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:29:33.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:33 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:33.441+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:29:34.076 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 19 09:29:33 smithi120 podman[152776]: 2024-09-19 09:29:33.693117061 +0000 UTC m=+0.016154609 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:29:34.076 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 19 09:29:33 smithi120 podman[152776]: 2024-09-19 09:29:33.816021397 +0000 UTC m=+0.139058939 container create f1d4ce4b619aca742d0fb279e6b5aa19296a4a468d618928b693969cfe89345c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4) 2024-09-19T09:29:34.076 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:33 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:33.897+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.107758+0000 front 2024-09-19T09:29:07.107785+0000 (oldest deadline 2024-09-19T09:29:31.207911+0000) 2024-09-19T09:29:34.437 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:33 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:33.971+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.519869+0000 front 2024-09-19T09:29:07.519673+0000 (oldest deadline 2024-09-19T09:29:28.619790+0000) 2024-09-19T09:29:34.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:34 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:34.079+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:34.576 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 19 09:29:34 smithi120 podman[152776]: 2024-09-19 09:29:34.12743368 +0000 UTC m=+0.450471235 container init f1d4ce4b619aca742d0fb279e6b5aa19296a4a468d618928b693969cfe89345c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, io.buildah.version=1.37.2) 2024-09-19T09:29:34.576 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 19 09:29:34 smithi120 podman[152776]: 2024-09-19 09:29:34.130785135 +0000 UTC m=+0.453822682 container start f1d4ce4b619aca742d0fb279e6b5aa19296a4a468d618928b693969cfe89345c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:29:34.576 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 19 09:29:34 smithi120 podman[152776]: 2024-09-19 09:29:34.219318044 +0000 UTC m=+0.542355604 container attach f1d4ce4b619aca742d0fb279e6b5aa19296a4a468d618928b693969cfe89345c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD) 2024-09-19T09:29:34.576 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 19 09:29:34 smithi120 podman[152776]: 2024-09-19 09:29:34.324637867 +0000 UTC m=+0.647675409 container died f1d4ce4b619aca742d0fb279e6b5aa19296a4a468d618928b693969cfe89345c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-19T09:29:34.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:34 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:34.393+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:29:34.938 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:34 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:34.455+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:29:34.956 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.3.service' 2024-09-19T09:29:34.996 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:29:34.997 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-19T09:29:34.997 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-19T09:29:34.997 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.4 2024-09-19T09:29:35.318 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 19 09:29:35 smithi153 systemd[1]: Stopping Ceph osd.4 for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:29:35.326 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:34 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:34.925+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.107758+0000 front 2024-09-19T09:29:07.107785+0000 (oldest deadline 2024-09-19T09:29:31.207911+0000) 2024-09-19T09:29:35.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:35 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:35.120+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:35.411 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:35 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:35.011+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.519869+0000 front 2024-09-19T09:29:07.519673+0000 (oldest deadline 2024-09-19T09:29:28.619790+0000) 2024-09-19T09:29:35.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:35 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:35.410+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:29:35.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:35 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:35.345+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:29:35.688 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 19 09:29:35 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-4[45982]: 2024-09-19T09:29:35.316+0000 7f2dab593640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:29:35.688 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 19 09:29:35 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-4[45982]: 2024-09-19T09:29:35.316+0000 7f2dab593640 -1 osd.4 88 *** Got signal Terminated *** 2024-09-19T09:29:35.688 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 19 09:29:35 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-4[45982]: 2024-09-19T09:29:35.316+0000 7f2dab593640 -1 osd.4 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:29:36.326 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:35 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:35.941+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.107758+0000 front 2024-09-19T09:29:07.107785+0000 (oldest deadline 2024-09-19T09:29:31.207911+0000) 2024-09-19T09:29:36.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:36 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:36.103+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:36.397 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:36 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:36.026+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.519869+0000 front 2024-09-19T09:29:07.519673+0000 (oldest deadline 2024-09-19T09:29:28.619790+0000) 2024-09-19T09:29:36.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:36 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:36.396+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:29:36.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:36 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:36.311+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:29:37.326 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:36 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:36.948+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.107758+0000 front 2024-09-19T09:29:07.107785+0000 (oldest deadline 2024-09-19T09:29:31.207911+0000) 2024-09-19T09:29:37.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:37 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:37.143+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:37.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:37 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:37.143+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:29:37.430 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:37 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:37.059+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.519869+0000 front 2024-09-19T09:29:07.519673+0000 (oldest deadline 2024-09-19T09:29:28.619790+0000) 2024-09-19T09:29:37.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:37 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:37.429+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:29:37.688 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 19 09:29:37 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-4[45982]: 2024-09-19T09:29:37.267+0000 7f2da7ba9640 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.911079+0000 front 2024-09-19T09:29:14.911105+0000 (oldest deadline 2024-09-19T09:29:37.211228+0000) 2024-09-19T09:29:37.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:37 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:37.284+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:29:38.326 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:37 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:37.948+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.107758+0000 front 2024-09-19T09:29:07.107785+0000 (oldest deadline 2024-09-19T09:29:31.207911+0000) 2024-09-19T09:29:38.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:38 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:38.128+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:38.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:38 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:38.128+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:29:38.393 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:38 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:38.069+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.519869+0000 front 2024-09-19T09:29:07.519673+0000 (oldest deadline 2024-09-19T09:29:28.619790+0000) 2024-09-19T09:29:38.393 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:38 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:38.022+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:29:38.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:38 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:38.391+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:29:38.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:38 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:38.294+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:29:38.688 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 19 09:29:38 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-4[45982]: 2024-09-19T09:29:38.218+0000 7f2da7ba9640 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.911079+0000 front 2024-09-19T09:29:14.911105+0000 (oldest deadline 2024-09-19T09:29:37.211228+0000) 2024-09-19T09:29:39.326 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:38 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:38.970+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.107758+0000 front 2024-09-19T09:29:07.107785+0000 (oldest deadline 2024-09-19T09:29:31.207911+0000) 2024-09-19T09:29:39.326 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:38 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:38.970+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:12.908918+0000 front 2024-09-19T09:29:12.908887+0000 (oldest deadline 2024-09-19T09:29:38.808899+0000) 2024-09-19T09:29:39.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:39 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:39.141+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:39.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:39 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:39.141+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:29:39.388 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:39 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:39.021+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:29:39.388 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:39 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:39.110+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.519869+0000 front 2024-09-19T09:29:07.519673+0000 (oldest deadline 2024-09-19T09:29:28.619790+0000) 2024-09-19T09:29:39.388 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:39 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:39.110+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.520884+0000 front 2024-09-19T09:29:14.520831+0000 (oldest deadline 2024-09-19T09:29:38.620849+0000) 2024-09-19T09:29:39.589 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 19 09:29:39 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-4[45982]: 2024-09-19T09:29:39.220+0000 7f2da7ba9640 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.911079+0000 front 2024-09-19T09:29:14.911105+0000 (oldest deadline 2024-09-19T09:29:37.211228+0000) 2024-09-19T09:29:39.589 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:39 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:39.309+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:29:39.590 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:39 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:39.309+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:29:39.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:39 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:39.386+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:29:40.326 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:39 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:39.956+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.107758+0000 front 2024-09-19T09:29:07.107785+0000 (oldest deadline 2024-09-19T09:29:31.207911+0000) 2024-09-19T09:29:40.326 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:39 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:39.956+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:12.908918+0000 front 2024-09-19T09:29:12.908887+0000 (oldest deadline 2024-09-19T09:29:38.808899+0000) 2024-09-19T09:29:40.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:40 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:40.155+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:40.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:40 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:40.155+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:29:40.387 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:39 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:39.998+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:29:40.387 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:40 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:40.118+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.519869+0000 front 2024-09-19T09:29:07.519673+0000 (oldest deadline 2024-09-19T09:29:28.619790+0000) 2024-09-19T09:29:40.388 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:40 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:40.118+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.520884+0000 front 2024-09-19T09:29:14.520831+0000 (oldest deadline 2024-09-19T09:29:38.620849+0000) 2024-09-19T09:29:40.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:40 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:40.386+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:29:40.688 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 19 09:29:40 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-4[45982]: 2024-09-19T09:29:40.217+0000 7f2da7ba9640 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.911079+0000 front 2024-09-19T09:29:14.911105+0000 (oldest deadline 2024-09-19T09:29:37.211228+0000) 2024-09-19T09:29:40.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:40 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:40.347+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:29:40.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:40 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:40.347+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:29:41.035 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 19 09:29:40 smithi153 podman[73084]: 2024-09-19 09:29:40.753056645 +0000 UTC m=+5.555150201 container died 7458524620cbaa0f19074bc09ac3329faf39cae19af117d6a18649e841f6a0ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-19T09:29:41.180 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:40 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:40.913+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.107758+0000 front 2024-09-19T09:29:07.107785+0000 (oldest deadline 2024-09-19T09:29:31.207911+0000) 2024-09-19T09:29:41.180 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:40 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:40.913+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:12.908918+0000 front 2024-09-19T09:29:12.908887+0000 (oldest deadline 2024-09-19T09:29:38.808899+0000) 2024-09-19T09:29:41.334 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:41 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:41.055+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.656187+0000 front 2024-09-19T09:29:14.656171+0000 (oldest deadline 2024-09-19T09:29:40.556311+0000) 2024-09-19T09:29:41.334 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 19 09:29:41 smithi153 podman[73084]: 2024-09-19 09:29:41.141994433 +0000 UTC m=+5.944087989 container cleanup 7458524620cbaa0f19074bc09ac3329faf39cae19af117d6a18649e841f6a0ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-4, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=) 2024-09-19T09:29:41.334 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 19 09:29:41 smithi153 bash[73084]: ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-4 2024-09-19T09:29:41.406 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:41 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:41.163+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.519869+0000 front 2024-09-19T09:29:07.519673+0000 (oldest deadline 2024-09-19T09:29:28.619790+0000) 2024-09-19T09:29:41.406 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:41 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:41.163+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.520884+0000 front 2024-09-19T09:29:14.520831+0000 (oldest deadline 2024-09-19T09:29:38.620849+0000) 2024-09-19T09:29:41.406 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:41 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:41.035+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:29:41.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:41 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:41.178+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:41.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:41 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:41.178+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:29:41.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:41 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:41.404+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:29:41.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:41 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:41.333+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:29:41.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:41 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:41.333+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:29:41.688 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 19 09:29:41 smithi153 podman[73100]: 2024-09-19 09:29:41.401637391 +0000 UTC m=+0.644045114 container remove 7458524620cbaa0f19074bc09ac3329faf39cae19af117d6a18649e841f6a0ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-19T09:29:42.006 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 19 09:29:41 smithi153 podman[73189]: 2024-09-19 09:29:41.613164601 +0000 UTC m=+0.015828033 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:29:42.006 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 19 09:29:41 smithi153 podman[73189]: 2024-09-19 09:29:41.752695291 +0000 UTC m=+0.155358713 container create a758814fdb32772013b3ec014a90a2e0ebe3f69f2eaf1773cedd84be29168258 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, io.buildah.version=1.37.2) 2024-09-19T09:29:42.227 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:41 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:41.896+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.107758+0000 front 2024-09-19T09:29:07.107785+0000 (oldest deadline 2024-09-19T09:29:31.207911+0000) 2024-09-19T09:29:42.227 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:41 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:41.896+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:12.908918+0000 front 2024-09-19T09:29:12.908887+0000 (oldest deadline 2024-09-19T09:29:38.808899+0000) 2024-09-19T09:29:42.334 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:42 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:42.058+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.656187+0000 front 2024-09-19T09:29:14.656171+0000 (oldest deadline 2024-09-19T09:29:40.556311+0000) 2024-09-19T09:29:42.334 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 19 09:29:42 smithi153 podman[73189]: 2024-09-19 09:29:42.005413314 +0000 UTC m=+0.408076736 container init a758814fdb32772013b3ec014a90a2e0ebe3f69f2eaf1773cedd84be29168258 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-19T09:29:42.334 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 19 09:29:42 smithi153 podman[73189]: 2024-09-19 09:29:42.008934999 +0000 UTC m=+0.411598431 container start a758814fdb32772013b3ec014a90a2e0ebe3f69f2eaf1773cedd84be29168258 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-19T09:29:42.334 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 19 09:29:42 smithi153 podman[73189]: 2024-09-19 09:29:42.097234071 +0000 UTC m=+0.499897503 container attach a758814fdb32772013b3ec014a90a2e0ebe3f69f2eaf1773cedd84be29168258 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-19T09:29:42.335 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 19 09:29:42 smithi153 podman[73189]: 2024-09-19 09:29:42.197512207 +0000 UTC m=+0.600175640 container died a758814fdb32772013b3ec014a90a2e0ebe3f69f2eaf1773cedd84be29168258 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-19T09:29:42.427 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:41 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:41.992+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:29:42.428 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:42 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:42.129+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.519869+0000 front 2024-09-19T09:29:07.519673+0000 (oldest deadline 2024-09-19T09:29:28.619790+0000) 2024-09-19T09:29:42.428 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:42 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:42.129+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.520884+0000 front 2024-09-19T09:29:14.520831+0000 (oldest deadline 2024-09-19T09:29:38.620849+0000) 2024-09-19T09:29:42.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:42 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:42.225+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:42.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:42 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:42.225+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:29:42.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:42 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:42.426+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:29:42.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:42 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:42.426+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:29:42.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:42 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:42.332+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:29:42.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:42 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:42.332+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:29:42.884 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.4.service' 2024-09-19T09:29:42.924 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:29:42.924 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-19T09:29:42.924 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-19T09:29:42.925 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.5 2024-09-19T09:29:43.188 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:43 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:43.059+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.656187+0000 front 2024-09-19T09:29:14.656171+0000 (oldest deadline 2024-09-19T09:29:40.556311+0000) 2024-09-19T09:29:43.236 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:42 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:42.855+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.107758+0000 front 2024-09-19T09:29:07.107785+0000 (oldest deadline 2024-09-19T09:29:31.207911+0000) 2024-09-19T09:29:43.236 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:42 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:42.855+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:12.908918+0000 front 2024-09-19T09:29:12.908887+0000 (oldest deadline 2024-09-19T09:29:38.808899+0000) 2024-09-19T09:29:43.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:43 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:43.018+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:29:43.437 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:42 smithi196 systemd[1]: Stopping Ceph osd.5 for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:29:43.438 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:43 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:43.106+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.519869+0000 front 2024-09-19T09:29:07.519673+0000 (oldest deadline 2024-09-19T09:29:28.619790+0000) 2024-09-19T09:29:43.438 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:43 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:43.106+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.520884+0000 front 2024-09-19T09:29:14.520831+0000 (oldest deadline 2024-09-19T09:29:38.620849+0000) 2024-09-19T09:29:43.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:43 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:43.234+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:43.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:43 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:43.234+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:29:43.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:43 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:43.348+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:29:43.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:43 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:43.348+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:29:43.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:43 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:43.445+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:29:43.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:43 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:43.445+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:29:43.938 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:43 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:43.532+0000 7f34339b8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:29:43.938 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:43 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:43.532+0000 7f34339b8640 -1 osd.5 88 *** Got signal Terminated *** 2024-09-19T09:29:43.938 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:43 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:43.532+0000 7f34339b8640 -1 osd.5 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:29:44.137 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:43 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:43.874+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.107758+0000 front 2024-09-19T09:29:07.107785+0000 (oldest deadline 2024-09-19T09:29:31.207911+0000) 2024-09-19T09:29:44.138 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:43 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:43.874+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:12.908918+0000 front 2024-09-19T09:29:12.908887+0000 (oldest deadline 2024-09-19T09:29:38.808899+0000) 2024-09-19T09:29:44.319 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:44 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:44.030+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.656187+0000 front 2024-09-19T09:29:14.656171+0000 (oldest deadline 2024-09-19T09:29:40.556311+0000) 2024-09-19T09:29:44.418 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:43 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:43.971+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:29:44.418 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:44 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:44.066+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.519869+0000 front 2024-09-19T09:29:07.519673+0000 (oldest deadline 2024-09-19T09:29:28.619790+0000) 2024-09-19T09:29:44.418 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:44 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:44.066+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.520884+0000 front 2024-09-19T09:29:14.520831+0000 (oldest deadline 2024-09-19T09:29:38.620849+0000) 2024-09-19T09:29:44.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:44 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:44.247+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:44.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:44 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:44.247+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:29:44.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:44 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:44.318+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:29:44.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:44 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:44.318+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:29:44.689 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:44 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:44.416+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:29:44.690 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:44 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:44.416+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:29:45.289 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:44 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:44.878+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.107758+0000 front 2024-09-19T09:29:07.107785+0000 (oldest deadline 2024-09-19T09:29:31.207911+0000) 2024-09-19T09:29:45.289 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:44 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:44.878+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:12.908918+0000 front 2024-09-19T09:29:12.908887+0000 (oldest deadline 2024-09-19T09:29:38.808899+0000) 2024-09-19T09:29:45.331 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:45 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:45.072+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.656187+0000 front 2024-09-19T09:29:14.656171+0000 (oldest deadline 2024-09-19T09:29:40.556311+0000) 2024-09-19T09:29:45.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:45 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:45.006+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:29:45.437 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:45 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:45.017+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.519869+0000 front 2024-09-19T09:29:07.519673+0000 (oldest deadline 2024-09-19T09:29:28.619790+0000) 2024-09-19T09:29:45.438 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:45 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:45.017+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.520884+0000 front 2024-09-19T09:29:14.520831+0000 (oldest deadline 2024-09-19T09:29:38.620849+0000) 2024-09-19T09:29:45.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:45 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:45.287+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:45.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:45 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:45.287+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:29:45.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:45 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:45.330+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:29:45.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:45 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:45.330+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:29:45.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:45 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:45.437+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:29:45.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:45 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:45.437+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:29:46.291 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:45 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:45.852+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.107758+0000 front 2024-09-19T09:29:07.107785+0000 (oldest deadline 2024-09-19T09:29:31.207911+0000) 2024-09-19T09:29:46.291 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:45 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:45.852+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:12.908918+0000 front 2024-09-19T09:29:12.908887+0000 (oldest deadline 2024-09-19T09:29:38.808899+0000) 2024-09-19T09:29:46.318 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:46 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:46.045+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.656187+0000 front 2024-09-19T09:29:14.656171+0000 (oldest deadline 2024-09-19T09:29:40.556311+0000) 2024-09-19T09:29:46.318 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:46 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:46.045+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.557075+0000 front 2024-09-19T09:29:20.556914+0000 (oldest deadline 2024-09-19T09:29:45.256692+0000) 2024-09-19T09:29:46.415 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:46 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:46.032+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:29:46.416 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:46 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:46.044+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.519869+0000 front 2024-09-19T09:29:07.519673+0000 (oldest deadline 2024-09-19T09:29:28.619790+0000) 2024-09-19T09:29:46.416 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:46 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:46.044+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.520884+0000 front 2024-09-19T09:29:14.520831+0000 (oldest deadline 2024-09-19T09:29:38.620849+0000) 2024-09-19T09:29:46.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:46 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:46.289+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:46.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:46 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:46.289+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:29:46.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:46 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:46.414+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:29:46.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:46 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:46.414+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:29:46.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:46 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:46.318+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:29:46.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:46 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:46.318+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:29:47.076 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:46 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:46.811+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.107758+0000 front 2024-09-19T09:29:07.107785+0000 (oldest deadline 2024-09-19T09:29:31.207911+0000) 2024-09-19T09:29:47.076 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:46 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:46.811+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:12.908918+0000 front 2024-09-19T09:29:12.908887+0000 (oldest deadline 2024-09-19T09:29:38.808899+0000) 2024-09-19T09:29:47.339 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:47 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:47.047+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.656187+0000 front 2024-09-19T09:29:14.656171+0000 (oldest deadline 2024-09-19T09:29:40.556311+0000) 2024-09-19T09:29:47.339 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:47 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:47.047+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.557075+0000 front 2024-09-19T09:29:20.556914+0000 (oldest deadline 2024-09-19T09:29:45.256692+0000) 2024-09-19T09:29:47.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:47 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:47.054+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:29:47.437 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:47 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:47.003+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.519869+0000 front 2024-09-19T09:29:07.519673+0000 (oldest deadline 2024-09-19T09:29:28.619790+0000) 2024-09-19T09:29:47.438 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:47 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:47.003+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.520884+0000 front 2024-09-19T09:29:14.520831+0000 (oldest deadline 2024-09-19T09:29:38.620849+0000) 2024-09-19T09:29:47.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:47 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:47.253+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:47.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:47 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:47.253+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:29:47.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:47 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:47.338+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:29:47.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:47 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:47.338+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:29:47.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:47 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:47.338+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:29:47.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:47 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:47.451+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:29:47.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:47 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:47.451+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:29:47.938 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:47 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:47.451+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:29:48.076 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:47 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:47.763+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.107758+0000 front 2024-09-19T09:29:07.107785+0000 (oldest deadline 2024-09-19T09:29:31.207911+0000) 2024-09-19T09:29:48.076 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:47 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:47.763+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:12.908918+0000 front 2024-09-19T09:29:12.908887+0000 (oldest deadline 2024-09-19T09:29:38.808899+0000) 2024-09-19T09:29:48.076 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:47 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:47.763+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.210102+0000 front 2024-09-19T09:29:22.210259+0000 (oldest deadline 2024-09-19T09:29:46.910079+0000) 2024-09-19T09:29:48.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:48 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:48.007+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:29:48.437 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:48 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:48.043+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.519869+0000 front 2024-09-19T09:29:07.519673+0000 (oldest deadline 2024-09-19T09:29:28.619790+0000) 2024-09-19T09:29:48.437 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:48 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:48.043+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.520884+0000 front 2024-09-19T09:29:14.520831+0000 (oldest deadline 2024-09-19T09:29:38.620849+0000) 2024-09-19T09:29:48.437 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:48 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5[46574]: 2024-09-19T09:29:48.043+0000 7f342bfc6640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:23.822429+0000 front 2024-09-19T09:29:23.822457+0000 (oldest deadline 2024-09-19T09:29:47.322617+0000) 2024-09-19T09:29:48.438 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:48 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:48.091+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.656187+0000 front 2024-09-19T09:29:14.656171+0000 (oldest deadline 2024-09-19T09:29:40.556311+0000) 2024-09-19T09:29:48.438 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:48 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:48.091+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.557075+0000 front 2024-09-19T09:29:20.556914+0000 (oldest deadline 2024-09-19T09:29:45.256692+0000) 2024-09-19T09:29:48.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:48 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:48.330+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:29:48.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:48 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:48.330+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:29:48.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:48 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:48.330+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:29:48.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:48 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:48.258+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:48.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:48 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:48.258+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:29:48.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:48 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:48.258+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:29:48.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:48 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:48.464+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:29:48.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:48 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:48.464+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:29:48.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:48 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:48.464+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:29:49.076 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:48 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:48.796+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.107758+0000 front 2024-09-19T09:29:07.107785+0000 (oldest deadline 2024-09-19T09:29:31.207911+0000) 2024-09-19T09:29:49.076 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:48 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:48.796+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:12.908918+0000 front 2024-09-19T09:29:12.908887+0000 (oldest deadline 2024-09-19T09:29:38.808899+0000) 2024-09-19T09:29:49.076 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:48 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:48.796+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.210102+0000 front 2024-09-19T09:29:22.210259+0000 (oldest deadline 2024-09-19T09:29:46.910079+0000) 2024-09-19T09:29:49.387 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:49 smithi196 podman[72813]: 2024-09-19 09:29:49.081391442 +0000 UTC m=+5.959821505 container died d04c42c0ee6d926c27c6ee96e433ea68a989ba14e27abab38f9ce540104d4c87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:29:49.387 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:49 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:49.026+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:29:49.388 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:49 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:49.026+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:23.849908+0000 front 2024-09-19T09:29:23.849990+0000 (oldest deadline 2024-09-19T09:29:48.550062+0000) 2024-09-19T09:29:49.438 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:49 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:49.100+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.656187+0000 front 2024-09-19T09:29:14.656171+0000 (oldest deadline 2024-09-19T09:29:40.556311+0000) 2024-09-19T09:29:49.438 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:49 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:49.100+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.557075+0000 front 2024-09-19T09:29:20.556914+0000 (oldest deadline 2024-09-19T09:29:45.256692+0000) 2024-09-19T09:29:49.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:49 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:49.336+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:29:49.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:49 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:49.336+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:29:49.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:49 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:49.336+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:29:49.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:49 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:49.229+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:49.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:49 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:49.229+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:29:49.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:49 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:49.229+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:29:49.687 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:49 smithi196 podman[72813]: 2024-09-19 09:29:49.491044331 +0000 UTC m=+6.369474375 container cleanup d04c42c0ee6d926c27c6ee96e433ea68a989ba14e27abab38f9ce540104d4c87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:29:49.688 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:49 smithi196 bash[72813]: ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5 2024-09-19T09:29:49.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:49 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:49.511+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:29:49.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:49 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:49.511+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:29:49.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:49 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:49.511+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:29:50.006 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:49 smithi196 podman[72827]: 2024-09-19 09:29:49.709287957 +0000 UTC m=+0.630865417 container remove d04c42c0ee6d926c27c6ee96e433ea68a989ba14e27abab38f9ce540104d4c87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-19T09:29:50.183 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:49 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:49.836+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.107758+0000 front 2024-09-19T09:29:07.107785+0000 (oldest deadline 2024-09-19T09:29:31.207911+0000) 2024-09-19T09:29:50.183 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:49 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:49.836+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:12.908918+0000 front 2024-09-19T09:29:12.908887+0000 (oldest deadline 2024-09-19T09:29:38.808899+0000) 2024-09-19T09:29:50.183 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:49 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:49.836+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.210102+0000 front 2024-09-19T09:29:22.210259+0000 (oldest deadline 2024-09-19T09:29:46.910079+0000) 2024-09-19T09:29:50.339 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:50 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:50.005+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:29:50.339 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:50 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:50.005+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:23.849908+0000 front 2024-09-19T09:29:23.849990+0000 (oldest deadline 2024-09-19T09:29:48.550062+0000) 2024-09-19T09:29:50.339 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:50 smithi196 podman[72921]: 2024-09-19 09:29:50.068784966 +0000 UTC m=+0.015801339 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:29:50.340 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:50 smithi196 podman[72921]: 2024-09-19 09:29:50.210212147 +0000 UTC m=+0.157228508 container create fdb96faa709af890fbf1ce3aea9873925aa40e2f452ed524825a7ba84c3fff24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-19T09:29:50.377 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:50 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:50.105+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.656187+0000 front 2024-09-19T09:29:14.656171+0000 (oldest deadline 2024-09-19T09:29:40.556311+0000) 2024-09-19T09:29:50.378 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:50 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:50.105+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.557075+0000 front 2024-09-19T09:29:20.556914+0000 (oldest deadline 2024-09-19T09:29:45.256692+0000) 2024-09-19T09:29:50.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:50 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:50.182+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:50.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:50 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:50.182+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:29:50.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:50 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:50.182+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:29:50.662 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:50 smithi196 podman[72921]: 2024-09-19 09:29:50.471196573 +0000 UTC m=+0.418212933 container init fdb96faa709af890fbf1ce3aea9873925aa40e2f452ed524825a7ba84c3fff24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4) 2024-09-19T09:29:50.662 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:50 smithi196 podman[72921]: 2024-09-19 09:29:50.474593003 +0000 UTC m=+0.421609371 container start fdb96faa709af890fbf1ce3aea9873925aa40e2f452ed524825a7ba84c3fff24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:29:50.662 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:50 smithi196 podman[72921]: 2024-09-19 09:29:50.554551797 +0000 UTC m=+0.501568167 container attach fdb96faa709af890fbf1ce3aea9873925aa40e2f452ed524825a7ba84c3fff24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-19T09:29:50.662 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:50 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:50.553+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:29:50.662 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:50 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:50.553+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:29:50.662 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:50 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:50.553+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:29:50.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:50 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:50.377+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:29:50.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:50 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:50.377+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:29:50.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:50 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:50.377+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:29:50.937 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 19 09:29:50 smithi196 podman[72921]: 2024-09-19 09:29:50.661116576 +0000 UTC m=+0.608132945 container died fdb96faa709af890fbf1ce3aea9873925aa40e2f452ed524825a7ba84c3fff24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:29:51.076 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:50 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:50.813+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.107758+0000 front 2024-09-19T09:29:07.107785+0000 (oldest deadline 2024-09-19T09:29:31.207911+0000) 2024-09-19T09:29:51.076 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:50 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:50.813+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:12.908918+0000 front 2024-09-19T09:29:12.908887+0000 (oldest deadline 2024-09-19T09:29:38.808899+0000) 2024-09-19T09:29:51.076 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:50 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:50.813+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.210102+0000 front 2024-09-19T09:29:22.210259+0000 (oldest deadline 2024-09-19T09:29:46.910079+0000) 2024-09-19T09:29:51.200 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:50 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:50.996+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:29:51.200 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:50 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:50.996+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:23.849908+0000 front 2024-09-19T09:29:23.849990+0000 (oldest deadline 2024-09-19T09:29:48.550062+0000) 2024-09-19T09:29:51.226 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.5.service' 2024-09-19T09:29:51.265 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:29:51.265 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-19T09:29:51.265 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-19T09:29:51.265 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.6 2024-09-19T09:29:51.438 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:51 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:51.117+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.656187+0000 front 2024-09-19T09:29:14.656171+0000 (oldest deadline 2024-09-19T09:29:40.556311+0000) 2024-09-19T09:29:51.438 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:51 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:51.117+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.557075+0000 front 2024-09-19T09:29:20.556914+0000 (oldest deadline 2024-09-19T09:29:45.256692+0000) 2024-09-19T09:29:51.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:51 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:51.349+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:29:51.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:51 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:51.349+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:29:51.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:51 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:51.349+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:29:51.552 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:51 smithi120 systemd[1]: Stopping Ceph osd.6 for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:29:51.553 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:51 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:51.183+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:51.553 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:51 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:51.183+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:29:51.553 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:51 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:51.183+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:29:51.826 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:51 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:51.551+0000 7f50f1455640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:29:51.826 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:51 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:51.551+0000 7f50f1455640 -1 osd.6 88 *** Got signal Terminated *** 2024-09-19T09:29:51.826 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:51 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:51.551+0000 7f50f1455640 -1 osd.6 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:29:51.826 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:51 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:51.784+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.107758+0000 front 2024-09-19T09:29:07.107785+0000 (oldest deadline 2024-09-19T09:29:31.207911+0000) 2024-09-19T09:29:51.826 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:51 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:51.784+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:12.908918+0000 front 2024-09-19T09:29:12.908887+0000 (oldest deadline 2024-09-19T09:29:38.808899+0000) 2024-09-19T09:29:51.826 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:51 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:51.784+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.210102+0000 front 2024-09-19T09:29:22.210259+0000 (oldest deadline 2024-09-19T09:29:46.910079+0000) 2024-09-19T09:29:51.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:51 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:51.508+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:29:51.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:51 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:51.508+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:29:51.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:51 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:51.508+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:29:52.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:52 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:52.046+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:29:52.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:52 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:52.046+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:23.849908+0000 front 2024-09-19T09:29:23.849990+0000 (oldest deadline 2024-09-19T09:29:48.550062+0000) 2024-09-19T09:29:52.438 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:52 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:52.111+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.656187+0000 front 2024-09-19T09:29:14.656171+0000 (oldest deadline 2024-09-19T09:29:40.556311+0000) 2024-09-19T09:29:52.438 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:52 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:52.111+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.557075+0000 front 2024-09-19T09:29:20.556914+0000 (oldest deadline 2024-09-19T09:29:45.256692+0000) 2024-09-19T09:29:52.438 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:52 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:52.310+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:29:52.438 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:52 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:52.310+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:29:52.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:52 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:52.310+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:29:52.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:52 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:52.176+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:52.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:52 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:52.176+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:29:52.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:52 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:52.177+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:29:52.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:52 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:52.534+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:29:52.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:52 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:52.534+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:29:52.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:52 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:52.534+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:29:53.076 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:52 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:52.792+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.107758+0000 front 2024-09-19T09:29:07.107785+0000 (oldest deadline 2024-09-19T09:29:31.207911+0000) 2024-09-19T09:29:53.076 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:52 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:52.792+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:12.908918+0000 front 2024-09-19T09:29:12.908887+0000 (oldest deadline 2024-09-19T09:29:38.808899+0000) 2024-09-19T09:29:53.076 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:52 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:52.792+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.210102+0000 front 2024-09-19T09:29:22.210259+0000 (oldest deadline 2024-09-19T09:29:46.910079+0000) 2024-09-19T09:29:53.076 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:52 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:52.792+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:26.910448+0000 front 2024-09-19T09:29:26.910429+0000 (oldest deadline 2024-09-19T09:29:52.711074+0000) 2024-09-19T09:29:53.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:53 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:53.093+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:29:53.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:53 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:53.093+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:23.849908+0000 front 2024-09-19T09:29:23.849990+0000 (oldest deadline 2024-09-19T09:29:48.550062+0000) 2024-09-19T09:29:53.438 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:53 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:53.093+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:28.550831+0000 front 2024-09-19T09:29:28.550655+0000 (oldest deadline 2024-09-19T09:29:52.650832+0000) 2024-09-19T09:29:53.438 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:53 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:53.136+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.656187+0000 front 2024-09-19T09:29:14.656171+0000 (oldest deadline 2024-09-19T09:29:40.556311+0000) 2024-09-19T09:29:53.438 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:53 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:53.136+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.557075+0000 front 2024-09-19T09:29:20.556914+0000 (oldest deadline 2024-09-19T09:29:45.256692+0000) 2024-09-19T09:29:53.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:53 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:53.296+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:29:53.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:53 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:53.296+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:29:53.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:53 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:53.296+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:29:53.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:53 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:53.296+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:29:53.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:53 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:53.182+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:53.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:53 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:53.182+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:29:53.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:53 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:53.182+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:29:53.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:53 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:53.571+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:29:53.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:53 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:53.571+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:29:53.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:53 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:53.571+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:29:54.077 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:53 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:53.809+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.107758+0000 front 2024-09-19T09:29:07.107785+0000 (oldest deadline 2024-09-19T09:29:31.207911+0000) 2024-09-19T09:29:54.077 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:53 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:53.809+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:12.908918+0000 front 2024-09-19T09:29:12.908887+0000 (oldest deadline 2024-09-19T09:29:38.808899+0000) 2024-09-19T09:29:54.077 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:53 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:53.809+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.210102+0000 front 2024-09-19T09:29:22.210259+0000 (oldest deadline 2024-09-19T09:29:46.910079+0000) 2024-09-19T09:29:54.077 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:53 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:53.809+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:26.910448+0000 front 2024-09-19T09:29:26.910429+0000 (oldest deadline 2024-09-19T09:29:52.711074+0000) 2024-09-19T09:29:54.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:54 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:54.066+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:29:54.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:54 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:54.066+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:23.849908+0000 front 2024-09-19T09:29:23.849990+0000 (oldest deadline 2024-09-19T09:29:48.550062+0000) 2024-09-19T09:29:54.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:54 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:54.066+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:28.550831+0000 front 2024-09-19T09:29:28.550655+0000 (oldest deadline 2024-09-19T09:29:52.650832+0000) 2024-09-19T09:29:54.438 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:54 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:54.103+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.656187+0000 front 2024-09-19T09:29:14.656171+0000 (oldest deadline 2024-09-19T09:29:40.556311+0000) 2024-09-19T09:29:54.438 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:54 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:54.103+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.557075+0000 front 2024-09-19T09:29:20.556914+0000 (oldest deadline 2024-09-19T09:29:45.256692+0000) 2024-09-19T09:29:54.438 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:54 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:54.345+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:29:54.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:54 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:54.345+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:29:54.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:54 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:54.345+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:29:54.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:54 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:54.345+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:29:54.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:54 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:54.167+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:54.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:54 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:54.167+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:29:54.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:54 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:54.167+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:29:54.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:54 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:54.522+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:29:54.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:54 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:54.522+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:29:54.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:54 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:54.522+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:29:55.076 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:54 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:54.816+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.107758+0000 front 2024-09-19T09:29:07.107785+0000 (oldest deadline 2024-09-19T09:29:31.207911+0000) 2024-09-19T09:29:55.076 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:54 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:54.816+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:12.908918+0000 front 2024-09-19T09:29:12.908887+0000 (oldest deadline 2024-09-19T09:29:38.808899+0000) 2024-09-19T09:29:55.076 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:54 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:54.816+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.210102+0000 front 2024-09-19T09:29:22.210259+0000 (oldest deadline 2024-09-19T09:29:46.910079+0000) 2024-09-19T09:29:55.076 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:54 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:54.816+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:26.910448+0000 front 2024-09-19T09:29:26.910429+0000 (oldest deadline 2024-09-19T09:29:52.711074+0000) 2024-09-19T09:29:55.372 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:55 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:55.119+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.656187+0000 front 2024-09-19T09:29:14.656171+0000 (oldest deadline 2024-09-19T09:29:40.556311+0000) 2024-09-19T09:29:55.373 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:55 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:55.119+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.557075+0000 front 2024-09-19T09:29:20.556914+0000 (oldest deadline 2024-09-19T09:29:45.256692+0000) 2024-09-19T09:29:55.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:55 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:55.073+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:29:55.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:55 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:55.073+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:23.849908+0000 front 2024-09-19T09:29:23.849990+0000 (oldest deadline 2024-09-19T09:29:48.550062+0000) 2024-09-19T09:29:55.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:55 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:55.073+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:28.550831+0000 front 2024-09-19T09:29:28.550655+0000 (oldest deadline 2024-09-19T09:29:52.650832+0000) 2024-09-19T09:29:55.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:55 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:55.141+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:55.631 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:55 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:55.141+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:29:55.631 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:55 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:55.141+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:29:55.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:55 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:55.372+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:29:55.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:55 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:55.372+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:29:55.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:55 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:55.372+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:29:55.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:55 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:55.372+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:29:55.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:55 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:55.513+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:29:55.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:55 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:55.513+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:29:55.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:55 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:55.513+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:29:56.141 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:55 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:55.840+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.107758+0000 front 2024-09-19T09:29:07.107785+0000 (oldest deadline 2024-09-19T09:29:31.207911+0000) 2024-09-19T09:29:56.141 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:55 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:55.840+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:12.908918+0000 front 2024-09-19T09:29:12.908887+0000 (oldest deadline 2024-09-19T09:29:38.808899+0000) 2024-09-19T09:29:56.141 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:55 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:55.840+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.210102+0000 front 2024-09-19T09:29:22.210259+0000 (oldest deadline 2024-09-19T09:29:46.910079+0000) 2024-09-19T09:29:56.142 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:55 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6[58561]: 2024-09-19T09:29:55.840+0000 7f50eda6b640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:26.910448+0000 front 2024-09-19T09:29:26.910429+0000 (oldest deadline 2024-09-19T09:29:52.711074+0000) 2024-09-19T09:29:56.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:56 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:56.031+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:29:56.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:56 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:56.031+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:23.849908+0000 front 2024-09-19T09:29:23.849990+0000 (oldest deadline 2024-09-19T09:29:48.550062+0000) 2024-09-19T09:29:56.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:56 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:56.031+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:28.550831+0000 front 2024-09-19T09:29:28.550655+0000 (oldest deadline 2024-09-19T09:29:52.650832+0000) 2024-09-19T09:29:56.438 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:56 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:56.120+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.656187+0000 front 2024-09-19T09:29:14.656171+0000 (oldest deadline 2024-09-19T09:29:40.556311+0000) 2024-09-19T09:29:56.438 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:56 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:56.120+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.557075+0000 front 2024-09-19T09:29:20.556914+0000 (oldest deadline 2024-09-19T09:29:45.256692+0000) 2024-09-19T09:29:56.438 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:56 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:56.120+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.557784+0000 front 2024-09-19T09:29:30.557864+0000 (oldest deadline 2024-09-19T09:29:55.857873+0000) 2024-09-19T09:29:56.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:56 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:56.344+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:29:56.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:56 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:56.344+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:29:56.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:56 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:56.344+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:29:56.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:56 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:56.344+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:29:56.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:56 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:56.140+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:56.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:56 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:56.140+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:29:56.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:56 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:56.140+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:29:56.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:56 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:56.485+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:29:56.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:56 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:56.485+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:29:56.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:56 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:56.485+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:29:57.295 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:56 smithi120 podman[152879]: 2024-09-19 09:29:56.988436573 +0000 UTC m=+5.528816894 container died 3b2af5d32d050335b47f7743ea69b90047e1bfbfb926d05369bcd948c9b27cbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-19T09:29:57.295 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:57 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:57.099+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:57.295 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:57 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:57.099+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:29:57.295 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:57 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:57.099+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:29:57.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:57 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:57.010+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:29:57.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:57 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:57.010+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:23.849908+0000 front 2024-09-19T09:29:23.849990+0000 (oldest deadline 2024-09-19T09:29:48.550062+0000) 2024-09-19T09:29:57.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:57 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:57.010+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:28.550831+0000 front 2024-09-19T09:29:28.550655+0000 (oldest deadline 2024-09-19T09:29:52.650832+0000) 2024-09-19T09:29:57.438 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:57 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:57.078+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.656187+0000 front 2024-09-19T09:29:14.656171+0000 (oldest deadline 2024-09-19T09:29:40.556311+0000) 2024-09-19T09:29:57.438 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:57 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:57.078+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.557075+0000 front 2024-09-19T09:29:20.556914+0000 (oldest deadline 2024-09-19T09:29:45.256692+0000) 2024-09-19T09:29:57.438 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:57 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:57.078+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.557784+0000 front 2024-09-19T09:29:30.557864+0000 (oldest deadline 2024-09-19T09:29:55.857873+0000) 2024-09-19T09:29:57.438 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:57 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:57.299+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:29:57.438 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:57 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:57.299+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:29:57.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:57 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:57.299+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:29:57.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:57 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:57.299+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:29:57.576 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:57 smithi120 podman[152879]: 2024-09-19 09:29:57.493948655 +0000 UTC m=+6.034329021 container cleanup 3b2af5d32d050335b47f7743ea69b90047e1bfbfb926d05369bcd948c9b27cbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-19T09:29:57.576 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:57 smithi120 bash[152879]: ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6 2024-09-19T09:29:57.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:57 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:57.518+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:29:57.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:57 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:57.518+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:29:57.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:57 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:57.518+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:29:58.076 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:57 smithi120 podman[152894]: 2024-09-19 09:29:57.695566602 +0000 UTC m=+0.702933479 container remove 3b2af5d32d050335b47f7743ea69b90047e1bfbfb926d05369bcd948c9b27cbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-19T09:29:58.400 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:58 smithi120 podman[152987]: 2024-09-19 09:29:58.010469429 +0000 UTC m=+0.015728515 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:29:58.400 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:58 smithi120 podman[152987]: 2024-09-19 09:29:58.138317663 +0000 UTC m=+0.143576744 container create 4e73b75ed7ffeb17818d5f65cf3aa172c56a21c861cef2a9e2e7e5be43877a4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-19T09:29:58.400 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:58 smithi120 podman[152987]: 2024-09-19 09:29:58.399386697 +0000 UTC m=+0.404645779 container init 4e73b75ed7ffeb17818d5f65cf3aa172c56a21c861cef2a9e2e7e5be43877a4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-19T09:29:58.400 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:58 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:58.142+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:58.400 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:58 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:58.142+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:29:58.400 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:58 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:58.142+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:29:58.401 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:58 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:58.142+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:31.438536+0000 front 2024-09-19T09:29:31.438561+0000 (oldest deadline 2024-09-19T09:29:57.338695+0000) 2024-09-19T09:29:58.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:58 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:58.056+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:29:58.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:58 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:58.056+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:23.849908+0000 front 2024-09-19T09:29:23.849990+0000 (oldest deadline 2024-09-19T09:29:48.550062+0000) 2024-09-19T09:29:58.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:58 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:58.056+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:28.550831+0000 front 2024-09-19T09:29:28.550655+0000 (oldest deadline 2024-09-19T09:29:52.650832+0000) 2024-09-19T09:29:58.438 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:58 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:58.110+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.656187+0000 front 2024-09-19T09:29:14.656171+0000 (oldest deadline 2024-09-19T09:29:40.556311+0000) 2024-09-19T09:29:58.438 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:58 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:58.110+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.557075+0000 front 2024-09-19T09:29:20.556914+0000 (oldest deadline 2024-09-19T09:29:45.256692+0000) 2024-09-19T09:29:58.438 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:58 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:58.110+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.557784+0000 front 2024-09-19T09:29:30.557864+0000 (oldest deadline 2024-09-19T09:29:55.857873+0000) 2024-09-19T09:29:58.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:58 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:58.278+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:29:58.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:58 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:58.278+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:29:58.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:58 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:58.278+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:29:58.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:58 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:58.278+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:29:58.826 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:58 smithi120 podman[152987]: 2024-09-19 09:29:58.402806237 +0000 UTC m=+0.408065321 container start 4e73b75ed7ffeb17818d5f65cf3aa172c56a21c861cef2a9e2e7e5be43877a4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:29:58.826 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:58 smithi120 podman[152987]: 2024-09-19 09:29:58.491109489 +0000 UTC m=+0.496368571 container attach 4e73b75ed7ffeb17818d5f65cf3aa172c56a21c861cef2a9e2e7e5be43877a4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux ) 2024-09-19T09:29:58.826 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 19 09:29:58 smithi120 podman[152987]: 2024-09-19 09:29:58.589995163 +0000 UTC m=+0.595254252 container died 4e73b75ed7ffeb17818d5f65cf3aa172c56a21c861cef2a9e2e7e5be43877a4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-6-deactivate, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-19T09:29:58.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:58 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:58.568+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:29:58.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:58 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:58.568+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:29:58.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:58 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:58.568+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:29:59.196 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:59 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:59.096+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:29:59.196 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:59 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:59.096+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:29:59.196 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:59 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:59.096+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:29:59.197 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:29:59 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:29:59.096+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:31.438536+0000 front 2024-09-19T09:29:31.438561+0000 (oldest deadline 2024-09-19T09:29:57.338695+0000) 2024-09-19T09:29:59.216 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.6.service' 2024-09-19T09:29:59.255 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:29:59.255 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-19T09:29:59.255 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-19T09:29:59.256 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.7 2024-09-19T09:29:59.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:59 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:59.037+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:29:59.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:59 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:59.037+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:23.849908+0000 front 2024-09-19T09:29:23.849990+0000 (oldest deadline 2024-09-19T09:29:48.550062+0000) 2024-09-19T09:29:59.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:29:59 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:29:59.037+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:28.550831+0000 front 2024-09-19T09:29:28.550655+0000 (oldest deadline 2024-09-19T09:29:52.650832+0000) 2024-09-19T09:29:59.438 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:59 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:59.144+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.656187+0000 front 2024-09-19T09:29:14.656171+0000 (oldest deadline 2024-09-19T09:29:40.556311+0000) 2024-09-19T09:29:59.438 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:59 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:59.144+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.557075+0000 front 2024-09-19T09:29:20.556914+0000 (oldest deadline 2024-09-19T09:29:45.256692+0000) 2024-09-19T09:29:59.438 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:59 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:59.144+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.557784+0000 front 2024-09-19T09:29:30.557864+0000 (oldest deadline 2024-09-19T09:29:55.857873+0000) 2024-09-19T09:29:59.439 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:59 smithi153 systemd[1]: Stopping Ceph osd.7 for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:29:59.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:59 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:59.324+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:29:59.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:59 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:59.324+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:29:59.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:59 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:59.324+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:29:59.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:29:59 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:29:59.324+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:29:59.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:59 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:59.531+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:29:59.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:59 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:59.531+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:29:59.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:29:59 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:29:59.531+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:29:59.938 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:59 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:59.561+0000 7f840d064640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:29:59.938 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:59 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:59.561+0000 7f840d064640 -1 osd.7 88 *** Got signal Terminated *** 2024-09-19T09:29:59.938 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:29:59 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:29:59.561+0000 7f840d064640 -1 osd.7 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:30:00.417 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:00 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:00.143+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:30:00.417 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:00 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:00.143+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:30:00.417 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:00 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:00.143+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:30:00.418 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:00 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:00.143+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:31.438536+0000 front 2024-09-19T09:29:31.438561+0000 (oldest deadline 2024-09-19T09:29:57.338695+0000) 2024-09-19T09:30:00.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:00 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:00.074+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:30:00.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:00 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:00.074+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:23.849908+0000 front 2024-09-19T09:29:23.849990+0000 (oldest deadline 2024-09-19T09:29:48.550062+0000) 2024-09-19T09:30:00.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:00 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:00.074+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:28.550831+0000 front 2024-09-19T09:29:28.550655+0000 (oldest deadline 2024-09-19T09:29:52.650832+0000) 2024-09-19T09:30:00.689 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:30:00 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:30:00.189+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.656187+0000 front 2024-09-19T09:29:14.656171+0000 (oldest deadline 2024-09-19T09:29:40.556311+0000) 2024-09-19T09:30:00.689 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:30:00 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:30:00.189+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.557075+0000 front 2024-09-19T09:29:20.556914+0000 (oldest deadline 2024-09-19T09:29:45.256692+0000) 2024-09-19T09:30:00.689 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:30:00 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:30:00.189+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.557784+0000 front 2024-09-19T09:29:30.557864+0000 (oldest deadline 2024-09-19T09:29:55.857873+0000) 2024-09-19T09:30:00.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:00 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:00.331+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:30:00.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:00 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:00.331+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:30:00.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:00 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:00.331+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:30:00.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:00 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:00.331+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:30:00.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:00 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:00.535+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:00.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:00 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:00.535+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:00.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:00 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:00.535+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:01.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:01 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:01.083+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:30:01.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:01 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:01.083+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:23.849908+0000 front 2024-09-19T09:29:23.849990+0000 (oldest deadline 2024-09-19T09:29:48.550062+0000) 2024-09-19T09:30:01.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:01 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:01.083+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:28.550831+0000 front 2024-09-19T09:29:28.550655+0000 (oldest deadline 2024-09-19T09:29:52.650832+0000) 2024-09-19T09:30:01.438 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:01 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:01.330+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:30:01.438 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:01 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:01.330+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:30:01.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:01 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:01.330+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:30:01.439 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:01 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:01.330+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:30:01.439 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:30:01 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:30:01.192+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.656187+0000 front 2024-09-19T09:29:14.656171+0000 (oldest deadline 2024-09-19T09:29:40.556311+0000) 2024-09-19T09:30:01.439 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:30:01 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:30:01.192+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.557075+0000 front 2024-09-19T09:29:20.556914+0000 (oldest deadline 2024-09-19T09:29:45.256692+0000) 2024-09-19T09:30:01.439 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:30:01 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:30:01.192+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.557784+0000 front 2024-09-19T09:29:30.557864+0000 (oldest deadline 2024-09-19T09:29:55.857873+0000) 2024-09-19T09:30:01.439 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:30:01 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:30:01.192+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:39.959015+0000 front 2024-09-19T09:29:39.959038+0000 (oldest deadline 2024-09-19T09:30:01.059155+0000) 2024-09-19T09:30:01.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:01 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:01.190+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:30:01.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:01 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:01.190+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:30:01.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:01 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:01.190+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:30:01.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:01 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:01.190+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:31.438536+0000 front 2024-09-19T09:29:31.438561+0000 (oldest deadline 2024-09-19T09:29:57.338695+0000) 2024-09-19T09:30:01.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:01 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:01.568+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:01.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:01 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:01.568+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:01.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:01 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:01.568+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:01.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:01 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:01.568+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:02.438 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:02 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:02.088+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:30:02.438 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:02 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:02.088+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:23.849908+0000 front 2024-09-19T09:29:23.849990+0000 (oldest deadline 2024-09-19T09:29:48.550062+0000) 2024-09-19T09:30:02.438 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:02 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:02.088+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:28.550831+0000 front 2024-09-19T09:29:28.550655+0000 (oldest deadline 2024-09-19T09:29:52.650832+0000) 2024-09-19T09:30:02.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:02 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:02.240+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:30:02.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:02 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:02.240+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:30:02.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:02 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:02.240+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:30:02.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:02 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:02.240+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:31.438536+0000 front 2024-09-19T09:29:31.438561+0000 (oldest deadline 2024-09-19T09:29:57.338695+0000) 2024-09-19T09:30:02.688 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:30:02 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:30:02.240+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.656187+0000 front 2024-09-19T09:29:14.656171+0000 (oldest deadline 2024-09-19T09:29:40.556311+0000) 2024-09-19T09:30:02.689 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:30:02 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:30:02.240+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.557075+0000 front 2024-09-19T09:29:20.556914+0000 (oldest deadline 2024-09-19T09:29:45.256692+0000) 2024-09-19T09:30:02.689 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:30:02 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:30:02.240+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.557784+0000 front 2024-09-19T09:29:30.557864+0000 (oldest deadline 2024-09-19T09:29:55.857873+0000) 2024-09-19T09:30:02.689 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:30:02 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:30:02.240+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:39.959015+0000 front 2024-09-19T09:29:39.959038+0000 (oldest deadline 2024-09-19T09:30:01.059155+0000) 2024-09-19T09:30:02.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:02 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:02.288+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:30:02.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:02 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:02.288+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:30:02.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:02 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:02.288+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:30:02.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:02 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:02.288+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:30:02.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:02 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:02.587+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:02.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:02 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:02.587+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:02.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:02 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:02.587+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:02.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:02 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:02.587+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:03.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:03 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:03.090+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:30:03.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:03 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:03.090+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:23.849908+0000 front 2024-09-19T09:29:23.849990+0000 (oldest deadline 2024-09-19T09:29:48.550062+0000) 2024-09-19T09:30:03.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:03 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:03.090+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:28.550831+0000 front 2024-09-19T09:29:28.550655+0000 (oldest deadline 2024-09-19T09:29:52.650832+0000) 2024-09-19T09:30:03.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:03 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:03.242+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:30:03.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:03 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:03.242+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:30:03.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:03 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:03.242+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:30:03.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:03 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:03.242+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:31.438536+0000 front 2024-09-19T09:29:31.438561+0000 (oldest deadline 2024-09-19T09:29:57.338695+0000) 2024-09-19T09:30:03.688 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:30:03 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:30:03.271+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.656187+0000 front 2024-09-19T09:29:14.656171+0000 (oldest deadline 2024-09-19T09:29:40.556311+0000) 2024-09-19T09:30:03.688 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:30:03 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:30:03.271+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.557075+0000 front 2024-09-19T09:29:20.556914+0000 (oldest deadline 2024-09-19T09:29:45.256692+0000) 2024-09-19T09:30:03.689 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:30:03 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:30:03.271+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.557784+0000 front 2024-09-19T09:29:30.557864+0000 (oldest deadline 2024-09-19T09:29:55.857873+0000) 2024-09-19T09:30:03.689 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:30:03 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:30:03.271+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:39.959015+0000 front 2024-09-19T09:29:39.959038+0000 (oldest deadline 2024-09-19T09:30:01.059155+0000) 2024-09-19T09:30:03.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:03 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:03.266+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:30:03.693 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:03 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:03.266+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:30:03.693 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:03 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:03.266+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:30:03.693 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:03 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:03.266+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:30:03.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:03 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:03.557+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:03.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:03 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:03.557+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:03.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:03 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:03.557+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:03.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:03 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:03.557+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:04.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:04 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:04.099+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:30:04.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:04 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:04.099+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:23.849908+0000 front 2024-09-19T09:29:23.849990+0000 (oldest deadline 2024-09-19T09:29:48.550062+0000) 2024-09-19T09:30:04.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:04 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:04.099+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:28.550831+0000 front 2024-09-19T09:29:28.550655+0000 (oldest deadline 2024-09-19T09:29:52.650832+0000) 2024-09-19T09:30:04.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:04 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:04.099+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:39.052473+0000 front 2024-09-19T09:29:39.052579+0000 (oldest deadline 2024-09-19T09:30:03.752464+0000) 2024-09-19T09:30:04.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:04 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:04.219+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:30:04.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:04 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:04.219+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:30:04.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:04 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:04.219+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:30:04.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:04 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:04.219+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:31.438536+0000 front 2024-09-19T09:29:31.438561+0000 (oldest deadline 2024-09-19T09:29:57.338695+0000) 2024-09-19T09:30:04.688 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:30:04 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:30:04.260+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.656187+0000 front 2024-09-19T09:29:14.656171+0000 (oldest deadline 2024-09-19T09:29:40.556311+0000) 2024-09-19T09:30:04.689 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:30:04 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:30:04.260+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.557075+0000 front 2024-09-19T09:29:20.556914+0000 (oldest deadline 2024-09-19T09:29:45.256692+0000) 2024-09-19T09:30:04.689 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:30:04 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:30:04.260+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.557784+0000 front 2024-09-19T09:29:30.557864+0000 (oldest deadline 2024-09-19T09:29:55.857873+0000) 2024-09-19T09:30:04.689 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:30:04 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7[52469]: 2024-09-19T09:30:04.260+0000 7f840967a640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:39.959015+0000 front 2024-09-19T09:29:39.959038+0000 (oldest deadline 2024-09-19T09:30:01.059155+0000) 2024-09-19T09:30:04.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:04 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:04.238+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:30:04.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:04 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:04.238+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:30:04.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:04 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:04.238+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:30:04.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:04 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:04.238+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:30:04.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:04 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:04.238+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:40.122552+0000 front 2024-09-19T09:29:40.122578+0000 (oldest deadline 2024-09-19T09:30:03.523459+0000) 2024-09-19T09:30:04.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:04 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:04.525+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:04.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:04 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:04.525+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:04.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:04 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:04.525+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:04.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:04 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:04.525+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:05.278 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:30:05 smithi153 podman[73292]: 2024-09-19 09:30:05.025497038 +0000 UTC m=+5.578608671 container died 217858a438cae2291eca2e810ebc9dce050d27f09a37366b1e4a21506a694d58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731) 2024-09-19T09:30:05.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:05 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:05.095+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:30:05.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:05 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:05.095+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:23.849908+0000 front 2024-09-19T09:29:23.849990+0000 (oldest deadline 2024-09-19T09:29:48.550062+0000) 2024-09-19T09:30:05.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:05 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:05.095+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:28.550831+0000 front 2024-09-19T09:29:28.550655+0000 (oldest deadline 2024-09-19T09:29:52.650832+0000) 2024-09-19T09:30:05.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:05 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:05.095+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:39.052473+0000 front 2024-09-19T09:29:39.052579+0000 (oldest deadline 2024-09-19T09:30:03.752464+0000) 2024-09-19T09:30:05.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:05 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:05.187+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:30:05.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:05 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:05.187+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:30:05.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:05 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:05.187+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:30:05.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:05 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:05.187+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:31.438536+0000 front 2024-09-19T09:29:31.438561+0000 (oldest deadline 2024-09-19T09:29:57.338695+0000) 2024-09-19T09:30:05.680 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:30:05 smithi153 podman[73292]: 2024-09-19 09:30:05.419738644 +0000 UTC m=+5.972850266 container cleanup 217858a438cae2291eca2e810ebc9dce050d27f09a37366b1e4a21506a694d58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-19T09:30:05.680 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:30:05 smithi153 bash[73292]: ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7 2024-09-19T09:30:05.680 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:05 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:05.276+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:30:05.680 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:05 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:05.276+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:30:05.680 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:05 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:05.276+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:30:05.680 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:05 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:05.276+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:30:05.680 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:05 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:05.276+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:40.122552+0000 front 2024-09-19T09:29:40.122578+0000 (oldest deadline 2024-09-19T09:30:03.523459+0000) 2024-09-19T09:30:05.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:05 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:05.529+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:05.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:05 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:05.529+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:05.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:05 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:05.529+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:05.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:05 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:05.529+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:05.938 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:30:05 smithi153 podman[73312]: 2024-09-19 09:30:05.679364732 +0000 UTC m=+0.648132038 container remove 217858a438cae2291eca2e810ebc9dce050d27f09a37366b1e4a21506a694d58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-09-19T09:30:06.298 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:30:06 smithi153 podman[73403]: 2024-09-19 09:30:05.930859734 +0000 UTC m=+0.015842374 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:30:06.298 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:30:06 smithi153 podman[73403]: 2024-09-19 09:30:06.063870334 +0000 UTC m=+0.148853027 container create ad186ac7f0f6f112e08b45db3612f78cca06ebb04a80d5f2d74560f8060a4335 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-19T09:30:06.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:06 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:06.131+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:30:06.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:06 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:06.131+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:23.849908+0000 front 2024-09-19T09:29:23.849990+0000 (oldest deadline 2024-09-19T09:29:48.550062+0000) 2024-09-19T09:30:06.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:06 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:06.131+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:28.550831+0000 front 2024-09-19T09:29:28.550655+0000 (oldest deadline 2024-09-19T09:29:52.650832+0000) 2024-09-19T09:30:06.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:06 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:06.131+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:39.052473+0000 front 2024-09-19T09:29:39.052579+0000 (oldest deadline 2024-09-19T09:30:03.752464+0000) 2024-09-19T09:30:06.554 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:30:06 smithi153 podman[73403]: 2024-09-19 09:30:06.334071633 +0000 UTC m=+0.419054261 container init ad186ac7f0f6f112e08b45db3612f78cca06ebb04a80d5f2d74560f8060a4335 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2) 2024-09-19T09:30:06.554 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:30:06 smithi153 podman[73403]: 2024-09-19 09:30:06.337419399 +0000 UTC m=+0.422402027 container start ad186ac7f0f6f112e08b45db3612f78cca06ebb04a80d5f2d74560f8060a4335 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:30:06.554 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:30:06 smithi153 podman[73403]: 2024-09-19 09:30:06.416096207 +0000 UTC m=+0.501078853 container attach ad186ac7f0f6f112e08b45db3612f78cca06ebb04a80d5f2d74560f8060a4335 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-19T09:30:06.555 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:06 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:06.297+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:30:06.555 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:06 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:06.297+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:30:06.555 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:06 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:06.297+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:30:06.555 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:06 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:06.297+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:30:06.555 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:06 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:06.297+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:40.122552+0000 front 2024-09-19T09:29:40.122578+0000 (oldest deadline 2024-09-19T09:30:03.523459+0000) 2024-09-19T09:30:06.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:06 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:06.206+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:30:06.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:06 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:06.206+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:30:06.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:06 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:06.206+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:30:06.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:06 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:06.206+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:31.438536+0000 front 2024-09-19T09:29:31.438561+0000 (oldest deadline 2024-09-19T09:29:57.338695+0000) 2024-09-19T09:30:06.869 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 19 09:30:06 smithi153 podman[73426]: 2024-09-19 09:30:06.553875822 +0000 UTC m=+0.016867455 container died ad186ac7f0f6f112e08b45db3612f78cca06ebb04a80d5f2d74560f8060a4335 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-7-deactivate, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-19T09:30:06.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:06 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:06.541+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:06.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:06 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:06.541+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:06.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:06 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:06.541+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:06.938 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:06 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:06.541+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:07.183 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.7.service' 2024-09-19T09:30:07.224 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:30:07.224 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-19T09:30:07.225 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-19T09:30:07.225 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.8 2024-09-19T09:30:07.431 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:07 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:07.264+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:30:07.431 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:07 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:07.264+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:30:07.431 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:07 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:07.264+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:30:07.431 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:07 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:07.264+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:30:07.432 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:07 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:07.264+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:40.122552+0000 front 2024-09-19T09:29:40.122578+0000 (oldest deadline 2024-09-19T09:30:03.523459+0000) 2024-09-19T09:30:07.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:07 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:07.128+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:30:07.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:07 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:07.128+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:23.849908+0000 front 2024-09-19T09:29:23.849990+0000 (oldest deadline 2024-09-19T09:29:48.550062+0000) 2024-09-19T09:30:07.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:07 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:07.128+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:28.550831+0000 front 2024-09-19T09:29:28.550655+0000 (oldest deadline 2024-09-19T09:29:52.650832+0000) 2024-09-19T09:30:07.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:07 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:07.128+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:39.052473+0000 front 2024-09-19T09:29:39.052579+0000 (oldest deadline 2024-09-19T09:30:03.752464+0000) 2024-09-19T09:30:07.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:07 smithi196 systemd[1]: Stopping Ceph osd.8 for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:30:07.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:07 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:07.238+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:30:07.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:07 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:07.238+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:30:07.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:07 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:07.238+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:30:07.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:07 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:07.238+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:31.438536+0000 front 2024-09-19T09:29:31.438561+0000 (oldest deadline 2024-09-19T09:29:57.338695+0000) 2024-09-19T09:30:07.937 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:07 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:07.551+0000 7fc1b656e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:30:07.937 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:07 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:07.551+0000 7fc1b656e640 -1 osd.8 88 *** Got signal Terminated *** 2024-09-19T09:30:07.937 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:07 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:07.551+0000 7fc1b656e640 -1 osd.8 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:30:07.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:07 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:07.551+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:07.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:07 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:07.551+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:07.938 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:07 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:07.551+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:07.938 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:07 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:07.551+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:08.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:08 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:08.174+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:30:08.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:08 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:08.174+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:23.849908+0000 front 2024-09-19T09:29:23.849990+0000 (oldest deadline 2024-09-19T09:29:48.550062+0000) 2024-09-19T09:30:08.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:08 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:08.174+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:28.550831+0000 front 2024-09-19T09:29:28.550655+0000 (oldest deadline 2024-09-19T09:29:52.650832+0000) 2024-09-19T09:30:08.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:08 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:08.174+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:39.052473+0000 front 2024-09-19T09:29:39.052579+0000 (oldest deadline 2024-09-19T09:30:03.752464+0000) 2024-09-19T09:30:08.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:08 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:08.192+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:30:08.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:08 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:08.192+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:30:08.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:08 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:08.192+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:30:08.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:08 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:08.192+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:31.438536+0000 front 2024-09-19T09:29:31.438561+0000 (oldest deadline 2024-09-19T09:29:57.338695+0000) 2024-09-19T09:30:08.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:08 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:08.273+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:30:08.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:08 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:08.273+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:30:08.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:08 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:08.273+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:30:08.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:08 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:08.273+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:30:08.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:08 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:08.273+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:40.122552+0000 front 2024-09-19T09:29:40.122578+0000 (oldest deadline 2024-09-19T09:30:03.523459+0000) 2024-09-19T09:30:08.714 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:08 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:08.525+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:08.714 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:08 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:08.525+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:08.714 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:08 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:08.525+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:08.714 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:08 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:08.525+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:09.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:09 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:09.134+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:30:09.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:09 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:09.134+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:23.849908+0000 front 2024-09-19T09:29:23.849990+0000 (oldest deadline 2024-09-19T09:29:48.550062+0000) 2024-09-19T09:30:09.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:09 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:09.134+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:28.550831+0000 front 2024-09-19T09:29:28.550655+0000 (oldest deadline 2024-09-19T09:29:52.650832+0000) 2024-09-19T09:30:09.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:09 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:09.134+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:39.052473+0000 front 2024-09-19T09:29:39.052579+0000 (oldest deadline 2024-09-19T09:30:03.752464+0000) 2024-09-19T09:30:09.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:09 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:09.206+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:30:09.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:09 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:09.206+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:30:09.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:09 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:09.206+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:30:09.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:09 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:09.206+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:31.438536+0000 front 2024-09-19T09:29:31.438561+0000 (oldest deadline 2024-09-19T09:29:57.338695+0000) 2024-09-19T09:30:09.590 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:09 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:09.233+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:30:09.590 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:09 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:09.233+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:30:09.590 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:09 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:09.233+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:30:09.590 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:09 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:09.233+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:30:09.590 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:09 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:09.233+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:40.122552+0000 front 2024-09-19T09:29:40.122578+0000 (oldest deadline 2024-09-19T09:30:03.523459+0000) 2024-09-19T09:30:09.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:09 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:09.547+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:09.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:09 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:09.547+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:09.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:09 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:09.547+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:09.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:09 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:09.547+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:10.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:10 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:10.174+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:30:10.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:10 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:10.174+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:30:10.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:10 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:10.174+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:30:10.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:10 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:10.174+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:31.438536+0000 front 2024-09-19T09:29:31.438561+0000 (oldest deadline 2024-09-19T09:29:57.338695+0000) 2024-09-19T09:30:10.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:10 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:10.108+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:30:10.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:10 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:10.108+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:23.849908+0000 front 2024-09-19T09:29:23.849990+0000 (oldest deadline 2024-09-19T09:29:48.550062+0000) 2024-09-19T09:30:10.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:10 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:10.108+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:28.550831+0000 front 2024-09-19T09:29:28.550655+0000 (oldest deadline 2024-09-19T09:29:52.650832+0000) 2024-09-19T09:30:10.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:10 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:10.108+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:39.052473+0000 front 2024-09-19T09:29:39.052579+0000 (oldest deadline 2024-09-19T09:30:03.752464+0000) 2024-09-19T09:30:10.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:10 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:10.268+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:30:10.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:10 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:10.268+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:30:10.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:10 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:10.268+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:30:10.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:10 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:10.268+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:30:10.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:10 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:10.268+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:40.122552+0000 front 2024-09-19T09:29:40.122578+0000 (oldest deadline 2024-09-19T09:30:03.523459+0000) 2024-09-19T09:30:10.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:10 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:10.268+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:43.524070+0000 front 2024-09-19T09:29:43.524014+0000 (oldest deadline 2024-09-19T09:30:09.924440+0000) 2024-09-19T09:30:10.938 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:10 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:10.517+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:10.939 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:10 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:10.517+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:10.939 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:10 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:10.517+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:10.939 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:10 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:10.517+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:11.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:11 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:11.151+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:30:11.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:11 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:11.151+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:23.849908+0000 front 2024-09-19T09:29:23.849990+0000 (oldest deadline 2024-09-19T09:29:48.550062+0000) 2024-09-19T09:30:11.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:11 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:11.151+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:28.550831+0000 front 2024-09-19T09:29:28.550655+0000 (oldest deadline 2024-09-19T09:29:52.650832+0000) 2024-09-19T09:30:11.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:11 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:11.151+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:39.052473+0000 front 2024-09-19T09:29:39.052579+0000 (oldest deadline 2024-09-19T09:30:03.752464+0000) 2024-09-19T09:30:11.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:11 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:11.195+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:30:11.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:11 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:11.195+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:30:11.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:11 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:11.195+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:30:11.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:11 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:11.195+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:31.438536+0000 front 2024-09-19T09:29:31.438561+0000 (oldest deadline 2024-09-19T09:29:57.338695+0000) 2024-09-19T09:30:11.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:11 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:11.231+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:30:11.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:11 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:11.231+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:30:11.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:11 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:11.231+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:30:11.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:11 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:11.231+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:30:11.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:11 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:11.231+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:40.122552+0000 front 2024-09-19T09:29:40.122578+0000 (oldest deadline 2024-09-19T09:30:03.523459+0000) 2024-09-19T09:30:11.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:11 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:11.231+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:43.524070+0000 front 2024-09-19T09:29:43.524014+0000 (oldest deadline 2024-09-19T09:30:09.924440+0000) 2024-09-19T09:30:11.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:11 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:11.474+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:11.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:11 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:11.474+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:11.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:11 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:11.474+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:11.938 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:11 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:11.474+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:11.938 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:11 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:11.474+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:45.560108+0000 front 2024-09-19T09:29:45.560135+0000 (oldest deadline 2024-09-19T09:30:11.460243+0000) 2024-09-19T09:30:12.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:12 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:12.166+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:11.647103+0000 front 2024-09-19T09:29:11.647144+0000 (oldest deadline 2024-09-19T09:29:37.547186+0000) 2024-09-19T09:30:12.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:12 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:12.166+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:23.849908+0000 front 2024-09-19T09:29:23.849990+0000 (oldest deadline 2024-09-19T09:29:48.550062+0000) 2024-09-19T09:30:12.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:12 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:12.166+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:28.550831+0000 front 2024-09-19T09:29:28.550655+0000 (oldest deadline 2024-09-19T09:29:52.650832+0000) 2024-09-19T09:30:12.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:12 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:12.166+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:39.052473+0000 front 2024-09-19T09:29:39.052579+0000 (oldest deadline 2024-09-19T09:30:03.752464+0000) 2024-09-19T09:30:12.437 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:12 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8[53360]: 2024-09-19T09:30:12.166+0000 7fc1aeb7c640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:43.752861+0000 front 2024-09-19T09:29:43.752879+0000 (oldest deadline 2024-09-19T09:30:11.353787+0000) 2024-09-19T09:30:12.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:12 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:12.202+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:30:12.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:12 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:12.202+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:30:12.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:12 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:12.202+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:30:12.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:12 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:12.202+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:31.438536+0000 front 2024-09-19T09:29:31.438561+0000 (oldest deadline 2024-09-19T09:29:57.338695+0000) 2024-09-19T09:30:12.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:12 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:12.216+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:30:12.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:12 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:12.216+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:30:12.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:12 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:12.216+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:30:12.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:12 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:12.216+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:30:12.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:12 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:12.216+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:40.122552+0000 front 2024-09-19T09:29:40.122578+0000 (oldest deadline 2024-09-19T09:30:03.523459+0000) 2024-09-19T09:30:12.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:12 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:12.216+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:43.524070+0000 front 2024-09-19T09:29:43.524014+0000 (oldest deadline 2024-09-19T09:30:09.924440+0000) 2024-09-19T09:30:12.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:12 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:12.455+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:12.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:12 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:12.455+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:12.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:12 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:12.455+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:12.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:12 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:12.455+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:12.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:12 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:12.455+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:45.560108+0000 front 2024-09-19T09:29:45.560135+0000 (oldest deadline 2024-09-19T09:30:11.460243+0000) 2024-09-19T09:30:13.335 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:13 smithi196 podman[73025]: 2024-09-19 09:30:13.076999154 +0000 UTC m=+5.654047357 container died e640ff09be7aeb6c3613d28c27a6be362ddc2bdcad3c57d9e582d49de6feaf53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-19T09:30:13.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:13 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:13.193+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:30:13.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:13 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:13.193+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:30:13.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:13 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:13.193+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:30:13.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:13 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:13.193+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:31.438536+0000 front 2024-09-19T09:29:31.438561+0000 (oldest deadline 2024-09-19T09:29:57.338695+0000) 2024-09-19T09:30:13.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:13 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:13.193+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:46.040741+0000 front 2024-09-19T09:29:46.040833+0000 (oldest deadline 2024-09-19T09:30:13.041164+0000) 2024-09-19T09:30:13.620 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:13 smithi196 podman[73025]: 2024-09-19 09:30:13.401476474 +0000 UTC m=+5.978524723 container cleanup e640ff09be7aeb6c3613d28c27a6be362ddc2bdcad3c57d9e582d49de6feaf53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD) 2024-09-19T09:30:13.620 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:13 smithi196 bash[73025]: ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8 2024-09-19T09:30:13.621 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:13 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:13.491+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:13.621 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:13 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:13.491+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:13.621 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:13 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:13.491+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:13.621 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:13 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:13.491+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:13.621 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:13 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:13.491+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:45.560108+0000 front 2024-09-19T09:29:45.560135+0000 (oldest deadline 2024-09-19T09:30:11.460243+0000) 2024-09-19T09:30:13.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:13 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:13.228+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:30:13.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:13 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:13.228+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:30:13.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:13 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:13.228+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:30:13.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:13 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:13.228+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:30:13.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:13 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:13.228+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:40.122552+0000 front 2024-09-19T09:29:40.122578+0000 (oldest deadline 2024-09-19T09:30:03.523459+0000) 2024-09-19T09:30:13.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:13 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:13.228+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:43.524070+0000 front 2024-09-19T09:29:43.524014+0000 (oldest deadline 2024-09-19T09:30:09.924440+0000) 2024-09-19T09:30:13.937 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:13 smithi196 podman[73052]: 2024-09-19 09:30:13.619233944 +0000 UTC m=+0.536794482 container remove e640ff09be7aeb6c3613d28c27a6be362ddc2bdcad3c57d9e582d49de6feaf53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-19T09:30:14.323 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:14 smithi196 podman[73140]: 2024-09-19 09:30:13.966051172 +0000 UTC m=+0.015889717 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:30:14.324 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:14 smithi196 podman[73140]: 2024-09-19 09:30:14.069989778 +0000 UTC m=+0.119828323 container create d666d3b4082c320857e36c646830e37643c6c7f3fe1da3413b46046e603824e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-2f599a4) 2024-09-19T09:30:14.324 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:14 smithi196 podman[73140]: 2024-09-19 09:30:14.322904723 +0000 UTC m=+0.372743262 container init d666d3b4082c320857e36c646830e37643c6c7f3fe1da3413b46046e603824e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-19T09:30:14.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:14 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:14.153+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:30:14.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:14 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:14.153+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:30:14.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:14 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:14.153+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:30:14.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:14 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:14.153+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:31.438536+0000 front 2024-09-19T09:29:31.438561+0000 (oldest deadline 2024-09-19T09:29:57.338695+0000) 2024-09-19T09:30:14.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:14 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:14.153+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:46.040741+0000 front 2024-09-19T09:29:46.040833+0000 (oldest deadline 2024-09-19T09:30:13.041164+0000) 2024-09-19T09:30:14.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:14 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:14.461+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:14.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:14 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:14.461+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:14.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:14 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:14.461+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:14.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:14 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:14.461+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:14.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:14 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:14.461+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:45.560108+0000 front 2024-09-19T09:29:45.560135+0000 (oldest deadline 2024-09-19T09:30:11.460243+0000) 2024-09-19T09:30:14.688 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:14 smithi196 podman[73140]: 2024-09-19 09:30:14.326428732 +0000 UTC m=+0.376267270 container start d666d3b4082c320857e36c646830e37643c6c7f3fe1da3413b46046e603824e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8-deactivate, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-19T09:30:14.688 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:14 smithi196 podman[73140]: 2024-09-19 09:30:14.422917332 +0000 UTC m=+0.472755876 container attach d666d3b4082c320857e36c646830e37643c6c7f3fe1da3413b46046e603824e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:30:14.688 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 19 09:30:14 smithi196 podman[73140]: 2024-09-19 09:30:14.518614884 +0000 UTC m=+0.568453425 container died d666d3b4082c320857e36c646830e37643c6c7f3fe1da3413b46046e603824e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4) 2024-09-19T09:30:14.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:14 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:14.266+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:30:14.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:14 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:14.266+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:30:14.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:14 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:14.266+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:30:14.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:14 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:14.266+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:30:14.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:14 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:14.266+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:40.122552+0000 front 2024-09-19T09:29:40.122578+0000 (oldest deadline 2024-09-19T09:30:03.523459+0000) 2024-09-19T09:30:14.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:14 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:14.266+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:43.524070+0000 front 2024-09-19T09:29:43.524014+0000 (oldest deadline 2024-09-19T09:30:09.924440+0000) 2024-09-19T09:30:15.122 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.8.service' 2024-09-19T09:30:15.163 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:30:15.163 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-19T09:30:15.163 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-19T09:30:15.163 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.9 2024-09-19T09:30:15.474 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:15 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:15.135+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:30:15.475 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:15 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:15.135+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:30:15.475 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:15 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:15.135+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:30:15.475 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:15 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:15.135+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:31.438536+0000 front 2024-09-19T09:29:31.438561+0000 (oldest deadline 2024-09-19T09:29:57.338695+0000) 2024-09-19T09:30:15.475 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:15 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:15.135+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:46.040741+0000 front 2024-09-19T09:29:46.040833+0000 (oldest deadline 2024-09-19T09:30:13.041164+0000) 2024-09-19T09:30:15.475 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:15 smithi120 systemd[1]: Stopping Ceph osd.9 for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:30:15.475 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:15 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:15.473+0000 7fda93f21640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:30:15.475 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:15 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:15.473+0000 7fda93f21640 -1 osd.9 88 *** Got signal Terminated *** 2024-09-19T09:30:15.476 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:15 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:15.473+0000 7fda93f21640 -1 osd.9 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:30:15.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:15 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:15.289+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:30:15.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:15 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:15.289+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:30:15.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:15 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:15.289+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:30:15.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:15 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:15.289+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:30:15.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:15 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:15.289+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:40.122552+0000 front 2024-09-19T09:29:40.122578+0000 (oldest deadline 2024-09-19T09:30:03.523459+0000) 2024-09-19T09:30:15.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:15 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:15.289+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:43.524070+0000 front 2024-09-19T09:29:43.524014+0000 (oldest deadline 2024-09-19T09:30:09.924440+0000) 2024-09-19T09:30:15.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:15 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:15.464+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:15.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:15 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:15.464+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:15.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:15 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:15.464+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:15.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:15 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:15.464+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:15.938 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:15 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:15.464+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:45.560108+0000 front 2024-09-19T09:29:45.560135+0000 (oldest deadline 2024-09-19T09:30:11.460243+0000) 2024-09-19T09:30:16.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:16 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:16.087+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:30:16.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:16 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:16.087+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:30:16.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:16 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:16.087+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:30:16.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:16 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:16.087+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:31.438536+0000 front 2024-09-19T09:29:31.438561+0000 (oldest deadline 2024-09-19T09:29:57.338695+0000) 2024-09-19T09:30:16.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:16 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:16.087+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:46.040741+0000 front 2024-09-19T09:29:46.040833+0000 (oldest deadline 2024-09-19T09:30:13.041164+0000) 2024-09-19T09:30:16.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:16 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:16.314+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:30:16.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:16 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:16.314+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:30:16.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:16 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:16.314+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:30:16.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:16 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:16.314+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:30:16.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:16 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:16.314+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:40.122552+0000 front 2024-09-19T09:29:40.122578+0000 (oldest deadline 2024-09-19T09:30:03.523459+0000) 2024-09-19T09:30:16.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:16 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:16.314+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:43.524070+0000 front 2024-09-19T09:29:43.524014+0000 (oldest deadline 2024-09-19T09:30:09.924440+0000) 2024-09-19T09:30:16.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:16 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:16.439+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:16.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:16 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:16.439+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:16.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:16 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:16.439+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:16.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:16 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:16.439+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:16.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:16 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:16.439+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:45.560108+0000 front 2024-09-19T09:29:45.560135+0000 (oldest deadline 2024-09-19T09:30:11.460243+0000) 2024-09-19T09:30:17.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:17 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:17.064+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:30:17.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:17 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:17.064+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:30:17.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:17 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:17.064+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:30:17.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:17 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:17.064+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:31.438536+0000 front 2024-09-19T09:29:31.438561+0000 (oldest deadline 2024-09-19T09:29:57.338695+0000) 2024-09-19T09:30:17.327 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:17 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:17.064+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:46.040741+0000 front 2024-09-19T09:29:46.040833+0000 (oldest deadline 2024-09-19T09:30:13.041164+0000) 2024-09-19T09:30:17.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:17 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:17.320+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:30:17.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:17 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:17.320+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:30:17.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:17 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:17.320+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:30:17.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:17 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:17.320+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:30:17.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:17 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:17.320+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:40.122552+0000 front 2024-09-19T09:29:40.122578+0000 (oldest deadline 2024-09-19T09:30:03.523459+0000) 2024-09-19T09:30:17.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:17 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:17.320+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:43.524070+0000 front 2024-09-19T09:29:43.524014+0000 (oldest deadline 2024-09-19T09:30:09.924440+0000) 2024-09-19T09:30:17.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:17 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:17.441+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:17.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:17 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:17.441+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:17.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:17 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:17.441+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:17.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:17 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:17.441+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:17.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:17 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:17.441+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:45.560108+0000 front 2024-09-19T09:29:45.560135+0000 (oldest deadline 2024-09-19T09:30:11.460243+0000) 2024-09-19T09:30:17.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:17 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:17.441+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.761182+0000 front 2024-09-19T09:29:53.761165+0000 (oldest deadline 2024-09-19T09:30:17.261139+0000) 2024-09-19T09:30:18.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:18 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:18.020+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:30:18.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:18 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:18.020+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:30:18.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:18 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:18.020+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:30:18.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:18 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:18.020+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:31.438536+0000 front 2024-09-19T09:29:31.438561+0000 (oldest deadline 2024-09-19T09:29:57.338695+0000) 2024-09-19T09:30:18.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:18 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:18.020+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:46.040741+0000 front 2024-09-19T09:29:46.040833+0000 (oldest deadline 2024-09-19T09:30:13.041164+0000) 2024-09-19T09:30:18.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:18 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:18.020+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.041542+0000 front 2024-09-19T09:29:53.041522+0000 (oldest deadline 2024-09-19T09:30:17.141726+0000) 2024-09-19T09:30:18.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:18 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:18.346+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:30:18.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:18 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:18.346+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:30:18.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:18 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:18.346+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:30:18.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:18 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:18.346+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:30:18.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:18 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:18.346+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:40.122552+0000 front 2024-09-19T09:29:40.122578+0000 (oldest deadline 2024-09-19T09:30:03.523459+0000) 2024-09-19T09:30:18.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:18 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:18.346+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:43.524070+0000 front 2024-09-19T09:29:43.524014+0000 (oldest deadline 2024-09-19T09:30:09.924440+0000) 2024-09-19T09:30:18.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:18 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:18.472+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:18.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:18 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:18.472+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:18.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:18 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:18.472+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:18.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:18 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:18.472+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:18.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:18 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:18.472+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:45.560108+0000 front 2024-09-19T09:29:45.560135+0000 (oldest deadline 2024-09-19T09:30:11.460243+0000) 2024-09-19T09:30:18.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:18 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:18.472+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.761182+0000 front 2024-09-19T09:29:53.761165+0000 (oldest deadline 2024-09-19T09:30:17.261139+0000) 2024-09-19T09:30:19.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:19 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:19.010+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:30:19.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:19 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:19.010+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:30:19.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:19 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:19.010+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:30:19.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:19 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:19.010+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:31.438536+0000 front 2024-09-19T09:29:31.438561+0000 (oldest deadline 2024-09-19T09:29:57.338695+0000) 2024-09-19T09:30:19.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:19 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:19.010+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:46.040741+0000 front 2024-09-19T09:29:46.040833+0000 (oldest deadline 2024-09-19T09:30:13.041164+0000) 2024-09-19T09:30:19.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:19 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:19.010+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.041542+0000 front 2024-09-19T09:29:53.041522+0000 (oldest deadline 2024-09-19T09:30:17.141726+0000) 2024-09-19T09:30:19.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:19 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:19.326+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:30:19.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:19 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:19.326+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:30:19.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:19 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:19.326+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:30:19.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:19 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:19.326+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:30:19.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:19 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:19.326+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:40.122552+0000 front 2024-09-19T09:29:40.122578+0000 (oldest deadline 2024-09-19T09:30:03.523459+0000) 2024-09-19T09:30:19.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:19 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:19.326+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:43.524070+0000 front 2024-09-19T09:29:43.524014+0000 (oldest deadline 2024-09-19T09:30:09.924440+0000) 2024-09-19T09:30:19.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:19 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:19.326+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.925731+0000 front 2024-09-19T09:29:53.925717+0000 (oldest deadline 2024-09-19T09:30:18.625544+0000) 2024-09-19T09:30:19.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:19 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:19.493+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:19.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:19 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:19.493+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:19.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:19 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:19.493+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:19.938 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:19 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:19.493+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:19.938 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:19 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:19.493+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:45.560108+0000 front 2024-09-19T09:29:45.560135+0000 (oldest deadline 2024-09-19T09:30:11.460243+0000) 2024-09-19T09:30:19.938 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:19 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:19.493+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.761182+0000 front 2024-09-19T09:29:53.761165+0000 (oldest deadline 2024-09-19T09:30:17.261139+0000) 2024-09-19T09:30:20.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:19 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:19.966+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:04.232472+0000 front 2024-09-19T09:29:04.232505+0000 (oldest deadline 2024-09-19T09:29:30.132534+0000) 2024-09-19T09:30:20.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:19 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:19.966+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:14.833715+0000 front 2024-09-19T09:29:14.833801+0000 (oldest deadline 2024-09-19T09:29:36.533822+0000) 2024-09-19T09:30:20.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:19 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:19.966+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:21.636409+0000 front 2024-09-19T09:29:21.636352+0000 (oldest deadline 2024-09-19T09:29:47.437125+0000) 2024-09-19T09:30:20.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:19 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:19.966+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:31.438536+0000 front 2024-09-19T09:29:31.438561+0000 (oldest deadline 2024-09-19T09:29:57.338695+0000) 2024-09-19T09:30:20.327 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:19 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:19.966+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:46.040741+0000 front 2024-09-19T09:29:46.040833+0000 (oldest deadline 2024-09-19T09:30:13.041164+0000) 2024-09-19T09:30:20.327 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:19 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9[65539]: 2024-09-19T09:30:19.966+0000 7fda8c52f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.041542+0000 front 2024-09-19T09:29:53.041522+0000 (oldest deadline 2024-09-19T09:30:17.141726+0000) 2024-09-19T09:30:20.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:20 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:20.308+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:30:20.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:20 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:20.308+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:30:20.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:20 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:20.308+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:30:20.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:20 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:20.308+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:30:20.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:20 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:20.308+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:40.122552+0000 front 2024-09-19T09:29:40.122578+0000 (oldest deadline 2024-09-19T09:30:03.523459+0000) 2024-09-19T09:30:20.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:20 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:20.308+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:43.524070+0000 front 2024-09-19T09:29:43.524014+0000 (oldest deadline 2024-09-19T09:30:09.924440+0000) 2024-09-19T09:30:20.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:20 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:20.308+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.925731+0000 front 2024-09-19T09:29:53.925717+0000 (oldest deadline 2024-09-19T09:30:18.625544+0000) 2024-09-19T09:30:20.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:20 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:20.471+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:20.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:20 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:20.471+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:20.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:20 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:20.471+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:20.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:20 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:20.471+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:20.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:20 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:20.471+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:45.560108+0000 front 2024-09-19T09:29:45.560135+0000 (oldest deadline 2024-09-19T09:30:11.460243+0000) 2024-09-19T09:30:20.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:20 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:20.471+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.761182+0000 front 2024-09-19T09:29:53.761165+0000 (oldest deadline 2024-09-19T09:30:17.261139+0000) 2024-09-19T09:30:21.326 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:21 smithi120 podman[153104]: 2024-09-19 09:30:21.063931315 +0000 UTC m=+5.710973203 container died f875f2996a9cc45f702398de9fb10c978b6366dc85e205725e95ecc24500b600 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:30:21.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:21 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:21.425+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:21.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:21 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:21.425+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:21.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:21 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:21.425+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:21.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:21 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:21.425+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:21.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:21 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:21.425+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:45.560108+0000 front 2024-09-19T09:29:45.560135+0000 (oldest deadline 2024-09-19T09:30:11.460243+0000) 2024-09-19T09:30:21.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:21 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:21.425+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.761182+0000 front 2024-09-19T09:29:53.761165+0000 (oldest deadline 2024-09-19T09:30:17.261139+0000) 2024-09-19T09:30:21.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:21 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:21.335+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:30:21.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:21 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:21.335+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:30:21.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:21 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:21.335+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:30:21.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:21 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:21.335+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:30:21.690 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:21 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:21.335+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:40.122552+0000 front 2024-09-19T09:29:40.122578+0000 (oldest deadline 2024-09-19T09:30:03.523459+0000) 2024-09-19T09:30:21.690 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:21 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:21.335+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:43.524070+0000 front 2024-09-19T09:29:43.524014+0000 (oldest deadline 2024-09-19T09:30:09.924440+0000) 2024-09-19T09:30:21.690 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:21 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:21.335+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.925731+0000 front 2024-09-19T09:29:53.925717+0000 (oldest deadline 2024-09-19T09:30:18.625544+0000) 2024-09-19T09:30:21.717 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:21 smithi120 podman[153104]: 2024-09-19 09:30:21.414055368 +0000 UTC m=+6.061097255 container cleanup f875f2996a9cc45f702398de9fb10c978b6366dc85e205725e95ecc24500b600 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:30:21.717 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:21 smithi120 bash[153104]: ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9 2024-09-19T09:30:22.017 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:21 smithi120 podman[153120]: 2024-09-19 09:30:21.71594459 +0000 UTC m=+0.647021408 container remove f875f2996a9cc45f702398de9fb10c978b6366dc85e205725e95ecc24500b600 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=) 2024-09-19T09:30:22.312 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:22 smithi120 podman[153208]: 2024-09-19 09:30:22.016846271 +0000 UTC m=+0.108420758 container create a0ddfe28319dac722bd6b4859647e42edaffd604691faf0bde679ad313508c9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:30:22.312 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:22 smithi120 podman[153208]: 2024-09-19 09:30:21.924112865 +0000 UTC m=+0.015687321 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:30:22.312 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:22 smithi120 podman[153208]: 2024-09-19 09:30:22.235977509 +0000 UTC m=+0.327551956 container init a0ddfe28319dac722bd6b4859647e42edaffd604691faf0bde679ad313508c9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-19T09:30:22.312 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:22 smithi120 podman[153208]: 2024-09-19 09:30:22.239221556 +0000 UTC m=+0.330795996 container start a0ddfe28319dac722bd6b4859647e42edaffd604691faf0bde679ad313508c9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2) 2024-09-19T09:30:22.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:22 smithi120 podman[153208]: 2024-09-19 09:30:22.310957432 +0000 UTC m=+0.402531871 container attach a0ddfe28319dac722bd6b4859647e42edaffd604691faf0bde679ad313508c9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-19T09:30:22.576 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 19 09:30:22 smithi120 podman[153208]: 2024-09-19 09:30:22.427597307 +0000 UTC m=+0.519171750 container died a0ddfe28319dac722bd6b4859647e42edaffd604691faf0bde679ad313508c9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4) 2024-09-19T09:30:22.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:22 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:22.356+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:30:22.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:22 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:22.356+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:30:22.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:22 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:22.356+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:30:22.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:22 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:22.356+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:30:22.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:22 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:22.356+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:40.122552+0000 front 2024-09-19T09:29:40.122578+0000 (oldest deadline 2024-09-19T09:30:03.523459+0000) 2024-09-19T09:30:22.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:22 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:22.356+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:43.524070+0000 front 2024-09-19T09:29:43.524014+0000 (oldest deadline 2024-09-19T09:30:09.924440+0000) 2024-09-19T09:30:22.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:22 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:22.356+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.925731+0000 front 2024-09-19T09:29:53.925717+0000 (oldest deadline 2024-09-19T09:30:18.625544+0000) 2024-09-19T09:30:22.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:22 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:22.438+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:22.938 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:22 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:22.438+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:22.938 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:22 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:22.438+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:22.938 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:22 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:22.438+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:22.938 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:22 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:22.438+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:45.560108+0000 front 2024-09-19T09:29:45.560135+0000 (oldest deadline 2024-09-19T09:30:11.460243+0000) 2024-09-19T09:30:22.938 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:22 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:22.438+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.761182+0000 front 2024-09-19T09:29:53.761165+0000 (oldest deadline 2024-09-19T09:30:17.261139+0000) 2024-09-19T09:30:23.027 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.9.service' 2024-09-19T09:30:23.066 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:30:23.066 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-19T09:30:23.067 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-19T09:30:23.067 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.10 2024-09-19T09:30:23.370 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:23 smithi153 systemd[1]: Stopping Ceph osd.10 for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:30:23.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:23 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:23.420+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:23.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:23 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:23.420+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:23.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:23 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:23.420+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:23.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:23 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:23.420+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:23.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:23 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:23.420+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:45.560108+0000 front 2024-09-19T09:29:45.560135+0000 (oldest deadline 2024-09-19T09:30:11.460243+0000) 2024-09-19T09:30:23.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:23 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:23.420+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.761182+0000 front 2024-09-19T09:29:53.761165+0000 (oldest deadline 2024-09-19T09:30:17.261139+0000) 2024-09-19T09:30:23.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:23 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:23.369+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:30:23.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:23 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:23.369+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:30:23.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:23 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:23.369+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:30:23.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:23 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:23.369+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:30:23.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:23 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:23.369+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:40.122552+0000 front 2024-09-19T09:29:40.122578+0000 (oldest deadline 2024-09-19T09:30:03.523459+0000) 2024-09-19T09:30:23.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:23 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:23.369+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:43.524070+0000 front 2024-09-19T09:29:43.524014+0000 (oldest deadline 2024-09-19T09:30:09.924440+0000) 2024-09-19T09:30:23.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:23 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:23.369+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.925731+0000 front 2024-09-19T09:29:53.925717+0000 (oldest deadline 2024-09-19T09:30:18.625544+0000) 2024-09-19T09:30:23.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:23 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:23.371+0000 7f4be8801640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:30:23.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:23 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:23.371+0000 7f4be8801640 -1 osd.10 88 *** Got signal Terminated *** 2024-09-19T09:30:23.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:23 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:23.371+0000 7f4be8801640 -1 osd.10 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:30:24.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:24 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:24.415+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:24.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:24 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:24.415+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:24.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:24 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:24.415+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:24.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:24 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:24.415+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:24.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:24 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:24.415+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:45.560108+0000 front 2024-09-19T09:29:45.560135+0000 (oldest deadline 2024-09-19T09:30:11.460243+0000) 2024-09-19T09:30:24.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:24 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:24.415+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.761182+0000 front 2024-09-19T09:29:53.761165+0000 (oldest deadline 2024-09-19T09:30:17.261139+0000) 2024-09-19T09:30:24.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:24 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:24.361+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:30:24.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:24 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:24.361+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:30:24.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:24 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:24.361+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:30:24.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:24 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:24.361+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:30:24.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:24 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:24.361+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:40.122552+0000 front 2024-09-19T09:29:40.122578+0000 (oldest deadline 2024-09-19T09:30:03.523459+0000) 2024-09-19T09:30:24.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:24 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:24.361+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:43.524070+0000 front 2024-09-19T09:29:43.524014+0000 (oldest deadline 2024-09-19T09:30:09.924440+0000) 2024-09-19T09:30:24.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:24 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:24.361+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.925731+0000 front 2024-09-19T09:29:53.925717+0000 (oldest deadline 2024-09-19T09:30:18.625544+0000) 2024-09-19T09:30:25.437 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:25 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:25.381+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:25.437 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:25 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:25.381+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:25.437 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:25 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:25.381+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:25.437 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:25 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:25.381+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:25.437 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:25 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:25.381+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:45.560108+0000 front 2024-09-19T09:29:45.560135+0000 (oldest deadline 2024-09-19T09:30:11.460243+0000) 2024-09-19T09:30:25.437 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:25 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:25.381+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.761182+0000 front 2024-09-19T09:29:53.761165+0000 (oldest deadline 2024-09-19T09:30:17.261139+0000) 2024-09-19T09:30:25.438 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:25 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:25.381+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6820 osd.7 since back 2024-09-19T09:30:04.163317+0000 front 2024-09-19T09:30:04.163305+0000 (oldest deadline 2024-09-19T09:30:25.263168+0000) 2024-09-19T09:30:25.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:25 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:25.315+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:30:25.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:25 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:25.315+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:30:25.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:25 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:25.315+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:30:25.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:25 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:25.315+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:30:25.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:25 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:25.315+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:40.122552+0000 front 2024-09-19T09:29:40.122578+0000 (oldest deadline 2024-09-19T09:30:03.523459+0000) 2024-09-19T09:30:25.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:25 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:25.315+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:43.524070+0000 front 2024-09-19T09:29:43.524014+0000 (oldest deadline 2024-09-19T09:30:09.924440+0000) 2024-09-19T09:30:25.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:25 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:25.315+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.925731+0000 front 2024-09-19T09:29:53.925717+0000 (oldest deadline 2024-09-19T09:30:18.625544+0000) 2024-09-19T09:30:26.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:26 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:26.387+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:26.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:26 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:26.387+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:26.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:26 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:26.387+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:26.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:26 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:26.387+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:26.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:26 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:26.387+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:45.560108+0000 front 2024-09-19T09:29:45.560135+0000 (oldest deadline 2024-09-19T09:30:11.460243+0000) 2024-09-19T09:30:26.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:26 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:26.387+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.761182+0000 front 2024-09-19T09:29:53.761165+0000 (oldest deadline 2024-09-19T09:30:17.261139+0000) 2024-09-19T09:30:26.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:26 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:26.388+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6820 osd.7 since back 2024-09-19T09:30:04.163317+0000 front 2024-09-19T09:30:04.163305+0000 (oldest deadline 2024-09-19T09:30:25.263168+0000) 2024-09-19T09:30:26.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:26 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:26.346+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:30:26.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:26 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:26.346+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:30:26.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:26 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:26.346+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:30:26.688 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:26 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:26.346+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:30:26.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:26 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:26.346+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:40.122552+0000 front 2024-09-19T09:29:40.122578+0000 (oldest deadline 2024-09-19T09:30:03.523459+0000) 2024-09-19T09:30:26.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:26 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:26.346+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:43.524070+0000 front 2024-09-19T09:29:43.524014+0000 (oldest deadline 2024-09-19T09:30:09.924440+0000) 2024-09-19T09:30:26.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:26 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:26.346+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.925731+0000 front 2024-09-19T09:29:53.925717+0000 (oldest deadline 2024-09-19T09:30:18.625544+0000) 2024-09-19T09:30:27.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:27 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:27.408+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:27.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:27 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:27.408+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:27.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:27 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:27.408+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:27.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:27 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:27.408+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:27.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:27 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:27.408+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:45.560108+0000 front 2024-09-19T09:29:45.560135+0000 (oldest deadline 2024-09-19T09:30:11.460243+0000) 2024-09-19T09:30:27.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:27 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:27.408+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.761182+0000 front 2024-09-19T09:29:53.761165+0000 (oldest deadline 2024-09-19T09:30:17.261139+0000) 2024-09-19T09:30:27.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:27 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:27.408+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6820 osd.7 since back 2024-09-19T09:30:04.163317+0000 front 2024-09-19T09:30:04.163305+0000 (oldest deadline 2024-09-19T09:30:25.263168+0000) 2024-09-19T09:30:27.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:27 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:27.307+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:30:27.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:27 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:27.307+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:30:27.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:27 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:27.307+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:30:27.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:27 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:27.307+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:30:27.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:27 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:27.307+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:40.122552+0000 front 2024-09-19T09:29:40.122578+0000 (oldest deadline 2024-09-19T09:30:03.523459+0000) 2024-09-19T09:30:27.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:27 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:27.307+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:43.524070+0000 front 2024-09-19T09:29:43.524014+0000 (oldest deadline 2024-09-19T09:30:09.924440+0000) 2024-09-19T09:30:27.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:27 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:27.307+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.925731+0000 front 2024-09-19T09:29:53.925717+0000 (oldest deadline 2024-09-19T09:30:18.625544+0000) 2024-09-19T09:30:28.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:28 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:28.331+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:05.216637+0000 front 2024-09-19T09:29:05.216452+0000 (oldest deadline 2024-09-19T09:29:31.116358+0000) 2024-09-19T09:30:28.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:28 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:28.331+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:13.918006+0000 front 2024-09-19T09:29:13.918028+0000 (oldest deadline 2024-09-19T09:29:39.218133+0000) 2024-09-19T09:30:28.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:28 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:28.331+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:20.919184+0000 front 2024-09-19T09:29:20.919102+0000 (oldest deadline 2024-09-19T09:29:46.819236+0000) 2024-09-19T09:30:28.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:28 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:28.331+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-19T09:29:30.221123+0000 front 2024-09-19T09:29:30.221030+0000 (oldest deadline 2024-09-19T09:29:53.121177+0000) 2024-09-19T09:30:28.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:28 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:28.331+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:40.122552+0000 front 2024-09-19T09:29:40.122578+0000 (oldest deadline 2024-09-19T09:30:03.523459+0000) 2024-09-19T09:30:28.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:28 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:28.331+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:43.524070+0000 front 2024-09-19T09:29:43.524014+0000 (oldest deadline 2024-09-19T09:30:09.924440+0000) 2024-09-19T09:30:28.689 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:28 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10[59008]: 2024-09-19T09:30:28.331+0000 7f4be0e0f640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.925731+0000 front 2024-09-19T09:29:53.925717+0000 (oldest deadline 2024-09-19T09:30:18.625544+0000) 2024-09-19T09:30:28.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:28 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:28.440+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:28.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:28 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:28.440+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:28.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:28 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:28.440+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:28.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:28 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:28.440+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:28.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:28 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:28.440+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:45.560108+0000 front 2024-09-19T09:29:45.560135+0000 (oldest deadline 2024-09-19T09:30:11.460243+0000) 2024-09-19T09:30:28.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:28 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:28.440+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.761182+0000 front 2024-09-19T09:29:53.761165+0000 (oldest deadline 2024-09-19T09:30:17.261139+0000) 2024-09-19T09:30:28.938 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:28 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:28.440+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6820 osd.7 since back 2024-09-19T09:30:04.163317+0000 front 2024-09-19T09:30:04.163305+0000 (oldest deadline 2024-09-19T09:30:25.263168+0000) 2024-09-19T09:30:29.165 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:28 smithi153 podman[73518]: 2024-09-19 09:30:28.89095683 +0000 UTC m=+5.631980392 container died 34e1434a3c2e9f1bb12f66a253fb3c58eb6bea199de44b97aa6e326c50e0e628 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-19T09:30:29.438 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:29 smithi153 podman[73518]: 2024-09-19 09:30:29.263017791 +0000 UTC m=+6.004041345 container cleanup 34e1434a3c2e9f1bb12f66a253fb3c58eb6bea199de44b97aa6e326c50e0e628 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731) 2024-09-19T09:30:29.438 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:29 smithi153 bash[73518]: ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10 2024-09-19T09:30:29.885 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:29 smithi153 podman[73533]: 2024-09-19 09:30:29.547891247 +0000 UTC m=+0.653805400 container remove 34e1434a3c2e9f1bb12f66a253fb3c58eb6bea199de44b97aa6e326c50e0e628 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:30:29.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:29 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:29.444+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:29.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:29 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:29.444+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:29.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:29 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:29.444+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:29.938 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:29 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:29.444+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:29.938 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:29 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:29.444+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:45.560108+0000 front 2024-09-19T09:29:45.560135+0000 (oldest deadline 2024-09-19T09:30:11.460243+0000) 2024-09-19T09:30:29.938 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:29 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:29.444+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.761182+0000 front 2024-09-19T09:29:53.761165+0000 (oldest deadline 2024-09-19T09:30:17.261139+0000) 2024-09-19T09:30:29.938 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:29 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:29.444+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6820 osd.7 since back 2024-09-19T09:30:04.163317+0000 front 2024-09-19T09:30:04.163305+0000 (oldest deadline 2024-09-19T09:30:25.263168+0000) 2024-09-19T09:30:30.177 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:29 smithi153 podman[73622]: 2024-09-19 09:30:29.787581163 +0000 UTC m=+0.016025300 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:30:30.177 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:29 smithi153 podman[73622]: 2024-09-19 09:30:29.898639685 +0000 UTC m=+0.127083816 container create f8ef2b46783de46563bb27d754105fb93244080dfd81efaea6b8da2a253cec2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-19T09:30:30.177 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:30 smithi153 podman[73622]: 2024-09-19 09:30:30.092522524 +0000 UTC m=+0.320966654 container init f8ef2b46783de46563bb27d754105fb93244080dfd81efaea6b8da2a253cec2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:30:30.177 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:30 smithi153 podman[73622]: 2024-09-19 09:30:30.09575811 +0000 UTC m=+0.324202243 container start f8ef2b46783de46563bb27d754105fb93244080dfd81efaea6b8da2a253cec2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-19T09:30:30.177 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:30 smithi153 podman[73622]: 2024-09-19 09:30:30.175658448 +0000 UTC m=+0.404102584 container attach f8ef2b46783de46563bb27d754105fb93244080dfd81efaea6b8da2a253cec2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True) 2024-09-19T09:30:30.438 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 19 09:30:30 smithi153 podman[73622]: 2024-09-19 09:30:30.281073733 +0000 UTC m=+0.509517867 container died f8ef2b46783de46563bb27d754105fb93244080dfd81efaea6b8da2a253cec2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-19T09:30:30.902 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.10.service' 2024-09-19T09:30:30.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:30 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:30.465+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:30.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:30 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:30.465+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:30.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:30 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:30.465+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:30.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:30 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:30.465+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:30.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:30 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:30.465+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:45.560108+0000 front 2024-09-19T09:29:45.560135+0000 (oldest deadline 2024-09-19T09:30:11.460243+0000) 2024-09-19T09:30:30.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:30 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:30.465+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.761182+0000 front 2024-09-19T09:29:53.761165+0000 (oldest deadline 2024-09-19T09:30:17.261139+0000) 2024-09-19T09:30:30.938 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:30 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:30.465+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6820 osd.7 since back 2024-09-19T09:30:04.163317+0000 front 2024-09-19T09:30:04.163305+0000 (oldest deadline 2024-09-19T09:30:25.263168+0000) 2024-09-19T09:30:30.941 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:30:30.942 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-19T09:30:30.942 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-19T09:30:30.942 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.11 2024-09-19T09:30:31.221 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:30 smithi196 systemd[1]: Stopping Ceph osd.11 for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:30:31.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:31 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:31.220+0000 7fa96a073640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:30:31.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:31 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:31.220+0000 7fa96a073640 -1 osd.11 88 *** Got signal Terminated *** 2024-09-19T09:30:31.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:31 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:31.220+0000 7fa96a073640 -1 osd.11 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:30:31.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:31 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:31.436+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:31.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:31 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:31.436+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:31.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:31 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:31.436+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:31.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:31 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:31.436+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:31.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:31 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:31.436+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:45.560108+0000 front 2024-09-19T09:29:45.560135+0000 (oldest deadline 2024-09-19T09:30:11.460243+0000) 2024-09-19T09:30:31.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:31 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:31.436+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.761182+0000 front 2024-09-19T09:29:53.761165+0000 (oldest deadline 2024-09-19T09:30:17.261139+0000) 2024-09-19T09:30:31.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:31 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:31.436+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6820 osd.7 since back 2024-09-19T09:30:04.163317+0000 front 2024-09-19T09:30:04.163305+0000 (oldest deadline 2024-09-19T09:30:25.263168+0000) 2024-09-19T09:30:32.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:32 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:32.418+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:32.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:32 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:32.418+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:32.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:32 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:32.418+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:32.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:32 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:32.418+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:32.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:32 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:32.418+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:45.560108+0000 front 2024-09-19T09:29:45.560135+0000 (oldest deadline 2024-09-19T09:30:11.460243+0000) 2024-09-19T09:30:32.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:32 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:32.418+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.761182+0000 front 2024-09-19T09:29:53.761165+0000 (oldest deadline 2024-09-19T09:30:17.261139+0000) 2024-09-19T09:30:32.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:32 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:32.418+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6820 osd.7 since back 2024-09-19T09:30:04.163317+0000 front 2024-09-19T09:30:04.163305+0000 (oldest deadline 2024-09-19T09:30:25.263168+0000) 2024-09-19T09:30:33.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:33 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:33.390+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:33.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:33 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:33.390+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:33.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:33 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:33.390+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:33.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:33 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:33.390+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:33.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:33 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:33.390+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:45.560108+0000 front 2024-09-19T09:29:45.560135+0000 (oldest deadline 2024-09-19T09:30:11.460243+0000) 2024-09-19T09:30:33.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:33 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:33.390+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.761182+0000 front 2024-09-19T09:29:53.761165+0000 (oldest deadline 2024-09-19T09:30:17.261139+0000) 2024-09-19T09:30:33.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:33 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:33.390+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6820 osd.7 since back 2024-09-19T09:30:04.163317+0000 front 2024-09-19T09:30:04.163305+0000 (oldest deadline 2024-09-19T09:30:25.263168+0000) 2024-09-19T09:30:34.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:34 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:34.401+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:34.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:34 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:34.401+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:34.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:34 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:34.401+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:34.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:34 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:34.401+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:34.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:34 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:34.401+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:45.560108+0000 front 2024-09-19T09:29:45.560135+0000 (oldest deadline 2024-09-19T09:30:11.460243+0000) 2024-09-19T09:30:34.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:34 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:34.401+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.761182+0000 front 2024-09-19T09:29:53.761165+0000 (oldest deadline 2024-09-19T09:30:17.261139+0000) 2024-09-19T09:30:34.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:34 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:34.401+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6820 osd.7 since back 2024-09-19T09:30:04.163317+0000 front 2024-09-19T09:30:04.163305+0000 (oldest deadline 2024-09-19T09:30:25.263168+0000) 2024-09-19T09:30:34.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:34 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:34.401+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6820 osd.8 since back 2024-09-19T09:30:10.564083+0000 front 2024-09-19T09:30:10.564116+0000 (oldest deadline 2024-09-19T09:30:33.464242+0000) 2024-09-19T09:30:35.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:35 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:35.418+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-19T09:29:07.351935+0000 front 2024-09-19T09:29:07.351880+0000 (oldest deadline 2024-09-19T09:29:32.051863+0000) 2024-09-19T09:30:35.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:35 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:35.418+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-19T09:29:15.553164+0000 front 2024-09-19T09:29:15.553115+0000 (oldest deadline 2024-09-19T09:29:41.453195+0000) 2024-09-19T09:30:35.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:35 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:35.418+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-19T09:29:22.554110+0000 front 2024-09-19T09:29:22.554151+0000 (oldest deadline 2024-09-19T09:29:46.654323+0000) 2024-09-19T09:30:35.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:35 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:35.418+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-19T09:29:38.058740+0000 front 2024-09-19T09:29:38.058654+0000 (oldest deadline 2024-09-19T09:30:01.558677+0000) 2024-09-19T09:30:35.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:35 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:35.418+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-19T09:29:45.560108+0000 front 2024-09-19T09:29:45.560135+0000 (oldest deadline 2024-09-19T09:30:11.460243+0000) 2024-09-19T09:30:35.687 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:35 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:35.418+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-19T09:29:53.761182+0000 front 2024-09-19T09:29:53.761165+0000 (oldest deadline 2024-09-19T09:30:17.261139+0000) 2024-09-19T09:30:35.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:35 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:35.418+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.153:6820 osd.7 since back 2024-09-19T09:30:04.163317+0000 front 2024-09-19T09:30:04.163305+0000 (oldest deadline 2024-09-19T09:30:25.263168+0000) 2024-09-19T09:30:35.688 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:35 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11[59917]: 2024-09-19T09:30:35.418+0000 7fa966689640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.196:6820 osd.8 since back 2024-09-19T09:30:10.564083+0000 front 2024-09-19T09:30:10.564116+0000 (oldest deadline 2024-09-19T09:30:33.464242+0000) 2024-09-19T09:30:36.937 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:36 smithi196 podman[73242]: 2024-09-19 09:30:36.661828787 +0000 UTC m=+5.533069477 container died 56c6c4d0cbee217c74f8152f88ac53b46100c63bdd017687eb075574fa4facf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-19T09:30:37.437 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:36 smithi196 podman[73242]: 2024-09-19 09:30:36.977269687 +0000 UTC m=+5.848510372 container cleanup 56c6c4d0cbee217c74f8152f88ac53b46100c63bdd017687eb075574fa4facf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-2f599a4, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-19T09:30:37.438 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:36 smithi196 bash[73242]: ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11 2024-09-19T09:30:37.438 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:37 smithi196 podman[73257]: 2024-09-19 09:30:37.195198918 +0000 UTC m=+0.533044921 container remove 56c6c4d0cbee217c74f8152f88ac53b46100c63bdd017687eb075574fa4facf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:30:37.874 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:37 smithi196 podman[73345]: 2024-09-19 09:30:37.498446228 +0000 UTC m=+0.015670886 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:30:37.874 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:37 smithi196 podman[73345]: 2024-09-19 09:30:37.595959739 +0000 UTC m=+0.113184392 container create dc508646909608962c96ca476d0b08c238286a5c93eda8c13c51cdf3009e2699 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4) 2024-09-19T09:30:37.874 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:37 smithi196 podman[73345]: 2024-09-19 09:30:37.814876612 +0000 UTC m=+0.332101256 container init dc508646909608962c96ca476d0b08c238286a5c93eda8c13c51cdf3009e2699 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-19T09:30:37.874 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:37 smithi196 podman[73345]: 2024-09-19 09:30:37.818655118 +0000 UTC m=+0.335879769 container start dc508646909608962c96ca476d0b08c238286a5c93eda8c13c51cdf3009e2699 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4) 2024-09-19T09:30:37.874 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:37 smithi196 podman[73345]: 2024-09-19 09:30:37.873043219 +0000 UTC m=+0.390267879 container attach dc508646909608962c96ca476d0b08c238286a5c93eda8c13c51cdf3009e2699 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:30:38.187 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 19 09:30:38 smithi196 podman[73345]: 2024-09-19 09:30:38.007899462 +0000 UTC m=+0.525124121 container died dc508646909608962c96ca476d0b08c238286a5c93eda8c13c51cdf3009e2699 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-19T09:30:38.625 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@osd.11.service' 2024-09-19T09:30:38.665 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:30:38.665 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-19T09:30:38.665 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-19T09:30:38.666 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.a 2024-09-19T09:30:39.076 INFO:journalctl@ceph.mds.a.smithi120.stdout:Sep 19 09:30:38 smithi120 systemd[1]: Stopping Ceph mds.a for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:30:39.576 INFO:journalctl@ceph.mds.a.smithi120.stdout:Sep 19 09:30:39 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-a[72252]: 2024-09-19T09:30:39.084+0000 7f419e6f7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:30:39.576 INFO:journalctl@ceph.mds.a.smithi120.stdout:Sep 19 09:30:39 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-a[72252]: 2024-09-19T09:30:39.084+0000 7f419e6f7640 -1 mds.a *** got signal Terminated *** 2024-09-19T09:30:43.326 INFO:journalctl@ceph.mds.a.smithi120.stdout:Sep 19 09:30:43 smithi120 podman[153326]: 2024-09-19 09:30:43.070019867 +0000 UTC m=+4.209419900 container died e6ec996a145b3ec7af7dafbe64641e48bfbdd4930a4c14cb7c786da944326951 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-a, ceph=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-19T09:30:43.826 INFO:journalctl@ceph.mds.a.smithi120.stdout:Sep 19 09:30:43 smithi120 podman[153326]: 2024-09-19 09:30:43.692371787 +0000 UTC m=+4.831771820 container cleanup e6ec996a145b3ec7af7dafbe64641e48bfbdd4930a4c14cb7c786da944326951 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-a, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-19T09:30:43.826 INFO:journalctl@ceph.mds.a.smithi120.stdout:Sep 19 09:30:43 smithi120 bash[153326]: ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-a 2024-09-19T09:30:44.179 INFO:journalctl@ceph.mds.a.smithi120.stdout:Sep 19 09:30:43 smithi120 podman[153338]: 2024-09-19 09:30:43.927411039 +0000 UTC m=+0.851414433 container remove e6ec996a145b3ec7af7dafbe64641e48bfbdd4930a4c14cb7c786da944326951 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:30:44.179 INFO:journalctl@ceph.mds.a.smithi120.stdout:Sep 19 09:30:44 smithi120 systemd[1]: ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.a.service: Deactivated successfully. 2024-09-19T09:30:44.179 INFO:journalctl@ceph.mds.a.smithi120.stdout:Sep 19 09:30:44 smithi120 systemd[1]: Stopped Ceph mds.a for 507393d2-7664-11ef-bceb-c7b262605968. 2024-09-19T09:30:44.179 INFO:journalctl@ceph.mds.a.smithi120.stdout:Sep 19 09:30:44 smithi120 systemd[1]: ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.a.service: Consumed 24.765s CPU time. 2024-09-19T09:30:44.180 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.a.service' 2024-09-19T09:30:44.219 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:30:44.220 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-19T09:30:44.220 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-19T09:30:44.220 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.d 2024-09-19T09:30:44.565 INFO:journalctl@ceph.mds.d.smithi120.stdout:Sep 19 09:30:44 smithi120 systemd[1]: Stopping Ceph mds.d for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:30:44.826 INFO:journalctl@ceph.mds.d.smithi120.stdout:Sep 19 09:30:44 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-d[71073]: 2024-09-19T09:30:44.563+0000 7f486ba36640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:30:44.826 INFO:journalctl@ceph.mds.d.smithi120.stdout:Sep 19 09:30:44 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-d[71073]: 2024-09-19T09:30:44.563+0000 7f486ba36640 -1 mds.d *** got signal Terminated *** 2024-09-19T09:30:47.915 INFO:journalctl@ceph.mds.d.smithi120.stdout:Sep 19 09:30:47 smithi120 podman[153444]: 2024-09-19 09:30:47.644993079 +0000 UTC m=+3.199431366 container died c4e057cca4e0c9d81156ebf49ba3c886c674506b3efb56de11b76f5af24867d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:30:48.326 INFO:journalctl@ceph.mds.d.smithi120.stdout:Sep 19 09:30:48 smithi120 podman[153444]: 2024-09-19 09:30:48.105644341 +0000 UTC m=+3.660082638 container cleanup c4e057cca4e0c9d81156ebf49ba3c886c674506b3efb56de11b76f5af24867d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-d, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True) 2024-09-19T09:30:48.326 INFO:journalctl@ceph.mds.d.smithi120.stdout:Sep 19 09:30:48 smithi120 bash[153444]: ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-d 2024-09-19T09:30:48.544 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.d.service' 2024-09-19T09:30:48.577 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:30:48.577 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-19T09:30:48.577 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-19T09:30:48.577 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.g 2024-09-19T09:30:48.903 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 19 09:30:48 smithi120 systemd[1]: Stopping Ceph mds.g for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:30:49.326 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 19 09:30:48 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-g[71844]: 2024-09-19T09:30:48.901+0000 7f135cc67640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:30:49.326 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 19 09:30:48 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-g[71844]: 2024-09-19T09:30:48.901+0000 7f135cc67640 -1 mds.g *** got signal Terminated *** 2024-09-19T09:30:51.326 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 19 09:30:51 smithi120 podman[153565]: 2024-09-19 09:30:51.073470626 +0000 UTC m=+2.285773064 container died 19915489a4a6142cc133389ea6329cc8a10cf9055313ecb1379692a65357af5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-g, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-09-19T09:30:51.826 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 19 09:30:51 smithi120 podman[153565]: 2024-09-19 09:30:51.56013742 +0000 UTC m=+2.772439857 container cleanup 19915489a4a6142cc133389ea6329cc8a10cf9055313ecb1379692a65357af5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-g, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-09-19T09:30:51.826 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 19 09:30:51 smithi120 bash[153565]: ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-g 2024-09-19T09:30:51.988 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.g.service' 2024-09-19T09:30:52.020 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:30:52.020 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-19T09:30:52.021 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-19T09:30:52.021 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.j 2024-09-19T09:30:52.365 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 19 09:30:52 smithi120 systemd[1]: Stopping Ceph mds.j for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:30:52.826 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 19 09:30:52 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-j[70682]: 2024-09-19T09:30:52.364+0000 7fc467033640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:30:52.826 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 19 09:30:52 smithi120 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-j[70682]: 2024-09-19T09:30:52.364+0000 7fc467033640 -1 mds.j *** got signal Terminated *** 2024-09-19T09:30:56.575 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 19 09:30:56 smithi120 podman[153686]: 2024-09-19 09:30:56.305135123 +0000 UTC m=+4.066960615 container died d3d45bf31a9201ebead060b2bcc534cf252155e1e07edc020a6fe55e0a0951c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-j, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:30:56.826 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 19 09:30:56 smithi120 podman[153686]: 2024-09-19 09:30:56.647689684 +0000 UTC m=+4.409515168 container cleanup d3d45bf31a9201ebead060b2bcc534cf252155e1e07edc020a6fe55e0a0951c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-09-19T09:30:56.826 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 19 09:30:56 smithi120 bash[153686]: ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-j 2024-09-19T09:30:57.120 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.j.service' 2024-09-19T09:30:57.152 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:30:57.152 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-19T09:30:57.152 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-19T09:30:57.152 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.b 2024-09-19T09:30:57.438 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 19 09:30:57 smithi153 systemd[1]: Stopping Ceph mds.b for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:30:57.938 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 19 09:30:57 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-b[64524]: 2024-09-19T09:30:57.561+0000 7f19f74cf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:30:57.938 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 19 09:30:57 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-b[64524]: 2024-09-19T09:30:57.561+0000 7f19f74cf640 -1 mds.b *** got signal Terminated *** 2024-09-19T09:30:59.389 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 19 09:30:59 smithi153 podman[73725]: 2024-09-19 09:30:59.112702582 +0000 UTC m=+1.762682304 container died 3c564dedccd8d27d314462e23175a2e06862c4f6b8359e3b397118358dc7e15b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-19T09:30:59.688 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 19 09:30:59 smithi153 podman[73725]: 2024-09-19 09:30:59.478977333 +0000 UTC m=+2.128957068 container cleanup 3c564dedccd8d27d314462e23175a2e06862c4f6b8359e3b397118358dc7e15b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-b, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-19T09:30:59.688 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 19 09:30:59 smithi153 bash[73725]: ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-b 2024-09-19T09:30:59.908 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.b.service' 2024-09-19T09:30:59.943 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:30:59.943 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-19T09:30:59.943 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-19T09:30:59.944 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.e 2024-09-19T09:31:00.267 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:31:00 smithi153 systemd[1]: Stopping Ceph mds.e for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:31:00.688 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:31:00 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-e[63911]: 2024-09-19T09:31:00.265+0000 7fbd3e77b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:31:00.688 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:31:00 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-e[63911]: 2024-09-19T09:31:00.265+0000 7fbd3e77b640 -1 mds.e *** got signal Terminated *** 2024-09-19T09:31:03.386 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:31:03 smithi153 podman[73857]: 2024-09-19 09:31:03.113852682 +0000 UTC m=+2.952294602 container died 8b60695bc827d25ff69cf5167dff1ccec15fc4ecac3153f1a544b35e4b1a4bea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:31:03.688 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:31:03 smithi153 podman[73857]: 2024-09-19 09:31:03.490802204 +0000 UTC m=+3.329244109 container cleanup 8b60695bc827d25ff69cf5167dff1ccec15fc4ecac3153f1a544b35e4b1a4bea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-e, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:31:03.689 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 19 09:31:03 smithi153 bash[73857]: ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-e 2024-09-19T09:31:03.873 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.e.service' 2024-09-19T09:31:03.907 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:31:03.907 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-19T09:31:03.907 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-19T09:31:03.907 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.h 2024-09-19T09:31:04.438 INFO:journalctl@ceph.mds.h.smithi153.stdout:Sep 19 09:31:03 smithi153 systemd[1]: Stopping Ceph mds.h for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:31:04.438 INFO:journalctl@ceph.mds.h.smithi153.stdout:Sep 19 09:31:04 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-h[64207]: 2024-09-19T09:31:04.236+0000 7f1e0ff67640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:31:04.438 INFO:journalctl@ceph.mds.h.smithi153.stdout:Sep 19 09:31:04 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-h[64207]: 2024-09-19T09:31:04.236+0000 7f1e0ff67640 -1 mds.h *** got signal Terminated *** 2024-09-19T09:31:07.413 INFO:journalctl@ceph.mds.h.smithi153.stdout:Sep 19 09:31:07 smithi153 podman[73979]: 2024-09-19 09:31:07.113573835 +0000 UTC m=+2.987715748 container died f6118a9454abd582df00fa71db555e97f5b3f911d9406374aac4af8061e11bcd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-19T09:31:07.688 INFO:journalctl@ceph.mds.h.smithi153.stdout:Sep 19 09:31:07 smithi153 podman[73979]: 2024-09-19 09:31:07.494634378 +0000 UTC m=+3.368776290 container cleanup f6118a9454abd582df00fa71db555e97f5b3f911d9406374aac4af8061e11bcd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-19T09:31:07.689 INFO:journalctl@ceph.mds.h.smithi153.stdout:Sep 19 09:31:07 smithi153 bash[73979]: ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-h 2024-09-19T09:31:07.877 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.h.service' 2024-09-19T09:31:07.910 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:31:07.910 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-19T09:31:07.910 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-19T09:31:07.910 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.k 2024-09-19T09:31:08.316 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 19 09:31:07 smithi153 systemd[1]: Stopping Ceph mds.k for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:31:08.592 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 19 09:31:08 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-k[63597]: 2024-09-19T09:31:08.314+0000 7fa604c72640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:31:08.592 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 19 09:31:08 smithi153 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-k[63597]: 2024-09-19T09:31:08.314+0000 7fa604c72640 -1 mds.k *** got signal Terminated *** 2024-09-19T09:31:11.366 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 19 09:31:11 smithi153 podman[74103]: 2024-09-19 09:31:11.108805919 +0000 UTC m=+2.975787163 container died 5c585088ff66e9be5114b6efad06840594f5e0823c090859f836ee7dcd79675f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-k, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-19T09:31:11.666 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 19 09:31:11 smithi153 podman[74103]: 2024-09-19 09:31:11.456246991 +0000 UTC m=+3.323228224 container cleanup 5c585088ff66e9be5114b6efad06840594f5e0823c090859f836ee7dcd79675f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-k, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:31:11.666 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 19 09:31:11 smithi153 bash[74103]: ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-k 2024-09-19T09:31:11.800 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.k.service' 2024-09-19T09:31:11.832 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:31:11.832 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-19T09:31:11.832 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-19T09:31:11.832 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.c 2024-09-19T09:31:12.187 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 09:31:11 smithi196 systemd[1]: Stopping Ceph mds.c for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:31:12.687 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 09:31:12 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-c[65022]: 2024-09-19T09:31:12.367+0000 7f6a12a12640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:31:12.687 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 09:31:12 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-c[65022]: 2024-09-19T09:31:12.367+0000 7f6a12a12640 -1 mds.c *** got signal Terminated *** 2024-09-19T09:31:15.501 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 09:31:15 smithi196 podman[73460]: 2024-09-19 09:31:15.242383264 +0000 UTC m=+3.210969097 container died ae427bccb0d4288f9ed0779455d2aa9a152ceab7bb56829c0f3dd55a064ef3c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-c, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:31:15.785 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 09:31:15 smithi196 podman[73460]: 2024-09-19 09:31:15.591617995 +0000 UTC m=+3.560203825 container cleanup ae427bccb0d4288f9ed0779455d2aa9a152ceab7bb56829c0f3dd55a064ef3c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-19T09:31:15.785 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 19 09:31:15 smithi196 bash[73460]: ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-c 2024-09-19T09:31:15.996 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.c.service' 2024-09-19T09:31:16.033 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:31:16.033 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-19T09:31:16.033 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-19T09:31:16.033 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.f 2024-09-19T09:31:16.346 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 09:31:16 smithi196 systemd[1]: Stopping Ceph mds.f for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:31:16.687 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 09:31:16 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-f[64721]: 2024-09-19T09:31:16.345+0000 7fd5436e8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:31:16.687 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 09:31:16 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-f[64721]: 2024-09-19T09:31:16.345+0000 7fd5436e8640 -1 mds.f *** got signal Terminated *** 2024-09-19T09:31:19.597 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 09:31:19 smithi196 podman[73593]: 2024-09-19 09:31:19.24935478 +0000 UTC m=+3.002286784 container died 79f3de5e93f2b475954bd25128db46ee6242d9af109d736359009d90e2897c5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-19T09:31:19.856 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 09:31:19 smithi196 podman[73593]: 2024-09-19 09:31:19.679819287 +0000 UTC m=+3.432751292 container cleanup 79f3de5e93f2b475954bd25128db46ee6242d9af109d736359009d90e2897c5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-f, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-09-19T09:31:19.856 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 09:31:19 smithi196 bash[73593]: ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-f 2024-09-19T09:31:20.107 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 09:31:19 smithi196 podman[73606]: 2024-09-19 09:31:19.85486194 +0000 UTC m=+0.606273633 container remove 79f3de5e93f2b475954bd25128db46ee6242d9af109d736359009d90e2897c5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-f, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-19T09:31:20.107 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 09:31:20 smithi196 systemd[1]: ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.f.service: Deactivated successfully. 2024-09-19T09:31:20.107 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 09:31:20 smithi196 systemd[1]: Stopped Ceph mds.f for 507393d2-7664-11ef-bceb-c7b262605968. 2024-09-19T09:31:20.108 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 19 09:31:20 smithi196 systemd[1]: ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.f.service: Consumed 40.812s CPU time. 2024-09-19T09:31:20.109 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.f.service' 2024-09-19T09:31:20.142 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:31:20.142 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-19T09:31:20.142 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-19T09:31:20.142 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.i 2024-09-19T09:31:20.437 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 09:31:20 smithi196 systemd[1]: Stopping Ceph mds.i for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:31:20.937 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 09:31:20 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-i[65325]: 2024-09-19T09:31:20.453+0000 7f3fe7b8e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:31:20.937 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 09:31:20 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-i[65325]: 2024-09-19T09:31:20.453+0000 7f3fe7b8e640 -1 mds.i *** got signal Terminated *** 2024-09-19T09:31:23.600 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 09:31:23 smithi196 podman[73713]: 2024-09-19 09:31:23.241970303 +0000 UTC m=+2.887294071 container died 92cd60c0b5c868fa3fe800895ca39f073b3a1105e2f492de73487fda2bd1e569 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-i, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-2f599a4) 2024-09-19T09:31:23.884 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 09:31:23 smithi196 podman[73713]: 2024-09-19 09:31:23.690218619 +0000 UTC m=+3.335542393 container cleanup 92cd60c0b5c868fa3fe800895ca39f073b3a1105e2f492de73487fda2bd1e569 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-i, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-19T09:31:23.884 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 09:31:23 smithi196 bash[73713]: ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-i 2024-09-19T09:31:24.122 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.i.service' 2024-09-19T09:31:24.143 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 09:31:23 smithi196 podman[73727]: 2024-09-19 09:31:23.883113038 +0000 UTC m=+0.632574378 container remove 92cd60c0b5c868fa3fe800895ca39f073b3a1105e2f492de73487fda2bd1e569 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-i, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-19T09:31:24.143 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 09:31:24 smithi196 systemd[1]: ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.i.service: Deactivated successfully. 2024-09-19T09:31:24.143 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 09:31:24 smithi196 systemd[1]: Stopped Ceph mds.i for 507393d2-7664-11ef-bceb-c7b262605968. 2024-09-19T09:31:24.144 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 19 09:31:24 smithi196 systemd[1]: ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.i.service: Consumed 53.665s CPU time. 2024-09-19T09:31:24.154 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:31:24.154 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-19T09:31:24.155 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-19T09:31:24.155 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.l 2024-09-19T09:31:24.437 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 09:31:24 smithi196 systemd[1]: Stopping Ceph mds.l for 507393d2-7664-11ef-bceb-c7b262605968... 2024-09-19T09:31:24.937 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 09:31:24 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-l[64418]: 2024-09-19T09:31:24.527+0000 7fe6f7f95640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:31:24.937 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 09:31:24 smithi196 ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-l[64418]: 2024-09-19T09:31:24.527+0000 7fe6f7f95640 -1 mds.l *** got signal Terminated *** 2024-09-19T09:31:27.537 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 09:31:27 smithi196 podman[73834]: 2024-09-19 09:31:27.242147037 +0000 UTC m=+2.872776884 container died 5dd429730b8796a48b19e9923b8b568c28b95032e43640760c3d9617f8ae21d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-l, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-19T09:31:27.854 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 09:31:27 smithi196 podman[73834]: 2024-09-19 09:31:27.610707292 +0000 UTC m=+3.241337147 container cleanup 5dd429730b8796a48b19e9923b8b568c28b95032e43640760c3d9617f8ae21d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-19T09:31:27.854 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 19 09:31:27 smithi196 bash[73834]: ceph-507393d2-7664-11ef-bceb-c7b262605968-mds-l 2024-09-19T09:31:27.991 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-507393d2-7664-11ef-bceb-c7b262605968@mds.l.service' 2024-09-19T09:31:28.023 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:31:28.024 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-19T09:31:28.024 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 507393d2-7664-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T09:31:28.224 INFO:teuthology.orchestra.run.smithi120.stdout:Deleting cluster with fsid: 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T09:31:30.747 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 507393d2-7664-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T09:31:30.950 INFO:teuthology.orchestra.run.smithi153.stdout:Deleting cluster with fsid: 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T09:31:33.492 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 507393d2-7664-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T09:31:33.693 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T09:31:36.145 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T09:31:36.178 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T09:31:36.211 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T09:31:36.239 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-19T09:31:36.241 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911876/remote/smithi120/crash 2024-09-19T09:31:36.242 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/crash -- . 2024-09-19T09:31:36.279 INFO:teuthology.orchestra.run.smithi120.stderr:tar: /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T09:31:36.279 INFO:teuthology.orchestra.run.smithi120.stderr:tar: Error is not recoverable: exiting now 2024-09-19T09:31:36.281 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911876/remote/smithi153/crash 2024-09-19T09:31:36.282 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/crash -- . 2024-09-19T09:31:36.320 INFO:teuthology.orchestra.run.smithi153.stderr:tar: /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T09:31:36.320 INFO:teuthology.orchestra.run.smithi153.stderr:tar: Error is not recoverable: exiting now 2024-09-19T09:31:36.322 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911876/remote/smithi196/crash 2024-09-19T09:31:36.323 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/crash -- . 2024-09-19T09:31:36.354 INFO:teuthology.orchestra.run.smithi196.stderr:tar: /var/lib/ceph/507393d2-7664-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T09:31:36.354 INFO:teuthology.orchestra.run.smithi196.stderr:tar: Error is not recoverable: exiting now 2024-09-19T09:31:36.355 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-19T09:31:36.356 DEBUG:teuthology.orchestra.run.smithi120:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-19T09:31:36.394 INFO:teuthology.orchestra.run.smithi120.stdout:2024-09-19T09:18:20.260930+0000 mds.b (mds.0) 124 : cluster [WRN] Scrub error on inode 0x1000000b60e (/volumes/qa/sv_1/f2bee15c-e720-4829-ac9c-1a0401c9d37d/client.0/tmp/clients/client5/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-09-19T09:31:36.394 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-09-19T09:31:36.395 DEBUG:teuthology.orchestra.run.smithi120:> sudo egrep '\[SEC\]' /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-19T09:31:36.471 DEBUG:teuthology.orchestra.run.smithi120:> sudo egrep '\[ERR\]' /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-19T09:31:36.546 DEBUG:teuthology.orchestra.run.smithi120:> sudo egrep '\[WRN\]' /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-19T09:31:36.619 INFO:teuthology.orchestra.run.smithi120.stdout:2024-09-19T09:18:20.260930+0000 mds.b (mds.0) 124 : cluster [WRN] Scrub error on inode 0x1000000b60e (/volumes/qa/sv_1/f2bee15c-e720-4829-ac9c-1a0401c9d37d/client.0/tmp/clients/client5/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-09-19T09:31:36.620 INFO:tasks.cephadm:Compressing logs... 2024-09-19T09:31:36.620 DEBUG:teuthology.orchestra.run.smithi120:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T09:31:36.663 DEBUG:teuthology.orchestra.run.smithi153:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T09:31:36.666 DEBUG:teuthology.orchestra.run.smithi196:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T09:31:36.686 INFO:teuthology.orchestra.run.smithi120.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-19T09:31:36.686 INFO:teuthology.orchestra.run.smithi120.stderr:: No such file or directory 2024-09-19T09:31:36.686 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.9.log 2024-09-19T09:31:36.686 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T09:31:36.686 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T09:31:36.686 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.a.log 2024-09-19T09:31:36.686 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T09:31:36.687 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-19T09:31:36.688 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.g.log 2024-09-19T09:31:36.689 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mon.a.log: 82.9% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T09:31:36.692 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-19T09:31:36.693 INFO:teuthology.orchestra.run.smithi120.stderr: 90.4% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T09:31:36.693 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.log 2024-09-19T09:31:36.693 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.d.log 2024-09-19T09:31:36.695 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.6.log: /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-19T09:31:36.695 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.d.log: 86.7% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T09:31:36.699 INFO:teuthology.orchestra.run.smithi153.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-19T09:31:36.700 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.j.log 2024-09-19T09:31:36.701 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-19T09:31:36.702 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-19T09:31:36.702 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-19T09:31:36.702 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T09:31:36.702 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.b.log 2024-09-19T09:31:36.705 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.k.log 2024-09-19T09:31:36.705 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-19T09:31:36.705 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T09:31:36.705 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T09:31:36.705 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-19T09:31:36.705 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.log 2024-09-19T09:31:36.705 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.h.log 2024-09-19T09:31:36.705 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-19T09:31:36.705 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.e.log 2024-09-19T09:31:36.706 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T09:31:36.706 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mgr.x.log: 94.7% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.d.log.gz 2024-09-19T09:31:36.707 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-19T09:31:36.707 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.j.log: 90.9% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T09:31:36.709 INFO:teuthology.orchestra.run.smithi196.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-19T09:31:36.709 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T09:31:36.709 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T09:31:36.710 INFO:teuthology.orchestra.run.smithi120.stderr: 94.8% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.j.log.gz 2024-09-19T09:31:36.713 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.i.log 2024-09-19T09:31:36.713 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-19T09:31:36.713 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T09:31:36.714 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-19T09:31:36.714 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-19T09:31:36.714 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-19T09:31:36.714 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.log 2024-09-19T09:31:36.714 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.l.log 2024-09-19T09:31:36.714 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-19T09:31:36.714 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mgr.z.log 2024-09-19T09:31:36.714 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.c.log 2024-09-19T09:31:36.714 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.f.log 2024-09-19T09:31:36.714 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T09:31:36.715 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T09:31:36.744 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 95.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T09:31:36.789 INFO:teuthology.orchestra.run.smithi120.stderr: 94.9% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.g.log.gz 2024-09-19T09:31:36.801 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 83.8% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T09:31:36.813 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 83.9% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T09:31:36.865 INFO:teuthology.orchestra.run.smithi196.stderr: 93.6% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mgr.z.log.gz 2024-09-19T09:31:36.873 INFO:teuthology.orchestra.run.smithi153.stderr: 93.5% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-19T09:31:36.904 INFO:teuthology.orchestra.run.smithi153.stderr: 91.0% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T09:31:36.920 INFO:teuthology.orchestra.run.smithi153.stderr: 90.6% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T09:31:36.927 INFO:teuthology.orchestra.run.smithi153.stderr: 86.7% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T09:31:36.977 INFO:teuthology.orchestra.run.smithi153.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T09:31:37.021 INFO:teuthology.orchestra.run.smithi196.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T09:31:37.022 INFO:teuthology.orchestra.run.smithi120.stderr: 90.2% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-19T09:31:37.033 INFO:teuthology.orchestra.run.smithi196.stderr: 86.7% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T09:31:37.195 INFO:teuthology.orchestra.run.smithi196.stderr: 90.6% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T09:31:37.346 INFO:teuthology.orchestra.run.smithi196.stderr: 90.9% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T09:31:39.717 INFO:teuthology.orchestra.run.smithi153.stderr: 93.8% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.k.log.gz 2024-09-19T09:31:40.944 INFO:teuthology.orchestra.run.smithi196.stderr: 91.6% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-19T09:31:40.985 INFO:teuthology.orchestra.run.smithi120.stderr: 92.4% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-19T09:31:44.780 INFO:teuthology.orchestra.run.smithi196.stderr: 94.9% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.l.log.gz 2024-09-19T09:31:45.323 INFO:teuthology.orchestra.run.smithi120.stderr: 95.7% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.a.log.gz 2024-09-19T09:31:55.936 INFO:teuthology.orchestra.run.smithi196.stderr: 96.8% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.f.log.gz 2024-09-19T09:31:58.942 INFO:teuthology.orchestra.run.smithi153.stderr: 91.6% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-19T09:32:09.058 INFO:teuthology.orchestra.run.smithi120.stderr: 91.4% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-19T09:32:12.573 INFO:teuthology.orchestra.run.smithi120.stderr: 91.7% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-19T09:32:13.281 INFO:teuthology.orchestra.run.smithi120.stderr: 91.2% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-19T09:32:13.331 INFO:teuthology.orchestra.run.smithi196.stderr: 95.4% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.i.log.gz 2024-09-19T09:32:18.799 INFO:teuthology.orchestra.run.smithi120.stderr: 91.4% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-19T09:32:18.801 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-19T09:32:18.801 INFO:teuthology.orchestra.run.smithi120.stderr:real 0m42.126s 2024-09-19T09:32:18.801 INFO:teuthology.orchestra.run.smithi120.stderr:user 2m35.016s 2024-09-19T09:32:18.801 INFO:teuthology.orchestra.run.smithi120.stderr:sys 0m5.015s 2024-09-19T09:34:36.493 INFO:teuthology.orchestra.run.smithi153.stderr: 91.4% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-19T09:35:04.403 INFO:teuthology.orchestra.run.smithi196.stderr: 91.3% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-19T09:35:56.044 INFO:teuthology.orchestra.run.smithi196.stderr: 91.3% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-19T09:36:36.013 INFO:teuthology.orchestra.run.smithi196.stderr: 91.1% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-19T09:36:41.749 INFO:teuthology.orchestra.run.smithi153.stderr: 94.8% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.h.log.gz 2024-09-19T09:36:48.687 INFO:teuthology.orchestra.run.smithi196.stderr: 91.2% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-19T09:38:19.112 INFO:teuthology.orchestra.run.smithi153.stderr: 94.6% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.e.log.gz 2024-09-19T09:38:22.012 INFO:teuthology.orchestra.run.smithi196.stderr: 93.5% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.c.log.gz 2024-09-19T09:38:22.014 INFO:teuthology.orchestra.run.smithi196.stderr: 2024-09-19T09:38:22.015 INFO:teuthology.orchestra.run.smithi196.stderr:real 6m45.333s 2024-09-19T09:38:22.015 INFO:teuthology.orchestra.run.smithi196.stderr:user 4m59.257s 2024-09-19T09:38:22.015 INFO:teuthology.orchestra.run.smithi196.stderr:sys 0m11.783s 2024-09-19T09:38:41.896 INFO:teuthology.orchestra.run.smithi153.stderr: 91.7% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-19T09:41:11.008 INFO:teuthology.orchestra.run.smithi153.stderr: 91.3% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-19T09:41:31.102 INFO:teuthology.orchestra.run.smithi153.stderr: 91.2% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-19T09:50:21.969 INFO:teuthology.orchestra.run.smithi153.stderr: 93.3% -- replaced with /var/log/ceph/507393d2-7664-11ef-bceb-c7b262605968/ceph-mds.b.log.gz 2024-09-19T09:50:22.013 INFO:teuthology.orchestra.run.smithi153.stderr: 2024-09-19T09:50:22.013 INFO:teuthology.orchestra.run.smithi153.stderr:real 18m45.331s 2024-09-19T09:50:22.013 INFO:teuthology.orchestra.run.smithi153.stderr:user 11m25.180s 2024-09-19T09:50:22.013 INFO:teuthology.orchestra.run.smithi153.stderr:sys 0m29.547s 2024-09-19T09:50:22.013 INFO:tasks.cephadm:Archiving logs... 2024-09-19T09:50:22.014 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911876/remote/smithi120/log 2024-09-19T09:50:22.015 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T09:50:46.677 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911876/remote/smithi153/log 2024-09-19T09:50:46.678 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T09:52:35.066 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911876/remote/smithi196/log 2024-09-19T09:52:35.101 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T09:53:24.621 INFO:tasks.cephadm:Removing cluster... 2024-09-19T09:53:24.622 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 507393d2-7664-11ef-bceb-c7b262605968 --force 2024-09-19T09:53:24.825 INFO:teuthology.orchestra.run.smithi120.stdout:Deleting cluster with fsid: 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T09:53:25.321 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 507393d2-7664-11ef-bceb-c7b262605968 --force 2024-09-19T09:53:26.247 INFO:teuthology.orchestra.run.smithi153.stdout:Deleting cluster with fsid: 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T09:53:27.064 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 507393d2-7664-11ef-bceb-c7b262605968 --force 2024-09-19T09:53:27.624 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: 507393d2-7664-11ef-bceb-c7b262605968 2024-09-19T09:53:28.287 INFO:tasks.cephadm:Removing cephadm ... 2024-09-19T09:53:28.287 DEBUG:teuthology.orchestra.run.smithi120:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T09:53:28.306 DEBUG:teuthology.orchestra.run.smithi153:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T09:53:28.325 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T09:53:28.341 INFO:tasks.cephadm:Teardown complete 2024-09-19T09:53:28.341 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-19T09:53:28.356 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-19T09:53:28.356 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T09:53:28.359 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T09:53:28.368 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T09:53:28.671 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-19T09:53:28.681 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-19T09:53:28.682 DEBUG:teuthology.orchestra.run.smithi120:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T09:53:28.685 DEBUG:teuthology.orchestra.run.smithi153:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T09:53:28.687 DEBUG:teuthology.orchestra.run.smithi196:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T09:53:28.699 INFO:teuthology.orchestra.run.smithi120.stderr:bash: line 1: ntpq: command not found 2024-09-19T09:53:28.702 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-09-19T09:53:28.702 INFO:teuthology.orchestra.run.smithi153.stderr:bash: line 1: ntpq: command not found 2024-09-19T09:53:28.704 INFO:teuthology.orchestra.run.smithi120.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T09:53:28.704 INFO:teuthology.orchestra.run.smithi120.stdout:=============================================================================== 2024-09-19T09:53:28.704 INFO:teuthology.orchestra.run.smithi120.stdout:^* hv01.front.sepia.ceph.com 3 6 377 18 +452us[ +479us] +/- 41ms 2024-09-19T09:53:28.704 INFO:teuthology.orchestra.run.smithi120.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 19 -1854us[-1827us] +/- 75ms 2024-09-19T09:53:28.704 INFO:teuthology.orchestra.run.smithi120.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 +772us[ +798us] +/- 64ms 2024-09-19T09:53:28.704 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:53:28.740 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T09:53:28.740 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-09-19T09:53:28.740 INFO:teuthology.orchestra.run.smithi196.stdout:^* hv01.front.sepia.ceph.com 3 6 377 19 +467us[ +485us] +/- 41ms 2024-09-19T09:53:28.740 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 21 -1839us[-1821us] +/- 75ms 2024-09-19T09:53:28.740 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 21 +802us[ +820us] +/- 64ms 2024-09-19T09:53:28.740 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:53:28.774 INFO:teuthology.orchestra.run.smithi153.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T09:53:28.774 INFO:teuthology.orchestra.run.smithi153.stdout:=============================================================================== 2024-09-19T09:53:28.774 INFO:teuthology.orchestra.run.smithi153.stdout:^* hv01.front.sepia.ceph.com 3 6 377 23 +462us[ +476us] +/- 41ms 2024-09-19T09:53:28.774 INFO:teuthology.orchestra.run.smithi153.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 16 -1826us[-1826us] +/- 75ms 2024-09-19T09:53:28.774 INFO:teuthology.orchestra.run.smithi153.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 23 +804us[ +804us] +/- 64ms 2024-09-19T09:53:28.775 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:53:28.794 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-19T09:53:28.803 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-19T09:53:28.804 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-19T09:53:28.819 DEBUG:teuthology.orchestra.run.smithi120:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T09:53:28.919 DEBUG:teuthology.orchestra.run.smithi153:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T09:53:29.032 DEBUG:teuthology.orchestra.run.smithi196:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T09:53:29.138 DEBUG:teuthology.orchestra.run.smithi120:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T09:53:29.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:53:29.169 DEBUG:teuthology.orchestra.run.smithi153:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T09:53:29.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:53:29.201 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T09:53:29.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:53:29.230 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-19T09:53:29.733 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-19T09:53:29.939 INFO:teuthology.task.internal:Duration was 4094.452485 seconds 2024-09-19T09:53:29.940 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-19T09:53:29.971 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-19T09:53:29.971 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T09:53:29.974 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T09:53:29.976 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T09:53:30.017 INFO:teuthology.orchestra.run.smithi120.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T09:53:30.026 INFO:teuthology.orchestra.run.smithi153.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T09:53:30.032 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T09:53:30.546 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-19T09:53:30.546 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi120.front.sepia.ceph.com 2024-09-19T09:53:30.547 DEBUG:teuthology.orchestra.run.smithi120:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T09:53:30.571 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi153.front.sepia.ceph.com 2024-09-19T09:53:30.571 DEBUG:teuthology.orchestra.run.smithi153:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T09:53:30.601 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi196.front.sepia.ceph.com 2024-09-19T09:53:30.601 DEBUG:teuthology.orchestra.run.smithi196:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T09:53:30.633 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-19T09:53:30.633 DEBUG:teuthology.orchestra.run.smithi120:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T09:53:30.635 DEBUG:teuthology.orchestra.run.smithi153:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T09:53:30.644 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T09:53:30.701 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-19T09:53:30.702 DEBUG:teuthology.orchestra.run.smithi120:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T09:53:30.708 DEBUG:teuthology.orchestra.run.smithi153:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T09:53:30.717 DEBUG:teuthology.orchestra.run.smithi196:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T09:53:33.146 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-19T09:53:33.157 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-19T09:53:33.157 DEBUG:teuthology.orchestra.run.smithi120:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T09:53:33.182 DEBUG:teuthology.orchestra.run.smithi153:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T09:53:33.231 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T09:53:33.275 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-19T09:53:33.287 DEBUG:teuthology.orchestra.run.smithi120:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T09:53:33.289 DEBUG:teuthology.orchestra.run.smithi153:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T09:53:33.294 DEBUG:teuthology.orchestra.run.smithi196:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T09:53:33.309 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern = core 2024-09-19T09:53:33.337 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern = core 2024-09-19T09:53:33.353 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = core 2024-09-19T09:53:33.380 DEBUG:teuthology.orchestra.run.smithi120:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T09:53:33.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:53:33.394 DEBUG:teuthology.orchestra.run.smithi153:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T09:53:33.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:53:33.412 DEBUG:teuthology.orchestra.run.smithi196:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T09:53:33.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:53:33.432 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-19T09:53:33.445 INFO:teuthology.task.internal:Transferring archived files... 2024-09-19T09:53:33.446 DEBUG:teuthology.misc:Transferring archived files from smithi120:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911876/remote/smithi120 2024-09-19T09:53:33.447 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T09:53:33.536 DEBUG:teuthology.misc:Transferring archived files from smithi153:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911876/remote/smithi153 2024-09-19T09:53:33.537 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T09:53:33.578 DEBUG:teuthology.misc:Transferring archived files from smithi196:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911876/remote/smithi196 2024-09-19T09:53:33.578 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T09:53:33.617 INFO:teuthology.task.internal:Removing archive directory... 2024-09-19T09:53:33.618 DEBUG:teuthology.orchestra.run.smithi120:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T09:53:33.621 DEBUG:teuthology.orchestra.run.smithi153:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T09:53:33.623 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T09:53:33.668 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-19T09:53:33.681 INFO:teuthology.task.internal:Not uploading archives. 2024-09-19T09:53:33.681 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-19T09:53:33.695 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-19T09:53:33.695 DEBUG:teuthology.orchestra.run.smithi120:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T09:53:33.698 DEBUG:teuthology.orchestra.run.smithi153:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T09:53:33.700 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T09:53:33.710 INFO:teuthology.orchestra.run.smithi120.stdout: 266063 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 09:53 /home/ubuntu/cephtest 2024-09-19T09:53:33.715 INFO:teuthology.orchestra.run.smithi153.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 09:53 /home/ubuntu/cephtest 2024-09-19T09:53:33.724 INFO:teuthology.orchestra.run.smithi196.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 09:53 /home/ubuntu/cephtest 2024-09-19T09:53:33.726 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-19T09:53:33.736 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-19T09:53:33.769 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} duration: 4094.452484846115 failure_reason: '"2024-09-19T09:18:20.260930+0000 mds.b (mds.0) 124 : cluster [WRN] Scrub error on inode 0x1000000b60e (/volumes/qa/sv_1/f2bee15c-e720-4829-ac9c-1a0401c9d37d/client.0/tmp/clients/client5/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details" in cluster log' flavor: default owner: scheduled_teuthology@teuthology success: false 2024-09-19T09:53:33.769 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T09:53:33.862 INFO:teuthology.run:FAIL